2024-04-21T08:20:05.533 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-21T08:20:05.533 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666510 --name leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi --verbose --description fs/workload/{0-ubuntu_22.04 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} --owner scheduled_leonidus@teuthology -- /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666510/orig.config.yaml 2024-04-21T08:20:05.596 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-21T08:20:05.695 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666510 branch: wip-lusov-quiescer description: fs/workload/{0-ubuntu_22.04 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} email: null first_in_suite: false job_id: '7666510' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: a630765c1f93c6e61d9b40c4b76437611d5ef29e subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: a630765c1f93c6e61d9b40c4b76437611d5ef29e kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: 4f6bc5273643aaf5660909e85400647a8951c10b owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7599 sha1: a630765c1f93c6e61d9b40c4b76437611d5ef29e sleep_before_teardown: 0 subset: 111/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa suite_relpath: qa suite_repo: https://github.com/leonid-s-usov/ceph-ceph suite_sha1: 4f6bc5273643aaf5660909e85400647a8951c10b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi043.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHvyUXKR7gN3zHbaPnlKREYK3PV15Bktz8Y9kw+xlm9fM2/qeSOAhC4Pz/GpLf9q25pOs4Nvl2cUsxn154piXtQ= smithi112.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFNvgHzHjc0QFXV1lqtW7olbE2CT+rABIqw7RYiHnBdu+PNNmnXviUOqVef8EeyRpn36i/TS7Pa1OrnYAXRLopA= smithi175.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO/mG4hwt9g8AfooYwlYtJtUA1tsPvLLoRi+wriTHjKdOnztrgK6ljh7zcX82n49LeVgtRnAMHe3fwGZLsOIOIg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-21_08:12:55 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-21T08:20:05.695 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa; will attempt to use it 2024-04-21T08:20:05.696 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks 2024-04-21T08:20:05.696 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-21T08:20:05.698 INFO:teuthology.task.internal:Checking packages... 2024-04-21T08:20:05.721 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a630765c1f93c6e61d9b40c4b76437611d5ef29e' 2024-04-21T08:20:05.721 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-21T08:20:05.722 INFO:teuthology.packaging:ref: None 2024-04-21T08:20:05.722 INFO:teuthology.packaging:tag: None 2024-04-21T08:20:05.722 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-21T08:20:05.722 INFO:teuthology.packaging:sha1: a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:20:05.722 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-21T08:20:05.963 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3154.ga630765c 2024-04-21T08:20:05.965 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-21T08:20:05.972 INFO:teuthology.task.internal:no buildpackages task found 2024-04-21T08:20:05.972 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-21T08:20:05.981 INFO:teuthology.task.internal:Saving configuration 2024-04-21T08:20:06.004 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-21T08:20:06.010 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-21T08:20:06.036 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666510', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 08:15:11.958922', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHvyUXKR7gN3zHbaPnlKREYK3PV15Bktz8Y9kw+xlm9fM2/qeSOAhC4Pz/GpLf9q25pOs4Nvl2cUsxn154piXtQ='} 2024-04-21T08:20:06.061 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666510', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 08:15:11.960764', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFNvgHzHjc0QFXV1lqtW7olbE2CT+rABIqw7RYiHnBdu+PNNmnXviUOqVef8EeyRpn36i/TS7Pa1OrnYAXRLopA='} 2024-04-21T08:20:06.085 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666510', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 08:15:11.962373', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO/mG4hwt9g8AfooYwlYtJtUA1tsPvLLoRi+wriTHjKdOnztrgK6ljh7zcX82n49LeVgtRnAMHe3fwGZLsOIOIg='} 2024-04-21T08:20:06.085 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-21T08:20:06.092 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-21T08:20:06.092 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-21T08:20:06.092 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-21T08:20:06.092 INFO:teuthology.run_tasks:Running task console_log... 2024-04-21T08:20:06.225 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2bc8c240d0>, signals=[15]) 2024-04-21T08:20:06.226 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-21T08:20:06.232 INFO:teuthology.task.internal:Opening connections... 2024-04-21T08:20:06.232 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi043.front.sepia.ceph.com 2024-04-21T08:20:06.234 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T08:20:06.317 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi112.front.sepia.ceph.com 2024-04-21T08:20:06.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T08:20:06.395 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi175.front.sepia.ceph.com 2024-04-21T08:20:06.396 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T08:20:06.472 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-21T08:20:06.479 DEBUG:teuthology.orchestra.run.smithi043:> uname -m 2024-04-21T08:20:06.498 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2024-04-21T08:20:06.499 DEBUG:teuthology.orchestra.run.smithi043:> cat /etc/os-release 2024-04-21T08:20:06.557 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="CentOS Stream" 2024-04-21T08:20:06.557 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="9" 2024-04-21T08:20:06.557 INFO:teuthology.orchestra.run.smithi043.stdout:ID="centos" 2024-04-21T08:20:06.557 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE="rhel fedora" 2024-04-21T08:20:06.557 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="9" 2024-04-21T08:20:06.557 INFO:teuthology.orchestra.run.smithi043.stdout:PLATFORM_ID="platform:el9" 2024-04-21T08:20:06.557 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-21T08:20:06.557 INFO:teuthology.orchestra.run.smithi043.stdout:ANSI_COLOR="0;31" 2024-04-21T08:20:06.557 INFO:teuthology.orchestra.run.smithi043.stdout:LOGO="fedora-logo-icon" 2024-04-21T08:20:06.558 INFO:teuthology.orchestra.run.smithi043.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-21T08:20:06.558 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://centos.org/" 2024-04-21T08:20:06.558 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-21T08:20:06.558 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-21T08:20:06.558 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-21T08:20:06.558 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2024-04-21T08:20:06.585 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2024-04-21T08:20:06.603 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2024-04-21T08:20:06.603 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-04-21T08:20:06.660 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2024-04-21T08:20:06.660 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2024-04-21T08:20:06.660 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2024-04-21T08:20:06.661 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2024-04-21T08:20:06.661 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2024-04-21T08:20:06.661 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2024-04-21T08:20:06.661 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-21T08:20:06.661 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-04-21T08:20:06.661 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2024-04-21T08:20:06.661 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-21T08:20:06.661 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2024-04-21T08:20:06.661 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-21T08:20:06.661 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-21T08:20:06.661 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-21T08:20:06.661 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-04-21T08:20:06.685 DEBUG:teuthology.orchestra.run.smithi175:> uname -m 2024-04-21T08:20:06.703 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2024-04-21T08:20:06.703 DEBUG:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2024-04-21T08:20:06.760 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="CentOS Stream" 2024-04-21T08:20:06.760 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="9" 2024-04-21T08:20:06.760 INFO:teuthology.orchestra.run.smithi175.stdout:ID="centos" 2024-04-21T08:20:06.760 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="rhel fedora" 2024-04-21T08:20:06.760 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="9" 2024-04-21T08:20:06.760 INFO:teuthology.orchestra.run.smithi175.stdout:PLATFORM_ID="platform:el9" 2024-04-21T08:20:06.760 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-21T08:20:06.760 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2024-04-21T08:20:06.761 INFO:teuthology.orchestra.run.smithi175.stdout:LOGO="fedora-logo-icon" 2024-04-21T08:20:06.761 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-21T08:20:06.761 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://centos.org/" 2024-04-21T08:20:06.761 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-21T08:20:06.761 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-21T08:20:06.761 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-21T08:20:06.761 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2024-04-21T08:20:06.776 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-21T08:20:06.788 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-21T08:20:06.844 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-21T08:20:06.844 DEBUG:teuthology.orchestra.run.smithi043:> test '!' -e /home/ubuntu/cephtest 2024-04-21T08:20:06.847 DEBUG:teuthology.orchestra.run.smithi112:> test '!' -e /home/ubuntu/cephtest 2024-04-21T08:20:06.850 DEBUG:teuthology.orchestra.run.smithi175:> test '!' -e /home/ubuntu/cephtest 2024-04-21T08:20:06.867 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-21T08:20:06.873 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-21T08:20:06.873 DEBUG:teuthology.orchestra.run.smithi043:> test -z $(ls -A /var/lib/ceph) 2024-04-21T08:20:06.905 DEBUG:teuthology.orchestra.run.smithi112:> test -z $(ls -A /var/lib/ceph) 2024-04-21T08:20:06.908 DEBUG:teuthology.orchestra.run.smithi175:> test -z $(ls -A /var/lib/ceph) 2024-04-21T08:20:07.015 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-21T08:20:07.093 INFO:teuthology.run_tasks:Running task kernel... 2024-04-21T08:20:07.107 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-21T08:20:07.107 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-21T08:20:07.107 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-21T08:20:07.108 DEBUG:teuthology.orchestra.run.smithi043:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-21T08:20:07.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:20:07.125 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-04-21T08:20:07.184 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-437.el9.x86_64 2024-04-21T08:20:07.184 INFO:teuthology.task.kernel:Running kernel on smithi043: 5.14.0-437.el9.x86_64 2024-04-21T08:20:07.184 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum install -y kernel 2024-04-21T08:20:09.395 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-21T08:20:11.188 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-04-21T08:20:15.605 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 8.6 MB/s | 6.2 MB 00:00 2024-04-21T08:20:17.056 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 16 kB 00:00 2024-04-21T08:20:18.260 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-04-21T08:20:23.374 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-21T08:20:25.322 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-21T08:20:25.322 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-21T08:20:25.420 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-04-21T08:20:25.421 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-04-21T08:20:25.421 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-04-21T08:20:25.533 DEBUG:teuthology.orchestra.run.smithi043:> echo no | sudo yum reinstall kernel || true 2024-04-21T08:20:26.054 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:03 ago on Sun 21 Apr 2024 08:20:23 AM UTC. 2024-04-21T08:20:26.178 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-04-21T08:20:26.195 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-04-21T08:20:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-04-21T08:20:26.195 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-04-21T08:20:26.195 INFO:teuthology.orchestra.run.smithi043.stdout:Reinstalling: 2024-04-21T08:20:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-21T08:20:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:20:26.195 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-04-21T08:20:26.195 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-04-21T08:20:26.196 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:20:26.196 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 255 k 2024-04-21T08:20:26.196 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 0 2024-04-21T08:20:26.196 INFO:teuthology.orchestra.run.smithi043.stderr:Operation aborted. 2024-04-21T08:20:26.251 INFO:teuthology.orchestra.run.smithi043.stdout:Is this ok [y/N]: 2024-04-21T08:20:26.251 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum reinstall -y kernel || true 2024-04-21T08:20:26.766 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:03 ago on Sun 21 Apr 2024 08:20:23 AM UTC. 2024-04-21T08:20:26.887 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-04-21T08:20:26.888 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-04-21T08:20:26.888 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-04-21T08:20:26.888 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-04-21T08:20:26.888 INFO:teuthology.orchestra.run.smithi043.stdout:Reinstalling: 2024-04-21T08:20:26.888 INFO:teuthology.orchestra.run.smithi043.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-21T08:20:26.888 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:20:26.888 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-04-21T08:20:26.888 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-04-21T08:20:26.888 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:20:26.889 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 255 k 2024-04-21T08:20:26.889 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 0 2024-04-21T08:20:26.889 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-04-21T08:20:27.034 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-5.14.0-437.el9.x86_64.rpm 3.1 MB/s | 255 kB 00:00 2024-04-21T08:20:27.035 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-04-21T08:20:27.035 INFO:teuthology.orchestra.run.smithi043.stdout:Total 1.7 MB/s | 255 kB 00:00 2024-04-21T08:20:27.053 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-04-21T08:20:27.243 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-04-21T08:20:27.244 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-04-21T08:20:27.249 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-04-21T08:20:27.249 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-04-21T08:20:27.363 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-04-21T08:20:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-21T08:20:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-21T08:20:28.092 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-21T08:20:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-21T08:20:28.760 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-21T08:20:28.760 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:20:28.760 INFO:teuthology.orchestra.run.smithi043.stdout:Reinstalled: 2024-04-21T08:20:28.760 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-21T08:20:28.760 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:20:28.761 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-04-21T08:20:28.956 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q kernel | sort -rV | head -n 1 2024-04-21T08:20:29.004 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-21T08:20:29.004 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-21T08:20:29.004 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-21T08:20:29.005 DEBUG:teuthology.parallel:result is None 2024-04-21T08:20:29.005 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-21T08:20:29.019 INFO:teuthology.task.internal:Creating test directory... 2024-04-21T08:20:29.020 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-21T08:20:29.022 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-21T08:20:29.025 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-21T08:20:29.042 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-21T08:20:29.054 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-21T08:20:29.069 INFO:teuthology.task.internal:Creating archive directory... 2024-04-21T08:20:29.070 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-21T08:20:29.077 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-21T08:20:29.086 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-21T08:20:29.200 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-21T08:20:29.212 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-21T08:20:29.212 DEBUG:teuthology.orchestra.run.smithi112:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-21T08:20:29.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:20:29.230 DEBUG:teuthology.orchestra.run.smithi175:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-21T08:20:29.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:20:29.248 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-21T08:20:29.251 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-21T08:20:29.273 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-21T08:20:29.279 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T08:20:29.296 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T08:20:29.321 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T08:20:29.346 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T08:20:29.349 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T08:20:29.364 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T08:20:29.366 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-21T08:20:29.377 INFO:teuthology.task.internal:Configuring sudo... 2024-04-21T08:20:29.377 DEBUG:teuthology.orchestra.run.smithi043:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-21T08:20:29.380 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-21T08:20:29.382 DEBUG:teuthology.orchestra.run.smithi175:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-21T08:20:29.408 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-21T08:20:29.421 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-21T08:20:29.421 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-21T08:20:29.445 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-21T08:20:29.449 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-21T08:20:29.465 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T08:20:29.542 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T08:20:29.590 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T08:20:29.672 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T08:20:29.713 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-21T08:20:29.714 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-21T08:20:29.779 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T08:20:29.818 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T08:20:29.875 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T08:20:29.956 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T08:20:30.002 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-21T08:20:30.002 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-21T08:20:30.067 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T08:20:30.112 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T08:20:30.157 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T08:20:30.242 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T08:20:30.280 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-21T08:20:30.280 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-21T08:20:30.345 DEBUG:teuthology.orchestra.run.smithi043:> sudo service rsyslog restart 2024-04-21T08:20:30.348 DEBUG:teuthology.orchestra.run.smithi112:> sudo service rsyslog restart 2024-04-21T08:20:30.353 DEBUG:teuthology.orchestra.run.smithi175:> sudo service rsyslog restart 2024-04-21T08:20:30.390 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T08:20:30.398 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T08:20:30.419 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T08:20:30.848 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-21T08:20:30.857 INFO:teuthology.task.internal:Starting timer... 2024-04-21T08:20:30.857 INFO:teuthology.run_tasks:Running task pcp... 2024-04-21T08:20:30.916 INFO:teuthology.run_tasks:Running task selinux... 2024-04-21T08:20:30.926 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-21T08:20:30.927 DEBUG:teuthology.orchestra.run.smithi043:> sudo service auditd rotate 2024-04-21T08:20:31.008 INFO:teuthology.orchestra.run.smithi043.stdout:Rotating logs: 2024-04-21T08:20:31.010 DEBUG:teuthology.orchestra.run.smithi112:> sudo service auditd rotate 2024-04-21T08:20:31.064 INFO:teuthology.orchestra.run.smithi112.stdout:Rotating logs: 2024-04-21T08:20:31.066 DEBUG:teuthology.orchestra.run.smithi175:> sudo service auditd rotate 2024-04-21T08:20:31.121 INFO:teuthology.orchestra.run.smithi175.stdout:Rotating logs: 2024-04-21T08:20:31.123 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-21T08:20:31.123 DEBUG:teuthology.orchestra.run.smithi043:> /usr/sbin/getenforce 2024-04-21T08:20:31.150 INFO:teuthology.orchestra.run.smithi043.stdout:Permissive 2024-04-21T08:20:31.150 DEBUG:teuthology.orchestra.run.smithi112:> /usr/sbin/getenforce 2024-04-21T08:20:31.174 INFO:teuthology.orchestra.run.smithi112.stdout:Permissive 2024-04-21T08:20:31.175 DEBUG:teuthology.orchestra.run.smithi175:> /usr/sbin/getenforce 2024-04-21T08:20:31.194 INFO:teuthology.orchestra.run.smithi175.stdout:Permissive 2024-04-21T08:20:31.194 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi043.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi112.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi175.front.sepia.ceph.com': 'permissive'} 2024-04-21T08:20:31.195 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T08:20:31.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:20:31.222 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T08:20:31.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:20:31.250 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T08:20:31.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:20:31.277 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-21T08:20:31.277 DEBUG:teuthology.orchestra.run.smithi043:> sudo /usr/sbin/setenforce permissive 2024-04-21T08:20:31.329 DEBUG:teuthology.orchestra.run.smithi112:> sudo /usr/sbin/setenforce permissive 2024-04-21T08:20:31.356 DEBUG:teuthology.orchestra.run.smithi175:> sudo /usr/sbin/setenforce permissive 2024-04-21T08:20:31.383 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-21T08:20:31.392 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-21T08:20:31.409 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-21T08:20:31.411 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi043.front.sepia.ceph.com,smithi112.front.sepia.ceph.com,smithi175.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-21T08:24:35.326 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi043.front.sepia.ceph.com'), Remote(name='ubuntu@smithi112.front.sepia.ceph.com'), Remote(name='ubuntu@smithi175.front.sepia.ceph.com')] 2024-04-21T08:24:35.327 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-04-21T08:24:35.328 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T08:24:35.411 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-04-21T08:24:35.495 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-04-21T08:24:35.495 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-21T08:24:35.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T08:24:35.575 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-04-21T08:24:35.661 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-21T08:24:35.662 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-04-21T08:24:35.662 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T08:24:35.777 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-04-21T08:24:35.813 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-04-21T08:24:35.813 INFO:teuthology.run_tasks:Running task clock... 2024-04-21T08:24:35.826 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-21T08:24:35.826 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-21T08:24:35.827 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T08:24:35.829 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-21T08:24:35.829 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T08:24:35.831 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-21T08:24:35.831 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T08:24:35.857 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-21T08:24:35.860 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-21T08:24:35.870 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-21T08:24:35.876 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-21T08:24:35.881 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-21T08:24:35.893 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: ntpd: command not found 2024-04-21T08:24:35.894 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-21T08:24:35.902 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: ntpd: command not found 2024-04-21T08:24:35.903 INFO:teuthology.orchestra.run.smithi112.stdout:506 Cannot talk to daemon 2024-04-21T08:24:35.913 INFO:teuthology.orchestra.run.smithi043.stdout:506 Cannot talk to daemon 2024-04-21T08:24:35.916 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-21T08:24:35.917 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: ntpd: command not found 2024-04-21T08:24:35.927 INFO:teuthology.orchestra.run.smithi175.stdout:506 Cannot talk to daemon 2024-04-21T08:24:35.928 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-21T08:24:35.929 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-21T08:24:35.940 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-21T08:24:35.943 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-21T08:24:35.953 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-21T08:24:35.988 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-04-21T08:24:35.992 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T08:24:35.992 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-04-21T08:24:35.992 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:24:35.992 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:24:35.992 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:24:35.992 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:24:36.005 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-04-21T08:24:36.008 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T08:24:36.008 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-04-21T08:24:36.008 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:24:36.008 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:24:36.008 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:24:36.008 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:24:36.009 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-04-21T08:24:36.012 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T08:24:36.012 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-04-21T08:24:36.012 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:24:36.012 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:24:36.012 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:24:36.012 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:24:36.013 INFO:teuthology.run_tasks:Running task install... 2024-04-21T08:24:36.022 DEBUG:teuthology.task.install:project ceph 2024-04-21T08:24:36.022 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e'}} 2024-04-21T08:24:36.022 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e'} 2024-04-21T08:24:36.022 INFO:teuthology.task.install:Using flavor: default 2024-04-21T08:24:36.028 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-21T08:24:36.028 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-21T08:24:36.029 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'tag': None, 'wait_for_package': False} 2024-04-21T08:24:36.029 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:24:36.031 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'tag': None, 'wait_for_package': False} 2024-04-21T08:24:36.032 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:24:36.032 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'tag': None, 'wait_for_package': False} 2024-04-21T08:24:36.033 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:24:36.188 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/ 2024-04-21T08:24:36.188 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3154.ga630765c 2024-04-21T08:24:36.257 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/ 2024-04-21T08:24:36.258 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3154.ga630765c 2024-04-21T08:24:36.269 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/ 2024-04-21T08:24:36.269 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3154.ga630765c 2024-04-21T08:24:36.305 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-21T08:24:36.305 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-21T08:24:36.305 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-21T08:24:36.330 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-21T08:24:36.331 DEBUG:teuthology.orchestra.run.smithi112:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a630765c1f93c6e61d9b40c4b76437611d5ef29e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-21T08:24:36.372 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-21T08:24:36.372 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-21T08:24:36.372 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-21T08:24:36.389 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-21T08:24:36.389 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-21T08:24:36.389 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-21T08:24:36.396 DEBUG:teuthology.orchestra.run.smithi112:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-21T08:24:36.399 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-21T08:24:36.399 DEBUG:teuthology.orchestra.run.smithi043:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a630765c1f93c6e61d9b40c4b76437611d5ef29e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-21T08:24:36.414 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-21T08:24:36.415 DEBUG:teuthology.orchestra.run.smithi175:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a630765c1f93c6e61d9b40c4b76437611d5ef29e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-21T08:24:36.466 DEBUG:teuthology.orchestra.run.smithi043:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-21T08:24:36.468 DEBUG:teuthology.orchestra.run.smithi112:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-21T08:24:36.480 DEBUG:teuthology.orchestra.run.smithi175:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-21T08:24:36.531 INFO:teuthology.orchestra.run.smithi112.stdout:check_obsoletes = 1 2024-04-21T08:24:36.533 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum clean all 2024-04-21T08:24:36.543 DEBUG:teuthology.orchestra.run.smithi043:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-21T08:24:36.553 DEBUG:teuthology.orchestra.run.smithi175:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-21T08:24:36.607 INFO:teuthology.orchestra.run.smithi043.stdout:check_obsoletes = 1 2024-04-21T08:24:36.609 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum clean all 2024-04-21T08:24:36.616 INFO:teuthology.orchestra.run.smithi175.stdout:check_obsoletes = 1 2024-04-21T08:24:36.618 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum clean all 2024-04-21T08:24:36.756 INFO:teuthology.orchestra.run.smithi112.stdout:48 files removed 2024-04-21T08:24:36.783 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-21T08:24:36.843 INFO:teuthology.orchestra.run.smithi043.stdout:48 files removed 2024-04-21T08:24:36.844 INFO:teuthology.orchestra.run.smithi175.stdout:48 files removed 2024-04-21T08:24:36.870 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-21T08:24:36.872 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-21T08:24:37.397 INFO:teuthology.orchestra.run.smithi112.stdout:ceph packages for x86_64 235 kB/s | 83 kB 00:00 2024-04-21T08:24:37.482 INFO:teuthology.orchestra.run.smithi175.stdout:ceph packages for x86_64 239 kB/s | 83 kB 00:00 2024-04-21T08:24:37.541 INFO:teuthology.orchestra.run.smithi043.stdout:ceph packages for x86_64 210 kB/s | 83 kB 00:00 2024-04-21T08:24:37.833 INFO:teuthology.orchestra.run.smithi175.stdout:ceph noarch packages 38 kB/s | 12 kB 00:00 2024-04-21T08:24:37.865 INFO:teuthology.orchestra.run.smithi112.stdout:ceph noarch packages 27 kB/s | 12 kB 00:00 2024-04-21T08:24:37.975 INFO:teuthology.orchestra.run.smithi043.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-04-21T08:24:38.258 INFO:teuthology.orchestra.run.smithi175.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-21T08:24:38.290 INFO:teuthology.orchestra.run.smithi112.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-21T08:24:38.351 INFO:teuthology.orchestra.run.smithi043.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-04-21T08:24:38.909 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-21T08:24:39.009 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-21T08:24:42.168 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 8.7 MB/s | 19 MB 00:02 2024-04-21T08:24:44.107 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 4.7 MB/s | 19 MB 00:04 2024-04-21T08:24:46.303 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-04-21T08:24:47.432 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 901 kB/s | 8.0 MB 00:09 2024-04-21T08:24:48.120 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 16 kB 00:00 2024-04-21T08:24:48.417 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 8.2 MB/s | 6.2 MB 00:00 2024-04-21T08:24:49.166 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-04-21T08:24:49.805 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-21T08:24:50.225 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 16 kB 00:00 2024-04-21T08:24:52.902 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 8.2 MB/s | 21 MB 00:02 2024-04-21T08:24:53.183 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-21T08:24:54.600 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 16 kB 00:00 2024-04-21T08:24:55.088 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-21T08:24:55.452 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 28 MB/s | 21 MB 00:00 2024-04-21T08:24:57.332 INFO:teuthology.orchestra.run.smithi043.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T08:24:57.332 INFO:teuthology.orchestra.run.smithi043.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T08:24:57.344 INFO:teuthology.orchestra.run.smithi043.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-21T08:24:57.344 INFO:teuthology.orchestra.run.smithi043.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-21T08:24:57.345 INFO:teuthology.orchestra.run.smithi043.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-21T08:24:57.346 INFO:teuthology.orchestra.run.smithi043.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-21T08:24:57.348 INFO:teuthology.orchestra.run.smithi043.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-21T08:24:57.350 INFO:teuthology.orchestra.run.smithi043.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-21T08:24:57.351 INFO:teuthology.orchestra.run.smithi043.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-21T08:24:57.414 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-04-21T08:24:57.423 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================= 2024-04-21T08:24:57.423 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-04-21T08:24:57.424 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================= 2024-04-21T08:24:57.424 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-04-21T08:24:57.424 INFO:teuthology.orchestra.run.smithi043.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-21T08:24:57.424 INFO:teuthology.orchestra.run.smithi043.stdout: ceph x86_64 2:19.0.0-3154.ga630765c.el9 ceph 6.6 k 2024-04-21T08:24:57.424 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T08:24:57.424 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 890 k 2024-04-21T08:24:57.424 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3154.ga630765c.el9 ceph 145 k 2024-04-21T08:24:57.424 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr x86_64 2:19.0.0-3154.ga630765c.el9 ceph 1.5 M 2024-04-21T08:24:57.424 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 142 k 2024-04-21T08:24:57.424 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 3.5 M 2024-04-21T08:24:57.424 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 7.4 M 2024-04-21T08:24:57.424 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 49 k 2024-04-21T08:24:57.424 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 11 M 2024-04-21T08:24:57.425 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test x86_64 2:19.0.0-3154.ga630765c.el9 ceph 49 M 2024-04-21T08:24:57.425 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 273 k 2024-04-21T08:24:57.425 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 762 k 2024-04-21T08:24:57.425 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 225 k 2024-04-21T08:24:57.425 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-shell x86_64 2:19.0.0-3154.ga630765c.el9 ceph 27 k 2024-04-21T08:24:57.425 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-top noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 18 k 2024-04-21T08:24:57.425 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-21T08:24:57.425 INFO:teuthology.orchestra.run.smithi043.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-21T08:24:57.425 INFO:teuthology.orchestra.run.smithi043.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-21T08:24:57.425 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 32 k 2024-04-21T08:24:57.425 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 746 k 2024-04-21T08:24:57.425 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 127 k 2024-04-21T08:24:57.425 INFO:teuthology.orchestra.run.smithi043.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-21T08:24:57.425 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-21T08:24:57.426 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-21T08:24:57.426 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-21T08:24:57.426 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-21T08:24:57.426 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-21T08:24:57.426 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs x86_64 2:19.0.0-3154.ga630765c.el9 ceph 162 k 2024-04-21T08:24:57.426 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados x86_64 2:19.0.0-3154.ga630765c.el9 ceph 321 k 2024-04-21T08:24:57.426 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 298 k 2024-04-21T08:24:57.426 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 100 k 2024-04-21T08:24:57.426 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 85 k 2024-04-21T08:24:57.426 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T08:24:57.426 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 171 k 2024-04-21T08:24:57.426 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-21T08:24:57.426 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrading: 2024-04-21T08:24:57.427 INFO:teuthology.orchestra.run.smithi043.stdout: librados2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.4 M 2024-04-21T08:24:57.427 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T08:24:57.427 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-04-21T08:24:57.428 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-21T08:24:57.428 INFO:teuthology.orchestra.run.smithi043.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-21T08:24:57.428 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-21T08:24:57.428 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 21 M 2024-04-21T08:24:57.428 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 34 k 2024-04-21T08:24:57.428 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds x86_64 2:19.0.0-3154.ga630765c.el9 ceph 2.4 M 2024-04-21T08:24:57.428 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 249 k 2024-04-21T08:24:57.428 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon x86_64 2:19.0.0-3154.ga630765c.el9 ceph 4.7 M 2024-04-21T08:24:57.428 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 17 M 2024-04-21T08:24:57.428 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 16 k 2024-04-21T08:24:57.429 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux x86_64 2:19.0.0-3154.ga630765c.el9 ceph 25 k 2024-04-21T08:24:57.429 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-21T08:24:57.429 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-21T08:24:57.429 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-21T08:24:57.429 INFO:teuthology.orchestra.run.smithi043.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-21T08:24:57.429 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-21T08:24:57.429 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-21T08:24:57.429 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-21T08:24:57.429 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-21T08:24:57.429 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite x86_64 2:19.0.0-3154.ga630765c.el9 ceph 163 k 2024-04-21T08:24:57.429 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-21T08:24:57.429 INFO:teuthology.orchestra.run.smithi043.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-21T08:24:57.429 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-21T08:24:57.430 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-21T08:24:57.430 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-21T08:24:57.430 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 501 k 2024-04-21T08:24:57.430 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-21T08:24:57.430 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T08:24:57.430 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-21T08:24:57.430 INFO:teuthology.orchestra.run.smithi043.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-21T08:24:57.430 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-21T08:24:57.430 INFO:teuthology.orchestra.run.smithi043.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-21T08:24:57.430 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-21T08:24:57.430 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-21T08:24:57.430 INFO:teuthology.orchestra.run.smithi043.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-21T08:24:57.430 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-21T08:24:57.430 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-21T08:24:57.431 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-21T08:24:57.431 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-21T08:24:57.431 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-21T08:24:57.431 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-21T08:24:57.431 INFO:teuthology.orchestra.run.smithi043.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-21T08:24:57.431 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-21T08:24:57.431 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-21T08:24:57.431 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-21T08:24:57.431 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 45 k 2024-04-21T08:24:57.431 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 135 k 2024-04-21T08:24:57.431 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-21T08:24:57.431 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-21T08:24:57.432 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-21T08:24:57.432 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-21T08:24:57.432 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-21T08:24:57.432 INFO:teuthology.orchestra.run.smithi043.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-21T08:24:57.432 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-21T08:24:57.432 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-21T08:24:57.432 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-21T08:24:57.432 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-21T08:24:57.432 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-21T08:24:57.432 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-21T08:24:57.432 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-21T08:24:57.432 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-21T08:24:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-21T08:24:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-21T08:24:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-21T08:24:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-21T08:24:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-21T08:24:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-21T08:24:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-21T08:24:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-21T08:24:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-21T08:24:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-21T08:24:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-21T08:24:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-21T08:24:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-21T08:24:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-21T08:24:57.434 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-21T08:24:57.434 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-21T08:24:57.434 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-21T08:24:57.434 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-21T08:24:57.434 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-21T08:24:57.434 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-21T08:24:57.434 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-21T08:24:57.434 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-21T08:24:57.434 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-21T08:24:57.434 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-21T08:24:57.434 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-21T08:24:57.434 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-21T08:24:57.434 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-21T08:24:57.434 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-21T08:24:57.435 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-21T08:24:57.435 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-21T08:24:57.435 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-21T08:24:57.435 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-21T08:24:57.435 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-21T08:24:57.435 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-21T08:24:57.435 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-21T08:24:57.435 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-21T08:24:57.435 INFO:teuthology.orchestra.run.smithi043.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-21T08:24:57.435 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-21T08:24:57.435 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-21T08:24:57.435 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-21T08:24:57.435 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-21T08:24:57.436 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-21T08:24:57.436 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-21T08:24:57.436 INFO:teuthology.orchestra.run.smithi043.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-21T08:24:57.436 INFO:teuthology.orchestra.run.smithi043.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-21T08:24:57.436 INFO:teuthology.orchestra.run.smithi043.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-21T08:24:57.436 INFO:teuthology.orchestra.run.smithi043.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-21T08:24:57.436 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-21T08:24:57.436 INFO:teuthology.orchestra.run.smithi043.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-21T08:24:57.436 INFO:teuthology.orchestra.run.smithi043.stdout:Installing weak dependencies: 2024-04-21T08:24:57.436 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-21T08:24:57.436 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:24:57.436 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-04-21T08:24:57.436 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================= 2024-04-21T08:24:57.436 INFO:teuthology.orchestra.run.smithi043.stdout:Install 148 Packages 2024-04-21T08:24:57.436 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrade 2 Packages 2024-04-21T08:24:57.437 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:24:57.437 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 218 M 2024-04-21T08:24:57.437 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-04-21T08:24:58.052 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-21T08:24:58.156 INFO:teuthology.orchestra.run.smithi043.stdout:(1/150): ceph-19.0.0-3154.ga630765c.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-04-21T08:24:58.481 INFO:teuthology.orchestra.run.smithi043.stdout:(2/150): ceph-base-19.0.0-3154.ga630765c.el9.x8 9.0 MB/s | 5.4 MB 00:00 2024-04-21T08:24:58.557 INFO:teuthology.orchestra.run.smithi043.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 1.9 MB/s | 145 kB 00:00 2024-04-21T08:24:58.707 INFO:teuthology.orchestra.run.smithi043.stdout:(4/150): ceph-fuse-19.0.0-3154.ga630765c.el9.x8 1.6 MB/s | 890 kB 00:00 2024-04-21T08:24:58.840 INFO:teuthology.orchestra.run.smithi043.stdout:(5/150): ceph-mds-19.0.0-3154.ga630765c.el9.x86 8.3 MB/s | 2.4 MB 00:00 2024-04-21T08:24:59.182 INFO:teuthology.orchestra.run.smithi043.stdout:(6/150): ceph-mon-19.0.0-3154.ga630765c.el9.x86 14 MB/s | 4.7 MB 00:00 2024-04-21T08:24:59.524 INFO:teuthology.orchestra.run.smithi043.stdout:(7/150): ceph-mgr-19.0.0-3154.ga630765c.el9.x86 1.8 MB/s | 1.5 MB 00:00 2024-04-21T08:25:00.187 INFO:teuthology.orchestra.run.smithi112.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T08:25:00.187 INFO:teuthology.orchestra.run.smithi112.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T08:25:00.198 INFO:teuthology.orchestra.run.smithi112.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-21T08:25:00.199 INFO:teuthology.orchestra.run.smithi112.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-21T08:25:00.200 INFO:teuthology.orchestra.run.smithi112.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-21T08:25:00.201 INFO:teuthology.orchestra.run.smithi112.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-21T08:25:00.202 INFO:teuthology.orchestra.run.smithi112.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-21T08:25:00.204 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-21T08:25:00.205 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-21T08:25:00.266 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-04-21T08:25:00.282 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-04-21T08:25:00.283 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-04-21T08:25:00.283 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-04-21T08:25:00.283 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-04-21T08:25:00.283 INFO:teuthology.orchestra.run.smithi112.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-21T08:25:00.283 INFO:teuthology.orchestra.run.smithi112.stdout: ceph x86_64 2:19.0.0-3154.ga630765c.el9 ceph 6.6 k 2024-04-21T08:25:00.283 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T08:25:00.283 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 890 k 2024-04-21T08:25:00.283 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3154.ga630765c.el9 ceph 145 k 2024-04-21T08:25:00.283 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr x86_64 2:19.0.0-3154.ga630765c.el9 ceph 1.5 M 2024-04-21T08:25:00.283 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 142 k 2024-04-21T08:25:00.284 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 3.5 M 2024-04-21T08:25:00.284 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 7.4 M 2024-04-21T08:25:00.284 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 49 k 2024-04-21T08:25:00.284 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 11 M 2024-04-21T08:25:00.284 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test x86_64 2:19.0.0-3154.ga630765c.el9 ceph 49 M 2024-04-21T08:25:00.284 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 273 k 2024-04-21T08:25:00.284 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 762 k 2024-04-21T08:25:00.284 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 225 k 2024-04-21T08:25:00.284 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell x86_64 2:19.0.0-3154.ga630765c.el9 ceph 27 k 2024-04-21T08:25:00.284 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 18 k 2024-04-21T08:25:00.284 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-21T08:25:00.284 INFO:teuthology.orchestra.run.smithi112.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-21T08:25:00.284 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-21T08:25:00.285 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 32 k 2024-04-21T08:25:00.285 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 746 k 2024-04-21T08:25:00.285 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 127 k 2024-04-21T08:25:00.285 INFO:teuthology.orchestra.run.smithi112.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-21T08:25:00.285 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-21T08:25:00.285 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-21T08:25:00.285 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-21T08:25:00.285 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-21T08:25:00.285 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-21T08:25:00.285 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs x86_64 2:19.0.0-3154.ga630765c.el9 ceph 162 k 2024-04-21T08:25:00.285 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados x86_64 2:19.0.0-3154.ga630765c.el9 ceph 321 k 2024-04-21T08:25:00.285 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 298 k 2024-04-21T08:25:00.285 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 100 k 2024-04-21T08:25:00.286 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 85 k 2024-04-21T08:25:00.286 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T08:25:00.286 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 171 k 2024-04-21T08:25:00.286 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-21T08:25:00.286 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrading: 2024-04-21T08:25:00.286 INFO:teuthology.orchestra.run.smithi112.stdout: librados2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.4 M 2024-04-21T08:25:00.286 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T08:25:00.286 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-04-21T08:25:00.286 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-21T08:25:00.286 INFO:teuthology.orchestra.run.smithi112.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-21T08:25:00.286 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-21T08:25:00.286 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 21 M 2024-04-21T08:25:00.286 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 34 k 2024-04-21T08:25:00.287 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds x86_64 2:19.0.0-3154.ga630765c.el9 ceph 2.4 M 2024-04-21T08:25:00.287 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 249 k 2024-04-21T08:25:00.287 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon x86_64 2:19.0.0-3154.ga630765c.el9 ceph 4.7 M 2024-04-21T08:25:00.287 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 17 M 2024-04-21T08:25:00.287 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 16 k 2024-04-21T08:25:00.287 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux x86_64 2:19.0.0-3154.ga630765c.el9 ceph 25 k 2024-04-21T08:25:00.287 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-21T08:25:00.287 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-21T08:25:00.287 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-21T08:25:00.287 INFO:teuthology.orchestra.run.smithi112.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-21T08:25:00.287 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-21T08:25:00.287 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-21T08:25:00.287 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-21T08:25:00.287 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-21T08:25:00.287 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite x86_64 2:19.0.0-3154.ga630765c.el9 ceph 163 k 2024-04-21T08:25:00.288 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-21T08:25:00.288 INFO:teuthology.orchestra.run.smithi112.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-21T08:25:00.288 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-21T08:25:00.288 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-21T08:25:00.288 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-21T08:25:00.288 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 501 k 2024-04-21T08:25:00.288 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-21T08:25:00.288 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T08:25:00.288 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-21T08:25:00.288 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-21T08:25:00.288 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-21T08:25:00.288 INFO:teuthology.orchestra.run.smithi112.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-21T08:25:00.289 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-21T08:25:00.289 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-21T08:25:00.289 INFO:teuthology.orchestra.run.smithi112.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-21T08:25:00.289 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-21T08:25:00.289 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-21T08:25:00.289 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-21T08:25:00.289 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-21T08:25:00.289 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-21T08:25:00.289 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-21T08:25:00.289 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-21T08:25:00.289 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-21T08:25:00.289 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-21T08:25:00.289 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-21T08:25:00.290 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 45 k 2024-04-21T08:25:00.290 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 135 k 2024-04-21T08:25:00.290 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-21T08:25:00.290 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-21T08:25:00.290 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-21T08:25:00.290 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-21T08:25:00.290 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-21T08:25:00.290 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-21T08:25:00.290 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-21T08:25:00.290 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-21T08:25:00.290 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-21T08:25:00.290 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-21T08:25:00.291 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-21T08:25:00.291 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-21T08:25:00.291 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-21T08:25:00.291 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-21T08:25:00.291 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-21T08:25:00.291 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-21T08:25:00.291 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-21T08:25:00.291 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-21T08:25:00.291 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-21T08:25:00.291 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-21T08:25:00.291 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-21T08:25:00.291 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-21T08:25:00.292 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-21T08:25:00.292 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-21T08:25:00.292 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-21T08:25:00.292 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-21T08:25:00.292 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-21T08:25:00.292 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-21T08:25:00.292 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-21T08:25:00.292 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-21T08:25:00.292 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-21T08:25:00.292 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-21T08:25:00.292 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-21T08:25:00.293 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-21T08:25:00.293 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-21T08:25:00.293 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-21T08:25:00.293 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-21T08:25:00.293 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-21T08:25:00.293 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-21T08:25:00.293 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-21T08:25:00.293 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-21T08:25:00.293 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-21T08:25:00.293 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-21T08:25:00.293 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-21T08:25:00.293 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-21T08:25:00.293 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-21T08:25:00.294 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-21T08:25:00.294 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-21T08:25:00.294 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-21T08:25:00.294 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-21T08:25:00.294 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-21T08:25:00.294 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-21T08:25:00.294 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-21T08:25:00.294 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-21T08:25:00.294 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-21T08:25:00.294 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-21T08:25:00.294 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-21T08:25:00.294 INFO:teuthology.orchestra.run.smithi112.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-21T08:25:00.294 INFO:teuthology.orchestra.run.smithi112.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-21T08:25:00.295 INFO:teuthology.orchestra.run.smithi112.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-21T08:25:00.295 INFO:teuthology.orchestra.run.smithi112.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-21T08:25:00.295 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-21T08:25:00.295 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-21T08:25:00.295 INFO:teuthology.orchestra.run.smithi112.stdout:Installing weak dependencies: 2024-04-21T08:25:00.295 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-21T08:25:00.295 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-21T08:25:00.295 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-04-21T08:25:00.295 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-04-21T08:25:00.295 INFO:teuthology.orchestra.run.smithi112.stdout:Install 148 Packages 2024-04-21T08:25:00.295 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrade 2 Packages 2024-04-21T08:25:00.295 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-21T08:25:00.296 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 218 M 2024-04-21T08:25:00.296 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-04-21T08:25:00.427 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-21T08:25:00.616 INFO:teuthology.orchestra.run.smithi043.stdout:(8/150): ceph-osd-19.0.0-3154.ga630765c.el9.x86 12 MB/s | 17 MB 00:01 2024-04-21T08:25:00.724 INFO:teuthology.orchestra.run.smithi043.stdout:(9/150): ceph-selinux-19.0.0-3154.ga630765c.el9 233 kB/s | 25 kB 00:00 2024-04-21T08:25:00.958 INFO:teuthology.orchestra.run.smithi043.stdout:(10/150): ceph-common-19.0.0-3154.ga630765c.el9 6.9 MB/s | 21 MB 00:03 2024-04-21T08:25:01.108 INFO:teuthology.orchestra.run.smithi043.stdout:(11/150): cephfs-mirror-19.0.0-3154.ga630765c.e 1.5 MB/s | 225 kB 00:00 2024-04-21T08:25:01.187 INFO:teuthology.orchestra.run.smithi112.stdout:(1/150): ceph-19.0.0-3154.ga630765c.el9.x86_64. 29 kB/s | 6.6 kB 00:00 2024-04-21T08:25:01.192 INFO:teuthology.orchestra.run.smithi043.stdout:(12/150): cephfs-shell-19.0.0-3154.ga630765c.el 326 kB/s | 27 kB 00:00 2024-04-21T08:25:01.276 INFO:teuthology.orchestra.run.smithi043.stdout:(13/150): libcephfs-devel-19.0.0-3154.ga630765c 380 kB/s | 32 kB 00:00 2024-04-21T08:25:01.468 INFO:teuthology.orchestra.run.smithi043.stdout:(14/150): libcephfs2-19.0.0-3154.ga630765c.el9. 3.8 MB/s | 746 kB 00:00 2024-04-21T08:25:01.569 INFO:teuthology.orchestra.run.smithi043.stdout:(15/150): libcephsqlite-19.0.0-3154.ga630765c.e 1.6 MB/s | 163 kB 00:00 2024-04-21T08:25:01.669 INFO:teuthology.orchestra.run.smithi043.stdout:(16/150): librados-devel-19.0.0-3154.ga630765c. 1.2 MB/s | 127 kB 00:00 2024-04-21T08:25:01.828 INFO:teuthology.orchestra.run.smithi043.stdout:(17/150): libradosstriper1-19.0.0-3154.ga630765 3.1 MB/s | 501 kB 00:00 2024-04-21T08:25:01.837 INFO:teuthology.orchestra.run.smithi112.stdout:(2/150): ceph-fuse-19.0.0-3154.ga630765c.el9.x8 1.3 MB/s | 890 kB 00:00 2024-04-21T08:25:01.904 INFO:teuthology.orchestra.run.smithi112.stdout:(3/150): ceph-base-19.0.0-3154.ga630765c.el9.x8 5.7 MB/s | 5.4 MB 00:00 2024-04-21T08:25:01.988 INFO:teuthology.orchestra.run.smithi112.stdout:(4/150): ceph-immutable-object-cache-19.0.0-315 965 kB/s | 145 kB 00:00 2024-04-21T08:25:02.196 INFO:teuthology.orchestra.run.smithi112.stdout:(5/150): ceph-mds-19.0.0-3154.ga630765c.el9.x86 8.1 MB/s | 2.4 MB 00:00 2024-04-21T08:25:02.211 INFO:teuthology.orchestra.run.smithi043.stdout:(18/150): ceph-radosgw-19.0.0-3154.ga630765c.el 3.9 MB/s | 11 MB 00:02 2024-04-21T08:25:02.336 INFO:teuthology.orchestra.run.smithi043.stdout:(19/150): python3-ceph-argparse-19.0.0-3154.ga6 362 kB/s | 45 kB 00:00 2024-04-21T08:25:02.595 INFO:teuthology.orchestra.run.smithi175.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T08:25:02.595 INFO:teuthology.orchestra.run.smithi175.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T08:25:02.606 INFO:teuthology.orchestra.run.smithi175.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-21T08:25:02.607 INFO:teuthology.orchestra.run.smithi175.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-21T08:25:02.608 INFO:teuthology.orchestra.run.smithi175.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-21T08:25:02.609 INFO:teuthology.orchestra.run.smithi175.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-21T08:25:02.610 INFO:teuthology.orchestra.run.smithi175.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-21T08:25:02.612 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-21T08:25:02.613 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-21T08:25:02.646 INFO:teuthology.orchestra.run.smithi112.stdout:(6/150): ceph-mon-19.0.0-3154.ga630765c.el9.x86 10 MB/s | 4.7 MB 00:00 2024-04-21T08:25:02.673 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-04-21T08:25:02.680 INFO:teuthology.orchestra.run.smithi112.stdout:(7/150): ceph-mgr-19.0.0-3154.ga630765c.el9.x86 2.2 MB/s | 1.5 MB 00:00 2024-04-21T08:25:02.692 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-04-21T08:25:02.693 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-04-21T08:25:02.693 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-04-21T08:25:02.693 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-04-21T08:25:02.693 INFO:teuthology.orchestra.run.smithi175.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-21T08:25:02.693 INFO:teuthology.orchestra.run.smithi175.stdout: ceph x86_64 2:19.0.0-3154.ga630765c.el9 ceph 6.6 k 2024-04-21T08:25:02.693 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T08:25:02.693 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 890 k 2024-04-21T08:25:02.693 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3154.ga630765c.el9 ceph 145 k 2024-04-21T08:25:02.693 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr x86_64 2:19.0.0-3154.ga630765c.el9 ceph 1.5 M 2024-04-21T08:25:02.693 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 142 k 2024-04-21T08:25:02.693 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 3.5 M 2024-04-21T08:25:02.693 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 7.4 M 2024-04-21T08:25:02.694 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 49 k 2024-04-21T08:25:02.694 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 11 M 2024-04-21T08:25:02.694 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test x86_64 2:19.0.0-3154.ga630765c.el9 ceph 49 M 2024-04-21T08:25:02.694 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 273 k 2024-04-21T08:25:02.694 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 762 k 2024-04-21T08:25:02.694 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 225 k 2024-04-21T08:25:02.694 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-shell x86_64 2:19.0.0-3154.ga630765c.el9 ceph 27 k 2024-04-21T08:25:02.694 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 18 k 2024-04-21T08:25:02.694 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-21T08:25:02.694 INFO:teuthology.orchestra.run.smithi175.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-21T08:25:02.694 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-21T08:25:02.694 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 32 k 2024-04-21T08:25:02.694 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 746 k 2024-04-21T08:25:02.695 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 127 k 2024-04-21T08:25:02.695 INFO:teuthology.orchestra.run.smithi175.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-21T08:25:02.695 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-21T08:25:02.695 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-21T08:25:02.695 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-21T08:25:02.696 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-21T08:25:02.696 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-21T08:25:02.696 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs x86_64 2:19.0.0-3154.ga630765c.el9 ceph 162 k 2024-04-21T08:25:02.696 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados x86_64 2:19.0.0-3154.ga630765c.el9 ceph 321 k 2024-04-21T08:25:02.696 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 298 k 2024-04-21T08:25:02.696 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 100 k 2024-04-21T08:25:02.696 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 85 k 2024-04-21T08:25:02.696 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T08:25:02.696 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 171 k 2024-04-21T08:25:02.697 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-21T08:25:02.697 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrading: 2024-04-21T08:25:02.697 INFO:teuthology.orchestra.run.smithi175.stdout: librados2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.4 M 2024-04-21T08:25:02.697 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T08:25:02.697 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-04-21T08:25:02.697 INFO:teuthology.orchestra.run.smithi175.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-21T08:25:02.697 INFO:teuthology.orchestra.run.smithi175.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-21T08:25:02.697 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-21T08:25:02.697 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 21 M 2024-04-21T08:25:02.697 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 34 k 2024-04-21T08:25:02.697 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds x86_64 2:19.0.0-3154.ga630765c.el9 ceph 2.4 M 2024-04-21T08:25:02.697 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 249 k 2024-04-21T08:25:02.698 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon x86_64 2:19.0.0-3154.ga630765c.el9 ceph 4.7 M 2024-04-21T08:25:02.698 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 17 M 2024-04-21T08:25:02.698 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 16 k 2024-04-21T08:25:02.698 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux x86_64 2:19.0.0-3154.ga630765c.el9 ceph 25 k 2024-04-21T08:25:02.698 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-21T08:25:02.698 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-21T08:25:02.698 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-21T08:25:02.698 INFO:teuthology.orchestra.run.smithi175.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-21T08:25:02.699 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-21T08:25:02.699 INFO:teuthology.orchestra.run.smithi175.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-21T08:25:02.699 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-21T08:25:02.699 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-21T08:25:02.699 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite x86_64 2:19.0.0-3154.ga630765c.el9 ceph 163 k 2024-04-21T08:25:02.699 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-21T08:25:02.699 INFO:teuthology.orchestra.run.smithi175.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-21T08:25:02.699 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-21T08:25:02.699 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-21T08:25:02.699 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-21T08:25:02.699 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 501 k 2024-04-21T08:25:02.699 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-21T08:25:02.700 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T08:25:02.700 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-21T08:25:02.700 INFO:teuthology.orchestra.run.smithi175.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-21T08:25:02.700 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-21T08:25:02.700 INFO:teuthology.orchestra.run.smithi175.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-21T08:25:02.700 INFO:teuthology.orchestra.run.smithi175.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-21T08:25:02.700 INFO:teuthology.orchestra.run.smithi175.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-21T08:25:02.700 INFO:teuthology.orchestra.run.smithi175.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-21T08:25:02.700 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-21T08:25:02.700 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-21T08:25:02.700 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-21T08:25:02.700 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-21T08:25:02.700 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-21T08:25:02.700 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-21T08:25:02.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-21T08:25:02.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-21T08:25:02.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-21T08:25:02.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-21T08:25:02.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 45 k 2024-04-21T08:25:02.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 135 k 2024-04-21T08:25:02.701 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-21T08:25:02.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-21T08:25:02.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-21T08:25:02.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-21T08:25:02.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-21T08:25:02.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-21T08:25:02.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-21T08:25:02.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-21T08:25:02.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-21T08:25:02.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-21T08:25:02.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-21T08:25:02.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-21T08:25:02.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-21T08:25:02.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-21T08:25:02.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-21T08:25:02.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-21T08:25:02.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-21T08:25:02.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-21T08:25:02.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-21T08:25:02.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-21T08:25:02.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-21T08:25:02.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-21T08:25:02.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-21T08:25:02.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-21T08:25:02.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-21T08:25:02.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-21T08:25:02.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-21T08:25:02.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-21T08:25:02.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-21T08:25:02.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-21T08:25:02.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-21T08:25:02.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-21T08:25:02.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-21T08:25:02.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-21T08:25:02.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-21T08:25:02.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-21T08:25:02.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-21T08:25:02.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-21T08:25:02.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-21T08:25:02.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-21T08:25:02.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-21T08:25:02.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-21T08:25:02.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-21T08:25:02.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-21T08:25:02.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-21T08:25:02.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-21T08:25:02.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-21T08:25:02.706 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-21T08:25:02.706 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-21T08:25:02.706 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-21T08:25:02.706 INFO:teuthology.orchestra.run.smithi175.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-21T08:25:02.706 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-21T08:25:02.706 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-21T08:25:02.706 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-21T08:25:02.706 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-21T08:25:02.706 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-21T08:25:02.706 INFO:teuthology.orchestra.run.smithi175.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-21T08:25:02.706 INFO:teuthology.orchestra.run.smithi175.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-21T08:25:02.707 INFO:teuthology.orchestra.run.smithi175.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-21T08:25:02.707 INFO:teuthology.orchestra.run.smithi175.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-21T08:25:02.707 INFO:teuthology.orchestra.run.smithi175.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-21T08:25:02.707 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-21T08:25:02.707 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-21T08:25:02.707 INFO:teuthology.orchestra.run.smithi175.stdout:Installing weak dependencies: 2024-04-21T08:25:02.707 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-21T08:25:02.707 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-21T08:25:02.707 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-04-21T08:25:02.707 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-04-21T08:25:02.707 INFO:teuthology.orchestra.run.smithi175.stdout:Install 148 Packages 2024-04-21T08:25:02.707 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrade 2 Packages 2024-04-21T08:25:02.707 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-21T08:25:02.708 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 218 M 2024-04-21T08:25:02.708 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-04-21T08:25:02.745 INFO:teuthology.orchestra.run.smithi043.stdout:(20/150): librgw2-19.0.0-3154.ga630765c.el9.x86 5.9 MB/s | 5.4 MB 00:00 2024-04-21T08:25:02.795 INFO:teuthology.orchestra.run.smithi043.stdout:(21/150): python3-ceph-common-19.0.0-3154.ga630 294 kB/s | 135 kB 00:00 2024-04-21T08:25:02.863 INFO:teuthology.orchestra.run.smithi043.stdout:(22/150): python3-cephfs-19.0.0-3154.ga630765c. 1.3 MB/s | 162 kB 00:00 2024-04-21T08:25:02.946 INFO:teuthology.orchestra.run.smithi043.stdout:(23/150): python3-rados-19.0.0-3154.ga630765c.e 2.1 MB/s | 321 kB 00:00 2024-04-21T08:25:03.021 INFO:teuthology.orchestra.run.smithi043.stdout:(24/150): python3-rbd-19.0.0-3154.ga630765c.el9 1.8 MB/s | 298 kB 00:00 2024-04-21T08:25:03.080 INFO:teuthology.orchestra.run.smithi043.stdout:(25/150): python3-rgw-19.0.0-3154.ga630765c.el9 745 kB/s | 100 kB 00:00 2024-04-21T08:25:03.147 INFO:teuthology.orchestra.run.smithi043.stdout:(26/150): rbd-fuse-19.0.0-3154.ga630765c.el9.x8 682 kB/s | 85 kB 00:00 2024-04-21T08:25:03.628 INFO:teuthology.orchestra.run.smithi175.stdout:(1/150): ceph-19.0.0-3154.ga630765c.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-04-21T08:25:03.814 INFO:teuthology.orchestra.run.smithi043.stdout:(27/150): rbd-mirror-19.0.0-3154.ga630765c.el9. 4.2 MB/s | 3.1 MB 00:00 2024-04-21T08:25:03.872 INFO:teuthology.orchestra.run.smithi112.stdout:(8/150): ceph-osd-19.0.0-3154.ga630765c.el9.x86 14 MB/s | 17 MB 00:01 2024-04-21T08:25:03.894 INFO:teuthology.orchestra.run.smithi175.stdout:(2/150): ceph-fuse-19.0.0-3154.ga630765c.el9.x8 3.3 MB/s | 890 kB 00:00 2024-04-21T08:25:03.914 INFO:teuthology.orchestra.run.smithi043.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3154.g 343 kB/s | 34 kB 00:00 2024-04-21T08:25:03.958 INFO:teuthology.orchestra.run.smithi112.stdout:(9/150): ceph-selinux-19.0.0-3154.ga630765c.el9 302 kB/s | 25 kB 00:00 2024-04-21T08:25:03.970 INFO:teuthology.orchestra.run.smithi175.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 1.9 MB/s | 145 kB 00:00 2024-04-21T08:25:04.031 INFO:teuthology.orchestra.run.smithi043.stdout:(29/150): rbd-nbd-19.0.0-3154.ga630765c.el9.x86 194 kB/s | 171 kB 00:00 2024-04-21T08:25:04.056 INFO:teuthology.orchestra.run.smithi043.stdout:(30/150): ceph-mgr-cephadm-19.0.0-3154.ga630765 1.0 MB/s | 142 kB 00:00 2024-04-21T08:25:05.095 INFO:teuthology.orchestra.run.smithi175.stdout:(4/150): ceph-mds-19.0.0-3154.ga630765c.el9.x86 2.1 MB/s | 2.4 MB 00:01 2024-04-21T08:25:05.487 INFO:teuthology.orchestra.run.smithi175.stdout:(5/150): ceph-mgr-19.0.0-3154.ga630765c.el9.x86 3.8 MB/s | 1.5 MB 00:00 2024-04-21T08:25:05.840 INFO:teuthology.orchestra.run.smithi043.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3154.ga6307 2.0 MB/s | 3.5 MB 00:01 2024-04-21T08:25:06.048 INFO:teuthology.orchestra.run.smithi043.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3154.ga6 1.2 MB/s | 249 kB 00:00 2024-04-21T08:25:06.132 INFO:teuthology.orchestra.run.smithi043.stdout:(33/150): ceph-mgr-diskprediction-local-19.0.0- 3.6 MB/s | 7.4 MB 00:02 2024-04-21T08:25:06.157 INFO:teuthology.orchestra.run.smithi043.stdout:(34/150): ceph-mgr-rook-19.0.0-3154.ga630765c.e 454 kB/s | 49 kB 00:00 2024-04-21T08:25:06.241 INFO:teuthology.orchestra.run.smithi043.stdout:(35/150): ceph-prometheus-alerts-19.0.0-3154.ga 152 kB/s | 16 kB 00:00 2024-04-21T08:25:06.332 INFO:teuthology.orchestra.run.smithi043.stdout:(36/150): ceph-volume-19.0.0-3154.ga630765c.el9 1.5 MB/s | 273 kB 00:00 2024-04-21T08:25:06.499 INFO:teuthology.orchestra.run.smithi043.stdout:(37/150): cephadm-19.0.0-3154.ga630765c.el9.noa 2.9 MB/s | 762 kB 00:00 2024-04-21T08:25:06.558 INFO:teuthology.orchestra.run.smithi043.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 80 kB 00:00 2024-04-21T08:25:06.579 INFO:teuthology.orchestra.run.smithi175.stdout:(6/150): ceph-mon-19.0.0-3154.ga630765c.el9.x86 4.3 MB/s | 4.7 MB 00:01 2024-04-21T08:25:06.583 INFO:teuthology.orchestra.run.smithi043.stdout:(39/150): cephfs-top-19.0.0-3154.ga630765c.el9. 73 kB/s | 18 kB 00:00 2024-04-21T08:25:06.661 INFO:teuthology.orchestra.run.smithi043.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2024-04-21T08:25:06.694 INFO:teuthology.orchestra.run.smithi043.stdout:(41/150): libgfortran-11.4.1-3.el9.x86_64.rpm 5.8 MB/s | 803 kB 00:00 2024-04-21T08:25:06.761 INFO:teuthology.orchestra.run.smithi043.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-04-21T08:25:06.762 INFO:teuthology.orchestra.run.smithi175.stdout:(7/150): ceph-base-19.0.0-3154.ga630765c.el9.x8 1.5 MB/s | 5.4 MB 00:03 2024-04-21T08:25:06.828 INFO:teuthology.orchestra.run.smithi043.stdout:(43/150): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-04-21T08:25:06.853 INFO:teuthology.orchestra.run.smithi043.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-04-21T08:25:06.912 INFO:teuthology.orchestra.run.smithi043.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-21T08:25:06.946 INFO:teuthology.orchestra.run.smithi043.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-04-21T08:25:06.962 INFO:teuthology.orchestra.run.smithi175.stdout:(8/150): ceph-common-19.0.0-3154.ga630765c.el9. 5.8 MB/s | 21 MB 00:03 2024-04-21T08:25:07.013 INFO:teuthology.orchestra.run.smithi043.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-04-21T08:25:07.021 INFO:teuthology.orchestra.run.smithi175.stdout:(9/150): ceph-selinux-19.0.0-3154.ga630765c.el9 433 kB/s | 25 kB 00:00 2024-04-21T08:25:07.238 INFO:teuthology.orchestra.run.smithi043.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-04-21T08:25:07.297 INFO:teuthology.orchestra.run.smithi043.stdout:(49/150): bison-3.7.4-5.el9.x86_64.rpm 3.6 MB/s | 943 kB 00:00 2024-04-21T08:25:07.322 INFO:teuthology.orchestra.run.smithi043.stdout:(50/150): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 107 kB 00:00 2024-04-21T08:25:07.364 INFO:teuthology.orchestra.run.smithi043.stdout:(51/150): elfutils-libelf-devel-0.190-2.el9.x86 321 kB/s | 21 kB 00:00 2024-04-21T08:25:07.389 INFO:teuthology.orchestra.run.smithi043.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-04-21T08:25:07.431 INFO:teuthology.orchestra.run.smithi043.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 490 kB/s | 33 kB 00:00 2024-04-21T08:25:07.506 INFO:teuthology.orchestra.run.smithi043.stdout:(54/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-04-21T08:25:07.523 INFO:teuthology.orchestra.run.smithi043.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 198 kB/s | 18 kB 00:00 2024-04-21T08:25:07.557 INFO:teuthology.orchestra.run.smithi043.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-04-21T08:25:07.607 INFO:teuthology.orchestra.run.smithi043.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-04-21T08:25:07.624 INFO:teuthology.orchestra.run.smithi043.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 675 kB/s | 45 kB 00:00 2024-04-21T08:25:07.741 INFO:teuthology.orchestra.run.smithi043.stdout:(59/150): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-04-21T08:25:07.741 INFO:teuthology.orchestra.run.smithi043.stdout:(60/150): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-04-21T08:25:07.766 INFO:teuthology.orchestra.run.smithi043.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 381 kB/s | 25 kB 00:00 2024-04-21T08:25:07.792 INFO:teuthology.orchestra.run.smithi043.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 710 kB/s | 47 kB 00:00 2024-04-21T08:25:07.842 INFO:teuthology.orchestra.run.smithi043.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-21T08:25:07.868 INFO:teuthology.orchestra.run.smithi043.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-04-21T08:25:07.909 INFO:teuthology.orchestra.run.smithi043.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 527 kB/s | 35 kB 00:00 2024-04-21T08:25:08.006 INFO:teuthology.orchestra.run.smithi112.stdout:(10/150): ceph-radosgw-19.0.0-3154.ga630765c.el 2.0 MB/s | 11 MB 00:05 2024-04-21T08:25:08.018 INFO:teuthology.orchestra.run.smithi043.stdout:(66/150): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-04-21T08:25:08.110 INFO:teuthology.orchestra.run.smithi043.stdout:(67/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 21 MB/s | 4.1 MB 00:00 2024-04-21T08:25:08.161 INFO:teuthology.orchestra.run.smithi043.stdout:(68/150): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-21T08:25:08.172 INFO:teuthology.orchestra.run.smithi112.stdout:(11/150): cephfs-mirror-19.0.0-3154.ga630765c.e 1.3 MB/s | 225 kB 00:00 2024-04-21T08:25:08.219 INFO:teuthology.orchestra.run.smithi043.stdout:(69/150): postgresql-contrib-13.14-1.el9.x86_64 8.2 MB/s | 911 kB 00:00 2024-04-21T08:25:08.244 INFO:teuthology.orchestra.run.smithi043.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-04-21T08:25:08.265 INFO:teuthology.orchestra.run.smithi112.stdout:(12/150): cephfs-shell-19.0.0-3154.ga630765c.el 294 kB/s | 27 kB 00:00 2024-04-21T08:25:08.337 INFO:teuthology.orchestra.run.smithi043.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-04-21T08:25:08.357 INFO:teuthology.orchestra.run.smithi112.stdout:(13/150): libcephfs-devel-19.0.0-3154.ga630765c 346 kB/s | 32 kB 00:00 2024-04-21T08:25:08.428 INFO:teuthology.orchestra.run.smithi043.stdout:(72/150): postgresql-server-13.14-1.el9.x86_64. 28 MB/s | 5.8 MB 00:00 2024-04-21T08:25:08.470 INFO:teuthology.orchestra.run.smithi043.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 655 kB/s | 87 kB 00:00 2024-04-21T08:25:08.621 INFO:teuthology.orchestra.run.smithi043.stdout:(74/150): python3-babel-2.9.1-2.el9.noarch.rpm 31 MB/s | 6.0 MB 00:00 2024-04-21T08:25:08.646 INFO:teuthology.orchestra.run.smithi043.stdout:(75/150): python3-devel-3.9.18-3.el9.x86_64.rpm 1.4 MB/s | 244 kB 00:00 2024-04-21T08:25:08.696 INFO:teuthology.orchestra.run.smithi043.stdout:(76/150): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-04-21T08:25:08.721 INFO:teuthology.orchestra.run.smithi043.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 341 kB/s | 26 kB 00:00 2024-04-21T08:25:08.764 INFO:teuthology.orchestra.run.smithi043.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 285 kB/s | 19 kB 00:00 2024-04-21T08:25:08.789 INFO:teuthology.orchestra.run.smithi043.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-04-21T08:25:08.831 INFO:teuthology.orchestra.run.smithi043.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 520 kB/s | 35 kB 00:00 2024-04-21T08:25:08.890 INFO:teuthology.orchestra.run.smithi112.stdout:(14/150): libcephfs2-19.0.0-3154.ga630765c.el9. 1.4 MB/s | 746 kB 00:00 2024-04-21T08:25:08.948 INFO:teuthology.orchestra.run.smithi043.stdout:(81/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-04-21T08:25:08.973 INFO:teuthology.orchestra.run.smithi043.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 250 kB 00:00 2024-04-21T08:25:09.015 INFO:teuthology.orchestra.run.smithi043.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-04-21T08:25:09.040 INFO:teuthology.orchestra.run.smithi043.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-04-21T08:25:09.048 INFO:teuthology.orchestra.run.smithi112.stdout:(15/150): libcephsqlite-19.0.0-3154.ga630765c.e 1.0 MB/s | 163 kB 00:00 2024-04-21T08:25:09.082 INFO:teuthology.orchestra.run.smithi043.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 629 kB/s | 42 kB 00:00 2024-04-21T08:25:09.107 INFO:teuthology.orchestra.run.smithi043.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-04-21T08:25:09.149 INFO:teuthology.orchestra.run.smithi043.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-21T08:25:09.175 INFO:teuthology.orchestra.run.smithi043.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-21T08:25:09.208 INFO:teuthology.orchestra.run.smithi043.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 869 kB/s | 51 kB 00:00 2024-04-21T08:25:09.216 INFO:teuthology.orchestra.run.smithi112.stdout:(16/150): librados-devel-19.0.0-3154.ga630765c. 760 kB/s | 127 kB 00:00 2024-04-21T08:25:09.233 INFO:teuthology.orchestra.run.smithi043.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 919 kB/s | 54 kB 00:00 2024-04-21T08:25:09.301 INFO:teuthology.orchestra.run.smithi043.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 625 kB/s | 42 kB 00:00 2024-04-21T08:25:09.376 INFO:teuthology.orchestra.run.smithi043.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 590 kB/s | 44 kB 00:00 2024-04-21T08:25:09.459 INFO:teuthology.orchestra.run.smithi043.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-04-21T08:25:09.557 INFO:teuthology.orchestra.run.smithi112.stdout:(17/150): libradosstriper1-19.0.0-3154.ga630765 1.4 MB/s | 501 kB 00:00 2024-04-21T08:25:09.618 INFO:teuthology.orchestra.run.smithi043.stdout:(94/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 40 MB/s | 16 MB 00:00 2024-04-21T08:25:09.621 INFO:teuthology.orchestra.run.smithi175.stdout:(10/150): ceph-osd-19.0.0-3154.ga630765c.el9.x8 5.5 MB/s | 17 MB 00:03 2024-04-21T08:25:09.652 INFO:teuthology.orchestra.run.smithi043.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 187 kB/s | 36 kB 00:00 2024-04-21T08:25:09.677 INFO:teuthology.orchestra.run.smithi043.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 805 kB/s | 47 kB 00:00 2024-04-21T08:25:09.696 INFO:teuthology.orchestra.run.smithi175.stdout:(11/150): cephfs-mirror-19.0.0-3154.ga630765c.e 2.9 MB/s | 225 kB 00:00 2024-04-21T08:25:09.736 INFO:teuthology.orchestra.run.smithi043.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-04-21T08:25:09.761 INFO:teuthology.orchestra.run.smithi043.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 692 kB/s | 58 kB 00:00 2024-04-21T08:25:09.763 INFO:teuthology.orchestra.run.smithi175.stdout:(12/150): cephfs-shell-19.0.0-3154.ga630765c.el 405 kB/s | 27 kB 00:00 2024-04-21T08:25:09.795 INFO:teuthology.orchestra.run.smithi043.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-21T08:25:09.820 INFO:teuthology.orchestra.run.smithi043.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-21T08:25:09.830 INFO:teuthology.orchestra.run.smithi175.stdout:(13/150): libcephfs-devel-19.0.0-3154.ga630765c 475 kB/s | 32 kB 00:00 2024-04-21T08:25:09.862 INFO:teuthology.orchestra.run.smithi043.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 670 kB/s | 45 kB 00:00 2024-04-21T08:25:09.937 INFO:teuthology.orchestra.run.smithi043.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 190 kB/s | 22 kB 00:00 2024-04-21T08:25:10.030 INFO:teuthology.orchestra.run.smithi175.stdout:(14/150): libcephfs2-19.0.0-3154.ga630765c.el9. 3.6 MB/s | 746 kB 00:00 2024-04-21T08:25:10.079 INFO:teuthology.orchestra.run.smithi043.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 3.9 MB/s | 863 kB 00:00 2024-04-21T08:25:10.105 INFO:teuthology.orchestra.run.smithi175.stdout:(15/150): libcephsqlite-19.0.0-3154.ga630765c.e 2.1 MB/s | 163 kB 00:00 2024-04-21T08:25:10.172 INFO:teuthology.orchestra.run.smithi175.stdout:(16/150): librados-devel-19.0.0-3154.ga630765c. 1.9 MB/s | 127 kB 00:00 2024-04-21T08:25:10.273 INFO:teuthology.orchestra.run.smithi175.stdout:(17/150): libradosstriper1-19.0.0-3154.ga630765 4.9 MB/s | 501 kB 00:00 2024-04-21T08:25:10.471 INFO:teuthology.orchestra.run.smithi043.stdout:(104/150): gperftools-libs-2.9.1-2.el9.x86_64.r 791 kB/s | 309 kB 00:00 2024-04-21T08:25:10.496 INFO:teuthology.orchestra.run.smithi043.stdout:(105/150): abseil-cpp-20211102.0-3.el9.x86_64.r 981 kB/s | 548 kB 00:00 2024-04-21T08:25:10.547 INFO:teuthology.orchestra.run.smithi043.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 260 kB/s | 19 kB 00:00 2024-04-21T08:25:10.616 INFO:teuthology.orchestra.run.smithi112.stdout:(18/150): ceph-test-19.0.0-3154.ga630765c.el9.x 7.4 MB/s | 49 MB 00:06 2024-04-21T08:25:10.697 INFO:teuthology.orchestra.run.smithi043.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 173 kB/s | 26 kB 00:00 2024-04-21T08:25:10.755 INFO:teuthology.orchestra.run.smithi043.stdout:(108/150): libarrow-9.0.0-10.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-21T08:25:10.781 INFO:teuthology.orchestra.run.smithi043.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 587 kB/s | 49 kB 00:00 2024-04-21T08:25:10.831 INFO:teuthology.orchestra.run.smithi043.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 899 kB/s | 67 kB 00:00 2024-04-21T08:25:10.841 INFO:teuthology.orchestra.run.smithi112.stdout:(19/150): ceph-common-19.0.0-3154.ga630765c.el9 2.1 MB/s | 21 MB 00:09 2024-04-21T08:25:10.856 INFO:teuthology.orchestra.run.smithi043.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-21T08:25:10.867 INFO:teuthology.orchestra.run.smithi112.stdout:(20/150): python3-ceph-argparse-19.0.0-3154.ga6 180 kB/s | 45 kB 00:00 2024-04-21T08:25:10.915 INFO:teuthology.orchestra.run.smithi043.stdout:(112/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 838 kB 00:00 2024-04-21T08:25:10.949 INFO:teuthology.orchestra.run.smithi043.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-04-21T08:25:10.958 INFO:teuthology.orchestra.run.smithi112.stdout:(21/150): python3-cephfs-19.0.0-3154.ga630765c. 1.7 MB/s | 162 kB 00:00 2024-04-21T08:25:10.984 INFO:teuthology.orchestra.run.smithi112.stdout:(22/150): python3-ceph-common-19.0.0-3154.ga630 948 kB/s | 135 kB 00:00 2024-04-21T08:25:10.990 INFO:teuthology.orchestra.run.smithi043.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 588 kB/s | 43 kB 00:00 2024-04-21T08:25:11.024 INFO:teuthology.orchestra.run.smithi043.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 428 kB/s | 32 kB 00:00 2024-04-21T08:25:11.040 INFO:teuthology.orchestra.run.smithi175.stdout:(18/150): librgw2-19.0.0-3154.ga630765c.el9.x86 7.0 MB/s | 5.4 MB 00:00 2024-04-21T08:25:11.057 INFO:teuthology.orchestra.run.smithi043.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-04-21T08:25:11.067 INFO:teuthology.orchestra.run.smithi112.stdout:(23/150): python3-rados-19.0.0-3154.ga630765c.e 2.9 MB/s | 321 kB 00:00 2024-04-21T08:25:11.098 INFO:teuthology.orchestra.run.smithi175.stdout:(19/150): python3-ceph-argparse-19.0.0-3154.ga6 775 kB/s | 45 kB 00:00 2024-04-21T08:25:11.099 INFO:teuthology.orchestra.run.smithi043.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-21T08:25:11.133 INFO:teuthology.orchestra.run.smithi043.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-04-21T08:25:11.134 INFO:teuthology.orchestra.run.smithi112.stdout:(24/150): python3-rbd-19.0.0-3154.ga630765c.el9 1.9 MB/s | 298 kB 00:00 2024-04-21T08:25:11.160 INFO:teuthology.orchestra.run.smithi112.stdout:(25/150): python3-rgw-19.0.0-3154.ga630765c.el9 1.1 MB/s | 100 kB 00:00 2024-04-21T08:25:11.175 INFO:teuthology.orchestra.run.smithi043.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-04-21T08:25:11.193 INFO:teuthology.orchestra.run.smithi112.stdout:(26/150): rbd-fuse-19.0.0-3154.ga630765c.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-21T08:25:11.200 INFO:teuthology.orchestra.run.smithi043.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 527 kB/s | 35 kB 00:00 2024-04-21T08:25:11.207 INFO:teuthology.orchestra.run.smithi175.stdout:(20/150): ceph-radosgw-19.0.0-3154.ga630765c.el 2.4 MB/s | 11 MB 00:04 2024-04-21T08:25:11.232 INFO:teuthology.orchestra.run.smithi175.stdout:(21/150): python3-ceph-common-19.0.0-3154.ga630 1.0 MB/s | 135 kB 00:00 2024-04-21T08:25:11.250 INFO:teuthology.orchestra.run.smithi043.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-04-21T08:25:11.283 INFO:teuthology.orchestra.run.smithi175.stdout:(22/150): python3-cephfs-19.0.0-3154.ga630765c. 2.1 MB/s | 162 kB 00:00 2024-04-21T08:25:11.302 INFO:teuthology.orchestra.run.smithi112.stdout:(27/150): rbd-nbd-19.0.0-3154.ga630765c.el9.x86 1.5 MB/s | 171 kB 00:00 2024-04-21T08:25:11.308 INFO:teuthology.orchestra.run.smithi175.stdout:(23/150): python3-rados-19.0.0-3154.ga630765c.e 4.2 MB/s | 321 kB 00:00 2024-04-21T08:25:11.317 INFO:teuthology.orchestra.run.smithi043.stdout:(122/150): python3-grpcio-1.46.7-10.el9.x86_64. 17 MB/s | 2.0 MB 00:00 2024-04-21T08:25:11.343 INFO:teuthology.orchestra.run.smithi043.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 1.6 MB/s | 144 kB 00:00 2024-04-21T08:25:11.375 INFO:teuthology.orchestra.run.smithi175.stdout:(24/150): python3-rgw-19.0.0-3154.ga630765c.el9 1.5 MB/s | 100 kB 00:00 2024-04-21T08:25:11.377 INFO:teuthology.orchestra.run.smithi112.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3154.g 457 kB/s | 34 kB 00:00 2024-04-21T08:25:11.384 INFO:teuthology.orchestra.run.smithi043.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 161 kB/s | 11 kB 00:00 2024-04-21T08:25:11.410 INFO:teuthology.orchestra.run.smithi043.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-04-21T08:25:11.425 INFO:teuthology.orchestra.run.smithi175.stdout:(25/150): python3-rbd-19.0.0-3154.ga630765c.el9 2.1 MB/s | 298 kB 00:00 2024-04-21T08:25:11.450 INFO:teuthology.orchestra.run.smithi175.stdout:(26/150): rbd-fuse-19.0.0-3154.ga630765c.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-21T08:25:11.452 INFO:teuthology.orchestra.run.smithi043.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 344 kB/s | 23 kB 00:00 2024-04-21T08:25:11.461 INFO:teuthology.orchestra.run.smithi112.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3154.ga630765 1.7 MB/s | 142 kB 00:00 2024-04-21T08:25:11.477 INFO:teuthology.orchestra.run.smithi043.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 289 kB/s | 19 kB 00:00 2024-04-21T08:25:11.517 INFO:teuthology.orchestra.run.smithi175.stdout:(27/150): rbd-nbd-19.0.0-3154.ga630765c.el9.x86 2.5 MB/s | 171 kB 00:00 2024-04-21T08:25:11.519 INFO:teuthology.orchestra.run.smithi043.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 294 kB/s | 20 kB 00:00 2024-04-21T08:25:11.584 INFO:teuthology.orchestra.run.smithi175.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3154.g 512 kB/s | 34 kB 00:00 2024-04-21T08:25:11.611 INFO:teuthology.orchestra.run.smithi112.stdout:(30/150): rbd-mirror-19.0.0-3154.ga630765c.el9. 6.8 MB/s | 3.1 MB 00:00 2024-04-21T08:25:11.620 INFO:teuthology.orchestra.run.smithi043.stdout:(129/150): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-04-21T08:25:11.643 INFO:teuthology.orchestra.run.smithi175.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3154.ga630765 2.4 MB/s | 142 kB 00:00 2024-04-21T08:25:11.645 INFO:teuthology.orchestra.run.smithi043.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 369 kB/s | 46 kB 00:00 2024-04-21T08:25:11.695 INFO:teuthology.orchestra.run.smithi043.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-21T08:25:11.712 INFO:teuthology.orchestra.run.smithi043.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 863 kB/s | 58 kB 00:00 2024-04-21T08:25:11.771 INFO:teuthology.orchestra.run.smithi043.stdout:(133/150): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-21T08:25:11.788 INFO:teuthology.orchestra.run.smithi043.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-04-21T08:25:11.855 INFO:teuthology.orchestra.run.smithi043.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-04-21T08:25:11.881 INFO:teuthology.orchestra.run.smithi043.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 308 kB/s | 28 kB 00:00 2024-04-21T08:25:11.922 INFO:teuthology.orchestra.run.smithi043.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 459 kB/s | 31 kB 00:00 2024-04-21T08:25:11.958 INFO:teuthology.orchestra.run.smithi043.stdout:(138/150): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-21T08:25:11.998 INFO:teuthology.orchestra.run.smithi043.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 786 kB/s | 59 kB 00:00 2024-04-21T08:25:12.023 INFO:teuthology.orchestra.run.smithi043.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 536 kB/s | 36 kB 00:00 2024-04-21T08:25:12.073 INFO:teuthology.orchestra.run.smithi043.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 680 kB/s | 51 kB 00:00 2024-04-21T08:25:12.098 INFO:teuthology.orchestra.run.smithi043.stdout:(142/150): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-21T08:25:12.140 INFO:teuthology.orchestra.run.smithi043.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-04-21T08:25:12.182 INFO:teuthology.orchestra.run.smithi043.stdout:(144/150): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-21T08:25:12.208 INFO:teuthology.orchestra.run.smithi043.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 298 kB/s | 20 kB 00:00 2024-04-21T08:25:12.237 INFO:teuthology.orchestra.run.smithi112.stdout:(31/150): librgw2-19.0.0-3154.ga630765c.el9.x86 2.0 MB/s | 5.4 MB 00:02 2024-04-21T08:25:12.258 INFO:teuthology.orchestra.run.smithi043.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-21T08:25:12.308 INFO:teuthology.orchestra.run.smithi043.stdout:(147/150): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-21T08:25:12.334 INFO:teuthology.orchestra.run.smithi175.stdout:(30/150): ceph-mgr-dashboard-19.0.0-3154.ga6307 5.1 MB/s | 3.5 MB 00:00 2024-04-21T08:25:12.404 INFO:teuthology.orchestra.run.smithi112.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3154.ga6 1.5 MB/s | 249 kB 00:00 2024-04-21T08:25:12.495 INFO:teuthology.orchestra.run.smithi112.stdout:(33/150): ceph-mgr-rook-19.0.0-3154.ga630765c.e 539 kB/s | 49 kB 00:00 2024-04-21T08:25:12.596 INFO:teuthology.orchestra.run.smithi112.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3154.ga 165 kB/s | 16 kB 00:00 2024-04-21T08:25:12.677 INFO:teuthology.orchestra.run.smithi175.stdout:(31/150): rbd-mirror-19.0.0-3154.ga630765c.el9. 2.5 MB/s | 3.1 MB 00:01 2024-04-21T08:25:12.796 INFO:teuthology.orchestra.run.smithi112.stdout:(35/150): ceph-mgr-diskprediction-local-19.0.0- 6.3 MB/s | 7.4 MB 00:01 2024-04-21T08:25:12.821 INFO:teuthology.orchestra.run.smithi112.stdout:(36/150): ceph-volume-19.0.0-3154.ga630765c.el9 1.2 MB/s | 273 kB 00:00 2024-04-21T08:25:12.905 INFO:teuthology.orchestra.run.smithi112.stdout:(37/150): cephfs-top-19.0.0-3154.ga630765c.el9. 219 kB/s | 18 kB 00:00 2024-04-21T08:25:12.938 INFO:teuthology.orchestra.run.smithi112.stdout:(38/150): cephadm-19.0.0-3154.ga630765c.el9.noa 5.2 MB/s | 762 kB 00:00 2024-04-21T08:25:13.039 INFO:teuthology.orchestra.run.smithi112.stdout:(39/150): ceph-mgr-dashboard-19.0.0-3154.ga6307 2.2 MB/s | 3.5 MB 00:01 2024-04-21T08:25:13.080 INFO:teuthology.orchestra.run.smithi112.stdout:(40/150): fuse-2.9.9-15.el9.x86_64.rpm 458 kB/s | 80 kB 00:00 2024-04-21T08:25:13.119 INFO:teuthology.orchestra.run.smithi175.stdout:(32/150): ceph-test-19.0.0-3154.ga630765c.el9.x 8.1 MB/s | 49 MB 00:06 2024-04-21T08:25:13.142 INFO:teuthology.orchestra.run.smithi043.stdout:(148/150): librados2-19.0.0-3154.ga630765c.el9. 3.8 MB/s | 3.4 MB 00:00 2024-04-21T08:25:13.144 INFO:teuthology.orchestra.run.smithi175.stdout:(33/150): ceph-mgr-modules-core-19.0.0-3154.ga6 533 kB/s | 249 kB 00:00 2024-04-21T08:25:13.169 INFO:teuthology.orchestra.run.smithi175.stdout:(34/150): ceph-mgr-rook-19.0.0-3154.ga630765c.e 986 kB/s | 49 kB 00:00 2024-04-21T08:25:13.197 INFO:teuthology.orchestra.run.smithi112.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-04-21T08:25:13.203 INFO:teuthology.orchestra.run.smithi175.stdout:(35/150): ceph-prometheus-alerts-19.0.0-3154.ga 281 kB/s | 16 kB 00:00 2024-04-21T08:25:13.231 INFO:teuthology.orchestra.run.smithi112.stdout:(42/150): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-21T08:25:13.253 INFO:teuthology.orchestra.run.smithi175.stdout:(36/150): ceph-volume-19.0.0-3154.ga630765c.el9 3.2 MB/s | 273 kB 00:00 2024-04-21T08:25:13.290 INFO:teuthology.orchestra.run.smithi112.stdout:(43/150): libgfortran-11.4.1-3.el9.x86_64.rpm 2.2 MB/s | 803 kB 00:00 2024-04-21T08:25:13.295 INFO:teuthology.orchestra.run.smithi175.stdout:(37/150): cephfs-top-19.0.0-3154.ga630765c.el9. 439 kB/s | 18 kB 00:00 2024-04-21T08:25:13.315 INFO:teuthology.orchestra.run.smithi112.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2024-04-21T08:25:13.349 INFO:teuthology.orchestra.run.smithi112.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-21T08:25:13.374 INFO:teuthology.orchestra.run.smithi112.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-21T08:25:13.407 INFO:teuthology.orchestra.run.smithi112.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-21T08:25:13.474 INFO:teuthology.orchestra.run.smithi112.stdout:(48/150): python3-cryptography-36.0.1-4.el9.x86 4.4 MB/s | 1.2 MB 00:00 2024-04-21T08:25:13.478 INFO:teuthology.orchestra.run.smithi175.stdout:(38/150): ceph-mgr-diskprediction-local-19.0.0- 6.5 MB/s | 7.4 MB 00:01 2024-04-21T08:25:13.512 INFO:teuthology.orchestra.run.smithi175.stdout:(39/150): cephadm-19.0.0-3154.ga630765c.el9.noa 2.4 MB/s | 762 kB 00:00 2024-04-21T08:25:13.575 INFO:teuthology.orchestra.run.smithi043.stdout:(149/150): librbd1-19.0.0-3154.ga630765c.el9.x8 2.5 MB/s | 3.1 MB 00:01 2024-04-21T08:25:13.650 INFO:teuthology.orchestra.run.smithi112.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 610 kB/s | 107 kB 00:00 2024-04-21T08:25:13.675 INFO:teuthology.orchestra.run.smithi112.stdout:(50/150): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-04-21T08:25:13.702 INFO:teuthology.orchestra.run.smithi112.stdout:(51/150): elfutils-libelf-devel-0.190-2.el9.x86 429 kB/s | 21 kB 00:00 2024-04-21T08:25:13.721 INFO:teuthology.orchestra.run.smithi175.stdout:(40/150): fuse-2.9.9-15.el9.x86_64.rpm 189 kB/s | 80 kB 00:00 2024-04-21T08:25:13.751 INFO:teuthology.orchestra.run.smithi112.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 649 kB/s | 33 kB 00:00 2024-04-21T08:25:13.776 INFO:teuthology.orchestra.run.smithi112.stdout:(53/150): bison-3.7.4-5.el9.x86_64.rpm 2.5 MB/s | 943 kB 00:00 2024-04-21T08:25:13.801 INFO:teuthology.orchestra.run.smithi112.stdout:(54/150): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-04-21T08:25:13.826 INFO:teuthology.orchestra.run.smithi112.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-04-21T08:25:13.852 INFO:teuthology.orchestra.run.smithi112.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-04-21T08:25:13.885 INFO:teuthology.orchestra.run.smithi112.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-21T08:25:13.911 INFO:teuthology.orchestra.run.smithi112.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 772 kB/s | 45 kB 00:00 2024-04-21T08:25:14.012 INFO:teuthology.orchestra.run.smithi175.stdout:(41/150): libquadmath-11.4.1-3.el9.x86_64.rpm 390 kB/s | 195 kB 00:00 2024-04-21T08:25:14.061 INFO:teuthology.orchestra.run.smithi112.stdout:(59/150): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-04-21T08:25:14.063 INFO:teuthology.orchestra.run.smithi175.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 741 kB/s | 253 kB 00:00 2024-04-21T08:25:14.086 INFO:teuthology.orchestra.run.smithi112.stdout:(60/150): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-04-21T08:25:14.136 INFO:teuthology.orchestra.run.smithi112.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 339 kB/s | 25 kB 00:00 2024-04-21T08:25:14.153 INFO:teuthology.orchestra.run.smithi112.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 707 kB/s | 47 kB 00:00 2024-04-21T08:25:14.172 INFO:teuthology.orchestra.run.smithi175.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 983 kB/s | 106 kB 00:00 2024-04-21T08:25:14.220 INFO:teuthology.orchestra.run.smithi112.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-21T08:25:14.246 INFO:teuthology.orchestra.run.smithi112.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-04-21T08:25:14.263 INFO:teuthology.orchestra.run.smithi112.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 844 kB/s | 35 kB 00:00 2024-04-21T08:25:14.280 INFO:teuthology.orchestra.run.smithi175.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-04-21T08:25:14.305 INFO:teuthology.orchestra.run.smithi175.stdout:(45/150): libgfortran-11.4.1-3.el9.x86_64.rpm 972 kB/s | 803 kB 00:00 2024-04-21T08:25:14.350 INFO:teuthology.orchestra.run.smithi043.stdout:(150/150): ceph-test-19.0.0-3154.ga630765c.el9. 3.6 MB/s | 49 MB 00:13 2024-04-21T08:25:14.358 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-04-21T08:25:14.358 INFO:teuthology.orchestra.run.smithi043.stdout:Total 13 MB/s | 218 MB 00:16 2024-04-21T08:25:14.389 INFO:teuthology.orchestra.run.smithi175.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-04-21T08:25:14.422 INFO:teuthology.orchestra.run.smithi175.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-04-21T08:25:14.464 INFO:teuthology.orchestra.run.smithi175.stdout:(48/150): python3-cryptography-36.0.1-4.el9.x86 2.7 MB/s | 1.2 MB 00:00 2024-04-21T08:25:14.614 INFO:teuthology.orchestra.run.smithi112.stdout:(66/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.5 MB/s | 3.0 MB 00:00 2024-04-21T08:25:14.640 INFO:teuthology.orchestra.run.smithi175.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 612 kB/s | 107 kB 00:00 2024-04-21T08:25:14.681 INFO:teuthology.orchestra.run.smithi175.stdout:(50/150): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-04-21T08:25:14.707 INFO:teuthology.orchestra.run.smithi175.stdout:(51/150): elfutils-libelf-devel-0.190-2.el9.x86 322 kB/s | 21 kB 00:00 2024-04-21T08:25:14.774 INFO:teuthology.orchestra.run.smithi175.stdout:(52/150): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-04-21T08:25:14.807 INFO:teuthology.orchestra.run.smithi175.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-04-21T08:25:14.832 INFO:teuthology.orchestra.run.smithi175.stdout:(54/150): flex-2.6.4-9.el9.x86_64.rpm 2.0 MB/s | 313 kB 00:00 2024-04-21T08:25:14.866 INFO:teuthology.orchestra.run.smithi175.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-04-21T08:25:14.891 INFO:teuthology.orchestra.run.smithi175.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-04-21T08:25:14.916 INFO:teuthology.orchestra.run.smithi175.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-21T08:25:14.950 INFO:teuthology.orchestra.run.smithi175.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 775 kB/s | 45 kB 00:00 2024-04-21T08:25:15.013 INFO:teuthology.orchestra.run.smithi112.stdout:(67/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 5.5 MB/s | 4.1 MB 00:00 2024-04-21T08:25:15.047 INFO:teuthology.orchestra.run.smithi112.stdout:(68/150): postgresql-13.14-1.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-04-21T08:25:15.050 INFO:teuthology.orchestra.run.smithi175.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-04-21T08:25:15.105 INFO:teuthology.orchestra.run.smithi112.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-21T08:25:15.109 INFO:teuthology.orchestra.run.smithi175.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 436 kB/s | 25 kB 00:00 2024-04-21T08:25:15.134 INFO:teuthology.orchestra.run.smithi175.stdout:(61/150): libtool-2.4.6-45.el9.x86_64.rpm 3.1 MB/s | 582 kB 00:00 2024-04-21T08:25:15.168 INFO:teuthology.orchestra.run.smithi175.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 813 kB/s | 47 kB 00:00 2024-04-21T08:25:15.205 INFO:teuthology.orchestra.run.smithi112.stdout:(70/150): postgresql-contrib-13.14-1.el9.x86_64 4.6 MB/s | 911 kB 00:00 2024-04-21T08:25:15.218 INFO:teuthology.orchestra.run.smithi175.stdout:(63/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.7 MB/s | 3.0 MB 00:00 2024-04-21T08:25:15.243 INFO:teuthology.orchestra.run.smithi175.stdout:(64/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-04-21T08:25:15.268 INFO:teuthology.orchestra.run.smithi175.stdout:(65/150): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-04-21T08:25:15.302 INFO:teuthology.orchestra.run.smithi175.stdout:(66/150): openblas-0.3.21-2.el9.x86_64.rpm 419 kB/s | 35 kB 00:00 2024-04-21T08:25:15.306 INFO:teuthology.orchestra.run.smithi112.stdout:(71/150): openblas-openmp-0.3.21-2.el9.x86_64.r 4.5 MB/s | 4.7 MB 00:01 2024-04-21T08:25:15.364 INFO:teuthology.orchestra.run.smithi112.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-21T08:25:15.423 INFO:teuthology.orchestra.run.smithi112.stdout:(73/150): protobuf-3.14.0-13.el9.x86_64.rpm 4.6 MB/s | 1.0 MB 00:00 2024-04-21T08:25:15.439 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-04-21T08:25:15.503 INFO:teuthology.orchestra.run.smithi175.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-04-21T08:25:15.515 INFO:teuthology.orchestra.run.smithi112.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-04-21T08:25:15.579 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-04-21T08:25:15.579 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-04-21T08:25:15.607 INFO:teuthology.orchestra.run.smithi112.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-04-21T08:25:15.638 INFO:teuthology.orchestra.run.smithi175.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 6.7 MB/s | 911 kB 00:00 2024-04-21T08:25:15.665 INFO:teuthology.orchestra.run.smithi112.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 439 kB/s | 26 kB 00:00 2024-04-21T08:25:15.695 INFO:teuthology.orchestra.run.smithi175.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-21T08:25:15.716 INFO:teuthology.orchestra.run.smithi112.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-21T08:25:15.774 INFO:teuthology.orchestra.run.smithi112.stdout:(78/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-21T08:25:15.833 INFO:teuthology.orchestra.run.smithi112.stdout:(79/150): python3-markupsafe-1.1.1-12.el9.x86_6 597 kB/s | 35 kB 00:00 2024-04-21T08:25:15.937 INFO:teuthology.orchestra.run.smithi175.stdout:(70/150): openblas-openmp-0.3.21-2.el9.x86_64.r 6.8 MB/s | 4.7 MB 00:00 2024-04-21T08:25:16.137 INFO:teuthology.orchestra.run.smithi175.stdout:(71/150): postgresql-server-13.14-1.el9.x86_64. 13 MB/s | 5.8 MB 00:00 2024-04-21T08:25:16.170 INFO:teuthology.orchestra.run.smithi175.stdout:(72/150): protobuf-3.14.0-13.el9.x86_64.rpm 4.3 MB/s | 1.0 MB 00:00 2024-04-21T08:25:16.196 INFO:teuthology.orchestra.run.smithi175.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-21T08:25:16.254 INFO:teuthology.orchestra.run.smithi175.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-21T08:25:16.300 INFO:teuthology.orchestra.run.smithi112.stdout:(80/150): postgresql-server-13.14-1.el9.x86_64. 4.8 MB/s | 5.8 MB 00:01 2024-04-21T08:25:16.313 INFO:teuthology.orchestra.run.smithi175.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-04-21T08:25:16.363 INFO:teuthology.orchestra.run.smithi175.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-04-21T08:25:16.422 INFO:teuthology.orchestra.run.smithi175.stdout:(77/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 3.6 MB/s | 4.1 MB 00:01 2024-04-21T08:25:16.425 INFO:teuthology.orchestra.run.smithi112.stdout:(81/150): python3-babel-2.9.1-2.el9.noarch.rpm 5.6 MB/s | 6.0 MB 00:01 2024-04-21T08:25:16.438 INFO:teuthology.orchestra.run.smithi175.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 254 kB/s | 19 kB 00:00 2024-04-21T08:25:16.454 INFO:teuthology.orchestra.run.smithi112.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-04-21T08:25:16.478 INFO:teuthology.orchestra.run.smithi112.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 4.2 MB/s | 222 kB 00:00 2024-04-21T08:25:16.497 INFO:teuthology.orchestra.run.smithi175.stdout:(79/150): python3-markupsafe-1.1.1-12.el9.x86_6 598 kB/s | 35 kB 00:00 2024-04-21T08:25:16.511 INFO:teuthology.orchestra.run.smithi112.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-21T08:25:16.522 INFO:teuthology.orchestra.run.smithi175.stdout:(80/150): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-04-21T08:25:16.536 INFO:teuthology.orchestra.run.smithi112.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 720 kB/s | 42 kB 00:00 2024-04-21T08:25:16.595 INFO:teuthology.orchestra.run.smithi112.stdout:(86/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-21T08:25:16.615 INFO:teuthology.orchestra.run.smithi175.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-04-21T08:25:16.629 INFO:teuthology.orchestra.run.smithi112.stdout:(87/150): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-04-21T08:25:16.696 INFO:teuthology.orchestra.run.smithi112.stdout:(88/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.9 MB/s | 5.1 MB 00:00 2024-04-21T08:25:16.713 INFO:teuthology.orchestra.run.smithi112.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 609 kB/s | 51 kB 00:00 2024-04-21T08:25:16.714 INFO:teuthology.orchestra.run.smithi175.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-04-21T08:25:16.738 INFO:teuthology.orchestra.run.smithi112.stdout:(90/150): python3-pyasn1-modules-0.4.8-6.el9.no 1.9 MB/s | 279 kB 00:00 2024-04-21T08:25:16.755 INFO:teuthology.orchestra.run.smithi112.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 910 kB/s | 54 kB 00:00 2024-04-21T08:25:16.780 INFO:teuthology.orchestra.run.smithi112.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 990 kB/s | 42 kB 00:00 2024-04-21T08:25:16.797 INFO:teuthology.orchestra.run.smithi112.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-04-21T08:25:16.823 INFO:teuthology.orchestra.run.smithi175.stdout:(83/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-04-21T08:25:16.839 INFO:teuthology.orchestra.run.smithi112.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-21T08:25:16.848 INFO:teuthology.orchestra.run.smithi175.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 578 kB/s | 77 kB 00:00 2024-04-21T08:25:16.856 INFO:teuthology.orchestra.run.smithi112.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 613 kB/s | 36 kB 00:00 2024-04-21T08:25:16.881 INFO:teuthology.orchestra.run.smithi112.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-21T08:25:16.882 INFO:teuthology.orchestra.run.smithi175.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 724 kB/s | 42 kB 00:00 2024-04-21T08:25:16.907 INFO:teuthology.orchestra.run.smithi175.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-21T08:25:16.923 INFO:teuthology.orchestra.run.smithi112.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-04-21T08:25:16.948 INFO:teuthology.orchestra.run.smithi112.stdout:(98/150): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 305 kB 00:00 2024-04-21T08:25:16.990 INFO:teuthology.orchestra.run.smithi112.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-04-21T08:25:17.007 INFO:teuthology.orchestra.run.smithi112.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-21T08:25:17.008 INFO:teuthology.orchestra.run.smithi175.stdout:(87/150): python3-babel-2.9.1-2.el9.noarch.rpm 7.1 MB/s | 6.0 MB 00:00 2024-04-21T08:25:17.033 INFO:teuthology.orchestra.run.smithi175.stdout:(88/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-04-21T08:25:17.041 INFO:teuthology.orchestra.run.smithi112.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-04-21T08:25:17.058 INFO:teuthology.orchestra.run.smithi175.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 279 kB 00:00 2024-04-21T08:25:17.083 INFO:teuthology.orchestra.run.smithi175.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 676 kB/s | 51 kB 00:00 2024-04-21T08:25:17.108 INFO:teuthology.orchestra.run.smithi175.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 711 kB/s | 54 kB 00:00 2024-04-21T08:25:17.142 INFO:teuthology.orchestra.run.smithi175.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 712 kB/s | 42 kB 00:00 2024-04-21T08:25:17.167 INFO:teuthology.orchestra.run.smithi175.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 754 kB/s | 44 kB 00:00 2024-04-21T08:25:17.226 INFO:teuthology.orchestra.run.smithi175.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 615 kB/s | 36 kB 00:00 2024-04-21T08:25:17.243 INFO:teuthology.orchestra.run.smithi175.stdout:(95/150): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-04-21T08:25:17.283 INFO:teuthology.orchestra.run.smithi112.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 81 kB/s | 22 kB 00:00 2024-04-21T08:25:17.285 INFO:teuthology.orchestra.run.smithi175.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 807 kB/s | 47 kB 00:00 2024-04-21T08:25:17.345 INFO:teuthology.orchestra.run.smithi175.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 960 kB/s | 58 kB 00:00 2024-04-21T08:25:17.379 INFO:teuthology.orchestra.run.smithi175.stdout:(98/150): socat-1.7.4.1-5.el9.x86_64.rpm 2.2 MB/s | 305 kB 00:00 2024-04-21T08:25:17.403 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-04-21T08:25:17.404 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-04-21T08:25:17.404 INFO:teuthology.orchestra.run.smithi175.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-21T08:25:17.438 INFO:teuthology.orchestra.run.smithi175.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-21T08:25:17.463 INFO:teuthology.orchestra.run.smithi175.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 762 kB/s | 45 kB 00:00 2024-04-21T08:25:17.555 INFO:teuthology.orchestra.run.smithi175.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-04-21T08:25:17.781 INFO:teuthology.orchestra.run.smithi175.stdout:(103/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-04-21T08:25:17.814 INFO:teuthology.orchestra.run.smithi175.stdout:(104/150): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-04-21T08:25:17.833 INFO:teuthology.orchestra.run.smithi112.stdout:(103/150): abseil-cpp-20211102.0-3.el9.x86_64.r 997 kB/s | 548 kB 00:00 2024-04-21T08:25:17.967 INFO:teuthology.orchestra.run.smithi112.stdout:(104/150): gperftools-libs-2.9.1-2.el9.x86_64.r 2.3 MB/s | 309 kB 00:00 2024-04-21T08:25:18.056 INFO:teuthology.orchestra.run.smithi175.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 80 kB/s | 19 kB 00:00 2024-04-21T08:25:18.067 INFO:teuthology.orchestra.run.smithi112.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-04-21T08:25:18.231 INFO:teuthology.orchestra.run.smithi175.stdout:(106/150): abseil-cpp-20211102.0-3.el9.x86_64.r 811 kB/s | 548 kB 00:00 2024-04-21T08:25:18.265 INFO:teuthology.orchestra.run.smithi175.stdout:(107/150): gperftools-libs-2.9.1-2.el9.x86_64.r 638 kB/s | 309 kB 00:00 2024-04-21T08:25:18.323 INFO:teuthology.orchestra.run.smithi175.stdout:(108/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 282 kB/s | 26 kB 00:00 2024-04-21T08:25:18.357 INFO:teuthology.orchestra.run.smithi175.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 528 kB/s | 49 kB 00:00 2024-04-21T08:25:18.416 INFO:teuthology.orchestra.run.smithi175.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 734 kB/s | 67 kB 00:00 2024-04-21T08:25:18.458 INFO:teuthology.orchestra.run.smithi175.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-21T08:25:18.591 INFO:teuthology.orchestra.run.smithi175.stdout:(112/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.7 MB/s | 838 kB 00:00 2024-04-21T08:25:18.650 INFO:teuthology.orchestra.run.smithi175.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-04-21T08:25:18.675 INFO:teuthology.orchestra.run.smithi175.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 520 kB/s | 43 kB 00:00 2024-04-21T08:25:18.734 INFO:teuthology.orchestra.run.smithi112.stdout:(106/150): libarrow-9.0.0-10.el9.x86_64.rpm 6.6 MB/s | 4.4 MB 00:00 2024-04-21T08:25:18.742 INFO:teuthology.orchestra.run.smithi175.stdout:(115/150): libarrow-9.0.0-10.el9.x86_64.rpm 6.4 MB/s | 4.4 MB 00:00 2024-04-21T08:25:18.759 INFO:teuthology.orchestra.run.smithi112.stdout:(107/150): protobuf-compiler-3.14.0-13.el9.x86_ 502 kB/s | 863 kB 00:01 2024-04-21T08:25:18.767 INFO:teuthology.orchestra.run.smithi175.stdout:(116/150): python3-cachetools-4.2.4-1.el9.noarc 274 kB/s | 32 kB 00:00 2024-04-21T08:25:18.792 INFO:teuthology.orchestra.run.smithi175.stdout:(117/150): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-04-21T08:25:18.835 INFO:teuthology.orchestra.run.smithi112.stdout:(108/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-04-21T08:25:18.843 INFO:teuthology.orchestra.run.smithi175.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-21T08:25:18.868 INFO:teuthology.orchestra.run.smithi175.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-21T08:25:18.935 INFO:teuthology.orchestra.run.smithi112.stdout:(109/150): libunwind-1.6.2-1.el9.x86_64.rpm 673 kB/s | 67 kB 00:00 2024-04-21T08:25:18.935 INFO:teuthology.orchestra.run.smithi175.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.1 MB/s | 303 kB 00:00 2024-04-21T08:25:18.977 INFO:teuthology.orchestra.run.smithi175.stdout:(121/150): python3-colorama-0.4.4-12.el9.noarch 264 kB/s | 35 kB 00:00 2024-04-21T08:25:19.002 INFO:teuthology.orchestra.run.smithi175.stdout:(122/150): python3-google-auth-2.28.2-1.el9.noa 1.5 MB/s | 203 kB 00:00 2024-04-21T08:25:19.077 INFO:teuthology.orchestra.run.smithi175.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-21T08:25:19.103 INFO:teuthology.orchestra.run.smithi175.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 106 kB/s | 11 kB 00:00 2024-04-21T08:25:19.152 INFO:teuthology.orchestra.run.smithi175.stdout:(125/150): python3-grpcio-1.46.7-10.el9.x86_64. 9.4 MB/s | 2.0 MB 00:00 2024-04-21T08:25:19.178 INFO:teuthology.orchestra.run.smithi175.stdout:(126/150): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-04-21T08:25:19.185 INFO:teuthology.orchestra.run.smithi112.stdout:(110/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.6 MB/s | 16 MB 00:02 2024-04-21T08:25:19.203 INFO:teuthology.orchestra.run.smithi175.stdout:(127/150): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-04-21T08:25:19.211 INFO:teuthology.orchestra.run.smithi112.stdout:(111/150): liboath-2.6.7-2.el9.x86_64.rpm 108 kB/s | 49 kB 00:00 2024-04-21T08:25:19.236 INFO:teuthology.orchestra.run.smithi112.stdout:(112/150): luarocks-3.9.2-1.el9.noarch.rpm 502 kB/s | 151 kB 00:00 2024-04-21T08:25:19.245 INFO:teuthology.orchestra.run.smithi175.stdout:(128/150): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-04-21T08:25:19.279 INFO:teuthology.orchestra.run.smithi175.stdout:(129/150): python3-jaraco-text-3.2.0-6.el9.noar 195 kB/s | 20 kB 00:00 2024-04-21T08:25:19.312 INFO:teuthology.orchestra.run.smithi175.stdout:(130/150): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-04-21T08:25:19.338 INFO:teuthology.orchestra.run.smithi175.stdout:(131/150): python3-logutils-0.3.5-21.el9.noarch 501 kB/s | 46 kB 00:00 2024-04-21T08:25:19.345 INFO:teuthology.orchestra.run.smithi112.stdout:(113/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 401 kB/s | 43 kB 00:00 2024-04-21T08:25:19.371 INFO:teuthology.orchestra.run.smithi175.stdout:(132/150): python3-more-itertools-8.12.0-2.el9. 855 kB/s | 79 kB 00:00 2024-04-21T08:25:19.413 INFO:teuthology.orchestra.run.smithi175.stdout:(133/150): python3-natsort-7.1.1-5.el9.noarch.r 575 kB/s | 58 kB 00:00 2024-04-21T08:25:19.438 INFO:teuthology.orchestra.run.smithi175.stdout:(134/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-04-21T08:25:19.445 INFO:teuthology.orchestra.run.smithi112.stdout:(114/150): python3-cachetools-4.2.4-1.el9.noarc 322 kB/s | 32 kB 00:00 2024-04-21T08:25:19.464 INFO:teuthology.orchestra.run.smithi175.stdout:(135/150): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-04-21T08:25:19.506 INFO:teuthology.orchestra.run.smithi175.stdout:(136/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 976 kB/s | 90 kB 00:00 2024-04-21T08:25:19.545 INFO:teuthology.orchestra.run.smithi112.stdout:(115/150): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-04-21T08:25:19.547 INFO:teuthology.orchestra.run.smithi175.stdout:(137/150): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-04-21T08:25:19.573 INFO:teuthology.orchestra.run.smithi175.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 283 kB/s | 31 kB 00:00 2024-04-21T08:25:19.604 INFO:teuthology.orchestra.run.smithi112.stdout:(116/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.0 MB/s | 838 kB 00:00 2024-04-21T08:25:19.606 INFO:teuthology.orchestra.run.smithi175.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-21T08:25:19.640 INFO:teuthology.orchestra.run.smithi175.stdout:(140/150): python3-rsa-4.9-2.el9.noarch.rpm 642 kB/s | 59 kB 00:00 2024-04-21T08:25:19.662 INFO:teuthology.orchestra.run.smithi112.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-04-21T08:25:19.674 INFO:teuthology.orchestra.run.smithi175.stdout:(141/150): python3-tempora-5.0.0-2.el9.noarch.r 355 kB/s | 36 kB 00:00 2024-04-21T08:25:19.699 INFO:teuthology.orchestra.run.smithi175.stdout:(142/150): python3-typing-extensions-4.4.0-2.el 554 kB/s | 51 kB 00:00 2024-04-21T08:25:19.729 INFO:teuthology.orchestra.run.smithi112.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-04-21T08:25:19.741 INFO:teuthology.orchestra.run.smithi175.stdout:(143/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-21T08:25:19.766 INFO:teuthology.orchestra.run.smithi175.stdout:(144/150): python3-websocket-client-1.2.3-2.el9 973 kB/s | 90 kB 00:00 2024-04-21T08:25:19.796 INFO:teuthology.orchestra.run.smithi112.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-04-21T08:25:19.800 INFO:teuthology.orchestra.run.smithi175.stdout:(145/150): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-04-21T08:25:19.830 INFO:teuthology.orchestra.run.smithi112.stdout:(120/150): python3-asyncssh-2.13.2-5.el9.noarch 886 kB/s | 548 kB 00:00 2024-04-21T08:25:19.833 INFO:teuthology.orchestra.run.smithi175.stdout:(146/150): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-04-21T08:25:19.847 INFO:teuthology.orchestra.run.smithi112.stdout:(121/150): python3-colorama-0.4.4-12.el9.noarch 301 kB/s | 35 kB 00:00 2024-04-21T08:25:19.867 INFO:teuthology.orchestra.run.smithi175.stdout:(147/150): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-21T08:25:19.905 INFO:teuthology.orchestra.run.smithi112.stdout:(122/150): python3-google-auth-2.28.2-1.el9.noa 1.8 MB/s | 203 kB 00:00 2024-04-21T08:25:19.925 INFO:teuthology.orchestra.run.smithi175.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-21T08:25:19.973 INFO:teuthology.orchestra.run.smithi112.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-21T08:25:20.006 INFO:teuthology.orchestra.run.smithi112.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 106 kB/s | 11 kB 00:00 2024-04-21T08:25:20.073 INFO:teuthology.orchestra.run.smithi112.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-04-21T08:25:20.107 INFO:teuthology.orchestra.run.smithi112.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-04-21T08:25:20.174 INFO:teuthology.orchestra.run.smithi112.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-04-21T08:25:20.207 INFO:teuthology.orchestra.run.smithi112.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-04-21T08:25:20.257 INFO:teuthology.orchestra.run.smithi112.stdout:(129/150): python3-grpcio-1.46.7-10.el9.x86_64. 4.8 MB/s | 2.0 MB 00:00 2024-04-21T08:25:20.409 INFO:teuthology.orchestra.run.smithi112.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 229 kB/s | 46 kB 00:00 2024-04-21T08:25:20.575 INFO:teuthology.orchestra.run.smithi112.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 249 kB/s | 79 kB 00:00 2024-04-21T08:25:20.600 INFO:teuthology.orchestra.run.smithi112.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 303 kB/s | 58 kB 00:00 2024-04-21T08:25:20.634 INFO:teuthology.orchestra.run.smithi112.stdout:(133/150): python3-kubernetes-26.1.0-2.el9.noar 2.2 MB/s | 1.0 MB 00:00 2024-04-21T08:25:20.709 INFO:teuthology.orchestra.run.smithi112.stdout:(134/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-04-21T08:25:20.734 INFO:teuthology.orchestra.run.smithi112.stdout:(135/150): python3-portend-3.1.0-2.el9.noarch.r 123 kB/s | 16 kB 00:00 2024-04-21T08:25:20.759 INFO:teuthology.orchestra.run.smithi112.stdout:(136/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 716 kB/s | 90 kB 00:00 2024-04-21T08:25:20.818 INFO:teuthology.orchestra.run.smithi112.stdout:(137/150): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-04-21T08:25:20.843 INFO:teuthology.orchestra.run.smithi112.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 282 kB/s | 31 kB 00:00 2024-04-21T08:25:20.868 INFO:teuthology.orchestra.run.smithi112.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-21T08:25:20.919 INFO:teuthology.orchestra.run.smithi112.stdout:(140/150): python3-rsa-4.9-2.el9.noarch.rpm 586 kB/s | 59 kB 00:00 2024-04-21T08:25:20.953 INFO:teuthology.orchestra.run.smithi112.stdout:(141/150): python3-tempora-5.0.0-2.el9.noarch.r 327 kB/s | 36 kB 00:00 2024-04-21T08:25:20.978 INFO:teuthology.orchestra.run.smithi112.stdout:(142/150): python3-typing-extensions-4.4.0-2.el 467 kB/s | 51 kB 00:00 2024-04-21T08:25:21.045 INFO:teuthology.orchestra.run.smithi112.stdout:(143/150): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-21T08:25:21.070 INFO:teuthology.orchestra.run.smithi112.stdout:(144/150): python3-websocket-client-1.2.3-2.el9 765 kB/s | 90 kB 00:00 2024-04-21T08:25:21.112 INFO:teuthology.orchestra.run.smithi112.stdout:(145/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-04-21T08:25:21.146 INFO:teuthology.orchestra.run.smithi112.stdout:(146/150): python3-zc-lockfile-2.0-10.el9.noarc 198 kB/s | 20 kB 00:00 2024-04-21T08:25:21.188 INFO:teuthology.orchestra.run.smithi112.stdout:(147/150): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-04-21T08:25:21.413 INFO:teuthology.orchestra.run.smithi112.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 5.3 MB/s | 1.6 MB 00:00 2024-04-21T08:25:21.884 INFO:teuthology.orchestra.run.smithi175.stdout:(149/150): librbd1-19.0.0-3154.ga630765c.el9.x8 1.6 MB/s | 3.1 MB 00:02 2024-04-21T08:25:22.272 INFO:teuthology.orchestra.run.smithi112.stdout:(149/150): librbd1-19.0.0-3154.ga630765c.el9.x8 2.9 MB/s | 3.1 MB 00:01 2024-04-21T08:25:22.467 INFO:teuthology.orchestra.run.smithi175.stdout:(150/150): librados2-19.0.0-3154.ga630765c.el9. 1.3 MB/s | 3.4 MB 00:02 2024-04-21T08:25:22.470 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-04-21T08:25:22.471 INFO:teuthology.orchestra.run.smithi175.stdout:Total 11 MB/s | 218 MB 00:19 2024-04-21T08:25:23.305 INFO:teuthology.orchestra.run.smithi112.stdout:(150/150): librados2-19.0.0-3154.ga630765c.el9. 1.6 MB/s | 3.4 MB 00:02 2024-04-21T08:25:23.307 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-04-21T08:25:23.308 INFO:teuthology.orchestra.run.smithi112.stdout:Total 9.5 MB/s | 218 MB 00:23 2024-04-21T08:25:23.427 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-04-21T08:25:23.526 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-04-21T08:25:23.547 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-21T08:25:23.652 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-04-21T08:25:23.652 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-04-21T08:25:23.712 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-21T08:25:23.759 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-21T08:25:24.102 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-21T08:25:24.221 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T08:25:24.329 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T08:25:24.355 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:25:24.377 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-04-21T08:25:24.431 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:25:24.481 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T08:25:24.503 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-04-21T08:25:24.503 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-04-21T08:25:24.509 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-21T08:25:24.535 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-21T08:25:24.698 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-21T08:25:24.758 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-21T08:25:24.785 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:25:24.891 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:25:24.920 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T08:25:24.982 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T08:25:25.004 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T08:25:25.026 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T08:25:25.223 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T08:25:25.254 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T08:25:25.367 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-21T08:25:25.379 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-04-21T08:25:25.380 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-04-21T08:25:25.471 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-21T08:25:25.503 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-21T08:25:25.540 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-21T08:25:25.616 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-21T08:25:25.655 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-21T08:25:25.699 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-21T08:25:25.748 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-21T08:25:25.784 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-21T08:25:25.857 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-21T08:25:25.907 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-21T08:25:25.936 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-21T08:25:25.980 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-21T08:25:26.059 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-21T08:25:26.091 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 30/152 2024-04-21T08:25:26.135 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 31/152 2024-04-21T08:25:26.163 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 32/152 2024-04-21T08:25:26.221 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-04-21T08:25:26.222 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-04-21T08:25:26.238 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-21T08:25:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-21T08:25:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-21T08:25:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-21T08:25:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-21T08:25:27.150 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-21T08:25:27.238 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-21T08:25:27.271 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-21T08:25:27.323 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-21T08:25:27.361 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 42/152 2024-04-21T08:25:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-21T08:25:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-21T08:25:27.525 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-21T08:25:27.562 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-21T08:25:27.599 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-21T08:25:27.646 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-21T08:25:27.682 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-21T08:25:27.725 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-21T08:25:27.767 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-21T08:25:27.911 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-21T08:25:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-21T08:25:28.169 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-21T08:25:28.212 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-21T08:25:28.311 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-21T08:25:28.343 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-21T08:25:28.384 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-21T08:25:28.426 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-21T08:25:28.459 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-21T08:25:28.537 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-21T08:25:28.668 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-21T08:25:28.700 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-21T08:25:28.728 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-21T08:25:28.762 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-21T08:25:28.807 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-21T08:25:28.844 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-21T08:25:28.884 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-21T08:25:28.914 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-21T08:25:28.948 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-21T08:25:29.065 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-21T08:25:29.116 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-21T08:25:29.150 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-21T08:25:29.242 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-21T08:25:29.289 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-21T08:25:29.319 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-21T08:25:29.368 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-21T08:25:29.479 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-21T08:25:29.531 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-21T08:25:30.306 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-21T08:25:30.351 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-21T08:25:30.385 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-21T08:25:30.492 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-21T08:25:30.544 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-21T08:25:30.626 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-21T08:25:30.662 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-21T08:25:30.847 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-21T08:25:30.876 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-21T08:25:30.957 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-21T08:25:31.166 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-04-21T08:25:31.228 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-21T08:25:31.360 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-21T08:25:31.609 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-21T08:25:31.650 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-21T08:25:31.781 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-21T08:25:32.011 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-21T08:25:32.023 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-04-21T08:25:32.070 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T08:25:32.091 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-21T08:25:32.166 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-21T08:25:32.193 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T08:25:32.221 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:25:32.313 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:25:32.353 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-21T08:25:32.354 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T08:25:32.384 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-21T08:25:32.409 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-21T08:25:32.568 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-21T08:25:32.615 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-21T08:25:32.643 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:25:32.737 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-21T08:25:32.738 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:25:32.761 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T08:25:32.806 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T08:25:32.817 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T08:25:32.828 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T08:25:32.851 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T08:25:32.945 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T08:25:32.968 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:25:33.051 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:25:33.062 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T08:25:33.093 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T08:25:33.094 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T08:25:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-21T08:25:33.122 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-21T08:25:33.148 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-21T08:25:33.189 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-21T08:25:33.202 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-21T08:25:33.217 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-21T08:25:33.251 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-21T08:25:33.320 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-21T08:25:33.320 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-21T08:25:33.335 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-21T08:25:33.362 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-21T08:25:33.370 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-21T08:25:33.390 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:25:33.445 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-21T08:25:33.477 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:25:33.486 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-21T08:25:33.499 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T08:25:33.529 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-21T08:25:33.562 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T08:25:33.570 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-21T08:25:33.579 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-21T08:25:33.583 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T08:25:33.604 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T08:25:33.606 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-21T08:25:33.613 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T08:25:33.677 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-21T08:25:33.705 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T08:25:33.730 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-21T08:25:33.735 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 98/152 2024-04-21T08:25:33.759 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-21T08:25:33.771 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-21T08:25:33.791 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T08:25:33.803 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-21T08:25:33.825 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T08:25:33.880 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-21T08:25:33.922 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 30/152 2024-04-21T08:25:33.931 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-21T08:25:33.965 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 31/152 2024-04-21T08:25:33.994 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 32/152 2024-04-21T08:25:34.029 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-21T08:25:34.067 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-21T08:25:34.074 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-21T08:25:34.117 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-21T08:25:34.117 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-21T08:25:34.184 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-21T08:25:34.193 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-21T08:25:34.225 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-21T08:25:34.267 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-21T08:25:34.276 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-21T08:25:34.301 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-21T08:25:34.307 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T08:25:34.328 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-21T08:25:34.339 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-21T08:25:34.379 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T08:25:34.396 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-21T08:25:34.399 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-21T08:25:34.401 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 102/152 2024-04-21T08:25:34.461 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-21T08:25:34.490 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-21T08:25:34.525 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-21T08:25:34.603 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-21T08:25:34.636 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 30/152 2024-04-21T08:25:34.689 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 31/152 2024-04-21T08:25:34.725 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 32/152 2024-04-21T08:25:34.798 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-21T08:25:34.839 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-21T08:25:34.916 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-21T08:25:34.959 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-21T08:25:35.051 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-21T08:25:35.055 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-21T08:25:35.086 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-21T08:25:35.118 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-21T08:25:35.137 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-21T08:25:35.175 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 42/152 2024-04-21T08:25:35.208 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-21T08:25:35.264 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-21T08:25:35.338 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-21T08:25:35.376 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-21T08:25:35.412 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-21T08:25:35.460 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-21T08:25:35.495 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-21T08:25:35.538 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-21T08:25:35.581 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-21T08:25:35.664 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-21T08:25:35.720 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-21T08:25:35.757 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-21T08:25:35.769 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-21T08:25:35.792 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-21T08:25:35.798 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-21T08:25:35.834 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-21T08:25:35.840 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-21T08:25:35.873 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 42/152 2024-04-21T08:25:35.906 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-21T08:25:35.937 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-21T08:25:35.962 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-21T08:25:35.973 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-21T08:25:36.014 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-21T08:25:36.035 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-21T08:25:36.039 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-21T08:25:36.064 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-21T08:25:36.073 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-21T08:25:36.110 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-21T08:25:36.140 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-21T08:25:36.157 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-21T08:25:36.193 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-21T08:25:36.228 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-21T08:25:36.269 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-21T08:25:36.270 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-21T08:25:36.312 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-21T08:25:36.340 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-21T08:25:36.375 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-21T08:25:36.413 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-21T08:25:36.419 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-21T08:25:36.457 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-21T08:25:36.458 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-21T08:25:36.488 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-21T08:25:36.496 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-21T08:25:36.526 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-21T08:25:36.530 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-21T08:25:36.561 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-21T08:25:36.594 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-21T08:25:36.628 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-21T08:25:36.636 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-21T08:25:36.661 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-21T08:25:36.671 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-21T08:25:36.711 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-21T08:25:36.737 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-21T08:25:36.750 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-21T08:25:36.762 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-21T08:25:36.775 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:25:36.801 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-21T08:25:36.814 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:25:36.831 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-21T08:25:36.838 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-21T08:25:36.879 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-21T08:25:36.966 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-21T08:25:36.979 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-21T08:25:37.003 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-21T08:25:37.034 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-21T08:25:37.038 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-21T08:25:37.073 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-21T08:25:37.117 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-21T08:25:37.155 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-21T08:25:37.195 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-21T08:25:37.225 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-21T08:25:37.250 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-21T08:25:37.284 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-21T08:25:37.326 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-21T08:25:37.360 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-21T08:25:37.383 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:25:37.426 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T08:25:37.450 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-21T08:25:37.499 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-21T08:25:37.520 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T08:25:37.520 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-21T08:25:37.520 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-21T08:25:37.521 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:25:37.530 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-21T08:25:37.553 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T08:25:37.578 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-21T08:25:37.680 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-21T08:25:37.742 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-21T08:25:37.798 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-21T08:25:37.847 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-21T08:25:37.880 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-21T08:25:37.984 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-21T08:25:38.039 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-21T08:25:38.120 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-21T08:25:38.165 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-21T08:25:38.343 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-21T08:25:38.371 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-21T08:25:38.450 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-21T08:25:38.507 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-21T08:25:38.562 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-21T08:25:38.596 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-21T08:25:38.698 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-21T08:25:38.754 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-21T08:25:38.843 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-21T08:25:38.881 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-21T08:25:39.057 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-21T08:25:39.087 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-21T08:25:39.120 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-21T08:25:39.163 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-21T08:25:39.231 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-21T08:25:39.831 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-21T08:25:40.169 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-21T08:25:40.514 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-21T08:25:40.600 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-21T08:25:40.620 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-21T08:25:40.653 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-21T08:25:40.967 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-21T08:25:41.007 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T08:25:41.098 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T08:25:41.128 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 98/152 2024-04-21T08:25:41.164 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-21T08:25:41.459 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-21T08:25:41.541 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-21T08:25:41.570 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-21T08:25:41.602 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-21T08:25:41.667 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-21T08:25:41.718 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T08:25:41.789 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T08:25:41.811 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 102/152 2024-04-21T08:25:41.914 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-21T08:25:41.965 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T08:25:42.056 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T08:25:42.086 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 98/152 2024-04-21T08:25:42.122 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-21T08:25:42.602 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-21T08:25:42.634 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T08:25:42.706 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T08:25:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 102/152 2024-04-21T08:25:44.054 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:25:44.091 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:25:44.637 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:25:44.677 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T08:25:44.786 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T08:25:44.786 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-21T08:25:44.786 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-21T08:25:44.786 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-21T08:25:44.820 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T08:25:44.979 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:25:45.016 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:25:45.558 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:25:45.603 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T08:25:45.692 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T08:25:45.692 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-21T08:25:45.692 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-21T08:25:45.692 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-21T08:25:45.721 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T08:25:49.474 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T08:25:49.475 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /sys 2024-04-21T08:25:49.475 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /proc 2024-04-21T08:25:49.475 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /mnt 2024-04-21T08:25:49.475 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /var/tmp 2024-04-21T08:25:49.475 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /home 2024-04-21T08:25:49.475 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /root 2024-04-21T08:25:49.475 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /tmp 2024-04-21T08:25:49.475 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:25:49.657 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T08:25:49.676 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T08:25:49.676 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:25:49.676 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-21T08:25:49.677 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:25:51.227 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T08:25:51.254 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T08:25:51.254 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:25:51.254 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-21T08:25:51.254 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T08:25:51.255 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T08:25:51.255 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:25:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T08:25:51.518 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T08:25:51.518 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:25:51.519 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-21T08:25:51.519 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T08:25:51.519 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T08:25:51.519 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:25:51.979 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T08:25:52.002 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T08:25:52.002 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:25:52.002 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-21T08:25:52.003 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T08:25:52.003 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T08:25:52.003 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:25:52.101 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-21T08:25:52.167 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-21T08:25:52.231 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-21T08:25:52.426 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-21T08:25:52.480 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-21T08:25:52.563 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-21T08:25:52.660 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-21T08:25:52.803 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-21T08:25:52.847 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-21T08:25:52.879 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-21T08:25:52.906 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-21T08:25:53.317 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T08:25:53.346 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T08:25:53.379 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 122/152 2024-04-21T08:25:53.442 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 123/152 2024-04-21T08:25:53.469 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T08:25:54.562 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T08:25:54.596 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T08:25:55.611 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T08:25:55.654 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T08:25:55.764 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T08:25:55.933 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 127/152 2024-04-21T08:25:55.965 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T08:25:55.989 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T08:25:55.989 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:25:55.989 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-21T08:25:55.989 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T08:25:55.990 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T08:25:55.990 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:25:56.040 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T08:25:56.052 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T08:25:56.154 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 130/152 2024-04-21T08:25:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 131/152 2024-04-21T08:25:56.264 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T08:25:56.264 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /sys 2024-04-21T08:25:56.264 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /proc 2024-04-21T08:25:56.264 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /mnt 2024-04-21T08:25:56.264 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /var/tmp 2024-04-21T08:25:56.264 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /home 2024-04-21T08:25:56.264 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /root 2024-04-21T08:25:56.264 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /tmp 2024-04-21T08:25:56.264 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-21T08:25:56.438 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T08:25:56.453 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T08:25:56.453 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:25:56.453 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-21T08:25:56.453 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-21T08:25:57.022 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T08:25:57.022 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /sys 2024-04-21T08:25:57.023 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /proc 2024-04-21T08:25:57.023 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /mnt 2024-04-21T08:25:57.023 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /var/tmp 2024-04-21T08:25:57.023 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /home 2024-04-21T08:25:57.023 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /root 2024-04-21T08:25:57.023 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /tmp 2024-04-21T08:25:57.023 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-21T08:25:57.181 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-21T08:25:57.208 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T08:25:57.210 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T08:25:57.224 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T08:25:57.224 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:25:57.224 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-21T08:25:57.224 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-21T08:25:57.233 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T08:25:57.234 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:25:57.234 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-21T08:25:57.234 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T08:25:57.234 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T08:25:57.234 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:25:57.277 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T08:25:57.300 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T08:25:57.300 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:25:57.300 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-21T08:25:57.300 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:25:57.361 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T08:25:57.385 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T08:25:57.385 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:25:57.385 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-21T08:25:57.385 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T08:25:57.385 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T08:25:57.385 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:25:57.704 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T08:25:57.727 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T08:25:57.727 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:25:57.728 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-21T08:25:57.728 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T08:25:57.728 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T08:25:57.728 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:25:57.932 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T08:25:57.958 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T08:25:57.958 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:25:57.958 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-21T08:25:57.958 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T08:25:57.958 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T08:25:57.958 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-21T08:25:58.174 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T08:25:58.196 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T08:25:58.196 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:25:58.197 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-21T08:25:58.197 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T08:25:58.197 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T08:25:58.197 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-21T08:25:58.641 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T08:25:58.664 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T08:25:58.664 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:25:58.664 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-21T08:25:58.664 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T08:25:58.664 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T08:25:58.664 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-21T08:25:58.695 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T08:25:58.721 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T08:25:58.721 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:25:58.721 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-21T08:25:58.721 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T08:25:58.721 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T08:25:58.721 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-21T08:25:58.772 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-21T08:25:58.829 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-21T08:25:58.950 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-21T08:25:58.962 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T08:25:58.985 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T08:25:58.985 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:25:58.985 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-21T08:25:58.985 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T08:25:58.985 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T08:25:58.986 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-21T08:25:59.071 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-21T08:25:59.117 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-21T08:25:59.196 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-21T08:25:59.284 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-21T08:25:59.425 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-21T08:25:59.438 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T08:25:59.459 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-21T08:25:59.460 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T08:25:59.460 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:25:59.460 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-21T08:25:59.460 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T08:25:59.460 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T08:25:59.461 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-21T08:25:59.490 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-21T08:25:59.518 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-21T08:25:59.568 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-21T08:25:59.626 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-21T08:25:59.682 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-21T08:25:59.874 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-21T08:25:59.922 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-21T08:25:59.953 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T08:25:59.983 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T08:26:00.003 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-21T08:26:00.015 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 122/152 2024-04-21T08:26:00.069 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 123/152 2024-04-21T08:26:00.091 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-21T08:26:00.098 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T08:26:00.233 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-21T08:26:00.272 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-21T08:26:00.304 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-21T08:26:00.340 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-21T08:26:00.718 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T08:26:00.756 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T08:26:00.796 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 122/152 2024-04-21T08:26:00.859 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 123/152 2024-04-21T08:26:00.887 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T08:26:01.145 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T08:26:01.183 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T08:26:01.938 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T08:26:01.972 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T08:26:02.153 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T08:26:02.183 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T08:26:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T08:26:02.447 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 127/152 2024-04-21T08:26:02.477 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T08:26:02.500 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T08:26:02.500 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:02.500 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-21T08:26:02.500 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T08:26:02.500 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T08:26:02.500 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-21T08:26:02.544 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T08:26:02.555 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T08:26:02.591 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 137/152 2024-04-21T08:26:02.633 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 138/152 2024-04-21T08:26:02.674 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 139/152 2024-04-21T08:26:02.678 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 130/152 2024-04-21T08:26:02.705 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-21T08:26:02.725 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 131/152 2024-04-21T08:26:02.732 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-21T08:26:02.927 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 142/152 2024-04-21T08:26:02.942 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T08:26:02.970 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-21T08:26:02.972 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T08:26:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 144/152 2024-04-21T08:26:03.077 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T08:26:03.252 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 127/152 2024-04-21T08:26:03.283 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T08:26:03.306 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T08:26:03.306 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:03.306 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-21T08:26:03.306 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T08:26:03.307 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T08:26:03.307 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-21T08:26:03.350 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T08:26:03.361 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T08:26:03.460 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 130/152 2024-04-21T08:26:03.516 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 131/152 2024-04-21T08:26:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:26:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:26:03.638 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-21T08:26:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:26:03.671 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T08:26:03.694 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T08:26:03.694 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:03.694 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-21T08:26:03.694 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T08:26:03.694 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T08:26:03.694 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-21T08:26:03.738 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T08:26:03.761 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T08:26:03.761 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:03.761 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-21T08:26:03.761 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-21T08:26:03.806 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T08:26:03.828 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T08:26:03.828 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:03.828 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-21T08:26:03.828 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T08:26:03.828 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T08:26:03.829 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-21T08:26:04.140 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T08:26:04.163 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T08:26:04.163 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:04.163 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-21T08:26:04.163 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T08:26:04.163 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T08:26:04.163 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-21T08:26:04.440 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-21T08:26:04.478 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T08:26:04.500 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 146/152 2024-04-21T08:26:04.501 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T08:26:04.501 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:04.501 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-21T08:26:04.501 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T08:26:04.501 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T08:26:04.501 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-21T08:26:04.545 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T08:26:04.567 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T08:26:04.567 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:04.567 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-21T08:26:04.567 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-21T08:26:04.586 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-21T08:26:04.613 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T08:26:04.634 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T08:26:04.635 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:04.635 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-21T08:26:04.635 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T08:26:04.635 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T08:26:04.635 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-21T08:26:04.653 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-21T08:26:04.737 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-21T08:26:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-21T08:26:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T08:26:04.849 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T08:26:04.849 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:26:04.947 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T08:26:04.970 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T08:26:04.970 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:04.970 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-21T08:26:04.970 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T08:26:04.970 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T08:26:04.970 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-21T08:26:08.868 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 137/152 2024-04-21T08:26:08.910 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 138/152 2024-04-21T08:26:08.951 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 139/152 2024-04-21T08:26:08.983 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-21T08:26:09.010 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-21T08:26:09.195 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 142/152 2024-04-21T08:26:09.239 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-21T08:26:09.268 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 144/152 2024-04-21T08:26:09.576 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 137/152 2024-04-21T08:26:09.625 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 138/152 2024-04-21T08:26:09.675 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 139/152 2024-04-21T08:26:09.707 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-21T08:26:09.734 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-21T08:26:09.916 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:26:09.925 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 142/152 2024-04-21T08:26:09.957 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:26:09.963 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-21T08:26:09.977 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:26:09.993 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 144/152 2024-04-21T08:26:10.628 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:26:10.628 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 1/152 2024-04-21T08:26:10.629 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2/152 2024-04-21T08:26:10.629 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 3/152 2024-04-21T08:26:10.629 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 4/152 2024-04-21T08:26:10.629 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 5/152 2024-04-21T08:26:10.629 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:26:10.629 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T08:26:10.629 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 8/152 2024-04-21T08:26:10.629 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 9/152 2024-04-21T08:26:10.629 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 10/152 2024-04-21T08:26:10.629 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 11/152 2024-04-21T08:26:10.629 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:26:10.629 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 13/152 2024-04-21T08:26:10.629 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 14/152 2024-04-21T08:26:10.629 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 15/152 2024-04-21T08:26:10.630 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 16/152 2024-04-21T08:26:10.630 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 17/152 2024-04-21T08:26:10.630 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 18/152 2024-04-21T08:26:10.630 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 19/152 2024-04-21T08:26:10.630 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 20/152 2024-04-21T08:26:10.630 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 21/152 2024-04-21T08:26:10.630 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 22/152 2024-04-21T08:26:10.630 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 23/152 2024-04-21T08:26:10.630 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 24/152 2024-04-21T08:26:10.630 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 25/152 2024-04-21T08:26:10.630 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 26/152 2024-04-21T08:26:10.630 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 27/152 2024-04-21T08:26:10.630 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 28/152 2024-04-21T08:26:10.630 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 29/152 2024-04-21T08:26:10.631 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:26:10.643 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 30/152 2024-04-21T08:26:10.643 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 31/152 2024-04-21T08:26:10.643 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 32/152 2024-04-21T08:26:10.643 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 33/152 2024-04-21T08:26:10.643 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 34/152 2024-04-21T08:26:10.643 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 35/152 2024-04-21T08:26:10.643 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 36/152 2024-04-21T08:26:10.643 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 37/152 2024-04-21T08:26:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 38/152 2024-04-21T08:26:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 39/152 2024-04-21T08:26:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-21T08:26:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-21T08:26:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-21T08:26:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-21T08:26:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-21T08:26:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-21T08:26:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-21T08:26:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-21T08:26:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-21T08:26:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-21T08:26:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-21T08:26:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-21T08:26:10.645 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-21T08:26:10.645 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-21T08:26:10.645 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-21T08:26:10.645 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-21T08:26:10.645 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-21T08:26:10.645 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-21T08:26:10.645 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-21T08:26:10.645 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-21T08:26:10.645 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-21T08:26:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-21T08:26:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-21T08:26:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-21T08:26:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-21T08:26:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-21T08:26:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-21T08:26:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-21T08:26:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-21T08:26:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-21T08:26:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-21T08:26:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-21T08:26:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-21T08:26:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-21T08:26:10.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-21T08:26:10.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-21T08:26:10.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-21T08:26:10.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-21T08:26:10.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-21T08:26:10.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-21T08:26:10.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-21T08:26:10.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-21T08:26:10.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-21T08:26:10.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-21T08:26:10.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-21T08:26:10.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-21T08:26:10.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-21T08:26:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-21T08:26:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-21T08:26:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-21T08:26:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-21T08:26:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-21T08:26:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-21T08:26:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-21T08:26:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-21T08:26:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-21T08:26:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-21T08:26:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-21T08:26:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-21T08:26:10.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-21T08:26:10.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-21T08:26:10.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-21T08:26:10.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-21T08:26:10.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-21T08:26:10.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-21T08:26:10.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-21T08:26:10.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-21T08:26:10.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-21T08:26:10.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-21T08:26:10.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-21T08:26:10.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-21T08:26:10.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-21T08:26:10.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-21T08:26:10.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-21T08:26:10.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-21T08:26:10.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-21T08:26:10.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-21T08:26:10.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-21T08:26:10.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-21T08:26:10.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-21T08:26:10.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-21T08:26:10.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-21T08:26:10.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-21T08:26:10.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-21T08:26:10.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-21T08:26:10.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-21T08:26:10.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-21T08:26:10.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-21T08:26:10.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-21T08:26:10.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-21T08:26:10.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-21T08:26:10.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-21T08:26:10.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-21T08:26:10.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-21T08:26:10.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-21T08:26:10.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-21T08:26:10.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-21T08:26:10.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-21T08:26:10.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-21T08:26:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-21T08:26:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-21T08:26:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-21T08:26:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-21T08:26:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-21T08:26:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-21T08:26:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-21T08:26:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-21T08:26:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-21T08:26:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-21T08:26:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 149/152 2024-04-21T08:26:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-21T08:26:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 151/152 2024-04-21T08:26:10.657 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:26:10.677 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:26:10.756 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 146/152 2024-04-21T08:26:10.861 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-21T08:26:10.902 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-21T08:26:10.984 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-21T08:26:11.015 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-21T08:26:11.015 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T08:26:11.098 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T08:26:11.099 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:26:11.479 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 146/152 2024-04-21T08:26:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-21T08:26:11.611 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-21T08:26:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-21T08:26:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-21T08:26:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T08:26:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T08:26:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:26:12.763 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:26:12.763 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:26:12.763 INFO:teuthology.orchestra.run.smithi043.stdout:Upgraded: 2024-04-21T08:26:12.763 INFO:teuthology.orchestra.run.smithi043.stdout: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.763 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.763 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-04-21T08:26:12.763 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-21T08:26:12.763 INFO:teuthology.orchestra.run.smithi043.stdout: automake-1.16.2-8.el9.noarch 2024-04-21T08:26:12.763 INFO:teuthology.orchestra.run.smithi043.stdout: bison-3.7.4-5.el9.x86_64 2024-04-21T08:26:12.763 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-21T08:26:12.763 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.764 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.764 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.764 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.764 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:12.764 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.764 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.764 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.764 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:12.764 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:12.764 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:12.764 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:12.764 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:12.764 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.764 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.765 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:12.765 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.765 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.765 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.765 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:12.765 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:12.765 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.765 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.765 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:12.765 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-21T08:26:12.765 INFO:teuthology.orchestra.run.smithi043.stdout: flex-2.6.4-9.el9.x86_64 2024-04-21T08:26:12.765 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-21T08:26:12.766 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-21T08:26:12.766 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-21T08:26:12.766 INFO:teuthology.orchestra.run.smithi043.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-21T08:26:12.766 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-21T08:26:12.766 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-21T08:26:12.766 INFO:teuthology.orchestra.run.smithi043.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-21T08:26:12.766 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-21T08:26:12.766 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-21T08:26:12.766 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.766 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.766 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.766 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-21T08:26:12.766 INFO:teuthology.orchestra.run.smithi043.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-21T08:26:12.767 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-21T08:26:12.767 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-21T08:26:12.767 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-21T08:26:12.767 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.767 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.767 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-21T08:26:12.767 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.767 INFO:teuthology.orchestra.run.smithi043.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-21T08:26:12.767 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-21T08:26:12.767 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-21T08:26:12.767 INFO:teuthology.orchestra.run.smithi043.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-21T08:26:12.767 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-21T08:26:12.767 INFO:teuthology.orchestra.run.smithi043.stdout: lua-5.4.4-4.el9.x86_64 2024-04-21T08:26:12.768 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-21T08:26:12.768 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-21T08:26:12.768 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-21T08:26:12.768 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-21T08:26:12.768 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-21T08:26:12.768 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-21T08:26:12.768 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-13.14-1.el9.x86_64 2024-04-21T08:26:12.768 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-21T08:26:12.768 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-21T08:26:12.768 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-21T08:26:12.768 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-21T08:26:12.768 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-21T08:26:12.768 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-21T08:26:12.768 INFO:teuthology.orchestra.run.smithi043.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-21T08:26:12.769 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-21T08:26:12.769 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-21T08:26:12.769 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-21T08:26:12.769 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.769 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.769 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.769 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-21T08:26:12.769 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-21T08:26:12.769 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-21T08:26:12.769 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-21T08:26:12.769 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-21T08:26:12.769 INFO:teuthology.orchestra.run.smithi043.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-21T08:26:12.769 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-21T08:26:12.770 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-21T08:26:12.770 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-21T08:26:12.770 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-21T08:26:12.770 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-21T08:26:12.770 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-21T08:26:12.770 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-21T08:26:12.770 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-21T08:26:12.770 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-21T08:26:12.770 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-21T08:26:12.770 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-21T08:26:12.770 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-21T08:26:12.770 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-21T08:26:12.770 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-21T08:26:12.770 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-21T08:26:12.771 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-21T08:26:12.771 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-21T08:26:12.771 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-21T08:26:12.771 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-21T08:26:12.771 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-21T08:26:12.771 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-21T08:26:12.771 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-21T08:26:12.771 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-21T08:26:12.771 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-21T08:26:12.771 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply-3.11-14.el9.noarch 2024-04-21T08:26:12.771 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-21T08:26:12.771 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-21T08:26:12.771 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-21T08:26:12.771 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-21T08:26:12.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-21T08:26:12.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-21T08:26:12.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-21T08:26:12.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-21T08:26:12.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-21T08:26:12.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-21T08:26:12.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-21T08:26:12.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-21T08:26:12.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-21T08:26:12.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-21T08:26:12.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-21T08:26:12.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-21T08:26:12.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-21T08:26:12.773 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-21T08:26:12.773 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-21T08:26:12.773 INFO:teuthology.orchestra.run.smithi043.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-21T08:26:12.773 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-21T08:26:12.773 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-21T08:26:12.773 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-21T08:26:12.773 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-21T08:26:12.773 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-21T08:26:12.773 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-21T08:26:12.773 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-21T08:26:12.773 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.773 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.773 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:12.773 INFO:teuthology.orchestra.run.smithi043.stdout: re2-1:20211101-3.el9.x86_64 2024-04-21T08:26:12.773 INFO:teuthology.orchestra.run.smithi043.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-21T08:26:12.773 INFO:teuthology.orchestra.run.smithi043.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-21T08:26:12.774 INFO:teuthology.orchestra.run.smithi043.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-21T08:26:12.774 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-21T08:26:12.774 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-21T08:26:12.774 INFO:teuthology.orchestra.run.smithi043.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-21T08:26:12.774 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:26:12.774 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-04-21T08:26:13.141 DEBUG:teuthology.parallel:result is None 2024-04-21T08:26:16.957 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:26:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 1/152 2024-04-21T08:26:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2/152 2024-04-21T08:26:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 3/152 2024-04-21T08:26:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 4/152 2024-04-21T08:26:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 5/152 2024-04-21T08:26:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:26:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T08:26:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 8/152 2024-04-21T08:26:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 9/152 2024-04-21T08:26:16.958 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 10/152 2024-04-21T08:26:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 11/152 2024-04-21T08:26:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:26:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 13/152 2024-04-21T08:26:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 14/152 2024-04-21T08:26:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 15/152 2024-04-21T08:26:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 16/152 2024-04-21T08:26:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 17/152 2024-04-21T08:26:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 18/152 2024-04-21T08:26:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 19/152 2024-04-21T08:26:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 20/152 2024-04-21T08:26:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 21/152 2024-04-21T08:26:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 22/152 2024-04-21T08:26:16.959 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 23/152 2024-04-21T08:26:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 24/152 2024-04-21T08:26:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 25/152 2024-04-21T08:26:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 26/152 2024-04-21T08:26:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 27/152 2024-04-21T08:26:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 28/152 2024-04-21T08:26:16.960 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 29/152 2024-04-21T08:26:16.971 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 30/152 2024-04-21T08:26:16.971 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 31/152 2024-04-21T08:26:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 32/152 2024-04-21T08:26:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 33/152 2024-04-21T08:26:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 34/152 2024-04-21T08:26:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 35/152 2024-04-21T08:26:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 36/152 2024-04-21T08:26:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 37/152 2024-04-21T08:26:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 38/152 2024-04-21T08:26:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 39/152 2024-04-21T08:26:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-21T08:26:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-21T08:26:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-21T08:26:16.972 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-21T08:26:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-21T08:26:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-21T08:26:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-21T08:26:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-21T08:26:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-21T08:26:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-21T08:26:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-21T08:26:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-21T08:26:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-21T08:26:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-21T08:26:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-21T08:26:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-21T08:26:16.973 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-21T08:26:16.974 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-21T08:26:16.974 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-21T08:26:16.974 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-21T08:26:16.974 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-21T08:26:16.974 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-21T08:26:16.974 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-21T08:26:16.974 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-21T08:26:16.974 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-21T08:26:16.974 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-21T08:26:16.974 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-21T08:26:16.974 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-21T08:26:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-21T08:26:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-21T08:26:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-21T08:26:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-21T08:26:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-21T08:26:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-21T08:26:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-21T08:26:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-21T08:26:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-21T08:26:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-21T08:26:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-21T08:26:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-21T08:26:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-21T08:26:16.975 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-21T08:26:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-21T08:26:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-21T08:26:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-21T08:26:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-21T08:26:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-21T08:26:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-21T08:26:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-21T08:26:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-21T08:26:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-21T08:26:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-21T08:26:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-21T08:26:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-21T08:26:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-21T08:26:16.976 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-21T08:26:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-21T08:26:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-21T08:26:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-21T08:26:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-21T08:26:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-21T08:26:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-21T08:26:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-21T08:26:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-21T08:26:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-21T08:26:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-21T08:26:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-21T08:26:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-21T08:26:16.977 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-21T08:26:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-21T08:26:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-21T08:26:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-21T08:26:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-21T08:26:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-21T08:26:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-21T08:26:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-21T08:26:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-21T08:26:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-21T08:26:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-21T08:26:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-21T08:26:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-21T08:26:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-21T08:26:16.978 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-21T08:26:16.979 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-21T08:26:16.979 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-21T08:26:16.979 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-21T08:26:16.979 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-21T08:26:16.979 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-21T08:26:16.979 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-21T08:26:16.979 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-21T08:26:16.979 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-21T08:26:16.979 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-21T08:26:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-21T08:26:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-21T08:26:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-21T08:26:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-21T08:26:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-21T08:26:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-21T08:26:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-21T08:26:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-21T08:26:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-21T08:26:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-21T08:26:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-21T08:26:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-21T08:26:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-21T08:26:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-21T08:26:16.980 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-21T08:26:16.981 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-21T08:26:16.981 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-21T08:26:16.981 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 149/152 2024-04-21T08:26:16.981 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-21T08:26:16.981 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 151/152 2024-04-21T08:26:17.616 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:26:17.616 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 1/152 2024-04-21T08:26:17.616 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2/152 2024-04-21T08:26:17.616 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 3/152 2024-04-21T08:26:17.616 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 4/152 2024-04-21T08:26:17.616 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 5/152 2024-04-21T08:26:17.616 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:26:17.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T08:26:17.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 8/152 2024-04-21T08:26:17.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 9/152 2024-04-21T08:26:17.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 10/152 2024-04-21T08:26:17.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 11/152 2024-04-21T08:26:17.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:26:17.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 13/152 2024-04-21T08:26:17.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 14/152 2024-04-21T08:26:17.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 15/152 2024-04-21T08:26:17.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 16/152 2024-04-21T08:26:17.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 17/152 2024-04-21T08:26:17.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 18/152 2024-04-21T08:26:17.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 19/152 2024-04-21T08:26:17.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 20/152 2024-04-21T08:26:17.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 21/152 2024-04-21T08:26:17.618 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 22/152 2024-04-21T08:26:17.618 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 23/152 2024-04-21T08:26:17.618 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 24/152 2024-04-21T08:26:17.618 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 25/152 2024-04-21T08:26:17.618 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 26/152 2024-04-21T08:26:17.618 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 27/152 2024-04-21T08:26:17.618 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 28/152 2024-04-21T08:26:17.630 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 29/152 2024-04-21T08:26:17.630 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 30/152 2024-04-21T08:26:17.630 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 31/152 2024-04-21T08:26:17.630 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 32/152 2024-04-21T08:26:17.630 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 33/152 2024-04-21T08:26:17.630 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 34/152 2024-04-21T08:26:17.630 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 35/152 2024-04-21T08:26:17.630 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 36/152 2024-04-21T08:26:17.630 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 37/152 2024-04-21T08:26:17.630 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 38/152 2024-04-21T08:26:17.631 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 39/152 2024-04-21T08:26:17.631 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-21T08:26:17.631 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-21T08:26:17.631 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-21T08:26:17.631 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-21T08:26:17.631 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-21T08:26:17.631 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-21T08:26:17.631 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-21T08:26:17.631 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-21T08:26:17.631 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-21T08:26:17.631 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-21T08:26:17.631 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-21T08:26:17.631 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-21T08:26:17.631 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-21T08:26:17.632 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-21T08:26:17.632 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-21T08:26:17.632 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-21T08:26:17.632 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-21T08:26:17.632 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-21T08:26:17.632 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-21T08:26:17.632 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-21T08:26:17.632 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-21T08:26:17.632 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-21T08:26:17.632 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-21T08:26:17.632 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-21T08:26:17.632 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-21T08:26:17.632 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-21T08:26:17.632 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-21T08:26:17.633 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-21T08:26:17.633 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-21T08:26:17.633 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-21T08:26:17.633 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-21T08:26:17.633 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-21T08:26:17.633 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-21T08:26:17.633 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-21T08:26:17.633 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-21T08:26:17.633 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-21T08:26:17.633 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-21T08:26:17.633 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-21T08:26:17.633 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-21T08:26:17.633 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-21T08:26:17.633 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-21T08:26:17.634 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-21T08:26:17.634 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-21T08:26:17.634 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-21T08:26:17.634 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-21T08:26:17.634 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-21T08:26:17.634 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-21T08:26:17.634 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-21T08:26:17.634 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-21T08:26:17.634 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-21T08:26:17.634 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-21T08:26:17.634 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-21T08:26:17.634 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-21T08:26:17.634 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-21T08:26:17.635 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-21T08:26:17.635 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-21T08:26:17.635 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-21T08:26:17.635 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-21T08:26:17.635 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-21T08:26:17.635 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-21T08:26:17.635 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-21T08:26:17.635 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-21T08:26:17.635 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-21T08:26:17.635 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-21T08:26:17.635 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-21T08:26:17.635 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-21T08:26:17.635 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-21T08:26:17.635 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-21T08:26:17.636 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-21T08:26:17.636 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-21T08:26:17.636 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-21T08:26:17.636 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-21T08:26:17.636 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-21T08:26:17.636 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-21T08:26:17.636 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-21T08:26:17.636 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-21T08:26:17.636 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-21T08:26:17.636 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-21T08:26:17.636 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-21T08:26:17.636 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-21T08:26:17.636 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-21T08:26:17.636 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-21T08:26:17.637 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-21T08:26:17.637 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-21T08:26:17.637 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-21T08:26:17.637 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-21T08:26:17.637 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-21T08:26:17.637 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-21T08:26:17.637 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-21T08:26:17.637 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-21T08:26:17.637 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-21T08:26:17.637 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-21T08:26:17.637 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-21T08:26:17.637 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-21T08:26:17.637 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-21T08:26:17.637 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-21T08:26:17.638 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-21T08:26:17.638 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-21T08:26:17.638 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-21T08:26:17.638 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-21T08:26:17.638 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-21T08:26:17.638 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-21T08:26:17.638 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-21T08:26:17.638 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-21T08:26:17.638 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-21T08:26:17.638 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-21T08:26:17.638 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-21T08:26:17.638 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-21T08:26:17.638 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-21T08:26:17.639 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 149/152 2024-04-21T08:26:17.639 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-21T08:26:17.639 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 151/152 2024-04-21T08:26:19.031 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:26:19.031 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-21T08:26:19.031 INFO:teuthology.orchestra.run.smithi175.stdout:Upgraded: 2024-04-21T08:26:19.031 INFO:teuthology.orchestra.run.smithi175.stdout: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.031 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.031 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-04-21T08:26:19.031 INFO:teuthology.orchestra.run.smithi175.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-21T08:26:19.031 INFO:teuthology.orchestra.run.smithi175.stdout: automake-1.16.2-8.el9.noarch 2024-04-21T08:26:19.031 INFO:teuthology.orchestra.run.smithi175.stdout: bison-3.7.4-5.el9.x86_64 2024-04-21T08:26:19.032 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-21T08:26:19.032 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.032 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.032 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.032 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.032 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:19.032 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.032 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.032 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.032 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:19.032 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:19.032 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:19.032 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:19.032 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:19.033 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.033 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.033 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:19.033 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.033 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.033 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.033 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:19.033 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:19.033 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.033 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.033 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:19.033 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-21T08:26:19.033 INFO:teuthology.orchestra.run.smithi175.stdout: flex-2.6.4-9.el9.x86_64 2024-04-21T08:26:19.033 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-21T08:26:19.034 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-21T08:26:19.034 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-21T08:26:19.034 INFO:teuthology.orchestra.run.smithi175.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-21T08:26:19.034 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-21T08:26:19.034 INFO:teuthology.orchestra.run.smithi175.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-21T08:26:19.034 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-21T08:26:19.034 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-21T08:26:19.034 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-21T08:26:19.034 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.034 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.034 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.034 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-21T08:26:19.035 INFO:teuthology.orchestra.run.smithi175.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-21T08:26:19.035 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-21T08:26:19.035 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-21T08:26:19.035 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-21T08:26:19.035 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.035 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.035 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-21T08:26:19.035 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.035 INFO:teuthology.orchestra.run.smithi175.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-21T08:26:19.035 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-21T08:26:19.035 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-21T08:26:19.035 INFO:teuthology.orchestra.run.smithi175.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-21T08:26:19.035 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-21T08:26:19.035 INFO:teuthology.orchestra.run.smithi175.stdout: lua-5.4.4-4.el9.x86_64 2024-04-21T08:26:19.036 INFO:teuthology.orchestra.run.smithi175.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-21T08:26:19.036 INFO:teuthology.orchestra.run.smithi175.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-21T08:26:19.036 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-21T08:26:19.036 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-21T08:26:19.036 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-21T08:26:19.036 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-21T08:26:19.036 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-13.14-1.el9.x86_64 2024-04-21T08:26:19.036 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-21T08:26:19.036 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-21T08:26:19.036 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-21T08:26:19.036 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-21T08:26:19.036 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-21T08:26:19.036 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-21T08:26:19.036 INFO:teuthology.orchestra.run.smithi175.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-21T08:26:19.037 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-21T08:26:19.037 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-21T08:26:19.037 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-21T08:26:19.037 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.037 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.037 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.037 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-21T08:26:19.037 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-21T08:26:19.037 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-21T08:26:19.037 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-21T08:26:19.037 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-21T08:26:19.037 INFO:teuthology.orchestra.run.smithi175.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-21T08:26:19.038 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-21T08:26:19.038 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-21T08:26:19.038 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-21T08:26:19.038 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-21T08:26:19.038 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-21T08:26:19.038 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-21T08:26:19.038 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-21T08:26:19.038 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-21T08:26:19.038 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-21T08:26:19.038 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-21T08:26:19.039 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-21T08:26:19.039 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-21T08:26:19.039 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-21T08:26:19.039 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-21T08:26:19.039 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-21T08:26:19.039 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-21T08:26:19.039 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-21T08:26:19.039 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-21T08:26:19.039 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-21T08:26:19.039 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-21T08:26:19.039 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-21T08:26:19.039 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-21T08:26:19.040 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-21T08:26:19.040 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-21T08:26:19.040 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply-3.11-14.el9.noarch 2024-04-21T08:26:19.040 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-21T08:26:19.040 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-21T08:26:19.040 INFO:teuthology.orchestra.run.smithi175.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-21T08:26:19.040 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-21T08:26:19.040 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-21T08:26:19.040 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-21T08:26:19.040 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-21T08:26:19.040 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-21T08:26:19.040 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-21T08:26:19.040 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.040 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.041 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-21T08:26:19.041 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-21T08:26:19.041 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-21T08:26:19.041 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.041 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-21T08:26:19.041 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-21T08:26:19.041 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-21T08:26:19.041 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-21T08:26:19.041 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-21T08:26:19.041 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-21T08:26:19.041 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-21T08:26:19.041 INFO:teuthology.orchestra.run.smithi175.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-21T08:26:19.041 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-21T08:26:19.041 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-21T08:26:19.042 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-21T08:26:19.042 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-21T08:26:19.042 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-21T08:26:19.042 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-21T08:26:19.042 INFO:teuthology.orchestra.run.smithi175.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-21T08:26:19.042 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.042 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.042 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.042 INFO:teuthology.orchestra.run.smithi175.stdout: re2-1:20211101-3.el9.x86_64 2024-04-21T08:26:19.042 INFO:teuthology.orchestra.run.smithi175.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-21T08:26:19.042 INFO:teuthology.orchestra.run.smithi175.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-21T08:26:19.042 INFO:teuthology.orchestra.run.smithi175.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-21T08:26:19.042 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-21T08:26:19.043 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-21T08:26:19.043 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-21T08:26:19.043 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-21T08:26:19.043 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-04-21T08:26:19.451 DEBUG:teuthology.parallel:result is None 2024-04-21T08:26:19.676 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:26:19.676 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-21T08:26:19.676 INFO:teuthology.orchestra.run.smithi112.stdout:Upgraded: 2024-04-21T08:26:19.676 INFO:teuthology.orchestra.run.smithi112.stdout: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.676 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.676 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-04-21T08:26:19.676 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-21T08:26:19.676 INFO:teuthology.orchestra.run.smithi112.stdout: automake-1.16.2-8.el9.noarch 2024-04-21T08:26:19.676 INFO:teuthology.orchestra.run.smithi112.stdout: bison-3.7.4-5.el9.x86_64 2024-04-21T08:26:19.676 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-21T08:26:19.677 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.677 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.677 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.677 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.677 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:19.677 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.677 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.677 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.677 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:19.677 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:19.677 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:19.677 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:19.677 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:19.677 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.678 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.678 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:19.678 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.678 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.678 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.678 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:19.678 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:19.678 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.678 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.678 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:19.678 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-21T08:26:19.678 INFO:teuthology.orchestra.run.smithi112.stdout: flex-2.6.4-9.el9.x86_64 2024-04-21T08:26:19.678 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-21T08:26:19.679 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-21T08:26:19.679 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-21T08:26:19.679 INFO:teuthology.orchestra.run.smithi112.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-21T08:26:19.679 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-21T08:26:19.679 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-21T08:26:19.679 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-21T08:26:19.679 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-21T08:26:19.679 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-21T08:26:19.679 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.679 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.679 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.679 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-21T08:26:19.679 INFO:teuthology.orchestra.run.smithi112.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-21T08:26:19.679 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-21T08:26:19.679 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-21T08:26:19.680 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-21T08:26:19.680 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.680 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.680 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-21T08:26:19.680 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.680 INFO:teuthology.orchestra.run.smithi112.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-21T08:26:19.680 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-21T08:26:19.680 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-21T08:26:19.680 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-21T08:26:19.680 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-21T08:26:19.680 INFO:teuthology.orchestra.run.smithi112.stdout: lua-5.4.4-4.el9.x86_64 2024-04-21T08:26:19.680 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-21T08:26:19.680 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-21T08:26:19.680 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-21T08:26:19.680 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-21T08:26:19.680 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-21T08:26:19.681 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-21T08:26:19.681 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-13.14-1.el9.x86_64 2024-04-21T08:26:19.681 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-21T08:26:19.681 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-21T08:26:19.681 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-21T08:26:19.681 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-21T08:26:19.681 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-21T08:26:19.681 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-21T08:26:19.681 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-21T08:26:19.681 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-21T08:26:19.681 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-21T08:26:19.681 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-21T08:26:19.681 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.681 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.681 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.682 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-21T08:26:19.682 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-21T08:26:19.682 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-21T08:26:19.682 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-21T08:26:19.682 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-21T08:26:19.682 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-21T08:26:19.682 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-21T08:26:19.682 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-21T08:26:19.682 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-21T08:26:19.682 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-21T08:26:19.682 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-21T08:26:19.682 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-21T08:26:19.682 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-21T08:26:19.682 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-21T08:26:19.683 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-21T08:26:19.683 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-21T08:26:19.683 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-21T08:26:19.683 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-21T08:26:19.683 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-21T08:26:19.683 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-21T08:26:19.683 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-21T08:26:19.683 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-21T08:26:19.683 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-21T08:26:19.683 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-21T08:26:19.683 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-21T08:26:19.683 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-21T08:26:19.683 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-21T08:26:19.683 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-21T08:26:19.684 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-21T08:26:19.684 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-21T08:26:19.684 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply-3.11-14.el9.noarch 2024-04-21T08:26:19.684 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-21T08:26:19.684 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-21T08:26:19.684 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-21T08:26:19.684 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-21T08:26:19.684 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-21T08:26:19.684 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-21T08:26:19.684 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-21T08:26:19.684 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-21T08:26:19.684 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-21T08:26:19.684 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.684 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.685 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-21T08:26:19.685 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-21T08:26:19.685 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-21T08:26:19.685 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.685 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-21T08:26:19.685 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-21T08:26:19.685 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-21T08:26:19.685 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-21T08:26:19.685 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-21T08:26:19.685 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-21T08:26:19.685 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-21T08:26:19.685 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-21T08:26:19.685 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-21T08:26:19.685 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-21T08:26:19.686 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-21T08:26:19.686 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-21T08:26:19.686 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-21T08:26:19.686 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-21T08:26:19.686 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-21T08:26:19.686 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.686 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.686 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:19.686 INFO:teuthology.orchestra.run.smithi112.stdout: re2-1:20211101-3.el9.x86_64 2024-04-21T08:26:19.686 INFO:teuthology.orchestra.run.smithi112.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-21T08:26:19.686 INFO:teuthology.orchestra.run.smithi112.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-21T08:26:19.686 INFO:teuthology.orchestra.run.smithi112.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-21T08:26:19.686 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-21T08:26:19.687 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-21T08:26:19.687 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-21T08:26:19.687 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-21T08:26:19.687 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-04-21T08:26:20.038 DEBUG:teuthology.parallel:result is None 2024-04-21T08:26:20.039 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:26:20.188 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-21T08:26:20.215 INFO:teuthology.orchestra.run.smithi043.stdout:19.0.0-3154.ga630765c.el9 2024-04-21T08:26:20.216 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3154.ga630765c.el9 2024-04-21T08:26:20.216 INFO:teuthology.task.install:The correct ceph version 19.0.0-3154.ga630765c is installed. 2024-04-21T08:26:20.218 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:26:20.356 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-21T08:26:20.378 INFO:teuthology.orchestra.run.smithi112.stdout:19.0.0-3154.ga630765c.el9 2024-04-21T08:26:20.379 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3154.ga630765c.el9 2024-04-21T08:26:20.379 INFO:teuthology.task.install:The correct ceph version 19.0.0-3154.ga630765c is installed. 2024-04-21T08:26:20.381 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:26:20.519 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-21T08:26:20.541 INFO:teuthology.orchestra.run.smithi175.stdout:19.0.0-3154.ga630765c.el9 2024-04-21T08:26:20.541 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3154.ga630765c.el9 2024-04-21T08:26:20.541 INFO:teuthology.task.install:The correct ceph version 19.0.0-3154.ga630765c is installed. 2024-04-21T08:26:20.543 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-21T08:26:20.543 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-21T08:26:20.543 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-21T08:26:20.575 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-21T08:26:20.575 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-21T08:26:20.601 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-21T08:26:20.601 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-21T08:26:20.629 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-21T08:26:20.629 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-21T08:26:20.630 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/daemon-helper 2024-04-21T08:26:20.659 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-21T08:26:20.726 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-21T08:26:20.726 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/daemon-helper 2024-04-21T08:26:20.752 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-21T08:26:20.815 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-21T08:26:20.815 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/daemon-helper 2024-04-21T08:26:20.838 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-21T08:26:20.904 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-21T08:26:20.904 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-21T08:26:20.905 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-21T08:26:20.933 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-21T08:26:21.002 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-21T08:26:21.002 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-21T08:26:21.025 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-21T08:26:21.089 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-21T08:26:21.089 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-21T08:26:21.115 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-21T08:26:21.178 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-21T08:26:21.178 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-21T08:26:21.179 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/stdin-killer 2024-04-21T08:26:21.208 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-21T08:26:21.277 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-21T08:26:21.277 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/stdin-killer 2024-04-21T08:26:21.300 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-21T08:26:21.363 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-21T08:26:21.363 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/stdin-killer 2024-04-21T08:26:21.386 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-21T08:26:21.449 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-21T08:26:21.569 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-21T08:26:21.569 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:26:21.570 INFO:tasks.cephadm:Cluster fsid is d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:26:21.570 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-21T08:26:21.570 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.43', 'mon.b': '172.21.15.112', 'mon.c': '172.21.15.175'} 2024-04-21T08:26:21.570 INFO:tasks.cephadm:First mon is mon.a on smithi043 2024-04-21T08:26:21.571 INFO:tasks.cephadm:First mgr is x 2024-04-21T08:26:21.571 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-21T08:26:21.571 DEBUG:teuthology.orchestra.run.smithi043:> sudo hostname $(hostname -s) 2024-04-21T08:26:21.602 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname $(hostname -s) 2024-04-21T08:26:21.627 DEBUG:teuthology.orchestra.run.smithi175:> sudo hostname $(hostname -s) 2024-04-21T08:26:21.650 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-21T08:26:21.650 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:26:21.866 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78740/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3154-ga630765c', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3154.ga630765c'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-19 11:13:53.331234', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-21T08:26:22.083 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 a630765c1f93c6e61d9b40c4b76437611d5ef29e from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:26:22.084 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/x86_64/flavors/default/cephadm 2024-04-21T08:26:22.084 DEBUG:teuthology.orchestra.run.smithi043:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-21T08:26:22.745 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r--. 1 ubuntu ubuntu 781532 Apr 21 08:26 /home/ubuntu/cephtest/cephadm 2024-04-21T08:26:22.745 DEBUG:teuthology.orchestra.run.smithi112:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-21T08:26:23.452 INFO:teuthology.orchestra.run.smithi112.stdout:-rw-r--r--. 1 ubuntu ubuntu 781532 Apr 21 08:26 /home/ubuntu/cephtest/cephadm 2024-04-21T08:26:23.452 DEBUG:teuthology.orchestra.run.smithi175:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-21T08:26:23.926 INFO:teuthology.orchestra.run.smithi175.stdout:-rw-r--r--. 1 ubuntu ubuntu 781532 Apr 21 08:26 /home/ubuntu/cephtest/cephadm 2024-04-21T08:26:23.926 DEBUG:teuthology.orchestra.run.smithi043:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-21T08:26:23.946 DEBUG:teuthology.orchestra.run.smithi112:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-21T08:26:23.966 DEBUG:teuthology.orchestra.run.smithi175:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-21T08:26:24.001 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e on all hosts... 2024-04-21T08:26:24.001 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e pull 2024-04-21T08:26:24.004 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e pull 2024-04-21T08:26:24.008 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e pull 2024-04-21T08:26:25.478 INFO:teuthology.orchestra.run.smithi112.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T08:26:25.592 INFO:teuthology.orchestra.run.smithi043.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T08:26:25.659 INFO:teuthology.orchestra.run.smithi175.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T08:26:56.898 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-21T08:26:56.898 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev)", 2024-04-21T08:26:56.898 INFO:teuthology.orchestra.run.smithi112.stdout: "image_id": "b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192", 2024-04-21T08:26:56.898 INFO:teuthology.orchestra.run.smithi112.stdout: "repo_digests": [ 2024-04-21T08:26:56.898 INFO:teuthology.orchestra.run.smithi112.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda" 2024-04-21T08:26:56.899 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-21T08:26:56.899 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-21T08:27:25.440 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-04-21T08:27:25.440 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev)", 2024-04-21T08:27:25.440 INFO:teuthology.orchestra.run.smithi175.stdout: "image_id": "b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192", 2024-04-21T08:27:25.440 INFO:teuthology.orchestra.run.smithi175.stdout: "repo_digests": [ 2024-04-21T08:27:25.440 INFO:teuthology.orchestra.run.smithi175.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda" 2024-04-21T08:27:25.440 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-04-21T08:27:25.440 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-04-21T08:28:02.916 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:28:02.916 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev)", 2024-04-21T08:28:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: "image_id": "b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192", 2024-04-21T08:28:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: "repo_digests": [ 2024-04-21T08:28:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda" 2024-04-21T08:28:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-21T08:28:02.917 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:28:02.943 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /etc/ceph 2024-04-21T08:28:02.988 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph 2024-04-21T08:28:03.020 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph 2024-04-21T08:28:03.053 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 777 /etc/ceph 2024-04-21T08:28:03.079 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 777 /etc/ceph 2024-04-21T08:28:03.106 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 777 /etc/ceph 2024-04-21T08:28:03.133 INFO:tasks.cephadm:Writing seed config... 2024-04-21T08:28:03.135 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-21T08:28:03.135 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-21T08:28:03.135 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-21T08:28:03.135 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-21T08:28:03.135 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-21T08:28:03.135 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-21T08:28:03.135 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-21T08:28:03.135 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-21T08:28:03.135 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-21T08:28:03.135 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-21T08:28:03.135 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-21T08:28:03.135 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-21T08:28:03.135 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-21T08:28:03.136 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-21T08:28:03.136 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-21T08:28:03.136 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-21T08:28:03.136 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-21T08:28:03.136 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-21T08:28:03.136 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-21T08:28:03.136 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-21T08:28:03.136 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-21T08:28:03.136 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-21T08:28:03.136 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-21T08:28:03.136 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-21T08:28:03.136 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-21T08:28:03.136 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-21T08:28:03.136 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-21T08:28:03.136 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-21T08:28:03.137 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-21T08:28:03.137 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-21T08:28:03.137 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-21T08:28:03.137 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-21T08:28:03.137 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-21T08:28:03.137 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-21T08:28:03.137 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-21T08:28:03.137 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-21T08:28:03.137 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-21T08:28:03.137 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-21T08:28:03.137 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-21T08:28:03.137 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-21T08:28:03.137 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-21T08:28:03.137 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-21T08:28:03.137 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-21T08:28:03.138 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-21T08:28:03.138 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-21T08:28:03.138 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-21T08:28:03.139 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-21T08:28:03.139 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-21T08:28:03.155 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = d57a7d98-ffb8-11ee-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-21T08:28:03.156 DEBUG:teuthology.orchestra.run.smithi043:mon.a> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mon.a.service 2024-04-21T08:28:03.199 DEBUG:teuthology.orchestra.run.smithi043:mgr.x> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mgr.x.service 2024-04-21T08:28:03.242 INFO:tasks.cephadm:Bootstrapping... 2024-04-21T08:28:03.242 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e -v bootstrap --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.43 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:28:03.441 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-04-21T08:28:03.441 INFO:teuthology.orchestra.run.smithi043.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e', '-v', 'bootstrap', '--fsid', 'd57a7d98-ffb8-11ee-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.43', '--skip-admin-label'] 2024-04-21T08:28:03.441 INFO:teuthology.orchestra.run.smithi043.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-21T08:28:03.442 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying podman|docker is present... 2024-04-21T08:28:03.483 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-21T08:28:03.483 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying lvm2 is present... 2024-04-21T08:28:03.483 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying time synchronization is in place... 2024-04-21T08:28:03.490 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-21T08:28:03.491 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-21T08:28:03.496 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-21T08:28:03.496 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-04-21T08:28:03.503 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout enabled 2024-04-21T08:28:03.509 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout active 2024-04-21T08:28:03.509 INFO:teuthology.orchestra.run.smithi043.stdout:Unit chronyd.service is enabled and running 2024-04-21T08:28:03.509 INFO:teuthology.orchestra.run.smithi043.stdout:Repeating the final host check... 2024-04-21T08:28:03.534 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-21T08:28:03.534 INFO:teuthology.orchestra.run.smithi043.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-21T08:28:03.534 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl is present 2024-04-21T08:28:03.534 INFO:teuthology.orchestra.run.smithi043.stdout:lvcreate is present 2024-04-21T08:28:03.540 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-21T08:28:03.540 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-21T08:28:03.546 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-21T08:28:03.546 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-04-21T08:28:03.553 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout enabled 2024-04-21T08:28:03.559 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout active 2024-04-21T08:28:03.559 INFO:teuthology.orchestra.run.smithi043.stdout:Unit chronyd.service is enabled and running 2024-04-21T08:28:03.560 INFO:teuthology.orchestra.run.smithi043.stdout:Host looks OK 2024-04-21T08:28:03.560 INFO:teuthology.orchestra.run.smithi043.stdout:Cluster fsid: d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:28:03.560 INFO:teuthology.orchestra.run.smithi043.stdout:Acquiring lock 139778477587616 on /run/cephadm/d57a7d98-ffb8-11ee-bc93-c7b262605968.lock 2024-04-21T08:28:03.560 INFO:teuthology.orchestra.run.smithi043.stdout:Lock 139778477587616 acquired on /run/cephadm/d57a7d98-ffb8-11ee-bc93-c7b262605968.lock 2024-04-21T08:28:03.560 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying IP 172.21.15.43 port 3300 ... 2024-04-21T08:28:03.560 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying IP 172.21.15.43 port 6789 ... 2024-04-21T08:28:03.560 INFO:teuthology.orchestra.run.smithi043.stdout:Base mon IP(s) is [172.21.15.43:3300, 172.21.15.43:6789], mon addrv is [v2:172.21.15.43:3300,v1:172.21.15.43:6789] 2024-04-21T08:28:03.564 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.43 metric 100 2024-04-21T08:28:03.564 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.43 metric 100 2024-04-21T08:28:03.566 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-21T08:28:03.566 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-21T08:28:03.566 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-21T08:28:03.569 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-21T08:28:03.569 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-21T08:28:03.569 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-21T08:28:03.569 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-21T08:28:03.569 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-04-21T08:28:03.569 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-21T08:28:03.570 INFO:teuthology.orchestra.run.smithi043.stdout:Mon IP `172.21.15.43` is in CIDR network `172.21.0.0/20` 2024-04-21T08:28:03.570 INFO:teuthology.orchestra.run.smithi043.stdout:Mon IP `172.21.15.43` is in CIDR network `172.21.0.0/20` 2024-04-21T08:28:03.570 INFO:teuthology.orchestra.run.smithi043.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-21T08:28:03.570 INFO:teuthology.orchestra.run.smithi043.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-21T08:28:03.571 INFO:teuthology.orchestra.run.smithi043.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T08:28:04.093 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 2024-04-21T08:28:04.093 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T08:28:04.093 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Getting image source signatures 2024-04-21T08:28:04.093 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying blob sha256:7eab8a2da10d06ac88aea1fd8998a2559b4de9d07c585e3b1e6ee6c4d35fc119 2024-04-21T08:28:04.093 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-21T08:28:04.093 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying config sha256:b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 2024-04-21T08:28:04.094 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-21T08:28:05.031 INFO:teuthology.orchestra.run.smithi043.stdout:ceph: stdout ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev) 2024-04-21T08:28:05.031 INFO:teuthology.orchestra.run.smithi043.stdout:Ceph version: ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev) 2024-04-21T08:28:05.031 INFO:teuthology.orchestra.run.smithi043.stdout:Extracting ceph user uid/gid from container image... 2024-04-21T08:28:05.877 INFO:teuthology.orchestra.run.smithi043.stdout:stat: stdout 167 167 2024-04-21T08:28:05.877 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial keys... 2024-04-21T08:28:06.829 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQAWziRmt6xKDBAA1YvOanOP2+oGlSmIdaEspA== 2024-04-21T08:28:07.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQAXziRm4JP8CBAAr5DjGM3WR3fvFEjleCYN7w== 2024-04-21T08:28:08.667 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQAYziRmmYvYBhAAZZAbGsNr7mEdF9tdFLB3Hg== 2024-04-21T08:28:08.668 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial monmap... 2024-04-21T08:28:09.565 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-21T08:28:09.566 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-21T08:28:09.566 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:28:09.566 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-21T08:28:09.566 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool for a [v2:172.21.15.43:3300,v1:172.21.15.43:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-21T08:28:09.566 INFO:teuthology.orchestra.run.smithi043.stdout:setting min_mon_release = quincy 2024-04-21T08:28:09.566 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: set fsid to d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:28:09.566 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-21T08:28:09.566 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:28:09.566 INFO:teuthology.orchestra.run.smithi043.stdout:Creating mon... 2024-04-21T08:28:10.547 INFO:teuthology.orchestra.run.smithi043.stdout:create mon.a on 2024-04-21T08:28:10.764 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-21T08:28:10.907 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-21T08:28:11.079 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968.target → /etc/systemd/system/ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968.target. 2024-04-21T08:28:11.079 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968.target → /etc/systemd/system/ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968.target. 2024-04-21T08:28:11.313 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mon.a 2024-04-21T08:28:11.313 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mon.a.service: Unit ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mon.a.service not loaded. 2024-04-21T08:28:11.478 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968.target.wants/ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@.service. 2024-04-21T08:28:12.040 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-21T08:28:12.040 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-04-21T08:28:12.047 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-21T08:28:12.047 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-04-21T08:28:12.047 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-04-21T08:28:12.048 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to enable service . firewalld.service is not available 2024-04-21T08:28:12.048 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mon to start... 2024-04-21T08:28:12.048 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mon... 2024-04-21T08:28:13.157 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout cluster: 2024-04-21T08:28:13.157 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout id: d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:28:13.158 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-21T08:28:13.158 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:13.158 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout services: 2024-04-21T08:28:13.158 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.610007s) 2024-04-21T08:28:13.158 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-21T08:28:13.158 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-21T08:28:13.158 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:13.158 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout data: 2024-04-21T08:28:13.158 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-21T08:28:13.158 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-21T08:28:13.158 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-21T08:28:13.158 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout pgs: 2024-04-21T08:28:13.158 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:13.159 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.501+0000 7f0ef7e14700 1 Processor -- start 2024-04-21T08:28:13.159 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.502+0000 7f0ef7e14700 1 -- start start 2024-04-21T08:28:13.159 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.502+0000 7f0ef7e14700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ef0108360 0x7f0ef0108780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:13.159 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.502+0000 7f0ef7e14700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef0108d50 con 0x7f0ef0108360 2024-04-21T08:28:13.159 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.502+0000 7f0ef5bb0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ef0108360 0x7f0ef0108780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:13.159 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.503+0000 7f0ef5bb0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ef0108360 0x7f0ef0108780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54202/0 (socket says 172.21.15.43:54202) 2024-04-21T08:28:13.159 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.503+0000 7f0ef5bb0700 1 -- 172.21.15.43:0/3827373432 learned_addr learned my addr 172.21.15.43:0/3827373432 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:13.159 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.503+0000 7f0ef5bb0700 1 -- 172.21.15.43:0/3827373432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef0109560 con 0x7f0ef0108360 2024-04-21T08:28:13.159 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.503+0000 7f0ef5bb0700 1 --2- 172.21.15.43:0/3827373432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ef0108360 0x7f0ef0108780 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0eec0098a0 tx=0x7f0eec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555dbecca5bd2ac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:13.159 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.503+0000 7f0ef4bae700 1 -- 172.21.15.43:0/3827373432 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0eec004160 con 0x7f0ef0108360 2024-04-21T08:28:13.159 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.503+0000 7f0ef4bae700 1 -- 172.21.15.43:0/3827373432 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eec004300 con 0x7f0ef0108360 2024-04-21T08:28:13.159 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.503+0000 7f0ef4bae700 1 -- 172.21.15.43:0/3827373432 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0eec0036a0 con 0x7f0ef0108360 2024-04-21T08:28:13.159 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.504+0000 7f0ef7e14700 1 -- 172.21.15.43:0/3827373432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ef0108360 msgr2=0x7f0ef0108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:13.159 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.504+0000 7f0ef7e14700 1 --2- 172.21.15.43:0/3827373432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ef0108360 0x7f0ef0108780 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0eec0098a0 tx=0x7f0eec009bb0 comp rx=0 tx=0).stop 2024-04-21T08:28:13.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.504+0000 7f0ef7e14700 1 -- 172.21.15.43:0/3827373432 shutdown_connections 2024-04-21T08:28:13.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.504+0000 7f0ef7e14700 1 --2- 172.21.15.43:0/3827373432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ef0108360 0x7f0ef0108780 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:13.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.504+0000 7f0ef7e14700 1 -- 172.21.15.43:0/3827373432 >> 172.21.15.43:0/3827373432 conn(0x7f0ef0103920 msgr2=0x7f0ef0105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:13.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.504+0000 7f0ef7e14700 1 -- 172.21.15.43:0/3827373432 shutdown_connections 2024-04-21T08:28:13.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.504+0000 7f0ef7e14700 1 -- 172.21.15.43:0/3827373432 wait complete. 2024-04-21T08:28:13.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.505+0000 7f0ef7e14700 1 Processor -- start 2024-04-21T08:28:13.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.505+0000 7f0ef7e14700 1 -- start start 2024-04-21T08:28:13.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.505+0000 7f0ef7e14700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ef0108360 0x7f0ef007db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:13.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.505+0000 7f0ef7e14700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef0109a90 con 0x7f0ef0108360 2024-04-21T08:28:13.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.505+0000 7f0ef5bb0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ef0108360 0x7f0ef007db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:13.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.505+0000 7f0ef5bb0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ef0108360 0x7f0ef007db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54218/0 (socket says 172.21.15.43:54218) 2024-04-21T08:28:13.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.505+0000 7f0ef5bb0700 1 -- 172.21.15.43:0/4204414024 learned_addr learned my addr 172.21.15.43:0/4204414024 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:13.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.506+0000 7f0ef5bb0700 1 -- 172.21.15.43:0/4204414024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef007e0a0 con 0x7f0ef0108360 2024-04-21T08:28:13.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.506+0000 7f0ef5bb0700 1 --2- 172.21.15.43:0/4204414024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ef0108360 0x7f0ef007db60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0108ff0 tx=0x7f0eec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:13.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.506+0000 7f0ee6ffd700 1 -- 172.21.15.43:0/4204414024 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0eec003db0 con 0x7f0ef0108360 2024-04-21T08:28:13.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.506+0000 7f0ee6ffd700 1 -- 172.21.15.43:0/4204414024 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eec011630 con 0x7f0ef0108360 2024-04-21T08:28:13.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.506+0000 7f0ef7e14700 1 -- 172.21.15.43:0/4204414024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef007c290 con 0x7f0ef0108360 2024-04-21T08:28:13.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.506+0000 7f0ee6ffd700 1 -- 172.21.15.43:0/4204414024 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0eec011930 con 0x7f0ef0108360 2024-04-21T08:28:13.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.506+0000 7f0ef7e14700 1 -- 172.21.15.43:0/4204414024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef007c7d0 con 0x7f0ef0108360 2024-04-21T08:28:13.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.506+0000 7f0ee6ffd700 1 -- 172.21.15.43:0/4204414024 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0eec011b10 con 0x7f0ef0108360 2024-04-21T08:28:13.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.506+0000 7f0ee6ffd700 1 -- 172.21.15.43:0/4204414024 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0eec01b790 con 0x7f0ef0108360 2024-04-21T08:28:13.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.507+0000 7f0ef7e14700 1 -- 172.21.15.43:0/4204414024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef00640a0 con 0x7f0ef0108360 2024-04-21T08:28:13.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.509+0000 7f0ee6ffd700 1 -- 172.21.15.43:0/4204414024 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0eec011e00 con 0x7f0ef0108360 2024-04-21T08:28:13.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.584+0000 7f0ef7e14700 1 -- 172.21.15.43:0/4204414024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f0ef0194f60 con 0x7f0ef0108360 2024-04-21T08:28:13.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.585+0000 7f0ee6ffd700 1 -- 172.21.15.43:0/4204414024 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f0eec004260 con 0x7f0ef0108360 2024-04-21T08:28:13.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.586+0000 7f0ef7e14700 1 -- 172.21.15.43:0/4204414024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ef0108360 msgr2=0x7f0ef007db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:13.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.586+0000 7f0ef7e14700 1 --2- 172.21.15.43:0/4204414024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ef0108360 0x7f0ef007db60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0108ff0 tx=0x7f0eec004750 comp rx=0 tx=0).stop 2024-04-21T08:28:13.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.586+0000 7f0ef7e14700 1 -- 172.21.15.43:0/4204414024 shutdown_connections 2024-04-21T08:28:13.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.586+0000 7f0ef7e14700 1 --2- 172.21.15.43:0/4204414024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ef0108360 0x7f0ef007db60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:13.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.586+0000 7f0ef7e14700 1 -- 172.21.15.43:0/4204414024 >> 172.21.15.43:0/4204414024 conn(0x7f0ef0103920 msgr2=0x7f0ef01926c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:13.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.586+0000 7f0ef7e14700 1 -- 172.21.15.43:0/4204414024 shutdown_connections 2024-04-21T08:28:13.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:12.586+0000 7f0ef7e14700 1 -- 172.21.15.43:0/4204414024 wait complete. 2024-04-21T08:28:13.162 INFO:teuthology.orchestra.run.smithi043.stdout:mon is available 2024-04-21T08:28:13.162 INFO:teuthology.orchestra.run.smithi043.stdout:Assimilating anything we can from ceph.conf... 2024-04-21T08:28:14.195 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:14.195 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [global] 2024-04-21T08:28:14.195 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout fsid = d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:28:14.195 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-21T08:28:14.195 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.43:3300,v1:172.21.15.43:6789] 2024-04-21T08:28:14.195 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-21T08:28:14.195 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-21T08:28:14.195 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-21T08:28:14.195 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-21T08:28:14.195 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:14.196 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-21T08:28:14.196 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-21T08:28:14.196 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:14.196 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [osd] 2024-04-21T08:28:14.196 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-21T08:28:14.196 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-21T08:28:14.196 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-21T08:28:14.196 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-21T08:28:14.196 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-21T08:28:14.196 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.574+0000 7f4e3ac90700 1 Processor -- start 2024-04-21T08:28:14.196 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.574+0000 7f4e3ac90700 1 -- start start 2024-04-21T08:28:14.196 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.574+0000 7f4e3ac90700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e341072a0 0x7f4e341076c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:14.196 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.575+0000 7f4e3ac90700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4e34107c90 con 0x7f4e341072a0 2024-04-21T08:28:14.196 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.575+0000 7f4e38a2c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e341072a0 0x7f4e341076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:14.197 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.575+0000 7f4e38a2c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e341072a0 0x7f4e341076c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54234/0 (socket says 172.21.15.43:54234) 2024-04-21T08:28:14.197 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.575+0000 7f4e38a2c700 1 -- 172.21.15.43:0/2040668795 learned_addr learned my addr 172.21.15.43:0/2040668795 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:14.197 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.575+0000 7f4e38a2c700 1 -- 172.21.15.43:0/2040668795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e341084a0 con 0x7f4e341072a0 2024-04-21T08:28:14.197 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.575+0000 7f4e38a2c700 1 --2- 172.21.15.43:0/2040668795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e341072a0 0x7f4e341076c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4e280098a0 tx=0x7f4e28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ffb16ab1b2f6711 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:14.197 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.575+0000 7f4e337fe700 1 -- 172.21.15.43:0/2040668795 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e28004160 con 0x7f4e341072a0 2024-04-21T08:28:14.197 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.575+0000 7f4e337fe700 1 -- 172.21.15.43:0/2040668795 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e28004300 con 0x7f4e341072a0 2024-04-21T08:28:14.197 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.575+0000 7f4e337fe700 1 -- 172.21.15.43:0/2040668795 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e280036a0 con 0x7f4e341072a0 2024-04-21T08:28:14.197 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.576+0000 7f4e3ac90700 1 -- 172.21.15.43:0/2040668795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e341072a0 msgr2=0x7f4e341076c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:14.197 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.576+0000 7f4e3ac90700 1 --2- 172.21.15.43:0/2040668795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e341072a0 0x7f4e341076c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4e280098a0 tx=0x7f4e28009bb0 comp rx=0 tx=0).stop 2024-04-21T08:28:14.197 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.576+0000 7f4e3ac90700 1 -- 172.21.15.43:0/2040668795 shutdown_connections 2024-04-21T08:28:14.197 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.576+0000 7f4e3ac90700 1 --2- 172.21.15.43:0/2040668795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e341072a0 0x7f4e341076c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:14.197 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.576+0000 7f4e3ac90700 1 -- 172.21.15.43:0/2040668795 >> 172.21.15.43:0/2040668795 conn(0x7f4e341028e0 msgr2=0x7f4e34104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:14.197 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.576+0000 7f4e3ac90700 1 -- 172.21.15.43:0/2040668795 shutdown_connections 2024-04-21T08:28:14.198 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.576+0000 7f4e3ac90700 1 -- 172.21.15.43:0/2040668795 wait complete. 2024-04-21T08:28:14.198 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.577+0000 7f4e3ac90700 1 Processor -- start 2024-04-21T08:28:14.198 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.577+0000 7f4e3ac90700 1 -- start start 2024-04-21T08:28:14.198 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.577+0000 7f4e3ac90700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e341072a0 0x7f4e3419b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:14.198 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.577+0000 7f4e3ac90700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4e341089d0 con 0x7f4e341072a0 2024-04-21T08:28:14.198 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.577+0000 7f4e38a2c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e341072a0 0x7f4e3419b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:14.198 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.577+0000 7f4e38a2c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e341072a0 0x7f4e3419b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54236/0 (socket says 172.21.15.43:54236) 2024-04-21T08:28:14.198 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.577+0000 7f4e38a2c700 1 -- 172.21.15.43:0/3925846143 learned_addr learned my addr 172.21.15.43:0/3925846143 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:14.198 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.577+0000 7f4e38a2c700 1 -- 172.21.15.43:0/3925846143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3419bf00 con 0x7f4e341072a0 2024-04-21T08:28:14.198 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.577+0000 7f4e38a2c700 1 --2- 172.21.15.43:0/3925846143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e341072a0 0x7f4e3419b9c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4e28000c00 tx=0x7f4e28004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:14.198 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.577+0000 7f4e31ffb700 1 -- 172.21.15.43:0/3925846143 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e28003db0 con 0x7f4e341072a0 2024-04-21T08:28:14.198 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.577+0000 7f4e31ffb700 1 -- 172.21.15.43:0/3925846143 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e28011630 con 0x7f4e341072a0 2024-04-21T08:28:14.198 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.578+0000 7f4e31ffb700 1 -- 172.21.15.43:0/3925846143 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e28011950 con 0x7f4e341072a0 2024-04-21T08:28:14.198 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.578+0000 7f4e3ac90700 1 -- 172.21.15.43:0/3925846143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e3419c190 con 0x7f4e341072a0 2024-04-21T08:28:14.199 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.578+0000 7f4e3ac90700 1 -- 172.21.15.43:0/3925846143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e3419ee80 con 0x7f4e341072a0 2024-04-21T08:28:14.199 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.578+0000 7f4e31ffb700 1 -- 172.21.15.43:0/3925846143 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4e28011af0 con 0x7f4e341072a0 2024-04-21T08:28:14.199 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.578+0000 7f4e31ffb700 1 -- 172.21.15.43:0/3925846143 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4e2801c450 con 0x7f4e341072a0 2024-04-21T08:28:14.199 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.579+0000 7f4e3ac90700 1 -- 172.21.15.43:0/3925846143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e340640a0 con 0x7f4e341072a0 2024-04-21T08:28:14.199 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.581+0000 7f4e31ffb700 1 -- 172.21.15.43:0/3925846143 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4e28032020 con 0x7f4e341072a0 2024-04-21T08:28:14.199 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.652+0000 7f4e3ac90700 1 -- 172.21.15.43:0/3925846143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f4e3402e100 con 0x7f4e341072a0 2024-04-21T08:28:14.199 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.655+0000 7f4e31ffb700 1 -- 172.21.15.43:0/3925846143 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f4e28004260 con 0x7f4e341072a0 2024-04-21T08:28:14.199 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.656+0000 7f4e31ffb700 1 -- 172.21.15.43:0/3925846143 <== mon.0 v2:172.21.15.43:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4e2801c6a0 con 0x7f4e341072a0 2024-04-21T08:28:14.199 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.657+0000 7f4e3ac90700 1 -- 172.21.15.43:0/3925846143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e341072a0 msgr2=0x7f4e3419b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:14.199 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.657+0000 7f4e3ac90700 1 --2- 172.21.15.43:0/3925846143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e341072a0 0x7f4e3419b9c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4e28000c00 tx=0x7f4e28004750 comp rx=0 tx=0).stop 2024-04-21T08:28:14.199 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.657+0000 7f4e3ac90700 1 -- 172.21.15.43:0/3925846143 shutdown_connections 2024-04-21T08:28:14.199 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.657+0000 7f4e3ac90700 1 --2- 172.21.15.43:0/3925846143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e341072a0 0x7f4e3419b9c0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:14.199 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.657+0000 7f4e3ac90700 1 -- 172.21.15.43:0/3925846143 >> 172.21.15.43:0/3925846143 conn(0x7f4e341028e0 msgr2=0x7f4e34192660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:14.199 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.657+0000 7f4e3ac90700 1 -- 172.21.15.43:0/3925846143 shutdown_connections 2024-04-21T08:28:14.200 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:13.657+0000 7f4e3ac90700 1 -- 172.21.15.43:0/3925846143 wait complete. 2024-04-21T08:28:14.200 INFO:teuthology.orchestra.run.smithi043.stdout:Generating new minimal ceph.conf... 2024-04-21T08:28:15.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.613+0000 7fa5edfeb700 1 Processor -- start 2024-04-21T08:28:15.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.613+0000 7fa5edfeb700 1 -- start start 2024-04-21T08:28:15.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.613+0000 7fa5edfeb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5e8108e50 0x7fa5e810b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:15.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.613+0000 7fa5edfeb700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e807be20 con 0x7fa5e8108e50 2024-04-21T08:28:15.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.614+0000 7fa5e77fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5e8108e50 0x7fa5e810b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:15.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.614+0000 7fa5e77fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5e8108e50 0x7fa5e810b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54240/0 (socket says 172.21.15.43:54240) 2024-04-21T08:28:15.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.614+0000 7fa5e77fe700 1 -- 172.21.15.43:0/118720620 learned_addr learned my addr 172.21.15.43:0/118720620 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:15.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.614+0000 7fa5e77fe700 1 -- 172.21.15.43:0/118720620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e810b780 con 0x7fa5e8108e50 2024-04-21T08:28:15.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.614+0000 7fa5e77fe700 1 --2- 172.21.15.43:0/118720620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5e8108e50 0x7fa5e810b240 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa5d80098a0 tx=0x7fa5d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39d11f12b993e46 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:15.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.614+0000 7fa5e67fc700 1 -- 172.21.15.43:0/118720620 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5d8004160 con 0x7fa5e8108e50 2024-04-21T08:28:15.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.614+0000 7fa5e67fc700 1 -- 172.21.15.43:0/118720620 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa5d8004300 con 0x7fa5e8108e50 2024-04-21T08:28:15.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.614+0000 7fa5e67fc700 1 -- 172.21.15.43:0/118720620 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5d8011af0 con 0x7fa5e8108e50 2024-04-21T08:28:15.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.615+0000 7fa5edfeb700 1 -- 172.21.15.43:0/118720620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5e8108e50 msgr2=0x7fa5e810b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:15.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.615+0000 7fa5edfeb700 1 --2- 172.21.15.43:0/118720620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5e8108e50 0x7fa5e810b240 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa5d80098a0 tx=0x7fa5d8009bb0 comp rx=0 tx=0).stop 2024-04-21T08:28:15.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.615+0000 7fa5edfeb700 1 -- 172.21.15.43:0/118720620 shutdown_connections 2024-04-21T08:28:15.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.615+0000 7fa5edfeb700 1 --2- 172.21.15.43:0/118720620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5e8108e50 0x7fa5e810b240 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:15.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.615+0000 7fa5edfeb700 1 -- 172.21.15.43:0/118720620 >> 172.21.15.43:0/118720620 conn(0x7fa5e8102860 msgr2=0x7fa5e8104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:15.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.615+0000 7fa5edfeb700 1 -- 172.21.15.43:0/118720620 shutdown_connections 2024-04-21T08:28:15.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.615+0000 7fa5edfeb700 1 -- 172.21.15.43:0/118720620 wait complete. 2024-04-21T08:28:15.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.616+0000 7fa5edfeb700 1 Processor -- start 2024-04-21T08:28:15.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.616+0000 7fa5edfeb700 1 -- start start 2024-04-21T08:28:15.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.616+0000 7fa5edfeb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5e8108e50 0x7fa5e819c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:15.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.616+0000 7fa5edfeb700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e810c6e0 con 0x7fa5e8108e50 2024-04-21T08:28:15.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.616+0000 7fa5e77fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5e8108e50 0x7fa5e819c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:15.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.616+0000 7fa5e77fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5e8108e50 0x7fa5e819c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54244/0 (socket says 172.21.15.43:54244) 2024-04-21T08:28:15.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.616+0000 7fa5e77fe700 1 -- 172.21.15.43:0/3897862067 learned_addr learned my addr 172.21.15.43:0/3897862067 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:15.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.616+0000 7fa5e77fe700 1 -- 172.21.15.43:0/3897862067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e819c700 con 0x7fa5e8108e50 2024-04-21T08:28:15.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.617+0000 7fa5e77fe700 1 --2- 172.21.15.43:0/3897862067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5e8108e50 0x7fa5e819c1c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8011ac0 tx=0x7fa5d8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:15.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.617+0000 7fa5e4ff9700 1 -- 172.21.15.43:0/3897862067 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5d8003b20 con 0x7fa5e8108e50 2024-04-21T08:28:15.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.617+0000 7fa5e4ff9700 1 -- 172.21.15.43:0/3897862067 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa5d801a8c0 con 0x7fa5e8108e50 2024-04-21T08:28:15.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.617+0000 7fa5edfeb700 1 -- 172.21.15.43:0/3897862067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e819c990 con 0x7fa5e8108e50 2024-04-21T08:28:15.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.617+0000 7fa5e4ff9700 1 -- 172.21.15.43:0/3897862067 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5d80248d0 con 0x7fa5e8108e50 2024-04-21T08:28:15.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.617+0000 7fa5edfeb700 1 -- 172.21.15.43:0/3897862067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e819f680 con 0x7fa5e8108e50 2024-04-21T08:28:15.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.617+0000 7fa5e4ff9700 1 -- 172.21.15.43:0/3897862067 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa5d8022020 con 0x7fa5e8108e50 2024-04-21T08:28:15.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.617+0000 7fa5e4ff9700 1 -- 172.21.15.43:0/3897862067 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa5d80193c0 con 0x7fa5e8108e50 2024-04-21T08:28:15.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.618+0000 7fa5edfeb700 1 -- 172.21.15.43:0/3897862067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e80640a0 con 0x7fa5e8108e50 2024-04-21T08:28:15.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.620+0000 7fa5e4ff9700 1 -- 172.21.15.43:0/3897862067 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fa5d802ddf0 con 0x7fa5e8108e50 2024-04-21T08:28:15.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.692+0000 7fa5edfeb700 1 -- 172.21.15.43:0/3897862067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fa5e819fbc0 con 0x7fa5e8108e50 2024-04-21T08:28:15.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.692+0000 7fa5e4ff9700 1 -- 172.21.15.43:0/3897862067 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fa5d801aa70 con 0x7fa5e8108e50 2024-04-21T08:28:15.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.693+0000 7fa5edfeb700 1 -- 172.21.15.43:0/3897862067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5e8108e50 msgr2=0x7fa5e819c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:15.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.693+0000 7fa5edfeb700 1 --2- 172.21.15.43:0/3897862067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5e8108e50 0x7fa5e819c1c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8011ac0 tx=0x7fa5d8011ba0 comp rx=0 tx=0).stop 2024-04-21T08:28:15.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.693+0000 7fa5edfeb700 1 -- 172.21.15.43:0/3897862067 shutdown_connections 2024-04-21T08:28:15.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.693+0000 7fa5edfeb700 1 --2- 172.21.15.43:0/3897862067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5e8108e50 0x7fa5e819c1c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:15.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.693+0000 7fa5edfeb700 1 -- 172.21.15.43:0/3897862067 >> 172.21.15.43:0/3897862067 conn(0x7fa5e8102860 msgr2=0x7fa5e81032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:15.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.693+0000 7fa5edfeb700 1 -- 172.21.15.43:0/3897862067 shutdown_connections 2024-04-21T08:28:15.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:14.693+0000 7fa5edfeb700 1 -- 172.21.15.43:0/3897862067 wait complete. 2024-04-21T08:28:15.238 INFO:teuthology.orchestra.run.smithi043.stdout:Restarting the monitor... 2024-04-21T08:28:15.506 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:15 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-a[27587]: 2024-04-21T08:28:15.505+0000 7fb40256a700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-21T08:28:15.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:15 smithi043 podman[27847]: 2024-04-21 08:28:15.67003889 +0000 UTC m=+0.263719682 container died 840bb477f04b880719e79551e5a713a8eddc2ea60430753ae4b904b9fffeed48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-a, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:28:16.296 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:16 smithi043 podman[27847]: 2024-04-21 08:28:16.047199469 +0000 UTC m=+0.640880271 container cleanup 840bb477f04b880719e79551e5a713a8eddc2ea60430753ae4b904b9fffeed48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-21T08:28:16.296 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:16 smithi043 bash[27847]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-a 2024-04-21T08:28:16.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:16 smithi043 podman[27861]: 2024-04-21 08:28:16.297894076 +0000 UTC m=+0.623603516 container remove 840bb477f04b880719e79551e5a713a8eddc2ea60430753ae4b904b9fffeed48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-21T08:28:16.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:16 smithi043 systemd[1]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-21T08:28:16.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:16 smithi043 systemd[1]: Stopped Ceph mon.a for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:28:16.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:16 smithi043 systemd[1]: Starting Ceph mon.a for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:28:17.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:16 smithi043 podman[27959]: 2024-04-21 08:28:16.553455957 +0000 UTC m=+0.018010829 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:28:18.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:17 smithi043 podman[27959]: 2024-04-21 08:28:17.733033303 +0000 UTC m=+1.197588199 container create 25b38f5f08346eddca90a08429d1cbee8da43b3de422220168e724d9855a4ec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-a, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True) 2024-04-21T08:28:20.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 podman[27959]: 2024-04-21 08:28:19.736762252 +0000 UTC m=+3.201317107 container init 25b38f5f08346eddca90a08429d1cbee8da43b3de422220168e724d9855a4ec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:28:20.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 podman[27959]: 2024-04-21 08:28:19.741228034 +0000 UTC m=+3.205782893 container start 25b38f5f08346eddca90a08429d1cbee8da43b3de422220168e724d9855a4ec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-a, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-21T08:28:20.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: set uid:gid to 167:167 (ceph:ceph) 2024-04-21T08:28:20.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev), process ceph-mon, pid 2 2024-04-21T08:28:20.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: pidfile_write: ignore empty --pid-file 2024-04-21T08:28:20.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: load: jerasure load: lrc 2024-04-21T08:28:20.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: RocksDB version: 7.9.2 2024-04-21T08:28:20.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Git sha 0 2024-04-21T08:28:20.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Compile date 2024-04-19 14:19:22 2024-04-21T08:28:20.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: DB SUMMARY 2024-04-21T08:28:20.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: DB Session ID: 9VIM28VVCITOE180TLE1 2024-04-21T08:28:20.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: CURRENT file: CURRENT 2024-04-21T08:28:20.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: IDENTITY file: IDENTITY 2024-04-21T08:28:20.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-21T08:28:20.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-21T08:28:20.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104526 ; 2024-04-21T08:28:20.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.error_if_exists: 0 2024-04-21T08:28:20.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.create_if_missing: 0 2024-04-21T08:28:20.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.paranoid_checks: 1 2024-04-21T08:28:20.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-21T08:28:20.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-21T08:28:20.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-21T08:28:20.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.env: 0x556a61052cc0 2024-04-21T08:28:20.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.fs: PosixFileSystem 2024-04-21T08:28:20.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.info_log: 0x556a61affbe0 2024-04-21T08:28:20.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_file_opening_threads: 16 2024-04-21T08:28:20.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.statistics: (nil) 2024-04-21T08:28:20.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.use_fsync: 0 2024-04-21T08:28:20.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_log_file_size: 0 2024-04-21T08:28:20.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-21T08:28:20.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-21T08:28:20.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.keep_log_file_num: 1000 2024-04-21T08:28:20.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.recycle_log_file_num: 0 2024-04-21T08:28:20.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.allow_fallocate: 1 2024-04-21T08:28:20.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.allow_mmap_reads: 0 2024-04-21T08:28:20.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.allow_mmap_writes: 0 2024-04-21T08:28:20.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.use_direct_reads: 0 2024-04-21T08:28:20.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-21T08:28:20.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.create_missing_column_families: 0 2024-04-21T08:28:20.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.db_log_dir: 2024-04-21T08:28:20.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.wal_dir: 2024-04-21T08:28:20.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-21T08:28:20.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-21T08:28:20.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-21T08:28:20.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-21T08:28:20.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-21T08:28:20.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-21T08:28:20.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.advise_random_on_open: 1 2024-04-21T08:28:20.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.db_write_buffer_size: 0 2024-04-21T08:28:20.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.write_buffer_manager: 0x556a61a930e0 2024-04-21T08:28:20.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-21T08:28:20.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-21T08:28:20.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-21T08:28:20.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.rate_limiter: (nil) 2024-04-21T08:28:20.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-21T08:28:20.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.wal_recovery_mode: 2 2024-04-21T08:28:20.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.enable_thread_tracking: 0 2024-04-21T08:28:20.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.enable_pipelined_write: 0 2024-04-21T08:28:20.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.unordered_write: 0 2024-04-21T08:28:20.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-21T08:28:20.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-21T08:28:20.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-21T08:28:20.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-21T08:28:20.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.row_cache: None 2024-04-21T08:28:20.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.wal_filter: None 2024-04-21T08:28:20.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-21T08:28:20.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.allow_ingest_behind: 0 2024-04-21T08:28:20.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.two_write_queues: 0 2024-04-21T08:28:20.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.manual_wal_flush: 0 2024-04-21T08:28:20.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.wal_compression: 0 2024-04-21T08:28:20.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.atomic_flush: 0 2024-04-21T08:28:20.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-21T08:28:20.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-21T08:28:20.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-21T08:28:20.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.log_readahead_size: 0 2024-04-21T08:28:20.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-21T08:28:20.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.best_efforts_recovery: 0 2024-04-21T08:28:20.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-21T08:28:20.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-21T08:28:20.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.allow_data_in_errors: 0 2024-04-21T08:28:20.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.db_host_id: __hostname__ 2024-04-21T08:28:20.070 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-21T08:28:20.070 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_background_jobs: 2 2024-04-21T08:28:20.070 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_background_compactions: -1 2024-04-21T08:28:20.070 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_subcompactions: 1 2024-04-21T08:28:20.070 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-21T08:28:20.070 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-21T08:28:20.070 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-21T08:28:20.070 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_total_wal_size: 0 2024-04-21T08:28:20.070 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-21T08:28:20.070 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-21T08:28:20.070 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-21T08:28:20.070 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-21T08:28:20.070 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_open_files: -1 2024-04-21T08:28:20.071 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.bytes_per_sync: 0 2024-04-21T08:28:20.071 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-21T08:28:20.071 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-21T08:28:20.071 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.compaction_readahead_size: 0 2024-04-21T08:28:20.071 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_background_flushes: -1 2024-04-21T08:28:20.071 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Compression algorithms supported: 2024-04-21T08:28:20.071 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-21T08:28:20.071 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: kZSTD supported: 0 2024-04-21T08:28:20.071 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: kXpressCompression supported: 0 2024-04-21T08:28:20.071 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: kLZ4HCCompression supported: 1 2024-04-21T08:28:20.071 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: kZlibCompression supported: 1 2024-04-21T08:28:20.071 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: kSnappyCompression supported: 1 2024-04-21T08:28:20.072 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: kLZ4Compression supported: 1 2024-04-21T08:28:20.072 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: kBZip2Compression supported: 0 2024-04-21T08:28:20.072 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-21T08:28:20.072 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-21T08:28:20.072 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-21T08:28:20.072 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-21T08:28:20.072 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-21T08:28:20.072 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.merge_operator: 2024-04-21T08:28:20.072 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.compaction_filter: None 2024-04-21T08:28:20.072 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.compaction_filter_factory: None 2024-04-21T08:28:20.072 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.sst_partitioner_factory: None 2024-04-21T08:28:20.072 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-21T08:28:20.072 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-21T08:28:20.072 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556a61affdc0) 2024-04-21T08:28:20.073 INFO:journalctl@ceph.mon.a.smithi043.stdout: cache_index_and_filter_blocks: 1 2024-04-21T08:28:20.073 INFO:journalctl@ceph.mon.a.smithi043.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-21T08:28:20.073 INFO:journalctl@ceph.mon.a.smithi043.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-21T08:28:20.073 INFO:journalctl@ceph.mon.a.smithi043.stdout: pin_top_level_index_and_filter: 1 2024-04-21T08:28:20.073 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_type: 0 2024-04-21T08:28:20.073 INFO:journalctl@ceph.mon.a.smithi043.stdout: data_block_index_type: 0 2024-04-21T08:28:20.073 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_shortening: 1 2024-04-21T08:28:20.073 INFO:journalctl@ceph.mon.a.smithi043.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-21T08:28:20.073 INFO:journalctl@ceph.mon.a.smithi043.stdout: checksum: 4 2024-04-21T08:28:20.073 INFO:journalctl@ceph.mon.a.smithi043.stdout: no_block_cache: 0 2024-04-21T08:28:20.073 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache: 0x556a61ac51f0 2024-04-21T08:28:20.073 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_name: BinnedLRUCache 2024-04-21T08:28:20.073 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_options: 2024-04-21T08:28:20.073 INFO:journalctl@ceph.mon.a.smithi043.stdout: capacity : 536870912 2024-04-21T08:28:20.074 INFO:journalctl@ceph.mon.a.smithi043.stdout: num_shard_bits : 4 2024-04-21T08:28:20.074 INFO:journalctl@ceph.mon.a.smithi043.stdout: strict_capacity_limit : 0 2024-04-21T08:28:20.074 INFO:journalctl@ceph.mon.a.smithi043.stdout: high_pri_pool_ratio: 0.000 2024-04-21T08:28:20.074 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_compressed: (nil) 2024-04-21T08:28:20.074 INFO:journalctl@ceph.mon.a.smithi043.stdout: persistent_cache: (nil) 2024-04-21T08:28:20.074 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_size: 4096 2024-04-21T08:28:20.074 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_size_deviation: 10 2024-04-21T08:28:20.074 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_restart_interval: 16 2024-04-21T08:28:20.074 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_block_restart_interval: 1 2024-04-21T08:28:20.075 INFO:journalctl@ceph.mon.a.smithi043.stdout: metadata_block_size: 4096 2024-04-21T08:28:20.075 INFO:journalctl@ceph.mon.a.smithi043.stdout: partition_filters: 0 2024-04-21T08:28:20.075 INFO:journalctl@ceph.mon.a.smithi043.stdout: use_delta_encoding: 1 2024-04-21T08:28:20.075 INFO:journalctl@ceph.mon.a.smithi043.stdout: filter_policy: bloomfilter 2024-04-21T08:28:20.075 INFO:journalctl@ceph.mon.a.smithi043.stdout: whole_key_filtering: 1 2024-04-21T08:28:20.075 INFO:journalctl@ceph.mon.a.smithi043.stdout: verify_compression: 0 2024-04-21T08:28:20.075 INFO:journalctl@ceph.mon.a.smithi043.stdout: read_amp_bytes_per_bit: 0 2024-04-21T08:28:20.075 INFO:journalctl@ceph.mon.a.smithi043.stdout: format_version: 5 2024-04-21T08:28:20.075 INFO:journalctl@ceph.mon.a.smithi043.stdout: enable_index_compression: 1 2024-04-21T08:28:20.075 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_align: 0 2024-04-21T08:28:20.075 INFO:journalctl@ceph.mon.a.smithi043.stdout: max_auto_readahead_size: 262144 2024-04-21T08:28:20.075 INFO:journalctl@ceph.mon.a.smithi043.stdout: prepopulate_block_cache: 0 2024-04-21T08:28:20.075 INFO:journalctl@ceph.mon.a.smithi043.stdout: initial_auto_readahead_size: 8192 2024-04-21T08:28:20.075 INFO:journalctl@ceph.mon.a.smithi043.stdout: num_file_reads_for_auto_readahead: 2 2024-04-21T08:28:20.076 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.write_buffer_size: 33554432 2024-04-21T08:28:20.076 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_write_buffer_number: 2 2024-04-21T08:28:20.076 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.compression: NoCompression 2024-04-21T08:28:20.076 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.bottommost_compression: Disabled 2024-04-21T08:28:20.077 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.prefix_extractor: nullptr 2024-04-21T08:28:20.077 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-21T08:28:20.077 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.num_levels: 7 2024-04-21T08:28:20.077 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-21T08:28:20.077 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-21T08:28:20.077 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-21T08:28:20.077 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-21T08:28:20.077 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-21T08:28:20.077 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-21T08:28:20.077 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-21T08:28:20.077 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-21T08:28:20.077 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-21T08:28:20.077 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-21T08:28:20.077 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-21T08:28:20.077 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-21T08:28:20.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-21T08:28:20.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.compression_opts.level: 32767 2024-04-21T08:28:20.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.compression_opts.strategy: 0 2024-04-21T08:28:20.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-21T08:28:20.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-21T08:28:20.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-21T08:28:20.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-21T08:28:20.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.compression_opts.enabled: false 2024-04-21T08:28:20.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-21T08:28:20.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-21T08:28:20.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-21T08:28:20.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-21T08:28:20.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.target_file_size_base: 67108864 2024-04-21T08:28:20.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-21T08:28:20.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-21T08:28:20.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-21T08:28:20.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-21T08:28:20.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-21T08:28:20.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-21T08:28:20.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-21T08:28:20.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-21T08:28:20.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-21T08:28:20.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-21T08:28:20.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-21T08:28:20.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-21T08:28:20.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-21T08:28:20.080 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-21T08:28:20.080 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.arena_block_size: 1048576 2024-04-21T08:28:20.080 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-21T08:28:20.080 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-21T08:28:20.080 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.disable_auto_compactions: 0 2024-04-21T08:28:20.080 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-21T08:28:20.080 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-21T08:28:20.080 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-21T08:28:20.080 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-21T08:28:20.080 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-21T08:28:20.080 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-21T08:28:20.080 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-21T08:28:20.080 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-21T08:28:20.080 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-21T08:28:20.081 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-21T08:28:20.081 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-21T08:28:20.081 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.inplace_update_support: 0 2024-04-21T08:28:20.081 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-21T08:28:20.081 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-21T08:28:20.081 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-21T08:28:20.081 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-21T08:28:20.081 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.bloom_locality: 0 2024-04-21T08:28:20.081 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.max_successive_merges: 0 2024-04-21T08:28:20.081 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-21T08:28:20.081 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.paranoid_file_checks: 0 2024-04-21T08:28:20.081 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.force_consistency_checks: 1 2024-04-21T08:28:20.081 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.report_bg_io_stats: 0 2024-04-21T08:28:20.081 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.ttl: 2592000 2024-04-21T08:28:20.081 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-21T08:28:20.082 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-21T08:28:20.082 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-21T08:28:20.082 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.enable_blob_files: false 2024-04-21T08:28:20.082 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.min_blob_size: 0 2024-04-21T08:28:20.082 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.blob_file_size: 268435456 2024-04-21T08:28:20.082 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-21T08:28:20.082 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-21T08:28:20.082 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-21T08:28:20.082 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-21T08:28:20.082 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-21T08:28:20.082 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.blob_file_starting_level: 0 2024-04-21T08:28:20.082 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-21T08:28:20.082 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-21T08:28:20.083 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-21T08:28:20.083 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0ff3a28d-bdf9-44a5-ac4c-7251add24865 2024-04-21T08:28:20.083 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713688099774410, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-21T08:28:20.083 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-21T08:28:20.083 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713688099775660, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97629, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14030, "raw_average_key_size": 49, "raw_value_size": 89991, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713688099, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0ff3a28d-bdf9-44a5-ac4c-7251add24865", "db_session_id": "9VIM28VVCITOE180TLE1", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-21T08:28:20.083 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713688099775728, "job": 1, "event": "recovery_finished"} 2024-04-21T08:28:20.083 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-21T08:28:20.083 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-21T08:28:20.083 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556a61bda000 2024-04-21T08:28:20.083 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: DB pointer 0x556a61bc4000 2024-04-21T08:28:20.083 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-21T08:28:20.083 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-21T08:28:20.083 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** DB Stats ** 2024-04-21T08:28:20.083 INFO:journalctl@ceph.mon.a.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-21T08:28:20.084 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-21T08:28:20.084 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-21T08:28:20.084 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-21T08:28:20.084 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-21T08:28:20.084 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-21T08:28:20.084 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-21T08:28:20.085 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-04-21T08:28:20.085 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** Compaction Stats [default] ** 2024-04-21T08:28:20.085 INFO:journalctl@ceph.mon.a.smithi043.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-21T08:28:20.085 INFO:journalctl@ceph.mon.a.smithi043.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-21T08:28:20.085 INFO:journalctl@ceph.mon.a.smithi043.stdout: L0 2/0 99.27 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-21T08:28:20.085 INFO:journalctl@ceph.mon.a.smithi043.stdout: Sum 2/0 99.27 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-21T08:28:20.085 INFO:journalctl@ceph.mon.a.smithi043.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-21T08:28:20.086 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-04-21T08:28:20.086 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** Compaction Stats [default] ** 2024-04-21T08:28:20.086 INFO:journalctl@ceph.mon.a.smithi043.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-21T08:28:20.086 INFO:journalctl@ceph.mon.a.smithi043.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-21T08:28:20.086 INFO:journalctl@ceph.mon.a.smithi043.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 118.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-21T08:28:20.087 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-04-21T08:28:20.087 INFO:journalctl@ceph.mon.a.smithi043.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-21T08:28:20.087 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-04-21T08:28:20.087 INFO:journalctl@ceph.mon.a.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-21T08:28:20.087 INFO:journalctl@ceph.mon.a.smithi043.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-21T08:28:20.087 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-21T08:28:20.087 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-21T08:28:20.088 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-21T08:28:20.088 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-21T08:28:20.088 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative compaction: 0.00 GB write, 25.99 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-21T08:28:20.088 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval compaction: 0.00 GB write, 25.99 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-21T08:28:20.088 INFO:journalctl@ceph.mon.a.smithi043.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-21T08:28:20.089 INFO:journalctl@ceph.mon.a.smithi043.stdout: Block cache BinnedLRUCache@0x556a61ac51f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-21T08:28:20.089 INFO:journalctl@ceph.mon.a.smithi043.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-21T08:28:20.089 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-04-21T08:28:20.089 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-21T08:28:20.089 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: starting mon.a rank 0 at public addrs [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] at bind addrs [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:28:20.089 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: mon.a@-1(???) e1 preinit fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:28:20.089 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: mon.a@-1(???).mds e1 new map 2024-04-21T08:28:20.090 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: mon.a@-1(???).mds e1 print_map 2024-04-21T08:28:20.090 INFO:journalctl@ceph.mon.a.smithi043.stdout: e1 2024-04-21T08:28:20.090 INFO:journalctl@ceph.mon.a.smithi043.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-21T08:28:20.090 INFO:journalctl@ceph.mon.a.smithi043.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-21T08:28:20.090 INFO:journalctl@ceph.mon.a.smithi043.stdout: legacy client fscid: -1 2024-04-21T08:28:20.090 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-04-21T08:28:20.091 INFO:journalctl@ceph.mon.a.smithi043.stdout: No filesystems configured 2024-04-21T08:28:20.091 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-21T08:28:20.091 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T08:28:20.091 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T08:28:20.091 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T08:28:20.091 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-21T08:28:20.092 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-21T08:28:20.092 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: monmap epoch 1 2024-04-21T08:28:20.092 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:28:20.092 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: last_changed 2024-04-21T08:28:08.995135+0000 2024-04-21T08:28:20.092 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: created 2024-04-21T08:28:08.995135+0000 2024-04-21T08:28:20.092 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: min_mon_release 19 (squid) 2024-04-21T08:28:20.093 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: election_strategy: 1 2024-04-21T08:28:20.093 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-04-21T08:28:20.093 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: fsmap 2024-04-21T08:28:20.093 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: osdmap e1: 0 total, 0 up, 0 in 2024-04-21T08:28:20.093 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:19 smithi043 ceph-mon[27978]: mgrmap e1: no daemons active 2024-04-21T08:28:20.596 INFO:teuthology.orchestra.run.smithi043.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-21T08:28:21.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:20 smithi043 bash[27959]: 25b38f5f08346eddca90a08429d1cbee8da43b3de422220168e724d9855a4ec4 2024-04-21T08:28:21.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:20 smithi043 systemd[1]: Started Ceph mon.a for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:28:25.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:24 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3514888264' entity='client.admin' 2024-04-21T08:28:26.541 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.602+0000 7f5836064700 1 Processor -- start 2024-04-21T08:28:26.541 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.602+0000 7f5836064700 1 -- start start 2024-04-21T08:28:26.541 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.602+0000 7f5836064700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58301084b0 0x7f58301088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:26.541 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.602+0000 7f5836064700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5830108ea0 con 0x7f58301084b0 2024-04-21T08:28:26.541 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.602+0000 7f582f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58301084b0 0x7f58301088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:26.542 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.602+0000 7f582f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58301084b0 0x7f58301088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55488/0 (socket says 172.21.15.43:55488) 2024-04-21T08:28:26.542 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.602+0000 7f582f7fe700 1 -- 172.21.15.43:0/2120070668 learned_addr learned my addr 172.21.15.43:0/2120070668 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:26.542 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.603+0000 7f582f7fe700 1 -- 172.21.15.43:0/2120070668 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5830109660 con 0x7f58301084b0 2024-04-21T08:28:26.542 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.603+0000 7f582f7fe700 1 --2- 172.21.15.43:0/2120070668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58301084b0 0x7f58301088d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f58200098a0 tx=0x7f5820009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0748b820517920 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:26.542 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.603+0000 7f582e7fc700 1 -- 172.21.15.43:0/2120070668 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5820004160 con 0x7f58301084b0 2024-04-21T08:28:26.542 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.603+0000 7f582e7fc700 1 -- 172.21.15.43:0/2120070668 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5820004300 con 0x7f58301084b0 2024-04-21T08:28:26.542 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.603+0000 7f582e7fc700 1 -- 172.21.15.43:0/2120070668 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5820011af0 con 0x7f58301084b0 2024-04-21T08:28:26.542 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.604+0000 7f5836064700 1 -- 172.21.15.43:0/2120070668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58301084b0 msgr2=0x7f58301088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:26.542 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.604+0000 7f5836064700 1 --2- 172.21.15.43:0/2120070668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58301084b0 0x7f58301088d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f58200098a0 tx=0x7f5820009bb0 comp rx=0 tx=0).stop 2024-04-21T08:28:26.542 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.604+0000 7f5836064700 1 -- 172.21.15.43:0/2120070668 shutdown_connections 2024-04-21T08:28:26.542 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.604+0000 7f5836064700 1 --2- 172.21.15.43:0/2120070668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58301084b0 0x7f58301088d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:26.542 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.604+0000 7f5836064700 1 -- 172.21.15.43:0/2120070668 >> 172.21.15.43:0/2120070668 conn(0x7f5830103af0 msgr2=0x7f5830105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:26.543 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.604+0000 7f5836064700 1 -- 172.21.15.43:0/2120070668 shutdown_connections 2024-04-21T08:28:26.543 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.604+0000 7f5836064700 1 -- 172.21.15.43:0/2120070668 wait complete. 2024-04-21T08:28:26.543 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.604+0000 7f5836064700 1 Processor -- start 2024-04-21T08:28:26.543 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.605+0000 7f5836064700 1 -- start start 2024-04-21T08:28:26.543 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.605+0000 7f5836064700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58301084b0 0x7f583019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:26.543 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.605+0000 7f5836064700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f583010c890 con 0x7f58301084b0 2024-04-21T08:28:26.543 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.605+0000 7f582f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58301084b0 0x7f583019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:26.543 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.605+0000 7f582f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58301084b0 0x7f583019e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55496/0 (socket says 172.21.15.43:55496) 2024-04-21T08:28:26.543 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.605+0000 7f582f7fe700 1 -- 172.21.15.43:0/3514888264 learned_addr learned my addr 172.21.15.43:0/3514888264 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:26.543 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.605+0000 7f582f7fe700 1 -- 172.21.15.43:0/3514888264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583019e940 con 0x7f58301084b0 2024-04-21T08:28:26.543 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.605+0000 7f582f7fe700 1 --2- 172.21.15.43:0/3514888264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58301084b0 0x7f583019e400 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5820003f40 tx=0x7f5820011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:26.543 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.605+0000 7f582cff9700 1 -- 172.21.15.43:0/3514888264 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58200038b0 con 0x7f58301084b0 2024-04-21T08:28:26.543 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.605+0000 7f582cff9700 1 -- 172.21.15.43:0/3514888264 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5820003a50 con 0x7f58301084b0 2024-04-21T08:28:26.543 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.605+0000 7f5836064700 1 -- 172.21.15.43:0/3514888264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f583019ebd0 con 0x7f58301084b0 2024-04-21T08:28:26.543 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.605+0000 7f582cff9700 1 -- 172.21.15.43:0/3514888264 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f582001a5c0 con 0x7f58301084b0 2024-04-21T08:28:26.544 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.606+0000 7f5836064700 1 -- 172.21.15.43:0/3514888264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58301a18c0 con 0x7f58301084b0 2024-04-21T08:28:26.544 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.606+0000 7f582cff9700 1 -- 172.21.15.43:0/3514888264 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f582001ac50 con 0x7f58301084b0 2024-04-21T08:28:26.544 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.606+0000 7f582cff9700 1 -- 172.21.15.43:0/3514888264 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5820019700 con 0x7f58301084b0 2024-04-21T08:28:26.544 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.606+0000 7f5836064700 1 -- 172.21.15.43:0/3514888264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58300640a0 con 0x7f58301084b0 2024-04-21T08:28:26.544 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.609+0000 7f582cff9700 1 -- 172.21.15.43:0/3514888264 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5820019900 con 0x7f58301084b0 2024-04-21T08:28:26.544 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.680+0000 7f5836064700 1 -- 172.21.15.43:0/3514888264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f58301a1a90 con 0x7f58301084b0 2024-04-21T08:28:26.544 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.682+0000 7f582cff9700 1 -- 172.21.15.43:0/3514888264 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f5820019ae0 con 0x7f58301084b0 2024-04-21T08:28:26.544 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.683+0000 7f5836064700 1 -- 172.21.15.43:0/3514888264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58301084b0 msgr2=0x7f583019e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:26.544 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.683+0000 7f5836064700 1 --2- 172.21.15.43:0/3514888264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58301084b0 0x7f583019e400 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5820003f40 tx=0x7f5820011890 comp rx=0 tx=0).stop 2024-04-21T08:28:26.544 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.683+0000 7f5836064700 1 -- 172.21.15.43:0/3514888264 shutdown_connections 2024-04-21T08:28:26.544 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.683+0000 7f5836064700 1 --2- 172.21.15.43:0/3514888264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58301084b0 0x7f583019e400 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:26.544 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.683+0000 7f5836064700 1 -- 172.21.15.43:0/3514888264 >> 172.21.15.43:0/3514888264 conn(0x7f5830103af0 msgr2=0x7f58301044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:26.544 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.683+0000 7f5836064700 1 -- 172.21.15.43:0/3514888264 shutdown_connections 2024-04-21T08:28:26.544 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.683+0000 7f5836064700 1 -- 172.21.15.43:0/3514888264 wait complete. 2024-04-21T08:28:26.545 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-21T08:28:26.566 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:28:26.566 INFO:teuthology.orchestra.run.smithi043.stdout:Creating mgr... 2024-04-21T08:28:26.567 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-21T08:28:26.567 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-21T08:28:26.826 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mgr.x 2024-04-21T08:28:26.826 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mgr.x.service: Unit ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-21T08:28:26.979 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968.target.wants/ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@.service. 2024-04-21T08:28:26.996 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:26 smithi043 systemd[1]: Starting Ceph mgr.x for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:28:27.312 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:27 smithi043 podman[28197]: 2024-04-21 08:28:27.125773304 +0000 UTC m=+0.018158034 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:28:27.399 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:27 smithi043 podman[28197]: 2024-04-21 08:28:27.236847848 +0000 UTC m=+0.129232621 container create 7477df00b6fb78720379d2845187ec7156d30543a4cffb70d358b6d6fab2e5dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD) 2024-04-21T08:28:27.511 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-21T08:28:27.511 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-04-21T08:28:27.519 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-21T08:28:27.519 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-04-21T08:28:27.519 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-04-21T08:28:27.519 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to enable service . firewalld.service is not available 2024-04-21T08:28:27.528 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-21T08:28:27.529 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-04-21T08:28:27.537 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-21T08:28:27.537 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-04-21T08:28:27.537 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-04-21T08:28:27.537 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-21T08:28:27.537 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr to start... 2024-04-21T08:28:27.537 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr... 2024-04-21T08:28:27.607 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:27 smithi043 podman[28197]: 2024-04-21 08:28:27.431171087 +0000 UTC m=+0.323555808 container init 7477df00b6fb78720379d2845187ec7156d30543a4cffb70d358b6d6fab2e5dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-21T08:28:27.607 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:27 smithi043 podman[28197]: 2024-04-21 08:28:27.434227466 +0000 UTC m=+0.326612177 container start 7477df00b6fb78720379d2845187ec7156d30543a4cffb70d358b6d6fab2e5dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:28:27.607 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:27 smithi043 bash[28197]: 7477df00b6fb78720379d2845187ec7156d30543a4cffb70d358b6d6fab2e5dd 2024-04-21T08:28:27.607 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:27 smithi043 systemd[1]: Started Ceph mgr.x for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:28:27.899 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:27 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:27.606+0000 7f8b6020d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:28:27.899 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:27 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:27.803+0000 7f8b6020d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:28:28.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:28 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1292425449' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-21T08:28:28.245 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:27 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:27.912+0000 7f8b6020d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:28:28.245 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:28 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:28.244+0000 7f8b6020d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:28:28.561 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:28 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:28.333+0000 7f8b6020d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:28:28.700 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:28.700 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:28:28.700 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "d57a7d98-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:28:28.700 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-04-21T08:28:28.700 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-21T08:28:28.700 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-21T08:28:28.700 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-21T08:28:28.700 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:28.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-21T08:28:28.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-21T08:28:28.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-04-21T08:28:28.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:28:28.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-21T08:28:28.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-04-21T08:28:28.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:28:28.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-21T08:28:28.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-21T08:28:28.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:28.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-21T08:28:28.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-21T08:28:28.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:28.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-21T08:28:28.702 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:28.702 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-21T08:28:28.702 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-21T08:28:28.702 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-21T08:28:28.702 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-21T08:28:28.702 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-21T08:28:28.702 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-21T08:28:28.702 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:28.702 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-21T08:28:28.702 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-21T08:28:28.702 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-21T08:28:28.702 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-21T08:28:28.702 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-21T08:28:28.702 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-21T08:28:28.703 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-21T08:28:28.703 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-21T08:28:28.703 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-21T08:28:28.703 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:28.703 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-21T08:28:28.703 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:28.703 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-21T08:28:28.703 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-21T08:28:28.703 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:28.703 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-21T08:28:28.703 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-21T08:28:28.703 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-21T08:28:28.703 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-21T08:28:28.703 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-21T08:28:28.704 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-21T08:28:28.704 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-04-21T08:28:28.704 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:28:28.704 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:28:28.704 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:28.704 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-21T08:28:28.704 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:28.704 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-04-21T08:28:11.976852+0000", 2024-04-21T08:28:28.704 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:28:28.704 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:28.704 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-21T08:28:28.704 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:28:28.704 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.001+0000 7f82a7c2c700 1 Processor -- start 2024-04-21T08:28:28.704 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.002+0000 7f82a7c2c700 1 -- start start 2024-04-21T08:28:28.705 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.002+0000 7f82a7c2c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82a0106610 0x7f82a0106a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:28.705 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.002+0000 7f82a7c2c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f82a0107000 con 0x7f82a0106610 2024-04-21T08:28:28.705 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.002+0000 7f82a59c8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82a0106610 0x7f82a0106a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:28.705 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.002+0000 7f82a59c8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82a0106610 0x7f82a0106a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35426/0 (socket says 172.21.15.43:35426) 2024-04-21T08:28:28.705 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.002+0000 7f82a59c8700 1 -- 172.21.15.43:0/1966410133 learned_addr learned my addr 172.21.15.43:0/1966410133 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:28.705 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.003+0000 7f82a59c8700 1 -- 172.21.15.43:0/1966410133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a01095f0 con 0x7f82a0106610 2024-04-21T08:28:28.705 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.003+0000 7f82a59c8700 1 --2- 172.21.15.43:0/1966410133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82a0106610 0x7f82a0106a30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f829c0098a0 tx=0x7f829c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b57363c07f7c5a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:28.705 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.003+0000 7f82a49c6700 1 -- 172.21.15.43:0/1966410133 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f829c004160 con 0x7f82a0106610 2024-04-21T08:28:28.705 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.003+0000 7f82a49c6700 1 -- 172.21.15.43:0/1966410133 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f829c004300 con 0x7f82a0106610 2024-04-21T08:28:28.705 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.003+0000 7f82a49c6700 1 -- 172.21.15.43:0/1966410133 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f829c011af0 con 0x7f82a0106610 2024-04-21T08:28:28.705 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.003+0000 7f82a7c2c700 1 -- 172.21.15.43:0/1966410133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82a0106610 msgr2=0x7f82a0106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:28.705 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.003+0000 7f82a7c2c700 1 --2- 172.21.15.43:0/1966410133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82a0106610 0x7f82a0106a30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f829c0098a0 tx=0x7f829c009bb0 comp rx=0 tx=0).stop 2024-04-21T08:28:28.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.004+0000 7f82a7c2c700 1 -- 172.21.15.43:0/1966410133 shutdown_connections 2024-04-21T08:28:28.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.004+0000 7f82a7c2c700 1 --2- 172.21.15.43:0/1966410133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82a0106610 0x7f82a0106a30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:28.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.004+0000 7f82a7c2c700 1 -- 172.21.15.43:0/1966410133 >> 172.21.15.43:0/1966410133 conn(0x7f82a0101f70 msgr2=0x7f82a01043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:28.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.004+0000 7f82a7c2c700 1 -- 172.21.15.43:0/1966410133 shutdown_connections 2024-04-21T08:28:28.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.004+0000 7f82a7c2c700 1 -- 172.21.15.43:0/1966410133 wait complete. 2024-04-21T08:28:28.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.004+0000 7f82a7c2c700 1 Processor -- start 2024-04-21T08:28:28.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.004+0000 7f82a7c2c700 1 -- start start 2024-04-21T08:28:28.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.005+0000 7f82a7c2c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82a0106610 0x7f82a0108e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:28.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.005+0000 7f82a7c2c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f82a010c9a0 con 0x7f82a0106610 2024-04-21T08:28:28.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.005+0000 7f82a59c8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82a0106610 0x7f82a0108e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:28.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.005+0000 7f82a59c8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82a0106610 0x7f82a0108e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35438/0 (socket says 172.21.15.43:35438) 2024-04-21T08:28:28.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.005+0000 7f82a59c8700 1 -- 172.21.15.43:0/1292425449 learned_addr learned my addr 172.21.15.43:0/1292425449 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:28.706 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.005+0000 7f82a59c8700 1 -- 172.21.15.43:0/1292425449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a01093b0 con 0x7f82a0106610 2024-04-21T08:28:28.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.005+0000 7f82a59c8700 1 --2- 172.21.15.43:0/1292425449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82a0106610 0x7f82a0108e70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f829c000c00 tx=0x7f829c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:28.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.005+0000 7f8292ffd700 1 -- 172.21.15.43:0/1292425449 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f829c003b20 con 0x7f82a0106610 2024-04-21T08:28:28.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.005+0000 7f8292ffd700 1 -- 172.21.15.43:0/1292425449 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f829c01a8c0 con 0x7f82a0106610 2024-04-21T08:28:28.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.005+0000 7f82a7c2c700 1 -- 172.21.15.43:0/1292425449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a01075a0 con 0x7f82a0106610 2024-04-21T08:28:28.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.005+0000 7f8292ffd700 1 -- 172.21.15.43:0/1292425449 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f829c024940 con 0x7f82a0106610 2024-04-21T08:28:28.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.005+0000 7f82a7c2c700 1 -- 172.21.15.43:0/1292425449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a0107ae0 con 0x7f82a0106610 2024-04-21T08:28:28.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.006+0000 7f8292ffd700 1 -- 172.21.15.43:0/1292425449 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f829c022020 con 0x7f82a0106610 2024-04-21T08:28:28.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.006+0000 7f8292ffd700 1 -- 172.21.15.43:0/1292425449 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f829c019610 con 0x7f82a0106610 2024-04-21T08:28:28.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.006+0000 7f82a7c2c700 1 -- 172.21.15.43:0/1292425449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82a00640a0 con 0x7f82a0106610 2024-04-21T08:28:28.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.009+0000 7f8292ffd700 1 -- 172.21.15.43:0/1292425449 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f829c01aa70 con 0x7f82a0106610 2024-04-21T08:28:28.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.082+0000 7f82a7c2c700 1 -- 172.21.15.43:0/1292425449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f82a0107d90 con 0x7f82a0106610 2024-04-21T08:28:28.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.082+0000 7f8292ffd700 1 -- 172.21.15.43:0/1292425449 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f829c01a430 con 0x7f82a0106610 2024-04-21T08:28:28.707 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.084+0000 7f82a7c2c700 1 -- 172.21.15.43:0/1292425449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82a0106610 msgr2=0x7f82a0108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:28.708 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.084+0000 7f82a7c2c700 1 --2- 172.21.15.43:0/1292425449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82a0106610 0x7f82a0108e70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f829c000c00 tx=0x7f829c011b70 comp rx=0 tx=0).stop 2024-04-21T08:28:28.708 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.084+0000 7f82a7c2c700 1 -- 172.21.15.43:0/1292425449 shutdown_connections 2024-04-21T08:28:28.708 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.084+0000 7f82a7c2c700 1 --2- 172.21.15.43:0/1292425449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82a0106610 0x7f82a0108e70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:28.708 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.084+0000 7f82a7c2c700 1 -- 172.21.15.43:0/1292425449 >> 172.21.15.43:0/1292425449 conn(0x7f82a0101f70 msgr2=0x7f82a01029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:28.708 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.084+0000 7f82a7c2c700 1 -- 172.21.15.43:0/1292425449 shutdown_connections 2024-04-21T08:28:28.708 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:28.084+0000 7f82a7c2c700 1 -- 172.21.15.43:0/1292425449 wait complete. 2024-04-21T08:28:28.708 INFO:teuthology.orchestra.run.smithi043.stdout:mgr not available, waiting (1/15)... 2024-04-21T08:28:29.956 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:29 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:29.604+0000 7f8b6020d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:28:29.957 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:29 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:29.780+0000 7f8b6020d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:28:29.957 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:29 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:29.956+0000 7f8b6020d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:28:30.560 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:30 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:30.254+0000 7f8b6020d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:28:30.560 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:30 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:30.349+0000 7f8b6020d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:28:30.560 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:30 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:30.559+0000 7f8b6020d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:28:30.811 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:30 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:30.763+0000 7f8b6020d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:28:31.410 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:31 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3353126956' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-21T08:28:31.811 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:31 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:31.407+0000 7f8b6020d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:28:32.205 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:31 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:31.844+0000 7f8b6020d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:28:32.205 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:31 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:31.944+0000 7f8b6020d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:28:32.205 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:32 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:32.153+0000 7f8b6020d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:28:33.027 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:32 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:32.699+0000 7f8b6020d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:28:33.028 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:32 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:32.794+0000 7f8b6020d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:28:33.291 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:33 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:33.047+0000 7f8b6020d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:28:33.291 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:33 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:33.190+0000 7f8b6020d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:28:33.561 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:33 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:33.291+0000 7f8b6020d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:28:33.561 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:33 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:33.412+0000 7f8b6020d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:28:34.061 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:33 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:33.732+0000 7f8b6020d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:28:34.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:34.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:28:34.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "d57a7d98-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:28:34.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-04-21T08:28:34.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-21T08:28:34.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-21T08:28:34.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-21T08:28:34.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:34.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-21T08:28:34.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-21T08:28:34.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-04-21T08:28:34.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:28:34.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-21T08:28:34.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-04-21T08:28:34.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:28:34.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-04-21T08:28:34.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-21T08:28:34.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:34.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-21T08:28:34.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-21T08:28:34.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:34.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-21T08:28:34.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:34.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-21T08:28:34.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-21T08:28:34.375 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-21T08:28:34.375 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-21T08:28:34.375 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-21T08:28:34.375 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-21T08:28:34.375 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:34.375 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-21T08:28:34.375 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-21T08:28:34.375 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-21T08:28:34.375 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-21T08:28:34.375 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-21T08:28:34.375 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-21T08:28:34.375 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-21T08:28:34.375 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-21T08:28:34.375 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-21T08:28:34.375 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:34.376 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-21T08:28:34.376 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:34.376 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-21T08:28:34.376 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-21T08:28:34.376 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:34.376 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-21T08:28:34.376 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-21T08:28:34.376 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-21T08:28:34.376 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-21T08:28:34.376 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-21T08:28:34.376 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-21T08:28:34.377 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-04-21T08:28:34.377 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:28:34.377 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:28:34.377 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:34.377 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-21T08:28:34.377 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:34.377 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-04-21T08:28:11.976852+0000", 2024-04-21T08:28:34.377 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:28:34.377 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:34.377 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-21T08:28:34.377 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:28:34.377 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.114+0000 7f2a75129700 1 Processor -- start 2024-04-21T08:28:34.377 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.114+0000 7f2a75129700 1 -- start start 2024-04-21T08:28:34.377 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.114+0000 7f2a75129700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a70108e70 0x7f2a7010b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:34.378 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.114+0000 7f2a75129700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7007be40 con 0x7f2a70108e70 2024-04-21T08:28:34.378 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.114+0000 7f2a6ed9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a70108e70 0x7f2a7010b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:34.378 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.115+0000 7f2a6ed9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a70108e70 0x7f2a7010b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35454/0 (socket says 172.21.15.43:35454) 2024-04-21T08:28:34.378 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.115+0000 7f2a6ed9d700 1 -- 172.21.15.43:0/431375223 learned_addr learned my addr 172.21.15.43:0/431375223 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:34.378 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.115+0000 7f2a6ed9d700 1 -- 172.21.15.43:0/431375223 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7010b7a0 con 0x7f2a70108e70 2024-04-21T08:28:34.378 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.115+0000 7f2a6ed9d700 1 --2- 172.21.15.43:0/431375223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a70108e70 0x7f2a7010b260 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2a600098a0 tx=0x7f2a60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a7125becb11ae0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:34.378 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.115+0000 7f2a6dd9b700 1 -- 172.21.15.43:0/431375223 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a60004160 con 0x7f2a70108e70 2024-04-21T08:28:34.378 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.115+0000 7f2a6dd9b700 1 -- 172.21.15.43:0/431375223 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2a60004300 con 0x7f2a70108e70 2024-04-21T08:28:34.378 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.115+0000 7f2a6dd9b700 1 -- 172.21.15.43:0/431375223 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a60011af0 con 0x7f2a70108e70 2024-04-21T08:28:34.378 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.116+0000 7f2a75129700 1 -- 172.21.15.43:0/431375223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a70108e70 msgr2=0x7f2a7010b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:34.378 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.116+0000 7f2a75129700 1 --2- 172.21.15.43:0/431375223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a70108e70 0x7f2a7010b260 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2a600098a0 tx=0x7f2a60009bb0 comp rx=0 tx=0).stop 2024-04-21T08:28:34.378 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.116+0000 7f2a75129700 1 -- 172.21.15.43:0/431375223 shutdown_connections 2024-04-21T08:28:34.378 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.116+0000 7f2a75129700 1 --2- 172.21.15.43:0/431375223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a70108e70 0x7f2a7010b260 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:34.378 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.116+0000 7f2a75129700 1 -- 172.21.15.43:0/431375223 >> 172.21.15.43:0/431375223 conn(0x7f2a701028e0 msgr2=0x7f2a70104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:34.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.116+0000 7f2a75129700 1 -- 172.21.15.43:0/431375223 shutdown_connections 2024-04-21T08:28:34.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.116+0000 7f2a75129700 1 -- 172.21.15.43:0/431375223 wait complete. 2024-04-21T08:28:34.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.117+0000 7f2a75129700 1 Processor -- start 2024-04-21T08:28:34.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.117+0000 7f2a75129700 1 -- start start 2024-04-21T08:28:34.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.117+0000 7f2a75129700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a70108e70 0x7f2a7019c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:34.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.117+0000 7f2a75129700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7010c6a0 con 0x7f2a70108e70 2024-04-21T08:28:34.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.117+0000 7f2a6ed9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a70108e70 0x7f2a7019c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:34.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.117+0000 7f2a6ed9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a70108e70 0x7f2a7019c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35468/0 (socket says 172.21.15.43:35468) 2024-04-21T08:28:34.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.117+0000 7f2a6ed9d700 1 -- 172.21.15.43:0/3353126956 learned_addr learned my addr 172.21.15.43:0/3353126956 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:34.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.117+0000 7f2a6ed9d700 1 -- 172.21.15.43:0/3353126956 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7019c6f0 con 0x7f2a70108e70 2024-04-21T08:28:34.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.118+0000 7f2a6ed9d700 1 --2- 172.21.15.43:0/3353126956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a70108e70 0x7f2a7019c1b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2a60011ac0 tx=0x7f2a60011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:34.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.118+0000 7f2a67fff700 1 -- 172.21.15.43:0/3353126956 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a60003b20 con 0x7f2a70108e70 2024-04-21T08:28:34.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.118+0000 7f2a67fff700 1 -- 172.21.15.43:0/3353126956 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2a6001a8c0 con 0x7f2a70108e70 2024-04-21T08:28:34.379 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.118+0000 7f2a75129700 1 -- 172.21.15.43:0/3353126956 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a7019c980 con 0x7f2a70108e70 2024-04-21T08:28:34.380 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.118+0000 7f2a67fff700 1 -- 172.21.15.43:0/3353126956 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a600248d0 con 0x7f2a70108e70 2024-04-21T08:28:34.380 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.118+0000 7f2a75129700 1 -- 172.21.15.43:0/3353126956 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a7019f670 con 0x7f2a70108e70 2024-04-21T08:28:34.380 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.118+0000 7f2a67fff700 1 -- 172.21.15.43:0/3353126956 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2a60022020 con 0x7f2a70108e70 2024-04-21T08:28:34.380 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.118+0000 7f2a67fff700 1 -- 172.21.15.43:0/3353126956 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2a60024e00 con 0x7f2a70108e70 2024-04-21T08:28:34.380 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.119+0000 7f2a75129700 1 -- 172.21.15.43:0/3353126956 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a700640a0 con 0x7f2a70108e70 2024-04-21T08:28:34.380 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.121+0000 7f2a67fff700 1 -- 172.21.15.43:0/3353126956 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f2a6001aa70 con 0x7f2a70108e70 2024-04-21T08:28:34.380 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.195+0000 7f2a75129700 1 -- 172.21.15.43:0/3353126956 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2a7019f8e0 con 0x7f2a70108e70 2024-04-21T08:28:34.380 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.196+0000 7f2a67fff700 1 -- 172.21.15.43:0/3353126956 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f2a6001a430 con 0x7f2a70108e70 2024-04-21T08:28:34.380 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.197+0000 7f2a75129700 1 -- 172.21.15.43:0/3353126956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a70108e70 msgr2=0x7f2a7019c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:34.380 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.197+0000 7f2a75129700 1 --2- 172.21.15.43:0/3353126956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a70108e70 0x7f2a7019c1b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2a60011ac0 tx=0x7f2a60011ba0 comp rx=0 tx=0).stop 2024-04-21T08:28:34.380 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.197+0000 7f2a75129700 1 -- 172.21.15.43:0/3353126956 shutdown_connections 2024-04-21T08:28:34.380 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.197+0000 7f2a75129700 1 --2- 172.21.15.43:0/3353126956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a70108e70 0x7f2a7019c1b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:34.380 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.197+0000 7f2a75129700 1 -- 172.21.15.43:0/3353126956 >> 172.21.15.43:0/3353126956 conn(0x7f2a701028e0 msgr2=0x7f2a70103330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:34.380 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.197+0000 7f2a75129700 1 -- 172.21.15.43:0/3353126956 shutdown_connections 2024-04-21T08:28:34.381 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:31.197+0000 7f2a75129700 1 -- 172.21.15.43:0/3353126956 wait complete. 2024-04-21T08:28:34.381 INFO:teuthology.orchestra.run.smithi043.stdout:mgr not available, waiting (2/15)... 2024-04-21T08:28:34.653 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:34 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:34.368+0000 7f8b6020d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:28:34.653 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:34 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:34.599+0000 7f8b6020d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:28:35.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:34 smithi043 ceph-mon[27978]: Activating manager daemon x 2024-04-21T08:28:35.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:34 smithi043 ceph-mon[27978]: mgrmap e2: x(active, starting, since 0.00314676s) 2024-04-21T08:28:35.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:34 smithi043 ceph-mon[27978]: from='mgr.14100 172.21.15.43:0/863228572' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T08:28:35.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:34 smithi043 ceph-mon[27978]: from='mgr.14100 172.21.15.43:0/863228572' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T08:28:35.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:34 smithi043 ceph-mon[27978]: from='mgr.14100 172.21.15.43:0/863228572' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T08:28:35.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:34 smithi043 ceph-mon[27978]: from='mgr.14100 172.21.15.43:0/863228572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:28:35.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:34 smithi043 ceph-mon[27978]: from='mgr.14100 172.21.15.43:0/863228572' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T08:28:35.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:34 smithi043 ceph-mon[27978]: Manager daemon x is now available 2024-04-21T08:28:35.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:34 smithi043 ceph-mon[27978]: from='mgr.14100 172.21.15.43:0/863228572' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T08:28:35.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:34 smithi043 ceph-mon[27978]: from='mgr.14100 172.21.15.43:0/863228572' entity='mgr.x' 2024-04-21T08:28:35.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:34 smithi043 ceph-mon[27978]: from='mgr.14100 172.21.15.43:0/863228572' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T08:28:35.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:34 smithi043 ceph-mon[27978]: from='mgr.14100 172.21.15.43:0/863228572' entity='mgr.x' 2024-04-21T08:28:35.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:34 smithi043 ceph-mon[27978]: from='mgr.14100 172.21.15.43:0/863228572' entity='mgr.x' 2024-04-21T08:28:36.786 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:36 smithi043 ceph-mon[27978]: mgrmap e3: x(active, since 1.00747s) 2024-04-21T08:28:37.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:37.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:28:37.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "d57a7d98-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:28:37.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-04-21T08:28:37.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-21T08:28:37.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-21T08:28:37.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-21T08:28:37.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:37.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-21T08:28:37.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-21T08:28:37.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-04-21T08:28:37.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:28:37.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-21T08:28:37.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-04-21T08:28:37.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:28:37.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 17, 2024-04-21T08:28:37.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-21T08:28:37.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:37.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-21T08:28:37.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-21T08:28:37.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:37.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-21T08:28:37.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:37.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-21T08:28:37.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-21T08:28:37.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-21T08:28:37.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-21T08:28:37.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-21T08:28:37.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-21T08:28:37.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:37.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-21T08:28:37.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-21T08:28:37.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-21T08:28:37.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-21T08:28:37.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-21T08:28:37.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-21T08:28:37.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-21T08:28:37.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-21T08:28:37.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-21T08:28:37.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:37.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-21T08:28:37.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:37.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-21T08:28:37.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-21T08:28:37.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:37.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-21T08:28:37.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-21T08:28:37.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-21T08:28:37.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-21T08:28:37.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-21T08:28:37.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-21T08:28:37.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-04-21T08:28:37.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:28:37.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:28:37.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:37.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-21T08:28:37.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:37.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-04-21T08:28:11.976852+0000", 2024-04-21T08:28:37.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:28:37.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:37.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-21T08:28:37.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:28:37.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.813+0000 7efd3eb71700 1 Processor -- start 2024-04-21T08:28:37.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.813+0000 7efd3eb71700 1 -- start start 2024-04-21T08:28:37.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.813+0000 7efd3eb71700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd3807de40 0x7efd3807c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:37.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.813+0000 7efd3eb71700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efd3807c8c0 con 0x7efd3807de40 2024-04-21T08:28:37.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.814+0000 7efd3c90d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd3807de40 0x7efd3807c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:37.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.814+0000 7efd3c90d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd3807de40 0x7efd3807c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53814/0 (socket says 172.21.15.43:53814) 2024-04-21T08:28:37.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.814+0000 7efd3c90d700 1 -- 172.21.15.43:0/1279206897 learned_addr learned my addr 172.21.15.43:0/1279206897 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:37.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.814+0000 7efd3c90d700 1 -- 172.21.15.43:0/1279206897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3807ca40 con 0x7efd3807de40 2024-04-21T08:28:37.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.814+0000 7efd3c90d700 1 --2- 172.21.15.43:0/1279206897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd3807de40 0x7efd3807c2f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efd280098a0 tx=0x7efd28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798dc4522e72bc47 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:37.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.814+0000 7efd377fe700 1 -- 172.21.15.43:0/1279206897 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd28004160 con 0x7efd3807de40 2024-04-21T08:28:37.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.815+0000 7efd377fe700 1 -- 172.21.15.43:0/1279206897 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efd28004300 con 0x7efd3807de40 2024-04-21T08:28:37.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.815+0000 7efd377fe700 1 -- 172.21.15.43:0/1279206897 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd28011af0 con 0x7efd3807de40 2024-04-21T08:28:37.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.815+0000 7efd3eb71700 1 -- 172.21.15.43:0/1279206897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd3807de40 msgr2=0x7efd3807c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:37.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.815+0000 7efd3eb71700 1 --2- 172.21.15.43:0/1279206897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd3807de40 0x7efd3807c2f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efd280098a0 tx=0x7efd28009bb0 comp rx=0 tx=0).stop 2024-04-21T08:28:37.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.815+0000 7efd3eb71700 1 -- 172.21.15.43:0/1279206897 shutdown_connections 2024-04-21T08:28:37.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.815+0000 7efd3eb71700 1 --2- 172.21.15.43:0/1279206897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd3807de40 0x7efd3807c2f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:37.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.815+0000 7efd3eb71700 1 -- 172.21.15.43:0/1279206897 >> 172.21.15.43:0/1279206897 conn(0x7efd38103af0 msgr2=0x7efd38105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:37.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.815+0000 7efd3eb71700 1 -- 172.21.15.43:0/1279206897 shutdown_connections 2024-04-21T08:28:37.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.815+0000 7efd3eb71700 1 -- 172.21.15.43:0/1279206897 wait complete. 2024-04-21T08:28:37.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.816+0000 7efd3eb71700 1 Processor -- start 2024-04-21T08:28:37.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.816+0000 7efd3eb71700 1 -- start start 2024-04-21T08:28:37.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.816+0000 7efd3eb71700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd3807de40 0x7efd3819e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:37.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.816+0000 7efd3eb71700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efd3810e930 con 0x7efd3807de40 2024-04-21T08:28:37.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.816+0000 7efd3c90d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd3807de40 0x7efd3819e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:37.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.816+0000 7efd3c90d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd3807de40 0x7efd3819e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53822/0 (socket says 172.21.15.43:53822) 2024-04-21T08:28:37.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.816+0000 7efd3c90d700 1 -- 172.21.15.43:0/2159811200 learned_addr learned my addr 172.21.15.43:0/2159811200 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:37.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.816+0000 7efd3c90d700 1 -- 172.21.15.43:0/2159811200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3819e900 con 0x7efd3807de40 2024-04-21T08:28:37.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.817+0000 7efd3c90d700 1 --2- 172.21.15.43:0/2159811200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd3807de40 0x7efd3819e3c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efd28003f40 tx=0x7efd28011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:37.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.817+0000 7efd35ffb700 1 -- 172.21.15.43:0/2159811200 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd280038b0 con 0x7efd3807de40 2024-04-21T08:28:37.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.817+0000 7efd35ffb700 1 -- 172.21.15.43:0/2159811200 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efd28003a50 con 0x7efd3807de40 2024-04-21T08:28:37.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.817+0000 7efd3eb71700 1 -- 172.21.15.43:0/2159811200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd3819eb90 con 0x7efd3807de40 2024-04-21T08:28:37.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.817+0000 7efd35ffb700 1 -- 172.21.15.43:0/2159811200 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd2801a5c0 con 0x7efd3807de40 2024-04-21T08:28:37.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.817+0000 7efd3eb71700 1 -- 172.21.15.43:0/2159811200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd381a1880 con 0x7efd3807de40 2024-04-21T08:28:37.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.818+0000 7efd35ffb700 1 -- 172.21.15.43:0/2159811200 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7efd2801a8c0 con 0x7efd3807de40 2024-04-21T08:28:37.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.818+0000 7efd3eb71700 1 -- 172.21.15.43:0/2159811200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd380640a0 con 0x7efd3807de40 2024-04-21T08:28:37.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.818+0000 7efd35ffb700 1 --2- 172.21.15.43:0/2159811200 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7efd200395d0 0x7efd2003ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:37.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.818+0000 7efd35ffb700 1 -- 172.21.15.43:0/2159811200 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efd2804da90 con 0x7efd3807de40 2024-04-21T08:28:37.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.818+0000 7efd37fff700 1 --2- 172.21.15.43:0/2159811200 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7efd200395d0 0x7efd2003ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:37.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.819+0000 7efd37fff700 1 --2- 172.21.15.43:0/2159811200 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7efd200395d0 0x7efd2003ba90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efd2c006fd0 tx=0x7efd2c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:37.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:36.824+0000 7efd35ffb700 1 -- 172.21.15.43:0/2159811200 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd2804d4b0 con 0x7efd3807de40 2024-04-21T08:28:37.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.119+0000 7efd3eb71700 1 -- 172.21.15.43:0/2159811200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7efd381a1a50 con 0x7efd3807de40 2024-04-21T08:28:37.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.120+0000 7efd35ffb700 1 -- 172.21.15.43:0/2159811200 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7efd381a1a50 con 0x7efd3807de40 2024-04-21T08:28:37.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.122+0000 7efd3eb71700 1 -- 172.21.15.43:0/2159811200 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7efd200395d0 msgr2=0x7efd2003ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:37.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.122+0000 7efd3eb71700 1 --2- 172.21.15.43:0/2159811200 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7efd200395d0 0x7efd2003ba90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efd2c006fd0 tx=0x7efd2c006e40 comp rx=0 tx=0).stop 2024-04-21T08:28:37.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.122+0000 7efd3eb71700 1 -- 172.21.15.43:0/2159811200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd3807de40 msgr2=0x7efd3819e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:37.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.122+0000 7efd3eb71700 1 --2- 172.21.15.43:0/2159811200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd3807de40 0x7efd3819e3c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efd28003f40 tx=0x7efd28011890 comp rx=0 tx=0).stop 2024-04-21T08:28:37.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.122+0000 7efd3eb71700 1 -- 172.21.15.43:0/2159811200 shutdown_connections 2024-04-21T08:28:37.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.122+0000 7efd3eb71700 1 --2- 172.21.15.43:0/2159811200 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7efd200395d0 0x7efd2003ba90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:37.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.122+0000 7efd3eb71700 1 --2- 172.21.15.43:0/2159811200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd3807de40 0x7efd3819e3c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:37.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.122+0000 7efd3eb71700 1 -- 172.21.15.43:0/2159811200 >> 172.21.15.43:0/2159811200 conn(0x7efd38103af0 msgr2=0x7efd38104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:37.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.122+0000 7efd3eb71700 1 -- 172.21.15.43:0/2159811200 shutdown_connections 2024-04-21T08:28:37.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.122+0000 7efd3eb71700 1 -- 172.21.15.43:0/2159811200 wait complete. 2024-04-21T08:28:37.737 INFO:teuthology.orchestra.run.smithi043.stdout:mgr is available 2024-04-21T08:28:37.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:37 smithi043 ceph-mon[27978]: mgrmap e4: x(active, since 2s) 2024-04-21T08:28:37.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:37 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2159811200' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-21T08:28:38.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:38 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3402471524' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-21T08:28:38.972 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:38.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [global] 2024-04-21T08:28:38.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout fsid = d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:28:38.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-21T08:28:38.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-21T08:28:38.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-21T08:28:38.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-21T08:28:38.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-21T08:28:38.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:38.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-21T08:28:38.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-21T08:28:38.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:38.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [osd] 2024-04-21T08:28:38.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-21T08:28:38.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-21T08:28:38.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-21T08:28:38.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-21T08:28:38.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-21T08:28:38.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.153+0000 7f13bb037700 1 Processor -- start 2024-04-21T08:28:38.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.153+0000 7f13bb037700 1 -- start start 2024-04-21T08:28:38.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.153+0000 7f13bb037700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13b41084b0 0x7f13b41088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:38.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.153+0000 7f13bb037700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f13b4108ea0 con 0x7f13b41084b0 2024-04-21T08:28:38.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.153+0000 7f13b8dd3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13b41084b0 0x7f13b41088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:38.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.153+0000 7f13b8dd3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13b41084b0 0x7f13b41088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53830/0 (socket says 172.21.15.43:53830) 2024-04-21T08:28:38.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.153+0000 7f13b8dd3700 1 -- 172.21.15.43:0/3715763720 learned_addr learned my addr 172.21.15.43:0/3715763720 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:38.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.154+0000 7f13b8dd3700 1 -- 172.21.15.43:0/3715763720 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b4109660 con 0x7f13b41084b0 2024-04-21T08:28:38.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.154+0000 7f13b8dd3700 1 --2- 172.21.15.43:0/3715763720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13b41084b0 0x7f13b41088d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f13a80098a0 tx=0x7f13a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0c0da5f27ab003 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:38.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.154+0000 7f13b37fe700 1 -- 172.21.15.43:0/3715763720 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13a8004160 con 0x7f13b41084b0 2024-04-21T08:28:38.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.154+0000 7f13b37fe700 1 -- 172.21.15.43:0/3715763720 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f13a8004300 con 0x7f13b41084b0 2024-04-21T08:28:38.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.154+0000 7f13b37fe700 1 -- 172.21.15.43:0/3715763720 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13a8011af0 con 0x7f13b41084b0 2024-04-21T08:28:38.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.154+0000 7f13bb037700 1 -- 172.21.15.43:0/3715763720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13b41084b0 msgr2=0x7f13b41088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:38.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.154+0000 7f13bb037700 1 --2- 172.21.15.43:0/3715763720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13b41084b0 0x7f13b41088d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f13a80098a0 tx=0x7f13a8009bb0 comp rx=0 tx=0).stop 2024-04-21T08:28:38.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.154+0000 7f13bb037700 1 -- 172.21.15.43:0/3715763720 shutdown_connections 2024-04-21T08:28:38.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.154+0000 7f13bb037700 1 --2- 172.21.15.43:0/3715763720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13b41084b0 0x7f13b41088d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:38.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.154+0000 7f13bb037700 1 -- 172.21.15.43:0/3715763720 >> 172.21.15.43:0/3715763720 conn(0x7f13b4103af0 msgr2=0x7f13b4105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:38.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.155+0000 7f13bb037700 1 -- 172.21.15.43:0/3715763720 shutdown_connections 2024-04-21T08:28:38.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.155+0000 7f13bb037700 1 -- 172.21.15.43:0/3715763720 wait complete. 2024-04-21T08:28:38.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.155+0000 7f13bb037700 1 Processor -- start 2024-04-21T08:28:38.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.155+0000 7f13bb037700 1 -- start start 2024-04-21T08:28:38.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.155+0000 7f13bb037700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13b41084b0 0x7f13b419c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:38.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.155+0000 7f13bb037700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f13b410c890 con 0x7f13b41084b0 2024-04-21T08:28:38.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.155+0000 7f13b8dd3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13b41084b0 0x7f13b419c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:38.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.155+0000 7f13b8dd3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13b41084b0 0x7f13b419c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53836/0 (socket says 172.21.15.43:53836) 2024-04-21T08:28:38.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.155+0000 7f13b8dd3700 1 -- 172.21.15.43:0/3402471524 learned_addr learned my addr 172.21.15.43:0/3402471524 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:38.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.156+0000 7f13b8dd3700 1 -- 172.21.15.43:0/3402471524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b419c740 con 0x7f13b41084b0 2024-04-21T08:28:38.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.156+0000 7f13b8dd3700 1 --2- 172.21.15.43:0/3402471524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13b41084b0 0x7f13b419c200 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f13a8003f40 tx=0x7f13a8011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:38.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.156+0000 7f13b1ffb700 1 -- 172.21.15.43:0/3402471524 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13a80038b0 con 0x7f13b41084b0 2024-04-21T08:28:38.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.156+0000 7f13b1ffb700 1 -- 172.21.15.43:0/3402471524 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f13a8003a50 con 0x7f13b41084b0 2024-04-21T08:28:38.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.156+0000 7f13bb037700 1 -- 172.21.15.43:0/3402471524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b419c9d0 con 0x7f13b41084b0 2024-04-21T08:28:38.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.156+0000 7f13b1ffb700 1 -- 172.21.15.43:0/3402471524 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13a801a5c0 con 0x7f13b41084b0 2024-04-21T08:28:38.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.156+0000 7f13bb037700 1 -- 172.21.15.43:0/3402471524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b419f6c0 con 0x7f13b41084b0 2024-04-21T08:28:38.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.157+0000 7f13b1ffb700 1 -- 172.21.15.43:0/3402471524 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f13a8024430 con 0x7f13b41084b0 2024-04-21T08:28:38.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.157+0000 7f13b1ffb700 1 --2- 172.21.15.43:0/3402471524 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f13a4039620 0x7f13a403bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:38.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.157+0000 7f13b1ffb700 1 -- 172.21.15.43:0/3402471524 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f13a804da30 con 0x7f13b41084b0 2024-04-21T08:28:38.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.157+0000 7f13bb037700 1 -- 172.21.15.43:0/3402471524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13b40640a0 con 0x7f13b41084b0 2024-04-21T08:28:38.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.158+0000 7f13b3fff700 1 --2- 172.21.15.43:0/3402471524 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f13a4039620 0x7f13a403bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:38.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.158+0000 7f13b3fff700 1 --2- 172.21.15.43:0/3402471524 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f13a4039620 0x7f13a403bae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f13a0006fd0 tx=0x7f13a0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:38.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.163+0000 7f13b1ffb700 1 -- 172.21.15.43:0/3402471524 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f13a804d450 con 0x7f13b41084b0 2024-04-21T08:28:38.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.369+0000 7f13bb037700 1 -- 172.21.15.43:0/3402471524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f13b41a0e10 con 0x7f13b41084b0 2024-04-21T08:28:38.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.370+0000 7f13b1ffb700 1 -- 172.21.15.43:0/3402471524 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f13a8032020 con 0x7f13b41084b0 2024-04-21T08:28:38.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.373+0000 7f13bb037700 1 -- 172.21.15.43:0/3402471524 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f13a4039620 msgr2=0x7f13a403bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:38.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.373+0000 7f13bb037700 1 --2- 172.21.15.43:0/3402471524 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f13a4039620 0x7f13a403bae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f13a0006fd0 tx=0x7f13a0006e40 comp rx=0 tx=0).stop 2024-04-21T08:28:38.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.373+0000 7f13bb037700 1 -- 172.21.15.43:0/3402471524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13b41084b0 msgr2=0x7f13b419c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:38.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.373+0000 7f13bb037700 1 --2- 172.21.15.43:0/3402471524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13b41084b0 0x7f13b419c200 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f13a8003f40 tx=0x7f13a8011890 comp rx=0 tx=0).stop 2024-04-21T08:28:38.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.373+0000 7f13bb037700 1 -- 172.21.15.43:0/3402471524 shutdown_connections 2024-04-21T08:28:38.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.373+0000 7f13bb037700 1 --2- 172.21.15.43:0/3402471524 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f13a4039620 0x7f13a403bae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:38.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.373+0000 7f13bb037700 1 --2- 172.21.15.43:0/3402471524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13b41084b0 0x7f13b419c200 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:38.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.373+0000 7f13bb037700 1 -- 172.21.15.43:0/3402471524 >> 172.21.15.43:0/3402471524 conn(0x7f13b4103af0 msgr2=0x7f13b4104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:38.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.373+0000 7f13bb037700 1 -- 172.21.15.43:0/3402471524 shutdown_connections 2024-04-21T08:28:38.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:38.373+0000 7f13bb037700 1 -- 172.21.15.43:0/3402471524 wait complete. 2024-04-21T08:28:38.978 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling cephadm module... 2024-04-21T08:28:40.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:39 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/927475544' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-21T08:28:40.035 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:39 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: ignoring --setuser ceph since I am not root 2024-04-21T08:28:40.035 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:39 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: ignoring --setgroup ceph since I am not root 2024-04-21T08:28:40.035 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:39 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:39.901+0000 7f465128b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:28:40.310 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:40 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:40.094+0000 7f465128b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:28:40.310 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:40 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:40.200+0000 7f465128b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:28:40.322 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.398+0000 7f250679a700 1 Processor -- start 2024-04-21T08:28:40.322 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.398+0000 7f250679a700 1 -- start start 2024-04-21T08:28:40.323 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.398+0000 7f250679a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2500108e70 0x7f250010b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:40.323 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.398+0000 7f250679a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f250007bea0 con 0x7f2500108e70 2024-04-21T08:28:40.323 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.398+0000 7f24fffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2500108e70 0x7f250010b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:40.323 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.398+0000 7f24fffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2500108e70 0x7f250010b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53838/0 (socket says 172.21.15.43:53838) 2024-04-21T08:28:40.323 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.398+0000 7f24fffff700 1 -- 172.21.15.43:0/1573261610 learned_addr learned my addr 172.21.15.43:0/1573261610 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:40.323 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.398+0000 7f24fffff700 1 -- 172.21.15.43:0/1573261610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250010b7a0 con 0x7f2500108e70 2024-04-21T08:28:40.323 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.399+0000 7f24fffff700 1 --2- 172.21.15.43:0/1573261610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2500108e70 0x7f250010b260 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f24f00098a0 tx=0x7f24f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110f0c3ea19dd8b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:40.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.399+0000 7f24feffd700 1 -- 172.21.15.43:0/1573261610 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24f0004160 con 0x7f2500108e70 2024-04-21T08:28:40.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.399+0000 7f24feffd700 1 -- 172.21.15.43:0/1573261610 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f24f0004300 con 0x7f2500108e70 2024-04-21T08:28:40.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.399+0000 7f24feffd700 1 -- 172.21.15.43:0/1573261610 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24f0011af0 con 0x7f2500108e70 2024-04-21T08:28:40.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.399+0000 7f250679a700 1 -- 172.21.15.43:0/1573261610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2500108e70 msgr2=0x7f250010b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:40.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.399+0000 7f250679a700 1 --2- 172.21.15.43:0/1573261610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2500108e70 0x7f250010b260 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f24f00098a0 tx=0x7f24f0009bb0 comp rx=0 tx=0).stop 2024-04-21T08:28:40.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.399+0000 7f250679a700 1 -- 172.21.15.43:0/1573261610 shutdown_connections 2024-04-21T08:28:40.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.399+0000 7f250679a700 1 --2- 172.21.15.43:0/1573261610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2500108e70 0x7f250010b260 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:40.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.399+0000 7f250679a700 1 -- 172.21.15.43:0/1573261610 >> 172.21.15.43:0/1573261610 conn(0x7f25001028c0 msgr2=0x7f2500104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:40.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.399+0000 7f250679a700 1 -- 172.21.15.43:0/1573261610 shutdown_connections 2024-04-21T08:28:40.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.399+0000 7f250679a700 1 -- 172.21.15.43:0/1573261610 wait complete. 2024-04-21T08:28:40.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.400+0000 7f250679a700 1 Processor -- start 2024-04-21T08:28:40.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.400+0000 7f250679a700 1 -- start start 2024-04-21T08:28:40.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.400+0000 7f250679a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2500108e70 0x7f25001086f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:40.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.400+0000 7f250679a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f250010c700 con 0x7f2500108e70 2024-04-21T08:28:40.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.400+0000 7f24fffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2500108e70 0x7f25001086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:40.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.401+0000 7f24fffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2500108e70 0x7f25001086f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53854/0 (socket says 172.21.15.43:53854) 2024-04-21T08:28:40.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.401+0000 7f24fffff700 1 -- 172.21.15.43:0/927475544 learned_addr learned my addr 172.21.15.43:0/927475544 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:40.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.401+0000 7f24fffff700 1 -- 172.21.15.43:0/927475544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2500108c30 con 0x7f2500108e70 2024-04-21T08:28:40.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.401+0000 7f24fffff700 1 --2- 172.21.15.43:0/927475544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2500108e70 0x7f25001086f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009fb0 tx=0x7f24f0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:40.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.401+0000 7f24fd7fa700 1 -- 172.21.15.43:0/927475544 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24f00036a0 con 0x7f2500108e70 2024-04-21T08:28:40.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.401+0000 7f24fd7fa700 1 -- 172.21.15.43:0/927475544 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f24f001a430 con 0x7f2500108e70 2024-04-21T08:28:40.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.401+0000 7f250679a700 1 -- 172.21.15.43:0/927475544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2500106e20 con 0x7f2500108e70 2024-04-21T08:28:40.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.401+0000 7f24fd7fa700 1 -- 172.21.15.43:0/927475544 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24f0019550 con 0x7f2500108e70 2024-04-21T08:28:40.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.401+0000 7f250679a700 1 -- 172.21.15.43:0/927475544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2500107360 con 0x7f2500108e70 2024-04-21T08:28:40.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.402+0000 7f24fd7fa700 1 -- 172.21.15.43:0/927475544 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f24f0019850 con 0x7f2500108e70 2024-04-21T08:28:40.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.402+0000 7f250679a700 1 -- 172.21.15.43:0/927475544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25000640a0 con 0x7f2500108e70 2024-04-21T08:28:40.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.402+0000 7f24fd7fa700 1 --2- 172.21.15.43:0/927475544 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f24e80395d0 0x7f24e803ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:40.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.402+0000 7f24fd7fa700 1 -- 172.21.15.43:0/927475544 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f24f004da50 con 0x7f2500108e70 2024-04-21T08:28:40.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.402+0000 7f24ff7fe700 1 --2- 172.21.15.43:0/927475544 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f24e80395d0 0x7f24e803ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:40.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.403+0000 7f24ff7fe700 1 --2- 172.21.15.43:0/927475544 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f24e80395d0 0x7f24e803ba90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f24f4006fd0 tx=0x7f24f4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:40.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.407+0000 7f24fd7fa700 1 -- 172.21.15.43:0/927475544 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f24f004d470 con 0x7f2500108e70 2024-04-21T08:28:40.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.673+0000 7f250679a700 1 -- 172.21.15.43:0/927475544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f2500107b50 con 0x7f2500108e70 2024-04-21T08:28:40.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.724+0000 7f24fd7fa700 1 -- 172.21.15.43:0/927475544 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f24f000bde0 con 0x7f2500108e70 2024-04-21T08:28:40.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.726+0000 7f24fd7fa700 1 -- 172.21.15.43:0/927475544 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f24f002d960 con 0x7f2500108e70 2024-04-21T08:28:40.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.727+0000 7f250679a700 1 -- 172.21.15.43:0/927475544 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f24e80395d0 msgr2=0x7f24e803ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:40.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.727+0000 7f250679a700 1 --2- 172.21.15.43:0/927475544 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f24e80395d0 0x7f24e803ba90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f24f4006fd0 tx=0x7f24f4006e40 comp rx=0 tx=0).stop 2024-04-21T08:28:40.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.727+0000 7f250679a700 1 -- 172.21.15.43:0/927475544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2500108e70 msgr2=0x7f25001086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:40.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.727+0000 7f250679a700 1 --2- 172.21.15.43:0/927475544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2500108e70 0x7f25001086f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009fb0 tx=0x7f24f0011940 comp rx=0 tx=0).stop 2024-04-21T08:28:40.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.727+0000 7f250679a700 1 -- 172.21.15.43:0/927475544 shutdown_connections 2024-04-21T08:28:40.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.727+0000 7f250679a700 1 --2- 172.21.15.43:0/927475544 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f24e80395d0 0x7f24e803ba90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:40.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.727+0000 7f250679a700 1 --2- 172.21.15.43:0/927475544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2500108e70 0x7f25001086f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:40.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.727+0000 7f250679a700 1 -- 172.21.15.43:0/927475544 >> 172.21.15.43:0/927475544 conn(0x7f25001028c0 msgr2=0x7f2500103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:40.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.727+0000 7f250679a700 1 -- 172.21.15.43:0/927475544 shutdown_connections 2024-04-21T08:28:40.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:39.727+0000 7f250679a700 1 -- 172.21.15.43:0/927475544 wait complete. 2024-04-21T08:28:40.561 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:40 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:40.527+0000 7f465128b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:28:40.946 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:40 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/927475544' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-21T08:28:40.946 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:40 smithi043 ceph-mon[27978]: mgrmap e5: x(active, since 5s) 2024-04-21T08:28:40.946 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:40 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:40.616+0000 7f465128b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:28:41.566 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:28:41.566 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-21T08:28:41.567 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-21T08:28:41.567 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-21T08:28:41.567 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-21T08:28:41.567 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:28:41.567 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.740+0000 7f4048567700 1 Processor -- start 2024-04-21T08:28:41.567 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.740+0000 7f4048567700 1 -- start start 2024-04-21T08:28:41.567 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.741+0000 7f4048567700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4040108e20 0x7f404010b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:41.567 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.741+0000 7f4048567700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f404007be40 con 0x7f4040108e20 2024-04-21T08:28:41.567 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.741+0000 7f4046303700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4040108e20 0x7f404010b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:41.567 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.741+0000 7f4046303700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4040108e20 0x7f404010b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53888/0 (socket says 172.21.15.43:53888) 2024-04-21T08:28:41.567 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.741+0000 7f4046303700 1 -- 172.21.15.43:0/3507511148 learned_addr learned my addr 172.21.15.43:0/3507511148 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:41.567 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.741+0000 7f4046303700 1 -- 172.21.15.43:0/3507511148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404010b750 con 0x7f4040108e20 2024-04-21T08:28:41.567 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.741+0000 7f4046303700 1 --2- 172.21.15.43:0/3507511148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4040108e20 0x7f404010b210 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f40340098a0 tx=0x7f4034009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa0462e283f7da1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:41.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.742+0000 7f4045301700 1 -- 172.21.15.43:0/3507511148 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4034004160 con 0x7f4040108e20 2024-04-21T08:28:41.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.742+0000 7f4045301700 1 -- 172.21.15.43:0/3507511148 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4034004300 con 0x7f4040108e20 2024-04-21T08:28:41.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.742+0000 7f4045301700 1 -- 172.21.15.43:0/3507511148 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4034011af0 con 0x7f4040108e20 2024-04-21T08:28:41.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.742+0000 7f4048567700 1 -- 172.21.15.43:0/3507511148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4040108e20 msgr2=0x7f404010b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:41.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.742+0000 7f4048567700 1 --2- 172.21.15.43:0/3507511148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4040108e20 0x7f404010b210 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f40340098a0 tx=0x7f4034009bb0 comp rx=0 tx=0).stop 2024-04-21T08:28:41.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.742+0000 7f4048567700 1 -- 172.21.15.43:0/3507511148 shutdown_connections 2024-04-21T08:28:41.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.742+0000 7f4048567700 1 --2- 172.21.15.43:0/3507511148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4040108e20 0x7f404010b210 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:41.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.742+0000 7f4048567700 1 -- 172.21.15.43:0/3507511148 >> 172.21.15.43:0/3507511148 conn(0x7f40401027d0 msgr2=0x7f4040104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:41.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.742+0000 7f4048567700 1 -- 172.21.15.43:0/3507511148 shutdown_connections 2024-04-21T08:28:41.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.742+0000 7f4048567700 1 -- 172.21.15.43:0/3507511148 wait complete. 2024-04-21T08:28:41.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.743+0000 7f4048567700 1 Processor -- start 2024-04-21T08:28:41.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.743+0000 7f4048567700 1 -- start start 2024-04-21T08:28:41.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.743+0000 7f4048567700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4040108e20 0x7f404019c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:41.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.743+0000 7f4048567700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f404010c6c0 con 0x7f4040108e20 2024-04-21T08:28:41.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.743+0000 7f4046303700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4040108e20 0x7f404019c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:41.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.743+0000 7f4046303700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4040108e20 0x7f404019c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53904/0 (socket says 172.21.15.43:53904) 2024-04-21T08:28:41.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.743+0000 7f4046303700 1 -- 172.21.15.43:0/3812969606 learned_addr learned my addr 172.21.15.43:0/3812969606 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:41.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.744+0000 7f4046303700 1 -- 172.21.15.43:0/3812969606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404019c720 con 0x7f4040108e20 2024-04-21T08:28:41.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.744+0000 7f4046303700 1 --2- 172.21.15.43:0/3812969606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4040108e20 0x7f404019c1e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4034003f40 tx=0x7f4034011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:41.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.744+0000 7f40337fe700 1 -- 172.21.15.43:0/3812969606 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40340038b0 con 0x7f4040108e20 2024-04-21T08:28:41.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.744+0000 7f40337fe700 1 -- 172.21.15.43:0/3812969606 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4034003a50 con 0x7f4040108e20 2024-04-21T08:28:41.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.744+0000 7f40337fe700 1 -- 172.21.15.43:0/3812969606 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f403401a5c0 con 0x7f4040108e20 2024-04-21T08:28:41.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.744+0000 7f4048567700 1 -- 172.21.15.43:0/3812969606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404019c9b0 con 0x7f4040108e20 2024-04-21T08:28:41.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.744+0000 7f4048567700 1 -- 172.21.15.43:0/3812969606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f404019f6a0 con 0x7f4040108e20 2024-04-21T08:28:41.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.745+0000 7f40337fe700 1 -- 172.21.15.43:0/3812969606 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f4034024430 con 0x7f4040108e20 2024-04-21T08:28:41.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.745+0000 7f4048567700 1 -- 172.21.15.43:0/3812969606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40400640a0 con 0x7f4040108e20 2024-04-21T08:28:41.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.745+0000 7f40337fe700 1 --2- 172.21.15.43:0/3812969606 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f402c039620 0x7f402c03bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:41.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.745+0000 7f40337fe700 1 -- 172.21.15.43:0/3812969606 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f403404dad0 con 0x7f4040108e20 2024-04-21T08:28:41.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.745+0000 7f4045b02700 1 -- 172.21.15.43:0/3812969606 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f402c039620 msgr2=0x7f402c03bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1559586579 2024-04-21T08:28:41.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.745+0000 7f4045b02700 1 --2- 172.21.15.43:0/3812969606 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f402c039620 0x7f402c03bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:28:41.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.750+0000 7f40337fe700 1 -- 172.21.15.43:0/3812969606 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f403404d4f0 con 0x7f4040108e20 2024-04-21T08:28:41.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.946+0000 7f4045b02700 1 -- 172.21.15.43:0/3812969606 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f402c039620 msgr2=0x7f402c03bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1559586579 2024-04-21T08:28:41.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:40.946+0000 7f4045b02700 1 --2- 172.21.15.43:0/3812969606 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f402c039620 0x7f402c03bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:28:41.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.034+0000 7f4048567700 1 -- 172.21.15.43:0/3812969606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f404019f870 con 0x7f4040108e20 2024-04-21T08:28:41.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.035+0000 7f40337fe700 1 -- 172.21.15.43:0/3812969606 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f404019f870 con 0x7f4040108e20 2024-04-21T08:28:41.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.039+0000 7f4048567700 1 -- 172.21.15.43:0/3812969606 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f402c039620 msgr2=0x7f402c03bae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:28:41.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.039+0000 7f4048567700 1 --2- 172.21.15.43:0/3812969606 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f402c039620 0x7f402c03bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:41.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.039+0000 7f4048567700 1 -- 172.21.15.43:0/3812969606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4040108e20 msgr2=0x7f404019c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:41.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.039+0000 7f4048567700 1 --2- 172.21.15.43:0/3812969606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4040108e20 0x7f404019c1e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4034003f40 tx=0x7f4034011890 comp rx=0 tx=0).stop 2024-04-21T08:28:41.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.039+0000 7f4048567700 1 -- 172.21.15.43:0/3812969606 shutdown_connections 2024-04-21T08:28:41.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.039+0000 7f4048567700 1 --2- 172.21.15.43:0/3812969606 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f402c039620 0x7f402c03bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:41.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.039+0000 7f4048567700 1 --2- 172.21.15.43:0/3812969606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4040108e20 0x7f404019c1e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:41.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.039+0000 7f4048567700 1 -- 172.21.15.43:0/3812969606 >> 172.21.15.43:0/3812969606 conn(0x7f40401027d0 msgr2=0x7f4040103270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:41.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.040+0000 7f4048567700 1 -- 172.21.15.43:0/3812969606 shutdown_connections 2024-04-21T08:28:41.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.040+0000 7f4048567700 1 -- 172.21.15.43:0/3812969606 wait complete. 2024-04-21T08:28:41.571 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for the mgr to restart... 2024-04-21T08:28:41.571 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr epoch 5... 2024-04-21T08:28:41.977 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:41 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3812969606' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-21T08:28:41.978 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:41 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:41.880+0000 7f465128b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:28:42.239 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:42 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:42.062+0000 7f465128b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:28:42.239 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:42 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:42.239+0000 7f465128b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:28:42.811 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:42 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:42.542+0000 7f465128b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:28:42.811 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:42 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:42.639+0000 7f465128b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:28:43.311 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:42 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:42.849+0000 7f465128b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:28:43.311 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:43 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:43.050+0000 7f465128b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:28:44.061 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:43 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:43.680+0000 7f465128b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:28:44.413 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:44 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:44.121+0000 7f465128b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:28:44.413 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:44 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:44.221+0000 7f465128b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:28:44.811 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:44 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:44.434+0000 7f465128b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:28:45.311 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:44 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:44.979+0000 7f465128b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:28:45.311 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:45 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:45.075+0000 7f465128b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:28:45.687 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:45 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:45.330+0000 7f465128b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:28:45.687 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:45 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:45.468+0000 7f465128b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:28:45.687 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:45 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:45.567+0000 7f465128b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:28:45.687 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:45 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:45.687+0000 7f465128b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:28:46.311 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:46 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:45.999+0000 7f465128b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:28:46.905 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:46 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:46.627+0000 7f465128b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:28:46.905 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:46 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:28:46.852+0000 7f465128b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:28:47.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:46 smithi043 ceph-mon[27978]: Active manager daemon x restarted 2024-04-21T08:28:47.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:46 smithi043 ceph-mon[27978]: Activating manager daemon x 2024-04-21T08:28:47.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:46 smithi043 ceph-mon[27978]: osdmap e2: 0 total, 0 up, 0 in 2024-04-21T08:28:47.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:46 smithi043 ceph-mon[27978]: mgrmap e6: x(active, starting, since 0.00337544s) 2024-04-21T08:28:47.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:46 smithi043 ceph-mon[27978]: from='mgr.14118 172.21.15.43:0/1296813352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:28:47.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:46 smithi043 ceph-mon[27978]: from='mgr.14118 172.21.15.43:0/1296813352' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T08:28:47.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:46 smithi043 ceph-mon[27978]: from='mgr.14118 172.21.15.43:0/1296813352' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T08:28:47.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:46 smithi043 ceph-mon[27978]: from='mgr.14118 172.21.15.43:0/1296813352' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T08:28:47.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:46 smithi043 ceph-mon[27978]: from='mgr.14118 172.21.15.43:0/1296813352' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T08:28:47.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:46 smithi043 ceph-mon[27978]: Manager daemon x is now available 2024-04-21T08:28:47.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:46 smithi043 ceph-mon[27978]: from='mgr.14118 172.21.15.43:0/1296813352' entity='mgr.x' 2024-04-21T08:28:47.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:46 smithi043 ceph-mon[27978]: from='mgr.14118 172.21.15.43:0/1296813352' entity='mgr.x' 2024-04-21T08:28:47.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:46 smithi043 ceph-mon[27978]: from='mgr.14118 172.21.15.43:0/1296813352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:28:47.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:46 smithi043 ceph-mon[27978]: from='mgr.14118 172.21.15.43:0/1296813352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:28:47.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:46 smithi043 ceph-mon[27978]: from='mgr.14118 172.21.15.43:0/1296813352' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T08:28:48.214 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:47 smithi043 ceph-mon[27978]: Found migration_current of "None". Setting to last migration. 2024-04-21T08:28:48.214 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:47 smithi043 ceph-mon[27978]: from='mgr.14118 172.21.15.43:0/1296813352' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T08:28:48.214 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:47 smithi043 ceph-mon[27978]: from='mgr.14118 172.21.15.43:0/1296813352' entity='mgr.x' 2024-04-21T08:28:48.214 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:47 smithi043 ceph-mon[27978]: from='mgr.14118 172.21.15.43:0/1296813352' entity='mgr.x' 2024-04-21T08:28:48.214 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:47 smithi043 ceph-mon[27978]: mgrmap e7: x(active, since 1.00558s) 2024-04-21T08:28:48.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:28:48.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-21T08:28:48.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-21T08:28:48.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:28:48.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.980+0000 7f845947e700 1 Processor -- start 2024-04-21T08:28:48.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.980+0000 7f845947e700 1 -- start start 2024-04-21T08:28:48.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.980+0000 7f845947e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84541072f0 0x7f8454107710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:48.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.980+0000 7f845947e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8454107ce0 con 0x7f84541072f0 2024-04-21T08:28:48.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.980+0000 7f8452ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84541072f0 0x7f8454107710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:48.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.981+0000 7f8452ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84541072f0 0x7f8454107710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53908/0 (socket says 172.21.15.43:53908) 2024-04-21T08:28:48.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.981+0000 7f8452ffd700 1 -- 172.21.15.43:0/481572184 learned_addr learned my addr 172.21.15.43:0/481572184 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:48.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.981+0000 7f8452ffd700 1 -- 172.21.15.43:0/481572184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84541084f0 con 0x7f84541072f0 2024-04-21T08:28:48.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.981+0000 7f8452ffd700 1 --2- 172.21.15.43:0/481572184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84541072f0 0x7f8454107710 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f84480098a0 tx=0x7f8448009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61cf54355e2e5f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:48.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.981+0000 7f8451ffb700 1 -- 172.21.15.43:0/481572184 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8448004160 con 0x7f84541072f0 2024-04-21T08:28:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.981+0000 7f8451ffb700 1 -- 172.21.15.43:0/481572184 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8448004300 con 0x7f84541072f0 2024-04-21T08:28:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.981+0000 7f8451ffb700 1 -- 172.21.15.43:0/481572184 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8448011af0 con 0x7f84541072f0 2024-04-21T08:28:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.982+0000 7f845947e700 1 -- 172.21.15.43:0/481572184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84541072f0 msgr2=0x7f8454107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.982+0000 7f845947e700 1 --2- 172.21.15.43:0/481572184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84541072f0 0x7f8454107710 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f84480098a0 tx=0x7f8448009bb0 comp rx=0 tx=0).stop 2024-04-21T08:28:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.982+0000 7f845947e700 1 -- 172.21.15.43:0/481572184 shutdown_connections 2024-04-21T08:28:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.982+0000 7f845947e700 1 --2- 172.21.15.43:0/481572184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84541072f0 0x7f8454107710 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.982+0000 7f845947e700 1 -- 172.21.15.43:0/481572184 >> 172.21.15.43:0/481572184 conn(0x7f8454102890 msgr2=0x7f8454104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.982+0000 7f845947e700 1 -- 172.21.15.43:0/481572184 shutdown_connections 2024-04-21T08:28:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.982+0000 7f845947e700 1 -- 172.21.15.43:0/481572184 wait complete. 2024-04-21T08:28:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.983+0000 7f845947e700 1 Processor -- start 2024-04-21T08:28:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.983+0000 7f845947e700 1 -- start start 2024-04-21T08:28:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.983+0000 7f845947e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84541072f0 0x7f845407ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.983+0000 7f845947e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f845410b720 con 0x7f84541072f0 2024-04-21T08:28:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.983+0000 7f8452ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84541072f0 0x7f845407ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.983+0000 7f8452ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84541072f0 0x7f845407ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53916/0 (socket says 172.21.15.43:53916) 2024-04-21T08:28:48.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.983+0000 7f8452ffd700 1 -- 172.21.15.43:0/3689063144 learned_addr learned my addr 172.21.15.43:0/3689063144 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:48.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.983+0000 7f8452ffd700 1 -- 172.21.15.43:0/3689063144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f845419efb0 con 0x7f84541072f0 2024-04-21T08:28:48.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.984+0000 7f8452ffd700 1 --2- 172.21.15.43:0/3689063144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84541072f0 0x7f845407ba40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8448003f40 tx=0x7f8448011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:48.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.984+0000 7f843bfff700 1 -- 172.21.15.43:0/3689063144 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84480038b0 con 0x7f84541072f0 2024-04-21T08:28:48.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.984+0000 7f845947e700 1 -- 172.21.15.43:0/3689063144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f845419f180 con 0x7f84541072f0 2024-04-21T08:28:48.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.984+0000 7f843bfff700 1 -- 172.21.15.43:0/3689063144 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8448003a50 con 0x7f84541072f0 2024-04-21T08:28:48.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.984+0000 7f843bfff700 1 -- 172.21.15.43:0/3689063144 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f844801a5c0 con 0x7f84541072f0 2024-04-21T08:28:48.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.984+0000 7f845947e700 1 -- 172.21.15.43:0/3689063144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f845419f520 con 0x7f84541072f0 2024-04-21T08:28:48.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.985+0000 7f843bfff700 1 -- 172.21.15.43:0/3689063144 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f844801a8c0 con 0x7f84541072f0 2024-04-21T08:28:48.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.985+0000 7f843bfff700 1 --2- 172.21.15.43:0/3689063144 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f843c039620 0x7f843c03bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:48.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.985+0000 7f845947e700 1 -- 172.21.15.43:0/3689063144 --> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8454069b50 con 0x7f843c039620 2024-04-21T08:28:48.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.985+0000 7f843bfff700 1 -- 172.21.15.43:0/3689063144 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f844804ddf0 con 0x7f84541072f0 2024-04-21T08:28:48.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.985+0000 7f84527fc700 1 -- 172.21.15.43:0/3689063144 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f843c039620 msgr2=0x7f843c03bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1559586579 2024-04-21T08:28:48.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.985+0000 7f84527fc700 1 --2- 172.21.15.43:0/3689063144 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f843c039620 0x7f843c03bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:28:48.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:42.186+0000 7f84527fc700 1 -- 172.21.15.43:0/3689063144 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f843c039620 msgr2=0x7f843c03bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1559586579 2024-04-21T08:28:48.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:42.186+0000 7f84527fc700 1 --2- 172.21.15.43:0/3689063144 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f843c039620 0x7f843c03bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:28:48.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:42.587+0000 7f84527fc700 1 -- 172.21.15.43:0/3689063144 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f843c039620 msgr2=0x7f843c03bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1559586579 2024-04-21T08:28:48.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:42.587+0000 7f84527fc700 1 --2- 172.21.15.43:0/3689063144 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f843c039620 0x7f843c03bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-21T08:28:48.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:43.388+0000 7f84527fc700 1 -- 172.21.15.43:0/3689063144 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f843c039620 msgr2=0x7f843c03bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1559586579 2024-04-21T08:28:48.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:43.388+0000 7f84527fc700 1 --2- 172.21.15.43:0/3689063144 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f843c039620 0x7f843c03bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-21T08:28:48.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.989+0000 7f84527fc700 1 -- 172.21.15.43:0/3689063144 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f843c039620 msgr2=0x7f843c03bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1559586579 2024-04-21T08:28:48.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.989+0000 7f84527fc700 1 --2- 172.21.15.43:0/3689063144 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f843c039620 0x7f843c03bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-21T08:28:48.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:46.856+0000 7f843bfff700 1 -- 172.21.15.43:0/3689063144 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f844802d910 con 0x7f84541072f0 2024-04-21T08:28:48.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:46.856+0000 7f843bfff700 1 -- 172.21.15.43:0/3689063144 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f843c039620 msgr2=0x7f843c03bae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:28:48.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:46.856+0000 7f843bfff700 1 --2- 172.21.15.43:0/3689063144 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f843c039620 0x7f843c03bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:48.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.858+0000 7f843bfff700 1 -- 172.21.15.43:0/3689063144 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f844804f980 con 0x7f84541072f0 2024-04-21T08:28:48.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.858+0000 7f843bfff700 1 --2- 172.21.15.43:0/3689063144 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f843c03cd20 0x7f843c03f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:48.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.858+0000 7f843bfff700 1 -- 172.21.15.43:0/3689063144 --> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8454069b50 con 0x7f843c03cd20 2024-04-21T08:28:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.859+0000 7f84527fc700 1 --2- 172.21.15.43:0/3689063144 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f843c03cd20 0x7f843c03f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.859+0000 7f84527fc700 1 --2- 172.21.15.43:0/3689063144 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f843c03cd20 0x7f843c03f110 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8444003a10 tx=0x7f8444009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.860+0000 7f843bfff700 1 -- 172.21.15.43:0/3689063144 <== mgr.14118 v2:172.21.15.43:6800/3200866696 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f8454069b50 con 0x7f843c03cd20 2024-04-21T08:28:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.867+0000 7f845947e700 1 -- 172.21.15.43:0/3689063144 --> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f845410b8a0 con 0x7f843c03cd20 2024-04-21T08:28:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.867+0000 7f843bfff700 1 -- 172.21.15.43:0/3689063144 <== mgr.14118 v2:172.21.15.43:6800/3200866696 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f845410b8a0 con 0x7f843c03cd20 2024-04-21T08:28:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.867+0000 7f845947e700 1 -- 172.21.15.43:0/3689063144 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f843c03cd20 msgr2=0x7f843c03f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.867+0000 7f845947e700 1 --2- 172.21.15.43:0/3689063144 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f843c03cd20 0x7f843c03f110 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8444003a10 tx=0x7f8444009290 comp rx=0 tx=0).stop 2024-04-21T08:28:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.867+0000 7f845947e700 1 -- 172.21.15.43:0/3689063144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84541072f0 msgr2=0x7f845407ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.867+0000 7f845947e700 1 --2- 172.21.15.43:0/3689063144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84541072f0 0x7f845407ba40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8448003f40 tx=0x7f8448011890 comp rx=0 tx=0).stop 2024-04-21T08:28:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.867+0000 7f845947e700 1 -- 172.21.15.43:0/3689063144 shutdown_connections 2024-04-21T08:28:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.867+0000 7f845947e700 1 --2- 172.21.15.43:0/3689063144 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f843c03cd20 0x7f843c03f110 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.867+0000 7f845947e700 1 --2- 172.21.15.43:0/3689063144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84541072f0 0x7f845407ba40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.867+0000 7f845947e700 1 --2- 172.21.15.43:0/3689063144 >> [v2:172.21.15.43:6800/1559586579,v1:172.21.15.43:6801/1559586579] conn(0x7f843c039620 0x7f843c03bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.867+0000 7f845947e700 1 -- 172.21.15.43:0/3689063144 >> 172.21.15.43:0/3689063144 conn(0x7f8454102890 msgr2=0x7f8454103330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.867+0000 7f845947e700 1 -- 172.21.15.43:0/3689063144 shutdown_connections 2024-04-21T08:28:48.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.867+0000 7f845947e700 1 -- 172.21.15.43:0/3689063144 wait complete. 2024-04-21T08:28:48.521 INFO:teuthology.orchestra.run.smithi043.stdout:mgr epoch 5 is available 2024-04-21T08:28:48.521 INFO:teuthology.orchestra.run.smithi043.stdout:Setting orchestrator backend to cephadm... 2024-04-21T08:28:49.174 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:48 smithi043 ceph-mon[27978]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-21T08:28:49.174 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:48 smithi043 ceph-mon[27978]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-21T08:28:49.174 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:48 smithi043 ceph-mon[27978]: [21/Apr/2024:08:28:48] ENGINE Bus STARTING 2024-04-21T08:28:49.175 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:48 smithi043 ceph-mon[27978]: [21/Apr/2024:08:28:48] ENGINE Serving on https://172.21.15.43:7150 2024-04-21T08:28:49.175 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:48 smithi043 ceph-mon[27978]: [21/Apr/2024:08:28:48] ENGINE Serving on http://172.21.15.43:8765 2024-04-21T08:28:49.175 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:48 smithi043 ceph-mon[27978]: [21/Apr/2024:08:28:48] ENGINE Bus STARTED 2024-04-21T08:28:49.175 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:48 smithi043 ceph-mon[27978]: from='mgr.14118 172.21.15.43:0/1296813352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:28:49.745 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.939+0000 7f0749e0c700 1 Processor -- start 2024-04-21T08:28:49.745 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.939+0000 7f0749e0c700 1 -- start start 2024-04-21T08:28:49.745 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.939+0000 7f0749e0c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0744108360 0x7f0744108780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:49.745 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.939+0000 7f0749e0c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0744108d50 con 0x7f0744108360 2024-04-21T08:28:49.745 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.940+0000 7f07437fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0744108360 0x7f0744108780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:49.745 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.940+0000 7f07437fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0744108360 0x7f0744108780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48140/0 (socket says 172.21.15.43:48140) 2024-04-21T08:28:49.745 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.940+0000 7f07437fe700 1 -- 172.21.15.43:0/263858010 learned_addr learned my addr 172.21.15.43:0/263858010 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:49.745 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.940+0000 7f07437fe700 1 -- 172.21.15.43:0/263858010 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0744109560 con 0x7f0744108360 2024-04-21T08:28:49.746 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.940+0000 7f07437fe700 1 --2- 172.21.15.43:0/263858010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0744108360 0x7f0744108780 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f073c009e80 tx=0x7f073c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b0f9bcb6f5a586 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:49.746 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.940+0000 7f07427fc700 1 -- 172.21.15.43:0/263858010 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f073c017070 con 0x7f0744108360 2024-04-21T08:28:49.746 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.940+0000 7f07427fc700 1 -- 172.21.15.43:0/263858010 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f073c00b8c0 con 0x7f0744108360 2024-04-21T08:28:49.746 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.940+0000 7f07427fc700 1 -- 172.21.15.43:0/263858010 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f073c004490 con 0x7f0744108360 2024-04-21T08:28:49.746 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.941+0000 7f0749e0c700 1 -- 172.21.15.43:0/263858010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0744108360 msgr2=0x7f0744108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:49.746 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.941+0000 7f0749e0c700 1 --2- 172.21.15.43:0/263858010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0744108360 0x7f0744108780 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f073c009e80 tx=0x7f073c00b330 comp rx=0 tx=0).stop 2024-04-21T08:28:49.746 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.941+0000 7f0749e0c700 1 -- 172.21.15.43:0/263858010 shutdown_connections 2024-04-21T08:28:49.746 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.941+0000 7f0749e0c700 1 --2- 172.21.15.43:0/263858010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0744108360 0x7f0744108780 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:49.746 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.941+0000 7f0749e0c700 1 -- 172.21.15.43:0/263858010 >> 172.21.15.43:0/263858010 conn(0x7f0744103920 msgr2=0x7f0744105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:49.746 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.941+0000 7f0749e0c700 1 -- 172.21.15.43:0/263858010 shutdown_connections 2024-04-21T08:28:49.746 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.941+0000 7f0749e0c700 1 -- 172.21.15.43:0/263858010 wait complete. 2024-04-21T08:28:49.746 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.942+0000 7f0749e0c700 1 Processor -- start 2024-04-21T08:28:49.746 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.942+0000 7f0749e0c700 1 -- start start 2024-04-21T08:28:49.747 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.942+0000 7f0749e0c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0744108360 0x7f074407db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:49.747 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.942+0000 7f0749e0c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f074410c910 con 0x7f0744108360 2024-04-21T08:28:49.747 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.942+0000 7f07437fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0744108360 0x7f074407db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:49.747 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.942+0000 7f07437fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0744108360 0x7f074407db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48146/0 (socket says 172.21.15.43:48146) 2024-04-21T08:28:49.747 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.942+0000 7f07437fe700 1 -- 172.21.15.43:0/1797274414 learned_addr learned my addr 172.21.15.43:0/1797274414 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:49.747 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.942+0000 7f07437fe700 1 -- 172.21.15.43:0/1797274414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074407e0a0 con 0x7f0744108360 2024-04-21T08:28:49.747 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.942+0000 7f07437fe700 1 --2- 172.21.15.43:0/1797274414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0744108360 0x7f074407db60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f073c004a60 tx=0x7f073c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:49.747 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.943+0000 7f0740ff9700 1 -- 172.21.15.43:0/1797274414 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f073c020020 con 0x7f0744108360 2024-04-21T08:28:49.747 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.943+0000 7f0749e0c700 1 -- 172.21.15.43:0/1797274414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f074407c290 con 0x7f0744108360 2024-04-21T08:28:49.747 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.943+0000 7f0740ff9700 1 -- 172.21.15.43:0/1797274414 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f073c003c60 con 0x7f0744108360 2024-04-21T08:28:49.747 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.943+0000 7f0740ff9700 1 -- 172.21.15.43:0/1797274414 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f073c017040 con 0x7f0744108360 2024-04-21T08:28:49.747 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.943+0000 7f0749e0c700 1 -- 172.21.15.43:0/1797274414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f074407c7d0 con 0x7f0744108360 2024-04-21T08:28:49.747 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.944+0000 7f0740ff9700 1 -- 172.21.15.43:0/1797274414 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f073c023020 con 0x7f0744108360 2024-04-21T08:28:49.747 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.944+0000 7f0749e0c700 1 -- 172.21.15.43:0/1797274414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07440640a0 con 0x7f0744108360 2024-04-21T08:28:49.748 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.944+0000 7f0740ff9700 1 --2- 172.21.15.43:0/1797274414 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f0730039670 0x7f073003bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:49.748 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.944+0000 7f0740ff9700 1 -- 172.21.15.43:0/1797274414 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f073c00f050 con 0x7f0744108360 2024-04-21T08:28:49.748 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.944+0000 7f0742ffd700 1 --2- 172.21.15.43:0/1797274414 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f0730039670 0x7f073003bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:49.748 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.944+0000 7f0742ffd700 1 --2- 172.21.15.43:0/1797274414 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f0730039670 0x7f073003bb30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f072c006fd0 tx=0x7f072c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:49.748 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.949+0000 7f0740ff9700 1 -- 172.21.15.43:0/1797274414 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f073c02a020 con 0x7f0744108360 2024-04-21T08:28:49.748 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.175+0000 7f0749e0c700 1 -- 172.21.15.43:0/1797274414 --> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f0744104a30 con 0x7f0730039670 2024-04-21T08:28:49.748 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.180+0000 7f0740ff9700 1 -- 172.21.15.43:0/1797274414 <== mgr.14118 v2:172.21.15.43:6800/3200866696 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0744104a30 con 0x7f0730039670 2024-04-21T08:28:49.748 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.182+0000 7f0749e0c700 1 -- 172.21.15.43:0/1797274414 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f0730039670 msgr2=0x7f073003bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:49.748 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.182+0000 7f0749e0c700 1 --2- 172.21.15.43:0/1797274414 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f0730039670 0x7f073003bb30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f072c006fd0 tx=0x7f072c006e40 comp rx=0 tx=0).stop 2024-04-21T08:28:49.748 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.182+0000 7f0749e0c700 1 -- 172.21.15.43:0/1797274414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0744108360 msgr2=0x7f074407db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:49.748 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.182+0000 7f0749e0c700 1 --2- 172.21.15.43:0/1797274414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0744108360 0x7f074407db60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f073c004a60 tx=0x7f073c00bd50 comp rx=0 tx=0).stop 2024-04-21T08:28:49.748 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.182+0000 7f0749e0c700 1 -- 172.21.15.43:0/1797274414 shutdown_connections 2024-04-21T08:28:49.748 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.182+0000 7f0749e0c700 1 --2- 172.21.15.43:0/1797274414 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f0730039670 0x7f073003bb30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:49.749 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.182+0000 7f0749e0c700 1 --2- 172.21.15.43:0/1797274414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0744108360 0x7f074407db60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:49.749 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.182+0000 7f0749e0c700 1 -- 172.21.15.43:0/1797274414 >> 172.21.15.43:0/1797274414 conn(0x7f0744103920 msgr2=0x7f0744104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:49.749 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.183+0000 7f0749e0c700 1 -- 172.21.15.43:0/1797274414 shutdown_connections 2024-04-21T08:28:49.749 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.183+0000 7f0749e0c700 1 -- 172.21.15.43:0/1797274414 wait complete. 2024-04-21T08:28:50.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:49 smithi043 ceph-mon[27978]: mgrmap e8: x(active, since 2s) 2024-04-21T08:28:50.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:49 smithi043 ceph-mon[27978]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:28:50.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:49 smithi043 ceph-mon[27978]: from='mgr.14118 172.21.15.43:0/1296813352' entity='mgr.x' 2024-04-21T08:28:50.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:49 smithi043 ceph-mon[27978]: from='mgr.14118 172.21.15.43:0/1296813352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:28:51.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:50 smithi043 ceph-mon[27978]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:28:51.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-21T08:28:51.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.184+0000 7f35a86ce700 1 Processor -- start 2024-04-21T08:28:51.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.185+0000 7f35a86ce700 1 -- start start 2024-04-21T08:28:51.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.185+0000 7f35a86ce700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35a0108430 0x7f35a0108850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:51.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.185+0000 7f35a86ce700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f35a0108e20 con 0x7f35a0108430 2024-04-21T08:28:51.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.185+0000 7f35a646a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35a0108430 0x7f35a0108850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:51.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.185+0000 7f35a646a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35a0108430 0x7f35a0108850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48148/0 (socket says 172.21.15.43:48148) 2024-04-21T08:28:51.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.185+0000 7f35a646a700 1 -- 172.21.15.43:0/398374849 learned_addr learned my addr 172.21.15.43:0/398374849 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:51.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.185+0000 7f35a646a700 1 -- 172.21.15.43:0/398374849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a01095e0 con 0x7f35a0108430 2024-04-21T08:28:51.014 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.185+0000 7f35a646a700 1 --2- 172.21.15.43:0/398374849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35a0108430 0x7f35a0108850 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f35900098a0 tx=0x7f3590009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6f34f3543e9bd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:51.014 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.186+0000 7f35a5468700 1 -- 172.21.15.43:0/398374849 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3590004160 con 0x7f35a0108430 2024-04-21T08:28:51.014 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.186+0000 7f35a5468700 1 -- 172.21.15.43:0/398374849 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3590004300 con 0x7f35a0108430 2024-04-21T08:28:51.014 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.186+0000 7f35a5468700 1 -- 172.21.15.43:0/398374849 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3590011af0 con 0x7f35a0108430 2024-04-21T08:28:51.014 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.186+0000 7f35a86ce700 1 -- 172.21.15.43:0/398374849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35a0108430 msgr2=0x7f35a0108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:51.014 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.186+0000 7f35a86ce700 1 --2- 172.21.15.43:0/398374849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35a0108430 0x7f35a0108850 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f35900098a0 tx=0x7f3590009bb0 comp rx=0 tx=0).stop 2024-04-21T08:28:51.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.186+0000 7f35a86ce700 1 -- 172.21.15.43:0/398374849 shutdown_connections 2024-04-21T08:28:51.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.186+0000 7f35a86ce700 1 --2- 172.21.15.43:0/398374849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35a0108430 0x7f35a0108850 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:51.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.186+0000 7f35a86ce700 1 -- 172.21.15.43:0/398374849 >> 172.21.15.43:0/398374849 conn(0x7f35a01039f0 msgr2=0x7f35a0105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:51.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.186+0000 7f35a86ce700 1 -- 172.21.15.43:0/398374849 shutdown_connections 2024-04-21T08:28:51.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.186+0000 7f35a86ce700 1 -- 172.21.15.43:0/398374849 wait complete. 2024-04-21T08:28:51.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.187+0000 7f35a86ce700 1 Processor -- start 2024-04-21T08:28:51.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.187+0000 7f35a86ce700 1 -- start start 2024-04-21T08:28:51.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.187+0000 7f35a86ce700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35a0108430 0x7f35a019c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:51.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.187+0000 7f35a86ce700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f35a010c810 con 0x7f35a0108430 2024-04-21T08:28:51.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.187+0000 7f35a646a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35a0108430 0x7f35a019c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:51.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.187+0000 7f35a646a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35a0108430 0x7f35a019c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48150/0 (socket says 172.21.15.43:48150) 2024-04-21T08:28:51.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.187+0000 7f35a646a700 1 -- 172.21.15.43:0/1190926143 learned_addr learned my addr 172.21.15.43:0/1190926143 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:51.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.188+0000 7f35a646a700 1 -- 172.21.15.43:0/1190926143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a019c700 con 0x7f35a0108430 2024-04-21T08:28:51.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.188+0000 7f35a646a700 1 --2- 172.21.15.43:0/1190926143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35a0108430 0x7f35a019c1c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3590003f40 tx=0x7f3590011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:51.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.188+0000 7f35977fe700 1 -- 172.21.15.43:0/1190926143 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35900038b0 con 0x7f35a0108430 2024-04-21T08:28:51.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.188+0000 7f35977fe700 1 -- 172.21.15.43:0/1190926143 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3590003a50 con 0x7f35a0108430 2024-04-21T08:28:51.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.188+0000 7f35a86ce700 1 -- 172.21.15.43:0/1190926143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a019c990 con 0x7f35a0108430 2024-04-21T08:28:51.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.188+0000 7f35977fe700 1 -- 172.21.15.43:0/1190926143 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f359001a5c0 con 0x7f35a0108430 2024-04-21T08:28:51.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.188+0000 7f35a86ce700 1 -- 172.21.15.43:0/1190926143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a019f680 con 0x7f35a0108430 2024-04-21T08:28:51.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.189+0000 7f35977fe700 1 -- 172.21.15.43:0/1190926143 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f359001a8c0 con 0x7f35a0108430 2024-04-21T08:28:51.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.189+0000 7f35977fe700 1 --2- 172.21.15.43:0/1190926143 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f358c039620 0x7f358c03bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:51.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.189+0000 7f35a86ce700 1 -- 172.21.15.43:0/1190926143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35a010c990 con 0x7f35a0108430 2024-04-21T08:28:51.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.189+0000 7f35977fe700 1 -- 172.21.15.43:0/1190926143 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f359004db10 con 0x7f35a0108430 2024-04-21T08:28:51.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.189+0000 7f35a5c69700 1 --2- 172.21.15.43:0/1190926143 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f358c039620 0x7f358c03bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:51.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.190+0000 7f35a5c69700 1 --2- 172.21.15.43:0/1190926143 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f358c039620 0x7f358c03bae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f359c006fd0 tx=0x7f359c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:51.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.195+0000 7f35977fe700 1 -- 172.21.15.43:0/1190926143 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3590019c70 con 0x7f35a0108430 2024-04-21T08:28:51.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.404+0000 7f35a86ce700 1 -- 172.21.15.43:0/1190926143 --> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f35a0104b20 con 0x7f358c039620 2024-04-21T08:28:51.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.405+0000 7f35977fe700 1 -- 172.21.15.43:0/1190926143 <== mgr.14118 v2:172.21.15.43:6800/3200866696 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f35a0104b20 con 0x7f358c039620 2024-04-21T08:28:51.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.408+0000 7f35a86ce700 1 -- 172.21.15.43:0/1190926143 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f358c039620 msgr2=0x7f358c03bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:51.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.408+0000 7f35a86ce700 1 --2- 172.21.15.43:0/1190926143 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f358c039620 0x7f358c03bae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f359c006fd0 tx=0x7f359c006e40 comp rx=0 tx=0).stop 2024-04-21T08:28:51.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.408+0000 7f35a86ce700 1 -- 172.21.15.43:0/1190926143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35a0108430 msgr2=0x7f35a019c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:51.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.408+0000 7f35a86ce700 1 --2- 172.21.15.43:0/1190926143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35a0108430 0x7f35a019c1c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3590003f40 tx=0x7f3590011890 comp rx=0 tx=0).stop 2024-04-21T08:28:51.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.408+0000 7f35a86ce700 1 -- 172.21.15.43:0/1190926143 shutdown_connections 2024-04-21T08:28:51.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.408+0000 7f35a86ce700 1 --2- 172.21.15.43:0/1190926143 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f358c039620 0x7f358c03bae0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:51.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.408+0000 7f35a86ce700 1 --2- 172.21.15.43:0/1190926143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35a0108430 0x7f35a019c1c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:51.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.408+0000 7f35a86ce700 1 -- 172.21.15.43:0/1190926143 >> 172.21.15.43:0/1190926143 conn(0x7f35a01039f0 msgr2=0x7f35a0104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:51.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.408+0000 7f35a86ce700 1 -- 172.21.15.43:0/1190926143 shutdown_connections 2024-04-21T08:28:51.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.408+0000 7f35a86ce700 1 -- 172.21.15.43:0/1190926143 wait complete. 2024-04-21T08:28:51.019 INFO:teuthology.orchestra.run.smithi043.stdout:Generating ssh key... 2024-04-21T08:28:52.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:51 smithi043 ceph-mon[27978]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:28:52.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:51 smithi043 ceph-mon[27978]: Generating ssh key... 2024-04-21T08:28:52.811 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: Generating public/private rsa key pair. 2024-04-21T08:28:52.811 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: Your identification has been saved in /tmp/tmprcu86ytr/key. 2024-04-21T08:28:52.812 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: Your public key has been saved in /tmp/tmprcu86ytr/key.pub. 2024-04-21T08:28:52.812 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: The key fingerprint is: 2024-04-21T08:28:52.812 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: SHA256:SaRrDVHDDPrfWgnajdEcb8EFwkDC9R8FEto2+kj2d9I ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:28:52.812 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: The key's randomart image is: 2024-04-21T08:28:52.812 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: +---[RSA 3072]----+ 2024-04-21T08:28:52.812 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: | .+BBo+.ooo | 2024-04-21T08:28:52.812 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: | ..=o=.+ o | 2024-04-21T08:28:52.812 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: | . o o * + | 2024-04-21T08:28:52.812 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: | . = * = o | 2024-04-21T08:28:52.812 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: | + S o + | 2024-04-21T08:28:52.812 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: | . * X o . | 2024-04-21T08:28:52.812 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: | . = B o E | 2024-04-21T08:28:52.812 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: | o . o | 2024-04-21T08:28:52.812 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: | . | 2024-04-21T08:28:52.813 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:28:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: +----[SHA256]-----+ 2024-04-21T08:28:53.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.459+0000 7fa39f8d7700 1 Processor -- start 2024-04-21T08:28:53.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.459+0000 7fa39f8d7700 1 -- start start 2024-04-21T08:28:53.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.460+0000 7fa39f8d7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa398108ef0 0x7fa39810b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:53.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.460+0000 7fa39f8d7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa39807bdc0 con 0x7fa398108ef0 2024-04-21T08:28:53.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.460+0000 7fa39d673700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa398108ef0 0x7fa39810b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:53.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.460+0000 7fa39d673700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa398108ef0 0x7fa39810b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48162/0 (socket says 172.21.15.43:48162) 2024-04-21T08:28:53.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.460+0000 7fa39d673700 1 -- 172.21.15.43:0/993006488 learned_addr learned my addr 172.21.15.43:0/993006488 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:53.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.460+0000 7fa39d673700 1 -- 172.21.15.43:0/993006488 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39810b820 con 0x7fa398108ef0 2024-04-21T08:28:53.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.460+0000 7fa39d673700 1 --2- 172.21.15.43:0/993006488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa398108ef0 0x7fa39810b2e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa3880098a0 tx=0x7fa388009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b592b4628eb95369 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:53.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.460+0000 7fa38ffff700 1 -- 172.21.15.43:0/993006488 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa388004160 con 0x7fa398108ef0 2024-04-21T08:28:53.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.461+0000 7fa38ffff700 1 -- 172.21.15.43:0/993006488 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa388004300 con 0x7fa398108ef0 2024-04-21T08:28:53.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.461+0000 7fa38ffff700 1 -- 172.21.15.43:0/993006488 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa388011af0 con 0x7fa398108ef0 2024-04-21T08:28:53.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.461+0000 7fa39f8d7700 1 -- 172.21.15.43:0/993006488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa398108ef0 msgr2=0x7fa39810b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:53.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.461+0000 7fa39f8d7700 1 --2- 172.21.15.43:0/993006488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa398108ef0 0x7fa39810b2e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa3880098a0 tx=0x7fa388009bb0 comp rx=0 tx=0).stop 2024-04-21T08:28:53.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.461+0000 7fa39f8d7700 1 -- 172.21.15.43:0/993006488 shutdown_connections 2024-04-21T08:28:53.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.461+0000 7fa39f8d7700 1 --2- 172.21.15.43:0/993006488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa398108ef0 0x7fa39810b2e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:53.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.461+0000 7fa39f8d7700 1 -- 172.21.15.43:0/993006488 >> 172.21.15.43:0/993006488 conn(0x7fa3981028e0 msgr2=0x7fa398104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:53.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.461+0000 7fa39f8d7700 1 -- 172.21.15.43:0/993006488 shutdown_connections 2024-04-21T08:28:53.068 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.461+0000 7fa39f8d7700 1 -- 172.21.15.43:0/993006488 wait complete. 2024-04-21T08:28:53.068 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.462+0000 7fa39f8d7700 1 Processor -- start 2024-04-21T08:28:53.068 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.462+0000 7fa39f8d7700 1 -- start start 2024-04-21T08:28:53.068 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.462+0000 7fa39f8d7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa398108ef0 0x7fa39819c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:53.068 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.462+0000 7fa39f8d7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa39810c6b0 con 0x7fa398108ef0 2024-04-21T08:28:53.068 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.462+0000 7fa39d673700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa398108ef0 0x7fa39819c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:53.068 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.462+0000 7fa39d673700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa398108ef0 0x7fa39819c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48170/0 (socket says 172.21.15.43:48170) 2024-04-21T08:28:53.068 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.462+0000 7fa39d673700 1 -- 172.21.15.43:0/3045445952 learned_addr learned my addr 172.21.15.43:0/3045445952 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:53.068 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.463+0000 7fa39d673700 1 -- 172.21.15.43:0/3045445952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39819c730 con 0x7fa398108ef0 2024-04-21T08:28:53.068 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.463+0000 7fa39d673700 1 --2- 172.21.15.43:0/3045445952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa398108ef0 0x7fa39819c1f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa388003f40 tx=0x7fa388011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:53.068 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.463+0000 7fa38e7fc700 1 -- 172.21.15.43:0/3045445952 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3880038b0 con 0x7fa398108ef0 2024-04-21T08:28:53.068 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.463+0000 7fa38e7fc700 1 -- 172.21.15.43:0/3045445952 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa388003a50 con 0x7fa398108ef0 2024-04-21T08:28:53.069 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.463+0000 7fa39f8d7700 1 -- 172.21.15.43:0/3045445952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa39819c9c0 con 0x7fa398108ef0 2024-04-21T08:28:53.069 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.463+0000 7fa38e7fc700 1 -- 172.21.15.43:0/3045445952 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa38801a5c0 con 0x7fa398108ef0 2024-04-21T08:28:53.069 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.463+0000 7fa39f8d7700 1 -- 172.21.15.43:0/3045445952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa39819f6b0 con 0x7fa398108ef0 2024-04-21T08:28:53.069 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.464+0000 7fa38e7fc700 1 -- 172.21.15.43:0/3045445952 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa388024430 con 0x7fa398108ef0 2024-04-21T08:28:53.069 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.464+0000 7fa39f8d7700 1 -- 172.21.15.43:0/3045445952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3980640a0 con 0x7fa398108ef0 2024-04-21T08:28:53.069 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.464+0000 7fa38e7fc700 1 --2- 172.21.15.43:0/3045445952 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fa384039670 0x7fa38403bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:53.069 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.464+0000 7fa38e7fc700 1 -- 172.21.15.43:0/3045445952 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa38804dbb0 con 0x7fa398108ef0 2024-04-21T08:28:53.069 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.464+0000 7fa39ce72700 1 --2- 172.21.15.43:0/3045445952 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fa384039670 0x7fa38403bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:53.069 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.465+0000 7fa39ce72700 1 --2- 172.21.15.43:0/3045445952 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fa384039670 0x7fa38403bb30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa394006fd0 tx=0x7fa394006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:53.069 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.469+0000 7fa38e7fc700 1 -- 172.21.15.43:0/3045445952 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa38802d910 con 0x7fa398108ef0 2024-04-21T08:28:53.069 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.678+0000 7fa39f8d7700 1 -- 172.21.15.43:0/3045445952 --> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fa398103a60 con 0x7fa384039670 2024-04-21T08:28:53.070 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.510+0000 7fa38e7fc700 1 -- 172.21.15.43:0/3045445952 <== mgr.14118 v2:172.21.15.43:6800/3200866696 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa398103a60 con 0x7fa384039670 2024-04-21T08:28:53.070 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.512+0000 7fa39f8d7700 1 -- 172.21.15.43:0/3045445952 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fa384039670 msgr2=0x7fa38403bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:53.070 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.512+0000 7fa39f8d7700 1 --2- 172.21.15.43:0/3045445952 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fa384039670 0x7fa38403bb30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa394006fd0 tx=0x7fa394006e40 comp rx=0 tx=0).stop 2024-04-21T08:28:53.070 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.512+0000 7fa39f8d7700 1 -- 172.21.15.43:0/3045445952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa398108ef0 msgr2=0x7fa39819c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:53.070 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.512+0000 7fa39f8d7700 1 --2- 172.21.15.43:0/3045445952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa398108ef0 0x7fa39819c1f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa388003f40 tx=0x7fa388011890 comp rx=0 tx=0).stop 2024-04-21T08:28:53.070 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.512+0000 7fa39f8d7700 1 -- 172.21.15.43:0/3045445952 shutdown_connections 2024-04-21T08:28:53.070 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.512+0000 7fa39f8d7700 1 --2- 172.21.15.43:0/3045445952 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fa384039670 0x7fa38403bb30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:53.070 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.512+0000 7fa39f8d7700 1 --2- 172.21.15.43:0/3045445952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa398108ef0 0x7fa39819c1f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:53.070 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.512+0000 7fa39f8d7700 1 -- 172.21.15.43:0/3045445952 >> 172.21.15.43:0/3045445952 conn(0x7fa3981028e0 msgr2=0x7fa398103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:53.070 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.512+0000 7fa39f8d7700 1 -- 172.21.15.43:0/3045445952 shutdown_connections 2024-04-21T08:28:53.071 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.512+0000 7fa39f8d7700 1 -- 172.21.15.43:0/3045445952 wait complete. 2024-04-21T08:28:53.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:53 smithi043 ceph-mon[27978]: from='mgr.14118 172.21.15.43:0/1296813352' entity='mgr.x' 2024-04-21T08:28:53.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:53 smithi043 ceph-mon[27978]: from='mgr.14118 172.21.15.43:0/1296813352' entity='mgr.x' 2024-04-21T08:28:54.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnfcfO6EmZIk+jlEg1uY5awyUdoyC7l44f6hqhPS7a72XoYQ+clUe6JS5eLwrGep3vYQ0AHRwmmUpilnnN22IwR2cmkn/g8X2y7znJ6GFAJj9lQp6DEwhkYjV/0KIbPZ5ftPGBYev7xTjaLLf1BwJ5G0LKpMAm+hQQwolUXuZ+TVUFuUZmtjtXnHk2Ye7gbpE8tcpLX94cK1MFiqaRww8pneNFIsb1V4Ha0a/ghXKy3vjQAVM/FNYgEcpQNWbfiQAQ7S6LDV0k004MclVPkIk/huIovQF36i0OO9jPFZ58xCet48B4j4j6oNasK/q3R8TjZmpcgok1sk0d5H0PWnFA21YyLE5JbVP+bGQvG0s2zvYRVJ+1eEcYB7GLpLsnanFLrfvtvSTybk0dyXZ4qGCxmJoKmSzjBMFGorvFjYcixW6ysjwh6y3HvhPXOPkG/ZOAE1KW6sdCB7YtHymiUp/OPz/dywgDVFMXl1C4QXACpgPuVerYQROLfVJIcU8XWu0= ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:28:54.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.498+0000 7f14e01f6700 1 Processor -- start 2024-04-21T08:28:54.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.498+0000 7f14e01f6700 1 -- start start 2024-04-21T08:28:54.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.499+0000 7f14e01f6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14d8106610 0x7f14d8106a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:54.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.499+0000 7f14e01f6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14d8107000 con 0x7f14d8106610 2024-04-21T08:28:54.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.499+0000 7f14ddf92700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14d8106610 0x7f14d8106a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:54.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.499+0000 7f14ddf92700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14d8106610 0x7f14d8106a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48174/0 (socket says 172.21.15.43:48174) 2024-04-21T08:28:54.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.499+0000 7f14ddf92700 1 -- 172.21.15.43:0/2332653222 learned_addr learned my addr 172.21.15.43:0/2332653222 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:54.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.499+0000 7f14ddf92700 1 -- 172.21.15.43:0/2332653222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d8107180 con 0x7f14d8106610 2024-04-21T08:28:54.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.499+0000 7f14ddf92700 1 --2- 172.21.15.43:0/2332653222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14d8106610 0x7f14d8106a30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f14d0009e80 tx=0x7f14d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb2a61c74a6f8ae server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:54.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.499+0000 7f14dcf90700 1 -- 172.21.15.43:0/2332653222 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14d0017070 con 0x7f14d8106610 2024-04-21T08:28:54.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.499+0000 7f14dcf90700 1 -- 172.21.15.43:0/2332653222 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f14d000b8c0 con 0x7f14d8106610 2024-04-21T08:28:54.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.499+0000 7f14dcf90700 1 -- 172.21.15.43:0/2332653222 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14d0004490 con 0x7f14d8106610 2024-04-21T08:28:54.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.500+0000 7f14e01f6700 1 -- 172.21.15.43:0/2332653222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14d8106610 msgr2=0x7f14d8106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:54.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.500+0000 7f14e01f6700 1 --2- 172.21.15.43:0/2332653222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14d8106610 0x7f14d8106a30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f14d0009e80 tx=0x7f14d000b330 comp rx=0 tx=0).stop 2024-04-21T08:28:54.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.500+0000 7f14e01f6700 1 -- 172.21.15.43:0/2332653222 shutdown_connections 2024-04-21T08:28:54.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.500+0000 7f14e01f6700 1 --2- 172.21.15.43:0/2332653222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14d8106610 0x7f14d8106a30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:54.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.500+0000 7f14e01f6700 1 -- 172.21.15.43:0/2332653222 >> 172.21.15.43:0/2332653222 conn(0x7f14d8101f70 msgr2=0x7f14d81043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:54.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.500+0000 7f14e01f6700 1 -- 172.21.15.43:0/2332653222 shutdown_connections 2024-04-21T08:28:54.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.500+0000 7f14e01f6700 1 -- 172.21.15.43:0/2332653222 wait complete. 2024-04-21T08:28:54.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.501+0000 7f14e01f6700 1 Processor -- start 2024-04-21T08:28:54.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.501+0000 7f14e01f6700 1 -- start start 2024-04-21T08:28:54.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.501+0000 7f14e01f6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14d8106610 0x7f14d819c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:54.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.501+0000 7f14e01f6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14d810a4f0 con 0x7f14d8106610 2024-04-21T08:28:54.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.501+0000 7f14ddf92700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14d8106610 0x7f14d819c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:54.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.501+0000 7f14ddf92700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14d8106610 0x7f14d819c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48178/0 (socket says 172.21.15.43:48178) 2024-04-21T08:28:54.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.501+0000 7f14ddf92700 1 -- 172.21.15.43:0/1846838093 learned_addr learned my addr 172.21.15.43:0/1846838093 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:54.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.501+0000 7f14ddf92700 1 -- 172.21.15.43:0/1846838093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d819c750 con 0x7f14d8106610 2024-04-21T08:28:54.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.501+0000 7f14ddf92700 1 --2- 172.21.15.43:0/1846838093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14d8106610 0x7f14d819c210 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f14d0000f80 tx=0x7f14d000bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:54.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.502+0000 7f14d6ffd700 1 -- 172.21.15.43:0/1846838093 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14d0020020 con 0x7f14d8106610 2024-04-21T08:28:54.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.502+0000 7f14d6ffd700 1 -- 172.21.15.43:0/1846838093 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f14d0012430 con 0x7f14d8106610 2024-04-21T08:28:54.258 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.502+0000 7f14e01f6700 1 -- 172.21.15.43:0/1846838093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d819c9e0 con 0x7f14d8106610 2024-04-21T08:28:54.258 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.502+0000 7f14d6ffd700 1 -- 172.21.15.43:0/1846838093 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14d0017040 con 0x7f14d8106610 2024-04-21T08:28:54.258 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.502+0000 7f14e01f6700 1 -- 172.21.15.43:0/1846838093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d819f6d0 con 0x7f14d8106610 2024-04-21T08:28:54.258 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.502+0000 7f14d6ffd700 1 -- 172.21.15.43:0/1846838093 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f14d0023020 con 0x7f14d8106610 2024-04-21T08:28:54.258 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.503+0000 7f14d6ffd700 1 --2- 172.21.15.43:0/1846838093 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f14c00396c0 0x7f14c003bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:54.258 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.503+0000 7f14d6ffd700 1 -- 172.21.15.43:0/1846838093 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f14d000f050 con 0x7f14d8106610 2024-04-21T08:28:54.258 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.503+0000 7f14e01f6700 1 -- 172.21.15.43:0/1846838093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14d80640a0 con 0x7f14d8106610 2024-04-21T08:28:54.258 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.503+0000 7f14dd791700 1 --2- 172.21.15.43:0/1846838093 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f14c00396c0 0x7f14c003bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:54.258 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.503+0000 7f14dd791700 1 --2- 172.21.15.43:0/1846838093 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f14c00396c0 0x7f14c003bb80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f14cc006fd0 tx=0x7f14cc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:54.258 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.508+0000 7f14d6ffd700 1 -- 172.21.15.43:0/1846838093 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f14d002a020 con 0x7f14d8106610 2024-04-21T08:28:54.258 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.715+0000 7f14e01f6700 1 -- 172.21.15.43:0/1846838093 --> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f14d8103180 con 0x7f14c00396c0 2024-04-21T08:28:54.258 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.716+0000 7f14d6ffd700 1 -- 172.21.15.43:0/1846838093 <== mgr.14118 v2:172.21.15.43:6800/3200866696 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f14d8103180 con 0x7f14c00396c0 2024-04-21T08:28:54.259 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.718+0000 7f14e01f6700 1 -- 172.21.15.43:0/1846838093 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f14c00396c0 msgr2=0x7f14c003bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:54.259 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.718+0000 7f14e01f6700 1 --2- 172.21.15.43:0/1846838093 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f14c00396c0 0x7f14c003bb80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f14cc006fd0 tx=0x7f14cc006e40 comp rx=0 tx=0).stop 2024-04-21T08:28:54.259 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.718+0000 7f14e01f6700 1 -- 172.21.15.43:0/1846838093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14d8106610 msgr2=0x7f14d819c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:54.259 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.718+0000 7f14e01f6700 1 --2- 172.21.15.43:0/1846838093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14d8106610 0x7f14d819c210 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f14d0000f80 tx=0x7f14d000bdc0 comp rx=0 tx=0).stop 2024-04-21T08:28:54.259 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.718+0000 7f14e01f6700 1 -- 172.21.15.43:0/1846838093 shutdown_connections 2024-04-21T08:28:54.259 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.718+0000 7f14e01f6700 1 --2- 172.21.15.43:0/1846838093 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f14c00396c0 0x7f14c003bb80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:54.259 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.718+0000 7f14e01f6700 1 --2- 172.21.15.43:0/1846838093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14d8106610 0x7f14d819c210 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:54.259 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.718+0000 7f14e01f6700 1 -- 172.21.15.43:0/1846838093 >> 172.21.15.43:0/1846838093 conn(0x7f14d8101f70 msgr2=0x7f14d8102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:54.259 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.718+0000 7f14e01f6700 1 -- 172.21.15.43:0/1846838093 shutdown_connections 2024-04-21T08:28:54.259 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.718+0000 7f14e01f6700 1 -- 172.21.15.43:0/1846838093 wait complete. 2024-04-21T08:28:54.259 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-21T08:28:54.259 INFO:teuthology.orchestra.run.smithi043.stdout:Adding key to root@localhost authorized_keys... 2024-04-21T08:28:54.259 INFO:teuthology.orchestra.run.smithi043.stdout:Adding host smithi043... 2024-04-21T08:28:54.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:54 smithi043 ceph-mon[27978]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:28:56.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:55 smithi043 ceph-mon[27978]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi043", "addr": "172.21.15.43", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:28:56.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:28:55 smithi043 ceph-mon[27978]: Deploying cephadm binary to smithi043 2024-04-21T08:28:59.712 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Added host 'smithi043' with addr '172.21.15.43' 2024-04-21T08:28:59.712 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.725+0000 7f436e3d1700 1 Processor -- start 2024-04-21T08:28:59.712 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.725+0000 7f436e3d1700 1 -- start start 2024-04-21T08:28:59.712 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.725+0000 7f436e3d1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4368108360 0x7f4368108780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:59.712 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.725+0000 7f436e3d1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4368108d50 con 0x7f4368108360 2024-04-21T08:28:59.712 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.726+0000 7f4367fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4368108360 0x7f4368108780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:59.712 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.726+0000 7f4367fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4368108360 0x7f4368108780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48192/0 (socket says 172.21.15.43:48192) 2024-04-21T08:28:59.712 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.726+0000 7f4367fff700 1 -- 172.21.15.43:0/4114748305 learned_addr learned my addr 172.21.15.43:0/4114748305 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:59.712 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.726+0000 7f4367fff700 1 -- 172.21.15.43:0/4114748305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4368109560 con 0x7f4368108360 2024-04-21T08:28:59.713 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.726+0000 7f4367fff700 1 --2- 172.21.15.43:0/4114748305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4368108360 0x7f4368108780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f43580098a0 tx=0x7f4358009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb43003f4ab51ac server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:59.713 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.726+0000 7f4366ffd700 1 -- 172.21.15.43:0/4114748305 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4358004160 con 0x7f4368108360 2024-04-21T08:28:59.713 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.726+0000 7f4366ffd700 1 -- 172.21.15.43:0/4114748305 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4358004300 con 0x7f4368108360 2024-04-21T08:28:59.713 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.726+0000 7f4366ffd700 1 -- 172.21.15.43:0/4114748305 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4358011af0 con 0x7f4368108360 2024-04-21T08:28:59.713 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.727+0000 7f436e3d1700 1 -- 172.21.15.43:0/4114748305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4368108360 msgr2=0x7f4368108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:59.713 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.727+0000 7f436e3d1700 1 --2- 172.21.15.43:0/4114748305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4368108360 0x7f4368108780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f43580098a0 tx=0x7f4358009bb0 comp rx=0 tx=0).stop 2024-04-21T08:28:59.713 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.727+0000 7f436e3d1700 1 -- 172.21.15.43:0/4114748305 shutdown_connections 2024-04-21T08:28:59.713 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.727+0000 7f436e3d1700 1 --2- 172.21.15.43:0/4114748305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4368108360 0x7f4368108780 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:59.713 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.727+0000 7f436e3d1700 1 -- 172.21.15.43:0/4114748305 >> 172.21.15.43:0/4114748305 conn(0x7f4368103920 msgr2=0x7f4368105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:59.713 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.727+0000 7f436e3d1700 1 -- 172.21.15.43:0/4114748305 shutdown_connections 2024-04-21T08:28:59.713 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.727+0000 7f436e3d1700 1 -- 172.21.15.43:0/4114748305 wait complete. 2024-04-21T08:28:59.713 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.727+0000 7f436e3d1700 1 Processor -- start 2024-04-21T08:28:59.713 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.728+0000 7f436e3d1700 1 -- start start 2024-04-21T08:28:59.713 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.728+0000 7f436e3d1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4368108360 0x7f436807db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:59.714 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.728+0000 7f436e3d1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f436810c910 con 0x7f4368108360 2024-04-21T08:28:59.714 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.728+0000 7f4367fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4368108360 0x7f436807db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:59.714 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.728+0000 7f4367fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4368108360 0x7f436807db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48206/0 (socket says 172.21.15.43:48206) 2024-04-21T08:28:59.714 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.728+0000 7f4367fff700 1 -- 172.21.15.43:0/3090944755 learned_addr learned my addr 172.21.15.43:0/3090944755 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:28:59.714 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.728+0000 7f4367fff700 1 -- 172.21.15.43:0/3090944755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436807e0a0 con 0x7f4368108360 2024-04-21T08:28:59.714 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.728+0000 7f4367fff700 1 --2- 172.21.15.43:0/3090944755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4368108360 0x7f436807db60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4358000c00 tx=0x7f4358011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:59.714 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.729+0000 7f43657fa700 1 -- 172.21.15.43:0/3090944755 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4358003b20 con 0x7f4368108360 2024-04-21T08:28:59.714 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.729+0000 7f43657fa700 1 -- 172.21.15.43:0/3090944755 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f435801a8c0 con 0x7f4368108360 2024-04-21T08:28:59.714 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.729+0000 7f436e3d1700 1 -- 172.21.15.43:0/3090944755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f436807c290 con 0x7f4368108360 2024-04-21T08:28:59.714 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.729+0000 7f436e3d1700 1 -- 172.21.15.43:0/3090944755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f436807c7d0 con 0x7f4368108360 2024-04-21T08:28:59.714 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.729+0000 7f43657fa700 1 -- 172.21.15.43:0/3090944755 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4358024940 con 0x7f4368108360 2024-04-21T08:28:59.714 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.730+0000 7f43657fa700 1 -- 172.21.15.43:0/3090944755 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4358022020 con 0x7f4368108360 2024-04-21T08:28:59.714 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.730+0000 7f43657fa700 1 --2- 172.21.15.43:0/3090944755 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f43500396c0 0x7f435003bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:59.714 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.730+0000 7f43657fa700 1 -- 172.21.15.43:0/3090944755 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f435804d790 con 0x7f4368108360 2024-04-21T08:28:59.715 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.730+0000 7f43677fe700 1 --2- 172.21.15.43:0/3090944755 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f43500396c0 0x7f435003bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:59.715 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.730+0000 7f436e3d1700 1 -- 172.21.15.43:0/3090944755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43680640a0 con 0x7f4368108360 2024-04-21T08:28:59.715 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.730+0000 7f43677fe700 1 --2- 172.21.15.43:0/3090944755 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f43500396c0 0x7f435003bb80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f435c006fd0 tx=0x7f435c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:59.715 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.735+0000 7f43657fa700 1 -- 172.21.15.43:0/3090944755 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43580172b0 con 0x7f4368108360 2024-04-21T08:28:59.715 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.949+0000 7f436e3d1700 1 -- 172.21.15.43:0/3090944755 --> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi043", "addr": "172.21.15.43", "target": ["mon-mgr", ""]}) -- 0x7f436807cee0 con 0x7f43500396c0 2024-04-21T08:28:59.715 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.161+0000 7f43657fa700 1 -- 172.21.15.43:0/3090944755 <== mgr.14118 v2:172.21.15.43:6800/3200866696 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f436807cee0 con 0x7f43500396c0 2024-04-21T08:28:59.715 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.163+0000 7f436e3d1700 1 -- 172.21.15.43:0/3090944755 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f43500396c0 msgr2=0x7f435003bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:59.715 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.163+0000 7f436e3d1700 1 --2- 172.21.15.43:0/3090944755 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f43500396c0 0x7f435003bb80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f435c006fd0 tx=0x7f435c006e40 comp rx=0 tx=0).stop 2024-04-21T08:28:59.715 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.163+0000 7f436e3d1700 1 -- 172.21.15.43:0/3090944755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4368108360 msgr2=0x7f436807db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:59.715 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.163+0000 7f436e3d1700 1 --2- 172.21.15.43:0/3090944755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4368108360 0x7f436807db60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4358000c00 tx=0x7f4358011b70 comp rx=0 tx=0).stop 2024-04-21T08:28:59.715 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.163+0000 7f436e3d1700 1 -- 172.21.15.43:0/3090944755 shutdown_connections 2024-04-21T08:28:59.715 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.164+0000 7f436e3d1700 1 --2- 172.21.15.43:0/3090944755 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f43500396c0 0x7f435003bb80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:59.715 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.164+0000 7f436e3d1700 1 --2- 172.21.15.43:0/3090944755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4368108360 0x7f436807db60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:59.715 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.164+0000 7f436e3d1700 1 -- 172.21.15.43:0/3090944755 >> 172.21.15.43:0/3090944755 conn(0x7f4368103920 msgr2=0x7f43681043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:59.715 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.164+0000 7f436e3d1700 1 -- 172.21.15.43:0/3090944755 shutdown_connections 2024-04-21T08:28:59.716 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.164+0000 7f436e3d1700 1 -- 172.21.15.43:0/3090944755 wait complete. 2024-04-21T08:28:59.716 INFO:teuthology.orchestra.run.smithi043.stdout:Deploying unmanaged mon service... 2024-04-21T08:29:00.424 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:00 smithi043 ceph-mon[27978]: from='mgr.14118 172.21.15.43:0/1296813352' entity='mgr.x' 2024-04-21T08:29:00.424 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:00 smithi043 ceph-mon[27978]: Added host smithi043 2024-04-21T08:29:00.424 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:00 smithi043 ceph-mon[27978]: from='mgr.14118 172.21.15.43:0/1296813352' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:29:01.046 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-21T08:29:01.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.196+0000 7f446b882700 1 Processor -- start 2024-04-21T08:29:01.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.196+0000 7f446b882700 1 -- start start 2024-04-21T08:29:01.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.196+0000 7f446b882700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4464107780 0x7f4464107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:01.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.196+0000 7f446b882700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4464108170 con 0x7f4464107780 2024-04-21T08:29:01.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.196+0000 7f446961e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4464107780 0x7f4464107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:01.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.196+0000 7f446961e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4464107780 0x7f4464107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44964/0 (socket says 172.21.15.43:44964) 2024-04-21T08:29:01.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.196+0000 7f446961e700 1 -- 172.21.15.43:0/3079805736 learned_addr learned my addr 172.21.15.43:0/3079805736 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:01.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.197+0000 7f446961e700 1 -- 172.21.15.43:0/3079805736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44641084a0 con 0x7f4464107780 2024-04-21T08:29:01.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.197+0000 7f446961e700 1 --2- 172.21.15.43:0/3079805736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4464107780 0x7f4464107ba0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f446000bb40 tx=0x7f446000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98983a7b01f0409f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:01.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.197+0000 7f445bfff700 1 -- 172.21.15.43:0/3079805736 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f446000dbe0 con 0x7f4464107780 2024-04-21T08:29:01.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.197+0000 7f445bfff700 1 -- 172.21.15.43:0/3079805736 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f44600044c0 con 0x7f4464107780 2024-04-21T08:29:01.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.197+0000 7f445bfff700 1 -- 172.21.15.43:0/3079805736 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4460010470 con 0x7f4464107780 2024-04-21T08:29:01.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.198+0000 7f446b882700 1 -- 172.21.15.43:0/3079805736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4464107780 msgr2=0x7f4464107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:01.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.198+0000 7f446b882700 1 --2- 172.21.15.43:0/3079805736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4464107780 0x7f4464107ba0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f446000bb40 tx=0x7f446000be50 comp rx=0 tx=0).stop 2024-04-21T08:29:01.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.198+0000 7f446b882700 1 -- 172.21.15.43:0/3079805736 shutdown_connections 2024-04-21T08:29:01.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.198+0000 7f446b882700 1 --2- 172.21.15.43:0/3079805736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4464107780 0x7f4464107ba0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:01.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.198+0000 7f446b882700 1 -- 172.21.15.43:0/3079805736 >> 172.21.15.43:0/3079805736 conn(0x7f4464102040 msgr2=0x7f4464104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:01.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.198+0000 7f446b882700 1 -- 172.21.15.43:0/3079805736 shutdown_connections 2024-04-21T08:29:01.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.198+0000 7f446b882700 1 -- 172.21.15.43:0/3079805736 wait complete. 2024-04-21T08:29:01.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.198+0000 7f446b882700 1 Processor -- start 2024-04-21T08:29:01.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.198+0000 7f446b882700 1 -- start start 2024-04-21T08:29:01.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.198+0000 7f446b882700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4464107780 0x7f446419e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:01.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.199+0000 7f446b882700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f44641062e0 con 0x7f4464107780 2024-04-21T08:29:01.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.199+0000 7f446961e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4464107780 0x7f446419e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:01.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.199+0000 7f446961e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4464107780 0x7f446419e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44968/0 (socket says 172.21.15.43:44968) 2024-04-21T08:29:01.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.199+0000 7f446961e700 1 -- 172.21.15.43:0/1437392598 learned_addr learned my addr 172.21.15.43:0/1437392598 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:01.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.199+0000 7f446961e700 1 -- 172.21.15.43:0/1437392598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446419e890 con 0x7f4464107780 2024-04-21T08:29:01.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.199+0000 7f446961e700 1 --2- 172.21.15.43:0/1437392598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4464107780 0x7f446419e350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f446000b340 tx=0x7f4460003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:01.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.199+0000 7f445a7fc700 1 -- 172.21.15.43:0/1437392598 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4460010720 con 0x7f4464107780 2024-04-21T08:29:01.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.199+0000 7f445a7fc700 1 -- 172.21.15.43:0/1437392598 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f446001d070 con 0x7f4464107780 2024-04-21T08:29:01.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.199+0000 7f446b882700 1 -- 172.21.15.43:0/1437392598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f446419eb20 con 0x7f4464107780 2024-04-21T08:29:01.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.199+0000 7f445a7fc700 1 -- 172.21.15.43:0/1437392598 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4460019b40 con 0x7f4464107780 2024-04-21T08:29:01.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.199+0000 7f446b882700 1 -- 172.21.15.43:0/1437392598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44641a1810 con 0x7f4464107780 2024-04-21T08:29:01.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.200+0000 7f445a7fc700 1 -- 172.21.15.43:0/1437392598 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4460028060 con 0x7f4464107780 2024-04-21T08:29:01.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.200+0000 7f446b882700 1 -- 172.21.15.43:0/1437392598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44640640a0 con 0x7f4464107780 2024-04-21T08:29:01.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.200+0000 7f445a7fc700 1 --2- 172.21.15.43:0/1437392598 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f44500396c0 0x7f445003bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:01.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.200+0000 7f445a7fc700 1 -- 172.21.15.43:0/1437392598 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f446004d8b0 con 0x7f4464107780 2024-04-21T08:29:01.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.200+0000 7f4468e1d700 1 --2- 172.21.15.43:0/1437392598 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f44500396c0 0x7f445003bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:01.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.201+0000 7f4468e1d700 1 --2- 172.21.15.43:0/1437392598 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f44500396c0 0x7f445003bb80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4454006fd0 tx=0x7f4454006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:01.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.205+0000 7f445a7fc700 1 -- 172.21.15.43:0/1437392598 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4460015070 con 0x7f4464107780 2024-04-21T08:29:01.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.424+0000 7f446b882700 1 -- 172.21.15.43:0/1437392598 --> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f4464062e80 con 0x7f44500396c0 2024-04-21T08:29:01.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.429+0000 7f445a7fc700 1 -- 172.21.15.43:0/1437392598 <== mgr.14118 v2:172.21.15.43:6800/3200866696 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4464062e80 con 0x7f44500396c0 2024-04-21T08:29:01.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.431+0000 7f446b882700 1 -- 172.21.15.43:0/1437392598 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f44500396c0 msgr2=0x7f445003bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:01.055 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.431+0000 7f446b882700 1 --2- 172.21.15.43:0/1437392598 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f44500396c0 0x7f445003bb80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4454006fd0 tx=0x7f4454006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:01.055 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.431+0000 7f446b882700 1 -- 172.21.15.43:0/1437392598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4464107780 msgr2=0x7f446419e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:01.055 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.431+0000 7f446b882700 1 --2- 172.21.15.43:0/1437392598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4464107780 0x7f446419e350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f446000b340 tx=0x7f4460003dc0 comp rx=0 tx=0).stop 2024-04-21T08:29:01.055 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.431+0000 7f446b882700 1 -- 172.21.15.43:0/1437392598 shutdown_connections 2024-04-21T08:29:01.055 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.431+0000 7f446b882700 1 --2- 172.21.15.43:0/1437392598 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f44500396c0 0x7f445003bb80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:01.055 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.431+0000 7f446b882700 1 --2- 172.21.15.43:0/1437392598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4464107780 0x7f446419e350 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:01.055 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.431+0000 7f446b882700 1 -- 172.21.15.43:0/1437392598 >> 172.21.15.43:0/1437392598 conn(0x7f4464102040 msgr2=0x7f4464190580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:01.055 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.431+0000 7f446b882700 1 -- 172.21.15.43:0/1437392598 shutdown_connections 2024-04-21T08:29:01.055 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.431+0000 7f446b882700 1 -- 172.21.15.43:0/1437392598 wait complete. 2024-04-21T08:29:01.055 INFO:teuthology.orchestra.run.smithi043.stdout:Deploying unmanaged mgr service... 2024-04-21T08:29:01.618 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:01 smithi043 ceph-mon[27978]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:29:01.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:01 smithi043 ceph-mon[27978]: Saving service mon spec with placement count:5 2024-04-21T08:29:01.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:01 smithi043 ceph-mon[27978]: from='mgr.14118 172.21.15.43:0/1296813352' entity='mgr.x' 2024-04-21T08:29:02.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-21T08:29:02.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.619+0000 7fe9def80700 1 Processor -- start 2024-04-21T08:29:02.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.619+0000 7fe9def80700 1 -- start start 2024-04-21T08:29:02.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.619+0000 7fe9def80700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d8108360 0x7fe9d8108780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:02.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.619+0000 7fe9def80700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d8108d50 con 0x7fe9d8108360 2024-04-21T08:29:02.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.620+0000 7fe9dcd1c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d8108360 0x7fe9d8108780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:02.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.620+0000 7fe9dcd1c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d8108360 0x7fe9d8108780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44982/0 (socket says 172.21.15.43:44982) 2024-04-21T08:29:02.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.620+0000 7fe9dcd1c700 1 -- 172.21.15.43:0/1987697907 learned_addr learned my addr 172.21.15.43:0/1987697907 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:02.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.620+0000 7fe9dcd1c700 1 -- 172.21.15.43:0/1987697907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d8109560 con 0x7fe9d8108360 2024-04-21T08:29:02.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.620+0000 7fe9dcd1c700 1 --2- 172.21.15.43:0/1987697907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d8108360 0x7fe9d8108780 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0009e80 tx=0x7fe9d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e071f4953130ecb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:02.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.620+0000 7fe9d77fe700 1 -- 172.21.15.43:0/1987697907 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9d0017070 con 0x7fe9d8108360 2024-04-21T08:29:02.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.620+0000 7fe9d77fe700 1 -- 172.21.15.43:0/1987697907 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe9d000b8c0 con 0x7fe9d8108360 2024-04-21T08:29:02.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.620+0000 7fe9d77fe700 1 -- 172.21.15.43:0/1987697907 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9d0004490 con 0x7fe9d8108360 2024-04-21T08:29:02.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.621+0000 7fe9def80700 1 -- 172.21.15.43:0/1987697907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d8108360 msgr2=0x7fe9d8108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:02.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.621+0000 7fe9def80700 1 --2- 172.21.15.43:0/1987697907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d8108360 0x7fe9d8108780 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0009e80 tx=0x7fe9d000b330 comp rx=0 tx=0).stop 2024-04-21T08:29:02.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.621+0000 7fe9def80700 1 -- 172.21.15.43:0/1987697907 shutdown_connections 2024-04-21T08:29:02.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.621+0000 7fe9def80700 1 --2- 172.21.15.43:0/1987697907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d8108360 0x7fe9d8108780 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:02.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.621+0000 7fe9def80700 1 -- 172.21.15.43:0/1987697907 >> 172.21.15.43:0/1987697907 conn(0x7fe9d8103920 msgr2=0x7fe9d8105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:02.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.621+0000 7fe9def80700 1 -- 172.21.15.43:0/1987697907 shutdown_connections 2024-04-21T08:29:02.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.621+0000 7fe9def80700 1 -- 172.21.15.43:0/1987697907 wait complete. 2024-04-21T08:29:02.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.622+0000 7fe9def80700 1 Processor -- start 2024-04-21T08:29:02.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.622+0000 7fe9def80700 1 -- start start 2024-04-21T08:29:02.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.622+0000 7fe9def80700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d8108360 0x7fe9d819c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:02.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.622+0000 7fe9def80700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d810c910 con 0x7fe9d8108360 2024-04-21T08:29:02.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.622+0000 7fe9dcd1c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d8108360 0x7fe9d819c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:02.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.622+0000 7fe9dcd1c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d8108360 0x7fe9d819c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44990/0 (socket says 172.21.15.43:44990) 2024-04-21T08:29:02.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.622+0000 7fe9dcd1c700 1 -- 172.21.15.43:0/1861124614 learned_addr learned my addr 172.21.15.43:0/1861124614 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:02.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.622+0000 7fe9dcd1c700 1 -- 172.21.15.43:0/1861124614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d819c6d0 con 0x7fe9d8108360 2024-04-21T08:29:02.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.622+0000 7fe9dcd1c700 1 --2- 172.21.15.43:0/1861124614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d8108360 0x7fe9d819c190 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe9d000b7b0 tx=0x7fe9d000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:02.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.623+0000 7fe9d5ffb700 1 -- 172.21.15.43:0/1861124614 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9d0020020 con 0x7fe9d8108360 2024-04-21T08:29:02.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.623+0000 7fe9d5ffb700 1 -- 172.21.15.43:0/1861124614 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe9d0012430 con 0x7fe9d8108360 2024-04-21T08:29:02.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.623+0000 7fe9def80700 1 -- 172.21.15.43:0/1861124614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d819c960 con 0x7fe9d8108360 2024-04-21T08:29:02.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.623+0000 7fe9d5ffb700 1 -- 172.21.15.43:0/1861124614 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9d0017040 con 0x7fe9d8108360 2024-04-21T08:29:02.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.623+0000 7fe9def80700 1 -- 172.21.15.43:0/1861124614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d819f650 con 0x7fe9d8108360 2024-04-21T08:29:02.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.624+0000 7fe9d5ffb700 1 -- 172.21.15.43:0/1861124614 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe9d0023020 con 0x7fe9d8108360 2024-04-21T08:29:02.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.624+0000 7fe9def80700 1 -- 172.21.15.43:0/1861124614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9d80640a0 con 0x7fe9d8108360 2024-04-21T08:29:02.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.624+0000 7fe9d5ffb700 1 --2- 172.21.15.43:0/1861124614 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fe9c0039670 0x7fe9c003bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:02.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.624+0000 7fe9d5ffb700 1 -- 172.21.15.43:0/1861124614 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe9d000f050 con 0x7fe9d8108360 2024-04-21T08:29:02.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.624+0000 7fe9d7fff700 1 --2- 172.21.15.43:0/1861124614 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fe9c0039670 0x7fe9c003bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:02.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.625+0000 7fe9d7fff700 1 --2- 172.21.15.43:0/1861124614 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fe9c0039670 0x7fe9c003bb30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc006fd0 tx=0x7fe9cc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:02.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.629+0000 7fe9d5ffb700 1 -- 172.21.15.43:0/1861124614 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe9d002a020 con 0x7fe9d8108360 2024-04-21T08:29:02.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.870+0000 7fe9def80700 1 -- 172.21.15.43:0/1861124614 --> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe9d8104a30 con 0x7fe9c0039670 2024-04-21T08:29:02.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.875+0000 7fe9d5ffb700 1 -- 172.21.15.43:0/1861124614 <== mgr.14118 v2:172.21.15.43:6800/3200866696 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe9d8104a30 con 0x7fe9c0039670 2024-04-21T08:29:02.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.877+0000 7fe9def80700 1 -- 172.21.15.43:0/1861124614 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fe9c0039670 msgr2=0x7fe9c003bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:02.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.877+0000 7fe9def80700 1 --2- 172.21.15.43:0/1861124614 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fe9c0039670 0x7fe9c003bb30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc006fd0 tx=0x7fe9cc006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:02.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.877+0000 7fe9def80700 1 -- 172.21.15.43:0/1861124614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d8108360 msgr2=0x7fe9d819c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:02.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.877+0000 7fe9def80700 1 --2- 172.21.15.43:0/1861124614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d8108360 0x7fe9d819c190 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe9d000b7b0 tx=0x7fe9d000be20 comp rx=0 tx=0).stop 2024-04-21T08:29:02.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.877+0000 7fe9def80700 1 -- 172.21.15.43:0/1861124614 shutdown_connections 2024-04-21T08:29:02.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.877+0000 7fe9def80700 1 --2- 172.21.15.43:0/1861124614 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fe9c0039670 0x7fe9c003bb30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:02.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.877+0000 7fe9def80700 1 --2- 172.21.15.43:0/1861124614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d8108360 0x7fe9d819c190 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:02.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.877+0000 7fe9def80700 1 -- 172.21.15.43:0/1861124614 >> 172.21.15.43:0/1861124614 conn(0x7fe9d8103920 msgr2=0x7fe9d8104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:02.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.877+0000 7fe9def80700 1 -- 172.21.15.43:0/1861124614 shutdown_connections 2024-04-21T08:29:02.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.877+0000 7fe9def80700 1 -- 172.21.15.43:0/1861124614 wait complete. 2024-04-21T08:29:02.906 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:02 smithi043 ceph-mon[27978]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:29:02.907 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:02 smithi043 ceph-mon[27978]: Saving service mgr spec with placement count:2 2024-04-21T08:29:02.907 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:02 smithi043 ceph-mon[27978]: from='mgr.14118 172.21.15.43:0/1296813352' entity='mgr.x' 2024-04-21T08:29:04.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.938+0000 7fc40e849700 1 Processor -- start 2024-04-21T08:29:04.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.938+0000 7fc40e849700 1 -- start start 2024-04-21T08:29:04.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.938+0000 7fc40e849700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4081088f0 0x7fc408108d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:04.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.938+0000 7fc40e849700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081092e0 con 0x7fc4081088f0 2024-04-21T08:29:04.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.938+0000 7fc407fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4081088f0 0x7fc408108d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:04.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.938+0000 7fc407fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4081088f0 0x7fc408108d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44996/0 (socket says 172.21.15.43:44996) 2024-04-21T08:29:04.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.938+0000 7fc407fff700 1 -- 172.21.15.43:0/1800584198 learned_addr learned my addr 172.21.15.43:0/1800584198 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:04.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.938+0000 7fc407fff700 1 -- 172.21.15.43:0/1800584198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4081063f0 con 0x7fc4081088f0 2024-04-21T08:29:04.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.938+0000 7fc407fff700 1 --2- 172.21.15.43:0/1800584198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4081088f0 0x7fc408108d10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc3f80098a0 tx=0x7fc3f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e07210cbbf94b5af server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:04.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.939+0000 7fc406ffd700 1 -- 172.21.15.43:0/1800584198 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3f8004160 con 0x7fc4081088f0 2024-04-21T08:29:04.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.939+0000 7fc406ffd700 1 -- 172.21.15.43:0/1800584198 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc3f8004300 con 0x7fc4081088f0 2024-04-21T08:29:04.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.939+0000 7fc406ffd700 1 -- 172.21.15.43:0/1800584198 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3f8011af0 con 0x7fc4081088f0 2024-04-21T08:29:04.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.939+0000 7fc40e849700 1 -- 172.21.15.43:0/1800584198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4081088f0 msgr2=0x7fc408108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:04.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.939+0000 7fc40e849700 1 --2- 172.21.15.43:0/1800584198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4081088f0 0x7fc408108d10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc3f80098a0 tx=0x7fc3f8009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:04.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.939+0000 7fc40e849700 1 -- 172.21.15.43:0/1800584198 shutdown_connections 2024-04-21T08:29:04.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.939+0000 7fc40e849700 1 --2- 172.21.15.43:0/1800584198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4081088f0 0x7fc408108d10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:04.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.939+0000 7fc40e849700 1 -- 172.21.15.43:0/1800584198 >> 172.21.15.43:0/1800584198 conn(0x7fc408101fb0 msgr2=0x7fc4081043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:04.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.939+0000 7fc40e849700 1 -- 172.21.15.43:0/1800584198 shutdown_connections 2024-04-21T08:29:04.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.939+0000 7fc40e849700 1 -- 172.21.15.43:0/1800584198 wait complete. 2024-04-21T08:29:04.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.940+0000 7fc40e849700 1 Processor -- start 2024-04-21T08:29:04.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.940+0000 7fc40e849700 1 -- start start 2024-04-21T08:29:04.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.940+0000 7fc40e849700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4081088f0 0x7fc40819e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:04.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.940+0000 7fc40e849700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc408107410 con 0x7fc4081088f0 2024-04-21T08:29:04.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.940+0000 7fc407fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4081088f0 0x7fc40819e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:04.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.941+0000 7fc407fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4081088f0 0x7fc40819e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44998/0 (socket says 172.21.15.43:44998) 2024-04-21T08:29:04.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.941+0000 7fc407fff700 1 -- 172.21.15.43:0/185396853 learned_addr learned my addr 172.21.15.43:0/185396853 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:04.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.941+0000 7fc407fff700 1 -- 172.21.15.43:0/185396853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40819e8e0 con 0x7fc4081088f0 2024-04-21T08:29:04.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.941+0000 7fc407fff700 1 --2- 172.21.15.43:0/185396853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4081088f0 0x7fc40819e3a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8003f40 tx=0x7fc3f8011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:04.014 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.941+0000 7fc4057fa700 1 -- 172.21.15.43:0/185396853 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3f80038b0 con 0x7fc4081088f0 2024-04-21T08:29:04.014 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.941+0000 7fc4057fa700 1 -- 172.21.15.43:0/185396853 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc3f8003a50 con 0x7fc4081088f0 2024-04-21T08:29:04.014 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.941+0000 7fc40e849700 1 -- 172.21.15.43:0/185396853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc40819eb70 con 0x7fc4081088f0 2024-04-21T08:29:04.014 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.941+0000 7fc4057fa700 1 -- 172.21.15.43:0/185396853 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3f801a5c0 con 0x7fc4081088f0 2024-04-21T08:29:04.014 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.941+0000 7fc40e849700 1 -- 172.21.15.43:0/185396853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4081a1860 con 0x7fc4081088f0 2024-04-21T08:29:04.014 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.942+0000 7fc4057fa700 1 -- 172.21.15.43:0/185396853 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fc3f801a870 con 0x7fc4081088f0 2024-04-21T08:29:04.014 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.942+0000 7fc40e849700 1 -- 172.21.15.43:0/185396853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4080640a0 con 0x7fc4081088f0 2024-04-21T08:29:04.014 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.942+0000 7fc4057fa700 1 --2- 172.21.15.43:0/185396853 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fc3f0039670 0x7fc3f003bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:04.014 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.942+0000 7fc4057fa700 1 -- 172.21.15.43:0/185396853 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc3f804dab0 con 0x7fc4081088f0 2024-04-21T08:29:04.014 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.942+0000 7fc4077fe700 1 --2- 172.21.15.43:0/185396853 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fc3f0039670 0x7fc3f003bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:04.014 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.943+0000 7fc4077fe700 1 --2- 172.21.15.43:0/185396853 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fc3f0039670 0x7fc3f003bb30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc006fd0 tx=0x7fc3fc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:04.014 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.947+0000 7fc4057fa700 1 -- 172.21.15.43:0/185396853 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc3f802d3f0 con 0x7fc4081088f0 2024-04-21T08:29:04.014 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.159+0000 7fc40e849700 1 -- 172.21.15.43:0/185396853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fc4081979a0 con 0x7fc4081088f0 2024-04-21T08:29:04.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.161+0000 7fc4057fa700 1 -- 172.21.15.43:0/185396853 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7fc4081979a0 con 0x7fc4081088f0 2024-04-21T08:29:04.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.164+0000 7fc40e849700 1 -- 172.21.15.43:0/185396853 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fc3f0039670 msgr2=0x7fc3f003bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:04.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.164+0000 7fc40e849700 1 --2- 172.21.15.43:0/185396853 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fc3f0039670 0x7fc3f003bb30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc006fd0 tx=0x7fc3fc006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:04.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.164+0000 7fc40e849700 1 -- 172.21.15.43:0/185396853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4081088f0 msgr2=0x7fc40819e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:04.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.164+0000 7fc40e849700 1 --2- 172.21.15.43:0/185396853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4081088f0 0x7fc40819e3a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8003f40 tx=0x7fc3f8011890 comp rx=0 tx=0).stop 2024-04-21T08:29:04.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.164+0000 7fc40e849700 1 -- 172.21.15.43:0/185396853 shutdown_connections 2024-04-21T08:29:04.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.164+0000 7fc40e849700 1 --2- 172.21.15.43:0/185396853 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fc3f0039670 0x7fc3f003bb30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:04.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.164+0000 7fc40e849700 1 --2- 172.21.15.43:0/185396853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4081088f0 0x7fc40819e3a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:04.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.164+0000 7fc40e849700 1 -- 172.21.15.43:0/185396853 >> 172.21.15.43:0/185396853 conn(0x7fc408101fb0 msgr2=0x7fc408194a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:04.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.164+0000 7fc40e849700 1 -- 172.21.15.43:0/185396853 shutdown_connections 2024-04-21T08:29:04.015 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.164+0000 7fc40e849700 1 -- 172.21.15.43:0/185396853 wait complete. 2024-04-21T08:29:04.306 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:04 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/185396853' entity='client.admin' 2024-04-21T08:29:05.445 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.413+0000 7f4f208c9700 1 Processor -- start 2024-04-21T08:29:05.445 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.413+0000 7f4f208c9700 1 -- start start 2024-04-21T08:29:05.445 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.413+0000 7f4f208c9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c106c80 0x7f4f1c1070a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:05.445 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.413+0000 7f4f208c9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c07be20 con 0x7f4f1c106c80 2024-04-21T08:29:05.445 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.413+0000 7f4f1a59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c106c80 0x7f4f1c1070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:05.446 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.413+0000 7f4f1a59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c106c80 0x7f4f1c1070a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45008/0 (socket says 172.21.15.43:45008) 2024-04-21T08:29:05.446 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.413+0000 7f4f1a59c700 1 -- 172.21.15.43:0/349986247 learned_addr learned my addr 172.21.15.43:0/349986247 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:05.446 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.414+0000 7f4f1a59c700 1 -- 172.21.15.43:0/349986247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1c1075e0 con 0x7f4f1c106c80 2024-04-21T08:29:05.446 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.414+0000 7f4f1a59c700 1 --2- 172.21.15.43:0/349986247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c106c80 0x7f4f1c1070a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4f100098a0 tx=0x7f4f10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432d57e49f16a10e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:05.446 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.414+0000 7f4f1959a700 1 -- 172.21.15.43:0/349986247 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f10004160 con 0x7f4f1c106c80 2024-04-21T08:29:05.446 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.414+0000 7f4f1959a700 1 -- 172.21.15.43:0/349986247 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4f10004300 con 0x7f4f1c106c80 2024-04-21T08:29:05.446 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.414+0000 7f4f1959a700 1 -- 172.21.15.43:0/349986247 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f10011af0 con 0x7f4f1c106c80 2024-04-21T08:29:05.446 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.415+0000 7f4f208c9700 1 -- 172.21.15.43:0/349986247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c106c80 msgr2=0x7f4f1c1070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:05.446 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.415+0000 7f4f208c9700 1 --2- 172.21.15.43:0/349986247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c106c80 0x7f4f1c1070a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4f100098a0 tx=0x7f4f10009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:05.446 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.415+0000 7f4f208c9700 1 -- 172.21.15.43:0/349986247 shutdown_connections 2024-04-21T08:29:05.446 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.415+0000 7f4f208c9700 1 --2- 172.21.15.43:0/349986247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c106c80 0x7f4f1c1070a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:05.446 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.415+0000 7f4f208c9700 1 -- 172.21.15.43:0/349986247 >> 172.21.15.43:0/349986247 conn(0x7f4f1c1028e0 msgr2=0x7f4f1c104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:05.446 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.415+0000 7f4f208c9700 1 -- 172.21.15.43:0/349986247 shutdown_connections 2024-04-21T08:29:05.447 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.415+0000 7f4f208c9700 1 -- 172.21.15.43:0/349986247 wait complete. 2024-04-21T08:29:05.447 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.416+0000 7f4f208c9700 1 Processor -- start 2024-04-21T08:29:05.447 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.416+0000 7f4f208c9700 1 -- start start 2024-04-21T08:29:05.447 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.416+0000 7f4f208c9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c106c80 0x7f4f1c19e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:05.447 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.416+0000 7f4f208c9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c1085a0 con 0x7f4f1c106c80 2024-04-21T08:29:05.447 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.416+0000 7f4f1a59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c106c80 0x7f4f1c19e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:05.447 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.416+0000 7f4f1a59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c106c80 0x7f4f1c19e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45018/0 (socket says 172.21.15.43:45018) 2024-04-21T08:29:05.447 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.416+0000 7f4f1a59c700 1 -- 172.21.15.43:0/751170402 learned_addr learned my addr 172.21.15.43:0/751170402 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:05.447 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.416+0000 7f4f1a59c700 1 -- 172.21.15.43:0/751170402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1c19e900 con 0x7f4f1c106c80 2024-04-21T08:29:05.447 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.417+0000 7f4f1a59c700 1 --2- 172.21.15.43:0/751170402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c106c80 0x7f4f1c19e3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4f10003f40 tx=0x7f4f10011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:05.447 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.417+0000 7f4f0b7fe700 1 -- 172.21.15.43:0/751170402 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f100038b0 con 0x7f4f1c106c80 2024-04-21T08:29:05.447 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.417+0000 7f4f0b7fe700 1 -- 172.21.15.43:0/751170402 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4f10003a50 con 0x7f4f1c106c80 2024-04-21T08:29:05.447 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.417+0000 7f4f208c9700 1 -- 172.21.15.43:0/751170402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1c19eb90 con 0x7f4f1c106c80 2024-04-21T08:29:05.447 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.417+0000 7f4f0b7fe700 1 -- 172.21.15.43:0/751170402 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f1001a5c0 con 0x7f4f1c106c80 2024-04-21T08:29:05.448 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.417+0000 7f4f208c9700 1 -- 172.21.15.43:0/751170402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f1c1a1880 con 0x7f4f1c106c80 2024-04-21T08:29:05.448 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.418+0000 7f4f208c9700 1 -- 172.21.15.43:0/751170402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f1c0640a0 con 0x7f4f1c106c80 2024-04-21T08:29:05.448 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.418+0000 7f4f0b7fe700 1 -- 172.21.15.43:0/751170402 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4f10024430 con 0x7f4f1c106c80 2024-04-21T08:29:05.448 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.418+0000 7f4f0b7fe700 1 --2- 172.21.15.43:0/751170402 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f4f04039670 0x7f4f0403bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:05.448 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.418+0000 7f4f0b7fe700 1 -- 172.21.15.43:0/751170402 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4f1004da80 con 0x7f4f1c106c80 2024-04-21T08:29:05.448 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.418+0000 7f4f19d9b700 1 --2- 172.21.15.43:0/751170402 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f4f04039670 0x7f4f0403bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:05.448 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.419+0000 7f4f19d9b700 1 --2- 172.21.15.43:0/751170402 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f4f04039670 0x7f4f0403bb30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c006fd0 tx=0x7f4f0c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:05.448 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.423+0000 7f4f0b7fe700 1 -- 172.21.15.43:0/751170402 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4f1002d910 con 0x7f4f1c106c80 2024-04-21T08:29:05.448 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.634+0000 7f4f208c9700 1 -- 172.21.15.43:0/751170402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f4f1c0008d0 con 0x7f4f1c106c80 2024-04-21T08:29:05.448 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.636+0000 7f4f0b7fe700 1 -- 172.21.15.43:0/751170402 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v7)=0 v7) ==== 130+0+0 (secure 0 0 0) 0x7f4f1002d3f0 con 0x7f4f1c106c80 2024-04-21T08:29:05.448 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.638+0000 7f4f208c9700 1 -- 172.21.15.43:0/751170402 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f4f04039670 msgr2=0x7f4f0403bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:05.448 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.638+0000 7f4f208c9700 1 --2- 172.21.15.43:0/751170402 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f4f04039670 0x7f4f0403bb30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c006fd0 tx=0x7f4f0c006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:05.448 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.638+0000 7f4f208c9700 1 -- 172.21.15.43:0/751170402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c106c80 msgr2=0x7f4f1c19e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:05.448 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.638+0000 7f4f208c9700 1 --2- 172.21.15.43:0/751170402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c106c80 0x7f4f1c19e3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4f10003f40 tx=0x7f4f10011890 comp rx=0 tx=0).stop 2024-04-21T08:29:05.449 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.638+0000 7f4f208c9700 1 -- 172.21.15.43:0/751170402 shutdown_connections 2024-04-21T08:29:05.449 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.639+0000 7f4f208c9700 1 --2- 172.21.15.43:0/751170402 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f4f04039670 0x7f4f0403bb30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:05.449 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.639+0000 7f4f208c9700 1 --2- 172.21.15.43:0/751170402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c106c80 0x7f4f1c19e3c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:05.449 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.639+0000 7f4f208c9700 1 -- 172.21.15.43:0/751170402 >> 172.21.15.43:0/751170402 conn(0x7f4f1c1028e0 msgr2=0x7f4f1c192710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:05.449 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.639+0000 7f4f208c9700 1 -- 172.21.15.43:0/751170402 shutdown_connections 2024-04-21T08:29:05.449 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.639+0000 7f4f208c9700 1 -- 172.21.15.43:0/751170402 wait complete. 2024-04-21T08:29:05.449 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling the dashboard module... 2024-04-21T08:29:05.692 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:05 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/751170402' entity='client.admin' 2024-04-21T08:29:05.692 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:05 smithi043 ceph-mon[27978]: from='mgr.14118 172.21.15.43:0/1296813352' entity='mgr.x' 2024-04-21T08:29:05.692 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:05 smithi043 ceph-mon[27978]: from='mgr.14118 172.21.15.43:0/1296813352' entity='mgr.x' 2024-04-21T08:29:06.756 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:06 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3542667679' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-21T08:29:06.756 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:06 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: ignoring --setuser ceph since I am not root 2024-04-21T08:29:06.756 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:06 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: ignoring --setgroup ceph since I am not root 2024-04-21T08:29:07.014 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:06 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:06.823+0000 7f0760cc6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:29:07.014 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:07 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:07.015+0000 7f0760cc6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:29:07.311 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:07 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:07.131+0000 7f0760cc6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:29:07.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.888+0000 7f634a78f700 1 Processor -- start 2024-04-21T08:29:07.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.888+0000 7f634a78f700 1 -- start start 2024-04-21T08:29:07.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.888+0000 7f634a78f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6344108e60 0x7f634410b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:07.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.888+0000 7f634a78f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f634407be60 con 0x7f6344108e60 2024-04-21T08:29:07.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.888+0000 7f6343fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6344108e60 0x7f634410b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:07.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.888+0000 7f6343fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6344108e60 0x7f634410b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45020/0 (socket says 172.21.15.43:45020) 2024-04-21T08:29:07.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.888+0000 7f6343fff700 1 -- 172.21.15.43:0/883511196 learned_addr learned my addr 172.21.15.43:0/883511196 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:07.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.889+0000 7f6343fff700 1 -- 172.21.15.43:0/883511196 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634410b790 con 0x7f6344108e60 2024-04-21T08:29:07.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.889+0000 7f6343fff700 1 --2- 172.21.15.43:0/883511196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6344108e60 0x7f634410b250 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f63340098a0 tx=0x7f6334009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe25fe7268703ed server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:07.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.889+0000 7f6342ffd700 1 -- 172.21.15.43:0/883511196 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6334004160 con 0x7f6344108e60 2024-04-21T08:29:07.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.889+0000 7f6342ffd700 1 -- 172.21.15.43:0/883511196 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6334004300 con 0x7f6344108e60 2024-04-21T08:29:07.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.889+0000 7f6342ffd700 1 -- 172.21.15.43:0/883511196 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6334011a20 con 0x7f6344108e60 2024-04-21T08:29:07.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.890+0000 7f634a78f700 1 -- 172.21.15.43:0/883511196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6344108e60 msgr2=0x7f634410b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:07.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.890+0000 7f634a78f700 1 --2- 172.21.15.43:0/883511196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6344108e60 0x7f634410b250 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f63340098a0 tx=0x7f6334009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:07.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.890+0000 7f634a78f700 1 -- 172.21.15.43:0/883511196 shutdown_connections 2024-04-21T08:29:07.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.890+0000 7f634a78f700 1 --2- 172.21.15.43:0/883511196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6344108e60 0x7f634410b250 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:07.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.890+0000 7f634a78f700 1 -- 172.21.15.43:0/883511196 >> 172.21.15.43:0/883511196 conn(0x7f6344102890 msgr2=0x7f6344104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:07.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.890+0000 7f634a78f700 1 -- 172.21.15.43:0/883511196 shutdown_connections 2024-04-21T08:29:07.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.890+0000 7f634a78f700 1 -- 172.21.15.43:0/883511196 wait complete. 2024-04-21T08:29:07.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.890+0000 7f634a78f700 1 Processor -- start 2024-04-21T08:29:07.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.890+0000 7f634a78f700 1 -- start start 2024-04-21T08:29:07.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.891+0000 7f634a78f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6344108e60 0x7f634419c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:07.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.891+0000 7f634a78f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f634410c790 con 0x7f6344108e60 2024-04-21T08:29:07.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.891+0000 7f6343fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6344108e60 0x7f634419c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:07.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.891+0000 7f6343fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6344108e60 0x7f634419c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45030/0 (socket says 172.21.15.43:45030) 2024-04-21T08:29:07.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.891+0000 7f6343fff700 1 -- 172.21.15.43:0/3542667679 learned_addr learned my addr 172.21.15.43:0/3542667679 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:07.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.891+0000 7f6343fff700 1 -- 172.21.15.43:0/3542667679 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634419c6d0 con 0x7f6344108e60 2024-04-21T08:29:07.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.891+0000 7f6343fff700 1 --2- 172.21.15.43:0/3542667679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6344108e60 0x7f634419c190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f63340116e0 tx=0x7f6334011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:07.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.892+0000 7f63417fa700 1 -- 172.21.15.43:0/3542667679 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6334003a60 con 0x7f6344108e60 2024-04-21T08:29:07.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.892+0000 7f63417fa700 1 -- 172.21.15.43:0/3542667679 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f633401b8f0 con 0x7f6344108e60 2024-04-21T08:29:07.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.892+0000 7f634a78f700 1 -- 172.21.15.43:0/3542667679 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634419c960 con 0x7f6344108e60 2024-04-21T08:29:07.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.892+0000 7f63417fa700 1 -- 172.21.15.43:0/3542667679 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6334019860 con 0x7f6344108e60 2024-04-21T08:29:07.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.892+0000 7f634a78f700 1 -- 172.21.15.43:0/3542667679 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f634419f650 con 0x7f6344108e60 2024-04-21T08:29:07.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.893+0000 7f63417fa700 1 -- 172.21.15.43:0/3542667679 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6334019b60 con 0x7f6344108e60 2024-04-21T08:29:07.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.893+0000 7f63417fa700 1 --2- 172.21.15.43:0/3542667679 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f632c039620 0x7f632c03bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:07.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.893+0000 7f63417fa700 1 -- 172.21.15.43:0/3542667679 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f633404dd20 con 0x7f6344108e60 2024-04-21T08:29:07.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.893+0000 7f63437fe700 1 --2- 172.21.15.43:0/3542667679 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f632c039620 0x7f632c03bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:07.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.893+0000 7f634a78f700 1 -- 172.21.15.43:0/3542667679 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63440640a0 con 0x7f6344108e60 2024-04-21T08:29:07.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.893+0000 7f63437fe700 1 --2- 172.21.15.43:0/3542667679 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f632c039620 0x7f632c03bae0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6338006fd0 tx=0x7f6338006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:07.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:05.898+0000 7f63417fa700 1 -- 172.21.15.43:0/3542667679 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f633401f030 con 0x7f6344108e60 2024-04-21T08:29:07.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:06.184+0000 7f634a78f700 1 -- 172.21.15.43:0/3542667679 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f634419fa40 con 0x7f6344108e60 2024-04-21T08:29:07.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:06.638+0000 7f63417fa700 1 -- 172.21.15.43:0/3542667679 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f6334029260 con 0x7f6344108e60 2024-04-21T08:29:07.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:06.639+0000 7f63417fa700 1 -- 172.21.15.43:0/3542667679 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f633401a8f0 con 0x7f6344108e60 2024-04-21T08:29:07.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:06.641+0000 7f634a78f700 1 -- 172.21.15.43:0/3542667679 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f632c039620 msgr2=0x7f632c03bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:07.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:06.641+0000 7f634a78f700 1 --2- 172.21.15.43:0/3542667679 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f632c039620 0x7f632c03bae0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6338006fd0 tx=0x7f6338006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:07.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:06.641+0000 7f634a78f700 1 -- 172.21.15.43:0/3542667679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6344108e60 msgr2=0x7f634419c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:07.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:06.641+0000 7f634a78f700 1 --2- 172.21.15.43:0/3542667679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6344108e60 0x7f634419c190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f63340116e0 tx=0x7f6334011aa0 comp rx=0 tx=0).stop 2024-04-21T08:29:07.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:06.641+0000 7f634a78f700 1 -- 172.21.15.43:0/3542667679 shutdown_connections 2024-04-21T08:29:07.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:06.641+0000 7f634a78f700 1 --2- 172.21.15.43:0/3542667679 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f632c039620 0x7f632c03bae0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:07.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:06.641+0000 7f634a78f700 1 --2- 172.21.15.43:0/3542667679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6344108e60 0x7f634419c190 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:07.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:06.641+0000 7f634a78f700 1 -- 172.21.15.43:0/3542667679 >> 172.21.15.43:0/3542667679 conn(0x7f6344102890 msgr2=0x7f6344103250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:07.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:06.641+0000 7f634a78f700 1 -- 172.21.15.43:0/3542667679 shutdown_connections 2024-04-21T08:29:07.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:06.641+0000 7f634a78f700 1 -- 172.21.15.43:0/3542667679 wait complete. 2024-04-21T08:29:07.692 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:07 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3542667679' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-21T08:29:07.692 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:07 smithi043 ceph-mon[27978]: mgrmap e9: x(active, since 19s) 2024-04-21T08:29:07.692 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:07 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:07.459+0000 7f0760cc6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:29:07.693 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:07 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:07.547+0000 7f0760cc6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:29:08.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:08 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2592878215' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-21T08:29:08.848 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:29:08.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-21T08:29:08.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-21T08:29:08.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-21T08:29:08.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-21T08:29:08.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:29:08.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.912+0000 7f6d0f9ca700 1 Processor -- start 2024-04-21T08:29:08.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.912+0000 7f6d0f9ca700 1 -- start start 2024-04-21T08:29:08.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.913+0000 7f6d0f9ca700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d08108e10 0x7f6d0810b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:08.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.913+0000 7f6d0f9ca700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0807bea0 con 0x7f6d08108e10 2024-04-21T08:29:08.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.913+0000 7f6d0d766700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d08108e10 0x7f6d0810b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:08.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.913+0000 7f6d0d766700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d08108e10 0x7f6d0810b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55678/0 (socket says 172.21.15.43:55678) 2024-04-21T08:29:08.851 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.913+0000 7f6d0d766700 1 -- 172.21.15.43:0/3393233990 learned_addr learned my addr 172.21.15.43:0/3393233990 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:08.851 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.913+0000 7f6d0d766700 1 -- 172.21.15.43:0/3393233990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0810b740 con 0x7f6d08108e10 2024-04-21T08:29:08.851 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.913+0000 7f6d0d766700 1 --2- 172.21.15.43:0/3393233990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d08108e10 0x7f6d0810b200 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80098a0 tx=0x7f6cf8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a9ca85b9b87d41 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:08.851 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.913+0000 7f6cfffff700 1 -- 172.21.15.43:0/3393233990 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cf8004160 con 0x7f6d08108e10 2024-04-21T08:29:08.851 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.913+0000 7f6cfffff700 1 -- 172.21.15.43:0/3393233990 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cf8004300 con 0x7f6d08108e10 2024-04-21T08:29:08.851 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.914+0000 7f6cfffff700 1 -- 172.21.15.43:0/3393233990 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cf8011a20 con 0x7f6d08108e10 2024-04-21T08:29:08.852 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.914+0000 7f6d0f9ca700 1 -- 172.21.15.43:0/3393233990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d08108e10 msgr2=0x7f6d0810b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:08.852 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.914+0000 7f6d0f9ca700 1 --2- 172.21.15.43:0/3393233990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d08108e10 0x7f6d0810b200 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80098a0 tx=0x7f6cf8009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:08.852 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.914+0000 7f6d0f9ca700 1 -- 172.21.15.43:0/3393233990 shutdown_connections 2024-04-21T08:29:08.852 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.914+0000 7f6d0f9ca700 1 --2- 172.21.15.43:0/3393233990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d08108e10 0x7f6d0810b200 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:08.852 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.914+0000 7f6d0f9ca700 1 -- 172.21.15.43:0/3393233990 >> 172.21.15.43:0/3393233990 conn(0x7f6d081028e0 msgr2=0x7f6d08104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:08.852 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.914+0000 7f6d0f9ca700 1 -- 172.21.15.43:0/3393233990 shutdown_connections 2024-04-21T08:29:08.853 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.914+0000 7f6d0f9ca700 1 -- 172.21.15.43:0/3393233990 wait complete. 2024-04-21T08:29:08.853 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.915+0000 7f6d0f9ca700 1 Processor -- start 2024-04-21T08:29:08.853 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.915+0000 7f6d0f9ca700 1 -- start start 2024-04-21T08:29:08.853 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.915+0000 7f6d0f9ca700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d08108e10 0x7f6d0819c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:08.853 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.915+0000 7f6d0f9ca700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0810c750 con 0x7f6d08108e10 2024-04-21T08:29:08.853 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.915+0000 7f6d0d766700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d08108e10 0x7f6d0819c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:08.854 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.915+0000 7f6d0d766700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d08108e10 0x7f6d0819c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55686/0 (socket says 172.21.15.43:55686) 2024-04-21T08:29:08.854 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.915+0000 7f6d0d766700 1 -- 172.21.15.43:0/2592878215 learned_addr learned my addr 172.21.15.43:0/2592878215 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:08.854 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.916+0000 7f6d0d766700 1 -- 172.21.15.43:0/2592878215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0819c6a0 con 0x7f6d08108e10 2024-04-21T08:29:08.854 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.916+0000 7f6d0d766700 1 --2- 172.21.15.43:0/2592878215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d08108e10 0x7f6d0819c160 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80116e0 tx=0x7f6cf8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:08.854 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.916+0000 7f6cfe7fc700 1 -- 172.21.15.43:0/2592878215 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cf8003a60 con 0x7f6d08108e10 2024-04-21T08:29:08.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.916+0000 7f6cfe7fc700 1 -- 172.21.15.43:0/2592878215 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cf801b8f0 con 0x7f6d08108e10 2024-04-21T08:29:08.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.916+0000 7f6d0f9ca700 1 -- 172.21.15.43:0/2592878215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d0819c930 con 0x7f6d08108e10 2024-04-21T08:29:08.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.916+0000 7f6cfe7fc700 1 -- 172.21.15.43:0/2592878215 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cf8019860 con 0x7f6d08108e10 2024-04-21T08:29:08.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.916+0000 7f6d0f9ca700 1 -- 172.21.15.43:0/2592878215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d0819f620 con 0x7f6d08108e10 2024-04-21T08:29:08.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.917+0000 7f6cfe7fc700 1 -- 172.21.15.43:0/2592878215 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f6cf8019b60 con 0x7f6d08108e10 2024-04-21T08:29:08.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.917+0000 7f6d0f9ca700 1 -- 172.21.15.43:0/2592878215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d080640a0 con 0x7f6d08108e10 2024-04-21T08:29:08.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.917+0000 7f6cfe7fc700 1 --2- 172.21.15.43:0/2592878215 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f6cf4039670 0x7f6cf403bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:08.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.917+0000 7f6d0cf65700 1 -- 172.21.15.43:0/2592878215 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f6cf4039670 msgr2=0x7f6cf403bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3200866696 2024-04-21T08:29:08.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.917+0000 7f6cfe7fc700 1 -- 172.21.15.43:0/2592878215 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6cf804dd10 con 0x7f6d08108e10 2024-04-21T08:29:08.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.917+0000 7f6d0cf65700 1 --2- 172.21.15.43:0/2592878215 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f6cf4039670 0x7f6cf403bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:29:08.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:07.922+0000 7f6cfe7fc700 1 -- 172.21.15.43:0/2592878215 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6cf801f030 con 0x7f6d08108e10 2024-04-21T08:29:08.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.118+0000 7f6d0cf65700 1 -- 172.21.15.43:0/2592878215 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f6cf4039670 msgr2=0x7f6cf403bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3200866696 2024-04-21T08:29:08.857 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.118+0000 7f6d0cf65700 1 --2- 172.21.15.43:0/2592878215 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f6cf4039670 0x7f6cf403bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:29:08.857 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.235+0000 7f6d0f9ca700 1 -- 172.21.15.43:0/2592878215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f6d0819fbd0 con 0x7f6d08108e10 2024-04-21T08:29:08.857 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.235+0000 7f6cfe7fc700 1 -- 172.21.15.43:0/2592878215 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f6cf8029260 con 0x7f6d08108e10 2024-04-21T08:29:08.857 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.238+0000 7f6d0f9ca700 1 -- 172.21.15.43:0/2592878215 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f6cf4039670 msgr2=0x7f6cf403bb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:29:08.857 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.238+0000 7f6d0f9ca700 1 --2- 172.21.15.43:0/2592878215 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f6cf4039670 0x7f6cf403bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:08.857 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.238+0000 7f6d0f9ca700 1 -- 172.21.15.43:0/2592878215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d08108e10 msgr2=0x7f6d0819c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:08.857 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.238+0000 7f6d0f9ca700 1 --2- 172.21.15.43:0/2592878215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d08108e10 0x7f6d0819c160 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80116e0 tx=0x7f6cf8011aa0 comp rx=0 tx=0).stop 2024-04-21T08:29:08.857 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.238+0000 7f6d0f9ca700 1 -- 172.21.15.43:0/2592878215 shutdown_connections 2024-04-21T08:29:08.858 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.238+0000 7f6d0f9ca700 1 --2- 172.21.15.43:0/2592878215 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7f6cf4039670 0x7f6cf403bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:08.858 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.238+0000 7f6d0f9ca700 1 --2- 172.21.15.43:0/2592878215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d08108e10 0x7f6d0819c160 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:08.858 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.238+0000 7f6d0f9ca700 1 -- 172.21.15.43:0/2592878215 >> 172.21.15.43:0/2592878215 conn(0x7f6d081028e0 msgr2=0x7f6d08103270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:08.858 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.238+0000 7f6d0f9ca700 1 -- 172.21.15.43:0/2592878215 shutdown_connections 2024-04-21T08:29:08.858 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.238+0000 7f6d0f9ca700 1 -- 172.21.15.43:0/2592878215 wait complete. 2024-04-21T08:29:08.859 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for the mgr to restart... 2024-04-21T08:29:08.859 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr epoch 9... 2024-04-21T08:29:09.093 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:08 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:08.821+0000 7f0760cc6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:29:09.094 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:09 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:09.009+0000 7f0760cc6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:29:09.500 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:09 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:09.187+0000 7f0760cc6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:29:09.500 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:09 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:09.501+0000 7f0760cc6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:29:09.808 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:09 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:09.599+0000 7f0760cc6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:29:09.809 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:09 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:09.809+0000 7f0760cc6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:29:10.061 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:10 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:10.011+0000 7f0760cc6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:29:11.061 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:10 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:10.641+0000 7f0760cc6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:29:11.394 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:11 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:11.084+0000 7f0760cc6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:29:11.394 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:11 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:11.184+0000 7f0760cc6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:29:11.394 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:11 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:11.394+0000 7f0760cc6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:29:12.311 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:11 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:11.965+0000 7f0760cc6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:29:12.311 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:12 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:12.062+0000 7f0760cc6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:29:12.681 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:12 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:12.318+0000 7f0760cc6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:29:12.681 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:12 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:12.458+0000 7f0760cc6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:29:12.681 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:12 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:12.558+0000 7f0760cc6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:29:12.681 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:12 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:12.681+0000 7f0760cc6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:29:13.311 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:13 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:13.022+0000 7f0760cc6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:29:13.939 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:13 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:13.656+0000 7f0760cc6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:29:13.939 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:29:13 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:29:13.887+0000 7f0760cc6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:29:14.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:13 smithi043 ceph-mon[27978]: Active manager daemon x restarted 2024-04-21T08:29:14.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:13 smithi043 ceph-mon[27978]: Activating manager daemon x 2024-04-21T08:29:14.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:13 smithi043 ceph-mon[27978]: osdmap e3: 0 total, 0 up, 0 in 2024-04-21T08:29:14.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:13 smithi043 ceph-mon[27978]: mgrmap e10: x(active, starting, since 0.00336625s) 2024-04-21T08:29:14.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:13 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:29:14.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:13 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T08:29:14.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:13 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T08:29:14.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:13 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T08:29:14.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:13 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T08:29:14.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:13 smithi043 ceph-mon[27978]: Manager daemon x is now available 2024-04-21T08:29:14.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:13 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:29:15.201 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T08:29:15.201 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T08:29:15.201 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:14 smithi043 ceph-mon[27978]: mgrmap e11: x(active, since 1.00678s) 2024-04-21T08:29:15.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:29:15.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-21T08:29:15.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-21T08:29:15.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:29:15.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.325+0000 7fcb5ef6c700 1 Processor -- start 2024-04-21T08:29:15.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.325+0000 7fcb5ef6c700 1 -- start start 2024-04-21T08:29:15.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.326+0000 7fcb5ef6c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb58107300 0x7fcb58107720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:15.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.326+0000 7fcb5ef6c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcb58107cf0 con 0x7fcb58107300 2024-04-21T08:29:15.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.326+0000 7fcb5cd08700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb58107300 0x7fcb58107720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:15.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.326+0000 7fcb5cd08700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb58107300 0x7fcb58107720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55690/0 (socket says 172.21.15.43:55690) 2024-04-21T08:29:15.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.326+0000 7fcb5cd08700 1 -- 172.21.15.43:0/2434642838 learned_addr learned my addr 172.21.15.43:0/2434642838 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:15.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.326+0000 7fcb5cd08700 1 -- 172.21.15.43:0/2434642838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb58108500 con 0x7fcb58107300 2024-04-21T08:29:15.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.326+0000 7fcb5cd08700 1 --2- 172.21.15.43:0/2434642838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb58107300 0x7fcb58107720 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00bb40 tx=0x7fcb4c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73837ec72de43941 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:15.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.326+0000 7fcb577fe700 1 -- 172.21.15.43:0/2434642838 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb4c00dbe0 con 0x7fcb58107300 2024-04-21T08:29:15.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.326+0000 7fcb577fe700 1 -- 172.21.15.43:0/2434642838 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb4c0044e0 con 0x7fcb58107300 2024-04-21T08:29:15.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.326+0000 7fcb577fe700 1 -- 172.21.15.43:0/2434642838 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb4c010470 con 0x7fcb58107300 2024-04-21T08:29:15.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.327+0000 7fcb5ef6c700 1 -- 172.21.15.43:0/2434642838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb58107300 msgr2=0x7fcb58107720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:15.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.327+0000 7fcb5ef6c700 1 --2- 172.21.15.43:0/2434642838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb58107300 0x7fcb58107720 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00bb40 tx=0x7fcb4c00be50 comp rx=0 tx=0).stop 2024-04-21T08:29:15.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.327+0000 7fcb5ef6c700 1 -- 172.21.15.43:0/2434642838 shutdown_connections 2024-04-21T08:29:15.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.327+0000 7fcb5ef6c700 1 --2- 172.21.15.43:0/2434642838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb58107300 0x7fcb58107720 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:15.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.327+0000 7fcb5ef6c700 1 -- 172.21.15.43:0/2434642838 >> 172.21.15.43:0/2434642838 conn(0x7fcb581028e0 msgr2=0x7fcb58104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:15.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.327+0000 7fcb5ef6c700 1 -- 172.21.15.43:0/2434642838 shutdown_connections 2024-04-21T08:29:15.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.327+0000 7fcb5ef6c700 1 -- 172.21.15.43:0/2434642838 wait complete. 2024-04-21T08:29:15.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.328+0000 7fcb5ef6c700 1 Processor -- start 2024-04-21T08:29:15.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.328+0000 7fcb5ef6c700 1 -- start start 2024-04-21T08:29:15.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.328+0000 7fcb5ef6c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb58107300 0x7fcb58199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:15.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.328+0000 7fcb5ef6c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5807afc0 con 0x7fcb58107300 2024-04-21T08:29:15.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.328+0000 7fcb5cd08700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb58107300 0x7fcb58199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:15.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.328+0000 7fcb5cd08700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb58107300 0x7fcb58199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55706/0 (socket says 172.21.15.43:55706) 2024-04-21T08:29:15.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.328+0000 7fcb5cd08700 1 -- 172.21.15.43:0/984289636 learned_addr learned my addr 172.21.15.43:0/984289636 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:15.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.328+0000 7fcb5cd08700 1 -- 172.21.15.43:0/984289636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5819a520 con 0x7fcb58107300 2024-04-21T08:29:15.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.329+0000 7fcb5cd08700 1 --2- 172.21.15.43:0/984289636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb58107300 0x7fcb58199fe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00d650 tx=0x7fcb4c003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:15.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.329+0000 7fcb55ffb700 1 -- 172.21.15.43:0/984289636 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb4c027070 con 0x7fcb58107300 2024-04-21T08:29:15.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.329+0000 7fcb55ffb700 1 -- 172.21.15.43:0/984289636 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb4c010c20 con 0x7fcb58107300 2024-04-21T08:29:15.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.329+0000 7fcb5ef6c700 1 -- 172.21.15.43:0/984289636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb5819a7b0 con 0x7fcb58107300 2024-04-21T08:29:15.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.329+0000 7fcb55ffb700 1 -- 172.21.15.43:0/984289636 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb4c022b10 con 0x7fcb58107300 2024-04-21T08:29:15.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.329+0000 7fcb5ef6c700 1 -- 172.21.15.43:0/984289636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb5819d4a0 con 0x7fcb58107300 2024-04-21T08:29:15.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.330+0000 7fcb55ffb700 1 -- 172.21.15.43:0/984289636 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fcb4c02a070 con 0x7fcb58107300 2024-04-21T08:29:15.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.330+0000 7fcb55ffb700 1 --2- 172.21.15.43:0/984289636 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fcb480396d0 0x7fcb4803bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:15.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.330+0000 7fcb55ffb700 1 -- 172.21.15.43:0/984289636 --> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcb4803c2a0 con 0x7fcb480396d0 2024-04-21T08:29:15.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.330+0000 7fcb55ffb700 1 -- 172.21.15.43:0/984289636 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcb4c01f070 con 0x7fcb58107300 2024-04-21T08:29:15.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.330+0000 7fcb57fff700 1 -- 172.21.15.43:0/984289636 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fcb480396d0 msgr2=0x7fcb4803bb90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3200866696 2024-04-21T08:29:15.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.330+0000 7fcb57fff700 1 --2- 172.21.15.43:0/984289636 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fcb480396d0 0x7fcb4803bb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:29:15.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.532+0000 7fcb57fff700 1 -- 172.21.15.43:0/984289636 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fcb480396d0 msgr2=0x7fcb4803bb90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3200866696 2024-04-21T08:29:15.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.532+0000 7fcb57fff700 1 --2- 172.21.15.43:0/984289636 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fcb480396d0 0x7fcb4803bb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:29:15.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.933+0000 7fcb57fff700 1 -- 172.21.15.43:0/984289636 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fcb480396d0 msgr2=0x7fcb4803bb90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3200866696 2024-04-21T08:29:15.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.933+0000 7fcb57fff700 1 --2- 172.21.15.43:0/984289636 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fcb480396d0 0x7fcb4803bb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-21T08:29:15.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:10.734+0000 7fcb57fff700 1 -- 172.21.15.43:0/984289636 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fcb480396d0 msgr2=0x7fcb4803bb90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3200866696 2024-04-21T08:29:15.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:10.734+0000 7fcb57fff700 1 --2- 172.21.15.43:0/984289636 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fcb480396d0 0x7fcb4803bb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-21T08:29:15.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.335+0000 7fcb57fff700 1 -- 172.21.15.43:0/984289636 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fcb480396d0 msgr2=0x7fcb4803bb90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3200866696 2024-04-21T08:29:15.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.335+0000 7fcb57fff700 1 --2- 172.21.15.43:0/984289636 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fcb480396d0 0x7fcb4803bb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-21T08:29:15.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:13.890+0000 7fcb55ffb700 1 -- 172.21.15.43:0/984289636 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7fcb4c022410 con 0x7fcb58107300 2024-04-21T08:29:15.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:13.891+0000 7fcb55ffb700 1 -- 172.21.15.43:0/984289636 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fcb480396d0 msgr2=0x7fcb4803bb90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:29:15.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:13.891+0000 7fcb55ffb700 1 --2- 172.21.15.43:0/984289636 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fcb480396d0 0x7fcb4803bb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:15.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.893+0000 7fcb55ffb700 1 -- 172.21.15.43:0/984289636 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7fcb4c0158d0 con 0x7fcb58107300 2024-04-21T08:29:15.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.893+0000 7fcb55ffb700 1 --2- 172.21.15.43:0/984289636 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fcb4803cfc0 0x7fcb4803f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:15.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.894+0000 7fcb55ffb700 1 -- 172.21.15.43:0/984289636 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcb4c049210 con 0x7fcb4803cfc0 2024-04-21T08:29:15.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.894+0000 7fcb57fff700 1 --2- 172.21.15.43:0/984289636 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fcb4803cfc0 0x7fcb4803f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:15.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.895+0000 7fcb57fff700 1 --2- 172.21.15.43:0/984289636 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fcb4803cfc0 0x7fcb4803f3b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fcb44003a10 tx=0x7fcb44009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:15.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.895+0000 7fcb55ffb700 1 -- 172.21.15.43:0/984289636 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fcb4c049210 con 0x7fcb4803cfc0 2024-04-21T08:29:15.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.902+0000 7fcb5ef6c700 1 -- 172.21.15.43:0/984289636 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fcb58069b50 con 0x7fcb4803cfc0 2024-04-21T08:29:15.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.902+0000 7fcb55ffb700 1 -- 172.21.15.43:0/984289636 <== mgr.14150 v2:172.21.15.43:6800/1577030772 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fcb58069b50 con 0x7fcb4803cfc0 2024-04-21T08:29:15.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.902+0000 7fcb5ef6c700 1 -- 172.21.15.43:0/984289636 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fcb4803cfc0 msgr2=0x7fcb4803f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:15.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.902+0000 7fcb5ef6c700 1 --2- 172.21.15.43:0/984289636 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fcb4803cfc0 0x7fcb4803f3b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fcb44003a10 tx=0x7fcb44009270 comp rx=0 tx=0).stop 2024-04-21T08:29:15.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.902+0000 7fcb5ef6c700 1 -- 172.21.15.43:0/984289636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb58107300 msgr2=0x7fcb58199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:15.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.902+0000 7fcb5ef6c700 1 --2- 172.21.15.43:0/984289636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb58107300 0x7fcb58199fe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00d650 tx=0x7fcb4c003ff0 comp rx=0 tx=0).stop 2024-04-21T08:29:15.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.902+0000 7fcb5ef6c700 1 -- 172.21.15.43:0/984289636 shutdown_connections 2024-04-21T08:29:15.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.902+0000 7fcb5ef6c700 1 --2- 172.21.15.43:0/984289636 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fcb4803cfc0 0x7fcb4803f3b0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:15.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.902+0000 7fcb5ef6c700 1 --2- 172.21.15.43:0/984289636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb58107300 0x7fcb58199fe0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:15.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.902+0000 7fcb5ef6c700 1 --2- 172.21.15.43:0/984289636 >> [v2:172.21.15.43:6800/3200866696,v1:172.21.15.43:6801/3200866696] conn(0x7fcb480396d0 0x7fcb4803bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:15.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.902+0000 7fcb5ef6c700 1 -- 172.21.15.43:0/984289636 >> 172.21.15.43:0/984289636 conn(0x7fcb581028e0 msgr2=0x7fcb58190620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:15.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.903+0000 7fcb5ef6c700 1 -- 172.21.15.43:0/984289636 shutdown_connections 2024-04-21T08:29:15.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.903+0000 7fcb5ef6c700 1 -- 172.21.15.43:0/984289636 wait complete. 2024-04-21T08:29:15.516 INFO:teuthology.orchestra.run.smithi043.stdout:mgr epoch 9 is available 2024-04-21T08:29:15.516 INFO:teuthology.orchestra.run.smithi043.stdout:Generating a dashboard self-signed certificate... 2024-04-21T08:29:16.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:15 smithi043 ceph-mon[27978]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-21T08:29:16.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:15 smithi043 ceph-mon[27978]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-21T08:29:17.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:16 smithi043 ceph-mon[27978]: [21/Apr/2024:08:29:15] ENGINE Bus STARTING 2024-04-21T08:29:17.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:16 smithi043 ceph-mon[27978]: mgrmap e12: x(active, since 2s) 2024-04-21T08:29:17.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:16 smithi043 ceph-mon[27978]: [21/Apr/2024:08:29:15] ENGINE Serving on http://172.21.15.43:8765 2024-04-21T08:29:17.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:16 smithi043 ceph-mon[27978]: [21/Apr/2024:08:29:16] ENGINE Serving on https://172.21.15.43:7150 2024-04-21T08:29:17.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:16 smithi043 ceph-mon[27978]: [21/Apr/2024:08:29:16] ENGINE Bus STARTED 2024-04-21T08:29:17.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:16 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:17.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:16 smithi043 ceph-mon[27978]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:29:17.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:16 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:17.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:16 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:17.252 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-21T08:29:17.252 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.293+0000 7f9631f44700 1 Processor -- start 2024-04-21T08:29:17.252 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.293+0000 7f9631f44700 1 -- start start 2024-04-21T08:29:17.252 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.293+0000 7f9631f44700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c1084b0 0x7f962c1088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:17.252 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.293+0000 7f9631f44700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f962c108ea0 con 0x7f962c1084b0 2024-04-21T08:29:17.252 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.293+0000 7f962b7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c1084b0 0x7f962c1088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:17.253 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.293+0000 7f962b7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c1084b0 0x7f962c1088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55774/0 (socket says 172.21.15.43:55774) 2024-04-21T08:29:17.253 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.293+0000 7f962b7fe700 1 -- 172.21.15.43:0/3030805349 learned_addr learned my addr 172.21.15.43:0/3030805349 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:17.253 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.293+0000 7f962b7fe700 1 -- 172.21.15.43:0/3030805349 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962c109660 con 0x7f962c1084b0 2024-04-21T08:29:17.253 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.294+0000 7f962b7fe700 1 --2- 172.21.15.43:0/3030805349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c1084b0 0x7f962c1088d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f961c0098a0 tx=0x7f961c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77084fc3a93a1d00 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:17.253 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.294+0000 7f962a7fc700 1 -- 172.21.15.43:0/3030805349 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f961c004160 con 0x7f962c1084b0 2024-04-21T08:29:17.253 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.294+0000 7f962a7fc700 1 -- 172.21.15.43:0/3030805349 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f961c004300 con 0x7f962c1084b0 2024-04-21T08:29:17.253 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.294+0000 7f962a7fc700 1 -- 172.21.15.43:0/3030805349 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f961c011a20 con 0x7f962c1084b0 2024-04-21T08:29:17.253 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.294+0000 7f9631f44700 1 -- 172.21.15.43:0/3030805349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c1084b0 msgr2=0x7f962c1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:17.253 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.294+0000 7f9631f44700 1 --2- 172.21.15.43:0/3030805349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c1084b0 0x7f962c1088d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f961c0098a0 tx=0x7f961c009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:17.253 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.294+0000 7f9631f44700 1 -- 172.21.15.43:0/3030805349 shutdown_connections 2024-04-21T08:29:17.253 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.294+0000 7f9631f44700 1 --2- 172.21.15.43:0/3030805349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c1084b0 0x7f962c1088d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:17.254 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.294+0000 7f9631f44700 1 -- 172.21.15.43:0/3030805349 >> 172.21.15.43:0/3030805349 conn(0x7f962c103af0 msgr2=0x7f962c105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:17.254 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.295+0000 7f9631f44700 1 -- 172.21.15.43:0/3030805349 shutdown_connections 2024-04-21T08:29:17.254 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.295+0000 7f9631f44700 1 -- 172.21.15.43:0/3030805349 wait complete. 2024-04-21T08:29:17.254 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.295+0000 7f9631f44700 1 Processor -- start 2024-04-21T08:29:17.254 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.295+0000 7f9631f44700 1 -- start start 2024-04-21T08:29:17.254 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.296+0000 7f9631f44700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c1084b0 0x7f962c19c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:17.254 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.296+0000 7f9631f44700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f962c10ca10 con 0x7f962c1084b0 2024-04-21T08:29:17.254 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.296+0000 7f962b7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c1084b0 0x7f962c19c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:17.254 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.296+0000 7f962b7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c1084b0 0x7f962c19c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55782/0 (socket says 172.21.15.43:55782) 2024-04-21T08:29:17.254 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.296+0000 7f962b7fe700 1 -- 172.21.15.43:0/1948831304 learned_addr learned my addr 172.21.15.43:0/1948831304 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:17.254 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.296+0000 7f962b7fe700 1 -- 172.21.15.43:0/1948831304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962c19c690 con 0x7f962c1084b0 2024-04-21T08:29:17.254 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.296+0000 7f962b7fe700 1 --2- 172.21.15.43:0/1948831304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c1084b0 0x7f962c19c150 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f961c009520 tx=0x7f961c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:17.254 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.296+0000 7f9628ff9700 1 -- 172.21.15.43:0/1948831304 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f961c0039e0 con 0x7f962c1084b0 2024-04-21T08:29:17.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.296+0000 7f9631f44700 1 -- 172.21.15.43:0/1948831304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f962c19c920 con 0x7f962c1084b0 2024-04-21T08:29:17.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.296+0000 7f9628ff9700 1 -- 172.21.15.43:0/1948831304 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f961c01b8f0 con 0x7f962c1084b0 2024-04-21T08:29:17.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.296+0000 7f9628ff9700 1 -- 172.21.15.43:0/1948831304 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f961c019770 con 0x7f962c1084b0 2024-04-21T08:29:17.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.296+0000 7f9631f44700 1 -- 172.21.15.43:0/1948831304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f962c19f610 con 0x7f962c1084b0 2024-04-21T08:29:17.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.297+0000 7f9628ff9700 1 -- 172.21.15.43:0/1948831304 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f961c019a70 con 0x7f962c1084b0 2024-04-21T08:29:17.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.297+0000 7f9631f44700 1 -- 172.21.15.43:0/1948831304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f962c0640a0 con 0x7f962c1084b0 2024-04-21T08:29:17.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.298+0000 7f9628ff9700 1 --2- 172.21.15.43:0/1948831304 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f96140396c0 0x7f961403bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:17.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.298+0000 7f9628ff9700 1 -- 172.21.15.43:0/1948831304 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f961c04e180 con 0x7f962c1084b0 2024-04-21T08:29:17.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.298+0000 7f962affd700 1 --2- 172.21.15.43:0/1948831304 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f96140396c0 0x7f961403bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:17.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.298+0000 7f962affd700 1 --2- 172.21.15.43:0/1948831304 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f96140396c0 0x7f961403bb80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9620006fd0 tx=0x7f9620006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:17.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.303+0000 7f9628ff9700 1 -- 172.21.15.43:0/1948831304 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f961c01f030 con 0x7f962c1084b0 2024-04-21T08:29:17.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.516+0000 7f9631f44700 1 -- 172.21.15.43:0/1948831304 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f962c104bb0 con 0x7f96140396c0 2024-04-21T08:29:17.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.592+0000 7f9628ff9700 1 -- 172.21.15.43:0/1948831304 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f962c104bb0 con 0x7f96140396c0 2024-04-21T08:29:17.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.594+0000 7f9631f44700 1 -- 172.21.15.43:0/1948831304 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f96140396c0 msgr2=0x7f961403bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:17.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.594+0000 7f9631f44700 1 --2- 172.21.15.43:0/1948831304 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f96140396c0 0x7f961403bb80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9620006fd0 tx=0x7f9620006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:17.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.594+0000 7f9631f44700 1 -- 172.21.15.43:0/1948831304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c1084b0 msgr2=0x7f962c19c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:17.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.594+0000 7f9631f44700 1 --2- 172.21.15.43:0/1948831304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c1084b0 0x7f962c19c150 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f961c009520 tx=0x7f961c011aa0 comp rx=0 tx=0).stop 2024-04-21T08:29:17.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.594+0000 7f9631f44700 1 -- 172.21.15.43:0/1948831304 shutdown_connections 2024-04-21T08:29:17.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.594+0000 7f9631f44700 1 --2- 172.21.15.43:0/1948831304 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f96140396c0 0x7f961403bb80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:17.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.594+0000 7f9631f44700 1 --2- 172.21.15.43:0/1948831304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c1084b0 0x7f962c19c150 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:17.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.594+0000 7f9631f44700 1 -- 172.21.15.43:0/1948831304 >> 172.21.15.43:0/1948831304 conn(0x7f962c103af0 msgr2=0x7f962c1044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:17.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.595+0000 7f9631f44700 1 -- 172.21.15.43:0/1948831304 shutdown_connections 2024-04-21T08:29:17.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.595+0000 7f9631f44700 1 -- 172.21.15.43:0/1948831304 wait complete. 2024-04-21T08:29:17.256 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial admin user... 2024-04-21T08:29:18.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:18 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:18.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:18 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:18.936 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$k5Zn0sWxBexATFZ9jfHGhe/c6v.QCSjS9T.q.EixfsOhjmMDfBYDi", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713688158, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-21T08:29:18.936 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.707+0000 7fd42587d700 1 Processor -- start 2024-04-21T08:29:18.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.707+0000 7fd42587d700 1 -- start start 2024-04-21T08:29:18.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.707+0000 7fd42587d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4201072a0 0x7fd4201076c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:18.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.707+0000 7fd42587d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd420107c90 con 0x7fd4201072a0 2024-04-21T08:29:18.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.708+0000 7fd41effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4201072a0 0x7fd4201076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:18.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.708+0000 7fd41effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4201072a0 0x7fd4201076c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49844/0 (socket says 172.21.15.43:49844) 2024-04-21T08:29:18.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.708+0000 7fd41effd700 1 -- 172.21.15.43:0/3684084083 learned_addr learned my addr 172.21.15.43:0/3684084083 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:18.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.708+0000 7fd41effd700 1 -- 172.21.15.43:0/3684084083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4201084a0 con 0x7fd4201072a0 2024-04-21T08:29:18.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.708+0000 7fd41effd700 1 --2- 172.21.15.43:0/3684084083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4201072a0 0x7fd4201076c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd4080098a0 tx=0x7fd408009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cae297b8992e154 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:18.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.708+0000 7fd41dffb700 1 -- 172.21.15.43:0/3684084083 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd408004160 con 0x7fd4201072a0 2024-04-21T08:29:18.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.708+0000 7fd41dffb700 1 -- 172.21.15.43:0/3684084083 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd408004300 con 0x7fd4201072a0 2024-04-21T08:29:18.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.708+0000 7fd41dffb700 1 -- 172.21.15.43:0/3684084083 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd408011a20 con 0x7fd4201072a0 2024-04-21T08:29:18.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.709+0000 7fd42587d700 1 -- 172.21.15.43:0/3684084083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4201072a0 msgr2=0x7fd4201076c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:18.937 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.709+0000 7fd42587d700 1 --2- 172.21.15.43:0/3684084083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4201072a0 0x7fd4201076c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd4080098a0 tx=0x7fd408009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:18.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.709+0000 7fd42587d700 1 -- 172.21.15.43:0/3684084083 shutdown_connections 2024-04-21T08:29:18.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.709+0000 7fd42587d700 1 --2- 172.21.15.43:0/3684084083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4201072a0 0x7fd4201076c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:18.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.709+0000 7fd42587d700 1 -- 172.21.15.43:0/3684084083 >> 172.21.15.43:0/3684084083 conn(0x7fd4201028e0 msgr2=0x7fd420104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:18.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.709+0000 7fd42587d700 1 -- 172.21.15.43:0/3684084083 shutdown_connections 2024-04-21T08:29:18.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.709+0000 7fd42587d700 1 -- 172.21.15.43:0/3684084083 wait complete. 2024-04-21T08:29:18.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.710+0000 7fd42587d700 1 Processor -- start 2024-04-21T08:29:18.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.710+0000 7fd42587d700 1 -- start start 2024-04-21T08:29:18.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.710+0000 7fd42587d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4201072a0 0x7fd42019c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:18.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.710+0000 7fd42587d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd42010b850 con 0x7fd4201072a0 2024-04-21T08:29:18.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.710+0000 7fd41effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4201072a0 0x7fd42019c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:18.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.710+0000 7fd41effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4201072a0 0x7fd42019c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49854/0 (socket says 172.21.15.43:49854) 2024-04-21T08:29:18.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.710+0000 7fd41effd700 1 -- 172.21.15.43:0/514475408 learned_addr learned my addr 172.21.15.43:0/514475408 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:18.938 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.711+0000 7fd41effd700 1 -- 172.21.15.43:0/514475408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd42019c700 con 0x7fd4201072a0 2024-04-21T08:29:18.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.711+0000 7fd41effd700 1 --2- 172.21.15.43:0/514475408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4201072a0 0x7fd42019c1c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd4080116e0 tx=0x7fd408011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:18.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.711+0000 7fd42487b700 1 -- 172.21.15.43:0/514475408 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4080039e0 con 0x7fd4201072a0 2024-04-21T08:29:18.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.711+0000 7fd42487b700 1 -- 172.21.15.43:0/514475408 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd40801b8f0 con 0x7fd4201072a0 2024-04-21T08:29:18.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.711+0000 7fd42587d700 1 -- 172.21.15.43:0/514475408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd42019c990 con 0x7fd4201072a0 2024-04-21T08:29:18.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.711+0000 7fd42487b700 1 -- 172.21.15.43:0/514475408 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd408023760 con 0x7fd4201072a0 2024-04-21T08:29:18.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.711+0000 7fd42587d700 1 -- 172.21.15.43:0/514475408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd42019f680 con 0x7fd4201072a0 2024-04-21T08:29:18.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.712+0000 7fd42487b700 1 -- 172.21.15.43:0/514475408 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fd4080239f0 con 0x7fd4201072a0 2024-04-21T08:29:18.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.712+0000 7fd42587d700 1 -- 172.21.15.43:0/514475408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd42010b9d0 con 0x7fd4201072a0 2024-04-21T08:29:18.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.712+0000 7fd42487b700 1 --2- 172.21.15.43:0/514475408 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd40c039670 0x7fd40c03bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:18.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.712+0000 7fd42487b700 1 -- 172.21.15.43:0/514475408 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd40804e1e0 con 0x7fd4201072a0 2024-04-21T08:29:18.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.713+0000 7fd41e7fc700 1 --2- 172.21.15.43:0/514475408 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd40c039670 0x7fd40c03bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:18.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.713+0000 7fd41e7fc700 1 --2- 172.21.15.43:0/514475408 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd40c039670 0x7fd40c03bb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd410006fd0 tx=0x7fd410006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:18.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.722+0000 7fd42487b700 1 -- 172.21.15.43:0/514475408 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd40801f030 con 0x7fd4201072a0 2024-04-21T08:29:18.939 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.951+0000 7fd42587d700 1 -- 172.21.15.43:0/514475408 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fd42002e1d0 con 0x7fd40c039670 2024-04-21T08:29:18.940 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.198+0000 7fd42487b700 1 -- 172.21.15.43:0/514475408 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fd42002e1d0 con 0x7fd40c039670 2024-04-21T08:29:18.940 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.200+0000 7fd42587d700 1 -- 172.21.15.43:0/514475408 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd40c039670 msgr2=0x7fd40c03bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:18.940 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.200+0000 7fd42587d700 1 --2- 172.21.15.43:0/514475408 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd40c039670 0x7fd40c03bb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd410006fd0 tx=0x7fd410006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:18.940 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.200+0000 7fd42587d700 1 -- 172.21.15.43:0/514475408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4201072a0 msgr2=0x7fd42019c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:18.940 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.200+0000 7fd42587d700 1 --2- 172.21.15.43:0/514475408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4201072a0 0x7fd42019c1c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd4080116e0 tx=0x7fd408011aa0 comp rx=0 tx=0).stop 2024-04-21T08:29:18.940 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.200+0000 7fd42587d700 1 -- 172.21.15.43:0/514475408 shutdown_connections 2024-04-21T08:29:18.940 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.200+0000 7fd42587d700 1 --2- 172.21.15.43:0/514475408 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd40c039670 0x7fd40c03bb30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:18.940 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.200+0000 7fd42587d700 1 --2- 172.21.15.43:0/514475408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4201072a0 0x7fd42019c1c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:18.940 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.200+0000 7fd42587d700 1 -- 172.21.15.43:0/514475408 >> 172.21.15.43:0/514475408 conn(0x7fd4201028e0 msgr2=0x7fd420192d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:18.940 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.201+0000 7fd42587d700 1 -- 172.21.15.43:0/514475408 shutdown_connections 2024-04-21T08:29:18.940 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.201+0000 7fd42587d700 1 -- 172.21.15.43:0/514475408 wait complete. 2024-04-21T08:29:18.941 INFO:teuthology.orchestra.run.smithi043.stdout:Fetching dashboard port number... 2024-04-21T08:29:19.665 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:19 smithi043 ceph-mon[27978]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:29:20.623 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:20 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/980739883' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-21T08:29:20.628 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 8443 2024-04-21T08:29:20.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.669+0000 7ff6dafd7700 1 Processor -- start 2024-04-21T08:29:20.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.669+0000 7ff6dafd7700 1 -- start start 2024-04-21T08:29:20.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.669+0000 7ff6dafd7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6d41084b0 0x7ff6d41088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:20.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.669+0000 7ff6dafd7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d4108ea0 con 0x7ff6d41084b0 2024-04-21T08:29:20.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.670+0000 7ff6d8d73700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6d41084b0 0x7ff6d41088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:20.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.670+0000 7ff6d8d73700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6d41084b0 0x7ff6d41088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49868/0 (socket says 172.21.15.43:49868) 2024-04-21T08:29:20.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.670+0000 7ff6d8d73700 1 -- 172.21.15.43:0/245819997 learned_addr learned my addr 172.21.15.43:0/245819997 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:20.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.670+0000 7ff6d8d73700 1 -- 172.21.15.43:0/245819997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d4109660 con 0x7ff6d41084b0 2024-04-21T08:29:20.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.670+0000 7ff6d8d73700 1 --2- 172.21.15.43:0/245819997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6d41084b0 0x7ff6d41088d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40098a0 tx=0x7ff6c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82451abb04a9bc39 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:20.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.670+0000 7ff6d37fe700 1 -- 172.21.15.43:0/245819997 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6c4004160 con 0x7ff6d41084b0 2024-04-21T08:29:20.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.670+0000 7ff6d37fe700 1 -- 172.21.15.43:0/245819997 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6c4004300 con 0x7ff6d41084b0 2024-04-21T08:29:20.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.670+0000 7ff6d37fe700 1 -- 172.21.15.43:0/245819997 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6c4011a20 con 0x7ff6d41084b0 2024-04-21T08:29:20.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.671+0000 7ff6dafd7700 1 -- 172.21.15.43:0/245819997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6d41084b0 msgr2=0x7ff6d41088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:20.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.671+0000 7ff6dafd7700 1 --2- 172.21.15.43:0/245819997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6d41084b0 0x7ff6d41088d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40098a0 tx=0x7ff6c4009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:20.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.671+0000 7ff6dafd7700 1 -- 172.21.15.43:0/245819997 shutdown_connections 2024-04-21T08:29:20.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.671+0000 7ff6dafd7700 1 --2- 172.21.15.43:0/245819997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6d41084b0 0x7ff6d41088d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:20.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.671+0000 7ff6dafd7700 1 -- 172.21.15.43:0/245819997 >> 172.21.15.43:0/245819997 conn(0x7ff6d4103af0 msgr2=0x7ff6d4105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:20.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.671+0000 7ff6dafd7700 1 -- 172.21.15.43:0/245819997 shutdown_connections 2024-04-21T08:29:20.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.671+0000 7ff6dafd7700 1 -- 172.21.15.43:0/245819997 wait complete. 2024-04-21T08:29:20.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.671+0000 7ff6dafd7700 1 Processor -- start 2024-04-21T08:29:20.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.671+0000 7ff6dafd7700 1 -- start start 2024-04-21T08:29:20.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.672+0000 7ff6dafd7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6d41084b0 0x7ff6d419e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:20.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.672+0000 7ff6dafd7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d410ca10 con 0x7ff6d41084b0 2024-04-21T08:29:20.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.672+0000 7ff6d8d73700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6d41084b0 0x7ff6d419e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:20.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.672+0000 7ff6d8d73700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6d41084b0 0x7ff6d419e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49872/0 (socket says 172.21.15.43:49872) 2024-04-21T08:29:20.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.672+0000 7ff6d8d73700 1 -- 172.21.15.43:0/980739883 learned_addr learned my addr 172.21.15.43:0/980739883 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:20.631 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.672+0000 7ff6d8d73700 1 -- 172.21.15.43:0/980739883 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d419e8a0 con 0x7ff6d41084b0 2024-04-21T08:29:20.631 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.672+0000 7ff6d8d73700 1 --2- 172.21.15.43:0/980739883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6d41084b0 0x7ff6d419e360 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40116e0 tx=0x7ff6c4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:20.631 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.672+0000 7ff6d1ffb700 1 -- 172.21.15.43:0/980739883 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6c40039e0 con 0x7ff6d41084b0 2024-04-21T08:29:20.631 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.672+0000 7ff6d1ffb700 1 -- 172.21.15.43:0/980739883 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6c401b8f0 con 0x7ff6d41084b0 2024-04-21T08:29:20.631 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.672+0000 7ff6dafd7700 1 -- 172.21.15.43:0/980739883 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d419eb30 con 0x7ff6d41084b0 2024-04-21T08:29:20.631 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.672+0000 7ff6d1ffb700 1 -- 172.21.15.43:0/980739883 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6c4023760 con 0x7ff6d41084b0 2024-04-21T08:29:20.631 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.673+0000 7ff6dafd7700 1 -- 172.21.15.43:0/980739883 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d41a1820 con 0x7ff6d41084b0 2024-04-21T08:29:20.631 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.673+0000 7ff6d1ffb700 1 -- 172.21.15.43:0/980739883 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7ff6c40239d0 con 0x7ff6d41084b0 2024-04-21T08:29:20.631 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.673+0000 7ff6dafd7700 1 -- 172.21.15.43:0/980739883 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6d40640a0 con 0x7ff6d41084b0 2024-04-21T08:29:20.631 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.674+0000 7ff6d1ffb700 1 --2- 172.21.15.43:0/980739883 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ff6bc039670 0x7ff6bc03bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:20.631 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.674+0000 7ff6d1ffb700 1 -- 172.21.15.43:0/980739883 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff6c404e1a0 con 0x7ff6d41084b0 2024-04-21T08:29:20.631 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.674+0000 7ff6d3fff700 1 --2- 172.21.15.43:0/980739883 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ff6bc039670 0x7ff6bc03bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:20.631 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.674+0000 7ff6d3fff700 1 --2- 172.21.15.43:0/980739883 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ff6bc039670 0x7ff6bc03bb30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8006fd0 tx=0x7ff6c8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:20.631 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.679+0000 7ff6d1ffb700 1 -- 172.21.15.43:0/980739883 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff6c401f030 con 0x7ff6d41084b0 2024-04-21T08:29:20.632 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.884+0000 7ff6dafd7700 1 -- 172.21.15.43:0/980739883 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7ff6d41a1dd0 con 0x7ff6d41084b0 2024-04-21T08:29:20.632 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.885+0000 7ff6d1ffb700 1 -- 172.21.15.43:0/980739883 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7ff6c4023dc0 con 0x7ff6d41084b0 2024-04-21T08:29:20.632 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.887+0000 7ff6dafd7700 1 -- 172.21.15.43:0/980739883 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ff6bc039670 msgr2=0x7ff6bc03bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:20.632 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.887+0000 7ff6dafd7700 1 --2- 172.21.15.43:0/980739883 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ff6bc039670 0x7ff6bc03bb30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8006fd0 tx=0x7ff6c8006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:20.632 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.887+0000 7ff6dafd7700 1 -- 172.21.15.43:0/980739883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6d41084b0 msgr2=0x7ff6d419e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:20.632 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.887+0000 7ff6dafd7700 1 --2- 172.21.15.43:0/980739883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6d41084b0 0x7ff6d419e360 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40116e0 tx=0x7ff6c4011aa0 comp rx=0 tx=0).stop 2024-04-21T08:29:20.632 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.887+0000 7ff6dafd7700 1 -- 172.21.15.43:0/980739883 shutdown_connections 2024-04-21T08:29:20.632 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.887+0000 7ff6dafd7700 1 --2- 172.21.15.43:0/980739883 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ff6bc039670 0x7ff6bc03bb30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:20.632 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.887+0000 7ff6dafd7700 1 --2- 172.21.15.43:0/980739883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6d41084b0 0x7ff6d419e360 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:20.632 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.887+0000 7ff6dafd7700 1 -- 172.21.15.43:0/980739883 >> 172.21.15.43:0/980739883 conn(0x7ff6d4103af0 msgr2=0x7ff6d407c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:20.632 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.887+0000 7ff6dafd7700 1 -- 172.21.15.43:0/980739883 shutdown_connections 2024-04-21T08:29:20.632 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.887+0000 7ff6dafd7700 1 -- 172.21.15.43:0/980739883 wait complete. 2024-04-21T08:29:20.637 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-21T08:29:20.637 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-04-21T08:29:20.644 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-21T08:29:20.644 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-04-21T08:29:20.644 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-04-21T08:29:20.644 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-21T08:29:20.646 INFO:teuthology.orchestra.run.smithi043.stdout:Ceph Dashboard is now available at: 2024-04-21T08:29:20.646 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:29:20.646 INFO:teuthology.orchestra.run.smithi043.stdout: URL: https://smithi043:8443/ 2024-04-21T08:29:20.646 INFO:teuthology.orchestra.run.smithi043.stdout: User: admin 2024-04-21T08:29:20.646 INFO:teuthology.orchestra.run.smithi043.stdout: Password: yg4yh0ydv0 2024-04-21T08:29:20.646 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:29:20.647 INFO:teuthology.orchestra.run.smithi043.stdout:Saving cluster configuration to /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config directory 2024-04-21T08:29:20.647 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling autotune for osd_memory_target 2024-04-21T08:29:21.802 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:21 smithi043 ceph-mon[27978]: mgrmap e13: x(active, since 6s) 2024-04-21T08:29:21.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.012+0000 7f846fd2c700 1 Processor -- start 2024-04-21T08:29:21.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.012+0000 7f846fd2c700 1 -- start start 2024-04-21T08:29:21.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.012+0000 7f846fd2c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84681084b0 0x7f84681088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:21.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.012+0000 7f846fd2c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8468108ea0 con 0x7f84681084b0 2024-04-21T08:29:21.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.012+0000 7f846dac8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84681084b0 0x7f84681088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:21.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.012+0000 7f846dac8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84681084b0 0x7f84681088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49874/0 (socket says 172.21.15.43:49874) 2024-04-21T08:29:21.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.012+0000 7f846dac8700 1 -- 172.21.15.43:0/1305802985 learned_addr learned my addr 172.21.15.43:0/1305802985 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:21.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.013+0000 7f846dac8700 1 -- 172.21.15.43:0/1305802985 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8468109660 con 0x7f84681084b0 2024-04-21T08:29:21.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.013+0000 7f846dac8700 1 --2- 172.21.15.43:0/1305802985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84681084b0 0x7f84681088d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f84640098a0 tx=0x7f8464009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47bc73352b6ebc80 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:21.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.013+0000 7f846cac6700 1 -- 172.21.15.43:0/1305802985 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8464004160 con 0x7f84681084b0 2024-04-21T08:29:21.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.013+0000 7f846cac6700 1 -- 172.21.15.43:0/1305802985 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8464004300 con 0x7f84681084b0 2024-04-21T08:29:21.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.013+0000 7f846cac6700 1 -- 172.21.15.43:0/1305802985 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8464011a20 con 0x7f84681084b0 2024-04-21T08:29:21.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.013+0000 7f846fd2c700 1 -- 172.21.15.43:0/1305802985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84681084b0 msgr2=0x7f84681088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:21.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.013+0000 7f846fd2c700 1 --2- 172.21.15.43:0/1305802985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84681084b0 0x7f84681088d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f84640098a0 tx=0x7f8464009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:21.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.014+0000 7f846fd2c700 1 -- 172.21.15.43:0/1305802985 shutdown_connections 2024-04-21T08:29:21.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.014+0000 7f846fd2c700 1 --2- 172.21.15.43:0/1305802985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84681084b0 0x7f84681088d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:21.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.014+0000 7f846fd2c700 1 -- 172.21.15.43:0/1305802985 >> 172.21.15.43:0/1305802985 conn(0x7f8468103af0 msgr2=0x7f8468105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:21.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.014+0000 7f846fd2c700 1 -- 172.21.15.43:0/1305802985 shutdown_connections 2024-04-21T08:29:21.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.014+0000 7f846fd2c700 1 -- 172.21.15.43:0/1305802985 wait complete. 2024-04-21T08:29:21.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.014+0000 7f846fd2c700 1 Processor -- start 2024-04-21T08:29:21.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.014+0000 7f846fd2c700 1 -- start start 2024-04-21T08:29:21.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.014+0000 7f846fd2c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84681084b0 0x7f846807db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:21.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.014+0000 7f846fd2c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f846810ca10 con 0x7f84681084b0 2024-04-21T08:29:21.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.015+0000 7f846dac8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84681084b0 0x7f846807db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:21.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.015+0000 7f846dac8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84681084b0 0x7f846807db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49886/0 (socket says 172.21.15.43:49886) 2024-04-21T08:29:21.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.015+0000 7f846dac8700 1 -- 172.21.15.43:0/1916634744 learned_addr learned my addr 172.21.15.43:0/1916634744 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:21.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.015+0000 7f846dac8700 1 -- 172.21.15.43:0/1916634744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846807e0a0 con 0x7f84681084b0 2024-04-21T08:29:21.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.015+0000 7f846dac8700 1 --2- 172.21.15.43:0/1916634744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84681084b0 0x7f846807db60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f84640116e0 tx=0x7f8464011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:21.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.015+0000 7f845effd700 1 -- 172.21.15.43:0/1916634744 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84640039e0 con 0x7f84681084b0 2024-04-21T08:29:21.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.015+0000 7f845effd700 1 -- 172.21.15.43:0/1916634744 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f846401b8f0 con 0x7f84681084b0 2024-04-21T08:29:21.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.015+0000 7f846fd2c700 1 -- 172.21.15.43:0/1916634744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846807c290 con 0x7f84681084b0 2024-04-21T08:29:21.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.015+0000 7f845effd700 1 -- 172.21.15.43:0/1916634744 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8464023760 con 0x7f84681084b0 2024-04-21T08:29:21.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.015+0000 7f846fd2c700 1 -- 172.21.15.43:0/1916634744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f846807c7d0 con 0x7f84681084b0 2024-04-21T08:29:21.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.016+0000 7f845effd700 1 -- 172.21.15.43:0/1916634744 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8464023900 con 0x7f84681084b0 2024-04-21T08:29:21.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.016+0000 7f845effd700 1 --2- 172.21.15.43:0/1916634744 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8454039770 0x7f845403bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:21.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.016+0000 7f845effd700 1 -- 172.21.15.43:0/1916634744 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f846404e3f0 con 0x7f84681084b0 2024-04-21T08:29:21.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.016+0000 7f846fd2c700 1 -- 172.21.15.43:0/1916634744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84680640a0 con 0x7f84681084b0 2024-04-21T08:29:21.812 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.016+0000 7f846d2c7700 1 --2- 172.21.15.43:0/1916634744 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8454039770 0x7f845403bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:21.812 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.017+0000 7f846d2c7700 1 --2- 172.21.15.43:0/1916634744 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8454039770 0x7f845403bc30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8458006fd0 tx=0x7f8458006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:21.812 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.022+0000 7f845effd700 1 -- 172.21.15.43:0/1916634744 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f846401f030 con 0x7f84681084b0 2024-04-21T08:29:21.812 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.228+0000 7f846fd2c700 1 -- 172.21.15.43:0/1916634744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f846807d100 con 0x7f84681084b0 2024-04-21T08:29:21.812 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.228+0000 7f845effd700 1 -- 172.21.15.43:0/1916634744 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f8464018c10 con 0x7f84681084b0 2024-04-21T08:29:21.812 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.231+0000 7f846fd2c700 1 -- 172.21.15.43:0/1916634744 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8454039770 msgr2=0x7f845403bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:21.812 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.231+0000 7f846fd2c700 1 --2- 172.21.15.43:0/1916634744 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8454039770 0x7f845403bc30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8458006fd0 tx=0x7f8458006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:21.812 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.231+0000 7f846fd2c700 1 -- 172.21.15.43:0/1916634744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84681084b0 msgr2=0x7f846807db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:21.812 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.231+0000 7f846fd2c700 1 --2- 172.21.15.43:0/1916634744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84681084b0 0x7f846807db60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f84640116e0 tx=0x7f8464011aa0 comp rx=0 tx=0).stop 2024-04-21T08:29:21.812 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.231+0000 7f846fd2c700 1 -- 172.21.15.43:0/1916634744 shutdown_connections 2024-04-21T08:29:21.812 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.231+0000 7f846fd2c700 1 --2- 172.21.15.43:0/1916634744 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8454039770 0x7f845403bc30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:21.812 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.231+0000 7f846fd2c700 1 --2- 172.21.15.43:0/1916634744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84681084b0 0x7f846807db60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:21.812 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.231+0000 7f846fd2c700 1 -- 172.21.15.43:0/1916634744 >> 172.21.15.43:0/1916634744 conn(0x7f8468103af0 msgr2=0x7f84681044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:21.812 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.231+0000 7f846fd2c700 1 -- 172.21.15.43:0/1916634744 shutdown_connections 2024-04-21T08:29:21.813 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.231+0000 7f846fd2c700 1 -- 172.21.15.43:0/1916634744 wait complete. 2024-04-21T08:29:23.116 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.244+0000 7f8fb8a70700 1 Processor -- start 2024-04-21T08:29:23.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.244+0000 7f8fb8a70700 1 -- start start 2024-04-21T08:29:23.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.245+0000 7f8fb8a70700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fb407de60 0x7f8fb407c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:23.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.245+0000 7f8fb8a70700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb407c8e0 con 0x7f8fb407de60 2024-04-21T08:29:23.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.245+0000 7f8fb259c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fb407de60 0x7f8fb407c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:23.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.245+0000 7f8fb259c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fb407de60 0x7f8fb407c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49894/0 (socket says 172.21.15.43:49894) 2024-04-21T08:29:23.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.245+0000 7f8fb259c700 1 -- 172.21.15.43:0/2258155607 learned_addr learned my addr 172.21.15.43:0/2258155607 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:23.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.245+0000 7f8fb259c700 1 -- 172.21.15.43:0/2258155607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb407ca60 con 0x7f8fb407de60 2024-04-21T08:29:23.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.245+0000 7f8fb259c700 1 --2- 172.21.15.43:0/2258155607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fb407de60 0x7f8fb407c310 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8fa40098a0 tx=0x7f8fa4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c244b221514112 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:23.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.246+0000 7f8fb159a700 1 -- 172.21.15.43:0/2258155607 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fa4004160 con 0x7f8fb407de60 2024-04-21T08:29:23.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.246+0000 7f8fb159a700 1 -- 172.21.15.43:0/2258155607 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fa4004300 con 0x7f8fb407de60 2024-04-21T08:29:23.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.246+0000 7f8fb159a700 1 -- 172.21.15.43:0/2258155607 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fa4011a20 con 0x7f8fb407de60 2024-04-21T08:29:23.117 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.246+0000 7f8fb8a70700 1 -- 172.21.15.43:0/2258155607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fb407de60 msgr2=0x7f8fb407c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:23.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.246+0000 7f8fb8a70700 1 --2- 172.21.15.43:0/2258155607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fb407de60 0x7f8fb407c310 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8fa40098a0 tx=0x7f8fa4009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:23.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.246+0000 7f8fb8a70700 1 -- 172.21.15.43:0/2258155607 shutdown_connections 2024-04-21T08:29:23.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.246+0000 7f8fb8a70700 1 --2- 172.21.15.43:0/2258155607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fb407de60 0x7f8fb407c310 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:23.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.246+0000 7f8fb8a70700 1 -- 172.21.15.43:0/2258155607 >> 172.21.15.43:0/2258155607 conn(0x7f8fb4103af0 msgr2=0x7f8fb4105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:23.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.246+0000 7f8fb8a70700 1 -- 172.21.15.43:0/2258155607 shutdown_connections 2024-04-21T08:29:23.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.246+0000 7f8fb8a70700 1 -- 172.21.15.43:0/2258155607 wait complete. 2024-04-21T08:29:23.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.247+0000 7f8fb8a70700 1 Processor -- start 2024-04-21T08:29:23.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.247+0000 7f8fb8a70700 1 -- start start 2024-04-21T08:29:23.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.247+0000 7f8fb8a70700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fb407de60 0x7f8fb419e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:23.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.247+0000 7f8fb8a70700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb410ea30 con 0x7f8fb407de60 2024-04-21T08:29:23.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.247+0000 7f8fb259c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fb407de60 0x7f8fb419e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:23.118 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.247+0000 7f8fb259c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fb407de60 0x7f8fb419e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49908/0 (socket says 172.21.15.43:49908) 2024-04-21T08:29:23.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.247+0000 7f8fb259c700 1 -- 172.21.15.43:0/4233408697 learned_addr learned my addr 172.21.15.43:0/4233408697 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:23.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.248+0000 7f8fb259c700 1 -- 172.21.15.43:0/4233408697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb419e890 con 0x7f8fb407de60 2024-04-21T08:29:23.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.248+0000 7f8fb259c700 1 --2- 172.21.15.43:0/4233408697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fb407de60 0x7f8fb419e350 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8fa40116e0 tx=0x7f8fa4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:23.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.248+0000 7f8fa37fe700 1 -- 172.21.15.43:0/4233408697 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fa4003a60 con 0x7f8fb407de60 2024-04-21T08:29:23.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.248+0000 7f8fa37fe700 1 -- 172.21.15.43:0/4233408697 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fa401b8f0 con 0x7f8fb407de60 2024-04-21T08:29:23.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.248+0000 7f8fa37fe700 1 -- 172.21.15.43:0/4233408697 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fa4019860 con 0x7f8fb407de60 2024-04-21T08:29:23.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.248+0000 7f8fb8a70700 1 -- 172.21.15.43:0/4233408697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb419eb20 con 0x7f8fb407de60 2024-04-21T08:29:23.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.248+0000 7f8fb8a70700 1 -- 172.21.15.43:0/4233408697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb41a1810 con 0x7f8fb407de60 2024-04-21T08:29:23.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.249+0000 7f8fa37fe700 1 -- 172.21.15.43:0/4233408697 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8fa401b440 con 0x7f8fb407de60 2024-04-21T08:29:23.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.249+0000 7f8fb8a70700 1 -- 172.21.15.43:0/4233408697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb40640a0 con 0x7f8fb407de60 2024-04-21T08:29:23.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.249+0000 7f8fa37fe700 1 --2- 172.21.15.43:0/4233408697 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8f9c039770 0x7f8f9c03bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:23.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.249+0000 7f8fa37fe700 1 -- 172.21.15.43:0/4233408697 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8fa404de90 con 0x7f8fb407de60 2024-04-21T08:29:23.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.249+0000 7f8fb1d9b700 1 --2- 172.21.15.43:0/4233408697 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8f9c039770 0x7f8f9c03bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:23.119 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.250+0000 7f8fb1d9b700 1 --2- 172.21.15.43:0/4233408697 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8f9c039770 0x7f8f9c03bc30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8006fd0 tx=0x7f8fa8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:23.120 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.254+0000 7f8fa37fe700 1 -- 172.21.15.43:0/4233408697 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8fa401f030 con 0x7f8fb407de60 2024-04-21T08:29:23.120 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.577+0000 7f8fb8a70700 1 -- 172.21.15.43:0/4233408697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f8fb41a1dc0 con 0x7f8fb407de60 2024-04-21T08:29:23.120 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.579+0000 7f8fa37fe700 1 -- 172.21.15.43:0/4233408697 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v26)=0 set mgr/dashboard/cluster/status v26) ==== 153+0+0 (secure 0 0 0) 0x7f8fa40223b0 con 0x7f8fb407de60 2024-04-21T08:29:23.120 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.581+0000 7f8fb8a70700 1 -- 172.21.15.43:0/4233408697 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8f9c039770 msgr2=0x7f8f9c03bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:23.120 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.581+0000 7f8fb8a70700 1 --2- 172.21.15.43:0/4233408697 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8f9c039770 0x7f8f9c03bc30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8006fd0 tx=0x7f8fa8006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:23.120 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.581+0000 7f8fb8a70700 1 -- 172.21.15.43:0/4233408697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fb407de60 msgr2=0x7f8fb419e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:23.120 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.581+0000 7f8fb8a70700 1 --2- 172.21.15.43:0/4233408697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fb407de60 0x7f8fb419e350 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8fa40116e0 tx=0x7f8fa4011aa0 comp rx=0 tx=0).stop 2024-04-21T08:29:23.120 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.581+0000 7f8fb8a70700 1 -- 172.21.15.43:0/4233408697 shutdown_connections 2024-04-21T08:29:23.120 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.582+0000 7f8fb8a70700 1 --2- 172.21.15.43:0/4233408697 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8f9c039770 0x7f8f9c03bc30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:23.120 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.582+0000 7f8fb8a70700 1 --2- 172.21.15.43:0/4233408697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fb407de60 0x7f8fb419e350 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:23.120 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.582+0000 7f8fb8a70700 1 -- 172.21.15.43:0/4233408697 >> 172.21.15.43:0/4233408697 conn(0x7f8fb4103af0 msgr2=0x7f8fb4104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:23.120 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.582+0000 7f8fb8a70700 1 -- 172.21.15.43:0/4233408697 shutdown_connections 2024-04-21T08:29:23.120 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:22.582+0000 7f8fb8a70700 1 -- 172.21.15.43:0/4233408697 wait complete. 2024-04-21T08:29:23.120 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-21T08:29:23.120 INFO:teuthology.orchestra.run.smithi043.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-21T08:29:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:29:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:29:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:29:23.121 INFO:teuthology.orchestra.run.smithi043.stdout:Or, if you are only running a single cluster on this host: 2024-04-21T08:29:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:29:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-21T08:29:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:29:23.121 INFO:teuthology.orchestra.run.smithi043.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-21T08:29:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:29:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: ceph telemetry on 2024-04-21T08:29:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:29:23.121 INFO:teuthology.orchestra.run.smithi043.stdout:For more information see: 2024-04-21T08:29:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:29:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-21T08:29:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:29:23.121 INFO:teuthology.orchestra.run.smithi043.stdout:Bootstrap complete. 2024-04-21T08:29:23.158 INFO:tasks.cephadm:Fetching config... 2024-04-21T08:29:23.158 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-21T08:29:23.158 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-21T08:29:23.174 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-21T08:29:23.175 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-21T08:29:23.175 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-21T08:29:23.231 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-21T08:29:23.231 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-21T08:29:23.231 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-21T08:29:23.295 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-21T08:29:23.296 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-21T08:29:23.296 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-21T08:29:23.351 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-21T08:29:23.351 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnfcfO6EmZIk+jlEg1uY5awyUdoyC7l44f6hqhPS7a72XoYQ+clUe6JS5eLwrGep3vYQ0AHRwmmUpilnnN22IwR2cmkn/g8X2y7znJ6GFAJj9lQp6DEwhkYjV/0KIbPZ5ftPGBYev7xTjaLLf1BwJ5G0LKpMAm+hQQwolUXuZ+TVUFuUZmtjtXnHk2Ye7gbpE8tcpLX94cK1MFiqaRww8pneNFIsb1V4Ha0a/ghXKy3vjQAVM/FNYgEcpQNWbfiQAQ7S6LDV0k004MclVPkIk/huIovQF36i0OO9jPFZ58xCet48B4j4j6oNasK/q3R8TjZmpcgok1sk0d5H0PWnFA21YyLE5JbVP+bGQvG0s2zvYRVJ+1eEcYB7GLpLsnanFLrfvtvSTybk0dyXZ4qGCxmJoKmSzjBMFGorvFjYcixW6ysjwh6y3HvhPXOPkG/ZOAE1KW6sdCB7YtHymiUp/OPz/dywgDVFMXl1C4QXACpgPuVerYQROLfVJIcU8XWu0= ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-21T08:29:23.425 INFO:teuthology.orchestra.run.smithi043.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnfcfO6EmZIk+jlEg1uY5awyUdoyC7l44f6hqhPS7a72XoYQ+clUe6JS5eLwrGep3vYQ0AHRwmmUpilnnN22IwR2cmkn/g8X2y7znJ6GFAJj9lQp6DEwhkYjV/0KIbPZ5ftPGBYev7xTjaLLf1BwJ5G0LKpMAm+hQQwolUXuZ+TVUFuUZmtjtXnHk2Ye7gbpE8tcpLX94cK1MFiqaRww8pneNFIsb1V4Ha0a/ghXKy3vjQAVM/FNYgEcpQNWbfiQAQ7S6LDV0k004MclVPkIk/huIovQF36i0OO9jPFZ58xCet48B4j4j6oNasK/q3R8TjZmpcgok1sk0d5H0PWnFA21YyLE5JbVP+bGQvG0s2zvYRVJ+1eEcYB7GLpLsnanFLrfvtvSTybk0dyXZ4qGCxmJoKmSzjBMFGorvFjYcixW6ysjwh6y3HvhPXOPkG/ZOAE1KW6sdCB7YtHymiUp/OPz/dywgDVFMXl1C4QXACpgPuVerYQROLfVJIcU8XWu0= ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:29:23.437 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnfcfO6EmZIk+jlEg1uY5awyUdoyC7l44f6hqhPS7a72XoYQ+clUe6JS5eLwrGep3vYQ0AHRwmmUpilnnN22IwR2cmkn/g8X2y7znJ6GFAJj9lQp6DEwhkYjV/0KIbPZ5ftPGBYev7xTjaLLf1BwJ5G0LKpMAm+hQQwolUXuZ+TVUFuUZmtjtXnHk2Ye7gbpE8tcpLX94cK1MFiqaRww8pneNFIsb1V4Ha0a/ghXKy3vjQAVM/FNYgEcpQNWbfiQAQ7S6LDV0k004MclVPkIk/huIovQF36i0OO9jPFZ58xCet48B4j4j6oNasK/q3R8TjZmpcgok1sk0d5H0PWnFA21YyLE5JbVP+bGQvG0s2zvYRVJ+1eEcYB7GLpLsnanFLrfvtvSTybk0dyXZ4qGCxmJoKmSzjBMFGorvFjYcixW6ysjwh6y3HvhPXOPkG/ZOAE1KW6sdCB7YtHymiUp/OPz/dywgDVFMXl1C4QXACpgPuVerYQROLfVJIcU8XWu0= ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-21T08:29:23.478 INFO:teuthology.orchestra.run.smithi112.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnfcfO6EmZIk+jlEg1uY5awyUdoyC7l44f6hqhPS7a72XoYQ+clUe6JS5eLwrGep3vYQ0AHRwmmUpilnnN22IwR2cmkn/g8X2y7znJ6GFAJj9lQp6DEwhkYjV/0KIbPZ5ftPGBYev7xTjaLLf1BwJ5G0LKpMAm+hQQwolUXuZ+TVUFuUZmtjtXnHk2Ye7gbpE8tcpLX94cK1MFiqaRww8pneNFIsb1V4Ha0a/ghXKy3vjQAVM/FNYgEcpQNWbfiQAQ7S6LDV0k004MclVPkIk/huIovQF36i0OO9jPFZ58xCet48B4j4j6oNasK/q3R8TjZmpcgok1sk0d5H0PWnFA21YyLE5JbVP+bGQvG0s2zvYRVJ+1eEcYB7GLpLsnanFLrfvtvSTybk0dyXZ4qGCxmJoKmSzjBMFGorvFjYcixW6ysjwh6y3HvhPXOPkG/ZOAE1KW6sdCB7YtHymiUp/OPz/dywgDVFMXl1C4QXACpgPuVerYQROLfVJIcU8XWu0= ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:29:23.490 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnfcfO6EmZIk+jlEg1uY5awyUdoyC7l44f6hqhPS7a72XoYQ+clUe6JS5eLwrGep3vYQ0AHRwmmUpilnnN22IwR2cmkn/g8X2y7znJ6GFAJj9lQp6DEwhkYjV/0KIbPZ5ftPGBYev7xTjaLLf1BwJ5G0LKpMAm+hQQwolUXuZ+TVUFuUZmtjtXnHk2Ye7gbpE8tcpLX94cK1MFiqaRww8pneNFIsb1V4Ha0a/ghXKy3vjQAVM/FNYgEcpQNWbfiQAQ7S6LDV0k004MclVPkIk/huIovQF36i0OO9jPFZ58xCet48B4j4j6oNasK/q3R8TjZmpcgok1sk0d5H0PWnFA21YyLE5JbVP+bGQvG0s2zvYRVJ+1eEcYB7GLpLsnanFLrfvtvSTybk0dyXZ4qGCxmJoKmSzjBMFGorvFjYcixW6ysjwh6y3HvhPXOPkG/ZOAE1KW6sdCB7YtHymiUp/OPz/dywgDVFMXl1C4QXACpgPuVerYQROLfVJIcU8XWu0= ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-21T08:29:23.530 INFO:teuthology.orchestra.run.smithi175.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnfcfO6EmZIk+jlEg1uY5awyUdoyC7l44f6hqhPS7a72XoYQ+clUe6JS5eLwrGep3vYQ0AHRwmmUpilnnN22IwR2cmkn/g8X2y7znJ6GFAJj9lQp6DEwhkYjV/0KIbPZ5ftPGBYev7xTjaLLf1BwJ5G0LKpMAm+hQQwolUXuZ+TVUFuUZmtjtXnHk2Ye7gbpE8tcpLX94cK1MFiqaRww8pneNFIsb1V4Ha0a/ghXKy3vjQAVM/FNYgEcpQNWbfiQAQ7S6LDV0k004MclVPkIk/huIovQF36i0OO9jPFZ58xCet48B4j4j6oNasK/q3R8TjZmpcgok1sk0d5H0PWnFA21YyLE5JbVP+bGQvG0s2zvYRVJ+1eEcYB7GLpLsnanFLrfvtvSTybk0dyXZ4qGCxmJoKmSzjBMFGorvFjYcixW6ysjwh6y3HvhPXOPkG/ZOAE1KW6sdCB7YtHymiUp/OPz/dywgDVFMXl1C4QXACpgPuVerYQROLfVJIcU8XWu0= ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:29:23.542 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-21T08:29:23.790 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:29:23.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:23 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/4233408697' entity='client.admin' 2024-04-21T08:29:25.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.842+0000 7fa7293b8700 1 -- 172.21.15.43:0/3838239499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724103bb0 msgr2=0x7fa724103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:25.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.842+0000 7fa7293b8700 1 --2- 172.21.15.43:0/3838239499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724103bb0 0x7fa724103fd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa714009910 tx=0x7fa714009c20 comp rx=0 tx=0).stop 2024-04-21T08:29:25.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.842+0000 7fa7293b8700 1 -- 172.21.15.43:0/3838239499 shutdown_connections 2024-04-21T08:29:25.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.842+0000 7fa7293b8700 1 --2- 172.21.15.43:0/3838239499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724103bb0 0x7fa724103fd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:25.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.842+0000 7fa7293b8700 1 -- 172.21.15.43:0/3838239499 >> 172.21.15.43:0/3838239499 conn(0x7fa7240fda90 msgr2=0x7fa7240ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:25.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.842+0000 7fa7293b8700 1 -- 172.21.15.43:0/3838239499 shutdown_connections 2024-04-21T08:29:25.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.843+0000 7fa7293b8700 1 -- 172.21.15.43:0/3838239499 wait complete. 2024-04-21T08:29:25.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.843+0000 7fa7293b8700 1 Processor -- start 2024-04-21T08:29:25.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.843+0000 7fa7293b8700 1 -- start start 2024-04-21T08:29:25.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.843+0000 7fa7293b8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724103bb0 0x7fa724076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:25.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.843+0000 7fa7293b8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa724108520 con 0x7fa724103bb0 2024-04-21T08:29:25.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.844+0000 7fa722ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724103bb0 0x7fa724076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:25.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.844+0000 7fa722ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724103bb0 0x7fa724076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49932/0 (socket says 172.21.15.43:49932) 2024-04-21T08:29:25.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.844+0000 7fa722ffd700 1 -- 172.21.15.43:0/4277657230 learned_addr learned my addr 172.21.15.43:0/4277657230 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:25.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.844+0000 7fa722ffd700 1 -- 172.21.15.43:0/4277657230 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa724076c40 con 0x7fa724103bb0 2024-04-21T08:29:25.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.844+0000 7fa722ffd700 1 --2- 172.21.15.43:0/4277657230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724103bb0 0x7fa724076700 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa714005800 tx=0x7fa714009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:25.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.844+0000 7fa70bfff700 1 -- 172.21.15.43:0/4277657230 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa714027070 con 0x7fa724103bb0 2024-04-21T08:29:25.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.844+0000 7fa70bfff700 1 -- 172.21.15.43:0/4277657230 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa71400be10 con 0x7fa724103bb0 2024-04-21T08:29:25.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.844+0000 7fa7293b8700 1 -- 172.21.15.43:0/4277657230 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa724074e30 con 0x7fa724103bb0 2024-04-21T08:29:25.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.844+0000 7fa70bfff700 1 -- 172.21.15.43:0/4277657230 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa71400f460 con 0x7fa724103bb0 2024-04-21T08:29:25.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.844+0000 7fa7293b8700 1 -- 172.21.15.43:0/4277657230 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa724075370 con 0x7fa724103bb0 2024-04-21T08:29:25.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.845+0000 7fa70bfff700 1 -- 172.21.15.43:0/4277657230 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa71402a070 con 0x7fa724103bb0 2024-04-21T08:29:25.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.845+0000 7fa7293b8700 1 -- 172.21.15.43:0/4277657230 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa724108ca0 con 0x7fa724103bb0 2024-04-21T08:29:25.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.845+0000 7fa70bfff700 1 --2- 172.21.15.43:0/4277657230 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa70c0396b0 0x7fa70c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:25.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.845+0000 7fa70bfff700 1 -- 172.21.15.43:0/4277657230 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa71401e030 con 0x7fa724103bb0 2024-04-21T08:29:25.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.845+0000 7fa7227fc700 1 --2- 172.21.15.43:0/4277657230 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa70c0396b0 0x7fa70c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:25.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.846+0000 7fa7227fc700 1 --2- 172.21.15.43:0/4277657230 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa70c0396b0 0x7fa70c03bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa718006fd0 tx=0x7fa718006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:25.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:25.850+0000 7fa70bfff700 1 -- 172.21.15.43:0/4277657230 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa714013070 con 0x7fa724103bb0 2024-04-21T08:29:26.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:26.057+0000 7fa7293b8700 1 -- 172.21.15.43:0/4277657230 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fa724190e90 con 0x7fa724103bb0 2024-04-21T08:29:26.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:26.059+0000 7fa70bfff700 1 -- 172.21.15.43:0/4277657230 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fa714031030 con 0x7fa724103bb0 2024-04-21T08:29:26.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:26.061+0000 7fa7293b8700 1 -- 172.21.15.43:0/4277657230 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa70c0396b0 msgr2=0x7fa70c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:26.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:26.061+0000 7fa7293b8700 1 --2- 172.21.15.43:0/4277657230 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa70c0396b0 0x7fa70c03bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa718006fd0 tx=0x7fa718006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:26.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:26.061+0000 7fa7293b8700 1 -- 172.21.15.43:0/4277657230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724103bb0 msgr2=0x7fa724076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:26.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:26.061+0000 7fa7293b8700 1 --2- 172.21.15.43:0/4277657230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724103bb0 0x7fa724076700 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa714005800 tx=0x7fa714009f60 comp rx=0 tx=0).stop 2024-04-21T08:29:26.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:26.061+0000 7fa7293b8700 1 -- 172.21.15.43:0/4277657230 shutdown_connections 2024-04-21T08:29:26.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:26.061+0000 7fa7293b8700 1 --2- 172.21.15.43:0/4277657230 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa70c0396b0 0x7fa70c03bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:26.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:26.061+0000 7fa7293b8700 1 --2- 172.21.15.43:0/4277657230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724103bb0 0x7fa724076700 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:26.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:26.061+0000 7fa7293b8700 1 -- 172.21.15.43:0/4277657230 >> 172.21.15.43:0/4277657230 conn(0x7fa7240fda90 msgr2=0x7fa7240fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:26.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:26.061+0000 7fa7293b8700 1 -- 172.21.15.43:0/4277657230 shutdown_connections 2024-04-21T08:29:26.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:26.061+0000 7fa7293b8700 1 -- 172.21.15.43:0/4277657230 wait complete. 2024-04-21T08:29:26.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:26 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:26.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:26 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:26.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:26 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi043", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:29:26.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:26 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:26.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:26 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:29:26.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:26 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:26.643 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-21T08:29:26.643 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-21T08:29:26.886 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:29:27.265 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:27 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/4277657230' entity='client.admin' 2024-04-21T08:29:27.266 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:27 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:29:27.266 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:27 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:28.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.409+0000 7f13f5077700 1 -- 172.21.15.43:0/1786632939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13f0103bb0 msgr2=0x7f13f0103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:28.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.409+0000 7f13f5077700 1 --2- 172.21.15.43:0/1786632939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13f0103bb0 0x7f13f0103fd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f13e000b0a0 tx=0x7f13e000b3b0 comp rx=0 tx=0).stop 2024-04-21T08:29:28.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.409+0000 7f13f5077700 1 -- 172.21.15.43:0/1786632939 shutdown_connections 2024-04-21T08:29:28.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.409+0000 7f13f5077700 1 --2- 172.21.15.43:0/1786632939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13f0103bb0 0x7f13f0103fd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:28.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.409+0000 7f13f5077700 1 -- 172.21.15.43:0/1786632939 >> 172.21.15.43:0/1786632939 conn(0x7f13f00fda90 msgr2=0x7f13f00ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:28.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.409+0000 7f13f5077700 1 -- 172.21.15.43:0/1786632939 shutdown_connections 2024-04-21T08:29:28.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.409+0000 7f13f5077700 1 -- 172.21.15.43:0/1786632939 wait complete. 2024-04-21T08:29:28.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.410+0000 7f13f5077700 1 Processor -- start 2024-04-21T08:29:28.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.410+0000 7f13f5077700 1 -- start start 2024-04-21T08:29:28.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.410+0000 7f13f5077700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13f0103bb0 0x7f13f0197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:28.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.410+0000 7f13f5077700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f13f0108520 con 0x7f13f0103bb0 2024-04-21T08:29:28.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.410+0000 7f13eed9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13f0103bb0 0x7f13f0197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:28.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.410+0000 7f13eed9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13f0103bb0 0x7f13f0197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40424/0 (socket says 172.21.15.43:40424) 2024-04-21T08:29:28.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.410+0000 7f13eed9d700 1 -- 172.21.15.43:0/2112527375 learned_addr learned my addr 172.21.15.43:0/2112527375 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:28.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.410+0000 7f13eed9d700 1 -- 172.21.15.43:0/2112527375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f01980d0 con 0x7f13f0103bb0 2024-04-21T08:29:28.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.411+0000 7f13eed9d700 1 --2- 172.21.15.43:0/2112527375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13f0103bb0 0x7f13f0197b90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f13e000d010 tx=0x7f13e0003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:28.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.411+0000 7f13e7fff700 1 -- 172.21.15.43:0/2112527375 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13e0021020 con 0x7f13f0103bb0 2024-04-21T08:29:28.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.411+0000 7f13e7fff700 1 -- 172.21.15.43:0/2112527375 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13e000bdd0 con 0x7f13f0103bb0 2024-04-21T08:29:28.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.411+0000 7f13e7fff700 1 -- 172.21.15.43:0/2112527375 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13e0017040 con 0x7f13f0103bb0 2024-04-21T08:29:28.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.411+0000 7f13f5077700 1 -- 172.21.15.43:0/2112527375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f0198360 con 0x7f13f0103bb0 2024-04-21T08:29:28.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.411+0000 7f13f5077700 1 -- 172.21.15.43:0/2112527375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f019b050 con 0x7f13f0103bb0 2024-04-21T08:29:28.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.412+0000 7f13e7fff700 1 -- 172.21.15.43:0/2112527375 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f13e0024020 con 0x7f13f0103bb0 2024-04-21T08:29:28.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.412+0000 7f13f5077700 1 -- 172.21.15.43:0/2112527375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13f0198820 con 0x7f13f0103bb0 2024-04-21T08:29:28.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.412+0000 7f13e7fff700 1 --2- 172.21.15.43:0/2112527375 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f13dc0396b0 0x7f13dc03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:28.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.412+0000 7f13e7fff700 1 -- 172.21.15.43:0/2112527375 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f13e000f050 con 0x7f13f0103bb0 2024-04-21T08:29:28.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.412+0000 7f13ee59c700 1 --2- 172.21.15.43:0/2112527375 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f13dc0396b0 0x7f13dc03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:28.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.412+0000 7f13ee59c700 1 --2- 172.21.15.43:0/2112527375 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f13dc0396b0 0x7f13dc03bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f13d8006fd0 tx=0x7f13d8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:28.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.417+0000 7f13e7fff700 1 -- 172.21.15.43:0/2112527375 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f13e002b020 con 0x7f13f0103bb0 2024-04-21T08:29:28.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.627+0000 7f13f5077700 1 -- 172.21.15.43:0/2112527375 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f13f0062e80 con 0x7f13dc0396b0 2024-04-21T08:29:28.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.631+0000 7f13e7fff700 1 -- 172.21.15.43:0/2112527375 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f13f0062e80 con 0x7f13dc0396b0 2024-04-21T08:29:28.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.633+0000 7f13f5077700 1 -- 172.21.15.43:0/2112527375 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f13dc0396b0 msgr2=0x7f13dc03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:28.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.633+0000 7f13f5077700 1 --2- 172.21.15.43:0/2112527375 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f13dc0396b0 0x7f13dc03bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f13d8006fd0 tx=0x7f13d8006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:28.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.633+0000 7f13f5077700 1 -- 172.21.15.43:0/2112527375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13f0103bb0 msgr2=0x7f13f0197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:28.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.633+0000 7f13f5077700 1 --2- 172.21.15.43:0/2112527375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13f0103bb0 0x7f13f0197b90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f13e000d010 tx=0x7f13e0003ce0 comp rx=0 tx=0).stop 2024-04-21T08:29:28.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.633+0000 7f13f5077700 1 -- 172.21.15.43:0/2112527375 shutdown_connections 2024-04-21T08:29:28.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.633+0000 7f13f5077700 1 --2- 172.21.15.43:0/2112527375 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f13dc0396b0 0x7f13dc03bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:28.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.633+0000 7f13f5077700 1 --2- 172.21.15.43:0/2112527375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13f0103bb0 0x7f13f0197b90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:28.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.633+0000 7f13f5077700 1 -- 172.21.15.43:0/2112527375 >> 172.21.15.43:0/2112527375 conn(0x7f13f00fda90 msgr2=0x7f13f00fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:28.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.633+0000 7f13f5077700 1 -- 172.21.15.43:0/2112527375 shutdown_connections 2024-04-21T08:29:28.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:28.633+0000 7f13f5077700 1 -- 172.21.15.43:0/2112527375 wait complete. 2024-04-21T08:29:29.416 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi112 2024-04-21T08:29:29.416 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-21T08:29:29.416 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/etc/ceph/ceph.conf 2024-04-21T08:29:29.435 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-21T08:29:29.435 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:29:29.493 INFO:tasks.cephadm:Adding host smithi112 to orchestrator... 2024-04-21T08:29:29.493 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph orch host add smithi112 2024-04-21T08:29:29.739 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:29:29.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:29 smithi043 ceph-mon[27978]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:29:29.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:29.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:29:29.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:29:29.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:29:29.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:29 smithi043 ceph-mon[27978]: Updating smithi043:/etc/ceph/ceph.conf 2024-04-21T08:29:29.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:29.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:29.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:30.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:30 smithi043 ceph-mon[27978]: Updating smithi043:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:29:30.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:30 smithi043 ceph-mon[27978]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:29:30.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:30 smithi043 ceph-mon[27978]: Updating smithi043:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:29:31.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.172+0000 7fc82c9f4700 1 -- 172.21.15.43:0/4025192151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc828103650 msgr2=0x7fc828103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:31.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.172+0000 7fc82c9f4700 1 --2- 172.21.15.43:0/4025192151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc828103650 0x7fc828103a30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc818009910 tx=0x7fc818009c20 comp rx=0 tx=0).stop 2024-04-21T08:29:31.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.172+0000 7fc82c9f4700 1 -- 172.21.15.43:0/4025192151 shutdown_connections 2024-04-21T08:29:31.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.172+0000 7fc82c9f4700 1 --2- 172.21.15.43:0/4025192151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc828103650 0x7fc828103a30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:31.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.172+0000 7fc82c9f4700 1 -- 172.21.15.43:0/4025192151 >> 172.21.15.43:0/4025192151 conn(0x7fc8280fcea0 msgr2=0x7fc8280ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:31.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.172+0000 7fc82c9f4700 1 -- 172.21.15.43:0/4025192151 shutdown_connections 2024-04-21T08:29:31.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.172+0000 7fc82c9f4700 1 -- 172.21.15.43:0/4025192151 wait complete. 2024-04-21T08:29:31.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.173+0000 7fc82c9f4700 1 Processor -- start 2024-04-21T08:29:31.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.173+0000 7fc82c9f4700 1 -- start start 2024-04-21T08:29:31.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.173+0000 7fc82c9f4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc828103650 0x7fc8281012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:31.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.173+0000 7fc82c9f4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc82810c9d0 con 0x7fc828103650 2024-04-21T08:29:31.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.173+0000 7fc82659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc828103650 0x7fc8281012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:31.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.173+0000 7fc82659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc828103650 0x7fc8281012b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40452/0 (socket says 172.21.15.43:40452) 2024-04-21T08:29:31.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.173+0000 7fc82659c700 1 -- 172.21.15.43:0/4002988093 learned_addr learned my addr 172.21.15.43:0/4002988093 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:31.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.173+0000 7fc82659c700 1 -- 172.21.15.43:0/4002988093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8281031a0 con 0x7fc828103650 2024-04-21T08:29:31.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.173+0000 7fc82659c700 1 --2- 172.21.15.43:0/4002988093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc828103650 0x7fc8281012b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc818006010 tx=0x7fc818005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:31.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.173+0000 7fc8177fe700 1 -- 172.21.15.43:0/4002988093 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc818026070 con 0x7fc828103650 2024-04-21T08:29:31.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.173+0000 7fc8177fe700 1 -- 172.21.15.43:0/4002988093 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc81800bc00 con 0x7fc828103650 2024-04-21T08:29:31.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.173+0000 7fc82c9f4700 1 -- 172.21.15.43:0/4002988093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8281017f0 con 0x7fc828103650 2024-04-21T08:29:31.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.174+0000 7fc8177fe700 1 -- 172.21.15.43:0/4002988093 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc818003910 con 0x7fc828103650 2024-04-21T08:29:31.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.174+0000 7fc82c9f4700 1 -- 172.21.15.43:0/4002988093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc828101cb0 con 0x7fc828103650 2024-04-21T08:29:31.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.174+0000 7fc8177fe700 1 -- 172.21.15.43:0/4002988093 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc818029070 con 0x7fc828103650 2024-04-21T08:29:31.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.175+0000 7fc8177fe700 1 --2- 172.21.15.43:0/4002988093 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc810039650 0x7fc81003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:31.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.175+0000 7fc8177fe700 1 -- 172.21.15.43:0/4002988093 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc81801e070 con 0x7fc828103650 2024-04-21T08:29:31.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.175+0000 7fc82c9f4700 1 -- 172.21.15.43:0/4002988093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc82806b190 con 0x7fc828103650 2024-04-21T08:29:31.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.175+0000 7fc825d9b700 1 --2- 172.21.15.43:0/4002988093 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc810039650 0x7fc81003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:31.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.175+0000 7fc825d9b700 1 --2- 172.21.15.43:0/4002988093 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc810039650 0x7fc81003bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc81c006fd0 tx=0x7fc81c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:31.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.180+0000 7fc8177fe700 1 -- 172.21.15.43:0/4002988093 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc818030070 con 0x7fc828103650 2024-04-21T08:29:31.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:31.390+0000 7fc82c9f4700 1 -- 172.21.15.43:0/4002988093 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi112", "target": ["mon-mgr", ""]}) -- 0x7fc8281027e0 con 0x7fc810039650 2024-04-21T08:29:32.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:31 smithi043 ceph-mon[27978]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi112", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:29:33.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:32 smithi043 ceph-mon[27978]: Deploying cephadm binary to smithi112 2024-04-21T08:29:35.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:34 smithi043 ceph-mon[27978]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:29:35.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:35.838+0000 7fc8177fe700 1 -- 172.21.15.43:0/4002988093 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fc8281027e0 con 0x7fc810039650 2024-04-21T08:29:35.838 INFO:teuthology.orchestra.run.smithi043.stdout:Added host 'smithi112' with addr '172.21.15.112' 2024-04-21T08:29:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:35.841+0000 7fc82c9f4700 1 -- 172.21.15.43:0/4002988093 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc810039650 msgr2=0x7fc81003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:35.841+0000 7fc82c9f4700 1 --2- 172.21.15.43:0/4002988093 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc810039650 0x7fc81003bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc81c006fd0 tx=0x7fc81c006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:35.841+0000 7fc82c9f4700 1 -- 172.21.15.43:0/4002988093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc828103650 msgr2=0x7fc8281012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:35.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:35.841+0000 7fc82c9f4700 1 --2- 172.21.15.43:0/4002988093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc828103650 0x7fc8281012b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc818006010 tx=0x7fc818005780 comp rx=0 tx=0).stop 2024-04-21T08:29:35.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:35.841+0000 7fc82c9f4700 1 -- 172.21.15.43:0/4002988093 shutdown_connections 2024-04-21T08:29:35.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:35.841+0000 7fc82c9f4700 1 --2- 172.21.15.43:0/4002988093 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc810039650 0x7fc81003bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:35.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:35.841+0000 7fc82c9f4700 1 --2- 172.21.15.43:0/4002988093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc828103650 0x7fc8281012b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:35.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:35.841+0000 7fc82c9f4700 1 -- 172.21.15.43:0/4002988093 >> 172.21.15.43:0/4002988093 conn(0x7fc8280fcea0 msgr2=0x7fc8281043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:35.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:35.841+0000 7fc82c9f4700 1 -- 172.21.15.43:0/4002988093 shutdown_connections 2024-04-21T08:29:35.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:35.841+0000 7fc82c9f4700 1 -- 172.21.15.43:0/4002988093 wait complete. 2024-04-21T08:29:36.458 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-21T08:29:36.707 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:29:37.142 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:36 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:37.142 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:36 smithi043 ceph-mon[27978]: Added host smithi112 2024-04-21T08:29:37.143 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:36 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:29:37.143 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:36 smithi043 ceph-mon[27978]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:29:37.143 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:36 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:38.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.095+0000 7f70bac8b700 1 -- 172.21.15.43:0/3459703949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b41015f0 msgr2=0x7f70b41019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:38.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.095+0000 7f70bac8b700 1 --2- 172.21.15.43:0/3459703949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b41015f0 0x7f70b41019d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f709c009910 tx=0x7f709c009c20 comp rx=0 tx=0).stop 2024-04-21T08:29:38.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.096+0000 7f70bac8b700 1 -- 172.21.15.43:0/3459703949 shutdown_connections 2024-04-21T08:29:38.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.096+0000 7f70bac8b700 1 --2- 172.21.15.43:0/3459703949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b41015f0 0x7f70b41019d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:38.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.096+0000 7f70bac8b700 1 -- 172.21.15.43:0/3459703949 >> 172.21.15.43:0/3459703949 conn(0x7f70b40fcea0 msgr2=0x7f70b40ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:38.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.096+0000 7f70bac8b700 1 -- 172.21.15.43:0/3459703949 shutdown_connections 2024-04-21T08:29:38.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.096+0000 7f70bac8b700 1 -- 172.21.15.43:0/3459703949 wait complete. 2024-04-21T08:29:38.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.096+0000 7f70bac8b700 1 Processor -- start 2024-04-21T08:29:38.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.096+0000 7f70bac8b700 1 -- start start 2024-04-21T08:29:38.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.096+0000 7f70bac8b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b41015f0 0x7f70b4103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:38.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.096+0000 7f70bac8b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f70b410b870 con 0x7f70b41015f0 2024-04-21T08:29:38.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.096+0000 7f70b8a27700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b41015f0 0x7f70b4103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:38.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.096+0000 7f70b8a27700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b41015f0 0x7f70b4103090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46230/0 (socket says 172.21.15.43:46230) 2024-04-21T08:29:38.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.097+0000 7f70b8a27700 1 -- 172.21.15.43:0/3757009701 learned_addr learned my addr 172.21.15.43:0/3757009701 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:38.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.097+0000 7f70b8a27700 1 -- 172.21.15.43:0/3757009701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b4104f80 con 0x7f70b41015f0 2024-04-21T08:29:38.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.097+0000 7f70b8a27700 1 --2- 172.21.15.43:0/3757009701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b41015f0 0x7f70b4103090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f709c0056d0 tx=0x7f709c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:38.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.097+0000 7f70b1ffb700 1 -- 172.21.15.43:0/3757009701 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f709c026070 con 0x7f70b41015f0 2024-04-21T08:29:38.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.097+0000 7f70b1ffb700 1 -- 172.21.15.43:0/3757009701 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f709c00bc00 con 0x7f70b41015f0 2024-04-21T08:29:38.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.097+0000 7f70bac8b700 1 -- 172.21.15.43:0/3757009701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b4103630 con 0x7f70b41015f0 2024-04-21T08:29:38.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.097+0000 7f70b1ffb700 1 -- 172.21.15.43:0/3757009701 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f709c003920 con 0x7f70b41015f0 2024-04-21T08:29:38.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.097+0000 7f70bac8b700 1 -- 172.21.15.43:0/3757009701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b4103af0 con 0x7f70b41015f0 2024-04-21T08:29:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.098+0000 7f70b1ffb700 1 -- 172.21.15.43:0/3757009701 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f709c029070 con 0x7f70b41015f0 2024-04-21T08:29:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.098+0000 7f70bac8b700 1 -- 172.21.15.43:0/3757009701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70b4102ad0 con 0x7f70b41015f0 2024-04-21T08:29:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.098+0000 7f70b1ffb700 1 --2- 172.21.15.43:0/3757009701 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f70a00396b0 0x7f70a003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.098+0000 7f70b1ffb700 1 -- 172.21.15.43:0/3757009701 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f709c01e070 con 0x7f70b41015f0 2024-04-21T08:29:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.099+0000 7f70b3fff700 1 --2- 172.21.15.43:0/3757009701 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f70a00396b0 0x7f70a003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.099+0000 7f70b3fff700 1 --2- 172.21.15.43:0/3757009701 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f70a00396b0 0x7f70a003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f70a4006fd0 tx=0x7f70a4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:38.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.103+0000 7f70b1ffb700 1 -- 172.21.15.43:0/3757009701 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f709c013070 con 0x7f70b41015f0 2024-04-21T08:29:38.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:37 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:38.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.318+0000 7f70bac8b700 1 -- 172.21.15.43:0/3757009701 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f70b41045c0 con 0x7f70a00396b0 2024-04-21T08:29:38.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.319+0000 7f70b1ffb700 1 -- 172.21.15.43:0/3757009701 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f70b41045c0 con 0x7f70a00396b0 2024-04-21T08:29:38.318 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:29:38.318 INFO:teuthology.orchestra.run.smithi043.stdout:[{"addr": "172.21.15.43", "hostname": "smithi043", "labels": [], "status": ""}, {"addr": "172.21.15.112", "hostname": "smithi112", "labels": [], "status": ""}] 2024-04-21T08:29:38.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.321+0000 7f70bac8b700 1 -- 172.21.15.43:0/3757009701 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f70a00396b0 msgr2=0x7f70a003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:38.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.321+0000 7f70bac8b700 1 --2- 172.21.15.43:0/3757009701 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f70a00396b0 0x7f70a003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f70a4006fd0 tx=0x7f70a4006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:38.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.321+0000 7f70bac8b700 1 -- 172.21.15.43:0/3757009701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b41015f0 msgr2=0x7f70b4103090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:38.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.321+0000 7f70bac8b700 1 --2- 172.21.15.43:0/3757009701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b41015f0 0x7f70b4103090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f709c0056d0 tx=0x7f709c0057b0 comp rx=0 tx=0).stop 2024-04-21T08:29:38.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.321+0000 7f70bac8b700 1 -- 172.21.15.43:0/3757009701 shutdown_connections 2024-04-21T08:29:38.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.321+0000 7f70bac8b700 1 --2- 172.21.15.43:0/3757009701 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f70a00396b0 0x7f70a003bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:38.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.321+0000 7f70bac8b700 1 --2- 172.21.15.43:0/3757009701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b41015f0 0x7f70b4103090 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:38.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.321+0000 7f70bac8b700 1 -- 172.21.15.43:0/3757009701 >> 172.21.15.43:0/3757009701 conn(0x7f70b40fcea0 msgr2=0x7f70b406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:38.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.321+0000 7f70bac8b700 1 -- 172.21.15.43:0/3757009701 shutdown_connections 2024-04-21T08:29:38.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:38.321+0000 7f70bac8b700 1 -- 172.21.15.43:0/3757009701 wait complete. 2024-04-21T08:29:38.900 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi175 2024-04-21T08:29:38.900 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-21T08:29:38.900 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/etc/ceph/ceph.conf 2024-04-21T08:29:38.920 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-21T08:29:38.921 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:29:38.977 INFO:tasks.cephadm:Adding host smithi175 to orchestrator... 2024-04-21T08:29:38.978 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph orch host add smithi175 2024-04-21T08:29:39.227 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:29:39.621 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:39 smithi043 ceph-mon[27978]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:29:39.621 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:39 smithi043 ceph-mon[27978]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:29:39.621 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:39 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:40.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.643+0000 7f60ef8d4700 1 -- 172.21.15.43:0/2777515001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60e81084e0 msgr2=0x7f60e81088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:40.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.643+0000 7f60ef8d4700 1 --2- 172.21.15.43:0/2777515001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60e81084e0 0x7f60e81088c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009910 tx=0x7f60e4009c20 comp rx=0 tx=0).stop 2024-04-21T08:29:40.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.643+0000 7f60ef8d4700 1 -- 172.21.15.43:0/2777515001 shutdown_connections 2024-04-21T08:29:40.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.643+0000 7f60ef8d4700 1 --2- 172.21.15.43:0/2777515001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60e81084e0 0x7f60e81088c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:40.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.643+0000 7f60ef8d4700 1 -- 172.21.15.43:0/2777515001 >> 172.21.15.43:0/2777515001 conn(0x7f60e80fdf80 msgr2=0x7f60e81003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:40.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.643+0000 7f60ef8d4700 1 -- 172.21.15.43:0/2777515001 shutdown_connections 2024-04-21T08:29:40.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.643+0000 7f60ef8d4700 1 -- 172.21.15.43:0/2777515001 wait complete. 2024-04-21T08:29:40.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.644+0000 7f60ef8d4700 1 Processor -- start 2024-04-21T08:29:40.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.644+0000 7f60ef8d4700 1 -- start start 2024-04-21T08:29:40.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.644+0000 7f60ef8d4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60e81084e0 0x7f60e819a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:40.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.644+0000 7f60ef8d4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f60e810c940 con 0x7f60e81084e0 2024-04-21T08:29:40.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.644+0000 7f60ed670700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60e81084e0 0x7f60e819a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:40.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.644+0000 7f60ed670700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60e81084e0 0x7f60e819a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46246/0 (socket says 172.21.15.43:46246) 2024-04-21T08:29:40.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.644+0000 7f60ed670700 1 -- 172.21.15.43:0/3773182471 learned_addr learned my addr 172.21.15.43:0/3773182471 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:40.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.645+0000 7f60ed670700 1 -- 172.21.15.43:0/3773182471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e819a620 con 0x7f60e81084e0 2024-04-21T08:29:40.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.645+0000 7f60ed670700 1 --2- 172.21.15.43:0/3773182471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60e81084e0 0x7f60e819a0e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f60e40056d0 tx=0x7f60e40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:40.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.645+0000 7f60de7fc700 1 -- 172.21.15.43:0/3773182471 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60e4026070 con 0x7f60e81084e0 2024-04-21T08:29:40.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.645+0000 7f60de7fc700 1 -- 172.21.15.43:0/3773182471 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60e400bba0 con 0x7f60e81084e0 2024-04-21T08:29:40.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.645+0000 7f60ef8d4700 1 -- 172.21.15.43:0/3773182471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e819a910 con 0x7f60e81084e0 2024-04-21T08:29:40.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.645+0000 7f60de7fc700 1 -- 172.21.15.43:0/3773182471 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60e4003920 con 0x7f60e81084e0 2024-04-21T08:29:40.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.645+0000 7f60ef8d4700 1 -- 172.21.15.43:0/3773182471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e819e9b0 con 0x7f60e81084e0 2024-04-21T08:29:40.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.646+0000 7f60de7fc700 1 -- 172.21.15.43:0/3773182471 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f60e4029070 con 0x7f60e81084e0 2024-04-21T08:29:40.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.646+0000 7f60ef8d4700 1 -- 172.21.15.43:0/3773182471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60e8103b90 con 0x7f60e81084e0 2024-04-21T08:29:40.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.646+0000 7f60de7fc700 1 --2- 172.21.15.43:0/3773182471 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f60d4039650 0x7f60d403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:40.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.646+0000 7f60de7fc700 1 -- 172.21.15.43:0/3773182471 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f60e401e070 con 0x7f60e81084e0 2024-04-21T08:29:40.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.646+0000 7f60ece6f700 1 --2- 172.21.15.43:0/3773182471 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f60d4039650 0x7f60d403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:40.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.647+0000 7f60ece6f700 1 --2- 172.21.15.43:0/3773182471 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f60d4039650 0x7f60d403bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f60d8006fd0 tx=0x7f60d8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:40.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.651+0000 7f60de7fc700 1 -- 172.21.15.43:0/3773182471 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60e4013070 con 0x7f60e81084e0 2024-04-21T08:29:40.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:40.862+0000 7f60ef8d4700 1 -- 172.21.15.43:0/3773182471 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi175", "target": ["mon-mgr", ""]}) -- 0x7f60e819b440 con 0x7f60d4039650 2024-04-21T08:29:41.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:41 smithi043 ceph-mon[27978]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:29:42.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:42 smithi043 ceph-mon[27978]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi175", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:29:42.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:42 smithi043 ceph-mon[27978]: Deploying cephadm binary to smithi175 2024-04-21T08:29:42.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:42 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:42.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:42 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:42.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:42 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:42.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:42 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:42.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:42 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:29:42.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:42 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:29:42.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:42 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:29:42.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:42 smithi043 ceph-mon[27978]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-21T08:29:44.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:43 smithi043 ceph-mon[27978]: Updating smithi112:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:29:44.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:43 smithi043 ceph-mon[27978]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:29:45.106 INFO:teuthology.orchestra.run.smithi043.stdout:Added host 'smithi175' with addr '172.21.15.175' 2024-04-21T08:29:45.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:45.107+0000 7f60de7fc700 1 -- 172.21.15.43:0/3773182471 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f60e819b440 con 0x7f60d4039650 2024-04-21T08:29:45.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:45.109+0000 7f60ef8d4700 1 -- 172.21.15.43:0/3773182471 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f60d4039650 msgr2=0x7f60d403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:45.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:45.109+0000 7f60ef8d4700 1 --2- 172.21.15.43:0/3773182471 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f60d4039650 0x7f60d403bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f60d8006fd0 tx=0x7f60d8006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:45.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:45.109+0000 7f60ef8d4700 1 -- 172.21.15.43:0/3773182471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60e81084e0 msgr2=0x7f60e819a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:45.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:45.109+0000 7f60ef8d4700 1 --2- 172.21.15.43:0/3773182471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60e81084e0 0x7f60e819a0e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f60e40056d0 tx=0x7f60e40057b0 comp rx=0 tx=0).stop 2024-04-21T08:29:45.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:45.109+0000 7f60ef8d4700 1 -- 172.21.15.43:0/3773182471 shutdown_connections 2024-04-21T08:29:45.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:45.109+0000 7f60ef8d4700 1 --2- 172.21.15.43:0/3773182471 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f60d4039650 0x7f60d403bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:45.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:45.109+0000 7f60ef8d4700 1 --2- 172.21.15.43:0/3773182471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60e81084e0 0x7f60e819a0e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:45.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:45.109+0000 7f60ef8d4700 1 -- 172.21.15.43:0/3773182471 >> 172.21.15.43:0/3773182471 conn(0x7f60e80fdf80 msgr2=0x7f60e8109f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:45.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:45.109+0000 7f60ef8d4700 1 -- 172.21.15.43:0/3773182471 shutdown_connections 2024-04-21T08:29:45.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:45.109+0000 7f60ef8d4700 1 -- 172.21.15.43:0/3773182471 wait complete. 2024-04-21T08:29:45.763 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:45 smithi043 ceph-mon[27978]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:29:45.763 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:45 smithi043 ceph-mon[27978]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:29:45.764 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:45 smithi043 ceph-mon[27978]: Updating smithi112:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:29:45.764 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:45 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:45.764 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:45 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:45.764 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:45 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:45.764 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:45 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:45.764 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:45 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:29:45.796 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-21T08:29:46.041 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:29:46.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:46 smithi043 ceph-mon[27978]: Added host smithi175 2024-04-21T08:29:46.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:46 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:46.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:46 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:47.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.596+0000 7fe13229c700 1 -- 172.21.15.43:0/2614546393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe12c1084e0 msgr2=0x7fe12c1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.596+0000 7fe13229c700 1 --2- 172.21.15.43:0/2614546393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe12c1084e0 0x7fe12c1088c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe120009910 tx=0x7fe120009c20 comp rx=0 tx=0).stop 2024-04-21T08:29:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.596+0000 7fe13229c700 1 -- 172.21.15.43:0/2614546393 shutdown_connections 2024-04-21T08:29:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.596+0000 7fe13229c700 1 --2- 172.21.15.43:0/2614546393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe12c1084e0 0x7fe12c1088c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.596+0000 7fe13229c700 1 -- 172.21.15.43:0/2614546393 >> 172.21.15.43:0/2614546393 conn(0x7fe12c0fdf80 msgr2=0x7fe12c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.596+0000 7fe13229c700 1 -- 172.21.15.43:0/2614546393 shutdown_connections 2024-04-21T08:29:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.596+0000 7fe13229c700 1 -- 172.21.15.43:0/2614546393 wait complete. 2024-04-21T08:29:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.597+0000 7fe13229c700 1 Processor -- start 2024-04-21T08:29:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.597+0000 7fe13229c700 1 -- start start 2024-04-21T08:29:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.597+0000 7fe13229c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe12c1084e0 0x7fe12c197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.597+0000 7fe13229c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c10c940 con 0x7fe12c1084e0 2024-04-21T08:29:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.598+0000 7fe12bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe12c1084e0 0x7fe12c197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.598+0000 7fe12bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe12c1084e0 0x7fe12c197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47762/0 (socket says 172.21.15.43:47762) 2024-04-21T08:29:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.598+0000 7fe12bfff700 1 -- 172.21.15.43:0/2406413795 learned_addr learned my addr 172.21.15.43:0/2406413795 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:47.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.598+0000 7fe12bfff700 1 -- 172.21.15.43:0/2406413795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12c1983f0 con 0x7fe12c1084e0 2024-04-21T08:29:47.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.598+0000 7fe12bfff700 1 --2- 172.21.15.43:0/2406413795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe12c1084e0 0x7fe12c197eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe1200056d0 tx=0x7fe1200057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:47.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.598+0000 7fe1297fa700 1 -- 172.21.15.43:0/2406413795 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe120026070 con 0x7fe12c1084e0 2024-04-21T08:29:47.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.598+0000 7fe1297fa700 1 -- 172.21.15.43:0/2406413795 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe12000bba0 con 0x7fe12c1084e0 2024-04-21T08:29:47.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.598+0000 7fe13229c700 1 -- 172.21.15.43:0/2406413795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe12c1986e0 con 0x7fe12c1084e0 2024-04-21T08:29:47.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.598+0000 7fe1297fa700 1 -- 172.21.15.43:0/2406413795 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe120003920 con 0x7fe12c1084e0 2024-04-21T08:29:47.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.598+0000 7fe13229c700 1 -- 172.21.15.43:0/2406413795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe12c19c780 con 0x7fe12c1084e0 2024-04-21T08:29:47.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.599+0000 7fe1297fa700 1 -- 172.21.15.43:0/2406413795 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe120029070 con 0x7fe12c1084e0 2024-04-21T08:29:47.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.599+0000 7fe13229c700 1 -- 172.21.15.43:0/2406413795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe12c103b90 con 0x7fe12c1084e0 2024-04-21T08:29:47.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.599+0000 7fe1297fa700 1 --2- 172.21.15.43:0/2406413795 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe114039650 0x7fe11403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:47.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.599+0000 7fe1297fa700 1 -- 172.21.15.43:0/2406413795 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe12001e070 con 0x7fe12c1084e0 2024-04-21T08:29:47.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.599+0000 7fe12b7fe700 1 --2- 172.21.15.43:0/2406413795 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe114039650 0x7fe11403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:47.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.600+0000 7fe12b7fe700 1 --2- 172.21.15.43:0/2406413795 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe114039650 0x7fe11403bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe11c006fd0 tx=0x7fe11c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:47.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.604+0000 7fe1297fa700 1 -- 172.21.15.43:0/2406413795 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe120013070 con 0x7fe12c1084e0 2024-04-21T08:29:47.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:47 smithi043 ceph-mon[27978]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:29:47.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.819+0000 7fe13229c700 1 -- 172.21.15.43:0/2406413795 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe12c199210 con 0x7fe114039650 2024-04-21T08:29:47.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.820+0000 7fe1297fa700 1 -- 172.21.15.43:0/2406413795 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7fe12c199210 con 0x7fe114039650 2024-04-21T08:29:47.819 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:29:47.819 INFO:teuthology.orchestra.run.smithi043.stdout:[{"addr": "172.21.15.43", "hostname": "smithi043", "labels": [], "status": ""}, {"addr": "172.21.15.112", "hostname": "smithi112", "labels": [], "status": ""}, {"addr": "172.21.15.175", "hostname": "smithi175", "labels": [], "status": ""}] 2024-04-21T08:29:47.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.822+0000 7fe13229c700 1 -- 172.21.15.43:0/2406413795 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe114039650 msgr2=0x7fe11403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:47.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.822+0000 7fe13229c700 1 --2- 172.21.15.43:0/2406413795 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe114039650 0x7fe11403bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe11c006fd0 tx=0x7fe11c006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:47.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.822+0000 7fe13229c700 1 -- 172.21.15.43:0/2406413795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe12c1084e0 msgr2=0x7fe12c197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:47.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.822+0000 7fe13229c700 1 --2- 172.21.15.43:0/2406413795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe12c1084e0 0x7fe12c197eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe1200056d0 tx=0x7fe1200057b0 comp rx=0 tx=0).stop 2024-04-21T08:29:47.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.822+0000 7fe13229c700 1 -- 172.21.15.43:0/2406413795 shutdown_connections 2024-04-21T08:29:47.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.822+0000 7fe13229c700 1 --2- 172.21.15.43:0/2406413795 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe114039650 0x7fe11403bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:47.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.822+0000 7fe13229c700 1 --2- 172.21.15.43:0/2406413795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe12c1084e0 0x7fe12c197eb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:47.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.822+0000 7fe13229c700 1 -- 172.21.15.43:0/2406413795 >> 172.21.15.43:0/2406413795 conn(0x7fe12c0fdf80 msgr2=0x7fe12c10a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:47.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.822+0000 7fe13229c700 1 -- 172.21.15.43:0/2406413795 shutdown_connections 2024-04-21T08:29:47.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:47.822+0000 7fe13229c700 1 -- 172.21.15.43:0/2406413795 wait complete. 2024-04-21T08:29:48.411 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-21T08:29:48.411 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-21T08:29:48.657 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:29:48.995 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:48 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:49.900 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:49 smithi043 ceph-mon[27978]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:29:49.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:49 smithi043 ceph-mon[27978]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:29:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.006+0000 7f53080bf700 1 -- 172.21.15.43:0/1051558298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5300103610 msgr2=0x7f53001039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.006+0000 7f53080bf700 1 --2- 172.21.15.43:0/1051558298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5300103610 0x7f53001039f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009910 tx=0x7f52fc009c20 comp rx=0 tx=0).stop 2024-04-21T08:29:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.006+0000 7f53080bf700 1 -- 172.21.15.43:0/1051558298 shutdown_connections 2024-04-21T08:29:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.006+0000 7f53080bf700 1 --2- 172.21.15.43:0/1051558298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5300103610 0x7f53001039f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.006+0000 7f53080bf700 1 -- 172.21.15.43:0/1051558298 >> 172.21.15.43:0/1051558298 conn(0x7f53000fce60 msgr2=0x7f53000ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.006+0000 7f53080bf700 1 -- 172.21.15.43:0/1051558298 shutdown_connections 2024-04-21T08:29:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.006+0000 7f53080bf700 1 -- 172.21.15.43:0/1051558298 wait complete. 2024-04-21T08:29:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.006+0000 7f53080bf700 1 Processor -- start 2024-04-21T08:29:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.006+0000 7f53080bf700 1 -- start start 2024-04-21T08:29:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.007+0000 7f53080bf700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5300103610 0x7f530019e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.007+0000 7f53080bf700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f530010c9b0 con 0x7f5300103610 2024-04-21T08:29:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.007+0000 7f5305e5b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5300103610 0x7f530019e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.007+0000 7f5305e5b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5300103610 0x7f530019e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47792/0 (socket says 172.21.15.43:47792) 2024-04-21T08:29:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.007+0000 7f5305e5b700 1 -- 172.21.15.43:0/2267614155 learned_addr learned my addr 172.21.15.43:0/2267614155 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:29:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.007+0000 7f5305e5b700 1 -- 172.21.15.43:0/2267614155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530019ea40 con 0x7f5300103610 2024-04-21T08:29:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.007+0000 7f5305e5b700 1 --2- 172.21.15.43:0/2267614155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5300103610 0x7f530019e500 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0056d0 tx=0x7f52fc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.007+0000 7f52f6ffd700 1 -- 172.21.15.43:0/2267614155 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52fc026070 con 0x7f5300103610 2024-04-21T08:29:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.007+0000 7f52f6ffd700 1 -- 172.21.15.43:0/2267614155 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52fc004370 con 0x7f5300103610 2024-04-21T08:29:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.007+0000 7f53080bf700 1 -- 172.21.15.43:0/2267614155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f530019ecd0 con 0x7f5300103610 2024-04-21T08:29:50.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.007+0000 7f52f6ffd700 1 -- 172.21.15.43:0/2267614155 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52fc0039b0 con 0x7f5300103610 2024-04-21T08:29:50.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.007+0000 7f53080bf700 1 -- 172.21.15.43:0/2267614155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f530019f130 con 0x7f5300103610 2024-04-21T08:29:50.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.008+0000 7f52f6ffd700 1 -- 172.21.15.43:0/2267614155 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f52fc029070 con 0x7f5300103610 2024-04-21T08:29:50.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.009+0000 7f53080bf700 1 -- 172.21.15.43:0/2267614155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f530006b1d0 con 0x7f5300103610 2024-04-21T08:29:50.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.009+0000 7f52f6ffd700 1 --2- 172.21.15.43:0/2267614155 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f52ec0396b0 0x7f52ec03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:50.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.009+0000 7f52f6ffd700 1 -- 172.21.15.43:0/2267614155 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f52fc01e070 con 0x7f5300103610 2024-04-21T08:29:50.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.009+0000 7f530565a700 1 --2- 172.21.15.43:0/2267614155 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f52ec0396b0 0x7f52ec03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:50.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.009+0000 7f530565a700 1 --2- 172.21.15.43:0/2267614155 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f52ec0396b0 0x7f52ec03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f52f0006fd0 tx=0x7f52f0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:50.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.014+0000 7f52f6ffd700 1 -- 172.21.15.43:0/2267614155 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f52fc013070 con 0x7f5300103610 2024-04-21T08:29:50.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.229+0000 7f53080bf700 1 -- 172.21.15.43:0/2267614155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f530019f860 con 0x7f5300103610 2024-04-21T08:29:50.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.702+0000 7f52f6ffd700 1 -- 172.21.15.43:0/2267614155 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f52fc00bc10 con 0x7f5300103610 2024-04-21T08:29:50.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.705+0000 7f53080bf700 1 -- 172.21.15.43:0/2267614155 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f52ec0396b0 msgr2=0x7f52ec03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:50.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.706+0000 7f53080bf700 1 --2- 172.21.15.43:0/2267614155 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f52ec0396b0 0x7f52ec03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f52f0006fd0 tx=0x7f52f0006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:50.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.706+0000 7f53080bf700 1 -- 172.21.15.43:0/2267614155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5300103610 msgr2=0x7f530019e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:50.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.706+0000 7f53080bf700 1 --2- 172.21.15.43:0/2267614155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5300103610 0x7f530019e500 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0056d0 tx=0x7f52fc0057b0 comp rx=0 tx=0).stop 2024-04-21T08:29:50.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.706+0000 7f53080bf700 1 -- 172.21.15.43:0/2267614155 shutdown_connections 2024-04-21T08:29:50.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.706+0000 7f53080bf700 1 --2- 172.21.15.43:0/2267614155 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f52ec0396b0 0x7f52ec03bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:50.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.706+0000 7f53080bf700 1 --2- 172.21.15.43:0/2267614155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5300103610 0x7f530019e500 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:50.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.706+0000 7f53080bf700 1 -- 172.21.15.43:0/2267614155 >> 172.21.15.43:0/2267614155 conn(0x7f53000fce60 msgr2=0x7f53001011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:50.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.706+0000 7f53080bf700 1 -- 172.21.15.43:0/2267614155 shutdown_connections 2024-04-21T08:29:50.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:29:50.706+0000 7f53080bf700 1 -- 172.21.15.43:0/2267614155 wait complete. 2024-04-21T08:29:50.707 INFO:teuthology.orchestra.run.smithi043.stderr:adjusted tunables profile to default 2024-04-21T08:29:51.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:50 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2267614155' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-21T08:29:51.308 INFO:tasks.cephadm:Adding mon.a on smithi043 2024-04-21T08:29:51.308 INFO:tasks.cephadm:Adding mon.b on smithi112 2024-04-21T08:29:51.308 INFO:tasks.cephadm:Adding mon.c on smithi175 2024-04-21T08:29:51.308 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph orch apply mon '3;smithi043:172.21.15.43=a;smithi112:172.21.15.112=b;smithi175:172.21.15.175=c' 2024-04-21T08:29:51.542 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:29:52.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:51 smithi043 ceph-mon[27978]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:29:52.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:51 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2267614155' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-21T08:29:52.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:51 smithi043 ceph-mon[27978]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:29:52.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:51 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:52.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:51 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:52.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:51 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:52.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:51 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:52.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:51 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi175", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:29:52.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:51 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:29:52.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:51 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:29:52.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:51 smithi043 ceph-mon[27978]: Updating smithi175:/etc/ceph/ceph.conf 2024-04-21T08:29:52.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:51 smithi043 ceph-mon[27978]: Updating smithi175:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:29:52.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:51 smithi043 ceph-mon[27978]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:29:52.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:51 smithi043 ceph-mon[27978]: Updating smithi175:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:29:52.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:51 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:52.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:51 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:52.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:51 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:53.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.027+0000 7fddd8a92700 1 -- 172.21.15.175:0/2622194873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddd41015b0 msgr2=0x7fddd4101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:53.027 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.027+0000 7fddd8a92700 1 --2- 172.21.15.175:0/2622194873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddd41015b0 0x7fddd4101990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fddc4009910 tx=0x7fddc4009c20 comp rx=0 tx=0).stop 2024-04-21T08:29:53.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.027+0000 7fddd8a92700 1 -- 172.21.15.175:0/2622194873 shutdown_connections 2024-04-21T08:29:53.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.027+0000 7fddd8a92700 1 --2- 172.21.15.175:0/2622194873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddd41015b0 0x7fddd4101990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:53.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.027+0000 7fddd8a92700 1 -- 172.21.15.175:0/2622194873 >> 172.21.15.175:0/2622194873 conn(0x7fddd40fce60 msgr2=0x7fddd40ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:53.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.027+0000 7fddd8a92700 1 -- 172.21.15.175:0/2622194873 shutdown_connections 2024-04-21T08:29:53.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.027+0000 7fddd8a92700 1 -- 172.21.15.175:0/2622194873 wait complete. 2024-04-21T08:29:53.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.028+0000 7fddd8a92700 1 Processor -- start 2024-04-21T08:29:53.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.028+0000 7fddd8a92700 1 -- start start 2024-04-21T08:29:53.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.028+0000 7fddd8a92700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddd41015b0 0x7fddd4197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:53.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.028+0000 7fddd8a92700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fddd410b830 con 0x7fddd41015b0 2024-04-21T08:29:53.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.028+0000 7fddd259c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddd41015b0 0x7fddd4197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:53.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.028+0000 7fddd259c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddd41015b0 0x7fddd4197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.175:53166/0 (socket says 172.21.15.175:53166) 2024-04-21T08:29:53.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.028+0000 7fddd259c700 1 -- 172.21.15.175:0/725199445 learned_addr learned my addr 172.21.15.175:0/725199445 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-21T08:29:53.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.029+0000 7fddd259c700 1 -- 172.21.15.175:0/725199445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd41983d0 con 0x7fddd41015b0 2024-04-21T08:29:53.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.029+0000 7fddd259c700 1 --2- 172.21.15.175:0/725199445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddd41015b0 0x7fddd4197e90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fddc40056d0 tx=0x7fddc40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:53.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.029+0000 7fddc37fe700 1 -- 172.21.15.175:0/725199445 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddc4026070 con 0x7fddd41015b0 2024-04-21T08:29:53.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.029+0000 7fddc37fe700 1 -- 172.21.15.175:0/725199445 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddc400bba0 con 0x7fddd41015b0 2024-04-21T08:29:53.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.029+0000 7fddd8a92700 1 -- 172.21.15.175:0/725199445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd41986c0 con 0x7fddd41015b0 2024-04-21T08:29:53.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.029+0000 7fddc37fe700 1 -- 172.21.15.175:0/725199445 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddc4003920 con 0x7fddd41015b0 2024-04-21T08:29:53.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.029+0000 7fddd8a92700 1 -- 172.21.15.175:0/725199445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd419c760 con 0x7fddd41015b0 2024-04-21T08:29:53.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.030+0000 7fddd8a92700 1 -- 172.21.15.175:0/725199445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd4102a90 con 0x7fddd41015b0 2024-04-21T08:29:53.030 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.030+0000 7fddc37fe700 1 -- 172.21.15.175:0/725199445 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fddc4029070 con 0x7fddd41015b0 2024-04-21T08:29:53.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.030+0000 7fddc37fe700 1 --2- 172.21.15.175:0/725199445 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fddbc0396b0 0x7fddbc03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:53.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.030+0000 7fddc37fe700 1 -- 172.21.15.175:0/725199445 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fddc401e070 con 0x7fddd41015b0 2024-04-21T08:29:53.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.030+0000 7fddd1d9b700 1 --2- 172.21.15.175:0/725199445 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fddbc0396b0 0x7fddbc03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:53.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.031+0000 7fddd1d9b700 1 --2- 172.21.15.175:0/725199445 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fddbc0396b0 0x7fddbc03bb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fddc8006fd0 tx=0x7fddc8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:53.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.035+0000 7fddc37fe700 1 -- 172.21.15.175:0/725199445 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fddc4013070 con 0x7fddd41015b0 2024-04-21T08:29:53.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:52 smithi043 ceph-mon[27978]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:29:53.242 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.242+0000 7fddd8a92700 1 -- 172.21.15.175:0/725199445 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi112:172.21.15.112=b;smithi175:172.21.15.175=c", "target": ["mon-mgr", ""]}) -- 0x7fddd41991f0 con 0x7fddbc0396b0 2024-04-21T08:29:53.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.249+0000 7fddc37fe700 1 -- 172.21.15.175:0/725199445 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fddd41991f0 con 0x7fddbc0396b0 2024-04-21T08:29:53.249 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled mon update... 2024-04-21T08:29:53.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.251+0000 7fddd8a92700 1 -- 172.21.15.175:0/725199445 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fddbc0396b0 msgr2=0x7fddbc03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:53.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.251+0000 7fddd8a92700 1 --2- 172.21.15.175:0/725199445 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fddbc0396b0 0x7fddbc03bb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fddc8006fd0 tx=0x7fddc8006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:53.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.251+0000 7fddd8a92700 1 -- 172.21.15.175:0/725199445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddd41015b0 msgr2=0x7fddd4197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:53.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.251+0000 7fddd8a92700 1 --2- 172.21.15.175:0/725199445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddd41015b0 0x7fddd4197e90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fddc40056d0 tx=0x7fddc40057b0 comp rx=0 tx=0).stop 2024-04-21T08:29:53.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.252+0000 7fddd8a92700 1 -- 172.21.15.175:0/725199445 shutdown_connections 2024-04-21T08:29:53.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.252+0000 7fddd8a92700 1 --2- 172.21.15.175:0/725199445 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fddbc0396b0 0x7fddbc03bb70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:53.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.252+0000 7fddd8a92700 1 --2- 172.21.15.175:0/725199445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddd41015b0 0x7fddd4197e90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:53.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.252+0000 7fddd8a92700 1 -- 172.21.15.175:0/725199445 >> 172.21.15.175:0/725199445 conn(0x7fddd40fce60 msgr2=0x7fddd406a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:53.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.252+0000 7fddd8a92700 1 -- 172.21.15.175:0/725199445 shutdown_connections 2024-04-21T08:29:53.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:53.252+0000 7fddd8a92700 1 -- 172.21.15.175:0/725199445 wait complete. 2024-04-21T08:29:53.829 DEBUG:teuthology.orchestra.run.smithi112:mon.b> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mon.b.service 2024-04-21T08:29:53.832 DEBUG:teuthology.orchestra.run.smithi175:mon.c> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mon.c.service 2024-04-21T08:29:53.834 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-21T08:29:53.834 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-21T08:29:54.539 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:29:54.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:54 smithi043 ceph-mon[27978]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi112:172.21.15.112=b;smithi175:172.21.15.175=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:29:54.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:54 smithi043 ceph-mon[27978]: Saving service mon spec with placement smithi043:172.21.15.43=a;smithi112:172.21.15.112=b;smithi175:172.21.15.175=c;count:3 2024-04-21T08:29:54.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:54 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:54.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:54 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:29:54.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:54 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:29:54.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:54 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:29:54.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:54 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:29:54.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:54 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:29:54.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:54 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:29:54.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:54 smithi043 ceph-mon[27978]: Deploying daemon mon.c on smithi175 2024-04-21T08:29:55.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:55 smithi043 ceph-mon[27978]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:29:55.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.862+0000 7f20502d4700 1 -- 172.21.15.175:0/3167265541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20481084e0 msgr2=0x7f20481088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:55.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.862+0000 7f20502d4700 1 --2- 172.21.15.175:0/3167265541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20481084e0 0x7f20481088c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2044009910 tx=0x7f2044009c20 comp rx=0 tx=0).stop 2024-04-21T08:29:55.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.863+0000 7f20502d4700 1 -- 172.21.15.175:0/3167265541 shutdown_connections 2024-04-21T08:29:55.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.863+0000 7f20502d4700 1 --2- 172.21.15.175:0/3167265541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20481084e0 0x7f20481088c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:55.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.863+0000 7f20502d4700 1 -- 172.21.15.175:0/3167265541 >> 172.21.15.175:0/3167265541 conn(0x7f20480fdf80 msgr2=0x7f20481003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:55.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.863+0000 7f20502d4700 1 -- 172.21.15.175:0/3167265541 shutdown_connections 2024-04-21T08:29:55.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.863+0000 7f20502d4700 1 -- 172.21.15.175:0/3167265541 wait complete. 2024-04-21T08:29:55.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.863+0000 7f20502d4700 1 Processor -- start 2024-04-21T08:29:55.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.863+0000 7f20502d4700 1 -- start start 2024-04-21T08:29:55.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.863+0000 7f20502d4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20481084e0 0x7f2048074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:55.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.863+0000 7f20502d4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f204810c940 con 0x7f20481084e0 2024-04-21T08:29:55.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.864+0000 7f204e070700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20481084e0 0x7f2048074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:55.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.864+0000 7f204e070700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20481084e0 0x7f2048074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.175:53192/0 (socket says 172.21.15.175:53192) 2024-04-21T08:29:55.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.864+0000 7f204e070700 1 -- 172.21.15.175:0/640509845 learned_addr learned my addr 172.21.15.175:0/640509845 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-21T08:29:55.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.864+0000 7f204e070700 1 -- 172.21.15.175:0/640509845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2048076c30 con 0x7f20481084e0 2024-04-21T08:29:55.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.864+0000 7f204e070700 1 --2- 172.21.15.175:0/640509845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20481084e0 0x7f2048074d40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f20440056d0 tx=0x7f20440057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:55.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.864+0000 7f203f7fe700 1 -- 172.21.15.175:0/640509845 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2044026070 con 0x7f20481084e0 2024-04-21T08:29:55.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.864+0000 7f203f7fe700 1 -- 172.21.15.175:0/640509845 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f204400bc00 con 0x7f20481084e0 2024-04-21T08:29:55.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.864+0000 7f20502d4700 1 -- 172.21.15.175:0/640509845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20480752e0 con 0x7f20481084e0 2024-04-21T08:29:55.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.864+0000 7f203f7fe700 1 -- 172.21.15.175:0/640509845 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2044003920 con 0x7f20481084e0 2024-04-21T08:29:55.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.864+0000 7f20502d4700 1 -- 172.21.15.175:0/640509845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20480757a0 con 0x7f20481084e0 2024-04-21T08:29:55.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.865+0000 7f20502d4700 1 -- 172.21.15.175:0/640509845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2048103b90 con 0x7f20481084e0 2024-04-21T08:29:55.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.865+0000 7f203f7fe700 1 -- 172.21.15.175:0/640509845 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2044029070 con 0x7f20481084e0 2024-04-21T08:29:55.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.865+0000 7f203f7fe700 1 --2- 172.21.15.175:0/640509845 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2034039650 0x7f203403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:55.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.865+0000 7f203f7fe700 1 -- 172.21.15.175:0/640509845 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f204401e070 con 0x7f20481084e0 2024-04-21T08:29:55.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.866+0000 7f204d86f700 1 --2- 172.21.15.175:0/640509845 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2034039650 0x7f203403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:55.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.866+0000 7f204d86f700 1 --2- 172.21.15.175:0/640509845 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2034039650 0x7f203403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2038006fd0 tx=0x7f2038006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:55.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:55.871+0000 7f203f7fe700 1 -- 172.21.15.175:0/640509845 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2044013070 con 0x7f20481084e0 2024-04-21T08:29:56.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:56.153+0000 7f20502d4700 1 -- 172.21.15.175:0/640509845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2048076270 con 0x7f20481084e0 2024-04-21T08:29:56.153 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:56.154+0000 7f203f7fe700 1 -- 172.21.15.175:0/640509845 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f2044030090 con 0x7f20481084e0 2024-04-21T08:29:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-21T08:29:56.153 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","modified":"2024-04-21T08:28:08.995135Z","created":"2024-04-21T08:28:08.995135Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-21T08:29:56.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:56.156+0000 7f20502d4700 1 -- 172.21.15.175:0/640509845 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2034039650 msgr2=0x7f203403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:56.155 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:56.156+0000 7f20502d4700 1 --2- 172.21.15.175:0/640509845 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2034039650 0x7f203403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2038006fd0 tx=0x7f2038006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:56.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:56.156+0000 7f20502d4700 1 -- 172.21.15.175:0/640509845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20481084e0 msgr2=0x7f2048074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:56.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:56.156+0000 7f20502d4700 1 --2- 172.21.15.175:0/640509845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20481084e0 0x7f2048074d40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f20440056d0 tx=0x7f20440057b0 comp rx=0 tx=0).stop 2024-04-21T08:29:56.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:56.156+0000 7f20502d4700 1 -- 172.21.15.175:0/640509845 shutdown_connections 2024-04-21T08:29:56.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:56.156+0000 7f20502d4700 1 --2- 172.21.15.175:0/640509845 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2034039650 0x7f203403bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:56.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:56.156+0000 7f20502d4700 1 --2- 172.21.15.175:0/640509845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20481084e0 0x7f2048074d40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:56.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:56.156+0000 7f20502d4700 1 -- 172.21.15.175:0/640509845 >> 172.21.15.175:0/640509845 conn(0x7f20480fdf80 msgr2=0x7f204810a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:56.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:56.156+0000 7f20502d4700 1 -- 172.21.15.175:0/640509845 shutdown_connections 2024-04-21T08:29:56.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:56.156+0000 7f20502d4700 1 -- 172.21.15.175:0/640509845 wait complete. 2024-04-21T08:29:56.157 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2024-04-21T08:29:56.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:56 smithi043 ceph-mon[27978]: from='client.? 172.21.15.175:0/640509845' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T08:29:57.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:57 smithi043 ceph-mon[27978]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:29:57.787 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-21T08:29:57.787 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-21T08:29:58.076 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:29:59.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:29:59 smithi043 ceph-mon[27978]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:29:59.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.749+0000 7efe97bed700 1 -- 172.21.15.175:0/649403389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe90108630 msgr2=0x7efe90108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.749+0000 7efe97bed700 1 --2- 172.21.15.175:0/649403389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe90108630 0x7efe90108a10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009910 tx=0x7efe8c009c20 comp rx=0 tx=0).stop 2024-04-21T08:29:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.749+0000 7efe97bed700 1 -- 172.21.15.175:0/649403389 shutdown_connections 2024-04-21T08:29:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.749+0000 7efe97bed700 1 --2- 172.21.15.175:0/649403389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe90108630 0x7efe90108a10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.749+0000 7efe97bed700 1 -- 172.21.15.175:0/649403389 >> 172.21.15.175:0/649403389 conn(0x7efe900fe050 msgr2=0x7efe90100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.749+0000 7efe97bed700 1 -- 172.21.15.175:0/649403389 shutdown_connections 2024-04-21T08:29:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.749+0000 7efe97bed700 1 -- 172.21.15.175:0/649403389 wait complete. 2024-04-21T08:29:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.750+0000 7efe97bed700 1 Processor -- start 2024-04-21T08:29:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.750+0000 7efe97bed700 1 -- start start 2024-04-21T08:29:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.750+0000 7efe97bed700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe90108630 0x7efe90197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.750+0000 7efe97bed700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efe9010caa0 con 0x7efe90108630 2024-04-21T08:29:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.750+0000 7efe95989700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe90108630 0x7efe90197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.750+0000 7efe95989700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe90108630 0x7efe90197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.175:50510/0 (socket says 172.21.15.175:50510) 2024-04-21T08:29:59.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.750+0000 7efe95989700 1 -- 172.21.15.175:0/958335121 learned_addr learned my addr 172.21.15.175:0/958335121 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-21T08:29:59.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.751+0000 7efe95989700 1 -- 172.21.15.175:0/958335121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe901984d0 con 0x7efe90108630 2024-04-21T08:29:59.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.751+0000 7efe95989700 1 --2- 172.21.15.175:0/958335121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe90108630 0x7efe90197f90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efe8c0056d0 tx=0x7efe8c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:59.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.751+0000 7efe86ffd700 1 -- 172.21.15.175:0/958335121 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe8c026070 con 0x7efe90108630 2024-04-21T08:29:59.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.751+0000 7efe97bed700 1 -- 172.21.15.175:0/958335121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe90198760 con 0x7efe90108630 2024-04-21T08:29:59.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.751+0000 7efe86ffd700 1 -- 172.21.15.175:0/958335121 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe8c00bc00 con 0x7efe90108630 2024-04-21T08:29:59.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.751+0000 7efe86ffd700 1 -- 172.21.15.175:0/958335121 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe8c003920 con 0x7efe90108630 2024-04-21T08:29:59.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.751+0000 7efe97bed700 1 -- 172.21.15.175:0/958335121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe9019c860 con 0x7efe90108630 2024-04-21T08:29:59.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.752+0000 7efe97bed700 1 -- 172.21.15.175:0/958335121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe90103ca0 con 0x7efe90108630 2024-04-21T08:29:59.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.752+0000 7efe86ffd700 1 -- 172.21.15.175:0/958335121 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7efe8c029070 con 0x7efe90108630 2024-04-21T08:29:59.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.752+0000 7efe86ffd700 1 --2- 172.21.15.175:0/958335121 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7efe7c039650 0x7efe7c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:59.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.752+0000 7efe86ffd700 1 -- 172.21.15.175:0/958335121 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efe8c01e070 con 0x7efe90108630 2024-04-21T08:29:59.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.753+0000 7efe95188700 1 --2- 172.21.15.175:0/958335121 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7efe7c039650 0x7efe7c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:59.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.753+0000 7efe95188700 1 --2- 172.21.15.175:0/958335121 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7efe7c039650 0x7efe7c03bb10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efe80006fd0 tx=0x7efe80006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:59.757 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:29:59.758+0000 7efe86ffd700 1 -- 172.21.15.175:0/958335121 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efe8c013070 con 0x7efe90108630 2024-04-21T08:30:00.008 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:29:59 smithi175 podman[31889]: 2024-04-21 08:29:59.906208047 +0000 UTC m=+0.118075924 container create 44beb75e459f9b983e3d3775928b4d24608fb57db35119d9abdd709e72249d20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-c, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-21T08:30:00.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:00.048+0000 7efe97bed700 1 -- 172.21.15.175:0/958335121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7efe901992f0 con 0x7efe90108630 2024-04-21T08:30:00.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:00.049+0000 7efe86ffd700 1 -- 172.21.15.175:0/958335121 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7efe8c030030 con 0x7efe90108630 2024-04-21T08:30:00.049 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-21T08:30:00.049 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","modified":"2024-04-21T08:28:08.995135Z","created":"2024-04-21T08:28:08.995135Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-21T08:30:00.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:00.051+0000 7efe97bed700 1 -- 172.21.15.175:0/958335121 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7efe7c039650 msgr2=0x7efe7c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:00.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:00.051+0000 7efe97bed700 1 --2- 172.21.15.175:0/958335121 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7efe7c039650 0x7efe7c03bb10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efe80006fd0 tx=0x7efe80006e40 comp rx=0 tx=0).stop 2024-04-21T08:30:00.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:00.051+0000 7efe97bed700 1 -- 172.21.15.175:0/958335121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe90108630 msgr2=0x7efe90197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:00.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:00.051+0000 7efe97bed700 1 --2- 172.21.15.175:0/958335121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe90108630 0x7efe90197f90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efe8c0056d0 tx=0x7efe8c0057b0 comp rx=0 tx=0).stop 2024-04-21T08:30:00.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:00.051+0000 7efe97bed700 1 -- 172.21.15.175:0/958335121 shutdown_connections 2024-04-21T08:30:00.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:00.051+0000 7efe97bed700 1 --2- 172.21.15.175:0/958335121 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7efe7c039650 0x7efe7c03bb10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:00.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:00.051+0000 7efe97bed700 1 --2- 172.21.15.175:0/958335121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe90108630 0x7efe90197f90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:00.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:00.051+0000 7efe97bed700 1 -- 172.21.15.175:0/958335121 >> 172.21.15.175:0/958335121 conn(0x7efe900fe050 msgr2=0x7efe900ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:00.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:00.052+0000 7efe97bed700 1 -- 172.21.15.175:0/958335121 shutdown_connections 2024-04-21T08:30:00.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:00.052+0000 7efe97bed700 1 -- 172.21.15.175:0/958335121 wait complete. 2024-04-21T08:30:00.052 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2024-04-21T08:30:00.262 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 podman[31889]: 2024-04-21 08:30:00.168740516 +0000 UTC m=+0.380608400 container init 44beb75e459f9b983e3d3775928b4d24608fb57db35119d9abdd709e72249d20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-21T08:30:00.262 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 podman[31889]: 2024-04-21 08:30:00.172087098 +0000 UTC m=+0.383954976 container start 44beb75e459f9b983e3d3775928b4d24608fb57db35119d9abdd709e72249d20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-21T08:30:00.263 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: set uid:gid to 167:167 (ceph:ceph) 2024-04-21T08:30:00.263 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev), process ceph-mon, pid 2 2024-04-21T08:30:00.263 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: pidfile_write: ignore empty --pid-file 2024-04-21T08:30:00.263 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: load: jerasure load: lrc 2024-04-21T08:30:00.263 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: RocksDB version: 7.9.2 2024-04-21T08:30:00.263 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Git sha 0 2024-04-21T08:30:00.263 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Compile date 2024-04-19 14:19:22 2024-04-21T08:30:00.263 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: DB SUMMARY 2024-04-21T08:30:00.263 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: DB Session ID: 4HQ5JXYF67KESJZQ5CLD 2024-04-21T08:30:00.263 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: CURRENT file: CURRENT 2024-04-21T08:30:00.263 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: IDENTITY file: IDENTITY 2024-04-21T08:30:00.263 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-21T08:30:00.263 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-21T08:30:00.264 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-21T08:30:00.264 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.error_if_exists: 0 2024-04-21T08:30:00.264 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.create_if_missing: 0 2024-04-21T08:30:00.264 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.paranoid_checks: 1 2024-04-21T08:30:00.264 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-21T08:30:00.264 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-21T08:30:00.264 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-21T08:30:00.264 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.env: 0x55f086179cc0 2024-04-21T08:30:00.264 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.fs: PosixFileSystem 2024-04-21T08:30:00.264 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.info_log: 0x55f086b69be0 2024-04-21T08:30:00.264 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_file_opening_threads: 16 2024-04-21T08:30:00.265 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.statistics: (nil) 2024-04-21T08:30:00.265 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.use_fsync: 0 2024-04-21T08:30:00.265 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_log_file_size: 0 2024-04-21T08:30:00.265 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-21T08:30:00.265 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-21T08:30:00.265 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.keep_log_file_num: 1000 2024-04-21T08:30:00.266 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.recycle_log_file_num: 0 2024-04-21T08:30:00.266 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.allow_fallocate: 1 2024-04-21T08:30:00.266 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.allow_mmap_reads: 0 2024-04-21T08:30:00.266 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.allow_mmap_writes: 0 2024-04-21T08:30:00.266 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.use_direct_reads: 0 2024-04-21T08:30:00.266 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-21T08:30:00.266 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.create_missing_column_families: 0 2024-04-21T08:30:00.266 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.db_log_dir: 2024-04-21T08:30:00.266 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.wal_dir: 2024-04-21T08:30:00.266 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-21T08:30:00.266 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-21T08:30:00.267 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-21T08:30:00.267 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-21T08:30:00.267 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-21T08:30:00.267 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-21T08:30:00.267 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.advise_random_on_open: 1 2024-04-21T08:30:00.267 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.db_write_buffer_size: 0 2024-04-21T08:30:00.267 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.write_buffer_manager: 0x55f086afd0e0 2024-04-21T08:30:00.267 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-21T08:30:00.267 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-21T08:30:00.267 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-21T08:30:00.267 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.rate_limiter: (nil) 2024-04-21T08:30:00.267 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-21T08:30:00.267 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.wal_recovery_mode: 2 2024-04-21T08:30:00.268 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.enable_thread_tracking: 0 2024-04-21T08:30:00.268 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.enable_pipelined_write: 0 2024-04-21T08:30:00.268 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.unordered_write: 0 2024-04-21T08:30:00.268 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-21T08:30:00.268 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-21T08:30:00.268 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-21T08:30:00.268 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-21T08:30:00.268 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.row_cache: None 2024-04-21T08:30:00.268 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.wal_filter: None 2024-04-21T08:30:00.268 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-21T08:30:00.268 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.allow_ingest_behind: 0 2024-04-21T08:30:00.268 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.two_write_queues: 0 2024-04-21T08:30:00.268 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.manual_wal_flush: 0 2024-04-21T08:30:00.268 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.wal_compression: 0 2024-04-21T08:30:00.268 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.atomic_flush: 0 2024-04-21T08:30:00.269 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-21T08:30:00.269 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-21T08:30:00.269 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-21T08:30:00.269 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.log_readahead_size: 0 2024-04-21T08:30:00.269 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-21T08:30:00.269 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.best_efforts_recovery: 0 2024-04-21T08:30:00.269 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-21T08:30:00.269 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-21T08:30:00.269 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.allow_data_in_errors: 0 2024-04-21T08:30:00.269 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.db_host_id: __hostname__ 2024-04-21T08:30:00.269 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-21T08:30:00.269 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_background_jobs: 2 2024-04-21T08:30:00.269 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_background_compactions: -1 2024-04-21T08:30:00.269 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_subcompactions: 1 2024-04-21T08:30:00.269 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-21T08:30:00.270 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-21T08:30:00.270 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-21T08:30:00.270 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_total_wal_size: 0 2024-04-21T08:30:00.270 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-21T08:30:00.270 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-21T08:30:00.270 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-21T08:30:00.270 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-21T08:30:00.270 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_open_files: -1 2024-04-21T08:30:00.270 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.bytes_per_sync: 0 2024-04-21T08:30:00.270 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-21T08:30:00.270 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-21T08:30:00.270 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.compaction_readahead_size: 0 2024-04-21T08:30:00.270 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_background_flushes: -1 2024-04-21T08:30:00.270 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Compression algorithms supported: 2024-04-21T08:30:00.271 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-21T08:30:00.271 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: kZSTD supported: 0 2024-04-21T08:30:00.271 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: kXpressCompression supported: 0 2024-04-21T08:30:00.271 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: kLZ4HCCompression supported: 1 2024-04-21T08:30:00.271 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: kZlibCompression supported: 1 2024-04-21T08:30:00.271 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: kSnappyCompression supported: 1 2024-04-21T08:30:00.271 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: kLZ4Compression supported: 1 2024-04-21T08:30:00.271 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: kBZip2Compression supported: 0 2024-04-21T08:30:00.271 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-21T08:30:00.271 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-21T08:30:00.271 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-21T08:30:00.271 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-21T08:30:00.271 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-21T08:30:00.271 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.merge_operator: 2024-04-21T08:30:00.272 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.compaction_filter: None 2024-04-21T08:30:00.272 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.compaction_filter_factory: None 2024-04-21T08:30:00.272 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.sst_partitioner_factory: None 2024-04-21T08:30:00.272 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-21T08:30:00.272 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-21T08:30:00.272 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f086b69dc0) 2024-04-21T08:30:00.272 INFO:journalctl@ceph.mon.c.smithi175.stdout: cache_index_and_filter_blocks: 1 2024-04-21T08:30:00.272 INFO:journalctl@ceph.mon.c.smithi175.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-21T08:30:00.272 INFO:journalctl@ceph.mon.c.smithi175.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-21T08:30:00.272 INFO:journalctl@ceph.mon.c.smithi175.stdout: pin_top_level_index_and_filter: 1 2024-04-21T08:30:00.272 INFO:journalctl@ceph.mon.c.smithi175.stdout: index_type: 0 2024-04-21T08:30:00.272 INFO:journalctl@ceph.mon.c.smithi175.stdout: data_block_index_type: 0 2024-04-21T08:30:00.272 INFO:journalctl@ceph.mon.c.smithi175.stdout: index_shortening: 1 2024-04-21T08:30:00.272 INFO:journalctl@ceph.mon.c.smithi175.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-21T08:30:00.272 INFO:journalctl@ceph.mon.c.smithi175.stdout: checksum: 4 2024-04-21T08:30:00.273 INFO:journalctl@ceph.mon.c.smithi175.stdout: no_block_cache: 0 2024-04-21T08:30:00.273 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_cache: 0x55f086b2f1f0 2024-04-21T08:30:00.273 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_cache_name: BinnedLRUCache 2024-04-21T08:30:00.273 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_cache_options: 2024-04-21T08:30:00.273 INFO:journalctl@ceph.mon.c.smithi175.stdout: capacity : 536870912 2024-04-21T08:30:00.273 INFO:journalctl@ceph.mon.c.smithi175.stdout: num_shard_bits : 4 2024-04-21T08:30:00.273 INFO:journalctl@ceph.mon.c.smithi175.stdout: strict_capacity_limit : 0 2024-04-21T08:30:00.273 INFO:journalctl@ceph.mon.c.smithi175.stdout: high_pri_pool_ratio: 0.000 2024-04-21T08:30:00.273 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_cache_compressed: (nil) 2024-04-21T08:30:00.273 INFO:journalctl@ceph.mon.c.smithi175.stdout: persistent_cache: (nil) 2024-04-21T08:30:00.273 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_size: 4096 2024-04-21T08:30:00.273 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_size_deviation: 10 2024-04-21T08:30:00.273 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_restart_interval: 16 2024-04-21T08:30:00.273 INFO:journalctl@ceph.mon.c.smithi175.stdout: index_block_restart_interval: 1 2024-04-21T08:30:00.274 INFO:journalctl@ceph.mon.c.smithi175.stdout: metadata_block_size: 4096 2024-04-21T08:30:00.274 INFO:journalctl@ceph.mon.c.smithi175.stdout: partition_filters: 0 2024-04-21T08:30:00.274 INFO:journalctl@ceph.mon.c.smithi175.stdout: use_delta_encoding: 1 2024-04-21T08:30:00.274 INFO:journalctl@ceph.mon.c.smithi175.stdout: filter_policy: bloomfilter 2024-04-21T08:30:00.274 INFO:journalctl@ceph.mon.c.smithi175.stdout: whole_key_filtering: 1 2024-04-21T08:30:00.274 INFO:journalctl@ceph.mon.c.smithi175.stdout: verify_compression: 0 2024-04-21T08:30:00.274 INFO:journalctl@ceph.mon.c.smithi175.stdout: read_amp_bytes_per_bit: 0 2024-04-21T08:30:00.274 INFO:journalctl@ceph.mon.c.smithi175.stdout: format_version: 5 2024-04-21T08:30:00.274 INFO:journalctl@ceph.mon.c.smithi175.stdout: enable_index_compression: 1 2024-04-21T08:30:00.274 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_align: 0 2024-04-21T08:30:00.274 INFO:journalctl@ceph.mon.c.smithi175.stdout: max_auto_readahead_size: 262144 2024-04-21T08:30:00.274 INFO:journalctl@ceph.mon.c.smithi175.stdout: prepopulate_block_cache: 0 2024-04-21T08:30:00.274 INFO:journalctl@ceph.mon.c.smithi175.stdout: initial_auto_readahead_size: 8192 2024-04-21T08:30:00.274 INFO:journalctl@ceph.mon.c.smithi175.stdout: num_file_reads_for_auto_readahead: 2 2024-04-21T08:30:00.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.write_buffer_size: 33554432 2024-04-21T08:30:00.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_write_buffer_number: 2 2024-04-21T08:30:00.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.compression: NoCompression 2024-04-21T08:30:00.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.bottommost_compression: Disabled 2024-04-21T08:30:00.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.prefix_extractor: nullptr 2024-04-21T08:30:00.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-21T08:30:00.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.num_levels: 7 2024-04-21T08:30:00.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-21T08:30:00.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-21T08:30:00.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-21T08:30:00.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-21T08:30:00.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-21T08:30:00.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-21T08:30:00.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-21T08:30:00.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-21T08:30:00.276 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-21T08:30:00.276 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-21T08:30:00.276 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-21T08:30:00.276 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-21T08:30:00.276 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-21T08:30:00.276 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.compression_opts.level: 32767 2024-04-21T08:30:00.276 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.compression_opts.strategy: 0 2024-04-21T08:30:00.276 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-21T08:30:00.276 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-21T08:30:00.276 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-21T08:30:00.276 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-21T08:30:00.276 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.compression_opts.enabled: false 2024-04-21T08:30:00.276 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-21T08:30:00.276 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-21T08:30:00.276 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-21T08:30:00.277 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-21T08:30:00.277 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.target_file_size_base: 67108864 2024-04-21T08:30:00.277 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-21T08:30:00.277 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-21T08:30:00.277 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-21T08:30:00.277 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-21T08:30:00.277 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-21T08:30:00.277 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-21T08:30:00.278 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-21T08:30:00.278 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-21T08:30:00.278 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-21T08:30:00.279 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-21T08:30:00.279 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-21T08:30:00.279 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-21T08:30:00.279 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-21T08:30:00.279 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-21T08:30:00.279 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.arena_block_size: 1048576 2024-04-21T08:30:00.279 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-21T08:30:00.279 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-21T08:30:00.279 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.disable_auto_compactions: 0 2024-04-21T08:30:00.279 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-21T08:30:00.279 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-21T08:30:00.279 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-21T08:30:00.279 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-21T08:30:00.279 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-21T08:30:00.280 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-21T08:30:00.280 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-21T08:30:00.280 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-21T08:30:00.280 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-21T08:30:00.280 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-21T08:30:00.280 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-21T08:30:00.280 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.inplace_update_support: 0 2024-04-21T08:30:00.280 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-21T08:30:00.280 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-21T08:30:00.280 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-21T08:30:00.280 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-21T08:30:00.280 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.bloom_locality: 0 2024-04-21T08:30:00.280 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.max_successive_merges: 0 2024-04-21T08:30:00.280 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-21T08:30:00.281 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.paranoid_file_checks: 0 2024-04-21T08:30:00.281 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.force_consistency_checks: 1 2024-04-21T08:30:00.281 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.report_bg_io_stats: 0 2024-04-21T08:30:00.281 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.ttl: 2592000 2024-04-21T08:30:00.281 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-21T08:30:00.281 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-21T08:30:00.281 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-21T08:30:00.281 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.enable_blob_files: false 2024-04-21T08:30:00.281 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.min_blob_size: 0 2024-04-21T08:30:00.281 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.blob_file_size: 268435456 2024-04-21T08:30:00.281 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-21T08:30:00.281 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-21T08:30:00.281 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-21T08:30:00.281 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-21T08:30:00.281 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-21T08:30:00.282 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.blob_file_starting_level: 0 2024-04-21T08:30:00.282 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-21T08:30:00.282 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-21T08:30:00.282 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-21T08:30:00.282 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: da7c369b-7c49-42cc-ab19-fdbeee66db2d 2024-04-21T08:30:00.282 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713688200202289, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-21T08:30:00.282 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-21T08:30:00.282 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713688200202721, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713688200, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "da7c369b-7c49-42cc-ab19-fdbeee66db2d", "db_session_id": "4HQ5JXYF67KESJZQ5CLD", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-21T08:30:00.282 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713688200202806, "job": 1, "event": "recovery_finished"} 2024-04-21T08:30:00.282 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-21T08:30:00.282 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-21T08:30:00.282 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f086c44000 2024-04-21T08:30:00.282 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: DB pointer 0x55f086c2e000 2024-04-21T08:30:00.282 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-21T08:30:00.283 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-21T08:30:00.283 INFO:journalctl@ceph.mon.c.smithi175.stdout: ** DB Stats ** 2024-04-21T08:30:00.283 INFO:journalctl@ceph.mon.c.smithi175.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-21T08:30:00.283 INFO:journalctl@ceph.mon.c.smithi175.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-21T08:30:00.283 INFO:journalctl@ceph.mon.c.smithi175.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-21T08:30:00.283 INFO:journalctl@ceph.mon.c.smithi175.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-21T08:30:00.283 INFO:journalctl@ceph.mon.c.smithi175.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-21T08:30:00.283 INFO:journalctl@ceph.mon.c.smithi175.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-21T08:30:00.283 INFO:journalctl@ceph.mon.c.smithi175.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-21T08:30:00.283 INFO:journalctl@ceph.mon.c.smithi175.stdout: 2024-04-21T08:30:00.283 INFO:journalctl@ceph.mon.c.smithi175.stdout: ** Compaction Stats [default] ** 2024-04-21T08:30:00.283 INFO:journalctl@ceph.mon.c.smithi175.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-21T08:30:00.283 INFO:journalctl@ceph.mon.c.smithi175.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-21T08:30:00.283 INFO:journalctl@ceph.mon.c.smithi175.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-21T08:30:00.283 INFO:journalctl@ceph.mon.c.smithi175.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-21T08:30:00.284 INFO:journalctl@ceph.mon.c.smithi175.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-21T08:30:00.284 INFO:journalctl@ceph.mon.c.smithi175.stdout: 2024-04-21T08:30:00.284 INFO:journalctl@ceph.mon.c.smithi175.stdout: ** Compaction Stats [default] ** 2024-04-21T08:30:00.284 INFO:journalctl@ceph.mon.c.smithi175.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-21T08:30:00.284 INFO:journalctl@ceph.mon.c.smithi175.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-21T08:30:00.284 INFO:journalctl@ceph.mon.c.smithi175.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-21T08:30:00.284 INFO:journalctl@ceph.mon.c.smithi175.stdout: 2024-04-21T08:30:00.284 INFO:journalctl@ceph.mon.c.smithi175.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-21T08:30:00.284 INFO:journalctl@ceph.mon.c.smithi175.stdout: 2024-04-21T08:30:00.284 INFO:journalctl@ceph.mon.c.smithi175.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-21T08:30:00.284 INFO:journalctl@ceph.mon.c.smithi175.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-21T08:30:00.284 INFO:journalctl@ceph.mon.c.smithi175.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-21T08:30:00.284 INFO:journalctl@ceph.mon.c.smithi175.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-21T08:30:00.284 INFO:journalctl@ceph.mon.c.smithi175.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-21T08:30:00.284 INFO:journalctl@ceph.mon.c.smithi175.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-21T08:30:00.285 INFO:journalctl@ceph.mon.c.smithi175.stdout: Cumulative compaction: 0.00 GB write, 0.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-21T08:30:00.285 INFO:journalctl@ceph.mon.c.smithi175.stdout: Interval compaction: 0.00 GB write, 0.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-21T08:30:00.285 INFO:journalctl@ceph.mon.c.smithi175.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-21T08:30:00.285 INFO:journalctl@ceph.mon.c.smithi175.stdout: Block cache BinnedLRUCache@0x55f086b2f1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-21T08:30:00.285 INFO:journalctl@ceph.mon.c.smithi175.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-21T08:30:00.285 INFO:journalctl@ceph.mon.c.smithi175.stdout: 2024-04-21T08:30:00.285 INFO:journalctl@ceph.mon.c.smithi175.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-21T08:30:00.285 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-21T08:30:00.285 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: using public_addr v2:172.21.15.175:0/0 -> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2024-04-21T08:30:00.285 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: starting mon.c rank -1 at public addrs [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] at bind addrs [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:30:00.285 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: mon.c@-1(???) e0 preinit fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:30:00.285 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: mon.c@-1(synchronizing).mds e1 new map 2024-04-21T08:30:00.285 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-21T08:30:00.285 INFO:journalctl@ceph.mon.c.smithi175.stdout: e1 2024-04-21T08:30:00.285 INFO:journalctl@ceph.mon.c.smithi175.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-21T08:30:00.286 INFO:journalctl@ceph.mon.c.smithi175.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-21T08:30:00.286 INFO:journalctl@ceph.mon.c.smithi175.stdout: legacy client fscid: -1 2024-04-21T08:30:00.286 INFO:journalctl@ceph.mon.c.smithi175.stdout: 2024-04-21T08:30:00.286 INFO:journalctl@ceph.mon.c.smithi175.stdout: No filesystems configured 2024-04-21T08:30:00.286 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-21T08:30:00.286 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-21T08:30:00.286 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-21T08:30:00.286 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-21T08:30:00.286 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-21T08:30:00.286 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-21T08:30:00.286 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-21T08:30:00.286 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T08:30:00.286 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T08:30:00.286 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T08:30:00.286 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:00.287 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:30:00.287 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:00.287 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:00.287 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi175", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:30:00.287 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: Deploying cephadm binary to smithi175 2024-04-21T08:30:00.287 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:00.287 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:00.287 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:00.287 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:00.287 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:30:00.287 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:00.287 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:00.287 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-21T08:30:00.287 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: Updating smithi112:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:00.288 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:00.288 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:00.288 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:30:00.288 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: Updating smithi112:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:30:00.288 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:00.288 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:00.288 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:00.288 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:00.288 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:00.288 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: Added host smithi175 2024-04-21T08:30:00.288 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:00.288 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:00.288 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:00.288 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:00.288 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:30:00.289 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:00.289 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2267614155' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-21T08:30:00.289 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:00.289 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2267614155' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-21T08:30:00.289 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:30:00.289 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:00.289 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:00.289 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:00.289 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:00.289 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi175", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:30:00.289 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:00.289 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:00.289 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: Updating smithi175:/etc/ceph/ceph.conf 2024-04-21T08:30:00.289 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: Updating smithi175:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:00.289 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:30:00.290 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: Updating smithi175:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:30:00.290 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:00.290 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:00.290 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:00.290 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:00.290 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi112:172.21.15.112=b;smithi175:172.21.15.175=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:30:00.290 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: Saving service mon spec with placement smithi043:172.21.15.43=a;smithi112:172.21.15.112=b;smithi175:172.21.15.175=c;count:3 2024-04-21T08:30:00.290 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:00.290 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:00.290 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:00.290 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:00.290 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:00.290 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:00.290 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:00.290 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: Deploying daemon mon.c on smithi175 2024-04-21T08:30:00.291 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:00.291 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: from='client.? 172.21.15.175:0/640509845' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T08:30:00.291 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:00.291 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:00.291 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 ceph-mon[31920]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-21T08:30:00.291 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 bash[31889]: 44beb75e459f9b983e3d3775928b4d24608fb57db35119d9abdd709e72249d20 2024-04-21T08:30:00.291 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:00 smithi175 systemd[1]: Started Ceph mon.c for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:30:01.882 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-21T08:30:01.883 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-21T08:30:02.132 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:30:05.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:05.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:30:05.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: mon.a calling monitor election 2024-04-21T08:30:05.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:05.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:05.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:05.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:05.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: mon.c calling monitor election 2024-04-21T08:30:05.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:05.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:05.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-21T08:30:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: monmap epoch 2 2024-04-21T08:30:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:30:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: last_changed 2024-04-21T08:30:00.226102+0000 2024-04-21T08:30:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: created 2024-04-21T08:28:08.995135+0000 2024-04-21T08:30:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: min_mon_release 19 (squid) 2024-04-21T08:30:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: election_strategy: 1 2024-04-21T08:30:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-04-21T08:30:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-04-21T08:30:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: fsmap 2024-04-21T08:30:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:30:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: mgrmap e13: x(active, since 51s) 2024-04-21T08:30:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: overall HEALTH_OK 2024-04-21T08:30:05.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:05.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:05.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:05.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:05.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:05 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:05.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:05.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:30:05.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: mon.a calling monitor election 2024-04-21T08:30:05.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:05.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:05.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:05.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:05.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: mon.c calling monitor election 2024-04-21T08:30:05.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:05.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:05.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:05.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:05.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-21T08:30:05.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: monmap epoch 2 2024-04-21T08:30:05.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:30:05.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: last_changed 2024-04-21T08:30:00.226102+0000 2024-04-21T08:30:05.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: created 2024-04-21T08:28:08.995135+0000 2024-04-21T08:30:05.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: min_mon_release 19 (squid) 2024-04-21T08:30:05.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: election_strategy: 1 2024-04-21T08:30:05.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-04-21T08:30:05.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-04-21T08:30:05.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: fsmap 2024-04-21T08:30:05.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:30:05.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: mgrmap e13: x(active, since 51s) 2024-04-21T08:30:05.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: overall HEALTH_OK 2024-04-21T08:30:05.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:05.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:05.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:05.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:05.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:05 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:06.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:06 smithi043 ceph-mon[27978]: Deploying daemon mon.b on smithi112 2024-04-21T08:30:06.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:06 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:06.631 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.631+0000 7fb91a15d700 1 -- 172.21.15.175:0/593777367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9000035c0 msgr2=0x7fb900005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:06.631 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.631+0000 7fb91a15d700 1 --2- 172.21.15.175:0/593777367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9000035c0 0x7fb900005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb914101380 tx=0x7fb90400feb0 comp rx=0 tx=0).stop 2024-04-21T08:30:06.631 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:06 smithi175 ceph-mon[31920]: Deploying daemon mon.b on smithi112 2024-04-21T08:30:06.631 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:06 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:06.632 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.632+0000 7fb91a15d700 1 -- 172.21.15.175:0/593777367 shutdown_connections 2024-04-21T08:30:06.632 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.632+0000 7fb91a15d700 1 --2- 172.21.15.175:0/593777367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9000035c0 0x7fb900005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:06.632 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.632+0000 7fb91a15d700 1 -- 172.21.15.175:0/593777367 >> 172.21.15.175:0/593777367 conn(0x7fb9140fcf80 msgr2=0x7fb9140ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:06.632 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.632+0000 7fb91a15d700 1 -- 172.21.15.175:0/593777367 shutdown_connections 2024-04-21T08:30:06.632 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.632+0000 7fb91a15d700 1 -- 172.21.15.175:0/593777367 wait complete. 2024-04-21T08:30:06.632 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.633+0000 7fb91a15d700 1 Processor -- start 2024-04-21T08:30:06.632 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.633+0000 7fb91a15d700 1 -- start start 2024-04-21T08:30:06.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.633+0000 7fb91a15d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9000035c0 0x7fb914195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:06.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.633+0000 7fb91a15d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb914196340 0x7fb91419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:06.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.633+0000 7fb91a15d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb9141082f0 con 0x7fb914196340 2024-04-21T08:30:06.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.633+0000 7fb91a15d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb91406bb80 con 0x7fb9000035c0 2024-04-21T08:30:06.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.633+0000 7fb9137fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9000035c0 0x7fb914195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:06.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.633+0000 7fb9137fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9000035c0 0x7fb914195e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:59830/0 (socket says 172.21.15.175:59830) 2024-04-21T08:30:06.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.633+0000 7fb90bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb914196340 0x7fb91419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:06.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.633+0000 7fb9137fe700 1 -- 172.21.15.175:0/2701230159 learned_addr learned my addr 172.21.15.175:0/2701230159 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-21T08:30:06.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.633+0000 7fb9137fe700 1 -- 172.21.15.175:0/2701230159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9000035c0 msgr2=0x7fb914195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-21T08:30:06.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.633+0000 7fb9137fe700 1 -- 172.21.15.175:0/2701230159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9000035c0 msgr2=0x7fb914195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-21T08:30:06.635 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.633+0000 7fb9137fe700 1 --2- 172.21.15.175:0/2701230159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9000035c0 0x7fb914195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-21T08:30:06.635 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.633+0000 7fb9137fe700 1 --2- 172.21.15.175:0/2701230159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9000035c0 0x7fb914195e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:30:06.635 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.634+0000 7fb90bfff700 1 -- 172.21.15.175:0/2701230159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9000035c0 msgr2=0x7fb914195e00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:30:06.635 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.634+0000 7fb90bfff700 1 --2- 172.21.15.175:0/2701230159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9000035c0 0x7fb914195e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:06.635 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.634+0000 7fb90bfff700 1 -- 172.21.15.175:0/2701230159 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb914196980 con 0x7fb914196340 2024-04-21T08:30:06.635 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.634+0000 7fb90bfff700 1 --2- 172.21.15.175:0/2701230159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb914196340 0x7fb91419a680 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00b4d0 tx=0x7fb8fc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:06.635 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.634+0000 7fb9117fa700 1 -- 172.21.15.175:0/2701230159 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb8fc011630 con 0x7fb914196340 2024-04-21T08:30:06.635 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.634+0000 7fb9117fa700 1 -- 172.21.15.175:0/2701230159 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8fc011c80 con 0x7fb914196340 2024-04-21T08:30:06.635 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.634+0000 7fb91a15d700 1 -- 172.21.15.175:0/2701230159 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb91419abc0 con 0x7fb914196340 2024-04-21T08:30:06.635 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.634+0000 7fb9117fa700 1 -- 172.21.15.175:0/2701230159 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb8fc010da0 con 0x7fb914196340 2024-04-21T08:30:06.635 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.634+0000 7fb91a15d700 1 -- 172.21.15.175:0/2701230159 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb91419b120 con 0x7fb914196340 2024-04-21T08:30:06.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.635+0000 7fb9117fa700 1 -- 172.21.15.175:0/2701230159 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb8fc018020 con 0x7fb914196340 2024-04-21T08:30:06.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.635+0000 7fb91a15d700 1 -- 172.21.15.175:0/2701230159 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb914102b90 con 0x7fb914196340 2024-04-21T08:30:06.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.635+0000 7fb9117fa700 1 --2- 172.21.15.175:0/2701230159 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8f403b650 0x7fb8f403db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:06.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.635+0000 7fb9117fa700 1 -- 172.21.15.175:0/2701230159 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb8fc04c1f0 con 0x7fb914196340 2024-04-21T08:30:06.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.636+0000 7fb9137fe700 1 --2- 172.21.15.175:0/2701230159 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8f403b650 0x7fb8f403db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:06.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.636+0000 7fb9137fe700 1 --2- 172.21.15.175:0/2701230159 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8f403b650 0x7fb8f403db10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb90400fa60 tx=0x7fb904005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:06.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.641+0000 7fb9117fa700 1 -- 172.21.15.175:0/2701230159 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb8fc01f020 con 0x7fb914196340 2024-04-21T08:30:06.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.922+0000 7fb91a15d700 1 -- 172.21.15.175:0/2701230159 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb9141971a0 con 0x7fb914196340 2024-04-21T08:30:06.922 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.923+0000 7fb9117fa700 1 -- 172.21.15.175:0/2701230159 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fb9141971a0 con 0x7fb914196340 2024-04-21T08:30:06.922 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-21T08:30:06.922 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":2,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","modified":"2024-04-21T08:30:00.226102Z","created":"2024-04-21T08:28:08.995135Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:3300","nonce":0},{"type":"v1","addr":"172.21.15.175:6789","nonce":0}]},"addr":"172.21.15.175:6789/0","public_addr":"172.21.15.175:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-21T08:30:06.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.925+0000 7fb91a15d700 1 -- 172.21.15.175:0/2701230159 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8f403b650 msgr2=0x7fb8f403db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:06.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.925+0000 7fb91a15d700 1 --2- 172.21.15.175:0/2701230159 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8f403b650 0x7fb8f403db10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb90400fa60 tx=0x7fb904005a90 comp rx=0 tx=0).stop 2024-04-21T08:30:06.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.925+0000 7fb91a15d700 1 -- 172.21.15.175:0/2701230159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb914196340 msgr2=0x7fb91419a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:06.924 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.925+0000 7fb91a15d700 1 --2- 172.21.15.175:0/2701230159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb914196340 0x7fb91419a680 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00b4d0 tx=0x7fb8fc00b890 comp rx=0 tx=0).stop 2024-04-21T08:30:06.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.925+0000 7fb91a15d700 1 -- 172.21.15.175:0/2701230159 shutdown_connections 2024-04-21T08:30:06.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.925+0000 7fb91a15d700 1 --2- 172.21.15.175:0/2701230159 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8f403b650 0x7fb8f403db10 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:06.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.925+0000 7fb91a15d700 1 --2- 172.21.15.175:0/2701230159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9000035c0 0x7fb914195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:06.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.925+0000 7fb91a15d700 1 --2- 172.21.15.175:0/2701230159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb914196340 0x7fb91419a680 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:06.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.925+0000 7fb91a15d700 1 -- 172.21.15.175:0/2701230159 >> 172.21.15.175:0/2701230159 conn(0x7fb9140fcf80 msgr2=0x7fb9140feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:06.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.925+0000 7fb91a15d700 1 -- 172.21.15.175:0/2701230159 shutdown_connections 2024-04-21T08:30:06.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:06.925+0000 7fb91a15d700 1 -- 172.21.15.175:0/2701230159 wait complete. 2024-04-21T08:30:06.925 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 2 2024-04-21T08:30:07.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:07 smithi043 ceph-mon[27978]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:07.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:07 smithi043 ceph-mon[27978]: from='client.? 172.21.15.175:0/2701230159' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T08:30:07.583 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:07 smithi175 ceph-mon[31920]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:07.584 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:07 smithi175 ceph-mon[31920]: from='client.? 172.21.15.175:0/2701230159' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T08:30:08.615 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-21T08:30:08.616 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-21T08:30:08.860 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:30:09.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:09 smithi112 systemd[1]: Started Ceph mon.b for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:30:14.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:30:14.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: mon.a calling monitor election 2024-04-21T08:30:14.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: mon.c calling monitor election 2024-04-21T08:30:14.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:14.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:14.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:14.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:14.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:14.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: mon.b calling monitor election 2024-04-21T08:30:14.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:14.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:14.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:14.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:14.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-21T08:30:14.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: monmap epoch 3 2024-04-21T08:30:14.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:30:14.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: last_changed 2024-04-21T08:30:09.128148+0000 2024-04-21T08:30:14.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: created 2024-04-21T08:28:08.995135+0000 2024-04-21T08:30:14.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: min_mon_release 19 (squid) 2024-04-21T08:30:14.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: election_strategy: 1 2024-04-21T08:30:14.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-04-21T08:30:14.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-04-21T08:30:14.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: 2: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.b 2024-04-21T08:30:14.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: fsmap 2024-04-21T08:30:14.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:30:14.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: mgrmap e13: x(active, since 60s) 2024-04-21T08:30:14.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: overall HEALTH_OK 2024-04-21T08:30:14.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:14.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:14.467 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:14.467 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:30:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: mon.a calling monitor election 2024-04-21T08:30:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: mon.c calling monitor election 2024-04-21T08:30:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:14.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: mon.b calling monitor election 2024-04-21T08:30:14.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:14.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:14.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:14.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:14.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-21T08:30:14.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: monmap epoch 3 2024-04-21T08:30:14.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:30:14.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: last_changed 2024-04-21T08:30:09.128148+0000 2024-04-21T08:30:14.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: created 2024-04-21T08:28:08.995135+0000 2024-04-21T08:30:14.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: min_mon_release 19 (squid) 2024-04-21T08:30:14.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: election_strategy: 1 2024-04-21T08:30:14.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-04-21T08:30:14.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-04-21T08:30:14.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: 2: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.b 2024-04-21T08:30:14.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: fsmap 2024-04-21T08:30:14.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:30:14.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: mgrmap e13: x(active, since 60s) 2024-04-21T08:30:14.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: overall HEALTH_OK 2024-04-21T08:30:14.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:14.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:14.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:14.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:14.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.552+0000 7f4231eb4700 1 -- 172.21.15.175:0/2034198270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42180035c0 msgr2=0x7f4218005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:14.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.552+0000 7f4231eb4700 1 --2- 172.21.15.175:0/2034198270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42180035c0 0x7f4218005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f422c102480 tx=0x7f421c00fe60 comp rx=0 tx=0).stop 2024-04-21T08:30:14.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.552+0000 7f4231eb4700 1 -- 172.21.15.175:0/2034198270 shutdown_connections 2024-04-21T08:30:14.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.552+0000 7f4231eb4700 1 --2- 172.21.15.175:0/2034198270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42180035c0 0x7f4218005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:14.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.552+0000 7f4231eb4700 1 -- 172.21.15.175:0/2034198270 >> 172.21.15.175:0/2034198270 conn(0x7f422c0fe040 msgr2=0x7f422c100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:14.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.553+0000 7f4231eb4700 1 -- 172.21.15.175:0/2034198270 shutdown_connections 2024-04-21T08:30:14.555 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.553+0000 7f4231eb4700 1 -- 172.21.15.175:0/2034198270 wait complete. 2024-04-21T08:30:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.553+0000 7f4231eb4700 1 Processor -- start 2024-04-21T08:30:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.553+0000 7f4231eb4700 1 -- start start 2024-04-21T08:30:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.553+0000 7f4231eb4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42180035c0 0x7f422c1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.553+0000 7f4231eb4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f422c198730 0x7f422c19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.553+0000 7f4231eb4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c19d000 0x7f422c19d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.553+0000 7f4231eb4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f422c10a450 con 0x7f422c19d000 2024-04-21T08:30:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.553+0000 7f4231eb4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f422c10a2d0 con 0x7f42180035c0 2024-04-21T08:30:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.553+0000 7f4231eb4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f422c10a5d0 con 0x7f422c198730 2024-04-21T08:30:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.554+0000 7f4223fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f422c198730 0x7f422c19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.554+0000 7f422bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c19d000 0x7f422c19d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.554+0000 7f4223fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f422c198730 0x7f422c19cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:42134/0 (socket says 172.21.15.175:42134) 2024-04-21T08:30:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.554+0000 7f422bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c19d000 0x7f422c19d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.175:60542/0 (socket says 172.21.15.175:60542) 2024-04-21T08:30:14.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.554+0000 7f4223fff700 1 -- 172.21.15.175:0/1981465855 learned_addr learned my addr 172.21.15.175:0/1981465855 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-21T08:30:14.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.554+0000 7f422b7fe700 1 --2- 172.21.15.175:0/1981465855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42180035c0 0x7f422c1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:14.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.554+0000 7f4223fff700 1 -- 172.21.15.175:0/1981465855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f422c198730 msgr2=0x7f422c19cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-21T08:30:14.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.554+0000 7f4223fff700 1 -- 172.21.15.175:0/1981465855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f422c198730 msgr2=0x7f422c19cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-21T08:30:14.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.554+0000 7f4223fff700 1 --2- 172.21.15.175:0/1981465855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f422c198730 0x7f422c19cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-21T08:30:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.554+0000 7f4223fff700 1 --2- 172.21.15.175:0/1981465855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f422c198730 0x7f422c19cac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:30:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.554+0000 7f422b7fe700 1 -- 172.21.15.175:0/1981465855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42180035c0 msgr2=0x7f422c1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-21T08:30:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.554+0000 7f422b7fe700 1 -- 172.21.15.175:0/1981465855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42180035c0 msgr2=0x7f422c1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-21T08:30:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.554+0000 7f422b7fe700 1 --2- 172.21.15.175:0/1981465855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42180035c0 0x7f422c1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-21T08:30:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.554+0000 7f422b7fe700 1 --2- 172.21.15.175:0/1981465855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42180035c0 0x7f422c1981f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:30:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.554+0000 7f422bfff700 1 -- 172.21.15.175:0/1981465855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f422c198730 msgr2=0x7f422c19cac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:30:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.554+0000 7f422bfff700 1 --2- 172.21.15.175:0/1981465855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f422c198730 0x7f422c19cac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.554+0000 7f422bfff700 1 -- 172.21.15.175:0/1981465855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42180035c0 msgr2=0x7f422c1981f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:30:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.554+0000 7f422bfff700 1 --2- 172.21.15.175:0/1981465855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42180035c0 0x7f422c1981f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.554+0000 7f422bfff700 1 -- 172.21.15.175:0/1981465855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422c1a4d70 con 0x7f422c19d000 2024-04-21T08:30:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.554+0000 7f422bfff700 1 --2- 172.21.15.175:0/1981465855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c19d000 0x7f422c19d4b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f422400bd50 tx=0x7f422400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:14.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.554+0000 7f42297fa700 1 -- 172.21.15.175:0/1981465855 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224019070 con 0x7f422c19d000 2024-04-21T08:30:14.559 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.554+0000 7f42297fa700 1 -- 172.21.15.175:0/1981465855 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f422400cd10 con 0x7f422c19d000 2024-04-21T08:30:14.559 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.555+0000 7f4231eb4700 1 -- 172.21.15.175:0/1981465855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f422c1a4fa0 con 0x7f422c19d000 2024-04-21T08:30:14.559 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.555+0000 7f42297fa700 1 -- 172.21.15.175:0/1981465855 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224012700 con 0x7f422c19d000 2024-04-21T08:30:14.559 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.555+0000 7f4231eb4700 1 -- 172.21.15.175:0/1981465855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f422c1a5480 con 0x7f422c19d000 2024-04-21T08:30:14.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.555+0000 7f42297fa700 1 -- 172.21.15.175:0/1981465855 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f422401c070 con 0x7f422c19d000 2024-04-21T08:30:14.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.556+0000 7f42297fa700 1 --2- 172.21.15.175:0/1981465855 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f420c039ae0 0x7f420c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:14.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.556+0000 7f42297fa700 1 -- 172.21.15.175:0/1981465855 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f422404d160 con 0x7f422c19d000 2024-04-21T08:30:14.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.556+0000 7f4231eb4700 1 -- 172.21.15.175:0/1981465855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f422c0640a0 con 0x7f422c19d000 2024-04-21T08:30:14.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.556+0000 7f422b7fe700 1 --2- 172.21.15.175:0/1981465855 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f420c039ae0 0x7f420c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:14.560 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.556+0000 7f422b7fe700 1 --2- 172.21.15.175:0/1981465855 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f420c039ae0 0x7f420c03bfa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f421c00fa10 tx=0x7f421c005de0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:14.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:30:14.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: mon.a calling monitor election 2024-04-21T08:30:14.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: mon.c calling monitor election 2024-04-21T08:30:14.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:14.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:14.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:14.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:14.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:14.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: mon.b calling monitor election 2024-04-21T08:30:14.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:14.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:14.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:14.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:14.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-21T08:30:14.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: monmap epoch 3 2024-04-21T08:30:14.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:30:14.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: last_changed 2024-04-21T08:30:09.128148+0000 2024-04-21T08:30:14.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: created 2024-04-21T08:28:08.995135+0000 2024-04-21T08:30:14.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: min_mon_release 19 (squid) 2024-04-21T08:30:14.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: election_strategy: 1 2024-04-21T08:30:14.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-04-21T08:30:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-04-21T08:30:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: 2: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.b 2024-04-21T08:30:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: fsmap 2024-04-21T08:30:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:30:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: mgrmap e13: x(active, since 60s) 2024-04-21T08:30:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: overall HEALTH_OK 2024-04-21T08:30:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:14.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.561+0000 7f42297fa700 1 -- 172.21.15.175:0/1981465855 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f422400e0a0 con 0x7f422c19d000 2024-04-21T08:30:14.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.915+0000 7f4231eb4700 1 -- 172.21.15.175:0/1981465855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f422c02df80 con 0x7f422c19d000 2024-04-21T08:30:14.916 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.916+0000 7f42297fa700 1 -- 172.21.15.175:0/1981465855 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1285 (secure 0 0 0) 0x7f4224007290 con 0x7f422c19d000 2024-04-21T08:30:14.917 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-21T08:30:14.917 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":3,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","modified":"2024-04-21T08:30:09.128148Z","created":"2024-04-21T08:28:08.995135Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:3300","nonce":0},{"type":"v1","addr":"172.21.15.175:6789","nonce":0}]},"addr":"172.21.15.175:6789/0","public_addr":"172.21.15.175:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-21T08:30:14.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.920+0000 7f4231eb4700 1 -- 172.21.15.175:0/1981465855 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f420c039ae0 msgr2=0x7f420c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:14.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.920+0000 7f4231eb4700 1 --2- 172.21.15.175:0/1981465855 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f420c039ae0 0x7f420c03bfa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f421c00fa10 tx=0x7f421c005de0 comp rx=0 tx=0).stop 2024-04-21T08:30:14.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.920+0000 7f4231eb4700 1 -- 172.21.15.175:0/1981465855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c19d000 msgr2=0x7f422c19d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:14.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.920+0000 7f4231eb4700 1 --2- 172.21.15.175:0/1981465855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c19d000 0x7f422c19d4b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f422400bd50 tx=0x7f422400be30 comp rx=0 tx=0).stop 2024-04-21T08:30:14.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.920+0000 7f4231eb4700 1 -- 172.21.15.175:0/1981465855 shutdown_connections 2024-04-21T08:30:14.920 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.920+0000 7f4231eb4700 1 --2- 172.21.15.175:0/1981465855 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f420c039ae0 0x7f420c03bfa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:14.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.920+0000 7f4231eb4700 1 --2- 172.21.15.175:0/1981465855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c19d000 0x7f422c19d4b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:14.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.920+0000 7f4231eb4700 1 --2- 172.21.15.175:0/1981465855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42180035c0 0x7f422c1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:14.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.920+0000 7f4231eb4700 1 --2- 172.21.15.175:0/1981465855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f422c198730 0x7f422c19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:14.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.920+0000 7f4231eb4700 1 -- 172.21.15.175:0/1981465855 >> 172.21.15.175:0/1981465855 conn(0x7f422c0fe040 msgr2=0x7f422c1071b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:14.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.920+0000 7f4231eb4700 1 -- 172.21.15.175:0/1981465855 shutdown_connections 2024-04-21T08:30:14.921 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:14.920+0000 7f4231eb4700 1 -- 172.21.15.175:0/1981465855 wait complete. 2024-04-21T08:30:14.922 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 3 2024-04-21T08:30:15.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:15 smithi175 ceph-mon[31920]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:15.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:15.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:15 smithi175 ceph-mon[31920]: from='client.? 172.21.15.175:0/1981465855' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T08:30:15.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:15.512 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-21T08:30:15.513 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-21T08:30:15.541 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:15 smithi043 ceph-mon[27978]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:15.541 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:15.541 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:15 smithi043 ceph-mon[27978]: from='client.? 172.21.15.175:0/1981465855' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T08:30:15.542 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:15.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:15 smithi112 ceph-mon[31085]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:15.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:15.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:15 smithi112 ceph-mon[31085]: from='client.? 172.21.15.175:0/1981465855' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T08:30:15.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:15.772 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:30:16.885 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:16 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:16.885 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:16 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:16.885 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:16 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:16.885 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:16 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:16.885 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:16 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:16.885 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:16 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:16.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:16 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:16.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:16 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:16.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:16 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:16.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:16 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:16.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:16 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:16.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:16 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:17.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:16 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:16 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:16 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:16 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:16 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:17.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:16 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:17.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.386+0000 7f6317b8f700 1 -- 172.21.15.43:0/3974267476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63101015f0 msgr2=0x7f63101019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:17.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.386+0000 7f6317b8f700 1 --2- 172.21.15.43:0/3974267476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63101015f0 0x7f63101019d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6300009910 tx=0x7f6300009c20 comp rx=0 tx=0).stop 2024-04-21T08:30:17.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.386+0000 7f6317b8f700 1 -- 172.21.15.43:0/3974267476 shutdown_connections 2024-04-21T08:30:17.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.386+0000 7f6317b8f700 1 --2- 172.21.15.43:0/3974267476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63101015f0 0x7f63101019d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:17.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.386+0000 7f6317b8f700 1 -- 172.21.15.43:0/3974267476 >> 172.21.15.43:0/3974267476 conn(0x7f6310076ea0 msgr2=0x7f63100772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:17.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.387+0000 7f6317b8f700 1 -- 172.21.15.43:0/3974267476 shutdown_connections 2024-04-21T08:30:17.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.387+0000 7f6317b8f700 1 -- 172.21.15.43:0/3974267476 wait complete. 2024-04-21T08:30:17.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.387+0000 7f6317b8f700 1 Processor -- start 2024-04-21T08:30:17.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.387+0000 7f6317b8f700 1 -- start start 2024-04-21T08:30:17.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.387+0000 7f6317b8f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63101015f0 0x7f6310103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:17.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.387+0000 7f6317b8f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6310103580 0x7f6310103a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:17.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.387+0000 7f6317b8f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63101a42f0 0x7f63101a66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:17.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.388+0000 7f6317b8f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f631010b820 con 0x7f6310103580 2024-04-21T08:30:17.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.388+0000 7f631512a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6310103580 0x7f6310103a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:17.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.388+0000 7f6317b8f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f631010b6a0 con 0x7f63101015f0 2024-04-21T08:30:17.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.388+0000 7f631512a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6310103580 0x7f6310103a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40108/0 (socket says 172.21.15.43:40108) 2024-04-21T08:30:17.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.388+0000 7f6317b8f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f631010b9a0 con 0x7f63101a42f0 2024-04-21T08:30:17.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.388+0000 7f631512a700 1 -- 172.21.15.43:0/1293936922 learned_addr learned my addr 172.21.15.43:0/1293936922 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:30:17.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.388+0000 7f631512a700 1 -- 172.21.15.43:0/1293936922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63101a42f0 msgr2=0x7f63101a66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:17.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.388+0000 7f631512a700 1 --2- 172.21.15.43:0/1293936922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63101a42f0 0x7f63101a66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:17.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.388+0000 7f631512a700 1 -- 172.21.15.43:0/1293936922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63101015f0 msgr2=0x7f6310103040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:17.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.388+0000 7f631512a700 1 --2- 172.21.15.43:0/1293936922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63101015f0 0x7f6310103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:17.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.388+0000 7f631512a700 1 -- 172.21.15.43:0/1293936922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63101a6e10 con 0x7f6310103580 2024-04-21T08:30:17.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.388+0000 7f631512a700 1 --2- 172.21.15.43:0/1293936922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6310103580 0x7f6310103a00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f630c00b780 tx=0x7f630c00bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:17.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.388+0000 7f6306ffd700 1 -- 172.21.15.43:0/1293936922 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c004230 con 0x7f6310103580 2024-04-21T08:30:17.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.388+0000 7f6306ffd700 1 -- 172.21.15.43:0/1293936922 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f630c0043d0 con 0x7f6310103580 2024-04-21T08:30:17.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.388+0000 7f6317b8f700 1 -- 172.21.15.43:0/1293936922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63101a7100 con 0x7f6310103580 2024-04-21T08:30:17.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.388+0000 7f6306ffd700 1 -- 172.21.15.43:0/1293936922 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c011bd0 con 0x7f6310103580 2024-04-21T08:30:17.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.388+0000 7f6317b8f700 1 -- 172.21.15.43:0/1293936922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63101a7690 con 0x7f6310103580 2024-04-21T08:30:17.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.389+0000 7f6306ffd700 1 -- 172.21.15.43:0/1293936922 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f630c01a400 con 0x7f6310103580 2024-04-21T08:30:17.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.389+0000 7f6306ffd700 1 --2- 172.21.15.43:0/1293936922 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f62fc039ae0 0x7f62fc03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:17.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.390+0000 7f6306ffd700 1 -- 172.21.15.43:0/1293936922 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f630c04c4c0 con 0x7f6310103580 2024-04-21T08:30:17.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.390+0000 7f631592b700 1 --2- 172.21.15.43:0/1293936922 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f62fc039ae0 0x7f62fc03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:17.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.390+0000 7f6317b8f700 1 -- 172.21.15.43:0/1293936922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63100640a0 con 0x7f6310103580 2024-04-21T08:30:17.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.390+0000 7f631592b700 1 --2- 172.21.15.43:0/1293936922 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f62fc039ae0 0x7f62fc03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f630000b5c0 tx=0x7f6300005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.395+0000 7f6306ffd700 1 -- 172.21.15.43:0/1293936922 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f630c014160 con 0x7f6310103580 2024-04-21T08:30:17.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.610+0000 7f6317b8f700 1 -- 172.21.15.43:0/1293936922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f63101a79b0 con 0x7f6310103580 2024-04-21T08:30:17.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.611+0000 7f6306ffd700 1 -- 172.21.15.43:0/1293936922 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f630c018510 con 0x7f6310103580 2024-04-21T08:30:17.610 INFO:teuthology.orchestra.run.smithi043.stdout:# minimal ceph.conf for d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:30:17.610 INFO:teuthology.orchestra.run.smithi043.stdout:[global] 2024-04-21T08:30:17.610 INFO:teuthology.orchestra.run.smithi043.stdout: fsid = d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:30:17.610 INFO:teuthology.orchestra.run.smithi043.stdout: mon_host = [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2024-04-21T08:30:17.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.613+0000 7f6317b8f700 1 -- 172.21.15.43:0/1293936922 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f62fc039ae0 msgr2=0x7f62fc03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:17.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.613+0000 7f6317b8f700 1 --2- 172.21.15.43:0/1293936922 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f62fc039ae0 0x7f62fc03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f630000b5c0 tx=0x7f6300005da0 comp rx=0 tx=0).stop 2024-04-21T08:30:17.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.613+0000 7f6317b8f700 1 -- 172.21.15.43:0/1293936922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6310103580 msgr2=0x7f6310103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:17.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.613+0000 7f6317b8f700 1 --2- 172.21.15.43:0/1293936922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6310103580 0x7f6310103a00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f630c00b780 tx=0x7f630c00bb40 comp rx=0 tx=0).stop 2024-04-21T08:30:17.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.613+0000 7f6317b8f700 1 -- 172.21.15.43:0/1293936922 shutdown_connections 2024-04-21T08:30:17.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.613+0000 7f6317b8f700 1 --2- 172.21.15.43:0/1293936922 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f62fc039ae0 0x7f62fc03bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:17.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.613+0000 7f6317b8f700 1 --2- 172.21.15.43:0/1293936922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63101a42f0 0x7f63101a66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:17.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.613+0000 7f6317b8f700 1 --2- 172.21.15.43:0/1293936922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63101015f0 0x7f6310103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:17.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.613+0000 7f6317b8f700 1 --2- 172.21.15.43:0/1293936922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6310103580 0x7f6310103a00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:17.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.613+0000 7f6317b8f700 1 -- 172.21.15.43:0/1293936922 >> 172.21.15.43:0/1293936922 conn(0x7f6310076ea0 msgr2=0x7f6310069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:17.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.613+0000 7f6317b8f700 1 -- 172.21.15.43:0/1293936922 shutdown_connections 2024-04-21T08:30:17.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:17.613+0000 7f6317b8f700 1 -- 172.21.15.43:0/1293936922 wait complete. 2024-04-21T08:30:17.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:17 smithi043 ceph-mon[27978]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:17.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:17 smithi043 ceph-mon[27978]: Updating smithi043:/etc/ceph/ceph.conf 2024-04-21T08:30:17.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:17 smithi043 ceph-mon[27978]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-21T08:30:17.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:17 smithi043 ceph-mon[27978]: Updating smithi175:/etc/ceph/ceph.conf 2024-04-21T08:30:17.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:17 smithi043 ceph-mon[27978]: Updating smithi043:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:17.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:17 smithi043 ceph-mon[27978]: Updating smithi175:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:17.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:17 smithi043 ceph-mon[27978]: Updating smithi112:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:17.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:17 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:17 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:17 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:17 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:17 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:17 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:17 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:17 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:17 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:17 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:17 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:17 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:17.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:17 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:30:17.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:17 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:17.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:17 smithi175 ceph-mon[31920]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:17.965 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:17 smithi175 ceph-mon[31920]: Updating smithi043:/etc/ceph/ceph.conf 2024-04-21T08:30:17.965 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:17 smithi175 ceph-mon[31920]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-21T08:30:17.965 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:17 smithi175 ceph-mon[31920]: Updating smithi175:/etc/ceph/ceph.conf 2024-04-21T08:30:17.965 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:17 smithi175 ceph-mon[31920]: Updating smithi043:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:17.965 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:17 smithi175 ceph-mon[31920]: Updating smithi175:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:17.965 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:17 smithi175 ceph-mon[31920]: Updating smithi112:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:17.965 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:17 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.965 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:17 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.965 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:17 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.965 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:17 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.965 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:17 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.965 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:17 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.965 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:17 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.966 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:17 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.966 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:17 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.966 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:17 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.966 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:17 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:17.966 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:17 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:17.966 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:17 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:30:17.966 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:17 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:18.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:17 smithi112 ceph-mon[31085]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:18.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:17 smithi112 ceph-mon[31085]: Updating smithi043:/etc/ceph/ceph.conf 2024-04-21T08:30:18.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:17 smithi112 ceph-mon[31085]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-21T08:30:18.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:17 smithi112 ceph-mon[31085]: Updating smithi175:/etc/ceph/ceph.conf 2024-04-21T08:30:18.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:17 smithi112 ceph-mon[31085]: Updating smithi043:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:18.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:17 smithi112 ceph-mon[31085]: Updating smithi175:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:18.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:17 smithi112 ceph-mon[31085]: Updating smithi112:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:18.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:17 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:18.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:17 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:18.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:17 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:18.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:17 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:18.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:17 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:18.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:17 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:18.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:17 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:18.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:17 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:18.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:17 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:18.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:17 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:18.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:17 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:18.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:17 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:18.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:17 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:30:18.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:17 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:18.203 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-21T08:30:18.203 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-21T08:30:18.203 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.conf 2024-04-21T08:30:18.271 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-21T08:30:18.271 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:30:18.338 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-21T08:30:18.338 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.conf 2024-04-21T08:30:18.370 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-21T08:30:18.370 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:30:18.438 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-21T08:30:18.439 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.conf 2024-04-21T08:30:18.470 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-21T08:30:18.470 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:30:18.536 INFO:tasks.cephadm:Adding mgr.x on smithi043 2024-04-21T08:30:18.536 INFO:tasks.cephadm:Adding mgr.y on smithi112 2024-04-21T08:30:18.536 INFO:tasks.cephadm:Adding mgr.z on smithi175 2024-04-21T08:30:18.536 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi043=x;smithi112=y;smithi175=z' 2024-04-21T08:30:18.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:18 smithi112 ceph-mon[31085]: Reconfiguring mon.a (unknown last config time)... 2024-04-21T08:30:18.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:18 smithi112 ceph-mon[31085]: Reconfiguring daemon mon.a on smithi043 2024-04-21T08:30:18.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:18 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1293936922' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:18.806 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:30:18.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:18 smithi043 ceph-mon[27978]: Reconfiguring mon.a (unknown last config time)... 2024-04-21T08:30:18.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:18 smithi043 ceph-mon[27978]: Reconfiguring daemon mon.a on smithi043 2024-04-21T08:30:18.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:18 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1293936922' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:18.928 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:18 smithi175 ceph-mon[31920]: Reconfiguring mon.a (unknown last config time)... 2024-04-21T08:30:18.928 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:18 smithi175 ceph-mon[31920]: Reconfiguring daemon mon.a on smithi043 2024-04-21T08:30:18.928 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:18 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1293936922' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:19.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:19 smithi112 ceph-mon[31085]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:19.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:19 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:19.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:19 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:19.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:19 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:19.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:19 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:19.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:19 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:30:19.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:19 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:19.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:19 smithi043 ceph-mon[27978]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:19.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:19 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:19.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:19 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:19.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:19 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:19.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:19 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:19.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:19 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:30:19.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:19 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:19.820 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:19 smithi175 ceph-mon[31920]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:19.820 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:19 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:19.820 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:19 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:19.820 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:19 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:19.820 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:19 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:19.820 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:19 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:30:19.820 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:19 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:20.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.225+0000 7f3d8f4e8700 1 -- 172.21.15.175:0/3579521162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d881025d0 msgr2=0x7f3d881029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:20.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.225+0000 7f3d8f4e8700 1 --2- 172.21.15.175:0/3579521162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d881025d0 0x7f3d881029f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3d78009910 tx=0x7f3d78009c20 comp rx=0 tx=0).stop 2024-04-21T08:30:20.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.225+0000 7f3d8f4e8700 1 -- 172.21.15.175:0/3579521162 shutdown_connections 2024-04-21T08:30:20.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.225+0000 7f3d8f4e8700 1 --2- 172.21.15.175:0/3579521162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d881025d0 0x7f3d881029f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:20.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.225+0000 7f3d8f4e8700 1 -- 172.21.15.175:0/3579521162 >> 172.21.15.175:0/3579521162 conn(0x7f3d880fdb70 msgr2=0x7f3d880fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:20.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.225+0000 7f3d8f4e8700 1 -- 172.21.15.175:0/3579521162 shutdown_connections 2024-04-21T08:30:20.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.225+0000 7f3d8f4e8700 1 -- 172.21.15.175:0/3579521162 wait complete. 2024-04-21T08:30:20.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.225+0000 7f3d8f4e8700 1 Processor -- start 2024-04-21T08:30:20.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.225+0000 7f3d8f4e8700 1 -- start start 2024-04-21T08:30:20.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.226+0000 7f3d8f4e8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d881025d0 0x7f3d88197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:20.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.226+0000 7f3d8f4e8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d88198270 0x7f3d8819d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:20.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.226+0000 7f3d8f4e8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d881986f0 0x7f3d8819f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:20.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.226+0000 7f3d8f4e8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3d88108510 con 0x7f3d88198270 2024-04-21T08:30:20.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.226+0000 7f3d8f4e8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3d88108390 con 0x7f3d881025d0 2024-04-21T08:30:20.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.226+0000 7f3d8f4e8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3d88108690 con 0x7f3d881986f0 2024-04-21T08:30:20.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.226+0000 7f3d8da85700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d881986f0 0x7f3d8819f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:20.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.226+0000 7f3d8ca83700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d88198270 0x7f3d8819d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:20.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.226+0000 7f3d8da85700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d881986f0 0x7f3d8819f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57054/0 (socket says 172.21.15.175:57054) 2024-04-21T08:30:20.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.226+0000 7f3d8da85700 1 -- 172.21.15.175:0/1396151635 learned_addr learned my addr 172.21.15.175:0/1396151635 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-21T08:30:20.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.226+0000 7f3d8d284700 1 --2- 172.21.15.175:0/1396151635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d881025d0 0x7f3d88197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:20.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.226+0000 7f3d8da85700 1 -- 172.21.15.175:0/1396151635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d881025d0 msgr2=0x7f3d88197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:20.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.226+0000 7f3d8da85700 1 --2- 172.21.15.175:0/1396151635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d881025d0 0x7f3d88197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:20.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.226+0000 7f3d8da85700 1 -- 172.21.15.175:0/1396151635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d88198270 msgr2=0x7f3d8819d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:20.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.226+0000 7f3d8da85700 1 --2- 172.21.15.175:0/1396151635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d88198270 0x7f3d8819d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:20.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.226+0000 7f3d8da85700 1 -- 172.21.15.175:0/1396151635 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d881a0150 con 0x7f3d881986f0 2024-04-21T08:30:20.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.226+0000 7f3d8d284700 1 --2- 172.21.15.175:0/1396151635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d881025d0 0x7f3d88197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:30:20.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.226+0000 7f3d8ca83700 1 --2- 172.21.15.175:0/1396151635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d88198270 0x7f3d8819d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:30:20.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.226+0000 7f3d8da85700 1 --2- 172.21.15.175:0/1396151635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d881986f0 0x7f3d8819f850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3d8000ba50 tx=0x7f3d8000bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:20.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.226+0000 7f3d7e7fc700 1 -- 172.21.15.175:0/1396151635 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d80019070 con 0x7f3d881986f0 2024-04-21T08:30:20.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.227+0000 7f3d7e7fc700 1 -- 172.21.15.175:0/1396151635 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d80014440 con 0x7f3d881986f0 2024-04-21T08:30:20.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.227+0000 7f3d8f4e8700 1 -- 172.21.15.175:0/1396151635 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d881a0440 con 0x7f3d881986f0 2024-04-21T08:30:20.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.227+0000 7f3d7e7fc700 1 -- 172.21.15.175:0/1396151635 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8000ce60 con 0x7f3d881986f0 2024-04-21T08:30:20.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.227+0000 7f3d8f4e8700 1 -- 172.21.15.175:0/1396151635 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d881a09b0 con 0x7f3d881986f0 2024-04-21T08:30:20.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.227+0000 7f3d7e7fc700 1 -- 172.21.15.175:0/1396151635 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3d8001c070 con 0x7f3d881986f0 2024-04-21T08:30:20.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.227+0000 7f3d8f4e8700 1 -- 172.21.15.175:0/1396151635 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d880640a0 con 0x7f3d881986f0 2024-04-21T08:30:20.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.228+0000 7f3d7e7fc700 1 --2- 172.21.15.175:0/1396151635 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f3d74039ae0 0x7f3d7403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:20.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.228+0000 7f3d7e7fc700 1 -- 172.21.15.175:0/1396151635 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3d8004be10 con 0x7f3d881986f0 2024-04-21T08:30:20.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.228+0000 7f3d8d284700 1 --2- 172.21.15.175:0/1396151635 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f3d74039ae0 0x7f3d7403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:20.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.228+0000 7f3d8d284700 1 --2- 172.21.15.175:0/1396151635 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f3d74039ae0 0x7f3d7403bfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3d78005190 tx=0x7f3d7801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:20.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.233+0000 7f3d7e7fc700 1 -- 172.21.15.175:0/1396151635 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d80014910 con 0x7f3d881986f0 2024-04-21T08:30:20.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.441+0000 7f3d8f4e8700 1 -- 172.21.15.175:0/1396151635 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi043=x;smithi112=y;smithi175=z", "target": ["mon-mgr", ""]}) -- 0x7f3d88062f10 con 0x7f3d74039ae0 2024-04-21T08:30:20.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.448+0000 7f3d7e7fc700 1 -- 172.21.15.175:0/1396151635 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3d88062f10 con 0x7f3d74039ae0 2024-04-21T08:30:20.447 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled mgr update... 2024-04-21T08:30:20.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.450+0000 7f3d8f4e8700 1 -- 172.21.15.175:0/1396151635 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f3d74039ae0 msgr2=0x7f3d7403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:20.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.450+0000 7f3d8f4e8700 1 --2- 172.21.15.175:0/1396151635 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f3d74039ae0 0x7f3d7403bfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3d78005190 tx=0x7f3d7801a040 comp rx=0 tx=0).stop 2024-04-21T08:30:20.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.450+0000 7f3d8f4e8700 1 -- 172.21.15.175:0/1396151635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d881986f0 msgr2=0x7f3d8819f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:20.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.450+0000 7f3d8f4e8700 1 --2- 172.21.15.175:0/1396151635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d881986f0 0x7f3d8819f850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3d8000ba50 tx=0x7f3d8000bd60 comp rx=0 tx=0).stop 2024-04-21T08:30:20.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.450+0000 7f3d8f4e8700 1 -- 172.21.15.175:0/1396151635 shutdown_connections 2024-04-21T08:30:20.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.450+0000 7f3d8f4e8700 1 --2- 172.21.15.175:0/1396151635 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f3d74039ae0 0x7f3d7403bfa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:20.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.450+0000 7f3d8f4e8700 1 --2- 172.21.15.175:0/1396151635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d881986f0 0x7f3d8819f850 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:20.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.450+0000 7f3d8f4e8700 1 --2- 172.21.15.175:0/1396151635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d881025d0 0x7f3d88197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:20.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.450+0000 7f3d8f4e8700 1 --2- 172.21.15.175:0/1396151635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d88198270 0x7f3d8819d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:20.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.450+0000 7f3d8f4e8700 1 -- 172.21.15.175:0/1396151635 >> 172.21.15.175:0/1396151635 conn(0x7f3d880fdb70 msgr2=0x7f3d880fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:20.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.450+0000 7f3d8f4e8700 1 -- 172.21.15.175:0/1396151635 shutdown_connections 2024-04-21T08:30:20.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:30:20.450+0000 7f3d8f4e8700 1 -- 172.21.15.175:0/1396151635 wait complete. 2024-04-21T08:30:20.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:20 smithi175 ceph-mon[31920]: Reconfiguring mon.b (monmap changed)... 2024-04-21T08:30:20.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:20 smithi175 ceph-mon[31920]: Reconfiguring daemon mon.b on smithi112 2024-04-21T08:30:20.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:20 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:21.013 DEBUG:teuthology.orchestra.run.smithi112:mgr.y> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mgr.y.service 2024-04-21T08:30:21.016 DEBUG:teuthology.orchestra.run.smithi175:mgr.z> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mgr.z.service 2024-04-21T08:30:21.018 INFO:tasks.cephadm:Deploying OSDs... 2024-04-21T08:30:21.018 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-21T08:30:21.018 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/scratch_devs of=/dev/stdout 2024-04-21T08:30:21.037 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:20 smithi112 ceph-mon[31085]: Reconfiguring mon.b (monmap changed)... 2024-04-21T08:30:21.037 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:20 smithi112 ceph-mon[31085]: Reconfiguring daemon mon.b on smithi112 2024-04-21T08:30:21.037 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:20 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:21.038 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-21T08:30:21.039 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_1 2024-04-21T08:30:21.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:20 smithi043 ceph-mon[27978]: Reconfiguring mon.b (monmap changed)... 2024-04-21T08:30:21.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:20 smithi043 ceph-mon[27978]: Reconfiguring daemon mon.b on smithi112 2024-04-21T08:30:21.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:20 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:21.079 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-21T08:30:21.079 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:30:21.079 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-21T08:30:21.079 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:30:21.079 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:30:21.079 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-04-21 08:29:24.162295386 +0000 2024-04-21T08:30:21.079 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-04-21 08:23:48.328887007 +0000 2024-04-21T08:30:21.079 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-04-21 08:23:48.328887007 +0000 2024-04-21T08:30:21.079 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-04-21 08:23:48.328887007 +0000 2024-04-21T08:30:21.079 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-21T08:30:21.148 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-21T08:30:21.148 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-21T08:30:21.148 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000115492 s, 4.4 MB/s 2024-04-21T08:30:21.150 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-21T08:30:21.209 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_2 2024-04-21T08:30:21.266 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-21T08:30:21.266 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:30:21.266 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-04-21T08:30:21.266 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:30:21.266 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:30:21.266 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-04-21 08:29:24.198294954 +0000 2024-04-21T08:30:21.267 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-04-21 08:23:48.676882779 +0000 2024-04-21T08:30:21.267 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-04-21 08:23:48.676882779 +0000 2024-04-21T08:30:21.267 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-04-21 08:23:48.676882779 +0000 2024-04-21T08:30:21.267 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-21T08:30:21.334 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-21T08:30:21.334 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-21T08:30:21.334 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000115862 s, 4.4 MB/s 2024-04-21T08:30:21.336 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-21T08:30:21.393 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_3 2024-04-21T08:30:21.450 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-21T08:30:21.450 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:30:21.450 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-21T08:30:21.450 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:30:21.450 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:30:21.450 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-04-21 08:29:24.239294461 +0000 2024-04-21T08:30:21.450 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-04-21 08:23:49.012878697 +0000 2024-04-21T08:30:21.450 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-04-21 08:23:49.012878697 +0000 2024-04-21T08:30:21.451 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-04-21 08:23:49.012878697 +0000 2024-04-21T08:30:21.451 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-21T08:30:21.514 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-21T08:30:21.514 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-21T08:30:21.514 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000110386 s, 4.6 MB/s 2024-04-21T08:30:21.515 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-21T08:30:21.573 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_4 2024-04-21T08:30:21.629 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-21T08:30:21.629 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:30:21.629 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-21T08:30:21.629 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:30:21.629 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:30:21.629 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-04-21 08:29:24.274294041 +0000 2024-04-21T08:30:21.629 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-04-21 08:23:49.353874554 +0000 2024-04-21T08:30:21.629 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-04-21 08:23:49.353874554 +0000 2024-04-21T08:30:21.629 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-04-21 08:23:49.353874554 +0000 2024-04-21T08:30:21.629 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-21T08:30:21.693 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-21T08:30:21.693 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-21T08:30:21.693 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000108696 s, 4.7 MB/s 2024-04-21T08:30:21.697 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-21T08:30:21.754 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-21T08:30:21.754 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/scratch_devs of=/dev/stdout 2024-04-21T08:30:21.774 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-21T08:30:21.774 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_1 2024-04-21T08:30:21.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:21 smithi043 ceph-mon[27978]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:21.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:21 smithi043 ceph-mon[27978]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi043=x;smithi112=y;smithi175=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:30:21.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:21 smithi043 ceph-mon[27978]: Saving service mgr spec with placement smithi043=x;smithi112=y;smithi175=z;count:3 2024-04-21T08:30:21.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:21 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:21.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:21 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:21.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:21 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:21.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:21 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:30:21.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:21 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:21.832 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-21T08:30:21.832 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:30:21.832 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-21T08:30:21.832 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:30:21.832 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:30:21.832 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-21 08:30:08.577258841 +0000 2024-04-21T08:30:21.832 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-21 08:24:16.987321136 +0000 2024-04-21T08:30:21.832 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-21 08:24:16.987321136 +0000 2024-04-21T08:30:21.832 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-21 08:24:16.987321136 +0000 2024-04-21T08:30:21.832 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-21T08:30:21.898 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-21T08:30:21.898 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-21T08:30:21.898 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000342844 s, 1.5 MB/s 2024-04-21T08:30:21.900 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-21T08:30:21.958 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_2 2024-04-21T08:30:21.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:21 smithi175 ceph-mon[31920]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:21.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:21 smithi175 ceph-mon[31920]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi043=x;smithi112=y;smithi175=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:30:21.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:21 smithi175 ceph-mon[31920]: Saving service mgr spec with placement smithi043=x;smithi112=y;smithi175=z;count:3 2024-04-21T08:30:21.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:21 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:21.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:21 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:21.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:21 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:21.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:21 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:30:21.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:21 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:22.015 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-21T08:30:22.015 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:30:22.015 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-21T08:30:22.015 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:30:22.015 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:30:22.015 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-21 08:30:08.577258841 +0000 2024-04-21T08:30:22.015 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-21 08:24:17.326312232 +0000 2024-04-21T08:30:22.015 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-21 08:24:17.326312232 +0000 2024-04-21T08:30:22.015 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-21 08:24:17.326312232 +0000 2024-04-21T08:30:22.016 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-21T08:30:22.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:21 smithi112 ceph-mon[31085]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:22.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:21 smithi112 ceph-mon[31085]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi043=x;smithi112=y;smithi175=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:30:22.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:21 smithi112 ceph-mon[31085]: Saving service mgr spec with placement smithi043=x;smithi112=y;smithi175=z;count:3 2024-04-21T08:30:22.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:21 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:22.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:21 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:22.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:21 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:22.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:21 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:30:22.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:21 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:22.068 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-21T08:30:22.068 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-21T08:30:22.068 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000196916 s, 2.6 MB/s 2024-04-21T08:30:22.070 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-21T08:30:22.126 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_3 2024-04-21T08:30:22.183 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-21T08:30:22.184 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:30:22.184 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-21T08:30:22.184 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:30:22.184 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:30:22.184 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-21 08:30:08.577258841 +0000 2024-04-21T08:30:22.184 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-21 08:24:17.680302935 +0000 2024-04-21T08:30:22.184 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-21 08:24:17.680302935 +0000 2024-04-21T08:30:22.185 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-21 08:24:17.680302935 +0000 2024-04-21T08:30:22.185 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-21T08:30:22.245 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-21T08:30:22.246 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-21T08:30:22.246 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000251411 s, 2.0 MB/s 2024-04-21T08:30:22.248 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-21T08:30:22.305 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_4 2024-04-21T08:30:22.360 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-21T08:30:22.360 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:30:22.360 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-21T08:30:22.361 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:30:22.361 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:30:22.361 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-21 08:30:08.578258815 +0000 2024-04-21T08:30:22.361 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-21 08:24:18.019294032 +0000 2024-04-21T08:30:22.361 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-21 08:24:18.019294032 +0000 2024-04-21T08:30:22.361 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-21 08:24:18.019294032 +0000 2024-04-21T08:30:22.361 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-21T08:30:22.423 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-21T08:30:22.423 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-21T08:30:22.423 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000232004 s, 2.2 MB/s 2024-04-21T08:30:22.424 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-21T08:30:22.480 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-21T08:30:22.480 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/scratch_devs of=/dev/stdout 2024-04-21T08:30:22.499 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-21T08:30:22.499 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_1 2024-04-21T08:30:22.556 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-21T08:30:22.556 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:30:22.556 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-21T08:30:22.556 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:30:22.556 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:30:22.556 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-04-21 08:29:58.842210724 +0000 2024-04-21T08:30:22.556 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-04-21 08:24:02.719830821 +0000 2024-04-21T08:30:22.556 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-04-21 08:24:02.719830821 +0000 2024-04-21T08:30:22.556 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-04-21 08:24:02.719830821 +0000 2024-04-21T08:30:22.556 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-21T08:30:22.622 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-04-21T08:30:22.623 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-04-21T08:30:22.623 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000284225 s, 1.8 MB/s 2024-04-21T08:30:22.624 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-21T08:30:22.682 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_2 2024-04-21T08:30:22.738 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-21T08:30:22.739 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:30:22.739 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-21T08:30:22.739 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:30:22.739 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:30:22.739 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-04-21 08:29:58.843210707 +0000 2024-04-21T08:30:22.739 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-04-21 08:24:03.061824930 +0000 2024-04-21T08:30:22.739 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-04-21 08:24:03.061824930 +0000 2024-04-21T08:30:22.739 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-04-21 08:24:03.061824930 +0000 2024-04-21T08:30:22.739 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-21T08:30:22.804 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-04-21T08:30:22.804 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-04-21T08:30:22.805 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000237184 s, 2.2 MB/s 2024-04-21T08:30:22.806 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-21T08:30:22.862 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_3 2024-04-21T08:30:22.917 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-21T08:30:22.917 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:30:22.917 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-21T08:30:22.917 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:30:22.917 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:30:22.917 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-04-21 08:29:58.843210707 +0000 2024-04-21T08:30:22.917 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-04-21 08:24:03.400819111 +0000 2024-04-21T08:30:22.917 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-04-21 08:24:03.400819111 +0000 2024-04-21T08:30:22.917 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-04-21 08:24:03.400819111 +0000 2024-04-21T08:30:22.918 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-21T08:30:22.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:22 smithi175 ceph-mon[31920]: Reconfiguring mon.c (monmap changed)... 2024-04-21T08:30:22.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:22 smithi175 ceph-mon[31920]: Reconfiguring daemon mon.c on smithi175 2024-04-21T08:30:22.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:22 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:22.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:22 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:22.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:22 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:22.978 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-04-21T08:30:22.979 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-04-21T08:30:22.979 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.00020641 s, 2.5 MB/s 2024-04-21T08:30:22.980 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-21T08:30:23.035 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_4 2024-04-21T08:30:23.043 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:22 smithi112 ceph-mon[31085]: Reconfiguring mon.c (monmap changed)... 2024-04-21T08:30:23.043 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:22 smithi112 ceph-mon[31085]: Reconfiguring daemon mon.c on smithi175 2024-04-21T08:30:23.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:22 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:23.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:22 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:23.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:22 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:23.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:22 smithi043 ceph-mon[27978]: Reconfiguring mon.c (monmap changed)... 2024-04-21T08:30:23.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:22 smithi043 ceph-mon[27978]: Reconfiguring daemon mon.c on smithi175 2024-04-21T08:30:23.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:22 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:23.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:22 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:23.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:22 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:23.090 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-21T08:30:23.090 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:30:23.090 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-21T08:30:23.091 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:30:23.091 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:30:23.091 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-04-21 08:29:58.843210707 +0000 2024-04-21T08:30:23.091 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-04-21 08:24:03.736813470 +0000 2024-04-21T08:30:23.091 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-04-21 08:24:03.736813470 +0000 2024-04-21T08:30:23.091 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-04-21 08:24:03.736813470 +0000 2024-04-21T08:30:23.091 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-21T08:30:23.153 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-04-21T08:30:23.153 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-04-21T08:30:23.153 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000240724 s, 2.1 MB/s 2024-04-21T08:30:23.154 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-21T08:30:23.209 INFO:tasks.cephadm:Deploying osd.0 on smithi043 with /dev/vg_nvme/lv_4... 2024-04-21T08:30:23.210 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-21T08:30:23.449 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:30:23.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:23 smithi043 ceph-mon[27978]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:23.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:23.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:23.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:23.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:30:23.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T08:30:23.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:30:23.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:23.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:23 smithi175 ceph-mon[31920]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:23.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:23.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:23.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:23.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:30:23.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T08:30:23.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:30:23.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:24.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:23 smithi112 ceph-mon[31085]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:24.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:24.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:24.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:24.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:30:24.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T08:30:24.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:30:24.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:24.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:24 smithi175 ceph-mon[31920]: Deploying daemon mgr.z on smithi175 2024-04-21T08:30:24.964 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:24 smithi175 systemd[1]: Starting Ceph mgr.z for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:30:25.043 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:24 smithi112 ceph-mon[31085]: Deploying daemon mgr.z on smithi175 2024-04-21T08:30:25.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:24 smithi043 ceph-mon[27978]: Deploying daemon mgr.z on smithi175 2024-04-21T08:30:25.283 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-21T08:30:25.283 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-21T08:30:25.283 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-04-21T08:30:25.283 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-04-21T08:30:25.283 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00825167 s, 1.3 GB/s 2024-04-21T08:30:25.283 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-04-21T08:30:25.284 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:24 smithi175 podman[33766]: 2024-04-21 08:30:24.898986102 +0000 UTC m=+0.018252976 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:30:25.284 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:24 smithi175 podman[33766]: 2024-04-21 08:30:24.999431373 +0000 UTC m=+0.118698232 container create b7aab1eaed462ac8ec5008d5fe9e3014779cef450f60e2e4f1bda5a5f9d0efd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-21T08:30:25.284 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:25 smithi175 podman[33766]: 2024-04-21 08:30:25.209841447 +0000 UTC m=+0.329108306 container init b7aab1eaed462ac8ec5008d5fe9e3014779cef450f60e2e4f1bda5a5f9d0efd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-21T08:30:25.284 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:25 smithi175 podman[33766]: 2024-04-21 08:30:25.212379848 +0000 UTC m=+0.331646714 container start b7aab1eaed462ac8ec5008d5fe9e3014779cef450f60e2e4f1bda5a5f9d0efd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-21T08:30:25.284 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:25 smithi175 ceph-mgr[33785]: -- 172.21.15.175:0/3210713774 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5564d3d805a0 con 0x5564d3d55000 2024-04-21T08:30:25.284 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:25 smithi175 bash[33766]: b7aab1eaed462ac8ec5008d5fe9e3014779cef450f60e2e4f1bda5a5f9d0efd2 2024-04-21T08:30:25.548 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:25 smithi175 systemd[1]: Started Ceph mgr.z for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:30:25.549 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:25 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:25.364+0000 7f873a29f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:30:25.549 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:25 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:25.549+0000 7f873a29f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:30:25.854 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:25 smithi043 ceph-mon[27978]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:25.854 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:25 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:25.854 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:25 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:25.854 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:25 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:25.854 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:25 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:30:25.854 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:25 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T08:30:25.854 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:25 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:30:25.854 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:25 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:25.856 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_4 2024-04-21T08:30:25.958 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:25 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:25.650+0000 7f873a29f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:30:25.958 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:25 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:25.959+0000 7f873a29f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:30:25.958 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:25 smithi175 ceph-mon[31920]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:25.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:25 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:25.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:25 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:25.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:25 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:25.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:25 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:30:25.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:25 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T08:30:25.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:25 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:30:25.959 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:25 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:26.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:25 smithi112 ceph-mon[31085]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:26.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:25 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:26.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:25 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:26.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:25 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:26.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:25 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:30:26.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:25 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T08:30:26.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:25 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:30:26.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:25 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:26.098 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:30:26.213 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:26 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:26.043+0000 7f873a29f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:30:26.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:26 smithi043 ceph-mon[27978]: Deploying daemon mgr.y on smithi112 2024-04-21T08:30:26.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:26 smithi175 ceph-mon[31920]: Deploying daemon mgr.y on smithi112 2024-04-21T08:30:26.970 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:26 smithi112 ceph-mon[31085]: Deploying daemon mgr.y on smithi112 2024-04-21T08:30:27.544 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:27 smithi112 systemd[1]: Starting Ceph mgr.y for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:30:27.576 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:27 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:27.237+0000 7f873a29f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:30:27.576 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:27 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:27.405+0000 7f873a29f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:30:27.576 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:27 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:27.577+0000 7f873a29f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:30:27.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.653+0000 7f2188838700 1 -- 172.21.15.43:0/3790265870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2184103ca0 msgr2=0x7f2184104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:27.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.653+0000 7f2188838700 1 --2- 172.21.15.43:0/3790265870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2184103ca0 0x7f2184104100 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f216c0099c0 tx=0x7f216c009cd0 comp rx=0 tx=0).stop 2024-04-21T08:30:27.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.653+0000 7f2188838700 1 -- 172.21.15.43:0/3790265870 shutdown_connections 2024-04-21T08:30:27.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.653+0000 7f2188838700 1 --2- 172.21.15.43:0/3790265870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2184102aa0 0x7f2184102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:27.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.653+0000 7f2188838700 1 --2- 172.21.15.43:0/3790265870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2184104640 0x7f218410aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:27.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.653+0000 7f2188838700 1 --2- 172.21.15.43:0/3790265870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2184103ca0 0x7f2184104100 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:27.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.653+0000 7f2188838700 1 -- 172.21.15.43:0/3790265870 >> 172.21.15.43:0/3790265870 conn(0x7f21840fe040 msgr2=0x7f2184100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:27.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.653+0000 7f2188838700 1 -- 172.21.15.43:0/3790265870 shutdown_connections 2024-04-21T08:30:27.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.653+0000 7f2188838700 1 -- 172.21.15.43:0/3790265870 wait complete. 2024-04-21T08:30:27.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.654+0000 7f2188838700 1 Processor -- start 2024-04-21T08:30:27.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.654+0000 7f2188838700 1 -- start start 2024-04-21T08:30:27.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.654+0000 7f2188838700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2184102aa0 0x7f2184198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:27.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.654+0000 7f2188838700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2184103ca0 0x7f21841990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:27.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.654+0000 7f2188838700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2184104640 0x7f21841a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:27.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.654+0000 7f2188838700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f218410e530 con 0x7f2184104640 2024-04-21T08:30:27.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.654+0000 7f2188838700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f218410e3b0 con 0x7f2184102aa0 2024-04-21T08:30:27.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.654+0000 7f2188838700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f218410e6b0 con 0x7f2184103ca0 2024-04-21T08:30:27.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.654+0000 7f218259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2184102aa0 0x7f2184198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:27.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.654+0000 7f2182d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2184104640 0x7f21841a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:27.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.654+0000 7f218259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2184102aa0 0x7f2184198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:35998/0 (socket says 172.21.15.43:35998) 2024-04-21T08:30:27.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.654+0000 7f218259c700 1 -- 172.21.15.43:0/1373040837 learned_addr learned my addr 172.21.15.43:0/1373040837 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:30:27.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.654+0000 7f2181d9b700 1 --2- 172.21.15.43:0/1373040837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2184103ca0 0x7f21841990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:27.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.654+0000 7f218259c700 1 -- 172.21.15.43:0/1373040837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2184103ca0 msgr2=0x7f21841990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:27.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.654+0000 7f218259c700 1 --2- 172.21.15.43:0/1373040837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2184103ca0 0x7f21841990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:27.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.654+0000 7f218259c700 1 -- 172.21.15.43:0/1373040837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2184104640 msgr2=0x7f21841a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:27.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.654+0000 7f218259c700 1 --2- 172.21.15.43:0/1373040837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2184104640 0x7f21841a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:27.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.654+0000 7f218259c700 1 -- 172.21.15.43:0/1373040837 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21841a07e0 con 0x7f2184102aa0 2024-04-21T08:30:27.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.654+0000 7f2182d9d700 1 --2- 172.21.15.43:0/1373040837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2184104640 0x7f21841a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:30:27.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.655+0000 7f218259c700 1 --2- 172.21.15.43:0/1373040837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2184102aa0 0x7f2184198b80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f217400e950 tx=0x7f217400ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:27.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.655+0000 7f217b7fe700 1 -- 172.21.15.43:0/1373040837 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174004230 con 0x7f2184102aa0 2024-04-21T08:30:27.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.655+0000 7f2188838700 1 -- 172.21.15.43:0/1373040837 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21841a0ad0 con 0x7f2184102aa0 2024-04-21T08:30:27.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.655+0000 7f2188838700 1 -- 172.21.15.43:0/1373040837 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21841a1010 con 0x7f2184102aa0 2024-04-21T08:30:27.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.655+0000 7f217b7fe700 1 -- 172.21.15.43:0/1373040837 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21740043d0 con 0x7f2184102aa0 2024-04-21T08:30:27.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.655+0000 7f217b7fe700 1 -- 172.21.15.43:0/1373040837 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174003cd0 con 0x7f2184102aa0 2024-04-21T08:30:27.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.656+0000 7f217b7fe700 1 -- 172.21.15.43:0/1373040837 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2174021440 con 0x7f2184102aa0 2024-04-21T08:30:27.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.656+0000 7f2188838700 1 -- 172.21.15.43:0/1373040837 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2164005360 con 0x7f2184102aa0 2024-04-21T08:30:27.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.656+0000 7f217b7fe700 1 --2- 172.21.15.43:0/1373040837 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2170039890 0x7f217003bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:27.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.656+0000 7f217b7fe700 1 -- 172.21.15.43:0/1373040837 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f217404cc70 con 0x7f2184102aa0 2024-04-21T08:30:27.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.656+0000 7f2181d9b700 1 --2- 172.21.15.43:0/1373040837 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2170039890 0x7f217003bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:27.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.657+0000 7f2181d9b700 1 --2- 172.21.15.43:0/1373040837 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2170039890 0x7f217003bd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f216c000c00 tx=0x7f216c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.661+0000 7f217b7fe700 1 -- 172.21.15.43:0/1373040837 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f217401fc70 con 0x7f2184102aa0 2024-04-21T08:30:27.862 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:27 smithi175 ceph-mon[31920]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:27.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:27.880+0000 7f2188838700 1 -- 172.21.15.43:0/1373040837 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2164000bf0 con 0x7f2170039890 2024-04-21T08:30:27.884 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:27 smithi112 ceph-mon[31085]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:27.885 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:27 smithi112 podman[32459]: 2024-04-21 08:30:27.522329343 +0000 UTC m=+0.017855178 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:30:27.885 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:27 smithi112 podman[32459]: 2024-04-21 08:30:27.632251841 +0000 UTC m=+0.127796427 container create 905d62cd1f86b9d74a2509f442789538a4a2cd97b544879d9c1aa0fb44563545 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416) 2024-04-21T08:30:27.885 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:27 smithi112 podman[32459]: 2024-04-21 08:30:27.859387113 +0000 UTC m=+0.354912949 container init 905d62cd1f86b9d74a2509f442789538a4a2cd97b544879d9c1aa0fb44563545 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:30:27.885 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:27 smithi112 podman[32459]: 2024-04-21 08:30:27.861829276 +0000 UTC m=+0.357355114 container start 905d62cd1f86b9d74a2509f442789538a4a2cd97b544879d9c1aa0fb44563545 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True) 2024-04-21T08:30:27.885 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:27 smithi112 ceph-mgr[32478]: -- 172.21.15.112:0/2473106657 <== mon.2 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562dc37f25a0 con 0x562dc37c7000 2024-04-21T08:30:28.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:27 smithi043 ceph-mon[27978]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:28.161 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:27 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:27.863+0000 7f873a29f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:30:28.162 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:27 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:27.954+0000 7f873a29f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:30:28.162 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:28 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:28.162+0000 7f873a29f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:30:28.197 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:27 smithi112 bash[32459]: 905d62cd1f86b9d74a2509f442789538a4a2cd97b544879d9c1aa0fb44563545 2024-04-21T08:30:28.197 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:27 smithi112 systemd[1]: Started Ceph mgr.y for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:30:28.197 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:28 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:28.011+0000 7f9960f7d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:30:28.197 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:28 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:28.197+0000 7f9960f7d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:30:28.463 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:28 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:28.362+0000 7f873a29f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:30:28.543 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:28 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:28.307+0000 7f9960f7d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:30:28.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:28 smithi043 ceph-mon[27978]: from='client.24107 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:30:28.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:28 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:30:28.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:28 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:30:28.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:28 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:28.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:28 smithi043 ceph-mon[27978]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:28.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:28 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:28.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:28 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:28.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:28 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:28.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:28 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:28.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:28 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:28.979 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:28 smithi175 ceph-mon[31920]: from='client.24107 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:30:28.979 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:28 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:30:28.979 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:28 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:30:28.979 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:28 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:28.979 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:28 smithi175 ceph-mon[31920]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:28.979 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:28 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:28.979 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:28 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:28.979 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:28 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:28.980 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:28 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:28.980 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:28 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:28.996 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:28 smithi112 ceph-mon[31085]: from='client.24107 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:30:28.996 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:28 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:30:28.996 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:28 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:30:28.996 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:28 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:28.996 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:28 smithi112 ceph-mon[31085]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:28.996 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:28 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:28.996 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:28 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:28.996 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:28 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:28.996 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:28 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:28.996 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:28 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:28.997 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:28 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:28.623+0000 7f9960f7d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:30:28.997 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:28 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:28.717+0000 7f9960f7d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:30:29.443 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:29 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:29.004+0000 7f873a29f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:30:29.713 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:29 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:29.444+0000 7f873a29f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:30:29.713 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:29 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:29.543+0000 7f873a29f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:30:30.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:30.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:30.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:30.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:30.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:30.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:30.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:30.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:30.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:29 smithi043 ceph-mon[27978]: Reconfiguring mgr.x (unknown last config time)... 2024-04-21T08:30:30.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:30:30.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:30:30.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:30.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:29 smithi043 ceph-mon[27978]: Reconfiguring daemon mgr.x on smithi043 2024-04-21T08:30:30.213 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:29 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:29.745+0000 7f873a29f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:30:30.381 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:29 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:30.381 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:29 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:30.381 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:29 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:30.381 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:29 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:30.381 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:29 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:30.381 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:29 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:30.382 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:29 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:30.382 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:29 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:30.382 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:29 smithi175 ceph-mon[31920]: Reconfiguring mgr.x (unknown last config time)... 2024-04-21T08:30:30.382 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:29 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:30:30.382 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:29 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:30:30.382 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:29 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:30.382 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:29 smithi175 ceph-mon[31920]: Reconfiguring daemon mgr.x on smithi043 2024-04-21T08:30:30.383 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:29 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:29.948+0000 7f9960f7d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:30:30.383 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:30 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:30.116+0000 7f9960f7d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:30:30.383 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:30 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:30.283+0000 7f9960f7d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:30:30.383 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:29 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:30.383 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:29 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:30.383 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:29 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:30.383 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:29 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:30.383 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:29 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:30.383 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:29 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:30.383 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:29 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:30.383 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:29 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:30.384 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:29 smithi112 ceph-mon[31085]: Reconfiguring mgr.x (unknown last config time)... 2024-04-21T08:30:30.384 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:29 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:30:30.384 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:29 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:30:30.384 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:29 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:30.384 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:29 smithi112 ceph-mon[31085]: Reconfiguring daemon mgr.x on smithi043 2024-04-21T08:30:30.614 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:30 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:30.283+0000 7f873a29f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:30:30.615 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:30 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:30.374+0000 7f873a29f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:30:30.859 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:30 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:30.569+0000 7f9960f7d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:30:30.859 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:30 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:30.661+0000 7f9960f7d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:30:30.859 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:30 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:30.860+0000 7f9960f7d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:30:30.959 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:30 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:30.615+0000 7f873a29f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:30:30.959 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:30 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:30.749+0000 7f873a29f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:30:30.959 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:30 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:30.844+0000 7f873a29f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:30:30.959 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:30 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:30.960+0000 7f873a29f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:30:31.293 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:31 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:31.050+0000 7f9960f7d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:30:31.544 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:31 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:31.287+0000 7f873a29f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:30:31.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:31 smithi043 ceph-mon[27978]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:31.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:31 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3861005647' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "070c2bde-f131-4523-b54c-d376a5015826"}]: dispatch 2024-04-21T08:30:31.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:31 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3861005647' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "070c2bde-f131-4523-b54c-d376a5015826"}]': finished 2024-04-21T08:30:31.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:31 smithi043 ceph-mon[27978]: osdmap e5: 1 total, 0 up, 1 in 2024-04-21T08:30:31.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:31 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:31.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:31 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/4128369706' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:30:31.884 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:31 smithi175 ceph-mon[31920]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:31.884 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:31 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3861005647' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "070c2bde-f131-4523-b54c-d376a5015826"}]: dispatch 2024-04-21T08:30:31.884 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:31 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3861005647' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "070c2bde-f131-4523-b54c-d376a5015826"}]': finished 2024-04-21T08:30:31.884 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:31 smithi175 ceph-mon[31920]: osdmap e5: 1 total, 0 up, 1 in 2024-04-21T08:30:31.884 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:31 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:31.884 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:31 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/4128369706' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:30:32.043 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:31 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:31.641+0000 7f9960f7d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:30:32.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:31 smithi112 ceph-mon[31085]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:32.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:31 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3861005647' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "070c2bde-f131-4523-b54c-d376a5015826"}]: dispatch 2024-04-21T08:30:32.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:31 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3861005647' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "070c2bde-f131-4523-b54c-d376a5015826"}]': finished 2024-04-21T08:30:32.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:31 smithi112 ceph-mon[31085]: osdmap e5: 1 total, 0 up, 1 in 2024-04-21T08:30:32.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:31 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:32.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:31 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/4128369706' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:30:32.213 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:31 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:31.884+0000 7f873a29f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:30:32.214 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:30:32 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:30:32.099+0000 7f873a29f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:30:32.350 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:32 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:32.057+0000 7f9960f7d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:30:32.350 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:32 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:32.151+0000 7f9960f7d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:30:32.350 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:32 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:32.350+0000 7f9960f7d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:30:32.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:32.553+0000 7f217b7fe700 1 -- 172.21.15.43:0/1373040837 <== mon.2 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f21740107b0 con 0x7f2184102aa0 2024-04-21T08:30:32.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:32 smithi112 ceph-mon[31085]: Standby manager daemon z started 2024-04-21T08:30:32.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:32 smithi112 ceph-mon[31085]: from='mgr.? 172.21.15.175:0/2412156453' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T08:30:32.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:32 smithi112 ceph-mon[31085]: from='mgr.? 172.21.15.175:0/2412156453' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:30:32.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:32 smithi112 ceph-mon[31085]: from='mgr.? 172.21.15.175:0/2412156453' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T08:30:32.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:32 smithi112 ceph-mon[31085]: from='mgr.? 172.21.15.175:0/2412156453' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:30:32.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:32 smithi043 ceph-mon[27978]: Standby manager daemon z started 2024-04-21T08:30:32.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:32 smithi043 ceph-mon[27978]: from='mgr.? 172.21.15.175:0/2412156453' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T08:30:32.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:32 smithi043 ceph-mon[27978]: from='mgr.? 172.21.15.175:0/2412156453' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:30:32.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:32 smithi043 ceph-mon[27978]: from='mgr.? 172.21.15.175:0/2412156453' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T08:30:32.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:32 smithi043 ceph-mon[27978]: from='mgr.? 172.21.15.175:0/2412156453' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:30:32.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:32 smithi175 ceph-mon[31920]: Standby manager daemon z started 2024-04-21T08:30:32.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:32 smithi175 ceph-mon[31920]: from='mgr.? 172.21.15.175:0/2412156453' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T08:30:32.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:32 smithi175 ceph-mon[31920]: from='mgr.? 172.21.15.175:0/2412156453' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:30:32.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:32 smithi175 ceph-mon[31920]: from='mgr.? 172.21.15.175:0/2412156453' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T08:30:32.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:32 smithi175 ceph-mon[31920]: from='mgr.? 172.21.15.175:0/2412156453' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:30:33.199 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:32 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:32.869+0000 7f9960f7d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:30:33.199 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:32 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:32.961+0000 7f9960f7d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:30:33.536 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:33 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:33.199+0000 7f9960f7d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:30:33.536 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:33 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:33.329+0000 7f9960f7d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:30:33.536 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:33 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:33.422+0000 7f9960f7d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:30:33.536 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:33 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:33.537+0000 7f9960f7d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:30:33.793 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:33 smithi112 ceph-mon[31085]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:33.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:33 smithi112 ceph-mon[31085]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-21T08:30:33.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:33 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T08:30:33.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:33 smithi043 ceph-mon[27978]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:33.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:33 smithi043 ceph-mon[27978]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-21T08:30:33.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:33 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T08:30:33.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:33 smithi175 ceph-mon[31920]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:33.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:33 smithi175 ceph-mon[31920]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-21T08:30:33.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:33 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T08:30:34.293 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:33 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:33.840+0000 7f9960f7d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:30:34.794 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:34 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:34.433+0000 7f9960f7d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:30:34.794 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:30:34 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:30:34.645+0000 7f9960f7d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:30:35.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:35.563+0000 7f217b7fe700 1 -- 172.21.15.43:0/1373040837 <== mon.2 v2:172.21.15.112:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f2174016370 con 0x7f2184102aa0 2024-04-21T08:30:35.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:35 smithi043 ceph-mon[27978]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:35.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:35 smithi043 ceph-mon[27978]: Standby manager daemon y started 2024-04-21T08:30:35.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:35 smithi043 ceph-mon[27978]: from='mgr.? 172.21.15.112:0/206006546' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T08:30:35.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:35 smithi043 ceph-mon[27978]: from='mgr.? 172.21.15.112:0/206006546' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:30:35.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:35 smithi043 ceph-mon[27978]: from='mgr.? 172.21.15.112:0/206006546' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T08:30:35.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:35 smithi043 ceph-mon[27978]: from='mgr.? 172.21.15.112:0/206006546' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:30:35.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:35 smithi175 ceph-mon[31920]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:35.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:35 smithi175 ceph-mon[31920]: Standby manager daemon y started 2024-04-21T08:30:35.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:35 smithi175 ceph-mon[31920]: from='mgr.? 172.21.15.112:0/206006546' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T08:30:35.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:35 smithi175 ceph-mon[31920]: from='mgr.? 172.21.15.112:0/206006546' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:30:35.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:35 smithi175 ceph-mon[31920]: from='mgr.? 172.21.15.112:0/206006546' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T08:30:35.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:35 smithi175 ceph-mon[31920]: from='mgr.? 172.21.15.112:0/206006546' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:30:36.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:35 smithi112 ceph-mon[31085]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:36.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:35 smithi112 ceph-mon[31085]: Standby manager daemon y started 2024-04-21T08:30:36.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:35 smithi112 ceph-mon[31085]: from='mgr.? 172.21.15.112:0/206006546' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T08:30:36.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:35 smithi112 ceph-mon[31085]: from='mgr.? 172.21.15.112:0/206006546' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:30:36.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:35 smithi112 ceph-mon[31085]: from='mgr.? 172.21.15.112:0/206006546' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T08:30:36.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:35 smithi112 ceph-mon[31085]: from='mgr.? 172.21.15.112:0/206006546' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:30:36.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:36 smithi175 ceph-mon[31920]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-21T08:30:36.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:36 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T08:30:36.723 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:36 smithi043 ceph-mon[27978]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-21T08:30:36.724 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:36 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T08:30:37.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:36 smithi112 ceph-mon[31085]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-21T08:30:37.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:36 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T08:30:37.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:37 smithi043 ceph-mon[27978]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:37.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:37 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:37.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:37 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:37.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:37 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:37.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:37 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:37.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:37 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:37.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:37 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:37.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:37 smithi175 ceph-mon[31920]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:37.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:37 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:37.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:37 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:37.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:37 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:37.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:37 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:37.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:37 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:37.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:37 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:38.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:37 smithi112 ceph-mon[31085]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:38.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:37 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:38.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:37 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:38.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:37 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:38.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:37 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:38.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:37 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:38.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:37 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:39.849 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:39 smithi043 ceph-mon[27978]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:39.849 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:39 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-21T08:30:39.849 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:39 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:39.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:39 smithi175 ceph-mon[31920]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:39.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:39 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-21T08:30:39.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:39 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:40.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:39 smithi112 ceph-mon[31085]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:40.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:39 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-21T08:30:40.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:39 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:40.862 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:40 smithi043 ceph-mon[27978]: Deploying daemon osd.0 on smithi043 2024-04-21T08:30:40.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:40 smithi175 ceph-mon[31920]: Deploying daemon osd.0 on smithi043 2024-04-21T08:30:41.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:40 smithi112 ceph-mon[31085]: Deploying daemon osd.0 on smithi043 2024-04-21T08:30:41.881 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:41 smithi043 ceph-mon[27978]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:41.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:41 smithi175 ceph-mon[31920]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:42.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:41 smithi112 ceph-mon[31085]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:43.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:43 smithi043 ceph-mon[27978]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:43.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:43 smithi175 ceph-mon[31920]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:44.043 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:43 smithi112 ceph-mon[31085]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:45.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:45 smithi112 ceph-mon[31085]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:45.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:45 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:45.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:45 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:45.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:45 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:45.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:45 smithi043 ceph-mon[27978]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:45.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:45 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:45.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:45 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:45.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:45 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:45.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:45 smithi175 ceph-mon[31920]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:45.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:45 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:45.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:45 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:45.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:45 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:47.788 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:47 smithi043 ceph-mon[27978]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:47.788 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:47 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:47.788 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:47 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:47.788 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:47 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:47.788 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:47 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:47.788 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:47 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:47.793 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:47 smithi112 ceph-mon[31085]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:47.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:47 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:47.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:47 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:47.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:47 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:47.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:47 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:47.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:47 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:47.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:47 smithi175 ceph-mon[31920]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:47.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:47 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:47.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:47 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:47.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:47 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:47.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:47 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:47.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:47 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:48.438 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 0 on host 'smithi043' 2024-04-21T08:30:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:48.439+0000 7f217b7fe700 1 -- 172.21.15.43:0/1373040837 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2164000bf0 con 0x7f2170039890 2024-04-21T08:30:48.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:48.441+0000 7f2188838700 1 -- 172.21.15.43:0/1373040837 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2170039890 msgr2=0x7f217003bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:48.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:48.441+0000 7f2188838700 1 --2- 172.21.15.43:0/1373040837 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2170039890 0x7f217003bd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f216c000c00 tx=0x7f216c005fb0 comp rx=0 tx=0).stop 2024-04-21T08:30:48.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:48.441+0000 7f2188838700 1 -- 172.21.15.43:0/1373040837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2184102aa0 msgr2=0x7f2184198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:48.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:48.441+0000 7f2188838700 1 --2- 172.21.15.43:0/1373040837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2184102aa0 0x7f2184198b80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f217400e950 tx=0x7f217400ec60 comp rx=0 tx=0).stop 2024-04-21T08:30:48.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:48.441+0000 7f2188838700 1 -- 172.21.15.43:0/1373040837 shutdown_connections 2024-04-21T08:30:48.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:48.441+0000 7f2188838700 1 --2- 172.21.15.43:0/1373040837 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2170039890 0x7f217003bd50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:48.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:48.441+0000 7f2188838700 1 --2- 172.21.15.43:0/1373040837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2184104640 0x7f21841a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:48.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:48.441+0000 7f2188838700 1 --2- 172.21.15.43:0/1373040837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2184102aa0 0x7f2184198b80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:48.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:48.441+0000 7f2188838700 1 --2- 172.21.15.43:0/1373040837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2184103ca0 0x7f21841990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:48.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:48.441+0000 7f2188838700 1 -- 172.21.15.43:0/1373040837 >> 172.21.15.43:0/1373040837 conn(0x7f21840fe040 msgr2=0x7f2184106fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:48.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:48.441+0000 7f2188838700 1 -- 172.21.15.43:0/1373040837 shutdown_connections 2024-04-21T08:30:48.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:30:48.441+0000 7f2188838700 1 -- 172.21.15.43:0/1373040837 wait complete. 2024-04-21T08:30:48.727 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:48 smithi043 ceph-mon[27978]: from='osd.0 [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-21T08:30:48.728 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:48 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:48.728 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:48 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:48.728 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:48 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:48.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:48 smithi112 ceph-mon[31085]: from='osd.0 [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-21T08:30:48.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:48 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:48.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:48 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:48.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:48 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:48.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:48 smithi175 ceph-mon[31920]: from='osd.0 [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-21T08:30:48.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:48 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:48.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:48 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:48.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:48 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:49.045 DEBUG:teuthology.orchestra.run.smithi043:osd.0> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.0.service 2024-04-21T08:30:49.047 INFO:tasks.cephadm:Deploying osd.1 on smithi112 with /dev/vg_nvme/lv_4... 2024-04-21T08:30:49.048 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-21T08:30:49.290 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:30:49.731 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:49 smithi112 ceph-mon[31085]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:49.732 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:49 smithi112 ceph-mon[31085]: from='osd.0 [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-21T08:30:49.732 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:49 smithi112 ceph-mon[31085]: osdmap e6: 1 total, 0 up, 1 in 2024-04-21T08:30:49.732 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:49 smithi112 ceph-mon[31085]: from='osd.0 [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-21T08:30:49.732 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:49 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:49.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:49 smithi043 ceph-mon[27978]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:49.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:49 smithi043 ceph-mon[27978]: from='osd.0 [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-21T08:30:49.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:49 smithi043 ceph-mon[27978]: osdmap e6: 1 total, 0 up, 1 in 2024-04-21T08:30:49.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:49 smithi043 ceph-mon[27978]: from='osd.0 [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-21T08:30:49.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:49 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:49.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:49 smithi175 ceph-mon[31920]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:49.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:49 smithi175 ceph-mon[31920]: from='osd.0 [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-21T08:30:49.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:49 smithi175 ceph-mon[31920]: osdmap e6: 1 total, 0 up, 1 in 2024-04-21T08:30:49.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:49 smithi175 ceph-mon[31920]: from='osd.0 [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-21T08:30:49.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:49 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:50.226 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 21 08:30:50 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-0[40338]: 2024-04-21T08:30:50.078+0000 7fef10f85700 -1 osd.0 0 waiting for initial osdmap 2024-04-21T08:30:50.226 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 21 08:30:50 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-0[40338]: 2024-04-21T08:30:50.081+0000 7fef08d70700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:30:50.701 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:50 smithi112 ceph-mon[31085]: from='osd.0 [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-21T08:30:50.702 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:50 smithi112 ceph-mon[31085]: osdmap e7: 1 total, 0 up, 1 in 2024-04-21T08:30:50.702 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:50 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:50.702 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:50 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:50.702 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:50 smithi112 ceph-mon[31085]: from='osd.0 [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768]' entity='osd.0' 2024-04-21T08:30:50.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:50 smithi043 ceph-mon[27978]: from='osd.0 [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-21T08:30:50.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:50 smithi043 ceph-mon[27978]: osdmap e7: 1 total, 0 up, 1 in 2024-04-21T08:30:50.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:50 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:50.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:50 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:50.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:50 smithi043 ceph-mon[27978]: from='osd.0 [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768]' entity='osd.0' 2024-04-21T08:30:50.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:50 smithi175 ceph-mon[31920]: from='osd.0 [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-21T08:30:50.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:50 smithi175 ceph-mon[31920]: osdmap e7: 1 total, 0 up, 1 in 2024-04-21T08:30:50.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:50 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:50.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:50 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:50.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:50 smithi175 ceph-mon[31920]: from='osd.0 [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768]' entity='osd.0' 2024-04-21T08:30:51.269 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-21T08:30:51.270 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-21T08:30:51.270 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-04-21T08:30:51.270 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-04-21T08:30:51.270 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114952 s, 912 MB/s 2024-04-21T08:30:51.270 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-04-21T08:30:51.591 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:51 smithi112 ceph-mon[31085]: purged_snaps scrub starts 2024-04-21T08:30:51.592 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:51 smithi112 ceph-mon[31085]: purged_snaps scrub ok 2024-04-21T08:30:51.592 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:51 smithi112 ceph-mon[31085]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:51.592 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:51 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:51.592 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:51 smithi112 ceph-mon[31085]: osd.0 [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768] boot 2024-04-21T08:30:51.592 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:51 smithi112 ceph-mon[31085]: osdmap e8: 1 total, 1 up, 1 in 2024-04-21T08:30:51.592 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:51 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:51.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:51 smithi043 ceph-mon[27978]: purged_snaps scrub starts 2024-04-21T08:30:51.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:51 smithi043 ceph-mon[27978]: purged_snaps scrub ok 2024-04-21T08:30:51.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:51 smithi043 ceph-mon[27978]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:51.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:51 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:51.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:51 smithi043 ceph-mon[27978]: osd.0 [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768] boot 2024-04-21T08:30:51.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:51 smithi043 ceph-mon[27978]: osdmap e8: 1 total, 1 up, 1 in 2024-04-21T08:30:51.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:51 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:51.896 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_4 2024-04-21T08:30:51.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:51 smithi175 ceph-mon[31920]: purged_snaps scrub starts 2024-04-21T08:30:51.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:51 smithi175 ceph-mon[31920]: purged_snaps scrub ok 2024-04-21T08:30:51.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:51 smithi175 ceph-mon[31920]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:51.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:51 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:51.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:51 smithi175 ceph-mon[31920]: osd.0 [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768] boot 2024-04-21T08:30:51.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:51 smithi175 ceph-mon[31920]: osdmap e8: 1 total, 1 up, 1 in 2024-04-21T08:30:51.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:51 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:52.145 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:30:53.225 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:53 smithi112 ceph-mon[31085]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:30:53.225 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:53 smithi112 ceph-mon[31085]: Detected new or changed devices on smithi043 2024-04-21T08:30:53.225 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:53 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:53.225 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:53 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:53.225 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:53 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:30:53.225 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:53 smithi112 ceph-mon[31085]: Adjusting osd_memory_target on smithi043 to 17093M 2024-04-21T08:30:53.225 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:53 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:53.225 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:53 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:53.226 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:53 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:53.226 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:53 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:53.226 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:53 smithi112 ceph-mon[31085]: osdmap e9: 1 total, 1 up, 1 in 2024-04-21T08:30:53.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:53 smithi043 ceph-mon[27978]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:30:53.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:53 smithi043 ceph-mon[27978]: Detected new or changed devices on smithi043 2024-04-21T08:30:53.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:53 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:53.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:53 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:53.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:53 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:30:53.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:53 smithi043 ceph-mon[27978]: Adjusting osd_memory_target on smithi043 to 17093M 2024-04-21T08:30:53.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:53 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:53.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:53 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:53.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:53 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:53.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:53 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:53.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:53 smithi043 ceph-mon[27978]: osdmap e9: 1 total, 1 up, 1 in 2024-04-21T08:30:53.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:53 smithi175 ceph-mon[31920]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:30:53.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:53 smithi175 ceph-mon[31920]: Detected new or changed devices on smithi043 2024-04-21T08:30:53.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:53 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:53.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:53 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:53.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:53 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:30:53.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:53 smithi175 ceph-mon[31920]: Adjusting osd_memory_target on smithi043 to 17093M 2024-04-21T08:30:53.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:53 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:53.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:53 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:53.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:53 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:53.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:53 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:30:53.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:53 smithi175 ceph-mon[31920]: osdmap e9: 1 total, 1 up, 1 in 2024-04-21T08:30:53.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.714+0000 7fd3d8796700 1 -- 172.21.15.112:0/678639386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3d0104640 msgr2=0x7fd3d010aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:53.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.714+0000 7fd3d8796700 1 --2- 172.21.15.112:0/678639386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3d0104640 0x7fd3d010aef0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd3c800b0d0 tx=0x7fd3c800b3e0 comp rx=0 tx=0).stop 2024-04-21T08:30:53.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.714+0000 7fd3d8796700 1 -- 172.21.15.112:0/678639386 shutdown_connections 2024-04-21T08:30:53.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.714+0000 7fd3d8796700 1 --2- 172.21.15.112:0/678639386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3d0102aa0 0x7fd3d0102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:53.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.714+0000 7fd3d8796700 1 --2- 172.21.15.112:0/678639386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3d0104640 0x7fd3d010aef0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:53.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.714+0000 7fd3d8796700 1 --2- 172.21.15.112:0/678639386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3d0103ca0 0x7fd3d0104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:53.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.714+0000 7fd3d8796700 1 -- 172.21.15.112:0/678639386 >> 172.21.15.112:0/678639386 conn(0x7fd3d00fe040 msgr2=0x7fd3d0100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:53.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.714+0000 7fd3d8796700 1 -- 172.21.15.112:0/678639386 shutdown_connections 2024-04-21T08:30:53.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.714+0000 7fd3d8796700 1 -- 172.21.15.112:0/678639386 wait complete. 2024-04-21T08:30:53.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.714+0000 7fd3d8796700 1 Processor -- start 2024-04-21T08:30:53.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.714+0000 7fd3d8796700 1 -- start start 2024-04-21T08:30:53.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.715+0000 7fd3d8796700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3d0102aa0 0x7fd3d0198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:53.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.715+0000 7fd3d8796700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3d0103ca0 0x7fd3d0199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:53.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.715+0000 7fd3d8796700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3d0104640 0x7fd3d01a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:53.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.715+0000 7fd3d8796700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d010e540 con 0x7fd3d0102aa0 2024-04-21T08:30:53.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.715+0000 7fd3d8796700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d010e3c0 con 0x7fd3d0103ca0 2024-04-21T08:30:53.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.715+0000 7fd3d8796700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d010e6c0 con 0x7fd3d0104640 2024-04-21T08:30:53.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.715+0000 7fd3d5d31700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3d0103ca0 0x7fd3d0199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:53.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.715+0000 7fd3d5d31700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3d0103ca0 0x7fd3d0199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48694/0 (socket says 172.21.15.112:48694) 2024-04-21T08:30:53.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.715+0000 7fd3d5d31700 1 -- 172.21.15.112:0/3009824775 learned_addr learned my addr 172.21.15.112:0/3009824775 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-21T08:30:53.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.715+0000 7fd3d6d33700 1 --2- 172.21.15.112:0/3009824775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3d0104640 0x7fd3d01a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:53.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.715+0000 7fd3d6532700 1 --2- 172.21.15.112:0/3009824775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3d0102aa0 0x7fd3d0198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:53.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.715+0000 7fd3d5d31700 1 -- 172.21.15.112:0/3009824775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3d0104640 msgr2=0x7fd3d01a00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:53.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.715+0000 7fd3d5d31700 1 --2- 172.21.15.112:0/3009824775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3d0104640 0x7fd3d01a00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:53.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.715+0000 7fd3d5d31700 1 -- 172.21.15.112:0/3009824775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3d0102aa0 msgr2=0x7fd3d0198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:53.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.715+0000 7fd3d5d31700 1 --2- 172.21.15.112:0/3009824775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3d0102aa0 0x7fd3d0198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:53.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.715+0000 7fd3d5d31700 1 -- 172.21.15.112:0/3009824775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d01a0760 con 0x7fd3d0103ca0 2024-04-21T08:30:53.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.715+0000 7fd3d5d31700 1 --2- 172.21.15.112:0/3009824775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3d0103ca0 0x7fd3d0199040 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000c9b0 tx=0x7fd3c000ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:53.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.715+0000 7fd3c77fe700 1 -- 172.21.15.112:0/3009824775 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0007b00 con 0x7fd3d0103ca0 2024-04-21T08:30:53.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.715+0000 7fd3d8796700 1 -- 172.21.15.112:0/3009824775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d01a0a50 con 0x7fd3d0103ca0 2024-04-21T08:30:53.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.715+0000 7fd3c77fe700 1 -- 172.21.15.112:0/3009824775 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3c000ce50 con 0x7fd3d0103ca0 2024-04-21T08:30:53.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.715+0000 7fd3c77fe700 1 -- 172.21.15.112:0/3009824775 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0018660 con 0x7fd3d0103ca0 2024-04-21T08:30:53.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.716+0000 7fd3d8796700 1 -- 172.21.15.112:0/3009824775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d01a0f90 con 0x7fd3d0103ca0 2024-04-21T08:30:53.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.716+0000 7fd3d8796700 1 -- 172.21.15.112:0/3009824775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3d0069730 con 0x7fd3d0103ca0 2024-04-21T08:30:53.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.718+0000 7fd3c77fe700 1 -- 172.21.15.112:0/3009824775 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fd3c000f450 con 0x7fd3d0103ca0 2024-04-21T08:30:53.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.718+0000 7fd3c77fe700 1 --2- 172.21.15.112:0/3009824775 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd3bc0a3bb0 0x7fd3bc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:53.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.718+0000 7fd3c77fe700 1 -- 172.21.15.112:0/3009824775 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fd3c00cea60 con 0x7fd3d0103ca0 2024-04-21T08:30:53.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.718+0000 7fd3d6532700 1 --2- 172.21.15.112:0/3009824775 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd3bc0a3bb0 0x7fd3bc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:53.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.719+0000 7fd3d6532700 1 --2- 172.21.15.112:0/3009824775 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd3bc0a3bb0 0x7fd3bc0a6070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0103b00 tx=0x7fd3cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:53.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.722+0000 7fd3c77fe700 1 -- 172.21.15.112:0/3009824775 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd3c0014030 con 0x7fd3d0103ca0 2024-04-21T08:30:53.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:30:53.927+0000 7fd3d8796700 1 -- 172.21.15.112:0/3009824775 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd3d0062e80 con 0x7fd3bc0a3bb0 2024-04-21T08:30:54.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:54 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:30:54.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:54 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:30:54.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:54 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:54.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:54 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:30:54.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:54 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:30:54.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:54 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:54.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:54 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:30:54.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:54 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:30:54.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:54 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:55.293 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:55 smithi112 ceph-mon[31085]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:30:55.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:55 smithi112 ceph-mon[31085]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:30:55.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:55 smithi043 ceph-mon[27978]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:30:55.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:55 smithi043 ceph-mon[27978]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:30:55.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:55 smithi175 ceph-mon[31920]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:30:55.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:55 smithi175 ceph-mon[31920]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:30:57.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:57 smithi112 ceph-mon[31085]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:30:57.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:57 smithi112 ceph-mon[31085]: from='client.? 172.21.15.112:0/3885411218' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f297cc81-3025-40c8-8081-1b201be2c4bd"}]: dispatch 2024-04-21T08:30:57.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:57 smithi112 ceph-mon[31085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f297cc81-3025-40c8-8081-1b201be2c4bd"}]: dispatch 2024-04-21T08:30:57.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:57 smithi112 ceph-mon[31085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f297cc81-3025-40c8-8081-1b201be2c4bd"}]': finished 2024-04-21T08:30:57.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:57 smithi112 ceph-mon[31085]: osdmap e10: 2 total, 1 up, 2 in 2024-04-21T08:30:57.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:57 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:30:57.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:57 smithi043 ceph-mon[27978]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:30:57.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:57 smithi043 ceph-mon[27978]: from='client.? 172.21.15.112:0/3885411218' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f297cc81-3025-40c8-8081-1b201be2c4bd"}]: dispatch 2024-04-21T08:30:57.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:57 smithi043 ceph-mon[27978]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f297cc81-3025-40c8-8081-1b201be2c4bd"}]: dispatch 2024-04-21T08:30:57.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:57 smithi043 ceph-mon[27978]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f297cc81-3025-40c8-8081-1b201be2c4bd"}]': finished 2024-04-21T08:30:57.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:57 smithi043 ceph-mon[27978]: osdmap e10: 2 total, 1 up, 2 in 2024-04-21T08:30:57.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:57 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:30:57.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:57 smithi175 ceph-mon[31920]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:30:57.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:57 smithi175 ceph-mon[31920]: from='client.? 172.21.15.112:0/3885411218' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f297cc81-3025-40c8-8081-1b201be2c4bd"}]: dispatch 2024-04-21T08:30:57.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:57 smithi175 ceph-mon[31920]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f297cc81-3025-40c8-8081-1b201be2c4bd"}]: dispatch 2024-04-21T08:30:57.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:57 smithi175 ceph-mon[31920]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f297cc81-3025-40c8-8081-1b201be2c4bd"}]': finished 2024-04-21T08:30:57.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:57 smithi175 ceph-mon[31920]: osdmap e10: 2 total, 1 up, 2 in 2024-04-21T08:30:57.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:57 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:30:58.293 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:58 smithi112 ceph-mon[31085]: from='client.? 172.21.15.112:0/4290030755' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:30:58.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:58 smithi043 ceph-mon[27978]: from='client.? 172.21.15.112:0/4290030755' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:30:58.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:58 smithi175 ceph-mon[31920]: from='client.? 172.21.15.112:0/4290030755' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:30:59.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:30:59 smithi112 ceph-mon[31085]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:30:59.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:30:59 smithi043 ceph-mon[27978]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:30:59.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:30:59 smithi175 ceph-mon[31920]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:01.293 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:01 smithi112 ceph-mon[31085]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:01.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:01 smithi043 ceph-mon[27978]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:01.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:01 smithi175 ceph-mon[31920]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:03.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:03 smithi112 ceph-mon[31085]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:03.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:03 smithi043 ceph-mon[27978]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:03.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:03 smithi175 ceph-mon[31920]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:04.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:04 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-21T08:31:04.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:04 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:04.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:04 smithi112 ceph-mon[31085]: Deploying daemon osd.1 on smithi112 2024-04-21T08:31:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:04 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-21T08:31:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:04 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:04 smithi043 ceph-mon[27978]: Deploying daemon osd.1 on smithi112 2024-04-21T08:31:04.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:04 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-21T08:31:04.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:04 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:04.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:04 smithi175 ceph-mon[31920]: Deploying daemon osd.1 on smithi112 2024-04-21T08:31:05.149 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:05 smithi112 ceph-mon[31085]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:05.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:05 smithi043 ceph-mon[27978]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:05.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:05 smithi175 ceph-mon[31920]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:07.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:07 smithi112 ceph-mon[31085]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:07.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:07 smithi043 ceph-mon[27978]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:07.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:07 smithi175 ceph-mon[31920]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:09.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:09 smithi112 ceph-mon[31085]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:09.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:09 smithi043 ceph-mon[27978]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:09.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:09 smithi175 ceph-mon[31920]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:10.293 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:10 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:10.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:10 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:10.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:10 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:10.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:10 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:10.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:10 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:10.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:10 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:10.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:10 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:10.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:10 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:10.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:10 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:11.253 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:11 smithi112 ceph-mon[31085]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:11.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:11 smithi175 ceph-mon[31920]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:11.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:11 smithi043 ceph-mon[27978]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:12.789 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 1 on host 'smithi112' 2024-04-21T08:31:12.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:31:12.789+0000 7fd3c77fe700 1 -- 172.21.15.112:0/3009824775 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd3d0062e80 con 0x7fd3bc0a3bb0 2024-04-21T08:31:12.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:31:12.792+0000 7fd3d8796700 1 -- 172.21.15.112:0/3009824775 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd3bc0a3bb0 msgr2=0x7fd3bc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:31:12.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:31:12.792+0000 7fd3d8796700 1 --2- 172.21.15.112:0/3009824775 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd3bc0a3bb0 0x7fd3bc0a6070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0103b00 tx=0x7fd3cc009260 comp rx=0 tx=0).stop 2024-04-21T08:31:12.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:31:12.792+0000 7fd3d8796700 1 -- 172.21.15.112:0/3009824775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3d0103ca0 msgr2=0x7fd3d0199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:31:12.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:31:12.792+0000 7fd3d8796700 1 --2- 172.21.15.112:0/3009824775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3d0103ca0 0x7fd3d0199040 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000c9b0 tx=0x7fd3c000ccc0 comp rx=0 tx=0).stop 2024-04-21T08:31:12.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:31:12.792+0000 7fd3d8796700 1 -- 172.21.15.112:0/3009824775 shutdown_connections 2024-04-21T08:31:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:31:12.792+0000 7fd3d8796700 1 --2- 172.21.15.112:0/3009824775 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd3bc0a3bb0 0x7fd3bc0a6070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:31:12.792+0000 7fd3d8796700 1 --2- 172.21.15.112:0/3009824775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3d0104640 0x7fd3d01a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:31:12.792+0000 7fd3d8796700 1 --2- 172.21.15.112:0/3009824775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3d0102aa0 0x7fd3d0198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:31:12.792+0000 7fd3d8796700 1 --2- 172.21.15.112:0/3009824775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3d0103ca0 0x7fd3d0199040 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:31:12.792+0000 7fd3d8796700 1 -- 172.21.15.112:0/3009824775 >> 172.21.15.112:0/3009824775 conn(0x7fd3d00fe040 msgr2=0x7fd3d0106fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:31:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:31:12.792+0000 7fd3d8796700 1 -- 172.21.15.112:0/3009824775 shutdown_connections 2024-04-21T08:31:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:31:12.792+0000 7fd3d8796700 1 -- 172.21.15.112:0/3009824775 wait complete. 2024-04-21T08:31:13.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:12 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:13.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:12 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:13.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:12 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:13.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:12 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:31:13.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:12 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:13.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:12 smithi112 ceph-mon[31085]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:13.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:12 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:13.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:12 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:13.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:12 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:13.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:12 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:13.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:12 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:13.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:12 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:13.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:12 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:31:13.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:12 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:13.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:12 smithi175 ceph-mon[31920]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:13.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:12 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:13.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:12 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:13.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:12 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:13.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:12 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:13.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:12 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:13.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:12 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:13.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:12 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:31:13.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:12 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:13.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:12 smithi043 ceph-mon[27978]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:13.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:12 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:13.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:12 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:13.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:12 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:13.406 DEBUG:teuthology.orchestra.run.smithi112:osd.1> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.1.service 2024-04-21T08:31:13.408 INFO:tasks.cephadm:Deploying osd.2 on smithi175 with /dev/vg_nvme/lv_4... 2024-04-21T08:31:13.408 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-21T08:31:13.653 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:31:14.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:13 smithi112 ceph-mon[31085]: from='osd.1 [v2:172.21.15.112:6800/2362092039,v1:172.21.15.112:6801/2362092039]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T08:31:14.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:13 smithi112 ceph-mon[31085]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T08:31:14.079 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:13 smithi175 ceph-mon[31920]: from='osd.1 [v2:172.21.15.112:6800/2362092039,v1:172.21.15.112:6801/2362092039]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T08:31:14.079 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:13 smithi175 ceph-mon[31920]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T08:31:14.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:13 smithi043 ceph-mon[27978]: from='osd.1 [v2:172.21.15.112:6800/2362092039,v1:172.21.15.112:6801/2362092039]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T08:31:14.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:13 smithi043 ceph-mon[27978]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T08:31:14.961 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:14 smithi112 ceph-mon[31085]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-21T08:31:14.962 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:14 smithi112 ceph-mon[31085]: from='osd.1 [v2:172.21.15.112:6800/2362092039,v1:172.21.15.112:6801/2362092039]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-21T08:31:14.962 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:14 smithi112 ceph-mon[31085]: osdmap e11: 2 total, 1 up, 2 in 2024-04-21T08:31:14.962 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:14.962 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:14 smithi112 ceph-mon[31085]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-21T08:31:14.962 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:14 smithi112 ceph-mon[31085]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:14.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:14 smithi175 ceph-mon[31920]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-21T08:31:14.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:14 smithi175 ceph-mon[31920]: from='osd.1 [v2:172.21.15.112:6800/2362092039,v1:172.21.15.112:6801/2362092039]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-21T08:31:14.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:14 smithi175 ceph-mon[31920]: osdmap e11: 2 total, 1 up, 2 in 2024-04-21T08:31:14.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:14.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:14 smithi175 ceph-mon[31920]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-21T08:31:14.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:14 smithi175 ceph-mon[31920]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:15.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:14 smithi043 ceph-mon[27978]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-21T08:31:15.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:14 smithi043 ceph-mon[27978]: from='osd.1 [v2:172.21.15.112:6800/2362092039,v1:172.21.15.112:6801/2362092039]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-21T08:31:15.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:14 smithi043 ceph-mon[27978]: osdmap e11: 2 total, 1 up, 2 in 2024-04-21T08:31:15.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:15.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:14 smithi043 ceph-mon[27978]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-21T08:31:15.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:14 smithi043 ceph-mon[27978]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:15.504 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-21T08:31:15.504 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-21T08:31:15.504 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-04-21T08:31:15.504 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-04-21T08:31:15.505 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113673 s, 922 MB/s 2024-04-21T08:31:15.505 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-04-21T08:31:15.874 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 21 08:31:15 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-1[35682]: 2024-04-21T08:31:15.570+0000 7f3be5ae7700 -1 osd.1 0 waiting for initial osdmap 2024-04-21T08:31:15.874 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 21 08:31:15 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-1[35682]: 2024-04-21T08:31:15.574+0000 7f3bdd8d2700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:31:16.044 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:15 smithi175 ceph-mon[31920]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-21T08:31:16.045 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:15 smithi175 ceph-mon[31920]: osdmap e12: 2 total, 1 up, 2 in 2024-04-21T08:31:16.045 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:16.045 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:16.045 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:15 smithi175 ceph-mon[31920]: from='osd.1 ' entity='osd.1' 2024-04-21T08:31:16.046 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi175:vg_nvme/lv_4 2024-04-21T08:31:16.286 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:31:16.293 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:15 smithi112 ceph-mon[31085]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-21T08:31:16.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:15 smithi112 ceph-mon[31085]: osdmap e12: 2 total, 1 up, 2 in 2024-04-21T08:31:16.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:16.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:16.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:15 smithi112 ceph-mon[31085]: from='osd.1 ' entity='osd.1' 2024-04-21T08:31:16.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:15 smithi043 ceph-mon[27978]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-21T08:31:16.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:15 smithi043 ceph-mon[27978]: osdmap e12: 2 total, 1 up, 2 in 2024-04-21T08:31:16.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:16.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:16.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:15 smithi043 ceph-mon[27978]: from='osd.1 ' entity='osd.1' 2024-04-21T08:31:17.089 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:16 smithi175 ceph-mon[31920]: purged_snaps scrub starts 2024-04-21T08:31:17.090 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:16 smithi175 ceph-mon[31920]: purged_snaps scrub ok 2024-04-21T08:31:17.090 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:16 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:17.090 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:16 smithi175 ceph-mon[31920]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:17.090 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:16 smithi175 ceph-mon[31920]: Detected new or changed devices on smithi112 2024-04-21T08:31:17.090 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:16 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:17.090 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:16 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:17.090 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:16 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:31:17.090 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:16 smithi175 ceph-mon[31920]: Adjusting osd_memory_target on smithi112 to 17088M 2024-04-21T08:31:17.090 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:16 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:17.090 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:16 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:17.090 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:16 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:31:17.090 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:16 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:17.091 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:16 smithi175 ceph-mon[31920]: osd.1 [v2:172.21.15.112:6800/2362092039,v1:172.21.15.112:6801/2362092039] boot 2024-04-21T08:31:17.091 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:16 smithi175 ceph-mon[31920]: osdmap e13: 2 total, 2 up, 2 in 2024-04-21T08:31:17.091 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:16 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:17.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:16 smithi112 ceph-mon[31085]: purged_snaps scrub starts 2024-04-21T08:31:17.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:16 smithi112 ceph-mon[31085]: purged_snaps scrub ok 2024-04-21T08:31:17.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:16 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:17.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:16 smithi112 ceph-mon[31085]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:17.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:16 smithi112 ceph-mon[31085]: Detected new or changed devices on smithi112 2024-04-21T08:31:17.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:16 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:17.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:16 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:17.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:16 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:31:17.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:16 smithi112 ceph-mon[31085]: Adjusting osd_memory_target on smithi112 to 17088M 2024-04-21T08:31:17.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:16 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:17.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:16 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:17.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:16 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:31:17.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:16 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:17.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:16 smithi112 ceph-mon[31085]: osd.1 [v2:172.21.15.112:6800/2362092039,v1:172.21.15.112:6801/2362092039] boot 2024-04-21T08:31:17.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:16 smithi112 ceph-mon[31085]: osdmap e13: 2 total, 2 up, 2 in 2024-04-21T08:31:17.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:16 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:17.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:16 smithi043 ceph-mon[27978]: purged_snaps scrub starts 2024-04-21T08:31:17.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:16 smithi043 ceph-mon[27978]: purged_snaps scrub ok 2024-04-21T08:31:17.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:16 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:17.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:16 smithi043 ceph-mon[27978]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:17.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:16 smithi043 ceph-mon[27978]: Detected new or changed devices on smithi112 2024-04-21T08:31:17.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:16 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:17.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:16 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:17.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:16 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:31:17.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:16 smithi043 ceph-mon[27978]: Adjusting osd_memory_target on smithi112 to 17088M 2024-04-21T08:31:17.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:16 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:17.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:16 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:17.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:16 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:31:17.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:16 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:17.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:16 smithi043 ceph-mon[27978]: osd.1 [v2:172.21.15.112:6800/2362092039,v1:172.21.15.112:6801/2362092039] boot 2024-04-21T08:31:17.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:16 smithi043 ceph-mon[27978]: osdmap e13: 2 total, 2 up, 2 in 2024-04-21T08:31:17.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:16 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:17.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.905+0000 7fd31cfd6700 1 -- 172.21.15.175:0/4189438357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd318102aa0 msgr2=0x7fd318102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:31:17.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.905+0000 7fd31cfd6700 1 --2- 172.21.15.175:0/4189438357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd318102aa0 0x7fd318102ec0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd308009990 tx=0x7fd308009ca0 comp rx=0 tx=0).stop 2024-04-21T08:31:17.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.905+0000 7fd31cfd6700 1 -- 172.21.15.175:0/4189438357 shutdown_connections 2024-04-21T08:31:17.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.905+0000 7fd31cfd6700 1 --2- 172.21.15.175:0/4189438357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd318102aa0 0x7fd318102ec0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:17.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.905+0000 7fd31cfd6700 1 --2- 172.21.15.175:0/4189438357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd318104640 0x7fd31810aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:17.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.905+0000 7fd31cfd6700 1 --2- 172.21.15.175:0/4189438357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd318103ca0 0x7fd318104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:17.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.905+0000 7fd31cfd6700 1 -- 172.21.15.175:0/4189438357 >> 172.21.15.175:0/4189438357 conn(0x7fd3180fe040 msgr2=0x7fd318100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:31:17.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.905+0000 7fd31cfd6700 1 -- 172.21.15.175:0/4189438357 shutdown_connections 2024-04-21T08:31:17.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.905+0000 7fd31cfd6700 1 -- 172.21.15.175:0/4189438357 wait complete. 2024-04-21T08:31:17.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.906+0000 7fd31cfd6700 1 Processor -- start 2024-04-21T08:31:17.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.906+0000 7fd31cfd6700 1 -- start start 2024-04-21T08:31:17.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.906+0000 7fd31cfd6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd318102aa0 0x7fd318198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:31:17.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.906+0000 7fd31cfd6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd318103ca0 0x7fd318199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:31:17.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.906+0000 7fd31cfd6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd318104640 0x7fd3181a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:31:17.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.906+0000 7fd31cfd6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd31810e590 con 0x7fd318102aa0 2024-04-21T08:31:17.908 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.906+0000 7fd31cfd6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd31810e410 con 0x7fd318103ca0 2024-04-21T08:31:17.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.906+0000 7fd31cfd6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd31810e710 con 0x7fd318104640 2024-04-21T08:31:17.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.906+0000 7fd316d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd318104640 0x7fd3181a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:31:17.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.906+0000 7fd316d9d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd318104640 0x7fd3181a0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:43576/0 (socket says 172.21.15.175:43576) 2024-04-21T08:31:17.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.907+0000 7fd316d9d700 1 -- 172.21.15.175:0/3151497713 learned_addr learned my addr 172.21.15.175:0/3151497713 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-21T08:31:17.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.907+0000 7fd31659c700 1 --2- 172.21.15.175:0/3151497713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd318102aa0 0x7fd318198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:31:17.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.907+0000 7fd315d9b700 1 --2- 172.21.15.175:0/3151497713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd318103ca0 0x7fd318199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:31:17.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.907+0000 7fd316d9d700 1 -- 172.21.15.175:0/3151497713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd318103ca0 msgr2=0x7fd318199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:31:17.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.907+0000 7fd316d9d700 1 --2- 172.21.15.175:0/3151497713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd318103ca0 0x7fd318199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:17.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.907+0000 7fd316d9d700 1 -- 172.21.15.175:0/3151497713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd318102aa0 msgr2=0x7fd318198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:31:17.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.907+0000 7fd316d9d700 1 --2- 172.21.15.175:0/3151497713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd318102aa0 0x7fd318198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:17.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.907+0000 7fd316d9d700 1 -- 172.21.15.175:0/3151497713 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3181a0900 con 0x7fd318104640 2024-04-21T08:31:17.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.907+0000 7fd316d9d700 1 --2- 172.21.15.175:0/3151497713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd318104640 0x7fd3181a0230 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd31000ada0 tx=0x7fd31000c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:31:17.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.907+0000 7fd3077fe700 1 -- 172.21.15.175:0/3151497713 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310017020 con 0x7fd318104640 2024-04-21T08:31:17.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.907+0000 7fd31cfd6700 1 -- 172.21.15.175:0/3151497713 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3181a0bf0 con 0x7fd318104640 2024-04-21T08:31:17.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.907+0000 7fd3077fe700 1 -- 172.21.15.175:0/3151497713 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd31000ce30 con 0x7fd318104640 2024-04-21T08:31:17.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.907+0000 7fd3077fe700 1 -- 172.21.15.175:0/3151497713 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310005350 con 0x7fd318104640 2024-04-21T08:31:17.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.907+0000 7fd31cfd6700 1 -- 172.21.15.175:0/3151497713 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3181a10e0 con 0x7fd318104640 2024-04-21T08:31:17.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.908+0000 7fd31cfd6700 1 -- 172.21.15.175:0/3151497713 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd318069730 con 0x7fd318104640 2024-04-21T08:31:17.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.909+0000 7fd3077fe700 1 -- 172.21.15.175:0/3151497713 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fd31000c980 con 0x7fd318104640 2024-04-21T08:31:17.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.910+0000 7fd3077fe700 1 --2- 172.21.15.175:0/3151497713 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd3000a3bb0 0x7fd3000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:31:17.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.910+0000 7fd3077fe700 1 -- 172.21.15.175:0/3151497713 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fd3100ce2d0 con 0x7fd318104640 2024-04-21T08:31:17.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.910+0000 7fd31659c700 1 --2- 172.21.15.175:0/3151497713 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd3000a3bb0 0x7fd3000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:31:17.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.910+0000 7fd31659c700 1 --2- 172.21.15.175:0/3151497713 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd3000a3bb0 0x7fd3000a6070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd308003790 tx=0x7fd308005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:31:17.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:17.915+0000 7fd3077fe700 1 -- 172.21.15.175:0/3151497713 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd310021020 con 0x7fd318104640 2024-04-21T08:31:18.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:18.122+0000 7fd31cfd6700 1 -- 172.21.15.175:0/3151497713 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd318062e80 con 0x7fd3000a3bb0 2024-04-21T08:31:19.178 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:18 smithi175 ceph-mon[31920]: osdmap e14: 2 total, 2 up, 2 in 2024-04-21T08:31:19.179 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:18 smithi175 ceph-mon[31920]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:19.179 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:18 smithi175 ceph-mon[31920]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:31:19.179 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:18 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:31:19.179 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:18 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:31:19.179 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:18 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:19.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:18 smithi112 ceph-mon[31085]: osdmap e14: 2 total, 2 up, 2 in 2024-04-21T08:31:19.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:18 smithi112 ceph-mon[31085]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:19.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:18 smithi112 ceph-mon[31085]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:31:19.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:18 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:31:19.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:18 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:31:19.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:18 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:19.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:18 smithi043 ceph-mon[27978]: osdmap e14: 2 total, 2 up, 2 in 2024-04-21T08:31:19.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:18 smithi043 ceph-mon[27978]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:19.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:18 smithi043 ceph-mon[27978]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:31:19.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:18 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:31:19.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:18 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:31:19.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:18 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:21.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:20 smithi112 ceph-mon[31085]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:21.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:20 smithi112 ceph-mon[31085]: from='client.? 172.21.15.175:0/1456740222' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca0c4942-852b-415e-9a75-8645f0435202"}]: dispatch 2024-04-21T08:31:21.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:20 smithi112 ceph-mon[31085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca0c4942-852b-415e-9a75-8645f0435202"}]: dispatch 2024-04-21T08:31:21.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:20 smithi112 ceph-mon[31085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca0c4942-852b-415e-9a75-8645f0435202"}]': finished 2024-04-21T08:31:21.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:20 smithi112 ceph-mon[31085]: osdmap e15: 3 total, 2 up, 3 in 2024-04-21T08:31:21.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:20 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:21.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:20 smithi043 ceph-mon[27978]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:21.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:20 smithi043 ceph-mon[27978]: from='client.? 172.21.15.175:0/1456740222' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca0c4942-852b-415e-9a75-8645f0435202"}]: dispatch 2024-04-21T08:31:21.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:20 smithi043 ceph-mon[27978]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca0c4942-852b-415e-9a75-8645f0435202"}]: dispatch 2024-04-21T08:31:21.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:20 smithi043 ceph-mon[27978]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca0c4942-852b-415e-9a75-8645f0435202"}]': finished 2024-04-21T08:31:21.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:20 smithi043 ceph-mon[27978]: osdmap e15: 3 total, 2 up, 3 in 2024-04-21T08:31:21.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:20 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:21.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:20 smithi175 ceph-mon[31920]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:21.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:20 smithi175 ceph-mon[31920]: from='client.? 172.21.15.175:0/1456740222' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca0c4942-852b-415e-9a75-8645f0435202"}]: dispatch 2024-04-21T08:31:21.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:20 smithi175 ceph-mon[31920]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca0c4942-852b-415e-9a75-8645f0435202"}]: dispatch 2024-04-21T08:31:21.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:20 smithi175 ceph-mon[31920]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca0c4942-852b-415e-9a75-8645f0435202"}]': finished 2024-04-21T08:31:21.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:20 smithi175 ceph-mon[31920]: osdmap e15: 3 total, 2 up, 3 in 2024-04-21T08:31:21.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:20 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:22.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:21 smithi112 ceph-mon[31085]: from='client.? 172.21.15.175:0/1162590304' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:31:22.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:21 smithi043 ceph-mon[27978]: from='client.? 172.21.15.175:0/1162590304' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:31:22.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:21 smithi175 ceph-mon[31920]: from='client.? 172.21.15.175:0/1162590304' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:31:23.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:22 smithi112 ceph-mon[31085]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:23.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:22 smithi043 ceph-mon[27978]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:23.365 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:22 smithi175 ceph-mon[31920]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:25.228 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:24 smithi112 ceph-mon[31085]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:25.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:24 smithi043 ceph-mon[27978]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:25.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:24 smithi175 ceph-mon[31920]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:27.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:26 smithi112 ceph-mon[31085]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:27.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:26 smithi043 ceph-mon[27978]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:27.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:26 smithi175 ceph-mon[31920]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:28.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:27 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-21T08:31:28.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:27 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:28.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:27 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-21T08:31:28.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:27 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:28.418 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:27 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-21T08:31:28.418 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:27 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:29.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:28 smithi112 ceph-mon[31085]: Deploying daemon osd.2 on smithi175 2024-04-21T08:31:29.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:28 smithi112 ceph-mon[31085]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:29.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:28 smithi043 ceph-mon[27978]: Deploying daemon osd.2 on smithi175 2024-04-21T08:31:29.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:28 smithi043 ceph-mon[27978]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:29.347 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:28 smithi175 ceph-mon[31920]: Deploying daemon osd.2 on smithi175 2024-04-21T08:31:29.347 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:28 smithi175 ceph-mon[31920]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:31.277 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:30 smithi175 ceph-mon[31920]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:31.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:30 smithi112 ceph-mon[31085]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:31.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:30 smithi043 ceph-mon[27978]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:33.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:32 smithi112 ceph-mon[31085]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:33.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:32 smithi043 ceph-mon[27978]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:33.412 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:32 smithi175 ceph-mon[31920]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:34.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:34 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:34.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:34 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:34.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:34 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:34.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:34 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:34.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:34 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:34.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:34 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:34.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:34 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:34.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:34 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:34.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:34 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:35.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:35 smithi175 ceph-mon[31920]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:35.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:35 smithi043 ceph-mon[27978]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:35.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:35 smithi112 ceph-mon[31085]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:36.495 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 2 on host 'smithi175' 2024-04-21T08:31:36.495 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:36.495+0000 7fd3077fe700 1 -- 172.21.15.175:0/3151497713 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd318062e80 con 0x7fd3000a3bb0 2024-04-21T08:31:36.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:36.497+0000 7fd31cfd6700 1 -- 172.21.15.175:0/3151497713 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd3000a3bb0 msgr2=0x7fd3000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:31:36.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:36.497+0000 7fd31cfd6700 1 --2- 172.21.15.175:0/3151497713 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd3000a3bb0 0x7fd3000a6070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd308003790 tx=0x7fd308005fb0 comp rx=0 tx=0).stop 2024-04-21T08:31:36.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:36.498+0000 7fd31cfd6700 1 -- 172.21.15.175:0/3151497713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd318104640 msgr2=0x7fd3181a0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:31:36.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:36.498+0000 7fd31cfd6700 1 --2- 172.21.15.175:0/3151497713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd318104640 0x7fd3181a0230 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd31000ada0 tx=0x7fd31000c330 comp rx=0 tx=0).stop 2024-04-21T08:31:36.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:36.498+0000 7fd31cfd6700 1 -- 172.21.15.175:0/3151497713 shutdown_connections 2024-04-21T08:31:36.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:36.498+0000 7fd31cfd6700 1 --2- 172.21.15.175:0/3151497713 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd3000a3bb0 0x7fd3000a6070 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:36.497 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:36.498+0000 7fd31cfd6700 1 --2- 172.21.15.175:0/3151497713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd318104640 0x7fd3181a0230 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:36.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:36.498+0000 7fd31cfd6700 1 --2- 172.21.15.175:0/3151497713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd318102aa0 0x7fd318198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:36.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:36.498+0000 7fd31cfd6700 1 --2- 172.21.15.175:0/3151497713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd318103ca0 0x7fd318199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:36.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:36.498+0000 7fd31cfd6700 1 -- 172.21.15.175:0/3151497713 >> 172.21.15.175:0/3151497713 conn(0x7fd3180fe040 msgr2=0x7fd3181070e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:31:36.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:36.498+0000 7fd31cfd6700 1 -- 172.21.15.175:0/3151497713 shutdown_connections 2024-04-21T08:31:36.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:31:36.498+0000 7fd31cfd6700 1 -- 172.21.15.175:0/3151497713 wait complete. 2024-04-21T08:31:37.132 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:36 smithi175 ceph-mon[31920]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:37.132 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:36 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:37.132 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:36 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:37.132 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:36 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:37.132 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:36 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:31:37.132 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:36 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:37.132 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:36 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:37.132 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:36 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:37.132 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:36 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:37.156 DEBUG:teuthology.orchestra.run.smithi175:osd.2> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.2.service 2024-04-21T08:31:37.159 INFO:tasks.cephadm:Deploying osd.3 on smithi043 with /dev/vg_nvme/lv_3... 2024-04-21T08:31:37.159 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-21T08:31:37.186 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:36 smithi043 ceph-mon[27978]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:37.186 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:36 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:37.186 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:36 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:37.186 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:36 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:37.186 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:36 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:31:37.186 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:36 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:37.186 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:36 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:37.186 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:36 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:37.186 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:36 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:37.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:36 smithi112 ceph-mon[31085]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:37.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:36 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:37.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:36 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:37.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:36 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:37.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:36 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:31:37.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:36 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:37.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:36 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:37.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:36 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:37.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:36 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:37.414 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:31:38.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:37 smithi175 ceph-mon[31920]: from='osd.2 [v2:172.21.15.175:6800/2080591921,v1:172.21.15.175:6801/2080591921]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T08:31:38.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:37 smithi175 ceph-mon[31920]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T08:31:38.277 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:37 smithi043 ceph-mon[27978]: from='osd.2 [v2:172.21.15.175:6800/2080591921,v1:172.21.15.175:6801/2080591921]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T08:31:38.277 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:37 smithi043 ceph-mon[27978]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T08:31:38.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:37 smithi112 ceph-mon[31085]: from='osd.2 [v2:172.21.15.175:6800/2080591921,v1:172.21.15.175:6801/2080591921]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T08:31:38.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:37 smithi112 ceph-mon[31085]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T08:31:39.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:38 smithi175 ceph-mon[31920]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:39.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:38 smithi175 ceph-mon[31920]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-21T08:31:39.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:38 smithi175 ceph-mon[31920]: osdmap e16: 3 total, 2 up, 3 in 2024-04-21T08:31:39.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:38 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:39.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:38 smithi175 ceph-mon[31920]: from='osd.2 [v2:172.21.15.175:6800/2080591921,v1:172.21.15.175:6801/2080591921]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-04-21T08:31:39.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:38 smithi175 ceph-mon[31920]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-04-21T08:31:39.214 INFO:journalctl@ceph.osd.2.smithi175.stdout:Apr 21 08:31:38 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-2[37058]: 2024-04-21T08:31:38.949+0000 7fc80c917700 -1 osd.2 0 waiting for initial osdmap 2024-04-21T08:31:39.214 INFO:journalctl@ceph.osd.2.smithi175.stdout:Apr 21 08:31:38 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-2[37058]: 2024-04-21T08:31:38.953+0000 7fc80870a700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:31:39.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:38 smithi112 ceph-mon[31085]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:39.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:38 smithi112 ceph-mon[31085]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-21T08:31:39.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:38 smithi112 ceph-mon[31085]: osdmap e16: 3 total, 2 up, 3 in 2024-04-21T08:31:39.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:38 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:39.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:38 smithi112 ceph-mon[31085]: from='osd.2 [v2:172.21.15.175:6800/2080591921,v1:172.21.15.175:6801/2080591921]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-04-21T08:31:39.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:38 smithi112 ceph-mon[31085]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-04-21T08:31:39.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:38 smithi043 ceph-mon[27978]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:39.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:38 smithi043 ceph-mon[27978]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-21T08:31:39.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:38 smithi043 ceph-mon[27978]: osdmap e16: 3 total, 2 up, 3 in 2024-04-21T08:31:39.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:38 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:39.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:38 smithi043 ceph-mon[27978]: from='osd.2 [v2:172.21.15.175:6800/2080591921,v1:172.21.15.175:6801/2080591921]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-04-21T08:31:39.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:38 smithi043 ceph-mon[27978]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-04-21T08:31:39.459 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-21T08:31:39.459 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-21T08:31:39.459 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-04-21T08:31:39.459 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-04-21T08:31:39.459 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00856342 s, 1.2 GB/s 2024-04-21T08:31:39.459 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-04-21T08:31:40.095 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:39 smithi175 ceph-mon[31920]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-04-21T08:31:40.095 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:39 smithi175 ceph-mon[31920]: osdmap e17: 3 total, 2 up, 3 in 2024-04-21T08:31:40.095 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:39 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:40.095 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:39 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:40.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_3 2024-04-21T08:31:40.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:39 smithi112 ceph-mon[31085]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-04-21T08:31:40.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:39 smithi112 ceph-mon[31085]: osdmap e17: 3 total, 2 up, 3 in 2024-04-21T08:31:40.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:39 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:40.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:39 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:40.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:39 smithi043 ceph-mon[27978]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-04-21T08:31:40.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:39 smithi043 ceph-mon[27978]: osdmap e17: 3 total, 2 up, 3 in 2024-04-21T08:31:40.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:39 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:40.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:39 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:40.397 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:31:41.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:40 smithi043 ceph-mon[27978]: purged_snaps scrub starts 2024-04-21T08:31:41.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:40 smithi043 ceph-mon[27978]: purged_snaps scrub ok 2024-04-21T08:31:41.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:40 smithi043 ceph-mon[27978]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:41.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:40 smithi043 ceph-mon[27978]: osd.2 [v2:172.21.15.175:6800/2080591921,v1:172.21.15.175:6801/2080591921] boot 2024-04-21T08:31:41.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:40 smithi043 ceph-mon[27978]: osdmap e18: 3 total, 3 up, 3 in 2024-04-21T08:31:41.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:40 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:41.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:40 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:31:41.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:40 smithi043 ceph-mon[27978]: Detected new or changed devices on smithi175 2024-04-21T08:31:41.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:40 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:41.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:40 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:41.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:40 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:31:41.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:40 smithi043 ceph-mon[27978]: Adjusting osd_memory_target on smithi175 to 17088M 2024-04-21T08:31:41.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:40 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:41.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:40 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:41.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:40 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:31:41.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:40 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:41.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:40 smithi175 ceph-mon[31920]: purged_snaps scrub starts 2024-04-21T08:31:41.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:40 smithi175 ceph-mon[31920]: purged_snaps scrub ok 2024-04-21T08:31:41.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:40 smithi175 ceph-mon[31920]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:41.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:40 smithi175 ceph-mon[31920]: osd.2 [v2:172.21.15.175:6800/2080591921,v1:172.21.15.175:6801/2080591921] boot 2024-04-21T08:31:41.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:40 smithi175 ceph-mon[31920]: osdmap e18: 3 total, 3 up, 3 in 2024-04-21T08:31:41.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:40 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:41.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:40 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:31:41.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:40 smithi175 ceph-mon[31920]: Detected new or changed devices on smithi175 2024-04-21T08:31:41.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:40 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:41.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:40 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:41.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:40 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:31:41.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:40 smithi175 ceph-mon[31920]: Adjusting osd_memory_target on smithi175 to 17088M 2024-04-21T08:31:41.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:40 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:41.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:40 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:41.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:40 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:31:41.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:40 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:41.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:40 smithi112 ceph-mon[31085]: purged_snaps scrub starts 2024-04-21T08:31:41.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:40 smithi112 ceph-mon[31085]: purged_snaps scrub ok 2024-04-21T08:31:41.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:40 smithi112 ceph-mon[31085]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:41.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:40 smithi112 ceph-mon[31085]: osd.2 [v2:172.21.15.175:6800/2080591921,v1:172.21.15.175:6801/2080591921] boot 2024-04-21T08:31:41.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:40 smithi112 ceph-mon[31085]: osdmap e18: 3 total, 3 up, 3 in 2024-04-21T08:31:41.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:40 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:41.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:40 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:31:41.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:40 smithi112 ceph-mon[31085]: Detected new or changed devices on smithi175 2024-04-21T08:31:41.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:40 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:41.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:40 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:41.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:40 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:31:41.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:40 smithi112 ceph-mon[31085]: Adjusting osd_memory_target on smithi175 to 17088M 2024-04-21T08:31:41.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:40 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:41.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:40 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:41.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:40 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:31:41.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:40 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:42.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.147+0000 7f6ee4e35700 1 -- 172.21.15.43:0/3228987377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ee01046d0 msgr2=0x7f6ee010af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:31:42.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.147+0000 7f6ee4e35700 1 --2- 172.21.15.43:0/3228987377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ee01046d0 0x7f6ee010af80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6ed800b0a0 tx=0x7f6ed800b3b0 comp rx=0 tx=0).stop 2024-04-21T08:31:42.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.147+0000 7f6ee4e35700 1 -- 172.21.15.43:0/3228987377 shutdown_connections 2024-04-21T08:31:42.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.147+0000 7f6ee4e35700 1 --2- 172.21.15.43:0/3228987377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ee01046d0 0x7f6ee010af80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:42.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.147+0000 7f6ee4e35700 1 --2- 172.21.15.43:0/3228987377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ee0100d50 0x7f6ee0101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:42.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.147+0000 7f6ee4e35700 1 --2- 172.21.15.43:0/3228987377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ee0103d30 0x7f6ee0104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:42.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.147+0000 7f6ee4e35700 1 -- 172.21.15.43:0/3228987377 >> 172.21.15.43:0/3228987377 conn(0x7f6ee00fc6d0 msgr2=0x7f6ee00feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:31:42.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.147+0000 7f6ee4e35700 1 -- 172.21.15.43:0/3228987377 shutdown_connections 2024-04-21T08:31:42.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.147+0000 7f6ee4e35700 1 -- 172.21.15.43:0/3228987377 wait complete. 2024-04-21T08:31:42.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.148+0000 7f6ee4e35700 1 Processor -- start 2024-04-21T08:31:42.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.148+0000 7f6ee4e35700 1 -- start start 2024-04-21T08:31:42.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.148+0000 7f6ee4e35700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ee0100d50 0x7f6ee0103690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:31:42.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.148+0000 7f6ee4e35700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ee0103d30 0x7f6ee0101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:31:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.148+0000 7f6ee4e35700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ee01046d0 0x7f6ee0102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:31:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.148+0000 7f6ee4e35700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee010e590 con 0x7f6ee0100d50 2024-04-21T08:31:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.148+0000 7f6ee4e35700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee010e410 con 0x7f6ee01046d0 2024-04-21T08:31:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.148+0000 7f6ee4e35700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee010e710 con 0x7f6ee0103d30 2024-04-21T08:31:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.148+0000 7f6ede59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ee0100d50 0x7f6ee0103690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:31:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.148+0000 7f6ede59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ee0100d50 0x7f6ee0103690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46432/0 (socket says 172.21.15.43:46432) 2024-04-21T08:31:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.149+0000 7f6ede59c700 1 -- 172.21.15.43:0/4047480216 learned_addr learned my addr 172.21.15.43:0/4047480216 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:31:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.149+0000 7f6eded9d700 1 --2- 172.21.15.43:0/4047480216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ee01046d0 0x7f6ee0102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:31:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.149+0000 7f6eddd9b700 1 --2- 172.21.15.43:0/4047480216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ee0103d30 0x7f6ee0101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:31:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.149+0000 7f6ede59c700 1 -- 172.21.15.43:0/4047480216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ee0103d30 msgr2=0x7f6ee0101ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:31:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.149+0000 7f6ede59c700 1 --2- 172.21.15.43:0/4047480216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ee0103d30 0x7f6ee0101ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.149+0000 7f6ede59c700 1 -- 172.21.15.43:0/4047480216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ee01046d0 msgr2=0x7f6ee0102220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:31:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.149+0000 7f6ede59c700 1 --2- 172.21.15.43:0/4047480216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ee01046d0 0x7f6ee0102220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:42.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.149+0000 7f6ede59c700 1 -- 172.21.15.43:0/4047480216 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee0102ad0 con 0x7f6ee0100d50 2024-04-21T08:31:42.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.149+0000 7f6eddd9b700 1 --2- 172.21.15.43:0/4047480216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ee0103d30 0x7f6ee0101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:31:42.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.149+0000 7f6ede59c700 1 --2- 172.21.15.43:0/4047480216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ee0100d50 0x7f6ee0103690 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000e970 tx=0x7f6ed000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:31:42.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.149+0000 7f6ecf7fe700 1 -- 172.21.15.43:0/4047480216 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed000cbd0 con 0x7f6ee0100d50 2024-04-21T08:31:42.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.149+0000 7f6ee4e35700 1 -- 172.21.15.43:0/4047480216 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee01a2d30 con 0x7f6ee0100d50 2024-04-21T08:31:42.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.149+0000 7f6ecf7fe700 1 -- 172.21.15.43:0/4047480216 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ed00044d0 con 0x7f6ee0100d50 2024-04-21T08:31:42.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.149+0000 7f6ecf7fe700 1 -- 172.21.15.43:0/4047480216 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0003df0 con 0x7f6ee0100d50 2024-04-21T08:31:42.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.149+0000 7f6ee4e35700 1 -- 172.21.15.43:0/4047480216 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee01a3210 con 0x7f6ee0100d50 2024-04-21T08:31:42.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.150+0000 7f6ee4e35700 1 -- 172.21.15.43:0/4047480216 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee002df80 con 0x7f6ee0100d50 2024-04-21T08:31:42.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.152+0000 7f6ecf7fe700 1 -- 172.21.15.43:0/4047480216 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f6ed0003f90 con 0x7f6ee0100d50 2024-04-21T08:31:42.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.152+0000 7f6ecf7fe700 1 --2- 172.21.15.43:0/4047480216 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f6ec80a3bb0 0x7f6ec80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:31:42.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.152+0000 7f6eddd9b700 1 --2- 172.21.15.43:0/4047480216 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f6ec80a3bb0 0x7f6ec80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:31:42.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.152+0000 7f6ecf7fe700 1 -- 172.21.15.43:0/4047480216 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7f6ed00cf260 con 0x7f6ee0100d50 2024-04-21T08:31:42.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.153+0000 7f6eddd9b700 1 --2- 172.21.15.43:0/4047480216 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f6ec80a3bb0 0x7f6ec80a6070 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4006fd0 tx=0x7f6ed4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:31:42.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.156+0000 7f6ecf7fe700 1 -- 172.21.15.43:0/4047480216 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ed009d180 con 0x7f6ee0100d50 2024-04-21T08:31:42.218 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:41 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-21T08:31:42.218 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:41 smithi175 ceph-mon[31920]: osdmap e19: 3 total, 3 up, 3 in 2024-04-21T08:31:42.218 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:41 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:31:42.278 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:41 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-21T08:31:42.278 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:41 smithi043 ceph-mon[27978]: osdmap e19: 3 total, 3 up, 3 in 2024-04-21T08:31:42.278 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:41 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:31:42.279 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 21 08:31:42 smithi043 sudo[44589]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T08:31:42.279 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 21 08:31:42 smithi043 sudo[44589]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:31:42.279 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 21 08:31:42 smithi043 sudo[44589]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:31:42.279 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 21 08:31:42 smithi043 sudo[44589]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:31:42.279 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 21 08:31:42 smithi043 sudo[44592]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T08:31:42.279 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 21 08:31:42 smithi043 sudo[44592]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:31:42.279 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 21 08:31:42 smithi043 sudo[44592]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:31:42.279 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 21 08:31:42 smithi043 sudo[44592]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:31:42.293 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:41 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-21T08:31:42.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:41 smithi112 ceph-mon[31085]: osdmap e19: 3 total, 3 up, 3 in 2024-04-21T08:31:42.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:41 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:31:42.294 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 21 08:31:42 smithi112 sudo[39505]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T08:31:42.294 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 21 08:31:42 smithi112 sudo[39505]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:31:42.294 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 21 08:31:42 smithi112 sudo[39505]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:31:42.294 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 21 08:31:42 smithi112 sudo[39505]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:31:42.294 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 21 08:31:42 smithi112 sudo[39508]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T08:31:42.294 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 21 08:31:42 smithi112 sudo[39508]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:31:42.294 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 21 08:31:42 smithi112 sudo[39508]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:31:42.294 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 21 08:31:42 smithi112 sudo[39508]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:31:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:42.389+0000 7f6ee4e35700 1 -- 172.21.15.43:0/4047480216 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6ee0062e80 con 0x7f6ec80a3bb0 2024-04-21T08:31:42.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 sudo[44615]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T08:31:42.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 sudo[44615]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:31:42.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 sudo[44615]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:31:42.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 sudo[44615]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:31:42.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 sudo[44618]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T08:31:42.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 sudo[44618]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:31:42.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 sudo[44618]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:31:42.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 sudo[44618]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:31:42.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 sudo[41124]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T08:31:42.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 sudo[41124]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:31:42.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 sudo[41124]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:31:42.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 sudo[41124]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:31:42.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 sudo[41127]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T08:31:42.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 sudo[41127]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:31:42.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 sudo[41127]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:31:42.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 sudo[41127]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:31:42.714 INFO:journalctl@ceph.osd.2.smithi175.stdout:Apr 21 08:31:42 smithi175 sudo[41118]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T08:31:42.715 INFO:journalctl@ceph.osd.2.smithi175.stdout:Apr 21 08:31:42 smithi175 sudo[41118]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:31:42.715 INFO:journalctl@ceph.osd.2.smithi175.stdout:Apr 21 08:31:42 smithi175 sudo[41118]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:31:42.715 INFO:journalctl@ceph.osd.2.smithi175.stdout:Apr 21 08:31:42 smithi175 sudo[41118]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:31:42.715 INFO:journalctl@ceph.osd.2.smithi175.stdout:Apr 21 08:31:42 smithi175 sudo[41121]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T08:31:42.715 INFO:journalctl@ceph.osd.2.smithi175.stdout:Apr 21 08:31:42 smithi175 sudo[41121]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:31:42.715 INFO:journalctl@ceph.osd.2.smithi175.stdout:Apr 21 08:31:42 smithi175 sudo[41121]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:31:42.715 INFO:journalctl@ceph.osd.2.smithi175.stdout:Apr 21 08:31:42 smithi175 sudo[41121]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:31:42.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 sudo[39511]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T08:31:42.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 sudo[39511]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:31:42.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 sudo[39511]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:31:42.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 sudo[39511]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:31:42.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 sudo[39514]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T08:31:42.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 sudo[39514]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:31:42.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 sudo[39514]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:31:42.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 sudo[39514]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:31:43.239 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:43.239 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T08:31:43.239 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-21T08:31:43.239 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: osdmap e20: 3 total, 3 up, 3 in 2024-04-21T08:31:43.239 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:43.239 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:43.239 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:43.239 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:31:43.240 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:31:43.240 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:31:43.240 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:31:43.240 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:31:43.240 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:31:43.240 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:31:43.240 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:31:43.240 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:31:43.240 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:31:43.240 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:31:43.240 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:31:43.240 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:31:43.240 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:31:43.240 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:31:43.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:31:43.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:31:43.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:31:43.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:42 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:43.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:43.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T08:31:43.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-21T08:31:43.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: osdmap e20: 3 total, 3 up, 3 in 2024-04-21T08:31:43.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:43.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:43.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:43.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:31:43.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:31:43.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:31:43.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:31:43.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:31:43.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:31:43.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:31:43.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:31:43.296 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:31:43.296 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:31:43.296 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:31:43.296 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:31:43.296 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:31:43.296 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:31:43.296 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:31:43.296 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:31:43.296 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:31:43.296 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:31:43.296 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:42 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:43.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:43.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T08:31:43.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-21T08:31:43.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: osdmap e20: 3 total, 3 up, 3 in 2024-04-21T08:31:43.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:43.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:43.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:43.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:31:43.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:31:43.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:31:43.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:31:43.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:31:43.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:31:43.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:31:43.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:31:43.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:31:43.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:31:43.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:31:43.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:31:43.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:31:43.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:31:43.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:31:43.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:31:43.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:31:43.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:31:43.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:42 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:43.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:31:43.988+0000 7f6ecf7fe700 1 -- 172.21.15.43:0/4047480216 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6ed00990f0 con 0x7f6ee0100d50 2024-04-21T08:31:44.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:43 smithi043 ceph-mon[27978]: osdmap e21: 3 total, 3 up, 3 in 2024-04-21T08:31:44.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:43 smithi112 ceph-mon[31085]: osdmap e21: 3 total, 3 up, 3 in 2024-04-21T08:31:44.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:43 smithi175 ceph-mon[31920]: osdmap e21: 3 total, 3 up, 3 in 2024-04-21T08:31:45.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:44 smithi112 ceph-mon[31085]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:45.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:44 smithi112 ceph-mon[31085]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-21T08:31:45.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:44 smithi112 ceph-mon[31085]: Cluster is now healthy 2024-04-21T08:31:45.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:44 smithi112 ceph-mon[31085]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-21T08:31:45.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:44 smithi043 ceph-mon[27978]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:45.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:44 smithi043 ceph-mon[27978]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-21T08:31:45.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:44 smithi043 ceph-mon[27978]: Cluster is now healthy 2024-04-21T08:31:45.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:44 smithi043 ceph-mon[27978]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-21T08:31:45.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:44 smithi175 ceph-mon[31920]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:45.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:44 smithi175 ceph-mon[31920]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-21T08:31:45.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:44 smithi175 ceph-mon[31920]: Cluster is now healthy 2024-04-21T08:31:45.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:44 smithi175 ceph-mon[31920]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-21T08:31:46.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:45 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3355977253' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2"}]: dispatch 2024-04-21T08:31:46.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:45 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3355977253' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2"}]': finished 2024-04-21T08:31:46.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:45 smithi112 ceph-mon[31085]: osdmap e22: 4 total, 3 up, 4 in 2024-04-21T08:31:46.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:45 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:31:46.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:45 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2977467180' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:31:46.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:45 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3355977253' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2"}]: dispatch 2024-04-21T08:31:46.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:45 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3355977253' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2"}]': finished 2024-04-21T08:31:46.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:45 smithi043 ceph-mon[27978]: osdmap e22: 4 total, 3 up, 4 in 2024-04-21T08:31:46.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:45 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:31:46.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:45 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2977467180' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:31:46.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:45 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3355977253' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2"}]: dispatch 2024-04-21T08:31:46.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:45 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3355977253' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2"}]': finished 2024-04-21T08:31:46.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:45 smithi175 ceph-mon[31920]: osdmap e22: 4 total, 3 up, 4 in 2024-04-21T08:31:46.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:45 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:31:46.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:45 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2977467180' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:31:47.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:47 smithi175 ceph-mon[31920]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:47.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:47 smithi112 ceph-mon[31085]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:47.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:47 smithi043 ceph-mon[27978]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:49.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:49 smithi175 ceph-mon[31920]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:49.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:49 smithi043 ceph-mon[27978]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:49.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:49 smithi112 ceph-mon[31085]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:51.377 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:51 smithi043 ceph-mon[27978]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:51.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:51 smithi175 ceph-mon[31920]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:51.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:51 smithi112 ceph-mon[31085]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:53.410 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:53 smithi043 ceph-mon[27978]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:53.410 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:53 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-21T08:31:53.410 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:53 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:53.410 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:53 smithi043 ceph-mon[27978]: Deploying daemon osd.3 on smithi043 2024-04-21T08:31:53.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:53 smithi175 ceph-mon[31920]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:53.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:53 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-21T08:31:53.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:53 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:53.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:53 smithi175 ceph-mon[31920]: Deploying daemon osd.3 on smithi043 2024-04-21T08:31:53.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:53 smithi112 ceph-mon[31085]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:53.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:53 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-21T08:31:53.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:53 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:53.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:53 smithi112 ceph-mon[31085]: Deploying daemon osd.3 on smithi043 2024-04-21T08:31:55.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:55 smithi175 ceph-mon[31920]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:55.518 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:55 smithi043 ceph-mon[27978]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:55.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:55 smithi112 ceph-mon[31085]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:57.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:57 smithi043 ceph-mon[27978]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:57.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:57 smithi175 ceph-mon[31920]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:57.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:57 smithi112 ceph-mon[31085]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:59.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:59 smithi112 ceph-mon[31085]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:59.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:59 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:59.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:59 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:59.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:31:59 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:59.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:59 smithi043 ceph-mon[27978]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:59.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:59 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:59.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:59 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:59.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:31:59 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:59.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:59 smithi175 ceph-mon[31920]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:59.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:59 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:59.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:59 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:31:59.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:31:59 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:01.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:01 smithi112 ceph-mon[31085]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:01.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:01 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:01.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:01 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:01.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:01 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:01.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:01 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:01.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:01 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:01.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:01 smithi043 ceph-mon[27978]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:01.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:01 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:01.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:01 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:01.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:01 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:01.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:01 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:01.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:01 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:01.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:01 smithi175 ceph-mon[31920]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:01.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:01 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:01.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:01 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:01.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:01 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:01.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:01 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:01.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:01 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:02.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:02.049+0000 7f6ecf7fe700 1 -- 172.21.15.43:0/4047480216 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6ee0062e80 con 0x7f6ec80a3bb0 2024-04-21T08:32:02.049 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 3 on host 'smithi043' 2024-04-21T08:32:02.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:02.052+0000 7f6ee4e35700 1 -- 172.21.15.43:0/4047480216 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f6ec80a3bb0 msgr2=0x7f6ec80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:02.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:02.052+0000 7f6ee4e35700 1 --2- 172.21.15.43:0/4047480216 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f6ec80a3bb0 0x7f6ec80a6070 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4006fd0 tx=0x7f6ed4008040 comp rx=0 tx=0).stop 2024-04-21T08:32:02.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:02.052+0000 7f6ee4e35700 1 -- 172.21.15.43:0/4047480216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ee0100d50 msgr2=0x7f6ee0103690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:02.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:02.052+0000 7f6ee4e35700 1 --2- 172.21.15.43:0/4047480216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ee0100d50 0x7f6ee0103690 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000e970 tx=0x7f6ed000ec80 comp rx=0 tx=0).stop 2024-04-21T08:32:02.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:02.052+0000 7f6ee4e35700 1 -- 172.21.15.43:0/4047480216 shutdown_connections 2024-04-21T08:32:02.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:02.052+0000 7f6ee4e35700 1 --2- 172.21.15.43:0/4047480216 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f6ec80a3bb0 0x7f6ec80a6070 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:02.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:02.052+0000 7f6ee4e35700 1 --2- 172.21.15.43:0/4047480216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ee01046d0 0x7f6ee0102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:02.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:02.052+0000 7f6ee4e35700 1 --2- 172.21.15.43:0/4047480216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ee0100d50 0x7f6ee0103690 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:02.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:02.052+0000 7f6ee4e35700 1 --2- 172.21.15.43:0/4047480216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ee0103d30 0x7f6ee0101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:02.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:02.052+0000 7f6ee4e35700 1 -- 172.21.15.43:0/4047480216 >> 172.21.15.43:0/4047480216 conn(0x7f6ee00fc6d0 msgr2=0x7f6ee0107080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:32:02.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:02.052+0000 7f6ee4e35700 1 -- 172.21.15.43:0/4047480216 shutdown_connections 2024-04-21T08:32:02.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:02.052+0000 7f6ee4e35700 1 -- 172.21.15.43:0/4047480216 wait complete. 2024-04-21T08:32:02.757 DEBUG:teuthology.orchestra.run.smithi043:osd.3> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.3.service 2024-04-21T08:32:02.760 INFO:tasks.cephadm:Deploying osd.4 on smithi112 with /dev/vg_nvme/lv_3... 2024-04-21T08:32:02.760 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-21T08:32:03.002 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:32:03.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:03 smithi112 ceph-mon[31085]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:03.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:03 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:03.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:03 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:03.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:03 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:03.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:03 smithi112 ceph-mon[31085]: from='osd.3 [v2:172.21.15.43:6810/117814050,v1:172.21.15.43:6811/117814050]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-21T08:32:03.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:03 smithi043 ceph-mon[27978]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:03.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:03 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:03.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:03 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:03.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:03 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:03.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:03 smithi043 ceph-mon[27978]: from='osd.3 [v2:172.21.15.43:6810/117814050,v1:172.21.15.43:6811/117814050]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-21T08:32:03.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:03 smithi175 ceph-mon[31920]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:03.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:03 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:03.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:03 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:03.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:03 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:03.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:03 smithi175 ceph-mon[31920]: from='osd.3 [v2:172.21.15.43:6810/117814050,v1:172.21.15.43:6811/117814050]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-21T08:32:04.277 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:04 smithi112 ceph-mon[31085]: from='osd.3 [v2:172.21.15.43:6810/117814050,v1:172.21.15.43:6811/117814050]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-21T08:32:04.278 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:04 smithi112 ceph-mon[31085]: osdmap e23: 4 total, 3 up, 4 in 2024-04-21T08:32:04.278 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:04 smithi112 ceph-mon[31085]: from='osd.3 [v2:172.21.15.43:6810/117814050,v1:172.21.15.43:6811/117814050]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-21T08:32:04.278 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:04 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:32:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:04 smithi043 ceph-mon[27978]: from='osd.3 [v2:172.21.15.43:6810/117814050,v1:172.21.15.43:6811/117814050]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-21T08:32:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:04 smithi043 ceph-mon[27978]: osdmap e23: 4 total, 3 up, 4 in 2024-04-21T08:32:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:04 smithi043 ceph-mon[27978]: from='osd.3 [v2:172.21.15.43:6810/117814050,v1:172.21.15.43:6811/117814050]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-21T08:32:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:04 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:32:04.311 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 21 08:32:04 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-3[46872]: 2024-04-21T08:32:04.030+0000 7fd5ddfb0700 -1 osd.3 0 waiting for initial osdmap 2024-04-21T08:32:04.311 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 21 08:32:04 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-3[46872]: 2024-04-21T08:32:04.036+0000 7fd5d9da3700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:32:04.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:04 smithi175 ceph-mon[31920]: from='osd.3 [v2:172.21.15.43:6810/117814050,v1:172.21.15.43:6811/117814050]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-21T08:32:04.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:04 smithi175 ceph-mon[31920]: osdmap e23: 4 total, 3 up, 4 in 2024-04-21T08:32:04.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:04 smithi175 ceph-mon[31920]: from='osd.3 [v2:172.21.15.43:6810/117814050,v1:172.21.15.43:6811/117814050]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-21T08:32:04.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:04 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:32:05.180 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-21T08:32:05.180 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-21T08:32:05.180 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-04-21T08:32:05.180 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-04-21T08:32:05.180 INFO:teuthology.orchestra.run.smithi112.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0133806 s, 784 MB/s 2024-04-21T08:32:05.180 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-04-21T08:32:05.278 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:05 smithi043 ceph-mon[27978]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:05.278 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:05 smithi043 ceph-mon[27978]: from='osd.3 [v2:172.21.15.43:6810/117814050,v1:172.21.15.43:6811/117814050]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-21T08:32:05.278 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:05 smithi043 ceph-mon[27978]: osdmap e24: 4 total, 3 up, 4 in 2024-04-21T08:32:05.278 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:05 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:32:05.278 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:05 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:32:05.278 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:05 smithi043 ceph-mon[27978]: osd.3 [v2:172.21.15.43:6810/117814050,v1:172.21.15.43:6811/117814050] boot 2024-04-21T08:32:05.279 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:05 smithi043 ceph-mon[27978]: osdmap e25: 4 total, 4 up, 4 in 2024-04-21T08:32:05.279 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:05 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:32:05.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:05 smithi112 ceph-mon[31085]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:05.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:05 smithi112 ceph-mon[31085]: from='osd.3 [v2:172.21.15.43:6810/117814050,v1:172.21.15.43:6811/117814050]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-21T08:32:05.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:05 smithi112 ceph-mon[31085]: osdmap e24: 4 total, 3 up, 4 in 2024-04-21T08:32:05.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:05 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:32:05.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:05 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:32:05.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:05 smithi112 ceph-mon[31085]: osd.3 [v2:172.21.15.43:6810/117814050,v1:172.21.15.43:6811/117814050] boot 2024-04-21T08:32:05.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:05 smithi112 ceph-mon[31085]: osdmap e25: 4 total, 4 up, 4 in 2024-04-21T08:32:05.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:05 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:32:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:05 smithi175 ceph-mon[31920]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:05 smithi175 ceph-mon[31920]: from='osd.3 [v2:172.21.15.43:6810/117814050,v1:172.21.15.43:6811/117814050]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-21T08:32:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:05 smithi175 ceph-mon[31920]: osdmap e24: 4 total, 3 up, 4 in 2024-04-21T08:32:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:05 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:32:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:05 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:32:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:05 smithi175 ceph-mon[31920]: osd.3 [v2:172.21.15.43:6810/117814050,v1:172.21.15.43:6811/117814050] boot 2024-04-21T08:32:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:05 smithi175 ceph-mon[31920]: osdmap e25: 4 total, 4 up, 4 in 2024-04-21T08:32:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:05 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:32:05.870 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_3 2024-04-21T08:32:06.111 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:32:06.236 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:06 smithi112 ceph-mon[31085]: purged_snaps scrub starts 2024-04-21T08:32:06.236 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:06 smithi112 ceph-mon[31085]: purged_snaps scrub ok 2024-04-21T08:32:06.236 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:06 smithi112 ceph-mon[31085]: osdmap e26: 4 total, 4 up, 4 in 2024-04-21T08:32:06.236 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:06 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:06.236 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:06 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:06.236 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:06 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:06.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:06 smithi043 ceph-mon[27978]: purged_snaps scrub starts 2024-04-21T08:32:06.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:06 smithi043 ceph-mon[27978]: purged_snaps scrub ok 2024-04-21T08:32:06.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:06 smithi043 ceph-mon[27978]: osdmap e26: 4 total, 4 up, 4 in 2024-04-21T08:32:06.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:06 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:06.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:06 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:06.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:06 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:06.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:06 smithi175 ceph-mon[31920]: purged_snaps scrub starts 2024-04-21T08:32:06.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:06 smithi175 ceph-mon[31920]: purged_snaps scrub ok 2024-04-21T08:32:06.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:06 smithi175 ceph-mon[31920]: osdmap e26: 4 total, 4 up, 4 in 2024-04-21T08:32:06.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:06 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:06.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:06 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:06.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:06 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:07.173 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:07 smithi112 ceph-mon[31085]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:07.173 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:07 smithi112 ceph-mon[31085]: Detected new or changed devices on smithi043 2024-04-21T08:32:07.173 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:07 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:07.173 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:07 smithi112 ceph-mon[31085]: Adjusting osd_memory_target on smithi043 to 8546M 2024-04-21T08:32:07.173 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:07 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:07.173 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:07 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:07.173 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:07 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:07.173 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:07 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:07.173 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:07 smithi112 ceph-mon[31085]: osdmap e27: 4 total, 4 up, 4 in 2024-04-21T08:32:07.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:07 smithi043 ceph-mon[27978]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:07.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:07 smithi043 ceph-mon[27978]: Detected new or changed devices on smithi043 2024-04-21T08:32:07.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:07 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:07.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:07 smithi043 ceph-mon[27978]: Adjusting osd_memory_target on smithi043 to 8546M 2024-04-21T08:32:07.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:07 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:07.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:07 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:07.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:07 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:07.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:07 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:07.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:07 smithi043 ceph-mon[27978]: osdmap e27: 4 total, 4 up, 4 in 2024-04-21T08:32:07.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:07 smithi175 ceph-mon[31920]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:07.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:07 smithi175 ceph-mon[31920]: Detected new or changed devices on smithi043 2024-04-21T08:32:07.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:07 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:07.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:07 smithi175 ceph-mon[31920]: Adjusting osd_memory_target on smithi043 to 8546M 2024-04-21T08:32:07.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:07 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:07.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:07 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:07.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:07 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:07.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:07 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:07.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:07 smithi175 ceph-mon[31920]: osdmap e27: 4 total, 4 up, 4 in 2024-04-21T08:32:07.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.679+0000 7f7d98373700 1 -- 172.21.15.112:0/3323973342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d90102ab0 msgr2=0x7f7d90102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:07.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.679+0000 7f7d98373700 1 --2- 172.21.15.112:0/3323973342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d90102ab0 0x7f7d90102ed0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00b0d0 tx=0x7f7d8c00b3e0 comp rx=0 tx=0).stop 2024-04-21T08:32:07.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.679+0000 7f7d98373700 1 -- 172.21.15.112:0/3323973342 shutdown_connections 2024-04-21T08:32:07.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.679+0000 7f7d98373700 1 --2- 172.21.15.112:0/3323973342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d90104650 0x7f7d9010af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:07.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.679+0000 7f7d98373700 1 --2- 172.21.15.112:0/3323973342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d90102ab0 0x7f7d90102ed0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:07.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.679+0000 7f7d98373700 1 --2- 172.21.15.112:0/3323973342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d90103cb0 0x7f7d90104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:07.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.679+0000 7f7d98373700 1 -- 172.21.15.112:0/3323973342 >> 172.21.15.112:0/3323973342 conn(0x7f7d900fe010 msgr2=0x7f7d90100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:32:07.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.679+0000 7f7d98373700 1 -- 172.21.15.112:0/3323973342 shutdown_connections 2024-04-21T08:32:07.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.679+0000 7f7d98373700 1 -- 172.21.15.112:0/3323973342 wait complete. 2024-04-21T08:32:07.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.680+0000 7f7d98373700 1 Processor -- start 2024-04-21T08:32:07.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.680+0000 7f7d98373700 1 -- start start 2024-04-21T08:32:07.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.680+0000 7f7d98373700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d90102ab0 0x7f7d90198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:32:07.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.680+0000 7f7d98373700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d90103cb0 0x7f7d90199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:32:07.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.680+0000 7f7d98373700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d90104650 0x7f7d901a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:32:07.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.680+0000 7f7d98373700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9010e540 con 0x7f7d90103cb0 2024-04-21T08:32:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.680+0000 7f7d98373700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9010e3c0 con 0x7f7d90104650 2024-04-21T08:32:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.680+0000 7f7d98373700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9010e6c0 con 0x7f7d90102ab0 2024-04-21T08:32:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.680+0000 7f7d96910700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d90104650 0x7f7d901a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:32:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.680+0000 7f7d96910700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d90104650 0x7f7d901a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39806/0 (socket says 172.21.15.112:39806) 2024-04-21T08:32:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.680+0000 7f7d96910700 1 -- 172.21.15.112:0/2061425860 learned_addr learned my addr 172.21.15.112:0/2061425860 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-21T08:32:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.680+0000 7f7d9610f700 1 --2- 172.21.15.112:0/2061425860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d90102ab0 0x7f7d90198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:32:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.680+0000 7f7d9590e700 1 --2- 172.21.15.112:0/2061425860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d90103cb0 0x7f7d90199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:32:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.680+0000 7f7d96910700 1 -- 172.21.15.112:0/2061425860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d90102ab0 msgr2=0x7f7d90198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.680+0000 7f7d96910700 1 --2- 172.21.15.112:0/2061425860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d90102ab0 0x7f7d90198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.680+0000 7f7d96910700 1 -- 172.21.15.112:0/2061425860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d90103cb0 msgr2=0x7f7d90199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.680+0000 7f7d96910700 1 --2- 172.21.15.112:0/2061425860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d90103cb0 0x7f7d90199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.680+0000 7f7d96910700 1 -- 172.21.15.112:0/2061425860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d901a0840 con 0x7f7d90104650 2024-04-21T08:32:07.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.681+0000 7f7d96910700 1 --2- 172.21.15.112:0/2061425860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d90104650 0x7f7d901a01a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800d980 tx=0x7f7d8800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:32:07.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.681+0000 7f7d877fe700 1 -- 172.21.15.112:0/2061425860 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d88017070 con 0x7f7d90104650 2024-04-21T08:32:07.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.681+0000 7f7d98373700 1 -- 172.21.15.112:0/2061425860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d901a0b30 con 0x7f7d90104650 2024-04-21T08:32:07.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.681+0000 7f7d877fe700 1 -- 172.21.15.112:0/2061425860 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d88005020 con 0x7f7d90104650 2024-04-21T08:32:07.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.681+0000 7f7d877fe700 1 -- 172.21.15.112:0/2061425860 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8800fa90 con 0x7f7d90104650 2024-04-21T08:32:07.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.681+0000 7f7d98373700 1 -- 172.21.15.112:0/2061425860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d901a1070 con 0x7f7d90104650 2024-04-21T08:32:07.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.682+0000 7f7d98373700 1 -- 172.21.15.112:0/2061425860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d90069730 con 0x7f7d90104650 2024-04-21T08:32:07.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.683+0000 7f7d877fe700 1 -- 172.21.15.112:0/2061425860 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7d8800fc30 con 0x7f7d90104650 2024-04-21T08:32:07.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.683+0000 7f7d877fe700 1 --2- 172.21.15.112:0/2061425860 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f7d7c0a3c40 0x7f7d7c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:32:07.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.683+0000 7f7d877fe700 1 -- 172.21.15.112:0/2061425860 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7f7d880cf3c0 con 0x7f7d90104650 2024-04-21T08:32:07.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.684+0000 7f7d9610f700 1 --2- 172.21.15.112:0/2061425860 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f7d7c0a3c40 0x7f7d7c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:32:07.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.684+0000 7f7d9610f700 1 --2- 172.21.15.112:0/2061425860 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f7d7c0a3c40 0x7f7d7c0a6100 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c009040 tx=0x7f7d8c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:32:07.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.687+0000 7f7d877fe700 1 -- 172.21.15.112:0/2061425860 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d88020020 con 0x7f7d90104650 2024-04-21T08:32:07.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:07.896+0000 7f7d98373700 1 -- 172.21.15.112:0/2061425860 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f7d90062e80 con 0x7f7d7c0a3c40 2024-04-21T08:32:08.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:08 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:32:08.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:08 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:32:08.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:08 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:08.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:08 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:32:08.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:08 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:32:08.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:08 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:08.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:08 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:32:08.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:08 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:32:08.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:08 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:09.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:09 smithi112 ceph-mon[31085]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:32:09.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:09 smithi112 ceph-mon[31085]: pgmap v104: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-21T08:32:09.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:09 smithi112 ceph-mon[31085]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-21T08:32:09.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:09 smithi112 ceph-mon[31085]: osdmap e28: 4 total, 4 up, 4 in 2024-04-21T08:32:09.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:09 smithi043 ceph-mon[27978]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:32:09.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:09 smithi043 ceph-mon[27978]: pgmap v104: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-21T08:32:09.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:09 smithi043 ceph-mon[27978]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-21T08:32:09.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:09 smithi043 ceph-mon[27978]: osdmap e28: 4 total, 4 up, 4 in 2024-04-21T08:32:09.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:09 smithi175 ceph-mon[31920]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:32:09.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:09 smithi175 ceph-mon[31920]: pgmap v104: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-21T08:32:09.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:09 smithi175 ceph-mon[31920]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-21T08:32:09.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:09 smithi175 ceph-mon[31920]: osdmap e28: 4 total, 4 up, 4 in 2024-04-21T08:32:10.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:10 smithi043 ceph-mon[27978]: osdmap e29: 4 total, 4 up, 4 in 2024-04-21T08:32:10.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:10 smithi175 ceph-mon[31920]: osdmap e29: 4 total, 4 up, 4 in 2024-04-21T08:32:10.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:10 smithi112 ceph-mon[31085]: osdmap e29: 4 total, 4 up, 4 in 2024-04-21T08:32:11.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:11 smithi112 ceph-mon[31085]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-21T08:32:11.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:11 smithi112 ceph-mon[31085]: from='client.? 172.21.15.112:0/595771152' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "87d594ea-199e-42db-8e47-2d535cbca3dc"}]: dispatch 2024-04-21T08:32:11.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:11 smithi112 ceph-mon[31085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "87d594ea-199e-42db-8e47-2d535cbca3dc"}]: dispatch 2024-04-21T08:32:11.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:11 smithi112 ceph-mon[31085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "87d594ea-199e-42db-8e47-2d535cbca3dc"}]': finished 2024-04-21T08:32:11.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:11 smithi112 ceph-mon[31085]: osdmap e30: 5 total, 4 up, 5 in 2024-04-21T08:32:11.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:11 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:11.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:11 smithi043 ceph-mon[27978]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-21T08:32:11.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:11 smithi043 ceph-mon[27978]: from='client.? 172.21.15.112:0/595771152' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "87d594ea-199e-42db-8e47-2d535cbca3dc"}]: dispatch 2024-04-21T08:32:11.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:11 smithi043 ceph-mon[27978]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "87d594ea-199e-42db-8e47-2d535cbca3dc"}]: dispatch 2024-04-21T08:32:11.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:11 smithi043 ceph-mon[27978]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "87d594ea-199e-42db-8e47-2d535cbca3dc"}]': finished 2024-04-21T08:32:11.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:11 smithi043 ceph-mon[27978]: osdmap e30: 5 total, 4 up, 5 in 2024-04-21T08:32:11.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:11 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:11.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:11 smithi175 ceph-mon[31920]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-21T08:32:11.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:11 smithi175 ceph-mon[31920]: from='client.? 172.21.15.112:0/595771152' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "87d594ea-199e-42db-8e47-2d535cbca3dc"}]: dispatch 2024-04-21T08:32:11.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:11 smithi175 ceph-mon[31920]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "87d594ea-199e-42db-8e47-2d535cbca3dc"}]: dispatch 2024-04-21T08:32:11.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:11 smithi175 ceph-mon[31920]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "87d594ea-199e-42db-8e47-2d535cbca3dc"}]': finished 2024-04-21T08:32:11.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:11 smithi175 ceph-mon[31920]: osdmap e30: 5 total, 4 up, 5 in 2024-04-21T08:32:11.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:11 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:12.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:12 smithi043 ceph-mon[27978]: from='client.? 172.21.15.112:0/3239191797' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:32:12.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:12 smithi175 ceph-mon[31920]: from='client.? 172.21.15.112:0/3239191797' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:32:12.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:12 smithi112 ceph-mon[31085]: from='client.? 172.21.15.112:0/3239191797' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:32:13.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:13 smithi043 ceph-mon[27978]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-04-21T08:32:13.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:13 smithi043 ceph-mon[27978]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-21T08:32:13.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:13 smithi043 ceph-mon[27978]: Cluster is now healthy 2024-04-21T08:32:13.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:13 smithi175 ceph-mon[31920]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-04-21T08:32:13.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:13 smithi175 ceph-mon[31920]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-21T08:32:13.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:13 smithi175 ceph-mon[31920]: Cluster is now healthy 2024-04-21T08:32:13.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:13 smithi112 ceph-mon[31085]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-04-21T08:32:13.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:13 smithi112 ceph-mon[31085]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-21T08:32:13.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:13 smithi112 ceph-mon[31085]: Cluster is now healthy 2024-04-21T08:32:15.293 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:15 smithi112 ceph-mon[31085]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:32:15.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:15 smithi043 ceph-mon[27978]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:32:15.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:15 smithi175 ceph-mon[31920]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:32:17.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:17 smithi175 ceph-mon[31920]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-21T08:32:17.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:17 smithi112 ceph-mon[31085]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-21T08:32:17.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:17 smithi043 ceph-mon[27978]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-21T08:32:18.440 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:18 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-21T08:32:18.440 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:18 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:18.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:18 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-21T08:32:18.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:18 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:18.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:18 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-21T08:32:18.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:18 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:19.408 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:19 smithi112 ceph-mon[31085]: Deploying daemon osd.4 on smithi112 2024-04-21T08:32:19.408 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:19 smithi112 ceph-mon[31085]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-21T08:32:19.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:19 smithi175 ceph-mon[31920]: Deploying daemon osd.4 on smithi112 2024-04-21T08:32:19.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:19 smithi175 ceph-mon[31920]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-21T08:32:19.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:19 smithi043 ceph-mon[27978]: Deploying daemon osd.4 on smithi112 2024-04-21T08:32:19.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:19 smithi043 ceph-mon[27978]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-21T08:32:21.445 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:21 smithi112 ceph-mon[31085]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T08:32:21.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:21 smithi175 ceph-mon[31920]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T08:32:21.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:21 smithi043 ceph-mon[27978]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T08:32:23.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:23 smithi175 ceph-mon[31920]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-04-21T08:32:23.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:23 smithi112 ceph-mon[31085]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-04-21T08:32:23.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:23 smithi043 ceph-mon[27978]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-04-21T08:32:24.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:24 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:24.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:24 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:24.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:24 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:24.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:24 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:24.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:24 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:24.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:24 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:24.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:24 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:24.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:24 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:24.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:24 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:25.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:25 smithi112 ceph-mon[31085]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:25.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:25 smithi175 ceph-mon[31920]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:25.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:25 smithi043 ceph-mon[27978]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:27.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:27.364+0000 7f7d877fe700 1 -- 172.21.15.112:0/2061425860 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7d90062e80 con 0x7f7d7c0a3c40 2024-04-21T08:32:27.364 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 4 on host 'smithi112' 2024-04-21T08:32:27.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:27.367+0000 7f7d98373700 1 -- 172.21.15.112:0/2061425860 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f7d7c0a3c40 msgr2=0x7f7d7c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:27.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:27.367+0000 7f7d98373700 1 --2- 172.21.15.112:0/2061425860 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f7d7c0a3c40 0x7f7d7c0a6100 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c009040 tx=0x7f7d8c00b040 comp rx=0 tx=0).stop 2024-04-21T08:32:27.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:27.367+0000 7f7d98373700 1 -- 172.21.15.112:0/2061425860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d90104650 msgr2=0x7f7d901a01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:27.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:27.367+0000 7f7d98373700 1 --2- 172.21.15.112:0/2061425860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d90104650 0x7f7d901a01a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800d980 tx=0x7f7d8800dc90 comp rx=0 tx=0).stop 2024-04-21T08:32:27.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:27.367+0000 7f7d98373700 1 -- 172.21.15.112:0/2061425860 shutdown_connections 2024-04-21T08:32:27.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:27.367+0000 7f7d98373700 1 --2- 172.21.15.112:0/2061425860 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f7d7c0a3c40 0x7f7d7c0a6100 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:27.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:27.367+0000 7f7d98373700 1 --2- 172.21.15.112:0/2061425860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d90102ab0 0x7f7d90198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:27.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:27.367+0000 7f7d98373700 1 --2- 172.21.15.112:0/2061425860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d90104650 0x7f7d901a01a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:27.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:27.367+0000 7f7d98373700 1 --2- 172.21.15.112:0/2061425860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d90103cb0 0x7f7d90199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:27.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:27.367+0000 7f7d98373700 1 -- 172.21.15.112:0/2061425860 >> 172.21.15.112:0/2061425860 conn(0x7f7d900fe010 msgr2=0x7f7d90107030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:32:27.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:27.367+0000 7f7d98373700 1 -- 172.21.15.112:0/2061425860 shutdown_connections 2024-04-21T08:32:27.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:32:27.367+0000 7f7d98373700 1 -- 172.21.15.112:0/2061425860 wait complete. 2024-04-21T08:32:27.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:27 smithi112 ceph-mon[31085]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:27.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:27 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:27.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:27 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:27.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:27 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:27.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:27 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:27.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:27 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:27.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:27 smithi112 ceph-mon[31085]: from='osd.4 [v2:172.21.15.112:6808/3007534004,v1:172.21.15.112:6809/3007534004]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T08:32:27.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:27 smithi112 ceph-mon[31085]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T08:32:27.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:27 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:27.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:27 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:27.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:27 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:27.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:27 smithi175 ceph-mon[31920]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:27.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:27 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:27.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:27 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:27.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:27 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:27.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:27 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:27.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:27 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:27.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:27 smithi175 ceph-mon[31920]: from='osd.4 [v2:172.21.15.112:6808/3007534004,v1:172.21.15.112:6809/3007534004]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T08:32:27.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:27 smithi175 ceph-mon[31920]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T08:32:27.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:27 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:27.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:27 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:27.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:27 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:27.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:27 smithi043 ceph-mon[27978]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:27.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:27 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:27.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:27 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:27.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:27 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:27.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:27 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:27.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:27 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:27.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:27 smithi043 ceph-mon[27978]: from='osd.4 [v2:172.21.15.112:6808/3007534004,v1:172.21.15.112:6809/3007534004]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T08:32:27.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:27 smithi043 ceph-mon[27978]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T08:32:27.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:27 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:27.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:27 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:27.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:27 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:28.030 DEBUG:teuthology.orchestra.run.smithi112:osd.4> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.4.service 2024-04-21T08:32:28.033 INFO:tasks.cephadm:Deploying osd.5 on smithi175 with /dev/vg_nvme/lv_3... 2024-04-21T08:32:28.033 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-21T08:32:28.280 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:32:28.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:28 smithi112 ceph-mon[31085]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-21T08:32:28.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:28 smithi112 ceph-mon[31085]: osdmap e31: 5 total, 4 up, 5 in 2024-04-21T08:32:28.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:28 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:28.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:28 smithi112 ceph-mon[31085]: from='osd.4 [v2:172.21.15.112:6808/3007534004,v1:172.21.15.112:6809/3007534004]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-21T08:32:28.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:28 smithi112 ceph-mon[31085]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-21T08:32:28.544 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 21 08:32:28 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-4[42328]: 2024-04-21T08:32:28.464+0000 7f957cad6700 -1 osd.4 0 waiting for initial osdmap 2024-04-21T08:32:28.545 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 21 08:32:28 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-4[42328]: 2024-04-21T08:32:28.469+0000 7f95788c9700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:32:28.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:28 smithi175 ceph-mon[31920]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-21T08:32:28.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:28 smithi175 ceph-mon[31920]: osdmap e31: 5 total, 4 up, 5 in 2024-04-21T08:32:28.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:28 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:28.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:28 smithi175 ceph-mon[31920]: from='osd.4 [v2:172.21.15.112:6808/3007534004,v1:172.21.15.112:6809/3007534004]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-21T08:32:28.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:28 smithi175 ceph-mon[31920]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-21T08:32:28.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:28 smithi043 ceph-mon[27978]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-21T08:32:28.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:28 smithi043 ceph-mon[27978]: osdmap e31: 5 total, 4 up, 5 in 2024-04-21T08:32:28.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:28 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:28.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:28 smithi043 ceph-mon[27978]: from='osd.4 [v2:172.21.15.112:6808/3007534004,v1:172.21.15.112:6809/3007534004]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-21T08:32:28.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:28 smithi043 ceph-mon[27978]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-21T08:32:29.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:29 smithi112 ceph-mon[31085]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:29.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:29 smithi112 ceph-mon[31085]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-21T08:32:29.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:29 smithi112 ceph-mon[31085]: osdmap e32: 5 total, 4 up, 5 in 2024-04-21T08:32:29.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:29 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:29.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:29 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:29.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:29 smithi175 ceph-mon[31920]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:29.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:29 smithi175 ceph-mon[31920]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-21T08:32:29.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:29 smithi175 ceph-mon[31920]: osdmap e32: 5 total, 4 up, 5 in 2024-04-21T08:32:29.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:29 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:29.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:29 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:29.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:29 smithi043 ceph-mon[27978]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:29.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:29 smithi043 ceph-mon[27978]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-21T08:32:29.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:29 smithi043 ceph-mon[27978]: osdmap e32: 5 total, 4 up, 5 in 2024-04-21T08:32:29.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:29.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:30.379 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-21T08:32:30.379 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-21T08:32:30.379 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-04-21T08:32:30.379 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-04-21T08:32:30.379 INFO:teuthology.orchestra.run.smithi175.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114845 s, 913 MB/s 2024-04-21T08:32:30.379 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 2024-04-21T08:32:30.379 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-04-21T08:32:30.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:30 smithi175 ceph-mon[31920]: purged_snaps scrub starts 2024-04-21T08:32:30.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:30 smithi175 ceph-mon[31920]: purged_snaps scrub ok 2024-04-21T08:32:30.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:30 smithi175 ceph-mon[31920]: osd.4 [v2:172.21.15.112:6808/3007534004,v1:172.21.15.112:6809/3007534004] boot 2024-04-21T08:32:30.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:30 smithi175 ceph-mon[31920]: osdmap e33: 5 total, 5 up, 5 in 2024-04-21T08:32:30.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:30 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:30.784 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:30 smithi112 ceph-mon[31085]: purged_snaps scrub starts 2024-04-21T08:32:30.784 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:30 smithi112 ceph-mon[31085]: purged_snaps scrub ok 2024-04-21T08:32:30.784 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:30 smithi112 ceph-mon[31085]: osd.4 [v2:172.21.15.112:6808/3007534004,v1:172.21.15.112:6809/3007534004] boot 2024-04-21T08:32:30.785 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:30 smithi112 ceph-mon[31085]: osdmap e33: 5 total, 5 up, 5 in 2024-04-21T08:32:30.785 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:30 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:30.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:30 smithi043 ceph-mon[27978]: purged_snaps scrub starts 2024-04-21T08:32:30.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:30 smithi043 ceph-mon[27978]: purged_snaps scrub ok 2024-04-21T08:32:30.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:30 smithi043 ceph-mon[27978]: osd.4 [v2:172.21.15.112:6808/3007534004,v1:172.21.15.112:6809/3007534004] boot 2024-04-21T08:32:30.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:30 smithi043 ceph-mon[27978]: osdmap e33: 5 total, 5 up, 5 in 2024-04-21T08:32:30.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:30 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:30.939 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi175:vg_nvme/lv_3 2024-04-21T08:32:31.172 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:32:31.581 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:31 smithi175 ceph-mon[31920]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:31.581 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:31 smithi175 ceph-mon[31920]: osdmap e34: 5 total, 5 up, 5 in 2024-04-21T08:32:31.581 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:31 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:31.581 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:31 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:31.582 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:31 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:31.582 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:31 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:31.582 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:31 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:31.582 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:31 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:31.582 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:31 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:31.582 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:31 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:31.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:31 smithi112 ceph-mon[31085]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:31.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:31 smithi112 ceph-mon[31085]: osdmap e34: 5 total, 5 up, 5 in 2024-04-21T08:32:31.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:31 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:31.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:31 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:31.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:31 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:31.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:31 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:31.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:31 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:31.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:31 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:31.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:31 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:31.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:31 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:31.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:31 smithi043 ceph-mon[27978]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:31.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:31 smithi043 ceph-mon[27978]: osdmap e34: 5 total, 5 up, 5 in 2024-04-21T08:32:31.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:31 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:31.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:31 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:31.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:31 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:31.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:31 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:31.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:31 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:31.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:31 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:31.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:31 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:31.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:31 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:32.638 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:32 smithi175 ceph-mon[31920]: Detected new or changed devices on smithi112 2024-04-21T08:32:32.638 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:32 smithi175 ceph-mon[31920]: Adjusting osd_memory_target on smithi112 to 8544M 2024-04-21T08:32:32.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:32 smithi112 ceph-mon[31085]: Detected new or changed devices on smithi112 2024-04-21T08:32:32.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:32 smithi112 ceph-mon[31085]: Adjusting osd_memory_target on smithi112 to 8544M 2024-04-21T08:32:32.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:32 smithi043 ceph-mon[27978]: Detected new or changed devices on smithi112 2024-04-21T08:32:32.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:32 smithi043 ceph-mon[27978]: Adjusting osd_memory_target on smithi112 to 8544M 2024-04-21T08:32:32.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.875+0000 7fc70cce9700 1 -- 172.21.15.175:0/2415547287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc708105e60 msgr2=0x7fc708108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:32.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.875+0000 7fc70cce9700 1 --2- 172.21.15.175:0/2415547287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc708105e60 0x7fc708108250 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0009990 tx=0x7fc6f0009ca0 comp rx=0 tx=0).stop 2024-04-21T08:32:32.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.875+0000 7fc70cce9700 1 -- 172.21.15.175:0/2415547287 shutdown_connections 2024-04-21T08:32:32.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.875+0000 7fc70cce9700 1 --2- 172.21.15.175:0/2415547287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc708108790 0x7fc70810ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:32.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.875+0000 7fc70cce9700 1 --2- 172.21.15.175:0/2415547287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc708103530 0x7fc708105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:32.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.875+0000 7fc70cce9700 1 --2- 172.21.15.175:0/2415547287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc708105e60 0x7fc708108250 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:32.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.875+0000 7fc70cce9700 1 -- 172.21.15.175:0/2415547287 >> 172.21.15.175:0/2415547287 conn(0x7fc7080fcf40 msgr2=0x7fc7080ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:32:32.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.875+0000 7fc70cce9700 1 -- 172.21.15.175:0/2415547287 shutdown_connections 2024-04-21T08:32:32.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.875+0000 7fc70cce9700 1 -- 172.21.15.175:0/2415547287 wait complete. 2024-04-21T08:32:32.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.875+0000 7fc70cce9700 1 Processor -- start 2024-04-21T08:32:32.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.876+0000 7fc70cce9700 1 -- start start 2024-04-21T08:32:32.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.876+0000 7fc70cce9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc708103530 0x7fc708198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:32:32.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.876+0000 7fc70cce9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc708105e60 0x7fc708199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:32:32.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.876+0000 7fc70cce9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc708108790 0x7fc7081a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:32:32.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.876+0000 7fc70cce9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc70810e530 con 0x7fc708103530 2024-04-21T08:32:32.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.876+0000 7fc70cce9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc70810e3b0 con 0x7fc708108790 2024-04-21T08:32:32.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.876+0000 7fc70cce9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc70810e6b0 con 0x7fc708105e60 2024-04-21T08:32:32.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.876+0000 7fc6fdd9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc708105e60 0x7fc708199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:32:32.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.876+0000 7fc70659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc708103530 0x7fc708198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:32:32.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.876+0000 7fc70659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc708103530 0x7fc708198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.175:39778/0 (socket says 172.21.15.175:39778) 2024-04-21T08:32:32.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.876+0000 7fc70659c700 1 -- 172.21.15.175:0/3826249235 learned_addr learned my addr 172.21.15.175:0/3826249235 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-21T08:32:32.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.876+0000 7fc706d9d700 1 --2- 172.21.15.175:0/3826249235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc708108790 0x7fc7081a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:32:32.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.876+0000 7fc6fdd9b700 1 -- 172.21.15.175:0/3826249235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc708108790 msgr2=0x7fc7081a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:32.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.876+0000 7fc6fdd9b700 1 --2- 172.21.15.175:0/3826249235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc708108790 0x7fc7081a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:32.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.876+0000 7fc6fdd9b700 1 -- 172.21.15.175:0/3826249235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc708103530 msgr2=0x7fc708198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:32.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.876+0000 7fc6fdd9b700 1 --2- 172.21.15.175:0/3826249235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc708103530 0x7fc708198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:32.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.876+0000 7fc6fdd9b700 1 -- 172.21.15.175:0/3826249235 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7081a0830 con 0x7fc708105e60 2024-04-21T08:32:32.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.877+0000 7fc6fdd9b700 1 --2- 172.21.15.175:0/3826249235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc708105e60 0x7fc708199110 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0015000 tx=0x7fc6f0009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:32:32.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.877+0000 7fc6fffff700 1 -- 172.21.15.175:0/3826249235 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0027070 con 0x7fc708105e60 2024-04-21T08:32:32.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.877+0000 7fc70cce9700 1 -- 172.21.15.175:0/3826249235 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7081a0ac0 con 0x7fc708105e60 2024-04-21T08:32:32.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.877+0000 7fc6fffff700 1 -- 172.21.15.175:0/3826249235 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6f0004000 con 0x7fc708105e60 2024-04-21T08:32:32.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.877+0000 7fc6fffff700 1 -- 172.21.15.175:0/3826249235 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0022c30 con 0x7fc708105e60 2024-04-21T08:32:32.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.877+0000 7fc70cce9700 1 -- 172.21.15.175:0/3826249235 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7081a0f70 con 0x7fc708105e60 2024-04-21T08:32:32.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.878+0000 7fc70cce9700 1 -- 172.21.15.175:0/3826249235 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc708069730 con 0x7fc708105e60 2024-04-21T08:32:32.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.879+0000 7fc6fffff700 1 -- 172.21.15.175:0/3826249235 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc6f002a070 con 0x7fc708105e60 2024-04-21T08:32:32.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.879+0000 7fc6fffff700 1 --2- 172.21.15.175:0/3826249235 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc6f40a3c40 0x7fc6f40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:32:32.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.880+0000 7fc70659c700 1 --2- 172.21.15.175:0/3826249235 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc6f40a3c40 0x7fc6f40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:32:32.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.880+0000 7fc6fffff700 1 -- 172.21.15.175:0/3826249235 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fc6f00d0340 con 0x7fc708105e60 2024-04-21T08:32:32.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.880+0000 7fc70659c700 1 --2- 172.21.15.175:0/3826249235 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc6f40a3c40 0x7fc6f40a6100 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8005950 tx=0x7fc6f800b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:32:32.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:32.884+0000 7fc6fffff700 1 -- 172.21.15.175:0/3826249235 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc6f005eff0 con 0x7fc708105e60 2024-04-21T08:32:33.090 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:33.091+0000 7fc70cce9700 1 -- 172.21.15.175:0/3826249235 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc708062f40 con 0x7fc6f40a3c40 2024-04-21T08:32:33.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:33 smithi112 ceph-mon[31085]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:33.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:33 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:32:33.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:33 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:32:33.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:33 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:33.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:33 smithi043 ceph-mon[27978]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:33.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:33 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:32:33.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:33 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:32:33.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:33 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:33.840 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:33 smithi175 ceph-mon[31920]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:33.840 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:33 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:32:33.840 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:33 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:32:33.840 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:33 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:34.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:34 smithi175 ceph-mon[31920]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:32:34.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:34 smithi112 ceph-mon[31085]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:32:34.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:34 smithi043 ceph-mon[27978]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:32:35.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:35 smithi043 ceph-mon[27978]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:35.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:35 smithi112 ceph-mon[31085]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:35.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:35 smithi175 ceph-mon[31920]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:36.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:36 smithi112 ceph-mon[31085]: from='client.? 172.21.15.175:0/815831014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93b83dce-3ad8-40ef-b472-5612b41fd044"}]: dispatch 2024-04-21T08:32:36.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:36 smithi112 ceph-mon[31085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93b83dce-3ad8-40ef-b472-5612b41fd044"}]: dispatch 2024-04-21T08:32:36.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:36 smithi112 ceph-mon[31085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93b83dce-3ad8-40ef-b472-5612b41fd044"}]': finished 2024-04-21T08:32:36.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:36 smithi112 ceph-mon[31085]: osdmap e35: 6 total, 5 up, 6 in 2024-04-21T08:32:36.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:36 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:36.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:36 smithi112 ceph-mon[31085]: from='client.? 172.21.15.175:0/3108025319' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:32:36.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:36 smithi043 ceph-mon[27978]: from='client.? 172.21.15.175:0/815831014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93b83dce-3ad8-40ef-b472-5612b41fd044"}]: dispatch 2024-04-21T08:32:36.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:36 smithi043 ceph-mon[27978]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93b83dce-3ad8-40ef-b472-5612b41fd044"}]: dispatch 2024-04-21T08:32:36.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:36 smithi043 ceph-mon[27978]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93b83dce-3ad8-40ef-b472-5612b41fd044"}]': finished 2024-04-21T08:32:36.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:36 smithi043 ceph-mon[27978]: osdmap e35: 6 total, 5 up, 6 in 2024-04-21T08:32:36.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:36 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:36.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:36 smithi043 ceph-mon[27978]: from='client.? 172.21.15.175:0/3108025319' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:32:36.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:36 smithi175 ceph-mon[31920]: from='client.? 172.21.15.175:0/815831014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93b83dce-3ad8-40ef-b472-5612b41fd044"}]: dispatch 2024-04-21T08:32:36.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:36 smithi175 ceph-mon[31920]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93b83dce-3ad8-40ef-b472-5612b41fd044"}]: dispatch 2024-04-21T08:32:36.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:36 smithi175 ceph-mon[31920]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93b83dce-3ad8-40ef-b472-5612b41fd044"}]': finished 2024-04-21T08:32:36.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:36 smithi175 ceph-mon[31920]: osdmap e35: 6 total, 5 up, 6 in 2024-04-21T08:32:36.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:36 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:36.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:36 smithi175 ceph-mon[31920]: from='client.? 172.21.15.175:0/3108025319' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:32:37.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:37 smithi175 ceph-mon[31920]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:38.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:37 smithi112 ceph-mon[31085]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:38.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:37 smithi043 ceph-mon[27978]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:39.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:39 smithi175 ceph-mon[31920]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:40.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:39 smithi112 ceph-mon[31085]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:40.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:39 smithi043 ceph-mon[27978]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:41.785 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:41 smithi175 ceph-mon[31920]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:42.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:41 smithi112 ceph-mon[31085]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:42.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:41 smithi043 ceph-mon[27978]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:43.810 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:43 smithi175 ceph-mon[31920]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:43.810 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:43 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-21T08:32:43.810 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:43 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:44.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:43 smithi112 ceph-mon[31085]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:44.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:43 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-21T08:32:44.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:43 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:44.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:43 smithi043 ceph-mon[27978]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:44.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:43 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-21T08:32:44.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:43 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:45.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:44 smithi112 ceph-mon[31085]: Deploying daemon osd.5 on smithi175 2024-04-21T08:32:45.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:44 smithi043 ceph-mon[27978]: Deploying daemon osd.5 on smithi175 2024-04-21T08:32:45.102 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:44 smithi175 ceph-mon[31920]: Deploying daemon osd.5 on smithi175 2024-04-21T08:32:46.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:45 smithi112 ceph-mon[31085]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:46.051 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:45 smithi175 ceph-mon[31920]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:46.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:45 smithi043 ceph-mon[27978]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:47.796 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:47 smithi175 ceph-mon[31920]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:48.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:47 smithi112 ceph-mon[31085]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:48.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:47 smithi043 ceph-mon[27978]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:49.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:49 smithi175 ceph-mon[31920]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:49.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:49 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:49.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:49 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:49.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:49 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:49.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:49 smithi112 ceph-mon[31085]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:49.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:49 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:49.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:49 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:49.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:49 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:49.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:49 smithi043 ceph-mon[27978]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:49.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:49 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:49.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:49 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:49.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:49 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:51.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:51 smithi175 ceph-mon[31920]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:51.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:51 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:51.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:51 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:51.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:51 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:51.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:51 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:51.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:51 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:51.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:51 smithi112 ceph-mon[31085]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:51.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:51 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:51.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:51 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:51.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:51 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:51.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:51 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:51.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:51 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:51.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:51 smithi043 ceph-mon[27978]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:51.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:51 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:51.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:51 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:51.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:51 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:51.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:51 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:51.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:51 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:52.161 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:52.161+0000 7fc6fffff700 1 -- 172.21.15.175:0/3826249235 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc708062f40 con 0x7fc6f40a3c40 2024-04-21T08:32:52.161 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 5 on host 'smithi175' 2024-04-21T08:32:52.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:52.164+0000 7fc70cce9700 1 -- 172.21.15.175:0/3826249235 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc6f40a3c40 msgr2=0x7fc6f40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:52.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:52.164+0000 7fc70cce9700 1 --2- 172.21.15.175:0/3826249235 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc6f40a3c40 0x7fc6f40a6100 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8005950 tx=0x7fc6f800b500 comp rx=0 tx=0).stop 2024-04-21T08:32:52.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:52.164+0000 7fc70cce9700 1 -- 172.21.15.175:0/3826249235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc708105e60 msgr2=0x7fc708199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:52.163 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:52.164+0000 7fc70cce9700 1 --2- 172.21.15.175:0/3826249235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc708105e60 0x7fc708199110 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0015000 tx=0x7fc6f0009580 comp rx=0 tx=0).stop 2024-04-21T08:32:52.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:52.164+0000 7fc70cce9700 1 -- 172.21.15.175:0/3826249235 shutdown_connections 2024-04-21T08:32:52.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:52.164+0000 7fc70cce9700 1 --2- 172.21.15.175:0/3826249235 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc6f40a3c40 0x7fc6f40a6100 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:52.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:52.164+0000 7fc70cce9700 1 --2- 172.21.15.175:0/3826249235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc708108790 0x7fc7081a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:52.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:52.164+0000 7fc70cce9700 1 --2- 172.21.15.175:0/3826249235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc708103530 0x7fc708198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:52.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:52.164+0000 7fc70cce9700 1 --2- 172.21.15.175:0/3826249235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc708105e60 0x7fc708199110 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:52.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:52.164+0000 7fc70cce9700 1 -- 172.21.15.175:0/3826249235 >> 172.21.15.175:0/3826249235 conn(0x7fc7080fcf40 msgr2=0x7fc708103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:32:52.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:52.164+0000 7fc70cce9700 1 -- 172.21.15.175:0/3826249235 shutdown_connections 2024-04-21T08:32:52.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:32:52.164+0000 7fc70cce9700 1 -- 172.21.15.175:0/3826249235 wait complete. 2024-04-21T08:32:52.889 DEBUG:teuthology.orchestra.run.smithi175:osd.5> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.5.service 2024-04-21T08:32:52.892 INFO:tasks.cephadm:Deploying osd.6 on smithi043 with /dev/vg_nvme/lv_2... 2024-04-21T08:32:52.892 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-21T08:32:53.151 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:32:53.274 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:53 smithi043 ceph-mon[27978]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:53.274 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:53 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:53.275 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:53 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:53.275 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:53 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:53.275 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:53 smithi043 ceph-mon[27978]: from='osd.5 [v2:172.21.15.175:6808/1901076703,v1:172.21.15.175:6809/1901076703]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T08:32:53.275 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:53 smithi043 ceph-mon[27978]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T08:32:53.441 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:53 smithi175 ceph-mon[31920]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:53.441 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:53 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:53.441 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:53 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:53.441 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:53 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:53.441 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:53 smithi175 ceph-mon[31920]: from='osd.5 [v2:172.21.15.175:6808/1901076703,v1:172.21.15.175:6809/1901076703]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T08:32:53.442 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:53 smithi175 ceph-mon[31920]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T08:32:53.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:53 smithi112 ceph-mon[31085]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:53.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:53 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:53.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:53 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:53.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:53 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:53.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:53 smithi112 ceph-mon[31085]: from='osd.5 [v2:172.21.15.175:6808/1901076703,v1:172.21.15.175:6809/1901076703]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T08:32:53.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:53 smithi112 ceph-mon[31085]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T08:32:54.386 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:54 smithi175 ceph-mon[31920]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-21T08:32:54.387 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:54 smithi175 ceph-mon[31920]: osdmap e36: 6 total, 5 up, 6 in 2024-04-21T08:32:54.387 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:54 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:54.387 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:54 smithi175 ceph-mon[31920]: from='osd.5 [v2:172.21.15.175:6808/1901076703,v1:172.21.15.175:6809/1901076703]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-04-21T08:32:54.387 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:54 smithi175 ceph-mon[31920]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-04-21T08:32:54.445 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:54 smithi043 ceph-mon[27978]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-21T08:32:54.445 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:54 smithi043 ceph-mon[27978]: osdmap e36: 6 total, 5 up, 6 in 2024-04-21T08:32:54.445 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:54 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:54.445 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:54 smithi043 ceph-mon[27978]: from='osd.5 [v2:172.21.15.175:6808/1901076703,v1:172.21.15.175:6809/1901076703]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-04-21T08:32:54.445 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:54 smithi043 ceph-mon[27978]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-04-21T08:32:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:54 smithi112 ceph-mon[31085]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-21T08:32:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:54 smithi112 ceph-mon[31085]: osdmap e36: 6 total, 5 up, 6 in 2024-04-21T08:32:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:54 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:54 smithi112 ceph-mon[31085]: from='osd.5 [v2:172.21.15.175:6808/1901076703,v1:172.21.15.175:6809/1901076703]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-04-21T08:32:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:54 smithi112 ceph-mon[31085]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-04-21T08:32:54.964 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:32:54 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5[44012]: 2024-04-21T08:32:54.883+0000 7fb4a4fca700 -1 osd.5 0 waiting for initial osdmap 2024-04-21T08:32:54.964 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:32:54 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5[44012]: 2024-04-21T08:32:54.891+0000 7fb49cdb5700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:32:55.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:55 smithi043 ceph-mon[27978]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:55.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:55 smithi043 ceph-mon[27978]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-04-21T08:32:55.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:55 smithi043 ceph-mon[27978]: osdmap e37: 6 total, 5 up, 6 in 2024-04-21T08:32:55.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:55 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:55.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:55 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:55.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:55 smithi043 ceph-mon[27978]: osdmap e38: 6 total, 5 up, 6 in 2024-04-21T08:32:55.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:55 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:55.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:55 smithi043 ceph-mon[27978]: from='osd.5 ' entity='osd.5' 2024-04-21T08:32:55.445 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:55 smithi112 ceph-mon[31085]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:55.445 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:55 smithi112 ceph-mon[31085]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-04-21T08:32:55.445 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:55 smithi112 ceph-mon[31085]: osdmap e37: 6 total, 5 up, 6 in 2024-04-21T08:32:55.445 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:55 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:55.445 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:55 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:55.445 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:55 smithi112 ceph-mon[31085]: osdmap e38: 6 total, 5 up, 6 in 2024-04-21T08:32:55.445 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:55 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:55.446 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:55 smithi112 ceph-mon[31085]: from='osd.5 ' entity='osd.5' 2024-04-21T08:32:55.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:55 smithi175 ceph-mon[31920]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:55.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:55 smithi175 ceph-mon[31920]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-04-21T08:32:55.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:55 smithi175 ceph-mon[31920]: osdmap e37: 6 total, 5 up, 6 in 2024-04-21T08:32:55.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:55 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:55.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:55 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:55.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:55 smithi175 ceph-mon[31920]: osdmap e38: 6 total, 5 up, 6 in 2024-04-21T08:32:55.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:55 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:55.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:55 smithi175 ceph-mon[31920]: from='osd.5 ' entity='osd.5' 2024-04-21T08:32:55.489 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-21T08:32:55.489 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-21T08:32:55.489 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-04-21T08:32:55.489 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-04-21T08:32:55.489 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00847671 s, 1.2 GB/s 2024-04-21T08:32:55.489 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-04-21T08:32:56.297 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_2 2024-04-21T08:32:56.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:56 smithi175 ceph-mon[31920]: purged_snaps scrub starts 2024-04-21T08:32:56.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:56 smithi175 ceph-mon[31920]: purged_snaps scrub ok 2024-04-21T08:32:56.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:56 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:56.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:56 smithi175 ceph-mon[31920]: osd.5 [v2:172.21.15.175:6808/1901076703,v1:172.21.15.175:6809/1901076703] boot 2024-04-21T08:32:56.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:56 smithi175 ceph-mon[31920]: osdmap e39: 6 total, 6 up, 6 in 2024-04-21T08:32:56.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:56 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:56.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:56 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:56.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:56 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:56.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:56 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:56.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:56 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:56.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:56 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:56.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:56 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:56.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:56 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:56.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:56 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:56.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:56 smithi112 ceph-mon[31085]: purged_snaps scrub starts 2024-04-21T08:32:56.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:56 smithi112 ceph-mon[31085]: purged_snaps scrub ok 2024-04-21T08:32:56.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:56 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:56.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:56 smithi112 ceph-mon[31085]: osd.5 [v2:172.21.15.175:6808/1901076703,v1:172.21.15.175:6809/1901076703] boot 2024-04-21T08:32:56.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:56 smithi112 ceph-mon[31085]: osdmap e39: 6 total, 6 up, 6 in 2024-04-21T08:32:56.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:56 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:56.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:56 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:56.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:56 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:56.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:56 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:56.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:56 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:56.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:56 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:56.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:56 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:56.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:56 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:56.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:56 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:56.545 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:32:56.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:56 smithi043 ceph-mon[27978]: purged_snaps scrub starts 2024-04-21T08:32:56.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:56 smithi043 ceph-mon[27978]: purged_snaps scrub ok 2024-04-21T08:32:56.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:56 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:56.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:56 smithi043 ceph-mon[27978]: osd.5 [v2:172.21.15.175:6808/1901076703,v1:172.21.15.175:6809/1901076703] boot 2024-04-21T08:32:56.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:56 smithi043 ceph-mon[27978]: osdmap e39: 6 total, 6 up, 6 in 2024-04-21T08:32:56.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:56 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:56.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:56 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:56.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:56 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:56.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:56 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:56.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:56 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:56.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:56 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:56.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:56 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:56 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:56 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:32:57.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:57 smithi043 ceph-mon[27978]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:57.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:57 smithi043 ceph-mon[27978]: Detected new or changed devices on smithi175 2024-04-21T08:32:57.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:57 smithi043 ceph-mon[27978]: Adjusting osd_memory_target on smithi175 to 8544M 2024-04-21T08:32:57.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:57 smithi043 ceph-mon[27978]: osdmap e40: 6 total, 6 up, 6 in 2024-04-21T08:32:57.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:57 smithi175 ceph-mon[31920]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:57.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:57 smithi175 ceph-mon[31920]: Detected new or changed devices on smithi175 2024-04-21T08:32:57.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:57 smithi175 ceph-mon[31920]: Adjusting osd_memory_target on smithi175 to 8544M 2024-04-21T08:32:57.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:57 smithi175 ceph-mon[31920]: osdmap e40: 6 total, 6 up, 6 in 2024-04-21T08:32:57.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:57 smithi112 ceph-mon[31085]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:57.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:57 smithi112 ceph-mon[31085]: Detected new or changed devices on smithi175 2024-04-21T08:32:57.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:57 smithi112 ceph-mon[31085]: Adjusting osd_memory_target on smithi175 to 8544M 2024-04-21T08:32:57.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:57 smithi112 ceph-mon[31085]: osdmap e40: 6 total, 6 up, 6 in 2024-04-21T08:32:58.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.327+0000 7fe705243700 1 -- 172.21.15.43:0/4082843103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7001019c0 msgr2=0x7fe700101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:58.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.327+0000 7fe705243700 1 --2- 172.21.15.43:0/4082843103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7001019c0 0x7fe700101de0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00099c0 tx=0x7fe6f0009cd0 comp rx=0 tx=0).stop 2024-04-21T08:32:58.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.327+0000 7fe705243700 1 -- 172.21.15.43:0/4082843103 shutdown_connections 2024-04-21T08:32:58.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.327+0000 7fe705243700 1 --2- 172.21.15.43:0/4082843103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe700103560 0x7fe700109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:58.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.327+0000 7fe705243700 1 --2- 172.21.15.43:0/4082843103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7001019c0 0x7fe700101de0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:58.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.327+0000 7fe705243700 1 --2- 172.21.15.43:0/4082843103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe700102bc0 0x7fe700103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:58.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.327+0000 7fe705243700 1 -- 172.21.15.43:0/4082843103 >> 172.21.15.43:0/4082843103 conn(0x7fe7000fcf60 msgr2=0x7fe7000ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:32:58.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.327+0000 7fe705243700 1 -- 172.21.15.43:0/4082843103 shutdown_connections 2024-04-21T08:32:58.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.327+0000 7fe705243700 1 -- 172.21.15.43:0/4082843103 wait complete. 2024-04-21T08:32:58.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.328+0000 7fe705243700 1 Processor -- start 2024-04-21T08:32:58.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.328+0000 7fe705243700 1 -- start start 2024-04-21T08:32:58.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.328+0000 7fe705243700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7001019c0 0x7fe70006bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:32:58.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.328+0000 7fe705243700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe700102bc0 0x7fe70019d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:32:58.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.328+0000 7fe705243700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe700103560 0x7fe70019ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:32:58.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.328+0000 7fe705243700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe70010d450 con 0x7fe700103560 2024-04-21T08:32:58.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.328+0000 7fe705243700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe70010d2d0 con 0x7fe7001019c0 2024-04-21T08:32:58.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.328+0000 7fe705243700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe70010d5d0 con 0x7fe700102bc0 2024-04-21T08:32:58.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.328+0000 7fe6ff59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe700103560 0x7fe70019ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:32:58.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.328+0000 7fe6ff59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe700103560 0x7fe70019ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41268/0 (socket says 172.21.15.43:41268) 2024-04-21T08:32:58.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.328+0000 7fe6ff59e700 1 -- 172.21.15.43:0/2152692779 learned_addr learned my addr 172.21.15.43:0/2152692779 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:32:58.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.329+0000 7fe6fed9d700 1 --2- 172.21.15.43:0/2152692779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7001019c0 0x7fe70006bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:32:58.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.329+0000 7fe6fe59c700 1 --2- 172.21.15.43:0/2152692779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe700102bc0 0x7fe70019d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:32:58.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.329+0000 7fe6ff59e700 1 -- 172.21.15.43:0/2152692779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe700102bc0 msgr2=0x7fe70019d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:58.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.329+0000 7fe6ff59e700 1 --2- 172.21.15.43:0/2152692779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe700102bc0 0x7fe70019d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:58.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.329+0000 7fe6ff59e700 1 -- 172.21.15.43:0/2152692779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7001019c0 msgr2=0x7fe70006bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:58.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.329+0000 7fe6ff59e700 1 --2- 172.21.15.43:0/2152692779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7001019c0 0x7fe70006bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:58.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.329+0000 7fe6ff59e700 1 -- 172.21.15.43:0/2152692779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7001a0840 con 0x7fe700103560 2024-04-21T08:32:58.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.329+0000 7fe6ff59e700 1 --2- 172.21.15.43:0/2152692779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe700103560 0x7fe70019ff70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8007ee0 tx=0x7fe6f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:32:58.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.329+0000 7fe6e7fff700 1 -- 172.21.15.43:0/2152692779 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f800dbd0 con 0x7fe700103560 2024-04-21T08:32:58.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.329+0000 7fe705243700 1 -- 172.21.15.43:0/2152692779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7001a0b30 con 0x7fe700103560 2024-04-21T08:32:58.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.329+0000 7fe6e7fff700 1 -- 172.21.15.43:0/2152692779 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6f800dd70 con 0x7fe700103560 2024-04-21T08:32:58.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.329+0000 7fe6e7fff700 1 -- 172.21.15.43:0/2152692779 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8005990 con 0x7fe700103560 2024-04-21T08:32:58.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.329+0000 7fe705243700 1 -- 172.21.15.43:0/2152692779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7001a1070 con 0x7fe700103560 2024-04-21T08:32:58.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.330+0000 7fe705243700 1 -- 172.21.15.43:0/2152692779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe700069730 con 0x7fe700103560 2024-04-21T08:32:58.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.332+0000 7fe6e7fff700 1 -- 172.21.15.43:0/2152692779 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe6f801a020 con 0x7fe700103560 2024-04-21T08:32:58.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.332+0000 7fe6e7fff700 1 --2- 172.21.15.43:0/2152692779 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe6e80a3c40 0x7fe6e80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:32:58.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.332+0000 7fe6fed9d700 1 --2- 172.21.15.43:0/2152692779 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe6e80a3c40 0x7fe6e80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:32:58.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.332+0000 7fe6e7fff700 1 -- 172.21.15.43:0/2152692779 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7fe6f80cf330 con 0x7fe700103560 2024-04-21T08:32:58.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.333+0000 7fe6fed9d700 1 --2- 172.21.15.43:0/2152692779 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe6e80a3c40 0x7fe6e80a6100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00038c0 tx=0x7fe6f000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:32:58.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.336+0000 7fe6e7fff700 1 -- 172.21.15.43:0/2152692779 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe6f8010070 con 0x7fe700103560 2024-04-21T08:32:58.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:32:58.554+0000 7fe705243700 1 -- 172.21.15.43:0/2152692779 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe700062e80 con 0x7fe6e80a3c40 2024-04-21T08:32:59.191 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:58 smithi043 ceph-mon[27978]: osdmap e41: 6 total, 6 up, 6 in 2024-04-21T08:32:59.191 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:58 smithi043 ceph-mon[27978]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:32:59.192 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:58 smithi043 ceph-mon[27978]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:32:59.192 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:58 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:32:59.192 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:58 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:32:59.192 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:58 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:59.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:58 smithi175 ceph-mon[31920]: osdmap e41: 6 total, 6 up, 6 in 2024-04-21T08:32:59.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:58 smithi175 ceph-mon[31920]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:32:59.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:58 smithi175 ceph-mon[31920]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:32:59.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:58 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:32:59.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:58 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:32:59.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:58 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:59.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:58 smithi112 ceph-mon[31085]: osdmap e41: 6 total, 6 up, 6 in 2024-04-21T08:32:59.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:58 smithi112 ceph-mon[31085]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:32:59.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:58 smithi112 ceph-mon[31085]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:32:59.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:58 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:32:59.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:58 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:32:59.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:58 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:00.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:32:59 smithi175 ceph-mon[31920]: osdmap e42: 6 total, 6 up, 6 in 2024-04-21T08:33:00.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:32:59 smithi112 ceph-mon[31085]: osdmap e42: 6 total, 6 up, 6 in 2024-04-21T08:33:00.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:32:59 smithi043 ceph-mon[27978]: osdmap e42: 6 total, 6 up, 6 in 2024-04-21T08:33:01.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:00 smithi043 ceph-mon[27978]: osdmap e43: 6 total, 6 up, 6 in 2024-04-21T08:33:01.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:00 smithi043 ceph-mon[27978]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:01.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:00 smithi175 ceph-mon[31920]: osdmap e43: 6 total, 6 up, 6 in 2024-04-21T08:33:01.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:00 smithi175 ceph-mon[31920]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:01.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:00 smithi112 ceph-mon[31085]: osdmap e43: 6 total, 6 up, 6 in 2024-04-21T08:33:01.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:00 smithi112 ceph-mon[31085]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:02.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:01 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3839067683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3abceb62-32a7-48ae-a367-41869d2471cf"}]: dispatch 2024-04-21T08:33:02.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:01 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3839067683' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3abceb62-32a7-48ae-a367-41869d2471cf"}]': finished 2024-04-21T08:33:02.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:01 smithi175 ceph-mon[31920]: osdmap e44: 7 total, 6 up, 7 in 2024-04-21T08:33:02.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:01 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:02.293 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:01 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3839067683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3abceb62-32a7-48ae-a367-41869d2471cf"}]: dispatch 2024-04-21T08:33:02.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:01 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3839067683' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3abceb62-32a7-48ae-a367-41869d2471cf"}]': finished 2024-04-21T08:33:02.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:01 smithi112 ceph-mon[31085]: osdmap e44: 7 total, 6 up, 7 in 2024-04-21T08:33:02.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:01 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:02.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:01 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3839067683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3abceb62-32a7-48ae-a367-41869d2471cf"}]: dispatch 2024-04-21T08:33:02.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:01 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3839067683' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3abceb62-32a7-48ae-a367-41869d2471cf"}]': finished 2024-04-21T08:33:02.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:01 smithi043 ceph-mon[27978]: osdmap e44: 7 total, 6 up, 7 in 2024-04-21T08:33:02.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:01 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:03.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:02 smithi175 ceph-mon[31920]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-04-21T08:33:03.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:02 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2408293297' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:33:03.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:02 smithi112 ceph-mon[31085]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-04-21T08:33:03.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:02 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2408293297' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:33:03.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:02 smithi043 ceph-mon[27978]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-04-21T08:33:03.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:02 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2408293297' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:33:05.259 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:04 smithi043 ceph-mon[27978]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:33:05.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:04 smithi112 ceph-mon[31085]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:33:05.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:04 smithi175 ceph-mon[31920]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:33:07.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:06 smithi112 ceph-mon[31085]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-21T08:33:07.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:06 smithi043 ceph-mon[27978]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-21T08:33:07.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:06 smithi175 ceph-mon[31920]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-21T08:33:09.212 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:08 smithi043 ceph-mon[27978]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-21T08:33:09.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:08 smithi112 ceph-mon[31085]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-21T08:33:09.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:08 smithi175 ceph-mon[31920]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-21T08:33:10.173 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:09 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-21T08:33:10.173 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:09 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:10.173 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:09 smithi043 ceph-mon[27978]: Deploying daemon osd.6 on smithi043 2024-04-21T08:33:10.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:10 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-21T08:33:10.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:10 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:10.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:10 smithi112 ceph-mon[31085]: Deploying daemon osd.6 on smithi043 2024-04-21T08:33:10.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:09 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-21T08:33:10.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:09 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:10.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:09 smithi175 ceph-mon[31920]: Deploying daemon osd.6 on smithi043 2024-04-21T08:33:11.211 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:10 smithi043 ceph-mon[27978]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T08:33:11.293 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:11 smithi112 ceph-mon[31085]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T08:33:11.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:10 smithi175 ceph-mon[31920]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T08:33:13.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:13 smithi043 ceph-mon[27978]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-04-21T08:33:13.293 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:13 smithi112 ceph-mon[31085]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-04-21T08:33:13.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:13 smithi175 ceph-mon[31920]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-04-21T08:33:15.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:15 smithi043 ceph-mon[27978]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:15.293 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:15 smithi112 ceph-mon[31085]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:15.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:15 smithi175 ceph-mon[31920]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:16.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:16 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:16.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:16 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:16.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:16 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:33:16.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:16 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:16.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:16 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:16.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:16 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:33:16.793 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:16 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:16.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:16 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:16.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:16 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:33:17.603 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:17 smithi043 ceph-mon[27978]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:17.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:17 smithi175 ceph-mon[31920]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:17.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:17 smithi112 ceph-mon[31085]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:33:19.133+0000 7fe6e7fff700 1 -- 172.21.15.43:0/2152692779 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe700062e80 con 0x7fe6e80a3c40 2024-04-21T08:33:19.133 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 6 on host 'smithi043' 2024-04-21T08:33:19.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:33:19.135+0000 7fe705243700 1 -- 172.21.15.43:0/2152692779 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe6e80a3c40 msgr2=0x7fe6e80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:33:19.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:33:19.135+0000 7fe705243700 1 --2- 172.21.15.43:0/2152692779 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe6e80a3c40 0x7fe6e80a6100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00038c0 tx=0x7fe6f000b540 comp rx=0 tx=0).stop 2024-04-21T08:33:19.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:33:19.135+0000 7fe705243700 1 -- 172.21.15.43:0/2152692779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe700103560 msgr2=0x7fe70019ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:33:19.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:33:19.135+0000 7fe705243700 1 --2- 172.21.15.43:0/2152692779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe700103560 0x7fe70019ff70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8007ee0 tx=0x7fe6f800d3b0 comp rx=0 tx=0).stop 2024-04-21T08:33:19.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:33:19.135+0000 7fe705243700 1 -- 172.21.15.43:0/2152692779 shutdown_connections 2024-04-21T08:33:19.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:33:19.135+0000 7fe705243700 1 --2- 172.21.15.43:0/2152692779 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe6e80a3c40 0x7fe6e80a6100 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:19.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:33:19.135+0000 7fe705243700 1 --2- 172.21.15.43:0/2152692779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe700103560 0x7fe70019ff70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:19.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:33:19.135+0000 7fe705243700 1 --2- 172.21.15.43:0/2152692779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7001019c0 0x7fe70006bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:19.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:33:19.135+0000 7fe705243700 1 --2- 172.21.15.43:0/2152692779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe700102bc0 0x7fe70019d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:19.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:33:19.135+0000 7fe705243700 1 -- 172.21.15.43:0/2152692779 >> 172.21.15.43:0/2152692779 conn(0x7fe7000fcf60 msgr2=0x7fe700105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:33:19.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:33:19.136+0000 7fe705243700 1 -- 172.21.15.43:0/2152692779 shutdown_connections 2024-04-21T08:33:19.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:33:19.136+0000 7fe705243700 1 -- 172.21.15.43:0/2152692779 wait complete. 2024-04-21T08:33:19.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:19 smithi175 ceph-mon[31920]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:19.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:19 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:19.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:19 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:19.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:19 smithi175 ceph-mon[31920]: from='osd.6 [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-21T08:33:19.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:19 smithi112 ceph-mon[31085]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:19.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:19 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:19.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:19 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:19.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:19 smithi112 ceph-mon[31085]: from='osd.6 [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-21T08:33:19.795 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:19 smithi043 ceph-mon[27978]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:19.796 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:19 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:19.796 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:19 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:19.796 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:19 smithi043 ceph-mon[27978]: from='osd.6 [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-21T08:33:19.825 DEBUG:teuthology.orchestra.run.smithi043:osd.6> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.6.service 2024-04-21T08:33:19.828 INFO:tasks.cephadm:Deploying osd.7 on smithi112 with /dev/vg_nvme/lv_2... 2024-04-21T08:33:19.828 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-21T08:33:20.071 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:33:20.454 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:20 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:20.454 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:20 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:20.455 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:20 smithi112 ceph-mon[31085]: from='osd.6 [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-21T08:33:20.455 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:20 smithi112 ceph-mon[31085]: osdmap e45: 7 total, 6 up, 7 in 2024-04-21T08:33:20.455 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:20 smithi112 ceph-mon[31085]: from='osd.6 [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-21T08:33:20.455 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:20 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:20.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:20 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:20.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:20 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:20.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:20 smithi175 ceph-mon[31920]: from='osd.6 [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-21T08:33:20.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:20 smithi175 ceph-mon[31920]: osdmap e45: 7 total, 6 up, 7 in 2024-04-21T08:33:20.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:20 smithi175 ceph-mon[31920]: from='osd.6 [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-21T08:33:20.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:20 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:20.753 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:20 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:20.754 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:20 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:20.754 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:20 smithi043 ceph-mon[27978]: from='osd.6 [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-21T08:33:20.754 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:20 smithi043 ceph-mon[27978]: osdmap e45: 7 total, 6 up, 7 in 2024-04-21T08:33:20.754 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:20 smithi043 ceph-mon[27978]: from='osd.6 [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-21T08:33:20.754 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:20 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:21.525 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:21 smithi112 ceph-mon[31085]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:21.526 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:21 smithi112 ceph-mon[31085]: from='osd.6 [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-21T08:33:21.526 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:21 smithi112 ceph-mon[31085]: osdmap e46: 7 total, 6 up, 7 in 2024-04-21T08:33:21.526 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:21 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:21.526 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:21 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:21.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:21 smithi043 ceph-mon[27978]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:21.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:21 smithi043 ceph-mon[27978]: from='osd.6 [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-21T08:33:21.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:21 smithi043 ceph-mon[27978]: osdmap e46: 7 total, 6 up, 7 in 2024-04-21T08:33:21.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:21 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:21.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:21 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:21.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:21 smithi175 ceph-mon[31920]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:21.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:21 smithi175 ceph-mon[31920]: from='osd.6 [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-21T08:33:21.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:21 smithi175 ceph-mon[31920]: osdmap e46: 7 total, 6 up, 7 in 2024-04-21T08:33:21.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:21 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:21.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:21 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:22.061 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:33:21 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:33:21.650+0000 7efcb3ca5700 -1 osd.6 0 waiting for initial osdmap 2024-04-21T08:33:22.061 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:33:21 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:33:21.661+0000 7efcaba90700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:33:22.111 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-21T08:33:22.111 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-21T08:33:22.111 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-04-21T08:33:22.111 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-04-21T08:33:22.111 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114072 s, 919 MB/s 2024-04-21T08:33:22.111 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-04-21T08:33:22.758 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:22 smithi112 ceph-mon[31085]: purged_snaps scrub starts 2024-04-21T08:33:22.758 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:22 smithi112 ceph-mon[31085]: purged_snaps scrub ok 2024-04-21T08:33:22.758 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:22 smithi112 ceph-mon[31085]: from='osd.6 [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202]' entity='osd.6' 2024-04-21T08:33:22.758 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:22 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:22.785 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_2 2024-04-21T08:33:22.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:22 smithi175 ceph-mon[31920]: purged_snaps scrub starts 2024-04-21T08:33:22.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:22 smithi175 ceph-mon[31920]: purged_snaps scrub ok 2024-04-21T08:33:22.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:22 smithi175 ceph-mon[31920]: from='osd.6 [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202]' entity='osd.6' 2024-04-21T08:33:22.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:22 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:23.024 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:33:23.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:22 smithi043 ceph-mon[27978]: purged_snaps scrub starts 2024-04-21T08:33:23.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:22 smithi043 ceph-mon[27978]: purged_snaps scrub ok 2024-04-21T08:33:23.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:22 smithi043 ceph-mon[27978]: from='osd.6 [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202]' entity='osd.6' 2024-04-21T08:33:23.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:22 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:23.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:23 smithi112 ceph-mon[31085]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:23.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:23 smithi112 ceph-mon[31085]: osd.6 [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202] boot 2024-04-21T08:33:23.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:23 smithi112 ceph-mon[31085]: osdmap e47: 7 total, 7 up, 7 in 2024-04-21T08:33:23.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:23.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:23.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:23.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:23.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:23.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:23.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:23.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:23.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:33:23.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:23.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:33:23.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:23.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:33:23.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:23.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:23 smithi175 ceph-mon[31920]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:23.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:23 smithi175 ceph-mon[31920]: osd.6 [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202] boot 2024-04-21T08:33:23.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:23 smithi175 ceph-mon[31920]: osdmap e47: 7 total, 7 up, 7 in 2024-04-21T08:33:23.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:23.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:23.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:23.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:23.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:23.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:23.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:23.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:23.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:33:23.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:23.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:33:23.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:23.965 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:33:23.965 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:24.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:23 smithi043 ceph-mon[27978]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:24.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:23 smithi043 ceph-mon[27978]: osd.6 [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202] boot 2024-04-21T08:33:24.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:23 smithi043 ceph-mon[27978]: osdmap e47: 7 total, 7 up, 7 in 2024-04-21T08:33:24.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:24.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:24.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:24.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:24.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:24.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:24.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:24.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:24.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:33:24.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:24.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:33:24.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:24.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:33:24.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:24.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.747+0000 7f857d59f700 1 -- 172.21.15.112:0/1738012279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8578102ba0 msgr2=0x7f8578103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:33:24.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.747+0000 7f857d59f700 1 --2- 172.21.15.112:0/1738012279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8578102ba0 0x7f8578103000 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f85600099c0 tx=0x7f8560009cd0 comp rx=0 tx=0).stop 2024-04-21T08:33:24.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.747+0000 7f857d59f700 1 -- 172.21.15.112:0/1738012279 shutdown_connections 2024-04-21T08:33:24.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.747+0000 7f857d59f700 1 --2- 172.21.15.112:0/1738012279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8578103540 0x7f8578109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:24.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.747+0000 7f857d59f700 1 --2- 172.21.15.112:0/1738012279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85781019a0 0x7f8578101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:24.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.747+0000 7f857d59f700 1 --2- 172.21.15.112:0/1738012279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8578102ba0 0x7f8578103000 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:24.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.747+0000 7f857d59f700 1 -- 172.21.15.112:0/1738012279 >> 172.21.15.112:0/1738012279 conn(0x7f85780fcf40 msgr2=0x7f85780ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:33:24.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.747+0000 7f857d59f700 1 -- 172.21.15.112:0/1738012279 shutdown_connections 2024-04-21T08:33:24.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.747+0000 7f857d59f700 1 -- 172.21.15.112:0/1738012279 wait complete. 2024-04-21T08:33:24.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.747+0000 7f857d59f700 1 Processor -- start 2024-04-21T08:33:24.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.747+0000 7f857d59f700 1 -- start start 2024-04-21T08:33:24.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.748+0000 7f857d59f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85781019a0 0x7f857806b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:33:24.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.748+0000 7f857d59f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8578102ba0 0x7f857819d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:33:24.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.748+0000 7f857d59f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8578103540 0x7f857819ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:33:24.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.748+0000 7f857d59f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f857810d440 con 0x7f8578102ba0 2024-04-21T08:33:24.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.748+0000 7f857d59f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f857810d2c0 con 0x7f85781019a0 2024-04-21T08:33:24.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.748+0000 7f857d59f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f857810d5c0 con 0x7f8578103540 2024-04-21T08:33:24.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.748+0000 7f8576ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85781019a0 0x7f857806b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:33:24.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.748+0000 7f8576ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85781019a0 0x7f857806b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51222/0 (socket says 172.21.15.112:51222) 2024-04-21T08:33:24.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.748+0000 7f8576ffd700 1 -- 172.21.15.112:0/3214065255 learned_addr learned my addr 172.21.15.112:0/3214065255 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-21T08:33:24.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.748+0000 7f85767fc700 1 --2- 172.21.15.112:0/3214065255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8578102ba0 0x7f857819d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:33:24.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.748+0000 7f85777fe700 1 --2- 172.21.15.112:0/3214065255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8578103540 0x7f857819ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:33:24.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.748+0000 7f8576ffd700 1 -- 172.21.15.112:0/3214065255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8578103540 msgr2=0x7f857819ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:33:24.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.748+0000 7f8576ffd700 1 --2- 172.21.15.112:0/3214065255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8578103540 0x7f857819ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:24.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.748+0000 7f8576ffd700 1 -- 172.21.15.112:0/3214065255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8578102ba0 msgr2=0x7f857819d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:33:24.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.748+0000 7f8576ffd700 1 --2- 172.21.15.112:0/3214065255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8578102ba0 0x7f857819d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:24.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.748+0000 7f8576ffd700 1 -- 172.21.15.112:0/3214065255 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85781a0820 con 0x7f85781019a0 2024-04-21T08:33:24.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.748+0000 7f8576ffd700 1 --2- 172.21.15.112:0/3214065255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85781019a0 0x7f857806b280 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8570007e00 tx=0x7f857000a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:33:24.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.748+0000 7f856ffff700 1 -- 172.21.15.112:0/3214065255 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8570016020 con 0x7f85781019a0 2024-04-21T08:33:24.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.748+0000 7f856ffff700 1 -- 172.21.15.112:0/3214065255 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f857000ad40 con 0x7f85781019a0 2024-04-21T08:33:24.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.748+0000 7f857d59f700 1 -- 172.21.15.112:0/3214065255 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85781a0b10 con 0x7f85781019a0 2024-04-21T08:33:24.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.748+0000 7f856ffff700 1 -- 172.21.15.112:0/3214065255 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8570011670 con 0x7f85781019a0 2024-04-21T08:33:24.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.748+0000 7f857d59f700 1 -- 172.21.15.112:0/3214065255 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85781a1020 con 0x7f85781019a0 2024-04-21T08:33:24.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.749+0000 7f857d59f700 1 -- 172.21.15.112:0/3214065255 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8578069730 con 0x7f85781019a0 2024-04-21T08:33:24.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.751+0000 7f856ffff700 1 -- 172.21.15.112:0/3214065255 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8570019020 con 0x7f85781019a0 2024-04-21T08:33:24.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.752+0000 7f856ffff700 1 --2- 172.21.15.112:0/3214065255 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f85640a3c40 0x7f85640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:33:24.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.752+0000 7f856ffff700 1 -- 172.21.15.112:0/3214065255 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f85700cfab0 con 0x7f85781019a0 2024-04-21T08:33:24.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.752+0000 7f85767fc700 1 --2- 172.21.15.112:0/3214065255 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f85640a3c40 0x7f85640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:33:24.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.753+0000 7f85767fc700 1 --2- 172.21.15.112:0/3214065255 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f85640a3c40 0x7f85640a6100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8560009990 tx=0x7f8560005bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:33:24.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.756+0000 7f856ffff700 1 -- 172.21.15.112:0/3214065255 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8570020020 con 0x7f85781019a0 2024-04-21T08:33:24.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:24.961+0000 7f857d59f700 1 -- 172.21.15.112:0/3214065255 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8578062e80 con 0x7f85640a3c40 2024-04-21T08:33:24.961 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:24 smithi112 ceph-mon[31085]: Detected new or changed devices on smithi043 2024-04-21T08:33:24.961 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:24 smithi112 ceph-mon[31085]: Adjusting osd_memory_target on smithi043 to 5697M 2024-04-21T08:33:24.961 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:24 smithi112 ceph-mon[31085]: osdmap e48: 7 total, 7 up, 7 in 2024-04-21T08:33:24.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:24 smithi175 ceph-mon[31920]: Detected new or changed devices on smithi043 2024-04-21T08:33:24.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:24 smithi175 ceph-mon[31920]: Adjusting osd_memory_target on smithi043 to 5697M 2024-04-21T08:33:24.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:24 smithi175 ceph-mon[31920]: osdmap e48: 7 total, 7 up, 7 in 2024-04-21T08:33:25.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:24 smithi043 ceph-mon[27978]: Detected new or changed devices on smithi043 2024-04-21T08:33:25.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:24 smithi043 ceph-mon[27978]: Adjusting osd_memory_target on smithi043 to 5697M 2024-04-21T08:33:25.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:24 smithi043 ceph-mon[27978]: osdmap e48: 7 total, 7 up, 7 in 2024-04-21T08:33:25.905 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:25 smithi112 ceph-mon[31085]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:25.906 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:25 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:33:25.906 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:25 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:33:25.906 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:25 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:25.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:25 smithi175 ceph-mon[31920]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:25.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:25 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:33:25.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:25 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:33:25.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:25 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:26.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:25 smithi043 ceph-mon[27978]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:26.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:25 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:33:26.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:25 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:33:26.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:25 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:26.912 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:26 smithi112 ceph-mon[31085]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:33:26.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:26 smithi175 ceph-mon[31920]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:33:27.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:26 smithi043 ceph-mon[27978]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:33:27.928 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:27 smithi112 ceph-mon[31085]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:27.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:27 smithi175 ceph-mon[31920]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:28.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:27 smithi043 ceph-mon[27978]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:28.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:28 smithi175 ceph-mon[31920]: from='client.? 172.21.15.112:0/2026738309' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "223d7e91-e4a6-42e0-b5fb-bbbf4844efd2"}]: dispatch 2024-04-21T08:33:28.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:28 smithi175 ceph-mon[31920]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "223d7e91-e4a6-42e0-b5fb-bbbf4844efd2"}]: dispatch 2024-04-21T08:33:28.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:28 smithi175 ceph-mon[31920]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "223d7e91-e4a6-42e0-b5fb-bbbf4844efd2"}]': finished 2024-04-21T08:33:28.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:28 smithi175 ceph-mon[31920]: osdmap e49: 8 total, 7 up, 8 in 2024-04-21T08:33:28.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:28 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:28.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:28 smithi175 ceph-mon[31920]: from='client.? 172.21.15.112:0/1494440569' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:33:29.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:28 smithi112 ceph-mon[31085]: from='client.? 172.21.15.112:0/2026738309' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "223d7e91-e4a6-42e0-b5fb-bbbf4844efd2"}]: dispatch 2024-04-21T08:33:29.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:28 smithi112 ceph-mon[31085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "223d7e91-e4a6-42e0-b5fb-bbbf4844efd2"}]: dispatch 2024-04-21T08:33:29.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:28 smithi112 ceph-mon[31085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "223d7e91-e4a6-42e0-b5fb-bbbf4844efd2"}]': finished 2024-04-21T08:33:29.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:28 smithi112 ceph-mon[31085]: osdmap e49: 8 total, 7 up, 8 in 2024-04-21T08:33:29.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:28 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:29.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:28 smithi112 ceph-mon[31085]: from='client.? 172.21.15.112:0/1494440569' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:33:29.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:28 smithi043 ceph-mon[27978]: from='client.? 172.21.15.112:0/2026738309' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "223d7e91-e4a6-42e0-b5fb-bbbf4844efd2"}]: dispatch 2024-04-21T08:33:29.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:28 smithi043 ceph-mon[27978]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "223d7e91-e4a6-42e0-b5fb-bbbf4844efd2"}]: dispatch 2024-04-21T08:33:29.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:28 smithi043 ceph-mon[27978]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "223d7e91-e4a6-42e0-b5fb-bbbf4844efd2"}]': finished 2024-04-21T08:33:29.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:28 smithi043 ceph-mon[27978]: osdmap e49: 8 total, 7 up, 8 in 2024-04-21T08:33:29.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:28 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:29.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:28 smithi043 ceph-mon[27978]: from='client.? 172.21.15.112:0/1494440569' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:33:29.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:29 smithi175 ceph-mon[31920]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:30.043 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:29 smithi112 ceph-mon[31085]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:30.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:29 smithi043 ceph-mon[27978]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:31.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:31 smithi175 ceph-mon[31920]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:32.010 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:31 smithi112 ceph-mon[31085]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:32.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:31 smithi043 ceph-mon[27978]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:33.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:33 smithi175 ceph-mon[31920]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:34.036 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:33 smithi112 ceph-mon[31085]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:34.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:33 smithi043 ceph-mon[27978]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:35.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:35 smithi175 ceph-mon[31920]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:35.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:35 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-21T08:33:35.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:35 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:35.970 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:35 smithi112 ceph-mon[31085]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:35.970 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:35 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-21T08:33:35.970 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:35 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:36.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:35 smithi043 ceph-mon[27978]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:36.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:35 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-21T08:33:36.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:35 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:36.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:36 smithi175 ceph-mon[31920]: Deploying daemon osd.7 on smithi112 2024-04-21T08:33:37.027 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:36 smithi112 ceph-mon[31085]: Deploying daemon osd.7 on smithi112 2024-04-21T08:33:37.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:36 smithi043 ceph-mon[27978]: Deploying daemon osd.7 on smithi112 2024-04-21T08:33:37.963 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:37 smithi112 ceph-mon[31085]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:37.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:37 smithi175 ceph-mon[31920]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:38.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:37 smithi043 ceph-mon[27978]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:39.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:39 smithi175 ceph-mon[31920]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:40.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:39 smithi112 ceph-mon[31085]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:40.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:39 smithi043 ceph-mon[27978]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:41.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:41 smithi175 ceph-mon[31920]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:41.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:41 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:41.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:41 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:41.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:41 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:33:41.980 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:41 smithi112 ceph-mon[31085]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:41.980 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:41 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:41.980 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:41 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:41.980 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:41 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:33:42.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:41 smithi043 ceph-mon[27978]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:42.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:41 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:42.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:41 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:42.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:41 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:33:43.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:43 smithi175 ceph-mon[31920]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:44.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:43 smithi112 ceph-mon[31085]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:44.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:43 smithi043 ceph-mon[27978]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:44.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:44.833+0000 7f856ffff700 1 -- 172.21.15.112:0/3214065255 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8578062e80 con 0x7f85640a3c40 2024-04-21T08:33:44.834 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 7 on host 'smithi112' 2024-04-21T08:33:44.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:44.836+0000 7f857d59f700 1 -- 172.21.15.112:0/3214065255 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f85640a3c40 msgr2=0x7f85640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:33:44.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:44.836+0000 7f857d59f700 1 --2- 172.21.15.112:0/3214065255 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f85640a3c40 0x7f85640a6100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8560009990 tx=0x7f8560005bc0 comp rx=0 tx=0).stop 2024-04-21T08:33:44.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:44.836+0000 7f857d59f700 1 -- 172.21.15.112:0/3214065255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85781019a0 msgr2=0x7f857806b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:33:44.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:44.836+0000 7f857d59f700 1 --2- 172.21.15.112:0/3214065255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85781019a0 0x7f857806b280 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8570007e00 tx=0x7f857000a340 comp rx=0 tx=0).stop 2024-04-21T08:33:44.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:44.836+0000 7f857d59f700 1 -- 172.21.15.112:0/3214065255 shutdown_connections 2024-04-21T08:33:44.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:44.836+0000 7f857d59f700 1 --2- 172.21.15.112:0/3214065255 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f85640a3c40 0x7f85640a6100 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:44.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:44.836+0000 7f857d59f700 1 --2- 172.21.15.112:0/3214065255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8578103540 0x7f857819ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:44.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:44.836+0000 7f857d59f700 1 --2- 172.21.15.112:0/3214065255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85781019a0 0x7f857806b280 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:44.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:44.836+0000 7f857d59f700 1 --2- 172.21.15.112:0/3214065255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8578102ba0 0x7f857819d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:44.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:44.836+0000 7f857d59f700 1 -- 172.21.15.112:0/3214065255 >> 172.21.15.112:0/3214065255 conn(0x7f85780fcf40 msgr2=0x7f8578105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:33:44.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:44.836+0000 7f857d59f700 1 -- 172.21.15.112:0/3214065255 shutdown_connections 2024-04-21T08:33:44.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:33:44.836+0000 7f857d59f700 1 -- 172.21.15.112:0/3214065255 wait complete. 2024-04-21T08:33:45.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:45 smithi112 ceph-mon[31085]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:45.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:45 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:45.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:45 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:45.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:45 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:45.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:45 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:33:45.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:45 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:45.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:45 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:45.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:45 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:45.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:45 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:33:45.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:45 smithi112 ceph-mon[31085]: from='osd.7 [v2:172.21.15.112:6816/1090541705,v1:172.21.15.112:6817/1090541705]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T08:33:45.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:45 smithi112 ceph-mon[31085]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T08:33:45.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:45 smithi043 ceph-mon[27978]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:45.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:45 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:45.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:45 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:45.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:45 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:45.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:45 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:33:45.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:45 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:45.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:45 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:45.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:45 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:45.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:45 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:33:45.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:45 smithi043 ceph-mon[27978]: from='osd.7 [v2:172.21.15.112:6816/1090541705,v1:172.21.15.112:6817/1090541705]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T08:33:45.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:45 smithi043 ceph-mon[27978]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T08:33:45.449 DEBUG:teuthology.orchestra.run.smithi112:osd.7> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.7.service 2024-04-21T08:33:45.452 INFO:tasks.cephadm:Deploying osd.8 on smithi175 with /dev/vg_nvme/lv_2... 2024-04-21T08:33:45.452 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-21T08:33:45.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:45 smithi175 ceph-mon[31920]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:45.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:45 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:45.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:45 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:45.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:45 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:45.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:45 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:33:45.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:45 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:45.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:45 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:45.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:45 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:45.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:45 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:33:45.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:45 smithi175 ceph-mon[31920]: from='osd.7 [v2:172.21.15.112:6816/1090541705,v1:172.21.15.112:6817/1090541705]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T08:33:45.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:45 smithi175 ceph-mon[31920]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T08:33:45.699 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:33:46.907 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:46 smithi112 ceph-mon[31085]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-21T08:33:46.907 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:46 smithi112 ceph-mon[31085]: osdmap e50: 8 total, 7 up, 8 in 2024-04-21T08:33:46.907 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:46 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:46.907 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:46 smithi112 ceph-mon[31085]: from='osd.7 [v2:172.21.15.112:6816/1090541705,v1:172.21.15.112:6817/1090541705]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-21T08:33:46.907 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:46 smithi112 ceph-mon[31085]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-21T08:33:46.907 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:46 smithi112 ceph-mon[31085]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:46.908 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:33:46 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:33:46.837+0000 7f4fe0c59700 -1 osd.7 0 waiting for initial osdmap 2024-04-21T08:33:46.908 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:33:46 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:33:46.845+0000 7f4fdca4c700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:33:47.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:46 smithi043 ceph-mon[27978]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-21T08:33:47.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:46 smithi043 ceph-mon[27978]: osdmap e50: 8 total, 7 up, 8 in 2024-04-21T08:33:47.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:46 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:47.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:46 smithi043 ceph-mon[27978]: from='osd.7 [v2:172.21.15.112:6816/1090541705,v1:172.21.15.112:6817/1090541705]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-21T08:33:47.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:46 smithi043 ceph-mon[27978]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-21T08:33:47.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:46 smithi043 ceph-mon[27978]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:47.091 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:46 smithi175 ceph-mon[31920]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-21T08:33:47.091 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:46 smithi175 ceph-mon[31920]: osdmap e50: 8 total, 7 up, 8 in 2024-04-21T08:33:47.091 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:46 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:47.091 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:46 smithi175 ceph-mon[31920]: from='osd.7 [v2:172.21.15.112:6816/1090541705,v1:172.21.15.112:6817/1090541705]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-21T08:33:47.091 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:46 smithi175 ceph-mon[31920]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-21T08:33:47.091 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:46 smithi175 ceph-mon[31920]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:47.820 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-21T08:33:47.820 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-21T08:33:47.820 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-04-21T08:33:47.820 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-04-21T08:33:47.820 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113761 s, 922 MB/s 2024-04-21T08:33:47.820 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-04-21T08:33:48.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:47 smithi112 ceph-mon[31085]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-21T08:33:48.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:47 smithi112 ceph-mon[31085]: osdmap e51: 8 total, 7 up, 8 in 2024-04-21T08:33:48.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:47 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:48.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:47 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:48.206 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:47 smithi175 ceph-mon[31920]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-21T08:33:48.206 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:47 smithi175 ceph-mon[31920]: osdmap e51: 8 total, 7 up, 8 in 2024-04-21T08:33:48.206 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:47 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:48.206 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:47 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:48.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:47 smithi043 ceph-mon[27978]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-21T08:33:48.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:47 smithi043 ceph-mon[27978]: osdmap e51: 8 total, 7 up, 8 in 2024-04-21T08:33:48.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:47 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:48.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:47 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:48.513 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi175:vg_nvme/lv_2 2024-04-21T08:33:48.804 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:33:49.099 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:48 smithi175 ceph-mon[31920]: purged_snaps scrub starts 2024-04-21T08:33:49.100 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:48 smithi175 ceph-mon[31920]: purged_snaps scrub ok 2024-04-21T08:33:49.100 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:48 smithi175 ceph-mon[31920]: osd.7 [v2:172.21.15.112:6816/1090541705,v1:172.21.15.112:6817/1090541705] boot 2024-04-21T08:33:49.100 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:48 smithi175 ceph-mon[31920]: osdmap e52: 8 total, 8 up, 8 in 2024-04-21T08:33:49.100 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:48 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:49.100 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:48 smithi175 ceph-mon[31920]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:49.100 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:48 smithi175 ceph-mon[31920]: Detected new or changed devices on smithi112 2024-04-21T08:33:49.100 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:48 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:49.100 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:48 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:49.100 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:48 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:49.100 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:48 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:49.100 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:48 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:49.100 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:48 smithi175 ceph-mon[31920]: Adjusting osd_memory_target on smithi112 to 5696M 2024-04-21T08:33:49.100 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:48 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:49.100 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:48 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:49.101 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:48 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:33:49.101 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:48 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:49.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:48 smithi112 ceph-mon[31085]: purged_snaps scrub starts 2024-04-21T08:33:49.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:48 smithi112 ceph-mon[31085]: purged_snaps scrub ok 2024-04-21T08:33:49.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:48 smithi112 ceph-mon[31085]: osd.7 [v2:172.21.15.112:6816/1090541705,v1:172.21.15.112:6817/1090541705] boot 2024-04-21T08:33:49.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:48 smithi112 ceph-mon[31085]: osdmap e52: 8 total, 8 up, 8 in 2024-04-21T08:33:49.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:48 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:49.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:48 smithi112 ceph-mon[31085]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:49.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:48 smithi112 ceph-mon[31085]: Detected new or changed devices on smithi112 2024-04-21T08:33:49.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:48 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:49.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:48 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:49.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:48 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:49.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:48 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:49.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:48 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:49.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:48 smithi112 ceph-mon[31085]: Adjusting osd_memory_target on smithi112 to 5696M 2024-04-21T08:33:49.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:48 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:49.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:48 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:49.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:48 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:33:49.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:48 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:49.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:48 smithi043 ceph-mon[27978]: purged_snaps scrub starts 2024-04-21T08:33:49.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:48 smithi043 ceph-mon[27978]: purged_snaps scrub ok 2024-04-21T08:33:49.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:48 smithi043 ceph-mon[27978]: osd.7 [v2:172.21.15.112:6816/1090541705,v1:172.21.15.112:6817/1090541705] boot 2024-04-21T08:33:49.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:48 smithi043 ceph-mon[27978]: osdmap e52: 8 total, 8 up, 8 in 2024-04-21T08:33:49.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:48 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:49.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:48 smithi043 ceph-mon[27978]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:49.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:48 smithi043 ceph-mon[27978]: Detected new or changed devices on smithi112 2024-04-21T08:33:49.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:48 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:49.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:48 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:49.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:48 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:49.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:48 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:49.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:48 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:49.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:48 smithi043 ceph-mon[27978]: Adjusting osd_memory_target on smithi112 to 5696M 2024-04-21T08:33:49.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:48 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:49.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:48 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:49.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:48 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:33:49.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:48 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:33:50.122 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:49 smithi175 ceph-mon[31920]: osdmap e53: 8 total, 8 up, 8 in 2024-04-21T08:33:50.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:49 smithi112 ceph-mon[31085]: osdmap e53: 8 total, 8 up, 8 in 2024-04-21T08:33:50.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:49 smithi043 ceph-mon[27978]: osdmap e53: 8 total, 8 up, 8 in 2024-04-21T08:33:50.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.373+0000 7fa2ef8b7700 1 -- 172.21.15.175:0/278898058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2e8103530 msgr2=0x7fa2e8105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:33:50.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.373+0000 7fa2ef8b7700 1 --2- 172.21.15.175:0/278898058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2e8103530 0x7fa2e8105920 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc00b0a0 tx=0x7fa2dc00b3b0 comp rx=0 tx=0).stop 2024-04-21T08:33:50.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.373+0000 7fa2ef8b7700 1 -- 172.21.15.175:0/278898058 shutdown_connections 2024-04-21T08:33:50.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.373+0000 7fa2ef8b7700 1 --2- 172.21.15.175:0/278898058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2e8103530 0x7fa2e8105920 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:50.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.373+0000 7fa2ef8b7700 1 --2- 172.21.15.175:0/278898058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2e8108790 0x7fa2e810ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:50.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.373+0000 7fa2ef8b7700 1 --2- 172.21.15.175:0/278898058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2e8105e60 0x7fa2e8108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:50.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.373+0000 7fa2ef8b7700 1 -- 172.21.15.175:0/278898058 >> 172.21.15.175:0/278898058 conn(0x7fa2e80fcf60 msgr2=0x7fa2e80ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:33:50.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.373+0000 7fa2ef8b7700 1 -- 172.21.15.175:0/278898058 shutdown_connections 2024-04-21T08:33:50.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.373+0000 7fa2ef8b7700 1 -- 172.21.15.175:0/278898058 wait complete. 2024-04-21T08:33:50.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.374+0000 7fa2ef8b7700 1 Processor -- start 2024-04-21T08:33:50.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.374+0000 7fa2ef8b7700 1 -- start start 2024-04-21T08:33:50.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.374+0000 7fa2ef8b7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2e8103530 0x7fa2e8198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:33:50.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.374+0000 7fa2ef8b7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2e8105e60 0x7fa2e81990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:33:50.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.374+0000 7fa2ef8b7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2e8108790 0x7fa2e81a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:33:50.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.374+0000 7fa2ef8b7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e810e550 con 0x7fa2e8108790 2024-04-21T08:33:50.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.374+0000 7fa2ef8b7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e810e3d0 con 0x7fa2e8103530 2024-04-21T08:33:50.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.374+0000 7fa2ef8b7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e810e6d0 con 0x7fa2e8105e60 2024-04-21T08:33:50.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.374+0000 7fa2ece52700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2e8105e60 0x7fa2e81990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:33:50.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.374+0000 7fa2ece52700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2e8105e60 0x7fa2e81990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:36988/0 (socket says 172.21.15.175:36988) 2024-04-21T08:33:50.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.374+0000 7fa2ece52700 1 -- 172.21.15.175:0/422264984 learned_addr learned my addr 172.21.15.175:0/422264984 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-21T08:33:50.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.374+0000 7fa2ed653700 1 --2- 172.21.15.175:0/422264984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2e8103530 0x7fa2e8198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:33:50.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.374+0000 7fa2ede54700 1 --2- 172.21.15.175:0/422264984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2e8108790 0x7fa2e81a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:33:50.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.375+0000 7fa2ece52700 1 -- 172.21.15.175:0/422264984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2e8103530 msgr2=0x7fa2e8198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:33:50.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.375+0000 7fa2ece52700 1 --2- 172.21.15.175:0/422264984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2e8103530 0x7fa2e8198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:50.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.375+0000 7fa2ece52700 1 -- 172.21.15.175:0/422264984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2e8108790 msgr2=0x7fa2e81a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:33:50.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.375+0000 7fa2ece52700 1 --2- 172.21.15.175:0/422264984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2e8108790 0x7fa2e81a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:50.395 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.375+0000 7fa2ece52700 1 -- 172.21.15.175:0/422264984 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e81a07f0 con 0x7fa2e8105e60 2024-04-21T08:33:50.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.375+0000 7fa2ece52700 1 --2- 172.21.15.175:0/422264984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2e8105e60 0x7fa2e81990d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400d980 tx=0x7fa2d400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:33:50.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.375+0000 7fa2e67fc700 1 -- 172.21.15.175:0/422264984 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4016070 con 0x7fa2e8105e60 2024-04-21T08:33:50.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.375+0000 7fa2e67fc700 1 -- 172.21.15.175:0/422264984 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2d4009d80 con 0x7fa2e8105e60 2024-04-21T08:33:50.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.375+0000 7fa2ef8b7700 1 -- 172.21.15.175:0/422264984 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e81a0ae0 con 0x7fa2e8105e60 2024-04-21T08:33:50.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.375+0000 7fa2e67fc700 1 -- 172.21.15.175:0/422264984 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d400f3c0 con 0x7fa2e8105e60 2024-04-21T08:33:50.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.375+0000 7fa2ef8b7700 1 -- 172.21.15.175:0/422264984 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e81a1020 con 0x7fa2e8105e60 2024-04-21T08:33:50.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.376+0000 7fa2ef8b7700 1 -- 172.21.15.175:0/422264984 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2e8069730 con 0x7fa2e8105e60 2024-04-21T08:33:50.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.377+0000 7fa2e67fc700 1 -- 172.21.15.175:0/422264984 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa2d4019070 con 0x7fa2e8105e60 2024-04-21T08:33:50.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.378+0000 7fa2e67fc700 1 --2- 172.21.15.175:0/422264984 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa2d80a3c40 0x7fa2d80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:33:50.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.378+0000 7fa2e67fc700 1 -- 172.21.15.175:0/422264984 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7fa2d40cf2f0 con 0x7fa2e8105e60 2024-04-21T08:33:50.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.378+0000 7fa2ed653700 1 --2- 172.21.15.175:0/422264984 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa2d80a3c40 0x7fa2d80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:33:50.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.378+0000 7fa2ed653700 1 --2- 172.21.15.175:0/422264984 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa2d80a3c40 0x7fa2d80a6100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009fd0 tx=0x7fa2dc00b7b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:33:50.396 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.381+0000 7fa2e67fc700 1 -- 172.21.15.175:0/422264984 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa2d409c810 con 0x7fa2e8105e60 2024-04-21T08:33:50.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:33:50.586+0000 7fa2ef8b7700 1 -- 172.21.15.175:0/422264984 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa2e8062e80 con 0x7fa2d80a3c40 2024-04-21T08:33:51.197 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:50 smithi175 ceph-mon[31920]: osdmap e54: 8 total, 8 up, 8 in 2024-04-21T08:33:51.197 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:50 smithi175 ceph-mon[31920]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:51.197 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:50 smithi175 ceph-mon[31920]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:33:51.197 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:50 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:33:51.198 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:50 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:33:51.198 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:50 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:51.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:50 smithi112 ceph-mon[31085]: osdmap e54: 8 total, 8 up, 8 in 2024-04-21T08:33:51.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:50 smithi112 ceph-mon[31085]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:51.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:50 smithi112 ceph-mon[31085]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:33:51.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:50 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:33:51.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:50 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:33:51.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:50 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:51.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:50 smithi043 ceph-mon[27978]: osdmap e54: 8 total, 8 up, 8 in 2024-04-21T08:33:51.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:50 smithi043 ceph-mon[27978]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:51.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:50 smithi043 ceph-mon[27978]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:33:51.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:50 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:33:51.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:50 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:33:51.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:50 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:52.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:51 smithi175 ceph-mon[31920]: osdmap e55: 8 total, 8 up, 8 in 2024-04-21T08:33:52.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:51 smithi112 ceph-mon[31085]: osdmap e55: 8 total, 8 up, 8 in 2024-04-21T08:33:52.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:51 smithi043 ceph-mon[27978]: osdmap e55: 8 total, 8 up, 8 in 2024-04-21T08:33:53.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:52 smithi175 ceph-mon[31920]: osdmap e56: 8 total, 8 up, 8 in 2024-04-21T08:33:53.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:52 smithi175 ceph-mon[31920]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:53.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:52 smithi112 ceph-mon[31085]: osdmap e56: 8 total, 8 up, 8 in 2024-04-21T08:33:53.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:52 smithi112 ceph-mon[31085]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:53.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:52 smithi043 ceph-mon[27978]: osdmap e56: 8 total, 8 up, 8 in 2024-04-21T08:33:53.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:52 smithi043 ceph-mon[27978]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:54.114 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:53 smithi175 ceph-mon[31920]: from='client.? 172.21.15.175:0/3799691966' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "277e3276-9d50-4233-9a4d-74b1a321f308"}]: dispatch 2024-04-21T08:33:54.115 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:53 smithi175 ceph-mon[31920]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "277e3276-9d50-4233-9a4d-74b1a321f308"}]: dispatch 2024-04-21T08:33:54.115 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:53 smithi175 ceph-mon[31920]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "277e3276-9d50-4233-9a4d-74b1a321f308"}]': finished 2024-04-21T08:33:54.115 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:53 smithi175 ceph-mon[31920]: osdmap e57: 9 total, 8 up, 9 in 2024-04-21T08:33:54.115 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:53 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:33:54.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:53 smithi112 ceph-mon[31085]: from='client.? 172.21.15.175:0/3799691966' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "277e3276-9d50-4233-9a4d-74b1a321f308"}]: dispatch 2024-04-21T08:33:54.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:53 smithi112 ceph-mon[31085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "277e3276-9d50-4233-9a4d-74b1a321f308"}]: dispatch 2024-04-21T08:33:54.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:53 smithi112 ceph-mon[31085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "277e3276-9d50-4233-9a4d-74b1a321f308"}]': finished 2024-04-21T08:33:54.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:53 smithi112 ceph-mon[31085]: osdmap e57: 9 total, 8 up, 9 in 2024-04-21T08:33:54.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:53 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:33:54.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:53 smithi043 ceph-mon[27978]: from='client.? 172.21.15.175:0/3799691966' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "277e3276-9d50-4233-9a4d-74b1a321f308"}]: dispatch 2024-04-21T08:33:54.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:53 smithi043 ceph-mon[27978]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "277e3276-9d50-4233-9a4d-74b1a321f308"}]: dispatch 2024-04-21T08:33:54.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:53 smithi043 ceph-mon[27978]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "277e3276-9d50-4233-9a4d-74b1a321f308"}]': finished 2024-04-21T08:33:54.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:53 smithi043 ceph-mon[27978]: osdmap e57: 9 total, 8 up, 9 in 2024-04-21T08:33:54.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:53 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:33:55.195 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:54 smithi112 ceph-mon[31085]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:55.195 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:54 smithi112 ceph-mon[31085]: from='client.? 172.21.15.175:0/552049459' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:33:55.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:54 smithi175 ceph-mon[31920]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:55.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:54 smithi175 ceph-mon[31920]: from='client.? 172.21.15.175:0/552049459' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:33:55.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:54 smithi043 ceph-mon[27978]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:55.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:54 smithi043 ceph-mon[27978]: from='client.? 172.21.15.175:0/552049459' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:33:57.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:56 smithi112 ceph-mon[31085]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:57.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:56 smithi043 ceph-mon[27978]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:57.416 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:56 smithi175 ceph-mon[31920]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:59.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:33:59 smithi112 ceph-mon[31085]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:59.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:33:59 smithi043 ceph-mon[27978]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:59.315 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:33:59 smithi175 ceph-mon[31920]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:01.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:01 smithi175 ceph-mon[31920]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:01.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:01 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-21T08:34:01.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:01 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:01.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:01 smithi112 ceph-mon[31085]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:01.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:01 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-21T08:34:01.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:01 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:01.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:01 smithi043 ceph-mon[27978]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:01.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:01 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-21T08:34:01.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:01 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:02.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:02 smithi112 ceph-mon[31085]: Deploying daemon osd.8 on smithi175 2024-04-21T08:34:02.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:02 smithi043 ceph-mon[27978]: Deploying daemon osd.8 on smithi175 2024-04-21T08:34:02.388 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:02 smithi175 ceph-mon[31920]: Deploying daemon osd.8 on smithi175 2024-04-21T08:34:03.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:03 smithi112 ceph-mon[31085]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:03.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:03 smithi043 ceph-mon[27978]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:03.444 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:03 smithi175 ceph-mon[31920]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:05.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:05 smithi112 ceph-mon[31085]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:05.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:05 smithi043 ceph-mon[27978]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:05.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:05 smithi175 ceph-mon[31920]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:07.148 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:07 smithi175 ceph-mon[31920]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:07.149 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:07 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:07.149 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:07 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:07.149 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:07 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:07.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:07 smithi112 ceph-mon[31085]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:07.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:07 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:07.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:07 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:07.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:07 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:07.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:07 smithi043 ceph-mon[27978]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:07.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:07 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:07.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:07 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:07.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:07 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:09.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:09 smithi175 ceph-mon[31920]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:09.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:09 smithi112 ceph-mon[31085]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:09.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:09 smithi043 ceph-mon[27978]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:10.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:34:10.106+0000 7fa2e67fc700 1 -- 172.21.15.175:0/422264984 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa2e8062e80 con 0x7fa2d80a3c40 2024-04-21T08:34:10.107 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 8 on host 'smithi175' 2024-04-21T08:34:10.108 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:34:10.109+0000 7fa2ef8b7700 1 -- 172.21.15.175:0/422264984 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa2d80a3c40 msgr2=0x7fa2d80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:34:10.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:34:10.109+0000 7fa2ef8b7700 1 --2- 172.21.15.175:0/422264984 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa2d80a3c40 0x7fa2d80a6100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009fd0 tx=0x7fa2dc00b7b0 comp rx=0 tx=0).stop 2024-04-21T08:34:10.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:34:10.109+0000 7fa2ef8b7700 1 -- 172.21.15.175:0/422264984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2e8105e60 msgr2=0x7fa2e81990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:34:10.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:34:10.109+0000 7fa2ef8b7700 1 --2- 172.21.15.175:0/422264984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2e8105e60 0x7fa2e81990d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400d980 tx=0x7fa2d400dc90 comp rx=0 tx=0).stop 2024-04-21T08:34:10.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:34:10.109+0000 7fa2ef8b7700 1 -- 172.21.15.175:0/422264984 shutdown_connections 2024-04-21T08:34:10.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:34:10.109+0000 7fa2ef8b7700 1 --2- 172.21.15.175:0/422264984 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa2d80a3c40 0x7fa2d80a6100 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:10.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:34:10.109+0000 7fa2ef8b7700 1 --2- 172.21.15.175:0/422264984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2e8108790 0x7fa2e81a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:10.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:34:10.109+0000 7fa2ef8b7700 1 --2- 172.21.15.175:0/422264984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2e8103530 0x7fa2e8198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:10.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:34:10.109+0000 7fa2ef8b7700 1 --2- 172.21.15.175:0/422264984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2e8105e60 0x7fa2e81990d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:10.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:34:10.109+0000 7fa2ef8b7700 1 -- 172.21.15.175:0/422264984 >> 172.21.15.175:0/422264984 conn(0x7fa2e80fcf60 msgr2=0x7fa2e8103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:34:10.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:34:10.109+0000 7fa2ef8b7700 1 -- 172.21.15.175:0/422264984 shutdown_connections 2024-04-21T08:34:10.109 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:34:10.109+0000 7fa2ef8b7700 1 -- 172.21.15.175:0/422264984 wait complete. 2024-04-21T08:34:10.795 DEBUG:teuthology.orchestra.run.smithi175:osd.8> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.8.service 2024-04-21T08:34:10.798 INFO:tasks.cephadm:Deploying osd.9 on smithi043 with /dev/vg_nvme/lv_1... 2024-04-21T08:34:10.798 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-21T08:34:11.072 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:34:11.195 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:11 smithi043 ceph-mon[27978]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:11.195 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:11 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:11.195 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:11 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:11.195 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:11 smithi043 ceph-mon[27978]: from='osd.8 [v2:172.21.15.175:6816/1369698308,v1:172.21.15.175:6817/1369698308]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T08:34:11.195 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:11 smithi043 ceph-mon[27978]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T08:34:11.195 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:11 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:11.195 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:11 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:11.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:11 smithi175 ceph-mon[31920]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:11.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:11 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:11.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:11 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:11.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:11 smithi175 ceph-mon[31920]: from='osd.8 [v2:172.21.15.175:6816/1369698308,v1:172.21.15.175:6817/1369698308]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T08:34:11.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:11 smithi175 ceph-mon[31920]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T08:34:11.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:11 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:11.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:11 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:11.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:11 smithi112 ceph-mon[31085]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:11.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:11 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:11.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:11 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:11.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:11 smithi112 ceph-mon[31085]: from='osd.8 [v2:172.21.15.175:6816/1369698308,v1:172.21.15.175:6817/1369698308]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T08:34:11.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:11 smithi112 ceph-mon[31085]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T08:34:11.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:11 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:11.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:11 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:12.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:12 smithi043 ceph-mon[27978]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-21T08:34:12.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:12 smithi043 ceph-mon[27978]: osdmap e58: 9 total, 8 up, 9 in 2024-04-21T08:34:12.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:12 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:12.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:12 smithi043 ceph-mon[27978]: from='osd.8 [v2:172.21.15.175:6816/1369698308,v1:172.21.15.175:6817/1369698308]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-04-21T08:34:12.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:12 smithi043 ceph-mon[27978]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-04-21T08:34:12.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:12 smithi175 ceph-mon[31920]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-21T08:34:12.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:12 smithi175 ceph-mon[31920]: osdmap e58: 9 total, 8 up, 9 in 2024-04-21T08:34:12.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:12 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:12.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:12 smithi175 ceph-mon[31920]: from='osd.8 [v2:172.21.15.175:6816/1369698308,v1:172.21.15.175:6817/1369698308]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-04-21T08:34:12.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:12 smithi175 ceph-mon[31920]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-04-21T08:34:12.464 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:34:12 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:34:12.071+0000 7f7bcebbd700 -1 osd.8 0 waiting for initial osdmap 2024-04-21T08:34:12.464 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:34:12 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:34:12.082+0000 7f7bca9b0700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:34:12.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:12 smithi112 ceph-mon[31085]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-21T08:34:12.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:12 smithi112 ceph-mon[31085]: osdmap e58: 9 total, 8 up, 9 in 2024-04-21T08:34:12.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:12 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:12.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:12 smithi112 ceph-mon[31085]: from='osd.8 [v2:172.21.15.175:6816/1369698308,v1:172.21.15.175:6817/1369698308]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-04-21T08:34:12.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:12 smithi112 ceph-mon[31085]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-04-21T08:34:13.407 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-21T08:34:13.408 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-21T08:34:13.408 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-04-21T08:34:13.408 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-04-21T08:34:13.408 INFO:teuthology.orchestra.run.smithi043.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0084241 s, 1.2 GB/s 2024-04-21T08:34:13.408 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-04-21T08:34:13.408 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:13 smithi043 ceph-mon[27978]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:13.408 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:13 smithi043 ceph-mon[27978]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-04-21T08:34:13.408 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:13 smithi043 ceph-mon[27978]: osdmap e59: 9 total, 8 up, 9 in 2024-04-21T08:34:13.408 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:13 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:13.408 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:13 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:13.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:13 smithi175 ceph-mon[31920]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:13.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:13 smithi175 ceph-mon[31920]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-04-21T08:34:13.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:13 smithi175 ceph-mon[31920]: osdmap e59: 9 total, 8 up, 9 in 2024-04-21T08:34:13.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:13 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:13.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:13 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:13.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:13 smithi112 ceph-mon[31085]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:13.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:13 smithi112 ceph-mon[31085]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-04-21T08:34:13.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:13 smithi112 ceph-mon[31085]: osdmap e59: 9 total, 8 up, 9 in 2024-04-21T08:34:13.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:13 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:13.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:13 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:14.157 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_1 2024-04-21T08:34:14.308 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:14 smithi175 ceph-mon[31920]: purged_snaps scrub starts 2024-04-21T08:34:14.309 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:14 smithi175 ceph-mon[31920]: purged_snaps scrub ok 2024-04-21T08:34:14.309 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:14.309 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:14 smithi175 ceph-mon[31920]: osd.8 [v2:172.21.15.175:6816/1369698308,v1:172.21.15.175:6817/1369698308] boot 2024-04-21T08:34:14.309 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:14 smithi175 ceph-mon[31920]: osdmap e60: 9 total, 9 up, 9 in 2024-04-21T08:34:14.309 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:14.412 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:34:14.539 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:14 smithi043 ceph-mon[27978]: purged_snaps scrub starts 2024-04-21T08:34:14.539 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:14 smithi043 ceph-mon[27978]: purged_snaps scrub ok 2024-04-21T08:34:14.539 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:14.540 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:14 smithi043 ceph-mon[27978]: osd.8 [v2:172.21.15.175:6816/1369698308,v1:172.21.15.175:6817/1369698308] boot 2024-04-21T08:34:14.540 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:14 smithi043 ceph-mon[27978]: osdmap e60: 9 total, 9 up, 9 in 2024-04-21T08:34:14.540 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:14.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:14 smithi112 ceph-mon[31085]: purged_snaps scrub starts 2024-04-21T08:34:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:14 smithi112 ceph-mon[31085]: purged_snaps scrub ok 2024-04-21T08:34:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:14 smithi112 ceph-mon[31085]: osd.8 [v2:172.21.15.175:6816/1369698308,v1:172.21.15.175:6817/1369698308] boot 2024-04-21T08:34:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:14 smithi112 ceph-mon[31085]: osdmap e60: 9 total, 9 up, 9 in 2024-04-21T08:34:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:15.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:15 smithi175 ceph-mon[31920]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:15.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:15 smithi175 ceph-mon[31920]: osdmap e61: 9 total, 9 up, 9 in 2024-04-21T08:34:15.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:15 smithi175 ceph-mon[31920]: Detected new or changed devices on smithi175 2024-04-21T08:34:15.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:15.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:15.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:15.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:15.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:15.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:15 smithi175 ceph-mon[31920]: Adjusting osd_memory_target on smithi175 to 5696M 2024-04-21T08:34:15.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:15.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:15.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:34:15.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:15.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:15.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:15.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:34:15.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:15.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:15 smithi112 ceph-mon[31085]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:15.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:15 smithi112 ceph-mon[31085]: osdmap e61: 9 total, 9 up, 9 in 2024-04-21T08:34:15.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:15 smithi112 ceph-mon[31085]: Detected new or changed devices on smithi175 2024-04-21T08:34:15.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:15.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:15.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:15.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:15.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:15.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:15 smithi112 ceph-mon[31085]: Adjusting osd_memory_target on smithi175 to 5696M 2024-04-21T08:34:15.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:15.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:15.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:34:15.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:15.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:15.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:15.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:34:15.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:15.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:15 smithi043 ceph-mon[27978]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:15.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:15 smithi043 ceph-mon[27978]: osdmap e61: 9 total, 9 up, 9 in 2024-04-21T08:34:15.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:15 smithi043 ceph-mon[27978]: Detected new or changed devices on smithi175 2024-04-21T08:34:15.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:15.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:15.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:15.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:15.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:15.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:15 smithi043 ceph-mon[27978]: Adjusting osd_memory_target on smithi175 to 5696M 2024-04-21T08:34:15.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:15.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:15.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:34:15.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:15.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:15.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:15.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:34:15.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:16.257 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:16 smithi043 ceph-mon[27978]: osdmap e62: 9 total, 9 up, 9 in 2024-04-21T08:34:16.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:16 smithi175 ceph-mon[31920]: osdmap e62: 9 total, 9 up, 9 in 2024-04-21T08:34:16.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.446+0000 7f56ff5f1700 1 -- 172.21.15.43:0/4289416623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f8103cc0 msgr2=0x7f56f8104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:34:16.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.446+0000 7f56ff5f1700 1 --2- 172.21.15.43:0/4289416623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f8103cc0 0x7f56f8104120 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f56e80099c0 tx=0x7f56e8009cd0 comp rx=0 tx=0).stop 2024-04-21T08:34:16.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.446+0000 7f56ff5f1700 1 -- 172.21.15.43:0/4289416623 shutdown_connections 2024-04-21T08:34:16.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.446+0000 7f56ff5f1700 1 --2- 172.21.15.43:0/4289416623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56f8104660 0x7f56f810af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:16.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.446+0000 7f56ff5f1700 1 --2- 172.21.15.43:0/4289416623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56f8102ac0 0x7f56f8102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:16.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.446+0000 7f56ff5f1700 1 --2- 172.21.15.43:0/4289416623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f8103cc0 0x7f56f8104120 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:16.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.446+0000 7f56ff5f1700 1 -- 172.21.15.43:0/4289416623 >> 172.21.15.43:0/4289416623 conn(0x7f56f80fe060 msgr2=0x7f56f81004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:34:16.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.446+0000 7f56ff5f1700 1 -- 172.21.15.43:0/4289416623 shutdown_connections 2024-04-21T08:34:16.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.446+0000 7f56ff5f1700 1 -- 172.21.15.43:0/4289416623 wait complete. 2024-04-21T08:34:16.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.447+0000 7f56ff5f1700 1 Processor -- start 2024-04-21T08:34:16.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.447+0000 7f56ff5f1700 1 -- start start 2024-04-21T08:34:16.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.447+0000 7f56ff5f1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56f8102ac0 0x7f56f8198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:34:16.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.447+0000 7f56ff5f1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f8103cc0 0x7f56f8199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:34:16.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.447+0000 7f56ff5f1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56f8104660 0x7f56f81a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:34:16.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.447+0000 7f56ff5f1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f56f810e560 con 0x7f56f8103cc0 2024-04-21T08:34:16.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.447+0000 7f56ff5f1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f56f810e3e0 con 0x7f56f8102ac0 2024-04-21T08:34:16.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.447+0000 7f56ff5f1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f56f810e6e0 con 0x7f56f8104660 2024-04-21T08:34:16.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.447+0000 7f56fcb8c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f8103cc0 0x7f56f8199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:34:16.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.447+0000 7f56fcb8c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f8103cc0 0x7f56f8199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53804/0 (socket says 172.21.15.43:53804) 2024-04-21T08:34:16.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.447+0000 7f56fcb8c700 1 -- 172.21.15.43:0/2970791318 learned_addr learned my addr 172.21.15.43:0/2970791318 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:34:16.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.447+0000 7f56fd38d700 1 --2- 172.21.15.43:0/2970791318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56f8102ac0 0x7f56f8198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:34:16.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.447+0000 7f56fdb8e700 1 --2- 172.21.15.43:0/2970791318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56f8104660 0x7f56f81a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:34:16.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.447+0000 7f56fcb8c700 1 -- 172.21.15.43:0/2970791318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56f8104660 msgr2=0x7f56f81a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:34:16.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.447+0000 7f56fcb8c700 1 --2- 172.21.15.43:0/2970791318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56f8104660 0x7f56f81a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:16.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.447+0000 7f56fcb8c700 1 -- 172.21.15.43:0/2970791318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56f8102ac0 msgr2=0x7f56f8198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:34:16.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.447+0000 7f56fcb8c700 1 --2- 172.21.15.43:0/2970791318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56f8102ac0 0x7f56f8198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:16.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.447+0000 7f56fcb8c700 1 -- 172.21.15.43:0/2970791318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f81a0830 con 0x7f56f8103cc0 2024-04-21T08:34:16.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.447+0000 7f56fdb8e700 1 --2- 172.21.15.43:0/2970791318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56f8104660 0x7f56f81a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:34:16.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.447+0000 7f56fcb8c700 1 --2- 172.21.15.43:0/2970791318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f8103cc0 0x7f56f8199110 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f56e8005910 tx=0x7f56e8009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:34:16.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.448+0000 7f56ee7fc700 1 -- 172.21.15.43:0/2970791318 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8027070 con 0x7f56f8103cc0 2024-04-21T08:34:16.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.448+0000 7f56ee7fc700 1 -- 172.21.15.43:0/2970791318 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56e80041a0 con 0x7f56f8103cc0 2024-04-21T08:34:16.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.448+0000 7f56ff5f1700 1 -- 172.21.15.43:0/2970791318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f81a0ac0 con 0x7f56f8103cc0 2024-04-21T08:34:16.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.448+0000 7f56ee7fc700 1 -- 172.21.15.43:0/2970791318 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8022da0 con 0x7f56f8103cc0 2024-04-21T08:34:16.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.448+0000 7f56ff5f1700 1 -- 172.21.15.43:0/2970791318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f81a0fa0 con 0x7f56f8103cc0 2024-04-21T08:34:16.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.449+0000 7f56ff5f1700 1 -- 172.21.15.43:0/2970791318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56f8069730 con 0x7f56f8103cc0 2024-04-21T08:34:16.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.450+0000 7f56ee7fc700 1 -- 172.21.15.43:0/2970791318 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f56e802a070 con 0x7f56f8103cc0 2024-04-21T08:34:16.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.451+0000 7f56ee7fc700 1 --2- 172.21.15.43:0/2970791318 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f56e40a3c40 0x7f56e40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:34:16.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.451+0000 7f56fd38d700 1 --2- 172.21.15.43:0/2970791318 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f56e40a3c40 0x7f56e40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:34:16.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.451+0000 7f56ee7fc700 1 -- 172.21.15.43:0/2970791318 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7f56e80d1070 con 0x7f56f8103cc0 2024-04-21T08:34:16.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.451+0000 7f56fd38d700 1 --2- 172.21.15.43:0/2970791318 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f56e40a3c40 0x7f56e40a6100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f56f8103b20 tx=0x7f56f400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:34:16.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.455+0000 7f56ee7fc700 1 -- 172.21.15.43:0/2970791318 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56e8031020 con 0x7f56f8103cc0 2024-04-21T08:34:16.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:16 smithi112 ceph-mon[31085]: osdmap e62: 9 total, 9 up, 9 in 2024-04-21T08:34:16.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:16.674+0000 7f56ff5f1700 1 -- 172.21.15.43:0/2970791318 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f56f8062f40 con 0x7f56e40a3c40 2024-04-21T08:34:17.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:17 smithi175 ceph-mon[31920]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:17.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:17 smithi175 ceph-mon[31920]: osdmap e63: 9 total, 9 up, 9 in 2024-04-21T08:34:17.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:17 smithi175 ceph-mon[31920]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:34:17.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:17 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:34:17.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:17 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:34:17.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:17 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:17.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:17 smithi043 ceph-mon[27978]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:17.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:17 smithi043 ceph-mon[27978]: osdmap e63: 9 total, 9 up, 9 in 2024-04-21T08:34:17.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:17 smithi043 ceph-mon[27978]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:34:17.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:17 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:34:17.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:17 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:34:17.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:17 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:17.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:17 smithi112 ceph-mon[31085]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:17.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:17 smithi112 ceph-mon[31085]: osdmap e63: 9 total, 9 up, 9 in 2024-04-21T08:34:17.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:17 smithi112 ceph-mon[31085]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:34:17.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:17 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:34:17.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:17 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:34:17.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:17 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:18.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:18 smithi175 ceph-mon[31920]: osdmap e64: 9 total, 9 up, 9 in 2024-04-21T08:34:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:18 smithi112 ceph-mon[31085]: osdmap e64: 9 total, 9 up, 9 in 2024-04-21T08:34:18.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:18 smithi043 ceph-mon[27978]: osdmap e64: 9 total, 9 up, 9 in 2024-04-21T08:34:19.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:19 smithi043 ceph-mon[27978]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:19.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:19 smithi175 ceph-mon[31920]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:19.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:19 smithi112 ceph-mon[31085]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:20.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:20 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/4114823240' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f530826b-3c2f-411b-9795-f4b3c6053a23"}]: dispatch 2024-04-21T08:34:20.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:20 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/4114823240' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f530826b-3c2f-411b-9795-f4b3c6053a23"}]': finished 2024-04-21T08:34:20.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:20 smithi175 ceph-mon[31920]: osdmap e65: 10 total, 9 up, 10 in 2024-04-21T08:34:20.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:20 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:20.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:20 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/4114823240' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f530826b-3c2f-411b-9795-f4b3c6053a23"}]: dispatch 2024-04-21T08:34:20.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:20 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/4114823240' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f530826b-3c2f-411b-9795-f4b3c6053a23"}]': finished 2024-04-21T08:34:20.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:20 smithi112 ceph-mon[31085]: osdmap e65: 10 total, 9 up, 10 in 2024-04-21T08:34:20.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:20 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:20.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:20 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/4114823240' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f530826b-3c2f-411b-9795-f4b3c6053a23"}]: dispatch 2024-04-21T08:34:20.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:20 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/4114823240' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f530826b-3c2f-411b-9795-f4b3c6053a23"}]': finished 2024-04-21T08:34:20.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:20 smithi043 ceph-mon[27978]: osdmap e65: 10 total, 9 up, 10 in 2024-04-21T08:34:20.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:20 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:21.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:21 smithi175 ceph-mon[31920]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-04-21T08:34:21.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:21 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/982309520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:34:21.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:21 smithi112 ceph-mon[31085]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-04-21T08:34:21.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:21 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/982309520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:34:21.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:21 smithi043 ceph-mon[27978]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-04-21T08:34:21.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:21 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/982309520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:34:23.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:23 smithi175 ceph-mon[31920]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:34:23.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:23 smithi112 ceph-mon[31085]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:34:23.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:23 smithi043 ceph-mon[27978]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:34:25.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:25 smithi043 ceph-mon[27978]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-21T08:34:25.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:25 smithi175 ceph-mon[31920]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-21T08:34:25.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:25 smithi112 ceph-mon[31085]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-21T08:34:27.407 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:27 smithi043 ceph-mon[27978]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-21T08:34:27.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:27 smithi175 ceph-mon[31920]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-21T08:34:27.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:27 smithi112 ceph-mon[31085]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-21T08:34:28.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:28 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-21T08:34:28.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:28 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:28.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:28 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-21T08:34:28.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:28 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:28.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:28 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-21T08:34:28.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:28 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:29.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:29 smithi175 ceph-mon[31920]: Deploying daemon osd.9 on smithi043 2024-04-21T08:34:29.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:29 smithi175 ceph-mon[31920]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T08:34:29.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:29 smithi112 ceph-mon[31085]: Deploying daemon osd.9 on smithi043 2024-04-21T08:34:29.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:29 smithi112 ceph-mon[31085]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T08:34:29.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:29 smithi043 ceph-mon[27978]: Deploying daemon osd.9 on smithi043 2024-04-21T08:34:29.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:29 smithi043 ceph-mon[27978]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T08:34:31.383 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:31 smithi043 ceph-mon[27978]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:31.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:31 smithi175 ceph-mon[31920]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:31.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:31 smithi112 ceph-mon[31085]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:33.418 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:33 smithi043 ceph-mon[27978]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:33.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:33 smithi175 ceph-mon[31920]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:33.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:33 smithi112 ceph-mon[31085]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:35.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:35 smithi043 ceph-mon[27978]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:35.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:35 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:35.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:35 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:35.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:35 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:35.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:35 smithi175 ceph-mon[31920]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:35.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:35 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:35.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:35 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:35.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:35 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:35.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:35 smithi112 ceph-mon[31085]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:35.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:35 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:35.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:35 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:35.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:35 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:37.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:37 smithi043 ceph-mon[27978]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:37.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:37 smithi175 ceph-mon[31920]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:37.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:37 smithi112 ceph-mon[31085]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:37.937 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 9 on host 'smithi043' 2024-04-21T08:34:37.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:37.938+0000 7f56ee7fc700 1 -- 172.21.15.43:0/2970791318 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f56f8062f40 con 0x7f56e40a3c40 2024-04-21T08:34:37.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:37.940+0000 7f56ff5f1700 1 -- 172.21.15.43:0/2970791318 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f56e40a3c40 msgr2=0x7f56e40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:34:37.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:37.940+0000 7f56ff5f1700 1 --2- 172.21.15.43:0/2970791318 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f56e40a3c40 0x7f56e40a6100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f56f8103b20 tx=0x7f56f400b410 comp rx=0 tx=0).stop 2024-04-21T08:34:37.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:37.941+0000 7f56ff5f1700 1 -- 172.21.15.43:0/2970791318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f8103cc0 msgr2=0x7f56f8199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:34:37.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:37.941+0000 7f56ff5f1700 1 --2- 172.21.15.43:0/2970791318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f8103cc0 0x7f56f8199110 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f56e8005910 tx=0x7f56e8009510 comp rx=0 tx=0).stop 2024-04-21T08:34:37.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:37.941+0000 7f56ff5f1700 1 -- 172.21.15.43:0/2970791318 shutdown_connections 2024-04-21T08:34:37.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:37.941+0000 7f56ff5f1700 1 --2- 172.21.15.43:0/2970791318 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f56e40a3c40 0x7f56e40a6100 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:37.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:37.941+0000 7f56ff5f1700 1 --2- 172.21.15.43:0/2970791318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56f8104660 0x7f56f81a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:37.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:37.941+0000 7f56ff5f1700 1 --2- 172.21.15.43:0/2970791318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56f8102ac0 0x7f56f8198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:37.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:37.941+0000 7f56ff5f1700 1 --2- 172.21.15.43:0/2970791318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f8103cc0 0x7f56f8199110 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:37.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:37.941+0000 7f56ff5f1700 1 -- 172.21.15.43:0/2970791318 >> 172.21.15.43:0/2970791318 conn(0x7f56f80fe060 msgr2=0x7f56f8107040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:34:37.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:37.941+0000 7f56ff5f1700 1 -- 172.21.15.43:0/2970791318 shutdown_connections 2024-04-21T08:34:37.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:34:37.941+0000 7f56ff5f1700 1 -- 172.21.15.43:0/2970791318 wait complete. 2024-04-21T08:34:38.732 DEBUG:teuthology.orchestra.run.smithi043:osd.9> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.9.service 2024-04-21T08:34:38.735 INFO:tasks.cephadm:Deploying osd.10 on smithi112 with /dev/vg_nvme/lv_1... 2024-04-21T08:34:38.735 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-21T08:34:38.981 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:34:39.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:38 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:39.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:38 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:39.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:38 smithi112 ceph-mon[31085]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:39.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:38 smithi112 ceph-mon[31085]: from='osd.9 [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-21T08:34:39.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:38 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:39.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:38 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:39.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:38 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:39.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:38 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:39.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:38 smithi043 ceph-mon[27978]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:39.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:38 smithi043 ceph-mon[27978]: from='osd.9 [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-21T08:34:39.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:38 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:39.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:38 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:39.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:38 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:39.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:38 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:39.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:38 smithi175 ceph-mon[31920]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:39.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:38 smithi175 ceph-mon[31920]: from='osd.9 [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-21T08:34:39.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:38 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:39.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:38 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:40.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:39 smithi043 ceph-mon[27978]: from='osd.9 [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-21T08:34:40.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:39 smithi043 ceph-mon[27978]: osdmap e66: 10 total, 9 up, 10 in 2024-04-21T08:34:40.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:39 smithi043 ceph-mon[27978]: from='osd.9 [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-21T08:34:40.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:39 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:40.212 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:39 smithi112 ceph-mon[31085]: from='osd.9 [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-21T08:34:40.212 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:39 smithi112 ceph-mon[31085]: osdmap e66: 10 total, 9 up, 10 in 2024-04-21T08:34:40.212 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:39 smithi112 ceph-mon[31085]: from='osd.9 [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-21T08:34:40.212 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:39 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:40.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:39 smithi175 ceph-mon[31920]: from='osd.9 [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-21T08:34:40.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:39 smithi175 ceph-mon[31920]: osdmap e66: 10 total, 9 up, 10 in 2024-04-21T08:34:40.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:39 smithi175 ceph-mon[31920]: from='osd.9 [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-04-21T08:34:40.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:39 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:40.728 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:34:40 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:34:40.445+0000 7fe60033f700 -1 osd.9 0 waiting for initial osdmap 2024-04-21T08:34:40.728 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:34:40 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:34:40.456+0000 7fe5f7929700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:34:40.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:40 smithi043 ceph-mon[27978]: from='osd.9 [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-21T08:34:40.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:40 smithi043 ceph-mon[27978]: osdmap e67: 10 total, 9 up, 10 in 2024-04-21T08:34:40.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:40 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:40.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:40 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:40.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:40 smithi043 ceph-mon[27978]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:40.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:40 smithi043 ceph-mon[27978]: from='osd.9 [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011]' entity='osd.9' 2024-04-21T08:34:41.125 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:40 smithi112 ceph-mon[31085]: from='osd.9 [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-21T08:34:41.125 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:40 smithi112 ceph-mon[31085]: osdmap e67: 10 total, 9 up, 10 in 2024-04-21T08:34:41.125 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:40 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:41.126 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:40 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:41.126 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:40 smithi112 ceph-mon[31085]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:41.126 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:40 smithi112 ceph-mon[31085]: from='osd.9 [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011]' entity='osd.9' 2024-04-21T08:34:41.126 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-21T08:34:41.126 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-21T08:34:41.126 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-04-21T08:34:41.126 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-04-21T08:34:41.126 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114349 s, 917 MB/s 2024-04-21T08:34:41.126 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-04-21T08:34:41.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:40 smithi175 ceph-mon[31920]: from='osd.9 [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-04-21T08:34:41.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:40 smithi175 ceph-mon[31920]: osdmap e67: 10 total, 9 up, 10 in 2024-04-21T08:34:41.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:40 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:41.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:40 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:41.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:40 smithi175 ceph-mon[31920]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:41.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:40 smithi175 ceph-mon[31920]: from='osd.9 [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011]' entity='osd.9' 2024-04-21T08:34:41.872 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_1 2024-04-21T08:34:41.923 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:41 smithi043 ceph-mon[27978]: purged_snaps scrub starts 2024-04-21T08:34:41.923 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:41 smithi043 ceph-mon[27978]: purged_snaps scrub ok 2024-04-21T08:34:42.119 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:34:42.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:41 smithi175 ceph-mon[31920]: purged_snaps scrub starts 2024-04-21T08:34:42.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:41 smithi175 ceph-mon[31920]: purged_snaps scrub ok 2024-04-21T08:34:42.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:41 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:42.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:41 smithi175 ceph-mon[31920]: osd.9 [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011] boot 2024-04-21T08:34:42.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:41 smithi175 ceph-mon[31920]: osdmap e68: 10 total, 10 up, 10 in 2024-04-21T08:34:42.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:41 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:42.242 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:41 smithi112 ceph-mon[31085]: purged_snaps scrub starts 2024-04-21T08:34:42.242 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:41 smithi112 ceph-mon[31085]: purged_snaps scrub ok 2024-04-21T08:34:42.242 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:41 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:42.242 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:41 smithi112 ceph-mon[31085]: osd.9 [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011] boot 2024-04-21T08:34:42.242 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:41 smithi112 ceph-mon[31085]: osdmap e68: 10 total, 10 up, 10 in 2024-04-21T08:34:42.242 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:41 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:42.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:41 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:42.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:41 smithi043 ceph-mon[27978]: osd.9 [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011] boot 2024-04-21T08:34:42.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:41 smithi043 ceph-mon[27978]: osdmap e68: 10 total, 10 up, 10 in 2024-04-21T08:34:42.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:41 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:43.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:43 smithi112 ceph-mon[31085]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:43.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:43 smithi112 ceph-mon[31085]: osdmap e69: 10 total, 10 up, 10 in 2024-04-21T08:34:43.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:43 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:43.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:43 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:43.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:43 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:43.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:43 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:43.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:43 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:43.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:43 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:43.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:43 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:43.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:43 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:43.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:43 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:34:43.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:43 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:43.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:43 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:43.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:43 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:43.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:43 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:34:43.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:43 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:43.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:43 smithi175 ceph-mon[31920]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:43.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:43 smithi175 ceph-mon[31920]: osdmap e69: 10 total, 10 up, 10 in 2024-04-21T08:34:43.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:43 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:43.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:43 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:43.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:43 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:43.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:43 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:43.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:43 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:43.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:43 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:43.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:43 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:43.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:43 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:43.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:43 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:34:43.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:43 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:43.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:43 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:43.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:43 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:43.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:43 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:34:43.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:43 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:43.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.776+0000 7f18331da700 1 -- 172.21.15.112:0/3751166043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f182c103ca0 msgr2=0x7f182c104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:34:43.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.776+0000 7f18331da700 1 --2- 172.21.15.112:0/3751166043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f182c103ca0 0x7f182c104100 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f18180099c0 tx=0x7f1818009cd0 comp rx=0 tx=0).stop 2024-04-21T08:34:43.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.777+0000 7f18331da700 1 -- 172.21.15.112:0/3751166043 shutdown_connections 2024-04-21T08:34:43.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.777+0000 7f18331da700 1 --2- 172.21.15.112:0/3751166043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f182c102aa0 0x7f182c102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:43.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.777+0000 7f18331da700 1 --2- 172.21.15.112:0/3751166043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182c104640 0x7f182c10aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:43.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.777+0000 7f18331da700 1 --2- 172.21.15.112:0/3751166043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f182c103ca0 0x7f182c104100 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.777+0000 7f18331da700 1 -- 172.21.15.112:0/3751166043 >> 172.21.15.112:0/3751166043 conn(0x7f182c0fe040 msgr2=0x7f182c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:34:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.777+0000 7f18331da700 1 -- 172.21.15.112:0/3751166043 shutdown_connections 2024-04-21T08:34:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.777+0000 7f18331da700 1 -- 172.21.15.112:0/3751166043 wait complete. 2024-04-21T08:34:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.777+0000 7f18331da700 1 Processor -- start 2024-04-21T08:34:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.777+0000 7f18331da700 1 -- start start 2024-04-21T08:34:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.777+0000 7f18331da700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182c102aa0 0x7f182c0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:34:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.777+0000 7f18331da700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f182c103ca0 0x7f182c074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:34:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.777+0000 7f18331da700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f182c104640 0x7f182c075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:34:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.777+0000 7f18331da700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f182c10e530 con 0x7f182c102aa0 2024-04-21T08:34:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.778+0000 7f18331da700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f182c10e3b0 con 0x7f182c103ca0 2024-04-21T08:34:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.778+0000 7f18331da700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f182c10e6b0 con 0x7f182c104640 2024-04-21T08:34:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.778+0000 7f182bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f182c103ca0 0x7f182c074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:34:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.778+0000 7f182bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f182c103ca0 0x7f182c074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39154/0 (socket says 172.21.15.112:39154) 2024-04-21T08:34:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.778+0000 7f182bfff700 1 -- 172.21.15.112:0/2564101997 learned_addr learned my addr 172.21.15.112:0/2564101997 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-21T08:34:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.778+0000 7f1830f76700 1 --2- 172.21.15.112:0/2564101997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182c102aa0 0x7f182c0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:34:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.778+0000 7f1831777700 1 --2- 172.21.15.112:0/2564101997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f182c104640 0x7f182c075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:34:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.778+0000 7f182bfff700 1 -- 172.21.15.112:0/2564101997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f182c104640 msgr2=0x7f182c075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:34:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.778+0000 7f182bfff700 1 --2- 172.21.15.112:0/2564101997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f182c104640 0x7f182c075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.778+0000 7f182bfff700 1 -- 172.21.15.112:0/2564101997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182c102aa0 msgr2=0x7f182c0767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:34:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.778+0000 7f182bfff700 1 --2- 172.21.15.112:0/2564101997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182c102aa0 0x7f182c0767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.778+0000 7f182bfff700 1 -- 172.21.15.112:0/2564101997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f182c075c20 con 0x7f182c103ca0 2024-04-21T08:34:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.778+0000 7f182bfff700 1 --2- 172.21.15.112:0/2564101997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f182c103ca0 0x7f182c074e30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1818003790 tx=0x7f1818005b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:34:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.778+0000 7f1829ffb700 1 -- 172.21.15.112:0/2564101997 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1818027070 con 0x7f182c103ca0 2024-04-21T08:34:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.778+0000 7f1829ffb700 1 -- 172.21.15.112:0/2564101997 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18180040e0 con 0x7f182c103ca0 2024-04-21T08:34:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.778+0000 7f1829ffb700 1 -- 172.21.15.112:0/2564101997 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1818022d60 con 0x7f182c103ca0 2024-04-21T08:34:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.778+0000 7f18331da700 1 -- 172.21.15.112:0/2564101997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f182c1a2d80 con 0x7f182c103ca0 2024-04-21T08:34:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.778+0000 7f18331da700 1 -- 172.21.15.112:0/2564101997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f182c1a31d0 con 0x7f182c103ca0 2024-04-21T08:34:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.779+0000 7f18331da700 1 -- 172.21.15.112:0/2564101997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f182c069730 con 0x7f182c103ca0 2024-04-21T08:34:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.781+0000 7f1829ffb700 1 -- 172.21.15.112:0/2564101997 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f181802a070 con 0x7f182c103ca0 2024-04-21T08:34:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.782+0000 7f1829ffb700 1 --2- 172.21.15.112:0/2564101997 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f181c0a3c40 0x7f181c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:34:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.782+0000 7f1829ffb700 1 -- 172.21.15.112:0/2564101997 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7f18180d0b50 con 0x7f182c103ca0 2024-04-21T08:34:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.782+0000 7f1830f76700 1 --2- 172.21.15.112:0/2564101997 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f181c0a3c40 0x7f181c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:34:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.783+0000 7f1830f76700 1 --2- 172.21.15.112:0/2564101997 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f181c0a3c40 0x7f181c0a6100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f182c103b00 tx=0x7f182000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:34:43.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.785+0000 7f1829ffb700 1 -- 172.21.15.112:0/2564101997 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1818013070 con 0x7f182c103ca0 2024-04-21T08:34:43.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:43 smithi043 ceph-mon[27978]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:43.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:43 smithi043 ceph-mon[27978]: osdmap e69: 10 total, 10 up, 10 in 2024-04-21T08:34:43.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:43 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:43.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:43 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:43.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:43 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:43.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:43 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:43.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:43 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:43.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:43 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:43.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:43 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:43.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:43 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:43.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:43 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:34:43.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:43 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:43.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:43 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:43.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:43 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:43.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:43 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:34:43.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:43 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:34:43.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:34:43.993+0000 7f18331da700 1 -- 172.21.15.112:0/2564101997 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f182c062e80 con 0x7f181c0a3c40 2024-04-21T08:34:44.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:44 smithi175 ceph-mon[31920]: Detected new or changed devices on smithi043 2024-04-21T08:34:44.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:44 smithi175 ceph-mon[31920]: Adjusting osd_memory_target on smithi043 to 4273M 2024-04-21T08:34:44.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:44 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:34:44.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:44 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:34:44.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:44 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:44.756 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:44 smithi112 ceph-mon[31085]: Detected new or changed devices on smithi043 2024-04-21T08:34:44.756 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:44 smithi112 ceph-mon[31085]: Adjusting osd_memory_target on smithi043 to 4273M 2024-04-21T08:34:44.756 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:44 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:34:44.757 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:44 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:34:44.757 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:44 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:44.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:44 smithi043 ceph-mon[27978]: Detected new or changed devices on smithi043 2024-04-21T08:34:44.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:44 smithi043 ceph-mon[27978]: Adjusting osd_memory_target on smithi043 to 4273M 2024-04-21T08:34:44.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:44 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:34:44.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:44 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:34:44.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:44 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:45.711 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:45 smithi112 ceph-mon[31085]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:45.711 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:45 smithi112 ceph-mon[31085]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:34:45.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:45 smithi175 ceph-mon[31920]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:45.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:45 smithi175 ceph-mon[31920]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:34:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:45 smithi043 ceph-mon[27978]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:45 smithi043 ceph-mon[27978]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:34:47.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:47 smithi175 ceph-mon[31920]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:47.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:47 smithi175 ceph-mon[31920]: from='client.? 172.21.15.112:0/3163207517' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "307394bf-0915-448c-b025-42db9d0691c0"}]: dispatch 2024-04-21T08:34:47.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:47 smithi175 ceph-mon[31920]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "307394bf-0915-448c-b025-42db9d0691c0"}]: dispatch 2024-04-21T08:34:47.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:47 smithi175 ceph-mon[31920]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "307394bf-0915-448c-b025-42db9d0691c0"}]': finished 2024-04-21T08:34:47.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:47 smithi175 ceph-mon[31920]: osdmap e70: 11 total, 10 up, 11 in 2024-04-21T08:34:47.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:47 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:34:47.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:47 smithi175 ceph-mon[31920]: from='client.? 172.21.15.112:0/1611959095' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:34:47.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:47 smithi112 ceph-mon[31085]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:47.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:47 smithi112 ceph-mon[31085]: from='client.? 172.21.15.112:0/3163207517' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "307394bf-0915-448c-b025-42db9d0691c0"}]: dispatch 2024-04-21T08:34:47.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:47 smithi112 ceph-mon[31085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "307394bf-0915-448c-b025-42db9d0691c0"}]: dispatch 2024-04-21T08:34:47.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:47 smithi112 ceph-mon[31085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "307394bf-0915-448c-b025-42db9d0691c0"}]': finished 2024-04-21T08:34:47.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:47 smithi112 ceph-mon[31085]: osdmap e70: 11 total, 10 up, 11 in 2024-04-21T08:34:47.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:47 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:34:47.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:47 smithi112 ceph-mon[31085]: from='client.? 172.21.15.112:0/1611959095' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:34:47.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:47 smithi043 ceph-mon[27978]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:47.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:47 smithi043 ceph-mon[27978]: from='client.? 172.21.15.112:0/3163207517' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "307394bf-0915-448c-b025-42db9d0691c0"}]: dispatch 2024-04-21T08:34:47.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:47 smithi043 ceph-mon[27978]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "307394bf-0915-448c-b025-42db9d0691c0"}]: dispatch 2024-04-21T08:34:47.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:47 smithi043 ceph-mon[27978]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "307394bf-0915-448c-b025-42db9d0691c0"}]': finished 2024-04-21T08:34:47.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:47 smithi043 ceph-mon[27978]: osdmap e70: 11 total, 10 up, 11 in 2024-04-21T08:34:47.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:47 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:34:47.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:47 smithi043 ceph-mon[27978]: from='client.? 172.21.15.112:0/1611959095' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:34:49.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:49 smithi175 ceph-mon[31920]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:49.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:49 smithi112 ceph-mon[31085]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:49.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:49 smithi043 ceph-mon[27978]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:51.715 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:51 smithi112 ceph-mon[31085]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:51.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:51 smithi043 ceph-mon[27978]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:51.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:51 smithi175 ceph-mon[31920]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:53.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:53 smithi112 ceph-mon[31085]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:53.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:53 smithi043 ceph-mon[27978]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:53.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:53 smithi175 ceph-mon[31920]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:54.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:54 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-21T08:34:54.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:54 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:54.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:54 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-21T08:34:54.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:54 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:54.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:54 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-21T08:34:54.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:54 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:55.702 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:55 smithi112 ceph-mon[31085]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:55.702 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:55 smithi112 ceph-mon[31085]: Deploying daemon osd.10 on smithi112 2024-04-21T08:34:55.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:55 smithi043 ceph-mon[27978]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:55.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:55 smithi043 ceph-mon[27978]: Deploying daemon osd.10 on smithi112 2024-04-21T08:34:55.866 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:55 smithi175 ceph-mon[31920]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:55.866 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:55 smithi175 ceph-mon[31920]: Deploying daemon osd.10 on smithi112 2024-04-21T08:34:57.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:57 smithi112 ceph-mon[31085]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:57.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:57 smithi043 ceph-mon[27978]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:57.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:57 smithi175 ceph-mon[31920]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:59.524 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:34:59 smithi112 ceph-mon[31085]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:59.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:34:59 smithi043 ceph-mon[27978]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:59.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:34:59 smithi175 ceph-mon[31920]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:35:01.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:01.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:01.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:00 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:01.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:00 smithi175 ceph-mon[31920]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:35:01.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:00 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:01.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:00 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:01.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:00 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:01.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:00 smithi112 ceph-mon[31085]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:35:01.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:00 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:01.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:00 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:01.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:00 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:01.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:00 smithi043 ceph-mon[27978]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:35:03.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:03 smithi112 ceph-mon[31085]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:35:03.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:03 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:03.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:03 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:03.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:03 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:03.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:03 smithi043 ceph-mon[27978]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:35:03.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:03 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:03.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:03 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:03.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:03 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:03.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:03 smithi175 ceph-mon[31920]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:35:03.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:03 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:03.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:03 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:03.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:03 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:03.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:35:03.813+0000 7f1829ffb700 1 -- 172.21.15.112:0/2564101997 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f182c062e80 con 0x7f181c0a3c40 2024-04-21T08:35:03.813 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 10 on host 'smithi112' 2024-04-21T08:35:03.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:35:03.815+0000 7f18331da700 1 -- 172.21.15.112:0/2564101997 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f181c0a3c40 msgr2=0x7f181c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:03.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:35:03.815+0000 7f18331da700 1 --2- 172.21.15.112:0/2564101997 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f181c0a3c40 0x7f181c0a6100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f182c103b00 tx=0x7f182000b410 comp rx=0 tx=0).stop 2024-04-21T08:35:03.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:35:03.815+0000 7f18331da700 1 -- 172.21.15.112:0/2564101997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f182c103ca0 msgr2=0x7f182c074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:03.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:35:03.815+0000 7f18331da700 1 --2- 172.21.15.112:0/2564101997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f182c103ca0 0x7f182c074e30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1818003790 tx=0x7f1818005b00 comp rx=0 tx=0).stop 2024-04-21T08:35:03.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:35:03.816+0000 7f18331da700 1 -- 172.21.15.112:0/2564101997 shutdown_connections 2024-04-21T08:35:03.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:35:03.816+0000 7f18331da700 1 --2- 172.21.15.112:0/2564101997 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f181c0a3c40 0x7f181c0a6100 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:03.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:35:03.816+0000 7f18331da700 1 --2- 172.21.15.112:0/2564101997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f182c104640 0x7f182c075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:03.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:35:03.816+0000 7f18331da700 1 --2- 172.21.15.112:0/2564101997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182c102aa0 0x7f182c0767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:03.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:35:03.816+0000 7f18331da700 1 --2- 172.21.15.112:0/2564101997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f182c103ca0 0x7f182c074e30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:03.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:35:03.816+0000 7f18331da700 1 -- 172.21.15.112:0/2564101997 >> 172.21.15.112:0/2564101997 conn(0x7f182c0fe040 msgr2=0x7f182c107060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:03.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:35:03.816+0000 7f18331da700 1 -- 172.21.15.112:0/2564101997 shutdown_connections 2024-04-21T08:35:03.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-21T08:35:03.816+0000 7f18331da700 1 -- 172.21.15.112:0/2564101997 wait complete. 2024-04-21T08:35:04.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:04 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:04.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:04 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:04.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:04 smithi112 ceph-mon[31085]: from='osd.10 [v2:172.21.15.112:6824/3645019377,v1:172.21.15.112:6825/3645019377]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T08:35:04.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:04 smithi112 ceph-mon[31085]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T08:35:04.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:04 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:04.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:04 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:04.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:04 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:04 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:04 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:04 smithi043 ceph-mon[27978]: from='osd.10 [v2:172.21.15.112:6824/3645019377,v1:172.21.15.112:6825/3645019377]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T08:35:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:04 smithi043 ceph-mon[27978]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T08:35:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:04 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:04 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:04 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:04.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:04 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:04.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:04 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:04.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:04 smithi175 ceph-mon[31920]: from='osd.10 [v2:172.21.15.112:6824/3645019377,v1:172.21.15.112:6825/3645019377]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T08:35:04.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:04 smithi175 ceph-mon[31920]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T08:35:04.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:04 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:04.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:04 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:04.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:04 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:04.518 DEBUG:teuthology.orchestra.run.smithi112:osd.10> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.10.service 2024-04-21T08:35:04.520 INFO:tasks.cephadm:Deploying osd.11 on smithi175 with /dev/vg_nvme/lv_1... 2024-04-21T08:35:04.520 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-21T08:35:04.768 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:35:05.144 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:05 smithi175 ceph-mon[31920]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:35:05.144 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:05 smithi175 ceph-mon[31920]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-21T08:35:05.145 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:05 smithi175 ceph-mon[31920]: osdmap e71: 11 total, 10 up, 11 in 2024-04-21T08:35:05.145 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:05 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:35:05.145 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:05 smithi175 ceph-mon[31920]: from='osd.10 [v2:172.21.15.112:6824/3645019377,v1:172.21.15.112:6825/3645019377]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-21T08:35:05.145 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:05 smithi175 ceph-mon[31920]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-21T08:35:05.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:05 smithi112 ceph-mon[31085]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:35:05.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:05 smithi112 ceph-mon[31085]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-21T08:35:05.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:05 smithi112 ceph-mon[31085]: osdmap e71: 11 total, 10 up, 11 in 2024-04-21T08:35:05.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:05 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:35:05.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:05 smithi112 ceph-mon[31085]: from='osd.10 [v2:172.21.15.112:6824/3645019377,v1:172.21.15.112:6825/3645019377]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-21T08:35:05.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:05 smithi112 ceph-mon[31085]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-21T08:35:05.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:05 smithi043 ceph-mon[27978]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:35:05.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:05 smithi043 ceph-mon[27978]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-21T08:35:05.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:05 smithi043 ceph-mon[27978]: osdmap e71: 11 total, 10 up, 11 in 2024-04-21T08:35:05.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:05 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:35:05.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:05 smithi043 ceph-mon[27978]: from='osd.10 [v2:172.21.15.112:6824/3645019377,v1:172.21.15.112:6825/3645019377]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-21T08:35:05.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:05 smithi043 ceph-mon[27978]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-21T08:35:06.031 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:06 smithi112 ceph-mon[31085]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-21T08:35:06.031 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:06 smithi112 ceph-mon[31085]: osdmap e72: 11 total, 10 up, 11 in 2024-04-21T08:35:06.031 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:06 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:35:06.031 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:06 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:35:06.031 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:35:05 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:35:05.718+0000 7f9e50c5b700 -1 osd.10 0 waiting for initial osdmap 2024-04-21T08:35:06.032 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:35:05 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:35:05.731+0000 7f9e48a46700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:35:06.200 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:06 smithi175 ceph-mon[31920]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-21T08:35:06.200 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:06 smithi175 ceph-mon[31920]: osdmap e72: 11 total, 10 up, 11 in 2024-04-21T08:35:06.200 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:06 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:35:06.200 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:06 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:35:06.201 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:06 smithi175 ceph-mon[31920]: from='osd.10 ' entity='osd.10' 2024-04-21T08:35:06.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:06 smithi112 ceph-mon[31085]: from='osd.10 ' entity='osd.10' 2024-04-21T08:35:06.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:06 smithi043 ceph-mon[27978]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-21T08:35:06.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:06 smithi043 ceph-mon[27978]: osdmap e72: 11 total, 10 up, 11 in 2024-04-21T08:35:06.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:06 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:35:06.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:06 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:35:06.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:06 smithi043 ceph-mon[27978]: from='osd.10 ' entity='osd.10' 2024-04-21T08:35:06.947 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-21T08:35:06.947 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-21T08:35:06.948 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-04-21T08:35:06.948 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-04-21T08:35:06.948 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116677 s, 899 MB/s 2024-04-21T08:35:06.948 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-04-21T08:35:07.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:07 smithi175 ceph-mon[31920]: purged_snaps scrub starts 2024-04-21T08:35:07.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:07 smithi175 ceph-mon[31920]: purged_snaps scrub ok 2024-04-21T08:35:07.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:07 smithi175 ceph-mon[31920]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:35:07.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:07 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:35:07.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:07 smithi175 ceph-mon[31920]: osd.10 [v2:172.21.15.112:6824/3645019377,v1:172.21.15.112:6825/3645019377] boot 2024-04-21T08:35:07.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:07 smithi175 ceph-mon[31920]: osdmap e73: 11 total, 11 up, 11 in 2024-04-21T08:35:07.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:07 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:35:07.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:07 smithi112 ceph-mon[31085]: purged_snaps scrub starts 2024-04-21T08:35:07.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:07 smithi112 ceph-mon[31085]: purged_snaps scrub ok 2024-04-21T08:35:07.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:07 smithi112 ceph-mon[31085]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:35:07.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:07 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:35:07.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:07 smithi112 ceph-mon[31085]: osd.10 [v2:172.21.15.112:6824/3645019377,v1:172.21.15.112:6825/3645019377] boot 2024-04-21T08:35:07.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:07 smithi112 ceph-mon[31085]: osdmap e73: 11 total, 11 up, 11 in 2024-04-21T08:35:07.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:07 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:35:07.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:07 smithi043 ceph-mon[27978]: purged_snaps scrub starts 2024-04-21T08:35:07.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:07 smithi043 ceph-mon[27978]: purged_snaps scrub ok 2024-04-21T08:35:07.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:07 smithi043 ceph-mon[27978]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:35:07.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:07 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:35:07.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:07 smithi043 ceph-mon[27978]: osd.10 [v2:172.21.15.112:6824/3645019377,v1:172.21.15.112:6825/3645019377] boot 2024-04-21T08:35:07.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:07 smithi043 ceph-mon[27978]: osdmap e73: 11 total, 11 up, 11 in 2024-04-21T08:35:07.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:07 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:35:07.775 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi175:vg_nvme/lv_1 2024-04-21T08:35:08.026 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:35:08.294 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:08 smithi175 ceph-mon[31920]: osdmap e74: 11 total, 11 up, 11 in 2024-04-21T08:35:08.294 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:08 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:08.294 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:08 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:08.294 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:08 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:08.294 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:08 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:08.294 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:08 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:08.294 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:08 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:08.294 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:08 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:08.294 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:08 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:08.294 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:08 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:08.295 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:08 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:08.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:08 smithi112 ceph-mon[31085]: osdmap e74: 11 total, 11 up, 11 in 2024-04-21T08:35:08.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:08 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:08.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:08 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:08.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:08 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:08.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:08 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:08.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:08 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:08.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:08 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:08.296 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:08 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:08.296 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:08 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:08.296 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:08 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:08.296 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:08 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:08.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:08 smithi043 ceph-mon[27978]: osdmap e74: 11 total, 11 up, 11 in 2024-04-21T08:35:08.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:08 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:08.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:08 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:08.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:08 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:08.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:08 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:08.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:08 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:08.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:08 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:08.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:08 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:08.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:08 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:08.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:08 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:08.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:08 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:09.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:09 smithi043 ceph-mon[27978]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:09.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:09 smithi043 ceph-mon[27978]: Detected new or changed devices on smithi112 2024-04-21T08:35:09.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:09 smithi043 ceph-mon[27978]: Adjusting osd_memory_target on smithi112 to 4272M 2024-04-21T08:35:09.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:09 smithi043 ceph-mon[27978]: osdmap e75: 11 total, 11 up, 11 in 2024-04-21T08:35:09.316 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:09 smithi175 ceph-mon[31920]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:09.316 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:09 smithi175 ceph-mon[31920]: Detected new or changed devices on smithi112 2024-04-21T08:35:09.316 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:09 smithi175 ceph-mon[31920]: Adjusting osd_memory_target on smithi112 to 4272M 2024-04-21T08:35:09.316 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:09 smithi175 ceph-mon[31920]: osdmap e75: 11 total, 11 up, 11 in 2024-04-21T08:35:09.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:09 smithi112 ceph-mon[31085]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:09.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:09 smithi112 ceph-mon[31085]: Detected new or changed devices on smithi112 2024-04-21T08:35:09.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:09 smithi112 ceph-mon[31085]: Adjusting osd_memory_target on smithi112 to 4272M 2024-04-21T08:35:09.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:09 smithi112 ceph-mon[31085]: osdmap e75: 11 total, 11 up, 11 in 2024-04-21T08:35:09.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.660+0000 7fb3a22bd700 1 -- 172.21.15.175:0/3869651706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb39c102ba0 msgr2=0x7fb39c103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:09.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.660+0000 7fb3a22bd700 1 --2- 172.21.15.175:0/3869651706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb39c102ba0 0x7fb39c103000 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb390009990 tx=0x7fb390009ca0 comp rx=0 tx=0).stop 2024-04-21T08:35:09.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.660+0000 7fb3a22bd700 1 -- 172.21.15.175:0/3869651706 shutdown_connections 2024-04-21T08:35:09.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.660+0000 7fb3a22bd700 1 --2- 172.21.15.175:0/3869651706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb39c103540 0x7fb39c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:09.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.660+0000 7fb3a22bd700 1 --2- 172.21.15.175:0/3869651706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb39c1019a0 0x7fb39c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:09.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.660+0000 7fb3a22bd700 1 --2- 172.21.15.175:0/3869651706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb39c102ba0 0x7fb39c103000 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:09.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.660+0000 7fb3a22bd700 1 -- 172.21.15.175:0/3869651706 >> 172.21.15.175:0/3869651706 conn(0x7fb39c0fcf40 msgr2=0x7fb39c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:09.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.661+0000 7fb3a22bd700 1 -- 172.21.15.175:0/3869651706 shutdown_connections 2024-04-21T08:35:09.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.661+0000 7fb3a22bd700 1 -- 172.21.15.175:0/3869651706 wait complete. 2024-04-21T08:35:09.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.661+0000 7fb3a22bd700 1 Processor -- start 2024-04-21T08:35:09.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.661+0000 7fb3a22bd700 1 -- start start 2024-04-21T08:35:09.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.661+0000 7fb3a22bd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb39c1019a0 0x7fb39c198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:09.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.661+0000 7fb3a22bd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb39c102ba0 0x7fb39c199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:09.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.661+0000 7fb3a22bd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb39c103540 0x7fb39c1a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:09.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.661+0000 7fb3a22bd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c10d530 con 0x7fb39c1019a0 2024-04-21T08:35:09.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.661+0000 7fb3a22bd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c10d3b0 con 0x7fb39c102ba0 2024-04-21T08:35:09.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.661+0000 7fb3a22bd700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c10d6b0 con 0x7fb39c103540 2024-04-21T08:35:09.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.662+0000 7fb3a085a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb39c103540 0x7fb39c1a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:09.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.662+0000 7fb3a085a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb39c103540 0x7fb39c1a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:53460/0 (socket says 172.21.15.175:53460) 2024-04-21T08:35:09.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.662+0000 7fb3a085a700 1 -- 172.21.15.175:0/3925329582 learned_addr learned my addr 172.21.15.175:0/3925329582 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-21T08:35:09.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.662+0000 7fb39bfff700 1 --2- 172.21.15.175:0/3925329582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb39c1019a0 0x7fb39c198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:09.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.662+0000 7fb39b7fe700 1 --2- 172.21.15.175:0/3925329582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb39c102ba0 0x7fb39c199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:09.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.662+0000 7fb3a085a700 1 -- 172.21.15.175:0/3925329582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb39c102ba0 msgr2=0x7fb39c199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:09.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.662+0000 7fb3a085a700 1 --2- 172.21.15.175:0/3925329582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb39c102ba0 0x7fb39c199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:09.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.662+0000 7fb3a085a700 1 -- 172.21.15.175:0/3925329582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb39c1019a0 msgr2=0x7fb39c198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:09.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.662+0000 7fb3a085a700 1 --2- 172.21.15.175:0/3925329582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb39c1019a0 0x7fb39c198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:09.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.662+0000 7fb3a085a700 1 -- 172.21.15.175:0/3925329582 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39c1a0970 con 0x7fb39c103540 2024-04-21T08:35:09.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.662+0000 7fb39b7fe700 1 --2- 172.21.15.175:0/3925329582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb39c102ba0 0x7fb39c199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:35:09.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.662+0000 7fb3a085a700 1 --2- 172.21.15.175:0/3925329582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb39c103540 0x7fb39c1a02a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb39400b970 tx=0x7fb39400bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:09.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.662+0000 7fb3997fa700 1 -- 172.21.15.175:0/3925329582 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb394019070 con 0x7fb39c103540 2024-04-21T08:35:09.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.662+0000 7fb3997fa700 1 -- 172.21.15.175:0/3925329582 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb39400cde0 con 0x7fb39c103540 2024-04-21T08:35:09.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.662+0000 7fb3997fa700 1 -- 172.21.15.175:0/3925329582 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb394007370 con 0x7fb39c103540 2024-04-21T08:35:09.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.662+0000 7fb3a22bd700 1 -- 172.21.15.175:0/3925329582 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb39c1a0c60 con 0x7fb39c103540 2024-04-21T08:35:09.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.662+0000 7fb3a22bd700 1 -- 172.21.15.175:0/3925329582 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb39c1a1150 con 0x7fb39c103540 2024-04-21T08:35:09.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.663+0000 7fb3a22bd700 1 -- 172.21.15.175:0/3925329582 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb39c069730 con 0x7fb39c103540 2024-04-21T08:35:09.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.664+0000 7fb3997fa700 1 -- 172.21.15.175:0/3925329582 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb39401c070 con 0x7fb39c103540 2024-04-21T08:35:09.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.665+0000 7fb3997fa700 1 --2- 172.21.15.175:0/3925329582 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb3840a3c40 0x7fb3840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:09.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.665+0000 7fb3997fa700 1 -- 172.21.15.175:0/3925329582 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7fb3940cf2f0 con 0x7fb39c103540 2024-04-21T08:35:09.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.665+0000 7fb39bfff700 1 --2- 172.21.15.175:0/3925329582 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb3840a3c40 0x7fb3840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:09.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.666+0000 7fb39bfff700 1 --2- 172.21.15.175:0/3925329582 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb3840a3c40 0x7fb3840a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb39c19a200 tx=0x7fb38c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:09.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.670+0000 7fb3997fa700 1 -- 172.21.15.175:0/3925329582 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb39409c320 con 0x7fb39c103540 2024-04-21T08:35:09.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:09.875+0000 7fb3a22bd700 1 -- 172.21.15.175:0/3925329582 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb39c06ae60 con 0x7fb3840a3c40 2024-04-21T08:35:10.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:10 smithi043 ceph-mon[27978]: osdmap e76: 11 total, 11 up, 11 in 2024-04-21T08:35:10.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:10 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:35:10.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:10 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:35:10.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:10 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:10.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:10 smithi175 ceph-mon[31920]: osdmap e76: 11 total, 11 up, 11 in 2024-04-21T08:35:10.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:10 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:35:10.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:10 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:35:10.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:10 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:10 smithi112 ceph-mon[31085]: osdmap e76: 11 total, 11 up, 11 in 2024-04-21T08:35:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:10 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:35:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:10 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:35:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:10 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:11.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:11 smithi043 ceph-mon[27978]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:35:11.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:11 smithi043 ceph-mon[27978]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T08:35:11.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:11 smithi043 ceph-mon[27978]: osdmap e77: 11 total, 11 up, 11 in 2024-04-21T08:35:11.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:11 smithi175 ceph-mon[31920]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:35:11.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:11 smithi175 ceph-mon[31920]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T08:35:11.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:11 smithi175 ceph-mon[31920]: osdmap e77: 11 total, 11 up, 11 in 2024-04-21T08:35:11.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:11 smithi112 ceph-mon[31085]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:35:11.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:11 smithi112 ceph-mon[31085]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T08:35:11.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:11 smithi112 ceph-mon[31085]: osdmap e77: 11 total, 11 up, 11 in 2024-04-21T08:35:13.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:13 smithi043 ceph-mon[27978]: pgmap v246: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-21T08:35:13.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:13 smithi043 ceph-mon[27978]: from='client.? 172.21.15.175:0/1224797775' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8114e33b-8fb2-4fe4-bf56-5f007c88ddbc"}]: dispatch 2024-04-21T08:35:13.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:13 smithi043 ceph-mon[27978]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8114e33b-8fb2-4fe4-bf56-5f007c88ddbc"}]: dispatch 2024-04-21T08:35:13.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:13 smithi043 ceph-mon[27978]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8114e33b-8fb2-4fe4-bf56-5f007c88ddbc"}]': finished 2024-04-21T08:35:13.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:13 smithi043 ceph-mon[27978]: osdmap e78: 12 total, 11 up, 12 in 2024-04-21T08:35:13.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:13 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:13.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:13 smithi175 ceph-mon[31920]: pgmap v246: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-21T08:35:13.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:13 smithi175 ceph-mon[31920]: from='client.? 172.21.15.175:0/1224797775' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8114e33b-8fb2-4fe4-bf56-5f007c88ddbc"}]: dispatch 2024-04-21T08:35:13.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:13 smithi175 ceph-mon[31920]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8114e33b-8fb2-4fe4-bf56-5f007c88ddbc"}]: dispatch 2024-04-21T08:35:13.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:13 smithi175 ceph-mon[31920]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8114e33b-8fb2-4fe4-bf56-5f007c88ddbc"}]': finished 2024-04-21T08:35:13.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:13 smithi175 ceph-mon[31920]: osdmap e78: 12 total, 11 up, 12 in 2024-04-21T08:35:13.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:13 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:13.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:13 smithi112 ceph-mon[31085]: pgmap v246: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-21T08:35:13.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:13 smithi112 ceph-mon[31085]: from='client.? 172.21.15.175:0/1224797775' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8114e33b-8fb2-4fe4-bf56-5f007c88ddbc"}]: dispatch 2024-04-21T08:35:13.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:13 smithi112 ceph-mon[31085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8114e33b-8fb2-4fe4-bf56-5f007c88ddbc"}]: dispatch 2024-04-21T08:35:13.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:13 smithi112 ceph-mon[31085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8114e33b-8fb2-4fe4-bf56-5f007c88ddbc"}]': finished 2024-04-21T08:35:13.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:13 smithi112 ceph-mon[31085]: osdmap e78: 12 total, 11 up, 12 in 2024-04-21T08:35:13.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:13 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:14.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:14 smithi043 ceph-mon[27978]: from='client.? 172.21.15.175:0/1804114595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:35:14.312 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:35:13 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:35:13.914+0000 7f06f0537700 -1 calc_pg_upmaps abort due to max <= 0 2024-04-21T08:35:14.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:14 smithi175 ceph-mon[31920]: from='client.? 172.21.15.175:0/1804114595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:35:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:14 smithi112 ceph-mon[31085]: from='client.? 172.21.15.175:0/1804114595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:35:15.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:15 smithi175 ceph-mon[31920]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-21T08:35:15.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:15 smithi112 ceph-mon[31085]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-21T08:35:15.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:15 smithi043 ceph-mon[27978]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-21T08:35:17.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:17 smithi043 ceph-mon[27978]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:17.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:17 smithi175 ceph-mon[31920]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:17.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:17 smithi112 ceph-mon[31085]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:19.180 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:19 smithi175 ceph-mon[31920]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:19.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:19 smithi112 ceph-mon[31085]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:19.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:19 smithi043 ceph-mon[27978]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:21.311 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:21 smithi175 ceph-mon[31920]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:21.312 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:21 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-21T08:35:21.312 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:21 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:21.312 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:21 smithi175 ceph-mon[31920]: Deploying daemon osd.11 on smithi175 2024-04-21T08:35:21.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:21 smithi112 ceph-mon[31085]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:21.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:21 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-21T08:35:21.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:21 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:21.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:21 smithi112 ceph-mon[31085]: Deploying daemon osd.11 on smithi175 2024-04-21T08:35:21.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:21 smithi043 ceph-mon[27978]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:21.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:21 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-21T08:35:21.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:21 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:21.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:21 smithi043 ceph-mon[27978]: Deploying daemon osd.11 on smithi175 2024-04-21T08:35:23.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:23 smithi175 ceph-mon[31920]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:23.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:23 smithi112 ceph-mon[31085]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:23.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:23 smithi043 ceph-mon[27978]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:25.346 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:25 smithi112 ceph-mon[31085]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:25.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:25 smithi175 ceph-mon[31920]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:25.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:25 smithi043 ceph-mon[27978]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:27.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:27 smithi112 ceph-mon[31085]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:27.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:27 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:27.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:27 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:27.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:27 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:27.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:27 smithi043 ceph-mon[27978]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:27.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:27 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:27.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:27 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:27.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:27 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:27.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:27 smithi175 ceph-mon[31920]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:27.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:27 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:27.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:27 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:27.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:27 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:29.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:29 smithi112 ceph-mon[31085]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:29.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:29 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:29.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:29 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:29.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:29 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:29.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:29 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:29.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:29 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:29.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:29 smithi043 ceph-mon[27978]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:29.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:29.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:29.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:29.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:29.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:29 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:29.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:29 smithi175 ceph-mon[31920]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:29.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:29 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:29.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:29 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:29.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:29 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:29.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:29 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:29.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:29 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:30.446 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 11 on host 'smithi175' 2024-04-21T08:35:30.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:30.447+0000 7fb3997fa700 1 -- 172.21.15.175:0/3925329582 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fb39c06ae60 con 0x7fb3840a3c40 2024-04-21T08:35:30.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:30.449+0000 7fb3a22bd700 1 -- 172.21.15.175:0/3925329582 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb3840a3c40 msgr2=0x7fb3840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:30.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:30.449+0000 7fb3a22bd700 1 --2- 172.21.15.175:0/3925329582 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb3840a3c40 0x7fb3840a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb39c19a200 tx=0x7fb38c009190 comp rx=0 tx=0).stop 2024-04-21T08:35:30.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:30.449+0000 7fb3a22bd700 1 -- 172.21.15.175:0/3925329582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb39c103540 msgr2=0x7fb39c1a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:30.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:30.449+0000 7fb3a22bd700 1 --2- 172.21.15.175:0/3925329582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb39c103540 0x7fb39c1a02a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb39400b970 tx=0x7fb39400bc80 comp rx=0 tx=0).stop 2024-04-21T08:35:30.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:30.450+0000 7fb3a22bd700 1 -- 172.21.15.175:0/3925329582 shutdown_connections 2024-04-21T08:35:30.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:30.450+0000 7fb3a22bd700 1 --2- 172.21.15.175:0/3925329582 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb3840a3c40 0x7fb3840a6100 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:30.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:30.450+0000 7fb3a22bd700 1 --2- 172.21.15.175:0/3925329582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb39c103540 0x7fb39c1a02a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:30.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:30.450+0000 7fb3a22bd700 1 --2- 172.21.15.175:0/3925329582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb39c1019a0 0x7fb39c198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:30.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:30.450+0000 7fb3a22bd700 1 --2- 172.21.15.175:0/3925329582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb39c102ba0 0x7fb39c199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:30.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:30.450+0000 7fb3a22bd700 1 -- 172.21.15.175:0/3925329582 >> 172.21.15.175:0/3925329582 conn(0x7fb39c0fcf40 msgr2=0x7fb39c06a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:30.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:30.450+0000 7fb3a22bd700 1 -- 172.21.15.175:0/3925329582 shutdown_connections 2024-04-21T08:35:30.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:30.450+0000 7fb3a22bd700 1 -- 172.21.15.175:0/3925329582 wait complete. 2024-04-21T08:35:30.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:30 smithi175 ceph-mon[31920]: from='osd.11 [v2:172.21.15.175:6824/2527264089,v1:172.21.15.175:6825/2527264089]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T08:35:30.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:30 smithi175 ceph-mon[31920]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T08:35:30.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:30 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:30.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:30 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:30.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:30 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:30.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:30 smithi112 ceph-mon[31085]: from='osd.11 [v2:172.21.15.175:6824/2527264089,v1:172.21.15.175:6825/2527264089]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T08:35:30.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:30 smithi112 ceph-mon[31085]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T08:35:30.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:30 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:30.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:30 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:30.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:30 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:30.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:30 smithi043 ceph-mon[27978]: from='osd.11 [v2:172.21.15.175:6824/2527264089,v1:172.21.15.175:6825/2527264089]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T08:35:30.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:30 smithi043 ceph-mon[27978]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T08:35:30.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:30 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:30.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:30 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:30.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:30 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:31.098 DEBUG:teuthology.orchestra.run.smithi175:osd.11> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.11.service 2024-04-21T08:35:31.100 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-21T08:35:31.101 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-21T08:35:31.358 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:35:31.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:31 smithi175 ceph-mon[31920]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:31.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:31 smithi175 ceph-mon[31920]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-21T08:35:31.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:31 smithi175 ceph-mon[31920]: osdmap e79: 12 total, 11 up, 12 in 2024-04-21T08:35:31.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:31 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:31.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:31 smithi175 ceph-mon[31920]: from='osd.11 [v2:172.21.15.175:6824/2527264089,v1:172.21.15.175:6825/2527264089]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-04-21T08:35:31.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:31 smithi175 ceph-mon[31920]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-04-21T08:35:31.714 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:35:31 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:35:31.550+0000 7f0d52f9d700 -1 osd.11 0 waiting for initial osdmap 2024-04-21T08:35:31.714 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:35:31 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:35:31.563+0000 7f0d4ed90700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:35:31.787 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:31 smithi043 ceph-mon[27978]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:31.788 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:31 smithi043 ceph-mon[27978]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-21T08:35:31.788 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:31 smithi043 ceph-mon[27978]: osdmap e79: 12 total, 11 up, 12 in 2024-04-21T08:35:31.788 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:31 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:31.788 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:31 smithi043 ceph-mon[27978]: from='osd.11 [v2:172.21.15.175:6824/2527264089,v1:172.21.15.175:6825/2527264089]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-04-21T08:35:31.788 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:31 smithi043 ceph-mon[27978]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-04-21T08:35:32.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:31 smithi112 ceph-mon[31085]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:32.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:31 smithi112 ceph-mon[31085]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-21T08:35:32.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:31 smithi112 ceph-mon[31085]: osdmap e79: 12 total, 11 up, 12 in 2024-04-21T08:35:32.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:31 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:32.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:31 smithi112 ceph-mon[31085]: from='osd.11 [v2:172.21.15.175:6824/2527264089,v1:172.21.15.175:6825/2527264089]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-04-21T08:35:32.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:31 smithi112 ceph-mon[31085]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-04-21T08:35:32.616 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:32 smithi175 ceph-mon[31920]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-04-21T08:35:32.617 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:32 smithi175 ceph-mon[31920]: osdmap e80: 12 total, 11 up, 12 in 2024-04-21T08:35:32.617 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:32 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:32.617 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:32 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:32.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:32 smithi043 ceph-mon[27978]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-04-21T08:35:32.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:32 smithi043 ceph-mon[27978]: osdmap e80: 12 total, 11 up, 12 in 2024-04-21T08:35:32.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:32 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:32.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:32 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:33.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:32 smithi112 ceph-mon[31085]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-04-21T08:35:33.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:32 smithi112 ceph-mon[31085]: osdmap e80: 12 total, 11 up, 12 in 2024-04-21T08:35:33.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:32 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:33.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:32 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:33.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.472+0000 7f22a6217700 1 -- 172.21.15.43:0/3662661375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22a0075440 msgr2=0x7f22a00758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:33.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.472+0000 7f22a6217700 1 --2- 172.21.15.43:0/3662661375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22a0075440 0x7f22a00758c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2288009990 tx=0x7f2288009ca0 comp rx=0 tx=0).stop 2024-04-21T08:35:33.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.472+0000 7f22a6217700 1 -- 172.21.15.43:0/3662661375 shutdown_connections 2024-04-21T08:35:33.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.473+0000 7f22a6217700 1 --2- 172.21.15.43:0/3662661375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22a0076a10 0x7f22a0074e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:33.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.473+0000 7f22a6217700 1 --2- 172.21.15.43:0/3662661375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22a010a980 0x7f22a010ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:33.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.473+0000 7f22a6217700 1 --2- 172.21.15.43:0/3662661375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22a0075440 0x7f22a00758c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:33.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.473+0000 7f22a6217700 1 -- 172.21.15.43:0/3662661375 >> 172.21.15.43:0/3662661375 conn(0x7f22a00fe040 msgr2=0x7f22a0100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:33.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.473+0000 7f22a6217700 1 -- 172.21.15.43:0/3662661375 shutdown_connections 2024-04-21T08:35:33.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.473+0000 7f22a6217700 1 -- 172.21.15.43:0/3662661375 wait complete. 2024-04-21T08:35:33.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.473+0000 7f22a6217700 1 Processor -- start 2024-04-21T08:35:33.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.473+0000 7f22a6217700 1 -- start start 2024-04-21T08:35:33.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.474+0000 7f22a6217700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22a0075440 0x7f22a019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.474+0000 7f22a6217700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22a0076a10 0x7f22a019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.474+0000 7f22a6217700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22a010a980 0x7f22a01a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.474+0000 7f22a6217700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f22a01106f0 con 0x7f22a0076a10 2024-04-21T08:35:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.474+0000 7f22a6217700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f22a0110570 con 0x7f22a0075440 2024-04-21T08:35:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.474+0000 7f22a6217700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22a0110870 con 0x7f22a010a980 2024-04-21T08:35:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.474+0000 7f229effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22a0076a10 0x7f22a019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:33.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.474+0000 7f229effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22a0076a10 0x7f22a019b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60094/0 (socket says 172.21.15.43:60094) 2024-04-21T08:35:33.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.474+0000 7f229effd700 1 -- 172.21.15.43:0/2847041770 learned_addr learned my addr 172.21.15.43:0/2847041770 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:35:33.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.474+0000 7f229f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22a0075440 0x7f22a019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:33.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.474+0000 7f229ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22a010a980 0x7f22a01a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:33.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.474+0000 7f229effd700 1 -- 172.21.15.43:0/2847041770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22a010a980 msgr2=0x7f22a01a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:33.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.474+0000 7f229effd700 1 --2- 172.21.15.43:0/2847041770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22a010a980 0x7f22a01a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.474+0000 7f229effd700 1 -- 172.21.15.43:0/2847041770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22a0075440 msgr2=0x7f22a019adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.474+0000 7f229effd700 1 --2- 172.21.15.43:0/2847041770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22a0075440 0x7f22a019adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.474+0000 7f229effd700 1 -- 172.21.15.43:0/2847041770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a01a2a10 con 0x7f22a0076a10 2024-04-21T08:35:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.474+0000 7f229effd700 1 --2- 172.21.15.43:0/2847041770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22a0076a10 0x7f22a019b2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f22880059d0 tx=0x7f22880047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.475+0000 7f229cff9700 1 -- 172.21.15.43:0/2847041770 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2288027070 con 0x7f22a0076a10 2024-04-21T08:35:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.475+0000 7f22a6217700 1 -- 172.21.15.43:0/2847041770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a01a2ca0 con 0x7f22a0076a10 2024-04-21T08:35:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.475+0000 7f229cff9700 1 -- 172.21.15.43:0/2847041770 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2288022470 con 0x7f22a0076a10 2024-04-21T08:35:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.475+0000 7f22a6217700 1 -- 172.21.15.43:0/2847041770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a01a3150 con 0x7f22a0076a10 2024-04-21T08:35:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.475+0000 7f229cff9700 1 -- 172.21.15.43:0/2847041770 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2288017650 con 0x7f22a0076a10 2024-04-21T08:35:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.476+0000 7f22a6217700 1 -- 172.21.15.43:0/2847041770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2280005360 con 0x7f22a0076a10 2024-04-21T08:35:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.477+0000 7f229cff9700 1 -- 172.21.15.43:0/2847041770 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2288009d10 con 0x7f22a0076a10 2024-04-21T08:35:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.478+0000 7f229cff9700 1 --2- 172.21.15.43:0/2847041770 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f228c0a3cc0 0x7f228c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.478+0000 7f229f7fe700 1 --2- 172.21.15.43:0/2847041770 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f228c0a3cc0 0x7f228c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.478+0000 7f229cff9700 1 -- 172.21.15.43:0/2847041770 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f22880d09b0 con 0x7f22a0076a10 2024-04-21T08:35:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.478+0000 7f229f7fe700 1 --2- 172.21.15.43:0/2847041770 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f228c0a3cc0 0x7f228c0a6180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2290009550 tx=0x7f22900091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.483+0000 7f229cff9700 1 -- 172.21.15.43:0/2847041770 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f228805dff0 con 0x7f22a0076a10 2024-04-21T08:35:33.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.692+0000 7f22a6217700 1 -- 172.21.15.43:0/2847041770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f2280005920 con 0x7f22a0076a10 2024-04-21T08:35:33.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.693+0000 7f229cff9700 1 -- 172.21.15.43:0/2847041770 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7f2288013070 con 0x7f22a0076a10 2024-04-21T08:35:33.693 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:35:33.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.695+0000 7f22a6217700 1 -- 172.21.15.43:0/2847041770 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f228c0a3cc0 msgr2=0x7f228c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:33.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.695+0000 7f22a6217700 1 --2- 172.21.15.43:0/2847041770 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f228c0a3cc0 0x7f228c0a6180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2290009550 tx=0x7f22900091b0 comp rx=0 tx=0).stop 2024-04-21T08:35:33.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.695+0000 7f22a6217700 1 -- 172.21.15.43:0/2847041770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22a0076a10 msgr2=0x7f22a019b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:33.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.695+0000 7f22a6217700 1 --2- 172.21.15.43:0/2847041770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22a0076a10 0x7f22a019b2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f22880059d0 tx=0x7f22880047b0 comp rx=0 tx=0).stop 2024-04-21T08:35:33.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.695+0000 7f22a6217700 1 -- 172.21.15.43:0/2847041770 shutdown_connections 2024-04-21T08:35:33.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.695+0000 7f22a6217700 1 --2- 172.21.15.43:0/2847041770 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f228c0a3cc0 0x7f228c0a6180 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:33.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.695+0000 7f22a6217700 1 --2- 172.21.15.43:0/2847041770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22a010a980 0x7f22a01a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:33.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.695+0000 7f22a6217700 1 --2- 172.21.15.43:0/2847041770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22a0075440 0x7f22a019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:33.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.695+0000 7f22a6217700 1 --2- 172.21.15.43:0/2847041770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22a0076a10 0x7f22a019b2f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:33.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.695+0000 7f22a6217700 1 -- 172.21.15.43:0/2847041770 >> 172.21.15.43:0/2847041770 conn(0x7f22a00fe040 msgr2=0x7f22a01068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:33.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.695+0000 7f22a6217700 1 -- 172.21.15.43:0/2847041770 shutdown_connections 2024-04-21T08:35:33.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:33.695+0000 7f22a6217700 1 -- 172.21.15.43:0/2847041770 wait complete. 2024-04-21T08:35:33.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:33 smithi175 ceph-mon[31920]: purged_snaps scrub starts 2024-04-21T08:35:33.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:33 smithi175 ceph-mon[31920]: purged_snaps scrub ok 2024-04-21T08:35:33.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:33 smithi175 ceph-mon[31920]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:33.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:33 smithi175 ceph-mon[31920]: osd.11 [v2:172.21.15.175:6824/2527264089,v1:172.21.15.175:6825/2527264089] boot 2024-04-21T08:35:33.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:33 smithi175 ceph-mon[31920]: osdmap e81: 12 total, 12 up, 12 in 2024-04-21T08:35:33.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:33 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:33.726 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:33 smithi043 ceph-mon[27978]: purged_snaps scrub starts 2024-04-21T08:35:33.726 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:33 smithi043 ceph-mon[27978]: purged_snaps scrub ok 2024-04-21T08:35:33.726 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:33 smithi043 ceph-mon[27978]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:33.726 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:33 smithi043 ceph-mon[27978]: osd.11 [v2:172.21.15.175:6824/2527264089,v1:172.21.15.175:6825/2527264089] boot 2024-04-21T08:35:33.726 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:33 smithi043 ceph-mon[27978]: osdmap e81: 12 total, 12 up, 12 in 2024-04-21T08:35:33.726 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:33 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:34.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:33 smithi112 ceph-mon[31085]: purged_snaps scrub starts 2024-04-21T08:35:34.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:33 smithi112 ceph-mon[31085]: purged_snaps scrub ok 2024-04-21T08:35:34.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:33 smithi112 ceph-mon[31085]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:34.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:33 smithi112 ceph-mon[31085]: osd.11 [v2:172.21.15.175:6824/2527264089,v1:172.21.15.175:6825/2527264089] boot 2024-04-21T08:35:34.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:33 smithi112 ceph-mon[31085]: osdmap e81: 12 total, 12 up, 12 in 2024-04-21T08:35:34.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:33 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:34.530 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1713688532,"num_in_osds":12,"osd_in_since":1713688512,"num_remapped_pgs":0} 2024-04-21T08:35:34.531 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-21T08:35:34.783 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:35:34.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:34 smithi043 ceph-mon[27978]: osdmap e82: 12 total, 12 up, 12 in 2024-04-21T08:35:34.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:34 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2847041770' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-21T08:35:34.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:34 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:34.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:34 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:34.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:34 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:34.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:34 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:34.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:34 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:34.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:34 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:34.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:34 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:34.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:34 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:34.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:34 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:34.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:34 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:34.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:34 smithi175 ceph-mon[31920]: osdmap e82: 12 total, 12 up, 12 in 2024-04-21T08:35:34.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:34 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2847041770' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-21T08:35:34.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:34 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:34.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:34 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:34.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:34 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:34.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:34 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:34.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:34 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:34.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:34 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:34.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:34 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:34.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:34 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:34.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:34 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:34.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:34 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:35.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:34 smithi112 ceph-mon[31085]: osdmap e82: 12 total, 12 up, 12 in 2024-04-21T08:35:35.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:34 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2847041770' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-21T08:35:35.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:34 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:35.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:34 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:35.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:34 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:35.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:34 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:35.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:34 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:35.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:34 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:35.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:34 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:35.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:34 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:35.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:34 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:35.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:34 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:35.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:35 smithi175 ceph-mon[31920]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:35.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:35 smithi175 ceph-mon[31920]: Detected new or changed devices on smithi175 2024-04-21T08:35:35.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:35 smithi175 ceph-mon[31920]: Adjusting osd_memory_target on smithi175 to 4272M 2024-04-21T08:35:36.035 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:35 smithi043 ceph-mon[27978]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:36.035 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:35 smithi043 ceph-mon[27978]: Detected new or changed devices on smithi175 2024-04-21T08:35:36.035 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:35 smithi043 ceph-mon[27978]: Adjusting osd_memory_target on smithi175 to 4272M 2024-04-21T08:35:36.043 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:35 smithi112 ceph-mon[31085]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:36.043 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:35 smithi112 ceph-mon[31085]: Detected new or changed devices on smithi175 2024-04-21T08:35:36.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:35 smithi112 ceph-mon[31085]: Adjusting osd_memory_target on smithi175 to 4272M 2024-04-21T08:35:36.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.551+0000 7f4d83148700 1 -- 172.21.15.43:0/773170198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d7c06a750 msgr2=0x7f4d7c06abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:36.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.551+0000 7f4d83148700 1 --2- 172.21.15.43:0/773170198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d7c06a750 0x7f4d7c06abb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4d700098f0 tx=0x7f4d70009c00 comp rx=0 tx=0).stop 2024-04-21T08:35:36.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.551+0000 7f4d83148700 1 -- 172.21.15.43:0/773170198 shutdown_connections 2024-04-21T08:35:36.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.551+0000 7f4d83148700 1 --2- 172.21.15.43:0/773170198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d7c108a30 0x7f4d7c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:36.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.551+0000 7f4d83148700 1 --2- 172.21.15.43:0/773170198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d7c06b0f0 0x7f4d7c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:36.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.551+0000 7f4d83148700 1 --2- 172.21.15.43:0/773170198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d7c06a750 0x7f4d7c06abb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:36.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.551+0000 7f4d83148700 1 -- 172.21.15.43:0/773170198 >> 172.21.15.43:0/773170198 conn(0x7f4d7c0788f0 msgr2=0x7f4d7c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:36.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.551+0000 7f4d83148700 1 -- 172.21.15.43:0/773170198 shutdown_connections 2024-04-21T08:35:36.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.551+0000 7f4d83148700 1 -- 172.21.15.43:0/773170198 wait complete. 2024-04-21T08:35:36.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.552+0000 7f4d83148700 1 Processor -- start 2024-04-21T08:35:36.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.552+0000 7f4d83148700 1 -- start start 2024-04-21T08:35:36.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.552+0000 7f4d83148700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d7c06a750 0x7f4d7c199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:36.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.552+0000 7f4d83148700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d7c06b0f0 0x7f4d7c1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:36.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.552+0000 7f4d83148700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d7c108a30 0x7f4d7c19da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:36.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.552+0000 7f4d83148700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c112bd0 con 0x7f4d7c06b0f0 2024-04-21T08:35:36.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.552+0000 7f4d83148700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c112a50 con 0x7f4d7c06a750 2024-04-21T08:35:36.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.552+0000 7f4d83148700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c112d50 con 0x7f4d7c108a30 2024-04-21T08:35:36.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.552+0000 7f4d7bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d7c06b0f0 0x7f4d7c1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:36.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.552+0000 7f4d7bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d7c06b0f0 0x7f4d7c1996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60114/0 (socket says 172.21.15.43:60114) 2024-04-21T08:35:36.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.552+0000 7f4d7bfff700 1 -- 172.21.15.43:0/2228563410 learned_addr learned my addr 172.21.15.43:0/2228563410 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:35:36.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.552+0000 7f4d80ee4700 1 --2- 172.21.15.43:0/2228563410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d7c06a750 0x7f4d7c199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:36.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.552+0000 7f4d816e5700 1 --2- 172.21.15.43:0/2228563410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d7c108a30 0x7f4d7c19da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:36.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.553+0000 7f4d7bfff700 1 -- 172.21.15.43:0/2228563410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d7c108a30 msgr2=0x7f4d7c19da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:36.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.553+0000 7f4d7bfff700 1 --2- 172.21.15.43:0/2228563410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d7c108a30 0x7f4d7c19da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:36.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.553+0000 7f4d7bfff700 1 -- 172.21.15.43:0/2228563410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d7c06a750 msgr2=0x7f4d7c199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:36.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.553+0000 7f4d7bfff700 1 --2- 172.21.15.43:0/2228563410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d7c06a750 0x7f4d7c199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:36.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.553+0000 7f4d7bfff700 1 -- 172.21.15.43:0/2228563410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d7c19e140 con 0x7f4d7c06b0f0 2024-04-21T08:35:36.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.553+0000 7f4d7bfff700 1 --2- 172.21.15.43:0/2228563410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d7c06b0f0 0x7f4d7c1996c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000f990 tx=0x7f4d7000f9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:36.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.553+0000 7f4d79ffb700 1 -- 172.21.15.43:0/2228563410 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70027070 con 0x7f4d7c06b0f0 2024-04-21T08:35:36.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.553+0000 7f4d79ffb700 1 -- 172.21.15.43:0/2228563410 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d70005570 con 0x7f4d7c06b0f0 2024-04-21T08:35:36.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.553+0000 7f4d83148700 1 -- 172.21.15.43:0/2228563410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d7c19e3d0 con 0x7f4d7c06b0f0 2024-04-21T08:35:36.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.553+0000 7f4d79ffb700 1 -- 172.21.15.43:0/2228563410 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70004930 con 0x7f4d7c06b0f0 2024-04-21T08:35:36.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.553+0000 7f4d83148700 1 -- 172.21.15.43:0/2228563410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d7c1a5cb0 con 0x7f4d7c06b0f0 2024-04-21T08:35:36.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.554+0000 7f4d83148700 1 -- 172.21.15.43:0/2228563410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d7c069730 con 0x7f4d7c06b0f0 2024-04-21T08:35:36.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.555+0000 7f4d79ffb700 1 -- 172.21.15.43:0/2228563410 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4d7002a070 con 0x7f4d7c06b0f0 2024-04-21T08:35:36.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.556+0000 7f4d79ffb700 1 --2- 172.21.15.43:0/2228563410 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f4d640a3c40 0x7f4d640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:36.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.556+0000 7f4d80ee4700 1 --2- 172.21.15.43:0/2228563410 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f4d640a3c40 0x7f4d640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:36.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.556+0000 7f4d79ffb700 1 -- 172.21.15.43:0/2228563410 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4d700d1010 con 0x7f4d7c06b0f0 2024-04-21T08:35:36.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.556+0000 7f4d80ee4700 1 --2- 172.21.15.43:0/2228563410 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f4d640a3c40 0x7f4d640a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009be0 tx=0x7f4d6c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:36.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.560+0000 7f4d79ffb700 1 -- 172.21.15.43:0/2228563410 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d70031070 con 0x7f4d7c06b0f0 2024-04-21T08:35:36.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.768+0000 7f4d83148700 1 -- 172.21.15.43:0/2228563410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d7c02df80 con 0x7f4d7c06b0f0 2024-04-21T08:35:36.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.769+0000 7f4d79ffb700 1 -- 172.21.15.43:0/2228563410 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18480 (secure 0 0 0) 0x7f4d70013030 con 0x7f4d7c06b0f0 2024-04-21T08:35:36.770 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:35:36.770 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":82,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:35:33.554402+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:35:36.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.771+0000 7f4d83148700 1 -- 172.21.15.43:0/2228563410 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f4d640a3c40 msgr2=0x7f4d640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:36.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.771+0000 7f4d83148700 1 --2- 172.21.15.43:0/2228563410 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f4d640a3c40 0x7f4d640a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009be0 tx=0x7f4d6c009260 comp rx=0 tx=0).stop 2024-04-21T08:35:36.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.771+0000 7f4d83148700 1 -- 172.21.15.43:0/2228563410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d7c06b0f0 msgr2=0x7f4d7c1996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:36.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.771+0000 7f4d83148700 1 --2- 172.21.15.43:0/2228563410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d7c06b0f0 0x7f4d7c1996c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000f990 tx=0x7f4d7000f9c0 comp rx=0 tx=0).stop 2024-04-21T08:35:36.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.771+0000 7f4d83148700 1 -- 172.21.15.43:0/2228563410 shutdown_connections 2024-04-21T08:35:36.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.771+0000 7f4d83148700 1 --2- 172.21.15.43:0/2228563410 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f4d640a3c40 0x7f4d640a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:36.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.771+0000 7f4d83148700 1 --2- 172.21.15.43:0/2228563410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d7c108a30 0x7f4d7c19da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:36.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.771+0000 7f4d83148700 1 --2- 172.21.15.43:0/2228563410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d7c06a750 0x7f4d7c199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:36.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.771+0000 7f4d83148700 1 --2- 172.21.15.43:0/2228563410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d7c06b0f0 0x7f4d7c1996c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:36.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.771+0000 7f4d83148700 1 -- 172.21.15.43:0/2228563410 >> 172.21.15.43:0/2228563410 conn(0x7f4d7c0788f0 msgr2=0x7f4d7c0ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:36.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.771+0000 7f4d83148700 1 -- 172.21.15.43:0/2228563410 shutdown_connections 2024-04-21T08:35:36.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:36.771+0000 7f4d83148700 1 -- 172.21.15.43:0/2228563410 wait complete. 2024-04-21T08:35:37.553 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:39.981405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-21T08:35:37.553 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-21T08:35:37.805 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:35:37.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:37 smithi043 ceph-mon[27978]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:37.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:37 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2228563410' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:35:37.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:37 smithi175 ceph-mon[31920]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:37.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:37 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2228563410' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:35:38.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:37 smithi112 ceph-mon[31085]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:38.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:37 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2228563410' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:35:39.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.731+0000 7f96b3e52700 1 -- 172.21.15.43:0/4124690455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96ac102ed0 msgr2=0x7f96ac10e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:39.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.731+0000 7f96b3e52700 1 --2- 172.21.15.43:0/4124690455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96ac102ed0 0x7f96ac10e8e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f96a400b0d0 tx=0x7f96a400b3e0 comp rx=0 tx=0).stop 2024-04-21T08:35:39.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.731+0000 7f96b3e52700 1 -- 172.21.15.43:0/4124690455 shutdown_connections 2024-04-21T08:35:39.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.731+0000 7f96b3e52700 1 --2- 172.21.15.43:0/4124690455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96ac102ed0 0x7f96ac10e8e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:39.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.731+0000 7f96b3e52700 1 --2- 172.21.15.43:0/4124690455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96ac107c30 0x7f96ac108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:39.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.731+0000 7f96b3e52700 1 --2- 172.21.15.43:0/4124690455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96ac102510 0x7f96ac102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:39.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.731+0000 7f96b3e52700 1 -- 172.21.15.43:0/4124690455 >> 172.21.15.43:0/4124690455 conn(0x7f96ac076c60 msgr2=0x7f96ac077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:39.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.731+0000 7f96b3e52700 1 -- 172.21.15.43:0/4124690455 shutdown_connections 2024-04-21T08:35:39.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.731+0000 7f96b3e52700 1 -- 172.21.15.43:0/4124690455 wait complete. 2024-04-21T08:35:39.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.732+0000 7f96b3e52700 1 Processor -- start 2024-04-21T08:35:39.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.732+0000 7f96b3e52700 1 -- start start 2024-04-21T08:35:39.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.732+0000 7f96b3e52700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96ac102510 0x7f96ac199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:39.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.732+0000 7f96b3e52700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96ac102ed0 0x7f96ac199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:39.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.732+0000 7f96b3e52700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96ac107c30 0x7f96ac19da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:39.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.732+0000 7f96b3e52700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac111cb0 con 0x7f96ac102ed0 2024-04-21T08:35:39.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.732+0000 7f96b3e52700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac111b30 con 0x7f96ac102510 2024-04-21T08:35:39.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.732+0000 7f96b3e52700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac111e30 con 0x7f96ac107c30 2024-04-21T08:35:39.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.732+0000 7f96b13ed700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96ac102ed0 0x7f96ac199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:39.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.732+0000 7f96b13ed700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96ac102ed0 0x7f96ac199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47042/0 (socket says 172.21.15.43:47042) 2024-04-21T08:35:39.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.732+0000 7f96b13ed700 1 -- 172.21.15.43:0/4066034074 learned_addr learned my addr 172.21.15.43:0/4066034074 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:35:39.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.732+0000 7f96b1bee700 1 --2- 172.21.15.43:0/4066034074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96ac102510 0x7f96ac199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:39.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.732+0000 7f96b23ef700 1 --2- 172.21.15.43:0/4066034074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96ac107c30 0x7f96ac19da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:39.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.732+0000 7f96b13ed700 1 -- 172.21.15.43:0/4066034074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96ac107c30 msgr2=0x7f96ac19da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:39.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.732+0000 7f96b13ed700 1 --2- 172.21.15.43:0/4066034074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96ac107c30 0x7f96ac19da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:39.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.732+0000 7f96b13ed700 1 -- 172.21.15.43:0/4066034074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96ac102510 msgr2=0x7f96ac199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:39.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.732+0000 7f96b13ed700 1 --2- 172.21.15.43:0/4066034074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96ac102510 0x7f96ac199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:39.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.732+0000 7f96b13ed700 1 -- 172.21.15.43:0/4066034074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ac19e1c0 con 0x7f96ac102ed0 2024-04-21T08:35:39.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.733+0000 7f96b13ed700 1 --2- 172.21.15.43:0/4066034074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96ac102ed0 0x7f96ac199740 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f969c00b530 tx=0x7f969c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:39.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.733+0000 7f96a2ffd700 1 -- 172.21.15.43:0/4066034074 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c015020 con 0x7f96ac102ed0 2024-04-21T08:35:39.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.733+0000 7f96a2ffd700 1 -- 172.21.15.43:0/4066034074 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f969c011ce0 con 0x7f96ac102ed0 2024-04-21T08:35:39.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.733+0000 7f96b3e52700 1 -- 172.21.15.43:0/4066034074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96ac19e4b0 con 0x7f96ac102ed0 2024-04-21T08:35:39.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.733+0000 7f96a2ffd700 1 -- 172.21.15.43:0/4066034074 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c010e40 con 0x7f96ac102ed0 2024-04-21T08:35:39.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.733+0000 7f96b3e52700 1 -- 172.21.15.43:0/4066034074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96ac1a5d30 con 0x7f96ac102ed0 2024-04-21T08:35:39.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.734+0000 7f96b3e52700 1 -- 172.21.15.43:0/4066034074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96ac069730 con 0x7f96ac102ed0 2024-04-21T08:35:39.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.735+0000 7f96a2ffd700 1 -- 172.21.15.43:0/4066034074 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f969c018020 con 0x7f96ac102ed0 2024-04-21T08:35:39.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.735+0000 7f96a2ffd700 1 --2- 172.21.15.43:0/4066034074 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f96980a3cc0 0x7f96980a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:39.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.736+0000 7f96b1bee700 1 --2- 172.21.15.43:0/4066034074 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f96980a3cc0 0x7f96980a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:39.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.736+0000 7f96a2ffd700 1 -- 172.21.15.43:0/4066034074 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f969c0cf730 con 0x7f96ac102ed0 2024-04-21T08:35:39.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.736+0000 7f96b1bee700 1 --2- 172.21.15.43:0/4066034074 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f96980a3cc0 0x7f96980a6180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f96a800ac50 tx=0x7f96a800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:39.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.740+0000 7f96a2ffd700 1 -- 172.21.15.43:0/4066034074 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f969c01f020 con 0x7f96ac102ed0 2024-04-21T08:35:39.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:39 smithi043 ceph-mon[27978]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:39.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.947+0000 7f96b3e52700 1 -- 172.21.15.43:0/4066034074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f96ac19a510 con 0x7f96ac102ed0 2024-04-21T08:35:39.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.948+0000 7f96a2ffd700 1 -- 172.21.15.43:0/4066034074 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f969c09c5b0 con 0x7f96ac102ed0 2024-04-21T08:35:39.948 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-21T08:35:39.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.950+0000 7f96b3e52700 1 -- 172.21.15.43:0/4066034074 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f96980a3cc0 msgr2=0x7f96980a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:39.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.950+0000 7f96b3e52700 1 --2- 172.21.15.43:0/4066034074 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f96980a3cc0 0x7f96980a6180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f96a800ac50 tx=0x7f96a800a380 comp rx=0 tx=0).stop 2024-04-21T08:35:39.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.950+0000 7f96b3e52700 1 -- 172.21.15.43:0/4066034074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96ac102ed0 msgr2=0x7f96ac199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:39.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.950+0000 7f96b3e52700 1 --2- 172.21.15.43:0/4066034074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96ac102ed0 0x7f96ac199740 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f969c00b530 tx=0x7f969c00b840 comp rx=0 tx=0).stop 2024-04-21T08:35:39.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.950+0000 7f96b3e52700 1 -- 172.21.15.43:0/4066034074 shutdown_connections 2024-04-21T08:35:39.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.950+0000 7f96b3e52700 1 --2- 172.21.15.43:0/4066034074 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f96980a3cc0 0x7f96980a6180 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:39.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.950+0000 7f96b3e52700 1 --2- 172.21.15.43:0/4066034074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96ac107c30 0x7f96ac19da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:39.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.950+0000 7f96b3e52700 1 --2- 172.21.15.43:0/4066034074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96ac102510 0x7f96ac199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:39.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.950+0000 7f96b3e52700 1 --2- 172.21.15.43:0/4066034074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96ac102ed0 0x7f96ac199740 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:39.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.950+0000 7f96b3e52700 1 -- 172.21.15.43:0/4066034074 >> 172.21.15.43:0/4066034074 conn(0x7f96ac076c60 msgr2=0x7f96ac101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:39.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.950+0000 7f96b3e52700 1 -- 172.21.15.43:0/4066034074 shutdown_connections 2024-04-21T08:35:39.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:39.950+0000 7f96b3e52700 1 -- 172.21.15.43:0/4066034074 wait complete. 2024-04-21T08:35:39.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:39 smithi175 ceph-mon[31920]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:40.043 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:39 smithi112 ceph-mon[31085]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:40.788 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:40 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/4066034074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:35:40.824 INFO:tasks.cephadm:Adding mds.a on smithi043 2024-04-21T08:35:40.824 INFO:tasks.cephadm:Adding mds.d on smithi043 2024-04-21T08:35:40.824 INFO:tasks.cephadm:Adding mds.g on smithi043 2024-04-21T08:35:40.824 INFO:tasks.cephadm:Adding mds.j on smithi043 2024-04-21T08:35:40.824 INFO:tasks.cephadm:Adding mds.b on smithi112 2024-04-21T08:35:40.824 INFO:tasks.cephadm:Adding mds.e on smithi112 2024-04-21T08:35:40.824 INFO:tasks.cephadm:Adding mds.h on smithi112 2024-04-21T08:35:40.824 INFO:tasks.cephadm:Adding mds.k on smithi112 2024-04-21T08:35:40.824 INFO:tasks.cephadm:Adding mds.c on smithi175 2024-04-21T08:35:40.825 INFO:tasks.cephadm:Adding mds.f on smithi175 2024-04-21T08:35:40.825 INFO:tasks.cephadm:Adding mds.i on smithi175 2024-04-21T08:35:40.825 INFO:tasks.cephadm:Adding mds.l on smithi175 2024-04-21T08:35:40.825 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l' 2024-04-21T08:35:40.852 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:40 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/4066034074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:35:41.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:40 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/4066034074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:35:41.073 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:35:41.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:41 smithi175 ceph-mon[31920]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:42.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:41 smithi112 ceph-mon[31085]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:42.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:41 smithi043 ceph-mon[27978]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:42.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.805+0000 7fd1c95f3700 1 -- 172.21.15.175:0/3563053501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1c4104670 msgr2=0x7fd1c410af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:42.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.805+0000 7fd1c95f3700 1 --2- 172.21.15.175:0/3563053501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1c4104670 0x7fd1c410af20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00b0a0 tx=0x7fd1bc00b3b0 comp rx=0 tx=0).stop 2024-04-21T08:35:42.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.805+0000 7fd1c95f3700 1 -- 172.21.15.175:0/3563053501 shutdown_connections 2024-04-21T08:35:42.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.805+0000 7fd1c95f3700 1 --2- 172.21.15.175:0/3563053501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1c4104670 0x7fd1c410af20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:42.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.805+0000 7fd1c95f3700 1 --2- 172.21.15.175:0/3563053501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c4102ad0 0x7fd1c4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:42.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.805+0000 7fd1c95f3700 1 --2- 172.21.15.175:0/3563053501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1c4103cd0 0x7fd1c4104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:42.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.805+0000 7fd1c95f3700 1 -- 172.21.15.175:0/3563053501 >> 172.21.15.175:0/3563053501 conn(0x7fd1c40fe090 msgr2=0x7fd1c41004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:42.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.805+0000 7fd1c95f3700 1 -- 172.21.15.175:0/3563053501 shutdown_connections 2024-04-21T08:35:42.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.806+0000 7fd1c95f3700 1 -- 172.21.15.175:0/3563053501 wait complete. 2024-04-21T08:35:42.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.806+0000 7fd1c95f3700 1 Processor -- start 2024-04-21T08:35:42.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.806+0000 7fd1c95f3700 1 -- start start 2024-04-21T08:35:42.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.806+0000 7fd1c95f3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1c4102ad0 0x7fd1c4198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:42.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.806+0000 7fd1c95f3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c4103cd0 0x7fd1c4199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:42.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.806+0000 7fd1c95f3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1c4104670 0x7fd1c41a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:42.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.806+0000 7fd1c95f3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c410e5e0 con 0x7fd1c4104670 2024-04-21T08:35:42.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.806+0000 7fd1c95f3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c410e460 con 0x7fd1c4103cd0 2024-04-21T08:35:42.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.806+0000 7fd1c95f3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c410e760 con 0x7fd1c4102ad0 2024-04-21T08:35:42.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.807+0000 7fd1c2ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1c4102ad0 0x7fd1c4198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:42.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.807+0000 7fd1c27fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c4103cd0 0x7fd1c4199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:42.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.807+0000 7fd1c2ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1c4102ad0 0x7fd1c4198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:53886/0 (socket says 172.21.15.175:53886) 2024-04-21T08:35:42.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.807+0000 7fd1c2ffd700 1 -- 172.21.15.175:0/3289002196 learned_addr learned my addr 172.21.15.175:0/3289002196 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-21T08:35:42.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.807+0000 7fd1c37fe700 1 --2- 172.21.15.175:0/3289002196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1c4104670 0x7fd1c41a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:42.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.807+0000 7fd1c2ffd700 1 -- 172.21.15.175:0/3289002196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c4103cd0 msgr2=0x7fd1c4199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:42.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.807+0000 7fd1c2ffd700 1 --2- 172.21.15.175:0/3289002196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c4103cd0 0x7fd1c4199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:42.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.807+0000 7fd1c2ffd700 1 -- 172.21.15.175:0/3289002196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1c4104670 msgr2=0x7fd1c41a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:42.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.807+0000 7fd1c2ffd700 1 --2- 172.21.15.175:0/3289002196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1c4104670 0x7fd1c41a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:42.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.807+0000 7fd1c2ffd700 1 -- 172.21.15.175:0/3289002196 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c41a0830 con 0x7fd1c4102ad0 2024-04-21T08:35:42.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.807+0000 7fd1c2ffd700 1 --2- 172.21.15.175:0/3289002196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1c4102ad0 0x7fd1c4198bd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400b530 tx=0x7fd1b400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:42.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.807+0000 7fd1abfff700 1 -- 172.21.15.175:0/3289002196 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4015020 con 0x7fd1c4102ad0 2024-04-21T08:35:42.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.807+0000 7fd1c95f3700 1 -- 172.21.15.175:0/3289002196 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c41a0b20 con 0x7fd1c4102ad0 2024-04-21T08:35:42.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.807+0000 7fd1abfff700 1 -- 172.21.15.175:0/3289002196 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1b4011ce0 con 0x7fd1c4102ad0 2024-04-21T08:35:42.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.807+0000 7fd1abfff700 1 -- 172.21.15.175:0/3289002196 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4010c60 con 0x7fd1c4102ad0 2024-04-21T08:35:42.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.807+0000 7fd1c95f3700 1 -- 172.21.15.175:0/3289002196 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c41a10b0 con 0x7fd1c4102ad0 2024-04-21T08:35:42.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.809+0000 7fd1c95f3700 1 -- 172.21.15.175:0/3289002196 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c4069730 con 0x7fd1c4102ad0 2024-04-21T08:35:42.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.809+0000 7fd1abfff700 1 -- 172.21.15.175:0/3289002196 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd1b4018020 con 0x7fd1c4102ad0 2024-04-21T08:35:42.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.810+0000 7fd1abfff700 1 --2- 172.21.15.175:0/3289002196 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd1ac0a3c40 0x7fd1ac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:42.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.810+0000 7fd1abfff700 1 -- 172.21.15.175:0/3289002196 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd1b40cf610 con 0x7fd1c4102ad0 2024-04-21T08:35:42.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.810+0000 7fd1c27fc700 1 --2- 172.21.15.175:0/3289002196 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd1ac0a3c40 0x7fd1ac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:42.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.811+0000 7fd1c27fc700 1 --2- 172.21.15.175:0/3289002196 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd1ac0a3c40 0x7fd1ac0a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8005950 tx=0x7fd1b800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:42.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:42.815+0000 7fd1abfff700 1 -- 172.21.15.175:0/3289002196 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd1b401f020 con 0x7fd1c4102ad0 2024-04-21T08:35:43.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:43.024+0000 7fd1c95f3700 1 -- 172.21.15.175:0/3289002196 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l", "target": ["mon-mgr", ""]}) -- 0x7fd1c4063080 con 0x7fd1ac0a3c40 2024-04-21T08:35:43.031 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:43.032+0000 7fd1abfff700 1 -- 172.21.15.175:0/3289002196 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fd1c4063080 con 0x7fd1ac0a3c40 2024-04-21T08:35:43.032 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled mds.all update... 2024-04-21T08:35:43.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:43.034+0000 7fd1c95f3700 1 -- 172.21.15.175:0/3289002196 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd1ac0a3c40 msgr2=0x7fd1ac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:43.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:43.034+0000 7fd1c95f3700 1 --2- 172.21.15.175:0/3289002196 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd1ac0a3c40 0x7fd1ac0a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8005950 tx=0x7fd1b800b410 comp rx=0 tx=0).stop 2024-04-21T08:35:43.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:43.034+0000 7fd1c95f3700 1 -- 172.21.15.175:0/3289002196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1c4102ad0 msgr2=0x7fd1c4198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:43.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:43.034+0000 7fd1c95f3700 1 --2- 172.21.15.175:0/3289002196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1c4102ad0 0x7fd1c4198bd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400b530 tx=0x7fd1b400b840 comp rx=0 tx=0).stop 2024-04-21T08:35:43.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:43.034+0000 7fd1c95f3700 1 -- 172.21.15.175:0/3289002196 shutdown_connections 2024-04-21T08:35:43.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:43.034+0000 7fd1c95f3700 1 --2- 172.21.15.175:0/3289002196 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd1ac0a3c40 0x7fd1ac0a6100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:43.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:43.034+0000 7fd1c95f3700 1 --2- 172.21.15.175:0/3289002196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1c4102ad0 0x7fd1c4198bd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:43.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:43.034+0000 7fd1c95f3700 1 --2- 172.21.15.175:0/3289002196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1c4104670 0x7fd1c41a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:43.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:43.034+0000 7fd1c95f3700 1 --2- 172.21.15.175:0/3289002196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1c4103cd0 0x7fd1c4199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:43.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:43.034+0000 7fd1c95f3700 1 -- 172.21.15.175:0/3289002196 >> 172.21.15.175:0/3289002196 conn(0x7fd1c40fe090 msgr2=0x7fd1c4106fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:43.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:43.035+0000 7fd1c95f3700 1 -- 172.21.15.175:0/3289002196 shutdown_connections 2024-04-21T08:35:43.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-21T08:35:43.035+0000 7fd1c95f3700 1 -- 172.21.15.175:0/3289002196 wait complete. 2024-04-21T08:35:43.699 DEBUG:teuthology.orchestra.run.smithi043:mds.a> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.a.service 2024-04-21T08:35:43.702 DEBUG:teuthology.orchestra.run.smithi043:mds.d> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.d.service 2024-04-21T08:35:43.745 DEBUG:teuthology.orchestra.run.smithi043:mds.g> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.g.service 2024-04-21T08:35:43.788 DEBUG:teuthology.orchestra.run.smithi043:mds.j> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.j.service 2024-04-21T08:35:43.831 DEBUG:teuthology.orchestra.run.smithi112:mds.b> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.b.service 2024-04-21T08:35:43.833 DEBUG:teuthology.orchestra.run.smithi112:mds.e> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.e.service 2024-04-21T08:35:43.852 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:43 smithi043 ceph-mon[27978]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:43.852 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:43 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:43.852 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:43 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:43.852 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:43 smithi112 ceph-mon[31085]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:43.852 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:43 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:43.853 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:43 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:43.853 DEBUG:teuthology.orchestra.run.smithi112:mds.h> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.h.service 2024-04-21T08:35:43.896 DEBUG:teuthology.orchestra.run.smithi112:mds.k> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.k.service 2024-04-21T08:35:43.939 DEBUG:teuthology.orchestra.run.smithi175:mds.c> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.c.service 2024-04-21T08:35:43.941 DEBUG:teuthology.orchestra.run.smithi175:mds.f> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.f.service 2024-04-21T08:35:43.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:43 smithi175 ceph-mon[31920]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:43.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:43 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:43.960 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:43 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:43.961 DEBUG:teuthology.orchestra.run.smithi175:mds.i> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.i.service 2024-04-21T08:35:44.004 DEBUG:teuthology.orchestra.run.smithi175:mds.l> sudo journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.l.service 2024-04-21T08:35:44.047 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-21T08:35:44.047 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:35:44.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.164+0000 7f5d3c32b640 1 -- 172.21.15.43:0/2004750167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d34100ea0 msgr2=0x7f5d341012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:44.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.164+0000 7f5d3c32b640 1 --2- 172.21.15.43:0/2004750167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d34100ea0 0x7f5d341012c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009ca0 tx=0x7f5d2c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.164+0000 7f5d3c32b640 1 -- 172.21.15.43:0/2004750167 shutdown_connections 2024-04-21T08:35:44.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.164+0000 7f5d3c32b640 1 --2- 172.21.15.43:0/2004750167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d341029b0 0x7f5d341092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.164+0000 7f5d3c32b640 1 --2- 172.21.15.43:0/2004750167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d34101ff0 0x7f5d34102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.164+0000 7f5d3c32b640 1 --2- 172.21.15.43:0/2004750167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d34100ea0 0x7f5d341012c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.164+0000 7f5d3c32b640 1 -- 172.21.15.43:0/2004750167 >> 172.21.15.43:0/2004750167 conn(0x7f5d340fc410 msgr2=0x7f5d340fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:44.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.164+0000 7f5d3c32b640 1 -- 172.21.15.43:0/2004750167 shutdown_connections 2024-04-21T08:35:44.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.164+0000 7f5d3c32b640 1 -- 172.21.15.43:0/2004750167 wait complete. 2024-04-21T08:35:44.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.165+0000 7f5d3c32b640 1 Processor -- start 2024-04-21T08:35:44.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.165+0000 7f5d3c32b640 1 -- start start 2024-04-21T08:35:44.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.165+0000 7f5d3c32b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d34100ea0 0x7f5d3410b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:44.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.165+0000 7f5d3c32b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d34101ff0 0x7f5d34109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:44.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.165+0000 7f5d3c32b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d341029b0 0x7f5d3410a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:44.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.165+0000 7f5d3c32b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3410db10 con 0x7f5d34101ff0 2024-04-21T08:35:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.165+0000 7f5d3c32b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3410d990 con 0x7f5d34100ea0 2024-04-21T08:35:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.165+0000 7f5d3c32b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3410dc90 con 0x7f5d341029b0 2024-04-21T08:35:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.165+0000 7f5d398c6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d34101ff0 0x7f5d34109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.165+0000 7f5d398c6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d34101ff0 0x7f5d34109cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47060/0 (socket says 172.21.15.43:47060) 2024-04-21T08:35:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.165+0000 7f5d398c6640 1 -- 172.21.15.43:0/597230583 learned_addr learned my addr 172.21.15.43:0/597230583 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:35:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.165+0000 7f5d3a0c7640 1 --2- 172.21.15.43:0/597230583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d34100ea0 0x7f5d3410b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.165+0000 7f5d3a8c8640 1 --2- 172.21.15.43:0/597230583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d341029b0 0x7f5d3410a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.165+0000 7f5d398c6640 1 -- 172.21.15.43:0/597230583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d341029b0 msgr2=0x7f5d3410a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.165+0000 7f5d398c6640 1 --2- 172.21.15.43:0/597230583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d341029b0 0x7f5d3410a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.165+0000 7f5d398c6640 1 -- 172.21.15.43:0/597230583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d34100ea0 msgr2=0x7f5d3410b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.165+0000 7f5d398c6640 1 --2- 172.21.15.43:0/597230583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d34100ea0 0x7f5d3410b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.165+0000 7f5d398c6640 1 -- 172.21.15.43:0/597230583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3410aaa0 con 0x7f5d34101ff0 2024-04-21T08:35:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.166+0000 7f5d398c6640 1 --2- 172.21.15.43:0/597230583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d34101ff0 0x7f5d34109cd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800e9c0 tx=0x7f5d2800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.166+0000 7f5d337fe640 1 -- 172.21.15.43:0/597230583 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2800cd80 con 0x7f5d34101ff0 2024-04-21T08:35:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.166+0000 7f5d337fe640 1 -- 172.21.15.43:0/597230583 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d280035f0 con 0x7f5d34101ff0 2024-04-21T08:35:44.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.166+0000 7f5d3c32b640 1 -- 172.21.15.43:0/597230583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d341a22b0 con 0x7f5d34101ff0 2024-04-21T08:35:44.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.166+0000 7f5d337fe640 1 -- 172.21.15.43:0/597230583 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d28010690 con 0x7f5d34101ff0 2024-04-21T08:35:44.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.166+0000 7f5d3c32b640 1 -- 172.21.15.43:0/597230583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d341a2800 con 0x7f5d34101ff0 2024-04-21T08:35:44.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.167+0000 7f5d3c32b640 1 -- 172.21.15.43:0/597230583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d341081b0 con 0x7f5d34101ff0 2024-04-21T08:35:44.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.168+0000 7f5d337fe640 1 -- 172.21.15.43:0/597230583 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5d28004000 con 0x7f5d34101ff0 2024-04-21T08:35:44.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.168+0000 7f5d337fe640 1 --2- 172.21.15.43:0/597230583 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f5d1c0a3c30 0x7f5d1c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:44.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.169+0000 7f5d3a0c7640 1 --2- 172.21.15.43:0/597230583 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f5d1c0a3c30 0x7f5d1c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:44.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.169+0000 7f5d337fe640 1 -- 172.21.15.43:0/597230583 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5d280d0440 con 0x7f5d34101ff0 2024-04-21T08:35:44.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.169+0000 7f5d3a0c7640 1 --2- 172.21.15.43:0/597230583 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f5d1c0a3c30 0x7f5d1c0a60f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c000f80 tx=0x7f5d2c032340 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:44.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.173+0000 7f5d337fe640 1 -- 172.21.15.43:0/597230583 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d2809d2c0 con 0x7f5d34101ff0 2024-04-21T08:35:44.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.362+0000 7f5d3c32b640 1 -- 172.21.15.43:0/597230583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d3402d810 con 0x7f5d34101ff0 2024-04-21T08:35:44.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.363+0000 7f5d337fe640 1 -- 172.21.15.43:0/597230583 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18480 (secure 0 0 0) 0x7f5d2809cbc0 con 0x7f5d34101ff0 2024-04-21T08:35:44.363 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:35:44.363 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":82,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:35:33.554402+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:35:44.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.366+0000 7f5d3c32b640 1 -- 172.21.15.43:0/597230583 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f5d1c0a3c30 msgr2=0x7f5d1c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:44.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.366+0000 7f5d3c32b640 1 --2- 172.21.15.43:0/597230583 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f5d1c0a3c30 0x7f5d1c0a60f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c000f80 tx=0x7f5d2c032340 comp rx=0 tx=0).stop 2024-04-21T08:35:44.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.366+0000 7f5d3c32b640 1 -- 172.21.15.43:0/597230583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d34101ff0 msgr2=0x7f5d34109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:44.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.366+0000 7f5d3c32b640 1 --2- 172.21.15.43:0/597230583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d34101ff0 0x7f5d34109cd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800e9c0 tx=0x7f5d2800ee80 comp rx=0 tx=0).stop 2024-04-21T08:35:44.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.366+0000 7f5d3c32b640 1 -- 172.21.15.43:0/597230583 shutdown_connections 2024-04-21T08:35:44.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.366+0000 7f5d3c32b640 1 --2- 172.21.15.43:0/597230583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d341029b0 0x7f5d3410a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.366+0000 7f5d3c32b640 1 --2- 172.21.15.43:0/597230583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d34101ff0 0x7f5d34109cd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.366+0000 7f5d3c32b640 1 --2- 172.21.15.43:0/597230583 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f5d1c0a3c30 0x7f5d1c0a60f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.366+0000 7f5d3c32b640 1 --2- 172.21.15.43:0/597230583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d34100ea0 0x7f5d3410b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.366+0000 7f5d3c32b640 1 -- 172.21.15.43:0/597230583 >> 172.21.15.43:0/597230583 conn(0x7f5d340fc410 msgr2=0x7f5d34105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:44.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.366+0000 7f5d3c32b640 1 -- 172.21.15.43:0/597230583 shutdown_connections 2024-04-21T08:35:44.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.366+0000 7f5d3c32b640 1 -- 172.21.15.43:0/597230583 wait complete. 2024-04-21T08:35:44.376 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:39.981405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-21T08:35:44.377 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:35:44.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.482+0000 7f434be93640 1 -- 172.21.15.43:0/3307205542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4344105360 msgr2=0x7f4344107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:44.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.482+0000 7f434be93640 1 --2- 172.21.15.43:0/3307205542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4344105360 0x7f4344107750 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4334009860 tx=0x7f433402bb70 comp rx=0 tx=0).stop 2024-04-21T08:35:44.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.482+0000 7f434be93640 1 -- 172.21.15.43:0/3307205542 shutdown_connections 2024-04-21T08:35:44.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.482+0000 7f434be93640 1 --2- 172.21.15.43:0/3307205542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4344107c90 0x7f434410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.482+0000 7f434be93640 1 --2- 172.21.15.43:0/3307205542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4344105360 0x7f4344107750 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.482+0000 7f434be93640 1 --2- 172.21.15.43:0/3307205542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4344102a30 0x7f4344104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.482+0000 7f434be93640 1 -- 172.21.15.43:0/3307205542 >> 172.21.15.43:0/3307205542 conn(0x7f43440fc440 msgr2=0x7f43440fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:44.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.482+0000 7f434be93640 1 -- 172.21.15.43:0/3307205542 shutdown_connections 2024-04-21T08:35:44.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.482+0000 7f434be93640 1 -- 172.21.15.43:0/3307205542 wait complete. 2024-04-21T08:35:44.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.483+0000 7f434be93640 1 Processor -- start 2024-04-21T08:35:44.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.483+0000 7f434be93640 1 -- start start 2024-04-21T08:35:44.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.483+0000 7f434be93640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4344102a30 0x7f4344198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:44.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.483+0000 7f434be93640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4344105360 0x7f4344198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.483+0000 7f434be93640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4344107c90 0x7f434419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.483+0000 7f434be93640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f434410daa0 con 0x7f4344102a30 2024-04-21T08:35:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.483+0000 7f434be93640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f434410d920 con 0x7f4344105360 2024-04-21T08:35:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.483+0000 7f434be93640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f434410dc20 con 0x7f4344107c90 2024-04-21T08:35:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.483+0000 7f4349c2f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4344102a30 0x7f4344198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.483+0000 7f4349c2f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4344102a30 0x7f4344198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47066/0 (socket says 172.21.15.43:47066) 2024-04-21T08:35:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.483+0000 7f4349c2f640 1 -- 172.21.15.43:0/82515649 learned_addr learned my addr 172.21.15.43:0/82515649 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:35:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.483+0000 7f434a430640 1 --2- 172.21.15.43:0/82515649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4344107c90 0x7f434419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.484+0000 7f434942e640 1 --2- 172.21.15.43:0/82515649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4344105360 0x7f4344198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.484+0000 7f4349c2f640 1 -- 172.21.15.43:0/82515649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4344107c90 msgr2=0x7f434419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.484+0000 7f4349c2f640 1 --2- 172.21.15.43:0/82515649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4344107c90 0x7f434419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.484+0000 7f4349c2f640 1 -- 172.21.15.43:0/82515649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4344105360 msgr2=0x7f4344198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.484+0000 7f4349c2f640 1 --2- 172.21.15.43:0/82515649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4344105360 0x7f4344198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.484+0000 7f4349c2f640 1 -- 172.21.15.43:0/82515649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f434419fe50 con 0x7f4344102a30 2024-04-21T08:35:44.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.484+0000 7f4349c2f640 1 --2- 172.21.15.43:0/82515649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4344102a30 0x7f4344198230 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f434000d950 tx=0x7f434000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:44.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.484+0000 7f433affd640 1 -- 172.21.15.43:0/82515649 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4340017070 con 0x7f4344102a30 2024-04-21T08:35:44.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.484+0000 7f434be93640 1 -- 172.21.15.43:0/82515649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43441a0140 con 0x7f4344102a30 2024-04-21T08:35:44.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.484+0000 7f433affd640 1 -- 172.21.15.43:0/82515649 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4340009c30 con 0x7f4344102a30 2024-04-21T08:35:44.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.484+0000 7f433affd640 1 -- 172.21.15.43:0/82515649 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434000f620 con 0x7f4344102a30 2024-04-21T08:35:44.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.484+0000 7f434be93640 1 -- 172.21.15.43:0/82515649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43441a06b0 con 0x7f4344102a30 2024-04-21T08:35:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.485+0000 7f434be93640 1 -- 172.21.15.43:0/82515649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43440fdf30 con 0x7f4344102a30 2024-04-21T08:35:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.486+0000 7f433affd640 1 -- 172.21.15.43:0/82515649 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4340010460 con 0x7f4344102a30 2024-04-21T08:35:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.487+0000 7f433affd640 1 --2- 172.21.15.43:0/82515649 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f43300a3bb0 0x7f43300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.487+0000 7f434942e640 1 --2- 172.21.15.43:0/82515649 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f43300a3bb0 0x7f43300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.487+0000 7f433affd640 1 -- 172.21.15.43:0/82515649 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f43400cfc10 con 0x7f4344102a30 2024-04-21T08:35:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.487+0000 7f434942e640 1 --2- 172.21.15.43:0/82515649 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f43300a3bb0 0x7f43300a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4334009830 tx=0x7f4334034580 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:44.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.490+0000 7f433affd640 1 -- 172.21.15.43:0/82515649 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f434009c9f0 con 0x7f4344102a30 2024-04-21T08:35:44.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.675+0000 7f434be93640 1 -- 172.21.15.43:0/82515649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f434402d5c0 con 0x7f4344102a30 2024-04-21T08:35:44.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.675+0000 7f433affd640 1 -- 172.21.15.43:0/82515649 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f4340009de0 con 0x7f4344102a30 2024-04-21T08:35:44.675 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-21T08:35:44.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.677+0000 7f434be93640 1 -- 172.21.15.43:0/82515649 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f43300a3bb0 msgr2=0x7f43300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:44.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.677+0000 7f434be93640 1 --2- 172.21.15.43:0/82515649 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f43300a3bb0 0x7f43300a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4334009830 tx=0x7f4334034580 comp rx=0 tx=0).stop 2024-04-21T08:35:44.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.677+0000 7f434be93640 1 -- 172.21.15.43:0/82515649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4344102a30 msgr2=0x7f4344198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:44.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.677+0000 7f434be93640 1 --2- 172.21.15.43:0/82515649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4344102a30 0x7f4344198230 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f434000d950 tx=0x7f434000de10 comp rx=0 tx=0).stop 2024-04-21T08:35:44.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.677+0000 7f434be93640 1 -- 172.21.15.43:0/82515649 shutdown_connections 2024-04-21T08:35:44.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.677+0000 7f434be93640 1 --2- 172.21.15.43:0/82515649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4344107c90 0x7f434419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.677+0000 7f434be93640 1 --2- 172.21.15.43:0/82515649 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f43300a3bb0 0x7f43300a6070 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.677+0000 7f434be93640 1 --2- 172.21.15.43:0/82515649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4344105360 0x7f4344198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.677+0000 7f434be93640 1 --2- 172.21.15.43:0/82515649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4344102a30 0x7f4344198230 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.677+0000 7f434be93640 1 -- 172.21.15.43:0/82515649 >> 172.21.15.43:0/82515649 conn(0x7f43440fc440 msgr2=0x7f4344103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:44.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.678+0000 7f434be93640 1 -- 172.21.15.43:0/82515649 shutdown_connections 2024-04-21T08:35:44.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.678+0000 7f434be93640 1 -- 172.21.15.43:0/82515649 wait complete. 2024-04-21T08:35:44.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:44 smithi043 ceph-mon[27978]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:35:44.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:44 smithi043 ceph-mon[27978]: Saving service mds.all spec with placement smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l;count:12 2024-04-21T08:35:44.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:44 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:44.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:44 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:44.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:44 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:44.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:44 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:44.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:44 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:44.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:44 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:44.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:44 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/597230583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:35:44.688 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:35:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.790+0000 7fe96f2b3640 1 -- 172.21.15.43:0/866287151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe968100e70 msgr2=0x7fe968101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.790+0000 7fe96f2b3640 1 --2- 172.21.15.43:0/866287151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe968100e70 0x7fe968101290 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe960009ca0 tx=0x7fe96002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.790+0000 7fe96f2b3640 1 -- 172.21.15.43:0/866287151 shutdown_connections 2024-04-21T08:35:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.790+0000 7fe96f2b3640 1 --2- 172.21.15.43:0/866287151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe968102980 0x7fe9681092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.790+0000 7fe96f2b3640 1 --2- 172.21.15.43:0/866287151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe968101fc0 0x7fe968102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.790+0000 7fe96f2b3640 1 --2- 172.21.15.43:0/866287151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe968100e70 0x7fe968101290 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.790+0000 7fe96f2b3640 1 -- 172.21.15.43:0/866287151 >> 172.21.15.43:0/866287151 conn(0x7fe9680fc3e0 msgr2=0x7fe9680fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.791+0000 7fe96f2b3640 1 -- 172.21.15.43:0/866287151 shutdown_connections 2024-04-21T08:35:44.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.791+0000 7fe96f2b3640 1 -- 172.21.15.43:0/866287151 wait complete. 2024-04-21T08:35:44.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.791+0000 7fe96f2b3640 1 Processor -- start 2024-04-21T08:35:44.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.791+0000 7fe96f2b3640 1 -- start start 2024-04-21T08:35:44.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.791+0000 7fe96f2b3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe968100e70 0x7fe968104df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:44.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.791+0000 7fe96f2b3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe968101fc0 0x7fe968103430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:44.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.791+0000 7fe96f2b3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe968102980 0x7fe968103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:44.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.791+0000 7fe96f2b3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe96810c9a0 con 0x7fe968100e70 2024-04-21T08:35:44.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.791+0000 7fe96f2b3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe96810c820 con 0x7fe968101fc0 2024-04-21T08:35:44.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.791+0000 7fe96f2b3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe96810cb20 con 0x7fe968102980 2024-04-21T08:35:44.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.791+0000 7fe96d04f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe968100e70 0x7fe968104df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:44.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.792+0000 7fe96d04f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe968100e70 0x7fe968104df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47074/0 (socket says 172.21.15.43:47074) 2024-04-21T08:35:44.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.792+0000 7fe96d04f640 1 -- 172.21.15.43:0/2719354025 learned_addr learned my addr 172.21.15.43:0/2719354025 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:35:44.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.792+0000 7fe96d850640 1 --2- 172.21.15.43:0/2719354025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe968102980 0x7fe968103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:44.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.792+0000 7fe96c84e640 1 --2- 172.21.15.43:0/2719354025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe968101fc0 0x7fe968103430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:44.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.792+0000 7fe96d04f640 1 -- 172.21.15.43:0/2719354025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe968102980 msgr2=0x7fe968103970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:44.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.792+0000 7fe96d04f640 1 --2- 172.21.15.43:0/2719354025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe968102980 0x7fe968103970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.792+0000 7fe96d04f640 1 -- 172.21.15.43:0/2719354025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe968101fc0 msgr2=0x7fe968103430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:44.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.792+0000 7fe96d04f640 1 --2- 172.21.15.43:0/2719354025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe968101fc0 0x7fe968103430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.792+0000 7fe96d04f640 1 -- 172.21.15.43:0/2719354025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe968104200 con 0x7fe968100e70 2024-04-21T08:35:44.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.792+0000 7fe96c84e640 1 --2- 172.21.15.43:0/2719354025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe968101fc0 0x7fe968103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:35:44.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.792+0000 7fe96d850640 1 --2- 172.21.15.43:0/2719354025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe968102980 0x7fe968103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:35:44.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.792+0000 7fe96d04f640 1 --2- 172.21.15.43:0/2719354025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe968100e70 0x7fe968104df0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe960005e30 tx=0x7fe960034830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:44.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.792+0000 7fe9667fc640 1 -- 172.21.15.43:0/2719354025 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960039040 con 0x7fe968100e70 2024-04-21T08:35:44.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.792+0000 7fe9667fc640 1 -- 172.21.15.43:0/2719354025 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe960035c30 con 0x7fe968100e70 2024-04-21T08:35:44.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.792+0000 7fe96f2b3640 1 -- 172.21.15.43:0/2719354025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9681a22b0 con 0x7fe968100e70 2024-04-21T08:35:44.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.792+0000 7fe9667fc640 1 -- 172.21.15.43:0/2719354025 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9600042a0 con 0x7fe968100e70 2024-04-21T08:35:44.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.792+0000 7fe96f2b3640 1 -- 172.21.15.43:0/2719354025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9681a2850 con 0x7fe968100e70 2024-04-21T08:35:44.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.793+0000 7fe96f2b3640 1 -- 172.21.15.43:0/2719354025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9681a2ab0 con 0x7fe968100e70 2024-04-21T08:35:44.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.794+0000 7fe9667fc640 1 -- 172.21.15.43:0/2719354025 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe960046030 con 0x7fe968100e70 2024-04-21T08:35:44.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.795+0000 7fe9667fc640 1 --2- 172.21.15.43:0/2719354025 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe9500a3c30 0x7fe9500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:44.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.795+0000 7fe96c84e640 1 --2- 172.21.15.43:0/2719354025 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe9500a3c30 0x7fe9500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:44.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.795+0000 7fe9667fc640 1 -- 172.21.15.43:0/2719354025 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe9600f1ef0 con 0x7fe968100e70 2024-04-21T08:35:44.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.796+0000 7fe96c84e640 1 --2- 172.21.15.43:0/2719354025 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe9500a3c30 0x7fe9500a60f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0058c0 tx=0x7fe95c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:44.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:44.799+0000 7fe9667fc640 1 -- 172.21.15.43:0/2719354025 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe96004d060 con 0x7fe968100e70 2024-04-21T08:35:44.841 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:44 smithi175 ceph-mon[31920]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:35:44.841 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:44 smithi175 ceph-mon[31920]: Saving service mds.all spec with placement smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l;count:12 2024-04-21T08:35:44.841 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:44 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:44.841 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:44 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:44.841 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:44 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:44.841 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:44 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:44.841 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:44 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:44.842 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:44 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:44.842 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:44 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/597230583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:35:45.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:45.025+0000 7fe96f2b3640 1 -- 172.21.15.43:0/2719354025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9681a2c80 con 0x7fe968100e70 2024-04-21T08:35:45.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:45.025+0000 7fe9667fc640 1 -- 172.21.15.43:0/2719354025 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fe9600bee00 con 0x7fe968100e70 2024-04-21T08:35:45.025 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:35:45.025 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-21T08:35:45.025 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 1 2024-04-21T08:35:45.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:45.027+0000 7fe96f2b3640 1 -- 172.21.15.43:0/2719354025 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe9500a3c30 msgr2=0x7fe9500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:45.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:45.027+0000 7fe96f2b3640 1 --2- 172.21.15.43:0/2719354025 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe9500a3c30 0x7fe9500a60f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0058c0 tx=0x7fe95c005850 comp rx=0 tx=0).stop 2024-04-21T08:35:45.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:45.027+0000 7fe96f2b3640 1 -- 172.21.15.43:0/2719354025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe968100e70 msgr2=0x7fe968104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:45.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:45.027+0000 7fe96f2b3640 1 --2- 172.21.15.43:0/2719354025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe968100e70 0x7fe968104df0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe960005e30 tx=0x7fe960034830 comp rx=0 tx=0).stop 2024-04-21T08:35:45.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:45.027+0000 7fe96f2b3640 1 -- 172.21.15.43:0/2719354025 shutdown_connections 2024-04-21T08:35:45.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:45.027+0000 7fe96f2b3640 1 --2- 172.21.15.43:0/2719354025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe968102980 0x7fe968103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:45.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:45.027+0000 7fe96f2b3640 1 --2- 172.21.15.43:0/2719354025 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe9500a3c30 0x7fe9500a60f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:45.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:45.027+0000 7fe96f2b3640 1 --2- 172.21.15.43:0/2719354025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe968101fc0 0x7fe968103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:45.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:45.027+0000 7fe96f2b3640 1 --2- 172.21.15.43:0/2719354025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe968100e70 0x7fe968104df0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:45.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:45.027+0000 7fe96f2b3640 1 -- 172.21.15.43:0/2719354025 >> 172.21.15.43:0/2719354025 conn(0x7fe9680fc3e0 msgr2=0x7fe9680fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:45.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:45.027+0000 7fe96f2b3640 1 -- 172.21.15.43:0/2719354025 shutdown_connections 2024-04-21T08:35:45.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:45.027+0000 7fe96f2b3640 1 -- 172.21.15.43:0/2719354025 wait complete. 2024-04-21T08:35:45.043 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:44 smithi112 ceph-mon[31085]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:35:45.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:44 smithi112 ceph-mon[31085]: Saving service mds.all spec with placement smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l;count:12 2024-04-21T08:35:45.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:44 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:45.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:44 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:45.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:44 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:45.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:44 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:45.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:44 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:45.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:44 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:45.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:44 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/597230583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:35:45.819 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:45 smithi175 ceph-mon[31920]: Deploying daemon mds.l on smithi175 2024-04-21T08:35:45.819 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:45 smithi175 ceph-mon[31920]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:45.819 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:45 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/82515649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:35:45.819 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:45 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2719354025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:46.043 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:45 smithi112 ceph-mon[31085]: Deploying daemon mds.l on smithi175 2024-04-21T08:35:46.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:45 smithi112 ceph-mon[31085]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:46.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:45 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/82515649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:35:46.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:45 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2719354025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:46.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:45 smithi043 ceph-mon[27978]: Deploying daemon mds.l on smithi175 2024-04-21T08:35:46.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:45 smithi043 ceph-mon[27978]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:46.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:45 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/82515649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:35:46.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:45 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2719354025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:46.209 INFO:journalctl@ceph.mds.l.smithi175.stdout:Apr 21 08:35:45 smithi175 systemd[1]: Starting Ceph mds.l for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:35:46.464 INFO:journalctl@ceph.mds.l.smithi175.stdout:Apr 21 08:35:46 smithi175 podman[61517]: 2024-04-21 08:35:46.114069623 +0000 UTC m=+0.017575660 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:35:46.464 INFO:journalctl@ceph.mds.l.smithi175.stdout:Apr 21 08:35:46 smithi175 podman[61517]: 2024-04-21 08:35:46.23219777 +0000 UTC m=+0.135703797 container create 7b28e9f656202ef7d3de2ed5832054f59c2ad50578f82b13e6f5eb4a8ad21bfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-l, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-21T08:35:46.964 INFO:journalctl@ceph.mds.l.smithi175.stdout:Apr 21 08:35:46 smithi175 podman[61517]: 2024-04-21 08:35:46.476673738 +0000 UTC m=+0.380179762 container init 7b28e9f656202ef7d3de2ed5832054f59c2ad50578f82b13e6f5eb4a8ad21bfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-l, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-21T08:35:46.964 INFO:journalctl@ceph.mds.l.smithi175.stdout:Apr 21 08:35:46 smithi175 podman[61517]: 2024-04-21 08:35:46.479265937 +0000 UTC m=+0.382771963 container start 7b28e9f656202ef7d3de2ed5832054f59c2ad50578f82b13e6f5eb4a8ad21bfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-l, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-21T08:35:46.964 INFO:journalctl@ceph.mds.l.smithi175.stdout:Apr 21 08:35:46 smithi175 ceph-mds[61536]: -- 172.21.15.175:0/2434201358 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558d07e05a40 con 0x558d07dd8c00 2024-04-21T08:35:46.964 INFO:journalctl@ceph.mds.l.smithi175.stdout:Apr 21 08:35:46 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-l[61532]: starting mds.l at 2024-04-21T08:35:46.964 INFO:journalctl@ceph.mds.l.smithi175.stdout:Apr 21 08:35:46 smithi175 bash[61517]: 7b28e9f656202ef7d3de2ed5832054f59c2ad50578f82b13e6f5eb4a8ad21bfb 2024-04-21T08:35:46.964 INFO:journalctl@ceph.mds.l.smithi175.stdout:Apr 21 08:35:46 smithi175 systemd[1]: Started Ceph mds.l for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:35:47.039 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:35:47.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.154+0000 7f26ce4a0640 1 -- 172.21.15.43:0/1792674443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c8100890 msgr2=0x7f26c8100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:47.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.154+0000 7f26ce4a0640 1 --2- 172.21.15.43:0/1792674443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c8100890 0x7f26c8100cb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f26b8009900 tx=0x7f26b802bc30 comp rx=0 tx=0).stop 2024-04-21T08:35:47.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.154+0000 7f26ce4a0640 1 -- 172.21.15.43:0/1792674443 shutdown_connections 2024-04-21T08:35:47.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.154+0000 7f26ce4a0640 1 --2- 172.21.15.43:0/1792674443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26c8101b90 0x7f26c8102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:47.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.154+0000 7f26ce4a0640 1 --2- 172.21.15.43:0/1792674443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26c81011f0 0x7f26c8101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:47.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.154+0000 7f26ce4a0640 1 --2- 172.21.15.43:0/1792674443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c8100890 0x7f26c8100cb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:47.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.154+0000 7f26ce4a0640 1 -- 172.21.15.43:0/1792674443 >> 172.21.15.43:0/1792674443 conn(0x7f26c80fc470 msgr2=0x7f26c80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:47.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.154+0000 7f26ce4a0640 1 -- 172.21.15.43:0/1792674443 shutdown_connections 2024-04-21T08:35:47.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.154+0000 7f26ce4a0640 1 -- 172.21.15.43:0/1792674443 wait complete. 2024-04-21T08:35:47.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.155+0000 7f26ce4a0640 1 Processor -- start 2024-04-21T08:35:47.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.155+0000 7f26ce4a0640 1 -- start start 2024-04-21T08:35:47.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.155+0000 7f26ce4a0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c8100890 0x7f26c819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:47.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.155+0000 7f26ce4a0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26c81011f0 0x7f26c819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:47.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.155+0000 7f26ce4a0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26c8101b90 0x7f26c81a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:47.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.155+0000 7f26ce4a0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f26c810fd30 con 0x7f26c8100890 2024-04-21T08:35:47.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.155+0000 7f26ce4a0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f26c810fbb0 con 0x7f26c8101b90 2024-04-21T08:35:47.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.155+0000 7f26ce4a0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26c810feb0 con 0x7f26c81011f0 2024-04-21T08:35:47.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.155+0000 7f26c7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c8100890 0x7f26c819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:47.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.155+0000 7f26c7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c8100890 0x7f26c819a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47406/0 (socket says 172.21.15.43:47406) 2024-04-21T08:35:47.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.156+0000 7f26c7fff640 1 -- 172.21.15.43:0/630004764 learned_addr learned my addr 172.21.15.43:0/630004764 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:35:47.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.156+0000 7f26c77fe640 1 --2- 172.21.15.43:0/630004764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26c81011f0 0x7f26c819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:47.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.156+0000 7f26cca3d640 1 --2- 172.21.15.43:0/630004764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26c8101b90 0x7f26c81a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:47.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.156+0000 7f26c7fff640 1 -- 172.21.15.43:0/630004764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26c81011f0 msgr2=0x7f26c819a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:47.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.156+0000 7f26c7fff640 1 --2- 172.21.15.43:0/630004764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26c81011f0 0x7f26c819a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:47.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.156+0000 7f26c7fff640 1 -- 172.21.15.43:0/630004764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26c8101b90 msgr2=0x7f26c81a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:47.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.156+0000 7f26c7fff640 1 --2- 172.21.15.43:0/630004764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26c8101b90 0x7f26c81a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:47.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.156+0000 7f26c7fff640 1 -- 172.21.15.43:0/630004764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c81a1f70 con 0x7f26c8100890 2024-04-21T08:35:47.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.156+0000 7f26c7fff640 1 --2- 172.21.15.43:0/630004764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c8100890 0x7f26c819a350 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f26b8000c00 tx=0x7f26b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:47.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.156+0000 7f26c57fa640 1 -- 172.21.15.43:0/630004764 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8043020 con 0x7f26c8100890 2024-04-21T08:35:47.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.156+0000 7f26ce4a0640 1 -- 172.21.15.43:0/630004764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c81a2200 con 0x7f26c8100890 2024-04-21T08:35:47.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.156+0000 7f26c57fa640 1 -- 172.21.15.43:0/630004764 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26b80054c0 con 0x7f26c8100890 2024-04-21T08:35:47.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.156+0000 7f26c57fa640 1 -- 172.21.15.43:0/630004764 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8003d40 con 0x7f26c8100890 2024-04-21T08:35:47.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.156+0000 7f26ce4a0640 1 -- 172.21.15.43:0/630004764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c81a27a0 con 0x7f26c8100890 2024-04-21T08:35:47.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.157+0000 7f26ce4a0640 1 -- 172.21.15.43:0/630004764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c810b770 con 0x7f26c8100890 2024-04-21T08:35:47.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.159+0000 7f26c57fa640 1 -- 172.21.15.43:0/630004764 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f26b8046020 con 0x7f26c8100890 2024-04-21T08:35:47.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.159+0000 7f26c57fa640 1 --2- 172.21.15.43:0/630004764 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f26b00a3bb0 0x7f26b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:47.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.159+0000 7f26c77fe640 1 --2- 172.21.15.43:0/630004764 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f26b00a3bb0 0x7f26b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:47.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.159+0000 7f26c57fa640 1 -- 172.21.15.43:0/630004764 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f26b80f2710 con 0x7f26c8100890 2024-04-21T08:35:47.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.160+0000 7f26c77fe640 1 --2- 172.21.15.43:0/630004764 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f26b00a3bb0 0x7f26b00a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f26c819b860 tx=0x7f26bc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:47.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.163+0000 7f26c57fa640 1 -- 172.21.15.43:0/630004764 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f26b804d020 con 0x7f26c8100890 2024-04-21T08:35:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.388+0000 7f26ce4a0640 1 -- 172.21.15.43:0/630004764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26c802d5c0 con 0x7f26c8100890 2024-04-21T08:35:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.388+0000 7f26c57fa640 1 -- 172.21.15.43:0/630004764 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7f26b80bf590 con 0x7f26c8100890 2024-04-21T08:35:47.387 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:35:47.388 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-21T08:35:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 2 2024-04-21T08:35:47.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.390+0000 7f26ce4a0640 1 -- 172.21.15.43:0/630004764 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f26b00a3bb0 msgr2=0x7f26b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:47.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.390+0000 7f26ce4a0640 1 --2- 172.21.15.43:0/630004764 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f26b00a3bb0 0x7f26b00a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f26c819b860 tx=0x7f26bc00a2d0 comp rx=0 tx=0).stop 2024-04-21T08:35:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.390+0000 7f26ce4a0640 1 -- 172.21.15.43:0/630004764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c8100890 msgr2=0x7f26c819a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.390+0000 7f26ce4a0640 1 --2- 172.21.15.43:0/630004764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c8100890 0x7f26c819a350 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f26b8000c00 tx=0x7f26b80042f0 comp rx=0 tx=0).stop 2024-04-21T08:35:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.390+0000 7f26ce4a0640 1 -- 172.21.15.43:0/630004764 shutdown_connections 2024-04-21T08:35:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.390+0000 7f26ce4a0640 1 --2- 172.21.15.43:0/630004764 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f26b00a3bb0 0x7f26b00a6070 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.390+0000 7f26ce4a0640 1 --2- 172.21.15.43:0/630004764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26c8101b90 0x7f26c81a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.390+0000 7f26ce4a0640 1 --2- 172.21.15.43:0/630004764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26c81011f0 0x7f26c819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.390+0000 7f26ce4a0640 1 --2- 172.21.15.43:0/630004764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c8100890 0x7f26c819a350 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.390+0000 7f26ce4a0640 1 -- 172.21.15.43:0/630004764 >> 172.21.15.43:0/630004764 conn(0x7f26c80fc470 msgr2=0x7f26c80fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.390+0000 7f26ce4a0640 1 -- 172.21.15.43:0/630004764 shutdown_connections 2024-04-21T08:35:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:47.390+0000 7f26ce4a0640 1 -- 172.21.15.43:0/630004764 wait complete. 2024-04-21T08:35:47.568 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:47 smithi112 ceph-mon[31085]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:47.568 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:47 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] up:boot 2024-04-21T08:35:47.568 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:47 smithi112 ceph-mon[31085]: fsmap 1 up:standby 2024-04-21T08:35:47.568 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:47 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:35:47.568 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:47 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:47.568 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:47 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:47.568 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:47 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:47.568 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:47 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:47.569 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:47 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:47.569 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:47 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:47.569 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:47 smithi112 ceph-mon[31085]: Deploying daemon mds.k on smithi112 2024-04-21T08:35:47.569 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:47 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/630004764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:47.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:47 smithi043 ceph-mon[27978]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:47.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:47 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] up:boot 2024-04-21T08:35:47.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:47 smithi043 ceph-mon[27978]: fsmap 1 up:standby 2024-04-21T08:35:47.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:47 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:35:47.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:47 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:47.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:47 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:47.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:47 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:47.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:47 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:47.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:47 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:47.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:47 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:47.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:47 smithi043 ceph-mon[27978]: Deploying daemon mds.k on smithi112 2024-04-21T08:35:47.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:47 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/630004764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:47.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:47 smithi175 ceph-mon[31920]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:47.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:47 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] up:boot 2024-04-21T08:35:47.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:47 smithi175 ceph-mon[31920]: fsmap 1 up:standby 2024-04-21T08:35:47.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:47 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:35:47.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:47 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:47.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:47 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:47.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:47 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:47.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:47 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:47.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:47 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:47.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:47 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:47.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:47 smithi175 ceph-mon[31920]: Deploying daemon mds.k on smithi112 2024-04-21T08:35:47.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:47 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/630004764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:49.294 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 21 08:35:49 smithi112 systemd[1]: Starting Ceph mds.k for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:35:49.401 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:35:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.518+0000 7f5028eba640 1 -- 172.21.15.43:0/1690258159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5024101fa0 msgr2=0x7f50241023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.518+0000 7f5028eba640 1 --2- 172.21.15.43:0/1690258159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5024101fa0 0x7f50241023c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5014009ca0 tx=0x7f501402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:35:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.518+0000 7f5028eba640 1 -- 172.21.15.43:0/1690258159 shutdown_connections 2024-04-21T08:35:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.518+0000 7f5028eba640 1 --2- 172.21.15.43:0/1690258159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5024103b50 0x7f502410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.518+0000 7f5028eba640 1 --2- 172.21.15.43:0/1690258159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5024103190 0x7f5024103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.518+0000 7f5028eba640 1 --2- 172.21.15.43:0/1690258159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5024101fa0 0x7f50241023c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.518+0000 7f5028eba640 1 -- 172.21.15.43:0/1690258159 >> 172.21.15.43:0/1690258159 conn(0x7f50240fd530 msgr2=0x7f50240ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.518+0000 7f5028eba640 1 -- 172.21.15.43:0/1690258159 shutdown_connections 2024-04-21T08:35:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.518+0000 7f5028eba640 1 -- 172.21.15.43:0/1690258159 wait complete. 2024-04-21T08:35:49.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.518+0000 7f5028eba640 1 Processor -- start 2024-04-21T08:35:49.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.518+0000 7f5028eba640 1 -- start start 2024-04-21T08:35:49.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.519+0000 7f5028eba640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5024101fa0 0x7f50241981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:49.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.519+0000 7f5028eba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5024103190 0x7f5024198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:49.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.519+0000 7f5028eba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5024103b50 0x7f502419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:49.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.519+0000 7f5028eba640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f502410daa0 con 0x7f5024103190 2024-04-21T08:35:49.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.519+0000 7f5028eba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f502410d920 con 0x7f5024103b50 2024-04-21T08:35:49.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.519+0000 7f5028eba640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f502410dc20 con 0x7f5024101fa0 2024-04-21T08:35:49.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.519+0000 7f5021d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5024103190 0x7f5024198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:49.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.519+0000 7f5021d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5024103190 0x7f5024198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47420/0 (socket says 172.21.15.43:47420) 2024-04-21T08:35:49.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.519+0000 7f5021d9b640 1 -- 172.21.15.43:0/1100199162 learned_addr learned my addr 172.21.15.43:0/1100199162 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:35:49.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.519+0000 7f5022d9d640 1 --2- 172.21.15.43:0/1100199162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5024103b50 0x7f502419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:49.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.519+0000 7f502259c640 1 --2- 172.21.15.43:0/1100199162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5024101fa0 0x7f50241981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:49.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.519+0000 7f5021d9b640 1 -- 172.21.15.43:0/1100199162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5024101fa0 msgr2=0x7f50241981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:49.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.519+0000 7f5021d9b640 1 --2- 172.21.15.43:0/1100199162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5024101fa0 0x7f50241981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:49.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.519+0000 7f5021d9b640 1 -- 172.21.15.43:0/1100199162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5024103b50 msgr2=0x7f502419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:49.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.519+0000 7f5021d9b640 1 --2- 172.21.15.43:0/1100199162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5024103b50 0x7f502419f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:49.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.519+0000 7f5021d9b640 1 -- 172.21.15.43:0/1100199162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502419fde0 con 0x7f5024103190 2024-04-21T08:35:49.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.519+0000 7f502259c640 1 --2- 172.21.15.43:0/1100199162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5024101fa0 0x7f50241981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:35:49.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.519+0000 7f5021d9b640 1 --2- 172.21.15.43:0/1100199162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5024103190 0x7f5024198700 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f500c00e9f0 tx=0x7f500c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:49.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.519+0000 7f501b7fe640 1 -- 172.21.15.43:0/1100199162 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c00cd80 con 0x7f5024103190 2024-04-21T08:35:49.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.519+0000 7f501b7fe640 1 -- 172.21.15.43:0/1100199162 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f500c0108e0 con 0x7f5024103190 2024-04-21T08:35:49.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.519+0000 7f5028eba640 1 -- 172.21.15.43:0/1100199162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50241a00d0 con 0x7f5024103190 2024-04-21T08:35:49.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.520+0000 7f501b7fe640 1 -- 172.21.15.43:0/1100199162 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c018980 con 0x7f5024103190 2024-04-21T08:35:49.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.520+0000 7f5028eba640 1 -- 172.21.15.43:0/1100199162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50241a0670 con 0x7f5024103190 2024-04-21T08:35:49.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.521+0000 7f5028eba640 1 -- 172.21.15.43:0/1100199162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5024109390 con 0x7f5024103190 2024-04-21T08:35:49.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.522+0000 7f501b7fe640 1 -- 172.21.15.43:0/1100199162 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f500c010430 con 0x7f5024103190 2024-04-21T08:35:49.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.522+0000 7f501b7fe640 1 --2- 172.21.15.43:0/1100199162 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f50100a3c30 0x7f50100a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:49.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.523+0000 7f502259c640 1 --2- 172.21.15.43:0/1100199162 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f50100a3c30 0x7f50100a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:49.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.523+0000 7f501b7fe640 1 -- 172.21.15.43:0/1100199162 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f500c0d0370 con 0x7f5024103190 2024-04-21T08:35:49.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.523+0000 7f502259c640 1 --2- 172.21.15.43:0/1100199162 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f50100a3c30 0x7f50100a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5014037040 tx=0x7f5014005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:49.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.527+0000 7f501b7fe640 1 -- 172.21.15.43:0/1100199162 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f500c09d150 con 0x7f5024103190 2024-04-21T08:35:49.689 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:49 smithi112 ceph-mon[31085]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:49.689 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 21 08:35:49 smithi112 podman[60624]: 2024-04-21 08:35:49.337779525 +0000 UTC m=+0.017831150 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:35:49.689 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 21 08:35:49 smithi112 podman[60624]: 2024-04-21 08:35:49.454516352 +0000 UTC m=+0.134567966 container create 45235075818ce263e8c0d652558f4c40fe1560f798a1c095675557bdbdd2bba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.build-date=20240416) 2024-04-21T08:35:49.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.753+0000 7f5028eba640 1 -- 172.21.15.43:0/1100199162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5024062b50 con 0x7f5024103190 2024-04-21T08:35:49.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.754+0000 7f501b7fe640 1 -- 172.21.15.43:0/1100199162 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7f500c09ca50 con 0x7f5024103190 2024-04-21T08:35:49.753 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:35:49.753 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-21T08:35:49.753 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 2 2024-04-21T08:35:49.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.756+0000 7f5028eba640 1 -- 172.21.15.43:0/1100199162 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f50100a3c30 msgr2=0x7f50100a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:49.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.756+0000 7f5028eba640 1 --2- 172.21.15.43:0/1100199162 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f50100a3c30 0x7f50100a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5014037040 tx=0x7f5014005ef0 comp rx=0 tx=0).stop 2024-04-21T08:35:49.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.756+0000 7f5028eba640 1 -- 172.21.15.43:0/1100199162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5024103190 msgr2=0x7f5024198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:49.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.756+0000 7f5028eba640 1 --2- 172.21.15.43:0/1100199162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5024103190 0x7f5024198700 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f500c00e9f0 tx=0x7f500c00eeb0 comp rx=0 tx=0).stop 2024-04-21T08:35:49.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.756+0000 7f5028eba640 1 -- 172.21.15.43:0/1100199162 shutdown_connections 2024-04-21T08:35:49.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.756+0000 7f5028eba640 1 --2- 172.21.15.43:0/1100199162 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f50100a3c30 0x7f50100a60f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:49.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.756+0000 7f5028eba640 1 --2- 172.21.15.43:0/1100199162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5024103b50 0x7f502419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:49.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.756+0000 7f5028eba640 1 --2- 172.21.15.43:0/1100199162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5024103190 0x7f5024198700 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:49.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.756+0000 7f5028eba640 1 --2- 172.21.15.43:0/1100199162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5024101fa0 0x7f50241981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:49.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.756+0000 7f5028eba640 1 -- 172.21.15.43:0/1100199162 >> 172.21.15.43:0/1100199162 conn(0x7f50240fd530 msgr2=0x7f5024106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:49.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.756+0000 7f5028eba640 1 -- 172.21.15.43:0/1100199162 shutdown_connections 2024-04-21T08:35:49.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:49.756+0000 7f5028eba640 1 -- 172.21.15.43:0/1100199162 wait complete. 2024-04-21T08:35:49.765 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:49 smithi043 ceph-mon[27978]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:49.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:49 smithi175 ceph-mon[31920]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:50.044 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 21 08:35:49 smithi112 podman[60624]: 2024-04-21 08:35:49.690487193 +0000 UTC m=+0.370538811 container init 45235075818ce263e8c0d652558f4c40fe1560f798a1c095675557bdbdd2bba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-21T08:35:50.044 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 21 08:35:49 smithi112 podman[60624]: 2024-04-21 08:35:49.693169974 +0000 UTC m=+0.373221598 container start 45235075818ce263e8c0d652558f4c40fe1560f798a1c095675557bdbdd2bba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:35:50.044 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 21 08:35:49 smithi112 ceph-mds[60643]: -- 172.21.15.112:0/649936411 <== mon.2 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cc38c79a40 con 0x55cc38c4d000 2024-04-21T08:35:50.044 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 21 08:35:49 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-k[60639]: starting mds.k at 2024-04-21T08:35:50.044 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 21 08:35:49 smithi112 bash[60624]: 45235075818ce263e8c0d652558f4c40fe1560f798a1c095675557bdbdd2bba1 2024-04-21T08:35:50.044 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 21 08:35:49 smithi112 systemd[1]: Started Ceph mds.k for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:35:50.791 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:50 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1100199162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:50.791 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:50 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:50.791 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:50 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:50.792 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:50 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:50.792 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:50 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:50.792 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:50 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:50.792 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:50 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:50.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:50 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1100199162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:50.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:50 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:50.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:50 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:50.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:50 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:50.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:50 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:50.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:50 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:50.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:50 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:50.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:50 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1100199162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:50.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:50 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:50.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:50 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:50.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:50 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:50.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:50 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:50.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:50 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:50.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:50 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:51.767 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:35:51.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:51 smithi112 ceph-mon[31085]: Deploying daemon mds.j on smithi043 2024-04-21T08:35:51.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:51 smithi112 ceph-mon[31085]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:51.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:51 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] up:boot 2024-04-21T08:35:51.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:51 smithi112 ceph-mon[31085]: fsmap 2 up:standby 2024-04-21T08:35:51.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:51 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:35:51.794 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:51 smithi043 ceph-mon[27978]: Deploying daemon mds.j on smithi043 2024-04-21T08:35:51.794 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:51 smithi043 ceph-mon[27978]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:51.795 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:51 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] up:boot 2024-04-21T08:35:51.795 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:51 smithi043 ceph-mon[27978]: fsmap 2 up:standby 2024-04-21T08:35:51.795 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:51 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:35:51.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:51 smithi175 ceph-mon[31920]: Deploying daemon mds.j on smithi043 2024-04-21T08:35:51.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:51 smithi175 ceph-mon[31920]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:51.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:51 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] up:boot 2024-04-21T08:35:51.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:51 smithi175 ceph-mon[31920]: fsmap 2 up:standby 2024-04-21T08:35:51.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:51 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:35:52.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.027+0000 7f0e1c585640 1 -- 172.21.15.43:0/1782472394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e14101fc0 msgr2=0x7f0e14102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:52.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.027+0000 7f0e1c585640 1 --2- 172.21.15.43:0/1782472394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e14101fc0 0x7f0e14102440 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0e10009900 tx=0x7f0e1002bc30 comp rx=0 tx=0).stop 2024-04-21T08:35:52.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.027+0000 7f0e1c585640 1 -- 172.21.15.43:0/1782472394 shutdown_connections 2024-04-21T08:35:52.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.027+0000 7f0e1c585640 1 --2- 172.21.15.43:0/1782472394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e14102980 0x7f0e141092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:52.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.027+0000 7f0e1c585640 1 --2- 172.21.15.43:0/1782472394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e14101fc0 0x7f0e14102440 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:52.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.027+0000 7f0e1c585640 1 --2- 172.21.15.43:0/1782472394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e14100e70 0x7f0e14101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:52.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.027+0000 7f0e1c585640 1 -- 172.21.15.43:0/1782472394 >> 172.21.15.43:0/1782472394 conn(0x7f0e140fc3e0 msgr2=0x7f0e140fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:52.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.027+0000 7f0e1c585640 1 -- 172.21.15.43:0/1782472394 shutdown_connections 2024-04-21T08:35:52.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.027+0000 7f0e1c585640 1 -- 172.21.15.43:0/1782472394 wait complete. 2024-04-21T08:35:52.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.028+0000 7f0e1c585640 1 Processor -- start 2024-04-21T08:35:52.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.028+0000 7f0e1c585640 1 -- start start 2024-04-21T08:35:52.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.028+0000 7f0e1c585640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e14100e70 0x7f0e141970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:52.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.028+0000 7f0e1c585640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e14101fc0 0x7f0e14197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:52.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.028+0000 7f0e1c585640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e14102980 0x7f0e1419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:52.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.028+0000 7f0e1c585640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1410c9c0 con 0x7f0e14100e70 2024-04-21T08:35:52.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.028+0000 7f0e1c585640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1410c840 con 0x7f0e14101fc0 2024-04-21T08:35:52.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.028+0000 7f0e1c585640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1410cb40 con 0x7f0e14102980 2024-04-21T08:35:52.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.028+0000 7f0e1a321640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e14100e70 0x7f0e141970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.028+0000 7f0e1a321640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e14100e70 0x7f0e141970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47442/0 (socket says 172.21.15.43:47442) 2024-04-21T08:35:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.028+0000 7f0e1a321640 1 -- 172.21.15.43:0/34507893 learned_addr learned my addr 172.21.15.43:0/34507893 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:35:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.028+0000 7f0e19b20640 1 --2- 172.21.15.43:0/34507893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e14101fc0 0x7f0e14197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.028+0000 7f0e1ab22640 1 --2- 172.21.15.43:0/34507893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e14102980 0x7f0e1419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.028+0000 7f0e1a321640 1 -- 172.21.15.43:0/34507893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e14102980 msgr2=0x7f0e1419e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.028+0000 7f0e1a321640 1 --2- 172.21.15.43:0/34507893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e14102980 0x7f0e1419e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.028+0000 7f0e1a321640 1 -- 172.21.15.43:0/34507893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e14101fc0 msgr2=0x7f0e14197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.028+0000 7f0e1a321640 1 --2- 172.21.15.43:0/34507893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e14101fc0 0x7f0e14197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.028+0000 7f0e1a321640 1 -- 172.21.15.43:0/34507893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1419ece0 con 0x7f0e14100e70 2024-04-21T08:35:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.029+0000 7f0e1a321640 1 --2- 172.21.15.43:0/34507893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e14100e70 0x7f0e141970c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400d770 tx=0x7f0e0400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.029+0000 7f0e0b7fe640 1 -- 172.21.15.43:0/34507893 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04017070 con 0x7f0e14100e70 2024-04-21T08:35:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.029+0000 7f0e1c585640 1 -- 172.21.15.43:0/34507893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1419efd0 con 0x7f0e14100e70 2024-04-21T08:35:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.029+0000 7f0e1c585640 1 -- 172.21.15.43:0/34507893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e1419f430 con 0x7f0e14100e70 2024-04-21T08:35:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.029+0000 7f0e0b7fe640 1 -- 172.21.15.43:0/34507893 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e040054d0 con 0x7f0e14100e70 2024-04-21T08:35:52.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.029+0000 7f0e0b7fe640 1 -- 172.21.15.43:0/34507893 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04004c30 con 0x7f0e14100e70 2024-04-21T08:35:52.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.030+0000 7f0e1c585640 1 -- 172.21.15.43:0/34507893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df80052c0 con 0x7f0e14100e70 2024-04-21T08:35:52.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.032+0000 7f0e0b7fe640 1 -- 172.21.15.43:0/34507893 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0e04009c30 con 0x7f0e14100e70 2024-04-21T08:35:52.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.033+0000 7f0e0b7fe640 1 --2- 172.21.15.43:0/34507893 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0e000a3bb0 0x7f0e000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:52.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.033+0000 7f0e19b20640 1 --2- 172.21.15.43:0/34507893 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0e000a3bb0 0x7f0e000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:52.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.033+0000 7f0e0b7fe640 1 -- 172.21.15.43:0/34507893 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0e040cfe00 con 0x7f0e14100e70 2024-04-21T08:35:52.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.033+0000 7f0e19b20640 1 --2- 172.21.15.43:0/34507893 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0e000a3bb0 0x7f0e000a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0e10005ab0 tx=0x7f0e10005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:52.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.036+0000 7f0e0b7fe640 1 -- 172.21.15.43:0/34507893 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e0409cbe0 con 0x7f0e14100e70 2024-04-21T08:35:52.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.269+0000 7f0e1c585640 1 -- 172.21.15.43:0/34507893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0df80055b0 con 0x7f0e14100e70 2024-04-21T08:35:52.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.269+0000 7f0e0b7fe640 1 -- 172.21.15.43:0/34507893 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f0e0409c4e0 con 0x7f0e14100e70 2024-04-21T08:35:52.269 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:35:52.269 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-21T08:35:52.269 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 3 2024-04-21T08:35:52.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.272+0000 7f0e1c585640 1 -- 172.21.15.43:0/34507893 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0e000a3bb0 msgr2=0x7f0e000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:52.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.272+0000 7f0e1c585640 1 --2- 172.21.15.43:0/34507893 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0e000a3bb0 0x7f0e000a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0e10005ab0 tx=0x7f0e10005fd0 comp rx=0 tx=0).stop 2024-04-21T08:35:52.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.272+0000 7f0e1c585640 1 -- 172.21.15.43:0/34507893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e14100e70 msgr2=0x7f0e141970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:52.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.272+0000 7f0e1c585640 1 --2- 172.21.15.43:0/34507893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e14100e70 0x7f0e141970c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400d770 tx=0x7f0e0400dc30 comp rx=0 tx=0).stop 2024-04-21T08:35:52.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.272+0000 7f0e1c585640 1 -- 172.21.15.43:0/34507893 shutdown_connections 2024-04-21T08:35:52.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.272+0000 7f0e1c585640 1 --2- 172.21.15.43:0/34507893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e14102980 0x7f0e1419e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:52.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.272+0000 7f0e1c585640 1 --2- 172.21.15.43:0/34507893 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0e000a3bb0 0x7f0e000a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:52.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.272+0000 7f0e1c585640 1 --2- 172.21.15.43:0/34507893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e14101fc0 0x7f0e14197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:52.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.272+0000 7f0e1c585640 1 --2- 172.21.15.43:0/34507893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e14100e70 0x7f0e141970c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:52.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.272+0000 7f0e1c585640 1 -- 172.21.15.43:0/34507893 >> 172.21.15.43:0/34507893 conn(0x7f0e140fc3e0 msgr2=0x7f0e141053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:52.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.272+0000 7f0e1c585640 1 -- 172.21.15.43:0/34507893 shutdown_connections 2024-04-21T08:35:52.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:52.272+0000 7f0e1c585640 1 -- 172.21.15.43:0/34507893 wait complete. 2024-04-21T08:35:52.416 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 21 08:35:52 smithi043 podman[66175]: 2024-04-21 08:35:52.281027798 +0000 UTC m=+0.019776049 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:35:52.416 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 21 08:35:52 smithi043 podman[66175]: 2024-04-21 08:35:52.417627691 +0000 UTC m=+0.156375973 container create 3d09652be3c85cc996ca41c1f5a68ccace2a5c4d3e575f4950634fb3dfc7f6fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-21T08:35:52.685 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:52 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/34507893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:52.793 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:52 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/34507893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:52.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:52 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/34507893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:53.061 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 21 08:35:52 smithi043 podman[66175]: 2024-04-21 08:35:52.686934148 +0000 UTC m=+0.425682388 container init 3d09652be3c85cc996ca41c1f5a68ccace2a5c4d3e575f4950634fb3dfc7f6fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-21T08:35:53.061 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 21 08:35:52 smithi043 podman[66175]: 2024-04-21 08:35:52.689727386 +0000 UTC m=+0.428475633 container start 3d09652be3c85cc996ca41c1f5a68ccace2a5c4d3e575f4950634fb3dfc7f6fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-21T08:35:53.061 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 21 08:35:52 smithi043 ceph-mds[66197]: -- 172.21.15.43:0/3678754640 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561a621d5a40 con 0x561a621a8800 2024-04-21T08:35:53.062 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 21 08:35:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-j[66193]: starting mds.j at 2024-04-21T08:35:53.062 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 21 08:35:52 smithi043 bash[66175]: 3d09652be3c85cc996ca41c1f5a68ccace2a5c4d3e575f4950634fb3dfc7f6fd 2024-04-21T08:35:53.062 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 21 08:35:52 smithi043 systemd[1]: Started Ceph mds.j for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:35:53.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:53 smithi112 ceph-mon[31085]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:53.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:53 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:53.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:53 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:53.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:53 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:53.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:53 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:53.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:53 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:53.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:53 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:53.797 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:53 smithi043 ceph-mon[27978]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:53.797 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:53 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:53.797 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:53 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:53.797 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:53 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:53.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:53 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:53.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:53 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:53.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:53 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:53.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:53 smithi175 ceph-mon[31920]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:53.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:53 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:53.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:53 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:53.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:53 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:53.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:53 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:53.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:53 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:53.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:53 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:54.284 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:35:54.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.395+0000 7fb1fd474640 1 -- 172.21.15.43:0/3984580276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1f81011a0 msgr2=0x7fb1f8101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:54.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.395+0000 7fb1fd474640 1 --2- 172.21.15.43:0/3984580276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1f81011a0 0x7fb1f8101600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009900 tx=0x7fb1e002bc30 comp rx=0 tx=0).stop 2024-04-21T08:35:54.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.395+0000 7fb1fd474640 1 -- 172.21.15.43:0/3984580276 shutdown_connections 2024-04-21T08:35:54.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.395+0000 7fb1fd474640 1 --2- 172.21.15.43:0/3984580276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f8101b40 0x7fb1f8101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:54.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.395+0000 7fb1fd474640 1 --2- 172.21.15.43:0/3984580276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1f81011a0 0x7fb1f8101600 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:54.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.395+0000 7fb1fd474640 1 --2- 172.21.15.43:0/3984580276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1f8100840 0x7fb1f8100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:54.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.395+0000 7fb1fd474640 1 -- 172.21.15.43:0/3984580276 >> 172.21.15.43:0/3984580276 conn(0x7fb1f80fc3e0 msgr2=0x7fb1f80fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:54.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.395+0000 7fb1fd474640 1 -- 172.21.15.43:0/3984580276 shutdown_connections 2024-04-21T08:35:54.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.395+0000 7fb1fd474640 1 -- 172.21.15.43:0/3984580276 wait complete. 2024-04-21T08:35:54.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.395+0000 7fb1fd474640 1 Processor -- start 2024-04-21T08:35:54.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.395+0000 7fb1fd474640 1 -- start start 2024-04-21T08:35:54.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.396+0000 7fb1fd474640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f8100840 0x7fb1f819a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:54.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.396+0000 7fb1fd474640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1f81011a0 0x7fb1f819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:54.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.396+0000 7fb1fd474640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1f8101b40 0x7fb1f81a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:54.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.396+0000 7fb1fd474640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f810fcc0 con 0x7fb1f8101b40 2024-04-21T08:35:54.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.396+0000 7fb1fd474640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f810fb40 con 0x7fb1f8100840 2024-04-21T08:35:54.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.396+0000 7fb1fd474640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f810fe40 con 0x7fb1f81011a0 2024-04-21T08:35:54.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.396+0000 7fb1f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1f8101b40 0x7fb1f81a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:54.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.396+0000 7fb1f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1f8101b40 0x7fb1f81a1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47476/0 (socket says 172.21.15.43:47476) 2024-04-21T08:35:54.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.396+0000 7fb1f77fe640 1 -- 172.21.15.43:0/2607285339 learned_addr learned my addr 172.21.15.43:0/2607285339 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:35:54.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.396+0000 7fb1f6ffd640 1 --2- 172.21.15.43:0/2607285339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f8100840 0x7fb1f819a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:54.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.396+0000 7fb1f67fc640 1 --2- 172.21.15.43:0/2607285339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1f81011a0 0x7fb1f819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:54.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.396+0000 7fb1f77fe640 1 -- 172.21.15.43:0/2607285339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1f81011a0 msgr2=0x7fb1f819a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:54.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.396+0000 7fb1f77fe640 1 --2- 172.21.15.43:0/2607285339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1f81011a0 0x7fb1f819a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:54.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.396+0000 7fb1f77fe640 1 -- 172.21.15.43:0/2607285339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f8100840 msgr2=0x7fb1f819a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:54.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.396+0000 7fb1f77fe640 1 --2- 172.21.15.43:0/2607285339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f8100840 0x7fb1f819a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:54.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.396+0000 7fb1f77fe640 1 -- 172.21.15.43:0/2607285339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f81a1fd0 con 0x7fb1f8101b40 2024-04-21T08:35:54.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.396+0000 7fb1f77fe640 1 --2- 172.21.15.43:0/2607285339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1f8101b40 0x7fb1f81a1960 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800eaa0 tx=0x7fb1e800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:54.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.396+0000 7fb1effff640 1 -- 172.21.15.43:0/2607285339 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e800cdb0 con 0x7fb1f8101b40 2024-04-21T08:35:54.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.396+0000 7fb1effff640 1 -- 172.21.15.43:0/2607285339 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e8005020 con 0x7fb1f8101b40 2024-04-21T08:35:54.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.396+0000 7fb1fd474640 1 -- 172.21.15.43:0/2607285339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f81a22c0 con 0x7fb1f8101b40 2024-04-21T08:35:54.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.396+0000 7fb1effff640 1 -- 172.21.15.43:0/2607285339 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e80116d0 con 0x7fb1f8101b40 2024-04-21T08:35:54.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.396+0000 7fb1fd474640 1 -- 172.21.15.43:0/2607285339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f81a27c0 con 0x7fb1f8101b40 2024-04-21T08:35:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.398+0000 7fb1fd474640 1 -- 172.21.15.43:0/2607285339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1f810b7e0 con 0x7fb1f8101b40 2024-04-21T08:35:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.398+0000 7fb1effff640 1 -- 172.21.15.43:0/2607285339 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb1e80051d0 con 0x7fb1f8101b40 2024-04-21T08:35:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.399+0000 7fb1effff640 1 --2- 172.21.15.43:0/2607285339 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb1e40a3bb0 0x7fb1e40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.399+0000 7fb1f6ffd640 1 --2- 172.21.15.43:0/2607285339 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb1e40a3bb0 0x7fb1e40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.399+0000 7fb1effff640 1 -- 172.21.15.43:0/2607285339 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb1e80d0160 con 0x7fb1f8101b40 2024-04-21T08:35:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.399+0000 7fb1f6ffd640 1 --2- 172.21.15.43:0/2607285339 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb1e40a3bb0 0x7fb1e40a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0005fd0 tx=0x7fb1f000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.403+0000 7fb1effff640 1 -- 172.21.15.43:0/2607285339 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb1e8016030 con 0x7fb1f8101b40 2024-04-21T08:35:54.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:54 smithi043 ceph-mon[27978]: Deploying daemon mds.d on smithi043 2024-04-21T08:35:54.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:54 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.43:6834/3697998894,v1:172.21.15.43:6835/3697998894] up:boot 2024-04-21T08:35:54.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:54 smithi043 ceph-mon[27978]: fsmap 3 up:standby 2024-04-21T08:35:54.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:54 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T08:35:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.634+0000 7fb1fd474640 1 -- 172.21.15.43:0/2607285339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1f802d7c0 con 0x7fb1f8101b40 2024-04-21T08:35:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.634+0000 7fb1effff640 1 -- 172.21.15.43:0/2607285339 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7fb1e809cfe0 con 0x7fb1f8101b40 2024-04-21T08:35:54.634 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:35:54.634 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-21T08:35:54.634 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 4 2024-04-21T08:35:54.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.636+0000 7fb1fd474640 1 -- 172.21.15.43:0/2607285339 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb1e40a3bb0 msgr2=0x7fb1e40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:54.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.636+0000 7fb1fd474640 1 --2- 172.21.15.43:0/2607285339 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb1e40a3bb0 0x7fb1e40a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0005fd0 tx=0x7fb1f000b390 comp rx=0 tx=0).stop 2024-04-21T08:35:54.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.636+0000 7fb1fd474640 1 -- 172.21.15.43:0/2607285339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1f8101b40 msgr2=0x7fb1f81a1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:54.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.636+0000 7fb1fd474640 1 --2- 172.21.15.43:0/2607285339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1f8101b40 0x7fb1f81a1960 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800eaa0 tx=0x7fb1e800ef60 comp rx=0 tx=0).stop 2024-04-21T08:35:54.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.636+0000 7fb1fd474640 1 -- 172.21.15.43:0/2607285339 shutdown_connections 2024-04-21T08:35:54.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.636+0000 7fb1fd474640 1 --2- 172.21.15.43:0/2607285339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1f8101b40 0x7fb1f81a1960 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:54.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.636+0000 7fb1fd474640 1 --2- 172.21.15.43:0/2607285339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1f81011a0 0x7fb1f819a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:54.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.636+0000 7fb1fd474640 1 --2- 172.21.15.43:0/2607285339 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb1e40a3bb0 0x7fb1e40a6070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:54.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.636+0000 7fb1fd474640 1 --2- 172.21.15.43:0/2607285339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f8100840 0x7fb1f819a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:54.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.636+0000 7fb1fd474640 1 -- 172.21.15.43:0/2607285339 >> 172.21.15.43:0/2607285339 conn(0x7fb1f80fc3e0 msgr2=0x7fb1f80fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:54.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.637+0000 7fb1fd474640 1 -- 172.21.15.43:0/2607285339 shutdown_connections 2024-04-21T08:35:54.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:54.637+0000 7fb1fd474640 1 -- 172.21.15.43:0/2607285339 wait complete. 2024-04-21T08:35:54.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:54 smithi175 ceph-mon[31920]: Deploying daemon mds.d on smithi043 2024-04-21T08:35:54.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:54 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.43:6834/3697998894,v1:172.21.15.43:6835/3697998894] up:boot 2024-04-21T08:35:54.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:54 smithi175 ceph-mon[31920]: fsmap 3 up:standby 2024-04-21T08:35:54.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:54 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T08:35:54.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:54 smithi112 ceph-mon[31085]: Deploying daemon mds.d on smithi043 2024-04-21T08:35:54.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:54 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.43:6834/3697998894,v1:172.21.15.43:6835/3697998894] up:boot 2024-04-21T08:35:54.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:54 smithi112 ceph-mon[31085]: fsmap 3 up:standby 2024-04-21T08:35:54.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:54 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T08:35:55.311 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 21 08:35:55 smithi043 systemd[1]: Starting Ceph mds.d for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:35:55.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:55 smithi043 ceph-mon[27978]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:55.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:55 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2607285339' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:55.646 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 21 08:35:55 smithi043 podman[66526]: 2024-04-21 08:35:55.283177063 +0000 UTC m=+0.017949321 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:35:55.646 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 21 08:35:55 smithi043 podman[66526]: 2024-04-21 08:35:55.39213306 +0000 UTC m=+0.126905285 container create 63a5476ed53054dfa441f882f7b5cc445f9388ed100e17fe5e1542c6e3bb610a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-21T08:35:55.646 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 21 08:35:55 smithi043 podman[66526]: 2024-04-21 08:35:55.620947984 +0000 UTC m=+0.355720222 container init 63a5476ed53054dfa441f882f7b5cc445f9388ed100e17fe5e1542c6e3bb610a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:35:55.646 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 21 08:35:55 smithi043 podman[66526]: 2024-04-21 08:35:55.623858687 +0000 UTC m=+0.358630920 container start 63a5476ed53054dfa441f882f7b5cc445f9388ed100e17fe5e1542c6e3bb610a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-d, RELEASE=wip-lusov-quiescer-a630765, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-21T08:35:55.646 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 21 08:35:55 smithi043 ceph-mds[66545]: -- 172.21.15.43:0/2568347440 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56433e539a40 con 0x56433e50cc00 2024-04-21T08:35:55.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:55 smithi112 ceph-mon[31085]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:55.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:55 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2607285339' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:55.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:55 smithi175 ceph-mon[31920]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:55.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:55 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2607285339' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:56.061 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 21 08:35:55 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-d[66541]: starting mds.d at 2024-04-21T08:35:56.061 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 21 08:35:55 smithi043 bash[66526]: 63a5476ed53054dfa441f882f7b5cc445f9388ed100e17fe5e1542c6e3bb610a 2024-04-21T08:35:56.062 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 21 08:35:55 smithi043 systemd[1]: Started Ceph mds.d for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:35:56.647 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:35:56.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.766+0000 7fdc82870640 1 -- 172.21.15.43:0/3222256468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc7c102ae0 msgr2=0x7fdc7c104ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:56.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.766+0000 7fdc82870640 1 --2- 172.21.15.43:0/3222256468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc7c102ae0 0x7fdc7c104ed0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009900 tx=0x7fdc7002bc30 comp rx=0 tx=0).stop 2024-04-21T08:35:56.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.766+0000 7fdc82870640 1 -- 172.21.15.43:0/3222256468 shutdown_connections 2024-04-21T08:35:56.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.766+0000 7fdc82870640 1 --2- 172.21.15.43:0/3222256468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc7c107d40 0x7fdc7c10a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:56.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.766+0000 7fdc82870640 1 --2- 172.21.15.43:0/3222256468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc7c105410 0x7fdc7c107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:56.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.766+0000 7fdc82870640 1 --2- 172.21.15.43:0/3222256468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc7c102ae0 0x7fdc7c104ed0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.766+0000 7fdc82870640 1 -- 172.21.15.43:0/3222256468 >> 172.21.15.43:0/3222256468 conn(0x7fdc7c0fc470 msgr2=0x7fdc7c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.766+0000 7fdc82870640 1 -- 172.21.15.43:0/3222256468 shutdown_connections 2024-04-21T08:35:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.767+0000 7fdc82870640 1 -- 172.21.15.43:0/3222256468 wait complete. 2024-04-21T08:35:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.767+0000 7fdc82870640 1 Processor -- start 2024-04-21T08:35:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.767+0000 7fdc82870640 1 -- start start 2024-04-21T08:35:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.767+0000 7fdc82870640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc7c102ae0 0x7fdc7c198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.767+0000 7fdc82870640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc7c105410 0x7fdc7c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.767+0000 7fdc82870640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc7c107d40 0x7fdc7c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.767+0000 7fdc82870640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c10db20 con 0x7fdc7c105410 2024-04-21T08:35:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.767+0000 7fdc82870640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c10d9a0 con 0x7fdc7c102ae0 2024-04-21T08:35:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.767+0000 7fdc82870640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c10dca0 con 0x7fdc7c107d40 2024-04-21T08:35:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.768+0000 7fdc7b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc7c105410 0x7fdc7c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.768+0000 7fdc7b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc7c105410 0x7fdc7c198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58020/0 (socket says 172.21.15.43:58020) 2024-04-21T08:35:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.768+0000 7fdc7b7fe640 1 -- 172.21.15.43:0/2500825972 learned_addr learned my addr 172.21.15.43:0/2500825972 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:35:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.768+0000 7fdc7bfff640 1 --2- 172.21.15.43:0/2500825972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc7c102ae0 0x7fdc7c198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.768+0000 7fdc80e0d640 1 --2- 172.21.15.43:0/2500825972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc7c107d40 0x7fdc7c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.768+0000 7fdc7b7fe640 1 -- 172.21.15.43:0/2500825972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc7c107d40 msgr2=0x7fdc7c19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.768+0000 7fdc7b7fe640 1 --2- 172.21.15.43:0/2500825972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc7c107d40 0x7fdc7c19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.768+0000 7fdc7b7fe640 1 -- 172.21.15.43:0/2500825972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc7c102ae0 msgr2=0x7fdc7c198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.768+0000 7fdc7b7fe640 1 --2- 172.21.15.43:0/2500825972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc7c102ae0 0x7fdc7c198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.768+0000 7fdc7b7fe640 1 -- 172.21.15.43:0/2500825972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7c19fe30 con 0x7fdc7c105410 2024-04-21T08:35:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.768+0000 7fdc80e0d640 1 --2- 172.21.15.43:0/2500825972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc7c107d40 0x7fdc7c19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:35:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.768+0000 7fdc7b7fe640 1 --2- 172.21.15.43:0/2500825972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc7c105410 0x7fdc7c198750 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00d740 tx=0x7fdc6c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.768+0000 7fdc797fa640 1 -- 172.21.15.43:0/2500825972 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c017070 con 0x7fdc7c105410 2024-04-21T08:35:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.768+0000 7fdc82870640 1 -- 172.21.15.43:0/2500825972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc7c1a0120 con 0x7fdc7c105410 2024-04-21T08:35:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.768+0000 7fdc797fa640 1 -- 172.21.15.43:0/2500825972 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc6c00de50 con 0x7fdc7c105410 2024-04-21T08:35:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.768+0000 7fdc797fa640 1 -- 172.21.15.43:0/2500825972 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c004ce0 con 0x7fdc7c105410 2024-04-21T08:35:56.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.768+0000 7fdc82870640 1 -- 172.21.15.43:0/2500825972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc7c1a0620 con 0x7fdc7c105410 2024-04-21T08:35:56.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.769+0000 7fdc82870640 1 -- 172.21.15.43:0/2500825972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc7c0fdf60 con 0x7fdc7c105410 2024-04-21T08:35:56.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.770+0000 7fdc797fa640 1 -- 172.21.15.43:0/2500825972 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdc6c004e80 con 0x7fdc7c105410 2024-04-21T08:35:56.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.771+0000 7fdc797fa640 1 --2- 172.21.15.43:0/2500825972 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fdc640a3bb0 0x7fdc640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:56.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.771+0000 7fdc7bfff640 1 --2- 172.21.15.43:0/2500825972 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fdc640a3bb0 0x7fdc640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:56.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.771+0000 7fdc797fa640 1 -- 172.21.15.43:0/2500825972 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fdc6c0cf9a0 con 0x7fdc7c105410 2024-04-21T08:35:56.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.772+0000 7fdc7bfff640 1 --2- 172.21.15.43:0/2500825972 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fdc640a3bb0 0x7fdc640a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009230 tx=0x7fdc70005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:56.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:56.776+0000 7fdc797fa640 1 -- 172.21.15.43:0/2500825972 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdc6c09c780 con 0x7fdc7c105410 2024-04-21T08:35:57.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:57.004+0000 7fdc82870640 1 -- 172.21.15.43:0/2500825972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc7c062b50 con 0x7fdc7c105410 2024-04-21T08:35:57.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:57.004+0000 7fdc797fa640 1 -- 172.21.15.43:0/2500825972 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7fdc6c09c080 con 0x7fdc7c105410 2024-04-21T08:35:57.004 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:35:57.004 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-21T08:35:57.004 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 5 2024-04-21T08:35:57.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:57.007+0000 7fdc82870640 1 -- 172.21.15.43:0/2500825972 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fdc640a3bb0 msgr2=0x7fdc640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:57.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:57.007+0000 7fdc82870640 1 --2- 172.21.15.43:0/2500825972 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fdc640a3bb0 0x7fdc640a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009230 tx=0x7fdc70005fd0 comp rx=0 tx=0).stop 2024-04-21T08:35:57.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:57.007+0000 7fdc82870640 1 -- 172.21.15.43:0/2500825972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc7c105410 msgr2=0x7fdc7c198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:57.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:57.007+0000 7fdc82870640 1 --2- 172.21.15.43:0/2500825972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc7c105410 0x7fdc7c198750 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00d740 tx=0x7fdc6c00dc00 comp rx=0 tx=0).stop 2024-04-21T08:35:57.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:57.007+0000 7fdc82870640 1 -- 172.21.15.43:0/2500825972 shutdown_connections 2024-04-21T08:35:57.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:57.007+0000 7fdc82870640 1 --2- 172.21.15.43:0/2500825972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc7c107d40 0x7fdc7c19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:57.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:57.007+0000 7fdc82870640 1 --2- 172.21.15.43:0/2500825972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc7c105410 0x7fdc7c198750 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:57.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:57.007+0000 7fdc82870640 1 --2- 172.21.15.43:0/2500825972 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fdc640a3bb0 0x7fdc640a6070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:57.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:57.007+0000 7fdc82870640 1 --2- 172.21.15.43:0/2500825972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc7c102ae0 0x7fdc7c198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:57.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:57.007+0000 7fdc82870640 1 -- 172.21.15.43:0/2500825972 >> 172.21.15.43:0/2500825972 conn(0x7fdc7c0fc470 msgr2=0x7fdc7c103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:57.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:57.007+0000 7fdc82870640 1 -- 172.21.15.43:0/2500825972 shutdown_connections 2024-04-21T08:35:57.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:57.007+0000 7fdc82870640 1 -- 172.21.15.43:0/2500825972 wait complete. 2024-04-21T08:35:57.016 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:56 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:57.016 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:56 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:57.017 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:56 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:57.017 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:56 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:57.017 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:56 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:57.017 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:56 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:57.017 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:56 smithi043 ceph-mon[27978]: Deploying daemon mds.e on smithi112 2024-04-21T08:35:57.017 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:56 smithi043 ceph-mon[27978]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:57.017 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:56 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.43:6836/1236755039,v1:172.21.15.43:6837/1236755039] up:boot 2024-04-21T08:35:57.017 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:56 smithi043 ceph-mon[27978]: fsmap 4 up:standby 2024-04-21T08:35:57.017 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:56 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T08:35:57.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:56 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:57.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:56 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:57.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:56 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:57.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:56 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:57.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:56 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:57.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:56 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:57.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:56 smithi175 ceph-mon[31920]: Deploying daemon mds.e on smithi112 2024-04-21T08:35:57.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:56 smithi175 ceph-mon[31920]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:57.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:56 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.43:6836/1236755039,v1:172.21.15.43:6837/1236755039] up:boot 2024-04-21T08:35:57.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:56 smithi175 ceph-mon[31920]: fsmap 4 up:standby 2024-04-21T08:35:57.215 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:56 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T08:35:57.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:56 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:57.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:56 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:57.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:56 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:57.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:56 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:57.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:56 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:57.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:56 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:57.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:56 smithi112 ceph-mon[31085]: Deploying daemon mds.e on smithi112 2024-04-21T08:35:57.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:56 smithi112 ceph-mon[31085]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:57.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:56 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.43:6836/1236755039,v1:172.21.15.43:6837/1236755039] up:boot 2024-04-21T08:35:57.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:56 smithi112 ceph-mon[31085]: fsmap 4 up:standby 2024-04-21T08:35:57.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:56 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T08:35:58.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:57 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2500825972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:58.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:57 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2500825972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:58.295 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 21 08:35:58 smithi112 systemd[1]: Starting Ceph mds.e for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:35:58.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:57 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2500825972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:58.637 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 21 08:35:58 smithi112 podman[60943]: 2024-04-21 08:35:58.243032975 +0000 UTC m=+0.018139343 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:35:58.637 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 21 08:35:58 smithi112 podman[60943]: 2024-04-21 08:35:58.377236242 +0000 UTC m=+0.152342632 container create cc359f26aee18c802f4bb7ad38baa028380957f4ed1050a61d7ecd89338ef7f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:35:58.968 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 21 08:35:58 smithi112 podman[60943]: 2024-04-21 08:35:58.638235446 +0000 UTC m=+0.413341795 container init cc359f26aee18c802f4bb7ad38baa028380957f4ed1050a61d7ecd89338ef7f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-21T08:35:58.968 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 21 08:35:58 smithi112 podman[60943]: 2024-04-21 08:35:58.640832746 +0000 UTC m=+0.415939099 container start cc359f26aee18c802f4bb7ad38baa028380957f4ed1050a61d7ecd89338ef7f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:35:58.968 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 21 08:35:58 smithi112 ceph-mds[60962]: -- 172.21.15.112:0/44209988 <== mon.2 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5643c2417a40 con 0x5643c23ea800 2024-04-21T08:35:58.968 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 21 08:35:58 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-e[60958]: starting mds.e at 2024-04-21T08:35:58.969 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 21 08:35:58 smithi112 bash[60943]: cc359f26aee18c802f4bb7ad38baa028380957f4ed1050a61d7ecd89338ef7f6 2024-04-21T08:35:58.969 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 21 08:35:58 smithi112 systemd[1]: Started Ceph mds.e for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:35:59.019 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:35:59.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.136+0000 7fe2c0b75640 1 -- 172.21.15.43:0/2091308459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2bc101fa0 msgr2=0x7fe2bc1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:59.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.136+0000 7fe2c0b75640 1 --2- 172.21.15.43:0/2091308459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2bc101fa0 0x7fe2bc1023c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009900 tx=0x7fe2ac02bc30 comp rx=0 tx=0).stop 2024-04-21T08:35:59.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.136+0000 7fe2c0b75640 1 -- 172.21.15.43:0/2091308459 shutdown_connections 2024-04-21T08:35:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.136+0000 7fe2c0b75640 1 --2- 172.21.15.43:0/2091308459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2bc103b50 0x7fe2bc10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.136+0000 7fe2c0b75640 1 --2- 172.21.15.43:0/2091308459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2bc103190 0x7fe2bc103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.136+0000 7fe2c0b75640 1 --2- 172.21.15.43:0/2091308459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2bc101fa0 0x7fe2bc1023c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.136+0000 7fe2c0b75640 1 -- 172.21.15.43:0/2091308459 >> 172.21.15.43:0/2091308459 conn(0x7fe2bc0fd530 msgr2=0x7fe2bc0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.136+0000 7fe2c0b75640 1 -- 172.21.15.43:0/2091308459 shutdown_connections 2024-04-21T08:35:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.136+0000 7fe2c0b75640 1 -- 172.21.15.43:0/2091308459 wait complete. 2024-04-21T08:35:59.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.137+0000 7fe2c0b75640 1 Processor -- start 2024-04-21T08:35:59.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.137+0000 7fe2c0b75640 1 -- start start 2024-04-21T08:35:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.137+0000 7fe2c0b75640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2bc101fa0 0x7fe2bc075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.137+0000 7fe2c0b75640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2bc103190 0x7fe2bc074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.137+0000 7fe2c0b75640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2bc103b50 0x7fe2bc074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.137+0000 7fe2c0b75640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc10dae0 con 0x7fe2bc103b50 2024-04-21T08:35:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.137+0000 7fe2c0b75640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc10d960 con 0x7fe2bc101fa0 2024-04-21T08:35:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.137+0000 7fe2c0b75640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc10dc60 con 0x7fe2bc103190 2024-04-21T08:35:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.137+0000 7fe2bad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2bc103b50 0x7fe2bc074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.137+0000 7fe2ba59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2bc101fa0 0x7fe2bc075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.137+0000 7fe2bad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2bc103b50 0x7fe2bc074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58038/0 (socket says 172.21.15.43:58038) 2024-04-21T08:35:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.137+0000 7fe2bad9d640 1 -- 172.21.15.43:0/3565327286 learned_addr learned my addr 172.21.15.43:0/3565327286 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:35:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.137+0000 7fe2b9d9b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2bc103190 0x7fe2bc074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.137+0000 7fe2bad9d640 1 -- 172.21.15.43:0/3565327286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2bc103190 msgr2=0x7fe2bc074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.137+0000 7fe2bad9d640 1 --2- 172.21.15.43:0/3565327286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2bc103190 0x7fe2bc074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.137+0000 7fe2bad9d640 1 -- 172.21.15.43:0/3565327286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2bc101fa0 msgr2=0x7fe2bc075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:59.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.137+0000 7fe2bad9d640 1 --2- 172.21.15.43:0/3565327286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2bc101fa0 0x7fe2bc075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:59.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.137+0000 7fe2bad9d640 1 -- 172.21.15.43:0/3565327286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2bc075110 con 0x7fe2bc103b50 2024-04-21T08:35:59.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.138+0000 7fe2bad9d640 1 --2- 172.21.15.43:0/3565327286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2bc103b50 0x7fe2bc074880 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400eaa0 tx=0x7fe2b400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:59.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.138+0000 7fe2ab7fe640 1 -- 172.21.15.43:0/3565327286 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b400cdb0 con 0x7fe2bc103b50 2024-04-21T08:35:59.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.138+0000 7fe2ab7fe640 1 -- 172.21.15.43:0/3565327286 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2b4005020 con 0x7fe2bc103b50 2024-04-21T08:35:59.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.138+0000 7fe2c0b75640 1 -- 172.21.15.43:0/3565327286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2bc1a22c0 con 0x7fe2bc103b50 2024-04-21T08:35:59.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.138+0000 7fe2ab7fe640 1 -- 172.21.15.43:0/3565327286 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40116d0 con 0x7fe2bc103b50 2024-04-21T08:35:59.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.138+0000 7fe2c0b75640 1 -- 172.21.15.43:0/3565327286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2bc1a27c0 con 0x7fe2bc103b50 2024-04-21T08:35:59.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.139+0000 7fe2c0b75640 1 -- 172.21.15.43:0/3565327286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2bc109340 con 0x7fe2bc103b50 2024-04-21T08:35:59.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.140+0000 7fe2ab7fe640 1 -- 172.21.15.43:0/3565327286 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe2b40051d0 con 0x7fe2bc103b50 2024-04-21T08:35:59.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.140+0000 7fe2ab7fe640 1 --2- 172.21.15.43:0/3565327286 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe2a40a3bb0 0x7fe2a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:59.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.141+0000 7fe2ba59c640 1 --2- 172.21.15.43:0/3565327286 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe2a40a3bb0 0x7fe2a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:59.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.141+0000 7fe2ab7fe640 1 -- 172.21.15.43:0/3565327286 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe2b40d0190 con 0x7fe2bc103b50 2024-04-21T08:35:59.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.141+0000 7fe2ba59c640 1 --2- 172.21.15.43:0/3565327286 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe2a40a3bb0 0x7fe2a40a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac005ab0 tx=0x7fe2ac005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:59.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.145+0000 7fe2ab7fe640 1 -- 172.21.15.43:0/3565327286 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe2b4016030 con 0x7fe2bc103b50 2024-04-21T08:35:59.293 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:58 smithi112 ceph-mon[31085]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:59.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:58 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:59.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:58 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:59.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:58 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:59.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:58 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:59.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:58 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:59.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:58 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:59.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:58 smithi043 ceph-mon[27978]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:59.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:58 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:59.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:58 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:59.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:58 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:59.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:58 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:59.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:58 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:59.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:58 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:59.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.370+0000 7fe2c0b75640 1 -- 172.21.15.43:0/3565327286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2bc062b50 con 0x7fe2bc103b50 2024-04-21T08:35:59.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.371+0000 7fe2ab7fe640 1 -- 172.21.15.43:0/3565327286 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7fe2b409c920 con 0x7fe2bc103b50 2024-04-21T08:35:59.370 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:35:59.370 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-21T08:35:59.370 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 6 2024-04-21T08:35:59.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.373+0000 7fe2c0b75640 1 -- 172.21.15.43:0/3565327286 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe2a40a3bb0 msgr2=0x7fe2a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:59.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.373+0000 7fe2c0b75640 1 --2- 172.21.15.43:0/3565327286 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe2a40a3bb0 0x7fe2a40a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac005ab0 tx=0x7fe2ac005fd0 comp rx=0 tx=0).stop 2024-04-21T08:35:59.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.373+0000 7fe2c0b75640 1 -- 172.21.15.43:0/3565327286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2bc103b50 msgr2=0x7fe2bc074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:59.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.373+0000 7fe2c0b75640 1 --2- 172.21.15.43:0/3565327286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2bc103b50 0x7fe2bc074880 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400eaa0 tx=0x7fe2b400ef60 comp rx=0 tx=0).stop 2024-04-21T08:35:59.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.373+0000 7fe2c0b75640 1 -- 172.21.15.43:0/3565327286 shutdown_connections 2024-04-21T08:35:59.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.373+0000 7fe2c0b75640 1 --2- 172.21.15.43:0/3565327286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2bc103b50 0x7fe2bc074880 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:59.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.373+0000 7fe2c0b75640 1 --2- 172.21.15.43:0/3565327286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2bc103190 0x7fe2bc074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:59.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.373+0000 7fe2c0b75640 1 --2- 172.21.15.43:0/3565327286 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe2a40a3bb0 0x7fe2a40a6070 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:59.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.373+0000 7fe2c0b75640 1 --2- 172.21.15.43:0/3565327286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2bc101fa0 0x7fe2bc075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:59.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.373+0000 7fe2c0b75640 1 -- 172.21.15.43:0/3565327286 >> 172.21.15.43:0/3565327286 conn(0x7fe2bc0fd530 msgr2=0x7fe2bc106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:59.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.373+0000 7fe2c0b75640 1 -- 172.21.15.43:0/3565327286 shutdown_connections 2024-04-21T08:35:59.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:35:59.373+0000 7fe2c0b75640 1 -- 172.21.15.43:0/3565327286 wait complete. 2024-04-21T08:35:59.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:58 smithi175 ceph-mon[31920]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:59.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:58 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:59.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:58 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:59.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:58 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:35:59.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:58 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:59.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:58 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:59.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:58 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:00.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:59 smithi175 ceph-mon[31920]: Deploying daemon mds.f on smithi175 2024-04-21T08:36:00.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:59 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.112:6834/3159749962,v1:172.21.15.112:6835/3159749962] up:boot 2024-04-21T08:36:00.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:59 smithi175 ceph-mon[31920]: fsmap 5 up:standby 2024-04-21T08:36:00.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:59 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:36:00.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:35:59 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3565327286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:00.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:59 smithi112 ceph-mon[31085]: Deploying daemon mds.f on smithi175 2024-04-21T08:36:00.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:59 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.112:6834/3159749962,v1:172.21.15.112:6835/3159749962] up:boot 2024-04-21T08:36:00.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:59 smithi112 ceph-mon[31085]: fsmap 5 up:standby 2024-04-21T08:36:00.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:59 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:36:00.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:35:59 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3565327286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:00.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:59 smithi043 ceph-mon[27978]: Deploying daemon mds.f on smithi175 2024-04-21T08:36:00.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:59 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.112:6834/3159749962,v1:172.21.15.112:6835/3159749962] up:boot 2024-04-21T08:36:00.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:59 smithi043 ceph-mon[27978]: fsmap 5 up:standby 2024-04-21T08:36:00.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:59 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:36:00.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:35:59 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3565327286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:01.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:00 smithi175 ceph-mon[31920]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:01.214 INFO:journalctl@ceph.mds.f.smithi175.stdout:Apr 21 08:36:01 smithi175 systemd[1]: Starting Ceph mds.f for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:36:01.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:00 smithi112 ceph-mon[31085]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:01.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:00 smithi043 ceph-mon[27978]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:01.384 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:36:01.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.499+0000 7fe4e6315640 1 -- 172.21.15.43:0/2458894693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e0103190 msgr2=0x7fe4e0103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:01.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.499+0000 7fe4e6315640 1 --2- 172.21.15.43:0/2458894693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e0103190 0x7fe4e0103610 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009900 tx=0x7fe4c802bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:01.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.499+0000 7fe4e6315640 1 -- 172.21.15.43:0/2458894693 shutdown_connections 2024-04-21T08:36:01.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.499+0000 7fe4e6315640 1 --2- 172.21.15.43:0/2458894693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4e0103b50 0x7fe4e010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:01.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.499+0000 7fe4e6315640 1 --2- 172.21.15.43:0/2458894693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e0103190 0x7fe4e0103610 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:01.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.499+0000 7fe4e6315640 1 --2- 172.21.15.43:0/2458894693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4e0101fa0 0x7fe4e01023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:01.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.499+0000 7fe4e6315640 1 -- 172.21.15.43:0/2458894693 >> 172.21.15.43:0/2458894693 conn(0x7fe4e00fd530 msgr2=0x7fe4e00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.499+0000 7fe4e6315640 1 -- 172.21.15.43:0/2458894693 shutdown_connections 2024-04-21T08:36:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.500+0000 7fe4e6315640 1 -- 172.21.15.43:0/2458894693 wait complete. 2024-04-21T08:36:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.500+0000 7fe4e6315640 1 Processor -- start 2024-04-21T08:36:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.500+0000 7fe4e6315640 1 -- start start 2024-04-21T08:36:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.500+0000 7fe4e6315640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4e0101fa0 0x7fe4e0198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.500+0000 7fe4e6315640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4e0103190 0x7fe4e01986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.500+0000 7fe4e6315640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e0103b50 0x7fe4e019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.500+0000 7fe4e6315640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e010dae0 con 0x7fe4e0103b50 2024-04-21T08:36:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.500+0000 7fe4e6315640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e010d960 con 0x7fe4e0103190 2024-04-21T08:36:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.500+0000 7fe4e6315640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e010dc60 con 0x7fe4e0101fa0 2024-04-21T08:36:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.501+0000 7fe4e48b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e0103b50 0x7fe4e019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.501+0000 7fe4e48b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e0103b50 0x7fe4e019f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58054/0 (socket says 172.21.15.43:58054) 2024-04-21T08:36:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.501+0000 7fe4e48b2640 1 -- 172.21.15.43:0/936869316 learned_addr learned my addr 172.21.15.43:0/936869316 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.501+0000 7fe4df7fe640 1 --2- 172.21.15.43:0/936869316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4e0103190 0x7fe4e01986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.501+0000 7fe4dffff640 1 --2- 172.21.15.43:0/936869316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4e0101fa0 0x7fe4e0198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.501+0000 7fe4e48b2640 1 -- 172.21.15.43:0/936869316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4e0101fa0 msgr2=0x7fe4e0198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.501+0000 7fe4e48b2640 1 --2- 172.21.15.43:0/936869316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4e0101fa0 0x7fe4e0198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.501+0000 7fe4e48b2640 1 -- 172.21.15.43:0/936869316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4e0103190 msgr2=0x7fe4e01986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.501+0000 7fe4e48b2640 1 --2- 172.21.15.43:0/936869316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4e0103190 0x7fe4e01986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.501+0000 7fe4e48b2640 1 -- 172.21.15.43:0/936869316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e019fd90 con 0x7fe4e0103b50 2024-04-21T08:36:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.501+0000 7fe4df7fe640 1 --2- 172.21.15.43:0/936869316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4e0103190 0x7fe4e01986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.501+0000 7fe4e48b2640 1 --2- 172.21.15.43:0/936869316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e0103b50 0x7fe4e019f720 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800eb10 tx=0x7fe4d800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.501+0000 7fe4dd7fa640 1 -- 172.21.15.43:0/936869316 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8004b20 con 0x7fe4e0103b50 2024-04-21T08:36:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.501+0000 7fe4dd7fa640 1 -- 172.21.15.43:0/936869316 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d8004cc0 con 0x7fe4e0103b50 2024-04-21T08:36:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.501+0000 7fe4e6315640 1 -- 172.21.15.43:0/936869316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e01a0080 con 0x7fe4e0103b50 2024-04-21T08:36:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.501+0000 7fe4dd7fa640 1 -- 172.21.15.43:0/936869316 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8010670 con 0x7fe4e0103b50 2024-04-21T08:36:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.501+0000 7fe4e6315640 1 -- 172.21.15.43:0/936869316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e01a0580 con 0x7fe4e0103b50 2024-04-21T08:36:01.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.502+0000 7fe4e6315640 1 -- 172.21.15.43:0/936869316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4e0109340 con 0x7fe4e0103b50 2024-04-21T08:36:01.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.503+0000 7fe4dd7fa640 1 -- 172.21.15.43:0/936869316 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe4d8005020 con 0x7fe4e0103b50 2024-04-21T08:36:01.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.504+0000 7fe4dd7fa640 1 --2- 172.21.15.43:0/936869316 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe4cc0a3bb0 0x7fe4cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:01.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.504+0000 7fe4dd7fa640 1 -- 172.21.15.43:0/936869316 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe4d80d0640 con 0x7fe4e0103b50 2024-04-21T08:36:01.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.504+0000 7fe4dffff640 1 --2- 172.21.15.43:0/936869316 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe4cc0a3bb0 0x7fe4cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:01.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.505+0000 7fe4dffff640 1 --2- 172.21.15.43:0/936869316 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe4cc0a3bb0 0x7fe4cc0a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0102ff0 tx=0x7fe4d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:01.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.509+0000 7fe4dd7fa640 1 -- 172.21.15.43:0/936869316 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe4d809d4c0 con 0x7fe4e0103b50 2024-04-21T08:36:01.562 INFO:journalctl@ceph.mds.f.smithi175.stdout:Apr 21 08:36:01 smithi175 podman[61839]: 2024-04-21 08:36:01.18507627 +0000 UTC m=+0.017917308 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:36:01.562 INFO:journalctl@ceph.mds.f.smithi175.stdout:Apr 21 08:36:01 smithi175 podman[61839]: 2024-04-21 08:36:01.310544507 +0000 UTC m=+0.143385529 container create 9b7c904433b3674f4e2bd30c41d9e7f3b7e4c16bf275a7a066f0935593355b76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-21T08:36:01.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.737+0000 7fe4e6315640 1 -- 172.21.15.43:0/936869316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4e002d5c0 con 0x7fe4e0103b50 2024-04-21T08:36:01.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.738+0000 7fe4dd7fa640 1 -- 172.21.15.43:0/936869316 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7fe4d809cdc0 con 0x7fe4e0103b50 2024-04-21T08:36:01.737 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:01.737 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-21T08:36:01.738 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 6 2024-04-21T08:36:01.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.740+0000 7fe4e6315640 1 -- 172.21.15.43:0/936869316 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe4cc0a3bb0 msgr2=0x7fe4cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:01.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.740+0000 7fe4e6315640 1 --2- 172.21.15.43:0/936869316 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe4cc0a3bb0 0x7fe4cc0a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0102ff0 tx=0x7fe4d0005f90 comp rx=0 tx=0).stop 2024-04-21T08:36:01.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.740+0000 7fe4e6315640 1 -- 172.21.15.43:0/936869316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e0103b50 msgr2=0x7fe4e019f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:01.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.740+0000 7fe4e6315640 1 --2- 172.21.15.43:0/936869316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e0103b50 0x7fe4e019f720 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800eb10 tx=0x7fe4d800c2d0 comp rx=0 tx=0).stop 2024-04-21T08:36:01.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.740+0000 7fe4e6315640 1 -- 172.21.15.43:0/936869316 shutdown_connections 2024-04-21T08:36:01.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.740+0000 7fe4e6315640 1 --2- 172.21.15.43:0/936869316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e0103b50 0x7fe4e019f720 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:01.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.740+0000 7fe4e6315640 1 --2- 172.21.15.43:0/936869316 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe4cc0a3bb0 0x7fe4cc0a6070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:01.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.740+0000 7fe4e6315640 1 --2- 172.21.15.43:0/936869316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4e0103190 0x7fe4e01986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:01.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.740+0000 7fe4e6315640 1 --2- 172.21.15.43:0/936869316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4e0101fa0 0x7fe4e0198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:01.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.740+0000 7fe4e6315640 1 -- 172.21.15.43:0/936869316 >> 172.21.15.43:0/936869316 conn(0x7fe4e00fd530 msgr2=0x7fe4e0106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:01.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.740+0000 7fe4e6315640 1 -- 172.21.15.43:0/936869316 shutdown_connections 2024-04-21T08:36:01.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:01.740+0000 7fe4e6315640 1 -- 172.21.15.43:0/936869316 wait complete. 2024-04-21T08:36:01.964 INFO:journalctl@ceph.mds.f.smithi175.stdout:Apr 21 08:36:01 smithi175 podman[61839]: 2024-04-21 08:36:01.563033839 +0000 UTC m=+0.395874861 container init 9b7c904433b3674f4e2bd30c41d9e7f3b7e4c16bf275a7a066f0935593355b76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-04-21T08:36:01.964 INFO:journalctl@ceph.mds.f.smithi175.stdout:Apr 21 08:36:01 smithi175 podman[61839]: 2024-04-21 08:36:01.566365293 +0000 UTC m=+0.399206325 container start 9b7c904433b3674f4e2bd30c41d9e7f3b7e4c16bf275a7a066f0935593355b76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-04-21T08:36:01.964 INFO:journalctl@ceph.mds.f.smithi175.stdout:Apr 21 08:36:01 smithi175 ceph-mds[61858]: -- 172.21.15.175:0/2581881656 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564eff895a40 con 0x564eff868800 2024-04-21T08:36:01.965 INFO:journalctl@ceph.mds.f.smithi175.stdout:Apr 21 08:36:01 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-f[61854]: starting mds.f at 2024-04-21T08:36:01.965 INFO:journalctl@ceph.mds.f.smithi175.stdout:Apr 21 08:36:01 smithi175 bash[61839]: 9b7c904433b3674f4e2bd30c41d9e7f3b7e4c16bf275a7a066f0935593355b76 2024-04-21T08:36:01.965 INFO:journalctl@ceph.mds.f.smithi175.stdout:Apr 21 08:36:01 smithi175 systemd[1]: Started Ceph mds.f for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:36:02.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:02 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:02.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:02 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:02.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:02 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/936869316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:02.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:02 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:02.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:02 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:02.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:02 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:02.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:02 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:02.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:02 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.175:6834/466532477,v1:172.21.15.175:6835/466532477] up:boot 2024-04-21T08:36:02.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:02 smithi175 ceph-mon[31920]: fsmap 6 up:standby 2024-04-21T08:36:02.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:02 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:36:03.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:02 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:03.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:02 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:03.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:02 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/936869316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:03.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:02 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:03.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:02 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:03.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:02 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:03.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:02 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:03.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:02 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.175:6834/466532477,v1:172.21.15.175:6835/466532477] up:boot 2024-04-21T08:36:03.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:02 smithi112 ceph-mon[31085]: fsmap 6 up:standby 2024-04-21T08:36:03.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:02 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:36:03.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:02 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:03.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:02 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:03.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:02 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/936869316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:03.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:02 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:03.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:02 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:03.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:02 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:03.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:02 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:03.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:02 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.175:6834/466532477,v1:172.21.15.175:6835/466532477] up:boot 2024-04-21T08:36:03.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:02 smithi043 ceph-mon[27978]: fsmap 6 up:standby 2024-04-21T08:36:03.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:02 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:36:03.752 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:36:03.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.868+0000 7fcad863f640 1 -- 172.21.15.43:0/4022539721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcad0102050 msgr2=0x7fcad01024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:03.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.868+0000 7fcad863f640 1 --2- 172.21.15.43:0/4022539721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcad0102050 0x7fcad01024d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009900 tx=0x7fcac002bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:03.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.869+0000 7fcad863f640 1 -- 172.21.15.43:0/4022539721 shutdown_connections 2024-04-21T08:36:03.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.869+0000 7fcad863f640 1 --2- 172.21.15.43:0/4022539721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcad0102a10 0x7fcad0109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:03.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.869+0000 7fcad863f640 1 --2- 172.21.15.43:0/4022539721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcad0102050 0x7fcad01024d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:03.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.869+0000 7fcad863f640 1 --2- 172.21.15.43:0/4022539721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcad0100e70 0x7fcad0101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:03.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.869+0000 7fcad863f640 1 -- 172.21.15.43:0/4022539721 >> 172.21.15.43:0/4022539721 conn(0x7fcad00fc470 msgr2=0x7fcad00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:03.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.869+0000 7fcad863f640 1 -- 172.21.15.43:0/4022539721 shutdown_connections 2024-04-21T08:36:03.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.869+0000 7fcad863f640 1 -- 172.21.15.43:0/4022539721 wait complete. 2024-04-21T08:36:03.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.869+0000 7fcad863f640 1 Processor -- start 2024-04-21T08:36:03.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.869+0000 7fcad863f640 1 -- start start 2024-04-21T08:36:03.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.869+0000 7fcad863f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcad0100e70 0x7fcad01981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.869+0000 7fcad863f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcad0102050 0x7fcad01986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.869+0000 7fcad863f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcad0102a10 0x7fcad019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.869+0000 7fcad863f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcad010db40 con 0x7fcad0102a10 2024-04-21T08:36:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.869+0000 7fcad863f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcad010d9c0 con 0x7fcad0102050 2024-04-21T08:36:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.869+0000 7fcad863f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcad010dcc0 con 0x7fcad0100e70 2024-04-21T08:36:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.870+0000 7fcad6bdc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcad0102a10 0x7fcad019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.870+0000 7fcad63db640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcad0100e70 0x7fcad01981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.870+0000 7fcad6bdc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcad0102a10 0x7fcad019f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58070/0 (socket says 172.21.15.43:58070) 2024-04-21T08:36:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.870+0000 7fcad6bdc640 1 -- 172.21.15.43:0/2260567535 learned_addr learned my addr 172.21.15.43:0/2260567535 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.870+0000 7fcad5bda640 1 --2- 172.21.15.43:0/2260567535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcad0102050 0x7fcad01986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.870+0000 7fcad6bdc640 1 -- 172.21.15.43:0/2260567535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcad0100e70 msgr2=0x7fcad01981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.870+0000 7fcad6bdc640 1 --2- 172.21.15.43:0/2260567535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcad0100e70 0x7fcad01981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.870+0000 7fcad6bdc640 1 -- 172.21.15.43:0/2260567535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcad0102050 msgr2=0x7fcad01986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.870+0000 7fcad6bdc640 1 --2- 172.21.15.43:0/2260567535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcad0102050 0x7fcad01986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:03.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.870+0000 7fcad6bdc640 1 -- 172.21.15.43:0/2260567535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad019fdc0 con 0x7fcad0102a10 2024-04-21T08:36:03.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.870+0000 7fcad6bdc640 1 --2- 172.21.15.43:0/2260567535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcad0102a10 0x7fcad019f750 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcac800eb10 tx=0x7fcac800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:03.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.870+0000 7fcac77fe640 1 -- 172.21.15.43:0/2260567535 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac8004b20 con 0x7fcad0102a10 2024-04-21T08:36:03.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.870+0000 7fcad863f640 1 -- 172.21.15.43:0/2260567535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad01a00b0 con 0x7fcad0102a10 2024-04-21T08:36:03.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.870+0000 7fcac77fe640 1 -- 172.21.15.43:0/2260567535 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcac8004cc0 con 0x7fcad0102a10 2024-04-21T08:36:03.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.870+0000 7fcac77fe640 1 -- 172.21.15.43:0/2260567535 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac8010670 con 0x7fcad0102a10 2024-04-21T08:36:03.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.870+0000 7fcad863f640 1 -- 172.21.15.43:0/2260567535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad01a05b0 con 0x7fcad0102a10 2024-04-21T08:36:03.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.871+0000 7fcad863f640 1 -- 172.21.15.43:0/2260567535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad0108290 con 0x7fcad0102a10 2024-04-21T08:36:03.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.872+0000 7fcac77fe640 1 -- 172.21.15.43:0/2260567535 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcac8005020 con 0x7fcad0102a10 2024-04-21T08:36:03.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.873+0000 7fcac77fe640 1 --2- 172.21.15.43:0/2260567535 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fcabc0a3bb0 0x7fcabc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:03.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.873+0000 7fcad63db640 1 --2- 172.21.15.43:0/2260567535 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fcabc0a3bb0 0x7fcabc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:03.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.873+0000 7fcac77fe640 1 -- 172.21.15.43:0/2260567535 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fcac80d05c0 con 0x7fcad0102a10 2024-04-21T08:36:03.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.874+0000 7fcad63db640 1 --2- 172.21.15.43:0/2260567535 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fcabc0a3bb0 0x7fcabc0a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcad0101bc0 tx=0x7fcacc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:03.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:03.877+0000 7fcac77fe640 1 -- 172.21.15.43:0/2260567535 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcac809d440 con 0x7fcad0102a10 2024-04-21T08:36:03.966 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:03 smithi175 ceph-mon[31920]: Deploying daemon mds.c on smithi175 2024-04-21T08:36:03.966 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:03 smithi175 ceph-mon[31920]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:03.967 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:36:03 smithi175 systemd[1]: Starting Ceph mds.c for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:36:04.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:03 smithi112 ceph-mon[31085]: Deploying daemon mds.c on smithi175 2024-04-21T08:36:04.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:03 smithi112 ceph-mon[31085]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:04.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:03 smithi043 ceph-mon[27978]: Deploying daemon mds.c on smithi175 2024-04-21T08:36:04.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:03 smithi043 ceph-mon[27978]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:04.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:04.105+0000 7fcad863f640 1 -- 172.21.15.43:0/2260567535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcad002d5c0 con 0x7fcad0102a10 2024-04-21T08:36:04.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:04.105+0000 7fcac77fe640 1 -- 172.21.15.43:0/2260567535 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7fcac809cd40 con 0x7fcad0102a10 2024-04-21T08:36:04.104 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:04.104 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-21T08:36:04.105 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 7 2024-04-21T08:36:04.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:04.107+0000 7fcad863f640 1 -- 172.21.15.43:0/2260567535 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fcabc0a3bb0 msgr2=0x7fcabc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:04.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:04.107+0000 7fcad863f640 1 --2- 172.21.15.43:0/2260567535 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fcabc0a3bb0 0x7fcabc0a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcad0101bc0 tx=0x7fcacc005f90 comp rx=0 tx=0).stop 2024-04-21T08:36:04.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:04.107+0000 7fcad863f640 1 -- 172.21.15.43:0/2260567535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcad0102a10 msgr2=0x7fcad019f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:04.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:04.107+0000 7fcad863f640 1 --2- 172.21.15.43:0/2260567535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcad0102a10 0x7fcad019f750 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcac800eb10 tx=0x7fcac800c2d0 comp rx=0 tx=0).stop 2024-04-21T08:36:04.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:04.107+0000 7fcad863f640 1 -- 172.21.15.43:0/2260567535 shutdown_connections 2024-04-21T08:36:04.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:04.107+0000 7fcad863f640 1 --2- 172.21.15.43:0/2260567535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcad0102a10 0x7fcad019f750 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:04.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:04.107+0000 7fcad863f640 1 --2- 172.21.15.43:0/2260567535 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fcabc0a3bb0 0x7fcabc0a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:04.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:04.107+0000 7fcad863f640 1 --2- 172.21.15.43:0/2260567535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcad0102050 0x7fcad01986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:04.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:04.107+0000 7fcad863f640 1 --2- 172.21.15.43:0/2260567535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcad0100e70 0x7fcad01981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:04.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:04.107+0000 7fcad863f640 1 -- 172.21.15.43:0/2260567535 >> 172.21.15.43:0/2260567535 conn(0x7fcad00fc470 msgr2=0x7fcad0105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:04.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:04.107+0000 7fcad863f640 1 -- 172.21.15.43:0/2260567535 shutdown_connections 2024-04-21T08:36:04.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:04.108+0000 7fcad863f640 1 -- 172.21.15.43:0/2260567535 wait complete. 2024-04-21T08:36:04.341 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:36:04 smithi175 podman[62149]: 2024-04-21 08:36:03.964907546 +0000 UTC m=+0.018575236 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:36:04.341 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:36:04 smithi175 podman[62149]: 2024-04-21 08:36:04.081727319 +0000 UTC m=+0.135395050 container create bc8e2184d8abade86548f8ef1359450b62f0faec5c6341cd01d1f41e816b8001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True) 2024-04-21T08:36:04.341 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:36:04 smithi175 podman[62149]: 2024-04-21 08:36:04.342662571 +0000 UTC m=+0.396330259 container init bc8e2184d8abade86548f8ef1359450b62f0faec5c6341cd01d1f41e816b8001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-21T08:36:04.714 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:36:04 smithi175 podman[62149]: 2024-04-21 08:36:04.345905297 +0000 UTC m=+0.399572987 container start bc8e2184d8abade86548f8ef1359450b62f0faec5c6341cd01d1f41e816b8001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-21T08:36:04.714 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:36:04 smithi175 ceph-mds[62168]: -- 172.21.15.175:0/2880151202 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fb51313a40 con 0x55fb512e6800 2024-04-21T08:36:04.714 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:36:04 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: starting mds.c at 2024-04-21T08:36:04.714 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:36:04 smithi175 bash[62149]: bc8e2184d8abade86548f8ef1359450b62f0faec5c6341cd01d1f41e816b8001 2024-04-21T08:36:04.714 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:36:04 smithi175 systemd[1]: Started Ceph mds.c for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:36:05.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:04 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2260567535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:05.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:04 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:05.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:04 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:05.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:04 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:05.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:04 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:05.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:04 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:05.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:04 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:05.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:04 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2260567535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:05.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:04 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:05.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:04 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:05.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:04 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:05.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:04 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:05.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:04 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:05.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:04 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:05.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:04 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2260567535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:05.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:04 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:05.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:04 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:05.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:04 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:05.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:04 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:05.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:04 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:05.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:04 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:06.022 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:05 smithi112 ceph-mon[31085]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:06.022 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:05 smithi112 ceph-mon[31085]: Deploying daemon mds.h on smithi112 2024-04-21T08:36:06.022 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:05 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.175:6836/3198693848,v1:172.21.15.175:6837/3198693848] up:boot 2024-04-21T08:36:06.022 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:05 smithi112 ceph-mon[31085]: fsmap 7 up:standby 2024-04-21T08:36:06.022 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:05 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:36:06.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:05 smithi043 ceph-mon[27978]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:06.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:05 smithi043 ceph-mon[27978]: Deploying daemon mds.h on smithi112 2024-04-21T08:36:06.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:05 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.175:6836/3198693848,v1:172.21.15.175:6837/3198693848] up:boot 2024-04-21T08:36:06.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:05 smithi043 ceph-mon[27978]: fsmap 7 up:standby 2024-04-21T08:36:06.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:05 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:36:06.119 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:36:06.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:05 smithi175 ceph-mon[31920]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:06.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:05 smithi175 ceph-mon[31920]: Deploying daemon mds.h on smithi112 2024-04-21T08:36:06.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:05 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.175:6836/3198693848,v1:172.21.15.175:6837/3198693848] up:boot 2024-04-21T08:36:06.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:05 smithi175 ceph-mon[31920]: fsmap 7 up:standby 2024-04-21T08:36:06.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:05 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:36:06.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.223+0000 7efdaaec0640 1 -- 172.21.15.43:0/3260108527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efda4105320 msgr2=0x7efda4107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:06.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.223+0000 7efdaaec0640 1 --2- 172.21.15.43:0/3260108527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efda4105320 0x7efda4107710 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efd90009900 tx=0x7efd9002bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:06.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.223+0000 7efdaaec0640 1 -- 172.21.15.43:0/3260108527 shutdown_connections 2024-04-21T08:36:06.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.223+0000 7efdaaec0640 1 --2- 172.21.15.43:0/3260108527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efda4107c50 0x7efda410a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:06.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.223+0000 7efdaaec0640 1 --2- 172.21.15.43:0/3260108527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efda4105320 0x7efda4107710 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:06.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.223+0000 7efdaaec0640 1 --2- 172.21.15.43:0/3260108527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efda41029f0 0x7efda4104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:06.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.223+0000 7efdaaec0640 1 -- 172.21.15.43:0/3260108527 >> 172.21.15.43:0/3260108527 conn(0x7efda40fc3e0 msgr2=0x7efda40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:06.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.224+0000 7efdaaec0640 1 -- 172.21.15.43:0/3260108527 shutdown_connections 2024-04-21T08:36:06.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.224+0000 7efdaaec0640 1 -- 172.21.15.43:0/3260108527 wait complete. 2024-04-21T08:36:06.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.224+0000 7efdaaec0640 1 Processor -- start 2024-04-21T08:36:06.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.224+0000 7efdaaec0640 1 -- start start 2024-04-21T08:36:06.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.224+0000 7efdaaec0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efda41029f0 0x7efda41981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:06.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.224+0000 7efdaaec0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efda4105320 0x7efda4198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:06.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.224+0000 7efdaaec0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efda4107c50 0x7efda419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:06.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.224+0000 7efdaaec0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efda410daf0 con 0x7efda4105320 2024-04-21T08:36:06.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.224+0000 7efdaaec0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efda410d970 con 0x7efda4107c50 2024-04-21T08:36:06.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.224+0000 7efdaaec0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efda410dc70 con 0x7efda41029f0 2024-04-21T08:36:06.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.224+0000 7efda3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efda4105320 0x7efda4198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:06.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.224+0000 7efda3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efda4105320 0x7efda4198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58092/0 (socket says 172.21.15.43:58092) 2024-04-21T08:36:06.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.224+0000 7efda3fff640 1 -- 172.21.15.43:0/1814462682 learned_addr learned my addr 172.21.15.43:0/1814462682 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:06.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.224+0000 7efda8c5c640 1 --2- 172.21.15.43:0/1814462682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efda41029f0 0x7efda41981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:06.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.225+0000 7efda945d640 1 --2- 172.21.15.43:0/1814462682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efda4107c50 0x7efda419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:06.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.225+0000 7efda3fff640 1 -- 172.21.15.43:0/1814462682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efda41029f0 msgr2=0x7efda41981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:06.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.225+0000 7efda3fff640 1 --2- 172.21.15.43:0/1814462682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efda41029f0 0x7efda41981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:06.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.225+0000 7efda3fff640 1 -- 172.21.15.43:0/1814462682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efda4107c50 msgr2=0x7efda419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:06.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.225+0000 7efda3fff640 1 --2- 172.21.15.43:0/1814462682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efda4107c50 0x7efda419f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:06.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.225+0000 7efda3fff640 1 -- 172.21.15.43:0/1814462682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda419fde0 con 0x7efda4105320 2024-04-21T08:36:06.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.225+0000 7efda3fff640 1 --2- 172.21.15.43:0/1814462682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efda4105320 0x7efda4198700 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efd90000c00 tx=0x7efd900040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:06.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.225+0000 7efda1ffb640 1 -- 172.21.15.43:0/1814462682 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90043020 con 0x7efda4105320 2024-04-21T08:36:06.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.225+0000 7efdaaec0640 1 -- 172.21.15.43:0/1814462682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda41a0070 con 0x7efda4105320 2024-04-21T08:36:06.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.225+0000 7efda1ffb640 1 -- 172.21.15.43:0/1814462682 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd900044c0 con 0x7efda4105320 2024-04-21T08:36:06.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.225+0000 7efda1ffb640 1 -- 172.21.15.43:0/1814462682 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90003c90 con 0x7efda4105320 2024-04-21T08:36:06.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.225+0000 7efdaaec0640 1 -- 172.21.15.43:0/1814462682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda41a0610 con 0x7efda4105320 2024-04-21T08:36:06.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.226+0000 7efdaaec0640 1 -- 172.21.15.43:0/1814462682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efda40fded0 con 0x7efda4105320 2024-04-21T08:36:06.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.227+0000 7efda1ffb640 1 -- 172.21.15.43:0/1814462682 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efd90046020 con 0x7efda4105320 2024-04-21T08:36:06.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.228+0000 7efda1ffb640 1 --2- 172.21.15.43:0/1814462682 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7efd940a3bb0 0x7efd940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:06.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.228+0000 7efda1ffb640 1 -- 172.21.15.43:0/1814462682 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7efd900f21f0 con 0x7efda4105320 2024-04-21T08:36:06.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.228+0000 7efda8c5c640 1 --2- 172.21.15.43:0/1814462682 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7efd940a3bb0 0x7efd940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:06.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.229+0000 7efda8c5c640 1 --2- 172.21.15.43:0/1814462682 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7efd940a3bb0 0x7efd940a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efd98005fd0 tx=0x7efd9800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:06.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.233+0000 7efda1ffb640 1 -- 172.21.15.43:0/1814462682 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd9004d020 con 0x7efda4105320 2024-04-21T08:36:06.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.459+0000 7efdaaec0640 1 -- 172.21.15.43:0/1814462682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efda402d5c0 con 0x7efda4105320 2024-04-21T08:36:06.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.459+0000 7efda1ffb640 1 -- 172.21.15.43:0/1814462682 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7efd900befd0 con 0x7efda4105320 2024-04-21T08:36:06.459 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:06.459 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-21T08:36:06.459 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 8 2024-04-21T08:36:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.462+0000 7efdaaec0640 1 -- 172.21.15.43:0/1814462682 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7efd940a3bb0 msgr2=0x7efd940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.462+0000 7efdaaec0640 1 --2- 172.21.15.43:0/1814462682 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7efd940a3bb0 0x7efd940a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efd98005fd0 tx=0x7efd9800b390 comp rx=0 tx=0).stop 2024-04-21T08:36:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.462+0000 7efdaaec0640 1 -- 172.21.15.43:0/1814462682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efda4105320 msgr2=0x7efda4198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.462+0000 7efdaaec0640 1 --2- 172.21.15.43:0/1814462682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efda4105320 0x7efda4198700 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efd90000c00 tx=0x7efd900040c0 comp rx=0 tx=0).stop 2024-04-21T08:36:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.462+0000 7efdaaec0640 1 -- 172.21.15.43:0/1814462682 shutdown_connections 2024-04-21T08:36:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.462+0000 7efdaaec0640 1 --2- 172.21.15.43:0/1814462682 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7efd940a3bb0 0x7efd940a6070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.462+0000 7efdaaec0640 1 --2- 172.21.15.43:0/1814462682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efda4107c50 0x7efda419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.462+0000 7efdaaec0640 1 --2- 172.21.15.43:0/1814462682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efda4105320 0x7efda4198700 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:06.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.462+0000 7efdaaec0640 1 --2- 172.21.15.43:0/1814462682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efda41029f0 0x7efda41981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:06.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.462+0000 7efdaaec0640 1 -- 172.21.15.43:0/1814462682 >> 172.21.15.43:0/1814462682 conn(0x7efda40fc3e0 msgr2=0x7efda4103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:06.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.462+0000 7efdaaec0640 1 -- 172.21.15.43:0/1814462682 shutdown_connections 2024-04-21T08:36:06.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:06.462+0000 7efdaaec0640 1 -- 172.21.15.43:0/1814462682 wait complete. 2024-04-21T08:36:06.972 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:06 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1814462682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:06.973 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 21 08:36:06 smithi112 systemd[1]: Starting Ceph mds.h for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:36:07.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:06 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1814462682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:07.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:06 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1814462682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:07.262 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 21 08:36:06 smithi112 podman[61250]: 2024-04-21 08:36:06.87613331 +0000 UTC m=+0.017783920 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:36:07.262 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 21 08:36:07 smithi112 podman[61250]: 2024-04-21 08:36:07.010983691 +0000 UTC m=+0.152634287 container create 74e1523b5639d2cccb7f21e93c565a16becdd07285169fd9f111f6c5dd45f226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-21T08:36:07.544 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 21 08:36:07 smithi112 podman[61250]: 2024-04-21 08:36:07.264069815 +0000 UTC m=+0.405720410 container init 74e1523b5639d2cccb7f21e93c565a16becdd07285169fd9f111f6c5dd45f226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:36:07.544 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 21 08:36:07 smithi112 podman[61250]: 2024-04-21 08:36:07.266872197 +0000 UTC m=+0.408522799 container start 74e1523b5639d2cccb7f21e93c565a16becdd07285169fd9f111f6c5dd45f226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-h, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-21T08:36:07.544 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 21 08:36:07 smithi112 ceph-mds[61269]: -- 172.21.15.112:0/2489699518 <== mon.2 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56256b011a40 con 0x56256afe4800 2024-04-21T08:36:07.544 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 21 08:36:07 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-h[61265]: starting mds.h at 2024-04-21T08:36:07.544 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 21 08:36:07 smithi112 bash[61250]: 74e1523b5639d2cccb7f21e93c565a16becdd07285169fd9f111f6c5dd45f226 2024-04-21T08:36:07.544 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 21 08:36:07 smithi112 systemd[1]: Started Ceph mds.h for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:36:08.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:07 smithi112 ceph-mon[31085]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:08.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:07 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:08.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:07 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:08.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:07 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:08.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:07 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:08.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:07 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:08.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:07 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:08.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:07 smithi043 ceph-mon[27978]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:08.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:07 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:08.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:07 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:08.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:07 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:08.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:07 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:08.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:07 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:08.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:07 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:08.114 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:07 smithi175 ceph-mon[31920]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:08.114 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:07 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:08.114 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:07 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:08.114 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:07 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:08.114 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:07 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:08.114 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:07 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:08.115 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:07 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:08.473 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:36:08.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.589+0000 7f6403e5c640 1 -- 172.21.15.43:0/2345571912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63fc0749b0 msgr2=0x7f63fc074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:08.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.589+0000 7f6403e5c640 1 --2- 172.21.15.43:0/2345571912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63fc0749b0 0x7f63fc074e30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f63ec009930 tx=0x7f63ec02bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:08.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.589+0000 7f6403e5c640 1 -- 172.21.15.43:0/2345571912 shutdown_connections 2024-04-21T08:36:08.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.589+0000 7f6403e5c640 1 --2- 172.21.15.43:0/2345571912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63fc10e560 0x7f63fc110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:08.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.589+0000 7f6403e5c640 1 --2- 172.21.15.43:0/2345571912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63fc0749b0 0x7f63fc074e30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:08.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.589+0000 7f6403e5c640 1 --2- 172.21.15.43:0/2345571912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63fc075f90 0x7f63fc0743e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:08.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.589+0000 7f6403e5c640 1 -- 172.21.15.43:0/2345571912 >> 172.21.15.43:0/2345571912 conn(0x7f63fc0fd500 msgr2=0x7f63fc0ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:08.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.589+0000 7f6403e5c640 1 -- 172.21.15.43:0/2345571912 shutdown_connections 2024-04-21T08:36:08.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.590+0000 7f6403e5c640 1 -- 172.21.15.43:0/2345571912 wait complete. 2024-04-21T08:36:08.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.590+0000 7f6403e5c640 1 Processor -- start 2024-04-21T08:36:08.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.590+0000 7f6403e5c640 1 -- start start 2024-04-21T08:36:08.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.590+0000 7f6403e5c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63fc0749b0 0x7f63fc19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:08.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.590+0000 7f6403e5c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63fc075f90 0x7f63fc19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:08.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.590+0000 7f6403e5c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63fc10e560 0x7f63fc1a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:08.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.590+0000 7f6403e5c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc1140d0 con 0x7f63fc0749b0 2024-04-21T08:36:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.590+0000 7f6403e5c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc113f50 con 0x7f63fc10e560 2024-04-21T08:36:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.590+0000 7f6403e5c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc114250 con 0x7f63fc075f90 2024-04-21T08:36:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.590+0000 7f6401bf8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63fc0749b0 0x7f63fc19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.590+0000 7f6401bf8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63fc0749b0 0x7f63fc19e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42104/0 (socket says 172.21.15.43:42104) 2024-04-21T08:36:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.590+0000 7f6401bf8640 1 -- 172.21.15.43:0/67783524 learned_addr learned my addr 172.21.15.43:0/67783524 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.591+0000 7f64023f9640 1 --2- 172.21.15.43:0/67783524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63fc10e560 0x7f63fc1a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.591+0000 7f64013f7640 1 --2- 172.21.15.43:0/67783524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63fc075f90 0x7f63fc19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.591+0000 7f6401bf8640 1 -- 172.21.15.43:0/67783524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63fc075f90 msgr2=0x7f63fc19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.591+0000 7f6401bf8640 1 --2- 172.21.15.43:0/67783524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63fc075f90 0x7f63fc19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.591+0000 7f6401bf8640 1 -- 172.21.15.43:0/67783524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63fc10e560 msgr2=0x7f63fc1a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.591+0000 7f6401bf8640 1 --2- 172.21.15.43:0/67783524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63fc10e560 0x7f63fc1a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.591+0000 7f6401bf8640 1 -- 172.21.15.43:0/67783524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63fc1a6420 con 0x7f63fc0749b0 2024-04-21T08:36:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.591+0000 7f6401bf8640 1 --2- 172.21.15.43:0/67783524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63fc0749b0 0x7f63fc19e800 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f63f800c890 tx=0x7f63f800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.591+0000 7f63f2ffd640 1 -- 172.21.15.43:0/67783524 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8004220 con 0x7f63fc0749b0 2024-04-21T08:36:08.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.591+0000 7f63f2ffd640 1 -- 172.21.15.43:0/67783524 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63f80043c0 con 0x7f63fc0749b0 2024-04-21T08:36:08.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.591+0000 7f6403e5c640 1 -- 172.21.15.43:0/67783524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63fc1a6710 con 0x7f63fc0749b0 2024-04-21T08:36:08.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.591+0000 7f63f2ffd640 1 -- 172.21.15.43:0/67783524 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8004a00 con 0x7f63fc0749b0 2024-04-21T08:36:08.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.591+0000 7f6403e5c640 1 -- 172.21.15.43:0/67783524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63fc1a6c30 con 0x7f63fc0749b0 2024-04-21T08:36:08.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.592+0000 7f6403e5c640 1 -- 172.21.15.43:0/67783524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63fc104620 con 0x7f63fc0749b0 2024-04-21T08:36:08.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.593+0000 7f63f2ffd640 1 -- 172.21.15.43:0/67783524 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f63f80079b0 con 0x7f63fc0749b0 2024-04-21T08:36:08.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.594+0000 7f63f2ffd640 1 --2- 172.21.15.43:0/67783524 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f63e80a3bb0 0x7f63e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:08.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.594+0000 7f64013f7640 1 --2- 172.21.15.43:0/67783524 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f63e80a3bb0 0x7f63e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:08.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.594+0000 7f63f2ffd640 1 -- 172.21.15.43:0/67783524 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f63f80cfd40 con 0x7f63fc0749b0 2024-04-21T08:36:08.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.594+0000 7f64013f7640 1 --2- 172.21.15.43:0/67783524 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f63e80a3bb0 0x7f63e80a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f63ec006010 tx=0x7f63ec02c740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:08.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.599+0000 7f63f2ffd640 1 -- 172.21.15.43:0/67783524 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f63f8014030 con 0x7f63fc0749b0 2024-04-21T08:36:08.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:08 smithi043 ceph-mon[27978]: Deploying daemon mds.i on smithi175 2024-04-21T08:36:08.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:08 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] up:boot 2024-04-21T08:36:08.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:08 smithi043 ceph-mon[27978]: fsmap 8 up:standby 2024-04-21T08:36:08.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:08 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:36:08.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.826+0000 7f6403e5c640 1 -- 172.21.15.43:0/67783524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63fc02d5c0 con 0x7f63fc0749b0 2024-04-21T08:36:08.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.827+0000 7f63f2ffd640 1 -- 172.21.15.43:0/67783524 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7180 (secure 0 0 0) 0x7f63f809cb20 con 0x7f63fc0749b0 2024-04-21T08:36:08.826 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:08.826 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-21T08:36:08.826 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2024-04-21T08:36:08.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.829+0000 7f6403e5c640 1 -- 172.21.15.43:0/67783524 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f63e80a3bb0 msgr2=0x7f63e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:08.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.829+0000 7f6403e5c640 1 --2- 172.21.15.43:0/67783524 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f63e80a3bb0 0x7f63e80a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f63ec006010 tx=0x7f63ec02c740 comp rx=0 tx=0).stop 2024-04-21T08:36:08.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.829+0000 7f6403e5c640 1 -- 172.21.15.43:0/67783524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63fc0749b0 msgr2=0x7f63fc19e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:08.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.829+0000 7f6403e5c640 1 --2- 172.21.15.43:0/67783524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63fc0749b0 0x7f63fc19e800 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f63f800c890 tx=0x7f63f800cd50 comp rx=0 tx=0).stop 2024-04-21T08:36:08.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.829+0000 7f6403e5c640 1 -- 172.21.15.43:0/67783524 shutdown_connections 2024-04-21T08:36:08.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.829+0000 7f6403e5c640 1 --2- 172.21.15.43:0/67783524 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f63e80a3bb0 0x7f63e80a6070 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:08.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.829+0000 7f6403e5c640 1 --2- 172.21.15.43:0/67783524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63fc10e560 0x7f63fc1a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:08.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.829+0000 7f6403e5c640 1 --2- 172.21.15.43:0/67783524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63fc075f90 0x7f63fc19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:08.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.829+0000 7f6403e5c640 1 --2- 172.21.15.43:0/67783524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63fc0749b0 0x7f63fc19e800 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:08.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.829+0000 7f6403e5c640 1 -- 172.21.15.43:0/67783524 >> 172.21.15.43:0/67783524 conn(0x7f63fc0fd500 msgr2=0x7f63fc1017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:08.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.829+0000 7f6403e5c640 1 -- 172.21.15.43:0/67783524 shutdown_connections 2024-04-21T08:36:08.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:08.829+0000 7f6403e5c640 1 -- 172.21.15.43:0/67783524 wait complete. 2024-04-21T08:36:09.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:08 smithi112 ceph-mon[31085]: Deploying daemon mds.i on smithi175 2024-04-21T08:36:09.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:08 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] up:boot 2024-04-21T08:36:09.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:08 smithi112 ceph-mon[31085]: fsmap 8 up:standby 2024-04-21T08:36:09.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:08 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:36:09.119 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:08 smithi175 ceph-mon[31920]: Deploying daemon mds.i on smithi175 2024-04-21T08:36:09.120 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:08 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] up:boot 2024-04-21T08:36:09.120 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:08 smithi175 ceph-mon[31920]: fsmap 8 up:standby 2024-04-21T08:36:09.120 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:08 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:36:09.714 INFO:journalctl@ceph.mds.i.smithi175.stdout:Apr 21 08:36:09 smithi175 systemd[1]: Starting Ceph mds.i for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:36:10.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:09 smithi112 ceph-mon[31085]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:10.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:09 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/67783524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:10.052 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:09 smithi175 ceph-mon[31920]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:10.052 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:09 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/67783524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:10.052 INFO:journalctl@ceph.mds.i.smithi175.stdout:Apr 21 08:36:09 smithi175 podman[62454]: 2024-04-21 08:36:09.769224586 +0000 UTC m=+0.018299363 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:36:10.052 INFO:journalctl@ceph.mds.i.smithi175.stdout:Apr 21 08:36:09 smithi175 podman[62454]: 2024-04-21 08:36:09.910733935 +0000 UTC m=+0.159808704 container create 03ad24450f0c3fae67212ea21f599898c554e679ad9b93e4fbabc7e339ef994a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-04-21T08:36:10.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:09 smithi043 ceph-mon[27978]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:10.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:09 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/67783524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:10.714 INFO:journalctl@ceph.mds.i.smithi175.stdout:Apr 21 08:36:10 smithi175 podman[62454]: 2024-04-21 08:36:10.244590427 +0000 UTC m=+0.493665198 container init 03ad24450f0c3fae67212ea21f599898c554e679ad9b93e4fbabc7e339ef994a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:36:10.714 INFO:journalctl@ceph.mds.i.smithi175.stdout:Apr 21 08:36:10 smithi175 podman[62454]: 2024-04-21 08:36:10.247198954 +0000 UTC m=+0.496273727 container start 03ad24450f0c3fae67212ea21f599898c554e679ad9b93e4fbabc7e339ef994a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-21T08:36:10.714 INFO:journalctl@ceph.mds.i.smithi175.stdout:Apr 21 08:36:10 smithi175 ceph-mds[62473]: -- 172.21.15.175:0/3494999369 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55919d2d1a40 con 0x55919d2a5000 2024-04-21T08:36:10.714 INFO:journalctl@ceph.mds.i.smithi175.stdout:Apr 21 08:36:10 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-i[62469]: starting mds.i at 2024-04-21T08:36:10.714 INFO:journalctl@ceph.mds.i.smithi175.stdout:Apr 21 08:36:10 smithi175 bash[62454]: 03ad24450f0c3fae67212ea21f599898c554e679ad9b93e4fbabc7e339ef994a 2024-04-21T08:36:10.714 INFO:journalctl@ceph.mds.i.smithi175.stdout:Apr 21 08:36:10 smithi175 systemd[1]: Started Ceph mds.i for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:36:10.840 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:36:10.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.945+0000 7f9de366d640 1 -- 172.21.15.43:0/3041373652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ddc107cb0 msgr2=0x7f9ddc10a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:10.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.945+0000 7f9de366d640 1 --2- 172.21.15.43:0/3041373652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ddc107cb0 0x7f9ddc10a0a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4009ca0 tx=0x7f9dd402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:10.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.945+0000 7f9de366d640 1 -- 172.21.15.43:0/3041373652 shutdown_connections 2024-04-21T08:36:10.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.945+0000 7f9de366d640 1 --2- 172.21.15.43:0/3041373652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ddc107cb0 0x7f9ddc10a0a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:10.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.945+0000 7f9de366d640 1 --2- 172.21.15.43:0/3041373652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ddc105380 0x7f9ddc107770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:10.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.945+0000 7f9de366d640 1 --2- 172.21.15.43:0/3041373652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ddc102a50 0x7f9ddc104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:10.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.945+0000 7f9de366d640 1 -- 172.21.15.43:0/3041373652 >> 172.21.15.43:0/3041373652 conn(0x7f9ddc0fc3e0 msgr2=0x7f9ddc0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:10.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.945+0000 7f9de366d640 1 -- 172.21.15.43:0/3041373652 shutdown_connections 2024-04-21T08:36:10.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.945+0000 7f9de366d640 1 -- 172.21.15.43:0/3041373652 wait complete. 2024-04-21T08:36:10.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.946+0000 7f9de366d640 1 Processor -- start 2024-04-21T08:36:10.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.946+0000 7f9de366d640 1 -- start start 2024-04-21T08:36:10.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.946+0000 7f9de366d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ddc102a50 0x7f9ddc198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:10.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.946+0000 7f9de366d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ddc105380 0x7f9ddc198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:10.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.946+0000 7f9de366d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ddc107cb0 0x7f9ddc19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:10.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.946+0000 7f9de366d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc10daa0 con 0x7f9ddc107cb0 2024-04-21T08:36:10.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.946+0000 7f9de366d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc10d920 con 0x7f9ddc102a50 2024-04-21T08:36:10.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.946+0000 7f9de366d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc10dc20 con 0x7f9ddc105380 2024-04-21T08:36:10.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.946+0000 7f9de1c0a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ddc107cb0 0x7f9ddc19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:10.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.946+0000 7f9de1c0a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ddc107cb0 0x7f9ddc19f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42120/0 (socket says 172.21.15.43:42120) 2024-04-21T08:36:10.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.946+0000 7f9de1c0a640 1 -- 172.21.15.43:0/360092211 learned_addr learned my addr 172.21.15.43:0/360092211 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:10.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.946+0000 7f9de1409640 1 --2- 172.21.15.43:0/360092211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ddc102a50 0x7f9ddc198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:10.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.946+0000 7f9de0c08640 1 --2- 172.21.15.43:0/360092211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ddc105380 0x7f9ddc198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:10.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.946+0000 7f9de1c0a640 1 -- 172.21.15.43:0/360092211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ddc105380 msgr2=0x7f9ddc198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:10.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.946+0000 7f9de1c0a640 1 --2- 172.21.15.43:0/360092211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ddc105380 0x7f9ddc198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:10.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.946+0000 7f9de1c0a640 1 -- 172.21.15.43:0/360092211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ddc102a50 msgr2=0x7f9ddc198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:10.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.946+0000 7f9de1c0a640 1 --2- 172.21.15.43:0/360092211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ddc102a50 0x7f9ddc198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:10.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.946+0000 7f9de1c0a640 1 -- 172.21.15.43:0/360092211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ddc19fd60 con 0x7f9ddc107cb0 2024-04-21T08:36:10.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.947+0000 7f9de1c0a640 1 --2- 172.21.15.43:0/360092211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ddc107cb0 0x7f9ddc19f6f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9dd402c700 tx=0x7f9dd40093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:10.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.947+0000 7f9dd27fc640 1 -- 172.21.15.43:0/360092211 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4041020 con 0x7f9ddc107cb0 2024-04-21T08:36:10.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.947+0000 7f9dd27fc640 1 -- 172.21.15.43:0/360092211 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dd4034d60 con 0x7f9ddc107cb0 2024-04-21T08:36:10.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.947+0000 7f9de366d640 1 -- 172.21.15.43:0/360092211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ddc19fff0 con 0x7f9ddc107cb0 2024-04-21T08:36:10.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.947+0000 7f9dd27fc640 1 -- 172.21.15.43:0/360092211 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4039040 con 0x7f9ddc107cb0 2024-04-21T08:36:10.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.947+0000 7f9de366d640 1 -- 172.21.15.43:0/360092211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ddc1a04b0 con 0x7f9ddc107cb0 2024-04-21T08:36:10.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.948+0000 7f9de366d640 1 -- 172.21.15.43:0/360092211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ddc0fded0 con 0x7f9ddc107cb0 2024-04-21T08:36:10.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.949+0000 7f9dd27fc640 1 -- 172.21.15.43:0/360092211 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9dd4044020 con 0x7f9ddc107cb0 2024-04-21T08:36:10.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.950+0000 7f9dd27fc640 1 --2- 172.21.15.43:0/360092211 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9dc80a3c30 0x7f9dc80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:10.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.950+0000 7f9de1409640 1 --2- 172.21.15.43:0/360092211 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9dc80a3c30 0x7f9dc80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:10.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.951+0000 7f9dd27fc640 1 -- 172.21.15.43:0/360092211 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9dd40f2190 con 0x7f9ddc107cb0 2024-04-21T08:36:10.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.951+0000 7f9de1409640 1 --2- 172.21.15.43:0/360092211 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9dc80a3c30 0x7f9dc80a60f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0095f0 tx=0x7f9dcc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:10.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:10.954+0000 7f9dd27fc640 1 -- 172.21.15.43:0/360092211 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9dd404b020 con 0x7f9ddc107cb0 2024-04-21T08:36:11.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:11.188+0000 7f9de366d640 1 -- 172.21.15.43:0/360092211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ddc02d5c0 con 0x7f9ddc107cb0 2024-04-21T08:36:11.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:11.188+0000 7f9dd27fc640 1 -- 172.21.15.43:0/360092211 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8010 (secure 0 0 0) 0x7f9dd40bf010 con 0x7f9ddc107cb0 2024-04-21T08:36:11.188 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:11.188 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-21T08:36:11.188 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 10 2024-04-21T08:36:11.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:11.190+0000 7f9de366d640 1 -- 172.21.15.43:0/360092211 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9dc80a3c30 msgr2=0x7f9dc80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:11.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:11.190+0000 7f9de366d640 1 --2- 172.21.15.43:0/360092211 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9dc80a3c30 0x7f9dc80a60f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0095f0 tx=0x7f9dcc005f90 comp rx=0 tx=0).stop 2024-04-21T08:36:11.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:11.190+0000 7f9de366d640 1 -- 172.21.15.43:0/360092211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ddc107cb0 msgr2=0x7f9ddc19f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:11.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:11.190+0000 7f9de366d640 1 --2- 172.21.15.43:0/360092211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ddc107cb0 0x7f9ddc19f6f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9dd402c700 tx=0x7f9dd40093f0 comp rx=0 tx=0).stop 2024-04-21T08:36:11.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:11.191+0000 7f9de366d640 1 -- 172.21.15.43:0/360092211 shutdown_connections 2024-04-21T08:36:11.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:11.191+0000 7f9de366d640 1 --2- 172.21.15.43:0/360092211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ddc107cb0 0x7f9ddc19f6f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:11.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:11.191+0000 7f9de366d640 1 --2- 172.21.15.43:0/360092211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ddc105380 0x7f9ddc198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:11.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:11.191+0000 7f9de366d640 1 --2- 172.21.15.43:0/360092211 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9dc80a3c30 0x7f9dc80a60f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:11.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:11.191+0000 7f9de366d640 1 --2- 172.21.15.43:0/360092211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ddc102a50 0x7f9ddc198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:11.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:11.191+0000 7f9de366d640 1 -- 172.21.15.43:0/360092211 >> 172.21.15.43:0/360092211 conn(0x7f9ddc0fc3e0 msgr2=0x7f9ddc103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:11.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:11.191+0000 7f9de366d640 1 -- 172.21.15.43:0/360092211 shutdown_connections 2024-04-21T08:36:11.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:11.191+0000 7f9de366d640 1 -- 172.21.15.43:0/360092211 wait complete. 2024-04-21T08:36:11.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:11 smithi175 ceph-mon[31920]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:11.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:11 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:11.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:11 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:11.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:11 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:11.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:11 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:11.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:11 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:11.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:11 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:11.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:11 smithi175 ceph-mon[31920]: Deploying daemon mds.g on smithi043 2024-04-21T08:36:11.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:11 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] up:boot 2024-04-21T08:36:11.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:11 smithi175 ceph-mon[31920]: fsmap 9 up:standby 2024-04-21T08:36:11.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:11 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:36:11.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:11 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/360092211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:11.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:11 smithi112 ceph-mon[31085]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:11.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:11 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:11.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:11 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:11.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:11 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:11.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:11 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:11.796 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:11 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:11.796 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:11 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:11.796 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:11 smithi112 ceph-mon[31085]: Deploying daemon mds.g on smithi043 2024-04-21T08:36:11.796 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:11 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] up:boot 2024-04-21T08:36:11.797 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:11 smithi112 ceph-mon[31085]: fsmap 9 up:standby 2024-04-21T08:36:11.797 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:11 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:36:11.797 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:11 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/360092211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:11.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:11 smithi043 ceph-mon[27978]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:11.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:11 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:11.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:11 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:11.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:11 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:11.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:11 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:11.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:11 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:11.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:11 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:11.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:11 smithi043 ceph-mon[27978]: Deploying daemon mds.g on smithi043 2024-04-21T08:36:11.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:11 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] up:boot 2024-04-21T08:36:11.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:11 smithi043 ceph-mon[27978]: fsmap 9 up:standby 2024-04-21T08:36:11.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:11 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:36:11.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:11 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/360092211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:12.811 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 21 08:36:12 smithi043 systemd[1]: Starting Ceph mds.g for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:36:13.174 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 21 08:36:12 smithi043 podman[67134]: 2024-04-21 08:36:12.785832785 +0000 UTC m=+0.018649026 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:36:13.175 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 21 08:36:12 smithi043 podman[67134]: 2024-04-21 08:36:12.91461838 +0000 UTC m=+0.147434618 container create a244b7e4086e783f885905983bb3b213e0c10f87d8068d8f73e30d9d4d2d1e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-21T08:36:13.175 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 21 08:36:13 smithi043 podman[67134]: 2024-04-21 08:36:13.175689617 +0000 UTC m=+0.408505856 container init a244b7e4086e783f885905983bb3b213e0c10f87d8068d8f73e30d9d4d2d1e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS) 2024-04-21T08:36:13.201 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:36:13.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.309+0000 7f49458f1640 1 -- 172.21.15.43:0/522612906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4940101fc0 msgr2=0x7f4940102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:13.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.309+0000 7f49458f1640 1 --2- 172.21.15.43:0/522612906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4940101fc0 0x7f4940102440 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4934009900 tx=0x7f493402bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:13.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.309+0000 7f49458f1640 1 -- 172.21.15.43:0/522612906 shutdown_connections 2024-04-21T08:36:13.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.309+0000 7f49458f1640 1 --2- 172.21.15.43:0/522612906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4940102980 0x7f49401092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:13.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.309+0000 7f49458f1640 1 --2- 172.21.15.43:0/522612906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4940101fc0 0x7f4940102440 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:13.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.309+0000 7f49458f1640 1 --2- 172.21.15.43:0/522612906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4940100e70 0x7f4940101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.309+0000 7f49458f1640 1 -- 172.21.15.43:0/522612906 >> 172.21.15.43:0/522612906 conn(0x7f49400fc3e0 msgr2=0x7f49400fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.309+0000 7f49458f1640 1 -- 172.21.15.43:0/522612906 shutdown_connections 2024-04-21T08:36:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.309+0000 7f49458f1640 1 -- 172.21.15.43:0/522612906 wait complete. 2024-04-21T08:36:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.310+0000 7f49458f1640 1 Processor -- start 2024-04-21T08:36:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.310+0000 7f49458f1640 1 -- start start 2024-04-21T08:36:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.310+0000 7f49458f1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4940100e70 0x7f4940069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.310+0000 7f49458f1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4940101fc0 0x7f494019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.310+0000 7f49458f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4940102980 0x7f494019f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.310+0000 7f49458f1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f494010c9a0 con 0x7f4940100e70 2024-04-21T08:36:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.310+0000 7f49458f1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f494010c820 con 0x7f4940102980 2024-04-21T08:36:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.310+0000 7f49458f1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f494010cb20 con 0x7f4940101fc0 2024-04-21T08:36:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.311+0000 7f493effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4940100e70 0x7f4940069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.311+0000 7f493effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4940100e70 0x7f4940069c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42146/0 (socket says 172.21.15.43:42146) 2024-04-21T08:36:13.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.311+0000 7f493effd640 1 -- 172.21.15.43:0/1525147125 learned_addr learned my addr 172.21.15.43:0/1525147125 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:13.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.311+0000 7f493e7fc640 1 --2- 172.21.15.43:0/1525147125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4940101fc0 0x7f494019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:13.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.311+0000 7f493f7fe640 1 --2- 172.21.15.43:0/1525147125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4940102980 0x7f494019f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:13.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.311+0000 7f493effd640 1 -- 172.21.15.43:0/1525147125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4940101fc0 msgr2=0x7f494019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:13.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.311+0000 7f493effd640 1 --2- 172.21.15.43:0/1525147125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4940101fc0 0x7f494019ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:13.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.311+0000 7f493effd640 1 -- 172.21.15.43:0/1525147125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4940102980 msgr2=0x7f494019f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:13.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.311+0000 7f493effd640 1 --2- 172.21.15.43:0/1525147125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4940102980 0x7f494019f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:13.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.311+0000 7f493effd640 1 -- 172.21.15.43:0/1525147125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494019fd70 con 0x7f4940100e70 2024-04-21T08:36:13.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.311+0000 7f493f7fe640 1 --2- 172.21.15.43:0/1525147125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4940102980 0x7f494019f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:13.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.311+0000 7f493effd640 1 --2- 172.21.15.43:0/1525147125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4940100e70 0x7f4940069c70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f493000cce0 tx=0x7f493000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:13.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.311+0000 7f49448ef640 1 -- 172.21.15.43:0/1525147125 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493000f650 con 0x7f4940100e70 2024-04-21T08:36:13.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.311+0000 7f49458f1640 1 -- 172.21.15.43:0/1525147125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49401a0060 con 0x7f4940100e70 2024-04-21T08:36:13.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.311+0000 7f49448ef640 1 -- 172.21.15.43:0/1525147125 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f493000fca0 con 0x7f4940100e70 2024-04-21T08:36:13.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.311+0000 7f49448ef640 1 -- 172.21.15.43:0/1525147125 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930018b40 con 0x7f4940100e70 2024-04-21T08:36:13.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.311+0000 7f49458f1640 1 -- 172.21.15.43:0/1525147125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49401a0580 con 0x7f4940100e70 2024-04-21T08:36:13.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.313+0000 7f49458f1640 1 -- 172.21.15.43:0/1525147125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4940068de0 con 0x7f4940100e70 2024-04-21T08:36:13.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.314+0000 7f49448ef640 1 -- 172.21.15.43:0/1525147125 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f493000f7f0 con 0x7f4940100e70 2024-04-21T08:36:13.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.314+0000 7f49448ef640 1 --2- 172.21.15.43:0/1525147125 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f49280a3bb0 0x7f49280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:13.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.315+0000 7f493e7fc640 1 --2- 172.21.15.43:0/1525147125 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f49280a3bb0 0x7f49280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:13.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.315+0000 7f49448ef640 1 -- 172.21.15.43:0/1525147125 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f49300d03e0 con 0x7f4940100e70 2024-04-21T08:36:13.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.315+0000 7f493e7fc640 1 --2- 172.21.15.43:0/1525147125 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f49280a3bb0 0x7f49280a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4934005ab0 tx=0x7f4934005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:13.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.319+0000 7f49448ef640 1 -- 172.21.15.43:0/1525147125 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4930014030 con 0x7f4940100e70 2024-04-21T08:36:13.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.555+0000 7f49458f1640 1 -- 172.21.15.43:0/1525147125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f494002d5c0 con 0x7f4940100e70 2024-04-21T08:36:13.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.556+0000 7f49448ef640 1 -- 172.21.15.43:0/1525147125 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7f493009d260 con 0x7f4940100e70 2024-04-21T08:36:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:13.555 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-21T08:36:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 11 2024-04-21T08:36:13.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.558+0000 7f49458f1640 1 -- 172.21.15.43:0/1525147125 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f49280a3bb0 msgr2=0x7f49280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:13.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.558+0000 7f49458f1640 1 --2- 172.21.15.43:0/1525147125 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f49280a3bb0 0x7f49280a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4934005ab0 tx=0x7f4934005fd0 comp rx=0 tx=0).stop 2024-04-21T08:36:13.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.558+0000 7f49458f1640 1 -- 172.21.15.43:0/1525147125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4940100e70 msgr2=0x7f4940069c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:13.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.558+0000 7f49458f1640 1 --2- 172.21.15.43:0/1525147125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4940100e70 0x7f4940069c70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f493000cce0 tx=0x7f493000a620 comp rx=0 tx=0).stop 2024-04-21T08:36:13.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.558+0000 7f49458f1640 1 -- 172.21.15.43:0/1525147125 shutdown_connections 2024-04-21T08:36:13.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.558+0000 7f49458f1640 1 --2- 172.21.15.43:0/1525147125 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f49280a3bb0 0x7f49280a6070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:13.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.558+0000 7f49458f1640 1 --2- 172.21.15.43:0/1525147125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4940102980 0x7f494019f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:13.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.558+0000 7f49458f1640 1 --2- 172.21.15.43:0/1525147125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4940101fc0 0x7f494019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:13.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.558+0000 7f49458f1640 1 --2- 172.21.15.43:0/1525147125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4940100e70 0x7f4940069c70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:13.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.558+0000 7f49458f1640 1 -- 172.21.15.43:0/1525147125 >> 172.21.15.43:0/1525147125 conn(0x7f49400fc3e0 msgr2=0x7f4940105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:13.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.558+0000 7f49458f1640 1 -- 172.21.15.43:0/1525147125 shutdown_connections 2024-04-21T08:36:13.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:13.558+0000 7f49458f1640 1 -- 172.21.15.43:0/1525147125 wait complete. 2024-04-21T08:36:13.561 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 21 08:36:13 smithi043 podman[67134]: 2024-04-21 08:36:13.17852179 +0000 UTC m=+0.411338029 container start a244b7e4086e783f885905983bb3b213e0c10f87d8068d8f73e30d9d4d2d1e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-21T08:36:13.561 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 21 08:36:13 smithi043 ceph-mds[67153]: -- 172.21.15.43:0/3657973902 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ac918d9a40 con 0x55ac918ac800 2024-04-21T08:36:13.561 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 21 08:36:13 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-g[67149]: starting mds.g at 2024-04-21T08:36:13.561 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 21 08:36:13 smithi043 bash[67134]: a244b7e4086e783f885905983bb3b213e0c10f87d8068d8f73e30d9d4d2d1e2a 2024-04-21T08:36:13.561 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 21 08:36:13 smithi043 systemd[1]: Started Ceph mds.g for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:36:13.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:13 smithi043 ceph-mon[27978]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:13.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:13 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:13.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:13 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:13.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:13 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:13.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:13 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:13.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:13 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:13.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:13 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:13.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:13 smithi175 ceph-mon[31920]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:13.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:13 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:13.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:13 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:13.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:13 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:13.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:13 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:13.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:13 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:13.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:13 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:13.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:13 smithi112 ceph-mon[31085]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:13.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:13 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:13.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:13 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:13.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:13 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:13.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:13 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:13.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:13 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:13.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:13 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:14.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:14 smithi043 ceph-mon[27978]: Deploying daemon mds.a on smithi043 2024-04-21T08:36:14.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:14 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.43:6838/931821965,v1:172.21.15.43:6839/931821965] up:boot 2024-04-21T08:36:14.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:14 smithi043 ceph-mon[27978]: fsmap 10 up:standby 2024-04-21T08:36:14.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:36:14.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:14 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1525147125' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:14.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:14 smithi175 ceph-mon[31920]: Deploying daemon mds.a on smithi043 2024-04-21T08:36:14.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:14 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.43:6838/931821965,v1:172.21.15.43:6839/931821965] up:boot 2024-04-21T08:36:14.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:14 smithi175 ceph-mon[31920]: fsmap 10 up:standby 2024-04-21T08:36:14.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:36:14.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:14 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1525147125' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:14.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:14 smithi112 ceph-mon[31085]: Deploying daemon mds.a on smithi043 2024-04-21T08:36:14.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:14 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.43:6838/931821965,v1:172.21.15.43:6839/931821965] up:boot 2024-04-21T08:36:14.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:14 smithi112 ceph-mon[31085]: fsmap 10 up:standby 2024-04-21T08:36:14.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:36:14.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:14 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1525147125' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:15.570 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:36:15.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.689+0000 7fc1973ad640 1 -- 172.21.15.43:0/2212532235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc190105380 msgr2=0x7fc190107770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:15.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.689+0000 7fc1973ad640 1 --2- 172.21.15.43:0/2212532235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc190105380 0x7fc190107770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc180009900 tx=0x7fc18002bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:15.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.689+0000 7fc1973ad640 1 -- 172.21.15.43:0/2212532235 shutdown_connections 2024-04-21T08:36:15.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.689+0000 7fc1973ad640 1 --2- 172.21.15.43:0/2212532235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc190107cb0 0x7fc19010a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:15.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.689+0000 7fc1973ad640 1 --2- 172.21.15.43:0/2212532235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc190105380 0x7fc190107770 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:15.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.689+0000 7fc1973ad640 1 --2- 172.21.15.43:0/2212532235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc190102a50 0x7fc190104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:15.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.689+0000 7fc1973ad640 1 -- 172.21.15.43:0/2212532235 >> 172.21.15.43:0/2212532235 conn(0x7fc1900fc3e0 msgr2=0x7fc1900fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:15.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.689+0000 7fc1973ad640 1 -- 172.21.15.43:0/2212532235 shutdown_connections 2024-04-21T08:36:15.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.689+0000 7fc1973ad640 1 -- 172.21.15.43:0/2212532235 wait complete. 2024-04-21T08:36:15.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.690+0000 7fc1973ad640 1 Processor -- start 2024-04-21T08:36:15.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.690+0000 7fc1973ad640 1 -- start start 2024-04-21T08:36:15.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.690+0000 7fc1973ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc190102a50 0x7fc190198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:15.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.690+0000 7fc1973ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc190105380 0x7fc190198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:15.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.690+0000 7fc1973ad640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc190107cb0 0x7fc19019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:15.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.690+0000 7fc194948640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc190105380 0x7fc190198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:15.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.690+0000 7fc194948640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc190105380 0x7fc190198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42160/0 (socket says 172.21.15.43:42160) 2024-04-21T08:36:15.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.690+0000 7fc194948640 1 -- 172.21.15.43:0/2574081301 learned_addr learned my addr 172.21.15.43:0/2574081301 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:15.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.690+0000 7fc195149640 1 --2- 172.21.15.43:0/2574081301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc190102a50 0x7fc190198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:15.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.690+0000 7fc1973ad640 1 -- 172.21.15.43:0/2574081301 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc19010daf0 con 0x7fc190105380 2024-04-21T08:36:15.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.690+0000 7fc1973ad640 1 -- 172.21.15.43:0/2574081301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc19010d970 con 0x7fc190102a50 2024-04-21T08:36:15.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.690+0000 7fc1973ad640 1 -- 172.21.15.43:0/2574081301 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc19010dc70 con 0x7fc190107cb0 2024-04-21T08:36:15.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.690+0000 7fc19594a640 1 --2- 172.21.15.43:0/2574081301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc190107cb0 0x7fc19019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:15.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.690+0000 7fc194948640 1 -- 172.21.15.43:0/2574081301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc190107cb0 msgr2=0x7fc19019f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:15.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.690+0000 7fc194948640 1 --2- 172.21.15.43:0/2574081301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc190107cb0 0x7fc19019f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:15.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.690+0000 7fc194948640 1 -- 172.21.15.43:0/2574081301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc190102a50 msgr2=0x7fc190198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.690+0000 7fc194948640 1 --2- 172.21.15.43:0/2574081301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc190102a50 0x7fc190198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.690+0000 7fc194948640 1 -- 172.21.15.43:0/2574081301 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19019fc70 con 0x7fc190105380 2024-04-21T08:36:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.690+0000 7fc19594a640 1 --2- 172.21.15.43:0/2574081301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc190107cb0 0x7fc19019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.690+0000 7fc195149640 1 --2- 172.21.15.43:0/2574081301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc190102a50 0x7fc190198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:36:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.691+0000 7fc194948640 1 --2- 172.21.15.43:0/2574081301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc190105380 0x7fc190198590 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc180036040 tx=0x7fc1800042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.691+0000 7fc1867fc640 1 -- 172.21.15.43:0/2574081301 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180043020 con 0x7fc190105380 2024-04-21T08:36:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.691+0000 7fc1867fc640 1 -- 172.21.15.43:0/2574081301 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1800044c0 con 0x7fc190105380 2024-04-21T08:36:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.691+0000 7fc1973ad640 1 -- 172.21.15.43:0/2574081301 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19019ff00 con 0x7fc190105380 2024-04-21T08:36:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.691+0000 7fc1867fc640 1 -- 172.21.15.43:0/2574081301 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18003e450 con 0x7fc190105380 2024-04-21T08:36:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.691+0000 7fc1973ad640 1 -- 172.21.15.43:0/2574081301 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1901a03e0 con 0x7fc190105380 2024-04-21T08:36:15.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.692+0000 7fc1973ad640 1 -- 172.21.15.43:0/2574081301 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1900fded0 con 0x7fc190105380 2024-04-21T08:36:15.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.693+0000 7fc1867fc640 1 -- 172.21.15.43:0/2574081301 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc180046020 con 0x7fc190105380 2024-04-21T08:36:15.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.694+0000 7fc1867fc640 1 --2- 172.21.15.43:0/2574081301 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc17c0a3bb0 0x7fc17c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:15.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.694+0000 7fc195149640 1 --2- 172.21.15.43:0/2574081301 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc17c0a3bb0 0x7fc17c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:15.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.694+0000 7fc1867fc640 1 -- 172.21.15.43:0/2574081301 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc1800f2530 con 0x7fc190105380 2024-04-21T08:36:15.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.694+0000 7fc195149640 1 --2- 172.21.15.43:0/2574081301 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc17c0a3bb0 0x7fc17c0a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00bfd0 tx=0x7fc18c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:15.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.699+0000 7fc1867fc640 1 -- 172.21.15.43:0/2574081301 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc18004d020 con 0x7fc190105380 2024-04-21T08:36:15.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:15 smithi175 ceph-mon[31920]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:15.788 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:15 smithi043 ceph-mon[27978]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:15.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:15 smithi112 ceph-mon[31085]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:15.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.933+0000 7fc1973ad640 1 -- 172.21.15.43:0/2574081301 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc19002d770 con 0x7fc190105380 2024-04-21T08:36:15.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.933+0000 7fc1867fc640 1 -- 172.21.15.43:0/2574081301 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7fc1800bf3b0 con 0x7fc190105380 2024-04-21T08:36:15.933 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:15.933 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-21T08:36:15.933 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 11 2024-04-21T08:36:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.936+0000 7fc1973ad640 1 -- 172.21.15.43:0/2574081301 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc17c0a3bb0 msgr2=0x7fc17c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.936+0000 7fc1973ad640 1 --2- 172.21.15.43:0/2574081301 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc17c0a3bb0 0x7fc17c0a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00bfd0 tx=0x7fc18c00b390 comp rx=0 tx=0).stop 2024-04-21T08:36:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.936+0000 7fc1973ad640 1 -- 172.21.15.43:0/2574081301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc190105380 msgr2=0x7fc190198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.936+0000 7fc1973ad640 1 --2- 172.21.15.43:0/2574081301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc190105380 0x7fc190198590 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc180036040 tx=0x7fc1800042d0 comp rx=0 tx=0).stop 2024-04-21T08:36:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.936+0000 7fc1973ad640 1 -- 172.21.15.43:0/2574081301 shutdown_connections 2024-04-21T08:36:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.936+0000 7fc1973ad640 1 --2- 172.21.15.43:0/2574081301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc190107cb0 0x7fc19019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.936+0000 7fc1973ad640 1 --2- 172.21.15.43:0/2574081301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc190105380 0x7fc190198590 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.936+0000 7fc1973ad640 1 --2- 172.21.15.43:0/2574081301 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc17c0a3bb0 0x7fc17c0a6070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.936+0000 7fc1973ad640 1 --2- 172.21.15.43:0/2574081301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc190102a50 0x7fc190198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.936+0000 7fc1973ad640 1 -- 172.21.15.43:0/2574081301 >> 172.21.15.43:0/2574081301 conn(0x7fc1900fc3e0 msgr2=0x7fc190103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.937+0000 7fc1973ad640 1 -- 172.21.15.43:0/2574081301 shutdown_connections 2024-04-21T08:36:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:15.937+0000 7fc1973ad640 1 -- 172.21.15.43:0/2574081301 wait complete. 2024-04-21T08:36:16.312 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 21 08:36:16 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-a[67537]: starting mds.a at 2024-04-21T08:36:16.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:16 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2574081301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:16.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:16 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:16.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:16 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:16.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:16 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:16.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:16 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2574081301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:16.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:16 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:16.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:16 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:16.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:16 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:16.811 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 21 08:36:16 smithi043 bash[67502]: 22dc542087dbc3ddf5da4ff364fad092e7c84e78ed3840facbe4ad281af22a1f 2024-04-21T08:36:16.811 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 21 08:36:16 smithi043 systemd[1]: Started Ceph mds.a for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:36:16.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:16 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2574081301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:16.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:16 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:16.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:16 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:16.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:16 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:17.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:17 smithi175 ceph-mon[31920]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:17.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:17 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:17.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:17 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:17.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:17 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:17.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:17 smithi175 ceph-mon[31920]: Deploying daemon mds.b on smithi112 2024-04-21T08:36:17.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:17 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.43:6840/623279035,v1:172.21.15.43:6841/623279035] up:boot 2024-04-21T08:36:17.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:17 smithi175 ceph-mon[31920]: fsmap 11 up:standby 2024-04-21T08:36:17.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:17 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:36:17.761 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:17 smithi112 ceph-mon[31085]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:17.761 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:17 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:17.761 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:17 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:17.761 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:17 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:17.761 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:17 smithi112 ceph-mon[31085]: Deploying daemon mds.b on smithi112 2024-04-21T08:36:17.761 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:17 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.43:6840/623279035,v1:172.21.15.43:6841/623279035] up:boot 2024-04-21T08:36:17.761 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:17 smithi112 ceph-mon[31085]: fsmap 11 up:standby 2024-04-21T08:36:17.762 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:17 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:36:17.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:17 smithi043 ceph-mon[27978]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:17.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:17 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:17.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:17 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:17.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:17 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:17.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:17 smithi043 ceph-mon[27978]: Deploying daemon mds.b on smithi112 2024-04-21T08:36:17.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:17 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.43:6840/623279035,v1:172.21.15.43:6841/623279035] up:boot 2024-04-21T08:36:17.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:17 smithi043 ceph-mon[27978]: fsmap 11 up:standby 2024-04-21T08:36:17.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:17 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:36:17.948 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:36:18.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.065+0000 7f026aaba640 1 -- 172.21.15.43:0/2381795753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0264102a10 msgr2=0x7f0264109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:18.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.065+0000 7f026aaba640 1 --2- 172.21.15.43:0/2381795753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0264102a10 0x7f0264109330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f025c009ca0 tx=0x7f025c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:18.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.065+0000 7f026aaba640 1 -- 172.21.15.43:0/2381795753 shutdown_connections 2024-04-21T08:36:18.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.065+0000 7f026aaba640 1 --2- 172.21.15.43:0/2381795753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0264102a10 0x7f0264109330 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:18.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.065+0000 7f026aaba640 1 --2- 172.21.15.43:0/2381795753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0264102050 0x7f02641024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:18.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.065+0000 7f026aaba640 1 --2- 172.21.15.43:0/2381795753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0264100e70 0x7f0264101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:18.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.065+0000 7f026aaba640 1 -- 172.21.15.43:0/2381795753 >> 172.21.15.43:0/2381795753 conn(0x7f02640fc470 msgr2=0x7f02640fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:18.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.066+0000 7f026aaba640 1 -- 172.21.15.43:0/2381795753 shutdown_connections 2024-04-21T08:36:18.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.066+0000 7f026aaba640 1 -- 172.21.15.43:0/2381795753 wait complete. 2024-04-21T08:36:18.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.066+0000 7f026aaba640 1 Processor -- start 2024-04-21T08:36:18.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.066+0000 7f026aaba640 1 -- start start 2024-04-21T08:36:18.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.066+0000 7f026aaba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0264100e70 0x7f026410b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:18.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.066+0000 7f026aaba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0264102050 0x7f0264109eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:18.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.066+0000 7f026aaba640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0264102a10 0x7f026410a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:18.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.066+0000 7f026aaba640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f026410db40 con 0x7f0264100e70 2024-04-21T08:36:18.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.066+0000 7f026aaba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f026410d9c0 con 0x7f0264102050 2024-04-21T08:36:18.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.066+0000 7f026aaba640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f026410dcc0 con 0x7f0264102a10 2024-04-21T08:36:18.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.066+0000 7f0268856640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0264100e70 0x7f026410b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:18.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.066+0000 7f0268856640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0264100e70 0x7f026410b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58666/0 (socket says 172.21.15.43:58666) 2024-04-21T08:36:18.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.066+0000 7f0268856640 1 -- 172.21.15.43:0/1701394513 learned_addr learned my addr 172.21.15.43:0/1701394513 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:18.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.067+0000 7f0263fff640 1 --2- 172.21.15.43:0/1701394513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0264102050 0x7f0264109eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:18.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.067+0000 7f0269057640 1 --2- 172.21.15.43:0/1701394513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0264102a10 0x7f026410a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:18.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.067+0000 7f0268856640 1 -- 172.21.15.43:0/1701394513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0264102a10 msgr2=0x7f026410a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:18.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.067+0000 7f0268856640 1 --2- 172.21.15.43:0/1701394513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0264102a10 0x7f026410a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:18.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.067+0000 7f0268856640 1 -- 172.21.15.43:0/1701394513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0264102050 msgr2=0x7f0264109eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:18.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.067+0000 7f0268856640 1 --2- 172.21.15.43:0/1701394513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0264102050 0x7f0264109eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:18.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.067+0000 7f0268856640 1 -- 172.21.15.43:0/1701394513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f026410ac80 con 0x7f0264100e70 2024-04-21T08:36:18.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.067+0000 7f0268856640 1 --2- 172.21.15.43:0/1701394513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0264100e70 0x7f026410b870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f025800d770 tx=0x7f025800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:18.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.067+0000 7f0261ffb640 1 -- 172.21.15.43:0/1701394513 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0258017070 con 0x7f0264100e70 2024-04-21T08:36:18.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.067+0000 7f0261ffb640 1 -- 172.21.15.43:0/1701394513 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0258009c30 con 0x7f0264100e70 2024-04-21T08:36:18.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.067+0000 7f026aaba640 1 -- 172.21.15.43:0/1701394513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02641a22f0 con 0x7f0264100e70 2024-04-21T08:36:18.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.067+0000 7f0261ffb640 1 -- 172.21.15.43:0/1701394513 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0258003cc0 con 0x7f0264100e70 2024-04-21T08:36:18.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.067+0000 7f026aaba640 1 -- 172.21.15.43:0/1701394513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02641a2890 con 0x7f0264100e70 2024-04-21T08:36:18.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.068+0000 7f026aaba640 1 -- 172.21.15.43:0/1701394513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02641081b0 con 0x7f0264100e70 2024-04-21T08:36:18.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.069+0000 7f0261ffb640 1 -- 172.21.15.43:0/1701394513 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0258003e60 con 0x7f0264100e70 2024-04-21T08:36:18.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.070+0000 7f0261ffb640 1 --2- 172.21.15.43:0/1701394513 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f02540a3c30 0x7f02540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:18.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.070+0000 7f0263fff640 1 --2- 172.21.15.43:0/1701394513 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f02540a3c30 0x7f02540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:18.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.070+0000 7f0261ffb640 1 -- 172.21.15.43:0/1701394513 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f02580cf920 con 0x7f0264100e70 2024-04-21T08:36:18.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.071+0000 7f0263fff640 1 --2- 172.21.15.43:0/1701394513 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f02540a3c30 0x7f02540a60f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0250005fd0 tx=0x7f0250005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:18.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.075+0000 7f0261ffb640 1 -- 172.21.15.43:0/1701394513 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0258009de0 con 0x7f0264100e70 2024-04-21T08:36:18.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.307+0000 7f026aaba640 1 -- 172.21.15.43:0/1701394513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f026402d5c0 con 0x7f0264100e70 2024-04-21T08:36:18.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.308+0000 7f0261ffb640 1 -- 172.21.15.43:0/1701394513 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9656 (secure 0 0 0) 0x7f025809c7a0 con 0x7f0264100e70 2024-04-21T08:36:18.307 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:18.307 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-21T08:36:18.307 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 12 2024-04-21T08:36:18.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.310+0000 7f026aaba640 1 -- 172.21.15.43:0/1701394513 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f02540a3c30 msgr2=0x7f02540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.310+0000 7f026aaba640 1 --2- 172.21.15.43:0/1701394513 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f02540a3c30 0x7f02540a60f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0250005fd0 tx=0x7f0250005850 comp rx=0 tx=0).stop 2024-04-21T08:36:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.310+0000 7f026aaba640 1 -- 172.21.15.43:0/1701394513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0264100e70 msgr2=0x7f026410b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.310+0000 7f026aaba640 1 --2- 172.21.15.43:0/1701394513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0264100e70 0x7f026410b870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f025800d770 tx=0x7f025800dc30 comp rx=0 tx=0).stop 2024-04-21T08:36:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.310+0000 7f026aaba640 1 -- 172.21.15.43:0/1701394513 shutdown_connections 2024-04-21T08:36:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.310+0000 7f026aaba640 1 --2- 172.21.15.43:0/1701394513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0264102a10 0x7f026410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.310+0000 7f026aaba640 1 --2- 172.21.15.43:0/1701394513 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f02540a3c30 0x7f02540a60f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.310+0000 7f026aaba640 1 --2- 172.21.15.43:0/1701394513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0264102050 0x7f0264109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.310+0000 7f026aaba640 1 --2- 172.21.15.43:0/1701394513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0264100e70 0x7f026410b870 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.310+0000 7f026aaba640 1 -- 172.21.15.43:0/1701394513 >> 172.21.15.43:0/1701394513 conn(0x7f02640fc470 msgr2=0x7f0264105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.310+0000 7f026aaba640 1 -- 172.21.15.43:0/1701394513 shutdown_connections 2024-04-21T08:36:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:18.310+0000 7f026aaba640 1 -- 172.21.15.43:0/1701394513 wait complete. 2024-04-21T08:36:18.499 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:18 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1701394513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:18.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:18 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1701394513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:18.794 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:36:18 smithi112 systemd[1]: Starting Ceph mds.b for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:36:18.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:18 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1701394513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:19.151 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:36:18 smithi112 podman[61563]: 2024-04-21 08:36:18.89157741 +0000 UTC m=+0.112288314 container create bea88b79092d11d5191c0b8ba200742cf9867263235771011a250f3d37e35dad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-b, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-21T08:36:19.152 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:36:18 smithi112 podman[61563]: 2024-04-21 08:36:18.797397675 +0000 UTC m=+0.018108584 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:36:19.152 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:36:19 smithi112 podman[61563]: 2024-04-21 08:36:19.152733616 +0000 UTC m=+0.373444517 container init bea88b79092d11d5191c0b8ba200742cf9867263235771011a250f3d37e35dad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-21T08:36:19.430 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:36:19 smithi112 podman[61563]: 2024-04-21 08:36:19.15545519 +0000 UTC m=+0.376166102 container start bea88b79092d11d5191c0b8ba200742cf9867263235771011a250f3d37e35dad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-21T08:36:19.431 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:36:19 smithi112 ceph-mds[61582]: -- 172.21.15.112:0/327207580 <== mon.2 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56133a5e3a40 con 0x56133a5b6800 2024-04-21T08:36:19.431 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:36:19 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-b[61578]: starting mds.b at 2024-04-21T08:36:19.431 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:36:19 smithi112 bash[61563]: bea88b79092d11d5191c0b8ba200742cf9867263235771011a250f3d37e35dad 2024-04-21T08:36:19.431 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:36:19 smithi112 systemd[1]: Started Ceph mds.b for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:36:19.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:19 smithi175 ceph-mon[31920]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:19.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:19 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:19.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:19 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:19.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:19 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:19.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:19 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:19.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:19 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:19.793 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:19 smithi112 ceph-mon[31085]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:19.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:19 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:19.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:19 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:19.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:19 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:19.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:19 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:19.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:19 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:19.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:19 smithi043 ceph-mon[27978]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:19.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:19 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:19.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:19 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:19.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:19 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:19.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:19 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:19.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:19 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:20.321 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:36:20.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.427+0000 7f847f1a4640 1 -- 172.21.15.43:0/827862492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8478103ab0 msgr2=0x7f847810a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:20.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.427+0000 7f847f1a4640 1 --2- 172.21.15.43:0/827862492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8478103ab0 0x7f847810a3d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8470009ca0 tx=0x7f847002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.427+0000 7f847f1a4640 1 -- 172.21.15.43:0/827862492 shutdown_connections 2024-04-21T08:36:20.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.427+0000 7f847f1a4640 1 --2- 172.21.15.43:0/827862492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8478103ab0 0x7f847810a3d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.427+0000 7f847f1a4640 1 --2- 172.21.15.43:0/827862492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84781030f0 0x7f8478103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.427+0000 7f847f1a4640 1 --2- 172.21.15.43:0/827862492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8478101f20 0x7f8478102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.427+0000 7f847f1a4640 1 -- 172.21.15.43:0/827862492 >> 172.21.15.43:0/827862492 conn(0x7f84780fd500 msgr2=0x7f84780ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:20.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.427+0000 7f847f1a4640 1 -- 172.21.15.43:0/827862492 shutdown_connections 2024-04-21T08:36:20.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.427+0000 7f847f1a4640 1 -- 172.21.15.43:0/827862492 wait complete. 2024-04-21T08:36:20.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.428+0000 7f847f1a4640 1 Processor -- start 2024-04-21T08:36:20.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.428+0000 7f847f1a4640 1 -- start start 2024-04-21T08:36:20.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.428+0000 7f847f1a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8478101f20 0x7f8478075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:20.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.428+0000 7f847f1a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84781030f0 0x7f8478074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:20.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.428+0000 7f847f1a4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8478103ab0 0x7f8478074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:20.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.428+0000 7f847f1a4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f847810daa0 con 0x7f84781030f0 2024-04-21T08:36:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.428+0000 7f847f1a4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f847810d920 con 0x7f8478101f20 2024-04-21T08:36:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.428+0000 7f847f1a4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f847810dc20 con 0x7f8478103ab0 2024-04-21T08:36:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.428+0000 7f846ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84781030f0 0x7f8478074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.428+0000 7f846ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84781030f0 0x7f8478074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58682/0 (socket says 172.21.15.43:58682) 2024-04-21T08:36:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.428+0000 7f846ffff640 1 -- 172.21.15.43:0/955179033 learned_addr learned my addr 172.21.15.43:0/955179033 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.428+0000 7f847cf40640 1 --2- 172.21.15.43:0/955179033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8478101f20 0x7f8478075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.428+0000 7f847d741640 1 --2- 172.21.15.43:0/955179033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8478103ab0 0x7f8478074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.428+0000 7f846ffff640 1 -- 172.21.15.43:0/955179033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8478103ab0 msgr2=0x7f8478074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.428+0000 7f846ffff640 1 --2- 172.21.15.43:0/955179033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8478103ab0 0x7f8478074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.428+0000 7f846ffff640 1 -- 172.21.15.43:0/955179033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8478101f20 msgr2=0x7f8478075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.428+0000 7f846ffff640 1 --2- 172.21.15.43:0/955179033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8478101f20 0x7f8478075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.428+0000 7f846ffff640 1 -- 172.21.15.43:0/955179033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8478075110 con 0x7f84781030f0 2024-04-21T08:36:20.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.429+0000 7f846ffff640 1 --2- 172.21.15.43:0/955179033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84781030f0 0x7f8478074340 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f846800e9f0 tx=0x7f846800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:20.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.429+0000 7f846dffb640 1 -- 172.21.15.43:0/955179033 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846800cd80 con 0x7f84781030f0 2024-04-21T08:36:20.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.429+0000 7f847f1a4640 1 -- 172.21.15.43:0/955179033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84781a2260 con 0x7f84781030f0 2024-04-21T08:36:20.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.429+0000 7f846dffb640 1 -- 172.21.15.43:0/955179033 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84680054d0 con 0x7f84781030f0 2024-04-21T08:36:20.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.429+0000 7f846dffb640 1 -- 172.21.15.43:0/955179033 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468010690 con 0x7f84781030f0 2024-04-21T08:36:20.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.429+0000 7f847f1a4640 1 -- 172.21.15.43:0/955179033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84781a27b0 con 0x7f84781030f0 2024-04-21T08:36:20.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.430+0000 7f847f1a4640 1 -- 172.21.15.43:0/955179033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84781093d0 con 0x7f84781030f0 2024-04-21T08:36:20.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.431+0000 7f846dffb640 1 -- 172.21.15.43:0/955179033 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8468004e90 con 0x7f84781030f0 2024-04-21T08:36:20.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.432+0000 7f846dffb640 1 --2- 172.21.15.43:0/955179033 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f84600a3bb0 0x7f84600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:20.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.432+0000 7f847cf40640 1 --2- 172.21.15.43:0/955179033 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f84600a3bb0 0x7f84600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:20.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.432+0000 7f846dffb640 1 -- 172.21.15.43:0/955179033 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f84680d0360 con 0x7f84781030f0 2024-04-21T08:36:20.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.433+0000 7f847cf40640 1 --2- 172.21.15.43:0/955179033 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f84600a3bb0 0x7f84600a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8474005fd0 tx=0x7f847400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:20.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.437+0000 7f846dffb640 1 -- 172.21.15.43:0/955179033 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f846809d1e0 con 0x7f84781030f0 2024-04-21T08:36:20.596 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:20 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] up:boot 2024-04-21T08:36:20.597 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:20 smithi043 ceph-mon[27978]: fsmap 12 up:standby 2024-04-21T08:36:20.597 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:20 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:36:20.597 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:20 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:36:20.612 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:20 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] up:boot 2024-04-21T08:36:20.613 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:20 smithi112 ceph-mon[31085]: fsmap 12 up:standby 2024-04-21T08:36:20.613 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:20 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:36:20.613 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:20 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:36:20.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.669+0000 7f847f1a4640 1 -- 172.21.15.43:0/955179033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f847802d7c0 con 0x7f84781030f0 2024-04-21T08:36:20.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.669+0000 7f846dffb640 1 -- 172.21.15.43:0/955179033 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10485 (secure 0 0 0) 0x7f846809cae0 con 0x7f84781030f0 2024-04-21T08:36:20.669 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:20.669 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-21T08:36:20.669 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 13 2024-04-21T08:36:20.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.671+0000 7f847f1a4640 1 -- 172.21.15.43:0/955179033 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f84600a3bb0 msgr2=0x7f84600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:20.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.671+0000 7f847f1a4640 1 --2- 172.21.15.43:0/955179033 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f84600a3bb0 0x7f84600a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8474005fd0 tx=0x7f847400b410 comp rx=0 tx=0).stop 2024-04-21T08:36:20.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.671+0000 7f847f1a4640 1 -- 172.21.15.43:0/955179033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84781030f0 msgr2=0x7f8478074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:20.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.671+0000 7f847f1a4640 1 --2- 172.21.15.43:0/955179033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84781030f0 0x7f8478074340 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f846800e9f0 tx=0x7f846800edf0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.671+0000 7f847f1a4640 1 -- 172.21.15.43:0/955179033 shutdown_connections 2024-04-21T08:36:20.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.671+0000 7f847f1a4640 1 --2- 172.21.15.43:0/955179033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8478103ab0 0x7f8478074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.671+0000 7f847f1a4640 1 --2- 172.21.15.43:0/955179033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84781030f0 0x7f8478074340 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.671+0000 7f847f1a4640 1 --2- 172.21.15.43:0/955179033 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f84600a3bb0 0x7f84600a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.671+0000 7f847f1a4640 1 --2- 172.21.15.43:0/955179033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8478101f20 0x7f8478075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.671+0000 7f847f1a4640 1 -- 172.21.15.43:0/955179033 >> 172.21.15.43:0/955179033 conn(0x7f84780fd500 msgr2=0x7f84781065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:20.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.672+0000 7f847f1a4640 1 -- 172.21.15.43:0/955179033 shutdown_connections 2024-04-21T08:36:20.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.672+0000 7f847f1a4640 1 -- 172.21.15.43:0/955179033 wait complete. 2024-04-21T08:36:20.682 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:36:20.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:20 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] up:boot 2024-04-21T08:36:20.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:20 smithi175 ceph-mon[31920]: fsmap 12 up:standby 2024-04-21T08:36:20.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:20 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:36:20.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:20 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:36:20.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.786+0000 7f161d550640 1 -- 172.21.15.43:0/516150103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1618105370 msgr2=0x7f1618107760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:20.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.786+0000 7f161d550640 1 --2- 172.21.15.43:0/516150103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1618105370 0x7f1618107760 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f160c009900 tx=0x7f160c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.786+0000 7f161d550640 1 -- 172.21.15.43:0/516150103 shutdown_connections 2024-04-21T08:36:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.786+0000 7f161d550640 1 --2- 172.21.15.43:0/516150103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1618107ca0 0x7f161810a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.786+0000 7f161d550640 1 --2- 172.21.15.43:0/516150103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1618105370 0x7f1618107760 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.786+0000 7f161d550640 1 --2- 172.21.15.43:0/516150103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1618102a40 0x7f1618104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.786+0000 7f161d550640 1 -- 172.21.15.43:0/516150103 >> 172.21.15.43:0/516150103 conn(0x7f16180fc470 msgr2=0x7f16180fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.786+0000 7f161d550640 1 -- 172.21.15.43:0/516150103 shutdown_connections 2024-04-21T08:36:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.786+0000 7f161d550640 1 -- 172.21.15.43:0/516150103 wait complete. 2024-04-21T08:36:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.787+0000 7f161d550640 1 Processor -- start 2024-04-21T08:36:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.787+0000 7f161d550640 1 -- start start 2024-04-21T08:36:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.787+0000 7f161d550640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1618102a40 0x7f16181981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.787+0000 7f161d550640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1618105370 0x7f1618198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.787+0000 7f161d550640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1618107ca0 0x7f161819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.787+0000 7f161d550640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f161810db40 con 0x7f1618102a40 2024-04-21T08:36:20.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.787+0000 7f161d550640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f161810d9c0 con 0x7f1618107ca0 2024-04-21T08:36:20.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.787+0000 7f161d550640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f161810dcc0 con 0x7f1618105370 2024-04-21T08:36:20.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.787+0000 7f1616ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1618102a40 0x7f16181981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:20.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.787+0000 7f1616ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1618102a40 0x7f16181981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58708/0 (socket says 172.21.15.43:58708) 2024-04-21T08:36:20.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.787+0000 7f1616ffd640 1 -- 172.21.15.43:0/2935833375 learned_addr learned my addr 172.21.15.43:0/2935833375 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:20.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.787+0000 7f16177fe640 1 --2- 172.21.15.43:0/2935833375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1618107ca0 0x7f161819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:20.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.787+0000 7f16167fc640 1 --2- 172.21.15.43:0/2935833375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1618105370 0x7f1618198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:20.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.787+0000 7f1616ffd640 1 -- 172.21.15.43:0/2935833375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1618105370 msgr2=0x7f1618198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:20.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.787+0000 7f1616ffd640 1 --2- 172.21.15.43:0/2935833375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1618105370 0x7f1618198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.787+0000 7f1616ffd640 1 -- 172.21.15.43:0/2935833375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1618107ca0 msgr2=0x7f161819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:20.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.787+0000 7f1616ffd640 1 --2- 172.21.15.43:0/2935833375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1618107ca0 0x7f161819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.787+0000 7f1616ffd640 1 -- 172.21.15.43:0/2935833375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161819fe00 con 0x7f1618102a40 2024-04-21T08:36:20.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.788+0000 7f1616ffd640 1 --2- 172.21.15.43:0/2935833375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1618102a40 0x7f16181981e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f161000e9f0 tx=0x7f161000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:20.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.788+0000 7f15fffff640 1 -- 172.21.15.43:0/2935833375 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161000cd80 con 0x7f1618102a40 2024-04-21T08:36:20.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.788+0000 7f15fffff640 1 -- 172.21.15.43:0/2935833375 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16100108e0 con 0x7f1618102a40 2024-04-21T08:36:20.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.788+0000 7f161d550640 1 -- 172.21.15.43:0/2935833375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16181a00f0 con 0x7f1618102a40 2024-04-21T08:36:20.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.788+0000 7f15fffff640 1 -- 172.21.15.43:0/2935833375 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1610018980 con 0x7f1618102a40 2024-04-21T08:36:20.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.788+0000 7f161d550640 1 -- 172.21.15.43:0/2935833375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16181a0690 con 0x7f1618102a40 2024-04-21T08:36:20.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.789+0000 7f161d550640 1 -- 172.21.15.43:0/2935833375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16180fdf60 con 0x7f1618102a40 2024-04-21T08:36:20.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.790+0000 7f15fffff640 1 -- 172.21.15.43:0/2935833375 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1610010430 con 0x7f1618102a40 2024-04-21T08:36:20.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.791+0000 7f15fffff640 1 --2- 172.21.15.43:0/2935833375 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f16000a3bb0 0x7f16000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:20.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.791+0000 7f16167fc640 1 --2- 172.21.15.43:0/2935833375 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f16000a3bb0 0x7f16000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:20.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.791+0000 7f15fffff640 1 -- 172.21.15.43:0/2935833375 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f16100d02e0 con 0x7f1618102a40 2024-04-21T08:36:20.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.791+0000 7f16167fc640 1 --2- 172.21.15.43:0/2935833375 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f16000a3bb0 0x7f16000a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f160c006010 tx=0x7f160c005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:20.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:20.796+0000 7f15fffff640 1 -- 172.21.15.43:0/2935833375 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f161009d160 con 0x7f1618102a40 2024-04-21T08:36:21.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.007+0000 7f161d550640 1 -- 172.21.15.43:0/2935833375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1618062b50 con 0x7f1618102a40 2024-04-21T08:36:21.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.009+0000 7f15fffff640 1 -- 172.21.15.43:0/2935833375 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18480 (secure 0 0 0) 0x7f161009ca60 con 0x7f1618102a40 2024-04-21T08:36:21.008 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:21.009 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":82,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:35:33.554402+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:36:21.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.011+0000 7f161d550640 1 -- 172.21.15.43:0/2935833375 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f16000a3bb0 msgr2=0x7f16000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:21.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.011+0000 7f161d550640 1 --2- 172.21.15.43:0/2935833375 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f16000a3bb0 0x7f16000a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f160c006010 tx=0x7f160c005be0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.011+0000 7f161d550640 1 -- 172.21.15.43:0/2935833375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1618102a40 msgr2=0x7f16181981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:21.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.011+0000 7f161d550640 1 --2- 172.21.15.43:0/2935833375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1618102a40 0x7f16181981e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f161000e9f0 tx=0x7f161000eeb0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.011+0000 7f161d550640 1 -- 172.21.15.43:0/2935833375 shutdown_connections 2024-04-21T08:36:21.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.011+0000 7f161d550640 1 --2- 172.21.15.43:0/2935833375 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f16000a3bb0 0x7f16000a6070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.011+0000 7f161d550640 1 --2- 172.21.15.43:0/2935833375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1618107ca0 0x7f161819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.011+0000 7f161d550640 1 --2- 172.21.15.43:0/2935833375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1618105370 0x7f1618198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.011+0000 7f161d550640 1 --2- 172.21.15.43:0/2935833375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1618102a40 0x7f16181981e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.011+0000 7f161d550640 1 -- 172.21.15.43:0/2935833375 >> 172.21.15.43:0/2935833375 conn(0x7f16180fc470 msgr2=0x7f1618103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:21.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.011+0000 7f161d550640 1 -- 172.21.15.43:0/2935833375 shutdown_connections 2024-04-21T08:36:21.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.011+0000 7f161d550640 1 -- 172.21.15.43:0/2935833375 wait complete. 2024-04-21T08:36:21.022 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:39.981405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-21T08:36:21.023 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:36:21.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.145+0000 7f8a93843640 1 -- 172.21.15.43:0/3577662412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a8c100610 msgr2=0x7f8a8c100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:21.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.145+0000 7f8a93843640 1 --2- 172.21.15.43:0/3577662412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a8c100610 0x7f8a8c100a30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800aa10 tx=0x7f8a8802cf70 comp rx=0 tx=0).stop 2024-04-21T08:36:21.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.145+0000 7f8a93843640 1 -- 172.21.15.43:0/3577662412 shutdown_connections 2024-04-21T08:36:21.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.145+0000 7f8a93843640 1 --2- 172.21.15.43:0/3577662412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a8c1021c0 0x7f8a8c108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.145+0000 7f8a93843640 1 --2- 172.21.15.43:0/3577662412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a8c101800 0x7f8a8c101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.145+0000 7f8a93843640 1 --2- 172.21.15.43:0/3577662412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a8c100610 0x7f8a8c100a30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.145+0000 7f8a93843640 1 -- 172.21.15.43:0/3577662412 >> 172.21.15.43:0/3577662412 conn(0x7f8a8c0fbb80 msgr2=0x7f8a8c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:21.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.145+0000 7f8a93843640 1 -- 172.21.15.43:0/3577662412 shutdown_connections 2024-04-21T08:36:21.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.145+0000 7f8a93843640 1 -- 172.21.15.43:0/3577662412 wait complete. 2024-04-21T08:36:21.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.146+0000 7f8a93843640 1 Processor -- start 2024-04-21T08:36:21.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.146+0000 7f8a93843640 1 -- start start 2024-04-21T08:36:21.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.146+0000 7f8a93843640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a8c100610 0x7f8a8c10b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:21.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.146+0000 7f8a93843640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a8c101800 0x7f8a8c109e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.146+0000 7f8a93843640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a8c1021c0 0x7f8a8c10a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.146+0000 7f8a93843640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c10dae0 con 0x7f8a8c100610 2024-04-21T08:36:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.146+0000 7f8a93843640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c10d960 con 0x7f8a8c1021c0 2024-04-21T08:36:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.146+0000 7f8a93843640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c10dc60 con 0x7f8a8c101800 2024-04-21T08:36:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.146+0000 7f8a915df640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a8c100610 0x7f8a8c10b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.146+0000 7f8a915df640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a8c100610 0x7f8a8c10b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58728/0 (socket says 172.21.15.43:58728) 2024-04-21T08:36:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.146+0000 7f8a915df640 1 -- 172.21.15.43:0/77945634 learned_addr learned my addr 172.21.15.43:0/77945634 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.146+0000 7f8a90dde640 1 --2- 172.21.15.43:0/77945634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a8c101800 0x7f8a8c109e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.146+0000 7f8a91de0640 1 --2- 172.21.15.43:0/77945634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a8c1021c0 0x7f8a8c10a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.146+0000 7f8a915df640 1 -- 172.21.15.43:0/77945634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a8c101800 msgr2=0x7f8a8c109e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.146+0000 7f8a915df640 1 --2- 172.21.15.43:0/77945634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a8c101800 0x7f8a8c109e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.146+0000 7f8a915df640 1 -- 172.21.15.43:0/77945634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a8c1021c0 msgr2=0x7f8a8c10a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:21.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.146+0000 7f8a915df640 1 --2- 172.21.15.43:0/77945634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a8c1021c0 0x7f8a8c10a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.146+0000 7f8a915df640 1 -- 172.21.15.43:0/77945634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a8c10ac20 con 0x7f8a8c100610 2024-04-21T08:36:21.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.146+0000 7f8a915df640 1 --2- 172.21.15.43:0/77945634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a8c100610 0x7f8a8c10b810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8a88000c00 tx=0x7f8a880048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:21.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.147+0000 7f8a827fc640 1 -- 172.21.15.43:0/77945634 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88042020 con 0x7f8a8c100610 2024-04-21T08:36:21.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.147+0000 7f8a827fc640 1 -- 172.21.15.43:0/77945634 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a88035910 con 0x7f8a8c100610 2024-04-21T08:36:21.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.147+0000 7f8a93843640 1 -- 172.21.15.43:0/77945634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a8c1a2340 con 0x7f8a8c100610 2024-04-21T08:36:21.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.147+0000 7f8a827fc640 1 -- 172.21.15.43:0/77945634 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88003850 con 0x7f8a8c100610 2024-04-21T08:36:21.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.147+0000 7f8a93843640 1 -- 172.21.15.43:0/77945634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a8c1a28e0 con 0x7f8a8c100610 2024-04-21T08:36:21.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.148+0000 7f8a93843640 1 -- 172.21.15.43:0/77945634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a8c107a20 con 0x7f8a8c100610 2024-04-21T08:36:21.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.149+0000 7f8a827fc640 1 -- 172.21.15.43:0/77945634 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8a88045020 con 0x7f8a8c100610 2024-04-21T08:36:21.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.150+0000 7f8a827fc640 1 --2- 172.21.15.43:0/77945634 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8a780a3c30 0x7f8a780a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:21.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.150+0000 7f8a90dde640 1 --2- 172.21.15.43:0/77945634 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8a780a3c30 0x7f8a780a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:21.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.150+0000 7f8a827fc640 1 -- 172.21.15.43:0/77945634 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8a880f22e0 con 0x7f8a8c100610 2024-04-21T08:36:21.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.150+0000 7f8a90dde640 1 --2- 172.21.15.43:0/77945634 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8a780a3c30 0x7f8a780a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c10b580 tx=0x7f8a7c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:21.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.155+0000 7f8a827fc640 1 -- 172.21.15.43:0/77945634 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8a8804c020 con 0x7f8a8c100610 2024-04-21T08:36:21.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.381+0000 7f8a93843640 1 -- 172.21.15.43:0/77945634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8a8c062b50 con 0x7f8a8c100610 2024-04-21T08:36:21.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.381+0000 7f8a827fc640 1 -- 172.21.15.43:0/77945634 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f8a880bf160 con 0x7f8a8c100610 2024-04-21T08:36:21.381 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-21T08:36:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.384+0000 7f8a93843640 1 -- 172.21.15.43:0/77945634 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8a780a3c30 msgr2=0x7f8a780a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.384+0000 7f8a93843640 1 --2- 172.21.15.43:0/77945634 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8a780a3c30 0x7f8a780a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c10b580 tx=0x7f8a7c005f90 comp rx=0 tx=0).stop 2024-04-21T08:36:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.384+0000 7f8a93843640 1 -- 172.21.15.43:0/77945634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a8c100610 msgr2=0x7f8a8c10b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.384+0000 7f8a93843640 1 --2- 172.21.15.43:0/77945634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a8c100610 0x7f8a8c10b810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8a88000c00 tx=0x7f8a880048f0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.384+0000 7f8a93843640 1 -- 172.21.15.43:0/77945634 shutdown_connections 2024-04-21T08:36:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.384+0000 7f8a93843640 1 --2- 172.21.15.43:0/77945634 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8a780a3c30 0x7f8a780a60f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.384+0000 7f8a93843640 1 --2- 172.21.15.43:0/77945634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a8c1021c0 0x7f8a8c10a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.384+0000 7f8a93843640 1 --2- 172.21.15.43:0/77945634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a8c101800 0x7f8a8c109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.384+0000 7f8a93843640 1 --2- 172.21.15.43:0/77945634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a8c100610 0x7f8a8c10b810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.384+0000 7f8a93843640 1 -- 172.21.15.43:0/77945634 >> 172.21.15.43:0/77945634 conn(0x7f8a8c0fbb80 msgr2=0x7f8a8c104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.384+0000 7f8a93843640 1 -- 172.21.15.43:0/77945634 shutdown_connections 2024-04-21T08:36:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.384+0000 7f8a93843640 1 -- 172.21.15.43:0/77945634 wait complete. 2024-04-21T08:36:21.394 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-21T08:36:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.500+0000 7ff27ea26640 1 -- 172.21.15.43:0/3635202975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff278103cc0 msgr2=0x7ff27810a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.500+0000 7ff27ea26640 1 --2- 172.21.15.43:0/3635202975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff278103cc0 0x7ff27810a550 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff270009cd0 tx=0x7ff27002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.500+0000 7ff27ea26640 1 -- 172.21.15.43:0/3635202975 shutdown_connections 2024-04-21T08:36:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.500+0000 7ff27ea26640 1 --2- 172.21.15.43:0/3635202975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff278103cc0 0x7ff27810a550 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.500+0000 7ff27ea26640 1 --2- 172.21.15.43:0/3635202975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff278103300 0x7ff278103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.500+0000 7ff27ea26640 1 --2- 172.21.15.43:0/3635202975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff278100290 0x7ff2781006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.500+0000 7ff27ea26640 1 -- 172.21.15.43:0/3635202975 >> 172.21.15.43:0/3635202975 conn(0x7ff2780fbbc0 msgr2=0x7ff2780fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.500+0000 7ff27ea26640 1 -- 172.21.15.43:0/3635202975 shutdown_connections 2024-04-21T08:36:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.500+0000 7ff27ea26640 1 -- 172.21.15.43:0/3635202975 wait complete. 2024-04-21T08:36:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.501+0000 7ff27ea26640 1 Processor -- start 2024-04-21T08:36:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.501+0000 7ff27ea26640 1 -- start start 2024-04-21T08:36:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.501+0000 7ff27ea26640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff278100290 0x7ff278198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.501+0000 7ff27ea26640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff278103300 0x7ff278198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.501+0000 7ff27ea26640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff278103cc0 0x7ff27819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.501+0000 7ff27ea26640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff27810dac0 con 0x7ff278103300 2024-04-21T08:36:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.501+0000 7ff27ea26640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff27810d940 con 0x7ff278103cc0 2024-04-21T08:36:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.501+0000 7ff27ea26640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff27810dc40 con 0x7ff278100290 2024-04-21T08:36:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.501+0000 7ff2777fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff278103300 0x7ff278198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.501+0000 7ff2777fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff278103300 0x7ff278198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58738/0 (socket says 172.21.15.43:58738) 2024-04-21T08:36:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.501+0000 7ff2777fe640 1 -- 172.21.15.43:0/1841595600 learned_addr learned my addr 172.21.15.43:0/1841595600 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.501+0000 7ff27cfc3640 1 --2- 172.21.15.43:0/1841595600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff278103cc0 0x7ff27819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.501+0000 7ff277fff640 1 --2- 172.21.15.43:0/1841595600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff278100290 0x7ff278198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.502+0000 7ff2777fe640 1 -- 172.21.15.43:0/1841595600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff278100290 msgr2=0x7ff278198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.502+0000 7ff2777fe640 1 --2- 172.21.15.43:0/1841595600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff278100290 0x7ff278198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.502+0000 7ff2777fe640 1 -- 172.21.15.43:0/1841595600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff278103cc0 msgr2=0x7ff27819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.502+0000 7ff2777fe640 1 --2- 172.21.15.43:0/1841595600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff278103cc0 0x7ff27819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.502+0000 7ff2777fe640 1 -- 172.21.15.43:0/1841595600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27819fd50 con 0x7ff278103300 2024-04-21T08:36:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.502+0000 7ff2777fe640 1 --2- 172.21.15.43:0/1841595600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff278103300 0x7ff278198670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff26800e9f0 tx=0x7ff26800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.502+0000 7ff2757fa640 1 -- 172.21.15.43:0/1841595600 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26800cd80 con 0x7ff278103300 2024-04-21T08:36:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.502+0000 7ff2757fa640 1 -- 172.21.15.43:0/1841595600 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2680108e0 con 0x7ff278103300 2024-04-21T08:36:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.502+0000 7ff27ea26640 1 -- 172.21.15.43:0/1841595600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff27819ffe0 con 0x7ff278103300 2024-04-21T08:36:21.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.502+0000 7ff2757fa640 1 -- 172.21.15.43:0/1841595600 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268018980 con 0x7ff278103300 2024-04-21T08:36:21.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.502+0000 7ff27ea26640 1 -- 172.21.15.43:0/1841595600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2781a0500 con 0x7ff278103300 2024-04-21T08:36:21.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.503+0000 7ff27ea26640 1 -- 172.21.15.43:0/1841595600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff278109430 con 0x7ff278103300 2024-04-21T08:36:21.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.504+0000 7ff2757fa640 1 -- 172.21.15.43:0/1841595600 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff268010430 con 0x7ff278103300 2024-04-21T08:36:21.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.505+0000 7ff2757fa640 1 --2- 172.21.15.43:0/1841595600 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ff2640a3bb0 0x7ff2640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:21.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.505+0000 7ff277fff640 1 --2- 172.21.15.43:0/1841595600 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ff2640a3bb0 0x7ff2640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:21.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.505+0000 7ff2757fa640 1 -- 172.21.15.43:0/1841595600 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff2680d06e0 con 0x7ff278103300 2024-04-21T08:36:21.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.505+0000 7ff277fff640 1 --2- 172.21.15.43:0/1841595600 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ff2640a3bb0 0x7ff2640a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff27806a330 tx=0x7ff260005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:21.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.508+0000 7ff2757fa640 1 -- 172.21.15.43:0/1841595600 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff26809cf10 con 0x7ff278103300 2024-04-21T08:36:21.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.706+0000 7ff27ea26640 1 -- 172.21.15.43:0/1841595600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff27802d810 con 0x7ff278103300 2024-04-21T08:36:21.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.707+0000 7ff2757fa640 1 -- 172.21.15.43:0/1841595600 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7ff2680a1210 con 0x7ff278103300 2024-04-21T08:36:21.706 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:21.706 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-04-21T08:36:21.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:36:21.706 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-04-21T08:36:21.706 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-04-21T08:36:21.706 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:36:21.707 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-04-21T08:36:21.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.709+0000 7ff27ea26640 1 -- 172.21.15.43:0/1841595600 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ff2640a3bb0 msgr2=0x7ff2640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:21.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.709+0000 7ff27ea26640 1 --2- 172.21.15.43:0/1841595600 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ff2640a3bb0 0x7ff2640a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff27806a330 tx=0x7ff260005f90 comp rx=0 tx=0).stop 2024-04-21T08:36:21.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.709+0000 7ff27ea26640 1 -- 172.21.15.43:0/1841595600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff278103300 msgr2=0x7ff278198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:21.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.709+0000 7ff27ea26640 1 --2- 172.21.15.43:0/1841595600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff278103300 0x7ff278198670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff26800e9f0 tx=0x7ff26800eeb0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.709+0000 7ff27ea26640 1 -- 172.21.15.43:0/1841595600 shutdown_connections 2024-04-21T08:36:21.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.709+0000 7ff27ea26640 1 --2- 172.21.15.43:0/1841595600 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ff2640a3bb0 0x7ff2640a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.709+0000 7ff27ea26640 1 --2- 172.21.15.43:0/1841595600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff278103cc0 0x7ff27819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.709+0000 7ff27ea26640 1 --2- 172.21.15.43:0/1841595600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff278103300 0x7ff278198670 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.709+0000 7ff27ea26640 1 --2- 172.21.15.43:0/1841595600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff278100290 0x7ff278198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.709+0000 7ff27ea26640 1 -- 172.21.15.43:0/1841595600 >> 172.21.15.43:0/1841595600 conn(0x7ff2780fbbc0 msgr2=0x7ff278106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:21.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.709+0000 7ff27ea26640 1 -- 172.21.15.43:0/1841595600 shutdown_connections 2024-04-21T08:36:21.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.709+0000 7ff27ea26640 1 -- 172.21.15.43:0/1841595600 wait complete. 2024-04-21T08:36:21.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:21 smithi175 ceph-mon[31920]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:21.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:21 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/955179033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:21.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:21 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2935833375' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:21.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:21 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:21.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:21 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:21.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:21 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:21.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:21 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:21.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:21 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/77945634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:36:21.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:21 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:21.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:21 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:21.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:21 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:21.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:21 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:36:21.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:21 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:21.718 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:21 smithi043 ceph-mon[27978]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:21.718 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:21 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/955179033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:21.718 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:21 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2935833375' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:21.719 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:21 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:21.719 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:21 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:21.719 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:21 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:21.719 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:21 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:21.719 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:21 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/77945634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:36:21.719 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:21 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:21.719 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:21 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:21.719 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:21 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:21.719 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:21 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:36:21.719 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:21 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:21.720 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-21T08:36:21.720 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-21T08:36:21.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:21 smithi112 ceph-mon[31085]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:21.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:21 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/955179033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:21.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:21 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2935833375' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:21.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:21 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:21.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:21 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:21.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:21 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:21.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:21 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:21.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:21 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/77945634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:36:21.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:21 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:21.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:21 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:21.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:21 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:21.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:21 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:36:21.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:21 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:21.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.828+0000 7f7284ac7640 1 -- 172.21.15.43:0/1738625416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7280102a70 msgr2=0x7f7280104e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:21.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.828+0000 7f7284ac7640 1 --2- 172.21.15.43:0/1738625416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7280102a70 0x7f7280104e60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7268009900 tx=0x7f726802bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:21.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.828+0000 7f7284ac7640 1 -- 172.21.15.43:0/1738625416 shutdown_connections 2024-04-21T08:36:21.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.828+0000 7f7284ac7640 1 --2- 172.21.15.43:0/1738625416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7280107cd0 0x7f728010a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.828+0000 7f7284ac7640 1 --2- 172.21.15.43:0/1738625416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72801053a0 0x7f7280107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.828+0000 7f7284ac7640 1 --2- 172.21.15.43:0/1738625416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7280102a70 0x7f7280104e60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.828+0000 7f7284ac7640 1 -- 172.21.15.43:0/1738625416 >> 172.21.15.43:0/1738625416 conn(0x7f72800fc400 msgr2=0x7f72800fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:21.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.828+0000 7f7284ac7640 1 -- 172.21.15.43:0/1738625416 shutdown_connections 2024-04-21T08:36:21.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.828+0000 7f7284ac7640 1 -- 172.21.15.43:0/1738625416 wait complete. 2024-04-21T08:36:21.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.829+0000 7f7284ac7640 1 Processor -- start 2024-04-21T08:36:21.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.829+0000 7f7284ac7640 1 -- start start 2024-04-21T08:36:21.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.829+0000 7f7284ac7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7280102a70 0x7f7280198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:21.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.829+0000 7f7284ac7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72801053a0 0x7f72801987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:21.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.829+0000 7f7284ac7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7280107cd0 0x7f728019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:21.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.829+0000 7f7284ac7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f728010dad0 con 0x7f7280107cd0 2024-04-21T08:36:21.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.829+0000 7f7284ac7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f728010d950 con 0x7f72801053a0 2024-04-21T08:36:21.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.829+0000 7f7284ac7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f728010dc50 con 0x7f7280102a70 2024-04-21T08:36:21.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.829+0000 7f727ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7280107cd0 0x7f728019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:21.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.829+0000 7f727dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72801053a0 0x7f72801987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:21.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.829+0000 7f727ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7280107cd0 0x7f728019f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58762/0 (socket says 172.21.15.43:58762) 2024-04-21T08:36:21.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.829+0000 7f727ed9d640 1 -- 172.21.15.43:0/2837889196 learned_addr learned my addr 172.21.15.43:0/2837889196 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:21.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.829+0000 7f727e59c640 1 --2- 172.21.15.43:0/2837889196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7280102a70 0x7f7280198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:21.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.830+0000 7f727ed9d640 1 -- 172.21.15.43:0/2837889196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7280102a70 msgr2=0x7f7280198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:21.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.830+0000 7f727ed9d640 1 --2- 172.21.15.43:0/2837889196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7280102a70 0x7f7280198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.830+0000 7f727ed9d640 1 -- 172.21.15.43:0/2837889196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72801053a0 msgr2=0x7f72801987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:21.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.830+0000 7f727ed9d640 1 --2- 172.21.15.43:0/2837889196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72801053a0 0x7f72801987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.830+0000 7f727ed9d640 1 -- 172.21.15.43:0/2837889196 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728019feb0 con 0x7f7280107cd0 2024-04-21T08:36:21.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.830+0000 7f727e59c640 1 --2- 172.21.15.43:0/2837889196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7280102a70 0x7f7280198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:36:21.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.830+0000 7f727dd9b640 1 --2- 172.21.15.43:0/2837889196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72801053a0 0x7f72801987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:36:21.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.830+0000 7f727ed9d640 1 --2- 172.21.15.43:0/2837889196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7280107cd0 0x7f728019f840 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7278009fd0 tx=0x7f727800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:21.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.830+0000 7f72777fe640 1 -- 172.21.15.43:0/2837889196 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7278017070 con 0x7f7280107cd0 2024-04-21T08:36:21.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.830+0000 7f7284ac7640 1 -- 172.21.15.43:0/2837889196 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72801a01a0 con 0x7f7280107cd0 2024-04-21T08:36:21.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.830+0000 7f72777fe640 1 -- 172.21.15.43:0/2837889196 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7278003e20 con 0x7f7280107cd0 2024-04-21T08:36:21.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.830+0000 7f72777fe640 1 -- 172.21.15.43:0/2837889196 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7278010460 con 0x7f7280107cd0 2024-04-21T08:36:21.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.830+0000 7f7284ac7640 1 -- 172.21.15.43:0/2837889196 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72801a06f0 con 0x7f7280107cd0 2024-04-21T08:36:21.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.831+0000 7f7284ac7640 1 -- 172.21.15.43:0/2837889196 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72800fdef0 con 0x7f7280107cd0 2024-04-21T08:36:21.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.832+0000 7f72777fe640 1 -- 172.21.15.43:0/2837889196 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7278003970 con 0x7f7280107cd0 2024-04-21T08:36:21.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.833+0000 7f72777fe640 1 --2- 172.21.15.43:0/2837889196 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f726c0a3bb0 0x7f726c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:21.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.833+0000 7f727e59c640 1 --2- 172.21.15.43:0/2837889196 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f726c0a3bb0 0x7f726c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:21.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.833+0000 7f72777fe640 1 -- 172.21.15.43:0/2837889196 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f72780cfc70 con 0x7f7280107cd0 2024-04-21T08:36:21.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.834+0000 7f727e59c640 1 --2- 172.21.15.43:0/2837889196 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f726c0a3bb0 0x7f726c0a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7268009230 tx=0x7f7268005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:21.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:21.841+0000 7f72777fe640 1 -- 172.21.15.43:0/2837889196 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f727809c4a0 con 0x7f7280107cd0 2024-04-21T08:36:22.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.029+0000 7f7284ac7640 1 -- 172.21.15.43:0/2837889196 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7280062b50 con 0x7f7280107cd0 2024-04-21T08:36:22.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.446+0000 7f72777fe640 1 -- 172.21.15.43:0/2837889196 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f72780a07a0 con 0x7f7280107cd0 2024-04-21T08:36:22.446 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-04-21T08:36:22.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.449+0000 7f7284ac7640 1 -- 172.21.15.43:0/2837889196 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f726c0a3bb0 msgr2=0x7f726c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:22.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.449+0000 7f7284ac7640 1 --2- 172.21.15.43:0/2837889196 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f726c0a3bb0 0x7f726c0a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7268009230 tx=0x7f7268005fd0 comp rx=0 tx=0).stop 2024-04-21T08:36:22.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.449+0000 7f7284ac7640 1 -- 172.21.15.43:0/2837889196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7280107cd0 msgr2=0x7f728019f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:22.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.449+0000 7f7284ac7640 1 --2- 172.21.15.43:0/2837889196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7280107cd0 0x7f728019f840 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7278009fd0 tx=0x7f727800b260 comp rx=0 tx=0).stop 2024-04-21T08:36:22.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.449+0000 7f7284ac7640 1 -- 172.21.15.43:0/2837889196 shutdown_connections 2024-04-21T08:36:22.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.449+0000 7f7284ac7640 1 --2- 172.21.15.43:0/2837889196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7280107cd0 0x7f728019f840 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:22.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.449+0000 7f7284ac7640 1 --2- 172.21.15.43:0/2837889196 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f726c0a3bb0 0x7f726c0a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:22.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.449+0000 7f7284ac7640 1 --2- 172.21.15.43:0/2837889196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72801053a0 0x7f72801987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:22.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.449+0000 7f7284ac7640 1 --2- 172.21.15.43:0/2837889196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7280102a70 0x7f7280198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:22.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.449+0000 7f7284ac7640 1 -- 172.21.15.43:0/2837889196 >> 172.21.15.43:0/2837889196 conn(0x7f72800fc400 msgr2=0x7f7280103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:22.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.450+0000 7f7284ac7640 1 -- 172.21.15.43:0/2837889196 shutdown_connections 2024-04-21T08:36:22.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.450+0000 7f7284ac7640 1 -- 172.21.15.43:0/2837889196 wait complete. 2024-04-21T08:36:22.467 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-21T08:36:22.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.579+0000 7f65a0524640 1 -- 172.21.15.43:0/914495695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6598102a00 msgr2=0x7f6598109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:22.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.579+0000 7f65a0524640 1 --2- 172.21.15.43:0/914495695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6598102a00 0x7f6598109320 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6590009ca0 tx=0x7f659002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:22.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.579+0000 7f65a0524640 1 -- 172.21.15.43:0/914495695 shutdown_connections 2024-04-21T08:36:22.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.579+0000 7f65a0524640 1 --2- 172.21.15.43:0/914495695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6598102a00 0x7f6598109320 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:22.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.579+0000 7f65a0524640 1 --2- 172.21.15.43:0/914495695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6598102040 0x7f65981024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:22.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.579+0000 7f65a0524640 1 --2- 172.21.15.43:0/914495695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6598100e60 0x7f6598101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:22.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.579+0000 7f65a0524640 1 -- 172.21.15.43:0/914495695 >> 172.21.15.43:0/914495695 conn(0x7f65980fc440 msgr2=0x7f65980fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:22.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.579+0000 7f65a0524640 1 -- 172.21.15.43:0/914495695 shutdown_connections 2024-04-21T08:36:22.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.579+0000 7f65a0524640 1 -- 172.21.15.43:0/914495695 wait complete. 2024-04-21T08:36:22.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.580+0000 7f65a0524640 1 Processor -- start 2024-04-21T08:36:22.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.580+0000 7f65a0524640 1 -- start start 2024-04-21T08:36:22.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.580+0000 7f65a0524640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6598100e60 0x7f659806b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:22.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.580+0000 7f65a0524640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6598102040 0x7f659819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:22.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.580+0000 7f65a0524640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6598102a00 0x7f659819f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:22.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.580+0000 7f65a0524640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f659810c9f0 con 0x7f6598102040 2024-04-21T08:36:22.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.580+0000 7f65a0524640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f659810c870 con 0x7f6598102a00 2024-04-21T08:36:22.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.580+0000 7f65a0524640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f659810cb70 con 0x7f6598100e60 2024-04-21T08:36:22.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.580+0000 7f659dabf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6598102040 0x7f659819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:22.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.580+0000 7f659dabf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6598102040 0x7f659819ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58776/0 (socket says 172.21.15.43:58776) 2024-04-21T08:36:22.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.580+0000 7f659dabf640 1 -- 172.21.15.43:0/3673040181 learned_addr learned my addr 172.21.15.43:0/3673040181 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:22.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.580+0000 7f659eac1640 1 --2- 172.21.15.43:0/3673040181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6598102a00 0x7f659819f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:22.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.580+0000 7f659e2c0640 1 --2- 172.21.15.43:0/3673040181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6598100e60 0x7f659806b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:22.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.580+0000 7f659dabf640 1 -- 172.21.15.43:0/3673040181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6598100e60 msgr2=0x7f659806b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:22.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.580+0000 7f659dabf640 1 --2- 172.21.15.43:0/3673040181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6598100e60 0x7f659806b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:22.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.580+0000 7f659dabf640 1 -- 172.21.15.43:0/3673040181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6598102a00 msgr2=0x7f659819f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:22.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.580+0000 7f659dabf640 1 --2- 172.21.15.43:0/3673040181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6598102a00 0x7f659819f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:22.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.580+0000 7f659dabf640 1 -- 172.21.15.43:0/3673040181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659819fdc0 con 0x7f6598102040 2024-04-21T08:36:22.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.580+0000 7f659eac1640 1 --2- 172.21.15.43:0/3673040181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6598102a00 0x7f659819f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:22.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.580+0000 7f659e2c0640 1 --2- 172.21.15.43:0/3673040181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6598100e60 0x7f659806b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:22.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.581+0000 7f659dabf640 1 --2- 172.21.15.43:0/3673040181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6598102040 0x7f659819ce90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f659400d950 tx=0x7f659400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:22.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.581+0000 7f658f7fe640 1 -- 172.21.15.43:0/3673040181 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594017070 con 0x7f6598102040 2024-04-21T08:36:22.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.581+0000 7f65a0524640 1 -- 172.21.15.43:0/3673040181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f659819fff0 con 0x7f6598102040 2024-04-21T08:36:22.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.581+0000 7f658f7fe640 1 -- 172.21.15.43:0/3673040181 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6594009c30 con 0x7f6598102040 2024-04-21T08:36:22.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.581+0000 7f658f7fe640 1 -- 172.21.15.43:0/3673040181 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659400f620 con 0x7f6598102040 2024-04-21T08:36:22.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.581+0000 7f65a0524640 1 -- 172.21.15.43:0/3673040181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65981a0510 con 0x7f6598102040 2024-04-21T08:36:22.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.582+0000 7f65a0524640 1 -- 172.21.15.43:0/3673040181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6598108200 con 0x7f6598102040 2024-04-21T08:36:22.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.583+0000 7f658f7fe640 1 -- 172.21.15.43:0/3673040181 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6594005030 con 0x7f6598102040 2024-04-21T08:36:22.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.584+0000 7f658f7fe640 1 --2- 172.21.15.43:0/3673040181 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f65840a3bb0 0x7f65840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:22.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.584+0000 7f659e2c0640 1 --2- 172.21.15.43:0/3673040181 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f65840a3bb0 0x7f65840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:22.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.584+0000 7f658f7fe640 1 -- 172.21.15.43:0/3673040181 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f65940cfd90 con 0x7f6598102040 2024-04-21T08:36:22.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.584+0000 7f659e2c0640 1 --2- 172.21.15.43:0/3673040181 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f65840a3bb0 0x7f65840a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6588005d00 tx=0x7f6588005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:22.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.588+0000 7f658f7fe640 1 -- 172.21.15.43:0/3673040181 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f659409ca00 con 0x7f6598102040 2024-04-21T08:36:22.635 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:22 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:36:22.635 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:22 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1841595600' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-21T08:36:22.635 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:22 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2837889196' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-21T08:36:22.651 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:22 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:36:22.651 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:22 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1841595600' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-21T08:36:22.651 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:22 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2837889196' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-21T08:36:22.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:22.776+0000 7f65a0524640 1 -- 172.21.15.43:0/3673040181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f659802d5c0 con 0x7f6598102040 2024-04-21T08:36:22.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:22 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:36:22.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:22 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1841595600' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-21T08:36:22.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:22 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2837889196' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-21T08:36:23.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.452+0000 7f658f7fe640 1 -- 172.21.15.43:0/3673040181 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) ==== 171+0+0 (secure 0 0 0) 0x7f6594009de0 con 0x7f6598102040 2024-04-21T08:36:23.451 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-04-21T08:36:23.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.456+0000 7f65a0524640 1 -- 172.21.15.43:0/3673040181 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f65840a3bb0 msgr2=0x7f65840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:23.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.456+0000 7f65a0524640 1 --2- 172.21.15.43:0/3673040181 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f65840a3bb0 0x7f65840a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6588005d00 tx=0x7f6588005ef0 comp rx=0 tx=0).stop 2024-04-21T08:36:23.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.456+0000 7f65a0524640 1 -- 172.21.15.43:0/3673040181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6598102040 msgr2=0x7f659819ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:23.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.456+0000 7f65a0524640 1 --2- 172.21.15.43:0/3673040181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6598102040 0x7f659819ce90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f659400d950 tx=0x7f659400de10 comp rx=0 tx=0).stop 2024-04-21T08:36:23.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.456+0000 7f65a0524640 1 -- 172.21.15.43:0/3673040181 shutdown_connections 2024-04-21T08:36:23.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.456+0000 7f65a0524640 1 --2- 172.21.15.43:0/3673040181 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f65840a3bb0 0x7f65840a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:23.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.456+0000 7f65a0524640 1 --2- 172.21.15.43:0/3673040181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6598102a00 0x7f659819f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:23.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.456+0000 7f65a0524640 1 --2- 172.21.15.43:0/3673040181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6598102040 0x7f659819ce90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:23.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.456+0000 7f65a0524640 1 --2- 172.21.15.43:0/3673040181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6598100e60 0x7f659806b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:23.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.456+0000 7f65a0524640 1 -- 172.21.15.43:0/3673040181 >> 172.21.15.43:0/3673040181 conn(0x7f65980fc440 msgr2=0x7f65981053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:23.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.456+0000 7f65a0524640 1 -- 172.21.15.43:0/3673040181 shutdown_connections 2024-04-21T08:36:23.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.456+0000 7f65a0524640 1 -- 172.21.15.43:0/3673040181 wait complete. 2024-04-21T08:36:23.475 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-21T08:36:23.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.582+0000 7f446fa5c640 1 -- 172.21.15.43:0/1584426568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4468101fa0 msgr2=0x7f44681023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:23.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.582+0000 7f446fa5c640 1 --2- 172.21.15.43:0/1584426568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4468101fa0 0x7f44681023c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4464009900 tx=0x7f446402bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:23.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.582+0000 7f446fa5c640 1 -- 172.21.15.43:0/1584426568 shutdown_connections 2024-04-21T08:36:23.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.582+0000 7f446fa5c640 1 --2- 172.21.15.43:0/1584426568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4468103b50 0x7f446810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:23.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.582+0000 7f446fa5c640 1 --2- 172.21.15.43:0/1584426568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4468103190 0x7f4468103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:23.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.582+0000 7f446fa5c640 1 --2- 172.21.15.43:0/1584426568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4468101fa0 0x7f44681023c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:23.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.582+0000 7f446fa5c640 1 -- 172.21.15.43:0/1584426568 >> 172.21.15.43:0/1584426568 conn(0x7f44680fd530 msgr2=0x7f44680ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:23.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.582+0000 7f446fa5c640 1 -- 172.21.15.43:0/1584426568 shutdown_connections 2024-04-21T08:36:23.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.582+0000 7f446fa5c640 1 -- 172.21.15.43:0/1584426568 wait complete. 2024-04-21T08:36:23.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.583+0000 7f446fa5c640 1 Processor -- start 2024-04-21T08:36:23.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.583+0000 7f446fa5c640 1 -- start start 2024-04-21T08:36:23.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.583+0000 7f446fa5c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4468101fa0 0x7f44681981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:23.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.583+0000 7f446fa5c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4468103190 0x7f4468198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:23.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.583+0000 7f446fa5c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4468103b50 0x7f446819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:23.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.583+0000 7f446fa5c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f446810dae0 con 0x7f4468101fa0 2024-04-21T08:36:23.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.583+0000 7f446fa5c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f446810d960 con 0x7f4468103b50 2024-04-21T08:36:23.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.583+0000 7f446fa5c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f446810dc60 con 0x7f4468103190 2024-04-21T08:36:23.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.583+0000 7f446d7f8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4468101fa0 0x7f44681981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:23.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.583+0000 7f446d7f8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4468101fa0 0x7f44681981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58796/0 (socket says 172.21.15.43:58796) 2024-04-21T08:36:23.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.583+0000 7f446d7f8640 1 -- 172.21.15.43:0/2450697091 learned_addr learned my addr 172.21.15.43:0/2450697091 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:23.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.583+0000 7f446dff9640 1 --2- 172.21.15.43:0/2450697091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4468103b50 0x7f446819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:23.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.583+0000 7f446cff7640 1 --2- 172.21.15.43:0/2450697091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4468103190 0x7f4468198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:23.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.583+0000 7f446d7f8640 1 -- 172.21.15.43:0/2450697091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4468103190 msgr2=0x7f4468198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:23.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.583+0000 7f446d7f8640 1 --2- 172.21.15.43:0/2450697091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4468103190 0x7f4468198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:23.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.583+0000 7f446d7f8640 1 -- 172.21.15.43:0/2450697091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4468103b50 msgr2=0x7f446819f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:23.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.583+0000 7f446d7f8640 1 --2- 172.21.15.43:0/2450697091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4468103b50 0x7f446819f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:23.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.583+0000 7f446d7f8640 1 -- 172.21.15.43:0/2450697091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446819fe10 con 0x7f4468101fa0 2024-04-21T08:36:23.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.583+0000 7f446cff7640 1 --2- 172.21.15.43:0/2450697091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4468103190 0x7f4468198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:23.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.583+0000 7f446dff9640 1 --2- 172.21.15.43:0/2450697091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4468103b50 0x7f446819f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:36:23.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.583+0000 7f446d7f8640 1 --2- 172.21.15.43:0/2450697091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4468101fa0 0x7f44681981f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4464000c00 tx=0x7f4464005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:23.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.584+0000 7f445e7fc640 1 -- 172.21.15.43:0/2450697091 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464043020 con 0x7f4468101fa0 2024-04-21T08:36:23.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.584+0000 7f445e7fc640 1 -- 172.21.15.43:0/2450697091 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4464004350 con 0x7f4468101fa0 2024-04-21T08:36:23.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.584+0000 7f445e7fc640 1 -- 172.21.15.43:0/2450697091 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464003b80 con 0x7f4468101fa0 2024-04-21T08:36:23.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.584+0000 7f446fa5c640 1 -- 172.21.15.43:0/2450697091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44681a00a0 con 0x7f4468101fa0 2024-04-21T08:36:23.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.584+0000 7f446fa5c640 1 -- 172.21.15.43:0/2450697091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44681a0640 con 0x7f4468101fa0 2024-04-21T08:36:23.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.585+0000 7f446fa5c640 1 -- 172.21.15.43:0/2450697091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44681093e0 con 0x7f4468101fa0 2024-04-21T08:36:23.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.587+0000 7f445e7fc640 1 -- 172.21.15.43:0/2450697091 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4464046020 con 0x7f4468101fa0 2024-04-21T08:36:23.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.587+0000 7f445e7fc640 1 --2- 172.21.15.43:0/2450697091 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f44540a3c30 0x7f44540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:23.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.587+0000 7f445e7fc640 1 -- 172.21.15.43:0/2450697091 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f44640f2800 con 0x7f4468101fa0 2024-04-21T08:36:23.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.587+0000 7f446cff7640 1 --2- 172.21.15.43:0/2450697091 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f44540a3c30 0x7f44540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:23.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.588+0000 7f446cff7640 1 --2- 172.21.15.43:0/2450697091 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f44540a3c30 0x7f44540a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4468199700 tx=0x7f4458005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:23.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.591+0000 7f445e7fc640 1 -- 172.21.15.43:0/2450697091 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f446404d020 con 0x7f4468101fa0 2024-04-21T08:36:23.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:23 smithi175 ceph-mon[31920]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:23.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:23 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2837889196' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-21T08:36:23.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:23 smithi175 ceph-mon[31920]: osdmap e83: 12 total, 12 up, 12 in 2024-04-21T08:36:23.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:23 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3673040181' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-21T08:36:23.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:23.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:23.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:23.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:23.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:23.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:23.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:23.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:36:23.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:23 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:23.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:23 smithi112 ceph-mon[31085]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:23.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:23 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2837889196' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-21T08:36:23.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:23 smithi112 ceph-mon[31085]: osdmap e83: 12 total, 12 up, 12 in 2024-04-21T08:36:23.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:23 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3673040181' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-21T08:36:23.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:23.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:23.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:23.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:23.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:23.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:23.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:23.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:36:23.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:23 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:23.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:23 smithi043 ceph-mon[27978]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:23.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:23 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2837889196' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-21T08:36:23.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:23 smithi043 ceph-mon[27978]: osdmap e83: 12 total, 12 up, 12 in 2024-04-21T08:36:23.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:23 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3673040181' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-21T08:36:23.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:23.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:23.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:23.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:23.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:23.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:23.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:23.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:36:23.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:23 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:23.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:23.820+0000 7f446fa5c640 1 -- 172.21.15.43:0/2450697091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f446802d5c0 con 0x7f4468101fa0 2024-04-21T08:36:24.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.453+0000 7f445e7fc640 1 -- 172.21.15.43:0/2450697091 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f44640bf390 con 0x7f4468101fa0 2024-04-21T08:36:24.453 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 2 and data pool 3 2024-04-21T08:36:24.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.456+0000 7f446fa5c640 1 -- 172.21.15.43:0/2450697091 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f44540a3c30 msgr2=0x7f44540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:24.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.456+0000 7f446fa5c640 1 --2- 172.21.15.43:0/2450697091 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f44540a3c30 0x7f44540a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4468199700 tx=0x7f4458005f90 comp rx=0 tx=0).stop 2024-04-21T08:36:24.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.456+0000 7f446fa5c640 1 -- 172.21.15.43:0/2450697091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4468101fa0 msgr2=0x7f44681981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:24.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.456+0000 7f446fa5c640 1 --2- 172.21.15.43:0/2450697091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4468101fa0 0x7f44681981f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4464000c00 tx=0x7f4464005dc0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.456+0000 7f446fa5c640 1 -- 172.21.15.43:0/2450697091 shutdown_connections 2024-04-21T08:36:24.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.456+0000 7f446fa5c640 1 --2- 172.21.15.43:0/2450697091 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f44540a3c30 0x7f44540a60f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.456+0000 7f446fa5c640 1 --2- 172.21.15.43:0/2450697091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4468103b50 0x7f446819f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.456+0000 7f446fa5c640 1 --2- 172.21.15.43:0/2450697091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4468103190 0x7f4468198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.456+0000 7f446fa5c640 1 --2- 172.21.15.43:0/2450697091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4468101fa0 0x7f44681981f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.456+0000 7f446fa5c640 1 -- 172.21.15.43:0/2450697091 >> 172.21.15.43:0/2450697091 conn(0x7f44680fd530 msgr2=0x7f44681065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:24.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.456+0000 7f446fa5c640 1 -- 172.21.15.43:0/2450697091 shutdown_connections 2024-04-21T08:36:24.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.456+0000 7f446fa5c640 1 -- 172.21.15.43:0/2450697091 wait complete. 2024-04-21T08:36:24.470 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-21T08:36:24.470 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-21T08:36:24.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:24 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3673040181' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-21T08:36:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:24 smithi043 ceph-mon[27978]: osdmap e84: 12 total, 12 up, 12 in 2024-04-21T08:36:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:24 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2450697091' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-21T08:36:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:24 smithi043 ceph-mon[27978]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-21T08:36:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:24 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:24.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:24 smithi043 ceph-mon[27978]: osdmap e85: 12 total, 12 up, 12 in 2024-04-21T08:36:24.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.579+0000 7fb37d3e2640 1 -- 172.21.15.43:0/1801429521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb378105340 msgr2=0x7fb378107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:24.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.579+0000 7fb37d3e2640 1 --2- 172.21.15.43:0/1801429521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb378105340 0x7fb378107730 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb368009900 tx=0x7fb36802bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:24.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.579+0000 7fb37d3e2640 1 -- 172.21.15.43:0/1801429521 shutdown_connections 2024-04-21T08:36:24.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.579+0000 7fb37d3e2640 1 --2- 172.21.15.43:0/1801429521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb378107c70 0x7fb37810a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.579+0000 7fb37d3e2640 1 --2- 172.21.15.43:0/1801429521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb378105340 0x7fb378107730 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.579+0000 7fb37d3e2640 1 --2- 172.21.15.43:0/1801429521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb378102a10 0x7fb378104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.579+0000 7fb37d3e2640 1 -- 172.21.15.43:0/1801429521 >> 172.21.15.43:0/1801429521 conn(0x7fb3780fc400 msgr2=0x7fb3780fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:24.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.579+0000 7fb37d3e2640 1 -- 172.21.15.43:0/1801429521 shutdown_connections 2024-04-21T08:36:24.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.579+0000 7fb37d3e2640 1 -- 172.21.15.43:0/1801429521 wait complete. 2024-04-21T08:36:24.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.580+0000 7fb37d3e2640 1 Processor -- start 2024-04-21T08:36:24.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.580+0000 7fb37d3e2640 1 -- start start 2024-04-21T08:36:24.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.580+0000 7fb37d3e2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb378102a10 0x7fb3781982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:24.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.580+0000 7fb37d3e2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb378105340 0x7fb378198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:24.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.580+0000 7fb37d3e2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb378107c70 0x7fb37819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:24.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.580+0000 7fb37d3e2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb37810db10 con 0x7fb378107c70 2024-04-21T08:36:24.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.580+0000 7fb37d3e2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb37810d990 con 0x7fb378105340 2024-04-21T08:36:24.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.580+0000 7fb37d3e2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb37810dc90 con 0x7fb378102a10 2024-04-21T08:36:24.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.580+0000 7fb3777fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb378107c70 0x7fb37819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:24.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.580+0000 7fb3777fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb378107c70 0x7fb37819f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58804/0 (socket says 172.21.15.43:58804) 2024-04-21T08:36:24.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.580+0000 7fb3777fe640 1 -- 172.21.15.43:0/725016512 learned_addr learned my addr 172.21.15.43:0/725016512 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:24.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.581+0000 7fb376ffd640 1 --2- 172.21.15.43:0/725016512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb378102a10 0x7fb3781982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:24.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.581+0000 7fb3767fc640 1 --2- 172.21.15.43:0/725016512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb378105340 0x7fb378198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:24.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.581+0000 7fb3777fe640 1 -- 172.21.15.43:0/725016512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb378102a10 msgr2=0x7fb3781982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:24.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.581+0000 7fb3777fe640 1 --2- 172.21.15.43:0/725016512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb378102a10 0x7fb3781982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.581+0000 7fb3777fe640 1 -- 172.21.15.43:0/725016512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb378105340 msgr2=0x7fb378198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:24.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.581+0000 7fb3777fe640 1 --2- 172.21.15.43:0/725016512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb378105340 0x7fb378198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.581+0000 7fb3777fe640 1 -- 172.21.15.43:0/725016512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37819ff10 con 0x7fb378107c70 2024-04-21T08:36:24.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.581+0000 7fb3777fe640 1 --2- 172.21.15.43:0/725016512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb378107c70 0x7fb37819f8a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb37000cad0 tx=0x7fb37000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:24.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.581+0000 7fb36ffff640 1 -- 172.21.15.43:0/725016512 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37000ad60 con 0x7fb378107c70 2024-04-21T08:36:24.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.581+0000 7fb36ffff640 1 -- 172.21.15.43:0/725016512 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb370005020 con 0x7fb378107c70 2024-04-21T08:36:24.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.581+0000 7fb37d3e2640 1 -- 172.21.15.43:0/725016512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3781a0200 con 0x7fb378107c70 2024-04-21T08:36:24.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.581+0000 7fb36ffff640 1 -- 172.21.15.43:0/725016512 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37000f6b0 con 0x7fb378107c70 2024-04-21T08:36:24.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.581+0000 7fb37d3e2640 1 -- 172.21.15.43:0/725016512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3781a0750 con 0x7fb378107c70 2024-04-21T08:36:24.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.582+0000 7fb37d3e2640 1 -- 172.21.15.43:0/725016512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3780fdef0 con 0x7fb378107c70 2024-04-21T08:36:24.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.583+0000 7fb36ffff640 1 -- 172.21.15.43:0/725016512 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb3700051d0 con 0x7fb378107c70 2024-04-21T08:36:24.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.584+0000 7fb36ffff640 1 --2- 172.21.15.43:0/725016512 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb3640a3bb0 0x7fb3640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:24.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.584+0000 7fb376ffd640 1 --2- 172.21.15.43:0/725016512 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb3640a3bb0 0x7fb3640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:24.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.584+0000 7fb36ffff640 1 -- 172.21.15.43:0/725016512 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fb3700d0400 con 0x7fb378107c70 2024-04-21T08:36:24.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.584+0000 7fb376ffd640 1 --2- 172.21.15.43:0/725016512 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb3640a3bb0 0x7fb3640a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb360009990 tx=0x7fb360009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:24.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.588+0000 7fb36ffff640 1 -- 172.21.15.43:0/725016512 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb370014030 con 0x7fb378107c70 2024-04-21T08:36:24.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:24 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3673040181' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-21T08:36:24.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:24 smithi175 ceph-mon[31920]: osdmap e84: 12 total, 12 up, 12 in 2024-04-21T08:36:24.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:24 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2450697091' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-21T08:36:24.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:24 smithi175 ceph-mon[31920]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-21T08:36:24.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:24 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:24.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:24 smithi175 ceph-mon[31920]: osdmap e85: 12 total, 12 up, 12 in 2024-04-21T08:36:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.785+0000 7fb37d3e2640 1 -- 172.21.15.43:0/725016512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fb37802d5c0 con 0x7fb378107c70 2024-04-21T08:36:24.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:24 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3673040181' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-21T08:36:24.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:24 smithi112 ceph-mon[31085]: osdmap e84: 12 total, 12 up, 12 in 2024-04-21T08:36:24.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:24 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2450697091' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-21T08:36:24.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:24 smithi112 ceph-mon[31085]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-21T08:36:24.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:24 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:36:24.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:24 smithi112 ceph-mon[31085]: osdmap e85: 12 total, 12 up, 12 in 2024-04-21T08:36:24.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.973+0000 7fb36ffff640 1 -- 172.21.15.43:0/725016512 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) ==== 157+0+0 (secure 0 0 0) 0x7fb37009ce90 con 0x7fb378107c70 2024-04-21T08:36:24.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.976+0000 7fb37d3e2640 1 -- 172.21.15.43:0/725016512 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb3640a3bb0 msgr2=0x7fb3640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:24.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.976+0000 7fb37d3e2640 1 --2- 172.21.15.43:0/725016512 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb3640a3bb0 0x7fb3640a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb360009990 tx=0x7fb360009110 comp rx=0 tx=0).stop 2024-04-21T08:36:24.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.976+0000 7fb37d3e2640 1 -- 172.21.15.43:0/725016512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb378107c70 msgr2=0x7fb37819f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:24.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.976+0000 7fb37d3e2640 1 --2- 172.21.15.43:0/725016512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb378107c70 0x7fb37819f8a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb37000cad0 tx=0x7fb37000ced0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.976+0000 7fb37d3e2640 1 -- 172.21.15.43:0/725016512 shutdown_connections 2024-04-21T08:36:24.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.976+0000 7fb37d3e2640 1 --2- 172.21.15.43:0/725016512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb378107c70 0x7fb37819f8a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.976+0000 7fb37d3e2640 1 --2- 172.21.15.43:0/725016512 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb3640a3bb0 0x7fb3640a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.976+0000 7fb37d3e2640 1 --2- 172.21.15.43:0/725016512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb378105340 0x7fb378198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.976+0000 7fb37d3e2640 1 --2- 172.21.15.43:0/725016512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb378102a10 0x7fb3781982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.976+0000 7fb37d3e2640 1 -- 172.21.15.43:0/725016512 >> 172.21.15.43:0/725016512 conn(0x7fb3780fc400 msgr2=0x7fb3781034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:24.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.976+0000 7fb37d3e2640 1 -- 172.21.15.43:0/725016512 shutdown_connections 2024-04-21T08:36:24.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:24.976+0000 7fb37d3e2640 1 -- 172.21.15.43:0/725016512 wait complete. 2024-04-21T08:36:24.987 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-21T08:36:25.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.138+0000 7fb8860d3640 1 -- 172.21.15.43:0/3509394563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb880102a00 msgr2=0x7fb880109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:25.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.138+0000 7fb8860d3640 1 --2- 172.21.15.43:0/3509394563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb880102a00 0x7fb880109320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb878009ca0 tx=0x7fb87802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:25.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.138+0000 7fb8860d3640 1 -- 172.21.15.43:0/3509394563 shutdown_connections 2024-04-21T08:36:25.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.138+0000 7fb8860d3640 1 --2- 172.21.15.43:0/3509394563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb880102a00 0x7fb880109320 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:25.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.138+0000 7fb8860d3640 1 --2- 172.21.15.43:0/3509394563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb880102040 0x7fb8801024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:25.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.138+0000 7fb8860d3640 1 --2- 172.21.15.43:0/3509394563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb880100e60 0x7fb880101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:25.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.138+0000 7fb8860d3640 1 -- 172.21.15.43:0/3509394563 >> 172.21.15.43:0/3509394563 conn(0x7fb8800fc440 msgr2=0x7fb8800fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:25.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.138+0000 7fb8860d3640 1 -- 172.21.15.43:0/3509394563 shutdown_connections 2024-04-21T08:36:25.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.138+0000 7fb8860d3640 1 -- 172.21.15.43:0/3509394563 wait complete. 2024-04-21T08:36:25.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.139+0000 7fb8860d3640 1 Processor -- start 2024-04-21T08:36:25.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.139+0000 7fb8860d3640 1 -- start start 2024-04-21T08:36:25.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.139+0000 7fb8860d3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb880100e60 0x7fb8801981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:25.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.139+0000 7fb8860d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb880102040 0x7fb880198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:25.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.139+0000 7fb8860d3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb880102a00 0x7fb88019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:25.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.139+0000 7fb8860d3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb88010c9f0 con 0x7fb880100e60 2024-04-21T08:36:25.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.139+0000 7fb8860d3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb88010c870 con 0x7fb880102040 2024-04-21T08:36:25.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.139+0000 7fb8860d3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb88010cb70 con 0x7fb880102a00 2024-04-21T08:36:25.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.139+0000 7fb87f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb880100e60 0x7fb8801981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:25.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.139+0000 7fb87f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb880100e60 0x7fb8801981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58824/0 (socket says 172.21.15.43:58824) 2024-04-21T08:36:25.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.139+0000 7fb87f7fe640 1 -- 172.21.15.43:0/2043289684 learned_addr learned my addr 172.21.15.43:0/2043289684 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:25.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.140+0000 7fb87effd640 1 --2- 172.21.15.43:0/2043289684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb880102040 0x7fb880198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:25.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.140+0000 7fb87f7fe640 1 -- 172.21.15.43:0/2043289684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb880102a00 msgr2=0x7fb88019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:25.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.140+0000 7fb87f7fe640 1 --2- 172.21.15.43:0/2043289684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb880102a00 0x7fb88019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:25.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.140+0000 7fb87f7fe640 1 -- 172.21.15.43:0/2043289684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb880102040 msgr2=0x7fb880198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:25.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.140+0000 7fb87f7fe640 1 --2- 172.21.15.43:0/2043289684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb880102040 0x7fb880198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:25.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.140+0000 7fb87f7fe640 1 -- 172.21.15.43:0/2043289684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88019fe10 con 0x7fb880100e60 2024-04-21T08:36:25.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.140+0000 7fb87effd640 1 --2- 172.21.15.43:0/2043289684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb880102040 0x7fb880198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:25.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.140+0000 7fb87f7fe640 1 --2- 172.21.15.43:0/2043289684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb880100e60 0x7fb8801981f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb87000d770 tx=0x7fb87000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:25.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.140+0000 7fb87cff9640 1 -- 172.21.15.43:0/2043289684 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870017070 con 0x7fb880100e60 2024-04-21T08:36:25.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.140+0000 7fb87cff9640 1 -- 172.21.15.43:0/2043289684 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb870009c30 con 0x7fb880100e60 2024-04-21T08:36:25.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.140+0000 7fb8860d3640 1 -- 172.21.15.43:0/2043289684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8801a0100 con 0x7fb880100e60 2024-04-21T08:36:25.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.140+0000 7fb87cff9640 1 -- 172.21.15.43:0/2043289684 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870003cc0 con 0x7fb880100e60 2024-04-21T08:36:25.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.140+0000 7fb8860d3640 1 -- 172.21.15.43:0/2043289684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8801a0650 con 0x7fb880100e60 2024-04-21T08:36:25.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.141+0000 7fb8860d3640 1 -- 172.21.15.43:0/2043289684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8801a08b0 con 0x7fb880100e60 2024-04-21T08:36:25.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.142+0000 7fb87cff9640 1 -- 172.21.15.43:0/2043289684 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb870003e60 con 0x7fb880100e60 2024-04-21T08:36:25.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.143+0000 7fb87cff9640 1 --2- 172.21.15.43:0/2043289684 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8680a3bb0 0x7fb8680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:25.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.143+0000 7fb87effd640 1 --2- 172.21.15.43:0/2043289684 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8680a3bb0 0x7fb8680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:25.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.143+0000 7fb87cff9640 1 -- 172.21.15.43:0/2043289684 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8277+0+0 (secure 0 0 0) 0x7fb8700cfe70 con 0x7fb880100e60 2024-04-21T08:36:25.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.143+0000 7fb87effd640 1 --2- 172.21.15.43:0/2043289684 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8680a3bb0 0x7fb8680a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb880199700 tx=0x7fb874005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:25.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.147+0000 7fb87cff9640 1 -- 172.21.15.43:0/2043289684 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb87009c820 con 0x7fb880100e60 2024-04-21T08:36:25.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:25.335+0000 7fb8860d3640 1 -- 172.21.15.43:0/2043289684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb8801a0b30 con 0x7fb880100e60 2024-04-21T08:36:25.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:25 smithi175 ceph-mon[31920]: pgmap v290: 97 pgs: 4 active+clean, 18 creating+peering, 75 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:25.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:25 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2450697091' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-21T08:36:25.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:25 smithi175 ceph-mon[31920]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-21T08:36:25.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:25 smithi175 ceph-mon[31920]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-21T08:36:25.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:25 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/725016512' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-21T08:36:25.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:25 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] up:active 2024-04-21T08:36:25.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:25 smithi175 ceph-mon[31920]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:25.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:25 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/725016512' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-21T08:36:25.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:25 smithi175 ceph-mon[31920]: osdmap e86: 12 total, 12 up, 12 in 2024-04-21T08:36:25.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:25 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2043289684' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-21T08:36:25.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:25 smithi112 ceph-mon[31085]: pgmap v290: 97 pgs: 4 active+clean, 18 creating+peering, 75 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:25.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:25 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2450697091' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-21T08:36:25.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:25 smithi112 ceph-mon[31085]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-21T08:36:25.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:25 smithi112 ceph-mon[31085]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-21T08:36:25.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:25 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/725016512' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-21T08:36:25.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:25 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] up:active 2024-04-21T08:36:25.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:25 smithi112 ceph-mon[31085]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:25.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:25 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/725016512' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-21T08:36:25.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:25 smithi112 ceph-mon[31085]: osdmap e86: 12 total, 12 up, 12 in 2024-04-21T08:36:25.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:25 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2043289684' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-21T08:36:25.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:25 smithi043 ceph-mon[27978]: pgmap v290: 97 pgs: 4 active+clean, 18 creating+peering, 75 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:25.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:25 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2450697091' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-21T08:36:25.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:25 smithi043 ceph-mon[27978]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-21T08:36:25.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:25 smithi043 ceph-mon[27978]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-21T08:36:25.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:25 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/725016512' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-21T08:36:25.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:25 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] up:active 2024-04-21T08:36:25.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:25 smithi043 ceph-mon[27978]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:25.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:25 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/725016512' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-21T08:36:25.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:25 smithi043 ceph-mon[27978]: osdmap e86: 12 total, 12 up, 12 in 2024-04-21T08:36:25.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:25 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2043289684' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-21T08:36:26.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:26.981+0000 7fb87cff9640 1 -- 172.21.15.43:0/2043289684 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) ==== 228+0+0 (secure 0 0 0) 0x7fb870009de0 con 0x7fb880100e60 2024-04-21T08:36:26.980 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data_ec' created 2024-04-21T08:36:26.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:26.985+0000 7fb8860d3640 1 -- 172.21.15.43:0/2043289684 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8680a3bb0 msgr2=0x7fb8680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:26.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:26.985+0000 7fb8860d3640 1 --2- 172.21.15.43:0/2043289684 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8680a3bb0 0x7fb8680a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb880199700 tx=0x7fb874005ec0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:26.985+0000 7fb8860d3640 1 -- 172.21.15.43:0/2043289684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb880100e60 msgr2=0x7fb8801981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:26.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:26.985+0000 7fb8860d3640 1 --2- 172.21.15.43:0/2043289684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb880100e60 0x7fb8801981f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb87000d770 tx=0x7fb87000dc30 comp rx=0 tx=0).stop 2024-04-21T08:36:26.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:26.985+0000 7fb8860d3640 1 -- 172.21.15.43:0/2043289684 shutdown_connections 2024-04-21T08:36:26.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:26.985+0000 7fb8860d3640 1 --2- 172.21.15.43:0/2043289684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb880102a00 0x7fb88019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:26.985+0000 7fb8860d3640 1 --2- 172.21.15.43:0/2043289684 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8680a3bb0 0x7fb8680a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:26.985+0000 7fb8860d3640 1 --2- 172.21.15.43:0/2043289684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb880102040 0x7fb880198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:26.985+0000 7fb8860d3640 1 --2- 172.21.15.43:0/2043289684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb880100e60 0x7fb8801981f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:26.985+0000 7fb8860d3640 1 -- 172.21.15.43:0/2043289684 >> 172.21.15.43:0/2043289684 conn(0x7fb8800fc440 msgr2=0x7fb8800fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:26.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:26.986+0000 7fb8860d3640 1 -- 172.21.15.43:0/2043289684 shutdown_connections 2024-04-21T08:36:26.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:26.986+0000 7fb8860d3640 1 -- 172.21.15.43:0/2043289684 wait complete. 2024-04-21T08:36:27.003 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-21T08:36:27.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.116+0000 7efcbdd32640 1 -- 172.21.15.43:0/1592693663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcb8103c00 msgr2=0x7efcb810a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:27.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.116+0000 7efcbdd32640 1 --2- 172.21.15.43:0/1592693663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcb8103c00 0x7efcb810a490 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efcb0009cd0 tx=0x7efcb002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.116+0000 7efcbdd32640 1 -- 172.21.15.43:0/1592693663 shutdown_connections 2024-04-21T08:36:27.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.116+0000 7efcbdd32640 1 --2- 172.21.15.43:0/1592693663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcb8103c00 0x7efcb810a490 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.116+0000 7efcbdd32640 1 --2- 172.21.15.43:0/1592693663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcb8103240 0x7efcb81036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.116+0000 7efcbdd32640 1 --2- 172.21.15.43:0/1592693663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcb8100270 0x7efcb8100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.116+0000 7efcbdd32640 1 -- 172.21.15.43:0/1592693663 >> 172.21.15.43:0/1592693663 conn(0x7efcb80fbb80 msgr2=0x7efcb80fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:27.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.116+0000 7efcbdd32640 1 -- 172.21.15.43:0/1592693663 shutdown_connections 2024-04-21T08:36:27.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.116+0000 7efcbdd32640 1 -- 172.21.15.43:0/1592693663 wait complete. 2024-04-21T08:36:27.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.116+0000 7efcbdd32640 1 Processor -- start 2024-04-21T08:36:27.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.117+0000 7efcbdd32640 1 -- start start 2024-04-21T08:36:27.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.117+0000 7efcbdd32640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcb8100270 0x7efcb8198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.117+0000 7efcbdd32640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcb8103240 0x7efcb8198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.117+0000 7efcbdd32640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcb8103c00 0x7efcb819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.117+0000 7efcbdd32640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efcb810db10 con 0x7efcb8100270 2024-04-21T08:36:27.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.117+0000 7efcbdd32640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efcb810d990 con 0x7efcb8103c00 2024-04-21T08:36:27.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.117+0000 7efcbdd32640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efcb810dc90 con 0x7efcb8103240 2024-04-21T08:36:27.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.117+0000 7efcb77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcb8100270 0x7efcb8198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.117+0000 7efcb77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcb8100270 0x7efcb8198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60478/0 (socket says 172.21.15.43:60478) 2024-04-21T08:36:27.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.117+0000 7efcb77fe640 1 -- 172.21.15.43:0/3411610153 learned_addr learned my addr 172.21.15.43:0/3411610153 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.117+0000 7efcb6ffd640 1 --2- 172.21.15.43:0/3411610153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcb8103240 0x7efcb8198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.117+0000 7efcb7fff640 1 --2- 172.21.15.43:0/3411610153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcb8103c00 0x7efcb819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.117+0000 7efcb77fe640 1 -- 172.21.15.43:0/3411610153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcb8103240 msgr2=0x7efcb8198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.117+0000 7efcb77fe640 1 --2- 172.21.15.43:0/3411610153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcb8103240 0x7efcb8198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.117+0000 7efcb77fe640 1 -- 172.21.15.43:0/3411610153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcb8103c00 msgr2=0x7efcb819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.117+0000 7efcb77fe640 1 --2- 172.21.15.43:0/3411610153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcb8103c00 0x7efcb819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.117+0000 7efcb77fe640 1 -- 172.21.15.43:0/3411610153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb819fda0 con 0x7efcb8100270 2024-04-21T08:36:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.117+0000 7efcb77fe640 1 --2- 172.21.15.43:0/3411610153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcb8100270 0x7efcb8198220 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efca800d770 tx=0x7efca800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.118+0000 7efcb4ff9640 1 -- 172.21.15.43:0/3411610153 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8017070 con 0x7efcb8100270 2024-04-21T08:36:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.118+0000 7efcbdd32640 1 -- 172.21.15.43:0/3411610153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb81a0090 con 0x7efcb8100270 2024-04-21T08:36:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.118+0000 7efcb4ff9640 1 -- 172.21.15.43:0/3411610153 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efca8009c30 con 0x7efcb8100270 2024-04-21T08:36:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.118+0000 7efcb4ff9640 1 -- 172.21.15.43:0/3411610153 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8003cc0 con 0x7efcb8100270 2024-04-21T08:36:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.118+0000 7efcbdd32640 1 -- 172.21.15.43:0/3411610153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb81a05e0 con 0x7efcb8100270 2024-04-21T08:36:27.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.119+0000 7efcbdd32640 1 -- 172.21.15.43:0/3411610153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcb8109370 con 0x7efcb8100270 2024-04-21T08:36:27.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.120+0000 7efcb4ff9640 1 -- 172.21.15.43:0/3411610153 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efca8003e60 con 0x7efcb8100270 2024-04-21T08:36:27.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.120+0000 7efcb4ff9640 1 --2- 172.21.15.43:0/3411610153 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7efca00a3bb0 0x7efca00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.121+0000 7efcb4ff9640 1 -- 172.21.15.43:0/3411610153 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7efca80d0340 con 0x7efcb8100270 2024-04-21T08:36:27.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.121+0000 7efcb6ffd640 1 --2- 172.21.15.43:0/3411610153 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7efca00a3bb0 0x7efca00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.121+0000 7efcb6ffd640 1 --2- 172.21.15.43:0/3411610153 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7efca00a3bb0 0x7efca00a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efcb8199690 tx=0x7efcac00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:27.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.124+0000 7efcb4ff9640 1 -- 172.21.15.43:0/3411610153 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efca8009de0 con 0x7efcb8100270 2024-04-21T08:36:27.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:26 smithi175 ceph-mon[31920]: pgmap v292: 97 pgs: 36 active+clean, 18 creating+peering, 43 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 1 op/s 2024-04-21T08:36:27.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:26 smithi175 ceph-mon[31920]: osdmap e87: 12 total, 12 up, 12 in 2024-04-21T08:36:27.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:26 smithi112 ceph-mon[31085]: pgmap v292: 97 pgs: 36 active+clean, 18 creating+peering, 43 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 1 op/s 2024-04-21T08:36:27.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:26 smithi112 ceph-mon[31085]: osdmap e87: 12 total, 12 up, 12 in 2024-04-21T08:36:27.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.312+0000 7efcbdd32640 1 -- 172.21.15.43:0/3411610153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7efcb802d5c0 con 0x7efcb8100270 2024-04-21T08:36:27.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:26 smithi043 ceph-mon[27978]: pgmap v292: 97 pgs: 36 active+clean, 18 creating+peering, 43 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 1 op/s 2024-04-21T08:36:27.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:26 smithi043 ceph-mon[27978]: osdmap e87: 12 total, 12 up, 12 in 2024-04-21T08:36:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.987+0000 7efcb4ff9640 1 -- 172.21.15.43:0/3411610153 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) ==== 169+0+0 (secure 0 0 0) 0x7efca809c4b0 con 0x7efcb8100270 2024-04-21T08:36:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:set pool 4 allow_ec_overwrites to true 2024-04-21T08:36:27.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.991+0000 7efcbdd32640 1 -- 172.21.15.43:0/3411610153 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7efca00a3bb0 msgr2=0x7efca00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:27.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.991+0000 7efcbdd32640 1 --2- 172.21.15.43:0/3411610153 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7efca00a3bb0 0x7efca00a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efcb8199690 tx=0x7efcac00b390 comp rx=0 tx=0).stop 2024-04-21T08:36:27.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.991+0000 7efcbdd32640 1 -- 172.21.15.43:0/3411610153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcb8100270 msgr2=0x7efcb8198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:27.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.991+0000 7efcbdd32640 1 --2- 172.21.15.43:0/3411610153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcb8100270 0x7efcb8198220 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efca800d770 tx=0x7efca800dc30 comp rx=0 tx=0).stop 2024-04-21T08:36:27.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.991+0000 7efcbdd32640 1 -- 172.21.15.43:0/3411610153 shutdown_connections 2024-04-21T08:36:27.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.991+0000 7efcbdd32640 1 --2- 172.21.15.43:0/3411610153 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7efca00a3bb0 0x7efca00a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.991+0000 7efcbdd32640 1 --2- 172.21.15.43:0/3411610153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcb8103c00 0x7efcb819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.991+0000 7efcbdd32640 1 --2- 172.21.15.43:0/3411610153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcb8103240 0x7efcb8198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.991+0000 7efcbdd32640 1 --2- 172.21.15.43:0/3411610153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcb8100270 0x7efcb8198220 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.991+0000 7efcbdd32640 1 -- 172.21.15.43:0/3411610153 >> 172.21.15.43:0/3411610153 conn(0x7efcb80fbb80 msgr2=0x7efcb8106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:27.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.991+0000 7efcbdd32640 1 -- 172.21.15.43:0/3411610153 shutdown_connections 2024-04-21T08:36:27.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:27.991+0000 7efcbdd32640 1 -- 172.21.15.43:0/3411610153 wait complete. 2024-04-21T08:36:28.013 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-21T08:36:28.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.125+0000 7f8777bf4640 1 -- 172.21.15.43:0/1716765225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87701031a0 msgr2=0x7f8770103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:28.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.125+0000 7f8777bf4640 1 --2- 172.21.15.43:0/1716765225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87701031a0 0x7f8770103620 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8760009900 tx=0x7f876002bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:28.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.125+0000 7f8777bf4640 1 -- 172.21.15.43:0/1716765225 shutdown_connections 2024-04-21T08:36:28.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.125+0000 7f8777bf4640 1 --2- 172.21.15.43:0/1716765225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8770103b60 0x7f877010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:28.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.125+0000 7f8777bf4640 1 --2- 172.21.15.43:0/1716765225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87701031a0 0x7f8770103620 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:28.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.125+0000 7f8777bf4640 1 --2- 172.21.15.43:0/1716765225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8770101fb0 0x7f87701023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:28.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.125+0000 7f8777bf4640 1 -- 172.21.15.43:0/1716765225 >> 172.21.15.43:0/1716765225 conn(0x7f87700fd520 msgr2=0x7f87700ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:28.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.125+0000 7f8777bf4640 1 -- 172.21.15.43:0/1716765225 shutdown_connections 2024-04-21T08:36:28.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.125+0000 7f8777bf4640 1 -- 172.21.15.43:0/1716765225 wait complete. 2024-04-21T08:36:28.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.126+0000 7f8777bf4640 1 Processor -- start 2024-04-21T08:36:28.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.126+0000 7f8777bf4640 1 -- start start 2024-04-21T08:36:28.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.126+0000 7f8777bf4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8770101fb0 0x7f87701981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:28.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.126+0000 7f8777bf4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87701031a0 0x7f8770198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:28.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.126+0000 7f8777bf4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8770103b60 0x7f877019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:28.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.126+0000 7f8777bf4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f877010daf0 con 0x7f8770101fb0 2024-04-21T08:36:28.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.126+0000 7f8777bf4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f877010d970 con 0x7f8770103b60 2024-04-21T08:36:28.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.126+0000 7f8777bf4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f877010dc70 con 0x7f87701031a0 2024-04-21T08:36:28.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.126+0000 7f8775990640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8770101fb0 0x7f87701981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:28.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.126+0000 7f8775990640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8770101fb0 0x7f87701981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60498/0 (socket says 172.21.15.43:60498) 2024-04-21T08:36:28.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.126+0000 7f8775990640 1 -- 172.21.15.43:0/3083189056 learned_addr learned my addr 172.21.15.43:0/3083189056 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:28.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.126+0000 7f8776191640 1 --2- 172.21.15.43:0/3083189056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8770103b60 0x7f877019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:28.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.126+0000 7f877518f640 1 --2- 172.21.15.43:0/3083189056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87701031a0 0x7f8770198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:28.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.126+0000 7f8775990640 1 -- 172.21.15.43:0/3083189056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87701031a0 msgr2=0x7f8770198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:28.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.126+0000 7f8775990640 1 --2- 172.21.15.43:0/3083189056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87701031a0 0x7f8770198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:28.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.126+0000 7f8775990640 1 -- 172.21.15.43:0/3083189056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8770103b60 msgr2=0x7f877019f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:28.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.126+0000 7f8775990640 1 --2- 172.21.15.43:0/3083189056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8770103b60 0x7f877019f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:28.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.126+0000 7f8775990640 1 -- 172.21.15.43:0/3083189056 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f877019fdf0 con 0x7f8770101fb0 2024-04-21T08:36:28.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.126+0000 7f877518f640 1 --2- 172.21.15.43:0/3083189056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87701031a0 0x7f8770198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:28.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.127+0000 7f8775990640 1 --2- 172.21.15.43:0/3083189056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8770101fb0 0x7f87701981d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f876c00c970 tx=0x7f876c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:28.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.127+0000 7f8766ffd640 1 -- 172.21.15.43:0/3083189056 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876c004220 con 0x7f8770101fb0 2024-04-21T08:36:28.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.127+0000 7f8766ffd640 1 -- 172.21.15.43:0/3083189056 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f876c0043c0 con 0x7f8770101fb0 2024-04-21T08:36:28.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.127+0000 7f8766ffd640 1 -- 172.21.15.43:0/3083189056 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876c00f450 con 0x7f8770101fb0 2024-04-21T08:36:28.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.127+0000 7f8777bf4640 1 -- 172.21.15.43:0/3083189056 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87701a00e0 con 0x7f8770101fb0 2024-04-21T08:36:28.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.127+0000 7f8777bf4640 1 -- 172.21.15.43:0/3083189056 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87701a0680 con 0x7f8770101fb0 2024-04-21T08:36:28.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.128+0000 7f8777bf4640 1 -- 172.21.15.43:0/3083189056 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8770109350 con 0x7f8770101fb0 2024-04-21T08:36:28.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.129+0000 7f8766ffd640 1 -- 172.21.15.43:0/3083189056 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f876c003610 con 0x7f8770101fb0 2024-04-21T08:36:28.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.129+0000 7f8766ffd640 1 --2- 172.21.15.43:0/3083189056 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f875c0a3c30 0x7f875c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:28.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.130+0000 7f877518f640 1 --2- 172.21.15.43:0/3083189056 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f875c0a3c30 0x7f875c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:28.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.130+0000 7f8766ffd640 1 -- 172.21.15.43:0/3083189056 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7f876c0d0880 con 0x7f8770101fb0 2024-04-21T08:36:28.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.130+0000 7f877518f640 1 --2- 172.21.15.43:0/3083189056 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f875c0a3c30 0x7f875c0a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f87600096c0 tx=0x7f8760000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:28.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.133+0000 7f8766ffd640 1 -- 172.21.15.43:0/3083189056 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f876c014030 con 0x7f8770101fb0 2024-04-21T08:36:28.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:27 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2043289684' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-21T08:36:28.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:27 smithi112 ceph-mon[31085]: osdmap e88: 12 total, 12 up, 12 in 2024-04-21T08:36:28.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:27 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3411610153' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-21T08:36:28.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:27 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2043289684' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-21T08:36:28.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:27 smithi043 ceph-mon[27978]: osdmap e88: 12 total, 12 up, 12 in 2024-04-21T08:36:28.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:27 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3411610153' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-21T08:36:28.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.363+0000 7f8777bf4640 1 -- 172.21.15.43:0/3083189056 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f877002d5c0 con 0x7f8770101fb0 2024-04-21T08:36:28.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:27 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2043289684' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-21T08:36:28.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:27 smithi175 ceph-mon[31920]: osdmap e88: 12 total, 12 up, 12 in 2024-04-21T08:36:28.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:27 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3411610153' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-21T08:36:28.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.988+0000 7f8766ffd640 1 -- 172.21.15.43:0/3083189056 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) ==== 137+0+0 (secure 0 0 0) 0x7f876c09d060 con 0x7f8770101fb0 2024-04-21T08:36:28.987 INFO:teuthology.orchestra.run.smithi043.stderr:added data pool 4 to fsmap 2024-04-21T08:36:28.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.992+0000 7f8777bf4640 1 -- 172.21.15.43:0/3083189056 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f875c0a3c30 msgr2=0x7f875c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:28.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.992+0000 7f8777bf4640 1 --2- 172.21.15.43:0/3083189056 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f875c0a3c30 0x7f875c0a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f87600096c0 tx=0x7f8760000bc0 comp rx=0 tx=0).stop 2024-04-21T08:36:28.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.992+0000 7f8777bf4640 1 -- 172.21.15.43:0/3083189056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8770101fb0 msgr2=0x7f87701981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:28.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.992+0000 7f8777bf4640 1 --2- 172.21.15.43:0/3083189056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8770101fb0 0x7f87701981d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f876c00c970 tx=0x7f876c00ce30 comp rx=0 tx=0).stop 2024-04-21T08:36:28.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.992+0000 7f8777bf4640 1 -- 172.21.15.43:0/3083189056 shutdown_connections 2024-04-21T08:36:28.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.992+0000 7f8777bf4640 1 --2- 172.21.15.43:0/3083189056 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f875c0a3c30 0x7f875c0a60f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:28.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.992+0000 7f8777bf4640 1 --2- 172.21.15.43:0/3083189056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8770103b60 0x7f877019f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:28.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.992+0000 7f8777bf4640 1 --2- 172.21.15.43:0/3083189056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87701031a0 0x7f8770198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:28.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.992+0000 7f8777bf4640 1 --2- 172.21.15.43:0/3083189056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8770101fb0 0x7f87701981d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:28.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.992+0000 7f8777bf4640 1 -- 172.21.15.43:0/3083189056 >> 172.21.15.43:0/3083189056 conn(0x7f87700fd520 msgr2=0x7f8770106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.992+0000 7f8777bf4640 1 -- 172.21.15.43:0/3083189056 shutdown_connections 2024-04-21T08:36:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:28.992+0000 7f8777bf4640 1 -- 172.21.15.43:0/3083189056 wait complete. 2024-04-21T08:36:29.005 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:36:29.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.114+0000 7fa3acd75640 1 -- 172.21.15.43:0/646578285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3a8107d20 msgr2=0x7fa3a810a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:29.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.114+0000 7fa3acd75640 1 --2- 172.21.15.43:0/646578285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3a8107d20 0x7fa3a810a110 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009ca0 tx=0x7fa3a002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.114+0000 7fa3acd75640 1 -- 172.21.15.43:0/646578285 shutdown_connections 2024-04-21T08:36:29.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.114+0000 7fa3acd75640 1 --2- 172.21.15.43:0/646578285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3a8107d20 0x7fa3a810a110 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.114+0000 7fa3acd75640 1 --2- 172.21.15.43:0/646578285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3a81053f0 0x7fa3a81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.114+0000 7fa3acd75640 1 --2- 172.21.15.43:0/646578285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3a8102ac0 0x7fa3a8104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.114+0000 7fa3acd75640 1 -- 172.21.15.43:0/646578285 >> 172.21.15.43:0/646578285 conn(0x7fa3a80fc470 msgr2=0x7fa3a80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:29.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.114+0000 7fa3acd75640 1 -- 172.21.15.43:0/646578285 shutdown_connections 2024-04-21T08:36:29.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.114+0000 7fa3acd75640 1 -- 172.21.15.43:0/646578285 wait complete. 2024-04-21T08:36:29.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.115+0000 7fa3acd75640 1 Processor -- start 2024-04-21T08:36:29.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.115+0000 7fa3acd75640 1 -- start start 2024-04-21T08:36:29.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.115+0000 7fa3acd75640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3a8102ac0 0x7fa3a8102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:29.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.115+0000 7fa3acd75640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3a81053f0 0x7fa3a8100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:29.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.115+0000 7fa3acd75640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3a8107d20 0x7fa3a8100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:29.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.115+0000 7fa3acd75640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a810dbe0 con 0x7fa3a8107d20 2024-04-21T08:36:29.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.115+0000 7fa3acd75640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a810da60 con 0x7fa3a81053f0 2024-04-21T08:36:29.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.115+0000 7fa3acd75640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a810dd60 con 0x7fa3a8102ac0 2024-04-21T08:36:29.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.115+0000 7fa3a6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3a8107d20 0x7fa3a8100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:29.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.115+0000 7fa3a6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3a8107d20 0x7fa3a8100fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60504/0 (socket says 172.21.15.43:60504) 2024-04-21T08:36:29.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.115+0000 7fa3a6d9d640 1 -- 172.21.15.43:0/2206877013 learned_addr learned my addr 172.21.15.43:0/2206877013 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:29.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.116+0000 7fa39dd9b640 1 --2- 172.21.15.43:0/2206877013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3a81053f0 0x7fa3a8100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:29.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.116+0000 7fa3a6d9d640 1 -- 172.21.15.43:0/2206877013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3a8102ac0 msgr2=0x7fa3a8102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:29.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.116+0000 7fa3a659c640 1 --2- 172.21.15.43:0/2206877013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3a8102ac0 0x7fa3a8102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:29.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.116+0000 7fa3a6d9d640 1 --2- 172.21.15.43:0/2206877013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3a8102ac0 0x7fa3a8102430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.116+0000 7fa3a6d9d640 1 -- 172.21.15.43:0/2206877013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3a81053f0 msgr2=0x7fa3a8100a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:29.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.116+0000 7fa3a6d9d640 1 --2- 172.21.15.43:0/2206877013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3a81053f0 0x7fa3a8100a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.116+0000 7fa3a6d9d640 1 -- 172.21.15.43:0/2206877013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a8101840 con 0x7fa3a8107d20 2024-04-21T08:36:29.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.116+0000 7fa39dd9b640 1 --2- 172.21.15.43:0/2206877013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3a81053f0 0x7fa3a8100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:29.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.116+0000 7fa3a6d9d640 1 --2- 172.21.15.43:0/2206877013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3a8107d20 0x7fa3a8100fb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa3a002c700 tx=0x7fa3a0035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:29.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.116+0000 7fa39ffff640 1 -- 172.21.15.43:0/2206877013 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0039040 con 0x7fa3a8107d20 2024-04-21T08:36:29.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.116+0000 7fa39ffff640 1 -- 172.21.15.43:0/2206877013 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3a0044070 con 0x7fa3a8107d20 2024-04-21T08:36:29.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.116+0000 7fa3acd75640 1 -- 172.21.15.43:0/2206877013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a81a2340 con 0x7fa3a8107d20 2024-04-21T08:36:29.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.116+0000 7fa39ffff640 1 -- 172.21.15.43:0/2206877013 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0004440 con 0x7fa3a8107d20 2024-04-21T08:36:29.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.116+0000 7fa3acd75640 1 -- 172.21.15.43:0/2206877013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a81a2910 con 0x7fa3a8107d20 2024-04-21T08:36:29.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.117+0000 7fa3acd75640 1 -- 172.21.15.43:0/2206877013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3a80fdf60 con 0x7fa3a8107d20 2024-04-21T08:36:29.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.118+0000 7fa39ffff640 1 -- 172.21.15.43:0/2206877013 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa3a00045e0 con 0x7fa3a8107d20 2024-04-21T08:36:29.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.119+0000 7fa39ffff640 1 --2- 172.21.15.43:0/2206877013 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa3940a3bb0 0x7fa3940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:29.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.119+0000 7fa3a659c640 1 --2- 172.21.15.43:0/2206877013 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa3940a3bb0 0x7fa3940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:29.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.119+0000 7fa39ffff640 1 -- 172.21.15.43:0/2206877013 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa3a00f2dd0 con 0x7fa3a8107d20 2024-04-21T08:36:29.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.120+0000 7fa3a659c640 1 --2- 172.21.15.43:0/2206877013 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa3940a3bb0 0x7fa3940a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa39800a810 tx=0x7fa398008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:29.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.124+0000 7fa39ffff640 1 -- 172.21.15.43:0/2206877013 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa3a00bf570 con 0x7fa3a8107d20 2024-04-21T08:36:29.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:28 smithi112 ceph-mon[31085]: pgmap v295: 129 pgs: 73 active+clean, 18 creating+peering, 38 unknown; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2024-04-21T08:36:29.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:28 smithi112 ceph-mon[31085]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T08:36:29.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:28 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3411610153' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-21T08:36:29.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:28 smithi112 ceph-mon[31085]: osdmap e89: 12 total, 12 up, 12 in 2024-04-21T08:36:29.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:28 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3083189056' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-21T08:36:29.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:28 smithi043 ceph-mon[27978]: pgmap v295: 129 pgs: 73 active+clean, 18 creating+peering, 38 unknown; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2024-04-21T08:36:29.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:28 smithi043 ceph-mon[27978]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T08:36:29.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:28 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3411610153' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-21T08:36:29.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:28 smithi043 ceph-mon[27978]: osdmap e89: 12 total, 12 up, 12 in 2024-04-21T08:36:29.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:28 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3083189056' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-21T08:36:29.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.354+0000 7fa3acd75640 1 -- 172.21.15.43:0/2206877013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3a8062b50 con 0x7fa3a8107d20 2024-04-21T08:36:29.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.355+0000 7fa39ffff640 1 -- 172.21.15.43:0/2206877013 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11803 (secure 0 0 0) 0x7fa3a00bee70 con 0x7fa3a8107d20 2024-04-21T08:36:29.354 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:29.355 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:36:28.364257+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24527,"qdb_cluster":[24527]},"id":1}]} 2024-04-21T08:36:29.355 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 16 2024-04-21T08:36:29.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.357+0000 7fa3acd75640 1 -- 172.21.15.43:0/2206877013 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa3940a3bb0 msgr2=0x7fa3940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:29.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.357+0000 7fa3acd75640 1 --2- 172.21.15.43:0/2206877013 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa3940a3bb0 0x7fa3940a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa39800a810 tx=0x7fa398008040 comp rx=0 tx=0).stop 2024-04-21T08:36:29.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.357+0000 7fa3acd75640 1 -- 172.21.15.43:0/2206877013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3a8107d20 msgr2=0x7fa3a8100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:29.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.357+0000 7fa3acd75640 1 --2- 172.21.15.43:0/2206877013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3a8107d20 0x7fa3a8100fb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa3a002c700 tx=0x7fa3a0035d60 comp rx=0 tx=0).stop 2024-04-21T08:36:29.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.358+0000 7fa3acd75640 1 -- 172.21.15.43:0/2206877013 shutdown_connections 2024-04-21T08:36:29.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.358+0000 7fa3acd75640 1 --2- 172.21.15.43:0/2206877013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3a8107d20 0x7fa3a8100fb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.358+0000 7fa3acd75640 1 --2- 172.21.15.43:0/2206877013 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa3940a3bb0 0x7fa3940a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.358+0000 7fa3acd75640 1 --2- 172.21.15.43:0/2206877013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3a81053f0 0x7fa3a8100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.358+0000 7fa3acd75640 1 --2- 172.21.15.43:0/2206877013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3a8102ac0 0x7fa3a8102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.358+0000 7fa3acd75640 1 -- 172.21.15.43:0/2206877013 >> 172.21.15.43:0/2206877013 conn(0x7fa3a80fc470 msgr2=0x7fa3a8103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:29.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.358+0000 7fa3acd75640 1 -- 172.21.15.43:0/2206877013 shutdown_connections 2024-04-21T08:36:29.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.358+0000 7fa3acd75640 1 -- 172.21.15.43:0/2206877013 wait complete. 2024-04-21T08:36:29.368 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:36:29.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:28 smithi175 ceph-mon[31920]: pgmap v295: 129 pgs: 73 active+clean, 18 creating+peering, 38 unknown; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2024-04-21T08:36:29.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:28 smithi175 ceph-mon[31920]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T08:36:29.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:28 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3411610153' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-21T08:36:29.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:28 smithi175 ceph-mon[31920]: osdmap e89: 12 total, 12 up, 12 in 2024-04-21T08:36:29.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:28 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3083189056' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-21T08:36:29.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.470+0000 7fbd73011640 1 -- 172.21.15.43:0/1350132614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd6c101810 msgr2=0x7fbd6c101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:29.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.470+0000 7fbd73011640 1 --2- 172.21.15.43:0/1350132614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd6c101810 0x7fbd6c101c90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009900 tx=0x7fbd5402bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:29.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.470+0000 7fbd73011640 1 -- 172.21.15.43:0/1350132614 shutdown_connections 2024-04-21T08:36:29.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.470+0000 7fbd73011640 1 --2- 172.21.15.43:0/1350132614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6c1021d0 0x7fbd6c108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.470+0000 7fbd73011640 1 --2- 172.21.15.43:0/1350132614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd6c101810 0x7fbd6c101c90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.470+0000 7fbd73011640 1 --2- 172.21.15.43:0/1350132614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd6c100620 0x7fbd6c100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.470+0000 7fbd73011640 1 -- 172.21.15.43:0/1350132614 >> 172.21.15.43:0/1350132614 conn(0x7fbd6c0fbb90 msgr2=0x7fbd6c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:29.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.470+0000 7fbd73011640 1 -- 172.21.15.43:0/1350132614 shutdown_connections 2024-04-21T08:36:29.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.470+0000 7fbd73011640 1 -- 172.21.15.43:0/1350132614 wait complete. 2024-04-21T08:36:29.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.471+0000 7fbd73011640 1 Processor -- start 2024-04-21T08:36:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.471+0000 7fbd73011640 1 -- start start 2024-04-21T08:36:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.471+0000 7fbd73011640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd6c100620 0x7fbd6c198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.471+0000 7fbd73011640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6c101810 0x7fbd6c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.471+0000 7fbd73011640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd6c1021d0 0x7fbd6c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.471+0000 7fbd73011640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c10b940 con 0x7fbd6c1021d0 2024-04-21T08:36:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.471+0000 7fbd73011640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c10b7c0 con 0x7fbd6c101810 2024-04-21T08:36:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.471+0000 7fbd73011640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c10bac0 con 0x7fbd6c100620 2024-04-21T08:36:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.471+0000 7fbd715ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd6c1021d0 0x7fbd6c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.471+0000 7fbd715ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd6c1021d0 0x7fbd6c19f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60524/0 (socket says 172.21.15.43:60524) 2024-04-21T08:36:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.471+0000 7fbd715ae640 1 -- 172.21.15.43:0/1064795169 learned_addr learned my addr 172.21.15.43:0/1064795169 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.471+0000 7fbd70dad640 1 --2- 172.21.15.43:0/1064795169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd6c100620 0x7fbd6c198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.471+0000 7fbd6bfff640 1 --2- 172.21.15.43:0/1064795169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6c101810 0x7fbd6c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.471+0000 7fbd715ae640 1 -- 172.21.15.43:0/1064795169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd6c100620 msgr2=0x7fbd6c198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.471+0000 7fbd715ae640 1 --2- 172.21.15.43:0/1064795169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd6c100620 0x7fbd6c198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.471+0000 7fbd715ae640 1 -- 172.21.15.43:0/1064795169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6c101810 msgr2=0x7fbd6c198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.471+0000 7fbd715ae640 1 --2- 172.21.15.43:0/1064795169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6c101810 0x7fbd6c198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.471+0000 7fbd715ae640 1 -- 172.21.15.43:0/1064795169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd6c19fe70 con 0x7fbd6c1021d0 2024-04-21T08:36:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.471+0000 7fbd6bfff640 1 --2- 172.21.15.43:0/1064795169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6c101810 0x7fbd6c198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.472+0000 7fbd715ae640 1 --2- 172.21.15.43:0/1064795169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd6c1021d0 0x7fbd6c19f800 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400de10 tx=0x7fbd6400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.472+0000 7fbd69ffb640 1 -- 172.21.15.43:0/1064795169 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd64003b70 con 0x7fbd6c1021d0 2024-04-21T08:36:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.472+0000 7fbd69ffb640 1 -- 172.21.15.43:0/1064795169 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd64003d10 con 0x7fbd6c1021d0 2024-04-21T08:36:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.472+0000 7fbd73011640 1 -- 172.21.15.43:0/1064795169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd6c1a0160 con 0x7fbd6c1021d0 2024-04-21T08:36:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.472+0000 7fbd69ffb640 1 -- 172.21.15.43:0/1064795169 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6400ea40 con 0x7fbd6c1021d0 2024-04-21T08:36:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.472+0000 7fbd73011640 1 -- 172.21.15.43:0/1064795169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd6c1a0660 con 0x7fbd6c1021d0 2024-04-21T08:36:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.473+0000 7fbd73011640 1 -- 172.21.15.43:0/1064795169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd6c1a08c0 con 0x7fbd6c1021d0 2024-04-21T08:36:29.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.474+0000 7fbd69ffb640 1 -- 172.21.15.43:0/1064795169 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbd6400ebe0 con 0x7fbd6c1021d0 2024-04-21T08:36:29.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.474+0000 7fbd69ffb640 1 --2- 172.21.15.43:0/1064795169 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fbd580a3bb0 0x7fbd580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:29.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.475+0000 7fbd69ffb640 1 -- 172.21.15.43:0/1064795169 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fbd640d0b20 con 0x7fbd6c1021d0 2024-04-21T08:36:29.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.475+0000 7fbd70dad640 1 --2- 172.21.15.43:0/1064795169 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fbd580a3bb0 0x7fbd580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:29.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.475+0000 7fbd70dad640 1 --2- 172.21.15.43:0/1064795169 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fbd580a3bb0 0x7fbd580a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c101670 tx=0x7fbd5c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:29.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.479+0000 7fbd69ffb640 1 -- 172.21.15.43:0/1064795169 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd6409d2c0 con 0x7fbd6c1021d0 2024-04-21T08:36:29.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.667+0000 7fbd73011640 1 -- 172.21.15.43:0/1064795169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd6c1a0bb0 con 0x7fbd6c1021d0 2024-04-21T08:36:29.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.669+0000 7fbd69ffb640 1 -- 172.21.15.43:0/1064795169 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24152 (secure 0 0 0) 0x7fbd6409cbc0 con 0x7fbd6c1021d0 2024-04-21T08:36:29.668 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:29.669 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":90,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:36:28.364093+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:36:29.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.671+0000 7fbd73011640 1 -- 172.21.15.43:0/1064795169 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fbd580a3bb0 msgr2=0x7fbd580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:29.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.671+0000 7fbd73011640 1 --2- 172.21.15.43:0/1064795169 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fbd580a3bb0 0x7fbd580a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c101670 tx=0x7fbd5c008040 comp rx=0 tx=0).stop 2024-04-21T08:36:29.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.671+0000 7fbd73011640 1 -- 172.21.15.43:0/1064795169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd6c1021d0 msgr2=0x7fbd6c19f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:29.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.671+0000 7fbd73011640 1 --2- 172.21.15.43:0/1064795169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd6c1021d0 0x7fbd6c19f800 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400de10 tx=0x7fbd6400b730 comp rx=0 tx=0).stop 2024-04-21T08:36:29.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.671+0000 7fbd73011640 1 -- 172.21.15.43:0/1064795169 shutdown_connections 2024-04-21T08:36:29.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.671+0000 7fbd73011640 1 --2- 172.21.15.43:0/1064795169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd6c1021d0 0x7fbd6c19f800 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.671+0000 7fbd73011640 1 --2- 172.21.15.43:0/1064795169 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fbd580a3bb0 0x7fbd580a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.671+0000 7fbd73011640 1 --2- 172.21.15.43:0/1064795169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6c101810 0x7fbd6c198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.671+0000 7fbd73011640 1 --2- 172.21.15.43:0/1064795169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd6c100620 0x7fbd6c198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.671+0000 7fbd73011640 1 -- 172.21.15.43:0/1064795169 >> 172.21.15.43:0/1064795169 conn(0x7fbd6c0fbb90 msgr2=0x7fbd6c106d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:29.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.671+0000 7fbd73011640 1 -- 172.21.15.43:0/1064795169 shutdown_connections 2024-04-21T08:36:29.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.671+0000 7fbd73011640 1 -- 172.21.15.43:0/1064795169 wait complete. 2024-04-21T08:36:29.682 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:36:29.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.785+0000 7f9b4010d640 1 -- 172.21.15.43:0/3828926141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b381021d0 msgr2=0x7f9b38108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:29.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.785+0000 7f9b4010d640 1 --2- 172.21.15.43:0/3828926141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b381021d0 0x7f9b38108a60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9b30009ca0 tx=0x7f9b3002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.785+0000 7f9b4010d640 1 -- 172.21.15.43:0/3828926141 shutdown_connections 2024-04-21T08:36:29.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.785+0000 7f9b4010d640 1 --2- 172.21.15.43:0/3828926141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b381021d0 0x7f9b38108a60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.785+0000 7f9b4010d640 1 --2- 172.21.15.43:0/3828926141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b38101810 0x7f9b38101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.785+0000 7f9b4010d640 1 --2- 172.21.15.43:0/3828926141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b38100620 0x7f9b38100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.785+0000 7f9b4010d640 1 -- 172.21.15.43:0/3828926141 >> 172.21.15.43:0/3828926141 conn(0x7f9b380fbb90 msgr2=0x7f9b380fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:29.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.785+0000 7f9b4010d640 1 -- 172.21.15.43:0/3828926141 shutdown_connections 2024-04-21T08:36:29.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.785+0000 7f9b4010d640 1 -- 172.21.15.43:0/3828926141 wait complete. 2024-04-21T08:36:29.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.786+0000 7f9b4010d640 1 Processor -- start 2024-04-21T08:36:29.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.786+0000 7f9b4010d640 1 -- start start 2024-04-21T08:36:29.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.786+0000 7f9b4010d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b38100620 0x7f9b380fdc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:29.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.786+0000 7f9b4010d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b38101810 0x7f9b3819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:29.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.786+0000 7f9b4010d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b381021d0 0x7f9b3819f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:29.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.786+0000 7f9b4010d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3810c1b0 con 0x7f9b38101810 2024-04-21T08:36:29.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.786+0000 7f9b4010d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3810c030 con 0x7f9b38100620 2024-04-21T08:36:29.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.786+0000 7f9b4010d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3810c330 con 0x7f9b381021d0 2024-04-21T08:36:29.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.786+0000 7f9b3d6a8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b38101810 0x7f9b3819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:29.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.786+0000 7f9b3d6a8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b38101810 0x7f9b3819cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60536/0 (socket says 172.21.15.43:60536) 2024-04-21T08:36:29.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.786+0000 7f9b3d6a8640 1 -- 172.21.15.43:0/305056432 learned_addr learned my addr 172.21.15.43:0/305056432 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:29.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.786+0000 7f9b3e6aa640 1 --2- 172.21.15.43:0/305056432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b381021d0 0x7f9b3819f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:29.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.786+0000 7f9b3dea9640 1 --2- 172.21.15.43:0/305056432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b38100620 0x7f9b380fdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:29.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.786+0000 7f9b3d6a8640 1 -- 172.21.15.43:0/305056432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b381021d0 msgr2=0x7f9b3819f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:29.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.786+0000 7f9b3d6a8640 1 --2- 172.21.15.43:0/305056432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b381021d0 0x7f9b3819f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.786+0000 7f9b3d6a8640 1 -- 172.21.15.43:0/305056432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b38100620 msgr2=0x7f9b380fdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:29.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.786+0000 7f9b3d6a8640 1 --2- 172.21.15.43:0/305056432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b38100620 0x7f9b380fdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.786+0000 7f9b3d6a8640 1 -- 172.21.15.43:0/305056432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b3819fe10 con 0x7f9b38101810 2024-04-21T08:36:29.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.786+0000 7f9b3e6aa640 1 --2- 172.21.15.43:0/305056432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b381021d0 0x7f9b3819f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:29.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.787+0000 7f9b3dea9640 1 --2- 172.21.15.43:0/305056432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b38100620 0x7f9b380fdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:29.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.787+0000 7f9b3d6a8640 1 --2- 172.21.15.43:0/305056432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b38101810 0x7f9b3819cf10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800b4a0 tx=0x7f9b2800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:29.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.787+0000 7f9b2effd640 1 -- 172.21.15.43:0/305056432 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28014070 con 0x7f9b38101810 2024-04-21T08:36:29.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.787+0000 7f9b2effd640 1 -- 172.21.15.43:0/305056432 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b28010ac0 con 0x7f9b38101810 2024-04-21T08:36:29.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.787+0000 7f9b4010d640 1 -- 172.21.15.43:0/305056432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b381a0100 con 0x7f9b38101810 2024-04-21T08:36:29.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.787+0000 7f9b2effd640 1 -- 172.21.15.43:0/305056432 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28017c30 con 0x7f9b38101810 2024-04-21T08:36:29.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.787+0000 7f9b4010d640 1 -- 172.21.15.43:0/305056432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b381a0650 con 0x7f9b38101810 2024-04-21T08:36:29.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.788+0000 7f9b4010d640 1 -- 172.21.15.43:0/305056432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b38107d70 con 0x7f9b38101810 2024-04-21T08:36:29.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.789+0000 7f9b2effd640 1 -- 172.21.15.43:0/305056432 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9b28017070 con 0x7f9b38101810 2024-04-21T08:36:29.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.790+0000 7f9b2effd640 1 --2- 172.21.15.43:0/305056432 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9b240a3bb0 0x7f9b240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:29.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.790+0000 7f9b3dea9640 1 --2- 172.21.15.43:0/305056432 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9b240a3bb0 0x7f9b240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:29.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.790+0000 7f9b2effd640 1 -- 172.21.15.43:0/305056432 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f9b280cfe30 con 0x7f9b38101810 2024-04-21T08:36:29.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.790+0000 7f9b3dea9640 1 --2- 172.21.15.43:0/305056432 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9b240a3bb0 0x7f9b240a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9b34005d00 tx=0x7f9b34005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:29.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.794+0000 7f9b2effd640 1 -- 172.21.15.43:0/305056432 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b2809c570 con 0x7f9b38101810 2024-04-21T08:36:29.794 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:36:29 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-b[61578]: 2024-04-21T08:36:29.433+0000 7fb798658700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-21T08:36:29.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.987+0000 7f9b4010d640 1 -- 172.21.15.43:0/305056432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b3802d5c0 con 0x7f9b38101810 2024-04-21T08:36:29.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.989+0000 7f9b2effd640 1 -- 172.21.15.43:0/305056432 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24152 (secure 0 0 0) 0x7f9b2809be70 con 0x7f9b38101810 2024-04-21T08:36:29.988 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:29.989 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":90,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:36:28.364093+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:36:29.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.991+0000 7f9b4010d640 1 -- 172.21.15.43:0/305056432 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9b240a3bb0 msgr2=0x7f9b240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:29.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.991+0000 7f9b4010d640 1 --2- 172.21.15.43:0/305056432 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9b240a3bb0 0x7f9b240a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9b34005d00 tx=0x7f9b34005ef0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.991+0000 7f9b4010d640 1 -- 172.21.15.43:0/305056432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b38101810 msgr2=0x7f9b3819cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:29.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.991+0000 7f9b4010d640 1 --2- 172.21.15.43:0/305056432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b38101810 0x7f9b3819cf10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800b4a0 tx=0x7f9b2800b960 comp rx=0 tx=0).stop 2024-04-21T08:36:29.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.991+0000 7f9b4010d640 1 -- 172.21.15.43:0/305056432 shutdown_connections 2024-04-21T08:36:29.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.991+0000 7f9b4010d640 1 --2- 172.21.15.43:0/305056432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b381021d0 0x7f9b3819f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.991+0000 7f9b4010d640 1 --2- 172.21.15.43:0/305056432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b38101810 0x7f9b3819cf10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.991+0000 7f9b4010d640 1 --2- 172.21.15.43:0/305056432 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9b240a3bb0 0x7f9b240a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.991+0000 7f9b4010d640 1 --2- 172.21.15.43:0/305056432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b38100620 0x7f9b380fdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.991+0000 7f9b4010d640 1 -- 172.21.15.43:0/305056432 >> 172.21.15.43:0/305056432 conn(0x7f9b380fbb90 msgr2=0x7f9b38104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:29.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.991+0000 7f9b4010d640 1 -- 172.21.15.43:0/305056432 shutdown_connections 2024-04-21T08:36:29.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:29.991+0000 7f9b4010d640 1 -- 172.21.15.43:0/305056432 wait complete. 2024-04-21T08:36:30.021 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:36:30.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.123+0000 7ffa8e3ad640 1 -- 172.21.15.43:0/2862116509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa88101fc0 msgr2=0x7ffa88102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:30.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.123+0000 7ffa8e3ad640 1 --2- 172.21.15.43:0/2862116509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa88101fc0 0x7ffa88102440 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ffa78009900 tx=0x7ffa7802bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:30.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.124+0000 7ffa8e3ad640 1 -- 172.21.15.43:0/2862116509 shutdown_connections 2024-04-21T08:36:30.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.124+0000 7ffa8e3ad640 1 --2- 172.21.15.43:0/2862116509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa88102980 0x7ffa881092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.124+0000 7ffa8e3ad640 1 --2- 172.21.15.43:0/2862116509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa88101fc0 0x7ffa88102440 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.124+0000 7ffa8e3ad640 1 --2- 172.21.15.43:0/2862116509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa88100e70 0x7ffa88101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.124+0000 7ffa8e3ad640 1 -- 172.21.15.43:0/2862116509 >> 172.21.15.43:0/2862116509 conn(0x7ffa880fc3e0 msgr2=0x7ffa880fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:30.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.124+0000 7ffa8e3ad640 1 -- 172.21.15.43:0/2862116509 shutdown_connections 2024-04-21T08:36:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.124+0000 7ffa8e3ad640 1 -- 172.21.15.43:0/2862116509 wait complete. 2024-04-21T08:36:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.124+0000 7ffa8e3ad640 1 Processor -- start 2024-04-21T08:36:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.124+0000 7ffa8e3ad640 1 -- start start 2024-04-21T08:36:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.124+0000 7ffa8e3ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa88100e70 0x7ffa88069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.124+0000 7ffa8e3ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa88101fc0 0x7ffa8819cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.124+0000 7ffa8e3ad640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa88102980 0x7ffa8819f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.124+0000 7ffa8e3ad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8810ca20 con 0x7ffa88100e70 2024-04-21T08:36:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.124+0000 7ffa8e3ad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8810c8a0 con 0x7ffa88101fc0 2024-04-21T08:36:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.124+0000 7ffa8e3ad640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8810cba0 con 0x7ffa88102980 2024-04-21T08:36:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.125+0000 7ffa87fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa88100e70 0x7ffa88069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.125+0000 7ffa87fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa88100e70 0x7ffa88069c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60552/0 (socket says 172.21.15.43:60552) 2024-04-21T08:36:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.125+0000 7ffa87fff640 1 -- 172.21.15.43:0/1306934691 learned_addr learned my addr 172.21.15.43:0/1306934691 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.125+0000 7ffa87fff640 1 -- 172.21.15.43:0/1306934691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa88102980 msgr2=0x7ffa8819f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:30.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.125+0000 7ffa8c94a640 1 --2- 172.21.15.43:0/1306934691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa88102980 0x7ffa8819f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.125+0000 7ffa87fff640 1 --2- 172.21.15.43:0/1306934691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa88102980 0x7ffa8819f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.125+0000 7ffa87fff640 1 -- 172.21.15.43:0/1306934691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa88101fc0 msgr2=0x7ffa8819cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:30.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.125+0000 7ffa877fe640 1 --2- 172.21.15.43:0/1306934691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa88101fc0 0x7ffa8819cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.125+0000 7ffa87fff640 1 --2- 172.21.15.43:0/1306934691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa88101fc0 0x7ffa8819cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.125+0000 7ffa87fff640 1 -- 172.21.15.43:0/1306934691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8819fdc0 con 0x7ffa88100e70 2024-04-21T08:36:30.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.125+0000 7ffa877fe640 1 --2- 172.21.15.43:0/1306934691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa88101fc0 0x7ffa8819cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:30.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.125+0000 7ffa87fff640 1 --2- 172.21.15.43:0/1306934691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa88100e70 0x7ffa88069c70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffa7000c890 tx=0x7ffa7000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:30.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.125+0000 7ffa857fa640 1 -- 172.21.15.43:0/1306934691 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70004220 con 0x7ffa88100e70 2024-04-21T08:36:30.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.125+0000 7ffa857fa640 1 -- 172.21.15.43:0/1306934691 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa700043c0 con 0x7ffa88100e70 2024-04-21T08:36:30.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.125+0000 7ffa8e3ad640 1 -- 172.21.15.43:0/1306934691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa881a00b0 con 0x7ffa88100e70 2024-04-21T08:36:30.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.125+0000 7ffa857fa640 1 -- 172.21.15.43:0/1306934691 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70004a00 con 0x7ffa88100e70 2024-04-21T08:36:30.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.125+0000 7ffa8e3ad640 1 -- 172.21.15.43:0/1306934691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa881a0600 con 0x7ffa88100e70 2024-04-21T08:36:30.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.126+0000 7ffa8e3ad640 1 -- 172.21.15.43:0/1306934691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa88108180 con 0x7ffa88100e70 2024-04-21T08:36:30.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.127+0000 7ffa857fa640 1 -- 172.21.15.43:0/1306934691 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ffa700079b0 con 0x7ffa88100e70 2024-04-21T08:36:30.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.128+0000 7ffa857fa640 1 --2- 172.21.15.43:0/1306934691 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ffa740a3bb0 0x7ffa740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.128+0000 7ffa857fa640 1 -- 172.21.15.43:0/1306934691 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ffa700d0430 con 0x7ffa88100e70 2024-04-21T08:36:30.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.128+0000 7ffa877fe640 1 --2- 172.21.15.43:0/1306934691 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ffa740a3bb0 0x7ffa740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.128+0000 7ffa877fe640 1 --2- 172.21.15.43:0/1306934691 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ffa740a3bb0 0x7ffa740a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffa78036040 tx=0x7ffa78005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:30.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.133+0000 7ffa857fa640 1 -- 172.21.15.43:0/1306934691 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa70014030 con 0x7ffa88100e70 2024-04-21T08:36:30.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:30 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3083189056' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-21T08:36:30.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:30 smithi112 ceph-mon[31085]: osdmap e90: 12 total, 12 up, 12 in 2024-04-21T08:36:30.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:30 smithi112 ceph-mon[31085]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:30.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:30 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2206877013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:30.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:30 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1064795169' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:30.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:30 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/305056432' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:30.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:30 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3083189056' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-21T08:36:30.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:30 smithi043 ceph-mon[27978]: osdmap e90: 12 total, 12 up, 12 in 2024-04-21T08:36:30.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:30 smithi043 ceph-mon[27978]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:30.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:30 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2206877013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:30.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:30 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1064795169' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:30.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:30 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/305056432' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:30.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.359+0000 7ffa8e3ad640 1 -- 172.21.15.43:0/1306934691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa8802d5c0 con 0x7ffa88100e70 2024-04-21T08:36:30.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.360+0000 7ffa857fa640 1 -- 172.21.15.43:0/1306934691 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11803 (secure 0 0 0) 0x7ffa7009cbd0 con 0x7ffa88100e70 2024-04-21T08:36:30.359 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:30.359 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:36:28.364257+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24527,"qdb_cluster":[24527]},"id":1}]} 2024-04-21T08:36:30.359 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 16 2024-04-21T08:36:30.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.361+0000 7ffa8e3ad640 1 -- 172.21.15.43:0/1306934691 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ffa740a3bb0 msgr2=0x7ffa740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:30.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.361+0000 7ffa8e3ad640 1 --2- 172.21.15.43:0/1306934691 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ffa740a3bb0 0x7ffa740a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffa78036040 tx=0x7ffa78005fd0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.361+0000 7ffa8e3ad640 1 -- 172.21.15.43:0/1306934691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa88100e70 msgr2=0x7ffa88069c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:30.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.361+0000 7ffa8e3ad640 1 --2- 172.21.15.43:0/1306934691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa88100e70 0x7ffa88069c70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffa7000c890 tx=0x7ffa7000cd50 comp rx=0 tx=0).stop 2024-04-21T08:36:30.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.362+0000 7ffa8e3ad640 1 -- 172.21.15.43:0/1306934691 shutdown_connections 2024-04-21T08:36:30.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.362+0000 7ffa8e3ad640 1 --2- 172.21.15.43:0/1306934691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa88102980 0x7ffa8819f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.362+0000 7ffa8e3ad640 1 --2- 172.21.15.43:0/1306934691 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ffa740a3bb0 0x7ffa740a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.362+0000 7ffa8e3ad640 1 --2- 172.21.15.43:0/1306934691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa88101fc0 0x7ffa8819cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.362+0000 7ffa8e3ad640 1 --2- 172.21.15.43:0/1306934691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa88100e70 0x7ffa88069c70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.362+0000 7ffa8e3ad640 1 -- 172.21.15.43:0/1306934691 >> 172.21.15.43:0/1306934691 conn(0x7ffa880fc3e0 msgr2=0x7ffa88105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:30.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.362+0000 7ffa8e3ad640 1 -- 172.21.15.43:0/1306934691 shutdown_connections 2024-04-21T08:36:30.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.362+0000 7ffa8e3ad640 1 -- 172.21.15.43:0/1306934691 wait complete. 2024-04-21T08:36:30.373 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:36:30.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:30 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3083189056' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-21T08:36:30.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:30 smithi175 ceph-mon[31920]: osdmap e90: 12 total, 12 up, 12 in 2024-04-21T08:36:30.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:30 smithi175 ceph-mon[31920]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:30.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:30 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2206877013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:30.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:30 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1064795169' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:30.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:30 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/305056432' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:30.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.477+0000 7f6dff111640 1 -- 172.21.15.43:0/1845546196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df8103b50 msgr2=0x7f6df810a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:30.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.477+0000 7f6dff111640 1 --2- 172.21.15.43:0/1845546196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df8103b50 0x7f6df810a3e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6df0009ca0 tx=0x7f6df002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.477+0000 7f6dff111640 1 -- 172.21.15.43:0/1845546196 shutdown_connections 2024-04-21T08:36:30.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.477+0000 7f6dff111640 1 --2- 172.21.15.43:0/1845546196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df8103b50 0x7f6df810a3e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.477+0000 7f6dff111640 1 --2- 172.21.15.43:0/1845546196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df8103190 0x7f6df8103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.477+0000 7f6dff111640 1 --2- 172.21.15.43:0/1845546196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df8101fa0 0x7f6df81023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.477+0000 7f6dff111640 1 -- 172.21.15.43:0/1845546196 >> 172.21.15.43:0/1845546196 conn(0x7f6df80fd530 msgr2=0x7f6df80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:30.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.477+0000 7f6dff111640 1 -- 172.21.15.43:0/1845546196 shutdown_connections 2024-04-21T08:36:30.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.477+0000 7f6dff111640 1 -- 172.21.15.43:0/1845546196 wait complete. 2024-04-21T08:36:30.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.478+0000 7f6dff111640 1 Processor -- start 2024-04-21T08:36:30.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.478+0000 7f6dff111640 1 -- start start 2024-04-21T08:36:30.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.478+0000 7f6dff111640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df8101fa0 0x7f6df8075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.478+0000 7f6dff111640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df8103190 0x7f6df8074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.478+0000 7f6dff111640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df8103b50 0x7f6df8074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.478+0000 7f6dff111640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6df810dae0 con 0x7f6df8103190 2024-04-21T08:36:30.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.478+0000 7f6dff111640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6df810d960 con 0x7f6df8101fa0 2024-04-21T08:36:30.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.478+0000 7f6dff111640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6df810dc60 con 0x7f6df8103b50 2024-04-21T08:36:30.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.478+0000 7f6df7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df8103190 0x7f6df8074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.478+0000 7f6df7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df8103190 0x7f6df8074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60564/0 (socket says 172.21.15.43:60564) 2024-04-21T08:36:30.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.478+0000 7f6df7fff640 1 -- 172.21.15.43:0/3742504367 learned_addr learned my addr 172.21.15.43:0/3742504367 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:30.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.478+0000 7f6dfcead640 1 --2- 172.21.15.43:0/3742504367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df8101fa0 0x7f6df8075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.478+0000 7f6df7fff640 1 -- 172.21.15.43:0/3742504367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df8103b50 msgr2=0x7f6df8074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:30.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.478+0000 7f6df7fff640 1 --2- 172.21.15.43:0/3742504367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df8103b50 0x7f6df8074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.478+0000 7f6df7fff640 1 -- 172.21.15.43:0/3742504367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df8101fa0 msgr2=0x7f6df8075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:30.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.478+0000 7f6df7fff640 1 --2- 172.21.15.43:0/3742504367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df8101fa0 0x7f6df8075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.478+0000 7f6df7fff640 1 -- 172.21.15.43:0/3742504367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df8075110 con 0x7f6df8103190 2024-04-21T08:36:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.479+0000 7f6df7fff640 1 --2- 172.21.15.43:0/3742504367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df8103190 0x7f6df8074340 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6de400d770 tx=0x7f6de400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.479+0000 7f6df5ffb640 1 -- 172.21.15.43:0/3742504367 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4017070 con 0x7f6df8103190 2024-04-21T08:36:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.479+0000 7f6df5ffb640 1 -- 172.21.15.43:0/3742504367 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6de40054d0 con 0x7f6df8103190 2024-04-21T08:36:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.479+0000 7f6dff111640 1 -- 172.21.15.43:0/3742504367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df81a2220 con 0x7f6df8103190 2024-04-21T08:36:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.479+0000 7f6df5ffb640 1 -- 172.21.15.43:0/3742504367 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4004c30 con 0x7f6df8103190 2024-04-21T08:36:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.479+0000 7f6dff111640 1 -- 172.21.15.43:0/3742504367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df81a2770 con 0x7f6df8103190 2024-04-21T08:36:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.480+0000 7f6dff111640 1 -- 172.21.15.43:0/3742504367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6df8109310 con 0x7f6df8103190 2024-04-21T08:36:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.481+0000 7f6df5ffb640 1 -- 172.21.15.43:0/3742504367 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6de4004e50 con 0x7f6df8103190 2024-04-21T08:36:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.482+0000 7f6df5ffb640 1 --2- 172.21.15.43:0/3742504367 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f6de80a3c30 0x7f6de80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.482+0000 7f6dfcead640 1 --2- 172.21.15.43:0/3742504367 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f6de80a3c30 0x7f6de80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.482+0000 7f6df5ffb640 1 -- 172.21.15.43:0/3742504367 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f6de40d01d0 con 0x7f6df8103190 2024-04-21T08:36:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.482+0000 7f6dfcead640 1 --2- 172.21.15.43:0/3742504367 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f6de80a3c30 0x7f6de80a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6df8102ff0 tx=0x7f6dec005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:30.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.486+0000 7f6df5ffb640 1 -- 172.21.15.43:0/3742504367 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6de409c910 con 0x7f6df8103190 2024-04-21T08:36:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.675+0000 7f6dff111640 1 -- 172.21.15.43:0/3742504367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6df8062b50 con 0x7f6df8103190 2024-04-21T08:36:30.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.677+0000 7f6df5ffb640 1 -- 172.21.15.43:0/3742504367 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24152 (secure 0 0 0) 0x7f6de409c210 con 0x7f6df8103190 2024-04-21T08:36:30.676 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:30.676 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":90,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:36:28.364093+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:36:30.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.679+0000 7f6dff111640 1 -- 172.21.15.43:0/3742504367 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f6de80a3c30 msgr2=0x7f6de80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:30.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.679+0000 7f6dff111640 1 --2- 172.21.15.43:0/3742504367 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f6de80a3c30 0x7f6de80a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6df8102ff0 tx=0x7f6dec005c60 comp rx=0 tx=0).stop 2024-04-21T08:36:30.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.679+0000 7f6dff111640 1 -- 172.21.15.43:0/3742504367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df8103190 msgr2=0x7f6df8074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:30.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.679+0000 7f6dff111640 1 --2- 172.21.15.43:0/3742504367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df8103190 0x7f6df8074340 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6de400d770 tx=0x7f6de400dc30 comp rx=0 tx=0).stop 2024-04-21T08:36:30.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.679+0000 7f6dff111640 1 -- 172.21.15.43:0/3742504367 shutdown_connections 2024-04-21T08:36:30.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.679+0000 7f6dff111640 1 --2- 172.21.15.43:0/3742504367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df8103b50 0x7f6df8074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.679+0000 7f6dff111640 1 --2- 172.21.15.43:0/3742504367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df8103190 0x7f6df8074340 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.679+0000 7f6dff111640 1 --2- 172.21.15.43:0/3742504367 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f6de80a3c30 0x7f6de80a60f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.679+0000 7f6dff111640 1 --2- 172.21.15.43:0/3742504367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df8101fa0 0x7f6df8075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.679+0000 7f6dff111640 1 -- 172.21.15.43:0/3742504367 >> 172.21.15.43:0/3742504367 conn(0x7f6df80fd530 msgr2=0x7f6df81064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.679+0000 7f6dff111640 1 -- 172.21.15.43:0/3742504367 shutdown_connections 2024-04-21T08:36:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.679+0000 7f6dff111640 1 -- 172.21.15.43:0/3742504367 wait complete. 2024-04-21T08:36:30.690 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-21T08:36:23.821253+0000', 'modified': '2024-04-21T08:36:28.364257+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24527}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24527': {'gid': 24527, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6839/2143575300', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6838', 'nonce': 2143575300}, {'type': 'v1', 'addr': '172.21.15.112:6839', 'nonce': 2143575300}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24527, 'qdb_cluster': [24527]} 2024-04-21T08:36:30.690 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-21T08:36:30.690 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-21T08:36:30.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.795+0000 7f0f2f59d640 1 -- 172.21.15.43:0/4176069135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f28101fa0 msgr2=0x7f0f281023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:30.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.795+0000 7f0f2f59d640 1 --2- 172.21.15.43:0/4176069135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f28101fa0 0x7f0f281023c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009900 tx=0x7f0f2002bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:30.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.795+0000 7f0f2f59d640 1 -- 172.21.15.43:0/4176069135 shutdown_connections 2024-04-21T08:36:30.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.795+0000 7f0f2f59d640 1 --2- 172.21.15.43:0/4176069135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f28103b50 0x7f0f2810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.795+0000 7f0f2f59d640 1 --2- 172.21.15.43:0/4176069135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f28103190 0x7f0f28103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.795+0000 7f0f2f59d640 1 --2- 172.21.15.43:0/4176069135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f28101fa0 0x7f0f281023c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.795+0000 7f0f2f59d640 1 -- 172.21.15.43:0/4176069135 >> 172.21.15.43:0/4176069135 conn(0x7f0f280fd530 msgr2=0x7f0f280ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:30.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.795+0000 7f0f2f59d640 1 -- 172.21.15.43:0/4176069135 shutdown_connections 2024-04-21T08:36:30.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.795+0000 7f0f2f59d640 1 -- 172.21.15.43:0/4176069135 wait complete. 2024-04-21T08:36:30.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.795+0000 7f0f2f59d640 1 Processor -- start 2024-04-21T08:36:30.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.796+0000 7f0f2f59d640 1 -- start start 2024-04-21T08:36:30.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.796+0000 7f0f2f59d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f28101fa0 0x7f0f28198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.796+0000 7f0f2f59d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f28103190 0x7f0f28198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.796+0000 7f0f2f59d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f28103b50 0x7f0f2819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.796+0000 7f0f2f59d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2810dae0 con 0x7f0f28101fa0 2024-04-21T08:36:30.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.796+0000 7f0f2f59d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2810d960 con 0x7f0f28103b50 2024-04-21T08:36:30.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.796+0000 7f0f2f59d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2810dc60 con 0x7f0f28103190 2024-04-21T08:36:30.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.796+0000 7f0f2d339640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f28101fa0 0x7f0f28198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.796+0000 7f0f2d339640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f28101fa0 0x7f0f28198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60574/0 (socket says 172.21.15.43:60574) 2024-04-21T08:36:30.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.796+0000 7f0f2d339640 1 -- 172.21.15.43:0/2610408402 learned_addr learned my addr 172.21.15.43:0/2610408402 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:30.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.796+0000 7f0f2db3a640 1 --2- 172.21.15.43:0/2610408402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f28103b50 0x7f0f2819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.796+0000 7f0f2cb38640 1 --2- 172.21.15.43:0/2610408402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f28103190 0x7f0f28198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.796+0000 7f0f2d339640 1 -- 172.21.15.43:0/2610408402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f28103190 msgr2=0x7f0f28198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:30.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.796+0000 7f0f2d339640 1 --2- 172.21.15.43:0/2610408402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f28103190 0x7f0f28198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.796+0000 7f0f2d339640 1 -- 172.21.15.43:0/2610408402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f28103b50 msgr2=0x7f0f2819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:30.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.796+0000 7f0f2d339640 1 --2- 172.21.15.43:0/2610408402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f28103b50 0x7f0f2819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.796+0000 7f0f2d339640 1 -- 172.21.15.43:0/2610408402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2819fe30 con 0x7f0f28101fa0 2024-04-21T08:36:30.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.796+0000 7f0f2db3a640 1 --2- 172.21.15.43:0/2610408402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f28103b50 0x7f0f2819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:30.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.796+0000 7f0f2d339640 1 --2- 172.21.15.43:0/2610408402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f28101fa0 0x7f0f28198210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0f20000c00 tx=0x7f0f20004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:30.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.796+0000 7f0f267fc640 1 -- 172.21.15.43:0/2610408402 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f20043020 con 0x7f0f28101fa0 2024-04-21T08:36:30.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.797+0000 7f0f267fc640 1 -- 172.21.15.43:0/2610408402 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f20004aa0 con 0x7f0f28101fa0 2024-04-21T08:36:30.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.797+0000 7f0f2f59d640 1 -- 172.21.15.43:0/2610408402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f281a00c0 con 0x7f0f28101fa0 2024-04-21T08:36:30.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.797+0000 7f0f267fc640 1 -- 172.21.15.43:0/2610408402 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f20003d00 con 0x7f0f28101fa0 2024-04-21T08:36:30.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.797+0000 7f0f2f59d640 1 -- 172.21.15.43:0/2610408402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f281a0610 con 0x7f0f28101fa0 2024-04-21T08:36:30.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.799+0000 7f0f267fc640 1 -- 172.21.15.43:0/2610408402 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0f20046020 con 0x7f0f28101fa0 2024-04-21T08:36:30.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.799+0000 7f0f3218d740 1 Processor -- start 2024-04-21T08:36:30.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.799+0000 7f0f3218d740 1 -- start start 2024-04-21T08:36:30.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.799+0000 7f0f3218d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be5257b250 0x55be5257b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.799+0000 7f0f3218d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be5257bc40 0x55be5257c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.799+0000 7f0f3218d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257c6e0 0x55be525832c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.799+0000 7f0f3218d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55be524dbb10 con 0x55be5257c6e0 2024-04-21T08:36:30.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.799+0000 7f0f3218d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55be52479240 con 0x55be5257b250 2024-04-21T08:36:30.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.799+0000 7f0f3218d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55be524d7ad0 con 0x55be5257bc40 2024-04-21T08:36:30.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.799+0000 7f0f2cb38640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257c6e0 0x55be525832c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.799+0000 7f0f2cb38640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257c6e0 0x55be525832c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60582/0 (socket says 172.21.15.43:60582) 2024-04-21T08:36:30.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.799+0000 7f0f2cb38640 1 -- 172.21.15.43:0/3959437761 learned_addr learned my addr 172.21.15.43:0/3959437761 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:30.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.799+0000 7f0f267fc640 1 --2- 172.21.15.43:0/2610408402 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0f100a3bb0 0x7f0f100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.800+0000 7f0f2db3a640 1 --2- 172.21.15.43:0/3959437761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be5257b250 0x55be5257b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.800+0000 7f0f2db3a640 1 --2- 172.21.15.43:0/2610408402 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0f100a3bb0 0x7f0f100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.800+0000 7f0f2d339640 1 --2- 172.21.15.43:0/3959437761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be5257bc40 0x55be5257c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.800+0000 7f0f2cb38640 1 -- 172.21.15.43:0/3959437761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be5257bc40 msgr2=0x55be5257c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:36:30.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.800+0000 7f0f2cb38640 1 --2- 172.21.15.43:0/3959437761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be5257bc40 0x55be5257c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.800+0000 7f0f267fc640 1 -- 172.21.15.43:0/2610408402 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0f200f2a60 con 0x7f0f28101fa0 2024-04-21T08:36:30.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.800+0000 7f0f2cb38640 1 -- 172.21.15.43:0/3959437761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be5257b250 msgr2=0x55be5257b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:36:30.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.800+0000 7f0f2cb38640 1 --2- 172.21.15.43:0/3959437761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be5257b250 0x55be5257b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.800+0000 7f0f2cb38640 1 -- 172.21.15.43:0/3959437761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be521ee2d0 con 0x55be5257c6e0 2024-04-21T08:36:30.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.800+0000 7f0f2db3a640 1 --2- 172.21.15.43:0/3959437761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be5257b250 0x55be5257b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:30.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.800+0000 7f0f2d339640 1 --2- 172.21.15.43:0/3959437761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be5257bc40 0x55be5257c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:30.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.800+0000 7f0f2cb38640 1 --2- 172.21.15.43:0/3959437761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257c6e0 0x55be525832c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c00ba70 tx=0x7f0f1c00bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5332debd94b0b737 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:30.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.800+0000 7f0f2db3a640 1 --2- 172.21.15.43:0/2610408402 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0f100a3bb0 0x7f0f100a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009270 tx=0x7f0f18005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:30.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.800+0000 7f0f0f7fe640 1 -- 172.21.15.43:0/3959437761 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c017420 con 0x55be5257c6e0 2024-04-21T08:36:30.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.800+0000 7f0f0f7fe640 1 -- 172.21.15.43:0/3959437761 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f1c017a70 con 0x55be5257c6e0 2024-04-21T08:36:30.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.800+0000 7f0f0f7fe640 1 -- 172.21.15.43:0/3959437761 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c00f990 con 0x55be5257c6e0 2024-04-21T08:36:30.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.801+0000 7f0f3218d740 1 -- 172.21.15.43:0/3959437761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257c6e0 msgr2=0x55be525832c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:30.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.801+0000 7f0f3218d740 1 --2- 172.21.15.43:0/3959437761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257c6e0 0x55be525832c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c00ba70 tx=0x7f0f1c00bf30 comp rx=0 tx=0).stop 2024-04-21T08:36:30.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.801+0000 7f0f3218d740 1 -- 172.21.15.43:0/3959437761 shutdown_connections 2024-04-21T08:36:30.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.801+0000 7f0f3218d740 1 --2- 172.21.15.43:0/3959437761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257c6e0 0x55be525832c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.801+0000 7f0f3218d740 1 --2- 172.21.15.43:0/3959437761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be5257bc40 0x55be5257c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.801+0000 7f0f3218d740 1 --2- 172.21.15.43:0/3959437761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be5257b250 0x55be5257b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.801+0000 7f0f3218d740 1 -- 172.21.15.43:0/3959437761 >> 172.21.15.43:0/3959437761 conn(0x55be52572f40 msgr2=0x55be52571490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:30.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.801+0000 7f0f3218d740 1 -- 172.21.15.43:0/3959437761 shutdown_connections 2024-04-21T08:36:30.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.801+0000 7f0f3218d740 1 -- 172.21.15.43:0/3959437761 wait complete. 2024-04-21T08:36:30.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.801+0000 7f0f3218d740 1 -- start start 2024-04-21T08:36:30.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.801+0000 7f0f3218d740 1 Processor -- start 2024-04-21T08:36:30.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.802+0000 7f0f3218d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be5257b250 0x55be52585260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.802+0000 7f0f3218d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be5257bc40 0x55be525857a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.802+0000 7f0f3218d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257c6e0 0x55be5258c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.802+0000 7f0f3218d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55be525838c0 con 0x55be5257c6e0 2024-04-21T08:36:30.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.802+0000 7f0f3218d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55be52464190 con 0x55be5257bc40 2024-04-21T08:36:30.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.802+0000 7f0f3218d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55be52583a40 con 0x55be5257b250 2024-04-21T08:36:30.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.802+0000 7f0f3218d740 10 client.0 _ll_register_callbacks cb 0x55be5245f8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:36:30.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.802+0000 7f0f2db3a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257c6e0 0x55be5258c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.802+0000 7f0f2db3a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257c6e0 0x55be5258c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60586/0 (socket says 172.21.15.43:60586) 2024-04-21T08:36:30.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.802+0000 7f0f2db3a640 1 -- 172.21.15.43:0/802940843 learned_addr learned my addr 172.21.15.43:0/802940843 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:30.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.802+0000 7f0f2d339640 1 --2- 172.21.15.43:0/802940843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be5257b250 0x55be52585260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.802+0000 7f0f2cb38640 1 --2- 172.21.15.43:0/802940843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be5257bc40 0x55be525857a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.802+0000 7f0f2db3a640 1 -- 172.21.15.43:0/802940843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be5257b250 msgr2=0x55be52585260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:36:30.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.802+0000 7f0f2db3a640 1 --2- 172.21.15.43:0/802940843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be5257b250 0x55be52585260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.802+0000 7f0f2db3a640 1 -- 172.21.15.43:0/802940843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be5257bc40 msgr2=0x55be525857a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:36:30.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.802+0000 7f0f2db3a640 1 --2- 172.21.15.43:0/802940843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be5257bc40 0x55be525857a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.802+0000 7f0f2db3a640 1 -- 172.21.15.43:0/802940843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be522b86b0 con 0x55be5257c6e0 2024-04-21T08:36:30.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.802+0000 7f0f2d339640 1 --2- 172.21.15.43:0/802940843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be5257b250 0x55be52585260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:30.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.802+0000 7f0f2cb38640 1 --2- 172.21.15.43:0/802940843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be5257bc40 0x55be525857a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:30.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.802+0000 7f0f2db3a640 1 --2- 172.21.15.43:0/802940843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257c6e0 0x55be5258c7e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0f180119c0 tx=0x7f0f18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d58ee5eddcb029 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:30.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.802+0000 7f0f0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:36:30.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.802+0000 7f0f0cff9640 1 -- 172.21.15.43:0/802940843 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f18024070 con 0x55be5257c6e0 2024-04-21T08:36:30.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.802+0000 7f0f0cff9640 1 -- 172.21.15.43:0/802940843 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0f1800fd80 con 0x55be5257c6e0 2024-04-21T08:36:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.802+0000 7f0f3218d740 1 -- 172.21.15.43:0/802940843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f1c017a70 con 0x55be5257c6e0 2024-04-21T08:36:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.803+0000 7f0f0cff9640 1 -- 172.21.15.43:0/802940843 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f0f1802f050 con 0x55be5257c6e0 2024-04-21T08:36:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.803+0000 7f0f3218d740 10 client.14766 fetch_fsmap learned FSMap version 16 2024-04-21T08:36:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.803+0000 7f0f3218d740 1 -- 172.21.15.43:0/802940843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x55be5258fb30 con 0x55be5257c6e0 2024-04-21T08:36:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.803+0000 7f0f0cff9640 1 -- 172.21.15.43:0/802940843 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f1c017a70 con 0x55be5257c6e0 2024-04-21T08:36:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.803+0000 7f0f0cff9640 1 -- 172.21.15.43:0/802940843 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 16) ==== 8342+0+0 (secure 0 0 0) 0x7f0f1801f070 con 0x55be5257c6e0 2024-04-21T08:36:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.803+0000 7f0f3218d740 10 client.14766 fetch_fsmap finished waiting for FSMap version 16 2024-04-21T08:36:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.803+0000 7f0f3218d740 10 client.14766 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T08:36:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.803+0000 7f0f3218d740 20 client.14766 populate_metadata read hostname 'smithi043' 2024-04-21T08:36:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.803+0000 7f0f3218d740 1 --2- 172.21.15.43:0/802940843 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55be52592250 0x55be525936c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.803+0000 7f0f3218d740 4 client.14766 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:36:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.803+0000 7f0f3218d740 1 -- 172.21.15.43:0/802940843 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be524e9530 con 0x55be52592250 2024-04-21T08:36:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.803+0000 7f0f2d339640 1 --2- 172.21.15.43:0/802940843 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55be52592250 0x55be525936c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.804+0000 7f0f2d339640 1 --2- 172.21.15.43:0/802940843 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55be52592250 0x55be525936c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6625e50b6b2ac8d9 server_cookie=c2351bb6508b7c4e in_seq=0 out_seq=0 2024-04-21T08:36:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.804+0000 7f0f0cff9640 10 client.14766 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:36:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.804+0000 7f0f0cff9640 1 -- 172.21.15.43:0/802940843 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55be524e9530 con 0x55be52592250 2024-04-21T08:36:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.804+0000 7f0f0cff9640 10 client.14766 handle_command_reply: tid=0 2024-04-21T08:36:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.804+0000 7f0f3218d740 1 client.14766 shutdown 2024-04-21T08:36:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.805+0000 7f0f3218d740 1 -- 172.21.15.43:0/802940843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257c6e0 msgr2=0x55be5258c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.805+0000 7f0f3218d740 1 --2- 172.21.15.43:0/802940843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257c6e0 0x55be5258c7e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0f180119c0 tx=0x7f0f18004a40 comp rx=0 tx=0).stop 2024-04-21T08:36:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.805+0000 7f0f3218d740 1 -- 172.21.15.43:0/802940843 shutdown_connections 2024-04-21T08:36:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.805+0000 7f0f3218d740 1 --2- 172.21.15.43:0/802940843 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55be52592250 0x55be525936c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.805+0000 7f0f3218d740 1 --2- 172.21.15.43:0/802940843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257c6e0 0x55be5258c7e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.805+0000 7f0f3218d740 1 --2- 172.21.15.43:0/802940843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be5257bc40 0x55be525857a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.805+0000 7f0f0cff9640 0 client.14766 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:36:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.805+0000 7f0f3218d740 1 --2- 172.21.15.43:0/802940843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be5257b250 0x55be52585260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.805+0000 7f0f3218d740 1 -- 172.21.15.43:0/802940843 >> 172.21.15.43:0/802940843 conn(0x55be52572f40 msgr2=0x55be52570a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:30.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.805+0000 7f0f3218d740 1 -- 172.21.15.43:0/802940843 shutdown_connections 2024-04-21T08:36:30.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.805+0000 7f0f3218d740 1 -- 172.21.15.43:0/802940843 wait complete. 2024-04-21T08:36:30.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.805+0000 7f0f3218d740 20 client.14766 trim_cache size 0 max 16384 2024-04-21T08:36:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.823+0000 7f0f3218d740 1 Processor -- start 2024-04-21T08:36:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.823+0000 7f0f3218d740 1 -- start start 2024-04-21T08:36:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.823+0000 7f0f3218d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be5257b250 0x55be5257fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.823+0000 7f0f3218d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be5257bc40 0x55be525800e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.823+0000 7f0f3218d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257c6e0 0x55be5258c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.823+0000 7f0f3218d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55be52591a00 con 0x55be5257c6e0 2024-04-21T08:36:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.823+0000 7f0f3218d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55be525920d0 con 0x55be5257b250 2024-04-21T08:36:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.823+0000 7f0f3218d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55be525927a0 con 0x55be5257bc40 2024-04-21T08:36:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.823+0000 7f0f2db3a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257c6e0 0x55be5258c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.823+0000 7f0f2db3a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257c6e0 0x55be5258c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60602/0 (socket says 172.21.15.43:60602) 2024-04-21T08:36:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.823+0000 7f0f2db3a640 1 -- 172.21.15.43:0/1814483372 learned_addr learned my addr 172.21.15.43:0/1814483372 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.824+0000 7f0f2cb38640 1 --2- 172.21.15.43:0/1814483372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be5257bc40 0x55be525800e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.824+0000 7f0f2d339640 1 --2- 172.21.15.43:0/1814483372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be5257b250 0x55be5257fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.824+0000 7f0f2db3a640 1 -- 172.21.15.43:0/1814483372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be5257bc40 msgr2=0x55be525800e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:36:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.824+0000 7f0f2db3a640 1 --2- 172.21.15.43:0/1814483372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be5257bc40 0x55be525800e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.824+0000 7f0f2db3a640 1 -- 172.21.15.43:0/1814483372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be5257b250 msgr2=0x55be5257fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:36:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.824+0000 7f0f2db3a640 1 --2- 172.21.15.43:0/1814483372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be5257b250 0x55be5257fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.824+0000 7f0f2db3a640 1 -- 172.21.15.43:0/1814483372 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be52591830 con 0x55be5257c6e0 2024-04-21T08:36:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.824+0000 7f0f2cb38640 1 --2- 172.21.15.43:0/1814483372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be5257bc40 0x55be525800e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.824+0000 7f0f2db3a640 1 --2- 172.21.15.43:0/1814483372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257c6e0 0x55be5258c7a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0f18005880 tx=0x7f0f18009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c010c03c9739dc7d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.824+0000 7f0f0effd640 1 -- 172.21.15.43:0/1814483372 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f18038070 con 0x55be5257c6e0 2024-04-21T08:36:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.824+0000 7f0f0effd640 1 -- 172.21.15.43:0/1814483372 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f1801a070 con 0x55be5257c6e0 2024-04-21T08:36:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.824+0000 7f0f0effd640 1 -- 172.21.15.43:0/1814483372 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f18024040 con 0x55be5257c6e0 2024-04-21T08:36:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.825+0000 7f0f3218d740 1 -- 172.21.15.43:0/1814483372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257c6e0 msgr2=0x55be5258c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.825+0000 7f0f3218d740 1 --2- 172.21.15.43:0/1814483372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257c6e0 0x55be5258c7a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0f18005880 tx=0x7f0f18009ac0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.825+0000 7f0f3218d740 1 -- 172.21.15.43:0/1814483372 shutdown_connections 2024-04-21T08:36:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.825+0000 7f0f3218d740 1 --2- 172.21.15.43:0/1814483372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257c6e0 0x55be5258c7a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.825+0000 7f0f3218d740 1 --2- 172.21.15.43:0/1814483372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be5257bc40 0x55be525800e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.825+0000 7f0f3218d740 1 --2- 172.21.15.43:0/1814483372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be5257b250 0x55be5257fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.825+0000 7f0f3218d740 1 -- 172.21.15.43:0/1814483372 >> 172.21.15.43:0/1814483372 conn(0x55be52572f40 msgr2=0x55be52585260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.825+0000 7f0f3218d740 1 -- 172.21.15.43:0/1814483372 shutdown_connections 2024-04-21T08:36:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.825+0000 7f0f3218d740 1 -- 172.21.15.43:0/1814483372 wait complete. 2024-04-21T08:36:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.825+0000 7f0f3218d740 1 -- start start 2024-04-21T08:36:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.825+0000 7f0f3218d740 1 Processor -- start 2024-04-21T08:36:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.825+0000 7f0f3218d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257b250 0x55be52588550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.825+0000 7f0f3218d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be5257bc40 0x55be52588a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.825+0000 7f0f3218d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be5257c6e0 0x55be5258b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.825+0000 7f0f3218d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55be52593540 con 0x55be5257b250 2024-04-21T08:36:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.825+0000 7f0f3218d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55be52592e70 con 0x55be5257c6e0 2024-04-21T08:36:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.825+0000 7f0f3218d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55be52593c10 con 0x55be5257bc40 2024-04-21T08:36:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.825+0000 7f0f3218d740 10 client.0 _ll_register_callbacks cb 0x7f0f18009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:36:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.826+0000 7f0f2d339640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257b250 0x55be52588550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.826+0000 7f0f2d339640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257b250 0x55be52588550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60610/0 (socket says 172.21.15.43:60610) 2024-04-21T08:36:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.826+0000 7f0f2d339640 1 -- 172.21.15.43:0/728245694 learned_addr learned my addr 172.21.15.43:0/728245694 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.826+0000 7f0f2cb38640 1 --2- 172.21.15.43:0/728245694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be5257bc40 0x55be52588a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.826+0000 7f0f2d339640 1 -- 172.21.15.43:0/728245694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be5257bc40 msgr2=0x55be52588a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:36:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.826+0000 7f0f2d339640 1 --2- 172.21.15.43:0/728245694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be5257bc40 0x55be52588a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.826+0000 7f0f2d339640 1 -- 172.21.15.43:0/728245694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be5257c6e0 msgr2=0x55be5258b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:36:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.826+0000 7f0f2d339640 1 --2- 172.21.15.43:0/728245694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be5257c6e0 0x55be5258b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.826+0000 7f0f2d339640 1 -- 172.21.15.43:0/728245694 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be5258bfe0 con 0x55be5257b250 2024-04-21T08:36:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.826+0000 7f0f2cb38640 1 --2- 172.21.15.43:0/728245694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be5257bc40 0x55be52588a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.826+0000 7f0f2d339640 1 --2- 172.21.15.43:0/728245694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257b250 0x55be52588550 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0f200bb4d0 tx=0x7f0f2002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84ff8e77381a36 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.826+0000 7f0f0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:36:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.826+0000 7f0f0cff9640 1 -- 172.21.15.43:0/728245694 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f200d7030 con 0x55be5257b250 2024-04-21T08:36:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.826+0000 7f0f0cff9640 1 -- 172.21.15.43:0/728245694 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0f200df0b0 con 0x55be5257b250 2024-04-21T08:36:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.826+0000 7f0f3218d740 1 -- 172.21.15.43:0/728245694 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f1801a070 con 0x55be5257b250 2024-04-21T08:36:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.826+0000 7f0f0cff9640 1 -- 172.21.15.43:0/728245694 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f0f200bf200 con 0x55be5257b250 2024-04-21T08:36:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.826+0000 7f0f3218d740 10 client.14772 fetch_fsmap learned FSMap version 16 2024-04-21T08:36:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.826+0000 7f0f3218d740 1 -- 172.21.15.43:0/728245694 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x55be525f8680 con 0x55be5257b250 2024-04-21T08:36:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.826+0000 7f0f0cff9640 1 -- 172.21.15.43:0/728245694 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f1801a070 con 0x55be5257b250 2024-04-21T08:36:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.827+0000 7f0f0cff9640 1 -- 172.21.15.43:0/728245694 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 16) ==== 8342+0+0 (secure 0 0 0) 0x7f0f200bb170 con 0x55be5257b250 2024-04-21T08:36:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.827+0000 7f0f3218d740 10 client.14772 fetch_fsmap finished waiting for FSMap version 16 2024-04-21T08:36:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.827+0000 7f0f3218d740 10 client.14772 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T08:36:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.827+0000 7f0f3218d740 20 client.14772 populate_metadata read hostname 'smithi043' 2024-04-21T08:36:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.827+0000 7f0f3218d740 1 --2- 172.21.15.43:0/728245694 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55be525fd310 0x55be525fc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.827+0000 7f0f3218d740 4 client.14772 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:36:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.827+0000 7f0f3218d740 1 -- 172.21.15.43:0/728245694 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "status"}) -- 0x55be524e33a0 con 0x55be525fd310 2024-04-21T08:36:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.827+0000 7f0f2cb38640 1 --2- 172.21.15.43:0/728245694 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55be525fd310 0x55be525fc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.827+0000 7f0f2cb38640 1 --2- 172.21.15.43:0/728245694 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55be525fd310 0x55be525fc020 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d898473a8896e8c server_cookie=5311a87c0c5383a6 in_seq=0 out_seq=0 2024-04-21T08:36:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.827+0000 7f0f0cff9640 10 client.14772 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:36:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.827+0000 7f0f0cff9640 1 -- 172.21.15.43:0/728245694 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55be524e33a0 con 0x55be525fd310 2024-04-21T08:36:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.827+0000 7f0f0cff9640 10 client.14772 handle_command_reply: tid=0 2024-04-21T08:36:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.827+0000 7f0f3218d740 1 client.14772 shutdown 2024-04-21T08:36:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.828+0000 7f0f3218d740 1 -- 172.21.15.43:0/728245694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257b250 msgr2=0x55be52588550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.828+0000 7f0f3218d740 1 --2- 172.21.15.43:0/728245694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257b250 0x55be52588550 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0f200bb4d0 tx=0x7f0f2002bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.828+0000 7f0f3218d740 1 -- 172.21.15.43:0/728245694 shutdown_connections 2024-04-21T08:36:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.828+0000 7f0f3218d740 1 --2- 172.21.15.43:0/728245694 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55be525fd310 0x55be525fc020 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.828+0000 7f0f3218d740 1 --2- 172.21.15.43:0/728245694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be5257c6e0 0x55be5258b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.828+0000 7f0f0cff9640 0 client.14772 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:36:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.828+0000 7f0f3218d740 1 --2- 172.21.15.43:0/728245694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be5257bc40 0x55be52588a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.828+0000 7f0f3218d740 1 --2- 172.21.15.43:0/728245694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be5257b250 0x55be52588550 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.828+0000 7f0f3218d740 1 -- 172.21.15.43:0/728245694 >> 172.21.15.43:0/728245694 conn(0x55be52572f40 msgr2=0x55be52585260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.828+0000 7f0f3218d740 1 -- 172.21.15.43:0/728245694 shutdown_connections 2024-04-21T08:36:30.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.828+0000 7f0f3218d740 1 -- 172.21.15.43:0/728245694 wait complete. 2024-04-21T08:36:30.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.828+0000 7f0f3218d740 20 client.14772 trim_cache size 0 max 16384 2024-04-21T08:36:30.835 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:36:30.835 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "d57a7d98-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:36:30.835 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-04-21T08:36:30.835 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24527, 2024-04-21T08:36:30.835 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-21T08:36:30.835 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-21T08:36:30.835 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-21T08:36:30.835 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 6.372832635, 2024-04-21T08:36:30.836 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 16, 2024-04-21T08:36:30.836 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 90, 2024-04-21T08:36:30.836 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T08:36:30.836 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 11.647881121999999 2024-04-21T08:36:30.836 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:36:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.828+0000 7f0f2f59d640 1 -- 172.21.15.43:0/2610408402 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0f100a3bb0 msgr2=0x7f0f100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.828+0000 7f0f2f59d640 1 --2- 172.21.15.43:0/2610408402 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0f100a3bb0 0x7f0f100a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009270 tx=0x7f0f18005bd0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.828+0000 7f0f2f59d640 1 -- 172.21.15.43:0/2610408402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f28101fa0 msgr2=0x7f0f28198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.828+0000 7f0f2f59d640 1 --2- 172.21.15.43:0/2610408402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f28101fa0 0x7f0f28198210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0f20000c00 tx=0x7f0f20004270 comp rx=0 tx=0).stop 2024-04-21T08:36:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.828+0000 7f0f2f59d640 1 -- 172.21.15.43:0/2610408402 shutdown_connections 2024-04-21T08:36:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.828+0000 7f0f2f59d640 1 --2- 172.21.15.43:0/2610408402 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0f100a3bb0 0x7f0f100a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.828+0000 7f0f2f59d640 1 --2- 172.21.15.43:0/2610408402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f28103b50 0x7f0f2819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.828+0000 7f0f2f59d640 1 --2- 172.21.15.43:0/2610408402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f28103190 0x7f0f28198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.828+0000 7f0f2f59d640 1 --2- 172.21.15.43:0/2610408402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f28101fa0 0x7f0f28198210 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.828+0000 7f0f2f59d640 1 -- 172.21.15.43:0/2610408402 >> 172.21.15.43:0/2610408402 conn(0x7f0f280fd530 msgr2=0x7f0f28106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.829+0000 7f0f2f59d640 1 -- 172.21.15.43:0/2610408402 shutdown_connections 2024-04-21T08:36:30.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:30.829+0000 7f0f2f59d640 1 -- 172.21.15.43:0/2610408402 wait complete. 2024-04-21T08:36:30.839 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-21T08:36:30.839 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-21T08:36:30.839 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph fs ls 2024-04-21T08:36:31.085 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:36:31.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:31 smithi112 ceph-mon[31085]: pgmap v298: 129 pgs: 112 active+clean, 17 unknown; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 9 op/s 2024-04-21T08:36:31.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:31 smithi112 ceph-mon[31085]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-21T08:36:31.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:31 smithi112 ceph-mon[31085]: Cluster is now healthy 2024-04-21T08:36:31.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:31 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1306934691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:31.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:31 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3742504367' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:31.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:31 smithi043 ceph-mon[27978]: pgmap v298: 129 pgs: 112 active+clean, 17 unknown; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 9 op/s 2024-04-21T08:36:31.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:31 smithi043 ceph-mon[27978]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-21T08:36:31.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:31 smithi043 ceph-mon[27978]: Cluster is now healthy 2024-04-21T08:36:31.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:31 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1306934691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:31.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:31 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3742504367' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:31.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:31 smithi175 ceph-mon[31920]: pgmap v298: 129 pgs: 112 active+clean, 17 unknown; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 9 op/s 2024-04-21T08:36:31.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:31 smithi175 ceph-mon[31920]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-21T08:36:31.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:31 smithi175 ceph-mon[31920]: Cluster is now healthy 2024-04-21T08:36:31.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:31 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1306934691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:31.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:31 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3742504367' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.795+0000 7f8f73d02700 1 -- 172.21.15.43:0/992935085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f6c101d20 msgr2=0x7f8f6c1021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.795+0000 7f8f73d02700 1 --2- 172.21.15.43:0/992935085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f6c101d20 0x7f8f6c1021a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009990 tx=0x7f8f5c009ca0 comp rx=0 tx=0).stop 2024-04-21T08:36:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.796+0000 7f8f73d02700 1 -- 172.21.15.43:0/992935085 shutdown_connections 2024-04-21T08:36:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.796+0000 7f8f73d02700 1 --2- 172.21.15.43:0/992935085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f6c1026e0 0x7f8f6c10e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.796+0000 7f8f73d02700 1 --2- 172.21.15.43:0/992935085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f6c106990 0x7f8f6c106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.796+0000 7f8f73d02700 1 --2- 172.21.15.43:0/992935085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f6c101d20 0x7f8f6c1021a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.796+0000 7f8f73d02700 1 -- 172.21.15.43:0/992935085 >> 172.21.15.43:0/992935085 conn(0x7f8f6c0788f0 msgr2=0x7f8f6c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.796+0000 7f8f73d02700 1 -- 172.21.15.43:0/992935085 shutdown_connections 2024-04-21T08:36:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.796+0000 7f8f73d02700 1 -- 172.21.15.43:0/992935085 wait complete. 2024-04-21T08:36:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.796+0000 7f8f73d02700 1 Processor -- start 2024-04-21T08:36:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.797+0000 7f8f73d02700 1 -- start start 2024-04-21T08:36:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.797+0000 7f8f73d02700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f6c101d20 0x7f8f6c199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.797+0000 7f8f73d02700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f6c1026e0 0x7f8f6c1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.797+0000 7f8f73d02700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f6c106990 0x7f8f6c19da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.797+0000 7f8f73d02700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c111b50 con 0x7f8f6c1026e0 2024-04-21T08:36:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.797+0000 7f8f73d02700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c1119d0 con 0x7f8f6c101d20 2024-04-21T08:36:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.797+0000 7f8f73d02700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c111cd0 con 0x7f8f6c106990 2024-04-21T08:36:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.797+0000 7f8f7129d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f6c1026e0 0x7f8f6c1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.797+0000 7f8f7129d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f6c1026e0 0x7f8f6c1996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60618/0 (socket says 172.21.15.43:60618) 2024-04-21T08:36:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.797+0000 7f8f7129d700 1 -- 172.21.15.43:0/90518785 learned_addr learned my addr 172.21.15.43:0/90518785 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.797+0000 7f8f71a9e700 1 --2- 172.21.15.43:0/90518785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f6c101d20 0x7f8f6c199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.797+0000 7f8f7229f700 1 --2- 172.21.15.43:0/90518785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f6c106990 0x7f8f6c19da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.797+0000 7f8f7129d700 1 -- 172.21.15.43:0/90518785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f6c106990 msgr2=0x7f8f6c19da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:32.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.797+0000 7f8f7129d700 1 --2- 172.21.15.43:0/90518785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f6c106990 0x7f8f6c19da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:32.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.797+0000 7f8f7129d700 1 -- 172.21.15.43:0/90518785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f6c101d20 msgr2=0x7f8f6c199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:32.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.797+0000 7f8f7129d700 1 --2- 172.21.15.43:0/90518785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f6c101d20 0x7f8f6c199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:32.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.797+0000 7f8f7129d700 1 -- 172.21.15.43:0/90518785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6c19e180 con 0x7f8f6c1026e0 2024-04-21T08:36:32.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.797+0000 7f8f7229f700 1 --2- 172.21.15.43:0/90518785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f6c106990 0x7f8f6c19da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:32.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.797+0000 7f8f71a9e700 1 --2- 172.21.15.43:0/90518785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f6c101d20 0x7f8f6c199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:32.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.798+0000 7f8f7129d700 1 --2- 172.21.15.43:0/90518785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f6c1026e0 0x7f8f6c1996b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c000c00 tx=0x7f8f5c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:32.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.798+0000 7f8f62ffd700 1 -- 172.21.15.43:0/90518785 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c027070 con 0x7f8f6c1026e0 2024-04-21T08:36:32.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.798+0000 7f8f73d02700 1 -- 172.21.15.43:0/90518785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6c19e410 con 0x7f8f6c1026e0 2024-04-21T08:36:32.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.798+0000 7f8f62ffd700 1 -- 172.21.15.43:0/90518785 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f5c003fa0 con 0x7f8f6c1026e0 2024-04-21T08:36:32.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.798+0000 7f8f62ffd700 1 -- 172.21.15.43:0/90518785 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c022db0 con 0x7f8f6c1026e0 2024-04-21T08:36:32.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.798+0000 7f8f73d02700 1 -- 172.21.15.43:0/90518785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f6c1a5cf0 con 0x7f8f6c1026e0 2024-04-21T08:36:32.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.799+0000 7f8f73d02700 1 -- 172.21.15.43:0/90518785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f6c103080 con 0x7f8f6c1026e0 2024-04-21T08:36:32.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.800+0000 7f8f62ffd700 1 -- 172.21.15.43:0/90518785 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8f5c02a070 con 0x7f8f6c1026e0 2024-04-21T08:36:32.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.801+0000 7f8f62ffd700 1 --2- 172.21.15.43:0/90518785 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8f580a3c40 0x7f8f580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:32.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.801+0000 7f8f71a9e700 1 --2- 172.21.15.43:0/90518785 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8f580a3c40 0x7f8f580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:32.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.801+0000 7f8f62ffd700 1 -- 172.21.15.43:0/90518785 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f8f5c0d1320 con 0x7f8f6c1026e0 2024-04-21T08:36:32.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.801+0000 7f8f71a9e700 1 --2- 172.21.15.43:0/90518785 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8f580a3c40 0x7f8f580a6100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8f680099d0 tx=0x7f8f68008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:32.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:32.805+0000 7f8f62ffd700 1 -- 172.21.15.43:0/90518785 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f5c013070 con 0x7f8f6c1026e0 2024-04-21T08:36:33.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:33 smithi043 ceph-mon[27978]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-04-21T08:36:33.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:33.062+0000 7f8f73d02700 1 -- 172.21.15.43:0/90518785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8f6c19a4d0 con 0x7f8f6c1026e0 2024-04-21T08:36:33.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:33.063+0000 7f8f62ffd700 1 -- 172.21.15.43:0/90518785 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+88 (secure 0 0 0) 0x7f8f5c09db50 con 0x7f8f6c1026e0 2024-04-21T08:36:33.062 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-21T08:36:33.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:33.065+0000 7f8f73d02700 1 -- 172.21.15.43:0/90518785 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8f580a3c40 msgr2=0x7f8f580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:33.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:33.065+0000 7f8f73d02700 1 --2- 172.21.15.43:0/90518785 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8f580a3c40 0x7f8f580a6100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8f680099d0 tx=0x7f8f68008040 comp rx=0 tx=0).stop 2024-04-21T08:36:33.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:33.065+0000 7f8f73d02700 1 -- 172.21.15.43:0/90518785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f6c1026e0 msgr2=0x7f8f6c1996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:33.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:33.065+0000 7f8f73d02700 1 --2- 172.21.15.43:0/90518785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f6c1026e0 0x7f8f6c1996b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c000c00 tx=0x7f8f5c0047b0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:33.065+0000 7f8f73d02700 1 -- 172.21.15.43:0/90518785 shutdown_connections 2024-04-21T08:36:33.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:33.065+0000 7f8f73d02700 1 --2- 172.21.15.43:0/90518785 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8f580a3c40 0x7f8f580a6100 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:33.065+0000 7f8f73d02700 1 --2- 172.21.15.43:0/90518785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f6c106990 0x7f8f6c19da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:33.065+0000 7f8f73d02700 1 --2- 172.21.15.43:0/90518785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f6c101d20 0x7f8f6c199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:33.065+0000 7f8f73d02700 1 --2- 172.21.15.43:0/90518785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f6c1026e0 0x7f8f6c1996b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:33.065+0000 7f8f73d02700 1 -- 172.21.15.43:0/90518785 >> 172.21.15.43:0/90518785 conn(0x7f8f6c0788f0 msgr2=0x7f8f6c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:33.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:33.065+0000 7f8f73d02700 1 -- 172.21.15.43:0/90518785 shutdown_connections 2024-04-21T08:36:33.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:33.065+0000 7f8f73d02700 1 -- 172.21.15.43:0/90518785 wait complete. 2024-04-21T08:36:33.293 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:33 smithi112 ceph-mon[31085]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-04-21T08:36:33.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:33 smithi175 ceph-mon[31920]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-04-21T08:36:33.817 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-21T08:36:33.817 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-21T08:36:33.817 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-04-21T08:36:33.817 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-21T08:36:33.817 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-21T08:36:33.817 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-21T08:36:33.817 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-21T08:36:33.817 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-21T08:36:33.818 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-21T08:36:33.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:36:33.818 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-04-21T08:36:33.835 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-21T08:36:33.835 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-21T08:36:33.835 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-04-21T08:36:33.835 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-21T08:36:33.835 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-04-21T08:36:33.835 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-21T08:36:33.835 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T08:36:33.835 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-04-21T08:36:33.835 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-04-21T08:36:33.835 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T08:36:33.835 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-04-21T08:36:33.835 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-04-21T08:36:33.836 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-21T08:36:33.836 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-04-21T08:36:33.836 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-04-21T08:36:33.836 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-21T08:36:33.836 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42172sec preferred_lft 42172sec 2024-04-21T08:36:33.836 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-04-21T08:36:33.836 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-21T08:36:33.836 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T08:36:33.836 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-04-21T08:36:33.836 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-04-21T08:36:33.837 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-21T08:36:33.837 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:36:33.837 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-21T08:36:33.837 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add name ceph-brx type bridge 2024-04-21T08:36:33.837 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr flush dev ceph-brx 2024-04-21T08:36:33.837 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set ceph-brx up 2024-04-21T08:36:33.837 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-21T08:36:33.837 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-21T08:36:33.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:36:34.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:36:34.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:36:34.022 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-21T08:36:34.037 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:34 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/90518785' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T08:36:34.048 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-04-21T08:36:34.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:36:34.050 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-04-21T08:36:34.178 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-04-21T08:36:34.178 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-21T08:36:34.178 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-21T08:36:34.178 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-21T08:36:34.178 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-21T08:36:34.179 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:36:34.179 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-21T08:36:34.179 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-21T08:36:34.179 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-21T08:36:34.179 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-21T08:36:34.180 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-21T08:36:34.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:36:34.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:34 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/90518785' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T08:36:34.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:36:34.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:36:34.414 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-21T08:36:34.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:36:34.430 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list-id 2024-04-21T08:36:34.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:34 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/90518785' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T08:36:34.485 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:36:34.485 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-21T08:36:34.485 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-21T08:36:34.485 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-21T08:36:34.486 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-21T08:36:34.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:36:34.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:36:34.604 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-21T08:36:34.604 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:36:34.604 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-21T08:36:34.605 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-21T08:36:34.605 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-21T08:36:34.605 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-21T08:36:34.605 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-21T08:36:34.605 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-21T08:36:34.605 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-21T08:36:34.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:36:34.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:36:34.731 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:36:34.731 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-21T08:36:34.731 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-04-21T08:36:34.731 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set dev brx.0 master ceph-brx 2024-04-21T08:36:34.731 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-21T08:36:34.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:36:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:36:34.808 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-21T08:36:34.808 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-21T08:36:34.808 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-21T08:36:34.828 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-21T08:36:34.828 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-21T08:36:34.828 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-21T08:36:34.885 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-21T08:36:34.950 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-21T08:36:35.006 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-21T08:36:35.006 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-21T08:36:35.006 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-21T08:36:35.006 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-21T08:36:35.006 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-21T08:36:35.006 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-21T08:36:35.006 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-21T08:36:35.007 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-21T08:36:35.007 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-21T08:36:35.007 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-21T08:36:35.007 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-21T08:36:35.007 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-21T08:36:35.007 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-21T08:36:35.007 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-21T08:36:35.007 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-21T08:36:35.007 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-21T08:36:35.007 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-21T08:36:35.007 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-21T08:36:35.007 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-21T08:36:35.008 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-21T08:36:35.008 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-21T08:36:35.008 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-21T08:36:35.008 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-21T08:36:35.008 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-21T08:36:35.008 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-21T08:36:35.008 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-21T08:36:35.008 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay 2024-04-21T08:36:35.008 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/b08fddaba5765ba87b548af2c196ded4fb07a2f2e1038d0e4632e5ab3ec67b84/merged 2024-04-21T08:36:35.008 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/9e5a6cb4ef143bb6438a72bf3c8816dcede626b6eea575f1da94354d8ec6f0eb/merged 2024-04-21T08:36:35.008 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/0 2024-04-21T08:36:35.009 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-21T08:36:35.009 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/9dd6b0958e358768394bc17b4abf868f2d1b0a2ce5d0b1b84a4b2d5b7ded8a7b/merged 2024-04-21T08:36:35.009 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/6a22612a34f2d841c5cbbc80abe942b562af933c31ead184189df69a824ba9dd/merged 2024-04-21T08:36:35.009 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/4e0ed9591beba11b4255655ea5e24f3c8dd7e9384c15db4f64cb6ff543528bc2/merged 2024-04-21T08:36:35.009 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/8cb5d07af860c312b7cb9e2be97c9cdfd861c608a19c27d69a471a0a0bce8255/merged 2024-04-21T08:36:35.009 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/b9f083ffb478b4d3c9f0d2c3c632fba97773098b47382b6ff398ca387587e451/merged 2024-04-21T08:36:35.009 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/0c7cabb1986c131ba65b5f5c27dea23c3fe6d0601dfb429c3f6f9486700a73b3/merged 2024-04-21T08:36:35.009 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/b873585374a1b6d64e64d0b4d813089c25bf3bac08e643efc98f3eec6c6d90b3/merged 2024-04-21T08:36:35.009 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/ef28d4e35d4092323b098b4349965dd1e24a2b0afd28dcf34148df9fbd164ec6/merged 2024-04-21T08:36:35.009 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-21T08:36:35.009 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-21T08:36:35.009 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-21T08:36:35.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:36:35.010 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-21T08:36:35.061 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-21T08:36:35.062 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-21T08:36:35.104 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-21T08:36:35.128 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-21T08:36:35.184 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-21T08:36:35.185 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-21T08:36:35.185 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-21T08:36:35.185 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-21T08:36:35.185 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-21T08:36:35.185 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-21T08:36:35.185 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-21T08:36:35.185 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-21T08:36:35.185 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-21T08:36:35.185 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-21T08:36:35.185 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-21T08:36:35.186 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-21T08:36:35.186 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-21T08:36:35.186 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-21T08:36:35.186 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-21T08:36:35.186 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-21T08:36:35.186 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-21T08:36:35.186 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-21T08:36:35.186 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-21T08:36:35.186 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-21T08:36:35.186 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-21T08:36:35.186 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-21T08:36:35.186 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-21T08:36:35.186 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-21T08:36:35.187 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-21T08:36:35.187 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-21T08:36:35.187 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay 2024-04-21T08:36:35.187 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/b08fddaba5765ba87b548af2c196ded4fb07a2f2e1038d0e4632e5ab3ec67b84/merged 2024-04-21T08:36:35.187 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/9e5a6cb4ef143bb6438a72bf3c8816dcede626b6eea575f1da94354d8ec6f0eb/merged 2024-04-21T08:36:35.187 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/0 2024-04-21T08:36:35.187 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-21T08:36:35.187 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/9dd6b0958e358768394bc17b4abf868f2d1b0a2ce5d0b1b84a4b2d5b7ded8a7b/merged 2024-04-21T08:36:35.187 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/6a22612a34f2d841c5cbbc80abe942b562af933c31ead184189df69a824ba9dd/merged 2024-04-21T08:36:35.187 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/4e0ed9591beba11b4255655ea5e24f3c8dd7e9384c15db4f64cb6ff543528bc2/merged 2024-04-21T08:36:35.187 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/8cb5d07af860c312b7cb9e2be97c9cdfd861c608a19c27d69a471a0a0bce8255/merged 2024-04-21T08:36:35.187 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/b9f083ffb478b4d3c9f0d2c3c632fba97773098b47382b6ff398ca387587e451/merged 2024-04-21T08:36:35.187 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/0c7cabb1986c131ba65b5f5c27dea23c3fe6d0601dfb429c3f6f9486700a73b3/merged 2024-04-21T08:36:35.188 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/b873585374a1b6d64e64d0b4d813089c25bf3bac08e643efc98f3eec6c6d90b3/merged 2024-04-21T08:36:35.188 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/ef28d4e35d4092323b098b4349965dd1e24a2b0afd28dcf34148df9fbd164ec6/merged 2024-04-21T08:36:35.188 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-21T08:36:35.188 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-21T08:36:35.188 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-21T08:36:35.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:36:35.188 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-21T08:36:35.210 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi043.stderr:2024-04-21T08:36:35.211+0000 7fbb20ad3480 -1 init, newargv = 0x55a971896e60 newargc=15 2024-04-21T08:36:35.210 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi043.stderr:ceph-fuse[69663]: starting ceph client 2024-04-21T08:36:35.216 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi043.stderr:ceph-fuse[69663]: starting fuse 2024-04-21T08:36:35.225 INFO:teuthology.orchestra.run.smithi043.stdout:115 2024-04-21T08:36:35.226 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [115] 2024-04-21T08:36:35.226 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2024-04-21T08:36:35.226 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2024-04-21T08:36:35.226 DEBUG:teuthology.orchestra.run.smithi043:> import re 2024-04-21T08:36:35.226 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-04-21T08:36:35.226 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2024-04-21T08:36:35.226 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-21T08:36:35.226 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2024-04-21T08:36:35.226 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-21T08:36:35.226 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2024-04-21T08:36:35.227 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-21T08:36:35.227 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-21T08:36:35.227 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2024-04-21T08:36:35.227 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2024-04-21T08:36:35.227 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2024-04-21T08:36:35.227 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2024-04-21T08:36:35.227 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-21T08:36:35.227 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2024-04-21T08:36:35.227 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-21T08:36:35.227 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-21T08:36:35.227 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-21T08:36:35.227 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2024-04-21T08:36:35.227 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2024-04-21T08:36:35.227 DEBUG:teuthology.orchestra.run.smithi043:> return f 2024-04-21T08:36:35.227 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-21T08:36:35.228 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-21T08:36:35.228 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.admin")) 2024-04-21T08:36:35.228 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-04-21T08:36:35.286 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:35 smithi043 ceph-mon[27978]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 585 B/s wr, 4 op/s 2024-04-21T08:36:35.293 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:35 smithi112 ceph-mon[31085]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 585 B/s wr, 4 op/s 2024-04-21T08:36:35.346 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.admin.69663.asok 2024-04-21T08:36:35.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:36:35.357 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.69663.asok 2024-04-21T08:36:35.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:36:35.357 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.69663.asok status 2024-04-21T08:36:35.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:35 smithi175 ceph-mon[31920]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 585 B/s wr, 4 op/s 2024-04-21T08:36:35.496 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:36:35.496 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2024-04-21T08:36:35.496 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "a630765c1f93c6e61d9b40c4b76437611d5ef29e", 2024-04-21T08:36:35.496 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev)", 2024-04-21T08:36:35.496 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "admin", 2024-04-21T08:36:35.497 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2024-04-21T08:36:35.497 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-21T08:36:35.497 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "69663", 2024-04-21T08:36:35.497 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2024-04-21T08:36:35.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:36:35.497 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2024-04-21T08:36:35.497 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2024-04-21T08:36:35.497 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 14784, 2024-04-21T08:36:35.497 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2024-04-21T08:36:35.497 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2024-04-21T08:36:35.497 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2024-04-21T08:36:35.497 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 14784 2024-04-21T08:36:35.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:36:35.498 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-21T08:36:35.498 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-21T08:36:35.498 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-21T08:36:35.498 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 4189640310 2024-04-21T08:36:35.498 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:36:35.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:36:35.498 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-21T08:36:35.498 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-21T08:36:35.498 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-21T08:36:35.498 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 4189640310 2024-04-21T08:36:35.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:36:35.498 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.14784 192.168.0.1:0/4189640310", 2024-04-21T08:36:35.499 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/4189640310", 2024-04-21T08:36:35.499 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2024-04-21T08:36:35.499 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 16, 2024-04-21T08:36:35.499 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 90, 2024-04-21T08:36:35.499 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2024-04-21T08:36:35.499 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2024-04-21T08:36:35.499 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2024-04-21T08:36:35.499 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:36:35.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:36:35.506 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2024-04-21T08:36:35.506 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-21T08:36:35.522 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2024-04-21T08:36:35.522 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-21T08:36:35.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:36:35.522 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-21T08:36:35.588 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-21T08:36:35.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-21T08:36:35.701 INFO:teuthology.orchestra.run.smithi043.stdout:# file: . 2024-04-21T08:36:35.701 INFO:teuthology.orchestra.run.smithi043.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-21T08:36:35.701 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:35.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:36:35.708 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-21T08:36:35.708 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T08:36:35.725 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:36:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.868+0000 7f44c1123640 1 -- 172.21.15.43:0/3328310597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44bc100e70 msgr2=0x7f44bc101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.868+0000 7f44c1123640 1 --2- 172.21.15.43:0/3328310597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44bc100e70 0x7f44bc101290 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009900 tx=0x7f44b002bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.868+0000 7f44c1123640 1 -- 172.21.15.43:0/3328310597 shutdown_connections 2024-04-21T08:36:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.868+0000 7f44c1123640 1 --2- 172.21.15.43:0/3328310597 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44bc102980 0x7f44bc1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.868+0000 7f44c1123640 1 --2- 172.21.15.43:0/3328310597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44bc101fc0 0x7f44bc102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.868+0000 7f44c1123640 1 --2- 172.21.15.43:0/3328310597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44bc100e70 0x7f44bc101290 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.868+0000 7f44c1123640 1 -- 172.21.15.43:0/3328310597 >> 172.21.15.43:0/3328310597 conn(0x7f44bc0fc3e0 msgr2=0x7f44bc0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.868+0000 7f44c1123640 1 -- 172.21.15.43:0/3328310597 shutdown_connections 2024-04-21T08:36:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.868+0000 7f44c1123640 1 -- 172.21.15.43:0/3328310597 wait complete. 2024-04-21T08:36:35.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.869+0000 7f44c1123640 1 Processor -- start 2024-04-21T08:36:35.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.869+0000 7f44c1123640 1 -- start start 2024-04-21T08:36:35.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.869+0000 7f44c1123640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44bc100e70 0x7f44bc104df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:35.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.869+0000 7f44c1123640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44bc101fc0 0x7f44bc103430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.869+0000 7f44c1123640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44bc102980 0x7f44bc103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.869+0000 7f44c1123640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc10c9c0 con 0x7f44bc100e70 2024-04-21T08:36:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.869+0000 7f44c1123640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc10c840 con 0x7f44bc101fc0 2024-04-21T08:36:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.869+0000 7f44c1123640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc10cb40 con 0x7f44bc102980 2024-04-21T08:36:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.870+0000 7f44bad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44bc100e70 0x7f44bc104df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.870+0000 7f44bad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44bc100e70 0x7f44bc104df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60642/0 (socket says 172.21.15.43:60642) 2024-04-21T08:36:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.870+0000 7f44bad9d640 1 -- 172.21.15.43:0/3650013205 learned_addr learned my addr 172.21.15.43:0/3650013205 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.870+0000 7f44bb59e640 1 --2- 172.21.15.43:0/3650013205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44bc102980 0x7f44bc103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.870+0000 7f44ba59c640 1 --2- 172.21.15.43:0/3650013205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44bc101fc0 0x7f44bc103430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.870+0000 7f44bad9d640 1 -- 172.21.15.43:0/3650013205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44bc102980 msgr2=0x7f44bc103970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.870+0000 7f44bad9d640 1 --2- 172.21.15.43:0/3650013205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44bc102980 0x7f44bc103970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.870+0000 7f44bad9d640 1 -- 172.21.15.43:0/3650013205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44bc101fc0 msgr2=0x7f44bc103430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:35.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.870+0000 7f44bad9d640 1 --2- 172.21.15.43:0/3650013205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44bc101fc0 0x7f44bc103430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.870+0000 7f44bad9d640 1 -- 172.21.15.43:0/3650013205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44bc104200 con 0x7f44bc100e70 2024-04-21T08:36:35.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.870+0000 7f44bad9d640 1 --2- 172.21.15.43:0/3650013205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44bc100e70 0x7f44bc104df0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f44b0006010 tx=0x7f44b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:35.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.870+0000 7f44a3fff640 1 -- 172.21.15.43:0/3650013205 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b0043020 con 0x7f44bc100e70 2024-04-21T08:36:35.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.870+0000 7f44a3fff640 1 -- 172.21.15.43:0/3650013205 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b0004aa0 con 0x7f44bc100e70 2024-04-21T08:36:35.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.870+0000 7f44c1123640 1 -- 172.21.15.43:0/3650013205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44bc1a22d0 con 0x7f44bc100e70 2024-04-21T08:36:35.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.870+0000 7f44a3fff640 1 -- 172.21.15.43:0/3650013205 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b0003c70 con 0x7f44bc100e70 2024-04-21T08:36:35.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.870+0000 7f44c1123640 1 -- 172.21.15.43:0/3650013205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44bc1a2820 con 0x7f44bc100e70 2024-04-21T08:36:35.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.872+0000 7f44c1123640 1 -- 172.21.15.43:0/3650013205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44bc1a2a20 con 0x7f44bc100e70 2024-04-21T08:36:35.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.874+0000 7f44a3fff640 1 -- 172.21.15.43:0/3650013205 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f44b0046020 con 0x7f44bc100e70 2024-04-21T08:36:35.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.874+0000 7f44a3fff640 1 --2- 172.21.15.43:0/3650013205 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f44a40a3bb0 0x7f44a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:35.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.875+0000 7f44ba59c640 1 --2- 172.21.15.43:0/3650013205 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f44a40a3bb0 0x7f44a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:35.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.875+0000 7f44a3fff640 1 -- 172.21.15.43:0/3650013205 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f44b00f3090 con 0x7f44bc100e70 2024-04-21T08:36:35.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.875+0000 7f44ba59c640 1 --2- 172.21.15.43:0/3650013205 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f44a40a3bb0 0x7f44a40a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f44ac005fd0 tx=0x7f44ac005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:35.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:35.878+0000 7f44a3fff640 1 -- 172.21.15.43:0/3650013205 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44b004d020 con 0x7f44bc100e70 2024-04-21T08:36:36.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.072+0000 7f44c1123640 1 -- 172.21.15.43:0/3650013205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44bc1a2bf0 con 0x7f44bc100e70 2024-04-21T08:36:36.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.074+0000 7f44a3fff640 1 -- 172.21.15.43:0/3650013205 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24152 (secure 0 0 0) 0x7f44b00bf830 con 0x7f44bc100e70 2024-04-21T08:36:36.073 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:36.073 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":90,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:36:28.364093+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:36:36.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.076+0000 7f44c1123640 1 -- 172.21.15.43:0/3650013205 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f44a40a3bb0 msgr2=0x7f44a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:36.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.076+0000 7f44c1123640 1 --2- 172.21.15.43:0/3650013205 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f44a40a3bb0 0x7f44a40a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f44ac005fd0 tx=0x7f44ac005850 comp rx=0 tx=0).stop 2024-04-21T08:36:36.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.076+0000 7f44c1123640 1 -- 172.21.15.43:0/3650013205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44bc100e70 msgr2=0x7f44bc104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:36.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.076+0000 7f44c1123640 1 --2- 172.21.15.43:0/3650013205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44bc100e70 0x7f44bc104df0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f44b0006010 tx=0x7f44b00042f0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.076+0000 7f44c1123640 1 -- 172.21.15.43:0/3650013205 shutdown_connections 2024-04-21T08:36:36.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.076+0000 7f44c1123640 1 --2- 172.21.15.43:0/3650013205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44bc102980 0x7f44bc103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.076+0000 7f44c1123640 1 --2- 172.21.15.43:0/3650013205 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f44a40a3bb0 0x7f44a40a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.076+0000 7f44c1123640 1 --2- 172.21.15.43:0/3650013205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44bc101fc0 0x7f44bc103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.076+0000 7f44c1123640 1 --2- 172.21.15.43:0/3650013205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44bc100e70 0x7f44bc104df0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.076+0000 7f44c1123640 1 -- 172.21.15.43:0/3650013205 >> 172.21.15.43:0/3650013205 conn(0x7f44bc0fc3e0 msgr2=0x7f44bc0fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:36.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.076+0000 7f44c1123640 1 -- 172.21.15.43:0/3650013205 shutdown_connections 2024-04-21T08:36:36.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.076+0000 7f44c1123640 1 -- 172.21.15.43:0/3650013205 wait complete. 2024-04-21T08:36:36.087 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:39.981405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:22.034751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:22.780795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:25.984261+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:36:36.087 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:36:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.195+0000 7f7930f15640 1 -- 172.21.15.43:0/459441391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f792c1029a0 msgr2=0x7f792c1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.195+0000 7f7930f15640 1 --2- 172.21.15.43:0/459441391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f792c1029a0 0x7f792c1092c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7924009ca0 tx=0x7f792402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.195+0000 7f7930f15640 1 -- 172.21.15.43:0/459441391 shutdown_connections 2024-04-21T08:36:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.195+0000 7f7930f15640 1 --2- 172.21.15.43:0/459441391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f792c1029a0 0x7f792c1092c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.195+0000 7f7930f15640 1 --2- 172.21.15.43:0/459441391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f792c101fe0 0x7f792c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.195+0000 7f7930f15640 1 --2- 172.21.15.43:0/459441391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f792c100e90 0x7f792c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.195+0000 7f7930f15640 1 -- 172.21.15.43:0/459441391 >> 172.21.15.43:0/459441391 conn(0x7f792c0fc400 msgr2=0x7f792c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.195+0000 7f7930f15640 1 -- 172.21.15.43:0/459441391 shutdown_connections 2024-04-21T08:36:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.195+0000 7f7930f15640 1 -- 172.21.15.43:0/459441391 wait complete. 2024-04-21T08:36:36.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.196+0000 7f7930f15640 1 Processor -- start 2024-04-21T08:36:36.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.196+0000 7f7930f15640 1 -- start start 2024-04-21T08:36:36.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.197+0000 7f7930f15640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f792c100e90 0x7f792c198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:36.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.197+0000 7f7930f15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f792c101fe0 0x7f792c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:36.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.197+0000 7f7930f15640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f792c1029a0 0x7f792c19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:36.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.197+0000 7f7930f15640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f792c10c9f0 con 0x7f792c100e90 2024-04-21T08:36:36.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.197+0000 7f7930f15640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f792c10c870 con 0x7f792c101fe0 2024-04-21T08:36:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.197+0000 7f7930f15640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f792c10cb70 con 0x7f792c1029a0 2024-04-21T08:36:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.197+0000 7f792a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f792c100e90 0x7f792c198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.197+0000 7f792a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f792c100e90 0x7f792c198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60652/0 (socket says 172.21.15.43:60652) 2024-04-21T08:36:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.197+0000 7f792a59c640 1 -- 172.21.15.43:0/2119677997 learned_addr learned my addr 172.21.15.43:0/2119677997 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.197+0000 7f792ad9d640 1 --2- 172.21.15.43:0/2119677997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f792c1029a0 0x7f792c19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.197+0000 7f7929d9b640 1 --2- 172.21.15.43:0/2119677997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f792c101fe0 0x7f792c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.197+0000 7f792a59c640 1 -- 172.21.15.43:0/2119677997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f792c1029a0 msgr2=0x7f792c19f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.197+0000 7f792a59c640 1 --2- 172.21.15.43:0/2119677997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f792c1029a0 0x7f792c19f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.197+0000 7f792a59c640 1 -- 172.21.15.43:0/2119677997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f792c101fe0 msgr2=0x7f792c198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.197+0000 7f792a59c640 1 --2- 172.21.15.43:0/2119677997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f792c101fe0 0x7f792c198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.197+0000 7f792a59c640 1 -- 172.21.15.43:0/2119677997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792c19fe20 con 0x7f792c100e90 2024-04-21T08:36:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.197+0000 7f7929d9b640 1 --2- 172.21.15.43:0/2119677997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f792c101fe0 0x7f792c198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.197+0000 7f792a59c640 1 --2- 172.21.15.43:0/2119677997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f792c100e90 0x7f792c198200 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f791c00d770 tx=0x7f791c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:36.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.197+0000 7f79237fe640 1 -- 172.21.15.43:0/2119677997 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c017070 con 0x7f792c100e90 2024-04-21T08:36:36.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.197+0000 7f79237fe640 1 -- 172.21.15.43:0/2119677997 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f791c0054d0 con 0x7f792c100e90 2024-04-21T08:36:36.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.197+0000 7f7930f15640 1 -- 172.21.15.43:0/2119677997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792c1a0110 con 0x7f792c100e90 2024-04-21T08:36:36.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.198+0000 7f79237fe640 1 -- 172.21.15.43:0/2119677997 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c004c30 con 0x7f792c100e90 2024-04-21T08:36:36.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.198+0000 7f7930f15640 1 -- 172.21.15.43:0/2119677997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f792c1a0610 con 0x7f792c100e90 2024-04-21T08:36:36.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.198+0000 7f7930f15640 1 -- 172.21.15.43:0/2119677997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f792c1a0870 con 0x7f792c100e90 2024-04-21T08:36:36.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.200+0000 7f79237fe640 1 -- 172.21.15.43:0/2119677997 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f791c004dd0 con 0x7f792c100e90 2024-04-21T08:36:36.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.200+0000 7f79237fe640 1 --2- 172.21.15.43:0/2119677997 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f79180a3bb0 0x7f79180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:36.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.200+0000 7f7929d9b640 1 --2- 172.21.15.43:0/2119677997 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f79180a3bb0 0x7f79180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:36.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.200+0000 7f79237fe640 1 -- 172.21.15.43:0/2119677997 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f791c0d05f0 con 0x7f792c100e90 2024-04-21T08:36:36.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.201+0000 7f7929d9b640 1 --2- 172.21.15.43:0/2119677997 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f79180a3bb0 0x7f79180a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f792c199710 tx=0x7f7914005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:36.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.208+0000 7f79237fe640 1 -- 172.21.15.43:0/2119677997 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f791c09cd90 con 0x7f792c100e90 2024-04-21T08:36:36.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.395+0000 7f7930f15640 1 -- 172.21.15.43:0/2119677997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f792c1a0a40 con 0x7f792c100e90 2024-04-21T08:36:36.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.395+0000 7f79237fe640 1 -- 172.21.15.43:0/2119677997 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f791c09c690 con 0x7f792c100e90 2024-04-21T08:36:36.395 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-21T08:36:36.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.397+0000 7f7930f15640 1 -- 172.21.15.43:0/2119677997 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f79180a3bb0 msgr2=0x7f79180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:36.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.397+0000 7f7930f15640 1 --2- 172.21.15.43:0/2119677997 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f79180a3bb0 0x7f79180a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f792c199710 tx=0x7f7914005f90 comp rx=0 tx=0).stop 2024-04-21T08:36:36.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.397+0000 7f7930f15640 1 -- 172.21.15.43:0/2119677997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f792c100e90 msgr2=0x7f792c198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:36.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.397+0000 7f7930f15640 1 --2- 172.21.15.43:0/2119677997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f792c100e90 0x7f792c198200 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f791c00d770 tx=0x7f791c00dc30 comp rx=0 tx=0).stop 2024-04-21T08:36:36.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.397+0000 7f7930f15640 1 -- 172.21.15.43:0/2119677997 shutdown_connections 2024-04-21T08:36:36.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.397+0000 7f7930f15640 1 --2- 172.21.15.43:0/2119677997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f792c1029a0 0x7f792c19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.397+0000 7f7930f15640 1 --2- 172.21.15.43:0/2119677997 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f79180a3bb0 0x7f79180a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.397+0000 7f7930f15640 1 --2- 172.21.15.43:0/2119677997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f792c101fe0 0x7f792c198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.397+0000 7f7930f15640 1 --2- 172.21.15.43:0/2119677997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f792c100e90 0x7f792c198200 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.397+0000 7f7930f15640 1 -- 172.21.15.43:0/2119677997 >> 172.21.15.43:0/2119677997 conn(0x7f792c0fc400 msgr2=0x7f792c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:36.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.397+0000 7f7930f15640 1 -- 172.21.15.43:0/2119677997 shutdown_connections 2024-04-21T08:36:36.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.398+0000 7f7930f15640 1 -- 172.21.15.43:0/2119677997 wait complete. 2024-04-21T08:36:36.408 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:36:36.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.514+0000 7f0cdc2fd640 1 -- 172.21.15.43:0/1280862611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd4100880 msgr2=0x7f0cd4100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:36.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.514+0000 7f0cdc2fd640 1 --2- 172.21.15.43:0/1280862611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd4100880 0x7f0cd4100ca0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009900 tx=0x7f0cc402bcd0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.514+0000 7f0cdc2fd640 1 -- 172.21.15.43:0/1280862611 shutdown_connections 2024-04-21T08:36:36.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.514+0000 7f0cdc2fd640 1 --2- 172.21.15.43:0/1280862611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cd4101b80 0x7f0cd4102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.514+0000 7f0cdc2fd640 1 --2- 172.21.15.43:0/1280862611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cd41011e0 0x7f0cd4101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.514+0000 7f0cdc2fd640 1 --2- 172.21.15.43:0/1280862611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd4100880 0x7f0cd4100ca0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.514+0000 7f0cdc2fd640 1 -- 172.21.15.43:0/1280862611 >> 172.21.15.43:0/1280862611 conn(0x7f0cd40fc400 msgr2=0x7f0cd40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:36.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.515+0000 7f0cdc2fd640 1 -- 172.21.15.43:0/1280862611 shutdown_connections 2024-04-21T08:36:36.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.515+0000 7f0cdc2fd640 1 -- 172.21.15.43:0/1280862611 wait complete. 2024-04-21T08:36:36.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.515+0000 7f0cdc2fd640 1 Processor -- start 2024-04-21T08:36:36.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.515+0000 7f0cdc2fd640 1 -- start start 2024-04-21T08:36:36.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.515+0000 7f0cdc2fd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd4100880 0x7f0cd419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:36.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.515+0000 7f0cdc2fd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cd41011e0 0x7f0cd419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:36.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.515+0000 7f0cdc2fd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cd4101b80 0x7f0cd41a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.515+0000 7f0cdc2fd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd410fd10 con 0x7f0cd4100880 2024-04-21T08:36:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.515+0000 7f0cdc2fd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd410fb90 con 0x7f0cd4101b80 2024-04-21T08:36:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.515+0000 7f0cdc2fd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd410fe90 con 0x7f0cd41011e0 2024-04-21T08:36:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.515+0000 7f0cda099640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd4100880 0x7f0cd419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.515+0000 7f0cda099640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd4100880 0x7f0cd419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60670/0 (socket says 172.21.15.43:60670) 2024-04-21T08:36:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.515+0000 7f0cda099640 1 -- 172.21.15.43:0/2301627046 learned_addr learned my addr 172.21.15.43:0/2301627046 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.516+0000 7f0cda89a640 1 --2- 172.21.15.43:0/2301627046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cd4101b80 0x7f0cd41a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.516+0000 7f0cd9898640 1 --2- 172.21.15.43:0/2301627046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cd41011e0 0x7f0cd419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.516+0000 7f0cda099640 1 -- 172.21.15.43:0/2301627046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cd41011e0 msgr2=0x7f0cd419a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.516+0000 7f0cda099640 1 --2- 172.21.15.43:0/2301627046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cd41011e0 0x7f0cd419a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.516+0000 7f0cda099640 1 -- 172.21.15.43:0/2301627046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cd4101b80 msgr2=0x7f0cd41a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.516+0000 7f0cda099640 1 --2- 172.21.15.43:0/2301627046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cd4101b80 0x7f0cd41a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.516+0000 7f0cda099640 1 -- 172.21.15.43:0/2301627046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd41a2060 con 0x7f0cd4100880 2024-04-21T08:36:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.516+0000 7f0cda099640 1 --2- 172.21.15.43:0/2301627046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd4100880 0x7f0cd419a440 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4006010 tx=0x7f0cc40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.516+0000 7f0ccb7fe640 1 -- 172.21.15.43:0/2301627046 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4043020 con 0x7f0cd4100880 2024-04-21T08:36:36.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.516+0000 7f0ccb7fe640 1 -- 172.21.15.43:0/2301627046 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cc402dd20 con 0x7f0cd4100880 2024-04-21T08:36:36.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.516+0000 7f0cdc2fd640 1 -- 172.21.15.43:0/2301627046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd41a22f0 con 0x7f0cd4100880 2024-04-21T08:36:36.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.516+0000 7f0ccb7fe640 1 -- 172.21.15.43:0/2301627046 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4003b20 con 0x7f0cd4100880 2024-04-21T08:36:36.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.516+0000 7f0cdc2fd640 1 -- 172.21.15.43:0/2301627046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd41a28c0 con 0x7f0cd4100880 2024-04-21T08:36:36.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.517+0000 7f0cdc2fd640 1 -- 172.21.15.43:0/2301627046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd410b850 con 0x7f0cd4100880 2024-04-21T08:36:36.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.518+0000 7f0ccb7fe640 1 -- 172.21.15.43:0/2301627046 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0cc4046020 con 0x7f0cd4100880 2024-04-21T08:36:36.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.519+0000 7f0ccb7fe640 1 --2- 172.21.15.43:0/2301627046 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0cc00a3bb0 0x7f0cc00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:36.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.519+0000 7f0ccb7fe640 1 -- 172.21.15.43:0/2301627046 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0cc40f28a0 con 0x7f0cd4100880 2024-04-21T08:36:36.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.519+0000 7f0cd9898640 1 --2- 172.21.15.43:0/2301627046 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0cc00a3bb0 0x7f0cc00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:36.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.519+0000 7f0cd9898640 1 --2- 172.21.15.43:0/2301627046 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0cc00a3bb0 0x7f0cc00a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0cd419b950 tx=0x7f0cd0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:36.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.523+0000 7f0ccb7fe640 1 -- 172.21.15.43:0/2301627046 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0cc404d020 con 0x7f0cd4100880 2024-04-21T08:36:36.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.712+0000 7f0cdc2fd640 1 -- 172.21.15.43:0/2301627046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0cd4062b50 con 0x7f0cd4100880 2024-04-21T08:36:36.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.712+0000 7f0ccb7fe640 1 -- 172.21.15.43:0/2301627046 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f0cc40befe0 con 0x7f0cd4100880 2024-04-21T08:36:36.712 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:36:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.714+0000 7f0cdc2fd640 1 -- 172.21.15.43:0/2301627046 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0cc00a3bb0 msgr2=0x7f0cc00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.714+0000 7f0cdc2fd640 1 --2- 172.21.15.43:0/2301627046 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0cc00a3bb0 0x7f0cc00a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0cd419b950 tx=0x7f0cd0005f90 comp rx=0 tx=0).stop 2024-04-21T08:36:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.714+0000 7f0cdc2fd640 1 -- 172.21.15.43:0/2301627046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd4100880 msgr2=0x7f0cd419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.714+0000 7f0cdc2fd640 1 --2- 172.21.15.43:0/2301627046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd4100880 0x7f0cd419a440 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4006010 tx=0x7f0cc40045e0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.714+0000 7f0cdc2fd640 1 -- 172.21.15.43:0/2301627046 shutdown_connections 2024-04-21T08:36:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.714+0000 7f0cdc2fd640 1 --2- 172.21.15.43:0/2301627046 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0cc00a3bb0 0x7f0cc00a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.714+0000 7f0cdc2fd640 1 --2- 172.21.15.43:0/2301627046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cd4101b80 0x7f0cd41a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.714+0000 7f0cdc2fd640 1 --2- 172.21.15.43:0/2301627046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cd41011e0 0x7f0cd419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.714+0000 7f0cdc2fd640 1 --2- 172.21.15.43:0/2301627046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd4100880 0x7f0cd419a440 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.714+0000 7f0cdc2fd640 1 -- 172.21.15.43:0/2301627046 >> 172.21.15.43:0/2301627046 conn(0x7f0cd40fc400 msgr2=0x7f0cd40fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.714+0000 7f0cdc2fd640 1 -- 172.21.15.43:0/2301627046 shutdown_connections 2024-04-21T08:36:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.714+0000 7f0cdc2fd640 1 -- 172.21.15.43:0/2301627046 wait complete. 2024-04-21T08:36:36.724 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:36:36.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.831+0000 7f9ab1e8c640 1 -- 172.21.15.43:0/3744450489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9aac107c90 msgr2=0x7f9aac10a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:36.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.831+0000 7f9ab1e8c640 1 --2- 172.21.15.43:0/3744450489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9aac107c90 0x7f9aac10a0e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009ca0 tx=0x7f9aa402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.831+0000 7f9ab1e8c640 1 -- 172.21.15.43:0/3744450489 shutdown_connections 2024-04-21T08:36:36.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.831+0000 7f9ab1e8c640 1 --2- 172.21.15.43:0/3744450489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9aac107c90 0x7f9aac10a0e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.831+0000 7f9ab1e8c640 1 --2- 172.21.15.43:0/3744450489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9aac105360 0x7f9aac107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.831+0000 7f9ab1e8c640 1 --2- 172.21.15.43:0/3744450489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9aac102a30 0x7f9aac104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.831+0000 7f9ab1e8c640 1 -- 172.21.15.43:0/3744450489 >> 172.21.15.43:0/3744450489 conn(0x7f9aac0fc440 msgr2=0x7f9aac0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:36.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.831+0000 7f9ab1e8c640 1 -- 172.21.15.43:0/3744450489 shutdown_connections 2024-04-21T08:36:36.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.831+0000 7f9ab1e8c640 1 -- 172.21.15.43:0/3744450489 wait complete. 2024-04-21T08:36:36.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.832+0000 7f9ab1e8c640 1 Processor -- start 2024-04-21T08:36:36.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.832+0000 7f9ab1e8c640 1 -- start start 2024-04-21T08:36:36.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.832+0000 7f9ab1e8c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9aac102a30 0x7f9aac1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:36.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.832+0000 7f9ab1e8c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9aac105360 0x7f9aac198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:36.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.832+0000 7f9ab1e8c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9aac107c90 0x7f9aac19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:36.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.832+0000 7f9ab1e8c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac10daf0 con 0x7f9aac105360 2024-04-21T08:36:36.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.832+0000 7f9ab1e8c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac10d970 con 0x7f9aac102a30 2024-04-21T08:36:36.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.832+0000 7f9ab1e8c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac10dc70 con 0x7f9aac107c90 2024-04-21T08:36:36.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.832+0000 7f9aaaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9aac105360 0x7f9aac198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:36.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.832+0000 7f9aaaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9aac105360 0x7f9aac198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53282/0 (socket says 172.21.15.43:53282) 2024-04-21T08:36:36.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.833+0000 7f9aaaffd640 1 -- 172.21.15.43:0/202957760 learned_addr learned my addr 172.21.15.43:0/202957760 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:36.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.833+0000 7f9aab7fe640 1 --2- 172.21.15.43:0/202957760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9aac102a30 0x7f9aac1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:36.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.833+0000 7f9aabfff640 1 --2- 172.21.15.43:0/202957760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9aac107c90 0x7f9aac19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:36.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.833+0000 7f9aaaffd640 1 -- 172.21.15.43:0/202957760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9aac107c90 msgr2=0x7f9aac19f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:36.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.833+0000 7f9aaaffd640 1 --2- 172.21.15.43:0/202957760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9aac107c90 0x7f9aac19f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.833+0000 7f9aaaffd640 1 -- 172.21.15.43:0/202957760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9aac102a30 msgr2=0x7f9aac1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:36.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.833+0000 7f9aaaffd640 1 --2- 172.21.15.43:0/202957760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9aac102a30 0x7f9aac1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.833+0000 7f9aaaffd640 1 -- 172.21.15.43:0/202957760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aac19fe10 con 0x7f9aac105360 2024-04-21T08:36:36.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.833+0000 7f9aab7fe640 1 --2- 172.21.15.43:0/202957760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9aac102a30 0x7f9aac1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:36:36.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.833+0000 7f9aaaffd640 1 --2- 172.21.15.43:0/202957760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9aac105360 0x7f9aac198730 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000e9c0 tx=0x7f9aa000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:36.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.833+0000 7f9aa8ff9640 1 -- 172.21.15.43:0/202957760 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa000cd80 con 0x7f9aac105360 2024-04-21T08:36:36.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.833+0000 7f9aa8ff9640 1 -- 172.21.15.43:0/202957760 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aa00054d0 con 0x7f9aac105360 2024-04-21T08:36:36.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.833+0000 7f9ab1e8c640 1 -- 172.21.15.43:0/202957760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aac1a0100 con 0x7f9aac105360 2024-04-21T08:36:36.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.833+0000 7f9aa8ff9640 1 -- 172.21.15.43:0/202957760 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0010430 con 0x7f9aac105360 2024-04-21T08:36:36.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.833+0000 7f9ab1e8c640 1 -- 172.21.15.43:0/202957760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aac1a0650 con 0x7f9aac105360 2024-04-21T08:36:36.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.834+0000 7f9ab1e8c640 1 -- 172.21.15.43:0/202957760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aac0fdf30 con 0x7f9aac105360 2024-04-21T08:36:36.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.835+0000 7f9aa8ff9640 1 -- 172.21.15.43:0/202957760 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9aa0004fd0 con 0x7f9aac105360 2024-04-21T08:36:36.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.836+0000 7f9aa8ff9640 1 --2- 172.21.15.43:0/202957760 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9a940a3bb0 0x7f9a940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:36.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.836+0000 7f9aab7fe640 1 --2- 172.21.15.43:0/202957760 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9a940a3bb0 0x7f9a940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:36.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.836+0000 7f9aa8ff9640 1 -- 172.21.15.43:0/202957760 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f9aa00d0a30 con 0x7f9aac105360 2024-04-21T08:36:36.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.837+0000 7f9aab7fe640 1 --2- 172.21.15.43:0/202957760 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9a940a3bb0 0x7f9a940a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c009fc0 tx=0x7f9a9c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:36.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:36.841+0000 7f9aa8ff9640 1 -- 172.21.15.43:0/202957760 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9aa009d1d0 con 0x7f9aac105360 2024-04-21T08:36:37.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.028+0000 7f9ab1e8c640 1 -- 172.21.15.43:0/202957760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9aac062b50 con 0x7f9aac105360 2024-04-21T08:36:37.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.028+0000 7f9aa8ff9640 1 -- 172.21.15.43:0/202957760 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f9aa009cad0 con 0x7f9aac105360 2024-04-21T08:36:37.027 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-21T08:36:37.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.030+0000 7f9ab1e8c640 1 -- 172.21.15.43:0/202957760 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9a940a3bb0 msgr2=0x7f9a940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:37.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.030+0000 7f9ab1e8c640 1 --2- 172.21.15.43:0/202957760 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9a940a3bb0 0x7f9a940a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c009fc0 tx=0x7f9a9c005f90 comp rx=0 tx=0).stop 2024-04-21T08:36:37.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.030+0000 7f9ab1e8c640 1 -- 172.21.15.43:0/202957760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9aac105360 msgr2=0x7f9aac198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:37.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.030+0000 7f9ab1e8c640 1 --2- 172.21.15.43:0/202957760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9aac105360 0x7f9aac198730 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000e9c0 tx=0x7f9aa000ee80 comp rx=0 tx=0).stop 2024-04-21T08:36:37.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.030+0000 7f9ab1e8c640 1 -- 172.21.15.43:0/202957760 shutdown_connections 2024-04-21T08:36:37.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.030+0000 7f9ab1e8c640 1 --2- 172.21.15.43:0/202957760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9aac107c90 0x7f9aac19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.030+0000 7f9ab1e8c640 1 --2- 172.21.15.43:0/202957760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9aac105360 0x7f9aac198730 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.030+0000 7f9ab1e8c640 1 --2- 172.21.15.43:0/202957760 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9a940a3bb0 0x7f9a940a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.030+0000 7f9ab1e8c640 1 --2- 172.21.15.43:0/202957760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9aac102a30 0x7f9aac1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.030+0000 7f9ab1e8c640 1 -- 172.21.15.43:0/202957760 >> 172.21.15.43:0/202957760 conn(0x7f9aac0fc440 msgr2=0x7f9aac103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:37.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.030+0000 7f9ab1e8c640 1 -- 172.21.15.43:0/202957760 shutdown_connections 2024-04-21T08:36:37.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.030+0000 7f9ab1e8c640 1 -- 172.21.15.43:0/202957760 wait complete. 2024-04-21T08:36:37.040 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:36:37.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.146+0000 7f3239681640 1 -- 172.21.15.43:0/61344525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3234101fe0 msgr2=0x7f3234102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:37.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.146+0000 7f3239681640 1 --2- 172.21.15.43:0/61344525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3234101fe0 0x7f3234102460 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3228009900 tx=0x7f322802bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:37.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.146+0000 7f3239681640 1 -- 172.21.15.43:0/61344525 shutdown_connections 2024-04-21T08:36:37.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.146+0000 7f3239681640 1 --2- 172.21.15.43:0/61344525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32341029a0 0x7f32341092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.146+0000 7f3239681640 1 --2- 172.21.15.43:0/61344525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3234101fe0 0x7f3234102460 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.146+0000 7f3239681640 1 --2- 172.21.15.43:0/61344525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3234100e90 0x7f32341012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.146+0000 7f3239681640 1 -- 172.21.15.43:0/61344525 >> 172.21.15.43:0/61344525 conn(0x7f32340fc400 msgr2=0x7f32340fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:37.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.146+0000 7f3239681640 1 -- 172.21.15.43:0/61344525 shutdown_connections 2024-04-21T08:36:37.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.146+0000 7f3239681640 1 -- 172.21.15.43:0/61344525 wait complete. 2024-04-21T08:36:37.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.146+0000 7f3239681640 1 Processor -- start 2024-04-21T08:36:37.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.147+0000 7f3239681640 1 -- start start 2024-04-21T08:36:37.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.147+0000 7f3239681640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3234100e90 0x7f3234069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:37.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.147+0000 7f3239681640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3234101fe0 0x7f323419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:37.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.147+0000 7f3239681640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32341029a0 0x7f323419f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:37.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.147+0000 7f3239681640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f323410c9e0 con 0x7f3234101fe0 2024-04-21T08:36:37.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.147+0000 7f3239681640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f323410c860 con 0x7f3234100e90 2024-04-21T08:36:37.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.147+0000 7f3239681640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f323410cb60 con 0x7f32341029a0 2024-04-21T08:36:37.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.147+0000 7f32327fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3234101fe0 0x7f323419cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:37.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.147+0000 7f32327fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3234101fe0 0x7f323419cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53304/0 (socket says 172.21.15.43:53304) 2024-04-21T08:36:37.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.147+0000 7f32327fc640 1 -- 172.21.15.43:0/3864396096 learned_addr learned my addr 172.21.15.43:0/3864396096 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:37.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.147+0000 7f3232ffd640 1 --2- 172.21.15.43:0/3864396096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3234100e90 0x7f3234069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:37.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.147+0000 7f32337fe640 1 --2- 172.21.15.43:0/3864396096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32341029a0 0x7f323419f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:37.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.147+0000 7f32327fc640 1 -- 172.21.15.43:0/3864396096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32341029a0 msgr2=0x7f323419f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:37.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.147+0000 7f32327fc640 1 --2- 172.21.15.43:0/3864396096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32341029a0 0x7f323419f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.147+0000 7f32327fc640 1 -- 172.21.15.43:0/3864396096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3234100e90 msgr2=0x7f3234069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.147+0000 7f32327fc640 1 --2- 172.21.15.43:0/3864396096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3234100e90 0x7f3234069cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.147+0000 7f32327fc640 1 -- 172.21.15.43:0/3864396096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323419fe60 con 0x7f3234101fe0 2024-04-21T08:36:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.147+0000 7f32327fc640 1 --2- 172.21.15.43:0/3864396096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3234101fe0 0x7f323419cf30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3228000c00 tx=0x7f3228005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.148+0000 7f321bfff640 1 -- 172.21.15.43:0/3864396096 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3228043020 con 0x7f3234101fe0 2024-04-21T08:36:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.148+0000 7f3239681640 1 -- 172.21.15.43:0/3864396096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32341a00f0 con 0x7f3234101fe0 2024-04-21T08:36:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.148+0000 7f321bfff640 1 -- 172.21.15.43:0/3864396096 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3228004350 con 0x7f3234101fe0 2024-04-21T08:36:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.148+0000 7f321bfff640 1 -- 172.21.15.43:0/3864396096 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3228003b80 con 0x7f3234101fe0 2024-04-21T08:36:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.148+0000 7f3239681640 1 -- 172.21.15.43:0/3864396096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32341a0690 con 0x7f3234101fe0 2024-04-21T08:36:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.149+0000 7f3239681640 1 -- 172.21.15.43:0/3864396096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3234108290 con 0x7f3234101fe0 2024-04-21T08:36:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.150+0000 7f321bfff640 1 -- 172.21.15.43:0/3864396096 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3228046020 con 0x7f3234101fe0 2024-04-21T08:36:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.150+0000 7f321bfff640 1 --2- 172.21.15.43:0/3864396096 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f321c0a3bb0 0x7f321c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.150+0000 7f3232ffd640 1 --2- 172.21.15.43:0/3864396096 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f321c0a3bb0 0x7f321c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.150+0000 7f321bfff640 1 -- 172.21.15.43:0/3864396096 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f32280f2760 con 0x7f3234101fe0 2024-04-21T08:36:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.151+0000 7f3232ffd640 1 --2- 172.21.15.43:0/3864396096 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f321c0a3bb0 0x7f321c0a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3224005fd0 tx=0x7f3224009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:37.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.155+0000 7f321bfff640 1 -- 172.21.15.43:0/3864396096 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f322804d020 con 0x7f3234101fe0 2024-04-21T08:36:37.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:37 smithi175 ceph-mon[31920]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 2 op/s 2024-04-21T08:36:37.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:37 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3650013205' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:37.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:37 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2119677997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:36:37.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:37 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2301627046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:36:37.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:37 smithi112 ceph-mon[31085]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 2 op/s 2024-04-21T08:36:37.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:37 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3650013205' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:37.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:37 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2119677997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:36:37.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:37 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2301627046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:36:37.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:37 smithi043 ceph-mon[27978]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 2 op/s 2024-04-21T08:36:37.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:37 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3650013205' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:37.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:37 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2119677997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:36:37.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:37 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2301627046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:36:37.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.339+0000 7f3239681640 1 -- 172.21.15.43:0/3864396096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f3234062b50 con 0x7f3234101fe0 2024-04-21T08:36:37.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.340+0000 7f321bfff640 1 -- 172.21.15.43:0/3864396096 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f32280bef00 con 0x7f3234101fe0 2024-04-21T08:36:37.339 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:36:37.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.342+0000 7f3239681640 1 -- 172.21.15.43:0/3864396096 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f321c0a3bb0 msgr2=0x7f321c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:37.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.342+0000 7f3239681640 1 --2- 172.21.15.43:0/3864396096 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f321c0a3bb0 0x7f321c0a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3224005fd0 tx=0x7f3224009500 comp rx=0 tx=0).stop 2024-04-21T08:36:37.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.342+0000 7f3239681640 1 -- 172.21.15.43:0/3864396096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3234101fe0 msgr2=0x7f323419cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:37.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.342+0000 7f3239681640 1 --2- 172.21.15.43:0/3864396096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3234101fe0 0x7f323419cf30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3228000c00 tx=0x7f3228005dc0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.342+0000 7f3239681640 1 -- 172.21.15.43:0/3864396096 shutdown_connections 2024-04-21T08:36:37.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.342+0000 7f3239681640 1 --2- 172.21.15.43:0/3864396096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32341029a0 0x7f323419f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.342+0000 7f3239681640 1 --2- 172.21.15.43:0/3864396096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3234101fe0 0x7f323419cf30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.342+0000 7f3239681640 1 --2- 172.21.15.43:0/3864396096 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f321c0a3bb0 0x7f321c0a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.342+0000 7f3239681640 1 --2- 172.21.15.43:0/3864396096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3234100e90 0x7f3234069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.342+0000 7f3239681640 1 -- 172.21.15.43:0/3864396096 >> 172.21.15.43:0/3864396096 conn(0x7f32340fc400 msgr2=0x7f3234105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:37.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.342+0000 7f3239681640 1 -- 172.21.15.43:0/3864396096 shutdown_connections 2024-04-21T08:36:37.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.342+0000 7f3239681640 1 -- 172.21.15.43:0/3864396096 wait complete. 2024-04-21T08:36:37.352 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-21T08:36:37.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.457+0000 7f2fb97b2640 1 -- 172.21.15.43:0/3015697883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fb4101fe0 msgr2=0x7f2fb4102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:37.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.457+0000 7f2fb97b2640 1 --2- 172.21.15.43:0/3015697883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fb4101fe0 0x7f2fb4102460 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009900 tx=0x7f2fa402bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:37.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.457+0000 7f2fb97b2640 1 -- 172.21.15.43:0/3015697883 shutdown_connections 2024-04-21T08:36:37.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.457+0000 7f2fb97b2640 1 --2- 172.21.15.43:0/3015697883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fb41029a0 0x7f2fb41092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.457+0000 7f2fb97b2640 1 --2- 172.21.15.43:0/3015697883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fb4101fe0 0x7f2fb4102460 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.457+0000 7f2fb97b2640 1 --2- 172.21.15.43:0/3015697883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fb4100e90 0x7f2fb41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.457+0000 7f2fb97b2640 1 -- 172.21.15.43:0/3015697883 >> 172.21.15.43:0/3015697883 conn(0x7f2fb40fc400 msgr2=0x7f2fb40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:37.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.457+0000 7f2fb97b2640 1 -- 172.21.15.43:0/3015697883 shutdown_connections 2024-04-21T08:36:37.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.457+0000 7f2fb97b2640 1 -- 172.21.15.43:0/3015697883 wait complete. 2024-04-21T08:36:37.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.457+0000 7f2fb97b2640 1 Processor -- start 2024-04-21T08:36:37.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.457+0000 7f2fb97b2640 1 -- start start 2024-04-21T08:36:37.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.458+0000 7f2fb97b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fb4100e90 0x7f2fb4104fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:37.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.458+0000 7f2fb97b2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fb4101fe0 0x7f2fb41035e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:37.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.458+0000 7f2fb97b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fb41029a0 0x7f2fb4103b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:37.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.458+0000 7f2fb97b2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb410c9f0 con 0x7f2fb4100e90 2024-04-21T08:36:37.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.458+0000 7f2fb97b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb410c870 con 0x7f2fb41029a0 2024-04-21T08:36:37.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.458+0000 7f2fb97b2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb410cb70 con 0x7f2fb4101fe0 2024-04-21T08:36:37.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.458+0000 7f2fb2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fb4100e90 0x7f2fb4104fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:37.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.458+0000 7f2fb2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fb4100e90 0x7f2fb4104fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53324/0 (socket says 172.21.15.43:53324) 2024-04-21T08:36:37.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.458+0000 7f2fb2ffd640 1 -- 172.21.15.43:0/2658278455 learned_addr learned my addr 172.21.15.43:0/2658278455 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:37.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.458+0000 7f2fb37fe640 1 --2- 172.21.15.43:0/2658278455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fb41029a0 0x7f2fb4103b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:37.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.458+0000 7f2fb27fc640 1 --2- 172.21.15.43:0/2658278455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fb4101fe0 0x7f2fb41035e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:37.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.458+0000 7f2fb2ffd640 1 -- 172.21.15.43:0/2658278455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fb4101fe0 msgr2=0x7f2fb41035e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:37.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.458+0000 7f2fb2ffd640 1 --2- 172.21.15.43:0/2658278455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fb4101fe0 0x7f2fb41035e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.458+0000 7f2fb2ffd640 1 -- 172.21.15.43:0/2658278455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fb41029a0 msgr2=0x7f2fb4103b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:37.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.458+0000 7f2fb2ffd640 1 --2- 172.21.15.43:0/2658278455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fb41029a0 0x7f2fb4103b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.458+0000 7f2fb2ffd640 1 -- 172.21.15.43:0/2658278455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb41043b0 con 0x7f2fb4100e90 2024-04-21T08:36:37.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.458+0000 7f2fb27fc640 1 --2- 172.21.15.43:0/2658278455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fb4101fe0 0x7f2fb41035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:37.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.458+0000 7f2fb2ffd640 1 --2- 172.21.15.43:0/2658278455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fb4100e90 0x7f2fb4104fa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800e9f0 tx=0x7f2fa800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:37.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.458+0000 7f2f9bfff640 1 -- 172.21.15.43:0/2658278455 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa800cd80 con 0x7f2fb4100e90 2024-04-21T08:36:37.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.458+0000 7f2f9bfff640 1 -- 172.21.15.43:0/2658278455 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fa80054d0 con 0x7f2fb4100e90 2024-04-21T08:36:37.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.458+0000 7f2f9bfff640 1 -- 172.21.15.43:0/2658278455 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8010690 con 0x7f2fb4100e90 2024-04-21T08:36:37.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.458+0000 7f2fb97b2640 1 -- 172.21.15.43:0/2658278455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb41a2390 con 0x7f2fb4100e90 2024-04-21T08:36:37.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.458+0000 7f2fb97b2640 1 -- 172.21.15.43:0/2658278455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb41a2890 con 0x7f2fb4100e90 2024-04-21T08:36:37.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.460+0000 7f2fb97b2640 1 -- 172.21.15.43:0/2658278455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb41a2bb0 con 0x7f2fb4100e90 2024-04-21T08:36:37.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.461+0000 7f2f9bfff640 1 -- 172.21.15.43:0/2658278455 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2fa8004e90 con 0x7f2fb4100e90 2024-04-21T08:36:37.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.461+0000 7f2f9bfff640 1 --2- 172.21.15.43:0/2658278455 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2f9c0a3bb0 0x7f2f9c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:37.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.461+0000 7f2f9bfff640 1 -- 172.21.15.43:0/2658278455 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f2fa80d0be0 con 0x7f2fb4100e90 2024-04-21T08:36:37.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.461+0000 7f2fb27fc640 1 --2- 172.21.15.43:0/2658278455 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2f9c0a3bb0 0x7f2f9c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:37.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.462+0000 7f2fb27fc640 1 --2- 172.21.15.43:0/2658278455 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2f9c0a3bb0 0x7f2f9c0a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4036040 tx=0x7f2fa4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:37.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.466+0000 7f2f9bfff640 1 -- 172.21.15.43:0/2658278455 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2fa809d380 con 0x7f2fb4100e90 2024-04-21T08:36:37.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.665+0000 7f2fb97b2640 1 -- 172.21.15.43:0/2658278455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2fb41a2d80 con 0x7f2fb4100e90 2024-04-21T08:36:37.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.665+0000 7f2f9bfff640 1 -- 172.21.15.43:0/2658278455 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+590 (secure 0 0 0) 0x7f2fa809cc80 con 0x7f2fb4100e90 2024-04-21T08:36:37.665 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/1916182222 2024-04-22T08:29:13.889526+0000 2024-04-21T08:36:37.665 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6801/3200866696 2024-04-22T08:29:13.889526+0000 2024-04-21T08:36:37.665 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6801/1559586579 2024-04-22T08:28:46.855699+0000 2024-04-21T08:36:37.665 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6800/3200866696 2024-04-22T08:29:13.889526+0000 2024-04-21T08:36:37.665 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/4047106671 2024-04-22T08:29:13.889526+0000 2024-04-21T08:36:37.665 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6800/1559586579 2024-04-22T08:28:46.855699+0000 2024-04-21T08:36:37.665 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/347461706 2024-04-22T08:29:13.889526+0000 2024-04-21T08:36:37.665 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/887926369 2024-04-22T08:28:46.855699+0000 2024-04-21T08:36:37.665 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/2598599313 2024-04-22T08:28:46.855699+0000 2024-04-21T08:36:37.665 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/2126312819 2024-04-22T08:28:46.855699+0000 2024-04-21T08:36:37.666 INFO:teuthology.orchestra.run.smithi043.stderr:listed 10 entries 2024-04-21T08:36:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.667+0000 7f2fb97b2640 1 -- 172.21.15.43:0/2658278455 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2f9c0a3bb0 msgr2=0x7f2f9c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.667+0000 7f2fb97b2640 1 --2- 172.21.15.43:0/2658278455 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2f9c0a3bb0 0x7f2f9c0a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4036040 tx=0x7f2fa4000bc0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.667+0000 7f2fb97b2640 1 -- 172.21.15.43:0/2658278455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fb4100e90 msgr2=0x7f2fb4104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.667+0000 7f2fb97b2640 1 --2- 172.21.15.43:0/2658278455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fb4100e90 0x7f2fb4104fa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800e9f0 tx=0x7f2fa800edf0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.667+0000 7f2fb97b2640 1 -- 172.21.15.43:0/2658278455 shutdown_connections 2024-04-21T08:36:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.667+0000 7f2fb97b2640 1 --2- 172.21.15.43:0/2658278455 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2f9c0a3bb0 0x7f2f9c0a6070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.667+0000 7f2fb97b2640 1 --2- 172.21.15.43:0/2658278455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fb41029a0 0x7f2fb4103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.667+0000 7f2fb97b2640 1 --2- 172.21.15.43:0/2658278455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fb4101fe0 0x7f2fb41035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.667+0000 7f2fb97b2640 1 --2- 172.21.15.43:0/2658278455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fb4100e90 0x7f2fb4104fa0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.667+0000 7f2fb97b2640 1 -- 172.21.15.43:0/2658278455 >> 172.21.15.43:0/2658278455 conn(0x7f2fb40fc400 msgr2=0x7f2fb40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.667+0000 7f2fb97b2640 1 -- 172.21.15.43:0/2658278455 shutdown_connections 2024-04-21T08:36:37.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.668+0000 7f2fb97b2640 1 -- 172.21.15.43:0/2658278455 wait complete. 2024-04-21T08:36:37.678 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-21T08:36:37.678 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T08:36:37.695 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:36:37.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.838+0000 7f831fe55640 1 -- 172.21.15.43:0/465566354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8318102050 msgr2=0x7f83181024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:37.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.838+0000 7f831fe55640 1 --2- 172.21.15.43:0/465566354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8318102050 0x7f83181024d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8308009900 tx=0x7f830802bc50 comp rx=0 tx=0).stop 2024-04-21T08:36:37.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.839+0000 7f831fe55640 1 -- 172.21.15.43:0/465566354 shutdown_connections 2024-04-21T08:36:37.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.839+0000 7f831fe55640 1 --2- 172.21.15.43:0/465566354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8318102a10 0x7f8318109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.839+0000 7f831fe55640 1 --2- 172.21.15.43:0/465566354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8318102050 0x7f83181024d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.839+0000 7f831fe55640 1 --2- 172.21.15.43:0/465566354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8318100e70 0x7f8318101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.839+0000 7f831fe55640 1 -- 172.21.15.43:0/465566354 >> 172.21.15.43:0/465566354 conn(0x7f83180fc470 msgr2=0x7f83180fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:37.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.839+0000 7f831fe55640 1 -- 172.21.15.43:0/465566354 shutdown_connections 2024-04-21T08:36:37.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.839+0000 7f831fe55640 1 -- 172.21.15.43:0/465566354 wait complete. 2024-04-21T08:36:37.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.839+0000 7f831fe55640 1 Processor -- start 2024-04-21T08:36:37.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.839+0000 7f831fe55640 1 -- start start 2024-04-21T08:36:37.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.839+0000 7f831fe55640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8318100e70 0x7f8318198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:37.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.839+0000 7f831fe55640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8318102050 0x7f83181986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:37.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.839+0000 7f831fe55640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8318102a10 0x7f831819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:37.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.839+0000 7f831fe55640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f831810dbe0 con 0x7f8318100e70 2024-04-21T08:36:37.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.839+0000 7f831fe55640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f831810da60 con 0x7f8318102a10 2024-04-21T08:36:37.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.839+0000 7f831fe55640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f831810dd60 con 0x7f8318102050 2024-04-21T08:36:37.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.839+0000 7f831dbf1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8318100e70 0x7f8318198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:37.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.839+0000 7f831dbf1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8318100e70 0x7f8318198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53336/0 (socket says 172.21.15.43:53336) 2024-04-21T08:36:37.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.839+0000 7f831dbf1640 1 -- 172.21.15.43:0/1396983060 learned_addr learned my addr 172.21.15.43:0/1396983060 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:37.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.840+0000 7f831dbf1640 1 -- 172.21.15.43:0/1396983060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8318102050 msgr2=0x7f83181986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:37.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.840+0000 7f831dbf1640 1 --2- 172.21.15.43:0/1396983060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8318102050 0x7f83181986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.840+0000 7f831dbf1640 1 -- 172.21.15.43:0/1396983060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8318102a10 msgr2=0x7f831819f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:37.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.840+0000 7f831dbf1640 1 --2- 172.21.15.43:0/1396983060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8318102a10 0x7f831819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:37.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.840+0000 7f831dbf1640 1 -- 172.21.15.43:0/1396983060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831819fdb0 con 0x7f8318100e70 2024-04-21T08:36:37.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.840+0000 7f831dbf1640 1 --2- 172.21.15.43:0/1396983060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8318100e70 0x7f8318198190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f831000b470 tx=0x7f831000b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:37.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.840+0000 7f830effd640 1 -- 172.21.15.43:0/1396983060 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310015020 con 0x7f8318100e70 2024-04-21T08:36:37.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.840+0000 7f830effd640 1 -- 172.21.15.43:0/1396983060 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8310011d40 con 0x7f8318100e70 2024-04-21T08:36:37.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.840+0000 7f831fe55640 1 -- 172.21.15.43:0/1396983060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83181a00a0 con 0x7f8318100e70 2024-04-21T08:36:37.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.840+0000 7f830effd640 1 -- 172.21.15.43:0/1396983060 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831000f340 con 0x7f8318100e70 2024-04-21T08:36:37.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.840+0000 7f831fe55640 1 -- 172.21.15.43:0/1396983060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83181a0670 con 0x7f8318100e70 2024-04-21T08:36:37.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.841+0000 7f831fe55640 1 -- 172.21.15.43:0/1396983060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8318108140 con 0x7f8318100e70 2024-04-21T08:36:37.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.842+0000 7f830effd640 1 -- 172.21.15.43:0/1396983060 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8310018020 con 0x7f8318100e70 2024-04-21T08:36:37.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.843+0000 7f830effd640 1 --2- 172.21.15.43:0/1396983060 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f83040a3c30 0x7f83040a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:37.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.843+0000 7f831d3f0640 1 --2- 172.21.15.43:0/1396983060 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f83040a3c30 0x7f83040a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:37.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.843+0000 7f830effd640 1 -- 172.21.15.43:0/1396983060 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f83100d0150 con 0x7f8318100e70 2024-04-21T08:36:37.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.843+0000 7f831d3f0640 1 --2- 172.21.15.43:0/1396983060 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f83040a3c30 0x7f83040a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8308005c20 tx=0x7f8308000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:37.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:37.848+0000 7f830effd640 1 -- 172.21.15.43:0/1396983060 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f831001f020 con 0x7f8318100e70 2024-04-21T08:36:38.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.035+0000 7f831fe55640 1 -- 172.21.15.43:0/1396983060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f831802d5c0 con 0x7f8318100e70 2024-04-21T08:36:38.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.036+0000 7f830effd640 1 -- 172.21.15.43:0/1396983060 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24152 (secure 0 0 0) 0x7f831000d040 con 0x7f8318100e70 2024-04-21T08:36:38.036 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:38.036 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":90,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:36:28.364093+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:36:38.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.038+0000 7f831fe55640 1 -- 172.21.15.43:0/1396983060 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f83040a3c30 msgr2=0x7f83040a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:38.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.038+0000 7f831fe55640 1 --2- 172.21.15.43:0/1396983060 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f83040a3c30 0x7f83040a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8308005c20 tx=0x7f8308000bc0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.038+0000 7f831fe55640 1 -- 172.21.15.43:0/1396983060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8318100e70 msgr2=0x7f8318198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:38.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.038+0000 7f831fe55640 1 --2- 172.21.15.43:0/1396983060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8318100e70 0x7f8318198190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f831000b470 tx=0x7f831000b930 comp rx=0 tx=0).stop 2024-04-21T08:36:38.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.038+0000 7f831fe55640 1 -- 172.21.15.43:0/1396983060 shutdown_connections 2024-04-21T08:36:38.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.038+0000 7f831fe55640 1 --2- 172.21.15.43:0/1396983060 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f83040a3c30 0x7f83040a60f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.038+0000 7f831fe55640 1 --2- 172.21.15.43:0/1396983060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8318102a10 0x7f831819f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.038+0000 7f831fe55640 1 --2- 172.21.15.43:0/1396983060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8318102050 0x7f83181986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.038+0000 7f831fe55640 1 --2- 172.21.15.43:0/1396983060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8318100e70 0x7f8318198190 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.038+0000 7f831fe55640 1 -- 172.21.15.43:0/1396983060 >> 172.21.15.43:0/1396983060 conn(0x7f83180fc470 msgr2=0x7f8318105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:38.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.039+0000 7f831fe55640 1 -- 172.21.15.43:0/1396983060 shutdown_connections 2024-04-21T08:36:38.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.039+0000 7f831fe55640 1 -- 172.21.15.43:0/1396983060 wait complete. 2024-04-21T08:36:38.049 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:39.981405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:22.034751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:22.780795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:25.984261+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:36:38.049 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:36:38.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.155+0000 7fb9201b2640 1 -- 172.21.15.43:0/2252423290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb918101fa0 msgr2=0x7fb9181023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.155+0000 7fb9201b2640 1 --2- 172.21.15.43:0/2252423290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb918101fa0 0x7fb9181023c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb914009900 tx=0x7fb91402bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.155+0000 7fb9201b2640 1 -- 172.21.15.43:0/2252423290 shutdown_connections 2024-04-21T08:36:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.155+0000 7fb9201b2640 1 --2- 172.21.15.43:0/2252423290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb918103b50 0x7fb91810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.155+0000 7fb9201b2640 1 --2- 172.21.15.43:0/2252423290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb918103190 0x7fb918103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.155+0000 7fb9201b2640 1 --2- 172.21.15.43:0/2252423290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb918101fa0 0x7fb9181023c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.155+0000 7fb9201b2640 1 -- 172.21.15.43:0/2252423290 >> 172.21.15.43:0/2252423290 conn(0x7fb9180fd530 msgr2=0x7fb9180ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.155+0000 7fb9201b2640 1 -- 172.21.15.43:0/2252423290 shutdown_connections 2024-04-21T08:36:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.155+0000 7fb9201b2640 1 -- 172.21.15.43:0/2252423290 wait complete. 2024-04-21T08:36:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.155+0000 7fb9201b2640 1 Processor -- start 2024-04-21T08:36:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.155+0000 7fb9201b2640 1 -- start start 2024-04-21T08:36:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.156+0000 7fb9201b2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb918101fa0 0x7fb9181982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:38.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.156+0000 7fb9201b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb918103190 0x7fb918198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:38.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.156+0000 7fb9201b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb918103b50 0x7fb91819f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:38.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.156+0000 7fb9201b2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb91810db40 con 0x7fb918103190 2024-04-21T08:36:38.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.156+0000 7fb9201b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb91810d9c0 con 0x7fb918103b50 2024-04-21T08:36:38.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.156+0000 7fb9201b2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb91810dcc0 con 0x7fb918101fa0 2024-04-21T08:36:38.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.156+0000 7fb91d74d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb918103190 0x7fb918198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:38.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.156+0000 7fb91d74d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb918103190 0x7fb918198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53362/0 (socket says 172.21.15.43:53362) 2024-04-21T08:36:38.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.156+0000 7fb91d74d640 1 -- 172.21.15.43:0/1011055732 learned_addr learned my addr 172.21.15.43:0/1011055732 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:38.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.156+0000 7fb91df4e640 1 --2- 172.21.15.43:0/1011055732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb918101fa0 0x7fb9181982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:38.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.156+0000 7fb91e74f640 1 --2- 172.21.15.43:0/1011055732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb918103b50 0x7fb91819f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:38.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.156+0000 7fb91d74d640 1 -- 172.21.15.43:0/1011055732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb918101fa0 msgr2=0x7fb9181982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:38.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.156+0000 7fb91d74d640 1 --2- 172.21.15.43:0/1011055732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb918101fa0 0x7fb9181982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.156+0000 7fb91d74d640 1 -- 172.21.15.43:0/1011055732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb918103b50 msgr2=0x7fb91819f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:38.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.156+0000 7fb91d74d640 1 --2- 172.21.15.43:0/1011055732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb918103b50 0x7fb91819f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.156+0000 7fb91d74d640 1 -- 172.21.15.43:0/1011055732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91819fee0 con 0x7fb918103190 2024-04-21T08:36:38.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.156+0000 7fb91e74f640 1 --2- 172.21.15.43:0/1011055732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb918103b50 0x7fb91819f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:38.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.156+0000 7fb91d74d640 1 --2- 172.21.15.43:0/1011055732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb918103190 0x7fb918198800 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb90800c970 tx=0x7fb90800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:38.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.156+0000 7fb90effd640 1 -- 172.21.15.43:0/1011055732 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb908007bb0 con 0x7fb918103190 2024-04-21T08:36:38.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.156+0000 7fb90effd640 1 -- 172.21.15.43:0/1011055732 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb908007d50 con 0x7fb918103190 2024-04-21T08:36:38.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.156+0000 7fb9201b2640 1 -- 172.21.15.43:0/1011055732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9181a01d0 con 0x7fb918103190 2024-04-21T08:36:38.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.156+0000 7fb90effd640 1 -- 172.21.15.43:0/1011055732 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90800f450 con 0x7fb918103190 2024-04-21T08:36:38.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.157+0000 7fb9201b2640 1 -- 172.21.15.43:0/1011055732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9181a07a0 con 0x7fb918103190 2024-04-21T08:36:38.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.158+0000 7fb9201b2640 1 -- 172.21.15.43:0/1011055732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb918109390 con 0x7fb918103190 2024-04-21T08:36:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.159+0000 7fb90effd640 1 -- 172.21.15.43:0/1011055732 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb908004f00 con 0x7fb918103190 2024-04-21T08:36:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.159+0000 7fb90effd640 1 --2- 172.21.15.43:0/1011055732 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb9040a3c30 0x7fb9040a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.160+0000 7fb91df4e640 1 --2- 172.21.15.43:0/1011055732 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb9040a3c30 0x7fb9040a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.160+0000 7fb90effd640 1 -- 172.21.15.43:0/1011055732 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb9080d07f0 con 0x7fb918103190 2024-04-21T08:36:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.160+0000 7fb91df4e640 1 --2- 172.21.15.43:0/1011055732 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb9040a3c30 0x7fb9040a60f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb914006010 tx=0x7fb91402c800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:38.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.163+0000 7fb90effd640 1 -- 172.21.15.43:0/1011055732 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb908014030 con 0x7fb918103190 2024-04-21T08:36:38.293 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:38 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/202957760' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:36:38.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:38 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3864396096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:36:38.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:38 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2658278455' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:36:38.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:38 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/202957760' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:36:38.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:38 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3864396096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:36:38.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:38 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2658278455' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:36:38.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.347+0000 7fb9201b2640 1 -- 172.21.15.43:0/1011055732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb91802d810 con 0x7fb918103190 2024-04-21T08:36:38.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.348+0000 7fb90effd640 1 -- 172.21.15.43:0/1011055732 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fb90809cfb0 con 0x7fb918103190 2024-04-21T08:36:38.347 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-21T08:36:38.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.350+0000 7fb9201b2640 1 -- 172.21.15.43:0/1011055732 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb9040a3c30 msgr2=0x7fb9040a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:38.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.350+0000 7fb9201b2640 1 --2- 172.21.15.43:0/1011055732 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb9040a3c30 0x7fb9040a60f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb914006010 tx=0x7fb91402c800 comp rx=0 tx=0).stop 2024-04-21T08:36:38.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.350+0000 7fb9201b2640 1 -- 172.21.15.43:0/1011055732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb918103190 msgr2=0x7fb918198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:38.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.350+0000 7fb9201b2640 1 --2- 172.21.15.43:0/1011055732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb918103190 0x7fb918198800 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb90800c970 tx=0x7fb90800ce30 comp rx=0 tx=0).stop 2024-04-21T08:36:38.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.350+0000 7fb9201b2640 1 -- 172.21.15.43:0/1011055732 shutdown_connections 2024-04-21T08:36:38.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.350+0000 7fb9201b2640 1 --2- 172.21.15.43:0/1011055732 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb9040a3c30 0x7fb9040a60f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.350+0000 7fb9201b2640 1 --2- 172.21.15.43:0/1011055732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb918103b50 0x7fb91819f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.350+0000 7fb9201b2640 1 --2- 172.21.15.43:0/1011055732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb918103190 0x7fb918198800 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.350+0000 7fb9201b2640 1 --2- 172.21.15.43:0/1011055732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb918101fa0 0x7fb9181982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.350+0000 7fb9201b2640 1 -- 172.21.15.43:0/1011055732 >> 172.21.15.43:0/1011055732 conn(0x7fb9180fd530 msgr2=0x7fb918106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:38.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.350+0000 7fb9201b2640 1 -- 172.21.15.43:0/1011055732 shutdown_connections 2024-04-21T08:36:38.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.350+0000 7fb9201b2640 1 -- 172.21.15.43:0/1011055732 wait complete. 2024-04-21T08:36:38.360 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:36:38.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:38 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/202957760' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:36:38.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:38 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3864396096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:36:38.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:38 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2658278455' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:36:38.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.465+0000 7f388b1ef640 1 -- 172.21.15.43:0/3704120200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38841024b0 msgr2=0x7f38841028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:38.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.465+0000 7f388b1ef640 1 --2- 172.21.15.43:0/3704120200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38841024b0 0x7f38841028d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3874009930 tx=0x7f387402bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:38.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.465+0000 7f388b1ef640 1 -- 172.21.15.43:0/3704120200 shutdown_connections 2024-04-21T08:36:38.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.465+0000 7f388b1ef640 1 --2- 172.21.15.43:0/3704120200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3884101b10 0x7f38841093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.465+0000 7f388b1ef640 1 --2- 172.21.15.43:0/3704120200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3884101150 0x7f38841015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.465+0000 7f388b1ef640 1 --2- 172.21.15.43:0/3704120200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38841024b0 0x7f38841028d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.465+0000 7f388b1ef640 1 -- 172.21.15.43:0/3704120200 >> 172.21.15.43:0/3704120200 conn(0x7f38840fbb80 msgr2=0x7f38840fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:38.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.465+0000 7f388b1ef640 1 -- 172.21.15.43:0/3704120200 shutdown_connections 2024-04-21T08:36:38.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.465+0000 7f388b1ef640 1 -- 172.21.15.43:0/3704120200 wait complete. 2024-04-21T08:36:38.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.466+0000 7f388b1ef640 1 Processor -- start 2024-04-21T08:36:38.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.466+0000 7f388b1ef640 1 -- start start 2024-04-21T08:36:38.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.466+0000 7f388b1ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3884101150 0x7f3884198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:38.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.466+0000 7f388b1ef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3884101b10 0x7f38841987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:38.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.466+0000 7f388b1ef640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38841024b0 0x7f388419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:38.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.466+0000 7f388b1ef640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f388410c9f0 con 0x7f3884101150 2024-04-21T08:36:38.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.466+0000 7f388b1ef640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f388410c870 con 0x7f3884101b10 2024-04-21T08:36:38.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.466+0000 7f388b1ef640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f388410cb70 con 0x7f38841024b0 2024-04-21T08:36:38.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.466+0000 7f3888f8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3884101150 0x7f3884198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:38.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.466+0000 7f3888f8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3884101150 0x7f3884198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53380/0 (socket says 172.21.15.43:53380) 2024-04-21T08:36:38.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.466+0000 7f3888f8b640 1 -- 172.21.15.43:0/220905310 learned_addr learned my addr 172.21.15.43:0/220905310 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:38.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.466+0000 7f3883fff640 1 --2- 172.21.15.43:0/220905310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3884101b10 0x7f38841987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:38.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.467+0000 7f3888f8b640 1 -- 172.21.15.43:0/220905310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38841024b0 msgr2=0x7f388419f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:38.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.467+0000 7f3888f8b640 1 --2- 172.21.15.43:0/220905310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38841024b0 0x7f388419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.467+0000 7f3888f8b640 1 -- 172.21.15.43:0/220905310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3884101b10 msgr2=0x7f38841987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:38.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.467+0000 7f3888f8b640 1 --2- 172.21.15.43:0/220905310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3884101b10 0x7f38841987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.467+0000 7f3888f8b640 1 -- 172.21.15.43:0/220905310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388419fe90 con 0x7f3884101150 2024-04-21T08:36:38.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.467+0000 7f3883fff640 1 --2- 172.21.15.43:0/220905310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3884101b10 0x7f38841987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:38.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.467+0000 7f3888f8b640 1 --2- 172.21.15.43:0/220905310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3884101150 0x7f3884198270 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3874000c00 tx=0x7f3874004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:38.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.467+0000 7f3881ffb640 1 -- 172.21.15.43:0/220905310 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3874043020 con 0x7f3884101150 2024-04-21T08:36:38.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.467+0000 7f3881ffb640 1 -- 172.21.15.43:0/220905310 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3874004aa0 con 0x7f3884101150 2024-04-21T08:36:38.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.467+0000 7f388b1ef640 1 -- 172.21.15.43:0/220905310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38841a0120 con 0x7f3884101150 2024-04-21T08:36:38.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.467+0000 7f3881ffb640 1 -- 172.21.15.43:0/220905310 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3874003cd0 con 0x7f3884101150 2024-04-21T08:36:38.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.467+0000 7f388b1ef640 1 -- 172.21.15.43:0/220905310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38841a06c0 con 0x7f3884101150 2024-04-21T08:36:38.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.468+0000 7f388b1ef640 1 -- 172.21.15.43:0/220905310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38841a08c0 con 0x7f3884101150 2024-04-21T08:36:38.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.469+0000 7f3881ffb640 1 -- 172.21.15.43:0/220905310 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3874046020 con 0x7f3884101150 2024-04-21T08:36:38.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.470+0000 7f3881ffb640 1 --2- 172.21.15.43:0/220905310 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f386c0a3c30 0x7f386c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:38.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.470+0000 7f3883fff640 1 --2- 172.21.15.43:0/220905310 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f386c0a3c30 0x7f386c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:38.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.470+0000 7f3881ffb640 1 -- 172.21.15.43:0/220905310 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f38740f2c40 con 0x7f3884101150 2024-04-21T08:36:38.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.470+0000 7f3883fff640 1 --2- 172.21.15.43:0/220905310 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f386c0a3c30 0x7f386c0a60f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3878005fd0 tx=0x7f3878005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:38.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.474+0000 7f3881ffb640 1 -- 172.21.15.43:0/220905310 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f387404d020 con 0x7f3884101150 2024-04-21T08:36:38.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.660+0000 7f388b1ef640 1 -- 172.21.15.43:0/220905310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f38841a0b30 con 0x7f3884101150 2024-04-21T08:36:38.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.660+0000 7f3881ffb640 1 -- 172.21.15.43:0/220905310 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f38740bf380 con 0x7f3884101150 2024-04-21T08:36:38.659 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:36:38.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.662+0000 7f388b1ef640 1 -- 172.21.15.43:0/220905310 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f386c0a3c30 msgr2=0x7f386c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:38.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.662+0000 7f388b1ef640 1 --2- 172.21.15.43:0/220905310 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f386c0a3c30 0x7f386c0a60f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3878005fd0 tx=0x7f3878005ea0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.662+0000 7f388b1ef640 1 -- 172.21.15.43:0/220905310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3884101150 msgr2=0x7f3884198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:38.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.662+0000 7f388b1ef640 1 --2- 172.21.15.43:0/220905310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3884101150 0x7f3884198270 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3874000c00 tx=0x7f3874004270 comp rx=0 tx=0).stop 2024-04-21T08:36:38.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.662+0000 7f388b1ef640 1 -- 172.21.15.43:0/220905310 shutdown_connections 2024-04-21T08:36:38.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.662+0000 7f388b1ef640 1 --2- 172.21.15.43:0/220905310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38841024b0 0x7f388419f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.662+0000 7f388b1ef640 1 --2- 172.21.15.43:0/220905310 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f386c0a3c30 0x7f386c0a60f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.662+0000 7f388b1ef640 1 --2- 172.21.15.43:0/220905310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3884101b10 0x7f38841987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.662+0000 7f388b1ef640 1 --2- 172.21.15.43:0/220905310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3884101150 0x7f3884198270 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.662+0000 7f388b1ef640 1 -- 172.21.15.43:0/220905310 >> 172.21.15.43:0/220905310 conn(0x7f38840fbb80 msgr2=0x7f38841000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:38.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.662+0000 7f388b1ef640 1 -- 172.21.15.43:0/220905310 shutdown_connections 2024-04-21T08:36:38.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.662+0000 7f388b1ef640 1 -- 172.21.15.43:0/220905310 wait complete. 2024-04-21T08:36:38.672 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:36:38.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.778+0000 7f8dc4eba640 1 -- 172.21.15.43:0/3147114844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8dc0101fe0 msgr2=0x7f8dc0102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:38.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.778+0000 7f8dc4eba640 1 --2- 172.21.15.43:0/3147114844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8dc0101fe0 0x7f8dc0102460 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8db4009900 tx=0x7f8db402bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:38.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.778+0000 7f8dc4eba640 1 -- 172.21.15.43:0/3147114844 shutdown_connections 2024-04-21T08:36:38.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.778+0000 7f8dc4eba640 1 --2- 172.21.15.43:0/3147114844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc01029a0 0x7f8dc01092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.778+0000 7f8dc4eba640 1 --2- 172.21.15.43:0/3147114844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8dc0101fe0 0x7f8dc0102460 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.778+0000 7f8dc4eba640 1 --2- 172.21.15.43:0/3147114844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dc0100e90 0x7f8dc01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.778+0000 7f8dc4eba640 1 -- 172.21.15.43:0/3147114844 >> 172.21.15.43:0/3147114844 conn(0x7f8dc00fc400 msgr2=0x7f8dc00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:38.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.778+0000 7f8dc4eba640 1 -- 172.21.15.43:0/3147114844 shutdown_connections 2024-04-21T08:36:38.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.778+0000 7f8dc4eba640 1 -- 172.21.15.43:0/3147114844 wait complete. 2024-04-21T08:36:38.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.779+0000 7f8dc4eba640 1 Processor -- start 2024-04-21T08:36:38.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.779+0000 7f8dc4eba640 1 -- start start 2024-04-21T08:36:38.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.779+0000 7f8dc4eba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc0100e90 0x7f8dc019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:38.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.779+0000 7f8dc4eba640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dc0101fe0 0x7f8dc019a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:38.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.779+0000 7f8dc4eba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8dc01029a0 0x7f8dc01a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:38.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.779+0000 7f8dc4eba640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc010c9f0 con 0x7f8dc01029a0 2024-04-21T08:36:38.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.779+0000 7f8dc4eba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc010c870 con 0x7f8dc0100e90 2024-04-21T08:36:38.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.779+0000 7f8dc4eba640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc010cb70 con 0x7f8dc0101fe0 2024-04-21T08:36:38.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.779+0000 7f8dbed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8dc01029a0 0x7f8dc01a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:38.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.779+0000 7f8dbed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8dc01029a0 0x7f8dc01a19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53392/0 (socket says 172.21.15.43:53392) 2024-04-21T08:36:38.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.779+0000 7f8dbed9d640 1 -- 172.21.15.43:0/4001842336 learned_addr learned my addr 172.21.15.43:0/4001842336 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:38.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.780+0000 7f8dbe59c640 1 --2- 172.21.15.43:0/4001842336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc0100e90 0x7f8dc019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:38.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.780+0000 7f8dbdd9b640 1 --2- 172.21.15.43:0/4001842336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dc0101fe0 0x7f8dc019a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:38.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.780+0000 7f8dbed9d640 1 -- 172.21.15.43:0/4001842336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dc0101fe0 msgr2=0x7f8dc019a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:38.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.780+0000 7f8dbed9d640 1 --2- 172.21.15.43:0/4001842336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dc0101fe0 0x7f8dc019a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.780+0000 7f8dbed9d640 1 -- 172.21.15.43:0/4001842336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc0100e90 msgr2=0x7f8dc019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:38.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.780+0000 7f8dbed9d640 1 --2- 172.21.15.43:0/4001842336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc0100e90 0x7f8dc019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.780+0000 7f8dbed9d640 1 -- 172.21.15.43:0/4001842336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc01a2060 con 0x7f8dc01029a0 2024-04-21T08:36:38.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.780+0000 7f8dbdd9b640 1 --2- 172.21.15.43:0/4001842336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dc0101fe0 0x7f8dc019a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:38.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.780+0000 7f8dbed9d640 1 --2- 172.21.15.43:0/4001842336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8dc01029a0 0x7f8dc01a19f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8db800eaa0 tx=0x7f8db800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:38.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.780+0000 7f8daf7fe640 1 -- 172.21.15.43:0/4001842336 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db800cdb0 con 0x7f8dc01029a0 2024-04-21T08:36:38.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.780+0000 7f8daf7fe640 1 -- 172.21.15.43:0/4001842336 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8db8005020 con 0x7f8dc01029a0 2024-04-21T08:36:38.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.780+0000 7f8daf7fe640 1 -- 172.21.15.43:0/4001842336 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db80116d0 con 0x7f8dc01029a0 2024-04-21T08:36:38.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.780+0000 7f8dc4eba640 1 -- 172.21.15.43:0/4001842336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dc01a2350 con 0x7f8dc01029a0 2024-04-21T08:36:38.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.780+0000 7f8dc4eba640 1 -- 172.21.15.43:0/4001842336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dc01a2850 con 0x7f8dc01029a0 2024-04-21T08:36:38.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.781+0000 7f8dc4eba640 1 -- 172.21.15.43:0/4001842336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dc01a2b10 con 0x7f8dc01029a0 2024-04-21T08:36:38.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.782+0000 7f8daf7fe640 1 -- 172.21.15.43:0/4001842336 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8db80051d0 con 0x7f8dc01029a0 2024-04-21T08:36:38.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.783+0000 7f8daf7fe640 1 --2- 172.21.15.43:0/4001842336 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8da80a3bb0 0x7f8da80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:38.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.783+0000 7f8dbe59c640 1 --2- 172.21.15.43:0/4001842336 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8da80a3bb0 0x7f8da80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:38.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.783+0000 7f8daf7fe640 1 -- 172.21.15.43:0/4001842336 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f8db80d0950 con 0x7f8dc01029a0 2024-04-21T08:36:38.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.783+0000 7f8dbe59c640 1 --2- 172.21.15.43:0/4001842336 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8da80a3bb0 0x7f8da80a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8db0009b60 tx=0x7f8db0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:38.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.787+0000 7f8daf7fe640 1 -- 172.21.15.43:0/4001842336 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8db8016030 con 0x7f8dc01029a0 2024-04-21T08:36:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.978+0000 7f8dc4eba640 1 -- 172.21.15.43:0/4001842336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8dc01a2ce0 con 0x7f8dc01029a0 2024-04-21T08:36:38.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.979+0000 7f8daf7fe640 1 -- 172.21.15.43:0/4001842336 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f8db809d090 con 0x7f8dc01029a0 2024-04-21T08:36:38.978 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-21T08:36:38.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.980+0000 7f8dc4eba640 1 -- 172.21.15.43:0/4001842336 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8da80a3bb0 msgr2=0x7f8da80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:38.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.980+0000 7f8dc4eba640 1 --2- 172.21.15.43:0/4001842336 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8da80a3bb0 0x7f8da80a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8db0009b60 tx=0x7f8db0005f90 comp rx=0 tx=0).stop 2024-04-21T08:36:38.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.980+0000 7f8dc4eba640 1 -- 172.21.15.43:0/4001842336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8dc01029a0 msgr2=0x7f8dc01a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:38.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.980+0000 7f8dc4eba640 1 --2- 172.21.15.43:0/4001842336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8dc01029a0 0x7f8dc01a19f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8db800eaa0 tx=0x7f8db800ef60 comp rx=0 tx=0).stop 2024-04-21T08:36:38.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.981+0000 7f8dc4eba640 1 -- 172.21.15.43:0/4001842336 shutdown_connections 2024-04-21T08:36:38.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.981+0000 7f8dc4eba640 1 --2- 172.21.15.43:0/4001842336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8dc01029a0 0x7f8dc01a19f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.981+0000 7f8dc4eba640 1 --2- 172.21.15.43:0/4001842336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dc0101fe0 0x7f8dc019a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.981+0000 7f8dc4eba640 1 --2- 172.21.15.43:0/4001842336 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8da80a3bb0 0x7f8da80a6070 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.981+0000 7f8dc4eba640 1 --2- 172.21.15.43:0/4001842336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc0100e90 0x7f8dc019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:38.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.981+0000 7f8dc4eba640 1 -- 172.21.15.43:0/4001842336 >> 172.21.15.43:0/4001842336 conn(0x7f8dc00fc400 msgr2=0x7f8dc0069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:38.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.981+0000 7f8dc4eba640 1 -- 172.21.15.43:0/4001842336 shutdown_connections 2024-04-21T08:36:38.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:38.981+0000 7f8dc4eba640 1 -- 172.21.15.43:0/4001842336 wait complete. 2024-04-21T08:36:38.991 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:36:39.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.098+0000 7f064025c640 1 -- 172.21.15.43:0/1926909061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0638103b50 msgr2=0x7f063810a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:39.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.098+0000 7f064025c640 1 --2- 172.21.15.43:0/1926909061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0638103b50 0x7f063810a3e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0630009ca0 tx=0x7f063002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:39.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.098+0000 7f064025c640 1 -- 172.21.15.43:0/1926909061 shutdown_connections 2024-04-21T08:36:39.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.098+0000 7f064025c640 1 --2- 172.21.15.43:0/1926909061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0638103b50 0x7f063810a3e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:39.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.098+0000 7f064025c640 1 --2- 172.21.15.43:0/1926909061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0638103190 0x7f0638103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:39.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.098+0000 7f064025c640 1 --2- 172.21.15.43:0/1926909061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0638101fa0 0x7f06381023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:39.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.098+0000 7f064025c640 1 -- 172.21.15.43:0/1926909061 >> 172.21.15.43:0/1926909061 conn(0x7f06380fd530 msgr2=0x7f06380ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:39.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.098+0000 7f064025c640 1 -- 172.21.15.43:0/1926909061 shutdown_connections 2024-04-21T08:36:39.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.098+0000 7f064025c640 1 -- 172.21.15.43:0/1926909061 wait complete. 2024-04-21T08:36:39.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.099+0000 7f064025c640 1 Processor -- start 2024-04-21T08:36:39.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.099+0000 7f064025c640 1 -- start start 2024-04-21T08:36:39.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.099+0000 7f064025c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0638101fa0 0x7f0638198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:39.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.099+0000 7f064025c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0638103190 0x7f0638198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:39.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.099+0000 7f064025c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0638103b50 0x7f063819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:39.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.099+0000 7f064025c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f063810dae0 con 0x7f0638103b50 2024-04-21T08:36:39.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.099+0000 7f064025c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f063810d960 con 0x7f0638101fa0 2024-04-21T08:36:39.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.099+0000 7f064025c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f063810dc60 con 0x7f0638103190 2024-04-21T08:36:39.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.099+0000 7f063e7f9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0638103b50 0x7f063819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:39.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.099+0000 7f063e7f9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0638103b50 0x7f063819f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53410/0 (socket says 172.21.15.43:53410) 2024-04-21T08:36:39.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.099+0000 7f063e7f9640 1 -- 172.21.15.43:0/3428105910 learned_addr learned my addr 172.21.15.43:0/3428105910 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:39.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.099+0000 7f063d7f7640 1 --2- 172.21.15.43:0/3428105910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0638103190 0x7f0638198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:39.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.100+0000 7f063dff8640 1 --2- 172.21.15.43:0/3428105910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0638101fa0 0x7f0638198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:39.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.100+0000 7f063e7f9640 1 -- 172.21.15.43:0/3428105910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0638103190 msgr2=0x7f0638198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:39.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.100+0000 7f063e7f9640 1 --2- 172.21.15.43:0/3428105910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0638103190 0x7f0638198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:39.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.100+0000 7f063e7f9640 1 -- 172.21.15.43:0/3428105910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0638101fa0 msgr2=0x7f0638198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:39.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.100+0000 7f063e7f9640 1 --2- 172.21.15.43:0/3428105910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0638101fa0 0x7f0638198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:39.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.100+0000 7f063e7f9640 1 -- 172.21.15.43:0/3428105910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063819fd20 con 0x7f0638103b50 2024-04-21T08:36:39.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.100+0000 7f063e7f9640 1 --2- 172.21.15.43:0/3428105910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0638103b50 0x7f063819f6b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f063002c700 tx=0x7f0630034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:39.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.100+0000 7f062effd640 1 -- 172.21.15.43:0/3428105910 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630039040 con 0x7f0638103b50 2024-04-21T08:36:39.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.100+0000 7f064025c640 1 -- 172.21.15.43:0/3428105910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f063819ffb0 con 0x7f0638103b50 2024-04-21T08:36:39.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.100+0000 7f062effd640 1 -- 172.21.15.43:0/3428105910 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0630044070 con 0x7f0638103b50 2024-04-21T08:36:39.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.100+0000 7f062effd640 1 -- 172.21.15.43:0/3428105910 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06300042a0 con 0x7f0638103b50 2024-04-21T08:36:39.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.100+0000 7f064025c640 1 -- 172.21.15.43:0/3428105910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06381a0470 con 0x7f0638103b50 2024-04-21T08:36:39.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.101+0000 7f064025c640 1 -- 172.21.15.43:0/3428105910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06381092c0 con 0x7f0638103b50 2024-04-21T08:36:39.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.102+0000 7f062effd640 1 -- 172.21.15.43:0/3428105910 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0630003970 con 0x7f0638103b50 2024-04-21T08:36:39.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.103+0000 7f062effd640 1 --2- 172.21.15.43:0/3428105910 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f06240a3bb0 0x7f06240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:39.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.103+0000 7f062effd640 1 -- 172.21.15.43:0/3428105910 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f06300f2a00 con 0x7f0638103b50 2024-04-21T08:36:39.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.103+0000 7f063dff8640 1 --2- 172.21.15.43:0/3428105910 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f06240a3bb0 0x7f06240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:39.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.103+0000 7f063dff8640 1 --2- 172.21.15.43:0/3428105910 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f06240a3bb0 0x7f06240a6070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0634005fd0 tx=0x7f0634005ce0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:39.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.107+0000 7f062effd640 1 -- 172.21.15.43:0/3428105910 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f06300bf1a0 con 0x7f0638103b50 2024-04-21T08:36:39.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.292+0000 7f064025c640 1 -- 172.21.15.43:0/3428105910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0638062b50 con 0x7f0638103b50 2024-04-21T08:36:39.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.293+0000 7f062effd640 1 -- 172.21.15.43:0/3428105910 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f06300beaa0 con 0x7f0638103b50 2024-04-21T08:36:39.292 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:36:39.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.294+0000 7f064025c640 1 -- 172.21.15.43:0/3428105910 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f06240a3bb0 msgr2=0x7f06240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:39.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.294+0000 7f064025c640 1 --2- 172.21.15.43:0/3428105910 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f06240a3bb0 0x7f06240a6070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0634005fd0 tx=0x7f0634005ce0 comp rx=0 tx=0).stop 2024-04-21T08:36:39.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.294+0000 7f064025c640 1 -- 172.21.15.43:0/3428105910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0638103b50 msgr2=0x7f063819f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:39.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.294+0000 7f064025c640 1 --2- 172.21.15.43:0/3428105910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0638103b50 0x7f063819f6b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f063002c700 tx=0x7f0630034760 comp rx=0 tx=0).stop 2024-04-21T08:36:39.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.294+0000 7f064025c640 1 -- 172.21.15.43:0/3428105910 shutdown_connections 2024-04-21T08:36:39.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.294+0000 7f064025c640 1 --2- 172.21.15.43:0/3428105910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0638103b50 0x7f063819f6b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:39.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.294+0000 7f064025c640 1 --2- 172.21.15.43:0/3428105910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0638103190 0x7f0638198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:39.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.294+0000 7f064025c640 1 --2- 172.21.15.43:0/3428105910 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f06240a3bb0 0x7f06240a6070 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:39.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.294+0000 7f064025c640 1 --2- 172.21.15.43:0/3428105910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0638101fa0 0x7f0638198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:39.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.294+0000 7f064025c640 1 -- 172.21.15.43:0/3428105910 >> 172.21.15.43:0/3428105910 conn(0x7f06380fd530 msgr2=0x7f0638106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:39.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.295+0000 7f064025c640 1 -- 172.21.15.43:0/3428105910 shutdown_connections 2024-04-21T08:36:39.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.295+0000 7f064025c640 1 -- 172.21.15.43:0/3428105910 wait complete. 2024-04-21T08:36:39.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:39 smithi112 ceph-mon[31085]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 410 B/s wr, 2 op/s 2024-04-21T08:36:39.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:39 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1396983060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:39.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:39 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1011055732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:36:39.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:39 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/220905310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:36:39.295 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:39 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/4001842336' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:36:39.304 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:39 smithi043 ceph-mon[27978]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 410 B/s wr, 2 op/s 2024-04-21T08:36:39.304 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:39 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1396983060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:39.304 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:39 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1011055732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:36:39.304 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:39 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/220905310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:36:39.304 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:39 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/4001842336' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:36:39.305 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-21T08:36:39.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.410+0000 7f9cc330e640 1 -- 172.21.15.43:0/3913730004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc101fe0 msgr2=0x7f9cbc102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:39.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.410+0000 7f9cc330e640 1 --2- 172.21.15.43:0/3913730004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc101fe0 0x7f9cbc102460 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009900 tx=0x7f9cb002bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:39.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.410+0000 7f9cc330e640 1 -- 172.21.15.43:0/3913730004 shutdown_connections 2024-04-21T08:36:39.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.410+0000 7f9cc330e640 1 --2- 172.21.15.43:0/3913730004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cbc1029a0 0x7f9cbc1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:39.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.410+0000 7f9cc330e640 1 --2- 172.21.15.43:0/3913730004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc101fe0 0x7f9cbc102460 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:39.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.410+0000 7f9cc330e640 1 --2- 172.21.15.43:0/3913730004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cbc100e90 0x7f9cbc1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:39.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.410+0000 7f9cc330e640 1 -- 172.21.15.43:0/3913730004 >> 172.21.15.43:0/3913730004 conn(0x7f9cbc0fc400 msgr2=0x7f9cbc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:39.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.410+0000 7f9cc330e640 1 -- 172.21.15.43:0/3913730004 shutdown_connections 2024-04-21T08:36:39.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.410+0000 7f9cc330e640 1 -- 172.21.15.43:0/3913730004 wait complete. 2024-04-21T08:36:39.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.411+0000 7f9cc330e640 1 Processor -- start 2024-04-21T08:36:39.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.411+0000 7f9cc330e640 1 -- start start 2024-04-21T08:36:39.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.411+0000 7f9cc330e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cbc100e90 0x7f9cbc1970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:39.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.411+0000 7f9cc330e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cbc101fe0 0x7f9cbc1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:39.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.411+0000 7f9cc330e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc1029a0 0x7f9cbc19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:39.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.411+0000 7f9cc330e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc10c9d0 con 0x7f9cbc1029a0 2024-04-21T08:36:39.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.411+0000 7f9cc330e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc10c850 con 0x7f9cbc100e90 2024-04-21T08:36:39.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.411+0000 7f9cc330e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc10cb50 con 0x7f9cbc101fe0 2024-04-21T08:36:39.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.411+0000 7f9cc18ab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc1029a0 0x7f9cbc19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:39.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.411+0000 7f9cc18ab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc1029a0 0x7f9cbc19e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53418/0 (socket says 172.21.15.43:53418) 2024-04-21T08:36:39.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.411+0000 7f9cc18ab640 1 -- 172.21.15.43:0/3461627904 learned_addr learned my addr 172.21.15.43:0/3461627904 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:39.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.411+0000 7f9cc10aa640 1 --2- 172.21.15.43:0/3461627904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cbc100e90 0x7f9cbc1970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:39.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.411+0000 7f9cc18ab640 1 -- 172.21.15.43:0/3461627904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cbc101fe0 msgr2=0x7f9cbc1975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:39.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.411+0000 7f9cc08a9640 1 --2- 172.21.15.43:0/3461627904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cbc101fe0 0x7f9cbc1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:39.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.411+0000 7f9cc18ab640 1 --2- 172.21.15.43:0/3461627904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cbc101fe0 0x7f9cbc1975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:39.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.411+0000 7f9cc18ab640 1 -- 172.21.15.43:0/3461627904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cbc100e90 msgr2=0x7f9cbc1970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:39.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.411+0000 7f9cc18ab640 1 --2- 172.21.15.43:0/3461627904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cbc100e90 0x7f9cbc1970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:39.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.411+0000 7f9cc18ab640 1 -- 172.21.15.43:0/3461627904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cbc19ecd0 con 0x7f9cbc1029a0 2024-04-21T08:36:39.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.411+0000 7f9cc08a9640 1 --2- 172.21.15.43:0/3461627904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cbc101fe0 0x7f9cbc1975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:39.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.412+0000 7f9cc18ab640 1 --2- 172.21.15.43:0/3461627904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc1029a0 0x7f9cbc19e660 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00cce0 tx=0x7f9cac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:39.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.412+0000 7f9cba7fc640 1 -- 172.21.15.43:0/3461627904 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac00f450 con 0x7f9cbc1029a0 2024-04-21T08:36:39.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.412+0000 7f9cba7fc640 1 -- 172.21.15.43:0/3461627904 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cac00faa0 con 0x7f9cbc1029a0 2024-04-21T08:36:39.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.412+0000 7f9cc330e640 1 -- 172.21.15.43:0/3461627904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cbc19efc0 con 0x7f9cbc1029a0 2024-04-21T08:36:39.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.412+0000 7f9cba7fc640 1 -- 172.21.15.43:0/3461627904 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac018b70 con 0x7f9cbc1029a0 2024-04-21T08:36:39.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.412+0000 7f9cc330e640 1 -- 172.21.15.43:0/3461627904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cbc19f430 con 0x7f9cbc1029a0 2024-04-21T08:36:39.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.413+0000 7f9cc330e640 1 -- 172.21.15.43:0/3461627904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cbc1081c0 con 0x7f9cbc1029a0 2024-04-21T08:36:39.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.414+0000 7f9cba7fc640 1 -- 172.21.15.43:0/3461627904 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9cac018d10 con 0x7f9cbc1029a0 2024-04-21T08:36:39.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.414+0000 7f9cba7fc640 1 --2- 172.21.15.43:0/3461627904 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9ca40a3bb0 0x7f9ca40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:39.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.415+0000 7f9cc10aa640 1 --2- 172.21.15.43:0/3461627904 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9ca40a3bb0 0x7f9ca40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:39.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.415+0000 7f9cba7fc640 1 -- 172.21.15.43:0/3461627904 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f9cac0d0be0 con 0x7f9cbc1029a0 2024-04-21T08:36:39.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.415+0000 7f9cc10aa640 1 --2- 172.21.15.43:0/3461627904 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9ca40a3bb0 0x7f9ca40a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4005fd0 tx=0x7f9cb4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:39.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.418+0000 7f9cba7fc640 1 -- 172.21.15.43:0/3461627904 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9cac014030 con 0x7f9cbc1029a0 2024-04-21T08:36:39.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:39 smithi175 ceph-mon[31920]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 410 B/s wr, 2 op/s 2024-04-21T08:36:39.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:39 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1396983060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:39.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:39 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1011055732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:36:39.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:39 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/220905310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:36:39.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:39 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/4001842336' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:36:39.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.620+0000 7f9cc330e640 1 -- 172.21.15.43:0/3461627904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9cbc02d810 con 0x7f9cbc1029a0 2024-04-21T08:36:39.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.621+0000 7f9cba7fc640 1 -- 172.21.15.43:0/3461627904 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+590 (secure 0 0 0) 0x7f9cac09d410 con 0x7f9cbc1029a0 2024-04-21T08:36:39.620 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/1916182222 2024-04-22T08:29:13.889526+0000 2024-04-21T08:36:39.620 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6801/3200866696 2024-04-22T08:29:13.889526+0000 2024-04-21T08:36:39.620 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6801/1559586579 2024-04-22T08:28:46.855699+0000 2024-04-21T08:36:39.620 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6800/3200866696 2024-04-22T08:29:13.889526+0000 2024-04-21T08:36:39.620 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/4047106671 2024-04-22T08:29:13.889526+0000 2024-04-21T08:36:39.621 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6800/1559586579 2024-04-22T08:28:46.855699+0000 2024-04-21T08:36:39.621 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/347461706 2024-04-22T08:29:13.889526+0000 2024-04-21T08:36:39.621 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/887926369 2024-04-22T08:28:46.855699+0000 2024-04-21T08:36:39.621 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/2598599313 2024-04-22T08:28:46.855699+0000 2024-04-21T08:36:39.621 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/2126312819 2024-04-22T08:28:46.855699+0000 2024-04-21T08:36:39.621 INFO:teuthology.orchestra.run.smithi043.stderr:listed 10 entries 2024-04-21T08:36:39.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.623+0000 7f9cc330e640 1 -- 172.21.15.43:0/3461627904 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9ca40a3bb0 msgr2=0x7f9ca40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:39.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.623+0000 7f9cc330e640 1 --2- 172.21.15.43:0/3461627904 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9ca40a3bb0 0x7f9ca40a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4005fd0 tx=0x7f9cb4005850 comp rx=0 tx=0).stop 2024-04-21T08:36:39.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.623+0000 7f9cc330e640 1 -- 172.21.15.43:0/3461627904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc1029a0 msgr2=0x7f9cbc19e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:39.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.623+0000 7f9cc330e640 1 --2- 172.21.15.43:0/3461627904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc1029a0 0x7f9cbc19e660 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00cce0 tx=0x7f9cac00a620 comp rx=0 tx=0).stop 2024-04-21T08:36:39.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.623+0000 7f9cc330e640 1 -- 172.21.15.43:0/3461627904 shutdown_connections 2024-04-21T08:36:39.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.623+0000 7f9cc330e640 1 --2- 172.21.15.43:0/3461627904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc1029a0 0x7f9cbc19e660 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:39.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.623+0000 7f9cc330e640 1 --2- 172.21.15.43:0/3461627904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cbc101fe0 0x7f9cbc1975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:39.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.623+0000 7f9cc330e640 1 --2- 172.21.15.43:0/3461627904 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9ca40a3bb0 0x7f9ca40a6070 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:39.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.623+0000 7f9cc330e640 1 --2- 172.21.15.43:0/3461627904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cbc100e90 0x7f9cbc1970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:39.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.623+0000 7f9cc330e640 1 -- 172.21.15.43:0/3461627904 >> 172.21.15.43:0/3461627904 conn(0x7f9cbc0fc400 msgr2=0x7f9cbc105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:39.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.623+0000 7f9cc330e640 1 -- 172.21.15.43:0/3461627904 shutdown_connections 2024-04-21T08:36:39.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:39.623+0000 7f9cc330e640 1 -- 172.21.15.43:0/3461627904 wait complete. 2024-04-21T08:36:39.633 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2024-04-21T08:36:39.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:36:39.633 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-21T08:36:39.656 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi043.stderr:ceph-fuse[69663]: fuse finished with error 0 and tester_r 0 2024-04-21T08:36:39.658 INFO:teuthology.orchestra.run:waiting for 300 2024-04-21T08:36:40.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:40 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3428105910' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:36:40.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:40 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3461627904' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:36:40.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:40 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3428105910' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:36:40.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:40 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3461627904' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:36:40.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:40 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3428105910' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:36:40.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:40 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3461627904' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:36:41.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:41 smithi112 ceph-mon[31085]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 372 B/s wr, 2 op/s 2024-04-21T08:36:41.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:41 smithi043 ceph-mon[27978]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 372 B/s wr, 2 op/s 2024-04-21T08:36:41.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:41 smithi175 ceph-mon[31920]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 372 B/s wr, 2 op/s 2024-04-21T08:36:43.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:43 smithi112 ceph-mon[31085]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-21T08:36:43.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:43 smithi043 ceph-mon[27978]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-21T08:36:43.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:43 smithi175 ceph-mon[31920]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-21T08:36:45.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:45 smithi112 ceph-mon[31085]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T08:36:45.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:45 smithi043 ceph-mon[27978]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T08:36:45.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:45 smithi175 ceph-mon[31920]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T08:36:45.760 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-21T08:36:45.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:36:45.760 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-21T08:36:45.787 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:36:45.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.940+0000 7f9058eb3640 1 -- 172.21.15.43:0/2078476106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9054101fa0 msgr2=0x7f90541023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:45.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.940+0000 7f9058eb3640 1 --2- 172.21.15.43:0/2078476106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9054101fa0 0x7f90541023c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9044009900 tx=0x7f904402bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:45.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.940+0000 7f9058eb3640 1 -- 172.21.15.43:0/2078476106 shutdown_connections 2024-04-21T08:36:45.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.940+0000 7f9058eb3640 1 --2- 172.21.15.43:0/2078476106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9054103b50 0x7f905410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:45.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.940+0000 7f9058eb3640 1 --2- 172.21.15.43:0/2078476106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9054103190 0x7f9054103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:45.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.940+0000 7f9058eb3640 1 --2- 172.21.15.43:0/2078476106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9054101fa0 0x7f90541023c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:45.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.940+0000 7f9058eb3640 1 -- 172.21.15.43:0/2078476106 >> 172.21.15.43:0/2078476106 conn(0x7f90540fd530 msgr2=0x7f90540ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:45.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.940+0000 7f9058eb3640 1 -- 172.21.15.43:0/2078476106 shutdown_connections 2024-04-21T08:36:45.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.940+0000 7f9058eb3640 1 -- 172.21.15.43:0/2078476106 wait complete. 2024-04-21T08:36:45.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.941+0000 7f9058eb3640 1 Processor -- start 2024-04-21T08:36:45.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.941+0000 7f9058eb3640 1 -- start start 2024-04-21T08:36:45.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.941+0000 7f9058eb3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9054101fa0 0x7f9054198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:45.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.941+0000 7f9058eb3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9054103190 0x7f90541986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:45.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.941+0000 7f9058eb3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9054103b50 0x7f905419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:45.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.941+0000 7f9058eb3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f905410dae0 con 0x7f9054103190 2024-04-21T08:36:45.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.941+0000 7f9058eb3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f905410d960 con 0x7f9054103b50 2024-04-21T08:36:45.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.941+0000 7f9058eb3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f905410dc60 con 0x7f9054101fa0 2024-04-21T08:36:45.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.941+0000 7f904bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9054103190 0x7f90541986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:45.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.941+0000 7f904bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9054103190 0x7f90541986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53432/0 (socket says 172.21.15.43:53432) 2024-04-21T08:36:45.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.941+0000 7f904bfff640 1 -- 172.21.15.43:0/3636013284 learned_addr learned my addr 172.21.15.43:0/3636013284 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:45.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.942+0000 7f9052d9d640 1 --2- 172.21.15.43:0/3636013284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9054103b50 0x7f905419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:45.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.942+0000 7f905259c640 1 --2- 172.21.15.43:0/3636013284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9054101fa0 0x7f9054198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:45.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.942+0000 7f904bfff640 1 -- 172.21.15.43:0/3636013284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9054101fa0 msgr2=0x7f9054198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:45.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.942+0000 7f904bfff640 1 --2- 172.21.15.43:0/3636013284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9054101fa0 0x7f9054198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:45.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.942+0000 7f904bfff640 1 -- 172.21.15.43:0/3636013284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9054103b50 msgr2=0x7f905419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:45.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.942+0000 7f904bfff640 1 --2- 172.21.15.43:0/3636013284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9054103b50 0x7f905419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:45.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.942+0000 7f904bfff640 1 -- 172.21.15.43:0/3636013284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905419fda0 con 0x7f9054103190 2024-04-21T08:36:45.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.942+0000 7f9052d9d640 1 --2- 172.21.15.43:0/3636013284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9054103b50 0x7f905419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:45.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.942+0000 7f905259c640 1 --2- 172.21.15.43:0/3636013284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9054101fa0 0x7f9054198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:45.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.942+0000 7f904bfff640 1 --2- 172.21.15.43:0/3636013284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9054103190 0x7f90541986c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f903c00d950 tx=0x7f903c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:45.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.942+0000 7f904b7fe640 1 -- 172.21.15.43:0/3636013284 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c017070 con 0x7f9054103190 2024-04-21T08:36:45.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.942+0000 7f904b7fe640 1 -- 172.21.15.43:0/3636013284 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f903c009c30 con 0x7f9054103190 2024-04-21T08:36:45.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.942+0000 7f9058eb3640 1 -- 172.21.15.43:0/3636013284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90541a0090 con 0x7f9054103190 2024-04-21T08:36:45.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.942+0000 7f904b7fe640 1 -- 172.21.15.43:0/3636013284 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c00f620 con 0x7f9054103190 2024-04-21T08:36:45.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.942+0000 7f9058eb3640 1 -- 172.21.15.43:0/3636013284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90541a05e0 con 0x7f9054103190 2024-04-21T08:36:45.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.943+0000 7f9058eb3640 1 -- 172.21.15.43:0/3636013284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90541092c0 con 0x7f9054103190 2024-04-21T08:36:45.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.945+0000 7f904b7fe640 1 -- 172.21.15.43:0/3636013284 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f903c010460 con 0x7f9054103190 2024-04-21T08:36:45.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.945+0000 7f904b7fe640 1 --2- 172.21.15.43:0/3636013284 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f90400a3bb0 0x7f90400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:45.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.946+0000 7f905259c640 1 --2- 172.21.15.43:0/3636013284 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f90400a3bb0 0x7f90400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:45.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.946+0000 7f904b7fe640 1 -- 172.21.15.43:0/3636013284 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f903c0d0320 con 0x7f9054103190 2024-04-21T08:36:45.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.946+0000 7f905259c640 1 --2- 172.21.15.43:0/3636013284 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f90400a3bb0 0x7f90400a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f90440096c0 tx=0x7f9044000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:45.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:45.950+0000 7f904b7fe640 1 -- 172.21.15.43:0/3636013284 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f903c09cac0 con 0x7f9054103190 2024-04-21T08:36:46.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.139+0000 7f9058eb3640 1 -- 172.21.15.43:0/3636013284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9054062b80 con 0x7f9054103190 2024-04-21T08:36:46.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.141+0000 7f904b7fe640 1 -- 172.21.15.43:0/3636013284 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24152 (secure 0 0 0) 0x7f903c009de0 con 0x7f9054103190 2024-04-21T08:36:46.141 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:46.141 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":90,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:36:28.364093+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:36:46.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.143+0000 7f9058eb3640 1 -- 172.21.15.43:0/3636013284 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f90400a3bb0 msgr2=0x7f90400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:46.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.143+0000 7f9058eb3640 1 --2- 172.21.15.43:0/3636013284 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f90400a3bb0 0x7f90400a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f90440096c0 tx=0x7f9044000bc0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.143+0000 7f9058eb3640 1 -- 172.21.15.43:0/3636013284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9054103190 msgr2=0x7f90541986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:46.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.143+0000 7f9058eb3640 1 --2- 172.21.15.43:0/3636013284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9054103190 0x7f90541986c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f903c00d950 tx=0x7f903c00de10 comp rx=0 tx=0).stop 2024-04-21T08:36:46.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.143+0000 7f9058eb3640 1 -- 172.21.15.43:0/3636013284 shutdown_connections 2024-04-21T08:36:46.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.143+0000 7f9058eb3640 1 --2- 172.21.15.43:0/3636013284 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f90400a3bb0 0x7f90400a6070 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.143+0000 7f9058eb3640 1 --2- 172.21.15.43:0/3636013284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9054103b50 0x7f905419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.143+0000 7f9058eb3640 1 --2- 172.21.15.43:0/3636013284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9054103190 0x7f90541986c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.143+0000 7f9058eb3640 1 --2- 172.21.15.43:0/3636013284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9054101fa0 0x7f9054198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.143+0000 7f9058eb3640 1 -- 172.21.15.43:0/3636013284 >> 172.21.15.43:0/3636013284 conn(0x7f90540fd530 msgr2=0x7f9054106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:46.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.143+0000 7f9058eb3640 1 -- 172.21.15.43:0/3636013284 shutdown_connections 2024-04-21T08:36:46.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.143+0000 7f9058eb3640 1 -- 172.21.15.43:0/3636013284 wait complete. 2024-04-21T08:36:46.155 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:36:46.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.258+0000 7f1a1cb15640 1 -- 172.21.15.43:0/172972638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a18102a40 msgr2=0x7f1a18104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:46.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.258+0000 7f1a1cb15640 1 --2- 172.21.15.43:0/172972638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a18102a40 0x7f1a18104e30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009900 tx=0x7f1a1002bc50 comp rx=0 tx=0).stop 2024-04-21T08:36:46.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.258+0000 7f1a1cb15640 1 -- 172.21.15.43:0/172972638 shutdown_connections 2024-04-21T08:36:46.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.258+0000 7f1a1cb15640 1 --2- 172.21.15.43:0/172972638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a18107ca0 0x7f1a1810a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.258+0000 7f1a1cb15640 1 --2- 172.21.15.43:0/172972638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a18105370 0x7f1a18107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.258+0000 7f1a1cb15640 1 --2- 172.21.15.43:0/172972638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a18102a40 0x7f1a18104e30 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.258+0000 7f1a1cb15640 1 -- 172.21.15.43:0/172972638 >> 172.21.15.43:0/172972638 conn(0x7f1a180fc470 msgr2=0x7f1a180fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:46.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.258+0000 7f1a1cb15640 1 -- 172.21.15.43:0/172972638 shutdown_connections 2024-04-21T08:36:46.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.258+0000 7f1a1cb15640 1 -- 172.21.15.43:0/172972638 wait complete. 2024-04-21T08:36:46.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.258+0000 7f1a1cb15640 1 Processor -- start 2024-04-21T08:36:46.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.258+0000 7f1a1cb15640 1 -- start start 2024-04-21T08:36:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.259+0000 7f1a1cb15640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a18102a40 0x7f1a18198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.259+0000 7f1a1cb15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a18105370 0x7f1a18198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.259+0000 7f1a1cb15640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a18107ca0 0x7f1a1819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.259+0000 7f1a1cb15640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1810dbd0 con 0x7f1a18107ca0 2024-04-21T08:36:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.259+0000 7f1a1cb15640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1810da50 con 0x7f1a18105370 2024-04-21T08:36:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.259+0000 7f1a1cb15640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1810dd50 con 0x7f1a18102a40 2024-04-21T08:36:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.259+0000 7f1a16d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a18107ca0 0x7f1a1819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.259+0000 7f1a16d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a18107ca0 0x7f1a1819f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53440/0 (socket says 172.21.15.43:53440) 2024-04-21T08:36:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.259+0000 7f1a16d9d640 1 -- 172.21.15.43:0/2766904562 learned_addr learned my addr 172.21.15.43:0/2766904562 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.259+0000 7f1a15d9b640 1 --2- 172.21.15.43:0/2766904562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a18105370 0x7f1a18198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.259+0000 7f1a1659c640 1 --2- 172.21.15.43:0/2766904562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a18102a40 0x7f1a18198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.259+0000 7f1a16d9d640 1 -- 172.21.15.43:0/2766904562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a18102a40 msgr2=0x7f1a18198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.259+0000 7f1a16d9d640 1 --2- 172.21.15.43:0/2766904562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a18102a40 0x7f1a18198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.259+0000 7f1a16d9d640 1 -- 172.21.15.43:0/2766904562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a18105370 msgr2=0x7f1a18198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:46.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.259+0000 7f1a16d9d640 1 --2- 172.21.15.43:0/2766904562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a18105370 0x7f1a18198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.259+0000 7f1a16d9d640 1 -- 172.21.15.43:0/2766904562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1819fe60 con 0x7f1a18107ca0 2024-04-21T08:36:46.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.259+0000 7f1a1659c640 1 --2- 172.21.15.43:0/2766904562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a18102a40 0x7f1a18198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:46.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.259+0000 7f1a16d9d640 1 --2- 172.21.15.43:0/2766904562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a18107ca0 0x7f1a1819f7f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800b990 tx=0x7f1a0800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:46.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.259+0000 7f1a077fe640 1 -- 172.21.15.43:0/2766904562 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08019070 con 0x7f1a18107ca0 2024-04-21T08:36:46.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.259+0000 7f1a1cb15640 1 -- 172.21.15.43:0/2766904562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a181a0150 con 0x7f1a18107ca0 2024-04-21T08:36:46.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.259+0000 7f1a077fe640 1 -- 172.21.15.43:0/2766904562 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a0800cba0 con 0x7f1a18107ca0 2024-04-21T08:36:46.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.259+0000 7f1a077fe640 1 -- 172.21.15.43:0/2766904562 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a080075d0 con 0x7f1a18107ca0 2024-04-21T08:36:46.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.260+0000 7f1a1cb15640 1 -- 172.21.15.43:0/2766904562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a181a0770 con 0x7f1a18107ca0 2024-04-21T08:36:46.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.261+0000 7f1a1cb15640 1 -- 172.21.15.43:0/2766904562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a180fdf60 con 0x7f1a18107ca0 2024-04-21T08:36:46.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.262+0000 7f1a077fe640 1 -- 172.21.15.43:0/2766904562 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1a0801c070 con 0x7f1a18107ca0 2024-04-21T08:36:46.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.262+0000 7f1a077fe640 1 --2- 172.21.15.43:0/2766904562 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f1a000a3bb0 0x7f1a000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:46.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.262+0000 7f1a077fe640 1 -- 172.21.15.43:0/2766904562 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f1a080cfbc0 con 0x7f1a18107ca0 2024-04-21T08:36:46.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.263+0000 7f1a1659c640 1 --2- 172.21.15.43:0/2766904562 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f1a000a3bb0 0x7f1a000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:46.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.263+0000 7f1a1659c640 1 --2- 172.21.15.43:0/2766904562 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f1a000a3bb0 0x7f1a000a6070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1a10036040 tx=0x7f1a10005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:46.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.267+0000 7f1a077fe640 1 -- 172.21.15.43:0/2766904562 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1a0809bd10 con 0x7f1a18107ca0 2024-04-21T08:36:46.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.457+0000 7f1a1cb15640 1 -- 172.21.15.43:0/2766904562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a18062b50 con 0x7f1a18107ca0 2024-04-21T08:36:46.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.459+0000 7f1a077fe640 1 -- 172.21.15.43:0/2766904562 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24152 (secure 0 0 0) 0x7f1a080bae20 con 0x7f1a18107ca0 2024-04-21T08:36:46.459 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:46.459 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":90,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:36:28.364093+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:36:46.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.461+0000 7f1a1cb15640 1 -- 172.21.15.43:0/2766904562 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f1a000a3bb0 msgr2=0x7f1a000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:46.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.461+0000 7f1a1cb15640 1 --2- 172.21.15.43:0/2766904562 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f1a000a3bb0 0x7f1a000a6070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1a10036040 tx=0x7f1a10005fd0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.461+0000 7f1a1cb15640 1 -- 172.21.15.43:0/2766904562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a18107ca0 msgr2=0x7f1a1819f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:46.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.461+0000 7f1a1cb15640 1 --2- 172.21.15.43:0/2766904562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a18107ca0 0x7f1a1819f7f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800b990 tx=0x7f1a0800be50 comp rx=0 tx=0).stop 2024-04-21T08:36:46.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.461+0000 7f1a1cb15640 1 -- 172.21.15.43:0/2766904562 shutdown_connections 2024-04-21T08:36:46.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.461+0000 7f1a1cb15640 1 --2- 172.21.15.43:0/2766904562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a18107ca0 0x7f1a1819f7f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.461+0000 7f1a1cb15640 1 --2- 172.21.15.43:0/2766904562 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f1a000a3bb0 0x7f1a000a6070 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.461+0000 7f1a1cb15640 1 --2- 172.21.15.43:0/2766904562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a18105370 0x7f1a18198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.461+0000 7f1a1cb15640 1 --2- 172.21.15.43:0/2766904562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a18102a40 0x7f1a18198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.461+0000 7f1a1cb15640 1 -- 172.21.15.43:0/2766904562 >> 172.21.15.43:0/2766904562 conn(0x7f1a180fc470 msgr2=0x7f1a18103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:46.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.461+0000 7f1a1cb15640 1 -- 172.21.15.43:0/2766904562 shutdown_connections 2024-04-21T08:36:46.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.461+0000 7f1a1cb15640 1 -- 172.21.15.43:0/2766904562 wait complete. 2024-04-21T08:36:46.473 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-21T08:36:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.578+0000 7fefeb055640 1 -- 172.21.15.43:0/3674313061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefe4103190 msgr2=0x7fefe4103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.578+0000 7fefeb055640 1 --2- 172.21.15.43:0/3674313061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefe4103190 0x7fefe4103610 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fefd0009900 tx=0x7fefd002bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.578+0000 7fefeb055640 1 -- 172.21.15.43:0/3674313061 shutdown_connections 2024-04-21T08:36:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.578+0000 7fefeb055640 1 --2- 172.21.15.43:0/3674313061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefe4103b50 0x7fefe410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.578+0000 7fefeb055640 1 --2- 172.21.15.43:0/3674313061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefe4103190 0x7fefe4103610 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.578+0000 7fefeb055640 1 --2- 172.21.15.43:0/3674313061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefe4101fa0 0x7fefe41023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.578+0000 7fefeb055640 1 -- 172.21.15.43:0/3674313061 >> 172.21.15.43:0/3674313061 conn(0x7fefe40fd530 msgr2=0x7fefe40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.578+0000 7fefeb055640 1 -- 172.21.15.43:0/3674313061 shutdown_connections 2024-04-21T08:36:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.578+0000 7fefeb055640 1 -- 172.21.15.43:0/3674313061 wait complete. 2024-04-21T08:36:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.579+0000 7fefeb055640 1 Processor -- start 2024-04-21T08:36:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.579+0000 7fefeb055640 1 -- start start 2024-04-21T08:36:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.579+0000 7fefeb055640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefe4101fa0 0x7fefe4075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.579+0000 7fefeb055640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefe4103190 0x7fefe4074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.579+0000 7fefeb055640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefe4103b50 0x7fefe4074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.579+0000 7fefeb055640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fefe410dae0 con 0x7fefe4101fa0 2024-04-21T08:36:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.579+0000 7fefeb055640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fefe410d960 con 0x7fefe4103190 2024-04-21T08:36:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.579+0000 7fefeb055640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fefe410dc60 con 0x7fefe4103b50 2024-04-21T08:36:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.579+0000 7fefe8df1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefe4101fa0 0x7fefe4075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.579+0000 7fefe8df1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefe4101fa0 0x7fefe4075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53448/0 (socket says 172.21.15.43:53448) 2024-04-21T08:36:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.579+0000 7fefe8df1640 1 -- 172.21.15.43:0/504787317 learned_addr learned my addr 172.21.15.43:0/504787317 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.579+0000 7fefe3fff640 1 --2- 172.21.15.43:0/504787317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefe4103190 0x7fefe4074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.580+0000 7fefe8df1640 1 -- 172.21.15.43:0/504787317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefe4103b50 msgr2=0x7fefe4074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.580+0000 7fefe8df1640 1 --2- 172.21.15.43:0/504787317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefe4103b50 0x7fefe4074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.580+0000 7fefe8df1640 1 -- 172.21.15.43:0/504787317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefe4103190 msgr2=0x7fefe4074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.580+0000 7fefe8df1640 1 --2- 172.21.15.43:0/504787317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefe4103190 0x7fefe4074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.580+0000 7fefe8df1640 1 -- 172.21.15.43:0/504787317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe4075110 con 0x7fefe4101fa0 2024-04-21T08:36:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.580+0000 7fefe8df1640 1 --2- 172.21.15.43:0/504787317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefe4101fa0 0x7fefe4075d00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fefd800d950 tx=0x7fefd800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.580+0000 7fefe1ffb640 1 -- 172.21.15.43:0/504787317 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd8017070 con 0x7fefe4101fa0 2024-04-21T08:36:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.580+0000 7fefeb055640 1 -- 172.21.15.43:0/504787317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe41a22f0 con 0x7fefe4101fa0 2024-04-21T08:36:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.580+0000 7fefe1ffb640 1 -- 172.21.15.43:0/504787317 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefd8009c30 con 0x7fefe4101fa0 2024-04-21T08:36:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.580+0000 7fefe1ffb640 1 -- 172.21.15.43:0/504787317 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd800f620 con 0x7fefe4101fa0 2024-04-21T08:36:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.580+0000 7fefeb055640 1 -- 172.21.15.43:0/504787317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe41a27f0 con 0x7fefe4101fa0 2024-04-21T08:36:46.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.581+0000 7fefeb055640 1 -- 172.21.15.43:0/504787317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe41093b0 con 0x7fefe4101fa0 2024-04-21T08:36:46.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.583+0000 7fefe1ffb640 1 -- 172.21.15.43:0/504787317 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fefd801a070 con 0x7fefe4101fa0 2024-04-21T08:36:46.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.583+0000 7fefe1ffb640 1 --2- 172.21.15.43:0/504787317 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fefd40a3bb0 0x7fefd40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.584+0000 7fefe3fff640 1 --2- 172.21.15.43:0/504787317 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fefd40a3bb0 0x7fefd40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.584+0000 7fefe1ffb640 1 -- 172.21.15.43:0/504787317 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fefd80d0110 con 0x7fefe4101fa0 2024-04-21T08:36:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.584+0000 7fefe3fff640 1 --2- 172.21.15.43:0/504787317 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fefd40a3bb0 0x7fefd40a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fefe4075a70 tx=0x7fefd0005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:46.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.588+0000 7fefe1ffb640 1 -- 172.21.15.43:0/504787317 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fefd8009de0 con 0x7fefe4101fa0 2024-04-21T08:36:46.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:46.816+0000 7fefeb055640 1 -- 172.21.15.43:0/504787317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fefe402d5c0 con 0x7fefe4101fa0 2024-04-21T08:36:47.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.040+0000 7fefe1ffb640 1 -- 172.21.15.43:0/504787317 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fefd809c8b0 con 0x7fefe4101fa0 2024-04-21T08:36:47.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.042+0000 7fefeb055640 1 -- 172.21.15.43:0/504787317 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fefd40a3bb0 msgr2=0x7fefd40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:47.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.042+0000 7fefeb055640 1 --2- 172.21.15.43:0/504787317 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fefd40a3bb0 0x7fefd40a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fefe4075a70 tx=0x7fefd0005be0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.042+0000 7fefeb055640 1 -- 172.21.15.43:0/504787317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefe4101fa0 msgr2=0x7fefe4075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:47.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.042+0000 7fefeb055640 1 --2- 172.21.15.43:0/504787317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefe4101fa0 0x7fefe4075d00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fefd800d950 tx=0x7fefd800de10 comp rx=0 tx=0).stop 2024-04-21T08:36:47.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.042+0000 7fefeb055640 1 -- 172.21.15.43:0/504787317 shutdown_connections 2024-04-21T08:36:47.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.042+0000 7fefeb055640 1 --2- 172.21.15.43:0/504787317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefe4103b50 0x7fefe4074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.042+0000 7fefeb055640 1 --2- 172.21.15.43:0/504787317 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fefd40a3bb0 0x7fefd40a6070 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.042+0000 7fefeb055640 1 --2- 172.21.15.43:0/504787317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefe4103190 0x7fefe4074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.042+0000 7fefeb055640 1 --2- 172.21.15.43:0/504787317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefe4101fa0 0x7fefe4075d00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.042+0000 7fefeb055640 1 -- 172.21.15.43:0/504787317 >> 172.21.15.43:0/504787317 conn(0x7fefe40fd530 msgr2=0x7fefe4106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:47.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.042+0000 7fefeb055640 1 -- 172.21.15.43:0/504787317 shutdown_connections 2024-04-21T08:36:47.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.042+0000 7fefeb055640 1 -- 172.21.15.43:0/504787317 wait complete. 2024-04-21T08:36:47.054 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-21T08:36:47.055 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-21T08:36:47.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.160+0000 7fe9a225b640 1 -- 172.21.15.43:0/3325559012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe99c103b50 msgr2=0x7fe99c10a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:47.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.160+0000 7fe9a225b640 1 --2- 172.21.15.43:0/3325559012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe99c103b50 0x7fe99c10a3e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe994009ca0 tx=0x7fe99402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.160+0000 7fe9a225b640 1 -- 172.21.15.43:0/3325559012 shutdown_connections 2024-04-21T08:36:47.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.160+0000 7fe9a225b640 1 --2- 172.21.15.43:0/3325559012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe99c103b50 0x7fe99c10a3e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.160+0000 7fe9a225b640 1 --2- 172.21.15.43:0/3325559012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe99c103190 0x7fe99c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.160+0000 7fe9a225b640 1 --2- 172.21.15.43:0/3325559012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe99c101fa0 0x7fe99c1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.160+0000 7fe9a225b640 1 -- 172.21.15.43:0/3325559012 >> 172.21.15.43:0/3325559012 conn(0x7fe99c0fd530 msgr2=0x7fe99c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:47.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.160+0000 7fe9a225b640 1 -- 172.21.15.43:0/3325559012 shutdown_connections 2024-04-21T08:36:47.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.160+0000 7fe9a225b640 1 -- 172.21.15.43:0/3325559012 wait complete. 2024-04-21T08:36:47.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.161+0000 7fe9a225b640 1 Processor -- start 2024-04-21T08:36:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.161+0000 7fe9a225b640 1 -- start start 2024-04-21T08:36:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.161+0000 7fe9a225b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe99c101fa0 0x7fe99c1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.161+0000 7fe9a225b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe99c103190 0x7fe99c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.161+0000 7fe9a225b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe99c103b50 0x7fe99c19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.161+0000 7fe9a225b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c10dae0 con 0x7fe99c101fa0 2024-04-21T08:36:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.161+0000 7fe9a225b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c10d960 con 0x7fe99c103190 2024-04-21T08:36:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.161+0000 7fe9a225b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c10dc60 con 0x7fe99c103b50 2024-04-21T08:36:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.161+0000 7fe99b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe99c101fa0 0x7fe99c1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.161+0000 7fe99b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe99c101fa0 0x7fe99c1981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35472/0 (socket says 172.21.15.43:35472) 2024-04-21T08:36:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.161+0000 7fe99b7fe640 1 -- 172.21.15.43:0/921695315 learned_addr learned my addr 172.21.15.43:0/921695315 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.162+0000 7fe99affd640 1 --2- 172.21.15.43:0/921695315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe99c103190 0x7fe99c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.162+0000 7fe99bfff640 1 --2- 172.21.15.43:0/921695315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe99c103b50 0x7fe99c19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.162+0000 7fe99b7fe640 1 -- 172.21.15.43:0/921695315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe99c103b50 msgr2=0x7fe99c19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.162+0000 7fe99b7fe640 1 --2- 172.21.15.43:0/921695315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe99c103b50 0x7fe99c19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.162+0000 7fe99b7fe640 1 -- 172.21.15.43:0/921695315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe99c103190 msgr2=0x7fe99c198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.162+0000 7fe99b7fe640 1 --2- 172.21.15.43:0/921695315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe99c103190 0x7fe99c198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.162+0000 7fe99b7fe640 1 -- 172.21.15.43:0/921695315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99c19fde0 con 0x7fe99c101fa0 2024-04-21T08:36:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.162+0000 7fe99bfff640 1 --2- 172.21.15.43:0/921695315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe99c103b50 0x7fe99c19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.162+0000 7fe99affd640 1 --2- 172.21.15.43:0/921695315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe99c103190 0x7fe99c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.162+0000 7fe99b7fe640 1 --2- 172.21.15.43:0/921695315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe99c101fa0 0x7fe99c1981c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00d950 tx=0x7fe98c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.162+0000 7fe998ff9640 1 -- 172.21.15.43:0/921695315 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c017070 con 0x7fe99c101fa0 2024-04-21T08:36:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.162+0000 7fe998ff9640 1 -- 172.21.15.43:0/921695315 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe98c009c30 con 0x7fe99c101fa0 2024-04-21T08:36:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.162+0000 7fe9a225b640 1 -- 172.21.15.43:0/921695315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe99c1a00d0 con 0x7fe99c101fa0 2024-04-21T08:36:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.162+0000 7fe998ff9640 1 -- 172.21.15.43:0/921695315 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c00f620 con 0x7fe99c101fa0 2024-04-21T08:36:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.162+0000 7fe9a225b640 1 -- 172.21.15.43:0/921695315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe99c1a05d0 con 0x7fe99c101fa0 2024-04-21T08:36:47.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.163+0000 7fe9a225b640 1 -- 172.21.15.43:0/921695315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe99c109330 con 0x7fe99c101fa0 2024-04-21T08:36:47.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.164+0000 7fe998ff9640 1 -- 172.21.15.43:0/921695315 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe98c01a070 con 0x7fe99c101fa0 2024-04-21T08:36:47.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.165+0000 7fe998ff9640 1 --2- 172.21.15.43:0/921695315 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe9840a3bb0 0x7fe9840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:47.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.165+0000 7fe99affd640 1 --2- 172.21.15.43:0/921695315 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe9840a3bb0 0x7fe9840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:47.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.165+0000 7fe998ff9640 1 -- 172.21.15.43:0/921695315 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe98c0d0130 con 0x7fe99c101fa0 2024-04-21T08:36:47.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.166+0000 7fe99affd640 1 --2- 172.21.15.43:0/921695315 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe9840a3bb0 0x7fe9840a6070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe99c1996d0 tx=0x7fe99000b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:47.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.170+0000 7fe998ff9640 1 -- 172.21.15.43:0/921695315 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe98c009de0 con 0x7fe99c101fa0 2024-04-21T08:36:47.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:47 smithi112 ceph-mon[31085]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T08:36:47.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:47 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3636013284' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:47.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:47 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2766904562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:47.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:47 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/504787317' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-21T08:36:47.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:47 smithi043 ceph-mon[27978]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T08:36:47.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:47 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3636013284' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:47.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:47 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2766904562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:47.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:47 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/504787317' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-21T08:36:47.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:47.399+0000 7fe9a225b640 1 -- 172.21.15.43:0/921695315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fe99c062b80 con 0x7fe99c101fa0 2024-04-21T08:36:47.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:47 smithi175 ceph-mon[31920]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T08:36:47.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:47 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3636013284' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:47.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:47 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2766904562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:47.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:47 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/504787317' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-21T08:36:48.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.045+0000 7fe998ff9640 1 -- 172.21.15.43:0/921695315 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fe98c09c8d0 con 0x7fe99c101fa0 2024-04-21T08:36:48.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.048+0000 7fe9a225b640 1 -- 172.21.15.43:0/921695315 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe9840a3bb0 msgr2=0x7fe9840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:48.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.048+0000 7fe9a225b640 1 --2- 172.21.15.43:0/921695315 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe9840a3bb0 0x7fe9840a6070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe99c1996d0 tx=0x7fe99000b5f0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.048+0000 7fe9a225b640 1 -- 172.21.15.43:0/921695315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe99c101fa0 msgr2=0x7fe99c1981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:48.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.048+0000 7fe9a225b640 1 --2- 172.21.15.43:0/921695315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe99c101fa0 0x7fe99c1981c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00d950 tx=0x7fe98c00de10 comp rx=0 tx=0).stop 2024-04-21T08:36:48.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.048+0000 7fe9a225b640 1 -- 172.21.15.43:0/921695315 shutdown_connections 2024-04-21T08:36:48.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.048+0000 7fe9a225b640 1 --2- 172.21.15.43:0/921695315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe99c103b50 0x7fe99c19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.048+0000 7fe9a225b640 1 --2- 172.21.15.43:0/921695315 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe9840a3bb0 0x7fe9840a6070 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.048+0000 7fe9a225b640 1 --2- 172.21.15.43:0/921695315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe99c103190 0x7fe99c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.048+0000 7fe9a225b640 1 --2- 172.21.15.43:0/921695315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe99c101fa0 0x7fe99c1981c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.048+0000 7fe9a225b640 1 -- 172.21.15.43:0/921695315 >> 172.21.15.43:0/921695315 conn(0x7fe99c0fd530 msgr2=0x7fe99c106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:48.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.049+0000 7fe9a225b640 1 -- 172.21.15.43:0/921695315 shutdown_connections 2024-04-21T08:36:48.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.049+0000 7fe9a225b640 1 -- 172.21.15.43:0/921695315 wait complete. 2024-04-21T08:36:48.064 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-21T08:36:48.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.171+0000 7f148bc93640 1 -- 172.21.15.43:0/2593238092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1484103b60 msgr2=0x7f148410a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:48.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.171+0000 7f148bc93640 1 --2- 172.21.15.43:0/2593238092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1484103b60 0x7f148410a3f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f147c009ca0 tx=0x7f147c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.171+0000 7f148bc93640 1 -- 172.21.15.43:0/2593238092 shutdown_connections 2024-04-21T08:36:48.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.171+0000 7f148bc93640 1 --2- 172.21.15.43:0/2593238092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1484103b60 0x7f148410a3f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.171+0000 7f148bc93640 1 --2- 172.21.15.43:0/2593238092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14841031a0 0x7f1484103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.171+0000 7f148bc93640 1 --2- 172.21.15.43:0/2593238092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1484101fb0 0x7f14841023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.171+0000 7f148bc93640 1 -- 172.21.15.43:0/2593238092 >> 172.21.15.43:0/2593238092 conn(0x7f14840fd520 msgr2=0x7f14840ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:48.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.171+0000 7f148bc93640 1 -- 172.21.15.43:0/2593238092 shutdown_connections 2024-04-21T08:36:48.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.171+0000 7f148bc93640 1 -- 172.21.15.43:0/2593238092 wait complete. 2024-04-21T08:36:48.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.172+0000 7f148bc93640 1 Processor -- start 2024-04-21T08:36:48.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.172+0000 7f148bc93640 1 -- start start 2024-04-21T08:36:48.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.172+0000 7f148bc93640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1484101fb0 0x7f1484198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:48.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.172+0000 7f148bc93640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14841031a0 0x7f14841986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:48.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.172+0000 7f148bc93640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1484103b60 0x7f148419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:48.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.172+0000 7f148bc93640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f148410daf0 con 0x7f1484103b60 2024-04-21T08:36:48.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.172+0000 7f148bc93640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f148410d970 con 0x7f14841031a0 2024-04-21T08:36:48.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.172+0000 7f148bc93640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f148410dc70 con 0x7f1484101fb0 2024-04-21T08:36:48.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.172+0000 7f148a230640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1484103b60 0x7f148419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:48.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.172+0000 7f148a230640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1484103b60 0x7f148419f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35490/0 (socket says 172.21.15.43:35490) 2024-04-21T08:36:48.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.172+0000 7f148a230640 1 -- 172.21.15.43:0/1026583084 learned_addr learned my addr 172.21.15.43:0/1026583084 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:48.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.172+0000 7f1489a2f640 1 --2- 172.21.15.43:0/1026583084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1484101fb0 0x7f1484198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:48.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.172+0000 7f148922e640 1 --2- 172.21.15.43:0/1026583084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14841031a0 0x7f14841986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:48.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.172+0000 7f148a230640 1 -- 172.21.15.43:0/1026583084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1484101fb0 msgr2=0x7f1484198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:48.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.172+0000 7f148a230640 1 --2- 172.21.15.43:0/1026583084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1484101fb0 0x7f1484198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.172+0000 7f148a230640 1 -- 172.21.15.43:0/1026583084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14841031a0 msgr2=0x7f14841986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:48.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.172+0000 7f148a230640 1 --2- 172.21.15.43:0/1026583084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14841031a0 0x7f14841986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.172+0000 7f148a230640 1 -- 172.21.15.43:0/1026583084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148419fd90 con 0x7f1484103b60 2024-04-21T08:36:48.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.172+0000 7f1489a2f640 1 --2- 172.21.15.43:0/1026583084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1484101fb0 0x7f1484198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:48.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.173+0000 7f148a230640 1 --2- 172.21.15.43:0/1026583084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1484103b60 0x7f148419f720 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f147c02c700 tx=0x7f147c034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:48.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.173+0000 7f147affd640 1 -- 172.21.15.43:0/1026583084 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c039040 con 0x7f1484103b60 2024-04-21T08:36:48.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.173+0000 7f147affd640 1 -- 172.21.15.43:0/1026583084 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f147c044070 con 0x7f1484103b60 2024-04-21T08:36:48.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.173+0000 7f148bc93640 1 -- 172.21.15.43:0/1026583084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14841a0020 con 0x7f1484103b60 2024-04-21T08:36:48.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.173+0000 7f147affd640 1 -- 172.21.15.43:0/1026583084 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c0042a0 con 0x7f1484103b60 2024-04-21T08:36:48.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.173+0000 7f148bc93640 1 -- 172.21.15.43:0/1026583084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14841a0510 con 0x7f1484103b60 2024-04-21T08:36:48.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.174+0000 7f148bc93640 1 -- 172.21.15.43:0/1026583084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14841093a0 con 0x7f1484103b60 2024-04-21T08:36:48.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.176+0000 7f147affd640 1 -- 172.21.15.43:0/1026583084 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f147c003970 con 0x7f1484103b60 2024-04-21T08:36:48.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.176+0000 7f147affd640 1 --2- 172.21.15.43:0/1026583084 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f14700a3bb0 0x7f14700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:48.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.177+0000 7f1489a2f640 1 --2- 172.21.15.43:0/1026583084 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f14700a3bb0 0x7f14700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:48.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.177+0000 7f147affd640 1 -- 172.21.15.43:0/1026583084 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f147c0f35e0 con 0x7f1484103b60 2024-04-21T08:36:48.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.177+0000 7f1489a2f640 1 --2- 172.21.15.43:0/1026583084 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f14700a3bb0 0x7f14700a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f14800058c0 tx=0x7f1480005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:48.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.180+0000 7f147affd640 1 -- 172.21.15.43:0/1026583084 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f147c0bfd20 con 0x7f1484103b60 2024-04-21T08:36:48.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:48 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/504787317' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-21T08:36:48.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:48 smithi043 ceph-mon[27978]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:48.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:48 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/921695315' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-21T08:36:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:48.408+0000 7f148bc93640 1 -- 172.21.15.43:0/1026583084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f148402d5c0 con 0x7f1484103b60 2024-04-21T08:36:48.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:48 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/504787317' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-21T08:36:48.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:48 smithi175 ceph-mon[31920]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:48.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:48 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/921695315' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-21T08:36:48.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:48 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/504787317' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-21T08:36:48.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:48 smithi112 ceph-mon[31085]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:48.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:48 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/921695315' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-21T08:36:49.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.052+0000 7f147affd640 1 -- 172.21.15.43:0/1026583084 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f147c0bf620 con 0x7f1484103b60 2024-04-21T08:36:49.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.055+0000 7f148bc93640 1 -- 172.21.15.43:0/1026583084 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f14700a3bb0 msgr2=0x7f14700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:49.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.055+0000 7f148bc93640 1 --2- 172.21.15.43:0/1026583084 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f14700a3bb0 0x7f14700a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f14800058c0 tx=0x7f1480005850 comp rx=0 tx=0).stop 2024-04-21T08:36:49.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.055+0000 7f148bc93640 1 -- 172.21.15.43:0/1026583084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1484103b60 msgr2=0x7f148419f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:49.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.055+0000 7f148bc93640 1 --2- 172.21.15.43:0/1026583084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1484103b60 0x7f148419f720 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f147c02c700 tx=0x7f147c034760 comp rx=0 tx=0).stop 2024-04-21T08:36:49.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.055+0000 7f148bc93640 1 -- 172.21.15.43:0/1026583084 shutdown_connections 2024-04-21T08:36:49.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.055+0000 7f148bc93640 1 --2- 172.21.15.43:0/1026583084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1484103b60 0x7f148419f720 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.055+0000 7f148bc93640 1 --2- 172.21.15.43:0/1026583084 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f14700a3bb0 0x7f14700a6070 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.055+0000 7f148bc93640 1 --2- 172.21.15.43:0/1026583084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14841031a0 0x7f14841986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.055+0000 7f148bc93640 1 --2- 172.21.15.43:0/1026583084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1484101fb0 0x7f1484198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.055+0000 7f148bc93640 1 -- 172.21.15.43:0/1026583084 >> 172.21.15.43:0/1026583084 conn(0x7f14840fd520 msgr2=0x7f1484106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:49.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.055+0000 7f148bc93640 1 -- 172.21.15.43:0/1026583084 shutdown_connections 2024-04-21T08:36:49.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.055+0000 7f148bc93640 1 -- 172.21.15.43:0/1026583084 wait complete. 2024-04-21T08:36:49.073 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-21T08:36:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.185+0000 7f189759e640 1 -- 172.21.15.43:0/1850365128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1898101a30 msgr2=0x7f1898101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.185+0000 7f189759e640 1 --2- 172.21.15.43:0/1850365128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1898101a30 0x7f1898101eb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1890009900 tx=0x7f189002bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.185+0000 7f189759e640 1 -- 172.21.15.43:0/1850365128 shutdown_connections 2024-04-21T08:36:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.185+0000 7f189759e640 1 --2- 172.21.15.43:0/1850365128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1898101a30 0x7f1898101eb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.185+0000 7f189759e640 1 --2- 172.21.15.43:0/1850365128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1898101090 0x7f18981014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.185+0000 7f189759e640 1 --2- 172.21.15.43:0/1850365128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1898100730 0x7f1898100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.185+0000 7f189759e640 1 -- 172.21.15.43:0/1850365128 >> 172.21.15.43:0/1850365128 conn(0x7f18980fc2f0 msgr2=0x7f18980fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.185+0000 7f189759e640 1 -- 172.21.15.43:0/1850365128 shutdown_connections 2024-04-21T08:36:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.185+0000 7f189759e640 1 -- 172.21.15.43:0/1850365128 wait complete. 2024-04-21T08:36:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.186+0000 7f189759e640 1 Processor -- start 2024-04-21T08:36:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.186+0000 7f189759e640 1 -- start start 2024-04-21T08:36:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.186+0000 7f189759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1898100730 0x7f189819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.186+0000 7f189759e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1898101090 0x7f189819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.186+0000 7f189759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1898101a30 0x7f18981a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.186+0000 7f189759e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f189810fbb0 con 0x7f1898101a30 2024-04-21T08:36:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.186+0000 7f189759e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f189810fa30 con 0x7f1898100730 2024-04-21T08:36:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.186+0000 7f189759e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f189810fd30 con 0x7f1898101090 2024-04-21T08:36:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.186+0000 7f1896d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1898101a30 0x7f18981a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.186+0000 7f1896d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1898101a30 0x7f18981a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35496/0 (socket says 172.21.15.43:35496) 2024-04-21T08:36:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.186+0000 7f189659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1898100730 0x7f189819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.186+0000 7f1896d9d640 1 -- 172.21.15.43:0/402142022 learned_addr learned my addr 172.21.15.43:0/402142022 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.186+0000 7f1896d9d640 1 -- 172.21.15.43:0/402142022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1898101090 msgr2=0x7f189819a6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:36:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.186+0000 7f1896d9d640 1 --2- 172.21.15.43:0/402142022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1898101090 0x7f189819a6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.186+0000 7f1896d9d640 1 -- 172.21.15.43:0/402142022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1898100730 msgr2=0x7f189819a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.186+0000 7f1896d9d640 1 --2- 172.21.15.43:0/402142022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1898100730 0x7f189819a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.186+0000 7f1896d9d640 1 -- 172.21.15.43:0/402142022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18981a1dd0 con 0x7f1898101a30 2024-04-21T08:36:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.186+0000 7f189659c640 1 --2- 172.21.15.43:0/402142022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1898100730 0x7f189819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:36:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.187+0000 7f1896d9d640 1 --2- 172.21.15.43:0/402142022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1898101a30 0x7f18981a1760 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1890009230 tx=0x7f18900040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.187+0000 7f18877fe640 1 -- 172.21.15.43:0/402142022 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890043020 con 0x7f1898101a30 2024-04-21T08:36:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.187+0000 7f189759e640 1 -- 172.21.15.43:0/402142022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18981a2060 con 0x7f1898101a30 2024-04-21T08:36:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.187+0000 7f18877fe640 1 -- 172.21.15.43:0/402142022 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18900044c0 con 0x7f1898101a30 2024-04-21T08:36:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.187+0000 7f18877fe640 1 -- 172.21.15.43:0/402142022 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890003bc0 con 0x7f1898101a30 2024-04-21T08:36:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.187+0000 7f189759e640 1 -- 172.21.15.43:0/402142022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18981a25b0 con 0x7f1898101a30 2024-04-21T08:36:49.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.188+0000 7f189759e640 1 -- 172.21.15.43:0/402142022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f189810b610 con 0x7f1898101a30 2024-04-21T08:36:49.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.189+0000 7f18877fe640 1 -- 172.21.15.43:0/402142022 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1890046020 con 0x7f1898101a30 2024-04-21T08:36:49.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.189+0000 7f18877fe640 1 --2- 172.21.15.43:0/402142022 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f18800a3bb0 0x7f18800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.190+0000 7f189659c640 1 --2- 172.21.15.43:0/402142022 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f18800a3bb0 0x7f18800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.190+0000 7f18877fe640 1 -- 172.21.15.43:0/402142022 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f18900f2930 con 0x7f1898101a30 2024-04-21T08:36:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.190+0000 7f189659c640 1 --2- 172.21.15.43:0/402142022 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f18800a3bb0 0x7f18800a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1888009fd0 tx=0x7f1888009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:49.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.194+0000 7f18877fe640 1 -- 172.21.15.43:0/402142022 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f189004d020 con 0x7f1898101a30 2024-04-21T08:36:49.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:49 smithi043 ceph-mon[27978]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T08:36:49.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:49 smithi043 ceph-mon[27978]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-21T08:36:49.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:49 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/921695315' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-21T08:36:49.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:49 smithi043 ceph-mon[27978]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-21T08:36:49.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:49 smithi043 ceph-mon[27978]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:49.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:49 smithi043 ceph-mon[27978]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-21T08:36:49.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:49 smithi043 ceph-mon[27978]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-21T08:36:49.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:49 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1026583084' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-21T08:36:49.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.424+0000 7f189759e640 1 -- 172.21.15.43:0/402142022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f1898062b50 con 0x7f1898101a30 2024-04-21T08:36:49.435 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:49 smithi112 ceph-mon[31085]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T08:36:49.435 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:49 smithi112 ceph-mon[31085]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-21T08:36:49.435 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:49 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/921695315' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-21T08:36:49.435 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:49 smithi112 ceph-mon[31085]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-21T08:36:49.435 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:49 smithi112 ceph-mon[31085]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:49.435 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:49 smithi112 ceph-mon[31085]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-21T08:36:49.435 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:49 smithi112 ceph-mon[31085]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-21T08:36:49.435 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:49 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1026583084' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-21T08:36:49.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:49 smithi175 ceph-mon[31920]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T08:36:49.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:49 smithi175 ceph-mon[31920]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-21T08:36:49.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:49 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/921695315' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-21T08:36:49.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:49 smithi175 ceph-mon[31920]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-21T08:36:49.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:49 smithi175 ceph-mon[31920]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:49.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:49 smithi175 ceph-mon[31920]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-21T08:36:49.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:49 smithi175 ceph-mon[31920]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-21T08:36:49.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:49 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1026583084' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-21T08:36:49.794 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:36:49 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-b[61578]: 2024-04-21T08:36:49.435+0000 7fb798658700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-21T08:36:49.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.874+0000 7f18877fe640 1 -- 172.21.15.43:0/402142022 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7f18900bf0d0 con 0x7f1898101a30 2024-04-21T08:36:49.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.876+0000 7f189759e640 1 -- 172.21.15.43:0/402142022 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f18800a3bb0 msgr2=0x7f18800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:49.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.876+0000 7f189759e640 1 --2- 172.21.15.43:0/402142022 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f18800a3bb0 0x7f18800a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1888009fd0 tx=0x7f1888009680 comp rx=0 tx=0).stop 2024-04-21T08:36:49.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.876+0000 7f189759e640 1 -- 172.21.15.43:0/402142022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1898101a30 msgr2=0x7f18981a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:49.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.876+0000 7f189759e640 1 --2- 172.21.15.43:0/402142022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1898101a30 0x7f18981a1760 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1890009230 tx=0x7f18900040c0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.876+0000 7f189759e640 1 -- 172.21.15.43:0/402142022 shutdown_connections 2024-04-21T08:36:49.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.876+0000 7f189759e640 1 --2- 172.21.15.43:0/402142022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1898101a30 0x7f18981a1760 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.877+0000 7f189759e640 1 --2- 172.21.15.43:0/402142022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1898101090 0x7f189819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.877+0000 7f189759e640 1 --2- 172.21.15.43:0/402142022 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f18800a3bb0 0x7f18800a6070 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.877+0000 7f189759e640 1 --2- 172.21.15.43:0/402142022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1898100730 0x7f189819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.877+0000 7f189759e640 1 -- 172.21.15.43:0/402142022 >> 172.21.15.43:0/402142022 conn(0x7f18980fc2f0 msgr2=0x7f18980fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:49.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.877+0000 7f189759e640 1 -- 172.21.15.43:0/402142022 shutdown_connections 2024-04-21T08:36:49.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:49.877+0000 7f189759e640 1 -- 172.21.15.43:0/402142022 wait complete. 2024-04-21T08:36:49.893 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-21T08:36:49.894 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-21T08:36:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.004+0000 7f125252c640 1 -- 172.21.15.43:0/3533074386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f124c102a00 msgr2=0x7f124c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.004+0000 7f125252c640 1 --2- 172.21.15.43:0/3533074386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f124c102a00 0x7f124c109320 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1244009ca0 tx=0x7f124402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.004+0000 7f125252c640 1 -- 172.21.15.43:0/3533074386 shutdown_connections 2024-04-21T08:36:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.004+0000 7f125252c640 1 --2- 172.21.15.43:0/3533074386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f124c102a00 0x7f124c109320 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.004+0000 7f125252c640 1 --2- 172.21.15.43:0/3533074386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f124c102040 0x7f124c1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.004+0000 7f125252c640 1 --2- 172.21.15.43:0/3533074386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f124c100e60 0x7f124c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.004+0000 7f125252c640 1 -- 172.21.15.43:0/3533074386 >> 172.21.15.43:0/3533074386 conn(0x7f124c0fc440 msgr2=0x7f124c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.004+0000 7f125252c640 1 -- 172.21.15.43:0/3533074386 shutdown_connections 2024-04-21T08:36:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.004+0000 7f125252c640 1 -- 172.21.15.43:0/3533074386 wait complete. 2024-04-21T08:36:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.004+0000 7f125252c640 1 Processor -- start 2024-04-21T08:36:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.004+0000 7f125252c640 1 -- start start 2024-04-21T08:36:50.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.005+0000 7f125252c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f124c100e60 0x7f124c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:50.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.005+0000 7f125252c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f124c102040 0x7f124c19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:50.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.005+0000 7f125252c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f124c102a00 0x7f124c1a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:50.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.005+0000 7f125252c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f124c10db30 con 0x7f124c102a00 2024-04-21T08:36:50.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.005+0000 7f125252c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f124c10d9b0 con 0x7f124c100e60 2024-04-21T08:36:50.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.005+0000 7f125252c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f124c10dcb0 con 0x7f124c102040 2024-04-21T08:36:50.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.005+0000 7f1250ac9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f124c102a00 0x7f124c1a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:50.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.005+0000 7f1250ac9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f124c102a00 0x7f124c1a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35514/0 (socket says 172.21.15.43:35514) 2024-04-21T08:36:50.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.005+0000 7f1250ac9640 1 -- 172.21.15.43:0/2612405215 learned_addr learned my addr 172.21.15.43:0/2612405215 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:50.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.005+0000 7f124bfff640 1 --2- 172.21.15.43:0/2612405215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f124c100e60 0x7f124c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:50.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.005+0000 7f1250ac9640 1 -- 172.21.15.43:0/2612405215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f124c102040 msgr2=0x7f124c19a830 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:36:50.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.005+0000 7f1250ac9640 1 --2- 172.21.15.43:0/2612405215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f124c102040 0x7f124c19a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:50.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.005+0000 7f1250ac9640 1 -- 172.21.15.43:0/2612405215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f124c100e60 msgr2=0x7f124c19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:50.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.005+0000 7f1250ac9640 1 --2- 172.21.15.43:0/2612405215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f124c100e60 0x7f124c19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.005+0000 7f1250ac9640 1 -- 172.21.15.43:0/2612405215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124c1a1f10 con 0x7f124c102a00 2024-04-21T08:36:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.005+0000 7f1250ac9640 1 --2- 172.21.15.43:0/2612405215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f124c102a00 0x7f124c1a18a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f124402c700 tx=0x7f12440093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.005+0000 7f12497fa640 1 -- 172.21.15.43:0/2612405215 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244041020 con 0x7f124c102a00 2024-04-21T08:36:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.005+0000 7f12497fa640 1 -- 172.21.15.43:0/2612405215 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1244034d60 con 0x7f124c102a00 2024-04-21T08:36:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.005+0000 7f125252c640 1 -- 172.21.15.43:0/2612405215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124c1a21a0 con 0x7f124c102a00 2024-04-21T08:36:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.005+0000 7f12497fa640 1 -- 172.21.15.43:0/2612405215 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244039040 con 0x7f124c102a00 2024-04-21T08:36:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.005+0000 7f125252c640 1 -- 172.21.15.43:0/2612405215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f124c1a2690 con 0x7f124c102a00 2024-04-21T08:36:50.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.006+0000 7f125252c640 1 -- 172.21.15.43:0/2612405215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f124c1a28f0 con 0x7f124c102a00 2024-04-21T08:36:50.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.007+0000 7f12497fa640 1 -- 172.21.15.43:0/2612405215 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1244044020 con 0x7f124c102a00 2024-04-21T08:36:50.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.008+0000 7f12497fa640 1 --2- 172.21.15.43:0/2612405215 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f12340a3bb0 0x7f12340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:50.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.008+0000 7f124bfff640 1 --2- 172.21.15.43:0/2612405215 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f12340a3bb0 0x7f12340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:50.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.008+0000 7f12497fa640 1 -- 172.21.15.43:0/2612405215 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f12440f25f0 con 0x7f124c102a00 2024-04-21T08:36:50.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.008+0000 7f124bfff640 1 --2- 172.21.15.43:0/2612405215 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f12340a3bb0 0x7f12340a6070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f124c101bb0 tx=0x7f123c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:50.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.013+0000 7f12497fa640 1 -- 172.21.15.43:0/2612405215 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f124404b020 con 0x7f124c102a00 2024-04-21T08:36:50.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.228+0000 7f125252c640 1 -- 172.21.15.43:0/2612405215 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f124c10b400 con 0x7f12340a3bb0 2024-04-21T08:36:50.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:50 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1026583084' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-21T08:36:50.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:50 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] up:active 2024-04-21T08:36:50.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:50 smithi043 ceph-mon[27978]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-21T08:36:50.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:50 smithi043 ceph-mon[27978]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-21T08:36:50.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:50 smithi043 ceph-mon[27978]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-21T08:36:50.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:50 smithi043 ceph-mon[27978]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-21T08:36:50.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:50 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/402142022' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-21T08:36:50.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:50 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/402142022' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-21T08:36:50.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:50 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] up:active 2024-04-21T08:36:50.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:50 smithi043 ceph-mon[27978]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-21T08:36:50.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:50 smithi043 ceph-mon[27978]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T08:36:50.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:50 smithi043 ceph-mon[27978]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-04-21T08:36:50.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:50 smithi043 ceph-mon[27978]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-21T08:36:50.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:50 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1026583084' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-21T08:36:50.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:50 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] up:active 2024-04-21T08:36:50.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:50 smithi175 ceph-mon[31920]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-21T08:36:50.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:50 smithi175 ceph-mon[31920]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-21T08:36:50.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:50 smithi175 ceph-mon[31920]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-21T08:36:50.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:50 smithi175 ceph-mon[31920]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-21T08:36:50.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:50 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/402142022' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-21T08:36:50.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:50 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/402142022' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-21T08:36:50.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:50 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] up:active 2024-04-21T08:36:50.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:50 smithi175 ceph-mon[31920]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-21T08:36:50.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:50 smithi175 ceph-mon[31920]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T08:36:50.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:50 smithi175 ceph-mon[31920]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-04-21T08:36:50.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:50 smithi175 ceph-mon[31920]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-21T08:36:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:50 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1026583084' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-21T08:36:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:50 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] up:active 2024-04-21T08:36:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:50 smithi112 ceph-mon[31085]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-21T08:36:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:50 smithi112 ceph-mon[31085]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-21T08:36:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:50 smithi112 ceph-mon[31085]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-21T08:36:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:50 smithi112 ceph-mon[31085]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-21T08:36:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:50 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/402142022' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-21T08:36:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:50 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/402142022' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-21T08:36:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:50 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] up:active 2024-04-21T08:36:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:50 smithi112 ceph-mon[31085]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-21T08:36:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:50 smithi112 ceph-mon[31085]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T08:36:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:50 smithi112 ceph-mon[31085]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-04-21T08:36:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:50 smithi112 ceph-mon[31085]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-21T08:36:50.561 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:36:50 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:36:50.233+0000 7f06c30bf700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:36:50.561 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:36:50 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:36:50.233+0000 7f06c30bf700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:36:50.561 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:36:50 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:36:50.233+0000 7f06c30bf700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:36:50.561 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:36:50 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:36:50.233+0000 7f06c30bf700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:36:50.561 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:36:50 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:36:50.233+0000 7f06c30bf700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:36:50.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.919+0000 7f12497fa640 1 -- 172.21.15.43:0/2612405215 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f124c10b400 con 0x7f12340a3bb0 2024-04-21T08:36:50.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.922+0000 7f125252c640 1 -- 172.21.15.43:0/2612405215 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f12340a3bb0 msgr2=0x7f12340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:50.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.922+0000 7f125252c640 1 --2- 172.21.15.43:0/2612405215 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f12340a3bb0 0x7f12340a6070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f124c101bb0 tx=0x7f123c005f90 comp rx=0 tx=0).stop 2024-04-21T08:36:50.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.922+0000 7f125252c640 1 -- 172.21.15.43:0/2612405215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f124c102a00 msgr2=0x7f124c1a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:50.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.922+0000 7f125252c640 1 --2- 172.21.15.43:0/2612405215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f124c102a00 0x7f124c1a18a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f124402c700 tx=0x7f12440093f0 comp rx=0 tx=0).stop 2024-04-21T08:36:50.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.922+0000 7f125252c640 1 -- 172.21.15.43:0/2612405215 shutdown_connections 2024-04-21T08:36:50.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.922+0000 7f125252c640 1 --2- 172.21.15.43:0/2612405215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f124c102a00 0x7f124c1a18a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:50.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.922+0000 7f125252c640 1 --2- 172.21.15.43:0/2612405215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f124c102040 0x7f124c19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:50.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.922+0000 7f125252c640 1 --2- 172.21.15.43:0/2612405215 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f12340a3bb0 0x7f12340a6070 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:50.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.922+0000 7f125252c640 1 --2- 172.21.15.43:0/2612405215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f124c100e60 0x7f124c19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:50.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.922+0000 7f125252c640 1 -- 172.21.15.43:0/2612405215 >> 172.21.15.43:0/2612405215 conn(0x7f124c0fc440 msgr2=0x7f124c109ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:50.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.922+0000 7f125252c640 1 -- 172.21.15.43:0/2612405215 shutdown_connections 2024-04-21T08:36:50.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:50.922+0000 7f125252c640 1 -- 172.21.15.43:0/2612405215 wait complete. 2024-04-21T08:36:50.936 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-21T08:36:51.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.048+0000 7feb6505c640 1 -- 172.21.15.43:0/2756563643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb60101b80 msgr2=0x7feb60102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:51.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.048+0000 7feb6505c640 1 --2- 172.21.15.43:0/2756563643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb60101b80 0x7feb60102000 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7feb58009ca0 tx=0x7feb5802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.048+0000 7feb6505c640 1 -- 172.21.15.43:0/2756563643 shutdown_connections 2024-04-21T08:36:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.048+0000 7feb6505c640 1 --2- 172.21.15.43:0/2756563643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb60101b80 0x7feb60102000 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.048+0000 7feb6505c640 1 --2- 172.21.15.43:0/2756563643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb601011e0 0x7feb60101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.048+0000 7feb6505c640 1 --2- 172.21.15.43:0/2756563643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb60100880 0x7feb60100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.048+0000 7feb6505c640 1 -- 172.21.15.43:0/2756563643 >> 172.21.15.43:0/2756563643 conn(0x7feb600fc440 msgr2=0x7feb600fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.048+0000 7feb6505c640 1 -- 172.21.15.43:0/2756563643 shutdown_connections 2024-04-21T08:36:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.048+0000 7feb6505c640 1 -- 172.21.15.43:0/2756563643 wait complete. 2024-04-21T08:36:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.049+0000 7feb6505c640 1 Processor -- start 2024-04-21T08:36:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.049+0000 7feb6505c640 1 -- start start 2024-04-21T08:36:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.049+0000 7feb6505c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb60100880 0x7feb6019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.049+0000 7feb6505c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb601011e0 0x7feb6019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.049+0000 7feb6505c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb60101b80 0x7feb601a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.049+0000 7feb6505c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feb6010fd00 con 0x7feb60100880 2024-04-21T08:36:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.049+0000 7feb6505c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb6010fb80 con 0x7feb60101b80 2024-04-21T08:36:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.049+0000 7feb6505c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feb6010fe80 con 0x7feb601011e0 2024-04-21T08:36:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.049+0000 7feb5ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb60100880 0x7feb6019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.049+0000 7feb5ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb60100880 0x7feb6019a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35556/0 (socket says 172.21.15.43:35556) 2024-04-21T08:36:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.049+0000 7feb5ed9d640 1 -- 172.21.15.43:0/89453528 learned_addr learned my addr 172.21.15.43:0/89453528 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.049+0000 7feb5f59e640 1 --2- 172.21.15.43:0/89453528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb60101b80 0x7feb601a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.049+0000 7feb5e59c640 1 --2- 172.21.15.43:0/89453528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb601011e0 0x7feb6019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.050+0000 7feb5ed9d640 1 -- 172.21.15.43:0/89453528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb601011e0 msgr2=0x7feb6019a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.050+0000 7feb5ed9d640 1 --2- 172.21.15.43:0/89453528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb601011e0 0x7feb6019a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.050+0000 7feb5ed9d640 1 -- 172.21.15.43:0/89453528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb60101b80 msgr2=0x7feb601a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.050+0000 7feb5ed9d640 1 --2- 172.21.15.43:0/89453528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb60101b80 0x7feb601a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.050+0000 7feb5ed9d640 1 -- 172.21.15.43:0/89453528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb601a1f40 con 0x7feb60100880 2024-04-21T08:36:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.050+0000 7feb5ed9d640 1 --2- 172.21.15.43:0/89453528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb60100880 0x7feb6019a320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7feb5000c890 tx=0x7feb5000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:51.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.050+0000 7feb47fff640 1 -- 172.21.15.43:0/89453528 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50004220 con 0x7feb60100880 2024-04-21T08:36:51.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.050+0000 7feb6505c640 1 -- 172.21.15.43:0/89453528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb601a2230 con 0x7feb60100880 2024-04-21T08:36:51.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.050+0000 7feb47fff640 1 -- 172.21.15.43:0/89453528 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb500043c0 con 0x7feb60100880 2024-04-21T08:36:51.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.050+0000 7feb47fff640 1 -- 172.21.15.43:0/89453528 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50004a00 con 0x7feb60100880 2024-04-21T08:36:51.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.050+0000 7feb6505c640 1 -- 172.21.15.43:0/89453528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb601a27d0 con 0x7feb60100880 2024-04-21T08:36:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.051+0000 7feb6505c640 1 -- 172.21.15.43:0/89453528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6010b5d0 con 0x7feb60100880 2024-04-21T08:36:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.052+0000 7feb47fff640 1 -- 172.21.15.43:0/89453528 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7feb500079b0 con 0x7feb60100880 2024-04-21T08:36:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.053+0000 7feb47fff640 1 --2- 172.21.15.43:0/89453528 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7feb480a3c30 0x7feb480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.053+0000 7feb5e59c640 1 --2- 172.21.15.43:0/89453528 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7feb480a3c30 0x7feb480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.053+0000 7feb47fff640 1 -- 172.21.15.43:0/89453528 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7feb500d0360 con 0x7feb60100880 2024-04-21T08:36:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.053+0000 7feb5e59c640 1 --2- 172.21.15.43:0/89453528 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7feb480a3c30 0x7feb480a60f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feb540095f0 tx=0x7feb54005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:51.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.057+0000 7feb47fff640 1 -- 172.21.15.43:0/89453528 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb50014030 con 0x7feb60100880 2024-04-21T08:36:51.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.272+0000 7feb6505c640 1 -- 172.21.15.43:0/89453528 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7feb60062390 con 0x7feb480a3c30 2024-04-21T08:36:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.297+0000 7feb47fff640 1 -- 172.21.15.43:0/89453528 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7feb60062390 con 0x7feb480a3c30 2024-04-21T08:36:51.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.299+0000 7feb6505c640 1 -- 172.21.15.43:0/89453528 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7feb480a3c30 msgr2=0x7feb480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:51.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.299+0000 7feb6505c640 1 --2- 172.21.15.43:0/89453528 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7feb480a3c30 0x7feb480a60f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feb540095f0 tx=0x7feb54005f90 comp rx=0 tx=0).stop 2024-04-21T08:36:51.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.299+0000 7feb6505c640 1 -- 172.21.15.43:0/89453528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb60100880 msgr2=0x7feb6019a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:51.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.299+0000 7feb6505c640 1 --2- 172.21.15.43:0/89453528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb60100880 0x7feb6019a320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7feb5000c890 tx=0x7feb5000cd50 comp rx=0 tx=0).stop 2024-04-21T08:36:51.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.299+0000 7feb6505c640 1 -- 172.21.15.43:0/89453528 shutdown_connections 2024-04-21T08:36:51.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.299+0000 7feb6505c640 1 --2- 172.21.15.43:0/89453528 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7feb480a3c30 0x7feb480a60f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.300+0000 7feb6505c640 1 --2- 172.21.15.43:0/89453528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb60101b80 0x7feb601a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.300+0000 7feb6505c640 1 --2- 172.21.15.43:0/89453528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb601011e0 0x7feb6019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.300+0000 7feb6505c640 1 --2- 172.21.15.43:0/89453528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb60100880 0x7feb6019a320 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.300+0000 7feb6505c640 1 -- 172.21.15.43:0/89453528 >> 172.21.15.43:0/89453528 conn(0x7feb600fc440 msgr2=0x7feb600fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:51.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.300+0000 7feb6505c640 1 -- 172.21.15.43:0/89453528 shutdown_connections 2024-04-21T08:36:51.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.300+0000 7feb6505c640 1 -- 172.21.15.43:0/89453528 wait complete. 2024-04-21T08:36:51.310 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:51 smithi043 ceph-mon[27978]: pgmap v308: 129 pgs: 129 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-04-21T08:36:51.310 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:51 smithi043 ceph-mon[27978]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:51.310 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:51 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] up:active 2024-04-21T08:36:51.310 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:51 smithi043 ceph-mon[27978]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-21T08:36:51.310 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:51 smithi043 ceph-mon[27978]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-21T08:36:51.310 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:51 smithi043 ceph-mon[27978]: Cluster is now healthy 2024-04-21T08:36:51.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:51 smithi043 ceph-mon[27978]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-04-21T08:36:51.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:51 smithi043 ceph-mon[27978]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-21T08:36:51.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:51 smithi043 ceph-mon[27978]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-04-21T08:36:51.312 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-21T08:36:51.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.419+0000 7fc546bb6640 1 -- 172.21.15.43:0/244810125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc540102040 msgr2=0x7fc5401024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:51.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.419+0000 7fc546bb6640 1 --2- 172.21.15.43:0/244810125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc540102040 0x7fc5401024c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009900 tx=0x7fc52c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:51.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.419+0000 7fc546bb6640 1 -- 172.21.15.43:0/244810125 shutdown_connections 2024-04-21T08:36:51.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.419+0000 7fc546bb6640 1 --2- 172.21.15.43:0/244810125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc540102a00 0x7fc540109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.419+0000 7fc546bb6640 1 --2- 172.21.15.43:0/244810125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc540102040 0x7fc5401024c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.419+0000 7fc546bb6640 1 --2- 172.21.15.43:0/244810125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc540100e60 0x7fc540101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.419+0000 7fc546bb6640 1 -- 172.21.15.43:0/244810125 >> 172.21.15.43:0/244810125 conn(0x7fc5400fc440 msgr2=0x7fc5400fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:51.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.419+0000 7fc546bb6640 1 -- 172.21.15.43:0/244810125 shutdown_connections 2024-04-21T08:36:51.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.419+0000 7fc546bb6640 1 -- 172.21.15.43:0/244810125 wait complete. 2024-04-21T08:36:51.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.420+0000 7fc546bb6640 1 Processor -- start 2024-04-21T08:36:51.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.420+0000 7fc546bb6640 1 -- start start 2024-04-21T08:36:51.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.420+0000 7fc546bb6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc540100e60 0x7fc54006b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:51.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.420+0000 7fc546bb6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc540102040 0x7fc54019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:51.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.420+0000 7fc546bb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc540102a00 0x7fc54019f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:51.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.420+0000 7fc546bb6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc54010ca70 con 0x7fc540100e60 2024-04-21T08:36:51.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.420+0000 7fc546bb6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc54010c8f0 con 0x7fc540102a00 2024-04-21T08:36:51.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.420+0000 7fc546bb6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc54010cbf0 con 0x7fc540102040 2024-04-21T08:36:51.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.420+0000 7fc544952640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc540100e60 0x7fc54006b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:51.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.420+0000 7fc544952640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc540100e60 0x7fc54006b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35566/0 (socket says 172.21.15.43:35566) 2024-04-21T08:36:51.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.420+0000 7fc544952640 1 -- 172.21.15.43:0/1715317279 learned_addr learned my addr 172.21.15.43:0/1715317279 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:51.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.420+0000 7fc545153640 1 --2- 172.21.15.43:0/1715317279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc540102a00 0x7fc54019f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:51.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.420+0000 7fc544952640 1 -- 172.21.15.43:0/1715317279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc540102040 msgr2=0x7fc54019cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:51.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.420+0000 7fc53ffff640 1 --2- 172.21.15.43:0/1715317279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc540102040 0x7fc54019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:51.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.420+0000 7fc544952640 1 --2- 172.21.15.43:0/1715317279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc540102040 0x7fc54019cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.420+0000 7fc544952640 1 -- 172.21.15.43:0/1715317279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc540102a00 msgr2=0x7fc54019f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.420+0000 7fc544952640 1 --2- 172.21.15.43:0/1715317279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc540102a00 0x7fc54019f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.420+0000 7fc544952640 1 -- 172.21.15.43:0/1715317279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54019fe70 con 0x7fc540100e60 2024-04-21T08:36:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.420+0000 7fc545153640 1 --2- 172.21.15.43:0/1715317279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc540102a00 0x7fc54019f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:36:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.420+0000 7fc544952640 1 --2- 172.21.15.43:0/1715317279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc540100e60 0x7fc54006b140 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc53400b450 tx=0x7fc53400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.421+0000 7fc53dffb640 1 -- 172.21.15.43:0/1715317279 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534015020 con 0x7fc540100e60 2024-04-21T08:36:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.421+0000 7fc53dffb640 1 -- 172.21.15.43:0/1715317279 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc534011d40 con 0x7fc540100e60 2024-04-21T08:36:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.421+0000 7fc546bb6640 1 -- 172.21.15.43:0/1715317279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5401a0160 con 0x7fc540100e60 2024-04-21T08:36:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.421+0000 7fc53dffb640 1 -- 172.21.15.43:0/1715317279 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53400f340 con 0x7fc540100e60 2024-04-21T08:36:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.421+0000 7fc546bb6640 1 -- 172.21.15.43:0/1715317279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5401a0780 con 0x7fc540100e60 2024-04-21T08:36:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.422+0000 7fc546bb6640 1 -- 172.21.15.43:0/1715317279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc540108250 con 0x7fc540100e60 2024-04-21T08:36:51.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.423+0000 7fc53dffb640 1 -- 172.21.15.43:0/1715317279 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc534018020 con 0x7fc540100e60 2024-04-21T08:36:51.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.424+0000 7fc53dffb640 1 --2- 172.21.15.43:0/1715317279 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc5300a3c30 0x7fc5300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:51.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.424+0000 7fc53dffb640 1 -- 172.21.15.43:0/1715317279 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc5340d00c0 con 0x7fc540100e60 2024-04-21T08:36:51.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.424+0000 7fc53ffff640 1 --2- 172.21.15.43:0/1715317279 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc5300a3c30 0x7fc5300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:51.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.424+0000 7fc53ffff640 1 --2- 172.21.15.43:0/1715317279 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc5300a3c30 0x7fc5300a60f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc52c036040 tx=0x7fc52c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:51.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.428+0000 7fc53dffb640 1 -- 172.21.15.43:0/1715317279 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc53401f020 con 0x7fc540100e60 2024-04-21T08:36:51.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:51 smithi175 ceph-mon[31920]: pgmap v308: 129 pgs: 129 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-04-21T08:36:51.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:51 smithi175 ceph-mon[31920]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:51.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:51 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] up:active 2024-04-21T08:36:51.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:51 smithi175 ceph-mon[31920]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-21T08:36:51.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:51 smithi175 ceph-mon[31920]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-21T08:36:51.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:51 smithi175 ceph-mon[31920]: Cluster is now healthy 2024-04-21T08:36:51.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:51 smithi175 ceph-mon[31920]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-04-21T08:36:51.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:51 smithi175 ceph-mon[31920]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-21T08:36:51.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:51 smithi175 ceph-mon[31920]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-04-21T08:36:51.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:51 smithi112 ceph-mon[31085]: pgmap v308: 129 pgs: 129 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-04-21T08:36:51.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:51 smithi112 ceph-mon[31085]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:51.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:51 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] up:active 2024-04-21T08:36:51.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:51 smithi112 ceph-mon[31085]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-21T08:36:51.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:51 smithi112 ceph-mon[31085]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-21T08:36:51.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:51 smithi112 ceph-mon[31085]: Cluster is now healthy 2024-04-21T08:36:51.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:51 smithi112 ceph-mon[31085]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-04-21T08:36:51.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:51 smithi112 ceph-mon[31085]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-21T08:36:51.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:51 smithi112 ceph-mon[31085]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-04-21T08:36:51.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.649+0000 7fc546bb6640 1 -- 172.21.15.43:0/1715317279 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fc540062390 con 0x7fc5300a3c30 2024-04-21T08:36:51.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.654+0000 7fc53dffb640 1 -- 172.21.15.43:0/1715317279 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fc540062390 con 0x7fc5300a3c30 2024-04-21T08:36:51.654 INFO:teuthology.orchestra.run.smithi043.stdout:/volumes/qa/sv_0/df6f3f29-e90a-4b28-9bb4-25abc88344ab 2024-04-21T08:36:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.656+0000 7fc546bb6640 1 -- 172.21.15.43:0/1715317279 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc5300a3c30 msgr2=0x7fc5300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.656+0000 7fc546bb6640 1 --2- 172.21.15.43:0/1715317279 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc5300a3c30 0x7fc5300a60f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc52c036040 tx=0x7fc52c000bc0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.656+0000 7fc546bb6640 1 -- 172.21.15.43:0/1715317279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc540100e60 msgr2=0x7fc54006b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.656+0000 7fc546bb6640 1 --2- 172.21.15.43:0/1715317279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc540100e60 0x7fc54006b140 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc53400b450 tx=0x7fc53400b910 comp rx=0 tx=0).stop 2024-04-21T08:36:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.656+0000 7fc546bb6640 1 -- 172.21.15.43:0/1715317279 shutdown_connections 2024-04-21T08:36:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.656+0000 7fc546bb6640 1 --2- 172.21.15.43:0/1715317279 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc5300a3c30 0x7fc5300a60f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.656+0000 7fc546bb6640 1 --2- 172.21.15.43:0/1715317279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc540102a00 0x7fc54019f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.656+0000 7fc546bb6640 1 --2- 172.21.15.43:0/1715317279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc540102040 0x7fc54019cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.656+0000 7fc546bb6640 1 --2- 172.21.15.43:0/1715317279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc540100e60 0x7fc54006b140 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.656+0000 7fc546bb6640 1 -- 172.21.15.43:0/1715317279 >> 172.21.15.43:0/1715317279 conn(0x7fc5400fc440 msgr2=0x7fc540105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.656+0000 7fc546bb6640 1 -- 172.21.15.43:0/1715317279 shutdown_connections 2024-04-21T08:36:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.656+0000 7fc546bb6640 1 -- 172.21.15.43:0/1715317279 wait complete. 2024-04-21T08:36:51.668 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-21T08:36:51.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.774+0000 7f1452767640 1 -- 172.21.15.43:0/2111507370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f144c1029a0 msgr2=0x7f144c1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:51.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.774+0000 7f1452767640 1 --2- 172.21.15.43:0/2111507370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f144c1029a0 0x7f144c1092c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1444009ca0 tx=0x7f144402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.775+0000 7f1452767640 1 -- 172.21.15.43:0/2111507370 shutdown_connections 2024-04-21T08:36:51.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.775+0000 7f1452767640 1 --2- 172.21.15.43:0/2111507370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f144c1029a0 0x7f144c1092c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.775+0000 7f1452767640 1 --2- 172.21.15.43:0/2111507370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f144c101fe0 0x7f144c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.775+0000 7f1452767640 1 --2- 172.21.15.43:0/2111507370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f144c100e90 0x7f144c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.775+0000 7f1452767640 1 -- 172.21.15.43:0/2111507370 >> 172.21.15.43:0/2111507370 conn(0x7f144c0fc400 msgr2=0x7f144c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:51.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.775+0000 7f1452767640 1 -- 172.21.15.43:0/2111507370 shutdown_connections 2024-04-21T08:36:51.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.775+0000 7f1452767640 1 -- 172.21.15.43:0/2111507370 wait complete. 2024-04-21T08:36:51.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.775+0000 7f1452767640 1 Processor -- start 2024-04-21T08:36:51.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.775+0000 7f1452767640 1 -- start start 2024-04-21T08:36:51.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.775+0000 7f1452767640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f144c100e90 0x7f144c069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:51.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.775+0000 7f1452767640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f144c101fe0 0x7f144c19cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:51.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.775+0000 7f1452767640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f144c1029a0 0x7f144c19f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:51.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.775+0000 7f1452767640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f144c10c9e0 con 0x7f144c1029a0 2024-04-21T08:36:51.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.775+0000 7f1452767640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f144c10c860 con 0x7f144c100e90 2024-04-21T08:36:51.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.775+0000 7f1452767640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f144c10cb60 con 0x7f144c101fe0 2024-04-21T08:36:51.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.776+0000 7f1450d04640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f144c1029a0 0x7f144c19f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:51.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.776+0000 7f1450d04640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f144c1029a0 0x7f144c19f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35576/0 (socket says 172.21.15.43:35576) 2024-04-21T08:36:51.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.776+0000 7f144bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f144c100e90 0x7f144c069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:51.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.776+0000 7f1450d04640 1 -- 172.21.15.43:0/2297773176 learned_addr learned my addr 172.21.15.43:0/2297773176 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:51.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.776+0000 7f1450d04640 1 -- 172.21.15.43:0/2297773176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f144c101fe0 msgr2=0x7f144c19cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:51.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.776+0000 7f144b7fe640 1 --2- 172.21.15.43:0/2297773176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f144c101fe0 0x7f144c19cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:51.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.776+0000 7f1450d04640 1 --2- 172.21.15.43:0/2297773176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f144c101fe0 0x7f144c19cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.776+0000 7f1450d04640 1 -- 172.21.15.43:0/2297773176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f144c100e90 msgr2=0x7f144c069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:51.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.776+0000 7f1450d04640 1 --2- 172.21.15.43:0/2297773176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f144c100e90 0x7f144c069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.776+0000 7f1450d04640 1 -- 172.21.15.43:0/2297773176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144c19fe60 con 0x7f144c1029a0 2024-04-21T08:36:51.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.776+0000 7f144bfff640 1 --2- 172.21.15.43:0/2297773176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f144c100e90 0x7f144c069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:36:51.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.776+0000 7f1450d04640 1 --2- 172.21.15.43:0/2297773176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f144c1029a0 0x7f144c19f550 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1444032520 tx=0x7f14440347b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:51.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.776+0000 7f14497fa640 1 -- 172.21.15.43:0/2297773176 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444039040 con 0x7f144c1029a0 2024-04-21T08:36:51.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.776+0000 7f14497fa640 1 -- 172.21.15.43:0/2297773176 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1444044070 con 0x7f144c1029a0 2024-04-21T08:36:51.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.776+0000 7f1452767640 1 -- 172.21.15.43:0/2297773176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f144c1a0090 con 0x7f144c1029a0 2024-04-21T08:36:51.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.776+0000 7f14497fa640 1 -- 172.21.15.43:0/2297773176 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14440042a0 con 0x7f144c1029a0 2024-04-21T08:36:51.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.776+0000 7f1452767640 1 -- 172.21.15.43:0/2297773176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f144c1a05d0 con 0x7f144c1029a0 2024-04-21T08:36:51.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.777+0000 7f1452767640 1 -- 172.21.15.43:0/2297773176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f144c108210 con 0x7f144c1029a0 2024-04-21T08:36:51.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.778+0000 7f14497fa640 1 -- 172.21.15.43:0/2297773176 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f14440325c0 con 0x7f144c1029a0 2024-04-21T08:36:51.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.779+0000 7f14497fa640 1 --2- 172.21.15.43:0/2297773176 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f14340a3c30 0x7f14340a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:51.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.779+0000 7f14497fa640 1 -- 172.21.15.43:0/2297773176 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f14440f28a0 con 0x7f144c1029a0 2024-04-21T08:36:51.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.779+0000 7f144bfff640 1 --2- 172.21.15.43:0/2297773176 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f14340a3c30 0x7f14340a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:51.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.779+0000 7f144bfff640 1 --2- 172.21.15.43:0/2297773176 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f14340a3c30 0x7f14340a60f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f143c009fc0 tx=0x7f143c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:51.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:51.783+0000 7f14497fa640 1 -- 172.21.15.43:0/2297773176 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f14440bf040 con 0x7f144c1029a0 2024-04-21T08:36:52.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.001+0000 7f1452767640 1 -- 172.21.15.43:0/2297773176 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f144c062420 con 0x7f14340a3c30 2024-04-21T08:36:52.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.026+0000 7f14497fa640 1 -- 172.21.15.43:0/2297773176 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f144c062420 con 0x7f14340a3c30 2024-04-21T08:36:52.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.028+0000 7f1452767640 1 -- 172.21.15.43:0/2297773176 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f14340a3c30 msgr2=0x7f14340a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:52.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.028+0000 7f1452767640 1 --2- 172.21.15.43:0/2297773176 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f14340a3c30 0x7f14340a60f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f143c009fc0 tx=0x7f143c005f90 comp rx=0 tx=0).stop 2024-04-21T08:36:52.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.028+0000 7f1452767640 1 -- 172.21.15.43:0/2297773176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f144c1029a0 msgr2=0x7f144c19f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:52.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.028+0000 7f1452767640 1 --2- 172.21.15.43:0/2297773176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f144c1029a0 0x7f144c19f550 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1444032520 tx=0x7f14440347b0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.028+0000 7f1452767640 1 -- 172.21.15.43:0/2297773176 shutdown_connections 2024-04-21T08:36:52.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.028+0000 7f1452767640 1 --2- 172.21.15.43:0/2297773176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f144c1029a0 0x7f144c19f550 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.028+0000 7f1452767640 1 --2- 172.21.15.43:0/2297773176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f144c101fe0 0x7f144c19cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.028+0000 7f1452767640 1 --2- 172.21.15.43:0/2297773176 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f14340a3c30 0x7f14340a60f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.028+0000 7f1452767640 1 --2- 172.21.15.43:0/2297773176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f144c100e90 0x7f144c069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.028+0000 7f1452767640 1 -- 172.21.15.43:0/2297773176 >> 172.21.15.43:0/2297773176 conn(0x7f144c0fc400 msgr2=0x7f144c1053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:52.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.028+0000 7f1452767640 1 -- 172.21.15.43:0/2297773176 shutdown_connections 2024-04-21T08:36:52.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.028+0000 7f1452767640 1 -- 172.21.15.43:0/2297773176 wait complete. 2024-04-21T08:36:52.039 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-21T08:36:52.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.148+0000 7fc9f2e3f640 1 -- 172.21.15.43:0/394408244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9ec102980 msgr2=0x7fc9ec1092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:52.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.148+0000 7fc9f2e3f640 1 --2- 172.21.15.43:0/394408244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9ec102980 0x7fc9ec1092a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009ca0 tx=0x7fc9e402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.148+0000 7fc9f2e3f640 1 -- 172.21.15.43:0/394408244 shutdown_connections 2024-04-21T08:36:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.148+0000 7fc9f2e3f640 1 --2- 172.21.15.43:0/394408244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9ec102980 0x7fc9ec1092a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.148+0000 7fc9f2e3f640 1 --2- 172.21.15.43:0/394408244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9ec101fc0 0x7fc9ec102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.148+0000 7fc9f2e3f640 1 --2- 172.21.15.43:0/394408244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9ec100e70 0x7fc9ec101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.148+0000 7fc9f2e3f640 1 -- 172.21.15.43:0/394408244 >> 172.21.15.43:0/394408244 conn(0x7fc9ec0fc3e0 msgr2=0x7fc9ec0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.148+0000 7fc9f2e3f640 1 -- 172.21.15.43:0/394408244 shutdown_connections 2024-04-21T08:36:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.148+0000 7fc9f2e3f640 1 -- 172.21.15.43:0/394408244 wait complete. 2024-04-21T08:36:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.148+0000 7fc9f2e3f640 1 Processor -- start 2024-04-21T08:36:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.149+0000 7fc9f2e3f640 1 -- start start 2024-04-21T08:36:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.149+0000 7fc9f2e3f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9ec100e70 0x7fc9ec1981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.149+0000 7fc9f2e3f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9ec101fc0 0x7fc9ec198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.149+0000 7fc9f2e3f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9ec102980 0x7fc9ec19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.149+0000 7fc9f2e3f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec10c9c0 con 0x7fc9ec100e70 2024-04-21T08:36:52.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.149+0000 7fc9f2e3f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec10c840 con 0x7fc9ec102980 2024-04-21T08:36:52.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.149+0000 7fc9f2e3f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec10cb40 con 0x7fc9ec101fc0 2024-04-21T08:36:52.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.149+0000 7fc9f0bdb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9ec100e70 0x7fc9ec1981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:52.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.149+0000 7fc9f0bdb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9ec100e70 0x7fc9ec1981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35594/0 (socket says 172.21.15.43:35594) 2024-04-21T08:36:52.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.149+0000 7fc9f0bdb640 1 -- 172.21.15.43:0/3807030323 learned_addr learned my addr 172.21.15.43:0/3807030323 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:52.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.149+0000 7fc9f13dc640 1 --2- 172.21.15.43:0/3807030323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9ec102980 0x7fc9ec19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:52.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.149+0000 7fc9ebfff640 1 --2- 172.21.15.43:0/3807030323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9ec101fc0 0x7fc9ec198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:52.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.149+0000 7fc9f0bdb640 1 -- 172.21.15.43:0/3807030323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9ec101fc0 msgr2=0x7fc9ec198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:52.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.149+0000 7fc9f0bdb640 1 --2- 172.21.15.43:0/3807030323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9ec101fc0 0x7fc9ec198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.149+0000 7fc9f0bdb640 1 -- 172.21.15.43:0/3807030323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9ec102980 msgr2=0x7fc9ec19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:52.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.149+0000 7fc9f0bdb640 1 --2- 172.21.15.43:0/3807030323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9ec102980 0x7fc9ec19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.149+0000 7fc9f0bdb640 1 -- 172.21.15.43:0/3807030323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ec19fd90 con 0x7fc9ec100e70 2024-04-21T08:36:52.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.149+0000 7fc9f0bdb640 1 --2- 172.21.15.43:0/3807030323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9ec100e70 0x7fc9ec1981e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000d770 tx=0x7fc9e000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:52.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.149+0000 7fc9e9ffb640 1 -- 172.21.15.43:0/3807030323 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0017070 con 0x7fc9ec100e70 2024-04-21T08:36:52.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.149+0000 7fc9e9ffb640 1 -- 172.21.15.43:0/3807030323 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9e00054d0 con 0x7fc9ec100e70 2024-04-21T08:36:52.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.149+0000 7fc9e9ffb640 1 -- 172.21.15.43:0/3807030323 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0004c30 con 0x7fc9ec100e70 2024-04-21T08:36:52.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.149+0000 7fc9f2e3f640 1 -- 172.21.15.43:0/3807030323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9ec1a0080 con 0x7fc9ec100e70 2024-04-21T08:36:52.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.150+0000 7fc9f2e3f640 1 -- 172.21.15.43:0/3807030323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9ec1a0580 con 0x7fc9ec100e70 2024-04-21T08:36:52.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.151+0000 7fc9f2e3f640 1 -- 172.21.15.43:0/3807030323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9ec1a07e0 con 0x7fc9ec100e70 2024-04-21T08:36:52.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.152+0000 7fc9e9ffb640 1 -- 172.21.15.43:0/3807030323 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc9e0004e10 con 0x7fc9ec100e70 2024-04-21T08:36:52.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.152+0000 7fc9e9ffb640 1 --2- 172.21.15.43:0/3807030323 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc9dc0a3bb0 0x7fc9dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:52.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.152+0000 7fc9e9ffb640 1 -- 172.21.15.43:0/3807030323 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc9e00d0100 con 0x7fc9ec100e70 2024-04-21T08:36:52.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.152+0000 7fc9ebfff640 1 --2- 172.21.15.43:0/3807030323 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc9dc0a3bb0 0x7fc9dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:52.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.153+0000 7fc9ebfff640 1 --2- 172.21.15.43:0/3807030323 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc9dc0a3bb0 0x7fc9dc0a6070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc9d80095f0 tx=0x7fc9d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:52.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.157+0000 7fc9e9ffb640 1 -- 172.21.15.43:0/3807030323 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc9e009c8a0 con 0x7fc9ec100e70 2024-04-21T08:36:52.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:52 smithi043 ceph-mon[27978]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:52.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:52 smithi043 ceph-mon[27978]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:52.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:52 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] up:active 2024-04-21T08:36:52.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:52 smithi043 ceph-mon[27978]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-21T08:36:52.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:52 smithi043 ceph-mon[27978]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:36:52.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.375+0000 7fc9f2e3f640 1 -- 172.21.15.43:0/3807030323 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fc9ec0fe5c0 con 0x7fc9dc0a3bb0 2024-04-21T08:36:52.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.380+0000 7fc9e9ffb640 1 -- 172.21.15.43:0/3807030323 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fc9ec0fe5c0 con 0x7fc9dc0a3bb0 2024-04-21T08:36:52.380 INFO:teuthology.orchestra.run.smithi043.stdout:/volumes/qa/sv_1/2d33bd43-04a4-4a16-bde2-90a5eaec28bf 2024-04-21T08:36:52.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.382+0000 7fc9f2e3f640 1 -- 172.21.15.43:0/3807030323 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc9dc0a3bb0 msgr2=0x7fc9dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:52.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.382+0000 7fc9f2e3f640 1 --2- 172.21.15.43:0/3807030323 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc9dc0a3bb0 0x7fc9dc0a6070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc9d80095f0 tx=0x7fc9d8005f90 comp rx=0 tx=0).stop 2024-04-21T08:36:52.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.382+0000 7fc9f2e3f640 1 -- 172.21.15.43:0/3807030323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9ec100e70 msgr2=0x7fc9ec1981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:52.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.382+0000 7fc9f2e3f640 1 --2- 172.21.15.43:0/3807030323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9ec100e70 0x7fc9ec1981e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000d770 tx=0x7fc9e000dc30 comp rx=0 tx=0).stop 2024-04-21T08:36:52.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.382+0000 7fc9f2e3f640 1 -- 172.21.15.43:0/3807030323 shutdown_connections 2024-04-21T08:36:52.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.382+0000 7fc9f2e3f640 1 --2- 172.21.15.43:0/3807030323 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc9dc0a3bb0 0x7fc9dc0a6070 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.382+0000 7fc9f2e3f640 1 --2- 172.21.15.43:0/3807030323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9ec102980 0x7fc9ec19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.382+0000 7fc9f2e3f640 1 --2- 172.21.15.43:0/3807030323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9ec101fc0 0x7fc9ec198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.382+0000 7fc9f2e3f640 1 --2- 172.21.15.43:0/3807030323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9ec100e70 0x7fc9ec1981e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.382+0000 7fc9f2e3f640 1 -- 172.21.15.43:0/3807030323 >> 172.21.15.43:0/3807030323 conn(0x7fc9ec0fc3e0 msgr2=0x7fc9ec0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:52.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.382+0000 7fc9f2e3f640 1 -- 172.21.15.43:0/3807030323 shutdown_connections 2024-04-21T08:36:52.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.382+0000 7fc9f2e3f640 1 -- 172.21.15.43:0/3807030323 wait complete. 2024-04-21T08:36:52.393 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:36:52.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:52 smithi175 ceph-mon[31920]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:52.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:52 smithi175 ceph-mon[31920]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:52.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:52 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] up:active 2024-04-21T08:36:52.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:52 smithi175 ceph-mon[31920]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-21T08:36:52.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:52 smithi175 ceph-mon[31920]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:36:52.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.496+0000 7fb28cd75640 1 -- 172.21.15.43:0/3631767165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb288107ca0 msgr2=0x7fb28810a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:52.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.496+0000 7fb28cd75640 1 --2- 172.21.15.43:0/3631767165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb288107ca0 0x7fb28810a0f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb280009ca0 tx=0x7fb28002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.496+0000 7fb28cd75640 1 -- 172.21.15.43:0/3631767165 shutdown_connections 2024-04-21T08:36:52.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.496+0000 7fb28cd75640 1 --2- 172.21.15.43:0/3631767165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb288107ca0 0x7fb28810a0f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.496+0000 7fb28cd75640 1 --2- 172.21.15.43:0/3631767165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb288105370 0x7fb288107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.496+0000 7fb28cd75640 1 --2- 172.21.15.43:0/3631767165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb288102a40 0x7fb288104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.496+0000 7fb28cd75640 1 -- 172.21.15.43:0/3631767165 >> 172.21.15.43:0/3631767165 conn(0x7fb2880fc470 msgr2=0x7fb2880fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:52.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.496+0000 7fb28cd75640 1 -- 172.21.15.43:0/3631767165 shutdown_connections 2024-04-21T08:36:52.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.496+0000 7fb28cd75640 1 -- 172.21.15.43:0/3631767165 wait complete. 2024-04-21T08:36:52.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.497+0000 7fb28cd75640 1 Processor -- start 2024-04-21T08:36:52.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.497+0000 7fb28cd75640 1 -- start start 2024-04-21T08:36:52.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.497+0000 7fb28cd75640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb288102a40 0x7fb288102360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:52.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.497+0000 7fb28cd75640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb288105370 0x7fb2881009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:52.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.497+0000 7fb28cd75640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb288107ca0 0x7fb288100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:52.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.497+0000 7fb28cd75640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb28810db00 con 0x7fb288102a40 2024-04-21T08:36:52.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.497+0000 7fb28cd75640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb28810d980 con 0x7fb288107ca0 2024-04-21T08:36:52.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.497+0000 7fb28cd75640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb28810dc80 con 0x7fb288105370 2024-04-21T08:36:52.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.497+0000 7fb28659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb288102a40 0x7fb288102360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:52.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.497+0000 7fb28659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb288102a40 0x7fb288102360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35602/0 (socket says 172.21.15.43:35602) 2024-04-21T08:36:52.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.497+0000 7fb28659c640 1 -- 172.21.15.43:0/1674830096 learned_addr learned my addr 172.21.15.43:0/1674830096 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:52.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.497+0000 7fb286d9d640 1 --2- 172.21.15.43:0/1674830096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb288107ca0 0x7fb288100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:52.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.497+0000 7fb285d9b640 1 --2- 172.21.15.43:0/1674830096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb288105370 0x7fb2881009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:52.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.498+0000 7fb28659c640 1 -- 172.21.15.43:0/1674830096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb288105370 msgr2=0x7fb2881009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:52.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.498+0000 7fb28659c640 1 --2- 172.21.15.43:0/1674830096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb288105370 0x7fb2881009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.498+0000 7fb28659c640 1 -- 172.21.15.43:0/1674830096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb288107ca0 msgr2=0x7fb288100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:52.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.498+0000 7fb28659c640 1 --2- 172.21.15.43:0/1674830096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb288107ca0 0x7fb288100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.498+0000 7fb28659c640 1 -- 172.21.15.43:0/1674830096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2881017f0 con 0x7fb288102a40 2024-04-21T08:36:52.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.498+0000 7fb285d9b640 1 --2- 172.21.15.43:0/1674830096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb288105370 0x7fb2881009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:52.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.498+0000 7fb286d9d640 1 --2- 172.21.15.43:0/1674830096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb288107ca0 0x7fb288100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:36:52.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.498+0000 7fb28659c640 1 --2- 172.21.15.43:0/1674830096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb288102a40 0x7fb288102360 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb27800ea60 tx=0x7fb27800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:52.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.498+0000 7fb2777fe640 1 -- 172.21.15.43:0/1674830096 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27800cde0 con 0x7fb288102a40 2024-04-21T08:36:52.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.498+0000 7fb2777fe640 1 -- 172.21.15.43:0/1674830096 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2780035f0 con 0x7fb288102a40 2024-04-21T08:36:52.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.498+0000 7fb28cd75640 1 -- 172.21.15.43:0/1674830096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2881a22a0 con 0x7fb288102a40 2024-04-21T08:36:52.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.498+0000 7fb2777fe640 1 -- 172.21.15.43:0/1674830096 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb278010690 con 0x7fb288102a40 2024-04-21T08:36:52.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.498+0000 7fb28cd75640 1 -- 172.21.15.43:0/1674830096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2881a27c0 con 0x7fb288102a40 2024-04-21T08:36:52.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.499+0000 7fb28cd75640 1 -- 172.21.15.43:0/1674830096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2880fdf60 con 0x7fb288102a40 2024-04-21T08:36:52.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.500+0000 7fb2777fe640 1 -- 172.21.15.43:0/1674830096 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb2780037a0 con 0x7fb288102a40 2024-04-21T08:36:52.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.501+0000 7fb2777fe640 1 --2- 172.21.15.43:0/1674830096 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb2700a3bb0 0x7fb2700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:52.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.501+0000 7fb285d9b640 1 --2- 172.21.15.43:0/1674830096 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb2700a3bb0 0x7fb2700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:52.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.501+0000 7fb2777fe640 1 -- 172.21.15.43:0/1674830096 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb2780d0980 con 0x7fb288102a40 2024-04-21T08:36:52.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.501+0000 7fb285d9b640 1 --2- 172.21.15.43:0/1674830096 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb2700a3bb0 0x7fb2700a6070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb2881020d0 tx=0x7fb27c005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:52.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.505+0000 7fb2777fe640 1 -- 172.21.15.43:0/1674830096 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb27809d120 con 0x7fb288102a40 2024-04-21T08:36:52.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:52 smithi112 ceph-mon[31085]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:52.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:52 smithi112 ceph-mon[31085]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:52.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:52 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] up:active 2024-04-21T08:36:52.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:52 smithi112 ceph-mon[31085]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-21T08:36:52.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:52 smithi112 ceph-mon[31085]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:36:52.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.735+0000 7fb28cd75640 1 -- 172.21.15.43:0/1674830096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb28802d5c0 con 0x7fb288102a40 2024-04-21T08:36:52.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.736+0000 7fb2777fe640 1 -- 172.21.15.43:0/1674830096 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11938 (secure 0 0 0) 0x7fb27809ca20 con 0x7fb288102a40 2024-04-21T08:36:52.736 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:52.736 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:36:51.902455+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24491,24527,24487,24425,24412]},"id":1}]} 2024-04-21T08:36:52.736 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 27 2024-04-21T08:36:52.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.738+0000 7fb28cd75640 1 -- 172.21.15.43:0/1674830096 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb2700a3bb0 msgr2=0x7fb2700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:52.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.738+0000 7fb28cd75640 1 --2- 172.21.15.43:0/1674830096 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb2700a3bb0 0x7fb2700a6070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb2881020d0 tx=0x7fb27c005e50 comp rx=0 tx=0).stop 2024-04-21T08:36:52.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.738+0000 7fb28cd75640 1 -- 172.21.15.43:0/1674830096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb288102a40 msgr2=0x7fb288102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:52.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.738+0000 7fb28cd75640 1 --2- 172.21.15.43:0/1674830096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb288102a40 0x7fb288102360 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb27800ea60 tx=0x7fb27800ef20 comp rx=0 tx=0).stop 2024-04-21T08:36:52.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.738+0000 7fb28cd75640 1 -- 172.21.15.43:0/1674830096 shutdown_connections 2024-04-21T08:36:52.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.738+0000 7fb28cd75640 1 --2- 172.21.15.43:0/1674830096 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb2700a3bb0 0x7fb2700a6070 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.738+0000 7fb28cd75640 1 --2- 172.21.15.43:0/1674830096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb288107ca0 0x7fb288100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.738+0000 7fb28cd75640 1 --2- 172.21.15.43:0/1674830096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb288105370 0x7fb2881009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.738+0000 7fb28cd75640 1 --2- 172.21.15.43:0/1674830096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb288102a40 0x7fb288102360 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.738+0000 7fb28cd75640 1 -- 172.21.15.43:0/1674830096 >> 172.21.15.43:0/1674830096 conn(0x7fb2880fc470 msgr2=0x7fb288103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:52.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.738+0000 7fb28cd75640 1 -- 172.21.15.43:0/1674830096 shutdown_connections 2024-04-21T08:36:52.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.738+0000 7fb28cd75640 1 -- 172.21.15.43:0/1674830096 wait complete. 2024-04-21T08:36:52.749 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:36:52.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.852+0000 7f187910e640 1 -- 172.21.15.43:0/2226380132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18741029b0 msgr2=0x7f18741092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:52.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.852+0000 7f187910e640 1 --2- 172.21.15.43:0/2226380132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18741029b0 0x7f18741092d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f186c009ca0 tx=0x7f186c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.852+0000 7f187910e640 1 -- 172.21.15.43:0/2226380132 shutdown_connections 2024-04-21T08:36:52.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.852+0000 7f187910e640 1 --2- 172.21.15.43:0/2226380132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18741029b0 0x7f18741092d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.852+0000 7f187910e640 1 --2- 172.21.15.43:0/2226380132 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1874101ff0 0x7f1874102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.852+0000 7f187910e640 1 --2- 172.21.15.43:0/2226380132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1874100ea0 0x7f18741012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.852+0000 7f187910e640 1 -- 172.21.15.43:0/2226380132 >> 172.21.15.43:0/2226380132 conn(0x7f18740fc410 msgr2=0x7f18740fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:52.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.852+0000 7f187910e640 1 -- 172.21.15.43:0/2226380132 shutdown_connections 2024-04-21T08:36:52.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.852+0000 7f187910e640 1 -- 172.21.15.43:0/2226380132 wait complete. 2024-04-21T08:36:52.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.852+0000 7f187910e640 1 Processor -- start 2024-04-21T08:36:52.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.853+0000 7f187910e640 1 -- start start 2024-04-21T08:36:52.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.853+0000 7f187910e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1874100ea0 0x7f1874104e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:52.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.853+0000 7f187910e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1874101ff0 0x7f1874103460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:52.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.853+0000 7f187910e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18741029b0 0x7f18741039a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:52.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.853+0000 7f187910e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f187410c9f0 con 0x7f1874100ea0 2024-04-21T08:36:52.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.853+0000 7f187910e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f187410c870 con 0x7f1874101ff0 2024-04-21T08:36:52.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.853+0000 7f187910e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f187410cb70 con 0x7f18741029b0 2024-04-21T08:36:52.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.853+0000 7f1872d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1874100ea0 0x7f1874104e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:52.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.853+0000 7f1872d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1874100ea0 0x7f1874104e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35616/0 (socket says 172.21.15.43:35616) 2024-04-21T08:36:52.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.853+0000 7f1872d9d640 1 -- 172.21.15.43:0/1410514650 learned_addr learned my addr 172.21.15.43:0/1410514650 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:52.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.853+0000 7f187259c640 1 --2- 172.21.15.43:0/1410514650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1874101ff0 0x7f1874103460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:52.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.853+0000 7f187359e640 1 --2- 172.21.15.43:0/1410514650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18741029b0 0x7f18741039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:52.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.853+0000 7f1872d9d640 1 -- 172.21.15.43:0/1410514650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18741029b0 msgr2=0x7f18741039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:52.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.853+0000 7f1872d9d640 1 --2- 172.21.15.43:0/1410514650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18741029b0 0x7f18741039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.853+0000 7f1872d9d640 1 -- 172.21.15.43:0/1410514650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1874101ff0 msgr2=0x7f1874103460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:52.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.853+0000 7f1872d9d640 1 --2- 172.21.15.43:0/1410514650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1874101ff0 0x7f1874103460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.853+0000 7f1872d9d640 1 -- 172.21.15.43:0/1410514650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1874104230 con 0x7f1874100ea0 2024-04-21T08:36:52.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.853+0000 7f187259c640 1 --2- 172.21.15.43:0/1410514650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1874101ff0 0x7f1874103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:36:52.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.853+0000 7f1872d9d640 1 --2- 172.21.15.43:0/1410514650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1874100ea0 0x7f1874104e20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f186400e9f0 tx=0x7f186400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:52.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.854+0000 7f185bfff640 1 -- 172.21.15.43:0/1410514650 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186400cd80 con 0x7f1874100ea0 2024-04-21T08:36:52.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.854+0000 7f187910e640 1 -- 172.21.15.43:0/1410514650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18741a2270 con 0x7f1874100ea0 2024-04-21T08:36:52.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.854+0000 7f185bfff640 1 -- 172.21.15.43:0/1410514650 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18640108e0 con 0x7f1874100ea0 2024-04-21T08:36:52.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.854+0000 7f185bfff640 1 -- 172.21.15.43:0/1410514650 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864018980 con 0x7f1874100ea0 2024-04-21T08:36:52.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.854+0000 7f187910e640 1 -- 172.21.15.43:0/1410514650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18741a27c0 con 0x7f1874100ea0 2024-04-21T08:36:52.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.855+0000 7f187910e640 1 -- 172.21.15.43:0/1410514650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18741a2a80 con 0x7f1874100ea0 2024-04-21T08:36:52.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.856+0000 7f185bfff640 1 -- 172.21.15.43:0/1410514650 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1864010430 con 0x7f1874100ea0 2024-04-21T08:36:52.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.856+0000 7f185bfff640 1 --2- 172.21.15.43:0/1410514650 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f185c0a3bb0 0x7f185c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:52.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.856+0000 7f187259c640 1 --2- 172.21.15.43:0/1410514650 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f185c0a3bb0 0x7f185c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:52.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.856+0000 7f185bfff640 1 -- 172.21.15.43:0/1410514650 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f18640d0930 con 0x7f1874100ea0 2024-04-21T08:36:52.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.857+0000 7f187259c640 1 --2- 172.21.15.43:0/1410514650 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f185c0a3bb0 0x7f185c0a6070 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1874104b90 tx=0x7f186800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:52.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.861+0000 7f185bfff640 1 -- 172.21.15.43:0/1410514650 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f186409ca80 con 0x7f1874100ea0 2024-04-21T08:36:52.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:52.918+0000 7f185bfff640 1 -- 172.21.15.43:0/1410514650 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f18640990f0 con 0x7f1874100ea0 2024-04-21T08:36:53.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.056+0000 7f187910e640 1 -- 172.21.15.43:0/1410514650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18741a2c50 con 0x7f1874100ea0 2024-04-21T08:36:53.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.057+0000 7f185bfff640 1 -- 172.21.15.43:0/1410514650 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24152 (secure 0 0 0) 0x7f186409d180 con 0x7f1874100ea0 2024-04-21T08:36:53.058 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:53.058 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":90,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:36:28.364093+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:36:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.059+0000 7f187910e640 1 -- 172.21.15.43:0/1410514650 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f185c0a3bb0 msgr2=0x7f185c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.059+0000 7f187910e640 1 --2- 172.21.15.43:0/1410514650 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f185c0a3bb0 0x7f185c0a6070 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1874104b90 tx=0x7f186800b390 comp rx=0 tx=0).stop 2024-04-21T08:36:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.059+0000 7f187910e640 1 -- 172.21.15.43:0/1410514650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1874100ea0 msgr2=0x7f1874104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.059+0000 7f187910e640 1 --2- 172.21.15.43:0/1410514650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1874100ea0 0x7f1874104e20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f186400e9f0 tx=0x7f186400eeb0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.059+0000 7f187910e640 1 -- 172.21.15.43:0/1410514650 shutdown_connections 2024-04-21T08:36:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.059+0000 7f187910e640 1 --2- 172.21.15.43:0/1410514650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18741029b0 0x7f18741039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.059+0000 7f187910e640 1 --2- 172.21.15.43:0/1410514650 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f185c0a3bb0 0x7f185c0a6070 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.059+0000 7f187910e640 1 --2- 172.21.15.43:0/1410514650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1874101ff0 0x7f1874103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.059+0000 7f187910e640 1 --2- 172.21.15.43:0/1410514650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1874100ea0 0x7f1874104e20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.059+0000 7f187910e640 1 -- 172.21.15.43:0/1410514650 >> 172.21.15.43:0/1410514650 conn(0x7f18740fc410 msgr2=0x7f18740fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:53.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.059+0000 7f187910e640 1 -- 172.21.15.43:0/1410514650 shutdown_connections 2024-04-21T08:36:53.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.059+0000 7f187910e640 1 -- 172.21.15.43:0/1410514650 wait complete. 2024-04-21T08:36:53.072 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-21T08:36:53.072 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T08:36:53.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.173+0000 7f6a2d011640 1 -- 172.21.15.43:0/1367207399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a28103190 msgr2=0x7f6a28103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:53.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.173+0000 7f6a2d011640 1 --2- 172.21.15.43:0/1367207399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a28103190 0x7f6a28103610 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009900 tx=0x7f6a1c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:53.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.173+0000 7f6a2d011640 1 -- 172.21.15.43:0/1367207399 shutdown_connections 2024-04-21T08:36:53.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.173+0000 7f6a2d011640 1 --2- 172.21.15.43:0/1367207399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a28103b50 0x7f6a2810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.173+0000 7f6a2d011640 1 --2- 172.21.15.43:0/1367207399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a28103190 0x7f6a28103610 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.173+0000 7f6a2d011640 1 --2- 172.21.15.43:0/1367207399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a28101fa0 0x7f6a281023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.173+0000 7f6a2d011640 1 -- 172.21.15.43:0/1367207399 >> 172.21.15.43:0/1367207399 conn(0x7f6a280fd530 msgr2=0x7f6a280ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:53.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.173+0000 7f6a2d011640 1 -- 172.21.15.43:0/1367207399 shutdown_connections 2024-04-21T08:36:53.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.173+0000 7f6a2d011640 1 -- 172.21.15.43:0/1367207399 wait complete. 2024-04-21T08:36:53.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.174+0000 7f6a2d011640 1 Processor -- start 2024-04-21T08:36:53.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.174+0000 7f6a2d011640 1 -- start start 2024-04-21T08:36:53.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.174+0000 7f6a2d011640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a28101fa0 0x7f6a2819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:53.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.174+0000 7f6a2d011640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a28103190 0x7f6a2819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:53.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.174+0000 7f6a2d011640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a28103b50 0x7f6a281a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:53.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.174+0000 7f6a2d011640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2810dae0 con 0x7f6a28103b50 2024-04-21T08:36:53.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.174+0000 7f6a2d011640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2810d960 con 0x7f6a28103190 2024-04-21T08:36:53.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.174+0000 7f6a2d011640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2810dc60 con 0x7f6a28101fa0 2024-04-21T08:36:53.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.175+0000 7f6a2759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a28103b50 0x7f6a281a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:53.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.175+0000 7f6a2759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a28103b50 0x7f6a281a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35634/0 (socket says 172.21.15.43:35634) 2024-04-21T08:36:53.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.175+0000 7f6a2759e640 1 -- 172.21.15.43:0/3640406585 learned_addr learned my addr 172.21.15.43:0/3640406585 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:53.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.175+0000 7f6a26d9d640 1 --2- 172.21.15.43:0/3640406585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a28101fa0 0x7f6a2819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:53.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.175+0000 7f6a2659c640 1 --2- 172.21.15.43:0/3640406585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a28103190 0x7f6a2819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:53.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.175+0000 7f6a2759e640 1 -- 172.21.15.43:0/3640406585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a28101fa0 msgr2=0x7f6a2819a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:53.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.175+0000 7f6a2759e640 1 --2- 172.21.15.43:0/3640406585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a28101fa0 0x7f6a2819a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.175+0000 7f6a2759e640 1 -- 172.21.15.43:0/3640406585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a28103190 msgr2=0x7f6a2819a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:53.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.175+0000 7f6a2759e640 1 --2- 172.21.15.43:0/3640406585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a28103190 0x7f6a2819a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.175+0000 7f6a2759e640 1 -- 172.21.15.43:0/3640406585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a281a1fa0 con 0x7f6a28103b50 2024-04-21T08:36:53.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.175+0000 7f6a2759e640 1 --2- 172.21.15.43:0/3640406585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a28103b50 0x7f6a281a1930 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000eb10 tx=0x7f6a2000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:53.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.175+0000 7f6a0ffff640 1 -- 172.21.15.43:0/3640406585 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a20004b20 con 0x7f6a28103b50 2024-04-21T08:36:53.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.175+0000 7f6a2d011640 1 -- 172.21.15.43:0/3640406585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a281a2290 con 0x7f6a28103b50 2024-04-21T08:36:53.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.175+0000 7f6a0ffff640 1 -- 172.21.15.43:0/3640406585 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a20004cc0 con 0x7f6a28103b50 2024-04-21T08:36:53.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.175+0000 7f6a0ffff640 1 -- 172.21.15.43:0/3640406585 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a20010670 con 0x7f6a28103b50 2024-04-21T08:36:53.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.175+0000 7f6a2d011640 1 -- 172.21.15.43:0/3640406585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a281a2760 con 0x7f6a28103b50 2024-04-21T08:36:53.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.176+0000 7f6a2d011640 1 -- 172.21.15.43:0/3640406585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a281a2a20 con 0x7f6a28103b50 2024-04-21T08:36:53.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.177+0000 7f6a0ffff640 1 -- 172.21.15.43:0/3640406585 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6a20005000 con 0x7f6a28103b50 2024-04-21T08:36:53.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.178+0000 7f6a0ffff640 1 --2- 172.21.15.43:0/3640406585 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f6a100a3c40 0x7f6a100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:53.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.178+0000 7f6a26d9d640 1 --2- 172.21.15.43:0/3640406585 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f6a100a3c40 0x7f6a100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:53.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.179+0000 7f6a0ffff640 1 -- 172.21.15.43:0/3640406585 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f6a200d0f60 con 0x7f6a28103b50 2024-04-21T08:36:53.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.179+0000 7f6a26d9d640 1 --2- 172.21.15.43:0/3640406585 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f6a100a3c40 0x7f6a100a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6a18005fd0 tx=0x7f6a1800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:53.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.182+0000 7f6a0ffff640 1 -- 172.21.15.43:0/3640406585 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a20053ff0 con 0x7f6a28103b50 2024-04-21T08:36:53.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.367+0000 7f6a2d011640 1 -- 172.21.15.43:0/3640406585 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6a281a2bf0 con 0x7f6a100a3c40 2024-04-21T08:36:53.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.377+0000 7f6a0ffff640 1 -- 172.21.15.43:0/3640406585 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340313 (secure 0 0 0) 0x7f6a281a2bf0 con 0x7f6a100a3c40 2024-04-21T08:36:53.383 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:53.385 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-21T08:36:53.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.381+0000 7f6a2d011640 1 -- 172.21.15.43:0/3640406585 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f6a100a3c40 msgr2=0x7f6a100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:53.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.381+0000 7f6a2d011640 1 --2- 172.21.15.43:0/3640406585 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f6a100a3c40 0x7f6a100a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6a18005fd0 tx=0x7f6a1800b390 comp rx=0 tx=0).stop 2024-04-21T08:36:53.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.381+0000 7f6a2d011640 1 -- 172.21.15.43:0/3640406585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a28103b50 msgr2=0x7f6a281a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:53.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.381+0000 7f6a2d011640 1 --2- 172.21.15.43:0/3640406585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a28103b50 0x7f6a281a1930 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000eb10 tx=0x7f6a2000c2d0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.381+0000 7f6a2d011640 1 -- 172.21.15.43:0/3640406585 shutdown_connections 2024-04-21T08:36:53.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.381+0000 7f6a2d011640 1 --2- 172.21.15.43:0/3640406585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a28103b50 0x7f6a281a1930 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.381+0000 7f6a2d011640 1 --2- 172.21.15.43:0/3640406585 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f6a100a3c40 0x7f6a100a6100 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.381+0000 7f6a2d011640 1 --2- 172.21.15.43:0/3640406585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a28103190 0x7f6a2819a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.381+0000 7f6a2d011640 1 --2- 172.21.15.43:0/3640406585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a28101fa0 0x7f6a2819a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.381+0000 7f6a2d011640 1 -- 172.21.15.43:0/3640406585 >> 172.21.15.43:0/3640406585 conn(0x7f6a280fd530 msgr2=0x7f6a28074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:53.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.381+0000 7f6a2d011640 1 -- 172.21.15.43:0/3640406585 shutdown_connections 2024-04-21T08:36:53.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.381+0000 7f6a2d011640 1 -- 172.21.15.43:0/3640406585 wait complete. 2024-04-21T08:36:53.391 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:53 smithi043 ceph-mon[27978]: pgmap v309: 129 pgs: 129 active+clean; 586 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-21T08:36:53.391 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:53 smithi043 ceph-mon[27978]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:53.391 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:53 smithi043 ceph-mon[27978]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:53.391 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:53 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1674830096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:53.391 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:53 smithi043 ceph-mon[27978]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-21T08:36:53.392 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:53 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1410514650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:53.392 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2024-04-21T08:36:51.981193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":599806,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":316,"num_write_kb":2993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":261,"ondisk_log_size":261,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379488,"kb_used_data":13680,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562336,"statfs":{"total":1151940427776,"available":1151551832064,"internally_reserved":0,"allocated":14008320,"data_stored":7586914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":374453564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4939,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":88,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003622"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.506924+0000","last_change":"2024-04-21T08:36:23.983921+0000","last_active":"2024-04-21T08:36:29.506924+0000","last_peered":"2024-04-21T08:36:29.506924+0000","last_clean":"2024-04-21T08:36:29.506924+0000","last_became_active":"2024-04-21T08:36:23.983638+0000","last_became_peered":"2024-04-21T08:36:23.983638+0000","last_unstale":"2024-04-21T08:36:29.506924+0000","last_undegraded":"2024-04-21T08:36:29.506924+0000","last_fullsized":"2024-04-21T08:36:29.506924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:29:44.646308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.643913+0000","last_change":"2024-04-21T08:36:23.990029+0000","last_active":"2024-04-21T08:36:29.643913+0000","last_peered":"2024-04-21T08:36:29.643913+0000","last_clean":"2024-04-21T08:36:29.643913+0000","last_became_active":"2024-04-21T08:36:23.989141+0000","last_became_peered":"2024-04-21T08:36:23.989141+0000","last_unstale":"2024-04-21T08:36:29.643913+0000","last_undegraded":"2024-04-21T08:36:29.643913+0000","last_fullsized":"2024-04-21T08:36:29.643913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:23:58.692342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.643897+0000","last_change":"2024-04-21T08:36:23.982205+0000","last_active":"2024-04-21T08:36:29.643897+0000","last_peered":"2024-04-21T08:36:29.643897+0000","last_clean":"2024-04-21T08:36:29.643897+0000","last_became_active":"2024-04-21T08:36:23.981876+0000","last_became_peered":"2024-04-21T08:36:23.981876+0000","last_unstale":"2024-04-21T08:36:29.643897+0000","last_undegraded":"2024-04-21T08:36:29.643897+0000","last_fullsized":"2024-04-21T08:36:29.643897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:12:27.121106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645335+0000","last_change":"2024-04-21T08:36:23.987425+0000","last_active":"2024-04-21T08:36:29.645335+0000","last_peered":"2024-04-21T08:36:29.645335+0000","last_clean":"2024-04-21T08:36:29.645335+0000","last_became_active":"2024-04-21T08:36:23.987109+0000","last_became_peered":"2024-04-21T08:36:23.987109+0000","last_unstale":"2024-04-21T08:36:29.645335+0000","last_undegraded":"2024-04-21T08:36:29.645335+0000","last_fullsized":"2024-04-21T08:36:29.645335+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:44.176324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644476+0000","last_change":"2024-04-21T08:36:23.990375+0000","last_active":"2024-04-21T08:36:29.644476+0000","last_peered":"2024-04-21T08:36:29.644476+0000","last_clean":"2024-04-21T08:36:29.644476+0000","last_became_active":"2024-04-21T08:36:23.990058+0000","last_became_peered":"2024-04-21T08:36:23.990058+0000","last_unstale":"2024-04-21T08:36:29.644476+0000","last_undegraded":"2024-04-21T08:36:29.644476+0000","last_fullsized":"2024-04-21T08:36:29.644476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:24:09.647528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644732+0000","last_change":"2024-04-21T08:36:23.979848+0000","last_active":"2024-04-21T08:36:29.644732+0000","last_peered":"2024-04-21T08:36:29.644732+0000","last_clean":"2024-04-21T08:36:29.644732+0000","last_became_active":"2024-04-21T08:36:23.979511+0000","last_became_peered":"2024-04-21T08:36:23.979511+0000","last_unstale":"2024-04-21T08:36:29.644732+0000","last_undegraded":"2024-04-21T08:36:29.644732+0000","last_fullsized":"2024-04-21T08:36:29.644732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:07:46.551369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992965+0000","last_change":"2024-04-21T08:36:23.985353+0000","last_active":"2024-04-21T08:36:28.992965+0000","last_peered":"2024-04-21T08:36:28.992965+0000","last_clean":"2024-04-21T08:36:28.992965+0000","last_became_active":"2024-04-21T08:36:23.984754+0000","last_became_peered":"2024-04-21T08:36:23.984754+0000","last_unstale":"2024-04-21T08:36:28.992965+0000","last_undegraded":"2024-04-21T08:36:28.992965+0000","last_fullsized":"2024-04-21T08:36:28.992965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:19:25.900776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644547+0000","last_change":"2024-04-21T08:36:23.987517+0000","last_active":"2024-04-21T08:36:29.644547+0000","last_peered":"2024-04-21T08:36:29.644547+0000","last_clean":"2024-04-21T08:36:29.644547+0000","last_became_active":"2024-04-21T08:36:23.987201+0000","last_became_peered":"2024-04-21T08:36:23.987201+0000","last_unstale":"2024-04-21T08:36:29.644547+0000","last_undegraded":"2024-04-21T08:36:29.644547+0000","last_fullsized":"2024-04-21T08:36:29.644547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:42:53.080863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645884+0000","last_change":"2024-04-21T08:36:23.989434+0000","last_active":"2024-04-21T08:36:29.645884+0000","last_peered":"2024-04-21T08:36:29.645884+0000","last_clean":"2024-04-21T08:36:29.645884+0000","last_became_active":"2024-04-21T08:36:23.989125+0000","last_became_peered":"2024-04-21T08:36:23.989125+0000","last_unstale":"2024-04-21T08:36:29.645884+0000","last_undegraded":"2024-04-21T08:36:29.645884+0000","last_fullsized":"2024-04-21T08:36:29.645884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:30:07.219056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645987+0000","last_change":"2024-04-21T08:36:23.980725+0000","last_active":"2024-04-21T08:36:29.645987+0000","last_peered":"2024-04-21T08:36:29.645987+0000","last_clean":"2024-04-21T08:36:29.645987+0000","last_became_active":"2024-04-21T08:36:23.980363+0000","last_became_peered":"2024-04-21T08:36:23.980363+0000","last_unstale":"2024-04-21T08:36:29.645987+0000","last_undegraded":"2024-04-21T08:36:29.645987+0000","last_fullsized":"2024-04-21T08:36:29.645987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:26:46.202030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992860+0000","last_change":"2024-04-21T08:36:23.989369+0000","last_active":"2024-04-21T08:36:28.992860+0000","last_peered":"2024-04-21T08:36:28.992860+0000","last_clean":"2024-04-21T08:36:28.992860+0000","last_became_active":"2024-04-21T08:36:23.989030+0000","last_became_peered":"2024-04-21T08:36:23.989030+0000","last_unstale":"2024-04-21T08:36:28.992860+0000","last_undegraded":"2024-04-21T08:36:28.992860+0000","last_fullsized":"2024-04-21T08:36:28.992860+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:23:13.725703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644985+0000","last_change":"2024-04-21T08:36:23.993485+0000","last_active":"2024-04-21T08:36:29.644985+0000","last_peered":"2024-04-21T08:36:29.644985+0000","last_clean":"2024-04-21T08:36:29.644985+0000","last_became_active":"2024-04-21T08:36:23.993124+0000","last_became_peered":"2024-04-21T08:36:23.993124+0000","last_unstale":"2024-04-21T08:36:29.644985+0000","last_undegraded":"2024-04-21T08:36:29.644985+0000","last_fullsized":"2024-04-21T08:36:29.644985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:53:16.249838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.991991+0000","last_change":"2024-04-21T08:36:23.983146+0000","last_active":"2024-04-21T08:36:28.991991+0000","last_peered":"2024-04-21T08:36:28.991991+0000","last_clean":"2024-04-21T08:36:28.991991+0000","last_became_active":"2024-04-21T08:36:23.982766+0000","last_became_peered":"2024-04-21T08:36:23.982766+0000","last_unstale":"2024-04-21T08:36:28.991991+0000","last_undegraded":"2024-04-21T08:36:28.991991+0000","last_fullsized":"2024-04-21T08:36:28.991991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:30:44.090929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644576+0000","last_change":"2024-04-21T08:36:23.988906+0000","last_active":"2024-04-21T08:36:29.644576+0000","last_peered":"2024-04-21T08:36:29.644576+0000","last_clean":"2024-04-21T08:36:29.644576+0000","last_became_active":"2024-04-21T08:36:23.988489+0000","last_became_peered":"2024-04-21T08:36:23.988489+0000","last_unstale":"2024-04-21T08:36:29.644576+0000","last_undegraded":"2024-04-21T08:36:29.644576+0000","last_fullsized":"2024-04-21T08:36:29.644576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:06:43.271650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.643779+0000","last_change":"2024-04-21T08:36:23.986112+0000","last_active":"2024-04-21T08:36:29.643779+0000","last_peered":"2024-04-21T08:36:29.643779+0000","last_clean":"2024-04-21T08:36:29.643779+0000","last_became_active":"2024-04-21T08:36:23.985801+0000","last_became_peered":"2024-04-21T08:36:23.985801+0000","last_unstale":"2024-04-21T08:36:29.643779+0000","last_undegraded":"2024-04-21T08:36:29.643779+0000","last_fullsized":"2024-04-21T08:36:29.643779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:28:45.793440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644227+0000","last_change":"2024-04-21T08:36:23.983136+0000","last_active":"2024-04-21T08:36:29.644227+0000","last_peered":"2024-04-21T08:36:29.644227+0000","last_clean":"2024-04-21T08:36:29.644227+0000","last_became_active":"2024-04-21T08:36:23.982698+0000","last_became_peered":"2024-04-21T08:36:23.982698+0000","last_unstale":"2024-04-21T08:36:29.644227+0000","last_undegraded":"2024-04-21T08:36:29.644227+0000","last_fullsized":"2024-04-21T08:36:29.644227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:34.750941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.506713+0000","last_change":"2024-04-21T08:36:23.984701+0000","last_active":"2024-04-21T08:36:29.506713+0000","last_peered":"2024-04-21T08:36:29.506713+0000","last_clean":"2024-04-21T08:36:29.506713+0000","last_became_active":"2024-04-21T08:36:23.984432+0000","last_became_peered":"2024-04-21T08:36:23.984432+0000","last_unstale":"2024-04-21T08:36:29.506713+0000","last_undegraded":"2024-04-21T08:36:29.506713+0000","last_fullsized":"2024-04-21T08:36:29.506713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:10:58.944380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645930+0000","last_change":"2024-04-21T08:36:23.982781+0000","last_active":"2024-04-21T08:36:29.645930+0000","last_peered":"2024-04-21T08:36:29.645930+0000","last_clean":"2024-04-21T08:36:29.645930+0000","last_became_active":"2024-04-21T08:36:23.982459+0000","last_became_peered":"2024-04-21T08:36:23.982459+0000","last_unstale":"2024-04-21T08:36:29.645930+0000","last_undegraded":"2024-04-21T08:36:29.645930+0000","last_fullsized":"2024-04-21T08:36:29.645930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:44.657504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644800+0000","last_change":"2024-04-21T08:36:23.992002+0000","last_active":"2024-04-21T08:36:29.644800+0000","last_peered":"2024-04-21T08:36:29.644800+0000","last_clean":"2024-04-21T08:36:29.644800+0000","last_became_active":"2024-04-21T08:36:23.991053+0000","last_became_peered":"2024-04-21T08:36:23.991053+0000","last_unstale":"2024-04-21T08:36:29.644800+0000","last_undegraded":"2024-04-21T08:36:29.644800+0000","last_fullsized":"2024-04-21T08:36:29.644800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:56:37.145949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644395+0000","last_change":"2024-04-21T08:36:28.013195+0000","last_active":"2024-04-21T08:36:29.644395+0000","last_peered":"2024-04-21T08:36:29.644395+0000","last_clean":"2024-04-21T08:36:29.644395+0000","last_became_active":"2024-04-21T08:36:28.012656+0000","last_became_peered":"2024-04-21T08:36:28.012656+0000","last_unstale":"2024-04-21T08:36:29.644395+0000","last_undegraded":"2024-04-21T08:36:29.644395+0000","last_fullsized":"2024-04-21T08:36:29.644395+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:12:03.133583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:50.907118+0000","last_change":"2024-04-21T08:36:23.468957+0000","last_active":"2024-04-21T08:36:50.907118+0000","last_peered":"2024-04-21T08:36:50.907118+0000","last_clean":"2024-04-21T08:36:50.907118+0000","last_became_active":"2024-04-21T08:36:23.468601+0000","last_became_peered":"2024-04-21T08:36:23.468601+0000","last_unstale":"2024-04-21T08:36:50.907118+0000","last_undegraded":"2024-04-21T08:36:50.907118+0000","last_fullsized":"2024-04-21T08:36:50.907118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:13:11.977780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645525+0000","last_change":"2024-04-21T08:36:23.993520+0000","last_active":"2024-04-21T08:36:29.645525+0000","last_peered":"2024-04-21T08:36:29.645525+0000","last_clean":"2024-04-21T08:36:29.645525+0000","last_became_active":"2024-04-21T08:36:23.993169+0000","last_became_peered":"2024-04-21T08:36:23.993169+0000","last_unstale":"2024-04-21T08:36:29.645525+0000","last_undegraded":"2024-04-21T08:36:29.645525+0000","last_fullsized":"2024-04-21T08:36:29.645525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:26:28.047696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644631+0000","last_change":"2024-04-21T08:36:28.014161+0000","last_active":"2024-04-21T08:36:29.644631+0000","last_peered":"2024-04-21T08:36:29.644631+0000","last_clean":"2024-04-21T08:36:29.644631+0000","last_became_active":"2024-04-21T08:36:28.013859+0000","last_became_peered":"2024-04-21T08:36:28.013859+0000","last_unstale":"2024-04-21T08:36:29.644631+0000","last_undegraded":"2024-04-21T08:36:29.644631+0000","last_fullsized":"2024-04-21T08:36:29.644631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:08.315334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645773+0000","last_change":"2024-04-21T08:36:23.475612+0000","last_active":"2024-04-21T08:36:29.645773+0000","last_peered":"2024-04-21T08:36:29.645773+0000","last_clean":"2024-04-21T08:36:29.645773+0000","last_became_active":"2024-04-21T08:36:23.475301+0000","last_became_peered":"2024-04-21T08:36:23.475301+0000","last_unstale":"2024-04-21T08:36:29.645773+0000","last_undegraded":"2024-04-21T08:36:29.645773+0000","last_fullsized":"2024-04-21T08:36:29.645773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:56.801630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644079+0000","last_change":"2024-04-21T08:36:23.984689+0000","last_active":"2024-04-21T08:36:29.644079+0000","last_peered":"2024-04-21T08:36:29.644079+0000","last_clean":"2024-04-21T08:36:29.644079+0000","last_became_active":"2024-04-21T08:36:23.984358+0000","last_became_peered":"2024-04-21T08:36:23.984358+0000","last_unstale":"2024-04-21T08:36:29.644079+0000","last_undegraded":"2024-04-21T08:36:29.644079+0000","last_fullsized":"2024-04-21T08:36:29.644079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:49:50.187466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644832+0000","last_change":"2024-04-21T08:36:28.007951+0000","last_active":"2024-04-21T08:36:29.644832+0000","last_peered":"2024-04-21T08:36:29.644832+0000","last_clean":"2024-04-21T08:36:29.644832+0000","last_became_active":"2024-04-21T08:36:28.007553+0000","last_became_peered":"2024-04-21T08:36:28.007553+0000","last_unstale":"2024-04-21T08:36:29.644832+0000","last_undegraded":"2024-04-21T08:36:29.644832+0000","last_fullsized":"2024-04-21T08:36:29.644832+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:10:54.019605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:49.069870+0000","last_change":"2024-04-21T08:36:23.473371+0000","last_active":"2024-04-21T08:36:49.069870+0000","last_peered":"2024-04-21T08:36:49.069870+0000","last_clean":"2024-04-21T08:36:49.069870+0000","last_became_active":"2024-04-21T08:36:23.472472+0000","last_became_peered":"2024-04-21T08:36:23.472472+0000","last_unstale":"2024-04-21T08:36:49.069870+0000","last_undegraded":"2024-04-21T08:36:49.069870+0000","last_fullsized":"2024-04-21T08:36:49.069870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:28:49.622873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645353+0000","last_change":"2024-04-21T08:36:23.987916+0000","last_active":"2024-04-21T08:36:29.645353+0000","last_peered":"2024-04-21T08:36:29.645353+0000","last_clean":"2024-04-21T08:36:29.645353+0000","last_became_active":"2024-04-21T08:36:23.986207+0000","last_became_peered":"2024-04-21T08:36:23.986207+0000","last_unstale":"2024-04-21T08:36:29.645353+0000","last_undegraded":"2024-04-21T08:36:29.645353+0000","last_fullsized":"2024-04-21T08:36:29.645353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:49:57.261078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644682+0000","last_change":"2024-04-21T08:36:28.006494+0000","last_active":"2024-04-21T08:36:29.644682+0000","last_peered":"2024-04-21T08:36:29.644682+0000","last_clean":"2024-04-21T08:36:29.644682+0000","last_became_active":"2024-04-21T08:36:28.006069+0000","last_became_peered":"2024-04-21T08:36:28.006069+0000","last_unstale":"2024-04-21T08:36:29.644682+0000","last_undegraded":"2024-04-21T08:36:29.644682+0000","last_fullsized":"2024-04-21T08:36:29.644682+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:04.608295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645118+0000","last_change":"2024-04-21T08:36:23.468923+0000","last_active":"2024-04-21T08:36:29.645118+0000","last_peered":"2024-04-21T08:36:29.645118+0000","last_clean":"2024-04-21T08:36:29.645118+0000","last_became_active":"2024-04-21T08:36:23.468509+0000","last_became_peered":"2024-04-21T08:36:23.468509+0000","last_unstale":"2024-04-21T08:36:29.645118+0000","last_undegraded":"2024-04-21T08:36:29.645118+0000","last_fullsized":"2024-04-21T08:36:29.645118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:44.452748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.995078+0000","last_change":"2024-04-21T08:36:23.994423+0000","last_active":"2024-04-21T08:36:28.995078+0000","last_peered":"2024-04-21T08:36:28.995078+0000","last_clean":"2024-04-21T08:36:28.995078+0000","last_became_active":"2024-04-21T08:36:23.993827+0000","last_became_peered":"2024-04-21T08:36:23.993827+0000","last_unstale":"2024-04-21T08:36:28.995078+0000","last_undegraded":"2024-04-21T08:36:28.995078+0000","last_fullsized":"2024-04-21T08:36:28.995078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:49:25.550301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644795+0000","last_change":"2024-04-21T08:36:28.004037+0000","last_active":"2024-04-21T08:36:29.644795+0000","last_peered":"2024-04-21T08:36:29.644795+0000","last_clean":"2024-04-21T08:36:29.644795+0000","last_became_active":"2024-04-21T08:36:28.003580+0000","last_became_peered":"2024-04-21T08:36:28.003580+0000","last_unstale":"2024-04-21T08:36:29.644795+0000","last_undegraded":"2024-04-21T08:36:29.644795+0000","last_fullsized":"2024-04-21T08:36:29.644795+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:04:51.380244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:50.884881+0000","last_change":"2024-04-21T08:36:23.468430+0000","last_active":"2024-04-21T08:36:50.884881+0000","last_peered":"2024-04-21T08:36:50.884881+0000","last_clean":"2024-04-21T08:36:50.884881+0000","last_became_active":"2024-04-21T08:36:23.468089+0000","last_became_peered":"2024-04-21T08:36:23.468089+0000","last_unstale":"2024-04-21T08:36:50.884881+0000","last_undegraded":"2024-04-21T08:36:50.884881+0000","last_fullsized":"2024-04-21T08:36:50.884881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:12:54.827034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644971+0000","last_change":"2024-04-21T08:36:23.985693+0000","last_active":"2024-04-21T08:36:29.644971+0000","last_peered":"2024-04-21T08:36:29.644971+0000","last_clean":"2024-04-21T08:36:29.644971+0000","last_became_active":"2024-04-21T08:36:23.985373+0000","last_became_peered":"2024-04-21T08:36:23.985373+0000","last_unstale":"2024-04-21T08:36:29.644971+0000","last_undegraded":"2024-04-21T08:36:29.644971+0000","last_fullsized":"2024-04-21T08:36:29.644971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:49:42.045542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644776+0000","last_change":"2024-04-21T08:36:28.004638+0000","last_active":"2024-04-21T08:36:29.644776+0000","last_peered":"2024-04-21T08:36:29.644776+0000","last_clean":"2024-04-21T08:36:29.644776+0000","last_became_active":"2024-04-21T08:36:28.002499+0000","last_became_peered":"2024-04-21T08:36:28.002499+0000","last_unstale":"2024-04-21T08:36:29.644776+0000","last_undegraded":"2024-04-21T08:36:29.644776+0000","last_fullsized":"2024-04-21T08:36:29.644776+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:45:46.934783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644598+0000","last_change":"2024-04-21T08:36:23.465783+0000","last_active":"2024-04-21T08:36:29.644598+0000","last_peered":"2024-04-21T08:36:29.644598+0000","last_clean":"2024-04-21T08:36:29.644598+0000","last_became_active":"2024-04-21T08:36:23.465478+0000","last_became_peered":"2024-04-21T08:36:23.465478+0000","last_unstale":"2024-04-21T08:36:29.644598+0000","last_undegraded":"2024-04-21T08:36:29.644598+0000","last_fullsized":"2024-04-21T08:36:29.644598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:56:06.275800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.994787+0000","last_change":"2024-04-21T08:36:23.995407+0000","last_active":"2024-04-21T08:36:28.994787+0000","last_peered":"2024-04-21T08:36:28.994787+0000","last_clean":"2024-04-21T08:36:28.994787+0000","last_became_active":"2024-04-21T08:36:23.995102+0000","last_became_peered":"2024-04-21T08:36:23.995102+0000","last_unstale":"2024-04-21T08:36:28.994787+0000","last_undegraded":"2024-04-21T08:36:28.994787+0000","last_fullsized":"2024-04-21T08:36:28.994787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:03:42.306628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.506803+0000","last_change":"2024-04-21T08:36:28.017068+0000","last_active":"2024-04-21T08:36:29.506803+0000","last_peered":"2024-04-21T08:36:29.506803+0000","last_clean":"2024-04-21T08:36:29.506803+0000","last_became_active":"2024-04-21T08:36:28.016770+0000","last_became_peered":"2024-04-21T08:36:28.016770+0000","last_unstale":"2024-04-21T08:36:29.506803+0000","last_undegraded":"2024-04-21T08:36:29.506803+0000","last_fullsized":"2024-04-21T08:36:29.506803+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:07:14.596090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.991419+0000","last_change":"2024-04-21T08:36:23.470430+0000","last_active":"2024-04-21T08:36:28.991419+0000","last_peered":"2024-04-21T08:36:28.991419+0000","last_clean":"2024-04-21T08:36:28.991419+0000","last_became_active":"2024-04-21T08:36:23.470109+0000","last_became_peered":"2024-04-21T08:36:23.470109+0000","last_unstale":"2024-04-21T08:36:28.991419+0000","last_undegraded":"2024-04-21T08:36:28.991419+0000","last_fullsized":"2024-04-21T08:36:28.991419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:18.176313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644710+0000","last_change":"2024-04-21T08:36:23.986668+0000","last_active":"2024-04-21T08:36:29.644710+0000","last_peered":"2024-04-21T08:36:29.644710+0000","last_clean":"2024-04-21T08:36:29.644710+0000","last_became_active":"2024-04-21T08:36:23.986344+0000","last_became_peered":"2024-04-21T08:36:23.986344+0000","last_unstale":"2024-04-21T08:36:29.644710+0000","last_undegraded":"2024-04-21T08:36:29.644710+0000","last_fullsized":"2024-04-21T08:36:29.644710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:56:17.744145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.643936+0000","last_change":"2024-04-21T08:36:28.009012+0000","last_active":"2024-04-21T08:36:29.643936+0000","last_peered":"2024-04-21T08:36:29.643936+0000","last_clean":"2024-04-21T08:36:29.643936+0000","last_became_active":"2024-04-21T08:36:28.008123+0000","last_became_peered":"2024-04-21T08:36:28.008123+0000","last_unstale":"2024-04-21T08:36:29.643936+0000","last_undegraded":"2024-04-21T08:36:29.643936+0000","last_fullsized":"2024-04-21T08:36:29.643936+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:33:18.881564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:49.890296+0000","last_change":"2024-04-21T08:36:23.467104+0000","last_active":"2024-04-21T08:36:49.890296+0000","last_peered":"2024-04-21T08:36:49.890296+0000","last_clean":"2024-04-21T08:36:49.890296+0000","last_became_active":"2024-04-21T08:36:23.466725+0000","last_became_peered":"2024-04-21T08:36:23.466725+0000","last_unstale":"2024-04-21T08:36:49.890296+0000","last_undegraded":"2024-04-21T08:36:49.890296+0000","last_fullsized":"2024-04-21T08:36:49.890296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:35:45.986448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644698+0000","last_change":"2024-04-21T08:36:23.987579+0000","last_active":"2024-04-21T08:36:29.644698+0000","last_peered":"2024-04-21T08:36:29.644698+0000","last_clean":"2024-04-21T08:36:29.644698+0000","last_became_active":"2024-04-21T08:36:23.987044+0000","last_became_peered":"2024-04-21T08:36:23.987044+0000","last_unstale":"2024-04-21T08:36:29.644698+0000","last_undegraded":"2024-04-21T08:36:29.644698+0000","last_fullsized":"2024-04-21T08:36:29.644698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:48:27.593090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645100+0000","last_change":"2024-04-21T08:36:28.006417+0000","last_active":"2024-04-21T08:36:29.645100+0000","last_peered":"2024-04-21T08:36:29.645100+0000","last_clean":"2024-04-21T08:36:29.645100+0000","last_became_active":"2024-04-21T08:36:28.006006+0000","last_became_peered":"2024-04-21T08:36:28.006006+0000","last_unstale":"2024-04-21T08:36:29.645100+0000","last_undegraded":"2024-04-21T08:36:29.645100+0000","last_fullsized":"2024-04-21T08:36:29.645100+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:16.173284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.646025+0000","last_change":"2024-04-21T08:36:23.471541+0000","last_active":"2024-04-21T08:36:29.646025+0000","last_peered":"2024-04-21T08:36:29.646025+0000","last_clean":"2024-04-21T08:36:29.646025+0000","last_became_active":"2024-04-21T08:36:23.471193+0000","last_became_peered":"2024-04-21T08:36:23.471193+0000","last_unstale":"2024-04-21T08:36:29.646025+0000","last_undegraded":"2024-04-21T08:36:29.646025+0000","last_fullsized":"2024-04-21T08:36:29.646025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:06:04.916707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645500+0000","last_change":"2024-04-21T08:36:23.986781+0000","last_active":"2024-04-21T08:36:29.645500+0000","last_peered":"2024-04-21T08:36:29.645500+0000","last_clean":"2024-04-21T08:36:29.645500+0000","last_became_active":"2024-04-21T08:36:23.985696+0000","last_became_peered":"2024-04-21T08:36:23.985696+0000","last_unstale":"2024-04-21T08:36:29.645500+0000","last_undegraded":"2024-04-21T08:36:29.645500+0000","last_fullsized":"2024-04-21T08:36:29.645500+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:40.556642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644245+0000","last_change":"2024-04-21T08:36:28.011621+0000","last_active":"2024-04-21T08:36:29.644245+0000","last_peered":"2024-04-21T08:36:29.644245+0000","last_clean":"2024-04-21T08:36:29.644245+0000","last_became_active":"2024-04-21T08:36:28.011136+0000","last_became_peered":"2024-04-21T08:36:28.011136+0000","last_unstale":"2024-04-21T08:36:29.644245+0000","last_undegraded":"2024-04-21T08:36:29.644245+0000","last_fullsized":"2024-04-21T08:36:29.644245+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:47.623445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:48.062410+0000","last_change":"2024-04-21T08:36:23.474019+0000","last_active":"2024-04-21T08:36:48.062410+0000","last_peered":"2024-04-21T08:36:48.062410+0000","last_clean":"2024-04-21T08:36:48.062410+0000","last_became_active":"2024-04-21T08:36:23.473690+0000","last_became_peered":"2024-04-21T08:36:23.473690+0000","last_unstale":"2024-04-21T08:36:48.062410+0000","last_undegraded":"2024-04-21T08:36:48.062410+0000","last_fullsized":"2024-04-21T08:36:48.062410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:38:32.913795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644171+0000","last_change":"2024-04-21T08:36:23.994434+0000","last_active":"2024-04-21T08:36:29.644171+0000","last_peered":"2024-04-21T08:36:29.644171+0000","last_clean":"2024-04-21T08:36:29.644171+0000","last_became_active":"2024-04-21T08:36:23.993675+0000","last_became_peered":"2024-04-21T08:36:23.993675+0000","last_unstale":"2024-04-21T08:36:29.644171+0000","last_undegraded":"2024-04-21T08:36:29.644171+0000","last_fullsized":"2024-04-21T08:36:29.644171+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:53:07.613445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645159+0000","last_change":"2024-04-21T08:36:28.004420+0000","last_active":"2024-04-21T08:36:29.645159+0000","last_peered":"2024-04-21T08:36:29.645159+0000","last_clean":"2024-04-21T08:36:29.645159+0000","last_became_active":"2024-04-21T08:36:28.003935+0000","last_became_peered":"2024-04-21T08:36:28.003935+0000","last_unstale":"2024-04-21T08:36:29.645159+0000","last_undegraded":"2024-04-21T08:36:29.645159+0000","last_fullsized":"2024-04-21T08:36:29.645159+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:10:03.255869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:49.882859+0000","last_change":"2024-04-21T08:36:23.465825+0000","last_active":"2024-04-21T08:36:49.882859+0000","last_peered":"2024-04-21T08:36:49.882859+0000","last_clean":"2024-04-21T08:36:49.882859+0000","last_became_active":"2024-04-21T08:36:23.465525+0000","last_became_peered":"2024-04-21T08:36:23.465525+0000","last_unstale":"2024-04-21T08:36:49.882859+0000","last_undegraded":"2024-04-21T08:36:49.882859+0000","last_fullsized":"2024-04-21T08:36:49.882859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:59:50.502210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645512+0000","last_change":"2024-04-21T08:36:23.987602+0000","last_active":"2024-04-21T08:36:29.645512+0000","last_peered":"2024-04-21T08:36:29.645512+0000","last_clean":"2024-04-21T08:36:29.645512+0000","last_became_active":"2024-04-21T08:36:23.987298+0000","last_became_peered":"2024-04-21T08:36:23.987298+0000","last_unstale":"2024-04-21T08:36:29.645512+0000","last_undegraded":"2024-04-21T08:36:29.645512+0000","last_fullsized":"2024-04-21T08:36:29.645512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:06.647676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645386+0000","last_change":"2024-04-21T08:36:28.005935+0000","last_active":"2024-04-21T08:36:29.645386+0000","last_peered":"2024-04-21T08:36:29.645386+0000","last_clean":"2024-04-21T08:36:29.645386+0000","last_became_active":"2024-04-21T08:36:28.005512+0000","last_became_peered":"2024-04-21T08:36:28.005512+0000","last_unstale":"2024-04-21T08:36:29.645386+0000","last_undegraded":"2024-04-21T08:36:29.645386+0000","last_fullsized":"2024-04-21T08:36:29.645386+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:23:20.298524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'10","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:39.661040+0000","last_change":"2024-04-21T08:36:23.467728+0000","last_active":"2024-04-21T08:36:39.661040+0000","last_peered":"2024-04-21T08:36:39.661040+0000","last_clean":"2024-04-21T08:36:39.661040+0000","last_became_active":"2024-04-21T08:36:23.467430+0000","last_became_peered":"2024-04-21T08:36:23.467430+0000","last_unstale":"2024-04-21T08:36:39.661040+0000","last_undegraded":"2024-04-21T08:36:39.661040+0000","last_fullsized":"2024-04-21T08:36:39.661040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:47:18.900911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645232+0000","last_change":"2024-04-21T08:36:23.989116+0000","last_active":"2024-04-21T08:36:29.645232+0000","last_peered":"2024-04-21T08:36:29.645232+0000","last_clean":"2024-04-21T08:36:29.645232+0000","last_became_active":"2024-04-21T08:36:23.988758+0000","last_became_peered":"2024-04-21T08:36:23.988758+0000","last_unstale":"2024-04-21T08:36:29.645232+0000","last_undegraded":"2024-04-21T08:36:29.645232+0000","last_fullsized":"2024-04-21T08:36:29.645232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:56:25.969905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.995497+0000","last_change":"2024-04-21T08:36:28.007370+0000","last_active":"2024-04-21T08:36:28.995497+0000","last_peered":"2024-04-21T08:36:28.995497+0000","last_clean":"2024-04-21T08:36:28.995497+0000","last_became_active":"2024-04-21T08:36:28.006972+0000","last_became_peered":"2024-04-21T08:36:28.006972+0000","last_unstale":"2024-04-21T08:36:28.995497+0000","last_undegraded":"2024-04-21T08:36:28.995497+0000","last_fullsized":"2024-04-21T08:36:28.995497+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:18:14.926058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:50.904775+0000","last_change":"2024-04-21T08:36:23.472658+0000","last_active":"2024-04-21T08:36:50.904775+0000","last_peered":"2024-04-21T08:36:50.904775+0000","last_clean":"2024-04-21T08:36:50.904775+0000","last_became_active":"2024-04-21T08:36:23.472318+0000","last_became_peered":"2024-04-21T08:36:23.472318+0000","last_unstale":"2024-04-21T08:36:50.904775+0000","last_undegraded":"2024-04-21T08:36:50.904775+0000","last_fullsized":"2024-04-21T08:36:50.904775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:28:27.497605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.991865+0000","last_change":"2024-04-21T08:36:23.986897+0000","last_active":"2024-04-21T08:36:28.991865+0000","last_peered":"2024-04-21T08:36:28.991865+0000","last_clean":"2024-04-21T08:36:28.991865+0000","last_became_active":"2024-04-21T08:36:23.986554+0000","last_became_peered":"2024-04-21T08:36:23.986554+0000","last_unstale":"2024-04-21T08:36:28.991865+0000","last_undegraded":"2024-04-21T08:36:28.991865+0000","last_fullsized":"2024-04-21T08:36:28.991865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:44:22.331207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.506740+0000","last_change":"2024-04-21T08:36:28.019186+0000","last_active":"2024-04-21T08:36:29.506740+0000","last_peered":"2024-04-21T08:36:29.506740+0000","last_clean":"2024-04-21T08:36:29.506740+0000","last_became_active":"2024-04-21T08:36:28.018774+0000","last_became_peered":"2024-04-21T08:36:28.018774+0000","last_unstale":"2024-04-21T08:36:29.506740+0000","last_undegraded":"2024-04-21T08:36:29.506740+0000","last_fullsized":"2024-04-21T08:36:29.506740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:35:33.548656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:49.890074+0000","last_change":"2024-04-21T08:36:23.472633+0000","last_active":"2024-04-21T08:36:49.890074+0000","last_peered":"2024-04-21T08:36:49.890074+0000","last_clean":"2024-04-21T08:36:49.890074+0000","last_became_active":"2024-04-21T08:36:23.472336+0000","last_became_peered":"2024-04-21T08:36:23.472336+0000","last_unstale":"2024-04-21T08:36:49.890074+0000","last_undegraded":"2024-04-21T08:36:49.890074+0000","last_fullsized":"2024-04-21T08:36:49.890074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:49.463492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992633+0000","last_change":"2024-04-21T08:36:23.994332+0000","last_active":"2024-04-21T08:36:28.992633+0000","last_peered":"2024-04-21T08:36:28.992633+0000","last_clean":"2024-04-21T08:36:28.992633+0000","last_became_active":"2024-04-21T08:36:23.993936+0000","last_became_peered":"2024-04-21T08:36:23.993936+0000","last_unstale":"2024-04-21T08:36:28.992633+0000","last_undegraded":"2024-04-21T08:36:28.992633+0000","last_fullsized":"2024-04-21T08:36:28.992633+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:26:35.330559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645888+0000","last_change":"2024-04-21T08:36:28.012549+0000","last_active":"2024-04-21T08:36:29.645888+0000","last_peered":"2024-04-21T08:36:29.645888+0000","last_clean":"2024-04-21T08:36:29.645888+0000","last_became_active":"2024-04-21T08:36:28.012151+0000","last_became_peered":"2024-04-21T08:36:28.012151+0000","last_unstale":"2024-04-21T08:36:29.645888+0000","last_undegraded":"2024-04-21T08:36:29.645888+0000","last_fullsized":"2024-04-21T08:36:29.645888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:55.680399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:49.069358+0000","last_change":"2024-04-21T08:36:23.464586+0000","last_active":"2024-04-21T08:36:49.069358+0000","last_peered":"2024-04-21T08:36:49.069358+0000","last_clean":"2024-04-21T08:36:49.069358+0000","last_became_active":"2024-04-21T08:36:23.464281+0000","last_became_peered":"2024-04-21T08:36:23.464281+0000","last_unstale":"2024-04-21T08:36:49.069358+0000","last_undegraded":"2024-04-21T08:36:49.069358+0000","last_fullsized":"2024-04-21T08:36:49.069358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:04:58.577283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992476+0000","last_change":"2024-04-21T08:36:23.989604+0000","last_active":"2024-04-21T08:36:28.992476+0000","last_peered":"2024-04-21T08:36:28.992476+0000","last_clean":"2024-04-21T08:36:28.992476+0000","last_became_active":"2024-04-21T08:36:23.989257+0000","last_became_peered":"2024-04-21T08:36:23.989257+0000","last_unstale":"2024-04-21T08:36:28.992476+0000","last_undegraded":"2024-04-21T08:36:28.992476+0000","last_fullsized":"2024-04-21T08:36:28.992476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:45.077943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.991866+0000","last_change":"2024-04-21T08:36:28.007959+0000","last_active":"2024-04-21T08:36:28.991866+0000","last_peered":"2024-04-21T08:36:28.991866+0000","last_clean":"2024-04-21T08:36:28.991866+0000","last_became_active":"2024-04-21T08:36:28.007571+0000","last_became_peered":"2024-04-21T08:36:28.007571+0000","last_unstale":"2024-04-21T08:36:28.991866+0000","last_undegraded":"2024-04-21T08:36:28.991866+0000","last_fullsized":"2024-04-21T08:36:28.991866+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:17:34.007013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:49.070282+0000","last_change":"2024-04-21T08:36:23.475054+0000","last_active":"2024-04-21T08:36:49.070282+0000","last_peered":"2024-04-21T08:36:49.070282+0000","last_clean":"2024-04-21T08:36:49.070282+0000","last_became_active":"2024-04-21T08:36:23.474606+0000","last_became_peered":"2024-04-21T08:36:23.474606+0000","last_unstale":"2024-04-21T08:36:49.070282+0000","last_undegraded":"2024-04-21T08:36:49.070282+0000","last_fullsized":"2024-04-21T08:36:49.070282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:22.559174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.994442+0000","last_change":"2024-04-21T08:36:23.995980+0000","last_active":"2024-04-21T08:36:28.994442+0000","last_peered":"2024-04-21T08:36:28.994442+0000","last_clean":"2024-04-21T08:36:28.994442+0000","last_became_active":"2024-04-21T08:36:23.995673+0000","last_became_peered":"2024-04-21T08:36:23.995673+0000","last_unstale":"2024-04-21T08:36:28.994442+0000","last_undegraded":"2024-04-21T08:36:28.994442+0000","last_fullsized":"2024-04-21T08:36:28.994442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:11:49.884384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.994565+0000","last_change":"2024-04-21T08:36:28.018792+0000","last_active":"2024-04-21T08:36:28.994565+0000","last_peered":"2024-04-21T08:36:28.994565+0000","last_clean":"2024-04-21T08:36:28.994565+0000","last_became_active":"2024-04-21T08:36:28.018373+0000","last_became_peered":"2024-04-21T08:36:28.018373+0000","last_unstale":"2024-04-21T08:36:28.994565+0000","last_undegraded":"2024-04-21T08:36:28.994565+0000","last_fullsized":"2024-04-21T08:36:28.994565+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:11.857661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644928+0000","last_change":"2024-04-21T08:36:23.469787+0000","last_active":"2024-04-21T08:36:29.644928+0000","last_peered":"2024-04-21T08:36:29.644928+0000","last_clean":"2024-04-21T08:36:29.644928+0000","last_became_active":"2024-04-21T08:36:23.469462+0000","last_became_peered":"2024-04-21T08:36:23.469462+0000","last_unstale":"2024-04-21T08:36:29.644928+0000","last_undegraded":"2024-04-21T08:36:29.644928+0000","last_fullsized":"2024-04-21T08:36:29.644928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:33:25.849528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.506828+0000","last_change":"2024-04-21T08:36:23.988492+0000","last_active":"2024-04-21T08:36:29.506828+0000","last_peered":"2024-04-21T08:36:29.506828+0000","last_clean":"2024-04-21T08:36:29.506828+0000","last_became_active":"2024-04-21T08:36:23.988133+0000","last_became_peered":"2024-04-21T08:36:23.988133+0000","last_unstale":"2024-04-21T08:36:29.506828+0000","last_undegraded":"2024-04-21T08:36:29.506828+0000","last_fullsized":"2024-04-21T08:36:29.506828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:01:25.320281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644889+0000","last_change":"2024-04-21T08:36:28.012454+0000","last_active":"2024-04-21T08:36:29.644889+0000","last_peered":"2024-04-21T08:36:29.644889+0000","last_clean":"2024-04-21T08:36:29.644889+0000","last_became_active":"2024-04-21T08:36:28.011985+0000","last_became_peered":"2024-04-21T08:36:28.011985+0000","last_unstale":"2024-04-21T08:36:29.644889+0000","last_undegraded":"2024-04-21T08:36:29.644889+0000","last_fullsized":"2024-04-21T08:36:29.644889+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:16:51.939691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645655+0000","last_change":"2024-04-21T08:36:23.470403+0000","last_active":"2024-04-21T08:36:29.645655+0000","last_peered":"2024-04-21T08:36:29.645655+0000","last_clean":"2024-04-21T08:36:29.645655+0000","last_became_active":"2024-04-21T08:36:23.470035+0000","last_became_peered":"2024-04-21T08:36:23.470035+0000","last_unstale":"2024-04-21T08:36:29.645655+0000","last_undegraded":"2024-04-21T08:36:29.645655+0000","last_fullsized":"2024-04-21T08:36:29.645655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:36:17.395660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645402+0000","last_change":"2024-04-21T08:36:23.985910+0000","last_active":"2024-04-21T08:36:29.645402+0000","last_peered":"2024-04-21T08:36:29.645402+0000","last_clean":"2024-04-21T08:36:29.645402+0000","last_became_active":"2024-04-21T08:36:23.985605+0000","last_became_peered":"2024-04-21T08:36:23.985605+0000","last_unstale":"2024-04-21T08:36:29.645402+0000","last_undegraded":"2024-04-21T08:36:29.645402+0000","last_fullsized":"2024-04-21T08:36:29.645402+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:37:09.367005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.991970+0000","last_change":"2024-04-21T08:36:28.015256+0000","last_active":"2024-04-21T08:36:28.991970+0000","last_peered":"2024-04-21T08:36:28.991970+0000","last_clean":"2024-04-21T08:36:28.991970+0000","last_became_active":"2024-04-21T08:36:28.014849+0000","last_became_peered":"2024-04-21T08:36:28.014849+0000","last_unstale":"2024-04-21T08:36:28.991970+0000","last_undegraded":"2024-04-21T08:36:28.991970+0000","last_fullsized":"2024-04-21T08:36:28.991970+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:22:05.466299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992928+0000","last_change":"2024-04-21T08:36:23.474966+0000","last_active":"2024-04-21T08:36:28.992928+0000","last_peered":"2024-04-21T08:36:28.992928+0000","last_clean":"2024-04-21T08:36:28.992928+0000","last_became_active":"2024-04-21T08:36:23.474374+0000","last_became_peered":"2024-04-21T08:36:23.474374+0000","last_unstale":"2024-04-21T08:36:28.992928+0000","last_undegraded":"2024-04-21T08:36:28.992928+0000","last_fullsized":"2024-04-21T08:36:28.992928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:57:20.237966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645137+0000","last_change":"2024-04-21T08:36:23.986879+0000","last_active":"2024-04-21T08:36:29.645137+0000","last_peered":"2024-04-21T08:36:29.645137+0000","last_clean":"2024-04-21T08:36:29.645137+0000","last_became_active":"2024-04-21T08:36:23.986552+0000","last_became_peered":"2024-04-21T08:36:23.986552+0000","last_unstale":"2024-04-21T08:36:29.645137+0000","last_undegraded":"2024-04-21T08:36:29.645137+0000","last_fullsized":"2024-04-21T08:36:29.645137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:24:14.325345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992057+0000","last_change":"2024-04-21T08:36:28.013169+0000","last_active":"2024-04-21T08:36:28.992057+0000","last_peered":"2024-04-21T08:36:28.992057+0000","last_clean":"2024-04-21T08:36:28.992057+0000","last_became_active":"2024-04-21T08:36:28.012791+0000","last_became_peered":"2024-04-21T08:36:28.012791+0000","last_unstale":"2024-04-21T08:36:28.992057+0000","last_undegraded":"2024-04-21T08:36:28.992057+0000","last_fullsized":"2024-04-21T08:36:28.992057+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:42:56.063159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.994739+0000","last_change":"2024-04-21T08:36:23.995209+0000","last_active":"2024-04-21T08:36:28.994739+0000","last_peered":"2024-04-21T08:36:28.994739+0000","last_clean":"2024-04-21T08:36:28.994739+0000","last_became_active":"2024-04-21T08:36:23.994866+0000","last_became_peered":"2024-04-21T08:36:23.994866+0000","last_unstale":"2024-04-21T08:36:28.994739+0000","last_undegraded":"2024-04-21T08:36:28.994739+0000","last_fullsized":"2024-04-21T08:36:28.994739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:33:45.582150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.991351+0000","last_change":"2024-04-21T08:36:23.471474+0000","last_active":"2024-04-21T08:36:28.991351+0000","last_peered":"2024-04-21T08:36:28.991351+0000","last_clean":"2024-04-21T08:36:28.991351+0000","last_became_active":"2024-04-21T08:36:23.471179+0000","last_became_peered":"2024-04-21T08:36:23.471179+0000","last_unstale":"2024-04-21T08:36:28.991351+0000","last_undegraded":"2024-04-21T08:36:28.991351+0000","last_fullsized":"2024-04-21T08:36:28.991351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:17.290051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992730+0000","last_change":"2024-04-21T08:36:28.009671+0000","last_active":"2024-04-21T08:36:28.992730+0000","last_peered":"2024-04-21T08:36:28.992730+0000","last_clean":"2024-04-21T08:36:28.992730+0000","last_became_active":"2024-04-21T08:36:28.009185+0000","last_became_peered":"2024-04-21T08:36:28.009185+0000","last_unstale":"2024-04-21T08:36:28.992730+0000","last_undegraded":"2024-04-21T08:36:28.992730+0000","last_fullsized":"2024-04-21T08:36:28.992730+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:11:05.156271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644926+0000","last_change":"2024-04-21T08:36:23.990272+0000","last_active":"2024-04-21T08:36:29.644926+0000","last_peered":"2024-04-21T08:36:29.644926+0000","last_clean":"2024-04-21T08:36:29.644926+0000","last_became_active":"2024-04-21T08:36:23.989940+0000","last_became_peered":"2024-04-21T08:36:23.989940+0000","last_unstale":"2024-04-21T08:36:29.644926+0000","last_undegraded":"2024-04-21T08:36:29.644926+0000","last_fullsized":"2024-04-21T08:36:29.644926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:27:50.705167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:48.062707+0000","last_change":"2024-04-21T08:36:23.474162+0000","last_active":"2024-04-21T08:36:48.062707+0000","last_peered":"2024-04-21T08:36:48.062707+0000","last_clean":"2024-04-21T08:36:48.062707+0000","last_became_active":"2024-04-21T08:36:23.473802+0000","last_became_peered":"2024-04-21T08:36:23.473802+0000","last_unstale":"2024-04-21T08:36:48.062707+0000","last_undegraded":"2024-04-21T08:36:48.062707+0000","last_fullsized":"2024-04-21T08:36:48.062707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:32:41.918431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":355,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992282+0000","last_change":"2024-04-21T08:35:10.062130+0000","last_active":"2024-04-21T08:36:28.992282+0000","last_peered":"2024-04-21T08:36:28.992282+0000","last_clean":"2024-04-21T08:36:28.992282+0000","last_became_active":"2024-04-21T08:35:10.061727+0000","last_became_peered":"2024-04-21T08:35:10.061727+0000","last_unstale":"2024-04-21T08:36:28.992282+0000","last_undegraded":"2024-04-21T08:36:28.992282+0000","last_fullsized":"2024-04-21T08:36:28.992282+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:31:40.950939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:31:40.950939+0000","last_clean_scrub_stamp":"2024-04-21T08:31:40.950939+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:35:52.360167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.995143+0000","last_change":"2024-04-21T08:36:28.012840+0000","last_active":"2024-04-21T08:36:28.995143+0000","last_peered":"2024-04-21T08:36:28.995143+0000","last_clean":"2024-04-21T08:36:28.995143+0000","last_became_active":"2024-04-21T08:36:28.012439+0000","last_became_peered":"2024-04-21T08:36:28.012439+0000","last_unstale":"2024-04-21T08:36:28.995143+0000","last_undegraded":"2024-04-21T08:36:28.995143+0000","last_fullsized":"2024-04-21T08:36:28.995143+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:36:39.260061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.991261+0000","last_change":"2024-04-21T08:36:23.984332+0000","last_active":"2024-04-21T08:36:28.991261+0000","last_peered":"2024-04-21T08:36:28.991261+0000","last_clean":"2024-04-21T08:36:28.991261+0000","last_became_active":"2024-04-21T08:36:23.984021+0000","last_became_peered":"2024-04-21T08:36:23.984021+0000","last_unstale":"2024-04-21T08:36:28.991261+0000","last_undegraded":"2024-04-21T08:36:28.991261+0000","last_fullsized":"2024-04-21T08:36:28.991261+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:05:21.090633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992569+0000","last_change":"2024-04-21T08:36:23.467522+0000","last_active":"2024-04-21T08:36:28.992569+0000","last_peered":"2024-04-21T08:36:28.992569+0000","last_clean":"2024-04-21T08:36:28.992569+0000","last_became_active":"2024-04-21T08:36:23.466379+0000","last_became_peered":"2024-04-21T08:36:23.466379+0000","last_unstale":"2024-04-21T08:36:28.992569+0000","last_undegraded":"2024-04-21T08:36:28.992569+0000","last_fullsized":"2024-04-21T08:36:28.992569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:04.185314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.506545+0000","last_change":"2024-04-21T08:36:28.015120+0000","last_active":"2024-04-21T08:36:29.506545+0000","last_peered":"2024-04-21T08:36:29.506545+0000","last_clean":"2024-04-21T08:36:29.506545+0000","last_became_active":"2024-04-21T08:36:28.014701+0000","last_became_peered":"2024-04-21T08:36:28.014701+0000","last_unstale":"2024-04-21T08:36:29.506545+0000","last_undegraded":"2024-04-21T08:36:29.506545+0000","last_fullsized":"2024-04-21T08:36:29.506545+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:23.127622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645322+0000","last_change":"2024-04-21T08:36:23.475335+0000","last_active":"2024-04-21T08:36:29.645322+0000","last_peered":"2024-04-21T08:36:29.645322+0000","last_clean":"2024-04-21T08:36:29.645322+0000","last_became_active":"2024-04-21T08:36:23.474739+0000","last_became_peered":"2024-04-21T08:36:23.474739+0000","last_unstale":"2024-04-21T08:36:29.645322+0000","last_undegraded":"2024-04-21T08:36:29.645322+0000","last_fullsized":"2024-04-21T08:36:29.645322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:36.119649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.991960+0000","last_change":"2024-04-21T08:36:23.977229+0000","last_active":"2024-04-21T08:36:28.991960+0000","last_peered":"2024-04-21T08:36:28.991960+0000","last_clean":"2024-04-21T08:36:28.991960+0000","last_became_active":"2024-04-21T08:36:23.976893+0000","last_became_peered":"2024-04-21T08:36:23.976893+0000","last_unstale":"2024-04-21T08:36:28.991960+0000","last_undegraded":"2024-04-21T08:36:28.991960+0000","last_fullsized":"2024-04-21T08:36:28.991960+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:31:54.226274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644806+0000","last_change":"2024-04-21T08:36:28.018581+0000","last_active":"2024-04-21T08:36:29.644806+0000","last_peered":"2024-04-21T08:36:29.644806+0000","last_clean":"2024-04-21T08:36:29.644806+0000","last_became_active":"2024-04-21T08:36:28.018021+0000","last_became_peered":"2024-04-21T08:36:28.018021+0000","last_unstale":"2024-04-21T08:36:29.644806+0000","last_undegraded":"2024-04-21T08:36:29.644806+0000","last_fullsized":"2024-04-21T08:36:29.644806+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:29:00.355532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'8","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:49.893410+0000","last_change":"2024-04-21T08:36:23.473339+0000","last_active":"2024-04-21T08:36:49.893410+0000","last_peered":"2024-04-21T08:36:49.893410+0000","last_clean":"2024-04-21T08:36:49.893410+0000","last_became_active":"2024-04-21T08:36:23.472994+0000","last_became_peered":"2024-04-21T08:36:23.472994+0000","last_unstale":"2024-04-21T08:36:49.893410+0000","last_undegraded":"2024-04-21T08:36:49.893410+0000","last_fullsized":"2024-04-21T08:36:49.893410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:42.762273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645598+0000","last_change":"2024-04-21T08:36:23.987354+0000","last_active":"2024-04-21T08:36:29.645598+0000","last_peered":"2024-04-21T08:36:29.645598+0000","last_clean":"2024-04-21T08:36:29.645598+0000","last_became_active":"2024-04-21T08:36:23.987032+0000","last_became_peered":"2024-04-21T08:36:23.987032+0000","last_unstale":"2024-04-21T08:36:29.645598+0000","last_undegraded":"2024-04-21T08:36:29.645598+0000","last_fullsized":"2024-04-21T08:36:29.645598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:01:05.164367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645079+0000","last_change":"2024-04-21T08:36:28.002582+0000","last_active":"2024-04-21T08:36:29.645079+0000","last_peered":"2024-04-21T08:36:29.645079+0000","last_clean":"2024-04-21T08:36:29.645079+0000","last_became_active":"2024-04-21T08:36:28.002133+0000","last_became_peered":"2024-04-21T08:36:28.002133+0000","last_unstale":"2024-04-21T08:36:29.645079+0000","last_undegraded":"2024-04-21T08:36:29.645079+0000","last_fullsized":"2024-04-21T08:36:29.645079+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:30:00.268015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:48.062755+0000","last_change":"2024-04-21T08:36:23.462840+0000","last_active":"2024-04-21T08:36:48.062755+0000","last_peered":"2024-04-21T08:36:48.062755+0000","last_clean":"2024-04-21T08:36:48.062755+0000","last_became_active":"2024-04-21T08:36:23.462391+0000","last_became_peered":"2024-04-21T08:36:23.462391+0000","last_unstale":"2024-04-21T08:36:48.062755+0000","last_undegraded":"2024-04-21T08:36:48.062755+0000","last_fullsized":"2024-04-21T08:36:48.062755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:33.230388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.643766+0000","last_change":"2024-04-21T08:36:23.994751+0000","last_active":"2024-04-21T08:36:29.643766+0000","last_peered":"2024-04-21T08:36:29.643766+0000","last_clean":"2024-04-21T08:36:29.643766+0000","last_became_active":"2024-04-21T08:36:23.994384+0000","last_became_peered":"2024-04-21T08:36:23.994384+0000","last_unstale":"2024-04-21T08:36:29.643766+0000","last_undegraded":"2024-04-21T08:36:29.643766+0000","last_fullsized":"2024-04-21T08:36:29.643766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:18.956893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645100+0000","last_change":"2024-04-21T08:36:28.004004+0000","last_active":"2024-04-21T08:36:29.645100+0000","last_peered":"2024-04-21T08:36:29.645100+0000","last_clean":"2024-04-21T08:36:29.645100+0000","last_became_active":"2024-04-21T08:36:28.003547+0000","last_became_peered":"2024-04-21T08:36:28.003547+0000","last_unstale":"2024-04-21T08:36:29.645100+0000","last_undegraded":"2024-04-21T08:36:29.645100+0000","last_fullsized":"2024-04-21T08:36:29.645100+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:21:40.215428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:48.063775+0000","last_change":"2024-04-21T08:36:23.468528+0000","last_active":"2024-04-21T08:36:48.063775+0000","last_peered":"2024-04-21T08:36:48.063775+0000","last_clean":"2024-04-21T08:36:48.063775+0000","last_became_active":"2024-04-21T08:36:23.467838+0000","last_became_peered":"2024-04-21T08:36:23.467838+0000","last_unstale":"2024-04-21T08:36:48.063775+0000","last_undegraded":"2024-04-21T08:36:48.063775+0000","last_fullsized":"2024-04-21T08:36:48.063775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:03:44.901930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644152+0000","last_change":"2024-04-21T08:36:23.988020+0000","last_active":"2024-04-21T08:36:29.644152+0000","last_peered":"2024-04-21T08:36:29.644152+0000","last_clean":"2024-04-21T08:36:29.644152+0000","last_became_active":"2024-04-21T08:36:23.987630+0000","last_became_peered":"2024-04-21T08:36:23.987630+0000","last_unstale":"2024-04-21T08:36:29.644152+0000","last_undegraded":"2024-04-21T08:36:29.644152+0000","last_fullsized":"2024-04-21T08:36:29.644152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:06.519001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.646042+0000","last_change":"2024-04-21T08:36:28.006984+0000","last_active":"2024-04-21T08:36:29.646042+0000","last_peered":"2024-04-21T08:36:29.646042+0000","last_clean":"2024-04-21T08:36:29.646042+0000","last_became_active":"2024-04-21T08:36:28.006548+0000","last_became_peered":"2024-04-21T08:36:28.006548+0000","last_unstale":"2024-04-21T08:36:29.646042+0000","last_undegraded":"2024-04-21T08:36:29.646042+0000","last_fullsized":"2024-04-21T08:36:29.646042+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:25:50.870658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:49.889752+0000","last_change":"2024-04-21T08:36:23.476266+0000","last_active":"2024-04-21T08:36:49.889752+0000","last_peered":"2024-04-21T08:36:49.889752+0000","last_clean":"2024-04-21T08:36:49.889752+0000","last_became_active":"2024-04-21T08:36:23.475948+0000","last_became_peered":"2024-04-21T08:36:23.475948+0000","last_unstale":"2024-04-21T08:36:49.889752+0000","last_undegraded":"2024-04-21T08:36:49.889752+0000","last_fullsized":"2024-04-21T08:36:49.889752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:41.529376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.506350+0000","last_change":"2024-04-21T08:36:23.981942+0000","last_active":"2024-04-21T08:36:29.506350+0000","last_peered":"2024-04-21T08:36:29.506350+0000","last_clean":"2024-04-21T08:36:29.506350+0000","last_became_active":"2024-04-21T08:36:23.981643+0000","last_became_peered":"2024-04-21T08:36:23.981643+0000","last_unstale":"2024-04-21T08:36:29.506350+0000","last_undegraded":"2024-04-21T08:36:29.506350+0000","last_fullsized":"2024-04-21T08:36:29.506350+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:26:24.722695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644817+0000","last_change":"2024-04-21T08:36:28.006980+0000","last_active":"2024-04-21T08:36:29.644817+0000","last_peered":"2024-04-21T08:36:29.644817+0000","last_clean":"2024-04-21T08:36:29.644817+0000","last_became_active":"2024-04-21T08:36:28.006530+0000","last_became_peered":"2024-04-21T08:36:28.006530+0000","last_unstale":"2024-04-21T08:36:29.644817+0000","last_undegraded":"2024-04-21T08:36:29.644817+0000","last_fullsized":"2024-04-21T08:36:29.644817+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:05:57.465612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:49.058714+0000","last_change":"2024-04-21T08:36:23.473253+0000","last_active":"2024-04-21T08:36:49.058714+0000","last_peered":"2024-04-21T08:36:49.058714+0000","last_clean":"2024-04-21T08:36:49.058714+0000","last_became_active":"2024-04-21T08:36:23.472942+0000","last_became_peered":"2024-04-21T08:36:23.472942+0000","last_unstale":"2024-04-21T08:36:49.058714+0000","last_undegraded":"2024-04-21T08:36:49.058714+0000","last_fullsized":"2024-04-21T08:36:49.058714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:32:04.894388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645383+0000","last_change":"2024-04-21T08:36:23.988355+0000","last_active":"2024-04-21T08:36:29.645383+0000","last_peered":"2024-04-21T08:36:29.645383+0000","last_clean":"2024-04-21T08:36:29.645383+0000","last_became_active":"2024-04-21T08:36:23.988049+0000","last_became_peered":"2024-04-21T08:36:23.988049+0000","last_unstale":"2024-04-21T08:36:29.645383+0000","last_undegraded":"2024-04-21T08:36:29.645383+0000","last_fullsized":"2024-04-21T08:36:29.645383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:57.348698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644979+0000","last_change":"2024-04-21T08:36:28.007343+0000","last_active":"2024-04-21T08:36:29.644979+0000","last_peered":"2024-04-21T08:36:29.644979+0000","last_clean":"2024-04-21T08:36:29.644979+0000","last_became_active":"2024-04-21T08:36:28.006719+0000","last_became_peered":"2024-04-21T08:36:28.006719+0000","last_unstale":"2024-04-21T08:36:29.644979+0000","last_undegraded":"2024-04-21T08:36:29.644979+0000","last_fullsized":"2024-04-21T08:36:29.644979+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:02:25.734584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:48.062571+0000","last_change":"2024-04-21T08:36:23.472416+0000","last_active":"2024-04-21T08:36:48.062571+0000","last_peered":"2024-04-21T08:36:48.062571+0000","last_clean":"2024-04-21T08:36:48.062571+0000","last_became_active":"2024-04-21T08:36:23.471500+0000","last_became_peered":"2024-04-21T08:36:23.471500+0000","last_unstale":"2024-04-21T08:36:48.062571+0000","last_undegraded":"2024-04-21T08:36:48.062571+0000","last_fullsized":"2024-04-21T08:36:48.062571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:39.706432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992554+0000","last_change":"2024-04-21T08:36:23.990653+0000","last_active":"2024-04-21T08:36:28.992554+0000","last_peered":"2024-04-21T08:36:28.992554+0000","last_clean":"2024-04-21T08:36:28.992554+0000","last_became_active":"2024-04-21T08:36:23.990324+0000","last_became_peered":"2024-04-21T08:36:23.990324+0000","last_unstale":"2024-04-21T08:36:28.992554+0000","last_undegraded":"2024-04-21T08:36:28.992554+0000","last_fullsized":"2024-04-21T08:36:28.992554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:08:59.417850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644619+0000","last_change":"2024-04-21T08:36:28.007956+0000","last_active":"2024-04-21T08:36:29.644619+0000","last_peered":"2024-04-21T08:36:29.644619+0000","last_clean":"2024-04-21T08:36:29.644619+0000","last_became_active":"2024-04-21T08:36:28.007475+0000","last_became_peered":"2024-04-21T08:36:28.007475+0000","last_unstale":"2024-04-21T08:36:29.644619+0000","last_undegraded":"2024-04-21T08:36:29.644619+0000","last_fullsized":"2024-04-21T08:36:29.644619+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:54.039359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:49.894247+0000","last_change":"2024-04-21T08:36:23.476887+0000","last_active":"2024-04-21T08:36:49.894247+0000","last_peered":"2024-04-21T08:36:49.894247+0000","last_clean":"2024-04-21T08:36:49.894247+0000","last_became_active":"2024-04-21T08:36:23.476574+0000","last_became_peered":"2024-04-21T08:36:23.476574+0000","last_unstale":"2024-04-21T08:36:49.894247+0000","last_undegraded":"2024-04-21T08:36:49.894247+0000","last_fullsized":"2024-04-21T08:36:49.894247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:37:47.692358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992902+0000","last_change":"2024-04-21T08:36:23.982845+0000","last_active":"2024-04-21T08:36:28.992902+0000","last_peered":"2024-04-21T08:36:28.992902+0000","last_clean":"2024-04-21T08:36:28.992902+0000","last_became_active":"2024-04-21T08:36:23.982491+0000","last_became_peered":"2024-04-21T08:36:23.982491+0000","last_unstale":"2024-04-21T08:36:28.992902+0000","last_undegraded":"2024-04-21T08:36:28.992902+0000","last_fullsized":"2024-04-21T08:36:28.992902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:48:46.413896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.643762+0000","last_change":"2024-04-21T08:36:28.010442+0000","last_active":"2024-04-21T08:36:29.643762+0000","last_peered":"2024-04-21T08:36:29.643762+0000","last_clean":"2024-04-21T08:36:29.643762+0000","last_became_active":"2024-04-21T08:36:28.010046+0000","last_became_peered":"2024-04-21T08:36:28.010046+0000","last_unstale":"2024-04-21T08:36:29.643762+0000","last_undegraded":"2024-04-21T08:36:29.643762+0000","last_fullsized":"2024-04-21T08:36:29.643762+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:18:36.382802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:49.891675+0000","last_change":"2024-04-21T08:36:23.473545+0000","last_active":"2024-04-21T08:36:49.891675+0000","last_peered":"2024-04-21T08:36:49.891675+0000","last_clean":"2024-04-21T08:36:49.891675+0000","last_became_active":"2024-04-21T08:36:23.473222+0000","last_became_peered":"2024-04-21T08:36:23.473222+0000","last_unstale":"2024-04-21T08:36:49.891675+0000","last_undegraded":"2024-04-21T08:36:49.891675+0000","last_fullsized":"2024-04-21T08:36:49.891675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:47:53.947187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992544+0000","last_change":"2024-04-21T08:36:23.990959+0000","last_active":"2024-04-21T08:36:28.992544+0000","last_peered":"2024-04-21T08:36:28.992544+0000","last_clean":"2024-04-21T08:36:28.992544+0000","last_became_active":"2024-04-21T08:36:23.990322+0000","last_became_peered":"2024-04-21T08:36:23.990322+0000","last_unstale":"2024-04-21T08:36:28.992544+0000","last_undegraded":"2024-04-21T08:36:28.992544+0000","last_fullsized":"2024-04-21T08:36:28.992544+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:12:20.859224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.994454+0000","last_change":"2024-04-21T08:36:28.006682+0000","last_active":"2024-04-21T08:36:28.994454+0000","last_peered":"2024-04-21T08:36:28.994454+0000","last_clean":"2024-04-21T08:36:28.994454+0000","last_became_active":"2024-04-21T08:36:28.006229+0000","last_became_peered":"2024-04-21T08:36:28.006229+0000","last_unstale":"2024-04-21T08:36:28.994454+0000","last_undegraded":"2024-04-21T08:36:28.994454+0000","last_fullsized":"2024-04-21T08:36:28.994454+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:30:12.723848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.991834+0000","last_change":"2024-04-21T08:36:23.473592+0000","last_active":"2024-04-21T08:36:28.991834+0000","last_peered":"2024-04-21T08:36:28.991834+0000","last_clean":"2024-04-21T08:36:28.991834+0000","last_became_active":"2024-04-21T08:36:23.473283+0000","last_became_peered":"2024-04-21T08:36:23.473283+0000","last_unstale":"2024-04-21T08:36:28.991834+0000","last_undegraded":"2024-04-21T08:36:28.991834+0000","last_fullsized":"2024-04-21T08:36:28.991834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:10.425249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992949+0000","last_change":"2024-04-21T08:36:23.985052+0000","last_active":"2024-04-21T08:36:28.992949+0000","last_peered":"2024-04-21T08:36:28.992949+0000","last_clean":"2024-04-21T08:36:28.992949+0000","last_became_active":"2024-04-21T08:36:23.984742+0000","last_became_peered":"2024-04-21T08:36:23.984742+0000","last_unstale":"2024-04-21T08:36:28.992949+0000","last_undegraded":"2024-04-21T08:36:28.992949+0000","last_fullsized":"2024-04-21T08:36:28.992949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:08:06.082974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.506488+0000","last_change":"2024-04-21T08:36:23.980905+0000","last_active":"2024-04-21T08:36:29.506488+0000","last_peered":"2024-04-21T08:36:29.506488+0000","last_clean":"2024-04-21T08:36:29.506488+0000","last_became_active":"2024-04-21T08:36:23.980590+0000","last_became_peered":"2024-04-21T08:36:23.980590+0000","last_unstale":"2024-04-21T08:36:29.506488+0000","last_undegraded":"2024-04-21T08:36:29.506488+0000","last_fullsized":"2024-04-21T08:36:29.506488+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:54:59.886014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644549+0000","last_change":"2024-04-21T08:36:23.980883+0000","last_active":"2024-04-21T08:36:29.644549+0000","last_peered":"2024-04-21T08:36:29.644549+0000","last_clean":"2024-04-21T08:36:29.644549+0000","last_became_active":"2024-04-21T08:36:23.980562+0000","last_became_peered":"2024-04-21T08:36:23.980562+0000","last_unstale":"2024-04-21T08:36:29.644549+0000","last_undegraded":"2024-04-21T08:36:29.644549+0000","last_fullsized":"2024-04-21T08:36:29.644549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:14:51.868438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644224+0000","last_change":"2024-04-21T08:36:23.991571+0000","last_active":"2024-04-21T08:36:29.644224+0000","last_peered":"2024-04-21T08:36:29.644224+0000","last_clean":"2024-04-21T08:36:29.644224+0000","last_became_active":"2024-04-21T08:36:23.991273+0000","last_became_peered":"2024-04-21T08:36:23.991273+0000","last_unstale":"2024-04-21T08:36:29.644224+0000","last_undegraded":"2024-04-21T08:36:29.644224+0000","last_fullsized":"2024-04-21T08:36:29.644224+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:08:23.895038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.643975+0000","last_change":"2024-04-21T08:36:23.994729+0000","last_active":"2024-04-21T08:36:29.643975+0000","last_peered":"2024-04-21T08:36:29.643975+0000","last_clean":"2024-04-21T08:36:29.643975+0000","last_became_active":"2024-04-21T08:36:23.994405+0000","last_became_peered":"2024-04-21T08:36:23.994405+0000","last_unstale":"2024-04-21T08:36:29.643975+0000","last_undegraded":"2024-04-21T08:36:29.643975+0000","last_fullsized":"2024-04-21T08:36:29.643975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:42.463225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.991724+0000","last_change":"2024-04-21T08:36:23.995421+0000","last_active":"2024-04-21T08:36:28.991724+0000","last_peered":"2024-04-21T08:36:28.991724+0000","last_clean":"2024-04-21T08:36:28.991724+0000","last_became_active":"2024-04-21T08:36:23.993674+0000","last_became_peered":"2024-04-21T08:36:23.993674+0000","last_unstale":"2024-04-21T08:36:28.991724+0000","last_undegraded":"2024-04-21T08:36:28.991724+0000","last_fullsized":"2024-04-21T08:36:28.991724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:31.013874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.506757+0000","last_change":"2024-04-21T08:36:23.986418+0000","last_active":"2024-04-21T08:36:29.506757+0000","last_peered":"2024-04-21T08:36:29.506757+0000","last_clean":"2024-04-21T08:36:29.506757+0000","last_became_active":"2024-04-21T08:36:23.986095+0000","last_became_peered":"2024-04-21T08:36:23.986095+0000","last_unstale":"2024-04-21T08:36:29.506757+0000","last_undegraded":"2024-04-21T08:36:29.506757+0000","last_fullsized":"2024-04-21T08:36:29.506757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:31:56.456218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644890+0000","last_change":"2024-04-21T08:36:23.982353+0000","last_active":"2024-04-21T08:36:29.644890+0000","last_peered":"2024-04-21T08:36:29.644890+0000","last_clean":"2024-04-21T08:36:29.644890+0000","last_became_active":"2024-04-21T08:36:23.982035+0000","last_became_peered":"2024-04-21T08:36:23.982035+0000","last_unstale":"2024-04-21T08:36:29.644890+0000","last_undegraded":"2024-04-21T08:36:29.644890+0000","last_fullsized":"2024-04-21T08:36:29.644890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:42:44.598407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645856+0000","last_change":"2024-04-21T08:36:23.996371+0000","last_active":"2024-04-21T08:36:29.645856+0000","last_peered":"2024-04-21T08:36:29.645856+0000","last_clean":"2024-04-21T08:36:29.645856+0000","last_became_active":"2024-04-21T08:36:23.995802+0000","last_became_peered":"2024-04-21T08:36:23.995802+0000","last_unstale":"2024-04-21T08:36:29.645856+0000","last_undegraded":"2024-04-21T08:36:29.645856+0000","last_fullsized":"2024-04-21T08:36:29.645856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:52:08.571567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.994538+0000","last_change":"2024-04-21T08:36:23.987275+0000","last_active":"2024-04-21T08:36:28.994538+0000","last_peered":"2024-04-21T08:36:28.994538+0000","last_clean":"2024-04-21T08:36:28.994538+0000","last_became_active":"2024-04-21T08:36:23.986967+0000","last_became_peered":"2024-04-21T08:36:23.986967+0000","last_unstale":"2024-04-21T08:36:28.994538+0000","last_undegraded":"2024-04-21T08:36:28.994538+0000","last_fullsized":"2024-04-21T08:36:28.994538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:50.432502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645254+0000","last_change":"2024-04-21T08:36:23.993046+0000","last_active":"2024-04-21T08:36:29.645254+0000","last_peered":"2024-04-21T08:36:29.645254+0000","last_clean":"2024-04-21T08:36:29.645254+0000","last_became_active":"2024-04-21T08:36:23.992693+0000","last_became_peered":"2024-04-21T08:36:23.992693+0000","last_unstale":"2024-04-21T08:36:29.645254+0000","last_undegraded":"2024-04-21T08:36:29.645254+0000","last_fullsized":"2024-04-21T08:36:29.645254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:12:06.287585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644962+0000","last_change":"2024-04-21T08:36:23.992968+0000","last_active":"2024-04-21T08:36:29.644962+0000","last_peered":"2024-04-21T08:36:29.644962+0000","last_clean":"2024-04-21T08:36:29.644962+0000","last_became_active":"2024-04-21T08:36:23.992614+0000","last_became_peered":"2024-04-21T08:36:23.992614+0000","last_unstale":"2024-04-21T08:36:29.644962+0000","last_undegraded":"2024-04-21T08:36:29.644962+0000","last_fullsized":"2024-04-21T08:36:29.644962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:10.015323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.991739+0000","last_change":"2024-04-21T08:36:23.981169+0000","last_active":"2024-04-21T08:36:28.991739+0000","last_peered":"2024-04-21T08:36:28.991739+0000","last_clean":"2024-04-21T08:36:28.991739+0000","last_became_active":"2024-04-21T08:36:23.980849+0000","last_became_peered":"2024-04-21T08:36:23.980849+0000","last_unstale":"2024-04-21T08:36:28.991739+0000","last_undegraded":"2024-04-21T08:36:28.991739+0000","last_fullsized":"2024-04-21T08:36:28.991739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:13:49.907370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.506592+0000","last_change":"2024-04-21T08:36:23.993998+0000","last_active":"2024-04-21T08:36:29.506592+0000","last_peered":"2024-04-21T08:36:29.506592+0000","last_clean":"2024-04-21T08:36:29.506592+0000","last_became_active":"2024-04-21T08:36:23.993664+0000","last_became_peered":"2024-04-21T08:36:23.993664+0000","last_unstale":"2024-04-21T08:36:29.506592+0000","last_undegraded":"2024-04-21T08:36:29.506592+0000","last_fullsized":"2024-04-21T08:36:29.506592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:31:49.076820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9438,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":135,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":315392,"data_stored":52021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124,"ondisk_log_size":124,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":481709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612631,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32752,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712400,"statfs":{"total":95995035648,"available":95961497600,"internally_reserved":0,"allocated":1613824,"data_stored":1076113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776157,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":486927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037794,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":482335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299430,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1593344,"data_stored":1072659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462955,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":995328,"data_stored":481709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":493903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":484278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":487523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":482117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":482353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724594,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32772,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712380,"statfs":{"total":95995035648,"available":95961477120,"internally_reserved":0,"allocated":1634304,"data_stored":1075288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":24893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":13196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:36:53.399 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T08:36:53.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:53 smithi175 ceph-mon[31920]: pgmap v309: 129 pgs: 129 active+clean; 586 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-21T08:36:53.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:53 smithi175 ceph-mon[31920]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:53.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:53 smithi175 ceph-mon[31920]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:53.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:53 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1674830096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:53.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:53 smithi175 ceph-mon[31920]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-21T08:36:53.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:53 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1410514650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.536+0000 7fd95f904640 1 -- 172.21.15.43:0/1476032311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd958103cb0 msgr2=0x7fd95810a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.536+0000 7fd95f904640 1 --2- 172.21.15.43:0/1476032311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd958103cb0 0x7fd95810a540 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd950009cd0 tx=0x7fd95002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.536+0000 7fd95f904640 1 -- 172.21.15.43:0/1476032311 shutdown_connections 2024-04-21T08:36:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.536+0000 7fd95f904640 1 --2- 172.21.15.43:0/1476032311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd958103cb0 0x7fd95810a540 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.536+0000 7fd95f904640 1 --2- 172.21.15.43:0/1476032311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9581032f0 0x7fd958103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.536+0000 7fd95f904640 1 --2- 172.21.15.43:0/1476032311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd958100280 0x7fd9581006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.536+0000 7fd95f904640 1 -- 172.21.15.43:0/1476032311 >> 172.21.15.43:0/1476032311 conn(0x7fd9580fbb90 msgr2=0x7fd9580fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.536+0000 7fd95f904640 1 -- 172.21.15.43:0/1476032311 shutdown_connections 2024-04-21T08:36:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.536+0000 7fd95f904640 1 -- 172.21.15.43:0/1476032311 wait complete. 2024-04-21T08:36:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.537+0000 7fd95f904640 1 Processor -- start 2024-04-21T08:36:53.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.537+0000 7fd95f904640 1 -- start start 2024-04-21T08:36:53.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.537+0000 7fd95f904640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd958100280 0x7fd958102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:53.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.537+0000 7fd95f904640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9581032f0 0x7fd9581012a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:53.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.537+0000 7fd95f904640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd958103cb0 0x7fd9581017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:53.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.537+0000 7fd95f904640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd95810daf0 con 0x7fd958100280 2024-04-21T08:36:53.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.537+0000 7fd95f904640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd95810d970 con 0x7fd9581032f0 2024-04-21T08:36:53.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.537+0000 7fd95f904640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd95810dc70 con 0x7fd958103cb0 2024-04-21T08:36:53.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.537+0000 7fd95d6a0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd958100280 0x7fd958102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:53.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.537+0000 7fd95d6a0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd958100280 0x7fd958102c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35654/0 (socket says 172.21.15.43:35654) 2024-04-21T08:36:53.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.537+0000 7fd95d6a0640 1 -- 172.21.15.43:0/966458231 learned_addr learned my addr 172.21.15.43:0/966458231 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:53.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.537+0000 7fd95dea1640 1 --2- 172.21.15.43:0/966458231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd958103cb0 0x7fd9581017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:53.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.537+0000 7fd95ce9f640 1 --2- 172.21.15.43:0/966458231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9581032f0 0x7fd9581012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:53.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.537+0000 7fd95d6a0640 1 -- 172.21.15.43:0/966458231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd958103cb0 msgr2=0x7fd9581017e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:53.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.537+0000 7fd95d6a0640 1 --2- 172.21.15.43:0/966458231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd958103cb0 0x7fd9581017e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.537+0000 7fd95d6a0640 1 -- 172.21.15.43:0/966458231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9581032f0 msgr2=0x7fd9581012a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:53.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.537+0000 7fd95d6a0640 1 --2- 172.21.15.43:0/966458231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9581032f0 0x7fd9581012a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.537+0000 7fd95d6a0640 1 -- 172.21.15.43:0/966458231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd958102070 con 0x7fd958100280 2024-04-21T08:36:53.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.537+0000 7fd95ce9f640 1 --2- 172.21.15.43:0/966458231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9581032f0 0x7fd9581012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:53.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.538+0000 7fd95d6a0640 1 --2- 172.21.15.43:0/966458231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd958100280 0x7fd958102c60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd95400e9c0 tx=0x7fd95400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:53.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.538+0000 7fd94e7fc640 1 -- 172.21.15.43:0/966458231 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95400cd80 con 0x7fd958100280 2024-04-21T08:36:53.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.538+0000 7fd94e7fc640 1 -- 172.21.15.43:0/966458231 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9540035f0 con 0x7fd958100280 2024-04-21T08:36:53.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.538+0000 7fd94e7fc640 1 -- 172.21.15.43:0/966458231 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd954010690 con 0x7fd958100280 2024-04-21T08:36:53.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.538+0000 7fd95f904640 1 -- 172.21.15.43:0/966458231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9581a2290 con 0x7fd958100280 2024-04-21T08:36:53.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.538+0000 7fd95f904640 1 -- 172.21.15.43:0/966458231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9581a2790 con 0x7fd958100280 2024-04-21T08:36:53.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.539+0000 7fd95f904640 1 -- 172.21.15.43:0/966458231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9581094a0 con 0x7fd958100280 2024-04-21T08:36:53.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.540+0000 7fd94e7fc640 1 -- 172.21.15.43:0/966458231 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd954004000 con 0x7fd958100280 2024-04-21T08:36:53.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.541+0000 7fd94e7fc640 1 --2- 172.21.15.43:0/966458231 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd9440a3c40 0x7fd9440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:53.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.541+0000 7fd95ce9f640 1 --2- 172.21.15.43:0/966458231 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd9440a3c40 0x7fd9440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:53.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.541+0000 7fd94e7fc640 1 -- 172.21.15.43:0/966458231 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fd9540d0e00 con 0x7fd958100280 2024-04-21T08:36:53.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.541+0000 7fd95ce9f640 1 --2- 172.21.15.43:0/966458231 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd9440a3c40 0x7fd9440a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd9480058c0 tx=0x7fd948005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:53.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.545+0000 7fd94e7fc640 1 -- 172.21.15.43:0/966458231 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd95409d440 con 0x7fd958100280 2024-04-21T08:36:53.549 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:53 smithi112 ceph-mon[31085]: pgmap v309: 129 pgs: 129 active+clean; 586 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-21T08:36:53.549 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:53 smithi112 ceph-mon[31085]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:53.549 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:53 smithi112 ceph-mon[31085]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:53.549 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:53 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1674830096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:53.549 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:53 smithi112 ceph-mon[31085]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-21T08:36:53.549 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:53 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1410514650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:53.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.728+0000 7fd95f904640 1 -- 172.21.15.43:0/966458231 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd958062390 con 0x7fd9440a3c40 2024-04-21T08:36:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.737+0000 7fd94e7fc640 1 -- 172.21.15.43:0/966458231 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340313 (secure 0 0 0) 0x7fd958062390 con 0x7fd9440a3c40 2024-04-21T08:36:53.743 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:53.745 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-21T08:36:53.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.740+0000 7fd95f904640 1 -- 172.21.15.43:0/966458231 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd9440a3c40 msgr2=0x7fd9440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:53.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.740+0000 7fd95f904640 1 --2- 172.21.15.43:0/966458231 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd9440a3c40 0x7fd9440a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd9480058c0 tx=0x7fd948005850 comp rx=0 tx=0).stop 2024-04-21T08:36:53.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.740+0000 7fd95f904640 1 -- 172.21.15.43:0/966458231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd958100280 msgr2=0x7fd958102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:53.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.740+0000 7fd95f904640 1 --2- 172.21.15.43:0/966458231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd958100280 0x7fd958102c60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd95400e9c0 tx=0x7fd95400ee80 comp rx=0 tx=0).stop 2024-04-21T08:36:53.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.741+0000 7fd95f904640 1 -- 172.21.15.43:0/966458231 shutdown_connections 2024-04-21T08:36:53.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.741+0000 7fd95f904640 1 --2- 172.21.15.43:0/966458231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd958103cb0 0x7fd9581017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.741+0000 7fd95f904640 1 --2- 172.21.15.43:0/966458231 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd9440a3c40 0x7fd9440a6100 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.741+0000 7fd95f904640 1 --2- 172.21.15.43:0/966458231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9581032f0 0x7fd9581012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.741+0000 7fd95f904640 1 --2- 172.21.15.43:0/966458231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd958100280 0x7fd958102c60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.741+0000 7fd95f904640 1 -- 172.21.15.43:0/966458231 >> 172.21.15.43:0/966458231 conn(0x7fd9580fbb90 msgr2=0x7fd958106670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:53.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.741+0000 7fd95f904640 1 -- 172.21.15.43:0/966458231 shutdown_connections 2024-04-21T08:36:53.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:53.741+0000 7fd95f904640 1 -- 172.21.15.43:0/966458231 wait complete. 2024-04-21T08:36:53.752 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2024-04-21T08:36:51.981193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":599806,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":316,"num_write_kb":2993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":261,"ondisk_log_size":261,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379488,"kb_used_data":13680,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562336,"statfs":{"total":1151940427776,"available":1151551832064,"internally_reserved":0,"allocated":14008320,"data_stored":7586914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":374453564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4939,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":88,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003622"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.506924+0000","last_change":"2024-04-21T08:36:23.983921+0000","last_active":"2024-04-21T08:36:29.506924+0000","last_peered":"2024-04-21T08:36:29.506924+0000","last_clean":"2024-04-21T08:36:29.506924+0000","last_became_active":"2024-04-21T08:36:23.983638+0000","last_became_peered":"2024-04-21T08:36:23.983638+0000","last_unstale":"2024-04-21T08:36:29.506924+0000","last_undegraded":"2024-04-21T08:36:29.506924+0000","last_fullsized":"2024-04-21T08:36:29.506924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:29:44.646308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.643913+0000","last_change":"2024-04-21T08:36:23.990029+0000","last_active":"2024-04-21T08:36:29.643913+0000","last_peered":"2024-04-21T08:36:29.643913+0000","last_clean":"2024-04-21T08:36:29.643913+0000","last_became_active":"2024-04-21T08:36:23.989141+0000","last_became_peered":"2024-04-21T08:36:23.989141+0000","last_unstale":"2024-04-21T08:36:29.643913+0000","last_undegraded":"2024-04-21T08:36:29.643913+0000","last_fullsized":"2024-04-21T08:36:29.643913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:23:58.692342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.643897+0000","last_change":"2024-04-21T08:36:23.982205+0000","last_active":"2024-04-21T08:36:29.643897+0000","last_peered":"2024-04-21T08:36:29.643897+0000","last_clean":"2024-04-21T08:36:29.643897+0000","last_became_active":"2024-04-21T08:36:23.981876+0000","last_became_peered":"2024-04-21T08:36:23.981876+0000","last_unstale":"2024-04-21T08:36:29.643897+0000","last_undegraded":"2024-04-21T08:36:29.643897+0000","last_fullsized":"2024-04-21T08:36:29.643897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:12:27.121106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645335+0000","last_change":"2024-04-21T08:36:23.987425+0000","last_active":"2024-04-21T08:36:29.645335+0000","last_peered":"2024-04-21T08:36:29.645335+0000","last_clean":"2024-04-21T08:36:29.645335+0000","last_became_active":"2024-04-21T08:36:23.987109+0000","last_became_peered":"2024-04-21T08:36:23.987109+0000","last_unstale":"2024-04-21T08:36:29.645335+0000","last_undegraded":"2024-04-21T08:36:29.645335+0000","last_fullsized":"2024-04-21T08:36:29.645335+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:44.176324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644476+0000","last_change":"2024-04-21T08:36:23.990375+0000","last_active":"2024-04-21T08:36:29.644476+0000","last_peered":"2024-04-21T08:36:29.644476+0000","last_clean":"2024-04-21T08:36:29.644476+0000","last_became_active":"2024-04-21T08:36:23.990058+0000","last_became_peered":"2024-04-21T08:36:23.990058+0000","last_unstale":"2024-04-21T08:36:29.644476+0000","last_undegraded":"2024-04-21T08:36:29.644476+0000","last_fullsized":"2024-04-21T08:36:29.644476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:24:09.647528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644732+0000","last_change":"2024-04-21T08:36:23.979848+0000","last_active":"2024-04-21T08:36:29.644732+0000","last_peered":"2024-04-21T08:36:29.644732+0000","last_clean":"2024-04-21T08:36:29.644732+0000","last_became_active":"2024-04-21T08:36:23.979511+0000","last_became_peered":"2024-04-21T08:36:23.979511+0000","last_unstale":"2024-04-21T08:36:29.644732+0000","last_undegraded":"2024-04-21T08:36:29.644732+0000","last_fullsized":"2024-04-21T08:36:29.644732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:07:46.551369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992965+0000","last_change":"2024-04-21T08:36:23.985353+0000","last_active":"2024-04-21T08:36:28.992965+0000","last_peered":"2024-04-21T08:36:28.992965+0000","last_clean":"2024-04-21T08:36:28.992965+0000","last_became_active":"2024-04-21T08:36:23.984754+0000","last_became_peered":"2024-04-21T08:36:23.984754+0000","last_unstale":"2024-04-21T08:36:28.992965+0000","last_undegraded":"2024-04-21T08:36:28.992965+0000","last_fullsized":"2024-04-21T08:36:28.992965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:19:25.900776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644547+0000","last_change":"2024-04-21T08:36:23.987517+0000","last_active":"2024-04-21T08:36:29.644547+0000","last_peered":"2024-04-21T08:36:29.644547+0000","last_clean":"2024-04-21T08:36:29.644547+0000","last_became_active":"2024-04-21T08:36:23.987201+0000","last_became_peered":"2024-04-21T08:36:23.987201+0000","last_unstale":"2024-04-21T08:36:29.644547+0000","last_undegraded":"2024-04-21T08:36:29.644547+0000","last_fullsized":"2024-04-21T08:36:29.644547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:42:53.080863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645884+0000","last_change":"2024-04-21T08:36:23.989434+0000","last_active":"2024-04-21T08:36:29.645884+0000","last_peered":"2024-04-21T08:36:29.645884+0000","last_clean":"2024-04-21T08:36:29.645884+0000","last_became_active":"2024-04-21T08:36:23.989125+0000","last_became_peered":"2024-04-21T08:36:23.989125+0000","last_unstale":"2024-04-21T08:36:29.645884+0000","last_undegraded":"2024-04-21T08:36:29.645884+0000","last_fullsized":"2024-04-21T08:36:29.645884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:30:07.219056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645987+0000","last_change":"2024-04-21T08:36:23.980725+0000","last_active":"2024-04-21T08:36:29.645987+0000","last_peered":"2024-04-21T08:36:29.645987+0000","last_clean":"2024-04-21T08:36:29.645987+0000","last_became_active":"2024-04-21T08:36:23.980363+0000","last_became_peered":"2024-04-21T08:36:23.980363+0000","last_unstale":"2024-04-21T08:36:29.645987+0000","last_undegraded":"2024-04-21T08:36:29.645987+0000","last_fullsized":"2024-04-21T08:36:29.645987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:26:46.202030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992860+0000","last_change":"2024-04-21T08:36:23.989369+0000","last_active":"2024-04-21T08:36:28.992860+0000","last_peered":"2024-04-21T08:36:28.992860+0000","last_clean":"2024-04-21T08:36:28.992860+0000","last_became_active":"2024-04-21T08:36:23.989030+0000","last_became_peered":"2024-04-21T08:36:23.989030+0000","last_unstale":"2024-04-21T08:36:28.992860+0000","last_undegraded":"2024-04-21T08:36:28.992860+0000","last_fullsized":"2024-04-21T08:36:28.992860+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:23:13.725703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644985+0000","last_change":"2024-04-21T08:36:23.993485+0000","last_active":"2024-04-21T08:36:29.644985+0000","last_peered":"2024-04-21T08:36:29.644985+0000","last_clean":"2024-04-21T08:36:29.644985+0000","last_became_active":"2024-04-21T08:36:23.993124+0000","last_became_peered":"2024-04-21T08:36:23.993124+0000","last_unstale":"2024-04-21T08:36:29.644985+0000","last_undegraded":"2024-04-21T08:36:29.644985+0000","last_fullsized":"2024-04-21T08:36:29.644985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:53:16.249838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.991991+0000","last_change":"2024-04-21T08:36:23.983146+0000","last_active":"2024-04-21T08:36:28.991991+0000","last_peered":"2024-04-21T08:36:28.991991+0000","last_clean":"2024-04-21T08:36:28.991991+0000","last_became_active":"2024-04-21T08:36:23.982766+0000","last_became_peered":"2024-04-21T08:36:23.982766+0000","last_unstale":"2024-04-21T08:36:28.991991+0000","last_undegraded":"2024-04-21T08:36:28.991991+0000","last_fullsized":"2024-04-21T08:36:28.991991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:30:44.090929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644576+0000","last_change":"2024-04-21T08:36:23.988906+0000","last_active":"2024-04-21T08:36:29.644576+0000","last_peered":"2024-04-21T08:36:29.644576+0000","last_clean":"2024-04-21T08:36:29.644576+0000","last_became_active":"2024-04-21T08:36:23.988489+0000","last_became_peered":"2024-04-21T08:36:23.988489+0000","last_unstale":"2024-04-21T08:36:29.644576+0000","last_undegraded":"2024-04-21T08:36:29.644576+0000","last_fullsized":"2024-04-21T08:36:29.644576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:06:43.271650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.643779+0000","last_change":"2024-04-21T08:36:23.986112+0000","last_active":"2024-04-21T08:36:29.643779+0000","last_peered":"2024-04-21T08:36:29.643779+0000","last_clean":"2024-04-21T08:36:29.643779+0000","last_became_active":"2024-04-21T08:36:23.985801+0000","last_became_peered":"2024-04-21T08:36:23.985801+0000","last_unstale":"2024-04-21T08:36:29.643779+0000","last_undegraded":"2024-04-21T08:36:29.643779+0000","last_fullsized":"2024-04-21T08:36:29.643779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:28:45.793440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644227+0000","last_change":"2024-04-21T08:36:23.983136+0000","last_active":"2024-04-21T08:36:29.644227+0000","last_peered":"2024-04-21T08:36:29.644227+0000","last_clean":"2024-04-21T08:36:29.644227+0000","last_became_active":"2024-04-21T08:36:23.982698+0000","last_became_peered":"2024-04-21T08:36:23.982698+0000","last_unstale":"2024-04-21T08:36:29.644227+0000","last_undegraded":"2024-04-21T08:36:29.644227+0000","last_fullsized":"2024-04-21T08:36:29.644227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:34.750941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.506713+0000","last_change":"2024-04-21T08:36:23.984701+0000","last_active":"2024-04-21T08:36:29.506713+0000","last_peered":"2024-04-21T08:36:29.506713+0000","last_clean":"2024-04-21T08:36:29.506713+0000","last_became_active":"2024-04-21T08:36:23.984432+0000","last_became_peered":"2024-04-21T08:36:23.984432+0000","last_unstale":"2024-04-21T08:36:29.506713+0000","last_undegraded":"2024-04-21T08:36:29.506713+0000","last_fullsized":"2024-04-21T08:36:29.506713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:10:58.944380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645930+0000","last_change":"2024-04-21T08:36:23.982781+0000","last_active":"2024-04-21T08:36:29.645930+0000","last_peered":"2024-04-21T08:36:29.645930+0000","last_clean":"2024-04-21T08:36:29.645930+0000","last_became_active":"2024-04-21T08:36:23.982459+0000","last_became_peered":"2024-04-21T08:36:23.982459+0000","last_unstale":"2024-04-21T08:36:29.645930+0000","last_undegraded":"2024-04-21T08:36:29.645930+0000","last_fullsized":"2024-04-21T08:36:29.645930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:44.657504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644800+0000","last_change":"2024-04-21T08:36:23.992002+0000","last_active":"2024-04-21T08:36:29.644800+0000","last_peered":"2024-04-21T08:36:29.644800+0000","last_clean":"2024-04-21T08:36:29.644800+0000","last_became_active":"2024-04-21T08:36:23.991053+0000","last_became_peered":"2024-04-21T08:36:23.991053+0000","last_unstale":"2024-04-21T08:36:29.644800+0000","last_undegraded":"2024-04-21T08:36:29.644800+0000","last_fullsized":"2024-04-21T08:36:29.644800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:56:37.145949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644395+0000","last_change":"2024-04-21T08:36:28.013195+0000","last_active":"2024-04-21T08:36:29.644395+0000","last_peered":"2024-04-21T08:36:29.644395+0000","last_clean":"2024-04-21T08:36:29.644395+0000","last_became_active":"2024-04-21T08:36:28.012656+0000","last_became_peered":"2024-04-21T08:36:28.012656+0000","last_unstale":"2024-04-21T08:36:29.644395+0000","last_undegraded":"2024-04-21T08:36:29.644395+0000","last_fullsized":"2024-04-21T08:36:29.644395+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:12:03.133583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:50.907118+0000","last_change":"2024-04-21T08:36:23.468957+0000","last_active":"2024-04-21T08:36:50.907118+0000","last_peered":"2024-04-21T08:36:50.907118+0000","last_clean":"2024-04-21T08:36:50.907118+0000","last_became_active":"2024-04-21T08:36:23.468601+0000","last_became_peered":"2024-04-21T08:36:23.468601+0000","last_unstale":"2024-04-21T08:36:50.907118+0000","last_undegraded":"2024-04-21T08:36:50.907118+0000","last_fullsized":"2024-04-21T08:36:50.907118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:13:11.977780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645525+0000","last_change":"2024-04-21T08:36:23.993520+0000","last_active":"2024-04-21T08:36:29.645525+0000","last_peered":"2024-04-21T08:36:29.645525+0000","last_clean":"2024-04-21T08:36:29.645525+0000","last_became_active":"2024-04-21T08:36:23.993169+0000","last_became_peered":"2024-04-21T08:36:23.993169+0000","last_unstale":"2024-04-21T08:36:29.645525+0000","last_undegraded":"2024-04-21T08:36:29.645525+0000","last_fullsized":"2024-04-21T08:36:29.645525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:26:28.047696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644631+0000","last_change":"2024-04-21T08:36:28.014161+0000","last_active":"2024-04-21T08:36:29.644631+0000","last_peered":"2024-04-21T08:36:29.644631+0000","last_clean":"2024-04-21T08:36:29.644631+0000","last_became_active":"2024-04-21T08:36:28.013859+0000","last_became_peered":"2024-04-21T08:36:28.013859+0000","last_unstale":"2024-04-21T08:36:29.644631+0000","last_undegraded":"2024-04-21T08:36:29.644631+0000","last_fullsized":"2024-04-21T08:36:29.644631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:08.315334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645773+0000","last_change":"2024-04-21T08:36:23.475612+0000","last_active":"2024-04-21T08:36:29.645773+0000","last_peered":"2024-04-21T08:36:29.645773+0000","last_clean":"2024-04-21T08:36:29.645773+0000","last_became_active":"2024-04-21T08:36:23.475301+0000","last_became_peered":"2024-04-21T08:36:23.475301+0000","last_unstale":"2024-04-21T08:36:29.645773+0000","last_undegraded":"2024-04-21T08:36:29.645773+0000","last_fullsized":"2024-04-21T08:36:29.645773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:56.801630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644079+0000","last_change":"2024-04-21T08:36:23.984689+0000","last_active":"2024-04-21T08:36:29.644079+0000","last_peered":"2024-04-21T08:36:29.644079+0000","last_clean":"2024-04-21T08:36:29.644079+0000","last_became_active":"2024-04-21T08:36:23.984358+0000","last_became_peered":"2024-04-21T08:36:23.984358+0000","last_unstale":"2024-04-21T08:36:29.644079+0000","last_undegraded":"2024-04-21T08:36:29.644079+0000","last_fullsized":"2024-04-21T08:36:29.644079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:49:50.187466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644832+0000","last_change":"2024-04-21T08:36:28.007951+0000","last_active":"2024-04-21T08:36:29.644832+0000","last_peered":"2024-04-21T08:36:29.644832+0000","last_clean":"2024-04-21T08:36:29.644832+0000","last_became_active":"2024-04-21T08:36:28.007553+0000","last_became_peered":"2024-04-21T08:36:28.007553+0000","last_unstale":"2024-04-21T08:36:29.644832+0000","last_undegraded":"2024-04-21T08:36:29.644832+0000","last_fullsized":"2024-04-21T08:36:29.644832+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:10:54.019605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:49.069870+0000","last_change":"2024-04-21T08:36:23.473371+0000","last_active":"2024-04-21T08:36:49.069870+0000","last_peered":"2024-04-21T08:36:49.069870+0000","last_clean":"2024-04-21T08:36:49.069870+0000","last_became_active":"2024-04-21T08:36:23.472472+0000","last_became_peered":"2024-04-21T08:36:23.472472+0000","last_unstale":"2024-04-21T08:36:49.069870+0000","last_undegraded":"2024-04-21T08:36:49.069870+0000","last_fullsized":"2024-04-21T08:36:49.069870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:28:49.622873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645353+0000","last_change":"2024-04-21T08:36:23.987916+0000","last_active":"2024-04-21T08:36:29.645353+0000","last_peered":"2024-04-21T08:36:29.645353+0000","last_clean":"2024-04-21T08:36:29.645353+0000","last_became_active":"2024-04-21T08:36:23.986207+0000","last_became_peered":"2024-04-21T08:36:23.986207+0000","last_unstale":"2024-04-21T08:36:29.645353+0000","last_undegraded":"2024-04-21T08:36:29.645353+0000","last_fullsized":"2024-04-21T08:36:29.645353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:49:57.261078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644682+0000","last_change":"2024-04-21T08:36:28.006494+0000","last_active":"2024-04-21T08:36:29.644682+0000","last_peered":"2024-04-21T08:36:29.644682+0000","last_clean":"2024-04-21T08:36:29.644682+0000","last_became_active":"2024-04-21T08:36:28.006069+0000","last_became_peered":"2024-04-21T08:36:28.006069+0000","last_unstale":"2024-04-21T08:36:29.644682+0000","last_undegraded":"2024-04-21T08:36:29.644682+0000","last_fullsized":"2024-04-21T08:36:29.644682+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:04.608295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645118+0000","last_change":"2024-04-21T08:36:23.468923+0000","last_active":"2024-04-21T08:36:29.645118+0000","last_peered":"2024-04-21T08:36:29.645118+0000","last_clean":"2024-04-21T08:36:29.645118+0000","last_became_active":"2024-04-21T08:36:23.468509+0000","last_became_peered":"2024-04-21T08:36:23.468509+0000","last_unstale":"2024-04-21T08:36:29.645118+0000","last_undegraded":"2024-04-21T08:36:29.645118+0000","last_fullsized":"2024-04-21T08:36:29.645118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:44.452748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.995078+0000","last_change":"2024-04-21T08:36:23.994423+0000","last_active":"2024-04-21T08:36:28.995078+0000","last_peered":"2024-04-21T08:36:28.995078+0000","last_clean":"2024-04-21T08:36:28.995078+0000","last_became_active":"2024-04-21T08:36:23.993827+0000","last_became_peered":"2024-04-21T08:36:23.993827+0000","last_unstale":"2024-04-21T08:36:28.995078+0000","last_undegraded":"2024-04-21T08:36:28.995078+0000","last_fullsized":"2024-04-21T08:36:28.995078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:49:25.550301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644795+0000","last_change":"2024-04-21T08:36:28.004037+0000","last_active":"2024-04-21T08:36:29.644795+0000","last_peered":"2024-04-21T08:36:29.644795+0000","last_clean":"2024-04-21T08:36:29.644795+0000","last_became_active":"2024-04-21T08:36:28.003580+0000","last_became_peered":"2024-04-21T08:36:28.003580+0000","last_unstale":"2024-04-21T08:36:29.644795+0000","last_undegraded":"2024-04-21T08:36:29.644795+0000","last_fullsized":"2024-04-21T08:36:29.644795+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:04:51.380244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:50.884881+0000","last_change":"2024-04-21T08:36:23.468430+0000","last_active":"2024-04-21T08:36:50.884881+0000","last_peered":"2024-04-21T08:36:50.884881+0000","last_clean":"2024-04-21T08:36:50.884881+0000","last_became_active":"2024-04-21T08:36:23.468089+0000","last_became_peered":"2024-04-21T08:36:23.468089+0000","last_unstale":"2024-04-21T08:36:50.884881+0000","last_undegraded":"2024-04-21T08:36:50.884881+0000","last_fullsized":"2024-04-21T08:36:50.884881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:12:54.827034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644971+0000","last_change":"2024-04-21T08:36:23.985693+0000","last_active":"2024-04-21T08:36:29.644971+0000","last_peered":"2024-04-21T08:36:29.644971+0000","last_clean":"2024-04-21T08:36:29.644971+0000","last_became_active":"2024-04-21T08:36:23.985373+0000","last_became_peered":"2024-04-21T08:36:23.985373+0000","last_unstale":"2024-04-21T08:36:29.644971+0000","last_undegraded":"2024-04-21T08:36:29.644971+0000","last_fullsized":"2024-04-21T08:36:29.644971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:49:42.045542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644776+0000","last_change":"2024-04-21T08:36:28.004638+0000","last_active":"2024-04-21T08:36:29.644776+0000","last_peered":"2024-04-21T08:36:29.644776+0000","last_clean":"2024-04-21T08:36:29.644776+0000","last_became_active":"2024-04-21T08:36:28.002499+0000","last_became_peered":"2024-04-21T08:36:28.002499+0000","last_unstale":"2024-04-21T08:36:29.644776+0000","last_undegraded":"2024-04-21T08:36:29.644776+0000","last_fullsized":"2024-04-21T08:36:29.644776+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:45:46.934783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644598+0000","last_change":"2024-04-21T08:36:23.465783+0000","last_active":"2024-04-21T08:36:29.644598+0000","last_peered":"2024-04-21T08:36:29.644598+0000","last_clean":"2024-04-21T08:36:29.644598+0000","last_became_active":"2024-04-21T08:36:23.465478+0000","last_became_peered":"2024-04-21T08:36:23.465478+0000","last_unstale":"2024-04-21T08:36:29.644598+0000","last_undegraded":"2024-04-21T08:36:29.644598+0000","last_fullsized":"2024-04-21T08:36:29.644598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:56:06.275800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.994787+0000","last_change":"2024-04-21T08:36:23.995407+0000","last_active":"2024-04-21T08:36:28.994787+0000","last_peered":"2024-04-21T08:36:28.994787+0000","last_clean":"2024-04-21T08:36:28.994787+0000","last_became_active":"2024-04-21T08:36:23.995102+0000","last_became_peered":"2024-04-21T08:36:23.995102+0000","last_unstale":"2024-04-21T08:36:28.994787+0000","last_undegraded":"2024-04-21T08:36:28.994787+0000","last_fullsized":"2024-04-21T08:36:28.994787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:03:42.306628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.506803+0000","last_change":"2024-04-21T08:36:28.017068+0000","last_active":"2024-04-21T08:36:29.506803+0000","last_peered":"2024-04-21T08:36:29.506803+0000","last_clean":"2024-04-21T08:36:29.506803+0000","last_became_active":"2024-04-21T08:36:28.016770+0000","last_became_peered":"2024-04-21T08:36:28.016770+0000","last_unstale":"2024-04-21T08:36:29.506803+0000","last_undegraded":"2024-04-21T08:36:29.506803+0000","last_fullsized":"2024-04-21T08:36:29.506803+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:07:14.596090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.991419+0000","last_change":"2024-04-21T08:36:23.470430+0000","last_active":"2024-04-21T08:36:28.991419+0000","last_peered":"2024-04-21T08:36:28.991419+0000","last_clean":"2024-04-21T08:36:28.991419+0000","last_became_active":"2024-04-21T08:36:23.470109+0000","last_became_peered":"2024-04-21T08:36:23.470109+0000","last_unstale":"2024-04-21T08:36:28.991419+0000","last_undegraded":"2024-04-21T08:36:28.991419+0000","last_fullsized":"2024-04-21T08:36:28.991419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:18.176313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644710+0000","last_change":"2024-04-21T08:36:23.986668+0000","last_active":"2024-04-21T08:36:29.644710+0000","last_peered":"2024-04-21T08:36:29.644710+0000","last_clean":"2024-04-21T08:36:29.644710+0000","last_became_active":"2024-04-21T08:36:23.986344+0000","last_became_peered":"2024-04-21T08:36:23.986344+0000","last_unstale":"2024-04-21T08:36:29.644710+0000","last_undegraded":"2024-04-21T08:36:29.644710+0000","last_fullsized":"2024-04-21T08:36:29.644710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:56:17.744145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.643936+0000","last_change":"2024-04-21T08:36:28.009012+0000","last_active":"2024-04-21T08:36:29.643936+0000","last_peered":"2024-04-21T08:36:29.643936+0000","last_clean":"2024-04-21T08:36:29.643936+0000","last_became_active":"2024-04-21T08:36:28.008123+0000","last_became_peered":"2024-04-21T08:36:28.008123+0000","last_unstale":"2024-04-21T08:36:29.643936+0000","last_undegraded":"2024-04-21T08:36:29.643936+0000","last_fullsized":"2024-04-21T08:36:29.643936+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:33:18.881564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:49.890296+0000","last_change":"2024-04-21T08:36:23.467104+0000","last_active":"2024-04-21T08:36:49.890296+0000","last_peered":"2024-04-21T08:36:49.890296+0000","last_clean":"2024-04-21T08:36:49.890296+0000","last_became_active":"2024-04-21T08:36:23.466725+0000","last_became_peered":"2024-04-21T08:36:23.466725+0000","last_unstale":"2024-04-21T08:36:49.890296+0000","last_undegraded":"2024-04-21T08:36:49.890296+0000","last_fullsized":"2024-04-21T08:36:49.890296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:35:45.986448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644698+0000","last_change":"2024-04-21T08:36:23.987579+0000","last_active":"2024-04-21T08:36:29.644698+0000","last_peered":"2024-04-21T08:36:29.644698+0000","last_clean":"2024-04-21T08:36:29.644698+0000","last_became_active":"2024-04-21T08:36:23.987044+0000","last_became_peered":"2024-04-21T08:36:23.987044+0000","last_unstale":"2024-04-21T08:36:29.644698+0000","last_undegraded":"2024-04-21T08:36:29.644698+0000","last_fullsized":"2024-04-21T08:36:29.644698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:48:27.593090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645100+0000","last_change":"2024-04-21T08:36:28.006417+0000","last_active":"2024-04-21T08:36:29.645100+0000","last_peered":"2024-04-21T08:36:29.645100+0000","last_clean":"2024-04-21T08:36:29.645100+0000","last_became_active":"2024-04-21T08:36:28.006006+0000","last_became_peered":"2024-04-21T08:36:28.006006+0000","last_unstale":"2024-04-21T08:36:29.645100+0000","last_undegraded":"2024-04-21T08:36:29.645100+0000","last_fullsized":"2024-04-21T08:36:29.645100+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:16.173284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.646025+0000","last_change":"2024-04-21T08:36:23.471541+0000","last_active":"2024-04-21T08:36:29.646025+0000","last_peered":"2024-04-21T08:36:29.646025+0000","last_clean":"2024-04-21T08:36:29.646025+0000","last_became_active":"2024-04-21T08:36:23.471193+0000","last_became_peered":"2024-04-21T08:36:23.471193+0000","last_unstale":"2024-04-21T08:36:29.646025+0000","last_undegraded":"2024-04-21T08:36:29.646025+0000","last_fullsized":"2024-04-21T08:36:29.646025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:06:04.916707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645500+0000","last_change":"2024-04-21T08:36:23.986781+0000","last_active":"2024-04-21T08:36:29.645500+0000","last_peered":"2024-04-21T08:36:29.645500+0000","last_clean":"2024-04-21T08:36:29.645500+0000","last_became_active":"2024-04-21T08:36:23.985696+0000","last_became_peered":"2024-04-21T08:36:23.985696+0000","last_unstale":"2024-04-21T08:36:29.645500+0000","last_undegraded":"2024-04-21T08:36:29.645500+0000","last_fullsized":"2024-04-21T08:36:29.645500+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:40.556642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644245+0000","last_change":"2024-04-21T08:36:28.011621+0000","last_active":"2024-04-21T08:36:29.644245+0000","last_peered":"2024-04-21T08:36:29.644245+0000","last_clean":"2024-04-21T08:36:29.644245+0000","last_became_active":"2024-04-21T08:36:28.011136+0000","last_became_peered":"2024-04-21T08:36:28.011136+0000","last_unstale":"2024-04-21T08:36:29.644245+0000","last_undegraded":"2024-04-21T08:36:29.644245+0000","last_fullsized":"2024-04-21T08:36:29.644245+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:47.623445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:48.062410+0000","last_change":"2024-04-21T08:36:23.474019+0000","last_active":"2024-04-21T08:36:48.062410+0000","last_peered":"2024-04-21T08:36:48.062410+0000","last_clean":"2024-04-21T08:36:48.062410+0000","last_became_active":"2024-04-21T08:36:23.473690+0000","last_became_peered":"2024-04-21T08:36:23.473690+0000","last_unstale":"2024-04-21T08:36:48.062410+0000","last_undegraded":"2024-04-21T08:36:48.062410+0000","last_fullsized":"2024-04-21T08:36:48.062410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:38:32.913795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644171+0000","last_change":"2024-04-21T08:36:23.994434+0000","last_active":"2024-04-21T08:36:29.644171+0000","last_peered":"2024-04-21T08:36:29.644171+0000","last_clean":"2024-04-21T08:36:29.644171+0000","last_became_active":"2024-04-21T08:36:23.993675+0000","last_became_peered":"2024-04-21T08:36:23.993675+0000","last_unstale":"2024-04-21T08:36:29.644171+0000","last_undegraded":"2024-04-21T08:36:29.644171+0000","last_fullsized":"2024-04-21T08:36:29.644171+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:53:07.613445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645159+0000","last_change":"2024-04-21T08:36:28.004420+0000","last_active":"2024-04-21T08:36:29.645159+0000","last_peered":"2024-04-21T08:36:29.645159+0000","last_clean":"2024-04-21T08:36:29.645159+0000","last_became_active":"2024-04-21T08:36:28.003935+0000","last_became_peered":"2024-04-21T08:36:28.003935+0000","last_unstale":"2024-04-21T08:36:29.645159+0000","last_undegraded":"2024-04-21T08:36:29.645159+0000","last_fullsized":"2024-04-21T08:36:29.645159+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:10:03.255869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:49.882859+0000","last_change":"2024-04-21T08:36:23.465825+0000","last_active":"2024-04-21T08:36:49.882859+0000","last_peered":"2024-04-21T08:36:49.882859+0000","last_clean":"2024-04-21T08:36:49.882859+0000","last_became_active":"2024-04-21T08:36:23.465525+0000","last_became_peered":"2024-04-21T08:36:23.465525+0000","last_unstale":"2024-04-21T08:36:49.882859+0000","last_undegraded":"2024-04-21T08:36:49.882859+0000","last_fullsized":"2024-04-21T08:36:49.882859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:59:50.502210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645512+0000","last_change":"2024-04-21T08:36:23.987602+0000","last_active":"2024-04-21T08:36:29.645512+0000","last_peered":"2024-04-21T08:36:29.645512+0000","last_clean":"2024-04-21T08:36:29.645512+0000","last_became_active":"2024-04-21T08:36:23.987298+0000","last_became_peered":"2024-04-21T08:36:23.987298+0000","last_unstale":"2024-04-21T08:36:29.645512+0000","last_undegraded":"2024-04-21T08:36:29.645512+0000","last_fullsized":"2024-04-21T08:36:29.645512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:06.647676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645386+0000","last_change":"2024-04-21T08:36:28.005935+0000","last_active":"2024-04-21T08:36:29.645386+0000","last_peered":"2024-04-21T08:36:29.645386+0000","last_clean":"2024-04-21T08:36:29.645386+0000","last_became_active":"2024-04-21T08:36:28.005512+0000","last_became_peered":"2024-04-21T08:36:28.005512+0000","last_unstale":"2024-04-21T08:36:29.645386+0000","last_undegraded":"2024-04-21T08:36:29.645386+0000","last_fullsized":"2024-04-21T08:36:29.645386+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:23:20.298524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'10","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:39.661040+0000","last_change":"2024-04-21T08:36:23.467728+0000","last_active":"2024-04-21T08:36:39.661040+0000","last_peered":"2024-04-21T08:36:39.661040+0000","last_clean":"2024-04-21T08:36:39.661040+0000","last_became_active":"2024-04-21T08:36:23.467430+0000","last_became_peered":"2024-04-21T08:36:23.467430+0000","last_unstale":"2024-04-21T08:36:39.661040+0000","last_undegraded":"2024-04-21T08:36:39.661040+0000","last_fullsized":"2024-04-21T08:36:39.661040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:47:18.900911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645232+0000","last_change":"2024-04-21T08:36:23.989116+0000","last_active":"2024-04-21T08:36:29.645232+0000","last_peered":"2024-04-21T08:36:29.645232+0000","last_clean":"2024-04-21T08:36:29.645232+0000","last_became_active":"2024-04-21T08:36:23.988758+0000","last_became_peered":"2024-04-21T08:36:23.988758+0000","last_unstale":"2024-04-21T08:36:29.645232+0000","last_undegraded":"2024-04-21T08:36:29.645232+0000","last_fullsized":"2024-04-21T08:36:29.645232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:56:25.969905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.995497+0000","last_change":"2024-04-21T08:36:28.007370+0000","last_active":"2024-04-21T08:36:28.995497+0000","last_peered":"2024-04-21T08:36:28.995497+0000","last_clean":"2024-04-21T08:36:28.995497+0000","last_became_active":"2024-04-21T08:36:28.006972+0000","last_became_peered":"2024-04-21T08:36:28.006972+0000","last_unstale":"2024-04-21T08:36:28.995497+0000","last_undegraded":"2024-04-21T08:36:28.995497+0000","last_fullsized":"2024-04-21T08:36:28.995497+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:18:14.926058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:50.904775+0000","last_change":"2024-04-21T08:36:23.472658+0000","last_active":"2024-04-21T08:36:50.904775+0000","last_peered":"2024-04-21T08:36:50.904775+0000","last_clean":"2024-04-21T08:36:50.904775+0000","last_became_active":"2024-04-21T08:36:23.472318+0000","last_became_peered":"2024-04-21T08:36:23.472318+0000","last_unstale":"2024-04-21T08:36:50.904775+0000","last_undegraded":"2024-04-21T08:36:50.904775+0000","last_fullsized":"2024-04-21T08:36:50.904775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:28:27.497605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.991865+0000","last_change":"2024-04-21T08:36:23.986897+0000","last_active":"2024-04-21T08:36:28.991865+0000","last_peered":"2024-04-21T08:36:28.991865+0000","last_clean":"2024-04-21T08:36:28.991865+0000","last_became_active":"2024-04-21T08:36:23.986554+0000","last_became_peered":"2024-04-21T08:36:23.986554+0000","last_unstale":"2024-04-21T08:36:28.991865+0000","last_undegraded":"2024-04-21T08:36:28.991865+0000","last_fullsized":"2024-04-21T08:36:28.991865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:44:22.331207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.506740+0000","last_change":"2024-04-21T08:36:28.019186+0000","last_active":"2024-04-21T08:36:29.506740+0000","last_peered":"2024-04-21T08:36:29.506740+0000","last_clean":"2024-04-21T08:36:29.506740+0000","last_became_active":"2024-04-21T08:36:28.018774+0000","last_became_peered":"2024-04-21T08:36:28.018774+0000","last_unstale":"2024-04-21T08:36:29.506740+0000","last_undegraded":"2024-04-21T08:36:29.506740+0000","last_fullsized":"2024-04-21T08:36:29.506740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:35:33.548656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:49.890074+0000","last_change":"2024-04-21T08:36:23.472633+0000","last_active":"2024-04-21T08:36:49.890074+0000","last_peered":"2024-04-21T08:36:49.890074+0000","last_clean":"2024-04-21T08:36:49.890074+0000","last_became_active":"2024-04-21T08:36:23.472336+0000","last_became_peered":"2024-04-21T08:36:23.472336+0000","last_unstale":"2024-04-21T08:36:49.890074+0000","last_undegraded":"2024-04-21T08:36:49.890074+0000","last_fullsized":"2024-04-21T08:36:49.890074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:49.463492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992633+0000","last_change":"2024-04-21T08:36:23.994332+0000","last_active":"2024-04-21T08:36:28.992633+0000","last_peered":"2024-04-21T08:36:28.992633+0000","last_clean":"2024-04-21T08:36:28.992633+0000","last_became_active":"2024-04-21T08:36:23.993936+0000","last_became_peered":"2024-04-21T08:36:23.993936+0000","last_unstale":"2024-04-21T08:36:28.992633+0000","last_undegraded":"2024-04-21T08:36:28.992633+0000","last_fullsized":"2024-04-21T08:36:28.992633+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:26:35.330559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645888+0000","last_change":"2024-04-21T08:36:28.012549+0000","last_active":"2024-04-21T08:36:29.645888+0000","last_peered":"2024-04-21T08:36:29.645888+0000","last_clean":"2024-04-21T08:36:29.645888+0000","last_became_active":"2024-04-21T08:36:28.012151+0000","last_became_peered":"2024-04-21T08:36:28.012151+0000","last_unstale":"2024-04-21T08:36:29.645888+0000","last_undegraded":"2024-04-21T08:36:29.645888+0000","last_fullsized":"2024-04-21T08:36:29.645888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:55.680399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:49.069358+0000","last_change":"2024-04-21T08:36:23.464586+0000","last_active":"2024-04-21T08:36:49.069358+0000","last_peered":"2024-04-21T08:36:49.069358+0000","last_clean":"2024-04-21T08:36:49.069358+0000","last_became_active":"2024-04-21T08:36:23.464281+0000","last_became_peered":"2024-04-21T08:36:23.464281+0000","last_unstale":"2024-04-21T08:36:49.069358+0000","last_undegraded":"2024-04-21T08:36:49.069358+0000","last_fullsized":"2024-04-21T08:36:49.069358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:04:58.577283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992476+0000","last_change":"2024-04-21T08:36:23.989604+0000","last_active":"2024-04-21T08:36:28.992476+0000","last_peered":"2024-04-21T08:36:28.992476+0000","last_clean":"2024-04-21T08:36:28.992476+0000","last_became_active":"2024-04-21T08:36:23.989257+0000","last_became_peered":"2024-04-21T08:36:23.989257+0000","last_unstale":"2024-04-21T08:36:28.992476+0000","last_undegraded":"2024-04-21T08:36:28.992476+0000","last_fullsized":"2024-04-21T08:36:28.992476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:45.077943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.991866+0000","last_change":"2024-04-21T08:36:28.007959+0000","last_active":"2024-04-21T08:36:28.991866+0000","last_peered":"2024-04-21T08:36:28.991866+0000","last_clean":"2024-04-21T08:36:28.991866+0000","last_became_active":"2024-04-21T08:36:28.007571+0000","last_became_peered":"2024-04-21T08:36:28.007571+0000","last_unstale":"2024-04-21T08:36:28.991866+0000","last_undegraded":"2024-04-21T08:36:28.991866+0000","last_fullsized":"2024-04-21T08:36:28.991866+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:17:34.007013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:49.070282+0000","last_change":"2024-04-21T08:36:23.475054+0000","last_active":"2024-04-21T08:36:49.070282+0000","last_peered":"2024-04-21T08:36:49.070282+0000","last_clean":"2024-04-21T08:36:49.070282+0000","last_became_active":"2024-04-21T08:36:23.474606+0000","last_became_peered":"2024-04-21T08:36:23.474606+0000","last_unstale":"2024-04-21T08:36:49.070282+0000","last_undegraded":"2024-04-21T08:36:49.070282+0000","last_fullsized":"2024-04-21T08:36:49.070282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:22.559174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.994442+0000","last_change":"2024-04-21T08:36:23.995980+0000","last_active":"2024-04-21T08:36:28.994442+0000","last_peered":"2024-04-21T08:36:28.994442+0000","last_clean":"2024-04-21T08:36:28.994442+0000","last_became_active":"2024-04-21T08:36:23.995673+0000","last_became_peered":"2024-04-21T08:36:23.995673+0000","last_unstale":"2024-04-21T08:36:28.994442+0000","last_undegraded":"2024-04-21T08:36:28.994442+0000","last_fullsized":"2024-04-21T08:36:28.994442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:11:49.884384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.994565+0000","last_change":"2024-04-21T08:36:28.018792+0000","last_active":"2024-04-21T08:36:28.994565+0000","last_peered":"2024-04-21T08:36:28.994565+0000","last_clean":"2024-04-21T08:36:28.994565+0000","last_became_active":"2024-04-21T08:36:28.018373+0000","last_became_peered":"2024-04-21T08:36:28.018373+0000","last_unstale":"2024-04-21T08:36:28.994565+0000","last_undegraded":"2024-04-21T08:36:28.994565+0000","last_fullsized":"2024-04-21T08:36:28.994565+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:11.857661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644928+0000","last_change":"2024-04-21T08:36:23.469787+0000","last_active":"2024-04-21T08:36:29.644928+0000","last_peered":"2024-04-21T08:36:29.644928+0000","last_clean":"2024-04-21T08:36:29.644928+0000","last_became_active":"2024-04-21T08:36:23.469462+0000","last_became_peered":"2024-04-21T08:36:23.469462+0000","last_unstale":"2024-04-21T08:36:29.644928+0000","last_undegraded":"2024-04-21T08:36:29.644928+0000","last_fullsized":"2024-04-21T08:36:29.644928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:33:25.849528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.506828+0000","last_change":"2024-04-21T08:36:23.988492+0000","last_active":"2024-04-21T08:36:29.506828+0000","last_peered":"2024-04-21T08:36:29.506828+0000","last_clean":"2024-04-21T08:36:29.506828+0000","last_became_active":"2024-04-21T08:36:23.988133+0000","last_became_peered":"2024-04-21T08:36:23.988133+0000","last_unstale":"2024-04-21T08:36:29.506828+0000","last_undegraded":"2024-04-21T08:36:29.506828+0000","last_fullsized":"2024-04-21T08:36:29.506828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:01:25.320281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644889+0000","last_change":"2024-04-21T08:36:28.012454+0000","last_active":"2024-04-21T08:36:29.644889+0000","last_peered":"2024-04-21T08:36:29.644889+0000","last_clean":"2024-04-21T08:36:29.644889+0000","last_became_active":"2024-04-21T08:36:28.011985+0000","last_became_peered":"2024-04-21T08:36:28.011985+0000","last_unstale":"2024-04-21T08:36:29.644889+0000","last_undegraded":"2024-04-21T08:36:29.644889+0000","last_fullsized":"2024-04-21T08:36:29.644889+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:16:51.939691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645655+0000","last_change":"2024-04-21T08:36:23.470403+0000","last_active":"2024-04-21T08:36:29.645655+0000","last_peered":"2024-04-21T08:36:29.645655+0000","last_clean":"2024-04-21T08:36:29.645655+0000","last_became_active":"2024-04-21T08:36:23.470035+0000","last_became_peered":"2024-04-21T08:36:23.470035+0000","last_unstale":"2024-04-21T08:36:29.645655+0000","last_undegraded":"2024-04-21T08:36:29.645655+0000","last_fullsized":"2024-04-21T08:36:29.645655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:36:17.395660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645402+0000","last_change":"2024-04-21T08:36:23.985910+0000","last_active":"2024-04-21T08:36:29.645402+0000","last_peered":"2024-04-21T08:36:29.645402+0000","last_clean":"2024-04-21T08:36:29.645402+0000","last_became_active":"2024-04-21T08:36:23.985605+0000","last_became_peered":"2024-04-21T08:36:23.985605+0000","last_unstale":"2024-04-21T08:36:29.645402+0000","last_undegraded":"2024-04-21T08:36:29.645402+0000","last_fullsized":"2024-04-21T08:36:29.645402+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:37:09.367005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.991970+0000","last_change":"2024-04-21T08:36:28.015256+0000","last_active":"2024-04-21T08:36:28.991970+0000","last_peered":"2024-04-21T08:36:28.991970+0000","last_clean":"2024-04-21T08:36:28.991970+0000","last_became_active":"2024-04-21T08:36:28.014849+0000","last_became_peered":"2024-04-21T08:36:28.014849+0000","last_unstale":"2024-04-21T08:36:28.991970+0000","last_undegraded":"2024-04-21T08:36:28.991970+0000","last_fullsized":"2024-04-21T08:36:28.991970+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:22:05.466299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992928+0000","last_change":"2024-04-21T08:36:23.474966+0000","last_active":"2024-04-21T08:36:28.992928+0000","last_peered":"2024-04-21T08:36:28.992928+0000","last_clean":"2024-04-21T08:36:28.992928+0000","last_became_active":"2024-04-21T08:36:23.474374+0000","last_became_peered":"2024-04-21T08:36:23.474374+0000","last_unstale":"2024-04-21T08:36:28.992928+0000","last_undegraded":"2024-04-21T08:36:28.992928+0000","last_fullsized":"2024-04-21T08:36:28.992928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:57:20.237966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645137+0000","last_change":"2024-04-21T08:36:23.986879+0000","last_active":"2024-04-21T08:36:29.645137+0000","last_peered":"2024-04-21T08:36:29.645137+0000","last_clean":"2024-04-21T08:36:29.645137+0000","last_became_active":"2024-04-21T08:36:23.986552+0000","last_became_peered":"2024-04-21T08:36:23.986552+0000","last_unstale":"2024-04-21T08:36:29.645137+0000","last_undegraded":"2024-04-21T08:36:29.645137+0000","last_fullsized":"2024-04-21T08:36:29.645137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:24:14.325345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992057+0000","last_change":"2024-04-21T08:36:28.013169+0000","last_active":"2024-04-21T08:36:28.992057+0000","last_peered":"2024-04-21T08:36:28.992057+0000","last_clean":"2024-04-21T08:36:28.992057+0000","last_became_active":"2024-04-21T08:36:28.012791+0000","last_became_peered":"2024-04-21T08:36:28.012791+0000","last_unstale":"2024-04-21T08:36:28.992057+0000","last_undegraded":"2024-04-21T08:36:28.992057+0000","last_fullsized":"2024-04-21T08:36:28.992057+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:42:56.063159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.994739+0000","last_change":"2024-04-21T08:36:23.995209+0000","last_active":"2024-04-21T08:36:28.994739+0000","last_peered":"2024-04-21T08:36:28.994739+0000","last_clean":"2024-04-21T08:36:28.994739+0000","last_became_active":"2024-04-21T08:36:23.994866+0000","last_became_peered":"2024-04-21T08:36:23.994866+0000","last_unstale":"2024-04-21T08:36:28.994739+0000","last_undegraded":"2024-04-21T08:36:28.994739+0000","last_fullsized":"2024-04-21T08:36:28.994739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:33:45.582150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.991351+0000","last_change":"2024-04-21T08:36:23.471474+0000","last_active":"2024-04-21T08:36:28.991351+0000","last_peered":"2024-04-21T08:36:28.991351+0000","last_clean":"2024-04-21T08:36:28.991351+0000","last_became_active":"2024-04-21T08:36:23.471179+0000","last_became_peered":"2024-04-21T08:36:23.471179+0000","last_unstale":"2024-04-21T08:36:28.991351+0000","last_undegraded":"2024-04-21T08:36:28.991351+0000","last_fullsized":"2024-04-21T08:36:28.991351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:17.290051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992730+0000","last_change":"2024-04-21T08:36:28.009671+0000","last_active":"2024-04-21T08:36:28.992730+0000","last_peered":"2024-04-21T08:36:28.992730+0000","last_clean":"2024-04-21T08:36:28.992730+0000","last_became_active":"2024-04-21T08:36:28.009185+0000","last_became_peered":"2024-04-21T08:36:28.009185+0000","last_unstale":"2024-04-21T08:36:28.992730+0000","last_undegraded":"2024-04-21T08:36:28.992730+0000","last_fullsized":"2024-04-21T08:36:28.992730+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:11:05.156271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644926+0000","last_change":"2024-04-21T08:36:23.990272+0000","last_active":"2024-04-21T08:36:29.644926+0000","last_peered":"2024-04-21T08:36:29.644926+0000","last_clean":"2024-04-21T08:36:29.644926+0000","last_became_active":"2024-04-21T08:36:23.989940+0000","last_became_peered":"2024-04-21T08:36:23.989940+0000","last_unstale":"2024-04-21T08:36:29.644926+0000","last_undegraded":"2024-04-21T08:36:29.644926+0000","last_fullsized":"2024-04-21T08:36:29.644926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:27:50.705167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:48.062707+0000","last_change":"2024-04-21T08:36:23.474162+0000","last_active":"2024-04-21T08:36:48.062707+0000","last_peered":"2024-04-21T08:36:48.062707+0000","last_clean":"2024-04-21T08:36:48.062707+0000","last_became_active":"2024-04-21T08:36:23.473802+0000","last_became_peered":"2024-04-21T08:36:23.473802+0000","last_unstale":"2024-04-21T08:36:48.062707+0000","last_undegraded":"2024-04-21T08:36:48.062707+0000","last_fullsized":"2024-04-21T08:36:48.062707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:32:41.918431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":355,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992282+0000","last_change":"2024-04-21T08:35:10.062130+0000","last_active":"2024-04-21T08:36:28.992282+0000","last_peered":"2024-04-21T08:36:28.992282+0000","last_clean":"2024-04-21T08:36:28.992282+0000","last_became_active":"2024-04-21T08:35:10.061727+0000","last_became_peered":"2024-04-21T08:35:10.061727+0000","last_unstale":"2024-04-21T08:36:28.992282+0000","last_undegraded":"2024-04-21T08:36:28.992282+0000","last_fullsized":"2024-04-21T08:36:28.992282+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:31:40.950939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:31:40.950939+0000","last_clean_scrub_stamp":"2024-04-21T08:31:40.950939+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:35:52.360167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.995143+0000","last_change":"2024-04-21T08:36:28.012840+0000","last_active":"2024-04-21T08:36:28.995143+0000","last_peered":"2024-04-21T08:36:28.995143+0000","last_clean":"2024-04-21T08:36:28.995143+0000","last_became_active":"2024-04-21T08:36:28.012439+0000","last_became_peered":"2024-04-21T08:36:28.012439+0000","last_unstale":"2024-04-21T08:36:28.995143+0000","last_undegraded":"2024-04-21T08:36:28.995143+0000","last_fullsized":"2024-04-21T08:36:28.995143+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:36:39.260061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.991261+0000","last_change":"2024-04-21T08:36:23.984332+0000","last_active":"2024-04-21T08:36:28.991261+0000","last_peered":"2024-04-21T08:36:28.991261+0000","last_clean":"2024-04-21T08:36:28.991261+0000","last_became_active":"2024-04-21T08:36:23.984021+0000","last_became_peered":"2024-04-21T08:36:23.984021+0000","last_unstale":"2024-04-21T08:36:28.991261+0000","last_undegraded":"2024-04-21T08:36:28.991261+0000","last_fullsized":"2024-04-21T08:36:28.991261+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:05:21.090633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992569+0000","last_change":"2024-04-21T08:36:23.467522+0000","last_active":"2024-04-21T08:36:28.992569+0000","last_peered":"2024-04-21T08:36:28.992569+0000","last_clean":"2024-04-21T08:36:28.992569+0000","last_became_active":"2024-04-21T08:36:23.466379+0000","last_became_peered":"2024-04-21T08:36:23.466379+0000","last_unstale":"2024-04-21T08:36:28.992569+0000","last_undegraded":"2024-04-21T08:36:28.992569+0000","last_fullsized":"2024-04-21T08:36:28.992569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:04.185314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.506545+0000","last_change":"2024-04-21T08:36:28.015120+0000","last_active":"2024-04-21T08:36:29.506545+0000","last_peered":"2024-04-21T08:36:29.506545+0000","last_clean":"2024-04-21T08:36:29.506545+0000","last_became_active":"2024-04-21T08:36:28.014701+0000","last_became_peered":"2024-04-21T08:36:28.014701+0000","last_unstale":"2024-04-21T08:36:29.506545+0000","last_undegraded":"2024-04-21T08:36:29.506545+0000","last_fullsized":"2024-04-21T08:36:29.506545+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:23.127622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645322+0000","last_change":"2024-04-21T08:36:23.475335+0000","last_active":"2024-04-21T08:36:29.645322+0000","last_peered":"2024-04-21T08:36:29.645322+0000","last_clean":"2024-04-21T08:36:29.645322+0000","last_became_active":"2024-04-21T08:36:23.474739+0000","last_became_peered":"2024-04-21T08:36:23.474739+0000","last_unstale":"2024-04-21T08:36:29.645322+0000","last_undegraded":"2024-04-21T08:36:29.645322+0000","last_fullsized":"2024-04-21T08:36:29.645322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:36.119649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.991960+0000","last_change":"2024-04-21T08:36:23.977229+0000","last_active":"2024-04-21T08:36:28.991960+0000","last_peered":"2024-04-21T08:36:28.991960+0000","last_clean":"2024-04-21T08:36:28.991960+0000","last_became_active":"2024-04-21T08:36:23.976893+0000","last_became_peered":"2024-04-21T08:36:23.976893+0000","last_unstale":"2024-04-21T08:36:28.991960+0000","last_undegraded":"2024-04-21T08:36:28.991960+0000","last_fullsized":"2024-04-21T08:36:28.991960+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:31:54.226274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644806+0000","last_change":"2024-04-21T08:36:28.018581+0000","last_active":"2024-04-21T08:36:29.644806+0000","last_peered":"2024-04-21T08:36:29.644806+0000","last_clean":"2024-04-21T08:36:29.644806+0000","last_became_active":"2024-04-21T08:36:28.018021+0000","last_became_peered":"2024-04-21T08:36:28.018021+0000","last_unstale":"2024-04-21T08:36:29.644806+0000","last_undegraded":"2024-04-21T08:36:29.644806+0000","last_fullsized":"2024-04-21T08:36:29.644806+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:29:00.355532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'8","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:49.893410+0000","last_change":"2024-04-21T08:36:23.473339+0000","last_active":"2024-04-21T08:36:49.893410+0000","last_peered":"2024-04-21T08:36:49.893410+0000","last_clean":"2024-04-21T08:36:49.893410+0000","last_became_active":"2024-04-21T08:36:23.472994+0000","last_became_peered":"2024-04-21T08:36:23.472994+0000","last_unstale":"2024-04-21T08:36:49.893410+0000","last_undegraded":"2024-04-21T08:36:49.893410+0000","last_fullsized":"2024-04-21T08:36:49.893410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:42.762273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645598+0000","last_change":"2024-04-21T08:36:23.987354+0000","last_active":"2024-04-21T08:36:29.645598+0000","last_peered":"2024-04-21T08:36:29.645598+0000","last_clean":"2024-04-21T08:36:29.645598+0000","last_became_active":"2024-04-21T08:36:23.987032+0000","last_became_peered":"2024-04-21T08:36:23.987032+0000","last_unstale":"2024-04-21T08:36:29.645598+0000","last_undegraded":"2024-04-21T08:36:29.645598+0000","last_fullsized":"2024-04-21T08:36:29.645598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:01:05.164367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645079+0000","last_change":"2024-04-21T08:36:28.002582+0000","last_active":"2024-04-21T08:36:29.645079+0000","last_peered":"2024-04-21T08:36:29.645079+0000","last_clean":"2024-04-21T08:36:29.645079+0000","last_became_active":"2024-04-21T08:36:28.002133+0000","last_became_peered":"2024-04-21T08:36:28.002133+0000","last_unstale":"2024-04-21T08:36:29.645079+0000","last_undegraded":"2024-04-21T08:36:29.645079+0000","last_fullsized":"2024-04-21T08:36:29.645079+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:30:00.268015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:48.062755+0000","last_change":"2024-04-21T08:36:23.462840+0000","last_active":"2024-04-21T08:36:48.062755+0000","last_peered":"2024-04-21T08:36:48.062755+0000","last_clean":"2024-04-21T08:36:48.062755+0000","last_became_active":"2024-04-21T08:36:23.462391+0000","last_became_peered":"2024-04-21T08:36:23.462391+0000","last_unstale":"2024-04-21T08:36:48.062755+0000","last_undegraded":"2024-04-21T08:36:48.062755+0000","last_fullsized":"2024-04-21T08:36:48.062755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:33.230388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.643766+0000","last_change":"2024-04-21T08:36:23.994751+0000","last_active":"2024-04-21T08:36:29.643766+0000","last_peered":"2024-04-21T08:36:29.643766+0000","last_clean":"2024-04-21T08:36:29.643766+0000","last_became_active":"2024-04-21T08:36:23.994384+0000","last_became_peered":"2024-04-21T08:36:23.994384+0000","last_unstale":"2024-04-21T08:36:29.643766+0000","last_undegraded":"2024-04-21T08:36:29.643766+0000","last_fullsized":"2024-04-21T08:36:29.643766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:18.956893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645100+0000","last_change":"2024-04-21T08:36:28.004004+0000","last_active":"2024-04-21T08:36:29.645100+0000","last_peered":"2024-04-21T08:36:29.645100+0000","last_clean":"2024-04-21T08:36:29.645100+0000","last_became_active":"2024-04-21T08:36:28.003547+0000","last_became_peered":"2024-04-21T08:36:28.003547+0000","last_unstale":"2024-04-21T08:36:29.645100+0000","last_undegraded":"2024-04-21T08:36:29.645100+0000","last_fullsized":"2024-04-21T08:36:29.645100+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:21:40.215428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:48.063775+0000","last_change":"2024-04-21T08:36:23.468528+0000","last_active":"2024-04-21T08:36:48.063775+0000","last_peered":"2024-04-21T08:36:48.063775+0000","last_clean":"2024-04-21T08:36:48.063775+0000","last_became_active":"2024-04-21T08:36:23.467838+0000","last_became_peered":"2024-04-21T08:36:23.467838+0000","last_unstale":"2024-04-21T08:36:48.063775+0000","last_undegraded":"2024-04-21T08:36:48.063775+0000","last_fullsized":"2024-04-21T08:36:48.063775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:03:44.901930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644152+0000","last_change":"2024-04-21T08:36:23.988020+0000","last_active":"2024-04-21T08:36:29.644152+0000","last_peered":"2024-04-21T08:36:29.644152+0000","last_clean":"2024-04-21T08:36:29.644152+0000","last_became_active":"2024-04-21T08:36:23.987630+0000","last_became_peered":"2024-04-21T08:36:23.987630+0000","last_unstale":"2024-04-21T08:36:29.644152+0000","last_undegraded":"2024-04-21T08:36:29.644152+0000","last_fullsized":"2024-04-21T08:36:29.644152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:06.519001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.646042+0000","last_change":"2024-04-21T08:36:28.006984+0000","last_active":"2024-04-21T08:36:29.646042+0000","last_peered":"2024-04-21T08:36:29.646042+0000","last_clean":"2024-04-21T08:36:29.646042+0000","last_became_active":"2024-04-21T08:36:28.006548+0000","last_became_peered":"2024-04-21T08:36:28.006548+0000","last_unstale":"2024-04-21T08:36:29.646042+0000","last_undegraded":"2024-04-21T08:36:29.646042+0000","last_fullsized":"2024-04-21T08:36:29.646042+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:25:50.870658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:49.889752+0000","last_change":"2024-04-21T08:36:23.476266+0000","last_active":"2024-04-21T08:36:49.889752+0000","last_peered":"2024-04-21T08:36:49.889752+0000","last_clean":"2024-04-21T08:36:49.889752+0000","last_became_active":"2024-04-21T08:36:23.475948+0000","last_became_peered":"2024-04-21T08:36:23.475948+0000","last_unstale":"2024-04-21T08:36:49.889752+0000","last_undegraded":"2024-04-21T08:36:49.889752+0000","last_fullsized":"2024-04-21T08:36:49.889752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:41.529376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.506350+0000","last_change":"2024-04-21T08:36:23.981942+0000","last_active":"2024-04-21T08:36:29.506350+0000","last_peered":"2024-04-21T08:36:29.506350+0000","last_clean":"2024-04-21T08:36:29.506350+0000","last_became_active":"2024-04-21T08:36:23.981643+0000","last_became_peered":"2024-04-21T08:36:23.981643+0000","last_unstale":"2024-04-21T08:36:29.506350+0000","last_undegraded":"2024-04-21T08:36:29.506350+0000","last_fullsized":"2024-04-21T08:36:29.506350+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:26:24.722695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644817+0000","last_change":"2024-04-21T08:36:28.006980+0000","last_active":"2024-04-21T08:36:29.644817+0000","last_peered":"2024-04-21T08:36:29.644817+0000","last_clean":"2024-04-21T08:36:29.644817+0000","last_became_active":"2024-04-21T08:36:28.006530+0000","last_became_peered":"2024-04-21T08:36:28.006530+0000","last_unstale":"2024-04-21T08:36:29.644817+0000","last_undegraded":"2024-04-21T08:36:29.644817+0000","last_fullsized":"2024-04-21T08:36:29.644817+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:05:57.465612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:49.058714+0000","last_change":"2024-04-21T08:36:23.473253+0000","last_active":"2024-04-21T08:36:49.058714+0000","last_peered":"2024-04-21T08:36:49.058714+0000","last_clean":"2024-04-21T08:36:49.058714+0000","last_became_active":"2024-04-21T08:36:23.472942+0000","last_became_peered":"2024-04-21T08:36:23.472942+0000","last_unstale":"2024-04-21T08:36:49.058714+0000","last_undegraded":"2024-04-21T08:36:49.058714+0000","last_fullsized":"2024-04-21T08:36:49.058714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:32:04.894388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645383+0000","last_change":"2024-04-21T08:36:23.988355+0000","last_active":"2024-04-21T08:36:29.645383+0000","last_peered":"2024-04-21T08:36:29.645383+0000","last_clean":"2024-04-21T08:36:29.645383+0000","last_became_active":"2024-04-21T08:36:23.988049+0000","last_became_peered":"2024-04-21T08:36:23.988049+0000","last_unstale":"2024-04-21T08:36:29.645383+0000","last_undegraded":"2024-04-21T08:36:29.645383+0000","last_fullsized":"2024-04-21T08:36:29.645383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:57.348698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644979+0000","last_change":"2024-04-21T08:36:28.007343+0000","last_active":"2024-04-21T08:36:29.644979+0000","last_peered":"2024-04-21T08:36:29.644979+0000","last_clean":"2024-04-21T08:36:29.644979+0000","last_became_active":"2024-04-21T08:36:28.006719+0000","last_became_peered":"2024-04-21T08:36:28.006719+0000","last_unstale":"2024-04-21T08:36:29.644979+0000","last_undegraded":"2024-04-21T08:36:29.644979+0000","last_fullsized":"2024-04-21T08:36:29.644979+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:02:25.734584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:48.062571+0000","last_change":"2024-04-21T08:36:23.472416+0000","last_active":"2024-04-21T08:36:48.062571+0000","last_peered":"2024-04-21T08:36:48.062571+0000","last_clean":"2024-04-21T08:36:48.062571+0000","last_became_active":"2024-04-21T08:36:23.471500+0000","last_became_peered":"2024-04-21T08:36:23.471500+0000","last_unstale":"2024-04-21T08:36:48.062571+0000","last_undegraded":"2024-04-21T08:36:48.062571+0000","last_fullsized":"2024-04-21T08:36:48.062571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:39.706432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992554+0000","last_change":"2024-04-21T08:36:23.990653+0000","last_active":"2024-04-21T08:36:28.992554+0000","last_peered":"2024-04-21T08:36:28.992554+0000","last_clean":"2024-04-21T08:36:28.992554+0000","last_became_active":"2024-04-21T08:36:23.990324+0000","last_became_peered":"2024-04-21T08:36:23.990324+0000","last_unstale":"2024-04-21T08:36:28.992554+0000","last_undegraded":"2024-04-21T08:36:28.992554+0000","last_fullsized":"2024-04-21T08:36:28.992554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:08:59.417850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644619+0000","last_change":"2024-04-21T08:36:28.007956+0000","last_active":"2024-04-21T08:36:29.644619+0000","last_peered":"2024-04-21T08:36:29.644619+0000","last_clean":"2024-04-21T08:36:29.644619+0000","last_became_active":"2024-04-21T08:36:28.007475+0000","last_became_peered":"2024-04-21T08:36:28.007475+0000","last_unstale":"2024-04-21T08:36:29.644619+0000","last_undegraded":"2024-04-21T08:36:29.644619+0000","last_fullsized":"2024-04-21T08:36:29.644619+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:54.039359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:49.894247+0000","last_change":"2024-04-21T08:36:23.476887+0000","last_active":"2024-04-21T08:36:49.894247+0000","last_peered":"2024-04-21T08:36:49.894247+0000","last_clean":"2024-04-21T08:36:49.894247+0000","last_became_active":"2024-04-21T08:36:23.476574+0000","last_became_peered":"2024-04-21T08:36:23.476574+0000","last_unstale":"2024-04-21T08:36:49.894247+0000","last_undegraded":"2024-04-21T08:36:49.894247+0000","last_fullsized":"2024-04-21T08:36:49.894247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:37:47.692358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992902+0000","last_change":"2024-04-21T08:36:23.982845+0000","last_active":"2024-04-21T08:36:28.992902+0000","last_peered":"2024-04-21T08:36:28.992902+0000","last_clean":"2024-04-21T08:36:28.992902+0000","last_became_active":"2024-04-21T08:36:23.982491+0000","last_became_peered":"2024-04-21T08:36:23.982491+0000","last_unstale":"2024-04-21T08:36:28.992902+0000","last_undegraded":"2024-04-21T08:36:28.992902+0000","last_fullsized":"2024-04-21T08:36:28.992902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:48:46.413896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.643762+0000","last_change":"2024-04-21T08:36:28.010442+0000","last_active":"2024-04-21T08:36:29.643762+0000","last_peered":"2024-04-21T08:36:29.643762+0000","last_clean":"2024-04-21T08:36:29.643762+0000","last_became_active":"2024-04-21T08:36:28.010046+0000","last_became_peered":"2024-04-21T08:36:28.010046+0000","last_unstale":"2024-04-21T08:36:29.643762+0000","last_undegraded":"2024-04-21T08:36:29.643762+0000","last_fullsized":"2024-04-21T08:36:29.643762+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:18:36.382802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:49.891675+0000","last_change":"2024-04-21T08:36:23.473545+0000","last_active":"2024-04-21T08:36:49.891675+0000","last_peered":"2024-04-21T08:36:49.891675+0000","last_clean":"2024-04-21T08:36:49.891675+0000","last_became_active":"2024-04-21T08:36:23.473222+0000","last_became_peered":"2024-04-21T08:36:23.473222+0000","last_unstale":"2024-04-21T08:36:49.891675+0000","last_undegraded":"2024-04-21T08:36:49.891675+0000","last_fullsized":"2024-04-21T08:36:49.891675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:47:53.947187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992544+0000","last_change":"2024-04-21T08:36:23.990959+0000","last_active":"2024-04-21T08:36:28.992544+0000","last_peered":"2024-04-21T08:36:28.992544+0000","last_clean":"2024-04-21T08:36:28.992544+0000","last_became_active":"2024-04-21T08:36:23.990322+0000","last_became_peered":"2024-04-21T08:36:23.990322+0000","last_unstale":"2024-04-21T08:36:28.992544+0000","last_undegraded":"2024-04-21T08:36:28.992544+0000","last_fullsized":"2024-04-21T08:36:28.992544+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:12:20.859224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.994454+0000","last_change":"2024-04-21T08:36:28.006682+0000","last_active":"2024-04-21T08:36:28.994454+0000","last_peered":"2024-04-21T08:36:28.994454+0000","last_clean":"2024-04-21T08:36:28.994454+0000","last_became_active":"2024-04-21T08:36:28.006229+0000","last_became_peered":"2024-04-21T08:36:28.006229+0000","last_unstale":"2024-04-21T08:36:28.994454+0000","last_undegraded":"2024-04-21T08:36:28.994454+0000","last_fullsized":"2024-04-21T08:36:28.994454+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:30:12.723848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.991834+0000","last_change":"2024-04-21T08:36:23.473592+0000","last_active":"2024-04-21T08:36:28.991834+0000","last_peered":"2024-04-21T08:36:28.991834+0000","last_clean":"2024-04-21T08:36:28.991834+0000","last_became_active":"2024-04-21T08:36:23.473283+0000","last_became_peered":"2024-04-21T08:36:23.473283+0000","last_unstale":"2024-04-21T08:36:28.991834+0000","last_undegraded":"2024-04-21T08:36:28.991834+0000","last_fullsized":"2024-04-21T08:36:28.991834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:10.425249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.992949+0000","last_change":"2024-04-21T08:36:23.985052+0000","last_active":"2024-04-21T08:36:28.992949+0000","last_peered":"2024-04-21T08:36:28.992949+0000","last_clean":"2024-04-21T08:36:28.992949+0000","last_became_active":"2024-04-21T08:36:23.984742+0000","last_became_peered":"2024-04-21T08:36:23.984742+0000","last_unstale":"2024-04-21T08:36:28.992949+0000","last_undegraded":"2024-04-21T08:36:28.992949+0000","last_fullsized":"2024-04-21T08:36:28.992949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:08:06.082974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.506488+0000","last_change":"2024-04-21T08:36:23.980905+0000","last_active":"2024-04-21T08:36:29.506488+0000","last_peered":"2024-04-21T08:36:29.506488+0000","last_clean":"2024-04-21T08:36:29.506488+0000","last_became_active":"2024-04-21T08:36:23.980590+0000","last_became_peered":"2024-04-21T08:36:23.980590+0000","last_unstale":"2024-04-21T08:36:29.506488+0000","last_undegraded":"2024-04-21T08:36:29.506488+0000","last_fullsized":"2024-04-21T08:36:29.506488+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:54:59.886014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644549+0000","last_change":"2024-04-21T08:36:23.980883+0000","last_active":"2024-04-21T08:36:29.644549+0000","last_peered":"2024-04-21T08:36:29.644549+0000","last_clean":"2024-04-21T08:36:29.644549+0000","last_became_active":"2024-04-21T08:36:23.980562+0000","last_became_peered":"2024-04-21T08:36:23.980562+0000","last_unstale":"2024-04-21T08:36:29.644549+0000","last_undegraded":"2024-04-21T08:36:29.644549+0000","last_fullsized":"2024-04-21T08:36:29.644549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:14:51.868438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644224+0000","last_change":"2024-04-21T08:36:23.991571+0000","last_active":"2024-04-21T08:36:29.644224+0000","last_peered":"2024-04-21T08:36:29.644224+0000","last_clean":"2024-04-21T08:36:29.644224+0000","last_became_active":"2024-04-21T08:36:23.991273+0000","last_became_peered":"2024-04-21T08:36:23.991273+0000","last_unstale":"2024-04-21T08:36:29.644224+0000","last_undegraded":"2024-04-21T08:36:29.644224+0000","last_fullsized":"2024-04-21T08:36:29.644224+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:08:23.895038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.643975+0000","last_change":"2024-04-21T08:36:23.994729+0000","last_active":"2024-04-21T08:36:29.643975+0000","last_peered":"2024-04-21T08:36:29.643975+0000","last_clean":"2024-04-21T08:36:29.643975+0000","last_became_active":"2024-04-21T08:36:23.994405+0000","last_became_peered":"2024-04-21T08:36:23.994405+0000","last_unstale":"2024-04-21T08:36:29.643975+0000","last_undegraded":"2024-04-21T08:36:29.643975+0000","last_fullsized":"2024-04-21T08:36:29.643975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:42.463225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.991724+0000","last_change":"2024-04-21T08:36:23.995421+0000","last_active":"2024-04-21T08:36:28.991724+0000","last_peered":"2024-04-21T08:36:28.991724+0000","last_clean":"2024-04-21T08:36:28.991724+0000","last_became_active":"2024-04-21T08:36:23.993674+0000","last_became_peered":"2024-04-21T08:36:23.993674+0000","last_unstale":"2024-04-21T08:36:28.991724+0000","last_undegraded":"2024-04-21T08:36:28.991724+0000","last_fullsized":"2024-04-21T08:36:28.991724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:31.013874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.506757+0000","last_change":"2024-04-21T08:36:23.986418+0000","last_active":"2024-04-21T08:36:29.506757+0000","last_peered":"2024-04-21T08:36:29.506757+0000","last_clean":"2024-04-21T08:36:29.506757+0000","last_became_active":"2024-04-21T08:36:23.986095+0000","last_became_peered":"2024-04-21T08:36:23.986095+0000","last_unstale":"2024-04-21T08:36:29.506757+0000","last_undegraded":"2024-04-21T08:36:29.506757+0000","last_fullsized":"2024-04-21T08:36:29.506757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:31:56.456218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644890+0000","last_change":"2024-04-21T08:36:23.982353+0000","last_active":"2024-04-21T08:36:29.644890+0000","last_peered":"2024-04-21T08:36:29.644890+0000","last_clean":"2024-04-21T08:36:29.644890+0000","last_became_active":"2024-04-21T08:36:23.982035+0000","last_became_peered":"2024-04-21T08:36:23.982035+0000","last_unstale":"2024-04-21T08:36:29.644890+0000","last_undegraded":"2024-04-21T08:36:29.644890+0000","last_fullsized":"2024-04-21T08:36:29.644890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:42:44.598407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645856+0000","last_change":"2024-04-21T08:36:23.996371+0000","last_active":"2024-04-21T08:36:29.645856+0000","last_peered":"2024-04-21T08:36:29.645856+0000","last_clean":"2024-04-21T08:36:29.645856+0000","last_became_active":"2024-04-21T08:36:23.995802+0000","last_became_peered":"2024-04-21T08:36:23.995802+0000","last_unstale":"2024-04-21T08:36:29.645856+0000","last_undegraded":"2024-04-21T08:36:29.645856+0000","last_fullsized":"2024-04-21T08:36:29.645856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:52:08.571567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.994538+0000","last_change":"2024-04-21T08:36:23.987275+0000","last_active":"2024-04-21T08:36:28.994538+0000","last_peered":"2024-04-21T08:36:28.994538+0000","last_clean":"2024-04-21T08:36:28.994538+0000","last_became_active":"2024-04-21T08:36:23.986967+0000","last_became_peered":"2024-04-21T08:36:23.986967+0000","last_unstale":"2024-04-21T08:36:28.994538+0000","last_undegraded":"2024-04-21T08:36:28.994538+0000","last_fullsized":"2024-04-21T08:36:28.994538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:50.432502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.645254+0000","last_change":"2024-04-21T08:36:23.993046+0000","last_active":"2024-04-21T08:36:29.645254+0000","last_peered":"2024-04-21T08:36:29.645254+0000","last_clean":"2024-04-21T08:36:29.645254+0000","last_became_active":"2024-04-21T08:36:23.992693+0000","last_became_peered":"2024-04-21T08:36:23.992693+0000","last_unstale":"2024-04-21T08:36:29.645254+0000","last_undegraded":"2024-04-21T08:36:29.645254+0000","last_fullsized":"2024-04-21T08:36:29.645254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:12:06.287585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.644962+0000","last_change":"2024-04-21T08:36:23.992968+0000","last_active":"2024-04-21T08:36:29.644962+0000","last_peered":"2024-04-21T08:36:29.644962+0000","last_clean":"2024-04-21T08:36:29.644962+0000","last_became_active":"2024-04-21T08:36:23.992614+0000","last_became_peered":"2024-04-21T08:36:23.992614+0000","last_unstale":"2024-04-21T08:36:29.644962+0000","last_undegraded":"2024-04-21T08:36:29.644962+0000","last_fullsized":"2024-04-21T08:36:29.644962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:10.015323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.991739+0000","last_change":"2024-04-21T08:36:23.981169+0000","last_active":"2024-04-21T08:36:28.991739+0000","last_peered":"2024-04-21T08:36:28.991739+0000","last_clean":"2024-04-21T08:36:28.991739+0000","last_became_active":"2024-04-21T08:36:23.980849+0000","last_became_peered":"2024-04-21T08:36:23.980849+0000","last_unstale":"2024-04-21T08:36:28.991739+0000","last_undegraded":"2024-04-21T08:36:28.991739+0000","last_fullsized":"2024-04-21T08:36:28.991739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:13:49.907370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T08:36:29.506592+0000","last_change":"2024-04-21T08:36:23.993998+0000","last_active":"2024-04-21T08:36:29.506592+0000","last_peered":"2024-04-21T08:36:29.506592+0000","last_clean":"2024-04-21T08:36:29.506592+0000","last_became_active":"2024-04-21T08:36:23.993664+0000","last_became_peered":"2024-04-21T08:36:23.993664+0000","last_unstale":"2024-04-21T08:36:29.506592+0000","last_undegraded":"2024-04-21T08:36:29.506592+0000","last_fullsized":"2024-04-21T08:36:29.506592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:31:49.076820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9438,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":135,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":315392,"data_stored":52021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124,"ondisk_log_size":124,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":481709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612631,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32752,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712400,"statfs":{"total":95995035648,"available":95961497600,"internally_reserved":0,"allocated":1613824,"data_stored":1076113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776157,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":486927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037794,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":482335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299430,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1593344,"data_stored":1072659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462955,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":995328,"data_stored":481709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":493903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":484278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":487523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":482117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":482353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724594,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32772,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712380,"statfs":{"total":95995035648,"available":95961477120,"internally_reserved":0,"allocated":1634304,"data_stored":1075288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":24893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":13196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:36:53.761 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-21T08:36:53.762 INFO:tasks.cephadm:Setting up client nodes... 2024-04-21T08:36:53.762 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-21T08:36:54.041 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:36:54.409 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:54 smithi043 ceph-mon[27978]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:36:54.409 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:54 smithi043 ceph-mon[27978]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:36:54.435 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:54 smithi112 ceph-mon[31085]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:36:54.435 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:54 smithi112 ceph-mon[31085]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:36:54.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:54 smithi175 ceph-mon[31920]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:36:54.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:54 smithi175 ceph-mon[31920]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:36:54.794 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:36:54 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-b[61578]: 2024-04-21T08:36:54.435+0000 7fb798658700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-21T08:36:54.794 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:36:54 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-b[61578]: 2024-04-21T08:36:54.435+0000 7fb798658700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-21T08:36:54.794 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:36:54 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-b[61578]: 2024-04-21T08:36:54.435+0000 7fb798658700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-21T08:36:55.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:55 smithi043 ceph-mon[27978]: pgmap v310: 129 pgs: 129 active+clean; 622 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7.5 KiB/s wr, 14 op/s 2024-04-21T08:36:55.365 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:55 smithi175 ceph-mon[31920]: pgmap v310: 129 pgs: 129 active+clean; 622 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7.5 KiB/s wr, 14 op/s 2024-04-21T08:36:55.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:55 smithi112 ceph-mon[31085]: pgmap v310: 129 pgs: 129 active+clean; 622 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7.5 KiB/s wr, 14 op/s 2024-04-21T08:36:55.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.900+0000 7f452c67e700 1 -- 172.21.15.43:0/2388479657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4524105f20 msgr2=0x7f4524108310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:55.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.900+0000 7f452c67e700 1 --2- 172.21.15.43:0/2388479657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4524105f20 0x7f4524108310 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4514009990 tx=0x7f4514009ca0 comp rx=0 tx=0).stop 2024-04-21T08:36:55.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.900+0000 7f452c67e700 1 -- 172.21.15.43:0/2388479657 shutdown_connections 2024-04-21T08:36:55.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.900+0000 7f452c67e700 1 --2- 172.21.15.43:0/2388479657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45241035f0 0x7f45241059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:55.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.900+0000 7f452c67e700 1 --2- 172.21.15.43:0/2388479657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4524108850 0x7f452410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:55.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.900+0000 7f452c67e700 1 --2- 172.21.15.43:0/2388479657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4524105f20 0x7f4524108310 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:55.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.900+0000 7f452c67e700 1 -- 172.21.15.43:0/2388479657 >> 172.21.15.43:0/2388479657 conn(0x7f45240fcf60 msgr2=0x7f45240ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:55.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.900+0000 7f452c67e700 1 -- 172.21.15.43:0/2388479657 shutdown_connections 2024-04-21T08:36:55.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.900+0000 7f452c67e700 1 -- 172.21.15.43:0/2388479657 wait complete. 2024-04-21T08:36:55.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.901+0000 7f452c67e700 1 Processor -- start 2024-04-21T08:36:55.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.901+0000 7f452c67e700 1 -- start start 2024-04-21T08:36:55.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.901+0000 7f452c67e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45241035f0 0x7f4524198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:55.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.901+0000 7f452c67e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4524105f20 0x7f4524199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:55.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.901+0000 7f452c67e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4524108850 0x7f45241a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.901+0000 7f452c67e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f452410e540 con 0x7f45241035f0 2024-04-21T08:36:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.901+0000 7f452c67e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f452410e3c0 con 0x7f4524108850 2024-04-21T08:36:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.901+0000 7f452c67e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f452410e6c0 con 0x7f4524105f20 2024-04-21T08:36:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.901+0000 7f452a41a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45241035f0 0x7f4524198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.901+0000 7f452a41a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45241035f0 0x7f4524198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35676/0 (socket says 172.21.15.43:35676) 2024-04-21T08:36:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.901+0000 7f452a41a700 1 -- 172.21.15.43:0/1061995514 learned_addr learned my addr 172.21.15.43:0/1061995514 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.902+0000 7f452ac1b700 1 --2- 172.21.15.43:0/1061995514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4524108850 0x7f45241a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.902+0000 7f4529c19700 1 --2- 172.21.15.43:0/1061995514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4524105f20 0x7f4524199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.902+0000 7f452a41a700 1 -- 172.21.15.43:0/1061995514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4524105f20 msgr2=0x7f4524199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.902+0000 7f452a41a700 1 --2- 172.21.15.43:0/1061995514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4524105f20 0x7f4524199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.902+0000 7f452a41a700 1 -- 172.21.15.43:0/1061995514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4524108850 msgr2=0x7f45241a0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.902+0000 7f452a41a700 1 --2- 172.21.15.43:0/1061995514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4524108850 0x7f45241a0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.902+0000 7f452a41a700 1 -- 172.21.15.43:0/1061995514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45241a0720 con 0x7f45241035f0 2024-04-21T08:36:55.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.902+0000 7f452a41a700 1 --2- 172.21.15.43:0/1061995514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45241035f0 0x7f4524198ac0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f452000d9b0 tx=0x7f452000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:55.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.902+0000 7f451b7fe700 1 -- 172.21.15.43:0/1061995514 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520017070 con 0x7f45241035f0 2024-04-21T08:36:55.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.902+0000 7f452c67e700 1 -- 172.21.15.43:0/1061995514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45241a0a10 con 0x7f45241035f0 2024-04-21T08:36:55.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.902+0000 7f451b7fe700 1 -- 172.21.15.43:0/1061995514 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4520005020 con 0x7f45241035f0 2024-04-21T08:36:55.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.902+0000 7f451b7fe700 1 -- 172.21.15.43:0/1061995514 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520009b50 con 0x7f45241035f0 2024-04-21T08:36:55.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.902+0000 7f452c67e700 1 -- 172.21.15.43:0/1061995514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45241a0f00 con 0x7f45241035f0 2024-04-21T08:36:55.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.903+0000 7f452c67e700 1 -- 172.21.15.43:0/1061995514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4524050f80 con 0x7f45241035f0 2024-04-21T08:36:55.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.904+0000 7f451b7fe700 1 -- 172.21.15.43:0/1061995514 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f45200051d0 con 0x7f45241035f0 2024-04-21T08:36:55.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.905+0000 7f451b7fe700 1 --2- 172.21.15.43:0/1061995514 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f45100a3d80 0x7f45100a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:55.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.905+0000 7f4529c19700 1 --2- 172.21.15.43:0/1061995514 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f45100a3d80 0x7f45100a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:55.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.905+0000 7f451b7fe700 1 -- 172.21.15.43:0/1061995514 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f45200d06c0 con 0x7f45241035f0 2024-04-21T08:36:55.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.906+0000 7f4529c19700 1 --2- 172.21.15.43:0/1061995514 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f45100a3d80 0x7f45100a6240 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f451400b5c0 tx=0x7f4514005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:55.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:55.910+0000 7f451b7fe700 1 -- 172.21.15.43:0/1061995514 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4520020020 con 0x7f45241035f0 2024-04-21T08:36:56.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:56.206+0000 7f452c67e700 1 -- 172.21.15.43:0/1061995514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f452402df80 con 0x7f45241035f0 2024-04-21T08:36:56.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:56.209+0000 7f451b7fe700 1 -- 172.21.15.43:0/1061995514 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f452009cd00 con 0x7f45241035f0 2024-04-21T08:36:56.209 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-04-21T08:36:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAo0CRmB5l2DBAA9Q+7Mvb5l3w0tp/Jvjgjzw== 2024-04-21T08:36:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:56.212+0000 7f452c67e700 1 -- 172.21.15.43:0/1061995514 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f45100a3d80 msgr2=0x7f45100a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:56.212+0000 7f452c67e700 1 --2- 172.21.15.43:0/1061995514 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f45100a3d80 0x7f45100a6240 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f451400b5c0 tx=0x7f4514005fd0 comp rx=0 tx=0).stop 2024-04-21T08:36:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:56.212+0000 7f452c67e700 1 -- 172.21.15.43:0/1061995514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45241035f0 msgr2=0x7f4524198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:56.212+0000 7f452c67e700 1 --2- 172.21.15.43:0/1061995514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45241035f0 0x7f4524198ac0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f452000d9b0 tx=0x7f452000dcc0 comp rx=0 tx=0).stop 2024-04-21T08:36:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:56.212+0000 7f452c67e700 1 -- 172.21.15.43:0/1061995514 shutdown_connections 2024-04-21T08:36:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:56.212+0000 7f452c67e700 1 --2- 172.21.15.43:0/1061995514 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f45100a3d80 0x7f45100a6240 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:56.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:56.212+0000 7f452c67e700 1 --2- 172.21.15.43:0/1061995514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4524108850 0x7f45241a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:56.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:56.212+0000 7f452c67e700 1 --2- 172.21.15.43:0/1061995514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45241035f0 0x7f4524198ac0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:56.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:56.212+0000 7f452c67e700 1 --2- 172.21.15.43:0/1061995514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4524105f20 0x7f4524199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:56.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:56.212+0000 7f452c67e700 1 -- 172.21.15.43:0/1061995514 >> 172.21.15.43:0/1061995514 conn(0x7f45240fcf60 msgr2=0x7f4524103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:56.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:56.212+0000 7f452c67e700 1 -- 172.21.15.43:0/1061995514 shutdown_connections 2024-04-21T08:36:56.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:56.212+0000 7f452c67e700 1 -- 172.21.15.43:0/1061995514 wait complete. 2024-04-21T08:36:56.958 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-21T08:36:56.958 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-21T08:36:56.958 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-21T08:36:56.995 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-21T08:36:56.996 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-21T08:36:56.996 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-21T08:36:57.289 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:36:57.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:57 smithi043 ceph-mon[27978]: pgmap v311: 129 pgs: 129 active+clean; 624 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 7.8 KiB/s wr, 19 op/s 2024-04-21T08:36:57.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:57 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1061995514' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-21T08:36:57.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:57 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1061995514' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-21T08:36:57.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:57 smithi175 ceph-mon[31920]: pgmap v311: 129 pgs: 129 active+clean; 624 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 7.8 KiB/s wr, 19 op/s 2024-04-21T08:36:57.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:57 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1061995514' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-21T08:36:57.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:57 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1061995514' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-21T08:36:57.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:57 smithi112 ceph-mon[31085]: pgmap v311: 129 pgs: 129 active+clean; 624 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 7.8 KiB/s wr, 19 op/s 2024-04-21T08:36:57.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:57 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1061995514' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-21T08:36:57.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:57 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1061995514' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-21T08:36:59.205 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:36:59 smithi043 ceph-mon[27978]: pgmap v312: 129 pgs: 129 active+clean; 629 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.5 KiB/s wr, 23 op/s 2024-04-21T08:36:59.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.204+0000 7fe3a3f61700 1 -- 172.21.15.43:0/990699912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe39c108a70 msgr2=0x7fe39c108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:59.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.204+0000 7fe3a3f61700 1 --2- 172.21.15.43:0/990699912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe39c108a70 0x7fe39c108e50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe398009990 tx=0x7fe398009ca0 comp rx=0 tx=0).stop 2024-04-21T08:36:59.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.205+0000 7fe3a3f61700 1 -- 172.21.15.43:0/990699912 shutdown_connections 2024-04-21T08:36:59.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.205+0000 7fe3a3f61700 1 --2- 172.21.15.43:0/990699912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe39c06bb30 0x7fe39c10f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:59.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.205+0000 7fe3a3f61700 1 --2- 172.21.15.43:0/990699912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe39c108a70 0x7fe39c108e50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:59.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.205+0000 7fe3a3f61700 1 --2- 172.21.15.43:0/990699912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe39c06b170 0x7fe39c06b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:59.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.205+0000 7fe3a3f61700 1 -- 172.21.15.43:0/990699912 >> 172.21.15.43:0/990699912 conn(0x7fe39c0777b0 msgr2=0x7fe39c077bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:59.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.205+0000 7fe3a3f61700 1 -- 172.21.15.43:0/990699912 shutdown_connections 2024-04-21T08:36:59.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.205+0000 7fe3a3f61700 1 -- 172.21.15.43:0/990699912 wait complete. 2024-04-21T08:36:59.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.206+0000 7fe3a3f61700 1 Processor -- start 2024-04-21T08:36:59.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.206+0000 7fe3a3f61700 1 -- start start 2024-04-21T08:36:59.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.206+0000 7fe3a3f61700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe39c06b170 0x7fe39c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:59.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.206+0000 7fe3a3f61700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe39c06bb30 0x7fe39c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:59.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.206+0000 7fe3a3f61700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe39c108a70 0x7fe39c19dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:59.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.206+0000 7fe3a3f61700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c112d30 con 0x7fe39c108a70 2024-04-21T08:36:59.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.206+0000 7fe3a3f61700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c112bb0 con 0x7fe39c06bb30 2024-04-21T08:36:59.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.206+0000 7fe3a3f61700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c112eb0 con 0x7fe39c06b170 2024-04-21T08:36:59.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.206+0000 7fe3a24fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe39c108a70 0x7fe39c19dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:59.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.206+0000 7fe3a24fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe39c108a70 0x7fe39c19dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35084/0 (socket says 172.21.15.43:35084) 2024-04-21T08:36:59.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.206+0000 7fe3a24fe700 1 -- 172.21.15.43:0/2639709663 learned_addr learned my addr 172.21.15.43:0/2639709663 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:36:59.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.206+0000 7fe3a14fc700 1 --2- 172.21.15.43:0/2639709663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe39c06bb30 0x7fe39c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:59.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.206+0000 7fe3a1cfd700 1 --2- 172.21.15.43:0/2639709663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe39c06b170 0x7fe39c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:59.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.206+0000 7fe3a24fe700 1 -- 172.21.15.43:0/2639709663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe39c06b170 msgr2=0x7fe39c199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:59.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.206+0000 7fe3a24fe700 1 --2- 172.21.15.43:0/2639709663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe39c06b170 0x7fe39c199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:59.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.206+0000 7fe3a24fe700 1 -- 172.21.15.43:0/2639709663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe39c06bb30 msgr2=0x7fe39c1997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:59.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.206+0000 7fe3a24fe700 1 --2- 172.21.15.43:0/2639709663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe39c06bb30 0x7fe39c1997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:59.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.206+0000 7fe3a24fe700 1 -- 172.21.15.43:0/2639709663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39c19e220 con 0x7fe39c108a70 2024-04-21T08:36:59.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.207+0000 7fe3a24fe700 1 --2- 172.21.15.43:0/2639709663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe39c108a70 0x7fe39c19dae0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe39400b970 tx=0x7fe39400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:59.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.207+0000 7fe392ffd700 1 -- 172.21.15.43:0/2639709663 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39400d860 con 0x7fe39c108a70 2024-04-21T08:36:59.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.207+0000 7fe3a3f61700 1 -- 172.21.15.43:0/2639709663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe39c19e510 con 0x7fe39c108a70 2024-04-21T08:36:59.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.207+0000 7fe392ffd700 1 -- 172.21.15.43:0/2639709663 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe39400e430 con 0x7fe39c108a70 2024-04-21T08:36:59.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.207+0000 7fe392ffd700 1 -- 172.21.15.43:0/2639709663 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394017660 con 0x7fe39c108a70 2024-04-21T08:36:59.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.207+0000 7fe3a3f61700 1 -- 172.21.15.43:0/2639709663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe39c1a5d90 con 0x7fe39c108a70 2024-04-21T08:36:59.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.208+0000 7fe3a3f61700 1 -- 172.21.15.43:0/2639709663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe39c1041a0 con 0x7fe39c108a70 2024-04-21T08:36:59.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.209+0000 7fe392ffd700 1 -- 172.21.15.43:0/2639709663 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe39400da00 con 0x7fe39c108a70 2024-04-21T08:36:59.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.210+0000 7fe392ffd700 1 --2- 172.21.15.43:0/2639709663 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe3880a3cd0 0x7fe3880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:59.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.210+0000 7fe3a1cfd700 1 --2- 172.21.15.43:0/2639709663 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe3880a3cd0 0x7fe3880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:59.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.210+0000 7fe392ffd700 1 -- 172.21.15.43:0/2639709663 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe3940d0750 con 0x7fe39c108a70 2024-04-21T08:36:59.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.210+0000 7fe3a1cfd700 1 --2- 172.21.15.43:0/2639709663 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe3880a3cd0 0x7fe3880a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe398009750 tx=0x7fe398005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:59.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.213+0000 7fe392ffd700 1 -- 172.21.15.43:0/2639709663 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe394013030 con 0x7fe39c108a70 2024-04-21T08:36:59.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:36:59 smithi175 ceph-mon[31920]: pgmap v312: 129 pgs: 129 active+clean; 629 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.5 KiB/s wr, 23 op/s 2024-04-21T08:36:59.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.481+0000 7fe3a3f61700 1 -- 172.21.15.43:0/2639709663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fe39c19a570 con 0x7fe39c108a70 2024-04-21T08:36:59.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.486+0000 7fe392ffd700 1 -- 172.21.15.43:0/2639709663 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264997 (secure 0 0 0) 0x7fe39409cd90 con 0x7fe39c108a70 2024-04-21T08:36:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:36:59.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.490+0000 7fe3a3f61700 1 -- 172.21.15.43:0/2639709663 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe3880a3cd0 msgr2=0x7fe3880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:59.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.490+0000 7fe3a3f61700 1 --2- 172.21.15.43:0/2639709663 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe3880a3cd0 0x7fe3880a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe398009750 tx=0x7fe398005960 comp rx=0 tx=0).stop 2024-04-21T08:36:59.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.490+0000 7fe3a3f61700 1 -- 172.21.15.43:0/2639709663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe39c108a70 msgr2=0x7fe39c19dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:59.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.490+0000 7fe3a3f61700 1 --2- 172.21.15.43:0/2639709663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe39c108a70 0x7fe39c19dae0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe39400b970 tx=0x7fe39400bc80 comp rx=0 tx=0).stop 2024-04-21T08:36:59.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.490+0000 7fe3a3f61700 1 -- 172.21.15.43:0/2639709663 shutdown_connections 2024-04-21T08:36:59.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.490+0000 7fe3a3f61700 1 --2- 172.21.15.43:0/2639709663 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe3880a3cd0 0x7fe3880a6190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:59.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.490+0000 7fe3a3f61700 1 --2- 172.21.15.43:0/2639709663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe39c06b170 0x7fe39c199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:59.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.490+0000 7fe3a3f61700 1 --2- 172.21.15.43:0/2639709663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe39c108a70 0x7fe39c19dae0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:59.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.490+0000 7fe3a3f61700 1 --2- 172.21.15.43:0/2639709663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe39c06bb30 0x7fe39c1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:59.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.490+0000 7fe3a3f61700 1 -- 172.21.15.43:0/2639709663 >> 172.21.15.43:0/2639709663 conn(0x7fe39c0777b0 msgr2=0x7fe39c1009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:59.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.490+0000 7fe3a3f61700 1 -- 172.21.15.43:0/2639709663 shutdown_connections 2024-04-21T08:36:59.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:36:59.490+0000 7fe3a3f61700 1 -- 172.21.15.43:0/2639709663 wait complete. 2024-04-21T08:36:59.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:36:59 smithi112 ceph-mon[31085]: pgmap v312: 129 pgs: 129 active+clean; 629 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.5 KiB/s wr, 23 op/s 2024-04-21T08:37:00.348 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:00 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2639709663' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-21T08:37:00.350 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1577030772},{"type":"v1","addr":"172.21.15.43:6801","nonce":1577030772}]},"active_addr":"172.21.15.43:6801/1577030772","active_change":"2024-04-21T08:29:13.889637+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.43:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":1174411284}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":126475132}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":4288951239}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":2382670190}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":3299368354}]}]} 2024-04-21T08:37:00.354 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-21T08:37:00.354 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-21T08:37:00.354 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-21T08:37:00.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:00 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2639709663' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-21T08:37:00.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:00 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2639709663' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-21T08:37:00.604 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:01.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:01 smithi043 ceph-mon[27978]: pgmap v313: 129 pgs: 129 active+clean; 629 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.5 KiB/s wr, 24 op/s 2024-04-21T08:37:01.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:01 smithi175 ceph-mon[31920]: pgmap v313: 129 pgs: 129 active+clean; 629 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.5 KiB/s wr, 24 op/s 2024-04-21T08:37:01.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:01 smithi112 ceph-mon[31085]: pgmap v313: 129 pgs: 129 active+clean; 629 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.5 KiB/s wr, 24 op/s 2024-04-21T08:37:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.522+0000 7f67c691a700 1 -- 172.21.15.43:0/3004302307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67c0108a60 msgr2=0x7f67c0108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.522+0000 7f67c691a700 1 --2- 172.21.15.43:0/3004302307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67c0108a60 0x7f67c0108e40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009990 tx=0x7f67b0009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.522+0000 7f67c691a700 1 -- 172.21.15.43:0/3004302307 shutdown_connections 2024-04-21T08:37:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.522+0000 7f67c691a700 1 --2- 172.21.15.43:0/3004302307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c006bbb0 0x7f67c010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.522+0000 7f67c691a700 1 --2- 172.21.15.43:0/3004302307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67c0108a60 0x7f67c0108e40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:02.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.522+0000 7f67c691a700 1 --2- 172.21.15.43:0/3004302307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67c006b1f0 0x7f67c006b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:02.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.522+0000 7f67c691a700 1 -- 172.21.15.43:0/3004302307 >> 172.21.15.43:0/3004302307 conn(0x7f67c00789e0 msgr2=0x7f67c0078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:02.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.522+0000 7f67c691a700 1 -- 172.21.15.43:0/3004302307 shutdown_connections 2024-04-21T08:37:02.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.522+0000 7f67c691a700 1 -- 172.21.15.43:0/3004302307 wait complete. 2024-04-21T08:37:02.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.522+0000 7f67c691a700 1 Processor -- start 2024-04-21T08:37:02.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.522+0000 7f67c691a700 1 -- start start 2024-04-21T08:37:02.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.523+0000 7f67c691a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c006b1f0 0x7f67c0199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:02.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.523+0000 7f67c691a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67c006bbb0 0x7f67c0199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:02.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.523+0000 7f67c691a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67c0108a60 0x7f67c019d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:02.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.523+0000 7f67c691a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f67c0112c60 con 0x7f67c006bbb0 2024-04-21T08:37:02.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.523+0000 7f67c691a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f67c0112ae0 con 0x7f67c006b1f0 2024-04-21T08:37:02.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.523+0000 7f67c691a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f67c0112de0 con 0x7f67c0108a60 2024-04-21T08:37:02.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.523+0000 7f67bf7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67c006bbb0 0x7f67c0199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:02.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.523+0000 7f67bf7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67c006bbb0 0x7f67c0199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35104/0 (socket says 172.21.15.43:35104) 2024-04-21T08:37:02.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.523+0000 7f67c4eb7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67c0108a60 0x7f67c019d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:02.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.523+0000 7f67bf7fe700 1 -- 172.21.15.43:0/206079852 learned_addr learned my addr 172.21.15.43:0/206079852 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:37:02.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.523+0000 7f67bffff700 1 --2- 172.21.15.43:0/206079852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c006b1f0 0x7f67c0199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:02.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.523+0000 7f67bf7fe700 1 -- 172.21.15.43:0/206079852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67c0108a60 msgr2=0x7f67c019d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:02.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.523+0000 7f67bf7fe700 1 --2- 172.21.15.43:0/206079852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67c0108a60 0x7f67c019d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:02.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.523+0000 7f67bf7fe700 1 -- 172.21.15.43:0/206079852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c006b1f0 msgr2=0x7f67c0199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:02.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.523+0000 7f67bf7fe700 1 --2- 172.21.15.43:0/206079852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c006b1f0 0x7f67c0199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:02.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.523+0000 7f67bf7fe700 1 -- 172.21.15.43:0/206079852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c019e0d0 con 0x7f67c006bbb0 2024-04-21T08:37:02.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.523+0000 7f67bffff700 1 --2- 172.21.15.43:0/206079852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c006b1f0 0x7f67c0199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:37:02.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.524+0000 7f67bf7fe700 1 --2- 172.21.15.43:0/206079852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67c006bbb0 0x7f67c0199650 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f67b400d980 tx=0x7f67b400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:02.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.524+0000 7f67bd7fa700 1 -- 172.21.15.43:0/206079852 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4017070 con 0x7f67c006bbb0 2024-04-21T08:37:02.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.524+0000 7f67bd7fa700 1 -- 172.21.15.43:0/206079852 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67b4005020 con 0x7f67c006bbb0 2024-04-21T08:37:02.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.524+0000 7f67c691a700 1 -- 172.21.15.43:0/206079852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c019e3c0 con 0x7f67c006bbb0 2024-04-21T08:37:02.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.524+0000 7f67bd7fa700 1 -- 172.21.15.43:0/206079852 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4009b50 con 0x7f67c006bbb0 2024-04-21T08:37:02.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.524+0000 7f67c691a700 1 -- 172.21.15.43:0/206079852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c01a5c40 con 0x7f67c006bbb0 2024-04-21T08:37:02.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.525+0000 7f67c691a700 1 -- 172.21.15.43:0/206079852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c010b060 con 0x7f67c006bbb0 2024-04-21T08:37:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.527+0000 7f67bd7fa700 1 -- 172.21.15.43:0/206079852 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f67b40051d0 con 0x7f67c006bbb0 2024-04-21T08:37:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.527+0000 7f67bd7fa700 1 --2- 172.21.15.43:0/206079852 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f67a80a3cd0 0x7f67a80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.528+0000 7f67bffff700 1 --2- 172.21.15.43:0/206079852 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f67a80a3cd0 0x7f67a80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.528+0000 7f67bd7fa700 1 -- 172.21.15.43:0/206079852 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f67b40d0af0 con 0x7f67c006bbb0 2024-04-21T08:37:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.528+0000 7f67bffff700 1 --2- 172.21.15.43:0/206079852 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f67a80a3cd0 0x7f67a80a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f67b0006010 tx=0x7f67b000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:02.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.531+0000 7f67bd7fa700 1 -- 172.21.15.43:0/206079852 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f67b4020020 con 0x7f67c006bbb0 2024-04-21T08:37:02.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.738+0000 7f67c691a700 1 -- 172.21.15.43:0/206079852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67c0069730 con 0x7f67c006bbb0 2024-04-21T08:37:02.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.740+0000 7f67bd7fa700 1 -- 172.21.15.43:0/206079852 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24152 (secure 0 0 0) 0x7f67c0069730 con 0x7f67c006bbb0 2024-04-21T08:37:02.740 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:37:02.741 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":90,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:36:28.364093+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:37:02.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.742+0000 7f67c691a700 1 -- 172.21.15.43:0/206079852 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f67a80a3cd0 msgr2=0x7f67a80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:02.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.742+0000 7f67c691a700 1 --2- 172.21.15.43:0/206079852 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f67a80a3cd0 0x7f67a80a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f67b0006010 tx=0x7f67b000b540 comp rx=0 tx=0).stop 2024-04-21T08:37:02.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.742+0000 7f67c691a700 1 -- 172.21.15.43:0/206079852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67c006bbb0 msgr2=0x7f67c0199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:02.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.742+0000 7f67c691a700 1 --2- 172.21.15.43:0/206079852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67c006bbb0 0x7f67c0199650 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f67b400d980 tx=0x7f67b400dc90 comp rx=0 tx=0).stop 2024-04-21T08:37:02.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.742+0000 7f67c691a700 1 -- 172.21.15.43:0/206079852 shutdown_connections 2024-04-21T08:37:02.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.742+0000 7f67c691a700 1 --2- 172.21.15.43:0/206079852 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f67a80a3cd0 0x7f67a80a6190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:02.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.742+0000 7f67c691a700 1 --2- 172.21.15.43:0/206079852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67c0108a60 0x7f67c019d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:02.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.742+0000 7f67c691a700 1 --2- 172.21.15.43:0/206079852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c006b1f0 0x7f67c0199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:02.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.742+0000 7f67c691a700 1 --2- 172.21.15.43:0/206079852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67c006bbb0 0x7f67c0199650 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:02.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.742+0000 7f67c691a700 1 -- 172.21.15.43:0/206079852 >> 172.21.15.43:0/206079852 conn(0x7f67c00789e0 msgr2=0x7f67c00ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:02.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.742+0000 7f67c691a700 1 -- 172.21.15.43:0/206079852 shutdown_connections 2024-04-21T08:37:02.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:02.742+0000 7f67c691a700 1 -- 172.21.15.43:0/206079852 wait complete. 2024-04-21T08:37:03.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:03 smithi175 ceph-mon[31920]: pgmap v314: 129 pgs: 129 active+clean; 629 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.1 KiB/s wr, 23 op/s 2024-04-21T08:37:03.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:03 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/206079852' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:37:03.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:03 smithi112 ceph-mon[31085]: pgmap v314: 129 pgs: 129 active+clean; 629 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.1 KiB/s wr, 23 op/s 2024-04-21T08:37:03.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:03 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/206079852' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:37:03.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:03 smithi043 ceph-mon[27978]: pgmap v314: 129 pgs: 129 active+clean; 629 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.1 KiB/s wr, 23 op/s 2024-04-21T08:37:03.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:03 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/206079852' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:37:03.662 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-21T08:37:03.662 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-21T08:37:03.916 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:05.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:05 smithi043 ceph-mon[27978]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 21 op/s 2024-04-21T08:37:05.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:05 smithi175 ceph-mon[31920]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 21 op/s 2024-04-21T08:37:05.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:05 smithi112 ceph-mon[31085]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 21 op/s 2024-04-21T08:37:05.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.961+0000 7f5950552700 1 -- 172.21.15.43:0/2587429427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5948108a70 msgr2=0x7f5948108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:05.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.961+0000 7f5950552700 1 --2- 172.21.15.43:0/2587429427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5948108a70 0x7f5948108e50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5944009990 tx=0x7f5944009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:05.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.961+0000 7f5950552700 1 -- 172.21.15.43:0/2587429427 shutdown_connections 2024-04-21T08:37:05.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.961+0000 7f5950552700 1 --2- 172.21.15.43:0/2587429427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5948103480 0x7f594810f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:05.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.961+0000 7f5950552700 1 --2- 172.21.15.43:0/2587429427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5948108a70 0x7f5948108e50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:05.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.961+0000 7f5950552700 1 --2- 172.21.15.43:0/2587429427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5948102ac0 0x7f5948102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:05.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.961+0000 7f5950552700 1 -- 172.21.15.43:0/2587429427 >> 172.21.15.43:0/2587429427 conn(0x7f5948079bb0 msgr2=0x7f5948079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:05.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.961+0000 7f5950552700 1 -- 172.21.15.43:0/2587429427 shutdown_connections 2024-04-21T08:37:05.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.961+0000 7f5950552700 1 -- 172.21.15.43:0/2587429427 wait complete. 2024-04-21T08:37:05.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.962+0000 7f5950552700 1 Processor -- start 2024-04-21T08:37:05.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.962+0000 7f5950552700 1 -- start start 2024-04-21T08:37:05.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.962+0000 7f5950552700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5948102ac0 0x7f5948199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:05.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.962+0000 7f5950552700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5948103480 0x7f5948199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:05.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.962+0000 7f5950552700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5948108a70 0x7f594819d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:05.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.962+0000 7f5950552700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5948112c80 con 0x7f5948103480 2024-04-21T08:37:05.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.962+0000 7f5950552700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5948112b00 con 0x7f5948108a70 2024-04-21T08:37:05.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.962+0000 7f5950552700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5948112e00 con 0x7f5948102ac0 2024-04-21T08:37:05.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.962+0000 7f594daed700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5948103480 0x7f5948199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:05.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.962+0000 7f594daed700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5948103480 0x7f5948199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35130/0 (socket says 172.21.15.43:35130) 2024-04-21T08:37:05.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.962+0000 7f594daed700 1 -- 172.21.15.43:0/2720369358 learned_addr learned my addr 172.21.15.43:0/2720369358 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:37:05.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.962+0000 7f594eaef700 1 --2- 172.21.15.43:0/2720369358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5948108a70 0x7f594819d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:05.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.962+0000 7f594e2ee700 1 --2- 172.21.15.43:0/2720369358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5948102ac0 0x7f5948199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:05.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.962+0000 7f594daed700 1 -- 172.21.15.43:0/2720369358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5948102ac0 msgr2=0x7f5948199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:05.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.962+0000 7f594daed700 1 --2- 172.21.15.43:0/2720369358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5948102ac0 0x7f5948199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:05.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.962+0000 7f594daed700 1 -- 172.21.15.43:0/2720369358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5948108a70 msgr2=0x7f594819d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:05.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.962+0000 7f594daed700 1 --2- 172.21.15.43:0/2720369358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5948108a70 0x7f594819d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:05.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.962+0000 7f594daed700 1 -- 172.21.15.43:0/2720369358 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594819e0d0 con 0x7f5948103480 2024-04-21T08:37:05.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.962+0000 7f594eaef700 1 --2- 172.21.15.43:0/2720369358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5948108a70 0x7f594819d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:37:05.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.963+0000 7f594daed700 1 --2- 172.21.15.43:0/2720369358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5948103480 0x7f5948199650 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f593800b530 tx=0x7f593800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:05.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.963+0000 7f593f7fe700 1 -- 172.21.15.43:0/2720369358 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5938015020 con 0x7f5948103480 2024-04-21T08:37:05.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.963+0000 7f593f7fe700 1 -- 172.21.15.43:0/2720369358 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5938011ce0 con 0x7f5948103480 2024-04-21T08:37:05.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.963+0000 7f5950552700 1 -- 172.21.15.43:0/2720369358 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f594819e3c0 con 0x7f5948103480 2024-04-21T08:37:05.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.963+0000 7f593f7fe700 1 -- 172.21.15.43:0/2720369358 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5938010d20 con 0x7f5948103480 2024-04-21T08:37:05.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.963+0000 7f5950552700 1 -- 172.21.15.43:0/2720369358 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59481a5c40 con 0x7f5948103480 2024-04-21T08:37:05.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.964+0000 7f5950552700 1 -- 172.21.15.43:0/2720369358 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5948069730 con 0x7f5948103480 2024-04-21T08:37:05.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.965+0000 7f593f7fe700 1 -- 172.21.15.43:0/2720369358 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5938018020 con 0x7f5948103480 2024-04-21T08:37:05.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.966+0000 7f593f7fe700 1 --2- 172.21.15.43:0/2720369358 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f59340a3cd0 0x7f59340a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:05.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.966+0000 7f594e2ee700 1 --2- 172.21.15.43:0/2720369358 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f59340a3cd0 0x7f59340a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:05.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.966+0000 7f593f7fe700 1 -- 172.21.15.43:0/2720369358 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f59380d04a0 con 0x7f5948103480 2024-04-21T08:37:05.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.966+0000 7f594e2ee700 1 --2- 172.21.15.43:0/2720369358 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f59340a3cd0 0x7f59340a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5944005cd0 tx=0x7f5944005c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:05.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:05.969+0000 7f593f7fe700 1 -- 172.21.15.43:0/2720369358 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f593801f020 con 0x7f5948103480 2024-04-21T08:37:06.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:06.179+0000 7f5950552700 1 -- 172.21.15.43:0/2720369358 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f594802df80 con 0x7f5948103480 2024-04-21T08:37:06.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:06.180+0000 7f593f7fe700 1 -- 172.21.15.43:0/2720369358 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24152 (secure 0 0 0) 0x7f593809cb90 con 0x7f5948103480 2024-04-21T08:37:06.180 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:37:06.180 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":90,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:36:28.364093+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:37:06.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:06.183+0000 7f5950552700 1 -- 172.21.15.43:0/2720369358 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f59340a3cd0 msgr2=0x7f59340a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:06.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:06.183+0000 7f5950552700 1 --2- 172.21.15.43:0/2720369358 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f59340a3cd0 0x7f59340a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5944005cd0 tx=0x7f5944005c40 comp rx=0 tx=0).stop 2024-04-21T08:37:06.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:06.183+0000 7f5950552700 1 -- 172.21.15.43:0/2720369358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5948103480 msgr2=0x7f5948199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:06.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:06.183+0000 7f5950552700 1 --2- 172.21.15.43:0/2720369358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5948103480 0x7f5948199650 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f593800b530 tx=0x7f593800b8f0 comp rx=0 tx=0).stop 2024-04-21T08:37:06.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:06.183+0000 7f5950552700 1 -- 172.21.15.43:0/2720369358 shutdown_connections 2024-04-21T08:37:06.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:06.183+0000 7f5950552700 1 --2- 172.21.15.43:0/2720369358 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f59340a3cd0 0x7f59340a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:06.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:06.183+0000 7f5950552700 1 --2- 172.21.15.43:0/2720369358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5948102ac0 0x7f5948199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:06.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:06.183+0000 7f5950552700 1 --2- 172.21.15.43:0/2720369358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5948108a70 0x7f594819d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:06.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:06.183+0000 7f5950552700 1 --2- 172.21.15.43:0/2720369358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5948103480 0x7f5948199650 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:06.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:06.183+0000 7f5950552700 1 -- 172.21.15.43:0/2720369358 >> 172.21.15.43:0/2720369358 conn(0x7f5948079bb0 msgr2=0x7f5948101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:06.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:06.183+0000 7f5950552700 1 -- 172.21.15.43:0/2720369358 shutdown_connections 2024-04-21T08:37:06.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:06.183+0000 7f5950552700 1 -- 172.21.15.43:0/2720369358 wait complete. 2024-04-21T08:37:06.906 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-21T08:37:06.906 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-21T08:37:06.906 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-21T08:37:06.907 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-21T08:37:06.907 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-21T08:37:06.908 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-21T08:37:06.908 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-21T08:37:06.908 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-21T08:37:06.909 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-21T08:37:06.909 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-21T08:37:06.910 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-21T08:37:06.910 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-21T08:37:07.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:07 smithi043 ceph-mon[27978]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 2.5 KiB/s wr, 15 op/s 2024-04-21T08:37:07.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:07 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2720369358' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:37:07.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:07 smithi175 ceph-mon[31920]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 2.5 KiB/s wr, 15 op/s 2024-04-21T08:37:07.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:07 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2720369358' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:37:07.525 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:07.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:07 smithi112 ceph-mon[31085]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 2.5 KiB/s wr, 15 op/s 2024-04-21T08:37:07.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:07 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2720369358' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:37:07.621 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:07.624 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:07.861 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:09.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:09 smithi043 ceph-mon[27978]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-04-21T08:37:09.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:09 smithi175 ceph-mon[31920]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-04-21T08:37:09.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:09 smithi112 ceph-mon[31085]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-04-21T08:37:11.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:11 smithi175 ceph-mon[31920]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-21T08:37:11.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:11 smithi112 ceph-mon[31085]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-21T08:37:11.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:11 smithi043 ceph-mon[27978]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-21T08:37:11.600 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.461+0000 7f80abedb700 1 -- 172.21.15.43:0/1089603296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a4101940 msgr2=0x7f80a4101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.461+0000 7f80abedb700 1 --2- 172.21.15.43:0/1089603296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a4101940 0x7f80a4101dc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8094009990 tx=0x7f8094009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.461+0000 7f80abedb700 1 -- 172.21.15.43:0/1089603296 shutdown_connections 2024-04-21T08:37:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.461+0000 7f80abedb700 1 --2- 172.21.15.43:0/1089603296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80a4102300 0x7f80a410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.461+0000 7f80abedb700 1 --2- 172.21.15.43:0/1089603296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80a4107990 0x7f80a4107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.461+0000 7f80abedb700 1 --2- 172.21.15.43:0/1089603296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a4101940 0x7f80a4101dc0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.461+0000 7f80abedb700 1 -- 172.21.15.43:0/1089603296 >> 172.21.15.43:0/1089603296 conn(0x7f80a40788f0 msgr2=0x7f80a4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.461+0000 7f80abedb700 1 -- 172.21.15.43:0/1089603296 shutdown_connections 2024-04-21T08:37:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.461+0000 7f80abedb700 1 -- 172.21.15.43:0/1089603296 wait complete. 2024-04-21T08:37:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.462+0000 7f80abedb700 1 Processor -- start 2024-04-21T08:37:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.462+0000 7f80abedb700 1 -- start start 2024-04-21T08:37:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.462+0000 7f80abedb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a4101940 0x7f80a4199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.462+0000 7f80abedb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80a4102300 0x7f80a419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.462+0000 7f80abedb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80a4107990 0x7f80a419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.462+0000 7f80abedb700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f80a4111af0 con 0x7f80a4101940 2024-04-21T08:37:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.462+0000 7f80abedb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80a4111970 con 0x7f80a4102300 2024-04-21T08:37:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.462+0000 7f80abedb700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80a4111c70 con 0x7f80a4107990 2024-04-21T08:37:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.462+0000 7f80a9c77700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a4101940 0x7f80a4199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.462+0000 7f80a9c77700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a4101940 0x7f80a4199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51244/0 (socket says 172.21.15.43:51244) 2024-04-21T08:37:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.462+0000 7f80a9c77700 1 -- 172.21.15.43:0/4258024006 learned_addr learned my addr 172.21.15.43:0/4258024006 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:37:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.462+0000 7f80a9476700 1 --2- 172.21.15.43:0/4258024006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80a4102300 0x7f80a419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.462+0000 7f80aa478700 1 --2- 172.21.15.43:0/4258024006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80a4107990 0x7f80a419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.463+0000 7f80a9c77700 1 -- 172.21.15.43:0/4258024006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80a4107990 msgr2=0x7f80a419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.463+0000 7f80a9c77700 1 --2- 172.21.15.43:0/4258024006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80a4107990 0x7f80a419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.463+0000 7f80a9c77700 1 -- 172.21.15.43:0/4258024006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80a4102300 msgr2=0x7f80a419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.463+0000 7f80a9c77700 1 --2- 172.21.15.43:0/4258024006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80a4102300 0x7f80a419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.463+0000 7f80a9c77700 1 -- 172.21.15.43:0/4258024006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a41a53a0 con 0x7f80a4101940 2024-04-21T08:37:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.463+0000 7f80aa478700 1 --2- 172.21.15.43:0/4258024006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80a4107990 0x7f80a419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:37:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.463+0000 7f80a9c77700 1 --2- 172.21.15.43:0/4258024006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a4101940 0x7f80a4199fb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f80a000c9b0 tx=0x7f80a000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.463+0000 7f809affd700 1 -- 172.21.15.43:0/4258024006 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0007b00 con 0x7f80a4101940 2024-04-21T08:37:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.463+0000 7f80abedb700 1 -- 172.21.15.43:0/4258024006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a41a5690 con 0x7f80a4101940 2024-04-21T08:37:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.463+0000 7f809affd700 1 -- 172.21.15.43:0/4258024006 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80a0007ca0 con 0x7f80a4101940 2024-04-21T08:37:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.463+0000 7f809affd700 1 -- 172.21.15.43:0/4258024006 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0018660 con 0x7f80a4101940 2024-04-21T08:37:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.463+0000 7f80abedb700 1 -- 172.21.15.43:0/4258024006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a41a5ba0 con 0x7f80a4101940 2024-04-21T08:37:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.464+0000 7f80abedb700 1 -- 172.21.15.43:0/4258024006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f80a40630d0 con 0x7f80a4101940 2024-04-21T08:37:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.465+0000 7f809affd700 1 -- 172.21.15.43:0/4258024006 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f80a000f450 con 0x7f80a4101940 2024-04-21T08:37:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.466+0000 7f809affd700 1 --2- 172.21.15.43:0/4258024006 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f80900a3cd0 0x7f80900a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.466+0000 7f809affd700 1 -- 172.21.15.43:0/4258024006 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f80a00d0b60 con 0x7f80a4101940 2024-04-21T08:37:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.466+0000 7f80a9476700 1 --2- 172.21.15.43:0/4258024006 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f80900a3cd0 0x7f80900a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.466+0000 7f809affd700 1 --2- 172.21.15.43:0/4258024006 >> [v2:172.21.15.175:6816/1369698308,v1:172.21.15.175:6817/1369698308] conn(0x7f80900ae990 0x7f80900b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.466+0000 7f809affd700 1 -- 172.21.15.43:0/4258024006 --> [v2:172.21.15.175:6816/1369698308,v1:172.21.15.175:6817/1369698308] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f80900b14a0 con 0x7f80900ae990 2024-04-21T08:37:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.466+0000 7f809affd700 1 -- 172.21.15.43:0/4258024006 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f80a00d0fb0 con 0x7f80a4101940 2024-04-21T08:37:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.466+0000 7f80a9476700 1 --2- 172.21.15.43:0/4258024006 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f80900a3cd0 0x7f80900a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f809400b5c0 tx=0x7f8094005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.466+0000 7f80aa478700 1 --2- 172.21.15.43:0/4258024006 >> [v2:172.21.15.175:6816/1369698308,v1:172.21.15.175:6817/1369698308] conn(0x7f80900ae990 0x7f80900b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.467+0000 7f80aa478700 1 --2- 172.21.15.43:0/4258024006 >> [v2:172.21.15.175:6816/1369698308,v1:172.21.15.175:6817/1369698308] conn(0x7f80900ae990 0x7f80900b0dd0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.468+0000 7f809affd700 1 -- 172.21.15.43:0/4258024006 <== osd.8 v2:172.21.15.175:6816/1369698308 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f80900b14a0 con 0x7f80900ae990 2024-04-21T08:37:12.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.484+0000 7f80abedb700 1 -- 172.21.15.43:0/4258024006 --> [v2:172.21.15.175:6816/1369698308,v1:172.21.15.175:6817/1369698308] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f80a4063550 con 0x7f80900ae990 2024-04-21T08:37:12.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.485+0000 7f809affd700 1 -- 172.21.15.43:0/4258024006 <== osd.8 v2:172.21.15.175:6816/1369698308 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f80a4063550 con 0x7f80900ae990 2024-04-21T08:37:12.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.485+0000 7f80abedb700 1 -- 172.21.15.43:0/4258024006 >> [v2:172.21.15.175:6816/1369698308,v1:172.21.15.175:6817/1369698308] conn(0x7f80900ae990 msgr2=0x7f80900b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:12.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.485+0000 7f80abedb700 1 --2- 172.21.15.43:0/4258024006 >> [v2:172.21.15.175:6816/1369698308,v1:172.21.15.175:6817/1369698308] conn(0x7f80900ae990 0x7f80900b0dd0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:12.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.485+0000 7f80abedb700 1 -- 172.21.15.43:0/4258024006 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f80900a3cd0 msgr2=0x7f80900a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:12.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.485+0000 7f80abedb700 1 --2- 172.21.15.43:0/4258024006 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f80900a3cd0 0x7f80900a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f809400b5c0 tx=0x7f8094005960 comp rx=0 tx=0).stop 2024-04-21T08:37:12.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.485+0000 7f80abedb700 1 -- 172.21.15.43:0/4258024006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a4101940 msgr2=0x7f80a4199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:12.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.485+0000 7f80abedb700 1 --2- 172.21.15.43:0/4258024006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a4101940 0x7f80a4199fb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f80a000c9b0 tx=0x7f80a000ccc0 comp rx=0 tx=0).stop 2024-04-21T08:37:12.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.485+0000 7f80aa478700 1 -- 172.21.15.43:0/4258024006 reap_dead start 2024-04-21T08:37:12.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.486+0000 7f80abedb700 1 -- 172.21.15.43:0/4258024006 shutdown_connections 2024-04-21T08:37:12.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.486+0000 7f80abedb700 1 -- 172.21.15.43:0/4258024006 >> 172.21.15.43:0/4258024006 conn(0x7f80a40788f0 msgr2=0x7f80a40ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:12.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.486+0000 7f80abedb700 1 -- 172.21.15.43:0/4258024006 shutdown_connections 2024-04-21T08:37:12.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.486+0000 7f80abedb700 1 -- 172.21.15.43:0/4258024006 wait complete. 2024-04-21T08:37:12.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.794+0000 7f7ccc6d3700 1 -- 172.21.15.43:0/2604650171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cc4103450 msgr2=0x7f7cc410f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:12.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.794+0000 7f7ccc6d3700 1 --2- 172.21.15.43:0/2604650171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cc4103450 0x7f7cc410f8b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00b0a0 tx=0x7f7cbc00b3b0 comp rx=0 tx=0).stop 2024-04-21T08:37:12.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.794+0000 7f7ccc6d3700 1 -- 172.21.15.43:0/2604650171 shutdown_connections 2024-04-21T08:37:12.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.794+0000 7f7ccc6d3700 1 --2- 172.21.15.43:0/2604650171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cc4108a50 0x7f7cc4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:12.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.794+0000 7f7ccc6d3700 1 --2- 172.21.15.43:0/2604650171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cc4103450 0x7f7cc410f8b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:12.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.794+0000 7f7ccc6d3700 1 --2- 172.21.15.43:0/2604650171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc4102a90 0x7f7cc4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:12.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.794+0000 7f7ccc6d3700 1 -- 172.21.15.43:0/2604650171 >> 172.21.15.43:0/2604650171 conn(0x7f7cc4079ba0 msgr2=0x7f7cc4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:12.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.794+0000 7f7ccc6d3700 1 -- 172.21.15.43:0/2604650171 shutdown_connections 2024-04-21T08:37:12.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.794+0000 7f7ccc6d3700 1 -- 172.21.15.43:0/2604650171 wait complete. 2024-04-21T08:37:12.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.795+0000 7f7ccc6d3700 1 Processor -- start 2024-04-21T08:37:12.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.795+0000 7f7ccc6d3700 1 -- start start 2024-04-21T08:37:12.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.795+0000 7f7ccc6d3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cc4102a90 0x7f7cc4198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:12.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.795+0000 7f7ccc6d3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cc4103450 0x7f7cc41994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:12.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.795+0000 7f7ccc6d3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc4108a50 0x7f7cc419d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:12.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.795+0000 7f7ccc6d3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc4112be0 con 0x7f7cc4103450 2024-04-21T08:37:12.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.795+0000 7f7ccc6d3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc4112a60 con 0x7f7cc4108a50 2024-04-21T08:37:12.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.795+0000 7f7ccc6d3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc4112d60 con 0x7f7cc4102a90 2024-04-21T08:37:12.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.795+0000 7f7cc9c6e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cc4103450 0x7f7cc41994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:12.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.795+0000 7f7cc9c6e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cc4103450 0x7f7cc41994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51254/0 (socket says 172.21.15.43:51254) 2024-04-21T08:37:12.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.795+0000 7f7cc9c6e700 1 -- 172.21.15.43:0/2475174084 learned_addr learned my addr 172.21.15.43:0/2475174084 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:37:12.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.795+0000 7f7ccac70700 1 --2- 172.21.15.43:0/2475174084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc4108a50 0x7f7cc419d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:12.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.795+0000 7f7cca46f700 1 --2- 172.21.15.43:0/2475174084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cc4102a90 0x7f7cc4198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:12.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.796+0000 7f7cc9c6e700 1 -- 172.21.15.43:0/2475174084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cc4102a90 msgr2=0x7f7cc4198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:12.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.796+0000 7f7cc9c6e700 1 --2- 172.21.15.43:0/2475174084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cc4102a90 0x7f7cc4198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:12.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.796+0000 7f7cc9c6e700 1 -- 172.21.15.43:0/2475174084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc4108a50 msgr2=0x7f7cc419d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:12.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.796+0000 7f7cc9c6e700 1 --2- 172.21.15.43:0/2475174084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc4108a50 0x7f7cc419d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:12.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.796+0000 7f7cc9c6e700 1 -- 172.21.15.43:0/2475174084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc419df50 con 0x7f7cc4103450 2024-04-21T08:37:12.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.796+0000 7f7ccac70700 1 --2- 172.21.15.43:0/2475174084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc4108a50 0x7f7cc419d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:37:12.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.796+0000 7f7cc9c6e700 1 --2- 172.21.15.43:0/2475174084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cc4103450 0x7f7cc41994d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800c9b0 tx=0x7f7cb800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:12.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.796+0000 7f7cb77fe700 1 -- 172.21.15.43:0/2475174084 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8007b00 con 0x7f7cc4103450 2024-04-21T08:37:12.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.796+0000 7f7cb77fe700 1 -- 172.21.15.43:0/2475174084 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cb8007ca0 con 0x7f7cc4103450 2024-04-21T08:37:12.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.796+0000 7f7ccc6d3700 1 -- 172.21.15.43:0/2475174084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc419e240 con 0x7f7cc4103450 2024-04-21T08:37:12.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.796+0000 7f7cb77fe700 1 -- 172.21.15.43:0/2475174084 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8018730 con 0x7f7cc4103450 2024-04-21T08:37:12.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.796+0000 7f7ccc6d3700 1 -- 172.21.15.43:0/2475174084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc41a5ac0 con 0x7f7cc4103450 2024-04-21T08:37:12.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.797+0000 7f7ccc6d3700 1 -- 172.21.15.43:0/2475174084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7cc40631d0 con 0x7f7cc4103450 2024-04-21T08:37:12.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.798+0000 7f7cb77fe700 1 -- 172.21.15.43:0/2475174084 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7cb800f450 con 0x7f7cc4103450 2024-04-21T08:37:12.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.799+0000 7f7cb77fe700 1 --2- 172.21.15.43:0/2475174084 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f7cb00a3d50 0x7f7cb00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:12.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.799+0000 7f7cca46f700 1 --2- 172.21.15.43:0/2475174084 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f7cb00a3d50 0x7f7cb00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:12.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.799+0000 7f7cb77fe700 1 -- 172.21.15.43:0/2475174084 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f7cb80d0ce0 con 0x7f7cc4103450 2024-04-21T08:37:12.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.799+0000 7f7cb77fe700 1 --2- 172.21.15.43:0/2475174084 >> [v2:172.21.15.112:6800/2362092039,v1:172.21.15.112:6801/2362092039] conn(0x7f7cb00aea10 0x7f7cb00b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:12.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.799+0000 7f7cb77fe700 1 -- 172.21.15.43:0/2475174084 --> [v2:172.21.15.112:6800/2362092039,v1:172.21.15.112:6801/2362092039] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7cb8007e50 con 0x7f7cb00aea10 2024-04-21T08:37:12.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.799+0000 7f7cb77fe700 1 -- 172.21.15.43:0/2475174084 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f7cb80d1090 con 0x7f7cc4103450 2024-04-21T08:37:12.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.799+0000 7f7cca46f700 1 --2- 172.21.15.43:0/2475174084 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f7cb00a3d50 0x7f7cb00a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0005fd0 tx=0x7f7cc000af20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:12.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.799+0000 7f7ccac70700 1 --2- 172.21.15.43:0/2475174084 >> [v2:172.21.15.112:6800/2362092039,v1:172.21.15.112:6801/2362092039] conn(0x7f7cb00aea10 0x7f7cb00b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:12.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.800+0000 7f7ccac70700 1 --2- 172.21.15.43:0/2475174084 >> [v2:172.21.15.112:6800/2362092039,v1:172.21.15.112:6801/2362092039] conn(0x7f7cb00aea10 0x7f7cb00b0e50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:12.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.801+0000 7f7cb77fe700 1 -- 172.21.15.43:0/2475174084 <== osd.1 v2:172.21.15.112:6800/2362092039 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7cb8007e50 con 0x7f7cb00aea10 2024-04-21T08:37:12.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.817+0000 7f7ccc6d3700 1 -- 172.21.15.43:0/2475174084 --> [v2:172.21.15.112:6800/2362092039,v1:172.21.15.112:6801/2362092039] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7cc40635d0 con 0x7f7cb00aea10 2024-04-21T08:37:12.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.818+0000 7f7cb77fe700 1 -- 172.21.15.43:0/2475174084 <== osd.1 v2:172.21.15.112:6800/2362092039 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7cc40635d0 con 0x7f7cb00aea10 2024-04-21T08:37:12.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.818+0000 7f7ccc6d3700 1 -- 172.21.15.43:0/2475174084 >> [v2:172.21.15.112:6800/2362092039,v1:172.21.15.112:6801/2362092039] conn(0x7f7cb00aea10 msgr2=0x7f7cb00b0e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.818+0000 7f7ccc6d3700 1 --2- 172.21.15.43:0/2475174084 >> [v2:172.21.15.112:6800/2362092039,v1:172.21.15.112:6801/2362092039] conn(0x7f7cb00aea10 0x7f7cb00b0e50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.818+0000 7f7ccc6d3700 1 -- 172.21.15.43:0/2475174084 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f7cb00a3d50 msgr2=0x7f7cb00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.818+0000 7f7ccc6d3700 1 --2- 172.21.15.43:0/2475174084 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f7cb00a3d50 0x7f7cb00a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0005fd0 tx=0x7f7cc000af20 comp rx=0 tx=0).stop 2024-04-21T08:37:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.818+0000 7f7ccc6d3700 1 -- 172.21.15.43:0/2475174084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cc4103450 msgr2=0x7f7cc41994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.818+0000 7f7ccc6d3700 1 --2- 172.21.15.43:0/2475174084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cc4103450 0x7f7cc41994d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800c9b0 tx=0x7f7cb800cd70 comp rx=0 tx=0).stop 2024-04-21T08:37:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.818+0000 7f7ccac70700 1 -- 172.21.15.43:0/2475174084 reap_dead start 2024-04-21T08:37:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.818+0000 7f7ccc6d3700 1 -- 172.21.15.43:0/2475174084 shutdown_connections 2024-04-21T08:37:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.818+0000 7f7ccc6d3700 1 -- 172.21.15.43:0/2475174084 >> 172.21.15.43:0/2475174084 conn(0x7f7cc4079ba0 msgr2=0x7f7cc4101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.818+0000 7f7ccc6d3700 1 -- 172.21.15.43:0/2475174084 shutdown_connections 2024-04-21T08:37:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:12.818+0000 7f7ccc6d3700 1 -- 172.21.15.43:0/2475174084 wait complete. 2024-04-21T08:37:13.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:13 smithi043 ceph-mon[27978]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-21T08:37:13.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:13 smithi175 ceph-mon[31920]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-21T08:37:13.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:13 smithi112 ceph-mon[31085]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-21T08:37:13.850 INFO:teuthology.orchestra.run.smithi043.stdout:257698037799 2024-04-21T08:37:13.851 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-21T08:37:14.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.282+0000 7f0a3685d700 1 -- 172.21.15.43:0/1031133380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a301034b0 msgr2=0x7f0a3010f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:14.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.282+0000 7f0a3685d700 1 --2- 172.21.15.43:0/1031133380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a301034b0 0x7f0a3010f920 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800b0d0 tx=0x7f0a2800b3e0 comp rx=0 tx=0).stop 2024-04-21T08:37:14.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.282+0000 7f0a3685d700 1 -- 172.21.15.43:0/1031133380 shutdown_connections 2024-04-21T08:37:14.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.282+0000 7f0a3685d700 1 --2- 172.21.15.43:0/1031133380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a301034b0 0x7f0a3010f920 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:14.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.282+0000 7f0a3685d700 1 --2- 172.21.15.43:0/1031133380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a30107940 0x7f0a30107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:14.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.282+0000 7f0a3685d700 1 --2- 172.21.15.43:0/1031133380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a30102af0 0x7f0a30102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:14.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.282+0000 7f0a3685d700 1 -- 172.21.15.43:0/1031133380 >> 172.21.15.43:0/1031133380 conn(0x7f0a30079ba0 msgr2=0x7f0a30079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:14.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.282+0000 7f0a3685d700 1 -- 172.21.15.43:0/1031133380 shutdown_connections 2024-04-21T08:37:14.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.283+0000 7f0a3685d700 1 -- 172.21.15.43:0/1031133380 wait complete. 2024-04-21T08:37:14.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.283+0000 7f0a3685d700 1 Processor -- start 2024-04-21T08:37:14.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.283+0000 7f0a3685d700 1 -- start start 2024-04-21T08:37:14.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.284+0000 7f0a3685d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a30102af0 0x7f0a300762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:14.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.284+0000 7f0a3685d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a301034b0 0x7f0a30076820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:14.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.284+0000 7f0a3685d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a30107940 0x7f0a3006e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:14.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.284+0000 7f0a3685d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3006cd40 con 0x7f0a30102af0 2024-04-21T08:37:14.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.284+0000 7f0a3685d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3006cbc0 con 0x7f0a30107940 2024-04-21T08:37:14.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.284+0000 7f0a3685d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3006cec0 con 0x7f0a301034b0 2024-04-21T08:37:14.284 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:14.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.284+0000 7f0a2f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a301034b0 0x7f0a30076820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:14.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.284+0000 7f0a2f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a301034b0 0x7f0a30076820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.43:53018/0 (socket says 172.21.15.43:53018) 2024-04-21T08:37:14.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.284+0000 7f0a2f7fe700 1 -- 172.21.15.43:0/568869297 learned_addr learned my addr 172.21.15.43:0/568869297 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:37:14.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.285+0000 7f0a34dfa700 1 --2- 172.21.15.43:0/568869297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a30107940 0x7f0a3006e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:14.285 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:14.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.285+0000 7f0a2ffff700 1 --2- 172.21.15.43:0/568869297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a30102af0 0x7f0a300762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:14.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.286+0000 7f0a2ffff700 1 -- 172.21.15.43:0/568869297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a301034b0 msgr2=0x7f0a30076820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:14.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.286+0000 7f0a2ffff700 1 --2- 172.21.15.43:0/568869297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a301034b0 0x7f0a30076820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:14.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.286+0000 7f0a2ffff700 1 -- 172.21.15.43:0/568869297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a30107940 msgr2=0x7f0a3006e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:14.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.286+0000 7f0a2ffff700 1 --2- 172.21.15.43:0/568869297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a30107940 0x7f0a3006e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:14.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.286+0000 7f0a2ffff700 1 -- 172.21.15.43:0/568869297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3006ebd0 con 0x7f0a30102af0 2024-04-21T08:37:14.287 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:14.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.286+0000 7f0a2ffff700 1 --2- 172.21.15.43:0/568869297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a30102af0 0x7f0a300762e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000e9f0 tx=0x7f0a2000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:14.287 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:14.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.287+0000 7f0a2d7fa700 1 -- 172.21.15.43:0/568869297 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2000cbd0 con 0x7f0a30102af0 2024-04-21T08:37:14.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.287+0000 7f0a2d7fa700 1 -- 172.21.15.43:0/568869297 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a2000cd70 con 0x7f0a30102af0 2024-04-21T08:37:14.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.287+0000 7f0a2d7fa700 1 -- 172.21.15.43:0/568869297 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a20018680 con 0x7f0a30102af0 2024-04-21T08:37:14.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.288+0000 7f0a3685d700 1 -- 172.21.15.43:0/568869297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a3006eec0 con 0x7f0a30102af0 2024-04-21T08:37:14.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.288+0000 7f0a3685d700 1 -- 172.21.15.43:0/568869297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a301ac350 con 0x7f0a30102af0 2024-04-21T08:37:14.290 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:14.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.291+0000 7f0a3685d700 1 -- 172.21.15.43:0/568869297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0a301013a0 con 0x7f0a30102af0 2024-04-21T08:37:14.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.292+0000 7f0a2d7fa700 1 -- 172.21.15.43:0/568869297 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0a20010430 con 0x7f0a30102af0 2024-04-21T08:37:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.293+0000 7f0a2d7fa700 1 --2- 172.21.15.43:0/568869297 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0a180a3d50 0x7f0a180a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:14.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.294+0000 7f0a2d7fa700 1 -- 172.21.15.43:0/568869297 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0a200d0b30 con 0x7f0a30102af0 2024-04-21T08:37:14.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.294+0000 7f0a2d7fa700 1 --2- 172.21.15.43:0/568869297 >> [v2:172.21.15.175:6808/1901076703,v1:172.21.15.175:6809/1901076703] conn(0x7f0a180aeeb0 0x7f0a180b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:14.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.294+0000 7f0a2d7fa700 1 -- 172.21.15.43:0/568869297 --> [v2:172.21.15.175:6808/1901076703,v1:172.21.15.175:6809/1901076703] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0a200056b0 con 0x7f0a180aeeb0 2024-04-21T08:37:14.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.294+0000 7f0a2d7fa700 1 -- 172.21.15.43:0/568869297 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f0a200d52c0 con 0x7f0a30102af0 2024-04-21T08:37:14.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.294+0000 7f0a2f7fe700 1 --2- 172.21.15.43:0/568869297 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0a180a3d50 0x7f0a180a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:14.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.295+0000 7f0a34dfa700 1 --2- 172.21.15.43:0/568869297 >> [v2:172.21.15.175:6808/1901076703,v1:172.21.15.175:6809/1901076703] conn(0x7f0a180aeeb0 0x7f0a180b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.295+0000 7f0a2f7fe700 1 --2- 172.21.15.43:0/568869297 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0a180a3d50 0x7f0a180a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0a3006fa70 tx=0x7f0a24009290 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.296+0000 7f0a34dfa700 1 --2- 172.21.15.43:0/568869297 >> [v2:172.21.15.175:6808/1901076703,v1:172.21.15.175:6809/1901076703] conn(0x7f0a180aeeb0 0x7f0a180b1350 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:14.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.296+0000 7f0a2d7fa700 1 -- 172.21.15.43:0/568869297 <== osd.5 v2:172.21.15.175:6808/1901076703 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a200056b0 con 0x7f0a180aeeb0 2024-04-21T08:37:14.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.323+0000 7f0a3685d700 1 -- 172.21.15.43:0/568869297 --> [v2:172.21.15.175:6808/1901076703,v1:172.21.15.175:6809/1901076703] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0a301041b0 con 0x7f0a180aeeb0 2024-04-21T08:37:14.323 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-21T08:37:14.323 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-21T08:37:14.323 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-21T08:37:14.323 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-21T08:37:14.323 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-21T08:37:14.323 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-21T08:37:14.323 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-04-21T08:37:14.323 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-21T08:37:14.323 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-21T08:37:14.324 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-04-21T08:37:14.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.324+0000 7f0a2d7fa700 1 -- 172.21.15.43:0/568869297 <== osd.5 v2:172.21.15.175:6808/1901076703 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0a301041b0 con 0x7f0a180aeeb0 2024-04-21T08:37:14.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.325+0000 7f0a3685d700 1 -- 172.21.15.43:0/568869297 >> [v2:172.21.15.175:6808/1901076703,v1:172.21.15.175:6809/1901076703] conn(0x7f0a180aeeb0 msgr2=0x7f0a180b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:14.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.325+0000 7f0a3685d700 1 --2- 172.21.15.43:0/568869297 >> [v2:172.21.15.175:6808/1901076703,v1:172.21.15.175:6809/1901076703] conn(0x7f0a180aeeb0 0x7f0a180b1350 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:14.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.325+0000 7f0a3685d700 1 -- 172.21.15.43:0/568869297 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0a180a3d50 msgr2=0x7f0a180a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:14.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.325+0000 7f0a3685d700 1 --2- 172.21.15.43:0/568869297 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0a180a3d50 0x7f0a180a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0a3006fa70 tx=0x7f0a24009290 comp rx=0 tx=0).stop 2024-04-21T08:37:14.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.326+0000 7f0a3685d700 1 -- 172.21.15.43:0/568869297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a30102af0 msgr2=0x7f0a300762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:14.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.326+0000 7f0a3685d700 1 --2- 172.21.15.43:0/568869297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a30102af0 0x7f0a300762e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000e9f0 tx=0x7f0a2000ed00 comp rx=0 tx=0).stop 2024-04-21T08:37:14.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.326+0000 7f0a34dfa700 1 -- 172.21.15.43:0/568869297 reap_dead start 2024-04-21T08:37:14.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.326+0000 7f0a3685d700 1 -- 172.21.15.43:0/568869297 shutdown_connections 2024-04-21T08:37:14.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.326+0000 7f0a3685d700 1 -- 172.21.15.43:0/568869297 >> 172.21.15.43:0/568869297 conn(0x7f0a30079ba0 msgr2=0x7f0a301009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:14.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.328+0000 7f0a3685d700 1 -- 172.21.15.43:0/568869297 shutdown_connections 2024-04-21T08:37:14.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:14.329+0000 7f0a3685d700 1 -- 172.21.15.43:0/568869297 wait complete. 2024-04-21T08:37:14.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-21T08:37:14.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-21T08:37:14.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-21T08:37:14.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-21T08:37:14.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-21T08:37:14.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-21T08:37:14.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-04-21T08:37:14.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-21T08:37:14.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-21T08:37:14.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-04-21T08:37:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-21T08:37:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-21T08:37:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-21T08:37:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-21T08:37:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-21T08:37:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-21T08:37:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-04-21T08:37:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-21T08:37:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-21T08:37:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-04-21T08:37:15.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:15 smithi175 ceph-mon[31920]: pgmap v320: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-21T08:37:15.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-21T08:37:15.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-21T08:37:15.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-21T08:37:15.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-21T08:37:15.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-21T08:37:15.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-21T08:37:15.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-04-21T08:37:15.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-21T08:37:15.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-21T08:37:15.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-04-21T08:37:15.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:15 smithi175 ceph-mon[31920]: osdmap e91: 12 total, 12 up, 12 in 2024-04-21T08:37:15.472 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:15 smithi043 ceph-mon[27978]: pgmap v320: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-21T08:37:15.472 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-21T08:37:15.472 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-21T08:37:15.472 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-21T08:37:15.472 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-21T08:37:15.472 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-21T08:37:15.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-21T08:37:15.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-04-21T08:37:15.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-21T08:37:15.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-21T08:37:15.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-04-21T08:37:15.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:15 smithi043 ceph-mon[27978]: osdmap e91: 12 total, 12 up, 12 in 2024-04-21T08:37:15.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:15 smithi112 ceph-mon[31085]: pgmap v320: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-21T08:37:15.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-21T08:37:15.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-21T08:37:15.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-21T08:37:15.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-21T08:37:15.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-21T08:37:15.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-21T08:37:15.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-04-21T08:37:15.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-21T08:37:15.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-21T08:37:15.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-04-21T08:37:15.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:15 smithi112 ceph-mon[31085]: osdmap e91: 12 total, 12 up, 12 in 2024-04-21T08:37:16.460 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:16 smithi043 ceph-mon[27978]: osdmap e92: 12 total, 12 up, 12 in 2024-04-21T08:37:16.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:16 smithi175 ceph-mon[31920]: osdmap e92: 12 total, 12 up, 12 in 2024-04-21T08:37:16.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:16 smithi112 ceph-mon[31085]: osdmap e92: 12 total, 12 up, 12 in 2024-04-21T08:37:17.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:17 smithi175 ceph-mon[31920]: pgmap v323: 129 pgs: 1 peering, 128 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-21T08:37:17.526 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:17 smithi043 ceph-mon[27978]: pgmap v323: 129 pgs: 1 peering, 128 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-21T08:37:17.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:17 smithi112 ceph-mon[31085]: pgmap v323: 129 pgs: 1 peering, 128 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-21T08:37:18.002 INFO:teuthology.orchestra.run.smithi043.stdout:167503724599 2024-04-21T08:37:18.002 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-21T08:37:19.224 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:19 smithi043 ceph-mon[27978]: pgmap v324: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:37:19.256 INFO:teuthology.orchestra.run.smithi043.stdout:55834574923 2024-04-21T08:37:19.256 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-21T08:37:19.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:19 smithi175 ceph-mon[31920]: pgmap v324: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:37:19.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:19 smithi112 ceph-mon[31085]: pgmap v324: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:37:21.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.991+0000 7eff02d64700 1 -- 172.21.15.43:0/672040528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efefc101da0 msgr2=0x7efefc10f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:21.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.991+0000 7eff02d64700 1 --2- 172.21.15.43:0/672040528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efefc101da0 0x7efefc10f810 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efef400b0a0 tx=0x7efef400b3b0 comp rx=0 tx=0).stop 2024-04-21T08:37:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.991+0000 7eff02d64700 1 -- 172.21.15.43:0/672040528 shutdown_connections 2024-04-21T08:37:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.991+0000 7eff02d64700 1 --2- 172.21.15.43:0/672040528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efefc107b00 0x7efefc107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.991+0000 7eff02d64700 1 --2- 172.21.15.43:0/672040528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efefc101da0 0x7efefc10f810 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.991+0000 7eff02d64700 1 --2- 172.21.15.43:0/672040528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efefc1013e0 0x7efefc101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.991+0000 7eff02d64700 1 -- 172.21.15.43:0/672040528 >> 172.21.15.43:0/672040528 conn(0x7efefc076c60 msgr2=0x7efefc077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.991+0000 7eff02d64700 1 -- 172.21.15.43:0/672040528 shutdown_connections 2024-04-21T08:37:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.991+0000 7eff02d64700 1 -- 172.21.15.43:0/672040528 wait complete. 2024-04-21T08:37:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.992+0000 7eff02d64700 1 Processor -- start 2024-04-21T08:37:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.992+0000 7eff02d64700 1 -- start start 2024-04-21T08:37:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.992+0000 7eff02d64700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efefc1013e0 0x7efefc19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.992+0000 7eff02d64700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efefc101da0 0x7efefc19b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.992+0000 7eff02d64700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efefc107b00 0x7efefc19fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.992+0000 7eff02d64700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efefc112c90 con 0x7efefc1013e0 2024-04-21T08:37:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.992+0000 7eff02d64700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efefc112b10 con 0x7efefc101da0 2024-04-21T08:37:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.992+0000 7eff02d64700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efefc112e10 con 0x7efefc107b00 2024-04-21T08:37:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.992+0000 7eff00b00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efefc1013e0 0x7efefc19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.992+0000 7eff00b00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efefc1013e0 0x7efefc19b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33656/0 (socket says 172.21.15.43:33656) 2024-04-21T08:37:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.992+0000 7eff00b00700 1 -- 172.21.15.43:0/2526191550 learned_addr learned my addr 172.21.15.43:0/2526191550 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:37:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.992+0000 7eff01301700 1 --2- 172.21.15.43:0/2526191550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efefc107b00 0x7efefc19fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.992+0000 7efefbfff700 1 --2- 172.21.15.43:0/2526191550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efefc101da0 0x7efefc19b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.993+0000 7eff00b00700 1 -- 172.21.15.43:0/2526191550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efefc107b00 msgr2=0x7efefc19fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.993+0000 7eff00b00700 1 --2- 172.21.15.43:0/2526191550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efefc107b00 0x7efefc19fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.993+0000 7eff00b00700 1 -- 172.21.15.43:0/2526191550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efefc101da0 msgr2=0x7efefc19b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.993+0000 7eff00b00700 1 --2- 172.21.15.43:0/2526191550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efefc101da0 0x7efefc19b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.993+0000 7eff00b00700 1 -- 172.21.15.43:0/2526191550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efefc1a0320 con 0x7efefc1013e0 2024-04-21T08:37:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.993+0000 7efefbfff700 1 --2- 172.21.15.43:0/2526191550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efefc101da0 0x7efefc19b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:37:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.993+0000 7eff00b00700 1 --2- 172.21.15.43:0/2526191550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efefc1013e0 0x7efefc19b360 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efef000d980 tx=0x7efef000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.993+0000 7efef9ffb700 1 -- 172.21.15.43:0/2526191550 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef0017030 con 0x7efefc1013e0 2024-04-21T08:37:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.993+0000 7efef9ffb700 1 -- 172.21.15.43:0/2526191550 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efef00044d0 con 0x7efefc1013e0 2024-04-21T08:37:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.993+0000 7eff02d64700 1 -- 172.21.15.43:0/2526191550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efefc1a0610 con 0x7efefc1013e0 2024-04-21T08:37:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.993+0000 7efef9ffb700 1 -- 172.21.15.43:0/2526191550 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef00039e0 con 0x7efefc1013e0 2024-04-21T08:37:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.993+0000 7eff02d64700 1 -- 172.21.15.43:0/2526191550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efefc1a7e90 con 0x7efefc1013e0 2024-04-21T08:37:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.994+0000 7eff02d64700 1 -- 172.21.15.43:0/2526191550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efefc103200 con 0x7efefc1013e0 2024-04-21T08:37:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.996+0000 7efef9ffb700 1 -- 172.21.15.43:0/2526191550 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efef001a070 con 0x7efefc1013e0 2024-04-21T08:37:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.996+0000 7efef9ffb700 1 --2- 172.21.15.43:0/2526191550 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7efeec0a3d50 0x7efeec0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.996+0000 7efefbfff700 1 --2- 172.21.15.43:0/2526191550 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7efeec0a3d50 0x7efeec0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.996+0000 7efef9ffb700 1 -- 172.21.15.43:0/2526191550 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7efef00d08a0 con 0x7efefc1013e0 2024-04-21T08:37:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.996+0000 7efef9ffb700 1 --2- 172.21.15.43:0/2526191550 >> [v2:172.21.15.175:6824/2527264089,v1:172.21.15.175:6825/2527264089] conn(0x7efeec0aeeb0 0x7efeec0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.996+0000 7efef9ffb700 1 -- 172.21.15.43:0/2526191550 --> [v2:172.21.15.175:6824/2527264089,v1:172.21.15.175:6825/2527264089] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efef0020e50 con 0x7efeec0aeeb0 2024-04-21T08:37:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.996+0000 7efef9ffb700 1 -- 172.21.15.43:0/2526191550 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7efef00d0cf0 con 0x7efefc1013e0 2024-04-21T08:37:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.997+0000 7efefbfff700 1 --2- 172.21.15.43:0/2526191550 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7efeec0a3d50 0x7efeec0a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efee8005950 tx=0x7efee80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.997+0000 7eff01301700 1 --2- 172.21.15.43:0/2526191550 >> [v2:172.21.15.175:6824/2527264089,v1:172.21.15.175:6825/2527264089] conn(0x7efeec0aeeb0 0x7efeec0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.997+0000 7eff01301700 1 --2- 172.21.15.43:0/2526191550 >> [v2:172.21.15.175:6824/2527264089,v1:172.21.15.175:6825/2527264089] conn(0x7efeec0aeeb0 0x7efeec0b1350 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:18.998+0000 7efef9ffb700 1 -- 172.21.15.43:0/2526191550 <== osd.11 v2:172.21.15.175:6824/2527264089 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efef0020e50 con 0x7efeec0aeeb0 2024-04-21T08:37:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:19.015+0000 7eff02d64700 1 -- 172.21.15.43:0/2526191550 --> [v2:172.21.15.175:6824/2527264089,v1:172.21.15.175:6825/2527264089] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efefc06ac90 con 0x7efeec0aeeb0 2024-04-21T08:37:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:19.015+0000 7efef9ffb700 1 -- 172.21.15.43:0/2526191550 <== osd.11 v2:172.21.15.175:6824/2527264089 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efefc06ac90 con 0x7efeec0aeeb0 2024-04-21T08:37:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:19.016+0000 7eff02d64700 1 -- 172.21.15.43:0/2526191550 >> [v2:172.21.15.175:6824/2527264089,v1:172.21.15.175:6825/2527264089] conn(0x7efeec0aeeb0 msgr2=0x7efeec0b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:19.016+0000 7eff02d64700 1 --2- 172.21.15.43:0/2526191550 >> [v2:172.21.15.175:6824/2527264089,v1:172.21.15.175:6825/2527264089] conn(0x7efeec0aeeb0 0x7efeec0b1350 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:19.016+0000 7eff02d64700 1 -- 172.21.15.43:0/2526191550 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7efeec0a3d50 msgr2=0x7efeec0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:19.016+0000 7eff02d64700 1 --2- 172.21.15.43:0/2526191550 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7efeec0a3d50 0x7efeec0a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efee8005950 tx=0x7efee80058e0 comp rx=0 tx=0).stop 2024-04-21T08:37:21.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:19.016+0000 7eff02d64700 1 -- 172.21.15.43:0/2526191550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efefc1013e0 msgr2=0x7efefc19b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:21.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:19.016+0000 7eff02d64700 1 --2- 172.21.15.43:0/2526191550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efefc1013e0 0x7efefc19b360 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efef000d980 tx=0x7efef000dc90 comp rx=0 tx=0).stop 2024-04-21T08:37:21.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:19.016+0000 7eff01301700 1 -- 172.21.15.43:0/2526191550 reap_dead start 2024-04-21T08:37:21.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:19.016+0000 7eff02d64700 1 -- 172.21.15.43:0/2526191550 shutdown_connections 2024-04-21T08:37:21.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:19.016+0000 7eff02d64700 1 -- 172.21.15.43:0/2526191550 >> 172.21.15.43:0/2526191550 conn(0x7efefc076c60 msgr2=0x7efefc102820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:21.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:19.016+0000 7eff02d64700 1 -- 172.21.15.43:0/2526191550 shutdown_connections 2024-04-21T08:37:21.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:19.016+0000 7eff02d64700 1 -- 172.21.15.43:0/2526191550 wait complete. 2024-04-21T08:37:21.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:21 smithi043 ceph-mon[27978]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:37:21.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:21 smithi175 ceph-mon[31920]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:37:21.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:21 smithi112 ceph-mon[31085]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:37:21.772 INFO:teuthology.orchestra.run.smithi043.stdout:347892351001 2024-04-21T08:37:21.772 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-21T08:37:23.051 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:23.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:23 smithi043 ceph-mon[27978]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:37:23.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:23 smithi175 ceph-mon[31920]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:37:23.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:23 smithi112 ceph-mon[31085]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:37:24.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:24 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:37:24.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:24 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:37:24.464 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:24 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:37:24.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.976+0000 7f51ac206700 1 -- 172.21.15.43:0/964275778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51a4107960 msgr2=0x7f51a4107d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:24.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.976+0000 7f51ac206700 1 --2- 172.21.15.43:0/964275778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51a4107960 0x7f51a4107d40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009990 tx=0x7f51a0009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:24.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.976+0000 7f51ac206700 1 -- 172.21.15.43:0/964275778 shutdown_connections 2024-04-21T08:37:24.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.976+0000 7f51ac206700 1 --2- 172.21.15.43:0/964275778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51a4102320 0x7f51a410e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:24.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.976+0000 7f51ac206700 1 --2- 172.21.15.43:0/964275778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51a4107960 0x7f51a4107d40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:24.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.976+0000 7f51ac206700 1 --2- 172.21.15.43:0/964275778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51a4101960 0x7f51a4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:24.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.976+0000 7f51ac206700 1 -- 172.21.15.43:0/964275778 >> 172.21.15.43:0/964275778 conn(0x7f51a40789e0 msgr2=0x7f51a4078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:24.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.976+0000 7f51ac206700 1 -- 172.21.15.43:0/964275778 shutdown_connections 2024-04-21T08:37:24.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.976+0000 7f51ac206700 1 -- 172.21.15.43:0/964275778 wait complete. 2024-04-21T08:37:24.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.977+0000 7f51ac206700 1 Processor -- start 2024-04-21T08:37:24.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.977+0000 7f51ac206700 1 -- start start 2024-04-21T08:37:24.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.977+0000 7f51ac206700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51a4101960 0x7f51a4199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:24.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.977+0000 7f51ac206700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51a4102320 0x7f51a4199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:24.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.977+0000 7f51ac206700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51a4107960 0x7f51a419db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:24.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.977+0000 7f51ac206700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4111b70 con 0x7f51a4102320 2024-04-21T08:37:24.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.977+0000 7f51ac206700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f51a41119f0 con 0x7f51a4107960 2024-04-21T08:37:24.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.977+0000 7f51ac206700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4111cf0 con 0x7f51a4101960 2024-04-21T08:37:24.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.977+0000 7f51a97a1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51a4102320 0x7f51a4199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:24.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.977+0000 7f51aa7a3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51a4107960 0x7f51a419db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:24.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.977+0000 7f51a9fa2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51a4101960 0x7f51a4199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:24.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.977+0000 7f51aa7a3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51a4107960 0x7f51a419db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:56582/0 (socket says 172.21.15.43:56582) 2024-04-21T08:37:24.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.977+0000 7f51a97a1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51a4102320 0x7f51a4199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33682/0 (socket says 172.21.15.43:33682) 2024-04-21T08:37:24.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.977+0000 7f51a9fa2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51a4101960 0x7f51a4199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.43:50596/0 (socket says 172.21.15.43:50596) 2024-04-21T08:37:24.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.977+0000 7f51aa7a3700 1 -- 172.21.15.43:0/1259448702 learned_addr learned my addr 172.21.15.43:0/1259448702 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:37:24.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.977+0000 7f51aa7a3700 1 -- 172.21.15.43:0/1259448702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51a4101960 msgr2=0x7f51a4199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:24.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.977+0000 7f51aa7a3700 1 --2- 172.21.15.43:0/1259448702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51a4101960 0x7f51a4199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:24.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.977+0000 7f51aa7a3700 1 -- 172.21.15.43:0/1259448702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51a4102320 msgr2=0x7f51a4199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:24.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.977+0000 7f51aa7a3700 1 --2- 172.21.15.43:0/1259448702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51a4102320 0x7f51a4199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:24.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.977+0000 7f51aa7a3700 1 -- 172.21.15.43:0/1259448702 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a419e240 con 0x7f51a4107960 2024-04-21T08:37:24.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.977+0000 7f51a97a1700 1 --2- 172.21.15.43:0/1259448702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51a4102320 0x7f51a4199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:37:24.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.978+0000 7f51aa7a3700 1 --2- 172.21.15.43:0/1259448702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51a4107960 0x7f51a419db00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f519c00c370 tx=0x7f519c00c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:24.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.978+0000 7f519affd700 1 -- 172.21.15.43:0/1259448702 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c0092e0 con 0x7f51a4107960 2024-04-21T08:37:24.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.978+0000 7f51ac206700 1 -- 172.21.15.43:0/1259448702 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a419e4d0 con 0x7f51a4107960 2024-04-21T08:37:24.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.978+0000 7f519affd700 1 -- 172.21.15.43:0/1259448702 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f519c003a00 con 0x7f51a4107960 2024-04-21T08:37:24.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.978+0000 7f519affd700 1 -- 172.21.15.43:0/1259448702 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c0046f0 con 0x7f51a4107960 2024-04-21T08:37:24.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.978+0000 7f51ac206700 1 -- 172.21.15.43:0/1259448702 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a40fe8c0 con 0x7f51a4107960 2024-04-21T08:37:24.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.979+0000 7f51ac206700 1 -- 172.21.15.43:0/1259448702 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f51a4063110 con 0x7f51a4107960 2024-04-21T08:37:24.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.980+0000 7f519affd700 1 -- 172.21.15.43:0/1259448702 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f519c01a020 con 0x7f51a4107960 2024-04-21T08:37:24.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.981+0000 7f519affd700 1 --2- 172.21.15.43:0/1259448702 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f51900a3d50 0x7f51900a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:24.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.981+0000 7f51a9fa2700 1 --2- 172.21.15.43:0/1259448702 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f51900a3d50 0x7f51900a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:24.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.981+0000 7f519affd700 1 -- 172.21.15.43:0/1259448702 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f519c0d0b80 con 0x7f51a4107960 2024-04-21T08:37:24.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.981+0000 7f519affd700 1 --2- 172.21.15.43:0/1259448702 >> [v2:172.21.15.112:6816/1090541705,v1:172.21.15.112:6817/1090541705] conn(0x7f51900aeeb0 0x7f51900b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:24.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.981+0000 7f519affd700 1 -- 172.21.15.43:0/1259448702 --> [v2:172.21.15.112:6816/1090541705,v1:172.21.15.112:6817/1090541705] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f519c024e50 con 0x7f51900aeeb0 2024-04-21T08:37:24.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.981+0000 7f519affd700 1 -- 172.21.15.43:0/1259448702 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f519c0d0f30 con 0x7f51a4107960 2024-04-21T08:37:24.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.981+0000 7f51a97a1700 1 --2- 172.21.15.43:0/1259448702 >> [v2:172.21.15.112:6816/1090541705,v1:172.21.15.112:6817/1090541705] conn(0x7f51900aeeb0 0x7f51900b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:24.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.981+0000 7f51a9fa2700 1 --2- 172.21.15.43:0/1259448702 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f51900a3d50 0x7f51900a6210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f51a000b2e0 tx=0x7f51a0005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:24.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.981+0000 7f51a97a1700 1 --2- 172.21.15.43:0/1259448702 >> [v2:172.21.15.112:6816/1090541705,v1:172.21.15.112:6817/1090541705] conn(0x7f51900aeeb0 0x7f51900b1350 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:24.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.982+0000 7f519affd700 1 -- 172.21.15.43:0/1259448702 <== osd.7 v2:172.21.15.112:6816/1090541705 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f519c024e50 con 0x7f51900aeeb0 2024-04-21T08:37:24.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.999+0000 7f51ac206700 1 -- 172.21.15.43:0/1259448702 --> [v2:172.21.15.112:6816/1090541705,v1:172.21.15.112:6817/1090541705] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f51a4063510 con 0x7f51900aeeb0 2024-04-21T08:37:24.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:24.999+0000 7f519affd700 1 -- 172.21.15.43:0/1259448702 <== osd.7 v2:172.21.15.112:6816/1090541705 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f51a4063510 con 0x7f51900aeeb0 2024-04-21T08:37:24.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.000+0000 7f51ac206700 1 -- 172.21.15.43:0/1259448702 >> [v2:172.21.15.112:6816/1090541705,v1:172.21.15.112:6817/1090541705] conn(0x7f51900aeeb0 msgr2=0x7f51900b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:24.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.000+0000 7f51ac206700 1 --2- 172.21.15.43:0/1259448702 >> [v2:172.21.15.112:6816/1090541705,v1:172.21.15.112:6817/1090541705] conn(0x7f51900aeeb0 0x7f51900b1350 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:24.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.000+0000 7f51ac206700 1 -- 172.21.15.43:0/1259448702 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f51900a3d50 msgr2=0x7f51900a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:25.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.000+0000 7f51ac206700 1 --2- 172.21.15.43:0/1259448702 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f51900a3d50 0x7f51900a6210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f51a000b2e0 tx=0x7f51a0005960 comp rx=0 tx=0).stop 2024-04-21T08:37:25.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.000+0000 7f51ac206700 1 -- 172.21.15.43:0/1259448702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51a4107960 msgr2=0x7f51a419db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:25.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.000+0000 7f51ac206700 1 --2- 172.21.15.43:0/1259448702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51a4107960 0x7f51a419db00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f519c00c370 tx=0x7f519c00c680 comp rx=0 tx=0).stop 2024-04-21T08:37:25.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.000+0000 7f51aa7a3700 1 -- 172.21.15.43:0/1259448702 reap_dead start 2024-04-21T08:37:25.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.000+0000 7f51ac206700 1 -- 172.21.15.43:0/1259448702 shutdown_connections 2024-04-21T08:37:25.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.000+0000 7f51ac206700 1 -- 172.21.15.43:0/1259448702 >> 172.21.15.43:0/1259448702 conn(0x7f51a40789e0 msgr2=0x7f51a406b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:25.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.000+0000 7f51ac206700 1 -- 172.21.15.43:0/1259448702 shutdown_connections 2024-04-21T08:37:25.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.000+0000 7f51ac206700 1 -- 172.21.15.43:0/1259448702 wait complete. 2024-04-21T08:37:25.260 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:25 smithi043 ceph-mon[27978]: pgmap v327: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 310 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:37:25.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:25 smithi175 ceph-mon[31920]: pgmap v327: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 310 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:37:25.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:25 smithi112 ceph-mon[31085]: pgmap v327: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 310 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:37:25.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.673+0000 7ffa8bd55700 1 -- 172.21.15.43:0/971882180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa8406afe0 msgr2=0x7ffa8406b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:25.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.673+0000 7ffa8bd55700 1 --2- 172.21.15.43:0/971882180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa8406afe0 0x7ffa8406b440 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009990 tx=0x7ffa74009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:25.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.673+0000 7ffa8bd55700 1 -- 172.21.15.43:0/971882180 shutdown_connections 2024-04-21T08:37:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.673+0000 7ffa8bd55700 1 --2- 172.21.15.43:0/971882180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa84108a20 0x7ffa84108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.673+0000 7ffa8bd55700 1 --2- 172.21.15.43:0/971882180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa8406b980 0x7ffa8410f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.673+0000 7ffa8bd55700 1 --2- 172.21.15.43:0/971882180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa8406afe0 0x7ffa8406b440 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.673+0000 7ffa8bd55700 1 -- 172.21.15.43:0/971882180 >> 172.21.15.43:0/971882180 conn(0x7ffa840788b0 msgr2=0x7ffa84078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.673+0000 7ffa8bd55700 1 -- 172.21.15.43:0/971882180 shutdown_connections 2024-04-21T08:37:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.673+0000 7ffa8bd55700 1 -- 172.21.15.43:0/971882180 wait complete. 2024-04-21T08:37:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.674+0000 7ffa8bd55700 1 Processor -- start 2024-04-21T08:37:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.674+0000 7ffa8bd55700 1 -- start start 2024-04-21T08:37:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.674+0000 7ffa8bd55700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa8406afe0 0x7ffa84199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.674+0000 7ffa8bd55700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa8406b980 0x7ffa84199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.675+0000 7ffa8bd55700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa84108a20 0x7ffa8419d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.675+0000 7ffa8bd55700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffa84112be0 con 0x7ffa8406afe0 2024-04-21T08:37:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.675+0000 7ffa8bd55700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffa84112a60 con 0x7ffa8406b980 2024-04-21T08:37:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.675+0000 7ffa8bd55700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa84112d60 con 0x7ffa84108a20 2024-04-21T08:37:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.675+0000 7ffa89af1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa8406afe0 0x7ffa84199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.675+0000 7ffa89af1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa8406afe0 0x7ffa84199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33696/0 (socket says 172.21.15.43:33696) 2024-04-21T08:37:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.675+0000 7ffa8a2f2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa84108a20 0x7ffa8419d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.675+0000 7ffa89af1700 1 -- 172.21.15.43:0/426185431 learned_addr learned my addr 172.21.15.43:0/426185431 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:37:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.675+0000 7ffa892f0700 1 --2- 172.21.15.43:0/426185431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa8406b980 0x7ffa84199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.675+0000 7ffa89af1700 1 -- 172.21.15.43:0/426185431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa84108a20 msgr2=0x7ffa8419d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.675+0000 7ffa89af1700 1 --2- 172.21.15.43:0/426185431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa84108a20 0x7ffa8419d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.675+0000 7ffa89af1700 1 -- 172.21.15.43:0/426185431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa8406b980 msgr2=0x7ffa84199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.675+0000 7ffa89af1700 1 --2- 172.21.15.43:0/426185431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa8406b980 0x7ffa84199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.675+0000 7ffa89af1700 1 -- 172.21.15.43:0/426185431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8419e120 con 0x7ffa8406afe0 2024-04-21T08:37:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.675+0000 7ffa89af1700 1 --2- 172.21.15.43:0/426185431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa8406afe0 0x7ffa84199110 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000ef10 tx=0x7ffa8000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.675+0000 7ffa7affd700 1 -- 172.21.15.43:0/426185431 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8000c9c0 con 0x7ffa8406afe0 2024-04-21T08:37:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.676+0000 7ffa8bd55700 1 -- 172.21.15.43:0/426185431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa8419e3b0 con 0x7ffa8406afe0 2024-04-21T08:37:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.676+0000 7ffa8bd55700 1 -- 172.21.15.43:0/426185431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa841a5c90 con 0x7ffa8406afe0 2024-04-21T08:37:25.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.676+0000 7ffa7affd700 1 -- 172.21.15.43:0/426185431 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa80019070 con 0x7ffa8406afe0 2024-04-21T08:37:25.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.676+0000 7ffa7affd700 1 -- 172.21.15.43:0/426185431 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80014480 con 0x7ffa8406afe0 2024-04-21T08:37:25.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.676+0000 7ffa8bd55700 1 -- 172.21.15.43:0/426185431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa840631d0 con 0x7ffa8406afe0 2024-04-21T08:37:25.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.678+0000 7ffa7affd700 1 -- 172.21.15.43:0/426185431 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ffa80012050 con 0x7ffa8406afe0 2024-04-21T08:37:25.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.678+0000 7ffa7affd700 1 --2- 172.21.15.43:0/426185431 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ffa700a3cd0 0x7ffa700a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:25.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.679+0000 7ffa7affd700 1 -- 172.21.15.43:0/426185431 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ffa800d0ad0 con 0x7ffa8406afe0 2024-04-21T08:37:25.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.679+0000 7ffa892f0700 1 --2- 172.21.15.43:0/426185431 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ffa700a3cd0 0x7ffa700a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:25.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.679+0000 7ffa7affd700 1 --2- 172.21.15.43:0/426185431 >> [v2:172.21.15.112:6808/3007534004,v1:172.21.15.112:6809/3007534004] conn(0x7ffa700aee30 0x7ffa700b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:25.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.679+0000 7ffa7affd700 1 -- 172.21.15.43:0/426185431 --> [v2:172.21.15.112:6808/3007534004,v1:172.21.15.112:6809/3007534004] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa700b19a0 con 0x7ffa700aee30 2024-04-21T08:37:25.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.679+0000 7ffa7affd700 1 -- 172.21.15.43:0/426185431 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7ffa800d0f20 con 0x7ffa8406afe0 2024-04-21T08:37:25.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.679+0000 7ffa8a2f2700 1 --2- 172.21.15.43:0/426185431 >> [v2:172.21.15.112:6808/3007534004,v1:172.21.15.112:6809/3007534004] conn(0x7ffa700aee30 0x7ffa700b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:25.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.679+0000 7ffa892f0700 1 --2- 172.21.15.43:0/426185431 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ffa700a3cd0 0x7ffa700a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400b2e0 tx=0x7ffa74005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:25.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.679+0000 7ffa8a2f2700 1 --2- 172.21.15.43:0/426185431 >> [v2:172.21.15.112:6808/3007534004,v1:172.21.15.112:6809/3007534004] conn(0x7ffa700aee30 0x7ffa700b12d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.680+0000 7ffa7affd700 1 -- 172.21.15.43:0/426185431 <== osd.4 v2:172.21.15.112:6808/3007534004 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa700b19a0 con 0x7ffa700aee30 2024-04-21T08:37:25.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.697+0000 7ffa8bd55700 1 -- 172.21.15.43:0/426185431 --> [v2:172.21.15.112:6808/3007534004,v1:172.21.15.112:6809/3007534004] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffa840635d0 con 0x7ffa700aee30 2024-04-21T08:37:25.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.697+0000 7ffa7affd700 1 -- 172.21.15.43:0/426185431 <== osd.4 v2:172.21.15.112:6808/3007534004 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffa840635d0 con 0x7ffa700aee30 2024-04-21T08:37:25.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.698+0000 7ffa8bd55700 1 -- 172.21.15.43:0/426185431 >> [v2:172.21.15.112:6808/3007534004,v1:172.21.15.112:6809/3007534004] conn(0x7ffa700aee30 msgr2=0x7ffa700b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:25.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.698+0000 7ffa8bd55700 1 --2- 172.21.15.43:0/426185431 >> [v2:172.21.15.112:6808/3007534004,v1:172.21.15.112:6809/3007534004] conn(0x7ffa700aee30 0x7ffa700b12d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:25.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.698+0000 7ffa8bd55700 1 -- 172.21.15.43:0/426185431 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ffa700a3cd0 msgr2=0x7ffa700a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:25.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.698+0000 7ffa8bd55700 1 --2- 172.21.15.43:0/426185431 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ffa700a3cd0 0x7ffa700a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400b2e0 tx=0x7ffa74005960 comp rx=0 tx=0).stop 2024-04-21T08:37:25.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.698+0000 7ffa8bd55700 1 -- 172.21.15.43:0/426185431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa8406afe0 msgr2=0x7ffa84199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:25.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.698+0000 7ffa8bd55700 1 --2- 172.21.15.43:0/426185431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa8406afe0 0x7ffa84199110 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000ef10 tx=0x7ffa8000c3a0 comp rx=0 tx=0).stop 2024-04-21T08:37:25.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.698+0000 7ffa8a2f2700 1 -- 172.21.15.43:0/426185431 reap_dead start 2024-04-21T08:37:25.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.698+0000 7ffa8bd55700 1 -- 172.21.15.43:0/426185431 shutdown_connections 2024-04-21T08:37:25.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.698+0000 7ffa8bd55700 1 -- 172.21.15.43:0/426185431 >> 172.21.15.43:0/426185431 conn(0x7ffa840788b0 msgr2=0x7ffa840ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:25.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.698+0000 7ffa8bd55700 1 -- 172.21.15.43:0/426185431 shutdown_connections 2024-04-21T08:37:25.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.698+0000 7ffa8bd55700 1 -- 172.21.15.43:0/426185431 wait complete. 2024-04-21T08:37:25.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.923+0000 7fb824207700 1 -- 172.21.15.43:0/488593879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb81c101030 msgr2=0x7fb81c1014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:25.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.923+0000 7fb824207700 1 --2- 172.21.15.43:0/488593879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb81c101030 0x7fb81c1014b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb80c009990 tx=0x7fb80c009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:25.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.923+0000 7fb824207700 1 -- 172.21.15.43:0/488593879 shutdown_connections 2024-04-21T08:37:25.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.923+0000 7fb824207700 1 --2- 172.21.15.43:0/488593879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb81c103950 0x7fb81c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:25.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.923+0000 7fb824207700 1 --2- 172.21.15.43:0/488593879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb81c101ad0 0x7fb81c1117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:25.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.923+0000 7fb824207700 1 --2- 172.21.15.43:0/488593879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb81c101030 0x7fb81c1014b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:25.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.923+0000 7fb824207700 1 -- 172.21.15.43:0/488593879 >> 172.21.15.43:0/488593879 conn(0x7fb81c077fb0 msgr2=0x7fb81c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:25.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.924+0000 7fb824207700 1 -- 172.21.15.43:0/488593879 shutdown_connections 2024-04-21T08:37:25.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.924+0000 7fb824207700 1 -- 172.21.15.43:0/488593879 wait complete. 2024-04-21T08:37:25.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.925+0000 7fb824207700 1 Processor -- start 2024-04-21T08:37:25.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.925+0000 7fb824207700 1 -- start start 2024-04-21T08:37:25.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.925+0000 7fb824207700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb81c101030 0x7fb81c19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:25.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.925+0000 7fb824207700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb81c101ad0 0x7fb81c19b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:25.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.925+0000 7fb824207700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb81c103950 0x7fb81c19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:25.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.925+0000 7fb824207700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c114e40 con 0x7fb81c103950 2024-04-21T08:37:25.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.925+0000 7fb824207700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c114cc0 con 0x7fb81c101ad0 2024-04-21T08:37:25.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.925+0000 7fb824207700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c114fc0 con 0x7fb81c101030 2024-04-21T08:37:25.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.925+0000 7fb8227a4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb81c103950 0x7fb81c19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:25.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.925+0000 7fb8227a4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb81c103950 0x7fb81c19fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33718/0 (socket says 172.21.15.43:33718) 2024-04-21T08:37:25.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.925+0000 7fb8227a4700 1 -- 172.21.15.43:0/1259047058 learned_addr learned my addr 172.21.15.43:0/1259047058 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:37:25.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.926+0000 7fb8217a2700 1 --2- 172.21.15.43:0/1259047058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb81c101ad0 0x7fb81c19b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:25.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.926+0000 7fb821fa3700 1 --2- 172.21.15.43:0/1259047058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb81c101030 0x7fb81c19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:25.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.926+0000 7fb8227a4700 1 -- 172.21.15.43:0/1259047058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb81c101030 msgr2=0x7fb81c19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:25.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.926+0000 7fb8227a4700 1 --2- 172.21.15.43:0/1259047058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb81c101030 0x7fb81c19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:25.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.926+0000 7fb8227a4700 1 -- 172.21.15.43:0/1259047058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb81c101ad0 msgr2=0x7fb81c19b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:25.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.926+0000 7fb8227a4700 1 --2- 172.21.15.43:0/1259047058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb81c101ad0 0x7fb81c19b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:25.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.926+0000 7fb8227a4700 1 -- 172.21.15.43:0/1259047058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81c1a0340 con 0x7fb81c103950 2024-04-21T08:37:25.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.926+0000 7fb8227a4700 1 --2- 172.21.15.43:0/1259047058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb81c103950 0x7fb81c19fc00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb81400ef30 tx=0x7fb81400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:25.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.926+0000 7fb812ffd700 1 -- 172.21.15.43:0/1259047058 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8140092e0 con 0x7fb81c103950 2024-04-21T08:37:25.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.926+0000 7fb824207700 1 -- 172.21.15.43:0/1259047058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb81c1a05d0 con 0x7fb81c103950 2024-04-21T08:37:25.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.926+0000 7fb824207700 1 -- 172.21.15.43:0/1259047058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb81c1a7eb0 con 0x7fb81c103950 2024-04-21T08:37:25.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.926+0000 7fb812ffd700 1 -- 172.21.15.43:0/1259047058 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb814004bb0 con 0x7fb81c103950 2024-04-21T08:37:25.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.926+0000 7fb812ffd700 1 -- 172.21.15.43:0/1259047058 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814004280 con 0x7fb81c103950 2024-04-21T08:37:25.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.928+0000 7fb824207700 1 -- 172.21.15.43:0/1259047058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb81c063150 con 0x7fb81c103950 2024-04-21T08:37:25.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.928+0000 7fb812ffd700 1 -- 172.21.15.43:0/1259047058 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb81401b030 con 0x7fb81c103950 2024-04-21T08:37:25.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.929+0000 7fb812ffd700 1 --2- 172.21.15.43:0/1259047058 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8080a3d50 0x7fb8080a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:25.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.929+0000 7fb812ffd700 1 -- 172.21.15.43:0/1259047058 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb8140d0300 con 0x7fb81c103950 2024-04-21T08:37:25.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.929+0000 7fb821fa3700 1 --2- 172.21.15.43:0/1259047058 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8080a3d50 0x7fb8080a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.929+0000 7fb812ffd700 1 --2- 172.21.15.43:0/1259047058 >> [v2:172.21.15.175:6800/2080591921,v1:172.21.15.175:6801/2080591921] conn(0x7fb8080aeeb0 0x7fb8080b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.929+0000 7fb812ffd700 1 -- 172.21.15.43:0/1259047058 --> [v2:172.21.15.175:6800/2080591921,v1:172.21.15.175:6801/2080591921] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb8080b1a20 con 0x7fb8080aeeb0 2024-04-21T08:37:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.929+0000 7fb812ffd700 1 -- 172.21.15.43:0/1259047058 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fb8140d7050 con 0x7fb81c103950 2024-04-21T08:37:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.930+0000 7fb821fa3700 1 --2- 172.21.15.43:0/1259047058 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8080a3d50 0x7fb8080a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb818009bd0 tx=0x7fb818009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.930+0000 7fb8217a2700 1 --2- 172.21.15.43:0/1259047058 >> [v2:172.21.15.175:6800/2080591921,v1:172.21.15.175:6801/2080591921] conn(0x7fb8080aeeb0 0x7fb8080b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.930+0000 7fb8217a2700 1 --2- 172.21.15.43:0/1259047058 >> [v2:172.21.15.175:6800/2080591921,v1:172.21.15.175:6801/2080591921] conn(0x7fb8080aeeb0 0x7fb8080b1350 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:25.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.931+0000 7fb812ffd700 1 -- 172.21.15.43:0/1259047058 <== osd.2 v2:172.21.15.175:6800/2080591921 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb8080b1a20 con 0x7fb8080aeeb0 2024-04-21T08:37:25.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.948+0000 7fb824207700 1 -- 172.21.15.43:0/1259047058 --> [v2:172.21.15.175:6800/2080591921,v1:172.21.15.175:6801/2080591921] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb81c0635d0 con 0x7fb8080aeeb0 2024-04-21T08:37:25.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.948+0000 7fb812ffd700 1 -- 172.21.15.43:0/1259047058 <== osd.2 v2:172.21.15.175:6800/2080591921 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb81c0635d0 con 0x7fb8080aeeb0 2024-04-21T08:37:25.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.948+0000 7fb824207700 1 -- 172.21.15.43:0/1259047058 >> [v2:172.21.15.175:6800/2080591921,v1:172.21.15.175:6801/2080591921] conn(0x7fb8080aeeb0 msgr2=0x7fb8080b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:25.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.948+0000 7fb824207700 1 --2- 172.21.15.43:0/1259047058 >> [v2:172.21.15.175:6800/2080591921,v1:172.21.15.175:6801/2080591921] conn(0x7fb8080aeeb0 0x7fb8080b1350 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:25.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.949+0000 7fb824207700 1 -- 172.21.15.43:0/1259047058 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8080a3d50 msgr2=0x7fb8080a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:25.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.949+0000 7fb824207700 1 --2- 172.21.15.43:0/1259047058 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8080a3d50 0x7fb8080a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb818009bd0 tx=0x7fb818009260 comp rx=0 tx=0).stop 2024-04-21T08:37:25.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.949+0000 7fb824207700 1 -- 172.21.15.43:0/1259047058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb81c103950 msgr2=0x7fb81c19fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:25.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.949+0000 7fb824207700 1 --2- 172.21.15.43:0/1259047058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb81c103950 0x7fb81c19fc00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb81400ef30 tx=0x7fb81400c3a0 comp rx=0 tx=0).stop 2024-04-21T08:37:25.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.949+0000 7fb8227a4700 1 -- 172.21.15.43:0/1259047058 reap_dead start 2024-04-21T08:37:25.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.949+0000 7fb824207700 1 -- 172.21.15.43:0/1259047058 shutdown_connections 2024-04-21T08:37:25.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.949+0000 7fb824207700 1 -- 172.21.15.43:0/1259047058 >> 172.21.15.43:0/1259047058 conn(0x7fb81c077fb0 msgr2=0x7fb81c0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:25.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.949+0000 7fb824207700 1 -- 172.21.15.43:0/1259047058 shutdown_connections 2024-04-21T08:37:25.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:25.949+0000 7fb824207700 1 -- 172.21.15.43:0/1259047058 wait complete. 2024-04-21T08:37:26.448 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:26 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:37:26.448 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:26 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:37:26.448 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:26 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:37:26.448 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:26 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:37:26.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:26 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:37:26.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:26 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:37:26.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:26 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:37:26.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:26 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:37:26.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:26 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:37:26.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:26 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:37:26.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:26 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:37:26.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:26 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:37:27.133 INFO:teuthology.orchestra.run.smithi043.stdout:141733920830 2024-04-21T08:37:27.134 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-21T08:37:27.438 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:27 smithi043 ceph-mon[27978]: pgmap v328: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 282 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:37:27.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:27 smithi112 ceph-mon[31085]: pgmap v328: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 282 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:37:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.545+0000 7f3273418700 1 -- 172.21.15.43:0/315235528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f326c105870 msgr2=0x7f326c1098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.545+0000 7f3273418700 1 --2- 172.21.15.43:0/315235528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f326c105870 0x7f326c1098c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f32680099c0 tx=0x7f3268009cd0 comp rx=0 tx=0).stop 2024-04-21T08:37:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.545+0000 7f3273418700 1 -- 172.21.15.43:0/315235528 shutdown_connections 2024-04-21T08:37:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.545+0000 7f3273418700 1 --2- 172.21.15.43:0/315235528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f326c104ec0 0x7f326c1052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.545+0000 7f3273418700 1 --2- 172.21.15.43:0/315235528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f326c109fa0 0x7f326c111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.545+0000 7f3273418700 1 --2- 172.21.15.43:0/315235528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f326c105870 0x7f326c1098c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.545+0000 7f3273418700 1 -- 172.21.15.43:0/315235528 >> 172.21.15.43:0/315235528 conn(0x7f326c100750 msgr2=0x7f326c102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.546+0000 7f3273418700 1 -- 172.21.15.43:0/315235528 shutdown_connections 2024-04-21T08:37:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.546+0000 7f3273418700 1 -- 172.21.15.43:0/315235528 wait complete. 2024-04-21T08:37:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.546+0000 7f3273418700 1 Processor -- start 2024-04-21T08:37:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.546+0000 7f3273418700 1 -- start start 2024-04-21T08:37:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.546+0000 7f3273418700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f326c104ec0 0x7f326c19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.546+0000 7f3273418700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f326c105870 0x7f326c19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.546+0000 7f3273418700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f326c109fa0 0x7f326c19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.546+0000 7f3273418700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f326c114f10 con 0x7f326c105870 2024-04-21T08:37:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.546+0000 7f3273418700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f326c114d90 con 0x7f326c109fa0 2024-04-21T08:37:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.546+0000 7f3273418700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f326c115090 con 0x7f326c104ec0 2024-04-21T08:37:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.547+0000 7f32709b3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f326c105870 0x7f326c19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.547+0000 7f32709b3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f326c105870 0x7f326c19b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45284/0 (socket says 172.21.15.43:45284) 2024-04-21T08:37:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.547+0000 7f32709b3700 1 -- 172.21.15.43:0/2460389627 learned_addr learned my addr 172.21.15.43:0/2460389627 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:37:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.547+0000 7f32711b4700 1 --2- 172.21.15.43:0/2460389627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f326c104ec0 0x7f326c19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.547+0000 7f32709b3700 1 -- 172.21.15.43:0/2460389627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f326c104ec0 msgr2=0x7f326c19b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.547+0000 7f32709b3700 1 --2- 172.21.15.43:0/2460389627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f326c104ec0 0x7f326c19b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.547+0000 7f32709b3700 1 -- 172.21.15.43:0/2460389627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f326c109fa0 msgr2=0x7f326c19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.547+0000 7f32719b5700 1 --2- 172.21.15.43:0/2460389627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f326c109fa0 0x7f326c19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.547+0000 7f32709b3700 1 --2- 172.21.15.43:0/2460389627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f326c109fa0 0x7f326c19fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:27.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.547+0000 7f32709b3700 1 -- 172.21.15.43:0/2460389627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326c1a0420 con 0x7f326c105870 2024-04-21T08:37:27.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.547+0000 7f32709b3700 1 --2- 172.21.15.43:0/2460389627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f326c105870 0x7f326c19b9a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f32680098e0 tx=0x7f3268003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:27.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.547+0000 7f32627fc700 1 -- 172.21.15.43:0/2460389627 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3268027070 con 0x7f326c105870 2024-04-21T08:37:27.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.547+0000 7f3273418700 1 -- 172.21.15.43:0/2460389627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f326c1a06b0 con 0x7f326c105870 2024-04-21T08:37:27.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.547+0000 7f32627fc700 1 -- 172.21.15.43:0/2460389627 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32680040a0 con 0x7f326c105870 2024-04-21T08:37:27.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.547+0000 7f32627fc700 1 -- 172.21.15.43:0/2460389627 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326800f460 con 0x7f326c105870 2024-04-21T08:37:27.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.547+0000 7f3273418700 1 -- 172.21.15.43:0/2460389627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f326c1a7f90 con 0x7f326c105870 2024-04-21T08:37:27.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.548+0000 7f3273418700 1 -- 172.21.15.43:0/2460389627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f326c0631d0 con 0x7f326c105870 2024-04-21T08:37:27.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.549+0000 7f32627fc700 1 -- 172.21.15.43:0/2460389627 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f326802a070 con 0x7f326c105870 2024-04-21T08:37:27.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.550+0000 7f32627fc700 1 --2- 172.21.15.43:0/2460389627 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f32580a3cd0 0x7f32580a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:27.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.550+0000 7f32627fc700 1 -- 172.21.15.43:0/2460389627 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f32680d2450 con 0x7f326c105870 2024-04-21T08:37:27.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.550+0000 7f32711b4700 1 --2- 172.21.15.43:0/2460389627 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f32580a3cd0 0x7f32580a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:27.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.550+0000 7f32627fc700 1 --2- 172.21.15.43:0/2460389627 >> [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011] conn(0x7f32580aee30 0x7f32580b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:27.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.550+0000 7f32627fc700 1 -- 172.21.15.43:0/2460389627 --> [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f32580b19a0 con 0x7f32580aee30 2024-04-21T08:37:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.550+0000 7f32627fc700 1 -- 172.21.15.43:0/2460389627 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f32680d28a0 con 0x7f326c105870 2024-04-21T08:37:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.550+0000 7f32719b5700 1 --2- 172.21.15.43:0/2460389627 >> [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011] conn(0x7f32580aee30 0x7f32580b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.550+0000 7f32711b4700 1 --2- 172.21.15.43:0/2460389627 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f32580a3cd0 0x7f32580a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f325c006fd0 tx=0x7f325c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.551+0000 7f32719b5700 1 --2- 172.21.15.43:0/2460389627 >> [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011] conn(0x7f32580aee30 0x7f32580b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.551+0000 7f32627fc700 1 -- 172.21.15.43:0/2460389627 <== osd.9 v2:172.21.15.43:6826/2518248011 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f32580b19a0 con 0x7f32580aee30 2024-04-21T08:37:27.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.568+0000 7f3273418700 1 -- 172.21.15.43:0/2460389627 --> [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f326c0635d0 con 0x7f32580aee30 2024-04-21T08:37:27.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.569+0000 7f32627fc700 1 -- 172.21.15.43:0/2460389627 <== osd.9 v2:172.21.15.43:6826/2518248011 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f326c0635d0 con 0x7f32580aee30 2024-04-21T08:37:27.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.569+0000 7f3273418700 1 -- 172.21.15.43:0/2460389627 >> [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011] conn(0x7f32580aee30 msgr2=0x7f32580b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:27.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.569+0000 7f3273418700 1 --2- 172.21.15.43:0/2460389627 >> [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011] conn(0x7f32580aee30 0x7f32580b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:27.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.569+0000 7f3273418700 1 -- 172.21.15.43:0/2460389627 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f32580a3cd0 msgr2=0x7f32580a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:27.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.569+0000 7f3273418700 1 --2- 172.21.15.43:0/2460389627 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f32580a3cd0 0x7f32580a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f325c006fd0 tx=0x7f325c008040 comp rx=0 tx=0).stop 2024-04-21T08:37:27.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.569+0000 7f3273418700 1 -- 172.21.15.43:0/2460389627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f326c105870 msgr2=0x7f326c19b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:27.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.569+0000 7f3273418700 1 --2- 172.21.15.43:0/2460389627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f326c105870 0x7f326c19b9a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f32680098e0 tx=0x7f3268003680 comp rx=0 tx=0).stop 2024-04-21T08:37:27.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.570+0000 7f32719b5700 1 -- 172.21.15.43:0/2460389627 reap_dead start 2024-04-21T08:37:27.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.570+0000 7f3273418700 1 -- 172.21.15.43:0/2460389627 shutdown_connections 2024-04-21T08:37:27.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.570+0000 7f3273418700 1 -- 172.21.15.43:0/2460389627 >> 172.21.15.43:0/2460389627 conn(0x7f326c100750 msgr2=0x7f326c102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:27.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.570+0000 7f3273418700 1 -- 172.21.15.43:0/2460389627 shutdown_connections 2024-04-21T08:37:27.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:27.570+0000 7f3273418700 1 -- 172.21.15.43:0/2460389627 wait complete. 2024-04-21T08:37:27.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:27 smithi175 ceph-mon[31920]: pgmap v328: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 282 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:37:28.198 INFO:teuthology.orchestra.run.smithi043.stdout:223338299438 2024-04-21T08:37:28.198 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-21T08:37:29.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.175+0000 7f2137d32700 1 -- 172.21.15.43:0/1536696587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f213006a750 msgr2=0x7f213006ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:29.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.175+0000 7f2137d32700 1 --2- 172.21.15.43:0/1536696587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f213006a750 0x7f213006ab30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f212c009990 tx=0x7f212c009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:29.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.175+0000 7f2137d32700 1 -- 172.21.15.43:0/1536696587 shutdown_connections 2024-04-21T08:37:29.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.175+0000 7f2137d32700 1 --2- 172.21.15.43:0/1536696587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f213010f4f0 0x7f21301118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:29.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.175+0000 7f2137d32700 1 --2- 172.21.15.43:0/1536696587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f213006a750 0x7f213006ab30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:29.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.175+0000 7f2137d32700 1 --2- 172.21.15.43:0/1536696587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f213006b070 0x7f213010efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:29.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.175+0000 7f2137d32700 1 -- 172.21.15.43:0/1536696587 >> 172.21.15.43:0/1536696587 conn(0x7f21300788f0 msgr2=0x7f2130078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:29.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.175+0000 7f2137d32700 1 -- 172.21.15.43:0/1536696587 shutdown_connections 2024-04-21T08:37:29.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.175+0000 7f2137d32700 1 -- 172.21.15.43:0/1536696587 wait complete. 2024-04-21T08:37:29.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.176+0000 7f2137d32700 1 Processor -- start 2024-04-21T08:37:29.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.176+0000 7f2137d32700 1 -- start start 2024-04-21T08:37:29.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.176+0000 7f2137d32700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f213006a750 0x7f213019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:29.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.176+0000 7f2137d32700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f213006b070 0x7f213019b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:29.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.176+0000 7f2137d32700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f213010f4f0 0x7f213019fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:29.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.176+0000 7f2137d32700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2130114e00 con 0x7f213006b070 2024-04-21T08:37:29.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.176+0000 7f2137d32700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2130114c80 con 0x7f213010f4f0 2024-04-21T08:37:29.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.176+0000 7f2137d32700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2130114f80 con 0x7f213006a750 2024-04-21T08:37:29.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.176+0000 7f21352cd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f213006b070 0x7f213019b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:29.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.176+0000 7f21352cd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f213006b070 0x7f213019b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45296/0 (socket says 172.21.15.43:45296) 2024-04-21T08:37:29.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.176+0000 7f21352cd700 1 -- 172.21.15.43:0/2486073959 learned_addr learned my addr 172.21.15.43:0/2486073959 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:37:29.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.176+0000 7f21362cf700 1 --2- 172.21.15.43:0/2486073959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f213010f4f0 0x7f213019fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:29.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.176+0000 7f2135ace700 1 --2- 172.21.15.43:0/2486073959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f213006a750 0x7f213019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:29.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.176+0000 7f21352cd700 1 -- 172.21.15.43:0/2486073959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f213006a750 msgr2=0x7f213019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:29.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.176+0000 7f21352cd700 1 --2- 172.21.15.43:0/2486073959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f213006a750 0x7f213019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:29.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.176+0000 7f21352cd700 1 -- 172.21.15.43:0/2486073959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f213010f4f0 msgr2=0x7f213019fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:29.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.176+0000 7f21352cd700 1 --2- 172.21.15.43:0/2486073959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f213010f4f0 0x7f213019fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:29.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.176+0000 7f21352cd700 1 -- 172.21.15.43:0/2486073959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21301a02f0 con 0x7f213006b070 2024-04-21T08:37:29.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.177+0000 7f21352cd700 1 --2- 172.21.15.43:0/2486073959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f213006b070 0x7f213019b870 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f212000c920 tx=0x7f212000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:29.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.177+0000 7f2126ffd700 1 -- 172.21.15.43:0/2486073959 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120007b00 con 0x7f213006b070 2024-04-21T08:37:29.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.177+0000 7f2126ffd700 1 -- 172.21.15.43:0/2486073959 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2120004ce0 con 0x7f213006b070 2024-04-21T08:37:29.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.177+0000 7f2137d32700 1 -- 172.21.15.43:0/2486073959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21301a05e0 con 0x7f213006b070 2024-04-21T08:37:29.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.177+0000 7f2126ffd700 1 -- 172.21.15.43:0/2486073959 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120005550 con 0x7f213006b070 2024-04-21T08:37:29.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.177+0000 7f2137d32700 1 -- 172.21.15.43:0/2486073959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21301a7e60 con 0x7f213006b070 2024-04-21T08:37:29.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.178+0000 7f2137d32700 1 -- 172.21.15.43:0/2486073959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2130063190 con 0x7f213006b070 2024-04-21T08:37:29.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.179+0000 7f2126ffd700 1 -- 172.21.15.43:0/2486073959 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2120007ca0 con 0x7f213006b070 2024-04-21T08:37:29.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.180+0000 7f2126ffd700 1 --2- 172.21.15.43:0/2486073959 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f211c0a3cd0 0x7f211c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:29.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.180+0000 7f2135ace700 1 --2- 172.21.15.43:0/2486073959 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f211c0a3cd0 0x7f211c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:29.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.180+0000 7f2126ffd700 1 -- 172.21.15.43:0/2486073959 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f21200d09a0 con 0x7f213006b070 2024-04-21T08:37:29.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.180+0000 7f2126ffd700 1 --2- 172.21.15.43:0/2486073959 >> [v2:172.21.15.43:6810/117814050,v1:172.21.15.43:6811/117814050] conn(0x7f211c0aee30 0x7f211c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:29.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.180+0000 7f2126ffd700 1 -- 172.21.15.43:0/2486073959 --> [v2:172.21.15.43:6810/117814050,v1:172.21.15.43:6811/117814050] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f211c0b19a0 con 0x7f211c0aee30 2024-04-21T08:37:29.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.180+0000 7f2126ffd700 1 -- 172.21.15.43:0/2486073959 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f21200d0df0 con 0x7f213006b070 2024-04-21T08:37:29.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.180+0000 7f21362cf700 1 --2- 172.21.15.43:0/2486073959 >> [v2:172.21.15.43:6810/117814050,v1:172.21.15.43:6811/117814050] conn(0x7f211c0aee30 0x7f211c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:29.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.180+0000 7f2135ace700 1 --2- 172.21.15.43:0/2486073959 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f211c0a3cd0 0x7f211c0a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f212c006010 tx=0x7f212c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:29.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.181+0000 7f21362cf700 1 --2- 172.21.15.43:0/2486073959 >> [v2:172.21.15.43:6810/117814050,v1:172.21.15.43:6811/117814050] conn(0x7f211c0aee30 0x7f211c0b12d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:29.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.181+0000 7f2126ffd700 1 -- 172.21.15.43:0/2486073959 <== osd.3 v2:172.21.15.43:6810/117814050 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f211c0b19a0 con 0x7f211c0aee30 2024-04-21T08:37:29.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.198+0000 7f2137d32700 1 -- 172.21.15.43:0/2486073959 --> [v2:172.21.15.43:6810/117814050,v1:172.21.15.43:6811/117814050] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2130063610 con 0x7f211c0aee30 2024-04-21T08:37:29.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.198+0000 7f2126ffd700 1 -- 172.21.15.43:0/2486073959 <== osd.3 v2:172.21.15.43:6810/117814050 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2130063610 con 0x7f211c0aee30 2024-04-21T08:37:29.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.198+0000 7f2137d32700 1 -- 172.21.15.43:0/2486073959 >> [v2:172.21.15.43:6810/117814050,v1:172.21.15.43:6811/117814050] conn(0x7f211c0aee30 msgr2=0x7f211c0b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:29.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.198+0000 7f2137d32700 1 --2- 172.21.15.43:0/2486073959 >> [v2:172.21.15.43:6810/117814050,v1:172.21.15.43:6811/117814050] conn(0x7f211c0aee30 0x7f211c0b12d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:29.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.199+0000 7f2137d32700 1 -- 172.21.15.43:0/2486073959 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f211c0a3cd0 msgr2=0x7f211c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:29.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.199+0000 7f2137d32700 1 --2- 172.21.15.43:0/2486073959 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f211c0a3cd0 0x7f211c0a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f212c006010 tx=0x7f212c01a040 comp rx=0 tx=0).stop 2024-04-21T08:37:29.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.199+0000 7f2137d32700 1 -- 172.21.15.43:0/2486073959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f213006b070 msgr2=0x7f213019b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:29.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.199+0000 7f2137d32700 1 --2- 172.21.15.43:0/2486073959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f213006b070 0x7f213019b870 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f212000c920 tx=0x7f212000cc30 comp rx=0 tx=0).stop 2024-04-21T08:37:29.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.199+0000 7f21362cf700 1 -- 172.21.15.43:0/2486073959 reap_dead start 2024-04-21T08:37:29.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.199+0000 7f2137d32700 1 -- 172.21.15.43:0/2486073959 shutdown_connections 2024-04-21T08:37:29.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.199+0000 7f2137d32700 1 -- 172.21.15.43:0/2486073959 >> 172.21.15.43:0/2486073959 conn(0x7f21300788f0 msgr2=0x7f21301109e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:29.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.199+0000 7f2137d32700 1 -- 172.21.15.43:0/2486073959 shutdown_connections 2024-04-21T08:37:29.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:29.199+0000 7f2137d32700 1 -- 172.21.15.43:0/2486073959 wait complete. 2024-04-21T08:37:29.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:29 smithi112 ceph-mon[31085]: pgmap v329: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:37:29.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:29 smithi043 ceph-mon[27978]: pgmap v329: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:37:29.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:29 smithi175 ceph-mon[31920]: pgmap v329: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:37:30.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.507+0000 7febf569b700 1 -- 172.21.15.43:0/2815217106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febf006b370 msgr2=0x7febf006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:30.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.507+0000 7febf569b700 1 --2- 172.21.15.43:0/2815217106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febf006b370 0x7febf006b750 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7febe00099c0 tx=0x7febe0009cd0 comp rx=0 tx=0).stop 2024-04-21T08:37:30.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.508+0000 7febf569b700 1 -- 172.21.15.43:0/2815217106 shutdown_connections 2024-04-21T08:37:30.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.508+0000 7febf569b700 1 --2- 172.21.15.43:0/2815217106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febf0107e10 0x7febf010f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:30.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.508+0000 7febf569b700 1 --2- 172.21.15.43:0/2815217106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febf006b370 0x7febf006b750 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:30.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.508+0000 7febf569b700 1 --2- 172.21.15.43:0/2815217106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febf006bd20 0x7febf01076e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:30.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.508+0000 7febf569b700 1 -- 172.21.15.43:0/2815217106 >> 172.21.15.43:0/2815217106 conn(0x7febf00788f0 msgr2=0x7febf0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:30.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.508+0000 7febf569b700 1 -- 172.21.15.43:0/2815217106 shutdown_connections 2024-04-21T08:37:30.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.508+0000 7febf569b700 1 -- 172.21.15.43:0/2815217106 wait complete. 2024-04-21T08:37:30.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.508+0000 7febf569b700 1 Processor -- start 2024-04-21T08:37:30.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.508+0000 7febf569b700 1 -- start start 2024-04-21T08:37:30.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.509+0000 7febf569b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febf006b370 0x7febf0199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:30.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.509+0000 7febf569b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febf006bd20 0x7febf01995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:30.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.509+0000 7febf569b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febf0107e10 0x7febf019d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:30.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.509+0000 7febf569b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7febf0112be0 con 0x7febf006b370 2024-04-21T08:37:30.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.509+0000 7febf569b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7febf0112a60 con 0x7febf006bd20 2024-04-21T08:37:30.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.509+0000 7febf569b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7febf0112d60 con 0x7febf0107e10 2024-04-21T08:37:30.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.509+0000 7febeeffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febf006b370 0x7febf0199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:30.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.509+0000 7febeeffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febf006b370 0x7febf0199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45302/0 (socket says 172.21.15.43:45302) 2024-04-21T08:37:30.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.509+0000 7febeeffd700 1 -- 172.21.15.43:0/2493836347 learned_addr learned my addr 172.21.15.43:0/2493836347 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:37:30.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.509+0000 7febee7fc700 1 --2- 172.21.15.43:0/2493836347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febf006bd20 0x7febf01995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:30.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.509+0000 7febef7fe700 1 --2- 172.21.15.43:0/2493836347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febf0107e10 0x7febf019d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:30.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.509+0000 7febeeffd700 1 -- 172.21.15.43:0/2493836347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febf0107e10 msgr2=0x7febf019d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:30.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.509+0000 7febeeffd700 1 --2- 172.21.15.43:0/2493836347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febf0107e10 0x7febf019d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:30.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.509+0000 7febeeffd700 1 -- 172.21.15.43:0/2493836347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febf006bd20 msgr2=0x7febf01995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:30.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.509+0000 7febeeffd700 1 --2- 172.21.15.43:0/2493836347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febf006bd20 0x7febf01995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:30.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.509+0000 7febeeffd700 1 -- 172.21.15.43:0/2493836347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf019e0a0 con 0x7febf006b370 2024-04-21T08:37:30.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.509+0000 7febeeffd700 1 --2- 172.21.15.43:0/2493836347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febf006b370 0x7febf0199090 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7febe000b580 tx=0x7febe00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:30.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.509+0000 7febd7fff700 1 -- 172.21.15.43:0/2493836347 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0027070 con 0x7febf006b370 2024-04-21T08:37:30.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.510+0000 7febf569b700 1 -- 172.21.15.43:0/2493836347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf019e330 con 0x7febf006b370 2024-04-21T08:37:30.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.510+0000 7febd7fff700 1 -- 172.21.15.43:0/2493836347 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febe0003f20 con 0x7febf006b370 2024-04-21T08:37:30.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.510+0000 7febd7fff700 1 -- 172.21.15.43:0/2493836347 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0022d10 con 0x7febf006b370 2024-04-21T08:37:30.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.510+0000 7febf569b700 1 -- 172.21.15.43:0/2493836347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf0101d20 con 0x7febf006b370 2024-04-21T08:37:30.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.510+0000 7febf569b700 1 -- 172.21.15.43:0/2493836347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7febf00631d0 con 0x7febf006b370 2024-04-21T08:37:30.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.512+0000 7febd7fff700 1 -- 172.21.15.43:0/2493836347 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7febe0022470 con 0x7febf006b370 2024-04-21T08:37:30.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.512+0000 7febd7fff700 1 --2- 172.21.15.43:0/2493836347 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7febd80a3cd0 0x7febd80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:30.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.512+0000 7febd7fff700 1 -- 172.21.15.43:0/2493836347 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7febe00d1b10 con 0x7febf006b370 2024-04-21T08:37:30.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.512+0000 7febee7fc700 1 --2- 172.21.15.43:0/2493836347 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7febd80a3cd0 0x7febd80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:30.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.512+0000 7febd7fff700 1 --2- 172.21.15.43:0/2493836347 >> [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202] conn(0x7febd80aed00 0x7febd80b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:30.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.512+0000 7febd7fff700 1 -- 172.21.15.43:0/2493836347 --> [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7febe000eea0 con 0x7febd80aed00 2024-04-21T08:37:30.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.512+0000 7febd7fff700 1 -- 172.21.15.43:0/2493836347 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7febe00d1f60 con 0x7febf006b370 2024-04-21T08:37:30.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.513+0000 7febef7fe700 1 --2- 172.21.15.43:0/2493836347 >> [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202] conn(0x7febd80aed00 0x7febd80b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:30.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.513+0000 7febee7fc700 1 --2- 172.21.15.43:0/2493836347 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7febd80a3cd0 0x7febd80a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7febf01018d0 tx=0x7febe40073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:30.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.513+0000 7febef7fe700 1 --2- 172.21.15.43:0/2493836347 >> [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202] conn(0x7febd80aed00 0x7febd80b11a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:30.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.514+0000 7febd7fff700 1 -- 172.21.15.43:0/2493836347 <== osd.6 v2:172.21.15.43:6818/4220470202 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7febe000eea0 con 0x7febd80aed00 2024-04-21T08:37:30.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.530+0000 7febf569b700 1 -- 172.21.15.43:0/2493836347 --> [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7febf0063650 con 0x7febd80aed00 2024-04-21T08:37:30.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.530+0000 7febd7fff700 1 -- 172.21.15.43:0/2493836347 <== osd.6 v2:172.21.15.43:6818/4220470202 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7febf0063650 con 0x7febd80aed00 2024-04-21T08:37:30.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.530+0000 7febf569b700 1 -- 172.21.15.43:0/2493836347 >> [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202] conn(0x7febd80aed00 msgr2=0x7febd80b11a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:30.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.530+0000 7febf569b700 1 --2- 172.21.15.43:0/2493836347 >> [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202] conn(0x7febd80aed00 0x7febd80b11a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:30.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.531+0000 7febf569b700 1 -- 172.21.15.43:0/2493836347 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7febd80a3cd0 msgr2=0x7febd80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:30.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.531+0000 7febf569b700 1 --2- 172.21.15.43:0/2493836347 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7febd80a3cd0 0x7febd80a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7febf01018d0 tx=0x7febe40073d0 comp rx=0 tx=0).stop 2024-04-21T08:37:30.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.531+0000 7febf569b700 1 -- 172.21.15.43:0/2493836347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febf006b370 msgr2=0x7febf0199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:30.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.531+0000 7febf569b700 1 --2- 172.21.15.43:0/2493836347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febf006b370 0x7febf0199090 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7febe000b580 tx=0x7febe00047b0 comp rx=0 tx=0).stop 2024-04-21T08:37:30.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.531+0000 7febef7fe700 1 -- 172.21.15.43:0/2493836347 reap_dead start 2024-04-21T08:37:30.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.531+0000 7febf569b700 1 -- 172.21.15.43:0/2493836347 shutdown_connections 2024-04-21T08:37:30.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.531+0000 7febf569b700 1 -- 172.21.15.43:0/2493836347 >> 172.21.15.43:0/2493836347 conn(0x7febf00788f0 msgr2=0x7febf00fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:30.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.531+0000 7febf569b700 1 -- 172.21.15.43:0/2493836347 shutdown_connections 2024-04-21T08:37:30.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:30.531+0000 7febf569b700 1 -- 172.21.15.43:0/2493836347 wait complete. 2024-04-21T08:37:31.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:31 smithi112 ceph-mon[31085]: pgmap v330: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-21T08:37:31.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:31 smithi043 ceph-mon[27978]: pgmap v330: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-21T08:37:31.641 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:31.646 INFO:teuthology.orchestra.run.smithi043.stdout:107374182467 2024-04-21T08:37:31.646 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-21T08:37:31.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:31 smithi175 ceph-mon[31920]: pgmap v330: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-21T08:37:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.789+0000 7fde94328700 1 -- 172.21.15.43:0/4133097162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde8c102300 msgr2=0x7fde8c10e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.789+0000 7fde94328700 1 --2- 172.21.15.43:0/4133097162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde8c102300 0x7fde8c10e7f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fde8400b0a0 tx=0x7fde8400b3b0 comp rx=0 tx=0).stop 2024-04-21T08:37:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.789+0000 7fde94328700 1 -- 172.21.15.43:0/4133097162 shutdown_connections 2024-04-21T08:37:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.789+0000 7fde94328700 1 --2- 172.21.15.43:0/4133097162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde8c107990 0x7fde8c107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.789+0000 7fde94328700 1 --2- 172.21.15.43:0/4133097162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde8c102300 0x7fde8c10e7f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.789+0000 7fde94328700 1 --2- 172.21.15.43:0/4133097162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde8c101940 0x7fde8c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.789+0000 7fde94328700 1 -- 172.21.15.43:0/4133097162 >> 172.21.15.43:0/4133097162 conn(0x7fde8c0788f0 msgr2=0x7fde8c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.789+0000 7fde94328700 1 -- 172.21.15.43:0/4133097162 shutdown_connections 2024-04-21T08:37:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.789+0000 7fde94328700 1 -- 172.21.15.43:0/4133097162 wait complete. 2024-04-21T08:37:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.790+0000 7fde94328700 1 Processor -- start 2024-04-21T08:37:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.790+0000 7fde94328700 1 -- start start 2024-04-21T08:37:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.790+0000 7fde94328700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde8c101940 0x7fde8c1034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.790+0000 7fde94328700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde8c102300 0x7fde8c103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.790+0000 7fde94328700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde8c107990 0x7fde8c103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.790+0000 7fde94328700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c111ae0 con 0x7fde8c102300 2024-04-21T08:37:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.790+0000 7fde94328700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c111960 con 0x7fde8c101940 2024-04-21T08:37:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.790+0000 7fde94328700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c111c60 con 0x7fde8c107990 2024-04-21T08:37:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.790+0000 7fde918c3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde8c102300 0x7fde8c103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.791+0000 7fde918c3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde8c102300 0x7fde8c103a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45312/0 (socket says 172.21.15.43:45312) 2024-04-21T08:37:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.791+0000 7fde918c3700 1 -- 172.21.15.43:0/427212671 learned_addr learned my addr 172.21.15.43:0/427212671 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:37:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.791+0000 7fde920c4700 1 --2- 172.21.15.43:0/427212671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde8c101940 0x7fde8c1034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.791+0000 7fde928c5700 1 --2- 172.21.15.43:0/427212671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde8c107990 0x7fde8c103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.791+0000 7fde918c3700 1 -- 172.21.15.43:0/427212671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde8c107990 msgr2=0x7fde8c103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.791+0000 7fde918c3700 1 --2- 172.21.15.43:0/427212671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde8c107990 0x7fde8c103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.791+0000 7fde918c3700 1 -- 172.21.15.43:0/427212671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde8c101940 msgr2=0x7fde8c1034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.791+0000 7fde918c3700 1 --2- 172.21.15.43:0/427212671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde8c101940 0x7fde8c1034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.791+0000 7fde918c3700 1 -- 172.21.15.43:0/427212671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8c1a7540 con 0x7fde8c102300 2024-04-21T08:37:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.791+0000 7fde920c4700 1 --2- 172.21.15.43:0/427212671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde8c101940 0x7fde8c1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:37:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.791+0000 7fde918c3700 1 --2- 172.21.15.43:0/427212671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde8c102300 0x7fde8c103a30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00d950 tx=0x7fde7c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.791+0000 7fde837fe700 1 -- 172.21.15.43:0/427212671 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c016070 con 0x7fde8c102300 2024-04-21T08:37:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.791+0000 7fde94328700 1 -- 172.21.15.43:0/427212671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde8c1a7830 con 0x7fde8c102300 2024-04-21T08:37:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.791+0000 7fde837fe700 1 -- 172.21.15.43:0/427212671 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde7c010460 con 0x7fde8c102300 2024-04-21T08:37:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.791+0000 7fde837fe700 1 -- 172.21.15.43:0/427212671 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c00f620 con 0x7fde8c102300 2024-04-21T08:37:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.792+0000 7fde94328700 1 -- 172.21.15.43:0/427212671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde8c1a7dc0 con 0x7fde8c102300 2024-04-21T08:37:32.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.793+0000 7fde94328700 1 -- 172.21.15.43:0/427212671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fde8c105880 con 0x7fde8c102300 2024-04-21T08:37:32.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.794+0000 7fde837fe700 1 -- 172.21.15.43:0/427212671 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fde7c019070 con 0x7fde8c102300 2024-04-21T08:37:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.794+0000 7fde837fe700 1 --2- 172.21.15.43:0/427212671 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fde780a3cd0 0x7fde780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.795+0000 7fde920c4700 1 --2- 172.21.15.43:0/427212671 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fde780a3cd0 0x7fde780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.795+0000 7fde837fe700 1 -- 172.21.15.43:0/427212671 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fde7c0d0910 con 0x7fde8c102300 2024-04-21T08:37:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.795+0000 7fde837fe700 1 --2- 172.21.15.43:0/427212671 >> [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768] conn(0x7fde780aee30 0x7fde780b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.795+0000 7fde837fe700 1 -- 172.21.15.43:0/427212671 --> [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fde780b19a0 con 0x7fde780aee30 2024-04-21T08:37:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.795+0000 7fde837fe700 1 -- 172.21.15.43:0/427212671 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fde7c0d52c0 con 0x7fde8c102300 2024-04-21T08:37:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.795+0000 7fde928c5700 1 --2- 172.21.15.43:0/427212671 >> [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768] conn(0x7fde780aee30 0x7fde780b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.795+0000 7fde920c4700 1 --2- 172.21.15.43:0/427212671 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fde780a3cd0 0x7fde780a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fde88005fd0 tx=0x7fde8800a5e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.795+0000 7fde928c5700 1 --2- 172.21.15.43:0/427212671 >> [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768] conn(0x7fde780aee30 0x7fde780b12d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.796+0000 7fde837fe700 1 -- 172.21.15.43:0/427212671 <== osd.0 v2:172.21.15.43:6802/4223002768 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fde780b19a0 con 0x7fde780aee30 2024-04-21T08:37:32.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.814+0000 7fde94328700 1 -- 172.21.15.43:0/427212671 --> [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fde8c103060 con 0x7fde780aee30 2024-04-21T08:37:32.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.814+0000 7fde837fe700 1 -- 172.21.15.43:0/427212671 <== osd.0 v2:172.21.15.43:6802/4223002768 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fde8c103060 con 0x7fde780aee30 2024-04-21T08:37:32.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.815+0000 7fde94328700 1 -- 172.21.15.43:0/427212671 >> [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768] conn(0x7fde780aee30 msgr2=0x7fde780b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:32.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.815+0000 7fde94328700 1 --2- 172.21.15.43:0/427212671 >> [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768] conn(0x7fde780aee30 0x7fde780b12d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:32.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.815+0000 7fde94328700 1 -- 172.21.15.43:0/427212671 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fde780a3cd0 msgr2=0x7fde780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:32.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.815+0000 7fde94328700 1 --2- 172.21.15.43:0/427212671 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fde780a3cd0 0x7fde780a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fde88005fd0 tx=0x7fde8800a5e0 comp rx=0 tx=0).stop 2024-04-21T08:37:32.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.815+0000 7fde94328700 1 -- 172.21.15.43:0/427212671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde8c102300 msgr2=0x7fde8c103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:32.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.815+0000 7fde94328700 1 --2- 172.21.15.43:0/427212671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde8c102300 0x7fde8c103a30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00d950 tx=0x7fde7c00dd10 comp rx=0 tx=0).stop 2024-04-21T08:37:32.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.815+0000 7fde928c5700 1 -- 172.21.15.43:0/427212671 reap_dead start 2024-04-21T08:37:32.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.815+0000 7fde94328700 1 -- 172.21.15.43:0/427212671 shutdown_connections 2024-04-21T08:37:32.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.815+0000 7fde94328700 1 -- 172.21.15.43:0/427212671 >> 172.21.15.43:0/427212671 conn(0x7fde8c0788f0 msgr2=0x7fde8c100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:32.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.815+0000 7fde94328700 1 -- 172.21.15.43:0/427212671 shutdown_connections 2024-04-21T08:37:32.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:32.815+0000 7fde94328700 1 -- 172.21.15.43:0/427212671 wait complete. 2024-04-21T08:37:33.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:33 smithi112 ceph-mon[31085]: pgmap v331: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-21T08:37:33.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:33 smithi043 ceph-mon[27978]: pgmap v331: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-21T08:37:33.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:33 smithi175 ceph-mon[31920]: pgmap v331: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-21T08:37:33.911 INFO:teuthology.orchestra.run.smithi043.stdout:77309411400 2024-04-21T08:37:33.911 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-21T08:37:34.470 INFO:teuthology.orchestra.run.smithi043.stdout:34359738452 2024-04-21T08:37:34.470 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-21T08:37:35.123 INFO:teuthology.orchestra.run.smithi043.stdout:201863462964 2024-04-21T08:37:35.124 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-21T08:37:35.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:35 smithi112 ceph-mon[31085]: pgmap v332: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-21T08:37:35.550 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:35 smithi043 ceph-mon[27978]: pgmap v332: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-21T08:37:35.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:35 smithi175 ceph-mon[31920]: pgmap v332: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-21T08:37:36.014 INFO:teuthology.orchestra.run.smithi043.stdout:292057776165 2024-04-21T08:37:36.014 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-21T08:37:36.257 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:36.258 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:36.279 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:36.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.290+0000 7f8ec5fc7700 1 -- 172.21.15.43:0/19267632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec011c220 msgr2=0x7f8ec011e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:36.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.290+0000 7f8ec5fc7700 1 --2- 172.21.15.43:0/19267632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec011c220 0x7f8ec011e610 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8eb00099e0 tx=0x7f8eb0009cf0 comp rx=0 tx=0).stop 2024-04-21T08:37:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.297+0000 7f8ec5fc7700 1 -- 172.21.15.43:0/19267632 shutdown_connections 2024-04-21T08:37:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.297+0000 7f8ec5fc7700 1 --2- 172.21.15.43:0/19267632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ec0111130 0x7f8ec0109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.297+0000 7f8ec5fc7700 1 --2- 172.21.15.43:0/19267632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec011c220 0x7f8ec011e610 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.297+0000 7f8ec5fc7700 1 --2- 172.21.15.43:0/19267632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec01115a0 0x7f8ec0109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.297+0000 7f8ec5fc7700 1 -- 172.21.15.43:0/19267632 >> 172.21.15.43:0/19267632 conn(0x7f8ec006da50 msgr2=0x7f8ec006de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.300+0000 7f8ec5fc7700 1 -- 172.21.15.43:0/19267632 shutdown_connections 2024-04-21T08:37:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.300+0000 7f8ec5fc7700 1 -- 172.21.15.43:0/19267632 wait complete. 2024-04-21T08:37:36.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.301+0000 7f8ec5fc7700 1 Processor -- start 2024-04-21T08:37:36.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.304+0000 7f8ec5fc7700 1 -- start start 2024-04-21T08:37:36.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.304+0000 7f8ec5fc7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec0111130 0x7f8ec01a7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.304+0000 7f8ec5fc7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ec01115a0 0x7f8ec01a8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.304+0000 7f8ec5fc7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec011c220 0x7f8ec01ac600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.304+0000 7f8ec5fc7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec0121a50 con 0x7f8ec0111130 2024-04-21T08:37:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.304+0000 7f8ec5fc7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec01218d0 con 0x7f8ec011c220 2024-04-21T08:37:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.304+0000 7f8ec5fc7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec0121bd0 con 0x7f8ec01115a0 2024-04-21T08:37:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.304+0000 7f8ebf7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec0111130 0x7f8ec01a7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.305+0000 7f8ebf7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec0111130 0x7f8ec01a7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45322/0 (socket says 172.21.15.43:45322) 2024-04-21T08:37:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.305+0000 7f8ebf7fe700 1 -- 172.21.15.43:0/2161613740 learned_addr learned my addr 172.21.15.43:0/2161613740 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:37:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.305+0000 7f8ebf7fe700 1 -- 172.21.15.43:0/2161613740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ec01115a0 msgr2=0x7f8ec01a8270 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:37:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.305+0000 7f8ebf7fe700 1 --2- 172.21.15.43:0/2161613740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ec01115a0 0x7f8ec01a8270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.305+0000 7f8ebf7fe700 1 -- 172.21.15.43:0/2161613740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec011c220 msgr2=0x7f8ec01ac600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:37:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.305+0000 7f8ebf7fe700 1 --2- 172.21.15.43:0/2161613740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec011c220 0x7f8ec01ac600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.305+0000 7f8ebf7fe700 1 -- 172.21.15.43:0/2161613740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec01acd40 con 0x7f8ec0111130 2024-04-21T08:37:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.305+0000 7f8ebf7fe700 1 --2- 172.21.15.43:0/2161613740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec0111130 0x7f8ec01a7d30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400ed60 tx=0x7f8eb400c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.309+0000 7f8ebcff9700 1 -- 172.21.15.43:0/2161613740 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb400cbc0 con 0x7f8ec0111130 2024-04-21T08:37:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.309+0000 7f8ebcff9700 1 -- 172.21.15.43:0/2161613740 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eb400cd60 con 0x7f8ec0111130 2024-04-21T08:37:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.310+0000 7f8ec5fc7700 1 -- 172.21.15.43:0/2161613740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec01ad030 con 0x7f8ec0111130 2024-04-21T08:37:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.310+0000 7f8ec5fc7700 1 -- 172.21.15.43:0/2161613740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec01b48b0 con 0x7f8ec0111130 2024-04-21T08:37:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.310+0000 7f8ebcff9700 1 -- 172.21.15.43:0/2161613740 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4018710 con 0x7f8ec0111130 2024-04-21T08:37:36.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.314+0000 7f8ec5fc7700 1 -- 172.21.15.43:0/2161613740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec006eeb0 con 0x7f8ec0111130 2024-04-21T08:37:36.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.324+0000 7f8ebcff9700 1 -- 172.21.15.43:0/2161613740 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8eb401f020 con 0x7f8ec0111130 2024-04-21T08:37:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.325+0000 7f8ebcff9700 1 --2- 172.21.15.43:0/2161613740 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8ea80a3c40 0x7f8ea80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.325+0000 7f8ebcff9700 1 -- 172.21.15.43:0/2161613740 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f8eb40d0880 con 0x7f8ec0111130 2024-04-21T08:37:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.326+0000 7f8ebcff9700 1 -- 172.21.15.43:0/2161613740 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8eb401f310 con 0x7f8ec0111130 2024-04-21T08:37:36.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.327+0000 7f8ebeffd700 1 --2- 172.21.15.43:0/2161613740 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8ea80a3c40 0x7f8ea80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:36.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.328+0000 7f8ebeffd700 1 --2- 172.21.15.43:0/2161613740 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8ea80a3c40 0x7f8ea80a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8006130 tx=0x7f8eb8007500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:36.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.547+0000 7f8ec5fc7700 1 -- 172.21.15.43:0/2161613740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f8ec01a9090 con 0x7f8ec0111130 2024-04-21T08:37:36.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.548+0000 7f8ebcff9700 1 -- 172.21.15.43:0/2161613740 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8eb4098020 con 0x7f8ec0111130 2024-04-21T08:37:36.547 INFO:teuthology.orchestra.run.smithi043.stdout:257698037804 2024-04-21T08:37:36.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.550+0000 7f8ec5fc7700 1 -- 172.21.15.43:0/2161613740 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8ea80a3c40 msgr2=0x7f8ea80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:36.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.550+0000 7f8ec5fc7700 1 --2- 172.21.15.43:0/2161613740 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8ea80a3c40 0x7f8ea80a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8006130 tx=0x7f8eb8007500 comp rx=0 tx=0).stop 2024-04-21T08:37:36.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.550+0000 7f8ec5fc7700 1 -- 172.21.15.43:0/2161613740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec0111130 msgr2=0x7f8ec01a7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:36.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.550+0000 7f8ec5fc7700 1 --2- 172.21.15.43:0/2161613740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec0111130 0x7f8ec01a7d30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400ed60 tx=0x7f8eb400c5b0 comp rx=0 tx=0).stop 2024-04-21T08:37:36.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.550+0000 7f8ec5fc7700 1 -- 172.21.15.43:0/2161613740 shutdown_connections 2024-04-21T08:37:36.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.550+0000 7f8ec5fc7700 1 --2- 172.21.15.43:0/2161613740 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8ea80a3c40 0x7f8ea80a6100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:36.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.550+0000 7f8ec5fc7700 1 --2- 172.21.15.43:0/2161613740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec011c220 0x7f8ec01ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:36.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.550+0000 7f8ec5fc7700 1 --2- 172.21.15.43:0/2161613740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec0111130 0x7f8ec01a7d30 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:36.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.550+0000 7f8ec5fc7700 1 --2- 172.21.15.43:0/2161613740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ec01115a0 0x7f8ec01a8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:36.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.550+0000 7f8ec5fc7700 1 -- 172.21.15.43:0/2161613740 >> 172.21.15.43:0/2161613740 conn(0x7f8ec006da50 msgr2=0x7f8ec011a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:36.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.550+0000 7f8ec5fc7700 1 -- 172.21.15.43:0/2161613740 shutdown_connections 2024-04-21T08:37:36.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:36.550+0000 7f8ec5fc7700 1 -- 172.21.15.43:0/2161613740 wait complete. 2024-04-21T08:37:37.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:37 smithi112 ceph-mon[31085]: pgmap v333: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:37:37.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:37 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2161613740' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-21T08:37:37.557 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:37 smithi043 ceph-mon[27978]: pgmap v333: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:37:37.557 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:37 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2161613740' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-21T08:37:37.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:37 smithi175 ceph-mon[31920]: pgmap v333: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:37:37.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:37 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2161613740' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-21T08:37:38.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.970+0000 7fb3c35dd700 1 -- 172.21.15.43:0/1753867942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3bc101940 msgr2=0x7fb3bc101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:38.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.970+0000 7fb3c35dd700 1 --2- 172.21.15.43:0/1753867942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3bc101940 0x7fb3bc101dc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009990 tx=0x7fb3ac009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:38.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.970+0000 7fb3c35dd700 1 -- 172.21.15.43:0/1753867942 shutdown_connections 2024-04-21T08:37:38.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.970+0000 7fb3c35dd700 1 --2- 172.21.15.43:0/1753867942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3bc107990 0x7fb3bc107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:38.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.970+0000 7fb3c35dd700 1 --2- 172.21.15.43:0/1753867942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3bc102300 0x7fb3bc10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:38.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.970+0000 7fb3c35dd700 1 --2- 172.21.15.43:0/1753867942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3bc101940 0x7fb3bc101dc0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:38.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.970+0000 7fb3c35dd700 1 -- 172.21.15.43:0/1753867942 >> 172.21.15.43:0/1753867942 conn(0x7fb3bc0788f0 msgr2=0x7fb3bc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:38.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.970+0000 7fb3c35dd700 1 -- 172.21.15.43:0/1753867942 shutdown_connections 2024-04-21T08:37:38.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.970+0000 7fb3c35dd700 1 -- 172.21.15.43:0/1753867942 wait complete. 2024-04-21T08:37:38.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.970+0000 7fb3c35dd700 1 Processor -- start 2024-04-21T08:37:38.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.971+0000 7fb3c35dd700 1 -- start start 2024-04-21T08:37:38.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.971+0000 7fb3c35dd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3bc101940 0x7fb3bc199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:38.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.971+0000 7fb3c35dd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3bc102300 0x7fb3bc19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:38.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.971+0000 7fb3c35dd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3bc107990 0x7fb3bc1a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:38.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.971+0000 7fb3c35dd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc111ae0 con 0x7fb3bc101940 2024-04-21T08:37:38.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.971+0000 7fb3c35dd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc111960 con 0x7fb3bc107990 2024-04-21T08:37:38.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.971+0000 7fb3c35dd700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc111c60 con 0x7fb3bc102300 2024-04-21T08:37:38.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.971+0000 7fb3c1379700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3bc101940 0x7fb3bc199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:38.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.971+0000 7fb3c1379700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3bc101940 0x7fb3bc199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48102/0 (socket says 172.21.15.43:48102) 2024-04-21T08:37:38.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.971+0000 7fb3c1379700 1 -- 172.21.15.43:0/544597726 learned_addr learned my addr 172.21.15.43:0/544597726 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:37:38.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.971+0000 7fb3c1b7a700 1 --2- 172.21.15.43:0/544597726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3bc107990 0x7fb3bc1a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:38.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.971+0000 7fb3c0b78700 1 --2- 172.21.15.43:0/544597726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3bc102300 0x7fb3bc19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:38.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.971+0000 7fb3c1379700 1 -- 172.21.15.43:0/544597726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3bc102300 msgr2=0x7fb3bc19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:38.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.971+0000 7fb3c1379700 1 --2- 172.21.15.43:0/544597726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3bc102300 0x7fb3bc19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:38.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.971+0000 7fb3c1379700 1 -- 172.21.15.43:0/544597726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3bc107990 msgr2=0x7fb3bc1a4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:38.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.971+0000 7fb3c1379700 1 --2- 172.21.15.43:0/544597726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3bc107990 0x7fb3bc1a4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:38.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.971+0000 7fb3c1379700 1 -- 172.21.15.43:0/544597726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3bc1a54c0 con 0x7fb3bc101940 2024-04-21T08:37:38.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.971+0000 7fb3c1379700 1 --2- 172.21.15.43:0/544597726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3bc101940 0x7fb3bc199f00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb3b800d950 tx=0x7fb3b800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:38.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.972+0000 7fb3b27fc700 1 -- 172.21.15.43:0/544597726 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8016070 con 0x7fb3bc101940 2024-04-21T08:37:38.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.972+0000 7fb3b27fc700 1 -- 172.21.15.43:0/544597726 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3b800fd40 con 0x7fb3bc101940 2024-04-21T08:37:38.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.972+0000 7fb3c35dd700 1 -- 172.21.15.43:0/544597726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3bc1a5750 con 0x7fb3bc101940 2024-04-21T08:37:38.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.972+0000 7fb3b27fc700 1 -- 172.21.15.43:0/544597726 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b800e3b0 con 0x7fb3bc101940 2024-04-21T08:37:38.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.972+0000 7fb3c35dd700 1 -- 172.21.15.43:0/544597726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3bc1a5cb0 con 0x7fb3bc101940 2024-04-21T08:37:38.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.972+0000 7fb3c35dd700 1 -- 172.21.15.43:0/544597726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb3bc063110 con 0x7fb3bc101940 2024-04-21T08:37:38.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.975+0000 7fb3b27fc700 1 -- 172.21.15.43:0/544597726 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb3b8019070 con 0x7fb3bc101940 2024-04-21T08:37:38.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.976+0000 7fb3b27fc700 1 --2- 172.21.15.43:0/544597726 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb3a80a3d50 0x7fb3a80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:38.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.976+0000 7fb3c0b78700 1 --2- 172.21.15.43:0/544597726 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb3a80a3d50 0x7fb3a80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:38.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.976+0000 7fb3b27fc700 1 -- 172.21.15.43:0/544597726 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb3b80d0680 con 0x7fb3bc101940 2024-04-21T08:37:38.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.976+0000 7fb3b27fc700 1 --2- 172.21.15.43:0/544597726 >> [v2:172.21.15.112:6824/3645019377,v1:172.21.15.112:6825/3645019377] conn(0x7fb3a80aeeb0 0x7fb3a80b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:38.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.976+0000 7fb3b27fc700 1 -- 172.21.15.43:0/544597726 --> [v2:172.21.15.112:6824/3645019377,v1:172.21.15.112:6825/3645019377] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb3b8014e80 con 0x7fb3a80aeeb0 2024-04-21T08:37:38.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.976+0000 7fb3b27fc700 1 -- 172.21.15.43:0/544597726 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fb3b80d0ad0 con 0x7fb3bc101940 2024-04-21T08:37:38.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.976+0000 7fb3c1b7a700 1 --2- 172.21.15.43:0/544597726 >> [v2:172.21.15.112:6824/3645019377,v1:172.21.15.112:6825/3645019377] conn(0x7fb3a80aeeb0 0x7fb3a80b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:38.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.976+0000 7fb3c0b78700 1 --2- 172.21.15.43:0/544597726 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb3a80a3d50 0x7fb3a80a6210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc078fd0 tx=0x7fb3ac00b560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:38.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.977+0000 7fb3c1b7a700 1 --2- 172.21.15.43:0/544597726 >> [v2:172.21.15.112:6824/3645019377,v1:172.21.15.112:6825/3645019377] conn(0x7fb3a80aeeb0 0x7fb3a80b1350 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:38.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.978+0000 7fb3b27fc700 1 -- 172.21.15.43:0/544597726 <== osd.10 v2:172.21.15.112:6824/3645019377 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb3b8014e80 con 0x7fb3a80aeeb0 2024-04-21T08:37:38.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.999+0000 7fb3c35dd700 1 -- 172.21.15.43:0/544597726 --> [v2:172.21.15.112:6824/3645019377,v1:172.21.15.112:6825/3645019377] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb3bc063540 con 0x7fb3a80aeeb0 2024-04-21T08:37:38.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:38.999+0000 7fb3b27fc700 1 -- 172.21.15.43:0/544597726 <== osd.10 v2:172.21.15.112:6824/3645019377 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb3bc063540 con 0x7fb3a80aeeb0 2024-04-21T08:37:38.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:39.000+0000 7fb3c35dd700 1 -- 172.21.15.43:0/544597726 >> [v2:172.21.15.112:6824/3645019377,v1:172.21.15.112:6825/3645019377] conn(0x7fb3a80aeeb0 msgr2=0x7fb3a80b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:38.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:39.000+0000 7fb3c35dd700 1 --2- 172.21.15.43:0/544597726 >> [v2:172.21.15.112:6824/3645019377,v1:172.21.15.112:6825/3645019377] conn(0x7fb3a80aeeb0 0x7fb3a80b1350 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:38.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:39.000+0000 7fb3c35dd700 1 -- 172.21.15.43:0/544597726 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb3a80a3d50 msgr2=0x7fb3a80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:38.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:39.000+0000 7fb3c35dd700 1 --2- 172.21.15.43:0/544597726 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb3a80a3d50 0x7fb3a80a6210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc078fd0 tx=0x7fb3ac00b560 comp rx=0 tx=0).stop 2024-04-21T08:37:39.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:39.000+0000 7fb3c35dd700 1 -- 172.21.15.43:0/544597726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3bc101940 msgr2=0x7fb3bc199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:39.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:39.000+0000 7fb3c35dd700 1 --2- 172.21.15.43:0/544597726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3bc101940 0x7fb3bc199f00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb3b800d950 tx=0x7fb3b800dc60 comp rx=0 tx=0).stop 2024-04-21T08:37:39.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:39.000+0000 7fb3c1b7a700 1 -- 172.21.15.43:0/544597726 reap_dead start 2024-04-21T08:37:39.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:39.000+0000 7fb3c35dd700 1 -- 172.21.15.43:0/544597726 shutdown_connections 2024-04-21T08:37:39.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:39.000+0000 7fb3c35dd700 1 -- 172.21.15.43:0/544597726 >> 172.21.15.43:0/544597726 conn(0x7fb3bc0788f0 msgr2=0x7fb3bc0ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:39.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:39.000+0000 7fb3c35dd700 1 -- 172.21.15.43:0/544597726 shutdown_connections 2024-04-21T08:37:39.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:39.000+0000 7fb3c35dd700 1 -- 172.21.15.43:0/544597726 wait complete. 2024-04-21T08:37:39.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:39 smithi112 ceph-mon[31085]: pgmap v334: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-21T08:37:39.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:39 smithi043 ceph-mon[27978]: pgmap v334: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-21T08:37:39.683 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037799 got 257698037804 for osd.8 2024-04-21T08:37:39.683 DEBUG:teuthology.parallel:result is None 2024-04-21T08:37:39.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:39 smithi175 ceph-mon[31920]: pgmap v334: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-21T08:37:40.607 INFO:teuthology.orchestra.run.smithi043.stdout:313532612642 2024-04-21T08:37:40.607 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-21T08:37:40.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.819+0000 7f8ed04b8700 1 -- 172.21.15.43:0/3726913783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec8107990 msgr2=0x7f8ec8107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:40.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.819+0000 7f8ed04b8700 1 --2- 172.21.15.43:0/3726913783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec8107990 0x7f8ec8107d70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009990 tx=0x7f8ec4009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:40.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.820+0000 7f8ed04b8700 1 -- 172.21.15.43:0/3726913783 shutdown_connections 2024-04-21T08:37:40.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.820+0000 7f8ed04b8700 1 --2- 172.21.15.43:0/3726913783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ec8102300 0x7f8ec810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:40.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.820+0000 7f8ed04b8700 1 --2- 172.21.15.43:0/3726913783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec8107990 0x7f8ec8107d70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:40.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.820+0000 7f8ed04b8700 1 --2- 172.21.15.43:0/3726913783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec8101940 0x7f8ec8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:40.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.820+0000 7f8ed04b8700 1 -- 172.21.15.43:0/3726913783 >> 172.21.15.43:0/3726913783 conn(0x7f8ec80788f0 msgr2=0x7f8ec8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:40.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.820+0000 7f8ed04b8700 1 -- 172.21.15.43:0/3726913783 shutdown_connections 2024-04-21T08:37:40.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.820+0000 7f8ed04b8700 1 -- 172.21.15.43:0/3726913783 wait complete. 2024-04-21T08:37:40.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.820+0000 7f8ed04b8700 1 Processor -- start 2024-04-21T08:37:40.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.820+0000 7f8ed04b8700 1 -- start start 2024-04-21T08:37:40.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.820+0000 7f8ed04b8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec8101940 0x7f8ec81990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:40.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.820+0000 7f8ed04b8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ec8102300 0x7f8ec81995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:40.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.820+0000 7f8ed04b8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec8107990 0x7f8ec819d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:40.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.820+0000 7f8ed04b8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec8111b00 con 0x7f8ec8107990 2024-04-21T08:37:40.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.821+0000 7f8ed04b8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec8111980 con 0x7f8ec8101940 2024-04-21T08:37:40.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.821+0000 7f8ed04b8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec8111c80 con 0x7f8ec8102300 2024-04-21T08:37:40.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.821+0000 7f8ecea55700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec8107990 0x7f8ec819d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:40.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.821+0000 7f8ecea55700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec8107990 0x7f8ec819d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48128/0 (socket says 172.21.15.43:48128) 2024-04-21T08:37:40.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.821+0000 7f8ecea55700 1 -- 172.21.15.43:0/1138186065 learned_addr learned my addr 172.21.15.43:0/1138186065 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:37:40.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.821+0000 7f8ecda53700 1 --2- 172.21.15.43:0/1138186065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ec8102300 0x7f8ec81995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:40.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.821+0000 7f8ece254700 1 --2- 172.21.15.43:0/1138186065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec8101940 0x7f8ec81990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:40.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.821+0000 7f8ecea55700 1 -- 172.21.15.43:0/1138186065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ec8102300 msgr2=0x7f8ec81995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:40.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.821+0000 7f8ecea55700 1 --2- 172.21.15.43:0/1138186065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ec8102300 0x7f8ec81995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:40.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.821+0000 7f8ecea55700 1 -- 172.21.15.43:0/1138186065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec8101940 msgr2=0x7f8ec81990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:40.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.821+0000 7f8ecea55700 1 --2- 172.21.15.43:0/1138186065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec8101940 0x7f8ec81990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:40.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.821+0000 7f8ecea55700 1 -- 172.21.15.43:0/1138186065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec819e070 con 0x7f8ec8107990 2024-04-21T08:37:40.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.821+0000 7f8ece254700 1 --2- 172.21.15.43:0/1138186065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec8101940 0x7f8ec81990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:37:40.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.821+0000 7f8ecea55700 1 --2- 172.21.15.43:0/1138186065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec8107990 0x7f8ec819d930 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0007ee0 tx=0x7f8ec000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:40.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.821+0000 7f8ebf7fe700 1 -- 172.21.15.43:0/1138186065 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec000dbd0 con 0x7f8ec8107990 2024-04-21T08:37:40.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.821+0000 7f8ebf7fe700 1 -- 172.21.15.43:0/1138186065 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ec000dd70 con 0x7f8ec8107990 2024-04-21T08:37:40.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.821+0000 7f8ed04b8700 1 -- 172.21.15.43:0/1138186065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec819e360 con 0x7f8ec8107990 2024-04-21T08:37:40.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.821+0000 7f8ed04b8700 1 -- 172.21.15.43:0/1138186065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec81a5be0 con 0x7f8ec8107990 2024-04-21T08:37:40.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.821+0000 7f8ebf7fe700 1 -- 172.21.15.43:0/1138186065 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0005990 con 0x7f8ec8107990 2024-04-21T08:37:40.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.822+0000 7f8ed04b8700 1 -- 172.21.15.43:0/1138186065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eac005360 con 0x7f8ec8107990 2024-04-21T08:37:40.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.823+0000 7f8ebf7fe700 1 -- 172.21.15.43:0/1138186065 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8ec001a020 con 0x7f8ec8107990 2024-04-21T08:37:40.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.824+0000 7f8ebf7fe700 1 --2- 172.21.15.43:0/1138186065 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8eb40a3cd0 0x7f8eb40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:40.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.824+0000 7f8ebf7fe700 1 -- 172.21.15.43:0/1138186065 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f8ec00cfed0 con 0x7f8ec8107990 2024-04-21T08:37:40.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.824+0000 7f8ece254700 1 --2- 172.21.15.43:0/1138186065 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8eb40a3cd0 0x7f8eb40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:40.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.825+0000 7f8ece254700 1 --2- 172.21.15.43:0/1138186065 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8eb40a3cd0 0x7f8eb40a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400b5c0 tx=0x7f8ec4005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:40.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:40.829+0000 7f8ebf7fe700 1 -- 172.21.15.43:0/1138186065 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ec0010070 con 0x7f8ec8107990 2024-04-21T08:37:41.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:41.040+0000 7f8ed04b8700 1 -- 172.21.15.43:0/1138186065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f8eac005920 con 0x7f8ec8107990 2024-04-21T08:37:41.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:41.040+0000 7f8ebf7fe700 1 -- 172.21.15.43:0/1138186065 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8ec0022250 con 0x7f8ec8107990 2024-04-21T08:37:41.040 INFO:teuthology.orchestra.run.smithi043.stdout:167503724604 2024-04-21T08:37:41.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:41.042+0000 7f8ed04b8700 1 -- 172.21.15.43:0/1138186065 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8eb40a3cd0 msgr2=0x7f8eb40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:41.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:41.042+0000 7f8ed04b8700 1 --2- 172.21.15.43:0/1138186065 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8eb40a3cd0 0x7f8eb40a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400b5c0 tx=0x7f8ec4005a90 comp rx=0 tx=0).stop 2024-04-21T08:37:41.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:41.042+0000 7f8ed04b8700 1 -- 172.21.15.43:0/1138186065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec8107990 msgr2=0x7f8ec819d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:41.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:41.042+0000 7f8ed04b8700 1 --2- 172.21.15.43:0/1138186065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec8107990 0x7f8ec819d930 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0007ee0 tx=0x7f8ec000d3b0 comp rx=0 tx=0).stop 2024-04-21T08:37:41.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:41.042+0000 7f8ed04b8700 1 -- 172.21.15.43:0/1138186065 shutdown_connections 2024-04-21T08:37:41.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:41.042+0000 7f8ed04b8700 1 --2- 172.21.15.43:0/1138186065 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8eb40a3cd0 0x7f8eb40a6190 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:41.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:41.042+0000 7f8ed04b8700 1 --2- 172.21.15.43:0/1138186065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec8107990 0x7f8ec819d930 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:41.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:41.042+0000 7f8ed04b8700 1 --2- 172.21.15.43:0/1138186065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec8101940 0x7f8ec81990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:41.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:41.042+0000 7f8ed04b8700 1 --2- 172.21.15.43:0/1138186065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ec8102300 0x7f8ec81995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:41.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:41.042+0000 7f8ed04b8700 1 -- 172.21.15.43:0/1138186065 >> 172.21.15.43:0/1138186065 conn(0x7f8ec80788f0 msgr2=0x7f8ec806a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:41.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:41.042+0000 7f8ed04b8700 1 -- 172.21.15.43:0/1138186065 shutdown_connections 2024-04-21T08:37:41.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:41.042+0000 7f8ed04b8700 1 -- 172.21.15.43:0/1138186065 wait complete. 2024-04-21T08:37:41.412 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:41 smithi043 ceph-mon[27978]: pgmap v335: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-21T08:37:41.412 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:41 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1138186065' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-21T08:37:41.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:41 smithi112 ceph-mon[31085]: pgmap v335: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-21T08:37:41.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:41 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1138186065' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-21T08:37:41.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:41 smithi175 ceph-mon[31920]: pgmap v335: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-21T08:37:41.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:41 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1138186065' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-21T08:37:43.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:43 smithi043 ceph-mon[27978]: pgmap v336: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:37:43.410 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724599 got 167503724604 for osd.5 2024-04-21T08:37:43.410 DEBUG:teuthology.parallel:result is None 2024-04-21T08:37:43.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:43 smithi112 ceph-mon[31085]: pgmap v336: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:37:43.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:43 smithi175 ceph-mon[31920]: pgmap v336: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:37:45.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.051+0000 7f0c46b42700 1 -- 172.21.15.43:0/25404339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c40101940 msgr2=0x7f0c40101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:45.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.051+0000 7f0c46b42700 1 --2- 172.21.15.43:0/25404339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c40101940 0x7f0c40101dc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009990 tx=0x7f0c2c009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.051+0000 7f0c46b42700 1 -- 172.21.15.43:0/25404339 shutdown_connections 2024-04-21T08:37:45.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.051+0000 7f0c46b42700 1 --2- 172.21.15.43:0/25404339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c40102300 0x7f0c4010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.051+0000 7f0c46b42700 1 --2- 172.21.15.43:0/25404339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c40107990 0x7f0c40107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.051+0000 7f0c46b42700 1 --2- 172.21.15.43:0/25404339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c40101940 0x7f0c40101dc0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.051+0000 7f0c46b42700 1 -- 172.21.15.43:0/25404339 >> 172.21.15.43:0/25404339 conn(0x7f0c400788f0 msgr2=0x7f0c40078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:45.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.051+0000 7f0c46b42700 1 -- 172.21.15.43:0/25404339 shutdown_connections 2024-04-21T08:37:45.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.051+0000 7f0c46b42700 1 -- 172.21.15.43:0/25404339 wait complete. 2024-04-21T08:37:45.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.052+0000 7f0c46b42700 1 Processor -- start 2024-04-21T08:37:45.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.052+0000 7f0c46b42700 1 -- start start 2024-04-21T08:37:45.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.052+0000 7f0c46b42700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c40101940 0x7f0c401034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:45.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.052+0000 7f0c46b42700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c40102300 0x7f0c40103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:45.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.052+0000 7f0c46b42700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c40107990 0x7f0c40103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:45.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.052+0000 7f0c46b42700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0c40111ae0 con 0x7f0c40101940 2024-04-21T08:37:45.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.052+0000 7f0c46b42700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c40111960 con 0x7f0c40107990 2024-04-21T08:37:45.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.052+0000 7f0c46b42700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0c40111c60 con 0x7f0c40102300 2024-04-21T08:37:45.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.052+0000 7f0c448de700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c40101940 0x7f0c401034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:45.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.052+0000 7f0c448de700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c40101940 0x7f0c401034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48144/0 (socket says 172.21.15.43:48144) 2024-04-21T08:37:45.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.052+0000 7f0c448de700 1 -- 172.21.15.43:0/2092819448 learned_addr learned my addr 172.21.15.43:0/2092819448 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:37:45.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.052+0000 7f0c3ffff700 1 --2- 172.21.15.43:0/2092819448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c40102300 0x7f0c40103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:45.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.052+0000 7f0c450df700 1 --2- 172.21.15.43:0/2092819448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c40107990 0x7f0c40103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:45.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.053+0000 7f0c448de700 1 -- 172.21.15.43:0/2092819448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c40102300 msgr2=0x7f0c40103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:45.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.053+0000 7f0c448de700 1 --2- 172.21.15.43:0/2092819448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c40102300 0x7f0c40103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.053+0000 7f0c448de700 1 -- 172.21.15.43:0/2092819448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c40107990 msgr2=0x7f0c40103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:45.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.053+0000 7f0c448de700 1 --2- 172.21.15.43:0/2092819448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c40107990 0x7f0c40103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.053+0000 7f0c448de700 1 -- 172.21.15.43:0/2092819448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c401a76f0 con 0x7f0c40101940 2024-04-21T08:37:45.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.053+0000 7f0c448de700 1 --2- 172.21.15.43:0/2092819448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c40101940 0x7f0c401034f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400d950 tx=0x7f0c3400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:45.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.053+0000 7f0c3dffb700 1 -- 172.21.15.43:0/2092819448 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c34016070 con 0x7f0c40101940 2024-04-21T08:37:45.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.053+0000 7f0c46b42700 1 -- 172.21.15.43:0/2092819448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c401a79e0 con 0x7f0c40101940 2024-04-21T08:37:45.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.053+0000 7f0c3dffb700 1 -- 172.21.15.43:0/2092819448 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c3400fd40 con 0x7f0c40101940 2024-04-21T08:37:45.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.053+0000 7f0c3dffb700 1 -- 172.21.15.43:0/2092819448 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3400e3b0 con 0x7f0c40101940 2024-04-21T08:37:45.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.053+0000 7f0c46b42700 1 -- 172.21.15.43:0/2092819448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c401a7ef0 con 0x7f0c40101940 2024-04-21T08:37:45.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.054+0000 7f0c46b42700 1 -- 172.21.15.43:0/2092819448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c40069730 con 0x7f0c40101940 2024-04-21T08:37:45.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.055+0000 7f0c3dffb700 1 -- 172.21.15.43:0/2092819448 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0c34019070 con 0x7f0c40101940 2024-04-21T08:37:45.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.056+0000 7f0c3dffb700 1 --2- 172.21.15.43:0/2092819448 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0c300a3cd0 0x7f0c300a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:45.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.056+0000 7f0c3ffff700 1 --2- 172.21.15.43:0/2092819448 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0c300a3cd0 0x7f0c300a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:45.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.056+0000 7f0c3dffb700 1 -- 172.21.15.43:0/2092819448 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f0c340d0380 con 0x7f0c40101940 2024-04-21T08:37:45.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.056+0000 7f0c3ffff700 1 --2- 172.21.15.43:0/2092819448 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0c300a3cd0 0x7f0c300a6190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c0098e0 tx=0x7f0c2c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:45.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.061+0000 7f0c3dffb700 1 -- 172.21.15.43:0/2092819448 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c3409c840 con 0x7f0c40101940 2024-04-21T08:37:45.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.278+0000 7f0c46b42700 1 -- 172.21.15.43:0/2092819448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f0c4002df80 con 0x7f0c40101940 2024-04-21T08:37:45.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.279+0000 7f0c3dffb700 1 -- 172.21.15.43:0/2092819448 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f0c3409c140 con 0x7f0c40101940 2024-04-21T08:37:45.278 INFO:teuthology.orchestra.run.smithi043.stdout:347892351006 2024-04-21T08:37:45.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.281+0000 7f0c46b42700 1 -- 172.21.15.43:0/2092819448 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0c300a3cd0 msgr2=0x7f0c300a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:45.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.281+0000 7f0c46b42700 1 --2- 172.21.15.43:0/2092819448 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0c300a3cd0 0x7f0c300a6190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c0098e0 tx=0x7f0c2c005960 comp rx=0 tx=0).stop 2024-04-21T08:37:45.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.281+0000 7f0c46b42700 1 -- 172.21.15.43:0/2092819448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c40101940 msgr2=0x7f0c401034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:45.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.281+0000 7f0c46b42700 1 --2- 172.21.15.43:0/2092819448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c40101940 0x7f0c401034f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400d950 tx=0x7f0c3400dc60 comp rx=0 tx=0).stop 2024-04-21T08:37:45.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.281+0000 7f0c46b42700 1 -- 172.21.15.43:0/2092819448 shutdown_connections 2024-04-21T08:37:45.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.281+0000 7f0c46b42700 1 --2- 172.21.15.43:0/2092819448 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0c300a3cd0 0x7f0c300a6190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.281+0000 7f0c46b42700 1 --2- 172.21.15.43:0/2092819448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c40107990 0x7f0c40103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.281+0000 7f0c46b42700 1 --2- 172.21.15.43:0/2092819448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c40101940 0x7f0c401034f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.281+0000 7f0c46b42700 1 --2- 172.21.15.43:0/2092819448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c40102300 0x7f0c40103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.281+0000 7f0c46b42700 1 -- 172.21.15.43:0/2092819448 >> 172.21.15.43:0/2092819448 conn(0x7f0c400788f0 msgr2=0x7f0c4006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:45.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.281+0000 7f0c46b42700 1 -- 172.21.15.43:0/2092819448 shutdown_connections 2024-04-21T08:37:45.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.281+0000 7f0c46b42700 1 -- 172.21.15.43:0/2092819448 wait complete. 2024-04-21T08:37:45.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.296+0000 7f66e3b04700 1 -- 172.21.15.43:0/2660965878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66dc108ab0 msgr2=0x7f66dc108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:45.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.296+0000 7f66e3b04700 1 --2- 172.21.15.43:0/2660965878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66dc108ab0 0x7f66dc108e90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009990 tx=0x7f66d8009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.296+0000 7f66e3b04700 1 -- 172.21.15.43:0/2660965878 shutdown_connections 2024-04-21T08:37:45.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.296+0000 7f66e3b04700 1 --2- 172.21.15.43:0/2660965878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66dc103480 0x7f66dc10f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.296+0000 7f66e3b04700 1 --2- 172.21.15.43:0/2660965878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66dc108ab0 0x7f66dc108e90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.296+0000 7f66e3b04700 1 --2- 172.21.15.43:0/2660965878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66dc102ac0 0x7f66dc102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.296+0000 7f66e3b04700 1 -- 172.21.15.43:0/2660965878 >> 172.21.15.43:0/2660965878 conn(0x7f66dc079bf0 msgr2=0x7f66dc07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:45.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.296+0000 7f66e3b04700 1 -- 172.21.15.43:0/2660965878 shutdown_connections 2024-04-21T08:37:45.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.296+0000 7f66e3b04700 1 -- 172.21.15.43:0/2660965878 wait complete. 2024-04-21T08:37:45.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.297+0000 7f66e3b04700 1 Processor -- start 2024-04-21T08:37:45.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.297+0000 7f66e3b04700 1 -- start start 2024-04-21T08:37:45.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.297+0000 7f66e3b04700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66dc102ac0 0x7f66dc1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:45.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.297+0000 7f66e3b04700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66dc103480 0x7f66dc199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:45.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.297+0000 7f66e3b04700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66dc108ab0 0x7f66dc19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:45.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.297+0000 7f66e3b04700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc112ce0 con 0x7f66dc103480 2024-04-21T08:37:45.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.297+0000 7f66e3b04700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc112b60 con 0x7f66dc108ab0 2024-04-21T08:37:45.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.297+0000 7f66e3b04700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc112e60 con 0x7f66dc102ac0 2024-04-21T08:37:45.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.297+0000 7f66e109f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66dc103480 0x7f66dc199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:45.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.297+0000 7f66e109f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66dc103480 0x7f66dc199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48170/0 (socket says 172.21.15.43:48170) 2024-04-21T08:37:45.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.297+0000 7f66e109f700 1 -- 172.21.15.43:0/2281803481 learned_addr learned my addr 172.21.15.43:0/2281803481 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:37:45.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.298+0000 7f66e20a1700 1 --2- 172.21.15.43:0/2281803481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66dc108ab0 0x7f66dc19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:45.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.298+0000 7f66e18a0700 1 --2- 172.21.15.43:0/2281803481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66dc102ac0 0x7f66dc1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:45.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.298+0000 7f66e109f700 1 -- 172.21.15.43:0/2281803481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66dc102ac0 msgr2=0x7f66dc1991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:45.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.298+0000 7f66e109f700 1 --2- 172.21.15.43:0/2281803481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66dc102ac0 0x7f66dc1991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.298+0000 7f66e109f700 1 -- 172.21.15.43:0/2281803481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66dc108ab0 msgr2=0x7f66dc19dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:45.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.298+0000 7f66e109f700 1 --2- 172.21.15.43:0/2281803481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66dc108ab0 0x7f66dc19dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.298+0000 7f66e109f700 1 -- 172.21.15.43:0/2281803481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66dc19e200 con 0x7f66dc103480 2024-04-21T08:37:45.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.298+0000 7f66e18a0700 1 --2- 172.21.15.43:0/2281803481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66dc102ac0 0x7f66dc1991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:37:45.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.298+0000 7f66e109f700 1 --2- 172.21.15.43:0/2281803481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66dc103480 0x7f66dc199730 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00c9b0 tx=0x7f66cc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:45.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.298+0000 7f66d2ffd700 1 -- 172.21.15.43:0/2281803481 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc007b00 con 0x7f66dc103480 2024-04-21T08:37:45.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.298+0000 7f66e3b04700 1 -- 172.21.15.43:0/2281803481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66dc19e4f0 con 0x7f66dc103480 2024-04-21T08:37:45.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.298+0000 7f66d2ffd700 1 -- 172.21.15.43:0/2281803481 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66cc007ca0 con 0x7f66dc103480 2024-04-21T08:37:45.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.298+0000 7f66e3b04700 1 -- 172.21.15.43:0/2281803481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66dc074c70 con 0x7f66dc103480 2024-04-21T08:37:45.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.298+0000 7f66d2ffd700 1 -- 172.21.15.43:0/2281803481 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc018660 con 0x7f66dc103480 2024-04-21T08:37:45.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.300+0000 7f66e3b04700 1 -- 172.21.15.43:0/2281803481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66c0005330 con 0x7f66dc103480 2024-04-21T08:37:45.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.302+0000 7f66d2ffd700 1 -- 172.21.15.43:0/2281803481 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f66cc018800 con 0x7f66dc103480 2024-04-21T08:37:45.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.302+0000 7f66d2ffd700 1 --2- 172.21.15.43:0/2281803481 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f66c80a3cd0 0x7f66c80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:45.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.302+0000 7f66e18a0700 1 --2- 172.21.15.43:0/2281803481 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f66c80a3cd0 0x7f66c80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:45.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.302+0000 7f66d2ffd700 1 -- 172.21.15.43:0/2281803481 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f66cc0d0a40 con 0x7f66dc103480 2024-04-21T08:37:45.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.303+0000 7f66e18a0700 1 --2- 172.21.15.43:0/2281803481 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f66c80a3cd0 0x7f66c80a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009750 tx=0x7f66d8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:45.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.306+0000 7f66d2ffd700 1 -- 172.21.15.43:0/2281803481 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f66cc014030 con 0x7f66dc103480 2024-04-21T08:37:45.398 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:45 smithi043 ceph-mon[27978]: pgmap v337: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-21T08:37:45.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.526+0000 7f66e3b04700 1 -- 172.21.15.43:0/2281803481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f66c00058f0 con 0x7f66dc103480 2024-04-21T08:37:45.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.526+0000 7f66d2ffd700 1 -- 172.21.15.43:0/2281803481 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f66cc09cff0 con 0x7f66dc103480 2024-04-21T08:37:45.525 INFO:teuthology.orchestra.run.smithi043.stdout:55834574929 2024-04-21T08:37:45.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.528+0000 7f66e3b04700 1 -- 172.21.15.43:0/2281803481 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f66c80a3cd0 msgr2=0x7f66c80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:45.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.528+0000 7f66e3b04700 1 --2- 172.21.15.43:0/2281803481 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f66c80a3cd0 0x7f66c80a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009750 tx=0x7f66d8005fd0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.528+0000 7f66e3b04700 1 -- 172.21.15.43:0/2281803481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66dc103480 msgr2=0x7f66dc199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:45.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.528+0000 7f66e3b04700 1 --2- 172.21.15.43:0/2281803481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66dc103480 0x7f66dc199730 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00c9b0 tx=0x7f66cc00ccc0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.528+0000 7f66e3b04700 1 -- 172.21.15.43:0/2281803481 shutdown_connections 2024-04-21T08:37:45.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.528+0000 7f66e3b04700 1 --2- 172.21.15.43:0/2281803481 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f66c80a3cd0 0x7f66c80a6190 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.528+0000 7f66e3b04700 1 --2- 172.21.15.43:0/2281803481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66dc102ac0 0x7f66dc1991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.528+0000 7f66e3b04700 1 --2- 172.21.15.43:0/2281803481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66dc108ab0 0x7f66dc19dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.528+0000 7f66e3b04700 1 --2- 172.21.15.43:0/2281803481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66dc103480 0x7f66dc199730 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.528+0000 7f66e3b04700 1 -- 172.21.15.43:0/2281803481 >> 172.21.15.43:0/2281803481 conn(0x7f66dc079bf0 msgr2=0x7f66dc100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:45.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.528+0000 7f66e3b04700 1 -- 172.21.15.43:0/2281803481 shutdown_connections 2024-04-21T08:37:45.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:45.528+0000 7f66e3b04700 1 -- 172.21.15.43:0/2281803481 wait complete. 2024-04-21T08:37:45.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:45 smithi112 ceph-mon[31085]: pgmap v337: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-21T08:37:45.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:45 smithi175 ceph-mon[31920]: pgmap v337: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-21T08:37:46.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:46 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2092819448' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-21T08:37:46.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:46 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2281803481' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-21T08:37:46.394 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351001 got 347892351006 for osd.11 2024-04-21T08:37:46.394 DEBUG:teuthology.parallel:result is None 2024-04-21T08:37:46.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:46 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2092819448' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-21T08:37:46.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:46 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2281803481' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-21T08:37:46.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:46 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2092819448' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-21T08:37:46.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:46 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2281803481' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-21T08:37:47.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.301+0000 7f2cb3411700 1 -- 172.21.15.43:0/2489118472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cac103950 msgr2=0x7f2cac06b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:47.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.301+0000 7f2cb3411700 1 --2- 172.21.15.43:0/2489118472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cac103950 0x7f2cac06b110 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009990 tx=0x7f2ca8009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:47.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.301+0000 7f2cb3411700 1 -- 172.21.15.43:0/2489118472 shutdown_connections 2024-04-21T08:37:47.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.301+0000 7f2cb3411700 1 --2- 172.21.15.43:0/2489118472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cac10f390 0x7f2cac111780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:47.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.301+0000 7f2cb3411700 1 --2- 172.21.15.43:0/2489118472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cac103950 0x7f2cac06b110 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.301+0000 7f2cb3411700 1 --2- 172.21.15.43:0/2489118472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2cac06b650 0x7f2cac06bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.301+0000 7f2cb3411700 1 -- 172.21.15.43:0/2489118472 >> 172.21.15.43:0/2489118472 conn(0x7f2cac0788f0 msgr2=0x7f2cac078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.301+0000 7f2cb3411700 1 -- 172.21.15.43:0/2489118472 shutdown_connections 2024-04-21T08:37:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.301+0000 7f2cb3411700 1 -- 172.21.15.43:0/2489118472 wait complete. 2024-04-21T08:37:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.301+0000 7f2cb3411700 1 Processor -- start 2024-04-21T08:37:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.302+0000 7f2cb3411700 1 -- start start 2024-04-21T08:37:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.302+0000 7f2cb3411700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cac06b650 0x7f2cac19b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.302+0000 7f2cb3411700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cac103950 0x7f2cac19b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.302+0000 7f2cb3411700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2cac10f390 0x7f2cac19fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.302+0000 7f2cb3411700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac114e50 con 0x7f2cac103950 2024-04-21T08:37:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.302+0000 7f2cb3411700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac114cd0 con 0x7f2cac06b650 2024-04-21T08:37:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.302+0000 7f2cb3411700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac114fd0 con 0x7f2cac10f390 2024-04-21T08:37:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.302+0000 7f2cb09ac700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cac103950 0x7f2cac19b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.302+0000 7f2cb09ac700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cac103950 0x7f2cac19b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49170/0 (socket says 172.21.15.43:49170) 2024-04-21T08:37:47.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.302+0000 7f2cb09ac700 1 -- 172.21.15.43:0/3554818900 learned_addr learned my addr 172.21.15.43:0/3554818900 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:37:47.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.302+0000 7f2cb19ae700 1 --2- 172.21.15.43:0/3554818900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2cac10f390 0x7f2cac19fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:47.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.302+0000 7f2cb11ad700 1 --2- 172.21.15.43:0/3554818900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cac06b650 0x7f2cac19b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:47.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.302+0000 7f2cb09ac700 1 -- 172.21.15.43:0/3554818900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2cac10f390 msgr2=0x7f2cac19fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:47.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.302+0000 7f2cb09ac700 1 --2- 172.21.15.43:0/3554818900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2cac10f390 0x7f2cac19fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:47.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.302+0000 7f2cb09ac700 1 -- 172.21.15.43:0/3554818900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cac06b650 msgr2=0x7f2cac19b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:47.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.302+0000 7f2cb09ac700 1 --2- 172.21.15.43:0/3554818900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cac06b650 0x7f2cac19b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:47.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.302+0000 7f2cb09ac700 1 -- 172.21.15.43:0/3554818900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cac1a02d0 con 0x7f2cac103950 2024-04-21T08:37:47.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.302+0000 7f2cb11ad700 1 --2- 172.21.15.43:0/3554818900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cac06b650 0x7f2cac19b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:37:47.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.303+0000 7f2cb09ac700 1 --2- 172.21.15.43:0/3554818900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cac103950 0x7f2cac19b800 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c00c920 tx=0x7f2c9c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:47.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.303+0000 7f2ca27fc700 1 -- 172.21.15.43:0/3554818900 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c007b00 con 0x7f2cac103950 2024-04-21T08:37:47.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.303+0000 7f2ca27fc700 1 -- 172.21.15.43:0/3554818900 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c9c004ce0 con 0x7f2cac103950 2024-04-21T08:37:47.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.303+0000 7f2ca27fc700 1 -- 172.21.15.43:0/3554818900 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c005550 con 0x7f2cac103950 2024-04-21T08:37:47.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.303+0000 7f2cb3411700 1 -- 172.21.15.43:0/3554818900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cac1a05c0 con 0x7f2cac103950 2024-04-21T08:37:47.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.303+0000 7f2cb3411700 1 -- 172.21.15.43:0/3554818900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cac1a7e40 con 0x7f2cac103950 2024-04-21T08:37:47.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.304+0000 7f2cb3411700 1 -- 172.21.15.43:0/3554818900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cac101f80 con 0x7f2cac103950 2024-04-21T08:37:47.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.306+0000 7f2ca27fc700 1 -- 172.21.15.43:0/3554818900 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2c9c007ca0 con 0x7f2cac103950 2024-04-21T08:37:47.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.307+0000 7f2ca27fc700 1 --2- 172.21.15.43:0/3554818900 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2c980a3cd0 0x7f2c980a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:47.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.307+0000 7f2ca27fc700 1 -- 172.21.15.43:0/3554818900 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f2c9c0d09c0 con 0x7f2cac103950 2024-04-21T08:37:47.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.307+0000 7f2cb11ad700 1 --2- 172.21.15.43:0/3554818900 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2c980a3cd0 0x7f2c980a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:47.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.308+0000 7f2cb11ad700 1 --2- 172.21.15.43:0/3554818900 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2c980a3cd0 0x7f2c980a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009750 tx=0x7f2ca8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:47.307 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:47.308 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:47.309 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:47.312 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:47.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.312+0000 7f2ca27fc700 1 -- 172.21.15.43:0/3554818900 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c9c014030 con 0x7f2cac103950 2024-04-21T08:37:47.447 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:47 smithi043 ceph-mon[27978]: pgmap v338: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:37:47.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.535+0000 7f2cb3411700 1 -- 172.21.15.43:0/3554818900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f2cac1a80e0 con 0x7f2cac103950 2024-04-21T08:37:47.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.535+0000 7f2ca27fc700 1 -- 172.21.15.43:0/3554818900 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2c9c014030 con 0x7f2cac103950 2024-04-21T08:37:47.535 INFO:teuthology.orchestra.run.smithi043.stdout:141733920834 2024-04-21T08:37:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.537+0000 7f2cb3411700 1 -- 172.21.15.43:0/3554818900 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2c980a3cd0 msgr2=0x7f2c980a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.537+0000 7f2cb3411700 1 --2- 172.21.15.43:0/3554818900 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2c980a3cd0 0x7f2c980a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009750 tx=0x7f2ca8005fd0 comp rx=0 tx=0).stop 2024-04-21T08:37:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.537+0000 7f2cb3411700 1 -- 172.21.15.43:0/3554818900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cac103950 msgr2=0x7f2cac19b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.537+0000 7f2cb3411700 1 --2- 172.21.15.43:0/3554818900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cac103950 0x7f2cac19b800 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c00c920 tx=0x7f2c9c00cc30 comp rx=0 tx=0).stop 2024-04-21T08:37:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.538+0000 7f2cb3411700 1 -- 172.21.15.43:0/3554818900 shutdown_connections 2024-04-21T08:37:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.538+0000 7f2cb3411700 1 --2- 172.21.15.43:0/3554818900 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2c980a3cd0 0x7f2c980a6190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.538+0000 7f2cb3411700 1 --2- 172.21.15.43:0/3554818900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2cac10f390 0x7f2cac19fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.538+0000 7f2cb3411700 1 --2- 172.21.15.43:0/3554818900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2cac06b650 0x7f2cac19b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.538+0000 7f2cb3411700 1 --2- 172.21.15.43:0/3554818900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cac103950 0x7f2cac19b800 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.538+0000 7f2cb3411700 1 -- 172.21.15.43:0/3554818900 >> 172.21.15.43:0/3554818900 conn(0x7f2cac0788f0 msgr2=0x7f2cac100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.538+0000 7f2cb3411700 1 -- 172.21.15.43:0/3554818900 shutdown_connections 2024-04-21T08:37:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:47.538+0000 7f2cb3411700 1 -- 172.21.15.43:0/3554818900 wait complete. 2024-04-21T08:37:47.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:47 smithi112 ceph-mon[31085]: pgmap v338: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:37:47.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:47 smithi175 ceph-mon[31920]: pgmap v338: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:37:47.804 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:47.805 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:47.840 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574923 got 55834574929 for osd.1 2024-04-21T08:37:47.840 DEBUG:teuthology.parallel:result is None 2024-04-21T08:37:48.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:48 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3554818900' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-21T08:37:48.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:48 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3554818900' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-21T08:37:48.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:48 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3554818900' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-21T08:37:48.825 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920830 got 141733920834 for osd.4 2024-04-21T08:37:48.825 DEBUG:teuthology.parallel:result is None 2024-04-21T08:37:49.116 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:49.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:49 smithi112 ceph-mon[31085]: pgmap v339: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-21T08:37:49.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:49 smithi043 ceph-mon[27978]: pgmap v339: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-21T08:37:49.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:49 smithi175 ceph-mon[31920]: pgmap v339: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-21T08:37:51.499 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:51 smithi043 ceph-mon[27978]: pgmap v340: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:37:51.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:51 smithi112 ceph-mon[31085]: pgmap v340: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:37:51.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:51 smithi175 ceph-mon[31920]: pgmap v340: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:37:52.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.482+0000 7fa9dac58700 1 -- 172.21.15.43:0/1144416218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9d4101dc0 msgr2=0x7fa9d410f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:52.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.482+0000 7fa9dac58700 1 --2- 172.21.15.43:0/1144416218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9d4101dc0 0x7fa9d410f830 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00b0a0 tx=0x7fa9cc00b3b0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.482+0000 7fa9dac58700 1 -- 172.21.15.43:0/1144416218 shutdown_connections 2024-04-21T08:37:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.482+0000 7fa9dac58700 1 --2- 172.21.15.43:0/1144416218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9d4101dc0 0x7fa9d410f830 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.482+0000 7fa9dac58700 1 --2- 172.21.15.43:0/1144416218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d4107b20 0x7fa9d4107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.482+0000 7fa9dac58700 1 --2- 172.21.15.43:0/1144416218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9d4101400 0x7fa9d4101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.482+0000 7fa9dac58700 1 -- 172.21.15.43:0/1144416218 >> 172.21.15.43:0/1144416218 conn(0x7fa9d4076c60 msgr2=0x7fa9d4077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.482+0000 7fa9dac58700 1 -- 172.21.15.43:0/1144416218 shutdown_connections 2024-04-21T08:37:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.482+0000 7fa9dac58700 1 -- 172.21.15.43:0/1144416218 wait complete. 2024-04-21T08:37:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.483+0000 7fa9dac58700 1 Processor -- start 2024-04-21T08:37:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.483+0000 7fa9dac58700 1 -- start start 2024-04-21T08:37:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.483+0000 7fa9dac58700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d4101400 0x7fa9d4199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.483+0000 7fa9dac58700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9d4101dc0 0x7fa9d41995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.483+0000 7fa9dac58700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9d4107b20 0x7fa9d419d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.483+0000 7fa9dac58700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4112cb0 con 0x7fa9d4107b20 2024-04-21T08:37:52.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.483+0000 7fa9dac58700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4112b30 con 0x7fa9d4101400 2024-04-21T08:37:52.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.483+0000 7fa9dac58700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4112e30 con 0x7fa9d4101dc0 2024-04-21T08:37:52.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.483+0000 7fa9d91f5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9d4107b20 0x7fa9d419d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:52.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.483+0000 7fa9d89f4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d4101400 0x7fa9d4199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:52.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.483+0000 7fa9d91f5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9d4107b20 0x7fa9d419d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49188/0 (socket says 172.21.15.43:49188) 2024-04-21T08:37:52.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.483+0000 7fa9d91f5700 1 -- 172.21.15.43:0/2897137497 learned_addr learned my addr 172.21.15.43:0/2897137497 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:37:52.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.483+0000 7fa9d3fff700 1 --2- 172.21.15.43:0/2897137497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9d4101dc0 0x7fa9d41995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:52.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.484+0000 7fa9d91f5700 1 -- 172.21.15.43:0/2897137497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9d4101dc0 msgr2=0x7fa9d41995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:52.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.484+0000 7fa9d91f5700 1 --2- 172.21.15.43:0/2897137497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9d4101dc0 0x7fa9d41995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.484+0000 7fa9d91f5700 1 -- 172.21.15.43:0/2897137497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d4101400 msgr2=0x7fa9d4199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:52.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.484+0000 7fa9d91f5700 1 --2- 172.21.15.43:0/2897137497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d4101400 0x7fa9d4199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.484+0000 7fa9d91f5700 1 -- 172.21.15.43:0/2897137497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d419e090 con 0x7fa9d4107b20 2024-04-21T08:37:52.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.484+0000 7fa9d91f5700 1 --2- 172.21.15.43:0/2897137497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9d4107b20 0x7fa9d419d950 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00b830 tx=0x7fa9cc004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:52.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.484+0000 7fa9d1ffb700 1 -- 172.21.15.43:0/2897137497 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc021070 con 0x7fa9d4107b20 2024-04-21T08:37:52.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.484+0000 7fa9d1ffb700 1 -- 172.21.15.43:0/2897137497 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9cc007e30 con 0x7fa9d4107b20 2024-04-21T08:37:52.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.484+0000 7fa9d1ffb700 1 -- 172.21.15.43:0/2897137497 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc0047d0 con 0x7fa9d4107b20 2024-04-21T08:37:52.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.484+0000 7fa9dac58700 1 -- 172.21.15.43:0/2897137497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d419e320 con 0x7fa9d4107b20 2024-04-21T08:37:52.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.484+0000 7fa9dac58700 1 -- 172.21.15.43:0/2897137497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d41a5c00 con 0x7fa9d4107b20 2024-04-21T08:37:52.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.486+0000 7fa9dac58700 1 -- 172.21.15.43:0/2897137497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d4069730 con 0x7fa9d4107b20 2024-04-21T08:37:52.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.487+0000 7fa9d1ffb700 1 -- 172.21.15.43:0/2897137497 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa9cc007980 con 0x7fa9d4107b20 2024-04-21T08:37:52.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.487+0000 7fa9d1ffb700 1 --2- 172.21.15.43:0/2897137497 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa9c40a3d50 0x7fa9c40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:52.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.487+0000 7fa9d1ffb700 1 -- 172.21.15.43:0/2897137497 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fa9cc00f020 con 0x7fa9d4107b20 2024-04-21T08:37:52.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.487+0000 7fa9d89f4700 1 --2- 172.21.15.43:0/2897137497 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa9c40a3d50 0x7fa9c40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:52.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.488+0000 7fa9d89f4700 1 --2- 172.21.15.43:0/2897137497 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa9c40a3d50 0x7fa9c40a6210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40781e0 tx=0x7fa9c800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:52.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.492+0000 7fa9d1ffb700 1 -- 172.21.15.43:0/2897137497 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9cc02a020 con 0x7fa9d4107b20 2024-04-21T08:37:52.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.703+0000 7fa9dac58700 1 -- 172.21.15.43:0/2897137497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fa9d41a5dd0 con 0x7fa9d4107b20 2024-04-21T08:37:52.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.704+0000 7fa9d1ffb700 1 -- 172.21.15.43:0/2897137497 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa9cc09e3b0 con 0x7fa9d4107b20 2024-04-21T08:37:52.703 INFO:teuthology.orchestra.run.smithi043.stdout:223338299443 2024-04-21T08:37:52.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.706+0000 7fa9dac58700 1 -- 172.21.15.43:0/2897137497 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa9c40a3d50 msgr2=0x7fa9c40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:52.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.706+0000 7fa9dac58700 1 --2- 172.21.15.43:0/2897137497 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa9c40a3d50 0x7fa9c40a6210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40781e0 tx=0x7fa9c800b410 comp rx=0 tx=0).stop 2024-04-21T08:37:52.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.706+0000 7fa9dac58700 1 -- 172.21.15.43:0/2897137497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9d4107b20 msgr2=0x7fa9d419d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:52.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.706+0000 7fa9dac58700 1 --2- 172.21.15.43:0/2897137497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9d4107b20 0x7fa9d419d950 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00b830 tx=0x7fa9cc004ce0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.706+0000 7fa9dac58700 1 -- 172.21.15.43:0/2897137497 shutdown_connections 2024-04-21T08:37:52.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.706+0000 7fa9dac58700 1 --2- 172.21.15.43:0/2897137497 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa9c40a3d50 0x7fa9c40a6210 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.706+0000 7fa9dac58700 1 --2- 172.21.15.43:0/2897137497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9d4107b20 0x7fa9d419d950 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.706+0000 7fa9dac58700 1 --2- 172.21.15.43:0/2897137497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d4101400 0x7fa9d4199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.706+0000 7fa9dac58700 1 --2- 172.21.15.43:0/2897137497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9d4101dc0 0x7fa9d41995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.706+0000 7fa9dac58700 1 -- 172.21.15.43:0/2897137497 >> 172.21.15.43:0/2897137497 conn(0x7fa9d4076c60 msgr2=0x7fa9d40ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:52.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.706+0000 7fa9dac58700 1 -- 172.21.15.43:0/2897137497 shutdown_connections 2024-04-21T08:37:52.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:52.706+0000 7fa9dac58700 1 -- 172.21.15.43:0/2897137497 wait complete. 2024-04-21T08:37:53.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:53 smithi043 ceph-mon[27978]: pgmap v341: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-21T08:37:53.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:53 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2897137497' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-21T08:37:53.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:53 smithi175 ceph-mon[31920]: pgmap v341: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-21T08:37:53.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:53 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2897137497' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-21T08:37:53.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:53 smithi112 ceph-mon[31085]: pgmap v341: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-21T08:37:53.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:53 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2897137497' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-21T08:37:55.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:55 smithi043 ceph-mon[27978]: pgmap v342: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-21T08:37:55.596 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:55 smithi112 ceph-mon[31085]: pgmap v342: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-21T08:37:55.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:55 smithi175 ceph-mon[31920]: pgmap v342: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-21T08:37:56.697 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299438 got 223338299443 for osd.7 2024-04-21T08:37:56.697 DEBUG:teuthology.parallel:result is None 2024-04-21T08:37:57.550 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:57 smithi043 ceph-mon[27978]: pgmap v343: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:37:57.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:57 smithi175 ceph-mon[31920]: pgmap v343: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:37:57.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:57 smithi112 ceph-mon[31085]: pgmap v343: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:37:58.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.099+0000 7fe73508d700 1 -- 172.21.15.43:0/1891108149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe73006b0f0 msgr2=0x7fe73010f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.099+0000 7fe73508d700 1 --2- 172.21.15.43:0/1891108149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe73006b0f0 0x7fe73010f900 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe728009990 tx=0x7fe728009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.099+0000 7fe73508d700 1 -- 172.21.15.43:0/1891108149 shutdown_connections 2024-04-21T08:37:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.099+0000 7fe73508d700 1 --2- 172.21.15.43:0/1891108149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe730108a30 0x7fe730108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.099+0000 7fe73508d700 1 --2- 172.21.15.43:0/1891108149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe73006b0f0 0x7fe73010f900 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.099+0000 7fe73508d700 1 --2- 172.21.15.43:0/1891108149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe73006a750 0x7fe73006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.099+0000 7fe73508d700 1 -- 172.21.15.43:0/1891108149 >> 172.21.15.43:0/1891108149 conn(0x7fe7300788f0 msgr2=0x7fe730078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.099+0000 7fe73508d700 1 -- 172.21.15.43:0/1891108149 shutdown_connections 2024-04-21T08:37:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.099+0000 7fe73508d700 1 -- 172.21.15.43:0/1891108149 wait complete. 2024-04-21T08:37:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.100+0000 7fe73508d700 1 Processor -- start 2024-04-21T08:37:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.100+0000 7fe73508d700 1 -- start start 2024-04-21T08:37:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.100+0000 7fe73508d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe73006a750 0x7fe730199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.100+0000 7fe73508d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe73006b0f0 0x7fe7301996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.100+0000 7fe73508d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe730108a30 0x7fe73019da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.100+0000 7fe73508d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe730112c20 con 0x7fe730108a30 2024-04-21T08:37:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.100+0000 7fe73508d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe730112aa0 con 0x7fe73006b0f0 2024-04-21T08:37:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.100+0000 7fe73508d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe730112da0 con 0x7fe73006a750 2024-04-21T08:37:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.100+0000 7fe72f59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe730108a30 0x7fe73019da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.101+0000 7fe72f59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe730108a30 0x7fe73019da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33504/0 (socket says 172.21.15.43:33504) 2024-04-21T08:37:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.101+0000 7fe72f59e700 1 -- 172.21.15.43:0/4217619099 learned_addr learned my addr 172.21.15.43:0/4217619099 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:37:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.101+0000 7fe72ed9d700 1 --2- 172.21.15.43:0/4217619099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe73006a750 0x7fe730199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.101+0000 7fe72e59c700 1 --2- 172.21.15.43:0/4217619099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe73006b0f0 0x7fe7301996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.101+0000 7fe72f59e700 1 -- 172.21.15.43:0/4217619099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe73006a750 msgr2=0x7fe730199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.101+0000 7fe72f59e700 1 --2- 172.21.15.43:0/4217619099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe73006a750 0x7fe730199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.101+0000 7fe72f59e700 1 -- 172.21.15.43:0/4217619099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe73006b0f0 msgr2=0x7fe7301996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.101+0000 7fe72f59e700 1 --2- 172.21.15.43:0/4217619099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe73006b0f0 0x7fe7301996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.101+0000 7fe72f59e700 1 -- 172.21.15.43:0/4217619099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73019e150 con 0x7fe730108a30 2024-04-21T08:37:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.101+0000 7fe72f59e700 1 --2- 172.21.15.43:0/4217619099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe730108a30 0x7fe73019da10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe7280047b0 tx=0x7fe728005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.101+0000 7fe717fff700 1 -- 172.21.15.43:0/4217619099 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe728004830 con 0x7fe730108a30 2024-04-21T08:37:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.101+0000 7fe73508d700 1 -- 172.21.15.43:0/4217619099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73019e3e0 con 0x7fe730108a30 2024-04-21T08:37:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.101+0000 7fe717fff700 1 -- 172.21.15.43:0/4217619099 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe728005020 con 0x7fe730108a30 2024-04-21T08:37:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.101+0000 7fe717fff700 1 -- 172.21.15.43:0/4217619099 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe728003cd0 con 0x7fe730108a30 2024-04-21T08:37:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.101+0000 7fe73508d700 1 -- 172.21.15.43:0/4217619099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe73019e780 con 0x7fe730108a30 2024-04-21T08:37:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.102+0000 7fe73508d700 1 -- 172.21.15.43:0/4217619099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe730069730 con 0x7fe730108a30 2024-04-21T08:37:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.104+0000 7fe717fff700 1 -- 172.21.15.43:0/4217619099 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe728005490 con 0x7fe730108a30 2024-04-21T08:37:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.104+0000 7fe717fff700 1 --2- 172.21.15.43:0/4217619099 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe7180a3cd0 0x7fe7180a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.104+0000 7fe72ed9d700 1 --2- 172.21.15.43:0/4217619099 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe7180a3cd0 0x7fe7180a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.104+0000 7fe717fff700 1 -- 172.21.15.43:0/4217619099 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fe7280d1780 con 0x7fe730108a30 2024-04-21T08:37:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.105+0000 7fe72ed9d700 1 --2- 172.21.15.43:0/4217619099 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe7180a3cd0 0x7fe7180a6190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe720009550 tx=0x7fe7200091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:58.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.108+0000 7fe717fff700 1 -- 172.21.15.43:0/4217619099 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe728013070 con 0x7fe730108a30 2024-04-21T08:37:58.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.318+0000 7fe73508d700 1 -- 172.21.15.43:0/4217619099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fe73002e140 con 0x7fe730108a30 2024-04-21T08:37:58.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.318+0000 7fe717fff700 1 -- 172.21.15.43:0/4217619099 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe72809dd30 con 0x7fe730108a30 2024-04-21T08:37:58.318 INFO:teuthology.orchestra.run.smithi043.stdout:77309411407 2024-04-21T08:37:58.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.320+0000 7fe73508d700 1 -- 172.21.15.43:0/4217619099 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe7180a3cd0 msgr2=0x7fe7180a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:58.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.320+0000 7fe73508d700 1 --2- 172.21.15.43:0/4217619099 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe7180a3cd0 0x7fe7180a6190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe720009550 tx=0x7fe7200091b0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.320+0000 7fe73508d700 1 -- 172.21.15.43:0/4217619099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe730108a30 msgr2=0x7fe73019da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:58.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.320+0000 7fe73508d700 1 --2- 172.21.15.43:0/4217619099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe730108a30 0x7fe73019da10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe7280047b0 tx=0x7fe728005ef0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.320+0000 7fe73508d700 1 -- 172.21.15.43:0/4217619099 shutdown_connections 2024-04-21T08:37:58.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.320+0000 7fe73508d700 1 --2- 172.21.15.43:0/4217619099 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fe7180a3cd0 0x7fe7180a6190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.320+0000 7fe73508d700 1 --2- 172.21.15.43:0/4217619099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe73006a750 0x7fe730199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.321+0000 7fe73508d700 1 --2- 172.21.15.43:0/4217619099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe730108a30 0x7fe73019da10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.321+0000 7fe73508d700 1 --2- 172.21.15.43:0/4217619099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe73006b0f0 0x7fe7301996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.321+0000 7fe73508d700 1 -- 172.21.15.43:0/4217619099 >> 172.21.15.43:0/4217619099 conn(0x7fe7300788f0 msgr2=0x7fe7300ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:58.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.321+0000 7fe73508d700 1 -- 172.21.15.43:0/4217619099 shutdown_connections 2024-04-21T08:37:58.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.321+0000 7fe73508d700 1 -- 172.21.15.43:0/4217619099 wait complete. 2024-04-21T08:37:58.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.772+0000 7f013a044700 1 -- 172.21.15.43:0/4038710163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0134103450 msgr2=0x7f013410f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:58.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.772+0000 7f013a044700 1 --2- 172.21.15.43:0/4038710163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0134103450 0x7f013410f8b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f012c00b0a0 tx=0x7f012c00b3b0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.772+0000 7f013a044700 1 -- 172.21.15.43:0/4038710163 shutdown_connections 2024-04-21T08:37:58.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.772+0000 7f013a044700 1 --2- 172.21.15.43:0/4038710163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0134108a50 0x7f0134108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.772+0000 7f013a044700 1 --2- 172.21.15.43:0/4038710163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0134103450 0x7f013410f8b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.772+0000 7f013a044700 1 --2- 172.21.15.43:0/4038710163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0134102a90 0x7f0134102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.772+0000 7f013a044700 1 -- 172.21.15.43:0/4038710163 >> 172.21.15.43:0/4038710163 conn(0x7f0134079ba0 msgr2=0x7f0134079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:58.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.772+0000 7f013a044700 1 -- 172.21.15.43:0/4038710163 shutdown_connections 2024-04-21T08:37:58.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.772+0000 7f013a044700 1 -- 172.21.15.43:0/4038710163 wait complete. 2024-04-21T08:37:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.772+0000 7f013a044700 1 Processor -- start 2024-04-21T08:37:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.773+0000 7f013a044700 1 -- start start 2024-04-21T08:37:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.773+0000 7f013a044700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0134102a90 0x7f01341991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.773+0000 7f013a044700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0134103450 0x7f0134199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.773+0000 7f013a044700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0134108a50 0x7f013419daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.773+0000 7f013a044700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0134112c60 con 0x7f0134108a50 2024-04-21T08:37:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.773+0000 7f013a044700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0134112ae0 con 0x7f0134103450 2024-04-21T08:37:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.773+0000 7f013a044700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0134112de0 con 0x7f0134102a90 2024-04-21T08:37:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.773+0000 7f0133fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0134108a50 0x7f013419daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:58.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.773+0000 7f0132ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0134103450 0x7f0134199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:58.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.773+0000 7f0133fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0134108a50 0x7f013419daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33522/0 (socket says 172.21.15.43:33522) 2024-04-21T08:37:58.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.773+0000 7f0133fff700 1 -- 172.21.15.43:0/1453272866 learned_addr learned my addr 172.21.15.43:0/1453272866 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:37:58.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.773+0000 7f01337fe700 1 --2- 172.21.15.43:0/1453272866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0134102a90 0x7f01341991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:58.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.773+0000 7f0133fff700 1 -- 172.21.15.43:0/1453272866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0134102a90 msgr2=0x7f01341991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:58.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.773+0000 7f0133fff700 1 --2- 172.21.15.43:0/1453272866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0134102a90 0x7f01341991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.773+0000 7f0133fff700 1 -- 172.21.15.43:0/1453272866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0134103450 msgr2=0x7f0134199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:58.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.773+0000 7f0133fff700 1 --2- 172.21.15.43:0/1453272866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0134103450 0x7f0134199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.773+0000 7f0133fff700 1 -- 172.21.15.43:0/1453272866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013419e1e0 con 0x7f0134108a50 2024-04-21T08:37:58.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.773+0000 7f0132ffd700 1 --2- 172.21.15.43:0/1453272866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0134103450 0x7f0134199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:37:58.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.774+0000 7f0133fff700 1 --2- 172.21.15.43:0/1453272866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0134108a50 0x7f013419daa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f012c0062a0 tx=0x7f012c004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:58.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.774+0000 7f0130ff9700 1 -- 172.21.15.43:0/1453272866 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c021070 con 0x7f0134108a50 2024-04-21T08:37:58.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.774+0000 7f0130ff9700 1 -- 172.21.15.43:0/1453272866 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f012c007c80 con 0x7f0134108a50 2024-04-21T08:37:58.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.774+0000 7f013a044700 1 -- 172.21.15.43:0/1453272866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013419e470 con 0x7f0134108a50 2024-04-21T08:37:58.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.774+0000 7f0130ff9700 1 -- 172.21.15.43:0/1453272866 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c01d450 con 0x7f0134108a50 2024-04-21T08:37:58.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.774+0000 7f013a044700 1 -- 172.21.15.43:0/1453272866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01341a5d50 con 0x7f0134108a50 2024-04-21T08:37:58.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.775+0000 7f013a044700 1 -- 172.21.15.43:0/1453272866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0134069730 con 0x7f0134108a50 2024-04-21T08:37:58.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.776+0000 7f0130ff9700 1 -- 172.21.15.43:0/1453272866 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f012c004000 con 0x7f0134108a50 2024-04-21T08:37:58.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.777+0000 7f0130ff9700 1 --2- 172.21.15.43:0/1453272866 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f011c0a3cd0 0x7f011c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:58.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.777+0000 7f01337fe700 1 --2- 172.21.15.43:0/1453272866 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f011c0a3cd0 0x7f011c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:58.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.777+0000 7f0130ff9700 1 -- 172.21.15.43:0/1453272866 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f012c00f020 con 0x7f0134108a50 2024-04-21T08:37:58.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.777+0000 7f01337fe700 1 --2- 172.21.15.43:0/1453272866 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f011c0a3cd0 0x7f011c0a6190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0124006fd0 tx=0x7f0124008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:58.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.780+0000 7f0130ff9700 1 -- 172.21.15.43:0/1453272866 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f012c02a020 con 0x7f0134108a50 2024-04-21T08:37:58.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.988+0000 7f013a044700 1 -- 172.21.15.43:0/1453272866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f013402df80 con 0x7f0134108a50 2024-04-21T08:37:58.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.989+0000 7f0130ff9700 1 -- 172.21.15.43:0/1453272866 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f012c09e960 con 0x7f0134108a50 2024-04-21T08:37:58.988 INFO:teuthology.orchestra.run.smithi043.stdout:34359738457 2024-04-21T08:37:58.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.991+0000 7f013a044700 1 -- 172.21.15.43:0/1453272866 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f011c0a3cd0 msgr2=0x7f011c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:58.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.991+0000 7f013a044700 1 --2- 172.21.15.43:0/1453272866 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f011c0a3cd0 0x7f011c0a6190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0124006fd0 tx=0x7f0124008040 comp rx=0 tx=0).stop 2024-04-21T08:37:58.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.991+0000 7f013a044700 1 -- 172.21.15.43:0/1453272866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0134108a50 msgr2=0x7f013419daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:58.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.991+0000 7f013a044700 1 --2- 172.21.15.43:0/1453272866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0134108a50 0x7f013419daa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f012c0062a0 tx=0x7f012c004e20 comp rx=0 tx=0).stop 2024-04-21T08:37:58.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.991+0000 7f013a044700 1 -- 172.21.15.43:0/1453272866 shutdown_connections 2024-04-21T08:37:58.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.991+0000 7f013a044700 1 --2- 172.21.15.43:0/1453272866 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f011c0a3cd0 0x7f011c0a6190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.991+0000 7f013a044700 1 --2- 172.21.15.43:0/1453272866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0134102a90 0x7f01341991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.991+0000 7f013a044700 1 --2- 172.21.15.43:0/1453272866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0134108a50 0x7f013419daa0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.991+0000 7f013a044700 1 --2- 172.21.15.43:0/1453272866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0134103450 0x7f0134199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.991+0000 7f013a044700 1 -- 172.21.15.43:0/1453272866 >> 172.21.15.43:0/1453272866 conn(0x7f0134079ba0 msgr2=0x7f0134101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:58.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.992+0000 7f013a044700 1 -- 172.21.15.43:0/1453272866 shutdown_connections 2024-04-21T08:37:58.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:37:58.992+0000 7f013a044700 1 -- 172.21.15.43:0/1453272866 wait complete. 2024-04-21T08:37:59.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:59 smithi043 ceph-mon[27978]: pgmap v344: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-21T08:37:59.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:59 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/4217619099' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-21T08:37:59.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:37:59 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1453272866' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-21T08:37:59.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:59 smithi175 ceph-mon[31920]: pgmap v344: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-21T08:37:59.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:59 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/4217619099' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-21T08:37:59.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:37:59 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1453272866' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-21T08:37:59.771 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411400 got 77309411407 for osd.2 2024-04-21T08:37:59.771 DEBUG:teuthology.parallel:result is None 2024-04-21T08:37:59.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:59 smithi112 ceph-mon[31085]: pgmap v344: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-21T08:37:59.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:59 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/4217619099' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-21T08:37:59.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:37:59 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1453272866' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-21T08:38:01.325 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738452 got 34359738457 for osd.0 2024-04-21T08:38:01.325 DEBUG:teuthology.parallel:result is None 2024-04-21T08:38:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.644+0000 7f134d480700 1 -- 172.21.15.43:0/664131343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1348075590 msgr2=0x7f134810ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.644+0000 7f134d480700 1 --2- 172.21.15.43:0/664131343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1348075590 0x7f134810ef00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f133c009990 tx=0x7f133c009ca0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.644+0000 7f134d480700 1 -- 172.21.15.43:0/664131343 shutdown_connections 2024-04-21T08:38:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.644+0000 7f134d480700 1 --2- 172.21.15.43:0/664131343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f134810f440 0x7f1348111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.644+0000 7f134d480700 1 --2- 172.21.15.43:0/664131343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1348074c70 0x7f1348075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.644+0000 7f134d480700 1 --2- 172.21.15.43:0/664131343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1348075590 0x7f134810ef00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.644+0000 7f134d480700 1 -- 172.21.15.43:0/664131343 >> 172.21.15.43:0/664131343 conn(0x7f1348079bf0 msgr2=0x7f134807a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.644+0000 7f134d480700 1 -- 172.21.15.43:0/664131343 shutdown_connections 2024-04-21T08:38:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.644+0000 7f134d480700 1 -- 172.21.15.43:0/664131343 wait complete. 2024-04-21T08:38:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.645+0000 7f134d480700 1 Processor -- start 2024-04-21T08:38:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.645+0000 7f134d480700 1 -- start start 2024-04-21T08:38:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.645+0000 7f134d480700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1348074c70 0x7f134819b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.645+0000 7f134d480700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1348075590 0x7f134819b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.645+0000 7f134d480700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134810f440 0x7f134819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.645+0000 7f134d480700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1348114e80 con 0x7f134810f440 2024-04-21T08:38:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.645+0000 7f134d480700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1348114d00 con 0x7f1348075590 2024-04-21T08:38:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.645+0000 7f134d480700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1348115000 con 0x7f1348074c70 2024-04-21T08:38:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.645+0000 7f13477fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134810f440 0x7f134819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.645+0000 7f13477fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134810f440 0x7f134819fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33534/0 (socket says 172.21.15.43:33534) 2024-04-21T08:38:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.645+0000 7f13477fe700 1 -- 172.21.15.43:0/3498913406 learned_addr learned my addr 172.21.15.43:0/3498913406 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.645+0000 7f1346ffd700 1 --2- 172.21.15.43:0/3498913406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1348074c70 0x7f134819b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.646+0000 7f13467fc700 1 --2- 172.21.15.43:0/3498913406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1348075590 0x7f134819b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.646+0000 7f13477fe700 1 -- 172.21.15.43:0/3498913406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1348074c70 msgr2=0x7f134819b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.646+0000 7f13477fe700 1 --2- 172.21.15.43:0/3498913406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1348074c70 0x7f134819b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.646+0000 7f13477fe700 1 -- 172.21.15.43:0/3498913406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1348075590 msgr2=0x7f134819b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.646+0000 7f13477fe700 1 --2- 172.21.15.43:0/3498913406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1348075590 0x7f134819b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.646+0000 7f13477fe700 1 -- 172.21.15.43:0/3498913406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13481a03b0 con 0x7f134810f440 2024-04-21T08:38:01.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.646+0000 7f13467fc700 1 --2- 172.21.15.43:0/3498913406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1348075590 0x7f134819b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:01.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.646+0000 7f13477fe700 1 --2- 172.21.15.43:0/3498913406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134810f440 0x7f134819fc70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1340007f10 tx=0x7f134000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:01.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.646+0000 7f132ffff700 1 -- 172.21.15.43:0/3498913406 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134000ddd0 con 0x7f134810f440 2024-04-21T08:38:01.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.646+0000 7f134d480700 1 -- 172.21.15.43:0/3498913406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13481a06a0 con 0x7f134810f440 2024-04-21T08:38:01.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.646+0000 7f132ffff700 1 -- 172.21.15.43:0/3498913406 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1340009e20 con 0x7f134810f440 2024-04-21T08:38:01.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.646+0000 7f132ffff700 1 -- 172.21.15.43:0/3498913406 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1340015400 con 0x7f134810f440 2024-04-21T08:38:01.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.646+0000 7f134d480700 1 -- 172.21.15.43:0/3498913406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13481a7f20 con 0x7f134810f440 2024-04-21T08:38:01.480 INFO:teuthology.orchestra.run.smithi043.stdout:292057776171 2024-04-21T08:38:01.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.647+0000 7f134d480700 1 -- 172.21.15.43:0/3498913406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1348069730 con 0x7f134810f440 2024-04-21T08:38:01.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.648+0000 7f132ffff700 1 -- 172.21.15.43:0/3498913406 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1340004cd0 con 0x7f134810f440 2024-04-21T08:38:01.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.649+0000 7f132ffff700 1 --2- 172.21.15.43:0/3498913406 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f13300a3cd0 0x7f13300a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:01.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.649+0000 7f1346ffd700 1 --2- 172.21.15.43:0/3498913406 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f13300a3cd0 0x7f13300a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:01.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.649+0000 7f132ffff700 1 -- 172.21.15.43:0/3498913406 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f13400d0980 con 0x7f134810f440 2024-04-21T08:38:01.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.649+0000 7f1346ffd700 1 --2- 172.21.15.43:0/3498913406 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f13300a3cd0 0x7f13300a6190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1338005950 tx=0x7f133800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:01.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.653+0000 7f132ffff700 1 -- 172.21.15.43:0/3498913406 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1340010070 con 0x7f134810f440 2024-04-21T08:38:01.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.865+0000 7f134d480700 1 -- 172.21.15.43:0/3498913406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f134819c700 con 0x7f134810f440 2024-04-21T08:38:01.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.865+0000 7f132ffff700 1 -- 172.21.15.43:0/3498913406 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1340004ad0 con 0x7f134810f440 2024-04-21T08:38:01.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.867+0000 7f134d480700 1 -- 172.21.15.43:0/3498913406 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f13300a3cd0 msgr2=0x7f13300a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:01.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.867+0000 7f134d480700 1 --2- 172.21.15.43:0/3498913406 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f13300a3cd0 0x7f13300a6190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1338005950 tx=0x7f133800b410 comp rx=0 tx=0).stop 2024-04-21T08:38:01.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.867+0000 7f134d480700 1 -- 172.21.15.43:0/3498913406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134810f440 msgr2=0x7f134819fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:01.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.867+0000 7f134d480700 1 --2- 172.21.15.43:0/3498913406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134810f440 0x7f134819fc70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1340007f10 tx=0x7f134000d3b0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.867+0000 7f134d480700 1 -- 172.21.15.43:0/3498913406 shutdown_connections 2024-04-21T08:38:01.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.867+0000 7f134d480700 1 --2- 172.21.15.43:0/3498913406 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f13300a3cd0 0x7f13300a6190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.867+0000 7f134d480700 1 --2- 172.21.15.43:0/3498913406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1348074c70 0x7f134819b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.867+0000 7f134d480700 1 --2- 172.21.15.43:0/3498913406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134810f440 0x7f134819fc70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.867+0000 7f134d480700 1 --2- 172.21.15.43:0/3498913406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1348075590 0x7f134819b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.867+0000 7f134d480700 1 -- 172.21.15.43:0/3498913406 >> 172.21.15.43:0/3498913406 conn(0x7f1348079bf0 msgr2=0x7f13480ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:01.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.867+0000 7f134d480700 1 -- 172.21.15.43:0/3498913406 shutdown_connections 2024-04-21T08:38:01.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.867+0000 7f134d480700 1 -- 172.21.15.43:0/3498913406 wait complete. 2024-04-21T08:38:01.482 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:01 smithi043 ceph-mon[27978]: pgmap v345: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:38:01.482 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:01 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3498913406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-21T08:38:01.482 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:01 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/237593772' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-21T08:38:01.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:01 smithi175 ceph-mon[31920]: pgmap v345: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:38:01.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:01 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3498913406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-21T08:38:01.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:01 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/237593772' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-21T08:38:01.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.965+0000 7f292497b700 1 -- 172.21.15.43:0/4121234676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2920108a30 msgr2=0x7f2920108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:01.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.965+0000 7f292497b700 1 --2- 172.21.15.43:0/4121234676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2920108a30 0x7f2920108e10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2910009990 tx=0x7f2910009ca0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.965+0000 7f292497b700 1 -- 172.21.15.43:0/4121234676 shutdown_connections 2024-04-21T08:38:01.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.965+0000 7f292497b700 1 --2- 172.21.15.43:0/4121234676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f292006b0f0 0x7f292010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.965+0000 7f292497b700 1 --2- 172.21.15.43:0/4121234676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2920108a30 0x7f2920108e10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.965+0000 7f292497b700 1 --2- 172.21.15.43:0/4121234676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f292006a750 0x7f292006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.965+0000 7f292497b700 1 -- 172.21.15.43:0/4121234676 >> 172.21.15.43:0/4121234676 conn(0x7f29200788f0 msgr2=0x7f2920078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:01.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.965+0000 7f292497b700 1 -- 172.21.15.43:0/4121234676 shutdown_connections 2024-04-21T08:38:01.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.965+0000 7f292497b700 1 -- 172.21.15.43:0/4121234676 wait complete. 2024-04-21T08:38:01.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.966+0000 7f292497b700 1 Processor -- start 2024-04-21T08:38:01.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.966+0000 7f292497b700 1 -- start start 2024-04-21T08:38:01.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.966+0000 7f292497b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f292006a750 0x7f2920101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:01.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.966+0000 7f292497b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f292006b0f0 0x7f2920101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:01.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.966+0000 7f292497b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2920108a30 0x7f2920102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:01.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.966+0000 7f292497b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2920112c80 con 0x7f292006a750 2024-04-21T08:38:01.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.966+0000 7f292497b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2920112b00 con 0x7f292006b0f0 2024-04-21T08:38:01.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.966+0000 7f292497b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2920112e00 con 0x7f2920108a30 2024-04-21T08:38:01.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.966+0000 7f291e59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f292006a750 0x7f2920101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:01.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.966+0000 7f291e59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f292006a750 0x7f2920101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33548/0 (socket says 172.21.15.43:33548) 2024-04-21T08:38:01.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.966+0000 7f291e59c700 1 -- 172.21.15.43:0/237593772 learned_addr learned my addr 172.21.15.43:0/237593772 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:01.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.966+0000 7f291ed9d700 1 --2- 172.21.15.43:0/237593772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2920108a30 0x7f2920102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:01.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.966+0000 7f291dd9b700 1 --2- 172.21.15.43:0/237593772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f292006b0f0 0x7f2920101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:01.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.966+0000 7f291e59c700 1 -- 172.21.15.43:0/237593772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2920108a30 msgr2=0x7f2920102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:01.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.967+0000 7f291e59c700 1 --2- 172.21.15.43:0/237593772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2920108a30 0x7f2920102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.967+0000 7f291e59c700 1 -- 172.21.15.43:0/237593772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f292006b0f0 msgr2=0x7f2920101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:01.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.967+0000 7f291e59c700 1 --2- 172.21.15.43:0/237593772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f292006b0f0 0x7f2920101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.967+0000 7f291e59c700 1 -- 172.21.15.43:0/237593772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29201a7690 con 0x7f292006a750 2024-04-21T08:38:01.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.967+0000 7f291ed9d700 1 --2- 172.21.15.43:0/237593772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2920108a30 0x7f2920102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:01.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.967+0000 7f291dd9b700 1 --2- 172.21.15.43:0/237593772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f292006b0f0 0x7f2920101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:01.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.967+0000 7f291e59c700 1 --2- 172.21.15.43:0/237593772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f292006a750 0x7f2920101a10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2910005910 tx=0x7f2910003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:01.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.967+0000 7f290f7fe700 1 -- 172.21.15.43:0/237593772 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2910027070 con 0x7f292006a750 2024-04-21T08:38:01.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.967+0000 7f290f7fe700 1 -- 172.21.15.43:0/237593772 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2910004140 con 0x7f292006a750 2024-04-21T08:38:01.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.967+0000 7f292497b700 1 -- 172.21.15.43:0/237593772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29201a7920 con 0x7f292006a750 2024-04-21T08:38:01.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.967+0000 7f290f7fe700 1 -- 172.21.15.43:0/237593772 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291000f460 con 0x7f292006a750 2024-04-21T08:38:01.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.967+0000 7f292497b700 1 -- 172.21.15.43:0/237593772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29201a7d70 con 0x7f292006a750 2024-04-21T08:38:01.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.968+0000 7f292497b700 1 -- 172.21.15.43:0/237593772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2920069730 con 0x7f292006a750 2024-04-21T08:38:01.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.969+0000 7f290f7fe700 1 -- 172.21.15.43:0/237593772 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f291002a070 con 0x7f292006a750 2024-04-21T08:38:01.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.970+0000 7f290f7fe700 1 --2- 172.21.15.43:0/237593772 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f29080a3cd0 0x7f29080a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.970+0000 7f291dd9b700 1 --2- 172.21.15.43:0/237593772 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f29080a3cd0 0x7f29080a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.970+0000 7f290f7fe700 1 -- 172.21.15.43:0/237593772 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f29100d1fb0 con 0x7f292006a750 2024-04-21T08:38:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.970+0000 7f291dd9b700 1 --2- 172.21.15.43:0/237593772 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f29080a3cd0 0x7f29080a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2914005fd0 tx=0x7f2914005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:00.973+0000 7f290f7fe700 1 -- 172.21.15.43:0/237593772 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2910031070 con 0x7f292006a750 2024-04-21T08:38:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:01.182+0000 7f292497b700 1 -- 172.21.15.43:0/237593772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f292002e140 con 0x7f292006a750 2024-04-21T08:38:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:01.183+0000 7f290f7fe700 1 -- 172.21.15.43:0/237593772 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2910013070 con 0x7f292006a750 2024-04-21T08:38:01.791 INFO:teuthology.orchestra.run.smithi043.stdout:201863462970 2024-04-21T08:38:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:01.185+0000 7f292497b700 1 -- 172.21.15.43:0/237593772 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f29080a3cd0 msgr2=0x7f29080a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:01.185+0000 7f292497b700 1 --2- 172.21.15.43:0/237593772 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f29080a3cd0 0x7f29080a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2914005fd0 tx=0x7f2914005e30 comp rx=0 tx=0).stop 2024-04-21T08:38:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:01.185+0000 7f292497b700 1 -- 172.21.15.43:0/237593772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f292006a750 msgr2=0x7f2920101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:01.185+0000 7f292497b700 1 --2- 172.21.15.43:0/237593772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f292006a750 0x7f2920101a10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2910005910 tx=0x7f2910003730 comp rx=0 tx=0).stop 2024-04-21T08:38:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:01.185+0000 7f292497b700 1 -- 172.21.15.43:0/237593772 shutdown_connections 2024-04-21T08:38:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:01.185+0000 7f292497b700 1 --2- 172.21.15.43:0/237593772 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f29080a3cd0 0x7f29080a6190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:01.185+0000 7f292497b700 1 --2- 172.21.15.43:0/237593772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2920108a30 0x7f2920102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:01.185+0000 7f292497b700 1 --2- 172.21.15.43:0/237593772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f292006a750 0x7f2920101a10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:01.185+0000 7f292497b700 1 --2- 172.21.15.43:0/237593772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f292006b0f0 0x7f2920101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:01.185+0000 7f292497b700 1 -- 172.21.15.43:0/237593772 >> 172.21.15.43:0/237593772 conn(0x7f29200788f0 msgr2=0x7f29200ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:01.185+0000 7f292497b700 1 -- 172.21.15.43:0/237593772 shutdown_connections 2024-04-21T08:38:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:01.185+0000 7f292497b700 1 -- 172.21.15.43:0/237593772 wait complete. 2024-04-21T08:38:01.793 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:01 smithi112 ceph-mon[31085]: pgmap v345: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:38:01.793 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:01 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3498913406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-21T08:38:01.793 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:01 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/237593772' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-21T08:38:02.441 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462964 got 201863462970 for osd.6 2024-04-21T08:38:02.441 DEBUG:teuthology.parallel:result is None 2024-04-21T08:38:02.885 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776165 got 292057776171 for osd.9 2024-04-21T08:38:02.885 DEBUG:teuthology.parallel:result is None 2024-04-21T08:38:03.357 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:03 smithi043 ceph-mon[27978]: pgmap v346: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:38:03.357 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:03 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:38:03.357 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:03 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:38:03.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.367+0000 7fbb8ab4a700 1 -- 172.21.15.43:0/2989382914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb84101030 msgr2=0x7fbb84101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:03.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.367+0000 7fbb8ab4a700 1 --2- 172.21.15.43:0/2989382914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb84101030 0x7fbb84101490 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbb78009990 tx=0x7fbb78009ca0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.367+0000 7fbb8ab4a700 1 -- 172.21.15.43:0/2989382914 shutdown_connections 2024-04-21T08:38:03.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.367+0000 7fbb8ab4a700 1 --2- 172.21.15.43:0/2989382914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb84108ad0 0x7fbb84108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.367+0000 7fbb8ab4a700 1 --2- 172.21.15.43:0/2989382914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb841019d0 0x7fbb8410f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.367+0000 7fbb8ab4a700 1 --2- 172.21.15.43:0/2989382914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb84101030 0x7fbb84101490 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.367+0000 7fbb8ab4a700 1 -- 172.21.15.43:0/2989382914 >> 172.21.15.43:0/2989382914 conn(0x7fbb84077fb0 msgr2=0x7fbb840ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.367+0000 7fbb8ab4a700 1 -- 172.21.15.43:0/2989382914 shutdown_connections 2024-04-21T08:38:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.367+0000 7fbb8ab4a700 1 -- 172.21.15.43:0/2989382914 wait complete. 2024-04-21T08:38:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.368+0000 7fbb8ab4a700 1 Processor -- start 2024-04-21T08:38:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.368+0000 7fbb8ab4a700 1 -- start start 2024-04-21T08:38:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.368+0000 7fbb8ab4a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb84101030 0x7fbb84102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.368+0000 7fbb8ab4a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb841019d0 0x7fbb84102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.368+0000 7fbb8ab4a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb84108ad0 0x7fbb84102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.368+0000 7fbb8ab4a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbb84112c00 con 0x7fbb841019d0 2024-04-21T08:38:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.368+0000 7fbb8ab4a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbb84112a80 con 0x7fbb84101030 2024-04-21T08:38:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.368+0000 7fbb8ab4a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbb84112d80 con 0x7fbb84108ad0 2024-04-21T08:38:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.368+0000 7fbb83fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb841019d0 0x7fbb84102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.368+0000 7fbb83fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb841019d0 0x7fbb84102750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33568/0 (socket says 172.21.15.43:33568) 2024-04-21T08:38:03.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.368+0000 7fbb83fff700 1 -- 172.21.15.43:0/1305799028 learned_addr learned my addr 172.21.15.43:0/1305799028 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:03.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.369+0000 7fbb888e6700 1 --2- 172.21.15.43:0/1305799028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb84101030 0x7fbb84102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:03.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.369+0000 7fbb890e7700 1 --2- 172.21.15.43:0/1305799028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb84108ad0 0x7fbb84102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:03.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.369+0000 7fbb83fff700 1 -- 172.21.15.43:0/1305799028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb84108ad0 msgr2=0x7fbb84102c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:03.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.369+0000 7fbb83fff700 1 --2- 172.21.15.43:0/1305799028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb84108ad0 0x7fbb84102c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.369+0000 7fbb83fff700 1 -- 172.21.15.43:0/1305799028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb84101030 msgr2=0x7fbb84102210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:03.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.369+0000 7fbb83fff700 1 --2- 172.21.15.43:0/1305799028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb84101030 0x7fbb84102210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.369+0000 7fbb83fff700 1 -- 172.21.15.43:0/1305799028 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb841a75b0 con 0x7fbb841019d0 2024-04-21T08:38:03.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.369+0000 7fbb888e6700 1 --2- 172.21.15.43:0/1305799028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb84101030 0x7fbb84102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:03.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.369+0000 7fbb890e7700 1 --2- 172.21.15.43:0/1305799028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb84108ad0 0x7fbb84102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:03.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.369+0000 7fbb83fff700 1 --2- 172.21.15.43:0/1305799028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb841019d0 0x7fbb84102750 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbb780057f0 tx=0x7fbb78003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:03.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.369+0000 7fbb81ffb700 1 -- 172.21.15.43:0/1305799028 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb78027070 con 0x7fbb841019d0 2024-04-21T08:38:03.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.369+0000 7fbb81ffb700 1 -- 172.21.15.43:0/1305799028 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb780041c0 con 0x7fbb841019d0 2024-04-21T08:38:03.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.369+0000 7fbb81ffb700 1 -- 172.21.15.43:0/1305799028 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb78022e60 con 0x7fbb841019d0 2024-04-21T08:38:03.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.369+0000 7fbb8ab4a700 1 -- 172.21.15.43:0/1305799028 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb841a7840 con 0x7fbb841019d0 2024-04-21T08:38:03.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.370+0000 7fbb8ab4a700 1 -- 172.21.15.43:0/1305799028 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb841a7d20 con 0x7fbb841019d0 2024-04-21T08:38:03.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.371+0000 7fbb8ab4a700 1 -- 172.21.15.43:0/1305799028 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb84069730 con 0x7fbb841019d0 2024-04-21T08:38:03.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.372+0000 7fbb81ffb700 1 -- 172.21.15.43:0/1305799028 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbb7802a070 con 0x7fbb841019d0 2024-04-21T08:38:03.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.373+0000 7fbb81ffb700 1 --2- 172.21.15.43:0/1305799028 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fbb6c0a3cd0 0x7fbb6c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:03.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.373+0000 7fbb888e6700 1 --2- 172.21.15.43:0/1305799028 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fbb6c0a3cd0 0x7fbb6c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:03.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.373+0000 7fbb81ffb700 1 -- 172.21.15.43:0/1305799028 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fbb780d1660 con 0x7fbb841019d0 2024-04-21T08:38:03.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.373+0000 7fbb888e6700 1 --2- 172.21.15.43:0/1305799028 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fbb6c0a3cd0 0x7fbb6c0a6190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbb74005fd0 tx=0x7fbb74005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:03.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.377+0000 7fbb81ffb700 1 -- 172.21.15.43:0/1305799028 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbb78013070 con 0x7fbb841019d0 2024-04-21T08:38:03.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.589+0000 7fbb8ab4a700 1 -- 172.21.15.43:0/1305799028 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fbb8402df80 con 0x7fbb841019d0 2024-04-21T08:38:03.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.590+0000 7fbb81ffb700 1 -- 172.21.15.43:0/1305799028 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fbb7809dbb0 con 0x7fbb841019d0 2024-04-21T08:38:03.589 INFO:teuthology.orchestra.run.smithi043.stdout:313532612646 2024-04-21T08:38:03.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.593+0000 7fbb8ab4a700 1 -- 172.21.15.43:0/1305799028 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fbb6c0a3cd0 msgr2=0x7fbb6c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:03.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.593+0000 7fbb8ab4a700 1 --2- 172.21.15.43:0/1305799028 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fbb6c0a3cd0 0x7fbb6c0a6190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbb74005fd0 tx=0x7fbb74005d90 comp rx=0 tx=0).stop 2024-04-21T08:38:03.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.593+0000 7fbb8ab4a700 1 -- 172.21.15.43:0/1305799028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb841019d0 msgr2=0x7fbb84102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:03.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.593+0000 7fbb8ab4a700 1 --2- 172.21.15.43:0/1305799028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb841019d0 0x7fbb84102750 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbb780057f0 tx=0x7fbb78003730 comp rx=0 tx=0).stop 2024-04-21T08:38:03.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.594+0000 7fbb8ab4a700 1 -- 172.21.15.43:0/1305799028 shutdown_connections 2024-04-21T08:38:03.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.594+0000 7fbb8ab4a700 1 --2- 172.21.15.43:0/1305799028 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fbb6c0a3cd0 0x7fbb6c0a6190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.594+0000 7fbb8ab4a700 1 --2- 172.21.15.43:0/1305799028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb84108ad0 0x7fbb84102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.594+0000 7fbb8ab4a700 1 --2- 172.21.15.43:0/1305799028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb84101030 0x7fbb84102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.594+0000 7fbb8ab4a700 1 --2- 172.21.15.43:0/1305799028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb841019d0 0x7fbb84102750 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.594+0000 7fbb8ab4a700 1 -- 172.21.15.43:0/1305799028 >> 172.21.15.43:0/1305799028 conn(0x7fbb84077fb0 msgr2=0x7fbb840ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:03.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.594+0000 7fbb8ab4a700 1 -- 172.21.15.43:0/1305799028 shutdown_connections 2024-04-21T08:38:03.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.594+0000 7fbb8ab4a700 1 -- 172.21.15.43:0/1305799028 wait complete. 2024-04-21T08:38:03.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.610+0000 7fcb01697700 1 -- 172.21.15.43:0/2388510037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcafc108a50 msgr2=0x7fcafc108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:03.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.610+0000 7fcb01697700 1 --2- 172.21.15.43:0/2388510037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcafc108a50 0x7fcafc108e30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009990 tx=0x7fcaec009ca0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.610+0000 7fcb01697700 1 -- 172.21.15.43:0/2388510037 shutdown_connections 2024-04-21T08:38:03.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.610+0000 7fcb01697700 1 --2- 172.21.15.43:0/2388510037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcafc06bab0 0x7fcafc10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.610+0000 7fcb01697700 1 --2- 172.21.15.43:0/2388510037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcafc108a50 0x7fcafc108e30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.610+0000 7fcb01697700 1 --2- 172.21.15.43:0/2388510037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcafc06b110 0x7fcafc06b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.610+0000 7fcb01697700 1 -- 172.21.15.43:0/2388510037 >> 172.21.15.43:0/2388510037 conn(0x7fcafc0788f0 msgr2=0x7fcafc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:03.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.610+0000 7fcb01697700 1 -- 172.21.15.43:0/2388510037 shutdown_connections 2024-04-21T08:38:03.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.610+0000 7fcb01697700 1 -- 172.21.15.43:0/2388510037 wait complete. 2024-04-21T08:38:03.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.611+0000 7fcb01697700 1 Processor -- start 2024-04-21T08:38:03.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.611+0000 7fcb01697700 1 -- start start 2024-04-21T08:38:03.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.611+0000 7fcb01697700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcafc06b110 0x7fcafc101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:03.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.611+0000 7fcb01697700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcafc06bab0 0x7fcafc102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:03.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.611+0000 7fcb01697700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcafc108a50 0x7fcafc102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:03.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.611+0000 7fcb01697700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc112bf0 con 0x7fcafc06bab0 2024-04-21T08:38:03.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.611+0000 7fcb01697700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc112a70 con 0x7fcafc06b110 2024-04-21T08:38:03.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.611+0000 7fcb01697700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc112d70 con 0x7fcafc108a50 2024-04-21T08:38:03.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.611+0000 7fcafaffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcafc06b110 0x7fcafc101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:03.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.611+0000 7fcafb7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcafc108a50 0x7fcafc102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:03.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.611+0000 7fcafaffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcafc06b110 0x7fcafc101df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:43340/0 (socket says 172.21.15.43:43340) 2024-04-21T08:38:03.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.612+0000 7fcafb7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcafc108a50 0x7fcafc102870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.43:41148/0 (socket says 172.21.15.43:41148) 2024-04-21T08:38:03.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.612+0000 7fcafb7fe700 1 -- 172.21.15.43:0/1207412143 learned_addr learned my addr 172.21.15.43:0/1207412143 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:03.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.612+0000 7fcafa7fc700 1 --2- 172.21.15.43:0/1207412143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcafc06bab0 0x7fcafc102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:03.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.612+0000 7fcafb7fe700 1 -- 172.21.15.43:0/1207412143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcafc06b110 msgr2=0x7fcafc101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:03.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.612+0000 7fcafb7fe700 1 --2- 172.21.15.43:0/1207412143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcafc06b110 0x7fcafc101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.612+0000 7fcafb7fe700 1 -- 172.21.15.43:0/1207412143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcafc06bab0 msgr2=0x7fcafc102330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:03.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.612+0000 7fcafb7fe700 1 --2- 172.21.15.43:0/1207412143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcafc06bab0 0x7fcafc102330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.612+0000 7fcafb7fe700 1 -- 172.21.15.43:0/1207412143 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcafc1a76f0 con 0x7fcafc108a50 2024-04-21T08:38:03.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.612+0000 7fcafa7fc700 1 --2- 172.21.15.43:0/1207412143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcafc06bab0 0x7fcafc102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:38:03.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.612+0000 7fcafb7fe700 1 --2- 172.21.15.43:0/1207412143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcafc108a50 0x7fcafc102870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400ef30 tx=0x7fcaf400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:03.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.612+0000 7fcae3fff700 1 -- 172.21.15.43:0/1207412143 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf40092e0 con 0x7fcafc108a50 2024-04-21T08:38:03.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.612+0000 7fcae3fff700 1 -- 172.21.15.43:0/1207412143 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaf4004bb0 con 0x7fcafc108a50 2024-04-21T08:38:03.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.612+0000 7fcb01697700 1 -- 172.21.15.43:0/1207412143 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcafc1a7980 con 0x7fcafc108a50 2024-04-21T08:38:03.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.612+0000 7fcae3fff700 1 -- 172.21.15.43:0/1207412143 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4004020 con 0x7fcafc108a50 2024-04-21T08:38:03.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.613+0000 7fcb01697700 1 -- 172.21.15.43:0/1207412143 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcafc1a7ec0 con 0x7fcafc108a50 2024-04-21T08:38:03.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.614+0000 7fcb01697700 1 -- 172.21.15.43:0/1207412143 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcafc104190 con 0x7fcafc108a50 2024-04-21T08:38:03.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.615+0000 7fcae3fff700 1 -- 172.21.15.43:0/1207412143 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcaf401b030 con 0x7fcafc108a50 2024-04-21T08:38:03.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.616+0000 7fcae3fff700 1 --2- 172.21.15.43:0/1207412143 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fcae40a3cd0 0x7fcae40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:03.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.616+0000 7fcae3fff700 1 -- 172.21.15.43:0/1207412143 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fcaf40d05f0 con 0x7fcafc108a50 2024-04-21T08:38:03.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.616+0000 7fcafaffd700 1 --2- 172.21.15.43:0/1207412143 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fcae40a3cd0 0x7fcae40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:03.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.616+0000 7fcafaffd700 1 --2- 172.21.15.43:0/1207412143 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fcae40a3cd0 0x7fcae40a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0098e0 tx=0x7fcaec005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:03.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.620+0000 7fcae3fff700 1 -- 172.21.15.43:0/1207412143 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcaf4022020 con 0x7fcafc108a50 2024-04-21T08:38:03.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:03 smithi175 ceph-mon[31920]: pgmap v346: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:38:03.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:03 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:38:03.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:03 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:38:03.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:03 smithi112 ceph-mon[31085]: pgmap v346: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:38:03.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:03 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:38:03.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:03 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:38:03.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.833+0000 7fcb01697700 1 -- 172.21.15.43:0/1207412143 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fcafc02df80 con 0x7fcafc108a50 2024-04-21T08:38:03.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.834+0000 7fcae3fff700 1 -- 172.21.15.43:0/1207412143 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcaf4010040 con 0x7fcafc108a50 2024-04-21T08:38:03.833 INFO:teuthology.orchestra.run.smithi043.stdout:107374182474 2024-04-21T08:38:03.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.836+0000 7fcb01697700 1 -- 172.21.15.43:0/1207412143 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fcae40a3cd0 msgr2=0x7fcae40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:03.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.836+0000 7fcb01697700 1 --2- 172.21.15.43:0/1207412143 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fcae40a3cd0 0x7fcae40a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0098e0 tx=0x7fcaec005960 comp rx=0 tx=0).stop 2024-04-21T08:38:03.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.836+0000 7fcb01697700 1 -- 172.21.15.43:0/1207412143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcafc108a50 msgr2=0x7fcafc102870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:03.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.836+0000 7fcb01697700 1 --2- 172.21.15.43:0/1207412143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcafc108a50 0x7fcafc102870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400ef30 tx=0x7fcaf400c3a0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.836+0000 7fcb01697700 1 -- 172.21.15.43:0/1207412143 shutdown_connections 2024-04-21T08:38:03.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.836+0000 7fcb01697700 1 --2- 172.21.15.43:0/1207412143 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fcae40a3cd0 0x7fcae40a6190 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.836+0000 7fcb01697700 1 --2- 172.21.15.43:0/1207412143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcafc108a50 0x7fcafc102870 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.836+0000 7fcb01697700 1 --2- 172.21.15.43:0/1207412143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcafc06b110 0x7fcafc101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.836+0000 7fcb01697700 1 --2- 172.21.15.43:0/1207412143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcafc06bab0 0x7fcafc102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.836+0000 7fcb01697700 1 -- 172.21.15.43:0/1207412143 >> 172.21.15.43:0/1207412143 conn(0x7fcafc0788f0 msgr2=0x7fcafc0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:03.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.836+0000 7fcb01697700 1 -- 172.21.15.43:0/1207412143 shutdown_connections 2024-04-21T08:38:03.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:03.836+0000 7fcb01697700 1 -- 172.21.15.43:0/1207412143 wait complete. 2024-04-21T08:38:04.555 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:04 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1305799028' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-21T08:38:04.556 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:04 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1207412143' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-21T08:38:04.556 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:04 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:04.556 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:04 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:38:04.556 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:04 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:38:04.584 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612642 got 313532612646 for osd.10 2024-04-21T08:38:04.585 DEBUG:teuthology.parallel:result is None 2024-04-21T08:38:04.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:04 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1305799028' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-21T08:38:04.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:04 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1207412143' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-21T08:38:04.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:04 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:04.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:04 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:38:04.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:04 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:38:04.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:04 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1305799028' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-21T08:38:04.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:04 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1207412143' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-21T08:38:04.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:04 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:04.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:04 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:38:04.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:04 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' 2024-04-21T08:38:05.182 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182467 got 107374182474 for osd.3 2024-04-21T08:38:05.183 DEBUG:teuthology.parallel:result is None 2024-04-21T08:38:05.183 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-21T08:38:05.183 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-21T08:38:05.437 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:05.560 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:05 smithi043 ceph-mon[27978]: pgmap v347: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-21T08:38:05.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:05 smithi175 ceph-mon[31920]: pgmap v347: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-21T08:38:05.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:05 smithi112 ceph-mon[31085]: pgmap v347: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-21T08:38:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.300+0000 7f291d72e700 1 -- 172.21.15.43:0/3558900766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2918107990 msgr2=0x7f2918107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.300+0000 7f291d72e700 1 --2- 172.21.15.43:0/3558900766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2918107990 0x7f2918107d70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2908009990 tx=0x7f2908009ca0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.300+0000 7f291d72e700 1 -- 172.21.15.43:0/3558900766 shutdown_connections 2024-04-21T08:38:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.300+0000 7f291d72e700 1 --2- 172.21.15.43:0/3558900766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2918102300 0x7f291810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.300+0000 7f291d72e700 1 --2- 172.21.15.43:0/3558900766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2918107990 0x7f2918107d70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.300+0000 7f291d72e700 1 --2- 172.21.15.43:0/3558900766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2918101940 0x7f2918101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.300+0000 7f291d72e700 1 -- 172.21.15.43:0/3558900766 >> 172.21.15.43:0/3558900766 conn(0x7f29180788f0 msgr2=0x7f2918078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.300+0000 7f291d72e700 1 -- 172.21.15.43:0/3558900766 shutdown_connections 2024-04-21T08:38:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.300+0000 7f291d72e700 1 -- 172.21.15.43:0/3558900766 wait complete. 2024-04-21T08:38:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.300+0000 7f291d72e700 1 Processor -- start 2024-04-21T08:38:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.301+0000 7f291d72e700 1 -- start start 2024-04-21T08:38:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.301+0000 7f291d72e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2918101940 0x7f2918199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.301+0000 7f291d72e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2918102300 0x7f29181996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.301+0000 7f291d72e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2918107990 0x7f291819d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.301+0000 7f291d72e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2918111b00 con 0x7f2918107990 2024-04-21T08:38:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.301+0000 7f291d72e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2918111980 con 0x7f2918101940 2024-04-21T08:38:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.301+0000 7f291d72e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2918111c80 con 0x7f2918102300 2024-04-21T08:38:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.301+0000 7f29177fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2918107990 0x7f291819d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.301+0000 7f29177fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2918107990 0x7f291819d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38520/0 (socket says 172.21.15.43:38520) 2024-04-21T08:38:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.301+0000 7f29177fe700 1 -- 172.21.15.43:0/1474815723 learned_addr learned my addr 172.21.15.43:0/1474815723 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.301+0000 7f2916ffd700 1 --2- 172.21.15.43:0/1474815723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2918101940 0x7f2918199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.301+0000 7f29167fc700 1 --2- 172.21.15.43:0/1474815723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2918102300 0x7f29181996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.301+0000 7f29177fe700 1 -- 172.21.15.43:0/1474815723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2918102300 msgr2=0x7f29181996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.301+0000 7f29177fe700 1 --2- 172.21.15.43:0/1474815723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2918102300 0x7f29181996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.301+0000 7f29177fe700 1 -- 172.21.15.43:0/1474815723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2918101940 msgr2=0x7f2918199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.301+0000 7f29177fe700 1 --2- 172.21.15.43:0/1474815723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2918101940 0x7f2918199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.301+0000 7f29177fe700 1 -- 172.21.15.43:0/1474815723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291819e120 con 0x7f2918107990 2024-04-21T08:38:07.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.301+0000 7f29167fc700 1 --2- 172.21.15.43:0/1474815723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2918102300 0x7f29181996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:07.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.302+0000 7f29177fe700 1 --2- 172.21.15.43:0/1474815723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2918107990 0x7f291819d9e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2910007d70 tx=0x7f291000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:07.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.302+0000 7f28fffff700 1 -- 172.21.15.43:0/1474815723 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2910016020 con 0x7f2918107990 2024-04-21T08:38:07.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.302+0000 7f291d72e700 1 -- 172.21.15.43:0/1474815723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291819e410 con 0x7f2918107990 2024-04-21T08:38:07.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.302+0000 7f28fffff700 1 -- 172.21.15.43:0/1474815723 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f291000ad40 con 0x7f2918107990 2024-04-21T08:38:07.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.302+0000 7f28fffff700 1 -- 172.21.15.43:0/1474815723 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29100115e0 con 0x7f2918107990 2024-04-21T08:38:07.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.302+0000 7f291d72e700 1 -- 172.21.15.43:0/1474815723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29181a5c90 con 0x7f2918107990 2024-04-21T08:38:07.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.303+0000 7f291d72e700 1 -- 172.21.15.43:0/1474815723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2918069730 con 0x7f2918107990 2024-04-21T08:38:07.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.304+0000 7f28fffff700 1 -- 172.21.15.43:0/1474815723 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2910019020 con 0x7f2918107990 2024-04-21T08:38:07.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.304+0000 7f28fffff700 1 --2- 172.21.15.43:0/1474815723 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f29000a3cd0 0x7f29000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:07.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.305+0000 7f2916ffd700 1 --2- 172.21.15.43:0/1474815723 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f29000a3cd0 0x7f29000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:07.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.305+0000 7f28fffff700 1 -- 172.21.15.43:0/1474815723 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f29100d06e0 con 0x7f2918107990 2024-04-21T08:38:07.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.305+0000 7f2916ffd700 1 --2- 172.21.15.43:0/1474815723 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f29000a3cd0 0x7f29000a6190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f290800b5c0 tx=0x7f2908005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:07.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.309+0000 7f28fffff700 1 -- 172.21.15.43:0/1474815723 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2910020020 con 0x7f2918107990 2024-04-21T08:38:07.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.513+0000 7f291d72e700 1 -- 172.21.15.43:0/1474815723 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2918100460 con 0x7f29000a3cd0 2024-04-21T08:38:07.513 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:07 smithi043 ceph-mon[27978]: pgmap v348: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:38:07.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.523+0000 7f28fffff700 1 -- 172.21.15.43:0/1474815723 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340398 (secure 0 0 0) 0x7f2918100460 con 0x7f29000a3cd0 2024-04-21T08:38:07.528 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:07.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.527+0000 7f291d72e700 1 -- 172.21.15.43:0/1474815723 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f29000a3cd0 msgr2=0x7f29000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:07.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.527+0000 7f291d72e700 1 --2- 172.21.15.43:0/1474815723 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f29000a3cd0 0x7f29000a6190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f290800b5c0 tx=0x7f2908005960 comp rx=0 tx=0).stop 2024-04-21T08:38:07.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.527+0000 7f291d72e700 1 -- 172.21.15.43:0/1474815723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2918107990 msgr2=0x7f291819d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:07.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.527+0000 7f291d72e700 1 --2- 172.21.15.43:0/1474815723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2918107990 0x7f291819d9e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2910007d70 tx=0x7f291000a340 comp rx=0 tx=0).stop 2024-04-21T08:38:07.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.527+0000 7f291d72e700 1 -- 172.21.15.43:0/1474815723 shutdown_connections 2024-04-21T08:38:07.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.527+0000 7f291d72e700 1 --2- 172.21.15.43:0/1474815723 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f29000a3cd0 0x7f29000a6190 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.527+0000 7f291d72e700 1 --2- 172.21.15.43:0/1474815723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2918107990 0x7f291819d9e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.527+0000 7f291d72e700 1 --2- 172.21.15.43:0/1474815723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2918101940 0x7f2918199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.527+0000 7f291d72e700 1 --2- 172.21.15.43:0/1474815723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2918102300 0x7f29181996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.527+0000 7f291d72e700 1 -- 172.21.15.43:0/1474815723 >> 172.21.15.43:0/1474815723 conn(0x7f29180788f0 msgr2=0x7f2918105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:07.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.528+0000 7f291d72e700 1 -- 172.21.15.43:0/1474815723 shutdown_connections 2024-04-21T08:38:07.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:07.528+0000 7f291d72e700 1 -- 172.21.15.43:0/1474815723 wait complete. 2024-04-21T08:38:07.536 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-21T08:38:07.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:07 smithi175 ceph-mon[31920]: pgmap v348: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:38:07.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:07 smithi112 ceph-mon[31085]: pgmap v348: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:38:08.326 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":348,"stamp":"2024-04-21T08:38:06.002215+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661553,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":903,"num_read_kb":578,"num_write":426,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380244,"kb_used_data":14384,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561580,"statfs":{"total":1151940427776,"available":1151551057920,"internally_reserved":0,"allocated":14729216,"data_stored":8024393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":374453564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003187"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.218236+0000","last_change":"2024-04-21T08:36:23.983921+0000","last_active":"2024-04-21T08:37:15.218236+0000","last_peered":"2024-04-21T08:37:15.218236+0000","last_clean":"2024-04-21T08:37:15.218236+0000","last_became_active":"2024-04-21T08:36:23.983638+0000","last_became_peered":"2024-04-21T08:36:23.983638+0000","last_unstale":"2024-04-21T08:37:15.218236+0000","last_undegraded":"2024-04-21T08:37:15.218236+0000","last_fullsized":"2024-04-21T08:37:15.218236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:29:44.646308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.115599+0000","last_change":"2024-04-21T08:36:23.990029+0000","last_active":"2024-04-21T08:37:15.115599+0000","last_peered":"2024-04-21T08:37:15.115599+0000","last_clean":"2024-04-21T08:37:15.115599+0000","last_became_active":"2024-04-21T08:36:23.989141+0000","last_became_peered":"2024-04-21T08:36:23.989141+0000","last_unstale":"2024-04-21T08:37:15.115599+0000","last_undegraded":"2024-04-21T08:37:15.115599+0000","last_fullsized":"2024-04-21T08:37:15.115599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:23:58.692342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.115320+0000","last_change":"2024-04-21T08:36:23.982205+0000","last_active":"2024-04-21T08:37:15.115320+0000","last_peered":"2024-04-21T08:37:15.115320+0000","last_clean":"2024-04-21T08:37:15.115320+0000","last_became_active":"2024-04-21T08:36:23.981876+0000","last_became_peered":"2024-04-21T08:36:23.981876+0000","last_unstale":"2024-04-21T08:37:15.115320+0000","last_undegraded":"2024-04-21T08:37:15.115320+0000","last_fullsized":"2024-04-21T08:37:15.115320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:12:27.121106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.114980+0000","last_change":"2024-04-21T08:36:23.987425+0000","last_active":"2024-04-21T08:37:15.114980+0000","last_peered":"2024-04-21T08:37:15.114980+0000","last_clean":"2024-04-21T08:37:15.114980+0000","last_became_active":"2024-04-21T08:36:23.987109+0000","last_became_peered":"2024-04-21T08:36:23.987109+0000","last_unstale":"2024-04-21T08:37:15.114980+0000","last_undegraded":"2024-04-21T08:37:15.114980+0000","last_fullsized":"2024-04-21T08:37:15.114980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:44.176324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.112980+0000","last_change":"2024-04-21T08:36:23.990375+0000","last_active":"2024-04-21T08:37:15.112980+0000","last_peered":"2024-04-21T08:37:15.112980+0000","last_clean":"2024-04-21T08:37:15.112980+0000","last_became_active":"2024-04-21T08:36:23.990058+0000","last_became_peered":"2024-04-21T08:36:23.990058+0000","last_unstale":"2024-04-21T08:37:15.112980+0000","last_undegraded":"2024-04-21T08:37:15.112980+0000","last_fullsized":"2024-04-21T08:37:15.112980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:24:09.647528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.117978+0000","last_change":"2024-04-21T08:36:23.979848+0000","last_active":"2024-04-21T08:37:15.117978+0000","last_peered":"2024-04-21T08:37:15.117978+0000","last_clean":"2024-04-21T08:37:15.117978+0000","last_became_active":"2024-04-21T08:36:23.979511+0000","last_became_peered":"2024-04-21T08:36:23.979511+0000","last_unstale":"2024-04-21T08:37:15.117978+0000","last_undegraded":"2024-04-21T08:37:15.117978+0000","last_fullsized":"2024-04-21T08:37:15.117978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:07:46.551369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.118755+0000","last_change":"2024-04-21T08:37:15.118755+0000","last_active":"2024-04-21T08:37:15.118755+0000","last_peered":"2024-04-21T08:37:15.118755+0000","last_clean":"2024-04-21T08:37:15.118755+0000","last_became_active":"2024-04-21T08:37:15.118347+0000","last_became_peered":"2024-04-21T08:37:15.118347+0000","last_unstale":"2024-04-21T08:37:15.118755+0000","last_undegraded":"2024-04-21T08:37:15.118755+0000","last_fullsized":"2024-04-21T08:37:15.118755+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:36:43.398325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.555143+0000","last_change":"2024-04-21T08:36:23.987517+0000","last_active":"2024-04-21T08:37:15.555143+0000","last_peered":"2024-04-21T08:37:15.555143+0000","last_clean":"2024-04-21T08:37:15.555143+0000","last_became_active":"2024-04-21T08:36:23.987201+0000","last_became_peered":"2024-04-21T08:36:23.987201+0000","last_unstale":"2024-04-21T08:37:15.555143+0000","last_undegraded":"2024-04-21T08:37:15.555143+0000","last_fullsized":"2024-04-21T08:37:15.555143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:42:53.080863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.120308+0000","last_change":"2024-04-21T08:36:23.989434+0000","last_active":"2024-04-21T08:37:15.120308+0000","last_peered":"2024-04-21T08:37:15.120308+0000","last_clean":"2024-04-21T08:37:15.120308+0000","last_became_active":"2024-04-21T08:36:23.989125+0000","last_became_peered":"2024-04-21T08:36:23.989125+0000","last_unstale":"2024-04-21T08:37:15.120308+0000","last_undegraded":"2024-04-21T08:37:15.120308+0000","last_fullsized":"2024-04-21T08:37:15.120308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:30:07.219056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.117822+0000","last_change":"2024-04-21T08:36:23.980725+0000","last_active":"2024-04-21T08:37:15.117822+0000","last_peered":"2024-04-21T08:37:15.117822+0000","last_clean":"2024-04-21T08:37:15.117822+0000","last_became_active":"2024-04-21T08:36:23.980363+0000","last_became_peered":"2024-04-21T08:36:23.980363+0000","last_unstale":"2024-04-21T08:37:15.117822+0000","last_undegraded":"2024-04-21T08:37:15.117822+0000","last_fullsized":"2024-04-21T08:37:15.117822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:26:46.202030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116293+0000","last_change":"2024-04-21T08:36:23.989369+0000","last_active":"2024-04-21T08:37:15.116293+0000","last_peered":"2024-04-21T08:37:15.116293+0000","last_clean":"2024-04-21T08:37:15.116293+0000","last_became_active":"2024-04-21T08:36:23.989030+0000","last_became_peered":"2024-04-21T08:36:23.989030+0000","last_unstale":"2024-04-21T08:37:15.116293+0000","last_undegraded":"2024-04-21T08:37:15.116293+0000","last_fullsized":"2024-04-21T08:37:15.116293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:23:13.725703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.114871+0000","last_change":"2024-04-21T08:36:23.993485+0000","last_active":"2024-04-21T08:37:15.114871+0000","last_peered":"2024-04-21T08:37:15.114871+0000","last_clean":"2024-04-21T08:37:15.114871+0000","last_became_active":"2024-04-21T08:36:23.993124+0000","last_became_peered":"2024-04-21T08:36:23.993124+0000","last_unstale":"2024-04-21T08:37:15.114871+0000","last_undegraded":"2024-04-21T08:37:15.114871+0000","last_fullsized":"2024-04-21T08:37:15.114871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:53:16.249838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.119846+0000","last_change":"2024-04-21T08:36:23.983146+0000","last_active":"2024-04-21T08:37:15.119846+0000","last_peered":"2024-04-21T08:37:15.119846+0000","last_clean":"2024-04-21T08:37:15.119846+0000","last_became_active":"2024-04-21T08:36:23.982766+0000","last_became_peered":"2024-04-21T08:36:23.982766+0000","last_unstale":"2024-04-21T08:37:15.119846+0000","last_undegraded":"2024-04-21T08:37:15.119846+0000","last_fullsized":"2024-04-21T08:37:15.119846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:30:44.090929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.114937+0000","last_change":"2024-04-21T08:36:23.988906+0000","last_active":"2024-04-21T08:37:15.114937+0000","last_peered":"2024-04-21T08:37:15.114937+0000","last_clean":"2024-04-21T08:37:15.114937+0000","last_became_active":"2024-04-21T08:36:23.988489+0000","last_became_peered":"2024-04-21T08:36:23.988489+0000","last_unstale":"2024-04-21T08:37:15.114937+0000","last_undegraded":"2024-04-21T08:37:15.114937+0000","last_fullsized":"2024-04-21T08:37:15.114937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:06:43.271650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.123343+0000","last_change":"2024-04-21T08:37:15.123343+0000","last_active":"2024-04-21T08:37:15.123343+0000","last_peered":"2024-04-21T08:37:15.123343+0000","last_clean":"2024-04-21T08:37:15.123343+0000","last_became_active":"2024-04-21T08:37:15.122959+0000","last_became_peered":"2024-04-21T08:37:15.122959+0000","last_unstale":"2024-04-21T08:37:15.123343+0000","last_undegraded":"2024-04-21T08:37:15.123343+0000","last_fullsized":"2024-04-21T08:37:15.123343+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:56:19.179879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.118945+0000","last_change":"2024-04-21T08:36:23.983136+0000","last_active":"2024-04-21T08:37:15.118945+0000","last_peered":"2024-04-21T08:37:15.118945+0000","last_clean":"2024-04-21T08:37:15.118945+0000","last_became_active":"2024-04-21T08:36:23.982698+0000","last_became_peered":"2024-04-21T08:36:23.982698+0000","last_unstale":"2024-04-21T08:37:15.118945+0000","last_undegraded":"2024-04-21T08:37:15.118945+0000","last_fullsized":"2024-04-21T08:37:15.118945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:34.750941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.218158+0000","last_change":"2024-04-21T08:36:23.984701+0000","last_active":"2024-04-21T08:37:15.218158+0000","last_peered":"2024-04-21T08:37:15.218158+0000","last_clean":"2024-04-21T08:37:15.218158+0000","last_became_active":"2024-04-21T08:36:23.984432+0000","last_became_peered":"2024-04-21T08:36:23.984432+0000","last_unstale":"2024-04-21T08:37:15.218158+0000","last_undegraded":"2024-04-21T08:37:15.218158+0000","last_fullsized":"2024-04-21T08:37:15.218158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:10:58.944380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.113513+0000","last_change":"2024-04-21T08:36:23.982781+0000","last_active":"2024-04-21T08:37:15.113513+0000","last_peered":"2024-04-21T08:37:15.113513+0000","last_clean":"2024-04-21T08:37:15.113513+0000","last_became_active":"2024-04-21T08:36:23.982459+0000","last_became_peered":"2024-04-21T08:36:23.982459+0000","last_unstale":"2024-04-21T08:37:15.113513+0000","last_undegraded":"2024-04-21T08:37:15.113513+0000","last_fullsized":"2024-04-21T08:37:15.113513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:44.657504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.122441+0000","last_change":"2024-04-21T08:36:23.992002+0000","last_active":"2024-04-21T08:37:15.122441+0000","last_peered":"2024-04-21T08:37:15.122441+0000","last_clean":"2024-04-21T08:37:15.122441+0000","last_became_active":"2024-04-21T08:36:23.991053+0000","last_became_peered":"2024-04-21T08:36:23.991053+0000","last_unstale":"2024-04-21T08:37:15.122441+0000","last_undegraded":"2024-04-21T08:37:15.122441+0000","last_fullsized":"2024-04-21T08:37:15.122441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:56:37.145949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.122197+0000","last_change":"2024-04-21T08:36:28.013195+0000","last_active":"2024-04-21T08:37:15.122197+0000","last_peered":"2024-04-21T08:37:15.122197+0000","last_clean":"2024-04-21T08:37:15.122197+0000","last_became_active":"2024-04-21T08:36:28.012656+0000","last_became_peered":"2024-04-21T08:36:28.012656+0000","last_unstale":"2024-04-21T08:37:15.122197+0000","last_undegraded":"2024-04-21T08:37:15.122197+0000","last_fullsized":"2024-04-21T08:37:15.122197+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:12:03.133583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":111,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:38:00.003704+0000","last_change":"2024-04-21T08:36:23.468957+0000","last_active":"2024-04-21T08:38:00.003704+0000","last_peered":"2024-04-21T08:38:00.003704+0000","last_clean":"2024-04-21T08:38:00.003704+0000","last_became_active":"2024-04-21T08:36:23.468601+0000","last_became_peered":"2024-04-21T08:36:23.468601+0000","last_unstale":"2024-04-21T08:38:00.003704+0000","last_undegraded":"2024-04-21T08:38:00.003704+0000","last_fullsized":"2024-04-21T08:38:00.003704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:13:11.977780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.123893+0000","last_change":"2024-04-21T08:37:15.123476+0000","last_active":"2024-04-21T08:37:15.123893+0000","last_peered":"2024-04-21T08:37:15.123893+0000","last_clean":"2024-04-21T08:37:15.123893+0000","last_became_active":"2024-04-21T08:37:15.123066+0000","last_became_peered":"2024-04-21T08:37:15.123066+0000","last_unstale":"2024-04-21T08:37:15.123893+0000","last_undegraded":"2024-04-21T08:37:15.123893+0000","last_fullsized":"2024-04-21T08:37:15.123893+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:15.786178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.115138+0000","last_change":"2024-04-21T08:36:28.014161+0000","last_active":"2024-04-21T08:37:15.115138+0000","last_peered":"2024-04-21T08:37:15.115138+0000","last_clean":"2024-04-21T08:37:15.115138+0000","last_became_active":"2024-04-21T08:36:28.013859+0000","last_became_peered":"2024-04-21T08:36:28.013859+0000","last_unstale":"2024-04-21T08:37:15.115138+0000","last_undegraded":"2024-04-21T08:37:15.115138+0000","last_fullsized":"2024-04-21T08:37:15.115138+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:08.315334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.556182+0000","last_change":"2024-04-21T08:36:23.475612+0000","last_active":"2024-04-21T08:37:15.556182+0000","last_peered":"2024-04-21T08:37:15.556182+0000","last_clean":"2024-04-21T08:37:15.556182+0000","last_became_active":"2024-04-21T08:36:23.475301+0000","last_became_peered":"2024-04-21T08:36:23.475301+0000","last_unstale":"2024-04-21T08:37:15.556182+0000","last_undegraded":"2024-04-21T08:37:15.556182+0000","last_fullsized":"2024-04-21T08:37:15.556182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:56.801630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.115131+0000","last_change":"2024-04-21T08:36:23.984689+0000","last_active":"2024-04-21T08:37:15.115131+0000","last_peered":"2024-04-21T08:37:15.115131+0000","last_clean":"2024-04-21T08:37:15.115131+0000","last_became_active":"2024-04-21T08:36:23.984358+0000","last_became_peered":"2024-04-21T08:36:23.984358+0000","last_unstale":"2024-04-21T08:37:15.115131+0000","last_undegraded":"2024-04-21T08:37:15.115131+0000","last_fullsized":"2024-04-21T08:37:15.115131+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:49:50.187466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.114110+0000","last_change":"2024-04-21T08:36:28.007951+0000","last_active":"2024-04-21T08:37:15.114110+0000","last_peered":"2024-04-21T08:37:15.114110+0000","last_clean":"2024-04-21T08:37:15.114110+0000","last_became_active":"2024-04-21T08:36:28.007553+0000","last_became_peered":"2024-04-21T08:36:28.007553+0000","last_unstale":"2024-04-21T08:37:15.114110+0000","last_undegraded":"2024-04-21T08:37:15.114110+0000","last_fullsized":"2024-04-21T08:37:15.114110+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:10:54.019605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.113691+0000","last_change":"2024-04-21T08:36:23.473371+0000","last_active":"2024-04-21T08:37:15.113691+0000","last_peered":"2024-04-21T08:37:15.113691+0000","last_clean":"2024-04-21T08:37:15.113691+0000","last_became_active":"2024-04-21T08:36:23.472472+0000","last_became_peered":"2024-04-21T08:36:23.472472+0000","last_unstale":"2024-04-21T08:37:15.113691+0000","last_undegraded":"2024-04-21T08:37:15.113691+0000","last_fullsized":"2024-04-21T08:37:15.113691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:28:49.622873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.119670+0000","last_change":"2024-04-21T08:36:23.987916+0000","last_active":"2024-04-21T08:37:15.119670+0000","last_peered":"2024-04-21T08:37:15.119670+0000","last_clean":"2024-04-21T08:37:15.119670+0000","last_became_active":"2024-04-21T08:36:23.986207+0000","last_became_peered":"2024-04-21T08:36:23.986207+0000","last_unstale":"2024-04-21T08:37:15.119670+0000","last_undegraded":"2024-04-21T08:37:15.119670+0000","last_fullsized":"2024-04-21T08:37:15.119670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:49:57.261078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.113934+0000","last_change":"2024-04-21T08:36:28.006494+0000","last_active":"2024-04-21T08:37:15.113934+0000","last_peered":"2024-04-21T08:37:15.113934+0000","last_clean":"2024-04-21T08:37:15.113934+0000","last_became_active":"2024-04-21T08:36:28.006069+0000","last_became_peered":"2024-04-21T08:36:28.006069+0000","last_unstale":"2024-04-21T08:37:15.113934+0000","last_undegraded":"2024-04-21T08:37:15.113934+0000","last_fullsized":"2024-04-21T08:37:15.113934+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:04.608295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.113186+0000","last_change":"2024-04-21T08:36:23.468923+0000","last_active":"2024-04-21T08:37:15.113186+0000","last_peered":"2024-04-21T08:37:15.113186+0000","last_clean":"2024-04-21T08:37:15.113186+0000","last_became_active":"2024-04-21T08:36:23.468509+0000","last_became_peered":"2024-04-21T08:36:23.468509+0000","last_unstale":"2024-04-21T08:37:15.113186+0000","last_undegraded":"2024-04-21T08:37:15.113186+0000","last_fullsized":"2024-04-21T08:37:15.113186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:44.452748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116926+0000","last_change":"2024-04-21T08:36:23.994423+0000","last_active":"2024-04-21T08:37:15.116926+0000","last_peered":"2024-04-21T08:37:15.116926+0000","last_clean":"2024-04-21T08:37:15.116926+0000","last_became_active":"2024-04-21T08:36:23.993827+0000","last_became_peered":"2024-04-21T08:36:23.993827+0000","last_unstale":"2024-04-21T08:37:15.116926+0000","last_undegraded":"2024-04-21T08:37:15.116926+0000","last_fullsized":"2024-04-21T08:37:15.116926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:49:25.550301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.122337+0000","last_change":"2024-04-21T08:36:28.004037+0000","last_active":"2024-04-21T08:37:15.122337+0000","last_peered":"2024-04-21T08:37:15.122337+0000","last_clean":"2024-04-21T08:37:15.122337+0000","last_became_active":"2024-04-21T08:36:28.003580+0000","last_became_peered":"2024-04-21T08:36:28.003580+0000","last_unstale":"2024-04-21T08:37:15.122337+0000","last_undegraded":"2024-04-21T08:37:15.122337+0000","last_fullsized":"2024-04-21T08:37:15.122337+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:04:51.380244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":108,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:59.991408+0000","last_change":"2024-04-21T08:36:23.468430+0000","last_active":"2024-04-21T08:37:59.991408+0000","last_peered":"2024-04-21T08:37:59.991408+0000","last_clean":"2024-04-21T08:37:59.991408+0000","last_became_active":"2024-04-21T08:36:23.468089+0000","last_became_peered":"2024-04-21T08:36:23.468089+0000","last_unstale":"2024-04-21T08:37:59.991408+0000","last_undegraded":"2024-04-21T08:37:59.991408+0000","last_fullsized":"2024-04-21T08:37:59.991408+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:12:54.827034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.117922+0000","last_change":"2024-04-21T08:37:15.117922+0000","last_active":"2024-04-21T08:37:15.117922+0000","last_peered":"2024-04-21T08:37:15.117922+0000","last_clean":"2024-04-21T08:37:15.117922+0000","last_became_active":"2024-04-21T08:37:15.117404+0000","last_became_peered":"2024-04-21T08:37:15.117404+0000","last_unstale":"2024-04-21T08:37:15.117922+0000","last_undegraded":"2024-04-21T08:37:15.117922+0000","last_fullsized":"2024-04-21T08:37:15.117922+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:39.682224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.121684+0000","last_change":"2024-04-21T08:36:28.004638+0000","last_active":"2024-04-21T08:37:15.121684+0000","last_peered":"2024-04-21T08:37:15.121684+0000","last_clean":"2024-04-21T08:37:15.121684+0000","last_became_active":"2024-04-21T08:36:28.002499+0000","last_became_peered":"2024-04-21T08:36:28.002499+0000","last_unstale":"2024-04-21T08:37:15.121684+0000","last_undegraded":"2024-04-21T08:37:15.121684+0000","last_fullsized":"2024-04-21T08:37:15.121684+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:45:46.934783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.126569+0000","last_change":"2024-04-21T08:37:15.126569+0000","last_active":"2024-04-21T08:37:15.126569+0000","last_peered":"2024-04-21T08:37:15.126569+0000","last_clean":"2024-04-21T08:37:15.126569+0000","last_became_active":"2024-04-21T08:37:15.121284+0000","last_became_peered":"2024-04-21T08:37:15.121284+0000","last_unstale":"2024-04-21T08:37:15.126569+0000","last_undegraded":"2024-04-21T08:37:15.126569+0000","last_fullsized":"2024-04-21T08:37:15.126569+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:02:26.153324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.557192+0000","last_change":"2024-04-21T08:37:15.557192+0000","last_active":"2024-04-21T08:37:15.557192+0000","last_peered":"2024-04-21T08:37:15.557192+0000","last_clean":"2024-04-21T08:37:15.557192+0000","last_became_active":"2024-04-21T08:37:15.556653+0000","last_became_peered":"2024-04-21T08:37:15.556653+0000","last_unstale":"2024-04-21T08:37:15.557192+0000","last_undegraded":"2024-04-21T08:37:15.557192+0000","last_fullsized":"2024-04-21T08:37:15.557192+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:42.722460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.218102+0000","last_change":"2024-04-21T08:36:28.017068+0000","last_active":"2024-04-21T08:37:15.218102+0000","last_peered":"2024-04-21T08:37:15.218102+0000","last_clean":"2024-04-21T08:37:15.218102+0000","last_became_active":"2024-04-21T08:36:28.016770+0000","last_became_peered":"2024-04-21T08:36:28.016770+0000","last_unstale":"2024-04-21T08:37:15.218102+0000","last_undegraded":"2024-04-21T08:37:15.218102+0000","last_fullsized":"2024-04-21T08:37:15.218102+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:07:14.596090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.119806+0000","last_change":"2024-04-21T08:36:23.470430+0000","last_active":"2024-04-21T08:37:15.119806+0000","last_peered":"2024-04-21T08:37:15.119806+0000","last_clean":"2024-04-21T08:37:15.119806+0000","last_became_active":"2024-04-21T08:36:23.470109+0000","last_became_peered":"2024-04-21T08:36:23.470109+0000","last_unstale":"2024-04-21T08:37:15.119806+0000","last_undegraded":"2024-04-21T08:37:15.119806+0000","last_fullsized":"2024-04-21T08:37:15.119806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:18.176313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.555284+0000","last_change":"2024-04-21T08:36:23.986668+0000","last_active":"2024-04-21T08:37:15.555284+0000","last_peered":"2024-04-21T08:37:15.555284+0000","last_clean":"2024-04-21T08:37:15.555284+0000","last_became_active":"2024-04-21T08:36:23.986344+0000","last_became_peered":"2024-04-21T08:36:23.986344+0000","last_unstale":"2024-04-21T08:37:15.555284+0000","last_undegraded":"2024-04-21T08:37:15.555284+0000","last_fullsized":"2024-04-21T08:37:15.555284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:56:17.744145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.115442+0000","last_change":"2024-04-21T08:36:28.009012+0000","last_active":"2024-04-21T08:37:15.115442+0000","last_peered":"2024-04-21T08:37:15.115442+0000","last_clean":"2024-04-21T08:37:15.115442+0000","last_became_active":"2024-04-21T08:36:28.008123+0000","last_became_peered":"2024-04-21T08:36:28.008123+0000","last_unstale":"2024-04-21T08:37:15.115442+0000","last_undegraded":"2024-04-21T08:37:15.115442+0000","last_fullsized":"2024-04-21T08:37:15.115442+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:33:18.881564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.117600+0000","last_change":"2024-04-21T08:36:23.467104+0000","last_active":"2024-04-21T08:37:15.117600+0000","last_peered":"2024-04-21T08:37:15.117600+0000","last_clean":"2024-04-21T08:37:15.117600+0000","last_became_active":"2024-04-21T08:36:23.466725+0000","last_became_peered":"2024-04-21T08:36:23.466725+0000","last_unstale":"2024-04-21T08:37:15.117600+0000","last_undegraded":"2024-04-21T08:37:15.117600+0000","last_fullsized":"2024-04-21T08:37:15.117600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:35:45.986448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.112600+0000","last_change":"2024-04-21T08:36:23.987579+0000","last_active":"2024-04-21T08:37:15.112600+0000","last_peered":"2024-04-21T08:37:15.112600+0000","last_clean":"2024-04-21T08:37:15.112600+0000","last_became_active":"2024-04-21T08:36:23.987044+0000","last_became_peered":"2024-04-21T08:36:23.987044+0000","last_unstale":"2024-04-21T08:37:15.112600+0000","last_undegraded":"2024-04-21T08:37:15.112600+0000","last_fullsized":"2024-04-21T08:37:15.112600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:48:27.593090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.555280+0000","last_change":"2024-04-21T08:36:28.006417+0000","last_active":"2024-04-21T08:37:15.555280+0000","last_peered":"2024-04-21T08:37:15.555280+0000","last_clean":"2024-04-21T08:37:15.555280+0000","last_became_active":"2024-04-21T08:36:28.006006+0000","last_became_peered":"2024-04-21T08:36:28.006006+0000","last_unstale":"2024-04-21T08:37:15.555280+0000","last_undegraded":"2024-04-21T08:37:15.555280+0000","last_fullsized":"2024-04-21T08:37:15.555280+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:16.173284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":175,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:38:01.993791+0000","last_change":"2024-04-21T08:36:23.471541+0000","last_active":"2024-04-21T08:38:01.993791+0000","last_peered":"2024-04-21T08:38:01.993791+0000","last_clean":"2024-04-21T08:38:01.993791+0000","last_became_active":"2024-04-21T08:36:23.471193+0000","last_became_peered":"2024-04-21T08:36:23.471193+0000","last_unstale":"2024-04-21T08:38:01.993791+0000","last_undegraded":"2024-04-21T08:38:01.993791+0000","last_fullsized":"2024-04-21T08:38:01.993791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:06:04.916707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":144,"num_read_kb":144,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.120272+0000","last_change":"2024-04-21T08:36:23.986781+0000","last_active":"2024-04-21T08:37:15.120272+0000","last_peered":"2024-04-21T08:37:15.120272+0000","last_clean":"2024-04-21T08:37:15.120272+0000","last_became_active":"2024-04-21T08:36:23.985696+0000","last_became_peered":"2024-04-21T08:36:23.985696+0000","last_unstale":"2024-04-21T08:37:15.120272+0000","last_undegraded":"2024-04-21T08:37:15.120272+0000","last_fullsized":"2024-04-21T08:37:15.120272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:40.556642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.112290+0000","last_change":"2024-04-21T08:36:28.011621+0000","last_active":"2024-04-21T08:37:15.112290+0000","last_peered":"2024-04-21T08:37:15.112290+0000","last_clean":"2024-04-21T08:37:15.112290+0000","last_became_active":"2024-04-21T08:36:28.011136+0000","last_became_peered":"2024-04-21T08:36:28.011136+0000","last_unstale":"2024-04-21T08:37:15.112290+0000","last_undegraded":"2024-04-21T08:37:15.112290+0000","last_fullsized":"2024-04-21T08:37:15.112290+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:47.623445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.115712+0000","last_change":"2024-04-21T08:36:23.474019+0000","last_active":"2024-04-21T08:37:15.115712+0000","last_peered":"2024-04-21T08:37:15.115712+0000","last_clean":"2024-04-21T08:37:15.115712+0000","last_became_active":"2024-04-21T08:36:23.473690+0000","last_became_peered":"2024-04-21T08:36:23.473690+0000","last_unstale":"2024-04-21T08:37:15.115712+0000","last_undegraded":"2024-04-21T08:37:15.115712+0000","last_fullsized":"2024-04-21T08:37:15.115712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:38:32.913795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.121455+0000","last_change":"2024-04-21T08:37:15.121455+0000","last_active":"2024-04-21T08:37:15.121455+0000","last_peered":"2024-04-21T08:37:15.121455+0000","last_clean":"2024-04-21T08:37:15.121455+0000","last_became_active":"2024-04-21T08:37:15.121113+0000","last_became_peered":"2024-04-21T08:37:15.121113+0000","last_unstale":"2024-04-21T08:37:15.121455+0000","last_undegraded":"2024-04-21T08:37:15.121455+0000","last_fullsized":"2024-04-21T08:37:15.121455+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:39:36.525495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.114358+0000","last_change":"2024-04-21T08:36:28.004420+0000","last_active":"2024-04-21T08:37:15.114358+0000","last_peered":"2024-04-21T08:37:15.114358+0000","last_clean":"2024-04-21T08:37:15.114358+0000","last_became_active":"2024-04-21T08:36:28.003935+0000","last_became_peered":"2024-04-21T08:36:28.003935+0000","last_unstale":"2024-04-21T08:37:15.114358+0000","last_undegraded":"2024-04-21T08:37:15.114358+0000","last_fullsized":"2024-04-21T08:37:15.114358+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:10:03.255869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.117190+0000","last_change":"2024-04-21T08:36:23.465825+0000","last_active":"2024-04-21T08:37:15.117190+0000","last_peered":"2024-04-21T08:37:15.117190+0000","last_clean":"2024-04-21T08:37:15.117190+0000","last_became_active":"2024-04-21T08:36:23.465525+0000","last_became_peered":"2024-04-21T08:36:23.465525+0000","last_unstale":"2024-04-21T08:37:15.117190+0000","last_undegraded":"2024-04-21T08:37:15.117190+0000","last_fullsized":"2024-04-21T08:37:15.117190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:59:50.502210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.119858+0000","last_change":"2024-04-21T08:36:23.987602+0000","last_active":"2024-04-21T08:37:15.119858+0000","last_peered":"2024-04-21T08:37:15.119858+0000","last_clean":"2024-04-21T08:37:15.119858+0000","last_became_active":"2024-04-21T08:36:23.987298+0000","last_became_peered":"2024-04-21T08:36:23.987298+0000","last_unstale":"2024-04-21T08:37:15.119858+0000","last_undegraded":"2024-04-21T08:37:15.119858+0000","last_fullsized":"2024-04-21T08:37:15.119858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:06.647676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.112441+0000","last_change":"2024-04-21T08:36:28.005935+0000","last_active":"2024-04-21T08:37:15.112441+0000","last_peered":"2024-04-21T08:37:15.112441+0000","last_clean":"2024-04-21T08:37:15.112441+0000","last_became_active":"2024-04-21T08:36:28.005512+0000","last_became_peered":"2024-04-21T08:36:28.005512+0000","last_unstale":"2024-04-21T08:37:15.112441+0000","last_undegraded":"2024-04-21T08:37:15.112441+0000","last_fullsized":"2024-04-21T08:37:15.112441+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:23:20.298524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'35","reported_seq":216,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:38:01.997949+0000","last_change":"2024-04-21T08:36:23.467728+0000","last_active":"2024-04-21T08:38:01.997949+0000","last_peered":"2024-04-21T08:38:01.997949+0000","last_clean":"2024-04-21T08:38:01.997949+0000","last_became_active":"2024-04-21T08:36:23.467430+0000","last_became_peered":"2024-04-21T08:36:23.467430+0000","last_unstale":"2024-04-21T08:38:01.997949+0000","last_undegraded":"2024-04-21T08:38:01.997949+0000","last_fullsized":"2024-04-21T08:38:01.997949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:47:18.900911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63356,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":154,"num_read_kb":139,"num_write":37,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.113320+0000","last_change":"2024-04-21T08:36:23.989116+0000","last_active":"2024-04-21T08:37:15.113320+0000","last_peered":"2024-04-21T08:37:15.113320+0000","last_clean":"2024-04-21T08:37:15.113320+0000","last_became_active":"2024-04-21T08:36:23.988758+0000","last_became_peered":"2024-04-21T08:36:23.988758+0000","last_unstale":"2024-04-21T08:37:15.113320+0000","last_undegraded":"2024-04-21T08:37:15.113320+0000","last_fullsized":"2024-04-21T08:37:15.113320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:56:25.969905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.117421+0000","last_change":"2024-04-21T08:36:28.007370+0000","last_active":"2024-04-21T08:37:15.117421+0000","last_peered":"2024-04-21T08:37:15.117421+0000","last_clean":"2024-04-21T08:37:15.117421+0000","last_became_active":"2024-04-21T08:36:28.006972+0000","last_became_peered":"2024-04-21T08:36:28.006972+0000","last_unstale":"2024-04-21T08:37:15.117421+0000","last_undegraded":"2024-04-21T08:37:15.117421+0000","last_fullsized":"2024-04-21T08:37:15.117421+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:18:14.926058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.121864+0000","last_change":"2024-04-21T08:36:23.472658+0000","last_active":"2024-04-21T08:37:15.121864+0000","last_peered":"2024-04-21T08:37:15.121864+0000","last_clean":"2024-04-21T08:37:15.121864+0000","last_became_active":"2024-04-21T08:36:23.472318+0000","last_became_peered":"2024-04-21T08:36:23.472318+0000","last_unstale":"2024-04-21T08:37:15.121864+0000","last_undegraded":"2024-04-21T08:37:15.121864+0000","last_fullsized":"2024-04-21T08:37:15.121864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:28:27.497605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.120626+0000","last_change":"2024-04-21T08:36:23.986897+0000","last_active":"2024-04-21T08:37:15.120626+0000","last_peered":"2024-04-21T08:37:15.120626+0000","last_clean":"2024-04-21T08:37:15.120626+0000","last_became_active":"2024-04-21T08:36:23.986554+0000","last_became_peered":"2024-04-21T08:36:23.986554+0000","last_unstale":"2024-04-21T08:37:15.120626+0000","last_undegraded":"2024-04-21T08:37:15.120626+0000","last_fullsized":"2024-04-21T08:37:15.120626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:44:22.331207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.218065+0000","last_change":"2024-04-21T08:36:28.019186+0000","last_active":"2024-04-21T08:37:15.218065+0000","last_peered":"2024-04-21T08:37:15.218065+0000","last_clean":"2024-04-21T08:37:15.218065+0000","last_became_active":"2024-04-21T08:36:28.018774+0000","last_became_peered":"2024-04-21T08:36:28.018774+0000","last_unstale":"2024-04-21T08:37:15.218065+0000","last_undegraded":"2024-04-21T08:37:15.218065+0000","last_fullsized":"2024-04-21T08:37:15.218065+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:35:33.548656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.117065+0000","last_change":"2024-04-21T08:36:23.472633+0000","last_active":"2024-04-21T08:37:15.117065+0000","last_peered":"2024-04-21T08:37:15.117065+0000","last_clean":"2024-04-21T08:37:15.117065+0000","last_became_active":"2024-04-21T08:36:23.472336+0000","last_became_peered":"2024-04-21T08:36:23.472336+0000","last_unstale":"2024-04-21T08:37:15.117065+0000","last_undegraded":"2024-04-21T08:37:15.117065+0000","last_fullsized":"2024-04-21T08:37:15.117065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:49.463492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.115208+0000","last_change":"2024-04-21T08:36:23.994332+0000","last_active":"2024-04-21T08:37:15.115208+0000","last_peered":"2024-04-21T08:37:15.115208+0000","last_clean":"2024-04-21T08:37:15.115208+0000","last_became_active":"2024-04-21T08:36:23.993936+0000","last_became_peered":"2024-04-21T08:36:23.993936+0000","last_unstale":"2024-04-21T08:37:15.115208+0000","last_undegraded":"2024-04-21T08:37:15.115208+0000","last_fullsized":"2024-04-21T08:37:15.115208+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:26:35.330559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.113463+0000","last_change":"2024-04-21T08:36:28.012549+0000","last_active":"2024-04-21T08:37:15.113463+0000","last_peered":"2024-04-21T08:37:15.113463+0000","last_clean":"2024-04-21T08:37:15.113463+0000","last_became_active":"2024-04-21T08:36:28.012151+0000","last_became_peered":"2024-04-21T08:36:28.012151+0000","last_unstale":"2024-04-21T08:37:15.113463+0000","last_undegraded":"2024-04-21T08:37:15.113463+0000","last_fullsized":"2024-04-21T08:37:15.113463+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:55.680399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":111,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:38:03.000284+0000","last_change":"2024-04-21T08:36:23.464586+0000","last_active":"2024-04-21T08:38:03.000284+0000","last_peered":"2024-04-21T08:38:03.000284+0000","last_clean":"2024-04-21T08:38:03.000284+0000","last_became_active":"2024-04-21T08:36:23.464281+0000","last_became_peered":"2024-04-21T08:36:23.464281+0000","last_unstale":"2024-04-21T08:38:03.000284+0000","last_undegraded":"2024-04-21T08:38:03.000284+0000","last_fullsized":"2024-04-21T08:38:03.000284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:04:58.577283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116093+0000","last_change":"2024-04-21T08:36:23.989604+0000","last_active":"2024-04-21T08:37:15.116093+0000","last_peered":"2024-04-21T08:37:15.116093+0000","last_clean":"2024-04-21T08:37:15.116093+0000","last_became_active":"2024-04-21T08:36:23.989257+0000","last_became_peered":"2024-04-21T08:36:23.989257+0000","last_unstale":"2024-04-21T08:37:15.116093+0000","last_undegraded":"2024-04-21T08:37:15.116093+0000","last_fullsized":"2024-04-21T08:37:15.116093+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:45.077943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.120255+0000","last_change":"2024-04-21T08:36:28.007959+0000","last_active":"2024-04-21T08:37:15.120255+0000","last_peered":"2024-04-21T08:37:15.120255+0000","last_clean":"2024-04-21T08:37:15.120255+0000","last_became_active":"2024-04-21T08:36:28.007571+0000","last_became_peered":"2024-04-21T08:36:28.007571+0000","last_unstale":"2024-04-21T08:37:15.120255+0000","last_undegraded":"2024-04-21T08:37:15.120255+0000","last_fullsized":"2024-04-21T08:37:15.120255+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:17:34.007013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.112793+0000","last_change":"2024-04-21T08:36:23.475054+0000","last_active":"2024-04-21T08:37:15.112793+0000","last_peered":"2024-04-21T08:37:15.112793+0000","last_clean":"2024-04-21T08:37:15.112793+0000","last_became_active":"2024-04-21T08:36:23.474606+0000","last_became_peered":"2024-04-21T08:36:23.474606+0000","last_unstale":"2024-04-21T08:37:15.112793+0000","last_undegraded":"2024-04-21T08:37:15.112793+0000","last_fullsized":"2024-04-21T08:37:15.112793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:22.559174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116692+0000","last_change":"2024-04-21T08:36:23.995980+0000","last_active":"2024-04-21T08:37:15.116692+0000","last_peered":"2024-04-21T08:37:15.116692+0000","last_clean":"2024-04-21T08:37:15.116692+0000","last_became_active":"2024-04-21T08:36:23.995673+0000","last_became_peered":"2024-04-21T08:36:23.995673+0000","last_unstale":"2024-04-21T08:37:15.116692+0000","last_undegraded":"2024-04-21T08:37:15.116692+0000","last_fullsized":"2024-04-21T08:37:15.116692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:11:49.884384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116929+0000","last_change":"2024-04-21T08:36:28.018792+0000","last_active":"2024-04-21T08:37:15.116929+0000","last_peered":"2024-04-21T08:37:15.116929+0000","last_clean":"2024-04-21T08:37:15.116929+0000","last_became_active":"2024-04-21T08:36:28.018373+0000","last_became_peered":"2024-04-21T08:36:28.018373+0000","last_unstale":"2024-04-21T08:37:15.116929+0000","last_undegraded":"2024-04-21T08:37:15.116929+0000","last_fullsized":"2024-04-21T08:37:15.116929+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:11.857661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.113413+0000","last_change":"2024-04-21T08:36:23.469787+0000","last_active":"2024-04-21T08:37:15.113413+0000","last_peered":"2024-04-21T08:37:15.113413+0000","last_clean":"2024-04-21T08:37:15.113413+0000","last_became_active":"2024-04-21T08:36:23.469462+0000","last_became_peered":"2024-04-21T08:36:23.469462+0000","last_unstale":"2024-04-21T08:37:15.113413+0000","last_undegraded":"2024-04-21T08:37:15.113413+0000","last_fullsized":"2024-04-21T08:37:15.113413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:33:25.849528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.218022+0000","last_change":"2024-04-21T08:36:23.988492+0000","last_active":"2024-04-21T08:37:15.218022+0000","last_peered":"2024-04-21T08:37:15.218022+0000","last_clean":"2024-04-21T08:37:15.218022+0000","last_became_active":"2024-04-21T08:36:23.988133+0000","last_became_peered":"2024-04-21T08:36:23.988133+0000","last_unstale":"2024-04-21T08:37:15.218022+0000","last_undegraded":"2024-04-21T08:37:15.218022+0000","last_fullsized":"2024-04-21T08:37:15.218022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:01:25.320281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.113025+0000","last_change":"2024-04-21T08:36:28.012454+0000","last_active":"2024-04-21T08:37:15.113025+0000","last_peered":"2024-04-21T08:37:15.113025+0000","last_clean":"2024-04-21T08:37:15.113025+0000","last_became_active":"2024-04-21T08:36:28.011985+0000","last_became_peered":"2024-04-21T08:36:28.011985+0000","last_unstale":"2024-04-21T08:37:15.113025+0000","last_undegraded":"2024-04-21T08:37:15.113025+0000","last_fullsized":"2024-04-21T08:37:15.113025+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:16:51.939691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.219428+0000","last_change":"2024-04-21T08:37:15.219428+0000","last_active":"2024-04-21T08:37:15.219428+0000","last_peered":"2024-04-21T08:37:15.219428+0000","last_clean":"2024-04-21T08:37:15.219428+0000","last_became_active":"2024-04-21T08:37:15.218923+0000","last_became_peered":"2024-04-21T08:37:15.218923+0000","last_unstale":"2024-04-21T08:37:15.219428+0000","last_undegraded":"2024-04-21T08:37:15.219428+0000","last_fullsized":"2024-04-21T08:37:15.219428+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:07:02.781701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.117495+0000","last_change":"2024-04-21T08:36:23.985910+0000","last_active":"2024-04-21T08:37:15.117495+0000","last_peered":"2024-04-21T08:37:15.117495+0000","last_clean":"2024-04-21T08:37:15.117495+0000","last_became_active":"2024-04-21T08:36:23.985605+0000","last_became_peered":"2024-04-21T08:36:23.985605+0000","last_unstale":"2024-04-21T08:37:15.117495+0000","last_undegraded":"2024-04-21T08:37:15.117495+0000","last_fullsized":"2024-04-21T08:37:15.117495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:37:09.367005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.120042+0000","last_change":"2024-04-21T08:36:28.015256+0000","last_active":"2024-04-21T08:37:15.120042+0000","last_peered":"2024-04-21T08:37:15.120042+0000","last_clean":"2024-04-21T08:37:15.120042+0000","last_became_active":"2024-04-21T08:36:28.014849+0000","last_became_peered":"2024-04-21T08:36:28.014849+0000","last_unstale":"2024-04-21T08:37:15.120042+0000","last_undegraded":"2024-04-21T08:37:15.120042+0000","last_fullsized":"2024-04-21T08:37:15.120042+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:22:05.466299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":105,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:38:02.007150+0000","last_change":"2024-04-21T08:36:23.474966+0000","last_active":"2024-04-21T08:38:02.007150+0000","last_peered":"2024-04-21T08:38:02.007150+0000","last_clean":"2024-04-21T08:38:02.007150+0000","last_became_active":"2024-04-21T08:36:23.474374+0000","last_became_peered":"2024-04-21T08:36:23.474374+0000","last_unstale":"2024-04-21T08:38:02.007150+0000","last_undegraded":"2024-04-21T08:38:02.007150+0000","last_fullsized":"2024-04-21T08:38:02.007150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:57:20.237966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.118887+0000","last_change":"2024-04-21T08:37:15.118887+0000","last_active":"2024-04-21T08:37:15.118887+0000","last_peered":"2024-04-21T08:37:15.118887+0000","last_clean":"2024-04-21T08:37:15.118887+0000","last_became_active":"2024-04-21T08:37:15.118510+0000","last_became_peered":"2024-04-21T08:37:15.118510+0000","last_unstale":"2024-04-21T08:37:15.118887+0000","last_undegraded":"2024-04-21T08:37:15.118887+0000","last_fullsized":"2024-04-21T08:37:15.118887+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:59:27.209083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.120380+0000","last_change":"2024-04-21T08:36:28.013169+0000","last_active":"2024-04-21T08:37:15.120380+0000","last_peered":"2024-04-21T08:37:15.120380+0000","last_clean":"2024-04-21T08:37:15.120380+0000","last_became_active":"2024-04-21T08:36:28.012791+0000","last_became_peered":"2024-04-21T08:36:28.012791+0000","last_unstale":"2024-04-21T08:37:15.120380+0000","last_undegraded":"2024-04-21T08:37:15.120380+0000","last_fullsized":"2024-04-21T08:37:15.120380+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:42:56.063159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116469+0000","last_change":"2024-04-21T08:36:23.995209+0000","last_active":"2024-04-21T08:37:15.116469+0000","last_peered":"2024-04-21T08:37:15.116469+0000","last_clean":"2024-04-21T08:37:15.116469+0000","last_became_active":"2024-04-21T08:36:23.994866+0000","last_became_peered":"2024-04-21T08:36:23.994866+0000","last_unstale":"2024-04-21T08:37:15.116469+0000","last_undegraded":"2024-04-21T08:37:15.116469+0000","last_fullsized":"2024-04-21T08:37:15.116469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:33:45.582150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":107,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:38:01.998246+0000","last_change":"2024-04-21T08:36:23.471474+0000","last_active":"2024-04-21T08:38:01.998246+0000","last_peered":"2024-04-21T08:38:01.998246+0000","last_clean":"2024-04-21T08:38:01.998246+0000","last_became_active":"2024-04-21T08:36:23.471179+0000","last_became_peered":"2024-04-21T08:36:23.471179+0000","last_unstale":"2024-04-21T08:38:01.998246+0000","last_undegraded":"2024-04-21T08:38:01.998246+0000","last_fullsized":"2024-04-21T08:38:01.998246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:17.290051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116176+0000","last_change":"2024-04-21T08:36:28.009671+0000","last_active":"2024-04-21T08:37:15.116176+0000","last_peered":"2024-04-21T08:37:15.116176+0000","last_clean":"2024-04-21T08:37:15.116176+0000","last_became_active":"2024-04-21T08:36:28.009185+0000","last_became_peered":"2024-04-21T08:36:28.009185+0000","last_unstale":"2024-04-21T08:37:15.116176+0000","last_undegraded":"2024-04-21T08:37:15.116176+0000","last_fullsized":"2024-04-21T08:37:15.116176+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:11:05.156271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.555286+0000","last_change":"2024-04-21T08:36:23.990272+0000","last_active":"2024-04-21T08:37:15.555286+0000","last_peered":"2024-04-21T08:37:15.555286+0000","last_clean":"2024-04-21T08:37:15.555286+0000","last_became_active":"2024-04-21T08:36:23.989940+0000","last_became_peered":"2024-04-21T08:36:23.989940+0000","last_unstale":"2024-04-21T08:37:15.555286+0000","last_undegraded":"2024-04-21T08:37:15.555286+0000","last_fullsized":"2024-04-21T08:37:15.555286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:27:50.705167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.114241+0000","last_change":"2024-04-21T08:36:23.474162+0000","last_active":"2024-04-21T08:37:15.114241+0000","last_peered":"2024-04-21T08:37:15.114241+0000","last_clean":"2024-04-21T08:37:15.114241+0000","last_became_active":"2024-04-21T08:36:23.473802+0000","last_became_peered":"2024-04-21T08:36:23.473802+0000","last_unstale":"2024-04-21T08:37:15.114241+0000","last_undegraded":"2024-04-21T08:37:15.114241+0000","last_fullsized":"2024-04-21T08:37:15.114241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:32:41.918431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":360,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116011+0000","last_change":"2024-04-21T08:35:10.062130+0000","last_active":"2024-04-21T08:37:15.116011+0000","last_peered":"2024-04-21T08:37:15.116011+0000","last_clean":"2024-04-21T08:37:15.116011+0000","last_became_active":"2024-04-21T08:35:10.061727+0000","last_became_peered":"2024-04-21T08:35:10.061727+0000","last_unstale":"2024-04-21T08:37:15.116011+0000","last_undegraded":"2024-04-21T08:37:15.116011+0000","last_fullsized":"2024-04-21T08:37:15.116011+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:31:40.950939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:31:40.950939+0000","last_clean_scrub_stamp":"2024-04-21T08:31:40.950939+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:35:52.360167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116481+0000","last_change":"2024-04-21T08:36:28.012840+0000","last_active":"2024-04-21T08:37:15.116481+0000","last_peered":"2024-04-21T08:37:15.116481+0000","last_clean":"2024-04-21T08:37:15.116481+0000","last_became_active":"2024-04-21T08:36:28.012439+0000","last_became_peered":"2024-04-21T08:36:28.012439+0000","last_unstale":"2024-04-21T08:37:15.116481+0000","last_undegraded":"2024-04-21T08:37:15.116481+0000","last_fullsized":"2024-04-21T08:37:15.116481+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:36:39.260061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.119658+0000","last_change":"2024-04-21T08:36:23.984332+0000","last_active":"2024-04-21T08:37:15.119658+0000","last_peered":"2024-04-21T08:37:15.119658+0000","last_clean":"2024-04-21T08:37:15.119658+0000","last_became_active":"2024-04-21T08:36:23.984021+0000","last_became_peered":"2024-04-21T08:36:23.984021+0000","last_unstale":"2024-04-21T08:37:15.119658+0000","last_undegraded":"2024-04-21T08:37:15.119658+0000","last_fullsized":"2024-04-21T08:37:15.119658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:05:21.090633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.115990+0000","last_change":"2024-04-21T08:36:23.467522+0000","last_active":"2024-04-21T08:37:15.115990+0000","last_peered":"2024-04-21T08:37:15.115990+0000","last_clean":"2024-04-21T08:37:15.115990+0000","last_became_active":"2024-04-21T08:36:23.466379+0000","last_became_peered":"2024-04-21T08:36:23.466379+0000","last_unstale":"2024-04-21T08:37:15.115990+0000","last_undegraded":"2024-04-21T08:37:15.115990+0000","last_fullsized":"2024-04-21T08:37:15.115990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:04.185314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.217837+0000","last_change":"2024-04-21T08:36:28.015120+0000","last_active":"2024-04-21T08:37:15.217837+0000","last_peered":"2024-04-21T08:37:15.217837+0000","last_clean":"2024-04-21T08:37:15.217837+0000","last_became_active":"2024-04-21T08:36:28.014701+0000","last_became_peered":"2024-04-21T08:36:28.014701+0000","last_unstale":"2024-04-21T08:37:15.217837+0000","last_undegraded":"2024-04-21T08:37:15.217837+0000","last_fullsized":"2024-04-21T08:37:15.217837+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:23.127622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.113440+0000","last_change":"2024-04-21T08:36:23.475335+0000","last_active":"2024-04-21T08:37:15.113440+0000","last_peered":"2024-04-21T08:37:15.113440+0000","last_clean":"2024-04-21T08:37:15.113440+0000","last_became_active":"2024-04-21T08:36:23.474739+0000","last_became_peered":"2024-04-21T08:36:23.474739+0000","last_unstale":"2024-04-21T08:37:15.113440+0000","last_undegraded":"2024-04-21T08:37:15.113440+0000","last_fullsized":"2024-04-21T08:37:15.113440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:36.119649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.120403+0000","last_change":"2024-04-21T08:36:23.977229+0000","last_active":"2024-04-21T08:37:15.120403+0000","last_peered":"2024-04-21T08:37:15.120403+0000","last_clean":"2024-04-21T08:37:15.120403+0000","last_became_active":"2024-04-21T08:36:23.976893+0000","last_became_peered":"2024-04-21T08:36:23.976893+0000","last_unstale":"2024-04-21T08:37:15.120403+0000","last_undegraded":"2024-04-21T08:37:15.120403+0000","last_fullsized":"2024-04-21T08:37:15.120403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:31:54.226274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.555157+0000","last_change":"2024-04-21T08:36:28.018581+0000","last_active":"2024-04-21T08:37:15.555157+0000","last_peered":"2024-04-21T08:37:15.555157+0000","last_clean":"2024-04-21T08:37:15.555157+0000","last_became_active":"2024-04-21T08:36:28.018021+0000","last_became_peered":"2024-04-21T08:36:28.018021+0000","last_unstale":"2024-04-21T08:37:15.555157+0000","last_undegraded":"2024-04-21T08:37:15.555157+0000","last_fullsized":"2024-04-21T08:37:15.555157+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:29:00.355532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.113645+0000","last_change":"2024-04-21T08:36:23.473339+0000","last_active":"2024-04-21T08:37:15.113645+0000","last_peered":"2024-04-21T08:37:15.113645+0000","last_clean":"2024-04-21T08:37:15.113645+0000","last_became_active":"2024-04-21T08:36:23.472994+0000","last_became_peered":"2024-04-21T08:36:23.472994+0000","last_unstale":"2024-04-21T08:37:15.113645+0000","last_undegraded":"2024-04-21T08:37:15.113645+0000","last_fullsized":"2024-04-21T08:37:15.113645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:42.762273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.555861+0000","last_change":"2024-04-21T08:36:23.987354+0000","last_active":"2024-04-21T08:37:15.555861+0000","last_peered":"2024-04-21T08:37:15.555861+0000","last_clean":"2024-04-21T08:37:15.555861+0000","last_became_active":"2024-04-21T08:36:23.987032+0000","last_became_peered":"2024-04-21T08:36:23.987032+0000","last_unstale":"2024-04-21T08:37:15.555861+0000","last_undegraded":"2024-04-21T08:37:15.555861+0000","last_fullsized":"2024-04-21T08:37:15.555861+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:01:05.164367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.121970+0000","last_change":"2024-04-21T08:36:28.002582+0000","last_active":"2024-04-21T08:37:15.121970+0000","last_peered":"2024-04-21T08:37:15.121970+0000","last_clean":"2024-04-21T08:37:15.121970+0000","last_became_active":"2024-04-21T08:36:28.002133+0000","last_became_peered":"2024-04-21T08:36:28.002133+0000","last_unstale":"2024-04-21T08:37:15.121970+0000","last_undegraded":"2024-04-21T08:37:15.121970+0000","last_fullsized":"2024-04-21T08:37:15.121970+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:30:00.268015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.114867+0000","last_change":"2024-04-21T08:36:23.462840+0000","last_active":"2024-04-21T08:37:15.114867+0000","last_peered":"2024-04-21T08:37:15.114867+0000","last_clean":"2024-04-21T08:37:15.114867+0000","last_became_active":"2024-04-21T08:36:23.462391+0000","last_became_peered":"2024-04-21T08:36:23.462391+0000","last_unstale":"2024-04-21T08:37:15.114867+0000","last_undegraded":"2024-04-21T08:37:15.114867+0000","last_fullsized":"2024-04-21T08:37:15.114867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:33.230388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.115561+0000","last_change":"2024-04-21T08:36:23.994751+0000","last_active":"2024-04-21T08:37:15.115561+0000","last_peered":"2024-04-21T08:37:15.115561+0000","last_clean":"2024-04-21T08:37:15.115561+0000","last_became_active":"2024-04-21T08:36:23.994384+0000","last_became_peered":"2024-04-21T08:36:23.994384+0000","last_unstale":"2024-04-21T08:37:15.115561+0000","last_undegraded":"2024-04-21T08:37:15.115561+0000","last_fullsized":"2024-04-21T08:37:15.115561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:18.956893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.114299+0000","last_change":"2024-04-21T08:36:28.004004+0000","last_active":"2024-04-21T08:37:15.114299+0000","last_peered":"2024-04-21T08:37:15.114299+0000","last_clean":"2024-04-21T08:37:15.114299+0000","last_became_active":"2024-04-21T08:36:28.003547+0000","last_became_peered":"2024-04-21T08:36:28.003547+0000","last_unstale":"2024-04-21T08:37:15.114299+0000","last_undegraded":"2024-04-21T08:37:15.114299+0000","last_fullsized":"2024-04-21T08:37:15.114299+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:21:40.215428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.222118+0000","last_change":"2024-04-21T08:37:15.222118+0000","last_active":"2024-04-21T08:37:15.222118+0000","last_peered":"2024-04-21T08:37:15.222118+0000","last_clean":"2024-04-21T08:37:15.222118+0000","last_became_active":"2024-04-21T08:37:15.219173+0000","last_became_peered":"2024-04-21T08:37:15.219173+0000","last_unstale":"2024-04-21T08:37:15.222118+0000","last_undegraded":"2024-04-21T08:37:15.222118+0000","last_fullsized":"2024-04-21T08:37:15.222118+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:12.931076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.112796+0000","last_change":"2024-04-21T08:36:23.988020+0000","last_active":"2024-04-21T08:37:15.112796+0000","last_peered":"2024-04-21T08:37:15.112796+0000","last_clean":"2024-04-21T08:37:15.112796+0000","last_became_active":"2024-04-21T08:36:23.987630+0000","last_became_peered":"2024-04-21T08:36:23.987630+0000","last_unstale":"2024-04-21T08:37:15.112796+0000","last_undegraded":"2024-04-21T08:37:15.112796+0000","last_fullsized":"2024-04-21T08:37:15.112796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:06.519001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.113936+0000","last_change":"2024-04-21T08:36:28.006984+0000","last_active":"2024-04-21T08:37:15.113936+0000","last_peered":"2024-04-21T08:37:15.113936+0000","last_clean":"2024-04-21T08:37:15.113936+0000","last_became_active":"2024-04-21T08:36:28.006548+0000","last_became_peered":"2024-04-21T08:36:28.006548+0000","last_unstale":"2024-04-21T08:37:15.113936+0000","last_undegraded":"2024-04-21T08:37:15.113936+0000","last_fullsized":"2024-04-21T08:37:15.113936+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:25:50.870658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.555439+0000","last_change":"2024-04-21T08:36:23.476266+0000","last_active":"2024-04-21T08:37:15.555439+0000","last_peered":"2024-04-21T08:37:15.555439+0000","last_clean":"2024-04-21T08:37:15.555439+0000","last_became_active":"2024-04-21T08:36:23.475948+0000","last_became_peered":"2024-04-21T08:36:23.475948+0000","last_unstale":"2024-04-21T08:37:15.555439+0000","last_undegraded":"2024-04-21T08:37:15.555439+0000","last_fullsized":"2024-04-21T08:37:15.555439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:41.529376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.218219+0000","last_change":"2024-04-21T08:36:23.981942+0000","last_active":"2024-04-21T08:37:15.218219+0000","last_peered":"2024-04-21T08:37:15.218219+0000","last_clean":"2024-04-21T08:37:15.218219+0000","last_became_active":"2024-04-21T08:36:23.981643+0000","last_became_peered":"2024-04-21T08:36:23.981643+0000","last_unstale":"2024-04-21T08:37:15.218219+0000","last_undegraded":"2024-04-21T08:37:15.218219+0000","last_fullsized":"2024-04-21T08:37:15.218219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:26:24.722695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.114243+0000","last_change":"2024-04-21T08:36:28.006980+0000","last_active":"2024-04-21T08:37:15.114243+0000","last_peered":"2024-04-21T08:37:15.114243+0000","last_clean":"2024-04-21T08:37:15.114243+0000","last_became_active":"2024-04-21T08:36:28.006530+0000","last_became_peered":"2024-04-21T08:36:28.006530+0000","last_unstale":"2024-04-21T08:37:15.114243+0000","last_undegraded":"2024-04-21T08:37:15.114243+0000","last_fullsized":"2024-04-21T08:37:15.114243+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:05:57.465612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.121956+0000","last_change":"2024-04-21T08:36:23.473253+0000","last_active":"2024-04-21T08:37:15.121956+0000","last_peered":"2024-04-21T08:37:15.121956+0000","last_clean":"2024-04-21T08:37:15.121956+0000","last_became_active":"2024-04-21T08:36:23.472942+0000","last_became_peered":"2024-04-21T08:36:23.472942+0000","last_unstale":"2024-04-21T08:37:15.121956+0000","last_undegraded":"2024-04-21T08:37:15.121956+0000","last_fullsized":"2024-04-21T08:37:15.121956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:32:04.894388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.555573+0000","last_change":"2024-04-21T08:36:23.988355+0000","last_active":"2024-04-21T08:37:15.555573+0000","last_peered":"2024-04-21T08:37:15.555573+0000","last_clean":"2024-04-21T08:37:15.555573+0000","last_became_active":"2024-04-21T08:36:23.988049+0000","last_became_peered":"2024-04-21T08:36:23.988049+0000","last_unstale":"2024-04-21T08:37:15.555573+0000","last_undegraded":"2024-04-21T08:37:15.555573+0000","last_fullsized":"2024-04-21T08:37:15.555573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:57.348698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.115166+0000","last_change":"2024-04-21T08:36:28.007343+0000","last_active":"2024-04-21T08:37:15.115166+0000","last_peered":"2024-04-21T08:37:15.115166+0000","last_clean":"2024-04-21T08:37:15.115166+0000","last_became_active":"2024-04-21T08:36:28.006719+0000","last_became_peered":"2024-04-21T08:36:28.006719+0000","last_unstale":"2024-04-21T08:37:15.115166+0000","last_undegraded":"2024-04-21T08:37:15.115166+0000","last_fullsized":"2024-04-21T08:37:15.115166+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:02:25.734584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.112987+0000","last_change":"2024-04-21T08:36:23.472416+0000","last_active":"2024-04-21T08:37:15.112987+0000","last_peered":"2024-04-21T08:37:15.112987+0000","last_clean":"2024-04-21T08:37:15.112987+0000","last_became_active":"2024-04-21T08:36:23.471500+0000","last_became_peered":"2024-04-21T08:36:23.471500+0000","last_unstale":"2024-04-21T08:37:15.112987+0000","last_undegraded":"2024-04-21T08:37:15.112987+0000","last_fullsized":"2024-04-21T08:37:15.112987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:39.706432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.115081+0000","last_change":"2024-04-21T08:36:23.990653+0000","last_active":"2024-04-21T08:37:15.115081+0000","last_peered":"2024-04-21T08:37:15.115081+0000","last_clean":"2024-04-21T08:37:15.115081+0000","last_became_active":"2024-04-21T08:36:23.990324+0000","last_became_peered":"2024-04-21T08:36:23.990324+0000","last_unstale":"2024-04-21T08:37:15.115081+0000","last_undegraded":"2024-04-21T08:37:15.115081+0000","last_fullsized":"2024-04-21T08:37:15.115081+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:08:59.417850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.119019+0000","last_change":"2024-04-21T08:36:28.007956+0000","last_active":"2024-04-21T08:37:15.119019+0000","last_peered":"2024-04-21T08:37:15.119019+0000","last_clean":"2024-04-21T08:37:15.119019+0000","last_became_active":"2024-04-21T08:36:28.007475+0000","last_became_peered":"2024-04-21T08:36:28.007475+0000","last_unstale":"2024-04-21T08:37:15.119019+0000","last_undegraded":"2024-04-21T08:37:15.119019+0000","last_fullsized":"2024-04-21T08:37:15.119019+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:54.039359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":109,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:38:04.055528+0000","last_change":"2024-04-21T08:36:23.476887+0000","last_active":"2024-04-21T08:38:04.055528+0000","last_peered":"2024-04-21T08:38:04.055528+0000","last_clean":"2024-04-21T08:38:04.055528+0000","last_became_active":"2024-04-21T08:36:23.476574+0000","last_became_peered":"2024-04-21T08:36:23.476574+0000","last_unstale":"2024-04-21T08:38:04.055528+0000","last_undegraded":"2024-04-21T08:38:04.055528+0000","last_fullsized":"2024-04-21T08:38:04.055528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:37:47.692358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116352+0000","last_change":"2024-04-21T08:36:23.982845+0000","last_active":"2024-04-21T08:37:15.116352+0000","last_peered":"2024-04-21T08:37:15.116352+0000","last_clean":"2024-04-21T08:37:15.116352+0000","last_became_active":"2024-04-21T08:36:23.982491+0000","last_became_peered":"2024-04-21T08:36:23.982491+0000","last_unstale":"2024-04-21T08:37:15.116352+0000","last_undegraded":"2024-04-21T08:37:15.116352+0000","last_fullsized":"2024-04-21T08:37:15.116352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:48:46.413896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.114931+0000","last_change":"2024-04-21T08:36:28.010442+0000","last_active":"2024-04-21T08:37:15.114931+0000","last_peered":"2024-04-21T08:37:15.114931+0000","last_clean":"2024-04-21T08:37:15.114931+0000","last_became_active":"2024-04-21T08:36:28.010046+0000","last_became_peered":"2024-04-21T08:36:28.010046+0000","last_unstale":"2024-04-21T08:37:15.114931+0000","last_undegraded":"2024-04-21T08:37:15.114931+0000","last_fullsized":"2024-04-21T08:37:15.114931+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:18:36.382802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.117204+0000","last_change":"2024-04-21T08:36:23.473545+0000","last_active":"2024-04-21T08:37:15.117204+0000","last_peered":"2024-04-21T08:37:15.117204+0000","last_clean":"2024-04-21T08:37:15.117204+0000","last_became_active":"2024-04-21T08:36:23.473222+0000","last_became_peered":"2024-04-21T08:36:23.473222+0000","last_unstale":"2024-04-21T08:37:15.117204+0000","last_undegraded":"2024-04-21T08:37:15.117204+0000","last_fullsized":"2024-04-21T08:37:15.117204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:47:53.947187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116161+0000","last_change":"2024-04-21T08:36:23.990959+0000","last_active":"2024-04-21T08:37:15.116161+0000","last_peered":"2024-04-21T08:37:15.116161+0000","last_clean":"2024-04-21T08:37:15.116161+0000","last_became_active":"2024-04-21T08:36:23.990322+0000","last_became_peered":"2024-04-21T08:36:23.990322+0000","last_unstale":"2024-04-21T08:37:15.116161+0000","last_undegraded":"2024-04-21T08:37:15.116161+0000","last_fullsized":"2024-04-21T08:37:15.116161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:12:20.859224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116010+0000","last_change":"2024-04-21T08:36:28.006682+0000","last_active":"2024-04-21T08:37:15.116010+0000","last_peered":"2024-04-21T08:37:15.116010+0000","last_clean":"2024-04-21T08:37:15.116010+0000","last_became_active":"2024-04-21T08:36:28.006229+0000","last_became_peered":"2024-04-21T08:36:28.006229+0000","last_unstale":"2024-04-21T08:37:15.116010+0000","last_undegraded":"2024-04-21T08:37:15.116010+0000","last_fullsized":"2024-04-21T08:37:15.116010+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:30:12.723848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.120234+0000","last_change":"2024-04-21T08:36:23.473592+0000","last_active":"2024-04-21T08:37:15.120234+0000","last_peered":"2024-04-21T08:37:15.120234+0000","last_clean":"2024-04-21T08:37:15.120234+0000","last_became_active":"2024-04-21T08:36:23.473283+0000","last_became_peered":"2024-04-21T08:36:23.473283+0000","last_unstale":"2024-04-21T08:37:15.120234+0000","last_undegraded":"2024-04-21T08:37:15.120234+0000","last_fullsized":"2024-04-21T08:37:15.120234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:10.425249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116060+0000","last_change":"2024-04-21T08:36:23.985052+0000","last_active":"2024-04-21T08:37:15.116060+0000","last_peered":"2024-04-21T08:37:15.116060+0000","last_clean":"2024-04-21T08:37:15.116060+0000","last_became_active":"2024-04-21T08:36:23.984742+0000","last_became_peered":"2024-04-21T08:36:23.984742+0000","last_unstale":"2024-04-21T08:37:15.116060+0000","last_undegraded":"2024-04-21T08:37:15.116060+0000","last_fullsized":"2024-04-21T08:37:15.116060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:08:06.082974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.218357+0000","last_change":"2024-04-21T08:36:23.980905+0000","last_active":"2024-04-21T08:37:15.218357+0000","last_peered":"2024-04-21T08:37:15.218357+0000","last_clean":"2024-04-21T08:37:15.218357+0000","last_became_active":"2024-04-21T08:36:23.980590+0000","last_became_peered":"2024-04-21T08:36:23.980590+0000","last_unstale":"2024-04-21T08:37:15.218357+0000","last_undegraded":"2024-04-21T08:37:15.218357+0000","last_fullsized":"2024-04-21T08:37:15.218357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:54:59.886014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.122358+0000","last_change":"2024-04-21T08:36:23.980883+0000","last_active":"2024-04-21T08:37:15.122358+0000","last_peered":"2024-04-21T08:37:15.122358+0000","last_clean":"2024-04-21T08:37:15.122358+0000","last_became_active":"2024-04-21T08:36:23.980562+0000","last_became_peered":"2024-04-21T08:36:23.980562+0000","last_unstale":"2024-04-21T08:37:15.122358+0000","last_undegraded":"2024-04-21T08:37:15.122358+0000","last_fullsized":"2024-04-21T08:37:15.122358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:14:51.868438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.121470+0000","last_change":"2024-04-21T08:36:23.991571+0000","last_active":"2024-04-21T08:37:15.121470+0000","last_peered":"2024-04-21T08:37:15.121470+0000","last_clean":"2024-04-21T08:37:15.121470+0000","last_became_active":"2024-04-21T08:36:23.991273+0000","last_became_peered":"2024-04-21T08:36:23.991273+0000","last_unstale":"2024-04-21T08:37:15.121470+0000","last_undegraded":"2024-04-21T08:37:15.121470+0000","last_fullsized":"2024-04-21T08:37:15.121470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:08:23.895038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.115122+0000","last_change":"2024-04-21T08:36:23.994729+0000","last_active":"2024-04-21T08:37:15.115122+0000","last_peered":"2024-04-21T08:37:15.115122+0000","last_clean":"2024-04-21T08:37:15.115122+0000","last_became_active":"2024-04-21T08:36:23.994405+0000","last_became_peered":"2024-04-21T08:36:23.994405+0000","last_unstale":"2024-04-21T08:37:15.115122+0000","last_undegraded":"2024-04-21T08:37:15.115122+0000","last_fullsized":"2024-04-21T08:37:15.115122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:42.463225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.120282+0000","last_change":"2024-04-21T08:36:23.995421+0000","last_active":"2024-04-21T08:37:15.120282+0000","last_peered":"2024-04-21T08:37:15.120282+0000","last_clean":"2024-04-21T08:37:15.120282+0000","last_became_active":"2024-04-21T08:36:23.993674+0000","last_became_peered":"2024-04-21T08:36:23.993674+0000","last_unstale":"2024-04-21T08:37:15.120282+0000","last_undegraded":"2024-04-21T08:37:15.120282+0000","last_fullsized":"2024-04-21T08:37:15.120282+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:31.013874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.218208+0000","last_change":"2024-04-21T08:36:23.986418+0000","last_active":"2024-04-21T08:37:15.218208+0000","last_peered":"2024-04-21T08:37:15.218208+0000","last_clean":"2024-04-21T08:37:15.218208+0000","last_became_active":"2024-04-21T08:36:23.986095+0000","last_became_peered":"2024-04-21T08:36:23.986095+0000","last_unstale":"2024-04-21T08:37:15.218208+0000","last_undegraded":"2024-04-21T08:37:15.218208+0000","last_fullsized":"2024-04-21T08:37:15.218208+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:31:56.456218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.122096+0000","last_change":"2024-04-21T08:36:23.982353+0000","last_active":"2024-04-21T08:37:15.122096+0000","last_peered":"2024-04-21T08:37:15.122096+0000","last_clean":"2024-04-21T08:37:15.122096+0000","last_became_active":"2024-04-21T08:36:23.982035+0000","last_became_peered":"2024-04-21T08:36:23.982035+0000","last_unstale":"2024-04-21T08:37:15.122096+0000","last_undegraded":"2024-04-21T08:37:15.122096+0000","last_fullsized":"2024-04-21T08:37:15.122096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:42:44.598407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.556088+0000","last_change":"2024-04-21T08:36:23.996371+0000","last_active":"2024-04-21T08:37:15.556088+0000","last_peered":"2024-04-21T08:37:15.556088+0000","last_clean":"2024-04-21T08:37:15.556088+0000","last_became_active":"2024-04-21T08:36:23.995802+0000","last_became_peered":"2024-04-21T08:36:23.995802+0000","last_unstale":"2024-04-21T08:37:15.556088+0000","last_undegraded":"2024-04-21T08:37:15.556088+0000","last_fullsized":"2024-04-21T08:37:15.556088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:52:08.571567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116331+0000","last_change":"2024-04-21T08:36:23.987275+0000","last_active":"2024-04-21T08:37:15.116331+0000","last_peered":"2024-04-21T08:37:15.116331+0000","last_clean":"2024-04-21T08:37:15.116331+0000","last_became_active":"2024-04-21T08:36:23.986967+0000","last_became_peered":"2024-04-21T08:36:23.986967+0000","last_unstale":"2024-04-21T08:37:15.116331+0000","last_undegraded":"2024-04-21T08:37:15.116331+0000","last_fullsized":"2024-04-21T08:37:15.116331+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:50.432502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.555792+0000","last_change":"2024-04-21T08:36:23.993046+0000","last_active":"2024-04-21T08:37:15.555792+0000","last_peered":"2024-04-21T08:37:15.555792+0000","last_clean":"2024-04-21T08:37:15.555792+0000","last_became_active":"2024-04-21T08:36:23.992693+0000","last_became_peered":"2024-04-21T08:36:23.992693+0000","last_unstale":"2024-04-21T08:37:15.555792+0000","last_undegraded":"2024-04-21T08:37:15.555792+0000","last_fullsized":"2024-04-21T08:37:15.555792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:12:06.287585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.119326+0000","last_change":"2024-04-21T08:36:23.992968+0000","last_active":"2024-04-21T08:37:15.119326+0000","last_peered":"2024-04-21T08:37:15.119326+0000","last_clean":"2024-04-21T08:37:15.119326+0000","last_became_active":"2024-04-21T08:36:23.992614+0000","last_became_peered":"2024-04-21T08:36:23.992614+0000","last_unstale":"2024-04-21T08:37:15.119326+0000","last_undegraded":"2024-04-21T08:37:15.119326+0000","last_fullsized":"2024-04-21T08:37:15.119326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:10.015323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.119980+0000","last_change":"2024-04-21T08:36:23.981169+0000","last_active":"2024-04-21T08:37:15.119980+0000","last_peered":"2024-04-21T08:37:15.119980+0000","last_clean":"2024-04-21T08:37:15.119980+0000","last_became_active":"2024-04-21T08:36:23.980849+0000","last_became_peered":"2024-04-21T08:36:23.980849+0000","last_unstale":"2024-04-21T08:37:15.119980+0000","last_undegraded":"2024-04-21T08:37:15.119980+0000","last_fullsized":"2024-04-21T08:37:15.119980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:13:49.907370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.218204+0000","last_change":"2024-04-21T08:36:23.993998+0000","last_active":"2024-04-21T08:37:15.218204+0000","last_peered":"2024-04-21T08:37:15.218204+0000","last_clean":"2024-04-21T08:37:15.218204+0000","last_became_active":"2024-04-21T08:36:23.993664+0000","last_became_peered":"2024-04-21T08:36:23.993664+0000","last_unstale":"2024-04-21T08:37:15.218204+0000","last_undegraded":"2024-04-21T08:37:15.218204+0000","last_fullsized":"2024-04-21T08:37:15.218204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:31:49.076820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70951,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":767,"num_read_kb":459,"num_write":243,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":212853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351010,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":506318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612647,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32876,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712276,"statfs":{"total":95995035648,"available":95961370624,"internally_reserved":0,"allocated":1740800,"data_stored":1156490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776172,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1073152,"data_stored":509953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037810,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":506235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299446,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1662976,"data_stored":1103123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462971,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":501361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":565747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":503208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":566749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":501069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":505826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724609,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712336,"statfs":{"total":95995035648,"available":95961432064,"internally_reserved":0,"allocated":1671168,"data_stored":1098314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:38:08.331 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-21T08:38:08.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:08 smithi043 ceph-mon[27978]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:08.585 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:08.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:08 smithi175 ceph-mon[31920]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:08.793 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:08 smithi112 ceph-mon[31085]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:09.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:09 smithi175 ceph-mon[31920]: pgmap v349: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-21T08:38:09.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:09 smithi112 ceph-mon[31085]: pgmap v349: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-21T08:38:09.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:09 smithi043 ceph-mon[27978]: pgmap v349: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-21T08:38:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.477+0000 7f3abc4ae700 1 -- 172.21.15.43:0/2031388208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab406b170 msgr2=0x7f3ab406b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:10.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.477+0000 7f3abc4ae700 1 --2- 172.21.15.43:0/2031388208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab406b170 0x7f3ab406b5f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8009990 tx=0x7f3aa8009ca0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.477+0000 7f3abc4ae700 1 -- 172.21.15.43:0/2031388208 shutdown_connections 2024-04-21T08:38:10.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.477+0000 7f3abc4ae700 1 --2- 172.21.15.43:0/2031388208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ab4108a70 0x7f3ab4108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.477+0000 7f3abc4ae700 1 --2- 172.21.15.43:0/2031388208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ab406bb30 0x7f3ab410f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.477+0000 7f3abc4ae700 1 --2- 172.21.15.43:0/2031388208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab406b170 0x7f3ab406b5f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.477+0000 7f3abc4ae700 1 -- 172.21.15.43:0/2031388208 >> 172.21.15.43:0/2031388208 conn(0x7f3ab40777b0 msgr2=0x7f3ab4077bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:10.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.477+0000 7f3abc4ae700 1 -- 172.21.15.43:0/2031388208 shutdown_connections 2024-04-21T08:38:10.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.477+0000 7f3abc4ae700 1 -- 172.21.15.43:0/2031388208 wait complete. 2024-04-21T08:38:10.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.478+0000 7f3abc4ae700 1 Processor -- start 2024-04-21T08:38:10.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.478+0000 7f3abc4ae700 1 -- start start 2024-04-21T08:38:10.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.478+0000 7f3abc4ae700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ab406b170 0x7f3ab4199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:10.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.478+0000 7f3abc4ae700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab406bb30 0x7f3ab4199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:10.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.478+0000 7f3abc4ae700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ab4108a70 0x7f3ab419d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:10.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.478+0000 7f3abc4ae700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab4112c50 con 0x7f3ab406bb30 2024-04-21T08:38:10.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.478+0000 7f3abc4ae700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab4112ad0 con 0x7f3ab406b170 2024-04-21T08:38:10.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.478+0000 7f3abc4ae700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab4112dd0 con 0x7f3ab4108a70 2024-04-21T08:38:10.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.478+0000 7f3ab9a49700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab406bb30 0x7f3ab4199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:10.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.479+0000 7f3ab9a49700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab406bb30 0x7f3ab4199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38538/0 (socket says 172.21.15.43:38538) 2024-04-21T08:38:10.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.479+0000 7f3ab9a49700 1 -- 172.21.15.43:0/3406256932 learned_addr learned my addr 172.21.15.43:0/3406256932 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:10.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.479+0000 7f3abaa4b700 1 --2- 172.21.15.43:0/3406256932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ab4108a70 0x7f3ab419d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:10.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.479+0000 7f3aba24a700 1 --2- 172.21.15.43:0/3406256932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ab406b170 0x7f3ab4199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:10.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.479+0000 7f3ab9a49700 1 -- 172.21.15.43:0/3406256932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ab4108a70 msgr2=0x7f3ab419d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:10.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.479+0000 7f3ab9a49700 1 --2- 172.21.15.43:0/3406256932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ab4108a70 0x7f3ab419d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.479+0000 7f3ab9a49700 1 -- 172.21.15.43:0/3406256932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ab406b170 msgr2=0x7f3ab4199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:10.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.479+0000 7f3ab9a49700 1 --2- 172.21.15.43:0/3406256932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ab406b170 0x7f3ab4199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.479+0000 7f3ab9a49700 1 -- 172.21.15.43:0/3406256932 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab419e130 con 0x7f3ab406bb30 2024-04-21T08:38:10.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.479+0000 7f3ab9a49700 1 --2- 172.21.15.43:0/3406256932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab406bb30 0x7f3ab4199660 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80047b0 tx=0x7f3aa8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:10.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.479+0000 7f3ab37fe700 1 -- 172.21.15.43:0/3406256932 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8026070 con 0x7f3ab406bb30 2024-04-21T08:38:10.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.479+0000 7f3ab37fe700 1 -- 172.21.15.43:0/3406256932 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aa8004c90 con 0x7f3ab406bb30 2024-04-21T08:38:10.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.479+0000 7f3abc4ae700 1 -- 172.21.15.43:0/3406256932 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab419e3c0 con 0x7f3ab406bb30 2024-04-21T08:38:10.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.479+0000 7f3ab37fe700 1 -- 172.21.15.43:0/3406256932 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa800fa10 con 0x7f3ab406bb30 2024-04-21T08:38:10.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.479+0000 7f3abc4ae700 1 -- 172.21.15.43:0/3406256932 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab41a5ca0 con 0x7f3ab406bb30 2024-04-21T08:38:10.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.480+0000 7f3abc4ae700 1 -- 172.21.15.43:0/3406256932 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ab41041a0 con 0x7f3ab406bb30 2024-04-21T08:38:10.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.482+0000 7f3ab37fe700 1 -- 172.21.15.43:0/3406256932 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3aa8029070 con 0x7f3ab406bb30 2024-04-21T08:38:10.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.482+0000 7f3ab37fe700 1 --2- 172.21.15.43:0/3406256932 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f3a9c0a3cd0 0x7f3a9c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:10.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.482+0000 7f3aba24a700 1 --2- 172.21.15.43:0/3406256932 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f3a9c0a3cd0 0x7f3a9c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:10.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.482+0000 7f3ab37fe700 1 -- 172.21.15.43:0/3406256932 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f3aa80d1530 con 0x7f3ab406bb30 2024-04-21T08:38:10.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.483+0000 7f3aba24a700 1 --2- 172.21.15.43:0/3406256932 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f3a9c0a3cd0 0x7f3a9c0a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4003f10 tx=0x7f3aa4009180 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:10.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.486+0000 7f3ab37fe700 1 -- 172.21.15.43:0/3406256932 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3aa8030020 con 0x7f3ab406bb30 2024-04-21T08:38:10.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.689+0000 7f3abc4ae700 1 -- 172.21.15.43:0/3406256932 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3ab419a480 con 0x7f3a9c0a3cd0 2024-04-21T08:38:10.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.699+0000 7f3ab37fe700 1 -- 172.21.15.43:0/3406256932 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340398 (secure 0 0 0) 0x7f3ab419a480 con 0x7f3a9c0a3cd0 2024-04-21T08:38:10.704 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:10.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.702+0000 7f3abc4ae700 1 -- 172.21.15.43:0/3406256932 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f3a9c0a3cd0 msgr2=0x7f3a9c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:10.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.702+0000 7f3abc4ae700 1 --2- 172.21.15.43:0/3406256932 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f3a9c0a3cd0 0x7f3a9c0a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4003f10 tx=0x7f3aa4009180 comp rx=0 tx=0).stop 2024-04-21T08:38:10.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.702+0000 7f3abc4ae700 1 -- 172.21.15.43:0/3406256932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab406bb30 msgr2=0x7f3ab4199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:10.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.702+0000 7f3abc4ae700 1 --2- 172.21.15.43:0/3406256932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab406bb30 0x7f3ab4199660 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80047b0 tx=0x7f3aa8005e80 comp rx=0 tx=0).stop 2024-04-21T08:38:10.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.702+0000 7f3abc4ae700 1 -- 172.21.15.43:0/3406256932 shutdown_connections 2024-04-21T08:38:10.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.702+0000 7f3abc4ae700 1 --2- 172.21.15.43:0/3406256932 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f3a9c0a3cd0 0x7f3a9c0a6190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.702+0000 7f3abc4ae700 1 --2- 172.21.15.43:0/3406256932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ab4108a70 0x7f3ab419d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.702+0000 7f3abc4ae700 1 --2- 172.21.15.43:0/3406256932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ab406b170 0x7f3ab4199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.702+0000 7f3abc4ae700 1 --2- 172.21.15.43:0/3406256932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab406bb30 0x7f3ab4199660 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.702+0000 7f3abc4ae700 1 -- 172.21.15.43:0/3406256932 >> 172.21.15.43:0/3406256932 conn(0x7f3ab40777b0 msgr2=0x7f3ab4100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:10.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.702+0000 7f3abc4ae700 1 -- 172.21.15.43:0/3406256932 shutdown_connections 2024-04-21T08:38:10.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:10.702+0000 7f3abc4ae700 1 -- 172.21.15.43:0/3406256932 wait complete. 2024-04-21T08:38:10.708 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-21T08:38:11.542 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:11 smithi043 ceph-mon[27978]: pgmap v350: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-21T08:38:11.542 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:11 smithi043 ceph-mon[27978]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:11.577 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2024-04-21T08:38:10.003572+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661553,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":947,"num_read_kb":602,"num_write":426,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380244,"kb_used_data":14384,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561580,"statfs":{"total":1151940427776,"available":1151551057920,"internally_reserved":0,"allocated":14729216,"data_stored":8024393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":374453564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":54,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003525"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.218236+0000","last_change":"2024-04-21T08:36:23.983921+0000","last_active":"2024-04-21T08:37:15.218236+0000","last_peered":"2024-04-21T08:37:15.218236+0000","last_clean":"2024-04-21T08:37:15.218236+0000","last_became_active":"2024-04-21T08:36:23.983638+0000","last_became_peered":"2024-04-21T08:36:23.983638+0000","last_unstale":"2024-04-21T08:37:15.218236+0000","last_undegraded":"2024-04-21T08:37:15.218236+0000","last_fullsized":"2024-04-21T08:37:15.218236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:29:44.646308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.115599+0000","last_change":"2024-04-21T08:36:23.990029+0000","last_active":"2024-04-21T08:37:15.115599+0000","last_peered":"2024-04-21T08:37:15.115599+0000","last_clean":"2024-04-21T08:37:15.115599+0000","last_became_active":"2024-04-21T08:36:23.989141+0000","last_became_peered":"2024-04-21T08:36:23.989141+0000","last_unstale":"2024-04-21T08:37:15.115599+0000","last_undegraded":"2024-04-21T08:37:15.115599+0000","last_fullsized":"2024-04-21T08:37:15.115599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:23:58.692342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.115320+0000","last_change":"2024-04-21T08:36:23.982205+0000","last_active":"2024-04-21T08:37:15.115320+0000","last_peered":"2024-04-21T08:37:15.115320+0000","last_clean":"2024-04-21T08:37:15.115320+0000","last_became_active":"2024-04-21T08:36:23.981876+0000","last_became_peered":"2024-04-21T08:36:23.981876+0000","last_unstale":"2024-04-21T08:37:15.115320+0000","last_undegraded":"2024-04-21T08:37:15.115320+0000","last_fullsized":"2024-04-21T08:37:15.115320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:12:27.121106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.114980+0000","last_change":"2024-04-21T08:36:23.987425+0000","last_active":"2024-04-21T08:37:15.114980+0000","last_peered":"2024-04-21T08:37:15.114980+0000","last_clean":"2024-04-21T08:37:15.114980+0000","last_became_active":"2024-04-21T08:36:23.987109+0000","last_became_peered":"2024-04-21T08:36:23.987109+0000","last_unstale":"2024-04-21T08:37:15.114980+0000","last_undegraded":"2024-04-21T08:37:15.114980+0000","last_fullsized":"2024-04-21T08:37:15.114980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:44.176324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.112980+0000","last_change":"2024-04-21T08:36:23.990375+0000","last_active":"2024-04-21T08:37:15.112980+0000","last_peered":"2024-04-21T08:37:15.112980+0000","last_clean":"2024-04-21T08:37:15.112980+0000","last_became_active":"2024-04-21T08:36:23.990058+0000","last_became_peered":"2024-04-21T08:36:23.990058+0000","last_unstale":"2024-04-21T08:37:15.112980+0000","last_undegraded":"2024-04-21T08:37:15.112980+0000","last_fullsized":"2024-04-21T08:37:15.112980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:24:09.647528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.117978+0000","last_change":"2024-04-21T08:36:23.979848+0000","last_active":"2024-04-21T08:37:15.117978+0000","last_peered":"2024-04-21T08:37:15.117978+0000","last_clean":"2024-04-21T08:37:15.117978+0000","last_became_active":"2024-04-21T08:36:23.979511+0000","last_became_peered":"2024-04-21T08:36:23.979511+0000","last_unstale":"2024-04-21T08:37:15.117978+0000","last_undegraded":"2024-04-21T08:37:15.117978+0000","last_fullsized":"2024-04-21T08:37:15.117978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:07:46.551369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.118755+0000","last_change":"2024-04-21T08:37:15.118755+0000","last_active":"2024-04-21T08:37:15.118755+0000","last_peered":"2024-04-21T08:37:15.118755+0000","last_clean":"2024-04-21T08:37:15.118755+0000","last_became_active":"2024-04-21T08:37:15.118347+0000","last_became_peered":"2024-04-21T08:37:15.118347+0000","last_unstale":"2024-04-21T08:37:15.118755+0000","last_undegraded":"2024-04-21T08:37:15.118755+0000","last_fullsized":"2024-04-21T08:37:15.118755+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:36:43.398325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.555143+0000","last_change":"2024-04-21T08:36:23.987517+0000","last_active":"2024-04-21T08:37:15.555143+0000","last_peered":"2024-04-21T08:37:15.555143+0000","last_clean":"2024-04-21T08:37:15.555143+0000","last_became_active":"2024-04-21T08:36:23.987201+0000","last_became_peered":"2024-04-21T08:36:23.987201+0000","last_unstale":"2024-04-21T08:37:15.555143+0000","last_undegraded":"2024-04-21T08:37:15.555143+0000","last_fullsized":"2024-04-21T08:37:15.555143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:42:53.080863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.120308+0000","last_change":"2024-04-21T08:36:23.989434+0000","last_active":"2024-04-21T08:37:15.120308+0000","last_peered":"2024-04-21T08:37:15.120308+0000","last_clean":"2024-04-21T08:37:15.120308+0000","last_became_active":"2024-04-21T08:36:23.989125+0000","last_became_peered":"2024-04-21T08:36:23.989125+0000","last_unstale":"2024-04-21T08:37:15.120308+0000","last_undegraded":"2024-04-21T08:37:15.120308+0000","last_fullsized":"2024-04-21T08:37:15.120308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:30:07.219056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.117822+0000","last_change":"2024-04-21T08:36:23.980725+0000","last_active":"2024-04-21T08:37:15.117822+0000","last_peered":"2024-04-21T08:37:15.117822+0000","last_clean":"2024-04-21T08:37:15.117822+0000","last_became_active":"2024-04-21T08:36:23.980363+0000","last_became_peered":"2024-04-21T08:36:23.980363+0000","last_unstale":"2024-04-21T08:37:15.117822+0000","last_undegraded":"2024-04-21T08:37:15.117822+0000","last_fullsized":"2024-04-21T08:37:15.117822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:26:46.202030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116293+0000","last_change":"2024-04-21T08:36:23.989369+0000","last_active":"2024-04-21T08:37:15.116293+0000","last_peered":"2024-04-21T08:37:15.116293+0000","last_clean":"2024-04-21T08:37:15.116293+0000","last_became_active":"2024-04-21T08:36:23.989030+0000","last_became_peered":"2024-04-21T08:36:23.989030+0000","last_unstale":"2024-04-21T08:37:15.116293+0000","last_undegraded":"2024-04-21T08:37:15.116293+0000","last_fullsized":"2024-04-21T08:37:15.116293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:23:13.725703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.114871+0000","last_change":"2024-04-21T08:36:23.993485+0000","last_active":"2024-04-21T08:37:15.114871+0000","last_peered":"2024-04-21T08:37:15.114871+0000","last_clean":"2024-04-21T08:37:15.114871+0000","last_became_active":"2024-04-21T08:36:23.993124+0000","last_became_peered":"2024-04-21T08:36:23.993124+0000","last_unstale":"2024-04-21T08:37:15.114871+0000","last_undegraded":"2024-04-21T08:37:15.114871+0000","last_fullsized":"2024-04-21T08:37:15.114871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:53:16.249838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.119846+0000","last_change":"2024-04-21T08:36:23.983146+0000","last_active":"2024-04-21T08:37:15.119846+0000","last_peered":"2024-04-21T08:37:15.119846+0000","last_clean":"2024-04-21T08:37:15.119846+0000","last_became_active":"2024-04-21T08:36:23.982766+0000","last_became_peered":"2024-04-21T08:36:23.982766+0000","last_unstale":"2024-04-21T08:37:15.119846+0000","last_undegraded":"2024-04-21T08:37:15.119846+0000","last_fullsized":"2024-04-21T08:37:15.119846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:30:44.090929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.114937+0000","last_change":"2024-04-21T08:36:23.988906+0000","last_active":"2024-04-21T08:37:15.114937+0000","last_peered":"2024-04-21T08:37:15.114937+0000","last_clean":"2024-04-21T08:37:15.114937+0000","last_became_active":"2024-04-21T08:36:23.988489+0000","last_became_peered":"2024-04-21T08:36:23.988489+0000","last_unstale":"2024-04-21T08:37:15.114937+0000","last_undegraded":"2024-04-21T08:37:15.114937+0000","last_fullsized":"2024-04-21T08:37:15.114937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:06:43.271650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.123343+0000","last_change":"2024-04-21T08:37:15.123343+0000","last_active":"2024-04-21T08:37:15.123343+0000","last_peered":"2024-04-21T08:37:15.123343+0000","last_clean":"2024-04-21T08:37:15.123343+0000","last_became_active":"2024-04-21T08:37:15.122959+0000","last_became_peered":"2024-04-21T08:37:15.122959+0000","last_unstale":"2024-04-21T08:37:15.123343+0000","last_undegraded":"2024-04-21T08:37:15.123343+0000","last_fullsized":"2024-04-21T08:37:15.123343+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:56:19.179879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.118945+0000","last_change":"2024-04-21T08:36:23.983136+0000","last_active":"2024-04-21T08:37:15.118945+0000","last_peered":"2024-04-21T08:37:15.118945+0000","last_clean":"2024-04-21T08:37:15.118945+0000","last_became_active":"2024-04-21T08:36:23.982698+0000","last_became_peered":"2024-04-21T08:36:23.982698+0000","last_unstale":"2024-04-21T08:37:15.118945+0000","last_undegraded":"2024-04-21T08:37:15.118945+0000","last_fullsized":"2024-04-21T08:37:15.118945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:34.750941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.218158+0000","last_change":"2024-04-21T08:36:23.984701+0000","last_active":"2024-04-21T08:37:15.218158+0000","last_peered":"2024-04-21T08:37:15.218158+0000","last_clean":"2024-04-21T08:37:15.218158+0000","last_became_active":"2024-04-21T08:36:23.984432+0000","last_became_peered":"2024-04-21T08:36:23.984432+0000","last_unstale":"2024-04-21T08:37:15.218158+0000","last_undegraded":"2024-04-21T08:37:15.218158+0000","last_fullsized":"2024-04-21T08:37:15.218158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:10:58.944380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.113513+0000","last_change":"2024-04-21T08:36:23.982781+0000","last_active":"2024-04-21T08:37:15.113513+0000","last_peered":"2024-04-21T08:37:15.113513+0000","last_clean":"2024-04-21T08:37:15.113513+0000","last_became_active":"2024-04-21T08:36:23.982459+0000","last_became_peered":"2024-04-21T08:36:23.982459+0000","last_unstale":"2024-04-21T08:37:15.113513+0000","last_undegraded":"2024-04-21T08:37:15.113513+0000","last_fullsized":"2024-04-21T08:37:15.113513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:44.657504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.122441+0000","last_change":"2024-04-21T08:36:23.992002+0000","last_active":"2024-04-21T08:37:15.122441+0000","last_peered":"2024-04-21T08:37:15.122441+0000","last_clean":"2024-04-21T08:37:15.122441+0000","last_became_active":"2024-04-21T08:36:23.991053+0000","last_became_peered":"2024-04-21T08:36:23.991053+0000","last_unstale":"2024-04-21T08:37:15.122441+0000","last_undegraded":"2024-04-21T08:37:15.122441+0000","last_fullsized":"2024-04-21T08:37:15.122441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:56:37.145949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.122197+0000","last_change":"2024-04-21T08:36:28.013195+0000","last_active":"2024-04-21T08:37:15.122197+0000","last_peered":"2024-04-21T08:37:15.122197+0000","last_clean":"2024-04-21T08:37:15.122197+0000","last_became_active":"2024-04-21T08:36:28.012656+0000","last_became_peered":"2024-04-21T08:36:28.012656+0000","last_unstale":"2024-04-21T08:37:15.122197+0000","last_undegraded":"2024-04-21T08:37:15.122197+0000","last_fullsized":"2024-04-21T08:37:15.122197+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:12:03.133583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":116,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:38:05.010663+0000","last_change":"2024-04-21T08:36:23.468957+0000","last_active":"2024-04-21T08:38:05.010663+0000","last_peered":"2024-04-21T08:38:05.010663+0000","last_clean":"2024-04-21T08:38:05.010663+0000","last_became_active":"2024-04-21T08:36:23.468601+0000","last_became_peered":"2024-04-21T08:36:23.468601+0000","last_unstale":"2024-04-21T08:38:05.010663+0000","last_undegraded":"2024-04-21T08:38:05.010663+0000","last_fullsized":"2024-04-21T08:38:05.010663+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:13:11.977780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.123893+0000","last_change":"2024-04-21T08:37:15.123476+0000","last_active":"2024-04-21T08:37:15.123893+0000","last_peered":"2024-04-21T08:37:15.123893+0000","last_clean":"2024-04-21T08:37:15.123893+0000","last_became_active":"2024-04-21T08:37:15.123066+0000","last_became_peered":"2024-04-21T08:37:15.123066+0000","last_unstale":"2024-04-21T08:37:15.123893+0000","last_undegraded":"2024-04-21T08:37:15.123893+0000","last_fullsized":"2024-04-21T08:37:15.123893+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:15.786178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.115138+0000","last_change":"2024-04-21T08:36:28.014161+0000","last_active":"2024-04-21T08:37:15.115138+0000","last_peered":"2024-04-21T08:37:15.115138+0000","last_clean":"2024-04-21T08:37:15.115138+0000","last_became_active":"2024-04-21T08:36:28.013859+0000","last_became_peered":"2024-04-21T08:36:28.013859+0000","last_unstale":"2024-04-21T08:37:15.115138+0000","last_undegraded":"2024-04-21T08:37:15.115138+0000","last_fullsized":"2024-04-21T08:37:15.115138+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:08.315334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.556182+0000","last_change":"2024-04-21T08:36:23.475612+0000","last_active":"2024-04-21T08:37:15.556182+0000","last_peered":"2024-04-21T08:37:15.556182+0000","last_clean":"2024-04-21T08:37:15.556182+0000","last_became_active":"2024-04-21T08:36:23.475301+0000","last_became_peered":"2024-04-21T08:36:23.475301+0000","last_unstale":"2024-04-21T08:37:15.556182+0000","last_undegraded":"2024-04-21T08:37:15.556182+0000","last_fullsized":"2024-04-21T08:37:15.556182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:56.801630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.115131+0000","last_change":"2024-04-21T08:36:23.984689+0000","last_active":"2024-04-21T08:37:15.115131+0000","last_peered":"2024-04-21T08:37:15.115131+0000","last_clean":"2024-04-21T08:37:15.115131+0000","last_became_active":"2024-04-21T08:36:23.984358+0000","last_became_peered":"2024-04-21T08:36:23.984358+0000","last_unstale":"2024-04-21T08:37:15.115131+0000","last_undegraded":"2024-04-21T08:37:15.115131+0000","last_fullsized":"2024-04-21T08:37:15.115131+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:49:50.187466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.114110+0000","last_change":"2024-04-21T08:36:28.007951+0000","last_active":"2024-04-21T08:37:15.114110+0000","last_peered":"2024-04-21T08:37:15.114110+0000","last_clean":"2024-04-21T08:37:15.114110+0000","last_became_active":"2024-04-21T08:36:28.007553+0000","last_became_peered":"2024-04-21T08:36:28.007553+0000","last_unstale":"2024-04-21T08:37:15.114110+0000","last_undegraded":"2024-04-21T08:37:15.114110+0000","last_fullsized":"2024-04-21T08:37:15.114110+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:10:54.019605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.113691+0000","last_change":"2024-04-21T08:36:23.473371+0000","last_active":"2024-04-21T08:37:15.113691+0000","last_peered":"2024-04-21T08:37:15.113691+0000","last_clean":"2024-04-21T08:37:15.113691+0000","last_became_active":"2024-04-21T08:36:23.472472+0000","last_became_peered":"2024-04-21T08:36:23.472472+0000","last_unstale":"2024-04-21T08:37:15.113691+0000","last_undegraded":"2024-04-21T08:37:15.113691+0000","last_fullsized":"2024-04-21T08:37:15.113691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:28:49.622873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.119670+0000","last_change":"2024-04-21T08:36:23.987916+0000","last_active":"2024-04-21T08:37:15.119670+0000","last_peered":"2024-04-21T08:37:15.119670+0000","last_clean":"2024-04-21T08:37:15.119670+0000","last_became_active":"2024-04-21T08:36:23.986207+0000","last_became_peered":"2024-04-21T08:36:23.986207+0000","last_unstale":"2024-04-21T08:37:15.119670+0000","last_undegraded":"2024-04-21T08:37:15.119670+0000","last_fullsized":"2024-04-21T08:37:15.119670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:49:57.261078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.113934+0000","last_change":"2024-04-21T08:36:28.006494+0000","last_active":"2024-04-21T08:37:15.113934+0000","last_peered":"2024-04-21T08:37:15.113934+0000","last_clean":"2024-04-21T08:37:15.113934+0000","last_became_active":"2024-04-21T08:36:28.006069+0000","last_became_peered":"2024-04-21T08:36:28.006069+0000","last_unstale":"2024-04-21T08:37:15.113934+0000","last_undegraded":"2024-04-21T08:37:15.113934+0000","last_fullsized":"2024-04-21T08:37:15.113934+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:04.608295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.113186+0000","last_change":"2024-04-21T08:36:23.468923+0000","last_active":"2024-04-21T08:37:15.113186+0000","last_peered":"2024-04-21T08:37:15.113186+0000","last_clean":"2024-04-21T08:37:15.113186+0000","last_became_active":"2024-04-21T08:36:23.468509+0000","last_became_peered":"2024-04-21T08:36:23.468509+0000","last_unstale":"2024-04-21T08:37:15.113186+0000","last_undegraded":"2024-04-21T08:37:15.113186+0000","last_fullsized":"2024-04-21T08:37:15.113186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:44.452748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116926+0000","last_change":"2024-04-21T08:36:23.994423+0000","last_active":"2024-04-21T08:37:15.116926+0000","last_peered":"2024-04-21T08:37:15.116926+0000","last_clean":"2024-04-21T08:37:15.116926+0000","last_became_active":"2024-04-21T08:36:23.993827+0000","last_became_peered":"2024-04-21T08:36:23.993827+0000","last_unstale":"2024-04-21T08:37:15.116926+0000","last_undegraded":"2024-04-21T08:37:15.116926+0000","last_fullsized":"2024-04-21T08:37:15.116926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:49:25.550301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.122337+0000","last_change":"2024-04-21T08:36:28.004037+0000","last_active":"2024-04-21T08:37:15.122337+0000","last_peered":"2024-04-21T08:37:15.122337+0000","last_clean":"2024-04-21T08:37:15.122337+0000","last_became_active":"2024-04-21T08:36:28.003580+0000","last_became_peered":"2024-04-21T08:36:28.003580+0000","last_unstale":"2024-04-21T08:37:15.122337+0000","last_undegraded":"2024-04-21T08:37:15.122337+0000","last_fullsized":"2024-04-21T08:37:15.122337+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:04:51.380244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":113,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:38:04.999237+0000","last_change":"2024-04-21T08:36:23.468430+0000","last_active":"2024-04-21T08:38:04.999237+0000","last_peered":"2024-04-21T08:38:04.999237+0000","last_clean":"2024-04-21T08:38:04.999237+0000","last_became_active":"2024-04-21T08:36:23.468089+0000","last_became_peered":"2024-04-21T08:36:23.468089+0000","last_unstale":"2024-04-21T08:38:04.999237+0000","last_undegraded":"2024-04-21T08:38:04.999237+0000","last_fullsized":"2024-04-21T08:38:04.999237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:12:54.827034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.117922+0000","last_change":"2024-04-21T08:37:15.117922+0000","last_active":"2024-04-21T08:37:15.117922+0000","last_peered":"2024-04-21T08:37:15.117922+0000","last_clean":"2024-04-21T08:37:15.117922+0000","last_became_active":"2024-04-21T08:37:15.117404+0000","last_became_peered":"2024-04-21T08:37:15.117404+0000","last_unstale":"2024-04-21T08:37:15.117922+0000","last_undegraded":"2024-04-21T08:37:15.117922+0000","last_fullsized":"2024-04-21T08:37:15.117922+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:39.682224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.121684+0000","last_change":"2024-04-21T08:36:28.004638+0000","last_active":"2024-04-21T08:37:15.121684+0000","last_peered":"2024-04-21T08:37:15.121684+0000","last_clean":"2024-04-21T08:37:15.121684+0000","last_became_active":"2024-04-21T08:36:28.002499+0000","last_became_peered":"2024-04-21T08:36:28.002499+0000","last_unstale":"2024-04-21T08:37:15.121684+0000","last_undegraded":"2024-04-21T08:37:15.121684+0000","last_fullsized":"2024-04-21T08:37:15.121684+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:45:46.934783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.126569+0000","last_change":"2024-04-21T08:37:15.126569+0000","last_active":"2024-04-21T08:37:15.126569+0000","last_peered":"2024-04-21T08:37:15.126569+0000","last_clean":"2024-04-21T08:37:15.126569+0000","last_became_active":"2024-04-21T08:37:15.121284+0000","last_became_peered":"2024-04-21T08:37:15.121284+0000","last_unstale":"2024-04-21T08:37:15.126569+0000","last_undegraded":"2024-04-21T08:37:15.126569+0000","last_fullsized":"2024-04-21T08:37:15.126569+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:02:26.153324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.557192+0000","last_change":"2024-04-21T08:37:15.557192+0000","last_active":"2024-04-21T08:37:15.557192+0000","last_peered":"2024-04-21T08:37:15.557192+0000","last_clean":"2024-04-21T08:37:15.557192+0000","last_became_active":"2024-04-21T08:37:15.556653+0000","last_became_peered":"2024-04-21T08:37:15.556653+0000","last_unstale":"2024-04-21T08:37:15.557192+0000","last_undegraded":"2024-04-21T08:37:15.557192+0000","last_fullsized":"2024-04-21T08:37:15.557192+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:42.722460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.218102+0000","last_change":"2024-04-21T08:36:28.017068+0000","last_active":"2024-04-21T08:37:15.218102+0000","last_peered":"2024-04-21T08:37:15.218102+0000","last_clean":"2024-04-21T08:37:15.218102+0000","last_became_active":"2024-04-21T08:36:28.016770+0000","last_became_peered":"2024-04-21T08:36:28.016770+0000","last_unstale":"2024-04-21T08:37:15.218102+0000","last_undegraded":"2024-04-21T08:37:15.218102+0000","last_fullsized":"2024-04-21T08:37:15.218102+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:07:14.596090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.119806+0000","last_change":"2024-04-21T08:36:23.470430+0000","last_active":"2024-04-21T08:37:15.119806+0000","last_peered":"2024-04-21T08:37:15.119806+0000","last_clean":"2024-04-21T08:37:15.119806+0000","last_became_active":"2024-04-21T08:36:23.470109+0000","last_became_peered":"2024-04-21T08:36:23.470109+0000","last_unstale":"2024-04-21T08:37:15.119806+0000","last_undegraded":"2024-04-21T08:37:15.119806+0000","last_fullsized":"2024-04-21T08:37:15.119806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:18.176313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.555284+0000","last_change":"2024-04-21T08:36:23.986668+0000","last_active":"2024-04-21T08:37:15.555284+0000","last_peered":"2024-04-21T08:37:15.555284+0000","last_clean":"2024-04-21T08:37:15.555284+0000","last_became_active":"2024-04-21T08:36:23.986344+0000","last_became_peered":"2024-04-21T08:36:23.986344+0000","last_unstale":"2024-04-21T08:37:15.555284+0000","last_undegraded":"2024-04-21T08:37:15.555284+0000","last_fullsized":"2024-04-21T08:37:15.555284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:56:17.744145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.115442+0000","last_change":"2024-04-21T08:36:28.009012+0000","last_active":"2024-04-21T08:37:15.115442+0000","last_peered":"2024-04-21T08:37:15.115442+0000","last_clean":"2024-04-21T08:37:15.115442+0000","last_became_active":"2024-04-21T08:36:28.008123+0000","last_became_peered":"2024-04-21T08:36:28.008123+0000","last_unstale":"2024-04-21T08:37:15.115442+0000","last_undegraded":"2024-04-21T08:37:15.115442+0000","last_fullsized":"2024-04-21T08:37:15.115442+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:33:18.881564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.117600+0000","last_change":"2024-04-21T08:36:23.467104+0000","last_active":"2024-04-21T08:37:15.117600+0000","last_peered":"2024-04-21T08:37:15.117600+0000","last_clean":"2024-04-21T08:37:15.117600+0000","last_became_active":"2024-04-21T08:36:23.466725+0000","last_became_peered":"2024-04-21T08:36:23.466725+0000","last_unstale":"2024-04-21T08:37:15.117600+0000","last_undegraded":"2024-04-21T08:37:15.117600+0000","last_fullsized":"2024-04-21T08:37:15.117600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:35:45.986448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.112600+0000","last_change":"2024-04-21T08:36:23.987579+0000","last_active":"2024-04-21T08:37:15.112600+0000","last_peered":"2024-04-21T08:37:15.112600+0000","last_clean":"2024-04-21T08:37:15.112600+0000","last_became_active":"2024-04-21T08:36:23.987044+0000","last_became_peered":"2024-04-21T08:36:23.987044+0000","last_unstale":"2024-04-21T08:37:15.112600+0000","last_undegraded":"2024-04-21T08:37:15.112600+0000","last_fullsized":"2024-04-21T08:37:15.112600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:48:27.593090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.555280+0000","last_change":"2024-04-21T08:36:28.006417+0000","last_active":"2024-04-21T08:37:15.555280+0000","last_peered":"2024-04-21T08:37:15.555280+0000","last_clean":"2024-04-21T08:37:15.555280+0000","last_became_active":"2024-04-21T08:36:28.006006+0000","last_became_peered":"2024-04-21T08:36:28.006006+0000","last_unstale":"2024-04-21T08:37:15.555280+0000","last_undegraded":"2024-04-21T08:37:15.555280+0000","last_fullsized":"2024-04-21T08:37:15.555280+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:16.173284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":184,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:38:06.058336+0000","last_change":"2024-04-21T08:36:23.471541+0000","last_active":"2024-04-21T08:38:06.058336+0000","last_peered":"2024-04-21T08:38:06.058336+0000","last_clean":"2024-04-21T08:38:06.058336+0000","last_became_active":"2024-04-21T08:36:23.471193+0000","last_became_peered":"2024-04-21T08:36:23.471193+0000","last_unstale":"2024-04-21T08:38:06.058336+0000","last_undegraded":"2024-04-21T08:38:06.058336+0000","last_fullsized":"2024-04-21T08:38:06.058336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:06:04.916707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":153,"num_read_kb":153,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.120272+0000","last_change":"2024-04-21T08:36:23.986781+0000","last_active":"2024-04-21T08:37:15.120272+0000","last_peered":"2024-04-21T08:37:15.120272+0000","last_clean":"2024-04-21T08:37:15.120272+0000","last_became_active":"2024-04-21T08:36:23.985696+0000","last_became_peered":"2024-04-21T08:36:23.985696+0000","last_unstale":"2024-04-21T08:37:15.120272+0000","last_undegraded":"2024-04-21T08:37:15.120272+0000","last_fullsized":"2024-04-21T08:37:15.120272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:40.556642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.112290+0000","last_change":"2024-04-21T08:36:28.011621+0000","last_active":"2024-04-21T08:37:15.112290+0000","last_peered":"2024-04-21T08:37:15.112290+0000","last_clean":"2024-04-21T08:37:15.112290+0000","last_became_active":"2024-04-21T08:36:28.011136+0000","last_became_peered":"2024-04-21T08:36:28.011136+0000","last_unstale":"2024-04-21T08:37:15.112290+0000","last_undegraded":"2024-04-21T08:37:15.112290+0000","last_fullsized":"2024-04-21T08:37:15.112290+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:47.623445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.115712+0000","last_change":"2024-04-21T08:36:23.474019+0000","last_active":"2024-04-21T08:37:15.115712+0000","last_peered":"2024-04-21T08:37:15.115712+0000","last_clean":"2024-04-21T08:37:15.115712+0000","last_became_active":"2024-04-21T08:36:23.473690+0000","last_became_peered":"2024-04-21T08:36:23.473690+0000","last_unstale":"2024-04-21T08:37:15.115712+0000","last_undegraded":"2024-04-21T08:37:15.115712+0000","last_fullsized":"2024-04-21T08:37:15.115712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:38:32.913795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.121455+0000","last_change":"2024-04-21T08:37:15.121455+0000","last_active":"2024-04-21T08:37:15.121455+0000","last_peered":"2024-04-21T08:37:15.121455+0000","last_clean":"2024-04-21T08:37:15.121455+0000","last_became_active":"2024-04-21T08:37:15.121113+0000","last_became_peered":"2024-04-21T08:37:15.121113+0000","last_unstale":"2024-04-21T08:37:15.121455+0000","last_undegraded":"2024-04-21T08:37:15.121455+0000","last_fullsized":"2024-04-21T08:37:15.121455+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:39:36.525495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.114358+0000","last_change":"2024-04-21T08:36:28.004420+0000","last_active":"2024-04-21T08:37:15.114358+0000","last_peered":"2024-04-21T08:37:15.114358+0000","last_clean":"2024-04-21T08:37:15.114358+0000","last_became_active":"2024-04-21T08:36:28.003935+0000","last_became_peered":"2024-04-21T08:36:28.003935+0000","last_unstale":"2024-04-21T08:37:15.114358+0000","last_undegraded":"2024-04-21T08:37:15.114358+0000","last_fullsized":"2024-04-21T08:37:15.114358+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:10:03.255869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.117190+0000","last_change":"2024-04-21T08:36:23.465825+0000","last_active":"2024-04-21T08:37:15.117190+0000","last_peered":"2024-04-21T08:37:15.117190+0000","last_clean":"2024-04-21T08:37:15.117190+0000","last_became_active":"2024-04-21T08:36:23.465525+0000","last_became_peered":"2024-04-21T08:36:23.465525+0000","last_unstale":"2024-04-21T08:37:15.117190+0000","last_undegraded":"2024-04-21T08:37:15.117190+0000","last_fullsized":"2024-04-21T08:37:15.117190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:59:50.502210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.119858+0000","last_change":"2024-04-21T08:36:23.987602+0000","last_active":"2024-04-21T08:37:15.119858+0000","last_peered":"2024-04-21T08:37:15.119858+0000","last_clean":"2024-04-21T08:37:15.119858+0000","last_became_active":"2024-04-21T08:36:23.987298+0000","last_became_peered":"2024-04-21T08:36:23.987298+0000","last_unstale":"2024-04-21T08:37:15.119858+0000","last_undegraded":"2024-04-21T08:37:15.119858+0000","last_fullsized":"2024-04-21T08:37:15.119858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:06.647676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.112441+0000","last_change":"2024-04-21T08:36:28.005935+0000","last_active":"2024-04-21T08:37:15.112441+0000","last_peered":"2024-04-21T08:37:15.112441+0000","last_clean":"2024-04-21T08:37:15.112441+0000","last_became_active":"2024-04-21T08:36:28.005512+0000","last_became_peered":"2024-04-21T08:36:28.005512+0000","last_unstale":"2024-04-21T08:37:15.112441+0000","last_undegraded":"2024-04-21T08:37:15.112441+0000","last_fullsized":"2024-04-21T08:37:15.112441+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:23:20.298524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'35","reported_seq":226,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:38:07.005221+0000","last_change":"2024-04-21T08:36:23.467728+0000","last_active":"2024-04-21T08:38:07.005221+0000","last_peered":"2024-04-21T08:38:07.005221+0000","last_clean":"2024-04-21T08:38:07.005221+0000","last_became_active":"2024-04-21T08:36:23.467430+0000","last_became_peered":"2024-04-21T08:36:23.467430+0000","last_unstale":"2024-04-21T08:38:07.005221+0000","last_undegraded":"2024-04-21T08:38:07.005221+0000","last_fullsized":"2024-04-21T08:38:07.005221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:47:18.900911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63356,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":164,"num_read_kb":144,"num_write":37,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.113320+0000","last_change":"2024-04-21T08:36:23.989116+0000","last_active":"2024-04-21T08:37:15.113320+0000","last_peered":"2024-04-21T08:37:15.113320+0000","last_clean":"2024-04-21T08:37:15.113320+0000","last_became_active":"2024-04-21T08:36:23.988758+0000","last_became_peered":"2024-04-21T08:36:23.988758+0000","last_unstale":"2024-04-21T08:37:15.113320+0000","last_undegraded":"2024-04-21T08:37:15.113320+0000","last_fullsized":"2024-04-21T08:37:15.113320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:56:25.969905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.117421+0000","last_change":"2024-04-21T08:36:28.007370+0000","last_active":"2024-04-21T08:37:15.117421+0000","last_peered":"2024-04-21T08:37:15.117421+0000","last_clean":"2024-04-21T08:37:15.117421+0000","last_became_active":"2024-04-21T08:36:28.006972+0000","last_became_peered":"2024-04-21T08:36:28.006972+0000","last_unstale":"2024-04-21T08:37:15.117421+0000","last_undegraded":"2024-04-21T08:37:15.117421+0000","last_fullsized":"2024-04-21T08:37:15.117421+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:18:14.926058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.121864+0000","last_change":"2024-04-21T08:36:23.472658+0000","last_active":"2024-04-21T08:37:15.121864+0000","last_peered":"2024-04-21T08:37:15.121864+0000","last_clean":"2024-04-21T08:37:15.121864+0000","last_became_active":"2024-04-21T08:36:23.472318+0000","last_became_peered":"2024-04-21T08:36:23.472318+0000","last_unstale":"2024-04-21T08:37:15.121864+0000","last_undegraded":"2024-04-21T08:37:15.121864+0000","last_fullsized":"2024-04-21T08:37:15.121864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:28:27.497605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.120626+0000","last_change":"2024-04-21T08:36:23.986897+0000","last_active":"2024-04-21T08:37:15.120626+0000","last_peered":"2024-04-21T08:37:15.120626+0000","last_clean":"2024-04-21T08:37:15.120626+0000","last_became_active":"2024-04-21T08:36:23.986554+0000","last_became_peered":"2024-04-21T08:36:23.986554+0000","last_unstale":"2024-04-21T08:37:15.120626+0000","last_undegraded":"2024-04-21T08:37:15.120626+0000","last_fullsized":"2024-04-21T08:37:15.120626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:44:22.331207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.218065+0000","last_change":"2024-04-21T08:36:28.019186+0000","last_active":"2024-04-21T08:37:15.218065+0000","last_peered":"2024-04-21T08:37:15.218065+0000","last_clean":"2024-04-21T08:37:15.218065+0000","last_became_active":"2024-04-21T08:36:28.018774+0000","last_became_peered":"2024-04-21T08:36:28.018774+0000","last_unstale":"2024-04-21T08:37:15.218065+0000","last_undegraded":"2024-04-21T08:37:15.218065+0000","last_fullsized":"2024-04-21T08:37:15.218065+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:35:33.548656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.117065+0000","last_change":"2024-04-21T08:36:23.472633+0000","last_active":"2024-04-21T08:37:15.117065+0000","last_peered":"2024-04-21T08:37:15.117065+0000","last_clean":"2024-04-21T08:37:15.117065+0000","last_became_active":"2024-04-21T08:36:23.472336+0000","last_became_peered":"2024-04-21T08:36:23.472336+0000","last_unstale":"2024-04-21T08:37:15.117065+0000","last_undegraded":"2024-04-21T08:37:15.117065+0000","last_fullsized":"2024-04-21T08:37:15.117065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:49.463492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.115208+0000","last_change":"2024-04-21T08:36:23.994332+0000","last_active":"2024-04-21T08:37:15.115208+0000","last_peered":"2024-04-21T08:37:15.115208+0000","last_clean":"2024-04-21T08:37:15.115208+0000","last_became_active":"2024-04-21T08:36:23.993936+0000","last_became_peered":"2024-04-21T08:36:23.993936+0000","last_unstale":"2024-04-21T08:37:15.115208+0000","last_undegraded":"2024-04-21T08:37:15.115208+0000","last_fullsized":"2024-04-21T08:37:15.115208+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:26:35.330559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.113463+0000","last_change":"2024-04-21T08:36:28.012549+0000","last_active":"2024-04-21T08:37:15.113463+0000","last_peered":"2024-04-21T08:37:15.113463+0000","last_clean":"2024-04-21T08:37:15.113463+0000","last_became_active":"2024-04-21T08:36:28.012151+0000","last_became_peered":"2024-04-21T08:36:28.012151+0000","last_unstale":"2024-04-21T08:37:15.113463+0000","last_undegraded":"2024-04-21T08:37:15.113463+0000","last_fullsized":"2024-04-21T08:37:15.113463+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:55.680399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":116,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:38:08.007140+0000","last_change":"2024-04-21T08:36:23.464586+0000","last_active":"2024-04-21T08:38:08.007140+0000","last_peered":"2024-04-21T08:38:08.007140+0000","last_clean":"2024-04-21T08:38:08.007140+0000","last_became_active":"2024-04-21T08:36:23.464281+0000","last_became_peered":"2024-04-21T08:36:23.464281+0000","last_unstale":"2024-04-21T08:38:08.007140+0000","last_undegraded":"2024-04-21T08:38:08.007140+0000","last_fullsized":"2024-04-21T08:38:08.007140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:04:58.577283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116093+0000","last_change":"2024-04-21T08:36:23.989604+0000","last_active":"2024-04-21T08:37:15.116093+0000","last_peered":"2024-04-21T08:37:15.116093+0000","last_clean":"2024-04-21T08:37:15.116093+0000","last_became_active":"2024-04-21T08:36:23.989257+0000","last_became_peered":"2024-04-21T08:36:23.989257+0000","last_unstale":"2024-04-21T08:37:15.116093+0000","last_undegraded":"2024-04-21T08:37:15.116093+0000","last_fullsized":"2024-04-21T08:37:15.116093+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:45.077943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.120255+0000","last_change":"2024-04-21T08:36:28.007959+0000","last_active":"2024-04-21T08:37:15.120255+0000","last_peered":"2024-04-21T08:37:15.120255+0000","last_clean":"2024-04-21T08:37:15.120255+0000","last_became_active":"2024-04-21T08:36:28.007571+0000","last_became_peered":"2024-04-21T08:36:28.007571+0000","last_unstale":"2024-04-21T08:37:15.120255+0000","last_undegraded":"2024-04-21T08:37:15.120255+0000","last_fullsized":"2024-04-21T08:37:15.120255+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:17:34.007013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.112793+0000","last_change":"2024-04-21T08:36:23.475054+0000","last_active":"2024-04-21T08:37:15.112793+0000","last_peered":"2024-04-21T08:37:15.112793+0000","last_clean":"2024-04-21T08:37:15.112793+0000","last_became_active":"2024-04-21T08:36:23.474606+0000","last_became_peered":"2024-04-21T08:36:23.474606+0000","last_unstale":"2024-04-21T08:37:15.112793+0000","last_undegraded":"2024-04-21T08:37:15.112793+0000","last_fullsized":"2024-04-21T08:37:15.112793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:22.559174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116692+0000","last_change":"2024-04-21T08:36:23.995980+0000","last_active":"2024-04-21T08:37:15.116692+0000","last_peered":"2024-04-21T08:37:15.116692+0000","last_clean":"2024-04-21T08:37:15.116692+0000","last_became_active":"2024-04-21T08:36:23.995673+0000","last_became_peered":"2024-04-21T08:36:23.995673+0000","last_unstale":"2024-04-21T08:37:15.116692+0000","last_undegraded":"2024-04-21T08:37:15.116692+0000","last_fullsized":"2024-04-21T08:37:15.116692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:11:49.884384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116929+0000","last_change":"2024-04-21T08:36:28.018792+0000","last_active":"2024-04-21T08:37:15.116929+0000","last_peered":"2024-04-21T08:37:15.116929+0000","last_clean":"2024-04-21T08:37:15.116929+0000","last_became_active":"2024-04-21T08:36:28.018373+0000","last_became_peered":"2024-04-21T08:36:28.018373+0000","last_unstale":"2024-04-21T08:37:15.116929+0000","last_undegraded":"2024-04-21T08:37:15.116929+0000","last_fullsized":"2024-04-21T08:37:15.116929+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:11.857661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.113413+0000","last_change":"2024-04-21T08:36:23.469787+0000","last_active":"2024-04-21T08:37:15.113413+0000","last_peered":"2024-04-21T08:37:15.113413+0000","last_clean":"2024-04-21T08:37:15.113413+0000","last_became_active":"2024-04-21T08:36:23.469462+0000","last_became_peered":"2024-04-21T08:36:23.469462+0000","last_unstale":"2024-04-21T08:37:15.113413+0000","last_undegraded":"2024-04-21T08:37:15.113413+0000","last_fullsized":"2024-04-21T08:37:15.113413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:33:25.849528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.218022+0000","last_change":"2024-04-21T08:36:23.988492+0000","last_active":"2024-04-21T08:37:15.218022+0000","last_peered":"2024-04-21T08:37:15.218022+0000","last_clean":"2024-04-21T08:37:15.218022+0000","last_became_active":"2024-04-21T08:36:23.988133+0000","last_became_peered":"2024-04-21T08:36:23.988133+0000","last_unstale":"2024-04-21T08:37:15.218022+0000","last_undegraded":"2024-04-21T08:37:15.218022+0000","last_fullsized":"2024-04-21T08:37:15.218022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:01:25.320281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.113025+0000","last_change":"2024-04-21T08:36:28.012454+0000","last_active":"2024-04-21T08:37:15.113025+0000","last_peered":"2024-04-21T08:37:15.113025+0000","last_clean":"2024-04-21T08:37:15.113025+0000","last_became_active":"2024-04-21T08:36:28.011985+0000","last_became_peered":"2024-04-21T08:36:28.011985+0000","last_unstale":"2024-04-21T08:37:15.113025+0000","last_undegraded":"2024-04-21T08:37:15.113025+0000","last_fullsized":"2024-04-21T08:37:15.113025+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:16:51.939691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.219428+0000","last_change":"2024-04-21T08:37:15.219428+0000","last_active":"2024-04-21T08:37:15.219428+0000","last_peered":"2024-04-21T08:37:15.219428+0000","last_clean":"2024-04-21T08:37:15.219428+0000","last_became_active":"2024-04-21T08:37:15.218923+0000","last_became_peered":"2024-04-21T08:37:15.218923+0000","last_unstale":"2024-04-21T08:37:15.219428+0000","last_undegraded":"2024-04-21T08:37:15.219428+0000","last_fullsized":"2024-04-21T08:37:15.219428+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:07:02.781701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.117495+0000","last_change":"2024-04-21T08:36:23.985910+0000","last_active":"2024-04-21T08:37:15.117495+0000","last_peered":"2024-04-21T08:37:15.117495+0000","last_clean":"2024-04-21T08:37:15.117495+0000","last_became_active":"2024-04-21T08:36:23.985605+0000","last_became_peered":"2024-04-21T08:36:23.985605+0000","last_unstale":"2024-04-21T08:37:15.117495+0000","last_undegraded":"2024-04-21T08:37:15.117495+0000","last_fullsized":"2024-04-21T08:37:15.117495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:37:09.367005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.120042+0000","last_change":"2024-04-21T08:36:28.015256+0000","last_active":"2024-04-21T08:37:15.120042+0000","last_peered":"2024-04-21T08:37:15.120042+0000","last_clean":"2024-04-21T08:37:15.120042+0000","last_became_active":"2024-04-21T08:36:28.014849+0000","last_became_peered":"2024-04-21T08:36:28.014849+0000","last_unstale":"2024-04-21T08:37:15.120042+0000","last_undegraded":"2024-04-21T08:37:15.120042+0000","last_fullsized":"2024-04-21T08:37:15.120042+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:22:05.466299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":110,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:38:07.014161+0000","last_change":"2024-04-21T08:36:23.474966+0000","last_active":"2024-04-21T08:38:07.014161+0000","last_peered":"2024-04-21T08:38:07.014161+0000","last_clean":"2024-04-21T08:38:07.014161+0000","last_became_active":"2024-04-21T08:36:23.474374+0000","last_became_peered":"2024-04-21T08:36:23.474374+0000","last_unstale":"2024-04-21T08:38:07.014161+0000","last_undegraded":"2024-04-21T08:38:07.014161+0000","last_fullsized":"2024-04-21T08:38:07.014161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:57:20.237966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.118887+0000","last_change":"2024-04-21T08:37:15.118887+0000","last_active":"2024-04-21T08:37:15.118887+0000","last_peered":"2024-04-21T08:37:15.118887+0000","last_clean":"2024-04-21T08:37:15.118887+0000","last_became_active":"2024-04-21T08:37:15.118510+0000","last_became_peered":"2024-04-21T08:37:15.118510+0000","last_unstale":"2024-04-21T08:37:15.118887+0000","last_undegraded":"2024-04-21T08:37:15.118887+0000","last_fullsized":"2024-04-21T08:37:15.118887+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:59:27.209083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.120380+0000","last_change":"2024-04-21T08:36:28.013169+0000","last_active":"2024-04-21T08:37:15.120380+0000","last_peered":"2024-04-21T08:37:15.120380+0000","last_clean":"2024-04-21T08:37:15.120380+0000","last_became_active":"2024-04-21T08:36:28.012791+0000","last_became_peered":"2024-04-21T08:36:28.012791+0000","last_unstale":"2024-04-21T08:37:15.120380+0000","last_undegraded":"2024-04-21T08:37:15.120380+0000","last_fullsized":"2024-04-21T08:37:15.120380+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:42:56.063159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116469+0000","last_change":"2024-04-21T08:36:23.995209+0000","last_active":"2024-04-21T08:37:15.116469+0000","last_peered":"2024-04-21T08:37:15.116469+0000","last_clean":"2024-04-21T08:37:15.116469+0000","last_became_active":"2024-04-21T08:36:23.994866+0000","last_became_peered":"2024-04-21T08:36:23.994866+0000","last_unstale":"2024-04-21T08:37:15.116469+0000","last_undegraded":"2024-04-21T08:37:15.116469+0000","last_fullsized":"2024-04-21T08:37:15.116469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:33:45.582150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":112,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:38:07.005365+0000","last_change":"2024-04-21T08:36:23.471474+0000","last_active":"2024-04-21T08:38:07.005365+0000","last_peered":"2024-04-21T08:38:07.005365+0000","last_clean":"2024-04-21T08:38:07.005365+0000","last_became_active":"2024-04-21T08:36:23.471179+0000","last_became_peered":"2024-04-21T08:36:23.471179+0000","last_unstale":"2024-04-21T08:38:07.005365+0000","last_undegraded":"2024-04-21T08:38:07.005365+0000","last_fullsized":"2024-04-21T08:38:07.005365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:17.290051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116176+0000","last_change":"2024-04-21T08:36:28.009671+0000","last_active":"2024-04-21T08:37:15.116176+0000","last_peered":"2024-04-21T08:37:15.116176+0000","last_clean":"2024-04-21T08:37:15.116176+0000","last_became_active":"2024-04-21T08:36:28.009185+0000","last_became_peered":"2024-04-21T08:36:28.009185+0000","last_unstale":"2024-04-21T08:37:15.116176+0000","last_undegraded":"2024-04-21T08:37:15.116176+0000","last_fullsized":"2024-04-21T08:37:15.116176+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:11:05.156271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.555286+0000","last_change":"2024-04-21T08:36:23.990272+0000","last_active":"2024-04-21T08:37:15.555286+0000","last_peered":"2024-04-21T08:37:15.555286+0000","last_clean":"2024-04-21T08:37:15.555286+0000","last_became_active":"2024-04-21T08:36:23.989940+0000","last_became_peered":"2024-04-21T08:36:23.989940+0000","last_unstale":"2024-04-21T08:37:15.555286+0000","last_undegraded":"2024-04-21T08:37:15.555286+0000","last_fullsized":"2024-04-21T08:37:15.555286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:27:50.705167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.114241+0000","last_change":"2024-04-21T08:36:23.474162+0000","last_active":"2024-04-21T08:37:15.114241+0000","last_peered":"2024-04-21T08:37:15.114241+0000","last_clean":"2024-04-21T08:37:15.114241+0000","last_became_active":"2024-04-21T08:36:23.473802+0000","last_became_peered":"2024-04-21T08:36:23.473802+0000","last_unstale":"2024-04-21T08:37:15.114241+0000","last_undegraded":"2024-04-21T08:37:15.114241+0000","last_fullsized":"2024-04-21T08:37:15.114241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:32:41.918431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":360,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116011+0000","last_change":"2024-04-21T08:35:10.062130+0000","last_active":"2024-04-21T08:37:15.116011+0000","last_peered":"2024-04-21T08:37:15.116011+0000","last_clean":"2024-04-21T08:37:15.116011+0000","last_became_active":"2024-04-21T08:35:10.061727+0000","last_became_peered":"2024-04-21T08:35:10.061727+0000","last_unstale":"2024-04-21T08:37:15.116011+0000","last_undegraded":"2024-04-21T08:37:15.116011+0000","last_fullsized":"2024-04-21T08:37:15.116011+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:31:40.950939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:31:40.950939+0000","last_clean_scrub_stamp":"2024-04-21T08:31:40.950939+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:35:52.360167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116481+0000","last_change":"2024-04-21T08:36:28.012840+0000","last_active":"2024-04-21T08:37:15.116481+0000","last_peered":"2024-04-21T08:37:15.116481+0000","last_clean":"2024-04-21T08:37:15.116481+0000","last_became_active":"2024-04-21T08:36:28.012439+0000","last_became_peered":"2024-04-21T08:36:28.012439+0000","last_unstale":"2024-04-21T08:37:15.116481+0000","last_undegraded":"2024-04-21T08:37:15.116481+0000","last_fullsized":"2024-04-21T08:37:15.116481+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:36:39.260061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.119658+0000","last_change":"2024-04-21T08:36:23.984332+0000","last_active":"2024-04-21T08:37:15.119658+0000","last_peered":"2024-04-21T08:37:15.119658+0000","last_clean":"2024-04-21T08:37:15.119658+0000","last_became_active":"2024-04-21T08:36:23.984021+0000","last_became_peered":"2024-04-21T08:36:23.984021+0000","last_unstale":"2024-04-21T08:37:15.119658+0000","last_undegraded":"2024-04-21T08:37:15.119658+0000","last_fullsized":"2024-04-21T08:37:15.119658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:05:21.090633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.115990+0000","last_change":"2024-04-21T08:36:23.467522+0000","last_active":"2024-04-21T08:37:15.115990+0000","last_peered":"2024-04-21T08:37:15.115990+0000","last_clean":"2024-04-21T08:37:15.115990+0000","last_became_active":"2024-04-21T08:36:23.466379+0000","last_became_peered":"2024-04-21T08:36:23.466379+0000","last_unstale":"2024-04-21T08:37:15.115990+0000","last_undegraded":"2024-04-21T08:37:15.115990+0000","last_fullsized":"2024-04-21T08:37:15.115990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:04.185314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.217837+0000","last_change":"2024-04-21T08:36:28.015120+0000","last_active":"2024-04-21T08:37:15.217837+0000","last_peered":"2024-04-21T08:37:15.217837+0000","last_clean":"2024-04-21T08:37:15.217837+0000","last_became_active":"2024-04-21T08:36:28.014701+0000","last_became_peered":"2024-04-21T08:36:28.014701+0000","last_unstale":"2024-04-21T08:37:15.217837+0000","last_undegraded":"2024-04-21T08:37:15.217837+0000","last_fullsized":"2024-04-21T08:37:15.217837+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:23.127622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.113440+0000","last_change":"2024-04-21T08:36:23.475335+0000","last_active":"2024-04-21T08:37:15.113440+0000","last_peered":"2024-04-21T08:37:15.113440+0000","last_clean":"2024-04-21T08:37:15.113440+0000","last_became_active":"2024-04-21T08:36:23.474739+0000","last_became_peered":"2024-04-21T08:36:23.474739+0000","last_unstale":"2024-04-21T08:37:15.113440+0000","last_undegraded":"2024-04-21T08:37:15.113440+0000","last_fullsized":"2024-04-21T08:37:15.113440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:36.119649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.120403+0000","last_change":"2024-04-21T08:36:23.977229+0000","last_active":"2024-04-21T08:37:15.120403+0000","last_peered":"2024-04-21T08:37:15.120403+0000","last_clean":"2024-04-21T08:37:15.120403+0000","last_became_active":"2024-04-21T08:36:23.976893+0000","last_became_peered":"2024-04-21T08:36:23.976893+0000","last_unstale":"2024-04-21T08:37:15.120403+0000","last_undegraded":"2024-04-21T08:37:15.120403+0000","last_fullsized":"2024-04-21T08:37:15.120403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:31:54.226274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.555157+0000","last_change":"2024-04-21T08:36:28.018581+0000","last_active":"2024-04-21T08:37:15.555157+0000","last_peered":"2024-04-21T08:37:15.555157+0000","last_clean":"2024-04-21T08:37:15.555157+0000","last_became_active":"2024-04-21T08:36:28.018021+0000","last_became_peered":"2024-04-21T08:36:28.018021+0000","last_unstale":"2024-04-21T08:37:15.555157+0000","last_undegraded":"2024-04-21T08:37:15.555157+0000","last_fullsized":"2024-04-21T08:37:15.555157+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:29:00.355532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.113645+0000","last_change":"2024-04-21T08:36:23.473339+0000","last_active":"2024-04-21T08:37:15.113645+0000","last_peered":"2024-04-21T08:37:15.113645+0000","last_clean":"2024-04-21T08:37:15.113645+0000","last_became_active":"2024-04-21T08:36:23.472994+0000","last_became_peered":"2024-04-21T08:36:23.472994+0000","last_unstale":"2024-04-21T08:37:15.113645+0000","last_undegraded":"2024-04-21T08:37:15.113645+0000","last_fullsized":"2024-04-21T08:37:15.113645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:42.762273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.555861+0000","last_change":"2024-04-21T08:36:23.987354+0000","last_active":"2024-04-21T08:37:15.555861+0000","last_peered":"2024-04-21T08:37:15.555861+0000","last_clean":"2024-04-21T08:37:15.555861+0000","last_became_active":"2024-04-21T08:36:23.987032+0000","last_became_peered":"2024-04-21T08:36:23.987032+0000","last_unstale":"2024-04-21T08:37:15.555861+0000","last_undegraded":"2024-04-21T08:37:15.555861+0000","last_fullsized":"2024-04-21T08:37:15.555861+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:01:05.164367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.121970+0000","last_change":"2024-04-21T08:36:28.002582+0000","last_active":"2024-04-21T08:37:15.121970+0000","last_peered":"2024-04-21T08:37:15.121970+0000","last_clean":"2024-04-21T08:37:15.121970+0000","last_became_active":"2024-04-21T08:36:28.002133+0000","last_became_peered":"2024-04-21T08:36:28.002133+0000","last_unstale":"2024-04-21T08:37:15.121970+0000","last_undegraded":"2024-04-21T08:37:15.121970+0000","last_fullsized":"2024-04-21T08:37:15.121970+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:30:00.268015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.114867+0000","last_change":"2024-04-21T08:36:23.462840+0000","last_active":"2024-04-21T08:37:15.114867+0000","last_peered":"2024-04-21T08:37:15.114867+0000","last_clean":"2024-04-21T08:37:15.114867+0000","last_became_active":"2024-04-21T08:36:23.462391+0000","last_became_peered":"2024-04-21T08:36:23.462391+0000","last_unstale":"2024-04-21T08:37:15.114867+0000","last_undegraded":"2024-04-21T08:37:15.114867+0000","last_fullsized":"2024-04-21T08:37:15.114867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:33.230388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.115561+0000","last_change":"2024-04-21T08:36:23.994751+0000","last_active":"2024-04-21T08:37:15.115561+0000","last_peered":"2024-04-21T08:37:15.115561+0000","last_clean":"2024-04-21T08:37:15.115561+0000","last_became_active":"2024-04-21T08:36:23.994384+0000","last_became_peered":"2024-04-21T08:36:23.994384+0000","last_unstale":"2024-04-21T08:37:15.115561+0000","last_undegraded":"2024-04-21T08:37:15.115561+0000","last_fullsized":"2024-04-21T08:37:15.115561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:18.956893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.114299+0000","last_change":"2024-04-21T08:36:28.004004+0000","last_active":"2024-04-21T08:37:15.114299+0000","last_peered":"2024-04-21T08:37:15.114299+0000","last_clean":"2024-04-21T08:37:15.114299+0000","last_became_active":"2024-04-21T08:36:28.003547+0000","last_became_peered":"2024-04-21T08:36:28.003547+0000","last_unstale":"2024-04-21T08:37:15.114299+0000","last_undegraded":"2024-04-21T08:37:15.114299+0000","last_fullsized":"2024-04-21T08:37:15.114299+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:21:40.215428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.222118+0000","last_change":"2024-04-21T08:37:15.222118+0000","last_active":"2024-04-21T08:37:15.222118+0000","last_peered":"2024-04-21T08:37:15.222118+0000","last_clean":"2024-04-21T08:37:15.222118+0000","last_became_active":"2024-04-21T08:37:15.219173+0000","last_became_peered":"2024-04-21T08:37:15.219173+0000","last_unstale":"2024-04-21T08:37:15.222118+0000","last_undegraded":"2024-04-21T08:37:15.222118+0000","last_fullsized":"2024-04-21T08:37:15.222118+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:12.931076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.112796+0000","last_change":"2024-04-21T08:36:23.988020+0000","last_active":"2024-04-21T08:37:15.112796+0000","last_peered":"2024-04-21T08:37:15.112796+0000","last_clean":"2024-04-21T08:37:15.112796+0000","last_became_active":"2024-04-21T08:36:23.987630+0000","last_became_peered":"2024-04-21T08:36:23.987630+0000","last_unstale":"2024-04-21T08:37:15.112796+0000","last_undegraded":"2024-04-21T08:37:15.112796+0000","last_fullsized":"2024-04-21T08:37:15.112796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:06.519001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.113936+0000","last_change":"2024-04-21T08:36:28.006984+0000","last_active":"2024-04-21T08:37:15.113936+0000","last_peered":"2024-04-21T08:37:15.113936+0000","last_clean":"2024-04-21T08:37:15.113936+0000","last_became_active":"2024-04-21T08:36:28.006548+0000","last_became_peered":"2024-04-21T08:36:28.006548+0000","last_unstale":"2024-04-21T08:37:15.113936+0000","last_undegraded":"2024-04-21T08:37:15.113936+0000","last_fullsized":"2024-04-21T08:37:15.113936+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:25:50.870658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.555439+0000","last_change":"2024-04-21T08:36:23.476266+0000","last_active":"2024-04-21T08:37:15.555439+0000","last_peered":"2024-04-21T08:37:15.555439+0000","last_clean":"2024-04-21T08:37:15.555439+0000","last_became_active":"2024-04-21T08:36:23.475948+0000","last_became_peered":"2024-04-21T08:36:23.475948+0000","last_unstale":"2024-04-21T08:37:15.555439+0000","last_undegraded":"2024-04-21T08:37:15.555439+0000","last_fullsized":"2024-04-21T08:37:15.555439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:41.529376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.218219+0000","last_change":"2024-04-21T08:36:23.981942+0000","last_active":"2024-04-21T08:37:15.218219+0000","last_peered":"2024-04-21T08:37:15.218219+0000","last_clean":"2024-04-21T08:37:15.218219+0000","last_became_active":"2024-04-21T08:36:23.981643+0000","last_became_peered":"2024-04-21T08:36:23.981643+0000","last_unstale":"2024-04-21T08:37:15.218219+0000","last_undegraded":"2024-04-21T08:37:15.218219+0000","last_fullsized":"2024-04-21T08:37:15.218219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:26:24.722695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.114243+0000","last_change":"2024-04-21T08:36:28.006980+0000","last_active":"2024-04-21T08:37:15.114243+0000","last_peered":"2024-04-21T08:37:15.114243+0000","last_clean":"2024-04-21T08:37:15.114243+0000","last_became_active":"2024-04-21T08:36:28.006530+0000","last_became_peered":"2024-04-21T08:36:28.006530+0000","last_unstale":"2024-04-21T08:37:15.114243+0000","last_undegraded":"2024-04-21T08:37:15.114243+0000","last_fullsized":"2024-04-21T08:37:15.114243+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:05:57.465612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.121956+0000","last_change":"2024-04-21T08:36:23.473253+0000","last_active":"2024-04-21T08:37:15.121956+0000","last_peered":"2024-04-21T08:37:15.121956+0000","last_clean":"2024-04-21T08:37:15.121956+0000","last_became_active":"2024-04-21T08:36:23.472942+0000","last_became_peered":"2024-04-21T08:36:23.472942+0000","last_unstale":"2024-04-21T08:37:15.121956+0000","last_undegraded":"2024-04-21T08:37:15.121956+0000","last_fullsized":"2024-04-21T08:37:15.121956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:32:04.894388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.555573+0000","last_change":"2024-04-21T08:36:23.988355+0000","last_active":"2024-04-21T08:37:15.555573+0000","last_peered":"2024-04-21T08:37:15.555573+0000","last_clean":"2024-04-21T08:37:15.555573+0000","last_became_active":"2024-04-21T08:36:23.988049+0000","last_became_peered":"2024-04-21T08:36:23.988049+0000","last_unstale":"2024-04-21T08:37:15.555573+0000","last_undegraded":"2024-04-21T08:37:15.555573+0000","last_fullsized":"2024-04-21T08:37:15.555573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:57.348698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.115166+0000","last_change":"2024-04-21T08:36:28.007343+0000","last_active":"2024-04-21T08:37:15.115166+0000","last_peered":"2024-04-21T08:37:15.115166+0000","last_clean":"2024-04-21T08:37:15.115166+0000","last_became_active":"2024-04-21T08:36:28.006719+0000","last_became_peered":"2024-04-21T08:36:28.006719+0000","last_unstale":"2024-04-21T08:37:15.115166+0000","last_undegraded":"2024-04-21T08:37:15.115166+0000","last_fullsized":"2024-04-21T08:37:15.115166+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:02:25.734584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.112987+0000","last_change":"2024-04-21T08:36:23.472416+0000","last_active":"2024-04-21T08:37:15.112987+0000","last_peered":"2024-04-21T08:37:15.112987+0000","last_clean":"2024-04-21T08:37:15.112987+0000","last_became_active":"2024-04-21T08:36:23.471500+0000","last_became_peered":"2024-04-21T08:36:23.471500+0000","last_unstale":"2024-04-21T08:37:15.112987+0000","last_undegraded":"2024-04-21T08:37:15.112987+0000","last_fullsized":"2024-04-21T08:37:15.112987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:39.706432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.115081+0000","last_change":"2024-04-21T08:36:23.990653+0000","last_active":"2024-04-21T08:37:15.115081+0000","last_peered":"2024-04-21T08:37:15.115081+0000","last_clean":"2024-04-21T08:37:15.115081+0000","last_became_active":"2024-04-21T08:36:23.990324+0000","last_became_peered":"2024-04-21T08:36:23.990324+0000","last_unstale":"2024-04-21T08:37:15.115081+0000","last_undegraded":"2024-04-21T08:37:15.115081+0000","last_fullsized":"2024-04-21T08:37:15.115081+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:08:59.417850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.119019+0000","last_change":"2024-04-21T08:36:28.007956+0000","last_active":"2024-04-21T08:37:15.119019+0000","last_peered":"2024-04-21T08:37:15.119019+0000","last_clean":"2024-04-21T08:37:15.119019+0000","last_became_active":"2024-04-21T08:36:28.007475+0000","last_became_peered":"2024-04-21T08:36:28.007475+0000","last_unstale":"2024-04-21T08:37:15.119019+0000","last_undegraded":"2024-04-21T08:37:15.119019+0000","last_fullsized":"2024-04-21T08:37:15.119019+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:54.039359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":109,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:38:04.055528+0000","last_change":"2024-04-21T08:36:23.476887+0000","last_active":"2024-04-21T08:38:04.055528+0000","last_peered":"2024-04-21T08:38:04.055528+0000","last_clean":"2024-04-21T08:38:04.055528+0000","last_became_active":"2024-04-21T08:36:23.476574+0000","last_became_peered":"2024-04-21T08:36:23.476574+0000","last_unstale":"2024-04-21T08:38:04.055528+0000","last_undegraded":"2024-04-21T08:38:04.055528+0000","last_fullsized":"2024-04-21T08:38:04.055528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:37:47.692358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116352+0000","last_change":"2024-04-21T08:36:23.982845+0000","last_active":"2024-04-21T08:37:15.116352+0000","last_peered":"2024-04-21T08:37:15.116352+0000","last_clean":"2024-04-21T08:37:15.116352+0000","last_became_active":"2024-04-21T08:36:23.982491+0000","last_became_peered":"2024-04-21T08:36:23.982491+0000","last_unstale":"2024-04-21T08:37:15.116352+0000","last_undegraded":"2024-04-21T08:37:15.116352+0000","last_fullsized":"2024-04-21T08:37:15.116352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:48:46.413896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.114931+0000","last_change":"2024-04-21T08:36:28.010442+0000","last_active":"2024-04-21T08:37:15.114931+0000","last_peered":"2024-04-21T08:37:15.114931+0000","last_clean":"2024-04-21T08:37:15.114931+0000","last_became_active":"2024-04-21T08:36:28.010046+0000","last_became_peered":"2024-04-21T08:36:28.010046+0000","last_unstale":"2024-04-21T08:37:15.114931+0000","last_undegraded":"2024-04-21T08:37:15.114931+0000","last_fullsized":"2024-04-21T08:37:15.114931+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:18:36.382802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.117204+0000","last_change":"2024-04-21T08:36:23.473545+0000","last_active":"2024-04-21T08:37:15.117204+0000","last_peered":"2024-04-21T08:37:15.117204+0000","last_clean":"2024-04-21T08:37:15.117204+0000","last_became_active":"2024-04-21T08:36:23.473222+0000","last_became_peered":"2024-04-21T08:36:23.473222+0000","last_unstale":"2024-04-21T08:37:15.117204+0000","last_undegraded":"2024-04-21T08:37:15.117204+0000","last_fullsized":"2024-04-21T08:37:15.117204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:47:53.947187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116161+0000","last_change":"2024-04-21T08:36:23.990959+0000","last_active":"2024-04-21T08:37:15.116161+0000","last_peered":"2024-04-21T08:37:15.116161+0000","last_clean":"2024-04-21T08:37:15.116161+0000","last_became_active":"2024-04-21T08:36:23.990322+0000","last_became_peered":"2024-04-21T08:36:23.990322+0000","last_unstale":"2024-04-21T08:37:15.116161+0000","last_undegraded":"2024-04-21T08:37:15.116161+0000","last_fullsized":"2024-04-21T08:37:15.116161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:12:20.859224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116010+0000","last_change":"2024-04-21T08:36:28.006682+0000","last_active":"2024-04-21T08:37:15.116010+0000","last_peered":"2024-04-21T08:37:15.116010+0000","last_clean":"2024-04-21T08:37:15.116010+0000","last_became_active":"2024-04-21T08:36:28.006229+0000","last_became_peered":"2024-04-21T08:36:28.006229+0000","last_unstale":"2024-04-21T08:37:15.116010+0000","last_undegraded":"2024-04-21T08:37:15.116010+0000","last_fullsized":"2024-04-21T08:37:15.116010+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:30:12.723848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.120234+0000","last_change":"2024-04-21T08:36:23.473592+0000","last_active":"2024-04-21T08:37:15.120234+0000","last_peered":"2024-04-21T08:37:15.120234+0000","last_clean":"2024-04-21T08:37:15.120234+0000","last_became_active":"2024-04-21T08:36:23.473283+0000","last_became_peered":"2024-04-21T08:36:23.473283+0000","last_unstale":"2024-04-21T08:37:15.120234+0000","last_undegraded":"2024-04-21T08:37:15.120234+0000","last_fullsized":"2024-04-21T08:37:15.120234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:10.425249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116060+0000","last_change":"2024-04-21T08:36:23.985052+0000","last_active":"2024-04-21T08:37:15.116060+0000","last_peered":"2024-04-21T08:37:15.116060+0000","last_clean":"2024-04-21T08:37:15.116060+0000","last_became_active":"2024-04-21T08:36:23.984742+0000","last_became_peered":"2024-04-21T08:36:23.984742+0000","last_unstale":"2024-04-21T08:37:15.116060+0000","last_undegraded":"2024-04-21T08:37:15.116060+0000","last_fullsized":"2024-04-21T08:37:15.116060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:08:06.082974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.218357+0000","last_change":"2024-04-21T08:36:23.980905+0000","last_active":"2024-04-21T08:37:15.218357+0000","last_peered":"2024-04-21T08:37:15.218357+0000","last_clean":"2024-04-21T08:37:15.218357+0000","last_became_active":"2024-04-21T08:36:23.980590+0000","last_became_peered":"2024-04-21T08:36:23.980590+0000","last_unstale":"2024-04-21T08:37:15.218357+0000","last_undegraded":"2024-04-21T08:37:15.218357+0000","last_fullsized":"2024-04-21T08:37:15.218357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:54:59.886014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.122358+0000","last_change":"2024-04-21T08:36:23.980883+0000","last_active":"2024-04-21T08:37:15.122358+0000","last_peered":"2024-04-21T08:37:15.122358+0000","last_clean":"2024-04-21T08:37:15.122358+0000","last_became_active":"2024-04-21T08:36:23.980562+0000","last_became_peered":"2024-04-21T08:36:23.980562+0000","last_unstale":"2024-04-21T08:37:15.122358+0000","last_undegraded":"2024-04-21T08:37:15.122358+0000","last_fullsized":"2024-04-21T08:37:15.122358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:14:51.868438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.121470+0000","last_change":"2024-04-21T08:36:23.991571+0000","last_active":"2024-04-21T08:37:15.121470+0000","last_peered":"2024-04-21T08:37:15.121470+0000","last_clean":"2024-04-21T08:37:15.121470+0000","last_became_active":"2024-04-21T08:36:23.991273+0000","last_became_peered":"2024-04-21T08:36:23.991273+0000","last_unstale":"2024-04-21T08:37:15.121470+0000","last_undegraded":"2024-04-21T08:37:15.121470+0000","last_fullsized":"2024-04-21T08:37:15.121470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:08:23.895038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.115122+0000","last_change":"2024-04-21T08:36:23.994729+0000","last_active":"2024-04-21T08:37:15.115122+0000","last_peered":"2024-04-21T08:37:15.115122+0000","last_clean":"2024-04-21T08:37:15.115122+0000","last_became_active":"2024-04-21T08:36:23.994405+0000","last_became_peered":"2024-04-21T08:36:23.994405+0000","last_unstale":"2024-04-21T08:37:15.115122+0000","last_undegraded":"2024-04-21T08:37:15.115122+0000","last_fullsized":"2024-04-21T08:37:15.115122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:42.463225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.120282+0000","last_change":"2024-04-21T08:36:23.995421+0000","last_active":"2024-04-21T08:37:15.120282+0000","last_peered":"2024-04-21T08:37:15.120282+0000","last_clean":"2024-04-21T08:37:15.120282+0000","last_became_active":"2024-04-21T08:36:23.993674+0000","last_became_peered":"2024-04-21T08:36:23.993674+0000","last_unstale":"2024-04-21T08:37:15.120282+0000","last_undegraded":"2024-04-21T08:37:15.120282+0000","last_fullsized":"2024-04-21T08:37:15.120282+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:31.013874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.218208+0000","last_change":"2024-04-21T08:36:23.986418+0000","last_active":"2024-04-21T08:37:15.218208+0000","last_peered":"2024-04-21T08:37:15.218208+0000","last_clean":"2024-04-21T08:37:15.218208+0000","last_became_active":"2024-04-21T08:36:23.986095+0000","last_became_peered":"2024-04-21T08:36:23.986095+0000","last_unstale":"2024-04-21T08:37:15.218208+0000","last_undegraded":"2024-04-21T08:37:15.218208+0000","last_fullsized":"2024-04-21T08:37:15.218208+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:31:56.456218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.122096+0000","last_change":"2024-04-21T08:36:23.982353+0000","last_active":"2024-04-21T08:37:15.122096+0000","last_peered":"2024-04-21T08:37:15.122096+0000","last_clean":"2024-04-21T08:37:15.122096+0000","last_became_active":"2024-04-21T08:36:23.982035+0000","last_became_peered":"2024-04-21T08:36:23.982035+0000","last_unstale":"2024-04-21T08:37:15.122096+0000","last_undegraded":"2024-04-21T08:37:15.122096+0000","last_fullsized":"2024-04-21T08:37:15.122096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:42:44.598407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.556088+0000","last_change":"2024-04-21T08:36:23.996371+0000","last_active":"2024-04-21T08:37:15.556088+0000","last_peered":"2024-04-21T08:37:15.556088+0000","last_clean":"2024-04-21T08:37:15.556088+0000","last_became_active":"2024-04-21T08:36:23.995802+0000","last_became_peered":"2024-04-21T08:36:23.995802+0000","last_unstale":"2024-04-21T08:37:15.556088+0000","last_undegraded":"2024-04-21T08:37:15.556088+0000","last_fullsized":"2024-04-21T08:37:15.556088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:52:08.571567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.116331+0000","last_change":"2024-04-21T08:36:23.987275+0000","last_active":"2024-04-21T08:37:15.116331+0000","last_peered":"2024-04-21T08:37:15.116331+0000","last_clean":"2024-04-21T08:37:15.116331+0000","last_became_active":"2024-04-21T08:36:23.986967+0000","last_became_peered":"2024-04-21T08:36:23.986967+0000","last_unstale":"2024-04-21T08:37:15.116331+0000","last_undegraded":"2024-04-21T08:37:15.116331+0000","last_fullsized":"2024-04-21T08:37:15.116331+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:50.432502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.555792+0000","last_change":"2024-04-21T08:36:23.993046+0000","last_active":"2024-04-21T08:37:15.555792+0000","last_peered":"2024-04-21T08:37:15.555792+0000","last_clean":"2024-04-21T08:37:15.555792+0000","last_became_active":"2024-04-21T08:36:23.992693+0000","last_became_peered":"2024-04-21T08:36:23.992693+0000","last_unstale":"2024-04-21T08:37:15.555792+0000","last_undegraded":"2024-04-21T08:37:15.555792+0000","last_fullsized":"2024-04-21T08:37:15.555792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:12:06.287585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.119326+0000","last_change":"2024-04-21T08:36:23.992968+0000","last_active":"2024-04-21T08:37:15.119326+0000","last_peered":"2024-04-21T08:37:15.119326+0000","last_clean":"2024-04-21T08:37:15.119326+0000","last_became_active":"2024-04-21T08:36:23.992614+0000","last_became_peered":"2024-04-21T08:36:23.992614+0000","last_unstale":"2024-04-21T08:37:15.119326+0000","last_undegraded":"2024-04-21T08:37:15.119326+0000","last_fullsized":"2024-04-21T08:37:15.119326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:10.015323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.119980+0000","last_change":"2024-04-21T08:36:23.981169+0000","last_active":"2024-04-21T08:37:15.119980+0000","last_peered":"2024-04-21T08:37:15.119980+0000","last_clean":"2024-04-21T08:37:15.119980+0000","last_became_active":"2024-04-21T08:36:23.980849+0000","last_became_peered":"2024-04-21T08:36:23.980849+0000","last_unstale":"2024-04-21T08:37:15.119980+0000","last_undegraded":"2024-04-21T08:37:15.119980+0000","last_fullsized":"2024-04-21T08:37:15.119980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:13:49.907370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-21T08:37:15.218204+0000","last_change":"2024-04-21T08:36:23.993998+0000","last_active":"2024-04-21T08:37:15.218204+0000","last_peered":"2024-04-21T08:37:15.218204+0000","last_clean":"2024-04-21T08:37:15.218204+0000","last_became_active":"2024-04-21T08:36:23.993664+0000","last_became_peered":"2024-04-21T08:36:23.993664+0000","last_unstale":"2024-04-21T08:37:15.218204+0000","last_undegraded":"2024-04-21T08:37:15.218204+0000","last_fullsized":"2024-04-21T08:37:15.218204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:31:49.076820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70951,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":811,"num_read_kb":483,"num_write":243,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":212853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351011,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":506318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612648,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32876,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712276,"statfs":{"total":95995035648,"available":95961370624,"internally_reserved":0,"allocated":1740800,"data_stored":1156490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776173,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1073152,"data_stored":509953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037811,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":506235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299447,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1662976,"data_stored":1103123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462972,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":501361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":565747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":503208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":566749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":501069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":505826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724610,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712336,"statfs":{"total":95995035648,"available":95961432064,"internally_reserved":0,"allocated":1671168,"data_stored":1098314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:38:11.582 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-21T08:38:11.583 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-21T08:38:11.583 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-21T08:38:11.583 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph health --format=json 2024-04-21T08:38:11.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:11 smithi175 ceph-mon[31920]: pgmap v350: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-21T08:38:11.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:11 smithi175 ceph-mon[31920]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:11.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:11 smithi112 ceph-mon[31085]: pgmap v350: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-21T08:38:11.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:11 smithi112 ceph-mon[31085]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:11.838 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:13.603 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:13 smithi043 ceph-mon[27978]: pgmap v351: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-21T08:38:13.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.606+0000 7f63b7302700 1 -- 172.21.15.43:0/4238189695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b006bb70 msgr2=0x7f63b010f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:13.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.606+0000 7f63b7302700 1 --2- 172.21.15.43:0/4238189695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b006bb70 0x7f63b010f900 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f63a80099c0 tx=0x7f63a8009cd0 comp rx=0 tx=0).stop 2024-04-21T08:38:13.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.606+0000 7f63b7302700 1 -- 172.21.15.43:0/4238189695 shutdown_connections 2024-04-21T08:38:13.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.606+0000 7f63b7302700 1 --2- 172.21.15.43:0/4238189695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63b0108a50 0x7f63b0108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:13.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.606+0000 7f63b7302700 1 --2- 172.21.15.43:0/4238189695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b006bb70 0x7f63b010f900 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:13.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.606+0000 7f63b7302700 1 --2- 172.21.15.43:0/4238189695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63b006b1b0 0x7f63b006b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:13.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.606+0000 7f63b7302700 1 -- 172.21.15.43:0/4238189695 >> 172.21.15.43:0/4238189695 conn(0x7f63b0077990 msgr2=0x7f63b0077da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:13.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.606+0000 7f63b7302700 1 -- 172.21.15.43:0/4238189695 shutdown_connections 2024-04-21T08:38:13.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.606+0000 7f63b7302700 1 -- 172.21.15.43:0/4238189695 wait complete. 2024-04-21T08:38:13.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.607+0000 7f63b7302700 1 Processor -- start 2024-04-21T08:38:13.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.607+0000 7f63b7302700 1 -- start start 2024-04-21T08:38:13.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.607+0000 7f63b7302700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63b006b1b0 0x7f63b01990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:13.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.607+0000 7f63b7302700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b006bb70 0x7f63b0199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:13.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.607+0000 7f63b7302700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63b0108a50 0x7f63b019d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:13.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.607+0000 7f63b7302700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f63b0112c10 con 0x7f63b006bb70 2024-04-21T08:38:13.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.607+0000 7f63b7302700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63b0112a90 con 0x7f63b006b1b0 2024-04-21T08:38:13.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.607+0000 7f63b7302700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f63b0112d90 con 0x7f63b0108a50 2024-04-21T08:38:13.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.607+0000 7f63b489d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b006bb70 0x7f63b0199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:13.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.607+0000 7f63b489d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b006bb70 0x7f63b0199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38558/0 (socket says 172.21.15.43:38558) 2024-04-21T08:38:13.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.607+0000 7f63b489d700 1 -- 172.21.15.43:0/1512909526 learned_addr learned my addr 172.21.15.43:0/1512909526 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:13.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.607+0000 7f63b509e700 1 --2- 172.21.15.43:0/1512909526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63b006b1b0 0x7f63b01990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:13.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.607+0000 7f63b589f700 1 --2- 172.21.15.43:0/1512909526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63b0108a50 0x7f63b019d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:13.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.607+0000 7f63b489d700 1 -- 172.21.15.43:0/1512909526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63b0108a50 msgr2=0x7f63b019d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:13.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.607+0000 7f63b489d700 1 --2- 172.21.15.43:0/1512909526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63b0108a50 0x7f63b019d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:13.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.607+0000 7f63b489d700 1 -- 172.21.15.43:0/1512909526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63b006b1b0 msgr2=0x7f63b01990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:13.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.607+0000 7f63b489d700 1 --2- 172.21.15.43:0/1512909526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63b006b1b0 0x7f63b01990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:13.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.607+0000 7f63b489d700 1 -- 172.21.15.43:0/1512909526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b019e0a0 con 0x7f63b006bb70 2024-04-21T08:38:13.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.608+0000 7f63b489d700 1 --2- 172.21.15.43:0/1512909526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b006bb70 0x7f63b0199620 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f63a400d950 tx=0x7f63a400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:13.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.608+0000 7f63ae7fc700 1 -- 172.21.15.43:0/1512909526 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4016070 con 0x7f63b006bb70 2024-04-21T08:38:13.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.608+0000 7f63b7302700 1 -- 172.21.15.43:0/1512909526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b019e390 con 0x7f63b006bb70 2024-04-21T08:38:13.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.608+0000 7f63ae7fc700 1 -- 172.21.15.43:0/1512909526 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63a400fd40 con 0x7f63b006bb70 2024-04-21T08:38:13.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.608+0000 7f63ae7fc700 1 -- 172.21.15.43:0/1512909526 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a400e3b0 con 0x7f63b006bb70 2024-04-21T08:38:13.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.608+0000 7f63b7302700 1 -- 172.21.15.43:0/1512909526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b01a5c10 con 0x7f63b006bb70 2024-04-21T08:38:13.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.609+0000 7f63b7302700 1 -- 172.21.15.43:0/1512909526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b0069730 con 0x7f63b006bb70 2024-04-21T08:38:13.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.610+0000 7f63ae7fc700 1 -- 172.21.15.43:0/1512909526 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f63a4019070 con 0x7f63b006bb70 2024-04-21T08:38:13.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.610+0000 7f63ae7fc700 1 --2- 172.21.15.43:0/1512909526 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f63980a3cd0 0x7f63980a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:13.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.611+0000 7f63b509e700 1 --2- 172.21.15.43:0/1512909526 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f63980a3cd0 0x7f63980a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:13.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.611+0000 7f63ae7fc700 1 -- 172.21.15.43:0/1512909526 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f63a40d0300 con 0x7f63b006bb70 2024-04-21T08:38:13.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.611+0000 7f63b509e700 1 --2- 172.21.15.43:0/1512909526 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f63980a3cd0 0x7f63980a6190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009550 tx=0x7f63a00091f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:13.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.615+0000 7f63ae7fc700 1 -- 172.21.15.43:0/1512909526 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f63a409c8c0 con 0x7f63b006bb70 2024-04-21T08:38:13.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:13 smithi175 ceph-mon[31920]: pgmap v351: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-21T08:38:13.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:13 smithi112 ceph-mon[31085]: pgmap v351: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-21T08:38:13.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.871+0000 7f63b7302700 1 -- 172.21.15.43:0/1512909526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f63b019a3f0 con 0x7f63b006bb70 2024-04-21T08:38:13.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.871+0000 7f63ae7fc700 1 -- 172.21.15.43:0/1512909526 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f63a409c1c0 con 0x7f63b006bb70 2024-04-21T08:38:13.871 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:13.871 INFO:teuthology.orchestra.run.smithi043.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-21T08:38:13.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.873+0000 7f63b7302700 1 -- 172.21.15.43:0/1512909526 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f63980a3cd0 msgr2=0x7f63980a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:13.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.873+0000 7f63b7302700 1 --2- 172.21.15.43:0/1512909526 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f63980a3cd0 0x7f63980a6190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009550 tx=0x7f63a00091f0 comp rx=0 tx=0).stop 2024-04-21T08:38:13.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.873+0000 7f63b7302700 1 -- 172.21.15.43:0/1512909526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b006bb70 msgr2=0x7f63b0199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:13.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.873+0000 7f63b7302700 1 --2- 172.21.15.43:0/1512909526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b006bb70 0x7f63b0199620 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f63a400d950 tx=0x7f63a400dc60 comp rx=0 tx=0).stop 2024-04-21T08:38:13.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.873+0000 7f63b7302700 1 -- 172.21.15.43:0/1512909526 shutdown_connections 2024-04-21T08:38:13.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.873+0000 7f63b7302700 1 --2- 172.21.15.43:0/1512909526 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f63980a3cd0 0x7f63980a6190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:13.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.873+0000 7f63b7302700 1 --2- 172.21.15.43:0/1512909526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63b0108a50 0x7f63b019d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:13.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.873+0000 7f63b7302700 1 --2- 172.21.15.43:0/1512909526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63b006b1b0 0x7f63b01990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:13.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.873+0000 7f63b7302700 1 --2- 172.21.15.43:0/1512909526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b006bb70 0x7f63b0199620 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:13.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.873+0000 7f63b7302700 1 -- 172.21.15.43:0/1512909526 >> 172.21.15.43:0/1512909526 conn(0x7f63b0077990 msgr2=0x7f63b0100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:13.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.873+0000 7f63b7302700 1 -- 172.21.15.43:0/1512909526 shutdown_connections 2024-04-21T08:38:13.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:13.873+0000 7f63b7302700 1 -- 172.21.15.43:0/1512909526 wait complete. 2024-04-21T08:38:14.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:14 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1512909526' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-21T08:38:14.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2024-04-21T08:38:14.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-04-21T08:38:14.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]: dispatch 2024-04-21T08:38:14.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-04-21T08:38:14.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:14 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-04-21T08:38:14.678 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-21T08:38:14.678 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:38:14.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:14 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1512909526' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-21T08:38:14.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2024-04-21T08:38:14.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-04-21T08:38:14.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]: dispatch 2024-04-21T08:38:14.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-04-21T08:38:14.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:14 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-04-21T08:38:14.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.785+0000 7fde2284e640 1 -- 172.21.15.43:0/3387159775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c103b50 msgr2=0x7fde1c10a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:14.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.785+0000 7fde2284e640 1 --2- 172.21.15.43:0/3387159775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c103b50 0x7fde1c10a3e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fde14009ca0 tx=0x7fde1402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:14.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.785+0000 7fde2284e640 1 -- 172.21.15.43:0/3387159775 shutdown_connections 2024-04-21T08:38:14.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.785+0000 7fde2284e640 1 --2- 172.21.15.43:0/3387159775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c103b50 0x7fde1c10a3e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:14.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.785+0000 7fde2284e640 1 --2- 172.21.15.43:0/3387159775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde1c103190 0x7fde1c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:14.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.785+0000 7fde2284e640 1 --2- 172.21.15.43:0/3387159775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde1c101fa0 0x7fde1c1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:14.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.785+0000 7fde2284e640 1 -- 172.21.15.43:0/3387159775 >> 172.21.15.43:0/3387159775 conn(0x7fde1c0fd530 msgr2=0x7fde1c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:14.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.785+0000 7fde2284e640 1 -- 172.21.15.43:0/3387159775 shutdown_connections 2024-04-21T08:38:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.785+0000 7fde2284e640 1 -- 172.21.15.43:0/3387159775 wait complete. 2024-04-21T08:38:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.785+0000 7fde2284e640 1 Processor -- start 2024-04-21T08:38:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.785+0000 7fde2284e640 1 -- start start 2024-04-21T08:38:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.786+0000 7fde2284e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde1c101fa0 0x7fde1c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.786+0000 7fde2284e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c103190 0x7fde1c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.786+0000 7fde2284e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde1c103b50 0x7fde1c19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.786+0000 7fde2284e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c10db60 con 0x7fde1c103190 2024-04-21T08:38:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.786+0000 7fde2284e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c10d9e0 con 0x7fde1c103b50 2024-04-21T08:38:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.786+0000 7fde2284e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c10dce0 con 0x7fde1c101fa0 2024-04-21T08:38:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.786+0000 7fde1b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c103190 0x7fde1c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.786+0000 7fde1b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c103190 0x7fde1c1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38578/0 (socket says 172.21.15.43:38578) 2024-04-21T08:38:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.786+0000 7fde1b7fe640 1 -- 172.21.15.43:0/1676865667 learned_addr learned my addr 172.21.15.43:0/1676865667 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.786+0000 7fde1bfff640 1 --2- 172.21.15.43:0/1676865667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde1c101fa0 0x7fde1c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.786+0000 7fde20deb640 1 --2- 172.21.15.43:0/1676865667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde1c103b50 0x7fde1c19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.786+0000 7fde1b7fe640 1 -- 172.21.15.43:0/1676865667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde1c101fa0 msgr2=0x7fde1c1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:14.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.786+0000 7fde1b7fe640 1 --2- 172.21.15.43:0/1676865667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde1c101fa0 0x7fde1c1981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:14.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.786+0000 7fde1b7fe640 1 -- 172.21.15.43:0/1676865667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde1c103b50 msgr2=0x7fde1c19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:14.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.786+0000 7fde1b7fe640 1 --2- 172.21.15.43:0/1676865667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde1c103b50 0x7fde1c19f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:14.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.786+0000 7fde1b7fe640 1 -- 172.21.15.43:0/1676865667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde1c19fdd0 con 0x7fde1c103190 2024-04-21T08:38:14.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.786+0000 7fde1b7fe640 1 --2- 172.21.15.43:0/1676865667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c103190 0x7fde1c1986f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fde1000c970 tx=0x7fde1000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:14.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.786+0000 7fde197fa640 1 -- 172.21.15.43:0/1676865667 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde10007bb0 con 0x7fde1c103190 2024-04-21T08:38:14.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.786+0000 7fde197fa640 1 -- 172.21.15.43:0/1676865667 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde10007d50 con 0x7fde1c103190 2024-04-21T08:38:14.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.786+0000 7fde2284e640 1 -- 172.21.15.43:0/1676865667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde1c1a00c0 con 0x7fde1c103190 2024-04-21T08:38:14.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.786+0000 7fde197fa640 1 -- 172.21.15.43:0/1676865667 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1000f450 con 0x7fde1c103190 2024-04-21T08:38:14.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.786+0000 7fde2284e640 1 -- 172.21.15.43:0/1676865667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde1c1a05c0 con 0x7fde1c103190 2024-04-21T08:38:14.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.787+0000 7fde2284e640 1 -- 172.21.15.43:0/1676865667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde1c1092c0 con 0x7fde1c103190 2024-04-21T08:38:14.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.789+0000 7fde197fa640 1 -- 172.21.15.43:0/1676865667 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fde10004f00 con 0x7fde1c103190 2024-04-21T08:38:14.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.789+0000 7fde197fa640 1 --2- 172.21.15.43:0/1676865667 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fde040a3bb0 0x7fde040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:14.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.789+0000 7fde1bfff640 1 --2- 172.21.15.43:0/1676865667 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fde040a3bb0 0x7fde040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:14.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.789+0000 7fde197fa640 1 -- 172.21.15.43:0/1676865667 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7fde100d09f0 con 0x7fde1c103190 2024-04-21T08:38:14.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.790+0000 7fde1bfff640 1 --2- 172.21.15.43:0/1676865667 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fde040a3bb0 0x7fde040a6070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fde1c102ff0 tx=0x7fde0c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:14.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.793+0000 7fde197fa640 1 -- 172.21.15.43:0/1676865667 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde10014030 con 0x7fde1c103190 2024-04-21T08:38:14.793 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:14 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1512909526' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-21T08:38:14.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2024-04-21T08:38:14.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-04-21T08:38:14.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]: dispatch 2024-04-21T08:38:14.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-04-21T08:38:14.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:14 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-04-21T08:38:14.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.982+0000 7fde2284e640 1 -- 172.21.15.43:0/1676865667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde1c02d810 con 0x7fde1c103190 2024-04-21T08:38:14.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.984+0000 7fde197fa640 1 -- 172.21.15.43:0/1676865667 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24856 (secure 0 0 0) 0x7fde1009cfe0 con 0x7fde1c103190 2024-04-21T08:38:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:14.984 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":93,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:38:14.324456+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:38:14.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.986+0000 7fde2284e640 1 -- 172.21.15.43:0/1676865667 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fde040a3bb0 msgr2=0x7fde040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:14.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.986+0000 7fde2284e640 1 --2- 172.21.15.43:0/1676865667 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fde040a3bb0 0x7fde040a6070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fde1c102ff0 tx=0x7fde0c008040 comp rx=0 tx=0).stop 2024-04-21T08:38:14.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.986+0000 7fde2284e640 1 -- 172.21.15.43:0/1676865667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c103190 msgr2=0x7fde1c1986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:14.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.986+0000 7fde2284e640 1 --2- 172.21.15.43:0/1676865667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c103190 0x7fde1c1986f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fde1000c970 tx=0x7fde1000ce30 comp rx=0 tx=0).stop 2024-04-21T08:38:14.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.986+0000 7fde2284e640 1 -- 172.21.15.43:0/1676865667 shutdown_connections 2024-04-21T08:38:14.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.986+0000 7fde2284e640 1 --2- 172.21.15.43:0/1676865667 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fde040a3bb0 0x7fde040a6070 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:14.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.986+0000 7fde2284e640 1 --2- 172.21.15.43:0/1676865667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde1c103b50 0x7fde1c19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:14.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.986+0000 7fde2284e640 1 --2- 172.21.15.43:0/1676865667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c103190 0x7fde1c1986f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:14.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.986+0000 7fde2284e640 1 --2- 172.21.15.43:0/1676865667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde1c101fa0 0x7fde1c1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:14.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.986+0000 7fde2284e640 1 -- 172.21.15.43:0/1676865667 >> 172.21.15.43:0/1676865667 conn(0x7fde1c0fd530 msgr2=0x7fde1c106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:14.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.986+0000 7fde2284e640 1 -- 172.21.15.43:0/1676865667 shutdown_connections 2024-04-21T08:38:14.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:14.986+0000 7fde2284e640 1 -- 172.21.15.43:0/1676865667 wait complete. 2024-04-21T08:38:14.998 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:39.981405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:22.034751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:22.780795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:25.984261+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:38:14.999 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:38:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.104+0000 7fb701848640 1 -- 172.21.15.43:0/1393747744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6fc101b80 msgr2=0x7fb6fc102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.104+0000 7fb701848640 1 --2- 172.21.15.43:0/1393747744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6fc101b80 0x7fb6fc102000 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009900 tx=0x7fb6f402bcd0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.105+0000 7fb701848640 1 -- 172.21.15.43:0/1393747744 shutdown_connections 2024-04-21T08:38:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.105+0000 7fb701848640 1 --2- 172.21.15.43:0/1393747744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6fc101b80 0x7fb6fc102000 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.105+0000 7fb701848640 1 --2- 172.21.15.43:0/1393747744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6fc1011e0 0x7fb6fc101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.105+0000 7fb701848640 1 --2- 172.21.15.43:0/1393747744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6fc100880 0x7fb6fc100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.105+0000 7fb701848640 1 -- 172.21.15.43:0/1393747744 >> 172.21.15.43:0/1393747744 conn(0x7fb6fc0fc400 msgr2=0x7fb6fc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:15.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.105+0000 7fb701848640 1 -- 172.21.15.43:0/1393747744 shutdown_connections 2024-04-21T08:38:15.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.105+0000 7fb701848640 1 -- 172.21.15.43:0/1393747744 wait complete. 2024-04-21T08:38:15.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.105+0000 7fb701848640 1 Processor -- start 2024-04-21T08:38:15.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.105+0000 7fb701848640 1 -- start start 2024-04-21T08:38:15.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.105+0000 7fb701848640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6fc100880 0x7fb6fc19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:15.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.105+0000 7fb701848640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6fc1011e0 0x7fb6fc19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:15.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.105+0000 7fb701848640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6fc101b80 0x7fb6fc1a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:15.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.105+0000 7fb701848640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc10fd90 con 0x7fb6fc101b80 2024-04-21T08:38:15.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.105+0000 7fb701848640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc10fc10 con 0x7fb6fc100880 2024-04-21T08:38:15.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.105+0000 7fb701848640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc10ff10 con 0x7fb6fc1011e0 2024-04-21T08:38:15.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.106+0000 7fb6fb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6fc101b80 0x7fb6fc1a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:15.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.106+0000 7fb6fb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6fc101b80 0x7fb6fc1a1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38596/0 (socket says 172.21.15.43:38596) 2024-04-21T08:38:15.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.106+0000 7fb6fb7fe640 1 -- 172.21.15.43:0/828271730 learned_addr learned my addr 172.21.15.43:0/828271730 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:15.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.106+0000 7fb6faffd640 1 --2- 172.21.15.43:0/828271730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6fc100880 0x7fb6fc19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:15.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.106+0000 7fb6fa7fc640 1 --2- 172.21.15.43:0/828271730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6fc1011e0 0x7fb6fc19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:15.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.106+0000 7fb6fb7fe640 1 -- 172.21.15.43:0/828271730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6fc1011e0 msgr2=0x7fb6fc19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:15.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.106+0000 7fb6fb7fe640 1 --2- 172.21.15.43:0/828271730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6fc1011e0 0x7fb6fc19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.106+0000 7fb6fb7fe640 1 -- 172.21.15.43:0/828271730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6fc100880 msgr2=0x7fb6fc19a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:15.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.106+0000 7fb6fb7fe640 1 --2- 172.21.15.43:0/828271730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6fc100880 0x7fb6fc19a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.106+0000 7fb6fb7fe640 1 -- 172.21.15.43:0/828271730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6fc1a2140 con 0x7fb6fc101b80 2024-04-21T08:38:15.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.106+0000 7fb6fb7fe640 1 --2- 172.21.15.43:0/828271730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6fc101b80 0x7fb6fc1a1ad0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009230 tx=0x7fb6f402dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:15.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.106+0000 7fb700846640 1 -- 172.21.15.43:0/828271730 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4043020 con 0x7fb6fc101b80 2024-04-21T08:38:15.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.106+0000 7fb700846640 1 -- 172.21.15.43:0/828271730 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6f402de30 con 0x7fb6fc101b80 2024-04-21T08:38:15.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.106+0000 7fb701848640 1 -- 172.21.15.43:0/828271730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6fc1a23d0 con 0x7fb6fc101b80 2024-04-21T08:38:15.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.106+0000 7fb700846640 1 -- 172.21.15.43:0/828271730 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4003c20 con 0x7fb6fc101b80 2024-04-21T08:38:15.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.106+0000 7fb701848640 1 -- 172.21.15.43:0/828271730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6fc1a29a0 con 0x7fb6fc101b80 2024-04-21T08:38:15.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.107+0000 7fb701848640 1 -- 172.21.15.43:0/828271730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6fc10b8a0 con 0x7fb6fc101b80 2024-04-21T08:38:15.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.108+0000 7fb700846640 1 -- 172.21.15.43:0/828271730 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb6f4046020 con 0x7fb6fc101b80 2024-04-21T08:38:15.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.109+0000 7fb700846640 1 --2- 172.21.15.43:0/828271730 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb6e40a3bb0 0x7fb6e40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:15.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.109+0000 7fb6faffd640 1 --2- 172.21.15.43:0/828271730 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb6e40a3bb0 0x7fb6e40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:15.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.109+0000 7fb700846640 1 -- 172.21.15.43:0/828271730 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7fb6f40f2c90 con 0x7fb6fc101b80 2024-04-21T08:38:15.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.110+0000 7fb6faffd640 1 --2- 172.21.15.43:0/828271730 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb6e40a3bb0 0x7fb6e40a6070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00a400 tx=0x7fb6ec00d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:15.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.114+0000 7fb700846640 1 -- 172.21.15.43:0/828271730 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6f404d020 con 0x7fb6fc101b80 2024-04-21T08:38:15.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.303+0000 7fb701848640 1 -- 172.21.15.43:0/828271730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb6fc062b50 con 0x7fb6fc101b80 2024-04-21T08:38:15.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.304+0000 7fb700846640 1 -- 172.21.15.43:0/828271730 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fb6f40bf280 con 0x7fb6fc101b80 2024-04-21T08:38:15.304 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-21T08:38:15.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.306+0000 7fb701848640 1 -- 172.21.15.43:0/828271730 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb6e40a3bb0 msgr2=0x7fb6e40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.306+0000 7fb701848640 1 --2- 172.21.15.43:0/828271730 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb6e40a3bb0 0x7fb6e40a6070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00a400 tx=0x7fb6ec00d040 comp rx=0 tx=0).stop 2024-04-21T08:38:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.306+0000 7fb701848640 1 -- 172.21.15.43:0/828271730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6fc101b80 msgr2=0x7fb6fc1a1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.306+0000 7fb701848640 1 --2- 172.21.15.43:0/828271730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6fc101b80 0x7fb6fc1a1ad0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009230 tx=0x7fb6f402dc40 comp rx=0 tx=0).stop 2024-04-21T08:38:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.306+0000 7fb701848640 1 -- 172.21.15.43:0/828271730 shutdown_connections 2024-04-21T08:38:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.306+0000 7fb701848640 1 --2- 172.21.15.43:0/828271730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6fc101b80 0x7fb6fc1a1ad0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.306+0000 7fb701848640 1 --2- 172.21.15.43:0/828271730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6fc1011e0 0x7fb6fc19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.306+0000 7fb701848640 1 --2- 172.21.15.43:0/828271730 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb6e40a3bb0 0x7fb6e40a6070 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.306+0000 7fb701848640 1 --2- 172.21.15.43:0/828271730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6fc100880 0x7fb6fc19a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.306+0000 7fb701848640 1 -- 172.21.15.43:0/828271730 >> 172.21.15.43:0/828271730 conn(0x7fb6fc0fc400 msgr2=0x7fb6fc0fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.306+0000 7fb701848640 1 -- 172.21.15.43:0/828271730 shutdown_connections 2024-04-21T08:38:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.306+0000 7fb701848640 1 -- 172.21.15.43:0/828271730 wait complete. 2024-04-21T08:38:15.319 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:38:15.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.428+0000 7fdb19611640 1 -- 172.21.15.43:0/538386297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb14107d30 msgr2=0x7fdb1410a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.428+0000 7fdb19611640 1 --2- 172.21.15.43:0/538386297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb14107d30 0x7fdb1410a120 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009ca0 tx=0x7fdb0c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.428+0000 7fdb19611640 1 -- 172.21.15.43:0/538386297 shutdown_connections 2024-04-21T08:38:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.428+0000 7fdb19611640 1 --2- 172.21.15.43:0/538386297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb14107d30 0x7fdb1410a120 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.428+0000 7fdb19611640 1 --2- 172.21.15.43:0/538386297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb14105400 0x7fdb141077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.428+0000 7fdb19611640 1 --2- 172.21.15.43:0/538386297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb14102ad0 0x7fdb14104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.428+0000 7fdb19611640 1 -- 172.21.15.43:0/538386297 >> 172.21.15.43:0/538386297 conn(0x7fdb140fc440 msgr2=0x7fdb140fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.428+0000 7fdb19611640 1 -- 172.21.15.43:0/538386297 shutdown_connections 2024-04-21T08:38:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.428+0000 7fdb19611640 1 -- 172.21.15.43:0/538386297 wait complete. 2024-04-21T08:38:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.429+0000 7fdb19611640 1 Processor -- start 2024-04-21T08:38:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.429+0000 7fdb19611640 1 -- start start 2024-04-21T08:38:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.429+0000 7fdb19611640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb14102ad0 0x7fdb141981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.429+0000 7fdb19611640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb14105400 0x7fdb14198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.429+0000 7fdb19611640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb14107d30 0x7fdb1419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.429+0000 7fdb19611640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1410db10 con 0x7fdb14102ad0 2024-04-21T08:38:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.429+0000 7fdb19611640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1410d990 con 0x7fdb14107d30 2024-04-21T08:38:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.429+0000 7fdb19611640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1410dc90 con 0x7fdb14105400 2024-04-21T08:38:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.429+0000 7fdb12ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb14102ad0 0x7fdb141981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.429+0000 7fdb12ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb14102ad0 0x7fdb141981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38616/0 (socket says 172.21.15.43:38616) 2024-04-21T08:38:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.429+0000 7fdb12ffd640 1 -- 172.21.15.43:0/1687031538 learned_addr learned my addr 172.21.15.43:0/1687031538 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.429+0000 7fdb137fe640 1 --2- 172.21.15.43:0/1687031538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb14107d30 0x7fdb1419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.429+0000 7fdb127fc640 1 --2- 172.21.15.43:0/1687031538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb14105400 0x7fdb14198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.429+0000 7fdb12ffd640 1 -- 172.21.15.43:0/1687031538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb14105400 msgr2=0x7fdb14198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.429+0000 7fdb12ffd640 1 --2- 172.21.15.43:0/1687031538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb14105400 0x7fdb14198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.429+0000 7fdb12ffd640 1 -- 172.21.15.43:0/1687031538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb14107d30 msgr2=0x7fdb1419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.429+0000 7fdb12ffd640 1 --2- 172.21.15.43:0/1687031538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb14107d30 0x7fdb1419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.429+0000 7fdb12ffd640 1 -- 172.21.15.43:0/1687031538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1419fdf0 con 0x7fdb14102ad0 2024-04-21T08:38:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.429+0000 7fdb127fc640 1 --2- 172.21.15.43:0/1687031538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb14105400 0x7fdb14198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.430+0000 7fdb12ffd640 1 --2- 172.21.15.43:0/1687031538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb14102ad0 0x7fdb141981d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400c970 tx=0x7fdb0400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:15.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.430+0000 7fdafbfff640 1 -- 172.21.15.43:0/1687031538 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04007bb0 con 0x7fdb14102ad0 2024-04-21T08:38:15.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.430+0000 7fdb19611640 1 -- 172.21.15.43:0/1687031538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb141a00e0 con 0x7fdb14102ad0 2024-04-21T08:38:15.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.430+0000 7fdafbfff640 1 -- 172.21.15.43:0/1687031538 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb04007d50 con 0x7fdb14102ad0 2024-04-21T08:38:15.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.430+0000 7fdafbfff640 1 -- 172.21.15.43:0/1687031538 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0400f450 con 0x7fdb14102ad0 2024-04-21T08:38:15.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.430+0000 7fdb19611640 1 -- 172.21.15.43:0/1687031538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb141a0630 con 0x7fdb14102ad0 2024-04-21T08:38:15.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.431+0000 7fdb19611640 1 -- 172.21.15.43:0/1687031538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb140fdf30 con 0x7fdb14102ad0 2024-04-21T08:38:15.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.432+0000 7fdafbfff640 1 -- 172.21.15.43:0/1687031538 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdb04004f20 con 0x7fdb14102ad0 2024-04-21T08:38:15.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.433+0000 7fdafbfff640 1 --2- 172.21.15.43:0/1687031538 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fdafc0a3bb0 0x7fdafc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:15.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.433+0000 7fdb127fc640 1 --2- 172.21.15.43:0/1687031538 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fdafc0a3bb0 0x7fdafc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:15.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.433+0000 7fdafbfff640 1 -- 172.21.15.43:0/1687031538 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fdb040d0560 con 0x7fdb14102ad0 2024-04-21T08:38:15.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.433+0000 7fdb127fc640 1 --2- 172.21.15.43:0/1687031538 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fdafc0a3bb0 0x7fdafc0a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdb141996e0 tx=0x7fdb0800b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:15.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.438+0000 7fdafbfff640 1 -- 172.21.15.43:0/1687031538 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb04014030 con 0x7fdb14102ad0 2024-04-21T08:38:15.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:15 smithi043 ceph-mon[27978]: pgmap v352: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-21T08:38:15.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-04-21T08:38:15.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-04-21T08:38:15.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]': finished 2024-04-21T08:38:15.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-04-21T08:38:15.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:15 smithi043 ceph-mon[27978]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-04-21T08:38:15.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:15 smithi043 ceph-mon[27978]: osdmap e93: 12 total, 12 up, 12 in 2024-04-21T08:38:15.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:15 smithi043 ceph-mon[27978]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-21T08:38:15.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:15 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1676865667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:15.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:15 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/828271730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:15.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:15 smithi043 ceph-mon[27978]: osdmap e94: 12 total, 12 up, 12 in 2024-04-21T08:38:15.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.628+0000 7fdb19611640 1 -- 172.21.15.43:0/1687031538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdb14062b50 con 0x7fdb14102ad0 2024-04-21T08:38:15.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.628+0000 7fdafbfff640 1 -- 172.21.15.43:0/1687031538 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fdb0409cb50 con 0x7fdb14102ad0 2024-04-21T08:38:15.629 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:38:15.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.630+0000 7fdb19611640 1 -- 172.21.15.43:0/1687031538 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fdafc0a3bb0 msgr2=0x7fdafc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:15.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.630+0000 7fdb19611640 1 --2- 172.21.15.43:0/1687031538 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fdafc0a3bb0 0x7fdafc0a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdb141996e0 tx=0x7fdb0800b5f0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.630+0000 7fdb19611640 1 -- 172.21.15.43:0/1687031538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb14102ad0 msgr2=0x7fdb141981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:15.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.630+0000 7fdb19611640 1 --2- 172.21.15.43:0/1687031538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb14102ad0 0x7fdb141981d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400c970 tx=0x7fdb0400ce30 comp rx=0 tx=0).stop 2024-04-21T08:38:15.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.630+0000 7fdb19611640 1 -- 172.21.15.43:0/1687031538 shutdown_connections 2024-04-21T08:38:15.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.630+0000 7fdb19611640 1 --2- 172.21.15.43:0/1687031538 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fdafc0a3bb0 0x7fdafc0a6070 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.630+0000 7fdb19611640 1 --2- 172.21.15.43:0/1687031538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb14107d30 0x7fdb1419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.630+0000 7fdb19611640 1 --2- 172.21.15.43:0/1687031538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb14105400 0x7fdb14198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.630+0000 7fdb19611640 1 --2- 172.21.15.43:0/1687031538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb14102ad0 0x7fdb141981d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.630+0000 7fdb19611640 1 -- 172.21.15.43:0/1687031538 >> 172.21.15.43:0/1687031538 conn(0x7fdb140fc440 msgr2=0x7fdb14103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:15.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.631+0000 7fdb19611640 1 -- 172.21.15.43:0/1687031538 shutdown_connections 2024-04-21T08:38:15.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.631+0000 7fdb19611640 1 -- 172.21.15.43:0/1687031538 wait complete. 2024-04-21T08:38:15.642 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:38:15.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:15 smithi175 ceph-mon[31920]: pgmap v352: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-21T08:38:15.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-04-21T08:38:15.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-04-21T08:38:15.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]': finished 2024-04-21T08:38:15.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-04-21T08:38:15.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:15 smithi175 ceph-mon[31920]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-04-21T08:38:15.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:15 smithi175 ceph-mon[31920]: osdmap e93: 12 total, 12 up, 12 in 2024-04-21T08:38:15.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:15 smithi175 ceph-mon[31920]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-21T08:38:15.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:15 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1676865667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:15.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:15 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/828271730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:15.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:15 smithi175 ceph-mon[31920]: osdmap e94: 12 total, 12 up, 12 in 2024-04-21T08:38:15.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.748+0000 7f407a243640 1 -- 172.21.15.43:0/1073380386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4074100e90 msgr2=0x7f40741012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:15.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.748+0000 7f407a243640 1 --2- 172.21.15.43:0/1073380386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4074100e90 0x7f40741012b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4064009900 tx=0x7f406402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:15.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.748+0000 7f407a243640 1 -- 172.21.15.43:0/1073380386 shutdown_connections 2024-04-21T08:38:15.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.748+0000 7f407a243640 1 --2- 172.21.15.43:0/1073380386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40741029a0 0x7f40741092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.748+0000 7f407a243640 1 --2- 172.21.15.43:0/1073380386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4074101fe0 0x7f4074102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.748+0000 7f407a243640 1 --2- 172.21.15.43:0/1073380386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4074100e90 0x7f40741012b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.748+0000 7f407a243640 1 -- 172.21.15.43:0/1073380386 >> 172.21.15.43:0/1073380386 conn(0x7f40740fc400 msgr2=0x7f40740fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:15.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.748+0000 7f407a243640 1 -- 172.21.15.43:0/1073380386 shutdown_connections 2024-04-21T08:38:15.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.748+0000 7f407a243640 1 -- 172.21.15.43:0/1073380386 wait complete. 2024-04-21T08:38:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.748+0000 7f407a243640 1 Processor -- start 2024-04-21T08:38:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.748+0000 7f407a243640 1 -- start start 2024-04-21T08:38:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.749+0000 7f407a243640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4074100e90 0x7f407419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.749+0000 7f407a243640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4074101fe0 0x7f407419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.749+0000 7f407a243640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40741029a0 0x7f40741a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.749+0000 7f407a243640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4074069c70 con 0x7f4074101fe0 2024-04-21T08:38:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.749+0000 7f407a243640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4074069af0 con 0x7f4074100e90 2024-04-21T08:38:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.749+0000 7f407a243640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4074069df0 con 0x7f40741029a0 2024-04-21T08:38:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.749+0000 7f4072ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4074101fe0 0x7f407419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.749+0000 7f4072ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4074101fe0 0x7f407419a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38630/0 (socket says 172.21.15.43:38630) 2024-04-21T08:38:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.749+0000 7f4072ffd640 1 -- 172.21.15.43:0/288929255 learned_addr learned my addr 172.21.15.43:0/288929255 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.749+0000 7f40737fe640 1 --2- 172.21.15.43:0/288929255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4074100e90 0x7f407419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.749+0000 7f4072ffd640 1 -- 172.21.15.43:0/288929255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40741029a0 msgr2=0x7f40741a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:15.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.749+0000 7f4073fff640 1 --2- 172.21.15.43:0/288929255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40741029a0 0x7f40741a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.749+0000 7f4072ffd640 1 --2- 172.21.15.43:0/288929255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40741029a0 0x7f40741a1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.749+0000 7f4072ffd640 1 -- 172.21.15.43:0/288929255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4074100e90 msgr2=0x7f407419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.749+0000 7f4072ffd640 1 --2- 172.21.15.43:0/288929255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4074100e90 0x7f407419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.749+0000 7f4072ffd640 1 -- 172.21.15.43:0/288929255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40741a20a0 con 0x7f4074101fe0 2024-04-21T08:38:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.749+0000 7f40737fe640 1 --2- 172.21.15.43:0/288929255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4074100e90 0x7f407419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.749+0000 7f4072ffd640 1 --2- 172.21.15.43:0/288929255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4074101fe0 0x7f407419a9c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f405c00c970 tx=0x7f405c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.749+0000 7f4070ff9640 1 -- 172.21.15.43:0/288929255 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c004220 con 0x7f4074101fe0 2024-04-21T08:38:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.749+0000 7f4070ff9640 1 -- 172.21.15.43:0/288929255 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f405c0043c0 con 0x7f4074101fe0 2024-04-21T08:38:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.749+0000 7f4070ff9640 1 -- 172.21.15.43:0/288929255 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c004a00 con 0x7f4074101fe0 2024-04-21T08:38:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.749+0000 7f407a243640 1 -- 172.21.15.43:0/288929255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40741a2390 con 0x7f4074101fe0 2024-04-21T08:38:15.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.749+0000 7f407a243640 1 -- 172.21.15.43:0/288929255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40741a2890 con 0x7f4074101fe0 2024-04-21T08:38:15.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.750+0000 7f407a243640 1 -- 172.21.15.43:0/288929255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40741a2b50 con 0x7f4074101fe0 2024-04-21T08:38:15.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.752+0000 7f4070ff9640 1 -- 172.21.15.43:0/288929255 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f405c0079b0 con 0x7f4074101fe0 2024-04-21T08:38:15.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.752+0000 7f4070ff9640 1 --2- 172.21.15.43:0/288929255 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f40600a3bb0 0x7f40600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:15.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.753+0000 7f40737fe640 1 --2- 172.21.15.43:0/288929255 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f40600a3bb0 0x7f40600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:15.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.753+0000 7f4070ff9640 1 -- 172.21.15.43:0/288929255 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f405c0d0600 con 0x7f4074101fe0 2024-04-21T08:38:15.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.753+0000 7f40737fe640 1 --2- 172.21.15.43:0/288929255 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f40600a3bb0 0x7f40600a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f40640096c0 tx=0x7f4064000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:15.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.757+0000 7f4070ff9640 1 -- 172.21.15.43:0/288929255 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f405c014030 con 0x7f4074101fe0 2024-04-21T08:38:15.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:15 smithi112 ceph-mon[31085]: pgmap v352: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-21T08:38:15.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-04-21T08:38:15.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-04-21T08:38:15.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]': finished 2024-04-21T08:38:15.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-04-21T08:38:15.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:15 smithi112 ceph-mon[31085]: from='mgr.14150 172.21.15.43:0/2798543409' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-04-21T08:38:15.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:15 smithi112 ceph-mon[31085]: osdmap e93: 12 total, 12 up, 12 in 2024-04-21T08:38:15.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:15 smithi112 ceph-mon[31085]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-21T08:38:15.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:15 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1676865667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:15.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:15 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/828271730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:15.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:15 smithi112 ceph-mon[31085]: osdmap e94: 12 total, 12 up, 12 in 2024-04-21T08:38:15.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.946+0000 7f407a243640 1 -- 172.21.15.43:0/288929255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f40741a3380 con 0x7f4074101fe0 2024-04-21T08:38:15.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.946+0000 7f4070ff9640 1 -- 172.21.15.43:0/288929255 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f405c09cbf0 con 0x7f4074101fe0 2024-04-21T08:38:15.947 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-21T08:38:15.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.948+0000 7f407a243640 1 -- 172.21.15.43:0/288929255 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f40600a3bb0 msgr2=0x7f40600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:15.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.948+0000 7f407a243640 1 --2- 172.21.15.43:0/288929255 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f40600a3bb0 0x7f40600a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f40640096c0 tx=0x7f4064000bc0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.948+0000 7f407a243640 1 -- 172.21.15.43:0/288929255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4074101fe0 msgr2=0x7f407419a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:15.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.948+0000 7f407a243640 1 --2- 172.21.15.43:0/288929255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4074101fe0 0x7f407419a9c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f405c00c970 tx=0x7f405c00ce30 comp rx=0 tx=0).stop 2024-04-21T08:38:15.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.949+0000 7f407a243640 1 -- 172.21.15.43:0/288929255 shutdown_connections 2024-04-21T08:38:15.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.949+0000 7f407a243640 1 --2- 172.21.15.43:0/288929255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40741029a0 0x7f40741a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.949+0000 7f407a243640 1 --2- 172.21.15.43:0/288929255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4074101fe0 0x7f407419a9c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.949+0000 7f407a243640 1 --2- 172.21.15.43:0/288929255 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f40600a3bb0 0x7f40600a6070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.949+0000 7f407a243640 1 --2- 172.21.15.43:0/288929255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4074100e90 0x7f407419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:15.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.949+0000 7f407a243640 1 -- 172.21.15.43:0/288929255 >> 172.21.15.43:0/288929255 conn(0x7f40740fc400 msgr2=0x7f407410be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:15.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.949+0000 7f407a243640 1 -- 172.21.15.43:0/288929255 shutdown_connections 2024-04-21T08:38:15.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:15.949+0000 7f407a243640 1 -- 172.21.15.43:0/288929255 wait complete. 2024-04-21T08:38:15.960 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:38:16.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.068+0000 7fb818481640 1 -- 172.21.15.43:0/865746092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb810100e90 msgr2=0x7fb8101012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:16.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.068+0000 7fb818481640 1 --2- 172.21.15.43:0/865746092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb810100e90 0x7fb8101012b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb80c009900 tx=0x7fb80c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:16.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.068+0000 7fb818481640 1 -- 172.21.15.43:0/865746092 shutdown_connections 2024-04-21T08:38:16.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.068+0000 7fb818481640 1 --2- 172.21.15.43:0/865746092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8101029a0 0x7fb8101092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.068+0000 7fb818481640 1 --2- 172.21.15.43:0/865746092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb810101fe0 0x7fb810102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.068+0000 7fb818481640 1 --2- 172.21.15.43:0/865746092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb810100e90 0x7fb8101012b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.068+0000 7fb818481640 1 -- 172.21.15.43:0/865746092 >> 172.21.15.43:0/865746092 conn(0x7fb8100fc400 msgr2=0x7fb8100fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:16.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.068+0000 7fb818481640 1 -- 172.21.15.43:0/865746092 shutdown_connections 2024-04-21T08:38:16.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.068+0000 7fb818481640 1 -- 172.21.15.43:0/865746092 wait complete. 2024-04-21T08:38:16.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.068+0000 7fb818481640 1 Processor -- start 2024-04-21T08:38:16.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.069+0000 7fb818481640 1 -- start start 2024-04-21T08:38:16.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.069+0000 7fb818481640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb810100e90 0x7fb810198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:16.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.069+0000 7fb818481640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb810101fe0 0x7fb810198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:16.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.069+0000 7fb818481640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8101029a0 0x7fb81019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:16.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.069+0000 7fb818481640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb81010c9c0 con 0x7fb8101029a0 2024-04-21T08:38:16.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.069+0000 7fb818481640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb81010c840 con 0x7fb810100e90 2024-04-21T08:38:16.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.069+0000 7fb818481640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb81010cb40 con 0x7fb810101fe0 2024-04-21T08:38:16.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.069+0000 7fb816a1e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8101029a0 0x7fb81019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:16.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.069+0000 7fb816a1e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8101029a0 0x7fb81019f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38654/0 (socket says 172.21.15.43:38654) 2024-04-21T08:38:16.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.069+0000 7fb816a1e640 1 -- 172.21.15.43:0/3585626011 learned_addr learned my addr 172.21.15.43:0/3585626011 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:16.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.069+0000 7fb815a1c640 1 --2- 172.21.15.43:0/3585626011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb810101fe0 0x7fb810198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:16.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.069+0000 7fb81621d640 1 --2- 172.21.15.43:0/3585626011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb810100e90 0x7fb810198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.069+0000 7fb816a1e640 1 -- 172.21.15.43:0/3585626011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb810101fe0 msgr2=0x7fb810198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.069+0000 7fb816a1e640 1 --2- 172.21.15.43:0/3585626011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb810101fe0 0x7fb810198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.069+0000 7fb816a1e640 1 -- 172.21.15.43:0/3585626011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb810100e90 msgr2=0x7fb810198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.069+0000 7fb816a1e640 1 --2- 172.21.15.43:0/3585626011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb810100e90 0x7fb810198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.069+0000 7fb816a1e640 1 -- 172.21.15.43:0/3585626011 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81019fe30 con 0x7fb8101029a0 2024-04-21T08:38:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.069+0000 7fb816a1e640 1 --2- 172.21.15.43:0/3585626011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8101029a0 0x7fb81019f7c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb80800cad0 tx=0x7fb80800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.069+0000 7fb8077fe640 1 -- 172.21.15.43:0/3585626011 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80800ae20 con 0x7fb8101029a0 2024-04-21T08:38:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.070+0000 7fb8077fe640 1 -- 172.21.15.43:0/3585626011 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb808005020 con 0x7fb8101029a0 2024-04-21T08:38:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.070+0000 7fb818481640 1 -- 172.21.15.43:0/3585626011 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8101a0120 con 0x7fb8101029a0 2024-04-21T08:38:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.070+0000 7fb8077fe640 1 -- 172.21.15.43:0/3585626011 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80800f6b0 con 0x7fb8101029a0 2024-04-21T08:38:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.070+0000 7fb818481640 1 -- 172.21.15.43:0/3585626011 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8101a0620 con 0x7fb8101029a0 2024-04-21T08:38:16.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.071+0000 7fb818481640 1 -- 172.21.15.43:0/3585626011 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8101a08e0 con 0x7fb8101029a0 2024-04-21T08:38:16.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.072+0000 7fb8077fe640 1 -- 172.21.15.43:0/3585626011 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb8080051d0 con 0x7fb8101029a0 2024-04-21T08:38:16.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.072+0000 7fb8077fe640 1 --2- 172.21.15.43:0/3585626011 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb7fc0a3bb0 0x7fb7fc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:16.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.072+0000 7fb81621d640 1 --2- 172.21.15.43:0/3585626011 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb7fc0a3bb0 0x7fb7fc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:16.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.073+0000 7fb8077fe640 1 -- 172.21.15.43:0/3585626011 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fb8080d0c40 con 0x7fb8101029a0 2024-04-21T08:38:16.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.073+0000 7fb81621d640 1 --2- 172.21.15.43:0/3585626011 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb7fc0a3bb0 0x7fb7fc0a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb80c000c00 tx=0x7fb80c005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:16.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.076+0000 7fb8077fe640 1 -- 172.21.15.43:0/3585626011 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb808014030 con 0x7fb8101029a0 2024-04-21T08:38:16.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.262+0000 7fb818481640 1 -- 172.21.15.43:0/3585626011 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb8101a0ab0 con 0x7fb8101029a0 2024-04-21T08:38:16.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.262+0000 7fb8077fe640 1 -- 172.21.15.43:0/3585626011 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fb80809d230 con 0x7fb8101029a0 2024-04-21T08:38:16.263 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:38:16.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.264+0000 7fb818481640 1 -- 172.21.15.43:0/3585626011 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb7fc0a3bb0 msgr2=0x7fb7fc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:16.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.264+0000 7fb818481640 1 --2- 172.21.15.43:0/3585626011 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb7fc0a3bb0 0x7fb7fc0a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb80c000c00 tx=0x7fb80c005be0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.264+0000 7fb818481640 1 -- 172.21.15.43:0/3585626011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8101029a0 msgr2=0x7fb81019f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:16.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.264+0000 7fb818481640 1 --2- 172.21.15.43:0/3585626011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8101029a0 0x7fb81019f7c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb80800cad0 tx=0x7fb80800cf90 comp rx=0 tx=0).stop 2024-04-21T08:38:16.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.264+0000 7fb818481640 1 -- 172.21.15.43:0/3585626011 shutdown_connections 2024-04-21T08:38:16.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.264+0000 7fb818481640 1 --2- 172.21.15.43:0/3585626011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8101029a0 0x7fb81019f7c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.264+0000 7fb818481640 1 --2- 172.21.15.43:0/3585626011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb810101fe0 0x7fb810198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.264+0000 7fb818481640 1 --2- 172.21.15.43:0/3585626011 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb7fc0a3bb0 0x7fb7fc0a6070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.264+0000 7fb818481640 1 --2- 172.21.15.43:0/3585626011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb810100e90 0x7fb810198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.264+0000 7fb818481640 1 -- 172.21.15.43:0/3585626011 >> 172.21.15.43:0/3585626011 conn(0x7fb8100fc400 msgr2=0x7fb810069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:16.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.264+0000 7fb818481640 1 -- 172.21.15.43:0/3585626011 shutdown_connections 2024-04-21T08:38:16.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.264+0000 7fb818481640 1 -- 172.21.15.43:0/3585626011 wait complete. 2024-04-21T08:38:16.276 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:38:16.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.382+0000 7fa4bd831640 1 -- 172.21.15.43:0/3666218320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4b8107ca0 msgr2=0x7fa4b810a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:16.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.382+0000 7fa4bd831640 1 --2- 172.21.15.43:0/3666218320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4b8107ca0 0x7fa4b810a0f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0009ca0 tx=0x7fa4b002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.382+0000 7fa4bd831640 1 -- 172.21.15.43:0/3666218320 shutdown_connections 2024-04-21T08:38:16.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.382+0000 7fa4bd831640 1 --2- 172.21.15.43:0/3666218320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4b8107ca0 0x7fa4b810a0f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.382+0000 7fa4bd831640 1 --2- 172.21.15.43:0/3666218320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4b8105370 0x7fa4b8107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.382+0000 7fa4bd831640 1 --2- 172.21.15.43:0/3666218320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4b8102a40 0x7fa4b8104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.382+0000 7fa4bd831640 1 -- 172.21.15.43:0/3666218320 >> 172.21.15.43:0/3666218320 conn(0x7fa4b80fc470 msgr2=0x7fa4b80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:16.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.382+0000 7fa4bd831640 1 -- 172.21.15.43:0/3666218320 shutdown_connections 2024-04-21T08:38:16.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.382+0000 7fa4bd831640 1 -- 172.21.15.43:0/3666218320 wait complete. 2024-04-21T08:38:16.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.383+0000 7fa4bd831640 1 Processor -- start 2024-04-21T08:38:16.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.383+0000 7fa4bd831640 1 -- start start 2024-04-21T08:38:16.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.383+0000 7fa4bd831640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4b8102a40 0x7fa4b81023b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:16.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.383+0000 7fa4bd831640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4b8105370 0x7fa4b81009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:16.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.383+0000 7fa4bd831640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4b8107ca0 0x7fa4b8100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:16.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.383+0000 7fa4bd831640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b810db00 con 0x7fa4b8102a40 2024-04-21T08:38:16.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.383+0000 7fa4bd831640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b810d980 con 0x7fa4b8105370 2024-04-21T08:38:16.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.383+0000 7fa4bd831640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b810dc80 con 0x7fa4b8107ca0 2024-04-21T08:38:16.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.383+0000 7fa4b6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4b8102a40 0x7fa4b81023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:16.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.383+0000 7fa4b6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4b8102a40 0x7fa4b81023b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38684/0 (socket says 172.21.15.43:38684) 2024-04-21T08:38:16.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.383+0000 7fa4b6ffd640 1 -- 172.21.15.43:0/3552190207 learned_addr learned my addr 172.21.15.43:0/3552190207 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:16.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.384+0000 7fa4b67fc640 1 --2- 172.21.15.43:0/3552190207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4b8105370 0x7fa4b81009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:16.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.384+0000 7fa4b6ffd640 1 -- 172.21.15.43:0/3552190207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4b8107ca0 msgr2=0x7fa4b8100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:16.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.384+0000 7fa4b6ffd640 1 --2- 172.21.15.43:0/3552190207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4b8107ca0 0x7fa4b8100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.384+0000 7fa4b6ffd640 1 -- 172.21.15.43:0/3552190207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4b8105370 msgr2=0x7fa4b81009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:16.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.384+0000 7fa4b6ffd640 1 --2- 172.21.15.43:0/3552190207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4b8105370 0x7fa4b81009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.384+0000 7fa4b6ffd640 1 -- 172.21.15.43:0/3552190207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b81017c0 con 0x7fa4b8102a40 2024-04-21T08:38:16.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.384+0000 7fa4b67fc640 1 --2- 172.21.15.43:0/3552190207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4b8105370 0x7fa4b81009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:16.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.384+0000 7fa4b6ffd640 1 --2- 172.21.15.43:0/3552190207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4b8102a40 0x7fa4b81023b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000c920 tx=0x7fa4a000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:16.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.384+0000 7fa4bc82f640 1 -- 172.21.15.43:0/3552190207 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0007c70 con 0x7fa4b8102a40 2024-04-21T08:38:16.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.384+0000 7fa4bc82f640 1 -- 172.21.15.43:0/3552190207 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4a0007e10 con 0x7fa4b8102a40 2024-04-21T08:38:16.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.384+0000 7fa4bd831640 1 -- 172.21.15.43:0/3552190207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b81a2230 con 0x7fa4b8102a40 2024-04-21T08:38:16.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.384+0000 7fa4bc82f640 1 -- 172.21.15.43:0/3552190207 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0018820 con 0x7fa4b8102a40 2024-04-21T08:38:16.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.384+0000 7fa4bd831640 1 -- 172.21.15.43:0/3552190207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b81a2780 con 0x7fa4b8102a40 2024-04-21T08:38:16.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.385+0000 7fa4bd831640 1 -- 172.21.15.43:0/3552190207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b80fdf60 con 0x7fa4b8102a40 2024-04-21T08:38:16.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.386+0000 7fa4bc82f640 1 -- 172.21.15.43:0/3552190207 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa4a000f450 con 0x7fa4b8102a40 2024-04-21T08:38:16.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.387+0000 7fa4bc82f640 1 --2- 172.21.15.43:0/3552190207 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa4a40a3bb0 0x7fa4a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:16.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.387+0000 7fa4b67fc640 1 --2- 172.21.15.43:0/3552190207 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa4a40a3bb0 0x7fa4a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:16.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.387+0000 7fa4bc82f640 1 -- 172.21.15.43:0/3552190207 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fa4a00d0cd0 con 0x7fa4b8102a40 2024-04-21T08:38:16.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.387+0000 7fa4b67fc640 1 --2- 172.21.15.43:0/3552190207 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa4a40a3bb0 0x7fa4a40a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8102120 tx=0x7fa4a8005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.390+0000 7fa4bc82f640 1 -- 172.21.15.43:0/3552190207 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa4a0014030 con 0x7fa4b8102a40 2024-04-21T08:38:16.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:16 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1687031538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:16.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:16 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/288929255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:16.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:16 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3585626011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:16.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.619+0000 7fa4bd831640 1 -- 172.21.15.43:0/3552190207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4b802d5c0 con 0x7fa4b8102a40 2024-04-21T08:38:16.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.620+0000 7fa4bc82f640 1 -- 172.21.15.43:0/3552190207 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11938 (secure 0 0 0) 0x7fa4a009d350 con 0x7fa4b8102a40 2024-04-21T08:38:16.620 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:16.620 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:36:51.902455+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24491,24527,24487,24425,24412]},"id":1}]} 2024-04-21T08:38:16.620 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 27 2024-04-21T08:38:16.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.621+0000 7fa4bd831640 1 -- 172.21.15.43:0/3552190207 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa4a40a3bb0 msgr2=0x7fa4a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:16.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.621+0000 7fa4bd831640 1 --2- 172.21.15.43:0/3552190207 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa4a40a3bb0 0x7fa4a40a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8102120 tx=0x7fa4a8005e50 comp rx=0 tx=0).stop 2024-04-21T08:38:16.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.621+0000 7fa4bd831640 1 -- 172.21.15.43:0/3552190207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4b8102a40 msgr2=0x7fa4b81023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:16.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.621+0000 7fa4bd831640 1 --2- 172.21.15.43:0/3552190207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4b8102a40 0x7fa4b81023b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000c920 tx=0x7fa4a000cde0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.622+0000 7fa4bd831640 1 -- 172.21.15.43:0/3552190207 shutdown_connections 2024-04-21T08:38:16.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.622+0000 7fa4bd831640 1 --2- 172.21.15.43:0/3552190207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4b8107ca0 0x7fa4b8100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.622+0000 7fa4bd831640 1 --2- 172.21.15.43:0/3552190207 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa4a40a3bb0 0x7fa4a40a6070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.622+0000 7fa4bd831640 1 --2- 172.21.15.43:0/3552190207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4b8105370 0x7fa4b81009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.622+0000 7fa4bd831640 1 --2- 172.21.15.43:0/3552190207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4b8102a40 0x7fa4b81023b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.622+0000 7fa4bd831640 1 -- 172.21.15.43:0/3552190207 >> 172.21.15.43:0/3552190207 conn(0x7fa4b80fc470 msgr2=0x7fa4b8103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:16.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.622+0000 7fa4bd831640 1 -- 172.21.15.43:0/3552190207 shutdown_connections 2024-04-21T08:38:16.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.622+0000 7fa4bd831640 1 -- 172.21.15.43:0/3552190207 wait complete. 2024-04-21T08:38:16.634 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:38:16.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:16 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1687031538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:16.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:16 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/288929255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:16.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:16 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3585626011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:16.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.736+0000 7f978e674640 1 -- 172.21.15.43:0/2509650375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9788100e70 msgr2=0x7f9788101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:16.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.736+0000 7f978e674640 1 --2- 172.21.15.43:0/2509650375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9788100e70 0x7f9788101290 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9778009900 tx=0x7f977802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:16.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.736+0000 7f978e674640 1 -- 172.21.15.43:0/2509650375 shutdown_connections 2024-04-21T08:38:16.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.736+0000 7f978e674640 1 --2- 172.21.15.43:0/2509650375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9788102980 0x7f97881092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.736+0000 7f978e674640 1 --2- 172.21.15.43:0/2509650375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9788101fc0 0x7f9788102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.736+0000 7f978e674640 1 --2- 172.21.15.43:0/2509650375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9788100e70 0x7f9788101290 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.736+0000 7f978e674640 1 -- 172.21.15.43:0/2509650375 >> 172.21.15.43:0/2509650375 conn(0x7f97880fc3e0 msgr2=0x7f97880fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:16.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.736+0000 7f978e674640 1 -- 172.21.15.43:0/2509650375 shutdown_connections 2024-04-21T08:38:16.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.736+0000 7f978e674640 1 -- 172.21.15.43:0/2509650375 wait complete. 2024-04-21T08:38:16.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.736+0000 7f978e674640 1 Processor -- start 2024-04-21T08:38:16.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.736+0000 7f978e674640 1 -- start start 2024-04-21T08:38:16.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.737+0000 7f978e674640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9788100e70 0x7f9788198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:16.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.737+0000 7f978e674640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9788101fc0 0x7f97881986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:16.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.737+0000 7f978e674640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9788102980 0x7f978819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:16.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.737+0000 7f978e674640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f978810c9c0 con 0x7f9788102980 2024-04-21T08:38:16.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.737+0000 7f978e674640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f978810c840 con 0x7f9788100e70 2024-04-21T08:38:16.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.737+0000 7f978e674640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f978810cb40 con 0x7f9788101fc0 2024-04-21T08:38:16.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.737+0000 7f978cc11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9788102980 0x7f978819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:16.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.737+0000 7f978cc11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9788102980 0x7f978819f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54836/0 (socket says 172.21.15.43:54836) 2024-04-21T08:38:16.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.737+0000 7f978cc11640 1 -- 172.21.15.43:0/4004372234 learned_addr learned my addr 172.21.15.43:0/4004372234 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:16.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.737+0000 7f97877fe640 1 --2- 172.21.15.43:0/4004372234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9788101fc0 0x7f97881986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:16.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.737+0000 7f9787fff640 1 --2- 172.21.15.43:0/4004372234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9788100e70 0x7f9788198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:16.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.737+0000 7f978cc11640 1 -- 172.21.15.43:0/4004372234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9788101fc0 msgr2=0x7f97881986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:16.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.737+0000 7f978cc11640 1 --2- 172.21.15.43:0/4004372234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9788101fc0 0x7f97881986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.737+0000 7f978cc11640 1 -- 172.21.15.43:0/4004372234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9788100e70 msgr2=0x7f9788198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.737+0000 7f978cc11640 1 --2- 172.21.15.43:0/4004372234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9788100e70 0x7f9788198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.737+0000 7f978cc11640 1 -- 172.21.15.43:0/4004372234 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978819fdb0 con 0x7f9788102980 2024-04-21T08:38:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.737+0000 7f978cc11640 1 --2- 172.21.15.43:0/4004372234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9788102980 0x7f978819f740 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f978000eaa0 tx=0x7f978000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.737+0000 7f97857fa640 1 -- 172.21.15.43:0/4004372234 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978000cdb0 con 0x7f9788102980 2024-04-21T08:38:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.737+0000 7f97857fa640 1 -- 172.21.15.43:0/4004372234 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9780005020 con 0x7f9788102980 2024-04-21T08:38:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.737+0000 7f978e674640 1 -- 172.21.15.43:0/4004372234 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97881a00a0 con 0x7f9788102980 2024-04-21T08:38:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.737+0000 7f97857fa640 1 -- 172.21.15.43:0/4004372234 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97800116d0 con 0x7f9788102980 2024-04-21T08:38:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.738+0000 7f978e674640 1 -- 172.21.15.43:0/4004372234 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97881a05a0 con 0x7f9788102980 2024-04-21T08:38:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.738+0000 7f978e674640 1 -- 172.21.15.43:0/4004372234 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97881a0860 con 0x7f9788102980 2024-04-21T08:38:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.740+0000 7f97857fa640 1 -- 172.21.15.43:0/4004372234 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f97800051d0 con 0x7f9788102980 2024-04-21T08:38:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.740+0000 7f97857fa640 1 --2- 172.21.15.43:0/4004372234 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f97700a3bb0 0x7f97700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.740+0000 7f9787fff640 1 --2- 172.21.15.43:0/4004372234 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f97700a3bb0 0x7f97700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.740+0000 7f97857fa640 1 -- 172.21.15.43:0/4004372234 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f97800d0de0 con 0x7f9788102980 2024-04-21T08:38:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.741+0000 7f9787fff640 1 --2- 172.21.15.43:0/4004372234 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f97700a3bb0 0x7f97700a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9778000c00 tx=0x7f9778005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:16.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.744+0000 7f97857fa640 1 -- 172.21.15.43:0/4004372234 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9780016030 con 0x7f9788102980 2024-04-21T08:38:16.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:16 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1687031538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:16.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:16 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/288929255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:16.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:16 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3585626011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:16.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.933+0000 7f978e674640 1 -- 172.21.15.43:0/4004372234 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97881a0a30 con 0x7f9788102980 2024-04-21T08:38:16.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.934+0000 7f97857fa640 1 -- 172.21.15.43:0/4004372234 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24856 (secure 0 0 0) 0x7f978009d3d0 con 0x7f9788102980 2024-04-21T08:38:16.935 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:16.935 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":94,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:38:15.326063+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:38:16.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.936+0000 7f978e674640 1 -- 172.21.15.43:0/4004372234 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f97700a3bb0 msgr2=0x7f97700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:16.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.936+0000 7f978e674640 1 --2- 172.21.15.43:0/4004372234 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f97700a3bb0 0x7f97700a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9778000c00 tx=0x7f9778005be0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.936+0000 7f978e674640 1 -- 172.21.15.43:0/4004372234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9788102980 msgr2=0x7f978819f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:16.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.936+0000 7f978e674640 1 --2- 172.21.15.43:0/4004372234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9788102980 0x7f978819f740 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f978000eaa0 tx=0x7f978000ef60 comp rx=0 tx=0).stop 2024-04-21T08:38:16.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.936+0000 7f978e674640 1 -- 172.21.15.43:0/4004372234 shutdown_connections 2024-04-21T08:38:16.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.936+0000 7f978e674640 1 --2- 172.21.15.43:0/4004372234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9788102980 0x7f978819f740 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.936+0000 7f978e674640 1 --2- 172.21.15.43:0/4004372234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9788101fc0 0x7f97881986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.936+0000 7f978e674640 1 --2- 172.21.15.43:0/4004372234 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f97700a3bb0 0x7f97700a6070 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.936+0000 7f978e674640 1 --2- 172.21.15.43:0/4004372234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9788100e70 0x7f9788198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:16.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.936+0000 7f978e674640 1 -- 172.21.15.43:0/4004372234 >> 172.21.15.43:0/4004372234 conn(0x7f97880fc3e0 msgr2=0x7f97880fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:16.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.937+0000 7f978e674640 1 -- 172.21.15.43:0/4004372234 shutdown_connections 2024-04-21T08:38:16.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:16.937+0000 7f978e674640 1 -- 172.21.15.43:0/4004372234 wait complete. 2024-04-21T08:38:16.948 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:39.981405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:22.034751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:22.780795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:25.984261+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:38:16.949 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:38:17.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.054+0000 7f86c8255640 1 -- 172.21.15.43:0/3610917628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86c0102a90 msgr2=0x7f86c0104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:17.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.054+0000 7f86c8255640 1 --2- 172.21.15.43:0/3610917628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86c0102a90 0x7f86c0104e80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f86bc009900 tx=0x7f86bc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:17.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.054+0000 7f86c8255640 1 -- 172.21.15.43:0/3610917628 shutdown_connections 2024-04-21T08:38:17.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.054+0000 7f86c8255640 1 --2- 172.21.15.43:0/3610917628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86c0107cf0 0x7f86c010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.054+0000 7f86c8255640 1 --2- 172.21.15.43:0/3610917628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86c01053c0 0x7f86c01077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.054+0000 7f86c8255640 1 --2- 172.21.15.43:0/3610917628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86c0102a90 0x7f86c0104e80 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.054+0000 7f86c8255640 1 -- 172.21.15.43:0/3610917628 >> 172.21.15.43:0/3610917628 conn(0x7f86c00fc400 msgr2=0x7f86c00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:17.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.054+0000 7f86c8255640 1 -- 172.21.15.43:0/3610917628 shutdown_connections 2024-04-21T08:38:17.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.054+0000 7f86c8255640 1 -- 172.21.15.43:0/3610917628 wait complete. 2024-04-21T08:38:17.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.055+0000 7f86c8255640 1 Processor -- start 2024-04-21T08:38:17.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.055+0000 7f86c8255640 1 -- start start 2024-04-21T08:38:17.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.055+0000 7f86c8255640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86c0102a90 0x7f86c01023b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:17.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.055+0000 7f86c8255640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86c01053c0 0x7f86c0100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:17.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.055+0000 7f86c8255640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86c0107cf0 0x7f86c0100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:17.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.055+0000 7f86c8255640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f86c010db30 con 0x7f86c0107cf0 2024-04-21T08:38:17.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.055+0000 7f86c8255640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f86c010d9b0 con 0x7f86c01053c0 2024-04-21T08:38:17.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.055+0000 7f86c8255640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f86c010dcb0 con 0x7f86c0102a90 2024-04-21T08:38:17.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.055+0000 7f86c67f2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86c0107cf0 0x7f86c0100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:17.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.055+0000 7f86c67f2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86c0107cf0 0x7f86c0100f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54852/0 (socket says 172.21.15.43:54852) 2024-04-21T08:38:17.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.055+0000 7f86c67f2640 1 -- 172.21.15.43:0/2579029896 learned_addr learned my addr 172.21.15.43:0/2579029896 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:17.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.055+0000 7f86c57f0640 1 --2- 172.21.15.43:0/2579029896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86c01053c0 0x7f86c0100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:17.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.055+0000 7f86c5ff1640 1 --2- 172.21.15.43:0/2579029896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86c0102a90 0x7f86c01023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:17.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.056+0000 7f86c67f2640 1 -- 172.21.15.43:0/2579029896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86c0102a90 msgr2=0x7f86c01023b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:17.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.056+0000 7f86c67f2640 1 --2- 172.21.15.43:0/2579029896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86c0102a90 0x7f86c01023b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.056+0000 7f86c67f2640 1 -- 172.21.15.43:0/2579029896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86c01053c0 msgr2=0x7f86c0100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:17.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.056+0000 7f86c67f2640 1 --2- 172.21.15.43:0/2579029896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86c01053c0 0x7f86c0100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.056+0000 7f86c67f2640 1 -- 172.21.15.43:0/2579029896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c0101840 con 0x7f86c0107cf0 2024-04-21T08:38:17.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.056+0000 7f86c57f0640 1 --2- 172.21.15.43:0/2579029896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86c01053c0 0x7f86c0100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:17.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.056+0000 7f86c67f2640 1 --2- 172.21.15.43:0/2579029896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86c0107cf0 0x7f86c0100f80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f86b800eaa0 tx=0x7f86b800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:17.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.056+0000 7f86b6ffd640 1 -- 172.21.15.43:0/2579029896 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b800cdb0 con 0x7f86c0107cf0 2024-04-21T08:38:17.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.056+0000 7f86b6ffd640 1 -- 172.21.15.43:0/2579029896 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86b8005020 con 0x7f86c0107cf0 2024-04-21T08:38:17.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.056+0000 7f86c8255640 1 -- 172.21.15.43:0/2579029896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c01a23f0 con 0x7f86c0107cf0 2024-04-21T08:38:17.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.056+0000 7f86b6ffd640 1 -- 172.21.15.43:0/2579029896 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b80116d0 con 0x7f86c0107cf0 2024-04-21T08:38:17.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.056+0000 7f86c8255640 1 -- 172.21.15.43:0/2579029896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c01a2940 con 0x7f86c0107cf0 2024-04-21T08:38:17.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.057+0000 7f86c8255640 1 -- 172.21.15.43:0/2579029896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c00fdef0 con 0x7f86c0107cf0 2024-04-21T08:38:17.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.058+0000 7f86b6ffd640 1 -- 172.21.15.43:0/2579029896 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f86b80051d0 con 0x7f86c0107cf0 2024-04-21T08:38:17.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.059+0000 7f86b6ffd640 1 --2- 172.21.15.43:0/2579029896 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f86ac0a3bb0 0x7f86ac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:17.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.059+0000 7f86c5ff1640 1 --2- 172.21.15.43:0/2579029896 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f86ac0a3bb0 0x7f86ac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:17.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.059+0000 7f86b6ffd640 1 -- 172.21.15.43:0/2579029896 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f86b80d0aa0 con 0x7f86c0107cf0 2024-04-21T08:38:17.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.059+0000 7f86c5ff1640 1 --2- 172.21.15.43:0/2579029896 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f86ac0a3bb0 0x7f86ac0a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0096c0 tx=0x7f86bc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:17.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.063+0000 7f86b6ffd640 1 -- 172.21.15.43:0/2579029896 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f86b8016030 con 0x7f86c0107cf0 2024-04-21T08:38:17.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.250+0000 7f86c8255640 1 -- 172.21.15.43:0/2579029896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f86c0062b50 con 0x7f86c0107cf0 2024-04-21T08:38:17.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.250+0000 7f86b6ffd640 1 -- 172.21.15.43:0/2579029896 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f86b809d090 con 0x7f86c0107cf0 2024-04-21T08:38:17.251 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-21T08:38:17.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.252+0000 7f86c8255640 1 -- 172.21.15.43:0/2579029896 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f86ac0a3bb0 msgr2=0x7f86ac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:17.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.252+0000 7f86c8255640 1 --2- 172.21.15.43:0/2579029896 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f86ac0a3bb0 0x7f86ac0a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0096c0 tx=0x7f86bc000bc0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.252+0000 7f86c8255640 1 -- 172.21.15.43:0/2579029896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86c0107cf0 msgr2=0x7f86c0100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:17.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.252+0000 7f86c8255640 1 --2- 172.21.15.43:0/2579029896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86c0107cf0 0x7f86c0100f80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f86b800eaa0 tx=0x7f86b800ef60 comp rx=0 tx=0).stop 2024-04-21T08:38:17.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.252+0000 7f86c8255640 1 -- 172.21.15.43:0/2579029896 shutdown_connections 2024-04-21T08:38:17.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.252+0000 7f86c8255640 1 --2- 172.21.15.43:0/2579029896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86c0107cf0 0x7f86c0100f80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.252+0000 7f86c8255640 1 --2- 172.21.15.43:0/2579029896 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f86ac0a3bb0 0x7f86ac0a6070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.252+0000 7f86c8255640 1 --2- 172.21.15.43:0/2579029896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86c01053c0 0x7f86c0100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.252+0000 7f86c8255640 1 --2- 172.21.15.43:0/2579029896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86c0102a90 0x7f86c01023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.252+0000 7f86c8255640 1 -- 172.21.15.43:0/2579029896 >> 172.21.15.43:0/2579029896 conn(0x7f86c00fc400 msgr2=0x7f86c0103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:17.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.252+0000 7f86c8255640 1 -- 172.21.15.43:0/2579029896 shutdown_connections 2024-04-21T08:38:17.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.253+0000 7f86c8255640 1 -- 172.21.15.43:0/2579029896 wait complete. 2024-04-21T08:38:17.264 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:38:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.370+0000 7f7172e81640 1 -- 172.21.15.43:0/1513082981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f716c102a30 msgr2=0x7f716c104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.370+0000 7f7172e81640 1 --2- 172.21.15.43:0/1513082981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f716c102a30 0x7f716c104e20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f715c009900 tx=0x7f715c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.370+0000 7f7172e81640 1 -- 172.21.15.43:0/1513082981 shutdown_connections 2024-04-21T08:38:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.370+0000 7f7172e81640 1 --2- 172.21.15.43:0/1513082981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f716c107c90 0x7f716c10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.370+0000 7f7172e81640 1 --2- 172.21.15.43:0/1513082981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716c105360 0x7f716c107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.370+0000 7f7172e81640 1 --2- 172.21.15.43:0/1513082981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f716c102a30 0x7f716c104e20 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.370+0000 7f7172e81640 1 -- 172.21.15.43:0/1513082981 >> 172.21.15.43:0/1513082981 conn(0x7f716c0fc440 msgr2=0x7f716c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.370+0000 7f7172e81640 1 -- 172.21.15.43:0/1513082981 shutdown_connections 2024-04-21T08:38:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.370+0000 7f7172e81640 1 -- 172.21.15.43:0/1513082981 wait complete. 2024-04-21T08:38:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.371+0000 7f7172e81640 1 Processor -- start 2024-04-21T08:38:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.371+0000 7f7172e81640 1 -- start start 2024-04-21T08:38:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.371+0000 7f7172e81640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716c102a30 0x7f716c1023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.371+0000 7f7172e81640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f716c105360 0x7f716c1009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.371+0000 7f7172e81640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f716c107c90 0x7f716c100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.371+0000 7f7172e81640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f716c10db40 con 0x7f716c105360 2024-04-21T08:38:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.371+0000 7f7172e81640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f716c10d9c0 con 0x7f716c107c90 2024-04-21T08:38:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.371+0000 7f7172e81640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f716c10dcc0 con 0x7f716c102a30 2024-04-21T08:38:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.371+0000 7f716bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f716c105360 0x7f716c1009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.371+0000 7f716bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f716c105360 0x7f716c1009e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54862/0 (socket says 172.21.15.43:54862) 2024-04-21T08:38:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.371+0000 7f716bfff640 1 -- 172.21.15.43:0/1788197620 learned_addr learned my addr 172.21.15.43:0/1788197620 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.371+0000 7f7170c1d640 1 --2- 172.21.15.43:0/1788197620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716c102a30 0x7f716c1023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.372+0000 7f717141e640 1 --2- 172.21.15.43:0/1788197620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f716c107c90 0x7f716c100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.372+0000 7f716bfff640 1 -- 172.21.15.43:0/1788197620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716c102a30 msgr2=0x7f716c1023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.372+0000 7f716bfff640 1 --2- 172.21.15.43:0/1788197620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716c102a30 0x7f716c1023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.372+0000 7f716bfff640 1 -- 172.21.15.43:0/1788197620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f716c107c90 msgr2=0x7f716c100f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.372+0000 7f716bfff640 1 --2- 172.21.15.43:0/1788197620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f716c107c90 0x7f716c100f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.372+0000 7f716bfff640 1 -- 172.21.15.43:0/1788197620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716c1017b0 con 0x7f716c105360 2024-04-21T08:38:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.372+0000 7f716bfff640 1 --2- 172.21.15.43:0/1788197620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f716c105360 0x7f716c1009e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f716000d770 tx=0x7f716000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.372+0000 7f7169ffb640 1 -- 172.21.15.43:0/1788197620 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160017070 con 0x7f716c105360 2024-04-21T08:38:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.372+0000 7f7169ffb640 1 -- 172.21.15.43:0/1788197620 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7160009c30 con 0x7f716c105360 2024-04-21T08:38:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.372+0000 7f7172e81640 1 -- 172.21.15.43:0/1788197620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f716c1a2340 con 0x7f716c105360 2024-04-21T08:38:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.372+0000 7f7169ffb640 1 -- 172.21.15.43:0/1788197620 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716000f620 con 0x7f716c105360 2024-04-21T08:38:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.372+0000 7f7172e81640 1 -- 172.21.15.43:0/1788197620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f716c1a28e0 con 0x7f716c105360 2024-04-21T08:38:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.373+0000 7f7172e81640 1 -- 172.21.15.43:0/1788197620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f716c0fdf30 con 0x7f716c105360 2024-04-21T08:38:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.374+0000 7f7169ffb640 1 -- 172.21.15.43:0/1788197620 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f716001a070 con 0x7f716c105360 2024-04-21T08:38:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.375+0000 7f7169ffb640 1 --2- 172.21.15.43:0/1788197620 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f71540a3c30 0x7f71540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.375+0000 7f7169ffb640 1 -- 172.21.15.43:0/1788197620 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f71600d02d0 con 0x7f716c105360 2024-04-21T08:38:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.375+0000 7f7170c1d640 1 --2- 172.21.15.43:0/1788197620 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f71540a3c30 0x7f71540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.375+0000 7f7170c1d640 1 --2- 172.21.15.43:0/1788197620 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f71540a3c30 0x7f71540a60f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f715c000c00 tx=0x7f715c02c7e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:17.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.379+0000 7f7169ffb640 1 -- 172.21.15.43:0/1788197620 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7160009de0 con 0x7f716c105360 2024-04-21T08:38:17.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:17 smithi043 ceph-mon[27978]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-21T08:38:17.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:17 smithi043 ceph-mon[27978]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-21T08:38:17.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:17 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3552190207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:17.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:17 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/4004372234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:17.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:17 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2579029896' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:17.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.566+0000 7f7172e81640 1 -- 172.21.15.43:0/1788197620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f716c062b50 con 0x7f716c105360 2024-04-21T08:38:17.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.567+0000 7f7169ffb640 1 -- 172.21.15.43:0/1788197620 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f716009c8c0 con 0x7f716c105360 2024-04-21T08:38:17.567 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:38:17.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.569+0000 7f7172e81640 1 -- 172.21.15.43:0/1788197620 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f71540a3c30 msgr2=0x7f71540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:17.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.569+0000 7f7172e81640 1 --2- 172.21.15.43:0/1788197620 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f71540a3c30 0x7f71540a60f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f715c000c00 tx=0x7f715c02c7e0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.569+0000 7f7172e81640 1 -- 172.21.15.43:0/1788197620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f716c105360 msgr2=0x7f716c1009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:17.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.569+0000 7f7172e81640 1 --2- 172.21.15.43:0/1788197620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f716c105360 0x7f716c1009e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f716000d770 tx=0x7f716000dc30 comp rx=0 tx=0).stop 2024-04-21T08:38:17.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.569+0000 7f7172e81640 1 -- 172.21.15.43:0/1788197620 shutdown_connections 2024-04-21T08:38:17.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.569+0000 7f7172e81640 1 --2- 172.21.15.43:0/1788197620 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f71540a3c30 0x7f71540a60f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.569+0000 7f7172e81640 1 --2- 172.21.15.43:0/1788197620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f716c107c90 0x7f716c100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.569+0000 7f7172e81640 1 --2- 172.21.15.43:0/1788197620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f716c105360 0x7f716c1009e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.569+0000 7f7172e81640 1 --2- 172.21.15.43:0/1788197620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716c102a30 0x7f716c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.569+0000 7f7172e81640 1 -- 172.21.15.43:0/1788197620 >> 172.21.15.43:0/1788197620 conn(0x7f716c0fc440 msgr2=0x7f716c1034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:17.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.569+0000 7f7172e81640 1 -- 172.21.15.43:0/1788197620 shutdown_connections 2024-04-21T08:38:17.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.569+0000 7f7172e81640 1 -- 172.21.15.43:0/1788197620 wait complete. 2024-04-21T08:38:17.581 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:38:17.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.686+0000 7f1c21540640 1 -- 172.21.15.43:0/294356775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c1c1031a0 msgr2=0x7f1c1c103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:17.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.686+0000 7f1c21540640 1 --2- 172.21.15.43:0/294356775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c1c1031a0 0x7f1c1c103620 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009900 tx=0x7f1c1002bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:17.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.686+0000 7f1c21540640 1 -- 172.21.15.43:0/294356775 shutdown_connections 2024-04-21T08:38:17.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.686+0000 7f1c21540640 1 --2- 172.21.15.43:0/294356775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c1c103b60 0x7f1c1c10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.686+0000 7f1c21540640 1 --2- 172.21.15.43:0/294356775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c1c1031a0 0x7f1c1c103620 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.686+0000 7f1c21540640 1 --2- 172.21.15.43:0/294356775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c1c101fb0 0x7f1c1c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.686+0000 7f1c21540640 1 -- 172.21.15.43:0/294356775 >> 172.21.15.43:0/294356775 conn(0x7f1c1c0fd520 msgr2=0x7f1c1c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:17.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.686+0000 7f1c21540640 1 -- 172.21.15.43:0/294356775 shutdown_connections 2024-04-21T08:38:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.686+0000 7f1c21540640 1 -- 172.21.15.43:0/294356775 wait complete. 2024-04-21T08:38:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.687+0000 7f1c21540640 1 Processor -- start 2024-04-21T08:38:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.687+0000 7f1c21540640 1 -- start start 2024-04-21T08:38:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.687+0000 7f1c21540640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c1c101fb0 0x7f1c1c075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.687+0000 7f1c21540640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c1c1031a0 0x7f1c1c074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.687+0000 7f1c21540640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c1c103b60 0x7f1c1c074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.687+0000 7f1c21540640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c10daf0 con 0x7f1c1c103b60 2024-04-21T08:38:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.687+0000 7f1c21540640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c10d970 con 0x7f1c1c1031a0 2024-04-21T08:38:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.687+0000 7f1c21540640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c10dc70 con 0x7f1c1c101fb0 2024-04-21T08:38:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.687+0000 7f1c1b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c1c103b60 0x7f1c1c074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.687+0000 7f1c1b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c1c103b60 0x7f1c1c074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54880/0 (socket says 172.21.15.43:54880) 2024-04-21T08:38:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.687+0000 7f1c1b7fe640 1 -- 172.21.15.43:0/3789816880 learned_addr learned my addr 172.21.15.43:0/3789816880 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.687+0000 7f1c1a7fc640 1 --2- 172.21.15.43:0/3789816880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c1c1031a0 0x7f1c1c074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.687+0000 7f1c1affd640 1 --2- 172.21.15.43:0/3789816880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c1c101fb0 0x7f1c1c075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:17.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.688+0000 7f1c1b7fe640 1 -- 172.21.15.43:0/3789816880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c1c101fb0 msgr2=0x7f1c1c075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:17.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.688+0000 7f1c1b7fe640 1 --2- 172.21.15.43:0/3789816880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c1c101fb0 0x7f1c1c075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.688+0000 7f1c1b7fe640 1 -- 172.21.15.43:0/3789816880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c1c1031a0 msgr2=0x7f1c1c074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:17.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.688+0000 7f1c1b7fe640 1 --2- 172.21.15.43:0/3789816880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c1c1031a0 0x7f1c1c074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.688+0000 7f1c1b7fe640 1 -- 172.21.15.43:0/3789816880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1c075110 con 0x7f1c1c103b60 2024-04-21T08:38:17.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.688+0000 7f1c1a7fc640 1 --2- 172.21.15.43:0/3789816880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c1c1031a0 0x7f1c1c074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:17.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.688+0000 7f1c1b7fe640 1 --2- 172.21.15.43:0/3789816880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c1c103b60 0x7f1c1c074880 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400eb10 tx=0x7f1c1400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:17.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.688+0000 7f1c03fff640 1 -- 172.21.15.43:0/3789816880 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14004b20 con 0x7f1c1c103b60 2024-04-21T08:38:17.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.688+0000 7f1c03fff640 1 -- 172.21.15.43:0/3789816880 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c14004cc0 con 0x7f1c1c103b60 2024-04-21T08:38:17.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.688+0000 7f1c21540640 1 -- 172.21.15.43:0/3789816880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1c1a22d0 con 0x7f1c1c103b60 2024-04-21T08:38:17.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.688+0000 7f1c03fff640 1 -- 172.21.15.43:0/3789816880 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14010670 con 0x7f1c1c103b60 2024-04-21T08:38:17.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.688+0000 7f1c21540640 1 -- 172.21.15.43:0/3789816880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c1c1a2820 con 0x7f1c1c103b60 2024-04-21T08:38:17.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.689+0000 7f1c21540640 1 -- 172.21.15.43:0/3789816880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c1c109350 con 0x7f1c1c103b60 2024-04-21T08:38:17.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.690+0000 7f1c03fff640 1 -- 172.21.15.43:0/3789816880 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1c14005020 con 0x7f1c1c103b60 2024-04-21T08:38:17.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.691+0000 7f1c03fff640 1 --2- 172.21.15.43:0/3789816880 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f1c040a3c30 0x7f1c040a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:17.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.691+0000 7f1c1affd640 1 --2- 172.21.15.43:0/3789816880 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f1c040a3c30 0x7f1c040a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:17.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.691+0000 7f1c03fff640 1 -- 172.21.15.43:0/3789816880 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f1c140d0df0 con 0x7f1c1c103b60 2024-04-21T08:38:17.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.691+0000 7f1c1affd640 1 --2- 172.21.15.43:0/3789816880 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f1c040a3c30 0x7f1c040a60f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c005fd0 tx=0x7f1c0c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:17.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.695+0000 7f1c03fff640 1 -- 172.21.15.43:0/3789816880 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c1409d380 con 0x7f1c1c103b60 2024-04-21T08:38:17.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:17 smithi175 ceph-mon[31920]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-21T08:38:17.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:17 smithi175 ceph-mon[31920]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-21T08:38:17.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:17 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3552190207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:17.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:17 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/4004372234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:17.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:17 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2579029896' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:17.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:17 smithi112 ceph-mon[31085]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-21T08:38:17.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:17 smithi112 ceph-mon[31085]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-21T08:38:17.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:17 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3552190207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:17.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:17 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/4004372234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:17.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:17 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2579029896' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.879+0000 7f1c21540640 1 -- 172.21.15.43:0/3789816880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1c1c062b50 con 0x7f1c1c103b60 2024-04-21T08:38:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.880+0000 7f1c03fff640 1 -- 172.21.15.43:0/3789816880 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f1c1409cc80 con 0x7f1c1c103b60 2024-04-21T08:38:17.880 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-21T08:38:17.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.882+0000 7f1c21540640 1 -- 172.21.15.43:0/3789816880 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f1c040a3c30 msgr2=0x7f1c040a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:17.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.882+0000 7f1c21540640 1 --2- 172.21.15.43:0/3789816880 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f1c040a3c30 0x7f1c040a60f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c005fd0 tx=0x7f1c0c00b410 comp rx=0 tx=0).stop 2024-04-21T08:38:17.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.882+0000 7f1c21540640 1 -- 172.21.15.43:0/3789816880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c1c103b60 msgr2=0x7f1c1c074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:17.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.882+0000 7f1c21540640 1 --2- 172.21.15.43:0/3789816880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c1c103b60 0x7f1c1c074880 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400eb10 tx=0x7f1c1400c2d0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.882+0000 7f1c21540640 1 -- 172.21.15.43:0/3789816880 shutdown_connections 2024-04-21T08:38:17.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.882+0000 7f1c21540640 1 --2- 172.21.15.43:0/3789816880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c1c103b60 0x7f1c1c074880 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.882+0000 7f1c21540640 1 --2- 172.21.15.43:0/3789816880 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f1c040a3c30 0x7f1c040a60f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.882+0000 7f1c21540640 1 --2- 172.21.15.43:0/3789816880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c1c1031a0 0x7f1c1c074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.882+0000 7f1c21540640 1 --2- 172.21.15.43:0/3789816880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c1c101fb0 0x7f1c1c075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.882+0000 7f1c21540640 1 -- 172.21.15.43:0/3789816880 >> 172.21.15.43:0/3789816880 conn(0x7f1c1c0fd520 msgr2=0x7f1c1c106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:17.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.882+0000 7f1c21540640 1 -- 172.21.15.43:0/3789816880 shutdown_connections 2024-04-21T08:38:17.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.882+0000 7f1c21540640 1 -- 172.21.15.43:0/3789816880 wait complete. 2024-04-21T08:38:17.893 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:38:18.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.999+0000 7fa4872e4640 1 -- 172.21.15.43:0/248048004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa480103b60 msgr2=0x7fa48010a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:18.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.999+0000 7fa4872e4640 1 --2- 172.21.15.43:0/248048004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa480103b60 0x7fa48010a3f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa478009ca0 tx=0x7fa47802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.999+0000 7fa4872e4640 1 -- 172.21.15.43:0/248048004 shutdown_connections 2024-04-21T08:38:18.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.999+0000 7fa4872e4640 1 --2- 172.21.15.43:0/248048004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa480103b60 0x7fa48010a3f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.999+0000 7fa4872e4640 1 --2- 172.21.15.43:0/248048004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4801031a0 0x7fa480103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.999+0000 7fa4872e4640 1 --2- 172.21.15.43:0/248048004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa480101fb0 0x7fa4801023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.999+0000 7fa4872e4640 1 -- 172.21.15.43:0/248048004 >> 172.21.15.43:0/248048004 conn(0x7fa4800fd520 msgr2=0x7fa4800ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:18.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.999+0000 7fa4872e4640 1 -- 172.21.15.43:0/248048004 shutdown_connections 2024-04-21T08:38:18.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.999+0000 7fa4872e4640 1 -- 172.21.15.43:0/248048004 wait complete. 2024-04-21T08:38:18.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.999+0000 7fa4872e4640 1 Processor -- start 2024-04-21T08:38:18.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:17.999+0000 7fa4872e4640 1 -- start start 2024-04-21T08:38:18.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.000+0000 7fa4872e4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa480101fb0 0x7fa4801981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:18.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.000+0000 7fa4872e4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4801031a0 0x7fa4801986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:18.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.000+0000 7fa4872e4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa480103b60 0x7fa48019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:18.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.000+0000 7fa4872e4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa48010dab0 con 0x7fa480103b60 2024-04-21T08:38:18.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.000+0000 7fa4872e4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa48010d930 con 0x7fa4801031a0 2024-04-21T08:38:18.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.000+0000 7fa4872e4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa48010dc30 con 0x7fa480101fb0 2024-04-21T08:38:18.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.000+0000 7fa485881640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa480103b60 0x7fa48019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:18.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.000+0000 7fa485881640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa480103b60 0x7fa48019f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54888/0 (socket says 172.21.15.43:54888) 2024-04-21T08:38:18.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.000+0000 7fa485881640 1 -- 172.21.15.43:0/1093091359 learned_addr learned my addr 172.21.15.43:0/1093091359 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:18.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.000+0000 7fa48487f640 1 --2- 172.21.15.43:0/1093091359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4801031a0 0x7fa4801986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:18.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.000+0000 7fa485881640 1 -- 172.21.15.43:0/1093091359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa480101fb0 msgr2=0x7fa4801981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:18.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.000+0000 7fa485881640 1 --2- 172.21.15.43:0/1093091359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa480101fb0 0x7fa4801981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.000+0000 7fa485881640 1 -- 172.21.15.43:0/1093091359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4801031a0 msgr2=0x7fa4801986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:18.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.000+0000 7fa485881640 1 --2- 172.21.15.43:0/1093091359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4801031a0 0x7fa4801986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.000+0000 7fa485881640 1 -- 172.21.15.43:0/1093091359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48019fdc0 con 0x7fa480103b60 2024-04-21T08:38:18.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.000+0000 7fa48487f640 1 --2- 172.21.15.43:0/1093091359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4801031a0 0x7fa4801986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:18.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.000+0000 7fa485881640 1 --2- 172.21.15.43:0/1093091359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa480103b60 0x7fa48019f750 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa478032520 tx=0x7fa4780347b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:18.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.000+0000 7fa4767fc640 1 -- 172.21.15.43:0/1093091359 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa478039040 con 0x7fa480103b60 2024-04-21T08:38:18.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.000+0000 7fa4767fc640 1 -- 172.21.15.43:0/1093091359 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa478044070 con 0x7fa480103b60 2024-04-21T08:38:18.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.000+0000 7fa4872e4640 1 -- 172.21.15.43:0/1093091359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4801a0050 con 0x7fa480103b60 2024-04-21T08:38:18.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.000+0000 7fa4767fc640 1 -- 172.21.15.43:0/1093091359 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4780042a0 con 0x7fa480103b60 2024-04-21T08:38:18.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.000+0000 7fa4872e4640 1 -- 172.21.15.43:0/1093091359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4801a0540 con 0x7fa480103b60 2024-04-21T08:38:18.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.001+0000 7fa4872e4640 1 -- 172.21.15.43:0/1093091359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4801093a0 con 0x7fa480103b60 2024-04-21T08:38:18.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.003+0000 7fa4767fc640 1 -- 172.21.15.43:0/1093091359 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa4780325c0 con 0x7fa480103b60 2024-04-21T08:38:18.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.003+0000 7fa4767fc640 1 --2- 172.21.15.43:0/1093091359 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa46c0a3bb0 0x7fa46c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:18.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.003+0000 7fa4767fc640 1 -- 172.21.15.43:0/1093091359 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fa4780f32a0 con 0x7fa480103b60 2024-04-21T08:38:18.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.003+0000 7fa485080640 1 --2- 172.21.15.43:0/1093091359 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa46c0a3bb0 0x7fa46c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:18.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.004+0000 7fa485080640 1 --2- 172.21.15.43:0/1093091359 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa46c0a3bb0 0x7fa46c0a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa480103000 tx=0x7fa47c005ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:18.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.007+0000 7fa4767fc640 1 -- 172.21.15.43:0/1093091359 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa4780bf890 con 0x7fa480103b60 2024-04-21T08:38:18.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.192+0000 7fa4872e4640 1 -- 172.21.15.43:0/1093091359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa48002d5c0 con 0x7fa480103b60 2024-04-21T08:38:18.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.193+0000 7fa4767fc640 1 -- 172.21.15.43:0/1093091359 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fa4780bf190 con 0x7fa480103b60 2024-04-21T08:38:18.193 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:38:18.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.195+0000 7fa4872e4640 1 -- 172.21.15.43:0/1093091359 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa46c0a3bb0 msgr2=0x7fa46c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:18.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.195+0000 7fa4872e4640 1 --2- 172.21.15.43:0/1093091359 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa46c0a3bb0 0x7fa46c0a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa480103000 tx=0x7fa47c005ed0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.195+0000 7fa4872e4640 1 -- 172.21.15.43:0/1093091359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa480103b60 msgr2=0x7fa48019f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:18.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.195+0000 7fa4872e4640 1 --2- 172.21.15.43:0/1093091359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa480103b60 0x7fa48019f750 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa478032520 tx=0x7fa4780347b0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.195+0000 7fa4872e4640 1 -- 172.21.15.43:0/1093091359 shutdown_connections 2024-04-21T08:38:18.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.195+0000 7fa4872e4640 1 --2- 172.21.15.43:0/1093091359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa480103b60 0x7fa48019f750 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.195+0000 7fa4872e4640 1 --2- 172.21.15.43:0/1093091359 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa46c0a3bb0 0x7fa46c0a6070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.195+0000 7fa4872e4640 1 --2- 172.21.15.43:0/1093091359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4801031a0 0x7fa4801986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.195+0000 7fa4872e4640 1 --2- 172.21.15.43:0/1093091359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa480101fb0 0x7fa4801981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.195+0000 7fa4872e4640 1 -- 172.21.15.43:0/1093091359 >> 172.21.15.43:0/1093091359 conn(0x7fa4800fd520 msgr2=0x7fa480106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:18.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.195+0000 7fa4872e4640 1 -- 172.21.15.43:0/1093091359 shutdown_connections 2024-04-21T08:38:18.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.195+0000 7fa4872e4640 1 -- 172.21.15.43:0/1093091359 wait complete. 2024-04-21T08:38:18.206 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:38:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.308+0000 7fa3f76d5640 1 -- 172.21.15.43:0/3848672485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f0101ff0 msgr2=0x7fa3f0102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.308+0000 7fa3f76d5640 1 --2- 172.21.15.43:0/3848672485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f0101ff0 0x7fa3f0102470 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009900 tx=0x7fa3e002bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.308+0000 7fa3f76d5640 1 -- 172.21.15.43:0/3848672485 shutdown_connections 2024-04-21T08:38:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.308+0000 7fa3f76d5640 1 --2- 172.21.15.43:0/3848672485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f01029b0 0x7fa3f01092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.308+0000 7fa3f76d5640 1 --2- 172.21.15.43:0/3848672485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f0101ff0 0x7fa3f0102470 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.308+0000 7fa3f76d5640 1 --2- 172.21.15.43:0/3848672485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3f0100ea0 0x7fa3f01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.308+0000 7fa3f76d5640 1 -- 172.21.15.43:0/3848672485 >> 172.21.15.43:0/3848672485 conn(0x7fa3f00fc410 msgr2=0x7fa3f00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.308+0000 7fa3f76d5640 1 -- 172.21.15.43:0/3848672485 shutdown_connections 2024-04-21T08:38:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.308+0000 7fa3f76d5640 1 -- 172.21.15.43:0/3848672485 wait complete. 2024-04-21T08:38:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.308+0000 7fa3f76d5640 1 Processor -- start 2024-04-21T08:38:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.308+0000 7fa3f76d5640 1 -- start start 2024-04-21T08:38:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.309+0000 7fa3f76d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f0100ea0 0x7fa3f010b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.309+0000 7fa3f76d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f0101ff0 0x7fa3f0109e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.309+0000 7fa3f76d5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3f01029b0 0x7fa3f010a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.309+0000 7fa3f76d5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f010db40 con 0x7fa3f0100ea0 2024-04-21T08:38:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.309+0000 7fa3f76d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f010d9c0 con 0x7fa3f0101ff0 2024-04-21T08:38:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.309+0000 7fa3f76d5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f010dcc0 con 0x7fa3f01029b0 2024-04-21T08:38:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.309+0000 7fa3f5471640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f0100ea0 0x7fa3f010b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.309+0000 7fa3f5471640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f0100ea0 0x7fa3f010b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54904/0 (socket says 172.21.15.43:54904) 2024-04-21T08:38:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.309+0000 7fa3f5471640 1 -- 172.21.15.43:0/3784611215 learned_addr learned my addr 172.21.15.43:0/3784611215 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.309+0000 7fa3f5c72640 1 --2- 172.21.15.43:0/3784611215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3f01029b0 0x7fa3f010a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.309+0000 7fa3f4c70640 1 --2- 172.21.15.43:0/3784611215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f0101ff0 0x7fa3f0109e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.309+0000 7fa3f5471640 1 -- 172.21.15.43:0/3784611215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3f01029b0 msgr2=0x7fa3f010a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.309+0000 7fa3f5471640 1 --2- 172.21.15.43:0/3784611215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3f01029b0 0x7fa3f010a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.309+0000 7fa3f5471640 1 -- 172.21.15.43:0/3784611215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f0101ff0 msgr2=0x7fa3f0109e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.309+0000 7fa3f5471640 1 --2- 172.21.15.43:0/3784611215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f0101ff0 0x7fa3f0109e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.309+0000 7fa3f5471640 1 -- 172.21.15.43:0/3784611215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f010ac50 con 0x7fa3f0100ea0 2024-04-21T08:38:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.309+0000 7fa3f5471640 1 --2- 172.21.15.43:0/3784611215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f0100ea0 0x7fa3f010b840 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00d770 tx=0x7fa3ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.309+0000 7fa3e67fc640 1 -- 172.21.15.43:0/3784611215 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec017070 con 0x7fa3f0100ea0 2024-04-21T08:38:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.309+0000 7fa3e67fc640 1 -- 172.21.15.43:0/3784611215 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3ec0054d0 con 0x7fa3f0100ea0 2024-04-21T08:38:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.309+0000 7fa3f76d5640 1 -- 172.21.15.43:0/3784611215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f01a22f0 con 0x7fa3f0100ea0 2024-04-21T08:38:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.309+0000 7fa3e67fc640 1 -- 172.21.15.43:0/3784611215 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec004c30 con 0x7fa3f0100ea0 2024-04-21T08:38:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.310+0000 7fa3f76d5640 1 -- 172.21.15.43:0/3784611215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f01a27c0 con 0x7fa3f0100ea0 2024-04-21T08:38:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.310+0000 7fa3f76d5640 1 -- 172.21.15.43:0/3784611215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f0108280 con 0x7fa3f0100ea0 2024-04-21T08:38:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.312+0000 7fa3e67fc640 1 -- 172.21.15.43:0/3784611215 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa3ec004e50 con 0x7fa3f0100ea0 2024-04-21T08:38:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.312+0000 7fa3e67fc640 1 --2- 172.21.15.43:0/3784611215 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa3dc0a3bb0 0x7fa3dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.312+0000 7fa3f4c70640 1 --2- 172.21.15.43:0/3784611215 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa3dc0a3bb0 0x7fa3dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:18.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.313+0000 7fa3e67fc640 1 -- 172.21.15.43:0/3784611215 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fa3ec0d0280 con 0x7fa3f0100ea0 2024-04-21T08:38:18.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.313+0000 7fa3f4c70640 1 --2- 172.21.15.43:0/3784611215 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa3dc0a3bb0 0x7fa3dc0a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0005ab0 tx=0x7fa3e0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:18.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.317+0000 7fa3e67fc640 1 -- 172.21.15.43:0/3784611215 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa3ec09c1c0 con 0x7fa3f0100ea0 2024-04-21T08:38:18.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.547+0000 7fa3f76d5640 1 -- 172.21.15.43:0/3784611215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3f002d5c0 con 0x7fa3f0100ea0 2024-04-21T08:38:18.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.548+0000 7fa3e67fc640 1 -- 172.21.15.43:0/3784611215 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11938 (secure 0 0 0) 0x7fa3ec0a04c0 con 0x7fa3f0100ea0 2024-04-21T08:38:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:18.548 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:36:51.902455+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24491,24527,24487,24425,24412]},"id":1}]} 2024-04-21T08:38:18.548 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 27 2024-04-21T08:38:18.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.549+0000 7fa3f76d5640 1 -- 172.21.15.43:0/3784611215 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa3dc0a3bb0 msgr2=0x7fa3dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:18.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.549+0000 7fa3f76d5640 1 --2- 172.21.15.43:0/3784611215 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa3dc0a3bb0 0x7fa3dc0a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0005ab0 tx=0x7fa3e0005fd0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.549+0000 7fa3f76d5640 1 -- 172.21.15.43:0/3784611215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f0100ea0 msgr2=0x7fa3f010b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:18.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.549+0000 7fa3f76d5640 1 --2- 172.21.15.43:0/3784611215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f0100ea0 0x7fa3f010b840 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00d770 tx=0x7fa3ec00dc30 comp rx=0 tx=0).stop 2024-04-21T08:38:18.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.550+0000 7fa3f76d5640 1 -- 172.21.15.43:0/3784611215 shutdown_connections 2024-04-21T08:38:18.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.550+0000 7fa3f76d5640 1 --2- 172.21.15.43:0/3784611215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3f01029b0 0x7fa3f010a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.550+0000 7fa3f76d5640 1 --2- 172.21.15.43:0/3784611215 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa3dc0a3bb0 0x7fa3dc0a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.550+0000 7fa3f76d5640 1 --2- 172.21.15.43:0/3784611215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f0101ff0 0x7fa3f0109e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.550+0000 7fa3f76d5640 1 --2- 172.21.15.43:0/3784611215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f0100ea0 0x7fa3f010b840 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.550+0000 7fa3f76d5640 1 -- 172.21.15.43:0/3784611215 >> 172.21.15.43:0/3784611215 conn(0x7fa3f00fc410 msgr2=0x7fa3f0105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:18.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.550+0000 7fa3f76d5640 1 -- 172.21.15.43:0/3784611215 shutdown_connections 2024-04-21T08:38:18.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.550+0000 7fa3f76d5640 1 -- 172.21.15.43:0/3784611215 wait complete. 2024-04-21T08:38:18.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:18 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1788197620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:18.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:18 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3789816880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:18.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:18 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1093091359' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:18.563 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:38:18.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.665+0000 7f56c5915640 1 -- 172.21.15.43:0/1284161841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56c0101ff0 msgr2=0x7f56c0102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:18.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.665+0000 7f56c5915640 1 --2- 172.21.15.43:0/1284161841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56c0101ff0 0x7f56c0102470 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f56a8009900 tx=0x7f56a802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:18.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.665+0000 7f56c5915640 1 -- 172.21.15.43:0/1284161841 shutdown_connections 2024-04-21T08:38:18.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.665+0000 7f56c5915640 1 --2- 172.21.15.43:0/1284161841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c01029b0 0x7f56c01092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.665+0000 7f56c5915640 1 --2- 172.21.15.43:0/1284161841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56c0101ff0 0x7f56c0102470 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.665+0000 7f56c5915640 1 --2- 172.21.15.43:0/1284161841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56c0100ea0 0x7f56c01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.665+0000 7f56c5915640 1 -- 172.21.15.43:0/1284161841 >> 172.21.15.43:0/1284161841 conn(0x7f56c00fc410 msgr2=0x7f56c00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:18.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.665+0000 7f56c5915640 1 -- 172.21.15.43:0/1284161841 shutdown_connections 2024-04-21T08:38:18.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.665+0000 7f56c5915640 1 -- 172.21.15.43:0/1284161841 wait complete. 2024-04-21T08:38:18.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.665+0000 7f56c5915640 1 Processor -- start 2024-04-21T08:38:18.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.666+0000 7f56c5915640 1 -- start start 2024-04-21T08:38:18.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.666+0000 7f56c5915640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56c0100ea0 0x7f56c006b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:18.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.666+0000 7f56c5915640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c0101ff0 0x7f56c019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:18.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.666+0000 7f56c5915640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56c01029b0 0x7f56c019f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:18.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.666+0000 7f56c5915640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f56c010caa0 con 0x7f56c0100ea0 2024-04-21T08:38:18.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.666+0000 7f56c5915640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f56c010c920 con 0x7f56c0101ff0 2024-04-21T08:38:18.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.666+0000 7f56c5915640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f56c010cc20 con 0x7f56c01029b0 2024-04-21T08:38:18.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.666+0000 7f56beffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56c0100ea0 0x7f56c006b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:18.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.666+0000 7f56beffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56c0100ea0 0x7f56c006b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54924/0 (socket says 172.21.15.43:54924) 2024-04-21T08:38:18.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.666+0000 7f56beffd640 1 -- 172.21.15.43:0/760554590 learned_addr learned my addr 172.21.15.43:0/760554590 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:18.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.666+0000 7f56be7fc640 1 --2- 172.21.15.43:0/760554590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c0101ff0 0x7f56c019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:18.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.666+0000 7f56beffd640 1 -- 172.21.15.43:0/760554590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56c01029b0 msgr2=0x7f56c019f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:18.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.666+0000 7f56beffd640 1 --2- 172.21.15.43:0/760554590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56c01029b0 0x7f56c019f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.666+0000 7f56beffd640 1 -- 172.21.15.43:0/760554590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c0101ff0 msgr2=0x7f56c019cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:18.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.666+0000 7f56beffd640 1 --2- 172.21.15.43:0/760554590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c0101ff0 0x7f56c019cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.666+0000 7f56beffd640 1 -- 172.21.15.43:0/760554590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c019fdc0 con 0x7f56c0100ea0 2024-04-21T08:38:18.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.666+0000 7f56beffd640 1 --2- 172.21.15.43:0/760554590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56c0100ea0 0x7f56c006b1d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f56b000b4a0 tx=0x7f56b000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:18.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.667+0000 7f56c4913640 1 -- 172.21.15.43:0/760554590 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b0014070 con 0x7f56c0100ea0 2024-04-21T08:38:18.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.667+0000 7f56c4913640 1 -- 172.21.15.43:0/760554590 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56b0010ac0 con 0x7f56c0100ea0 2024-04-21T08:38:18.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.667+0000 7f56c5915640 1 -- 172.21.15.43:0/760554590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c01a00b0 con 0x7f56c0100ea0 2024-04-21T08:38:18.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.667+0000 7f56c4913640 1 -- 172.21.15.43:0/760554590 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b0017c30 con 0x7f56c0100ea0 2024-04-21T08:38:18.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.667+0000 7f56c5915640 1 -- 172.21.15.43:0/760554590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c01a0600 con 0x7f56c0100ea0 2024-04-21T08:38:18.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.668+0000 7f56c5915640 1 -- 172.21.15.43:0/760554590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56c0108150 con 0x7f56c0100ea0 2024-04-21T08:38:18.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.669+0000 7f56c4913640 1 -- 172.21.15.43:0/760554590 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f56b0017070 con 0x7f56c0100ea0 2024-04-21T08:38:18.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.669+0000 7f56c4913640 1 --2- 172.21.15.43:0/760554590 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f56ac0a3bb0 0x7f56ac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:18.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.669+0000 7f56be7fc640 1 --2- 172.21.15.43:0/760554590 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f56ac0a3bb0 0x7f56ac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:18.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.670+0000 7f56c4913640 1 -- 172.21.15.43:0/760554590 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f56b00cfc40 con 0x7f56c0100ea0 2024-04-21T08:38:18.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.670+0000 7f56be7fc640 1 --2- 172.21.15.43:0/760554590 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f56ac0a3bb0 0x7f56ac0a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f56a802c240 tx=0x7f56a8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:18.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.674+0000 7f56c4913640 1 -- 172.21.15.43:0/760554590 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56b009c230 con 0x7f56c0100ea0 2024-04-21T08:38:18.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:18 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1788197620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:18.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:18 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3789816880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:18.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:18 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1093091359' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:18.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:18 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1788197620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:18.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:18 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3789816880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:18.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:18 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1093091359' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:18.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.862+0000 7f56c5915640 1 -- 172.21.15.43:0/760554590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56c0062bf0 con 0x7f56c0100ea0 2024-04-21T08:38:18.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.863+0000 7f56c4913640 1 -- 172.21.15.43:0/760554590 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24856 (secure 0 0 0) 0x7f56b009bb30 con 0x7f56c0100ea0 2024-04-21T08:38:18.864 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:18.864 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":94,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:38:15.326063+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:38:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.865+0000 7f56c5915640 1 -- 172.21.15.43:0/760554590 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f56ac0a3bb0 msgr2=0x7f56ac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.865+0000 7f56c5915640 1 --2- 172.21.15.43:0/760554590 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f56ac0a3bb0 0x7f56ac0a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f56a802c240 tx=0x7f56a8000bc0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.865+0000 7f56c5915640 1 -- 172.21.15.43:0/760554590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56c0100ea0 msgr2=0x7f56c006b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.865+0000 7f56c5915640 1 --2- 172.21.15.43:0/760554590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56c0100ea0 0x7f56c006b1d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f56b000b4a0 tx=0x7f56b000b960 comp rx=0 tx=0).stop 2024-04-21T08:38:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.866+0000 7f56c5915640 1 -- 172.21.15.43:0/760554590 shutdown_connections 2024-04-21T08:38:18.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.866+0000 7f56c5915640 1 --2- 172.21.15.43:0/760554590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56c01029b0 0x7f56c019f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.866+0000 7f56c5915640 1 --2- 172.21.15.43:0/760554590 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f56ac0a3bb0 0x7f56ac0a6070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.866+0000 7f56c5915640 1 --2- 172.21.15.43:0/760554590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c0101ff0 0x7f56c019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.866+0000 7f56c5915640 1 --2- 172.21.15.43:0/760554590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56c0100ea0 0x7f56c006b1d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.866+0000 7f56c5915640 1 -- 172.21.15.43:0/760554590 >> 172.21.15.43:0/760554590 conn(0x7f56c00fc410 msgr2=0x7f56c0105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:18.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.866+0000 7f56c5915640 1 -- 172.21.15.43:0/760554590 shutdown_connections 2024-04-21T08:38:18.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.866+0000 7f56c5915640 1 -- 172.21.15.43:0/760554590 wait complete. 2024-04-21T08:38:18.878 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:38:18.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.981+0000 7f7b4345c640 1 -- 172.21.15.43:0/511340733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b3c103190 msgr2=0x7f7b3c103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:18.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.981+0000 7f7b4345c640 1 --2- 172.21.15.43:0/511340733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b3c103190 0x7f7b3c103610 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7b38009900 tx=0x7f7b3802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:18.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.981+0000 7f7b4345c640 1 -- 172.21.15.43:0/511340733 shutdown_connections 2024-04-21T08:38:18.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.981+0000 7f7b4345c640 1 --2- 172.21.15.43:0/511340733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b3c103b50 0x7f7b3c10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.981+0000 7f7b4345c640 1 --2- 172.21.15.43:0/511340733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b3c103190 0x7f7b3c103610 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.981+0000 7f7b4345c640 1 --2- 172.21.15.43:0/511340733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b3c101fa0 0x7f7b3c1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.981+0000 7f7b4345c640 1 -- 172.21.15.43:0/511340733 >> 172.21.15.43:0/511340733 conn(0x7f7b3c0fd530 msgr2=0x7f7b3c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:18.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.981+0000 7f7b4345c640 1 -- 172.21.15.43:0/511340733 shutdown_connections 2024-04-21T08:38:18.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.981+0000 7f7b4345c640 1 -- 172.21.15.43:0/511340733 wait complete. 2024-04-21T08:38:18.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.982+0000 7f7b4345c640 1 Processor -- start 2024-04-21T08:38:18.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.982+0000 7f7b4345c640 1 -- start start 2024-04-21T08:38:18.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.982+0000 7f7b4345c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b3c101fa0 0x7f7b3c198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:18.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.982+0000 7f7b4345c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b3c103190 0x7f7b3c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:18.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.982+0000 7f7b4345c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b3c103b50 0x7f7b3c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:18.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.982+0000 7f7b4345c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3c10db30 con 0x7f7b3c103190 2024-04-21T08:38:18.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.982+0000 7f7b4345c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3c10d9b0 con 0x7f7b3c103b50 2024-04-21T08:38:18.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.982+0000 7f7b4345c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3c10dcb0 con 0x7f7b3c101fa0 2024-04-21T08:38:18.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.982+0000 7f7b409f7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b3c103190 0x7f7b3c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:18.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.982+0000 7f7b409f7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b3c103190 0x7f7b3c198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54942/0 (socket says 172.21.15.43:54942) 2024-04-21T08:38:18.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.982+0000 7f7b409f7640 1 -- 172.21.15.43:0/3766377596 learned_addr learned my addr 172.21.15.43:0/3766377596 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:18.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.982+0000 7f7b419f9640 1 --2- 172.21.15.43:0/3766377596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b3c103b50 0x7f7b3c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:18.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.982+0000 7f7b411f8640 1 --2- 172.21.15.43:0/3766377596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b3c101fa0 0x7f7b3c198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:18.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.982+0000 7f7b409f7640 1 -- 172.21.15.43:0/3766377596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b3c101fa0 msgr2=0x7f7b3c198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:18.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.982+0000 7f7b409f7640 1 --2- 172.21.15.43:0/3766377596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b3c101fa0 0x7f7b3c198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.982+0000 7f7b409f7640 1 -- 172.21.15.43:0/3766377596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b3c103b50 msgr2=0x7f7b3c19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:18.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.982+0000 7f7b409f7640 1 --2- 172.21.15.43:0/3766377596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b3c103b50 0x7f7b3c19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.982+0000 7f7b409f7640 1 -- 172.21.15.43:0/3766377596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b3c19fe30 con 0x7f7b3c103190 2024-04-21T08:38:18.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.982+0000 7f7b419f9640 1 --2- 172.21.15.43:0/3766377596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b3c103b50 0x7f7b3c19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:18.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.983+0000 7f7b409f7640 1 --2- 172.21.15.43:0/3766377596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b3c103190 0x7f7b3c198750 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7b38000c00 tx=0x7f7b380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:18.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.983+0000 7f7b327fc640 1 -- 172.21.15.43:0/3766377596 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b38043020 con 0x7f7b3c103190 2024-04-21T08:38:18.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.983+0000 7f7b4345c640 1 -- 172.21.15.43:0/3766377596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b3c1a00c0 con 0x7f7b3c103190 2024-04-21T08:38:18.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.983+0000 7f7b327fc640 1 -- 172.21.15.43:0/3766377596 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b380054c0 con 0x7f7b3c103190 2024-04-21T08:38:18.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.983+0000 7f7b327fc640 1 -- 172.21.15.43:0/3766377596 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b38003d80 con 0x7f7b3c103190 2024-04-21T08:38:18.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.983+0000 7f7b4345c640 1 -- 172.21.15.43:0/3766377596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b3c1a0660 con 0x7f7b3c103190 2024-04-21T08:38:18.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.984+0000 7f7b4345c640 1 -- 172.21.15.43:0/3766377596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b3c109330 con 0x7f7b3c103190 2024-04-21T08:38:18.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.985+0000 7f7b327fc640 1 -- 172.21.15.43:0/3766377596 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7b38046020 con 0x7f7b3c103190 2024-04-21T08:38:18.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.985+0000 7f7b327fc640 1 --2- 172.21.15.43:0/3766377596 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f7b280a3bb0 0x7f7b280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:18.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.986+0000 7f7b327fc640 1 -- 172.21.15.43:0/3766377596 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f7b380f3200 con 0x7f7b3c103190 2024-04-21T08:38:18.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.986+0000 7f7b411f8640 1 --2- 172.21.15.43:0/3766377596 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f7b280a3bb0 0x7f7b280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:18.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.986+0000 7f7b411f8640 1 --2- 172.21.15.43:0/3766377596 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f7b280a3bb0 0x7f7b280a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c005db0 tx=0x7f7b2c005d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:18.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:18.989+0000 7f7b327fc640 1 -- 172.21.15.43:0/3766377596 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b3804d020 con 0x7f7b3c103190 2024-04-21T08:38:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.221+0000 7f7b4345c640 1 -- 172.21.15.43:0/3766377596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b3c02d5c0 con 0x7f7b3c103190 2024-04-21T08:38:19.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.222+0000 7f7b327fc640 1 -- 172.21.15.43:0/3766377596 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11938 (secure 0 0 0) 0x7f7b380bf910 con 0x7f7b3c103190 2024-04-21T08:38:19.222 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:19.222 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:36:51.902455+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24491,24527,24487,24425,24412]},"id":1}]} 2024-04-21T08:38:19.222 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 27 2024-04-21T08:38:19.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.224+0000 7f7b4345c640 1 -- 172.21.15.43:0/3766377596 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f7b280a3bb0 msgr2=0x7f7b280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.224+0000 7f7b4345c640 1 --2- 172.21.15.43:0/3766377596 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f7b280a3bb0 0x7f7b280a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c005db0 tx=0x7f7b2c005d40 comp rx=0 tx=0).stop 2024-04-21T08:38:19.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.224+0000 7f7b4345c640 1 -- 172.21.15.43:0/3766377596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b3c103190 msgr2=0x7f7b3c198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.224+0000 7f7b4345c640 1 --2- 172.21.15.43:0/3766377596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b3c103190 0x7f7b3c198750 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7b38000c00 tx=0x7f7b380042f0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.224+0000 7f7b4345c640 1 -- 172.21.15.43:0/3766377596 shutdown_connections 2024-04-21T08:38:19.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.224+0000 7f7b4345c640 1 --2- 172.21.15.43:0/3766377596 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f7b280a3bb0 0x7f7b280a6070 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.224+0000 7f7b4345c640 1 --2- 172.21.15.43:0/3766377596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b3c103b50 0x7f7b3c19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.224+0000 7f7b4345c640 1 --2- 172.21.15.43:0/3766377596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b3c103190 0x7f7b3c198750 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.224+0000 7f7b4345c640 1 --2- 172.21.15.43:0/3766377596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b3c101fa0 0x7f7b3c198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.224+0000 7f7b4345c640 1 -- 172.21.15.43:0/3766377596 >> 172.21.15.43:0/3766377596 conn(0x7f7b3c0fd530 msgr2=0x7f7b3c106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.224+0000 7f7b4345c640 1 -- 172.21.15.43:0/3766377596 shutdown_connections 2024-04-21T08:38:19.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.224+0000 7f7b4345c640 1 -- 172.21.15.43:0/3766377596 wait complete. 2024-04-21T08:38:19.236 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-21T08:36:23.821253+0000', 'modified': '2024-04-21T08:36:51.902455+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24527, 'mds_1': 24491, 'mds_2': 24487, 'mds_3': 24425, 'mds_4': 24412}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14634': {'gid': 14634, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.43:6839/931821965', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6838', 'nonce': 931821965}, {'type': 'v1', 'addr': '172.21.15.43:6839', 'nonce': 931821965}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14652': {'gid': 14652, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.43:6841/623279035', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6840', 'nonce': 623279035}, {'type': 'v1', 'addr': '172.21.15.43:6841', 'nonce': 623279035}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24412': {'gid': 24412, 'name': 'l', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.175:6833/187945366', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6832', 'nonce': 187945366}, {'type': 'v1', 'addr': '172.21.15.175:6833', 'nonce': 187945366}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.112:6833/1516781885', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6832', 'nonce': 1516781885}, {'type': 'v1', 'addr': '172.21.15.112:6833', 'nonce': 1516781885}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6835/466532477', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6834', 'nonce': 466532477}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 466532477}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.112:6835/3159749962', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6834', 'nonce': 3159749962}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 3159749962}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6837/3198693848', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6836', 'nonce': 3198693848}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 3198693848}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.175:6839/3664054463', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 3664054463}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 3664054463}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.112:6837/3054508488', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6836', 'nonce': 3054508488}, {'type': 'v1', 'addr': '172.21.15.112:6837', 'nonce': 3054508488}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24527': {'gid': 24527, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6839/2143575300', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6838', 'nonce': 2143575300}, {'type': 'v1', 'addr': '172.21.15.112:6839', 'nonce': 2143575300}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24527, 'qdb_cluster': [24491, 24527, 24487, 24425, 24412]} 2024-04-21T08:38:19.236 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-21T08:38:19.236 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-04-21T08:38:19.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.339+0000 7f7d8c270640 1 -- 172.21.15.43:0/4013033082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d84102a00 msgr2=0x7f7d84109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.339+0000 7f7d8c270640 1 --2- 172.21.15.43:0/4013033082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d84102a00 0x7f7d84109320 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c009ca0 tx=0x7f7d7c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.339+0000 7f7d8c270640 1 -- 172.21.15.43:0/4013033082 shutdown_connections 2024-04-21T08:38:19.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.339+0000 7f7d8c270640 1 --2- 172.21.15.43:0/4013033082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d84102a00 0x7f7d84109320 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.339+0000 7f7d8c270640 1 --2- 172.21.15.43:0/4013033082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d84102040 0x7f7d841024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.339+0000 7f7d8c270640 1 --2- 172.21.15.43:0/4013033082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d84100e60 0x7f7d84101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.339+0000 7f7d8c270640 1 -- 172.21.15.43:0/4013033082 >> 172.21.15.43:0/4013033082 conn(0x7f7d840fc440 msgr2=0x7f7d840fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.339+0000 7f7d8c270640 1 -- 172.21.15.43:0/4013033082 shutdown_connections 2024-04-21T08:38:19.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.339+0000 7f7d8c270640 1 -- 172.21.15.43:0/4013033082 wait complete. 2024-04-21T08:38:19.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.340+0000 7f7d8c270640 1 Processor -- start 2024-04-21T08:38:19.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.340+0000 7f7d8c270640 1 -- start start 2024-04-21T08:38:19.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.340+0000 7f7d8c270640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d84100e60 0x7f7d8410b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.340+0000 7f7d8c270640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d84102040 0x7f7d84109ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.340+0000 7f7d8c270640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d84102a00 0x7f7d8410a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.340+0000 7f7d8c270640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8410db30 con 0x7f7d84100e60 2024-04-21T08:38:19.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.340+0000 7f7d8c270640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8410d9b0 con 0x7f7d84102040 2024-04-21T08:38:19.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.340+0000 7f7d8c270640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8410dcb0 con 0x7f7d84102a00 2024-04-21T08:38:19.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.340+0000 7f7d8a00c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d84100e60 0x7f7d8410b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.340+0000 7f7d8a00c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d84100e60 0x7f7d8410b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54966/0 (socket says 172.21.15.43:54966) 2024-04-21T08:38:19.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.340+0000 7f7d8a00c640 1 -- 172.21.15.43:0/3515691085 learned_addr learned my addr 172.21.15.43:0/3515691085 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.340+0000 7f7d8980b640 1 --2- 172.21.15.43:0/3515691085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d84102040 0x7f7d84109ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.340+0000 7f7d8a80d640 1 --2- 172.21.15.43:0/3515691085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d84102a00 0x7f7d8410a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.340+0000 7f7d8a00c640 1 -- 172.21.15.43:0/3515691085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d84102a00 msgr2=0x7f7d8410a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.340+0000 7f7d8a00c640 1 --2- 172.21.15.43:0/3515691085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d84102a00 0x7f7d8410a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.340+0000 7f7d8a00c640 1 -- 172.21.15.43:0/3515691085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d84102040 msgr2=0x7f7d84109ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.340+0000 7f7d8a00c640 1 --2- 172.21.15.43:0/3515691085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d84102040 0x7f7d84109ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.340+0000 7f7d8a00c640 1 -- 172.21.15.43:0/3515691085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8410ac70 con 0x7f7d84100e60 2024-04-21T08:38:19.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.340+0000 7f7d8980b640 1 --2- 172.21.15.43:0/3515691085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d84102040 0x7f7d84109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:19.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.341+0000 7f7d8a00c640 1 --2- 172.21.15.43:0/3515691085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d84100e60 0x7f7d8410b860 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400d950 tx=0x7f7d7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.341+0000 7f7d7b7fe640 1 -- 172.21.15.43:0/3515691085 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d74017020 con 0x7f7d84100e60 2024-04-21T08:38:19.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.341+0000 7f7d7b7fe640 1 -- 172.21.15.43:0/3515691085 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d74009c30 con 0x7f7d84100e60 2024-04-21T08:38:19.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.341+0000 7f7d8c270640 1 -- 172.21.15.43:0/3515691085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d841a22d0 con 0x7f7d84100e60 2024-04-21T08:38:19.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.341+0000 7f7d7b7fe640 1 -- 172.21.15.43:0/3515691085 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7400f690 con 0x7f7d84100e60 2024-04-21T08:38:19.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.341+0000 7f7d8c270640 1 -- 172.21.15.43:0/3515691085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d841a2870 con 0x7f7d84100e60 2024-04-21T08:38:19.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.343+0000 7f7d7b7fe640 1 -- 172.21.15.43:0/3515691085 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7d7401a020 con 0x7f7d84100e60 2024-04-21T08:38:19.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.343+0000 7f7d8ee5c740 1 Processor -- start 2024-04-21T08:38:19.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.344+0000 7f7d8ee5c740 1 -- start start 2024-04-21T08:38:19.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.344+0000 7f7d8ee5c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578d1438e40 0x5578d1439260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.344+0000 7f7d8ee5c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d1439830 0x5578d1439cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.344+0000 7f7d8ee5c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578d143a2d0 0x5578d1440eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.344+0000 7f7d8ee5c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5578d139db10 con 0x5578d1439830 2024-04-21T08:38:19.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.344+0000 7f7d8ee5c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5578d133b240 con 0x5578d1438e40 2024-04-21T08:38:19.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.344+0000 7f7d8ee5c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5578d1399ad0 con 0x5578d143a2d0 2024-04-21T08:38:19.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.344+0000 7f7d8a00c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d1439830 0x5578d1439cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.344+0000 7f7d8a00c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d1439830 0x5578d1439cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54982/0 (socket says 172.21.15.43:54982) 2024-04-21T08:38:19.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.344+0000 7f7d7b7fe640 1 --2- 172.21.15.43:0/3515691085 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f7d700a3c30 0x7f7d700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.344+0000 7f7d8a00c640 1 -- 172.21.15.43:0/1523461069 learned_addr learned my addr 172.21.15.43:0/1523461069 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.344+0000 7f7d8980b640 1 --2- 172.21.15.43:0/1523461069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578d143a2d0 0x5578d1440eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.344+0000 7f7d8a80d640 1 --2- 172.21.15.43:0/1523461069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578d1438e40 0x5578d1439260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.344+0000 7f7d8a80d640 1 --2- 172.21.15.43:0/3515691085 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f7d700a3c30 0x7f7d700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.344+0000 7f7d8980b640 1 -- 172.21.15.43:0/1523461069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578d1438e40 msgr2=0x5578d1439260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.344+0000 7f7d7b7fe640 1 -- 172.21.15.43:0/3515691085 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f7d740d0820 con 0x7f7d84100e60 2024-04-21T08:38:19.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.344+0000 7f7d8980b640 1 --2- 172.21.15.43:0/1523461069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578d1438e40 0x5578d1439260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.344+0000 7f7d8980b640 1 -- 172.21.15.43:0/1523461069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d1439830 msgr2=0x5578d1439cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.344+0000 7f7d8980b640 1 --2- 172.21.15.43:0/1523461069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d1439830 0x5578d1439cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.344+0000 7f7d8980b640 1 -- 172.21.15.43:0/1523461069 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578d10b02d0 con 0x5578d143a2d0 2024-04-21T08:38:19.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.344+0000 7f7d8a00c640 1 --2- 172.21.15.43:0/1523461069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d1439830 0x5578d1439cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:38:19.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.345+0000 7f7d8a80d640 1 --2- 172.21.15.43:0/3515691085 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f7d700a3c30 0x7f7d700a60f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c006010 tx=0x7f7d7c032340 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.345+0000 7f7d8980b640 1 --2- 172.21.15.43:0/1523461069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578d143a2d0 0x5578d1440eb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009b80 tx=0x7f7d800117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98607a18ea607e12 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.345+0000 7f7d78bfe640 1 -- 172.21.15.43:0/1523461069 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80005020 con 0x5578d143a2d0 2024-04-21T08:38:19.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.345+0000 7f7d78bfe640 1 -- 172.21.15.43:0/1523461069 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d800047a0 con 0x5578d143a2d0 2024-04-21T08:38:19.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.345+0000 7f7d78bfe640 1 -- 172.21.15.43:0/1523461069 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d800126b0 con 0x5578d143a2d0 2024-04-21T08:38:19.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.346+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/1523461069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578d143a2d0 msgr2=0x5578d1440eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.346+0000 7f7d8ee5c740 1 --2- 172.21.15.43:0/1523461069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578d143a2d0 0x5578d1440eb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009b80 tx=0x7f7d800117b0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.346+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/1523461069 shutdown_connections 2024-04-21T08:38:19.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.346+0000 7f7d8ee5c740 1 --2- 172.21.15.43:0/1523461069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578d143a2d0 0x5578d1440eb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.346+0000 7f7d8ee5c740 1 --2- 172.21.15.43:0/1523461069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d1439830 0x5578d1439cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.346+0000 7f7d8ee5c740 1 --2- 172.21.15.43:0/1523461069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578d1438e40 0x5578d1439260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.346+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/1523461069 >> 172.21.15.43:0/1523461069 conn(0x5578d1434f40 msgr2=0x5578d1433490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.346+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/1523461069 shutdown_connections 2024-04-21T08:38:19.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.346+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/1523461069 wait complete. 2024-04-21T08:38:19.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.346+0000 7f7d8ee5c740 1 -- start start 2024-04-21T08:38:19.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.346+0000 7f7d8ee5c740 1 Processor -- start 2024-04-21T08:38:19.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.346+0000 7f7d8ee5c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578d1438e40 0x5578d1442e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.346+0000 7f7d8ee5c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578d1439830 0x5578d1443360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.346+0000 7f7d8ee5c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d143a2d0 0x5578d144a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.346+0000 7f7d8ee5c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5578d14414b0 con 0x5578d143a2d0 2024-04-21T08:38:19.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.346+0000 7f7d8ee5c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5578d1326190 con 0x5578d1439830 2024-04-21T08:38:19.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.346+0000 7f7d8ee5c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5578d1441630 con 0x5578d1438e40 2024-04-21T08:38:19.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.346+0000 7f7d8ee5c740 10 client.0 _ll_register_callbacks cb 0x5578d13218b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:19.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.346+0000 7f7d8a80d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d143a2d0 0x5578d144a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.347+0000 7f7d8a80d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d143a2d0 0x5578d144a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54998/0 (socket says 172.21.15.43:54998) 2024-04-21T08:38:19.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.347+0000 7f7d8a80d640 1 -- 172.21.15.43:0/820605267 learned_addr learned my addr 172.21.15.43:0/820605267 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.347+0000 7f7d8a00c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578d1438e40 0x5578d1442e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.347+0000 7f7d8980b640 1 --2- 172.21.15.43:0/820605267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578d1439830 0x5578d1443360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.347+0000 7f7d8a80d640 1 -- 172.21.15.43:0/820605267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578d1438e40 msgr2=0x5578d1442e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.347+0000 7f7d8a80d640 1 --2- 172.21.15.43:0/820605267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578d1438e40 0x5578d1442e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.347+0000 7f7d8a80d640 1 -- 172.21.15.43:0/820605267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578d1439830 msgr2=0x5578d1443360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.347+0000 7f7d8a80d640 1 --2- 172.21.15.43:0/820605267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578d1439830 0x5578d1443360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.347+0000 7f7d8a80d640 1 -- 172.21.15.43:0/820605267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578d117a6b0 con 0x5578d143a2d0 2024-04-21T08:38:19.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.347+0000 7f7d8a80d640 1 --2- 172.21.15.43:0/820605267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d143a2d0 0x5578d144a3a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c036000 tx=0x7f7d7c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b13b8ee2b5ba30 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.347+0000 7f7d6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:38:19.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.347+0000 7f7d6dffb640 1 -- 172.21.15.43:0/820605267 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c053060 con 0x5578d143a2d0 2024-04-21T08:38:19.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.347+0000 7f7d6dffb640 1 -- 172.21.15.43:0/820605267 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f7d7c004450 con 0x5578d143a2d0 2024-04-21T08:38:19.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.347+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/820605267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d800047a0 con 0x5578d143a2d0 2024-04-21T08:38:19.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.347+0000 7f7d6dffb640 1 -- 172.21.15.43:0/820605267 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7d7c039070 con 0x5578d143a2d0 2024-04-21T08:38:19.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.348+0000 7f7d8ee5c740 10 client.15234 fetch_fsmap learned FSMap version 27 2024-04-21T08:38:19.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.348+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/820605267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5578d144d780 con 0x5578d143a2d0 2024-04-21T08:38:19.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.348+0000 7f7d6dffb640 1 -- 172.21.15.43:0/820605267 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d800047a0 con 0x5578d143a2d0 2024-04-21T08:38:19.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.348+0000 7f7d6dffb640 1 -- 172.21.15.43:0/820605267 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7d7c04d070 con 0x5578d143a2d0 2024-04-21T08:38:19.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.348+0000 7f7d8ee5c740 10 client.15234 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:38:19.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.348+0000 7f7d8ee5c740 10 client.15234 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-21T08:38:19.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.348+0000 7f7d8ee5c740 20 client.15234 populate_metadata read hostname 'smithi043' 2024-04-21T08:38:19.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.348+0000 7f7d8ee5c740 1 --2- 172.21.15.43:0/820605267 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5578d1450330 0x5578d14517a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.348+0000 7f7d8ee5c740 4 client.15234 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:38:19.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.348+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/820605267 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578d13ab530 con 0x5578d1450330 2024-04-21T08:38:19.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.348+0000 7f7d8a00c640 1 --2- 172.21.15.43:0/820605267 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5578d1450330 0x5578d14517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.349+0000 7f7d8a00c640 1 --2- 172.21.15.43:0/820605267 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5578d1450330 0x5578d14517a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7dba3ec9a5f94d9a server_cookie=dff17e684f8179e8 in_seq=0 out_seq=0 2024-04-21T08:38:19.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.349+0000 7f7d6dffb640 10 client.15234 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:38:19.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.350+0000 7f7d6dffb640 1 -- 172.21.15.43:0/820605267 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578d13ab530 con 0x5578d1450330 2024-04-21T08:38:19.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.350+0000 7f7d6dffb640 10 client.15234 handle_command_reply: tid=0 2024-04-21T08:38:19.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.350+0000 7f7d8ee5c740 1 client.15234 shutdown 2024-04-21T08:38:19.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.350+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/820605267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d143a2d0 msgr2=0x5578d144a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.350+0000 7f7d8ee5c740 1 --2- 172.21.15.43:0/820605267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d143a2d0 0x5578d144a3a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c036000 tx=0x7f7d7c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.350+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/820605267 shutdown_connections 2024-04-21T08:38:19.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.350+0000 7f7d8ee5c740 1 --2- 172.21.15.43:0/820605267 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5578d1450330 0x5578d14517a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.350+0000 7f7d8ee5c740 1 --2- 172.21.15.43:0/820605267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d143a2d0 0x5578d144a3a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.350+0000 7f7d6dffb640 0 client.15234 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:38:19.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.350+0000 7f7d8ee5c740 1 --2- 172.21.15.43:0/820605267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578d1439830 0x5578d1443360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.350+0000 7f7d8ee5c740 1 --2- 172.21.15.43:0/820605267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578d1438e40 0x5578d1442e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.350+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/820605267 >> 172.21.15.43:0/820605267 conn(0x5578d1434f40 msgr2=0x5578d1432a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.351+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/820605267 shutdown_connections 2024-04-21T08:38:19.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.351+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/820605267 wait complete. 2024-04-21T08:38:19.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.351+0000 7f7d8ee5c740 20 client.15234 trim_cache size 0 max 16384 2024-04-21T08:38:19.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.369+0000 7f7d8ee5c740 1 Processor -- start 2024-04-21T08:38:19.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.369+0000 7f7d8ee5c740 1 -- start start 2024-04-21T08:38:19.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.369+0000 7f7d8ee5c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578d1438e40 0x5578d142e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.369+0000 7f7d8ee5c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d1439830 0x5578d144a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.369+0000 7f7d8ee5c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578d143a2d0 0x5578d1433420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.369+0000 7f7d8ee5c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5578d144fae0 con 0x5578d1439830 2024-04-21T08:38:19.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.369+0000 7f7d8ee5c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5578d14501b0 con 0x5578d143a2d0 2024-04-21T08:38:19.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.369+0000 7f7d8ee5c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5578d1450880 con 0x5578d1438e40 2024-04-21T08:38:19.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.369+0000 7f7d8980b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d1439830 0x5578d144a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.369+0000 7f7d8980b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d1439830 0x5578d144a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55006/0 (socket says 172.21.15.43:55006) 2024-04-21T08:38:19.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.369+0000 7f7d8980b640 1 -- 172.21.15.43:0/3506506521 learned_addr learned my addr 172.21.15.43:0/3506506521 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.370+0000 7f7d8a80d640 1 --2- 172.21.15.43:0/3506506521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578d143a2d0 0x5578d1433420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.370+0000 7f7d8a00c640 1 --2- 172.21.15.43:0/3506506521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578d1438e40 0x5578d142e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.370+0000 7f7d8980b640 1 -- 172.21.15.43:0/3506506521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578d1438e40 msgr2=0x5578d142e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.370+0000 7f7d8980b640 1 --2- 172.21.15.43:0/3506506521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578d1438e40 0x5578d142e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.370+0000 7f7d8980b640 1 -- 172.21.15.43:0/3506506521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578d143a2d0 msgr2=0x5578d1433420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.370+0000 7f7d8980b640 1 --2- 172.21.15.43:0/3506506521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578d143a2d0 0x5578d1433420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.370+0000 7f7d8980b640 1 -- 172.21.15.43:0/3506506521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578d1434630 con 0x5578d1439830 2024-04-21T08:38:19.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.370+0000 7f7d8a00c640 1 --2- 172.21.15.43:0/3506506521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578d1438e40 0x5578d142e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:19.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.370+0000 7f7d8a80d640 1 --2- 172.21.15.43:0/3506506521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578d143a2d0 0x5578d1433420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:19.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.370+0000 7f7d8980b640 1 --2- 172.21.15.43:0/3506506521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d1439830 0x5578d144a360 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7d80005b30 tx=0x7f7d800097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2c41e24884b3d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.370+0000 7f7d6ffff640 1 -- 172.21.15.43:0/3506506521 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80028020 con 0x5578d1439830 2024-04-21T08:38:19.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.370+0000 7f7d6ffff640 1 -- 172.21.15.43:0/3506506521 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d800095b0 con 0x5578d1439830 2024-04-21T08:38:19.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.370+0000 7f7d6ffff640 1 -- 172.21.15.43:0/3506506521 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80004ae0 con 0x5578d1439830 2024-04-21T08:38:19.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.371+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/3506506521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d1439830 msgr2=0x5578d144a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.371+0000 7f7d8ee5c740 1 --2- 172.21.15.43:0/3506506521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d1439830 0x5578d144a360 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7d80005b30 tx=0x7f7d800097e0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.371+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/3506506521 shutdown_connections 2024-04-21T08:38:19.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.371+0000 7f7d8ee5c740 1 --2- 172.21.15.43:0/3506506521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578d143a2d0 0x5578d1433420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.371+0000 7f7d8ee5c740 1 --2- 172.21.15.43:0/3506506521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d1439830 0x5578d144a360 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.371+0000 7f7d8ee5c740 1 --2- 172.21.15.43:0/3506506521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578d1438e40 0x5578d142e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.371+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/3506506521 >> 172.21.15.43:0/3506506521 conn(0x5578d1434f40 msgr2=0x5578d143d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.371+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/3506506521 shutdown_connections 2024-04-21T08:38:19.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.371+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/3506506521 wait complete. 2024-04-21T08:38:19.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.371+0000 7f7d8ee5c740 1 -- start start 2024-04-21T08:38:19.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.371+0000 7f7d8ee5c740 1 Processor -- start 2024-04-21T08:38:19.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.371+0000 7f7d8ee5c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578d1438e40 0x5578d1445530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.371+0000 7f7d8ee5c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d1439830 0x5578d1445ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.371+0000 7f7d8ee5c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578d143a2d0 0x5578d14b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.371+0000 7f7d8ee5c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5578d1451620 con 0x5578d1439830 2024-04-21T08:38:19.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.371+0000 7f7d8ee5c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5578d1450f50 con 0x5578d1438e40 2024-04-21T08:38:19.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.371+0000 7f7d8ee5c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5578d1451cf0 con 0x5578d143a2d0 2024-04-21T08:38:19.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.371+0000 7f7d8ee5c740 10 client.0 _ll_register_callbacks cb 0x7f7d80005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:19.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.371+0000 7f7d8980b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d1439830 0x5578d1445ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.372+0000 7f7d8980b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d1439830 0x5578d1445ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55022/0 (socket says 172.21.15.43:55022) 2024-04-21T08:38:19.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.372+0000 7f7d8980b640 1 -- 172.21.15.43:0/526370178 learned_addr learned my addr 172.21.15.43:0/526370178 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.372+0000 7f7d8a80d640 1 --2- 172.21.15.43:0/526370178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578d143a2d0 0x5578d14b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.372+0000 7f7d8a00c640 1 --2- 172.21.15.43:0/526370178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578d1438e40 0x5578d1445530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.372+0000 7f7d8980b640 1 -- 172.21.15.43:0/526370178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578d143a2d0 msgr2=0x5578d14b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.372+0000 7f7d8980b640 1 --2- 172.21.15.43:0/526370178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578d143a2d0 0x5578d14b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.372+0000 7f7d8980b640 1 -- 172.21.15.43:0/526370178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578d1438e40 msgr2=0x5578d1445530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.372+0000 7f7d8980b640 1 --2- 172.21.15.43:0/526370178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578d1438e40 0x5578d1445530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.372+0000 7f7d8980b640 1 -- 172.21.15.43:0/526370178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578d14b4810 con 0x5578d1439830 2024-04-21T08:38:19.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.372+0000 7f7d8a00c640 1 --2- 172.21.15.43:0/526370178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578d1438e40 0x5578d1445530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:19.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.372+0000 7f7d8a80d640 1 --2- 172.21.15.43:0/526370178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578d143a2d0 0x5578d14b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:19.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.372+0000 7f7d8980b640 1 --2- 172.21.15.43:0/526370178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d1439830 0x5578d1445ad0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7d80005b80 tx=0x7f7d800117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3273d4f95b77f3ac server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.372+0000 7f7d6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:38:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.372+0000 7f7d6dffb640 1 -- 172.21.15.43:0/526370178 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80031070 con 0x5578d1439830 2024-04-21T08:38:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.372+0000 7f7d6dffb640 1 -- 172.21.15.43:0/526370178 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f7d800047e0 con 0x5578d1439830 2024-04-21T08:38:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.372+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/526370178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d800095b0 con 0x5578d1439830 2024-04-21T08:38:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.372+0000 7f7d6dffb640 1 -- 172.21.15.43:0/526370178 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7d8003d050 con 0x5578d1439830 2024-04-21T08:38:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.372+0000 7f7d8ee5c740 10 client.15240 fetch_fsmap learned FSMap version 27 2024-04-21T08:38:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.372+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/526370178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5578d14b5a20 con 0x5578d1439830 2024-04-21T08:38:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.372+0000 7f7d6dffb640 1 -- 172.21.15.43:0/526370178 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d800095b0 con 0x5578d1439830 2024-04-21T08:38:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.373+0000 7f7d6dffb640 1 -- 172.21.15.43:0/526370178 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7d80024c30 con 0x5578d1439830 2024-04-21T08:38:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.373+0000 7f7d8ee5c740 10 client.15240 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:38:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.373+0000 7f7d8ee5c740 10 client.15240 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-21T08:38:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.373+0000 7f7d8ee5c740 20 client.15240 populate_metadata read hostname 'smithi043' 2024-04-21T08:38:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.373+0000 7f7d8ee5c740 1 --2- 172.21.15.43:0/526370178 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5578d14b9f30 0x5578d14b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.373+0000 7f7d8ee5c740 4 client.15240 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:38:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.373+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/526370178 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "status"}) -- 0x5578d13a53a0 con 0x5578d14b9f30 2024-04-21T08:38:19.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.373+0000 7f7d8a00c640 1 --2- 172.21.15.43:0/526370178 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5578d14b9f30 0x5578d14b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.373+0000 7f7d8a00c640 1 --2- 172.21.15.43:0/526370178 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5578d14b9f30 0x5578d14b8c40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a8403a7a808050af server_cookie=ee22b54898d046dc in_seq=0 out_seq=0 2024-04-21T08:38:19.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.373+0000 7f7d6dffb640 10 client.15240 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:38:19.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d6dffb640 1 -- 172.21.15.43:0/526370178 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5578d13a53a0 con 0x5578d14b9f30 2024-04-21T08:38:19.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d6dffb640 10 client.15240 handle_command_reply: tid=0 2024-04-21T08:38:19.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d8ee5c740 1 client.15240 shutdown 2024-04-21T08:38:19.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/526370178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d1439830 msgr2=0x5578d1445ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d8ee5c740 1 --2- 172.21.15.43:0/526370178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d1439830 0x5578d1445ad0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7d80005b80 tx=0x7f7d800117b0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/526370178 shutdown_connections 2024-04-21T08:38:19.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d8ee5c740 1 --2- 172.21.15.43:0/526370178 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5578d14b9f30 0x5578d14b8c40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d8ee5c740 1 --2- 172.21.15.43:0/526370178 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578d143a2d0 0x5578d14b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d8ee5c740 1 --2- 172.21.15.43:0/526370178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578d1439830 0x5578d1445ad0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d6dffb640 0 client.15240 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:38:19.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d8ee5c740 1 --2- 172.21.15.43:0/526370178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578d1438e40 0x5578d1445530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/526370178 >> 172.21.15.43:0/526370178 conn(0x5578d1434f40 msgr2=0x5578d1442cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/526370178 shutdown_connections 2024-04-21T08:38:19.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d8ee5c740 1 -- 172.21.15.43:0/526370178 wait complete. 2024-04-21T08:38:19.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d8ee5c740 20 client.15240 trim_cache size 0 max 16384 2024-04-21T08:38:19.381 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:38:19.381 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "d57a7d98-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:38:19.381 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 4, 2024-04-21T08:38:19.381 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24412, 2024-04-21T08:38:19.381 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-21T08:38:19.381 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-21T08:38:19.381 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-21T08:38:19.381 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 88.481185273999998, 2024-04-21T08:38:19.382 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 27, 2024-04-21T08:38:19.382 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 94, 2024-04-21T08:38:19.382 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 90, 2024-04-21T08:38:19.382 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 152.87096642099999 2024-04-21T08:38:19.382 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:38:19.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d8c270640 1 -- 172.21.15.43:0/3515691085 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f7d700a3c30 msgr2=0x7f7d700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d8c270640 1 --2- 172.21.15.43:0/3515691085 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f7d700a3c30 0x7f7d700a60f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c006010 tx=0x7f7d7c032340 comp rx=0 tx=0).stop 2024-04-21T08:38:19.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d8c270640 1 -- 172.21.15.43:0/3515691085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d84100e60 msgr2=0x7f7d8410b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d8c270640 1 --2- 172.21.15.43:0/3515691085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d84100e60 0x7f7d8410b860 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400d950 tx=0x7f7d7400de10 comp rx=0 tx=0).stop 2024-04-21T08:38:19.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d8c270640 1 -- 172.21.15.43:0/3515691085 shutdown_connections 2024-04-21T08:38:19.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d8c270640 1 --2- 172.21.15.43:0/3515691085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d84102a00 0x7f7d8410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d8c270640 1 --2- 172.21.15.43:0/3515691085 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f7d700a3c30 0x7f7d700a60f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d8c270640 1 --2- 172.21.15.43:0/3515691085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d84102040 0x7f7d84109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d8c270640 1 --2- 172.21.15.43:0/3515691085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d84100e60 0x7f7d8410b860 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.374+0000 7f7d8c270640 1 -- 172.21.15.43:0/3515691085 >> 172.21.15.43:0/3515691085 conn(0x7f7d840fc440 msgr2=0x7f7d841053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.375+0000 7f7d8c270640 1 -- 172.21.15.43:0/3515691085 shutdown_connections 2024-04-21T08:38:19.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.375+0000 7f7d8c270640 1 -- 172.21.15.43:0/3515691085 wait complete. 2024-04-21T08:38:19.385 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-21T08:38:19.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.488+0000 7f578914b640 1 -- 172.21.15.43:0/2940901921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5784107d50 msgr2=0x7f578410a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.488+0000 7f578914b640 1 --2- 172.21.15.43:0/2940901921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5784107d50 0x7f578410a140 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f577c009900 tx=0x7f577c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:19.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.488+0000 7f578914b640 1 -- 172.21.15.43:0/2940901921 shutdown_connections 2024-04-21T08:38:19.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.488+0000 7f578914b640 1 --2- 172.21.15.43:0/2940901921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5784107d50 0x7f578410a140 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.488+0000 7f578914b640 1 --2- 172.21.15.43:0/2940901921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5784105420 0x7f5784107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.488+0000 7f578914b640 1 --2- 172.21.15.43:0/2940901921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5784102af0 0x7f5784104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.488+0000 7f578914b640 1 -- 172.21.15.43:0/2940901921 >> 172.21.15.43:0/2940901921 conn(0x7f57840fc440 msgr2=0x7f57840fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.488+0000 7f578914b640 1 -- 172.21.15.43:0/2940901921 shutdown_connections 2024-04-21T08:38:19.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.488+0000 7f578914b640 1 -- 172.21.15.43:0/2940901921 wait complete. 2024-04-21T08:38:19.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.489+0000 7f578914b640 1 Processor -- start 2024-04-21T08:38:19.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.489+0000 7f578914b640 1 -- start start 2024-04-21T08:38:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.489+0000 7f578914b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5784102af0 0x7f5784198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.489+0000 7f578914b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5784105420 0x7f57841987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.489+0000 7f578914b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5784107d50 0x7f578419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.489+0000 7f578914b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f578410dbb0 con 0x7f5784107d50 2024-04-21T08:38:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.489+0000 7f578914b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f578410da30 con 0x7f5784102af0 2024-04-21T08:38:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.489+0000 7f578914b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f578410dd30 con 0x7f5784105420 2024-04-21T08:38:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.490+0000 7f578359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5784107d50 0x7f578419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.490+0000 7f578359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5784107d50 0x7f578419f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55034/0 (socket says 172.21.15.43:55034) 2024-04-21T08:38:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.490+0000 7f578359e640 1 -- 172.21.15.43:0/11046525 learned_addr learned my addr 172.21.15.43:0/11046525 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.490+0000 7f5782d9d640 1 --2- 172.21.15.43:0/11046525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5784102af0 0x7f5784198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.490+0000 7f578359e640 1 -- 172.21.15.43:0/11046525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5784105420 msgr2=0x7f57841987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.490+0000 7f578259c640 1 --2- 172.21.15.43:0/11046525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5784105420 0x7f57841987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.490+0000 7f578359e640 1 --2- 172.21.15.43:0/11046525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5784105420 0x7f57841987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.490+0000 7f578359e640 1 -- 172.21.15.43:0/11046525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5784102af0 msgr2=0x7f5784198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.490+0000 7f578359e640 1 --2- 172.21.15.43:0/11046525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5784102af0 0x7f5784198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.490+0000 7f578359e640 1 -- 172.21.15.43:0/11046525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578419fe80 con 0x7f5784107d50 2024-04-21T08:38:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.490+0000 7f5782d9d640 1 --2- 172.21.15.43:0/11046525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5784102af0 0x7f5784198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.490+0000 7f578259c640 1 --2- 172.21.15.43:0/11046525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5784105420 0x7f57841987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.490+0000 7f578359e640 1 --2- 172.21.15.43:0/11046525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5784107d50 0x7f578419f810 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f577c036040 tx=0x7f577c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.490+0000 7f576bfff640 1 -- 172.21.15.43:0/11046525 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c043020 con 0x7f5784107d50 2024-04-21T08:38:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.490+0000 7f578914b640 1 -- 172.21.15.43:0/11046525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57841a0110 con 0x7f5784107d50 2024-04-21T08:38:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.490+0000 7f576bfff640 1 -- 172.21.15.43:0/11046525 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f577c004490 con 0x7f5784107d50 2024-04-21T08:38:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.490+0000 7f576bfff640 1 -- 172.21.15.43:0/11046525 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c003d10 con 0x7f5784107d50 2024-04-21T08:38:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.490+0000 7f578914b640 1 -- 172.21.15.43:0/11046525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57841a0730 con 0x7f5784107d50 2024-04-21T08:38:19.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.492+0000 7f576bfff640 1 -- 172.21.15.43:0/11046525 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f577c046020 con 0x7f5784107d50 2024-04-21T08:38:19.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.493+0000 7f578be14740 1 Processor -- start 2024-04-21T08:38:19.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.493+0000 7f578be14740 1 -- start start 2024-04-21T08:38:19.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.493+0000 7f578be14740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557288f8ee40 0x557288f8f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.493+0000 7f578be14740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f8f830 0x557288f8fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.493+0000 7f578be14740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557288f902d0 0x557288f96eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.493+0000 7f578be14740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557288ef3b10 con 0x557288f8f830 2024-04-21T08:38:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.493+0000 7f578be14740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557288e91240 con 0x557288f8ee40 2024-04-21T08:38:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.493+0000 7f578be14740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557288eefad0 con 0x557288f902d0 2024-04-21T08:38:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.493+0000 7f576bfff640 1 --2- 172.21.15.43:0/11046525 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f576c0a3bb0 0x7f576c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.493+0000 7f5782d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f8f830 0x557288f8fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.493+0000 7f578259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557288f8ee40 0x557288f8f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.493+0000 7f5782d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f8f830 0x557288f8fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55050/0 (socket says 172.21.15.43:55050) 2024-04-21T08:38:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.493+0000 7f5782d9d640 1 -- 172.21.15.43:0/4155895018 learned_addr learned my addr 172.21.15.43:0/4155895018 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.493+0000 7f578359e640 1 --2- 172.21.15.43:0/11046525 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f576c0a3bb0 0x7f576c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.493+0000 7f578259c640 1 --2- 172.21.15.43:0/4155895018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557288f902d0 0x557288f96eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.493+0000 7f576bfff640 1 -- 172.21.15.43:0/11046525 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f577c0f2c80 con 0x7f5784107d50 2024-04-21T08:38:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.493+0000 7f5782d9d640 1 -- 172.21.15.43:0/4155895018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557288f902d0 msgr2=0x557288f96eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.493+0000 7f5782d9d640 1 --2- 172.21.15.43:0/4155895018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557288f902d0 0x557288f96eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.493+0000 7f5782d9d640 1 -- 172.21.15.43:0/4155895018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557288f8ee40 msgr2=0x557288f8f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.493+0000 7f5782d9d640 1 --2- 172.21.15.43:0/4155895018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557288f8ee40 0x557288f8f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.493+0000 7f5782d9d640 1 -- 172.21.15.43:0/4155895018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557288c062d0 con 0x557288f8f830 2024-04-21T08:38:19.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.494+0000 7f5782d9d640 1 --2- 172.21.15.43:0/4155895018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f8f830 0x557288f8fcb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5774007fb0 tx=0x7f577400c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255391f38dd87479 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.494+0000 7f578359e640 1 --2- 172.21.15.43:0/11046525 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f576c0a3bb0 0x7f576c0a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f577c0c72b0 tx=0x7f577c0916b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.494+0000 7f57697fa640 1 -- 172.21.15.43:0/4155895018 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774014070 con 0x557288f8f830 2024-04-21T08:38:19.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.494+0000 7f57697fa640 1 -- 172.21.15.43:0/4155895018 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f577400fd00 con 0x557288f8f830 2024-04-21T08:38:19.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.494+0000 7f57697fa640 1 -- 172.21.15.43:0/4155895018 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577400ec10 con 0x557288f8f830 2024-04-21T08:38:19.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.494+0000 7f578be14740 1 -- 172.21.15.43:0/4155895018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f8f830 msgr2=0x557288f8fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.494+0000 7f578be14740 1 --2- 172.21.15.43:0/4155895018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f8f830 0x557288f8fcb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5774007fb0 tx=0x7f577400c750 comp rx=0 tx=0).stop 2024-04-21T08:38:19.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.495+0000 7f578be14740 1 -- 172.21.15.43:0/4155895018 shutdown_connections 2024-04-21T08:38:19.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.495+0000 7f578be14740 1 --2- 172.21.15.43:0/4155895018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557288f902d0 0x557288f96eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.495+0000 7f578be14740 1 --2- 172.21.15.43:0/4155895018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f8f830 0x557288f8fcb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.495+0000 7f578be14740 1 --2- 172.21.15.43:0/4155895018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557288f8ee40 0x557288f8f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.495+0000 7f578be14740 1 -- 172.21.15.43:0/4155895018 >> 172.21.15.43:0/4155895018 conn(0x557288f8af40 msgr2=0x557288f89490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.495+0000 7f578be14740 1 -- 172.21.15.43:0/4155895018 shutdown_connections 2024-04-21T08:38:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.495+0000 7f578be14740 1 -- 172.21.15.43:0/4155895018 wait complete. 2024-04-21T08:38:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.495+0000 7f578be14740 1 -- start start 2024-04-21T08:38:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.495+0000 7f578be14740 1 Processor -- start 2024-04-21T08:38:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.495+0000 7f578be14740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557288f8ee40 0x557288f98e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.495+0000 7f578be14740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557288f8f830 0x557288f99370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.495+0000 7f578be14740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f902d0 0x557288fa03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.495+0000 7f578be14740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557288f974b0 con 0x557288f902d0 2024-04-21T08:38:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.495+0000 7f578be14740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557288e7c190 con 0x557288f8ee40 2024-04-21T08:38:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.495+0000 7f578be14740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557288f97630 con 0x557288f8f830 2024-04-21T08:38:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.495+0000 7f578be14740 10 client.0 _ll_register_callbacks cb 0x557288e778b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.495+0000 7f578259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f902d0 0x557288fa03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.495+0000 7f578259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f902d0 0x557288fa03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55060/0 (socket says 172.21.15.43:55060) 2024-04-21T08:38:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.495+0000 7f578259c640 1 -- 172.21.15.43:0/1611780075 learned_addr learned my addr 172.21.15.43:0/1611780075 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.495+0000 7f5782d9d640 1 --2- 172.21.15.43:0/1611780075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557288f8f830 0x557288f99370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.495+0000 7f578259c640 1 --2- 172.21.15.43:0/1611780075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557288f8ee40 0x557288f98e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.496+0000 7f578259c640 1 -- 172.21.15.43:0/1611780075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557288f8f830 msgr2=0x557288f99370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.496+0000 7f578259c640 1 --2- 172.21.15.43:0/1611780075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557288f8f830 0x557288f99370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.496+0000 7f578259c640 1 -- 172.21.15.43:0/1611780075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557288f8ee40 msgr2=0x557288f98e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.496+0000 7f578259c640 1 --2- 172.21.15.43:0/1611780075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557288f8ee40 0x557288f98e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.496+0000 7f578259c640 1 -- 172.21.15.43:0/1611780075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557288cd06b0 con 0x557288f902d0 2024-04-21T08:38:19.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.496+0000 7f5782d9d640 1 --2- 172.21.15.43:0/1611780075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557288f8f830 0x557288f99370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:19.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.496+0000 7f578259c640 1 --2- 172.21.15.43:0/1611780075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f902d0 0x557288fa03b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f578406aff0 tx=0x7f577800b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce30d2e89640e01 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.496+0000 7f5766ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:38:19.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.496+0000 7f5766ff5640 1 -- 172.21.15.43:0/1611780075 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5778010ae0 con 0x557288f902d0 2024-04-21T08:38:19.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.496+0000 7f5766ff5640 1 -- 172.21.15.43:0/1611780075 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f5778010c80 con 0x557288f902d0 2024-04-21T08:38:19.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.496+0000 7f578be14740 1 -- 172.21.15.43:0/1611780075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f577400fd00 con 0x557288f902d0 2024-04-21T08:38:19.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.496+0000 7f5766ff5640 1 -- 172.21.15.43:0/1611780075 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5778004020 con 0x557288f902d0 2024-04-21T08:38:19.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.496+0000 7f578be14740 10 client.15252 fetch_fsmap learned FSMap version 27 2024-04-21T08:38:19.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.496+0000 7f578be14740 1 -- 172.21.15.43:0/1611780075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557288fa37f0 con 0x557288f902d0 2024-04-21T08:38:19.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.496+0000 7f5766ff5640 1 -- 172.21.15.43:0/1611780075 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f577400fd00 con 0x557288f902d0 2024-04-21T08:38:19.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.496+0000 7f5766ff5640 1 -- 172.21.15.43:0/1611780075 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f577800f040 con 0x557288f902d0 2024-04-21T08:38:19.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.497+0000 7f578be14740 10 client.15252 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:38:19.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.497+0000 7f578be14740 10 client.15252 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-21T08:38:19.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.497+0000 7f578be14740 20 client.15252 populate_metadata read hostname 'smithi043' 2024-04-21T08:38:19.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.497+0000 7f578be14740 1 --2- 172.21.15.43:0/1611780075 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x557288fa5e80 0x557288fa72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.497+0000 7f578be14740 4 client.15252 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:38:19.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.497+0000 7f578be14740 1 -- 172.21.15.43:0/1611780075 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557288f01530 con 0x557288fa5e80 2024-04-21T08:38:19.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.497+0000 7f5782d9d640 1 --2- 172.21.15.43:0/1611780075 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x557288fa5e80 0x557288fa72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.497+0000 7f5766ff5640 10 client.15252 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:38:19.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.497+0000 7f5782d9d640 1 --2- 172.21.15.43:0/1611780075 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x557288fa5e80 0x557288fa72f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=abe2bb95358a3592 server_cookie=1a58f2c5a1f1dbea in_seq=0 out_seq=0 2024-04-21T08:38:19.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.498+0000 7f5766ff5640 1 -- 172.21.15.43:0/1611780075 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557288f01530 con 0x557288fa5e80 2024-04-21T08:38:19.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.498+0000 7f5766ff5640 10 client.15252 handle_command_reply: tid=0 2024-04-21T08:38:19.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.498+0000 7f578be14740 1 client.15252 shutdown 2024-04-21T08:38:19.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.498+0000 7f578be14740 1 -- 172.21.15.43:0/1611780075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f902d0 msgr2=0x557288fa03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.498+0000 7f578be14740 1 --2- 172.21.15.43:0/1611780075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f902d0 0x557288fa03b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f578406aff0 tx=0x7f577800b3a0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.498+0000 7f578be14740 1 -- 172.21.15.43:0/1611780075 shutdown_connections 2024-04-21T08:38:19.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.498+0000 7f578be14740 1 --2- 172.21.15.43:0/1611780075 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x557288fa5e80 0x557288fa72f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.498+0000 7f578be14740 1 --2- 172.21.15.43:0/1611780075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f902d0 0x557288fa03b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.498+0000 7f578be14740 1 --2- 172.21.15.43:0/1611780075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557288f8f830 0x557288f99370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.498+0000 7f578be14740 1 --2- 172.21.15.43:0/1611780075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557288f8ee40 0x557288f98e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.498+0000 7f5766ff5640 0 client.15252 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:38:19.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.498+0000 7f578be14740 1 -- 172.21.15.43:0/1611780075 >> 172.21.15.43:0/1611780075 conn(0x557288f8af40 msgr2=0x557288f88a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.498+0000 7f578be14740 1 -- 172.21.15.43:0/1611780075 shutdown_connections 2024-04-21T08:38:19.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.498+0000 7f578be14740 1 -- 172.21.15.43:0/1611780075 wait complete. 2024-04-21T08:38:19.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.498+0000 7f578be14740 20 client.15252 trim_cache size 0 max 16384 2024-04-21T08:38:19.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.517+0000 7f578be14740 1 Processor -- start 2024-04-21T08:38:19.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.517+0000 7f578be14740 1 -- start start 2024-04-21T08:38:19.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.517+0000 7f578be14740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f8ee40 0x557288f92e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.517+0000 7f578be14740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557288f8f830 0x557288f84800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.517+0000 7f578be14740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557288f902d0 0x557288f96da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.517+0000 7f578be14740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557288fa5630 con 0x557288f8ee40 2024-04-21T08:38:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.517+0000 7f578be14740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557288fa5d00 con 0x557288f902d0 2024-04-21T08:38:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.517+0000 7f578be14740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557288fa63d0 con 0x557288f8f830 2024-04-21T08:38:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.517+0000 7f578259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f8ee40 0x557288f92e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.517+0000 7f578259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f8ee40 0x557288f92e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55074/0 (socket says 172.21.15.43:55074) 2024-04-21T08:38:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.517+0000 7f578259c640 1 -- 172.21.15.43:0/691181793 learned_addr learned my addr 172.21.15.43:0/691181793 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.517+0000 7f578259c640 1 --2- 172.21.15.43:0/691181793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557288f902d0 0x557288f96da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.517+0000 7f5782d9d640 1 --2- 172.21.15.43:0/691181793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557288f8f830 0x557288f84800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.517+0000 7f578259c640 1 -- 172.21.15.43:0/691181793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557288f8f830 msgr2=0x557288f84800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.517+0000 7f578259c640 1 --2- 172.21.15.43:0/691181793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557288f8f830 0x557288f84800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.517+0000 7f578259c640 1 -- 172.21.15.43:0/691181793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557288f902d0 msgr2=0x557288f96da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.517+0000 7f578259c640 1 --2- 172.21.15.43:0/691181793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557288f902d0 0x557288f96da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.517+0000 7f578259c640 1 -- 172.21.15.43:0/691181793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557288f96a20 con 0x557288f8ee40 2024-04-21T08:38:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.517+0000 7f5782d9d640 1 --2- 172.21.15.43:0/691181793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557288f8f830 0x557288f84800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.518+0000 7f578259c640 1 --2- 172.21.15.43:0/691181793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f8ee40 0x557288f92e90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5778005910 tx=0x7f5778005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c32aa4b73cc37ad server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.518+0000 7f5768ff9640 1 -- 172.21.15.43:0/691181793 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57780041c0 con 0x557288f8ee40 2024-04-21T08:38:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.518+0000 7f5768ff9640 1 -- 172.21.15.43:0/691181793 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5778004360 con 0x557288f8ee40 2024-04-21T08:38:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.518+0000 7f5768ff9640 1 -- 172.21.15.43:0/691181793 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5778003bf0 con 0x557288f8ee40 2024-04-21T08:38:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.518+0000 7f578be14740 1 -- 172.21.15.43:0/691181793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f8ee40 msgr2=0x557288f92e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.518+0000 7f578be14740 1 --2- 172.21.15.43:0/691181793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f8ee40 0x557288f92e90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5778005910 tx=0x7f5778005e50 comp rx=0 tx=0).stop 2024-04-21T08:38:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.518+0000 7f578be14740 1 -- 172.21.15.43:0/691181793 shutdown_connections 2024-04-21T08:38:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.518+0000 7f578be14740 1 --2- 172.21.15.43:0/691181793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557288f902d0 0x557288f96da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.518+0000 7f578be14740 1 --2- 172.21.15.43:0/691181793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557288f8f830 0x557288f84800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.518+0000 7f578be14740 1 --2- 172.21.15.43:0/691181793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f8ee40 0x557288f92e90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.518+0000 7f578be14740 1 -- 172.21.15.43:0/691181793 >> 172.21.15.43:0/691181793 conn(0x557288f8af40 msgr2=0x557288f89400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.518+0000 7f578be14740 1 -- 172.21.15.43:0/691181793 shutdown_connections 2024-04-21T08:38:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.518+0000 7f578be14740 1 -- 172.21.15.43:0/691181793 wait complete. 2024-04-21T08:38:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.519+0000 7f578be14740 1 -- start start 2024-04-21T08:38:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.519+0000 7f578be14740 1 Processor -- start 2024-04-21T08:38:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.519+0000 7f578be14740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557288f8ee40 0x557288f96060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.519+0000 7f578be14740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557288f8f830 0x55728900be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.519+0000 7f578be14740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f902d0 0x55728900e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.519+0000 7f578be14740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557288fa7170 con 0x557288f902d0 2024-04-21T08:38:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.519+0000 7f578be14740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557288fa6aa0 con 0x557288f8ee40 2024-04-21T08:38:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.519+0000 7f578be14740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557288fa7840 con 0x557288f8f830 2024-04-21T08:38:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.519+0000 7f578be14740 10 client.0 _ll_register_callbacks cb 0x7f577800bd00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.519+0000 7f578259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f902d0 0x55728900e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.519+0000 7f578259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f902d0 0x55728900e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55090/0 (socket says 172.21.15.43:55090) 2024-04-21T08:38:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.519+0000 7f578259c640 1 -- 172.21.15.43:0/3178760529 learned_addr learned my addr 172.21.15.43:0/3178760529 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.519+0000 7f5782d9d640 1 --2- 172.21.15.43:0/3178760529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557288f8f830 0x55728900be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.519+0000 7f578259c640 1 --2- 172.21.15.43:0/3178760529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557288f8ee40 0x557288f96060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.519+0000 7f578259c640 1 -- 172.21.15.43:0/3178760529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557288f8f830 msgr2=0x55728900be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.519+0000 7f578259c640 1 --2- 172.21.15.43:0/3178760529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557288f8f830 0x55728900be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.519+0000 7f578259c640 1 -- 172.21.15.43:0/3178760529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557288f8ee40 msgr2=0x557288f96060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.519+0000 7f578259c640 1 --2- 172.21.15.43:0/3178760529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557288f8ee40 0x557288f96060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.519+0000 7f578259c640 1 -- 172.21.15.43:0/3178760529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55728900eb10 con 0x557288f902d0 2024-04-21T08:38:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.520+0000 7f578259c640 1 --2- 172.21.15.43:0/3178760529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f902d0 0x55728900e380 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5778003d20 tx=0x7f5778004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d28eaf97e124a3e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.520+0000 7f5766ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:38:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.520+0000 7f5766ff5640 1 -- 172.21.15.43:0/3178760529 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57780035f0 con 0x557288f902d0 2024-04-21T08:38:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.520+0000 7f5766ff5640 1 -- 172.21.15.43:0/3178760529 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f5778020470 con 0x557288f902d0 2024-04-21T08:38:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.520+0000 7f578be14740 1 -- 172.21.15.43:0/3178760529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5778004360 con 0x557288f902d0 2024-04-21T08:38:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.520+0000 7f5766ff5640 1 -- 172.21.15.43:0/3178760529 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5778020670 con 0x557288f902d0 2024-04-21T08:38:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.520+0000 7f578be14740 10 client.15258 fetch_fsmap learned FSMap version 27 2024-04-21T08:38:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.520+0000 7f578be14740 1 -- 172.21.15.43:0/3178760529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55728900fd80 con 0x557288f902d0 2024-04-21T08:38:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.520+0000 7f5766ff5640 1 -- 172.21.15.43:0/3178760529 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5778004360 con 0x557288f902d0 2024-04-21T08:38:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.520+0000 7f5766ff5640 1 -- 172.21.15.43:0/3178760529 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f577803b470 con 0x557288f902d0 2024-04-21T08:38:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.520+0000 7f578be14740 10 client.15258 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:38:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.520+0000 7f578be14740 10 client.15258 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-21T08:38:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.520+0000 7f578be14740 20 client.15258 populate_metadata read hostname 'smithi043' 2024-04-21T08:38:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.520+0000 7f578be14740 1 --2- 172.21.15.43:0/3178760529 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x557289014500 0x557289011010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.520+0000 7f578be14740 4 client.15258 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:38:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.520+0000 7f578be14740 1 -- 172.21.15.43:0/3178760529 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "status"}) -- 0x557288efb3a0 con 0x557289014500 2024-04-21T08:38:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.521+0000 7f5782d9d640 1 --2- 172.21.15.43:0/3178760529 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x557289014500 0x557289011010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.521+0000 7f5766ff5640 10 client.15258 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:38:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.521+0000 7f5782d9d640 1 --2- 172.21.15.43:0/3178760529 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x557289014500 0x557289011010 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=241ba6fc1671e12e server_cookie=68512d13b189b562 in_seq=0 out_seq=0 2024-04-21T08:38:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.521+0000 7f5766ff5640 1 -- 172.21.15.43:0/3178760529 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557288efb3a0 con 0x557289014500 2024-04-21T08:38:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.521+0000 7f5766ff5640 10 client.15258 handle_command_reply: tid=0 2024-04-21T08:38:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.521+0000 7f578be14740 1 client.15258 shutdown 2024-04-21T08:38:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.521+0000 7f578be14740 1 -- 172.21.15.43:0/3178760529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f902d0 msgr2=0x55728900e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.521+0000 7f578be14740 1 --2- 172.21.15.43:0/3178760529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f902d0 0x55728900e380 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5778003d20 tx=0x7f5778004510 comp rx=0 tx=0).stop 2024-04-21T08:38:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.522+0000 7f578be14740 1 -- 172.21.15.43:0/3178760529 shutdown_connections 2024-04-21T08:38:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.522+0000 7f578be14740 1 --2- 172.21.15.43:0/3178760529 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x557289014500 0x557289011010 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.522+0000 7f578be14740 1 --2- 172.21.15.43:0/3178760529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557288f902d0 0x55728900e380 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.522+0000 7f5766ff5640 0 client.15258 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:38:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.522+0000 7f578be14740 1 --2- 172.21.15.43:0/3178760529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557288f8f830 0x55728900be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.522+0000 7f578be14740 1 --2- 172.21.15.43:0/3178760529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557288f8ee40 0x557288f96060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.522+0000 7f578be14740 1 -- 172.21.15.43:0/3178760529 >> 172.21.15.43:0/3178760529 conn(0x557288f8af40 msgr2=0x557288f84800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.522+0000 7f578be14740 1 -- 172.21.15.43:0/3178760529 shutdown_connections 2024-04-21T08:38:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.522+0000 7f578be14740 1 -- 172.21.15.43:0/3178760529 wait complete. 2024-04-21T08:38:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.522+0000 7f578be14740 20 client.15258 trim_cache size 0 max 16384 2024-04-21T08:38:19.529 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:38:19.529 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "d57a7d98-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:38:19.529 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 3, 2024-04-21T08:38:19.529 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24425, 2024-04-21T08:38:19.530 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-21T08:38:19.530 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-21T08:38:19.530 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-21T08:38:19.530 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 89.641271095999997, 2024-04-21T08:38:19.530 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 27, 2024-04-21T08:38:19.530 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 94, 2024-04-21T08:38:19.530 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 90, 2024-04-21T08:38:19.530 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 149.805032657 2024-04-21T08:38:19.530 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:38:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.522+0000 7f578914b640 1 -- 172.21.15.43:0/11046525 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f576c0a3bb0 msgr2=0x7f576c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.522+0000 7f578914b640 1 --2- 172.21.15.43:0/11046525 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f576c0a3bb0 0x7f576c0a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f577c0c72b0 tx=0x7f577c0916b0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.522+0000 7f578914b640 1 -- 172.21.15.43:0/11046525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5784107d50 msgr2=0x7f578419f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.522+0000 7f578914b640 1 --2- 172.21.15.43:0/11046525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5784107d50 0x7f578419f810 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f577c036040 tx=0x7f577c004270 comp rx=0 tx=0).stop 2024-04-21T08:38:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.522+0000 7f578914b640 1 -- 172.21.15.43:0/11046525 shutdown_connections 2024-04-21T08:38:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.522+0000 7f578914b640 1 --2- 172.21.15.43:0/11046525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5784107d50 0x7f578419f810 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.522+0000 7f578914b640 1 --2- 172.21.15.43:0/11046525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5784105420 0x7f57841987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.522+0000 7f578914b640 1 --2- 172.21.15.43:0/11046525 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f576c0a3bb0 0x7f576c0a6070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.522+0000 7f578914b640 1 --2- 172.21.15.43:0/11046525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5784102af0 0x7f5784198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.522+0000 7f578914b640 1 -- 172.21.15.43:0/11046525 >> 172.21.15.43:0/11046525 conn(0x7f57840fc440 msgr2=0x7f57841034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.522+0000 7f578914b640 1 -- 172.21.15.43:0/11046525 shutdown_connections 2024-04-21T08:38:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.522+0000 7f578914b640 1 -- 172.21.15.43:0/11046525 wait complete. 2024-04-21T08:38:19.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:19 smithi043 ceph-mon[27978]: pgmap v356: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 12 op/s 2024-04-21T08:38:19.533 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:19 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3784611215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:19.533 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:19 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/760554590' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:19.533 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:19 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3766377596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:19.534 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-21T08:38:19.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.636+0000 7fd3ab225640 1 -- 172.21.15.43:0/1697584101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a4107c90 msgr2=0x7fd3a410a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.636+0000 7fd3ab225640 1 --2- 172.21.15.43:0/1697584101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a4107c90 0x7fd3a410a0e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009ca0 tx=0x7fd39c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.636+0000 7fd3ab225640 1 -- 172.21.15.43:0/1697584101 shutdown_connections 2024-04-21T08:38:19.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.636+0000 7fd3ab225640 1 --2- 172.21.15.43:0/1697584101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a4107c90 0x7fd3a410a0e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.636+0000 7fd3ab225640 1 --2- 172.21.15.43:0/1697584101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3a4105360 0x7fd3a4107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.636+0000 7fd3ab225640 1 --2- 172.21.15.43:0/1697584101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3a4102a30 0x7fd3a4104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.636+0000 7fd3ab225640 1 -- 172.21.15.43:0/1697584101 >> 172.21.15.43:0/1697584101 conn(0x7fd3a40fc440 msgr2=0x7fd3a40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.636+0000 7fd3ab225640 1 -- 172.21.15.43:0/1697584101 shutdown_connections 2024-04-21T08:38:19.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.637+0000 7fd3ab225640 1 -- 172.21.15.43:0/1697584101 wait complete. 2024-04-21T08:38:19.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.637+0000 7fd3ab225640 1 Processor -- start 2024-04-21T08:38:19.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.637+0000 7fd3ab225640 1 -- start start 2024-04-21T08:38:19.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.637+0000 7fd3ab225640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a4102a30 0x7fd3a41981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.637+0000 7fd3ab225640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3a4105360 0x7fd3a4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.637+0000 7fd3ab225640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3a4107c90 0x7fd3a419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.637+0000 7fd3ab225640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a410db50 con 0x7fd3a4102a30 2024-04-21T08:38:19.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.637+0000 7fd3ab225640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a410d9d0 con 0x7fd3a4107c90 2024-04-21T08:38:19.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.637+0000 7fd3ab225640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a410dcd0 con 0x7fd3a4105360 2024-04-21T08:38:19.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.637+0000 7fd3a8fc1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a4102a30 0x7fd3a41981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.637+0000 7fd3a8fc1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a4102a30 0x7fd3a41981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55114/0 (socket says 172.21.15.43:55114) 2024-04-21T08:38:19.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.637+0000 7fd3a8fc1640 1 -- 172.21.15.43:0/2791800844 learned_addr learned my addr 172.21.15.43:0/2791800844 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.637+0000 7fd3a3fff640 1 --2- 172.21.15.43:0/2791800844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3a4105360 0x7fd3a4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.638+0000 7fd3a97c2640 1 --2- 172.21.15.43:0/2791800844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3a4107c90 0x7fd3a419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.638+0000 7fd3a8fc1640 1 -- 172.21.15.43:0/2791800844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3a4105360 msgr2=0x7fd3a4198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.638+0000 7fd3a8fc1640 1 --2- 172.21.15.43:0/2791800844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3a4105360 0x7fd3a4198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.638+0000 7fd3a8fc1640 1 -- 172.21.15.43:0/2791800844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3a4107c90 msgr2=0x7fd3a419f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.638+0000 7fd3a8fc1640 1 --2- 172.21.15.43:0/2791800844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3a4107c90 0x7fd3a419f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.638+0000 7fd3a8fc1640 1 -- 172.21.15.43:0/2791800844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a419fe10 con 0x7fd3a4102a30 2024-04-21T08:38:19.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.638+0000 7fd3a97c2640 1 --2- 172.21.15.43:0/2791800844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3a4107c90 0x7fd3a419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:19.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.638+0000 7fd3a8fc1640 1 --2- 172.21.15.43:0/2791800844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a4102a30 0x7fd3a41981f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd39400b4a0 tx=0x7fd39400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.638+0000 7fd3a1ffb640 1 -- 172.21.15.43:0/2791800844 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394014070 con 0x7fd3a4102a30 2024-04-21T08:38:19.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.638+0000 7fd3a1ffb640 1 -- 172.21.15.43:0/2791800844 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd394010ac0 con 0x7fd3a4102a30 2024-04-21T08:38:19.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.638+0000 7fd3ab225640 1 -- 172.21.15.43:0/2791800844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a41a0100 con 0x7fd3a4102a30 2024-04-21T08:38:19.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.638+0000 7fd3a1ffb640 1 -- 172.21.15.43:0/2791800844 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394017c30 con 0x7fd3a4102a30 2024-04-21T08:38:19.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.638+0000 7fd3ab225640 1 -- 172.21.15.43:0/2791800844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a41a0600 con 0x7fd3a4102a30 2024-04-21T08:38:19.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.640+0000 7fd3a1ffb640 1 -- 172.21.15.43:0/2791800844 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd394017070 con 0x7fd3a4102a30 2024-04-21T08:38:19.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.641+0000 7fd3ade12740 1 Processor -- start 2024-04-21T08:38:19.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.641+0000 7fd3ade12740 1 -- start start 2024-04-21T08:38:19.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.641+0000 7fd3ade12740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7203ace40 0x55c7203ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.641+0000 7fd3ade12740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ad830 0x55c7203adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.641+0000 7fd3ade12740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7203ae2d0 0x55c7203b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.641+0000 7fd3a1ffb640 1 --2- 172.21.15.43:0/2791800844 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd38c0a3bb0 0x7fd38c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.641+0000 7fd3ade12740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c720311b10 con 0x55c7203ad830 2024-04-21T08:38:19.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.641+0000 7fd3ade12740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c7202af240 con 0x55c7203ae2d0 2024-04-21T08:38:19.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.641+0000 7fd3ade12740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c72030dad0 con 0x55c7203ace40 2024-04-21T08:38:19.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.641+0000 7fd3a8fc1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ad830 0x55c7203adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.641+0000 7fd3a8fc1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ad830 0x55c7203adcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55126/0 (socket says 172.21.15.43:55126) 2024-04-21T08:38:19.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.641+0000 7fd3a97c2640 1 --2- 172.21.15.43:0/2791800844 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd38c0a3bb0 0x7fd38c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.641+0000 7fd3a8fc1640 1 -- 172.21.15.43:0/3022183514 learned_addr learned my addr 172.21.15.43:0/3022183514 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.641+0000 7fd3a3fff640 1 --2- 172.21.15.43:0/3022183514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7203ae2d0 0x55c7203b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.641+0000 7fd3a1ffb640 1 -- 172.21.15.43:0/2791800844 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fd3940cfbf0 con 0x7fd3a4102a30 2024-04-21T08:38:19.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.641+0000 7fd3a97c2640 1 --2- 172.21.15.43:0/3022183514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7203ace40 0x55c7203ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.641+0000 7fd3a8fc1640 1 -- 172.21.15.43:0/3022183514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7203ace40 msgr2=0x55c7203ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.641+0000 7fd3a8fc1640 1 --2- 172.21.15.43:0/3022183514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7203ace40 0x55c7203ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.641+0000 7fd3a8fc1640 1 -- 172.21.15.43:0/3022183514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7203ae2d0 msgr2=0x55c7203b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.641+0000 7fd3a8fc1640 1 --2- 172.21.15.43:0/3022183514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7203ae2d0 0x55c7203b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.641+0000 7fd3a8fc1640 1 -- 172.21.15.43:0/3022183514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7200242d0 con 0x55c7203ad830 2024-04-21T08:38:19.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.642+0000 7fd3a97c2640 1 --2- 172.21.15.43:0/2791800844 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd38c0a3bb0 0x7fd38c0a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009a60 tx=0x7fd39c032ca0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.642+0000 7fd3a8fc1640 1 --2- 172.21.15.43:0/3022183514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ad830 0x55c7203adcb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd394097ee0 tx=0x7fd39406d510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a13c4be67662008 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.642+0000 7fd38affd640 1 -- 172.21.15.43:0/3022183514 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39401e040 con 0x55c7203ad830 2024-04-21T08:38:19.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.642+0000 7fd38affd640 1 -- 172.21.15.43:0/3022183514 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd39401e1e0 con 0x55c7203ad830 2024-04-21T08:38:19.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.642+0000 7fd38affd640 1 -- 172.21.15.43:0/3022183514 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3940a46f0 con 0x55c7203ad830 2024-04-21T08:38:19.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.642+0000 7fd3ade12740 1 -- 172.21.15.43:0/3022183514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ad830 msgr2=0x55c7203adcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.642+0000 7fd3ade12740 1 --2- 172.21.15.43:0/3022183514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ad830 0x55c7203adcb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd394097ee0 tx=0x7fd39406d510 comp rx=0 tx=0).stop 2024-04-21T08:38:19.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.643+0000 7fd3ade12740 1 -- 172.21.15.43:0/3022183514 shutdown_connections 2024-04-21T08:38:19.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.643+0000 7fd3ade12740 1 --2- 172.21.15.43:0/3022183514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7203ae2d0 0x55c7203b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.643+0000 7fd3ade12740 1 --2- 172.21.15.43:0/3022183514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ad830 0x55c7203adcb0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.643+0000 7fd3ade12740 1 --2- 172.21.15.43:0/3022183514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7203ace40 0x55c7203ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.643+0000 7fd3ade12740 1 -- 172.21.15.43:0/3022183514 >> 172.21.15.43:0/3022183514 conn(0x55c7203a8f40 msgr2=0x55c7203a7490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.643+0000 7fd3ade12740 1 -- 172.21.15.43:0/3022183514 shutdown_connections 2024-04-21T08:38:19.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.643+0000 7fd3ade12740 1 -- 172.21.15.43:0/3022183514 wait complete. 2024-04-21T08:38:19.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.643+0000 7fd3ade12740 1 -- start start 2024-04-21T08:38:19.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.643+0000 7fd3ade12740 1 Processor -- start 2024-04-21T08:38:19.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.643+0000 7fd3ade12740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ace40 0x55c7203b6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.643+0000 7fd3ade12740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7203ad830 0x55c7203b73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.643+0000 7fd3ade12740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7203ae2d0 0x55c7203be3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.643+0000 7fd3ade12740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c7203b5450 con 0x55c7203ace40 2024-04-21T08:38:19.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.643+0000 7fd3ade12740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c72029a190 con 0x55c7203ad830 2024-04-21T08:38:19.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.643+0000 7fd3ade12740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c7203b55d0 con 0x55c7203ae2d0 2024-04-21T08:38:19.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.643+0000 7fd3ade12740 10 client.0 _ll_register_callbacks cb 0x55c7202958b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:19.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.643+0000 7fd3a8fc1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ace40 0x55c7203b6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.643+0000 7fd3a8fc1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ace40 0x55c7203b6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55128/0 (socket says 172.21.15.43:55128) 2024-04-21T08:38:19.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.644+0000 7fd3a8fc1640 1 -- 172.21.15.43:0/3878498113 learned_addr learned my addr 172.21.15.43:0/3878498113 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.644+0000 7fd3a97c2640 1 --2- 172.21.15.43:0/3878498113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7203ae2d0 0x55c7203be3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.644+0000 7fd3a3fff640 1 --2- 172.21.15.43:0/3878498113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7203ad830 0x55c7203b73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.644+0000 7fd3a8fc1640 1 -- 172.21.15.43:0/3878498113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7203ae2d0 msgr2=0x55c7203be3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.644+0000 7fd3a8fc1640 1 --2- 172.21.15.43:0/3878498113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7203ae2d0 0x55c7203be3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.644+0000 7fd3a8fc1640 1 -- 172.21.15.43:0/3878498113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7203ad830 msgr2=0x55c7203b73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.644+0000 7fd3a8fc1640 1 --2- 172.21.15.43:0/3878498113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7203ad830 0x55c7203b73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.644+0000 7fd3a8fc1640 1 -- 172.21.15.43:0/3878498113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7200ee6b0 con 0x55c7203ace40 2024-04-21T08:38:19.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.644+0000 7fd3a97c2640 1 --2- 172.21.15.43:0/3878498113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7203ae2d0 0x55c7203be3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:19.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.644+0000 7fd3a3fff640 1 --2- 172.21.15.43:0/3878498113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7203ad830 0x55c7203b73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:19.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.644+0000 7fd3a8fc1640 1 --2- 172.21.15.43:0/3878498113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ace40 0x55c7203b6e60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd3940cc340 tx=0x7fd39400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483eed6595ace08b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.644+0000 7fd383fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:38:19.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.644+0000 7fd383fff640 1 -- 172.21.15.43:0/3878498113 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39401e040 con 0x55c7203ace40 2024-04-21T08:38:19.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.644+0000 7fd383fff640 1 -- 172.21.15.43:0/3878498113 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fd394010c70 con 0x55c7203ace40 2024-04-21T08:38:19.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.644+0000 7fd3ade12740 1 -- 172.21.15.43:0/3878498113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c72031f530 con 0x55c7203ace40 2024-04-21T08:38:19.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.644+0000 7fd383fff640 1 -- 172.21.15.43:0/3878498113 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd39409c180 con 0x55c7203ace40 2024-04-21T08:38:19.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.644+0000 7fd3ade12740 10 client.15270 fetch_fsmap learned FSMap version 27 2024-04-21T08:38:19.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.644+0000 7fd3ade12740 1 -- 172.21.15.43:0/3878498113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c7203c1750 con 0x55c7203ace40 2024-04-21T08:38:19.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.645+0000 7fd383fff640 1 -- 172.21.15.43:0/3878498113 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c72031f530 con 0x55c7203ace40 2024-04-21T08:38:19.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.645+0000 7fd383fff640 1 -- 172.21.15.43:0/3878498113 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd3940a4260 con 0x55c7203ace40 2024-04-21T08:38:19.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.645+0000 7fd3ade12740 10 client.15270 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:38:19.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.645+0000 7fd3ade12740 10 client.15270 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T08:38:19.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.645+0000 7fd3ade12740 20 client.15270 populate_metadata read hostname 'smithi043' 2024-04-21T08:38:19.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.645+0000 7fd3ade12740 1 --2- 172.21.15.43:0/3878498113 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55c7203c38c0 0x55c7203c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.645+0000 7fd3ade12740 4 client.15270 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:38:19.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.645+0000 7fd3ade12740 1 -- 172.21.15.43:0/3878498113 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7203193a0 con 0x55c7203c38c0 2024-04-21T08:38:19.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.645+0000 7fd3a3fff640 1 --2- 172.21.15.43:0/3878498113 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55c7203c38c0 0x55c7203c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.645+0000 7fd3a3fff640 1 --2- 172.21.15.43:0/3878498113 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55c7203c38c0 0x55c7203c4d30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf91236d4518ab8e server_cookie=8bb7241a1a6c185f in_seq=0 out_seq=0 2024-04-21T08:38:19.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.645+0000 7fd383fff640 10 client.15270 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:38:19.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.646+0000 7fd383fff640 1 -- 172.21.15.43:0/3878498113 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c7203193a0 con 0x55c7203c38c0 2024-04-21T08:38:19.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.646+0000 7fd383fff640 10 client.15270 handle_command_reply: tid=0 2024-04-21T08:38:19.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.646+0000 7fd3ade12740 1 client.15270 shutdown 2024-04-21T08:38:19.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.646+0000 7fd3ade12740 1 -- 172.21.15.43:0/3878498113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ace40 msgr2=0x55c7203b6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.646+0000 7fd3ade12740 1 --2- 172.21.15.43:0/3878498113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ace40 0x55c7203b6e60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd3940cc340 tx=0x7fd39400bf90 comp rx=0 tx=0).stop 2024-04-21T08:38:19.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.646+0000 7fd3ade12740 1 -- 172.21.15.43:0/3878498113 shutdown_connections 2024-04-21T08:38:19.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.646+0000 7fd3ade12740 1 --2- 172.21.15.43:0/3878498113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7203ae2d0 0x55c7203be3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.646+0000 7fd3ade12740 1 --2- 172.21.15.43:0/3878498113 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55c7203c38c0 0x55c7203c4d30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.646+0000 7fd3ade12740 1 --2- 172.21.15.43:0/3878498113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7203ad830 0x55c7203b73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.646+0000 7fd3ade12740 1 --2- 172.21.15.43:0/3878498113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ace40 0x55c7203b6e60 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.646+0000 7fd383fff640 0 client.15270 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:38:19.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.646+0000 7fd3ade12740 1 -- 172.21.15.43:0/3878498113 >> 172.21.15.43:0/3878498113 conn(0x55c7203a8f40 msgr2=0x55c7203a6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.646+0000 7fd3ade12740 1 -- 172.21.15.43:0/3878498113 shutdown_connections 2024-04-21T08:38:19.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.647+0000 7fd3ade12740 1 -- 172.21.15.43:0/3878498113 wait complete. 2024-04-21T08:38:19.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.647+0000 7fd3ade12740 20 client.15270 trim_cache size 0 max 16384 2024-04-21T08:38:19.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.665+0000 7fd3ade12740 1 Processor -- start 2024-04-21T08:38:19.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.665+0000 7fd3ade12740 1 -- start start 2024-04-21T08:38:19.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.665+0000 7fd3ade12740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ace40 0x55c7203a2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.665+0000 7fd3ade12740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7203ad830 0x55c7203a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.665+0000 7fd3ade12740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7203ae2d0 0x55c7203a7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.665+0000 7fd3ade12740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c7203c3070 con 0x55c7203ace40 2024-04-21T08:38:19.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.665+0000 7fd3ade12740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c7203c3740 con 0x55c7203ad830 2024-04-21T08:38:19.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.665+0000 7fd3ade12740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c7203c3e10 con 0x55c7203ae2d0 2024-04-21T08:38:19.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.665+0000 7fd3a8fc1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ace40 0x55c7203a2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.665+0000 7fd3a8fc1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ace40 0x55c7203a2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55132/0 (socket says 172.21.15.43:55132) 2024-04-21T08:38:19.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.665+0000 7fd3a8fc1640 1 -- 172.21.15.43:0/609395956 learned_addr learned my addr 172.21.15.43:0/609395956 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.665+0000 7fd3a97c2640 1 --2- 172.21.15.43:0/609395956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7203ae2d0 0x55c7203a7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.665+0000 7fd3a8fc1640 1 -- 172.21.15.43:0/609395956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7203ae2d0 msgr2=0x55c7203a7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.665+0000 7fd3a8fc1640 1 --2- 172.21.15.43:0/609395956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7203ae2d0 0x55c7203a7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.665+0000 7fd3a8fc1640 1 -- 172.21.15.43:0/609395956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7203ad830 msgr2=0x55c7203a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.665+0000 7fd3a8fc1640 1 --2- 172.21.15.43:0/609395956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7203ad830 0x55c7203a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.665+0000 7fd3a8fc1640 1 -- 172.21.15.43:0/609395956 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7203a8230 con 0x55c7203ace40 2024-04-21T08:38:19.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.665+0000 7fd3a97c2640 1 --2- 172.21.15.43:0/609395956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7203ae2d0 0x55c7203a7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:19.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.666+0000 7fd3a8fc1640 1 --2- 172.21.15.43:0/609395956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ace40 0x55c7203a2800 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd39409ba10 tx=0x7fd3940b5130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ce887cd38bd28a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.666+0000 7fd38a7fc640 1 -- 172.21.15.43:0/609395956 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39401e040 con 0x55c7203ace40 2024-04-21T08:38:19.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.666+0000 7fd38a7fc640 1 -- 172.21.15.43:0/609395956 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd39409ba80 con 0x55c7203ace40 2024-04-21T08:38:19.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.666+0000 7fd38a7fc640 1 -- 172.21.15.43:0/609395956 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3940092b0 con 0x55c7203ace40 2024-04-21T08:38:19.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.666+0000 7fd3ade12740 1 -- 172.21.15.43:0/609395956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ace40 msgr2=0x55c7203a2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.666+0000 7fd3ade12740 1 --2- 172.21.15.43:0/609395956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ace40 0x55c7203a2800 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd39409ba10 tx=0x7fd3940b5130 comp rx=0 tx=0).stop 2024-04-21T08:38:19.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.666+0000 7fd3ade12740 1 -- 172.21.15.43:0/609395956 shutdown_connections 2024-04-21T08:38:19.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.666+0000 7fd3ade12740 1 --2- 172.21.15.43:0/609395956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7203ae2d0 0x55c7203a7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.666+0000 7fd3ade12740 1 --2- 172.21.15.43:0/609395956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7203ad830 0x55c7203a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.666+0000 7fd3ade12740 1 --2- 172.21.15.43:0/609395956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ace40 0x55c7203a2800 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.666+0000 7fd3ade12740 1 -- 172.21.15.43:0/609395956 >> 172.21.15.43:0/609395956 conn(0x55c7203a8f40 msgr2=0x55c7203b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.666+0000 7fd3ade12740 1 -- 172.21.15.43:0/609395956 shutdown_connections 2024-04-21T08:38:19.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.666+0000 7fd3ade12740 1 -- 172.21.15.43:0/609395956 wait complete. 2024-04-21T08:38:19.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.667+0000 7fd3ade12740 1 -- start start 2024-04-21T08:38:19.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.667+0000 7fd3ade12740 1 Processor -- start 2024-04-21T08:38:19.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.667+0000 7fd3ade12740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7203ace40 0x55c7203b9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.667+0000 7fd3ade12740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7203ad830 0x55c7203b9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.667+0000 7fd3ade12740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ae2d0 0x55c72042bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.667+0000 7fd3ade12740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c7203c4bb0 con 0x55c7203ae2d0 2024-04-21T08:38:19.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.667+0000 7fd3ade12740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c7203c44e0 con 0x55c7203ad830 2024-04-21T08:38:19.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.667+0000 7fd3ade12740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c7203c5280 con 0x55c7203ace40 2024-04-21T08:38:19.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.667+0000 7fd3ade12740 10 client.0 _ll_register_callbacks cb 0x7fd39800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:19.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.667+0000 7fd3a97c2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ae2d0 0x55c72042bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.667+0000 7fd3a97c2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ae2d0 0x55c72042bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55140/0 (socket says 172.21.15.43:55140) 2024-04-21T08:38:19.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.667+0000 7fd3a97c2640 1 -- 172.21.15.43:0/3523151912 learned_addr learned my addr 172.21.15.43:0/3523151912 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.667+0000 7fd3a8fc1640 1 --2- 172.21.15.43:0/3523151912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7203ace40 0x55c7203b9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.667+0000 7fd3a3fff640 1 --2- 172.21.15.43:0/3523151912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7203ad830 0x55c7203b9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.668+0000 7fd3a97c2640 1 -- 172.21.15.43:0/3523151912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7203ace40 msgr2=0x55c7203b9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.668+0000 7fd3a97c2640 1 --2- 172.21.15.43:0/3523151912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7203ace40 0x55c7203b9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.668+0000 7fd3a97c2640 1 -- 172.21.15.43:0/3523151912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7203ad830 msgr2=0x55c7203b9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.668+0000 7fd3a97c2640 1 --2- 172.21.15.43:0/3523151912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7203ad830 0x55c7203b9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.668+0000 7fd3a97c2640 1 -- 172.21.15.43:0/3523151912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c72042c5a0 con 0x55c7203ae2d0 2024-04-21T08:38:19.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.668+0000 7fd3a97c2640 1 --2- 172.21.15.43:0/3523151912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ae2d0 0x55c72042bf90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd39c032ad0 tx=0x7fd39c032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228a7d365a1b7a56 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.668+0000 7fd383fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:38:19.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.668+0000 7fd383fff640 1 -- 172.21.15.43:0/3523151912 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c052070 con 0x55c7203ae2d0 2024-04-21T08:38:19.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.668+0000 7fd383fff640 1 -- 172.21.15.43:0/3523151912 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fd39c004250 con 0x55c7203ae2d0 2024-04-21T08:38:19.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.668+0000 7fd3ade12740 1 -- 172.21.15.43:0/3523151912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd39409ba80 con 0x55c7203ae2d0 2024-04-21T08:38:19.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.668+0000 7fd383fff640 1 -- 172.21.15.43:0/3523151912 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd39c05e050 con 0x55c7203ae2d0 2024-04-21T08:38:19.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.668+0000 7fd3ade12740 10 client.15276 fetch_fsmap learned FSMap version 27 2024-04-21T08:38:19.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.668+0000 7fd3ade12740 1 -- 172.21.15.43:0/3523151912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c72042d910 con 0x55c7203ae2d0 2024-04-21T08:38:19.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.668+0000 7fd383fff640 1 -- 172.21.15.43:0/3523151912 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd39409ba80 con 0x55c7203ae2d0 2024-04-21T08:38:19.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.668+0000 7fd383fff640 1 -- 172.21.15.43:0/3523151912 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd39c03f070 con 0x55c7203ae2d0 2024-04-21T08:38:19.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.669+0000 7fd3ade12740 10 client.15276 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:38:19.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.669+0000 7fd3ade12740 10 client.15276 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T08:38:19.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.669+0000 7fd3ade12740 20 client.15276 populate_metadata read hostname 'smithi043' 2024-04-21T08:38:19.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.669+0000 7fd3ade12740 1 --2- 172.21.15.43:0/3523151912 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55c720432480 0x55c72042ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.669+0000 7fd3ade12740 4 client.15276 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:38:19.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.669+0000 7fd3ade12740 1 -- 172.21.15.43:0/3523151912 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "status"}) -- 0x55c720295960 con 0x55c720432480 2024-04-21T08:38:19.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.669+0000 7fd3a8fc1640 1 --2- 172.21.15.43:0/3523151912 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55c720432480 0x55c72042ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.669+0000 7fd383fff640 10 client.15276 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:38:19.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.669+0000 7fd3a8fc1640 1 --2- 172.21.15.43:0/3523151912 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55c720432480 0x55c72042ef90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9695a77a9bf87eea server_cookie=8529c694deb3b88f in_seq=0 out_seq=0 2024-04-21T08:38:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.670+0000 7fd383fff640 1 -- 172.21.15.43:0/3523151912 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c720295960 con 0x55c720432480 2024-04-21T08:38:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.670+0000 7fd383fff640 10 client.15276 handle_command_reply: tid=0 2024-04-21T08:38:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.670+0000 7fd3ade12740 1 client.15276 shutdown 2024-04-21T08:38:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.670+0000 7fd3ade12740 1 -- 172.21.15.43:0/3523151912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ae2d0 msgr2=0x55c72042bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.670+0000 7fd3ade12740 1 --2- 172.21.15.43:0/3523151912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ae2d0 0x55c72042bf90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd39c032ad0 tx=0x7fd39c032340 comp rx=0 tx=0).stop 2024-04-21T08:38:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.670+0000 7fd3ade12740 1 -- 172.21.15.43:0/3523151912 shutdown_connections 2024-04-21T08:38:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.670+0000 7fd3ade12740 1 --2- 172.21.15.43:0/3523151912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7203ae2d0 0x55c72042bf90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.670+0000 7fd3ade12740 1 --2- 172.21.15.43:0/3523151912 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55c720432480 0x55c72042ef90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.670+0000 7fd3ade12740 1 --2- 172.21.15.43:0/3523151912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7203ad830 0x55c7203b9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.670+0000 7fd3ade12740 1 --2- 172.21.15.43:0/3523151912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7203ace40 0x55c7203b9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.670+0000 7fd383fff640 0 client.15276 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:38:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.670+0000 7fd3ade12740 1 -- 172.21.15.43:0/3523151912 >> 172.21.15.43:0/3523151912 conn(0x55c7203a8f40 msgr2=0x55c7203b4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.670+0000 7fd3ade12740 1 -- 172.21.15.43:0/3523151912 shutdown_connections 2024-04-21T08:38:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.670+0000 7fd3ade12740 1 -- 172.21.15.43:0/3523151912 wait complete. 2024-04-21T08:38:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.670+0000 7fd3ade12740 20 client.15276 trim_cache size 0 max 16384 2024-04-21T08:38:19.676 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:38:19.676 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "d57a7d98-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:38:19.676 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 2, 2024-04-21T08:38:19.676 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24487, 2024-04-21T08:38:19.676 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-21T08:38:19.676 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-21T08:38:19.676 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-21T08:38:19.676 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 90.610703509999993, 2024-04-21T08:38:19.676 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 27, 2024-04-21T08:38:19.676 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 94, 2024-04-21T08:38:19.676 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 90, 2024-04-21T08:38:19.677 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 129.39885256900001 2024-04-21T08:38:19.677 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:38:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.670+0000 7fd3ab225640 1 -- 172.21.15.43:0/2791800844 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd38c0a3bb0 msgr2=0x7fd38c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.670+0000 7fd3ab225640 1 --2- 172.21.15.43:0/2791800844 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd38c0a3bb0 0x7fd38c0a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009a60 tx=0x7fd39c032ca0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.670+0000 7fd3ab225640 1 -- 172.21.15.43:0/2791800844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a4102a30 msgr2=0x7fd3a41981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.670+0000 7fd3ab225640 1 --2- 172.21.15.43:0/2791800844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a4102a30 0x7fd3a41981f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd39400b4a0 tx=0x7fd39400b960 comp rx=0 tx=0).stop 2024-04-21T08:38:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.670+0000 7fd3ab225640 1 -- 172.21.15.43:0/2791800844 shutdown_connections 2024-04-21T08:38:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.670+0000 7fd3ab225640 1 --2- 172.21.15.43:0/2791800844 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd38c0a3bb0 0x7fd38c0a6070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.670+0000 7fd3ab225640 1 --2- 172.21.15.43:0/2791800844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3a4107c90 0x7fd3a419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.670+0000 7fd3ab225640 1 --2- 172.21.15.43:0/2791800844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3a4105360 0x7fd3a4198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.670+0000 7fd3ab225640 1 --2- 172.21.15.43:0/2791800844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a4102a30 0x7fd3a41981f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.671+0000 7fd3ab225640 1 -- 172.21.15.43:0/2791800844 >> 172.21.15.43:0/2791800844 conn(0x7fd3a40fc440 msgr2=0x7fd3a4103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.671+0000 7fd3ab225640 1 -- 172.21.15.43:0/2791800844 shutdown_connections 2024-04-21T08:38:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.671+0000 7fd3ab225640 1 -- 172.21.15.43:0/2791800844 wait complete. 2024-04-21T08:38:19.682 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-21T08:38:19.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:19 smithi175 ceph-mon[31920]: pgmap v356: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 12 op/s 2024-04-21T08:38:19.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:19 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3784611215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:19.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:19 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/760554590' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:19.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:19 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3766377596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:19.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.785+0000 7f21a2d98640 1 -- 172.21.15.43:0/973930550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f219c100ea0 msgr2=0x7f219c1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.785+0000 7f21a2d98640 1 --2- 172.21.15.43:0/973930550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f219c100ea0 0x7f219c1012c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2194009ca0 tx=0x7f219402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.785+0000 7f21a2d98640 1 -- 172.21.15.43:0/973930550 shutdown_connections 2024-04-21T08:38:19.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.785+0000 7f21a2d98640 1 --2- 172.21.15.43:0/973930550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f219c1029b0 0x7f219c1092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.785+0000 7f21a2d98640 1 --2- 172.21.15.43:0/973930550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f219c101ff0 0x7f219c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.785+0000 7f21a2d98640 1 --2- 172.21.15.43:0/973930550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f219c100ea0 0x7f219c1012c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.785+0000 7f21a2d98640 1 -- 172.21.15.43:0/973930550 >> 172.21.15.43:0/973930550 conn(0x7f219c0fc410 msgr2=0x7f219c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.785+0000 7f21a2d98640 1 -- 172.21.15.43:0/973930550 shutdown_connections 2024-04-21T08:38:19.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.785+0000 7f21a2d98640 1 -- 172.21.15.43:0/973930550 wait complete. 2024-04-21T08:38:19.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.785+0000 7f21a2d98640 1 Processor -- start 2024-04-21T08:38:19.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.785+0000 7f21a2d98640 1 -- start start 2024-04-21T08:38:19.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.785+0000 7f21a2d98640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f219c100ea0 0x7f219c198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.785+0000 7f21a2d98640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f219c101ff0 0x7f219c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.785+0000 7f21a2d98640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f219c1029b0 0x7f219c19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.785+0000 7f21a2d98640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f219c10c9d0 con 0x7f219c100ea0 2024-04-21T08:38:19.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.785+0000 7f21a2d98640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f219c10c850 con 0x7f219c1029b0 2024-04-21T08:38:19.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.785+0000 7f21a2d98640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f219c10cb50 con 0x7f219c101ff0 2024-04-21T08:38:19.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.786+0000 7f21a0b34640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f219c100ea0 0x7f219c198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.786+0000 7f21a0b34640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f219c100ea0 0x7f219c198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55154/0 (socket says 172.21.15.43:55154) 2024-04-21T08:38:19.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.786+0000 7f21a0b34640 1 -- 172.21.15.43:0/3734685943 learned_addr learned my addr 172.21.15.43:0/3734685943 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.786+0000 7f21a1335640 1 --2- 172.21.15.43:0/3734685943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f219c1029b0 0x7f219c19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.786+0000 7f21a0b34640 1 -- 172.21.15.43:0/3734685943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f219c101ff0 msgr2=0x7f219c198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.786+0000 7f21a0b34640 1 --2- 172.21.15.43:0/3734685943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f219c101ff0 0x7f219c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.786+0000 7f21a0b34640 1 -- 172.21.15.43:0/3734685943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f219c1029b0 msgr2=0x7f219c19f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.786+0000 7f21a0b34640 1 --2- 172.21.15.43:0/3734685943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f219c1029b0 0x7f219c19f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.786+0000 7f21a0b34640 1 -- 172.21.15.43:0/3734685943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219c19fe20 con 0x7f219c100ea0 2024-04-21T08:38:19.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.786+0000 7f21a1335640 1 --2- 172.21.15.43:0/3734685943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f219c1029b0 0x7f219c19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:19.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.786+0000 7f21a0b34640 1 --2- 172.21.15.43:0/3734685943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f219c100ea0 0x7f219c198200 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2194005cc0 tx=0x7f2194034830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.786+0000 7f2199ffb640 1 -- 172.21.15.43:0/3734685943 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194039040 con 0x7f219c100ea0 2024-04-21T08:38:19.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.786+0000 7f21a2d98640 1 -- 172.21.15.43:0/3734685943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f219c1a00b0 con 0x7f219c100ea0 2024-04-21T08:38:19.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.786+0000 7f2199ffb640 1 -- 172.21.15.43:0/3734685943 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2194044070 con 0x7f219c100ea0 2024-04-21T08:38:19.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.786+0000 7f2199ffb640 1 -- 172.21.15.43:0/3734685943 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21940042a0 con 0x7f219c100ea0 2024-04-21T08:38:19.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.786+0000 7f21a2d98640 1 -- 172.21.15.43:0/3734685943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f219c1a0570 con 0x7f219c100ea0 2024-04-21T08:38:19.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.789+0000 7f2199ffb640 1 -- 172.21.15.43:0/3734685943 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2194003970 con 0x7f219c100ea0 2024-04-21T08:38:19.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.789+0000 7f21a5981740 1 Processor -- start 2024-04-21T08:38:19.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.789+0000 7f21a5981740 1 -- start start 2024-04-21T08:38:19.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.789+0000 7f21a5981740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd7197c250 0x55bd7197c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.789+0000 7f21a5981740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd7197cc40 0x55bd7197d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.789+0000 7f21a5981740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197d6e0 0x55bd719842c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.789+0000 7f21a5981740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bd718dcb10 con 0x55bd7197d6e0 2024-04-21T08:38:19.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.789+0000 7f21a5981740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bd7187a240 con 0x55bd7197c250 2024-04-21T08:38:19.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.789+0000 7f21a5981740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bd718d8ad0 con 0x55bd7197cc40 2024-04-21T08:38:19.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.789+0000 7f219bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197d6e0 0x55bd719842c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.789+0000 7f219bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197d6e0 0x55bd719842c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55158/0 (socket says 172.21.15.43:55158) 2024-04-21T08:38:19.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.789+0000 7f219bfff640 1 -- 172.21.15.43:0/2132557804 learned_addr learned my addr 172.21.15.43:0/2132557804 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.789+0000 7f21a0b34640 1 --2- 172.21.15.43:0/2132557804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd7197cc40 0x55bd7197d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.789+0000 7f2199ffb640 1 --2- 172.21.15.43:0/3734685943 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f21880a3bb0 0x7f21880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.789+0000 7f21a1335640 1 --2- 172.21.15.43:0/2132557804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd7197c250 0x55bd7197c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.789+0000 7f219bfff640 1 -- 172.21.15.43:0/2132557804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd7197cc40 msgr2=0x55bd7197d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.789+0000 7f219bfff640 1 --2- 172.21.15.43:0/2132557804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd7197cc40 0x55bd7197d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.789+0000 7f21a1335640 1 --2- 172.21.15.43:0/3734685943 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f21880a3bb0 0x7f21880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.790+0000 7f219bfff640 1 -- 172.21.15.43:0/2132557804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd7197c250 msgr2=0x55bd7197c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.790+0000 7f219bfff640 1 --2- 172.21.15.43:0/2132557804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd7197c250 0x55bd7197c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.790+0000 7f219bfff640 1 -- 172.21.15.43:0/2132557804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd715ef2d0 con 0x55bd7197d6e0 2024-04-21T08:38:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.790+0000 7f2199ffb640 1 -- 172.21.15.43:0/3734685943 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f21940f2d00 con 0x7f219c100ea0 2024-04-21T08:38:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.790+0000 7f219bfff640 1 --2- 172.21.15.43:0/2132557804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197d6e0 0x55bd719842c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f21840058c0 tx=0x7f218400d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9fb74eccf97b55 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.790+0000 7f2192ffd640 1 -- 172.21.15.43:0/2132557804 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2184014070 con 0x55bd7197d6e0 2024-04-21T08:38:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.790+0000 7f21a1335640 1 --2- 172.21.15.43:0/3734685943 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f21880a3bb0 0x7f21880a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55bd71973200 tx=0x7f218c005d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.790+0000 7f2192ffd640 1 -- 172.21.15.43:0/2132557804 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f218400dd10 con 0x55bd7197d6e0 2024-04-21T08:38:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.790+0000 7f2192ffd640 1 -- 172.21.15.43:0/2132557804 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21840049e0 con 0x55bd7197d6e0 2024-04-21T08:38:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.791+0000 7f21a5981740 1 -- 172.21.15.43:0/2132557804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197d6e0 msgr2=0x55bd719842c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.791+0000 7f21a5981740 1 --2- 172.21.15.43:0/2132557804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197d6e0 0x55bd719842c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f21840058c0 tx=0x7f218400d2e0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.791+0000 7f21a5981740 1 -- 172.21.15.43:0/2132557804 shutdown_connections 2024-04-21T08:38:19.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.791+0000 7f21a5981740 1 --2- 172.21.15.43:0/2132557804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197d6e0 0x55bd719842c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.791+0000 7f21a5981740 1 --2- 172.21.15.43:0/2132557804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd7197cc40 0x55bd7197d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.791+0000 7f21a5981740 1 --2- 172.21.15.43:0/2132557804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd7197c250 0x55bd7197c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.791+0000 7f21a5981740 1 -- 172.21.15.43:0/2132557804 >> 172.21.15.43:0/2132557804 conn(0x55bd71973f40 msgr2=0x55bd71972490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.791+0000 7f21a5981740 1 -- 172.21.15.43:0/2132557804 shutdown_connections 2024-04-21T08:38:19.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.791+0000 7f21a5981740 1 -- 172.21.15.43:0/2132557804 wait complete. 2024-04-21T08:38:19.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.791+0000 7f21a5981740 1 -- start start 2024-04-21T08:38:19.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.791+0000 7f21a5981740 1 Processor -- start 2024-04-21T08:38:19.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.791+0000 7f21a5981740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197c250 0x55bd719862f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.791+0000 7f21a5981740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd7197cc40 0x55bd71986830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.791+0000 7f21a5981740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd7197d6e0 0x55bd7198d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.791+0000 7f21a5981740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bd719848c0 con 0x55bd7197c250 2024-04-21T08:38:19.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.791+0000 7f21a5981740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bd71865190 con 0x55bd7197d6e0 2024-04-21T08:38:19.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.791+0000 7f21a5981740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bd71984a40 con 0x55bd7197cc40 2024-04-21T08:38:19.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.792+0000 7f21a5981740 10 client.0 _ll_register_callbacks cb 0x55bd718608b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:19.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.792+0000 7f21a0b34640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197c250 0x55bd719862f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.792+0000 7f21a0b34640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197c250 0x55bd719862f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55168/0 (socket says 172.21.15.43:55168) 2024-04-21T08:38:19.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.792+0000 7f21a0b34640 1 -- 172.21.15.43:0/2843561595 learned_addr learned my addr 172.21.15.43:0/2843561595 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.792+0000 7f21a0b34640 1 -- 172.21.15.43:0/2843561595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd7197cc40 msgr2=0x55bd71986830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.792+0000 7f219bfff640 1 --2- 172.21.15.43:0/2843561595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd7197cc40 0x55bd71986830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.792+0000 7f21a1335640 1 --2- 172.21.15.43:0/2843561595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd7197d6e0 0x55bd7198d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.792+0000 7f21a0b34640 1 --2- 172.21.15.43:0/2843561595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd7197cc40 0x55bd71986830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.792+0000 7f21a0b34640 1 -- 172.21.15.43:0/2843561595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd7197d6e0 msgr2=0x55bd7198d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.792+0000 7f21a0b34640 1 --2- 172.21.15.43:0/2843561595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd7197d6e0 0x55bd7198d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.792+0000 7f21a0b34640 1 -- 172.21.15.43:0/2843561595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd716b96b0 con 0x55bd7197c250 2024-04-21T08:38:19.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.792+0000 7f219bfff640 1 --2- 172.21.15.43:0/2843561595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd7197cc40 0x55bd71986830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:19.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.792+0000 7f21a1335640 1 --2- 172.21.15.43:0/2843561595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd7197d6e0 0x55bd7198d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:19.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.792+0000 7f21a0b34640 1 --2- 172.21.15.43:0/2843561595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197c250 0x55bd719862f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2194009430 tx=0x7f2194005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297f718335ed86bb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.792+0000 7f217bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:38:19.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.792+0000 7f217bfff640 1 -- 172.21.15.43:0/2843561595 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21940df270 con 0x55bd7197c250 2024-04-21T08:38:19.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.792+0000 7f217bfff640 1 -- 172.21.15.43:0/2843561595 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f219403e3a0 con 0x55bd7197c250 2024-04-21T08:38:19.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.792+0000 7f21a5981740 1 -- 172.21.15.43:0/2843561595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f218400dd10 con 0x55bd7197c250 2024-04-21T08:38:19.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.792+0000 7f217bfff640 1 -- 172.21.15.43:0/2843561595 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f219403e5a0 con 0x55bd7197c250 2024-04-21T08:38:19.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.792+0000 7f21a5981740 10 client.15288 fetch_fsmap learned FSMap version 27 2024-04-21T08:38:19.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.792+0000 7f21a5981740 1 -- 172.21.15.43:0/2843561595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bd71990be0 con 0x55bd7197c250 2024-04-21T08:38:19.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.793+0000 7f217bfff640 1 -- 172.21.15.43:0/2843561595 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f218400dd10 con 0x55bd7197c250 2024-04-21T08:38:19.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.793+0000 7f217bfff640 1 -- 172.21.15.43:0/2843561595 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f219403c030 con 0x55bd7197c250 2024-04-21T08:38:19.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.793+0000 7f21a5981740 10 client.15288 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:38:19.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.793+0000 7f21a5981740 10 client.15288 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-21T08:38:19.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.793+0000 7f21a5981740 20 client.15288 populate_metadata read hostname 'smithi043' 2024-04-21T08:38:19.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.793+0000 7f21a5981740 1 --2- 172.21.15.43:0/2843561595 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55bd71993310 0x55bd71994780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.793+0000 7f21a5981740 4 client.15288 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:38:19.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.793+0000 7f21a5981740 1 -- 172.21.15.43:0/2843561595 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd718ea530 con 0x55bd71993310 2024-04-21T08:38:19.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.793+0000 7f219bfff640 1 --2- 172.21.15.43:0/2843561595 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55bd71993310 0x55bd71994780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.793+0000 7f219bfff640 1 --2- 172.21.15.43:0/2843561595 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55bd71993310 0x55bd71994780 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=998f577abd35a39d server_cookie=ef55d285998cf2b9 in_seq=0 out_seq=0 2024-04-21T08:38:19.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.793+0000 7f217bfff640 10 client.15288 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:38:19.799 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:19 smithi112 ceph-mon[31085]: pgmap v356: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 12 op/s 2024-04-21T08:38:19.799 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:19 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3784611215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:19.799 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:19 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/760554590' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:19.799 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:19 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3766377596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:19.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.794+0000 7f217bfff640 1 -- 172.21.15.43:0/2843561595 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bd718ea530 con 0x55bd71993310 2024-04-21T08:38:19.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.794+0000 7f217bfff640 10 client.15288 handle_command_reply: tid=0 2024-04-21T08:38:19.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.794+0000 7f21a5981740 1 client.15288 shutdown 2024-04-21T08:38:19.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.794+0000 7f21a5981740 1 -- 172.21.15.43:0/2843561595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197c250 msgr2=0x55bd719862f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.794+0000 7f21a5981740 1 --2- 172.21.15.43:0/2843561595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197c250 0x55bd719862f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2194009430 tx=0x7f2194005ef0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.794+0000 7f21a5981740 1 -- 172.21.15.43:0/2843561595 shutdown_connections 2024-04-21T08:38:19.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.794+0000 7f21a5981740 1 --2- 172.21.15.43:0/2843561595 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55bd71993310 0x55bd71994780 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.794+0000 7f21a5981740 1 --2- 172.21.15.43:0/2843561595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd7197d6e0 0x55bd7198d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.794+0000 7f21a5981740 1 --2- 172.21.15.43:0/2843561595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd7197cc40 0x55bd71986830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.794+0000 7f217bfff640 0 client.15288 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:38:19.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.794+0000 7f21a5981740 1 --2- 172.21.15.43:0/2843561595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197c250 0x55bd719862f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.794+0000 7f21a5981740 1 -- 172.21.15.43:0/2843561595 >> 172.21.15.43:0/2843561595 conn(0x55bd71973f40 msgr2=0x55bd71971a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.795+0000 7f21a5981740 1 -- 172.21.15.43:0/2843561595 shutdown_connections 2024-04-21T08:38:19.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.795+0000 7f21a5981740 1 -- 172.21.15.43:0/2843561595 wait complete. 2024-04-21T08:38:19.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.795+0000 7f21a5981740 20 client.15288 trim_cache size 0 max 16384 2024-04-21T08:38:19.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.813+0000 7f21a5981740 1 Processor -- start 2024-04-21T08:38:19.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.813+0000 7f21a5981740 1 -- start start 2024-04-21T08:38:19.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.813+0000 7f21a5981740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197c250 0x55bd71991f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.813+0000 7f21a5981740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd7197cc40 0x55bd7198d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.813+0000 7f21a5981740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd7197d6e0 0x55bd71992470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.813+0000 7f21a5981740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bd71992ac0 con 0x55bd7197c250 2024-04-21T08:38:19.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.813+0000 7f21a5981740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bd71993190 con 0x55bd7197d6e0 2024-04-21T08:38:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.813+0000 7f21a5981740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bd71993860 con 0x55bd7197cc40 2024-04-21T08:38:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.813+0000 7f21a0b34640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197c250 0x55bd71991f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.813+0000 7f21a0b34640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197c250 0x55bd71991f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55174/0 (socket says 172.21.15.43:55174) 2024-04-21T08:38:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.813+0000 7f21a0b34640 1 -- 172.21.15.43:0/4090057890 learned_addr learned my addr 172.21.15.43:0/4090057890 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.813+0000 7f219bfff640 1 --2- 172.21.15.43:0/4090057890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd7197cc40 0x55bd7198d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.813+0000 7f21a1335640 1 --2- 172.21.15.43:0/4090057890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd7197d6e0 0x55bd71992470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.813+0000 7f21a0b34640 1 -- 172.21.15.43:0/4090057890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd7197cc40 msgr2=0x55bd7198d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.813+0000 7f21a0b34640 1 --2- 172.21.15.43:0/4090057890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd7197cc40 0x55bd7198d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.813+0000 7f21a0b34640 1 -- 172.21.15.43:0/4090057890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd7197d6e0 msgr2=0x55bd71992470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.813+0000 7f21a0b34640 1 --2- 172.21.15.43:0/4090057890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd7197d6e0 0x55bd71992470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.814+0000 7f21a0b34640 1 -- 172.21.15.43:0/4090057890 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd7196cd80 con 0x55bd7197c250 2024-04-21T08:38:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.814+0000 7f21a1335640 1 --2- 172.21.15.43:0/4090057890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd7197d6e0 0x55bd71992470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.814+0000 7f21a0b34640 1 --2- 172.21.15.43:0/4090057890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197c250 0x55bd71991f30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f21940bcea0 tx=0x7f21940095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546930d93a68c88e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.814+0000 7f21927fc640 1 -- 172.21.15.43:0/4090057890 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21940d8170 con 0x55bd7197c250 2024-04-21T08:38:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.814+0000 7f21927fc640 1 -- 172.21.15.43:0/4090057890 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21940a9040 con 0x55bd7197c250 2024-04-21T08:38:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.814+0000 7f21927fc640 1 -- 172.21.15.43:0/4090057890 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21940bf0d0 con 0x55bd7197c250 2024-04-21T08:38:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.814+0000 7f21a5981740 1 -- 172.21.15.43:0/4090057890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197c250 msgr2=0x55bd71991f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.814+0000 7f21a5981740 1 --2- 172.21.15.43:0/4090057890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197c250 0x55bd71991f30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f21940bcea0 tx=0x7f21940095c0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.814+0000 7f21a5981740 1 -- 172.21.15.43:0/4090057890 shutdown_connections 2024-04-21T08:38:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.814+0000 7f21a5981740 1 --2- 172.21.15.43:0/4090057890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd7197d6e0 0x55bd71992470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.814+0000 7f21a5981740 1 --2- 172.21.15.43:0/4090057890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd7197cc40 0x55bd7198d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.814+0000 7f21a5981740 1 --2- 172.21.15.43:0/4090057890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197c250 0x55bd71991f30 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.814+0000 7f21a5981740 1 -- 172.21.15.43:0/4090057890 >> 172.21.15.43:0/4090057890 conn(0x55bd71973f40 msgr2=0x55bd71980c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.815+0000 7f21a5981740 1 -- 172.21.15.43:0/4090057890 shutdown_connections 2024-04-21T08:38:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.815+0000 7f21a5981740 1 -- 172.21.15.43:0/4090057890 wait complete. 2024-04-21T08:38:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.815+0000 7f21a5981740 1 -- start start 2024-04-21T08:38:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.815+0000 7f21a5981740 1 Processor -- start 2024-04-21T08:38:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.815+0000 7f21a5981740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197c250 0x55bd71989810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.815+0000 7f21a5981740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd7197cc40 0x55bd7198bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.815+0000 7f21a5981740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd7197d6e0 0x55bd7198c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.815+0000 7f21a5981740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bd71994600 con 0x55bd7197c250 2024-04-21T08:38:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.815+0000 7f21a5981740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bd71993f30 con 0x55bd7197cc40 2024-04-21T08:38:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.815+0000 7f21a5981740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bd71994cd0 con 0x55bd7197d6e0 2024-04-21T08:38:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.815+0000 7f21a5981740 10 client.0 _ll_register_callbacks cb 0x7f2194032340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.815+0000 7f21a0b34640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197c250 0x55bd71989810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.815+0000 7f21a0b34640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197c250 0x55bd71989810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55180/0 (socket says 172.21.15.43:55180) 2024-04-21T08:38:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.815+0000 7f21a0b34640 1 -- 172.21.15.43:0/3131308682 learned_addr learned my addr 172.21.15.43:0/3131308682 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.815+0000 7f21a1335640 1 --2- 172.21.15.43:0/3131308682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd7197d6e0 0x55bd7198c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.816+0000 7f21a0b34640 1 -- 172.21.15.43:0/3131308682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd7197d6e0 msgr2=0x55bd7198c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.816+0000 7f21a0b34640 1 --2- 172.21.15.43:0/3131308682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd7197d6e0 0x55bd7198c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.816+0000 7f21a0b34640 1 -- 172.21.15.43:0/3131308682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd7197cc40 msgr2=0x55bd7198bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.816+0000 7f21a0b34640 1 --2- 172.21.15.43:0/3131308682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd7197cc40 0x55bd7198bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.816+0000 7f21a0b34640 1 -- 172.21.15.43:0/3131308682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd7198cb10 con 0x55bd7197c250 2024-04-21T08:38:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.816+0000 7f21a0b34640 1 --2- 172.21.15.43:0/3131308682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197c250 0x55bd71989810 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f219403d040 tx=0x7f219402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf699e5be093ac20 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.816+0000 7f217bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:38:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.816+0000 7f217bfff640 1 -- 172.21.15.43:0/3131308682 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194087070 con 0x55bd7197c250 2024-04-21T08:38:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.816+0000 7f217bfff640 1 -- 172.21.15.43:0/3131308682 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f21940325c0 con 0x55bd7197c250 2024-04-21T08:38:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.816+0000 7f21a5981740 1 -- 172.21.15.43:0/3131308682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21940a9040 con 0x55bd7197c250 2024-04-21T08:38:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.816+0000 7f217bfff640 1 -- 172.21.15.43:0/3131308682 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f21940d8170 con 0x55bd7197c250 2024-04-21T08:38:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.816+0000 7f21a5981740 10 client.15294 fetch_fsmap learned FSMap version 27 2024-04-21T08:38:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.816+0000 7f21a5981740 1 -- 172.21.15.43:0/3131308682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bd7198cb10 con 0x55bd7197c250 2024-04-21T08:38:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.816+0000 7f217bfff640 1 -- 172.21.15.43:0/3131308682 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21940a9040 con 0x55bd7197c250 2024-04-21T08:38:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.816+0000 7f217bfff640 1 -- 172.21.15.43:0/3131308682 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f21940be040 con 0x55bd7197c250 2024-04-21T08:38:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.816+0000 7f21a5981740 10 client.15294 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:38:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.816+0000 7f21a5981740 10 client.15294 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-21T08:38:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.816+0000 7f21a5981740 20 client.15294 populate_metadata read hostname 'smithi043' 2024-04-21T08:38:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.816+0000 7f21a5981740 1 --2- 172.21.15.43:0/3131308682 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55bd719fd960 0x55bd719fc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.816+0000 7f21a5981740 4 client.15294 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:38:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.816+0000 7f21a5981740 1 -- 172.21.15.43:0/3131308682 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "status"}) -- 0x55bd718e43a0 con 0x55bd719fd960 2024-04-21T08:38:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.817+0000 7f219bfff640 1 --2- 172.21.15.43:0/3131308682 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55bd719fd960 0x55bd719fc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.817+0000 7f219bfff640 1 --2- 172.21.15.43:0/3131308682 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55bd719fd960 0x55bd719fc670 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d14a8235a052105 server_cookie=5f69a99da6584a24 in_seq=0 out_seq=0 2024-04-21T08:38:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.817+0000 7f217bfff640 10 client.15294 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:38:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.817+0000 7f217bfff640 1 -- 172.21.15.43:0/3131308682 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bd718e43a0 con 0x55bd719fd960 2024-04-21T08:38:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.817+0000 7f217bfff640 10 client.15294 handle_command_reply: tid=0 2024-04-21T08:38:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.817+0000 7f21a5981740 1 client.15294 shutdown 2024-04-21T08:38:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.817+0000 7f21a5981740 1 -- 172.21.15.43:0/3131308682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197c250 msgr2=0x55bd71989810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.817+0000 7f21a5981740 1 --2- 172.21.15.43:0/3131308682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197c250 0x55bd71989810 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f219403d040 tx=0x7f219402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.818+0000 7f21a5981740 1 -- 172.21.15.43:0/3131308682 shutdown_connections 2024-04-21T08:38:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.818+0000 7f21a5981740 1 --2- 172.21.15.43:0/3131308682 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55bd719fd960 0x55bd719fc670 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.818+0000 7f21a5981740 1 --2- 172.21.15.43:0/3131308682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bd7197d6e0 0x55bd7198c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.818+0000 7f21a5981740 1 --2- 172.21.15.43:0/3131308682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bd7197cc40 0x55bd7198bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.818+0000 7f21a5981740 1 --2- 172.21.15.43:0/3131308682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd7197c250 0x55bd71989810 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.818+0000 7f217bfff640 0 client.15294 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:38:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.818+0000 7f21a5981740 1 -- 172.21.15.43:0/3131308682 >> 172.21.15.43:0/3131308682 conn(0x55bd71973f40 msgr2=0x55bd71985ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.818+0000 7f21a5981740 1 -- 172.21.15.43:0/3131308682 shutdown_connections 2024-04-21T08:38:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.818+0000 7f21a5981740 1 -- 172.21.15.43:0/3131308682 wait complete. 2024-04-21T08:38:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.818+0000 7f21a5981740 20 client.15294 trim_cache size 0 max 16384 2024-04-21T08:38:19.824 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:38:19.824 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "d57a7d98-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:38:19.824 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 1, 2024-04-21T08:38:19.824 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24491, 2024-04-21T08:38:19.824 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-21T08:38:19.824 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-21T08:38:19.824 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-21T08:38:19.824 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 91.765309384999995, 2024-04-21T08:38:19.824 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 27, 2024-04-21T08:38:19.824 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 94, 2024-04-21T08:38:19.824 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 90, 2024-04-21T08:38:19.824 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 132.527482771 2024-04-21T08:38:19.825 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:38:19.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.818+0000 7f21a2d98640 1 -- 172.21.15.43:0/3734685943 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f21880a3bb0 msgr2=0x7f21880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.818+0000 7f21a2d98640 1 --2- 172.21.15.43:0/3734685943 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f21880a3bb0 0x7f21880a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55bd71973200 tx=0x7f218c005d60 comp rx=0 tx=0).stop 2024-04-21T08:38:19.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.818+0000 7f21a2d98640 1 -- 172.21.15.43:0/3734685943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f219c100ea0 msgr2=0x7f219c198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.818+0000 7f21a2d98640 1 --2- 172.21.15.43:0/3734685943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f219c100ea0 0x7f219c198200 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2194005cc0 tx=0x7f2194034830 comp rx=0 tx=0).stop 2024-04-21T08:38:19.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.818+0000 7f21a2d98640 1 -- 172.21.15.43:0/3734685943 shutdown_connections 2024-04-21T08:38:19.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.818+0000 7f21a2d98640 1 --2- 172.21.15.43:0/3734685943 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f21880a3bb0 0x7f21880a6070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.818+0000 7f21a2d98640 1 --2- 172.21.15.43:0/3734685943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f219c1029b0 0x7f219c19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.818+0000 7f21a2d98640 1 --2- 172.21.15.43:0/3734685943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f219c101ff0 0x7f219c198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.818+0000 7f21a2d98640 1 --2- 172.21.15.43:0/3734685943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f219c100ea0 0x7f219c198200 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.818+0000 7f21a2d98640 1 -- 172.21.15.43:0/3734685943 >> 172.21.15.43:0/3734685943 conn(0x7f219c0fc410 msgr2=0x7f219c06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.818+0000 7f21a2d98640 1 -- 172.21.15.43:0/3734685943 shutdown_connections 2024-04-21T08:38:19.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.818+0000 7f21a2d98640 1 -- 172.21.15.43:0/3734685943 wait complete. 2024-04-21T08:38:19.829 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-21T08:38:19.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.933+0000 7f2204bd5640 1 -- 172.21.15.43:0/1773868158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2200100e90 msgr2=0x7f22001012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.933+0000 7f2204bd5640 1 --2- 172.21.15.43:0/1773868158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2200100e90 0x7f22001012b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f21f8009900 tx=0x7f21f802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:19.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.934+0000 7f2204bd5640 1 -- 172.21.15.43:0/1773868158 shutdown_connections 2024-04-21T08:38:19.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.934+0000 7f2204bd5640 1 --2- 172.21.15.43:0/1773868158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22001029a0 0x7f22001092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.934+0000 7f2204bd5640 1 --2- 172.21.15.43:0/1773868158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2200101fe0 0x7f2200102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.934+0000 7f2204bd5640 1 --2- 172.21.15.43:0/1773868158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2200100e90 0x7f22001012b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.934+0000 7f2204bd5640 1 -- 172.21.15.43:0/1773868158 >> 172.21.15.43:0/1773868158 conn(0x7f22000fc400 msgr2=0x7f22000fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.934+0000 7f2204bd5640 1 -- 172.21.15.43:0/1773868158 shutdown_connections 2024-04-21T08:38:19.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.934+0000 7f2204bd5640 1 -- 172.21.15.43:0/1773868158 wait complete. 2024-04-21T08:38:19.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.934+0000 7f2204bd5640 1 Processor -- start 2024-04-21T08:38:19.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.934+0000 7f2204bd5640 1 -- start start 2024-04-21T08:38:19.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.935+0000 7f2204bd5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2200100e90 0x7f2200198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.935+0000 7f2204bd5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2200101fe0 0x7f2200198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.935+0000 7f2204bd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22001029a0 0x7f220019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.935+0000 7f2204bd5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f220010db30 con 0x7f2200101fe0 2024-04-21T08:38:19.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.935+0000 7f2204bd5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f220010d9b0 con 0x7f22001029a0 2024-04-21T08:38:19.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.935+0000 7f2204bd5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f220010dcb0 con 0x7f2200100e90 2024-04-21T08:38:19.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.935+0000 7f21fdd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2200101fe0 0x7f2200198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.935+0000 7f21fdd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2200101fe0 0x7f2200198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55194/0 (socket says 172.21.15.43:55194) 2024-04-21T08:38:19.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.935+0000 7f21fdd9b640 1 -- 172.21.15.43:0/3319772348 learned_addr learned my addr 172.21.15.43:0/3319772348 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.935+0000 7f21fe59c640 1 --2- 172.21.15.43:0/3319772348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2200100e90 0x7f2200198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.935+0000 7f21fed9d640 1 --2- 172.21.15.43:0/3319772348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22001029a0 0x7f220019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.935+0000 7f21fdd9b640 1 -- 172.21.15.43:0/3319772348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2200100e90 msgr2=0x7f2200198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.935+0000 7f21fdd9b640 1 --2- 172.21.15.43:0/3319772348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2200100e90 0x7f2200198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.935+0000 7f21fdd9b640 1 -- 172.21.15.43:0/3319772348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22001029a0 msgr2=0x7f220019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.935+0000 7f21fdd9b640 1 --2- 172.21.15.43:0/3319772348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22001029a0 0x7f220019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.935+0000 7f21fdd9b640 1 -- 172.21.15.43:0/3319772348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220019fe60 con 0x7f2200101fe0 2024-04-21T08:38:19.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.935+0000 7f21fdd9b640 1 --2- 172.21.15.43:0/3319772348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2200101fe0 0x7f2200198780 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f21e800d770 tx=0x7f21e800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.935+0000 7f21f77fe640 1 -- 172.21.15.43:0/3319772348 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8017070 con 0x7f2200101fe0 2024-04-21T08:38:19.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.935+0000 7f21f77fe640 1 -- 172.21.15.43:0/3319772348 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21e8009c30 con 0x7f2200101fe0 2024-04-21T08:38:19.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.935+0000 7f2204bd5640 1 -- 172.21.15.43:0/3319772348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22001a0150 con 0x7f2200101fe0 2024-04-21T08:38:19.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.935+0000 7f21f77fe640 1 -- 172.21.15.43:0/3319772348 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8003cc0 con 0x7f2200101fe0 2024-04-21T08:38:19.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.935+0000 7f2204bd5640 1 -- 172.21.15.43:0/3319772348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22001a06f0 con 0x7f2200101fe0 2024-04-21T08:38:19.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.938+0000 7f21f77fe640 1 -- 172.21.15.43:0/3319772348 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f21e8003e60 con 0x7f2200101fe0 2024-04-21T08:38:19.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.938+0000 7f22077ca740 1 Processor -- start 2024-04-21T08:38:19.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.938+0000 7f22077ca740 1 -- start start 2024-04-21T08:38:19.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.938+0000 7f22077ca740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c8e40 0x55aa858c9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.938+0000 7f22077ca740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa858c9830 0x55aa858c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.938+0000 7f22077ca740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aa858ca2d0 0x55aa858d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.938+0000 7f22077ca740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aa8582db10 con 0x55aa858c8e40 2024-04-21T08:38:19.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.938+0000 7f22077ca740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55aa857cb240 con 0x55aa858ca2d0 2024-04-21T08:38:19.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.938+0000 7f22077ca740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55aa85829ad0 con 0x55aa858c9830 2024-04-21T08:38:19.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.938+0000 7f21f77fe640 1 --2- 172.21.15.43:0/3319772348 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f21ec0a3c30 0x7f21ec0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.938+0000 7f21fed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c8e40 0x55aa858c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.938+0000 7f21fed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c8e40 0x55aa858c9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55200/0 (socket says 172.21.15.43:55200) 2024-04-21T08:38:19.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.938+0000 7f21fed9d640 1 -- 172.21.15.43:0/1808410389 learned_addr learned my addr 172.21.15.43:0/1808410389 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.939+0000 7f21fed9d640 1 --2- 172.21.15.43:0/3319772348 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f21ec0a3c30 0x7f21ec0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.939+0000 7f21fe59c640 1 --2- 172.21.15.43:0/1808410389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa858c9830 0x55aa858c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.939+0000 7f21f77fe640 1 -- 172.21.15.43:0/3319772348 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f21e80d0100 con 0x7f2200101fe0 2024-04-21T08:38:19.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.939+0000 7f21fdd9b640 1 --2- 172.21.15.43:0/1808410389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aa858ca2d0 0x55aa858d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.939+0000 7f21fed9d640 1 -- 172.21.15.43:0/1808410389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa858c9830 msgr2=0x55aa858c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.939+0000 7f21fed9d640 1 --2- 172.21.15.43:0/1808410389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa858c9830 0x55aa858c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.939+0000 7f21fed9d640 1 -- 172.21.15.43:0/1808410389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aa858ca2d0 msgr2=0x55aa858d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.939+0000 7f21fed9d640 1 --2- 172.21.15.43:0/1808410389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aa858ca2d0 0x55aa858d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.939+0000 7f21fed9d640 1 -- 172.21.15.43:0/1808410389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa855402d0 con 0x55aa858c8e40 2024-04-21T08:38:19.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.939+0000 7f21fdd9b640 1 --2- 172.21.15.43:0/1808410389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aa858ca2d0 0x55aa858d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:19.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.939+0000 7f21fed9d640 1 --2- 172.21.15.43:0/1808410389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c8e40 0x55aa858c9260 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f21f0013dc0 tx=0x7f21f0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d561fa5b51729c51 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.939+0000 7f21fed9d640 1 --2- 172.21.15.43:0/3319772348 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f21ec0a3c30 0x7f21ec0a60f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f21f0010c40 tx=0x7f21f001b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.939+0000 7f21f4ff9640 1 -- 172.21.15.43:0/1808410389 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f0021070 con 0x55aa858c8e40 2024-04-21T08:38:19.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.939+0000 7f21f4ff9640 1 -- 172.21.15.43:0/1808410389 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21f000aba0 con 0x55aa858c8e40 2024-04-21T08:38:19.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.939+0000 7f21f4ff9640 1 -- 172.21.15.43:0/1808410389 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f0004310 con 0x55aa858c8e40 2024-04-21T08:38:19.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.940+0000 7f22077ca740 1 -- 172.21.15.43:0/1808410389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c8e40 msgr2=0x55aa858c9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.940+0000 7f22077ca740 1 --2- 172.21.15.43:0/1808410389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c8e40 0x55aa858c9260 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f21f0013dc0 tx=0x7f21f0010740 comp rx=0 tx=0).stop 2024-04-21T08:38:19.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.940+0000 7f22077ca740 1 -- 172.21.15.43:0/1808410389 shutdown_connections 2024-04-21T08:38:19.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.940+0000 7f22077ca740 1 --2- 172.21.15.43:0/1808410389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aa858ca2d0 0x55aa858d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.940+0000 7f22077ca740 1 --2- 172.21.15.43:0/1808410389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa858c9830 0x55aa858c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.940+0000 7f22077ca740 1 --2- 172.21.15.43:0/1808410389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c8e40 0x55aa858c9260 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.940+0000 7f22077ca740 1 -- 172.21.15.43:0/1808410389 >> 172.21.15.43:0/1808410389 conn(0x55aa858c4f40 msgr2=0x55aa858c3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.940+0000 7f22077ca740 1 -- 172.21.15.43:0/1808410389 shutdown_connections 2024-04-21T08:38:19.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.940+0000 7f22077ca740 1 -- 172.21.15.43:0/1808410389 wait complete. 2024-04-21T08:38:19.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.940+0000 7f22077ca740 1 -- start start 2024-04-21T08:38:19.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.940+0000 7f22077ca740 1 Processor -- start 2024-04-21T08:38:19.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.941+0000 7f22077ca740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c8e40 0x55aa858d2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.941+0000 7f22077ca740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa858c9830 0x55aa858d33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.941+0000 7f22077ca740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aa858ca2d0 0x55aa858da430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.941+0000 7f22077ca740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aa858d1450 con 0x55aa858c8e40 2024-04-21T08:38:19.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.941+0000 7f22077ca740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55aa857b6190 con 0x55aa858ca2d0 2024-04-21T08:38:19.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.941+0000 7f22077ca740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55aa858d15d0 con 0x55aa858c9830 2024-04-21T08:38:19.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.941+0000 7f21fe59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c8e40 0x55aa858d2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.941+0000 7f22077ca740 10 client.0 _ll_register_callbacks cb 0x55aa857b18b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:19.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.941+0000 7f21fe59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c8e40 0x55aa858d2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55206/0 (socket says 172.21.15.43:55206) 2024-04-21T08:38:19.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.941+0000 7f21fe59c640 1 -- 172.21.15.43:0/279129354 learned_addr learned my addr 172.21.15.43:0/279129354 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.941+0000 7f21fed9d640 1 --2- 172.21.15.43:0/279129354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aa858ca2d0 0x55aa858da430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.941+0000 7f21fdd9b640 1 --2- 172.21.15.43:0/279129354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa858c9830 0x55aa858d33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.941+0000 7f21fe59c640 1 -- 172.21.15.43:0/279129354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa858c9830 msgr2=0x55aa858d33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.941+0000 7f21fe59c640 1 --2- 172.21.15.43:0/279129354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa858c9830 0x55aa858d33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.941+0000 7f21fe59c640 1 -- 172.21.15.43:0/279129354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aa858ca2d0 msgr2=0x55aa858da430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.941+0000 7f21fe59c640 1 --2- 172.21.15.43:0/279129354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aa858ca2d0 0x55aa858da430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.941+0000 7f21fe59c640 1 -- 172.21.15.43:0/279129354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa8560a6b0 con 0x55aa858c8e40 2024-04-21T08:38:19.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.941+0000 7f21fe59c640 1 --2- 172.21.15.43:0/279129354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c8e40 0x55aa858d2eb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f21f8009900 tx=0x7f21f8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f38e086618d1db server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.941+0000 7f21e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:38:19.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.942+0000 7f21e5ffb640 1 -- 172.21.15.43:0/279129354 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f8005030 con 0x55aa858c8e40 2024-04-21T08:38:19.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.942+0000 7f21e5ffb640 1 -- 172.21.15.43:0/279129354 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f21f80051d0 con 0x55aa858c8e40 2024-04-21T08:38:19.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.942+0000 7f22077ca740 1 -- 172.21.15.43:0/279129354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21f000aba0 con 0x55aa858c8e40 2024-04-21T08:38:19.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.942+0000 7f21e5ffb640 1 -- 172.21.15.43:0/279129354 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f21f8051050 con 0x55aa858c8e40 2024-04-21T08:38:19.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.942+0000 7f22077ca740 10 client.15306 fetch_fsmap learned FSMap version 27 2024-04-21T08:38:19.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.942+0000 7f22077ca740 1 -- 172.21.15.43:0/279129354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aa858dd810 con 0x55aa858c8e40 2024-04-21T08:38:19.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.942+0000 7f21e5ffb640 1 -- 172.21.15.43:0/279129354 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21f000aba0 con 0x55aa858c8e40 2024-04-21T08:38:19.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.942+0000 7f21e5ffb640 1 -- 172.21.15.43:0/279129354 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f21f8003680 con 0x55aa858c8e40 2024-04-21T08:38:19.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.942+0000 7f22077ca740 10 client.15306 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:38:19.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.942+0000 7f22077ca740 10 client.15306 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T08:38:19.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.942+0000 7f22077ca740 20 client.15306 populate_metadata read hostname 'smithi043' 2024-04-21T08:38:19.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.942+0000 7f22077ca740 1 --2- 172.21.15.43:0/279129354 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55aa858dfec0 0x55aa858e1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.942+0000 7f22077ca740 4 client.15306 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:38:19.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.942+0000 7f22077ca740 1 -- 172.21.15.43:0/279129354 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa8583b530 con 0x55aa858dfec0 2024-04-21T08:38:19.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.942+0000 7f21fdd9b640 1 --2- 172.21.15.43:0/279129354 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55aa858dfec0 0x55aa858e1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.943+0000 7f21fdd9b640 1 --2- 172.21.15.43:0/279129354 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55aa858dfec0 0x55aa858e1330 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48a9bdc65d2e7e5d server_cookie=e610b13d8b897267 in_seq=0 out_seq=0 2024-04-21T08:38:19.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.943+0000 7f21e5ffb640 10 client.15306 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:38:19.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.943+0000 7f21e5ffb640 1 -- 172.21.15.43:0/279129354 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aa8583b530 con 0x55aa858dfec0 2024-04-21T08:38:19.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.943+0000 7f21e5ffb640 10 client.15306 handle_command_reply: tid=0 2024-04-21T08:38:19.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.943+0000 7f22077ca740 1 client.15306 shutdown 2024-04-21T08:38:19.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.944+0000 7f22077ca740 1 -- 172.21.15.43:0/279129354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c8e40 msgr2=0x55aa858d2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.944+0000 7f22077ca740 1 --2- 172.21.15.43:0/279129354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c8e40 0x55aa858d2eb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f21f8009900 tx=0x7f21f8004270 comp rx=0 tx=0).stop 2024-04-21T08:38:19.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.944+0000 7f22077ca740 1 -- 172.21.15.43:0/279129354 shutdown_connections 2024-04-21T08:38:19.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.944+0000 7f22077ca740 1 --2- 172.21.15.43:0/279129354 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55aa858dfec0 0x55aa858e1330 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.944+0000 7f22077ca740 1 --2- 172.21.15.43:0/279129354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aa858ca2d0 0x55aa858da430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.944+0000 7f22077ca740 1 --2- 172.21.15.43:0/279129354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa858c9830 0x55aa858d33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.944+0000 7f21e5ffb640 0 client.15306 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:38:19.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.944+0000 7f22077ca740 1 --2- 172.21.15.43:0/279129354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c8e40 0x55aa858d2eb0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.944+0000 7f22077ca740 1 -- 172.21.15.43:0/279129354 >> 172.21.15.43:0/279129354 conn(0x55aa858c4f40 msgr2=0x55aa858c2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.944+0000 7f22077ca740 1 -- 172.21.15.43:0/279129354 shutdown_connections 2024-04-21T08:38:19.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.944+0000 7f22077ca740 1 -- 172.21.15.43:0/279129354 wait complete. 2024-04-21T08:38:19.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.944+0000 7f22077ca740 20 client.15306 trim_cache size 0 max 16384 2024-04-21T08:38:19.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.962+0000 7f22077ca740 1 Processor -- start 2024-04-21T08:38:19.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.962+0000 7f22077ca740 1 -- start start 2024-04-21T08:38:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.962+0000 7f22077ca740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa858c8e40 0x55aa858c3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.962+0000 7f22077ca740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c9830 0x55aa858c3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.962+0000 7f22077ca740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aa858ca2d0 0x55aa858da3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.962+0000 7f22077ca740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aa858df670 con 0x55aa858c9830 2024-04-21T08:38:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.962+0000 7f22077ca740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55aa858dfd40 con 0x55aa858ca2d0 2024-04-21T08:38:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.962+0000 7f22077ca740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55aa858e0410 con 0x55aa858c8e40 2024-04-21T08:38:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.962+0000 7f21fdd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c9830 0x55aa858c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.962+0000 7f21fdd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c9830 0x55aa858c3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55208/0 (socket says 172.21.15.43:55208) 2024-04-21T08:38:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.962+0000 7f21fdd9b640 1 -- 172.21.15.43:0/756949556 learned_addr learned my addr 172.21.15.43:0/756949556 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.962+0000 7f21fe59c640 1 --2- 172.21.15.43:0/756949556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa858c8e40 0x55aa858c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.962+0000 7f21fdd9b640 1 -- 172.21.15.43:0/756949556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa858c8e40 msgr2=0x55aa858c3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.962+0000 7f21fdd9b640 1 --2- 172.21.15.43:0/756949556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa858c8e40 0x55aa858c3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.963+0000 7f21fdd9b640 1 -- 172.21.15.43:0/756949556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aa858ca2d0 msgr2=0x55aa858da3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.963+0000 7f21fdd9b640 1 --2- 172.21.15.43:0/756949556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aa858ca2d0 0x55aa858da3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.963+0000 7f21fdd9b640 1 -- 172.21.15.43:0/756949556 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa858db8b0 con 0x55aa858c9830 2024-04-21T08:38:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.963+0000 7f21fe59c640 1 --2- 172.21.15.43:0/756949556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa858c8e40 0x55aa858c3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.963+0000 7f21fdd9b640 1 --2- 172.21.15.43:0/756949556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c9830 0x55aa858c3940 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009610 tx=0x7f21e80982d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d199e7f259f315 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.963+0000 7f21e7fff640 1 -- 172.21.15.43:0/756949556 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e80ba070 con 0x55aa858c9830 2024-04-21T08:38:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.963+0000 7f21e7fff640 1 -- 172.21.15.43:0/756949556 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21e809bff0 con 0x55aa858c9830 2024-04-21T08:38:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.963+0000 7f21e7fff640 1 -- 172.21.15.43:0/756949556 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e80b5070 con 0x55aa858c9830 2024-04-21T08:38:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.963+0000 7f22077ca740 1 -- 172.21.15.43:0/756949556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c9830 msgr2=0x55aa858c3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.963+0000 7f22077ca740 1 --2- 172.21.15.43:0/756949556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c9830 0x55aa858c3940 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009610 tx=0x7f21e80982d0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.963+0000 7f22077ca740 1 -- 172.21.15.43:0/756949556 shutdown_connections 2024-04-21T08:38:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.963+0000 7f22077ca740 1 --2- 172.21.15.43:0/756949556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aa858ca2d0 0x55aa858da3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.963+0000 7f22077ca740 1 --2- 172.21.15.43:0/756949556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c9830 0x55aa858c3940 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.963+0000 7f22077ca740 1 --2- 172.21.15.43:0/756949556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa858c8e40 0x55aa858c3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.963+0000 7f22077ca740 1 -- 172.21.15.43:0/756949556 >> 172.21.15.43:0/756949556 conn(0x55aa858c4f40 msgr2=0x55aa858cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.963+0000 7f22077ca740 1 -- 172.21.15.43:0/756949556 shutdown_connections 2024-04-21T08:38:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.963+0000 7f22077ca740 1 -- 172.21.15.43:0/756949556 wait complete. 2024-04-21T08:38:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.964+0000 7f22077ca740 1 -- start start 2024-04-21T08:38:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.964+0000 7f22077ca740 1 Processor -- start 2024-04-21T08:38:19.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.964+0000 7f22077ca740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c8e40 0x55aa858d6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.964+0000 7f22077ca740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa858c9830 0x55aa858d8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.964+0000 7f22077ca740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aa858ca2d0 0x55aa858d9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.964+0000 7f22077ca740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aa858e11b0 con 0x55aa858c8e40 2024-04-21T08:38:19.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.964+0000 7f22077ca740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55aa858e0ae0 con 0x55aa858ca2d0 2024-04-21T08:38:19.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.964+0000 7f22077ca740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55aa858e1880 con 0x55aa858c9830 2024-04-21T08:38:19.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.964+0000 7f22077ca740 10 client.0 _ll_register_callbacks cb 0x7f21f8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:19.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.964+0000 7f21fe59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c8e40 0x55aa858d6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.964+0000 7f21fe59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c8e40 0x55aa858d6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55222/0 (socket says 172.21.15.43:55222) 2024-04-21T08:38:19.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.964+0000 7f21fe59c640 1 -- 172.21.15.43:0/314932557 learned_addr learned my addr 172.21.15.43:0/314932557 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:19.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.964+0000 7f21fed9d640 1 --2- 172.21.15.43:0/314932557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aa858ca2d0 0x55aa858d9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.964+0000 7f21fdd9b640 1 --2- 172.21.15.43:0/314932557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa858c9830 0x55aa858d8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.964+0000 7f21fe59c640 1 -- 172.21.15.43:0/314932557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa858c9830 msgr2=0x55aa858d8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.964+0000 7f21fe59c640 1 --2- 172.21.15.43:0/314932557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa858c9830 0x55aa858d8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.964+0000 7f21fe59c640 1 -- 172.21.15.43:0/314932557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aa858ca2d0 msgr2=0x55aa858d9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:19.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.964+0000 7f21fe59c640 1 --2- 172.21.15.43:0/314932557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aa858ca2d0 0x55aa858d9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.964+0000 7f21fe59c640 1 -- 172.21.15.43:0/314932557 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa858d9940 con 0x55aa858c8e40 2024-04-21T08:38:19.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.964+0000 7f21fdd9b640 1 --2- 172.21.15.43:0/314932557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa858c9830 0x55aa858d8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:19.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.965+0000 7f21fe59c640 1 --2- 172.21.15.43:0/314932557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c8e40 0x55aa858d6620 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f21f8004650 tx=0x7f21f8003fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469bd09ca4cbb1c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:19.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.965+0000 7f21e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:38:19.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.965+0000 7f21e5ffb640 1 -- 172.21.15.43:0/314932557 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f802cc80 con 0x55aa858c8e40 2024-04-21T08:38:19.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.965+0000 7f21e5ffb640 1 -- 172.21.15.43:0/314932557 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f21f8034440 con 0x55aa858c8e40 2024-04-21T08:38:19.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.965+0000 7f22077ca740 1 -- 172.21.15.43:0/314932557 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21e809bff0 con 0x55aa858c8e40 2024-04-21T08:38:19.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.965+0000 7f21e5ffb640 1 -- 172.21.15.43:0/314932557 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f21f8005020 con 0x55aa858c8e40 2024-04-21T08:38:19.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.965+0000 7f22077ca740 10 client.15312 fetch_fsmap learned FSMap version 27 2024-04-21T08:38:19.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.965+0000 7f22077ca740 1 -- 172.21.15.43:0/314932557 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aa85946160 con 0x55aa858c8e40 2024-04-21T08:38:19.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.965+0000 7f21e5ffb640 1 -- 172.21.15.43:0/314932557 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21e809bff0 con 0x55aa858c8e40 2024-04-21T08:38:19.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.965+0000 7f21e5ffb640 1 -- 172.21.15.43:0/314932557 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f21f8034680 con 0x55aa858c8e40 2024-04-21T08:38:19.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.965+0000 7f22077ca740 10 client.15312 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:38:19.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.965+0000 7f22077ca740 10 client.15312 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T08:38:19.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.965+0000 7f22077ca740 20 client.15312 populate_metadata read hostname 'smithi043' 2024-04-21T08:38:19.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.965+0000 7f22077ca740 1 --2- 172.21.15.43:0/314932557 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55aa8594ad00 0x55aa85949a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:19.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.965+0000 7f22077ca740 4 client.15312 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:38:19.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.965+0000 7f22077ca740 1 -- 172.21.15.43:0/314932557 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "status"}) -- 0x55aa858353a0 con 0x55aa8594ad00 2024-04-21T08:38:19.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.966+0000 7f21fdd9b640 1 --2- 172.21.15.43:0/314932557 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55aa8594ad00 0x55aa85949a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:19.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.966+0000 7f21fdd9b640 1 --2- 172.21.15.43:0/314932557 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55aa8594ad00 0x55aa85949a10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60853d3014d4eb20 server_cookie=bffaed42e5f4b07b in_seq=0 out_seq=0 2024-04-21T08:38:19.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.966+0000 7f21e5ffb640 10 client.15312 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:38:19.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.966+0000 7f21e5ffb640 1 -- 172.21.15.43:0/314932557 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55aa858353a0 con 0x55aa8594ad00 2024-04-21T08:38:19.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.966+0000 7f21e5ffb640 10 client.15312 handle_command_reply: tid=0 2024-04-21T08:38:19.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.966+0000 7f22077ca740 1 client.15312 shutdown 2024-04-21T08:38:19.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.966+0000 7f22077ca740 1 -- 172.21.15.43:0/314932557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c8e40 msgr2=0x55aa858d6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.966+0000 7f22077ca740 1 --2- 172.21.15.43:0/314932557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c8e40 0x55aa858d6620 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f21f8004650 tx=0x7f21f8003fc0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.966+0000 7f22077ca740 1 -- 172.21.15.43:0/314932557 shutdown_connections 2024-04-21T08:38:19.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.966+0000 7f22077ca740 1 --2- 172.21.15.43:0/314932557 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55aa8594ad00 0x55aa85949a10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.966+0000 7f22077ca740 1 --2- 172.21.15.43:0/314932557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aa858ca2d0 0x55aa858d9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.966+0000 7f22077ca740 1 --2- 172.21.15.43:0/314932557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aa858c9830 0x55aa858d8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.966+0000 7f21e5ffb640 0 client.15312 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:38:19.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.966+0000 7f22077ca740 1 --2- 172.21.15.43:0/314932557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa858c8e40 0x55aa858d6620 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.966+0000 7f22077ca740 1 -- 172.21.15.43:0/314932557 >> 172.21.15.43:0/314932557 conn(0x55aa858c4f40 msgr2=0x55aa858c2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.967+0000 7f22077ca740 1 -- 172.21.15.43:0/314932557 shutdown_connections 2024-04-21T08:38:19.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.967+0000 7f22077ca740 1 -- 172.21.15.43:0/314932557 wait complete. 2024-04-21T08:38:19.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.967+0000 7f22077ca740 20 client.15312 trim_cache size 0 max 16384 2024-04-21T08:38:19.973 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:38:19.973 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "d57a7d98-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:38:19.973 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-04-21T08:38:19.973 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24527, 2024-04-21T08:38:19.973 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-21T08:38:19.973 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-21T08:38:19.974 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-21T08:38:19.974 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 115.512657877, 2024-04-21T08:38:19.974 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 27, 2024-04-21T08:38:19.974 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 94, 2024-04-21T08:38:19.974 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 90, 2024-04-21T08:38:19.974 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 120.787709191 2024-04-21T08:38:19.974 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:38:19.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.967+0000 7f2204bd5640 1 -- 172.21.15.43:0/3319772348 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f21ec0a3c30 msgr2=0x7f21ec0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.967+0000 7f2204bd5640 1 --2- 172.21.15.43:0/3319772348 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f21ec0a3c30 0x7f21ec0a60f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f21f0010c40 tx=0x7f21f001b000 comp rx=0 tx=0).stop 2024-04-21T08:38:19.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.967+0000 7f2204bd5640 1 -- 172.21.15.43:0/3319772348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2200101fe0 msgr2=0x7f2200198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:19.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.967+0000 7f2204bd5640 1 --2- 172.21.15.43:0/3319772348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2200101fe0 0x7f2200198780 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f21e800d770 tx=0x7f21e800dc30 comp rx=0 tx=0).stop 2024-04-21T08:38:19.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.967+0000 7f2204bd5640 1 -- 172.21.15.43:0/3319772348 shutdown_connections 2024-04-21T08:38:19.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.967+0000 7f2204bd5640 1 --2- 172.21.15.43:0/3319772348 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f21ec0a3c30 0x7f21ec0a60f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.967+0000 7f2204bd5640 1 --2- 172.21.15.43:0/3319772348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22001029a0 0x7f220019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.967+0000 7f2204bd5640 1 --2- 172.21.15.43:0/3319772348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2200101fe0 0x7f2200198780 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.967+0000 7f2204bd5640 1 --2- 172.21.15.43:0/3319772348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2200100e90 0x7f2200198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:19.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.967+0000 7f2204bd5640 1 -- 172.21.15.43:0/3319772348 >> 172.21.15.43:0/3319772348 conn(0x7f22000fc400 msgr2=0x7f2200105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:19.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.967+0000 7f2204bd5640 1 -- 172.21.15.43:0/3319772348 shutdown_connections 2024-04-21T08:38:19.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:19.967+0000 7f2204bd5640 1 -- 172.21.15.43:0/3319772348 wait complete. 2024-04-21T08:38:19.978 INFO:tasks.cephadm:Setup complete, yielding 2024-04-21T08:38:19.979 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-21T08:38:19.987 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-04-21T08:38:19.988 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-21T08:38:20.235 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:20.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:20 smithi043 ceph-mon[27978]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-21T08:38:20.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:20 smithi043 ceph-mon[27978]: Cluster is now healthy 2024-04-21T08:38:20.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:20 smithi175 ceph-mon[31920]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-21T08:38:20.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:20 smithi175 ceph-mon[31920]: Cluster is now healthy 2024-04-21T08:38:20.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:20 smithi112 ceph-mon[31085]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-21T08:38:20.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:20 smithi112 ceph-mon[31085]: Cluster is now healthy 2024-04-21T08:38:21.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:21 smithi175 ceph-mon[31920]: pgmap v357: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s; 14 B/s, 0 objects/s recovering 2024-04-21T08:38:21.719 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:21 smithi043 ceph-mon[27978]: pgmap v357: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s; 14 B/s, 0 objects/s recovering 2024-04-21T08:38:21.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:21 smithi112 ceph-mon[31085]: pgmap v357: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s; 14 B/s, 0 objects/s recovering 2024-04-21T08:38:22.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.138+0000 7f184f754700 1 -- 172.21.15.43:0/2762087868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1848103540 msgr2=0x7f1848109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:22.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.138+0000 7f184f754700 1 --2- 172.21.15.43:0/2762087868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1848103540 0x7f1848109df0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f184000b0d0 tx=0x7f184000b3e0 comp rx=0 tx=0).stop 2024-04-21T08:38:22.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.138+0000 7f184f754700 1 -- 172.21.15.43:0/2762087868 shutdown_connections 2024-04-21T08:38:22.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.138+0000 7f184f754700 1 --2- 172.21.15.43:0/2762087868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18481019a0 0x7f1848101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:22.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.138+0000 7f184f754700 1 --2- 172.21.15.43:0/2762087868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1848103540 0x7f1848109df0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:22.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.138+0000 7f184f754700 1 --2- 172.21.15.43:0/2762087868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1848102ba0 0x7f1848103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:22.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.138+0000 7f184f754700 1 -- 172.21.15.43:0/2762087868 >> 172.21.15.43:0/2762087868 conn(0x7f18480fcf40 msgr2=0x7f18480ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:22.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.138+0000 7f184f754700 1 -- 172.21.15.43:0/2762087868 shutdown_connections 2024-04-21T08:38:22.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.138+0000 7f184f754700 1 -- 172.21.15.43:0/2762087868 wait complete. 2024-04-21T08:38:22.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.139+0000 7f184f754700 1 Processor -- start 2024-04-21T08:38:22.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.139+0000 7f184f754700 1 -- start start 2024-04-21T08:38:22.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.139+0000 7f184f754700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18481019a0 0x7f1848198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:22.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.139+0000 7f184f754700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1848102ba0 0x7f1848199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:22.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.139+0000 7f184f754700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1848103540 0x7f18481a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:22.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.139+0000 7f184f754700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f184810d420 con 0x7f1848102ba0 2024-04-21T08:38:22.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.139+0000 7f184f754700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f184810d2a0 con 0x7f1848103540 2024-04-21T08:38:22.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.139+0000 7f184f754700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f184810d5a0 con 0x7f18481019a0 2024-04-21T08:38:22.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.139+0000 7f184ccef700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1848102ba0 0x7f1848199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:22.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.139+0000 7f184ccef700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1848102ba0 0x7f1848199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55250/0 (socket says 172.21.15.43:55250) 2024-04-21T08:38:22.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.139+0000 7f184ccef700 1 -- 172.21.15.43:0/2945136174 learned_addr learned my addr 172.21.15.43:0/2945136174 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:22.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.140+0000 7f184dcf1700 1 --2- 172.21.15.43:0/2945136174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1848103540 0x7f18481a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:22.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.140+0000 7f184d4f0700 1 --2- 172.21.15.43:0/2945136174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18481019a0 0x7f1848198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:22.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.140+0000 7f184ccef700 1 -- 172.21.15.43:0/2945136174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18481019a0 msgr2=0x7f1848198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:22.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.140+0000 7f184ccef700 1 --2- 172.21.15.43:0/2945136174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18481019a0 0x7f1848198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:22.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.140+0000 7f184ccef700 1 -- 172.21.15.43:0/2945136174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1848103540 msgr2=0x7f18481a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:22.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.140+0000 7f184ccef700 1 --2- 172.21.15.43:0/2945136174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1848103540 0x7f18481a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:22.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.140+0000 7f184ccef700 1 -- 172.21.15.43:0/2945136174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18481a07b0 con 0x7f1848102ba0 2024-04-21T08:38:22.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.140+0000 7f184d4f0700 1 --2- 172.21.15.43:0/2945136174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18481019a0 0x7f1848198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:22.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.140+0000 7f184ccef700 1 --2- 172.21.15.43:0/2945136174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1848102ba0 0x7f1848199090 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f184400c950 tx=0x7f184400cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:22.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.140+0000 7f183e7fc700 1 -- 172.21.15.43:0/2945136174 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844007ac0 con 0x7f1848102ba0 2024-04-21T08:38:22.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.140+0000 7f184f754700 1 -- 172.21.15.43:0/2945136174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18481a0aa0 con 0x7f1848102ba0 2024-04-21T08:38:22.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.140+0000 7f183e7fc700 1 -- 172.21.15.43:0/2945136174 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1844004ce0 con 0x7f1848102ba0 2024-04-21T08:38:22.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.140+0000 7f183e7fc700 1 -- 172.21.15.43:0/2945136174 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844005550 con 0x7f1848102ba0 2024-04-21T08:38:22.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.140+0000 7f184f754700 1 -- 172.21.15.43:0/2945136174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18481a1030 con 0x7f1848102ba0 2024-04-21T08:38:22.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.141+0000 7f184f754700 1 -- 172.21.15.43:0/2945136174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1848069730 con 0x7f1848102ba0 2024-04-21T08:38:22.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.142+0000 7f183e7fc700 1 -- 172.21.15.43:0/2945136174 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1844007c60 con 0x7f1848102ba0 2024-04-21T08:38:22.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.143+0000 7f183e7fc700 1 --2- 172.21.15.43:0/2945136174 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f18340a3c40 0x7f18340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:22.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.143+0000 7f184d4f0700 1 --2- 172.21.15.43:0/2945136174 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f18340a3c40 0x7f18340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:22.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.143+0000 7f183e7fc700 1 -- 172.21.15.43:0/2945136174 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f18440d0810 con 0x7f1848102ba0 2024-04-21T08:38:22.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.143+0000 7f184d4f0700 1 --2- 172.21.15.43:0/2945136174 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f18340a3c40 0x7f18340a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1838005fd0 tx=0x7f1838005e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:22.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.147+0000 7f183e7fc700 1 -- 172.21.15.43:0/2945136174 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1844014030 con 0x7f1848102ba0 2024-04-21T08:38:22.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.373+0000 7f184f754700 1 -- 172.21.15.43:0/2945136174 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f18480ff0d0 con 0x7f18340a3c40 2024-04-21T08:38:22.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.374+0000 7f183e7fc700 1 -- 172.21.15.43:0/2945136174 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f18480ff0d0 con 0x7f18340a3c40 2024-04-21T08:38:22.374 INFO:teuthology.orchestra.run.smithi043.stdout:Backend: cephadm 2024-04-21T08:38:22.374 INFO:teuthology.orchestra.run.smithi043.stdout:Available: Yes 2024-04-21T08:38:22.374 INFO:teuthology.orchestra.run.smithi043.stdout:Paused: No 2024-04-21T08:38:22.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.376+0000 7f184f754700 1 -- 172.21.15.43:0/2945136174 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f18340a3c40 msgr2=0x7f18340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:22.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.376+0000 7f184f754700 1 --2- 172.21.15.43:0/2945136174 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f18340a3c40 0x7f18340a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1838005fd0 tx=0x7f1838005e40 comp rx=0 tx=0).stop 2024-04-21T08:38:22.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.376+0000 7f184f754700 1 -- 172.21.15.43:0/2945136174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1848102ba0 msgr2=0x7f1848199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:22.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.376+0000 7f184f754700 1 --2- 172.21.15.43:0/2945136174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1848102ba0 0x7f1848199090 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f184400c950 tx=0x7f184400cd10 comp rx=0 tx=0).stop 2024-04-21T08:38:22.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.377+0000 7f184f754700 1 -- 172.21.15.43:0/2945136174 shutdown_connections 2024-04-21T08:38:22.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.377+0000 7f184f754700 1 --2- 172.21.15.43:0/2945136174 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f18340a3c40 0x7f18340a6100 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:22.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.377+0000 7f184f754700 1 --2- 172.21.15.43:0/2945136174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18481019a0 0x7f1848198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:22.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.377+0000 7f184f754700 1 --2- 172.21.15.43:0/2945136174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1848103540 0x7f18481a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:22.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.377+0000 7f184f754700 1 --2- 172.21.15.43:0/2945136174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1848102ba0 0x7f1848199090 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:22.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.377+0000 7f184f754700 1 -- 172.21.15.43:0/2945136174 >> 172.21.15.43:0/2945136174 conn(0x7f18480fcf40 msgr2=0x7f18480fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:22.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.377+0000 7f184f754700 1 -- 172.21.15.43:0/2945136174 shutdown_connections 2024-04-21T08:38:22.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:22.377+0000 7f184f754700 1 -- 172.21.15.43:0/2945136174 wait complete. 2024-04-21T08:38:23.158 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-21T08:38:23.414 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:23.711 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:23 smithi043 ceph-mon[27978]: pgmap v358: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 10 op/s; 14 B/s, 0 objects/s recovering 2024-04-21T08:38:23.711 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:23 smithi043 ceph-mon[27978]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:23.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:23 smithi175 ceph-mon[31920]: pgmap v358: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 10 op/s; 14 B/s, 0 objects/s recovering 2024-04-21T08:38:23.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:23 smithi175 ceph-mon[31920]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:23.793 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:23 smithi112 ceph-mon[31085]: pgmap v358: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 10 op/s; 14 B/s, 0 objects/s recovering 2024-04-21T08:38:23.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:23 smithi112 ceph-mon[31085]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:25.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.184+0000 7f62f59dd700 1 -- 172.21.15.43:0/1014413095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62f0103560 msgr2=0x7f62f0109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:25.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.184+0000 7f62f59dd700 1 --2- 172.21.15.43:0/1014413095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62f0103560 0x7f62f0109e10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f62e800b0d0 tx=0x7f62e800b3e0 comp rx=0 tx=0).stop 2024-04-21T08:38:25.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.184+0000 7f62f59dd700 1 -- 172.21.15.43:0/1014413095 shutdown_connections 2024-04-21T08:38:25.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.184+0000 7f62f59dd700 1 --2- 172.21.15.43:0/1014413095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62f0103560 0x7f62f0109e10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:25.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.184+0000 7f62f59dd700 1 --2- 172.21.15.43:0/1014413095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62f01019c0 0x7f62f0101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:25.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.184+0000 7f62f59dd700 1 --2- 172.21.15.43:0/1014413095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62f0102bc0 0x7f62f0103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:25.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.184+0000 7f62f59dd700 1 -- 172.21.15.43:0/1014413095 >> 172.21.15.43:0/1014413095 conn(0x7f62f00fcf60 msgr2=0x7f62f00ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:25.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.184+0000 7f62f59dd700 1 -- 172.21.15.43:0/1014413095 shutdown_connections 2024-04-21T08:38:25.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.184+0000 7f62f59dd700 1 -- 172.21.15.43:0/1014413095 wait complete. 2024-04-21T08:38:25.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.185+0000 7f62f59dd700 1 Processor -- start 2024-04-21T08:38:25.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.185+0000 7f62f59dd700 1 -- start start 2024-04-21T08:38:25.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.185+0000 7f62f59dd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62f01019c0 0x7f62f006bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:25.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.185+0000 7f62f59dd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62f0102bc0 0x7f62f019d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:25.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.185+0000 7f62f59dd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62f0103560 0x7f62f019fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:25.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.185+0000 7f62f59dd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f62f010d490 con 0x7f62f0103560 2024-04-21T08:38:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.185+0000 7f62f59dd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f62f010d310 con 0x7f62f0102bc0 2024-04-21T08:38:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.185+0000 7f62f59dd700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f62f010d610 con 0x7f62f01019c0 2024-04-21T08:38:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.185+0000 7f62ef7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62f0103560 0x7f62f019fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.185+0000 7f62ef7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62f0103560 0x7f62f019fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55262/0 (socket says 172.21.15.43:55262) 2024-04-21T08:38:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.185+0000 7f62ef7fe700 1 -- 172.21.15.43:0/3620416818 learned_addr learned my addr 172.21.15.43:0/3620416818 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.186+0000 7f62ee7fc700 1 --2- 172.21.15.43:0/3620416818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62f0102bc0 0x7f62f019d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.186+0000 7f62eeffd700 1 --2- 172.21.15.43:0/3620416818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62f01019c0 0x7f62f006bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.186+0000 7f62ef7fe700 1 -- 172.21.15.43:0/3620416818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62f01019c0 msgr2=0x7f62f006bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.186+0000 7f62ef7fe700 1 --2- 172.21.15.43:0/3620416818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62f01019c0 0x7f62f006bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.186+0000 7f62ef7fe700 1 -- 172.21.15.43:0/3620416818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62f0102bc0 msgr2=0x7f62f019d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.186+0000 7f62ef7fe700 1 --2- 172.21.15.43:0/3620416818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62f0102bc0 0x7f62f019d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.186+0000 7f62ef7fe700 1 -- 172.21.15.43:0/3620416818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f01a0710 con 0x7f62f0103560 2024-04-21T08:38:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.186+0000 7f62ef7fe700 1 --2- 172.21.15.43:0/3620416818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62f0103560 0x7f62f019fe40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f62e8004c80 tx=0x7f62e8004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.186+0000 7f62f49db700 1 -- 172.21.15.43:0/3620416818 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e801f030 con 0x7f62f0103560 2024-04-21T08:38:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.186+0000 7f62f49db700 1 -- 172.21.15.43:0/3620416818 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62e8005040 con 0x7f62f0103560 2024-04-21T08:38:25.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.186+0000 7f62f59dd700 1 -- 172.21.15.43:0/3620416818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f01a09a0 con 0x7f62f0103560 2024-04-21T08:38:25.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.186+0000 7f62f49db700 1 -- 172.21.15.43:0/3620416818 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e801b690 con 0x7f62f0103560 2024-04-21T08:38:25.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.186+0000 7f62f59dd700 1 -- 172.21.15.43:0/3620416818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f01a0da0 con 0x7f62f0103560 2024-04-21T08:38:25.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.187+0000 7f62f59dd700 1 -- 172.21.15.43:0/3620416818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62f002df80 con 0x7f62f0103560 2024-04-21T08:38:25.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.188+0000 7f62f49db700 1 -- 172.21.15.43:0/3620416818 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f62e8022030 con 0x7f62f0103560 2024-04-21T08:38:25.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.189+0000 7f62f49db700 1 --2- 172.21.15.43:0/3620416818 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f62d80a3c40 0x7f62d80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:25.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.189+0000 7f62eeffd700 1 --2- 172.21.15.43:0/3620416818 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f62d80a3c40 0x7f62d80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:25.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.189+0000 7f62f49db700 1 -- 172.21.15.43:0/3620416818 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f62e80d11f0 con 0x7f62f0103560 2024-04-21T08:38:25.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.190+0000 7f62eeffd700 1 --2- 172.21.15.43:0/3620416818 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f62d80a3c40 0x7f62d80a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f62f0102a20 tx=0x7f62e000a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:25.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.193+0000 7f62f49db700 1 -- 172.21.15.43:0/3620416818 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f62e809d870 con 0x7f62f0103560 2024-04-21T08:38:25.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.412+0000 7f62f59dd700 1 -- 172.21.15.43:0/3620416818 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f62f0062e80 con 0x7f62d80a3c40 2024-04-21T08:38:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.426+0000 7f62f49db700 1 -- 172.21.15.43:0/3620416818 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f62f0062e80 con 0x7f62d80a3c40 2024-04-21T08:38:25.426 INFO:teuthology.orchestra.run.smithi043.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-21T08:38:25.426 INFO:teuthology.orchestra.run.smithi043.stdout:mds.a smithi043 running (2m) 22s ago 2m 17.6M - 19.0.0-3154-ga630765c b45a994450e2 22dc542087db 2024-04-21T08:38:25.426 INFO:teuthology.orchestra.run.smithi043.stdout:mds.b smithi112 running (2m) 60s ago 2m 28.4M - 19.0.0-3154-ga630765c b45a994450e2 bea88b79092d 2024-04-21T08:38:25.426 INFO:teuthology.orchestra.run.smithi043.stdout:mds.c smithi175 running (2m) 60s ago 2m 16.4M - 19.0.0-3154-ga630765c b45a994450e2 bc8e2184d8ab 2024-04-21T08:38:25.426 INFO:teuthology.orchestra.run.smithi043.stdout:mds.d smithi043 running (2m) 22s ago 2m 14.1M - 19.0.0-3154-ga630765c b45a994450e2 63a5476ed530 2024-04-21T08:38:25.427 INFO:teuthology.orchestra.run.smithi043.stdout:mds.e smithi112 running (2m) 60s ago 2m 18.7M - 19.0.0-3154-ga630765c b45a994450e2 cc359f26aee1 2024-04-21T08:38:25.427 INFO:teuthology.orchestra.run.smithi043.stdout:mds.f smithi175 running (2m) 60s ago 2m 19.8M - 19.0.0-3154-ga630765c b45a994450e2 9b7c904433b3 2024-04-21T08:38:25.427 INFO:teuthology.orchestra.run.smithi043.stdout:mds.g smithi043 running (2m) 22s ago 2m 17.7M - 19.0.0-3154-ga630765c b45a994450e2 a244b7e4086e 2024-04-21T08:38:25.427 INFO:teuthology.orchestra.run.smithi043.stdout:mds.h smithi112 running (2m) 60s ago 2m 20.3M - 19.0.0-3154-ga630765c b45a994450e2 74e1523b5639 2024-04-21T08:38:25.427 INFO:teuthology.orchestra.run.smithi043.stdout:mds.i smithi175 running (2m) 60s ago 2m 18.9M - 19.0.0-3154-ga630765c b45a994450e2 03ad24450f0c 2024-04-21T08:38:25.427 INFO:teuthology.orchestra.run.smithi043.stdout:mds.j smithi043 running (2m) 22s ago 2m 16.3M - 19.0.0-3154-ga630765c b45a994450e2 3d09652be3c8 2024-04-21T08:38:25.427 INFO:teuthology.orchestra.run.smithi043.stdout:mds.k smithi112 running (2m) 60s ago 2m 17.2M - 19.0.0-3154-ga630765c b45a994450e2 45235075818c 2024-04-21T08:38:25.427 INFO:teuthology.orchestra.run.smithi043.stdout:mds.l smithi175 running (2m) 60s ago 2m 19.9M - 19.0.0-3154-ga630765c b45a994450e2 7b28e9f65620 2024-04-21T08:38:25.427 INFO:teuthology.orchestra.run.smithi043.stdout:mgr.x smithi043 *:9283,8765 running (9m) 22s ago 9m 517M - 19.0.0-3154-ga630765c b45a994450e2 7477df00b6fb 2024-04-21T08:38:25.427 INFO:teuthology.orchestra.run.smithi043.stdout:mgr.y smithi112 *:8443,8765 running (7m) 60s ago 7m 454M - 19.0.0-3154-ga630765c b45a994450e2 905d62cd1f86 2024-04-21T08:38:25.427 INFO:teuthology.orchestra.run.smithi043.stdout:mgr.z smithi175 *:8443,8765 running (8m) 60s ago 8m 452M - 19.0.0-3154-ga630765c b45a994450e2 b7aab1eaed46 2024-04-21T08:38:25.427 INFO:teuthology.orchestra.run.smithi043.stdout:mon.a smithi043 running (10m) 22s ago 10m 50.3M 2048M 19.0.0-3154-ga630765c b45a994450e2 25b38f5f0834 2024-04-21T08:38:25.427 INFO:teuthology.orchestra.run.smithi043.stdout:mon.b smithi112 running (8m) 60s ago 8m 40.9M 2048M 19.0.0-3154-ga630765c b45a994450e2 86479c34e9f6 2024-04-21T08:38:25.427 INFO:teuthology.orchestra.run.smithi043.stdout:mon.c smithi175 running (8m) 60s ago 8m 39.8M 2048M 19.0.0-3154-ga630765c b45a994450e2 44beb75e459f 2024-04-21T08:38:25.427 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0 smithi043 running (7m) 22s ago 7m 53.3M 4273M 19.0.0-3154-ga630765c b45a994450e2 4c5fd87f8f4f 2024-04-21T08:38:25.427 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1 smithi112 running (7m) 60s ago 7m 57.2M 4272M 19.0.0-3154-ga630765c b45a994450e2 0afe0693ecae 2024-04-21T08:38:25.428 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2 smithi175 running (6m) 60s ago 6m 46.2M 4272M 19.0.0-3154-ga630765c b45a994450e2 0a0f70c3df28 2024-04-21T08:38:25.428 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3 smithi043 running (6m) 22s ago 6m 44.6M 4273M 19.0.0-3154-ga630765c b45a994450e2 656d0767e0ea 2024-04-21T08:38:25.428 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4 smithi112 running (6m) 60s ago 6m 45.7M 4272M 19.0.0-3154-ga630765c b45a994450e2 828ee7946705 2024-04-21T08:38:25.428 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5 smithi175 running (5m) 60s ago 5m 53.3M 4272M 19.0.0-3154-ga630765c b45a994450e2 db669708d411 2024-04-21T08:38:25.428 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6 smithi043 running (5m) 22s ago 5m 53.1M 4273M 19.0.0-3154-ga630765c b45a994450e2 6d2108af23d5 2024-04-21T08:38:25.428 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7 smithi112 running (4m) 60s ago 4m 44.9M 4272M 19.0.0-3154-ga630765c b45a994450e2 fe54368f5892 2024-04-21T08:38:25.428 INFO:teuthology.orchestra.run.smithi043.stdout:osd.8 smithi175 running (4m) 60s ago 4m 43.7M 4272M 19.0.0-3154-ga630765c b45a994450e2 72864771c994 2024-04-21T08:38:25.428 INFO:teuthology.orchestra.run.smithi043.stdout:osd.9 smithi043 running (3m) 22s ago 3m 51.2M 4273M 19.0.0-3154-ga630765c b45a994450e2 1dbf59ed06ab 2024-04-21T08:38:25.428 INFO:teuthology.orchestra.run.smithi043.stdout:osd.10 smithi112 running (3m) 60s ago 3m 53.2M 4272M 19.0.0-3154-ga630765c b45a994450e2 b835457b5eb9 2024-04-21T08:38:25.428 INFO:teuthology.orchestra.run.smithi043.stdout:osd.11 smithi175 running (2m) 60s ago 2m 41.9M 4272M 19.0.0-3154-ga630765c b45a994450e2 a4905d0f54e2 2024-04-21T08:38:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.428+0000 7f62f59dd700 1 -- 172.21.15.43:0/3620416818 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f62d80a3c40 msgr2=0x7f62d80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.428+0000 7f62f59dd700 1 --2- 172.21.15.43:0/3620416818 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f62d80a3c40 0x7f62d80a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f62f0102a20 tx=0x7f62e000a3b0 comp rx=0 tx=0).stop 2024-04-21T08:38:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.428+0000 7f62f59dd700 1 -- 172.21.15.43:0/3620416818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62f0103560 msgr2=0x7f62f019fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.428+0000 7f62f59dd700 1 --2- 172.21.15.43:0/3620416818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62f0103560 0x7f62f019fe40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f62e8004c80 tx=0x7f62e8004cb0 comp rx=0 tx=0).stop 2024-04-21T08:38:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.428+0000 7f62f59dd700 1 -- 172.21.15.43:0/3620416818 shutdown_connections 2024-04-21T08:38:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.428+0000 7f62f59dd700 1 --2- 172.21.15.43:0/3620416818 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f62d80a3c40 0x7f62d80a6100 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.428+0000 7f62f59dd700 1 --2- 172.21.15.43:0/3620416818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62f01019c0 0x7f62f006bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.428+0000 7f62f59dd700 1 --2- 172.21.15.43:0/3620416818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62f0103560 0x7f62f019fe40 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:25.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.428+0000 7f62f59dd700 1 --2- 172.21.15.43:0/3620416818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62f0102bc0 0x7f62f019d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:25.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.428+0000 7f62f59dd700 1 -- 172.21.15.43:0/3620416818 >> 172.21.15.43:0/3620416818 conn(0x7f62f00fcf60 msgr2=0x7f62f0105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:25.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.428+0000 7f62f59dd700 1 -- 172.21.15.43:0/3620416818 shutdown_connections 2024-04-21T08:38:25.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:25.428+0000 7f62f59dd700 1 -- 172.21.15.43:0/3620416818 wait complete. 2024-04-21T08:38:25.439 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:25 smithi043 ceph-mon[27978]: pgmap v359: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 951 B/s wr, 11 op/s; 12 B/s, 0 objects/s recovering 2024-04-21T08:38:25.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:25 smithi175 ceph-mon[31920]: pgmap v359: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 951 B/s wr, 11 op/s; 12 B/s, 0 objects/s recovering 2024-04-21T08:38:25.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:25 smithi112 ceph-mon[31085]: pgmap v359: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 951 B/s wr, 11 op/s; 12 B/s, 0 objects/s recovering 2024-04-21T08:38:26.143 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-21T08:38:26.401 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:26.412 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:26 smithi043 ceph-mon[27978]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:26.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:26 smithi175 ceph-mon[31920]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:26.793 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:26 smithi112 ceph-mon[31085]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:27.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:27 smithi043 ceph-mon[27978]: pgmap v360: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 11 op/s; 10 B/s, 0 objects/s recovering 2024-04-21T08:38:27.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:27 smithi175 ceph-mon[31920]: pgmap v360: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 11 op/s; 10 B/s, 0 objects/s recovering 2024-04-21T08:38:27.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:27 smithi112 ceph-mon[31085]: pgmap v360: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 11 op/s; 10 B/s, 0 objects/s recovering 2024-04-21T08:38:28.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.128+0000 7fcefb2a0700 1 -- 172.21.15.43:0/3048625647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef4104660 msgr2=0x7fcef410af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:28.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.128+0000 7fcefb2a0700 1 --2- 172.21.15.43:0/3048625647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef4104660 0x7fcef410af10 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fceec00b0d0 tx=0x7fceec00b3e0 comp rx=0 tx=0).stop 2024-04-21T08:38:28.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.128+0000 7fcefb2a0700 1 -- 172.21.15.43:0/3048625647 shutdown_connections 2024-04-21T08:38:28.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.128+0000 7fcefb2a0700 1 --2- 172.21.15.43:0/3048625647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcef4102ac0 0x7fcef4102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:28.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.128+0000 7fcefb2a0700 1 --2- 172.21.15.43:0/3048625647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef4104660 0x7fcef410af10 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:28.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.128+0000 7fcefb2a0700 1 --2- 172.21.15.43:0/3048625647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcef4103cc0 0x7fcef4104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:28.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.128+0000 7fcefb2a0700 1 -- 172.21.15.43:0/3048625647 >> 172.21.15.43:0/3048625647 conn(0x7fcef40fe060 msgr2=0x7fcef41004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:28.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.129+0000 7fcefb2a0700 1 -- 172.21.15.43:0/3048625647 shutdown_connections 2024-04-21T08:38:28.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.129+0000 7fcefb2a0700 1 -- 172.21.15.43:0/3048625647 wait complete. 2024-04-21T08:38:28.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.129+0000 7fcefb2a0700 1 Processor -- start 2024-04-21T08:38:28.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.129+0000 7fcefb2a0700 1 -- start start 2024-04-21T08:38:28.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.129+0000 7fcefb2a0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef4102ac0 0x7fcef40767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:28.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.129+0000 7fcefb2a0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcef4103cc0 0x7fcef4074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:28.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.129+0000 7fcefb2a0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcef4104660 0x7fcef4075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:28.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.129+0000 7fcefb2a0700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcef410e5d0 con 0x7fcef4102ac0 2024-04-21T08:38:28.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.129+0000 7fcefb2a0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcef410e450 con 0x7fcef4103cc0 2024-04-21T08:38:28.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.130+0000 7fcefb2a0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcef410e750 con 0x7fcef4104660 2024-04-21T08:38:28.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.130+0000 7fcef903c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef4102ac0 0x7fcef40767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:28.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.130+0000 7fcef903c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef4102ac0 0x7fcef40767e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33520/0 (socket says 172.21.15.43:33520) 2024-04-21T08:38:28.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.130+0000 7fcef903c700 1 -- 172.21.15.43:0/3645189024 learned_addr learned my addr 172.21.15.43:0/3645189024 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:28.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.130+0000 7fcef883b700 1 --2- 172.21.15.43:0/3645189024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcef4103cc0 0x7fcef4074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:28.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.130+0000 7fcef983d700 1 --2- 172.21.15.43:0/3645189024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcef4104660 0x7fcef4075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:28.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.130+0000 7fcef903c700 1 -- 172.21.15.43:0/3645189024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcef4104660 msgr2=0x7fcef4075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:28.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.130+0000 7fcef903c700 1 --2- 172.21.15.43:0/3645189024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcef4104660 0x7fcef4075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:28.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.130+0000 7fcef903c700 1 -- 172.21.15.43:0/3645189024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcef4103cc0 msgr2=0x7fcef4074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:28.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.130+0000 7fcef903c700 1 --2- 172.21.15.43:0/3645189024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcef4103cc0 0x7fcef4074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:28.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.130+0000 7fcef903c700 1 -- 172.21.15.43:0/3645189024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef4075c20 con 0x7fcef4102ac0 2024-04-21T08:38:28.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.130+0000 7fcef983d700 1 --2- 172.21.15.43:0/3645189024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcef4104660 0x7fcef4075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:28.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.130+0000 7fcef903c700 1 --2- 172.21.15.43:0/3645189024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef4102ac0 0x7fcef40767e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcef000b500 tx=0x7fcef000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:28.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.130+0000 7fceea7fc700 1 -- 172.21.15.43:0/3645189024 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef0015020 con 0x7fcef4102ac0 2024-04-21T08:38:28.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.130+0000 7fceea7fc700 1 -- 172.21.15.43:0/3645189024 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcef0011ce0 con 0x7fcef4102ac0 2024-04-21T08:38:28.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.130+0000 7fcefb2a0700 1 -- 172.21.15.43:0/3645189024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcef41a2ce0 con 0x7fcef4102ac0 2024-04-21T08:38:28.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.130+0000 7fceea7fc700 1 -- 172.21.15.43:0/3645189024 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef0010d20 con 0x7fcef4102ac0 2024-04-21T08:38:28.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.131+0000 7fcefb2a0700 1 -- 172.21.15.43:0/3645189024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcef41a31a0 con 0x7fcef4102ac0 2024-04-21T08:38:28.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.131+0000 7fcefb2a0700 1 -- 172.21.15.43:0/3645189024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcef4069730 con 0x7fcef4102ac0 2024-04-21T08:38:28.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.133+0000 7fceea7fc700 1 -- 172.21.15.43:0/3645189024 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcef0018020 con 0x7fcef4102ac0 2024-04-21T08:38:28.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.133+0000 7fceea7fc700 1 --2- 172.21.15.43:0/3645189024 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fcee00a3c40 0x7fcee00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:28.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.133+0000 7fcef883b700 1 --2- 172.21.15.43:0/3645189024 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fcee00a3c40 0x7fcee00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:28.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.133+0000 7fceea7fc700 1 -- 172.21.15.43:0/3645189024 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fcef00d04b0 con 0x7fcef4102ac0 2024-04-21T08:38:28.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.134+0000 7fcef883b700 1 --2- 172.21.15.43:0/3645189024 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fcee00a3c40 0x7fcee00a6100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcee400ac50 tx=0x7fcee400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:28.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.137+0000 7fceea7fc700 1 -- 172.21.15.43:0/3645189024 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcef001f020 con 0x7fcef4102ac0 2024-04-21T08:38:28.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.356+0000 7fcefb2a0700 1 -- 172.21.15.43:0/3645189024 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fcef4062e80 con 0x7fcee00a3c40 2024-04-21T08:38:28.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.362+0000 7fceea7fc700 1 -- 172.21.15.43:0/3645189024 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fcef4062e80 con 0x7fcee00a3c40 2024-04-21T08:38:28.361 INFO:teuthology.orchestra.run.smithi043.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-21T08:38:28.361 INFO:teuthology.orchestra.run.smithi043.stdout:mds.all 12/12 63s ago 2m smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l;count:12 2024-04-21T08:38:28.361 INFO:teuthology.orchestra.run.smithi043.stdout:mgr 3/3 63s ago 8m smithi043=x;smithi112=y;smithi175=z;count:3 2024-04-21T08:38:28.361 INFO:teuthology.orchestra.run.smithi043.stdout:mon 3/3 63s ago 8m smithi043:172.21.15.43=a;smithi112:172.21.15.112=b;smithi175:172.21.15.175=c;count:3 2024-04-21T08:38:28.362 INFO:teuthology.orchestra.run.smithi043.stdout:osd 12 63s ago - 2024-04-21T08:38:28.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.364+0000 7fcefb2a0700 1 -- 172.21.15.43:0/3645189024 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fcee00a3c40 msgr2=0x7fcee00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:28.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.364+0000 7fcefb2a0700 1 --2- 172.21.15.43:0/3645189024 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fcee00a3c40 0x7fcee00a6100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcee400ac50 tx=0x7fcee400a380 comp rx=0 tx=0).stop 2024-04-21T08:38:28.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.364+0000 7fcefb2a0700 1 -- 172.21.15.43:0/3645189024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef4102ac0 msgr2=0x7fcef40767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:28.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.364+0000 7fcefb2a0700 1 --2- 172.21.15.43:0/3645189024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef4102ac0 0x7fcef40767e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcef000b500 tx=0x7fcef000b810 comp rx=0 tx=0).stop 2024-04-21T08:38:28.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.364+0000 7fcefb2a0700 1 -- 172.21.15.43:0/3645189024 shutdown_connections 2024-04-21T08:38:28.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.364+0000 7fcefb2a0700 1 --2- 172.21.15.43:0/3645189024 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fcee00a3c40 0x7fcee00a6100 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:28.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.364+0000 7fcefb2a0700 1 --2- 172.21.15.43:0/3645189024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcef4104660 0x7fcef4075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:28.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.364+0000 7fcefb2a0700 1 --2- 172.21.15.43:0/3645189024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef4102ac0 0x7fcef40767e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:28.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.364+0000 7fcefb2a0700 1 --2- 172.21.15.43:0/3645189024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcef4103cc0 0x7fcef4074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:28.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.364+0000 7fcefb2a0700 1 -- 172.21.15.43:0/3645189024 >> 172.21.15.43:0/3645189024 conn(0x7fcef40fe060 msgr2=0x7fcef4106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:28.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.364+0000 7fcefb2a0700 1 -- 172.21.15.43:0/3645189024 shutdown_connections 2024-04-21T08:38:28.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:28.364+0000 7fcefb2a0700 1 -- 172.21.15.43:0/3645189024 wait complete. 2024-04-21T08:38:29.075 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-21T08:38:29.325 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:29.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:29 smithi175 ceph-mon[31920]: pgmap v361: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s; 9 B/s, 0 objects/s recovering 2024-04-21T08:38:29.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:29 smithi175 ceph-mon[31920]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:29 smithi043 ceph-mon[27978]: pgmap v361: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s; 9 B/s, 0 objects/s recovering 2024-04-21T08:38:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:29 smithi043 ceph-mon[27978]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:29.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:29 smithi112 ceph-mon[31085]: pgmap v361: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s; 9 B/s, 0 objects/s recovering 2024-04-21T08:38:29.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:29 smithi112 ceph-mon[31085]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:31.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.018+0000 7f93d78c1700 1 -- 172.21.15.43:0/2956021893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0102bd0 msgr2=0x7f93d0103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:31.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.018+0000 7f93d78c1700 1 --2- 172.21.15.43:0/2956021893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0102bd0 0x7f93d0103050 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f93c4009990 tx=0x7f93c4009ca0 comp rx=0 tx=0).stop 2024-04-21T08:38:31.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.018+0000 7f93d78c1700 1 -- 172.21.15.43:0/2956021893 shutdown_connections 2024-04-21T08:38:31.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.018+0000 7f93d78c1700 1 --2- 172.21.15.43:0/2956021893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93d0101eb0 0x7f93d01022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:31.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.018+0000 7f93d78c1700 1 --2- 172.21.15.43:0/2956021893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93d0103590 0x7f93d0109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:31.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.018+0000 7f93d78c1700 1 --2- 172.21.15.43:0/2956021893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0102bd0 0x7f93d0103050 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:31.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.018+0000 7f93d78c1700 1 -- 172.21.15.43:0/2956021893 >> 172.21.15.43:0/2956021893 conn(0x7f93d00fc6d0 msgr2=0x7f93d00feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:31.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.018+0000 7f93d78c1700 1 -- 172.21.15.43:0/2956021893 shutdown_connections 2024-04-21T08:38:31.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.018+0000 7f93d78c1700 1 -- 172.21.15.43:0/2956021893 wait complete. 2024-04-21T08:38:31.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.019+0000 7f93d78c1700 1 Processor -- start 2024-04-21T08:38:31.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.019+0000 7f93d78c1700 1 -- start start 2024-04-21T08:38:31.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.019+0000 7f93d78c1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0101eb0 0x7f93d0198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:31.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.019+0000 7f93d78c1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93d0102bd0 0x7f93d0199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.019+0000 7f93d78c1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93d0103590 0x7f93d01a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.019+0000 7f93d78c1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f93d010d440 con 0x7f93d0101eb0 2024-04-21T08:38:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.019+0000 7f93d78c1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f93d010d2c0 con 0x7f93d0102bd0 2024-04-21T08:38:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.019+0000 7f93d78c1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f93d010d5c0 con 0x7f93d0103590 2024-04-21T08:38:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.019+0000 7f93d565d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0101eb0 0x7f93d0198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.019+0000 7f93d565d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0101eb0 0x7f93d0198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33536/0 (socket says 172.21.15.43:33536) 2024-04-21T08:38:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.019+0000 7f93d565d700 1 -- 172.21.15.43:0/1002654657 learned_addr learned my addr 172.21.15.43:0/1002654657 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.019+0000 7f93d5e5e700 1 --2- 172.21.15.43:0/1002654657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93d0103590 0x7f93d01a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.019+0000 7f93d4e5c700 1 --2- 172.21.15.43:0/1002654657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93d0102bd0 0x7f93d0199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.020+0000 7f93d565d700 1 -- 172.21.15.43:0/1002654657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93d0103590 msgr2=0x7f93d01a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.020+0000 7f93d565d700 1 --2- 172.21.15.43:0/1002654657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93d0103590 0x7f93d01a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.020+0000 7f93d565d700 1 -- 172.21.15.43:0/1002654657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93d0102bd0 msgr2=0x7f93d0199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.020+0000 7f93d565d700 1 --2- 172.21.15.43:0/1002654657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93d0102bd0 0x7f93d0199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.020+0000 7f93d565d700 1 -- 172.21.15.43:0/1002654657 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d01a0820 con 0x7f93d0101eb0 2024-04-21T08:38:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.020+0000 7f93d565d700 1 --2- 172.21.15.43:0/1002654657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0101eb0 0x7f93d0198bc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f93c800add0 tx=0x7f93c800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:31.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.020+0000 7f93ce7fc700 1 -- 172.21.15.43:0/1002654657 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8017070 con 0x7f93d0101eb0 2024-04-21T08:38:31.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.020+0000 7f93d78c1700 1 -- 172.21.15.43:0/1002654657 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d01a0b10 con 0x7f93d0101eb0 2024-04-21T08:38:31.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.020+0000 7f93ce7fc700 1 -- 172.21.15.43:0/1002654657 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93c8004bb0 con 0x7f93d0101eb0 2024-04-21T08:38:31.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.020+0000 7f93ce7fc700 1 -- 172.21.15.43:0/1002654657 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c800ce20 con 0x7f93d0101eb0 2024-04-21T08:38:31.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.020+0000 7f93d78c1700 1 -- 172.21.15.43:0/1002654657 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d01a1000 con 0x7f93d0101eb0 2024-04-21T08:38:31.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.021+0000 7f93d78c1700 1 -- 172.21.15.43:0/1002654657 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93d0069730 con 0x7f93d0101eb0 2024-04-21T08:38:31.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.022+0000 7f93ce7fc700 1 -- 172.21.15.43:0/1002654657 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f93c801a070 con 0x7f93d0101eb0 2024-04-21T08:38:31.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.023+0000 7f93ce7fc700 1 --2- 172.21.15.43:0/1002654657 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f93b80a3c40 0x7f93b80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:31.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.023+0000 7f93d4e5c700 1 --2- 172.21.15.43:0/1002654657 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f93b80a3c40 0x7f93b80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:31.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.023+0000 7f93ce7fc700 1 -- 172.21.15.43:0/1002654657 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f93c80cf880 con 0x7f93d0101eb0 2024-04-21T08:38:31.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.023+0000 7f93d4e5c700 1 --2- 172.21.15.43:0/1002654657 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f93b80a3c40 0x7f93b80a6100 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f93c4009750 tx=0x7f93c400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:31.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.027+0000 7f93ce7fc700 1 -- 172.21.15.43:0/1002654657 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f93c809be70 con 0x7f93d0101eb0 2024-04-21T08:38:31.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.241+0000 7f93d78c1700 1 -- 172.21.15.43:0/1002654657 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f93d00fe8b0 con 0x7f93b80a3c40 2024-04-21T08:38:31.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.242+0000 7f93ce7fc700 1 -- 172.21.15.43:0/1002654657 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f93d00fe8b0 con 0x7f93b80a3c40 2024-04-21T08:38:31.241 INFO:teuthology.orchestra.run.smithi043.stdout:HOST ADDR LABELS STATUS 2024-04-21T08:38:31.241 INFO:teuthology.orchestra.run.smithi043.stdout:smithi043 172.21.15.43 2024-04-21T08:38:31.242 INFO:teuthology.orchestra.run.smithi043.stdout:smithi112 172.21.15.112 2024-04-21T08:38:31.242 INFO:teuthology.orchestra.run.smithi043.stdout:smithi175 172.21.15.175 2024-04-21T08:38:31.242 INFO:teuthology.orchestra.run.smithi043.stdout:3 hosts in cluster 2024-04-21T08:38:31.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.244+0000 7f93d78c1700 1 -- 172.21.15.43:0/1002654657 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f93b80a3c40 msgr2=0x7f93b80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:31.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.244+0000 7f93d78c1700 1 --2- 172.21.15.43:0/1002654657 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f93b80a3c40 0x7f93b80a6100 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f93c4009750 tx=0x7f93c400b540 comp rx=0 tx=0).stop 2024-04-21T08:38:31.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.244+0000 7f93d78c1700 1 -- 172.21.15.43:0/1002654657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0101eb0 msgr2=0x7f93d0198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:31.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.244+0000 7f93d78c1700 1 --2- 172.21.15.43:0/1002654657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0101eb0 0x7f93d0198bc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f93c800add0 tx=0x7f93c800c330 comp rx=0 tx=0).stop 2024-04-21T08:38:31.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.244+0000 7f93d78c1700 1 -- 172.21.15.43:0/1002654657 shutdown_connections 2024-04-21T08:38:31.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.244+0000 7f93d78c1700 1 --2- 172.21.15.43:0/1002654657 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f93b80a3c40 0x7f93b80a6100 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:31.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.244+0000 7f93d78c1700 1 --2- 172.21.15.43:0/1002654657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93d0103590 0x7f93d01a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:31.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.244+0000 7f93d78c1700 1 --2- 172.21.15.43:0/1002654657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0101eb0 0x7f93d0198bc0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:31.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.244+0000 7f93d78c1700 1 --2- 172.21.15.43:0/1002654657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93d0102bd0 0x7f93d0199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:31.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.244+0000 7f93d78c1700 1 -- 172.21.15.43:0/1002654657 >> 172.21.15.43:0/1002654657 conn(0x7f93d00fc6d0 msgr2=0x7f93d00fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:31.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.244+0000 7f93d78c1700 1 -- 172.21.15.43:0/1002654657 shutdown_connections 2024-04-21T08:38:31.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:31.244+0000 7f93d78c1700 1 -- 172.21.15.43:0/1002654657 wait complete. 2024-04-21T08:38:31.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:31 smithi175 ceph-mon[31920]: pgmap v362: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s; 9 B/s, 0 objects/s recovering 2024-04-21T08:38:31.720 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:31 smithi043 ceph-mon[27978]: pgmap v362: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s; 9 B/s, 0 objects/s recovering 2024-04-21T08:38:31.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:31 smithi112 ceph-mon[31085]: pgmap v362: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s; 9 B/s, 0 objects/s recovering 2024-04-21T08:38:32.041 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-21T08:38:32.291 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:32.664 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:32 smithi043 ceph-mon[27978]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:32.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:32 smithi175 ceph-mon[31920]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:32.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:32 smithi112 ceph-mon[31085]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:33.712 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:33 smithi043 ceph-mon[27978]: pgmap v363: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:38:33.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:33 smithi175 ceph-mon[31920]: pgmap v363: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:38:33.793 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:33 smithi112 ceph-mon[31085]: pgmap v363: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:38:33.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.963+0000 7f9bb1caf700 1 -- 172.21.15.43:0/3440347886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bac1087b0 msgr2=0x7f9bac10ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:33.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.964+0000 7f9bb1caf700 1 --2- 172.21.15.43:0/3440347886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bac1087b0 0x7f9bac10ac20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9ba400b0a0 tx=0x7f9ba400b3b0 comp rx=0 tx=0).stop 2024-04-21T08:38:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.964+0000 7f9bb1caf700 1 -- 172.21.15.43:0/3440347886 shutdown_connections 2024-04-21T08:38:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.964+0000 7f9bb1caf700 1 --2- 172.21.15.43:0/3440347886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bac1087b0 0x7f9bac10ac20 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.964+0000 7f9bb1caf700 1 --2- 172.21.15.43:0/3440347886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bac103550 0x7f9bac105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.964+0000 7f9bb1caf700 1 --2- 172.21.15.43:0/3440347886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bac105e80 0x7f9bac108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.964+0000 7f9bb1caf700 1 -- 172.21.15.43:0/3440347886 >> 172.21.15.43:0/3440347886 conn(0x7f9bac0fcf60 msgr2=0x7f9bac0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.964+0000 7f9bb1caf700 1 -- 172.21.15.43:0/3440347886 shutdown_connections 2024-04-21T08:38:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.964+0000 7f9bb1caf700 1 -- 172.21.15.43:0/3440347886 wait complete. 2024-04-21T08:38:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.964+0000 7f9bb1caf700 1 Processor -- start 2024-04-21T08:38:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.964+0000 7f9bb1caf700 1 -- start start 2024-04-21T08:38:33.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.965+0000 7f9bb1caf700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bac103550 0x7f9bac198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:33.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.965+0000 7f9bb1caf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bac105e80 0x7f9bac1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:33.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.965+0000 7f9bb1caf700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bac1087b0 0x7f9bac1a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:33.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.965+0000 7f9bb1caf700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac10e5a0 con 0x7f9bac103550 2024-04-21T08:38:33.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.965+0000 7f9bb1caf700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac10e420 con 0x7f9bac105e80 2024-04-21T08:38:33.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.965+0000 7f9bb1caf700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac10e720 con 0x7f9bac1087b0 2024-04-21T08:38:33.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.965+0000 7f9bab7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bac103550 0x7f9bac198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:33.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.965+0000 7f9bab7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bac103550 0x7f9bac198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33562/0 (socket says 172.21.15.43:33562) 2024-04-21T08:38:33.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.965+0000 7f9bab7fe700 1 -- 172.21.15.43:0/1186211781 learned_addr learned my addr 172.21.15.43:0/1186211781 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:33.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.965+0000 7f9baaffd700 1 --2- 172.21.15.43:0/1186211781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bac105e80 0x7f9bac1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:33.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.965+0000 7f9babfff700 1 --2- 172.21.15.43:0/1186211781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bac1087b0 0x7f9bac1a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:33.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.965+0000 7f9bab7fe700 1 -- 172.21.15.43:0/1186211781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bac1087b0 msgr2=0x7f9bac1a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:33.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.965+0000 7f9bab7fe700 1 --2- 172.21.15.43:0/1186211781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bac1087b0 0x7f9bac1a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:33.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.965+0000 7f9bab7fe700 1 -- 172.21.15.43:0/1186211781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bac105e80 msgr2=0x7f9bac1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:33.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.965+0000 7f9bab7fe700 1 --2- 172.21.15.43:0/1186211781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bac105e80 0x7f9bac1991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:33.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.965+0000 7f9bab7fe700 1 -- 172.21.15.43:0/1186211781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bac1a0840 con 0x7f9bac103550 2024-04-21T08:38:33.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.966+0000 7f9bab7fe700 1 --2- 172.21.15.43:0/1186211781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bac103550 0x7f9bac198c80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000e9d0 tx=0x7f9ba000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:33.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.966+0000 7f9ba8ff9700 1 -- 172.21.15.43:0/1186211781 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba000cbd0 con 0x7f9bac103550 2024-04-21T08:38:33.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.966+0000 7f9ba8ff9700 1 -- 172.21.15.43:0/1186211781 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ba000cd70 con 0x7f9bac103550 2024-04-21T08:38:33.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.966+0000 7f9bb1caf700 1 -- 172.21.15.43:0/1186211781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bac1a0b30 con 0x7f9bac103550 2024-04-21T08:38:33.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.966+0000 7f9ba8ff9700 1 -- 172.21.15.43:0/1186211781 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0018680 con 0x7f9bac103550 2024-04-21T08:38:33.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.966+0000 7f9bb1caf700 1 -- 172.21.15.43:0/1186211781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bac1a10c0 con 0x7f9bac103550 2024-04-21T08:38:33.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.967+0000 7f9bb1caf700 1 -- 172.21.15.43:0/1186211781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bac069730 con 0x7f9bac103550 2024-04-21T08:38:33.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.968+0000 7f9ba8ff9700 1 -- 172.21.15.43:0/1186211781 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9ba0010430 con 0x7f9bac103550 2024-04-21T08:38:33.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.969+0000 7f9ba8ff9700 1 --2- 172.21.15.43:0/1186211781 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9b940a3cc0 0x7f9b940a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:33.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.969+0000 7f9baaffd700 1 --2- 172.21.15.43:0/1186211781 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9b940a3cc0 0x7f9b940a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:33.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.969+0000 7f9ba8ff9700 1 -- 172.21.15.43:0/1186211781 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f9ba00d0e20 con 0x7f9bac103550 2024-04-21T08:38:33.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.969+0000 7f9baaffd700 1 --2- 172.21.15.43:0/1186211781 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9b940a3cc0 0x7f9b940a6180 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c009a10 tx=0x7f9b9c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:33.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:33.972+0000 7f9ba8ff9700 1 -- 172.21.15.43:0/1186211781 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ba009d410 con 0x7f9bac103550 2024-04-21T08:38:34.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:34.191+0000 7f9bb1caf700 1 -- 172.21.15.43:0/1186211781 --> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f9bac062e80 con 0x7f9b940a3cc0 2024-04-21T08:38:34.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:34.193+0000 7f9ba8ff9700 1 -- 172.21.15.43:0/1186211781 <== mgr.14150 v2:172.21.15.43:6800/1577030772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f9bac062e80 con 0x7f9b940a3cc0 2024-04-21T08:38:34.192 INFO:teuthology.orchestra.run.smithi043.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-21T08:38:34.192 INFO:teuthology.orchestra.run.smithi043.stdout:smithi043 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300GH480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-21T08:38:34.192 INFO:teuthology.orchestra.run.smithi043.stdout:smithi112 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NV400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-21T08:38:34.192 INFO:teuthology.orchestra.run.smithi043.stdout:smithi175 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300M0400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-21T08:38:34.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:34.195+0000 7f9bb1caf700 1 -- 172.21.15.43:0/1186211781 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9b940a3cc0 msgr2=0x7f9b940a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:34.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:34.195+0000 7f9bb1caf700 1 --2- 172.21.15.43:0/1186211781 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9b940a3cc0 0x7f9b940a6180 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c009a10 tx=0x7f9b9c009190 comp rx=0 tx=0).stop 2024-04-21T08:38:34.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:34.195+0000 7f9bb1caf700 1 -- 172.21.15.43:0/1186211781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bac103550 msgr2=0x7f9bac198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:34.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:34.195+0000 7f9bb1caf700 1 --2- 172.21.15.43:0/1186211781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bac103550 0x7f9bac198c80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000e9d0 tx=0x7f9ba000ece0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:34.195+0000 7f9bb1caf700 1 -- 172.21.15.43:0/1186211781 shutdown_connections 2024-04-21T08:38:34.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:34.195+0000 7f9bb1caf700 1 --2- 172.21.15.43:0/1186211781 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f9b940a3cc0 0x7f9b940a6180 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:34.195+0000 7f9bb1caf700 1 --2- 172.21.15.43:0/1186211781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9bac1087b0 0x7f9bac1a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:34.195+0000 7f9bb1caf700 1 --2- 172.21.15.43:0/1186211781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bac103550 0x7f9bac198c80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:34.195+0000 7f9bb1caf700 1 --2- 172.21.15.43:0/1186211781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bac105e80 0x7f9bac1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:34.195+0000 7f9bb1caf700 1 -- 172.21.15.43:0/1186211781 >> 172.21.15.43:0/1186211781 conn(0x7f9bac0fcf60 msgr2=0x7f9bac103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:34.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:34.195+0000 7f9bb1caf700 1 -- 172.21.15.43:0/1186211781 shutdown_connections 2024-04-21T08:38:34.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:34.195+0000 7f9bb1caf700 1 -- 172.21.15.43:0/1186211781 wait complete. 2024-04-21T08:38:34.902 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-21T08:38:34.911 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-04-21T08:38:34.911 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-21T08:38:35.168 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:35.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:35 smithi043 ceph-mon[27978]: pgmap v364: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-21T08:38:35.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:35 smithi043 ceph-mon[27978]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:35.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:35 smithi175 ceph-mon[31920]: pgmap v364: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-21T08:38:35.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:35 smithi175 ceph-mon[31920]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:35.793 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:35 smithi112 ceph-mon[31085]: pgmap v364: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-21T08:38:35.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:35 smithi112 ceph-mon[31085]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:36.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.995+0000 7fafd325b700 1 -- 172.21.15.43:0/2099605387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafcc103ca0 msgr2=0x7fafcc104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:36.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.995+0000 7fafd325b700 1 --2- 172.21.15.43:0/2099605387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafcc103ca0 0x7fafcc104100 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fafb80099c0 tx=0x7fafb8009cd0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.995+0000 7fafd325b700 1 -- 172.21.15.43:0/2099605387 shutdown_connections 2024-04-21T08:38:36.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.995+0000 7fafd325b700 1 --2- 172.21.15.43:0/2099605387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafcc102aa0 0x7fafcc102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.995+0000 7fafd325b700 1 --2- 172.21.15.43:0/2099605387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafcc104640 0x7fafcc10aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.995+0000 7fafd325b700 1 --2- 172.21.15.43:0/2099605387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafcc103ca0 0x7fafcc104100 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.995+0000 7fafd325b700 1 -- 172.21.15.43:0/2099605387 >> 172.21.15.43:0/2099605387 conn(0x7fafcc0fe040 msgr2=0x7fafcc100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:36.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.995+0000 7fafd325b700 1 -- 172.21.15.43:0/2099605387 shutdown_connections 2024-04-21T08:38:36.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.995+0000 7fafd325b700 1 -- 172.21.15.43:0/2099605387 wait complete. 2024-04-21T08:38:36.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.996+0000 7fafd325b700 1 Processor -- start 2024-04-21T08:38:36.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.996+0000 7fafd325b700 1 -- start start 2024-04-21T08:38:36.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.996+0000 7fafd325b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafcc102aa0 0x7fafcc0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:36.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.996+0000 7fafd325b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafcc103ca0 0x7fafcc074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:36.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.996+0000 7fafd325b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafcc104640 0x7fafcc075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:36.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.996+0000 7fafd325b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc10e510 con 0x7fafcc104640 2024-04-21T08:38:36.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.996+0000 7fafd325b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc10e390 con 0x7fafcc102aa0 2024-04-21T08:38:36.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.996+0000 7fafd325b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc10e690 con 0x7fafcc103ca0 2024-04-21T08:38:36.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.996+0000 7fafd17f8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafcc104640 0x7fafcc075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:36.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.996+0000 7fafd17f8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafcc104640 0x7fafcc075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35848/0 (socket says 172.21.15.43:35848) 2024-04-21T08:38:36.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.996+0000 7fafd17f8700 1 -- 172.21.15.43:0/3516129049 learned_addr learned my addr 172.21.15.43:0/3516129049 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:36.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.996+0000 7fafd0ff7700 1 --2- 172.21.15.43:0/3516129049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafcc102aa0 0x7fafcc0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:36.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.997+0000 7fafcbfff700 1 --2- 172.21.15.43:0/3516129049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafcc103ca0 0x7fafcc074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:36.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.997+0000 7fafd17f8700 1 -- 172.21.15.43:0/3516129049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafcc103ca0 msgr2=0x7fafcc074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:36.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.997+0000 7fafd17f8700 1 --2- 172.21.15.43:0/3516129049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafcc103ca0 0x7fafcc074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.997+0000 7fafd17f8700 1 -- 172.21.15.43:0/3516129049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafcc102aa0 msgr2=0x7fafcc0767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:36.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.997+0000 7fafd17f8700 1 --2- 172.21.15.43:0/3516129049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafcc102aa0 0x7fafcc0767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.997+0000 7fafd17f8700 1 -- 172.21.15.43:0/3516129049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafcc075c20 con 0x7fafcc104640 2024-04-21T08:38:36.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.997+0000 7fafd17f8700 1 --2- 172.21.15.43:0/3516129049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafcc104640 0x7fafcc075370 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fafc4007ee0 tx=0x7fafc400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:36.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.997+0000 7fafc9ffb700 1 -- 172.21.15.43:0/3516129049 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc400dbd0 con 0x7fafcc104640 2024-04-21T08:38:36.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.997+0000 7fafc9ffb700 1 -- 172.21.15.43:0/3516129049 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafc400dd70 con 0x7fafcc104640 2024-04-21T08:38:36.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.997+0000 7fafd325b700 1 -- 172.21.15.43:0/3516129049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafcc1a2d50 con 0x7fafcc104640 2024-04-21T08:38:36.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.997+0000 7fafc9ffb700 1 -- 172.21.15.43:0/3516129049 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4005990 con 0x7fafcc104640 2024-04-21T08:38:36.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.998+0000 7fafd325b700 1 -- 172.21.15.43:0/3516129049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafcc1a3200 con 0x7fafcc104640 2024-04-21T08:38:37.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:36.998+0000 7fafd325b700 1 -- 172.21.15.43:0/3516129049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafcc069730 con 0x7fafcc104640 2024-04-21T08:38:37.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:37.000+0000 7fafc9ffb700 1 -- 172.21.15.43:0/3516129049 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fafc401a020 con 0x7fafcc104640 2024-04-21T08:38:37.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:37.001+0000 7fafc9ffb700 1 --2- 172.21.15.43:0/3516129049 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fafbc0a3c40 0x7fafbc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:37.001+0000 7fafd0ff7700 1 --2- 172.21.15.43:0/3516129049 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fafbc0a3c40 0x7fafbc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:37.001+0000 7fafc9ffb700 1 -- 172.21.15.43:0/3516129049 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fafc40cff90 con 0x7fafcc104640 2024-04-21T08:38:37.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:37.001+0000 7fafd0ff7700 1 --2- 172.21.15.43:0/3516129049 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fafbc0a3c40 0x7fafbc0a6100 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fafcc103b00 tx=0x7fafc000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:37.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:37.005+0000 7fafc9ffb700 1 -- 172.21.15.43:0/3516129049 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fafc4010070 con 0x7fafcc104640 2024-04-21T08:38:37.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:37.260+0000 7fafd325b700 1 -- 172.21.15.43:0/3516129049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fafcc063650 con 0x7fafcc104640 2024-04-21T08:38:37.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:37.260+0000 7fafc9ffb700 1 -- 172.21.15.43:0/3516129049 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3096 (secure 0 0 0) 0x7fafc409c580 con 0x7fafcc104640 2024-04-21T08:38:37.260 INFO:teuthology.orchestra.run.smithi043.stdout:e27 2024-04-21T08:38:37.260 INFO:teuthology.orchestra.run.smithi043.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-21T08:38:37.260 INFO:teuthology.orchestra.run.smithi043.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-21T08:38:37.260 INFO:teuthology.orchestra.run.smithi043.stdout:legacy client fscid: 1 2024-04-21T08:38:37.260 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:37.260 INFO:teuthology.orchestra.run.smithi043.stdout:Filesystem 'cephfs' (1) 2024-04-21T08:38:37.260 INFO:teuthology.orchestra.run.smithi043.stdout:fs_name cephfs 2024-04-21T08:38:37.260 INFO:teuthology.orchestra.run.smithi043.stdout:epoch 27 2024-04-21T08:38:37.260 INFO:teuthology.orchestra.run.smithi043.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-21T08:38:37.260 INFO:teuthology.orchestra.run.smithi043.stdout:created 2024-04-21T08:36:23.821253+0000 2024-04-21T08:38:37.260 INFO:teuthology.orchestra.run.smithi043.stdout:modified 2024-04-21T08:36:51.902455+0000 2024-04-21T08:38:37.260 INFO:teuthology.orchestra.run.smithi043.stdout:tableserver 0 2024-04-21T08:38:37.261 INFO:teuthology.orchestra.run.smithi043.stdout:root 0 2024-04-21T08:38:37.261 INFO:teuthology.orchestra.run.smithi043.stdout:session_timeout 300 2024-04-21T08:38:37.261 INFO:teuthology.orchestra.run.smithi043.stdout:session_autoclose 300 2024-04-21T08:38:37.261 INFO:teuthology.orchestra.run.smithi043.stdout:max_file_size 1099511627776 2024-04-21T08:38:37.261 INFO:teuthology.orchestra.run.smithi043.stdout:max_xattr_size 65536 2024-04-21T08:38:37.261 INFO:teuthology.orchestra.run.smithi043.stdout:required_client_features {} 2024-04-21T08:38:37.261 INFO:teuthology.orchestra.run.smithi043.stdout:last_failure 0 2024-04-21T08:38:37.261 INFO:teuthology.orchestra.run.smithi043.stdout:last_failure_osd_epoch 0 2024-04-21T08:38:37.261 INFO:teuthology.orchestra.run.smithi043.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-21T08:38:37.261 INFO:teuthology.orchestra.run.smithi043.stdout:max_mds 5 2024-04-21T08:38:37.261 INFO:teuthology.orchestra.run.smithi043.stdout:in 0,1,2,3,4 2024-04-21T08:38:37.261 INFO:teuthology.orchestra.run.smithi043.stdout:up {0=24527,1=24491,2=24487,3=24425,4=24412} 2024-04-21T08:38:37.261 INFO:teuthology.orchestra.run.smithi043.stdout:failed 2024-04-21T08:38:37.261 INFO:teuthology.orchestra.run.smithi043.stdout:damaged 2024-04-21T08:38:37.261 INFO:teuthology.orchestra.run.smithi043.stdout:stopped 2024-04-21T08:38:37.261 INFO:teuthology.orchestra.run.smithi043.stdout:data_pools [3,4] 2024-04-21T08:38:37.262 INFO:teuthology.orchestra.run.smithi043.stdout:metadata_pool 2 2024-04-21T08:38:37.262 INFO:teuthology.orchestra.run.smithi043.stdout:inline_data disabled 2024-04-21T08:38:37.262 INFO:teuthology.orchestra.run.smithi043.stdout:balancer 2024-04-21T08:38:37.262 INFO:teuthology.orchestra.run.smithi043.stdout:bal_rank_mask -1 2024-04-21T08:38:37.262 INFO:teuthology.orchestra.run.smithi043.stdout:standby_count_wanted 0 2024-04-21T08:38:37.262 INFO:teuthology.orchestra.run.smithi043.stdout:qdb_cluster leader: 24527 members: 24491,24527,24487,24425,24412 2024-04-21T08:38:37.262 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.b{0:24527} state up:active seq 3 addr [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:38:37.262 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.c{0:24466} state up:standby-replay seq 1 addr [v2:172.21.15.175:6836/3198693848,v1:172.21.15.175:6837/3198693848] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:38:37.262 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.h{1:24491} state up:active seq 12 addr [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:38:37.262 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.e{1:24458} state up:standby-replay seq 1 addr [v2:172.21.15.112:6834/3159749962,v1:172.21.15.112:6835/3159749962] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:38:37.262 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.i{2:24487} state up:active seq 11 addr [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:38:37.262 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.f{2:24454} state up:standby-replay seq 1 addr [v2:172.21.15.175:6834/466532477,v1:172.21.15.175:6835/466532477] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:38:37.262 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.k{3:24425} state up:active seq 17 addr [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:38:37.262 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.a{3:14652} state up:standby-replay seq 1 addr [v2:172.21.15.43:6840/623279035,v1:172.21.15.43:6841/623279035] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:38:37.263 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.l{4:24412} state up:active seq 18 addr [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:38:37.263 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.g{4:14634} state up:standby-replay seq 1 addr [v2:172.21.15.43:6838/931821965,v1:172.21.15.43:6839/931821965] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:38:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:37.263 INFO:teuthology.orchestra.run.smithi043.stdout:Standby daemons: 2024-04-21T08:38:37.263 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:37.263 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.j{-1:14556} state up:standby seq 1 addr [v2:172.21.15.43:6834/3697998894,v1:172.21.15.43:6835/3697998894] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:38:37.263 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.d{-1:14568} state up:standby seq 1 addr [v2:172.21.15.43:6836/1236755039,v1:172.21.15.43:6837/1236755039] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:38:37.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:37.263+0000 7fafd325b700 1 -- 172.21.15.43:0/3516129049 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fafbc0a3c40 msgr2=0x7fafbc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:37.263+0000 7fafd325b700 1 --2- 172.21.15.43:0/3516129049 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fafbc0a3c40 0x7fafbc0a6100 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fafcc103b00 tx=0x7fafc000a380 comp rx=0 tx=0).stop 2024-04-21T08:38:37.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:37.263+0000 7fafd325b700 1 -- 172.21.15.43:0/3516129049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafcc104640 msgr2=0x7fafcc075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:37.263+0000 7fafd325b700 1 --2- 172.21.15.43:0/3516129049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafcc104640 0x7fafcc075370 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fafc4007ee0 tx=0x7fafc400d3b0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:37.264+0000 7fafd325b700 1 -- 172.21.15.43:0/3516129049 shutdown_connections 2024-04-21T08:38:37.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:37.264+0000 7fafd325b700 1 --2- 172.21.15.43:0/3516129049 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fafbc0a3c40 0x7fafbc0a6100 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:37.264+0000 7fafd325b700 1 --2- 172.21.15.43:0/3516129049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafcc104640 0x7fafcc075370 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:37.264+0000 7fafd325b700 1 --2- 172.21.15.43:0/3516129049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafcc102aa0 0x7fafcc0767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:37.264+0000 7fafd325b700 1 --2- 172.21.15.43:0/3516129049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafcc103ca0 0x7fafcc074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:37.264+0000 7fafd325b700 1 -- 172.21.15.43:0/3516129049 >> 172.21.15.43:0/3516129049 conn(0x7fafcc0fe040 msgr2=0x7fafcc106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:37.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:37.264+0000 7fafd325b700 1 -- 172.21.15.43:0/3516129049 shutdown_connections 2024-04-21T08:38:37.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:37.264+0000 7fafd325b700 1 -- 172.21.15.43:0/3516129049 wait complete. 2024-04-21T08:38:37.265 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 27 2024-04-21T08:38:37.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:37 smithi043 ceph-mon[27978]: pgmap v365: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:38:37.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:37 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3516129049' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-21T08:38:37.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:37 smithi175 ceph-mon[31920]: pgmap v365: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:38:37.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:37 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3516129049' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-21T08:38:37.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:37 smithi112 ceph-mon[31085]: pgmap v365: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:38:37.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:37 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3516129049' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-21T08:38:38.053 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-21T08:38:38.312 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:39.701 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:39 smithi043 ceph-mon[27978]: pgmap v366: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-21T08:38:39.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:39 smithi175 ceph-mon[31920]: pgmap v366: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-21T08:38:39.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:39 smithi112 ceph-mon[31085]: pgmap v366: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-21T08:38:40.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.058+0000 7f0114b5d700 1 -- 172.21.15.43:0/1824465107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0110104660 msgr2=0x7f011010af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:40.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.058+0000 7f0114b5d700 1 --2- 172.21.15.43:0/1824465107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0110104660 0x7f011010af10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f01080099c0 tx=0x7f0108009cd0 comp rx=0 tx=0).stop 2024-04-21T08:38:40.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.058+0000 7f0114b5d700 1 -- 172.21.15.43:0/1824465107 shutdown_connections 2024-04-21T08:38:40.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.058+0000 7f0114b5d700 1 --2- 172.21.15.43:0/1824465107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0110102ac0 0x7f0110102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:40.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.058+0000 7f0114b5d700 1 --2- 172.21.15.43:0/1824465107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0110104660 0x7f011010af10 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:40.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.058+0000 7f0114b5d700 1 --2- 172.21.15.43:0/1824465107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0110103cc0 0x7f0110104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:40.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.058+0000 7f0114b5d700 1 -- 172.21.15.43:0/1824465107 >> 172.21.15.43:0/1824465107 conn(0x7f01100fe060 msgr2=0x7f01101004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:40.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.058+0000 7f0114b5d700 1 -- 172.21.15.43:0/1824465107 shutdown_connections 2024-04-21T08:38:40.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.058+0000 7f0114b5d700 1 -- 172.21.15.43:0/1824465107 wait complete. 2024-04-21T08:38:40.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.059+0000 7f0114b5d700 1 Processor -- start 2024-04-21T08:38:40.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.059+0000 7f0114b5d700 1 -- start start 2024-04-21T08:38:40.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.059+0000 7f0114b5d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0110102ac0 0x7f01100767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:40.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.059+0000 7f0114b5d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0110103cc0 0x7f0110074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:40.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.059+0000 7f0114b5d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0110104660 0x7f0110075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:40.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.059+0000 7f0114b5d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f011010e560 con 0x7f0110104660 2024-04-21T08:38:40.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.059+0000 7f0114b5d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f011010e3e0 con 0x7f0110103cc0 2024-04-21T08:38:40.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.059+0000 7f0114b5d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f011010e6e0 con 0x7f0110102ac0 2024-04-21T08:38:40.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.059+0000 7f010ed9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0110104660 0x7f0110075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:40.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.059+0000 7f010ed9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0110104660 0x7f0110075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35870/0 (socket says 172.21.15.43:35870) 2024-04-21T08:38:40.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.059+0000 7f010ed9d700 1 -- 172.21.15.43:0/3595426523 learned_addr learned my addr 172.21.15.43:0/3595426523 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:40.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.059+0000 7f010e59c700 1 --2- 172.21.15.43:0/3595426523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0110102ac0 0x7f01100767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:40.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.059+0000 7f010dd9b700 1 --2- 172.21.15.43:0/3595426523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0110103cc0 0x7f0110074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:40.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.059+0000 7f010ed9d700 1 -- 172.21.15.43:0/3595426523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0110102ac0 msgr2=0x7f01100767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:40.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.059+0000 7f010ed9d700 1 --2- 172.21.15.43:0/3595426523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0110102ac0 0x7f01100767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:40.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.059+0000 7f010ed9d700 1 -- 172.21.15.43:0/3595426523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0110103cc0 msgr2=0x7f0110074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:40.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.059+0000 7f010ed9d700 1 --2- 172.21.15.43:0/3595426523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0110103cc0 0x7f0110074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:40.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.059+0000 7f010ed9d700 1 -- 172.21.15.43:0/3595426523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0110075c20 con 0x7f0110104660 2024-04-21T08:38:40.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.060+0000 7f010ed9d700 1 --2- 172.21.15.43:0/3595426523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0110104660 0x7f0110075370 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f01080059d0 tx=0x7f01080047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:40.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.060+0000 7f01077fe700 1 -- 172.21.15.43:0/3595426523 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0108003c30 con 0x7f0110104660 2024-04-21T08:38:40.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.060+0000 7f0114b5d700 1 -- 172.21.15.43:0/3595426523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01101a2c80 con 0x7f0110104660 2024-04-21T08:38:40.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.060+0000 7f01077fe700 1 -- 172.21.15.43:0/3595426523 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0108004830 con 0x7f0110104660 2024-04-21T08:38:40.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.060+0000 7f01077fe700 1 -- 172.21.15.43:0/3595426523 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01080052f0 con 0x7f0110104660 2024-04-21T08:38:40.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.060+0000 7f0114b5d700 1 -- 172.21.15.43:0/3595426523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01101a3070 con 0x7f0110104660 2024-04-21T08:38:40.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.061+0000 7f0114b5d700 1 -- 172.21.15.43:0/3595426523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f011002df80 con 0x7f0110104660 2024-04-21T08:38:40.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.062+0000 7f01077fe700 1 -- 172.21.15.43:0/3595426523 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0108029060 con 0x7f0110104660 2024-04-21T08:38:40.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.063+0000 7f01077fe700 1 --2- 172.21.15.43:0/3595426523 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f00fc0a3c40 0x7f00fc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:40.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.063+0000 7f010e59c700 1 --2- 172.21.15.43:0/3595426523 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f00fc0a3c40 0x7f00fc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:40.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.063+0000 7f01077fe700 1 -- 172.21.15.43:0/3595426523 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f01080d1700 con 0x7f0110104660 2024-04-21T08:38:40.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.063+0000 7f010e59c700 1 --2- 172.21.15.43:0/3595426523 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f00fc0a3c40 0x7f00fc0a6100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0110103b20 tx=0x7f0100009180 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:40.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.066+0000 7f01077fe700 1 -- 172.21.15.43:0/3595426523 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f010805dff0 con 0x7f0110104660 2024-04-21T08:38:40.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.274+0000 7f0114b5d700 1 -- 172.21.15.43:0/3595426523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f011002e340 con 0x7f0110104660 2024-04-21T08:38:40.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.275+0000 7f01077fe700 1 -- 172.21.15.43:0/3595426523 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) ==== 56+0+5738 (secure 0 0 0) 0x7f0108013070 con 0x7f0110104660 2024-04-21T08:38:40.275 INFO:teuthology.orchestra.run.smithi043.stdout:epoch 94 2024-04-21T08:38:40.275 INFO:teuthology.orchestra.run.smithi043.stdout:fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:38:40.275 INFO:teuthology.orchestra.run.smithi043.stdout:created 2024-04-21T08:28:11.976386+0000 2024-04-21T08:38:40.275 INFO:teuthology.orchestra.run.smithi043.stdout:modified 2024-04-21T08:38:15.326063+0000 2024-04-21T08:38:40.275 INFO:teuthology.orchestra.run.smithi043.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-21T08:38:40.275 INFO:teuthology.orchestra.run.smithi043.stdout:crush_version 27 2024-04-21T08:38:40.275 INFO:teuthology.orchestra.run.smithi043.stdout:full_ratio 0.95 2024-04-21T08:38:40.275 INFO:teuthology.orchestra.run.smithi043.stdout:backfillfull_ratio 0.9 2024-04-21T08:38:40.275 INFO:teuthology.orchestra.run.smithi043.stdout:nearfull_ratio 0.85 2024-04-21T08:38:40.275 INFO:teuthology.orchestra.run.smithi043.stdout:require_min_compat_client luminous 2024-04-21T08:38:40.275 INFO:teuthology.orchestra.run.smithi043.stdout:min_compat_client luminous 2024-04-21T08:38:40.275 INFO:teuthology.orchestra.run.smithi043.stdout:require_osd_release squid 2024-04-21T08:38:40.275 INFO:teuthology.orchestra.run.smithi043.stdout:stretch_mode_enabled false 2024-04-21T08:38:40.276 INFO:teuthology.orchestra.run.smithi043.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-21T08:38:40.276 INFO:teuthology.orchestra.run.smithi043.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-04-21T08:38:40.276 INFO:teuthology.orchestra.run.smithi043.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-21T08:38:40.276 INFO:teuthology.orchestra.run.smithi043.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-21T08:38:40.276 INFO:teuthology.orchestra.run.smithi043.stdout:max_osd 12 2024-04-21T08:38:40.276 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0 up in weight 1 up_from 8 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6802/4223002768,v1:172.21.15.43:6803/4223002768] [v2:172.21.15.43:6804/4223002768,v1:172.21.15.43:6805/4223002768] exists,up 070c2bde-f131-4523-b54c-d376a5015826 2024-04-21T08:38:40.276 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1 up in weight 1 up_from 13 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6800/2362092039,v1:172.21.15.112:6801/2362092039] [v2:172.21.15.112:6802/2362092039,v1:172.21.15.112:6803/2362092039] exists,up f297cc81-3025-40c8-8081-1b201be2c4bd 2024-04-21T08:38:40.276 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6800/2080591921,v1:172.21.15.175:6801/2080591921] [v2:172.21.15.175:6802/2080591921,v1:172.21.15.175:6803/2080591921] exists,up ca0c4942-852b-415e-9a75-8645f0435202 2024-04-21T08:38:40.276 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3 up in weight 1 up_from 25 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6810/117814050,v1:172.21.15.43:6811/117814050] [v2:172.21.15.43:6812/117814050,v1:172.21.15.43:6813/117814050] exists,up e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2 2024-04-21T08:38:40.276 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4 up in weight 1 up_from 33 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6808/3007534004,v1:172.21.15.112:6809/3007534004] [v2:172.21.15.112:6810/3007534004,v1:172.21.15.112:6811/3007534004] exists,up 87d594ea-199e-42db-8e47-2d535cbca3dc 2024-04-21T08:38:40.276 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5 up in weight 1 up_from 39 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6808/1901076703,v1:172.21.15.175:6809/1901076703] [v2:172.21.15.175:6810/1901076703,v1:172.21.15.175:6811/1901076703] exists,up 93b83dce-3ad8-40ef-b472-5612b41fd044 2024-04-21T08:38:40.276 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6 up in weight 1 up_from 47 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6818/4220470202,v1:172.21.15.43:6819/4220470202] [v2:172.21.15.43:6820/4220470202,v1:172.21.15.43:6821/4220470202] exists,up 3abceb62-32a7-48ae-a367-41869d2471cf 2024-04-21T08:38:40.276 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7 up in weight 1 up_from 52 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6816/1090541705,v1:172.21.15.112:6817/1090541705] [v2:172.21.15.112:6818/1090541705,v1:172.21.15.112:6819/1090541705] exists,up 223d7e91-e4a6-42e0-b5fb-bbbf4844efd2 2024-04-21T08:38:40.276 INFO:teuthology.orchestra.run.smithi043.stdout:osd.8 up in weight 1 up_from 60 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6816/1369698308,v1:172.21.15.175:6817/1369698308] [v2:172.21.15.175:6818/1369698308,v1:172.21.15.175:6819/1369698308] exists,up 277e3276-9d50-4233-9a4d-74b1a321f308 2024-04-21T08:38:40.277 INFO:teuthology.orchestra.run.smithi043.stdout:osd.9 up in weight 1 up_from 68 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6826/2518248011,v1:172.21.15.43:6827/2518248011] [v2:172.21.15.43:6828/2518248011,v1:172.21.15.43:6829/2518248011] exists,up f530826b-3c2f-411b-9795-f4b3c6053a23 2024-04-21T08:38:40.277 INFO:teuthology.orchestra.run.smithi043.stdout:osd.10 up in weight 1 up_from 73 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6824/3645019377,v1:172.21.15.112:6825/3645019377] [v2:172.21.15.112:6826/3645019377,v1:172.21.15.112:6827/3645019377] exists,up 307394bf-0915-448c-b025-42db9d0691c0 2024-04-21T08:38:40.277 INFO:teuthology.orchestra.run.smithi043.stdout:osd.11 up in weight 1 up_from 81 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6824/2527264089,v1:172.21.15.175:6825/2527264089] [v2:172.21.15.175:6826/2527264089,v1:172.21.15.175:6827/2527264089] exists,up 8114e33b-8fb2-4fe4-bf56-5f007c88ddbc 2024-04-21T08:38:40.277 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 2.4 [2,11] 2024-04-21T08:38:40.277 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 2.8 [2,11] 2024-04-21T08:38:40.277 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 2.1a [8,4] 2024-04-21T08:38:40.277 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.3 [4,9] 2024-04-21T08:38:40.277 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.12 [3,9] 2024-04-21T08:38:40.277 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.17 [4,6] 2024-04-21T08:38:40.277 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.1a [5,7] 2024-04-21T08:38:40.277 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.1b [4,6] 2024-04-21T08:38:40.277 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.1e [3,9] 2024-04-21T08:38:40.277 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.31 [4,8] 2024-04-21T08:38:40.277 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.39 [5,8] 2024-04-21T08:38:40.277 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 4.2 [9,1] 2024-04-21T08:38:40.277 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 4.5 [5,8] 2024-04-21T08:38:40.278 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 4.6 [9,11] 2024-04-21T08:38:40.278 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 4.13 [9,8] 2024-04-21T08:38:40.278 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/1916182222 expires 2024-04-22T08:29:13.889526+0000 2024-04-21T08:38:40.278 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:6801/3200866696 expires 2024-04-22T08:29:13.889526+0000 2024-04-21T08:38:40.278 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:6801/1559586579 expires 2024-04-22T08:28:46.855699+0000 2024-04-21T08:38:40.278 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:6800/3200866696 expires 2024-04-22T08:29:13.889526+0000 2024-04-21T08:38:40.278 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/4047106671 expires 2024-04-22T08:29:13.889526+0000 2024-04-21T08:38:40.278 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:6800/1559586579 expires 2024-04-22T08:28:46.855699+0000 2024-04-21T08:38:40.278 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/347461706 expires 2024-04-22T08:29:13.889526+0000 2024-04-21T08:38:40.278 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/887926369 expires 2024-04-22T08:28:46.855699+0000 2024-04-21T08:38:40.278 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/2598599313 expires 2024-04-22T08:28:46.855699+0000 2024-04-21T08:38:40.278 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/2126312819 expires 2024-04-22T08:28:46.855699+0000 2024-04-21T08:38:40.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.277+0000 7f0114b5d700 1 -- 172.21.15.43:0/3595426523 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f00fc0a3c40 msgr2=0x7f00fc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:40.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.277+0000 7f0114b5d700 1 --2- 172.21.15.43:0/3595426523 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f00fc0a3c40 0x7f00fc0a6100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0110103b20 tx=0x7f0100009180 comp rx=0 tx=0).stop 2024-04-21T08:38:40.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.277+0000 7f0114b5d700 1 -- 172.21.15.43:0/3595426523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0110104660 msgr2=0x7f0110075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:40.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.277+0000 7f0114b5d700 1 --2- 172.21.15.43:0/3595426523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0110104660 0x7f0110075370 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f01080059d0 tx=0x7f01080047b0 comp rx=0 tx=0).stop 2024-04-21T08:38:40.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.278+0000 7f0114b5d700 1 -- 172.21.15.43:0/3595426523 shutdown_connections 2024-04-21T08:38:40.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.278+0000 7f0114b5d700 1 --2- 172.21.15.43:0/3595426523 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f00fc0a3c40 0x7f00fc0a6100 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:40.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.278+0000 7f0114b5d700 1 --2- 172.21.15.43:0/3595426523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0110102ac0 0x7f01100767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:40.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.278+0000 7f0114b5d700 1 --2- 172.21.15.43:0/3595426523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0110104660 0x7f0110075370 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:40.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.278+0000 7f0114b5d700 1 --2- 172.21.15.43:0/3595426523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0110103cc0 0x7f0110074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:40.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.278+0000 7f0114b5d700 1 -- 172.21.15.43:0/3595426523 >> 172.21.15.43:0/3595426523 conn(0x7f01100fe060 msgr2=0x7f0110106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:40.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.278+0000 7f0114b5d700 1 -- 172.21.15.43:0/3595426523 shutdown_connections 2024-04-21T08:38:40.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:40.278+0000 7f0114b5d700 1 -- 172.21.15.43:0/3595426523 wait complete. 2024-04-21T08:38:40.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:40 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3595426523' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-21T08:38:40.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:40 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3595426523' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-21T08:38:40.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:40 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3595426523' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-21T08:38:41.212 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-21T08:38:41.224 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:38:41.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.338+0000 7fb311688640 1 -- 172.21.15.43:0/1173710646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb30c102050 msgr2=0x7fb30c1024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:41.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.339+0000 7fb311688640 1 --2- 172.21.15.43:0/1173710646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb30c102050 0x7fb30c1024d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4009900 tx=0x7fb2f402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:41.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.339+0000 7fb311688640 1 -- 172.21.15.43:0/1173710646 shutdown_connections 2024-04-21T08:38:41.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.339+0000 7fb311688640 1 --2- 172.21.15.43:0/1173710646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb30c102a10 0x7fb30c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:41.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.339+0000 7fb311688640 1 --2- 172.21.15.43:0/1173710646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb30c102050 0x7fb30c1024d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:41.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.339+0000 7fb311688640 1 --2- 172.21.15.43:0/1173710646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb30c100e70 0x7fb30c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:41.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.339+0000 7fb311688640 1 -- 172.21.15.43:0/1173710646 >> 172.21.15.43:0/1173710646 conn(0x7fb30c0fc470 msgr2=0x7fb30c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:41.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.339+0000 7fb311688640 1 -- 172.21.15.43:0/1173710646 shutdown_connections 2024-04-21T08:38:41.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.339+0000 7fb311688640 1 -- 172.21.15.43:0/1173710646 wait complete. 2024-04-21T08:38:41.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.339+0000 7fb311688640 1 Processor -- start 2024-04-21T08:38:41.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.339+0000 7fb311688640 1 -- start start 2024-04-21T08:38:41.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.339+0000 7fb311688640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb30c100e70 0x7fb30c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:41.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.339+0000 7fb311688640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb30c102050 0x7fb30c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:41.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.339+0000 7fb311688640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb30c102a10 0x7fb30c19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:41.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.339+0000 7fb311688640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c10ca50 con 0x7fb30c100e70 2024-04-21T08:38:41.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.339+0000 7fb311688640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c10c8d0 con 0x7fb30c102050 2024-04-21T08:38:41.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.339+0000 7fb311688640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c10cbd0 con 0x7fb30c102a10 2024-04-21T08:38:41.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.339+0000 7fb30affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb30c100e70 0x7fb30c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:41.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.340+0000 7fb30affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb30c100e70 0x7fb30c1981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35890/0 (socket says 172.21.15.43:35890) 2024-04-21T08:38:41.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.340+0000 7fb30affd640 1 -- 172.21.15.43:0/694410195 learned_addr learned my addr 172.21.15.43:0/694410195 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:41.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.340+0000 7fb30affd640 1 -- 172.21.15.43:0/694410195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb30c102a10 msgr2=0x7fb30c19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:41.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.340+0000 7fb30affd640 1 --2- 172.21.15.43:0/694410195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb30c102a10 0x7fb30c19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:41.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.340+0000 7fb30affd640 1 -- 172.21.15.43:0/694410195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb30c102050 msgr2=0x7fb30c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:41.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.340+0000 7fb303fff640 1 --2- 172.21.15.43:0/694410195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb30c102050 0x7fb30c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:41.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.340+0000 7fb30affd640 1 --2- 172.21.15.43:0/694410195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb30c102050 0x7fb30c1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:41.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.340+0000 7fb30affd640 1 -- 172.21.15.43:0/694410195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30c19fdd0 con 0x7fb30c100e70 2024-04-21T08:38:41.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.340+0000 7fb30affd640 1 --2- 172.21.15.43:0/694410195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb30c100e70 0x7fb30c1981b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc00d950 tx=0x7fb2fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:41.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.340+0000 7fb308ff9640 1 -- 172.21.15.43:0/694410195 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc017020 con 0x7fb30c100e70 2024-04-21T08:38:41.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.340+0000 7fb311688640 1 -- 172.21.15.43:0/694410195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb30c1a00c0 con 0x7fb30c100e70 2024-04-21T08:38:41.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.340+0000 7fb308ff9640 1 -- 172.21.15.43:0/694410195 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2fc009c30 con 0x7fb30c100e70 2024-04-21T08:38:41.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.340+0000 7fb308ff9640 1 -- 172.21.15.43:0/694410195 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc00f6b0 con 0x7fb30c100e70 2024-04-21T08:38:41.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.340+0000 7fb311688640 1 -- 172.21.15.43:0/694410195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb30c1a0610 con 0x7fb30c100e70 2024-04-21T08:38:41.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.341+0000 7fb311688640 1 -- 172.21.15.43:0/694410195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb30c1a0930 con 0x7fb30c100e70 2024-04-21T08:38:41.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.343+0000 7fb308ff9640 1 -- 172.21.15.43:0/694410195 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb2fc01a020 con 0x7fb30c100e70 2024-04-21T08:38:41.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.343+0000 7fb308ff9640 1 --2- 172.21.15.43:0/694410195 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb2f80a3bb0 0x7fb2f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:41.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.343+0000 7fb308ff9640 1 -- 172.21.15.43:0/694410195 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fb2fc0d0cf0 con 0x7fb30c100e70 2024-04-21T08:38:41.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.343+0000 7fb303fff640 1 --2- 172.21.15.43:0/694410195 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb2f80a3bb0 0x7fb2f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:41.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.344+0000 7fb303fff640 1 --2- 172.21.15.43:0/694410195 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb2f80a3bb0 0x7fb2f80a6070 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb2f40096c0 tx=0x7fb2f4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:41.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.347+0000 7fb308ff9640 1 -- 172.21.15.43:0/694410195 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb2fc009de0 con 0x7fb30c100e70 2024-04-21T08:38:41.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.537+0000 7fb311688640 1 -- 172.21.15.43:0/694410195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb30c1a0b00 con 0x7fb30c100e70 2024-04-21T08:38:41.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.539+0000 7fb308ff9640 1 -- 172.21.15.43:0/694410195 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24856 (secure 0 0 0) 0x7fb2fc09d2e0 con 0x7fb30c100e70 2024-04-21T08:38:41.538 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:41.538 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":94,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:38:15.326063+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:38:41.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.540+0000 7fb311688640 1 -- 172.21.15.43:0/694410195 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb2f80a3bb0 msgr2=0x7fb2f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:41.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.540+0000 7fb311688640 1 --2- 172.21.15.43:0/694410195 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb2f80a3bb0 0x7fb2f80a6070 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb2f40096c0 tx=0x7fb2f4005fd0 comp rx=0 tx=0).stop 2024-04-21T08:38:41.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.540+0000 7fb311688640 1 -- 172.21.15.43:0/694410195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb30c100e70 msgr2=0x7fb30c1981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:41.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.540+0000 7fb311688640 1 --2- 172.21.15.43:0/694410195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb30c100e70 0x7fb30c1981b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc00d950 tx=0x7fb2fc00de10 comp rx=0 tx=0).stop 2024-04-21T08:38:41.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.540+0000 7fb311688640 1 -- 172.21.15.43:0/694410195 shutdown_connections 2024-04-21T08:38:41.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.540+0000 7fb311688640 1 --2- 172.21.15.43:0/694410195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb30c102a10 0x7fb30c19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:41.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.540+0000 7fb311688640 1 --2- 172.21.15.43:0/694410195 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb2f80a3bb0 0x7fb2f80a6070 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:41.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.540+0000 7fb311688640 1 --2- 172.21.15.43:0/694410195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb30c102050 0x7fb30c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:41.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.540+0000 7fb311688640 1 --2- 172.21.15.43:0/694410195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb30c100e70 0x7fb30c1981b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:41.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.541+0000 7fb311688640 1 -- 172.21.15.43:0/694410195 >> 172.21.15.43:0/694410195 conn(0x7fb30c0fc470 msgr2=0x7fb30c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:41.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.541+0000 7fb311688640 1 -- 172.21.15.43:0/694410195 shutdown_connections 2024-04-21T08:38:41.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.541+0000 7fb311688640 1 -- 172.21.15.43:0/694410195 wait complete. 2024-04-21T08:38:41.551 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:39.981405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:22.034751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:22.780795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:25.984261+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:38:41.552 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:38:41.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.658+0000 7f8eda9f4640 1 -- 172.21.15.43:0/4083800215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed4100e60 msgr2=0x7f8ed4101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:41.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.658+0000 7f8eda9f4640 1 --2- 172.21.15.43:0/4083800215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed4100e60 0x7f8ed4101280 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009900 tx=0x7f8ec402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:41.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.658+0000 7f8eda9f4640 1 -- 172.21.15.43:0/4083800215 shutdown_connections 2024-04-21T08:38:41.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.658+0000 7f8eda9f4640 1 --2- 172.21.15.43:0/4083800215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ed4102a00 0x7f8ed4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:41.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.658+0000 7f8eda9f4640 1 --2- 172.21.15.43:0/4083800215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ed4102040 0x7f8ed41024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:41.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.658+0000 7f8eda9f4640 1 --2- 172.21.15.43:0/4083800215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed4100e60 0x7f8ed4101280 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.658+0000 7f8eda9f4640 1 -- 172.21.15.43:0/4083800215 >> 172.21.15.43:0/4083800215 conn(0x7f8ed40fc440 msgr2=0x7f8ed40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.658+0000 7f8eda9f4640 1 -- 172.21.15.43:0/4083800215 shutdown_connections 2024-04-21T08:38:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.658+0000 7f8eda9f4640 1 -- 172.21.15.43:0/4083800215 wait complete. 2024-04-21T08:38:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.659+0000 7f8eda9f4640 1 Processor -- start 2024-04-21T08:38:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.659+0000 7f8eda9f4640 1 -- start start 2024-04-21T08:38:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.659+0000 7f8eda9f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed4100e60 0x7f8ed4198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.659+0000 7f8eda9f4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ed4102040 0x7f8ed4198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.659+0000 7f8eda9f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ed4102a00 0x7f8ed419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.659+0000 7f8eda9f4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed410c9d0 con 0x7f8ed4100e60 2024-04-21T08:38:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.659+0000 7f8eda9f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed410c850 con 0x7f8ed4102a00 2024-04-21T08:38:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.659+0000 7f8eda9f4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed410cb50 con 0x7f8ed4102040 2024-04-21T08:38:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.659+0000 7f8ed3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed4100e60 0x7f8ed4198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.659+0000 7f8ed3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed4100e60 0x7f8ed4198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35906/0 (socket says 172.21.15.43:35906) 2024-04-21T08:38:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.659+0000 7f8ed3fff640 1 -- 172.21.15.43:0/3485650426 learned_addr learned my addr 172.21.15.43:0/3485650426 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.660+0000 7f8ed8f91640 1 --2- 172.21.15.43:0/3485650426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ed4102a00 0x7f8ed419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.660+0000 7f8ed3fff640 1 -- 172.21.15.43:0/3485650426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ed4102040 msgr2=0x7f8ed4198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.660+0000 7f8ed3fff640 1 --2- 172.21.15.43:0/3485650426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ed4102040 0x7f8ed4198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.660+0000 7f8ed3fff640 1 -- 172.21.15.43:0/3485650426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ed4102a00 msgr2=0x7f8ed419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.660+0000 7f8ed3fff640 1 --2- 172.21.15.43:0/3485650426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ed4102a00 0x7f8ed419f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.660+0000 7f8ed3fff640 1 -- 172.21.15.43:0/3485650426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed419fe70 con 0x7f8ed4100e60 2024-04-21T08:38:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.660+0000 7f8ed3fff640 1 --2- 172.21.15.43:0/3485650426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed4100e60 0x7f8ed4198250 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4000c00 tx=0x7f8ec4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.660+0000 7f8ed17fa640 1 -- 172.21.15.43:0/3485650426 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4043020 con 0x7f8ed4100e60 2024-04-21T08:38:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.660+0000 7f8ed17fa640 1 -- 172.21.15.43:0/3485650426 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ec4004aa0 con 0x7f8ed4100e60 2024-04-21T08:38:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.660+0000 7f8eda9f4640 1 -- 172.21.15.43:0/3485650426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed41a0100 con 0x7f8ed4100e60 2024-04-21T08:38:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.660+0000 7f8ed17fa640 1 -- 172.21.15.43:0/3485650426 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4003d00 con 0x7f8ed4100e60 2024-04-21T08:38:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.660+0000 7f8eda9f4640 1 -- 172.21.15.43:0/3485650426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed41a0650 con 0x7f8ed4100e60 2024-04-21T08:38:41.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.661+0000 7f8eda9f4640 1 -- 172.21.15.43:0/3485650426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ed41a0850 con 0x7f8ed4100e60 2024-04-21T08:38:41.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.662+0000 7f8ed17fa640 1 -- 172.21.15.43:0/3485650426 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8ec4046020 con 0x7f8ed4100e60 2024-04-21T08:38:41.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.663+0000 7f8ed17fa640 1 --2- 172.21.15.43:0/3485650426 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8ebc0a3bb0 0x7f8ebc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:41.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.663+0000 7f8ed37fe640 1 --2- 172.21.15.43:0/3485650426 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8ebc0a3bb0 0x7f8ebc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:41.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.663+0000 7f8ed17fa640 1 -- 172.21.15.43:0/3485650426 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f8ec40f2e30 con 0x7f8ed4100e60 2024-04-21T08:38:41.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.663+0000 7f8ed37fe640 1 --2- 172.21.15.43:0/3485650426 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8ebc0a3bb0 0x7f8ebc0a6070 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8005fd0 tx=0x7f8ec8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:41.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.667+0000 7f8ed17fa640 1 -- 172.21.15.43:0/3485650426 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ec404d020 con 0x7f8ed4100e60 2024-04-21T08:38:41.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:41 smithi175 ceph-mon[31920]: pgmap v367: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:38:41.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:41 smithi112 ceph-mon[31085]: pgmap v367: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:38:41.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:41 smithi043 ceph-mon[27978]: pgmap v367: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:38:41.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.854+0000 7f8eda9f4640 1 -- 172.21.15.43:0/3485650426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8ed41a0a20 con 0x7f8ed4100e60 2024-04-21T08:38:41.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.855+0000 7f8ed17fa640 1 -- 172.21.15.43:0/3485650426 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f8ec40bf420 con 0x7f8ed4100e60 2024-04-21T08:38:41.854 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-21T08:38:41.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.857+0000 7f8eda9f4640 1 -- 172.21.15.43:0/3485650426 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8ebc0a3bb0 msgr2=0x7f8ebc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:41.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.857+0000 7f8eda9f4640 1 --2- 172.21.15.43:0/3485650426 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8ebc0a3bb0 0x7f8ebc0a6070 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8005fd0 tx=0x7f8ec8005850 comp rx=0 tx=0).stop 2024-04-21T08:38:41.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.857+0000 7f8eda9f4640 1 -- 172.21.15.43:0/3485650426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed4100e60 msgr2=0x7f8ed4198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:41.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.857+0000 7f8eda9f4640 1 --2- 172.21.15.43:0/3485650426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed4100e60 0x7f8ed4198250 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4000c00 tx=0x7f8ec4004270 comp rx=0 tx=0).stop 2024-04-21T08:38:41.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.857+0000 7f8eda9f4640 1 -- 172.21.15.43:0/3485650426 shutdown_connections 2024-04-21T08:38:41.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.857+0000 7f8eda9f4640 1 --2- 172.21.15.43:0/3485650426 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f8ebc0a3bb0 0x7f8ebc0a6070 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:41.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.857+0000 7f8eda9f4640 1 --2- 172.21.15.43:0/3485650426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ed4102a00 0x7f8ed419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:41.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.857+0000 7f8eda9f4640 1 --2- 172.21.15.43:0/3485650426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ed4102040 0x7f8ed4198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:41.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.857+0000 7f8eda9f4640 1 --2- 172.21.15.43:0/3485650426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed4100e60 0x7f8ed4198250 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:41.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.857+0000 7f8eda9f4640 1 -- 172.21.15.43:0/3485650426 >> 172.21.15.43:0/3485650426 conn(0x7f8ed40fc440 msgr2=0x7f8ed406a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:41.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.857+0000 7f8eda9f4640 1 -- 172.21.15.43:0/3485650426 shutdown_connections 2024-04-21T08:38:41.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.857+0000 7f8eda9f4640 1 -- 172.21.15.43:0/3485650426 wait complete. 2024-04-21T08:38:41.867 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:38:41.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.974+0000 7fea7c375640 1 -- 172.21.15.43:0/1148696040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea74102040 msgr2=0x7fea741024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:41.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.974+0000 7fea7c375640 1 --2- 172.21.15.43:0/1148696040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea74102040 0x7fea741024c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fea64009900 tx=0x7fea6402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:41.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.974+0000 7fea7c375640 1 -- 172.21.15.43:0/1148696040 shutdown_connections 2024-04-21T08:38:41.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.974+0000 7fea7c375640 1 --2- 172.21.15.43:0/1148696040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea74102a00 0x7fea74109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:41.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.974+0000 7fea7c375640 1 --2- 172.21.15.43:0/1148696040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea74102040 0x7fea741024c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:41.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.974+0000 7fea7c375640 1 --2- 172.21.15.43:0/1148696040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea74100e60 0x7fea74101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:41.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.974+0000 7fea7c375640 1 -- 172.21.15.43:0/1148696040 >> 172.21.15.43:0/1148696040 conn(0x7fea740fc440 msgr2=0x7fea740fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:41.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.974+0000 7fea7c375640 1 -- 172.21.15.43:0/1148696040 shutdown_connections 2024-04-21T08:38:41.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.974+0000 7fea7c375640 1 -- 172.21.15.43:0/1148696040 wait complete. 2024-04-21T08:38:41.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.975+0000 7fea7c375640 1 Processor -- start 2024-04-21T08:38:41.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.975+0000 7fea7c375640 1 -- start start 2024-04-21T08:38:41.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.975+0000 7fea7c375640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea74100e60 0x7fea7410b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:41.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.975+0000 7fea7c375640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea74102040 0x7fea74109ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:41.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.975+0000 7fea7c375640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea74102a00 0x7fea7410a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:41.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.975+0000 7fea7c375640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fea7410db90 con 0x7fea74100e60 2024-04-21T08:38:41.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.975+0000 7fea7c375640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fea7410da10 con 0x7fea74102a00 2024-04-21T08:38:41.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.975+0000 7fea7c375640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fea7410dd10 con 0x7fea74102040 2024-04-21T08:38:41.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.975+0000 7fea7a111640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea74100e60 0x7fea7410b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:41.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.975+0000 7fea7a111640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea74100e60 0x7fea7410b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35926/0 (socket says 172.21.15.43:35926) 2024-04-21T08:38:41.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.975+0000 7fea7a111640 1 -- 172.21.15.43:0/207662175 learned_addr learned my addr 172.21.15.43:0/207662175 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:41.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.975+0000 7fea79910640 1 --2- 172.21.15.43:0/207662175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea74102040 0x7fea74109ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:41.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.976+0000 7fea7a111640 1 -- 172.21.15.43:0/207662175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea74102040 msgr2=0x7fea74109ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:41.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.976+0000 7fea7a111640 1 --2- 172.21.15.43:0/207662175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea74102040 0x7fea74109ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:41.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.976+0000 7fea7a111640 1 -- 172.21.15.43:0/207662175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea74102a00 msgr2=0x7fea7410a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:41.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.976+0000 7fea7a912640 1 --2- 172.21.15.43:0/207662175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea74102a00 0x7fea7410a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:41.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.976+0000 7fea7a111640 1 --2- 172.21.15.43:0/207662175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea74102a00 0x7fea7410a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:41.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.976+0000 7fea7a111640 1 -- 172.21.15.43:0/207662175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7410ac70 con 0x7fea74100e60 2024-04-21T08:38:41.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.976+0000 7fea7a912640 1 --2- 172.21.15.43:0/207662175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea74102a00 0x7fea7410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:41.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.976+0000 7fea7a111640 1 --2- 172.21.15.43:0/207662175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea74100e60 0x7fea7410b860 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fea7000d950 tx=0x7fea7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:41.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.976+0000 7fea6b7fe640 1 -- 172.21.15.43:0/207662175 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea70017020 con 0x7fea74100e60 2024-04-21T08:38:41.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.976+0000 7fea6b7fe640 1 -- 172.21.15.43:0/207662175 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea70009c30 con 0x7fea74100e60 2024-04-21T08:38:41.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.976+0000 7fea6b7fe640 1 -- 172.21.15.43:0/207662175 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7000f6b0 con 0x7fea74100e60 2024-04-21T08:38:41.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.976+0000 7fea7c375640 1 -- 172.21.15.43:0/207662175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea741a2310 con 0x7fea74100e60 2024-04-21T08:38:41.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.976+0000 7fea7c375640 1 -- 172.21.15.43:0/207662175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea741a2880 con 0x7fea74100e60 2024-04-21T08:38:41.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.977+0000 7fea7c375640 1 -- 172.21.15.43:0/207662175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea741082d0 con 0x7fea74100e60 2024-04-21T08:38:41.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.978+0000 7fea6b7fe640 1 -- 172.21.15.43:0/207662175 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fea7001a020 con 0x7fea74100e60 2024-04-21T08:38:41.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.979+0000 7fea6b7fe640 1 --2- 172.21.15.43:0/207662175 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fea600a3c30 0x7fea600a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:41.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.979+0000 7fea6b7fe640 1 -- 172.21.15.43:0/207662175 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fea700d04d0 con 0x7fea74100e60 2024-04-21T08:38:41.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.979+0000 7fea79910640 1 --2- 172.21.15.43:0/207662175 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fea600a3c30 0x7fea600a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:41.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.979+0000 7fea79910640 1 --2- 172.21.15.43:0/207662175 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fea600a3c30 0x7fea600a60f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fea64000c00 tx=0x7fea6402c7e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:41.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:41.983+0000 7fea6b7fe640 1 -- 172.21.15.43:0/207662175 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea70009de0 con 0x7fea74100e60 2024-04-21T08:38:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.171+0000 7fea7c375640 1 -- 172.21.15.43:0/207662175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fea74062b50 con 0x7fea74100e60 2024-04-21T08:38:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.171+0000 7fea6b7fe640 1 -- 172.21.15.43:0/207662175 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fea7009ca60 con 0x7fea74100e60 2024-04-21T08:38:42.171 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:38:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.173+0000 7fea7c375640 1 -- 172.21.15.43:0/207662175 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fea600a3c30 msgr2=0x7fea600a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.173+0000 7fea7c375640 1 --2- 172.21.15.43:0/207662175 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fea600a3c30 0x7fea600a60f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fea64000c00 tx=0x7fea6402c7e0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.173+0000 7fea7c375640 1 -- 172.21.15.43:0/207662175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea74100e60 msgr2=0x7fea7410b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.173+0000 7fea7c375640 1 --2- 172.21.15.43:0/207662175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea74100e60 0x7fea7410b860 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fea7000d950 tx=0x7fea7000de10 comp rx=0 tx=0).stop 2024-04-21T08:38:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.173+0000 7fea7c375640 1 -- 172.21.15.43:0/207662175 shutdown_connections 2024-04-21T08:38:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.173+0000 7fea7c375640 1 --2- 172.21.15.43:0/207662175 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fea600a3c30 0x7fea600a60f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.173+0000 7fea7c375640 1 --2- 172.21.15.43:0/207662175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea74102a00 0x7fea7410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.173+0000 7fea7c375640 1 --2- 172.21.15.43:0/207662175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea74102040 0x7fea74109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.173+0000 7fea7c375640 1 --2- 172.21.15.43:0/207662175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea74100e60 0x7fea7410b860 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.173+0000 7fea7c375640 1 -- 172.21.15.43:0/207662175 >> 172.21.15.43:0/207662175 conn(0x7fea740fc440 msgr2=0x7fea741054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:42.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.174+0000 7fea7c375640 1 -- 172.21.15.43:0/207662175 shutdown_connections 2024-04-21T08:38:42.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.174+0000 7fea7c375640 1 -- 172.21.15.43:0/207662175 wait complete. 2024-04-21T08:38:42.184 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:38:42.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.290+0000 7fd1e52b2640 1 -- 172.21.15.43:0/3900749181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e0107c90 msgr2=0x7fd1e010a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:42.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.290+0000 7fd1e52b2640 1 --2- 172.21.15.43:0/3900749181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e0107c90 0x7fd1e010a0e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009ca0 tx=0x7fd1d802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.290+0000 7fd1e52b2640 1 -- 172.21.15.43:0/3900749181 shutdown_connections 2024-04-21T08:38:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.290+0000 7fd1e52b2640 1 --2- 172.21.15.43:0/3900749181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e0107c90 0x7fd1e010a0e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.290+0000 7fd1e52b2640 1 --2- 172.21.15.43:0/3900749181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1e0105360 0x7fd1e0107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.290+0000 7fd1e52b2640 1 --2- 172.21.15.43:0/3900749181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1e0102a30 0x7fd1e0104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.290+0000 7fd1e52b2640 1 -- 172.21.15.43:0/3900749181 >> 172.21.15.43:0/3900749181 conn(0x7fd1e00fc440 msgr2=0x7fd1e00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.290+0000 7fd1e52b2640 1 -- 172.21.15.43:0/3900749181 shutdown_connections 2024-04-21T08:38:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.290+0000 7fd1e52b2640 1 -- 172.21.15.43:0/3900749181 wait complete. 2024-04-21T08:38:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.291+0000 7fd1e52b2640 1 Processor -- start 2024-04-21T08:38:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.291+0000 7fd1e52b2640 1 -- start start 2024-04-21T08:38:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.291+0000 7fd1e52b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1e0102a30 0x7fd1e0198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.291+0000 7fd1e52b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e0105360 0x7fd1e0198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.291+0000 7fd1e52b2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1e0107c90 0x7fd1e019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.291+0000 7fd1e52b2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e010db50 con 0x7fd1e0105360 2024-04-21T08:38:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.291+0000 7fd1e52b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e010d9d0 con 0x7fd1e0102a30 2024-04-21T08:38:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.291+0000 7fd1e52b2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e010dcd0 con 0x7fd1e0107c90 2024-04-21T08:38:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.291+0000 7fd1de7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e0105360 0x7fd1e0198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.291+0000 7fd1de7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e0105360 0x7fd1e0198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35948/0 (socket says 172.21.15.43:35948) 2024-04-21T08:38:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.291+0000 7fd1de7fc640 1 -- 172.21.15.43:0/1479821738 learned_addr learned my addr 172.21.15.43:0/1479821738 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.291+0000 7fd1deffd640 1 --2- 172.21.15.43:0/1479821738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1e0102a30 0x7fd1e0198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.291+0000 7fd1df7fe640 1 --2- 172.21.15.43:0/1479821738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1e0107c90 0x7fd1e019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.291+0000 7fd1de7fc640 1 -- 172.21.15.43:0/1479821738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1e0107c90 msgr2=0x7fd1e019f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.291+0000 7fd1de7fc640 1 --2- 172.21.15.43:0/1479821738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1e0107c90 0x7fd1e019f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.291+0000 7fd1de7fc640 1 -- 172.21.15.43:0/1479821738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1e0102a30 msgr2=0x7fd1e0198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.291+0000 7fd1de7fc640 1 --2- 172.21.15.43:0/1479821738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1e0102a30 0x7fd1e0198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.291+0000 7fd1de7fc640 1 -- 172.21.15.43:0/1479821738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e019fe40 con 0x7fd1e0105360 2024-04-21T08:38:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.292+0000 7fd1de7fc640 1 --2- 172.21.15.43:0/1479821738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e0105360 0x7fd1e0198760 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400b450 tx=0x7fd1d400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.292+0000 7fd1c7fff640 1 -- 172.21.15.43:0/1479821738 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4015020 con 0x7fd1e0105360 2024-04-21T08:38:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.292+0000 7fd1c7fff640 1 -- 172.21.15.43:0/1479821738 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1d4011d40 con 0x7fd1e0105360 2024-04-21T08:38:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.292+0000 7fd1e52b2640 1 -- 172.21.15.43:0/1479821738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e01a0130 con 0x7fd1e0105360 2024-04-21T08:38:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.292+0000 7fd1c7fff640 1 -- 172.21.15.43:0/1479821738 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d400f340 con 0x7fd1e0105360 2024-04-21T08:38:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.292+0000 7fd1e52b2640 1 -- 172.21.15.43:0/1479821738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e01a06d0 con 0x7fd1e0105360 2024-04-21T08:38:42.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.293+0000 7fd1e52b2640 1 -- 172.21.15.43:0/1479821738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1e00fdf30 con 0x7fd1e0105360 2024-04-21T08:38:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.294+0000 7fd1c7fff640 1 -- 172.21.15.43:0/1479821738 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd1d4018020 con 0x7fd1e0105360 2024-04-21T08:38:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.294+0000 7fd1c7fff640 1 --2- 172.21.15.43:0/1479821738 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd1c80a3c30 0x7fd1c80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.294+0000 7fd1deffd640 1 --2- 172.21.15.43:0/1479821738 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd1c80a3c30 0x7fd1c80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.295+0000 7fd1c7fff640 1 -- 172.21.15.43:0/1479821738 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fd1d40d0210 con 0x7fd1e0105360 2024-04-21T08:38:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.295+0000 7fd1deffd640 1 --2- 172.21.15.43:0/1479821738 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd1c80a3c30 0x7fd1c80a60f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000bbe0 tx=0x7fd1d000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.299+0000 7fd1c7fff640 1 -- 172.21.15.43:0/1479821738 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd1d401f020 con 0x7fd1e0105360 2024-04-21T08:38:42.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.490+0000 7fd1e52b2640 1 -- 172.21.15.43:0/1479821738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd1e0062b50 con 0x7fd1e0105360 2024-04-21T08:38:42.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.490+0000 7fd1c7fff640 1 -- 172.21.15.43:0/1479821738 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fd1d400d040 con 0x7fd1e0105360 2024-04-21T08:38:42.490 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-21T08:38:42.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.493+0000 7fd1e52b2640 1 -- 172.21.15.43:0/1479821738 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd1c80a3c30 msgr2=0x7fd1c80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:42.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.493+0000 7fd1e52b2640 1 --2- 172.21.15.43:0/1479821738 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd1c80a3c30 0x7fd1c80a60f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000bbe0 tx=0x7fd1d000b390 comp rx=0 tx=0).stop 2024-04-21T08:38:42.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.493+0000 7fd1e52b2640 1 -- 172.21.15.43:0/1479821738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e0105360 msgr2=0x7fd1e0198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:42.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.493+0000 7fd1e52b2640 1 --2- 172.21.15.43:0/1479821738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e0105360 0x7fd1e0198760 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400b450 tx=0x7fd1d400b910 comp rx=0 tx=0).stop 2024-04-21T08:38:42.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.493+0000 7fd1e52b2640 1 -- 172.21.15.43:0/1479821738 shutdown_connections 2024-04-21T08:38:42.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.493+0000 7fd1e52b2640 1 --2- 172.21.15.43:0/1479821738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1e0107c90 0x7fd1e019f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.493+0000 7fd1e52b2640 1 --2- 172.21.15.43:0/1479821738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e0105360 0x7fd1e0198760 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.493+0000 7fd1e52b2640 1 --2- 172.21.15.43:0/1479821738 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fd1c80a3c30 0x7fd1c80a60f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.493+0000 7fd1e52b2640 1 --2- 172.21.15.43:0/1479821738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1e0102a30 0x7fd1e0198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.493+0000 7fd1e52b2640 1 -- 172.21.15.43:0/1479821738 >> 172.21.15.43:0/1479821738 conn(0x7fd1e00fc440 msgr2=0x7fd1e0103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:42.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.493+0000 7fd1e52b2640 1 -- 172.21.15.43:0/1479821738 shutdown_connections 2024-04-21T08:38:42.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.493+0000 7fd1e52b2640 1 -- 172.21.15.43:0/1479821738 wait complete. 2024-04-21T08:38:42.503 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:38:42.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.610+0000 7f1f4812b640 1 -- 172.21.15.43:0/3003451650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f401029a0 msgr2=0x7f1f401092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:42.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.610+0000 7f1f4812b640 1 --2- 172.21.15.43:0/3003451650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f401029a0 0x7f1f401092c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009ca0 tx=0x7f1f3802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.610+0000 7f1f4812b640 1 -- 172.21.15.43:0/3003451650 shutdown_connections 2024-04-21T08:38:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.610+0000 7f1f4812b640 1 --2- 172.21.15.43:0/3003451650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f401029a0 0x7f1f401092c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.610+0000 7f1f4812b640 1 --2- 172.21.15.43:0/3003451650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f40101fe0 0x7f1f40102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.610+0000 7f1f4812b640 1 --2- 172.21.15.43:0/3003451650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f40100e90 0x7f1f401012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.610+0000 7f1f4812b640 1 -- 172.21.15.43:0/3003451650 >> 172.21.15.43:0/3003451650 conn(0x7f1f400fc400 msgr2=0x7f1f400fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.610+0000 7f1f4812b640 1 -- 172.21.15.43:0/3003451650 shutdown_connections 2024-04-21T08:38:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.610+0000 7f1f4812b640 1 -- 172.21.15.43:0/3003451650 wait complete. 2024-04-21T08:38:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.610+0000 7f1f4812b640 1 Processor -- start 2024-04-21T08:38:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.611+0000 7f1f4812b640 1 -- start start 2024-04-21T08:38:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.611+0000 7f1f4812b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f40100e90 0x7f1f40198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.611+0000 7f1f4812b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f40101fe0 0x7f1f40198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:42.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.611+0000 7f1f4812b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f401029a0 0x7f1f4019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:42.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.611+0000 7f1f4812b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4010c9f0 con 0x7f1f401029a0 2024-04-21T08:38:42.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.611+0000 7f1f4812b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4010c870 con 0x7f1f40101fe0 2024-04-21T08:38:42.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.611+0000 7f1f4812b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4010cb70 con 0x7f1f40100e90 2024-04-21T08:38:42.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.611+0000 7f1f466c8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f401029a0 0x7f1f4019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:42.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.611+0000 7f1f466c8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f401029a0 0x7f1f4019f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35966/0 (socket says 172.21.15.43:35966) 2024-04-21T08:38:42.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.611+0000 7f1f466c8640 1 -- 172.21.15.43:0/2923529134 learned_addr learned my addr 172.21.15.43:0/2923529134 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:42.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.611+0000 7f1f45ec7640 1 --2- 172.21.15.43:0/2923529134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f40100e90 0x7f1f40198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:42.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.611+0000 7f1f456c6640 1 --2- 172.21.15.43:0/2923529134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f40101fe0 0x7f1f40198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:42.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.611+0000 7f1f466c8640 1 -- 172.21.15.43:0/2923529134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f40100e90 msgr2=0x7f1f40198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:42.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.611+0000 7f1f466c8640 1 --2- 172.21.15.43:0/2923529134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f40100e90 0x7f1f40198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.611+0000 7f1f466c8640 1 -- 172.21.15.43:0/2923529134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f40101fe0 msgr2=0x7f1f40198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:42.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.611+0000 7f1f466c8640 1 --2- 172.21.15.43:0/2923529134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f40101fe0 0x7f1f40198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.611+0000 7f1f466c8640 1 -- 172.21.15.43:0/2923529134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4019fe70 con 0x7f1f401029a0 2024-04-21T08:38:42.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.611+0000 7f1f466c8640 1 --2- 172.21.15.43:0/2923529134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f401029a0 0x7f1f4019f800 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1f38005e30 tx=0x7f1f38034830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:42.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.611+0000 7f1f36ffd640 1 -- 172.21.15.43:0/2923529134 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38039040 con 0x7f1f401029a0 2024-04-21T08:38:42.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.612+0000 7f1f36ffd640 1 -- 172.21.15.43:0/2923529134 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f38035c30 con 0x7f1f401029a0 2024-04-21T08:38:42.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.612+0000 7f1f4812b640 1 -- 172.21.15.43:0/2923529134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f401a0100 con 0x7f1f401029a0 2024-04-21T08:38:42.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.612+0000 7f1f36ffd640 1 -- 172.21.15.43:0/2923529134 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f380042a0 con 0x7f1f401029a0 2024-04-21T08:38:42.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.612+0000 7f1f4812b640 1 -- 172.21.15.43:0/2923529134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f401a0640 con 0x7f1f401029a0 2024-04-21T08:38:42.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.613+0000 7f1f4812b640 1 -- 172.21.15.43:0/2923529134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f401a08a0 con 0x7f1f401029a0 2024-04-21T08:38:42.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.614+0000 7f1f36ffd640 1 -- 172.21.15.43:0/2923529134 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1f38046030 con 0x7f1f401029a0 2024-04-21T08:38:42.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.614+0000 7f1f36ffd640 1 --2- 172.21.15.43:0/2923529134 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f1f2c0a3c30 0x7f1f2c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:42.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.614+0000 7f1f45ec7640 1 --2- 172.21.15.43:0/2923529134 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f1f2c0a3c30 0x7f1f2c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:42.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.614+0000 7f1f36ffd640 1 -- 172.21.15.43:0/2923529134 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f1f380f2120 con 0x7f1f401029a0 2024-04-21T08:38:42.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.615+0000 7f1f45ec7640 1 --2- 172.21.15.43:0/2923529134 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f1f2c0a3c30 0x7f1f2c0a60f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c005fd0 tx=0x7f1f3c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:42.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.619+0000 7f1f36ffd640 1 -- 172.21.15.43:0/2923529134 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f3804d060 con 0x7f1f401029a0 2024-04-21T08:38:42.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:42 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/694410195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:42.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:42 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3485650426' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:42.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:42 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/207662175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:42.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:42 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/694410195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:42.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:42 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3485650426' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:42.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:42 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/207662175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:42.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.805+0000 7f1f4812b640 1 -- 172.21.15.43:0/2923529134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1f401a0b10 con 0x7f1f401029a0 2024-04-21T08:38:42.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.805+0000 7f1f36ffd640 1 -- 172.21.15.43:0/2923529134 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f1f380be7a0 con 0x7f1f401029a0 2024-04-21T08:38:42.804 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:38:42.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.807+0000 7f1f4812b640 1 -- 172.21.15.43:0/2923529134 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f1f2c0a3c30 msgr2=0x7f1f2c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:42.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.807+0000 7f1f4812b640 1 --2- 172.21.15.43:0/2923529134 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f1f2c0a3c30 0x7f1f2c0a60f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c005fd0 tx=0x7f1f3c00b390 comp rx=0 tx=0).stop 2024-04-21T08:38:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.807+0000 7f1f4812b640 1 -- 172.21.15.43:0/2923529134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f401029a0 msgr2=0x7f1f4019f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.807+0000 7f1f4812b640 1 --2- 172.21.15.43:0/2923529134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f401029a0 0x7f1f4019f800 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1f38005e30 tx=0x7f1f38034830 comp rx=0 tx=0).stop 2024-04-21T08:38:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.807+0000 7f1f4812b640 1 -- 172.21.15.43:0/2923529134 shutdown_connections 2024-04-21T08:38:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.807+0000 7f1f4812b640 1 --2- 172.21.15.43:0/2923529134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f401029a0 0x7f1f4019f800 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.807+0000 7f1f4812b640 1 --2- 172.21.15.43:0/2923529134 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f1f2c0a3c30 0x7f1f2c0a60f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.807+0000 7f1f4812b640 1 --2- 172.21.15.43:0/2923529134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f40101fe0 0x7f1f40198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.807+0000 7f1f4812b640 1 --2- 172.21.15.43:0/2923529134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f40100e90 0x7f1f40198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.807+0000 7f1f4812b640 1 -- 172.21.15.43:0/2923529134 >> 172.21.15.43:0/2923529134 conn(0x7f1f400fc400 msgr2=0x7f1f400fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.807+0000 7f1f4812b640 1 -- 172.21.15.43:0/2923529134 shutdown_connections 2024-04-21T08:38:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.807+0000 7f1f4812b640 1 -- 172.21.15.43:0/2923529134 wait complete. 2024-04-21T08:38:42.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:42 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/694410195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:42.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:42 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3485650426' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:42.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:42 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/207662175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:42.818 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:38:42.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.920+0000 7f270eef2640 1 -- 172.21.15.43:0/887332547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2708102980 msgr2=0x7f27081092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:42.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.920+0000 7f270eef2640 1 --2- 172.21.15.43:0/887332547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2708102980 0x7f27081092a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2700009ca0 tx=0x7f270002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.920+0000 7f270eef2640 1 -- 172.21.15.43:0/887332547 shutdown_connections 2024-04-21T08:38:42.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.920+0000 7f270eef2640 1 --2- 172.21.15.43:0/887332547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2708102980 0x7f27081092a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.920+0000 7f270eef2640 1 --2- 172.21.15.43:0/887332547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2708101fc0 0x7f2708102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.920+0000 7f270eef2640 1 --2- 172.21.15.43:0/887332547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2708100e70 0x7f2708101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.920+0000 7f270eef2640 1 -- 172.21.15.43:0/887332547 >> 172.21.15.43:0/887332547 conn(0x7f27080fc3e0 msgr2=0x7f27080fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:42.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.920+0000 7f270eef2640 1 -- 172.21.15.43:0/887332547 shutdown_connections 2024-04-21T08:38:42.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.920+0000 7f270eef2640 1 -- 172.21.15.43:0/887332547 wait complete. 2024-04-21T08:38:42.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.921+0000 7f270eef2640 1 Processor -- start 2024-04-21T08:38:42.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.921+0000 7f270eef2640 1 -- start start 2024-04-21T08:38:42.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.921+0000 7f270eef2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2708100e70 0x7f2708198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:42.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.921+0000 7f270eef2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2708101fc0 0x7f2708198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:42.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.921+0000 7f270eef2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2708102980 0x7f270819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:42.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.921+0000 7f270eef2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f270810db10 con 0x7f2708101fc0 2024-04-21T08:38:42.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.921+0000 7f270eef2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f270810d990 con 0x7f2708100e70 2024-04-21T08:38:42.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.921+0000 7f270eef2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f270810dc90 con 0x7f2708102980 2024-04-21T08:38:42.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.921+0000 7f2707fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2708101fc0 0x7f2708198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:42.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.921+0000 7f2707fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2708101fc0 0x7f2708198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35990/0 (socket says 172.21.15.43:35990) 2024-04-21T08:38:42.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.921+0000 7f2707fff640 1 -- 172.21.15.43:0/3299485217 learned_addr learned my addr 172.21.15.43:0/3299485217 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:42.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.921+0000 7f270cc8e640 1 --2- 172.21.15.43:0/3299485217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2708100e70 0x7f2708198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:42.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.921+0000 7f270d48f640 1 --2- 172.21.15.43:0/3299485217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2708102980 0x7f270819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:42.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.921+0000 7f2707fff640 1 -- 172.21.15.43:0/3299485217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2708102980 msgr2=0x7f270819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:42.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.921+0000 7f2707fff640 1 --2- 172.21.15.43:0/3299485217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2708102980 0x7f270819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.921+0000 7f2707fff640 1 -- 172.21.15.43:0/3299485217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2708100e70 msgr2=0x7f2708198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:42.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.921+0000 7f2707fff640 1 --2- 172.21.15.43:0/3299485217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2708100e70 0x7f2708198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.921+0000 7f2707fff640 1 -- 172.21.15.43:0/3299485217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270819fd50 con 0x7f2708101fc0 2024-04-21T08:38:42.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.921+0000 7f270d48f640 1 --2- 172.21.15.43:0/3299485217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2708102980 0x7f270819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:42.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.922+0000 7f2707fff640 1 --2- 172.21.15.43:0/3299485217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2708101fc0 0x7f2708198670 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f26f400d770 tx=0x7f26f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:42.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.922+0000 7f2705ffb640 1 -- 172.21.15.43:0/3299485217 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4017070 con 0x7f2708101fc0 2024-04-21T08:38:42.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.922+0000 7f2705ffb640 1 -- 172.21.15.43:0/3299485217 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26f4009c30 con 0x7f2708101fc0 2024-04-21T08:38:42.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.922+0000 7f270eef2640 1 -- 172.21.15.43:0/3299485217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f270819ffe0 con 0x7f2708101fc0 2024-04-21T08:38:42.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.922+0000 7f2705ffb640 1 -- 172.21.15.43:0/3299485217 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4003cc0 con 0x7f2708101fc0 2024-04-21T08:38:42.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.922+0000 7f270eef2640 1 -- 172.21.15.43:0/3299485217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27081a0500 con 0x7f2708101fc0 2024-04-21T08:38:42.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.923+0000 7f270eef2640 1 -- 172.21.15.43:0/3299485217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2708108180 con 0x7f2708101fc0 2024-04-21T08:38:42.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.924+0000 7f2705ffb640 1 -- 172.21.15.43:0/3299485217 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f26f4003e60 con 0x7f2708101fc0 2024-04-21T08:38:42.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.924+0000 7f2705ffb640 1 --2- 172.21.15.43:0/3299485217 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f26f80a3bb0 0x7f26f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:42.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.925+0000 7f270cc8e640 1 --2- 172.21.15.43:0/3299485217 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f26f80a3bb0 0x7f26f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:42.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.925+0000 7f2705ffb640 1 -- 172.21.15.43:0/3299485217 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f26f40d0240 con 0x7f2708101fc0 2024-04-21T08:38:42.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.925+0000 7f270cc8e640 1 --2- 172.21.15.43:0/3299485217 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f26f80a3bb0 0x7f26f80a6070 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f26fc005fd0 tx=0x7f26fc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:42.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:42.929+0000 7f2705ffb640 1 -- 172.21.15.43:0/3299485217 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f26f409c830 con 0x7f2708101fc0 2024-04-21T08:38:43.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.156+0000 7f270eef2640 1 -- 172.21.15.43:0/3299485217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2708062c50 con 0x7f2708101fc0 2024-04-21T08:38:43.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.157+0000 7f2705ffb640 1 -- 172.21.15.43:0/3299485217 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11938 (secure 0 0 0) 0x7f26f4009de0 con 0x7f2708101fc0 2024-04-21T08:38:43.156 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:43.156 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:36:51.902455+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24491,24527,24487,24425,24412]},"id":1}]} 2024-04-21T08:38:43.157 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 27 2024-04-21T08:38:43.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.159+0000 7f270eef2640 1 -- 172.21.15.43:0/3299485217 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f26f80a3bb0 msgr2=0x7f26f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:43.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.159+0000 7f270eef2640 1 --2- 172.21.15.43:0/3299485217 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f26f80a3bb0 0x7f26f80a6070 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f26fc005fd0 tx=0x7f26fc005850 comp rx=0 tx=0).stop 2024-04-21T08:38:43.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.159+0000 7f270eef2640 1 -- 172.21.15.43:0/3299485217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2708101fc0 msgr2=0x7f2708198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:43.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.159+0000 7f270eef2640 1 --2- 172.21.15.43:0/3299485217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2708101fc0 0x7f2708198670 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f26f400d770 tx=0x7f26f400dc30 comp rx=0 tx=0).stop 2024-04-21T08:38:43.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.159+0000 7f270eef2640 1 -- 172.21.15.43:0/3299485217 shutdown_connections 2024-04-21T08:38:43.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.159+0000 7f270eef2640 1 --2- 172.21.15.43:0/3299485217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2708102980 0x7f270819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.159+0000 7f270eef2640 1 --2- 172.21.15.43:0/3299485217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2708101fc0 0x7f2708198670 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.159+0000 7f270eef2640 1 --2- 172.21.15.43:0/3299485217 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f26f80a3bb0 0x7f26f80a6070 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.159+0000 7f270eef2640 1 --2- 172.21.15.43:0/3299485217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2708100e70 0x7f2708198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.159+0000 7f270eef2640 1 -- 172.21.15.43:0/3299485217 >> 172.21.15.43:0/3299485217 conn(0x7f27080fc3e0 msgr2=0x7f2708105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:43.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.159+0000 7f270eef2640 1 -- 172.21.15.43:0/3299485217 shutdown_connections 2024-04-21T08:38:43.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.160+0000 7f270eef2640 1 -- 172.21.15.43:0/3299485217 wait complete. 2024-04-21T08:38:43.170 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:38:43.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.273+0000 7fce4a6c8640 1 -- 172.21.15.43:0/2840607420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce44102050 msgr2=0x7fce441024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:43.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.273+0000 7fce4a6c8640 1 --2- 172.21.15.43:0/2840607420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce44102050 0x7fce441024d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fce38009900 tx=0x7fce3802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:43.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.273+0000 7fce4a6c8640 1 -- 172.21.15.43:0/2840607420 shutdown_connections 2024-04-21T08:38:43.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.273+0000 7fce4a6c8640 1 --2- 172.21.15.43:0/2840607420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce44102a10 0x7fce44109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.273+0000 7fce4a6c8640 1 --2- 172.21.15.43:0/2840607420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce44102050 0x7fce441024d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.273+0000 7fce4a6c8640 1 --2- 172.21.15.43:0/2840607420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce44100e70 0x7fce44101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.273+0000 7fce4a6c8640 1 -- 172.21.15.43:0/2840607420 >> 172.21.15.43:0/2840607420 conn(0x7fce440fc470 msgr2=0x7fce440fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:43.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.273+0000 7fce4a6c8640 1 -- 172.21.15.43:0/2840607420 shutdown_connections 2024-04-21T08:38:43.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.273+0000 7fce4a6c8640 1 -- 172.21.15.43:0/2840607420 wait complete. 2024-04-21T08:38:43.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.274+0000 7fce4a6c8640 1 Processor -- start 2024-04-21T08:38:43.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.274+0000 7fce4a6c8640 1 -- start start 2024-04-21T08:38:43.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.274+0000 7fce4a6c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce44100e70 0x7fce44104e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:43.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.274+0000 7fce4a6c8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce44102050 0x7fce441034c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:43.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.274+0000 7fce4a6c8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce44102a10 0x7fce44103a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:43.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.274+0000 7fce4a6c8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fce4410ca80 con 0x7fce44102a10 2024-04-21T08:38:43.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.274+0000 7fce4a6c8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fce4410c900 con 0x7fce44100e70 2024-04-21T08:38:43.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.274+0000 7fce4a6c8640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fce4410cc00 con 0x7fce44102050 2024-04-21T08:38:43.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.274+0000 7fce48c65640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce44102a10 0x7fce44103a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:43.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.274+0000 7fce48c65640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce44102a10 0x7fce44103a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36000/0 (socket says 172.21.15.43:36000) 2024-04-21T08:38:43.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.274+0000 7fce48c65640 1 -- 172.21.15.43:0/3332650037 learned_addr learned my addr 172.21.15.43:0/3332650037 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:43.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.274+0000 7fce437fe640 1 --2- 172.21.15.43:0/3332650037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce44102050 0x7fce441034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:43.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.274+0000 7fce43fff640 1 --2- 172.21.15.43:0/3332650037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce44100e70 0x7fce44104e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:43.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.274+0000 7fce48c65640 1 -- 172.21.15.43:0/3332650037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce44102050 msgr2=0x7fce441034c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:43.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.274+0000 7fce48c65640 1 --2- 172.21.15.43:0/3332650037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce44102050 0x7fce441034c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.274+0000 7fce48c65640 1 -- 172.21.15.43:0/3332650037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce44100e70 msgr2=0x7fce44104e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:43.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.274+0000 7fce48c65640 1 --2- 172.21.15.43:0/3332650037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce44100e70 0x7fce44104e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.274+0000 7fce48c65640 1 -- 172.21.15.43:0/3332650037 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce44104290 con 0x7fce44102a10 2024-04-21T08:38:43.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.275+0000 7fce48c65640 1 --2- 172.21.15.43:0/3332650037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce44102a10 0x7fce44103a00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00cce0 tx=0x7fce3c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:43.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.275+0000 7fce417fa640 1 -- 172.21.15.43:0/3332650037 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c00f450 con 0x7fce44102a10 2024-04-21T08:38:43.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.275+0000 7fce4a6c8640 1 -- 172.21.15.43:0/3332650037 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce441a22e0 con 0x7fce44102a10 2024-04-21T08:38:43.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.275+0000 7fce417fa640 1 -- 172.21.15.43:0/3332650037 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce3c00faa0 con 0x7fce44102a10 2024-04-21T08:38:43.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.275+0000 7fce417fa640 1 -- 172.21.15.43:0/3332650037 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c018a80 con 0x7fce44102a10 2024-04-21T08:38:43.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.275+0000 7fce4a6c8640 1 -- 172.21.15.43:0/3332650037 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce441a2880 con 0x7fce44102a10 2024-04-21T08:38:43.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.276+0000 7fce4a6c8640 1 -- 172.21.15.43:0/3332650037 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce44069230 con 0x7fce44102a10 2024-04-21T08:38:43.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.277+0000 7fce417fa640 1 -- 172.21.15.43:0/3332650037 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fce3c00fc50 con 0x7fce44102a10 2024-04-21T08:38:43.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.278+0000 7fce417fa640 1 --2- 172.21.15.43:0/3332650037 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fce2c0a3bb0 0x7fce2c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:43.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.278+0000 7fce417fa640 1 -- 172.21.15.43:0/3332650037 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fce3c0d0bf0 con 0x7fce44102a10 2024-04-21T08:38:43.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.278+0000 7fce43fff640 1 --2- 172.21.15.43:0/3332650037 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fce2c0a3bb0 0x7fce2c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:43.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.278+0000 7fce43fff640 1 --2- 172.21.15.43:0/3332650037 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fce2c0a3bb0 0x7fce2c0a6070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fce44101bc0 tx=0x7fce34009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:43.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.282+0000 7fce417fa640 1 -- 172.21.15.43:0/3332650037 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce3c014030 con 0x7fce44102a10 2024-04-21T08:38:43.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.471+0000 7fce4a6c8640 1 -- 172.21.15.43:0/3332650037 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce441a2d60 con 0x7fce44102a10 2024-04-21T08:38:43.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.472+0000 7fce417fa640 1 -- 172.21.15.43:0/3332650037 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24856 (secure 0 0 0) 0x7fce441a2d60 con 0x7fce44102a10 2024-04-21T08:38:43.472 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:43.472 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":94,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:38:15.326063+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:38:43.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.474+0000 7fce4a6c8640 1 -- 172.21.15.43:0/3332650037 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fce2c0a3bb0 msgr2=0x7fce2c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:43.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.474+0000 7fce4a6c8640 1 --2- 172.21.15.43:0/3332650037 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fce2c0a3bb0 0x7fce2c0a6070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fce44101bc0 tx=0x7fce34009110 comp rx=0 tx=0).stop 2024-04-21T08:38:43.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.474+0000 7fce4a6c8640 1 -- 172.21.15.43:0/3332650037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce44102a10 msgr2=0x7fce44103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:43.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.474+0000 7fce4a6c8640 1 --2- 172.21.15.43:0/3332650037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce44102a10 0x7fce44103a00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00cce0 tx=0x7fce3c00a620 comp rx=0 tx=0).stop 2024-04-21T08:38:43.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.474+0000 7fce4a6c8640 1 -- 172.21.15.43:0/3332650037 shutdown_connections 2024-04-21T08:38:43.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.474+0000 7fce4a6c8640 1 --2- 172.21.15.43:0/3332650037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce44102a10 0x7fce44103a00 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.474+0000 7fce4a6c8640 1 --2- 172.21.15.43:0/3332650037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce44102050 0x7fce441034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.474+0000 7fce4a6c8640 1 --2- 172.21.15.43:0/3332650037 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fce2c0a3bb0 0x7fce2c0a6070 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.474+0000 7fce4a6c8640 1 --2- 172.21.15.43:0/3332650037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce44100e70 0x7fce44104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.474+0000 7fce4a6c8640 1 -- 172.21.15.43:0/3332650037 >> 172.21.15.43:0/3332650037 conn(0x7fce440fc470 msgr2=0x7fce440fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:43.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.474+0000 7fce4a6c8640 1 -- 172.21.15.43:0/3332650037 shutdown_connections 2024-04-21T08:38:43.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.474+0000 7fce4a6c8640 1 -- 172.21.15.43:0/3332650037 wait complete. 2024-04-21T08:38:43.485 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:39.981405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:22.034751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:22.780795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:25.984261+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:38:43.486 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:38:43.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.592+0000 7fc65ce63640 1 -- 172.21.15.43:0/3561366939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6581031a0 msgr2=0x7fc658103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:43.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.592+0000 7fc65ce63640 1 --2- 172.21.15.43:0/3561366939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6581031a0 0x7fc658103620 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009900 tx=0x7fc64c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:43.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.592+0000 7fc65ce63640 1 -- 172.21.15.43:0/3561366939 shutdown_connections 2024-04-21T08:38:43.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.592+0000 7fc65ce63640 1 --2- 172.21.15.43:0/3561366939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc658103b60 0x7fc65810a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.592+0000 7fc65ce63640 1 --2- 172.21.15.43:0/3561366939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6581031a0 0x7fc658103620 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.592+0000 7fc65ce63640 1 --2- 172.21.15.43:0/3561366939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc658101fb0 0x7fc6581023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.592+0000 7fc65ce63640 1 -- 172.21.15.43:0/3561366939 >> 172.21.15.43:0/3561366939 conn(0x7fc6580fd520 msgr2=0x7fc6580ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:43.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.592+0000 7fc65ce63640 1 -- 172.21.15.43:0/3561366939 shutdown_connections 2024-04-21T08:38:43.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.592+0000 7fc65ce63640 1 -- 172.21.15.43:0/3561366939 wait complete. 2024-04-21T08:38:43.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.593+0000 7fc65ce63640 1 Processor -- start 2024-04-21T08:38:43.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.593+0000 7fc65ce63640 1 -- start start 2024-04-21T08:38:43.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.593+0000 7fc65ce63640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc658101fb0 0x7fc658075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:43.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.593+0000 7fc65ce63640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6581031a0 0x7fc658074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:43.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.593+0000 7fc65ce63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc658103b60 0x7fc658074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:43.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.593+0000 7fc65ce63640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc65810daf0 con 0x7fc658103b60 2024-04-21T08:38:43.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.593+0000 7fc65ce63640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc65810d970 con 0x7fc658101fb0 2024-04-21T08:38:43.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.593+0000 7fc65ce63640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc65810dc70 con 0x7fc6581031a0 2024-04-21T08:38:43.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.593+0000 7fc656d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc658103b60 0x7fc658074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:43.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.593+0000 7fc656d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc658103b60 0x7fc658074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36018/0 (socket says 172.21.15.43:36018) 2024-04-21T08:38:43.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.593+0000 7fc656d9d640 1 -- 172.21.15.43:0/281359621 learned_addr learned my addr 172.21.15.43:0/281359621 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:43.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.593+0000 7fc65659c640 1 --2- 172.21.15.43:0/281359621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc658101fb0 0x7fc658075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:43.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.593+0000 7fc655d9b640 1 --2- 172.21.15.43:0/281359621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6581031a0 0x7fc658074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:43.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.594+0000 7fc656d9d640 1 -- 172.21.15.43:0/281359621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6581031a0 msgr2=0x7fc658074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:43.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.594+0000 7fc656d9d640 1 --2- 172.21.15.43:0/281359621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6581031a0 0x7fc658074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.594+0000 7fc656d9d640 1 -- 172.21.15.43:0/281359621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc658101fb0 msgr2=0x7fc658075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:43.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.594+0000 7fc656d9d640 1 --2- 172.21.15.43:0/281359621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc658101fb0 0x7fc658075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.594+0000 7fc656d9d640 1 -- 172.21.15.43:0/281359621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc658075110 con 0x7fc658103b60 2024-04-21T08:38:43.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.594+0000 7fc656d9d640 1 --2- 172.21.15.43:0/281359621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc658103b60 0x7fc658074880 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc65000eaa0 tx=0x7fc65000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:43.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.594+0000 7fc6477fe640 1 -- 172.21.15.43:0/281359621 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65000cdb0 con 0x7fc658103b60 2024-04-21T08:38:43.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.594+0000 7fc65ce63640 1 -- 172.21.15.43:0/281359621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6581a22c0 con 0x7fc658103b60 2024-04-21T08:38:43.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.594+0000 7fc6477fe640 1 -- 172.21.15.43:0/281359621 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc650005020 con 0x7fc658103b60 2024-04-21T08:38:43.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.594+0000 7fc6477fe640 1 -- 172.21.15.43:0/281359621 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6500116d0 con 0x7fc658103b60 2024-04-21T08:38:43.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.594+0000 7fc65ce63640 1 -- 172.21.15.43:0/281359621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6581a2860 con 0x7fc658103b60 2024-04-21T08:38:43.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.595+0000 7fc65ce63640 1 -- 172.21.15.43:0/281359621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc658109350 con 0x7fc658103b60 2024-04-21T08:38:43.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.596+0000 7fc6477fe640 1 -- 172.21.15.43:0/281359621 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc6500054a0 con 0x7fc658103b60 2024-04-21T08:38:43.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.597+0000 7fc6477fe640 1 --2- 172.21.15.43:0/281359621 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc6400a3bb0 0x7fc6400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:43.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.597+0000 7fc65659c640 1 --2- 172.21.15.43:0/281359621 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc6400a3bb0 0x7fc6400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:43.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.597+0000 7fc6477fe640 1 -- 172.21.15.43:0/281359621 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fc6500d0a10 con 0x7fc658103b60 2024-04-21T08:38:43.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.598+0000 7fc65659c640 1 --2- 172.21.15.43:0/281359621 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc6400a3bb0 0x7fc6400a6070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc658103000 tx=0x7fc648005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.602+0000 7fc6477fe640 1 -- 172.21.15.43:0/281359621 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc650016030 con 0x7fc658103b60 2024-04-21T08:38:43.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:43 smithi175 ceph-mon[31920]: pgmap v368: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:38:43.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:43 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1479821738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:43.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:43 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2923529134' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:43.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:43 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3299485217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:43.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.791+0000 7fc65ce63640 1 -- 172.21.15.43:0/281359621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc658062b50 con 0x7fc658103b60 2024-04-21T08:38:43.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.791+0000 7fc6477fe640 1 -- 172.21.15.43:0/281359621 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fc65009cfa0 con 0x7fc658103b60 2024-04-21T08:38:43.791 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-21T08:38:43.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.794+0000 7fc65ce63640 1 -- 172.21.15.43:0/281359621 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc6400a3bb0 msgr2=0x7fc6400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:43.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.794+0000 7fc65ce63640 1 --2- 172.21.15.43:0/281359621 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc6400a3bb0 0x7fc6400a6070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc658103000 tx=0x7fc648005f90 comp rx=0 tx=0).stop 2024-04-21T08:38:43.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.794+0000 7fc65ce63640 1 -- 172.21.15.43:0/281359621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc658103b60 msgr2=0x7fc658074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:43.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.794+0000 7fc65ce63640 1 --2- 172.21.15.43:0/281359621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc658103b60 0x7fc658074880 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc65000eaa0 tx=0x7fc65000ef60 comp rx=0 tx=0).stop 2024-04-21T08:38:43.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.794+0000 7fc65ce63640 1 -- 172.21.15.43:0/281359621 shutdown_connections 2024-04-21T08:38:43.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.794+0000 7fc65ce63640 1 --2- 172.21.15.43:0/281359621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc658103b60 0x7fc658074880 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.794+0000 7fc65ce63640 1 --2- 172.21.15.43:0/281359621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6581031a0 0x7fc658074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.794+0000 7fc65ce63640 1 --2- 172.21.15.43:0/281359621 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fc6400a3bb0 0x7fc6400a6070 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.794+0000 7fc65ce63640 1 --2- 172.21.15.43:0/281359621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc658101fb0 0x7fc658075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.794+0000 7fc65ce63640 1 -- 172.21.15.43:0/281359621 >> 172.21.15.43:0/281359621 conn(0x7fc6580fd520 msgr2=0x7fc658106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:43.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.794+0000 7fc65ce63640 1 -- 172.21.15.43:0/281359621 shutdown_connections 2024-04-21T08:38:43.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.794+0000 7fc65ce63640 1 -- 172.21.15.43:0/281359621 wait complete. 2024-04-21T08:38:43.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:43 smithi112 ceph-mon[31085]: pgmap v368: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:38:43.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:43 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1479821738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:43.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:43 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2923529134' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:43.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:43 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3299485217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:43.803 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:43 smithi043 ceph-mon[27978]: pgmap v368: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:38:43.803 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:43 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1479821738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:43.803 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:43 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2923529134' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:43.803 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:43 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3299485217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:43.804 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:38:43.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.910+0000 7f395e08c640 1 -- 172.21.15.43:0/776068629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3958100e90 msgr2=0x7f39581012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:43.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.910+0000 7f395e08c640 1 --2- 172.21.15.43:0/776068629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3958100e90 0x7f39581012b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3948009900 tx=0x7f394802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:43.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.910+0000 7f395e08c640 1 -- 172.21.15.43:0/776068629 shutdown_connections 2024-04-21T08:38:43.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.910+0000 7f395e08c640 1 --2- 172.21.15.43:0/776068629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39581029a0 0x7f39581092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.910+0000 7f395e08c640 1 --2- 172.21.15.43:0/776068629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3958101fe0 0x7f3958102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.910+0000 7f395e08c640 1 --2- 172.21.15.43:0/776068629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3958100e90 0x7f39581012b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.910+0000 7f395e08c640 1 -- 172.21.15.43:0/776068629 >> 172.21.15.43:0/776068629 conn(0x7f39580fc400 msgr2=0x7f39580fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:43.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.910+0000 7f395e08c640 1 -- 172.21.15.43:0/776068629 shutdown_connections 2024-04-21T08:38:43.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.910+0000 7f395e08c640 1 -- 172.21.15.43:0/776068629 wait complete. 2024-04-21T08:38:43.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.910+0000 7f395e08c640 1 Processor -- start 2024-04-21T08:38:43.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.910+0000 7f395e08c640 1 -- start start 2024-04-21T08:38:43.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.911+0000 7f395e08c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3958100e90 0x7f3958198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:43.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.911+0000 7f395e08c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3958101fe0 0x7f3958198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:43.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.911+0000 7f395e08c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39581029a0 0x7f395819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:43.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.911+0000 7f395e08c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f395810daf0 con 0x7f3958101fe0 2024-04-21T08:38:43.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.911+0000 7f395e08c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f395810d970 con 0x7f39581029a0 2024-04-21T08:38:43.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.911+0000 7f395e08c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f395810dc70 con 0x7f3958100e90 2024-04-21T08:38:43.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.911+0000 7f394effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3958101fe0 0x7f3958198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:43.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.911+0000 7f394effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3958101fe0 0x7f3958198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36036/0 (socket says 172.21.15.43:36036) 2024-04-21T08:38:43.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.911+0000 7f394effd640 1 -- 172.21.15.43:0/1531839152 learned_addr learned my addr 172.21.15.43:0/1531839152 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:43.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.911+0000 7f3957fff640 1 --2- 172.21.15.43:0/1531839152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39581029a0 0x7f395819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:43.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.911+0000 7f39577fe640 1 --2- 172.21.15.43:0/1531839152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3958100e90 0x7f3958198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:43.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.911+0000 7f394effd640 1 -- 172.21.15.43:0/1531839152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3958100e90 msgr2=0x7f3958198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:43.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.911+0000 7f394effd640 1 --2- 172.21.15.43:0/1531839152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3958100e90 0x7f3958198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.911+0000 7f394effd640 1 -- 172.21.15.43:0/1531839152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39581029a0 msgr2=0x7f395819f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:43.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.911+0000 7f394effd640 1 --2- 172.21.15.43:0/1531839152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39581029a0 0x7f395819f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:43.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.911+0000 7f394effd640 1 -- 172.21.15.43:0/1531839152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f395819fe50 con 0x7f3958101fe0 2024-04-21T08:38:43.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.911+0000 7f394effd640 1 --2- 172.21.15.43:0/1531839152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3958101fe0 0x7f3958198770 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f394000c970 tx=0x7f394000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:43.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.911+0000 7f39557fa640 1 -- 172.21.15.43:0/1531839152 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940004220 con 0x7f3958101fe0 2024-04-21T08:38:43.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.911+0000 7f39557fa640 1 -- 172.21.15.43:0/1531839152 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39400043c0 con 0x7f3958101fe0 2024-04-21T08:38:43.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.911+0000 7f395e08c640 1 -- 172.21.15.43:0/1531839152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39581a0140 con 0x7f3958101fe0 2024-04-21T08:38:43.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.911+0000 7f39557fa640 1 -- 172.21.15.43:0/1531839152 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394000f450 con 0x7f3958101fe0 2024-04-21T08:38:43.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.911+0000 7f395e08c640 1 -- 172.21.15.43:0/1531839152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39581a0690 con 0x7f3958101fe0 2024-04-21T08:38:43.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.912+0000 7f395e08c640 1 -- 172.21.15.43:0/1531839152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3958108310 con 0x7f3958101fe0 2024-04-21T08:38:43.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.914+0000 7f39557fa640 1 -- 172.21.15.43:0/1531839152 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3940003610 con 0x7f3958101fe0 2024-04-21T08:38:43.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.914+0000 7f39557fa640 1 --2- 172.21.15.43:0/1531839152 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f39440a3bb0 0x7f39440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:43.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.914+0000 7f39557fa640 1 -- 172.21.15.43:0/1531839152 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f39400d0a70 con 0x7f3958101fe0 2024-04-21T08:38:43.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.914+0000 7f39577fe640 1 --2- 172.21.15.43:0/1531839152 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f39440a3bb0 0x7f39440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:43.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.915+0000 7f39577fe640 1 --2- 172.21.15.43:0/1531839152 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f39440a3bb0 0x7f39440a6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f39480096c0 tx=0x7f3948000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:43.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:43.918+0000 7f39557fa640 1 -- 172.21.15.43:0/1531839152 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3940014030 con 0x7f3958101fe0 2024-04-21T08:38:44.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.106+0000 7f395e08c640 1 -- 172.21.15.43:0/1531839152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f395802d810 con 0x7f3958101fe0 2024-04-21T08:38:44.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.106+0000 7f39557fa640 1 -- 172.21.15.43:0/1531839152 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f394009d060 con 0x7f3958101fe0 2024-04-21T08:38:44.106 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:38:44.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.108+0000 7f395e08c640 1 -- 172.21.15.43:0/1531839152 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f39440a3bb0 msgr2=0x7f39440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.108+0000 7f395e08c640 1 --2- 172.21.15.43:0/1531839152 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f39440a3bb0 0x7f39440a6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f39480096c0 tx=0x7f3948000bc0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.108+0000 7f395e08c640 1 -- 172.21.15.43:0/1531839152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3958101fe0 msgr2=0x7f3958198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.108+0000 7f395e08c640 1 --2- 172.21.15.43:0/1531839152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3958101fe0 0x7f3958198770 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f394000c970 tx=0x7f394000ce30 comp rx=0 tx=0).stop 2024-04-21T08:38:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.108+0000 7f395e08c640 1 -- 172.21.15.43:0/1531839152 shutdown_connections 2024-04-21T08:38:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.108+0000 7f395e08c640 1 --2- 172.21.15.43:0/1531839152 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f39440a3bb0 0x7f39440a6070 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.108+0000 7f395e08c640 1 --2- 172.21.15.43:0/1531839152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39581029a0 0x7f395819f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.108+0000 7f395e08c640 1 --2- 172.21.15.43:0/1531839152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3958101fe0 0x7f3958198770 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.108+0000 7f395e08c640 1 --2- 172.21.15.43:0/1531839152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3958100e90 0x7f3958198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.108+0000 7f395e08c640 1 -- 172.21.15.43:0/1531839152 >> 172.21.15.43:0/1531839152 conn(0x7f39580fc400 msgr2=0x7f39581054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.108+0000 7f395e08c640 1 -- 172.21.15.43:0/1531839152 shutdown_connections 2024-04-21T08:38:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.108+0000 7f395e08c640 1 -- 172.21.15.43:0/1531839152 wait complete. 2024-04-21T08:38:44.119 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:38:44.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.225+0000 7f8885952640 1 -- 172.21.15.43:0/1055788994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8880103180 msgr2=0x7f8880103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:44.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.225+0000 7f8885952640 1 --2- 172.21.15.43:0/1055788994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8880103180 0x7f8880103600 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8874009900 tx=0x7f887402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:44.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.225+0000 7f8885952640 1 -- 172.21.15.43:0/1055788994 shutdown_connections 2024-04-21T08:38:44.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.225+0000 7f8885952640 1 --2- 172.21.15.43:0/1055788994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8880103b40 0x7f888010a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.225+0000 7f8885952640 1 --2- 172.21.15.43:0/1055788994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8880103180 0x7f8880103600 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.225+0000 7f8885952640 1 --2- 172.21.15.43:0/1055788994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8880101f90 0x7f88801023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.225+0000 7f8885952640 1 -- 172.21.15.43:0/1055788994 >> 172.21.15.43:0/1055788994 conn(0x7f88800fd500 msgr2=0x7f88800ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:44.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.225+0000 7f8885952640 1 -- 172.21.15.43:0/1055788994 shutdown_connections 2024-04-21T08:38:44.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.225+0000 7f8885952640 1 -- 172.21.15.43:0/1055788994 wait complete. 2024-04-21T08:38:44.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.226+0000 7f8885952640 1 Processor -- start 2024-04-21T08:38:44.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.226+0000 7f8885952640 1 -- start start 2024-04-21T08:38:44.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.226+0000 7f8885952640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8880101f90 0x7f88801981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:44.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.226+0000 7f8885952640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8880103180 0x7f8880198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:44.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.226+0000 7f8885952640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8880103b40 0x7f888019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:44.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.226+0000 7f8885952640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f888010dad0 con 0x7f8880101f90 2024-04-21T08:38:44.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.226+0000 7f8885952640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f888010d950 con 0x7f8880103b40 2024-04-21T08:38:44.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.226+0000 7f8885952640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f888010dc50 con 0x7f8880103180 2024-04-21T08:38:44.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.226+0000 7f887effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8880101f90 0x7f88801981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:44.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.226+0000 7f887effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8880101f90 0x7f88801981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36058/0 (socket says 172.21.15.43:36058) 2024-04-21T08:38:44.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.226+0000 7f887effd640 1 -- 172.21.15.43:0/3585613632 learned_addr learned my addr 172.21.15.43:0/3585613632 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:44.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.227+0000 7f887effd640 1 -- 172.21.15.43:0/3585613632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8880103180 msgr2=0x7f8880198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:44.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.227+0000 7f887effd640 1 --2- 172.21.15.43:0/3585613632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8880103180 0x7f8880198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.227+0000 7f887effd640 1 -- 172.21.15.43:0/3585613632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8880103b40 msgr2=0x7f888019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:44.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.227+0000 7f887effd640 1 --2- 172.21.15.43:0/3585613632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8880103b40 0x7f888019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.227+0000 7f887effd640 1 -- 172.21.15.43:0/3585613632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888019fe00 con 0x7f8880101f90 2024-04-21T08:38:44.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.227+0000 7f887effd640 1 --2- 172.21.15.43:0/3585613632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8880101f90 0x7f88801981e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f887000c970 tx=0x7f887000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:44.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.227+0000 7f8884950640 1 -- 172.21.15.43:0/3585613632 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8870004220 con 0x7f8880101f90 2024-04-21T08:38:44.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.227+0000 7f8884950640 1 -- 172.21.15.43:0/3585613632 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88700043c0 con 0x7f8880101f90 2024-04-21T08:38:44.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.227+0000 7f8884950640 1 -- 172.21.15.43:0/3585613632 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887000f450 con 0x7f8880101f90 2024-04-21T08:38:44.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.227+0000 7f8885952640 1 -- 172.21.15.43:0/3585613632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88801a00f0 con 0x7f8880101f90 2024-04-21T08:38:44.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.227+0000 7f8885952640 1 -- 172.21.15.43:0/3585613632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88801a0690 con 0x7f8880101f90 2024-04-21T08:38:44.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.228+0000 7f8885952640 1 -- 172.21.15.43:0/3585613632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8880109420 con 0x7f8880101f90 2024-04-21T08:38:44.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.229+0000 7f8884950640 1 -- 172.21.15.43:0/3585613632 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8870003610 con 0x7f8880101f90 2024-04-21T08:38:44.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.230+0000 7f8884950640 1 --2- 172.21.15.43:0/3585613632 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f88680a3c30 0x7f88680a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:44.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.230+0000 7f8884950640 1 -- 172.21.15.43:0/3585613632 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f88700d06a0 con 0x7f8880101f90 2024-04-21T08:38:44.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.230+0000 7f887e7fc640 1 --2- 172.21.15.43:0/3585613632 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f88680a3c30 0x7f88680a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:44.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.231+0000 7f887e7fc640 1 --2- 172.21.15.43:0/3585613632 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f88680a3c30 0x7f88680a60f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8874036040 tx=0x7f8874000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:44.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.234+0000 7f8884950640 1 -- 172.21.15.43:0/3585613632 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8870014030 con 0x7f8880101f90 2024-04-21T08:38:44.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.420+0000 7f8885952640 1 -- 172.21.15.43:0/3585613632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8880062c50 con 0x7f8880101f90 2024-04-21T08:38:44.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.420+0000 7f8884950640 1 -- 172.21.15.43:0/3585613632 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f887009cc90 con 0x7f8880101f90 2024-04-21T08:38:44.420 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-21T08:38:44.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.422+0000 7f8885952640 1 -- 172.21.15.43:0/3585613632 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f88680a3c30 msgr2=0x7f88680a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:44.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.422+0000 7f8885952640 1 --2- 172.21.15.43:0/3585613632 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f88680a3c30 0x7f88680a60f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8874036040 tx=0x7f8874000bc0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.422+0000 7f8885952640 1 -- 172.21.15.43:0/3585613632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8880101f90 msgr2=0x7f88801981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:44.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.422+0000 7f8885952640 1 --2- 172.21.15.43:0/3585613632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8880101f90 0x7f88801981e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f887000c970 tx=0x7f887000ce30 comp rx=0 tx=0).stop 2024-04-21T08:38:44.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.422+0000 7f8885952640 1 -- 172.21.15.43:0/3585613632 shutdown_connections 2024-04-21T08:38:44.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.422+0000 7f8885952640 1 --2- 172.21.15.43:0/3585613632 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f88680a3c30 0x7f88680a60f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.422+0000 7f8885952640 1 --2- 172.21.15.43:0/3585613632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8880103b40 0x7f888019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.422+0000 7f8885952640 1 --2- 172.21.15.43:0/3585613632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8880103180 0x7f8880198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.422+0000 7f8885952640 1 --2- 172.21.15.43:0/3585613632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8880101f90 0x7f88801981e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.422+0000 7f8885952640 1 -- 172.21.15.43:0/3585613632 >> 172.21.15.43:0/3585613632 conn(0x7f88800fd500 msgr2=0x7f88801065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:44.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.423+0000 7f8885952640 1 -- 172.21.15.43:0/3585613632 shutdown_connections 2024-04-21T08:38:44.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.423+0000 7f8885952640 1 -- 172.21.15.43:0/3585613632 wait complete. 2024-04-21T08:38:44.433 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:38:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.542+0000 7f613006d640 1 -- 172.21.15.43:0/1023335838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61281031a0 msgr2=0x7f6128103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.542+0000 7f613006d640 1 --2- 172.21.15.43:0/1023335838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61281031a0 0x7f6128103620 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6118009900 tx=0x7f611802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.542+0000 7f613006d640 1 -- 172.21.15.43:0/1023335838 shutdown_connections 2024-04-21T08:38:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.542+0000 7f613006d640 1 --2- 172.21.15.43:0/1023335838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6128103b60 0x7f612810a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.542+0000 7f613006d640 1 --2- 172.21.15.43:0/1023335838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61281031a0 0x7f6128103620 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.542+0000 7f613006d640 1 --2- 172.21.15.43:0/1023335838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6128101fb0 0x7f61281023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.542+0000 7f613006d640 1 -- 172.21.15.43:0/1023335838 >> 172.21.15.43:0/1023335838 conn(0x7f61280fd520 msgr2=0x7f61280ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.542+0000 7f613006d640 1 -- 172.21.15.43:0/1023335838 shutdown_connections 2024-04-21T08:38:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.542+0000 7f613006d640 1 -- 172.21.15.43:0/1023335838 wait complete. 2024-04-21T08:38:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.543+0000 7f613006d640 1 Processor -- start 2024-04-21T08:38:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.543+0000 7f613006d640 1 -- start start 2024-04-21T08:38:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.543+0000 7f613006d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6128101fb0 0x7f6128075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.543+0000 7f613006d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61281031a0 0x7f6128074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.543+0000 7f613006d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6128103b60 0x7f6128074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.543+0000 7f613006d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f612810daf0 con 0x7f6128103b60 2024-04-21T08:38:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.543+0000 7f613006d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f612810d970 con 0x7f6128101fb0 2024-04-21T08:38:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.543+0000 7f613006d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f612810dc70 con 0x7f61281031a0 2024-04-21T08:38:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.543+0000 7f612e60a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6128103b60 0x7f6128074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.543+0000 7f612e60a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6128103b60 0x7f6128074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36076/0 (socket says 172.21.15.43:36076) 2024-04-21T08:38:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.543+0000 7f612e60a640 1 -- 172.21.15.43:0/2784163321 learned_addr learned my addr 172.21.15.43:0/2784163321 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.543+0000 7f612de09640 1 --2- 172.21.15.43:0/2784163321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6128101fb0 0x7f6128075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.543+0000 7f612d608640 1 --2- 172.21.15.43:0/2784163321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61281031a0 0x7f6128074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.543+0000 7f612e60a640 1 -- 172.21.15.43:0/2784163321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61281031a0 msgr2=0x7f6128074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.543+0000 7f612e60a640 1 --2- 172.21.15.43:0/2784163321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61281031a0 0x7f6128074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.543+0000 7f612e60a640 1 -- 172.21.15.43:0/2784163321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6128101fb0 msgr2=0x7f6128075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.543+0000 7f612e60a640 1 --2- 172.21.15.43:0/2784163321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6128101fb0 0x7f6128075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.544+0000 7f612e60a640 1 -- 172.21.15.43:0/2784163321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6128075110 con 0x7f6128103b60 2024-04-21T08:38:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.544+0000 7f612d608640 1 --2- 172.21.15.43:0/2784163321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61281031a0 0x7f6128074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.544+0000 7f612de09640 1 --2- 172.21.15.43:0/2784163321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6128101fb0 0x7f6128075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.544+0000 7f612e60a640 1 --2- 172.21.15.43:0/2784163321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6128103b60 0x7f6128074880 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f612000eaa0 tx=0x7f612000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.544+0000 7f611effd640 1 -- 172.21.15.43:0/2784163321 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612000cdb0 con 0x7f6128103b60 2024-04-21T08:38:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.544+0000 7f611effd640 1 -- 172.21.15.43:0/2784163321 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6120005020 con 0x7f6128103b60 2024-04-21T08:38:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.544+0000 7f613006d640 1 -- 172.21.15.43:0/2784163321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61281a22e0 con 0x7f6128103b60 2024-04-21T08:38:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.544+0000 7f611effd640 1 -- 172.21.15.43:0/2784163321 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61200116d0 con 0x7f6128103b60 2024-04-21T08:38:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.544+0000 7f613006d640 1 -- 172.21.15.43:0/2784163321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61281a2880 con 0x7f6128103b60 2024-04-21T08:38:44.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.545+0000 7f613006d640 1 -- 172.21.15.43:0/2784163321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61281093a0 con 0x7f6128103b60 2024-04-21T08:38:44.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.546+0000 7f611effd640 1 -- 172.21.15.43:0/2784163321 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f61200051d0 con 0x7f6128103b60 2024-04-21T08:38:44.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.547+0000 7f611effd640 1 --2- 172.21.15.43:0/2784163321 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f61140a3c30 0x7f61140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:44.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.547+0000 7f611effd640 1 -- 172.21.15.43:0/2784163321 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f61200d0e10 con 0x7f6128103b60 2024-04-21T08:38:44.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.547+0000 7f612de09640 1 --2- 172.21.15.43:0/2784163321 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f61140a3c30 0x7f61140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:44.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.547+0000 7f612de09640 1 --2- 172.21.15.43:0/2784163321 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f61140a3c30 0x7f61140a60f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6128103000 tx=0x7f612400a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:44.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.550+0000 7f611effd640 1 -- 172.21.15.43:0/2784163321 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6120016030 con 0x7f6128103b60 2024-04-21T08:38:44.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:44 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3332650037' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:44.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:44 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/281359621' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:44.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:44 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1531839152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:44.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:44 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3585613632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:44.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.737+0000 7f613006d640 1 -- 172.21.15.43:0/2784163321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f612802d5c0 con 0x7f6128103b60 2024-04-21T08:38:44.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.738+0000 7f611effd640 1 -- 172.21.15.43:0/2784163321 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f612009d480 con 0x7f6128103b60 2024-04-21T08:38:44.737 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:38:44.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.740+0000 7f613006d640 1 -- 172.21.15.43:0/2784163321 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f61140a3c30 msgr2=0x7f61140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:44.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.740+0000 7f613006d640 1 --2- 172.21.15.43:0/2784163321 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f61140a3c30 0x7f61140a60f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6128103000 tx=0x7f612400a560 comp rx=0 tx=0).stop 2024-04-21T08:38:44.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.740+0000 7f613006d640 1 -- 172.21.15.43:0/2784163321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6128103b60 msgr2=0x7f6128074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:44.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.740+0000 7f613006d640 1 --2- 172.21.15.43:0/2784163321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6128103b60 0x7f6128074880 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f612000eaa0 tx=0x7f612000ef60 comp rx=0 tx=0).stop 2024-04-21T08:38:44.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.740+0000 7f613006d640 1 -- 172.21.15.43:0/2784163321 shutdown_connections 2024-04-21T08:38:44.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.740+0000 7f613006d640 1 --2- 172.21.15.43:0/2784163321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6128103b60 0x7f6128074880 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.740+0000 7f613006d640 1 --2- 172.21.15.43:0/2784163321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61281031a0 0x7f6128074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.740+0000 7f613006d640 1 --2- 172.21.15.43:0/2784163321 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f61140a3c30 0x7f61140a60f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.740+0000 7f613006d640 1 --2- 172.21.15.43:0/2784163321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6128101fb0 0x7f6128075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.740+0000 7f613006d640 1 -- 172.21.15.43:0/2784163321 >> 172.21.15.43:0/2784163321 conn(0x7f61280fd520 msgr2=0x7f6128106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:44.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.740+0000 7f613006d640 1 -- 172.21.15.43:0/2784163321 shutdown_connections 2024-04-21T08:38:44.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.740+0000 7f613006d640 1 -- 172.21.15.43:0/2784163321 wait complete. 2024-04-21T08:38:44.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:44 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3332650037' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:44.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:44 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/281359621' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:44.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:44 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1531839152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:44.749 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:44 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3585613632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:44.751 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:38:44.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:44 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3332650037' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:44.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:44 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/281359621' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:44.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:44 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1531839152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:44.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:44 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3585613632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:44.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.853+0000 7f98b7e14640 1 -- 172.21.15.43:0/3690857010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98b0107d60 msgr2=0x7f98b010a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:44.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.853+0000 7f98b7e14640 1 --2- 172.21.15.43:0/3690857010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98b0107d60 0x7f98b010a150 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009ca0 tx=0x7f98a802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.853+0000 7f98b7e14640 1 -- 172.21.15.43:0/3690857010 shutdown_connections 2024-04-21T08:38:44.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.853+0000 7f98b7e14640 1 --2- 172.21.15.43:0/3690857010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98b0107d60 0x7f98b010a150 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.853+0000 7f98b7e14640 1 --2- 172.21.15.43:0/3690857010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98b0105430 0x7f98b0107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.853+0000 7f98b7e14640 1 --2- 172.21.15.43:0/3690857010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98b0102b00 0x7f98b0104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.853+0000 7f98b7e14640 1 -- 172.21.15.43:0/3690857010 >> 172.21.15.43:0/3690857010 conn(0x7f98b00fc470 msgr2=0x7f98b00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:44.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.853+0000 7f98b7e14640 1 -- 172.21.15.43:0/3690857010 shutdown_connections 2024-04-21T08:38:44.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.853+0000 7f98b7e14640 1 -- 172.21.15.43:0/3690857010 wait complete. 2024-04-21T08:38:44.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.854+0000 7f98b7e14640 1 Processor -- start 2024-04-21T08:38:44.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.854+0000 7f98b7e14640 1 -- start start 2024-04-21T08:38:44.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.854+0000 7f98b7e14640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98b0102b00 0x7f98b0102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:44.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.854+0000 7f98b7e14640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98b0105430 0x7f98b0100ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:44.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.854+0000 7f98b7e14640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98b0107d60 0x7f98b0100ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.854+0000 7f98b7e14640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f98b010db10 con 0x7f98b0102b00 2024-04-21T08:38:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.854+0000 7f98b7e14640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f98b010d990 con 0x7f98b0107d60 2024-04-21T08:38:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.854+0000 7f98b7e14640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98b010dc90 con 0x7f98b0105430 2024-04-21T08:38:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.854+0000 7f98b5bb0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98b0102b00 0x7f98b0102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.854+0000 7f98b5bb0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98b0102b00 0x7f98b0102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36090/0 (socket says 172.21.15.43:36090) 2024-04-21T08:38:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.854+0000 7f98b5bb0640 1 -- 172.21.15.43:0/1992026708 learned_addr learned my addr 172.21.15.43:0/1992026708 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.854+0000 7f98b63b1640 1 --2- 172.21.15.43:0/1992026708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98b0107d60 0x7f98b0100ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.855+0000 7f98b5bb0640 1 -- 172.21.15.43:0/1992026708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98b0105430 msgr2=0x7f98b0100ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.855+0000 7f98b5bb0640 1 --2- 172.21.15.43:0/1992026708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98b0105430 0x7f98b0100ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.855+0000 7f98b5bb0640 1 -- 172.21.15.43:0/1992026708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98b0107d60 msgr2=0x7f98b0100ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.855+0000 7f98b5bb0640 1 --2- 172.21.15.43:0/1992026708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98b0107d60 0x7f98b0100ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.855+0000 7f98b5bb0640 1 -- 172.21.15.43:0/1992026708 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b0101880 con 0x7f98b0102b00 2024-04-21T08:38:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.855+0000 7f98b5bb0640 1 --2- 172.21.15.43:0/1992026708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98b0102b00 0x7f98b0102470 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00d950 tx=0x7f98ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.855+0000 7f98a6ffd640 1 -- 172.21.15.43:0/1992026708 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac017070 con 0x7f98b0102b00 2024-04-21T08:38:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.855+0000 7f98b7e14640 1 -- 172.21.15.43:0/1992026708 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b01a22d0 con 0x7f98b0102b00 2024-04-21T08:38:44.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.855+0000 7f98a6ffd640 1 -- 172.21.15.43:0/1992026708 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98ac009c30 con 0x7f98b0102b00 2024-04-21T08:38:44.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.855+0000 7f98a6ffd640 1 -- 172.21.15.43:0/1992026708 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac00f620 con 0x7f98b0102b00 2024-04-21T08:38:44.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.855+0000 7f98b7e14640 1 -- 172.21.15.43:0/1992026708 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b01a2870 con 0x7f98b0102b00 2024-04-21T08:38:44.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.856+0000 7f98b7e14640 1 -- 172.21.15.43:0/1992026708 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98b00fdf60 con 0x7f98b0102b00 2024-04-21T08:38:44.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.857+0000 7f98a6ffd640 1 -- 172.21.15.43:0/1992026708 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f98ac010460 con 0x7f98b0102b00 2024-04-21T08:38:44.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.858+0000 7f98a6ffd640 1 --2- 172.21.15.43:0/1992026708 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f989c0a3bb0 0x7f989c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:44.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.858+0000 7f98b53af640 1 --2- 172.21.15.43:0/1992026708 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f989c0a3bb0 0x7f989c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:44.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.858+0000 7f98a6ffd640 1 -- 172.21.15.43:0/1992026708 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f98ac0d0460 con 0x7f98b0102b00 2024-04-21T08:38:44.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.858+0000 7f98b53af640 1 --2- 172.21.15.43:0/1992026708 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f989c0a3bb0 0x7f989c0a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f98a0005fd0 tx=0x7f98a0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:44.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:44.861+0000 7f98a6ffd640 1 -- 172.21.15.43:0/1992026708 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f98ac09c9f0 con 0x7f98b0102b00 2024-04-21T08:38:45.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.089+0000 7f98b7e14640 1 -- 172.21.15.43:0/1992026708 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98b002d820 con 0x7f98b0102b00 2024-04-21T08:38:45.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.090+0000 7f98a6ffd640 1 -- 172.21.15.43:0/1992026708 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11938 (secure 0 0 0) 0x7f98ac009de0 con 0x7f98b0102b00 2024-04-21T08:38:45.089 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:45.090 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:36:51.902455+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24491,24527,24487,24425,24412]},"id":1}]} 2024-04-21T08:38:45.090 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 27 2024-04-21T08:38:45.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.092+0000 7f98b7e14640 1 -- 172.21.15.43:0/1992026708 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f989c0a3bb0 msgr2=0x7f989c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.092+0000 7f98b7e14640 1 --2- 172.21.15.43:0/1992026708 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f989c0a3bb0 0x7f989c0a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f98a0005fd0 tx=0x7f98a0005850 comp rx=0 tx=0).stop 2024-04-21T08:38:45.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.092+0000 7f98b7e14640 1 -- 172.21.15.43:0/1992026708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98b0102b00 msgr2=0x7f98b0102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.092+0000 7f98b7e14640 1 --2- 172.21.15.43:0/1992026708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98b0102b00 0x7f98b0102470 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00d950 tx=0x7f98ac00de10 comp rx=0 tx=0).stop 2024-04-21T08:38:45.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.093+0000 7f98b7e14640 1 -- 172.21.15.43:0/1992026708 shutdown_connections 2024-04-21T08:38:45.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.093+0000 7f98b7e14640 1 --2- 172.21.15.43:0/1992026708 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f989c0a3bb0 0x7f989c0a6070 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.093+0000 7f98b7e14640 1 --2- 172.21.15.43:0/1992026708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98b0107d60 0x7f98b0100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.093+0000 7f98b7e14640 1 --2- 172.21.15.43:0/1992026708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98b0105430 0x7f98b0100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.093+0000 7f98b7e14640 1 --2- 172.21.15.43:0/1992026708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98b0102b00 0x7f98b0102470 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.093+0000 7f98b7e14640 1 -- 172.21.15.43:0/1992026708 >> 172.21.15.43:0/1992026708 conn(0x7f98b00fc470 msgr2=0x7f98b0103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.093+0000 7f98b7e14640 1 -- 172.21.15.43:0/1992026708 shutdown_connections 2024-04-21T08:38:45.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.093+0000 7f98b7e14640 1 -- 172.21.15.43:0/1992026708 wait complete. 2024-04-21T08:38:45.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:38:45.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.206+0000 7fca22477640 1 -- 172.21.15.43:0/2871000637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca1c107ce0 msgr2=0x7fca1c10a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.206+0000 7fca22477640 1 --2- 172.21.15.43:0/2871000637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca1c107ce0 0x7fca1c10a0d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fca14009ca0 tx=0x7fca1402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.206+0000 7fca22477640 1 -- 172.21.15.43:0/2871000637 shutdown_connections 2024-04-21T08:38:45.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.206+0000 7fca22477640 1 --2- 172.21.15.43:0/2871000637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca1c107ce0 0x7fca1c10a0d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.206+0000 7fca22477640 1 --2- 172.21.15.43:0/2871000637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca1c1053b0 0x7fca1c1077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.206+0000 7fca22477640 1 --2- 172.21.15.43:0/2871000637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca1c102a80 0x7fca1c104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.206+0000 7fca22477640 1 -- 172.21.15.43:0/2871000637 >> 172.21.15.43:0/2871000637 conn(0x7fca1c0fc410 msgr2=0x7fca1c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.206+0000 7fca22477640 1 -- 172.21.15.43:0/2871000637 shutdown_connections 2024-04-21T08:38:45.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.206+0000 7fca22477640 1 -- 172.21.15.43:0/2871000637 wait complete. 2024-04-21T08:38:45.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.207+0000 7fca22477640 1 Processor -- start 2024-04-21T08:38:45.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.207+0000 7fca22477640 1 -- start start 2024-04-21T08:38:45.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.207+0000 7fca22477640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca1c102a80 0x7fca1c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.207+0000 7fca22477640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca1c1053b0 0x7fca1c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.207+0000 7fca22477640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca1c107ce0 0x7fca1c19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.207+0000 7fca22477640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c10dae0 con 0x7fca1c102a80 2024-04-21T08:38:45.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.207+0000 7fca22477640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c10d960 con 0x7fca1c1053b0 2024-04-21T08:38:45.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.207+0000 7fca22477640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c10dc60 con 0x7fca1c107ce0 2024-04-21T08:38:45.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.207+0000 7fca1bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca1c102a80 0x7fca1c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.207+0000 7fca1bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca1c102a80 0x7fca1c1981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36098/0 (socket says 172.21.15.43:36098) 2024-04-21T08:38:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.207+0000 7fca1bfff640 1 -- 172.21.15.43:0/2213780956 learned_addr learned my addr 172.21.15.43:0/2213780956 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.208+0000 7fca1b7fe640 1 --2- 172.21.15.43:0/2213780956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca1c1053b0 0x7fca1c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.208+0000 7fca20a14640 1 --2- 172.21.15.43:0/2213780956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca1c107ce0 0x7fca1c19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.208+0000 7fca1bfff640 1 -- 172.21.15.43:0/2213780956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca1c107ce0 msgr2=0x7fca1c19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.208+0000 7fca1bfff640 1 --2- 172.21.15.43:0/2213780956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca1c107ce0 0x7fca1c19f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.208+0000 7fca1bfff640 1 -- 172.21.15.43:0/2213780956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca1c1053b0 msgr2=0x7fca1c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.208+0000 7fca1bfff640 1 --2- 172.21.15.43:0/2213780956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca1c1053b0 0x7fca1c1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.208+0000 7fca1bfff640 1 -- 172.21.15.43:0/2213780956 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1c19fdd0 con 0x7fca1c102a80 2024-04-21T08:38:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.208+0000 7fca1bfff640 1 --2- 172.21.15.43:0/2213780956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca1c102a80 0x7fca1c1981b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00d770 tx=0x7fca0c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.208+0000 7fca197fa640 1 -- 172.21.15.43:0/2213780956 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c017070 con 0x7fca1c102a80 2024-04-21T08:38:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.208+0000 7fca197fa640 1 -- 172.21.15.43:0/2213780956 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca0c0054d0 con 0x7fca1c102a80 2024-04-21T08:38:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.208+0000 7fca22477640 1 -- 172.21.15.43:0/2213780956 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca1c1a00c0 con 0x7fca1c102a80 2024-04-21T08:38:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.208+0000 7fca197fa640 1 -- 172.21.15.43:0/2213780956 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c004c30 con 0x7fca1c102a80 2024-04-21T08:38:45.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.208+0000 7fca22477640 1 -- 172.21.15.43:0/2213780956 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca1c1a05c0 con 0x7fca1c102a80 2024-04-21T08:38:45.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.209+0000 7fca22477640 1 -- 172.21.15.43:0/2213780956 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca1c0fdf00 con 0x7fca1c102a80 2024-04-21T08:38:45.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.210+0000 7fca197fa640 1 -- 172.21.15.43:0/2213780956 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fca0c004e10 con 0x7fca1c102a80 2024-04-21T08:38:45.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.211+0000 7fca197fa640 1 --2- 172.21.15.43:0/2213780956 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fca080a3bb0 0x7fca080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.211+0000 7fca1b7fe640 1 --2- 172.21.15.43:0/2213780956 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fca080a3bb0 0x7fca080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.211+0000 7fca197fa640 1 -- 172.21.15.43:0/2213780956 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fca0c0d03c0 con 0x7fca1c102a80 2024-04-21T08:38:45.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.211+0000 7fca1b7fe640 1 --2- 172.21.15.43:0/2213780956 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fca080a3bb0 0x7fca080a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fca1c1996c0 tx=0x7fca0400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.216+0000 7fca197fa640 1 -- 172.21.15.43:0/2213780956 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fca0c09c950 con 0x7fca1c102a80 2024-04-21T08:38:45.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.403+0000 7fca22477640 1 -- 172.21.15.43:0/2213780956 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca1c062b50 con 0x7fca1c102a80 2024-04-21T08:38:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.405+0000 7fca197fa640 1 -- 172.21.15.43:0/2213780956 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24856 (secure 0 0 0) 0x7fca0c09c250 con 0x7fca1c102a80 2024-04-21T08:38:45.404 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:45.404 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":94,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:38:15.326063+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:38:45.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.407+0000 7fca22477640 1 -- 172.21.15.43:0/2213780956 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fca080a3bb0 msgr2=0x7fca080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.407+0000 7fca22477640 1 --2- 172.21.15.43:0/2213780956 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fca080a3bb0 0x7fca080a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fca1c1996c0 tx=0x7fca0400a4c0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.407+0000 7fca22477640 1 -- 172.21.15.43:0/2213780956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca1c102a80 msgr2=0x7fca1c1981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.407+0000 7fca22477640 1 --2- 172.21.15.43:0/2213780956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca1c102a80 0x7fca1c1981b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00d770 tx=0x7fca0c00dc30 comp rx=0 tx=0).stop 2024-04-21T08:38:45.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.407+0000 7fca22477640 1 -- 172.21.15.43:0/2213780956 shutdown_connections 2024-04-21T08:38:45.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.407+0000 7fca22477640 1 --2- 172.21.15.43:0/2213780956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca1c107ce0 0x7fca1c19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.407+0000 7fca22477640 1 --2- 172.21.15.43:0/2213780956 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fca080a3bb0 0x7fca080a6070 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.407+0000 7fca22477640 1 --2- 172.21.15.43:0/2213780956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca1c1053b0 0x7fca1c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.407+0000 7fca22477640 1 --2- 172.21.15.43:0/2213780956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca1c102a80 0x7fca1c1981b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.407+0000 7fca22477640 1 -- 172.21.15.43:0/2213780956 >> 172.21.15.43:0/2213780956 conn(0x7fca1c0fc410 msgr2=0x7fca1c1033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.407+0000 7fca22477640 1 -- 172.21.15.43:0/2213780956 shutdown_connections 2024-04-21T08:38:45.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.407+0000 7fca22477640 1 -- 172.21.15.43:0/2213780956 wait complete. 2024-04-21T08:38:45.418 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-21T08:36:23.821253+0000', 'modified': '2024-04-21T08:36:51.902455+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24527, 'mds_1': 24491, 'mds_2': 24487, 'mds_3': 24425, 'mds_4': 24412}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14634': {'gid': 14634, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.43:6839/931821965', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6838', 'nonce': 931821965}, {'type': 'v1', 'addr': '172.21.15.43:6839', 'nonce': 931821965}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14652': {'gid': 14652, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.43:6841/623279035', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6840', 'nonce': 623279035}, {'type': 'v1', 'addr': '172.21.15.43:6841', 'nonce': 623279035}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24412': {'gid': 24412, 'name': 'l', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.175:6833/187945366', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6832', 'nonce': 187945366}, {'type': 'v1', 'addr': '172.21.15.175:6833', 'nonce': 187945366}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.112:6833/1516781885', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6832', 'nonce': 1516781885}, {'type': 'v1', 'addr': '172.21.15.112:6833', 'nonce': 1516781885}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6835/466532477', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6834', 'nonce': 466532477}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 466532477}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.112:6835/3159749962', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6834', 'nonce': 3159749962}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 3159749962}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6837/3198693848', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6836', 'nonce': 3198693848}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 3198693848}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.175:6839/3664054463', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 3664054463}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 3664054463}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.112:6837/3054508488', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6836', 'nonce': 3054508488}, {'type': 'v1', 'addr': '172.21.15.112:6837', 'nonce': 3054508488}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24527': {'gid': 24527, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6839/2143575300', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6838', 'nonce': 2143575300}, {'type': 'v1', 'addr': '172.21.15.112:6839', 'nonce': 2143575300}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24527, 'qdb_cluster': [24491, 24527, 24487, 24425, 24412]} 2024-04-21T08:38:45.418 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-21T08:38:45.418 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-04-21T08:38:45.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.522+0000 7f7289c6d640 1 -- 172.21.15.43:0/4135649693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7284102a10 msgr2=0x7f7284104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.522+0000 7f7289c6d640 1 --2- 172.21.15.43:0/4135649693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7284102a10 0x7f7284104e00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7274009900 tx=0x7f727402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:45.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.523+0000 7f7289c6d640 1 -- 172.21.15.43:0/4135649693 shutdown_connections 2024-04-21T08:38:45.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.523+0000 7f7289c6d640 1 --2- 172.21.15.43:0/4135649693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7284107c70 0x7f728410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.523+0000 7f7289c6d640 1 --2- 172.21.15.43:0/4135649693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7284105340 0x7f7284107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.523+0000 7f7289c6d640 1 --2- 172.21.15.43:0/4135649693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7284102a10 0x7f7284104e00 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.523+0000 7f7289c6d640 1 -- 172.21.15.43:0/4135649693 >> 172.21.15.43:0/4135649693 conn(0x7f72840fc400 msgr2=0x7f72840fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.523+0000 7f7289c6d640 1 -- 172.21.15.43:0/4135649693 shutdown_connections 2024-04-21T08:38:45.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.523+0000 7f7289c6d640 1 -- 172.21.15.43:0/4135649693 wait complete. 2024-04-21T08:38:45.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.523+0000 7f7289c6d640 1 Processor -- start 2024-04-21T08:38:45.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.523+0000 7f7289c6d640 1 -- start start 2024-04-21T08:38:45.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.524+0000 7f7289c6d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7284102a10 0x7f7284102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.524+0000 7f7289c6d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7284105340 0x7f72841009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.524+0000 7f7289c6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7284107c70 0x7f7284100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.524+0000 7f7289c6d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f728410dac0 con 0x7f7284107c70 2024-04-21T08:38:45.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.524+0000 7f7289c6d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f728410d940 con 0x7f7284105340 2024-04-21T08:38:45.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.524+0000 7f7289c6d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f728410dc40 con 0x7f7284102a10 2024-04-21T08:38:45.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.524+0000 7f7283fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7284107c70 0x7f7284100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.524+0000 7f7283fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7284107c70 0x7f7284100f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36122/0 (socket says 172.21.15.43:36122) 2024-04-21T08:38:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.524+0000 7f7283fff640 1 -- 172.21.15.43:0/973662019 learned_addr learned my addr 172.21.15.43:0/973662019 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.524+0000 7f72837fe640 1 --2- 172.21.15.43:0/973662019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7284102a10 0x7f7284102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.524+0000 7f7282ffd640 1 --2- 172.21.15.43:0/973662019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7284105340 0x7f72841009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.524+0000 7f7283fff640 1 -- 172.21.15.43:0/973662019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7284102a10 msgr2=0x7f7284102380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.524+0000 7f7283fff640 1 --2- 172.21.15.43:0/973662019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7284102a10 0x7f7284102380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.524+0000 7f7283fff640 1 -- 172.21.15.43:0/973662019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7284105340 msgr2=0x7f72841009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.524+0000 7f7283fff640 1 --2- 172.21.15.43:0/973662019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7284105340 0x7f72841009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.524+0000 7f7283fff640 1 -- 172.21.15.43:0/973662019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7284101790 con 0x7f7284107c70 2024-04-21T08:38:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.524+0000 7f72837fe640 1 --2- 172.21.15.43:0/973662019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7284102a10 0x7f7284102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.524+0000 7f7282ffd640 1 --2- 172.21.15.43:0/973662019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7284105340 0x7f72841009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.524+0000 7f7283fff640 1 --2- 172.21.15.43:0/973662019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7284107c70 0x7f7284100f00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f727c00cad0 tx=0x7f727c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.524+0000 7f7280ff9640 1 -- 172.21.15.43:0/973662019 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c00ae20 con 0x7f7284107c70 2024-04-21T08:38:45.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.524+0000 7f7289c6d640 1 -- 172.21.15.43:0/973662019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72841a2380 con 0x7f7284107c70 2024-04-21T08:38:45.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.524+0000 7f7280ff9640 1 -- 172.21.15.43:0/973662019 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f727c005020 con 0x7f7284107c70 2024-04-21T08:38:45.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.524+0000 7f7280ff9640 1 -- 172.21.15.43:0/973662019 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c00f6b0 con 0x7f7284107c70 2024-04-21T08:38:45.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.524+0000 7f7289c6d640 1 -- 172.21.15.43:0/973662019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72841a28d0 con 0x7f7284107c70 2024-04-21T08:38:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.527+0000 7f7280ff9640 1 -- 172.21.15.43:0/973662019 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f727c0051d0 con 0x7f7284107c70 2024-04-21T08:38:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.527+0000 7f728c868740 1 Processor -- start 2024-04-21T08:38:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.527+0000 7f728c868740 1 -- start start 2024-04-21T08:38:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.527+0000 7f728c868740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562568bc0e40 0x562568bc1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.527+0000 7f728c868740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc1830 0x562568bc1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.527+0000 7f728c868740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562568bc22d0 0x562568bc8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.527+0000 7f728c868740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562568b25b10 con 0x562568bc1830 2024-04-21T08:38:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.527+0000 7f728c868740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562568ac3240 con 0x562568bc0e40 2024-04-21T08:38:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.527+0000 7f728c868740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562568b21ad0 con 0x562568bc22d0 2024-04-21T08:38:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.527+0000 7f72837fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc1830 0x562568bc1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.527+0000 7f72837fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc1830 0x562568bc1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36136/0 (socket says 172.21.15.43:36136) 2024-04-21T08:38:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.527+0000 7f72837fe640 1 -- 172.21.15.43:0/3530125502 learned_addr learned my addr 172.21.15.43:0/3530125502 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.527+0000 7f7282ffd640 1 --2- 172.21.15.43:0/3530125502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562568bc0e40 0x562568bc1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.527+0000 7f7280ff9640 1 --2- 172.21.15.43:0/973662019 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f726c0a3bb0 0x7f726c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.527+0000 7f7282ffd640 1 --2- 172.21.15.43:0/3530125502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562568bc22d0 0x562568bc8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.527+0000 7f7283fff640 1 --2- 172.21.15.43:0/973662019 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f726c0a3bb0 0x7f726c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.527+0000 7f72837fe640 1 -- 172.21.15.43:0/3530125502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562568bc22d0 msgr2=0x562568bc8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.527+0000 7f72837fe640 1 --2- 172.21.15.43:0/3530125502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562568bc22d0 0x562568bc8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.527+0000 7f72837fe640 1 -- 172.21.15.43:0/3530125502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562568bc0e40 msgr2=0x562568bc1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.527+0000 7f72837fe640 1 --2- 172.21.15.43:0/3530125502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562568bc0e40 0x562568bc1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.527+0000 7f72837fe640 1 -- 172.21.15.43:0/3530125502 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625688382d0 con 0x562568bc1830 2024-04-21T08:38:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.527+0000 7f7282ffd640 1 --2- 172.21.15.43:0/3530125502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562568bc22d0 0x562568bc8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.528+0000 7f7280ff9640 1 -- 172.21.15.43:0/973662019 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f727c0d0960 con 0x7f7284107c70 2024-04-21T08:38:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.528+0000 7f72837fe640 1 --2- 172.21.15.43:0/3530125502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc1830 0x562568bc1cb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7274009900 tx=0x7f727402c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c25720acc2ef6b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.528+0000 7f7283fff640 1 --2- 172.21.15.43:0/973662019 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f726c0a3bb0 0x7f726c0a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f727c0ce3e0 tx=0x7f727c087040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.528+0000 7f7269ffb640 1 -- 172.21.15.43:0/3530125502 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7274044020 con 0x562568bc1830 2024-04-21T08:38:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.528+0000 7f7269ffb640 1 -- 172.21.15.43:0/3530125502 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7274005040 con 0x562568bc1830 2024-04-21T08:38:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.528+0000 7f7269ffb640 1 -- 172.21.15.43:0/3530125502 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7274004a40 con 0x562568bc1830 2024-04-21T08:38:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.528+0000 7f728c868740 1 -- 172.21.15.43:0/3530125502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc1830 msgr2=0x562568bc1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.528+0000 7f728c868740 1 --2- 172.21.15.43:0/3530125502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc1830 0x562568bc1cb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7274009900 tx=0x7f727402c960 comp rx=0 tx=0).stop 2024-04-21T08:38:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.529+0000 7f728c868740 1 -- 172.21.15.43:0/3530125502 shutdown_connections 2024-04-21T08:38:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.529+0000 7f728c868740 1 --2- 172.21.15.43:0/3530125502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562568bc22d0 0x562568bc8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.529+0000 7f728c868740 1 --2- 172.21.15.43:0/3530125502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc1830 0x562568bc1cb0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.529+0000 7f728c868740 1 --2- 172.21.15.43:0/3530125502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562568bc0e40 0x562568bc1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.529+0000 7f728c868740 1 -- 172.21.15.43:0/3530125502 >> 172.21.15.43:0/3530125502 conn(0x562568bbcf40 msgr2=0x562568bbb490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.529+0000 7f728c868740 1 -- 172.21.15.43:0/3530125502 shutdown_connections 2024-04-21T08:38:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.529+0000 7f728c868740 1 -- 172.21.15.43:0/3530125502 wait complete. 2024-04-21T08:38:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.529+0000 7f728c868740 1 -- start start 2024-04-21T08:38:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.529+0000 7f728c868740 1 Processor -- start 2024-04-21T08:38:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.529+0000 7f728c868740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc0e40 0x562568bcae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.529+0000 7f728c868740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562568bc1830 0x562568bcb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.529+0000 7f728c868740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562568bc22d0 0x562568bd23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.529+0000 7f728c868740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562568bc9450 con 0x562568bc0e40 2024-04-21T08:38:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.529+0000 7f728c868740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562568aae190 con 0x562568bc1830 2024-04-21T08:38:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.529+0000 7f728c868740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562568bc95d0 con 0x562568bc22d0 2024-04-21T08:38:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.529+0000 7f728c868740 10 client.0 _ll_register_callbacks cb 0x562568aa98b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.529+0000 7f7282ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc0e40 0x562568bcae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.530+0000 7f7282ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc0e40 0x562568bcae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36138/0 (socket says 172.21.15.43:36138) 2024-04-21T08:38:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.530+0000 7f7282ffd640 1 -- 172.21.15.43:0/2133597122 learned_addr learned my addr 172.21.15.43:0/2133597122 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.530+0000 7f72837fe640 1 --2- 172.21.15.43:0/2133597122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562568bc1830 0x562568bcb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.530+0000 7f7282ffd640 1 -- 172.21.15.43:0/2133597122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562568bc22d0 msgr2=0x562568bd23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.530+0000 7f7282ffd640 1 --2- 172.21.15.43:0/2133597122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562568bc22d0 0x562568bd23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.530+0000 7f7282ffd640 1 -- 172.21.15.43:0/2133597122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562568bc1830 msgr2=0x562568bcb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.530+0000 7f7282ffd640 1 --2- 172.21.15.43:0/2133597122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562568bc1830 0x562568bcb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.530+0000 7f7282ffd640 1 -- 172.21.15.43:0/2133597122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625689026b0 con 0x562568bc0e40 2024-04-21T08:38:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.530+0000 7f72837fe640 1 --2- 172.21.15.43:0/2133597122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562568bc1830 0x562568bcb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.530+0000 7f7282ffd640 1 --2- 172.21.15.43:0/2133597122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc0e40 0x562568bcae20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f72780058c0 tx=0x7f7278011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8a1a46a1615f35 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.530+0000 7f7262ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:38:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.530+0000 7f7262ffd640 1 -- 172.21.15.43:0/2133597122 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727800ed90 con 0x562568bc0e40 2024-04-21T08:38:45.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.530+0000 7f7262ffd640 1 -- 172.21.15.43:0/2133597122 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f7278005020 con 0x562568bc0e40 2024-04-21T08:38:45.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.530+0000 7f728c868740 1 -- 172.21.15.43:0/2133597122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7274005040 con 0x562568bc0e40 2024-04-21T08:38:45.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.530+0000 7f7262ffd640 1 -- 172.21.15.43:0/2133597122 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7278005220 con 0x562568bc0e40 2024-04-21T08:38:45.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.530+0000 7f728c868740 10 client.15444 fetch_fsmap learned FSMap version 27 2024-04-21T08:38:45.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.530+0000 7f728c868740 1 -- 172.21.15.43:0/2133597122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562568bd5750 con 0x562568bc0e40 2024-04-21T08:38:45.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.530+0000 7f7262ffd640 1 -- 172.21.15.43:0/2133597122 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7274005040 con 0x562568bc0e40 2024-04-21T08:38:45.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.531+0000 7f7262ffd640 1 -- 172.21.15.43:0/2133597122 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f727801b5c0 con 0x562568bc0e40 2024-04-21T08:38:45.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.531+0000 7f728c868740 10 client.15444 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:38:45.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.531+0000 7f728c868740 10 client.15444 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-21T08:38:45.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.531+0000 7f728c868740 20 client.15444 populate_metadata read hostname 'smithi043' 2024-04-21T08:38:45.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.531+0000 7f728c868740 1 --2- 172.21.15.43:0/2133597122 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x562568bd7db0 0x562568bd9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.531+0000 7f728c868740 4 client.15444 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:38:45.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.531+0000 7f728c868740 1 -- 172.21.15.43:0/2133597122 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562568b33530 con 0x562568bd7db0 2024-04-21T08:38:45.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.531+0000 7f72837fe640 1 --2- 172.21.15.43:0/2133597122 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x562568bd7db0 0x562568bd9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.531+0000 7f7262ffd640 10 client.15444 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:38:45.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.531+0000 7f72837fe640 1 --2- 172.21.15.43:0/2133597122 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x562568bd7db0 0x562568bd9220 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2fda5ed213a59316 server_cookie=bbedd64607689031 in_seq=0 out_seq=0 2024-04-21T08:38:45.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.532+0000 7f7262ffd640 1 -- 172.21.15.43:0/2133597122 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562568b33530 con 0x562568bd7db0 2024-04-21T08:38:45.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.532+0000 7f7262ffd640 10 client.15444 handle_command_reply: tid=0 2024-04-21T08:38:45.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.532+0000 7f728c868740 1 client.15444 shutdown 2024-04-21T08:38:45.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.532+0000 7f728c868740 1 -- 172.21.15.43:0/2133597122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc0e40 msgr2=0x562568bcae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.532+0000 7f728c868740 1 --2- 172.21.15.43:0/2133597122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc0e40 0x562568bcae20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f72780058c0 tx=0x7f7278011f10 comp rx=0 tx=0).stop 2024-04-21T08:38:45.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.532+0000 7f728c868740 1 -- 172.21.15.43:0/2133597122 shutdown_connections 2024-04-21T08:38:45.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.532+0000 7f728c868740 1 --2- 172.21.15.43:0/2133597122 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x562568bd7db0 0x562568bd9220 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.532+0000 7f728c868740 1 --2- 172.21.15.43:0/2133597122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562568bc22d0 0x562568bd23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.532+0000 7f728c868740 1 --2- 172.21.15.43:0/2133597122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562568bc1830 0x562568bcb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.532+0000 7f7262ffd640 0 client.15444 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:38:45.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.532+0000 7f728c868740 1 --2- 172.21.15.43:0/2133597122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc0e40 0x562568bcae20 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.532+0000 7f728c868740 1 -- 172.21.15.43:0/2133597122 >> 172.21.15.43:0/2133597122 conn(0x562568bbcf40 msgr2=0x562568bbaa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.532+0000 7f728c868740 1 -- 172.21.15.43:0/2133597122 shutdown_connections 2024-04-21T08:38:45.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.533+0000 7f728c868740 1 -- 172.21.15.43:0/2133597122 wait complete. 2024-04-21T08:38:45.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.533+0000 7f728c868740 20 client.15444 trim_cache size 0 max 16384 2024-04-21T08:38:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.550+0000 7f728c868740 1 Processor -- start 2024-04-21T08:38:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.551+0000 7f728c868740 1 -- start start 2024-04-21T08:38:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.551+0000 7f728c868740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562568bc0e40 0x562568bd6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.551+0000 7f728c868740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562568bc1830 0x562568bd2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.551+0000 7f728c868740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc22d0 0x562568bc5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.551+0000 7f728c868740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562568bd7560 con 0x562568bc22d0 2024-04-21T08:38:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.551+0000 7f728c868740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562568bd7c30 con 0x562568bc1830 2024-04-21T08:38:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.551+0000 7f728c868740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562568bd8300 con 0x562568bc0e40 2024-04-21T08:38:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.551+0000 7f7282ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc22d0 0x562568bc5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.551+0000 7f7282ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc22d0 0x562568bc5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36140/0 (socket says 172.21.15.43:36140) 2024-04-21T08:38:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.551+0000 7f7282ffd640 1 -- 172.21.15.43:0/2726388733 learned_addr learned my addr 172.21.15.43:0/2726388733 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.551+0000 7f72837fe640 1 --2- 172.21.15.43:0/2726388733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562568bc1830 0x562568bd2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.551+0000 7f7282ffd640 1 -- 172.21.15.43:0/2726388733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562568bc0e40 msgr2=0x562568bd6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.551+0000 7f7282ffd640 1 --2- 172.21.15.43:0/2726388733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562568bc0e40 0x562568bd6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.551+0000 7f7282ffd640 1 -- 172.21.15.43:0/2726388733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562568bc1830 msgr2=0x562568bd2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.551+0000 7f7282ffd640 1 --2- 172.21.15.43:0/2726388733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562568bc1830 0x562568bd2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.551+0000 7f7282ffd640 1 -- 172.21.15.43:0/2726388733 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562568bcbe10 con 0x562568bc22d0 2024-04-21T08:38:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.551+0000 7f72837fe640 1 --2- 172.21.15.43:0/2726388733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562568bc1830 0x562568bd2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.551+0000 7f7282ffd640 1 --2- 172.21.15.43:0/2726388733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc22d0 0x562568bc5840 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7278004700 tx=0x7f727800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc233ad2e94bc82a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.551+0000 7f72697fa640 1 -- 172.21.15.43:0/2726388733 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727802f030 con 0x562568bc22d0 2024-04-21T08:38:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.552+0000 7f72697fa640 1 -- 172.21.15.43:0/2726388733 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7278005020 con 0x562568bc22d0 2024-04-21T08:38:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.552+0000 7f72697fa640 1 -- 172.21.15.43:0/2726388733 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7278004280 con 0x562568bc22d0 2024-04-21T08:38:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.552+0000 7f728c868740 1 -- 172.21.15.43:0/2726388733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc22d0 msgr2=0x562568bc5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.552+0000 7f728c868740 1 --2- 172.21.15.43:0/2726388733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc22d0 0x562568bc5840 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7278004700 tx=0x7f727800b910 comp rx=0 tx=0).stop 2024-04-21T08:38:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.552+0000 7f728c868740 1 -- 172.21.15.43:0/2726388733 shutdown_connections 2024-04-21T08:38:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.552+0000 7f728c868740 1 --2- 172.21.15.43:0/2726388733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc22d0 0x562568bc5840 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.552+0000 7f728c868740 1 --2- 172.21.15.43:0/2726388733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562568bc1830 0x562568bd2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.552+0000 7f728c868740 1 --2- 172.21.15.43:0/2726388733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562568bc0e40 0x562568bd6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.552+0000 7f728c868740 1 -- 172.21.15.43:0/2726388733 >> 172.21.15.43:0/2726388733 conn(0x562568bbcf40 msgr2=0x562568bca0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.552+0000 7f728c868740 1 -- 172.21.15.43:0/2726388733 shutdown_connections 2024-04-21T08:38:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.552+0000 7f728c868740 1 -- 172.21.15.43:0/2726388733 wait complete. 2024-04-21T08:38:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.552+0000 7f728c868740 1 -- start start 2024-04-21T08:38:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.553+0000 7f728c868740 1 Processor -- start 2024-04-21T08:38:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.553+0000 7f728c868740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562568bc0e40 0x562568bce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.553+0000 7f728c868740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562568bc1830 0x562568bd0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.553+0000 7f728c868740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc22d0 0x562568bd10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.553+0000 7f728c868740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562568bd90a0 con 0x562568bc22d0 2024-04-21T08:38:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.553+0000 7f728c868740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562568bd89d0 con 0x562568bc1830 2024-04-21T08:38:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.553+0000 7f728c868740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562568bd9770 con 0x562568bc0e40 2024-04-21T08:38:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.553+0000 7f728c868740 10 client.0 _ll_register_callbacks cb 0x7f72780116c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.553+0000 7f7282ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc22d0 0x562568bd10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.553+0000 7f7282ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc22d0 0x562568bd10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36146/0 (socket says 172.21.15.43:36146) 2024-04-21T08:38:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.553+0000 7f7282ffd640 1 -- 172.21.15.43:0/1012152430 learned_addr learned my addr 172.21.15.43:0/1012152430 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.553+0000 7f7282ffd640 1 --2- 172.21.15.43:0/1012152430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562568bc0e40 0x562568bce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.553+0000 7f72837fe640 1 --2- 172.21.15.43:0/1012152430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562568bc1830 0x562568bd0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.553+0000 7f7282ffd640 1 -- 172.21.15.43:0/1012152430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562568bc0e40 msgr2=0x562568bce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.553+0000 7f7282ffd640 1 --2- 172.21.15.43:0/1012152430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562568bc0e40 0x562568bce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.553+0000 7f7282ffd640 1 -- 172.21.15.43:0/1012152430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562568bc1830 msgr2=0x562568bd0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.553+0000 7f7282ffd640 1 --2- 172.21.15.43:0/1012152430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562568bc1830 0x562568bd0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.553+0000 7f7282ffd640 1 -- 172.21.15.43:0/1012152430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562568bd18f0 con 0x562568bc22d0 2024-04-21T08:38:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.553+0000 7f72837fe640 1 --2- 172.21.15.43:0/1012152430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562568bc1830 0x562568bd0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.553+0000 7f7282ffd640 1 --2- 172.21.15.43:0/1012152430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc22d0 0x562568bd10c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f72780051f0 tx=0x7f727801ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4114192018af007 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.553+0000 7f7262ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:38:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.554+0000 7f7262ffd640 1 -- 172.21.15.43:0/1012152430 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727802f030 con 0x562568bc22d0 2024-04-21T08:38:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.554+0000 7f7262ffd640 1 -- 172.21.15.43:0/1012152430 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f727800ed90 con 0x562568bc22d0 2024-04-21T08:38:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.554+0000 7f728c868740 1 -- 172.21.15.43:0/1012152430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7278005020 con 0x562568bc22d0 2024-04-21T08:38:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.554+0000 7f7262ffd640 1 -- 172.21.15.43:0/1012152430 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7278004730 con 0x562568bc22d0 2024-04-21T08:38:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.554+0000 7f728c868740 10 client.15450 fetch_fsmap learned FSMap version 27 2024-04-21T08:38:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.554+0000 7f728c868740 1 -- 172.21.15.43:0/1012152430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562568c3e050 con 0x562568bc22d0 2024-04-21T08:38:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.554+0000 7f7262ffd640 1 -- 172.21.15.43:0/1012152430 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7278005020 con 0x562568bc22d0 2024-04-21T08:38:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.554+0000 7f7262ffd640 1 -- 172.21.15.43:0/1012152430 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7278003b50 con 0x562568bc22d0 2024-04-21T08:38:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.554+0000 7f728c868740 10 client.15450 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:38:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.554+0000 7f728c868740 10 client.15450 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-21T08:38:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.554+0000 7f728c868740 20 client.15450 populate_metadata read hostname 'smithi043' 2024-04-21T08:38:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.554+0000 7f728c868740 1 --2- 172.21.15.43:0/1012152430 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x562568c42800 0x562568c41510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.554+0000 7f728c868740 4 client.15450 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:38:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.554+0000 7f728c868740 1 -- 172.21.15.43:0/1012152430 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "status"}) -- 0x562568b2d3a0 con 0x562568c42800 2024-04-21T08:38:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.554+0000 7f72837fe640 1 --2- 172.21.15.43:0/1012152430 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x562568c42800 0x562568c41510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.555+0000 7f7262ffd640 10 client.15450 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:38:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.555+0000 7f72837fe640 1 --2- 172.21.15.43:0/1012152430 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x562568c42800 0x562568c41510 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=14cd96cab797ceac server_cookie=d57a693a593ec5e0 in_seq=0 out_seq=0 2024-04-21T08:38:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.555+0000 7f7262ffd640 1 -- 172.21.15.43:0/1012152430 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x562568b2d3a0 con 0x562568c42800 2024-04-21T08:38:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.555+0000 7f7262ffd640 10 client.15450 handle_command_reply: tid=0 2024-04-21T08:38:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.555+0000 7f728c868740 1 client.15450 shutdown 2024-04-21T08:38:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.555+0000 7f728c868740 1 -- 172.21.15.43:0/1012152430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc22d0 msgr2=0x562568bd10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.555+0000 7f728c868740 1 --2- 172.21.15.43:0/1012152430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc22d0 0x562568bd10c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f72780051f0 tx=0x7f727801ba60 comp rx=0 tx=0).stop 2024-04-21T08:38:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.555+0000 7f728c868740 1 -- 172.21.15.43:0/1012152430 shutdown_connections 2024-04-21T08:38:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.555+0000 7f728c868740 1 --2- 172.21.15.43:0/1012152430 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x562568c42800 0x562568c41510 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.555+0000 7f728c868740 1 --2- 172.21.15.43:0/1012152430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562568bc22d0 0x562568bd10c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.555+0000 7f728c868740 1 --2- 172.21.15.43:0/1012152430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562568bc1830 0x562568bd0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.555+0000 7f728c868740 1 --2- 172.21.15.43:0/1012152430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562568bc0e40 0x562568bce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.555+0000 7f7262ffd640 0 client.15450 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:38:45.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.555+0000 7f728c868740 1 -- 172.21.15.43:0/1012152430 >> 172.21.15.43:0/1012152430 conn(0x562568bbcf40 msgr2=0x562568bc5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.555+0000 7f728c868740 1 -- 172.21.15.43:0/1012152430 shutdown_connections 2024-04-21T08:38:45.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.555+0000 7f728c868740 1 -- 172.21.15.43:0/1012152430 wait complete. 2024-04-21T08:38:45.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.555+0000 7f728c868740 20 client.15450 trim_cache size 0 max 16384 2024-04-21T08:38:45.561 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:38:45.561 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "d57a7d98-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:38:45.561 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 4, 2024-04-21T08:38:45.561 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24412, 2024-04-21T08:38:45.561 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-21T08:38:45.562 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-21T08:38:45.562 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-21T08:38:45.562 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 114.661665868, 2024-04-21T08:38:45.562 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 27, 2024-04-21T08:38:45.562 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 94, 2024-04-21T08:38:45.562 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 90, 2024-04-21T08:38:45.562 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 179.051446201 2024-04-21T08:38:45.562 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:38:45.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.556+0000 7f7289c6d640 1 -- 172.21.15.43:0/973662019 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f726c0a3bb0 msgr2=0x7f726c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.556+0000 7f7289c6d640 1 --2- 172.21.15.43:0/973662019 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f726c0a3bb0 0x7f726c0a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f727c0ce3e0 tx=0x7f727c087040 comp rx=0 tx=0).stop 2024-04-21T08:38:45.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.556+0000 7f7289c6d640 1 -- 172.21.15.43:0/973662019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7284107c70 msgr2=0x7f7284100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.556+0000 7f7289c6d640 1 --2- 172.21.15.43:0/973662019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7284107c70 0x7f7284100f00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f727c00cad0 tx=0x7f727c00cf90 comp rx=0 tx=0).stop 2024-04-21T08:38:45.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.556+0000 7f7289c6d640 1 -- 172.21.15.43:0/973662019 shutdown_connections 2024-04-21T08:38:45.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.556+0000 7f7289c6d640 1 --2- 172.21.15.43:0/973662019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7284107c70 0x7f7284100f00 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.556+0000 7f7289c6d640 1 --2- 172.21.15.43:0/973662019 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f726c0a3bb0 0x7f726c0a6070 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.556+0000 7f7289c6d640 1 --2- 172.21.15.43:0/973662019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7284105340 0x7f72841009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.556+0000 7f7289c6d640 1 --2- 172.21.15.43:0/973662019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7284102a10 0x7f7284102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.556+0000 7f7289c6d640 1 -- 172.21.15.43:0/973662019 >> 172.21.15.43:0/973662019 conn(0x7f72840fc400 msgr2=0x7f7284103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.556+0000 7f7289c6d640 1 -- 172.21.15.43:0/973662019 shutdown_connections 2024-04-21T08:38:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.556+0000 7f7289c6d640 1 -- 172.21.15.43:0/973662019 wait complete. 2024-04-21T08:38:45.566 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-21T08:38:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.670+0000 7ff161e77640 1 -- 172.21.15.43:0/3507898449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff15c101fe0 msgr2=0x7ff15c102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.670+0000 7ff161e77640 1 --2- 172.21.15.43:0/3507898449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff15c101fe0 0x7ff15c102460 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff150009900 tx=0x7ff15002bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.671+0000 7ff161e77640 1 -- 172.21.15.43:0/3507898449 shutdown_connections 2024-04-21T08:38:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.671+0000 7ff161e77640 1 --2- 172.21.15.43:0/3507898449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff15c1029a0 0x7ff15c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.671+0000 7ff161e77640 1 --2- 172.21.15.43:0/3507898449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff15c101fe0 0x7ff15c102460 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.671+0000 7ff161e77640 1 --2- 172.21.15.43:0/3507898449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff15c100e90 0x7ff15c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.671+0000 7ff161e77640 1 -- 172.21.15.43:0/3507898449 >> 172.21.15.43:0/3507898449 conn(0x7ff15c0fc400 msgr2=0x7ff15c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.671+0000 7ff161e77640 1 -- 172.21.15.43:0/3507898449 shutdown_connections 2024-04-21T08:38:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.671+0000 7ff161e77640 1 -- 172.21.15.43:0/3507898449 wait complete. 2024-04-21T08:38:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.671+0000 7ff161e77640 1 Processor -- start 2024-04-21T08:38:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.671+0000 7ff161e77640 1 -- start start 2024-04-21T08:38:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.671+0000 7ff161e77640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff15c100e90 0x7ff15c1971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.671+0000 7ff161e77640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff15c101fe0 0x7ff15c197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.671+0000 7ff161e77640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff15c1029a0 0x7ff15c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.671+0000 7ff161e77640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff15c10ca40 con 0x7ff15c100e90 2024-04-21T08:38:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.671+0000 7ff161e77640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff15c10c8c0 con 0x7ff15c1029a0 2024-04-21T08:38:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.671+0000 7ff161e77640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff15c10cbc0 con 0x7ff15c101fe0 2024-04-21T08:38:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.672+0000 7ff15b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff15c100e90 0x7ff15c1971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.672+0000 7ff15b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff15c100e90 0x7ff15c1971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36150/0 (socket says 172.21.15.43:36150) 2024-04-21T08:38:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.672+0000 7ff15b7fe640 1 -- 172.21.15.43:0/1062714664 learned_addr learned my addr 172.21.15.43:0/1062714664 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.672+0000 7ff15affd640 1 --2- 172.21.15.43:0/1062714664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff15c101fe0 0x7ff15c197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.672+0000 7ff15b7fe640 1 -- 172.21.15.43:0/1062714664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff15c101fe0 msgr2=0x7ff15c197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.672+0000 7ff15b7fe640 1 --2- 172.21.15.43:0/1062714664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff15c101fe0 0x7ff15c197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.672+0000 7ff15bfff640 1 --2- 172.21.15.43:0/1062714664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff15c1029a0 0x7ff15c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.672+0000 7ff15b7fe640 1 -- 172.21.15.43:0/1062714664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff15c1029a0 msgr2=0x7ff15c19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.672+0000 7ff15b7fe640 1 --2- 172.21.15.43:0/1062714664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff15c1029a0 0x7ff15c19e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.672+0000 7ff15b7fe640 1 -- 172.21.15.43:0/1062714664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff15c19edf0 con 0x7ff15c100e90 2024-04-21T08:38:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.672+0000 7ff15bfff640 1 --2- 172.21.15.43:0/1062714664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff15c1029a0 0x7ff15c19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.672+0000 7ff15b7fe640 1 --2- 172.21.15.43:0/1062714664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff15c100e90 0x7ff15c1971d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00d990 tx=0x7ff14c00de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.672+0000 7ff158ff9640 1 -- 172.21.15.43:0/1062714664 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c01b070 con 0x7ff15c100e90 2024-04-21T08:38:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.672+0000 7ff158ff9640 1 -- 172.21.15.43:0/1062714664 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff14c00edf0 con 0x7ff15c100e90 2024-04-21T08:38:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.672+0000 7ff161e77640 1 -- 172.21.15.43:0/1062714664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff15c069140 con 0x7ff15c100e90 2024-04-21T08:38:45.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.672+0000 7ff158ff9640 1 -- 172.21.15.43:0/1062714664 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c014800 con 0x7ff15c100e90 2024-04-21T08:38:45.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.672+0000 7ff161e77640 1 -- 172.21.15.43:0/1062714664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff15c069730 con 0x7ff15c100e90 2024-04-21T08:38:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.675+0000 7ff158ff9640 1 -- 172.21.15.43:0/1062714664 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff14c01e070 con 0x7ff15c100e90 2024-04-21T08:38:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.675+0000 7ff164a5f740 1 Processor -- start 2024-04-21T08:38:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.675+0000 7ff164a5f740 1 -- start start 2024-04-21T08:38:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.675+0000 7ff164a5f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f458e40 0x56240f459260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.675+0000 7ff164a5f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56240f459830 0x56240f459cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.675+0000 7ff164a5f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56240f45a2d0 0x56240f460eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.675+0000 7ff164a5f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56240f3bdb10 con 0x56240f458e40 2024-04-21T08:38:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.675+0000 7ff164a5f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56240f35b240 con 0x56240f45a2d0 2024-04-21T08:38:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.675+0000 7ff164a5f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56240f3b9ad0 con 0x56240f459830 2024-04-21T08:38:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.675+0000 7ff158ff9640 1 --2- 172.21.15.43:0/1062714664 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ff1440a3c30 0x7ff1440a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.675+0000 7ff15bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f458e40 0x56240f459260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.675+0000 7ff15bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f458e40 0x56240f459260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36156/0 (socket says 172.21.15.43:36156) 2024-04-21T08:38:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.675+0000 7ff15bfff640 1 -- 172.21.15.43:0/2074379161 learned_addr learned my addr 172.21.15.43:0/2074379161 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.676+0000 7ff15affd640 1 --2- 172.21.15.43:0/2074379161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56240f45a2d0 0x56240f460eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.676+0000 7ff15bfff640 1 --2- 172.21.15.43:0/1062714664 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ff1440a3c30 0x7ff1440a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.676+0000 7ff158ff9640 1 -- 172.21.15.43:0/1062714664 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7ff14c0d2010 con 0x7ff15c100e90 2024-04-21T08:38:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.676+0000 7ff15b7fe640 1 --2- 172.21.15.43:0/2074379161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56240f459830 0x56240f459cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.676+0000 7ff15bfff640 1 -- 172.21.15.43:0/2074379161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56240f459830 msgr2=0x56240f459cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.676+0000 7ff15bfff640 1 --2- 172.21.15.43:0/2074379161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56240f459830 0x56240f459cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.676+0000 7ff15bfff640 1 -- 172.21.15.43:0/2074379161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56240f45a2d0 msgr2=0x56240f460eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.676+0000 7ff15bfff640 1 --2- 172.21.15.43:0/2074379161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56240f45a2d0 0x56240f460eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.676+0000 7ff15bfff640 1 -- 172.21.15.43:0/2074379161 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56240f0d02d0 con 0x56240f458e40 2024-04-21T08:38:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.676+0000 7ff15bfff640 1 --2- 172.21.15.43:0/2074379161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f458e40 0x56240f459260 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff154012e00 tx=0x7ff15400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f7126955d9af3d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.676+0000 7ff15bfff640 1 --2- 172.21.15.43:0/1062714664 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ff1440a3c30 0x7ff1440a60f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff15400fc30 tx=0x7ff15401a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.676+0000 7ff141ffb640 1 -- 172.21.15.43:0/2074379161 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154020070 con 0x56240f458e40 2024-04-21T08:38:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.676+0000 7ff141ffb640 1 -- 172.21.15.43:0/2074379161 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff154008b50 con 0x56240f458e40 2024-04-21T08:38:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.676+0000 7ff141ffb640 1 -- 172.21.15.43:0/2074379161 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15400d3d0 con 0x56240f458e40 2024-04-21T08:38:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.677+0000 7ff164a5f740 1 -- 172.21.15.43:0/2074379161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f458e40 msgr2=0x56240f459260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.677+0000 7ff164a5f740 1 --2- 172.21.15.43:0/2074379161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f458e40 0x56240f459260 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff154012e00 tx=0x7ff15400f730 comp rx=0 tx=0).stop 2024-04-21T08:38:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.677+0000 7ff164a5f740 1 -- 172.21.15.43:0/2074379161 shutdown_connections 2024-04-21T08:38:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.677+0000 7ff164a5f740 1 --2- 172.21.15.43:0/2074379161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56240f45a2d0 0x56240f460eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.677+0000 7ff164a5f740 1 --2- 172.21.15.43:0/2074379161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56240f459830 0x56240f459cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.677+0000 7ff164a5f740 1 --2- 172.21.15.43:0/2074379161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f458e40 0x56240f459260 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.677+0000 7ff164a5f740 1 -- 172.21.15.43:0/2074379161 >> 172.21.15.43:0/2074379161 conn(0x56240f454f40 msgr2=0x56240f453490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.677+0000 7ff164a5f740 1 -- 172.21.15.43:0/2074379161 shutdown_connections 2024-04-21T08:38:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.677+0000 7ff164a5f740 1 -- 172.21.15.43:0/2074379161 wait complete. 2024-04-21T08:38:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.677+0000 7ff164a5f740 1 -- start start 2024-04-21T08:38:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.677+0000 7ff164a5f740 1 Processor -- start 2024-04-21T08:38:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.678+0000 7ff164a5f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56240f458e40 0x56240f462ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.678+0000 7ff164a5f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f459830 0x56240f463400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.678+0000 7ff164a5f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56240f45a2d0 0x56240f46a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.678+0000 7ff164a5f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56240f461450 con 0x56240f459830 2024-04-21T08:38:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.678+0000 7ff164a5f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56240f346190 con 0x56240f458e40 2024-04-21T08:38:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.678+0000 7ff164a5f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56240f4615d0 con 0x56240f45a2d0 2024-04-21T08:38:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.678+0000 7ff164a5f740 10 client.0 _ll_register_callbacks cb 0x56240f3418b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.678+0000 7ff15affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f459830 0x56240f463400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.678+0000 7ff15affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f459830 0x56240f463400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36168/0 (socket says 172.21.15.43:36168) 2024-04-21T08:38:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.678+0000 7ff15affd640 1 -- 172.21.15.43:0/797658917 learned_addr learned my addr 172.21.15.43:0/797658917 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.678+0000 7ff15bfff640 1 --2- 172.21.15.43:0/797658917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56240f45a2d0 0x56240f46a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.678+0000 7ff15b7fe640 1 --2- 172.21.15.43:0/797658917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56240f458e40 0x56240f462ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.678+0000 7ff15affd640 1 -- 172.21.15.43:0/797658917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56240f45a2d0 msgr2=0x56240f46a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.678+0000 7ff15affd640 1 --2- 172.21.15.43:0/797658917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56240f45a2d0 0x56240f46a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.678+0000 7ff15affd640 1 -- 172.21.15.43:0/797658917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56240f458e40 msgr2=0x56240f462ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.678+0000 7ff15affd640 1 --2- 172.21.15.43:0/797658917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56240f458e40 0x56240f462ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.678+0000 7ff15affd640 1 -- 172.21.15.43:0/797658917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56240f19a6b0 con 0x56240f459830 2024-04-21T08:38:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.678+0000 7ff15b7fe640 1 --2- 172.21.15.43:0/797658917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56240f458e40 0x56240f462ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.678+0000 7ff15affd640 1 --2- 172.21.15.43:0/797658917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f459830 0x56240f463400 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff150009900 tx=0x7ff150004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f0aeff191fc93c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.678+0000 7ff13affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:38:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.678+0000 7ff13affd640 1 -- 172.21.15.43:0/797658917 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150005450 con 0x56240f459830 2024-04-21T08:38:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.678+0000 7ff13affd640 1 -- 172.21.15.43:0/797658917 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7ff1500046c0 con 0x56240f459830 2024-04-21T08:38:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.678+0000 7ff164a5f740 1 -- 172.21.15.43:0/797658917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff154008b50 con 0x56240f459830 2024-04-21T08:38:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.679+0000 7ff13affd640 1 -- 172.21.15.43:0/797658917 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff150052050 con 0x56240f459830 2024-04-21T08:38:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.679+0000 7ff164a5f740 10 client.15462 fetch_fsmap learned FSMap version 27 2024-04-21T08:38:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.679+0000 7ff164a5f740 1 -- 172.21.15.43:0/797658917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56240f46d820 con 0x56240f459830 2024-04-21T08:38:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.679+0000 7ff13affd640 1 -- 172.21.15.43:0/797658917 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff154008b50 con 0x56240f459830 2024-04-21T08:38:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.679+0000 7ff13affd640 1 -- 172.21.15.43:0/797658917 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff1500037a0 con 0x56240f459830 2024-04-21T08:38:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.679+0000 7ff164a5f740 10 client.15462 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:38:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.679+0000 7ff164a5f740 10 client.15462 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-21T08:38:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.679+0000 7ff164a5f740 20 client.15462 populate_metadata read hostname 'smithi043' 2024-04-21T08:38:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.679+0000 7ff164a5f740 1 --2- 172.21.15.43:0/797658917 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56240f46fe90 0x56240f471300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.679+0000 7ff164a5f740 4 client.15462 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:38:45.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.679+0000 7ff164a5f740 1 -- 172.21.15.43:0/797658917 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56240f3cb530 con 0x56240f46fe90 2024-04-21T08:38:45.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.680+0000 7ff15b7fe640 1 --2- 172.21.15.43:0/797658917 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56240f46fe90 0x56240f471300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.680+0000 7ff15b7fe640 1 --2- 172.21.15.43:0/797658917 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56240f46fe90 0x56240f471300 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eb725e3e154cbb91 server_cookie=f5d6943b5de51a01 in_seq=0 out_seq=0 2024-04-21T08:38:45.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.680+0000 7ff13affd640 10 client.15462 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:38:45.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.681+0000 7ff13affd640 1 -- 172.21.15.43:0/797658917 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56240f3cb530 con 0x56240f46fe90 2024-04-21T08:38:45.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.681+0000 7ff13affd640 10 client.15462 handle_command_reply: tid=0 2024-04-21T08:38:45.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.681+0000 7ff164a5f740 1 client.15462 shutdown 2024-04-21T08:38:45.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.681+0000 7ff164a5f740 1 -- 172.21.15.43:0/797658917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f459830 msgr2=0x56240f463400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.681+0000 7ff164a5f740 1 --2- 172.21.15.43:0/797658917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f459830 0x56240f463400 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff150009900 tx=0x7ff150004270 comp rx=0 tx=0).stop 2024-04-21T08:38:45.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.681+0000 7ff164a5f740 1 -- 172.21.15.43:0/797658917 shutdown_connections 2024-04-21T08:38:45.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.681+0000 7ff164a5f740 1 --2- 172.21.15.43:0/797658917 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56240f46fe90 0x56240f471300 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.681+0000 7ff164a5f740 1 --2- 172.21.15.43:0/797658917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56240f45a2d0 0x56240f46a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.681+0000 7ff164a5f740 1 --2- 172.21.15.43:0/797658917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f459830 0x56240f463400 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.681+0000 7ff13affd640 0 client.15462 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:38:45.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.681+0000 7ff164a5f740 1 --2- 172.21.15.43:0/797658917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56240f458e40 0x56240f462ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.681+0000 7ff164a5f740 1 -- 172.21.15.43:0/797658917 >> 172.21.15.43:0/797658917 conn(0x56240f454f40 msgr2=0x56240f452a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.681+0000 7ff164a5f740 1 -- 172.21.15.43:0/797658917 shutdown_connections 2024-04-21T08:38:45.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.681+0000 7ff164a5f740 1 -- 172.21.15.43:0/797658917 wait complete. 2024-04-21T08:38:45.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.681+0000 7ff164a5f740 20 client.15462 trim_cache size 0 max 16384 2024-04-21T08:38:45.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.699+0000 7ff164a5f740 1 Processor -- start 2024-04-21T08:38:45.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.699+0000 7ff164a5f740 1 -- start start 2024-04-21T08:38:45.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.699+0000 7ff164a5f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56240f458e40 0x56240f45d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.699+0000 7ff164a5f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56240f459830 0x56240f46a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.699+0000 7ff164a5f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f45a2d0 0x56240f45ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.699+0000 7ff164a5f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56240f46f640 con 0x56240f45a2d0 2024-04-21T08:38:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.700+0000 7ff164a5f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56240f46fd10 con 0x56240f459830 2024-04-21T08:38:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.700+0000 7ff164a5f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56240f4703e0 con 0x56240f458e40 2024-04-21T08:38:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.700+0000 7ff15bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f45a2d0 0x56240f45ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.700+0000 7ff15bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f45a2d0 0x56240f45ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36170/0 (socket says 172.21.15.43:36170) 2024-04-21T08:38:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.700+0000 7ff15bfff640 1 -- 172.21.15.43:0/1009995110 learned_addr learned my addr 172.21.15.43:0/1009995110 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.700+0000 7ff15bfff640 1 -- 172.21.15.43:0/1009995110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56240f458e40 msgr2=0x56240f45d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.700+0000 7ff15bfff640 1 --2- 172.21.15.43:0/1009995110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56240f458e40 0x56240f45d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.700+0000 7ff15bfff640 1 -- 172.21.15.43:0/1009995110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56240f459830 msgr2=0x56240f46a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.700+0000 7ff15bfff640 1 --2- 172.21.15.43:0/1009995110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56240f459830 0x56240f46a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.700+0000 7ff15bfff640 1 -- 172.21.15.43:0/1009995110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56240f471630 con 0x56240f45a2d0 2024-04-21T08:38:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.700+0000 7ff15bfff640 1 --2- 172.21.15.43:0/1009995110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f45a2d0 0x56240f45ddf0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff15401cfd0 tx=0x7ff154004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b482dd9326757f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.700+0000 7ff1417fa640 1 -- 172.21.15.43:0/1009995110 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154020040 con 0x56240f45a2d0 2024-04-21T08:38:45.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.700+0000 7ff1417fa640 1 -- 172.21.15.43:0/1009995110 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff154008b50 con 0x56240f45a2d0 2024-04-21T08:38:45.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.700+0000 7ff1417fa640 1 -- 172.21.15.43:0/1009995110 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15401ccc0 con 0x56240f45a2d0 2024-04-21T08:38:45.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.701+0000 7ff164a5f740 1 -- 172.21.15.43:0/1009995110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f45a2d0 msgr2=0x56240f45ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.701+0000 7ff164a5f740 1 --2- 172.21.15.43:0/1009995110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f45a2d0 0x56240f45ddf0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff15401cfd0 tx=0x7ff154004a00 comp rx=0 tx=0).stop 2024-04-21T08:38:45.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.701+0000 7ff164a5f740 1 -- 172.21.15.43:0/1009995110 shutdown_connections 2024-04-21T08:38:45.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.701+0000 7ff164a5f740 1 --2- 172.21.15.43:0/1009995110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f45a2d0 0x56240f45ddf0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.701+0000 7ff164a5f740 1 --2- 172.21.15.43:0/1009995110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56240f459830 0x56240f46a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.701+0000 7ff164a5f740 1 --2- 172.21.15.43:0/1009995110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56240f458e40 0x56240f45d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.701+0000 7ff164a5f740 1 -- 172.21.15.43:0/1009995110 >> 172.21.15.43:0/1009995110 conn(0x56240f454f40 msgr2=0x56240f45cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.701+0000 7ff164a5f740 1 -- 172.21.15.43:0/1009995110 shutdown_connections 2024-04-21T08:38:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.701+0000 7ff164a5f740 1 -- 172.21.15.43:0/1009995110 wait complete. 2024-04-21T08:38:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.701+0000 7ff164a5f740 1 -- start start 2024-04-21T08:38:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.701+0000 7ff164a5f740 1 Processor -- start 2024-04-21T08:38:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.702+0000 7ff164a5f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f458e40 0x56240f464620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.702+0000 7ff164a5f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56240f459830 0x56240f464b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.702+0000 7ff164a5f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56240f45a2d0 0x56240f469820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.702+0000 7ff164a5f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56240f471180 con 0x56240f458e40 2024-04-21T08:38:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.702+0000 7ff164a5f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56240f470ab0 con 0x56240f45a2d0 2024-04-21T08:38:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.702+0000 7ff164a5f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56240f471850 con 0x56240f459830 2024-04-21T08:38:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.702+0000 7ff164a5f740 10 client.0 _ll_register_callbacks cb 0x7ff150009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.702+0000 7ff15b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f458e40 0x56240f464620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.702+0000 7ff15b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f458e40 0x56240f464620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36186/0 (socket says 172.21.15.43:36186) 2024-04-21T08:38:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.702+0000 7ff15b7fe640 1 -- 172.21.15.43:0/2138351232 learned_addr learned my addr 172.21.15.43:0/2138351232 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:45.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.702+0000 7ff15affd640 1 --2- 172.21.15.43:0/2138351232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56240f459830 0x56240f464b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.702+0000 7ff15b7fe640 1 -- 172.21.15.43:0/2138351232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56240f459830 msgr2=0x56240f464b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.702+0000 7ff15b7fe640 1 --2- 172.21.15.43:0/2138351232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56240f459830 0x56240f464b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.702+0000 7ff15b7fe640 1 -- 172.21.15.43:0/2138351232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56240f45a2d0 msgr2=0x56240f469820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.702+0000 7ff15b7fe640 1 --2- 172.21.15.43:0/2138351232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56240f45a2d0 0x56240f469820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.702+0000 7ff15b7fe640 1 -- 172.21.15.43:0/2138351232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56240f469e60 con 0x56240f458e40 2024-04-21T08:38:45.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.702+0000 7ff15b7fe640 1 --2- 172.21.15.43:0/2138351232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f458e40 0x56240f464620 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00e9b0 tx=0x7ff14c0a23a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40018acb4a7947ee server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.702+0000 7ff13affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:38:45.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.702+0000 7ff13affd640 1 -- 172.21.15.43:0/2138351232 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c0b6070 con 0x56240f458e40 2024-04-21T08:38:45.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.702+0000 7ff13affd640 1 -- 172.21.15.43:0/2138351232 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7ff14c09a570 con 0x56240f458e40 2024-04-21T08:38:45.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.702+0000 7ff164a5f740 1 -- 172.21.15.43:0/2138351232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff154008b50 con 0x56240f458e40 2024-04-21T08:38:45.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.702+0000 7ff13affd640 1 -- 172.21.15.43:0/2138351232 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff14c0b7270 con 0x56240f458e40 2024-04-21T08:38:45.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.702+0000 7ff164a5f740 10 client.15468 fetch_fsmap learned FSMap version 27 2024-04-21T08:38:45.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.702+0000 7ff164a5f740 1 -- 172.21.15.43:0/2138351232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56240f4d66b0 con 0x56240f458e40 2024-04-21T08:38:45.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.703+0000 7ff13affd640 1 -- 172.21.15.43:0/2138351232 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff154008b50 con 0x56240f458e40 2024-04-21T08:38:45.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.703+0000 7ff13affd640 1 -- 172.21.15.43:0/2138351232 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff14c00e040 con 0x56240f458e40 2024-04-21T08:38:45.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.703+0000 7ff164a5f740 10 client.15468 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:38:45.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.703+0000 7ff164a5f740 10 client.15468 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-21T08:38:45.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.703+0000 7ff164a5f740 20 client.15468 populate_metadata read hostname 'smithi043' 2024-04-21T08:38:45.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.703+0000 7ff164a5f740 1 --2- 172.21.15.43:0/2138351232 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56240f4db280 0x56240f4d76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.703+0000 7ff164a5f740 4 client.15468 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:38:45.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.703+0000 7ff164a5f740 1 -- 172.21.15.43:0/2138351232 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "status"}) -- 0x56240f3c53a0 con 0x56240f4db280 2024-04-21T08:38:45.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.703+0000 7ff15affd640 1 --2- 172.21.15.43:0/2138351232 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56240f4db280 0x56240f4d76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.703+0000 7ff13affd640 10 client.15468 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:38:45.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.703+0000 7ff15affd640 1 --2- 172.21.15.43:0/2138351232 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56240f4db280 0x56240f4d76c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=895483c5f1dec1a5 server_cookie=e2b80b0536da7923 in_seq=0 out_seq=0 2024-04-21T08:38:45.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.704+0000 7ff13affd640 1 -- 172.21.15.43:0/2138351232 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56240f3c53a0 con 0x56240f4db280 2024-04-21T08:38:45.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.704+0000 7ff13affd640 10 client.15468 handle_command_reply: tid=0 2024-04-21T08:38:45.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.704+0000 7ff164a5f740 1 client.15468 shutdown 2024-04-21T08:38:45.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.704+0000 7ff164a5f740 1 -- 172.21.15.43:0/2138351232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f458e40 msgr2=0x56240f464620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.704+0000 7ff164a5f740 1 --2- 172.21.15.43:0/2138351232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f458e40 0x56240f464620 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00e9b0 tx=0x7ff14c0a23a0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.704+0000 7ff164a5f740 1 -- 172.21.15.43:0/2138351232 shutdown_connections 2024-04-21T08:38:45.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.704+0000 7ff164a5f740 1 --2- 172.21.15.43:0/2138351232 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56240f4db280 0x56240f4d76c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.704+0000 7ff164a5f740 1 --2- 172.21.15.43:0/2138351232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56240f45a2d0 0x56240f469820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.704+0000 7ff164a5f740 1 --2- 172.21.15.43:0/2138351232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56240f459830 0x56240f464b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.704+0000 7ff13affd640 0 client.15468 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:38:45.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.704+0000 7ff164a5f740 1 --2- 172.21.15.43:0/2138351232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56240f458e40 0x56240f464620 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.704+0000 7ff164a5f740 1 -- 172.21.15.43:0/2138351232 >> 172.21.15.43:0/2138351232 conn(0x56240f454f40 msgr2=0x56240f462ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.704+0000 7ff164a5f740 1 -- 172.21.15.43:0/2138351232 shutdown_connections 2024-04-21T08:38:45.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.704+0000 7ff164a5f740 1 -- 172.21.15.43:0/2138351232 wait complete. 2024-04-21T08:38:45.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.704+0000 7ff164a5f740 20 client.15468 trim_cache size 0 max 16384 2024-04-21T08:38:45.710 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:38:45.710 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "d57a7d98-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:38:45.710 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 3, 2024-04-21T08:38:45.710 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24425, 2024-04-21T08:38:45.710 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-21T08:38:45.710 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-21T08:38:45.710 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-21T08:38:45.710 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 115.822642972, 2024-04-21T08:38:45.710 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 27, 2024-04-21T08:38:45.710 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 94, 2024-04-21T08:38:45.710 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 90, 2024-04-21T08:38:45.710 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 175.98640581500001 2024-04-21T08:38:45.711 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:38:45.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.705+0000 7ff161e77640 1 -- 172.21.15.43:0/1062714664 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ff1440a3c30 msgr2=0x7ff1440a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.705+0000 7ff161e77640 1 --2- 172.21.15.43:0/1062714664 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ff1440a3c30 0x7ff1440a60f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff15400fc30 tx=0x7ff15401a000 comp rx=0 tx=0).stop 2024-04-21T08:38:45.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.705+0000 7ff161e77640 1 -- 172.21.15.43:0/1062714664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff15c100e90 msgr2=0x7ff15c1971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.705+0000 7ff161e77640 1 --2- 172.21.15.43:0/1062714664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff15c100e90 0x7ff15c1971d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00d990 tx=0x7ff14c00de50 comp rx=0 tx=0).stop 2024-04-21T08:38:45.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.705+0000 7ff161e77640 1 -- 172.21.15.43:0/1062714664 shutdown_connections 2024-04-21T08:38:45.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.705+0000 7ff161e77640 1 --2- 172.21.15.43:0/1062714664 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ff1440a3c30 0x7ff1440a60f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.705+0000 7ff161e77640 1 --2- 172.21.15.43:0/1062714664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff15c1029a0 0x7ff15c19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.705+0000 7ff161e77640 1 --2- 172.21.15.43:0/1062714664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff15c101fe0 0x7ff15c197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.705+0000 7ff161e77640 1 --2- 172.21.15.43:0/1062714664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff15c100e90 0x7ff15c1971d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.705+0000 7ff161e77640 1 -- 172.21.15.43:0/1062714664 >> 172.21.15.43:0/1062714664 conn(0x7ff15c0fc400 msgr2=0x7ff15c1054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.705+0000 7ff161e77640 1 -- 172.21.15.43:0/1062714664 shutdown_connections 2024-04-21T08:38:45.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.705+0000 7ff161e77640 1 -- 172.21.15.43:0/1062714664 wait complete. 2024-04-21T08:38:45.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:45 smithi175 ceph-mon[31920]: pgmap v369: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-21T08:38:45.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:45 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2784163321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:45.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:45 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1992026708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:45.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:45 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2213780956' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:45.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:45 smithi043 ceph-mon[27978]: pgmap v369: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-21T08:38:45.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:45 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2784163321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:45.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:45 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1992026708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:45.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:45 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2213780956' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:45.715 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-21T08:38:45.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:45 smithi112 ceph-mon[31085]: pgmap v369: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-21T08:38:45.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:45 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2784163321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:45.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:45 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1992026708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:45.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:45 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2213780956' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:45.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.819+0000 7f381c443640 1 -- 172.21.15.43:0/2710582659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3814107c90 msgr2=0x7f381410a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.819+0000 7f381c443640 1 --2- 172.21.15.43:0/2710582659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3814107c90 0x7f381410a0e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f380c009ca0 tx=0x7f380c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.819+0000 7f381c443640 1 -- 172.21.15.43:0/2710582659 shutdown_connections 2024-04-21T08:38:45.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.819+0000 7f381c443640 1 --2- 172.21.15.43:0/2710582659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3814107c90 0x7f381410a0e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.819+0000 7f381c443640 1 --2- 172.21.15.43:0/2710582659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3814105360 0x7f3814107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.819+0000 7f381c443640 1 --2- 172.21.15.43:0/2710582659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3814102a30 0x7f3814104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.819+0000 7f381c443640 1 -- 172.21.15.43:0/2710582659 >> 172.21.15.43:0/2710582659 conn(0x7f38140fc440 msgr2=0x7f38140fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.819+0000 7f381c443640 1 -- 172.21.15.43:0/2710582659 shutdown_connections 2024-04-21T08:38:45.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.819+0000 7f381c443640 1 -- 172.21.15.43:0/2710582659 wait complete. 2024-04-21T08:38:45.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.820+0000 7f381c443640 1 Processor -- start 2024-04-21T08:38:45.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.820+0000 7f381c443640 1 -- start start 2024-04-21T08:38:45.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.820+0000 7f381c443640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3814102a30 0x7f381419a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.820+0000 7f381c443640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3814105360 0x7f381419a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.820+0000 7f381c443640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3814107c90 0x7f38141a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.820+0000 7f381c443640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3814101870 con 0x7f3814102a30 2024-04-21T08:38:45.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.820+0000 7f381c443640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38141016f0 con 0x7f3814105360 2024-04-21T08:38:45.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.820+0000 7f381c443640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f38141019f0 con 0x7f3814107c90 2024-04-21T08:38:45.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.820+0000 7f381a1df640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3814102a30 0x7f381419a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.820+0000 7f381a1df640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3814102a30 0x7f381419a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36194/0 (socket says 172.21.15.43:36194) 2024-04-21T08:38:45.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.820+0000 7f381a1df640 1 -- 172.21.15.43:0/1838961006 learned_addr learned my addr 172.21.15.43:0/1838961006 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:45.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.821+0000 7f381a9e0640 1 --2- 172.21.15.43:0/1838961006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3814107c90 0x7f38141a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.821+0000 7f381a1df640 1 -- 172.21.15.43:0/1838961006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3814107c90 msgr2=0x7f38141a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.821+0000 7f381a1df640 1 --2- 172.21.15.43:0/1838961006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3814107c90 0x7f38141a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.821+0000 7f38199de640 1 --2- 172.21.15.43:0/1838961006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3814105360 0x7f381419a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.821+0000 7f381a1df640 1 -- 172.21.15.43:0/1838961006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3814105360 msgr2=0x7f381419a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.821+0000 7f381a1df640 1 --2- 172.21.15.43:0/1838961006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3814105360 0x7f381419a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.821+0000 7f381a1df640 1 -- 172.21.15.43:0/1838961006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38141a1f40 con 0x7f3814102a30 2024-04-21T08:38:45.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.821+0000 7f381a9e0640 1 --2- 172.21.15.43:0/1838961006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3814107c90 0x7f38141a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:45.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.821+0000 7f381a1df640 1 --2- 172.21.15.43:0/1838961006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3814102a30 0x7f381419a320 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f380400e9f0 tx=0x7f380400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.821+0000 7f380b7fe640 1 -- 172.21.15.43:0/1838961006 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380400cd80 con 0x7f3814102a30 2024-04-21T08:38:45.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.821+0000 7f380b7fe640 1 -- 172.21.15.43:0/1838961006 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38040108e0 con 0x7f3814102a30 2024-04-21T08:38:45.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.821+0000 7f381c443640 1 -- 172.21.15.43:0/1838961006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38141a2230 con 0x7f3814102a30 2024-04-21T08:38:45.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.821+0000 7f380b7fe640 1 -- 172.21.15.43:0/1838961006 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3804018980 con 0x7f3814102a30 2024-04-21T08:38:45.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.821+0000 7f381c443640 1 -- 172.21.15.43:0/1838961006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38141a2780 con 0x7f3814102a30 2024-04-21T08:38:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.823+0000 7f380b7fe640 1 -- 172.21.15.43:0/1838961006 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3804010430 con 0x7f3814102a30 2024-04-21T08:38:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.824+0000 7f381f036740 1 Processor -- start 2024-04-21T08:38:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.824+0000 7f381f036740 1 -- start start 2024-04-21T08:38:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.824+0000 7f381f036740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71622ee40 0x55d71622f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.824+0000 7f381f036740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d71622f830 0x55d71622fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.824+0000 7f381f036740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7162302d0 0x55d716236eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.824+0000 7f381f036740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d716193b10 con 0x55d7162302d0 2024-04-21T08:38:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.824+0000 7f381f036740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d716131240 con 0x55d71622ee40 2024-04-21T08:38:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.824+0000 7f381f036740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d71618fad0 con 0x55d71622f830 2024-04-21T08:38:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.824+0000 7f38199de640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7162302d0 0x55d716236eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.824+0000 7f380b7fe640 1 --2- 172.21.15.43:0/1838961006 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f38000a3bb0 0x7f38000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.824+0000 7f38199de640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7162302d0 0x55d716236eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36206/0 (socket says 172.21.15.43:36206) 2024-04-21T08:38:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.824+0000 7f38199de640 1 -- 172.21.15.43:0/838538303 learned_addr learned my addr 172.21.15.43:0/838538303 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.824+0000 7f381a9e0640 1 --2- 172.21.15.43:0/838538303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71622ee40 0x55d71622f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.825+0000 7f381a9e0640 1 --2- 172.21.15.43:0/1838961006 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f38000a3bb0 0x7f38000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.825+0000 7f380b7fe640 1 -- 172.21.15.43:0/1838961006 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f38040d0c20 con 0x7f3814102a30 2024-04-21T08:38:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.825+0000 7f381a1df640 1 --2- 172.21.15.43:0/838538303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d71622f830 0x55d71622fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.825+0000 7f38199de640 1 -- 172.21.15.43:0/838538303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d71622f830 msgr2=0x55d71622fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.825+0000 7f38199de640 1 --2- 172.21.15.43:0/838538303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d71622f830 0x55d71622fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.825+0000 7f38199de640 1 -- 172.21.15.43:0/838538303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71622ee40 msgr2=0x55d71622f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.825+0000 7f38199de640 1 --2- 172.21.15.43:0/838538303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71622ee40 0x55d71622f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.825+0000 7f38199de640 1 -- 172.21.15.43:0/838538303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d715ea62d0 con 0x55d7162302d0 2024-04-21T08:38:45.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.825+0000 7f38199de640 1 --2- 172.21.15.43:0/838538303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7162302d0 0x55d716236eb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f381000bae0 tx=0x7f381000d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51efa1c2cf0ce603 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.825+0000 7f381a9e0640 1 --2- 172.21.15.43:0/1838961006 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f38000a3bb0 0x7f38000a6070 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55d71622a200 tx=0x7f380c032340 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.825+0000 7f3808bfe640 1 -- 172.21.15.43:0/838538303 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810017420 con 0x55d7162302d0 2024-04-21T08:38:45.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.825+0000 7f3808bfe640 1 -- 172.21.15.43:0/838538303 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3810017a70 con 0x55d7162302d0 2024-04-21T08:38:45.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.825+0000 7f3808bfe640 1 -- 172.21.15.43:0/838538303 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381000f9d0 con 0x55d7162302d0 2024-04-21T08:38:45.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.826+0000 7f381f036740 1 -- 172.21.15.43:0/838538303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7162302d0 msgr2=0x55d716236eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.826+0000 7f381f036740 1 --2- 172.21.15.43:0/838538303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7162302d0 0x55d716236eb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f381000bae0 tx=0x7f381000d340 comp rx=0 tx=0).stop 2024-04-21T08:38:45.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.826+0000 7f381f036740 1 -- 172.21.15.43:0/838538303 shutdown_connections 2024-04-21T08:38:45.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.826+0000 7f381f036740 1 --2- 172.21.15.43:0/838538303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7162302d0 0x55d716236eb0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.826+0000 7f381f036740 1 --2- 172.21.15.43:0/838538303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d71622f830 0x55d71622fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.826+0000 7f381f036740 1 --2- 172.21.15.43:0/838538303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71622ee40 0x55d71622f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.826+0000 7f381f036740 1 -- 172.21.15.43:0/838538303 >> 172.21.15.43:0/838538303 conn(0x55d71622af40 msgr2=0x55d716229490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.826+0000 7f381f036740 1 -- 172.21.15.43:0/838538303 shutdown_connections 2024-04-21T08:38:45.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.826+0000 7f381f036740 1 -- 172.21.15.43:0/838538303 wait complete. 2024-04-21T08:38:45.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.826+0000 7f381f036740 1 -- start start 2024-04-21T08:38:45.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.826+0000 7f381f036740 1 Processor -- start 2024-04-21T08:38:45.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.827+0000 7f381f036740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d71622ee40 0x55d716238ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.827+0000 7f381f036740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71622f830 0x55d716239430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.827+0000 7f381f036740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7162302d0 0x55d716240470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.827+0000 7f381f036740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d7162374b0 con 0x55d7162302d0 2024-04-21T08:38:45.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.827+0000 7f381f036740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d71611c190 con 0x55d71622f830 2024-04-21T08:38:45.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.827+0000 7f381f036740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d716237630 con 0x55d71622ee40 2024-04-21T08:38:45.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.827+0000 7f381f036740 10 client.0 _ll_register_callbacks cb 0x55d7161178b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:45.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.827+0000 7f381a9e0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7162302d0 0x55d716240470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.827+0000 7f381a9e0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7162302d0 0x55d716240470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36222/0 (socket says 172.21.15.43:36222) 2024-04-21T08:38:45.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.827+0000 7f381a9e0640 1 -- 172.21.15.43:0/3984840768 learned_addr learned my addr 172.21.15.43:0/3984840768 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:45.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.827+0000 7f381a1df640 1 --2- 172.21.15.43:0/3984840768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d71622ee40 0x55d716238ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.827+0000 7f38199de640 1 --2- 172.21.15.43:0/3984840768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71622f830 0x55d716239430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.827+0000 7f381a9e0640 1 -- 172.21.15.43:0/3984840768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d71622ee40 msgr2=0x55d716238ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.827+0000 7f381a9e0640 1 --2- 172.21.15.43:0/3984840768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d71622ee40 0x55d716238ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.827+0000 7f381a9e0640 1 -- 172.21.15.43:0/3984840768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71622f830 msgr2=0x55d716239430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.827+0000 7f381a9e0640 1 --2- 172.21.15.43:0/3984840768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71622f830 0x55d716239430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.827+0000 7f381a9e0640 1 -- 172.21.15.43:0/3984840768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d715f706b0 con 0x55d7162302d0 2024-04-21T08:38:45.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.827+0000 7f38199de640 1 --2- 172.21.15.43:0/3984840768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71622f830 0x55d716239430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:45.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.827+0000 7f381a9e0640 1 --2- 172.21.15.43:0/3984840768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7162302d0 0x55d716240470 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f380c036000 tx=0x7f380c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575291c0fe7544f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.827+0000 7f37fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:38:45.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.827+0000 7f37fdffb640 1 -- 172.21.15.43:0/3984840768 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c052060 con 0x55d7162302d0 2024-04-21T08:38:45.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.827+0000 7f37fdffb640 1 -- 172.21.15.43:0/3984840768 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f380c004250 con 0x55d7162302d0 2024-04-21T08:38:45.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.828+0000 7f381f036740 1 -- 172.21.15.43:0/3984840768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3810017a70 con 0x55d7162302d0 2024-04-21T08:38:45.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.828+0000 7f37fdffb640 1 -- 172.21.15.43:0/3984840768 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f380c059050 con 0x55d7162302d0 2024-04-21T08:38:45.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.828+0000 7f381f036740 10 client.15480 fetch_fsmap learned FSMap version 27 2024-04-21T08:38:45.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.828+0000 7f381f036740 1 -- 172.21.15.43:0/3984840768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d7162438a0 con 0x55d7162302d0 2024-04-21T08:38:45.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.828+0000 7f37fdffb640 1 -- 172.21.15.43:0/3984840768 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3810017a70 con 0x55d7162302d0 2024-04-21T08:38:45.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.828+0000 7f37fdffb640 1 -- 172.21.15.43:0/3984840768 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f380c04db70 con 0x55d7162302d0 2024-04-21T08:38:45.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.828+0000 7f381f036740 10 client.15480 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:38:45.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.828+0000 7f381f036740 10 client.15480 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T08:38:45.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.828+0000 7f381f036740 20 client.15480 populate_metadata read hostname 'smithi043' 2024-04-21T08:38:45.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.828+0000 7f381f036740 1 --2- 172.21.15.43:0/3984840768 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55d716245fa0 0x55d716247410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.828+0000 7f381f036740 4 client.15480 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:38:45.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.828+0000 7f381f036740 1 -- 172.21.15.43:0/3984840768 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7161a1530 con 0x55d716245fa0 2024-04-21T08:38:45.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.829+0000 7f381a1df640 1 --2- 172.21.15.43:0/3984840768 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55d716245fa0 0x55d716247410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.829+0000 7f381a1df640 1 --2- 172.21.15.43:0/3984840768 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55d716245fa0 0x55d716247410 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=989cde49e31a877a server_cookie=cd1de15014f27403 in_seq=0 out_seq=0 2024-04-21T08:38:45.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.829+0000 7f37fdffb640 10 client.15480 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:38:45.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.830+0000 7f37fdffb640 1 -- 172.21.15.43:0/3984840768 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d7161a1530 con 0x55d716245fa0 2024-04-21T08:38:45.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.830+0000 7f37fdffb640 10 client.15480 handle_command_reply: tid=0 2024-04-21T08:38:45.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.830+0000 7f381f036740 1 client.15480 shutdown 2024-04-21T08:38:45.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.830+0000 7f381f036740 1 -- 172.21.15.43:0/3984840768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7162302d0 msgr2=0x55d716240470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.830+0000 7f381f036740 1 --2- 172.21.15.43:0/3984840768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7162302d0 0x55d716240470 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f380c036000 tx=0x7f380c0095c0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.830+0000 7f381f036740 1 -- 172.21.15.43:0/3984840768 shutdown_connections 2024-04-21T08:38:45.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.830+0000 7f381f036740 1 --2- 172.21.15.43:0/3984840768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7162302d0 0x55d716240470 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.830+0000 7f381f036740 1 --2- 172.21.15.43:0/3984840768 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55d716245fa0 0x55d716247410 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.830+0000 7f381f036740 1 --2- 172.21.15.43:0/3984840768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71622f830 0x55d716239430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.830+0000 7f381f036740 1 --2- 172.21.15.43:0/3984840768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d71622ee40 0x55d716238ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.830+0000 7f37fdffb640 0 client.15480 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:38:45.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.830+0000 7f381f036740 1 -- 172.21.15.43:0/3984840768 >> 172.21.15.43:0/3984840768 conn(0x55d71622af40 msgr2=0x55d716228a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.830+0000 7f381f036740 1 -- 172.21.15.43:0/3984840768 shutdown_connections 2024-04-21T08:38:45.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.830+0000 7f381f036740 1 -- 172.21.15.43:0/3984840768 wait complete. 2024-04-21T08:38:45.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.830+0000 7f381f036740 20 client.15480 trim_cache size 0 max 16384 2024-04-21T08:38:45.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.848+0000 7f381f036740 1 Processor -- start 2024-04-21T08:38:45.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.848+0000 7f381f036740 1 -- start start 2024-04-21T08:38:45.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.848+0000 7f381f036740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71622ee40 0x55d716233840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.848+0000 7f381f036740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d71622f830 0x55d716240430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.848+0000 7f381f036740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7162302d0 0x55d716233d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.848+0000 7f381f036740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d716245750 con 0x55d71622f830 2024-04-21T08:38:45.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.848+0000 7f381f036740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d716245e20 con 0x55d71622ee40 2024-04-21T08:38:45.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.848+0000 7f381f036740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d7162464f0 con 0x55d7162302d0 2024-04-21T08:38:45.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.848+0000 7f38199de640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d71622f830 0x55d716240430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.848+0000 7f38199de640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d71622f830 0x55d716240430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36232/0 (socket says 172.21.15.43:36232) 2024-04-21T08:38:45.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.848+0000 7f38199de640 1 -- 172.21.15.43:0/2084070465 learned_addr learned my addr 172.21.15.43:0/2084070465 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:45.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.849+0000 7f381a1df640 1 --2- 172.21.15.43:0/2084070465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71622ee40 0x55d716233840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.849+0000 7f38199de640 1 -- 172.21.15.43:0/2084070465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7162302d0 msgr2=0x55d716233d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.849+0000 7f38199de640 1 --2- 172.21.15.43:0/2084070465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7162302d0 0x55d716233d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.849+0000 7f38199de640 1 -- 172.21.15.43:0/2084070465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71622ee40 msgr2=0x55d716233840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.849+0000 7f38199de640 1 --2- 172.21.15.43:0/2084070465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71622ee40 0x55d716233840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.849+0000 7f38199de640 1 -- 172.21.15.43:0/2084070465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d716237fd0 con 0x55d71622f830 2024-04-21T08:38:45.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.849+0000 7f38199de640 1 --2- 172.21.15.43:0/2084070465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d71622f830 0x55d716240430 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3810019040 tx=0x7f3810005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb86f28d432352f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.849+0000 7f37fffff640 1 -- 172.21.15.43:0/2084070465 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381000dd20 con 0x55d71622f830 2024-04-21T08:38:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.849+0000 7f37fffff640 1 -- 172.21.15.43:0/2084070465 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38100054e0 con 0x55d71622f830 2024-04-21T08:38:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.849+0000 7f37fffff640 1 -- 172.21.15.43:0/2084070465 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381000f5e0 con 0x55d71622f830 2024-04-21T08:38:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.850+0000 7f381f036740 1 -- 172.21.15.43:0/2084070465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d71622f830 msgr2=0x55d716240430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.850+0000 7f381f036740 1 --2- 172.21.15.43:0/2084070465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d71622f830 0x55d716240430 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3810019040 tx=0x7f3810005c70 comp rx=0 tx=0).stop 2024-04-21T08:38:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.850+0000 7f381f036740 1 -- 172.21.15.43:0/2084070465 shutdown_connections 2024-04-21T08:38:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.850+0000 7f381f036740 1 --2- 172.21.15.43:0/2084070465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7162302d0 0x55d716233d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.850+0000 7f381f036740 1 --2- 172.21.15.43:0/2084070465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d71622f830 0x55d716240430 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.850+0000 7f381f036740 1 --2- 172.21.15.43:0/2084070465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71622ee40 0x55d716233840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.850+0000 7f381f036740 1 -- 172.21.15.43:0/2084070465 >> 172.21.15.43:0/2084070465 conn(0x55d71622af40 msgr2=0x55d716236e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.850+0000 7f381f036740 1 -- 172.21.15.43:0/2084070465 shutdown_connections 2024-04-21T08:38:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.850+0000 7f381f036740 1 -- 172.21.15.43:0/2084070465 wait complete. 2024-04-21T08:38:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.850+0000 7f381f036740 1 -- start start 2024-04-21T08:38:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.850+0000 7f381f036740 1 Processor -- start 2024-04-21T08:38:45.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.850+0000 7f381f036740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71622ee40 0x55d716228330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.850+0000 7f381f036740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d71622f830 0x55d716228870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.850+0000 7f381f036740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7162302d0 0x55d7162a9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.850+0000 7f381f036740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d716247290 con 0x55d71622f830 2024-04-21T08:38:45.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.850+0000 7f381f036740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d716246bc0 con 0x55d71622ee40 2024-04-21T08:38:45.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.850+0000 7f381f036740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d716247960 con 0x55d7162302d0 2024-04-21T08:38:45.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.850+0000 7f381f036740 10 client.0 _ll_register_callbacks cb 0x7f380c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:45.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.850+0000 7f38199de640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d71622f830 0x55d716228870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.850+0000 7f38199de640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d71622f830 0x55d716228870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36246/0 (socket says 172.21.15.43:36246) 2024-04-21T08:38:45.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.850+0000 7f38199de640 1 -- 172.21.15.43:0/4019701075 learned_addr learned my addr 172.21.15.43:0/4019701075 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:45.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.851+0000 7f381a9e0640 1 --2- 172.21.15.43:0/4019701075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7162302d0 0x55d7162a9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.851+0000 7f381a1df640 1 --2- 172.21.15.43:0/4019701075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71622ee40 0x55d716228330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.851+0000 7f38199de640 1 -- 172.21.15.43:0/4019701075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7162302d0 msgr2=0x55d7162a9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.851+0000 7f38199de640 1 --2- 172.21.15.43:0/4019701075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7162302d0 0x55d7162a9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.851+0000 7f38199de640 1 -- 172.21.15.43:0/4019701075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71622ee40 msgr2=0x55d716228330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.851+0000 7f38199de640 1 --2- 172.21.15.43:0/4019701075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71622ee40 0x55d716228330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.851+0000 7f38199de640 1 -- 172.21.15.43:0/4019701075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7162aa550 con 0x55d71622f830 2024-04-21T08:38:45.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.851+0000 7f381a1df640 1 --2- 172.21.15.43:0/4019701075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71622ee40 0x55d716228330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:45.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.851+0000 7f38199de640 1 --2- 172.21.15.43:0/4019701075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d71622f830 0x55d716228870 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f381000d840 tx=0x7f381000dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee75f88ecc4ecd13 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.851+0000 7f37fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:38:45.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.851+0000 7f37fdffb640 1 -- 172.21.15.43:0/4019701075 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810031030 con 0x55d71622f830 2024-04-21T08:38:45.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.851+0000 7f37fdffb640 1 -- 172.21.15.43:0/4019701075 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f3810004e00 con 0x55d71622f830 2024-04-21T08:38:45.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.851+0000 7f381f036740 1 -- 172.21.15.43:0/4019701075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38100054e0 con 0x55d71622f830 2024-04-21T08:38:45.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.851+0000 7f37fdffb640 1 -- 172.21.15.43:0/4019701075 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f381003e050 con 0x55d71622f830 2024-04-21T08:38:45.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.851+0000 7f381f036740 10 client.15486 fetch_fsmap learned FSMap version 27 2024-04-21T08:38:45.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.851+0000 7f381f036740 1 -- 172.21.15.43:0/4019701075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d7162ab850 con 0x55d71622f830 2024-04-21T08:38:45.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.852+0000 7f37fdffb640 1 -- 172.21.15.43:0/4019701075 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38100054e0 con 0x55d71622f830 2024-04-21T08:38:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.852+0000 7f37fdffb640 1 -- 172.21.15.43:0/4019701075 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f38100384d0 con 0x55d71622f830 2024-04-21T08:38:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.852+0000 7f381f036740 10 client.15486 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:38:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.852+0000 7f381f036740 10 client.15486 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T08:38:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.852+0000 7f381f036740 20 client.15486 populate_metadata read hostname 'smithi043' 2024-04-21T08:38:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.852+0000 7f381f036740 1 --2- 172.21.15.43:0/4019701075 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55d7162afdb0 0x55d7162ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.852+0000 7f381f036740 4 client.15486 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:38:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.852+0000 7f381f036740 1 -- 172.21.15.43:0/4019701075 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "status"}) -- 0x55d71619b3a0 con 0x55d7162afdb0 2024-04-21T08:38:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.852+0000 7f381a1df640 1 --2- 172.21.15.43:0/4019701075 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55d7162afdb0 0x55d7162ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.852+0000 7f37fdffb640 10 client.15486 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:38:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.852+0000 7f381a1df640 1 --2- 172.21.15.43:0/4019701075 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55d7162afdb0 0x55d7162ac8c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=96b666c86754b598 server_cookie=8078dbbdfb63cc99 in_seq=0 out_seq=0 2024-04-21T08:38:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f37fdffb640 1 -- 172.21.15.43:0/4019701075 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55d71619b3a0 con 0x55d7162afdb0 2024-04-21T08:38:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f37fdffb640 10 client.15486 handle_command_reply: tid=0 2024-04-21T08:38:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f381f036740 1 client.15486 shutdown 2024-04-21T08:38:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f381f036740 1 -- 172.21.15.43:0/4019701075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d71622f830 msgr2=0x55d716228870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f381f036740 1 --2- 172.21.15.43:0/4019701075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d71622f830 0x55d716228870 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f381000d840 tx=0x7f381000dda0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f381f036740 1 -- 172.21.15.43:0/4019701075 shutdown_connections 2024-04-21T08:38:45.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f381f036740 1 --2- 172.21.15.43:0/4019701075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7162302d0 0x55d7162a9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f381f036740 1 --2- 172.21.15.43:0/4019701075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d71622f830 0x55d716228870 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f381f036740 1 --2- 172.21.15.43:0/4019701075 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55d7162afdb0 0x55d7162ac8c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f381f036740 1 --2- 172.21.15.43:0/4019701075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d71622ee40 0x55d716228330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f381f036740 1 -- 172.21.15.43:0/4019701075 >> 172.21.15.43:0/4019701075 conn(0x55d71622af40 msgr2=0x55d716229350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f37fdffb640 0 client.15486 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:38:45.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f381f036740 1 -- 172.21.15.43:0/4019701075 shutdown_connections 2024-04-21T08:38:45.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f381f036740 1 -- 172.21.15.43:0/4019701075 wait complete. 2024-04-21T08:38:45.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f381f036740 20 client.15486 trim_cache size 0 max 16384 2024-04-21T08:38:45.860 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:38:45.860 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "d57a7d98-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:38:45.860 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 2, 2024-04-21T08:38:45.860 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24487, 2024-04-21T08:38:45.860 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-21T08:38:45.860 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-21T08:38:45.860 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-21T08:38:45.860 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 116.792747291, 2024-04-21T08:38:45.860 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 27, 2024-04-21T08:38:45.860 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 94, 2024-04-21T08:38:45.860 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 90, 2024-04-21T08:38:45.860 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 155.580896026 2024-04-21T08:38:45.860 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:38:45.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f381c443640 1 -- 172.21.15.43:0/1838961006 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f38000a3bb0 msgr2=0x7f38000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f381c443640 1 --2- 172.21.15.43:0/1838961006 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f38000a3bb0 0x7f38000a6070 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55d71622a200 tx=0x7f380c032340 comp rx=0 tx=0).stop 2024-04-21T08:38:45.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f381c443640 1 -- 172.21.15.43:0/1838961006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3814102a30 msgr2=0x7f381419a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f381c443640 1 --2- 172.21.15.43:0/1838961006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3814102a30 0x7f381419a320 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f380400e9f0 tx=0x7f380400eeb0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f381c443640 1 -- 172.21.15.43:0/1838961006 shutdown_connections 2024-04-21T08:38:45.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f381c443640 1 --2- 172.21.15.43:0/1838961006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3814107c90 0x7f38141a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f381c443640 1 --2- 172.21.15.43:0/1838961006 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f38000a3bb0 0x7f38000a6070 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f381c443640 1 --2- 172.21.15.43:0/1838961006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3814105360 0x7f381419a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f381c443640 1 --2- 172.21.15.43:0/1838961006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3814102a30 0x7f381419a320 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f381c443640 1 -- 172.21.15.43:0/1838961006 >> 172.21.15.43:0/1838961006 conn(0x7f38140fc440 msgr2=0x7f3814108070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f381c443640 1 -- 172.21.15.43:0/1838961006 shutdown_connections 2024-04-21T08:38:45.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.853+0000 7f381c443640 1 -- 172.21.15.43:0/1838961006 wait complete. 2024-04-21T08:38:45.863 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-21T08:38:45.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.967+0000 7f99e8bb6640 1 -- 172.21.15.43:0/452732140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99e4103b50 msgr2=0x7f99e410a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.967+0000 7f99e8bb6640 1 --2- 172.21.15.43:0/452732140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99e4103b50 0x7f99e410a3e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f99dc009ca0 tx=0x7f99dc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.967+0000 7f99e8bb6640 1 -- 172.21.15.43:0/452732140 shutdown_connections 2024-04-21T08:38:45.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.967+0000 7f99e8bb6640 1 --2- 172.21.15.43:0/452732140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99e4103b50 0x7f99e410a3e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.967+0000 7f99e8bb6640 1 --2- 172.21.15.43:0/452732140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99e4103190 0x7f99e4103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.967+0000 7f99e8bb6640 1 --2- 172.21.15.43:0/452732140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e4101fa0 0x7f99e41023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.967+0000 7f99e8bb6640 1 -- 172.21.15.43:0/452732140 >> 172.21.15.43:0/452732140 conn(0x7f99e40fd530 msgr2=0x7f99e40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.967+0000 7f99e8bb6640 1 -- 172.21.15.43:0/452732140 shutdown_connections 2024-04-21T08:38:45.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.967+0000 7f99e8bb6640 1 -- 172.21.15.43:0/452732140 wait complete. 2024-04-21T08:38:45.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.968+0000 7f99e8bb6640 1 Processor -- start 2024-04-21T08:38:45.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.968+0000 7f99e8bb6640 1 -- start start 2024-04-21T08:38:45.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.968+0000 7f99e8bb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99e4101fa0 0x7f99e4075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.968+0000 7f99e8bb6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99e4103190 0x7f99e4074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.968+0000 7f99e8bb6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e4103b50 0x7f99e4074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.968+0000 7f99e8bb6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f99e410dae0 con 0x7f99e4103190 2024-04-21T08:38:45.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.968+0000 7f99e8bb6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f99e410d960 con 0x7f99e4101fa0 2024-04-21T08:38:45.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.968+0000 7f99e8bb6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f99e410dc60 con 0x7f99e4103b50 2024-04-21T08:38:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.968+0000 7f99e1d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99e4103190 0x7f99e4074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.968+0000 7f99e1d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99e4103190 0x7f99e4074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36270/0 (socket says 172.21.15.43:36270) 2024-04-21T08:38:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.968+0000 7f99e1d9b640 1 -- 172.21.15.43:0/3773432786 learned_addr learned my addr 172.21.15.43:0/3773432786 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.968+0000 7f99e259c640 1 --2- 172.21.15.43:0/3773432786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99e4101fa0 0x7f99e4075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.968+0000 7f99e2d9d640 1 --2- 172.21.15.43:0/3773432786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e4103b50 0x7f99e4074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.968+0000 7f99e1d9b640 1 -- 172.21.15.43:0/3773432786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e4103b50 msgr2=0x7f99e4074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.968+0000 7f99e1d9b640 1 --2- 172.21.15.43:0/3773432786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e4103b50 0x7f99e4074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.968+0000 7f99e1d9b640 1 -- 172.21.15.43:0/3773432786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99e4101fa0 msgr2=0x7f99e4075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.968+0000 7f99e1d9b640 1 --2- 172.21.15.43:0/3773432786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99e4101fa0 0x7f99e4075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.968+0000 7f99e1d9b640 1 -- 172.21.15.43:0/3773432786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e4075110 con 0x7f99e4103190 2024-04-21T08:38:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.968+0000 7f99e2d9d640 1 --2- 172.21.15.43:0/3773432786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e4103b50 0x7f99e4074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.969+0000 7f99e1d9b640 1 --2- 172.21.15.43:0/3773432786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99e4103190 0x7f99e4074340 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00c890 tx=0x7f99cc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.969+0000 7f99db7fe640 1 -- 172.21.15.43:0/3773432786 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc004220 con 0x7f99e4103190 2024-04-21T08:38:45.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.969+0000 7f99db7fe640 1 -- 172.21.15.43:0/3773432786 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99cc0043c0 con 0x7f99e4103190 2024-04-21T08:38:45.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.969+0000 7f99e8bb6640 1 -- 172.21.15.43:0/3773432786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e41a2280 con 0x7f99e4103190 2024-04-21T08:38:45.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.969+0000 7f99db7fe640 1 -- 172.21.15.43:0/3773432786 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc004a00 con 0x7f99e4103190 2024-04-21T08:38:45.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.969+0000 7f99e8bb6640 1 -- 172.21.15.43:0/3773432786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e41a27d0 con 0x7f99e4103190 2024-04-21T08:38:45.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.971+0000 7f99db7fe640 1 -- 172.21.15.43:0/3773432786 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f99cc0079b0 con 0x7f99e4103190 2024-04-21T08:38:45.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.972+0000 7f99eb7ac740 1 Processor -- start 2024-04-21T08:38:45.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.972+0000 7f99eb7ac740 1 -- start start 2024-04-21T08:38:45.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.972+0000 7f99eb7ac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565372b7fe40 0x565372b80260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.972+0000 7f99eb7ac740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b80830 0x565372b80cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.972+0000 7f99eb7ac740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565372b812d0 0x565372b87eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.972+0000 7f99eb7ac740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x565372ae4b10 con 0x565372b80830 2024-04-21T08:38:45.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.972+0000 7f99eb7ac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x565372a82240 con 0x565372b7fe40 2024-04-21T08:38:45.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.972+0000 7f99eb7ac740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x565372ae0ad0 con 0x565372b812d0 2024-04-21T08:38:45.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.972+0000 7f99e259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b80830 0x565372b80cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.972+0000 7f99e259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b80830 0x565372b80cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36280/0 (socket says 172.21.15.43:36280) 2024-04-21T08:38:45.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.972+0000 7f99e259c640 1 -- 172.21.15.43:0/1337395479 learned_addr learned my addr 172.21.15.43:0/1337395479 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:45.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.972+0000 7f99db7fe640 1 --2- 172.21.15.43:0/3773432786 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f99d00a3bb0 0x7f99d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.972+0000 7f99e1d9b640 1 --2- 172.21.15.43:0/1337395479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565372b812d0 0x565372b87eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.972+0000 7f99e2d9d640 1 --2- 172.21.15.43:0/1337395479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565372b7fe40 0x565372b80260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.972+0000 7f99e2d9d640 1 --2- 172.21.15.43:0/3773432786 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f99d00a3bb0 0x7f99d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.972+0000 7f99e259c640 1 -- 172.21.15.43:0/1337395479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565372b812d0 msgr2=0x565372b87eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.972+0000 7f99e259c640 1 --2- 172.21.15.43:0/1337395479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565372b812d0 0x565372b87eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.973+0000 7f99e259c640 1 -- 172.21.15.43:0/1337395479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565372b7fe40 msgr2=0x565372b80260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.973+0000 7f99e259c640 1 --2- 172.21.15.43:0/1337395479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565372b7fe40 0x565372b80260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.973+0000 7f99e259c640 1 -- 172.21.15.43:0/1337395479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653727f72d0 con 0x565372b80830 2024-04-21T08:38:45.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.973+0000 7f99db7fe640 1 -- 172.21.15.43:0/3773432786 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f99cc0d05f0 con 0x7f99e4103190 2024-04-21T08:38:45.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.973+0000 7f99e259c640 1 --2- 172.21.15.43:0/1337395479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b80830 0x565372b80cb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f99d400bcb0 tx=0x7f99d40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52dbfc6f85c56252 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.973+0000 7f99e2d9d640 1 --2- 172.21.15.43:0/3773432786 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f99d00a3bb0 0x7f99d00a6070 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f99dc032380 tx=0x7f99dc0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.973+0000 7f99d8ff9640 1 -- 172.21.15.43:0/1337395479 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4005020 con 0x565372b80830 2024-04-21T08:38:45.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.973+0000 7f99d8ff9640 1 -- 172.21.15.43:0/1337395479 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99d40047a0 con 0x565372b80830 2024-04-21T08:38:45.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.973+0000 7f99d8ff9640 1 -- 172.21.15.43:0/1337395479 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4011730 con 0x565372b80830 2024-04-21T08:38:45.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.974+0000 7f99eb7ac740 1 -- 172.21.15.43:0/1337395479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b80830 msgr2=0x565372b80cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.974+0000 7f99eb7ac740 1 --2- 172.21.15.43:0/1337395479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b80830 0x565372b80cb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f99d400bcb0 tx=0x7f99d40107a0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.974+0000 7f99eb7ac740 1 -- 172.21.15.43:0/1337395479 shutdown_connections 2024-04-21T08:38:45.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.974+0000 7f99eb7ac740 1 --2- 172.21.15.43:0/1337395479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565372b812d0 0x565372b87eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.974+0000 7f99eb7ac740 1 --2- 172.21.15.43:0/1337395479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b80830 0x565372b80cb0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.974+0000 7f99eb7ac740 1 --2- 172.21.15.43:0/1337395479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565372b7fe40 0x565372b80260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.974+0000 7f99eb7ac740 1 -- 172.21.15.43:0/1337395479 >> 172.21.15.43:0/1337395479 conn(0x565372b7bf40 msgr2=0x565372b7a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.974+0000 7f99eb7ac740 1 -- 172.21.15.43:0/1337395479 shutdown_connections 2024-04-21T08:38:45.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.974+0000 7f99eb7ac740 1 -- 172.21.15.43:0/1337395479 wait complete. 2024-04-21T08:38:45.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.974+0000 7f99eb7ac740 1 -- start start 2024-04-21T08:38:45.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.974+0000 7f99eb7ac740 1 Processor -- start 2024-04-21T08:38:45.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.974+0000 7f99eb7ac740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565372b7fe40 0x565372b89e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.974+0000 7f99eb7ac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565372b80830 0x565372b8a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.974+0000 7f99eb7ac740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b812d0 0x565372b913d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.974+0000 7f99eb7ac740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x565372b884b0 con 0x565372b812d0 2024-04-21T08:38:45.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.974+0000 7f99eb7ac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x565372a6d190 con 0x565372b80830 2024-04-21T08:38:45.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.974+0000 7f99eb7ac740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x565372b88630 con 0x565372b7fe40 2024-04-21T08:38:45.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.974+0000 7f99eb7ac740 10 client.0 _ll_register_callbacks cb 0x565372a688b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:45.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.974+0000 7f99e2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b812d0 0x565372b913d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.975+0000 7f99e2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b812d0 0x565372b913d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36292/0 (socket says 172.21.15.43:36292) 2024-04-21T08:38:45.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.975+0000 7f99e2d9d640 1 -- 172.21.15.43:0/2220388974 learned_addr learned my addr 172.21.15.43:0/2220388974 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:45.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.975+0000 7f99e259c640 1 --2- 172.21.15.43:0/2220388974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565372b7fe40 0x565372b89e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.975+0000 7f99e1d9b640 1 --2- 172.21.15.43:0/2220388974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565372b80830 0x565372b8a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.975+0000 7f99e2d9d640 1 -- 172.21.15.43:0/2220388974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565372b7fe40 msgr2=0x565372b89e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.975+0000 7f99e2d9d640 1 --2- 172.21.15.43:0/2220388974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565372b7fe40 0x565372b89e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.975+0000 7f99e2d9d640 1 -- 172.21.15.43:0/2220388974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565372b80830 msgr2=0x565372b8a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:45.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.975+0000 7f99e2d9d640 1 --2- 172.21.15.43:0/2220388974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565372b80830 0x565372b8a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.975+0000 7f99e2d9d640 1 -- 172.21.15.43:0/2220388974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653728c16b0 con 0x565372b812d0 2024-04-21T08:38:45.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.975+0000 7f99e1d9b640 1 --2- 172.21.15.43:0/2220388974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565372b80830 0x565372b8a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:45.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.975+0000 7f99e2d9d640 1 --2- 172.21.15.43:0/2220388974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b812d0 0x565372b913d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f99dc032520 tx=0x7f99dc02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa58c1f148ea773d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.975+0000 7f99c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:38:45.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.975+0000 7f99c9ffb640 1 -- 172.21.15.43:0/2220388974 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99dc052060 con 0x565372b812d0 2024-04-21T08:38:45.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.975+0000 7f99c9ffb640 1 -- 172.21.15.43:0/2220388974 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f99dc003e00 con 0x565372b812d0 2024-04-21T08:38:45.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.975+0000 7f99eb7ac740 1 -- 172.21.15.43:0/2220388974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99d40047a0 con 0x565372b812d0 2024-04-21T08:38:45.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.975+0000 7f99c9ffb640 1 -- 172.21.15.43:0/2220388974 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f99dc039040 con 0x565372b812d0 2024-04-21T08:38:45.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.975+0000 7f99eb7ac740 10 client.15498 fetch_fsmap learned FSMap version 27 2024-04-21T08:38:45.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.975+0000 7f99eb7ac740 1 -- 172.21.15.43:0/2220388974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x565372b947b0 con 0x565372b812d0 2024-04-21T08:38:45.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.975+0000 7f99c9ffb640 1 -- 172.21.15.43:0/2220388974 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99d40047a0 con 0x565372b812d0 2024-04-21T08:38:45.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.976+0000 7f99c9ffb640 1 -- 172.21.15.43:0/2220388974 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f99dc042820 con 0x565372b812d0 2024-04-21T08:38:45.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.976+0000 7f99eb7ac740 10 client.15498 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:38:45.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.976+0000 7f99eb7ac740 10 client.15498 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-21T08:38:45.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.976+0000 7f99eb7ac740 20 client.15498 populate_metadata read hostname 'smithi043' 2024-04-21T08:38:45.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.976+0000 7f99eb7ac740 1 --2- 172.21.15.43:0/2220388974 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x565372b96e70 0x565372b982e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.976+0000 7f99eb7ac740 4 client.15498 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:38:45.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.976+0000 7f99eb7ac740 1 -- 172.21.15.43:0/2220388974 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565372af2530 con 0x565372b96e70 2024-04-21T08:38:45.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.976+0000 7f99e259c640 1 --2- 172.21.15.43:0/2220388974 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x565372b96e70 0x565372b982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.976+0000 7f99e259c640 1 --2- 172.21.15.43:0/2220388974 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x565372b96e70 0x565372b982e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dd3a1f14b0bb425 server_cookie=a80071bbca568c37 in_seq=0 out_seq=0 2024-04-21T08:38:45.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.976+0000 7f99c9ffb640 10 client.15498 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:38:45.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.977+0000 7f99c9ffb640 1 -- 172.21.15.43:0/2220388974 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565372af2530 con 0x565372b96e70 2024-04-21T08:38:45.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.977+0000 7f99c9ffb640 10 client.15498 handle_command_reply: tid=0 2024-04-21T08:38:45.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.977+0000 7f99eb7ac740 1 client.15498 shutdown 2024-04-21T08:38:45.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.977+0000 7f99eb7ac740 1 -- 172.21.15.43:0/2220388974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b812d0 msgr2=0x565372b913d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.977+0000 7f99eb7ac740 1 --2- 172.21.15.43:0/2220388974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b812d0 0x565372b913d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f99dc032520 tx=0x7f99dc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.977+0000 7f99eb7ac740 1 -- 172.21.15.43:0/2220388974 shutdown_connections 2024-04-21T08:38:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.977+0000 7f99eb7ac740 1 --2- 172.21.15.43:0/2220388974 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x565372b96e70 0x565372b982e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.977+0000 7f99eb7ac740 1 --2- 172.21.15.43:0/2220388974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b812d0 0x565372b913d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.977+0000 7f99eb7ac740 1 --2- 172.21.15.43:0/2220388974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565372b80830 0x565372b8a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.977+0000 7f99eb7ac740 1 --2- 172.21.15.43:0/2220388974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565372b7fe40 0x565372b89e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.977+0000 7f99eb7ac740 1 -- 172.21.15.43:0/2220388974 >> 172.21.15.43:0/2220388974 conn(0x565372b7bf40 msgr2=0x565372b79a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.977+0000 7f99c9ffb640 0 client.15498 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:38:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.977+0000 7f99eb7ac740 1 -- 172.21.15.43:0/2220388974 shutdown_connections 2024-04-21T08:38:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.977+0000 7f99eb7ac740 1 -- 172.21.15.43:0/2220388974 wait complete. 2024-04-21T08:38:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.978+0000 7f99eb7ac740 20 client.15498 trim_cache size 0 max 16384 2024-04-21T08:38:46.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.996+0000 7f99eb7ac740 1 Processor -- start 2024-04-21T08:38:46.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.996+0000 7f99eb7ac740 1 -- start start 2024-04-21T08:38:46.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.996+0000 7f99eb7ac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565372b7fe40 0x565372b88a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.996+0000 7f99eb7ac740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b80830 0x565372b912f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.996+0000 7f99eb7ac740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565372b812d0 0x565372b89e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.996+0000 7f99eb7ac740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x565372b96620 con 0x565372b80830 2024-04-21T08:38:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.996+0000 7f99eb7ac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x565372b96cf0 con 0x565372b7fe40 2024-04-21T08:38:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.996+0000 7f99eb7ac740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x565372b973c0 con 0x565372b812d0 2024-04-21T08:38:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.996+0000 7f99e1d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b80830 0x565372b912f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.996+0000 7f99e1d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b80830 0x565372b912f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36294/0 (socket says 172.21.15.43:36294) 2024-04-21T08:38:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.996+0000 7f99e1d9b640 1 -- 172.21.15.43:0/3835214049 learned_addr learned my addr 172.21.15.43:0/3835214049 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.996+0000 7f99e259c640 1 --2- 172.21.15.43:0/3835214049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565372b7fe40 0x565372b88a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.996+0000 7f99e1d9b640 1 -- 172.21.15.43:0/3835214049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565372b812d0 msgr2=0x565372b89e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.996+0000 7f99e1d9b640 1 --2- 172.21.15.43:0/3835214049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565372b812d0 0x565372b89e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.996+0000 7f99e1d9b640 1 -- 172.21.15.43:0/3835214049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565372b7fe40 msgr2=0x565372b88a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.996+0000 7f99e1d9b640 1 --2- 172.21.15.43:0/3835214049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565372b7fe40 0x565372b88a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.996+0000 7f99e1d9b640 1 -- 172.21.15.43:0/3835214049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565372b97870 con 0x565372b80830 2024-04-21T08:38:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.996+0000 7f99e1d9b640 1 --2- 172.21.15.43:0/3835214049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b80830 0x565372b912f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f99cc0cdca0 tx=0x7f99cc0c7590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9950fbafc23639af server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:46.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.996+0000 7f99cbfff640 1 -- 172.21.15.43:0/3835214049 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc01d040 con 0x565372b80830 2024-04-21T08:38:46.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.996+0000 7f99cbfff640 1 -- 172.21.15.43:0/3835214049 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99cc09cb80 con 0x565372b80830 2024-04-21T08:38:46.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.996+0000 7f99cbfff640 1 -- 172.21.15.43:0/3835214049 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc0d0260 con 0x565372b80830 2024-04-21T08:38:46.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.997+0000 7f99eb7ac740 1 -- 172.21.15.43:0/3835214049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b80830 msgr2=0x565372b912f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.997+0000 7f99eb7ac740 1 --2- 172.21.15.43:0/3835214049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b80830 0x565372b912f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f99cc0cdca0 tx=0x7f99cc0c7590 comp rx=0 tx=0).stop 2024-04-21T08:38:46.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.997+0000 7f99eb7ac740 1 -- 172.21.15.43:0/3835214049 shutdown_connections 2024-04-21T08:38:46.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.997+0000 7f99eb7ac740 1 --2- 172.21.15.43:0/3835214049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565372b812d0 0x565372b89e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.997+0000 7f99eb7ac740 1 --2- 172.21.15.43:0/3835214049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b80830 0x565372b912f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.997+0000 7f99eb7ac740 1 --2- 172.21.15.43:0/3835214049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565372b7fe40 0x565372b88a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.997+0000 7f99eb7ac740 1 -- 172.21.15.43:0/3835214049 >> 172.21.15.43:0/3835214049 conn(0x565372b7bf40 msgr2=0x565372b75800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:46.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.997+0000 7f99eb7ac740 1 -- 172.21.15.43:0/3835214049 shutdown_connections 2024-04-21T08:38:46.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.997+0000 7f99eb7ac740 1 -- 172.21.15.43:0/3835214049 wait complete. 2024-04-21T08:38:46.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.997+0000 7f99eb7ac740 1 -- start start 2024-04-21T08:38:46.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.997+0000 7f99eb7ac740 1 Processor -- start 2024-04-21T08:38:46.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.998+0000 7f99eb7ac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565372b7fe40 0x565372b8cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.998+0000 7f99eb7ac740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b80830 0x565372b8d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.998+0000 7f99eb7ac740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565372b812d0 0x565372b902b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.998+0000 7f99eb7ac740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x565372b98160 con 0x565372b80830 2024-04-21T08:38:46.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.998+0000 7f99eb7ac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x565372b97a90 con 0x565372b7fe40 2024-04-21T08:38:46.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.998+0000 7f99eb7ac740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x565372b98830 con 0x565372b812d0 2024-04-21T08:38:46.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.998+0000 7f99eb7ac740 10 client.0 _ll_register_callbacks cb 0x7f99dc037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:46.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.998+0000 7f99e1d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b80830 0x565372b8d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.998+0000 7f99e1d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b80830 0x565372b8d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36304/0 (socket says 172.21.15.43:36304) 2024-04-21T08:38:46.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.998+0000 7f99e1d9b640 1 -- 172.21.15.43:0/3587054278 learned_addr learned my addr 172.21.15.43:0/3587054278 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:46.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.998+0000 7f99e259c640 1 --2- 172.21.15.43:0/3587054278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565372b7fe40 0x565372b8cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.998+0000 7f99e2d9d640 1 --2- 172.21.15.43:0/3587054278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565372b812d0 0x565372b902b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.998+0000 7f99e1d9b640 1 -- 172.21.15.43:0/3587054278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565372b812d0 msgr2=0x565372b902b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:46.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.998+0000 7f99e1d9b640 1 --2- 172.21.15.43:0/3587054278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565372b812d0 0x565372b902b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.998+0000 7f99e1d9b640 1 -- 172.21.15.43:0/3587054278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565372b7fe40 msgr2=0x565372b8cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:46.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.998+0000 7f99e1d9b640 1 --2- 172.21.15.43:0/3587054278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565372b7fe40 0x565372b8cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.998+0000 7f99e1d9b640 1 -- 172.21.15.43:0/3587054278 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565372b908a0 con 0x565372b80830 2024-04-21T08:38:46.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.998+0000 7f99e1d9b640 1 --2- 172.21.15.43:0/3587054278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b80830 0x565372b8d290 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f99cc0cdc70 tx=0x7f99cc09c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7626c1dbfca1881 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:46.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.998+0000 7f99c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:38:46.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.998+0000 7f99c9ffb640 1 -- 172.21.15.43:0/3587054278 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc0b4020 con 0x565372b80830 2024-04-21T08:38:46.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.998+0000 7f99c9ffb640 1 -- 172.21.15.43:0/3587054278 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f99cc09b8d0 con 0x565372b80830 2024-04-21T08:38:46.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.998+0000 7f99eb7ac740 1 -- 172.21.15.43:0/3587054278 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99cc09cb80 con 0x565372b80830 2024-04-21T08:38:46.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.998+0000 7f99c9ffb640 1 -- 172.21.15.43:0/3587054278 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f99cc01d040 con 0x565372b80830 2024-04-21T08:38:46.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.999+0000 7f99eb7ac740 10 client.15504 fetch_fsmap learned FSMap version 27 2024-04-21T08:38:46.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.999+0000 7f99eb7ac740 1 -- 172.21.15.43:0/3587054278 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x565372bfd000 con 0x565372b80830 2024-04-21T08:38:46.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.999+0000 7f99c9ffb640 1 -- 172.21.15.43:0/3587054278 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99cc09cb80 con 0x565372b80830 2024-04-21T08:38:46.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.999+0000 7f99c9ffb640 1 -- 172.21.15.43:0/3587054278 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f99cc0bb140 con 0x565372b80830 2024-04-21T08:38:46.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.999+0000 7f99eb7ac740 10 client.15504 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:38:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.999+0000 7f99eb7ac740 10 client.15504 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-21T08:38:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.999+0000 7f99eb7ac740 20 client.15504 populate_metadata read hostname 'smithi043' 2024-04-21T08:38:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.999+0000 7f99eb7ac740 1 --2- 172.21.15.43:0/3587054278 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x565372c016e0 0x565372c003f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.999+0000 7f99eb7ac740 4 client.15504 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:38:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.999+0000 7f99eb7ac740 1 -- 172.21.15.43:0/3587054278 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "status"}) -- 0x565372aec3a0 con 0x565372c016e0 2024-04-21T08:38:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.999+0000 7f99e259c640 1 --2- 172.21.15.43:0/3587054278 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x565372c016e0 0x565372c003f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.999+0000 7f99c9ffb640 10 client.15504 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:38:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:45.999+0000 7f99e259c640 1 --2- 172.21.15.43:0/3587054278 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x565372c016e0 0x565372c003f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1cec07c22433250b server_cookie=93a2fe1f9529f696 in_seq=0 out_seq=0 2024-04-21T08:38:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99c9ffb640 1 -- 172.21.15.43:0/3587054278 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x565372aec3a0 con 0x565372c016e0 2024-04-21T08:38:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99c9ffb640 10 client.15504 handle_command_reply: tid=0 2024-04-21T08:38:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99eb7ac740 1 client.15504 shutdown 2024-04-21T08:38:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99eb7ac740 1 -- 172.21.15.43:0/3587054278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b80830 msgr2=0x565372b8d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99eb7ac740 1 --2- 172.21.15.43:0/3587054278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b80830 0x565372b8d290 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f99cc0cdc70 tx=0x7f99cc09c480 comp rx=0 tx=0).stop 2024-04-21T08:38:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99eb7ac740 1 -- 172.21.15.43:0/3587054278 shutdown_connections 2024-04-21T08:38:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99eb7ac740 1 --2- 172.21.15.43:0/3587054278 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x565372c016e0 0x565372c003f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99eb7ac740 1 --2- 172.21.15.43:0/3587054278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565372b812d0 0x565372b902b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99c9ffb640 0 client.15504 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:38:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99eb7ac740 1 --2- 172.21.15.43:0/3587054278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565372b80830 0x565372b8d290 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99eb7ac740 1 --2- 172.21.15.43:0/3587054278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565372b7fe40 0x565372b8cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99eb7ac740 1 -- 172.21.15.43:0/3587054278 >> 172.21.15.43:0/3587054278 conn(0x565372b7bf40 msgr2=0x565372b88a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99eb7ac740 1 -- 172.21.15.43:0/3587054278 shutdown_connections 2024-04-21T08:38:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99eb7ac740 1 -- 172.21.15.43:0/3587054278 wait complete. 2024-04-21T08:38:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99eb7ac740 20 client.15504 trim_cache size 0 max 16384 2024-04-21T08:38:46.006 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:38:46.007 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "d57a7d98-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:38:46.007 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 1, 2024-04-21T08:38:46.007 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24491, 2024-04-21T08:38:46.007 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-21T08:38:46.007 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-21T08:38:46.007 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-21T08:38:46.007 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 117.94661565, 2024-04-21T08:38:46.007 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 27, 2024-04-21T08:38:46.007 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 94, 2024-04-21T08:38:46.007 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 90, 2024-04-21T08:38:46.007 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 158.708788618 2024-04-21T08:38:46.007 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:38:46.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99e8bb6640 1 -- 172.21.15.43:0/3773432786 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f99d00a3bb0 msgr2=0x7f99d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99e8bb6640 1 --2- 172.21.15.43:0/3773432786 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f99d00a3bb0 0x7f99d00a6070 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f99dc032380 tx=0x7f99dc0095c0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99e8bb6640 1 -- 172.21.15.43:0/3773432786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99e4103190 msgr2=0x7f99e4074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99e8bb6640 1 --2- 172.21.15.43:0/3773432786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99e4103190 0x7f99e4074340 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00c890 tx=0x7f99cc00cd50 comp rx=0 tx=0).stop 2024-04-21T08:38:46.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99e8bb6640 1 -- 172.21.15.43:0/3773432786 shutdown_connections 2024-04-21T08:38:46.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99e8bb6640 1 --2- 172.21.15.43:0/3773432786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e4103b50 0x7f99e4074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99e8bb6640 1 --2- 172.21.15.43:0/3773432786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99e4103190 0x7f99e4074340 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99e8bb6640 1 --2- 172.21.15.43:0/3773432786 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f99d00a3bb0 0x7f99d00a6070 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99e8bb6640 1 --2- 172.21.15.43:0/3773432786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99e4101fa0 0x7f99e4075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.000+0000 7f99e8bb6640 1 -- 172.21.15.43:0/3773432786 >> 172.21.15.43:0/3773432786 conn(0x7f99e40fd530 msgr2=0x7f99e4106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:46.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.001+0000 7f99e8bb6640 1 -- 172.21.15.43:0/3773432786 shutdown_connections 2024-04-21T08:38:46.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.001+0000 7f99e8bb6640 1 -- 172.21.15.43:0/3773432786 wait complete. 2024-04-21T08:38:46.011 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-21T08:38:46.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.116+0000 7fa5f40c8640 1 -- 172.21.15.43:0/673486759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5ec1031a0 msgr2=0x7fa5ec103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.116+0000 7fa5f40c8640 1 --2- 172.21.15.43:0/673486759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5ec1031a0 0x7fa5ec103620 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009900 tx=0x7fa5e002bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:46.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.116+0000 7fa5f40c8640 1 -- 172.21.15.43:0/673486759 shutdown_connections 2024-04-21T08:38:46.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.116+0000 7fa5f40c8640 1 --2- 172.21.15.43:0/673486759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5ec103b60 0x7fa5ec10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.116+0000 7fa5f40c8640 1 --2- 172.21.15.43:0/673486759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5ec1031a0 0x7fa5ec103620 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.116+0000 7fa5f40c8640 1 --2- 172.21.15.43:0/673486759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5ec101fb0 0x7fa5ec1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.116+0000 7fa5f40c8640 1 -- 172.21.15.43:0/673486759 >> 172.21.15.43:0/673486759 conn(0x7fa5ec0fd520 msgr2=0x7fa5ec0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:46.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.116+0000 7fa5f40c8640 1 -- 172.21.15.43:0/673486759 shutdown_connections 2024-04-21T08:38:46.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.116+0000 7fa5f40c8640 1 -- 172.21.15.43:0/673486759 wait complete. 2024-04-21T08:38:46.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.117+0000 7fa5f40c8640 1 Processor -- start 2024-04-21T08:38:46.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.117+0000 7fa5f40c8640 1 -- start start 2024-04-21T08:38:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.117+0000 7fa5f40c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5ec101fb0 0x7fa5ec198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.117+0000 7fa5f40c8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5ec1031a0 0x7fa5ec198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.117+0000 7fa5f40c8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5ec103b60 0x7fa5ec19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.117+0000 7fa5f40c8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec10daf0 con 0x7fa5ec103b60 2024-04-21T08:38:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.117+0000 7fa5f40c8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec10d970 con 0x7fa5ec101fb0 2024-04-21T08:38:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.117+0000 7fa5f40c8640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec10dc70 con 0x7fa5ec1031a0 2024-04-21T08:38:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.117+0000 7fa5f2665640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5ec103b60 0x7fa5ec19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.117+0000 7fa5f2665640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5ec103b60 0x7fa5ec19f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36326/0 (socket says 172.21.15.43:36326) 2024-04-21T08:38:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.117+0000 7fa5f2665640 1 -- 172.21.15.43:0/4210294006 learned_addr learned my addr 172.21.15.43:0/4210294006 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.117+0000 7fa5f1e64640 1 --2- 172.21.15.43:0/4210294006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5ec101fb0 0x7fa5ec198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.117+0000 7fa5f1663640 1 --2- 172.21.15.43:0/4210294006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5ec1031a0 0x7fa5ec198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.117+0000 7fa5f2665640 1 -- 172.21.15.43:0/4210294006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5ec1031a0 msgr2=0x7fa5ec198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.117+0000 7fa5f2665640 1 --2- 172.21.15.43:0/4210294006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5ec1031a0 0x7fa5ec198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.117+0000 7fa5f2665640 1 -- 172.21.15.43:0/4210294006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5ec101fb0 msgr2=0x7fa5ec198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.117+0000 7fa5f2665640 1 --2- 172.21.15.43:0/4210294006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5ec101fb0 0x7fa5ec198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.117+0000 7fa5f2665640 1 -- 172.21.15.43:0/4210294006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ec19fe20 con 0x7fa5ec103b60 2024-04-21T08:38:46.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.117+0000 7fa5f1e64640 1 --2- 172.21.15.43:0/4210294006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5ec101fb0 0x7fa5ec198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:46.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.118+0000 7fa5f2665640 1 --2- 172.21.15.43:0/4210294006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5ec103b60 0x7fa5ec19f7b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400cad0 tx=0x7fa5e400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:46.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.118+0000 7fa5deffd640 1 -- 172.21.15.43:0/4210294006 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e400ae20 con 0x7fa5ec103b60 2024-04-21T08:38:46.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.118+0000 7fa5deffd640 1 -- 172.21.15.43:0/4210294006 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5e4005020 con 0x7fa5ec103b60 2024-04-21T08:38:46.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.118+0000 7fa5f40c8640 1 -- 172.21.15.43:0/4210294006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5ec1a0110 con 0x7fa5ec103b60 2024-04-21T08:38:46.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.118+0000 7fa5deffd640 1 -- 172.21.15.43:0/4210294006 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e400f6b0 con 0x7fa5ec103b60 2024-04-21T08:38:46.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.118+0000 7fa5f40c8640 1 -- 172.21.15.43:0/4210294006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5ec1a0660 con 0x7fa5ec103b60 2024-04-21T08:38:46.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.120+0000 7fa5deffd640 1 -- 172.21.15.43:0/4210294006 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa5e40051d0 con 0x7fa5ec103b60 2024-04-21T08:38:46.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.120+0000 7fa5f6d96740 1 Processor -- start 2024-04-21T08:38:46.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.121+0000 7fa5f6d96740 1 -- start start 2024-04-21T08:38:46.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.121+0000 7fa5f6d96740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cdad4e40 0x5589cdad5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.121+0000 7fa5f6d96740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad5830 0x5589cdad5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.121+0000 7fa5f6d96740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589cdad62d0 0x5589cdadceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.121+0000 7fa5f6d96740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5589cda39b10 con 0x5589cdad5830 2024-04-21T08:38:46.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.121+0000 7fa5f6d96740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5589cd9d7240 con 0x5589cdad4e40 2024-04-21T08:38:46.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.121+0000 7fa5f6d96740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5589cda35ad0 con 0x5589cdad62d0 2024-04-21T08:38:46.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.121+0000 7fa5f1e64640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad5830 0x5589cdad5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.121+0000 7fa5f1e64640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad5830 0x5589cdad5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36340/0 (socket says 172.21.15.43:36340) 2024-04-21T08:38:46.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.121+0000 7fa5f1e64640 1 -- 172.21.15.43:0/3158733782 learned_addr learned my addr 172.21.15.43:0/3158733782 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:46.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.121+0000 7fa5f1663640 1 --2- 172.21.15.43:0/3158733782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cdad4e40 0x5589cdad5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.121+0000 7fa5deffd640 1 --2- 172.21.15.43:0/4210294006 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa5d80a3bb0 0x7fa5d80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.121+0000 7fa5f1663640 1 --2- 172.21.15.43:0/3158733782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589cdad62d0 0x5589cdadceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.121+0000 7fa5f1e64640 1 -- 172.21.15.43:0/3158733782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589cdad62d0 msgr2=0x5589cdadceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:46.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.121+0000 7fa5f1e64640 1 --2- 172.21.15.43:0/3158733782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589cdad62d0 0x5589cdadceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.121+0000 7fa5f2665640 1 --2- 172.21.15.43:0/4210294006 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa5d80a3bb0 0x7fa5d80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.121+0000 7fa5f1e64640 1 -- 172.21.15.43:0/3158733782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cdad4e40 msgr2=0x5589cdad5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:46.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.121+0000 7fa5f1e64640 1 --2- 172.21.15.43:0/3158733782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cdad4e40 0x5589cdad5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.121+0000 7fa5f1e64640 1 -- 172.21.15.43:0/3158733782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589cd74c2d0 con 0x5589cdad5830 2024-04-21T08:38:46.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.121+0000 7fa5f1663640 1 --2- 172.21.15.43:0/3158733782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589cdad62d0 0x5589cdadceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:46.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.121+0000 7fa5deffd640 1 -- 172.21.15.43:0/4210294006 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fa5e40d08b0 con 0x7fa5ec103b60 2024-04-21T08:38:46.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.122+0000 7fa5f1e64640 1 --2- 172.21.15.43:0/3158733782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad5830 0x5589cdad5cb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8009cd0 tx=0x7fa5e80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad85fe0939bcc8c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:46.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.122+0000 7fa5d7fff640 1 -- 172.21.15.43:0/3158733782 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8012450 con 0x5589cdad5830 2024-04-21T08:38:46.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.122+0000 7fa5d7fff640 1 -- 172.21.15.43:0/3158733782 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5e8012aa0 con 0x5589cdad5830 2024-04-21T08:38:46.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.122+0000 7fa5d7fff640 1 -- 172.21.15.43:0/3158733782 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e800fc00 con 0x5589cdad5830 2024-04-21T08:38:46.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.122+0000 7fa5f6d96740 1 -- 172.21.15.43:0/3158733782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad5830 msgr2=0x5589cdad5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.122+0000 7fa5f6d96740 1 --2- 172.21.15.43:0/3158733782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad5830 0x5589cdad5cb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8009cd0 tx=0x7fa5e80117b0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.122+0000 7fa5f2665640 1 --2- 172.21.15.43:0/4210294006 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa5d80a3bb0 0x7fa5d80a6070 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa5e409c730 tx=0x7fa5e4091c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:46.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.123+0000 7fa5f6d96740 1 -- 172.21.15.43:0/3158733782 shutdown_connections 2024-04-21T08:38:46.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.123+0000 7fa5f6d96740 1 --2- 172.21.15.43:0/3158733782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589cdad62d0 0x5589cdadceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.123+0000 7fa5f6d96740 1 --2- 172.21.15.43:0/3158733782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad5830 0x5589cdad5cb0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.123+0000 7fa5f6d96740 1 --2- 172.21.15.43:0/3158733782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cdad4e40 0x5589cdad5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.123+0000 7fa5f6d96740 1 -- 172.21.15.43:0/3158733782 >> 172.21.15.43:0/3158733782 conn(0x5589cdad0f40 msgr2=0x5589cdacf490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:46.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.123+0000 7fa5f6d96740 1 -- 172.21.15.43:0/3158733782 shutdown_connections 2024-04-21T08:38:46.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.123+0000 7fa5f6d96740 1 -- 172.21.15.43:0/3158733782 wait complete. 2024-04-21T08:38:46.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.123+0000 7fa5f6d96740 1 -- start start 2024-04-21T08:38:46.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.123+0000 7fa5f6d96740 1 Processor -- start 2024-04-21T08:38:46.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.123+0000 7fa5f6d96740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589cdad4e40 0x5589cdadee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.123+0000 7fa5f6d96740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cdad5830 0x5589cdadf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.123+0000 7fa5f6d96740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad62d0 0x5589cdae6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.123+0000 7fa5f6d96740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5589cdadd4b0 con 0x5589cdad62d0 2024-04-21T08:38:46.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.123+0000 7fa5f6d96740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5589cd9c2190 con 0x5589cdad5830 2024-04-21T08:38:46.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.123+0000 7fa5f6d96740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5589cdadd630 con 0x5589cdad4e40 2024-04-21T08:38:46.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.123+0000 7fa5f6d96740 10 client.0 _ll_register_callbacks cb 0x5589cd9bd8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:46.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.124+0000 7fa5f1663640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad62d0 0x5589cdae6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.124+0000 7fa5f1663640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad62d0 0x5589cdae6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36342/0 (socket says 172.21.15.43:36342) 2024-04-21T08:38:46.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.124+0000 7fa5f1663640 1 -- 172.21.15.43:0/1642368241 learned_addr learned my addr 172.21.15.43:0/1642368241 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:46.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.124+0000 7fa5f1663640 1 --2- 172.21.15.43:0/1642368241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589cdad4e40 0x5589cdadee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.124+0000 7fa5f1663640 1 -- 172.21.15.43:0/1642368241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589cdad4e40 msgr2=0x5589cdadee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:46.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.124+0000 7fa5f1663640 1 --2- 172.21.15.43:0/1642368241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589cdad4e40 0x5589cdadee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.124+0000 7fa5f1663640 1 -- 172.21.15.43:0/1642368241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cdad5830 msgr2=0x5589cdadf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:46.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.124+0000 7fa5f1e64640 1 --2- 172.21.15.43:0/1642368241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cdad5830 0x5589cdadf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.124+0000 7fa5f1663640 1 --2- 172.21.15.43:0/1642368241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cdad5830 0x5589cdadf3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.124+0000 7fa5f1663640 1 -- 172.21.15.43:0/1642368241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589cd8166b0 con 0x5589cdad62d0 2024-04-21T08:38:46.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.124+0000 7fa5f1663640 1 --2- 172.21.15.43:0/1642368241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad62d0 0x5589cdae6370 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0004650 tx=0x7fa5e002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713f680a2e92ed4c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:46.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.124+0000 7fa5d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:38:46.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.124+0000 7fa5d57fa640 1 -- 172.21.15.43:0/1642368241 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0005450 con 0x5589cdad62d0 2024-04-21T08:38:46.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.124+0000 7fa5f6d96740 1 -- 172.21.15.43:0/1642368241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5e8012aa0 con 0x5589cdad62d0 2024-04-21T08:38:46.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.124+0000 7fa5d57fa640 1 -- 172.21.15.43:0/1642368241 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fa5e00046c0 con 0x5589cdad62d0 2024-04-21T08:38:46.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.124+0000 7fa5d57fa640 1 -- 172.21.15.43:0/1642368241 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa5e0055050 con 0x5589cdad62d0 2024-04-21T08:38:46.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.124+0000 7fa5f6d96740 10 client.15516 fetch_fsmap learned FSMap version 27 2024-04-21T08:38:46.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.124+0000 7fa5f6d96740 1 -- 172.21.15.43:0/1642368241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5589cdae9750 con 0x5589cdad62d0 2024-04-21T08:38:46.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.125+0000 7fa5d57fa640 1 -- 172.21.15.43:0/1642368241 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5e8012aa0 con 0x5589cdad62d0 2024-04-21T08:38:46.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.125+0000 7fa5d57fa640 1 -- 172.21.15.43:0/1642368241 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa5e0040040 con 0x5589cdad62d0 2024-04-21T08:38:46.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.125+0000 7fa5f6d96740 10 client.15516 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:38:46.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.125+0000 7fa5f6d96740 10 client.15516 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T08:38:46.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.125+0000 7fa5f6d96740 20 client.15516 populate_metadata read hostname 'smithi043' 2024-04-21T08:38:46.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.125+0000 7fa5f6d96740 1 --2- 172.21.15.43:0/1642368241 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5589cdaebda0 0x5589cdaed210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.125+0000 7fa5f6d96740 4 client.15516 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:38:46.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.125+0000 7fa5f6d96740 1 -- 172.21.15.43:0/1642368241 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589cda47530 con 0x5589cdaebda0 2024-04-21T08:38:46.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.125+0000 7fa5f1e64640 1 --2- 172.21.15.43:0/1642368241 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5589cdaebda0 0x5589cdaed210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.125+0000 7fa5f1e64640 1 --2- 172.21.15.43:0/1642368241 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5589cdaebda0 0x5589cdaed210 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9795e29e044cc2b4 server_cookie=c0c0af0cd1495551 in_seq=0 out_seq=0 2024-04-21T08:38:46.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.125+0000 7fa5d57fa640 10 client.15516 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:38:46.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.126+0000 7fa5d57fa640 1 -- 172.21.15.43:0/1642368241 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5589cda47530 con 0x5589cdaebda0 2024-04-21T08:38:46.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.126+0000 7fa5d57fa640 10 client.15516 handle_command_reply: tid=0 2024-04-21T08:38:46.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.126+0000 7fa5f6d96740 1 client.15516 shutdown 2024-04-21T08:38:46.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.126+0000 7fa5f6d96740 1 -- 172.21.15.43:0/1642368241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad62d0 msgr2=0x5589cdae6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.126+0000 7fa5f6d96740 1 --2- 172.21.15.43:0/1642368241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad62d0 0x5589cdae6370 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0004650 tx=0x7fa5e002bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:46.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.126+0000 7fa5f6d96740 1 -- 172.21.15.43:0/1642368241 shutdown_connections 2024-04-21T08:38:46.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.126+0000 7fa5f6d96740 1 --2- 172.21.15.43:0/1642368241 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5589cdaebda0 0x5589cdaed210 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.126+0000 7fa5f6d96740 1 --2- 172.21.15.43:0/1642368241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad62d0 0x5589cdae6370 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.126+0000 7fa5f6d96740 1 --2- 172.21.15.43:0/1642368241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cdad5830 0x5589cdadf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.126+0000 7fa5f6d96740 1 --2- 172.21.15.43:0/1642368241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589cdad4e40 0x5589cdadee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.126+0000 7fa5d57fa640 0 client.15516 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:38:46.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.126+0000 7fa5f6d96740 1 -- 172.21.15.43:0/1642368241 >> 172.21.15.43:0/1642368241 conn(0x5589cdad0f40 msgr2=0x5589cdacea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:46.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.126+0000 7fa5f6d96740 1 -- 172.21.15.43:0/1642368241 shutdown_connections 2024-04-21T08:38:46.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.126+0000 7fa5f6d96740 1 -- 172.21.15.43:0/1642368241 wait complete. 2024-04-21T08:38:46.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.127+0000 7fa5f6d96740 20 client.15516 trim_cache size 0 max 16384 2024-04-21T08:38:46.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.145+0000 7fa5f6d96740 1 Processor -- start 2024-04-21T08:38:46.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.145+0000 7fa5f6d96740 1 -- start start 2024-04-21T08:38:46.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.145+0000 7fa5f6d96740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad4e40 0x5589cdadce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.145+0000 7fa5f6d96740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cdad5830 0x5589cdadd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.145+0000 7fa5f6d96740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589cdad62d0 0x5589cdad9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.145+0000 7fa5f6d96740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5589cdaeb550 con 0x5589cdad4e40 2024-04-21T08:38:46.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.145+0000 7fa5f6d96740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5589cdaebc20 con 0x5589cdad5830 2024-04-21T08:38:46.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.145+0000 7fa5f6d96740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5589cdaec2f0 con 0x5589cdad62d0 2024-04-21T08:38:46.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.145+0000 7fa5f1663640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad4e40 0x5589cdadce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.145+0000 7fa5f1663640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad4e40 0x5589cdadce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36344/0 (socket says 172.21.15.43:36344) 2024-04-21T08:38:46.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.145+0000 7fa5f1663640 1 -- 172.21.15.43:0/241189118 learned_addr learned my addr 172.21.15.43:0/241189118 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:46.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.146+0000 7fa5f1663640 1 -- 172.21.15.43:0/241189118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589cdad62d0 msgr2=0x5589cdad9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:46.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.146+0000 7fa5f1e64640 1 --2- 172.21.15.43:0/241189118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cdad5830 0x5589cdadd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.146+0000 7fa5f1663640 1 --2- 172.21.15.43:0/241189118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589cdad62d0 0x5589cdad9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.146+0000 7fa5f1663640 1 -- 172.21.15.43:0/241189118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cdad5830 msgr2=0x5589cdadd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:46.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.146+0000 7fa5f1663640 1 --2- 172.21.15.43:0/241189118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cdad5830 0x5589cdadd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.146+0000 7fa5f1663640 1 -- 172.21.15.43:0/241189118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589cdadfe50 con 0x5589cdad4e40 2024-04-21T08:38:46.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.146+0000 7fa5f1663640 1 --2- 172.21.15.43:0/241189118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad4e40 0x5589cdadce60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa5e002cab0 tx=0x7fa5e0009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3ade0da6a6c254 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.146+0000 7fa5d77fe640 1 -- 172.21.15.43:0/241189118 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e002cc80 con 0x5589cdad4e40 2024-04-21T08:38:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.146+0000 7fa5d77fe640 1 -- 172.21.15.43:0/241189118 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5e00035f0 con 0x5589cdad4e40 2024-04-21T08:38:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.146+0000 7fa5d77fe640 1 -- 172.21.15.43:0/241189118 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0004b80 con 0x5589cdad4e40 2024-04-21T08:38:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.147+0000 7fa5f6d96740 1 -- 172.21.15.43:0/241189118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad4e40 msgr2=0x5589cdadce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.147+0000 7fa5f6d96740 1 --2- 172.21.15.43:0/241189118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad4e40 0x5589cdadce60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa5e002cab0 tx=0x7fa5e0009830 comp rx=0 tx=0).stop 2024-04-21T08:38:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.147+0000 7fa5f6d96740 1 -- 172.21.15.43:0/241189118 shutdown_connections 2024-04-21T08:38:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.147+0000 7fa5f6d96740 1 --2- 172.21.15.43:0/241189118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589cdad62d0 0x5589cdad9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.147+0000 7fa5f6d96740 1 --2- 172.21.15.43:0/241189118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cdad5830 0x5589cdadd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.147+0000 7fa5f6d96740 1 --2- 172.21.15.43:0/241189118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad4e40 0x5589cdadce60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.147+0000 7fa5f6d96740 1 -- 172.21.15.43:0/241189118 >> 172.21.15.43:0/241189118 conn(0x5589cdad0f40 msgr2=0x5589cdadee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.147+0000 7fa5f6d96740 1 -- 172.21.15.43:0/241189118 shutdown_connections 2024-04-21T08:38:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.147+0000 7fa5f6d96740 1 -- 172.21.15.43:0/241189118 wait complete. 2024-04-21T08:38:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.147+0000 7fa5f6d96740 1 -- start start 2024-04-21T08:38:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.147+0000 7fa5f6d96740 1 Processor -- start 2024-04-21T08:38:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.147+0000 7fa5f6d96740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cdad4e40 0x5589cdad70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.147+0000 7fa5f6d96740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad5830 0x5589cdad7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.147+0000 7fa5f6d96740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589cdad62d0 0x5589cdad80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.147+0000 7fa5f6d96740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5589cdaed090 con 0x5589cdad5830 2024-04-21T08:38:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.147+0000 7fa5f6d96740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5589cdaec9c0 con 0x5589cdad4e40 2024-04-21T08:38:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.147+0000 7fa5f6d96740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5589cdaed760 con 0x5589cdad62d0 2024-04-21T08:38:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.148+0000 7fa5f6d96740 10 client.0 _ll_register_callbacks cb 0x7fa5e0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.148+0000 7fa5f1e64640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad5830 0x5589cdad7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.148+0000 7fa5f1e64640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad5830 0x5589cdad7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36354/0 (socket says 172.21.15.43:36354) 2024-04-21T08:38:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.148+0000 7fa5f1e64640 1 -- 172.21.15.43:0/893091180 learned_addr learned my addr 172.21.15.43:0/893091180 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.148+0000 7fa5f1663640 1 --2- 172.21.15.43:0/893091180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589cdad62d0 0x5589cdad80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.148+0000 7fa5f1663640 1 --2- 172.21.15.43:0/893091180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cdad4e40 0x5589cdad70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.148+0000 7fa5f1e64640 1 -- 172.21.15.43:0/893091180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589cdad62d0 msgr2=0x5589cdad80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.148+0000 7fa5f1e64640 1 --2- 172.21.15.43:0/893091180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589cdad62d0 0x5589cdad80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.148+0000 7fa5f1e64640 1 -- 172.21.15.43:0/893091180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cdad4e40 msgr2=0x5589cdad70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.148+0000 7fa5f1e64640 1 --2- 172.21.15.43:0/893091180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cdad4e40 0x5589cdad70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.148+0000 7fa5f1e64640 1 -- 172.21.15.43:0/893091180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589cdad86d0 con 0x5589cdad5830 2024-04-21T08:38:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.148+0000 7fa5f1663640 1 --2- 172.21.15.43:0/893091180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589cdad62d0 0x5589cdad80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.148+0000 7fa5f1e64640 1 --2- 172.21.15.43:0/893091180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad5830 0x5589cdad7610 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8009cd0 tx=0x7fa5e8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0dfbfeb7fe6322 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.148+0000 7fa5d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:38:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.148+0000 7fa5d57fa640 1 -- 172.21.15.43:0/893091180 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e800f440 con 0x5589cdad5830 2024-04-21T08:38:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.148+0000 7fa5d57fa640 1 -- 172.21.15.43:0/893091180 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fa5e800f910 con 0x5589cdad5830 2024-04-21T08:38:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.148+0000 7fa5f6d96740 1 -- 172.21.15.43:0/893091180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5e00035f0 con 0x5589cdad5830 2024-04-21T08:38:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.148+0000 7fa5d57fa640 1 -- 172.21.15.43:0/893091180 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa5e8037050 con 0x5589cdad5830 2024-04-21T08:38:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.148+0000 7fa5f6d96740 10 client.15522 fetch_fsmap learned FSMap version 27 2024-04-21T08:38:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.148+0000 7fa5f6d96740 1 -- 172.21.15.43:0/893091180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5589cdb56280 con 0x5589cdad5830 2024-04-21T08:38:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.149+0000 7fa5d57fa640 1 -- 172.21.15.43:0/893091180 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5e00035f0 con 0x5589cdad5830 2024-04-21T08:38:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.149+0000 7fa5d57fa640 1 -- 172.21.15.43:0/893091180 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa5e80050d0 con 0x5589cdad5830 2024-04-21T08:38:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.149+0000 7fa5f6d96740 10 client.15522 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:38:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.149+0000 7fa5f6d96740 10 client.15522 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T08:38:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.149+0000 7fa5f6d96740 20 client.15522 populate_metadata read hostname 'smithi043' 2024-04-21T08:38:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.149+0000 7fa5f6d96740 1 --2- 172.21.15.43:0/893091180 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5589cdb5aca0 0x5589cdb599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.149+0000 7fa5f6d96740 4 client.15522 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:38:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.149+0000 7fa5f6d96740 1 -- 172.21.15.43:0/893091180 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "status"}) -- 0x5589cda413a0 con 0x5589cdb5aca0 2024-04-21T08:38:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.149+0000 7fa5f1663640 1 --2- 172.21.15.43:0/893091180 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5589cdb5aca0 0x5589cdb599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.150+0000 7fa5f1663640 1 --2- 172.21.15.43:0/893091180 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5589cdb5aca0 0x5589cdb599b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb7dc8d83025f7a8 server_cookie=e01bc2e0fcc2ee68 in_seq=0 out_seq=0 2024-04-21T08:38:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.150+0000 7fa5d57fa640 10 client.15522 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:38:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.150+0000 7fa5d57fa640 1 -- 172.21.15.43:0/893091180 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5589cda413a0 con 0x5589cdb5aca0 2024-04-21T08:38:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.150+0000 7fa5d57fa640 10 client.15522 handle_command_reply: tid=0 2024-04-21T08:38:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.150+0000 7fa5f6d96740 1 client.15522 shutdown 2024-04-21T08:38:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.150+0000 7fa5f6d96740 1 -- 172.21.15.43:0/893091180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad5830 msgr2=0x5589cdad7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.150+0000 7fa5f6d96740 1 --2- 172.21.15.43:0/893091180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad5830 0x5589cdad7610 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8009cd0 tx=0x7fa5e8009920 comp rx=0 tx=0).stop 2024-04-21T08:38:46.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.150+0000 7fa5f6d96740 1 -- 172.21.15.43:0/893091180 shutdown_connections 2024-04-21T08:38:46.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.150+0000 7fa5f6d96740 1 --2- 172.21.15.43:0/893091180 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5589cdb5aca0 0x5589cdb599b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.150+0000 7fa5f6d96740 1 --2- 172.21.15.43:0/893091180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589cdad62d0 0x5589cdad80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.150+0000 7fa5d57fa640 0 client.15522 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:38:46.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.150+0000 7fa5f6d96740 1 --2- 172.21.15.43:0/893091180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589cdad5830 0x5589cdad7610 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.150+0000 7fa5f6d96740 1 --2- 172.21.15.43:0/893091180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cdad4e40 0x5589cdad70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.150+0000 7fa5f6d96740 1 -- 172.21.15.43:0/893091180 >> 172.21.15.43:0/893091180 conn(0x5589cdad0f40 msgr2=0x5589cdacf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:46.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.150+0000 7fa5f6d96740 1 -- 172.21.15.43:0/893091180 shutdown_connections 2024-04-21T08:38:46.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.150+0000 7fa5f6d96740 1 -- 172.21.15.43:0/893091180 wait complete. 2024-04-21T08:38:46.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.150+0000 7fa5f6d96740 20 client.15522 trim_cache size 0 max 16384 2024-04-21T08:38:46.156 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:38:46.157 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "d57a7d98-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:38:46.157 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-04-21T08:38:46.157 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24527, 2024-04-21T08:38:46.157 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-21T08:38:46.157 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-21T08:38:46.157 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-21T08:38:46.157 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 141.695080762, 2024-04-21T08:38:46.157 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 27, 2024-04-21T08:38:46.157 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 94, 2024-04-21T08:38:46.157 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 90, 2024-04-21T08:38:46.157 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 146.97013023100001 2024-04-21T08:38:46.157 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:38:46.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.151+0000 7fa5f40c8640 1 -- 172.21.15.43:0/4210294006 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa5d80a3bb0 msgr2=0x7fa5d80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.151+0000 7fa5f40c8640 1 --2- 172.21.15.43:0/4210294006 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa5d80a3bb0 0x7fa5d80a6070 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa5e409c730 tx=0x7fa5e4091c80 comp rx=0 tx=0).stop 2024-04-21T08:38:46.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.151+0000 7fa5f40c8640 1 -- 172.21.15.43:0/4210294006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5ec103b60 msgr2=0x7fa5ec19f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.151+0000 7fa5f40c8640 1 --2- 172.21.15.43:0/4210294006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5ec103b60 0x7fa5ec19f7b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400cad0 tx=0x7fa5e400cf90 comp rx=0 tx=0).stop 2024-04-21T08:38:46.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.151+0000 7fa5f40c8640 1 -- 172.21.15.43:0/4210294006 shutdown_connections 2024-04-21T08:38:46.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.151+0000 7fa5f40c8640 1 --2- 172.21.15.43:0/4210294006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5ec103b60 0x7fa5ec19f7b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.151+0000 7fa5f40c8640 1 --2- 172.21.15.43:0/4210294006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5ec1031a0 0x7fa5ec198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.151+0000 7fa5f40c8640 1 --2- 172.21.15.43:0/4210294006 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa5d80a3bb0 0x7fa5d80a6070 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.151+0000 7fa5f40c8640 1 --2- 172.21.15.43:0/4210294006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5ec101fb0 0x7fa5ec198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.151+0000 7fa5f40c8640 1 -- 172.21.15.43:0/4210294006 >> 172.21.15.43:0/4210294006 conn(0x7fa5ec0fd520 msgr2=0x7fa5ec106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:46.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.151+0000 7fa5f40c8640 1 -- 172.21.15.43:0/4210294006 shutdown_connections 2024-04-21T08:38:46.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.151+0000 7fa5f40c8640 1 -- 172.21.15.43:0/4210294006 wait complete. 2024-04-21T08:38:46.161 INFO:teuthology.run_tasks:Running task sequential... 2024-04-21T08:38:46.170 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-21T08:38:46.170 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-21T08:38:46.171 INFO:teuthology.task.print:Enabling mgr modules 2024-04-21T08:38:46.171 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-21T08:38:46.171 INFO:teuthology.task.exec:Executing custom commands... 2024-04-21T08:38:46.172 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-04-21T08:38:46.172 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-21T08:38:46.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.275+0000 7fa9d6ffd640 1 -- 172.21.15.43:0/402983137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9d0100030 msgr2=0x7fa9d01004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.275+0000 7fa9d6ffd640 1 --2- 172.21.15.43:0/402983137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9d0100030 0x7fa9d01004b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8009900 tx=0x7fa9b802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.275+0000 7fa9d6ffd640 1 -- 172.21.15.43:0/402983137 shutdown_connections 2024-04-21T08:38:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.275+0000 7fa9d6ffd640 1 --2- 172.21.15.43:0/402983137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9d01009f0 0x7fa9d010a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.275+0000 7fa9d6ffd640 1 --2- 172.21.15.43:0/402983137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9d0100030 0x7fa9d01004b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.275+0000 7fa9d6ffd640 1 --2- 172.21.15.43:0/402983137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d01035c0 0x7fa9d01039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.275+0000 7fa9d6ffd640 1 -- 172.21.15.43:0/402983137 >> 172.21.15.43:0/402983137 conn(0x7fa9d00fbbd0 msgr2=0x7fa9d00fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.275+0000 7fa9d6ffd640 1 -- 172.21.15.43:0/402983137 shutdown_connections 2024-04-21T08:38:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.275+0000 7fa9d6ffd640 1 -- 172.21.15.43:0/402983137 wait complete. 2024-04-21T08:38:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.276+0000 7fa9d6ffd640 1 Processor -- start 2024-04-21T08:38:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.276+0000 7fa9d6ffd640 1 -- start start 2024-04-21T08:38:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.276+0000 7fa9d6ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9d0100030 0x7fa9d0198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.276+0000 7fa9d6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9d01009f0 0x7fa9d0198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.276+0000 7fa9d6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d01035c0 0x7fa9d019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.276+0000 7fa9d6ffd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d010db50 con 0x7fa9d01009f0 2024-04-21T08:38:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.276+0000 7fa9d6ffd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d010d9d0 con 0x7fa9d01035c0 2024-04-21T08:38:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.276+0000 7fa9d6ffd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d010dcd0 con 0x7fa9d0100030 2024-04-21T08:38:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.276+0000 7fa9cffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9d01009f0 0x7fa9d0198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.276+0000 7fa9cffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9d01009f0 0x7fa9d0198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36372/0 (socket says 172.21.15.43:36372) 2024-04-21T08:38:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.276+0000 7fa9cffff640 1 -- 172.21.15.43:0/3303728118 learned_addr learned my addr 172.21.15.43:0/3303728118 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.276+0000 7fa9d4d99640 1 --2- 172.21.15.43:0/3303728118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9d0100030 0x7fa9d0198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.276+0000 7fa9d559a640 1 --2- 172.21.15.43:0/3303728118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d01035c0 0x7fa9d019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.276+0000 7fa9cffff640 1 -- 172.21.15.43:0/3303728118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9d0100030 msgr2=0x7fa9d0198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.276+0000 7fa9cffff640 1 --2- 172.21.15.43:0/3303728118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9d0100030 0x7fa9d0198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.276+0000 7fa9cffff640 1 -- 172.21.15.43:0/3303728118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d01035c0 msgr2=0x7fa9d019f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.276+0000 7fa9cffff640 1 --2- 172.21.15.43:0/3303728118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d01035c0 0x7fa9d019f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.276+0000 7fa9cffff640 1 -- 172.21.15.43:0/3303728118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d019fd50 con 0x7fa9d01009f0 2024-04-21T08:38:46.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.277+0000 7fa9cffff640 1 --2- 172.21.15.43:0/3303728118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9d01009f0 0x7fa9d0198670 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8005ab0 tx=0x7fa9b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:46.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.277+0000 7fa9cdffb640 1 -- 172.21.15.43:0/3303728118 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8043020 con 0x7fa9d01009f0 2024-04-21T08:38:46.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.277+0000 7fa9cdffb640 1 -- 172.21.15.43:0/3303728118 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b8004a80 con 0x7fa9d01009f0 2024-04-21T08:38:46.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.277+0000 7fa9d6ffd640 1 -- 172.21.15.43:0/3303728118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d019ffe0 con 0x7fa9d01009f0 2024-04-21T08:38:46.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.277+0000 7fa9cdffb640 1 -- 172.21.15.43:0/3303728118 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8003cd0 con 0x7fa9d01009f0 2024-04-21T08:38:46.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.277+0000 7fa9d6ffd640 1 -- 172.21.15.43:0/3303728118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d01a04a0 con 0x7fa9d01009f0 2024-04-21T08:38:46.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.278+0000 7fa9d6ffd640 1 -- 172.21.15.43:0/3303728118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d0109480 con 0x7fa9d01009f0 2024-04-21T08:38:46.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.279+0000 7fa9cdffb640 1 -- 172.21.15.43:0/3303728118 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa9b8046020 con 0x7fa9d01009f0 2024-04-21T08:38:46.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.280+0000 7fa9cdffb640 1 --2- 172.21.15.43:0/3303728118 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa9bc0a3c50 0x7fa9bc0a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.280+0000 7fa9d4d99640 1 --2- 172.21.15.43:0/3303728118 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa9bc0a3c50 0x7fa9bc0a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.280+0000 7fa9cdffb640 1 -- 172.21.15.43:0/3303728118 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fa9b80f2af0 con 0x7fa9d01009f0 2024-04-21T08:38:46.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.280+0000 7fa9d4d99640 1 --2- 172.21.15.43:0/3303728118 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa9bc0a3c50 0x7fa9bc0a6110 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0005fd0 tx=0x7fa9c000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:46.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.284+0000 7fa9cdffb640 1 -- 172.21.15.43:0/3303728118 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9b804d020 con 0x7fa9d01009f0 2024-04-21T08:38:46.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:46.525+0000 7fa9d6ffd640 1 -- 172.21.15.43:0/3303728118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fa9d0062bf0 con 0x7fa9d01009f0 2024-04-21T08:38:47.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.468+0000 7fa9cdffb640 1 -- 172.21.15.43:0/3303728118 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fa9b80bf0e0 con 0x7fa9d01009f0 2024-04-21T08:38:47.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.471+0000 7fa9d6ffd640 1 -- 172.21.15.43:0/3303728118 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa9bc0a3c50 msgr2=0x7fa9bc0a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:47.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.471+0000 7fa9d6ffd640 1 --2- 172.21.15.43:0/3303728118 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa9bc0a3c50 0x7fa9bc0a6110 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0005fd0 tx=0x7fa9c000b410 comp rx=0 tx=0).stop 2024-04-21T08:38:47.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.471+0000 7fa9d6ffd640 1 -- 172.21.15.43:0/3303728118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9d01009f0 msgr2=0x7fa9d0198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:47.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.471+0000 7fa9d6ffd640 1 --2- 172.21.15.43:0/3303728118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9d01009f0 0x7fa9d0198670 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8005ab0 tx=0x7fa9b80042f0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.471+0000 7fa9d6ffd640 1 -- 172.21.15.43:0/3303728118 shutdown_connections 2024-04-21T08:38:47.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.471+0000 7fa9d6ffd640 1 --2- 172.21.15.43:0/3303728118 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fa9bc0a3c50 0x7fa9bc0a6110 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.471+0000 7fa9d6ffd640 1 --2- 172.21.15.43:0/3303728118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9d01035c0 0x7fa9d019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.471+0000 7fa9d6ffd640 1 --2- 172.21.15.43:0/3303728118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9d01009f0 0x7fa9d0198670 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.471+0000 7fa9d6ffd640 1 --2- 172.21.15.43:0/3303728118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9d0100030 0x7fa9d0198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.471+0000 7fa9d6ffd640 1 -- 172.21.15.43:0/3303728118 >> 172.21.15.43:0/3303728118 conn(0x7fa9d00fbbd0 msgr2=0x7fa9d0106650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:47.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.473+0000 7fa9d6ffd640 1 -- 172.21.15.43:0/3303728118 shutdown_connections 2024-04-21T08:38:47.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.474+0000 7fa9d6ffd640 1 -- 172.21.15.43:0/3303728118 wait complete. 2024-04-21T08:38:47.492 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-21T08:38:47.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.604+0000 7f2acb481640 1 -- 172.21.15.43:0/2673695454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ac41011e0 msgr2=0x7f2ac4101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:47.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.604+0000 7f2acb481640 1 --2- 172.21.15.43:0/2673695454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ac41011e0 0x7f2ac4101640 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4009900 tx=0x7f2ab402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:47.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.605+0000 7f2acb481640 1 -- 172.21.15.43:0/2673695454 shutdown_connections 2024-04-21T08:38:47.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.605+0000 7f2acb481640 1 --2- 172.21.15.43:0/2673695454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ac4101b80 0x7f2ac4102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.605+0000 7f2acb481640 1 --2- 172.21.15.43:0/2673695454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ac41011e0 0x7f2ac4101640 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.605+0000 7f2acb481640 1 --2- 172.21.15.43:0/2673695454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac4100880 0x7f2ac4100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.605+0000 7f2acb481640 1 -- 172.21.15.43:0/2673695454 >> 172.21.15.43:0/2673695454 conn(0x7f2ac40fc420 msgr2=0x7f2ac40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:47.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.605+0000 7f2acb481640 1 -- 172.21.15.43:0/2673695454 shutdown_connections 2024-04-21T08:38:47.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.605+0000 7f2acb481640 1 -- 172.21.15.43:0/2673695454 wait complete. 2024-04-21T08:38:47.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.605+0000 7f2acb481640 1 Processor -- start 2024-04-21T08:38:47.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.606+0000 7f2acb481640 1 -- start start 2024-04-21T08:38:47.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.606+0000 7f2acb481640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ac4100880 0x7f2ac419a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:47.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.606+0000 7f2acb481640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ac41011e0 0x7f2ac419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:47.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.606+0000 7f2acb481640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac4101b80 0x7f2ac41a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:47.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.606+0000 7f2acb481640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac410fd00 con 0x7f2ac41011e0 2024-04-21T08:38:47.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.606+0000 7f2acb481640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac410fb80 con 0x7f2ac4100880 2024-04-21T08:38:47.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.606+0000 7f2acb481640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac410fe80 con 0x7f2ac4101b80 2024-04-21T08:38:47.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.606+0000 7f2ac8a1c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ac41011e0 0x7f2ac419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:47.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.606+0000 7f2ac921d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ac4100880 0x7f2ac419a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:47.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.606+0000 7f2ac8a1c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ac41011e0 0x7f2ac419a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60114/0 (socket says 172.21.15.43:60114) 2024-04-21T08:38:47.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.606+0000 7f2ac8a1c640 1 -- 172.21.15.43:0/4136532545 learned_addr learned my addr 172.21.15.43:0/4136532545 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:47.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.606+0000 7f2ac921d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ac4100880 0x7f2ac419a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:49716/0 (socket says 172.21.15.43:49716) 2024-04-21T08:38:47.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.606+0000 7f2ac9a1e640 1 --2- 172.21.15.43:0/4136532545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac4101b80 0x7f2ac41a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:47.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.606+0000 7f2ac8a1c640 1 -- 172.21.15.43:0/4136532545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac4101b80 msgr2=0x7f2ac41a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:47.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.606+0000 7f2ac8a1c640 1 --2- 172.21.15.43:0/4136532545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac4101b80 0x7f2ac41a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.606+0000 7f2ac8a1c640 1 -- 172.21.15.43:0/4136532545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ac4100880 msgr2=0x7f2ac419a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:47.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.606+0000 7f2ac8a1c640 1 --2- 172.21.15.43:0/4136532545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ac4100880 0x7f2ac419a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.606+0000 7f2ac8a1c640 1 -- 172.21.15.43:0/4136532545 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac41a1f80 con 0x7f2ac41011e0 2024-04-21T08:38:47.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.606+0000 7f2ac921d640 1 --2- 172.21.15.43:0/4136532545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ac4100880 0x7f2ac419a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:38:47.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.606+0000 7f2ac8a1c640 1 --2- 172.21.15.43:0/4136532545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ac41011e0 0x7f2ac419a8a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4006010 tx=0x7f2ab4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:47.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.607+0000 7f2aba7fc640 1 -- 172.21.15.43:0/4136532545 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4043020 con 0x7f2ac41011e0 2024-04-21T08:38:47.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.607+0000 7f2aba7fc640 1 -- 172.21.15.43:0/4136532545 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ab4004440 con 0x7f2ac41011e0 2024-04-21T08:38:47.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.607+0000 7f2aba7fc640 1 -- 172.21.15.43:0/4136532545 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4003bf0 con 0x7f2ac41011e0 2024-04-21T08:38:47.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.607+0000 7f2acb481640 1 -- 172.21.15.43:0/4136532545 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac41a2210 con 0x7f2ac41011e0 2024-04-21T08:38:47.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.607+0000 7f2acb481640 1 -- 172.21.15.43:0/4136532545 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac41a2710 con 0x7f2ac41011e0 2024-04-21T08:38:47.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.608+0000 7f2acb481640 1 -- 172.21.15.43:0/4136532545 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ac410b5d0 con 0x7f2ac41011e0 2024-04-21T08:38:47.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.609+0000 7f2aba7fc640 1 -- 172.21.15.43:0/4136532545 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f2ab4046020 con 0x7f2ac41011e0 2024-04-21T08:38:47.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.609+0000 7f2aba7fc640 1 --2- 172.21.15.43:0/4136532545 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2ab00a3ca0 0x7f2ab00a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:47.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.609+0000 7f2ac921d640 1 -- 172.21.15.43:0/4136532545 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2ab00a3ca0 msgr2=0x7f2ab00a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1577030772 2024-04-21T08:38:47.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.610+0000 7f2aba7fc640 1 -- 172.21.15.43:0/4136532545 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f2ab40f2bb0 con 0x7f2ac41011e0 2024-04-21T08:38:47.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.610+0000 7f2ac921d640 1 --2- 172.21.15.43:0/4136532545 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2ab00a3ca0 0x7f2ab00a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:38:47.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.614+0000 7f2aba7fc640 1 -- 172.21.15.43:0/4136532545 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ab404d020 con 0x7f2ac41011e0 2024-04-21T08:38:47.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:47 smithi175 ceph-mon[31920]: pgmap v370: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-04-21T08:38:47.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:47 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3303728118' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-21T08:38:47.714 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:47 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: ignoring --setuser ceph since I am not root 2024-04-21T08:38:47.714 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:47 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: ignoring --setgroup ceph since I am not root 2024-04-21T08:38:47.714 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:47 smithi175 ceph-mgr[33785]: -- 172.21.15.175:0/2299361346 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d04aa185a0 con 0x55d04a9ecc00 2024-04-21T08:38:47.714 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:47 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:47.651+0000 7fd360358200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:38:47.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:47 smithi112 ceph-mon[31085]: pgmap v370: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-04-21T08:38:47.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:47 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3303728118' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-21T08:38:47.794 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:47 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: ignoring --setuser ceph since I am not root 2024-04-21T08:38:47.794 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:47 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: ignoring --setgroup ceph since I am not root 2024-04-21T08:38:47.794 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:47 smithi112 ceph-mgr[32478]: -- 172.21.15.112:0/2716729499 <== mon.2 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5610a43b85a0 con 0x5610a438d000 2024-04-21T08:38:47.795 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:47 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:47.645+0000 7fdf3a074200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:38:47.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.806+0000 7f2acb481640 1 -- 172.21.15.43:0/4136532545 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f2ac402d5c0 con 0x7f2ac41011e0 2024-04-21T08:38:47.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.810+0000 7f2aba7fc640 1 -- 172.21.15.43:0/4136532545 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f2ab40bf140 con 0x7f2ac41011e0 2024-04-21T08:38:47.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.811+0000 7f2ac921d640 1 -- 172.21.15.43:0/4136532545 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2ab00a3ca0 msgr2=0x7f2ab00a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1577030772 2024-04-21T08:38:47.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.811+0000 7f2ac921d640 1 --2- 172.21.15.43:0/4136532545 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2ab00a3ca0 0x7f2ab00a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:38:47.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:47 smithi043 ceph-mon[27978]: pgmap v370: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-04-21T08:38:47.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:47 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3303728118' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-21T08:38:47.811 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:47 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: ignoring --setuser ceph since I am not root 2024-04-21T08:38:47.811 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:47 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: ignoring --setgroup ceph since I am not root 2024-04-21T08:38:47.811 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:47 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:47.663+0000 7fa51cf15200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:38:47.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.812+0000 7f2acb481640 1 -- 172.21.15.43:0/4136532545 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2ab00a3ca0 msgr2=0x7f2ab00a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:38:47.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.812+0000 7f2acb481640 1 --2- 172.21.15.43:0/4136532545 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2ab00a3ca0 0x7f2ab00a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.812+0000 7f2acb481640 1 -- 172.21.15.43:0/4136532545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ac41011e0 msgr2=0x7f2ac419a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:47.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.812+0000 7f2acb481640 1 --2- 172.21.15.43:0/4136532545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ac41011e0 0x7f2ac419a8a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4006010 tx=0x7f2ab4004040 comp rx=0 tx=0).stop 2024-04-21T08:38:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.812+0000 7f2acb481640 1 -- 172.21.15.43:0/4136532545 shutdown_connections 2024-04-21T08:38:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.812+0000 7f2acb481640 1 --2- 172.21.15.43:0/4136532545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac4101b80 0x7f2ac41a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.812+0000 7f2acb481640 1 --2- 172.21.15.43:0/4136532545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ac41011e0 0x7f2ac419a8a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.812+0000 7f2acb481640 1 --2- 172.21.15.43:0/4136532545 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f2ab00a3ca0 0x7f2ab00a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.812+0000 7f2acb481640 1 --2- 172.21.15.43:0/4136532545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ac4100880 0x7f2ac419a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.812+0000 7f2acb481640 1 -- 172.21.15.43:0/4136532545 >> 172.21.15.43:0/4136532545 conn(0x7f2ac40fc420 msgr2=0x7f2ac40fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.812+0000 7f2acb481640 1 -- 172.21.15.43:0/4136532545 shutdown_connections 2024-04-21T08:38:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.812+0000 7f2acb481640 1 -- 172.21.15.43:0/4136532545 wait complete. 2024-04-21T08:38:47.823 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-21T08:38:47.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.927+0000 7f560c7d5640 1 -- 172.21.15.43:0/1384587513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5604102a00 msgr2=0x7f5604109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:47.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.927+0000 7f560c7d5640 1 --2- 172.21.15.43:0/1384587513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5604102a00 0x7f5604109320 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009ca0 tx=0x7f55fc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.928+0000 7f560c7d5640 1 -- 172.21.15.43:0/1384587513 shutdown_connections 2024-04-21T08:38:47.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.928+0000 7f560c7d5640 1 --2- 172.21.15.43:0/1384587513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5604102a00 0x7f5604109320 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.928+0000 7f560c7d5640 1 --2- 172.21.15.43:0/1384587513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5604102040 0x7f56041024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.928+0000 7f560c7d5640 1 --2- 172.21.15.43:0/1384587513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5604100e60 0x7f5604101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.928+0000 7f560c7d5640 1 -- 172.21.15.43:0/1384587513 >> 172.21.15.43:0/1384587513 conn(0x7f56040fc420 msgr2=0x7f56040fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:47.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.928+0000 7f560c7d5640 1 -- 172.21.15.43:0/1384587513 shutdown_connections 2024-04-21T08:38:47.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.928+0000 7f560c7d5640 1 -- 172.21.15.43:0/1384587513 wait complete. 2024-04-21T08:38:47.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.928+0000 7f560c7d5640 1 Processor -- start 2024-04-21T08:38:47.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.928+0000 7f560c7d5640 1 -- start start 2024-04-21T08:38:47.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.928+0000 7f560c7d5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5604100e60 0x7f5604198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:47.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.928+0000 7f560c7d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5604102040 0x7f5604198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:47.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.928+0000 7f560c7d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5604102a00 0x7f560419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:47.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.928+0000 7f560c7d5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f560410c9f0 con 0x7f5604102040 2024-04-21T08:38:47.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.928+0000 7f560c7d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f560410c870 con 0x7f5604102a00 2024-04-21T08:38:47.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.928+0000 7f560c7d5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f560410cb70 con 0x7f5604100e60 2024-04-21T08:38:47.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.929+0000 7f5609d70640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5604102040 0x7f5604198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:47.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.929+0000 7f5609d70640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5604102040 0x7f5604198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60124/0 (socket says 172.21.15.43:60124) 2024-04-21T08:38:47.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.929+0000 7f5609d70640 1 -- 172.21.15.43:0/850309736 learned_addr learned my addr 172.21.15.43:0/850309736 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:47.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.929+0000 7f560a571640 1 --2- 172.21.15.43:0/850309736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5604100e60 0x7f5604198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:47.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.929+0000 7f560ad72640 1 --2- 172.21.15.43:0/850309736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5604102a00 0x7f560419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:47.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.929+0000 7f5609d70640 1 -- 172.21.15.43:0/850309736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5604100e60 msgr2=0x7f5604198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:47.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.929+0000 7f5609d70640 1 --2- 172.21.15.43:0/850309736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5604100e60 0x7f5604198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.929+0000 7f5609d70640 1 -- 172.21.15.43:0/850309736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5604102a00 msgr2=0x7f560419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:47.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.929+0000 7f5609d70640 1 --2- 172.21.15.43:0/850309736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5604102a00 0x7f560419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.929+0000 7f5609d70640 1 -- 172.21.15.43:0/850309736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560419fd20 con 0x7f5604102040 2024-04-21T08:38:47.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.929+0000 7f5609d70640 1 --2- 172.21.15.43:0/850309736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5604102040 0x7f5604198640 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f560000e9f0 tx=0x7f560000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:47.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.929+0000 7f55fb7fe640 1 -- 172.21.15.43:0/850309736 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560000cd80 con 0x7f5604102040 2024-04-21T08:38:47.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.929+0000 7f55fb7fe640 1 -- 172.21.15.43:0/850309736 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56000108e0 con 0x7f5604102040 2024-04-21T08:38:47.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.929+0000 7f560c7d5640 1 -- 172.21.15.43:0/850309736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560419ffb0 con 0x7f5604102040 2024-04-21T08:38:47.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.929+0000 7f55fb7fe640 1 -- 172.21.15.43:0/850309736 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5600018980 con 0x7f5604102040 2024-04-21T08:38:47.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.929+0000 7f560c7d5640 1 -- 172.21.15.43:0/850309736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56041a04d0 con 0x7f5604102040 2024-04-21T08:38:47.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.930+0000 7f560c7d5640 1 -- 172.21.15.43:0/850309736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56041a0700 con 0x7f5604102040 2024-04-21T08:38:47.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.932+0000 7f55fb7fe640 1 -- 172.21.15.43:0/850309736 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f5600010430 con 0x7f5604102040 2024-04-21T08:38:47.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.932+0000 7f55fb7fe640 1 --2- 172.21.15.43:0/850309736 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f55f00a3ca0 0x7f55f00a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:47.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.932+0000 7f560a571640 1 -- 172.21.15.43:0/850309736 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f55f00a3ca0 msgr2=0x7f55f00a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1577030772 2024-04-21T08:38:47.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.933+0000 7f55fb7fe640 1 -- 172.21.15.43:0/850309736 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f56000d0f70 con 0x7f5604102040 2024-04-21T08:38:47.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.933+0000 7f560a571640 1 --2- 172.21.15.43:0/850309736 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f55f00a3ca0 0x7f55f00a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:38:47.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:47.936+0000 7f55fb7fe640 1 -- 172.21.15.43:0/850309736 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f560009ceb0 con 0x7f5604102040 2024-04-21T08:38:48.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:48.129+0000 7f560c7d5640 1 -- 172.21.15.43:0/850309736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f56041a08d0 con 0x7f5604102040 2024-04-21T08:38:48.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:48.133+0000 7f55fb7fe640 1 -- 172.21.15.43:0/850309736 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7f56000a11b0 con 0x7f5604102040 2024-04-21T08:38:48.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:48.134+0000 7f560a571640 1 -- 172.21.15.43:0/850309736 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f55f00a3ca0 msgr2=0x7f55f00a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1577030772 2024-04-21T08:38:48.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:48.134+0000 7f560a571640 1 --2- 172.21.15.43:0/850309736 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f55f00a3ca0 0x7f55f00a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:38:48.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:48.135+0000 7f560c7d5640 1 -- 172.21.15.43:0/850309736 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f55f00a3ca0 msgr2=0x7f55f00a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:38:48.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:48.135+0000 7f560c7d5640 1 --2- 172.21.15.43:0/850309736 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f55f00a3ca0 0x7f55f00a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:48.135+0000 7f560c7d5640 1 -- 172.21.15.43:0/850309736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5604102040 msgr2=0x7f5604198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:48.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:48.135+0000 7f560c7d5640 1 --2- 172.21.15.43:0/850309736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5604102040 0x7f5604198640 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f560000e9f0 tx=0x7f560000eeb0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:48.135+0000 7f560c7d5640 1 -- 172.21.15.43:0/850309736 shutdown_connections 2024-04-21T08:38:48.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:48.135+0000 7f560c7d5640 1 --2- 172.21.15.43:0/850309736 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f55f00a3ca0 0x7f55f00a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:48.135+0000 7f560c7d5640 1 --2- 172.21.15.43:0/850309736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5604102a00 0x7f560419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:48.135+0000 7f560c7d5640 1 --2- 172.21.15.43:0/850309736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5604102040 0x7f5604198640 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:48.135+0000 7f560c7d5640 1 --2- 172.21.15.43:0/850309736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5604100e60 0x7f5604198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:48.135+0000 7f560c7d5640 1 -- 172.21.15.43:0/850309736 >> 172.21.15.43:0/850309736 conn(0x7f56040fc420 msgr2=0x7f56040fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:48.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:48.135+0000 7f560c7d5640 1 -- 172.21.15.43:0/850309736 shutdown_connections 2024-04-21T08:38:48.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:48.135+0000 7f560c7d5640 1 -- 172.21.15.43:0/850309736 wait complete. 2024-04-21T08:38:48.144 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:47 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:47.858+0000 7fa51cf15200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:38:48.144 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:47 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:47.970+0000 7fa51cf15200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:38:48.145 INFO:teuthology.run_tasks:Running task kclient... 2024-04-21T08:38:48.162 INFO:tasks.kclient:Mounting kernel clients... 2024-04-21T08:38:48.162 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-21T08:38:48.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:38:48.162 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-21T08:38:48.179 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-21T08:38:48.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:38:48.180 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-21T08:38:48.213 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:47 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:47.834+0000 7fd360358200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:38:48.213 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:47 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:47.937+0000 7fd360358200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:38:48.240 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:47 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:47.828+0000 7fdf3a074200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:38:48.240 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:47 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:47.931+0000 7fdf3a074200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:38:48.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:38:48.245 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2024-04-21T08:38:48.345 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-21T08:38:48.345 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-21T08:38:48.345 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-21T08:38:48.345 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph fs ls 2024-04-21T08:38:48.399 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:48 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:48.305+0000 7fa51cf15200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:38:48.399 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:48 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:48.400+0000 7fa51cf15200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:38:48.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:48 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3303728118' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-21T08:38:48.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:48 smithi112 ceph-mon[31085]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-21T08:38:48.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:48 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/4136532545' entity='client.admin' 2024-04-21T08:38:48.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:48 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/850309736' entity='client.admin' 2024-04-21T08:38:48.544 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:48 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:48.240+0000 7fdf3a074200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:38:48.544 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:48 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:48.325+0000 7fdf3a074200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:38:48.608 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:48.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:48 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3303728118' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-21T08:38:48.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:48 smithi175 ceph-mon[31920]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-21T08:38:48.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:48 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/4136532545' entity='client.admin' 2024-04-21T08:38:48.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:48 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/850309736' entity='client.admin' 2024-04-21T08:38:48.714 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:48 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:48.248+0000 7fd360358200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:38:48.714 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:48 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:48.332+0000 7fd360358200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:38:48.731 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:48 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3303728118' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-21T08:38:48.732 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:48 smithi043 ceph-mon[27978]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-21T08:38:48.732 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:48 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/4136532545' entity='client.admin' 2024-04-21T08:38:48.732 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:48 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/850309736' entity='client.admin' 2024-04-21T08:38:49.785 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:49 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:49.680+0000 7fa51cf15200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:38:49.793 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:49 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:49.505+0000 7fdf3a074200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:38:49.794 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:49 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:49.673+0000 7fdf3a074200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:38:49.846 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:49 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:49.506+0000 7fd360358200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:38:49.846 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:49 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:49.674+0000 7fd360358200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:38:49.846 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:49 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:49.846+0000 7fd360358200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:38:50.046 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:49 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:49.865+0000 7fa51cf15200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:38:50.046 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:50 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:50.046+0000 7fa51cf15200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:38:50.131 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:49 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:49.843+0000 7fdf3a074200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:38:50.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.387+0000 7f3ce6cfa700 1 -- 172.21.15.43:0/865084232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce0108a80 msgr2=0x7f3ce0108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:50.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.387+0000 7f3ce6cfa700 1 --2- 172.21.15.43:0/865084232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce0108a80 0x7f3ce0108e60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4009990 tx=0x7f3cd4009ca0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.387+0000 7f3ce6cfa700 1 -- 172.21.15.43:0/865084232 shutdown_connections 2024-04-21T08:38:50.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.387+0000 7f3ce6cfa700 1 --2- 172.21.15.43:0/865084232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ce0103450 0x7f3ce010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.387+0000 7f3ce6cfa700 1 --2- 172.21.15.43:0/865084232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce0108a80 0x7f3ce0108e60 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.387+0000 7f3ce6cfa700 1 --2- 172.21.15.43:0/865084232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ce0102a90 0x7f3ce0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.387+0000 7f3ce6cfa700 1 -- 172.21.15.43:0/865084232 >> 172.21.15.43:0/865084232 conn(0x7f3ce0079bf0 msgr2=0x7f3ce007a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:50.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.387+0000 7f3ce6cfa700 1 -- 172.21.15.43:0/865084232 shutdown_connections 2024-04-21T08:38:50.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.388+0000 7f3ce6cfa700 1 -- 172.21.15.43:0/865084232 wait complete. 2024-04-21T08:38:50.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.388+0000 7f3ce6cfa700 1 Processor -- start 2024-04-21T08:38:50.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.388+0000 7f3ce6cfa700 1 -- start start 2024-04-21T08:38:50.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.388+0000 7f3ce6cfa700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce0102a90 0x7f3ce0199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:50.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.388+0000 7f3ce6cfa700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ce0103450 0x7f3ce0199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:50.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.388+0000 7f3ce6cfa700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ce0108a80 0x7f3ce019d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:50.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.388+0000 7f3ce6cfa700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce0112c30 con 0x7f3ce0102a90 2024-04-21T08:38:50.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.388+0000 7f3ce6cfa700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce0112ab0 con 0x7f3ce0103450 2024-04-21T08:38:50.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.388+0000 7f3ce6cfa700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce0112db0 con 0x7f3ce0108a80 2024-04-21T08:38:50.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.388+0000 7f3ce4a96700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce0102a90 0x7f3ce0199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:50.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.388+0000 7f3ce4a96700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce0102a90 0x7f3ce0199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60140/0 (socket says 172.21.15.43:60140) 2024-04-21T08:38:50.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.388+0000 7f3ce4a96700 1 -- 172.21.15.43:0/98258067 learned_addr learned my addr 172.21.15.43:0/98258067 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:50.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.388+0000 7f3ce5297700 1 --2- 172.21.15.43:0/98258067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ce0108a80 0x7f3ce019d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:50.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.388+0000 7f3cdffff700 1 --2- 172.21.15.43:0/98258067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ce0103450 0x7f3ce0199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:50.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.389+0000 7f3ce4a96700 1 -- 172.21.15.43:0/98258067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ce0108a80 msgr2=0x7f3ce019d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:50.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.389+0000 7f3ce4a96700 1 --2- 172.21.15.43:0/98258067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ce0108a80 0x7f3ce019d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.389+0000 7f3ce4a96700 1 -- 172.21.15.43:0/98258067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ce0103450 msgr2=0x7f3ce0199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:50.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.389+0000 7f3ce4a96700 1 --2- 172.21.15.43:0/98258067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ce0103450 0x7f3ce0199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.389+0000 7f3ce4a96700 1 -- 172.21.15.43:0/98258067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce019e110 con 0x7f3ce0102a90 2024-04-21T08:38:50.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.389+0000 7f3ce4a96700 1 --2- 172.21.15.43:0/98258067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce0102a90 0x7f3ce0199100 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4000c00 tx=0x7f3cd400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:50.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.389+0000 7f3cddffb700 1 -- 172.21.15.43:0/98258067 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4027070 con 0x7f3ce0102a90 2024-04-21T08:38:50.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.389+0000 7f3ce6cfa700 1 -- 172.21.15.43:0/98258067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce019e3a0 con 0x7f3ce0102a90 2024-04-21T08:38:50.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.389+0000 7f3cddffb700 1 -- 172.21.15.43:0/98258067 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cd4022470 con 0x7f3ce0102a90 2024-04-21T08:38:50.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.389+0000 7f3cddffb700 1 -- 172.21.15.43:0/98258067 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4017650 con 0x7f3ce0102a90 2024-04-21T08:38:50.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.389+0000 7f3ce6cfa700 1 -- 172.21.15.43:0/98258067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce01a5c80 con 0x7f3ce0102a90 2024-04-21T08:38:50.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.391+0000 7f3ce6cfa700 1 -- 172.21.15.43:0/98258067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ce0069730 con 0x7f3ce0102a90 2024-04-21T08:38:50.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.392+0000 7f3cddffb700 1 -- 172.21.15.43:0/98258067 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f3cd402a070 con 0x7f3ce0102a90 2024-04-21T08:38:50.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.392+0000 7f3cddffb700 1 --2- 172.21.15.43:0/98258067 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f3cd00a3d10 0x7f3cd00a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:50.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.392+0000 7f3cdffff700 1 -- 172.21.15.43:0/98258067 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f3cd00a3d10 msgr2=0x7f3cd00a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1577030772 2024-04-21T08:38:50.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.393+0000 7f3cddffb700 1 -- 172.21.15.43:0/98258067 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f3cd40d1920 con 0x7f3ce0102a90 2024-04-21T08:38:50.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.393+0000 7f3cdffff700 1 --2- 172.21.15.43:0/98258067 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f3cd00a3d10 0x7f3cd00a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:38:50.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.400+0000 7f3cddffb700 1 -- 172.21.15.43:0/98258067 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3cd4013070 con 0x7f3ce0102a90 2024-04-21T08:38:50.421 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:50 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:50.131+0000 7fdf3a074200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:38:50.421 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:50 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:50.221+0000 7fdf3a074200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:38:50.431 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:50 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:50.136+0000 7fd360358200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:38:50.431 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:50 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:50.227+0000 7fd360358200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:38:50.431 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:50 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:50.431+0000 7fd360358200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:38:50.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.594+0000 7f3cdffff700 1 -- 172.21.15.43:0/98258067 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f3cd00a3d10 msgr2=0x7f3cd00a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1577030772 2024-04-21T08:38:50.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.594+0000 7f3cdffff700 1 --2- 172.21.15.43:0/98258067 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f3cd00a3d10 0x7f3cd00a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:38:50.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.655+0000 7f3ce6cfa700 1 -- 172.21.15.43:0/98258067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3ce019a460 con 0x7f3ce0102a90 2024-04-21T08:38:50.654 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:50 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:50.356+0000 7fa51cf15200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:38:50.654 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:50 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:50.459+0000 7fa51cf15200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:38:50.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.655+0000 7f3cddffb700 1 -- 172.21.15.43:0/98258067 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+88 (secure 0 0 0) 0x7f3cd409dfa0 con 0x7f3ce0102a90 2024-04-21T08:38:50.655 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-21T08:38:50.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.657+0000 7f3ce6cfa700 1 -- 172.21.15.43:0/98258067 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f3cd00a3d10 msgr2=0x7f3cd00a61d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:38:50.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.657+0000 7f3ce6cfa700 1 --2- 172.21.15.43:0/98258067 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f3cd00a3d10 0x7f3cd00a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.657+0000 7f3ce6cfa700 1 -- 172.21.15.43:0/98258067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce0102a90 msgr2=0x7f3ce0199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:50.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.657+0000 7f3ce6cfa700 1 --2- 172.21.15.43:0/98258067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce0102a90 0x7f3ce0199100 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4000c00 tx=0x7f3cd400bfd0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.658+0000 7f3ce6cfa700 1 -- 172.21.15.43:0/98258067 shutdown_connections 2024-04-21T08:38:50.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.658+0000 7f3ce6cfa700 1 --2- 172.21.15.43:0/98258067 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f3cd00a3d10 0x7f3cd00a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.658+0000 7f3ce6cfa700 1 --2- 172.21.15.43:0/98258067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ce0108a80 0x7f3ce019d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.658+0000 7f3ce6cfa700 1 --2- 172.21.15.43:0/98258067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce0102a90 0x7f3ce0199100 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.658+0000 7f3ce6cfa700 1 --2- 172.21.15.43:0/98258067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ce0103450 0x7f3ce0199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.658+0000 7f3ce6cfa700 1 -- 172.21.15.43:0/98258067 >> 172.21.15.43:0/98258067 conn(0x7f3ce0079bf0 msgr2=0x7f3ce0101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:50.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.658+0000 7f3ce6cfa700 1 -- 172.21.15.43:0/98258067 shutdown_connections 2024-04-21T08:38:50.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:50.658+0000 7f3ce6cfa700 1 -- 172.21.15.43:0/98258067 wait complete. 2024-04-21T08:38:50.706 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:50 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:50.624+0000 7fd360358200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:38:50.706 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:50 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:50.422+0000 7fdf3a074200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:38:50.706 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:50 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:50.613+0000 7fdf3a074200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:38:50.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:50 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/98258067' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T08:38:51.026 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:50 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/98258067' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T08:38:51.026 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:50 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:50.675+0000 7fa51cf15200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:38:51.026 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:50 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:50.881+0000 7fa51cf15200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:38:51.043 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:50 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/98258067' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T08:38:51.381 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-21T08:38:51.382 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-21T08:38:51.382 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-04-21T08:38:51.382 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-21T08:38:51.382 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-21T08:38:51.382 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-21T08:38:51.382 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-21T08:38:51.382 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-21T08:38:51.383 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-21T08:38:51.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:38:51.383 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-04-21T08:38:51.400 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-21T08:38:51.400 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-21T08:38:51.400 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-04-21T08:38:51.400 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-21T08:38:51.400 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-04-21T08:38:51.401 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-21T08:38:51.401 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T08:38:51.401 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-04-21T08:38:51.401 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-04-21T08:38:51.401 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T08:38:51.401 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-04-21T08:38:51.401 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-04-21T08:38:51.401 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-21T08:38:51.401 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-04-21T08:38:51.401 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-04-21T08:38:51.401 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-21T08:38:51.401 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42034sec preferred_lft 42034sec 2024-04-21T08:38:51.401 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-04-21T08:38:51.401 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-21T08:38:51.401 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T08:38:51.402 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-04-21T08:38:51.402 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-04-21T08:38:51.402 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-21T08:38:51.402 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:38:51.402 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-21T08:38:51.402 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add name ceph-brx type bridge 2024-04-21T08:38:51.402 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr flush dev ceph-brx 2024-04-21T08:38:51.402 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set ceph-brx up 2024-04-21T08:38:51.402 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-21T08:38:51.403 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-21T08:38:51.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:38:51.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:38:51.544 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:51 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:51.206+0000 7fdf3a074200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:38:51.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:38:51.547 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-21T08:38:51.573 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-04-21T08:38:51.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:38:51.575 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-04-21T08:38:51.633 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-04-21T08:38:51.633 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-21T08:38:51.633 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-21T08:38:51.633 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-21T08:38:51.633 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-21T08:38:51.634 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:38:51.634 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-21T08:38:51.634 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-21T08:38:51.634 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-21T08:38:51.634 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-21T08:38:51.634 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-21T08:38:51.640 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:51 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:51.222+0000 7fd360358200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:38:51.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:38:51.742 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:51 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:51.520+0000 7fa51cf15200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:38:51.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:38:51.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:38:51.774 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-21T08:38:51.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:38:51.790 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list-id 2024-04-21T08:38:51.846 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:38:51.846 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-21T08:38:51.846 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-21T08:38:51.846 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-21T08:38:51.846 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-21T08:38:51.918 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:51 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:51.624+0000 7fdf3a074200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:38:51.919 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:51 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:51.718+0000 7fdf3a074200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:38:51.919 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:51 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:51.918+0000 7fdf3a074200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:38:51.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:38:51.937 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:51 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:51.641+0000 7fd360358200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:38:51.937 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:51 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:51.737+0000 7fd360358200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:38:51.937 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:51 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:51.937+0000 7fd360358200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:38:51.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:38:51.965 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-21T08:38:51.966 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:38:51.966 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-21T08:38:51.966 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-21T08:38:51.967 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-21T08:38:51.967 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-21T08:38:51.967 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-21T08:38:51.967 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-21T08:38:51.967 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-21T08:38:52.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:38:52.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:38:52.086 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:38:52.086 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-21T08:38:52.086 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-04-21T08:38:52.086 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set dev brx.0 master ceph-brx 2024-04-21T08:38:52.086 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-21T08:38:52.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:38:52.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:38:52.165 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-21T08:38:52.165 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-21T08:38:52.180 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-21T08:38:52.181 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-21T08:38:52.181 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-21T08:38:52.237 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-21T08:38:52.237 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-21T08:38:52.237 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/2d33bd43-04a4-4a16-bde2-90a5eaec28bf 2024-04-21T08:38:52.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:38:52.237 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/2d33bd43-04a4-4a16-bde2-90a5eaec28bf /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-04-21T08:38:52.299 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:51 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:51.996+0000 7fa51cf15200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:38:52.299 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:52.109+0000 7fa51cf15200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:38:52.320 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-04-21T08:38:52.320 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-04-21T08:38:52.320 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-04-21T08:38:52.461 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-04-21T08:38:52.461 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-04-21T08:38:52.461 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-04-21T08:38:52.461 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.112:3300,172.21.15.175:3300" 2024-04-21T08:38:52.461 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:3300,172.21.15.112:3300,172.21.15.175:3300:/volumes/qa/sv_1/2d33bd43-04a4-4a16-bde2-90a5eaec28bf 2024-04-21T08:38:52.461 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,key=0,fsid=d57a7d98-ffb8-11ee-bc93-c7b262605968" will pass to kernel 2024-04-21T08:38:52.462 INFO:teuthology.orchestra.run.smithi043.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-21T08:38:52.462 INFO:teuthology.orchestra.run.smithi043.stdout: You just mounted a file system that supports labels which does not 2024-04-21T08:38:52.462 INFO:teuthology.orchestra.run.smithi043.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-21T08:38:52.463 INFO:teuthology.orchestra.run.smithi043.stdout: applications will generate AVC messages and not be allowed access to 2024-04-21T08:38:52.463 INFO:teuthology.orchestra.run.smithi043.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-21T08:38:52.464 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-21T08:38:52.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:38:52.464 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-21T08:38:52.532 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-21T08:38:52.561 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:52.331+0000 7fa51cf15200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:38:52.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-21T08:38:52.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:38:52.620 INFO:teuthology.orchestra.run.smithi043.stdout:client25175 2024-04-21T08:38:52.620 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-21T08:38:52.620 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T08:38:52.637 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-21T08:38:52.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-21T08:38:52.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:38:52.735 INFO:teuthology.orchestra.run.smithi043.stdout:client25175 2024-04-21T08:38:52.735 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-21T08:38:52.775 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:52 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:52.438+0000 7fdf3a074200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:38:52.775 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:52 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:52.532+0000 7fdf3a074200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:38:52.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-21T08:38:52.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:38:52.793 INFO:teuthology.orchestra.run.smithi043.stdout:client25175 2024-04-21T08:38:52.794 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968.client25175/status') 2024-04-21T08:38:52.794 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:52 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:52.464+0000 7fd360358200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:38:52.794 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:52 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:52.554+0000 7fd360358200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:38:52.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:38:52.854 INFO:teuthology.orchestra.run.smithi043.stdout:instance: client.25175 (3)172.21.15.43:0/1879805656 2024-04-21T08:38:52.854 INFO:teuthology.orchestra.run.smithi043.stdout:blocklisted: false 2024-04-21T08:38:52.854 INFO:teuthology.orchestra.run.smithi043.stderr:0+1 records in 2024-04-21T08:38:52.854 INFO:teuthology.orchestra.run.smithi043.stderr:0+1 records out 2024-04-21T08:38:52.855 INFO:teuthology.orchestra.run.smithi043.stderr:71 bytes copied, 2.442e-05 s, 2.9 MB/s 2024-04-21T08:38:52.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:38:52.861 INFO:teuthology.run_tasks:Running task exec... 2024-04-21T08:38:52.870 INFO:teuthology.task.exec:Executing custom commands... 2024-04-21T08:38:52.870 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-04-21T08:38:52.871 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-21T08:38:52.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.977+0000 7f0e8259d640 1 -- 172.21.15.43:0/2644578355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e7c102870 msgr2=0x7f0e7c104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.977+0000 7f0e8259d640 1 --2- 172.21.15.43:0/2644578355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e7c102870 0x7f0e7c104c60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009900 tx=0x7f0e7802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:52.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.977+0000 7f0e8259d640 1 -- 172.21.15.43:0/2644578355 shutdown_connections 2024-04-21T08:38:52.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.977+0000 7f0e8259d640 1 --2- 172.21.15.43:0/2644578355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e7c107ad0 0x7f0e7c109f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.977+0000 7f0e8259d640 1 --2- 172.21.15.43:0/2644578355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e7c1051a0 0x7f0e7c107590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.977+0000 7f0e8259d640 1 --2- 172.21.15.43:0/2644578355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e7c102870 0x7f0e7c104c60 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.977+0000 7f0e8259d640 1 -- 172.21.15.43:0/2644578355 >> 172.21.15.43:0/2644578355 conn(0x7f0e7c0fc280 msgr2=0x7f0e7c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.977+0000 7f0e8259d640 1 -- 172.21.15.43:0/2644578355 shutdown_connections 2024-04-21T08:38:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.977+0000 7f0e8259d640 1 -- 172.21.15.43:0/2644578355 wait complete. 2024-04-21T08:38:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.977+0000 7f0e8259d640 1 Processor -- start 2024-04-21T08:38:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.977+0000 7f0e8259d640 1 -- start start 2024-04-21T08:38:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.978+0000 7f0e8259d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e7c102870 0x7f0e7c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.978+0000 7f0e8259d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e7c1051a0 0x7f0e7c1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.978+0000 7f0e8259d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e7c107ad0 0x7f0e7c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.978+0000 7f0e8259d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c10d920 con 0x7f0e7c102870 2024-04-21T08:38:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.978+0000 7f0e8259d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c10d7a0 con 0x7f0e7c107ad0 2024-04-21T08:38:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.978+0000 7f0e8259d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c10daa0 con 0x7f0e7c1051a0 2024-04-21T08:38:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.978+0000 7f0e8159b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e7c102870 0x7f0e7c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.978+0000 7f0e8159b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e7c102870 0x7f0e7c198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60162/0 (socket says 172.21.15.43:60162) 2024-04-21T08:38:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.978+0000 7f0e8159b640 1 -- 172.21.15.43:0/3683078825 learned_addr learned my addr 172.21.15.43:0/3683078825 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.978+0000 7f0e80d9a640 1 --2- 172.21.15.43:0/3683078825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e7c1051a0 0x7f0e7c1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.978+0000 7f0e81d9c640 1 --2- 172.21.15.43:0/3683078825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e7c107ad0 0x7f0e7c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.978+0000 7f0e8159b640 1 -- 172.21.15.43:0/3683078825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e7c1051a0 msgr2=0x7f0e7c1985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.978+0000 7f0e8159b640 1 --2- 172.21.15.43:0/3683078825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e7c1051a0 0x7f0e7c1985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.978+0000 7f0e8159b640 1 -- 172.21.15.43:0/3683078825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e7c107ad0 msgr2=0x7f0e7c19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.978+0000 7f0e8159b640 1 --2- 172.21.15.43:0/3683078825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e7c107ad0 0x7f0e7c19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.978+0000 7f0e8159b640 1 -- 172.21.15.43:0/3683078825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e7c19fcb0 con 0x7f0e7c102870 2024-04-21T08:38:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.978+0000 7f0e8159b640 1 --2- 172.21.15.43:0/3683078825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e7c102870 0x7f0e7c198090 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0e78006010 tx=0x7f0e780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.978+0000 7f0e727fc640 1 -- 172.21.15.43:0/3683078825 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78043020 con 0x7f0e7c102870 2024-04-21T08:38:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.978+0000 7f0e727fc640 1 -- 172.21.15.43:0/3683078825 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e78004aa0 con 0x7f0e7c102870 2024-04-21T08:38:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.978+0000 7f0e8259d640 1 -- 172.21.15.43:0/3683078825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e7c19ff40 con 0x7f0e7c102870 2024-04-21T08:38:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.978+0000 7f0e727fc640 1 -- 172.21.15.43:0/3683078825 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78003c70 con 0x7f0e7c102870 2024-04-21T08:38:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.979+0000 7f0e8259d640 1 -- 172.21.15.43:0/3683078825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e7c1a03a0 con 0x7f0e7c102870 2024-04-21T08:38:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.980+0000 7f0e8259d640 1 -- 172.21.15.43:0/3683078825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e7c0fdd70 con 0x7f0e7c102870 2024-04-21T08:38:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.981+0000 7f0e727fc640 1 -- 172.21.15.43:0/3683078825 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f0e78046020 con 0x7f0e7c102870 2024-04-21T08:38:52.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.981+0000 7f0e727fc640 1 --2- 172.21.15.43:0/3683078825 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0e680a3bf0 0x7f0e680a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.981+0000 7f0e80d9a640 1 -- 172.21.15.43:0/3683078825 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0e680a3bf0 msgr2=0x7f0e680a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1577030772 2024-04-21T08:38:52.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.982+0000 7f0e727fc640 1 -- 172.21.15.43:0/3683078825 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f0e780f31f0 con 0x7f0e7c102870 2024-04-21T08:38:52.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.982+0000 7f0e80d9a640 1 --2- 172.21.15.43:0/3683078825 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0e680a3bf0 0x7f0e680a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:38:52.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:52.989+0000 7f0e727fc640 1 -- 172.21.15.43:0/3683078825 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e7804d020 con 0x7f0e7c102870 2024-04-21T08:38:53.044 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:52 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:52.775+0000 7fdf3a074200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:38:53.044 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:52 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:52.907+0000 7fdf3a074200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:38:53.044 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:53 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:53.002+0000 7fdf3a074200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:38:53.134 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:52 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:52.793+0000 7fd360358200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:38:53.135 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:52 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:52.925+0000 7fd360358200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:38:53.135 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:53 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:53.020+0000 7fd360358200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:38:53.135 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:53 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:53.135+0000 7fd360358200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:38:53.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.183+0000 7f0e80d9a640 1 -- 172.21.15.43:0/3683078825 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0e680a3bf0 msgr2=0x7f0e680a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1577030772 2024-04-21T08:38:53.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.183+0000 7f0e80d9a640 1 --2- 172.21.15.43:0/3683078825 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0e680a3bf0 0x7f0e680a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:38:53.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.218+0000 7f0e8259d640 1 -- 172.21.15.43:0/3683078825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f0e7c02d810 con 0x7f0e7c102870 2024-04-21T08:38:53.263 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:52.896+0000 7fa51cf15200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:38:53.263 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:53 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:53.003+0000 7fa51cf15200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:38:53.263 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:53 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:53.264+0000 7fa51cf15200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:38:53.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.270+0000 7f0e727fc640 1 -- 172.21.15.43:0/3683078825 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7f0e780bf7e0 con 0x7f0e7c102870 2024-04-21T08:38:53.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.273+0000 7f0e8259d640 1 -- 172.21.15.43:0/3683078825 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0e680a3bf0 msgr2=0x7f0e680a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:38:53.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.273+0000 7f0e8259d640 1 --2- 172.21.15.43:0/3683078825 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0e680a3bf0 0x7f0e680a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.273+0000 7f0e8259d640 1 -- 172.21.15.43:0/3683078825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e7c102870 msgr2=0x7f0e7c198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:53.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.273+0000 7f0e8259d640 1 --2- 172.21.15.43:0/3683078825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e7c102870 0x7f0e7c198090 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0e78006010 tx=0x7f0e780042f0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.273+0000 7f0e8259d640 1 -- 172.21.15.43:0/3683078825 shutdown_connections 2024-04-21T08:38:53.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.273+0000 7f0e8259d640 1 --2- 172.21.15.43:0/3683078825 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f0e680a3bf0 0x7f0e680a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.273+0000 7f0e8259d640 1 --2- 172.21.15.43:0/3683078825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e7c107ad0 0x7f0e7c19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.273+0000 7f0e8259d640 1 --2- 172.21.15.43:0/3683078825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e7c1051a0 0x7f0e7c1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.273+0000 7f0e8259d640 1 --2- 172.21.15.43:0/3683078825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e7c102870 0x7f0e7c198090 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.273+0000 7f0e8259d640 1 -- 172.21.15.43:0/3683078825 >> 172.21.15.43:0/3683078825 conn(0x7f0e7c0fc280 msgr2=0x7f0e7c103290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:53.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.273+0000 7f0e8259d640 1 -- 172.21.15.43:0/3683078825 shutdown_connections 2024-04-21T08:38:53.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.273+0000 7f0e8259d640 1 -- 172.21.15.43:0/3683078825 wait complete. 2024-04-21T08:38:53.285 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-21T08:38:53.299 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-21T08:38:53.311 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:38:53.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.416+0000 7fb88abf4640 1 -- 172.21.15.43:0/1468158836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb884102a90 msgr2=0x7fb884104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:53.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.416+0000 7fb88abf4640 1 --2- 172.21.15.43:0/1468158836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb884102a90 0x7fb884104e80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb878009900 tx=0x7fb87802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:53.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.416+0000 7fb88abf4640 1 -- 172.21.15.43:0/1468158836 shutdown_connections 2024-04-21T08:38:53.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.416+0000 7fb88abf4640 1 --2- 172.21.15.43:0/1468158836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb884107cf0 0x7fb88410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.416+0000 7fb88abf4640 1 --2- 172.21.15.43:0/1468158836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8841053c0 0x7fb8841077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.416+0000 7fb88abf4640 1 --2- 172.21.15.43:0/1468158836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb884102a90 0x7fb884104e80 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.416+0000 7fb88abf4640 1 -- 172.21.15.43:0/1468158836 >> 172.21.15.43:0/1468158836 conn(0x7fb8840fc3e0 msgr2=0x7fb8840fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:53.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.416+0000 7fb88abf4640 1 -- 172.21.15.43:0/1468158836 shutdown_connections 2024-04-21T08:38:53.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.416+0000 7fb88abf4640 1 -- 172.21.15.43:0/1468158836 wait complete. 2024-04-21T08:38:53.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.417+0000 7fb88abf4640 1 Processor -- start 2024-04-21T08:38:53.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.417+0000 7fb88abf4640 1 -- start start 2024-04-21T08:38:53.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.417+0000 7fb88abf4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb884102a90 0x7fb8841981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:53.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.417+0000 7fb88abf4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8841053c0 0x7fb884198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:53.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.417+0000 7fb88abf4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb884107cf0 0x7fb88419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:53.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.417+0000 7fb88abf4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb88410daa0 con 0x7fb8841053c0 2024-04-21T08:38:53.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.417+0000 7fb88abf4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb88410d920 con 0x7fb884107cf0 2024-04-21T08:38:53.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.417+0000 7fb88abf4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb88410dc20 con 0x7fb884102a90 2024-04-21T08:38:53.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.417+0000 7fb883fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8841053c0 0x7fb884198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:53.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.417+0000 7fb883fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8841053c0 0x7fb884198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60178/0 (socket says 172.21.15.43:60178) 2024-04-21T08:38:53.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.417+0000 7fb883fff640 1 -- 172.21.15.43:0/1665659222 learned_addr learned my addr 172.21.15.43:0/1665659222 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:53.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.417+0000 7fb888990640 1 --2- 172.21.15.43:0/1665659222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb884102a90 0x7fb8841981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:53.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.417+0000 7fb889191640 1 --2- 172.21.15.43:0/1665659222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb884107cf0 0x7fb88419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:53.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.417+0000 7fb883fff640 1 -- 172.21.15.43:0/1665659222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb884102a90 msgr2=0x7fb8841981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:53.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.417+0000 7fb883fff640 1 --2- 172.21.15.43:0/1665659222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb884102a90 0x7fb8841981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.417+0000 7fb883fff640 1 -- 172.21.15.43:0/1665659222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb884107cf0 msgr2=0x7fb88419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:53.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.417+0000 7fb883fff640 1 --2- 172.21.15.43:0/1665659222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb884107cf0 0x7fb88419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.417+0000 7fb883fff640 1 -- 172.21.15.43:0/1665659222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88419fdf0 con 0x7fb8841053c0 2024-04-21T08:38:53.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.418+0000 7fb883fff640 1 --2- 172.21.15.43:0/1665659222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8841053c0 0x7fb884198710 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb87000e9c0 tx=0x7fb87000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:53.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.418+0000 7fb881ffb640 1 -- 172.21.15.43:0/1665659222 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87000cd80 con 0x7fb8841053c0 2024-04-21T08:38:53.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.418+0000 7fb88abf4640 1 -- 172.21.15.43:0/1665659222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8841a00e0 con 0x7fb8841053c0 2024-04-21T08:38:53.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.418+0000 7fb881ffb640 1 -- 172.21.15.43:0/1665659222 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8700054d0 con 0x7fb8841053c0 2024-04-21T08:38:53.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.418+0000 7fb881ffb640 1 -- 172.21.15.43:0/1665659222 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870010430 con 0x7fb8841053c0 2024-04-21T08:38:53.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.418+0000 7fb88abf4640 1 -- 172.21.15.43:0/1665659222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8841a0630 con 0x7fb8841053c0 2024-04-21T08:38:53.420 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:53 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3683078825' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-21T08:38:53.420 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:53 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:53.117+0000 7fdf3a074200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:38:53.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.419+0000 7fb88abf4640 1 -- 172.21.15.43:0/1665659222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8840fded0 con 0x7fb8841053c0 2024-04-21T08:38:53.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.420+0000 7fb881ffb640 1 -- 172.21.15.43:0/1665659222 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fb870004fd0 con 0x7fb8841053c0 2024-04-21T08:38:53.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.421+0000 7fb881ffb640 1 --2- 172.21.15.43:0/1665659222 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8740a3c00 0x7fb8740a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:53.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.421+0000 7fb888990640 1 -- 172.21.15.43:0/1665659222 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8740a3c00 msgr2=0x7fb8740a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1577030772 2024-04-21T08:38:53.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.421+0000 7fb881ffb640 1 -- 172.21.15.43:0/1665659222 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fb8700d0c40 con 0x7fb8841053c0 2024-04-21T08:38:53.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.421+0000 7fb888990640 1 --2- 172.21.15.43:0/1665659222 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8740a3c00 0x7fb8740a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:38:53.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.425+0000 7fb881ffb640 1 -- 172.21.15.43:0/1665659222 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb87009d230 con 0x7fb8841053c0 2024-04-21T08:38:53.440 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:53 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3683078825' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-21T08:38:53.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:53 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3683078825' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-21T08:38:53.533 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:53 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:53.411+0000 7fa51cf15200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:38:53.533 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:53 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:53.533+0000 7fa51cf15200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:38:53.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.622+0000 7fb888990640 1 -- 172.21.15.43:0/1665659222 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8740a3c00 msgr2=0x7fb8740a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1577030772 2024-04-21T08:38:53.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.622+0000 7fb888990640 1 --2- 172.21.15.43:0/1665659222 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8740a3c00 0x7fb8740a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:38:53.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.635+0000 7fb88abf4640 1 -- 172.21.15.43:0/1665659222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb884062b50 con 0x7fb8841053c0 2024-04-21T08:38:53.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.637+0000 7fb881ffb640 1 -- 172.21.15.43:0/1665659222 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24856 (secure 0 0 0) 0x7fb87009cb30 con 0x7fb8841053c0 2024-04-21T08:38:53.636 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:53.636 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":94,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:38:15.326063+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:38:53.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.639+0000 7fb88abf4640 1 -- 172.21.15.43:0/1665659222 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8740a3c00 msgr2=0x7fb8740a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:38:53.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.639+0000 7fb88abf4640 1 --2- 172.21.15.43:0/1665659222 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8740a3c00 0x7fb8740a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.639+0000 7fb88abf4640 1 -- 172.21.15.43:0/1665659222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8841053c0 msgr2=0x7fb884198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:53.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.639+0000 7fb88abf4640 1 --2- 172.21.15.43:0/1665659222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8841053c0 0x7fb884198710 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb87000e9c0 tx=0x7fb87000ee80 comp rx=0 tx=0).stop 2024-04-21T08:38:53.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.639+0000 7fb88abf4640 1 -- 172.21.15.43:0/1665659222 shutdown_connections 2024-04-21T08:38:53.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.639+0000 7fb88abf4640 1 --2- 172.21.15.43:0/1665659222 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fb8740a3c00 0x7fb8740a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.639+0000 7fb88abf4640 1 --2- 172.21.15.43:0/1665659222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb884107cf0 0x7fb88419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.639+0000 7fb88abf4640 1 --2- 172.21.15.43:0/1665659222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8841053c0 0x7fb884198710 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.639+0000 7fb88abf4640 1 --2- 172.21.15.43:0/1665659222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb884102a90 0x7fb8841981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.639+0000 7fb88abf4640 1 -- 172.21.15.43:0/1665659222 >> 172.21.15.43:0/1665659222 conn(0x7fb8840fc3e0 msgr2=0x7fb884103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:53.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.639+0000 7fb88abf4640 1 -- 172.21.15.43:0/1665659222 shutdown_connections 2024-04-21T08:38:53.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.639+0000 7fb88abf4640 1 -- 172.21.15.43:0/1665659222 wait complete. 2024-04-21T08:38:53.649 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:39.981405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:22.034751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:22.780795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:25.984261+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:38:53.650 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:38:53.713 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:53 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:53.440+0000 7fd360358200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:38:53.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.758+0000 7feab2443640 1 -- 172.21.15.43:0/4105257618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaac107c70 msgr2=0x7feaac10a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.758+0000 7feab2443640 1 --2- 172.21.15.43:0/4105257618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaac107c70 0x7feaac10a0c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009ca0 tx=0x7feaa402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.758+0000 7feab2443640 1 -- 172.21.15.43:0/4105257618 shutdown_connections 2024-04-21T08:38:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.758+0000 7feab2443640 1 --2- 172.21.15.43:0/4105257618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaac107c70 0x7feaac10a0c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.758+0000 7feab2443640 1 --2- 172.21.15.43:0/4105257618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaac105340 0x7feaac107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.758+0000 7feab2443640 1 --2- 172.21.15.43:0/4105257618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaac102a10 0x7feaac104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.758+0000 7feab2443640 1 -- 172.21.15.43:0/4105257618 >> 172.21.15.43:0/4105257618 conn(0x7feaac0fc400 msgr2=0x7feaac0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.759+0000 7feab2443640 1 -- 172.21.15.43:0/4105257618 shutdown_connections 2024-04-21T08:38:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.759+0000 7feab2443640 1 -- 172.21.15.43:0/4105257618 wait complete. 2024-04-21T08:38:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.759+0000 7feab2443640 1 Processor -- start 2024-04-21T08:38:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.759+0000 7feab2443640 1 -- start start 2024-04-21T08:38:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.759+0000 7feab2443640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaac102a10 0x7feaac1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.759+0000 7feab2443640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaac105340 0x7feaac198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.759+0000 7feab2443640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaac107c70 0x7feaac19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:53.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.759+0000 7feab2443640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feaac10dac0 con 0x7feaac107c70 2024-04-21T08:38:53.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.759+0000 7feab2443640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feaac10d940 con 0x7feaac105340 2024-04-21T08:38:53.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.759+0000 7feab2443640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feaac10dc40 con 0x7feaac102a10 2024-04-21T08:38:53.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.759+0000 7feab09e0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaac107c70 0x7feaac19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:53.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.760+0000 7feab09e0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaac107c70 0x7feaac19f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60184/0 (socket says 172.21.15.43:60184) 2024-04-21T08:38:53.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.760+0000 7feaab7fe640 1 --2- 172.21.15.43:0/113428980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaac105340 0x7feaac198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:53.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.760+0000 7feab09e0640 1 -- 172.21.15.43:0/113428980 learned_addr learned my addr 172.21.15.43:0/113428980 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:53.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.760+0000 7feaabfff640 1 --2- 172.21.15.43:0/113428980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaac102a10 0x7feaac1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:53.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.760+0000 7feaab7fe640 1 -- 172.21.15.43:0/113428980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaac102a10 msgr2=0x7feaac1981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:53.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.760+0000 7feaab7fe640 1 --2- 172.21.15.43:0/113428980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaac102a10 0x7feaac1981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.760+0000 7feaab7fe640 1 -- 172.21.15.43:0/113428980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaac107c70 msgr2=0x7feaac19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:53.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.760+0000 7feaab7fe640 1 --2- 172.21.15.43:0/113428980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaac107c70 0x7feaac19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.760+0000 7feaab7fe640 1 -- 172.21.15.43:0/113428980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaac19fde0 con 0x7feaac105340 2024-04-21T08:38:53.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.760+0000 7feab09e0640 1 --2- 172.21.15.43:0/113428980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaac107c70 0x7feaac19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:38:53.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.760+0000 7feaab7fe640 1 --2- 172.21.15.43:0/113428980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaac105340 0x7feaac198700 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00e9c0 tx=0x7fea9c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:53.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.760+0000 7feaa97fa640 1 -- 172.21.15.43:0/113428980 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c00ccc0 con 0x7feaac105340 2024-04-21T08:38:53.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.760+0000 7feab2443640 1 -- 172.21.15.43:0/113428980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaac1a00d0 con 0x7feaac105340 2024-04-21T08:38:53.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.760+0000 7feaa97fa640 1 -- 172.21.15.43:0/113428980 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea9c0108e0 con 0x7feaac105340 2024-04-21T08:38:53.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.760+0000 7feaa97fa640 1 -- 172.21.15.43:0/113428980 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c0187a0 con 0x7feaac105340 2024-04-21T08:38:53.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.760+0000 7feab2443640 1 -- 172.21.15.43:0/113428980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaac1a0620 con 0x7feaac105340 2024-04-21T08:38:53.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.761+0000 7feab2443640 1 -- 172.21.15.43:0/113428980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaac0fdef0 con 0x7feaac105340 2024-04-21T08:38:53.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.762+0000 7feaa97fa640 1 -- 172.21.15.43:0/113428980 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fea9c010430 con 0x7feaac105340 2024-04-21T08:38:53.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.763+0000 7feaa97fa640 1 --2- 172.21.15.43:0/113428980 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fea940a3c00 0x7fea940a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:53.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.763+0000 7feaabfff640 1 -- 172.21.15.43:0/113428980 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fea940a3c00 msgr2=0x7fea940a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1577030772 2024-04-21T08:38:53.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.763+0000 7feaa97fa640 1 -- 172.21.15.43:0/113428980 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fea9c0d0bc0 con 0x7feaac105340 2024-04-21T08:38:53.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.763+0000 7feaabfff640 1 --2- 172.21.15.43:0/113428980 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fea940a3c00 0x7fea940a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:38:53.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.767+0000 7feaa97fa640 1 -- 172.21.15.43:0/113428980 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea9c09cb00 con 0x7feaac105340 2024-04-21T08:38:53.793 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:53 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:53.418+0000 7fdf3a074200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:38:53.811 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:53 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:53.660+0000 7fa51cf15200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:38:53.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.954+0000 7feab2443640 1 -- 172.21.15.43:0/113428980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feaac02d810 con 0x7feaac105340 2024-04-21T08:38:53.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.954+0000 7feaa97fa640 1 -- 172.21.15.43:0/113428980 <== mon.2 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fea9c0a0e00 con 0x7feaac105340 2024-04-21T08:38:53.954 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-21T08:38:53.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.956+0000 7feab2443640 1 -- 172.21.15.43:0/113428980 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fea940a3c00 msgr2=0x7fea940a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:38:53.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.956+0000 7feab2443640 1 --2- 172.21.15.43:0/113428980 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fea940a3c00 0x7fea940a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.956+0000 7feab2443640 1 -- 172.21.15.43:0/113428980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaac105340 msgr2=0x7feaac198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:53.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.956+0000 7feab2443640 1 --2- 172.21.15.43:0/113428980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaac105340 0x7feaac198700 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00e9c0 tx=0x7fea9c00ee80 comp rx=0 tx=0).stop 2024-04-21T08:38:53.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.956+0000 7feab2443640 1 -- 172.21.15.43:0/113428980 shutdown_connections 2024-04-21T08:38:53.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.956+0000 7feab2443640 1 --2- 172.21.15.43:0/113428980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaac107c70 0x7feaac19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.956+0000 7feab2443640 1 --2- 172.21.15.43:0/113428980 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fea940a3c00 0x7fea940a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.956+0000 7feab2443640 1 --2- 172.21.15.43:0/113428980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaac105340 0x7feaac198700 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.956+0000 7feab2443640 1 --2- 172.21.15.43:0/113428980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaac102a10 0x7feaac1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.956+0000 7feab2443640 1 -- 172.21.15.43:0/113428980 >> 172.21.15.43:0/113428980 conn(0x7feaac0fc400 msgr2=0x7feaac1033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:53.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.956+0000 7feab2443640 1 -- 172.21.15.43:0/113428980 shutdown_connections 2024-04-21T08:38:53.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:53.956+0000 7feab2443640 1 -- 172.21.15.43:0/113428980 wait complete. 2024-04-21T08:38:53.967 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:38:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.075+0000 7f2393e4a640 1 -- 172.21.15.43:0/2123048603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f238c101b60 msgr2=0x7f238c101fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.075+0000 7f2393e4a640 1 --2- 172.21.15.43:0/2123048603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f238c101b60 0x7f238c101fe0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2384009ca0 tx=0x7f238402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.075+0000 7f2393e4a640 1 -- 172.21.15.43:0/2123048603 shutdown_connections 2024-04-21T08:38:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.075+0000 7f2393e4a640 1 --2- 172.21.15.43:0/2123048603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f238c101b60 0x7f238c101fe0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.075+0000 7f2393e4a640 1 --2- 172.21.15.43:0/2123048603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f238c1011c0 0x7f238c101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.075+0000 7f2393e4a640 1 --2- 172.21.15.43:0/2123048603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f238c100860 0x7f238c100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.075+0000 7f2393e4a640 1 -- 172.21.15.43:0/2123048603 >> 172.21.15.43:0/2123048603 conn(0x7f238c0fc400 msgr2=0x7f238c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.075+0000 7f2393e4a640 1 -- 172.21.15.43:0/2123048603 shutdown_connections 2024-04-21T08:38:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.075+0000 7f2393e4a640 1 -- 172.21.15.43:0/2123048603 wait complete. 2024-04-21T08:38:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.076+0000 7f2393e4a640 1 Processor -- start 2024-04-21T08:38:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.076+0000 7f2393e4a640 1 -- start start 2024-04-21T08:38:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.076+0000 7f2393e4a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f238c100860 0x7f238c19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.076+0000 7f2393e4a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f238c1011c0 0x7f238c19a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.076+0000 7f2393e4a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f238c101b60 0x7f238c1a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.076+0000 7f2393e4a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f238c10fce0 con 0x7f238c101b60 2024-04-21T08:38:54.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.076+0000 7f2393e4a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f238c10fb60 con 0x7f238c1011c0 2024-04-21T08:38:54.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.076+0000 7f2393e4a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f238c10fe60 con 0x7f238c100860 2024-04-21T08:38:54.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.076+0000 7f23923e7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f238c101b60 0x7f238c1a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.076+0000 7f2391be6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f238c100860 0x7f238c19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.076+0000 7f2391be6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f238c100860 0x7f238c19a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.43:49106/0 (socket says 172.21.15.43:49106) 2024-04-21T08:38:54.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.076+0000 7f2391be6640 1 -- 172.21.15.43:0/2993316560 learned_addr learned my addr 172.21.15.43:0/2993316560 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:54.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.076+0000 7f23913e5640 1 --2- 172.21.15.43:0/2993316560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f238c1011c0 0x7f238c19a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.076+0000 7f23923e7640 1 -- 172.21.15.43:0/2993316560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f238c100860 msgr2=0x7f238c19a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.076+0000 7f23923e7640 1 --2- 172.21.15.43:0/2993316560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f238c100860 0x7f238c19a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.076+0000 7f23923e7640 1 -- 172.21.15.43:0/2993316560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f238c1011c0 msgr2=0x7f238c19a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.076+0000 7f23923e7640 1 --2- 172.21.15.43:0/2993316560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f238c1011c0 0x7f238c19a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.076+0000 7f23923e7640 1 -- 172.21.15.43:0/2993316560 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238c1a1fa0 con 0x7f238c101b60 2024-04-21T08:38:54.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.076+0000 7f23923e7640 1 --2- 172.21.15.43:0/2993316560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f238c101b60 0x7f238c1a1930 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f238402c630 tx=0x7f23840346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:54.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.077+0000 7f2382ffd640 1 -- 172.21.15.43:0/2993316560 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2384041020 con 0x7f238c101b60 2024-04-21T08:38:54.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.077+0000 7f2382ffd640 1 -- 172.21.15.43:0/2993316560 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2384034ca0 con 0x7f238c101b60 2024-04-21T08:38:54.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.077+0000 7f2393e4a640 1 -- 172.21.15.43:0/2993316560 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f238c1a2230 con 0x7f238c101b60 2024-04-21T08:38:54.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.077+0000 7f2382ffd640 1 -- 172.21.15.43:0/2993316560 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2384039040 con 0x7f238c101b60 2024-04-21T08:38:54.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.077+0000 7f2393e4a640 1 -- 172.21.15.43:0/2993316560 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f238c1a2770 con 0x7f238c101b60 2024-04-21T08:38:54.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.078+0000 7f2393e4a640 1 -- 172.21.15.43:0/2993316560 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f238c10b7e0 con 0x7f238c101b60 2024-04-21T08:38:54.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.080+0000 7f2382ffd640 1 -- 172.21.15.43:0/2993316560 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f2384044020 con 0x7f238c101b60 2024-04-21T08:38:54.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.080+0000 7f2382ffd640 1 --2- 172.21.15.43:0/2993316560 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f23780a3c80 0x7f23780a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.080+0000 7f2391be6640 1 -- 172.21.15.43:0/2993316560 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f23780a3c80 msgr2=0x7f23780a6140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1577030772 2024-04-21T08:38:54.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.080+0000 7f2382ffd640 1 -- 172.21.15.43:0/2993316560 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f23840f28a0 con 0x7f238c101b60 2024-04-21T08:38:54.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.080+0000 7f2391be6640 1 --2- 172.21.15.43:0/2993316560 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f23780a3c80 0x7f23780a6140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:38:54.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.083+0000 7f2382ffd640 1 -- 172.21.15.43:0/2993316560 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f238404b020 con 0x7f238c101b60 2024-04-21T08:38:54.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.274+0000 7f2393e4a640 1 -- 172.21.15.43:0/2993316560 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f238c02d5c0 con 0x7f238c101b60 2024-04-21T08:38:54.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.274+0000 7f2382ffd640 1 -- 172.21.15.43:0/2993316560 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f23840bee90 con 0x7f238c101b60 2024-04-21T08:38:54.274 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:38:54.275 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:53 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:53.984+0000 7fa51cf15200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:38:54.275 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:54 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:54.016+0000 7fdf3a074200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:38:54.275 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:38:54 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y[32474]: 2024-04-21T08:38:54.235+0000 7fdf3a074200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:38:54.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.277+0000 7f2393e4a640 1 -- 172.21.15.43:0/2993316560 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f23780a3c80 msgr2=0x7f23780a6140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:38:54.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.277+0000 7f2393e4a640 1 --2- 172.21.15.43:0/2993316560 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f23780a3c80 0x7f23780a6140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.277+0000 7f2393e4a640 1 -- 172.21.15.43:0/2993316560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f238c101b60 msgr2=0x7f238c1a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.277+0000 7f2393e4a640 1 --2- 172.21.15.43:0/2993316560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f238c101b60 0x7f238c1a1930 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f238402c630 tx=0x7f23840346d0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.277+0000 7f2393e4a640 1 -- 172.21.15.43:0/2993316560 shutdown_connections 2024-04-21T08:38:54.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.277+0000 7f2393e4a640 1 --2- 172.21.15.43:0/2993316560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f238c101b60 0x7f238c1a1930 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.277+0000 7f2393e4a640 1 --2- 172.21.15.43:0/2993316560 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7f23780a3c80 0x7f23780a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.277+0000 7f2393e4a640 1 --2- 172.21.15.43:0/2993316560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f238c1011c0 0x7f238c19a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.277+0000 7f2393e4a640 1 --2- 172.21.15.43:0/2993316560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f238c100860 0x7f238c19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.277+0000 7f2393e4a640 1 -- 172.21.15.43:0/2993316560 >> 172.21.15.43:0/2993316560 conn(0x7f238c0fc400 msgr2=0x7f238c0fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:54.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.277+0000 7f2393e4a640 1 -- 172.21.15.43:0/2993316560 shutdown_connections 2024-04-21T08:38:54.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.277+0000 7f2393e4a640 1 -- 172.21.15.43:0/2993316560 wait complete. 2024-04-21T08:38:54.291 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:38:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.398+0000 7fdfbb9a4640 1 -- 172.21.15.43:0/3323034765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfb41029a0 msgr2=0x7fdfb41092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.398+0000 7fdfbb9a4640 1 --2- 172.21.15.43:0/3323034765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfb41029a0 0x7fdfb41092c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdfac009ca0 tx=0x7fdfac02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.398+0000 7fdfbb9a4640 1 -- 172.21.15.43:0/3323034765 shutdown_connections 2024-04-21T08:38:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.398+0000 7fdfbb9a4640 1 --2- 172.21.15.43:0/3323034765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfb41029a0 0x7fdfb41092c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.398+0000 7fdfbb9a4640 1 --2- 172.21.15.43:0/3323034765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfb4101fe0 0x7fdfb4102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.398+0000 7fdfbb9a4640 1 --2- 172.21.15.43:0/3323034765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfb4100e90 0x7fdfb41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.398+0000 7fdfbb9a4640 1 -- 172.21.15.43:0/3323034765 >> 172.21.15.43:0/3323034765 conn(0x7fdfb40fc400 msgr2=0x7fdfb40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.398+0000 7fdfbb9a4640 1 -- 172.21.15.43:0/3323034765 shutdown_connections 2024-04-21T08:38:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.398+0000 7fdfbb9a4640 1 -- 172.21.15.43:0/3323034765 wait complete. 2024-04-21T08:38:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.399+0000 7fdfbb9a4640 1 Processor -- start 2024-04-21T08:38:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.399+0000 7fdfbb9a4640 1 -- start start 2024-04-21T08:38:54.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.399+0000 7fdfbb9a4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfb4100e90 0x7fdfb4069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.399+0000 7fdfbb9a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfb4101fe0 0x7fdfb419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.399+0000 7fdfbb9a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfb41029a0 0x7fdfb419f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.399+0000 7fdfbb9a4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb410c9d0 con 0x7fdfb41029a0 2024-04-21T08:38:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.399+0000 7fdfbb9a4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb410c850 con 0x7fdfb4101fe0 2024-04-21T08:38:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.399+0000 7fdfbb9a4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb410cb50 con 0x7fdfb4100e90 2024-04-21T08:38:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.399+0000 7fdfb9f41640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfb41029a0 0x7fdfb419f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.399+0000 7fdfb9f41640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfb41029a0 0x7fdfb419f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60216/0 (socket says 172.21.15.43:60216) 2024-04-21T08:38:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.399+0000 7fdfb9f41640 1 -- 172.21.15.43:0/393926609 learned_addr learned my addr 172.21.15.43:0/393926609 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.399+0000 7fdfb9740640 1 --2- 172.21.15.43:0/393926609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfb4100e90 0x7fdfb4069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.399+0000 7fdfb8f3f640 1 --2- 172.21.15.43:0/393926609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfb4101fe0 0x7fdfb419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.399+0000 7fdfb9f41640 1 -- 172.21.15.43:0/393926609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfb4100e90 msgr2=0x7fdfb4069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.399+0000 7fdfb9f41640 1 --2- 172.21.15.43:0/393926609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfb4100e90 0x7fdfb4069cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.399+0000 7fdfb9f41640 1 -- 172.21.15.43:0/393926609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfb4101fe0 msgr2=0x7fdfb419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.399+0000 7fdfb9f41640 1 --2- 172.21.15.43:0/393926609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfb4101fe0 0x7fdfb419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.399+0000 7fdfb9f41640 1 -- 172.21.15.43:0/393926609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb419fdb0 con 0x7fdfb41029a0 2024-04-21T08:38:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.399+0000 7fdfb9740640 1 --2- 172.21.15.43:0/393926609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfb4100e90 0x7fdfb4069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.400+0000 7fdfb9f41640 1 --2- 172.21.15.43:0/393926609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfb41029a0 0x7fdfb419f4f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdfac02c700 tx=0x7fdfac034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:54.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.400+0000 7fdfaa7fc640 1 -- 172.21.15.43:0/393926609 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac039040 con 0x7fdfb41029a0 2024-04-21T08:38:54.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.400+0000 7fdfaa7fc640 1 -- 172.21.15.43:0/393926609 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfac044070 con 0x7fdfb41029a0 2024-04-21T08:38:54.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.400+0000 7fdfbb9a4640 1 -- 172.21.15.43:0/393926609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb41a0040 con 0x7fdfb41029a0 2024-04-21T08:38:54.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.400+0000 7fdfaa7fc640 1 -- 172.21.15.43:0/393926609 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac0042a0 con 0x7fdfb41029a0 2024-04-21T08:38:54.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.400+0000 7fdfbb9a4640 1 -- 172.21.15.43:0/393926609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb41a0530 con 0x7fdfb41029a0 2024-04-21T08:38:54.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.401+0000 7fdfbb9a4640 1 -- 172.21.15.43:0/393926609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfb4108270 con 0x7fdfb41029a0 2024-04-21T08:38:54.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.402+0000 7fdfaa7fc640 1 -- 172.21.15.43:0/393926609 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7fdfac003970 con 0x7fdfb41029a0 2024-04-21T08:38:54.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.403+0000 7fdfaa7fc640 1 --2- 172.21.15.43:0/393926609 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fdfa00a3c00 0x7fdfa00a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.403+0000 7fdfb9740640 1 -- 172.21.15.43:0/393926609 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fdfa00a3c00 msgr2=0x7fdfa00a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1577030772 2024-04-21T08:38:54.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.403+0000 7fdfaa7fc640 1 -- 172.21.15.43:0/393926609 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fdfac0f2cf0 con 0x7fdfb41029a0 2024-04-21T08:38:54.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.403+0000 7fdfb9740640 1 --2- 172.21.15.43:0/393926609 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fdfa00a3c00 0x7fdfa00a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:38:54.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.408+0000 7fdfaa7fc640 1 -- 172.21.15.43:0/393926609 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdfac0f7070 con 0x7fdfb41029a0 2024-04-21T08:38:54.463 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:54 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:54.042+0000 7fd360358200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:38:54.464 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:38:54 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z[33781]: 2024-04-21T08:38:54.257+0000 7fd360358200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:38:54.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:54 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3683078825' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-21T08:38:54.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:54 smithi175 ceph-mon[31920]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:38:54.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:54 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1665659222' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:54.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:54 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/113428980' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:54.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:54 smithi175 ceph-mon[31920]: Standby manager daemon y restarted 2024-04-21T08:38:54.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:54 smithi175 ceph-mon[31920]: Standby manager daemon y started 2024-04-21T08:38:54.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:54 smithi175 ceph-mon[31920]: from='mgr.? 172.21.15.112:0/1960313429' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T08:38:54.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:54 smithi175 ceph-mon[31920]: from='mgr.? 172.21.15.112:0/1960313429' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:38:54.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:54 smithi175 ceph-mon[31920]: from='mgr.? 172.21.15.112:0/1960313429' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T08:38:54.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:54 smithi175 ceph-mon[31920]: from='mgr.? 172.21.15.112:0/1960313429' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:38:54.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:54 smithi175 ceph-mon[31920]: Standby manager daemon z restarted 2024-04-21T08:38:54.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:54 smithi175 ceph-mon[31920]: Standby manager daemon z started 2024-04-21T08:38:54.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:54 smithi175 ceph-mon[31920]: from='mgr.? 172.21.15.175:0/902628852' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T08:38:54.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:54 smithi175 ceph-mon[31920]: from='mgr.? 172.21.15.175:0/902628852' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:38:54.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:54 smithi175 ceph-mon[31920]: from='mgr.? 172.21.15.175:0/902628852' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T08:38:54.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:54 smithi175 ceph-mon[31920]: from='mgr.? 172.21.15.175:0/902628852' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:38:54.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:54 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2993316560' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:54 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3683078825' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-21T08:38:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:54 smithi112 ceph-mon[31085]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:38:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:54 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1665659222' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:54 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/113428980' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:54 smithi112 ceph-mon[31085]: Standby manager daemon y restarted 2024-04-21T08:38:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:54 smithi112 ceph-mon[31085]: Standby manager daemon y started 2024-04-21T08:38:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:54 smithi112 ceph-mon[31085]: from='mgr.? 172.21.15.112:0/1960313429' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T08:38:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:54 smithi112 ceph-mon[31085]: from='mgr.? 172.21.15.112:0/1960313429' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:38:54.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:54 smithi112 ceph-mon[31085]: from='mgr.? 172.21.15.112:0/1960313429' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T08:38:54.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:54 smithi112 ceph-mon[31085]: from='mgr.? 172.21.15.112:0/1960313429' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:38:54.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:54 smithi112 ceph-mon[31085]: Standby manager daemon z restarted 2024-04-21T08:38:54.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:54 smithi112 ceph-mon[31085]: Standby manager daemon z started 2024-04-21T08:38:54.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:54 smithi112 ceph-mon[31085]: from='mgr.? 172.21.15.175:0/902628852' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T08:38:54.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:54 smithi112 ceph-mon[31085]: from='mgr.? 172.21.15.175:0/902628852' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:38:54.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:54 smithi112 ceph-mon[31085]: from='mgr.? 172.21.15.175:0/902628852' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T08:38:54.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:54 smithi112 ceph-mon[31085]: from='mgr.? 172.21.15.175:0/902628852' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:38:54.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:54 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2993316560' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:54.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:54 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3683078825' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-21T08:38:54.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:54 smithi043 ceph-mon[27978]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:38:54.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:54 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1665659222' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:54.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:54 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/113428980' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:54.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:54 smithi043 ceph-mon[27978]: Standby manager daemon y restarted 2024-04-21T08:38:54.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:54 smithi043 ceph-mon[27978]: Standby manager daemon y started 2024-04-21T08:38:54.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:54 smithi043 ceph-mon[27978]: from='mgr.? 172.21.15.112:0/1960313429' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T08:38:54.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:54 smithi043 ceph-mon[27978]: from='mgr.? 172.21.15.112:0/1960313429' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:38:54.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:54 smithi043 ceph-mon[27978]: from='mgr.? 172.21.15.112:0/1960313429' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T08:38:54.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:54 smithi043 ceph-mon[27978]: from='mgr.? 172.21.15.112:0/1960313429' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:38:54.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:54 smithi043 ceph-mon[27978]: Standby manager daemon z restarted 2024-04-21T08:38:54.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:54 smithi043 ceph-mon[27978]: Standby manager daemon z started 2024-04-21T08:38:54.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:54 smithi043 ceph-mon[27978]: from='mgr.? 172.21.15.175:0/902628852' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T08:38:54.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:54 smithi043 ceph-mon[27978]: from='mgr.? 172.21.15.175:0/902628852' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:38:54.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:54 smithi043 ceph-mon[27978]: from='mgr.? 172.21.15.175:0/902628852' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T08:38:54.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:54 smithi043 ceph-mon[27978]: from='mgr.? 172.21.15.175:0/902628852' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:38:54.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:54 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2993316560' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:54.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.596+0000 7fdfbb9a4640 1 -- 172.21.15.43:0/393926609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdfb4062b50 con 0x7fdfb41029a0 2024-04-21T08:38:54.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.597+0000 7fdfaa7fc640 1 -- 172.21.15.43:0/393926609 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fdfac0bf2e0 con 0x7fdfb41029a0 2024-04-21T08:38:54.596 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-21T08:38:54.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.599+0000 7fdfbb9a4640 1 -- 172.21.15.43:0/393926609 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fdfa00a3c00 msgr2=0x7fdfa00a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:38:54.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.599+0000 7fdfbb9a4640 1 --2- 172.21.15.43:0/393926609 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fdfa00a3c00 0x7fdfa00a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.599+0000 7fdfbb9a4640 1 -- 172.21.15.43:0/393926609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfb41029a0 msgr2=0x7fdfb419f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.599+0000 7fdfbb9a4640 1 --2- 172.21.15.43:0/393926609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfb41029a0 0x7fdfb419f4f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdfac02c700 tx=0x7fdfac034760 comp rx=0 tx=0).stop 2024-04-21T08:38:54.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.599+0000 7fdfbb9a4640 1 -- 172.21.15.43:0/393926609 shutdown_connections 2024-04-21T08:38:54.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.599+0000 7fdfbb9a4640 1 --2- 172.21.15.43:0/393926609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfb41029a0 0x7fdfb419f4f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.599+0000 7fdfbb9a4640 1 --2- 172.21.15.43:0/393926609 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7fdfa00a3c00 0x7fdfa00a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.599+0000 7fdfbb9a4640 1 --2- 172.21.15.43:0/393926609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfb4101fe0 0x7fdfb419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.599+0000 7fdfbb9a4640 1 --2- 172.21.15.43:0/393926609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfb4100e90 0x7fdfb4069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.599+0000 7fdfbb9a4640 1 -- 172.21.15.43:0/393926609 >> 172.21.15.43:0/393926609 conn(0x7fdfb40fc400 msgr2=0x7fdfb4105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:54.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.599+0000 7fdfbb9a4640 1 -- 172.21.15.43:0/393926609 shutdown_connections 2024-04-21T08:38:54.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.599+0000 7fdfbb9a4640 1 -- 172.21.15.43:0/393926609 wait complete. 2024-04-21T08:38:54.610 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:38:54.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.717+0000 7ff8d868c640 1 -- 172.21.15.43:0/3477592522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d0100860 msgr2=0x7ff8d0100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.717+0000 7ff8d868c640 1 --2- 172.21.15.43:0/3477592522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d0100860 0x7ff8d0100c80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc009900 tx=0x7ff8cc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:54.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.717+0000 7ff8d868c640 1 -- 172.21.15.43:0/3477592522 shutdown_connections 2024-04-21T08:38:54.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.717+0000 7ff8d868c640 1 --2- 172.21.15.43:0/3477592522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8d0101b60 0x7ff8d0101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.717+0000 7ff8d868c640 1 --2- 172.21.15.43:0/3477592522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8d01011c0 0x7ff8d0101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.717+0000 7ff8d868c640 1 --2- 172.21.15.43:0/3477592522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d0100860 0x7ff8d0100c80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.717+0000 7ff8d868c640 1 -- 172.21.15.43:0/3477592522 >> 172.21.15.43:0/3477592522 conn(0x7ff8d00fc400 msgr2=0x7ff8d00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:54.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.717+0000 7ff8d868c640 1 -- 172.21.15.43:0/3477592522 shutdown_connections 2024-04-21T08:38:54.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.717+0000 7ff8d868c640 1 -- 172.21.15.43:0/3477592522 wait complete. 2024-04-21T08:38:54.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.718+0000 7ff8d868c640 1 Processor -- start 2024-04-21T08:38:54.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.718+0000 7ff8d868c640 1 -- start start 2024-04-21T08:38:54.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.718+0000 7ff8d868c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8d0100860 0x7ff8d019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.718+0000 7ff8d868c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d01011c0 0x7ff8d019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.718+0000 7ff8d868c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8d0101b60 0x7ff8d01a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.718+0000 7ff8d868c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d010fce0 con 0x7ff8d01011c0 2024-04-21T08:38:54.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.718+0000 7ff8d868c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d010fb60 con 0x7ff8d0101b60 2024-04-21T08:38:54.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.718+0000 7ff8d868c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d010fe60 con 0x7ff8d0100860 2024-04-21T08:38:54.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.718+0000 7ff8d5c27640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d01011c0 0x7ff8d019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.718+0000 7ff8d5c27640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d01011c0 0x7ff8d019a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60232/0 (socket says 172.21.15.43:60232) 2024-04-21T08:38:54.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.718+0000 7ff8d5c27640 1 -- 172.21.15.43:0/8601972 learned_addr learned my addr 172.21.15.43:0/8601972 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:54.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.718+0000 7ff8d6428640 1 --2- 172.21.15.43:0/8601972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8d0100860 0x7ff8d019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.718+0000 7ff8d6c29640 1 --2- 172.21.15.43:0/8601972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8d0101b60 0x7ff8d01a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.719+0000 7ff8d5c27640 1 -- 172.21.15.43:0/8601972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8d0100860 msgr2=0x7ff8d019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.719+0000 7ff8d5c27640 1 --2- 172.21.15.43:0/8601972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8d0100860 0x7ff8d019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.719+0000 7ff8d5c27640 1 -- 172.21.15.43:0/8601972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8d0101b60 msgr2=0x7ff8d01a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.719+0000 7ff8d5c27640 1 --2- 172.21.15.43:0/8601972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8d0101b60 0x7ff8d01a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.719+0000 7ff8d5c27640 1 -- 172.21.15.43:0/8601972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d01a2080 con 0x7ff8d01011c0 2024-04-21T08:38:54.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.719+0000 7ff8d6c29640 1 --2- 172.21.15.43:0/8601972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8d0101b60 0x7ff8d01a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:54.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.719+0000 7ff8d5c27640 1 --2- 172.21.15.43:0/8601972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d01011c0 0x7ff8d019a9a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000e9f0 tx=0x7ff8c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:54.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.719+0000 7ff8c77fe640 1 -- 172.21.15.43:0/8601972 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c000cd80 con 0x7ff8d01011c0 2024-04-21T08:38:54.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.719+0000 7ff8d868c640 1 -- 172.21.15.43:0/8601972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8d01a2370 con 0x7ff8d01011c0 2024-04-21T08:38:54.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.719+0000 7ff8c77fe640 1 -- 172.21.15.43:0/8601972 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8c00054d0 con 0x7ff8d01011c0 2024-04-21T08:38:54.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.719+0000 7ff8c77fe640 1 -- 172.21.15.43:0/8601972 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0010690 con 0x7ff8d01011c0 2024-04-21T08:38:54.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.719+0000 7ff8d868c640 1 -- 172.21.15.43:0/8601972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8d01a28c0 con 0x7ff8d01011c0 2024-04-21T08:38:54.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.720+0000 7ff8d868c640 1 -- 172.21.15.43:0/8601972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8d010b8d0 con 0x7ff8d01011c0 2024-04-21T08:38:54.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.721+0000 7ff8c77fe640 1 -- 172.21.15.43:0/8601972 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7ff8c0004e50 con 0x7ff8d01011c0 2024-04-21T08:38:54.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.722+0000 7ff8c77fe640 1 --2- 172.21.15.43:0/8601972 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ff8bc0a3c00 0x7ff8bc0a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.722+0000 7ff8d6428640 1 -- 172.21.15.43:0/8601972 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ff8bc0a3c00 msgr2=0x7ff8bc0a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1577030772 2024-04-21T08:38:54.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.722+0000 7ff8c77fe640 1 -- 172.21.15.43:0/8601972 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7ff8c00d0f20 con 0x7ff8d01011c0 2024-04-21T08:38:54.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.722+0000 7ff8d6428640 1 --2- 172.21.15.43:0/8601972 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ff8bc0a3c00 0x7ff8bc0a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:38:54.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.725+0000 7ff8c77fe640 1 -- 172.21.15.43:0/8601972 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff8c0053ff0 con 0x7ff8d01011c0 2024-04-21T08:38:54.879 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:54 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:54.645+0000 7fa51cf15200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:38:54.879 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:54 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:54.879+0000 7fa51cf15200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:38:54.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.892+0000 7ff8c77fe640 1 -- 172.21.15.43:0/8601972 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7ff8c0099480 con 0x7ff8d01011c0 2024-04-21T08:38:54.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.893+0000 7ff8c77fe640 1 -- 172.21.15.43:0/8601972 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ff8bc0a3c00 msgr2=0x7ff8bc0a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:38:54.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.893+0000 7ff8c77fe640 1 --2- 172.21.15.43:0/8601972 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ff8bc0a3c00 0x7ff8bc0a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.922+0000 7ff8d868c640 1 -- 172.21.15.43:0/8601972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff8d002d7c0 con 0x7ff8d01011c0 2024-04-21T08:38:54.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.922+0000 7ff8c77fe640 1 -- 172.21.15.43:0/8601972 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7ff8c009d510 con 0x7ff8d01011c0 2024-04-21T08:38:54.922 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:38:54.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.924+0000 7ff8d868c640 1 -- 172.21.15.43:0/8601972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d01011c0 msgr2=0x7ff8d019a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.924+0000 7ff8d868c640 1 --2- 172.21.15.43:0/8601972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d01011c0 0x7ff8d019a9a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000e9f0 tx=0x7ff8c000edf0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.924+0000 7ff8d868c640 1 -- 172.21.15.43:0/8601972 shutdown_connections 2024-04-21T08:38:54.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.924+0000 7ff8d868c640 1 --2- 172.21.15.43:0/8601972 >> [v2:172.21.15.43:6800/1577030772,v1:172.21.15.43:6801/1577030772] conn(0x7ff8bc0a3c00 0x7ff8bc0a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.924+0000 7ff8d868c640 1 --2- 172.21.15.43:0/8601972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8d0101b60 0x7ff8d01a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.924+0000 7ff8d868c640 1 --2- 172.21.15.43:0/8601972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d01011c0 0x7ff8d019a9a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.924+0000 7ff8d868c640 1 --2- 172.21.15.43:0/8601972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8d0100860 0x7ff8d019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.924+0000 7ff8d868c640 1 -- 172.21.15.43:0/8601972 >> 172.21.15.43:0/8601972 conn(0x7ff8d00fc400 msgr2=0x7ff8d00fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:54.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.925+0000 7ff8d868c640 1 -- 172.21.15.43:0/8601972 shutdown_connections 2024-04-21T08:38:54.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:54.925+0000 7ff8d868c640 1 -- 172.21.15.43:0/8601972 wait complete. 2024-04-21T08:38:54.936 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:38:55.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.059+0000 7f572b26d640 1 -- 172.21.15.43:0/404369750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5724105b00 msgr2=0x7f5724105f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.059+0000 7f572b26d640 1 --2- 172.21.15.43:0/404369750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5724105b00 0x7f5724105f80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f571c009ca0 tx=0x7f571c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.059+0000 7f572b26d640 1 -- 172.21.15.43:0/404369750 shutdown_connections 2024-04-21T08:38:55.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.059+0000 7f572b26d640 1 --2- 172.21.15.43:0/404369750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5724105b00 0x7f5724105f80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.059+0000 7f572b26d640 1 --2- 172.21.15.43:0/404369750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5724103150 0x7f57241055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.059+0000 7f572b26d640 1 --2- 172.21.15.43:0/404369750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57241007e0 0x7f5724102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.059+0000 7f572b26d640 1 -- 172.21.15.43:0/404369750 >> 172.21.15.43:0/404369750 conn(0x7f57240fc3e0 msgr2=0x7f57240fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:55.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.059+0000 7f572b26d640 1 -- 172.21.15.43:0/404369750 shutdown_connections 2024-04-21T08:38:55.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.059+0000 7f572b26d640 1 -- 172.21.15.43:0/404369750 wait complete. 2024-04-21T08:38:55.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.060+0000 7f572b26d640 1 Processor -- start 2024-04-21T08:38:55.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.060+0000 7f572b26d640 1 -- start start 2024-04-21T08:38:55.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.060+0000 7f572b26d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57241007e0 0x7f572419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:55.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.060+0000 7f572b26d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5724103150 0x7f572419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:55.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.060+0000 7f572b26d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5724105b00 0x7f57241a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:55.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.060+0000 7f572b26d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5724114090 con 0x7f57241007e0 2024-04-21T08:38:55.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.060+0000 7f572b26d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5724113f10 con 0x7f5724103150 2024-04-21T08:38:55.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.060+0000 7f572b26d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5724114210 con 0x7f5724105b00 2024-04-21T08:38:55.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.060+0000 7f572980a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5724105b00 0x7f57241a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:55.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.060+0000 7f5729009640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57241007e0 0x7f572419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:55.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.060+0000 7f572980a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5724105b00 0x7f57241a5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.43:49200/0 (socket says 172.21.15.43:49200) 2024-04-21T08:38:55.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.060+0000 7f572980a640 1 -- 172.21.15.43:0/3652962261 learned_addr learned my addr 172.21.15.43:0/3652962261 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:55.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.060+0000 7f5728808640 1 --2- 172.21.15.43:0/3652962261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5724103150 0x7f572419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:55.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.060+0000 7f572980a640 1 -- 172.21.15.43:0/3652962261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5724103150 msgr2=0x7f572419ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.060+0000 7f572980a640 1 --2- 172.21.15.43:0/3652962261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5724103150 0x7f572419ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.060+0000 7f572980a640 1 -- 172.21.15.43:0/3652962261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57241007e0 msgr2=0x7f572419e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.060+0000 7f572980a640 1 --2- 172.21.15.43:0/3652962261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57241007e0 0x7f572419e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.060+0000 7f572980a640 1 -- 172.21.15.43:0/3652962261 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57241a6310 con 0x7f5724105b00 2024-04-21T08:38:55.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.061+0000 7f572980a640 1 --2- 172.21.15.43:0/3652962261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5724105b00 0x7f57241a5ca0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f571c032520 tx=0x7f571c0347b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:55.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.061+0000 7f571a7fc640 1 -- 172.21.15.43:0/3652962261 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c039040 con 0x7f5724105b00 2024-04-21T08:38:55.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.061+0000 7f572b26d640 1 -- 172.21.15.43:0/3652962261 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57241a65a0 con 0x7f5724105b00 2024-04-21T08:38:55.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.061+0000 7f571a7fc640 1 -- 172.21.15.43:0/3652962261 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f571c044070 con 0x7f5724105b00 2024-04-21T08:38:55.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.061+0000 7f571a7fc640 1 -- 172.21.15.43:0/3652962261 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c0042a0 con 0x7f5724105b00 2024-04-21T08:38:55.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.061+0000 7f572b26d640 1 -- 172.21.15.43:0/3652962261 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57241a6a90 con 0x7f5724105b00 2024-04-21T08:38:55.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.062+0000 7f572b26d640 1 -- 172.21.15.43:0/3652962261 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57240fded0 con 0x7f5724105b00 2024-04-21T08:38:55.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.063+0000 7f571a7fc640 1 -- 172.21.15.43:0/3652962261 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f571c003970 con 0x7f5724105b00 2024-04-21T08:38:55.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.064+0000 7f571a7fc640 1 -- 172.21.15.43:0/3652962261 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f571c0f2590 con 0x7f5724105b00 2024-04-21T08:38:55.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.070+0000 7f571a7fc640 1 -- 172.21.15.43:0/3652962261 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f571c035d30 con 0x7f5724105b00 2024-04-21T08:38:55.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.301+0000 7f572b26d640 1 -- 172.21.15.43:0/3652962261 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5724062b50 con 0x7f5724105b00 2024-04-21T08:38:55.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.303+0000 7f571a7fc640 1 -- 172.21.15.43:0/3652962261 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+25233 (secure 0 0 0) 0x7f571c0bee20 con 0x7f5724105b00 2024-04-21T08:38:55.302 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:55.303 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":95,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:38:54.881943+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/126475132":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/4288951239":"2024-04-22T08:38:54.881882+0000","172.21.15.43:6801/1577030772":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/2382670190":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/1174411284":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/1577030772":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:38:55.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.305+0000 7f572b26d640 1 -- 172.21.15.43:0/3652962261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5724105b00 msgr2=0x7f57241a5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.305+0000 7f572b26d640 1 --2- 172.21.15.43:0/3652962261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5724105b00 0x7f57241a5ca0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f571c032520 tx=0x7f571c0347b0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.305+0000 7f572b26d640 1 -- 172.21.15.43:0/3652962261 shutdown_connections 2024-04-21T08:38:55.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.305+0000 7f572b26d640 1 --2- 172.21.15.43:0/3652962261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5724105b00 0x7f57241a5ca0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.305+0000 7f572b26d640 1 --2- 172.21.15.43:0/3652962261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5724103150 0x7f572419ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.305+0000 7f572b26d640 1 --2- 172.21.15.43:0/3652962261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57241007e0 0x7f572419e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.305+0000 7f572b26d640 1 -- 172.21.15.43:0/3652962261 >> 172.21.15.43:0/3652962261 conn(0x7f57240fc3e0 msgr2=0x7f5724101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:55.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.305+0000 7f572b26d640 1 -- 172.21.15.43:0/3652962261 shutdown_connections 2024-04-21T08:38:55.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.305+0000 7f572b26d640 1 -- 172.21.15.43:0/3652962261 wait complete. 2024-04-21T08:38:55.316 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:39.981405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:22.034751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:22.780795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:25.984261+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:38:55.316 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:38:55.366 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-21T08:38:55.366 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/393926609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:55.367 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: Active manager daemon x restarted 2024-04-21T08:38:55.367 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: Activating manager daemon x 2024-04-21T08:38:55.367 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: osdmap e95: 12 total, 12 up, 12 in 2024-04-21T08:38:55.367 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: mgrmap e21: x(active, starting, since 0.0126942s), standbys: z, y 2024-04-21T08:38:55.367 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:38:55.367 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:55.367 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:38:55.367 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T08:38:55.367 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T08:38:55.367 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:38:55.367 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:38:55.367 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:38:55.367 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:38:55.368 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:38:55.368 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:38:55.368 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:38:55.368 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:38:55.368 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:38:55.368 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:38:55.368 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T08:38:55.368 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T08:38:55.368 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T08:38:55.368 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:38:55.368 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:38:55.368 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:38:55.368 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:38:55.368 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:38:55.368 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:38:55.369 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:38:55.369 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:38:55.369 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:38:55.369 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:38:55.369 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:38:55.369 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:38:55.369 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T08:38:55.369 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T08:38:55.369 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T08:38:55.369 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: Manager daemon x is now available 2024-04-21T08:38:55.369 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/8601972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:55.369 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:55.369 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T08:38:55.370 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:55 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T08:38:55.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.439+0000 7f1bb0a48640 1 -- 172.21.15.43:0/1272776736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bac107d10 msgr2=0x7f1bac10a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.439+0000 7f1bb0a48640 1 --2- 172.21.15.43:0/1272776736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bac107d10 0x7f1bac10a100 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009ca0 tx=0x7f1ba402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.439+0000 7f1bb0a48640 1 -- 172.21.15.43:0/1272776736 shutdown_connections 2024-04-21T08:38:55.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.439+0000 7f1bb0a48640 1 --2- 172.21.15.43:0/1272776736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bac107d10 0x7f1bac10a100 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.439+0000 7f1bb0a48640 1 --2- 172.21.15.43:0/1272776736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bac1053e0 0x7f1bac1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.439+0000 7f1bb0a48640 1 --2- 172.21.15.43:0/1272776736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bac102ab0 0x7f1bac104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.439+0000 7f1bb0a48640 1 -- 172.21.15.43:0/1272776736 >> 172.21.15.43:0/1272776736 conn(0x7f1bac0fc400 msgr2=0x7f1bac0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:55.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.439+0000 7f1bb0a48640 1 -- 172.21.15.43:0/1272776736 shutdown_connections 2024-04-21T08:38:55.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.439+0000 7f1bb0a48640 1 -- 172.21.15.43:0/1272776736 wait complete. 2024-04-21T08:38:55.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.440+0000 7f1bb0a48640 1 Processor -- start 2024-04-21T08:38:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.440+0000 7f1bb0a48640 1 -- start start 2024-04-21T08:38:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.440+0000 7f1bb0a48640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bac102ab0 0x7f1bac1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.440+0000 7f1bb0a48640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bac1053e0 0x7f1bac198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.440+0000 7f1bb0a48640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bac107d10 0x7f1bac19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.440+0000 7f1bb0a48640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac10daf0 con 0x7f1bac1053e0 2024-04-21T08:38:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.440+0000 7f1bb0a48640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac10d970 con 0x7f1bac107d10 2024-04-21T08:38:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.441+0000 7f1bb0a48640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac10dc70 con 0x7f1bac102ab0 2024-04-21T08:38:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.441+0000 7f1ba9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bac1053e0 0x7f1bac198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.441+0000 7f1ba9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bac1053e0 0x7f1bac198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60414/0 (socket says 172.21.15.43:60414) 2024-04-21T08:38:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.441+0000 7f1baa59c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bac102ab0 0x7f1bac1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.441+0000 7f1ba9d9b640 1 -- 172.21.15.43:0/2526325762 learned_addr learned my addr 172.21.15.43:0/2526325762 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.441+0000 7f1baad9d640 1 --2- 172.21.15.43:0/2526325762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bac107d10 0x7f1bac19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:55.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.441+0000 7f1ba9d9b640 1 -- 172.21.15.43:0/2526325762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bac102ab0 msgr2=0x7f1bac1981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.441+0000 7f1ba9d9b640 1 --2- 172.21.15.43:0/2526325762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bac102ab0 0x7f1bac1981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.441+0000 7f1ba9d9b640 1 -- 172.21.15.43:0/2526325762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bac107d10 msgr2=0x7f1bac19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.441+0000 7f1ba9d9b640 1 --2- 172.21.15.43:0/2526325762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bac107d10 0x7f1bac19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.441+0000 7f1ba9d9b640 1 -- 172.21.15.43:0/2526325762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bac19fde0 con 0x7f1bac1053e0 2024-04-21T08:38:55.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.441+0000 7f1ba9d9b640 1 --2- 172.21.15.43:0/2526325762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bac1053e0 0x7f1bac198700 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000e9a0 tx=0x7f1ba000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:55.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.441+0000 7f1b9b7fe640 1 -- 172.21.15.43:0/2526325762 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba000cd80 con 0x7f1bac1053e0 2024-04-21T08:38:55.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.441+0000 7f1bb0a48640 1 -- 172.21.15.43:0/2526325762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bac1a00d0 con 0x7f1bac1053e0 2024-04-21T08:38:55.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.441+0000 7f1b9b7fe640 1 -- 172.21.15.43:0/2526325762 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ba00035f0 con 0x7f1bac1053e0 2024-04-21T08:38:55.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.441+0000 7f1b9b7fe640 1 -- 172.21.15.43:0/2526325762 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0010690 con 0x7f1bac1053e0 2024-04-21T08:38:55.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.441+0000 7f1bb0a48640 1 -- 172.21.15.43:0/2526325762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bac1a0620 con 0x7f1bac1053e0 2024-04-21T08:38:55.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.443+0000 7f1bb0a48640 1 -- 172.21.15.43:0/2526325762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bac0fdef0 con 0x7f1bac1053e0 2024-04-21T08:38:55.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.444+0000 7f1b9b7fe640 1 -- 172.21.15.43:0/2526325762 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f1ba0003fb0 con 0x7f1bac1053e0 2024-04-21T08:38:55.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.444+0000 7f1b9b7fe640 1 -- 172.21.15.43:0/2526325762 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f1ba00d0560 con 0x7f1bac1053e0 2024-04-21T08:38:55.446 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-21T08:38:55.446 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/393926609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:55.446 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: Active manager daemon x restarted 2024-04-21T08:38:55.446 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: Activating manager daemon x 2024-04-21T08:38:55.446 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: osdmap e95: 12 total, 12 up, 12 in 2024-04-21T08:38:55.446 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: mgrmap e21: x(active, starting, since 0.0126942s), standbys: z, y 2024-04-21T08:38:55.447 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:38:55.447 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:55.447 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:38:55.447 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T08:38:55.447 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T08:38:55.447 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:38:55.447 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:38:55.447 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:38:55.447 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:38:55.447 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:38:55.447 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:38:55.448 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:38:55.448 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:38:55.448 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:38:55.448 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:38:55.448 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T08:38:55.448 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T08:38:55.448 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T08:38:55.448 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:38:55.448 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:38:55.448 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:38:55.448 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:38:55.448 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:38:55.448 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:38:55.448 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:38:55.448 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:38:55.449 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:38:55.449 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:38:55.449 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:38:55.449 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:38:55.449 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T08:38:55.449 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T08:38:55.449 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T08:38:55.449 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: Manager daemon x is now available 2024-04-21T08:38:55.449 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/8601972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:55.449 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:55.449 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T08:38:55.449 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:55 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T08:38:55.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.449+0000 7f1b9b7fe640 1 -- 172.21.15.43:0/2526325762 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ba009ce50 con 0x7f1bac1053e0 2024-04-21T08:38:55.562 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:55.169+0000 7fa47e84d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:38:55.562 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:55.169+0000 7fa47e84d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:38:55.562 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:55.169+0000 7fa47e84d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:38:55.562 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:55.169+0000 7fa47e84d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:38:55.562 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:55.169+0000 7fa47e84d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:38:55.562 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:55.170+0000 7fa47c809700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:38:55.562 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:55.170+0000 7fa47c809700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:38:55.562 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:55.170+0000 7fa47c809700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:38:55.563 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:55.170+0000 7fa47c809700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:38:55.563 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:38:55.170+0000 7fa47c809700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:38:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-21T08:38:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/393926609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: Active manager daemon x restarted 2024-04-21T08:38:55.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: Activating manager daemon x 2024-04-21T08:38:55.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: osdmap e95: 12 total, 12 up, 12 in 2024-04-21T08:38:55.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: mgrmap e21: x(active, starting, since 0.0126942s), standbys: z, y 2024-04-21T08:38:55.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:38:55.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:55.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:38:55.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T08:38:55.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T08:38:55.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:38:55.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:38:55.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:38:55.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:38:55.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:38:55.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:38:55.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:38:55.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:38:55.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:38:55.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:38:55.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T08:38:55.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T08:38:55.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T08:38:55.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:38:55.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:38:55.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:38:55.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:38:55.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:38:55.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:38:55.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:38:55.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:38:55.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:38:55.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:38:55.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:38:55.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:38:55.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T08:38:55.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T08:38:55.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T08:38:55.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: Manager daemon x is now available 2024-04-21T08:38:55.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/8601972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:55.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:55.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T08:38:55.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:55 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T08:38:55.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.649+0000 7f1bb0a48640 1 -- 172.21.15.43:0/2526325762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1bac062b50 con 0x7f1bac1053e0 2024-04-21T08:38:55.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.649+0000 7f1b9b7fe640 1 -- 172.21.15.43:0/2526325762 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f1ba009c750 con 0x7f1bac1053e0 2024-04-21T08:38:55.649 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-21T08:38:55.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.651+0000 7f1bb0a48640 1 -- 172.21.15.43:0/2526325762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bac1053e0 msgr2=0x7f1bac198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.651+0000 7f1bb0a48640 1 --2- 172.21.15.43:0/2526325762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bac1053e0 0x7f1bac198700 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000e9a0 tx=0x7f1ba000ee60 comp rx=0 tx=0).stop 2024-04-21T08:38:55.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.651+0000 7f1bb0a48640 1 -- 172.21.15.43:0/2526325762 shutdown_connections 2024-04-21T08:38:55.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.651+0000 7f1bb0a48640 1 --2- 172.21.15.43:0/2526325762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bac107d10 0x7f1bac19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.652+0000 7f1bb0a48640 1 --2- 172.21.15.43:0/2526325762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bac1053e0 0x7f1bac198700 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.652+0000 7f1bb0a48640 1 --2- 172.21.15.43:0/2526325762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bac102ab0 0x7f1bac1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.652+0000 7f1bb0a48640 1 -- 172.21.15.43:0/2526325762 >> 172.21.15.43:0/2526325762 conn(0x7f1bac0fc400 msgr2=0x7f1bac103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:55.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.652+0000 7f1bb0a48640 1 -- 172.21.15.43:0/2526325762 shutdown_connections 2024-04-21T08:38:55.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.652+0000 7f1bb0a48640 1 -- 172.21.15.43:0/2526325762 wait complete. 2024-04-21T08:38:55.663 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:38:55.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.771+0000 7f2258f7e640 1 -- 172.21.15.43:0/3363886715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2254101fe0 msgr2=0x7f2254102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.771+0000 7f2258f7e640 1 --2- 172.21.15.43:0/3363886715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2254101fe0 0x7f2254102460 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f223c009900 tx=0x7f223c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:55.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.771+0000 7f2258f7e640 1 -- 172.21.15.43:0/3363886715 shutdown_connections 2024-04-21T08:38:55.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.771+0000 7f2258f7e640 1 --2- 172.21.15.43:0/3363886715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22541029a0 0x7f22541092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.771+0000 7f2258f7e640 1 --2- 172.21.15.43:0/3363886715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2254101fe0 0x7f2254102460 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.771+0000 7f2258f7e640 1 --2- 172.21.15.43:0/3363886715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2254100e90 0x7f22541012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.771+0000 7f2258f7e640 1 -- 172.21.15.43:0/3363886715 >> 172.21.15.43:0/3363886715 conn(0x7f22540fc400 msgr2=0x7f22540fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:55.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.771+0000 7f2258f7e640 1 -- 172.21.15.43:0/3363886715 shutdown_connections 2024-04-21T08:38:55.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.771+0000 7f2258f7e640 1 -- 172.21.15.43:0/3363886715 wait complete. 2024-04-21T08:38:55.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.772+0000 7f2258f7e640 1 Processor -- start 2024-04-21T08:38:55.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.772+0000 7f2258f7e640 1 -- start start 2024-04-21T08:38:55.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.772+0000 7f2258f7e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2254100e90 0x7f2254069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:55.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.772+0000 7f2258f7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2254101fe0 0x7f225419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:55.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.772+0000 7f2258f7e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22541029a0 0x7f225419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:55.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.772+0000 7f2258f7e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f225410c9f0 con 0x7f22541029a0 2024-04-21T08:38:55.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.772+0000 7f2258f7e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f225410c870 con 0x7f2254101fe0 2024-04-21T08:38:55.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.772+0000 7f2258f7e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f225410cb70 con 0x7f2254100e90 2024-04-21T08:38:55.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.772+0000 7f2252d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22541029a0 0x7f225419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:55.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.772+0000 7f2252d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22541029a0 0x7f225419f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60436/0 (socket says 172.21.15.43:60436) 2024-04-21T08:38:55.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.772+0000 7f2251d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2254101fe0 0x7f225419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:55.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.772+0000 7f2252d9d640 1 -- 172.21.15.43:0/3969303431 learned_addr learned my addr 172.21.15.43:0/3969303431 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:55.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.772+0000 7f225259c640 1 --2- 172.21.15.43:0/3969303431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2254100e90 0x7f2254069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:55.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.772+0000 7f2252d9d640 1 -- 172.21.15.43:0/3969303431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2254100e90 msgr2=0x7f2254069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.772+0000 7f2252d9d640 1 --2- 172.21.15.43:0/3969303431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2254100e90 0x7f2254069cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.772+0000 7f2252d9d640 1 -- 172.21.15.43:0/3969303431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2254101fe0 msgr2=0x7f225419cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.772+0000 7f2252d9d640 1 --2- 172.21.15.43:0/3969303431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2254101fe0 0x7f225419cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.772+0000 7f2252d9d640 1 -- 172.21.15.43:0/3969303431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225419fea0 con 0x7f22541029a0 2024-04-21T08:38:55.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.772+0000 7f2252d9d640 1 --2- 172.21.15.43:0/3969303431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22541029a0 0x7f225419f5e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f224c009fd0 tx=0x7f224c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:55.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.773+0000 7f224b7fe640 1 -- 172.21.15.43:0/3969303431 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c017070 con 0x7f22541029a0 2024-04-21T08:38:55.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.773+0000 7f224b7fe640 1 -- 172.21.15.43:0/3969303431 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f224c003e20 con 0x7f22541029a0 2024-04-21T08:38:55.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.773+0000 7f2258f7e640 1 -- 172.21.15.43:0/3969303431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22541a0190 con 0x7f22541029a0 2024-04-21T08:38:55.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.773+0000 7f224b7fe640 1 -- 172.21.15.43:0/3969303431 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c010460 con 0x7f22541029a0 2024-04-21T08:38:55.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.773+0000 7f2258f7e640 1 -- 172.21.15.43:0/3969303431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22541a0660 con 0x7f22541029a0 2024-04-21T08:38:55.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.774+0000 7f2258f7e640 1 -- 172.21.15.43:0/3969303431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2254108310 con 0x7f22541029a0 2024-04-21T08:38:55.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.775+0000 7f224b7fe640 1 -- 172.21.15.43:0/3969303431 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f224c003970 con 0x7f22541029a0 2024-04-21T08:38:55.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.776+0000 7f224b7fe640 1 -- 172.21.15.43:0/3969303431 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f224c0d0060 con 0x7f22541029a0 2024-04-21T08:38:55.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.780+0000 7f224b7fe640 1 -- 172.21.15.43:0/3969303431 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f224c09c300 con 0x7f22541029a0 2024-04-21T08:38:55.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.902+0000 7f224b7fe640 1 -- 172.21.15.43:0/3969303431 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f224c098050 con 0x7f22541029a0 2024-04-21T08:38:55.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.902+0000 7f224b7fe640 1 --2- 172.21.15.43:0/3969303431 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f224001db50 0x7f22400ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:55.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.906+0000 7f225259c640 1 --2- 172.21.15.43:0/3969303431 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f224001db50 0x7f22400ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:55.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.910+0000 7f225259c640 1 --2- 172.21.15.43:0/3969303431 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f224001db50 0x7f22400ae150 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2244005fd0 tx=0x7f224400b390 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:55.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.991+0000 7f2258f7e640 1 -- 172.21.15.43:0/3969303431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2254062b50 con 0x7f22541029a0 2024-04-21T08:38:55.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.992+0000 7f224b7fe640 1 -- 172.21.15.43:0/3969303431 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7f224c056a10 con 0x7f22541029a0 2024-04-21T08:38:55.991 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:38:55.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.994+0000 7f2258f7e640 1 -- 172.21.15.43:0/3969303431 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f224001db50 msgr2=0x7f22400ae150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.994+0000 7f2258f7e640 1 --2- 172.21.15.43:0/3969303431 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f224001db50 0x7f22400ae150 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2244005fd0 tx=0x7f224400b390 comp rx=0 tx=0).stop 2024-04-21T08:38:55.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.994+0000 7f2258f7e640 1 -- 172.21.15.43:0/3969303431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22541029a0 msgr2=0x7f225419f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.994+0000 7f2258f7e640 1 --2- 172.21.15.43:0/3969303431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22541029a0 0x7f225419f5e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f224c009fd0 tx=0x7f224c00b260 comp rx=0 tx=0).stop 2024-04-21T08:38:55.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.994+0000 7f2258f7e640 1 -- 172.21.15.43:0/3969303431 shutdown_connections 2024-04-21T08:38:55.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.994+0000 7f2258f7e640 1 --2- 172.21.15.43:0/3969303431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22541029a0 0x7f225419f5e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.994+0000 7f2258f7e640 1 --2- 172.21.15.43:0/3969303431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2254101fe0 0x7f225419cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.994+0000 7f2258f7e640 1 --2- 172.21.15.43:0/3969303431 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f224001db50 0x7f22400ae150 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.994+0000 7f2258f7e640 1 --2- 172.21.15.43:0/3969303431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2254100e90 0x7f2254069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.994+0000 7f2258f7e640 1 -- 172.21.15.43:0/3969303431 >> 172.21.15.43:0/3969303431 conn(0x7f22540fc400 msgr2=0x7f22541054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:55.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.994+0000 7f2258f7e640 1 -- 172.21.15.43:0/3969303431 shutdown_connections 2024-04-21T08:38:55.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:55.994+0000 7f2258f7e640 1 -- 172.21.15.43:0/3969303431 wait complete. 2024-04-21T08:38:56.006 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:38:56.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.117+0000 7fe1eb954640 1 -- 172.21.15.43:0/4154599839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e4103b50 msgr2=0x7fe1e410a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.117+0000 7fe1eb954640 1 --2- 172.21.15.43:0/4154599839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e4103b50 0x7fe1e410a3e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009900 tx=0x7fe1dc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:56.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.117+0000 7fe1eb954640 1 -- 172.21.15.43:0/4154599839 shutdown_connections 2024-04-21T08:38:56.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.117+0000 7fe1eb954640 1 --2- 172.21.15.43:0/4154599839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e4103b50 0x7fe1e410a3e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.117+0000 7fe1eb954640 1 --2- 172.21.15.43:0/4154599839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1e4103190 0x7fe1e4103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.117+0000 7fe1eb954640 1 --2- 172.21.15.43:0/4154599839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1e4101fa0 0x7fe1e41023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.117+0000 7fe1eb954640 1 -- 172.21.15.43:0/4154599839 >> 172.21.15.43:0/4154599839 conn(0x7fe1e40fd530 msgr2=0x7fe1e40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:56.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.117+0000 7fe1eb954640 1 -- 172.21.15.43:0/4154599839 shutdown_connections 2024-04-21T08:38:56.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.117+0000 7fe1eb954640 1 -- 172.21.15.43:0/4154599839 wait complete. 2024-04-21T08:38:56.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.118+0000 7fe1eb954640 1 Processor -- start 2024-04-21T08:38:56.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.118+0000 7fe1eb954640 1 -- start start 2024-04-21T08:38:56.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.118+0000 7fe1eb954640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e4101fa0 0x7fe1e4198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:56.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.118+0000 7fe1eb954640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1e4103190 0x7fe1e41986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:56.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.118+0000 7fe1eb954640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1e4103b50 0x7fe1e419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:56.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.118+0000 7fe1eb954640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e410dae0 con 0x7fe1e4101fa0 2024-04-21T08:38:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.118+0000 7fe1eb954640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e410d960 con 0x7fe1e4103190 2024-04-21T08:38:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.118+0000 7fe1eb954640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e410dc60 con 0x7fe1e4103b50 2024-04-21T08:38:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.118+0000 7fe1e96f0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e4101fa0 0x7fe1e4198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.118+0000 7fe1e96f0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e4101fa0 0x7fe1e4198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60460/0 (socket says 172.21.15.43:60460) 2024-04-21T08:38:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.118+0000 7fe1e96f0640 1 -- 172.21.15.43:0/1387283183 learned_addr learned my addr 172.21.15.43:0/1387283183 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.118+0000 7fe1e9ef1640 1 --2- 172.21.15.43:0/1387283183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1e4103b50 0x7fe1e419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.118+0000 7fe1e8eef640 1 --2- 172.21.15.43:0/1387283183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1e4103190 0x7fe1e41986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.118+0000 7fe1e96f0640 1 -- 172.21.15.43:0/1387283183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1e4103b50 msgr2=0x7fe1e419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.118+0000 7fe1e96f0640 1 --2- 172.21.15.43:0/1387283183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1e4103b50 0x7fe1e419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.118+0000 7fe1e96f0640 1 -- 172.21.15.43:0/1387283183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1e4103190 msgr2=0x7fe1e41986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.118+0000 7fe1e96f0640 1 --2- 172.21.15.43:0/1387283183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1e4103190 0x7fe1e41986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.118+0000 7fe1e96f0640 1 -- 172.21.15.43:0/1387283183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e419fd80 con 0x7fe1e4101fa0 2024-04-21T08:38:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.118+0000 7fe1e96f0640 1 --2- 172.21.15.43:0/1387283183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e4101fa0 0x7fe1e4198160 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400ebc0 tx=0x7fe1d400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.119+0000 7fe1e27fc640 1 -- 172.21.15.43:0/1387283183 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4003b70 con 0x7fe1e4101fa0 2024-04-21T08:38:56.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.119+0000 7fe1e27fc640 1 -- 172.21.15.43:0/1387283183 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1d4003d10 con 0x7fe1e4101fa0 2024-04-21T08:38:56.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.119+0000 7fe1eb954640 1 -- 172.21.15.43:0/1387283183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e41a0070 con 0x7fe1e4101fa0 2024-04-21T08:38:56.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.119+0000 7fe1e27fc640 1 -- 172.21.15.43:0/1387283183 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4019800 con 0x7fe1e4101fa0 2024-04-21T08:38:56.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.119+0000 7fe1eb954640 1 -- 172.21.15.43:0/1387283183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e41a05c0 con 0x7fe1e4101fa0 2024-04-21T08:38:56.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.120+0000 7fe1eb954640 1 -- 172.21.15.43:0/1387283183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1e4109340 con 0x7fe1e4101fa0 2024-04-21T08:38:56.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.121+0000 7fe1e27fc640 1 -- 172.21.15.43:0/1387283183 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fe1d40199a0 con 0x7fe1e4101fa0 2024-04-21T08:38:56.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.122+0000 7fe1e27fc640 1 --2- 172.21.15.43:0/1387283183 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe1cc0a39b0 0x7fe1cc0a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:56.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.122+0000 7fe1e8eef640 1 --2- 172.21.15.43:0/1387283183 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe1cc0a39b0 0x7fe1cc0a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:56.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.122+0000 7fe1e27fc640 1 -- 172.21.15.43:0/1387283183 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fe1d40d0e60 con 0x7fe1e4101fa0 2024-04-21T08:38:56.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.122+0000 7fe1e8eef640 1 --2- 172.21.15.43:0/1387283183 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe1cc0a39b0 0x7fe1cc0a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4199670 tx=0x7fe1d8005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:56.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.126+0000 7fe1e27fc640 1 -- 172.21.15.43:0/1387283183 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe1d409d600 con 0x7fe1e4101fa0 2024-04-21T08:38:56.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.316+0000 7fe1eb954640 1 -- 172.21.15.43:0/1387283183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe1e4062b50 con 0x7fe1e4101fa0 2024-04-21T08:38:56.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.317+0000 7fe1e27fc640 1 -- 172.21.15.43:0/1387283183 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7fe1d409cf00 con 0x7fe1e4101fa0 2024-04-21T08:38:56.316 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-21T08:38:56.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.319+0000 7fe1eb954640 1 -- 172.21.15.43:0/1387283183 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe1cc0a39b0 msgr2=0x7fe1cc0a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.319+0000 7fe1eb954640 1 --2- 172.21.15.43:0/1387283183 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe1cc0a39b0 0x7fe1cc0a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4199670 tx=0x7fe1d8005f90 comp rx=0 tx=0).stop 2024-04-21T08:38:56.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.319+0000 7fe1eb954640 1 -- 172.21.15.43:0/1387283183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e4101fa0 msgr2=0x7fe1e4198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.319+0000 7fe1eb954640 1 --2- 172.21.15.43:0/1387283183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e4101fa0 0x7fe1e4198160 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400ebc0 tx=0x7fe1d400c6a0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.319+0000 7fe1eb954640 1 -- 172.21.15.43:0/1387283183 shutdown_connections 2024-04-21T08:38:56.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.319+0000 7fe1eb954640 1 --2- 172.21.15.43:0/1387283183 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe1cc0a39b0 0x7fe1cc0a5e70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.319+0000 7fe1eb954640 1 --2- 172.21.15.43:0/1387283183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1e4103b50 0x7fe1e419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.319+0000 7fe1eb954640 1 --2- 172.21.15.43:0/1387283183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1e4103190 0x7fe1e41986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.319+0000 7fe1eb954640 1 --2- 172.21.15.43:0/1387283183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e4101fa0 0x7fe1e4198160 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.319+0000 7fe1eb954640 1 -- 172.21.15.43:0/1387283183 >> 172.21.15.43:0/1387283183 conn(0x7fe1e40fd530 msgr2=0x7fe1e4106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:56.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.320+0000 7fe1eb954640 1 -- 172.21.15.43:0/1387283183 shutdown_connections 2024-04-21T08:38:56.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.320+0000 7fe1eb954640 1 -- 172.21.15.43:0/1387283183 wait complete. 2024-04-21T08:38:56.330 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:38:56.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.441+0000 7f00391a4640 1 -- 172.21.15.43:0/2328865974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0034102040 msgr2=0x7f00341024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.441+0000 7f00391a4640 1 --2- 172.21.15.43:0/2328865974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0034102040 0x7f00341024c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f001c009900 tx=0x7f001c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:56.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.441+0000 7f00391a4640 1 -- 172.21.15.43:0/2328865974 shutdown_connections 2024-04-21T08:38:56.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.441+0000 7f00391a4640 1 --2- 172.21.15.43:0/2328865974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0034102a00 0x7f0034109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.441+0000 7f00391a4640 1 --2- 172.21.15.43:0/2328865974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0034102040 0x7f00341024c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.441+0000 7f00391a4640 1 --2- 172.21.15.43:0/2328865974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0034100e60 0x7f0034101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.441+0000 7f00391a4640 1 -- 172.21.15.43:0/2328865974 >> 172.21.15.43:0/2328865974 conn(0x7f00340fc440 msgr2=0x7f00340fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:56.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.441+0000 7f00391a4640 1 -- 172.21.15.43:0/2328865974 shutdown_connections 2024-04-21T08:38:56.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.441+0000 7f00391a4640 1 -- 172.21.15.43:0/2328865974 wait complete. 2024-04-21T08:38:56.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.442+0000 7f00391a4640 1 Processor -- start 2024-04-21T08:38:56.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.442+0000 7f00391a4640 1 -- start start 2024-04-21T08:38:56.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.442+0000 7f00391a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0034100e60 0x7f00341981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:56.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.442+0000 7f00391a4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0034102040 0x7f0034198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:56.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.442+0000 7f00391a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0034102a00 0x7f003419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:56.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.442+0000 7f00391a4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f003410db30 con 0x7f0034100e60 2024-04-21T08:38:56.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.442+0000 7f00391a4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f003410d9b0 con 0x7f0034102a00 2024-04-21T08:38:56.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.442+0000 7f00391a4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f003410dcb0 con 0x7f0034102040 2024-04-21T08:38:56.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.443+0000 7f0032d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0034100e60 0x7f00341981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:56.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.443+0000 7f003259c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0034102040 0x7f0034198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:56.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.443+0000 7f0032d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0034100e60 0x7f00341981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60474/0 (socket says 172.21.15.43:60474) 2024-04-21T08:38:56.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.443+0000 7f0032d9d640 1 -- 172.21.15.43:0/3411229034 learned_addr learned my addr 172.21.15.43:0/3411229034 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:56.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.443+0000 7f003359e640 1 --2- 172.21.15.43:0/3411229034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0034102a00 0x7f003419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:56.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.443+0000 7f0032d9d640 1 -- 172.21.15.43:0/3411229034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0034102040 msgr2=0x7f0034198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.443+0000 7f0032d9d640 1 --2- 172.21.15.43:0/3411229034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0034102040 0x7f0034198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.443+0000 7f0032d9d640 1 -- 172.21.15.43:0/3411229034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0034102a00 msgr2=0x7f003419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.443+0000 7f0032d9d640 1 --2- 172.21.15.43:0/3411229034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0034102a00 0x7f003419f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.443+0000 7f0032d9d640 1 -- 172.21.15.43:0/3411229034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003419fde0 con 0x7f0034100e60 2024-04-21T08:38:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.443+0000 7f0032d9d640 1 --2- 172.21.15.43:0/3411229034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0034100e60 0x7f00341981c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f002400eb70 tx=0x7f002400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.443+0000 7f002bfff640 1 -- 172.21.15.43:0/3411229034 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024003b70 con 0x7f0034100e60 2024-04-21T08:38:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.443+0000 7f002bfff640 1 -- 172.21.15.43:0/3411229034 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f002400ee70 con 0x7f0034100e60 2024-04-21T08:38:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.443+0000 7f00391a4640 1 -- 172.21.15.43:0/3411229034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00341a00d0 con 0x7f0034100e60 2024-04-21T08:38:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.443+0000 7f002bfff640 1 -- 172.21.15.43:0/3411229034 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024010650 con 0x7f0034100e60 2024-04-21T08:38:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.443+0000 7f00391a4640 1 -- 172.21.15.43:0/3411229034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00341a05a0 con 0x7f0034100e60 2024-04-21T08:38:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.444+0000 7f00391a4640 1 -- 172.21.15.43:0/3411229034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0034108280 con 0x7f0034100e60 2024-04-21T08:38:56.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.445+0000 7f002bfff640 1 -- 172.21.15.43:0/3411229034 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f0024004020 con 0x7f0034100e60 2024-04-21T08:38:56.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.446+0000 7f002bfff640 1 --2- 172.21.15.43:0/3411229034 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f00200a39b0 0x7f00200a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:56.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.446+0000 7f003259c640 1 --2- 172.21.15.43:0/3411229034 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f00200a39b0 0x7f00200a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:56.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.446+0000 7f002bfff640 1 -- 172.21.15.43:0/3411229034 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f00240d0b90 con 0x7f0034100e60 2024-04-21T08:38:56.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.447+0000 7f003259c640 1 --2- 172.21.15.43:0/3411229034 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f00200a39b0 0x7f00200a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f001c009230 tx=0x7f001c005fd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:56.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.451+0000 7f002bfff640 1 -- 172.21.15.43:0/3411229034 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f002409d330 con 0x7f0034100e60 2024-04-21T08:38:56.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:56 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3652962261' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:56.606 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:56 smithi112 ceph-mon[31085]: [21/Apr/2024:08:38:55] ENGINE Bus STARTING 2024-04-21T08:38:56.606 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:56 smithi112 ceph-mon[31085]: [21/Apr/2024:08:38:55] ENGINE Serving on http://172.21.15.43:8765 2024-04-21T08:38:56.606 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:56 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2526325762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:56.606 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:56 smithi112 ceph-mon[31085]: mgrmap e22: x(active, since 1.02147s), standbys: z, y 2024-04-21T08:38:56.606 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:56 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3969303431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:56.607 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:56 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3652962261' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:56.607 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:56 smithi043 ceph-mon[27978]: [21/Apr/2024:08:38:55] ENGINE Bus STARTING 2024-04-21T08:38:56.607 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:56 smithi043 ceph-mon[27978]: [21/Apr/2024:08:38:55] ENGINE Serving on http://172.21.15.43:8765 2024-04-21T08:38:56.607 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:56 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2526325762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:56.607 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:56 smithi043 ceph-mon[27978]: mgrmap e22: x(active, since 1.02147s), standbys: z, y 2024-04-21T08:38:56.607 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:56 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3969303431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:56.622 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:56 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3652962261' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:56.622 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:56 smithi175 ceph-mon[31920]: [21/Apr/2024:08:38:55] ENGINE Bus STARTING 2024-04-21T08:38:56.622 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:56 smithi175 ceph-mon[31920]: [21/Apr/2024:08:38:55] ENGINE Serving on http://172.21.15.43:8765 2024-04-21T08:38:56.622 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:56 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2526325762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:56.622 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:56 smithi175 ceph-mon[31920]: mgrmap e22: x(active, since 1.02147s), standbys: z, y 2024-04-21T08:38:56.622 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:56 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3969303431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:56.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.638+0000 7f00391a4640 1 -- 172.21.15.43:0/3411229034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0034062b50 con 0x7f0034100e60 2024-04-21T08:38:56.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.638+0000 7f002bfff640 1 -- 172.21.15.43:0/3411229034 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7f002409cc30 con 0x7f0034100e60 2024-04-21T08:38:56.637 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:38:56.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.640+0000 7f00391a4640 1 -- 172.21.15.43:0/3411229034 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f00200a39b0 msgr2=0x7f00200a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.640+0000 7f00391a4640 1 --2- 172.21.15.43:0/3411229034 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f00200a39b0 0x7f00200a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f001c009230 tx=0x7f001c005fd0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.640+0000 7f00391a4640 1 -- 172.21.15.43:0/3411229034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0034100e60 msgr2=0x7f00341981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.640+0000 7f00391a4640 1 --2- 172.21.15.43:0/3411229034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0034100e60 0x7f00341981c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f002400eb70 tx=0x7f002400c6a0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.640+0000 7f00391a4640 1 -- 172.21.15.43:0/3411229034 shutdown_connections 2024-04-21T08:38:56.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.640+0000 7f00391a4640 1 --2- 172.21.15.43:0/3411229034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0034102a00 0x7f003419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.640+0000 7f00391a4640 1 --2- 172.21.15.43:0/3411229034 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f00200a39b0 0x7f00200a5e70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.640+0000 7f00391a4640 1 --2- 172.21.15.43:0/3411229034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0034102040 0x7f0034198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.640+0000 7f00391a4640 1 --2- 172.21.15.43:0/3411229034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0034100e60 0x7f00341981c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.640+0000 7f00391a4640 1 -- 172.21.15.43:0/3411229034 >> 172.21.15.43:0/3411229034 conn(0x7f00340fc440 msgr2=0x7f0034105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:56.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.640+0000 7f00391a4640 1 -- 172.21.15.43:0/3411229034 shutdown_connections 2024-04-21T08:38:56.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.640+0000 7f00391a4640 1 -- 172.21.15.43:0/3411229034 wait complete. 2024-04-21T08:38:56.650 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-21T08:38:56.650 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:38:56.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.755+0000 7f0e57552640 1 -- 172.21.15.43:0/1863999982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e50103180 msgr2=0x7f0e50103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.755+0000 7f0e57552640 1 --2- 172.21.15.43:0/1863999982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e50103180 0x7f0e50103600 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0e40009900 tx=0x7f0e4002bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:56.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.755+0000 7f0e57552640 1 -- 172.21.15.43:0/1863999982 shutdown_connections 2024-04-21T08:38:56.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.755+0000 7f0e57552640 1 --2- 172.21.15.43:0/1863999982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e50103b40 0x7f0e5010a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.755+0000 7f0e57552640 1 --2- 172.21.15.43:0/1863999982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e50103180 0x7f0e50103600 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.755+0000 7f0e57552640 1 --2- 172.21.15.43:0/1863999982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e50101f90 0x7f0e501023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.755+0000 7f0e57552640 1 -- 172.21.15.43:0/1863999982 >> 172.21.15.43:0/1863999982 conn(0x7f0e500fd500 msgr2=0x7f0e500ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:56.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.755+0000 7f0e57552640 1 -- 172.21.15.43:0/1863999982 shutdown_connections 2024-04-21T08:38:56.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.755+0000 7f0e57552640 1 -- 172.21.15.43:0/1863999982 wait complete. 2024-04-21T08:38:56.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.755+0000 7f0e57552640 1 Processor -- start 2024-04-21T08:38:56.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.755+0000 7f0e57552640 1 -- start start 2024-04-21T08:38:56.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.755+0000 7f0e57552640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e50101f90 0x7f0e50198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:56.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.755+0000 7f0e57552640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e50103180 0x7f0e50198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:56.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.755+0000 7f0e57552640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e50103b40 0x7f0e5019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:56.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.755+0000 7f0e57552640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5010dad0 con 0x7f0e50101f90 2024-04-21T08:38:56.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.755+0000 7f0e57552640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5010d950 con 0x7f0e50103180 2024-04-21T08:38:56.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.755+0000 7f0e57552640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5010dc50 con 0x7f0e50103b40 2024-04-21T08:38:56.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.756+0000 7f0e552ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e50101f90 0x7f0e50198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:56.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.756+0000 7f0e552ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e50101f90 0x7f0e50198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38514/0 (socket says 172.21.15.43:38514) 2024-04-21T08:38:56.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.756+0000 7f0e552ee640 1 -- 172.21.15.43:0/3037815156 learned_addr learned my addr 172.21.15.43:0/3037815156 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:56.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.756+0000 7f0e54aed640 1 --2- 172.21.15.43:0/3037815156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e50103180 0x7f0e50198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:56.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.756+0000 7f0e55aef640 1 --2- 172.21.15.43:0/3037815156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e50103b40 0x7f0e5019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:56.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.756+0000 7f0e552ee640 1 -- 172.21.15.43:0/3037815156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e50103b40 msgr2=0x7f0e5019f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.756+0000 7f0e552ee640 1 --2- 172.21.15.43:0/3037815156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e50103b40 0x7f0e5019f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.756+0000 7f0e552ee640 1 -- 172.21.15.43:0/3037815156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e50103180 msgr2=0x7f0e50198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.756+0000 7f0e552ee640 1 --2- 172.21.15.43:0/3037815156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e50103180 0x7f0e50198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.756+0000 7f0e552ee640 1 -- 172.21.15.43:0/3037815156 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e5019fd50 con 0x7f0e50101f90 2024-04-21T08:38:56.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.756+0000 7f0e552ee640 1 --2- 172.21.15.43:0/3037815156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e50101f90 0x7f0e50198130 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00e9c0 tx=0x7f0e4c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:56.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.756+0000 7f0e467fc640 1 -- 172.21.15.43:0/3037815156 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c00cd80 con 0x7f0e50101f90 2024-04-21T08:38:56.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.756+0000 7f0e467fc640 1 -- 172.21.15.43:0/3037815156 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e4c0035f0 con 0x7f0e50101f90 2024-04-21T08:38:56.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.756+0000 7f0e57552640 1 -- 172.21.15.43:0/3037815156 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e5019ffe0 con 0x7f0e50101f90 2024-04-21T08:38:56.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.756+0000 7f0e467fc640 1 -- 172.21.15.43:0/3037815156 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c010690 con 0x7f0e50101f90 2024-04-21T08:38:56.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.756+0000 7f0e57552640 1 -- 172.21.15.43:0/3037815156 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e501a04a0 con 0x7f0e50101f90 2024-04-21T08:38:56.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.757+0000 7f0e57552640 1 -- 172.21.15.43:0/3037815156 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e501092b0 con 0x7f0e50101f90 2024-04-21T08:38:56.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.758+0000 7f0e467fc640 1 -- 172.21.15.43:0/3037815156 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f0e4c0037a0 con 0x7f0e50101f90 2024-04-21T08:38:56.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.759+0000 7f0e467fc640 1 --2- 172.21.15.43:0/3037815156 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0e3c0a39b0 0x7f0e3c0a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:56.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.759+0000 7f0e467fc640 1 -- 172.21.15.43:0/3037815156 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f0e4c0d0830 con 0x7f0e50101f90 2024-04-21T08:38:56.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.759+0000 7f0e54aed640 1 --2- 172.21.15.43:0/3037815156 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0e3c0a39b0 0x7f0e3c0a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:56.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.760+0000 7f0e54aed640 1 --2- 172.21.15.43:0/3037815156 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0e3c0a39b0 0x7f0e3c0a5e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0e40009230 tx=0x7f0e40005fd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:56.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:56.763+0000 7f0e467fc640 1 -- 172.21.15.43:0/3037815156 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e4c09d010 con 0x7f0e50101f90 2024-04-21T08:38:57.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.015+0000 7f0e57552640 1 -- 172.21.15.43:0/3037815156 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e50062b80 con 0x7f0e50101f90 2024-04-21T08:38:57.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.017+0000 7f0e467fc640 1 -- 172.21.15.43:0/3037815156 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7f0e4c09c910 con 0x7f0e50101f90 2024-04-21T08:38:57.016 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:57.016 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:38:57.016 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:38:57.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.019+0000 7f0e57552640 1 -- 172.21.15.43:0/3037815156 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0e3c0a39b0 msgr2=0x7f0e3c0a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.019+0000 7f0e57552640 1 --2- 172.21.15.43:0/3037815156 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0e3c0a39b0 0x7f0e3c0a5e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0e40009230 tx=0x7f0e40005fd0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.019+0000 7f0e57552640 1 -- 172.21.15.43:0/3037815156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e50101f90 msgr2=0x7f0e50198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.019+0000 7f0e57552640 1 --2- 172.21.15.43:0/3037815156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e50101f90 0x7f0e50198130 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00e9c0 tx=0x7f0e4c00ee80 comp rx=0 tx=0).stop 2024-04-21T08:38:57.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.019+0000 7f0e57552640 1 -- 172.21.15.43:0/3037815156 shutdown_connections 2024-04-21T08:38:57.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.019+0000 7f0e57552640 1 --2- 172.21.15.43:0/3037815156 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0e3c0a39b0 0x7f0e3c0a5e70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.019+0000 7f0e57552640 1 --2- 172.21.15.43:0/3037815156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e50103b40 0x7f0e5019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.019+0000 7f0e57552640 1 --2- 172.21.15.43:0/3037815156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e50103180 0x7f0e50198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.019+0000 7f0e57552640 1 --2- 172.21.15.43:0/3037815156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e50101f90 0x7f0e50198130 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.019+0000 7f0e57552640 1 -- 172.21.15.43:0/3037815156 >> 172.21.15.43:0/3037815156 conn(0x7f0e500fd500 msgr2=0x7f0e50106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:57.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.020+0000 7f0e57552640 1 -- 172.21.15.43:0/3037815156 shutdown_connections 2024-04-21T08:38:57.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.020+0000 7f0e57552640 1 -- 172.21.15.43:0/3037815156 wait complete. 2024-04-21T08:38:57.031 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-21T08:38:57.031 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-21T08:38:57.032 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T08:38:57.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.146+0000 7f42fa23f640 1 -- 172.21.15.43:0/272337428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42f4107cb0 msgr2=0x7f42f410a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.146+0000 7f42fa23f640 1 --2- 172.21.15.43:0/272337428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42f4107cb0 0x7f42f410a0a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009ca0 tx=0x7f42ec02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.147+0000 7f42fa23f640 1 -- 172.21.15.43:0/272337428 shutdown_connections 2024-04-21T08:38:57.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.147+0000 7f42fa23f640 1 --2- 172.21.15.43:0/272337428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42f4107cb0 0x7f42f410a0a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.147+0000 7f42fa23f640 1 --2- 172.21.15.43:0/272337428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42f4105380 0x7f42f4107770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.147+0000 7f42fa23f640 1 --2- 172.21.15.43:0/272337428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42f4102a50 0x7f42f4104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.147+0000 7f42fa23f640 1 -- 172.21.15.43:0/272337428 >> 172.21.15.43:0/272337428 conn(0x7f42f40fc3e0 msgr2=0x7f42f40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:57.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.147+0000 7f42fa23f640 1 -- 172.21.15.43:0/272337428 shutdown_connections 2024-04-21T08:38:57.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.147+0000 7f42fa23f640 1 -- 172.21.15.43:0/272337428 wait complete. 2024-04-21T08:38:57.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.147+0000 7f42fa23f640 1 Processor -- start 2024-04-21T08:38:57.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.148+0000 7f42fa23f640 1 -- start start 2024-04-21T08:38:57.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.148+0000 7f42fa23f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42f4102a50 0x7f42f41981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:57.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.148+0000 7f42fa23f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42f4105380 0x7f42f41986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:57.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.148+0000 7f42fa23f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42f4107cb0 0x7f42f419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:57.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.148+0000 7f42fa23f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f42f410daa0 con 0x7f42f4107cb0 2024-04-21T08:38:57.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.148+0000 7f42fa23f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f42f410d920 con 0x7f42f4102a50 2024-04-21T08:38:57.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.148+0000 7f42fa23f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f42f410dc20 con 0x7f42f4105380 2024-04-21T08:38:57.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.148+0000 7f42f3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42f4107cb0 0x7f42f419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:57.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.148+0000 7f42f2ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42f4105380 0x7f42f41986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:57.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.148+0000 7f42f37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42f4102a50 0x7f42f41981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:57.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.148+0000 7f42f3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42f4107cb0 0x7f42f419f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38550/0 (socket says 172.21.15.43:38550) 2024-04-21T08:38:57.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.148+0000 7f42f37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42f4102a50 0x7f42f41981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:55462/0 (socket says 172.21.15.43:55462) 2024-04-21T08:38:57.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.148+0000 7f42f2ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42f4105380 0x7f42f41986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.43:49552/0 (socket says 172.21.15.43:49552) 2024-04-21T08:38:57.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.148+0000 7f42f3fff640 1 -- 172.21.15.43:0/2591663744 learned_addr learned my addr 172.21.15.43:0/2591663744 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:57.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.148+0000 7f42f37fe640 1 -- 172.21.15.43:0/2591663744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42f4105380 msgr2=0x7f42f41986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.148+0000 7f42f37fe640 1 --2- 172.21.15.43:0/2591663744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42f4105380 0x7f42f41986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.148+0000 7f42f37fe640 1 -- 172.21.15.43:0/2591663744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42f4107cb0 msgr2=0x7f42f419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.148+0000 7f42f37fe640 1 --2- 172.21.15.43:0/2591663744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42f4107cb0 0x7f42f419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.148+0000 7f42f37fe640 1 -- 172.21.15.43:0/2591663744 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f419fd80 con 0x7f42f4102a50 2024-04-21T08:38:57.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.148+0000 7f42f3fff640 1 --2- 172.21.15.43:0/2591663744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42f4107cb0 0x7f42f419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:38:57.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.149+0000 7f42f37fe640 1 --2- 172.21.15.43:0/2591663744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42f4102a50 0x7f42f41981b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f42e400e9f0 tx=0x7f42e400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:57.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.149+0000 7f42f0ff9640 1 -- 172.21.15.43:0/2591663744 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e400cd80 con 0x7f42f4102a50 2024-04-21T08:38:57.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.149+0000 7f42f0ff9640 1 -- 172.21.15.43:0/2591663744 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42e40035f0 con 0x7f42f4102a50 2024-04-21T08:38:57.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.149+0000 7f42f0ff9640 1 -- 172.21.15.43:0/2591663744 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e4010670 con 0x7f42f4102a50 2024-04-21T08:38:57.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.149+0000 7f42fa23f640 1 -- 172.21.15.43:0/2591663744 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f41a0070 con 0x7f42f4102a50 2024-04-21T08:38:57.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.149+0000 7f42fa23f640 1 -- 172.21.15.43:0/2591663744 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f41a05c0 con 0x7f42f4102a50 2024-04-21T08:38:57.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.150+0000 7f42fa23f640 1 -- 172.21.15.43:0/2591663744 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f40fded0 con 0x7f42f4102a50 2024-04-21T08:38:57.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.152+0000 7f42f0ff9640 1 -- 172.21.15.43:0/2591663744 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f42e40037a0 con 0x7f42f4102a50 2024-04-21T08:38:57.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.153+0000 7f42f0ff9640 1 --2- 172.21.15.43:0/2591663744 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f42dc0a3a30 0x7f42dc0a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:57.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.153+0000 7f42f2ffd640 1 --2- 172.21.15.43:0/2591663744 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f42dc0a3a30 0x7f42dc0a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:57.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.153+0000 7f42f0ff9640 1 -- 172.21.15.43:0/2591663744 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f42e40d07a0 con 0x7f42f4102a50 2024-04-21T08:38:57.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.153+0000 7f42f2ffd640 1 --2- 172.21.15.43:0/2591663744 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f42dc0a3a30 0x7f42dc0a5ef0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f42f4199670 tx=0x7f42e800b390 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:57.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.157+0000 7f42f0ff9640 1 -- 172.21.15.43:0/2591663744 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f42e409cf40 con 0x7f42f4102a50 2024-04-21T08:38:57.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.302+0000 7f42f0ff9640 1 -- 172.21.15.43:0/2591663744 <== mon.2 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f42e4009310 con 0x7f42f4102a50 2024-04-21T08:38:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.361+0000 7f42fa23f640 1 -- 172.21.15.43:0/2591663744 --> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f42f4062390 con 0x7f42dc0a3a30 2024-04-21T08:38:57.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.372+0000 7f42f0ff9640 1 -- 172.21.15.43:0/2591663744 <== mgr.15546 v2:172.21.15.43:6800/1482273906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339423 (secure 0 0 0) 0x7f42f4062390 con 0x7f42dc0a3a30 2024-04-21T08:38:57.375 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:57.377 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-21T08:38:57.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.375+0000 7f42fa23f640 1 -- 172.21.15.43:0/2591663744 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f42dc0a3a30 msgr2=0x7f42dc0a5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.375+0000 7f42fa23f640 1 --2- 172.21.15.43:0/2591663744 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f42dc0a3a30 0x7f42dc0a5ef0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f42f4199670 tx=0x7f42e800b390 comp rx=0 tx=0).stop 2024-04-21T08:38:57.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.375+0000 7f42fa23f640 1 -- 172.21.15.43:0/2591663744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42f4102a50 msgr2=0x7f42f41981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.375+0000 7f42fa23f640 1 --2- 172.21.15.43:0/2591663744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42f4102a50 0x7f42f41981b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f42e400e9f0 tx=0x7f42e400eeb0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.375+0000 7f42fa23f640 1 -- 172.21.15.43:0/2591663744 shutdown_connections 2024-04-21T08:38:57.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.375+0000 7f42fa23f640 1 --2- 172.21.15.43:0/2591663744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42f4107cb0 0x7f42f419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.375+0000 7f42fa23f640 1 --2- 172.21.15.43:0/2591663744 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f42dc0a3a30 0x7f42dc0a5ef0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.375+0000 7f42fa23f640 1 --2- 172.21.15.43:0/2591663744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42f4105380 0x7f42f41986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.375+0000 7f42fa23f640 1 --2- 172.21.15.43:0/2591663744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42f4102a50 0x7f42f41981b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.375+0000 7f42fa23f640 1 -- 172.21.15.43:0/2591663744 >> 172.21.15.43:0/2591663744 conn(0x7f42f40fc3e0 msgr2=0x7f42f4103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:57.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.375+0000 7f42fa23f640 1 -- 172.21.15.43:0/2591663744 shutdown_connections 2024-04-21T08:38:57.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.375+0000 7f42fa23f640 1 -- 172.21.15.43:0/2591663744 wait complete. 2024-04-21T08:38:57.386 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-04-21T08:38:56.901532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1194985,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1528,"num_read_kb":935,"num_write":560,"num_write_kb":4416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":947,"ondisk_log_size":947,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386508,"kb_used_data":16520,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124555316,"statfs":{"total":1151940427776,"available":1151544643584,"internally_reserved":0,"allocated":16916480,"data_stored":9972833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":378713404},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.989687"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005622+0000","last_change":"2024-04-21T08:36:23.983921+0000","last_active":"2024-04-21T08:38:55.005622+0000","last_peered":"2024-04-21T08:38:55.005622+0000","last_clean":"2024-04-21T08:38:55.005622+0000","last_became_active":"2024-04-21T08:36:23.983638+0000","last_became_peered":"2024-04-21T08:36:23.983638+0000","last_unstale":"2024-04-21T08:38:55.005622+0000","last_undegraded":"2024-04-21T08:38:55.005622+0000","last_fullsized":"2024-04-21T08:38:55.005622+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:29:44.646308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004430+0000","last_change":"2024-04-21T08:36:23.990029+0000","last_active":"2024-04-21T08:38:55.004430+0000","last_peered":"2024-04-21T08:38:55.004430+0000","last_clean":"2024-04-21T08:38:55.004430+0000","last_became_active":"2024-04-21T08:36:23.989141+0000","last_became_peered":"2024-04-21T08:36:23.989141+0000","last_unstale":"2024-04-21T08:38:55.004430+0000","last_undegraded":"2024-04-21T08:38:55.004430+0000","last_fullsized":"2024-04-21T08:38:55.004430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:23:58.692342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004234+0000","last_change":"2024-04-21T08:36:23.982205+0000","last_active":"2024-04-21T08:38:55.004234+0000","last_peered":"2024-04-21T08:38:55.004234+0000","last_clean":"2024-04-21T08:38:55.004234+0000","last_became_active":"2024-04-21T08:36:23.981876+0000","last_became_peered":"2024-04-21T08:36:23.981876+0000","last_unstale":"2024-04-21T08:38:55.004234+0000","last_undegraded":"2024-04-21T08:38:55.004234+0000","last_fullsized":"2024-04-21T08:38:55.004234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:12:27.121106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006754+0000","last_change":"2024-04-21T08:36:23.987425+0000","last_active":"2024-04-21T08:38:55.006754+0000","last_peered":"2024-04-21T08:38:55.006754+0000","last_clean":"2024-04-21T08:38:55.006754+0000","last_became_active":"2024-04-21T08:36:23.987109+0000","last_became_peered":"2024-04-21T08:36:23.987109+0000","last_unstale":"2024-04-21T08:38:55.006754+0000","last_undegraded":"2024-04-21T08:38:55.006754+0000","last_fullsized":"2024-04-21T08:38:55.006754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:44.176324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890536+0000","last_change":"2024-04-21T08:36:23.990375+0000","last_active":"2024-04-21T08:38:54.890536+0000","last_peered":"2024-04-21T08:38:54.890536+0000","last_clean":"2024-04-21T08:38:54.890536+0000","last_became_active":"2024-04-21T08:36:23.990058+0000","last_became_peered":"2024-04-21T08:36:23.990058+0000","last_unstale":"2024-04-21T08:38:54.890536+0000","last_undegraded":"2024-04-21T08:38:54.890536+0000","last_fullsized":"2024-04-21T08:38:54.890536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:24:09.647528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888221+0000","last_change":"2024-04-21T08:36:23.979848+0000","last_active":"2024-04-21T08:38:54.888221+0000","last_peered":"2024-04-21T08:38:54.888221+0000","last_clean":"2024-04-21T08:38:54.888221+0000","last_became_active":"2024-04-21T08:36:23.979511+0000","last_became_peered":"2024-04-21T08:36:23.979511+0000","last_unstale":"2024-04-21T08:38:54.888221+0000","last_undegraded":"2024-04-21T08:38:54.888221+0000","last_fullsized":"2024-04-21T08:38:54.888221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:07:46.551369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006232+0000","last_change":"2024-04-21T08:37:15.118755+0000","last_active":"2024-04-21T08:38:55.006232+0000","last_peered":"2024-04-21T08:38:55.006232+0000","last_clean":"2024-04-21T08:38:55.006232+0000","last_became_active":"2024-04-21T08:37:15.118347+0000","last_became_peered":"2024-04-21T08:37:15.118347+0000","last_unstale":"2024-04-21T08:38:55.006232+0000","last_undegraded":"2024-04-21T08:38:55.006232+0000","last_fullsized":"2024-04-21T08:38:55.006232+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:36:43.398325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886870+0000","last_change":"2024-04-21T08:36:23.987517+0000","last_active":"2024-04-21T08:38:54.886870+0000","last_peered":"2024-04-21T08:38:54.886870+0000","last_clean":"2024-04-21T08:38:54.886870+0000","last_became_active":"2024-04-21T08:36:23.987201+0000","last_became_peered":"2024-04-21T08:36:23.987201+0000","last_unstale":"2024-04-21T08:38:54.886870+0000","last_undegraded":"2024-04-21T08:38:54.886870+0000","last_fullsized":"2024-04-21T08:38:54.886870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:42:53.080863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104900+0000","last_change":"2024-04-21T08:36:23.989434+0000","last_active":"2024-04-21T08:38:55.104900+0000","last_peered":"2024-04-21T08:38:55.104900+0000","last_clean":"2024-04-21T08:38:55.104900+0000","last_became_active":"2024-04-21T08:36:23.989125+0000","last_became_peered":"2024-04-21T08:36:23.989125+0000","last_unstale":"2024-04-21T08:38:55.104900+0000","last_undegraded":"2024-04-21T08:38:55.104900+0000","last_fullsized":"2024-04-21T08:38:55.104900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:30:07.219056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888449+0000","last_change":"2024-04-21T08:36:23.980725+0000","last_active":"2024-04-21T08:38:54.888449+0000","last_peered":"2024-04-21T08:38:54.888449+0000","last_clean":"2024-04-21T08:38:54.888449+0000","last_became_active":"2024-04-21T08:36:23.980363+0000","last_became_peered":"2024-04-21T08:36:23.980363+0000","last_unstale":"2024-04-21T08:38:54.888449+0000","last_undegraded":"2024-04-21T08:38:54.888449+0000","last_fullsized":"2024-04-21T08:38:54.888449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:26:46.202030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006978+0000","last_change":"2024-04-21T08:36:23.989369+0000","last_active":"2024-04-21T08:38:55.006978+0000","last_peered":"2024-04-21T08:38:55.006978+0000","last_clean":"2024-04-21T08:38:55.006978+0000","last_became_active":"2024-04-21T08:36:23.989030+0000","last_became_peered":"2024-04-21T08:36:23.989030+0000","last_unstale":"2024-04-21T08:38:55.006978+0000","last_undegraded":"2024-04-21T08:38:55.006978+0000","last_fullsized":"2024-04-21T08:38:55.006978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:23:13.725703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006691+0000","last_change":"2024-04-21T08:36:23.993485+0000","last_active":"2024-04-21T08:38:55.006691+0000","last_peered":"2024-04-21T08:38:55.006691+0000","last_clean":"2024-04-21T08:38:55.006691+0000","last_became_active":"2024-04-21T08:36:23.993124+0000","last_became_peered":"2024-04-21T08:36:23.993124+0000","last_unstale":"2024-04-21T08:38:55.006691+0000","last_undegraded":"2024-04-21T08:38:55.006691+0000","last_fullsized":"2024-04-21T08:38:55.006691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:53:16.249838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076569+0000","last_change":"2024-04-21T08:36:23.983146+0000","last_active":"2024-04-21T08:38:55.076569+0000","last_peered":"2024-04-21T08:38:55.076569+0000","last_clean":"2024-04-21T08:38:55.076569+0000","last_became_active":"2024-04-21T08:36:23.982766+0000","last_became_peered":"2024-04-21T08:36:23.982766+0000","last_unstale":"2024-04-21T08:38:55.076569+0000","last_undegraded":"2024-04-21T08:38:55.076569+0000","last_fullsized":"2024-04-21T08:38:55.076569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:30:44.090929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.003129+0000","last_change":"2024-04-21T08:36:23.988906+0000","last_active":"2024-04-21T08:38:55.003129+0000","last_peered":"2024-04-21T08:38:55.003129+0000","last_clean":"2024-04-21T08:38:55.003129+0000","last_became_active":"2024-04-21T08:36:23.988489+0000","last_became_peered":"2024-04-21T08:36:23.988489+0000","last_unstale":"2024-04-21T08:38:55.003129+0000","last_undegraded":"2024-04-21T08:38:55.003129+0000","last_fullsized":"2024-04-21T08:38:55.003129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:06:43.271650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.003060+0000","last_change":"2024-04-21T08:37:15.123343+0000","last_active":"2024-04-21T08:38:55.003060+0000","last_peered":"2024-04-21T08:38:55.003060+0000","last_clean":"2024-04-21T08:38:55.003060+0000","last_became_active":"2024-04-21T08:37:15.122959+0000","last_became_peered":"2024-04-21T08:37:15.122959+0000","last_unstale":"2024-04-21T08:38:55.003060+0000","last_undegraded":"2024-04-21T08:38:55.003060+0000","last_fullsized":"2024-04-21T08:38:55.003060+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:56:19.179879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104471+0000","last_change":"2024-04-21T08:36:23.983136+0000","last_active":"2024-04-21T08:38:55.104471+0000","last_peered":"2024-04-21T08:38:55.104471+0000","last_clean":"2024-04-21T08:38:55.104471+0000","last_became_active":"2024-04-21T08:36:23.982698+0000","last_became_peered":"2024-04-21T08:36:23.982698+0000","last_unstale":"2024-04-21T08:38:55.104471+0000","last_undegraded":"2024-04-21T08:38:55.104471+0000","last_fullsized":"2024-04-21T08:38:55.104471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:34.750941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005563+0000","last_change":"2024-04-21T08:36:23.984701+0000","last_active":"2024-04-21T08:38:55.005563+0000","last_peered":"2024-04-21T08:38:55.005563+0000","last_clean":"2024-04-21T08:38:55.005563+0000","last_became_active":"2024-04-21T08:36:23.984432+0000","last_became_peered":"2024-04-21T08:36:23.984432+0000","last_unstale":"2024-04-21T08:38:55.005563+0000","last_undegraded":"2024-04-21T08:38:55.005563+0000","last_fullsized":"2024-04-21T08:38:55.005563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:10:58.944380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.996823+0000","last_change":"2024-04-21T08:36:23.982781+0000","last_active":"2024-04-21T08:38:54.996823+0000","last_peered":"2024-04-21T08:38:54.996823+0000","last_clean":"2024-04-21T08:38:54.996823+0000","last_became_active":"2024-04-21T08:36:23.982459+0000","last_became_peered":"2024-04-21T08:36:23.982459+0000","last_unstale":"2024-04-21T08:38:54.996823+0000","last_undegraded":"2024-04-21T08:38:54.996823+0000","last_fullsized":"2024-04-21T08:38:54.996823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:44.657504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.105000+0000","last_change":"2024-04-21T08:36:23.992002+0000","last_active":"2024-04-21T08:38:55.105000+0000","last_peered":"2024-04-21T08:38:55.105000+0000","last_clean":"2024-04-21T08:38:55.105000+0000","last_became_active":"2024-04-21T08:36:23.991053+0000","last_became_peered":"2024-04-21T08:36:23.991053+0000","last_unstale":"2024-04-21T08:38:55.105000+0000","last_undegraded":"2024-04-21T08:38:55.105000+0000","last_fullsized":"2024-04-21T08:38:55.105000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:56:37.145949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104844+0000","last_change":"2024-04-21T08:36:28.013195+0000","last_active":"2024-04-21T08:38:55.104844+0000","last_peered":"2024-04-21T08:38:55.104844+0000","last_clean":"2024-04-21T08:38:55.104844+0000","last_became_active":"2024-04-21T08:36:28.012656+0000","last_became_peered":"2024-04-21T08:36:28.012656+0000","last_unstale":"2024-04-21T08:38:55.104844+0000","last_undegraded":"2024-04-21T08:38:55.104844+0000","last_fullsized":"2024-04-21T08:38:55.104844+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:12:03.133583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"95'14","reported_seq":173,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.074681+0000","last_change":"2024-04-21T08:36:23.468957+0000","last_active":"2024-04-21T08:38:55.074681+0000","last_peered":"2024-04-21T08:38:55.074681+0000","last_clean":"2024-04-21T08:38:55.074681+0000","last_became_active":"2024-04-21T08:36:23.468601+0000","last_became_peered":"2024-04-21T08:36:23.468601+0000","last_unstale":"2024-04-21T08:38:55.074681+0000","last_undegraded":"2024-04-21T08:38:55.074681+0000","last_fullsized":"2024-04-21T08:38:55.074681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:13:11.977780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":125,"num_read_kb":125,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.996013+0000","last_change":"2024-04-21T08:37:15.123476+0000","last_active":"2024-04-21T08:38:54.996013+0000","last_peered":"2024-04-21T08:38:54.996013+0000","last_clean":"2024-04-21T08:38:54.996013+0000","last_became_active":"2024-04-21T08:37:15.123066+0000","last_became_peered":"2024-04-21T08:37:15.123066+0000","last_unstale":"2024-04-21T08:38:54.996013+0000","last_undegraded":"2024-04-21T08:38:54.996013+0000","last_fullsized":"2024-04-21T08:38:54.996013+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:15.786178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005900+0000","last_change":"2024-04-21T08:36:28.014161+0000","last_active":"2024-04-21T08:38:55.005900+0000","last_peered":"2024-04-21T08:38:55.005900+0000","last_clean":"2024-04-21T08:38:55.005900+0000","last_became_active":"2024-04-21T08:36:28.013859+0000","last_became_peered":"2024-04-21T08:36:28.013859+0000","last_unstale":"2024-04-21T08:38:55.005900+0000","last_undegraded":"2024-04-21T08:38:55.005900+0000","last_fullsized":"2024-04-21T08:38:55.005900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:08.315334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887665+0000","last_change":"2024-04-21T08:36:23.475612+0000","last_active":"2024-04-21T08:38:54.887665+0000","last_peered":"2024-04-21T08:38:54.887665+0000","last_clean":"2024-04-21T08:38:54.887665+0000","last_became_active":"2024-04-21T08:36:23.475301+0000","last_became_peered":"2024-04-21T08:36:23.475301+0000","last_unstale":"2024-04-21T08:38:54.887665+0000","last_undegraded":"2024-04-21T08:38:54.887665+0000","last_fullsized":"2024-04-21T08:38:54.887665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:56.801630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004414+0000","last_change":"2024-04-21T08:36:23.984689+0000","last_active":"2024-04-21T08:38:55.004414+0000","last_peered":"2024-04-21T08:38:55.004414+0000","last_clean":"2024-04-21T08:38:55.004414+0000","last_became_active":"2024-04-21T08:36:23.984358+0000","last_became_peered":"2024-04-21T08:36:23.984358+0000","last_unstale":"2024-04-21T08:38:55.004414+0000","last_undegraded":"2024-04-21T08:38:55.004414+0000","last_fullsized":"2024-04-21T08:38:55.004414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:49:50.187466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006203+0000","last_change":"2024-04-21T08:36:28.007951+0000","last_active":"2024-04-21T08:38:55.006203+0000","last_peered":"2024-04-21T08:38:55.006203+0000","last_clean":"2024-04-21T08:38:55.006203+0000","last_became_active":"2024-04-21T08:36:28.007553+0000","last_became_peered":"2024-04-21T08:36:28.007553+0000","last_unstale":"2024-04-21T08:38:55.006203+0000","last_undegraded":"2024-04-21T08:38:55.006203+0000","last_fullsized":"2024-04-21T08:38:55.006203+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:10:54.019605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890572+0000","last_change":"2024-04-21T08:36:23.473371+0000","last_active":"2024-04-21T08:38:54.890572+0000","last_peered":"2024-04-21T08:38:54.890572+0000","last_clean":"2024-04-21T08:38:54.890572+0000","last_became_active":"2024-04-21T08:36:23.472472+0000","last_became_peered":"2024-04-21T08:36:23.472472+0000","last_unstale":"2024-04-21T08:38:54.890572+0000","last_undegraded":"2024-04-21T08:38:54.890572+0000","last_fullsized":"2024-04-21T08:38:54.890572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:28:49.622873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104901+0000","last_change":"2024-04-21T08:36:23.987916+0000","last_active":"2024-04-21T08:38:55.104901+0000","last_peered":"2024-04-21T08:38:55.104901+0000","last_clean":"2024-04-21T08:38:55.104901+0000","last_became_active":"2024-04-21T08:36:23.986207+0000","last_became_peered":"2024-04-21T08:36:23.986207+0000","last_unstale":"2024-04-21T08:38:55.104901+0000","last_undegraded":"2024-04-21T08:38:55.104901+0000","last_fullsized":"2024-04-21T08:38:55.104901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:49:57.261078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006246+0000","last_change":"2024-04-21T08:36:28.006494+0000","last_active":"2024-04-21T08:38:55.006246+0000","last_peered":"2024-04-21T08:38:55.006246+0000","last_clean":"2024-04-21T08:38:55.006246+0000","last_became_active":"2024-04-21T08:36:28.006069+0000","last_became_peered":"2024-04-21T08:36:28.006069+0000","last_unstale":"2024-04-21T08:38:55.006246+0000","last_undegraded":"2024-04-21T08:38:55.006246+0000","last_fullsized":"2024-04-21T08:38:55.006246+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:04.608295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890975+0000","last_change":"2024-04-21T08:36:23.468923+0000","last_active":"2024-04-21T08:38:54.890975+0000","last_peered":"2024-04-21T08:38:54.890975+0000","last_clean":"2024-04-21T08:38:54.890975+0000","last_became_active":"2024-04-21T08:36:23.468509+0000","last_became_peered":"2024-04-21T08:36:23.468509+0000","last_unstale":"2024-04-21T08:38:54.890975+0000","last_undegraded":"2024-04-21T08:38:54.890975+0000","last_fullsized":"2024-04-21T08:38:54.890975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:44.452748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008259+0000","last_change":"2024-04-21T08:36:23.994423+0000","last_active":"2024-04-21T08:38:55.008259+0000","last_peered":"2024-04-21T08:38:55.008259+0000","last_clean":"2024-04-21T08:38:55.008259+0000","last_became_active":"2024-04-21T08:36:23.993827+0000","last_became_peered":"2024-04-21T08:36:23.993827+0000","last_unstale":"2024-04-21T08:38:55.008259+0000","last_undegraded":"2024-04-21T08:38:55.008259+0000","last_fullsized":"2024-04-21T08:38:55.008259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:49:25.550301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104711+0000","last_change":"2024-04-21T08:36:28.004037+0000","last_active":"2024-04-21T08:38:55.104711+0000","last_peered":"2024-04-21T08:38:55.104711+0000","last_clean":"2024-04-21T08:38:55.104711+0000","last_became_active":"2024-04-21T08:36:28.003580+0000","last_became_peered":"2024-04-21T08:36:28.003580+0000","last_unstale":"2024-04-21T08:38:55.104711+0000","last_undegraded":"2024-04-21T08:38:55.104711+0000","last_fullsized":"2024-04-21T08:38:55.104711+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:04:51.380244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'8","reported_seq":172,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.181108+0000","last_change":"2024-04-21T08:36:23.468430+0000","last_active":"2024-04-21T08:38:55.181108+0000","last_peered":"2024-04-21T08:38:55.181108+0000","last_clean":"2024-04-21T08:38:55.181108+0000","last_became_active":"2024-04-21T08:36:23.468089+0000","last_became_peered":"2024-04-21T08:36:23.468089+0000","last_unstale":"2024-04-21T08:38:55.181108+0000","last_undegraded":"2024-04-21T08:38:55.181108+0000","last_fullsized":"2024-04-21T08:38:55.181108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:12:54.827034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":130,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890458+0000","last_change":"2024-04-21T08:37:15.117922+0000","last_active":"2024-04-21T08:38:54.890458+0000","last_peered":"2024-04-21T08:38:54.890458+0000","last_clean":"2024-04-21T08:38:54.890458+0000","last_became_active":"2024-04-21T08:37:15.117404+0000","last_became_peered":"2024-04-21T08:37:15.117404+0000","last_unstale":"2024-04-21T08:38:54.890458+0000","last_undegraded":"2024-04-21T08:38:54.890458+0000","last_fullsized":"2024-04-21T08:38:54.890458+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:39.682224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104672+0000","last_change":"2024-04-21T08:36:28.004638+0000","last_active":"2024-04-21T08:38:55.104672+0000","last_peered":"2024-04-21T08:38:55.104672+0000","last_clean":"2024-04-21T08:38:55.104672+0000","last_became_active":"2024-04-21T08:36:28.002499+0000","last_became_peered":"2024-04-21T08:36:28.002499+0000","last_unstale":"2024-04-21T08:38:55.104672+0000","last_undegraded":"2024-04-21T08:38:55.104672+0000","last_fullsized":"2024-04-21T08:38:55.104672+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:45:46.934783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":55,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888089+0000","last_change":"2024-04-21T08:37:15.126569+0000","last_active":"2024-04-21T08:38:54.888089+0000","last_peered":"2024-04-21T08:38:54.888089+0000","last_clean":"2024-04-21T08:38:54.888089+0000","last_became_active":"2024-04-21T08:37:15.121284+0000","last_became_peered":"2024-04-21T08:37:15.121284+0000","last_unstale":"2024-04-21T08:38:54.888089+0000","last_undegraded":"2024-04-21T08:38:54.888089+0000","last_fullsized":"2024-04-21T08:38:54.888089+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:02:26.153324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008374+0000","last_change":"2024-04-21T08:37:15.557192+0000","last_active":"2024-04-21T08:38:55.008374+0000","last_peered":"2024-04-21T08:38:55.008374+0000","last_clean":"2024-04-21T08:38:55.008374+0000","last_became_active":"2024-04-21T08:37:15.556653+0000","last_became_peered":"2024-04-21T08:37:15.556653+0000","last_unstale":"2024-04-21T08:38:55.008374+0000","last_undegraded":"2024-04-21T08:38:55.008374+0000","last_fullsized":"2024-04-21T08:38:55.008374+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:42.722460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005527+0000","last_change":"2024-04-21T08:36:28.017068+0000","last_active":"2024-04-21T08:38:55.005527+0000","last_peered":"2024-04-21T08:38:55.005527+0000","last_clean":"2024-04-21T08:38:55.005527+0000","last_became_active":"2024-04-21T08:36:28.016770+0000","last_became_peered":"2024-04-21T08:36:28.016770+0000","last_unstale":"2024-04-21T08:38:55.005527+0000","last_undegraded":"2024-04-21T08:38:55.005527+0000","last_fullsized":"2024-04-21T08:38:55.005527+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:07:14.596090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076311+0000","last_change":"2024-04-21T08:36:23.470430+0000","last_active":"2024-04-21T08:38:55.076311+0000","last_peered":"2024-04-21T08:38:55.076311+0000","last_clean":"2024-04-21T08:38:55.076311+0000","last_became_active":"2024-04-21T08:36:23.470109+0000","last_became_peered":"2024-04-21T08:36:23.470109+0000","last_unstale":"2024-04-21T08:38:55.076311+0000","last_undegraded":"2024-04-21T08:38:55.076311+0000","last_fullsized":"2024-04-21T08:38:55.076311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:18.176313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887026+0000","last_change":"2024-04-21T08:36:23.986668+0000","last_active":"2024-04-21T08:38:54.887026+0000","last_peered":"2024-04-21T08:38:54.887026+0000","last_clean":"2024-04-21T08:38:54.887026+0000","last_became_active":"2024-04-21T08:36:23.986344+0000","last_became_peered":"2024-04-21T08:36:23.986344+0000","last_unstale":"2024-04-21T08:38:54.887026+0000","last_undegraded":"2024-04-21T08:38:54.887026+0000","last_fullsized":"2024-04-21T08:38:54.887026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:56:17.744145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004263+0000","last_change":"2024-04-21T08:36:28.009012+0000","last_active":"2024-04-21T08:38:55.004263+0000","last_peered":"2024-04-21T08:38:55.004263+0000","last_clean":"2024-04-21T08:38:55.004263+0000","last_became_active":"2024-04-21T08:36:28.008123+0000","last_became_peered":"2024-04-21T08:36:28.008123+0000","last_unstale":"2024-04-21T08:38:55.004263+0000","last_undegraded":"2024-04-21T08:38:55.004263+0000","last_fullsized":"2024-04-21T08:38:55.004263+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:33:18.881564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007854+0000","last_change":"2024-04-21T08:36:23.467104+0000","last_active":"2024-04-21T08:38:55.007854+0000","last_peered":"2024-04-21T08:38:55.007854+0000","last_clean":"2024-04-21T08:38:55.007854+0000","last_became_active":"2024-04-21T08:36:23.466725+0000","last_became_peered":"2024-04-21T08:36:23.466725+0000","last_unstale":"2024-04-21T08:38:55.007854+0000","last_undegraded":"2024-04-21T08:38:55.007854+0000","last_fullsized":"2024-04-21T08:38:55.007854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:35:45.986448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.995844+0000","last_change":"2024-04-21T08:36:23.987579+0000","last_active":"2024-04-21T08:38:54.995844+0000","last_peered":"2024-04-21T08:38:54.995844+0000","last_clean":"2024-04-21T08:38:54.995844+0000","last_became_active":"2024-04-21T08:36:23.987044+0000","last_became_peered":"2024-04-21T08:36:23.987044+0000","last_unstale":"2024-04-21T08:38:54.995844+0000","last_undegraded":"2024-04-21T08:38:54.995844+0000","last_fullsized":"2024-04-21T08:38:54.995844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:48:27.593090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886895+0000","last_change":"2024-04-21T08:36:28.006417+0000","last_active":"2024-04-21T08:38:54.886895+0000","last_peered":"2024-04-21T08:38:54.886895+0000","last_clean":"2024-04-21T08:38:54.886895+0000","last_became_active":"2024-04-21T08:36:28.006006+0000","last_became_peered":"2024-04-21T08:36:28.006006+0000","last_unstale":"2024-04-21T08:38:54.886895+0000","last_undegraded":"2024-04-21T08:38:54.886895+0000","last_fullsized":"2024-04-21T08:38:54.886895+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:16.173284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'8","reported_seq":292,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.181085+0000","last_change":"2024-04-21T08:36:23.471541+0000","last_active":"2024-04-21T08:38:55.181085+0000","last_peered":"2024-04-21T08:38:55.181085+0000","last_clean":"2024-04-21T08:38:55.181085+0000","last_became_active":"2024-04-21T08:36:23.471193+0000","last_became_peered":"2024-04-21T08:36:23.471193+0000","last_unstale":"2024-04-21T08:38:55.181085+0000","last_undegraded":"2024-04-21T08:38:55.181085+0000","last_fullsized":"2024-04-21T08:38:55.181085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:06:04.916707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":250,"num_read_kb":250,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104848+0000","last_change":"2024-04-21T08:36:23.986781+0000","last_active":"2024-04-21T08:38:55.104848+0000","last_peered":"2024-04-21T08:38:55.104848+0000","last_clean":"2024-04-21T08:38:55.104848+0000","last_became_active":"2024-04-21T08:36:23.985696+0000","last_became_peered":"2024-04-21T08:36:23.985696+0000","last_unstale":"2024-04-21T08:38:55.104848+0000","last_undegraded":"2024-04-21T08:38:55.104848+0000","last_fullsized":"2024-04-21T08:38:55.104848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:40.556642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.995747+0000","last_change":"2024-04-21T08:36:28.011621+0000","last_active":"2024-04-21T08:38:54.995747+0000","last_peered":"2024-04-21T08:38:54.995747+0000","last_clean":"2024-04-21T08:38:54.995747+0000","last_became_active":"2024-04-21T08:36:28.011136+0000","last_became_peered":"2024-04-21T08:36:28.011136+0000","last_unstale":"2024-04-21T08:38:54.995747+0000","last_undegraded":"2024-04-21T08:38:54.995747+0000","last_fullsized":"2024-04-21T08:38:54.995747+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:47.623445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006842+0000","last_change":"2024-04-21T08:36:23.474019+0000","last_active":"2024-04-21T08:38:55.006842+0000","last_peered":"2024-04-21T08:38:55.006842+0000","last_clean":"2024-04-21T08:38:55.006842+0000","last_became_active":"2024-04-21T08:36:23.473690+0000","last_became_peered":"2024-04-21T08:36:23.473690+0000","last_unstale":"2024-04-21T08:38:55.006842+0000","last_undegraded":"2024-04-21T08:38:55.006842+0000","last_fullsized":"2024-04-21T08:38:55.006842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:38:32.913795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004613+0000","last_change":"2024-04-21T08:37:15.121455+0000","last_active":"2024-04-21T08:38:55.004613+0000","last_peered":"2024-04-21T08:38:55.004613+0000","last_clean":"2024-04-21T08:38:55.004613+0000","last_became_active":"2024-04-21T08:37:15.121113+0000","last_became_peered":"2024-04-21T08:37:15.121113+0000","last_unstale":"2024-04-21T08:38:55.004613+0000","last_undegraded":"2024-04-21T08:38:55.004613+0000","last_fullsized":"2024-04-21T08:38:55.004613+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:39:36.525495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006512+0000","last_change":"2024-04-21T08:38:15.413849+0000","last_active":"2024-04-21T08:38:55.006512+0000","last_peered":"2024-04-21T08:38:55.006512+0000","last_clean":"2024-04-21T08:38:55.006512+0000","last_became_active":"2024-04-21T08:38:15.413361+0000","last_became_peered":"2024-04-21T08:38:15.413361+0000","last_unstale":"2024-04-21T08:38:55.006512+0000","last_undegraded":"2024-04-21T08:38:55.006512+0000","last_fullsized":"2024-04-21T08:38:55.006512+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:16:49.197800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008173+0000","last_change":"2024-04-21T08:36:23.465825+0000","last_active":"2024-04-21T08:38:55.008173+0000","last_peered":"2024-04-21T08:38:55.008173+0000","last_clean":"2024-04-21T08:38:55.008173+0000","last_became_active":"2024-04-21T08:36:23.465525+0000","last_became_peered":"2024-04-21T08:36:23.465525+0000","last_unstale":"2024-04-21T08:38:55.008173+0000","last_undegraded":"2024-04-21T08:38:55.008173+0000","last_fullsized":"2024-04-21T08:38:55.008173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:59:50.502210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.105005+0000","last_change":"2024-04-21T08:36:23.987602+0000","last_active":"2024-04-21T08:38:55.105005+0000","last_peered":"2024-04-21T08:38:55.105005+0000","last_clean":"2024-04-21T08:38:55.105005+0000","last_became_active":"2024-04-21T08:36:23.987298+0000","last_became_peered":"2024-04-21T08:36:23.987298+0000","last_unstale":"2024-04-21T08:38:55.105005+0000","last_undegraded":"2024-04-21T08:38:55.105005+0000","last_fullsized":"2024-04-21T08:38:55.105005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:06.647676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.995946+0000","last_change":"2024-04-21T08:36:28.005935+0000","last_active":"2024-04-21T08:38:54.995946+0000","last_peered":"2024-04-21T08:38:54.995946+0000","last_clean":"2024-04-21T08:38:54.995946+0000","last_became_active":"2024-04-21T08:36:28.005512+0000","last_became_peered":"2024-04-21T08:36:28.005512+0000","last_unstale":"2024-04-21T08:38:54.995946+0000","last_undegraded":"2024-04-21T08:38:54.995946+0000","last_fullsized":"2024-04-21T08:38:54.995946+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:23:20.298524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"95'43","reported_seq":341,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.178912+0000","last_change":"2024-04-21T08:36:23.467728+0000","last_active":"2024-04-21T08:38:55.178912+0000","last_peered":"2024-04-21T08:38:55.178912+0000","last_clean":"2024-04-21T08:38:55.178912+0000","last_became_active":"2024-04-21T08:36:23.467430+0000","last_became_peered":"2024-04-21T08:36:23.467430+0000","last_unstale":"2024-04-21T08:38:55.178912+0000","last_undegraded":"2024-04-21T08:38:55.178912+0000","last_fullsized":"2024-04-21T08:38:55.178912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:47:18.900911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70068,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":264,"num_read_kb":201,"num_write":45,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890735+0000","last_change":"2024-04-21T08:36:23.989116+0000","last_active":"2024-04-21T08:38:54.890735+0000","last_peered":"2024-04-21T08:38:54.890735+0000","last_clean":"2024-04-21T08:38:54.890735+0000","last_became_active":"2024-04-21T08:36:23.988758+0000","last_became_peered":"2024-04-21T08:36:23.988758+0000","last_unstale":"2024-04-21T08:38:54.890735+0000","last_undegraded":"2024-04-21T08:38:54.890735+0000","last_fullsized":"2024-04-21T08:38:54.890735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:56:25.969905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008450+0000","last_change":"2024-04-21T08:36:28.007370+0000","last_active":"2024-04-21T08:38:55.008450+0000","last_peered":"2024-04-21T08:38:55.008450+0000","last_clean":"2024-04-21T08:38:55.008450+0000","last_became_active":"2024-04-21T08:36:28.006972+0000","last_became_peered":"2024-04-21T08:36:28.006972+0000","last_unstale":"2024-04-21T08:38:55.008450+0000","last_undegraded":"2024-04-21T08:38:55.008450+0000","last_fullsized":"2024-04-21T08:38:55.008450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:18:14.926058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104708+0000","last_change":"2024-04-21T08:36:23.472658+0000","last_active":"2024-04-21T08:38:55.104708+0000","last_peered":"2024-04-21T08:38:55.104708+0000","last_clean":"2024-04-21T08:38:55.104708+0000","last_became_active":"2024-04-21T08:36:23.472318+0000","last_became_peered":"2024-04-21T08:36:23.472318+0000","last_unstale":"2024-04-21T08:38:55.104708+0000","last_undegraded":"2024-04-21T08:38:55.104708+0000","last_fullsized":"2024-04-21T08:38:55.104708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:28:27.497605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076501+0000","last_change":"2024-04-21T08:38:15.338892+0000","last_active":"2024-04-21T08:38:55.076501+0000","last_peered":"2024-04-21T08:38:55.076501+0000","last_clean":"2024-04-21T08:38:55.076501+0000","last_became_active":"2024-04-21T08:38:15.338396+0000","last_became_peered":"2024-04-21T08:38:15.338396+0000","last_unstale":"2024-04-21T08:38:55.076501+0000","last_undegraded":"2024-04-21T08:38:55.076501+0000","last_fullsized":"2024-04-21T08:38:55.076501+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:19.670635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005342+0000","last_change":"2024-04-21T08:36:28.019186+0000","last_active":"2024-04-21T08:38:55.005342+0000","last_peered":"2024-04-21T08:38:55.005342+0000","last_clean":"2024-04-21T08:38:55.005342+0000","last_became_active":"2024-04-21T08:36:28.018774+0000","last_became_peered":"2024-04-21T08:36:28.018774+0000","last_unstale":"2024-04-21T08:38:55.005342+0000","last_undegraded":"2024-04-21T08:38:55.005342+0000","last_fullsized":"2024-04-21T08:38:55.005342+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:35:33.548656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008059+0000","last_change":"2024-04-21T08:36:23.472633+0000","last_active":"2024-04-21T08:38:55.008059+0000","last_peered":"2024-04-21T08:38:55.008059+0000","last_clean":"2024-04-21T08:38:55.008059+0000","last_became_active":"2024-04-21T08:36:23.472336+0000","last_became_peered":"2024-04-21T08:36:23.472336+0000","last_unstale":"2024-04-21T08:38:55.008059+0000","last_undegraded":"2024-04-21T08:38:55.008059+0000","last_fullsized":"2024-04-21T08:38:55.008059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:49.463492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006193+0000","last_change":"2024-04-21T08:36:23.994332+0000","last_active":"2024-04-21T08:38:55.006193+0000","last_peered":"2024-04-21T08:38:55.006193+0000","last_clean":"2024-04-21T08:38:55.006193+0000","last_became_active":"2024-04-21T08:36:23.993936+0000","last_became_peered":"2024-04-21T08:36:23.993936+0000","last_unstale":"2024-04-21T08:38:55.006193+0000","last_undegraded":"2024-04-21T08:38:55.006193+0000","last_fullsized":"2024-04-21T08:38:55.006193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:26:35.330559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.996887+0000","last_change":"2024-04-21T08:36:28.012549+0000","last_active":"2024-04-21T08:38:54.996887+0000","last_peered":"2024-04-21T08:38:54.996887+0000","last_clean":"2024-04-21T08:38:54.996887+0000","last_became_active":"2024-04-21T08:36:28.012151+0000","last_became_peered":"2024-04-21T08:36:28.012151+0000","last_unstale":"2024-04-21T08:38:54.996887+0000","last_undegraded":"2024-04-21T08:38:54.996887+0000","last_fullsized":"2024-04-21T08:38:54.996887+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:55.680399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'8","reported_seq":172,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076760+0000","last_change":"2024-04-21T08:36:23.464586+0000","last_active":"2024-04-21T08:38:55.076760+0000","last_peered":"2024-04-21T08:38:55.076760+0000","last_clean":"2024-04-21T08:38:55.076760+0000","last_became_active":"2024-04-21T08:36:23.464281+0000","last_became_peered":"2024-04-21T08:36:23.464281+0000","last_unstale":"2024-04-21T08:38:55.076760+0000","last_undegraded":"2024-04-21T08:38:55.076760+0000","last_fullsized":"2024-04-21T08:38:55.076760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:04:58.577283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":131,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006132+0000","last_change":"2024-04-21T08:36:23.989604+0000","last_active":"2024-04-21T08:38:55.006132+0000","last_peered":"2024-04-21T08:38:55.006132+0000","last_clean":"2024-04-21T08:38:55.006132+0000","last_became_active":"2024-04-21T08:36:23.989257+0000","last_became_peered":"2024-04-21T08:36:23.989257+0000","last_unstale":"2024-04-21T08:38:55.006132+0000","last_undegraded":"2024-04-21T08:38:55.006132+0000","last_fullsized":"2024-04-21T08:38:55.006132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:45.077943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077854+0000","last_change":"2024-04-21T08:36:28.007959+0000","last_active":"2024-04-21T08:38:55.077854+0000","last_peered":"2024-04-21T08:38:55.077854+0000","last_clean":"2024-04-21T08:38:55.077854+0000","last_became_active":"2024-04-21T08:36:28.007571+0000","last_became_peered":"2024-04-21T08:36:28.007571+0000","last_unstale":"2024-04-21T08:38:55.077854+0000","last_undegraded":"2024-04-21T08:38:55.077854+0000","last_fullsized":"2024-04-21T08:38:55.077854+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:17:34.007013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890204+0000","last_change":"2024-04-21T08:36:23.475054+0000","last_active":"2024-04-21T08:38:54.890204+0000","last_peered":"2024-04-21T08:38:54.890204+0000","last_clean":"2024-04-21T08:38:54.890204+0000","last_became_active":"2024-04-21T08:36:23.474606+0000","last_became_peered":"2024-04-21T08:36:23.474606+0000","last_unstale":"2024-04-21T08:38:54.890204+0000","last_undegraded":"2024-04-21T08:38:54.890204+0000","last_fullsized":"2024-04-21T08:38:54.890204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:22.559174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007909+0000","last_change":"2024-04-21T08:36:23.995980+0000","last_active":"2024-04-21T08:38:55.007909+0000","last_peered":"2024-04-21T08:38:55.007909+0000","last_clean":"2024-04-21T08:38:55.007909+0000","last_became_active":"2024-04-21T08:36:23.995673+0000","last_became_peered":"2024-04-21T08:36:23.995673+0000","last_unstale":"2024-04-21T08:38:55.007909+0000","last_undegraded":"2024-04-21T08:38:55.007909+0000","last_fullsized":"2024-04-21T08:38:55.007909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:11:49.884384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007831+0000","last_change":"2024-04-21T08:36:28.018792+0000","last_active":"2024-04-21T08:38:55.007831+0000","last_peered":"2024-04-21T08:38:55.007831+0000","last_clean":"2024-04-21T08:38:55.007831+0000","last_became_active":"2024-04-21T08:36:28.018373+0000","last_became_peered":"2024-04-21T08:36:28.018373+0000","last_unstale":"2024-04-21T08:38:55.007831+0000","last_undegraded":"2024-04-21T08:38:55.007831+0000","last_fullsized":"2024-04-21T08:38:55.007831+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:11.857661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890763+0000","last_change":"2024-04-21T08:36:23.469787+0000","last_active":"2024-04-21T08:38:54.890763+0000","last_peered":"2024-04-21T08:38:54.890763+0000","last_clean":"2024-04-21T08:38:54.890763+0000","last_became_active":"2024-04-21T08:36:23.469462+0000","last_became_peered":"2024-04-21T08:36:23.469462+0000","last_unstale":"2024-04-21T08:38:54.890763+0000","last_undegraded":"2024-04-21T08:38:54.890763+0000","last_fullsized":"2024-04-21T08:38:54.890763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:33:25.849528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005561+0000","last_change":"2024-04-21T08:36:23.988492+0000","last_active":"2024-04-21T08:38:55.005561+0000","last_peered":"2024-04-21T08:38:55.005561+0000","last_clean":"2024-04-21T08:38:55.005561+0000","last_became_active":"2024-04-21T08:36:23.988133+0000","last_became_peered":"2024-04-21T08:36:23.988133+0000","last_unstale":"2024-04-21T08:38:55.005561+0000","last_undegraded":"2024-04-21T08:38:55.005561+0000","last_fullsized":"2024-04-21T08:38:55.005561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:01:25.320281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890332+0000","last_change":"2024-04-21T08:38:15.415043+0000","last_active":"2024-04-21T08:38:54.890332+0000","last_peered":"2024-04-21T08:38:54.890332+0000","last_clean":"2024-04-21T08:38:54.890332+0000","last_became_active":"2024-04-21T08:38:15.414460+0000","last_became_peered":"2024-04-21T08:38:15.414460+0000","last_unstale":"2024-04-21T08:38:54.890332+0000","last_undegraded":"2024-04-21T08:38:54.890332+0000","last_fullsized":"2024-04-21T08:38:54.890332+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:50:17.202423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.996393+0000","last_change":"2024-04-21T08:37:15.219428+0000","last_active":"2024-04-21T08:38:54.996393+0000","last_peered":"2024-04-21T08:38:54.996393+0000","last_clean":"2024-04-21T08:38:54.996393+0000","last_became_active":"2024-04-21T08:37:15.218923+0000","last_became_peered":"2024-04-21T08:37:15.218923+0000","last_unstale":"2024-04-21T08:38:54.996393+0000","last_undegraded":"2024-04-21T08:38:54.996393+0000","last_fullsized":"2024-04-21T08:38:54.996393+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:07:02.781701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888120+0000","last_change":"2024-04-21T08:36:23.985910+0000","last_active":"2024-04-21T08:38:54.888120+0000","last_peered":"2024-04-21T08:38:54.888120+0000","last_clean":"2024-04-21T08:38:54.888120+0000","last_became_active":"2024-04-21T08:36:23.985605+0000","last_became_peered":"2024-04-21T08:36:23.985605+0000","last_unstale":"2024-04-21T08:38:54.888120+0000","last_undegraded":"2024-04-21T08:38:54.888120+0000","last_fullsized":"2024-04-21T08:38:54.888120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:37:09.367005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077277+0000","last_change":"2024-04-21T08:36:28.015256+0000","last_active":"2024-04-21T08:38:55.077277+0000","last_peered":"2024-04-21T08:38:55.077277+0000","last_clean":"2024-04-21T08:38:55.077277+0000","last_became_active":"2024-04-21T08:36:28.014849+0000","last_became_peered":"2024-04-21T08:36:28.014849+0000","last_unstale":"2024-04-21T08:38:55.077277+0000","last_undegraded":"2024-04-21T08:38:55.077277+0000","last_fullsized":"2024-04-21T08:38:55.077277+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:22:05.466299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"95'6","reported_seq":165,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.075177+0000","last_change":"2024-04-21T08:36:23.474966+0000","last_active":"2024-04-21T08:38:55.075177+0000","last_peered":"2024-04-21T08:38:55.075177+0000","last_clean":"2024-04-21T08:38:55.075177+0000","last_became_active":"2024-04-21T08:36:23.474374+0000","last_became_peered":"2024-04-21T08:36:23.474374+0000","last_unstale":"2024-04-21T08:38:55.075177+0000","last_undegraded":"2024-04-21T08:38:55.075177+0000","last_fullsized":"2024-04-21T08:38:55.075177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:57:20.237966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888138+0000","last_change":"2024-04-21T08:37:15.118887+0000","last_active":"2024-04-21T08:38:54.888138+0000","last_peered":"2024-04-21T08:38:54.888138+0000","last_clean":"2024-04-21T08:38:54.888138+0000","last_became_active":"2024-04-21T08:37:15.118510+0000","last_became_peered":"2024-04-21T08:37:15.118510+0000","last_unstale":"2024-04-21T08:38:54.888138+0000","last_undegraded":"2024-04-21T08:38:54.888138+0000","last_fullsized":"2024-04-21T08:38:54.888138+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:59:27.209083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077841+0000","last_change":"2024-04-21T08:36:28.013169+0000","last_active":"2024-04-21T08:38:55.077841+0000","last_peered":"2024-04-21T08:38:55.077841+0000","last_clean":"2024-04-21T08:38:55.077841+0000","last_became_active":"2024-04-21T08:36:28.012791+0000","last_became_peered":"2024-04-21T08:36:28.012791+0000","last_unstale":"2024-04-21T08:38:55.077841+0000","last_undegraded":"2024-04-21T08:38:55.077841+0000","last_fullsized":"2024-04-21T08:38:55.077841+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:42:56.063159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007983+0000","last_change":"2024-04-21T08:36:23.995209+0000","last_active":"2024-04-21T08:38:55.007983+0000","last_peered":"2024-04-21T08:38:55.007983+0000","last_clean":"2024-04-21T08:38:55.007983+0000","last_became_active":"2024-04-21T08:36:23.994866+0000","last_became_peered":"2024-04-21T08:36:23.994866+0000","last_unstale":"2024-04-21T08:38:55.007983+0000","last_undegraded":"2024-04-21T08:38:55.007983+0000","last_fullsized":"2024-04-21T08:38:55.007983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:33:45.582150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'7","reported_seq":168,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076172+0000","last_change":"2024-04-21T08:36:23.471474+0000","last_active":"2024-04-21T08:38:55.076172+0000","last_peered":"2024-04-21T08:38:55.076172+0000","last_clean":"2024-04-21T08:38:55.076172+0000","last_became_active":"2024-04-21T08:36:23.471179+0000","last_became_peered":"2024-04-21T08:36:23.471179+0000","last_unstale":"2024-04-21T08:38:55.076172+0000","last_undegraded":"2024-04-21T08:38:55.076172+0000","last_fullsized":"2024-04-21T08:38:55.076172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:17.290051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":127,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006553+0000","last_change":"2024-04-21T08:38:15.344332+0000","last_active":"2024-04-21T08:38:55.006553+0000","last_peered":"2024-04-21T08:38:55.006553+0000","last_clean":"2024-04-21T08:38:55.006553+0000","last_became_active":"2024-04-21T08:38:15.339914+0000","last_became_peered":"2024-04-21T08:38:15.339914+0000","last_unstale":"2024-04-21T08:38:55.006553+0000","last_undegraded":"2024-04-21T08:38:55.006553+0000","last_fullsized":"2024-04-21T08:38:55.006553+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:39:24.049323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886832+0000","last_change":"2024-04-21T08:36:23.990272+0000","last_active":"2024-04-21T08:38:54.886832+0000","last_peered":"2024-04-21T08:38:54.886832+0000","last_clean":"2024-04-21T08:38:54.886832+0000","last_became_active":"2024-04-21T08:36:23.989940+0000","last_became_peered":"2024-04-21T08:36:23.989940+0000","last_unstale":"2024-04-21T08:38:54.886832+0000","last_undegraded":"2024-04-21T08:38:54.886832+0000","last_fullsized":"2024-04-21T08:38:54.886832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:27:50.705167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006322+0000","last_change":"2024-04-21T08:36:23.474162+0000","last_active":"2024-04-21T08:38:55.006322+0000","last_peered":"2024-04-21T08:38:55.006322+0000","last_clean":"2024-04-21T08:38:55.006322+0000","last_became_active":"2024-04-21T08:36:23.473802+0000","last_became_peered":"2024-04-21T08:36:23.473802+0000","last_unstale":"2024-04-21T08:38:55.006322+0000","last_undegraded":"2024-04-21T08:38:55.006322+0000","last_fullsized":"2024-04-21T08:38:55.006322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:32:41.918431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":366,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005841+0000","last_change":"2024-04-21T08:35:10.062130+0000","last_active":"2024-04-21T08:38:55.005841+0000","last_peered":"2024-04-21T08:38:55.005841+0000","last_clean":"2024-04-21T08:38:55.005841+0000","last_became_active":"2024-04-21T08:35:10.061727+0000","last_became_peered":"2024-04-21T08:35:10.061727+0000","last_unstale":"2024-04-21T08:38:55.005841+0000","last_undegraded":"2024-04-21T08:38:55.005841+0000","last_fullsized":"2024-04-21T08:38:55.005841+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:31:40.950939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:31:40.950939+0000","last_clean_scrub_stamp":"2024-04-21T08:31:40.950939+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:35:52.360167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008148+0000","last_change":"2024-04-21T08:38:15.415007+0000","last_active":"2024-04-21T08:38:55.008148+0000","last_peered":"2024-04-21T08:38:55.008148+0000","last_clean":"2024-04-21T08:38:55.008148+0000","last_became_active":"2024-04-21T08:38:15.414478+0000","last_became_peered":"2024-04-21T08:38:15.414478+0000","last_unstale":"2024-04-21T08:38:55.008148+0000","last_undegraded":"2024-04-21T08:38:55.008148+0000","last_fullsized":"2024-04-21T08:38:55.008148+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:27.703964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.075651+0000","last_change":"2024-04-21T08:36:23.984332+0000","last_active":"2024-04-21T08:38:55.075651+0000","last_peered":"2024-04-21T08:38:55.075651+0000","last_clean":"2024-04-21T08:38:55.075651+0000","last_became_active":"2024-04-21T08:36:23.984021+0000","last_became_peered":"2024-04-21T08:36:23.984021+0000","last_unstale":"2024-04-21T08:38:55.075651+0000","last_undegraded":"2024-04-21T08:38:55.075651+0000","last_fullsized":"2024-04-21T08:38:55.075651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:05:21.090633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005972+0000","last_change":"2024-04-21T08:36:23.467522+0000","last_active":"2024-04-21T08:38:55.005972+0000","last_peered":"2024-04-21T08:38:55.005972+0000","last_clean":"2024-04-21T08:38:55.005972+0000","last_became_active":"2024-04-21T08:36:23.466379+0000","last_became_peered":"2024-04-21T08:36:23.466379+0000","last_unstale":"2024-04-21T08:38:55.005972+0000","last_undegraded":"2024-04-21T08:38:55.005972+0000","last_fullsized":"2024-04-21T08:38:55.005972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:04.185314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005193+0000","last_change":"2024-04-21T08:36:28.015120+0000","last_active":"2024-04-21T08:38:55.005193+0000","last_peered":"2024-04-21T08:38:55.005193+0000","last_clean":"2024-04-21T08:38:55.005193+0000","last_became_active":"2024-04-21T08:36:28.014701+0000","last_became_peered":"2024-04-21T08:36:28.014701+0000","last_unstale":"2024-04-21T08:38:55.005193+0000","last_undegraded":"2024-04-21T08:38:55.005193+0000","last_fullsized":"2024-04-21T08:38:55.005193+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:23.127622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890642+0000","last_change":"2024-04-21T08:36:23.475335+0000","last_active":"2024-04-21T08:38:54.890642+0000","last_peered":"2024-04-21T08:38:54.890642+0000","last_clean":"2024-04-21T08:38:54.890642+0000","last_became_active":"2024-04-21T08:36:23.474739+0000","last_became_peered":"2024-04-21T08:36:23.474739+0000","last_unstale":"2024-04-21T08:38:54.890642+0000","last_undegraded":"2024-04-21T08:38:54.890642+0000","last_fullsized":"2024-04-21T08:38:54.890642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:36.119649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077395+0000","last_change":"2024-04-21T08:36:23.977229+0000","last_active":"2024-04-21T08:38:55.077395+0000","last_peered":"2024-04-21T08:38:55.077395+0000","last_clean":"2024-04-21T08:38:55.077395+0000","last_became_active":"2024-04-21T08:36:23.976893+0000","last_became_peered":"2024-04-21T08:36:23.976893+0000","last_unstale":"2024-04-21T08:38:55.077395+0000","last_undegraded":"2024-04-21T08:38:55.077395+0000","last_fullsized":"2024-04-21T08:38:55.077395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:31:54.226274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886835+0000","last_change":"2024-04-21T08:36:28.018581+0000","last_active":"2024-04-21T08:38:54.886835+0000","last_peered":"2024-04-21T08:38:54.886835+0000","last_clean":"2024-04-21T08:38:54.886835+0000","last_became_active":"2024-04-21T08:36:28.018021+0000","last_became_peered":"2024-04-21T08:36:28.018021+0000","last_unstale":"2024-04-21T08:38:54.886835+0000","last_undegraded":"2024-04-21T08:38:54.886835+0000","last_fullsized":"2024-04-21T08:38:54.886835+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:29:00.355532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'48","reported_seq":105,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.124941+0000","last_change":"2024-04-21T08:36:23.473339+0000","last_active":"2024-04-21T08:38:55.124941+0000","last_peered":"2024-04-21T08:38:55.124941+0000","last_clean":"2024-04-21T08:38:55.124941+0000","last_became_active":"2024-04-21T08:36:23.472994+0000","last_became_peered":"2024-04-21T08:36:23.472994+0000","last_unstale":"2024-04-21T08:38:55.124941+0000","last_undegraded":"2024-04-21T08:38:55.124941+0000","last_fullsized":"2024-04-21T08:38:55.124941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:42.762273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887378+0000","last_change":"2024-04-21T08:36:23.987354+0000","last_active":"2024-04-21T08:38:54.887378+0000","last_peered":"2024-04-21T08:38:54.887378+0000","last_clean":"2024-04-21T08:38:54.887378+0000","last_became_active":"2024-04-21T08:36:23.987032+0000","last_became_peered":"2024-04-21T08:36:23.987032+0000","last_unstale":"2024-04-21T08:38:54.887378+0000","last_undegraded":"2024-04-21T08:38:54.887378+0000","last_fullsized":"2024-04-21T08:38:54.887378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:01:05.164367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104455+0000","last_change":"2024-04-21T08:36:28.002582+0000","last_active":"2024-04-21T08:38:55.104455+0000","last_peered":"2024-04-21T08:38:55.104455+0000","last_clean":"2024-04-21T08:38:55.104455+0000","last_became_active":"2024-04-21T08:36:28.002133+0000","last_became_peered":"2024-04-21T08:36:28.002133+0000","last_unstale":"2024-04-21T08:38:55.104455+0000","last_undegraded":"2024-04-21T08:38:55.104455+0000","last_fullsized":"2024-04-21T08:38:55.104455+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:30:00.268015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007058+0000","last_change":"2024-04-21T08:36:23.462840+0000","last_active":"2024-04-21T08:38:55.007058+0000","last_peered":"2024-04-21T08:38:55.007058+0000","last_clean":"2024-04-21T08:38:55.007058+0000","last_became_active":"2024-04-21T08:36:23.462391+0000","last_became_peered":"2024-04-21T08:36:23.462391+0000","last_unstale":"2024-04-21T08:38:55.007058+0000","last_undegraded":"2024-04-21T08:38:55.007058+0000","last_fullsized":"2024-04-21T08:38:55.007058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:33.230388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004380+0000","last_change":"2024-04-21T08:36:23.994751+0000","last_active":"2024-04-21T08:38:55.004380+0000","last_peered":"2024-04-21T08:38:55.004380+0000","last_clean":"2024-04-21T08:38:55.004380+0000","last_became_active":"2024-04-21T08:36:23.994384+0000","last_became_peered":"2024-04-21T08:36:23.994384+0000","last_unstale":"2024-04-21T08:38:55.004380+0000","last_undegraded":"2024-04-21T08:38:55.004380+0000","last_fullsized":"2024-04-21T08:38:55.004380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:18.956893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006981+0000","last_change":"2024-04-21T08:36:28.004004+0000","last_active":"2024-04-21T08:38:55.006981+0000","last_peered":"2024-04-21T08:38:55.006981+0000","last_clean":"2024-04-21T08:38:55.006981+0000","last_became_active":"2024-04-21T08:36:28.003547+0000","last_became_peered":"2024-04-21T08:36:28.003547+0000","last_unstale":"2024-04-21T08:38:55.006981+0000","last_undegraded":"2024-04-21T08:38:55.006981+0000","last_fullsized":"2024-04-21T08:38:55.006981+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:21:40.215428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104510+0000","last_change":"2024-04-21T08:37:15.222118+0000","last_active":"2024-04-21T08:38:55.104510+0000","last_peered":"2024-04-21T08:38:55.104510+0000","last_clean":"2024-04-21T08:38:55.104510+0000","last_became_active":"2024-04-21T08:37:15.219173+0000","last_became_peered":"2024-04-21T08:37:15.219173+0000","last_unstale":"2024-04-21T08:38:55.104510+0000","last_undegraded":"2024-04-21T08:38:55.104510+0000","last_fullsized":"2024-04-21T08:38:55.104510+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:12.931076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890299+0000","last_change":"2024-04-21T08:36:23.988020+0000","last_active":"2024-04-21T08:38:54.890299+0000","last_peered":"2024-04-21T08:38:54.890299+0000","last_clean":"2024-04-21T08:38:54.890299+0000","last_became_active":"2024-04-21T08:36:23.987630+0000","last_became_peered":"2024-04-21T08:36:23.987630+0000","last_unstale":"2024-04-21T08:38:54.890299+0000","last_undegraded":"2024-04-21T08:38:54.890299+0000","last_fullsized":"2024-04-21T08:38:54.890299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:06.519001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.997084+0000","last_change":"2024-04-21T08:36:28.006984+0000","last_active":"2024-04-21T08:38:54.997084+0000","last_peered":"2024-04-21T08:38:54.997084+0000","last_clean":"2024-04-21T08:38:54.997084+0000","last_became_active":"2024-04-21T08:36:28.006548+0000","last_became_peered":"2024-04-21T08:36:28.006548+0000","last_unstale":"2024-04-21T08:38:54.997084+0000","last_undegraded":"2024-04-21T08:38:54.997084+0000","last_fullsized":"2024-04-21T08:38:54.997084+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:25:50.870658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886907+0000","last_change":"2024-04-21T08:36:23.476266+0000","last_active":"2024-04-21T08:38:54.886907+0000","last_peered":"2024-04-21T08:38:54.886907+0000","last_clean":"2024-04-21T08:38:54.886907+0000","last_became_active":"2024-04-21T08:36:23.475948+0000","last_became_peered":"2024-04-21T08:36:23.475948+0000","last_unstale":"2024-04-21T08:38:54.886907+0000","last_undegraded":"2024-04-21T08:38:54.886907+0000","last_fullsized":"2024-04-21T08:38:54.886907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:41.529376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005217+0000","last_change":"2024-04-21T08:36:23.981942+0000","last_active":"2024-04-21T08:38:55.005217+0000","last_peered":"2024-04-21T08:38:55.005217+0000","last_clean":"2024-04-21T08:38:55.005217+0000","last_became_active":"2024-04-21T08:36:23.981643+0000","last_became_peered":"2024-04-21T08:36:23.981643+0000","last_unstale":"2024-04-21T08:38:55.005217+0000","last_undegraded":"2024-04-21T08:38:55.005217+0000","last_fullsized":"2024-04-21T08:38:55.005217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:26:24.722695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006307+0000","last_change":"2024-04-21T08:36:28.006980+0000","last_active":"2024-04-21T08:38:55.006307+0000","last_peered":"2024-04-21T08:38:55.006307+0000","last_clean":"2024-04-21T08:38:55.006307+0000","last_became_active":"2024-04-21T08:36:28.006530+0000","last_became_peered":"2024-04-21T08:36:28.006530+0000","last_unstale":"2024-04-21T08:38:55.006307+0000","last_undegraded":"2024-04-21T08:38:55.006307+0000","last_fullsized":"2024-04-21T08:38:55.006307+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:05:57.465612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104573+0000","last_change":"2024-04-21T08:36:23.473253+0000","last_active":"2024-04-21T08:38:55.104573+0000","last_peered":"2024-04-21T08:38:55.104573+0000","last_clean":"2024-04-21T08:38:55.104573+0000","last_became_active":"2024-04-21T08:36:23.472942+0000","last_became_peered":"2024-04-21T08:36:23.472942+0000","last_unstale":"2024-04-21T08:38:55.104573+0000","last_undegraded":"2024-04-21T08:38:55.104573+0000","last_fullsized":"2024-04-21T08:38:55.104573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:32:04.894388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887248+0000","last_change":"2024-04-21T08:36:23.988355+0000","last_active":"2024-04-21T08:38:54.887248+0000","last_peered":"2024-04-21T08:38:54.887248+0000","last_clean":"2024-04-21T08:38:54.887248+0000","last_became_active":"2024-04-21T08:36:23.988049+0000","last_became_peered":"2024-04-21T08:36:23.988049+0000","last_unstale":"2024-04-21T08:38:54.887248+0000","last_undegraded":"2024-04-21T08:38:54.887248+0000","last_fullsized":"2024-04-21T08:38:54.887248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:57.348698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006797+0000","last_change":"2024-04-21T08:36:28.007343+0000","last_active":"2024-04-21T08:38:55.006797+0000","last_peered":"2024-04-21T08:38:55.006797+0000","last_clean":"2024-04-21T08:38:55.006797+0000","last_became_active":"2024-04-21T08:36:28.006719+0000","last_became_peered":"2024-04-21T08:36:28.006719+0000","last_unstale":"2024-04-21T08:38:55.006797+0000","last_undegraded":"2024-04-21T08:38:55.006797+0000","last_fullsized":"2024-04-21T08:38:55.006797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:02:25.734584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890843+0000","last_change":"2024-04-21T08:36:23.472416+0000","last_active":"2024-04-21T08:38:54.890843+0000","last_peered":"2024-04-21T08:38:54.890843+0000","last_clean":"2024-04-21T08:38:54.890843+0000","last_became_active":"2024-04-21T08:36:23.471500+0000","last_became_peered":"2024-04-21T08:36:23.471500+0000","last_unstale":"2024-04-21T08:38:54.890843+0000","last_undegraded":"2024-04-21T08:38:54.890843+0000","last_fullsized":"2024-04-21T08:38:54.890843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:39.706432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006128+0000","last_change":"2024-04-21T08:36:23.990653+0000","last_active":"2024-04-21T08:38:55.006128+0000","last_peered":"2024-04-21T08:38:55.006128+0000","last_clean":"2024-04-21T08:38:55.006128+0000","last_became_active":"2024-04-21T08:36:23.990324+0000","last_became_peered":"2024-04-21T08:36:23.990324+0000","last_unstale":"2024-04-21T08:38:55.006128+0000","last_undegraded":"2024-04-21T08:38:55.006128+0000","last_fullsized":"2024-04-21T08:38:55.006128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:08:59.417850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104492+0000","last_change":"2024-04-21T08:36:28.007956+0000","last_active":"2024-04-21T08:38:55.104492+0000","last_peered":"2024-04-21T08:38:55.104492+0000","last_clean":"2024-04-21T08:38:55.104492+0000","last_became_active":"2024-04-21T08:36:28.007475+0000","last_became_peered":"2024-04-21T08:36:28.007475+0000","last_unstale":"2024-04-21T08:38:55.104492+0000","last_undegraded":"2024-04-21T08:38:55.104492+0000","last_fullsized":"2024-04-21T08:38:55.104492+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:54.039359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'8","reported_seq":170,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104857+0000","last_change":"2024-04-21T08:36:23.476887+0000","last_active":"2024-04-21T08:38:55.104857+0000","last_peered":"2024-04-21T08:38:55.104857+0000","last_clean":"2024-04-21T08:38:55.104857+0000","last_became_active":"2024-04-21T08:36:23.476574+0000","last_became_peered":"2024-04-21T08:36:23.476574+0000","last_unstale":"2024-04-21T08:38:55.104857+0000","last_undegraded":"2024-04-21T08:38:55.104857+0000","last_fullsized":"2024-04-21T08:38:55.104857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:37:47.692358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2010,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006973+0000","last_change":"2024-04-21T08:36:23.982845+0000","last_active":"2024-04-21T08:38:55.006973+0000","last_peered":"2024-04-21T08:38:55.006973+0000","last_clean":"2024-04-21T08:38:55.006973+0000","last_became_active":"2024-04-21T08:36:23.982491+0000","last_became_peered":"2024-04-21T08:36:23.982491+0000","last_unstale":"2024-04-21T08:38:55.006973+0000","last_undegraded":"2024-04-21T08:38:55.006973+0000","last_fullsized":"2024-04-21T08:38:55.006973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:48:46.413896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.002896+0000","last_change":"2024-04-21T08:36:28.010442+0000","last_active":"2024-04-21T08:38:55.002896+0000","last_peered":"2024-04-21T08:38:55.002896+0000","last_clean":"2024-04-21T08:38:55.002896+0000","last_became_active":"2024-04-21T08:36:28.010046+0000","last_became_peered":"2024-04-21T08:36:28.010046+0000","last_unstale":"2024-04-21T08:38:55.002896+0000","last_undegraded":"2024-04-21T08:38:55.002896+0000","last_fullsized":"2024-04-21T08:38:55.002896+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:18:36.382802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"95'41","reported_seq":83,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.122392+0000","last_change":"2024-04-21T08:36:23.473545+0000","last_active":"2024-04-21T08:38:55.122392+0000","last_peered":"2024-04-21T08:38:55.122392+0000","last_clean":"2024-04-21T08:38:55.122392+0000","last_became_active":"2024-04-21T08:36:23.473222+0000","last_became_peered":"2024-04-21T08:36:23.473222+0000","last_unstale":"2024-04-21T08:38:55.122392+0000","last_undegraded":"2024-04-21T08:38:55.122392+0000","last_fullsized":"2024-04-21T08:38:55.122392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:47:53.947187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006600+0000","last_change":"2024-04-21T08:36:23.990959+0000","last_active":"2024-04-21T08:38:55.006600+0000","last_peered":"2024-04-21T08:38:55.006600+0000","last_clean":"2024-04-21T08:38:55.006600+0000","last_became_active":"2024-04-21T08:36:23.990322+0000","last_became_peered":"2024-04-21T08:36:23.990322+0000","last_unstale":"2024-04-21T08:38:55.006600+0000","last_undegraded":"2024-04-21T08:38:55.006600+0000","last_fullsized":"2024-04-21T08:38:55.006600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:12:20.859224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007717+0000","last_change":"2024-04-21T08:36:28.006682+0000","last_active":"2024-04-21T08:38:55.007717+0000","last_peered":"2024-04-21T08:38:55.007717+0000","last_clean":"2024-04-21T08:38:55.007717+0000","last_became_active":"2024-04-21T08:36:28.006229+0000","last_became_peered":"2024-04-21T08:36:28.006229+0000","last_unstale":"2024-04-21T08:38:55.007717+0000","last_undegraded":"2024-04-21T08:38:55.007717+0000","last_fullsized":"2024-04-21T08:38:55.007717+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:30:12.723848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077921+0000","last_change":"2024-04-21T08:36:23.473592+0000","last_active":"2024-04-21T08:38:55.077921+0000","last_peered":"2024-04-21T08:38:55.077921+0000","last_clean":"2024-04-21T08:38:55.077921+0000","last_became_active":"2024-04-21T08:36:23.473283+0000","last_became_peered":"2024-04-21T08:36:23.473283+0000","last_unstale":"2024-04-21T08:38:55.077921+0000","last_undegraded":"2024-04-21T08:38:55.077921+0000","last_fullsized":"2024-04-21T08:38:55.077921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:10.425249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006895+0000","last_change":"2024-04-21T08:36:23.985052+0000","last_active":"2024-04-21T08:38:55.006895+0000","last_peered":"2024-04-21T08:38:55.006895+0000","last_clean":"2024-04-21T08:38:55.006895+0000","last_became_active":"2024-04-21T08:36:23.984742+0000","last_became_peered":"2024-04-21T08:36:23.984742+0000","last_unstale":"2024-04-21T08:38:55.006895+0000","last_undegraded":"2024-04-21T08:38:55.006895+0000","last_fullsized":"2024-04-21T08:38:55.006895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:08:06.082974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005201+0000","last_change":"2024-04-21T08:36:23.980905+0000","last_active":"2024-04-21T08:38:55.005201+0000","last_peered":"2024-04-21T08:38:55.005201+0000","last_clean":"2024-04-21T08:38:55.005201+0000","last_became_active":"2024-04-21T08:36:23.980590+0000","last_became_peered":"2024-04-21T08:36:23.980590+0000","last_unstale":"2024-04-21T08:38:55.005201+0000","last_undegraded":"2024-04-21T08:38:55.005201+0000","last_fullsized":"2024-04-21T08:38:55.005201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:54:59.886014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104725+0000","last_change":"2024-04-21T08:36:23.980883+0000","last_active":"2024-04-21T08:38:55.104725+0000","last_peered":"2024-04-21T08:38:55.104725+0000","last_clean":"2024-04-21T08:38:55.104725+0000","last_became_active":"2024-04-21T08:36:23.980562+0000","last_became_peered":"2024-04-21T08:36:23.980562+0000","last_unstale":"2024-04-21T08:38:55.104725+0000","last_undegraded":"2024-04-21T08:38:55.104725+0000","last_fullsized":"2024-04-21T08:38:55.104725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:14:51.868438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104692+0000","last_change":"2024-04-21T08:36:23.991571+0000","last_active":"2024-04-21T08:38:55.104692+0000","last_peered":"2024-04-21T08:38:55.104692+0000","last_clean":"2024-04-21T08:38:55.104692+0000","last_became_active":"2024-04-21T08:36:23.991273+0000","last_became_peered":"2024-04-21T08:36:23.991273+0000","last_unstale":"2024-04-21T08:38:55.104692+0000","last_undegraded":"2024-04-21T08:38:55.104692+0000","last_fullsized":"2024-04-21T08:38:55.104692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:08:23.895038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.003835+0000","last_change":"2024-04-21T08:36:23.994729+0000","last_active":"2024-04-21T08:38:55.003835+0000","last_peered":"2024-04-21T08:38:55.003835+0000","last_clean":"2024-04-21T08:38:55.003835+0000","last_became_active":"2024-04-21T08:36:23.994405+0000","last_became_peered":"2024-04-21T08:36:23.994405+0000","last_unstale":"2024-04-21T08:38:55.003835+0000","last_undegraded":"2024-04-21T08:38:55.003835+0000","last_fullsized":"2024-04-21T08:38:55.003835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:42.463225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077385+0000","last_change":"2024-04-21T08:36:23.995421+0000","last_active":"2024-04-21T08:38:55.077385+0000","last_peered":"2024-04-21T08:38:55.077385+0000","last_clean":"2024-04-21T08:38:55.077385+0000","last_became_active":"2024-04-21T08:36:23.993674+0000","last_became_peered":"2024-04-21T08:36:23.993674+0000","last_unstale":"2024-04-21T08:38:55.077385+0000","last_undegraded":"2024-04-21T08:38:55.077385+0000","last_fullsized":"2024-04-21T08:38:55.077385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:31.013874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005504+0000","last_change":"2024-04-21T08:36:23.986418+0000","last_active":"2024-04-21T08:38:55.005504+0000","last_peered":"2024-04-21T08:38:55.005504+0000","last_clean":"2024-04-21T08:38:55.005504+0000","last_became_active":"2024-04-21T08:36:23.986095+0000","last_became_peered":"2024-04-21T08:36:23.986095+0000","last_unstale":"2024-04-21T08:38:55.005504+0000","last_undegraded":"2024-04-21T08:38:55.005504+0000","last_fullsized":"2024-04-21T08:38:55.005504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:31:56.456218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104918+0000","last_change":"2024-04-21T08:36:23.982353+0000","last_active":"2024-04-21T08:38:55.104918+0000","last_peered":"2024-04-21T08:38:55.104918+0000","last_clean":"2024-04-21T08:38:55.104918+0000","last_became_active":"2024-04-21T08:36:23.982035+0000","last_became_peered":"2024-04-21T08:36:23.982035+0000","last_unstale":"2024-04-21T08:38:55.104918+0000","last_undegraded":"2024-04-21T08:38:55.104918+0000","last_fullsized":"2024-04-21T08:38:55.104918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:42:44.598407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887832+0000","last_change":"2024-04-21T08:36:23.996371+0000","last_active":"2024-04-21T08:38:54.887832+0000","last_peered":"2024-04-21T08:38:54.887832+0000","last_clean":"2024-04-21T08:38:54.887832+0000","last_became_active":"2024-04-21T08:36:23.995802+0000","last_became_peered":"2024-04-21T08:36:23.995802+0000","last_unstale":"2024-04-21T08:38:54.887832+0000","last_undegraded":"2024-04-21T08:38:54.887832+0000","last_fullsized":"2024-04-21T08:38:54.887832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:52:08.571567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007826+0000","last_change":"2024-04-21T08:36:23.987275+0000","last_active":"2024-04-21T08:38:55.007826+0000","last_peered":"2024-04-21T08:38:55.007826+0000","last_clean":"2024-04-21T08:38:55.007826+0000","last_became_active":"2024-04-21T08:36:23.986967+0000","last_became_peered":"2024-04-21T08:36:23.986967+0000","last_unstale":"2024-04-21T08:38:55.007826+0000","last_undegraded":"2024-04-21T08:38:55.007826+0000","last_fullsized":"2024-04-21T08:38:55.007826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:50.432502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886991+0000","last_change":"2024-04-21T08:36:23.993046+0000","last_active":"2024-04-21T08:38:54.886991+0000","last_peered":"2024-04-21T08:38:54.886991+0000","last_clean":"2024-04-21T08:38:54.886991+0000","last_became_active":"2024-04-21T08:36:23.992693+0000","last_became_peered":"2024-04-21T08:36:23.992693+0000","last_unstale":"2024-04-21T08:38:54.886991+0000","last_undegraded":"2024-04-21T08:38:54.886991+0000","last_fullsized":"2024-04-21T08:38:54.886991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:12:06.287585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104706+0000","last_change":"2024-04-21T08:36:23.992968+0000","last_active":"2024-04-21T08:38:55.104706+0000","last_peered":"2024-04-21T08:38:55.104706+0000","last_clean":"2024-04-21T08:38:55.104706+0000","last_became_active":"2024-04-21T08:36:23.992614+0000","last_became_peered":"2024-04-21T08:36:23.992614+0000","last_unstale":"2024-04-21T08:38:55.104706+0000","last_undegraded":"2024-04-21T08:38:55.104706+0000","last_fullsized":"2024-04-21T08:38:55.104706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:10.015323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076758+0000","last_change":"2024-04-21T08:36:23.981169+0000","last_active":"2024-04-21T08:38:55.076758+0000","last_peered":"2024-04-21T08:38:55.076758+0000","last_clean":"2024-04-21T08:38:55.076758+0000","last_became_active":"2024-04-21T08:36:23.980849+0000","last_became_peered":"2024-04-21T08:36:23.980849+0000","last_unstale":"2024-04-21T08:38:55.076758+0000","last_undegraded":"2024-04-21T08:38:55.076758+0000","last_fullsized":"2024-04-21T08:38:55.076758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:13:49.907370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005358+0000","last_change":"2024-04-21T08:36:23.993998+0000","last_active":"2024-04-21T08:38:55.005358+0000","last_peered":"2024-04-21T08:38:55.005358+0000","last_clean":"2024-04-21T08:38:55.005358+0000","last_became_active":"2024-04-21T08:36:23.993664+0000","last_became_peered":"2024-04-21T08:36:23.993664+0000","last_unstale":"2024-04-21T08:38:55.005358+0000","last_undegraded":"2024-04-21T08:38:55.005358+0000","last_fullsized":"2024-04-21T08:38:55.005358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:31:49.076820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":604383,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1392,"num_read_kb":816,"num_write":377,"num_write_kb":1476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1813149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351019,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1093632,"data_stored":535779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612657,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37220,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707932,"statfs":{"total":95995035648,"available":95956922368,"internally_reserved":0,"allocated":1929216,"data_stored":1324346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776182,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36820,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708332,"statfs":{"total":95995035648,"available":95957331968,"internally_reserved":0,"allocated":1515520,"data_stored":932181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037819,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28496,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716656,"statfs":{"total":95995035648,"available":95965855744,"internally_reserved":0,"allocated":1511424,"data_stored":932559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299455,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1712128,"data_stored":1133590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462980,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1093632,"data_stored":530764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1167360,"data_stored":601862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1118208,"data_stored":532669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1187840,"data_stored":603454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1054897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920847,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":535370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724619,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32984,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712168,"statfs":{"total":95995035648,"available":95961260032,"internally_reserved":0,"allocated":1851392,"data_stored":1255362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":204327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:38:57.392 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T08:38:57.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.530+0000 7ff7289b2640 1 -- 172.21.15.43:0/1256211189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7241011a0 msgr2=0x7ff724101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.530+0000 7ff7289b2640 1 --2- 172.21.15.43:0/1256211189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7241011a0 0x7ff724101600 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff70c009900 tx=0x7ff70c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:57.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.530+0000 7ff7289b2640 1 -- 172.21.15.43:0/1256211189 shutdown_connections 2024-04-21T08:38:57.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.530+0000 7ff7289b2640 1 --2- 172.21.15.43:0/1256211189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff724101b40 0x7ff724101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.530+0000 7ff7289b2640 1 --2- 172.21.15.43:0/1256211189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7241011a0 0x7ff724101600 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.530+0000 7ff7289b2640 1 --2- 172.21.15.43:0/1256211189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff724100840 0x7ff724100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.530+0000 7ff7289b2640 1 -- 172.21.15.43:0/1256211189 >> 172.21.15.43:0/1256211189 conn(0x7ff7240fc3e0 msgr2=0x7ff7240fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:57.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.531+0000 7ff7289b2640 1 -- 172.21.15.43:0/1256211189 shutdown_connections 2024-04-21T08:38:57.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.531+0000 7ff7289b2640 1 -- 172.21.15.43:0/1256211189 wait complete. 2024-04-21T08:38:57.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.531+0000 7ff7289b2640 1 Processor -- start 2024-04-21T08:38:57.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.532+0000 7ff7289b2640 1 -- start start 2024-04-21T08:38:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.532+0000 7ff7289b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff724100840 0x7ff72419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.532+0000 7ff7289b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7241011a0 0x7ff72419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.532+0000 7ff7289b2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff724101b40 0x7ff7241a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.532+0000 7ff7289b2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff72410fcc0 con 0x7ff724100840 2024-04-21T08:38:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.532+0000 7ff7289b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff72410fb40 con 0x7ff7241011a0 2024-04-21T08:38:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.532+0000 7ff7289b2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff72410fe40 con 0x7ff724101b40 2024-04-21T08:38:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.532+0000 7ff72259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff724100840 0x7ff72419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.532+0000 7ff72259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff724100840 0x7ff72419a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38572/0 (socket says 172.21.15.43:38572) 2024-04-21T08:38:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.532+0000 7ff72259c640 1 -- 172.21.15.43:0/926261605 learned_addr learned my addr 172.21.15.43:0/926261605 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.532+0000 7ff722d9d640 1 --2- 172.21.15.43:0/926261605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff724101b40 0x7ff7241a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.532+0000 7ff721d9b640 1 --2- 172.21.15.43:0/926261605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7241011a0 0x7ff72419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.532+0000 7ff72259c640 1 -- 172.21.15.43:0/926261605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff724101b40 msgr2=0x7ff7241a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.532+0000 7ff72259c640 1 --2- 172.21.15.43:0/926261605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff724101b40 0x7ff7241a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.532+0000 7ff72259c640 1 -- 172.21.15.43:0/926261605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7241011a0 msgr2=0x7ff72419a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.532+0000 7ff72259c640 1 --2- 172.21.15.43:0/926261605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7241011a0 0x7ff72419a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.532+0000 7ff72259c640 1 -- 172.21.15.43:0/926261605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7241a1fd0 con 0x7ff724100840 2024-04-21T08:38:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.532+0000 7ff72259c640 1 --2- 172.21.15.43:0/926261605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff724100840 0x7ff72419a400 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff71400e9f0 tx=0x7ff71400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.533+0000 7ff71b7fe640 1 -- 172.21.15.43:0/926261605 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71400cd80 con 0x7ff724100840 2024-04-21T08:38:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.533+0000 7ff7289b2640 1 -- 172.21.15.43:0/926261605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7241a22c0 con 0x7ff724100840 2024-04-21T08:38:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.533+0000 7ff71b7fe640 1 -- 172.21.15.43:0/926261605 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7140108e0 con 0x7ff724100840 2024-04-21T08:38:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.533+0000 7ff71b7fe640 1 -- 172.21.15.43:0/926261605 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714018980 con 0x7ff724100840 2024-04-21T08:38:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.533+0000 7ff7289b2640 1 -- 172.21.15.43:0/926261605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7241a27e0 con 0x7ff724100840 2024-04-21T08:38:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.534+0000 7ff7289b2640 1 -- 172.21.15.43:0/926261605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff72410b7b0 con 0x7ff724100840 2024-04-21T08:38:57.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.535+0000 7ff71b7fe640 1 -- 172.21.15.43:0/926261605 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7ff714010430 con 0x7ff724100840 2024-04-21T08:38:57.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.535+0000 7ff71b7fe640 1 --2- 172.21.15.43:0/926261605 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff7100a3d20 0x7ff7100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:57.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.536+0000 7ff721d9b640 1 --2- 172.21.15.43:0/926261605 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff7100a3d20 0x7ff7100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:57.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.536+0000 7ff71b7fe640 1 -- 172.21.15.43:0/926261605 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7ff7140d13a0 con 0x7ff724100840 2024-04-21T08:38:57.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.536+0000 7ff721d9b640 1 --2- 172.21.15.43:0/926261605 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff7100a3d20 0x7ff7100a61e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff70c009230 tx=0x7ff70c005fd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:57.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.539+0000 7ff71b7fe640 1 -- 172.21.15.43:0/926261605 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff71409d020 con 0x7ff724100840 2024-04-21T08:38:57.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:57 smithi112 ceph-mon[31085]: [21/Apr/2024:08:38:55] ENGINE Serving on https://172.21.15.43:7150 2024-04-21T08:38:57.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:57 smithi112 ceph-mon[31085]: [21/Apr/2024:08:38:55] ENGINE Bus STARTED 2024-04-21T08:38:57.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:57 smithi112 ceph-mon[31085]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:38:57.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:57 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1387283183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:57.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:57 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3411229034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:57.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:57 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3037815156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:57.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:57 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:57.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:57 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:57.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:57 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:57.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:57 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:57.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:57 smithi043 ceph-mon[27978]: [21/Apr/2024:08:38:55] ENGINE Serving on https://172.21.15.43:7150 2024-04-21T08:38:57.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:57 smithi043 ceph-mon[27978]: [21/Apr/2024:08:38:55] ENGINE Bus STARTED 2024-04-21T08:38:57.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:57 smithi043 ceph-mon[27978]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:38:57.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:57 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1387283183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:57.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:57 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3411229034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:57.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:57 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3037815156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:57.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:57 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:57.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:57 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:57.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:57 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:57.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:57 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:57.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:57 smithi175 ceph-mon[31920]: [21/Apr/2024:08:38:55] ENGINE Serving on https://172.21.15.43:7150 2024-04-21T08:38:57.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:57 smithi175 ceph-mon[31920]: [21/Apr/2024:08:38:55] ENGINE Bus STARTED 2024-04-21T08:38:57.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:57 smithi175 ceph-mon[31920]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:38:57.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:57 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1387283183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:57.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:57 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3411229034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:57.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:57 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3037815156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:57.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:57 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:57.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:57 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:57.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:57 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:57.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:57 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:57.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.722+0000 7ff7289b2640 1 -- 172.21.15.43:0/926261605 --> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff724062390 con 0x7ff7100a3d20 2024-04-21T08:38:57.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.732+0000 7ff71b7fe640 1 -- 172.21.15.43:0/926261605 <== mgr.15546 v2:172.21.15.43:6800/1482273906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339423 (secure 0 0 0) 0x7ff724062390 con 0x7ff7100a3d20 2024-04-21T08:38:57.735 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:57.737 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-21T08:38:57.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.735+0000 7ff7289b2640 1 -- 172.21.15.43:0/926261605 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff7100a3d20 msgr2=0x7ff7100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.735+0000 7ff7289b2640 1 --2- 172.21.15.43:0/926261605 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff7100a3d20 0x7ff7100a61e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff70c009230 tx=0x7ff70c005fd0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.735+0000 7ff7289b2640 1 -- 172.21.15.43:0/926261605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff724100840 msgr2=0x7ff72419a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.735+0000 7ff7289b2640 1 --2- 172.21.15.43:0/926261605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff724100840 0x7ff72419a400 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff71400e9f0 tx=0x7ff71400eeb0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.735+0000 7ff7289b2640 1 -- 172.21.15.43:0/926261605 shutdown_connections 2024-04-21T08:38:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.735+0000 7ff7289b2640 1 --2- 172.21.15.43:0/926261605 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff7100a3d20 0x7ff7100a61e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.735+0000 7ff7289b2640 1 --2- 172.21.15.43:0/926261605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff724101b40 0x7ff7241a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.735+0000 7ff7289b2640 1 --2- 172.21.15.43:0/926261605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7241011a0 0x7ff72419a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.735+0000 7ff7289b2640 1 --2- 172.21.15.43:0/926261605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff724100840 0x7ff72419a400 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.735+0000 7ff7289b2640 1 -- 172.21.15.43:0/926261605 >> 172.21.15.43:0/926261605 conn(0x7ff7240fc3e0 msgr2=0x7ff7240fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.735+0000 7ff7289b2640 1 -- 172.21.15.43:0/926261605 shutdown_connections 2024-04-21T08:38:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.735+0000 7ff7289b2640 1 -- 172.21.15.43:0/926261605 wait complete. 2024-04-21T08:38:57.746 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-04-21T08:38:56.901532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1194985,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1528,"num_read_kb":935,"num_write":560,"num_write_kb":4416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":947,"ondisk_log_size":947,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386508,"kb_used_data":16520,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124555316,"statfs":{"total":1151940427776,"available":1151544643584,"internally_reserved":0,"allocated":16916480,"data_stored":9972833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":378713404},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.989687"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005622+0000","last_change":"2024-04-21T08:36:23.983921+0000","last_active":"2024-04-21T08:38:55.005622+0000","last_peered":"2024-04-21T08:38:55.005622+0000","last_clean":"2024-04-21T08:38:55.005622+0000","last_became_active":"2024-04-21T08:36:23.983638+0000","last_became_peered":"2024-04-21T08:36:23.983638+0000","last_unstale":"2024-04-21T08:38:55.005622+0000","last_undegraded":"2024-04-21T08:38:55.005622+0000","last_fullsized":"2024-04-21T08:38:55.005622+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:29:44.646308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004430+0000","last_change":"2024-04-21T08:36:23.990029+0000","last_active":"2024-04-21T08:38:55.004430+0000","last_peered":"2024-04-21T08:38:55.004430+0000","last_clean":"2024-04-21T08:38:55.004430+0000","last_became_active":"2024-04-21T08:36:23.989141+0000","last_became_peered":"2024-04-21T08:36:23.989141+0000","last_unstale":"2024-04-21T08:38:55.004430+0000","last_undegraded":"2024-04-21T08:38:55.004430+0000","last_fullsized":"2024-04-21T08:38:55.004430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:23:58.692342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004234+0000","last_change":"2024-04-21T08:36:23.982205+0000","last_active":"2024-04-21T08:38:55.004234+0000","last_peered":"2024-04-21T08:38:55.004234+0000","last_clean":"2024-04-21T08:38:55.004234+0000","last_became_active":"2024-04-21T08:36:23.981876+0000","last_became_peered":"2024-04-21T08:36:23.981876+0000","last_unstale":"2024-04-21T08:38:55.004234+0000","last_undegraded":"2024-04-21T08:38:55.004234+0000","last_fullsized":"2024-04-21T08:38:55.004234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:12:27.121106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006754+0000","last_change":"2024-04-21T08:36:23.987425+0000","last_active":"2024-04-21T08:38:55.006754+0000","last_peered":"2024-04-21T08:38:55.006754+0000","last_clean":"2024-04-21T08:38:55.006754+0000","last_became_active":"2024-04-21T08:36:23.987109+0000","last_became_peered":"2024-04-21T08:36:23.987109+0000","last_unstale":"2024-04-21T08:38:55.006754+0000","last_undegraded":"2024-04-21T08:38:55.006754+0000","last_fullsized":"2024-04-21T08:38:55.006754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:44.176324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890536+0000","last_change":"2024-04-21T08:36:23.990375+0000","last_active":"2024-04-21T08:38:54.890536+0000","last_peered":"2024-04-21T08:38:54.890536+0000","last_clean":"2024-04-21T08:38:54.890536+0000","last_became_active":"2024-04-21T08:36:23.990058+0000","last_became_peered":"2024-04-21T08:36:23.990058+0000","last_unstale":"2024-04-21T08:38:54.890536+0000","last_undegraded":"2024-04-21T08:38:54.890536+0000","last_fullsized":"2024-04-21T08:38:54.890536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:24:09.647528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888221+0000","last_change":"2024-04-21T08:36:23.979848+0000","last_active":"2024-04-21T08:38:54.888221+0000","last_peered":"2024-04-21T08:38:54.888221+0000","last_clean":"2024-04-21T08:38:54.888221+0000","last_became_active":"2024-04-21T08:36:23.979511+0000","last_became_peered":"2024-04-21T08:36:23.979511+0000","last_unstale":"2024-04-21T08:38:54.888221+0000","last_undegraded":"2024-04-21T08:38:54.888221+0000","last_fullsized":"2024-04-21T08:38:54.888221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:07:46.551369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006232+0000","last_change":"2024-04-21T08:37:15.118755+0000","last_active":"2024-04-21T08:38:55.006232+0000","last_peered":"2024-04-21T08:38:55.006232+0000","last_clean":"2024-04-21T08:38:55.006232+0000","last_became_active":"2024-04-21T08:37:15.118347+0000","last_became_peered":"2024-04-21T08:37:15.118347+0000","last_unstale":"2024-04-21T08:38:55.006232+0000","last_undegraded":"2024-04-21T08:38:55.006232+0000","last_fullsized":"2024-04-21T08:38:55.006232+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:36:43.398325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886870+0000","last_change":"2024-04-21T08:36:23.987517+0000","last_active":"2024-04-21T08:38:54.886870+0000","last_peered":"2024-04-21T08:38:54.886870+0000","last_clean":"2024-04-21T08:38:54.886870+0000","last_became_active":"2024-04-21T08:36:23.987201+0000","last_became_peered":"2024-04-21T08:36:23.987201+0000","last_unstale":"2024-04-21T08:38:54.886870+0000","last_undegraded":"2024-04-21T08:38:54.886870+0000","last_fullsized":"2024-04-21T08:38:54.886870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:42:53.080863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104900+0000","last_change":"2024-04-21T08:36:23.989434+0000","last_active":"2024-04-21T08:38:55.104900+0000","last_peered":"2024-04-21T08:38:55.104900+0000","last_clean":"2024-04-21T08:38:55.104900+0000","last_became_active":"2024-04-21T08:36:23.989125+0000","last_became_peered":"2024-04-21T08:36:23.989125+0000","last_unstale":"2024-04-21T08:38:55.104900+0000","last_undegraded":"2024-04-21T08:38:55.104900+0000","last_fullsized":"2024-04-21T08:38:55.104900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:30:07.219056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888449+0000","last_change":"2024-04-21T08:36:23.980725+0000","last_active":"2024-04-21T08:38:54.888449+0000","last_peered":"2024-04-21T08:38:54.888449+0000","last_clean":"2024-04-21T08:38:54.888449+0000","last_became_active":"2024-04-21T08:36:23.980363+0000","last_became_peered":"2024-04-21T08:36:23.980363+0000","last_unstale":"2024-04-21T08:38:54.888449+0000","last_undegraded":"2024-04-21T08:38:54.888449+0000","last_fullsized":"2024-04-21T08:38:54.888449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:26:46.202030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006978+0000","last_change":"2024-04-21T08:36:23.989369+0000","last_active":"2024-04-21T08:38:55.006978+0000","last_peered":"2024-04-21T08:38:55.006978+0000","last_clean":"2024-04-21T08:38:55.006978+0000","last_became_active":"2024-04-21T08:36:23.989030+0000","last_became_peered":"2024-04-21T08:36:23.989030+0000","last_unstale":"2024-04-21T08:38:55.006978+0000","last_undegraded":"2024-04-21T08:38:55.006978+0000","last_fullsized":"2024-04-21T08:38:55.006978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:23:13.725703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006691+0000","last_change":"2024-04-21T08:36:23.993485+0000","last_active":"2024-04-21T08:38:55.006691+0000","last_peered":"2024-04-21T08:38:55.006691+0000","last_clean":"2024-04-21T08:38:55.006691+0000","last_became_active":"2024-04-21T08:36:23.993124+0000","last_became_peered":"2024-04-21T08:36:23.993124+0000","last_unstale":"2024-04-21T08:38:55.006691+0000","last_undegraded":"2024-04-21T08:38:55.006691+0000","last_fullsized":"2024-04-21T08:38:55.006691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:53:16.249838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076569+0000","last_change":"2024-04-21T08:36:23.983146+0000","last_active":"2024-04-21T08:38:55.076569+0000","last_peered":"2024-04-21T08:38:55.076569+0000","last_clean":"2024-04-21T08:38:55.076569+0000","last_became_active":"2024-04-21T08:36:23.982766+0000","last_became_peered":"2024-04-21T08:36:23.982766+0000","last_unstale":"2024-04-21T08:38:55.076569+0000","last_undegraded":"2024-04-21T08:38:55.076569+0000","last_fullsized":"2024-04-21T08:38:55.076569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:30:44.090929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.003129+0000","last_change":"2024-04-21T08:36:23.988906+0000","last_active":"2024-04-21T08:38:55.003129+0000","last_peered":"2024-04-21T08:38:55.003129+0000","last_clean":"2024-04-21T08:38:55.003129+0000","last_became_active":"2024-04-21T08:36:23.988489+0000","last_became_peered":"2024-04-21T08:36:23.988489+0000","last_unstale":"2024-04-21T08:38:55.003129+0000","last_undegraded":"2024-04-21T08:38:55.003129+0000","last_fullsized":"2024-04-21T08:38:55.003129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:06:43.271650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.003060+0000","last_change":"2024-04-21T08:37:15.123343+0000","last_active":"2024-04-21T08:38:55.003060+0000","last_peered":"2024-04-21T08:38:55.003060+0000","last_clean":"2024-04-21T08:38:55.003060+0000","last_became_active":"2024-04-21T08:37:15.122959+0000","last_became_peered":"2024-04-21T08:37:15.122959+0000","last_unstale":"2024-04-21T08:38:55.003060+0000","last_undegraded":"2024-04-21T08:38:55.003060+0000","last_fullsized":"2024-04-21T08:38:55.003060+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:56:19.179879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104471+0000","last_change":"2024-04-21T08:36:23.983136+0000","last_active":"2024-04-21T08:38:55.104471+0000","last_peered":"2024-04-21T08:38:55.104471+0000","last_clean":"2024-04-21T08:38:55.104471+0000","last_became_active":"2024-04-21T08:36:23.982698+0000","last_became_peered":"2024-04-21T08:36:23.982698+0000","last_unstale":"2024-04-21T08:38:55.104471+0000","last_undegraded":"2024-04-21T08:38:55.104471+0000","last_fullsized":"2024-04-21T08:38:55.104471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:34.750941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005563+0000","last_change":"2024-04-21T08:36:23.984701+0000","last_active":"2024-04-21T08:38:55.005563+0000","last_peered":"2024-04-21T08:38:55.005563+0000","last_clean":"2024-04-21T08:38:55.005563+0000","last_became_active":"2024-04-21T08:36:23.984432+0000","last_became_peered":"2024-04-21T08:36:23.984432+0000","last_unstale":"2024-04-21T08:38:55.005563+0000","last_undegraded":"2024-04-21T08:38:55.005563+0000","last_fullsized":"2024-04-21T08:38:55.005563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:10:58.944380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.996823+0000","last_change":"2024-04-21T08:36:23.982781+0000","last_active":"2024-04-21T08:38:54.996823+0000","last_peered":"2024-04-21T08:38:54.996823+0000","last_clean":"2024-04-21T08:38:54.996823+0000","last_became_active":"2024-04-21T08:36:23.982459+0000","last_became_peered":"2024-04-21T08:36:23.982459+0000","last_unstale":"2024-04-21T08:38:54.996823+0000","last_undegraded":"2024-04-21T08:38:54.996823+0000","last_fullsized":"2024-04-21T08:38:54.996823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:44.657504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.105000+0000","last_change":"2024-04-21T08:36:23.992002+0000","last_active":"2024-04-21T08:38:55.105000+0000","last_peered":"2024-04-21T08:38:55.105000+0000","last_clean":"2024-04-21T08:38:55.105000+0000","last_became_active":"2024-04-21T08:36:23.991053+0000","last_became_peered":"2024-04-21T08:36:23.991053+0000","last_unstale":"2024-04-21T08:38:55.105000+0000","last_undegraded":"2024-04-21T08:38:55.105000+0000","last_fullsized":"2024-04-21T08:38:55.105000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:56:37.145949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104844+0000","last_change":"2024-04-21T08:36:28.013195+0000","last_active":"2024-04-21T08:38:55.104844+0000","last_peered":"2024-04-21T08:38:55.104844+0000","last_clean":"2024-04-21T08:38:55.104844+0000","last_became_active":"2024-04-21T08:36:28.012656+0000","last_became_peered":"2024-04-21T08:36:28.012656+0000","last_unstale":"2024-04-21T08:38:55.104844+0000","last_undegraded":"2024-04-21T08:38:55.104844+0000","last_fullsized":"2024-04-21T08:38:55.104844+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:12:03.133583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"95'14","reported_seq":173,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.074681+0000","last_change":"2024-04-21T08:36:23.468957+0000","last_active":"2024-04-21T08:38:55.074681+0000","last_peered":"2024-04-21T08:38:55.074681+0000","last_clean":"2024-04-21T08:38:55.074681+0000","last_became_active":"2024-04-21T08:36:23.468601+0000","last_became_peered":"2024-04-21T08:36:23.468601+0000","last_unstale":"2024-04-21T08:38:55.074681+0000","last_undegraded":"2024-04-21T08:38:55.074681+0000","last_fullsized":"2024-04-21T08:38:55.074681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:13:11.977780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":125,"num_read_kb":125,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.996013+0000","last_change":"2024-04-21T08:37:15.123476+0000","last_active":"2024-04-21T08:38:54.996013+0000","last_peered":"2024-04-21T08:38:54.996013+0000","last_clean":"2024-04-21T08:38:54.996013+0000","last_became_active":"2024-04-21T08:37:15.123066+0000","last_became_peered":"2024-04-21T08:37:15.123066+0000","last_unstale":"2024-04-21T08:38:54.996013+0000","last_undegraded":"2024-04-21T08:38:54.996013+0000","last_fullsized":"2024-04-21T08:38:54.996013+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:15.786178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005900+0000","last_change":"2024-04-21T08:36:28.014161+0000","last_active":"2024-04-21T08:38:55.005900+0000","last_peered":"2024-04-21T08:38:55.005900+0000","last_clean":"2024-04-21T08:38:55.005900+0000","last_became_active":"2024-04-21T08:36:28.013859+0000","last_became_peered":"2024-04-21T08:36:28.013859+0000","last_unstale":"2024-04-21T08:38:55.005900+0000","last_undegraded":"2024-04-21T08:38:55.005900+0000","last_fullsized":"2024-04-21T08:38:55.005900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:08.315334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887665+0000","last_change":"2024-04-21T08:36:23.475612+0000","last_active":"2024-04-21T08:38:54.887665+0000","last_peered":"2024-04-21T08:38:54.887665+0000","last_clean":"2024-04-21T08:38:54.887665+0000","last_became_active":"2024-04-21T08:36:23.475301+0000","last_became_peered":"2024-04-21T08:36:23.475301+0000","last_unstale":"2024-04-21T08:38:54.887665+0000","last_undegraded":"2024-04-21T08:38:54.887665+0000","last_fullsized":"2024-04-21T08:38:54.887665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:56.801630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004414+0000","last_change":"2024-04-21T08:36:23.984689+0000","last_active":"2024-04-21T08:38:55.004414+0000","last_peered":"2024-04-21T08:38:55.004414+0000","last_clean":"2024-04-21T08:38:55.004414+0000","last_became_active":"2024-04-21T08:36:23.984358+0000","last_became_peered":"2024-04-21T08:36:23.984358+0000","last_unstale":"2024-04-21T08:38:55.004414+0000","last_undegraded":"2024-04-21T08:38:55.004414+0000","last_fullsized":"2024-04-21T08:38:55.004414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:49:50.187466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006203+0000","last_change":"2024-04-21T08:36:28.007951+0000","last_active":"2024-04-21T08:38:55.006203+0000","last_peered":"2024-04-21T08:38:55.006203+0000","last_clean":"2024-04-21T08:38:55.006203+0000","last_became_active":"2024-04-21T08:36:28.007553+0000","last_became_peered":"2024-04-21T08:36:28.007553+0000","last_unstale":"2024-04-21T08:38:55.006203+0000","last_undegraded":"2024-04-21T08:38:55.006203+0000","last_fullsized":"2024-04-21T08:38:55.006203+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:10:54.019605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890572+0000","last_change":"2024-04-21T08:36:23.473371+0000","last_active":"2024-04-21T08:38:54.890572+0000","last_peered":"2024-04-21T08:38:54.890572+0000","last_clean":"2024-04-21T08:38:54.890572+0000","last_became_active":"2024-04-21T08:36:23.472472+0000","last_became_peered":"2024-04-21T08:36:23.472472+0000","last_unstale":"2024-04-21T08:38:54.890572+0000","last_undegraded":"2024-04-21T08:38:54.890572+0000","last_fullsized":"2024-04-21T08:38:54.890572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:28:49.622873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104901+0000","last_change":"2024-04-21T08:36:23.987916+0000","last_active":"2024-04-21T08:38:55.104901+0000","last_peered":"2024-04-21T08:38:55.104901+0000","last_clean":"2024-04-21T08:38:55.104901+0000","last_became_active":"2024-04-21T08:36:23.986207+0000","last_became_peered":"2024-04-21T08:36:23.986207+0000","last_unstale":"2024-04-21T08:38:55.104901+0000","last_undegraded":"2024-04-21T08:38:55.104901+0000","last_fullsized":"2024-04-21T08:38:55.104901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:49:57.261078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006246+0000","last_change":"2024-04-21T08:36:28.006494+0000","last_active":"2024-04-21T08:38:55.006246+0000","last_peered":"2024-04-21T08:38:55.006246+0000","last_clean":"2024-04-21T08:38:55.006246+0000","last_became_active":"2024-04-21T08:36:28.006069+0000","last_became_peered":"2024-04-21T08:36:28.006069+0000","last_unstale":"2024-04-21T08:38:55.006246+0000","last_undegraded":"2024-04-21T08:38:55.006246+0000","last_fullsized":"2024-04-21T08:38:55.006246+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:04.608295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890975+0000","last_change":"2024-04-21T08:36:23.468923+0000","last_active":"2024-04-21T08:38:54.890975+0000","last_peered":"2024-04-21T08:38:54.890975+0000","last_clean":"2024-04-21T08:38:54.890975+0000","last_became_active":"2024-04-21T08:36:23.468509+0000","last_became_peered":"2024-04-21T08:36:23.468509+0000","last_unstale":"2024-04-21T08:38:54.890975+0000","last_undegraded":"2024-04-21T08:38:54.890975+0000","last_fullsized":"2024-04-21T08:38:54.890975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:44.452748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008259+0000","last_change":"2024-04-21T08:36:23.994423+0000","last_active":"2024-04-21T08:38:55.008259+0000","last_peered":"2024-04-21T08:38:55.008259+0000","last_clean":"2024-04-21T08:38:55.008259+0000","last_became_active":"2024-04-21T08:36:23.993827+0000","last_became_peered":"2024-04-21T08:36:23.993827+0000","last_unstale":"2024-04-21T08:38:55.008259+0000","last_undegraded":"2024-04-21T08:38:55.008259+0000","last_fullsized":"2024-04-21T08:38:55.008259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:49:25.550301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104711+0000","last_change":"2024-04-21T08:36:28.004037+0000","last_active":"2024-04-21T08:38:55.104711+0000","last_peered":"2024-04-21T08:38:55.104711+0000","last_clean":"2024-04-21T08:38:55.104711+0000","last_became_active":"2024-04-21T08:36:28.003580+0000","last_became_peered":"2024-04-21T08:36:28.003580+0000","last_unstale":"2024-04-21T08:38:55.104711+0000","last_undegraded":"2024-04-21T08:38:55.104711+0000","last_fullsized":"2024-04-21T08:38:55.104711+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:04:51.380244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'8","reported_seq":172,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.181108+0000","last_change":"2024-04-21T08:36:23.468430+0000","last_active":"2024-04-21T08:38:55.181108+0000","last_peered":"2024-04-21T08:38:55.181108+0000","last_clean":"2024-04-21T08:38:55.181108+0000","last_became_active":"2024-04-21T08:36:23.468089+0000","last_became_peered":"2024-04-21T08:36:23.468089+0000","last_unstale":"2024-04-21T08:38:55.181108+0000","last_undegraded":"2024-04-21T08:38:55.181108+0000","last_fullsized":"2024-04-21T08:38:55.181108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:12:54.827034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":130,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890458+0000","last_change":"2024-04-21T08:37:15.117922+0000","last_active":"2024-04-21T08:38:54.890458+0000","last_peered":"2024-04-21T08:38:54.890458+0000","last_clean":"2024-04-21T08:38:54.890458+0000","last_became_active":"2024-04-21T08:37:15.117404+0000","last_became_peered":"2024-04-21T08:37:15.117404+0000","last_unstale":"2024-04-21T08:38:54.890458+0000","last_undegraded":"2024-04-21T08:38:54.890458+0000","last_fullsized":"2024-04-21T08:38:54.890458+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:39.682224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104672+0000","last_change":"2024-04-21T08:36:28.004638+0000","last_active":"2024-04-21T08:38:55.104672+0000","last_peered":"2024-04-21T08:38:55.104672+0000","last_clean":"2024-04-21T08:38:55.104672+0000","last_became_active":"2024-04-21T08:36:28.002499+0000","last_became_peered":"2024-04-21T08:36:28.002499+0000","last_unstale":"2024-04-21T08:38:55.104672+0000","last_undegraded":"2024-04-21T08:38:55.104672+0000","last_fullsized":"2024-04-21T08:38:55.104672+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:45:46.934783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":55,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888089+0000","last_change":"2024-04-21T08:37:15.126569+0000","last_active":"2024-04-21T08:38:54.888089+0000","last_peered":"2024-04-21T08:38:54.888089+0000","last_clean":"2024-04-21T08:38:54.888089+0000","last_became_active":"2024-04-21T08:37:15.121284+0000","last_became_peered":"2024-04-21T08:37:15.121284+0000","last_unstale":"2024-04-21T08:38:54.888089+0000","last_undegraded":"2024-04-21T08:38:54.888089+0000","last_fullsized":"2024-04-21T08:38:54.888089+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:02:26.153324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008374+0000","last_change":"2024-04-21T08:37:15.557192+0000","last_active":"2024-04-21T08:38:55.008374+0000","last_peered":"2024-04-21T08:38:55.008374+0000","last_clean":"2024-04-21T08:38:55.008374+0000","last_became_active":"2024-04-21T08:37:15.556653+0000","last_became_peered":"2024-04-21T08:37:15.556653+0000","last_unstale":"2024-04-21T08:38:55.008374+0000","last_undegraded":"2024-04-21T08:38:55.008374+0000","last_fullsized":"2024-04-21T08:38:55.008374+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:42.722460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005527+0000","last_change":"2024-04-21T08:36:28.017068+0000","last_active":"2024-04-21T08:38:55.005527+0000","last_peered":"2024-04-21T08:38:55.005527+0000","last_clean":"2024-04-21T08:38:55.005527+0000","last_became_active":"2024-04-21T08:36:28.016770+0000","last_became_peered":"2024-04-21T08:36:28.016770+0000","last_unstale":"2024-04-21T08:38:55.005527+0000","last_undegraded":"2024-04-21T08:38:55.005527+0000","last_fullsized":"2024-04-21T08:38:55.005527+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:07:14.596090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076311+0000","last_change":"2024-04-21T08:36:23.470430+0000","last_active":"2024-04-21T08:38:55.076311+0000","last_peered":"2024-04-21T08:38:55.076311+0000","last_clean":"2024-04-21T08:38:55.076311+0000","last_became_active":"2024-04-21T08:36:23.470109+0000","last_became_peered":"2024-04-21T08:36:23.470109+0000","last_unstale":"2024-04-21T08:38:55.076311+0000","last_undegraded":"2024-04-21T08:38:55.076311+0000","last_fullsized":"2024-04-21T08:38:55.076311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:18.176313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887026+0000","last_change":"2024-04-21T08:36:23.986668+0000","last_active":"2024-04-21T08:38:54.887026+0000","last_peered":"2024-04-21T08:38:54.887026+0000","last_clean":"2024-04-21T08:38:54.887026+0000","last_became_active":"2024-04-21T08:36:23.986344+0000","last_became_peered":"2024-04-21T08:36:23.986344+0000","last_unstale":"2024-04-21T08:38:54.887026+0000","last_undegraded":"2024-04-21T08:38:54.887026+0000","last_fullsized":"2024-04-21T08:38:54.887026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:56:17.744145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004263+0000","last_change":"2024-04-21T08:36:28.009012+0000","last_active":"2024-04-21T08:38:55.004263+0000","last_peered":"2024-04-21T08:38:55.004263+0000","last_clean":"2024-04-21T08:38:55.004263+0000","last_became_active":"2024-04-21T08:36:28.008123+0000","last_became_peered":"2024-04-21T08:36:28.008123+0000","last_unstale":"2024-04-21T08:38:55.004263+0000","last_undegraded":"2024-04-21T08:38:55.004263+0000","last_fullsized":"2024-04-21T08:38:55.004263+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:33:18.881564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007854+0000","last_change":"2024-04-21T08:36:23.467104+0000","last_active":"2024-04-21T08:38:55.007854+0000","last_peered":"2024-04-21T08:38:55.007854+0000","last_clean":"2024-04-21T08:38:55.007854+0000","last_became_active":"2024-04-21T08:36:23.466725+0000","last_became_peered":"2024-04-21T08:36:23.466725+0000","last_unstale":"2024-04-21T08:38:55.007854+0000","last_undegraded":"2024-04-21T08:38:55.007854+0000","last_fullsized":"2024-04-21T08:38:55.007854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:35:45.986448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.995844+0000","last_change":"2024-04-21T08:36:23.987579+0000","last_active":"2024-04-21T08:38:54.995844+0000","last_peered":"2024-04-21T08:38:54.995844+0000","last_clean":"2024-04-21T08:38:54.995844+0000","last_became_active":"2024-04-21T08:36:23.987044+0000","last_became_peered":"2024-04-21T08:36:23.987044+0000","last_unstale":"2024-04-21T08:38:54.995844+0000","last_undegraded":"2024-04-21T08:38:54.995844+0000","last_fullsized":"2024-04-21T08:38:54.995844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:48:27.593090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886895+0000","last_change":"2024-04-21T08:36:28.006417+0000","last_active":"2024-04-21T08:38:54.886895+0000","last_peered":"2024-04-21T08:38:54.886895+0000","last_clean":"2024-04-21T08:38:54.886895+0000","last_became_active":"2024-04-21T08:36:28.006006+0000","last_became_peered":"2024-04-21T08:36:28.006006+0000","last_unstale":"2024-04-21T08:38:54.886895+0000","last_undegraded":"2024-04-21T08:38:54.886895+0000","last_fullsized":"2024-04-21T08:38:54.886895+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:16.173284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'8","reported_seq":292,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.181085+0000","last_change":"2024-04-21T08:36:23.471541+0000","last_active":"2024-04-21T08:38:55.181085+0000","last_peered":"2024-04-21T08:38:55.181085+0000","last_clean":"2024-04-21T08:38:55.181085+0000","last_became_active":"2024-04-21T08:36:23.471193+0000","last_became_peered":"2024-04-21T08:36:23.471193+0000","last_unstale":"2024-04-21T08:38:55.181085+0000","last_undegraded":"2024-04-21T08:38:55.181085+0000","last_fullsized":"2024-04-21T08:38:55.181085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:06:04.916707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":250,"num_read_kb":250,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104848+0000","last_change":"2024-04-21T08:36:23.986781+0000","last_active":"2024-04-21T08:38:55.104848+0000","last_peered":"2024-04-21T08:38:55.104848+0000","last_clean":"2024-04-21T08:38:55.104848+0000","last_became_active":"2024-04-21T08:36:23.985696+0000","last_became_peered":"2024-04-21T08:36:23.985696+0000","last_unstale":"2024-04-21T08:38:55.104848+0000","last_undegraded":"2024-04-21T08:38:55.104848+0000","last_fullsized":"2024-04-21T08:38:55.104848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:40.556642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.995747+0000","last_change":"2024-04-21T08:36:28.011621+0000","last_active":"2024-04-21T08:38:54.995747+0000","last_peered":"2024-04-21T08:38:54.995747+0000","last_clean":"2024-04-21T08:38:54.995747+0000","last_became_active":"2024-04-21T08:36:28.011136+0000","last_became_peered":"2024-04-21T08:36:28.011136+0000","last_unstale":"2024-04-21T08:38:54.995747+0000","last_undegraded":"2024-04-21T08:38:54.995747+0000","last_fullsized":"2024-04-21T08:38:54.995747+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:47.623445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006842+0000","last_change":"2024-04-21T08:36:23.474019+0000","last_active":"2024-04-21T08:38:55.006842+0000","last_peered":"2024-04-21T08:38:55.006842+0000","last_clean":"2024-04-21T08:38:55.006842+0000","last_became_active":"2024-04-21T08:36:23.473690+0000","last_became_peered":"2024-04-21T08:36:23.473690+0000","last_unstale":"2024-04-21T08:38:55.006842+0000","last_undegraded":"2024-04-21T08:38:55.006842+0000","last_fullsized":"2024-04-21T08:38:55.006842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:38:32.913795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004613+0000","last_change":"2024-04-21T08:37:15.121455+0000","last_active":"2024-04-21T08:38:55.004613+0000","last_peered":"2024-04-21T08:38:55.004613+0000","last_clean":"2024-04-21T08:38:55.004613+0000","last_became_active":"2024-04-21T08:37:15.121113+0000","last_became_peered":"2024-04-21T08:37:15.121113+0000","last_unstale":"2024-04-21T08:38:55.004613+0000","last_undegraded":"2024-04-21T08:38:55.004613+0000","last_fullsized":"2024-04-21T08:38:55.004613+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:39:36.525495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006512+0000","last_change":"2024-04-21T08:38:15.413849+0000","last_active":"2024-04-21T08:38:55.006512+0000","last_peered":"2024-04-21T08:38:55.006512+0000","last_clean":"2024-04-21T08:38:55.006512+0000","last_became_active":"2024-04-21T08:38:15.413361+0000","last_became_peered":"2024-04-21T08:38:15.413361+0000","last_unstale":"2024-04-21T08:38:55.006512+0000","last_undegraded":"2024-04-21T08:38:55.006512+0000","last_fullsized":"2024-04-21T08:38:55.006512+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:16:49.197800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008173+0000","last_change":"2024-04-21T08:36:23.465825+0000","last_active":"2024-04-21T08:38:55.008173+0000","last_peered":"2024-04-21T08:38:55.008173+0000","last_clean":"2024-04-21T08:38:55.008173+0000","last_became_active":"2024-04-21T08:36:23.465525+0000","last_became_peered":"2024-04-21T08:36:23.465525+0000","last_unstale":"2024-04-21T08:38:55.008173+0000","last_undegraded":"2024-04-21T08:38:55.008173+0000","last_fullsized":"2024-04-21T08:38:55.008173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:59:50.502210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.105005+0000","last_change":"2024-04-21T08:36:23.987602+0000","last_active":"2024-04-21T08:38:55.105005+0000","last_peered":"2024-04-21T08:38:55.105005+0000","last_clean":"2024-04-21T08:38:55.105005+0000","last_became_active":"2024-04-21T08:36:23.987298+0000","last_became_peered":"2024-04-21T08:36:23.987298+0000","last_unstale":"2024-04-21T08:38:55.105005+0000","last_undegraded":"2024-04-21T08:38:55.105005+0000","last_fullsized":"2024-04-21T08:38:55.105005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:06.647676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.995946+0000","last_change":"2024-04-21T08:36:28.005935+0000","last_active":"2024-04-21T08:38:54.995946+0000","last_peered":"2024-04-21T08:38:54.995946+0000","last_clean":"2024-04-21T08:38:54.995946+0000","last_became_active":"2024-04-21T08:36:28.005512+0000","last_became_peered":"2024-04-21T08:36:28.005512+0000","last_unstale":"2024-04-21T08:38:54.995946+0000","last_undegraded":"2024-04-21T08:38:54.995946+0000","last_fullsized":"2024-04-21T08:38:54.995946+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:23:20.298524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"95'43","reported_seq":341,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.178912+0000","last_change":"2024-04-21T08:36:23.467728+0000","last_active":"2024-04-21T08:38:55.178912+0000","last_peered":"2024-04-21T08:38:55.178912+0000","last_clean":"2024-04-21T08:38:55.178912+0000","last_became_active":"2024-04-21T08:36:23.467430+0000","last_became_peered":"2024-04-21T08:36:23.467430+0000","last_unstale":"2024-04-21T08:38:55.178912+0000","last_undegraded":"2024-04-21T08:38:55.178912+0000","last_fullsized":"2024-04-21T08:38:55.178912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:47:18.900911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70068,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":264,"num_read_kb":201,"num_write":45,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890735+0000","last_change":"2024-04-21T08:36:23.989116+0000","last_active":"2024-04-21T08:38:54.890735+0000","last_peered":"2024-04-21T08:38:54.890735+0000","last_clean":"2024-04-21T08:38:54.890735+0000","last_became_active":"2024-04-21T08:36:23.988758+0000","last_became_peered":"2024-04-21T08:36:23.988758+0000","last_unstale":"2024-04-21T08:38:54.890735+0000","last_undegraded":"2024-04-21T08:38:54.890735+0000","last_fullsized":"2024-04-21T08:38:54.890735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:56:25.969905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008450+0000","last_change":"2024-04-21T08:36:28.007370+0000","last_active":"2024-04-21T08:38:55.008450+0000","last_peered":"2024-04-21T08:38:55.008450+0000","last_clean":"2024-04-21T08:38:55.008450+0000","last_became_active":"2024-04-21T08:36:28.006972+0000","last_became_peered":"2024-04-21T08:36:28.006972+0000","last_unstale":"2024-04-21T08:38:55.008450+0000","last_undegraded":"2024-04-21T08:38:55.008450+0000","last_fullsized":"2024-04-21T08:38:55.008450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:18:14.926058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104708+0000","last_change":"2024-04-21T08:36:23.472658+0000","last_active":"2024-04-21T08:38:55.104708+0000","last_peered":"2024-04-21T08:38:55.104708+0000","last_clean":"2024-04-21T08:38:55.104708+0000","last_became_active":"2024-04-21T08:36:23.472318+0000","last_became_peered":"2024-04-21T08:36:23.472318+0000","last_unstale":"2024-04-21T08:38:55.104708+0000","last_undegraded":"2024-04-21T08:38:55.104708+0000","last_fullsized":"2024-04-21T08:38:55.104708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:28:27.497605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076501+0000","last_change":"2024-04-21T08:38:15.338892+0000","last_active":"2024-04-21T08:38:55.076501+0000","last_peered":"2024-04-21T08:38:55.076501+0000","last_clean":"2024-04-21T08:38:55.076501+0000","last_became_active":"2024-04-21T08:38:15.338396+0000","last_became_peered":"2024-04-21T08:38:15.338396+0000","last_unstale":"2024-04-21T08:38:55.076501+0000","last_undegraded":"2024-04-21T08:38:55.076501+0000","last_fullsized":"2024-04-21T08:38:55.076501+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:19.670635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005342+0000","last_change":"2024-04-21T08:36:28.019186+0000","last_active":"2024-04-21T08:38:55.005342+0000","last_peered":"2024-04-21T08:38:55.005342+0000","last_clean":"2024-04-21T08:38:55.005342+0000","last_became_active":"2024-04-21T08:36:28.018774+0000","last_became_peered":"2024-04-21T08:36:28.018774+0000","last_unstale":"2024-04-21T08:38:55.005342+0000","last_undegraded":"2024-04-21T08:38:55.005342+0000","last_fullsized":"2024-04-21T08:38:55.005342+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:35:33.548656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008059+0000","last_change":"2024-04-21T08:36:23.472633+0000","last_active":"2024-04-21T08:38:55.008059+0000","last_peered":"2024-04-21T08:38:55.008059+0000","last_clean":"2024-04-21T08:38:55.008059+0000","last_became_active":"2024-04-21T08:36:23.472336+0000","last_became_peered":"2024-04-21T08:36:23.472336+0000","last_unstale":"2024-04-21T08:38:55.008059+0000","last_undegraded":"2024-04-21T08:38:55.008059+0000","last_fullsized":"2024-04-21T08:38:55.008059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:49.463492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006193+0000","last_change":"2024-04-21T08:36:23.994332+0000","last_active":"2024-04-21T08:38:55.006193+0000","last_peered":"2024-04-21T08:38:55.006193+0000","last_clean":"2024-04-21T08:38:55.006193+0000","last_became_active":"2024-04-21T08:36:23.993936+0000","last_became_peered":"2024-04-21T08:36:23.993936+0000","last_unstale":"2024-04-21T08:38:55.006193+0000","last_undegraded":"2024-04-21T08:38:55.006193+0000","last_fullsized":"2024-04-21T08:38:55.006193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:26:35.330559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.996887+0000","last_change":"2024-04-21T08:36:28.012549+0000","last_active":"2024-04-21T08:38:54.996887+0000","last_peered":"2024-04-21T08:38:54.996887+0000","last_clean":"2024-04-21T08:38:54.996887+0000","last_became_active":"2024-04-21T08:36:28.012151+0000","last_became_peered":"2024-04-21T08:36:28.012151+0000","last_unstale":"2024-04-21T08:38:54.996887+0000","last_undegraded":"2024-04-21T08:38:54.996887+0000","last_fullsized":"2024-04-21T08:38:54.996887+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:55.680399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'8","reported_seq":172,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076760+0000","last_change":"2024-04-21T08:36:23.464586+0000","last_active":"2024-04-21T08:38:55.076760+0000","last_peered":"2024-04-21T08:38:55.076760+0000","last_clean":"2024-04-21T08:38:55.076760+0000","last_became_active":"2024-04-21T08:36:23.464281+0000","last_became_peered":"2024-04-21T08:36:23.464281+0000","last_unstale":"2024-04-21T08:38:55.076760+0000","last_undegraded":"2024-04-21T08:38:55.076760+0000","last_fullsized":"2024-04-21T08:38:55.076760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:04:58.577283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":131,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006132+0000","last_change":"2024-04-21T08:36:23.989604+0000","last_active":"2024-04-21T08:38:55.006132+0000","last_peered":"2024-04-21T08:38:55.006132+0000","last_clean":"2024-04-21T08:38:55.006132+0000","last_became_active":"2024-04-21T08:36:23.989257+0000","last_became_peered":"2024-04-21T08:36:23.989257+0000","last_unstale":"2024-04-21T08:38:55.006132+0000","last_undegraded":"2024-04-21T08:38:55.006132+0000","last_fullsized":"2024-04-21T08:38:55.006132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:45.077943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077854+0000","last_change":"2024-04-21T08:36:28.007959+0000","last_active":"2024-04-21T08:38:55.077854+0000","last_peered":"2024-04-21T08:38:55.077854+0000","last_clean":"2024-04-21T08:38:55.077854+0000","last_became_active":"2024-04-21T08:36:28.007571+0000","last_became_peered":"2024-04-21T08:36:28.007571+0000","last_unstale":"2024-04-21T08:38:55.077854+0000","last_undegraded":"2024-04-21T08:38:55.077854+0000","last_fullsized":"2024-04-21T08:38:55.077854+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:17:34.007013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890204+0000","last_change":"2024-04-21T08:36:23.475054+0000","last_active":"2024-04-21T08:38:54.890204+0000","last_peered":"2024-04-21T08:38:54.890204+0000","last_clean":"2024-04-21T08:38:54.890204+0000","last_became_active":"2024-04-21T08:36:23.474606+0000","last_became_peered":"2024-04-21T08:36:23.474606+0000","last_unstale":"2024-04-21T08:38:54.890204+0000","last_undegraded":"2024-04-21T08:38:54.890204+0000","last_fullsized":"2024-04-21T08:38:54.890204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:22.559174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007909+0000","last_change":"2024-04-21T08:36:23.995980+0000","last_active":"2024-04-21T08:38:55.007909+0000","last_peered":"2024-04-21T08:38:55.007909+0000","last_clean":"2024-04-21T08:38:55.007909+0000","last_became_active":"2024-04-21T08:36:23.995673+0000","last_became_peered":"2024-04-21T08:36:23.995673+0000","last_unstale":"2024-04-21T08:38:55.007909+0000","last_undegraded":"2024-04-21T08:38:55.007909+0000","last_fullsized":"2024-04-21T08:38:55.007909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:11:49.884384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007831+0000","last_change":"2024-04-21T08:36:28.018792+0000","last_active":"2024-04-21T08:38:55.007831+0000","last_peered":"2024-04-21T08:38:55.007831+0000","last_clean":"2024-04-21T08:38:55.007831+0000","last_became_active":"2024-04-21T08:36:28.018373+0000","last_became_peered":"2024-04-21T08:36:28.018373+0000","last_unstale":"2024-04-21T08:38:55.007831+0000","last_undegraded":"2024-04-21T08:38:55.007831+0000","last_fullsized":"2024-04-21T08:38:55.007831+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:11.857661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890763+0000","last_change":"2024-04-21T08:36:23.469787+0000","last_active":"2024-04-21T08:38:54.890763+0000","last_peered":"2024-04-21T08:38:54.890763+0000","last_clean":"2024-04-21T08:38:54.890763+0000","last_became_active":"2024-04-21T08:36:23.469462+0000","last_became_peered":"2024-04-21T08:36:23.469462+0000","last_unstale":"2024-04-21T08:38:54.890763+0000","last_undegraded":"2024-04-21T08:38:54.890763+0000","last_fullsized":"2024-04-21T08:38:54.890763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:33:25.849528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005561+0000","last_change":"2024-04-21T08:36:23.988492+0000","last_active":"2024-04-21T08:38:55.005561+0000","last_peered":"2024-04-21T08:38:55.005561+0000","last_clean":"2024-04-21T08:38:55.005561+0000","last_became_active":"2024-04-21T08:36:23.988133+0000","last_became_peered":"2024-04-21T08:36:23.988133+0000","last_unstale":"2024-04-21T08:38:55.005561+0000","last_undegraded":"2024-04-21T08:38:55.005561+0000","last_fullsized":"2024-04-21T08:38:55.005561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:01:25.320281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890332+0000","last_change":"2024-04-21T08:38:15.415043+0000","last_active":"2024-04-21T08:38:54.890332+0000","last_peered":"2024-04-21T08:38:54.890332+0000","last_clean":"2024-04-21T08:38:54.890332+0000","last_became_active":"2024-04-21T08:38:15.414460+0000","last_became_peered":"2024-04-21T08:38:15.414460+0000","last_unstale":"2024-04-21T08:38:54.890332+0000","last_undegraded":"2024-04-21T08:38:54.890332+0000","last_fullsized":"2024-04-21T08:38:54.890332+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:50:17.202423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.996393+0000","last_change":"2024-04-21T08:37:15.219428+0000","last_active":"2024-04-21T08:38:54.996393+0000","last_peered":"2024-04-21T08:38:54.996393+0000","last_clean":"2024-04-21T08:38:54.996393+0000","last_became_active":"2024-04-21T08:37:15.218923+0000","last_became_peered":"2024-04-21T08:37:15.218923+0000","last_unstale":"2024-04-21T08:38:54.996393+0000","last_undegraded":"2024-04-21T08:38:54.996393+0000","last_fullsized":"2024-04-21T08:38:54.996393+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:07:02.781701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888120+0000","last_change":"2024-04-21T08:36:23.985910+0000","last_active":"2024-04-21T08:38:54.888120+0000","last_peered":"2024-04-21T08:38:54.888120+0000","last_clean":"2024-04-21T08:38:54.888120+0000","last_became_active":"2024-04-21T08:36:23.985605+0000","last_became_peered":"2024-04-21T08:36:23.985605+0000","last_unstale":"2024-04-21T08:38:54.888120+0000","last_undegraded":"2024-04-21T08:38:54.888120+0000","last_fullsized":"2024-04-21T08:38:54.888120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:37:09.367005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077277+0000","last_change":"2024-04-21T08:36:28.015256+0000","last_active":"2024-04-21T08:38:55.077277+0000","last_peered":"2024-04-21T08:38:55.077277+0000","last_clean":"2024-04-21T08:38:55.077277+0000","last_became_active":"2024-04-21T08:36:28.014849+0000","last_became_peered":"2024-04-21T08:36:28.014849+0000","last_unstale":"2024-04-21T08:38:55.077277+0000","last_undegraded":"2024-04-21T08:38:55.077277+0000","last_fullsized":"2024-04-21T08:38:55.077277+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:22:05.466299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"95'6","reported_seq":165,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.075177+0000","last_change":"2024-04-21T08:36:23.474966+0000","last_active":"2024-04-21T08:38:55.075177+0000","last_peered":"2024-04-21T08:38:55.075177+0000","last_clean":"2024-04-21T08:38:55.075177+0000","last_became_active":"2024-04-21T08:36:23.474374+0000","last_became_peered":"2024-04-21T08:36:23.474374+0000","last_unstale":"2024-04-21T08:38:55.075177+0000","last_undegraded":"2024-04-21T08:38:55.075177+0000","last_fullsized":"2024-04-21T08:38:55.075177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:57:20.237966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888138+0000","last_change":"2024-04-21T08:37:15.118887+0000","last_active":"2024-04-21T08:38:54.888138+0000","last_peered":"2024-04-21T08:38:54.888138+0000","last_clean":"2024-04-21T08:38:54.888138+0000","last_became_active":"2024-04-21T08:37:15.118510+0000","last_became_peered":"2024-04-21T08:37:15.118510+0000","last_unstale":"2024-04-21T08:38:54.888138+0000","last_undegraded":"2024-04-21T08:38:54.888138+0000","last_fullsized":"2024-04-21T08:38:54.888138+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:59:27.209083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077841+0000","last_change":"2024-04-21T08:36:28.013169+0000","last_active":"2024-04-21T08:38:55.077841+0000","last_peered":"2024-04-21T08:38:55.077841+0000","last_clean":"2024-04-21T08:38:55.077841+0000","last_became_active":"2024-04-21T08:36:28.012791+0000","last_became_peered":"2024-04-21T08:36:28.012791+0000","last_unstale":"2024-04-21T08:38:55.077841+0000","last_undegraded":"2024-04-21T08:38:55.077841+0000","last_fullsized":"2024-04-21T08:38:55.077841+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:42:56.063159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007983+0000","last_change":"2024-04-21T08:36:23.995209+0000","last_active":"2024-04-21T08:38:55.007983+0000","last_peered":"2024-04-21T08:38:55.007983+0000","last_clean":"2024-04-21T08:38:55.007983+0000","last_became_active":"2024-04-21T08:36:23.994866+0000","last_became_peered":"2024-04-21T08:36:23.994866+0000","last_unstale":"2024-04-21T08:38:55.007983+0000","last_undegraded":"2024-04-21T08:38:55.007983+0000","last_fullsized":"2024-04-21T08:38:55.007983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:33:45.582150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'7","reported_seq":168,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076172+0000","last_change":"2024-04-21T08:36:23.471474+0000","last_active":"2024-04-21T08:38:55.076172+0000","last_peered":"2024-04-21T08:38:55.076172+0000","last_clean":"2024-04-21T08:38:55.076172+0000","last_became_active":"2024-04-21T08:36:23.471179+0000","last_became_peered":"2024-04-21T08:36:23.471179+0000","last_unstale":"2024-04-21T08:38:55.076172+0000","last_undegraded":"2024-04-21T08:38:55.076172+0000","last_fullsized":"2024-04-21T08:38:55.076172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:17.290051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":127,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006553+0000","last_change":"2024-04-21T08:38:15.344332+0000","last_active":"2024-04-21T08:38:55.006553+0000","last_peered":"2024-04-21T08:38:55.006553+0000","last_clean":"2024-04-21T08:38:55.006553+0000","last_became_active":"2024-04-21T08:38:15.339914+0000","last_became_peered":"2024-04-21T08:38:15.339914+0000","last_unstale":"2024-04-21T08:38:55.006553+0000","last_undegraded":"2024-04-21T08:38:55.006553+0000","last_fullsized":"2024-04-21T08:38:55.006553+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:39:24.049323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886832+0000","last_change":"2024-04-21T08:36:23.990272+0000","last_active":"2024-04-21T08:38:54.886832+0000","last_peered":"2024-04-21T08:38:54.886832+0000","last_clean":"2024-04-21T08:38:54.886832+0000","last_became_active":"2024-04-21T08:36:23.989940+0000","last_became_peered":"2024-04-21T08:36:23.989940+0000","last_unstale":"2024-04-21T08:38:54.886832+0000","last_undegraded":"2024-04-21T08:38:54.886832+0000","last_fullsized":"2024-04-21T08:38:54.886832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:27:50.705167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006322+0000","last_change":"2024-04-21T08:36:23.474162+0000","last_active":"2024-04-21T08:38:55.006322+0000","last_peered":"2024-04-21T08:38:55.006322+0000","last_clean":"2024-04-21T08:38:55.006322+0000","last_became_active":"2024-04-21T08:36:23.473802+0000","last_became_peered":"2024-04-21T08:36:23.473802+0000","last_unstale":"2024-04-21T08:38:55.006322+0000","last_undegraded":"2024-04-21T08:38:55.006322+0000","last_fullsized":"2024-04-21T08:38:55.006322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:32:41.918431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":366,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005841+0000","last_change":"2024-04-21T08:35:10.062130+0000","last_active":"2024-04-21T08:38:55.005841+0000","last_peered":"2024-04-21T08:38:55.005841+0000","last_clean":"2024-04-21T08:38:55.005841+0000","last_became_active":"2024-04-21T08:35:10.061727+0000","last_became_peered":"2024-04-21T08:35:10.061727+0000","last_unstale":"2024-04-21T08:38:55.005841+0000","last_undegraded":"2024-04-21T08:38:55.005841+0000","last_fullsized":"2024-04-21T08:38:55.005841+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:31:40.950939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:31:40.950939+0000","last_clean_scrub_stamp":"2024-04-21T08:31:40.950939+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:35:52.360167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008148+0000","last_change":"2024-04-21T08:38:15.415007+0000","last_active":"2024-04-21T08:38:55.008148+0000","last_peered":"2024-04-21T08:38:55.008148+0000","last_clean":"2024-04-21T08:38:55.008148+0000","last_became_active":"2024-04-21T08:38:15.414478+0000","last_became_peered":"2024-04-21T08:38:15.414478+0000","last_unstale":"2024-04-21T08:38:55.008148+0000","last_undegraded":"2024-04-21T08:38:55.008148+0000","last_fullsized":"2024-04-21T08:38:55.008148+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:27.703964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.075651+0000","last_change":"2024-04-21T08:36:23.984332+0000","last_active":"2024-04-21T08:38:55.075651+0000","last_peered":"2024-04-21T08:38:55.075651+0000","last_clean":"2024-04-21T08:38:55.075651+0000","last_became_active":"2024-04-21T08:36:23.984021+0000","last_became_peered":"2024-04-21T08:36:23.984021+0000","last_unstale":"2024-04-21T08:38:55.075651+0000","last_undegraded":"2024-04-21T08:38:55.075651+0000","last_fullsized":"2024-04-21T08:38:55.075651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:05:21.090633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005972+0000","last_change":"2024-04-21T08:36:23.467522+0000","last_active":"2024-04-21T08:38:55.005972+0000","last_peered":"2024-04-21T08:38:55.005972+0000","last_clean":"2024-04-21T08:38:55.005972+0000","last_became_active":"2024-04-21T08:36:23.466379+0000","last_became_peered":"2024-04-21T08:36:23.466379+0000","last_unstale":"2024-04-21T08:38:55.005972+0000","last_undegraded":"2024-04-21T08:38:55.005972+0000","last_fullsized":"2024-04-21T08:38:55.005972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:04.185314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005193+0000","last_change":"2024-04-21T08:36:28.015120+0000","last_active":"2024-04-21T08:38:55.005193+0000","last_peered":"2024-04-21T08:38:55.005193+0000","last_clean":"2024-04-21T08:38:55.005193+0000","last_became_active":"2024-04-21T08:36:28.014701+0000","last_became_peered":"2024-04-21T08:36:28.014701+0000","last_unstale":"2024-04-21T08:38:55.005193+0000","last_undegraded":"2024-04-21T08:38:55.005193+0000","last_fullsized":"2024-04-21T08:38:55.005193+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:23.127622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890642+0000","last_change":"2024-04-21T08:36:23.475335+0000","last_active":"2024-04-21T08:38:54.890642+0000","last_peered":"2024-04-21T08:38:54.890642+0000","last_clean":"2024-04-21T08:38:54.890642+0000","last_became_active":"2024-04-21T08:36:23.474739+0000","last_became_peered":"2024-04-21T08:36:23.474739+0000","last_unstale":"2024-04-21T08:38:54.890642+0000","last_undegraded":"2024-04-21T08:38:54.890642+0000","last_fullsized":"2024-04-21T08:38:54.890642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:36.119649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077395+0000","last_change":"2024-04-21T08:36:23.977229+0000","last_active":"2024-04-21T08:38:55.077395+0000","last_peered":"2024-04-21T08:38:55.077395+0000","last_clean":"2024-04-21T08:38:55.077395+0000","last_became_active":"2024-04-21T08:36:23.976893+0000","last_became_peered":"2024-04-21T08:36:23.976893+0000","last_unstale":"2024-04-21T08:38:55.077395+0000","last_undegraded":"2024-04-21T08:38:55.077395+0000","last_fullsized":"2024-04-21T08:38:55.077395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:31:54.226274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886835+0000","last_change":"2024-04-21T08:36:28.018581+0000","last_active":"2024-04-21T08:38:54.886835+0000","last_peered":"2024-04-21T08:38:54.886835+0000","last_clean":"2024-04-21T08:38:54.886835+0000","last_became_active":"2024-04-21T08:36:28.018021+0000","last_became_peered":"2024-04-21T08:36:28.018021+0000","last_unstale":"2024-04-21T08:38:54.886835+0000","last_undegraded":"2024-04-21T08:38:54.886835+0000","last_fullsized":"2024-04-21T08:38:54.886835+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:29:00.355532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'48","reported_seq":105,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.124941+0000","last_change":"2024-04-21T08:36:23.473339+0000","last_active":"2024-04-21T08:38:55.124941+0000","last_peered":"2024-04-21T08:38:55.124941+0000","last_clean":"2024-04-21T08:38:55.124941+0000","last_became_active":"2024-04-21T08:36:23.472994+0000","last_became_peered":"2024-04-21T08:36:23.472994+0000","last_unstale":"2024-04-21T08:38:55.124941+0000","last_undegraded":"2024-04-21T08:38:55.124941+0000","last_fullsized":"2024-04-21T08:38:55.124941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:42.762273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887378+0000","last_change":"2024-04-21T08:36:23.987354+0000","last_active":"2024-04-21T08:38:54.887378+0000","last_peered":"2024-04-21T08:38:54.887378+0000","last_clean":"2024-04-21T08:38:54.887378+0000","last_became_active":"2024-04-21T08:36:23.987032+0000","last_became_peered":"2024-04-21T08:36:23.987032+0000","last_unstale":"2024-04-21T08:38:54.887378+0000","last_undegraded":"2024-04-21T08:38:54.887378+0000","last_fullsized":"2024-04-21T08:38:54.887378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:01:05.164367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104455+0000","last_change":"2024-04-21T08:36:28.002582+0000","last_active":"2024-04-21T08:38:55.104455+0000","last_peered":"2024-04-21T08:38:55.104455+0000","last_clean":"2024-04-21T08:38:55.104455+0000","last_became_active":"2024-04-21T08:36:28.002133+0000","last_became_peered":"2024-04-21T08:36:28.002133+0000","last_unstale":"2024-04-21T08:38:55.104455+0000","last_undegraded":"2024-04-21T08:38:55.104455+0000","last_fullsized":"2024-04-21T08:38:55.104455+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:30:00.268015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007058+0000","last_change":"2024-04-21T08:36:23.462840+0000","last_active":"2024-04-21T08:38:55.007058+0000","last_peered":"2024-04-21T08:38:55.007058+0000","last_clean":"2024-04-21T08:38:55.007058+0000","last_became_active":"2024-04-21T08:36:23.462391+0000","last_became_peered":"2024-04-21T08:36:23.462391+0000","last_unstale":"2024-04-21T08:38:55.007058+0000","last_undegraded":"2024-04-21T08:38:55.007058+0000","last_fullsized":"2024-04-21T08:38:55.007058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:33.230388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004380+0000","last_change":"2024-04-21T08:36:23.994751+0000","last_active":"2024-04-21T08:38:55.004380+0000","last_peered":"2024-04-21T08:38:55.004380+0000","last_clean":"2024-04-21T08:38:55.004380+0000","last_became_active":"2024-04-21T08:36:23.994384+0000","last_became_peered":"2024-04-21T08:36:23.994384+0000","last_unstale":"2024-04-21T08:38:55.004380+0000","last_undegraded":"2024-04-21T08:38:55.004380+0000","last_fullsized":"2024-04-21T08:38:55.004380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:18.956893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006981+0000","last_change":"2024-04-21T08:36:28.004004+0000","last_active":"2024-04-21T08:38:55.006981+0000","last_peered":"2024-04-21T08:38:55.006981+0000","last_clean":"2024-04-21T08:38:55.006981+0000","last_became_active":"2024-04-21T08:36:28.003547+0000","last_became_peered":"2024-04-21T08:36:28.003547+0000","last_unstale":"2024-04-21T08:38:55.006981+0000","last_undegraded":"2024-04-21T08:38:55.006981+0000","last_fullsized":"2024-04-21T08:38:55.006981+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:21:40.215428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104510+0000","last_change":"2024-04-21T08:37:15.222118+0000","last_active":"2024-04-21T08:38:55.104510+0000","last_peered":"2024-04-21T08:38:55.104510+0000","last_clean":"2024-04-21T08:38:55.104510+0000","last_became_active":"2024-04-21T08:37:15.219173+0000","last_became_peered":"2024-04-21T08:37:15.219173+0000","last_unstale":"2024-04-21T08:38:55.104510+0000","last_undegraded":"2024-04-21T08:38:55.104510+0000","last_fullsized":"2024-04-21T08:38:55.104510+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:12.931076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890299+0000","last_change":"2024-04-21T08:36:23.988020+0000","last_active":"2024-04-21T08:38:54.890299+0000","last_peered":"2024-04-21T08:38:54.890299+0000","last_clean":"2024-04-21T08:38:54.890299+0000","last_became_active":"2024-04-21T08:36:23.987630+0000","last_became_peered":"2024-04-21T08:36:23.987630+0000","last_unstale":"2024-04-21T08:38:54.890299+0000","last_undegraded":"2024-04-21T08:38:54.890299+0000","last_fullsized":"2024-04-21T08:38:54.890299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:06.519001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.997084+0000","last_change":"2024-04-21T08:36:28.006984+0000","last_active":"2024-04-21T08:38:54.997084+0000","last_peered":"2024-04-21T08:38:54.997084+0000","last_clean":"2024-04-21T08:38:54.997084+0000","last_became_active":"2024-04-21T08:36:28.006548+0000","last_became_peered":"2024-04-21T08:36:28.006548+0000","last_unstale":"2024-04-21T08:38:54.997084+0000","last_undegraded":"2024-04-21T08:38:54.997084+0000","last_fullsized":"2024-04-21T08:38:54.997084+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:25:50.870658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886907+0000","last_change":"2024-04-21T08:36:23.476266+0000","last_active":"2024-04-21T08:38:54.886907+0000","last_peered":"2024-04-21T08:38:54.886907+0000","last_clean":"2024-04-21T08:38:54.886907+0000","last_became_active":"2024-04-21T08:36:23.475948+0000","last_became_peered":"2024-04-21T08:36:23.475948+0000","last_unstale":"2024-04-21T08:38:54.886907+0000","last_undegraded":"2024-04-21T08:38:54.886907+0000","last_fullsized":"2024-04-21T08:38:54.886907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:41.529376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005217+0000","last_change":"2024-04-21T08:36:23.981942+0000","last_active":"2024-04-21T08:38:55.005217+0000","last_peered":"2024-04-21T08:38:55.005217+0000","last_clean":"2024-04-21T08:38:55.005217+0000","last_became_active":"2024-04-21T08:36:23.981643+0000","last_became_peered":"2024-04-21T08:36:23.981643+0000","last_unstale":"2024-04-21T08:38:55.005217+0000","last_undegraded":"2024-04-21T08:38:55.005217+0000","last_fullsized":"2024-04-21T08:38:55.005217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:26:24.722695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006307+0000","last_change":"2024-04-21T08:36:28.006980+0000","last_active":"2024-04-21T08:38:55.006307+0000","last_peered":"2024-04-21T08:38:55.006307+0000","last_clean":"2024-04-21T08:38:55.006307+0000","last_became_active":"2024-04-21T08:36:28.006530+0000","last_became_peered":"2024-04-21T08:36:28.006530+0000","last_unstale":"2024-04-21T08:38:55.006307+0000","last_undegraded":"2024-04-21T08:38:55.006307+0000","last_fullsized":"2024-04-21T08:38:55.006307+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:05:57.465612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104573+0000","last_change":"2024-04-21T08:36:23.473253+0000","last_active":"2024-04-21T08:38:55.104573+0000","last_peered":"2024-04-21T08:38:55.104573+0000","last_clean":"2024-04-21T08:38:55.104573+0000","last_became_active":"2024-04-21T08:36:23.472942+0000","last_became_peered":"2024-04-21T08:36:23.472942+0000","last_unstale":"2024-04-21T08:38:55.104573+0000","last_undegraded":"2024-04-21T08:38:55.104573+0000","last_fullsized":"2024-04-21T08:38:55.104573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:32:04.894388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887248+0000","last_change":"2024-04-21T08:36:23.988355+0000","last_active":"2024-04-21T08:38:54.887248+0000","last_peered":"2024-04-21T08:38:54.887248+0000","last_clean":"2024-04-21T08:38:54.887248+0000","last_became_active":"2024-04-21T08:36:23.988049+0000","last_became_peered":"2024-04-21T08:36:23.988049+0000","last_unstale":"2024-04-21T08:38:54.887248+0000","last_undegraded":"2024-04-21T08:38:54.887248+0000","last_fullsized":"2024-04-21T08:38:54.887248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:57.348698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006797+0000","last_change":"2024-04-21T08:36:28.007343+0000","last_active":"2024-04-21T08:38:55.006797+0000","last_peered":"2024-04-21T08:38:55.006797+0000","last_clean":"2024-04-21T08:38:55.006797+0000","last_became_active":"2024-04-21T08:36:28.006719+0000","last_became_peered":"2024-04-21T08:36:28.006719+0000","last_unstale":"2024-04-21T08:38:55.006797+0000","last_undegraded":"2024-04-21T08:38:55.006797+0000","last_fullsized":"2024-04-21T08:38:55.006797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:02:25.734584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890843+0000","last_change":"2024-04-21T08:36:23.472416+0000","last_active":"2024-04-21T08:38:54.890843+0000","last_peered":"2024-04-21T08:38:54.890843+0000","last_clean":"2024-04-21T08:38:54.890843+0000","last_became_active":"2024-04-21T08:36:23.471500+0000","last_became_peered":"2024-04-21T08:36:23.471500+0000","last_unstale":"2024-04-21T08:38:54.890843+0000","last_undegraded":"2024-04-21T08:38:54.890843+0000","last_fullsized":"2024-04-21T08:38:54.890843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:39.706432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006128+0000","last_change":"2024-04-21T08:36:23.990653+0000","last_active":"2024-04-21T08:38:55.006128+0000","last_peered":"2024-04-21T08:38:55.006128+0000","last_clean":"2024-04-21T08:38:55.006128+0000","last_became_active":"2024-04-21T08:36:23.990324+0000","last_became_peered":"2024-04-21T08:36:23.990324+0000","last_unstale":"2024-04-21T08:38:55.006128+0000","last_undegraded":"2024-04-21T08:38:55.006128+0000","last_fullsized":"2024-04-21T08:38:55.006128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:08:59.417850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104492+0000","last_change":"2024-04-21T08:36:28.007956+0000","last_active":"2024-04-21T08:38:55.104492+0000","last_peered":"2024-04-21T08:38:55.104492+0000","last_clean":"2024-04-21T08:38:55.104492+0000","last_became_active":"2024-04-21T08:36:28.007475+0000","last_became_peered":"2024-04-21T08:36:28.007475+0000","last_unstale":"2024-04-21T08:38:55.104492+0000","last_undegraded":"2024-04-21T08:38:55.104492+0000","last_fullsized":"2024-04-21T08:38:55.104492+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:54.039359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'8","reported_seq":170,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104857+0000","last_change":"2024-04-21T08:36:23.476887+0000","last_active":"2024-04-21T08:38:55.104857+0000","last_peered":"2024-04-21T08:38:55.104857+0000","last_clean":"2024-04-21T08:38:55.104857+0000","last_became_active":"2024-04-21T08:36:23.476574+0000","last_became_peered":"2024-04-21T08:36:23.476574+0000","last_unstale":"2024-04-21T08:38:55.104857+0000","last_undegraded":"2024-04-21T08:38:55.104857+0000","last_fullsized":"2024-04-21T08:38:55.104857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:37:47.692358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2010,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006973+0000","last_change":"2024-04-21T08:36:23.982845+0000","last_active":"2024-04-21T08:38:55.006973+0000","last_peered":"2024-04-21T08:38:55.006973+0000","last_clean":"2024-04-21T08:38:55.006973+0000","last_became_active":"2024-04-21T08:36:23.982491+0000","last_became_peered":"2024-04-21T08:36:23.982491+0000","last_unstale":"2024-04-21T08:38:55.006973+0000","last_undegraded":"2024-04-21T08:38:55.006973+0000","last_fullsized":"2024-04-21T08:38:55.006973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:48:46.413896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.002896+0000","last_change":"2024-04-21T08:36:28.010442+0000","last_active":"2024-04-21T08:38:55.002896+0000","last_peered":"2024-04-21T08:38:55.002896+0000","last_clean":"2024-04-21T08:38:55.002896+0000","last_became_active":"2024-04-21T08:36:28.010046+0000","last_became_peered":"2024-04-21T08:36:28.010046+0000","last_unstale":"2024-04-21T08:38:55.002896+0000","last_undegraded":"2024-04-21T08:38:55.002896+0000","last_fullsized":"2024-04-21T08:38:55.002896+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:18:36.382802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"95'41","reported_seq":83,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.122392+0000","last_change":"2024-04-21T08:36:23.473545+0000","last_active":"2024-04-21T08:38:55.122392+0000","last_peered":"2024-04-21T08:38:55.122392+0000","last_clean":"2024-04-21T08:38:55.122392+0000","last_became_active":"2024-04-21T08:36:23.473222+0000","last_became_peered":"2024-04-21T08:36:23.473222+0000","last_unstale":"2024-04-21T08:38:55.122392+0000","last_undegraded":"2024-04-21T08:38:55.122392+0000","last_fullsized":"2024-04-21T08:38:55.122392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:47:53.947187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006600+0000","last_change":"2024-04-21T08:36:23.990959+0000","last_active":"2024-04-21T08:38:55.006600+0000","last_peered":"2024-04-21T08:38:55.006600+0000","last_clean":"2024-04-21T08:38:55.006600+0000","last_became_active":"2024-04-21T08:36:23.990322+0000","last_became_peered":"2024-04-21T08:36:23.990322+0000","last_unstale":"2024-04-21T08:38:55.006600+0000","last_undegraded":"2024-04-21T08:38:55.006600+0000","last_fullsized":"2024-04-21T08:38:55.006600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:12:20.859224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007717+0000","last_change":"2024-04-21T08:36:28.006682+0000","last_active":"2024-04-21T08:38:55.007717+0000","last_peered":"2024-04-21T08:38:55.007717+0000","last_clean":"2024-04-21T08:38:55.007717+0000","last_became_active":"2024-04-21T08:36:28.006229+0000","last_became_peered":"2024-04-21T08:36:28.006229+0000","last_unstale":"2024-04-21T08:38:55.007717+0000","last_undegraded":"2024-04-21T08:38:55.007717+0000","last_fullsized":"2024-04-21T08:38:55.007717+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:30:12.723848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077921+0000","last_change":"2024-04-21T08:36:23.473592+0000","last_active":"2024-04-21T08:38:55.077921+0000","last_peered":"2024-04-21T08:38:55.077921+0000","last_clean":"2024-04-21T08:38:55.077921+0000","last_became_active":"2024-04-21T08:36:23.473283+0000","last_became_peered":"2024-04-21T08:36:23.473283+0000","last_unstale":"2024-04-21T08:38:55.077921+0000","last_undegraded":"2024-04-21T08:38:55.077921+0000","last_fullsized":"2024-04-21T08:38:55.077921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:10.425249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006895+0000","last_change":"2024-04-21T08:36:23.985052+0000","last_active":"2024-04-21T08:38:55.006895+0000","last_peered":"2024-04-21T08:38:55.006895+0000","last_clean":"2024-04-21T08:38:55.006895+0000","last_became_active":"2024-04-21T08:36:23.984742+0000","last_became_peered":"2024-04-21T08:36:23.984742+0000","last_unstale":"2024-04-21T08:38:55.006895+0000","last_undegraded":"2024-04-21T08:38:55.006895+0000","last_fullsized":"2024-04-21T08:38:55.006895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:08:06.082974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005201+0000","last_change":"2024-04-21T08:36:23.980905+0000","last_active":"2024-04-21T08:38:55.005201+0000","last_peered":"2024-04-21T08:38:55.005201+0000","last_clean":"2024-04-21T08:38:55.005201+0000","last_became_active":"2024-04-21T08:36:23.980590+0000","last_became_peered":"2024-04-21T08:36:23.980590+0000","last_unstale":"2024-04-21T08:38:55.005201+0000","last_undegraded":"2024-04-21T08:38:55.005201+0000","last_fullsized":"2024-04-21T08:38:55.005201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:54:59.886014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104725+0000","last_change":"2024-04-21T08:36:23.980883+0000","last_active":"2024-04-21T08:38:55.104725+0000","last_peered":"2024-04-21T08:38:55.104725+0000","last_clean":"2024-04-21T08:38:55.104725+0000","last_became_active":"2024-04-21T08:36:23.980562+0000","last_became_peered":"2024-04-21T08:36:23.980562+0000","last_unstale":"2024-04-21T08:38:55.104725+0000","last_undegraded":"2024-04-21T08:38:55.104725+0000","last_fullsized":"2024-04-21T08:38:55.104725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:14:51.868438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104692+0000","last_change":"2024-04-21T08:36:23.991571+0000","last_active":"2024-04-21T08:38:55.104692+0000","last_peered":"2024-04-21T08:38:55.104692+0000","last_clean":"2024-04-21T08:38:55.104692+0000","last_became_active":"2024-04-21T08:36:23.991273+0000","last_became_peered":"2024-04-21T08:36:23.991273+0000","last_unstale":"2024-04-21T08:38:55.104692+0000","last_undegraded":"2024-04-21T08:38:55.104692+0000","last_fullsized":"2024-04-21T08:38:55.104692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:08:23.895038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.003835+0000","last_change":"2024-04-21T08:36:23.994729+0000","last_active":"2024-04-21T08:38:55.003835+0000","last_peered":"2024-04-21T08:38:55.003835+0000","last_clean":"2024-04-21T08:38:55.003835+0000","last_became_active":"2024-04-21T08:36:23.994405+0000","last_became_peered":"2024-04-21T08:36:23.994405+0000","last_unstale":"2024-04-21T08:38:55.003835+0000","last_undegraded":"2024-04-21T08:38:55.003835+0000","last_fullsized":"2024-04-21T08:38:55.003835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:42.463225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077385+0000","last_change":"2024-04-21T08:36:23.995421+0000","last_active":"2024-04-21T08:38:55.077385+0000","last_peered":"2024-04-21T08:38:55.077385+0000","last_clean":"2024-04-21T08:38:55.077385+0000","last_became_active":"2024-04-21T08:36:23.993674+0000","last_became_peered":"2024-04-21T08:36:23.993674+0000","last_unstale":"2024-04-21T08:38:55.077385+0000","last_undegraded":"2024-04-21T08:38:55.077385+0000","last_fullsized":"2024-04-21T08:38:55.077385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:31.013874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005504+0000","last_change":"2024-04-21T08:36:23.986418+0000","last_active":"2024-04-21T08:38:55.005504+0000","last_peered":"2024-04-21T08:38:55.005504+0000","last_clean":"2024-04-21T08:38:55.005504+0000","last_became_active":"2024-04-21T08:36:23.986095+0000","last_became_peered":"2024-04-21T08:36:23.986095+0000","last_unstale":"2024-04-21T08:38:55.005504+0000","last_undegraded":"2024-04-21T08:38:55.005504+0000","last_fullsized":"2024-04-21T08:38:55.005504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:31:56.456218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104918+0000","last_change":"2024-04-21T08:36:23.982353+0000","last_active":"2024-04-21T08:38:55.104918+0000","last_peered":"2024-04-21T08:38:55.104918+0000","last_clean":"2024-04-21T08:38:55.104918+0000","last_became_active":"2024-04-21T08:36:23.982035+0000","last_became_peered":"2024-04-21T08:36:23.982035+0000","last_unstale":"2024-04-21T08:38:55.104918+0000","last_undegraded":"2024-04-21T08:38:55.104918+0000","last_fullsized":"2024-04-21T08:38:55.104918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:42:44.598407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887832+0000","last_change":"2024-04-21T08:36:23.996371+0000","last_active":"2024-04-21T08:38:54.887832+0000","last_peered":"2024-04-21T08:38:54.887832+0000","last_clean":"2024-04-21T08:38:54.887832+0000","last_became_active":"2024-04-21T08:36:23.995802+0000","last_became_peered":"2024-04-21T08:36:23.995802+0000","last_unstale":"2024-04-21T08:38:54.887832+0000","last_undegraded":"2024-04-21T08:38:54.887832+0000","last_fullsized":"2024-04-21T08:38:54.887832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:52:08.571567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007826+0000","last_change":"2024-04-21T08:36:23.987275+0000","last_active":"2024-04-21T08:38:55.007826+0000","last_peered":"2024-04-21T08:38:55.007826+0000","last_clean":"2024-04-21T08:38:55.007826+0000","last_became_active":"2024-04-21T08:36:23.986967+0000","last_became_peered":"2024-04-21T08:36:23.986967+0000","last_unstale":"2024-04-21T08:38:55.007826+0000","last_undegraded":"2024-04-21T08:38:55.007826+0000","last_fullsized":"2024-04-21T08:38:55.007826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:50.432502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886991+0000","last_change":"2024-04-21T08:36:23.993046+0000","last_active":"2024-04-21T08:38:54.886991+0000","last_peered":"2024-04-21T08:38:54.886991+0000","last_clean":"2024-04-21T08:38:54.886991+0000","last_became_active":"2024-04-21T08:36:23.992693+0000","last_became_peered":"2024-04-21T08:36:23.992693+0000","last_unstale":"2024-04-21T08:38:54.886991+0000","last_undegraded":"2024-04-21T08:38:54.886991+0000","last_fullsized":"2024-04-21T08:38:54.886991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:12:06.287585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104706+0000","last_change":"2024-04-21T08:36:23.992968+0000","last_active":"2024-04-21T08:38:55.104706+0000","last_peered":"2024-04-21T08:38:55.104706+0000","last_clean":"2024-04-21T08:38:55.104706+0000","last_became_active":"2024-04-21T08:36:23.992614+0000","last_became_peered":"2024-04-21T08:36:23.992614+0000","last_unstale":"2024-04-21T08:38:55.104706+0000","last_undegraded":"2024-04-21T08:38:55.104706+0000","last_fullsized":"2024-04-21T08:38:55.104706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:10.015323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076758+0000","last_change":"2024-04-21T08:36:23.981169+0000","last_active":"2024-04-21T08:38:55.076758+0000","last_peered":"2024-04-21T08:38:55.076758+0000","last_clean":"2024-04-21T08:38:55.076758+0000","last_became_active":"2024-04-21T08:36:23.980849+0000","last_became_peered":"2024-04-21T08:36:23.980849+0000","last_unstale":"2024-04-21T08:38:55.076758+0000","last_undegraded":"2024-04-21T08:38:55.076758+0000","last_fullsized":"2024-04-21T08:38:55.076758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:13:49.907370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005358+0000","last_change":"2024-04-21T08:36:23.993998+0000","last_active":"2024-04-21T08:38:55.005358+0000","last_peered":"2024-04-21T08:38:55.005358+0000","last_clean":"2024-04-21T08:38:55.005358+0000","last_became_active":"2024-04-21T08:36:23.993664+0000","last_became_peered":"2024-04-21T08:36:23.993664+0000","last_unstale":"2024-04-21T08:38:55.005358+0000","last_undegraded":"2024-04-21T08:38:55.005358+0000","last_fullsized":"2024-04-21T08:38:55.005358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:31:49.076820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":604383,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1392,"num_read_kb":816,"num_write":377,"num_write_kb":1476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1813149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351019,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1093632,"data_stored":535779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612657,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37220,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707932,"statfs":{"total":95995035648,"available":95956922368,"internally_reserved":0,"allocated":1929216,"data_stored":1324346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776182,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36820,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708332,"statfs":{"total":95995035648,"available":95957331968,"internally_reserved":0,"allocated":1515520,"data_stored":932181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037819,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28496,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716656,"statfs":{"total":95995035648,"available":95965855744,"internally_reserved":0,"allocated":1511424,"data_stored":932559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299455,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1712128,"data_stored":1133590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462980,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1093632,"data_stored":530764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1167360,"data_stored":601862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1118208,"data_stored":532669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1187840,"data_stored":603454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1054897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920847,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":535370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724619,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32984,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712168,"statfs":{"total":95995035648,"available":95961260032,"internally_reserved":0,"allocated":1851392,"data_stored":1255362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":204327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:38:57.751 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-21T08:38:57.751 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T08:38:57.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.894+0000 7f7f6e52e640 1 -- 172.21.15.43:0/400384357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f68107c50 msgr2=0x7f7f6810a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.894+0000 7f7f6e52e640 1 --2- 172.21.15.43:0/400384357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f68107c50 0x7f7f6810a0a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009ca0 tx=0x7f7f6002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.894+0000 7f7f6e52e640 1 -- 172.21.15.43:0/400384357 shutdown_connections 2024-04-21T08:38:57.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.894+0000 7f7f6e52e640 1 --2- 172.21.15.43:0/400384357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f68107c50 0x7f7f6810a0a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.894+0000 7f7f6e52e640 1 --2- 172.21.15.43:0/400384357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f68105320 0x7f7f68107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.894+0000 7f7f6e52e640 1 --2- 172.21.15.43:0/400384357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f681029f0 0x7f7f68104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.894+0000 7f7f6e52e640 1 -- 172.21.15.43:0/400384357 >> 172.21.15.43:0/400384357 conn(0x7f7f680fc3e0 msgr2=0x7f7f680fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:57.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.894+0000 7f7f6e52e640 1 -- 172.21.15.43:0/400384357 shutdown_connections 2024-04-21T08:38:57.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.894+0000 7f7f6e52e640 1 -- 172.21.15.43:0/400384357 wait complete. 2024-04-21T08:38:57.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.895+0000 7f7f6e52e640 1 Processor -- start 2024-04-21T08:38:57.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.895+0000 7f7f6e52e640 1 -- start start 2024-04-21T08:38:57.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.895+0000 7f7f6e52e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f681029f0 0x7f7f68198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:57.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.895+0000 7f7f6e52e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f68105320 0x7f7f681986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:57.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.895+0000 7f7f6e52e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f68107c50 0x7f7f6819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:57.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.895+0000 7f7f6e52e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6810dac0 con 0x7f7f68105320 2024-04-21T08:38:57.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.895+0000 7f7f6e52e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6810d940 con 0x7f7f681029f0 2024-04-21T08:38:57.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.895+0000 7f7f6e52e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6810dc40 con 0x7f7f68107c50 2024-04-21T08:38:57.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.895+0000 7f7f677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f68105320 0x7f7f681986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:57.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.895+0000 7f7f677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f68105320 0x7f7f681986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38590/0 (socket says 172.21.15.43:38590) 2024-04-21T08:38:57.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.895+0000 7f7f677fe640 1 -- 172.21.15.43:0/2828731796 learned_addr learned my addr 172.21.15.43:0/2828731796 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:57.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.896+0000 7f7f67fff640 1 --2- 172.21.15.43:0/2828731796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f681029f0 0x7f7f68198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:57.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.896+0000 7f7f6cacb640 1 --2- 172.21.15.43:0/2828731796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f68107c50 0x7f7f6819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:57.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.896+0000 7f7f6cacb640 1 -- 172.21.15.43:0/2828731796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f681029f0 msgr2=0x7f7f68198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.896+0000 7f7f6cacb640 1 --2- 172.21.15.43:0/2828731796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f681029f0 0x7f7f68198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.896+0000 7f7f6cacb640 1 -- 172.21.15.43:0/2828731796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f68105320 msgr2=0x7f7f681986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.896+0000 7f7f6cacb640 1 --2- 172.21.15.43:0/2828731796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f68105320 0x7f7f681986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.896+0000 7f7f6cacb640 1 -- 172.21.15.43:0/2828731796 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6819fdb0 con 0x7f7f68107c50 2024-04-21T08:38:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.896+0000 7f7f677fe640 1 --2- 172.21.15.43:0/2828731796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f68105320 0x7f7f681986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:38:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.896+0000 7f7f67fff640 1 --2- 172.21.15.43:0/2828731796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f681029f0 0x7f7f68198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:38:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.896+0000 7f7f6cacb640 1 --2- 172.21.15.43:0/2828731796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f68107c50 0x7f7f6819f740 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009bf0 tx=0x7f7f60034670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.896+0000 7f7f657fa640 1 -- 172.21.15.43:0/2828731796 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60041020 con 0x7f7f68107c50 2024-04-21T08:38:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.896+0000 7f7f657fa640 1 -- 172.21.15.43:0/2828731796 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f60034ca0 con 0x7f7f68107c50 2024-04-21T08:38:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.896+0000 7f7f6e52e640 1 -- 172.21.15.43:0/2828731796 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f681a0040 con 0x7f7f68107c50 2024-04-21T08:38:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.896+0000 7f7f657fa640 1 -- 172.21.15.43:0/2828731796 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60039040 con 0x7f7f68107c50 2024-04-21T08:38:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.896+0000 7f7f6e52e640 1 -- 172.21.15.43:0/2828731796 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f681a0550 con 0x7f7f68107c50 2024-04-21T08:38:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.897+0000 7f7f6e52e640 1 -- 172.21.15.43:0/2828731796 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f680fded0 con 0x7f7f68107c50 2024-04-21T08:38:57.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.898+0000 7f7f657fa640 1 -- 172.21.15.43:0/2828731796 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f7f60044020 con 0x7f7f68107c50 2024-04-21T08:38:57.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.899+0000 7f7f657fa640 1 --2- 172.21.15.43:0/2828731796 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7f500a3da0 0x7f7f500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:57.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.899+0000 7f7f657fa640 1 -- 172.21.15.43:0/2828731796 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f7f600f2660 con 0x7f7f68107c50 2024-04-21T08:38:57.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.899+0000 7f7f67fff640 1 --2- 172.21.15.43:0/2828731796 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7f500a3da0 0x7f7f500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:57.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.900+0000 7f7f67fff640 1 --2- 172.21.15.43:0/2828731796 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7f500a3da0 0x7f7f500a6260 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009fc0 tx=0x7f7f58005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:57.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:57.904+0000 7f7f657fa640 1 -- 172.21.15.43:0/2828731796 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7f6004b020 con 0x7f7f68107c50 2024-04-21T08:38:58.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.091+0000 7f7f6e52e640 1 -- 172.21.15.43:0/2828731796 --> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7f68062390 con 0x7f7f500a3da0 2024-04-21T08:38:58.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.101+0000 7f7f657fa640 1 -- 172.21.15.43:0/2828731796 <== mgr.15546 v2:172.21.15.43:6800/1482273906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339423 (secure 0 0 0) 0x7f7f68062390 con 0x7f7f500a3da0 2024-04-21T08:38:58.105 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-21T08:38:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.105+0000 7f7f6e52e640 1 -- 172.21.15.43:0/2828731796 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7f500a3da0 msgr2=0x7f7f500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.105+0000 7f7f6e52e640 1 --2- 172.21.15.43:0/2828731796 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7f500a3da0 0x7f7f500a6260 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009fc0 tx=0x7f7f58005f90 comp rx=0 tx=0).stop 2024-04-21T08:38:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.105+0000 7f7f6e52e640 1 -- 172.21.15.43:0/2828731796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f68107c50 msgr2=0x7f7f6819f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.105+0000 7f7f6e52e640 1 --2- 172.21.15.43:0/2828731796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f68107c50 0x7f7f6819f740 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009bf0 tx=0x7f7f60034670 comp rx=0 tx=0).stop 2024-04-21T08:38:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.105+0000 7f7f6e52e640 1 -- 172.21.15.43:0/2828731796 shutdown_connections 2024-04-21T08:38:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.105+0000 7f7f6e52e640 1 --2- 172.21.15.43:0/2828731796 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7f500a3da0 0x7f7f500a6260 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.105+0000 7f7f6e52e640 1 --2- 172.21.15.43:0/2828731796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f68107c50 0x7f7f6819f740 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.105+0000 7f7f6e52e640 1 --2- 172.21.15.43:0/2828731796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f68105320 0x7f7f681986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.105+0000 7f7f6e52e640 1 --2- 172.21.15.43:0/2828731796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f681029f0 0x7f7f68198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.105+0000 7f7f6e52e640 1 -- 172.21.15.43:0/2828731796 >> 172.21.15.43:0/2828731796 conn(0x7f7f680fc3e0 msgr2=0x7f7f68103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.105+0000 7f7f6e52e640 1 -- 172.21.15.43:0/2828731796 shutdown_connections 2024-04-21T08:38:58.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.105+0000 7f7f6e52e640 1 -- 172.21.15.43:0/2828731796 wait complete. 2024-04-21T08:38:58.116 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-04-21T08:38:56.901532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1194985,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1528,"num_read_kb":935,"num_write":560,"num_write_kb":4416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":947,"ondisk_log_size":947,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386508,"kb_used_data":16520,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124555316,"statfs":{"total":1151940427776,"available":1151544643584,"internally_reserved":0,"allocated":16916480,"data_stored":9972833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":378713404},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.989687"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005622+0000","last_change":"2024-04-21T08:36:23.983921+0000","last_active":"2024-04-21T08:38:55.005622+0000","last_peered":"2024-04-21T08:38:55.005622+0000","last_clean":"2024-04-21T08:38:55.005622+0000","last_became_active":"2024-04-21T08:36:23.983638+0000","last_became_peered":"2024-04-21T08:36:23.983638+0000","last_unstale":"2024-04-21T08:38:55.005622+0000","last_undegraded":"2024-04-21T08:38:55.005622+0000","last_fullsized":"2024-04-21T08:38:55.005622+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:29:44.646308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004430+0000","last_change":"2024-04-21T08:36:23.990029+0000","last_active":"2024-04-21T08:38:55.004430+0000","last_peered":"2024-04-21T08:38:55.004430+0000","last_clean":"2024-04-21T08:38:55.004430+0000","last_became_active":"2024-04-21T08:36:23.989141+0000","last_became_peered":"2024-04-21T08:36:23.989141+0000","last_unstale":"2024-04-21T08:38:55.004430+0000","last_undegraded":"2024-04-21T08:38:55.004430+0000","last_fullsized":"2024-04-21T08:38:55.004430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:23:58.692342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004234+0000","last_change":"2024-04-21T08:36:23.982205+0000","last_active":"2024-04-21T08:38:55.004234+0000","last_peered":"2024-04-21T08:38:55.004234+0000","last_clean":"2024-04-21T08:38:55.004234+0000","last_became_active":"2024-04-21T08:36:23.981876+0000","last_became_peered":"2024-04-21T08:36:23.981876+0000","last_unstale":"2024-04-21T08:38:55.004234+0000","last_undegraded":"2024-04-21T08:38:55.004234+0000","last_fullsized":"2024-04-21T08:38:55.004234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:12:27.121106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006754+0000","last_change":"2024-04-21T08:36:23.987425+0000","last_active":"2024-04-21T08:38:55.006754+0000","last_peered":"2024-04-21T08:38:55.006754+0000","last_clean":"2024-04-21T08:38:55.006754+0000","last_became_active":"2024-04-21T08:36:23.987109+0000","last_became_peered":"2024-04-21T08:36:23.987109+0000","last_unstale":"2024-04-21T08:38:55.006754+0000","last_undegraded":"2024-04-21T08:38:55.006754+0000","last_fullsized":"2024-04-21T08:38:55.006754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:44.176324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890536+0000","last_change":"2024-04-21T08:36:23.990375+0000","last_active":"2024-04-21T08:38:54.890536+0000","last_peered":"2024-04-21T08:38:54.890536+0000","last_clean":"2024-04-21T08:38:54.890536+0000","last_became_active":"2024-04-21T08:36:23.990058+0000","last_became_peered":"2024-04-21T08:36:23.990058+0000","last_unstale":"2024-04-21T08:38:54.890536+0000","last_undegraded":"2024-04-21T08:38:54.890536+0000","last_fullsized":"2024-04-21T08:38:54.890536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:24:09.647528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888221+0000","last_change":"2024-04-21T08:36:23.979848+0000","last_active":"2024-04-21T08:38:54.888221+0000","last_peered":"2024-04-21T08:38:54.888221+0000","last_clean":"2024-04-21T08:38:54.888221+0000","last_became_active":"2024-04-21T08:36:23.979511+0000","last_became_peered":"2024-04-21T08:36:23.979511+0000","last_unstale":"2024-04-21T08:38:54.888221+0000","last_undegraded":"2024-04-21T08:38:54.888221+0000","last_fullsized":"2024-04-21T08:38:54.888221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:07:46.551369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006232+0000","last_change":"2024-04-21T08:37:15.118755+0000","last_active":"2024-04-21T08:38:55.006232+0000","last_peered":"2024-04-21T08:38:55.006232+0000","last_clean":"2024-04-21T08:38:55.006232+0000","last_became_active":"2024-04-21T08:37:15.118347+0000","last_became_peered":"2024-04-21T08:37:15.118347+0000","last_unstale":"2024-04-21T08:38:55.006232+0000","last_undegraded":"2024-04-21T08:38:55.006232+0000","last_fullsized":"2024-04-21T08:38:55.006232+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:36:43.398325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886870+0000","last_change":"2024-04-21T08:36:23.987517+0000","last_active":"2024-04-21T08:38:54.886870+0000","last_peered":"2024-04-21T08:38:54.886870+0000","last_clean":"2024-04-21T08:38:54.886870+0000","last_became_active":"2024-04-21T08:36:23.987201+0000","last_became_peered":"2024-04-21T08:36:23.987201+0000","last_unstale":"2024-04-21T08:38:54.886870+0000","last_undegraded":"2024-04-21T08:38:54.886870+0000","last_fullsized":"2024-04-21T08:38:54.886870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:42:53.080863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104900+0000","last_change":"2024-04-21T08:36:23.989434+0000","last_active":"2024-04-21T08:38:55.104900+0000","last_peered":"2024-04-21T08:38:55.104900+0000","last_clean":"2024-04-21T08:38:55.104900+0000","last_became_active":"2024-04-21T08:36:23.989125+0000","last_became_peered":"2024-04-21T08:36:23.989125+0000","last_unstale":"2024-04-21T08:38:55.104900+0000","last_undegraded":"2024-04-21T08:38:55.104900+0000","last_fullsized":"2024-04-21T08:38:55.104900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:30:07.219056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888449+0000","last_change":"2024-04-21T08:36:23.980725+0000","last_active":"2024-04-21T08:38:54.888449+0000","last_peered":"2024-04-21T08:38:54.888449+0000","last_clean":"2024-04-21T08:38:54.888449+0000","last_became_active":"2024-04-21T08:36:23.980363+0000","last_became_peered":"2024-04-21T08:36:23.980363+0000","last_unstale":"2024-04-21T08:38:54.888449+0000","last_undegraded":"2024-04-21T08:38:54.888449+0000","last_fullsized":"2024-04-21T08:38:54.888449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:26:46.202030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006978+0000","last_change":"2024-04-21T08:36:23.989369+0000","last_active":"2024-04-21T08:38:55.006978+0000","last_peered":"2024-04-21T08:38:55.006978+0000","last_clean":"2024-04-21T08:38:55.006978+0000","last_became_active":"2024-04-21T08:36:23.989030+0000","last_became_peered":"2024-04-21T08:36:23.989030+0000","last_unstale":"2024-04-21T08:38:55.006978+0000","last_undegraded":"2024-04-21T08:38:55.006978+0000","last_fullsized":"2024-04-21T08:38:55.006978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:23:13.725703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006691+0000","last_change":"2024-04-21T08:36:23.993485+0000","last_active":"2024-04-21T08:38:55.006691+0000","last_peered":"2024-04-21T08:38:55.006691+0000","last_clean":"2024-04-21T08:38:55.006691+0000","last_became_active":"2024-04-21T08:36:23.993124+0000","last_became_peered":"2024-04-21T08:36:23.993124+0000","last_unstale":"2024-04-21T08:38:55.006691+0000","last_undegraded":"2024-04-21T08:38:55.006691+0000","last_fullsized":"2024-04-21T08:38:55.006691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:53:16.249838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076569+0000","last_change":"2024-04-21T08:36:23.983146+0000","last_active":"2024-04-21T08:38:55.076569+0000","last_peered":"2024-04-21T08:38:55.076569+0000","last_clean":"2024-04-21T08:38:55.076569+0000","last_became_active":"2024-04-21T08:36:23.982766+0000","last_became_peered":"2024-04-21T08:36:23.982766+0000","last_unstale":"2024-04-21T08:38:55.076569+0000","last_undegraded":"2024-04-21T08:38:55.076569+0000","last_fullsized":"2024-04-21T08:38:55.076569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:30:44.090929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.003129+0000","last_change":"2024-04-21T08:36:23.988906+0000","last_active":"2024-04-21T08:38:55.003129+0000","last_peered":"2024-04-21T08:38:55.003129+0000","last_clean":"2024-04-21T08:38:55.003129+0000","last_became_active":"2024-04-21T08:36:23.988489+0000","last_became_peered":"2024-04-21T08:36:23.988489+0000","last_unstale":"2024-04-21T08:38:55.003129+0000","last_undegraded":"2024-04-21T08:38:55.003129+0000","last_fullsized":"2024-04-21T08:38:55.003129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:06:43.271650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.003060+0000","last_change":"2024-04-21T08:37:15.123343+0000","last_active":"2024-04-21T08:38:55.003060+0000","last_peered":"2024-04-21T08:38:55.003060+0000","last_clean":"2024-04-21T08:38:55.003060+0000","last_became_active":"2024-04-21T08:37:15.122959+0000","last_became_peered":"2024-04-21T08:37:15.122959+0000","last_unstale":"2024-04-21T08:38:55.003060+0000","last_undegraded":"2024-04-21T08:38:55.003060+0000","last_fullsized":"2024-04-21T08:38:55.003060+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:56:19.179879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104471+0000","last_change":"2024-04-21T08:36:23.983136+0000","last_active":"2024-04-21T08:38:55.104471+0000","last_peered":"2024-04-21T08:38:55.104471+0000","last_clean":"2024-04-21T08:38:55.104471+0000","last_became_active":"2024-04-21T08:36:23.982698+0000","last_became_peered":"2024-04-21T08:36:23.982698+0000","last_unstale":"2024-04-21T08:38:55.104471+0000","last_undegraded":"2024-04-21T08:38:55.104471+0000","last_fullsized":"2024-04-21T08:38:55.104471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:34.750941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005563+0000","last_change":"2024-04-21T08:36:23.984701+0000","last_active":"2024-04-21T08:38:55.005563+0000","last_peered":"2024-04-21T08:38:55.005563+0000","last_clean":"2024-04-21T08:38:55.005563+0000","last_became_active":"2024-04-21T08:36:23.984432+0000","last_became_peered":"2024-04-21T08:36:23.984432+0000","last_unstale":"2024-04-21T08:38:55.005563+0000","last_undegraded":"2024-04-21T08:38:55.005563+0000","last_fullsized":"2024-04-21T08:38:55.005563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:10:58.944380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.996823+0000","last_change":"2024-04-21T08:36:23.982781+0000","last_active":"2024-04-21T08:38:54.996823+0000","last_peered":"2024-04-21T08:38:54.996823+0000","last_clean":"2024-04-21T08:38:54.996823+0000","last_became_active":"2024-04-21T08:36:23.982459+0000","last_became_peered":"2024-04-21T08:36:23.982459+0000","last_unstale":"2024-04-21T08:38:54.996823+0000","last_undegraded":"2024-04-21T08:38:54.996823+0000","last_fullsized":"2024-04-21T08:38:54.996823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:44.657504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.105000+0000","last_change":"2024-04-21T08:36:23.992002+0000","last_active":"2024-04-21T08:38:55.105000+0000","last_peered":"2024-04-21T08:38:55.105000+0000","last_clean":"2024-04-21T08:38:55.105000+0000","last_became_active":"2024-04-21T08:36:23.991053+0000","last_became_peered":"2024-04-21T08:36:23.991053+0000","last_unstale":"2024-04-21T08:38:55.105000+0000","last_undegraded":"2024-04-21T08:38:55.105000+0000","last_fullsized":"2024-04-21T08:38:55.105000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:56:37.145949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104844+0000","last_change":"2024-04-21T08:36:28.013195+0000","last_active":"2024-04-21T08:38:55.104844+0000","last_peered":"2024-04-21T08:38:55.104844+0000","last_clean":"2024-04-21T08:38:55.104844+0000","last_became_active":"2024-04-21T08:36:28.012656+0000","last_became_peered":"2024-04-21T08:36:28.012656+0000","last_unstale":"2024-04-21T08:38:55.104844+0000","last_undegraded":"2024-04-21T08:38:55.104844+0000","last_fullsized":"2024-04-21T08:38:55.104844+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:12:03.133583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"95'14","reported_seq":173,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.074681+0000","last_change":"2024-04-21T08:36:23.468957+0000","last_active":"2024-04-21T08:38:55.074681+0000","last_peered":"2024-04-21T08:38:55.074681+0000","last_clean":"2024-04-21T08:38:55.074681+0000","last_became_active":"2024-04-21T08:36:23.468601+0000","last_became_peered":"2024-04-21T08:36:23.468601+0000","last_unstale":"2024-04-21T08:38:55.074681+0000","last_undegraded":"2024-04-21T08:38:55.074681+0000","last_fullsized":"2024-04-21T08:38:55.074681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:13:11.977780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":125,"num_read_kb":125,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.996013+0000","last_change":"2024-04-21T08:37:15.123476+0000","last_active":"2024-04-21T08:38:54.996013+0000","last_peered":"2024-04-21T08:38:54.996013+0000","last_clean":"2024-04-21T08:38:54.996013+0000","last_became_active":"2024-04-21T08:37:15.123066+0000","last_became_peered":"2024-04-21T08:37:15.123066+0000","last_unstale":"2024-04-21T08:38:54.996013+0000","last_undegraded":"2024-04-21T08:38:54.996013+0000","last_fullsized":"2024-04-21T08:38:54.996013+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:15.786178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005900+0000","last_change":"2024-04-21T08:36:28.014161+0000","last_active":"2024-04-21T08:38:55.005900+0000","last_peered":"2024-04-21T08:38:55.005900+0000","last_clean":"2024-04-21T08:38:55.005900+0000","last_became_active":"2024-04-21T08:36:28.013859+0000","last_became_peered":"2024-04-21T08:36:28.013859+0000","last_unstale":"2024-04-21T08:38:55.005900+0000","last_undegraded":"2024-04-21T08:38:55.005900+0000","last_fullsized":"2024-04-21T08:38:55.005900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:08.315334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887665+0000","last_change":"2024-04-21T08:36:23.475612+0000","last_active":"2024-04-21T08:38:54.887665+0000","last_peered":"2024-04-21T08:38:54.887665+0000","last_clean":"2024-04-21T08:38:54.887665+0000","last_became_active":"2024-04-21T08:36:23.475301+0000","last_became_peered":"2024-04-21T08:36:23.475301+0000","last_unstale":"2024-04-21T08:38:54.887665+0000","last_undegraded":"2024-04-21T08:38:54.887665+0000","last_fullsized":"2024-04-21T08:38:54.887665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:56.801630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004414+0000","last_change":"2024-04-21T08:36:23.984689+0000","last_active":"2024-04-21T08:38:55.004414+0000","last_peered":"2024-04-21T08:38:55.004414+0000","last_clean":"2024-04-21T08:38:55.004414+0000","last_became_active":"2024-04-21T08:36:23.984358+0000","last_became_peered":"2024-04-21T08:36:23.984358+0000","last_unstale":"2024-04-21T08:38:55.004414+0000","last_undegraded":"2024-04-21T08:38:55.004414+0000","last_fullsized":"2024-04-21T08:38:55.004414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:49:50.187466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006203+0000","last_change":"2024-04-21T08:36:28.007951+0000","last_active":"2024-04-21T08:38:55.006203+0000","last_peered":"2024-04-21T08:38:55.006203+0000","last_clean":"2024-04-21T08:38:55.006203+0000","last_became_active":"2024-04-21T08:36:28.007553+0000","last_became_peered":"2024-04-21T08:36:28.007553+0000","last_unstale":"2024-04-21T08:38:55.006203+0000","last_undegraded":"2024-04-21T08:38:55.006203+0000","last_fullsized":"2024-04-21T08:38:55.006203+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:10:54.019605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890572+0000","last_change":"2024-04-21T08:36:23.473371+0000","last_active":"2024-04-21T08:38:54.890572+0000","last_peered":"2024-04-21T08:38:54.890572+0000","last_clean":"2024-04-21T08:38:54.890572+0000","last_became_active":"2024-04-21T08:36:23.472472+0000","last_became_peered":"2024-04-21T08:36:23.472472+0000","last_unstale":"2024-04-21T08:38:54.890572+0000","last_undegraded":"2024-04-21T08:38:54.890572+0000","last_fullsized":"2024-04-21T08:38:54.890572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:28:49.622873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104901+0000","last_change":"2024-04-21T08:36:23.987916+0000","last_active":"2024-04-21T08:38:55.104901+0000","last_peered":"2024-04-21T08:38:55.104901+0000","last_clean":"2024-04-21T08:38:55.104901+0000","last_became_active":"2024-04-21T08:36:23.986207+0000","last_became_peered":"2024-04-21T08:36:23.986207+0000","last_unstale":"2024-04-21T08:38:55.104901+0000","last_undegraded":"2024-04-21T08:38:55.104901+0000","last_fullsized":"2024-04-21T08:38:55.104901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:49:57.261078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006246+0000","last_change":"2024-04-21T08:36:28.006494+0000","last_active":"2024-04-21T08:38:55.006246+0000","last_peered":"2024-04-21T08:38:55.006246+0000","last_clean":"2024-04-21T08:38:55.006246+0000","last_became_active":"2024-04-21T08:36:28.006069+0000","last_became_peered":"2024-04-21T08:36:28.006069+0000","last_unstale":"2024-04-21T08:38:55.006246+0000","last_undegraded":"2024-04-21T08:38:55.006246+0000","last_fullsized":"2024-04-21T08:38:55.006246+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:04.608295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890975+0000","last_change":"2024-04-21T08:36:23.468923+0000","last_active":"2024-04-21T08:38:54.890975+0000","last_peered":"2024-04-21T08:38:54.890975+0000","last_clean":"2024-04-21T08:38:54.890975+0000","last_became_active":"2024-04-21T08:36:23.468509+0000","last_became_peered":"2024-04-21T08:36:23.468509+0000","last_unstale":"2024-04-21T08:38:54.890975+0000","last_undegraded":"2024-04-21T08:38:54.890975+0000","last_fullsized":"2024-04-21T08:38:54.890975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:44.452748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008259+0000","last_change":"2024-04-21T08:36:23.994423+0000","last_active":"2024-04-21T08:38:55.008259+0000","last_peered":"2024-04-21T08:38:55.008259+0000","last_clean":"2024-04-21T08:38:55.008259+0000","last_became_active":"2024-04-21T08:36:23.993827+0000","last_became_peered":"2024-04-21T08:36:23.993827+0000","last_unstale":"2024-04-21T08:38:55.008259+0000","last_undegraded":"2024-04-21T08:38:55.008259+0000","last_fullsized":"2024-04-21T08:38:55.008259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:49:25.550301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104711+0000","last_change":"2024-04-21T08:36:28.004037+0000","last_active":"2024-04-21T08:38:55.104711+0000","last_peered":"2024-04-21T08:38:55.104711+0000","last_clean":"2024-04-21T08:38:55.104711+0000","last_became_active":"2024-04-21T08:36:28.003580+0000","last_became_peered":"2024-04-21T08:36:28.003580+0000","last_unstale":"2024-04-21T08:38:55.104711+0000","last_undegraded":"2024-04-21T08:38:55.104711+0000","last_fullsized":"2024-04-21T08:38:55.104711+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:04:51.380244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'8","reported_seq":172,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.181108+0000","last_change":"2024-04-21T08:36:23.468430+0000","last_active":"2024-04-21T08:38:55.181108+0000","last_peered":"2024-04-21T08:38:55.181108+0000","last_clean":"2024-04-21T08:38:55.181108+0000","last_became_active":"2024-04-21T08:36:23.468089+0000","last_became_peered":"2024-04-21T08:36:23.468089+0000","last_unstale":"2024-04-21T08:38:55.181108+0000","last_undegraded":"2024-04-21T08:38:55.181108+0000","last_fullsized":"2024-04-21T08:38:55.181108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:12:54.827034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":130,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890458+0000","last_change":"2024-04-21T08:37:15.117922+0000","last_active":"2024-04-21T08:38:54.890458+0000","last_peered":"2024-04-21T08:38:54.890458+0000","last_clean":"2024-04-21T08:38:54.890458+0000","last_became_active":"2024-04-21T08:37:15.117404+0000","last_became_peered":"2024-04-21T08:37:15.117404+0000","last_unstale":"2024-04-21T08:38:54.890458+0000","last_undegraded":"2024-04-21T08:38:54.890458+0000","last_fullsized":"2024-04-21T08:38:54.890458+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:39.682224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104672+0000","last_change":"2024-04-21T08:36:28.004638+0000","last_active":"2024-04-21T08:38:55.104672+0000","last_peered":"2024-04-21T08:38:55.104672+0000","last_clean":"2024-04-21T08:38:55.104672+0000","last_became_active":"2024-04-21T08:36:28.002499+0000","last_became_peered":"2024-04-21T08:36:28.002499+0000","last_unstale":"2024-04-21T08:38:55.104672+0000","last_undegraded":"2024-04-21T08:38:55.104672+0000","last_fullsized":"2024-04-21T08:38:55.104672+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:45:46.934783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":55,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888089+0000","last_change":"2024-04-21T08:37:15.126569+0000","last_active":"2024-04-21T08:38:54.888089+0000","last_peered":"2024-04-21T08:38:54.888089+0000","last_clean":"2024-04-21T08:38:54.888089+0000","last_became_active":"2024-04-21T08:37:15.121284+0000","last_became_peered":"2024-04-21T08:37:15.121284+0000","last_unstale":"2024-04-21T08:38:54.888089+0000","last_undegraded":"2024-04-21T08:38:54.888089+0000","last_fullsized":"2024-04-21T08:38:54.888089+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:02:26.153324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008374+0000","last_change":"2024-04-21T08:37:15.557192+0000","last_active":"2024-04-21T08:38:55.008374+0000","last_peered":"2024-04-21T08:38:55.008374+0000","last_clean":"2024-04-21T08:38:55.008374+0000","last_became_active":"2024-04-21T08:37:15.556653+0000","last_became_peered":"2024-04-21T08:37:15.556653+0000","last_unstale":"2024-04-21T08:38:55.008374+0000","last_undegraded":"2024-04-21T08:38:55.008374+0000","last_fullsized":"2024-04-21T08:38:55.008374+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:42.722460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005527+0000","last_change":"2024-04-21T08:36:28.017068+0000","last_active":"2024-04-21T08:38:55.005527+0000","last_peered":"2024-04-21T08:38:55.005527+0000","last_clean":"2024-04-21T08:38:55.005527+0000","last_became_active":"2024-04-21T08:36:28.016770+0000","last_became_peered":"2024-04-21T08:36:28.016770+0000","last_unstale":"2024-04-21T08:38:55.005527+0000","last_undegraded":"2024-04-21T08:38:55.005527+0000","last_fullsized":"2024-04-21T08:38:55.005527+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:07:14.596090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076311+0000","last_change":"2024-04-21T08:36:23.470430+0000","last_active":"2024-04-21T08:38:55.076311+0000","last_peered":"2024-04-21T08:38:55.076311+0000","last_clean":"2024-04-21T08:38:55.076311+0000","last_became_active":"2024-04-21T08:36:23.470109+0000","last_became_peered":"2024-04-21T08:36:23.470109+0000","last_unstale":"2024-04-21T08:38:55.076311+0000","last_undegraded":"2024-04-21T08:38:55.076311+0000","last_fullsized":"2024-04-21T08:38:55.076311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:18.176313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887026+0000","last_change":"2024-04-21T08:36:23.986668+0000","last_active":"2024-04-21T08:38:54.887026+0000","last_peered":"2024-04-21T08:38:54.887026+0000","last_clean":"2024-04-21T08:38:54.887026+0000","last_became_active":"2024-04-21T08:36:23.986344+0000","last_became_peered":"2024-04-21T08:36:23.986344+0000","last_unstale":"2024-04-21T08:38:54.887026+0000","last_undegraded":"2024-04-21T08:38:54.887026+0000","last_fullsized":"2024-04-21T08:38:54.887026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:56:17.744145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004263+0000","last_change":"2024-04-21T08:36:28.009012+0000","last_active":"2024-04-21T08:38:55.004263+0000","last_peered":"2024-04-21T08:38:55.004263+0000","last_clean":"2024-04-21T08:38:55.004263+0000","last_became_active":"2024-04-21T08:36:28.008123+0000","last_became_peered":"2024-04-21T08:36:28.008123+0000","last_unstale":"2024-04-21T08:38:55.004263+0000","last_undegraded":"2024-04-21T08:38:55.004263+0000","last_fullsized":"2024-04-21T08:38:55.004263+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:33:18.881564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007854+0000","last_change":"2024-04-21T08:36:23.467104+0000","last_active":"2024-04-21T08:38:55.007854+0000","last_peered":"2024-04-21T08:38:55.007854+0000","last_clean":"2024-04-21T08:38:55.007854+0000","last_became_active":"2024-04-21T08:36:23.466725+0000","last_became_peered":"2024-04-21T08:36:23.466725+0000","last_unstale":"2024-04-21T08:38:55.007854+0000","last_undegraded":"2024-04-21T08:38:55.007854+0000","last_fullsized":"2024-04-21T08:38:55.007854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:35:45.986448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.995844+0000","last_change":"2024-04-21T08:36:23.987579+0000","last_active":"2024-04-21T08:38:54.995844+0000","last_peered":"2024-04-21T08:38:54.995844+0000","last_clean":"2024-04-21T08:38:54.995844+0000","last_became_active":"2024-04-21T08:36:23.987044+0000","last_became_peered":"2024-04-21T08:36:23.987044+0000","last_unstale":"2024-04-21T08:38:54.995844+0000","last_undegraded":"2024-04-21T08:38:54.995844+0000","last_fullsized":"2024-04-21T08:38:54.995844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:48:27.593090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886895+0000","last_change":"2024-04-21T08:36:28.006417+0000","last_active":"2024-04-21T08:38:54.886895+0000","last_peered":"2024-04-21T08:38:54.886895+0000","last_clean":"2024-04-21T08:38:54.886895+0000","last_became_active":"2024-04-21T08:36:28.006006+0000","last_became_peered":"2024-04-21T08:36:28.006006+0000","last_unstale":"2024-04-21T08:38:54.886895+0000","last_undegraded":"2024-04-21T08:38:54.886895+0000","last_fullsized":"2024-04-21T08:38:54.886895+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:16.173284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'8","reported_seq":292,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.181085+0000","last_change":"2024-04-21T08:36:23.471541+0000","last_active":"2024-04-21T08:38:55.181085+0000","last_peered":"2024-04-21T08:38:55.181085+0000","last_clean":"2024-04-21T08:38:55.181085+0000","last_became_active":"2024-04-21T08:36:23.471193+0000","last_became_peered":"2024-04-21T08:36:23.471193+0000","last_unstale":"2024-04-21T08:38:55.181085+0000","last_undegraded":"2024-04-21T08:38:55.181085+0000","last_fullsized":"2024-04-21T08:38:55.181085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:06:04.916707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":250,"num_read_kb":250,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104848+0000","last_change":"2024-04-21T08:36:23.986781+0000","last_active":"2024-04-21T08:38:55.104848+0000","last_peered":"2024-04-21T08:38:55.104848+0000","last_clean":"2024-04-21T08:38:55.104848+0000","last_became_active":"2024-04-21T08:36:23.985696+0000","last_became_peered":"2024-04-21T08:36:23.985696+0000","last_unstale":"2024-04-21T08:38:55.104848+0000","last_undegraded":"2024-04-21T08:38:55.104848+0000","last_fullsized":"2024-04-21T08:38:55.104848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:40.556642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.995747+0000","last_change":"2024-04-21T08:36:28.011621+0000","last_active":"2024-04-21T08:38:54.995747+0000","last_peered":"2024-04-21T08:38:54.995747+0000","last_clean":"2024-04-21T08:38:54.995747+0000","last_became_active":"2024-04-21T08:36:28.011136+0000","last_became_peered":"2024-04-21T08:36:28.011136+0000","last_unstale":"2024-04-21T08:38:54.995747+0000","last_undegraded":"2024-04-21T08:38:54.995747+0000","last_fullsized":"2024-04-21T08:38:54.995747+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:47.623445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006842+0000","last_change":"2024-04-21T08:36:23.474019+0000","last_active":"2024-04-21T08:38:55.006842+0000","last_peered":"2024-04-21T08:38:55.006842+0000","last_clean":"2024-04-21T08:38:55.006842+0000","last_became_active":"2024-04-21T08:36:23.473690+0000","last_became_peered":"2024-04-21T08:36:23.473690+0000","last_unstale":"2024-04-21T08:38:55.006842+0000","last_undegraded":"2024-04-21T08:38:55.006842+0000","last_fullsized":"2024-04-21T08:38:55.006842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:38:32.913795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004613+0000","last_change":"2024-04-21T08:37:15.121455+0000","last_active":"2024-04-21T08:38:55.004613+0000","last_peered":"2024-04-21T08:38:55.004613+0000","last_clean":"2024-04-21T08:38:55.004613+0000","last_became_active":"2024-04-21T08:37:15.121113+0000","last_became_peered":"2024-04-21T08:37:15.121113+0000","last_unstale":"2024-04-21T08:38:55.004613+0000","last_undegraded":"2024-04-21T08:38:55.004613+0000","last_fullsized":"2024-04-21T08:38:55.004613+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:39:36.525495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006512+0000","last_change":"2024-04-21T08:38:15.413849+0000","last_active":"2024-04-21T08:38:55.006512+0000","last_peered":"2024-04-21T08:38:55.006512+0000","last_clean":"2024-04-21T08:38:55.006512+0000","last_became_active":"2024-04-21T08:38:15.413361+0000","last_became_peered":"2024-04-21T08:38:15.413361+0000","last_unstale":"2024-04-21T08:38:55.006512+0000","last_undegraded":"2024-04-21T08:38:55.006512+0000","last_fullsized":"2024-04-21T08:38:55.006512+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:16:49.197800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008173+0000","last_change":"2024-04-21T08:36:23.465825+0000","last_active":"2024-04-21T08:38:55.008173+0000","last_peered":"2024-04-21T08:38:55.008173+0000","last_clean":"2024-04-21T08:38:55.008173+0000","last_became_active":"2024-04-21T08:36:23.465525+0000","last_became_peered":"2024-04-21T08:36:23.465525+0000","last_unstale":"2024-04-21T08:38:55.008173+0000","last_undegraded":"2024-04-21T08:38:55.008173+0000","last_fullsized":"2024-04-21T08:38:55.008173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:59:50.502210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.105005+0000","last_change":"2024-04-21T08:36:23.987602+0000","last_active":"2024-04-21T08:38:55.105005+0000","last_peered":"2024-04-21T08:38:55.105005+0000","last_clean":"2024-04-21T08:38:55.105005+0000","last_became_active":"2024-04-21T08:36:23.987298+0000","last_became_peered":"2024-04-21T08:36:23.987298+0000","last_unstale":"2024-04-21T08:38:55.105005+0000","last_undegraded":"2024-04-21T08:38:55.105005+0000","last_fullsized":"2024-04-21T08:38:55.105005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:06.647676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.995946+0000","last_change":"2024-04-21T08:36:28.005935+0000","last_active":"2024-04-21T08:38:54.995946+0000","last_peered":"2024-04-21T08:38:54.995946+0000","last_clean":"2024-04-21T08:38:54.995946+0000","last_became_active":"2024-04-21T08:36:28.005512+0000","last_became_peered":"2024-04-21T08:36:28.005512+0000","last_unstale":"2024-04-21T08:38:54.995946+0000","last_undegraded":"2024-04-21T08:38:54.995946+0000","last_fullsized":"2024-04-21T08:38:54.995946+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:23:20.298524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"95'43","reported_seq":341,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.178912+0000","last_change":"2024-04-21T08:36:23.467728+0000","last_active":"2024-04-21T08:38:55.178912+0000","last_peered":"2024-04-21T08:38:55.178912+0000","last_clean":"2024-04-21T08:38:55.178912+0000","last_became_active":"2024-04-21T08:36:23.467430+0000","last_became_peered":"2024-04-21T08:36:23.467430+0000","last_unstale":"2024-04-21T08:38:55.178912+0000","last_undegraded":"2024-04-21T08:38:55.178912+0000","last_fullsized":"2024-04-21T08:38:55.178912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:47:18.900911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70068,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":264,"num_read_kb":201,"num_write":45,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890735+0000","last_change":"2024-04-21T08:36:23.989116+0000","last_active":"2024-04-21T08:38:54.890735+0000","last_peered":"2024-04-21T08:38:54.890735+0000","last_clean":"2024-04-21T08:38:54.890735+0000","last_became_active":"2024-04-21T08:36:23.988758+0000","last_became_peered":"2024-04-21T08:36:23.988758+0000","last_unstale":"2024-04-21T08:38:54.890735+0000","last_undegraded":"2024-04-21T08:38:54.890735+0000","last_fullsized":"2024-04-21T08:38:54.890735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:56:25.969905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008450+0000","last_change":"2024-04-21T08:36:28.007370+0000","last_active":"2024-04-21T08:38:55.008450+0000","last_peered":"2024-04-21T08:38:55.008450+0000","last_clean":"2024-04-21T08:38:55.008450+0000","last_became_active":"2024-04-21T08:36:28.006972+0000","last_became_peered":"2024-04-21T08:36:28.006972+0000","last_unstale":"2024-04-21T08:38:55.008450+0000","last_undegraded":"2024-04-21T08:38:55.008450+0000","last_fullsized":"2024-04-21T08:38:55.008450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:18:14.926058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104708+0000","last_change":"2024-04-21T08:36:23.472658+0000","last_active":"2024-04-21T08:38:55.104708+0000","last_peered":"2024-04-21T08:38:55.104708+0000","last_clean":"2024-04-21T08:38:55.104708+0000","last_became_active":"2024-04-21T08:36:23.472318+0000","last_became_peered":"2024-04-21T08:36:23.472318+0000","last_unstale":"2024-04-21T08:38:55.104708+0000","last_undegraded":"2024-04-21T08:38:55.104708+0000","last_fullsized":"2024-04-21T08:38:55.104708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:28:27.497605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076501+0000","last_change":"2024-04-21T08:38:15.338892+0000","last_active":"2024-04-21T08:38:55.076501+0000","last_peered":"2024-04-21T08:38:55.076501+0000","last_clean":"2024-04-21T08:38:55.076501+0000","last_became_active":"2024-04-21T08:38:15.338396+0000","last_became_peered":"2024-04-21T08:38:15.338396+0000","last_unstale":"2024-04-21T08:38:55.076501+0000","last_undegraded":"2024-04-21T08:38:55.076501+0000","last_fullsized":"2024-04-21T08:38:55.076501+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:19.670635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005342+0000","last_change":"2024-04-21T08:36:28.019186+0000","last_active":"2024-04-21T08:38:55.005342+0000","last_peered":"2024-04-21T08:38:55.005342+0000","last_clean":"2024-04-21T08:38:55.005342+0000","last_became_active":"2024-04-21T08:36:28.018774+0000","last_became_peered":"2024-04-21T08:36:28.018774+0000","last_unstale":"2024-04-21T08:38:55.005342+0000","last_undegraded":"2024-04-21T08:38:55.005342+0000","last_fullsized":"2024-04-21T08:38:55.005342+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:35:33.548656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008059+0000","last_change":"2024-04-21T08:36:23.472633+0000","last_active":"2024-04-21T08:38:55.008059+0000","last_peered":"2024-04-21T08:38:55.008059+0000","last_clean":"2024-04-21T08:38:55.008059+0000","last_became_active":"2024-04-21T08:36:23.472336+0000","last_became_peered":"2024-04-21T08:36:23.472336+0000","last_unstale":"2024-04-21T08:38:55.008059+0000","last_undegraded":"2024-04-21T08:38:55.008059+0000","last_fullsized":"2024-04-21T08:38:55.008059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:49.463492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006193+0000","last_change":"2024-04-21T08:36:23.994332+0000","last_active":"2024-04-21T08:38:55.006193+0000","last_peered":"2024-04-21T08:38:55.006193+0000","last_clean":"2024-04-21T08:38:55.006193+0000","last_became_active":"2024-04-21T08:36:23.993936+0000","last_became_peered":"2024-04-21T08:36:23.993936+0000","last_unstale":"2024-04-21T08:38:55.006193+0000","last_undegraded":"2024-04-21T08:38:55.006193+0000","last_fullsized":"2024-04-21T08:38:55.006193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:26:35.330559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.996887+0000","last_change":"2024-04-21T08:36:28.012549+0000","last_active":"2024-04-21T08:38:54.996887+0000","last_peered":"2024-04-21T08:38:54.996887+0000","last_clean":"2024-04-21T08:38:54.996887+0000","last_became_active":"2024-04-21T08:36:28.012151+0000","last_became_peered":"2024-04-21T08:36:28.012151+0000","last_unstale":"2024-04-21T08:38:54.996887+0000","last_undegraded":"2024-04-21T08:38:54.996887+0000","last_fullsized":"2024-04-21T08:38:54.996887+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:55.680399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'8","reported_seq":172,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076760+0000","last_change":"2024-04-21T08:36:23.464586+0000","last_active":"2024-04-21T08:38:55.076760+0000","last_peered":"2024-04-21T08:38:55.076760+0000","last_clean":"2024-04-21T08:38:55.076760+0000","last_became_active":"2024-04-21T08:36:23.464281+0000","last_became_peered":"2024-04-21T08:36:23.464281+0000","last_unstale":"2024-04-21T08:38:55.076760+0000","last_undegraded":"2024-04-21T08:38:55.076760+0000","last_fullsized":"2024-04-21T08:38:55.076760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:04:58.577283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":131,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006132+0000","last_change":"2024-04-21T08:36:23.989604+0000","last_active":"2024-04-21T08:38:55.006132+0000","last_peered":"2024-04-21T08:38:55.006132+0000","last_clean":"2024-04-21T08:38:55.006132+0000","last_became_active":"2024-04-21T08:36:23.989257+0000","last_became_peered":"2024-04-21T08:36:23.989257+0000","last_unstale":"2024-04-21T08:38:55.006132+0000","last_undegraded":"2024-04-21T08:38:55.006132+0000","last_fullsized":"2024-04-21T08:38:55.006132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:45.077943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077854+0000","last_change":"2024-04-21T08:36:28.007959+0000","last_active":"2024-04-21T08:38:55.077854+0000","last_peered":"2024-04-21T08:38:55.077854+0000","last_clean":"2024-04-21T08:38:55.077854+0000","last_became_active":"2024-04-21T08:36:28.007571+0000","last_became_peered":"2024-04-21T08:36:28.007571+0000","last_unstale":"2024-04-21T08:38:55.077854+0000","last_undegraded":"2024-04-21T08:38:55.077854+0000","last_fullsized":"2024-04-21T08:38:55.077854+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:17:34.007013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890204+0000","last_change":"2024-04-21T08:36:23.475054+0000","last_active":"2024-04-21T08:38:54.890204+0000","last_peered":"2024-04-21T08:38:54.890204+0000","last_clean":"2024-04-21T08:38:54.890204+0000","last_became_active":"2024-04-21T08:36:23.474606+0000","last_became_peered":"2024-04-21T08:36:23.474606+0000","last_unstale":"2024-04-21T08:38:54.890204+0000","last_undegraded":"2024-04-21T08:38:54.890204+0000","last_fullsized":"2024-04-21T08:38:54.890204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:22.559174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007909+0000","last_change":"2024-04-21T08:36:23.995980+0000","last_active":"2024-04-21T08:38:55.007909+0000","last_peered":"2024-04-21T08:38:55.007909+0000","last_clean":"2024-04-21T08:38:55.007909+0000","last_became_active":"2024-04-21T08:36:23.995673+0000","last_became_peered":"2024-04-21T08:36:23.995673+0000","last_unstale":"2024-04-21T08:38:55.007909+0000","last_undegraded":"2024-04-21T08:38:55.007909+0000","last_fullsized":"2024-04-21T08:38:55.007909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:11:49.884384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007831+0000","last_change":"2024-04-21T08:36:28.018792+0000","last_active":"2024-04-21T08:38:55.007831+0000","last_peered":"2024-04-21T08:38:55.007831+0000","last_clean":"2024-04-21T08:38:55.007831+0000","last_became_active":"2024-04-21T08:36:28.018373+0000","last_became_peered":"2024-04-21T08:36:28.018373+0000","last_unstale":"2024-04-21T08:38:55.007831+0000","last_undegraded":"2024-04-21T08:38:55.007831+0000","last_fullsized":"2024-04-21T08:38:55.007831+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:11.857661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890763+0000","last_change":"2024-04-21T08:36:23.469787+0000","last_active":"2024-04-21T08:38:54.890763+0000","last_peered":"2024-04-21T08:38:54.890763+0000","last_clean":"2024-04-21T08:38:54.890763+0000","last_became_active":"2024-04-21T08:36:23.469462+0000","last_became_peered":"2024-04-21T08:36:23.469462+0000","last_unstale":"2024-04-21T08:38:54.890763+0000","last_undegraded":"2024-04-21T08:38:54.890763+0000","last_fullsized":"2024-04-21T08:38:54.890763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:33:25.849528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005561+0000","last_change":"2024-04-21T08:36:23.988492+0000","last_active":"2024-04-21T08:38:55.005561+0000","last_peered":"2024-04-21T08:38:55.005561+0000","last_clean":"2024-04-21T08:38:55.005561+0000","last_became_active":"2024-04-21T08:36:23.988133+0000","last_became_peered":"2024-04-21T08:36:23.988133+0000","last_unstale":"2024-04-21T08:38:55.005561+0000","last_undegraded":"2024-04-21T08:38:55.005561+0000","last_fullsized":"2024-04-21T08:38:55.005561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:01:25.320281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890332+0000","last_change":"2024-04-21T08:38:15.415043+0000","last_active":"2024-04-21T08:38:54.890332+0000","last_peered":"2024-04-21T08:38:54.890332+0000","last_clean":"2024-04-21T08:38:54.890332+0000","last_became_active":"2024-04-21T08:38:15.414460+0000","last_became_peered":"2024-04-21T08:38:15.414460+0000","last_unstale":"2024-04-21T08:38:54.890332+0000","last_undegraded":"2024-04-21T08:38:54.890332+0000","last_fullsized":"2024-04-21T08:38:54.890332+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:50:17.202423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.996393+0000","last_change":"2024-04-21T08:37:15.219428+0000","last_active":"2024-04-21T08:38:54.996393+0000","last_peered":"2024-04-21T08:38:54.996393+0000","last_clean":"2024-04-21T08:38:54.996393+0000","last_became_active":"2024-04-21T08:37:15.218923+0000","last_became_peered":"2024-04-21T08:37:15.218923+0000","last_unstale":"2024-04-21T08:38:54.996393+0000","last_undegraded":"2024-04-21T08:38:54.996393+0000","last_fullsized":"2024-04-21T08:38:54.996393+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:07:02.781701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888120+0000","last_change":"2024-04-21T08:36:23.985910+0000","last_active":"2024-04-21T08:38:54.888120+0000","last_peered":"2024-04-21T08:38:54.888120+0000","last_clean":"2024-04-21T08:38:54.888120+0000","last_became_active":"2024-04-21T08:36:23.985605+0000","last_became_peered":"2024-04-21T08:36:23.985605+0000","last_unstale":"2024-04-21T08:38:54.888120+0000","last_undegraded":"2024-04-21T08:38:54.888120+0000","last_fullsized":"2024-04-21T08:38:54.888120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:37:09.367005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077277+0000","last_change":"2024-04-21T08:36:28.015256+0000","last_active":"2024-04-21T08:38:55.077277+0000","last_peered":"2024-04-21T08:38:55.077277+0000","last_clean":"2024-04-21T08:38:55.077277+0000","last_became_active":"2024-04-21T08:36:28.014849+0000","last_became_peered":"2024-04-21T08:36:28.014849+0000","last_unstale":"2024-04-21T08:38:55.077277+0000","last_undegraded":"2024-04-21T08:38:55.077277+0000","last_fullsized":"2024-04-21T08:38:55.077277+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:22:05.466299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"95'6","reported_seq":165,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.075177+0000","last_change":"2024-04-21T08:36:23.474966+0000","last_active":"2024-04-21T08:38:55.075177+0000","last_peered":"2024-04-21T08:38:55.075177+0000","last_clean":"2024-04-21T08:38:55.075177+0000","last_became_active":"2024-04-21T08:36:23.474374+0000","last_became_peered":"2024-04-21T08:36:23.474374+0000","last_unstale":"2024-04-21T08:38:55.075177+0000","last_undegraded":"2024-04-21T08:38:55.075177+0000","last_fullsized":"2024-04-21T08:38:55.075177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:57:20.237966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888138+0000","last_change":"2024-04-21T08:37:15.118887+0000","last_active":"2024-04-21T08:38:54.888138+0000","last_peered":"2024-04-21T08:38:54.888138+0000","last_clean":"2024-04-21T08:38:54.888138+0000","last_became_active":"2024-04-21T08:37:15.118510+0000","last_became_peered":"2024-04-21T08:37:15.118510+0000","last_unstale":"2024-04-21T08:38:54.888138+0000","last_undegraded":"2024-04-21T08:38:54.888138+0000","last_fullsized":"2024-04-21T08:38:54.888138+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:59:27.209083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077841+0000","last_change":"2024-04-21T08:36:28.013169+0000","last_active":"2024-04-21T08:38:55.077841+0000","last_peered":"2024-04-21T08:38:55.077841+0000","last_clean":"2024-04-21T08:38:55.077841+0000","last_became_active":"2024-04-21T08:36:28.012791+0000","last_became_peered":"2024-04-21T08:36:28.012791+0000","last_unstale":"2024-04-21T08:38:55.077841+0000","last_undegraded":"2024-04-21T08:38:55.077841+0000","last_fullsized":"2024-04-21T08:38:55.077841+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:42:56.063159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007983+0000","last_change":"2024-04-21T08:36:23.995209+0000","last_active":"2024-04-21T08:38:55.007983+0000","last_peered":"2024-04-21T08:38:55.007983+0000","last_clean":"2024-04-21T08:38:55.007983+0000","last_became_active":"2024-04-21T08:36:23.994866+0000","last_became_peered":"2024-04-21T08:36:23.994866+0000","last_unstale":"2024-04-21T08:38:55.007983+0000","last_undegraded":"2024-04-21T08:38:55.007983+0000","last_fullsized":"2024-04-21T08:38:55.007983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:33:45.582150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'7","reported_seq":168,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076172+0000","last_change":"2024-04-21T08:36:23.471474+0000","last_active":"2024-04-21T08:38:55.076172+0000","last_peered":"2024-04-21T08:38:55.076172+0000","last_clean":"2024-04-21T08:38:55.076172+0000","last_became_active":"2024-04-21T08:36:23.471179+0000","last_became_peered":"2024-04-21T08:36:23.471179+0000","last_unstale":"2024-04-21T08:38:55.076172+0000","last_undegraded":"2024-04-21T08:38:55.076172+0000","last_fullsized":"2024-04-21T08:38:55.076172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:17.290051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":127,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006553+0000","last_change":"2024-04-21T08:38:15.344332+0000","last_active":"2024-04-21T08:38:55.006553+0000","last_peered":"2024-04-21T08:38:55.006553+0000","last_clean":"2024-04-21T08:38:55.006553+0000","last_became_active":"2024-04-21T08:38:15.339914+0000","last_became_peered":"2024-04-21T08:38:15.339914+0000","last_unstale":"2024-04-21T08:38:55.006553+0000","last_undegraded":"2024-04-21T08:38:55.006553+0000","last_fullsized":"2024-04-21T08:38:55.006553+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:39:24.049323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886832+0000","last_change":"2024-04-21T08:36:23.990272+0000","last_active":"2024-04-21T08:38:54.886832+0000","last_peered":"2024-04-21T08:38:54.886832+0000","last_clean":"2024-04-21T08:38:54.886832+0000","last_became_active":"2024-04-21T08:36:23.989940+0000","last_became_peered":"2024-04-21T08:36:23.989940+0000","last_unstale":"2024-04-21T08:38:54.886832+0000","last_undegraded":"2024-04-21T08:38:54.886832+0000","last_fullsized":"2024-04-21T08:38:54.886832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:27:50.705167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006322+0000","last_change":"2024-04-21T08:36:23.474162+0000","last_active":"2024-04-21T08:38:55.006322+0000","last_peered":"2024-04-21T08:38:55.006322+0000","last_clean":"2024-04-21T08:38:55.006322+0000","last_became_active":"2024-04-21T08:36:23.473802+0000","last_became_peered":"2024-04-21T08:36:23.473802+0000","last_unstale":"2024-04-21T08:38:55.006322+0000","last_undegraded":"2024-04-21T08:38:55.006322+0000","last_fullsized":"2024-04-21T08:38:55.006322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:32:41.918431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":366,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005841+0000","last_change":"2024-04-21T08:35:10.062130+0000","last_active":"2024-04-21T08:38:55.005841+0000","last_peered":"2024-04-21T08:38:55.005841+0000","last_clean":"2024-04-21T08:38:55.005841+0000","last_became_active":"2024-04-21T08:35:10.061727+0000","last_became_peered":"2024-04-21T08:35:10.061727+0000","last_unstale":"2024-04-21T08:38:55.005841+0000","last_undegraded":"2024-04-21T08:38:55.005841+0000","last_fullsized":"2024-04-21T08:38:55.005841+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:31:40.950939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:31:40.950939+0000","last_clean_scrub_stamp":"2024-04-21T08:31:40.950939+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:35:52.360167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008148+0000","last_change":"2024-04-21T08:38:15.415007+0000","last_active":"2024-04-21T08:38:55.008148+0000","last_peered":"2024-04-21T08:38:55.008148+0000","last_clean":"2024-04-21T08:38:55.008148+0000","last_became_active":"2024-04-21T08:38:15.414478+0000","last_became_peered":"2024-04-21T08:38:15.414478+0000","last_unstale":"2024-04-21T08:38:55.008148+0000","last_undegraded":"2024-04-21T08:38:55.008148+0000","last_fullsized":"2024-04-21T08:38:55.008148+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:27.703964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.075651+0000","last_change":"2024-04-21T08:36:23.984332+0000","last_active":"2024-04-21T08:38:55.075651+0000","last_peered":"2024-04-21T08:38:55.075651+0000","last_clean":"2024-04-21T08:38:55.075651+0000","last_became_active":"2024-04-21T08:36:23.984021+0000","last_became_peered":"2024-04-21T08:36:23.984021+0000","last_unstale":"2024-04-21T08:38:55.075651+0000","last_undegraded":"2024-04-21T08:38:55.075651+0000","last_fullsized":"2024-04-21T08:38:55.075651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:05:21.090633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005972+0000","last_change":"2024-04-21T08:36:23.467522+0000","last_active":"2024-04-21T08:38:55.005972+0000","last_peered":"2024-04-21T08:38:55.005972+0000","last_clean":"2024-04-21T08:38:55.005972+0000","last_became_active":"2024-04-21T08:36:23.466379+0000","last_became_peered":"2024-04-21T08:36:23.466379+0000","last_unstale":"2024-04-21T08:38:55.005972+0000","last_undegraded":"2024-04-21T08:38:55.005972+0000","last_fullsized":"2024-04-21T08:38:55.005972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:04.185314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005193+0000","last_change":"2024-04-21T08:36:28.015120+0000","last_active":"2024-04-21T08:38:55.005193+0000","last_peered":"2024-04-21T08:38:55.005193+0000","last_clean":"2024-04-21T08:38:55.005193+0000","last_became_active":"2024-04-21T08:36:28.014701+0000","last_became_peered":"2024-04-21T08:36:28.014701+0000","last_unstale":"2024-04-21T08:38:55.005193+0000","last_undegraded":"2024-04-21T08:38:55.005193+0000","last_fullsized":"2024-04-21T08:38:55.005193+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:23.127622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890642+0000","last_change":"2024-04-21T08:36:23.475335+0000","last_active":"2024-04-21T08:38:54.890642+0000","last_peered":"2024-04-21T08:38:54.890642+0000","last_clean":"2024-04-21T08:38:54.890642+0000","last_became_active":"2024-04-21T08:36:23.474739+0000","last_became_peered":"2024-04-21T08:36:23.474739+0000","last_unstale":"2024-04-21T08:38:54.890642+0000","last_undegraded":"2024-04-21T08:38:54.890642+0000","last_fullsized":"2024-04-21T08:38:54.890642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:36.119649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077395+0000","last_change":"2024-04-21T08:36:23.977229+0000","last_active":"2024-04-21T08:38:55.077395+0000","last_peered":"2024-04-21T08:38:55.077395+0000","last_clean":"2024-04-21T08:38:55.077395+0000","last_became_active":"2024-04-21T08:36:23.976893+0000","last_became_peered":"2024-04-21T08:36:23.976893+0000","last_unstale":"2024-04-21T08:38:55.077395+0000","last_undegraded":"2024-04-21T08:38:55.077395+0000","last_fullsized":"2024-04-21T08:38:55.077395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:31:54.226274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886835+0000","last_change":"2024-04-21T08:36:28.018581+0000","last_active":"2024-04-21T08:38:54.886835+0000","last_peered":"2024-04-21T08:38:54.886835+0000","last_clean":"2024-04-21T08:38:54.886835+0000","last_became_active":"2024-04-21T08:36:28.018021+0000","last_became_peered":"2024-04-21T08:36:28.018021+0000","last_unstale":"2024-04-21T08:38:54.886835+0000","last_undegraded":"2024-04-21T08:38:54.886835+0000","last_fullsized":"2024-04-21T08:38:54.886835+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:29:00.355532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'48","reported_seq":105,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.124941+0000","last_change":"2024-04-21T08:36:23.473339+0000","last_active":"2024-04-21T08:38:55.124941+0000","last_peered":"2024-04-21T08:38:55.124941+0000","last_clean":"2024-04-21T08:38:55.124941+0000","last_became_active":"2024-04-21T08:36:23.472994+0000","last_became_peered":"2024-04-21T08:36:23.472994+0000","last_unstale":"2024-04-21T08:38:55.124941+0000","last_undegraded":"2024-04-21T08:38:55.124941+0000","last_fullsized":"2024-04-21T08:38:55.124941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:42.762273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887378+0000","last_change":"2024-04-21T08:36:23.987354+0000","last_active":"2024-04-21T08:38:54.887378+0000","last_peered":"2024-04-21T08:38:54.887378+0000","last_clean":"2024-04-21T08:38:54.887378+0000","last_became_active":"2024-04-21T08:36:23.987032+0000","last_became_peered":"2024-04-21T08:36:23.987032+0000","last_unstale":"2024-04-21T08:38:54.887378+0000","last_undegraded":"2024-04-21T08:38:54.887378+0000","last_fullsized":"2024-04-21T08:38:54.887378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:01:05.164367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104455+0000","last_change":"2024-04-21T08:36:28.002582+0000","last_active":"2024-04-21T08:38:55.104455+0000","last_peered":"2024-04-21T08:38:55.104455+0000","last_clean":"2024-04-21T08:38:55.104455+0000","last_became_active":"2024-04-21T08:36:28.002133+0000","last_became_peered":"2024-04-21T08:36:28.002133+0000","last_unstale":"2024-04-21T08:38:55.104455+0000","last_undegraded":"2024-04-21T08:38:55.104455+0000","last_fullsized":"2024-04-21T08:38:55.104455+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:30:00.268015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007058+0000","last_change":"2024-04-21T08:36:23.462840+0000","last_active":"2024-04-21T08:38:55.007058+0000","last_peered":"2024-04-21T08:38:55.007058+0000","last_clean":"2024-04-21T08:38:55.007058+0000","last_became_active":"2024-04-21T08:36:23.462391+0000","last_became_peered":"2024-04-21T08:36:23.462391+0000","last_unstale":"2024-04-21T08:38:55.007058+0000","last_undegraded":"2024-04-21T08:38:55.007058+0000","last_fullsized":"2024-04-21T08:38:55.007058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:33.230388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004380+0000","last_change":"2024-04-21T08:36:23.994751+0000","last_active":"2024-04-21T08:38:55.004380+0000","last_peered":"2024-04-21T08:38:55.004380+0000","last_clean":"2024-04-21T08:38:55.004380+0000","last_became_active":"2024-04-21T08:36:23.994384+0000","last_became_peered":"2024-04-21T08:36:23.994384+0000","last_unstale":"2024-04-21T08:38:55.004380+0000","last_undegraded":"2024-04-21T08:38:55.004380+0000","last_fullsized":"2024-04-21T08:38:55.004380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:18.956893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006981+0000","last_change":"2024-04-21T08:36:28.004004+0000","last_active":"2024-04-21T08:38:55.006981+0000","last_peered":"2024-04-21T08:38:55.006981+0000","last_clean":"2024-04-21T08:38:55.006981+0000","last_became_active":"2024-04-21T08:36:28.003547+0000","last_became_peered":"2024-04-21T08:36:28.003547+0000","last_unstale":"2024-04-21T08:38:55.006981+0000","last_undegraded":"2024-04-21T08:38:55.006981+0000","last_fullsized":"2024-04-21T08:38:55.006981+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:21:40.215428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104510+0000","last_change":"2024-04-21T08:37:15.222118+0000","last_active":"2024-04-21T08:38:55.104510+0000","last_peered":"2024-04-21T08:38:55.104510+0000","last_clean":"2024-04-21T08:38:55.104510+0000","last_became_active":"2024-04-21T08:37:15.219173+0000","last_became_peered":"2024-04-21T08:37:15.219173+0000","last_unstale":"2024-04-21T08:38:55.104510+0000","last_undegraded":"2024-04-21T08:38:55.104510+0000","last_fullsized":"2024-04-21T08:38:55.104510+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:12.931076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890299+0000","last_change":"2024-04-21T08:36:23.988020+0000","last_active":"2024-04-21T08:38:54.890299+0000","last_peered":"2024-04-21T08:38:54.890299+0000","last_clean":"2024-04-21T08:38:54.890299+0000","last_became_active":"2024-04-21T08:36:23.987630+0000","last_became_peered":"2024-04-21T08:36:23.987630+0000","last_unstale":"2024-04-21T08:38:54.890299+0000","last_undegraded":"2024-04-21T08:38:54.890299+0000","last_fullsized":"2024-04-21T08:38:54.890299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:06.519001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.997084+0000","last_change":"2024-04-21T08:36:28.006984+0000","last_active":"2024-04-21T08:38:54.997084+0000","last_peered":"2024-04-21T08:38:54.997084+0000","last_clean":"2024-04-21T08:38:54.997084+0000","last_became_active":"2024-04-21T08:36:28.006548+0000","last_became_peered":"2024-04-21T08:36:28.006548+0000","last_unstale":"2024-04-21T08:38:54.997084+0000","last_undegraded":"2024-04-21T08:38:54.997084+0000","last_fullsized":"2024-04-21T08:38:54.997084+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:25:50.870658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886907+0000","last_change":"2024-04-21T08:36:23.476266+0000","last_active":"2024-04-21T08:38:54.886907+0000","last_peered":"2024-04-21T08:38:54.886907+0000","last_clean":"2024-04-21T08:38:54.886907+0000","last_became_active":"2024-04-21T08:36:23.475948+0000","last_became_peered":"2024-04-21T08:36:23.475948+0000","last_unstale":"2024-04-21T08:38:54.886907+0000","last_undegraded":"2024-04-21T08:38:54.886907+0000","last_fullsized":"2024-04-21T08:38:54.886907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:41.529376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005217+0000","last_change":"2024-04-21T08:36:23.981942+0000","last_active":"2024-04-21T08:38:55.005217+0000","last_peered":"2024-04-21T08:38:55.005217+0000","last_clean":"2024-04-21T08:38:55.005217+0000","last_became_active":"2024-04-21T08:36:23.981643+0000","last_became_peered":"2024-04-21T08:36:23.981643+0000","last_unstale":"2024-04-21T08:38:55.005217+0000","last_undegraded":"2024-04-21T08:38:55.005217+0000","last_fullsized":"2024-04-21T08:38:55.005217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:26:24.722695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006307+0000","last_change":"2024-04-21T08:36:28.006980+0000","last_active":"2024-04-21T08:38:55.006307+0000","last_peered":"2024-04-21T08:38:55.006307+0000","last_clean":"2024-04-21T08:38:55.006307+0000","last_became_active":"2024-04-21T08:36:28.006530+0000","last_became_peered":"2024-04-21T08:36:28.006530+0000","last_unstale":"2024-04-21T08:38:55.006307+0000","last_undegraded":"2024-04-21T08:38:55.006307+0000","last_fullsized":"2024-04-21T08:38:55.006307+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:05:57.465612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104573+0000","last_change":"2024-04-21T08:36:23.473253+0000","last_active":"2024-04-21T08:38:55.104573+0000","last_peered":"2024-04-21T08:38:55.104573+0000","last_clean":"2024-04-21T08:38:55.104573+0000","last_became_active":"2024-04-21T08:36:23.472942+0000","last_became_peered":"2024-04-21T08:36:23.472942+0000","last_unstale":"2024-04-21T08:38:55.104573+0000","last_undegraded":"2024-04-21T08:38:55.104573+0000","last_fullsized":"2024-04-21T08:38:55.104573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:32:04.894388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887248+0000","last_change":"2024-04-21T08:36:23.988355+0000","last_active":"2024-04-21T08:38:54.887248+0000","last_peered":"2024-04-21T08:38:54.887248+0000","last_clean":"2024-04-21T08:38:54.887248+0000","last_became_active":"2024-04-21T08:36:23.988049+0000","last_became_peered":"2024-04-21T08:36:23.988049+0000","last_unstale":"2024-04-21T08:38:54.887248+0000","last_undegraded":"2024-04-21T08:38:54.887248+0000","last_fullsized":"2024-04-21T08:38:54.887248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:57.348698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006797+0000","last_change":"2024-04-21T08:36:28.007343+0000","last_active":"2024-04-21T08:38:55.006797+0000","last_peered":"2024-04-21T08:38:55.006797+0000","last_clean":"2024-04-21T08:38:55.006797+0000","last_became_active":"2024-04-21T08:36:28.006719+0000","last_became_peered":"2024-04-21T08:36:28.006719+0000","last_unstale":"2024-04-21T08:38:55.006797+0000","last_undegraded":"2024-04-21T08:38:55.006797+0000","last_fullsized":"2024-04-21T08:38:55.006797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:02:25.734584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890843+0000","last_change":"2024-04-21T08:36:23.472416+0000","last_active":"2024-04-21T08:38:54.890843+0000","last_peered":"2024-04-21T08:38:54.890843+0000","last_clean":"2024-04-21T08:38:54.890843+0000","last_became_active":"2024-04-21T08:36:23.471500+0000","last_became_peered":"2024-04-21T08:36:23.471500+0000","last_unstale":"2024-04-21T08:38:54.890843+0000","last_undegraded":"2024-04-21T08:38:54.890843+0000","last_fullsized":"2024-04-21T08:38:54.890843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:39.706432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006128+0000","last_change":"2024-04-21T08:36:23.990653+0000","last_active":"2024-04-21T08:38:55.006128+0000","last_peered":"2024-04-21T08:38:55.006128+0000","last_clean":"2024-04-21T08:38:55.006128+0000","last_became_active":"2024-04-21T08:36:23.990324+0000","last_became_peered":"2024-04-21T08:36:23.990324+0000","last_unstale":"2024-04-21T08:38:55.006128+0000","last_undegraded":"2024-04-21T08:38:55.006128+0000","last_fullsized":"2024-04-21T08:38:55.006128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:08:59.417850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104492+0000","last_change":"2024-04-21T08:36:28.007956+0000","last_active":"2024-04-21T08:38:55.104492+0000","last_peered":"2024-04-21T08:38:55.104492+0000","last_clean":"2024-04-21T08:38:55.104492+0000","last_became_active":"2024-04-21T08:36:28.007475+0000","last_became_peered":"2024-04-21T08:36:28.007475+0000","last_unstale":"2024-04-21T08:38:55.104492+0000","last_undegraded":"2024-04-21T08:38:55.104492+0000","last_fullsized":"2024-04-21T08:38:55.104492+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:54.039359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'8","reported_seq":170,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104857+0000","last_change":"2024-04-21T08:36:23.476887+0000","last_active":"2024-04-21T08:38:55.104857+0000","last_peered":"2024-04-21T08:38:55.104857+0000","last_clean":"2024-04-21T08:38:55.104857+0000","last_became_active":"2024-04-21T08:36:23.476574+0000","last_became_peered":"2024-04-21T08:36:23.476574+0000","last_unstale":"2024-04-21T08:38:55.104857+0000","last_undegraded":"2024-04-21T08:38:55.104857+0000","last_fullsized":"2024-04-21T08:38:55.104857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:37:47.692358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2010,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006973+0000","last_change":"2024-04-21T08:36:23.982845+0000","last_active":"2024-04-21T08:38:55.006973+0000","last_peered":"2024-04-21T08:38:55.006973+0000","last_clean":"2024-04-21T08:38:55.006973+0000","last_became_active":"2024-04-21T08:36:23.982491+0000","last_became_peered":"2024-04-21T08:36:23.982491+0000","last_unstale":"2024-04-21T08:38:55.006973+0000","last_undegraded":"2024-04-21T08:38:55.006973+0000","last_fullsized":"2024-04-21T08:38:55.006973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:48:46.413896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.002896+0000","last_change":"2024-04-21T08:36:28.010442+0000","last_active":"2024-04-21T08:38:55.002896+0000","last_peered":"2024-04-21T08:38:55.002896+0000","last_clean":"2024-04-21T08:38:55.002896+0000","last_became_active":"2024-04-21T08:36:28.010046+0000","last_became_peered":"2024-04-21T08:36:28.010046+0000","last_unstale":"2024-04-21T08:38:55.002896+0000","last_undegraded":"2024-04-21T08:38:55.002896+0000","last_fullsized":"2024-04-21T08:38:55.002896+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:18:36.382802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"95'41","reported_seq":83,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.122392+0000","last_change":"2024-04-21T08:36:23.473545+0000","last_active":"2024-04-21T08:38:55.122392+0000","last_peered":"2024-04-21T08:38:55.122392+0000","last_clean":"2024-04-21T08:38:55.122392+0000","last_became_active":"2024-04-21T08:36:23.473222+0000","last_became_peered":"2024-04-21T08:36:23.473222+0000","last_unstale":"2024-04-21T08:38:55.122392+0000","last_undegraded":"2024-04-21T08:38:55.122392+0000","last_fullsized":"2024-04-21T08:38:55.122392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:47:53.947187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006600+0000","last_change":"2024-04-21T08:36:23.990959+0000","last_active":"2024-04-21T08:38:55.006600+0000","last_peered":"2024-04-21T08:38:55.006600+0000","last_clean":"2024-04-21T08:38:55.006600+0000","last_became_active":"2024-04-21T08:36:23.990322+0000","last_became_peered":"2024-04-21T08:36:23.990322+0000","last_unstale":"2024-04-21T08:38:55.006600+0000","last_undegraded":"2024-04-21T08:38:55.006600+0000","last_fullsized":"2024-04-21T08:38:55.006600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:12:20.859224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007717+0000","last_change":"2024-04-21T08:36:28.006682+0000","last_active":"2024-04-21T08:38:55.007717+0000","last_peered":"2024-04-21T08:38:55.007717+0000","last_clean":"2024-04-21T08:38:55.007717+0000","last_became_active":"2024-04-21T08:36:28.006229+0000","last_became_peered":"2024-04-21T08:36:28.006229+0000","last_unstale":"2024-04-21T08:38:55.007717+0000","last_undegraded":"2024-04-21T08:38:55.007717+0000","last_fullsized":"2024-04-21T08:38:55.007717+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:30:12.723848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077921+0000","last_change":"2024-04-21T08:36:23.473592+0000","last_active":"2024-04-21T08:38:55.077921+0000","last_peered":"2024-04-21T08:38:55.077921+0000","last_clean":"2024-04-21T08:38:55.077921+0000","last_became_active":"2024-04-21T08:36:23.473283+0000","last_became_peered":"2024-04-21T08:36:23.473283+0000","last_unstale":"2024-04-21T08:38:55.077921+0000","last_undegraded":"2024-04-21T08:38:55.077921+0000","last_fullsized":"2024-04-21T08:38:55.077921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:10.425249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006895+0000","last_change":"2024-04-21T08:36:23.985052+0000","last_active":"2024-04-21T08:38:55.006895+0000","last_peered":"2024-04-21T08:38:55.006895+0000","last_clean":"2024-04-21T08:38:55.006895+0000","last_became_active":"2024-04-21T08:36:23.984742+0000","last_became_peered":"2024-04-21T08:36:23.984742+0000","last_unstale":"2024-04-21T08:38:55.006895+0000","last_undegraded":"2024-04-21T08:38:55.006895+0000","last_fullsized":"2024-04-21T08:38:55.006895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:08:06.082974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005201+0000","last_change":"2024-04-21T08:36:23.980905+0000","last_active":"2024-04-21T08:38:55.005201+0000","last_peered":"2024-04-21T08:38:55.005201+0000","last_clean":"2024-04-21T08:38:55.005201+0000","last_became_active":"2024-04-21T08:36:23.980590+0000","last_became_peered":"2024-04-21T08:36:23.980590+0000","last_unstale":"2024-04-21T08:38:55.005201+0000","last_undegraded":"2024-04-21T08:38:55.005201+0000","last_fullsized":"2024-04-21T08:38:55.005201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:54:59.886014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104725+0000","last_change":"2024-04-21T08:36:23.980883+0000","last_active":"2024-04-21T08:38:55.104725+0000","last_peered":"2024-04-21T08:38:55.104725+0000","last_clean":"2024-04-21T08:38:55.104725+0000","last_became_active":"2024-04-21T08:36:23.980562+0000","last_became_peered":"2024-04-21T08:36:23.980562+0000","last_unstale":"2024-04-21T08:38:55.104725+0000","last_undegraded":"2024-04-21T08:38:55.104725+0000","last_fullsized":"2024-04-21T08:38:55.104725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:14:51.868438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104692+0000","last_change":"2024-04-21T08:36:23.991571+0000","last_active":"2024-04-21T08:38:55.104692+0000","last_peered":"2024-04-21T08:38:55.104692+0000","last_clean":"2024-04-21T08:38:55.104692+0000","last_became_active":"2024-04-21T08:36:23.991273+0000","last_became_peered":"2024-04-21T08:36:23.991273+0000","last_unstale":"2024-04-21T08:38:55.104692+0000","last_undegraded":"2024-04-21T08:38:55.104692+0000","last_fullsized":"2024-04-21T08:38:55.104692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:08:23.895038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.003835+0000","last_change":"2024-04-21T08:36:23.994729+0000","last_active":"2024-04-21T08:38:55.003835+0000","last_peered":"2024-04-21T08:38:55.003835+0000","last_clean":"2024-04-21T08:38:55.003835+0000","last_became_active":"2024-04-21T08:36:23.994405+0000","last_became_peered":"2024-04-21T08:36:23.994405+0000","last_unstale":"2024-04-21T08:38:55.003835+0000","last_undegraded":"2024-04-21T08:38:55.003835+0000","last_fullsized":"2024-04-21T08:38:55.003835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:42.463225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077385+0000","last_change":"2024-04-21T08:36:23.995421+0000","last_active":"2024-04-21T08:38:55.077385+0000","last_peered":"2024-04-21T08:38:55.077385+0000","last_clean":"2024-04-21T08:38:55.077385+0000","last_became_active":"2024-04-21T08:36:23.993674+0000","last_became_peered":"2024-04-21T08:36:23.993674+0000","last_unstale":"2024-04-21T08:38:55.077385+0000","last_undegraded":"2024-04-21T08:38:55.077385+0000","last_fullsized":"2024-04-21T08:38:55.077385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:31.013874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005504+0000","last_change":"2024-04-21T08:36:23.986418+0000","last_active":"2024-04-21T08:38:55.005504+0000","last_peered":"2024-04-21T08:38:55.005504+0000","last_clean":"2024-04-21T08:38:55.005504+0000","last_became_active":"2024-04-21T08:36:23.986095+0000","last_became_peered":"2024-04-21T08:36:23.986095+0000","last_unstale":"2024-04-21T08:38:55.005504+0000","last_undegraded":"2024-04-21T08:38:55.005504+0000","last_fullsized":"2024-04-21T08:38:55.005504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:31:56.456218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104918+0000","last_change":"2024-04-21T08:36:23.982353+0000","last_active":"2024-04-21T08:38:55.104918+0000","last_peered":"2024-04-21T08:38:55.104918+0000","last_clean":"2024-04-21T08:38:55.104918+0000","last_became_active":"2024-04-21T08:36:23.982035+0000","last_became_peered":"2024-04-21T08:36:23.982035+0000","last_unstale":"2024-04-21T08:38:55.104918+0000","last_undegraded":"2024-04-21T08:38:55.104918+0000","last_fullsized":"2024-04-21T08:38:55.104918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:42:44.598407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887832+0000","last_change":"2024-04-21T08:36:23.996371+0000","last_active":"2024-04-21T08:38:54.887832+0000","last_peered":"2024-04-21T08:38:54.887832+0000","last_clean":"2024-04-21T08:38:54.887832+0000","last_became_active":"2024-04-21T08:36:23.995802+0000","last_became_peered":"2024-04-21T08:36:23.995802+0000","last_unstale":"2024-04-21T08:38:54.887832+0000","last_undegraded":"2024-04-21T08:38:54.887832+0000","last_fullsized":"2024-04-21T08:38:54.887832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:52:08.571567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007826+0000","last_change":"2024-04-21T08:36:23.987275+0000","last_active":"2024-04-21T08:38:55.007826+0000","last_peered":"2024-04-21T08:38:55.007826+0000","last_clean":"2024-04-21T08:38:55.007826+0000","last_became_active":"2024-04-21T08:36:23.986967+0000","last_became_peered":"2024-04-21T08:36:23.986967+0000","last_unstale":"2024-04-21T08:38:55.007826+0000","last_undegraded":"2024-04-21T08:38:55.007826+0000","last_fullsized":"2024-04-21T08:38:55.007826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:50.432502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886991+0000","last_change":"2024-04-21T08:36:23.993046+0000","last_active":"2024-04-21T08:38:54.886991+0000","last_peered":"2024-04-21T08:38:54.886991+0000","last_clean":"2024-04-21T08:38:54.886991+0000","last_became_active":"2024-04-21T08:36:23.992693+0000","last_became_peered":"2024-04-21T08:36:23.992693+0000","last_unstale":"2024-04-21T08:38:54.886991+0000","last_undegraded":"2024-04-21T08:38:54.886991+0000","last_fullsized":"2024-04-21T08:38:54.886991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:12:06.287585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104706+0000","last_change":"2024-04-21T08:36:23.992968+0000","last_active":"2024-04-21T08:38:55.104706+0000","last_peered":"2024-04-21T08:38:55.104706+0000","last_clean":"2024-04-21T08:38:55.104706+0000","last_became_active":"2024-04-21T08:36:23.992614+0000","last_became_peered":"2024-04-21T08:36:23.992614+0000","last_unstale":"2024-04-21T08:38:55.104706+0000","last_undegraded":"2024-04-21T08:38:55.104706+0000","last_fullsized":"2024-04-21T08:38:55.104706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:10.015323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076758+0000","last_change":"2024-04-21T08:36:23.981169+0000","last_active":"2024-04-21T08:38:55.076758+0000","last_peered":"2024-04-21T08:38:55.076758+0000","last_clean":"2024-04-21T08:38:55.076758+0000","last_became_active":"2024-04-21T08:36:23.980849+0000","last_became_peered":"2024-04-21T08:36:23.980849+0000","last_unstale":"2024-04-21T08:38:55.076758+0000","last_undegraded":"2024-04-21T08:38:55.076758+0000","last_fullsized":"2024-04-21T08:38:55.076758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:13:49.907370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005358+0000","last_change":"2024-04-21T08:36:23.993998+0000","last_active":"2024-04-21T08:38:55.005358+0000","last_peered":"2024-04-21T08:38:55.005358+0000","last_clean":"2024-04-21T08:38:55.005358+0000","last_became_active":"2024-04-21T08:36:23.993664+0000","last_became_peered":"2024-04-21T08:36:23.993664+0000","last_unstale":"2024-04-21T08:38:55.005358+0000","last_undegraded":"2024-04-21T08:38:55.005358+0000","last_fullsized":"2024-04-21T08:38:55.005358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:31:49.076820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":604383,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1392,"num_read_kb":816,"num_write":377,"num_write_kb":1476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1813149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351019,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1093632,"data_stored":535779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612657,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37220,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707932,"statfs":{"total":95995035648,"available":95956922368,"internally_reserved":0,"allocated":1929216,"data_stored":1324346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776182,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36820,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708332,"statfs":{"total":95995035648,"available":95957331968,"internally_reserved":0,"allocated":1515520,"data_stored":932181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037819,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28496,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716656,"statfs":{"total":95995035648,"available":95965855744,"internally_reserved":0,"allocated":1511424,"data_stored":932559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299455,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1712128,"data_stored":1133590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462980,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1093632,"data_stored":530764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1167360,"data_stored":601862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1118208,"data_stored":532669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1187840,"data_stored":603454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1054897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920847,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":535370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724619,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32984,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712168,"statfs":{"total":95995035648,"available":95961260032,"internally_reserved":0,"allocated":1851392,"data_stored":1255362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":204327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:38:58.124 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:38:58.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.264+0000 7f98c4681640 1 -- 172.21.15.43:0/2166690180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98bc101fc0 msgr2=0x7f98bc102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.264+0000 7f98c4681640 1 --2- 172.21.15.43:0/2166690180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98bc101fc0 0x7f98bc102440 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f98b8009900 tx=0x7f98b802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.264+0000 7f98c4681640 1 -- 172.21.15.43:0/2166690180 shutdown_connections 2024-04-21T08:38:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.264+0000 7f98c4681640 1 --2- 172.21.15.43:0/2166690180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98bc102980 0x7f98bc1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.264+0000 7f98c4681640 1 --2- 172.21.15.43:0/2166690180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98bc101fc0 0x7f98bc102440 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.264+0000 7f98c4681640 1 --2- 172.21.15.43:0/2166690180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98bc100e70 0x7f98bc101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.264+0000 7f98c4681640 1 -- 172.21.15.43:0/2166690180 >> 172.21.15.43:0/2166690180 conn(0x7f98bc0fc3e0 msgr2=0x7f98bc0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.264+0000 7f98c4681640 1 -- 172.21.15.43:0/2166690180 shutdown_connections 2024-04-21T08:38:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.264+0000 7f98c4681640 1 -- 172.21.15.43:0/2166690180 wait complete. 2024-04-21T08:38:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.265+0000 7f98c4681640 1 Processor -- start 2024-04-21T08:38:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.265+0000 7f98c4681640 1 -- start start 2024-04-21T08:38:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.265+0000 7f98c4681640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98bc100e70 0x7f98bc19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.265+0000 7f98c4681640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98bc101fc0 0x7f98bc19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.265+0000 7f98c4681640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98bc102980 0x7f98bc1a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.265+0000 7f98c4681640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc10c9c0 con 0x7f98bc102980 2024-04-21T08:38:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.265+0000 7f98c4681640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc10c840 con 0x7f98bc100e70 2024-04-21T08:38:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.265+0000 7f98c4681640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc10cb40 con 0x7f98bc101fc0 2024-04-21T08:38:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.265+0000 7f98c2c1e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98bc102980 0x7f98bc1a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.265+0000 7f98c241d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98bc100e70 0x7f98bc19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.265+0000 7f98c2c1e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98bc102980 0x7f98bc1a1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38600/0 (socket says 172.21.15.43:38600) 2024-04-21T08:38:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.265+0000 7f98c2c1e640 1 -- 172.21.15.43:0/3859683110 learned_addr learned my addr 172.21.15.43:0/3859683110 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.265+0000 7f98c1c1c640 1 --2- 172.21.15.43:0/3859683110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98bc101fc0 0x7f98bc19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.265+0000 7f98c2c1e640 1 -- 172.21.15.43:0/3859683110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98bc101fc0 msgr2=0x7f98bc19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.265+0000 7f98c2c1e640 1 --2- 172.21.15.43:0/3859683110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98bc101fc0 0x7f98bc19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.265+0000 7f98c2c1e640 1 -- 172.21.15.43:0/3859683110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98bc100e70 msgr2=0x7f98bc19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.265+0000 7f98c2c1e640 1 --2- 172.21.15.43:0/3859683110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98bc100e70 0x7f98bc19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.265+0000 7f98c2c1e640 1 -- 172.21.15.43:0/3859683110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98bc1a20a0 con 0x7f98bc102980 2024-04-21T08:38:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.265+0000 7f98c241d640 1 --2- 172.21.15.43:0/3859683110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98bc100e70 0x7f98bc19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:38:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.266+0000 7f98c2c1e640 1 --2- 172.21.15.43:0/3859683110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98bc102980 0x7f98bc1a1a30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f98b400eaa0 tx=0x7f98b400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.266+0000 7f98b37fe640 1 -- 172.21.15.43:0/3859683110 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b400cdb0 con 0x7f98bc102980 2024-04-21T08:38:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.266+0000 7f98c4681640 1 -- 172.21.15.43:0/3859683110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98bc1a2390 con 0x7f98bc102980 2024-04-21T08:38:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.266+0000 7f98b37fe640 1 -- 172.21.15.43:0/3859683110 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98b4005020 con 0x7f98bc102980 2024-04-21T08:38:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.266+0000 7f98c4681640 1 -- 172.21.15.43:0/3859683110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98bc1a28b0 con 0x7f98bc102980 2024-04-21T08:38:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.266+0000 7f98b37fe640 1 -- 172.21.15.43:0/3859683110 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b401f060 con 0x7f98bc102980 2024-04-21T08:38:58.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.267+0000 7f98c4681640 1 -- 172.21.15.43:0/3859683110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98bc1a2b70 con 0x7f98bc102980 2024-04-21T08:38:58.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.269+0000 7f98b37fe640 1 -- 172.21.15.43:0/3859683110 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f98b40051d0 con 0x7f98bc102980 2024-04-21T08:38:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.269+0000 7f98b37fe640 1 --2- 172.21.15.43:0/3859683110 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f98a80a3d20 0x7f98a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.270+0000 7f98c241d640 1 --2- 172.21.15.43:0/3859683110 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f98a80a3d20 0x7f98a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.270+0000 7f98b37fe640 1 -- 172.21.15.43:0/3859683110 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f98b40d11d0 con 0x7f98bc102980 2024-04-21T08:38:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.270+0000 7f98c241d640 1 --2- 172.21.15.43:0/3859683110 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f98a80a3d20 0x7f98a80a61e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009fc0 tx=0x7f98ac005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:58.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.273+0000 7f98b37fe640 1 -- 172.21.15.43:0/3859683110 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f98b4016030 con 0x7f98bc102980 2024-04-21T08:38:58.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.467+0000 7f98c4681640 1 -- 172.21.15.43:0/3859683110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f98bc1a2d40 con 0x7f98bc102980 2024-04-21T08:38:58.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.469+0000 7f98b37fe640 1 -- 172.21.15.43:0/3859683110 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+25233 (secure 0 0 0) 0x7f98b409d440 con 0x7f98bc102980 2024-04-21T08:38:58.468 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:38:58.468 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":95,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:38:54.881943+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/126475132":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/4288951239":"2024-04-22T08:38:54.881882+0000","172.21.15.43:6801/1577030772":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/2382670190":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/1174411284":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/1577030772":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:38:58.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.471+0000 7f98c4681640 1 -- 172.21.15.43:0/3859683110 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f98a80a3d20 msgr2=0x7f98a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.471+0000 7f98c4681640 1 --2- 172.21.15.43:0/3859683110 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f98a80a3d20 0x7f98a80a61e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009fc0 tx=0x7f98ac005f90 comp rx=0 tx=0).stop 2024-04-21T08:38:58.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.471+0000 7f98c4681640 1 -- 172.21.15.43:0/3859683110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98bc102980 msgr2=0x7f98bc1a1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.471+0000 7f98c4681640 1 --2- 172.21.15.43:0/3859683110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98bc102980 0x7f98bc1a1a30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f98b400eaa0 tx=0x7f98b400ef60 comp rx=0 tx=0).stop 2024-04-21T08:38:58.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.471+0000 7f98c4681640 1 -- 172.21.15.43:0/3859683110 shutdown_connections 2024-04-21T08:38:58.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.471+0000 7f98c4681640 1 --2- 172.21.15.43:0/3859683110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98bc102980 0x7f98bc1a1a30 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.471+0000 7f98c4681640 1 --2- 172.21.15.43:0/3859683110 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f98a80a3d20 0x7f98a80a61e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.471+0000 7f98c4681640 1 --2- 172.21.15.43:0/3859683110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98bc101fc0 0x7f98bc19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.471+0000 7f98c4681640 1 --2- 172.21.15.43:0/3859683110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98bc100e70 0x7f98bc19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.471+0000 7f98c4681640 1 -- 172.21.15.43:0/3859683110 >> 172.21.15.43:0/3859683110 conn(0x7f98bc0fc3e0 msgr2=0x7f98bc0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:58.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.471+0000 7f98c4681640 1 -- 172.21.15.43:0/3859683110 shutdown_connections 2024-04-21T08:38:58.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.471+0000 7f98c4681640 1 -- 172.21.15.43:0/3859683110 wait complete. 2024-04-21T08:38:58.480 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:58 smithi043 ceph-mon[27978]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:38:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:58 smithi043 ceph-mon[27978]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-21T08:38:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:58 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:58 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:58 smithi043 ceph-mon[27978]: from='client.25286 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:58 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:58 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:58 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:58 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:58 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:58 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:58 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:58 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:58 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:58.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:58 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:58.482 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:58 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:58.482 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:58 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:58.483 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:39.981405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:22.034751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:22.780795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:25.984261+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:38:58.483 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:38:58.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:58 smithi112 ceph-mon[31085]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:38:58.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:58 smithi112 ceph-mon[31085]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-21T08:38:58.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:58 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:58.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:58 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:58.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:58 smithi112 ceph-mon[31085]: from='client.25286 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:58.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:58 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:58.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:58 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:58.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:58 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:58.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:58 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:58.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:58 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:58.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:58 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:58.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:58 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:58.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:58 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:58.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:58 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:58.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:58 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:58.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:58 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:58.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:58 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.594+0000 7f825217e640 1 -- 172.21.15.43:0/1532333076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f824c1029a0 msgr2=0x7f824c1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.594+0000 7f825217e640 1 --2- 172.21.15.43:0/1532333076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f824c1029a0 0x7f824c1092c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8244009ca0 tx=0x7f824402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.594+0000 7f825217e640 1 -- 172.21.15.43:0/1532333076 shutdown_connections 2024-04-21T08:38:58.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.594+0000 7f825217e640 1 --2- 172.21.15.43:0/1532333076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f824c1029a0 0x7f824c1092c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.594+0000 7f825217e640 1 --2- 172.21.15.43:0/1532333076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f824c101fe0 0x7f824c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.594+0000 7f825217e640 1 --2- 172.21.15.43:0/1532333076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f824c100e90 0x7f824c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.594+0000 7f825217e640 1 -- 172.21.15.43:0/1532333076 >> 172.21.15.43:0/1532333076 conn(0x7f824c0fc400 msgr2=0x7f824c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:58.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.594+0000 7f825217e640 1 -- 172.21.15.43:0/1532333076 shutdown_connections 2024-04-21T08:38:58.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.594+0000 7f825217e640 1 -- 172.21.15.43:0/1532333076 wait complete. 2024-04-21T08:38:58.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.594+0000 7f825217e640 1 Processor -- start 2024-04-21T08:38:58.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.595+0000 7f825217e640 1 -- start start 2024-04-21T08:38:58.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.595+0000 7f825217e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f824c100e90 0x7f824c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:58.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.595+0000 7f825217e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f824c101fe0 0x7f824c19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:58.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.595+0000 7f825217e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f824c1029a0 0x7f824c1a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:58.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.595+0000 7f825217e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f824c10db30 con 0x7f824c100e90 2024-04-21T08:38:58.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.595+0000 7f825217e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f824c10d9b0 con 0x7f824c1029a0 2024-04-21T08:38:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.595+0000 7f825217e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f824c10dcb0 con 0x7f824c101fe0 2024-04-21T08:38:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.595+0000 7f824b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f824c100e90 0x7f824c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.595+0000 7f824b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f824c100e90 0x7f824c19a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38610/0 (socket says 172.21.15.43:38610) 2024-04-21T08:38:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.595+0000 7f824b7fe640 1 -- 172.21.15.43:0/2557471155 learned_addr learned my addr 172.21.15.43:0/2557471155 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.595+0000 7f824affd640 1 --2- 172.21.15.43:0/2557471155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f824c101fe0 0x7f824c19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.595+0000 7f824bfff640 1 --2- 172.21.15.43:0/2557471155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f824c1029a0 0x7f824c1a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.595+0000 7f824b7fe640 1 -- 172.21.15.43:0/2557471155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f824c101fe0 msgr2=0x7f824c19a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.595+0000 7f824b7fe640 1 --2- 172.21.15.43:0/2557471155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f824c101fe0 0x7f824c19a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.595+0000 7f824b7fe640 1 -- 172.21.15.43:0/2557471155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f824c1029a0 msgr2=0x7f824c1a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.595+0000 7f824b7fe640 1 --2- 172.21.15.43:0/2557471155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f824c1029a0 0x7f824c1a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.595+0000 7f824b7fe640 1 -- 172.21.15.43:0/2557471155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824c1a2000 con 0x7f824c100e90 2024-04-21T08:38:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.595+0000 7f824b7fe640 1 --2- 172.21.15.43:0/2557471155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f824c100e90 0x7f824c19a3e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f823c00e9f0 tx=0x7f823c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.596+0000 7f8248ff9640 1 -- 172.21.15.43:0/2557471155 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c00cd80 con 0x7f824c100e90 2024-04-21T08:38:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.596+0000 7f825217e640 1 -- 172.21.15.43:0/2557471155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824c1a22f0 con 0x7f824c100e90 2024-04-21T08:38:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.596+0000 7f8248ff9640 1 -- 172.21.15.43:0/2557471155 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f823c0054d0 con 0x7f824c100e90 2024-04-21T08:38:58.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.596+0000 7f8248ff9640 1 -- 172.21.15.43:0/2557471155 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c010690 con 0x7f824c100e90 2024-04-21T08:38:58.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.596+0000 7f825217e640 1 -- 172.21.15.43:0/2557471155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f824c1a2840 con 0x7f824c100e90 2024-04-21T08:38:58.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.597+0000 7f825217e640 1 -- 172.21.15.43:0/2557471155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f824c1a2b00 con 0x7f824c100e90 2024-04-21T08:38:58.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.598+0000 7f8248ff9640 1 -- 172.21.15.43:0/2557471155 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f823c004e90 con 0x7f824c100e90 2024-04-21T08:38:58.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.599+0000 7f8248ff9640 1 --2- 172.21.15.43:0/2557471155 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f82340a3d20 0x7f82340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:58.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.599+0000 7f824affd640 1 --2- 172.21.15.43:0/2557471155 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f82340a3d20 0x7f82340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:58.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.599+0000 7f8248ff9640 1 -- 172.21.15.43:0/2557471155 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f823c0d0fc0 con 0x7f824c100e90 2024-04-21T08:38:58.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.599+0000 7f824affd640 1 --2- 172.21.15.43:0/2557471155 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f82340a3d20 0x7f82340a61e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f824c19b8f0 tx=0x7f8240005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:58.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.604+0000 7f8248ff9640 1 -- 172.21.15.43:0/2557471155 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f823c09d290 con 0x7f824c100e90 2024-04-21T08:38:58.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:58 smithi175 ceph-mon[31920]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:38:58.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:58 smithi175 ceph-mon[31920]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-21T08:38:58.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:58 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:58.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:58 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:58.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:58 smithi175 ceph-mon[31920]: from='client.25286 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:58.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:58 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:58.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:58 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:58.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:58 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:58.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:58 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:58.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:58 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:58.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:58 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:58.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:58 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:58.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:58 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:58.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:58 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:58.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:58 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:58.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:58 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:58.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:58 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:58.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.793+0000 7f825217e640 1 -- 172.21.15.43:0/2557471155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f824c1a2cd0 con 0x7f824c100e90 2024-04-21T08:38:58.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.793+0000 7f8248ff9640 1 -- 172.21.15.43:0/2557471155 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f823c09cb90 con 0x7f824c100e90 2024-04-21T08:38:58.792 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-21T08:38:58.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.795+0000 7f825217e640 1 -- 172.21.15.43:0/2557471155 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f82340a3d20 msgr2=0x7f82340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.795+0000 7f825217e640 1 --2- 172.21.15.43:0/2557471155 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f82340a3d20 0x7f82340a61e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f824c19b8f0 tx=0x7f8240005f90 comp rx=0 tx=0).stop 2024-04-21T08:38:58.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.795+0000 7f825217e640 1 -- 172.21.15.43:0/2557471155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f824c100e90 msgr2=0x7f824c19a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.795+0000 7f825217e640 1 --2- 172.21.15.43:0/2557471155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f824c100e90 0x7f824c19a3e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f823c00e9f0 tx=0x7f823c00edf0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.796+0000 7f825217e640 1 -- 172.21.15.43:0/2557471155 shutdown_connections 2024-04-21T08:38:58.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.796+0000 7f825217e640 1 --2- 172.21.15.43:0/2557471155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f824c1029a0 0x7f824c1a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.796+0000 7f825217e640 1 --2- 172.21.15.43:0/2557471155 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f82340a3d20 0x7f82340a61e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.796+0000 7f825217e640 1 --2- 172.21.15.43:0/2557471155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f824c101fe0 0x7f824c19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.796+0000 7f825217e640 1 --2- 172.21.15.43:0/2557471155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f824c100e90 0x7f824c19a3e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.796+0000 7f825217e640 1 -- 172.21.15.43:0/2557471155 >> 172.21.15.43:0/2557471155 conn(0x7f824c0fc400 msgr2=0x7f824c109cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:58.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.796+0000 7f825217e640 1 -- 172.21.15.43:0/2557471155 shutdown_connections 2024-04-21T08:38:58.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.796+0000 7f825217e640 1 -- 172.21.15.43:0/2557471155 wait complete. 2024-04-21T08:38:58.807 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:38:58.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.917+0000 7fdd2dbb1640 1 -- 172.21.15.43:0/2394293677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd28100e90 msgr2=0x7fdd281012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.917+0000 7fdd2dbb1640 1 --2- 172.21.15.43:0/2394293677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd28100e90 0x7fdd281012b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdd18009900 tx=0x7fdd1802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:58.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.917+0000 7fdd2dbb1640 1 -- 172.21.15.43:0/2394293677 shutdown_connections 2024-04-21T08:38:58.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.917+0000 7fdd2dbb1640 1 --2- 172.21.15.43:0/2394293677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd281029a0 0x7fdd281092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.917+0000 7fdd2dbb1640 1 --2- 172.21.15.43:0/2394293677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd28101fe0 0x7fdd28102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.917+0000 7fdd2dbb1640 1 --2- 172.21.15.43:0/2394293677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd28100e90 0x7fdd281012b0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.917+0000 7fdd2dbb1640 1 -- 172.21.15.43:0/2394293677 >> 172.21.15.43:0/2394293677 conn(0x7fdd280fc440 msgr2=0x7fdd280fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:58.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.917+0000 7fdd2dbb1640 1 -- 172.21.15.43:0/2394293677 shutdown_connections 2024-04-21T08:38:58.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.917+0000 7fdd2dbb1640 1 -- 172.21.15.43:0/2394293677 wait complete. 2024-04-21T08:38:58.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.918+0000 7fdd2dbb1640 1 Processor -- start 2024-04-21T08:38:58.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.918+0000 7fdd2dbb1640 1 -- start start 2024-04-21T08:38:58.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.918+0000 7fdd2dbb1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd28100e90 0x7fdd281981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:58.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.918+0000 7fdd2dbb1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd28101fe0 0x7fdd28198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:58.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.918+0000 7fdd2dbb1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd281029a0 0x7fdd2819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:58.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.918+0000 7fdd2dbb1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2810c9e0 con 0x7fdd28100e90 2024-04-21T08:38:58.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.918+0000 7fdd2dbb1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2810c860 con 0x7fdd281029a0 2024-04-21T08:38:58.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.918+0000 7fdd2dbb1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2810cb60 con 0x7fdd28101fe0 2024-04-21T08:38:58.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.918+0000 7fdd277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd28100e90 0x7fdd281981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:58.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.918+0000 7fdd277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd28100e90 0x7fdd281981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38626/0 (socket says 172.21.15.43:38626) 2024-04-21T08:38:58.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.918+0000 7fdd277fe640 1 -- 172.21.15.43:0/3547881760 learned_addr learned my addr 172.21.15.43:0/3547881760 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:58.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.918+0000 7fdd26ffd640 1 --2- 172.21.15.43:0/3547881760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd28101fe0 0x7fdd28198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:58.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.918+0000 7fdd27fff640 1 --2- 172.21.15.43:0/3547881760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd281029a0 0x7fdd2819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:58.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.919+0000 7fdd27fff640 1 -- 172.21.15.43:0/3547881760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd28101fe0 msgr2=0x7fdd28198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.919+0000 7fdd27fff640 1 --2- 172.21.15.43:0/3547881760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd28101fe0 0x7fdd28198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.919+0000 7fdd27fff640 1 -- 172.21.15.43:0/3547881760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd28100e90 msgr2=0x7fdd281981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.919+0000 7fdd27fff640 1 --2- 172.21.15.43:0/3547881760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd28100e90 0x7fdd281981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.919+0000 7fdd27fff640 1 -- 172.21.15.43:0/3547881760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd2819fde0 con 0x7fdd281029a0 2024-04-21T08:38:58.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.919+0000 7fdd277fe640 1 --2- 172.21.15.43:0/3547881760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd28100e90 0x7fdd281981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:38:58.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.919+0000 7fdd27fff640 1 --2- 172.21.15.43:0/3547881760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd281029a0 0x7fdd2819f770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009fd0 tx=0x7fdd2000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:58.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.919+0000 7fdd24ff9640 1 -- 172.21.15.43:0/3547881760 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2000ce20 con 0x7fdd281029a0 2024-04-21T08:38:58.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.919+0000 7fdd24ff9640 1 -- 172.21.15.43:0/3547881760 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd20005020 con 0x7fdd281029a0 2024-04-21T08:38:58.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.919+0000 7fdd2dbb1640 1 -- 172.21.15.43:0/3547881760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd281a00d0 con 0x7fdd281029a0 2024-04-21T08:38:58.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.919+0000 7fdd24ff9640 1 -- 172.21.15.43:0/3547881760 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20010690 con 0x7fdd281029a0 2024-04-21T08:38:58.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.919+0000 7fdd2dbb1640 1 -- 172.21.15.43:0/3547881760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd281a0620 con 0x7fdd281029a0 2024-04-21T08:38:58.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.920+0000 7fdd2dbb1640 1 -- 172.21.15.43:0/3547881760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd281a08e0 con 0x7fdd281029a0 2024-04-21T08:38:58.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.921+0000 7fdd24ff9640 1 -- 172.21.15.43:0/3547881760 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7fdd20004b20 con 0x7fdd281029a0 2024-04-21T08:38:58.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.922+0000 7fdd24ff9640 1 --2- 172.21.15.43:0/3547881760 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdd100a3d20 0x7fdd100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:58.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.922+0000 7fdd277fe640 1 --2- 172.21.15.43:0/3547881760 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdd100a3d20 0x7fdd100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:58.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.922+0000 7fdd24ff9640 1 -- 172.21.15.43:0/3547881760 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fdd200d1250 con 0x7fdd281029a0 2024-04-21T08:38:58.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.923+0000 7fdd277fe640 1 --2- 172.21.15.43:0/3547881760 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdd100a3d20 0x7fdd100a61e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdd180096c0 tx=0x7fdd1802c7e0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:58.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:58.927+0000 7fdd24ff9640 1 -- 172.21.15.43:0/3547881760 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd2009d520 con 0x7fdd281029a0 2024-04-21T08:38:59.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.117+0000 7fdd2dbb1640 1 -- 172.21.15.43:0/3547881760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdd281a0ab0 con 0x7fdd281029a0 2024-04-21T08:38:59.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.118+0000 7fdd24ff9640 1 -- 172.21.15.43:0/3547881760 <== mon.2 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7fdd2009ce20 con 0x7fdd281029a0 2024-04-21T08:38:59.117 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:38:59.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.120+0000 7fdd2dbb1640 1 -- 172.21.15.43:0/3547881760 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdd100a3d20 msgr2=0x7fdd100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.120+0000 7fdd2dbb1640 1 --2- 172.21.15.43:0/3547881760 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdd100a3d20 0x7fdd100a61e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdd180096c0 tx=0x7fdd1802c7e0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.120+0000 7fdd2dbb1640 1 -- 172.21.15.43:0/3547881760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd281029a0 msgr2=0x7fdd2819f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.120+0000 7fdd2dbb1640 1 --2- 172.21.15.43:0/3547881760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd281029a0 0x7fdd2819f770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009fd0 tx=0x7fdd2000ef10 comp rx=0 tx=0).stop 2024-04-21T08:38:59.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.120+0000 7fdd2dbb1640 1 -- 172.21.15.43:0/3547881760 shutdown_connections 2024-04-21T08:38:59.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.120+0000 7fdd2dbb1640 1 --2- 172.21.15.43:0/3547881760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd281029a0 0x7fdd2819f770 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.120+0000 7fdd2dbb1640 1 --2- 172.21.15.43:0/3547881760 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdd100a3d20 0x7fdd100a61e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.120+0000 7fdd2dbb1640 1 --2- 172.21.15.43:0/3547881760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd28101fe0 0x7fdd28198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.120+0000 7fdd2dbb1640 1 --2- 172.21.15.43:0/3547881760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd28100e90 0x7fdd281981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.120+0000 7fdd2dbb1640 1 -- 172.21.15.43:0/3547881760 >> 172.21.15.43:0/3547881760 conn(0x7fdd280fc440 msgr2=0x7fdd280fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:59.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.120+0000 7fdd2dbb1640 1 -- 172.21.15.43:0/3547881760 shutdown_connections 2024-04-21T08:38:59.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.120+0000 7fdd2dbb1640 1 -- 172.21.15.43:0/3547881760 wait complete. 2024-04-21T08:38:59.132 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:38:59.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.246+0000 7fdf50d15640 1 -- 172.21.15.43:0/1406402641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf4c106160 msgr2=0x7fdf4c10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.246+0000 7fdf50d15640 1 --2- 172.21.15.43:0/1406402641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf4c106160 0x7fdf4c10dd30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdf44009ca0 tx=0x7fdf4402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.246+0000 7fdf50d15640 1 -- 172.21.15.43:0/1406402641 shutdown_connections 2024-04-21T08:38:59.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.246+0000 7fdf50d15640 1 --2- 172.21.15.43:0/1406402641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf4c106160 0x7fdf4c10dd30 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.246+0000 7fdf50d15640 1 --2- 172.21.15.43:0/1406402641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf4c101a40 0x7fdf4c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.246+0000 7fdf50d15640 1 --2- 172.21.15.43:0/1406402641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf4c101090 0x7fdf4c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.246+0000 7fdf50d15640 1 -- 172.21.15.43:0/1406402641 >> 172.21.15.43:0/1406402641 conn(0x7fdf4c077f50 msgr2=0x7fdf4c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:59.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.246+0000 7fdf50d15640 1 -- 172.21.15.43:0/1406402641 shutdown_connections 2024-04-21T08:38:59.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.246+0000 7fdf50d15640 1 -- 172.21.15.43:0/1406402641 wait complete. 2024-04-21T08:38:59.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.247+0000 7fdf50d15640 1 Processor -- start 2024-04-21T08:38:59.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.247+0000 7fdf50d15640 1 -- start start 2024-04-21T08:38:59.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.247+0000 7fdf50d15640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf4c101090 0x7fdf4c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.247+0000 7fdf50d15640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf4c106160 0x7fdf4c198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.247+0000 7fdf50d15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf4c19cfd0 0x7fdf4c19d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.247+0000 7fdf50d15640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c1110d0 con 0x7fdf4c101090 2024-04-21T08:38:59.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.247+0000 7fdf50d15640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c110f50 con 0x7fdf4c19cfd0 2024-04-21T08:38:59.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.247+0000 7fdf50d15640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c111250 con 0x7fdf4c106160 2024-04-21T08:38:59.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.248+0000 7fdf4a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf4c101090 0x7fdf4c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.248+0000 7fdf4a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf4c101090 0x7fdf4c198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38652/0 (socket says 172.21.15.43:38652) 2024-04-21T08:38:59.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.248+0000 7fdf4a59c640 1 -- 172.21.15.43:0/381264264 learned_addr learned my addr 172.21.15.43:0/381264264 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:59.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.248+0000 7fdf49d9b640 1 --2- 172.21.15.43:0/381264264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf4c106160 0x7fdf4c198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.248+0000 7fdf4ad9d640 1 --2- 172.21.15.43:0/381264264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf4c19cfd0 0x7fdf4c19d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.248+0000 7fdf4a59c640 1 -- 172.21.15.43:0/381264264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf4c106160 msgr2=0x7fdf4c198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.248+0000 7fdf4a59c640 1 --2- 172.21.15.43:0/381264264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf4c106160 0x7fdf4c198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.248+0000 7fdf4a59c640 1 -- 172.21.15.43:0/381264264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf4c19cfd0 msgr2=0x7fdf4c19d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.248+0000 7fdf4a59c640 1 --2- 172.21.15.43:0/381264264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf4c19cfd0 0x7fdf4c19d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.248+0000 7fdf4a59c640 1 -- 172.21.15.43:0/381264264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4c19dac0 con 0x7fdf4c101090 2024-04-21T08:38:59.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.248+0000 7fdf4a59c640 1 --2- 172.21.15.43:0/381264264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf4c101090 0x7fdf4c198710 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdf3400e9f0 tx=0x7fdf3400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:59.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.248+0000 7fdf437fe640 1 -- 172.21.15.43:0/381264264 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3400cd80 con 0x7fdf4c101090 2024-04-21T08:38:59.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.248+0000 7fdf50d15640 1 -- 172.21.15.43:0/381264264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf4c19ddb0 con 0x7fdf4c101090 2024-04-21T08:38:59.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.248+0000 7fdf437fe640 1 -- 172.21.15.43:0/381264264 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf340054d0 con 0x7fdf4c101090 2024-04-21T08:38:59.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.248+0000 7fdf437fe640 1 -- 172.21.15.43:0/381264264 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf34010690 con 0x7fdf4c101090 2024-04-21T08:38:59.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.248+0000 7fdf50d15640 1 -- 172.21.15.43:0/381264264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf4c1a5550 con 0x7fdf4c101090 2024-04-21T08:38:59.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.249+0000 7fdf50d15640 1 -- 172.21.15.43:0/381264264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf4c068d70 con 0x7fdf4c101090 2024-04-21T08:38:59.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.250+0000 7fdf437fe640 1 -- 172.21.15.43:0/381264264 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7fdf34004e90 con 0x7fdf4c101090 2024-04-21T08:38:59.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.251+0000 7fdf437fe640 1 --2- 172.21.15.43:0/381264264 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdf380a3d20 0x7fdf380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.251+0000 7fdf437fe640 1 -- 172.21.15.43:0/381264264 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fdf340d1070 con 0x7fdf4c101090 2024-04-21T08:38:59.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.251+0000 7fdf49d9b640 1 --2- 172.21.15.43:0/381264264 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdf380a3d20 0x7fdf380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.252+0000 7fdf49d9b640 1 --2- 172.21.15.43:0/381264264 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdf380a3d20 0x7fdf380a61e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c0095f0 tx=0x7fdf3c005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:59.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.256+0000 7fdf437fe640 1 -- 172.21.15.43:0/381264264 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf3409d340 con 0x7fdf4c101090 2024-04-21T08:38:59.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.454+0000 7fdf50d15640 1 -- 172.21.15.43:0/381264264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdf4c101a40 con 0x7fdf4c101090 2024-04-21T08:38:59.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.458+0000 7fdf437fe640 1 -- 172.21.15.43:0/381264264 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7fdf3409cc40 con 0x7fdf4c101090 2024-04-21T08:38:59.458 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-21T08:38:59.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.461+0000 7fdf50d15640 1 -- 172.21.15.43:0/381264264 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdf380a3d20 msgr2=0x7fdf380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.461+0000 7fdf50d15640 1 --2- 172.21.15.43:0/381264264 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdf380a3d20 0x7fdf380a61e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c0095f0 tx=0x7fdf3c005f90 comp rx=0 tx=0).stop 2024-04-21T08:38:59.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.461+0000 7fdf50d15640 1 -- 172.21.15.43:0/381264264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf4c101090 msgr2=0x7fdf4c198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.461+0000 7fdf50d15640 1 --2- 172.21.15.43:0/381264264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf4c101090 0x7fdf4c198710 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdf3400e9f0 tx=0x7fdf3400edf0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.461+0000 7fdf50d15640 1 -- 172.21.15.43:0/381264264 shutdown_connections 2024-04-21T08:38:59.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.461+0000 7fdf50d15640 1 --2- 172.21.15.43:0/381264264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf4c19cfd0 0x7fdf4c19d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.461+0000 7fdf50d15640 1 --2- 172.21.15.43:0/381264264 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdf380a3d20 0x7fdf380a61e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.461+0000 7fdf50d15640 1 --2- 172.21.15.43:0/381264264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf4c106160 0x7fdf4c198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.461+0000 7fdf50d15640 1 --2- 172.21.15.43:0/381264264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf4c101090 0x7fdf4c198710 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.461+0000 7fdf50d15640 1 -- 172.21.15.43:0/381264264 >> 172.21.15.43:0/381264264 conn(0x7fdf4c077f50 msgr2=0x7fdf4c069450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:59.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.461+0000 7fdf50d15640 1 -- 172.21.15.43:0/381264264 shutdown_connections 2024-04-21T08:38:59.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.461+0000 7fdf50d15640 1 -- 172.21.15.43:0/381264264 wait complete. 2024-04-21T08:38:59.479 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:38:59.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.591+0000 7f3fdfc31640 1 -- 172.21.15.43:0/457907296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fd8107260 msgr2=0x7f3fd810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.591+0000 7f3fdfc31640 1 --2- 172.21.15.43:0/457907296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fd8107260 0x7f3fd810ee30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0009ca0 tx=0x7f3fd002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.591+0000 7f3fdfc31640 1 -- 172.21.15.43:0/457907296 shutdown_connections 2024-04-21T08:38:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.591+0000 7f3fdfc31640 1 --2- 172.21.15.43:0/457907296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fd8107260 0x7f3fd810ee30 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.591+0000 7f3fdfc31640 1 --2- 172.21.15.43:0/457907296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fd8102b40 0x7f3fd8106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.591+0000 7f3fdfc31640 1 --2- 172.21.15.43:0/457907296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fd8102190 0x7f3fd8102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.591+0000 7f3fdfc31640 1 -- 172.21.15.43:0/457907296 >> 172.21.15.43:0/457907296 conn(0x7f3fd80793a0 msgr2=0x7f3fd80797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.591+0000 7f3fdfc31640 1 -- 172.21.15.43:0/457907296 shutdown_connections 2024-04-21T08:38:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.591+0000 7f3fdfc31640 1 -- 172.21.15.43:0/457907296 wait complete. 2024-04-21T08:38:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.592+0000 7f3fdfc31640 1 Processor -- start 2024-04-21T08:38:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.592+0000 7f3fdfc31640 1 -- start start 2024-04-21T08:38:59.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.592+0000 7f3fdfc31640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fd8102190 0x7f3fd8074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.592+0000 7f3fdfc31640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fd8107260 0x7f3fd8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.592+0000 7f3fdfc31640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fd8074cd0 0x7f3fd81a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.592+0000 7f3fdfc31640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd81121b0 con 0x7f3fd8102190 2024-04-21T08:38:59.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.592+0000 7f3fdfc31640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd8112030 con 0x7f3fd8074cd0 2024-04-21T08:38:59.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.592+0000 7f3fdfc31640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd8112330 con 0x7f3fd8107260 2024-04-21T08:38:59.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.592+0000 7f3fdd9cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fd8102190 0x7f3fd8074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.592+0000 7f3fdd9cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fd8102190 0x7f3fd8074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38664/0 (socket says 172.21.15.43:38664) 2024-04-21T08:38:59.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.592+0000 7f3fdd9cd640 1 -- 172.21.15.43:0/325364908 learned_addr learned my addr 172.21.15.43:0/325364908 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:59.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.592+0000 7f3fde1ce640 1 --2- 172.21.15.43:0/325364908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fd8074cd0 0x7f3fd81a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.592+0000 7f3fdd1cc640 1 --2- 172.21.15.43:0/325364908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fd8107260 0x7f3fd8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.592+0000 7f3fdd9cd640 1 -- 172.21.15.43:0/325364908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fd8107260 msgr2=0x7f3fd8074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.592+0000 7f3fdd9cd640 1 --2- 172.21.15.43:0/325364908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fd8107260 0x7f3fd8074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.592+0000 7f3fdd9cd640 1 -- 172.21.15.43:0/325364908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fd8074cd0 msgr2=0x7f3fd81a68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.592+0000 7f3fdd9cd640 1 --2- 172.21.15.43:0/325364908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fd8074cd0 0x7f3fd81a68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.592+0000 7f3fdd9cd640 1 -- 172.21.15.43:0/325364908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd81a6fc0 con 0x7f3fd8102190 2024-04-21T08:38:59.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.592+0000 7f3fde1ce640 1 --2- 172.21.15.43:0/325364908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fd8074cd0 0x7f3fd81a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:59.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.592+0000 7f3fdd1cc640 1 --2- 172.21.15.43:0/325364908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fd8107260 0x7f3fd8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:59.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.592+0000 7f3fdd9cd640 1 --2- 172.21.15.43:0/325364908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fd8102190 0x7f3fd8074250 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3fd400c890 tx=0x7f3fd400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:59.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.593+0000 7f3fceffd640 1 -- 172.21.15.43:0/325364908 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4004220 con 0x7f3fd8102190 2024-04-21T08:38:59.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.593+0000 7f3fceffd640 1 -- 172.21.15.43:0/325364908 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fd40043c0 con 0x7f3fd8102190 2024-04-21T08:38:59.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.593+0000 7f3fdfc31640 1 -- 172.21.15.43:0/325364908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fd81a72b0 con 0x7f3fd8102190 2024-04-21T08:38:59.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.593+0000 7f3fceffd640 1 -- 172.21.15.43:0/325364908 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4004a00 con 0x7f3fd8102190 2024-04-21T08:38:59.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.593+0000 7f3fdfc31640 1 -- 172.21.15.43:0/325364908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fd81a7850 con 0x7f3fd8102190 2024-04-21T08:38:59.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.594+0000 7f3fdfc31640 1 -- 172.21.15.43:0/325364908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fd80691c0 con 0x7f3fd8102190 2024-04-21T08:38:59.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.595+0000 7f3fceffd640 1 -- 172.21.15.43:0/325364908 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f3fd40079b0 con 0x7f3fd8102190 2024-04-21T08:38:59.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.595+0000 7f3fceffd640 1 --2- 172.21.15.43:0/325364908 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3fc40a3e60 0x7f3fc40a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.596+0000 7f3fdd1cc640 1 --2- 172.21.15.43:0/325364908 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3fc40a3e60 0x7f3fc40a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.596+0000 7f3fceffd640 1 -- 172.21.15.43:0/325364908 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f3fd40d0f10 con 0x7f3fd8102190 2024-04-21T08:38:59.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.596+0000 7f3fdd1cc640 1 --2- 172.21.15.43:0/325364908 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3fc40a3e60 0x7f3fc40a6320 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8005e00 tx=0x7f3fc8005c70 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:59.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.599+0000 7f3fceffd640 1 -- 172.21.15.43:0/325364908 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3fd4014030 con 0x7f3fd8102190 2024-04-21T08:38:59.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:59 smithi175 ceph-mon[31920]: from='client.15690 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:59.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:59 smithi175 ceph-mon[31920]: from='client.25192 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:59.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:59 smithi175 ceph-mon[31920]: Adjusting osd_memory_target on smithi175 to 176.2M 2024-04-21T08:38:59.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:59 smithi175 ceph-mon[31920]: Unable to set osd_memory_target on smithi175 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-21T08:38:59.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:59 smithi175 ceph-mon[31920]: Adjusting osd_memory_target on smithi112 to 176.2M 2024-04-21T08:38:59.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:59 smithi175 ceph-mon[31920]: Unable to set osd_memory_target on smithi112 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-04-21T08:38:59.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:59 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:59.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:59 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:59.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:59 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:59.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:59 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:59.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:59 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:59.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:59 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:59.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:59 smithi175 ceph-mon[31920]: Adjusting osd_memory_target on smithi043 to 177.2M 2024-04-21T08:38:59.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:59 smithi175 ceph-mon[31920]: Unable to set osd_memory_target on smithi043 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-21T08:38:59.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:59 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:59.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:59 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:38:59.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:59 smithi175 ceph-mon[31920]: Updating smithi043:/etc/ceph/ceph.conf 2024-04-21T08:38:59.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:59 smithi175 ceph-mon[31920]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-21T08:38:59.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:59 smithi175 ceph-mon[31920]: Updating smithi175:/etc/ceph/ceph.conf 2024-04-21T08:38:59.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:59 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3859683110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:59.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:59 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2557471155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:59.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:38:59 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3547881760' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:59.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.793+0000 7f3fdfc31640 1 -- 172.21.15.43:0/325364908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f3fd8102b40 con 0x7f3fd8102190 2024-04-21T08:38:59.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.793+0000 7f3fceffd640 1 -- 172.21.15.43:0/325364908 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7f3fd409d0f0 con 0x7f3fd8102190 2024-04-21T08:38:59.793 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:38:59.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:59 smithi112 ceph-mon[31085]: from='client.15690 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:59.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:59 smithi112 ceph-mon[31085]: from='client.25192 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:59.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:59 smithi112 ceph-mon[31085]: Adjusting osd_memory_target on smithi175 to 176.2M 2024-04-21T08:38:59.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:59 smithi112 ceph-mon[31085]: Unable to set osd_memory_target on smithi175 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-21T08:38:59.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:59 smithi112 ceph-mon[31085]: Adjusting osd_memory_target on smithi112 to 176.2M 2024-04-21T08:38:59.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:59 smithi112 ceph-mon[31085]: Unable to set osd_memory_target on smithi112 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-04-21T08:38:59.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:59 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:59.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:59 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:59.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:59 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:59.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:59 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:59.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:59 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:59.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:59 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:59.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:59 smithi112 ceph-mon[31085]: Adjusting osd_memory_target on smithi043 to 177.2M 2024-04-21T08:38:59.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:59 smithi112 ceph-mon[31085]: Unable to set osd_memory_target on smithi043 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-21T08:38:59.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:59 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:59.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:59 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:38:59.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:59 smithi112 ceph-mon[31085]: Updating smithi043:/etc/ceph/ceph.conf 2024-04-21T08:38:59.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:59 smithi112 ceph-mon[31085]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-21T08:38:59.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:59 smithi112 ceph-mon[31085]: Updating smithi175:/etc/ceph/ceph.conf 2024-04-21T08:38:59.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:59 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3859683110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:59.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:59 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2557471155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:59.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:38:59 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3547881760' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:59.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.796+0000 7f3fdfc31640 1 -- 172.21.15.43:0/325364908 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3fc40a3e60 msgr2=0x7f3fc40a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.796+0000 7f3fdfc31640 1 --2- 172.21.15.43:0/325364908 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3fc40a3e60 0x7f3fc40a6320 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8005e00 tx=0x7f3fc8005c70 comp rx=0 tx=0).stop 2024-04-21T08:38:59.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.796+0000 7f3fdfc31640 1 -- 172.21.15.43:0/325364908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fd8102190 msgr2=0x7f3fd8074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.796+0000 7f3fdfc31640 1 --2- 172.21.15.43:0/325364908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fd8102190 0x7f3fd8074250 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3fd400c890 tx=0x7f3fd400cd50 comp rx=0 tx=0).stop 2024-04-21T08:38:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.796+0000 7f3fdfc31640 1 -- 172.21.15.43:0/325364908 shutdown_connections 2024-04-21T08:38:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.796+0000 7f3fdfc31640 1 --2- 172.21.15.43:0/325364908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fd8074cd0 0x7f3fd81a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.796+0000 7f3fdfc31640 1 --2- 172.21.15.43:0/325364908 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3fc40a3e60 0x7f3fc40a6320 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.796+0000 7f3fdfc31640 1 --2- 172.21.15.43:0/325364908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fd8107260 0x7f3fd8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.796+0000 7f3fdfc31640 1 --2- 172.21.15.43:0/325364908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fd8102190 0x7f3fd8074250 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.796+0000 7f3fdfc31640 1 -- 172.21.15.43:0/325364908 >> 172.21.15.43:0/325364908 conn(0x7f3fd80793a0 msgr2=0x7f3fd80ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.796+0000 7f3fdfc31640 1 -- 172.21.15.43:0/325364908 shutdown_connections 2024-04-21T08:38:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.796+0000 7f3fdfc31640 1 -- 172.21.15.43:0/325364908 wait complete. 2024-04-21T08:38:59.805 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:59 smithi043 ceph-mon[27978]: from='client.15690 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:59.805 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:59 smithi043 ceph-mon[27978]: from='client.25192 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:59.805 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:59 smithi043 ceph-mon[27978]: Adjusting osd_memory_target on smithi175 to 176.2M 2024-04-21T08:38:59.806 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:59 smithi043 ceph-mon[27978]: Unable to set osd_memory_target on smithi175 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-21T08:38:59.806 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:59 smithi043 ceph-mon[27978]: Adjusting osd_memory_target on smithi112 to 176.2M 2024-04-21T08:38:59.806 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:59 smithi043 ceph-mon[27978]: Unable to set osd_memory_target on smithi112 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-04-21T08:38:59.806 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:59 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:59.806 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:59 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:38:59.806 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:59 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:59.806 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:59 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:59.806 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:59 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:59.806 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:59 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:59.806 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:59 smithi043 ceph-mon[27978]: Adjusting osd_memory_target on smithi043 to 177.2M 2024-04-21T08:38:59.806 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:59 smithi043 ceph-mon[27978]: Unable to set osd_memory_target on smithi043 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-21T08:38:59.806 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:59 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:59.807 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:59 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:38:59.807 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:59 smithi043 ceph-mon[27978]: Updating smithi043:/etc/ceph/ceph.conf 2024-04-21T08:38:59.807 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:59 smithi043 ceph-mon[27978]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-21T08:38:59.807 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:59 smithi043 ceph-mon[27978]: Updating smithi175:/etc/ceph/ceph.conf 2024-04-21T08:38:59.807 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:59 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3859683110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:59.807 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:59 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2557471155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:59.807 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:38:59 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3547881760' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:59.808 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:38:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.911+0000 7fdb19d8e640 1 -- 172.21.15.43:0/268744134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb141021a0 msgr2=0x7fdb14102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.911+0000 7fdb19d8e640 1 --2- 172.21.15.43:0/268744134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb141021a0 0x7fdb14102580 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdb04009900 tx=0x7fdb0402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.911+0000 7fdb19d8e640 1 -- 172.21.15.43:0/268744134 shutdown_connections 2024-04-21T08:38:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.911+0000 7fdb19d8e640 1 --2- 172.21.15.43:0/268744134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb14107270 0x7fdb1410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.911+0000 7fdb19d8e640 1 --2- 172.21.15.43:0/268744134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb14102b50 0x7fdb14106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.911+0000 7fdb19d8e640 1 --2- 172.21.15.43:0/268744134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb141021a0 0x7fdb14102580 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.911+0000 7fdb19d8e640 1 -- 172.21.15.43:0/268744134 >> 172.21.15.43:0/268744134 conn(0x7fdb140793a0 msgr2=0x7fdb140797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.911+0000 7fdb19d8e640 1 -- 172.21.15.43:0/268744134 shutdown_connections 2024-04-21T08:38:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.911+0000 7fdb19d8e640 1 -- 172.21.15.43:0/268744134 wait complete. 2024-04-21T08:38:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.912+0000 7fdb19d8e640 1 Processor -- start 2024-04-21T08:38:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.912+0000 7fdb19d8e640 1 -- start start 2024-04-21T08:38:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.912+0000 7fdb19d8e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb141021a0 0x7fdb14074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.912+0000 7fdb19d8e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb14107270 0x7fdb14074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.912+0000 7fdb19d8e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb14074cd0 0x7fdb141a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.912+0000 7fdb19d8e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdb141121d0 con 0x7fdb14107270 2024-04-21T08:38:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.912+0000 7fdb19d8e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdb14112050 con 0x7fdb14074cd0 2024-04-21T08:38:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.912+0000 7fdb19d8e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdb14112350 con 0x7fdb141021a0 2024-04-21T08:38:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.912+0000 7fdb0bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb14107270 0x7fdb14074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.912+0000 7fdb0bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb14107270 0x7fdb14074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38682/0 (socket says 172.21.15.43:38682) 2024-04-21T08:38:59.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.912+0000 7fdb0bfff640 1 -- 172.21.15.43:0/340692463 learned_addr learned my addr 172.21.15.43:0/340692463 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:38:59.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.913+0000 7fdb137fe640 1 --2- 172.21.15.43:0/340692463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb141021a0 0x7fdb14074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.913+0000 7fdb13fff640 1 --2- 172.21.15.43:0/340692463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb14074cd0 0x7fdb141a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.913+0000 7fdb0bfff640 1 -- 172.21.15.43:0/340692463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb141021a0 msgr2=0x7fdb14074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.913+0000 7fdb0bfff640 1 --2- 172.21.15.43:0/340692463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb141021a0 0x7fdb14074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.913+0000 7fdb0bfff640 1 -- 172.21.15.43:0/340692463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb14074cd0 msgr2=0x7fdb141a68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.913+0000 7fdb0bfff640 1 --2- 172.21.15.43:0/340692463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb14074cd0 0x7fdb141a68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.913+0000 7fdb0bfff640 1 -- 172.21.15.43:0/340692463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb141a6fa0 con 0x7fdb14107270 2024-04-21T08:38:59.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.913+0000 7fdb0bfff640 1 --2- 172.21.15.43:0/340692463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb14107270 0x7fdb14074790 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdafc00e9f0 tx=0x7fdafc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:59.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.913+0000 7fdb117fa640 1 -- 172.21.15.43:0/340692463 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc00ccc0 con 0x7fdb14107270 2024-04-21T08:38:59.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.913+0000 7fdb117fa640 1 -- 172.21.15.43:0/340692463 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdafc0054d0 con 0x7fdb14107270 2024-04-21T08:38:59.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.913+0000 7fdb117fa640 1 -- 172.21.15.43:0/340692463 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc010430 con 0x7fdb14107270 2024-04-21T08:38:59.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.913+0000 7fdb19d8e640 1 -- 172.21.15.43:0/340692463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb141a7290 con 0x7fdb14107270 2024-04-21T08:38:59.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.913+0000 7fdb19d8e640 1 -- 172.21.15.43:0/340692463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb141a7790 con 0x7fdb14107270 2024-04-21T08:38:59.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.914+0000 7fdb19d8e640 1 -- 172.21.15.43:0/340692463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb14069230 con 0x7fdb14107270 2024-04-21T08:38:59.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.915+0000 7fdb117fa640 1 -- 172.21.15.43:0/340692463 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fdafc0046c0 con 0x7fdb14107270 2024-04-21T08:38:59.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.916+0000 7fdb117fa640 1 --2- 172.21.15.43:0/340692463 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdb000a3de0 0x7fdb000a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.916+0000 7fdb137fe640 1 --2- 172.21.15.43:0/340692463 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdb000a3de0 0x7fdb000a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.916+0000 7fdb117fa640 1 -- 172.21.15.43:0/340692463 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fdafc0d1200 con 0x7fdb14107270 2024-04-21T08:38:59.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.917+0000 7fdb137fe640 1 --2- 172.21.15.43:0/340692463 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdb000a3de0 0x7fdb000a62a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdb04000c00 tx=0x7fdb04004040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:59.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:38:59.921+0000 7fdb117fa640 1 -- 172.21.15.43:0/340692463 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdafc09d360 con 0x7fdb14107270 2024-04-21T08:39:00.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.152+0000 7fdb19d8e640 1 -- 172.21.15.43:0/340692463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb1410a4a0 con 0x7fdb14107270 2024-04-21T08:39:00.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.153+0000 7fdb117fa640 1 -- 172.21.15.43:0/340692463 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7fdafc0bbe20 con 0x7fdb14107270 2024-04-21T08:39:00.152 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:39:00.152 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:39:00.153 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:39:00.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.155+0000 7fdb19d8e640 1 -- 172.21.15.43:0/340692463 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdb000a3de0 msgr2=0x7fdb000a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.155+0000 7fdb19d8e640 1 --2- 172.21.15.43:0/340692463 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdb000a3de0 0x7fdb000a62a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdb04000c00 tx=0x7fdb04004040 comp rx=0 tx=0).stop 2024-04-21T08:39:00.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.155+0000 7fdb19d8e640 1 -- 172.21.15.43:0/340692463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb14107270 msgr2=0x7fdb14074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.155+0000 7fdb19d8e640 1 --2- 172.21.15.43:0/340692463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb14107270 0x7fdb14074790 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdafc00e9f0 tx=0x7fdafc00eeb0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.155+0000 7fdb19d8e640 1 -- 172.21.15.43:0/340692463 shutdown_connections 2024-04-21T08:39:00.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.155+0000 7fdb19d8e640 1 --2- 172.21.15.43:0/340692463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb14074cd0 0x7fdb141a68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.155+0000 7fdb19d8e640 1 --2- 172.21.15.43:0/340692463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb14107270 0x7fdb14074790 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.155+0000 7fdb19d8e640 1 --2- 172.21.15.43:0/340692463 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdb000a3de0 0x7fdb000a62a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.155+0000 7fdb19d8e640 1 --2- 172.21.15.43:0/340692463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb141021a0 0x7fdb14074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.155+0000 7fdb19d8e640 1 -- 172.21.15.43:0/340692463 >> 172.21.15.43:0/340692463 conn(0x7fdb140793a0 msgr2=0x7fdb140ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:00.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.155+0000 7fdb19d8e640 1 -- 172.21.15.43:0/340692463 shutdown_connections 2024-04-21T08:39:00.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.155+0000 7fdb19d8e640 1 -- 172.21.15.43:0/340692463 wait complete. 2024-04-21T08:39:00.165 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:39:00.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.269+0000 7f6b9c58f640 1 -- 172.21.15.43:0/1478117361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b94102ea0 msgr2=0x7f6b9406a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.269+0000 7f6b9c58f640 1 --2- 172.21.15.43:0/1478117361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b94102ea0 0x7f6b9406a640 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6b90009ca0 tx=0x7f6b9002c180 comp rx=0 tx=0).stop 2024-04-21T08:39:00.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.269+0000 7f6b9c58f640 1 -- 172.21.15.43:0/1478117361 shutdown_connections 2024-04-21T08:39:00.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.269+0000 7f6b9c58f640 1 --2- 172.21.15.43:0/1478117361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b9410e810 0x7f6b94110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.269+0000 7f6b9c58f640 1 --2- 172.21.15.43:0/1478117361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b9406ab80 0x7f6b9406b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.269+0000 7f6b9c58f640 1 --2- 172.21.15.43:0/1478117361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b94102ea0 0x7f6b9406a640 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.269+0000 7f6b9c58f640 1 -- 172.21.15.43:0/1478117361 >> 172.21.15.43:0/1478117361 conn(0x7f6b94077e30 msgr2=0x7f6b94078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:00.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.269+0000 7f6b9c58f640 1 -- 172.21.15.43:0/1478117361 shutdown_connections 2024-04-21T08:39:00.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.270+0000 7f6b9c58f640 1 -- 172.21.15.43:0/1478117361 wait complete. 2024-04-21T08:39:00.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.270+0000 7f6b9c58f640 1 Processor -- start 2024-04-21T08:39:00.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.270+0000 7f6b9c58f640 1 -- start start 2024-04-21T08:39:00.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.270+0000 7f6b9c58f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b9406ab80 0x7f6b941a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.270+0000 7f6b9c58f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b94102ea0 0x7f6b941a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.270+0000 7f6b9c58f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b9410e810 0x7f6b9419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.270+0000 7f6b9c58f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6b94114460 con 0x7f6b94102ea0 2024-04-21T08:39:00.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.270+0000 7f6b9c58f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941142e0 con 0x7f6b9410e810 2024-04-21T08:39:00.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.270+0000 7f6b9c58f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941145e0 con 0x7f6b9406ab80 2024-04-21T08:39:00.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.270+0000 7f6b99b2a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b94102ea0 0x7f6b941a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.270+0000 7f6b99b2a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b94102ea0 0x7f6b941a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38704/0 (socket says 172.21.15.43:38704) 2024-04-21T08:39:00.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.270+0000 7f6b99b2a640 1 -- 172.21.15.43:0/3293512509 learned_addr learned my addr 172.21.15.43:0/3293512509 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:00.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.270+0000 7f6b9a32b640 1 --2- 172.21.15.43:0/3293512509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b9406ab80 0x7f6b941a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.270+0000 7f6b9ab2c640 1 --2- 172.21.15.43:0/3293512509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b9410e810 0x7f6b9419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.271+0000 7f6b99b2a640 1 -- 172.21.15.43:0/3293512509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b9406ab80 msgr2=0x7f6b941a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.271+0000 7f6b99b2a640 1 --2- 172.21.15.43:0/3293512509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b9406ab80 0x7f6b941a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.271+0000 7f6b99b2a640 1 -- 172.21.15.43:0/3293512509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b9410e810 msgr2=0x7f6b9419a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.271+0000 7f6b99b2a640 1 --2- 172.21.15.43:0/3293512509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b9410e810 0x7f6b9419a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.271+0000 7f6b99b2a640 1 -- 172.21.15.43:0/3293512509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9419aff0 con 0x7f6b94102ea0 2024-04-21T08:39:00.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.271+0000 7f6b99b2a640 1 --2- 172.21.15.43:0/3293512509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b94102ea0 0x7f6b941a0bf0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400b4a0 tx=0x7f6b8400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:00.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.271+0000 7f6b8b7fe640 1 -- 172.21.15.43:0/3293512509 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84014070 con 0x7f6b94102ea0 2024-04-21T08:39:00.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.271+0000 7f6b8b7fe640 1 -- 172.21.15.43:0/3293512509 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b84010ac0 con 0x7f6b94102ea0 2024-04-21T08:39:00.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.271+0000 7f6b9c58f640 1 -- 172.21.15.43:0/3293512509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9419b2e0 con 0x7f6b94102ea0 2024-04-21T08:39:00.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.271+0000 7f6b8b7fe640 1 -- 172.21.15.43:0/3293512509 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84017c30 con 0x7f6b94102ea0 2024-04-21T08:39:00.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.271+0000 7f6b9c58f640 1 -- 172.21.15.43:0/3293512509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b941a7500 con 0x7f6b94102ea0 2024-04-21T08:39:00.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.272+0000 7f6b9c58f640 1 -- 172.21.15.43:0/3293512509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b94101490 con 0x7f6b94102ea0 2024-04-21T08:39:00.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.273+0000 7f6b8b7fe640 1 -- 172.21.15.43:0/3293512509 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f6b84017070 con 0x7f6b94102ea0 2024-04-21T08:39:00.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.274+0000 7f6b8b7fe640 1 --2- 172.21.15.43:0/3293512509 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6b800a3e30 0x7f6b800a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.274+0000 7f6b9a32b640 1 --2- 172.21.15.43:0/3293512509 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6b800a3e30 0x7f6b800a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.274+0000 7f6b8b7fe640 1 -- 172.21.15.43:0/3293512509 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f6b840d02a0 con 0x7f6b94102ea0 2024-04-21T08:39:00.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.274+0000 7f6b9a32b640 1 --2- 172.21.15.43:0/3293512509 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6b800a3e30 0x7f6b800a62f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6b90009430 tx=0x7f6b90005c50 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:00.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.277+0000 7f6b8b7fe640 1 -- 172.21.15.43:0/3293512509 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6b8409be60 con 0x7f6b94102ea0 2024-04-21T08:39:00.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.468+0000 7f6b9c58f640 1 -- 172.21.15.43:0/3293512509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b941a77f0 con 0x7f6b94102ea0 2024-04-21T08:39:00.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.469+0000 7f6b8b7fe640 1 -- 172.21.15.43:0/3293512509 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+25233 (secure 0 0 0) 0x7f6b840a0160 con 0x7f6b94102ea0 2024-04-21T08:39:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:39:00.469 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":95,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:38:54.881943+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/126475132":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/4288951239":"2024-04-22T08:38:54.881882+0000","172.21.15.43:6801/1577030772":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/2382670190":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/1174411284":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/1577030772":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:39:00.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.471+0000 7f6b9c58f640 1 -- 172.21.15.43:0/3293512509 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6b800a3e30 msgr2=0x7f6b800a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.471+0000 7f6b9c58f640 1 --2- 172.21.15.43:0/3293512509 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6b800a3e30 0x7f6b800a62f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6b90009430 tx=0x7f6b90005c50 comp rx=0 tx=0).stop 2024-04-21T08:39:00.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.471+0000 7f6b9c58f640 1 -- 172.21.15.43:0/3293512509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b94102ea0 msgr2=0x7f6b941a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.471+0000 7f6b9c58f640 1 --2- 172.21.15.43:0/3293512509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b94102ea0 0x7f6b941a0bf0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400b4a0 tx=0x7f6b8400b960 comp rx=0 tx=0).stop 2024-04-21T08:39:00.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.471+0000 7f6b9c58f640 1 -- 172.21.15.43:0/3293512509 shutdown_connections 2024-04-21T08:39:00.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.471+0000 7f6b9c58f640 1 --2- 172.21.15.43:0/3293512509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b9410e810 0x7f6b9419a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.471+0000 7f6b9c58f640 1 --2- 172.21.15.43:0/3293512509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b94102ea0 0x7f6b941a0bf0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.471+0000 7f6b9c58f640 1 --2- 172.21.15.43:0/3293512509 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6b800a3e30 0x7f6b800a62f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.471+0000 7f6b9c58f640 1 --2- 172.21.15.43:0/3293512509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b9406ab80 0x7f6b941a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.471+0000 7f6b9c58f640 1 -- 172.21.15.43:0/3293512509 >> 172.21.15.43:0/3293512509 conn(0x7f6b94077e30 msgr2=0x7f6b9410f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:00.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.472+0000 7f6b9c58f640 1 -- 172.21.15.43:0/3293512509 shutdown_connections 2024-04-21T08:39:00.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.472+0000 7f6b9c58f640 1 -- 172.21.15.43:0/3293512509 wait complete. 2024-04-21T08:39:00.482 DEBUG:tasks.fwd_scrub:Yielding 2024-04-21T08:39:00.482 INFO:teuthology.run_tasks:Running task exec... 2024-04-21T08:39:00.492 INFO:teuthology.task.exec:Executing custom commands... 2024-04-21T08:39:00.492 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-04-21T08:39:00.492 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-21T08:39:00.493 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-21T08:39:00.493 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:39:00.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:39:00.493 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:39:00.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.599+0000 7f4f051ce640 1 -- 172.21.15.43:0/3901780735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f00102b70 msgr2=0x7f4f00106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.599+0000 7f4f051ce640 1 --2- 172.21.15.43:0/3901780735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f00102b70 0x7f4f00106bb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4009900 tx=0x7f4ef402bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:00.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.599+0000 7f4f051ce640 1 -- 172.21.15.43:0/3901780735 shutdown_connections 2024-04-21T08:39:00.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.599+0000 7f4f051ce640 1 --2- 172.21.15.43:0/3901780735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f00107290 0x7f4f0010ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.599+0000 7f4f051ce640 1 --2- 172.21.15.43:0/3901780735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f00102b70 0x7f4f00106bb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.599+0000 7f4f051ce640 1 --2- 172.21.15.43:0/3901780735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f001021c0 0x7f4f001025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.599+0000 7f4f051ce640 1 -- 172.21.15.43:0/3901780735 >> 172.21.15.43:0/3901780735 conn(0x7f4f000fda20 msgr2=0x7f4f000ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:00.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.599+0000 7f4f051ce640 1 -- 172.21.15.43:0/3901780735 shutdown_connections 2024-04-21T08:39:00.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.599+0000 7f4f051ce640 1 -- 172.21.15.43:0/3901780735 wait complete. 2024-04-21T08:39:00.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.600+0000 7f57837e6640 1 -- 172.21.15.43:0/1113950944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f577c101a40 msgr2=0x7f577c105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.600+0000 7f57837e6640 1 --2- 172.21.15.43:0/1113950944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f577c101a40 0x7f577c105a80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f576c009900 tx=0x7f576c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.600+0000 7f57837e6640 1 -- 172.21.15.43:0/1113950944 shutdown_connections 2024-04-21T08:39:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.600+0000 7f57837e6640 1 --2- 172.21.15.43:0/1113950944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f577c106160 0x7f577c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.600+0000 7f57837e6640 1 --2- 172.21.15.43:0/1113950944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f577c101a40 0x7f577c105a80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.600+0000 7f57837e6640 1 --2- 172.21.15.43:0/1113950944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f577c101090 0x7f577c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.600+0000 7f57837e6640 1 -- 172.21.15.43:0/1113950944 >> 172.21.15.43:0/1113950944 conn(0x7f577c077f50 msgr2=0x7f577c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.600+0000 7f57837e6640 1 -- 172.21.15.43:0/1113950944 shutdown_connections 2024-04-21T08:39:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.600+0000 7f57837e6640 1 -- 172.21.15.43:0/1113950944 wait complete. 2024-04-21T08:39:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.600+0000 7f4f051ce640 1 Processor -- start 2024-04-21T08:39:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.600+0000 7f4f051ce640 1 -- start start 2024-04-21T08:39:00.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.600+0000 7f57837e6640 1 Processor -- start 2024-04-21T08:39:00.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.600+0000 7f57837e6640 1 -- start start 2024-04-21T08:39:00.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f4f051ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f001021c0 0x7f4f001986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f4f051ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f00107290 0x7f4f00198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f4f051ce640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f0019cf80 0x7f4f0019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f4f051ce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4f00112200 con 0x7f4f00107290 2024-04-21T08:39:00.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f4f051ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4f00112080 con 0x7f4f001021c0 2024-04-21T08:39:00.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f4f051ce640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f00112380 con 0x7f4f0019cf80 2024-04-21T08:39:00.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f57837e6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f577c101090 0x7f577c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f57837e6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f577c106160 0x7f577c198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f4efe59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f00107290 0x7f4f00198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f57837e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f577c19d020 0x7f577c19d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f4efe59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f00107290 0x7f4f00198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38736/0 (socket says 172.21.15.43:38736) 2024-04-21T08:39:00.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f4efe59c640 1 -- 172.21.15.43:0/2297740344 learned_addr learned my addr 172.21.15.43:0/2297740344 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:00.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f4eff59e640 1 --2- 172.21.15.43:0/2297740344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f0019cf80 0x7f4f0019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f4efed9d640 1 --2- 172.21.15.43:0/2297740344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f001021c0 0x7f4f001986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f57837e6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f577c1110d0 con 0x7f577c19d020 2024-04-21T08:39:00.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f57837e6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f577c110f50 con 0x7f577c101090 2024-04-21T08:39:00.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f57837e6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f577c111250 con 0x7f577c106160 2024-04-21T08:39:00.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f5781582640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f577c101090 0x7f577c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f5781d83640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f577c19d020 0x7f577c19d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f5781582640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f577c101090 0x7f577c198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:55678/0 (socket says 172.21.15.43:55678) 2024-04-21T08:39:00.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f5780d81640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f577c106160 0x7f577c198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f5781582640 1 -- 172.21.15.43:0/2920528011 learned_addr learned my addr 172.21.15.43:0/2920528011 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:00.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f4efe59c640 1 -- 172.21.15.43:0/2297740344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f0019cf80 msgr2=0x7f4f0019d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f4efe59c640 1 --2- 172.21.15.43:0/2297740344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f0019cf80 0x7f4f0019d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f4efe59c640 1 -- 172.21.15.43:0/2297740344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f001021c0 msgr2=0x7f4f001986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f4efe59c640 1 --2- 172.21.15.43:0/2297740344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f001021c0 0x7f4f001986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f4efe59c640 1 -- 172.21.15.43:0/2297740344 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0019da70 con 0x7f4f00107290 2024-04-21T08:39:00.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f4efed9d640 1 --2- 172.21.15.43:0/2297740344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f001021c0 0x7f4f001986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:00.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f5780d81640 1 -- 172.21.15.43:0/2920528011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f577c101090 msgr2=0x7f577c198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f5780d81640 1 --2- 172.21.15.43:0/2920528011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f577c101090 0x7f577c198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f5780d81640 1 -- 172.21.15.43:0/2920528011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f577c19d020 msgr2=0x7f577c19d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f5780d81640 1 --2- 172.21.15.43:0/2920528011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f577c19d020 0x7f577c19d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f5780d81640 1 -- 172.21.15.43:0/2920528011 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577c19db10 con 0x7f577c106160 2024-04-21T08:39:00.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f5781582640 1 --2- 172.21.15.43:0/2920528011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f577c101090 0x7f577c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:00.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.601+0000 7f4efe59c640 1 --2- 172.21.15.43:0/2297740344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f00107290 0x7f4f00198c00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4000c00 tx=0x7f4ef402c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:00.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.602+0000 7f5780d81640 1 --2- 172.21.15.43:0/2920528011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f577c106160 0x7f577c198ca0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f576c036040 tx=0x7f576c02c640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:00.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.602+0000 7f4ee7fff640 1 -- 172.21.15.43:0/2297740344 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4043020 con 0x7f4f00107290 2024-04-21T08:39:00.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.602+0000 7f4f051ce640 1 -- 172.21.15.43:0/2297740344 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f0019dd00 con 0x7f4f00107290 2024-04-21T08:39:00.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.602+0000 7f4ee7fff640 1 -- 172.21.15.43:0/2297740344 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef402caf0 con 0x7f4f00107290 2024-04-21T08:39:00.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.602+0000 7f4ee7fff640 1 -- 172.21.15.43:0/2297740344 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4003b20 con 0x7f4f00107290 2024-04-21T08:39:00.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.602+0000 7f57727fc640 1 -- 172.21.15.43:0/2920528011 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c043020 con 0x7f577c106160 2024-04-21T08:39:00.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.602+0000 7f4f051ce640 1 -- 172.21.15.43:0/2297740344 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f001a5520 con 0x7f4f00107290 2024-04-21T08:39:00.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.602+0000 7f57837e6640 1 -- 172.21.15.43:0/2920528011 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f577c19dda0 con 0x7f577c106160 2024-04-21T08:39:00.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.602+0000 7f57727fc640 1 -- 172.21.15.43:0/2920528011 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f576c02ca70 con 0x7f577c106160 2024-04-21T08:39:00.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.602+0000 7f57727fc640 1 -- 172.21.15.43:0/2920528011 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c003a10 con 0x7f577c106160 2024-04-21T08:39:00.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.603+0000 7f57837e6640 1 -- 172.21.15.43:0/2920528011 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f577c102d40 con 0x7f577c106160 2024-04-21T08:39:00.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.603+0000 7f4f051ce640 1 -- 172.21.15.43:0/2297740344 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f0004fce0 con 0x7f4f00107290 2024-04-21T08:39:00.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.604+0000 7f4ee7fff640 1 -- 172.21.15.43:0/2297740344 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f4ef4046020 con 0x7f4f00107290 2024-04-21T08:39:00.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.605+0000 7f57727fc640 1 -- 172.21.15.43:0/2920528011 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f576c046020 con 0x7f577c106160 2024-04-21T08:39:00.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.605+0000 7f4ee7fff640 1 --2- 172.21.15.43:0/2297740344 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4ee80a3e60 0x7f4ee80a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.605+0000 7f4efed9d640 1 --2- 172.21.15.43:0/2297740344 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4ee80a3e60 0x7f4ee80a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.605+0000 7f4ee7fff640 1 -- 172.21.15.43:0/2297740344 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f4ef40f3010 con 0x7f4f00107290 2024-04-21T08:39:00.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.606+0000 7f57727fc640 1 --2- 172.21.15.43:0/2920528011 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f57680a3db0 0x7f57680a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.606+0000 7f4efed9d640 1 --2- 172.21.15.43:0/2297740344 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4ee80a3e60 0x7f4ee80a6320 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009fc0 tx=0x7f4ef0005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:00.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.606+0000 7f5781582640 1 --2- 172.21.15.43:0/2920528011 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f57680a3db0 0x7f57680a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.606+0000 7f57863d4740 1 Processor -- start 2024-04-21T08:39:00.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.606+0000 7f57727fc640 1 -- 172.21.15.43:0/2920528011 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f576c0f2bc0 con 0x7f577c106160 2024-04-21T08:39:00.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.606+0000 7f57863d4740 1 -- start start 2024-04-21T08:39:00.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.606+0000 7f57863d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb0f5768e0 0x55bb0f576cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.606+0000 7f57863d4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0f577290 0x55bb0f57f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.606+0000 7f57863d4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f57fe10 0x55bb0f582260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.606+0000 7f57863d4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bb0f4daf00 con 0x55bb0f57fe10 2024-04-21T08:39:00.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.606+0000 7f57863d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bb0f478590 con 0x55bb0f5768e0 2024-04-21T08:39:00.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.606+0000 7f57863d4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bb0f4d6ec0 con 0x55bb0f577290 2024-04-21T08:39:00.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.606+0000 7f5781582640 1 --2- 172.21.15.43:0/2920528011 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f57680a3db0 0x7f57680a6270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f57780105c0 tx=0x7f5778012040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:00.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.606+0000 7f5781d83640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f57fe10 0x55bb0f582260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.606+0000 7f5781d83640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f57fe10 0x55bb0f582260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38752/0 (socket says 172.21.15.43:38752) 2024-04-21T08:39:00.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.606+0000 7f5781d83640 1 -- 172.21.15.43:0/3886852247 learned_addr learned my addr 172.21.15.43:0/3886852247 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:00.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.606+0000 7f5781582640 1 --2- 172.21.15.43:0/3886852247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb0f5768e0 0x55bb0f576cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.606+0000 7f5780d81640 1 --2- 172.21.15.43:0/3886852247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0f577290 0x55bb0f57f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.606+0000 7f5781d83640 1 -- 172.21.15.43:0/3886852247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0f577290 msgr2=0x55bb0f57f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:00.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.606+0000 7f5781d83640 1 --2- 172.21.15.43:0/3886852247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0f577290 0x55bb0f57f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.606+0000 7f5781d83640 1 -- 172.21.15.43:0/3886852247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb0f5768e0 msgr2=0x55bb0f576cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:00.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.606+0000 7f5781d83640 1 --2- 172.21.15.43:0/3886852247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb0f5768e0 0x55bb0f576cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.607+0000 7f5781d83640 1 -- 172.21.15.43:0/3886852247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb0f2b57e0 con 0x55bb0f57fe10 2024-04-21T08:39:00.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.607+0000 7f5781582640 1 --2- 172.21.15.43:0/3886852247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb0f5768e0 0x55bb0f576cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:00.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.607+0000 7f5781d83640 1 --2- 172.21.15.43:0/3886852247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f57fe10 0x55bb0f582260 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5774009aa0 tx=0x7f5774009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff83f42062150b20 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:00.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.609+0000 7f57677fe640 1 -- 172.21.15.43:0/3886852247 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774004b20 con 0x55bb0f57fe10 2024-04-21T08:39:00.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.609+0000 7f57677fe640 1 -- 172.21.15.43:0/3886852247 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5774004cc0 con 0x55bb0f57fe10 2024-04-21T08:39:00.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.609+0000 7f57677fe640 1 -- 172.21.15.43:0/3886852247 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57740042b0 con 0x55bb0f57fe10 2024-04-21T08:39:00.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.609+0000 7f4ee7fff640 1 -- 172.21.15.43:0/2297740344 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ef404d020 con 0x7f4f00107290 2024-04-21T08:39:00.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.610+0000 7f57863d4740 1 -- 172.21.15.43:0/3886852247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f57fe10 msgr2=0x55bb0f582260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.610+0000 7f57863d4740 1 --2- 172.21.15.43:0/3886852247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f57fe10 0x55bb0f582260 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5774009aa0 tx=0x7f5774009500 comp rx=0 tx=0).stop 2024-04-21T08:39:00.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.610+0000 7f57863d4740 1 -- 172.21.15.43:0/3886852247 shutdown_connections 2024-04-21T08:39:00.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.610+0000 7f57863d4740 1 --2- 172.21.15.43:0/3886852247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f57fe10 0x55bb0f582260 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.610+0000 7f57863d4740 1 --2- 172.21.15.43:0/3886852247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0f577290 0x55bb0f57f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.610+0000 7f57863d4740 1 --2- 172.21.15.43:0/3886852247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb0f5768e0 0x55bb0f576cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.610+0000 7f57863d4740 1 -- 172.21.15.43:0/3886852247 >> 172.21.15.43:0/3886852247 conn(0x55bb0f570fb0 msgr2=0x55bb0f571390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:00.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.610+0000 7f57863d4740 1 -- 172.21.15.43:0/3886852247 shutdown_connections 2024-04-21T08:39:00.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.610+0000 7f57863d4740 1 -- 172.21.15.43:0/3886852247 wait complete. 2024-04-21T08:39:00.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.610+0000 7f57863d4740 1 -- start start 2024-04-21T08:39:00.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.610+0000 7f57863d4740 1 Processor -- start 2024-04-21T08:39:00.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.611+0000 7f57863d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb0f5768e0 0x55bb0f57cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.611+0000 7f57863d4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0f577290 0x55bb0f57d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.611+0000 7f57863d4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f57fe10 0x55bb0f57d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.611+0000 7f57863d4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bb0f582d80 con 0x55bb0f57fe10 2024-04-21T08:39:00.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.611+0000 7f57863d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bb0f494580 con 0x55bb0f5768e0 2024-04-21T08:39:00.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.611+0000 7f57863d4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bb0f582f00 con 0x55bb0f577290 2024-04-21T08:39:00.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.611+0000 7f57863d4740 10 client.0 _ll_register_callbacks cb 0x55bb0f49a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:00.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.611+0000 7f5781d83640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f57fe10 0x55bb0f57d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.611+0000 7f5781d83640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f57fe10 0x55bb0f57d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38762/0 (socket says 172.21.15.43:38762) 2024-04-21T08:39:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.611+0000 7f5781d83640 1 -- 172.21.15.43:0/2435464951 learned_addr learned my addr 172.21.15.43:0/2435464951 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.611+0000 7f5780d81640 1 --2- 172.21.15.43:0/2435464951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0f577290 0x55bb0f57d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.611+0000 7f5781582640 1 --2- 172.21.15.43:0/2435464951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb0f5768e0 0x55bb0f57cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.611+0000 7f5781d83640 1 -- 172.21.15.43:0/2435464951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0f577290 msgr2=0x55bb0f57d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.611+0000 7f5781d83640 1 --2- 172.21.15.43:0/2435464951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0f577290 0x55bb0f57d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.611+0000 7f5781d83640 1 -- 172.21.15.43:0/2435464951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb0f5768e0 msgr2=0x55bb0f57cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.611+0000 7f5781d83640 1 --2- 172.21.15.43:0/2435464951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb0f5768e0 0x55bb0f57cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.611+0000 7f5781d83640 1 -- 172.21.15.43:0/2435464951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb0f315f70 con 0x55bb0f57fe10 2024-04-21T08:39:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.612+0000 7f5781d83640 1 --2- 172.21.15.43:0/2435464951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f57fe10 0x55bb0f57d820 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5774000f80 tx=0x7f5774004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d5a5fda3b61f9b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.612+0000 7f5764ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.612+0000 7f5764ff9640 1 -- 172.21.15.43:0/2435464951 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577400bd30 con 0x55bb0f57fe10 2024-04-21T08:39:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.612+0000 7f5764ff9640 1 -- 172.21.15.43:0/2435464951 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f5774004050 con 0x55bb0f57fe10 2024-04-21T08:39:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.612+0000 7f57863d4740 1 -- 172.21.15.43:0/2435464951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5774004cc0 con 0x55bb0f57fe10 2024-04-21T08:39:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.612+0000 7f5764ff9640 1 -- 172.21.15.43:0/2435464951 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5774004250 con 0x55bb0f57fe10 2024-04-21T08:39:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.612+0000 7f57863d4740 10 client.15756 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.612+0000 7f57863d4740 1 -- 172.21.15.43:0/2435464951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bb0f1ad170 con 0x55bb0f57fe10 2024-04-21T08:39:00.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.612+0000 7f5764ff9640 1 -- 172.21.15.43:0/2435464951 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5774004cc0 con 0x55bb0f57fe10 2024-04-21T08:39:00.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.612+0000 7f5764ff9640 1 -- 172.21.15.43:0/2435464951 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5774005600 con 0x55bb0f57fe10 2024-04-21T08:39:00.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.612+0000 7f57863d4740 10 client.15756 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:00.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.612+0000 7f57863d4740 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:00.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.613+0000 7f57863d4740 20 client.15756 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:00.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.613+0000 7f57863d4740 1 --2- 172.21.15.43:0/2435464951 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55bb0f590230 0x55bb0f592620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.613+0000 7f57863d4740 4 client.15756 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:00.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.613+0000 7f57863d4740 1 -- 172.21.15.43:0/2435464951 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb0f4e8920 con 0x55bb0f590230 2024-04-21T08:39:00.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.613+0000 7f5781582640 1 --2- 172.21.15.43:0/2435464951 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55bb0f590230 0x55bb0f592620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.613+0000 7f5781582640 1 --2- 172.21.15.43:0/2435464951 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55bb0f590230 0x55bb0f592620 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2089364d9e73ff93 server_cookie=241e32ac64a95210 in_seq=0 out_seq=0 2024-04-21T08:39:00.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.613+0000 7f5764ff9640 10 client.15756 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:00.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.614+0000 7f5764ff9640 1 -- 172.21.15.43:0/2435464951 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bb0f4e8920 con 0x55bb0f590230 2024-04-21T08:39:00.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.614+0000 7f5764ff9640 10 client.15756 handle_command_reply: tid=0 2024-04-21T08:39:00.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.614+0000 7f57863d4740 1 client.15756 shutdown 2024-04-21T08:39:00.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.614+0000 7f57863d4740 1 -- 172.21.15.43:0/2435464951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f57fe10 msgr2=0x55bb0f57d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.614+0000 7f57863d4740 1 --2- 172.21.15.43:0/2435464951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f57fe10 0x55bb0f57d820 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5774000f80 tx=0x7f5774004a40 comp rx=0 tx=0).stop 2024-04-21T08:39:00.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.614+0000 7f57863d4740 1 -- 172.21.15.43:0/2435464951 shutdown_connections 2024-04-21T08:39:00.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.614+0000 7f57863d4740 1 --2- 172.21.15.43:0/2435464951 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55bb0f590230 0x55bb0f592620 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.614+0000 7f57863d4740 1 --2- 172.21.15.43:0/2435464951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f57fe10 0x55bb0f57d820 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.614+0000 7f57863d4740 1 --2- 172.21.15.43:0/2435464951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0f577290 0x55bb0f57d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.614+0000 7f5764ff9640 0 client.15756 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:00.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.614+0000 7f57863d4740 1 --2- 172.21.15.43:0/2435464951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb0f5768e0 0x55bb0f57cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.614+0000 7f57863d4740 1 -- 172.21.15.43:0/2435464951 >> 172.21.15.43:0/2435464951 conn(0x55bb0f570fb0 msgr2=0x55bb0f572d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:00.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.614+0000 7f57863d4740 1 -- 172.21.15.43:0/2435464951 shutdown_connections 2024-04-21T08:39:00.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.614+0000 7f57863d4740 1 -- 172.21.15.43:0/2435464951 wait complete. 2024-04-21T08:39:00.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.614+0000 7f57863d4740 20 client.15756 trim_cache size 0 max 16384 2024-04-21T08:39:00.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.632+0000 7f57863d4740 1 Processor -- start 2024-04-21T08:39:00.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.633+0000 7f57863d4740 1 -- start start 2024-04-21T08:39:00.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.633+0000 7f57863d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb0f5768e0 0x55bb0f58bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.633+0000 7f57863d4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f577290 0x55bb0f57f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.633+0000 7f57863d4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0f57fe10 0x55bb0f56fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.633+0000 7f57863d4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f577c1ad1e0 con 0x55bb0f577290 2024-04-21T08:39:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.633+0000 7f57863d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f577c1aed20 con 0x55bb0f5768e0 2024-04-21T08:39:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.633+0000 7f57863d4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f577c1ad8b0 con 0x55bb0f57fe10 2024-04-21T08:39:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.633+0000 7f5780d81640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f577290 0x55bb0f57f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.633+0000 7f5780d81640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f577290 0x55bb0f57f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38770/0 (socket says 172.21.15.43:38770) 2024-04-21T08:39:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.633+0000 7f5780d81640 1 -- 172.21.15.43:0/894623007 learned_addr learned my addr 172.21.15.43:0/894623007 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.633+0000 7f5781582640 1 --2- 172.21.15.43:0/894623007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb0f5768e0 0x55bb0f58bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.633+0000 7f5781d83640 1 --2- 172.21.15.43:0/894623007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0f57fe10 0x55bb0f56fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.633+0000 7f5780d81640 1 -- 172.21.15.43:0/894623007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0f57fe10 msgr2=0x55bb0f56fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.633+0000 7f5780d81640 1 --2- 172.21.15.43:0/894623007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0f57fe10 0x55bb0f56fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.633+0000 7f5780d81640 1 -- 172.21.15.43:0/894623007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb0f5768e0 msgr2=0x55bb0f58bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.633+0000 7f5780d81640 1 --2- 172.21.15.43:0/894623007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb0f5768e0 0x55bb0f58bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.633+0000 7f5780d81640 1 -- 172.21.15.43:0/894623007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb0f57b2f0 con 0x55bb0f577290 2024-04-21T08:39:00.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.633+0000 7f5780d81640 1 --2- 172.21.15.43:0/894623007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f577290 0x55bb0f57f430 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f576c005ec0 tx=0x7f576c0f2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225e6b1cc01b324a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:00.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.633+0000 7f5766ffd640 1 -- 172.21.15.43:0/894623007 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c0bee30 con 0x55bb0f577290 2024-04-21T08:39:00.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.634+0000 7f5766ffd640 1 -- 172.21.15.43:0/894623007 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f576c0be730 con 0x55bb0f577290 2024-04-21T08:39:00.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.634+0000 7f5766ffd640 1 -- 172.21.15.43:0/894623007 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c005340 con 0x55bb0f577290 2024-04-21T08:39:00.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.634+0000 7f57863d4740 1 -- 172.21.15.43:0/894623007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f577290 msgr2=0x55bb0f57f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.634+0000 7f57863d4740 1 --2- 172.21.15.43:0/894623007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f577290 0x55bb0f57f430 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f576c005ec0 tx=0x7f576c0f2040 comp rx=0 tx=0).stop 2024-04-21T08:39:00.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.634+0000 7f57863d4740 1 -- 172.21.15.43:0/894623007 shutdown_connections 2024-04-21T08:39:00.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.634+0000 7f57863d4740 1 --2- 172.21.15.43:0/894623007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0f57fe10 0x55bb0f56fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.634+0000 7f57863d4740 1 --2- 172.21.15.43:0/894623007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f577290 0x55bb0f57f430 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.634+0000 7f57863d4740 1 --2- 172.21.15.43:0/894623007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb0f5768e0 0x55bb0f58bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.634+0000 7f57863d4740 1 -- 172.21.15.43:0/894623007 >> 172.21.15.43:0/894623007 conn(0x55bb0f570fb0 msgr2=0x55bb0f56f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:00.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.634+0000 7f57863d4740 1 -- 172.21.15.43:0/894623007 shutdown_connections 2024-04-21T08:39:00.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.634+0000 7f57863d4740 1 -- 172.21.15.43:0/894623007 wait complete. 2024-04-21T08:39:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.635+0000 7f57863d4740 1 -- start start 2024-04-21T08:39:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.635+0000 7f57863d4740 1 Processor -- start 2024-04-21T08:39:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.635+0000 7f57863d4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0f5768e0 0x55bb0f583da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.635+0000 7f57863d4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f577290 0x55bb0f5842e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.635+0000 7f57863d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb0f57fe10 0x55bb0f584820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.635+0000 7f57863d4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f577c1ae650 con 0x55bb0f577290 2024-04-21T08:39:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.635+0000 7f57863d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f577c1adf80 con 0x55bb0f57fe10 2024-04-21T08:39:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.635+0000 7f57863d4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f577c1ab1c0 con 0x55bb0f5768e0 2024-04-21T08:39:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.635+0000 7f57863d4740 10 client.0 _ll_register_callbacks cb 0x7f5774009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.635+0000 7f5780d81640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f577290 0x55bb0f5842e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.635+0000 7f5780d81640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f577290 0x55bb0f5842e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38784/0 (socket says 172.21.15.43:38784) 2024-04-21T08:39:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.635+0000 7f5780d81640 1 -- 172.21.15.43:0/2010289139 learned_addr learned my addr 172.21.15.43:0/2010289139 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.635+0000 7f5781d83640 1 --2- 172.21.15.43:0/2010289139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb0f57fe10 0x55bb0f584820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.635+0000 7f5781582640 1 --2- 172.21.15.43:0/2010289139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0f5768e0 0x55bb0f583da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.636+0000 7f5780d81640 1 -- 172.21.15.43:0/2010289139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0f5768e0 msgr2=0x55bb0f583da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.636+0000 7f5780d81640 1 --2- 172.21.15.43:0/2010289139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0f5768e0 0x55bb0f583da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.636+0000 7f5780d81640 1 -- 172.21.15.43:0/2010289139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb0f57fe10 msgr2=0x55bb0f584820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.636+0000 7f5780d81640 1 --2- 172.21.15.43:0/2010289139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb0f57fe10 0x55bb0f584820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.636+0000 7f5780d81640 1 -- 172.21.15.43:0/2010289139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb0f5f56d0 con 0x55bb0f577290 2024-04-21T08:39:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.636+0000 7f5780d81640 1 --2- 172.21.15.43:0/2010289139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f577290 0x55bb0f5842e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f576c0bb100 tx=0x7f576c0f0360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85324ca1bdc31f5a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.636+0000 7f5764ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.636+0000 7f5764ff9640 1 -- 172.21.15.43:0/2010289139 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c0a1020 con 0x55bb0f577290 2024-04-21T08:39:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.636+0000 7f5764ff9640 1 -- 172.21.15.43:0/2010289139 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f576c0c7120 con 0x55bb0f577290 2024-04-21T08:39:00.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.636+0000 7f57863d4740 1 -- 172.21.15.43:0/2010289139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f576c0be730 con 0x55bb0f577290 2024-04-21T08:39:00.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.636+0000 7f5764ff9640 1 -- 172.21.15.43:0/2010289139 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f576c0dd050 con 0x55bb0f577290 2024-04-21T08:39:00.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.636+0000 7f57863d4740 10 client.15762 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:00.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.636+0000 7f57863d4740 1 -- 172.21.15.43:0/2010289139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bb0f5f69f0 con 0x55bb0f577290 2024-04-21T08:39:00.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.636+0000 7f5764ff9640 1 -- 172.21.15.43:0/2010289139 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f576c0be730 con 0x55bb0f577290 2024-04-21T08:39:00.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.637+0000 7f5764ff9640 1 -- 172.21.15.43:0/2010289139 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f576c003c40 con 0x55bb0f577290 2024-04-21T08:39:00.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.637+0000 7f57863d4740 10 client.15762 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:00.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.637+0000 7f57863d4740 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:00.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.637+0000 7f57863d4740 20 client.15762 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:00.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.637+0000 7f57863d4740 1 --2- 172.21.15.43:0/2010289139 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55bb0f5f6ec0 0x55bb0f5f92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.637+0000 7f57863d4740 4 client.15762 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:39:00.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.637+0000 7f57863d4740 1 -- 172.21.15.43:0/2010289139 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bb0f4e2790 con 0x55bb0f5f6ec0 2024-04-21T08:39:00.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.637+0000 7f5781582640 1 --2- 172.21.15.43:0/2010289139 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55bb0f5f6ec0 0x55bb0f5f92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.637+0000 7f5764ff9640 10 client.15762 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:00.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.637+0000 7f5781582640 1 --2- 172.21.15.43:0/2010289139 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55bb0f5f6ec0 0x55bb0f5f92b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6680839564b49f3 server_cookie=1c9aaa986a01b893 in_seq=0 out_seq=0 2024-04-21T08:39:00.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.638+0000 7f5764ff9640 1 -- 172.21.15.43:0/2010289139 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bb0f4e2790 con 0x55bb0f5f6ec0 2024-04-21T08:39:00.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.638+0000 7f5764ff9640 10 client.15762 handle_command_reply: tid=0 2024-04-21T08:39:00.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.638+0000 7f57863d4740 1 client.15762 shutdown 2024-04-21T08:39:00.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.638+0000 7f57863d4740 1 -- 172.21.15.43:0/2010289139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f577290 msgr2=0x55bb0f5842e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.638+0000 7f57863d4740 1 --2- 172.21.15.43:0/2010289139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f577290 0x55bb0f5842e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f576c0bb100 tx=0x7f576c0f0360 comp rx=0 tx=0).stop 2024-04-21T08:39:00.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.638+0000 7f57863d4740 1 -- 172.21.15.43:0/2010289139 shutdown_connections 2024-04-21T08:39:00.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.638+0000 7f57863d4740 1 --2- 172.21.15.43:0/2010289139 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55bb0f5f6ec0 0x55bb0f5f92b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.638+0000 7f57863d4740 1 --2- 172.21.15.43:0/2010289139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb0f57fe10 0x55bb0f584820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.639+0000 7f5764ff9640 0 client.15762 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:00.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.639+0000 7f57863d4740 1 --2- 172.21.15.43:0/2010289139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb0f577290 0x55bb0f5842e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.639+0000 7f57863d4740 1 --2- 172.21.15.43:0/2010289139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb0f5768e0 0x55bb0f583da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.639+0000 7f57863d4740 1 -- 172.21.15.43:0/2010289139 >> 172.21.15.43:0/2010289139 conn(0x55bb0f570fb0 msgr2=0x55bb0f570010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:00.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.639+0000 7f57863d4740 1 -- 172.21.15.43:0/2010289139 shutdown_connections 2024-04-21T08:39:00.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.639+0000 7f57863d4740 1 -- 172.21.15.43:0/2010289139 wait complete. 2024-04-21T08:39:00.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.639+0000 7f57863d4740 20 client.15762 trim_cache size 0 max 16384 2024-04-21T08:39:00.645 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:39:00.645 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:39:00.645 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "d2c011c5-55be-418f-b535-a24127a92cbf", 2024-04-21T08:39:00.646 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:39:00.646 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:39:00.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.639+0000 7f57837e6640 1 -- 172.21.15.43:0/2920528011 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f57680a3db0 msgr2=0x7f57680a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.639+0000 7f57837e6640 1 --2- 172.21.15.43:0/2920528011 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f57680a3db0 0x7f57680a6270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f57780105c0 tx=0x7f5778012040 comp rx=0 tx=0).stop 2024-04-21T08:39:00.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.639+0000 7f57837e6640 1 -- 172.21.15.43:0/2920528011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f577c106160 msgr2=0x7f577c198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.639+0000 7f57837e6640 1 --2- 172.21.15.43:0/2920528011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f577c106160 0x7f577c198ca0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f576c036040 tx=0x7f576c02c640 comp rx=0 tx=0).stop 2024-04-21T08:39:00.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.639+0000 7f57837e6640 1 -- 172.21.15.43:0/2920528011 shutdown_connections 2024-04-21T08:39:00.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.639+0000 7f57837e6640 1 --2- 172.21.15.43:0/2920528011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f577c19d020 0x7f577c19d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.639+0000 7f57837e6640 1 --2- 172.21.15.43:0/2920528011 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f57680a3db0 0x7f57680a6270 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.639+0000 7f57837e6640 1 --2- 172.21.15.43:0/2920528011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f577c106160 0x7f577c198ca0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.639+0000 7f57837e6640 1 --2- 172.21.15.43:0/2920528011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f577c101090 0x7f577c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.639+0000 7f57837e6640 1 -- 172.21.15.43:0/2920528011 >> 172.21.15.43:0/2920528011 conn(0x7f577c077f50 msgr2=0x7f577c0694e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:00.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.639+0000 7f57837e6640 1 -- 172.21.15.43:0/2920528011 shutdown_connections 2024-04-21T08:39:00.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.639+0000 7f57837e6640 1 -- 172.21.15.43:0/2920528011 wait complete. 2024-04-21T08:39:00.649 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:39:00.649 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:39:00.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: Updating smithi112:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:39:00.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: Updating smithi043:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:39:00.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: Updating smithi175:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:39:00.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:39:00.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:39:00.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:39:00.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:39:00.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: Updating smithi175:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:39:00.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: Updating smithi112:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:39:00.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: Updating smithi043:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:39:00.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/381264264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:39:00.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-21T08:39:00.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:39:00.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:39:00.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:39:00.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:39:00.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:39:00.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:39:00.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:39:00.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:39:00.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:00.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:39:00.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:39:00.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/325364908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:39:00.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:00 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/340692463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:00.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.755+0000 7f1a17704640 1 -- 172.21.15.43:0/4049409182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a10101a30 msgr2=0x7f1a10105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.755+0000 7f1a17704640 1 --2- 172.21.15.43:0/4049409182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a10101a30 0x7f1a10105a70 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009900 tx=0x7f1a0c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:00.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.755+0000 7f1a17704640 1 -- 172.21.15.43:0/4049409182 shutdown_connections 2024-04-21T08:39:00.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.755+0000 7f1a17704640 1 --2- 172.21.15.43:0/4049409182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a10106150 0x7f1a1010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.755+0000 7f1a17704640 1 --2- 172.21.15.43:0/4049409182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a10101a30 0x7f1a10105a70 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.755+0000 7f1a17704640 1 --2- 172.21.15.43:0/4049409182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a10101080 0x7f1a10101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.755+0000 7f1a17704640 1 -- 172.21.15.43:0/4049409182 >> 172.21.15.43:0/4049409182 conn(0x7f1a10078000 msgr2=0x7f1a10078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:00.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.755+0000 7f1a17704640 1 -- 172.21.15.43:0/4049409182 shutdown_connections 2024-04-21T08:39:00.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.755+0000 7f1a17704640 1 -- 172.21.15.43:0/4049409182 wait complete. 2024-04-21T08:39:00.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.756+0000 7f1a17704640 1 Processor -- start 2024-04-21T08:39:00.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.756+0000 7f1a17704640 1 -- start start 2024-04-21T08:39:00.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.756+0000 7f1a17704640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a10101080 0x7f1a101994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.756+0000 7f1a17704640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a10106150 0x7f1a10199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.756+0000 7f1a17704640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a1019a010 0x7f1a101a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.756+0000 7f1a17704640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a10111070 con 0x7f1a1019a010 2024-04-21T08:39:00.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.756+0000 7f1a17704640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a10110ef0 con 0x7f1a10106150 2024-04-21T08:39:00.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.756+0000 7f1a17704640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1a101111f0 con 0x7f1a10101080 2024-04-21T08:39:00.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.756+0000 7f1a15ca1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a1019a010 0x7f1a101a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.756+0000 7f1a154a0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a10101080 0x7f1a101994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.756+0000 7f1a15ca1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a1019a010 0x7f1a101a4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38794/0 (socket says 172.21.15.43:38794) 2024-04-21T08:39:00.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.756+0000 7f1a15ca1640 1 -- 172.21.15.43:0/709863854 learned_addr learned my addr 172.21.15.43:0/709863854 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:00.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.756+0000 7f1a14c9f640 1 --2- 172.21.15.43:0/709863854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a10106150 0x7f1a10199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.756+0000 7f1a154a0640 1 -- 172.21.15.43:0/709863854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a10106150 msgr2=0x7f1a10199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.756+0000 7f1a154a0640 1 --2- 172.21.15.43:0/709863854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a10106150 0x7f1a10199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.756+0000 7f1a154a0640 1 -- 172.21.15.43:0/709863854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a1019a010 msgr2=0x7f1a101a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.756+0000 7f1a154a0640 1 --2- 172.21.15.43:0/709863854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a1019a010 0x7f1a101a4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.756+0000 7f1a154a0640 1 -- 172.21.15.43:0/709863854 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a101a4d90 con 0x7f1a10101080 2024-04-21T08:39:00.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.756+0000 7f1a15ca1640 1 --2- 172.21.15.43:0/709863854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a1019a010 0x7f1a101a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:00.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.757+0000 7f1a154a0640 1 --2- 172.21.15.43:0/709863854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a10101080 0x7f1a101994f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400e9f0 tx=0x7f1a0400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:00.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.757+0000 7f1a027fc640 1 -- 172.21.15.43:0/709863854 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0400ccc0 con 0x7f1a10101080 2024-04-21T08:39:00.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.757+0000 7f1a027fc640 1 -- 172.21.15.43:0/709863854 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a040108e0 con 0x7f1a10101080 2024-04-21T08:39:00.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.757+0000 7f1a027fc640 1 -- 172.21.15.43:0/709863854 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04018880 con 0x7f1a10101080 2024-04-21T08:39:00.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.757+0000 7f1a17704640 1 -- 172.21.15.43:0/709863854 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a101a5080 con 0x7f1a10101080 2024-04-21T08:39:00.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.757+0000 7f1a17704640 1 -- 172.21.15.43:0/709863854 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a101a55a0 con 0x7f1a10101080 2024-04-21T08:39:00.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.759+0000 7f1a027fc640 1 -- 172.21.15.43:0/709863854 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f1a04010430 con 0x7f1a10101080 2024-04-21T08:39:00.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.759+0000 7f1a1a3d5740 1 Processor -- start 2024-04-21T08:39:00.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.760+0000 7f1a1a3d5740 1 -- start start 2024-04-21T08:39:00.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.760+0000 7f1a1a3d5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633175072b0 0x563317507690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.760+0000 7f1a1a3d5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563317507bd0 0x563317514950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.760+0000 7f1a1a3d5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563317514e90 0x563317517280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.760+0000 7f1a1a3d5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56331746c1e0 con 0x5633175072b0 2024-04-21T08:39:00.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.760+0000 7f1a1a3d5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563317409390 con 0x563317514e90 2024-04-21T08:39:00.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.760+0000 7f1a1a3d5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5633174681a0 con 0x563317507bd0 2024-04-21T08:39:00.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.760+0000 7f1a027fc640 1 --2- 172.21.15.43:0/709863854 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f19fc0a3db0 0x7f19fc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.760+0000 7f1a14c9f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563317514e90 0x563317517280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.760+0000 7f1a15ca1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633175072b0 0x563317507690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.760+0000 7f1a154a0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563317507bd0 0x563317514950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.760+0000 7f1a14c9f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563317514e90 0x563317517280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:55750/0 (socket says 172.21.15.43:55750) 2024-04-21T08:39:00.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.760+0000 7f1a14c9f640 1 -- 172.21.15.43:0/4227891663 learned_addr learned my addr 172.21.15.43:0/4227891663 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:00.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.760+0000 7f1a027fc640 1 -- 172.21.15.43:0/709863854 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f1a040d10d0 con 0x7f1a10101080 2024-04-21T08:39:00.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.760+0000 7f1a15ca1640 1 --2- 172.21.15.43:0/709863854 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f19fc0a3db0 0x7f19fc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.760+0000 7f1a14c9f640 1 -- 172.21.15.43:0/4227891663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563317507bd0 msgr2=0x563317514950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:00.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.760+0000 7f1a14c9f640 1 --2- 172.21.15.43:0/4227891663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563317507bd0 0x563317514950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.760+0000 7f1a14c9f640 1 -- 172.21.15.43:0/4227891663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633175072b0 msgr2=0x563317507690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:00.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.760+0000 7f1a14c9f640 1 --2- 172.21.15.43:0/4227891663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633175072b0 0x563317507690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.760+0000 7f1a14c9f640 1 -- 172.21.15.43:0/4227891663 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633172487d0 con 0x563317514e90 2024-04-21T08:39:00.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.761+0000 7f1a14c9f640 1 --2- 172.21.15.43:0/4227891663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563317514e90 0x563317517280 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009900 tx=0x7f1a0c005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7c5b7a2ac21f5c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:00.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.761+0000 7f1a15ca1640 1 --2- 172.21.15.43:0/709863854 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f19fc0a3db0 0x7f19fc0a6270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1a080094f0 tx=0x7f1a08009680 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:00.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.761+0000 7f19fb7fe640 1 -- 172.21.15.43:0/4227891663 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c045020 con 0x563317514e90 2024-04-21T08:39:00.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.761+0000 7f19fb7fe640 1 -- 172.21.15.43:0/4227891663 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a0c02c7e0 con 0x563317514e90 2024-04-21T08:39:00.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.761+0000 7f19fb7fe640 1 -- 172.21.15.43:0/4227891663 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c0055c0 con 0x563317514e90 2024-04-21T08:39:00.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.761+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/4227891663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563317514e90 msgr2=0x563317517280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.761+0000 7f1a1a3d5740 1 --2- 172.21.15.43:0/4227891663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563317514e90 0x563317517280 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009900 tx=0x7f1a0c005bf0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.762+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/4227891663 shutdown_connections 2024-04-21T08:39:00.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.762+0000 7f1a1a3d5740 1 --2- 172.21.15.43:0/4227891663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563317514e90 0x563317517280 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.762+0000 7f1a1a3d5740 1 --2- 172.21.15.43:0/4227891663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563317507bd0 0x563317514950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.762+0000 7f1a1a3d5740 1 --2- 172.21.15.43:0/4227891663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633175072b0 0x563317507690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.762+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/4227891663 >> 172.21.15.43:0/4227891663 conn(0x563317501cb0 msgr2=0x563317502090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:00.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.762+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/4227891663 shutdown_connections 2024-04-21T08:39:00.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.762+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/4227891663 wait complete. 2024-04-21T08:39:00.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.762+0000 7f1a1a3d5740 1 -- start start 2024-04-21T08:39:00.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.762+0000 7f1a1a3d5740 1 Processor -- start 2024-04-21T08:39:00.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.762+0000 7f1a1a3d5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5633175072b0 0x563317511ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.762+0000 7f1a1a3d5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563317507bd0 0x563317512400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.762+0000 7f1a1a3d5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563317514e90 0x563317512940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.762+0000 7f1a1a3d5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563317517bf0 con 0x563317514e90 2024-04-21T08:39:00.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.762+0000 7f1a1a3d5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5633174252f0 con 0x563317507bd0 2024-04-21T08:39:00.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.762+0000 7f1a1a3d5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563317517d70 con 0x5633175072b0 2024-04-21T08:39:00.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.763+0000 7f1a1a3d5740 10 client.0 _ll_register_callbacks cb 0x563317432580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:00.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.763+0000 7f1a15ca1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563317514e90 0x563317512940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.763+0000 7f1a15ca1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563317514e90 0x563317512940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38818/0 (socket says 172.21.15.43:38818) 2024-04-21T08:39:00.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.763+0000 7f1a15ca1640 1 -- 172.21.15.43:0/294473026 learned_addr learned my addr 172.21.15.43:0/294473026 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:00.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.763+0000 7f1a154a0640 1 --2- 172.21.15.43:0/294473026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5633175072b0 0x563317511ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.763+0000 7f1a14c9f640 1 --2- 172.21.15.43:0/294473026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563317507bd0 0x563317512400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.763+0000 7f1a15ca1640 1 -- 172.21.15.43:0/294473026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5633175072b0 msgr2=0x563317511ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:00.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.763+0000 7f1a15ca1640 1 --2- 172.21.15.43:0/294473026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5633175072b0 0x563317511ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.763+0000 7f1a15ca1640 1 -- 172.21.15.43:0/294473026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563317507bd0 msgr2=0x563317512400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:00.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.763+0000 7f1a15ca1640 1 --2- 172.21.15.43:0/294473026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563317507bd0 0x563317512400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.763+0000 7f1a15ca1640 1 -- 172.21.15.43:0/294473026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56331717e3f0 con 0x563317514e90 2024-04-21T08:39:00.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.763+0000 7f1a15ca1640 1 --2- 172.21.15.43:0/294473026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563317514e90 0x563317512940 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800cf10 tx=0x7f1a08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490ddaa6b4e5b929 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:00.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.763+0000 7f19f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:00.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.763+0000 7f19f8ff9640 1 -- 172.21.15.43:0/294473026 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08004b20 con 0x563317514e90 2024-04-21T08:39:00.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.763+0000 7f19f8ff9640 1 -- 172.21.15.43:0/294473026 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f1a08004cc0 con 0x563317514e90 2024-04-21T08:39:00.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.763+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/294473026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a0c02c7e0 con 0x563317514e90 2024-04-21T08:39:00.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.764+0000 7f19f8ff9640 1 -- 172.21.15.43:0/294473026 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1a08005020 con 0x563317514e90 2024-04-21T08:39:00.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.764+0000 7f1a1a3d5740 10 client.15774 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:00.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.764+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/294473026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56331713e0f0 con 0x563317514e90 2024-04-21T08:39:00.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.764+0000 7f19f8ff9640 1 -- 172.21.15.43:0/294473026 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a0c02c7e0 con 0x563317514e90 2024-04-21T08:39:00.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.764+0000 7f19f8ff9640 1 -- 172.21.15.43:0/294473026 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1a0800f040 con 0x563317514e90 2024-04-21T08:39:00.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.764+0000 7f1a1a3d5740 10 client.15774 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:00.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.764+0000 7f1a1a3d5740 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:00.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.764+0000 7f1a1a3d5740 20 client.15774 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:00.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.764+0000 7f1a1a3d5740 1 --2- 172.21.15.43:0/294473026 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563317524ee0 0x563317527310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.764+0000 7f1a1a3d5740 4 client.15774 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:00.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.764+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/294473026 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563317479c00 con 0x563317524ee0 2024-04-21T08:39:00.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.764+0000 7f1a154a0640 1 --2- 172.21.15.43:0/294473026 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563317524ee0 0x563317527310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.765+0000 7f1a154a0640 1 --2- 172.21.15.43:0/294473026 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563317524ee0 0x563317527310 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff71018baa78eea0 server_cookie=d2b5735d427f9a27 in_seq=0 out_seq=0 2024-04-21T08:39:00.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.765+0000 7f19f8ff9640 10 client.15774 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:00.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.765+0000 7f19f8ff9640 1 -- 172.21.15.43:0/294473026 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563317479c00 con 0x563317524ee0 2024-04-21T08:39:00.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.765+0000 7f19f8ff9640 10 client.15774 handle_command_reply: tid=0 2024-04-21T08:39:00.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.765+0000 7f1a1a3d5740 1 client.15774 shutdown 2024-04-21T08:39:00.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.765+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/294473026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563317514e90 msgr2=0x563317512940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.765+0000 7f1a1a3d5740 1 --2- 172.21.15.43:0/294473026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563317514e90 0x563317512940 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800cf10 tx=0x7f1a08004a40 comp rx=0 tx=0).stop 2024-04-21T08:39:00.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.765+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/294473026 shutdown_connections 2024-04-21T08:39:00.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.765+0000 7f1a1a3d5740 1 --2- 172.21.15.43:0/294473026 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563317524ee0 0x563317527310 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.765+0000 7f1a1a3d5740 1 --2- 172.21.15.43:0/294473026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563317514e90 0x563317512940 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.765+0000 7f19f8ff9640 0 client.15774 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:00.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.765+0000 7f1a1a3d5740 1 --2- 172.21.15.43:0/294473026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563317507bd0 0x563317512400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.765+0000 7f1a1a3d5740 1 --2- 172.21.15.43:0/294473026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5633175072b0 0x563317511ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.765+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/294473026 >> 172.21.15.43:0/294473026 conn(0x563317501cb0 msgr2=0x563317503a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:00.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.766+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/294473026 shutdown_connections 2024-04-21T08:39:00.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.766+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/294473026 wait complete. 2024-04-21T08:39:00.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.766+0000 7f1a1a3d5740 20 client.15774 trim_cache size 0 max 16384 2024-04-21T08:39:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.784+0000 7f1a1a3d5740 1 Processor -- start 2024-04-21T08:39:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.784+0000 7f1a1a3d5740 1 -- start start 2024-04-21T08:39:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.784+0000 7f1a1a3d5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5633175072b0 0x5633175145f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.784+0000 7f1a1a3d5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563317507bd0 0x563317510e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.784+0000 7f1a1a3d5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563317514e90 0x563317520af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.784+0000 7f1a1a3d5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a101ad160 con 0x563317514e90 2024-04-21T08:39:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.784+0000 7f1a1a3d5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a101aeca0 con 0x563317507bd0 2024-04-21T08:39:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.784+0000 7f1a1a3d5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1a101ad830 con 0x5633175072b0 2024-04-21T08:39:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.784+0000 7f1a15ca1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563317514e90 0x563317520af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.784+0000 7f1a15ca1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563317514e90 0x563317520af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38826/0 (socket says 172.21.15.43:38826) 2024-04-21T08:39:00.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.784+0000 7f1a15ca1640 1 -- 172.21.15.43:0/1955176037 learned_addr learned my addr 172.21.15.43:0/1955176037 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:00.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.784+0000 7f1a154a0640 1 --2- 172.21.15.43:0/1955176037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5633175072b0 0x5633175145f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.784+0000 7f1a14c9f640 1 --2- 172.21.15.43:0/1955176037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563317507bd0 0x563317510e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.784+0000 7f1a15ca1640 1 -- 172.21.15.43:0/1955176037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5633175072b0 msgr2=0x5633175145f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:00.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.784+0000 7f1a15ca1640 1 --2- 172.21.15.43:0/1955176037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5633175072b0 0x5633175145f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.784+0000 7f1a15ca1640 1 -- 172.21.15.43:0/1955176037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563317507bd0 msgr2=0x563317510e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:00.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.784+0000 7f1a15ca1640 1 --2- 172.21.15.43:0/1955176037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563317507bd0 0x563317510e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.784+0000 7f1a15ca1640 1 -- 172.21.15.43:0/1955176037 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563317517280 con 0x563317514e90 2024-04-21T08:39:00.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.784+0000 7f1a14c9f640 1 --2- 172.21.15.43:0/1955176037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563317507bd0 0x563317510e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:00.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.785+0000 7f1a15ca1640 1 --2- 172.21.15.43:0/1955176037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563317514e90 0x563317520af0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1a08029630 tx=0x7f1a08013590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf841fb2987870a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:00.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.785+0000 7f19faffd640 1 -- 172.21.15.43:0/1955176037 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0803a020 con 0x563317514e90 2024-04-21T08:39:00.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.785+0000 7f19faffd640 1 -- 172.21.15.43:0/1955176037 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a08025070 con 0x563317514e90 2024-04-21T08:39:00.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.785+0000 7f19faffd640 1 -- 172.21.15.43:0/1955176037 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08013280 con 0x563317514e90 2024-04-21T08:39:00.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.785+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/1955176037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563317514e90 msgr2=0x563317520af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.785+0000 7f1a1a3d5740 1 --2- 172.21.15.43:0/1955176037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563317514e90 0x563317520af0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1a08029630 tx=0x7f1a08013590 comp rx=0 tx=0).stop 2024-04-21T08:39:00.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.785+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/1955176037 shutdown_connections 2024-04-21T08:39:00.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.785+0000 7f1a1a3d5740 1 --2- 172.21.15.43:0/1955176037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563317514e90 0x563317520af0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.785+0000 7f1a1a3d5740 1 --2- 172.21.15.43:0/1955176037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563317507bd0 0x563317510e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.785+0000 7f1a1a3d5740 1 --2- 172.21.15.43:0/1955176037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5633175072b0 0x5633175145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.785+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/1955176037 >> 172.21.15.43:0/1955176037 conn(0x563317501cb0 msgr2=0x563317500d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:00.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.786+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/1955176037 shutdown_connections 2024-04-21T08:39:00.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.786+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/1955176037 wait complete. 2024-04-21T08:39:00.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.786+0000 7f1a1a3d5740 1 -- start start 2024-04-21T08:39:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.786+0000 7f1a1a3d5740 1 Processor -- start 2024-04-21T08:39:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.786+0000 7f1a1a3d5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633175072b0 0x56331750eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.786+0000 7f1a1a3d5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563317507bd0 0x56331750f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.786+0000 7f1a1a3d5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563317514e90 0x56331758a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.786+0000 7f1a1a3d5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a101ae5d0 con 0x5633175072b0 2024-04-21T08:39:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.786+0000 7f1a1a3d5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a101adf00 con 0x563317514e90 2024-04-21T08:39:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.786+0000 7f1a1a3d5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1a101ab140 con 0x563317507bd0 2024-04-21T08:39:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.786+0000 7f1a1a3d5740 10 client.0 _ll_register_callbacks cb 0x7f1a080136e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.786+0000 7f1a154a0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633175072b0 0x56331750eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.786+0000 7f1a154a0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633175072b0 0x56331750eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38838/0 (socket says 172.21.15.43:38838) 2024-04-21T08:39:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.787+0000 7f1a154a0640 1 -- 172.21.15.43:0/158343991 learned_addr learned my addr 172.21.15.43:0/158343991 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.787+0000 7f1a14c9f640 1 --2- 172.21.15.43:0/158343991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563317507bd0 0x56331750f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.787+0000 7f1a15ca1640 1 --2- 172.21.15.43:0/158343991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563317514e90 0x56331758a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.787+0000 7f1a154a0640 1 -- 172.21.15.43:0/158343991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563317507bd0 msgr2=0x56331750f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.787+0000 7f1a154a0640 1 --2- 172.21.15.43:0/158343991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563317507bd0 0x56331750f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.787+0000 7f1a154a0640 1 -- 172.21.15.43:0/158343991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563317514e90 msgr2=0x56331758a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.787+0000 7f1a154a0640 1 --2- 172.21.15.43:0/158343991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563317514e90 0x56331758a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.787+0000 7f1a154a0640 1 -- 172.21.15.43:0/158343991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56331750fcd0 con 0x5633175072b0 2024-04-21T08:39:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.787+0000 7f1a154a0640 1 --2- 172.21.15.43:0/158343991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633175072b0 0x56331750eec0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1a04099550 tx=0x7f1a0406e5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1feace9261cf04d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.787+0000 7f19f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.787+0000 7f19f8ff9640 1 -- 172.21.15.43:0/158343991 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0409ae80 con 0x5633175072b0 2024-04-21T08:39:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.787+0000 7f19f8ff9640 1 -- 172.21.15.43:0/158343991 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f1a0408e040 con 0x5633175072b0 2024-04-21T08:39:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.787+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/158343991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a08025070 con 0x5633175072b0 2024-04-21T08:39:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.787+0000 7f19f8ff9640 1 -- 172.21.15.43:0/158343991 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1a04010a90 con 0x5633175072b0 2024-04-21T08:39:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.787+0000 7f1a1a3d5740 10 client.15780 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.787+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/158343991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56331758b9e0 con 0x5633175072b0 2024-04-21T08:39:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.788+0000 7f19f8ff9640 1 -- 172.21.15.43:0/158343991 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a08025070 con 0x5633175072b0 2024-04-21T08:39:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.788+0000 7f19f8ff9640 1 -- 172.21.15.43:0/158343991 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1a040ce5e0 con 0x5633175072b0 2024-04-21T08:39:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.788+0000 7f1a1a3d5740 10 client.15780 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.788+0000 7f1a1a3d5740 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.788+0000 7f1a1a3d5740 20 client.15780 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.788+0000 7f1a1a3d5740 1 --2- 172.21.15.43:0/158343991 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56331758c090 0x56331758e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.788+0000 7f1a1a3d5740 4 client.15780 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:39:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.788+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/158343991 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563317473a70 con 0x56331758c090 2024-04-21T08:39:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.788+0000 7f1a14c9f640 1 --2- 172.21.15.43:0/158343991 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56331758c090 0x56331758e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.788+0000 7f19f8ff9640 10 client.15780 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.788+0000 7f1a14c9f640 1 --2- 172.21.15.43:0/158343991 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56331758c090 0x56331758e4b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1af05e1ac9a36217 server_cookie=cff4eef6f318de33 in_seq=0 out_seq=0 2024-04-21T08:39:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f19f8ff9640 1 -- 172.21.15.43:0/158343991 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563317473a70 con 0x56331758c090 2024-04-21T08:39:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f19f8ff9640 10 client.15780 handle_command_reply: tid=0 2024-04-21T08:39:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f1a1a3d5740 1 client.15780 shutdown 2024-04-21T08:39:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/158343991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633175072b0 msgr2=0x56331750eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f1a1a3d5740 1 --2- 172.21.15.43:0/158343991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633175072b0 0x56331750eec0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1a04099550 tx=0x7f1a0406e5c0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/158343991 shutdown_connections 2024-04-21T08:39:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f1a1a3d5740 1 --2- 172.21.15.43:0/158343991 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56331758c090 0x56331758e4b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f1a1a3d5740 1 --2- 172.21.15.43:0/158343991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563317514e90 0x56331758a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f1a1a3d5740 1 --2- 172.21.15.43:0/158343991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563317507bd0 0x56331750f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f1a1a3d5740 1 --2- 172.21.15.43:0/158343991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5633175072b0 0x56331750eec0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/158343991 >> 172.21.15.43:0/158343991 conn(0x563317501cb0 msgr2=0x563317511e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f19f8ff9640 0 client.15780 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/158343991 shutdown_connections 2024-04-21T08:39:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f1a1a3d5740 1 -- 172.21.15.43:0/158343991 wait complete. 2024-04-21T08:39:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f1a1a3d5740 20 client.15780 trim_cache size 0 max 16384 2024-04-21T08:39:00.796 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:39:00.796 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:39:00.796 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-21T08:39:00.796 INFO:teuthology.orchestra.run.smithi043.stdout: "d2c011c5-55be-418f-b535-a24127a92cbf": { 2024-04-21T08:39:00.797 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-21T08:39:00.797 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "d2c011c5-55be-418f-b535-a24127a92cbf", 2024-04-21T08:39:00.797 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-21T08:39:00.797 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:39:00.797 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:39:00.797 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:39:00.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f1a17704640 1 -- 172.21.15.43:0/709863854 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f19fc0a3db0 msgr2=0x7f19fc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f1a17704640 1 --2- 172.21.15.43:0/709863854 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f19fc0a3db0 0x7f19fc0a6270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1a080094f0 tx=0x7f1a08009680 comp rx=0 tx=0).stop 2024-04-21T08:39:00.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f1a17704640 1 -- 172.21.15.43:0/709863854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a10101080 msgr2=0x7f1a101994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f1a17704640 1 --2- 172.21.15.43:0/709863854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a10101080 0x7f1a101994f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400e9f0 tx=0x7f1a0400eeb0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f1a17704640 1 -- 172.21.15.43:0/709863854 shutdown_connections 2024-04-21T08:39:00.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f1a17704640 1 --2- 172.21.15.43:0/709863854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a1019a010 0x7f1a101a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f1a17704640 1 --2- 172.21.15.43:0/709863854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a10106150 0x7f1a10199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f1a17704640 1 --2- 172.21.15.43:0/709863854 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f19fc0a3db0 0x7f19fc0a6270 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f1a17704640 1 --2- 172.21.15.43:0/709863854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a10101080 0x7f1a101994f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.789+0000 7f1a17704640 1 -- 172.21.15.43:0/709863854 >> 172.21.15.43:0/709863854 conn(0x7f1a10078000 msgr2=0x7f1a100ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:00.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.790+0000 7f1a17704640 1 -- 172.21.15.43:0/709863854 shutdown_connections 2024-04-21T08:39:00.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.790+0000 7f1a17704640 1 -- 172.21.15.43:0/709863854 wait complete. 2024-04-21T08:39:00.800 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: Updating smithi112:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:39:00.800 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: Updating smithi043:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:39:00.800 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: Updating smithi175:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:39:00.800 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:39:00.800 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:39:00.800 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:39:00.800 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:39:00.800 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: Updating smithi175:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:39:00.800 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: Updating smithi112:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:39:00.800 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: Updating smithi043:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:39:00.800 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/381264264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:39:00.801 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-21T08:39:00.801 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:39:00.801 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:39:00.801 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:39:00.801 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:39:00.801 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:39:00.801 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:39:00.801 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:39:00.801 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:39:00.801 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:00.801 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:39:00.801 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:39:00.801 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/325364908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:39:00.802 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:00 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/340692463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:00.802 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: Updating smithi112:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:39:00.802 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: Updating smithi043:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:39:00.802 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: Updating smithi175:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:39:00.802 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:39:00.802 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:39:00.802 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:39:00.802 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:39:00.802 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: Updating smithi175:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:39:00.802 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: Updating smithi112:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:39:00.803 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: Updating smithi043:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:39:00.803 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/381264264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:39:00.803 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-21T08:39:00.803 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:39:00.803 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:39:00.803 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:39:00.803 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:39:00.803 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:39:00.803 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:39:00.803 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:39:00.803 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:39:00.803 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:00.804 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:39:00.804 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:39:00.804 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/325364908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:39:00.804 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:00 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/340692463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:00.804 INFO:tasks.cephfs.filesystem:scrub status for tag:d2c011c5-55be-418f-b535-a24127a92cbf - {'path': '/', 'tag': 'd2c011c5-55be-418f-b535-a24127a92cbf', 'options': 'recursive,force'} 2024-04-21T08:39:00.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.828+0000 7f4f051ce640 1 -- 172.21.15.43:0/2297740344 --> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f4f00102b70 con 0x7f4ee80a3e60 2024-04-21T08:39:00.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.891+0000 7f4ee7fff640 1 -- 172.21.15.43:0/2297740344 <== mgr.15546 v2:172.21.15.43:6800/1482273906 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4f00102b70 con 0x7f4ee80a3e60 2024-04-21T08:39:00.891 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule set for path / 2024-04-21T08:39:00.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.893+0000 7f4f051ce640 1 -- 172.21.15.43:0/2297740344 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4ee80a3e60 msgr2=0x7f4ee80a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.893+0000 7f4f051ce640 1 --2- 172.21.15.43:0/2297740344 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4ee80a3e60 0x7f4ee80a6320 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009fc0 tx=0x7f4ef0005f90 comp rx=0 tx=0).stop 2024-04-21T08:39:00.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.893+0000 7f4f051ce640 1 -- 172.21.15.43:0/2297740344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f00107290 msgr2=0x7f4f00198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.893+0000 7f4f051ce640 1 --2- 172.21.15.43:0/2297740344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f00107290 0x7f4f00198c00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4000c00 tx=0x7f4ef402c660 comp rx=0 tx=0).stop 2024-04-21T08:39:00.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.894+0000 7f4f051ce640 1 -- 172.21.15.43:0/2297740344 shutdown_connections 2024-04-21T08:39:00.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.894+0000 7f4f051ce640 1 --2- 172.21.15.43:0/2297740344 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4ee80a3e60 0x7f4ee80a6320 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.894+0000 7f4f051ce640 1 --2- 172.21.15.43:0/2297740344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f0019cf80 0x7f4f0019d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.894+0000 7f4f051ce640 1 --2- 172.21.15.43:0/2297740344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f00107290 0x7f4f00198c00 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.894+0000 7f4f051ce640 1 --2- 172.21.15.43:0/2297740344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f001021c0 0x7f4f001986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.894+0000 7f4f051ce640 1 -- 172.21.15.43:0/2297740344 >> 172.21.15.43:0/2297740344 conn(0x7f4f000fda20 msgr2=0x7f4f000ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:00.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.894+0000 7f4f051ce640 1 -- 172.21.15.43:0/2297740344 shutdown_connections 2024-04-21T08:39:00.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:00.894+0000 7f4f051ce640 1 -- 172.21.15.43:0/2297740344 wait complete. 2024-04-21T08:39:00.904 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-21T08:39:01.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.050+0000 7f89a7c8c640 1 -- 172.21.15.43:0/2739436789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a0106170 msgr2=0x7f89a010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.050+0000 7f89a7c8c640 1 --2- 172.21.15.43:0/2739436789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a0106170 0x7f89a010dd40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f899c009ca0 tx=0x7f899c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.050+0000 7f89a7c8c640 1 -- 172.21.15.43:0/2739436789 shutdown_connections 2024-04-21T08:39:01.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.050+0000 7f89a7c8c640 1 --2- 172.21.15.43:0/2739436789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a0106170 0x7f89a010dd40 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.050+0000 7f89a7c8c640 1 --2- 172.21.15.43:0/2739436789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89a0101aa0 0x7f89a0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.050+0000 7f89a7c8c640 1 --2- 172.21.15.43:0/2739436789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89a01010f0 0x7f89a01014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.050+0000 7f89a7c8c640 1 -- 172.21.15.43:0/2739436789 >> 172.21.15.43:0/2739436789 conn(0x7f89a0077fd0 msgr2=0x7f89a00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.050+0000 7f89a7c8c640 1 -- 172.21.15.43:0/2739436789 shutdown_connections 2024-04-21T08:39:01.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.050+0000 7f89a7c8c640 1 -- 172.21.15.43:0/2739436789 wait complete. 2024-04-21T08:39:01.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.050+0000 7f89a7c8c640 1 Processor -- start 2024-04-21T08:39:01.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.050+0000 7f89a7c8c640 1 -- start start 2024-04-21T08:39:01.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.050+0000 7f89a7c8c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a01010f0 0x7f89a01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.050+0000 7f89a7c8c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89a0106170 0x7f89a0198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.050+0000 7f89a7c8c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89a019cf40 0x7f89a019d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.050+0000 7f89a7c8c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f89a01121f0 con 0x7f89a01010f0 2024-04-21T08:39:01.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.050+0000 7f89a7c8c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89a0112070 con 0x7f89a0106170 2024-04-21T08:39:01.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.051+0000 7f89a7c8c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f89a0112370 con 0x7f89a019cf40 2024-04-21T08:39:01.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.051+0000 7f89a5a28640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a01010f0 0x7f89a01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.051+0000 7f89a5a28640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a01010f0 0x7f89a01986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38862/0 (socket says 172.21.15.43:38862) 2024-04-21T08:39:01.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.051+0000 7f89a5a28640 1 -- 172.21.15.43:0/3259898195 learned_addr learned my addr 172.21.15.43:0/3259898195 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:01.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.051+0000 7f89a5227640 1 --2- 172.21.15.43:0/3259898195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89a0106170 0x7f89a0198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.051+0000 7f89a6229640 1 --2- 172.21.15.43:0/3259898195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89a019cf40 0x7f89a019d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.051+0000 7f89a5a28640 1 -- 172.21.15.43:0/3259898195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89a019cf40 msgr2=0x7f89a019d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.051+0000 7f89a5a28640 1 --2- 172.21.15.43:0/3259898195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89a019cf40 0x7f89a019d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.051+0000 7f89a5a28640 1 -- 172.21.15.43:0/3259898195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89a0106170 msgr2=0x7f89a0198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.051+0000 7f89a5a28640 1 --2- 172.21.15.43:0/3259898195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89a0106170 0x7f89a0198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.051+0000 7f89a5a28640 1 -- 172.21.15.43:0/3259898195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a019da30 con 0x7f89a01010f0 2024-04-21T08:39:01.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.051+0000 7f89a5a28640 1 --2- 172.21.15.43:0/3259898195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a01010f0 0x7f89a01986d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f899800d950 tx=0x7f899800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.051+0000 7f8996ffd640 1 -- 172.21.15.43:0/3259898195 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8998017070 con 0x7f89a01010f0 2024-04-21T08:39:01.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.051+0000 7f8996ffd640 1 -- 172.21.15.43:0/3259898195 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8998009c30 con 0x7f89a01010f0 2024-04-21T08:39:01.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.051+0000 7f89a7c8c640 1 -- 172.21.15.43:0/3259898195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a019dd20 con 0x7f89a01010f0 2024-04-21T08:39:01.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.051+0000 7f8996ffd640 1 -- 172.21.15.43:0/3259898195 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899800f620 con 0x7f89a01010f0 2024-04-21T08:39:01.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.051+0000 7f89a7c8c640 1 -- 172.21.15.43:0/3259898195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a01a5560 con 0x7f89a01010f0 2024-04-21T08:39:01.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.052+0000 7f89a7c8c640 1 -- 172.21.15.43:0/3259898195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89a004fce0 con 0x7f89a01010f0 2024-04-21T08:39:01.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.054+0000 7f8996ffd640 1 -- 172.21.15.43:0/3259898195 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f8998010460 con 0x7f89a01010f0 2024-04-21T08:39:01.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.055+0000 7f8996ffd640 1 --2- 172.21.15.43:0/3259898195 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f898c0a3ee0 0x7f898c0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.055+0000 7f89a5227640 1 --2- 172.21.15.43:0/3259898195 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f898c0a3ee0 0x7f898c0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.055+0000 7f8996ffd640 1 -- 172.21.15.43:0/3259898195 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f89980d0a20 con 0x7f89a01010f0 2024-04-21T08:39:01.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.055+0000 7f89a5227640 1 --2- 172.21.15.43:0/3259898195 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f898c0a3ee0 0x7f898c0a63a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f89a01009b0 tx=0x7f8990005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.058+0000 7f8996ffd640 1 -- 172.21.15.43:0/3259898195 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f899809cba0 con 0x7f89a01010f0 2024-04-21T08:39:01.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.272+0000 7f89a7c8c640 1 -- 172.21.15.43:0/3259898195 --> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f89a0101aa0 con 0x7f898c0a3ee0 2024-04-21T08:39:01.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.330+0000 7f8996ffd640 1 -- 172.21.15.43:0/3259898195 <== mgr.15546 v2:172.21.15.43:6800/1482273906 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f89a0101aa0 con 0x7f898c0a3ee0 2024-04-21T08:39:01.329 INFO:teuthology.orchestra.run.smithi043.stdout:Retention added to path / 2024-04-21T08:39:01.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.332+0000 7f89a7c8c640 1 -- 172.21.15.43:0/3259898195 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f898c0a3ee0 msgr2=0x7f898c0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.332+0000 7f89a7c8c640 1 --2- 172.21.15.43:0/3259898195 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f898c0a3ee0 0x7f898c0a63a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f89a01009b0 tx=0x7f8990005f90 comp rx=0 tx=0).stop 2024-04-21T08:39:01.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.332+0000 7f89a7c8c640 1 -- 172.21.15.43:0/3259898195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a01010f0 msgr2=0x7f89a01986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.332+0000 7f89a7c8c640 1 --2- 172.21.15.43:0/3259898195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a01010f0 0x7f89a01986d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f899800d950 tx=0x7f899800de10 comp rx=0 tx=0).stop 2024-04-21T08:39:01.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.332+0000 7f89a7c8c640 1 -- 172.21.15.43:0/3259898195 shutdown_connections 2024-04-21T08:39:01.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.332+0000 7f89a7c8c640 1 --2- 172.21.15.43:0/3259898195 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f898c0a3ee0 0x7f898c0a63a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.332+0000 7f89a7c8c640 1 --2- 172.21.15.43:0/3259898195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89a019cf40 0x7f89a019d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.332+0000 7f89a7c8c640 1 --2- 172.21.15.43:0/3259898195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89a0106170 0x7f89a0198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.332+0000 7f89a7c8c640 1 --2- 172.21.15.43:0/3259898195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a01010f0 0x7f89a01986d0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.332+0000 7f89a7c8c640 1 -- 172.21.15.43:0/3259898195 >> 172.21.15.43:0/3259898195 conn(0x7f89a0077fd0 msgr2=0x7f89a00ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.332+0000 7f89a7c8c640 1 -- 172.21.15.43:0/3259898195 shutdown_connections 2024-04-21T08:39:01.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.332+0000 7f89a7c8c640 1 -- 172.21.15.43:0/3259898195 wait complete. 2024-04-21T08:39:01.342 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-21T08:39:01.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.446+0000 7fb0b67fd640 1 -- 172.21.15.43:0/1284935339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0b0107290 msgr2=0x7fb0b010ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.446+0000 7fb0b67fd640 1 --2- 172.21.15.43:0/1284935339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0b0107290 0x7fb0b010ee60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009ca0 tx=0x7fb0a802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.446+0000 7fb0b67fd640 1 -- 172.21.15.43:0/1284935339 shutdown_connections 2024-04-21T08:39:01.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.446+0000 7fb0b67fd640 1 --2- 172.21.15.43:0/1284935339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0b0107290 0x7fb0b010ee60 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.446+0000 7fb0b67fd640 1 --2- 172.21.15.43:0/1284935339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0b0102b70 0x7fb0b0106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.446+0000 7fb0b67fd640 1 --2- 172.21.15.43:0/1284935339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0b01021c0 0x7fb0b01025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.446+0000 7fb0b67fd640 1 -- 172.21.15.43:0/1284935339 >> 172.21.15.43:0/1284935339 conn(0x7fb0b00fda20 msgr2=0x7fb0b00ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.446+0000 7fb0b67fd640 1 -- 172.21.15.43:0/1284935339 shutdown_connections 2024-04-21T08:39:01.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.446+0000 7fb0b67fd640 1 -- 172.21.15.43:0/1284935339 wait complete. 2024-04-21T08:39:01.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.446+0000 7fb0b67fd640 1 Processor -- start 2024-04-21T08:39:01.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.446+0000 7fb0b67fd640 1 -- start start 2024-04-21T08:39:01.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.447+0000 7fb0b67fd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0b01021c0 0x7fb0b0198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.447+0000 7fb0b67fd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0b0107290 0x7fb0b0198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.447+0000 7fb0b67fd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0b019cfb0 0x7fb0b019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.447+0000 7fb0b67fd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b0112200 con 0x7fb0b019cfb0 2024-04-21T08:39:01.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.447+0000 7fb0b67fd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b0112080 con 0x7fb0b01021c0 2024-04-21T08:39:01.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.447+0000 7fb0b67fd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b0112380 con 0x7fb0b0107290 2024-04-21T08:39:01.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.447+0000 7fb0b4d9a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0b019cfb0 0x7fb0b019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.447+0000 7fb0b4d9a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0b019cfb0 0x7fb0b019d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38882/0 (socket says 172.21.15.43:38882) 2024-04-21T08:39:01.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.447+0000 7fb0b4d9a640 1 -- 172.21.15.43:0/4191586435 learned_addr learned my addr 172.21.15.43:0/4191586435 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:01.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.447+0000 7fb0affff640 1 --2- 172.21.15.43:0/4191586435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0b01021c0 0x7fb0b0198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.447+0000 7fb0af7fe640 1 --2- 172.21.15.43:0/4191586435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0b0107290 0x7fb0b0198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.447+0000 7fb0b4d9a640 1 -- 172.21.15.43:0/4191586435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0b0107290 msgr2=0x7fb0b0198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.447+0000 7fb0b4d9a640 1 --2- 172.21.15.43:0/4191586435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0b0107290 0x7fb0b0198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.447+0000 7fb0b4d9a640 1 -- 172.21.15.43:0/4191586435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0b01021c0 msgr2=0x7fb0b0198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.447+0000 7fb0b4d9a640 1 --2- 172.21.15.43:0/4191586435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0b01021c0 0x7fb0b0198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.447+0000 7fb0b4d9a640 1 -- 172.21.15.43:0/4191586435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b019daa0 con 0x7fb0b019cfb0 2024-04-21T08:39:01.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.447+0000 7fb0b4d9a640 1 --2- 172.21.15.43:0/4191586435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0b019cfb0 0x7fb0b019d460 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8005e30 tx=0x7fb0a8034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.447+0000 7fb0ad7fa640 1 -- 172.21.15.43:0/4191586435 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8039040 con 0x7fb0b019cfb0 2024-04-21T08:39:01.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.447+0000 7fb0ad7fa640 1 -- 172.21.15.43:0/4191586435 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a8044070 con 0x7fb0b019cfb0 2024-04-21T08:39:01.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.448+0000 7fb0b67fd640 1 -- 172.21.15.43:0/4191586435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b019dd30 con 0x7fb0b019cfb0 2024-04-21T08:39:01.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.448+0000 7fb0ad7fa640 1 -- 172.21.15.43:0/4191586435 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8003e60 con 0x7fb0b019cfb0 2024-04-21T08:39:01.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.448+0000 7fb0b67fd640 1 -- 172.21.15.43:0/4191586435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b01a54f0 con 0x7fb0b019cfb0 2024-04-21T08:39:01.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.449+0000 7fb0b67fd640 1 -- 172.21.15.43:0/4191586435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0b004fce0 con 0x7fb0b019cfb0 2024-04-21T08:39:01.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.450+0000 7fb0ad7fa640 1 -- 172.21.15.43:0/4191586435 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fb0a80324c0 con 0x7fb0b019cfb0 2024-04-21T08:39:01.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.450+0000 7fb0ad7fa640 1 --2- 172.21.15.43:0/4191586435 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb0980a3e60 0x7fb0980a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.451+0000 7fb0affff640 1 --2- 172.21.15.43:0/4191586435 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb0980a3e60 0x7fb0980a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.451+0000 7fb0ad7fa640 1 -- 172.21.15.43:0/4191586435 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fb0a80f3a40 con 0x7fb0b019cfb0 2024-04-21T08:39:01.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.451+0000 7fb0affff640 1 --2- 172.21.15.43:0/4191586435 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb0980a3e60 0x7fb0980a6320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0005fd0 tx=0x7fb0a000b590 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.454+0000 7fb0ad7fa640 1 -- 172.21.15.43:0/4191586435 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb0a80bfcb0 con 0x7fb0b019cfb0 2024-04-21T08:39:01.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.664+0000 7fb0b67fd640 1 -- 172.21.15.43:0/4191586435 --> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fb0b0102b70 con 0x7fb0980a3e60 2024-04-21T08:39:01.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.672+0000 7fb0ad7fa640 1 -- 172.21.15.43:0/4191586435 <== mgr.15546 v2:172.21.15.43:6800/1482273906 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fb0b0102b70 con 0x7fb0980a3e60 2024-04-21T08:39:01.671 INFO:teuthology.orchestra.run.smithi043.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-21T00:00:00", "created": "2024-04-21T08:39:00", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-21T08:39:01.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.674+0000 7fb0b67fd640 1 -- 172.21.15.43:0/4191586435 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb0980a3e60 msgr2=0x7fb0980a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.674+0000 7fb0b67fd640 1 --2- 172.21.15.43:0/4191586435 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb0980a3e60 0x7fb0980a6320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0005fd0 tx=0x7fb0a000b590 comp rx=0 tx=0).stop 2024-04-21T08:39:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.674+0000 7fb0b67fd640 1 -- 172.21.15.43:0/4191586435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0b019cfb0 msgr2=0x7fb0b019d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.674+0000 7fb0b67fd640 1 --2- 172.21.15.43:0/4191586435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0b019cfb0 0x7fb0b019d460 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8005e30 tx=0x7fb0a8034760 comp rx=0 tx=0).stop 2024-04-21T08:39:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.674+0000 7fb0b67fd640 1 -- 172.21.15.43:0/4191586435 shutdown_connections 2024-04-21T08:39:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.674+0000 7fb0b67fd640 1 --2- 172.21.15.43:0/4191586435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0b019cfb0 0x7fb0b019d460 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.674+0000 7fb0b67fd640 1 --2- 172.21.15.43:0/4191586435 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb0980a3e60 0x7fb0980a6320 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.674+0000 7fb0b67fd640 1 --2- 172.21.15.43:0/4191586435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0b0107290 0x7fb0b0198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.674+0000 7fb0b67fd640 1 --2- 172.21.15.43:0/4191586435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0b01021c0 0x7fb0b0198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.674+0000 7fb0b67fd640 1 -- 172.21.15.43:0/4191586435 >> 172.21.15.43:0/4191586435 conn(0x7fb0b00fda20 msgr2=0x7fb0b00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.674+0000 7fb0b67fd640 1 -- 172.21.15.43:0/4191586435 shutdown_connections 2024-04-21T08:39:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.674+0000 7fb0b67fd640 1 -- 172.21.15.43:0/4191586435 wait complete. 2024-04-21T08:39:01.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:01 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3293512509' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:39:01.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:01 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:39:01.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:01 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:39:01.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:01 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:39:01.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:01 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:39:01.684 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-21T08:39:01.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:01 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3293512509' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:39:01.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:01 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:39:01.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:01 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:39:01.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:01 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:39:01.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:01 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:39:01.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.789+0000 7fd6f07c3640 1 -- 172.21.15.43:0/1957041154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e8102b50 msgr2=0x7fd6e8106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.789+0000 7fd6f07c3640 1 --2- 172.21.15.43:0/1957041154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e8102b50 0x7fd6e8106b90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009900 tx=0x7fd6d802bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:01.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.789+0000 7fd6f07c3640 1 -- 172.21.15.43:0/1957041154 shutdown_connections 2024-04-21T08:39:01.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.789+0000 7fd6f07c3640 1 --2- 172.21.15.43:0/1957041154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6e8107270 0x7fd6e810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.789+0000 7fd6f07c3640 1 --2- 172.21.15.43:0/1957041154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e8102b50 0x7fd6e8106b90 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.789+0000 7fd6f07c3640 1 --2- 172.21.15.43:0/1957041154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6e81021a0 0x7fd6e8102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.789+0000 7fd6f07c3640 1 -- 172.21.15.43:0/1957041154 >> 172.21.15.43:0/1957041154 conn(0x7fd6e80793e0 msgr2=0x7fd6e80797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.789+0000 7fd6f07c3640 1 -- 172.21.15.43:0/1957041154 shutdown_connections 2024-04-21T08:39:01.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.789+0000 7fd6f07c3640 1 -- 172.21.15.43:0/1957041154 wait complete. 2024-04-21T08:39:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.790+0000 7fd6f07c3640 1 Processor -- start 2024-04-21T08:39:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.790+0000 7fd6f07c3640 1 -- start start 2024-04-21T08:39:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.790+0000 7fd6f07c3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e81021a0 0x7fd6e8074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.790+0000 7fd6f07c3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6e8107270 0x7fd6e8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.790+0000 7fd6f07c3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6e8074cd0 0x7fd6e81a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.790+0000 7fd6f07c3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e81121b0 con 0x7fd6e81021a0 2024-04-21T08:39:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.790+0000 7fd6f07c3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e8112030 con 0x7fd6e8074cd0 2024-04-21T08:39:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.790+0000 7fd6f07c3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e8112330 con 0x7fd6e8107270 2024-04-21T08:39:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.790+0000 7fd6ee55f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e81021a0 0x7fd6e8074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.790+0000 7fd6ee55f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e81021a0 0x7fd6e8074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38898/0 (socket says 172.21.15.43:38898) 2024-04-21T08:39:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.790+0000 7fd6ee55f640 1 -- 172.21.15.43:0/3600412522 learned_addr learned my addr 172.21.15.43:0/3600412522 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.790+0000 7fd6edd5e640 1 --2- 172.21.15.43:0/3600412522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6e8107270 0x7fd6e8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.790+0000 7fd6eed60640 1 --2- 172.21.15.43:0/3600412522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6e8074cd0 0x7fd6e81a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.790+0000 7fd6ee55f640 1 -- 172.21.15.43:0/3600412522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6e8107270 msgr2=0x7fd6e8074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.790+0000 7fd6ee55f640 1 --2- 172.21.15.43:0/3600412522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6e8107270 0x7fd6e8074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.790+0000 7fd6ee55f640 1 -- 172.21.15.43:0/3600412522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6e8074cd0 msgr2=0x7fd6e81a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.790+0000 7fd6ee55f640 1 --2- 172.21.15.43:0/3600412522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6e8074cd0 0x7fd6e81a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.790+0000 7fd6ee55f640 1 -- 172.21.15.43:0/3600412522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e81a6fe0 con 0x7fd6e81021a0 2024-04-21T08:39:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.791+0000 7fd6ee55f640 1 --2- 172.21.15.43:0/3600412522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e81021a0 0x7fd6e8074250 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400e9f0 tx=0x7fd6e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.791+0000 7fd6df7fe640 1 -- 172.21.15.43:0/3600412522 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e400cd80 con 0x7fd6e81021a0 2024-04-21T08:39:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.791+0000 7fd6df7fe640 1 -- 172.21.15.43:0/3600412522 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6e40054d0 con 0x7fd6e81021a0 2024-04-21T08:39:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.791+0000 7fd6f07c3640 1 -- 172.21.15.43:0/3600412522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e81a72d0 con 0x7fd6e81021a0 2024-04-21T08:39:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.791+0000 7fd6df7fe640 1 -- 172.21.15.43:0/3600412522 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4010690 con 0x7fd6e81021a0 2024-04-21T08:39:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.791+0000 7fd6f07c3640 1 -- 172.21.15.43:0/3600412522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e81a77a0 con 0x7fd6e81021a0 2024-04-21T08:39:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.792+0000 7fd6f07c3640 1 -- 172.21.15.43:0/3600412522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e804fce0 con 0x7fd6e81021a0 2024-04-21T08:39:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.793+0000 7fd6df7fe640 1 -- 172.21.15.43:0/3600412522 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fd6e4004e50 con 0x7fd6e81021a0 2024-04-21T08:39:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.794+0000 7fd6df7fe640 1 --2- 172.21.15.43:0/3600412522 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd6d40a3e90 0x7fd6d40a6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.794+0000 7fd6edd5e640 1 --2- 172.21.15.43:0/3600412522 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd6d40a3e90 0x7fd6d40a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.794+0000 7fd6df7fe640 1 -- 172.21.15.43:0/3600412522 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd6e40d1260 con 0x7fd6e81021a0 2024-04-21T08:39:01.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.795+0000 7fd6edd5e640 1 --2- 172.21.15.43:0/3600412522 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd6d40a3e90 0x7fd6d40a6350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8006010 tx=0x7fd6d802c690 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:01 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3293512509' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:39:01.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:01 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:39:01.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:01 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:39:01.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:01 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:39:01.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:01 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:39:01.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:01.799+0000 7fd6df7fe640 1 -- 172.21.15.43:0/3600412522 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd6e409d3c0 con 0x7fd6e81021a0 2024-04-21T08:39:02.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.009+0000 7fd6f07c3640 1 -- 172.21.15.43:0/3600412522 --> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fd6e810a510 con 0x7fd6d40a3e90 2024-04-21T08:39:02.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.015+0000 7fd6df7fe640 1 -- 172.21.15.43:0/3600412522 <== mgr.15546 v2:172.21.15.43:6800/1482273906 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fd6d8004140 con 0x7fd6d40a3e90 2024-04-21T08:39:02.014 INFO:teuthology.orchestra.run.smithi043.stdout:/ 1m 6m3h 2024-04-21T08:39:02.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.018+0000 7fd6f07c3640 1 -- 172.21.15.43:0/3600412522 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd6d40a3e90 msgr2=0x7fd6d40a6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:02.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.018+0000 7fd6f07c3640 1 --2- 172.21.15.43:0/3600412522 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd6d40a3e90 0x7fd6d40a6350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8006010 tx=0x7fd6d802c690 comp rx=0 tx=0).stop 2024-04-21T08:39:02.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.018+0000 7fd6f07c3640 1 -- 172.21.15.43:0/3600412522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e81021a0 msgr2=0x7fd6e8074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:02.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.018+0000 7fd6f07c3640 1 --2- 172.21.15.43:0/3600412522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e81021a0 0x7fd6e8074250 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400e9f0 tx=0x7fd6e400edf0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.018+0000 7fd6f07c3640 1 -- 172.21.15.43:0/3600412522 shutdown_connections 2024-04-21T08:39:02.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.018+0000 7fd6f07c3640 1 --2- 172.21.15.43:0/3600412522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6e8074cd0 0x7fd6e81a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.018+0000 7fd6f07c3640 1 --2- 172.21.15.43:0/3600412522 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd6d40a3e90 0x7fd6d40a6350 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.018+0000 7fd6f07c3640 1 --2- 172.21.15.43:0/3600412522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6e8107270 0x7fd6e8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.018+0000 7fd6f07c3640 1 --2- 172.21.15.43:0/3600412522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e81021a0 0x7fd6e8074250 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.018+0000 7fd6f07c3640 1 -- 172.21.15.43:0/3600412522 >> 172.21.15.43:0/3600412522 conn(0x7fd6e80793e0 msgr2=0x7fd6e80ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:02.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.018+0000 7fd6f07c3640 1 -- 172.21.15.43:0/3600412522 shutdown_connections 2024-04-21T08:39:02.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.018+0000 7fd6f07c3640 1 -- 172.21.15.43:0/3600412522 wait complete. 2024-04-21T08:39:02.028 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-21T08:39:02.053 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-21T08:39:02.064 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-21T08:39:02.077 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-04-21T08:39:02.077 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-21T08:39:02.096 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-21T08:39:02.115 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:39:02.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.219+0000 7f479bfc7640 1 -- 172.21.15.43:0/925461829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4794105e30 msgr2=0x7f4794106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:02.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.219+0000 7f479bfc7640 1 --2- 172.21.15.43:0/925461829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4794105e30 0x7f4794106210 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f478c009930 tx=0x7f478c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:02.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.219+0000 7f479bfc7640 1 -- 172.21.15.43:0/925461829 shutdown_connections 2024-04-21T08:39:02.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.219+0000 7f479bfc7640 1 --2- 172.21.15.43:0/925461829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4794101b60 0x7f479410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.219+0000 7f479bfc7640 1 --2- 172.21.15.43:0/925461829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47941011a0 0x7f4794101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.219+0000 7f479bfc7640 1 --2- 172.21.15.43:0/925461829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4794105e30 0x7f4794106210 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.219+0000 7f479bfc7640 1 -- 172.21.15.43:0/925461829 >> 172.21.15.43:0/925461829 conn(0x7f4794077e30 msgr2=0x7f4794078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:02.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.219+0000 7f479bfc7640 1 -- 172.21.15.43:0/925461829 shutdown_connections 2024-04-21T08:39:02.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.219+0000 7f479bfc7640 1 -- 172.21.15.43:0/925461829 wait complete. 2024-04-21T08:39:02.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.219+0000 7f479bfc7640 1 Processor -- start 2024-04-21T08:39:02.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.219+0000 7f479bfc7640 1 -- start start 2024-04-21T08:39:02.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.220+0000 7f479bfc7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47941011a0 0x7f4794199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:02.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.220+0000 7f479bfc7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4794101b60 0x7f4794199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:02.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.220+0000 7f479bfc7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4794105e30 0x7f47941a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:02.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.220+0000 7f479bfc7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f47941110e0 con 0x7f47941011a0 2024-04-21T08:39:02.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.220+0000 7f479bfc7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4794110f60 con 0x7f4794105e30 2024-04-21T08:39:02.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.220+0000 7f479bfc7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4794111260 con 0x7f4794101b60 2024-04-21T08:39:02.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.220+0000 7f4799d63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47941011a0 0x7f4794199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:02.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.220+0000 7f4799d63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47941011a0 0x7f4794199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38926/0 (socket says 172.21.15.43:38926) 2024-04-21T08:39:02.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.220+0000 7f4799d63640 1 -- 172.21.15.43:0/1090926062 learned_addr learned my addr 172.21.15.43:0/1090926062 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:02.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.220+0000 7f4799562640 1 --2- 172.21.15.43:0/1090926062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4794101b60 0x7f4794199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:02.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.220+0000 7f4799d63640 1 -- 172.21.15.43:0/1090926062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4794101b60 msgr2=0x7f4794199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:02.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.220+0000 7f4799d63640 1 --2- 172.21.15.43:0/1090926062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4794101b60 0x7f4794199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.220+0000 7f4799d63640 1 -- 172.21.15.43:0/1090926062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4794105e30 msgr2=0x7f47941a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:02.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.220+0000 7f4799d63640 1 --2- 172.21.15.43:0/1090926062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4794105e30 0x7f47941a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.220+0000 7f4799d63640 1 -- 172.21.15.43:0/1090926062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47941a4b40 con 0x7f47941011a0 2024-04-21T08:39:02.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.220+0000 7f4799562640 1 --2- 172.21.15.43:0/1090926062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4794101b60 0x7f4794199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:02.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.220+0000 7f4799d63640 1 --2- 172.21.15.43:0/1090926062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47941011a0 0x7f4794199580 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f478c009230 tx=0x7f478c004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:02.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.220+0000 7f4792ffd640 1 -- 172.21.15.43:0/1090926062 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c043020 con 0x7f47941011a0 2024-04-21T08:39:02.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.220+0000 7f4792ffd640 1 -- 172.21.15.43:0/1090926062 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f478c004350 con 0x7f47941011a0 2024-04-21T08:39:02.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.220+0000 7f479bfc7640 1 -- 172.21.15.43:0/1090926062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47941a4d10 con 0x7f47941011a0 2024-04-21T08:39:02.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.220+0000 7f4792ffd640 1 -- 172.21.15.43:0/1090926062 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c003b50 con 0x7f47941011a0 2024-04-21T08:39:02.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.221+0000 7f479bfc7640 1 -- 172.21.15.43:0/1090926062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47941a51d0 con 0x7f47941011a0 2024-04-21T08:39:02.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.221+0000 7f479bfc7640 1 -- 172.21.15.43:0/1090926062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4794068de0 con 0x7f47941011a0 2024-04-21T08:39:02.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.223+0000 7f4792ffd640 1 -- 172.21.15.43:0/1090926062 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f478c046020 con 0x7f47941011a0 2024-04-21T08:39:02.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.223+0000 7f4792ffd640 1 --2- 172.21.15.43:0/1090926062 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f477c0a3db0 0x7f477c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:02.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.223+0000 7f4799562640 1 --2- 172.21.15.43:0/1090926062 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f477c0a3db0 0x7f477c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:02.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.223+0000 7f4792ffd640 1 -- 172.21.15.43:0/1090926062 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f478c0f3270 con 0x7f47941011a0 2024-04-21T08:39:02.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.224+0000 7f4799562640 1 --2- 172.21.15.43:0/1090926062 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f477c0a3db0 0x7f477c0a6270 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4794069c30 tx=0x7f478800b5f0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:02.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.227+0000 7f4792ffd640 1 -- 172.21.15.43:0/1090926062 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f478c04d020 con 0x7f47941011a0 2024-04-21T08:39:02.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.419+0000 7f479bfc7640 1 -- 172.21.15.43:0/1090926062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4794108da0 con 0x7f47941011a0 2024-04-21T08:39:02.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.421+0000 7f4792ffd640 1 -- 172.21.15.43:0/1090926062 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+25233 (secure 0 0 0) 0x7f478c0bf600 con 0x7f47941011a0 2024-04-21T08:39:02.420 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:39:02.420 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":95,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:38:54.881943+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/126475132":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/4288951239":"2024-04-22T08:38:54.881882+0000","172.21.15.43:6801/1577030772":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/2382670190":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/1174411284":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/1577030772":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:39:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.423+0000 7f479bfc7640 1 -- 172.21.15.43:0/1090926062 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f477c0a3db0 msgr2=0x7f477c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.423+0000 7f479bfc7640 1 --2- 172.21.15.43:0/1090926062 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f477c0a3db0 0x7f477c0a6270 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4794069c30 tx=0x7f478800b5f0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.423+0000 7f479bfc7640 1 -- 172.21.15.43:0/1090926062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47941011a0 msgr2=0x7f4794199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.423+0000 7f479bfc7640 1 --2- 172.21.15.43:0/1090926062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47941011a0 0x7f4794199580 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f478c009230 tx=0x7f478c004bb0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.423+0000 7f479bfc7640 1 -- 172.21.15.43:0/1090926062 shutdown_connections 2024-04-21T08:39:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.423+0000 7f479bfc7640 1 --2- 172.21.15.43:0/1090926062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4794105e30 0x7f47941a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.423+0000 7f479bfc7640 1 --2- 172.21.15.43:0/1090926062 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f477c0a3db0 0x7f477c0a6270 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.423+0000 7f479bfc7640 1 --2- 172.21.15.43:0/1090926062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4794101b60 0x7f4794199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.423+0000 7f479bfc7640 1 --2- 172.21.15.43:0/1090926062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47941011a0 0x7f4794199580 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.423+0000 7f479bfc7640 1 -- 172.21.15.43:0/1090926062 >> 172.21.15.43:0/1090926062 conn(0x7f4794077e30 msgr2=0x7f47940fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:02.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.423+0000 7f479bfc7640 1 -- 172.21.15.43:0/1090926062 shutdown_connections 2024-04-21T08:39:02.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.423+0000 7f479bfc7640 1 -- 172.21.15.43:0/1090926062 wait complete. 2024-04-21T08:39:02.433 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:39.981405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:22.034751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:22.780795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:25.984261+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:39:02.434 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:39:02.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.540+0000 7f19188c1640 1 -- 172.21.15.43:0/3531885960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1914074300 msgr2=0x7f19140746e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:02.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.540+0000 7f19188c1640 1 --2- 172.21.15.43:0/3531885960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1914074300 0x7f19140746e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1904009900 tx=0x7f190402bcd0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.540+0000 7f19188c1640 1 -- 172.21.15.43:0/3531885960 shutdown_connections 2024-04-21T08:39:02.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.540+0000 7f19188c1640 1 --2- 172.21.15.43:0/3531885960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f191410e7e0 0x7f1914110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.540+0000 7f19188c1640 1 --2- 172.21.15.43:0/3531885960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1914074c20 0x7f191410e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.540+0000 7f19188c1640 1 --2- 172.21.15.43:0/3531885960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1914074300 0x7f19140746e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.540+0000 7f19188c1640 1 -- 172.21.15.43:0/3531885960 >> 172.21.15.43:0/3531885960 conn(0x7f19140fd990 msgr2=0x7f19140ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:02.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.540+0000 7f19188c1640 1 -- 172.21.15.43:0/3531885960 shutdown_connections 2024-04-21T08:39:02.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.540+0000 7f19188c1640 1 -- 172.21.15.43:0/3531885960 wait complete. 2024-04-21T08:39:02.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.541+0000 7f19188c1640 1 Processor -- start 2024-04-21T08:39:02.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.541+0000 7f19188c1640 1 -- start start 2024-04-21T08:39:02.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.541+0000 7f19188c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1914074300 0x7f19141a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:02.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.541+0000 7f19188c1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1914074c20 0x7f19141a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:02.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.541+0000 7f19188c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f191410e7e0 0x7f191419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:02.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.541+0000 7f19188c1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f19141143a0 con 0x7f191410e7e0 2024-04-21T08:39:02.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.541+0000 7f19188c1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1914114220 con 0x7f1914074300 2024-04-21T08:39:02.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.541+0000 7f19188c1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1914114520 con 0x7f1914074c20 2024-04-21T08:39:02.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.541+0000 7f1912d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f191410e7e0 0x7f191419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:02.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.541+0000 7f1912d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f191410e7e0 0x7f191419a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38942/0 (socket says 172.21.15.43:38942) 2024-04-21T08:39:02.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.541+0000 7f1912d9d640 1 -- 172.21.15.43:0/1015066095 learned_addr learned my addr 172.21.15.43:0/1015066095 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:02.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.542+0000 7f191259c640 1 --2- 172.21.15.43:0/1015066095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1914074300 0x7f19141a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:02.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.542+0000 7f1909d9b640 1 --2- 172.21.15.43:0/1015066095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1914074c20 0x7f19141a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:02.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.542+0000 7f1912d9d640 1 -- 172.21.15.43:0/1015066095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1914074c20 msgr2=0x7f19141a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:02.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.542+0000 7f1912d9d640 1 --2- 172.21.15.43:0/1015066095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1914074c20 0x7f19141a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.542+0000 7f1912d9d640 1 -- 172.21.15.43:0/1015066095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1914074300 msgr2=0x7f19141a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:02.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.542+0000 7f1912d9d640 1 --2- 172.21.15.43:0/1015066095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1914074300 0x7f19141a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.542+0000 7f1912d9d640 1 -- 172.21.15.43:0/1015066095 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191419b020 con 0x7f191410e7e0 2024-04-21T08:39:02.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.542+0000 7f1912d9d640 1 --2- 172.21.15.43:0/1015066095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f191410e7e0 0x7f191419a760 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f190c00eaa0 tx=0x7f190c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:02.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.542+0000 7f190bfff640 1 -- 172.21.15.43:0/1015066095 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c00cdb0 con 0x7f191410e7e0 2024-04-21T08:39:02.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.542+0000 7f19188c1640 1 -- 172.21.15.43:0/1015066095 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f191419b310 con 0x7f191410e7e0 2024-04-21T08:39:02.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.542+0000 7f190bfff640 1 -- 172.21.15.43:0/1015066095 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f190c005020 con 0x7f191410e7e0 2024-04-21T08:39:02.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.542+0000 7f190bfff640 1 -- 172.21.15.43:0/1015066095 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c0116d0 con 0x7f191410e7e0 2024-04-21T08:39:02.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.542+0000 7f19188c1640 1 -- 172.21.15.43:0/1015066095 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19141a7620 con 0x7f191410e7e0 2024-04-21T08:39:02.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.543+0000 7f19188c1640 1 -- 172.21.15.43:0/1015066095 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19140759b0 con 0x7f191410e7e0 2024-04-21T08:39:02.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.544+0000 7f190bfff640 1 -- 172.21.15.43:0/1015066095 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f190c0054a0 con 0x7f191410e7e0 2024-04-21T08:39:02.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.545+0000 7f190bfff640 1 --2- 172.21.15.43:0/1015066095 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f19000a3db0 0x7f19000a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:02.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.545+0000 7f191259c640 1 --2- 172.21.15.43:0/1015066095 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f19000a3db0 0x7f19000a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:02.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.545+0000 7f190bfff640 1 -- 172.21.15.43:0/1015066095 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f190c0d0fb0 con 0x7f191410e7e0 2024-04-21T08:39:02.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.545+0000 7f191259c640 1 --2- 172.21.15.43:0/1015066095 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f19000a3db0 0x7f19000a6270 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1904000c00 tx=0x7f190402c750 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:02.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.549+0000 7f190bfff640 1 -- 172.21.15.43:0/1015066095 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f190c016030 con 0x7f191410e7e0 2024-04-21T08:39:02.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:02 smithi175 ceph-mon[31920]: from='client.15747 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:39:02.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:02 smithi175 ceph-mon[31920]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 64 KiB/s wr, 30 op/s 2024-04-21T08:39:02.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:02 smithi175 ceph-mon[31920]: from='client.15786 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:39:02.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:02 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1090926062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:39:02.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.736+0000 7f19188c1640 1 -- 172.21.15.43:0/1015066095 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f191419b540 con 0x7f191410e7e0 2024-04-21T08:39:02.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.736+0000 7f190bfff640 1 -- 172.21.15.43:0/1015066095 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f190c09d0b0 con 0x7f191410e7e0 2024-04-21T08:39:02.736 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-21T08:39:02.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.738+0000 7f19188c1640 1 -- 172.21.15.43:0/1015066095 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f19000a3db0 msgr2=0x7f19000a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:02.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.738+0000 7f19188c1640 1 --2- 172.21.15.43:0/1015066095 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f19000a3db0 0x7f19000a6270 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1904000c00 tx=0x7f190402c750 comp rx=0 tx=0).stop 2024-04-21T08:39:02.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.738+0000 7f19188c1640 1 -- 172.21.15.43:0/1015066095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f191410e7e0 msgr2=0x7f191419a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:02.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.738+0000 7f19188c1640 1 --2- 172.21.15.43:0/1015066095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f191410e7e0 0x7f191419a760 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f190c00eaa0 tx=0x7f190c00ef60 comp rx=0 tx=0).stop 2024-04-21T08:39:02.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.738+0000 7f19188c1640 1 -- 172.21.15.43:0/1015066095 shutdown_connections 2024-04-21T08:39:02.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.738+0000 7f19188c1640 1 --2- 172.21.15.43:0/1015066095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f191410e7e0 0x7f191419a760 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.738+0000 7f19188c1640 1 --2- 172.21.15.43:0/1015066095 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f19000a3db0 0x7f19000a6270 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.738+0000 7f19188c1640 1 --2- 172.21.15.43:0/1015066095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1914074c20 0x7f19141a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.738+0000 7f19188c1640 1 --2- 172.21.15.43:0/1015066095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1914074300 0x7f19141a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.738+0000 7f19188c1640 1 -- 172.21.15.43:0/1015066095 >> 172.21.15.43:0/1015066095 conn(0x7f19140fd990 msgr2=0x7f19140fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:02.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.738+0000 7f19188c1640 1 -- 172.21.15.43:0/1015066095 shutdown_connections 2024-04-21T08:39:02.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.738+0000 7f19188c1640 1 -- 172.21.15.43:0/1015066095 wait complete. 2024-04-21T08:39:02.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:02 smithi043 ceph-mon[27978]: from='client.15747 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:39:02.748 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:02 smithi043 ceph-mon[27978]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 64 KiB/s wr, 30 op/s 2024-04-21T08:39:02.748 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:02 smithi043 ceph-mon[27978]: from='client.15786 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:39:02.748 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:02 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1090926062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:39:02.749 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:39:02.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:02 smithi112 ceph-mon[31085]: from='client.15747 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:39:02.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:02 smithi112 ceph-mon[31085]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 64 KiB/s wr, 30 op/s 2024-04-21T08:39:02.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:02 smithi112 ceph-mon[31085]: from='client.15786 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:39:02.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:02 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1090926062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:39:02.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.855+0000 7f3b39796640 1 -- 172.21.15.43:0/1478950920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b34102b40 msgr2=0x7f3b34106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:02.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.855+0000 7f3b39796640 1 --2- 172.21.15.43:0/1478950920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b34102b40 0x7f3b34106b80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009900 tx=0x7f3b2802bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:02.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.855+0000 7f3b39796640 1 -- 172.21.15.43:0/1478950920 shutdown_connections 2024-04-21T08:39:02.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.855+0000 7f3b39796640 1 --2- 172.21.15.43:0/1478950920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b34107260 0x7f3b3410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.855+0000 7f3b39796640 1 --2- 172.21.15.43:0/1478950920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b34102b40 0x7f3b34106b80 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.855+0000 7f3b39796640 1 --2- 172.21.15.43:0/1478950920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b34102190 0x7f3b34102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.855+0000 7f3b39796640 1 -- 172.21.15.43:0/1478950920 >> 172.21.15.43:0/1478950920 conn(0x7f3b340793a0 msgr2=0x7f3b340797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:02.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.855+0000 7f3b39796640 1 -- 172.21.15.43:0/1478950920 shutdown_connections 2024-04-21T08:39:02.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.855+0000 7f3b39796640 1 -- 172.21.15.43:0/1478950920 wait complete. 2024-04-21T08:39:02.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.856+0000 7f3b39796640 1 Processor -- start 2024-04-21T08:39:02.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.856+0000 7f3b39796640 1 -- start start 2024-04-21T08:39:02.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.856+0000 7f3b39796640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b34102190 0x7f3b34198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:02.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.856+0000 7f3b39796640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b34107260 0x7f3b34198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:02.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.856+0000 7f3b39796640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b3419d020 0x7f3b3419d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:02.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.856+0000 7f3b39796640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3b341121a0 con 0x7f3b34107260 2024-04-21T08:39:02.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.856+0000 7f3b39796640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3b34112020 con 0x7f3b3419d020 2024-04-21T08:39:02.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.856+0000 7f3b39796640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3b34112320 con 0x7f3b34102190 2024-04-21T08:39:02.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.856+0000 7f3b327fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b34107260 0x7f3b34198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:02.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.856+0000 7f3b327fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b34107260 0x7f3b34198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38960/0 (socket says 172.21.15.43:38960) 2024-04-21T08:39:02.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.856+0000 7f3b327fc640 1 -- 172.21.15.43:0/3472036348 learned_addr learned my addr 172.21.15.43:0/3472036348 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:02.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.857+0000 7f3b337fe640 1 --2- 172.21.15.43:0/3472036348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b3419d020 0x7f3b3419d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:02.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.857+0000 7f3b32ffd640 1 --2- 172.21.15.43:0/3472036348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b34102190 0x7f3b34198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:02.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.857+0000 7f3b327fc640 1 -- 172.21.15.43:0/3472036348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b34102190 msgr2=0x7f3b34198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:02.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.857+0000 7f3b327fc640 1 --2- 172.21.15.43:0/3472036348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b34102190 0x7f3b34198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.857+0000 7f3b327fc640 1 -- 172.21.15.43:0/3472036348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b3419d020 msgr2=0x7f3b3419d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:02.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.857+0000 7f3b327fc640 1 --2- 172.21.15.43:0/3472036348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b3419d020 0x7f3b3419d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:02.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.857+0000 7f3b327fc640 1 -- 172.21.15.43:0/3472036348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3419db10 con 0x7f3b34107260 2024-04-21T08:39:02.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.857+0000 7f3b337fe640 1 --2- 172.21.15.43:0/3472036348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b3419d020 0x7f3b3419d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:02.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.857+0000 7f3b327fc640 1 --2- 172.21.15.43:0/3472036348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b34107260 0x7f3b34198ca0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3b28036040 tx=0x7f3b280045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:02.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.857+0000 7f3b1bfff640 1 -- 172.21.15.43:0/3472036348 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28043020 con 0x7f3b34107260 2024-04-21T08:39:02.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.857+0000 7f3b39796640 1 -- 172.21.15.43:0/3472036348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b3419dda0 con 0x7f3b34107260 2024-04-21T08:39:02.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.857+0000 7f3b1bfff640 1 -- 172.21.15.43:0/3472036348 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b2802ca70 con 0x7f3b34107260 2024-04-21T08:39:02.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.857+0000 7f3b1bfff640 1 -- 172.21.15.43:0/3472036348 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28003ac0 con 0x7f3b34107260 2024-04-21T08:39:02.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.857+0000 7f3b39796640 1 -- 172.21.15.43:0/3472036348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b341a55f0 con 0x7f3b34107260 2024-04-21T08:39:02.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.858+0000 7f3b39796640 1 -- 172.21.15.43:0/3472036348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b340691c0 con 0x7f3b34107260 2024-04-21T08:39:02.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.859+0000 7f3b1bfff640 1 -- 172.21.15.43:0/3472036348 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f3b28046020 con 0x7f3b34107260 2024-04-21T08:39:02.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.860+0000 7f3b1bfff640 1 --2- 172.21.15.43:0/3472036348 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3b1c0a3db0 0x7f3b1c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:02.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.860+0000 7f3b1bfff640 1 -- 172.21.15.43:0/3472036348 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f3b280f31f0 con 0x7f3b34107260 2024-04-21T08:39:02.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.860+0000 7f3b32ffd640 1 --2- 172.21.15.43:0/3472036348 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3b1c0a3db0 0x7f3b1c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:02.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.861+0000 7f3b32ffd640 1 --2- 172.21.15.43:0/3472036348 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3b1c0a3db0 0x7f3b1c0a6270 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009b60 tx=0x7f3b24005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:02.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:02.864+0000 7f3b1bfff640 1 -- 172.21.15.43:0/3472036348 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3b2804d020 con 0x7f3b34107260 2024-04-21T08:39:03.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.049+0000 7f3b39796640 1 -- 172.21.15.43:0/3472036348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3b34102b40 con 0x7f3b34107260 2024-04-21T08:39:03.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.050+0000 7f3b1bfff640 1 -- 172.21.15.43:0/3472036348 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7f3b28047e40 con 0x7f3b34107260 2024-04-21T08:39:03.049 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:39:03.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.052+0000 7f3b39796640 1 -- 172.21.15.43:0/3472036348 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3b1c0a3db0 msgr2=0x7f3b1c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:03.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.052+0000 7f3b39796640 1 --2- 172.21.15.43:0/3472036348 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3b1c0a3db0 0x7f3b1c0a6270 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009b60 tx=0x7f3b24005f90 comp rx=0 tx=0).stop 2024-04-21T08:39:03.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.052+0000 7f3b39796640 1 -- 172.21.15.43:0/3472036348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b34107260 msgr2=0x7f3b34198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:03.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.052+0000 7f3b39796640 1 --2- 172.21.15.43:0/3472036348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b34107260 0x7f3b34198ca0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3b28036040 tx=0x7f3b280045e0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.052+0000 7f3b39796640 1 -- 172.21.15.43:0/3472036348 shutdown_connections 2024-04-21T08:39:03.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.052+0000 7f3b39796640 1 --2- 172.21.15.43:0/3472036348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b3419d020 0x7f3b3419d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.052+0000 7f3b39796640 1 --2- 172.21.15.43:0/3472036348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b34107260 0x7f3b34198ca0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.052+0000 7f3b39796640 1 --2- 172.21.15.43:0/3472036348 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3b1c0a3db0 0x7f3b1c0a6270 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.052+0000 7f3b39796640 1 --2- 172.21.15.43:0/3472036348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b34102190 0x7f3b34198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.052+0000 7f3b39796640 1 -- 172.21.15.43:0/3472036348 >> 172.21.15.43:0/3472036348 conn(0x7f3b340793a0 msgr2=0x7f3b34100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:03.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.052+0000 7f3b39796640 1 -- 172.21.15.43:0/3472036348 shutdown_connections 2024-04-21T08:39:03.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.052+0000 7f3b39796640 1 -- 172.21.15.43:0/3472036348 wait complete. 2024-04-21T08:39:03.062 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:39:03.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.169+0000 7f2041643640 1 -- 172.21.15.43:0/2899385089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f203c06b1f0 msgr2=0x7f203c106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:03.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.169+0000 7f2041643640 1 --2- 172.21.15.43:0/2899385089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f203c06b1f0 0x7f203c106b90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2024009900 tx=0x7f202402bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:03.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.169+0000 7f2041643640 1 -- 172.21.15.43:0/2899385089 shutdown_connections 2024-04-21T08:39:03.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.169+0000 7f2041643640 1 --2- 172.21.15.43:0/2899385089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f203c107270 0x7f203c10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.169+0000 7f2041643640 1 --2- 172.21.15.43:0/2899385089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f203c06b1f0 0x7f203c106b90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.169+0000 7f2041643640 1 --2- 172.21.15.43:0/2899385089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f203c06a840 0x7f203c06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.169+0000 7f2041643640 1 -- 172.21.15.43:0/2899385089 >> 172.21.15.43:0/2899385089 conn(0x7f203c077eb0 msgr2=0x7f203c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:03.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.169+0000 7f2041643640 1 -- 172.21.15.43:0/2899385089 shutdown_connections 2024-04-21T08:39:03.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.169+0000 7f2041643640 1 -- 172.21.15.43:0/2899385089 wait complete. 2024-04-21T08:39:03.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.170+0000 7f2041643640 1 Processor -- start 2024-04-21T08:39:03.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.170+0000 7f2041643640 1 -- start start 2024-04-21T08:39:03.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.170+0000 7f2041643640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f203c06a840 0x7f203c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:03.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.170+0000 7f2041643640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f203c107270 0x7f203c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:03.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.170+0000 7f2041643640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f203c19cff0 0x7f203c19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:03.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.170+0000 7f2041643640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f203c1121b0 con 0x7f203c107270 2024-04-21T08:39:03.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.170+0000 7f2041643640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f203c112030 con 0x7f203c19cff0 2024-04-21T08:39:03.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.170+0000 7f2041643640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f203c112330 con 0x7f203c06a840 2024-04-21T08:39:03.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.170+0000 7f2033fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f203c107270 0x7f203c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:03.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.170+0000 7f2033fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f203c107270 0x7f203c198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38976/0 (socket says 172.21.15.43:38976) 2024-04-21T08:39:03.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.170+0000 7f2033fff640 1 -- 172.21.15.43:0/3833831408 learned_addr learned my addr 172.21.15.43:0/3833831408 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:03.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.170+0000 7f203affd640 1 --2- 172.21.15.43:0/3833831408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f203c06a840 0x7f203c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:03.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.170+0000 7f203b7fe640 1 --2- 172.21.15.43:0/3833831408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f203c19cff0 0x7f203c19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:03.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.170+0000 7f2033fff640 1 -- 172.21.15.43:0/3833831408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f203c06a840 msgr2=0x7f203c198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:03.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.170+0000 7f2033fff640 1 --2- 172.21.15.43:0/3833831408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f203c06a840 0x7f203c198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.170+0000 7f2033fff640 1 -- 172.21.15.43:0/3833831408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f203c19cff0 msgr2=0x7f203c19d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:03.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.170+0000 7f2033fff640 1 --2- 172.21.15.43:0/3833831408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f203c19cff0 0x7f203c19d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.171+0000 7f2033fff640 1 -- 172.21.15.43:0/3833831408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f203c19dae0 con 0x7f203c107270 2024-04-21T08:39:03.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.171+0000 7f203b7fe640 1 --2- 172.21.15.43:0/3833831408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f203c19cff0 0x7f203c19d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:03.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.171+0000 7f2033fff640 1 --2- 172.21.15.43:0/3833831408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f203c107270 0x7f203c198c70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2024036040 tx=0x7f202402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:03.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.171+0000 7f2038ff9640 1 -- 172.21.15.43:0/3833831408 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2024043020 con 0x7f203c107270 2024-04-21T08:39:03.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.171+0000 7f2041643640 1 -- 172.21.15.43:0/3833831408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f203c19dd70 con 0x7f203c107270 2024-04-21T08:39:03.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.171+0000 7f2038ff9640 1 -- 172.21.15.43:0/3833831408 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f202402cbe0 con 0x7f203c107270 2024-04-21T08:39:03.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.171+0000 7f2038ff9640 1 -- 172.21.15.43:0/3833831408 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2024003bf0 con 0x7f203c107270 2024-04-21T08:39:03.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.171+0000 7f2041643640 1 -- 172.21.15.43:0/3833831408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f203c1a55c0 con 0x7f203c107270 2024-04-21T08:39:03.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.172+0000 7f2041643640 1 -- 172.21.15.43:0/3833831408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f203c0691c0 con 0x7f203c107270 2024-04-21T08:39:03.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.173+0000 7f2038ff9640 1 -- 172.21.15.43:0/3833831408 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f2024046020 con 0x7f203c107270 2024-04-21T08:39:03.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.174+0000 7f2038ff9640 1 --2- 172.21.15.43:0/3833831408 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f20280a3db0 0x7f20280a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:03.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.174+0000 7f203affd640 1 --2- 172.21.15.43:0/3833831408 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f20280a3db0 0x7f20280a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:03.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.174+0000 7f2038ff9640 1 -- 172.21.15.43:0/3833831408 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f20240f2ed0 con 0x7f203c107270 2024-04-21T08:39:03.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.174+0000 7f203affd640 1 --2- 172.21.15.43:0/3833831408 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f20280a3db0 0x7f20280a6270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f202c005fd0 tx=0x7f202c00b390 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:03.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.178+0000 7f2038ff9640 1 -- 172.21.15.43:0/3833831408 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f202404d020 con 0x7f203c107270 2024-04-21T08:39:03.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.365+0000 7f2041643640 1 -- 172.21.15.43:0/3833831408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f203c06b1f0 con 0x7f203c107270 2024-04-21T08:39:03.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.365+0000 7f2038ff9640 1 -- 172.21.15.43:0/3833831408 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7f20240bf140 con 0x7f203c107270 2024-04-21T08:39:03.365 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-21T08:39:03.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.367+0000 7f2041643640 1 -- 172.21.15.43:0/3833831408 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f20280a3db0 msgr2=0x7f20280a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:03.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.367+0000 7f2041643640 1 --2- 172.21.15.43:0/3833831408 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f20280a3db0 0x7f20280a6270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f202c005fd0 tx=0x7f202c00b390 comp rx=0 tx=0).stop 2024-04-21T08:39:03.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.367+0000 7f2041643640 1 -- 172.21.15.43:0/3833831408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f203c107270 msgr2=0x7f203c198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:03.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.367+0000 7f2041643640 1 --2- 172.21.15.43:0/3833831408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f203c107270 0x7f203c198c70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2024036040 tx=0x7f202402c7e0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.367+0000 7f2041643640 1 -- 172.21.15.43:0/3833831408 shutdown_connections 2024-04-21T08:39:03.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.367+0000 7f2041643640 1 --2- 172.21.15.43:0/3833831408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f203c19cff0 0x7f203c19d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.367+0000 7f2041643640 1 --2- 172.21.15.43:0/3833831408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f203c107270 0x7f203c198c70 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.367+0000 7f2041643640 1 --2- 172.21.15.43:0/3833831408 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f20280a3db0 0x7f20280a6270 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.367+0000 7f2041643640 1 --2- 172.21.15.43:0/3833831408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f203c06a840 0x7f203c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.367+0000 7f2041643640 1 -- 172.21.15.43:0/3833831408 >> 172.21.15.43:0/3833831408 conn(0x7f203c077eb0 msgr2=0x7f203c0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:03.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.367+0000 7f2041643640 1 -- 172.21.15.43:0/3833831408 shutdown_connections 2024-04-21T08:39:03.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.367+0000 7f2041643640 1 -- 172.21.15.43:0/3833831408 wait complete. 2024-04-21T08:39:03.378 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:39:03.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.485+0000 7fb32d14b640 1 -- 172.21.15.43:0/3593177490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb328101a40 msgr2=0x7fb328105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:03.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.485+0000 7fb32d14b640 1 --2- 172.21.15.43:0/3593177490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb328101a40 0x7fb328105a80 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb310009900 tx=0x7fb31002bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:03.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.485+0000 7fb32d14b640 1 -- 172.21.15.43:0/3593177490 shutdown_connections 2024-04-21T08:39:03.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.485+0000 7fb32d14b640 1 --2- 172.21.15.43:0/3593177490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb328106160 0x7fb32810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.485+0000 7fb32d14b640 1 --2- 172.21.15.43:0/3593177490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb328101a40 0x7fb328105a80 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.485+0000 7fb32d14b640 1 --2- 172.21.15.43:0/3593177490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb328101090 0x7fb328101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.485+0000 7fb32d14b640 1 -- 172.21.15.43:0/3593177490 >> 172.21.15.43:0/3593177490 conn(0x7fb328077f50 msgr2=0x7fb328078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:03.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.486+0000 7fb32d14b640 1 -- 172.21.15.43:0/3593177490 shutdown_connections 2024-04-21T08:39:03.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.486+0000 7fb32d14b640 1 -- 172.21.15.43:0/3593177490 wait complete. 2024-04-21T08:39:03.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.486+0000 7fb32d14b640 1 Processor -- start 2024-04-21T08:39:03.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.486+0000 7fb32d14b640 1 -- start start 2024-04-21T08:39:03.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.487+0000 7fb32d14b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb328101090 0x7fb32819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:03.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.487+0000 7fb32d14b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb328106160 0x7fb32819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:03.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.487+0000 7fb32d14b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb328198590 0x7fb328198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:03.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.487+0000 7fb32d14b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb3281110a0 con 0x7fb328106160 2024-04-21T08:39:03.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.487+0000 7fb32d14b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb328110f20 con 0x7fb328101090 2024-04-21T08:39:03.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.487+0000 7fb32d14b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb328111220 con 0x7fb328198590 2024-04-21T08:39:03.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.487+0000 7fb32659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb328106160 0x7fb32819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:03.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.487+0000 7fb32659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb328106160 0x7fb32819e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38990/0 (socket says 172.21.15.43:38990) 2024-04-21T08:39:03.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.487+0000 7fb32659c640 1 -- 172.21.15.43:0/1602352499 learned_addr learned my addr 172.21.15.43:0/1602352499 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:03.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.487+0000 7fb32759e640 1 --2- 172.21.15.43:0/1602352499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb328198590 0x7fb328198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:03.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.487+0000 7fb326d9d640 1 --2- 172.21.15.43:0/1602352499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb328101090 0x7fb32819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:03.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.487+0000 7fb32659c640 1 -- 172.21.15.43:0/1602352499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb328198590 msgr2=0x7fb328198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:03.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.487+0000 7fb32659c640 1 --2- 172.21.15.43:0/1602352499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb328198590 0x7fb328198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.487+0000 7fb32659c640 1 -- 172.21.15.43:0/1602352499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb328101090 msgr2=0x7fb32819e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:03.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.487+0000 7fb32659c640 1 --2- 172.21.15.43:0/1602352499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb328101090 0x7fb32819e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.487+0000 7fb32659c640 1 -- 172.21.15.43:0/1602352499 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3281990b0 con 0x7fb328106160 2024-04-21T08:39:03.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.487+0000 7fb32759e640 1 --2- 172.21.15.43:0/1602352499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb328198590 0x7fb328198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:03.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.487+0000 7fb32659c640 1 --2- 172.21.15.43:0/1602352499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb328106160 0x7fb32819e920 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb310036040 tx=0x7fb31002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:03.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.487+0000 7fb31ffff640 1 -- 172.21.15.43:0/1602352499 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb310043020 con 0x7fb328106160 2024-04-21T08:39:03.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.487+0000 7fb31ffff640 1 -- 172.21.15.43:0/1602352499 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb31002cbe0 con 0x7fb328106160 2024-04-21T08:39:03.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.487+0000 7fb32d14b640 1 -- 172.21.15.43:0/1602352499 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb328199340 con 0x7fb328106160 2024-04-21T08:39:03.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.488+0000 7fb31ffff640 1 -- 172.21.15.43:0/1602352499 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb310003bf0 con 0x7fb328106160 2024-04-21T08:39:03.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.488+0000 7fb32d14b640 1 -- 172.21.15.43:0/1602352499 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3281a5590 con 0x7fb328106160 2024-04-21T08:39:03.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.489+0000 7fb32d14b640 1 -- 172.21.15.43:0/1602352499 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb328068d70 con 0x7fb328106160 2024-04-21T08:39:03.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.490+0000 7fb31ffff640 1 -- 172.21.15.43:0/1602352499 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fb310046020 con 0x7fb328106160 2024-04-21T08:39:03.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.490+0000 7fb31ffff640 1 --2- 172.21.15.43:0/1602352499 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb3140a3db0 0x7fb3140a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:03.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.490+0000 7fb326d9d640 1 --2- 172.21.15.43:0/1602352499 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb3140a3db0 0x7fb3140a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:03.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.490+0000 7fb31ffff640 1 -- 172.21.15.43:0/1602352499 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fb3100f35a0 con 0x7fb328106160 2024-04-21T08:39:03.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.491+0000 7fb326d9d640 1 --2- 172.21.15.43:0/1602352499 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb3140a3db0 0x7fb3140a6270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb318005fd0 tx=0x7fb318005850 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:03.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.494+0000 7fb31ffff640 1 -- 172.21.15.43:0/1602352499 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb31004d020 con 0x7fb328106160 2024-04-21T08:39:03.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.681+0000 7fb32d14b640 1 -- 172.21.15.43:0/1602352499 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb32810a400 con 0x7fb328106160 2024-04-21T08:39:03.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.681+0000 7fb31ffff640 1 -- 172.21.15.43:0/1602352499 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7fb31003e450 con 0x7fb328106160 2024-04-21T08:39:03.680 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:39:03.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.683+0000 7fb32d14b640 1 -- 172.21.15.43:0/1602352499 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb3140a3db0 msgr2=0x7fb3140a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:03.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.683+0000 7fb32d14b640 1 --2- 172.21.15.43:0/1602352499 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb3140a3db0 0x7fb3140a6270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb318005fd0 tx=0x7fb318005850 comp rx=0 tx=0).stop 2024-04-21T08:39:03.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.683+0000 7fb32d14b640 1 -- 172.21.15.43:0/1602352499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb328106160 msgr2=0x7fb32819e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:03.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.683+0000 7fb32d14b640 1 --2- 172.21.15.43:0/1602352499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb328106160 0x7fb32819e920 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb310036040 tx=0x7fb31002c7e0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.683+0000 7fb32d14b640 1 -- 172.21.15.43:0/1602352499 shutdown_connections 2024-04-21T08:39:03.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.683+0000 7fb32d14b640 1 --2- 172.21.15.43:0/1602352499 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb3140a3db0 0x7fb3140a6270 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.683+0000 7fb32d14b640 1 --2- 172.21.15.43:0/1602352499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb328198590 0x7fb328198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.683+0000 7fb32d14b640 1 --2- 172.21.15.43:0/1602352499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb328106160 0x7fb32819e920 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.683+0000 7fb32d14b640 1 --2- 172.21.15.43:0/1602352499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb328101090 0x7fb32819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.683+0000 7fb32d14b640 1 -- 172.21.15.43:0/1602352499 >> 172.21.15.43:0/1602352499 conn(0x7fb328077f50 msgr2=0x7fb328069510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:03.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.683+0000 7fb32d14b640 1 -- 172.21.15.43:0/1602352499 shutdown_connections 2024-04-21T08:39:03.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.683+0000 7fb32d14b640 1 -- 172.21.15.43:0/1602352499 wait complete. 2024-04-21T08:39:03.692 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:03 smithi043 ceph-mon[27978]: from='client.15792 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:39:03.693 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:03 smithi043 ceph-mon[27978]: from='client.15798 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:39:03.693 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:03 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1015066095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:39:03.693 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:03 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3472036348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:39:03.693 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:03 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3833831408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:39:03.694 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:39:03.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:03 smithi112 ceph-mon[31085]: from='client.15792 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:39:03.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:03 smithi112 ceph-mon[31085]: from='client.15798 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:39:03.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:03 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1015066095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:39:03.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:03 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3472036348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:39:03.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:03 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3833831408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:39:03.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.797+0000 7f90e5785640 1 -- 172.21.15.43:0/2759957606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e006a640 msgr2=0x7f90e006aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:03.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.797+0000 7f90e5785640 1 --2- 172.21.15.43:0/2759957606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e006a640 0x7f90e006aac0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009930 tx=0x7f90c802bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:03.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.797+0000 7f90e5785640 1 -- 172.21.15.43:0/2759957606 shutdown_connections 2024-04-21T08:39:03.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.797+0000 7f90e5785640 1 --2- 172.21.15.43:0/2759957606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90e006b000 0x7f90e010ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.797+0000 7f90e5785640 1 --2- 172.21.15.43:0/2759957606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e006a640 0x7f90e006aac0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.797+0000 7f90e5785640 1 --2- 172.21.15.43:0/2759957606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90e0108040 0x7f90e0108420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.797+0000 7f90e5785640 1 -- 172.21.15.43:0/2759957606 >> 172.21.15.43:0/2759957606 conn(0x7f90e0077e30 msgr2=0x7f90e0078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:03.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.797+0000 7f90e5785640 1 -- 172.21.15.43:0/2759957606 shutdown_connections 2024-04-21T08:39:03.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.797+0000 7f90e5785640 1 -- 172.21.15.43:0/2759957606 wait complete. 2024-04-21T08:39:03.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.798+0000 7f90e5785640 1 Processor -- start 2024-04-21T08:39:03.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.798+0000 7f90e5785640 1 -- start start 2024-04-21T08:39:03.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.798+0000 7f90e5785640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90e006a640 0x7f90e01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:03.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.798+0000 7f90e5785640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90e006b000 0x7f90e0198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:03.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.798+0000 7f90e5785640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e0108040 0x7f90e019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:03.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.798+0000 7f90e5785640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f90e0112190 con 0x7f90e0108040 2024-04-21T08:39:03.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.798+0000 7f90e5785640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f90e0112010 con 0x7f90e006b000 2024-04-21T08:39:03.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.798+0000 7f90e5785640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f90e0112310 con 0x7f90e006a640 2024-04-21T08:39:03.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.798+0000 7f90df7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e0108040 0x7f90e019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:03.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.798+0000 7f90df7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e0108040 0x7f90e019cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38998/0 (socket says 172.21.15.43:38998) 2024-04-21T08:39:03.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.798+0000 7f90df7fe640 1 -- 172.21.15.43:0/3468297608 learned_addr learned my addr 172.21.15.43:0/3468297608 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:03.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.798+0000 7f90deffd640 1 --2- 172.21.15.43:0/3468297608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90e006a640 0x7f90e01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:03.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.798+0000 7f90d7fff640 1 --2- 172.21.15.43:0/3468297608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90e006b000 0x7f90e0198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:03.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.798+0000 7f90df7fe640 1 -- 172.21.15.43:0/3468297608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90e006a640 msgr2=0x7f90e01986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:03.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.798+0000 7f90df7fe640 1 --2- 172.21.15.43:0/3468297608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90e006a640 0x7f90e01986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.798+0000 7f90df7fe640 1 -- 172.21.15.43:0/3468297608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90e006b000 msgr2=0x7f90e0198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:03.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.798+0000 7f90df7fe640 1 --2- 172.21.15.43:0/3468297608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90e006b000 0x7f90e0198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.798+0000 7f90df7fe640 1 -- 172.21.15.43:0/3468297608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e019d6c0 con 0x7f90e0108040 2024-04-21T08:39:03.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.799+0000 7f90df7fe640 1 --2- 172.21.15.43:0/3468297608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e0108040 0x7f90e019cf90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f90d800eb10 tx=0x7f90d800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:03.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.799+0000 7f90dcff9640 1 -- 172.21.15.43:0/3468297608 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8004b20 con 0x7f90e0108040 2024-04-21T08:39:03.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.799+0000 7f90dcff9640 1 -- 172.21.15.43:0/3468297608 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90d8004cc0 con 0x7f90e0108040 2024-04-21T08:39:03.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.799+0000 7f90dcff9640 1 -- 172.21.15.43:0/3468297608 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8010670 con 0x7f90e0108040 2024-04-21T08:39:03.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.799+0000 7f90e5785640 1 -- 172.21.15.43:0/3468297608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e019d9b0 con 0x7f90e0108040 2024-04-21T08:39:03.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.799+0000 7f90e5785640 1 -- 172.21.15.43:0/3468297608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e01a5240 con 0x7f90e0108040 2024-04-21T08:39:03.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.800+0000 7f90e5785640 1 -- 172.21.15.43:0/3468297608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90e0069230 con 0x7f90e0108040 2024-04-21T08:39:03.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.801+0000 7f90dcff9640 1 -- 172.21.15.43:0/3468297608 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f90d8005020 con 0x7f90e0108040 2024-04-21T08:39:03.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.802+0000 7f90dcff9640 1 --2- 172.21.15.43:0/3468297608 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f90cc0a3e30 0x7f90cc0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:03.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.802+0000 7f90deffd640 1 --2- 172.21.15.43:0/3468297608 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f90cc0a3e30 0x7f90cc0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:03.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.802+0000 7f90dcff9640 1 -- 172.21.15.43:0/3468297608 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f90d80d1330 con 0x7f90e0108040 2024-04-21T08:39:03.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.802+0000 7f90deffd640 1 --2- 172.21.15.43:0/3468297608 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f90cc0a3e30 0x7f90cc0a62f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009bd0 tx=0x7f90d0005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:03.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.806+0000 7f90dcff9640 1 -- 172.21.15.43:0/3468297608 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90d809d430 con 0x7f90e0108040 2024-04-21T08:39:03.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:03 smithi175 ceph-mon[31920]: from='client.15792 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:39:03.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:03 smithi175 ceph-mon[31920]: from='client.15798 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:39:03.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:03 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1015066095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:39:03.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:03 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3472036348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:39:03.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:03 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3833831408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:39:03.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.996+0000 7f90e5785640 1 -- 172.21.15.43:0/3468297608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f90e0109e60 con 0x7f90e0108040 2024-04-21T08:39:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:03.998+0000 7f90dcff9640 1 -- 172.21.15.43:0/3468297608 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+25233 (secure 0 0 0) 0x7f90d809cd30 con 0x7f90e0108040 2024-04-21T08:39:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:39:03.997 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":95,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:38:54.881943+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/126475132":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/4288951239":"2024-04-22T08:38:54.881882+0000","172.21.15.43:6801/1577030772":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/2382670190":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/1174411284":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/1577030772":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:39:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.000+0000 7f90e5785640 1 -- 172.21.15.43:0/3468297608 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f90cc0a3e30 msgr2=0x7f90cc0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.000+0000 7f90e5785640 1 --2- 172.21.15.43:0/3468297608 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f90cc0a3e30 0x7f90cc0a62f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009bd0 tx=0x7f90d0005f90 comp rx=0 tx=0).stop 2024-04-21T08:39:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.000+0000 7f90e5785640 1 -- 172.21.15.43:0/3468297608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e0108040 msgr2=0x7f90e019cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.000+0000 7f90e5785640 1 --2- 172.21.15.43:0/3468297608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e0108040 0x7f90e019cf90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f90d800eb10 tx=0x7f90d800c2d0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.000+0000 7f90e5785640 1 -- 172.21.15.43:0/3468297608 shutdown_connections 2024-04-21T08:39:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.000+0000 7f90e5785640 1 --2- 172.21.15.43:0/3468297608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e0108040 0x7f90e019cf90 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.000+0000 7f90e5785640 1 --2- 172.21.15.43:0/3468297608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90e006b000 0x7f90e0198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.000+0000 7f90e5785640 1 --2- 172.21.15.43:0/3468297608 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f90cc0a3e30 0x7f90cc0a62f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.000+0000 7f90e5785640 1 --2- 172.21.15.43:0/3468297608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90e006a640 0x7f90e01986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.000+0000 7f90e5785640 1 -- 172.21.15.43:0/3468297608 >> 172.21.15.43:0/3468297608 conn(0x7f90e0077e30 msgr2=0x7f90e00ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.000+0000 7f90e5785640 1 -- 172.21.15.43:0/3468297608 shutdown_connections 2024-04-21T08:39:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.000+0000 7f90e5785640 1 -- 172.21.15.43:0/3468297608 wait complete. 2024-04-21T08:39:04.010 INFO:tasks.quiescer.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:39.981405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:22.034751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:22.780795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:25.984261+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:39:04.011 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:39:04.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.117+0000 7f74ff0c8640 1 -- 172.21.15.43:0/3994825715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74f806b1f0 msgr2=0x7f74f8106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:04.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.117+0000 7f74ff0c8640 1 --2- 172.21.15.43:0/3994825715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74f806b1f0 0x7f74f8106b90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f74ec009900 tx=0x7f74ec02bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:04.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.117+0000 7f74ff0c8640 1 -- 172.21.15.43:0/3994825715 shutdown_connections 2024-04-21T08:39:04.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.117+0000 7f74ff0c8640 1 --2- 172.21.15.43:0/3994825715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74f8107270 0x7f74f810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.117+0000 7f74ff0c8640 1 --2- 172.21.15.43:0/3994825715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74f806b1f0 0x7f74f8106b90 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.117+0000 7f74ff0c8640 1 --2- 172.21.15.43:0/3994825715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74f806a840 0x7f74f806ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.117+0000 7f74ff0c8640 1 -- 172.21.15.43:0/3994825715 >> 172.21.15.43:0/3994825715 conn(0x7f74f8077eb0 msgr2=0x7f74f80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:04.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.117+0000 7f74ff0c8640 1 -- 172.21.15.43:0/3994825715 shutdown_connections 2024-04-21T08:39:04.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.118+0000 7f74ff0c8640 1 -- 172.21.15.43:0/3994825715 wait complete. 2024-04-21T08:39:04.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.118+0000 7f74ff0c8640 1 Processor -- start 2024-04-21T08:39:04.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.118+0000 7f74ff0c8640 1 -- start start 2024-04-21T08:39:04.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.118+0000 7f74ff0c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74f806a840 0x7f74f8198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:04.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.118+0000 7f74ff0c8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74f8107270 0x7f74f8198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:04.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.118+0000 7f74ff0c8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74f819cff0 0x7f74f819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:04.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.118+0000 7f74ff0c8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f74f81121b0 con 0x7f74f819cff0 2024-04-21T08:39:04.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.118+0000 7f74ff0c8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f74f8112030 con 0x7f74f806a840 2024-04-21T08:39:04.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.118+0000 7f74ff0c8640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74f8112330 con 0x7f74f8107270 2024-04-21T08:39:04.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.118+0000 7f74fd665640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74f819cff0 0x7f74f819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:04.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.118+0000 7f74fd665640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74f819cff0 0x7f74f819d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39012/0 (socket says 172.21.15.43:39012) 2024-04-21T08:39:04.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.118+0000 7f74fd665640 1 -- 172.21.15.43:0/3019134007 learned_addr learned my addr 172.21.15.43:0/3019134007 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:04.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.118+0000 7f74fce64640 1 --2- 172.21.15.43:0/3019134007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74f806a840 0x7f74f8198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:04.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.118+0000 7f74f7fff640 1 --2- 172.21.15.43:0/3019134007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74f8107270 0x7f74f8198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:04.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.119+0000 7f74fd665640 1 -- 172.21.15.43:0/3019134007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74f8107270 msgr2=0x7f74f8198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:04.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.119+0000 7f74fd665640 1 --2- 172.21.15.43:0/3019134007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74f8107270 0x7f74f8198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.119+0000 7f74fd665640 1 -- 172.21.15.43:0/3019134007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74f806a840 msgr2=0x7f74f8198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:04.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.119+0000 7f74fd665640 1 --2- 172.21.15.43:0/3019134007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74f806a840 0x7f74f8198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.119+0000 7f74fd665640 1 -- 172.21.15.43:0/3019134007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f819dae0 con 0x7f74f819cff0 2024-04-21T08:39:04.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.119+0000 7f74f7fff640 1 --2- 172.21.15.43:0/3019134007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74f8107270 0x7f74f8198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:04.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.119+0000 7f74fce64640 1 --2- 172.21.15.43:0/3019134007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74f806a840 0x7f74f8198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:04.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.119+0000 7f74fd665640 1 --2- 172.21.15.43:0/3019134007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74f819cff0 0x7f74f819d4a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f74f000eaa0 tx=0x7f74f000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:04.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.119+0000 7f74f5ffb640 1 -- 172.21.15.43:0/3019134007 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f000cdb0 con 0x7f74f819cff0 2024-04-21T08:39:04.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.119+0000 7f74ff0c8640 1 -- 172.21.15.43:0/3019134007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f819ddd0 con 0x7f74f819cff0 2024-04-21T08:39:04.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.119+0000 7f74f5ffb640 1 -- 172.21.15.43:0/3019134007 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74f0005020 con 0x7f74f819cff0 2024-04-21T08:39:04.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.119+0000 7f74f5ffb640 1 -- 172.21.15.43:0/3019134007 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f00116d0 con 0x7f74f819cff0 2024-04-21T08:39:04.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.119+0000 7f74ff0c8640 1 -- 172.21.15.43:0/3019134007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f81a5590 con 0x7f74f819cff0 2024-04-21T08:39:04.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.120+0000 7f74ff0c8640 1 -- 172.21.15.43:0/3019134007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74f80691c0 con 0x7f74f819cff0 2024-04-21T08:39:04.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.121+0000 7f74f5ffb640 1 -- 172.21.15.43:0/3019134007 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f74f00051d0 con 0x7f74f819cff0 2024-04-21T08:39:04.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.122+0000 7f74f5ffb640 1 --2- 172.21.15.43:0/3019134007 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f74e00a3db0 0x7f74e00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:04.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.122+0000 7f74fce64640 1 --2- 172.21.15.43:0/3019134007 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f74e00a3db0 0x7f74e00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:04.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.122+0000 7f74f5ffb640 1 -- 172.21.15.43:0/3019134007 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f74f00d0fe0 con 0x7f74f819cff0 2024-04-21T08:39:04.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.122+0000 7f74fce64640 1 --2- 172.21.15.43:0/3019134007 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f74e00a3db0 0x7f74e00a6270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f74e8005fd0 tx=0x7f74e8005ea0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:04.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.126+0000 7f74f5ffb640 1 -- 172.21.15.43:0/3019134007 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f74f0016030 con 0x7f74f819cff0 2024-04-21T08:39:04.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.313+0000 7f74ff0c8640 1 -- 172.21.15.43:0/3019134007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f74f810a520 con 0x7f74f819cff0 2024-04-21T08:39:04.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.314+0000 7f74f5ffb640 1 -- 172.21.15.43:0/3019134007 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f74f009d120 con 0x7f74f819cff0 2024-04-21T08:39:04.313 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-21T08:39:04.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.316+0000 7f74ff0c8640 1 -- 172.21.15.43:0/3019134007 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f74e00a3db0 msgr2=0x7f74e00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:04.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.316+0000 7f74ff0c8640 1 --2- 172.21.15.43:0/3019134007 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f74e00a3db0 0x7f74e00a6270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f74e8005fd0 tx=0x7f74e8005ea0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.316+0000 7f74ff0c8640 1 -- 172.21.15.43:0/3019134007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74f819cff0 msgr2=0x7f74f819d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:04.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.316+0000 7f74ff0c8640 1 --2- 172.21.15.43:0/3019134007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74f819cff0 0x7f74f819d4a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f74f000eaa0 tx=0x7f74f000ef60 comp rx=0 tx=0).stop 2024-04-21T08:39:04.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.316+0000 7f74ff0c8640 1 -- 172.21.15.43:0/3019134007 shutdown_connections 2024-04-21T08:39:04.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.316+0000 7f74ff0c8640 1 --2- 172.21.15.43:0/3019134007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74f819cff0 0x7f74f819d4a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.316+0000 7f74ff0c8640 1 --2- 172.21.15.43:0/3019134007 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f74e00a3db0 0x7f74e00a6270 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.316+0000 7f74ff0c8640 1 --2- 172.21.15.43:0/3019134007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74f8107270 0x7f74f8198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.316+0000 7f74ff0c8640 1 --2- 172.21.15.43:0/3019134007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74f806a840 0x7f74f8198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.316+0000 7f74ff0c8640 1 -- 172.21.15.43:0/3019134007 >> 172.21.15.43:0/3019134007 conn(0x7f74f8077eb0 msgr2=0x7f74f80ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:04.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.316+0000 7f74ff0c8640 1 -- 172.21.15.43:0/3019134007 shutdown_connections 2024-04-21T08:39:04.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.316+0000 7f74ff0c8640 1 -- 172.21.15.43:0/3019134007 wait complete. 2024-04-21T08:39:04.326 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:39:04.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.433+0000 7fcbaf38f640 1 -- 172.21.15.43:0/2210582073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba806a840 msgr2=0x7fcba806ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:04.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.433+0000 7fcbaf38f640 1 --2- 172.21.15.43:0/2210582073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba806a840 0x7fcba806ac20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fcba4009900 tx=0x7fcba402bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:04.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.433+0000 7fcbaf38f640 1 -- 172.21.15.43:0/2210582073 shutdown_connections 2024-04-21T08:39:04.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.433+0000 7fcbaf38f640 1 --2- 172.21.15.43:0/2210582073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcba8107270 0x7fcba810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.433+0000 7fcbaf38f640 1 --2- 172.21.15.43:0/2210582073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcba806b1f0 0x7fcba8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.433+0000 7fcbaf38f640 1 --2- 172.21.15.43:0/2210582073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba806a840 0x7fcba806ac20 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.433+0000 7fcbaf38f640 1 -- 172.21.15.43:0/2210582073 >> 172.21.15.43:0/2210582073 conn(0x7fcba8077eb0 msgr2=0x7fcba80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:04.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.433+0000 7fcbaf38f640 1 -- 172.21.15.43:0/2210582073 shutdown_connections 2024-04-21T08:39:04.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.433+0000 7fcbaf38f640 1 -- 172.21.15.43:0/2210582073 wait complete. 2024-04-21T08:39:04.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.434+0000 7fcbaf38f640 1 Processor -- start 2024-04-21T08:39:04.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.434+0000 7fcbaf38f640 1 -- start start 2024-04-21T08:39:04.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.434+0000 7fcbaf38f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba806a840 0x7fcba81986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:04.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.434+0000 7fcbaf38f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcba8107270 0x7fcba8198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:04.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.434+0000 7fcbaf38f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcba819cf90 0x7fcba819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:04.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.434+0000 7fcbaf38f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81121b0 con 0x7fcba806a840 2024-04-21T08:39:04.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.434+0000 7fcbaf38f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8112030 con 0x7fcba8107270 2024-04-21T08:39:04.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.434+0000 7fcbaf38f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8112330 con 0x7fcba819cf90 2024-04-21T08:39:04.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.434+0000 7fcbad12b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba806a840 0x7fcba81986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:04.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.434+0000 7fcbad12b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba806a840 0x7fcba81986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39028/0 (socket says 172.21.15.43:39028) 2024-04-21T08:39:04.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.434+0000 7fcbad12b640 1 -- 172.21.15.43:0/84915204 learned_addr learned my addr 172.21.15.43:0/84915204 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:04.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.434+0000 7fcbac92a640 1 --2- 172.21.15.43:0/84915204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcba8107270 0x7fcba8198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:04.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.434+0000 7fcbad92c640 1 --2- 172.21.15.43:0/84915204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcba819cf90 0x7fcba819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:04.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.434+0000 7fcbad12b640 1 -- 172.21.15.43:0/84915204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcba819cf90 msgr2=0x7fcba819d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:04.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.434+0000 7fcbad12b640 1 --2- 172.21.15.43:0/84915204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcba819cf90 0x7fcba819d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.434+0000 7fcbad12b640 1 -- 172.21.15.43:0/84915204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcba8107270 msgr2=0x7fcba8198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:04.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.434+0000 7fcbad12b640 1 --2- 172.21.15.43:0/84915204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcba8107270 0x7fcba8198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.434+0000 7fcbad12b640 1 -- 172.21.15.43:0/84915204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba819da80 con 0x7fcba806a840 2024-04-21T08:39:04.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.435+0000 7fcbad12b640 1 --2- 172.21.15.43:0/84915204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba806a840 0x7fcba81986d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fcba4005940 tx=0x7fcba402c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:04.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.435+0000 7fcb9e7fc640 1 -- 172.21.15.43:0/84915204 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba4043020 con 0x7fcba806a840 2024-04-21T08:39:04.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.435+0000 7fcb9e7fc640 1 -- 172.21.15.43:0/84915204 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba402ca70 con 0x7fcba806a840 2024-04-21T08:39:04.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.435+0000 7fcbaf38f640 1 -- 172.21.15.43:0/84915204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba819dd10 con 0x7fcba806a840 2024-04-21T08:39:04.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.435+0000 7fcb9e7fc640 1 -- 172.21.15.43:0/84915204 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba40039f0 con 0x7fcba806a840 2024-04-21T08:39:04.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.435+0000 7fcbaf38f640 1 -- 172.21.15.43:0/84915204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba81a5560 con 0x7fcba806a840 2024-04-21T08:39:04.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.436+0000 7fcbaf38f640 1 -- 172.21.15.43:0/84915204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcba80691c0 con 0x7fcba806a840 2024-04-21T08:39:04.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.437+0000 7fcb9e7fc640 1 -- 172.21.15.43:0/84915204 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fcba4046020 con 0x7fcba806a840 2024-04-21T08:39:04.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.438+0000 7fcb9e7fc640 1 --2- 172.21.15.43:0/84915204 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcb940a3e30 0x7fcb940a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:04.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.438+0000 7fcbac92a640 1 --2- 172.21.15.43:0/84915204 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcb940a3e30 0x7fcb940a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:04.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.438+0000 7fcb9e7fc640 1 -- 172.21.15.43:0/84915204 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fcba40f33e0 con 0x7fcba806a840 2024-04-21T08:39:04.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.438+0000 7fcbac92a640 1 --2- 172.21.15.43:0/84915204 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcb940a3e30 0x7fcb940a62f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcba80ff930 tx=0x7fcb98005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:04.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.442+0000 7fcb9e7fc640 1 -- 172.21.15.43:0/84915204 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcba404d020 con 0x7fcba806a840 2024-04-21T08:39:04.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.627+0000 7fcbaf38f640 1 -- 172.21.15.43:0/84915204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcba810a4a0 con 0x7fcba806a840 2024-04-21T08:39:04.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.628+0000 7fcb9e7fc640 1 -- 172.21.15.43:0/84915204 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7fcba402cc20 con 0x7fcba806a840 2024-04-21T08:39:04.627 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:39:04.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.629+0000 7fcbaf38f640 1 -- 172.21.15.43:0/84915204 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcb940a3e30 msgr2=0x7fcb940a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:04.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.629+0000 7fcbaf38f640 1 --2- 172.21.15.43:0/84915204 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcb940a3e30 0x7fcb940a62f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcba80ff930 tx=0x7fcb98005f90 comp rx=0 tx=0).stop 2024-04-21T08:39:04.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.629+0000 7fcbaf38f640 1 -- 172.21.15.43:0/84915204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba806a840 msgr2=0x7fcba81986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:04.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.629+0000 7fcbaf38f640 1 --2- 172.21.15.43:0/84915204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba806a840 0x7fcba81986d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fcba4005940 tx=0x7fcba402c5f0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.630+0000 7fcbaf38f640 1 -- 172.21.15.43:0/84915204 shutdown_connections 2024-04-21T08:39:04.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.630+0000 7fcbaf38f640 1 --2- 172.21.15.43:0/84915204 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcb940a3e30 0x7fcb940a62f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.630+0000 7fcbaf38f640 1 --2- 172.21.15.43:0/84915204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcba819cf90 0x7fcba819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.630+0000 7fcbaf38f640 1 --2- 172.21.15.43:0/84915204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcba8107270 0x7fcba8198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.630+0000 7fcbaf38f640 1 --2- 172.21.15.43:0/84915204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba806a840 0x7fcba81986d0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.630+0000 7fcbaf38f640 1 -- 172.21.15.43:0/84915204 >> 172.21.15.43:0/84915204 conn(0x7fcba8077eb0 msgr2=0x7fcba8108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:04.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.630+0000 7fcbaf38f640 1 -- 172.21.15.43:0/84915204 shutdown_connections 2024-04-21T08:39:04.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.630+0000 7fcbaf38f640 1 -- 172.21.15.43:0/84915204 wait complete. 2024-04-21T08:39:04.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:04 smithi043 ceph-mon[27978]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 94 KiB/s wr, 50 op/s 2024-04-21T08:39:04.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:04 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1602352499' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:39:04.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:04 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3468297608' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:39:04.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:04 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3019134007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:39:04.640 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:39:04.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.746+0000 7f5069c5c640 1 -- 172.21.15.43:0/2775211006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5064107260 msgr2=0x7f506410ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:04.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.746+0000 7f5069c5c640 1 --2- 172.21.15.43:0/2775211006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5064107260 0x7f506410ee30 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f505c009ca0 tx=0x7f505c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.746+0000 7f5069c5c640 1 -- 172.21.15.43:0/2775211006 shutdown_connections 2024-04-21T08:39:04.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.746+0000 7f5069c5c640 1 --2- 172.21.15.43:0/2775211006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5064107260 0x7f506410ee30 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.746+0000 7f5069c5c640 1 --2- 172.21.15.43:0/2775211006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5064102b40 0x7f5064106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.746+0000 7f5069c5c640 1 --2- 172.21.15.43:0/2775211006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5064102190 0x7f5064102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.746+0000 7f5069c5c640 1 -- 172.21.15.43:0/2775211006 >> 172.21.15.43:0/2775211006 conn(0x7f50640793a0 msgr2=0x7f50640797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:04.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.746+0000 7f5069c5c640 1 -- 172.21.15.43:0/2775211006 shutdown_connections 2024-04-21T08:39:04.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.746+0000 7f5069c5c640 1 -- 172.21.15.43:0/2775211006 wait complete. 2024-04-21T08:39:04.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.747+0000 7f5069c5c640 1 Processor -- start 2024-04-21T08:39:04.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.747+0000 7f5069c5c640 1 -- start start 2024-04-21T08:39:04.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.747+0000 7f5069c5c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5064102190 0x7f5064198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:04.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.747+0000 7f5069c5c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5064107260 0x7f5064198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:04.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.747+0000 7f5069c5c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f506419cfd0 0x7f506419d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:04.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.747+0000 7f5069c5c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f50641121b0 con 0x7f506419cfd0 2024-04-21T08:39:04.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.747+0000 7f5069c5c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5064112030 con 0x7f5064107260 2024-04-21T08:39:04.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.747+0000 7f5069c5c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5064112330 con 0x7f5064102190 2024-04-21T08:39:04.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.747+0000 7f5063fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f506419cfd0 0x7f506419d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:04.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.747+0000 7f5063fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f506419cfd0 0x7f506419d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39042/0 (socket says 172.21.15.43:39042) 2024-04-21T08:39:04.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.747+0000 7f5063fff640 1 -- 172.21.15.43:0/3944114967 learned_addr learned my addr 172.21.15.43:0/3944114967 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:04.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.747+0000 7f5062ffd640 1 --2- 172.21.15.43:0/3944114967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5064107260 0x7f5064198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:04.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.748+0000 7f5063fff640 1 -- 172.21.15.43:0/3944114967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5064102190 msgr2=0x7f5064198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:04.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.748+0000 7f50637fe640 1 --2- 172.21.15.43:0/3944114967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5064102190 0x7f5064198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:04.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.748+0000 7f5063fff640 1 --2- 172.21.15.43:0/3944114967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5064102190 0x7f5064198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.748+0000 7f5063fff640 1 -- 172.21.15.43:0/3944114967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5064107260 msgr2=0x7f5064198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:04.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.748+0000 7f5063fff640 1 --2- 172.21.15.43:0/3944114967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5064107260 0x7f5064198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.748+0000 7f5063fff640 1 -- 172.21.15.43:0/3944114967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f506419dac0 con 0x7f506419cfd0 2024-04-21T08:39:04.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.748+0000 7f5062ffd640 1 --2- 172.21.15.43:0/3944114967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5064107260 0x7f5064198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:04.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.748+0000 7f5063fff640 1 --2- 172.21.15.43:0/3944114967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f506419cfd0 0x7f506419d480 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f505c005e30 tx=0x7f505c034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:04.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.748+0000 7f5060ff9640 1 -- 172.21.15.43:0/3944114967 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c039040 con 0x7f506419cfd0 2024-04-21T08:39:04.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.748+0000 7f5060ff9640 1 -- 172.21.15.43:0/3944114967 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f505c044070 con 0x7f506419cfd0 2024-04-21T08:39:04.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.748+0000 7f5069c5c640 1 -- 172.21.15.43:0/3944114967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f506419dd50 con 0x7f506419cfd0 2024-04-21T08:39:04.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.748+0000 7f5060ff9640 1 -- 172.21.15.43:0/3944114967 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c003e60 con 0x7f506419cfd0 2024-04-21T08:39:04.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.748+0000 7f5069c5c640 1 -- 172.21.15.43:0/3944114967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50641a5510 con 0x7f506419cfd0 2024-04-21T08:39:04.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.749+0000 7f5069c5c640 1 -- 172.21.15.43:0/3944114967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50640691c0 con 0x7f506419cfd0 2024-04-21T08:39:04.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.750+0000 7f5060ff9640 1 -- 172.21.15.43:0/3944114967 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f505c0324c0 con 0x7f506419cfd0 2024-04-21T08:39:04.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.751+0000 7f5060ff9640 1 --2- 172.21.15.43:0/3944114967 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f504c0a3db0 0x7f504c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:04.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.751+0000 7f50637fe640 1 --2- 172.21.15.43:0/3944114967 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f504c0a3db0 0x7f504c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:04.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.751+0000 7f5060ff9640 1 -- 172.21.15.43:0/3944114967 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f505c0f3af0 con 0x7f506419cfd0 2024-04-21T08:39:04.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.751+0000 7f50637fe640 1 --2- 172.21.15.43:0/3944114967 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f504c0a3db0 0x7f504c0a6270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5054005fd0 tx=0x7f5054005850 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:04.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.755+0000 7f5060ff9640 1 -- 172.21.15.43:0/3944114967 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f505c0bfd00 con 0x7f506419cfd0 2024-04-21T08:39:04.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:04 smithi112 ceph-mon[31085]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 94 KiB/s wr, 50 op/s 2024-04-21T08:39:04.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:04 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1602352499' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:39:04.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:04 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3468297608' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:39:04.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:04 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3019134007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:39:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.941+0000 7f5069c5c640 1 -- 172.21.15.43:0/3944114967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5064102b40 con 0x7f506419cfd0 2024-04-21T08:39:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.941+0000 7f5060ff9640 1 -- 172.21.15.43:0/3944114967 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7f505c0bf600 con 0x7f506419cfd0 2024-04-21T08:39:04.940 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-21T08:39:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.943+0000 7f5069c5c640 1 -- 172.21.15.43:0/3944114967 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f504c0a3db0 msgr2=0x7f504c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.943+0000 7f5069c5c640 1 --2- 172.21.15.43:0/3944114967 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f504c0a3db0 0x7f504c0a6270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5054005fd0 tx=0x7f5054005850 comp rx=0 tx=0).stop 2024-04-21T08:39:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.943+0000 7f5069c5c640 1 -- 172.21.15.43:0/3944114967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f506419cfd0 msgr2=0x7f506419d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.943+0000 7f5069c5c640 1 --2- 172.21.15.43:0/3944114967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f506419cfd0 0x7f506419d480 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f505c005e30 tx=0x7f505c034760 comp rx=0 tx=0).stop 2024-04-21T08:39:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.943+0000 7f5069c5c640 1 -- 172.21.15.43:0/3944114967 shutdown_connections 2024-04-21T08:39:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.943+0000 7f5069c5c640 1 --2- 172.21.15.43:0/3944114967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f506419cfd0 0x7f506419d480 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.943+0000 7f5069c5c640 1 --2- 172.21.15.43:0/3944114967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5064107260 0x7f5064198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.943+0000 7f5069c5c640 1 --2- 172.21.15.43:0/3944114967 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f504c0a3db0 0x7f504c0a6270 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.943+0000 7f5069c5c640 1 --2- 172.21.15.43:0/3944114967 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5064102190 0x7f5064198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.943+0000 7f5069c5c640 1 -- 172.21.15.43:0/3944114967 >> 172.21.15.43:0/3944114967 conn(0x7f50640793a0 msgr2=0x7f50640ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.943+0000 7f5069c5c640 1 -- 172.21.15.43:0/3944114967 shutdown_connections 2024-04-21T08:39:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:04.943+0000 7f5069c5c640 1 -- 172.21.15.43:0/3944114967 wait complete. 2024-04-21T08:39:04.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:39:04.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:04 smithi175 ceph-mon[31920]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 94 KiB/s wr, 50 op/s 2024-04-21T08:39:04.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:04 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1602352499' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:39:04.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:04 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3468297608' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:39:04.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:04 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3019134007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:39:05.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.060+0000 7fe52986d640 1 -- 172.21.15.43:0/1027744242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe524101090 msgr2=0x7fe524101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:05.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.060+0000 7fe52986d640 1 --2- 172.21.15.43:0/1027744242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe524101090 0x7fe524101470 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe514009900 tx=0x7fe51402bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:05.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.060+0000 7fe52986d640 1 -- 172.21.15.43:0/1027744242 shutdown_connections 2024-04-21T08:39:05.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.060+0000 7fe52986d640 1 --2- 172.21.15.43:0/1027744242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe524106160 0x7fe52410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.060+0000 7fe52986d640 1 --2- 172.21.15.43:0/1027744242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe524101a40 0x7fe524105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.060+0000 7fe52986d640 1 --2- 172.21.15.43:0/1027744242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe524101090 0x7fe524101470 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.060+0000 7fe52986d640 1 -- 172.21.15.43:0/1027744242 >> 172.21.15.43:0/1027744242 conn(0x7fe524077f50 msgr2=0x7fe524078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:05.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.060+0000 7fe52986d640 1 -- 172.21.15.43:0/1027744242 shutdown_connections 2024-04-21T08:39:05.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.060+0000 7fe52986d640 1 -- 172.21.15.43:0/1027744242 wait complete. 2024-04-21T08:39:05.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.060+0000 7fe52986d640 1 Processor -- start 2024-04-21T08:39:05.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.061+0000 7fe52986d640 1 -- start start 2024-04-21T08:39:05.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.061+0000 7fe52986d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe524101090 0x7fe52419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:05.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.061+0000 7fe52986d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe524106160 0x7fe52419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:05.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.061+0000 7fe52986d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5241985b0 0x7fe524198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:05.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.061+0000 7fe52986d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe5241110c0 con 0x7fe524106160 2024-04-21T08:39:05.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.061+0000 7fe52986d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe524110f40 con 0x7fe5241985b0 2024-04-21T08:39:05.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.061+0000 7fe52986d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe524111240 con 0x7fe524101090 2024-04-21T08:39:05.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.061+0000 7fe51bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe524106160 0x7fe52419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:05.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.061+0000 7fe51bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe524106160 0x7fe52419e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39060/0 (socket says 172.21.15.43:39060) 2024-04-21T08:39:05.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.061+0000 7fe51bfff640 1 -- 172.21.15.43:0/816044815 learned_addr learned my addr 172.21.15.43:0/816044815 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:05.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.061+0000 7fe522ffd640 1 --2- 172.21.15.43:0/816044815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe524101090 0x7fe52419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:05.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.061+0000 7fe5237fe640 1 --2- 172.21.15.43:0/816044815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5241985b0 0x7fe524198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:05.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.061+0000 7fe51bfff640 1 -- 172.21.15.43:0/816044815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe524101090 msgr2=0x7fe52419e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:05.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.061+0000 7fe51bfff640 1 --2- 172.21.15.43:0/816044815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe524101090 0x7fe52419e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.061+0000 7fe51bfff640 1 -- 172.21.15.43:0/816044815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5241985b0 msgr2=0x7fe524198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:05.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.061+0000 7fe51bfff640 1 --2- 172.21.15.43:0/816044815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5241985b0 0x7fe524198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.061+0000 7fe51bfff640 1 -- 172.21.15.43:0/816044815 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5241990d0 con 0x7fe524106160 2024-04-21T08:39:05.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.061+0000 7fe51bfff640 1 --2- 172.21.15.43:0/816044815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe524106160 0x7fe52419e940 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe50c00c970 tx=0x7fe50c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:05.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.061+0000 7fe52886b640 1 -- 172.21.15.43:0/816044815 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c007bb0 con 0x7fe524106160 2024-04-21T08:39:05.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.062+0000 7fe52886b640 1 -- 172.21.15.43:0/816044815 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe50c007d50 con 0x7fe524106160 2024-04-21T08:39:05.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.062+0000 7fe52986d640 1 -- 172.21.15.43:0/816044815 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5241993c0 con 0x7fe524106160 2024-04-21T08:39:05.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.062+0000 7fe52886b640 1 -- 172.21.15.43:0/816044815 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c018730 con 0x7fe524106160 2024-04-21T08:39:05.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.062+0000 7fe52986d640 1 -- 172.21.15.43:0/816044815 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5241a5610 con 0x7fe524106160 2024-04-21T08:39:05.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.063+0000 7fe52986d640 1 -- 172.21.15.43:0/816044815 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe524068d70 con 0x7fe524106160 2024-04-21T08:39:05.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.064+0000 7fe52886b640 1 -- 172.21.15.43:0/816044815 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fe50c00f990 con 0x7fe524106160 2024-04-21T08:39:05.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.065+0000 7fe52886b640 1 --2- 172.21.15.43:0/816044815 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe5100a3e30 0x7fe5100a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:05.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.065+0000 7fe522ffd640 1 --2- 172.21.15.43:0/816044815 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe5100a3e30 0x7fe5100a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:05.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.065+0000 7fe52886b640 1 -- 172.21.15.43:0/816044815 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fe50c0d0d60 con 0x7fe524106160 2024-04-21T08:39:05.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.065+0000 7fe522ffd640 1 --2- 172.21.15.43:0/816044815 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe5100a3e30 0x7fe5100a62f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe514006010 tx=0x7fe514005e40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:05.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.070+0000 7fe52886b640 1 -- 172.21.15.43:0/816044815 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe50c014030 con 0x7fe524106160 2024-04-21T08:39:05.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.255+0000 7fe52986d640 1 -- 172.21.15.43:0/816044815 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe524101ad0 con 0x7fe524106160 2024-04-21T08:39:05.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.256+0000 7fe52886b640 1 -- 172.21.15.43:0/816044815 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7fe50c09cec0 con 0x7fe524106160 2024-04-21T08:39:05.255 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:39:05.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.258+0000 7fe52986d640 1 -- 172.21.15.43:0/816044815 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe5100a3e30 msgr2=0x7fe5100a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:05.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.258+0000 7fe52986d640 1 --2- 172.21.15.43:0/816044815 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe5100a3e30 0x7fe5100a62f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe514006010 tx=0x7fe514005e40 comp rx=0 tx=0).stop 2024-04-21T08:39:05.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.258+0000 7fe52986d640 1 -- 172.21.15.43:0/816044815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe524106160 msgr2=0x7fe52419e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:05.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.258+0000 7fe52986d640 1 --2- 172.21.15.43:0/816044815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe524106160 0x7fe52419e940 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe50c00c970 tx=0x7fe50c00ce30 comp rx=0 tx=0).stop 2024-04-21T08:39:05.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.258+0000 7fe52986d640 1 -- 172.21.15.43:0/816044815 shutdown_connections 2024-04-21T08:39:05.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.258+0000 7fe52986d640 1 --2- 172.21.15.43:0/816044815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5241985b0 0x7fe524198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.258+0000 7fe52986d640 1 --2- 172.21.15.43:0/816044815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe524106160 0x7fe52419e940 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.258+0000 7fe52986d640 1 --2- 172.21.15.43:0/816044815 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe5100a3e30 0x7fe5100a62f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.258+0000 7fe52986d640 1 --2- 172.21.15.43:0/816044815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe524101090 0x7fe52419e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.258+0000 7fe52986d640 1 -- 172.21.15.43:0/816044815 >> 172.21.15.43:0/816044815 conn(0x7fe524077f50 msgr2=0x7fe524104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:05.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.258+0000 7fe52986d640 1 -- 172.21.15.43:0/816044815 shutdown_connections 2024-04-21T08:39:05.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.258+0000 7fe52986d640 1 -- 172.21.15.43:0/816044815 wait complete. 2024-04-21T08:39:05.268 INFO:tasks.quiescer.ceph_manager:waiting for clean 2024-04-21T08:39:05.268 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T08:39:05.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.373+0000 7f6a9b255640 1 -- 172.21.15.43:0/1896292929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a941028f0 msgr2=0x7f6a9410edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:05.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.373+0000 7f6a9b255640 1 --2- 172.21.15.43:0/1896292929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a941028f0 0x7f6a9410edb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009ca0 tx=0x7f6a8c02c180 comp rx=0 tx=0).stop 2024-04-21T08:39:05.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.373+0000 7f6a9b255640 1 -- 172.21.15.43:0/1896292929 shutdown_connections 2024-04-21T08:39:05.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.373+0000 7f6a9b255640 1 --2- 172.21.15.43:0/1896292929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a941028f0 0x7f6a9410edb0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.373+0000 7f6a9b255640 1 --2- 172.21.15.43:0/1896292929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a94101f50 0x7f6a941023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.373+0000 7f6a9b255640 1 --2- 172.21.15.43:0/1896292929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a94108010 0x7f6a941083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.373+0000 7f6a9b255640 1 -- 172.21.15.43:0/1896292929 >> 172.21.15.43:0/1896292929 conn(0x7f6a940793e0 msgr2=0x7f6a940797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:05.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.373+0000 7f6a9b255640 1 -- 172.21.15.43:0/1896292929 shutdown_connections 2024-04-21T08:39:05.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.373+0000 7f6a9b255640 1 -- 172.21.15.43:0/1896292929 wait complete. 2024-04-21T08:39:05.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.374+0000 7f6a9b255640 1 Processor -- start 2024-04-21T08:39:05.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.374+0000 7f6a9b255640 1 -- start start 2024-04-21T08:39:05.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.374+0000 7f6a9b255640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a94101f50 0x7f6a941986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:05.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.374+0000 7f6a9b255640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a941028f0 0x7f6a94198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:05.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.374+0000 7f6a9b255640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a94108010 0x7f6a9419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:05.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.374+0000 7f6a9b255640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941121a0 con 0x7f6a941028f0 2024-04-21T08:39:05.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.374+0000 7f6a9b255640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94112020 con 0x7f6a94108010 2024-04-21T08:39:05.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.374+0000 7f6a9b255640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94112320 con 0x7f6a94101f50 2024-04-21T08:39:05.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.374+0000 7f6a93fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a941028f0 0x7f6a94198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:05.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.374+0000 7f6a93fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a941028f0 0x7f6a94198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39076/0 (socket says 172.21.15.43:39076) 2024-04-21T08:39:05.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.374+0000 7f6a93fff640 1 -- 172.21.15.43:0/2703957739 learned_addr learned my addr 172.21.15.43:0/2703957739 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:05.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.375+0000 7f6a98ff1640 1 --2- 172.21.15.43:0/2703957739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a94101f50 0x7f6a941986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:05.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.375+0000 7f6a997f2640 1 --2- 172.21.15.43:0/2703957739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a94108010 0x7f6a9419cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:05.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.375+0000 7f6a93fff640 1 -- 172.21.15.43:0/2703957739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a94101f50 msgr2=0x7f6a941986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:05.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.375+0000 7f6a93fff640 1 --2- 172.21.15.43:0/2703957739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a94101f50 0x7f6a941986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.375+0000 7f6a93fff640 1 -- 172.21.15.43:0/2703957739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a94108010 msgr2=0x7f6a9419cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:05.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.375+0000 7f6a93fff640 1 --2- 172.21.15.43:0/2703957739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a94108010 0x7f6a9419cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.375+0000 7f6a93fff640 1 -- 172.21.15.43:0/2703957739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9419d660 con 0x7f6a941028f0 2024-04-21T08:39:05.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.375+0000 7f6a98ff1640 1 --2- 172.21.15.43:0/2703957739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a94101f50 0x7f6a941986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:05.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.375+0000 7f6a997f2640 1 --2- 172.21.15.43:0/2703957739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a94108010 0x7f6a9419cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:05.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.375+0000 7f6a93fff640 1 --2- 172.21.15.43:0/2703957739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a941028f0 0x7f6a94198c00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6a8000d950 tx=0x7f6a8000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:05.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.375+0000 7f6a91ffb640 1 -- 172.21.15.43:0/2703957739 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80017070 con 0x7f6a941028f0 2024-04-21T08:39:05.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.375+0000 7f6a91ffb640 1 -- 172.21.15.43:0/2703957739 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a80009c30 con 0x7f6a941028f0 2024-04-21T08:39:05.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.375+0000 7f6a9b255640 1 -- 172.21.15.43:0/2703957739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a9419d950 con 0x7f6a941028f0 2024-04-21T08:39:05.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.375+0000 7f6a91ffb640 1 -- 172.21.15.43:0/2703957739 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8000f620 con 0x7f6a941028f0 2024-04-21T08:39:05.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.375+0000 7f6a9b255640 1 -- 172.21.15.43:0/2703957739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a941a51e0 con 0x7f6a941028f0 2024-04-21T08:39:05.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.377+0000 7f6a9b255640 1 -- 172.21.15.43:0/2703957739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a94068de0 con 0x7f6a941028f0 2024-04-21T08:39:05.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.377+0000 7f6a91ffb640 1 -- 172.21.15.43:0/2703957739 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f6a80010460 con 0x7f6a941028f0 2024-04-21T08:39:05.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.378+0000 7f6a91ffb640 1 --2- 172.21.15.43:0/2703957739 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6a840a3e30 0x7f6a840a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:05.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.378+0000 7f6a98ff1640 1 --2- 172.21.15.43:0/2703957739 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6a840a3e30 0x7f6a840a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:05.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.378+0000 7f6a91ffb640 1 -- 172.21.15.43:0/2703957739 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f6a800d0a60 con 0x7f6a941028f0 2024-04-21T08:39:05.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.379+0000 7f6a98ff1640 1 --2- 172.21.15.43:0/2703957739 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6a840a3e30 0x7f6a840a62f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6a880058c0 tx=0x7f6a88005850 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:05.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.382+0000 7f6a91ffb640 1 -- 172.21.15.43:0/2703957739 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a8009cb60 con 0x7f6a941028f0 2024-04-21T08:39:05.559 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:05 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/84915204' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:39:05.560 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:05 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3944114967' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:39:05.560 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:05 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/816044815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:39:05.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.583+0000 7f6a9b255640 1 -- 172.21.15.43:0/2703957739 --> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6a94062390 con 0x7f6a840a3e30 2024-04-21T08:39:05.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.593+0000 7f6a91ffb640 1 -- 172.21.15.43:0/2703957739 <== mgr.15546 v2:172.21.15.43:6800/1482273906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339443 (secure 0 0 0) 0x7f6a94062390 con 0x7f6a840a3e30 2024-04-21T08:39:05.597 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:39:05.599 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-21T08:39:05.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.597+0000 7f6a9b255640 1 -- 172.21.15.43:0/2703957739 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6a840a3e30 msgr2=0x7f6a840a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:05.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.597+0000 7f6a9b255640 1 --2- 172.21.15.43:0/2703957739 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6a840a3e30 0x7f6a840a62f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6a880058c0 tx=0x7f6a88005850 comp rx=0 tx=0).stop 2024-04-21T08:39:05.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.597+0000 7f6a9b255640 1 -- 172.21.15.43:0/2703957739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a941028f0 msgr2=0x7f6a94198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:05.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.597+0000 7f6a9b255640 1 --2- 172.21.15.43:0/2703957739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a941028f0 0x7f6a94198c00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6a8000d950 tx=0x7f6a8000de10 comp rx=0 tx=0).stop 2024-04-21T08:39:05.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.597+0000 7f6a9b255640 1 -- 172.21.15.43:0/2703957739 shutdown_connections 2024-04-21T08:39:05.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.597+0000 7f6a9b255640 1 --2- 172.21.15.43:0/2703957739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a94108010 0x7f6a9419cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.597+0000 7f6a9b255640 1 --2- 172.21.15.43:0/2703957739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a941028f0 0x7f6a94198c00 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.597+0000 7f6a9b255640 1 --2- 172.21.15.43:0/2703957739 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6a840a3e30 0x7f6a840a62f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.597+0000 7f6a9b255640 1 --2- 172.21.15.43:0/2703957739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a94101f50 0x7f6a941986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.597+0000 7f6a9b255640 1 -- 172.21.15.43:0/2703957739 >> 172.21.15.43:0/2703957739 conn(0x7f6a940793e0 msgr2=0x7f6a941009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:05.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.597+0000 7f6a9b255640 1 -- 172.21.15.43:0/2703957739 shutdown_connections 2024-04-21T08:39:05.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.597+0000 7f6a9b255640 1 -- 172.21.15.43:0/2703957739 wait complete. 2024-04-21T08:39:05.608 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-21T08:39:04.903196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1391617,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1795,"num_read_kb":1277,"num_write":648,"num_write_kb":5070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1003,"ondisk_log_size":1003,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386504,"kb_used_data":16520,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124555320,"statfs":{"total":1151940427776,"available":1151544647680,"internally_reserved":0,"allocated":16916480,"data_stored":9972833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":378713404},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":267,"num_read_kb":342,"num_write":88,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.991351"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005622+0000","last_change":"2024-04-21T08:36:23.983921+0000","last_active":"2024-04-21T08:38:55.005622+0000","last_peered":"2024-04-21T08:38:55.005622+0000","last_clean":"2024-04-21T08:38:55.005622+0000","last_became_active":"2024-04-21T08:36:23.983638+0000","last_became_peered":"2024-04-21T08:36:23.983638+0000","last_unstale":"2024-04-21T08:38:55.005622+0000","last_undegraded":"2024-04-21T08:38:55.005622+0000","last_fullsized":"2024-04-21T08:38:55.005622+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:29:44.646308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004430+0000","last_change":"2024-04-21T08:36:23.990029+0000","last_active":"2024-04-21T08:38:55.004430+0000","last_peered":"2024-04-21T08:38:55.004430+0000","last_clean":"2024-04-21T08:38:55.004430+0000","last_became_active":"2024-04-21T08:36:23.989141+0000","last_became_peered":"2024-04-21T08:36:23.989141+0000","last_unstale":"2024-04-21T08:38:55.004430+0000","last_undegraded":"2024-04-21T08:38:55.004430+0000","last_fullsized":"2024-04-21T08:38:55.004430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:23:58.692342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004234+0000","last_change":"2024-04-21T08:36:23.982205+0000","last_active":"2024-04-21T08:38:55.004234+0000","last_peered":"2024-04-21T08:38:55.004234+0000","last_clean":"2024-04-21T08:38:55.004234+0000","last_became_active":"2024-04-21T08:36:23.981876+0000","last_became_peered":"2024-04-21T08:36:23.981876+0000","last_unstale":"2024-04-21T08:38:55.004234+0000","last_undegraded":"2024-04-21T08:38:55.004234+0000","last_fullsized":"2024-04-21T08:38:55.004234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:12:27.121106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006754+0000","last_change":"2024-04-21T08:36:23.987425+0000","last_active":"2024-04-21T08:38:55.006754+0000","last_peered":"2024-04-21T08:38:55.006754+0000","last_clean":"2024-04-21T08:38:55.006754+0000","last_became_active":"2024-04-21T08:36:23.987109+0000","last_became_peered":"2024-04-21T08:36:23.987109+0000","last_unstale":"2024-04-21T08:38:55.006754+0000","last_undegraded":"2024-04-21T08:38:55.006754+0000","last_fullsized":"2024-04-21T08:38:55.006754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:44.176324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890536+0000","last_change":"2024-04-21T08:36:23.990375+0000","last_active":"2024-04-21T08:38:54.890536+0000","last_peered":"2024-04-21T08:38:54.890536+0000","last_clean":"2024-04-21T08:38:54.890536+0000","last_became_active":"2024-04-21T08:36:23.990058+0000","last_became_peered":"2024-04-21T08:36:23.990058+0000","last_unstale":"2024-04-21T08:38:54.890536+0000","last_undegraded":"2024-04-21T08:38:54.890536+0000","last_fullsized":"2024-04-21T08:38:54.890536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:24:09.647528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888221+0000","last_change":"2024-04-21T08:36:23.979848+0000","last_active":"2024-04-21T08:38:54.888221+0000","last_peered":"2024-04-21T08:38:54.888221+0000","last_clean":"2024-04-21T08:38:54.888221+0000","last_became_active":"2024-04-21T08:36:23.979511+0000","last_became_peered":"2024-04-21T08:36:23.979511+0000","last_unstale":"2024-04-21T08:38:54.888221+0000","last_undegraded":"2024-04-21T08:38:54.888221+0000","last_fullsized":"2024-04-21T08:38:54.888221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:07:46.551369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006232+0000","last_change":"2024-04-21T08:37:15.118755+0000","last_active":"2024-04-21T08:38:55.006232+0000","last_peered":"2024-04-21T08:38:55.006232+0000","last_clean":"2024-04-21T08:38:55.006232+0000","last_became_active":"2024-04-21T08:37:15.118347+0000","last_became_peered":"2024-04-21T08:37:15.118347+0000","last_unstale":"2024-04-21T08:38:55.006232+0000","last_undegraded":"2024-04-21T08:38:55.006232+0000","last_fullsized":"2024-04-21T08:38:55.006232+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:36:43.398325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886870+0000","last_change":"2024-04-21T08:36:23.987517+0000","last_active":"2024-04-21T08:38:54.886870+0000","last_peered":"2024-04-21T08:38:54.886870+0000","last_clean":"2024-04-21T08:38:54.886870+0000","last_became_active":"2024-04-21T08:36:23.987201+0000","last_became_peered":"2024-04-21T08:36:23.987201+0000","last_unstale":"2024-04-21T08:38:54.886870+0000","last_undegraded":"2024-04-21T08:38:54.886870+0000","last_fullsized":"2024-04-21T08:38:54.886870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:42:53.080863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104900+0000","last_change":"2024-04-21T08:36:23.989434+0000","last_active":"2024-04-21T08:38:55.104900+0000","last_peered":"2024-04-21T08:38:55.104900+0000","last_clean":"2024-04-21T08:38:55.104900+0000","last_became_active":"2024-04-21T08:36:23.989125+0000","last_became_peered":"2024-04-21T08:36:23.989125+0000","last_unstale":"2024-04-21T08:38:55.104900+0000","last_undegraded":"2024-04-21T08:38:55.104900+0000","last_fullsized":"2024-04-21T08:38:55.104900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:30:07.219056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888449+0000","last_change":"2024-04-21T08:36:23.980725+0000","last_active":"2024-04-21T08:38:54.888449+0000","last_peered":"2024-04-21T08:38:54.888449+0000","last_clean":"2024-04-21T08:38:54.888449+0000","last_became_active":"2024-04-21T08:36:23.980363+0000","last_became_peered":"2024-04-21T08:36:23.980363+0000","last_unstale":"2024-04-21T08:38:54.888449+0000","last_undegraded":"2024-04-21T08:38:54.888449+0000","last_fullsized":"2024-04-21T08:38:54.888449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:26:46.202030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006978+0000","last_change":"2024-04-21T08:36:23.989369+0000","last_active":"2024-04-21T08:38:55.006978+0000","last_peered":"2024-04-21T08:38:55.006978+0000","last_clean":"2024-04-21T08:38:55.006978+0000","last_became_active":"2024-04-21T08:36:23.989030+0000","last_became_peered":"2024-04-21T08:36:23.989030+0000","last_unstale":"2024-04-21T08:38:55.006978+0000","last_undegraded":"2024-04-21T08:38:55.006978+0000","last_fullsized":"2024-04-21T08:38:55.006978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:23:13.725703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006691+0000","last_change":"2024-04-21T08:36:23.993485+0000","last_active":"2024-04-21T08:38:55.006691+0000","last_peered":"2024-04-21T08:38:55.006691+0000","last_clean":"2024-04-21T08:38:55.006691+0000","last_became_active":"2024-04-21T08:36:23.993124+0000","last_became_peered":"2024-04-21T08:36:23.993124+0000","last_unstale":"2024-04-21T08:38:55.006691+0000","last_undegraded":"2024-04-21T08:38:55.006691+0000","last_fullsized":"2024-04-21T08:38:55.006691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:53:16.249838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076569+0000","last_change":"2024-04-21T08:36:23.983146+0000","last_active":"2024-04-21T08:38:55.076569+0000","last_peered":"2024-04-21T08:38:55.076569+0000","last_clean":"2024-04-21T08:38:55.076569+0000","last_became_active":"2024-04-21T08:36:23.982766+0000","last_became_peered":"2024-04-21T08:36:23.982766+0000","last_unstale":"2024-04-21T08:38:55.076569+0000","last_undegraded":"2024-04-21T08:38:55.076569+0000","last_fullsized":"2024-04-21T08:38:55.076569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:30:44.090929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.003129+0000","last_change":"2024-04-21T08:36:23.988906+0000","last_active":"2024-04-21T08:38:55.003129+0000","last_peered":"2024-04-21T08:38:55.003129+0000","last_clean":"2024-04-21T08:38:55.003129+0000","last_became_active":"2024-04-21T08:36:23.988489+0000","last_became_peered":"2024-04-21T08:36:23.988489+0000","last_unstale":"2024-04-21T08:38:55.003129+0000","last_undegraded":"2024-04-21T08:38:55.003129+0000","last_fullsized":"2024-04-21T08:38:55.003129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:06:43.271650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.003060+0000","last_change":"2024-04-21T08:37:15.123343+0000","last_active":"2024-04-21T08:38:55.003060+0000","last_peered":"2024-04-21T08:38:55.003060+0000","last_clean":"2024-04-21T08:38:55.003060+0000","last_became_active":"2024-04-21T08:37:15.122959+0000","last_became_peered":"2024-04-21T08:37:15.122959+0000","last_unstale":"2024-04-21T08:38:55.003060+0000","last_undegraded":"2024-04-21T08:38:55.003060+0000","last_fullsized":"2024-04-21T08:38:55.003060+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:56:19.179879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104471+0000","last_change":"2024-04-21T08:36:23.983136+0000","last_active":"2024-04-21T08:38:55.104471+0000","last_peered":"2024-04-21T08:38:55.104471+0000","last_clean":"2024-04-21T08:38:55.104471+0000","last_became_active":"2024-04-21T08:36:23.982698+0000","last_became_peered":"2024-04-21T08:36:23.982698+0000","last_unstale":"2024-04-21T08:38:55.104471+0000","last_undegraded":"2024-04-21T08:38:55.104471+0000","last_fullsized":"2024-04-21T08:38:55.104471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:34.750941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005563+0000","last_change":"2024-04-21T08:36:23.984701+0000","last_active":"2024-04-21T08:38:55.005563+0000","last_peered":"2024-04-21T08:38:55.005563+0000","last_clean":"2024-04-21T08:38:55.005563+0000","last_became_active":"2024-04-21T08:36:23.984432+0000","last_became_peered":"2024-04-21T08:36:23.984432+0000","last_unstale":"2024-04-21T08:38:55.005563+0000","last_undegraded":"2024-04-21T08:38:55.005563+0000","last_fullsized":"2024-04-21T08:38:55.005563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:10:58.944380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.996823+0000","last_change":"2024-04-21T08:36:23.982781+0000","last_active":"2024-04-21T08:38:54.996823+0000","last_peered":"2024-04-21T08:38:54.996823+0000","last_clean":"2024-04-21T08:38:54.996823+0000","last_became_active":"2024-04-21T08:36:23.982459+0000","last_became_peered":"2024-04-21T08:36:23.982459+0000","last_unstale":"2024-04-21T08:38:54.996823+0000","last_undegraded":"2024-04-21T08:38:54.996823+0000","last_fullsized":"2024-04-21T08:38:54.996823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:44.657504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.105000+0000","last_change":"2024-04-21T08:36:23.992002+0000","last_active":"2024-04-21T08:38:55.105000+0000","last_peered":"2024-04-21T08:38:55.105000+0000","last_clean":"2024-04-21T08:38:55.105000+0000","last_became_active":"2024-04-21T08:36:23.991053+0000","last_became_peered":"2024-04-21T08:36:23.991053+0000","last_unstale":"2024-04-21T08:38:55.105000+0000","last_undegraded":"2024-04-21T08:38:55.105000+0000","last_fullsized":"2024-04-21T08:38:55.105000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:56:37.145949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104888+0000","last_change":"2024-04-21T08:36:28.013195+0000","last_active":"2024-04-21T08:38:55.104888+0000","last_peered":"2024-04-21T08:38:55.104888+0000","last_clean":"2024-04-21T08:38:55.104888+0000","last_became_active":"2024-04-21T08:36:28.012656+0000","last_became_peered":"2024-04-21T08:36:28.012656+0000","last_unstale":"2024-04-21T08:38:55.104888+0000","last_undegraded":"2024-04-21T08:38:55.104888+0000","last_fullsized":"2024-04-21T08:38:55.104888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:12:03.133583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"95'14","reported_seq":179,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:39:00.080989+0000","last_change":"2024-04-21T08:36:23.468957+0000","last_active":"2024-04-21T08:39:00.080989+0000","last_peered":"2024-04-21T08:39:00.080989+0000","last_clean":"2024-04-21T08:39:00.080989+0000","last_became_active":"2024-04-21T08:36:23.468601+0000","last_became_peered":"2024-04-21T08:36:23.468601+0000","last_unstale":"2024-04-21T08:39:00.080989+0000","last_undegraded":"2024-04-21T08:39:00.080989+0000","last_fullsized":"2024-04-21T08:39:00.080989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:13:11.977780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":131,"num_read_kb":131,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.996013+0000","last_change":"2024-04-21T08:37:15.123476+0000","last_active":"2024-04-21T08:38:54.996013+0000","last_peered":"2024-04-21T08:38:54.996013+0000","last_clean":"2024-04-21T08:38:54.996013+0000","last_became_active":"2024-04-21T08:37:15.123066+0000","last_became_peered":"2024-04-21T08:37:15.123066+0000","last_unstale":"2024-04-21T08:38:54.996013+0000","last_undegraded":"2024-04-21T08:38:54.996013+0000","last_fullsized":"2024-04-21T08:38:54.996013+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:15.786178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005900+0000","last_change":"2024-04-21T08:36:28.014161+0000","last_active":"2024-04-21T08:38:55.005900+0000","last_peered":"2024-04-21T08:38:55.005900+0000","last_clean":"2024-04-21T08:38:55.005900+0000","last_became_active":"2024-04-21T08:36:28.013859+0000","last_became_peered":"2024-04-21T08:36:28.013859+0000","last_unstale":"2024-04-21T08:38:55.005900+0000","last_undegraded":"2024-04-21T08:38:55.005900+0000","last_fullsized":"2024-04-21T08:38:55.005900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:08.315334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887665+0000","last_change":"2024-04-21T08:36:23.475612+0000","last_active":"2024-04-21T08:38:54.887665+0000","last_peered":"2024-04-21T08:38:54.887665+0000","last_clean":"2024-04-21T08:38:54.887665+0000","last_became_active":"2024-04-21T08:36:23.475301+0000","last_became_peered":"2024-04-21T08:36:23.475301+0000","last_unstale":"2024-04-21T08:38:54.887665+0000","last_undegraded":"2024-04-21T08:38:54.887665+0000","last_fullsized":"2024-04-21T08:38:54.887665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:56.801630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004414+0000","last_change":"2024-04-21T08:36:23.984689+0000","last_active":"2024-04-21T08:38:55.004414+0000","last_peered":"2024-04-21T08:38:55.004414+0000","last_clean":"2024-04-21T08:38:55.004414+0000","last_became_active":"2024-04-21T08:36:23.984358+0000","last_became_peered":"2024-04-21T08:36:23.984358+0000","last_unstale":"2024-04-21T08:38:55.004414+0000","last_undegraded":"2024-04-21T08:38:55.004414+0000","last_fullsized":"2024-04-21T08:38:55.004414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:49:50.187466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006257+0000","last_change":"2024-04-21T08:36:28.007951+0000","last_active":"2024-04-21T08:38:55.006257+0000","last_peered":"2024-04-21T08:38:55.006257+0000","last_clean":"2024-04-21T08:38:55.006257+0000","last_became_active":"2024-04-21T08:36:28.007553+0000","last_became_peered":"2024-04-21T08:36:28.007553+0000","last_unstale":"2024-04-21T08:38:55.006257+0000","last_undegraded":"2024-04-21T08:38:55.006257+0000","last_fullsized":"2024-04-21T08:38:55.006257+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:10:54.019605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890572+0000","last_change":"2024-04-21T08:36:23.473371+0000","last_active":"2024-04-21T08:38:54.890572+0000","last_peered":"2024-04-21T08:38:54.890572+0000","last_clean":"2024-04-21T08:38:54.890572+0000","last_became_active":"2024-04-21T08:36:23.472472+0000","last_became_peered":"2024-04-21T08:36:23.472472+0000","last_unstale":"2024-04-21T08:38:54.890572+0000","last_undegraded":"2024-04-21T08:38:54.890572+0000","last_fullsized":"2024-04-21T08:38:54.890572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:28:49.622873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104901+0000","last_change":"2024-04-21T08:36:23.987916+0000","last_active":"2024-04-21T08:38:55.104901+0000","last_peered":"2024-04-21T08:38:55.104901+0000","last_clean":"2024-04-21T08:38:55.104901+0000","last_became_active":"2024-04-21T08:36:23.986207+0000","last_became_peered":"2024-04-21T08:36:23.986207+0000","last_unstale":"2024-04-21T08:38:55.104901+0000","last_undegraded":"2024-04-21T08:38:55.104901+0000","last_fullsized":"2024-04-21T08:38:55.104901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:49:57.261078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006246+0000","last_change":"2024-04-21T08:36:28.006494+0000","last_active":"2024-04-21T08:38:55.006246+0000","last_peered":"2024-04-21T08:38:55.006246+0000","last_clean":"2024-04-21T08:38:55.006246+0000","last_became_active":"2024-04-21T08:36:28.006069+0000","last_became_peered":"2024-04-21T08:36:28.006069+0000","last_unstale":"2024-04-21T08:38:55.006246+0000","last_undegraded":"2024-04-21T08:38:55.006246+0000","last_fullsized":"2024-04-21T08:38:55.006246+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:04.608295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890975+0000","last_change":"2024-04-21T08:36:23.468923+0000","last_active":"2024-04-21T08:38:54.890975+0000","last_peered":"2024-04-21T08:38:54.890975+0000","last_clean":"2024-04-21T08:38:54.890975+0000","last_became_active":"2024-04-21T08:36:23.468509+0000","last_became_peered":"2024-04-21T08:36:23.468509+0000","last_unstale":"2024-04-21T08:38:54.890975+0000","last_undegraded":"2024-04-21T08:38:54.890975+0000","last_fullsized":"2024-04-21T08:38:54.890975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:44.452748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008259+0000","last_change":"2024-04-21T08:36:23.994423+0000","last_active":"2024-04-21T08:38:55.008259+0000","last_peered":"2024-04-21T08:38:55.008259+0000","last_clean":"2024-04-21T08:38:55.008259+0000","last_became_active":"2024-04-21T08:36:23.993827+0000","last_became_peered":"2024-04-21T08:36:23.993827+0000","last_unstale":"2024-04-21T08:38:55.008259+0000","last_undegraded":"2024-04-21T08:38:55.008259+0000","last_fullsized":"2024-04-21T08:38:55.008259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:49:25.550301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104894+0000","last_change":"2024-04-21T08:36:28.004037+0000","last_active":"2024-04-21T08:38:55.104894+0000","last_peered":"2024-04-21T08:38:55.104894+0000","last_clean":"2024-04-21T08:38:55.104894+0000","last_became_active":"2024-04-21T08:36:28.003580+0000","last_became_peered":"2024-04-21T08:36:28.003580+0000","last_unstale":"2024-04-21T08:38:55.104894+0000","last_undegraded":"2024-04-21T08:38:55.104894+0000","last_fullsized":"2024-04-21T08:38:55.104894+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:04:51.380244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'8","reported_seq":178,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:59.079811+0000","last_change":"2024-04-21T08:36:23.468430+0000","last_active":"2024-04-21T08:38:59.079811+0000","last_peered":"2024-04-21T08:38:59.079811+0000","last_clean":"2024-04-21T08:38:59.079811+0000","last_became_active":"2024-04-21T08:36:23.468089+0000","last_became_peered":"2024-04-21T08:36:23.468089+0000","last_unstale":"2024-04-21T08:38:59.079811+0000","last_undegraded":"2024-04-21T08:38:59.079811+0000","last_fullsized":"2024-04-21T08:38:59.079811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:12:54.827034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890458+0000","last_change":"2024-04-21T08:37:15.117922+0000","last_active":"2024-04-21T08:38:54.890458+0000","last_peered":"2024-04-21T08:38:54.890458+0000","last_clean":"2024-04-21T08:38:54.890458+0000","last_became_active":"2024-04-21T08:37:15.117404+0000","last_became_peered":"2024-04-21T08:37:15.117404+0000","last_unstale":"2024-04-21T08:38:54.890458+0000","last_undegraded":"2024-04-21T08:38:54.890458+0000","last_fullsized":"2024-04-21T08:38:54.890458+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:39.682224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104742+0000","last_change":"2024-04-21T08:36:28.004638+0000","last_active":"2024-04-21T08:38:55.104742+0000","last_peered":"2024-04-21T08:38:55.104742+0000","last_clean":"2024-04-21T08:38:55.104742+0000","last_became_active":"2024-04-21T08:36:28.002499+0000","last_became_peered":"2024-04-21T08:36:28.002499+0000","last_unstale":"2024-04-21T08:38:55.104742+0000","last_undegraded":"2024-04-21T08:38:55.104742+0000","last_fullsized":"2024-04-21T08:38:55.104742+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:45:46.934783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":55,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888089+0000","last_change":"2024-04-21T08:37:15.126569+0000","last_active":"2024-04-21T08:38:54.888089+0000","last_peered":"2024-04-21T08:38:54.888089+0000","last_clean":"2024-04-21T08:38:54.888089+0000","last_became_active":"2024-04-21T08:37:15.121284+0000","last_became_peered":"2024-04-21T08:37:15.121284+0000","last_unstale":"2024-04-21T08:38:54.888089+0000","last_undegraded":"2024-04-21T08:38:54.888089+0000","last_fullsized":"2024-04-21T08:38:54.888089+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:02:26.153324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008374+0000","last_change":"2024-04-21T08:37:15.557192+0000","last_active":"2024-04-21T08:38:55.008374+0000","last_peered":"2024-04-21T08:38:55.008374+0000","last_clean":"2024-04-21T08:38:55.008374+0000","last_became_active":"2024-04-21T08:37:15.556653+0000","last_became_peered":"2024-04-21T08:37:15.556653+0000","last_unstale":"2024-04-21T08:38:55.008374+0000","last_undegraded":"2024-04-21T08:38:55.008374+0000","last_fullsized":"2024-04-21T08:38:55.008374+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:42.722460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005527+0000","last_change":"2024-04-21T08:36:28.017068+0000","last_active":"2024-04-21T08:38:55.005527+0000","last_peered":"2024-04-21T08:38:55.005527+0000","last_clean":"2024-04-21T08:38:55.005527+0000","last_became_active":"2024-04-21T08:36:28.016770+0000","last_became_peered":"2024-04-21T08:36:28.016770+0000","last_unstale":"2024-04-21T08:38:55.005527+0000","last_undegraded":"2024-04-21T08:38:55.005527+0000","last_fullsized":"2024-04-21T08:38:55.005527+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:07:14.596090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076311+0000","last_change":"2024-04-21T08:36:23.470430+0000","last_active":"2024-04-21T08:38:55.076311+0000","last_peered":"2024-04-21T08:38:55.076311+0000","last_clean":"2024-04-21T08:38:55.076311+0000","last_became_active":"2024-04-21T08:36:23.470109+0000","last_became_peered":"2024-04-21T08:36:23.470109+0000","last_unstale":"2024-04-21T08:38:55.076311+0000","last_undegraded":"2024-04-21T08:38:55.076311+0000","last_fullsized":"2024-04-21T08:38:55.076311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:18.176313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887026+0000","last_change":"2024-04-21T08:36:23.986668+0000","last_active":"2024-04-21T08:38:54.887026+0000","last_peered":"2024-04-21T08:38:54.887026+0000","last_clean":"2024-04-21T08:38:54.887026+0000","last_became_active":"2024-04-21T08:36:23.986344+0000","last_became_peered":"2024-04-21T08:36:23.986344+0000","last_unstale":"2024-04-21T08:38:54.887026+0000","last_undegraded":"2024-04-21T08:38:54.887026+0000","last_fullsized":"2024-04-21T08:38:54.887026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:56:17.744145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004263+0000","last_change":"2024-04-21T08:36:28.009012+0000","last_active":"2024-04-21T08:38:55.004263+0000","last_peered":"2024-04-21T08:38:55.004263+0000","last_clean":"2024-04-21T08:38:55.004263+0000","last_became_active":"2024-04-21T08:36:28.008123+0000","last_became_peered":"2024-04-21T08:36:28.008123+0000","last_unstale":"2024-04-21T08:38:55.004263+0000","last_undegraded":"2024-04-21T08:38:55.004263+0000","last_fullsized":"2024-04-21T08:38:55.004263+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:33:18.881564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007854+0000","last_change":"2024-04-21T08:36:23.467104+0000","last_active":"2024-04-21T08:38:55.007854+0000","last_peered":"2024-04-21T08:38:55.007854+0000","last_clean":"2024-04-21T08:38:55.007854+0000","last_became_active":"2024-04-21T08:36:23.466725+0000","last_became_peered":"2024-04-21T08:36:23.466725+0000","last_unstale":"2024-04-21T08:38:55.007854+0000","last_undegraded":"2024-04-21T08:38:55.007854+0000","last_fullsized":"2024-04-21T08:38:55.007854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:35:45.986448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.995844+0000","last_change":"2024-04-21T08:36:23.987579+0000","last_active":"2024-04-21T08:38:54.995844+0000","last_peered":"2024-04-21T08:38:54.995844+0000","last_clean":"2024-04-21T08:38:54.995844+0000","last_became_active":"2024-04-21T08:36:23.987044+0000","last_became_peered":"2024-04-21T08:36:23.987044+0000","last_unstale":"2024-04-21T08:38:54.995844+0000","last_undegraded":"2024-04-21T08:38:54.995844+0000","last_fullsized":"2024-04-21T08:38:54.995844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:48:27.593090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886895+0000","last_change":"2024-04-21T08:36:28.006417+0000","last_active":"2024-04-21T08:38:54.886895+0000","last_peered":"2024-04-21T08:38:54.886895+0000","last_clean":"2024-04-21T08:38:54.886895+0000","last_became_active":"2024-04-21T08:36:28.006006+0000","last_became_peered":"2024-04-21T08:36:28.006006+0000","last_unstale":"2024-04-21T08:38:54.886895+0000","last_undegraded":"2024-04-21T08:38:54.886895+0000","last_fullsized":"2024-04-21T08:38:54.886895+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:16.173284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'8","reported_seq":302,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:39:00.080410+0000","last_change":"2024-04-21T08:36:23.471541+0000","last_active":"2024-04-21T08:39:00.080410+0000","last_peered":"2024-04-21T08:39:00.080410+0000","last_clean":"2024-04-21T08:39:00.080410+0000","last_became_active":"2024-04-21T08:36:23.471193+0000","last_became_peered":"2024-04-21T08:36:23.471193+0000","last_unstale":"2024-04-21T08:39:00.080410+0000","last_undegraded":"2024-04-21T08:39:00.080410+0000","last_fullsized":"2024-04-21T08:39:00.080410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:06:04.916707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":260,"num_read_kb":260,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104848+0000","last_change":"2024-04-21T08:36:23.986781+0000","last_active":"2024-04-21T08:38:55.104848+0000","last_peered":"2024-04-21T08:38:55.104848+0000","last_clean":"2024-04-21T08:38:55.104848+0000","last_became_active":"2024-04-21T08:36:23.985696+0000","last_became_peered":"2024-04-21T08:36:23.985696+0000","last_unstale":"2024-04-21T08:38:55.104848+0000","last_undegraded":"2024-04-21T08:38:55.104848+0000","last_fullsized":"2024-04-21T08:38:55.104848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:40.556642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.995747+0000","last_change":"2024-04-21T08:36:28.011621+0000","last_active":"2024-04-21T08:38:54.995747+0000","last_peered":"2024-04-21T08:38:54.995747+0000","last_clean":"2024-04-21T08:38:54.995747+0000","last_became_active":"2024-04-21T08:36:28.011136+0000","last_became_peered":"2024-04-21T08:36:28.011136+0000","last_unstale":"2024-04-21T08:38:54.995747+0000","last_undegraded":"2024-04-21T08:38:54.995747+0000","last_fullsized":"2024-04-21T08:38:54.995747+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:47.623445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006842+0000","last_change":"2024-04-21T08:36:23.474019+0000","last_active":"2024-04-21T08:38:55.006842+0000","last_peered":"2024-04-21T08:38:55.006842+0000","last_clean":"2024-04-21T08:38:55.006842+0000","last_became_active":"2024-04-21T08:36:23.473690+0000","last_became_peered":"2024-04-21T08:36:23.473690+0000","last_unstale":"2024-04-21T08:38:55.006842+0000","last_undegraded":"2024-04-21T08:38:55.006842+0000","last_fullsized":"2024-04-21T08:38:55.006842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:38:32.913795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004613+0000","last_change":"2024-04-21T08:37:15.121455+0000","last_active":"2024-04-21T08:38:55.004613+0000","last_peered":"2024-04-21T08:38:55.004613+0000","last_clean":"2024-04-21T08:38:55.004613+0000","last_became_active":"2024-04-21T08:37:15.121113+0000","last_became_peered":"2024-04-21T08:37:15.121113+0000","last_unstale":"2024-04-21T08:38:55.004613+0000","last_undegraded":"2024-04-21T08:38:55.004613+0000","last_fullsized":"2024-04-21T08:38:55.004613+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:39:36.525495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006512+0000","last_change":"2024-04-21T08:38:15.413849+0000","last_active":"2024-04-21T08:38:55.006512+0000","last_peered":"2024-04-21T08:38:55.006512+0000","last_clean":"2024-04-21T08:38:55.006512+0000","last_became_active":"2024-04-21T08:38:15.413361+0000","last_became_peered":"2024-04-21T08:38:15.413361+0000","last_unstale":"2024-04-21T08:38:55.006512+0000","last_undegraded":"2024-04-21T08:38:55.006512+0000","last_fullsized":"2024-04-21T08:38:55.006512+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:16:49.197800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008173+0000","last_change":"2024-04-21T08:36:23.465825+0000","last_active":"2024-04-21T08:38:55.008173+0000","last_peered":"2024-04-21T08:38:55.008173+0000","last_clean":"2024-04-21T08:38:55.008173+0000","last_became_active":"2024-04-21T08:36:23.465525+0000","last_became_peered":"2024-04-21T08:36:23.465525+0000","last_unstale":"2024-04-21T08:38:55.008173+0000","last_undegraded":"2024-04-21T08:38:55.008173+0000","last_fullsized":"2024-04-21T08:38:55.008173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:59:50.502210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.105005+0000","last_change":"2024-04-21T08:36:23.987602+0000","last_active":"2024-04-21T08:38:55.105005+0000","last_peered":"2024-04-21T08:38:55.105005+0000","last_clean":"2024-04-21T08:38:55.105005+0000","last_became_active":"2024-04-21T08:36:23.987298+0000","last_became_peered":"2024-04-21T08:36:23.987298+0000","last_unstale":"2024-04-21T08:38:55.105005+0000","last_undegraded":"2024-04-21T08:38:55.105005+0000","last_fullsized":"2024-04-21T08:38:55.105005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:06.647676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.995946+0000","last_change":"2024-04-21T08:36:28.005935+0000","last_active":"2024-04-21T08:38:54.995946+0000","last_peered":"2024-04-21T08:38:54.995946+0000","last_clean":"2024-04-21T08:38:54.995946+0000","last_became_active":"2024-04-21T08:36:28.005512+0000","last_became_peered":"2024-04-21T08:36:28.005512+0000","last_unstale":"2024-04-21T08:38:54.995946+0000","last_undegraded":"2024-04-21T08:38:54.995946+0000","last_fullsized":"2024-04-21T08:38:54.995946+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:23:20.298524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"95'43","reported_seq":353,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:39:00.080992+0000","last_change":"2024-04-21T08:36:23.467728+0000","last_active":"2024-04-21T08:39:00.080992+0000","last_peered":"2024-04-21T08:39:00.080992+0000","last_clean":"2024-04-21T08:39:00.080992+0000","last_became_active":"2024-04-21T08:36:23.467430+0000","last_became_peered":"2024-04-21T08:36:23.467430+0000","last_unstale":"2024-04-21T08:39:00.080992+0000","last_undegraded":"2024-04-21T08:39:00.080992+0000","last_fullsized":"2024-04-21T08:39:00.080992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:47:18.900911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70068,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":276,"num_read_kb":208,"num_write":45,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890735+0000","last_change":"2024-04-21T08:36:23.989116+0000","last_active":"2024-04-21T08:38:54.890735+0000","last_peered":"2024-04-21T08:38:54.890735+0000","last_clean":"2024-04-21T08:38:54.890735+0000","last_became_active":"2024-04-21T08:36:23.988758+0000","last_became_peered":"2024-04-21T08:36:23.988758+0000","last_unstale":"2024-04-21T08:38:54.890735+0000","last_undegraded":"2024-04-21T08:38:54.890735+0000","last_fullsized":"2024-04-21T08:38:54.890735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:56:25.969905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008450+0000","last_change":"2024-04-21T08:36:28.007370+0000","last_active":"2024-04-21T08:38:55.008450+0000","last_peered":"2024-04-21T08:38:55.008450+0000","last_clean":"2024-04-21T08:38:55.008450+0000","last_became_active":"2024-04-21T08:36:28.006972+0000","last_became_peered":"2024-04-21T08:36:28.006972+0000","last_unstale":"2024-04-21T08:38:55.008450+0000","last_undegraded":"2024-04-21T08:38:55.008450+0000","last_fullsized":"2024-04-21T08:38:55.008450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:18:14.926058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104708+0000","last_change":"2024-04-21T08:36:23.472658+0000","last_active":"2024-04-21T08:38:55.104708+0000","last_peered":"2024-04-21T08:38:55.104708+0000","last_clean":"2024-04-21T08:38:55.104708+0000","last_became_active":"2024-04-21T08:36:23.472318+0000","last_became_peered":"2024-04-21T08:36:23.472318+0000","last_unstale":"2024-04-21T08:38:55.104708+0000","last_undegraded":"2024-04-21T08:38:55.104708+0000","last_fullsized":"2024-04-21T08:38:55.104708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:28:27.497605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076501+0000","last_change":"2024-04-21T08:38:15.338892+0000","last_active":"2024-04-21T08:38:55.076501+0000","last_peered":"2024-04-21T08:38:55.076501+0000","last_clean":"2024-04-21T08:38:55.076501+0000","last_became_active":"2024-04-21T08:38:15.338396+0000","last_became_peered":"2024-04-21T08:38:15.338396+0000","last_unstale":"2024-04-21T08:38:55.076501+0000","last_undegraded":"2024-04-21T08:38:55.076501+0000","last_fullsized":"2024-04-21T08:38:55.076501+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:19.670635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005342+0000","last_change":"2024-04-21T08:36:28.019186+0000","last_active":"2024-04-21T08:38:55.005342+0000","last_peered":"2024-04-21T08:38:55.005342+0000","last_clean":"2024-04-21T08:38:55.005342+0000","last_became_active":"2024-04-21T08:36:28.018774+0000","last_became_peered":"2024-04-21T08:36:28.018774+0000","last_unstale":"2024-04-21T08:38:55.005342+0000","last_undegraded":"2024-04-21T08:38:55.005342+0000","last_fullsized":"2024-04-21T08:38:55.005342+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:35:33.548656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008059+0000","last_change":"2024-04-21T08:36:23.472633+0000","last_active":"2024-04-21T08:38:55.008059+0000","last_peered":"2024-04-21T08:38:55.008059+0000","last_clean":"2024-04-21T08:38:55.008059+0000","last_became_active":"2024-04-21T08:36:23.472336+0000","last_became_peered":"2024-04-21T08:36:23.472336+0000","last_unstale":"2024-04-21T08:38:55.008059+0000","last_undegraded":"2024-04-21T08:38:55.008059+0000","last_fullsized":"2024-04-21T08:38:55.008059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:49.463492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006193+0000","last_change":"2024-04-21T08:36:23.994332+0000","last_active":"2024-04-21T08:38:55.006193+0000","last_peered":"2024-04-21T08:38:55.006193+0000","last_clean":"2024-04-21T08:38:55.006193+0000","last_became_active":"2024-04-21T08:36:23.993936+0000","last_became_peered":"2024-04-21T08:36:23.993936+0000","last_unstale":"2024-04-21T08:38:55.006193+0000","last_undegraded":"2024-04-21T08:38:55.006193+0000","last_fullsized":"2024-04-21T08:38:55.006193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:26:35.330559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.996887+0000","last_change":"2024-04-21T08:36:28.012549+0000","last_active":"2024-04-21T08:38:54.996887+0000","last_peered":"2024-04-21T08:38:54.996887+0000","last_clean":"2024-04-21T08:38:54.996887+0000","last_became_active":"2024-04-21T08:36:28.012151+0000","last_became_peered":"2024-04-21T08:36:28.012151+0000","last_unstale":"2024-04-21T08:38:54.996887+0000","last_undegraded":"2024-04-21T08:38:54.996887+0000","last_fullsized":"2024-04-21T08:38:54.996887+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:55.680399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'8","reported_seq":178,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:59.081251+0000","last_change":"2024-04-21T08:36:23.464586+0000","last_active":"2024-04-21T08:38:59.081251+0000","last_peered":"2024-04-21T08:38:59.081251+0000","last_clean":"2024-04-21T08:38:59.081251+0000","last_became_active":"2024-04-21T08:36:23.464281+0000","last_became_peered":"2024-04-21T08:36:23.464281+0000","last_unstale":"2024-04-21T08:38:59.081251+0000","last_undegraded":"2024-04-21T08:38:59.081251+0000","last_fullsized":"2024-04-21T08:38:59.081251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:04:58.577283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":137,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006132+0000","last_change":"2024-04-21T08:36:23.989604+0000","last_active":"2024-04-21T08:38:55.006132+0000","last_peered":"2024-04-21T08:38:55.006132+0000","last_clean":"2024-04-21T08:38:55.006132+0000","last_became_active":"2024-04-21T08:36:23.989257+0000","last_became_peered":"2024-04-21T08:36:23.989257+0000","last_unstale":"2024-04-21T08:38:55.006132+0000","last_undegraded":"2024-04-21T08:38:55.006132+0000","last_fullsized":"2024-04-21T08:38:55.006132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:45.077943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077854+0000","last_change":"2024-04-21T08:36:28.007959+0000","last_active":"2024-04-21T08:38:55.077854+0000","last_peered":"2024-04-21T08:38:55.077854+0000","last_clean":"2024-04-21T08:38:55.077854+0000","last_became_active":"2024-04-21T08:36:28.007571+0000","last_became_peered":"2024-04-21T08:36:28.007571+0000","last_unstale":"2024-04-21T08:38:55.077854+0000","last_undegraded":"2024-04-21T08:38:55.077854+0000","last_fullsized":"2024-04-21T08:38:55.077854+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:17:34.007013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890204+0000","last_change":"2024-04-21T08:36:23.475054+0000","last_active":"2024-04-21T08:38:54.890204+0000","last_peered":"2024-04-21T08:38:54.890204+0000","last_clean":"2024-04-21T08:38:54.890204+0000","last_became_active":"2024-04-21T08:36:23.474606+0000","last_became_peered":"2024-04-21T08:36:23.474606+0000","last_unstale":"2024-04-21T08:38:54.890204+0000","last_undegraded":"2024-04-21T08:38:54.890204+0000","last_fullsized":"2024-04-21T08:38:54.890204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:22.559174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007909+0000","last_change":"2024-04-21T08:36:23.995980+0000","last_active":"2024-04-21T08:38:55.007909+0000","last_peered":"2024-04-21T08:38:55.007909+0000","last_clean":"2024-04-21T08:38:55.007909+0000","last_became_active":"2024-04-21T08:36:23.995673+0000","last_became_peered":"2024-04-21T08:36:23.995673+0000","last_unstale":"2024-04-21T08:38:55.007909+0000","last_undegraded":"2024-04-21T08:38:55.007909+0000","last_fullsized":"2024-04-21T08:38:55.007909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:11:49.884384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007831+0000","last_change":"2024-04-21T08:36:28.018792+0000","last_active":"2024-04-21T08:38:55.007831+0000","last_peered":"2024-04-21T08:38:55.007831+0000","last_clean":"2024-04-21T08:38:55.007831+0000","last_became_active":"2024-04-21T08:36:28.018373+0000","last_became_peered":"2024-04-21T08:36:28.018373+0000","last_unstale":"2024-04-21T08:38:55.007831+0000","last_undegraded":"2024-04-21T08:38:55.007831+0000","last_fullsized":"2024-04-21T08:38:55.007831+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:11.857661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"95'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890811+0000","last_change":"2024-04-21T08:36:23.469787+0000","last_active":"2024-04-21T08:38:54.890811+0000","last_peered":"2024-04-21T08:38:54.890811+0000","last_clean":"2024-04-21T08:38:54.890811+0000","last_became_active":"2024-04-21T08:36:23.469462+0000","last_became_peered":"2024-04-21T08:36:23.469462+0000","last_unstale":"2024-04-21T08:38:54.890811+0000","last_undegraded":"2024-04-21T08:38:54.890811+0000","last_fullsized":"2024-04-21T08:38:54.890811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:33:25.849528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005561+0000","last_change":"2024-04-21T08:36:23.988492+0000","last_active":"2024-04-21T08:38:55.005561+0000","last_peered":"2024-04-21T08:38:55.005561+0000","last_clean":"2024-04-21T08:38:55.005561+0000","last_became_active":"2024-04-21T08:36:23.988133+0000","last_became_peered":"2024-04-21T08:36:23.988133+0000","last_unstale":"2024-04-21T08:38:55.005561+0000","last_undegraded":"2024-04-21T08:38:55.005561+0000","last_fullsized":"2024-04-21T08:38:55.005561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:01:25.320281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890332+0000","last_change":"2024-04-21T08:38:15.415043+0000","last_active":"2024-04-21T08:38:54.890332+0000","last_peered":"2024-04-21T08:38:54.890332+0000","last_clean":"2024-04-21T08:38:54.890332+0000","last_became_active":"2024-04-21T08:38:15.414460+0000","last_became_peered":"2024-04-21T08:38:15.414460+0000","last_unstale":"2024-04-21T08:38:54.890332+0000","last_undegraded":"2024-04-21T08:38:54.890332+0000","last_fullsized":"2024-04-21T08:38:54.890332+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:50:17.202423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.996393+0000","last_change":"2024-04-21T08:37:15.219428+0000","last_active":"2024-04-21T08:38:54.996393+0000","last_peered":"2024-04-21T08:38:54.996393+0000","last_clean":"2024-04-21T08:38:54.996393+0000","last_became_active":"2024-04-21T08:37:15.218923+0000","last_became_peered":"2024-04-21T08:37:15.218923+0000","last_unstale":"2024-04-21T08:38:54.996393+0000","last_undegraded":"2024-04-21T08:38:54.996393+0000","last_fullsized":"2024-04-21T08:38:54.996393+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:07:02.781701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888120+0000","last_change":"2024-04-21T08:36:23.985910+0000","last_active":"2024-04-21T08:38:54.888120+0000","last_peered":"2024-04-21T08:38:54.888120+0000","last_clean":"2024-04-21T08:38:54.888120+0000","last_became_active":"2024-04-21T08:36:23.985605+0000","last_became_peered":"2024-04-21T08:36:23.985605+0000","last_unstale":"2024-04-21T08:38:54.888120+0000","last_undegraded":"2024-04-21T08:38:54.888120+0000","last_fullsized":"2024-04-21T08:38:54.888120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:37:09.367005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077277+0000","last_change":"2024-04-21T08:36:28.015256+0000","last_active":"2024-04-21T08:38:55.077277+0000","last_peered":"2024-04-21T08:38:55.077277+0000","last_clean":"2024-04-21T08:38:55.077277+0000","last_became_active":"2024-04-21T08:36:28.014849+0000","last_became_peered":"2024-04-21T08:36:28.014849+0000","last_unstale":"2024-04-21T08:38:55.077277+0000","last_undegraded":"2024-04-21T08:38:55.077277+0000","last_fullsized":"2024-04-21T08:38:55.077277+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:22:05.466299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"95'6","reported_seq":171,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:59.080426+0000","last_change":"2024-04-21T08:36:23.474966+0000","last_active":"2024-04-21T08:38:59.080426+0000","last_peered":"2024-04-21T08:38:59.080426+0000","last_clean":"2024-04-21T08:38:59.080426+0000","last_became_active":"2024-04-21T08:36:23.474374+0000","last_became_peered":"2024-04-21T08:36:23.474374+0000","last_unstale":"2024-04-21T08:38:59.080426+0000","last_undegraded":"2024-04-21T08:38:59.080426+0000","last_fullsized":"2024-04-21T08:38:59.080426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:57:20.237966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888138+0000","last_change":"2024-04-21T08:37:15.118887+0000","last_active":"2024-04-21T08:38:54.888138+0000","last_peered":"2024-04-21T08:38:54.888138+0000","last_clean":"2024-04-21T08:38:54.888138+0000","last_became_active":"2024-04-21T08:37:15.118510+0000","last_became_peered":"2024-04-21T08:37:15.118510+0000","last_unstale":"2024-04-21T08:38:54.888138+0000","last_undegraded":"2024-04-21T08:38:54.888138+0000","last_fullsized":"2024-04-21T08:38:54.888138+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:59:27.209083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077841+0000","last_change":"2024-04-21T08:36:28.013169+0000","last_active":"2024-04-21T08:38:55.077841+0000","last_peered":"2024-04-21T08:38:55.077841+0000","last_clean":"2024-04-21T08:38:55.077841+0000","last_became_active":"2024-04-21T08:36:28.012791+0000","last_became_peered":"2024-04-21T08:36:28.012791+0000","last_unstale":"2024-04-21T08:38:55.077841+0000","last_undegraded":"2024-04-21T08:38:55.077841+0000","last_fullsized":"2024-04-21T08:38:55.077841+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:42:56.063159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007983+0000","last_change":"2024-04-21T08:36:23.995209+0000","last_active":"2024-04-21T08:38:55.007983+0000","last_peered":"2024-04-21T08:38:55.007983+0000","last_clean":"2024-04-21T08:38:55.007983+0000","last_became_active":"2024-04-21T08:36:23.994866+0000","last_became_peered":"2024-04-21T08:36:23.994866+0000","last_unstale":"2024-04-21T08:38:55.007983+0000","last_undegraded":"2024-04-21T08:38:55.007983+0000","last_fullsized":"2024-04-21T08:38:55.007983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:33:45.582150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'7","reported_seq":173,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:59.080838+0000","last_change":"2024-04-21T08:36:23.471474+0000","last_active":"2024-04-21T08:38:59.080838+0000","last_peered":"2024-04-21T08:38:59.080838+0000","last_clean":"2024-04-21T08:38:59.080838+0000","last_became_active":"2024-04-21T08:36:23.471179+0000","last_became_peered":"2024-04-21T08:36:23.471179+0000","last_unstale":"2024-04-21T08:38:59.080838+0000","last_undegraded":"2024-04-21T08:38:59.080838+0000","last_fullsized":"2024-04-21T08:38:59.080838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:17.290051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":132,"num_read_kb":132,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006553+0000","last_change":"2024-04-21T08:38:15.344332+0000","last_active":"2024-04-21T08:38:55.006553+0000","last_peered":"2024-04-21T08:38:55.006553+0000","last_clean":"2024-04-21T08:38:55.006553+0000","last_became_active":"2024-04-21T08:38:15.339914+0000","last_became_peered":"2024-04-21T08:38:15.339914+0000","last_unstale":"2024-04-21T08:38:55.006553+0000","last_undegraded":"2024-04-21T08:38:55.006553+0000","last_fullsized":"2024-04-21T08:38:55.006553+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:39:24.049323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886832+0000","last_change":"2024-04-21T08:36:23.990272+0000","last_active":"2024-04-21T08:38:54.886832+0000","last_peered":"2024-04-21T08:38:54.886832+0000","last_clean":"2024-04-21T08:38:54.886832+0000","last_became_active":"2024-04-21T08:36:23.989940+0000","last_became_peered":"2024-04-21T08:36:23.989940+0000","last_unstale":"2024-04-21T08:38:54.886832+0000","last_undegraded":"2024-04-21T08:38:54.886832+0000","last_fullsized":"2024-04-21T08:38:54.886832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:27:50.705167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006322+0000","last_change":"2024-04-21T08:36:23.474162+0000","last_active":"2024-04-21T08:38:55.006322+0000","last_peered":"2024-04-21T08:38:55.006322+0000","last_clean":"2024-04-21T08:38:55.006322+0000","last_became_active":"2024-04-21T08:36:23.473802+0000","last_became_peered":"2024-04-21T08:36:23.473802+0000","last_unstale":"2024-04-21T08:38:55.006322+0000","last_undegraded":"2024-04-21T08:38:55.006322+0000","last_fullsized":"2024-04-21T08:38:55.006322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:32:41.918431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"95'148","reported_seq":410,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:57.010247+0000","last_change":"2024-04-21T08:35:10.062130+0000","last_active":"2024-04-21T08:38:57.010247+0000","last_peered":"2024-04-21T08:38:57.010247+0000","last_clean":"2024-04-21T08:38:57.010247+0000","last_became_active":"2024-04-21T08:35:10.061727+0000","last_became_peered":"2024-04-21T08:35:10.061727+0000","last_unstale":"2024-04-21T08:38:57.010247+0000","last_undegraded":"2024-04-21T08:38:57.010247+0000","last_fullsized":"2024-04-21T08:38:57.010247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:31:40.950939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:31:40.950939+0000","last_clean_scrub_stamp":"2024-04-21T08:31:40.950939+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:35:52.360167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008148+0000","last_change":"2024-04-21T08:38:15.415007+0000","last_active":"2024-04-21T08:38:55.008148+0000","last_peered":"2024-04-21T08:38:55.008148+0000","last_clean":"2024-04-21T08:38:55.008148+0000","last_became_active":"2024-04-21T08:38:15.414478+0000","last_became_peered":"2024-04-21T08:38:15.414478+0000","last_unstale":"2024-04-21T08:38:55.008148+0000","last_undegraded":"2024-04-21T08:38:55.008148+0000","last_fullsized":"2024-04-21T08:38:55.008148+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:27.703964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.075651+0000","last_change":"2024-04-21T08:36:23.984332+0000","last_active":"2024-04-21T08:38:55.075651+0000","last_peered":"2024-04-21T08:38:55.075651+0000","last_clean":"2024-04-21T08:38:55.075651+0000","last_became_active":"2024-04-21T08:36:23.984021+0000","last_became_peered":"2024-04-21T08:36:23.984021+0000","last_unstale":"2024-04-21T08:38:55.075651+0000","last_undegraded":"2024-04-21T08:38:55.075651+0000","last_fullsized":"2024-04-21T08:38:55.075651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:05:21.090633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005972+0000","last_change":"2024-04-21T08:36:23.467522+0000","last_active":"2024-04-21T08:38:55.005972+0000","last_peered":"2024-04-21T08:38:55.005972+0000","last_clean":"2024-04-21T08:38:55.005972+0000","last_became_active":"2024-04-21T08:36:23.466379+0000","last_became_peered":"2024-04-21T08:36:23.466379+0000","last_unstale":"2024-04-21T08:38:55.005972+0000","last_undegraded":"2024-04-21T08:38:55.005972+0000","last_fullsized":"2024-04-21T08:38:55.005972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:04.185314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005193+0000","last_change":"2024-04-21T08:36:28.015120+0000","last_active":"2024-04-21T08:38:55.005193+0000","last_peered":"2024-04-21T08:38:55.005193+0000","last_clean":"2024-04-21T08:38:55.005193+0000","last_became_active":"2024-04-21T08:36:28.014701+0000","last_became_peered":"2024-04-21T08:36:28.014701+0000","last_unstale":"2024-04-21T08:38:55.005193+0000","last_undegraded":"2024-04-21T08:38:55.005193+0000","last_fullsized":"2024-04-21T08:38:55.005193+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:23.127622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890642+0000","last_change":"2024-04-21T08:36:23.475335+0000","last_active":"2024-04-21T08:38:54.890642+0000","last_peered":"2024-04-21T08:38:54.890642+0000","last_clean":"2024-04-21T08:38:54.890642+0000","last_became_active":"2024-04-21T08:36:23.474739+0000","last_became_peered":"2024-04-21T08:36:23.474739+0000","last_unstale":"2024-04-21T08:38:54.890642+0000","last_undegraded":"2024-04-21T08:38:54.890642+0000","last_fullsized":"2024-04-21T08:38:54.890642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:36.119649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077395+0000","last_change":"2024-04-21T08:36:23.977229+0000","last_active":"2024-04-21T08:38:55.077395+0000","last_peered":"2024-04-21T08:38:55.077395+0000","last_clean":"2024-04-21T08:38:55.077395+0000","last_became_active":"2024-04-21T08:36:23.976893+0000","last_became_peered":"2024-04-21T08:36:23.976893+0000","last_unstale":"2024-04-21T08:38:55.077395+0000","last_undegraded":"2024-04-21T08:38:55.077395+0000","last_fullsized":"2024-04-21T08:38:55.077395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:31:54.226274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886835+0000","last_change":"2024-04-21T08:36:28.018581+0000","last_active":"2024-04-21T08:38:54.886835+0000","last_peered":"2024-04-21T08:38:54.886835+0000","last_clean":"2024-04-21T08:38:54.886835+0000","last_became_active":"2024-04-21T08:36:28.018021+0000","last_became_peered":"2024-04-21T08:36:28.018021+0000","last_unstale":"2024-04-21T08:38:54.886835+0000","last_undegraded":"2024-04-21T08:38:54.886835+0000","last_fullsized":"2024-04-21T08:38:54.886835+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:29:00.355532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'73","reported_seq":160,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:39:00.890509+0000","last_change":"2024-04-21T08:36:23.473339+0000","last_active":"2024-04-21T08:39:00.890509+0000","last_peered":"2024-04-21T08:39:00.890509+0000","last_clean":"2024-04-21T08:39:00.890509+0000","last_became_active":"2024-04-21T08:36:23.472994+0000","last_became_peered":"2024-04-21T08:36:23.472994+0000","last_unstale":"2024-04-21T08:39:00.890509+0000","last_undegraded":"2024-04-21T08:39:00.890509+0000","last_fullsized":"2024-04-21T08:39:00.890509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:42.762273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":182,"num_read_kb":160,"num_write":129,"num_write_kb":1156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887378+0000","last_change":"2024-04-21T08:36:23.987354+0000","last_active":"2024-04-21T08:38:54.887378+0000","last_peered":"2024-04-21T08:38:54.887378+0000","last_clean":"2024-04-21T08:38:54.887378+0000","last_became_active":"2024-04-21T08:36:23.987032+0000","last_became_peered":"2024-04-21T08:36:23.987032+0000","last_unstale":"2024-04-21T08:38:54.887378+0000","last_undegraded":"2024-04-21T08:38:54.887378+0000","last_fullsized":"2024-04-21T08:38:54.887378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:01:05.164367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104455+0000","last_change":"2024-04-21T08:36:28.002582+0000","last_active":"2024-04-21T08:38:55.104455+0000","last_peered":"2024-04-21T08:38:55.104455+0000","last_clean":"2024-04-21T08:38:55.104455+0000","last_became_active":"2024-04-21T08:36:28.002133+0000","last_became_peered":"2024-04-21T08:36:28.002133+0000","last_unstale":"2024-04-21T08:38:55.104455+0000","last_undegraded":"2024-04-21T08:38:55.104455+0000","last_fullsized":"2024-04-21T08:38:55.104455+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:30:00.268015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007058+0000","last_change":"2024-04-21T08:36:23.462840+0000","last_active":"2024-04-21T08:38:55.007058+0000","last_peered":"2024-04-21T08:38:55.007058+0000","last_clean":"2024-04-21T08:38:55.007058+0000","last_became_active":"2024-04-21T08:36:23.462391+0000","last_became_peered":"2024-04-21T08:36:23.462391+0000","last_unstale":"2024-04-21T08:38:55.007058+0000","last_undegraded":"2024-04-21T08:38:55.007058+0000","last_fullsized":"2024-04-21T08:38:55.007058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:33.230388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004380+0000","last_change":"2024-04-21T08:36:23.994751+0000","last_active":"2024-04-21T08:38:55.004380+0000","last_peered":"2024-04-21T08:38:55.004380+0000","last_clean":"2024-04-21T08:38:55.004380+0000","last_became_active":"2024-04-21T08:36:23.994384+0000","last_became_peered":"2024-04-21T08:36:23.994384+0000","last_unstale":"2024-04-21T08:38:55.004380+0000","last_undegraded":"2024-04-21T08:38:55.004380+0000","last_fullsized":"2024-04-21T08:38:55.004380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:18.956893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006981+0000","last_change":"2024-04-21T08:36:28.004004+0000","last_active":"2024-04-21T08:38:55.006981+0000","last_peered":"2024-04-21T08:38:55.006981+0000","last_clean":"2024-04-21T08:38:55.006981+0000","last_became_active":"2024-04-21T08:36:28.003547+0000","last_became_peered":"2024-04-21T08:36:28.003547+0000","last_unstale":"2024-04-21T08:38:55.006981+0000","last_undegraded":"2024-04-21T08:38:55.006981+0000","last_fullsized":"2024-04-21T08:38:55.006981+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:21:40.215428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104510+0000","last_change":"2024-04-21T08:37:15.222118+0000","last_active":"2024-04-21T08:38:55.104510+0000","last_peered":"2024-04-21T08:38:55.104510+0000","last_clean":"2024-04-21T08:38:55.104510+0000","last_became_active":"2024-04-21T08:37:15.219173+0000","last_became_peered":"2024-04-21T08:37:15.219173+0000","last_unstale":"2024-04-21T08:38:55.104510+0000","last_undegraded":"2024-04-21T08:38:55.104510+0000","last_fullsized":"2024-04-21T08:38:55.104510+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:12.931076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890299+0000","last_change":"2024-04-21T08:36:23.988020+0000","last_active":"2024-04-21T08:38:54.890299+0000","last_peered":"2024-04-21T08:38:54.890299+0000","last_clean":"2024-04-21T08:38:54.890299+0000","last_became_active":"2024-04-21T08:36:23.987630+0000","last_became_peered":"2024-04-21T08:36:23.987630+0000","last_unstale":"2024-04-21T08:38:54.890299+0000","last_undegraded":"2024-04-21T08:38:54.890299+0000","last_fullsized":"2024-04-21T08:38:54.890299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:06.519001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.997084+0000","last_change":"2024-04-21T08:36:28.006984+0000","last_active":"2024-04-21T08:38:54.997084+0000","last_peered":"2024-04-21T08:38:54.997084+0000","last_clean":"2024-04-21T08:38:54.997084+0000","last_became_active":"2024-04-21T08:36:28.006548+0000","last_became_peered":"2024-04-21T08:36:28.006548+0000","last_unstale":"2024-04-21T08:38:54.997084+0000","last_undegraded":"2024-04-21T08:38:54.997084+0000","last_fullsized":"2024-04-21T08:38:54.997084+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:25:50.870658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886907+0000","last_change":"2024-04-21T08:36:23.476266+0000","last_active":"2024-04-21T08:38:54.886907+0000","last_peered":"2024-04-21T08:38:54.886907+0000","last_clean":"2024-04-21T08:38:54.886907+0000","last_became_active":"2024-04-21T08:36:23.475948+0000","last_became_peered":"2024-04-21T08:36:23.475948+0000","last_unstale":"2024-04-21T08:38:54.886907+0000","last_undegraded":"2024-04-21T08:38:54.886907+0000","last_fullsized":"2024-04-21T08:38:54.886907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:41.529376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005217+0000","last_change":"2024-04-21T08:36:23.981942+0000","last_active":"2024-04-21T08:38:55.005217+0000","last_peered":"2024-04-21T08:38:55.005217+0000","last_clean":"2024-04-21T08:38:55.005217+0000","last_became_active":"2024-04-21T08:36:23.981643+0000","last_became_peered":"2024-04-21T08:36:23.981643+0000","last_unstale":"2024-04-21T08:38:55.005217+0000","last_undegraded":"2024-04-21T08:38:55.005217+0000","last_fullsized":"2024-04-21T08:38:55.005217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:26:24.722695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006307+0000","last_change":"2024-04-21T08:36:28.006980+0000","last_active":"2024-04-21T08:38:55.006307+0000","last_peered":"2024-04-21T08:38:55.006307+0000","last_clean":"2024-04-21T08:38:55.006307+0000","last_became_active":"2024-04-21T08:36:28.006530+0000","last_became_peered":"2024-04-21T08:36:28.006530+0000","last_unstale":"2024-04-21T08:38:55.006307+0000","last_undegraded":"2024-04-21T08:38:55.006307+0000","last_fullsized":"2024-04-21T08:38:55.006307+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:05:57.465612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"95'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104627+0000","last_change":"2024-04-21T08:36:23.473253+0000","last_active":"2024-04-21T08:38:55.104627+0000","last_peered":"2024-04-21T08:38:55.104627+0000","last_clean":"2024-04-21T08:38:55.104627+0000","last_became_active":"2024-04-21T08:36:23.472942+0000","last_became_peered":"2024-04-21T08:36:23.472942+0000","last_unstale":"2024-04-21T08:38:55.104627+0000","last_undegraded":"2024-04-21T08:38:55.104627+0000","last_fullsized":"2024-04-21T08:38:55.104627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:32:04.894388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887248+0000","last_change":"2024-04-21T08:36:23.988355+0000","last_active":"2024-04-21T08:38:54.887248+0000","last_peered":"2024-04-21T08:38:54.887248+0000","last_clean":"2024-04-21T08:38:54.887248+0000","last_became_active":"2024-04-21T08:36:23.988049+0000","last_became_peered":"2024-04-21T08:36:23.988049+0000","last_unstale":"2024-04-21T08:38:54.887248+0000","last_undegraded":"2024-04-21T08:38:54.887248+0000","last_fullsized":"2024-04-21T08:38:54.887248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:57.348698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006797+0000","last_change":"2024-04-21T08:36:28.007343+0000","last_active":"2024-04-21T08:38:55.006797+0000","last_peered":"2024-04-21T08:38:55.006797+0000","last_clean":"2024-04-21T08:38:55.006797+0000","last_became_active":"2024-04-21T08:36:28.006719+0000","last_became_peered":"2024-04-21T08:36:28.006719+0000","last_unstale":"2024-04-21T08:38:55.006797+0000","last_undegraded":"2024-04-21T08:38:55.006797+0000","last_fullsized":"2024-04-21T08:38:55.006797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:02:25.734584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890843+0000","last_change":"2024-04-21T08:36:23.472416+0000","last_active":"2024-04-21T08:38:54.890843+0000","last_peered":"2024-04-21T08:38:54.890843+0000","last_clean":"2024-04-21T08:38:54.890843+0000","last_became_active":"2024-04-21T08:36:23.471500+0000","last_became_peered":"2024-04-21T08:36:23.471500+0000","last_unstale":"2024-04-21T08:38:54.890843+0000","last_undegraded":"2024-04-21T08:38:54.890843+0000","last_fullsized":"2024-04-21T08:38:54.890843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:39.706432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006128+0000","last_change":"2024-04-21T08:36:23.990653+0000","last_active":"2024-04-21T08:38:55.006128+0000","last_peered":"2024-04-21T08:38:55.006128+0000","last_clean":"2024-04-21T08:38:55.006128+0000","last_became_active":"2024-04-21T08:36:23.990324+0000","last_became_peered":"2024-04-21T08:36:23.990324+0000","last_unstale":"2024-04-21T08:38:55.006128+0000","last_undegraded":"2024-04-21T08:38:55.006128+0000","last_fullsized":"2024-04-21T08:38:55.006128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:08:59.417850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104492+0000","last_change":"2024-04-21T08:36:28.007956+0000","last_active":"2024-04-21T08:38:55.104492+0000","last_peered":"2024-04-21T08:38:55.104492+0000","last_clean":"2024-04-21T08:38:55.104492+0000","last_became_active":"2024-04-21T08:36:28.007475+0000","last_became_peered":"2024-04-21T08:36:28.007475+0000","last_unstale":"2024-04-21T08:38:55.104492+0000","last_undegraded":"2024-04-21T08:38:55.104492+0000","last_fullsized":"2024-04-21T08:38:55.104492+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:54.039359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'8","reported_seq":175,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:59.164772+0000","last_change":"2024-04-21T08:36:23.476887+0000","last_active":"2024-04-21T08:38:59.164772+0000","last_peered":"2024-04-21T08:38:59.164772+0000","last_clean":"2024-04-21T08:38:59.164772+0000","last_became_active":"2024-04-21T08:36:23.476574+0000","last_became_peered":"2024-04-21T08:36:23.476574+0000","last_unstale":"2024-04-21T08:38:59.164772+0000","last_undegraded":"2024-04-21T08:38:59.164772+0000","last_fullsized":"2024-04-21T08:38:59.164772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:37:47.692358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2010,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006973+0000","last_change":"2024-04-21T08:36:23.982845+0000","last_active":"2024-04-21T08:38:55.006973+0000","last_peered":"2024-04-21T08:38:55.006973+0000","last_clean":"2024-04-21T08:38:55.006973+0000","last_became_active":"2024-04-21T08:36:23.982491+0000","last_became_peered":"2024-04-21T08:36:23.982491+0000","last_unstale":"2024-04-21T08:38:55.006973+0000","last_undegraded":"2024-04-21T08:38:55.006973+0000","last_fullsized":"2024-04-21T08:38:55.006973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:48:46.413896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.002896+0000","last_change":"2024-04-21T08:36:28.010442+0000","last_active":"2024-04-21T08:38:55.002896+0000","last_peered":"2024-04-21T08:38:55.002896+0000","last_clean":"2024-04-21T08:38:55.002896+0000","last_became_active":"2024-04-21T08:36:28.010046+0000","last_became_peered":"2024-04-21T08:36:28.010046+0000","last_unstale":"2024-04-21T08:38:55.002896+0000","last_undegraded":"2024-04-21T08:38:55.002896+0000","last_fullsized":"2024-04-21T08:38:55.002896+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:18:36.382802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"95'59","reported_seq":111,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:39:00.885741+0000","last_change":"2024-04-21T08:36:23.473545+0000","last_active":"2024-04-21T08:39:00.885741+0000","last_peered":"2024-04-21T08:39:00.885741+0000","last_clean":"2024-04-21T08:39:00.885741+0000","last_became_active":"2024-04-21T08:36:23.473222+0000","last_became_peered":"2024-04-21T08:36:23.473222+0000","last_unstale":"2024-04-21T08:39:00.885741+0000","last_undegraded":"2024-04-21T08:39:00.885741+0000","last_fullsized":"2024-04-21T08:39:00.885741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:47:53.947187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":20,"num_write":68,"num_write_kb":807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006600+0000","last_change":"2024-04-21T08:36:23.990959+0000","last_active":"2024-04-21T08:38:55.006600+0000","last_peered":"2024-04-21T08:38:55.006600+0000","last_clean":"2024-04-21T08:38:55.006600+0000","last_became_active":"2024-04-21T08:36:23.990322+0000","last_became_peered":"2024-04-21T08:36:23.990322+0000","last_unstale":"2024-04-21T08:38:55.006600+0000","last_undegraded":"2024-04-21T08:38:55.006600+0000","last_fullsized":"2024-04-21T08:38:55.006600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:12:20.859224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007717+0000","last_change":"2024-04-21T08:36:28.006682+0000","last_active":"2024-04-21T08:38:55.007717+0000","last_peered":"2024-04-21T08:38:55.007717+0000","last_clean":"2024-04-21T08:38:55.007717+0000","last_became_active":"2024-04-21T08:36:28.006229+0000","last_became_peered":"2024-04-21T08:36:28.006229+0000","last_unstale":"2024-04-21T08:38:55.007717+0000","last_undegraded":"2024-04-21T08:38:55.007717+0000","last_fullsized":"2024-04-21T08:38:55.007717+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:30:12.723848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077921+0000","last_change":"2024-04-21T08:36:23.473592+0000","last_active":"2024-04-21T08:38:55.077921+0000","last_peered":"2024-04-21T08:38:55.077921+0000","last_clean":"2024-04-21T08:38:55.077921+0000","last_became_active":"2024-04-21T08:36:23.473283+0000","last_became_peered":"2024-04-21T08:36:23.473283+0000","last_unstale":"2024-04-21T08:38:55.077921+0000","last_undegraded":"2024-04-21T08:38:55.077921+0000","last_fullsized":"2024-04-21T08:38:55.077921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:10.425249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006895+0000","last_change":"2024-04-21T08:36:23.985052+0000","last_active":"2024-04-21T08:38:55.006895+0000","last_peered":"2024-04-21T08:38:55.006895+0000","last_clean":"2024-04-21T08:38:55.006895+0000","last_became_active":"2024-04-21T08:36:23.984742+0000","last_became_peered":"2024-04-21T08:36:23.984742+0000","last_unstale":"2024-04-21T08:38:55.006895+0000","last_undegraded":"2024-04-21T08:38:55.006895+0000","last_fullsized":"2024-04-21T08:38:55.006895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:08:06.082974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005201+0000","last_change":"2024-04-21T08:36:23.980905+0000","last_active":"2024-04-21T08:38:55.005201+0000","last_peered":"2024-04-21T08:38:55.005201+0000","last_clean":"2024-04-21T08:38:55.005201+0000","last_became_active":"2024-04-21T08:36:23.980590+0000","last_became_peered":"2024-04-21T08:36:23.980590+0000","last_unstale":"2024-04-21T08:38:55.005201+0000","last_undegraded":"2024-04-21T08:38:55.005201+0000","last_fullsized":"2024-04-21T08:38:55.005201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:54:59.886014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104725+0000","last_change":"2024-04-21T08:36:23.980883+0000","last_active":"2024-04-21T08:38:55.104725+0000","last_peered":"2024-04-21T08:38:55.104725+0000","last_clean":"2024-04-21T08:38:55.104725+0000","last_became_active":"2024-04-21T08:36:23.980562+0000","last_became_peered":"2024-04-21T08:36:23.980562+0000","last_unstale":"2024-04-21T08:38:55.104725+0000","last_undegraded":"2024-04-21T08:38:55.104725+0000","last_fullsized":"2024-04-21T08:38:55.104725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:14:51.868438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104692+0000","last_change":"2024-04-21T08:36:23.991571+0000","last_active":"2024-04-21T08:38:55.104692+0000","last_peered":"2024-04-21T08:38:55.104692+0000","last_clean":"2024-04-21T08:38:55.104692+0000","last_became_active":"2024-04-21T08:36:23.991273+0000","last_became_peered":"2024-04-21T08:36:23.991273+0000","last_unstale":"2024-04-21T08:38:55.104692+0000","last_undegraded":"2024-04-21T08:38:55.104692+0000","last_fullsized":"2024-04-21T08:38:55.104692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:08:23.895038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.003835+0000","last_change":"2024-04-21T08:36:23.994729+0000","last_active":"2024-04-21T08:38:55.003835+0000","last_peered":"2024-04-21T08:38:55.003835+0000","last_clean":"2024-04-21T08:38:55.003835+0000","last_became_active":"2024-04-21T08:36:23.994405+0000","last_became_peered":"2024-04-21T08:36:23.994405+0000","last_unstale":"2024-04-21T08:38:55.003835+0000","last_undegraded":"2024-04-21T08:38:55.003835+0000","last_fullsized":"2024-04-21T08:38:55.003835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:42.463225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077385+0000","last_change":"2024-04-21T08:36:23.995421+0000","last_active":"2024-04-21T08:38:55.077385+0000","last_peered":"2024-04-21T08:38:55.077385+0000","last_clean":"2024-04-21T08:38:55.077385+0000","last_became_active":"2024-04-21T08:36:23.993674+0000","last_became_peered":"2024-04-21T08:36:23.993674+0000","last_unstale":"2024-04-21T08:38:55.077385+0000","last_undegraded":"2024-04-21T08:38:55.077385+0000","last_fullsized":"2024-04-21T08:38:55.077385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:31.013874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005504+0000","last_change":"2024-04-21T08:36:23.986418+0000","last_active":"2024-04-21T08:38:55.005504+0000","last_peered":"2024-04-21T08:38:55.005504+0000","last_clean":"2024-04-21T08:38:55.005504+0000","last_became_active":"2024-04-21T08:36:23.986095+0000","last_became_peered":"2024-04-21T08:36:23.986095+0000","last_unstale":"2024-04-21T08:38:55.005504+0000","last_undegraded":"2024-04-21T08:38:55.005504+0000","last_fullsized":"2024-04-21T08:38:55.005504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:31:56.456218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104918+0000","last_change":"2024-04-21T08:36:23.982353+0000","last_active":"2024-04-21T08:38:55.104918+0000","last_peered":"2024-04-21T08:38:55.104918+0000","last_clean":"2024-04-21T08:38:55.104918+0000","last_became_active":"2024-04-21T08:36:23.982035+0000","last_became_peered":"2024-04-21T08:36:23.982035+0000","last_unstale":"2024-04-21T08:38:55.104918+0000","last_undegraded":"2024-04-21T08:38:55.104918+0000","last_fullsized":"2024-04-21T08:38:55.104918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:42:44.598407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887832+0000","last_change":"2024-04-21T08:36:23.996371+0000","last_active":"2024-04-21T08:38:54.887832+0000","last_peered":"2024-04-21T08:38:54.887832+0000","last_clean":"2024-04-21T08:38:54.887832+0000","last_became_active":"2024-04-21T08:36:23.995802+0000","last_became_peered":"2024-04-21T08:36:23.995802+0000","last_unstale":"2024-04-21T08:38:54.887832+0000","last_undegraded":"2024-04-21T08:38:54.887832+0000","last_fullsized":"2024-04-21T08:38:54.887832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:52:08.571567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007826+0000","last_change":"2024-04-21T08:36:23.987275+0000","last_active":"2024-04-21T08:38:55.007826+0000","last_peered":"2024-04-21T08:38:55.007826+0000","last_clean":"2024-04-21T08:38:55.007826+0000","last_became_active":"2024-04-21T08:36:23.986967+0000","last_became_peered":"2024-04-21T08:36:23.986967+0000","last_unstale":"2024-04-21T08:38:55.007826+0000","last_undegraded":"2024-04-21T08:38:55.007826+0000","last_fullsized":"2024-04-21T08:38:55.007826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:50.432502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886991+0000","last_change":"2024-04-21T08:36:23.993046+0000","last_active":"2024-04-21T08:38:54.886991+0000","last_peered":"2024-04-21T08:38:54.886991+0000","last_clean":"2024-04-21T08:38:54.886991+0000","last_became_active":"2024-04-21T08:36:23.992693+0000","last_became_peered":"2024-04-21T08:36:23.992693+0000","last_unstale":"2024-04-21T08:38:54.886991+0000","last_undegraded":"2024-04-21T08:38:54.886991+0000","last_fullsized":"2024-04-21T08:38:54.886991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:12:06.287585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104706+0000","last_change":"2024-04-21T08:36:23.992968+0000","last_active":"2024-04-21T08:38:55.104706+0000","last_peered":"2024-04-21T08:38:55.104706+0000","last_clean":"2024-04-21T08:38:55.104706+0000","last_became_active":"2024-04-21T08:36:23.992614+0000","last_became_peered":"2024-04-21T08:36:23.992614+0000","last_unstale":"2024-04-21T08:38:55.104706+0000","last_undegraded":"2024-04-21T08:38:55.104706+0000","last_fullsized":"2024-04-21T08:38:55.104706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:10.015323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076758+0000","last_change":"2024-04-21T08:36:23.981169+0000","last_active":"2024-04-21T08:38:55.076758+0000","last_peered":"2024-04-21T08:38:55.076758+0000","last_clean":"2024-04-21T08:38:55.076758+0000","last_became_active":"2024-04-21T08:36:23.980849+0000","last_became_peered":"2024-04-21T08:36:23.980849+0000","last_unstale":"2024-04-21T08:38:55.076758+0000","last_undegraded":"2024-04-21T08:38:55.076758+0000","last_fullsized":"2024-04-21T08:38:55.076758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:13:49.907370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005358+0000","last_change":"2024-04-21T08:36:23.993998+0000","last_active":"2024-04-21T08:38:55.005358+0000","last_peered":"2024-04-21T08:38:55.005358+0000","last_clean":"2024-04-21T08:38:55.005358+0000","last_became_active":"2024-04-21T08:36:23.993664+0000","last_became_peered":"2024-04-21T08:36:23.993664+0000","last_unstale":"2024-04-21T08:38:55.005358+0000","last_undegraded":"2024-04-21T08:38:55.005358+0000","last_fullsized":"2024-04-21T08:38:55.005358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:31:49.076820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":801015,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1558,"num_read_kb":947,"num_write":445,"num_write_kb":2130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2403045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":353,"ondisk_log_size":353,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351020,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1093632,"data_stored":535779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612658,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37220,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707932,"statfs":{"total":95995035648,"available":95956922368,"internally_reserved":0,"allocated":1929216,"data_stored":1324346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776183,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36820,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708332,"statfs":{"total":95995035648,"available":95957331968,"internally_reserved":0,"allocated":1515520,"data_stored":932181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037820,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":932559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299456,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1712128,"data_stored":1133590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462981,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1093632,"data_stored":530764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1167360,"data_stored":601862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1118208,"data_stored":532669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1187840,"data_stored":603454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1054897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920848,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":535370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724620,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32984,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712168,"statfs":{"total":95995035648,"available":95961260032,"internally_reserved":0,"allocated":1851392,"data_stored":1255362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":331975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":400959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:39:05.614 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T08:39:05.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.754+0000 7f08fb84a640 1 -- 172.21.15.43:0/732908541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f4102e80 msgr2=0x7f08f40690e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:05.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.754+0000 7f08fb84a640 1 --2- 172.21.15.43:0/732908541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f4102e80 0x7f08f40690e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009900 tx=0x7f08ec02bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:05.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.754+0000 7f08fb84a640 1 -- 172.21.15.43:0/732908541 shutdown_connections 2024-04-21T08:39:05.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.754+0000 7f08fb84a640 1 --2- 172.21.15.43:0/732908541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08f410e810 0x7f08f4110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.754+0000 7f08fb84a640 1 --2- 172.21.15.43:0/732908541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08f4069620 0x7f08f4069a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.754+0000 7f08fb84a640 1 --2- 172.21.15.43:0/732908541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f4102e80 0x7f08f40690e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.754+0000 7f08fb84a640 1 -- 172.21.15.43:0/732908541 >> 172.21.15.43:0/732908541 conn(0x7f08f4077e30 msgr2=0x7f08f4078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:05.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.754+0000 7f08fb84a640 1 -- 172.21.15.43:0/732908541 shutdown_connections 2024-04-21T08:39:05.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.754+0000 7f08fb84a640 1 -- 172.21.15.43:0/732908541 wait complete. 2024-04-21T08:39:05.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.755+0000 7f08fb84a640 1 Processor -- start 2024-04-21T08:39:05.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.755+0000 7f08fb84a640 1 -- start start 2024-04-21T08:39:05.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.756+0000 7f08fb84a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08f4069620 0x7f08f41a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:05.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.756+0000 7f08fb84a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08f4102e80 0x7f08f41a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:05.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.756+0000 7f08fb84a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f410e810 0x7f08f419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:05.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.756+0000 7f08fb84a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f08f41143b0 con 0x7f08f410e810 2024-04-21T08:39:05.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.756+0000 7f08fb84a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08f4114230 con 0x7f08f4102e80 2024-04-21T08:39:05.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.756+0000 7f08fb84a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f08f4114530 con 0x7f08f4069620 2024-04-21T08:39:05.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.756+0000 7f08f9de7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f410e810 0x7f08f419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:05.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.756+0000 7f08f9de7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f410e810 0x7f08f419a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39098/0 (socket says 172.21.15.43:39098) 2024-04-21T08:39:05.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.756+0000 7f08f9de7640 1 -- 172.21.15.43:0/3433437849 learned_addr learned my addr 172.21.15.43:0/3433437849 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:05.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.756+0000 7f08f95e6640 1 --2- 172.21.15.43:0/3433437849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08f4069620 0x7f08f41a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:05.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.756+0000 7f08f8de5640 1 --2- 172.21.15.43:0/3433437849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08f4102e80 0x7f08f41a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:05.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.756+0000 7f08f9de7640 1 -- 172.21.15.43:0/3433437849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08f4069620 msgr2=0x7f08f41a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:05.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.756+0000 7f08f9de7640 1 --2- 172.21.15.43:0/3433437849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08f4069620 0x7f08f41a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.756+0000 7f08f9de7640 1 -- 172.21.15.43:0/3433437849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08f4102e80 msgr2=0x7f08f41a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:05.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.756+0000 7f08f9de7640 1 --2- 172.21.15.43:0/3433437849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08f4102e80 0x7f08f41a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.756+0000 7f08f9de7640 1 -- 172.21.15.43:0/3433437849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f419aff0 con 0x7f08f410e810 2024-04-21T08:39:05.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.756+0000 7f08f9de7640 1 --2- 172.21.15.43:0/3433437849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f410e810 0x7f08f419a730 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009fd0 tx=0x7f08f000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:05.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.756+0000 7f08ea7fc640 1 -- 172.21.15.43:0/3433437849 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f0004d20 con 0x7f08f410e810 2024-04-21T08:39:05.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.757+0000 7f08ea7fc640 1 -- 172.21.15.43:0/3433437849 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08f0005020 con 0x7f08f410e810 2024-04-21T08:39:05.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.757+0000 7f08fb84a640 1 -- 172.21.15.43:0/3433437849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08f419b2e0 con 0x7f08f410e810 2024-04-21T08:39:05.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.757+0000 7f08ea7fc640 1 -- 172.21.15.43:0/3433437849 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f00116b0 con 0x7f08f410e810 2024-04-21T08:39:05.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.757+0000 7f08fb84a640 1 -- 172.21.15.43:0/3433437849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08f41a75f0 con 0x7f08f410e810 2024-04-21T08:39:05.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.758+0000 7f08fb84a640 1 -- 172.21.15.43:0/3433437849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08f4101430 con 0x7f08f410e810 2024-04-21T08:39:05.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.759+0000 7f08ea7fc640 1 -- 172.21.15.43:0/3433437849 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f08f00051d0 con 0x7f08f410e810 2024-04-21T08:39:05.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.760+0000 7f08ea7fc640 1 --2- 172.21.15.43:0/3433437849 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f08e00a3db0 0x7f08e00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:05.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.760+0000 7f08f95e6640 1 --2- 172.21.15.43:0/3433437849 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f08e00a3db0 0x7f08e00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:05.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.760+0000 7f08ea7fc640 1 -- 172.21.15.43:0/3433437849 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f08f00d0f10 con 0x7f08f410e810 2024-04-21T08:39:05.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.760+0000 7f08f95e6640 1 --2- 172.21.15.43:0/3433437849 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f08e00a3db0 0x7f08e00a6270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f08ec006010 tx=0x7f08ec005e40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:05.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.764+0000 7f08ea7fc640 1 -- 172.21.15.43:0/3433437849 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f08f0016030 con 0x7f08f410e810 2024-04-21T08:39:05.793 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:05 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/84915204' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:39:05.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:05 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3944114967' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:39:05.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:05 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/816044815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:39:05.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.953+0000 7f08fb84a640 1 -- 172.21.15.43:0/3433437849 --> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f08f410fe70 con 0x7f08e00a3db0 2024-04-21T08:39:05.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.962+0000 7f08ea7fc640 1 -- 172.21.15.43:0/3433437849 <== mgr.15546 v2:172.21.15.43:6800/1482273906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339443 (secure 0 0 0) 0x7f08f410fe70 con 0x7f08e00a3db0 2024-04-21T08:39:05.965 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:39:05.967 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-21T08:39:05.969 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:05 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/84915204' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:39:05.969 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:05 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3944114967' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:39:05.969 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:05 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/816044815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:39:05.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.966+0000 7f08fb84a640 1 -- 172.21.15.43:0/3433437849 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f08e00a3db0 msgr2=0x7f08e00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:05.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.966+0000 7f08fb84a640 1 --2- 172.21.15.43:0/3433437849 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f08e00a3db0 0x7f08e00a6270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f08ec006010 tx=0x7f08ec005e40 comp rx=0 tx=0).stop 2024-04-21T08:39:05.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.966+0000 7f08fb84a640 1 -- 172.21.15.43:0/3433437849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f410e810 msgr2=0x7f08f419a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:05.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.966+0000 7f08fb84a640 1 --2- 172.21.15.43:0/3433437849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f410e810 0x7f08f419a730 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009fd0 tx=0x7f08f000ee60 comp rx=0 tx=0).stop 2024-04-21T08:39:05.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.966+0000 7f08fb84a640 1 -- 172.21.15.43:0/3433437849 shutdown_connections 2024-04-21T08:39:05.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.966+0000 7f08fb84a640 1 --2- 172.21.15.43:0/3433437849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f410e810 0x7f08f419a730 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.966+0000 7f08fb84a640 1 --2- 172.21.15.43:0/3433437849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08f4102e80 0x7f08f41a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.966+0000 7f08fb84a640 1 --2- 172.21.15.43:0/3433437849 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f08e00a3db0 0x7f08e00a6270 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.966+0000 7f08fb84a640 1 --2- 172.21.15.43:0/3433437849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08f4069620 0x7f08f41a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:05.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.966+0000 7f08fb84a640 1 -- 172.21.15.43:0/3433437849 >> 172.21.15.43:0/3433437849 conn(0x7f08f4077e30 msgr2=0x7f08f410f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:05.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.966+0000 7f08fb84a640 1 -- 172.21.15.43:0/3433437849 shutdown_connections 2024-04-21T08:39:05.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:05.966+0000 7f08fb84a640 1 -- 172.21.15.43:0/3433437849 wait complete. 2024-04-21T08:39:05.976 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-21T08:39:04.903196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1391617,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1795,"num_read_kb":1277,"num_write":648,"num_write_kb":5070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1003,"ondisk_log_size":1003,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386504,"kb_used_data":16520,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124555320,"statfs":{"total":1151940427776,"available":1151544647680,"internally_reserved":0,"allocated":16916480,"data_stored":9972833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":378713404},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":267,"num_read_kb":342,"num_write":88,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.991351"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005622+0000","last_change":"2024-04-21T08:36:23.983921+0000","last_active":"2024-04-21T08:38:55.005622+0000","last_peered":"2024-04-21T08:38:55.005622+0000","last_clean":"2024-04-21T08:38:55.005622+0000","last_became_active":"2024-04-21T08:36:23.983638+0000","last_became_peered":"2024-04-21T08:36:23.983638+0000","last_unstale":"2024-04-21T08:38:55.005622+0000","last_undegraded":"2024-04-21T08:38:55.005622+0000","last_fullsized":"2024-04-21T08:38:55.005622+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:29:44.646308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004430+0000","last_change":"2024-04-21T08:36:23.990029+0000","last_active":"2024-04-21T08:38:55.004430+0000","last_peered":"2024-04-21T08:38:55.004430+0000","last_clean":"2024-04-21T08:38:55.004430+0000","last_became_active":"2024-04-21T08:36:23.989141+0000","last_became_peered":"2024-04-21T08:36:23.989141+0000","last_unstale":"2024-04-21T08:38:55.004430+0000","last_undegraded":"2024-04-21T08:38:55.004430+0000","last_fullsized":"2024-04-21T08:38:55.004430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:23:58.692342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004234+0000","last_change":"2024-04-21T08:36:23.982205+0000","last_active":"2024-04-21T08:38:55.004234+0000","last_peered":"2024-04-21T08:38:55.004234+0000","last_clean":"2024-04-21T08:38:55.004234+0000","last_became_active":"2024-04-21T08:36:23.981876+0000","last_became_peered":"2024-04-21T08:36:23.981876+0000","last_unstale":"2024-04-21T08:38:55.004234+0000","last_undegraded":"2024-04-21T08:38:55.004234+0000","last_fullsized":"2024-04-21T08:38:55.004234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:12:27.121106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006754+0000","last_change":"2024-04-21T08:36:23.987425+0000","last_active":"2024-04-21T08:38:55.006754+0000","last_peered":"2024-04-21T08:38:55.006754+0000","last_clean":"2024-04-21T08:38:55.006754+0000","last_became_active":"2024-04-21T08:36:23.987109+0000","last_became_peered":"2024-04-21T08:36:23.987109+0000","last_unstale":"2024-04-21T08:38:55.006754+0000","last_undegraded":"2024-04-21T08:38:55.006754+0000","last_fullsized":"2024-04-21T08:38:55.006754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:44.176324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890536+0000","last_change":"2024-04-21T08:36:23.990375+0000","last_active":"2024-04-21T08:38:54.890536+0000","last_peered":"2024-04-21T08:38:54.890536+0000","last_clean":"2024-04-21T08:38:54.890536+0000","last_became_active":"2024-04-21T08:36:23.990058+0000","last_became_peered":"2024-04-21T08:36:23.990058+0000","last_unstale":"2024-04-21T08:38:54.890536+0000","last_undegraded":"2024-04-21T08:38:54.890536+0000","last_fullsized":"2024-04-21T08:38:54.890536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:24:09.647528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888221+0000","last_change":"2024-04-21T08:36:23.979848+0000","last_active":"2024-04-21T08:38:54.888221+0000","last_peered":"2024-04-21T08:38:54.888221+0000","last_clean":"2024-04-21T08:38:54.888221+0000","last_became_active":"2024-04-21T08:36:23.979511+0000","last_became_peered":"2024-04-21T08:36:23.979511+0000","last_unstale":"2024-04-21T08:38:54.888221+0000","last_undegraded":"2024-04-21T08:38:54.888221+0000","last_fullsized":"2024-04-21T08:38:54.888221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:07:46.551369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006232+0000","last_change":"2024-04-21T08:37:15.118755+0000","last_active":"2024-04-21T08:38:55.006232+0000","last_peered":"2024-04-21T08:38:55.006232+0000","last_clean":"2024-04-21T08:38:55.006232+0000","last_became_active":"2024-04-21T08:37:15.118347+0000","last_became_peered":"2024-04-21T08:37:15.118347+0000","last_unstale":"2024-04-21T08:38:55.006232+0000","last_undegraded":"2024-04-21T08:38:55.006232+0000","last_fullsized":"2024-04-21T08:38:55.006232+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:36:43.398325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886870+0000","last_change":"2024-04-21T08:36:23.987517+0000","last_active":"2024-04-21T08:38:54.886870+0000","last_peered":"2024-04-21T08:38:54.886870+0000","last_clean":"2024-04-21T08:38:54.886870+0000","last_became_active":"2024-04-21T08:36:23.987201+0000","last_became_peered":"2024-04-21T08:36:23.987201+0000","last_unstale":"2024-04-21T08:38:54.886870+0000","last_undegraded":"2024-04-21T08:38:54.886870+0000","last_fullsized":"2024-04-21T08:38:54.886870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:42:53.080863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104900+0000","last_change":"2024-04-21T08:36:23.989434+0000","last_active":"2024-04-21T08:38:55.104900+0000","last_peered":"2024-04-21T08:38:55.104900+0000","last_clean":"2024-04-21T08:38:55.104900+0000","last_became_active":"2024-04-21T08:36:23.989125+0000","last_became_peered":"2024-04-21T08:36:23.989125+0000","last_unstale":"2024-04-21T08:38:55.104900+0000","last_undegraded":"2024-04-21T08:38:55.104900+0000","last_fullsized":"2024-04-21T08:38:55.104900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:30:07.219056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888449+0000","last_change":"2024-04-21T08:36:23.980725+0000","last_active":"2024-04-21T08:38:54.888449+0000","last_peered":"2024-04-21T08:38:54.888449+0000","last_clean":"2024-04-21T08:38:54.888449+0000","last_became_active":"2024-04-21T08:36:23.980363+0000","last_became_peered":"2024-04-21T08:36:23.980363+0000","last_unstale":"2024-04-21T08:38:54.888449+0000","last_undegraded":"2024-04-21T08:38:54.888449+0000","last_fullsized":"2024-04-21T08:38:54.888449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:26:46.202030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006978+0000","last_change":"2024-04-21T08:36:23.989369+0000","last_active":"2024-04-21T08:38:55.006978+0000","last_peered":"2024-04-21T08:38:55.006978+0000","last_clean":"2024-04-21T08:38:55.006978+0000","last_became_active":"2024-04-21T08:36:23.989030+0000","last_became_peered":"2024-04-21T08:36:23.989030+0000","last_unstale":"2024-04-21T08:38:55.006978+0000","last_undegraded":"2024-04-21T08:38:55.006978+0000","last_fullsized":"2024-04-21T08:38:55.006978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:23:13.725703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006691+0000","last_change":"2024-04-21T08:36:23.993485+0000","last_active":"2024-04-21T08:38:55.006691+0000","last_peered":"2024-04-21T08:38:55.006691+0000","last_clean":"2024-04-21T08:38:55.006691+0000","last_became_active":"2024-04-21T08:36:23.993124+0000","last_became_peered":"2024-04-21T08:36:23.993124+0000","last_unstale":"2024-04-21T08:38:55.006691+0000","last_undegraded":"2024-04-21T08:38:55.006691+0000","last_fullsized":"2024-04-21T08:38:55.006691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:53:16.249838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076569+0000","last_change":"2024-04-21T08:36:23.983146+0000","last_active":"2024-04-21T08:38:55.076569+0000","last_peered":"2024-04-21T08:38:55.076569+0000","last_clean":"2024-04-21T08:38:55.076569+0000","last_became_active":"2024-04-21T08:36:23.982766+0000","last_became_peered":"2024-04-21T08:36:23.982766+0000","last_unstale":"2024-04-21T08:38:55.076569+0000","last_undegraded":"2024-04-21T08:38:55.076569+0000","last_fullsized":"2024-04-21T08:38:55.076569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:30:44.090929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.003129+0000","last_change":"2024-04-21T08:36:23.988906+0000","last_active":"2024-04-21T08:38:55.003129+0000","last_peered":"2024-04-21T08:38:55.003129+0000","last_clean":"2024-04-21T08:38:55.003129+0000","last_became_active":"2024-04-21T08:36:23.988489+0000","last_became_peered":"2024-04-21T08:36:23.988489+0000","last_unstale":"2024-04-21T08:38:55.003129+0000","last_undegraded":"2024-04-21T08:38:55.003129+0000","last_fullsized":"2024-04-21T08:38:55.003129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:06:43.271650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.003060+0000","last_change":"2024-04-21T08:37:15.123343+0000","last_active":"2024-04-21T08:38:55.003060+0000","last_peered":"2024-04-21T08:38:55.003060+0000","last_clean":"2024-04-21T08:38:55.003060+0000","last_became_active":"2024-04-21T08:37:15.122959+0000","last_became_peered":"2024-04-21T08:37:15.122959+0000","last_unstale":"2024-04-21T08:38:55.003060+0000","last_undegraded":"2024-04-21T08:38:55.003060+0000","last_fullsized":"2024-04-21T08:38:55.003060+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:56:19.179879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104471+0000","last_change":"2024-04-21T08:36:23.983136+0000","last_active":"2024-04-21T08:38:55.104471+0000","last_peered":"2024-04-21T08:38:55.104471+0000","last_clean":"2024-04-21T08:38:55.104471+0000","last_became_active":"2024-04-21T08:36:23.982698+0000","last_became_peered":"2024-04-21T08:36:23.982698+0000","last_unstale":"2024-04-21T08:38:55.104471+0000","last_undegraded":"2024-04-21T08:38:55.104471+0000","last_fullsized":"2024-04-21T08:38:55.104471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:34.750941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005563+0000","last_change":"2024-04-21T08:36:23.984701+0000","last_active":"2024-04-21T08:38:55.005563+0000","last_peered":"2024-04-21T08:38:55.005563+0000","last_clean":"2024-04-21T08:38:55.005563+0000","last_became_active":"2024-04-21T08:36:23.984432+0000","last_became_peered":"2024-04-21T08:36:23.984432+0000","last_unstale":"2024-04-21T08:38:55.005563+0000","last_undegraded":"2024-04-21T08:38:55.005563+0000","last_fullsized":"2024-04-21T08:38:55.005563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:10:58.944380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.996823+0000","last_change":"2024-04-21T08:36:23.982781+0000","last_active":"2024-04-21T08:38:54.996823+0000","last_peered":"2024-04-21T08:38:54.996823+0000","last_clean":"2024-04-21T08:38:54.996823+0000","last_became_active":"2024-04-21T08:36:23.982459+0000","last_became_peered":"2024-04-21T08:36:23.982459+0000","last_unstale":"2024-04-21T08:38:54.996823+0000","last_undegraded":"2024-04-21T08:38:54.996823+0000","last_fullsized":"2024-04-21T08:38:54.996823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:44.657504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.105000+0000","last_change":"2024-04-21T08:36:23.992002+0000","last_active":"2024-04-21T08:38:55.105000+0000","last_peered":"2024-04-21T08:38:55.105000+0000","last_clean":"2024-04-21T08:38:55.105000+0000","last_became_active":"2024-04-21T08:36:23.991053+0000","last_became_peered":"2024-04-21T08:36:23.991053+0000","last_unstale":"2024-04-21T08:38:55.105000+0000","last_undegraded":"2024-04-21T08:38:55.105000+0000","last_fullsized":"2024-04-21T08:38:55.105000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:56:37.145949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104888+0000","last_change":"2024-04-21T08:36:28.013195+0000","last_active":"2024-04-21T08:38:55.104888+0000","last_peered":"2024-04-21T08:38:55.104888+0000","last_clean":"2024-04-21T08:38:55.104888+0000","last_became_active":"2024-04-21T08:36:28.012656+0000","last_became_peered":"2024-04-21T08:36:28.012656+0000","last_unstale":"2024-04-21T08:38:55.104888+0000","last_undegraded":"2024-04-21T08:38:55.104888+0000","last_fullsized":"2024-04-21T08:38:55.104888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:12:03.133583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"95'14","reported_seq":179,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:39:00.080989+0000","last_change":"2024-04-21T08:36:23.468957+0000","last_active":"2024-04-21T08:39:00.080989+0000","last_peered":"2024-04-21T08:39:00.080989+0000","last_clean":"2024-04-21T08:39:00.080989+0000","last_became_active":"2024-04-21T08:36:23.468601+0000","last_became_peered":"2024-04-21T08:36:23.468601+0000","last_unstale":"2024-04-21T08:39:00.080989+0000","last_undegraded":"2024-04-21T08:39:00.080989+0000","last_fullsized":"2024-04-21T08:39:00.080989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:13:11.977780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":131,"num_read_kb":131,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.996013+0000","last_change":"2024-04-21T08:37:15.123476+0000","last_active":"2024-04-21T08:38:54.996013+0000","last_peered":"2024-04-21T08:38:54.996013+0000","last_clean":"2024-04-21T08:38:54.996013+0000","last_became_active":"2024-04-21T08:37:15.123066+0000","last_became_peered":"2024-04-21T08:37:15.123066+0000","last_unstale":"2024-04-21T08:38:54.996013+0000","last_undegraded":"2024-04-21T08:38:54.996013+0000","last_fullsized":"2024-04-21T08:38:54.996013+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:15.786178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005900+0000","last_change":"2024-04-21T08:36:28.014161+0000","last_active":"2024-04-21T08:38:55.005900+0000","last_peered":"2024-04-21T08:38:55.005900+0000","last_clean":"2024-04-21T08:38:55.005900+0000","last_became_active":"2024-04-21T08:36:28.013859+0000","last_became_peered":"2024-04-21T08:36:28.013859+0000","last_unstale":"2024-04-21T08:38:55.005900+0000","last_undegraded":"2024-04-21T08:38:55.005900+0000","last_fullsized":"2024-04-21T08:38:55.005900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:08.315334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887665+0000","last_change":"2024-04-21T08:36:23.475612+0000","last_active":"2024-04-21T08:38:54.887665+0000","last_peered":"2024-04-21T08:38:54.887665+0000","last_clean":"2024-04-21T08:38:54.887665+0000","last_became_active":"2024-04-21T08:36:23.475301+0000","last_became_peered":"2024-04-21T08:36:23.475301+0000","last_unstale":"2024-04-21T08:38:54.887665+0000","last_undegraded":"2024-04-21T08:38:54.887665+0000","last_fullsized":"2024-04-21T08:38:54.887665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:56.801630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004414+0000","last_change":"2024-04-21T08:36:23.984689+0000","last_active":"2024-04-21T08:38:55.004414+0000","last_peered":"2024-04-21T08:38:55.004414+0000","last_clean":"2024-04-21T08:38:55.004414+0000","last_became_active":"2024-04-21T08:36:23.984358+0000","last_became_peered":"2024-04-21T08:36:23.984358+0000","last_unstale":"2024-04-21T08:38:55.004414+0000","last_undegraded":"2024-04-21T08:38:55.004414+0000","last_fullsized":"2024-04-21T08:38:55.004414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:49:50.187466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006257+0000","last_change":"2024-04-21T08:36:28.007951+0000","last_active":"2024-04-21T08:38:55.006257+0000","last_peered":"2024-04-21T08:38:55.006257+0000","last_clean":"2024-04-21T08:38:55.006257+0000","last_became_active":"2024-04-21T08:36:28.007553+0000","last_became_peered":"2024-04-21T08:36:28.007553+0000","last_unstale":"2024-04-21T08:38:55.006257+0000","last_undegraded":"2024-04-21T08:38:55.006257+0000","last_fullsized":"2024-04-21T08:38:55.006257+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:10:54.019605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890572+0000","last_change":"2024-04-21T08:36:23.473371+0000","last_active":"2024-04-21T08:38:54.890572+0000","last_peered":"2024-04-21T08:38:54.890572+0000","last_clean":"2024-04-21T08:38:54.890572+0000","last_became_active":"2024-04-21T08:36:23.472472+0000","last_became_peered":"2024-04-21T08:36:23.472472+0000","last_unstale":"2024-04-21T08:38:54.890572+0000","last_undegraded":"2024-04-21T08:38:54.890572+0000","last_fullsized":"2024-04-21T08:38:54.890572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:28:49.622873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104901+0000","last_change":"2024-04-21T08:36:23.987916+0000","last_active":"2024-04-21T08:38:55.104901+0000","last_peered":"2024-04-21T08:38:55.104901+0000","last_clean":"2024-04-21T08:38:55.104901+0000","last_became_active":"2024-04-21T08:36:23.986207+0000","last_became_peered":"2024-04-21T08:36:23.986207+0000","last_unstale":"2024-04-21T08:38:55.104901+0000","last_undegraded":"2024-04-21T08:38:55.104901+0000","last_fullsized":"2024-04-21T08:38:55.104901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:49:57.261078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006246+0000","last_change":"2024-04-21T08:36:28.006494+0000","last_active":"2024-04-21T08:38:55.006246+0000","last_peered":"2024-04-21T08:38:55.006246+0000","last_clean":"2024-04-21T08:38:55.006246+0000","last_became_active":"2024-04-21T08:36:28.006069+0000","last_became_peered":"2024-04-21T08:36:28.006069+0000","last_unstale":"2024-04-21T08:38:55.006246+0000","last_undegraded":"2024-04-21T08:38:55.006246+0000","last_fullsized":"2024-04-21T08:38:55.006246+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:04.608295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890975+0000","last_change":"2024-04-21T08:36:23.468923+0000","last_active":"2024-04-21T08:38:54.890975+0000","last_peered":"2024-04-21T08:38:54.890975+0000","last_clean":"2024-04-21T08:38:54.890975+0000","last_became_active":"2024-04-21T08:36:23.468509+0000","last_became_peered":"2024-04-21T08:36:23.468509+0000","last_unstale":"2024-04-21T08:38:54.890975+0000","last_undegraded":"2024-04-21T08:38:54.890975+0000","last_fullsized":"2024-04-21T08:38:54.890975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:44.452748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008259+0000","last_change":"2024-04-21T08:36:23.994423+0000","last_active":"2024-04-21T08:38:55.008259+0000","last_peered":"2024-04-21T08:38:55.008259+0000","last_clean":"2024-04-21T08:38:55.008259+0000","last_became_active":"2024-04-21T08:36:23.993827+0000","last_became_peered":"2024-04-21T08:36:23.993827+0000","last_unstale":"2024-04-21T08:38:55.008259+0000","last_undegraded":"2024-04-21T08:38:55.008259+0000","last_fullsized":"2024-04-21T08:38:55.008259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:49:25.550301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104894+0000","last_change":"2024-04-21T08:36:28.004037+0000","last_active":"2024-04-21T08:38:55.104894+0000","last_peered":"2024-04-21T08:38:55.104894+0000","last_clean":"2024-04-21T08:38:55.104894+0000","last_became_active":"2024-04-21T08:36:28.003580+0000","last_became_peered":"2024-04-21T08:36:28.003580+0000","last_unstale":"2024-04-21T08:38:55.104894+0000","last_undegraded":"2024-04-21T08:38:55.104894+0000","last_fullsized":"2024-04-21T08:38:55.104894+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:04:51.380244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'8","reported_seq":178,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:59.079811+0000","last_change":"2024-04-21T08:36:23.468430+0000","last_active":"2024-04-21T08:38:59.079811+0000","last_peered":"2024-04-21T08:38:59.079811+0000","last_clean":"2024-04-21T08:38:59.079811+0000","last_became_active":"2024-04-21T08:36:23.468089+0000","last_became_peered":"2024-04-21T08:36:23.468089+0000","last_unstale":"2024-04-21T08:38:59.079811+0000","last_undegraded":"2024-04-21T08:38:59.079811+0000","last_fullsized":"2024-04-21T08:38:59.079811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:12:54.827034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890458+0000","last_change":"2024-04-21T08:37:15.117922+0000","last_active":"2024-04-21T08:38:54.890458+0000","last_peered":"2024-04-21T08:38:54.890458+0000","last_clean":"2024-04-21T08:38:54.890458+0000","last_became_active":"2024-04-21T08:37:15.117404+0000","last_became_peered":"2024-04-21T08:37:15.117404+0000","last_unstale":"2024-04-21T08:38:54.890458+0000","last_undegraded":"2024-04-21T08:38:54.890458+0000","last_fullsized":"2024-04-21T08:38:54.890458+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:39.682224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104742+0000","last_change":"2024-04-21T08:36:28.004638+0000","last_active":"2024-04-21T08:38:55.104742+0000","last_peered":"2024-04-21T08:38:55.104742+0000","last_clean":"2024-04-21T08:38:55.104742+0000","last_became_active":"2024-04-21T08:36:28.002499+0000","last_became_peered":"2024-04-21T08:36:28.002499+0000","last_unstale":"2024-04-21T08:38:55.104742+0000","last_undegraded":"2024-04-21T08:38:55.104742+0000","last_fullsized":"2024-04-21T08:38:55.104742+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:45:46.934783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":55,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888089+0000","last_change":"2024-04-21T08:37:15.126569+0000","last_active":"2024-04-21T08:38:54.888089+0000","last_peered":"2024-04-21T08:38:54.888089+0000","last_clean":"2024-04-21T08:38:54.888089+0000","last_became_active":"2024-04-21T08:37:15.121284+0000","last_became_peered":"2024-04-21T08:37:15.121284+0000","last_unstale":"2024-04-21T08:38:54.888089+0000","last_undegraded":"2024-04-21T08:38:54.888089+0000","last_fullsized":"2024-04-21T08:38:54.888089+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:02:26.153324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008374+0000","last_change":"2024-04-21T08:37:15.557192+0000","last_active":"2024-04-21T08:38:55.008374+0000","last_peered":"2024-04-21T08:38:55.008374+0000","last_clean":"2024-04-21T08:38:55.008374+0000","last_became_active":"2024-04-21T08:37:15.556653+0000","last_became_peered":"2024-04-21T08:37:15.556653+0000","last_unstale":"2024-04-21T08:38:55.008374+0000","last_undegraded":"2024-04-21T08:38:55.008374+0000","last_fullsized":"2024-04-21T08:38:55.008374+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:42.722460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005527+0000","last_change":"2024-04-21T08:36:28.017068+0000","last_active":"2024-04-21T08:38:55.005527+0000","last_peered":"2024-04-21T08:38:55.005527+0000","last_clean":"2024-04-21T08:38:55.005527+0000","last_became_active":"2024-04-21T08:36:28.016770+0000","last_became_peered":"2024-04-21T08:36:28.016770+0000","last_unstale":"2024-04-21T08:38:55.005527+0000","last_undegraded":"2024-04-21T08:38:55.005527+0000","last_fullsized":"2024-04-21T08:38:55.005527+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:07:14.596090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076311+0000","last_change":"2024-04-21T08:36:23.470430+0000","last_active":"2024-04-21T08:38:55.076311+0000","last_peered":"2024-04-21T08:38:55.076311+0000","last_clean":"2024-04-21T08:38:55.076311+0000","last_became_active":"2024-04-21T08:36:23.470109+0000","last_became_peered":"2024-04-21T08:36:23.470109+0000","last_unstale":"2024-04-21T08:38:55.076311+0000","last_undegraded":"2024-04-21T08:38:55.076311+0000","last_fullsized":"2024-04-21T08:38:55.076311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:18.176313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887026+0000","last_change":"2024-04-21T08:36:23.986668+0000","last_active":"2024-04-21T08:38:54.887026+0000","last_peered":"2024-04-21T08:38:54.887026+0000","last_clean":"2024-04-21T08:38:54.887026+0000","last_became_active":"2024-04-21T08:36:23.986344+0000","last_became_peered":"2024-04-21T08:36:23.986344+0000","last_unstale":"2024-04-21T08:38:54.887026+0000","last_undegraded":"2024-04-21T08:38:54.887026+0000","last_fullsized":"2024-04-21T08:38:54.887026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:56:17.744145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004263+0000","last_change":"2024-04-21T08:36:28.009012+0000","last_active":"2024-04-21T08:38:55.004263+0000","last_peered":"2024-04-21T08:38:55.004263+0000","last_clean":"2024-04-21T08:38:55.004263+0000","last_became_active":"2024-04-21T08:36:28.008123+0000","last_became_peered":"2024-04-21T08:36:28.008123+0000","last_unstale":"2024-04-21T08:38:55.004263+0000","last_undegraded":"2024-04-21T08:38:55.004263+0000","last_fullsized":"2024-04-21T08:38:55.004263+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:33:18.881564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007854+0000","last_change":"2024-04-21T08:36:23.467104+0000","last_active":"2024-04-21T08:38:55.007854+0000","last_peered":"2024-04-21T08:38:55.007854+0000","last_clean":"2024-04-21T08:38:55.007854+0000","last_became_active":"2024-04-21T08:36:23.466725+0000","last_became_peered":"2024-04-21T08:36:23.466725+0000","last_unstale":"2024-04-21T08:38:55.007854+0000","last_undegraded":"2024-04-21T08:38:55.007854+0000","last_fullsized":"2024-04-21T08:38:55.007854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:35:45.986448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.995844+0000","last_change":"2024-04-21T08:36:23.987579+0000","last_active":"2024-04-21T08:38:54.995844+0000","last_peered":"2024-04-21T08:38:54.995844+0000","last_clean":"2024-04-21T08:38:54.995844+0000","last_became_active":"2024-04-21T08:36:23.987044+0000","last_became_peered":"2024-04-21T08:36:23.987044+0000","last_unstale":"2024-04-21T08:38:54.995844+0000","last_undegraded":"2024-04-21T08:38:54.995844+0000","last_fullsized":"2024-04-21T08:38:54.995844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:48:27.593090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886895+0000","last_change":"2024-04-21T08:36:28.006417+0000","last_active":"2024-04-21T08:38:54.886895+0000","last_peered":"2024-04-21T08:38:54.886895+0000","last_clean":"2024-04-21T08:38:54.886895+0000","last_became_active":"2024-04-21T08:36:28.006006+0000","last_became_peered":"2024-04-21T08:36:28.006006+0000","last_unstale":"2024-04-21T08:38:54.886895+0000","last_undegraded":"2024-04-21T08:38:54.886895+0000","last_fullsized":"2024-04-21T08:38:54.886895+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:16.173284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'8","reported_seq":302,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:39:00.080410+0000","last_change":"2024-04-21T08:36:23.471541+0000","last_active":"2024-04-21T08:39:00.080410+0000","last_peered":"2024-04-21T08:39:00.080410+0000","last_clean":"2024-04-21T08:39:00.080410+0000","last_became_active":"2024-04-21T08:36:23.471193+0000","last_became_peered":"2024-04-21T08:36:23.471193+0000","last_unstale":"2024-04-21T08:39:00.080410+0000","last_undegraded":"2024-04-21T08:39:00.080410+0000","last_fullsized":"2024-04-21T08:39:00.080410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:06:04.916707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":260,"num_read_kb":260,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104848+0000","last_change":"2024-04-21T08:36:23.986781+0000","last_active":"2024-04-21T08:38:55.104848+0000","last_peered":"2024-04-21T08:38:55.104848+0000","last_clean":"2024-04-21T08:38:55.104848+0000","last_became_active":"2024-04-21T08:36:23.985696+0000","last_became_peered":"2024-04-21T08:36:23.985696+0000","last_unstale":"2024-04-21T08:38:55.104848+0000","last_undegraded":"2024-04-21T08:38:55.104848+0000","last_fullsized":"2024-04-21T08:38:55.104848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:40.556642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.995747+0000","last_change":"2024-04-21T08:36:28.011621+0000","last_active":"2024-04-21T08:38:54.995747+0000","last_peered":"2024-04-21T08:38:54.995747+0000","last_clean":"2024-04-21T08:38:54.995747+0000","last_became_active":"2024-04-21T08:36:28.011136+0000","last_became_peered":"2024-04-21T08:36:28.011136+0000","last_unstale":"2024-04-21T08:38:54.995747+0000","last_undegraded":"2024-04-21T08:38:54.995747+0000","last_fullsized":"2024-04-21T08:38:54.995747+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:47.623445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006842+0000","last_change":"2024-04-21T08:36:23.474019+0000","last_active":"2024-04-21T08:38:55.006842+0000","last_peered":"2024-04-21T08:38:55.006842+0000","last_clean":"2024-04-21T08:38:55.006842+0000","last_became_active":"2024-04-21T08:36:23.473690+0000","last_became_peered":"2024-04-21T08:36:23.473690+0000","last_unstale":"2024-04-21T08:38:55.006842+0000","last_undegraded":"2024-04-21T08:38:55.006842+0000","last_fullsized":"2024-04-21T08:38:55.006842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:38:32.913795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004613+0000","last_change":"2024-04-21T08:37:15.121455+0000","last_active":"2024-04-21T08:38:55.004613+0000","last_peered":"2024-04-21T08:38:55.004613+0000","last_clean":"2024-04-21T08:38:55.004613+0000","last_became_active":"2024-04-21T08:37:15.121113+0000","last_became_peered":"2024-04-21T08:37:15.121113+0000","last_unstale":"2024-04-21T08:38:55.004613+0000","last_undegraded":"2024-04-21T08:38:55.004613+0000","last_fullsized":"2024-04-21T08:38:55.004613+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:39:36.525495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006512+0000","last_change":"2024-04-21T08:38:15.413849+0000","last_active":"2024-04-21T08:38:55.006512+0000","last_peered":"2024-04-21T08:38:55.006512+0000","last_clean":"2024-04-21T08:38:55.006512+0000","last_became_active":"2024-04-21T08:38:15.413361+0000","last_became_peered":"2024-04-21T08:38:15.413361+0000","last_unstale":"2024-04-21T08:38:55.006512+0000","last_undegraded":"2024-04-21T08:38:55.006512+0000","last_fullsized":"2024-04-21T08:38:55.006512+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:16:49.197800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008173+0000","last_change":"2024-04-21T08:36:23.465825+0000","last_active":"2024-04-21T08:38:55.008173+0000","last_peered":"2024-04-21T08:38:55.008173+0000","last_clean":"2024-04-21T08:38:55.008173+0000","last_became_active":"2024-04-21T08:36:23.465525+0000","last_became_peered":"2024-04-21T08:36:23.465525+0000","last_unstale":"2024-04-21T08:38:55.008173+0000","last_undegraded":"2024-04-21T08:38:55.008173+0000","last_fullsized":"2024-04-21T08:38:55.008173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:59:50.502210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.105005+0000","last_change":"2024-04-21T08:36:23.987602+0000","last_active":"2024-04-21T08:38:55.105005+0000","last_peered":"2024-04-21T08:38:55.105005+0000","last_clean":"2024-04-21T08:38:55.105005+0000","last_became_active":"2024-04-21T08:36:23.987298+0000","last_became_peered":"2024-04-21T08:36:23.987298+0000","last_unstale":"2024-04-21T08:38:55.105005+0000","last_undegraded":"2024-04-21T08:38:55.105005+0000","last_fullsized":"2024-04-21T08:38:55.105005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:06.647676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.995946+0000","last_change":"2024-04-21T08:36:28.005935+0000","last_active":"2024-04-21T08:38:54.995946+0000","last_peered":"2024-04-21T08:38:54.995946+0000","last_clean":"2024-04-21T08:38:54.995946+0000","last_became_active":"2024-04-21T08:36:28.005512+0000","last_became_peered":"2024-04-21T08:36:28.005512+0000","last_unstale":"2024-04-21T08:38:54.995946+0000","last_undegraded":"2024-04-21T08:38:54.995946+0000","last_fullsized":"2024-04-21T08:38:54.995946+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:23:20.298524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"95'43","reported_seq":353,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:39:00.080992+0000","last_change":"2024-04-21T08:36:23.467728+0000","last_active":"2024-04-21T08:39:00.080992+0000","last_peered":"2024-04-21T08:39:00.080992+0000","last_clean":"2024-04-21T08:39:00.080992+0000","last_became_active":"2024-04-21T08:36:23.467430+0000","last_became_peered":"2024-04-21T08:36:23.467430+0000","last_unstale":"2024-04-21T08:39:00.080992+0000","last_undegraded":"2024-04-21T08:39:00.080992+0000","last_fullsized":"2024-04-21T08:39:00.080992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:47:18.900911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70068,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":276,"num_read_kb":208,"num_write":45,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890735+0000","last_change":"2024-04-21T08:36:23.989116+0000","last_active":"2024-04-21T08:38:54.890735+0000","last_peered":"2024-04-21T08:38:54.890735+0000","last_clean":"2024-04-21T08:38:54.890735+0000","last_became_active":"2024-04-21T08:36:23.988758+0000","last_became_peered":"2024-04-21T08:36:23.988758+0000","last_unstale":"2024-04-21T08:38:54.890735+0000","last_undegraded":"2024-04-21T08:38:54.890735+0000","last_fullsized":"2024-04-21T08:38:54.890735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:56:25.969905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008450+0000","last_change":"2024-04-21T08:36:28.007370+0000","last_active":"2024-04-21T08:38:55.008450+0000","last_peered":"2024-04-21T08:38:55.008450+0000","last_clean":"2024-04-21T08:38:55.008450+0000","last_became_active":"2024-04-21T08:36:28.006972+0000","last_became_peered":"2024-04-21T08:36:28.006972+0000","last_unstale":"2024-04-21T08:38:55.008450+0000","last_undegraded":"2024-04-21T08:38:55.008450+0000","last_fullsized":"2024-04-21T08:38:55.008450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:18:14.926058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104708+0000","last_change":"2024-04-21T08:36:23.472658+0000","last_active":"2024-04-21T08:38:55.104708+0000","last_peered":"2024-04-21T08:38:55.104708+0000","last_clean":"2024-04-21T08:38:55.104708+0000","last_became_active":"2024-04-21T08:36:23.472318+0000","last_became_peered":"2024-04-21T08:36:23.472318+0000","last_unstale":"2024-04-21T08:38:55.104708+0000","last_undegraded":"2024-04-21T08:38:55.104708+0000","last_fullsized":"2024-04-21T08:38:55.104708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:28:27.497605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076501+0000","last_change":"2024-04-21T08:38:15.338892+0000","last_active":"2024-04-21T08:38:55.076501+0000","last_peered":"2024-04-21T08:38:55.076501+0000","last_clean":"2024-04-21T08:38:55.076501+0000","last_became_active":"2024-04-21T08:38:15.338396+0000","last_became_peered":"2024-04-21T08:38:15.338396+0000","last_unstale":"2024-04-21T08:38:55.076501+0000","last_undegraded":"2024-04-21T08:38:55.076501+0000","last_fullsized":"2024-04-21T08:38:55.076501+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:19.670635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005342+0000","last_change":"2024-04-21T08:36:28.019186+0000","last_active":"2024-04-21T08:38:55.005342+0000","last_peered":"2024-04-21T08:38:55.005342+0000","last_clean":"2024-04-21T08:38:55.005342+0000","last_became_active":"2024-04-21T08:36:28.018774+0000","last_became_peered":"2024-04-21T08:36:28.018774+0000","last_unstale":"2024-04-21T08:38:55.005342+0000","last_undegraded":"2024-04-21T08:38:55.005342+0000","last_fullsized":"2024-04-21T08:38:55.005342+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:35:33.548656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008059+0000","last_change":"2024-04-21T08:36:23.472633+0000","last_active":"2024-04-21T08:38:55.008059+0000","last_peered":"2024-04-21T08:38:55.008059+0000","last_clean":"2024-04-21T08:38:55.008059+0000","last_became_active":"2024-04-21T08:36:23.472336+0000","last_became_peered":"2024-04-21T08:36:23.472336+0000","last_unstale":"2024-04-21T08:38:55.008059+0000","last_undegraded":"2024-04-21T08:38:55.008059+0000","last_fullsized":"2024-04-21T08:38:55.008059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:49.463492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006193+0000","last_change":"2024-04-21T08:36:23.994332+0000","last_active":"2024-04-21T08:38:55.006193+0000","last_peered":"2024-04-21T08:38:55.006193+0000","last_clean":"2024-04-21T08:38:55.006193+0000","last_became_active":"2024-04-21T08:36:23.993936+0000","last_became_peered":"2024-04-21T08:36:23.993936+0000","last_unstale":"2024-04-21T08:38:55.006193+0000","last_undegraded":"2024-04-21T08:38:55.006193+0000","last_fullsized":"2024-04-21T08:38:55.006193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:26:35.330559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.996887+0000","last_change":"2024-04-21T08:36:28.012549+0000","last_active":"2024-04-21T08:38:54.996887+0000","last_peered":"2024-04-21T08:38:54.996887+0000","last_clean":"2024-04-21T08:38:54.996887+0000","last_became_active":"2024-04-21T08:36:28.012151+0000","last_became_peered":"2024-04-21T08:36:28.012151+0000","last_unstale":"2024-04-21T08:38:54.996887+0000","last_undegraded":"2024-04-21T08:38:54.996887+0000","last_fullsized":"2024-04-21T08:38:54.996887+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:55.680399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'8","reported_seq":178,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:59.081251+0000","last_change":"2024-04-21T08:36:23.464586+0000","last_active":"2024-04-21T08:38:59.081251+0000","last_peered":"2024-04-21T08:38:59.081251+0000","last_clean":"2024-04-21T08:38:59.081251+0000","last_became_active":"2024-04-21T08:36:23.464281+0000","last_became_peered":"2024-04-21T08:36:23.464281+0000","last_unstale":"2024-04-21T08:38:59.081251+0000","last_undegraded":"2024-04-21T08:38:59.081251+0000","last_fullsized":"2024-04-21T08:38:59.081251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:04:58.577283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":137,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006132+0000","last_change":"2024-04-21T08:36:23.989604+0000","last_active":"2024-04-21T08:38:55.006132+0000","last_peered":"2024-04-21T08:38:55.006132+0000","last_clean":"2024-04-21T08:38:55.006132+0000","last_became_active":"2024-04-21T08:36:23.989257+0000","last_became_peered":"2024-04-21T08:36:23.989257+0000","last_unstale":"2024-04-21T08:38:55.006132+0000","last_undegraded":"2024-04-21T08:38:55.006132+0000","last_fullsized":"2024-04-21T08:38:55.006132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:45.077943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077854+0000","last_change":"2024-04-21T08:36:28.007959+0000","last_active":"2024-04-21T08:38:55.077854+0000","last_peered":"2024-04-21T08:38:55.077854+0000","last_clean":"2024-04-21T08:38:55.077854+0000","last_became_active":"2024-04-21T08:36:28.007571+0000","last_became_peered":"2024-04-21T08:36:28.007571+0000","last_unstale":"2024-04-21T08:38:55.077854+0000","last_undegraded":"2024-04-21T08:38:55.077854+0000","last_fullsized":"2024-04-21T08:38:55.077854+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:17:34.007013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890204+0000","last_change":"2024-04-21T08:36:23.475054+0000","last_active":"2024-04-21T08:38:54.890204+0000","last_peered":"2024-04-21T08:38:54.890204+0000","last_clean":"2024-04-21T08:38:54.890204+0000","last_became_active":"2024-04-21T08:36:23.474606+0000","last_became_peered":"2024-04-21T08:36:23.474606+0000","last_unstale":"2024-04-21T08:38:54.890204+0000","last_undegraded":"2024-04-21T08:38:54.890204+0000","last_fullsized":"2024-04-21T08:38:54.890204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:22.559174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007909+0000","last_change":"2024-04-21T08:36:23.995980+0000","last_active":"2024-04-21T08:38:55.007909+0000","last_peered":"2024-04-21T08:38:55.007909+0000","last_clean":"2024-04-21T08:38:55.007909+0000","last_became_active":"2024-04-21T08:36:23.995673+0000","last_became_peered":"2024-04-21T08:36:23.995673+0000","last_unstale":"2024-04-21T08:38:55.007909+0000","last_undegraded":"2024-04-21T08:38:55.007909+0000","last_fullsized":"2024-04-21T08:38:55.007909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:11:49.884384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007831+0000","last_change":"2024-04-21T08:36:28.018792+0000","last_active":"2024-04-21T08:38:55.007831+0000","last_peered":"2024-04-21T08:38:55.007831+0000","last_clean":"2024-04-21T08:38:55.007831+0000","last_became_active":"2024-04-21T08:36:28.018373+0000","last_became_peered":"2024-04-21T08:36:28.018373+0000","last_unstale":"2024-04-21T08:38:55.007831+0000","last_undegraded":"2024-04-21T08:38:55.007831+0000","last_fullsized":"2024-04-21T08:38:55.007831+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:11.857661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"95'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890811+0000","last_change":"2024-04-21T08:36:23.469787+0000","last_active":"2024-04-21T08:38:54.890811+0000","last_peered":"2024-04-21T08:38:54.890811+0000","last_clean":"2024-04-21T08:38:54.890811+0000","last_became_active":"2024-04-21T08:36:23.469462+0000","last_became_peered":"2024-04-21T08:36:23.469462+0000","last_unstale":"2024-04-21T08:38:54.890811+0000","last_undegraded":"2024-04-21T08:38:54.890811+0000","last_fullsized":"2024-04-21T08:38:54.890811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:33:25.849528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005561+0000","last_change":"2024-04-21T08:36:23.988492+0000","last_active":"2024-04-21T08:38:55.005561+0000","last_peered":"2024-04-21T08:38:55.005561+0000","last_clean":"2024-04-21T08:38:55.005561+0000","last_became_active":"2024-04-21T08:36:23.988133+0000","last_became_peered":"2024-04-21T08:36:23.988133+0000","last_unstale":"2024-04-21T08:38:55.005561+0000","last_undegraded":"2024-04-21T08:38:55.005561+0000","last_fullsized":"2024-04-21T08:38:55.005561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:01:25.320281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890332+0000","last_change":"2024-04-21T08:38:15.415043+0000","last_active":"2024-04-21T08:38:54.890332+0000","last_peered":"2024-04-21T08:38:54.890332+0000","last_clean":"2024-04-21T08:38:54.890332+0000","last_became_active":"2024-04-21T08:38:15.414460+0000","last_became_peered":"2024-04-21T08:38:15.414460+0000","last_unstale":"2024-04-21T08:38:54.890332+0000","last_undegraded":"2024-04-21T08:38:54.890332+0000","last_fullsized":"2024-04-21T08:38:54.890332+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:50:17.202423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.996393+0000","last_change":"2024-04-21T08:37:15.219428+0000","last_active":"2024-04-21T08:38:54.996393+0000","last_peered":"2024-04-21T08:38:54.996393+0000","last_clean":"2024-04-21T08:38:54.996393+0000","last_became_active":"2024-04-21T08:37:15.218923+0000","last_became_peered":"2024-04-21T08:37:15.218923+0000","last_unstale":"2024-04-21T08:38:54.996393+0000","last_undegraded":"2024-04-21T08:38:54.996393+0000","last_fullsized":"2024-04-21T08:38:54.996393+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:07:02.781701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888120+0000","last_change":"2024-04-21T08:36:23.985910+0000","last_active":"2024-04-21T08:38:54.888120+0000","last_peered":"2024-04-21T08:38:54.888120+0000","last_clean":"2024-04-21T08:38:54.888120+0000","last_became_active":"2024-04-21T08:36:23.985605+0000","last_became_peered":"2024-04-21T08:36:23.985605+0000","last_unstale":"2024-04-21T08:38:54.888120+0000","last_undegraded":"2024-04-21T08:38:54.888120+0000","last_fullsized":"2024-04-21T08:38:54.888120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:37:09.367005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077277+0000","last_change":"2024-04-21T08:36:28.015256+0000","last_active":"2024-04-21T08:38:55.077277+0000","last_peered":"2024-04-21T08:38:55.077277+0000","last_clean":"2024-04-21T08:38:55.077277+0000","last_became_active":"2024-04-21T08:36:28.014849+0000","last_became_peered":"2024-04-21T08:36:28.014849+0000","last_unstale":"2024-04-21T08:38:55.077277+0000","last_undegraded":"2024-04-21T08:38:55.077277+0000","last_fullsized":"2024-04-21T08:38:55.077277+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:22:05.466299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"95'6","reported_seq":171,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:59.080426+0000","last_change":"2024-04-21T08:36:23.474966+0000","last_active":"2024-04-21T08:38:59.080426+0000","last_peered":"2024-04-21T08:38:59.080426+0000","last_clean":"2024-04-21T08:38:59.080426+0000","last_became_active":"2024-04-21T08:36:23.474374+0000","last_became_peered":"2024-04-21T08:36:23.474374+0000","last_unstale":"2024-04-21T08:38:59.080426+0000","last_undegraded":"2024-04-21T08:38:59.080426+0000","last_fullsized":"2024-04-21T08:38:59.080426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:57:20.237966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888138+0000","last_change":"2024-04-21T08:37:15.118887+0000","last_active":"2024-04-21T08:38:54.888138+0000","last_peered":"2024-04-21T08:38:54.888138+0000","last_clean":"2024-04-21T08:38:54.888138+0000","last_became_active":"2024-04-21T08:37:15.118510+0000","last_became_peered":"2024-04-21T08:37:15.118510+0000","last_unstale":"2024-04-21T08:38:54.888138+0000","last_undegraded":"2024-04-21T08:38:54.888138+0000","last_fullsized":"2024-04-21T08:38:54.888138+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:59:27.209083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077841+0000","last_change":"2024-04-21T08:36:28.013169+0000","last_active":"2024-04-21T08:38:55.077841+0000","last_peered":"2024-04-21T08:38:55.077841+0000","last_clean":"2024-04-21T08:38:55.077841+0000","last_became_active":"2024-04-21T08:36:28.012791+0000","last_became_peered":"2024-04-21T08:36:28.012791+0000","last_unstale":"2024-04-21T08:38:55.077841+0000","last_undegraded":"2024-04-21T08:38:55.077841+0000","last_fullsized":"2024-04-21T08:38:55.077841+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:42:56.063159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007983+0000","last_change":"2024-04-21T08:36:23.995209+0000","last_active":"2024-04-21T08:38:55.007983+0000","last_peered":"2024-04-21T08:38:55.007983+0000","last_clean":"2024-04-21T08:38:55.007983+0000","last_became_active":"2024-04-21T08:36:23.994866+0000","last_became_peered":"2024-04-21T08:36:23.994866+0000","last_unstale":"2024-04-21T08:38:55.007983+0000","last_undegraded":"2024-04-21T08:38:55.007983+0000","last_fullsized":"2024-04-21T08:38:55.007983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:33:45.582150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'7","reported_seq":173,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:59.080838+0000","last_change":"2024-04-21T08:36:23.471474+0000","last_active":"2024-04-21T08:38:59.080838+0000","last_peered":"2024-04-21T08:38:59.080838+0000","last_clean":"2024-04-21T08:38:59.080838+0000","last_became_active":"2024-04-21T08:36:23.471179+0000","last_became_peered":"2024-04-21T08:36:23.471179+0000","last_unstale":"2024-04-21T08:38:59.080838+0000","last_undegraded":"2024-04-21T08:38:59.080838+0000","last_fullsized":"2024-04-21T08:38:59.080838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:17.290051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":132,"num_read_kb":132,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006553+0000","last_change":"2024-04-21T08:38:15.344332+0000","last_active":"2024-04-21T08:38:55.006553+0000","last_peered":"2024-04-21T08:38:55.006553+0000","last_clean":"2024-04-21T08:38:55.006553+0000","last_became_active":"2024-04-21T08:38:15.339914+0000","last_became_peered":"2024-04-21T08:38:15.339914+0000","last_unstale":"2024-04-21T08:38:55.006553+0000","last_undegraded":"2024-04-21T08:38:55.006553+0000","last_fullsized":"2024-04-21T08:38:55.006553+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:39:24.049323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886832+0000","last_change":"2024-04-21T08:36:23.990272+0000","last_active":"2024-04-21T08:38:54.886832+0000","last_peered":"2024-04-21T08:38:54.886832+0000","last_clean":"2024-04-21T08:38:54.886832+0000","last_became_active":"2024-04-21T08:36:23.989940+0000","last_became_peered":"2024-04-21T08:36:23.989940+0000","last_unstale":"2024-04-21T08:38:54.886832+0000","last_undegraded":"2024-04-21T08:38:54.886832+0000","last_fullsized":"2024-04-21T08:38:54.886832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:27:50.705167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006322+0000","last_change":"2024-04-21T08:36:23.474162+0000","last_active":"2024-04-21T08:38:55.006322+0000","last_peered":"2024-04-21T08:38:55.006322+0000","last_clean":"2024-04-21T08:38:55.006322+0000","last_became_active":"2024-04-21T08:36:23.473802+0000","last_became_peered":"2024-04-21T08:36:23.473802+0000","last_unstale":"2024-04-21T08:38:55.006322+0000","last_undegraded":"2024-04-21T08:38:55.006322+0000","last_fullsized":"2024-04-21T08:38:55.006322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:32:41.918431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"95'148","reported_seq":410,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:57.010247+0000","last_change":"2024-04-21T08:35:10.062130+0000","last_active":"2024-04-21T08:38:57.010247+0000","last_peered":"2024-04-21T08:38:57.010247+0000","last_clean":"2024-04-21T08:38:57.010247+0000","last_became_active":"2024-04-21T08:35:10.061727+0000","last_became_peered":"2024-04-21T08:35:10.061727+0000","last_unstale":"2024-04-21T08:38:57.010247+0000","last_undegraded":"2024-04-21T08:38:57.010247+0000","last_fullsized":"2024-04-21T08:38:57.010247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:31:40.950939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:31:40.950939+0000","last_clean_scrub_stamp":"2024-04-21T08:31:40.950939+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:35:52.360167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008148+0000","last_change":"2024-04-21T08:38:15.415007+0000","last_active":"2024-04-21T08:38:55.008148+0000","last_peered":"2024-04-21T08:38:55.008148+0000","last_clean":"2024-04-21T08:38:55.008148+0000","last_became_active":"2024-04-21T08:38:15.414478+0000","last_became_peered":"2024-04-21T08:38:15.414478+0000","last_unstale":"2024-04-21T08:38:55.008148+0000","last_undegraded":"2024-04-21T08:38:55.008148+0000","last_fullsized":"2024-04-21T08:38:55.008148+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:27.703964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.075651+0000","last_change":"2024-04-21T08:36:23.984332+0000","last_active":"2024-04-21T08:38:55.075651+0000","last_peered":"2024-04-21T08:38:55.075651+0000","last_clean":"2024-04-21T08:38:55.075651+0000","last_became_active":"2024-04-21T08:36:23.984021+0000","last_became_peered":"2024-04-21T08:36:23.984021+0000","last_unstale":"2024-04-21T08:38:55.075651+0000","last_undegraded":"2024-04-21T08:38:55.075651+0000","last_fullsized":"2024-04-21T08:38:55.075651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:05:21.090633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005972+0000","last_change":"2024-04-21T08:36:23.467522+0000","last_active":"2024-04-21T08:38:55.005972+0000","last_peered":"2024-04-21T08:38:55.005972+0000","last_clean":"2024-04-21T08:38:55.005972+0000","last_became_active":"2024-04-21T08:36:23.466379+0000","last_became_peered":"2024-04-21T08:36:23.466379+0000","last_unstale":"2024-04-21T08:38:55.005972+0000","last_undegraded":"2024-04-21T08:38:55.005972+0000","last_fullsized":"2024-04-21T08:38:55.005972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:04.185314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005193+0000","last_change":"2024-04-21T08:36:28.015120+0000","last_active":"2024-04-21T08:38:55.005193+0000","last_peered":"2024-04-21T08:38:55.005193+0000","last_clean":"2024-04-21T08:38:55.005193+0000","last_became_active":"2024-04-21T08:36:28.014701+0000","last_became_peered":"2024-04-21T08:36:28.014701+0000","last_unstale":"2024-04-21T08:38:55.005193+0000","last_undegraded":"2024-04-21T08:38:55.005193+0000","last_fullsized":"2024-04-21T08:38:55.005193+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:23.127622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890642+0000","last_change":"2024-04-21T08:36:23.475335+0000","last_active":"2024-04-21T08:38:54.890642+0000","last_peered":"2024-04-21T08:38:54.890642+0000","last_clean":"2024-04-21T08:38:54.890642+0000","last_became_active":"2024-04-21T08:36:23.474739+0000","last_became_peered":"2024-04-21T08:36:23.474739+0000","last_unstale":"2024-04-21T08:38:54.890642+0000","last_undegraded":"2024-04-21T08:38:54.890642+0000","last_fullsized":"2024-04-21T08:38:54.890642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:36.119649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077395+0000","last_change":"2024-04-21T08:36:23.977229+0000","last_active":"2024-04-21T08:38:55.077395+0000","last_peered":"2024-04-21T08:38:55.077395+0000","last_clean":"2024-04-21T08:38:55.077395+0000","last_became_active":"2024-04-21T08:36:23.976893+0000","last_became_peered":"2024-04-21T08:36:23.976893+0000","last_unstale":"2024-04-21T08:38:55.077395+0000","last_undegraded":"2024-04-21T08:38:55.077395+0000","last_fullsized":"2024-04-21T08:38:55.077395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:31:54.226274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886835+0000","last_change":"2024-04-21T08:36:28.018581+0000","last_active":"2024-04-21T08:38:54.886835+0000","last_peered":"2024-04-21T08:38:54.886835+0000","last_clean":"2024-04-21T08:38:54.886835+0000","last_became_active":"2024-04-21T08:36:28.018021+0000","last_became_peered":"2024-04-21T08:36:28.018021+0000","last_unstale":"2024-04-21T08:38:54.886835+0000","last_undegraded":"2024-04-21T08:38:54.886835+0000","last_fullsized":"2024-04-21T08:38:54.886835+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:29:00.355532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'73","reported_seq":160,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:39:00.890509+0000","last_change":"2024-04-21T08:36:23.473339+0000","last_active":"2024-04-21T08:39:00.890509+0000","last_peered":"2024-04-21T08:39:00.890509+0000","last_clean":"2024-04-21T08:39:00.890509+0000","last_became_active":"2024-04-21T08:36:23.472994+0000","last_became_peered":"2024-04-21T08:36:23.472994+0000","last_unstale":"2024-04-21T08:39:00.890509+0000","last_undegraded":"2024-04-21T08:39:00.890509+0000","last_fullsized":"2024-04-21T08:39:00.890509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:42.762273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":182,"num_read_kb":160,"num_write":129,"num_write_kb":1156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887378+0000","last_change":"2024-04-21T08:36:23.987354+0000","last_active":"2024-04-21T08:38:54.887378+0000","last_peered":"2024-04-21T08:38:54.887378+0000","last_clean":"2024-04-21T08:38:54.887378+0000","last_became_active":"2024-04-21T08:36:23.987032+0000","last_became_peered":"2024-04-21T08:36:23.987032+0000","last_unstale":"2024-04-21T08:38:54.887378+0000","last_undegraded":"2024-04-21T08:38:54.887378+0000","last_fullsized":"2024-04-21T08:38:54.887378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:01:05.164367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104455+0000","last_change":"2024-04-21T08:36:28.002582+0000","last_active":"2024-04-21T08:38:55.104455+0000","last_peered":"2024-04-21T08:38:55.104455+0000","last_clean":"2024-04-21T08:38:55.104455+0000","last_became_active":"2024-04-21T08:36:28.002133+0000","last_became_peered":"2024-04-21T08:36:28.002133+0000","last_unstale":"2024-04-21T08:38:55.104455+0000","last_undegraded":"2024-04-21T08:38:55.104455+0000","last_fullsized":"2024-04-21T08:38:55.104455+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:30:00.268015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007058+0000","last_change":"2024-04-21T08:36:23.462840+0000","last_active":"2024-04-21T08:38:55.007058+0000","last_peered":"2024-04-21T08:38:55.007058+0000","last_clean":"2024-04-21T08:38:55.007058+0000","last_became_active":"2024-04-21T08:36:23.462391+0000","last_became_peered":"2024-04-21T08:36:23.462391+0000","last_unstale":"2024-04-21T08:38:55.007058+0000","last_undegraded":"2024-04-21T08:38:55.007058+0000","last_fullsized":"2024-04-21T08:38:55.007058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:33.230388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004380+0000","last_change":"2024-04-21T08:36:23.994751+0000","last_active":"2024-04-21T08:38:55.004380+0000","last_peered":"2024-04-21T08:38:55.004380+0000","last_clean":"2024-04-21T08:38:55.004380+0000","last_became_active":"2024-04-21T08:36:23.994384+0000","last_became_peered":"2024-04-21T08:36:23.994384+0000","last_unstale":"2024-04-21T08:38:55.004380+0000","last_undegraded":"2024-04-21T08:38:55.004380+0000","last_fullsized":"2024-04-21T08:38:55.004380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:18.956893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006981+0000","last_change":"2024-04-21T08:36:28.004004+0000","last_active":"2024-04-21T08:38:55.006981+0000","last_peered":"2024-04-21T08:38:55.006981+0000","last_clean":"2024-04-21T08:38:55.006981+0000","last_became_active":"2024-04-21T08:36:28.003547+0000","last_became_peered":"2024-04-21T08:36:28.003547+0000","last_unstale":"2024-04-21T08:38:55.006981+0000","last_undegraded":"2024-04-21T08:38:55.006981+0000","last_fullsized":"2024-04-21T08:38:55.006981+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:21:40.215428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104510+0000","last_change":"2024-04-21T08:37:15.222118+0000","last_active":"2024-04-21T08:38:55.104510+0000","last_peered":"2024-04-21T08:38:55.104510+0000","last_clean":"2024-04-21T08:38:55.104510+0000","last_became_active":"2024-04-21T08:37:15.219173+0000","last_became_peered":"2024-04-21T08:37:15.219173+0000","last_unstale":"2024-04-21T08:38:55.104510+0000","last_undegraded":"2024-04-21T08:38:55.104510+0000","last_fullsized":"2024-04-21T08:38:55.104510+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:12.931076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890299+0000","last_change":"2024-04-21T08:36:23.988020+0000","last_active":"2024-04-21T08:38:54.890299+0000","last_peered":"2024-04-21T08:38:54.890299+0000","last_clean":"2024-04-21T08:38:54.890299+0000","last_became_active":"2024-04-21T08:36:23.987630+0000","last_became_peered":"2024-04-21T08:36:23.987630+0000","last_unstale":"2024-04-21T08:38:54.890299+0000","last_undegraded":"2024-04-21T08:38:54.890299+0000","last_fullsized":"2024-04-21T08:38:54.890299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:06.519001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.997084+0000","last_change":"2024-04-21T08:36:28.006984+0000","last_active":"2024-04-21T08:38:54.997084+0000","last_peered":"2024-04-21T08:38:54.997084+0000","last_clean":"2024-04-21T08:38:54.997084+0000","last_became_active":"2024-04-21T08:36:28.006548+0000","last_became_peered":"2024-04-21T08:36:28.006548+0000","last_unstale":"2024-04-21T08:38:54.997084+0000","last_undegraded":"2024-04-21T08:38:54.997084+0000","last_fullsized":"2024-04-21T08:38:54.997084+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:25:50.870658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886907+0000","last_change":"2024-04-21T08:36:23.476266+0000","last_active":"2024-04-21T08:38:54.886907+0000","last_peered":"2024-04-21T08:38:54.886907+0000","last_clean":"2024-04-21T08:38:54.886907+0000","last_became_active":"2024-04-21T08:36:23.475948+0000","last_became_peered":"2024-04-21T08:36:23.475948+0000","last_unstale":"2024-04-21T08:38:54.886907+0000","last_undegraded":"2024-04-21T08:38:54.886907+0000","last_fullsized":"2024-04-21T08:38:54.886907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:41.529376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005217+0000","last_change":"2024-04-21T08:36:23.981942+0000","last_active":"2024-04-21T08:38:55.005217+0000","last_peered":"2024-04-21T08:38:55.005217+0000","last_clean":"2024-04-21T08:38:55.005217+0000","last_became_active":"2024-04-21T08:36:23.981643+0000","last_became_peered":"2024-04-21T08:36:23.981643+0000","last_unstale":"2024-04-21T08:38:55.005217+0000","last_undegraded":"2024-04-21T08:38:55.005217+0000","last_fullsized":"2024-04-21T08:38:55.005217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:26:24.722695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006307+0000","last_change":"2024-04-21T08:36:28.006980+0000","last_active":"2024-04-21T08:38:55.006307+0000","last_peered":"2024-04-21T08:38:55.006307+0000","last_clean":"2024-04-21T08:38:55.006307+0000","last_became_active":"2024-04-21T08:36:28.006530+0000","last_became_peered":"2024-04-21T08:36:28.006530+0000","last_unstale":"2024-04-21T08:38:55.006307+0000","last_undegraded":"2024-04-21T08:38:55.006307+0000","last_fullsized":"2024-04-21T08:38:55.006307+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:05:57.465612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"95'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104627+0000","last_change":"2024-04-21T08:36:23.473253+0000","last_active":"2024-04-21T08:38:55.104627+0000","last_peered":"2024-04-21T08:38:55.104627+0000","last_clean":"2024-04-21T08:38:55.104627+0000","last_became_active":"2024-04-21T08:36:23.472942+0000","last_became_peered":"2024-04-21T08:36:23.472942+0000","last_unstale":"2024-04-21T08:38:55.104627+0000","last_undegraded":"2024-04-21T08:38:55.104627+0000","last_fullsized":"2024-04-21T08:38:55.104627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:32:04.894388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887248+0000","last_change":"2024-04-21T08:36:23.988355+0000","last_active":"2024-04-21T08:38:54.887248+0000","last_peered":"2024-04-21T08:38:54.887248+0000","last_clean":"2024-04-21T08:38:54.887248+0000","last_became_active":"2024-04-21T08:36:23.988049+0000","last_became_peered":"2024-04-21T08:36:23.988049+0000","last_unstale":"2024-04-21T08:38:54.887248+0000","last_undegraded":"2024-04-21T08:38:54.887248+0000","last_fullsized":"2024-04-21T08:38:54.887248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:57.348698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006797+0000","last_change":"2024-04-21T08:36:28.007343+0000","last_active":"2024-04-21T08:38:55.006797+0000","last_peered":"2024-04-21T08:38:55.006797+0000","last_clean":"2024-04-21T08:38:55.006797+0000","last_became_active":"2024-04-21T08:36:28.006719+0000","last_became_peered":"2024-04-21T08:36:28.006719+0000","last_unstale":"2024-04-21T08:38:55.006797+0000","last_undegraded":"2024-04-21T08:38:55.006797+0000","last_fullsized":"2024-04-21T08:38:55.006797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:02:25.734584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890843+0000","last_change":"2024-04-21T08:36:23.472416+0000","last_active":"2024-04-21T08:38:54.890843+0000","last_peered":"2024-04-21T08:38:54.890843+0000","last_clean":"2024-04-21T08:38:54.890843+0000","last_became_active":"2024-04-21T08:36:23.471500+0000","last_became_peered":"2024-04-21T08:36:23.471500+0000","last_unstale":"2024-04-21T08:38:54.890843+0000","last_undegraded":"2024-04-21T08:38:54.890843+0000","last_fullsized":"2024-04-21T08:38:54.890843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:39.706432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006128+0000","last_change":"2024-04-21T08:36:23.990653+0000","last_active":"2024-04-21T08:38:55.006128+0000","last_peered":"2024-04-21T08:38:55.006128+0000","last_clean":"2024-04-21T08:38:55.006128+0000","last_became_active":"2024-04-21T08:36:23.990324+0000","last_became_peered":"2024-04-21T08:36:23.990324+0000","last_unstale":"2024-04-21T08:38:55.006128+0000","last_undegraded":"2024-04-21T08:38:55.006128+0000","last_fullsized":"2024-04-21T08:38:55.006128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:08:59.417850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104492+0000","last_change":"2024-04-21T08:36:28.007956+0000","last_active":"2024-04-21T08:38:55.104492+0000","last_peered":"2024-04-21T08:38:55.104492+0000","last_clean":"2024-04-21T08:38:55.104492+0000","last_became_active":"2024-04-21T08:36:28.007475+0000","last_became_peered":"2024-04-21T08:36:28.007475+0000","last_unstale":"2024-04-21T08:38:55.104492+0000","last_undegraded":"2024-04-21T08:38:55.104492+0000","last_fullsized":"2024-04-21T08:38:55.104492+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:54.039359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'8","reported_seq":175,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:59.164772+0000","last_change":"2024-04-21T08:36:23.476887+0000","last_active":"2024-04-21T08:38:59.164772+0000","last_peered":"2024-04-21T08:38:59.164772+0000","last_clean":"2024-04-21T08:38:59.164772+0000","last_became_active":"2024-04-21T08:36:23.476574+0000","last_became_peered":"2024-04-21T08:36:23.476574+0000","last_unstale":"2024-04-21T08:38:59.164772+0000","last_undegraded":"2024-04-21T08:38:59.164772+0000","last_fullsized":"2024-04-21T08:38:59.164772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:37:47.692358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2010,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006973+0000","last_change":"2024-04-21T08:36:23.982845+0000","last_active":"2024-04-21T08:38:55.006973+0000","last_peered":"2024-04-21T08:38:55.006973+0000","last_clean":"2024-04-21T08:38:55.006973+0000","last_became_active":"2024-04-21T08:36:23.982491+0000","last_became_peered":"2024-04-21T08:36:23.982491+0000","last_unstale":"2024-04-21T08:38:55.006973+0000","last_undegraded":"2024-04-21T08:38:55.006973+0000","last_fullsized":"2024-04-21T08:38:55.006973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:48:46.413896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.002896+0000","last_change":"2024-04-21T08:36:28.010442+0000","last_active":"2024-04-21T08:38:55.002896+0000","last_peered":"2024-04-21T08:38:55.002896+0000","last_clean":"2024-04-21T08:38:55.002896+0000","last_became_active":"2024-04-21T08:36:28.010046+0000","last_became_peered":"2024-04-21T08:36:28.010046+0000","last_unstale":"2024-04-21T08:38:55.002896+0000","last_undegraded":"2024-04-21T08:38:55.002896+0000","last_fullsized":"2024-04-21T08:38:55.002896+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:18:36.382802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"95'59","reported_seq":111,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:39:00.885741+0000","last_change":"2024-04-21T08:36:23.473545+0000","last_active":"2024-04-21T08:39:00.885741+0000","last_peered":"2024-04-21T08:39:00.885741+0000","last_clean":"2024-04-21T08:39:00.885741+0000","last_became_active":"2024-04-21T08:36:23.473222+0000","last_became_peered":"2024-04-21T08:36:23.473222+0000","last_unstale":"2024-04-21T08:39:00.885741+0000","last_undegraded":"2024-04-21T08:39:00.885741+0000","last_fullsized":"2024-04-21T08:39:00.885741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:47:53.947187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":20,"num_write":68,"num_write_kb":807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006600+0000","last_change":"2024-04-21T08:36:23.990959+0000","last_active":"2024-04-21T08:38:55.006600+0000","last_peered":"2024-04-21T08:38:55.006600+0000","last_clean":"2024-04-21T08:38:55.006600+0000","last_became_active":"2024-04-21T08:36:23.990322+0000","last_became_peered":"2024-04-21T08:36:23.990322+0000","last_unstale":"2024-04-21T08:38:55.006600+0000","last_undegraded":"2024-04-21T08:38:55.006600+0000","last_fullsized":"2024-04-21T08:38:55.006600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:12:20.859224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007717+0000","last_change":"2024-04-21T08:36:28.006682+0000","last_active":"2024-04-21T08:38:55.007717+0000","last_peered":"2024-04-21T08:38:55.007717+0000","last_clean":"2024-04-21T08:38:55.007717+0000","last_became_active":"2024-04-21T08:36:28.006229+0000","last_became_peered":"2024-04-21T08:36:28.006229+0000","last_unstale":"2024-04-21T08:38:55.007717+0000","last_undegraded":"2024-04-21T08:38:55.007717+0000","last_fullsized":"2024-04-21T08:38:55.007717+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:30:12.723848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077921+0000","last_change":"2024-04-21T08:36:23.473592+0000","last_active":"2024-04-21T08:38:55.077921+0000","last_peered":"2024-04-21T08:38:55.077921+0000","last_clean":"2024-04-21T08:38:55.077921+0000","last_became_active":"2024-04-21T08:36:23.473283+0000","last_became_peered":"2024-04-21T08:36:23.473283+0000","last_unstale":"2024-04-21T08:38:55.077921+0000","last_undegraded":"2024-04-21T08:38:55.077921+0000","last_fullsized":"2024-04-21T08:38:55.077921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:10.425249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006895+0000","last_change":"2024-04-21T08:36:23.985052+0000","last_active":"2024-04-21T08:38:55.006895+0000","last_peered":"2024-04-21T08:38:55.006895+0000","last_clean":"2024-04-21T08:38:55.006895+0000","last_became_active":"2024-04-21T08:36:23.984742+0000","last_became_peered":"2024-04-21T08:36:23.984742+0000","last_unstale":"2024-04-21T08:38:55.006895+0000","last_undegraded":"2024-04-21T08:38:55.006895+0000","last_fullsized":"2024-04-21T08:38:55.006895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:08:06.082974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005201+0000","last_change":"2024-04-21T08:36:23.980905+0000","last_active":"2024-04-21T08:38:55.005201+0000","last_peered":"2024-04-21T08:38:55.005201+0000","last_clean":"2024-04-21T08:38:55.005201+0000","last_became_active":"2024-04-21T08:36:23.980590+0000","last_became_peered":"2024-04-21T08:36:23.980590+0000","last_unstale":"2024-04-21T08:38:55.005201+0000","last_undegraded":"2024-04-21T08:38:55.005201+0000","last_fullsized":"2024-04-21T08:38:55.005201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:54:59.886014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104725+0000","last_change":"2024-04-21T08:36:23.980883+0000","last_active":"2024-04-21T08:38:55.104725+0000","last_peered":"2024-04-21T08:38:55.104725+0000","last_clean":"2024-04-21T08:38:55.104725+0000","last_became_active":"2024-04-21T08:36:23.980562+0000","last_became_peered":"2024-04-21T08:36:23.980562+0000","last_unstale":"2024-04-21T08:38:55.104725+0000","last_undegraded":"2024-04-21T08:38:55.104725+0000","last_fullsized":"2024-04-21T08:38:55.104725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:14:51.868438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104692+0000","last_change":"2024-04-21T08:36:23.991571+0000","last_active":"2024-04-21T08:38:55.104692+0000","last_peered":"2024-04-21T08:38:55.104692+0000","last_clean":"2024-04-21T08:38:55.104692+0000","last_became_active":"2024-04-21T08:36:23.991273+0000","last_became_peered":"2024-04-21T08:36:23.991273+0000","last_unstale":"2024-04-21T08:38:55.104692+0000","last_undegraded":"2024-04-21T08:38:55.104692+0000","last_fullsized":"2024-04-21T08:38:55.104692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:08:23.895038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.003835+0000","last_change":"2024-04-21T08:36:23.994729+0000","last_active":"2024-04-21T08:38:55.003835+0000","last_peered":"2024-04-21T08:38:55.003835+0000","last_clean":"2024-04-21T08:38:55.003835+0000","last_became_active":"2024-04-21T08:36:23.994405+0000","last_became_peered":"2024-04-21T08:36:23.994405+0000","last_unstale":"2024-04-21T08:38:55.003835+0000","last_undegraded":"2024-04-21T08:38:55.003835+0000","last_fullsized":"2024-04-21T08:38:55.003835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:42.463225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077385+0000","last_change":"2024-04-21T08:36:23.995421+0000","last_active":"2024-04-21T08:38:55.077385+0000","last_peered":"2024-04-21T08:38:55.077385+0000","last_clean":"2024-04-21T08:38:55.077385+0000","last_became_active":"2024-04-21T08:36:23.993674+0000","last_became_peered":"2024-04-21T08:36:23.993674+0000","last_unstale":"2024-04-21T08:38:55.077385+0000","last_undegraded":"2024-04-21T08:38:55.077385+0000","last_fullsized":"2024-04-21T08:38:55.077385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:31.013874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005504+0000","last_change":"2024-04-21T08:36:23.986418+0000","last_active":"2024-04-21T08:38:55.005504+0000","last_peered":"2024-04-21T08:38:55.005504+0000","last_clean":"2024-04-21T08:38:55.005504+0000","last_became_active":"2024-04-21T08:36:23.986095+0000","last_became_peered":"2024-04-21T08:36:23.986095+0000","last_unstale":"2024-04-21T08:38:55.005504+0000","last_undegraded":"2024-04-21T08:38:55.005504+0000","last_fullsized":"2024-04-21T08:38:55.005504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:31:56.456218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104918+0000","last_change":"2024-04-21T08:36:23.982353+0000","last_active":"2024-04-21T08:38:55.104918+0000","last_peered":"2024-04-21T08:38:55.104918+0000","last_clean":"2024-04-21T08:38:55.104918+0000","last_became_active":"2024-04-21T08:36:23.982035+0000","last_became_peered":"2024-04-21T08:36:23.982035+0000","last_unstale":"2024-04-21T08:38:55.104918+0000","last_undegraded":"2024-04-21T08:38:55.104918+0000","last_fullsized":"2024-04-21T08:38:55.104918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:42:44.598407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887832+0000","last_change":"2024-04-21T08:36:23.996371+0000","last_active":"2024-04-21T08:38:54.887832+0000","last_peered":"2024-04-21T08:38:54.887832+0000","last_clean":"2024-04-21T08:38:54.887832+0000","last_became_active":"2024-04-21T08:36:23.995802+0000","last_became_peered":"2024-04-21T08:36:23.995802+0000","last_unstale":"2024-04-21T08:38:54.887832+0000","last_undegraded":"2024-04-21T08:38:54.887832+0000","last_fullsized":"2024-04-21T08:38:54.887832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:52:08.571567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007826+0000","last_change":"2024-04-21T08:36:23.987275+0000","last_active":"2024-04-21T08:38:55.007826+0000","last_peered":"2024-04-21T08:38:55.007826+0000","last_clean":"2024-04-21T08:38:55.007826+0000","last_became_active":"2024-04-21T08:36:23.986967+0000","last_became_peered":"2024-04-21T08:36:23.986967+0000","last_unstale":"2024-04-21T08:38:55.007826+0000","last_undegraded":"2024-04-21T08:38:55.007826+0000","last_fullsized":"2024-04-21T08:38:55.007826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:50.432502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886991+0000","last_change":"2024-04-21T08:36:23.993046+0000","last_active":"2024-04-21T08:38:54.886991+0000","last_peered":"2024-04-21T08:38:54.886991+0000","last_clean":"2024-04-21T08:38:54.886991+0000","last_became_active":"2024-04-21T08:36:23.992693+0000","last_became_peered":"2024-04-21T08:36:23.992693+0000","last_unstale":"2024-04-21T08:38:54.886991+0000","last_undegraded":"2024-04-21T08:38:54.886991+0000","last_fullsized":"2024-04-21T08:38:54.886991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:12:06.287585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104706+0000","last_change":"2024-04-21T08:36:23.992968+0000","last_active":"2024-04-21T08:38:55.104706+0000","last_peered":"2024-04-21T08:38:55.104706+0000","last_clean":"2024-04-21T08:38:55.104706+0000","last_became_active":"2024-04-21T08:36:23.992614+0000","last_became_peered":"2024-04-21T08:36:23.992614+0000","last_unstale":"2024-04-21T08:38:55.104706+0000","last_undegraded":"2024-04-21T08:38:55.104706+0000","last_fullsized":"2024-04-21T08:38:55.104706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:10.015323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076758+0000","last_change":"2024-04-21T08:36:23.981169+0000","last_active":"2024-04-21T08:38:55.076758+0000","last_peered":"2024-04-21T08:38:55.076758+0000","last_clean":"2024-04-21T08:38:55.076758+0000","last_became_active":"2024-04-21T08:36:23.980849+0000","last_became_peered":"2024-04-21T08:36:23.980849+0000","last_unstale":"2024-04-21T08:38:55.076758+0000","last_undegraded":"2024-04-21T08:38:55.076758+0000","last_fullsized":"2024-04-21T08:38:55.076758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:13:49.907370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005358+0000","last_change":"2024-04-21T08:36:23.993998+0000","last_active":"2024-04-21T08:38:55.005358+0000","last_peered":"2024-04-21T08:38:55.005358+0000","last_clean":"2024-04-21T08:38:55.005358+0000","last_became_active":"2024-04-21T08:36:23.993664+0000","last_became_peered":"2024-04-21T08:36:23.993664+0000","last_unstale":"2024-04-21T08:38:55.005358+0000","last_undegraded":"2024-04-21T08:38:55.005358+0000","last_fullsized":"2024-04-21T08:38:55.005358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:31:49.076820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":801015,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1558,"num_read_kb":947,"num_write":445,"num_write_kb":2130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2403045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":353,"ondisk_log_size":353,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351020,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1093632,"data_stored":535779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612658,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37220,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707932,"statfs":{"total":95995035648,"available":95956922368,"internally_reserved":0,"allocated":1929216,"data_stored":1324346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776183,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36820,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708332,"statfs":{"total":95995035648,"available":95957331968,"internally_reserved":0,"allocated":1515520,"data_stored":932181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037820,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":932559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299456,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1712128,"data_stored":1133590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462981,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1093632,"data_stored":530764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1167360,"data_stored":601862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1118208,"data_stored":532669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1187840,"data_stored":603454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1054897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920848,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":535370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724620,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32984,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712168,"statfs":{"total":95995035648,"available":95961260032,"internally_reserved":0,"allocated":1851392,"data_stored":1255362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":331975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":400959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:39:05.981 INFO:tasks.quiescer.ceph_manager:clean! 2024-04-21T08:39:05.982 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T08:39:06.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.123+0000 7fa98ab04640 1 -- 172.21.15.43:0/856007800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa984101b60 msgr2=0x7fa98410dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:06.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.123+0000 7fa98ab04640 1 --2- 172.21.15.43:0/856007800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa984101b60 0x7fa98410dc90 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009cd0 tx=0x7fa97c02c0e0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.123+0000 7fa98ab04640 1 -- 172.21.15.43:0/856007800 shutdown_connections 2024-04-21T08:39:06.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.123+0000 7fa98ab04640 1 --2- 172.21.15.43:0/856007800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa984101b60 0x7fa98410dc90 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.123+0000 7fa98ab04640 1 --2- 172.21.15.43:0/856007800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9841011a0 0x7fa984101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.123+0000 7fa98ab04640 1 --2- 172.21.15.43:0/856007800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa984105e30 0x7fa984106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.123+0000 7fa98ab04640 1 -- 172.21.15.43:0/856007800 >> 172.21.15.43:0/856007800 conn(0x7fa984077e30 msgr2=0x7fa984078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:06.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.123+0000 7fa98ab04640 1 -- 172.21.15.43:0/856007800 shutdown_connections 2024-04-21T08:39:06.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.123+0000 7fa98ab04640 1 -- 172.21.15.43:0/856007800 wait complete. 2024-04-21T08:39:06.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.123+0000 7fa98ab04640 1 Processor -- start 2024-04-21T08:39:06.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.124+0000 7fa98ab04640 1 -- start start 2024-04-21T08:39:06.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.124+0000 7fa98ab04640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9841011a0 0x7fa984198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:06.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.124+0000 7fa98ab04640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa984101b60 0x7fa984198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:06.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.124+0000 7fa98ab04640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa984105e30 0x7fa98419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:06.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.124+0000 7fa98ab04640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa984111140 con 0x7fa9841011a0 2024-04-21T08:39:06.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.124+0000 7fa98ab04640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa984110fc0 con 0x7fa984105e30 2024-04-21T08:39:06.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.124+0000 7fa98ab04640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9841112c0 con 0x7fa984101b60 2024-04-21T08:39:06.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.124+0000 7fa9888a0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9841011a0 0x7fa984198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:06.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.124+0000 7fa9888a0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9841011a0 0x7fa984198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39112/0 (socket says 172.21.15.43:39112) 2024-04-21T08:39:06.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.124+0000 7fa9888a0640 1 -- 172.21.15.43:0/3798859863 learned_addr learned my addr 172.21.15.43:0/3798859863 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:06.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.124+0000 7fa983fff640 1 --2- 172.21.15.43:0/3798859863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa984101b60 0x7fa984198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:06.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.124+0000 7fa9890a1640 1 --2- 172.21.15.43:0/3798859863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa984105e30 0x7fa98419cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:06.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.124+0000 7fa9888a0640 1 -- 172.21.15.43:0/3798859863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa984101b60 msgr2=0x7fa984198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:06.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.124+0000 7fa9888a0640 1 --2- 172.21.15.43:0/3798859863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa984101b60 0x7fa984198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.124+0000 7fa9888a0640 1 -- 172.21.15.43:0/3798859863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa984105e30 msgr2=0x7fa98419cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:06.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.124+0000 7fa9888a0640 1 --2- 172.21.15.43:0/3798859863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa984105e30 0x7fa98419cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.124+0000 7fa9888a0640 1 -- 172.21.15.43:0/3798859863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98419d660 con 0x7fa9841011a0 2024-04-21T08:39:06.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.124+0000 7fa9890a1640 1 --2- 172.21.15.43:0/3798859863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa984105e30 0x7fa98419cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:06.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.124+0000 7fa9888a0640 1 --2- 172.21.15.43:0/3798859863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9841011a0 0x7fa984198670 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa96c00b4a0 tx=0x7fa96c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:06.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.125+0000 7fa981ffb640 1 -- 172.21.15.43:0/3798859863 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c015020 con 0x7fa9841011a0 2024-04-21T08:39:06.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.125+0000 7fa981ffb640 1 -- 172.21.15.43:0/3798859863 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa96c011ce0 con 0x7fa9841011a0 2024-04-21T08:39:06.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.125+0000 7fa98ab04640 1 -- 172.21.15.43:0/3798859863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa98419d950 con 0x7fa9841011a0 2024-04-21T08:39:06.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.125+0000 7fa981ffb640 1 -- 172.21.15.43:0/3798859863 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c010dc0 con 0x7fa9841011a0 2024-04-21T08:39:06.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.125+0000 7fa98ab04640 1 -- 172.21.15.43:0/3798859863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9841a51e0 con 0x7fa9841011a0 2024-04-21T08:39:06.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.126+0000 7fa98ab04640 1 -- 172.21.15.43:0/3798859863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa984102570 con 0x7fa9841011a0 2024-04-21T08:39:06.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.128+0000 7fa981ffb640 1 -- 172.21.15.43:0/3798859863 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fa96c018020 con 0x7fa9841011a0 2024-04-21T08:39:06.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.128+0000 7fa981ffb640 1 --2- 172.21.15.43:0/3798859863 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa9700a3db0 0x7fa9700a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:06.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.128+0000 7fa981ffb640 1 -- 172.21.15.43:0/3798859863 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fa96c0d05e0 con 0x7fa9841011a0 2024-04-21T08:39:06.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.128+0000 7fa983fff640 1 --2- 172.21.15.43:0/3798859863 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa9700a3db0 0x7fa9700a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:06.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.129+0000 7fa983fff640 1 --2- 172.21.15.43:0/3798859863 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa9700a3db0 0x7fa9700a6270 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa9840fe4c0 tx=0x7fa97400a2d0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:06.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.133+0000 7fa981ffb640 1 -- 172.21.15.43:0/3798859863 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa96c01f020 con 0x7fa9841011a0 2024-04-21T08:39:06.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.318+0000 7fa98ab04640 1 -- 172.21.15.43:0/3798859863 --> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa984062390 con 0x7fa9700a3db0 2024-04-21T08:39:06.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.328+0000 7fa981ffb640 1 -- 172.21.15.43:0/3798859863 <== mgr.15546 v2:172.21.15.43:6800/1482273906 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339443 (secure 0 0 0) 0x7fa984062390 con 0x7fa9700a3db0 2024-04-21T08:39:06.331 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:39:06.332 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-21T08:39:06.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.332+0000 7fa98ab04640 1 -- 172.21.15.43:0/3798859863 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa9700a3db0 msgr2=0x7fa9700a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:06.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.332+0000 7fa98ab04640 1 --2- 172.21.15.43:0/3798859863 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa9700a3db0 0x7fa9700a6270 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa9840fe4c0 tx=0x7fa97400a2d0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.332+0000 7fa98ab04640 1 -- 172.21.15.43:0/3798859863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9841011a0 msgr2=0x7fa984198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:06.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.332+0000 7fa98ab04640 1 --2- 172.21.15.43:0/3798859863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9841011a0 0x7fa984198670 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa96c00b4a0 tx=0x7fa96c00b8a0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.332+0000 7fa98ab04640 1 -- 172.21.15.43:0/3798859863 shutdown_connections 2024-04-21T08:39:06.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.332+0000 7fa98ab04640 1 --2- 172.21.15.43:0/3798859863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa984105e30 0x7fa98419cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.332+0000 7fa98ab04640 1 --2- 172.21.15.43:0/3798859863 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa9700a3db0 0x7fa9700a6270 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.332+0000 7fa98ab04640 1 --2- 172.21.15.43:0/3798859863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa984101b60 0x7fa984198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.332+0000 7fa98ab04640 1 --2- 172.21.15.43:0/3798859863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9841011a0 0x7fa984198670 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.332+0000 7fa98ab04640 1 -- 172.21.15.43:0/3798859863 >> 172.21.15.43:0/3798859863 conn(0x7fa984077e30 msgr2=0x7fa984069320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:06.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.332+0000 7fa98ab04640 1 -- 172.21.15.43:0/3798859863 shutdown_connections 2024-04-21T08:39:06.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.332+0000 7fa98ab04640 1 -- 172.21.15.43:0/3798859863 wait complete. 2024-04-21T08:39:06.342 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-21T08:39:04.903196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1391617,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1795,"num_read_kb":1277,"num_write":648,"num_write_kb":5070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1003,"ondisk_log_size":1003,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386504,"kb_used_data":16520,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124555320,"statfs":{"total":1151940427776,"available":1151544647680,"internally_reserved":0,"allocated":16916480,"data_stored":9972833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":378713404},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":267,"num_read_kb":342,"num_write":88,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.991351"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005622+0000","last_change":"2024-04-21T08:36:23.983921+0000","last_active":"2024-04-21T08:38:55.005622+0000","last_peered":"2024-04-21T08:38:55.005622+0000","last_clean":"2024-04-21T08:38:55.005622+0000","last_became_active":"2024-04-21T08:36:23.983638+0000","last_became_peered":"2024-04-21T08:36:23.983638+0000","last_unstale":"2024-04-21T08:38:55.005622+0000","last_undegraded":"2024-04-21T08:38:55.005622+0000","last_fullsized":"2024-04-21T08:38:55.005622+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:29:44.646308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004430+0000","last_change":"2024-04-21T08:36:23.990029+0000","last_active":"2024-04-21T08:38:55.004430+0000","last_peered":"2024-04-21T08:38:55.004430+0000","last_clean":"2024-04-21T08:38:55.004430+0000","last_became_active":"2024-04-21T08:36:23.989141+0000","last_became_peered":"2024-04-21T08:36:23.989141+0000","last_unstale":"2024-04-21T08:38:55.004430+0000","last_undegraded":"2024-04-21T08:38:55.004430+0000","last_fullsized":"2024-04-21T08:38:55.004430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:23:58.692342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004234+0000","last_change":"2024-04-21T08:36:23.982205+0000","last_active":"2024-04-21T08:38:55.004234+0000","last_peered":"2024-04-21T08:38:55.004234+0000","last_clean":"2024-04-21T08:38:55.004234+0000","last_became_active":"2024-04-21T08:36:23.981876+0000","last_became_peered":"2024-04-21T08:36:23.981876+0000","last_unstale":"2024-04-21T08:38:55.004234+0000","last_undegraded":"2024-04-21T08:38:55.004234+0000","last_fullsized":"2024-04-21T08:38:55.004234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:12:27.121106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006754+0000","last_change":"2024-04-21T08:36:23.987425+0000","last_active":"2024-04-21T08:38:55.006754+0000","last_peered":"2024-04-21T08:38:55.006754+0000","last_clean":"2024-04-21T08:38:55.006754+0000","last_became_active":"2024-04-21T08:36:23.987109+0000","last_became_peered":"2024-04-21T08:36:23.987109+0000","last_unstale":"2024-04-21T08:38:55.006754+0000","last_undegraded":"2024-04-21T08:38:55.006754+0000","last_fullsized":"2024-04-21T08:38:55.006754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:44.176324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890536+0000","last_change":"2024-04-21T08:36:23.990375+0000","last_active":"2024-04-21T08:38:54.890536+0000","last_peered":"2024-04-21T08:38:54.890536+0000","last_clean":"2024-04-21T08:38:54.890536+0000","last_became_active":"2024-04-21T08:36:23.990058+0000","last_became_peered":"2024-04-21T08:36:23.990058+0000","last_unstale":"2024-04-21T08:38:54.890536+0000","last_undegraded":"2024-04-21T08:38:54.890536+0000","last_fullsized":"2024-04-21T08:38:54.890536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:24:09.647528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888221+0000","last_change":"2024-04-21T08:36:23.979848+0000","last_active":"2024-04-21T08:38:54.888221+0000","last_peered":"2024-04-21T08:38:54.888221+0000","last_clean":"2024-04-21T08:38:54.888221+0000","last_became_active":"2024-04-21T08:36:23.979511+0000","last_became_peered":"2024-04-21T08:36:23.979511+0000","last_unstale":"2024-04-21T08:38:54.888221+0000","last_undegraded":"2024-04-21T08:38:54.888221+0000","last_fullsized":"2024-04-21T08:38:54.888221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:07:46.551369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006232+0000","last_change":"2024-04-21T08:37:15.118755+0000","last_active":"2024-04-21T08:38:55.006232+0000","last_peered":"2024-04-21T08:38:55.006232+0000","last_clean":"2024-04-21T08:38:55.006232+0000","last_became_active":"2024-04-21T08:37:15.118347+0000","last_became_peered":"2024-04-21T08:37:15.118347+0000","last_unstale":"2024-04-21T08:38:55.006232+0000","last_undegraded":"2024-04-21T08:38:55.006232+0000","last_fullsized":"2024-04-21T08:38:55.006232+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:36:43.398325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886870+0000","last_change":"2024-04-21T08:36:23.987517+0000","last_active":"2024-04-21T08:38:54.886870+0000","last_peered":"2024-04-21T08:38:54.886870+0000","last_clean":"2024-04-21T08:38:54.886870+0000","last_became_active":"2024-04-21T08:36:23.987201+0000","last_became_peered":"2024-04-21T08:36:23.987201+0000","last_unstale":"2024-04-21T08:38:54.886870+0000","last_undegraded":"2024-04-21T08:38:54.886870+0000","last_fullsized":"2024-04-21T08:38:54.886870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:42:53.080863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104900+0000","last_change":"2024-04-21T08:36:23.989434+0000","last_active":"2024-04-21T08:38:55.104900+0000","last_peered":"2024-04-21T08:38:55.104900+0000","last_clean":"2024-04-21T08:38:55.104900+0000","last_became_active":"2024-04-21T08:36:23.989125+0000","last_became_peered":"2024-04-21T08:36:23.989125+0000","last_unstale":"2024-04-21T08:38:55.104900+0000","last_undegraded":"2024-04-21T08:38:55.104900+0000","last_fullsized":"2024-04-21T08:38:55.104900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:30:07.219056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888449+0000","last_change":"2024-04-21T08:36:23.980725+0000","last_active":"2024-04-21T08:38:54.888449+0000","last_peered":"2024-04-21T08:38:54.888449+0000","last_clean":"2024-04-21T08:38:54.888449+0000","last_became_active":"2024-04-21T08:36:23.980363+0000","last_became_peered":"2024-04-21T08:36:23.980363+0000","last_unstale":"2024-04-21T08:38:54.888449+0000","last_undegraded":"2024-04-21T08:38:54.888449+0000","last_fullsized":"2024-04-21T08:38:54.888449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:26:46.202030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006978+0000","last_change":"2024-04-21T08:36:23.989369+0000","last_active":"2024-04-21T08:38:55.006978+0000","last_peered":"2024-04-21T08:38:55.006978+0000","last_clean":"2024-04-21T08:38:55.006978+0000","last_became_active":"2024-04-21T08:36:23.989030+0000","last_became_peered":"2024-04-21T08:36:23.989030+0000","last_unstale":"2024-04-21T08:38:55.006978+0000","last_undegraded":"2024-04-21T08:38:55.006978+0000","last_fullsized":"2024-04-21T08:38:55.006978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:23:13.725703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006691+0000","last_change":"2024-04-21T08:36:23.993485+0000","last_active":"2024-04-21T08:38:55.006691+0000","last_peered":"2024-04-21T08:38:55.006691+0000","last_clean":"2024-04-21T08:38:55.006691+0000","last_became_active":"2024-04-21T08:36:23.993124+0000","last_became_peered":"2024-04-21T08:36:23.993124+0000","last_unstale":"2024-04-21T08:38:55.006691+0000","last_undegraded":"2024-04-21T08:38:55.006691+0000","last_fullsized":"2024-04-21T08:38:55.006691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:53:16.249838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076569+0000","last_change":"2024-04-21T08:36:23.983146+0000","last_active":"2024-04-21T08:38:55.076569+0000","last_peered":"2024-04-21T08:38:55.076569+0000","last_clean":"2024-04-21T08:38:55.076569+0000","last_became_active":"2024-04-21T08:36:23.982766+0000","last_became_peered":"2024-04-21T08:36:23.982766+0000","last_unstale":"2024-04-21T08:38:55.076569+0000","last_undegraded":"2024-04-21T08:38:55.076569+0000","last_fullsized":"2024-04-21T08:38:55.076569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:30:44.090929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.003129+0000","last_change":"2024-04-21T08:36:23.988906+0000","last_active":"2024-04-21T08:38:55.003129+0000","last_peered":"2024-04-21T08:38:55.003129+0000","last_clean":"2024-04-21T08:38:55.003129+0000","last_became_active":"2024-04-21T08:36:23.988489+0000","last_became_peered":"2024-04-21T08:36:23.988489+0000","last_unstale":"2024-04-21T08:38:55.003129+0000","last_undegraded":"2024-04-21T08:38:55.003129+0000","last_fullsized":"2024-04-21T08:38:55.003129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:06:43.271650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.003060+0000","last_change":"2024-04-21T08:37:15.123343+0000","last_active":"2024-04-21T08:38:55.003060+0000","last_peered":"2024-04-21T08:38:55.003060+0000","last_clean":"2024-04-21T08:38:55.003060+0000","last_became_active":"2024-04-21T08:37:15.122959+0000","last_became_peered":"2024-04-21T08:37:15.122959+0000","last_unstale":"2024-04-21T08:38:55.003060+0000","last_undegraded":"2024-04-21T08:38:55.003060+0000","last_fullsized":"2024-04-21T08:38:55.003060+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:56:19.179879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104471+0000","last_change":"2024-04-21T08:36:23.983136+0000","last_active":"2024-04-21T08:38:55.104471+0000","last_peered":"2024-04-21T08:38:55.104471+0000","last_clean":"2024-04-21T08:38:55.104471+0000","last_became_active":"2024-04-21T08:36:23.982698+0000","last_became_peered":"2024-04-21T08:36:23.982698+0000","last_unstale":"2024-04-21T08:38:55.104471+0000","last_undegraded":"2024-04-21T08:38:55.104471+0000","last_fullsized":"2024-04-21T08:38:55.104471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:34.750941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005563+0000","last_change":"2024-04-21T08:36:23.984701+0000","last_active":"2024-04-21T08:38:55.005563+0000","last_peered":"2024-04-21T08:38:55.005563+0000","last_clean":"2024-04-21T08:38:55.005563+0000","last_became_active":"2024-04-21T08:36:23.984432+0000","last_became_peered":"2024-04-21T08:36:23.984432+0000","last_unstale":"2024-04-21T08:38:55.005563+0000","last_undegraded":"2024-04-21T08:38:55.005563+0000","last_fullsized":"2024-04-21T08:38:55.005563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:10:58.944380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.996823+0000","last_change":"2024-04-21T08:36:23.982781+0000","last_active":"2024-04-21T08:38:54.996823+0000","last_peered":"2024-04-21T08:38:54.996823+0000","last_clean":"2024-04-21T08:38:54.996823+0000","last_became_active":"2024-04-21T08:36:23.982459+0000","last_became_peered":"2024-04-21T08:36:23.982459+0000","last_unstale":"2024-04-21T08:38:54.996823+0000","last_undegraded":"2024-04-21T08:38:54.996823+0000","last_fullsized":"2024-04-21T08:38:54.996823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:44.657504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.105000+0000","last_change":"2024-04-21T08:36:23.992002+0000","last_active":"2024-04-21T08:38:55.105000+0000","last_peered":"2024-04-21T08:38:55.105000+0000","last_clean":"2024-04-21T08:38:55.105000+0000","last_became_active":"2024-04-21T08:36:23.991053+0000","last_became_peered":"2024-04-21T08:36:23.991053+0000","last_unstale":"2024-04-21T08:38:55.105000+0000","last_undegraded":"2024-04-21T08:38:55.105000+0000","last_fullsized":"2024-04-21T08:38:55.105000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:56:37.145949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104888+0000","last_change":"2024-04-21T08:36:28.013195+0000","last_active":"2024-04-21T08:38:55.104888+0000","last_peered":"2024-04-21T08:38:55.104888+0000","last_clean":"2024-04-21T08:38:55.104888+0000","last_became_active":"2024-04-21T08:36:28.012656+0000","last_became_peered":"2024-04-21T08:36:28.012656+0000","last_unstale":"2024-04-21T08:38:55.104888+0000","last_undegraded":"2024-04-21T08:38:55.104888+0000","last_fullsized":"2024-04-21T08:38:55.104888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:12:03.133583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"95'14","reported_seq":179,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:39:00.080989+0000","last_change":"2024-04-21T08:36:23.468957+0000","last_active":"2024-04-21T08:39:00.080989+0000","last_peered":"2024-04-21T08:39:00.080989+0000","last_clean":"2024-04-21T08:39:00.080989+0000","last_became_active":"2024-04-21T08:36:23.468601+0000","last_became_peered":"2024-04-21T08:36:23.468601+0000","last_unstale":"2024-04-21T08:39:00.080989+0000","last_undegraded":"2024-04-21T08:39:00.080989+0000","last_fullsized":"2024-04-21T08:39:00.080989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:13:11.977780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":131,"num_read_kb":131,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.996013+0000","last_change":"2024-04-21T08:37:15.123476+0000","last_active":"2024-04-21T08:38:54.996013+0000","last_peered":"2024-04-21T08:38:54.996013+0000","last_clean":"2024-04-21T08:38:54.996013+0000","last_became_active":"2024-04-21T08:37:15.123066+0000","last_became_peered":"2024-04-21T08:37:15.123066+0000","last_unstale":"2024-04-21T08:38:54.996013+0000","last_undegraded":"2024-04-21T08:38:54.996013+0000","last_fullsized":"2024-04-21T08:38:54.996013+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:15.786178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005900+0000","last_change":"2024-04-21T08:36:28.014161+0000","last_active":"2024-04-21T08:38:55.005900+0000","last_peered":"2024-04-21T08:38:55.005900+0000","last_clean":"2024-04-21T08:38:55.005900+0000","last_became_active":"2024-04-21T08:36:28.013859+0000","last_became_peered":"2024-04-21T08:36:28.013859+0000","last_unstale":"2024-04-21T08:38:55.005900+0000","last_undegraded":"2024-04-21T08:38:55.005900+0000","last_fullsized":"2024-04-21T08:38:55.005900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:14:08.315334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887665+0000","last_change":"2024-04-21T08:36:23.475612+0000","last_active":"2024-04-21T08:38:54.887665+0000","last_peered":"2024-04-21T08:38:54.887665+0000","last_clean":"2024-04-21T08:38:54.887665+0000","last_became_active":"2024-04-21T08:36:23.475301+0000","last_became_peered":"2024-04-21T08:36:23.475301+0000","last_unstale":"2024-04-21T08:38:54.887665+0000","last_undegraded":"2024-04-21T08:38:54.887665+0000","last_fullsized":"2024-04-21T08:38:54.887665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:56.801630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004414+0000","last_change":"2024-04-21T08:36:23.984689+0000","last_active":"2024-04-21T08:38:55.004414+0000","last_peered":"2024-04-21T08:38:55.004414+0000","last_clean":"2024-04-21T08:38:55.004414+0000","last_became_active":"2024-04-21T08:36:23.984358+0000","last_became_peered":"2024-04-21T08:36:23.984358+0000","last_unstale":"2024-04-21T08:38:55.004414+0000","last_undegraded":"2024-04-21T08:38:55.004414+0000","last_fullsized":"2024-04-21T08:38:55.004414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:49:50.187466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006257+0000","last_change":"2024-04-21T08:36:28.007951+0000","last_active":"2024-04-21T08:38:55.006257+0000","last_peered":"2024-04-21T08:38:55.006257+0000","last_clean":"2024-04-21T08:38:55.006257+0000","last_became_active":"2024-04-21T08:36:28.007553+0000","last_became_peered":"2024-04-21T08:36:28.007553+0000","last_unstale":"2024-04-21T08:38:55.006257+0000","last_undegraded":"2024-04-21T08:38:55.006257+0000","last_fullsized":"2024-04-21T08:38:55.006257+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:10:54.019605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890572+0000","last_change":"2024-04-21T08:36:23.473371+0000","last_active":"2024-04-21T08:38:54.890572+0000","last_peered":"2024-04-21T08:38:54.890572+0000","last_clean":"2024-04-21T08:38:54.890572+0000","last_became_active":"2024-04-21T08:36:23.472472+0000","last_became_peered":"2024-04-21T08:36:23.472472+0000","last_unstale":"2024-04-21T08:38:54.890572+0000","last_undegraded":"2024-04-21T08:38:54.890572+0000","last_fullsized":"2024-04-21T08:38:54.890572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:28:49.622873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104901+0000","last_change":"2024-04-21T08:36:23.987916+0000","last_active":"2024-04-21T08:38:55.104901+0000","last_peered":"2024-04-21T08:38:55.104901+0000","last_clean":"2024-04-21T08:38:55.104901+0000","last_became_active":"2024-04-21T08:36:23.986207+0000","last_became_peered":"2024-04-21T08:36:23.986207+0000","last_unstale":"2024-04-21T08:38:55.104901+0000","last_undegraded":"2024-04-21T08:38:55.104901+0000","last_fullsized":"2024-04-21T08:38:55.104901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:49:57.261078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006246+0000","last_change":"2024-04-21T08:36:28.006494+0000","last_active":"2024-04-21T08:38:55.006246+0000","last_peered":"2024-04-21T08:38:55.006246+0000","last_clean":"2024-04-21T08:38:55.006246+0000","last_became_active":"2024-04-21T08:36:28.006069+0000","last_became_peered":"2024-04-21T08:36:28.006069+0000","last_unstale":"2024-04-21T08:38:55.006246+0000","last_undegraded":"2024-04-21T08:38:55.006246+0000","last_fullsized":"2024-04-21T08:38:55.006246+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:04.608295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890975+0000","last_change":"2024-04-21T08:36:23.468923+0000","last_active":"2024-04-21T08:38:54.890975+0000","last_peered":"2024-04-21T08:38:54.890975+0000","last_clean":"2024-04-21T08:38:54.890975+0000","last_became_active":"2024-04-21T08:36:23.468509+0000","last_became_peered":"2024-04-21T08:36:23.468509+0000","last_unstale":"2024-04-21T08:38:54.890975+0000","last_undegraded":"2024-04-21T08:38:54.890975+0000","last_fullsized":"2024-04-21T08:38:54.890975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:44.452748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008259+0000","last_change":"2024-04-21T08:36:23.994423+0000","last_active":"2024-04-21T08:38:55.008259+0000","last_peered":"2024-04-21T08:38:55.008259+0000","last_clean":"2024-04-21T08:38:55.008259+0000","last_became_active":"2024-04-21T08:36:23.993827+0000","last_became_peered":"2024-04-21T08:36:23.993827+0000","last_unstale":"2024-04-21T08:38:55.008259+0000","last_undegraded":"2024-04-21T08:38:55.008259+0000","last_fullsized":"2024-04-21T08:38:55.008259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:49:25.550301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104894+0000","last_change":"2024-04-21T08:36:28.004037+0000","last_active":"2024-04-21T08:38:55.104894+0000","last_peered":"2024-04-21T08:38:55.104894+0000","last_clean":"2024-04-21T08:38:55.104894+0000","last_became_active":"2024-04-21T08:36:28.003580+0000","last_became_peered":"2024-04-21T08:36:28.003580+0000","last_unstale":"2024-04-21T08:38:55.104894+0000","last_undegraded":"2024-04-21T08:38:55.104894+0000","last_fullsized":"2024-04-21T08:38:55.104894+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:04:51.380244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'8","reported_seq":178,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:59.079811+0000","last_change":"2024-04-21T08:36:23.468430+0000","last_active":"2024-04-21T08:38:59.079811+0000","last_peered":"2024-04-21T08:38:59.079811+0000","last_clean":"2024-04-21T08:38:59.079811+0000","last_became_active":"2024-04-21T08:36:23.468089+0000","last_became_peered":"2024-04-21T08:36:23.468089+0000","last_unstale":"2024-04-21T08:38:59.079811+0000","last_undegraded":"2024-04-21T08:38:59.079811+0000","last_fullsized":"2024-04-21T08:38:59.079811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:12:54.827034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890458+0000","last_change":"2024-04-21T08:37:15.117922+0000","last_active":"2024-04-21T08:38:54.890458+0000","last_peered":"2024-04-21T08:38:54.890458+0000","last_clean":"2024-04-21T08:38:54.890458+0000","last_became_active":"2024-04-21T08:37:15.117404+0000","last_became_peered":"2024-04-21T08:37:15.117404+0000","last_unstale":"2024-04-21T08:38:54.890458+0000","last_undegraded":"2024-04-21T08:38:54.890458+0000","last_fullsized":"2024-04-21T08:38:54.890458+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:32:39.682224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104742+0000","last_change":"2024-04-21T08:36:28.004638+0000","last_active":"2024-04-21T08:38:55.104742+0000","last_peered":"2024-04-21T08:38:55.104742+0000","last_clean":"2024-04-21T08:38:55.104742+0000","last_became_active":"2024-04-21T08:36:28.002499+0000","last_became_peered":"2024-04-21T08:36:28.002499+0000","last_unstale":"2024-04-21T08:38:55.104742+0000","last_undegraded":"2024-04-21T08:38:55.104742+0000","last_fullsized":"2024-04-21T08:38:55.104742+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:45:46.934783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":55,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888089+0000","last_change":"2024-04-21T08:37:15.126569+0000","last_active":"2024-04-21T08:38:54.888089+0000","last_peered":"2024-04-21T08:38:54.888089+0000","last_clean":"2024-04-21T08:38:54.888089+0000","last_became_active":"2024-04-21T08:37:15.121284+0000","last_became_peered":"2024-04-21T08:37:15.121284+0000","last_unstale":"2024-04-21T08:38:54.888089+0000","last_undegraded":"2024-04-21T08:38:54.888089+0000","last_fullsized":"2024-04-21T08:38:54.888089+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:02:26.153324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008374+0000","last_change":"2024-04-21T08:37:15.557192+0000","last_active":"2024-04-21T08:38:55.008374+0000","last_peered":"2024-04-21T08:38:55.008374+0000","last_clean":"2024-04-21T08:38:55.008374+0000","last_became_active":"2024-04-21T08:37:15.556653+0000","last_became_peered":"2024-04-21T08:37:15.556653+0000","last_unstale":"2024-04-21T08:38:55.008374+0000","last_undegraded":"2024-04-21T08:38:55.008374+0000","last_fullsized":"2024-04-21T08:38:55.008374+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:42.722460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005527+0000","last_change":"2024-04-21T08:36:28.017068+0000","last_active":"2024-04-21T08:38:55.005527+0000","last_peered":"2024-04-21T08:38:55.005527+0000","last_clean":"2024-04-21T08:38:55.005527+0000","last_became_active":"2024-04-21T08:36:28.016770+0000","last_became_peered":"2024-04-21T08:36:28.016770+0000","last_unstale":"2024-04-21T08:38:55.005527+0000","last_undegraded":"2024-04-21T08:38:55.005527+0000","last_fullsized":"2024-04-21T08:38:55.005527+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:07:14.596090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076311+0000","last_change":"2024-04-21T08:36:23.470430+0000","last_active":"2024-04-21T08:38:55.076311+0000","last_peered":"2024-04-21T08:38:55.076311+0000","last_clean":"2024-04-21T08:38:55.076311+0000","last_became_active":"2024-04-21T08:36:23.470109+0000","last_became_peered":"2024-04-21T08:36:23.470109+0000","last_unstale":"2024-04-21T08:38:55.076311+0000","last_undegraded":"2024-04-21T08:38:55.076311+0000","last_fullsized":"2024-04-21T08:38:55.076311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:18.176313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887026+0000","last_change":"2024-04-21T08:36:23.986668+0000","last_active":"2024-04-21T08:38:54.887026+0000","last_peered":"2024-04-21T08:38:54.887026+0000","last_clean":"2024-04-21T08:38:54.887026+0000","last_became_active":"2024-04-21T08:36:23.986344+0000","last_became_peered":"2024-04-21T08:36:23.986344+0000","last_unstale":"2024-04-21T08:38:54.887026+0000","last_undegraded":"2024-04-21T08:38:54.887026+0000","last_fullsized":"2024-04-21T08:38:54.887026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:56:17.744145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004263+0000","last_change":"2024-04-21T08:36:28.009012+0000","last_active":"2024-04-21T08:38:55.004263+0000","last_peered":"2024-04-21T08:38:55.004263+0000","last_clean":"2024-04-21T08:38:55.004263+0000","last_became_active":"2024-04-21T08:36:28.008123+0000","last_became_peered":"2024-04-21T08:36:28.008123+0000","last_unstale":"2024-04-21T08:38:55.004263+0000","last_undegraded":"2024-04-21T08:38:55.004263+0000","last_fullsized":"2024-04-21T08:38:55.004263+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:33:18.881564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007854+0000","last_change":"2024-04-21T08:36:23.467104+0000","last_active":"2024-04-21T08:38:55.007854+0000","last_peered":"2024-04-21T08:38:55.007854+0000","last_clean":"2024-04-21T08:38:55.007854+0000","last_became_active":"2024-04-21T08:36:23.466725+0000","last_became_peered":"2024-04-21T08:36:23.466725+0000","last_unstale":"2024-04-21T08:38:55.007854+0000","last_undegraded":"2024-04-21T08:38:55.007854+0000","last_fullsized":"2024-04-21T08:38:55.007854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:35:45.986448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.995844+0000","last_change":"2024-04-21T08:36:23.987579+0000","last_active":"2024-04-21T08:38:54.995844+0000","last_peered":"2024-04-21T08:38:54.995844+0000","last_clean":"2024-04-21T08:38:54.995844+0000","last_became_active":"2024-04-21T08:36:23.987044+0000","last_became_peered":"2024-04-21T08:36:23.987044+0000","last_unstale":"2024-04-21T08:38:54.995844+0000","last_undegraded":"2024-04-21T08:38:54.995844+0000","last_fullsized":"2024-04-21T08:38:54.995844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:48:27.593090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886895+0000","last_change":"2024-04-21T08:36:28.006417+0000","last_active":"2024-04-21T08:38:54.886895+0000","last_peered":"2024-04-21T08:38:54.886895+0000","last_clean":"2024-04-21T08:38:54.886895+0000","last_became_active":"2024-04-21T08:36:28.006006+0000","last_became_peered":"2024-04-21T08:36:28.006006+0000","last_unstale":"2024-04-21T08:38:54.886895+0000","last_undegraded":"2024-04-21T08:38:54.886895+0000","last_fullsized":"2024-04-21T08:38:54.886895+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:16.173284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'8","reported_seq":302,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:39:00.080410+0000","last_change":"2024-04-21T08:36:23.471541+0000","last_active":"2024-04-21T08:39:00.080410+0000","last_peered":"2024-04-21T08:39:00.080410+0000","last_clean":"2024-04-21T08:39:00.080410+0000","last_became_active":"2024-04-21T08:36:23.471193+0000","last_became_peered":"2024-04-21T08:36:23.471193+0000","last_unstale":"2024-04-21T08:39:00.080410+0000","last_undegraded":"2024-04-21T08:39:00.080410+0000","last_fullsized":"2024-04-21T08:39:00.080410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:06:04.916707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":260,"num_read_kb":260,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104848+0000","last_change":"2024-04-21T08:36:23.986781+0000","last_active":"2024-04-21T08:38:55.104848+0000","last_peered":"2024-04-21T08:38:55.104848+0000","last_clean":"2024-04-21T08:38:55.104848+0000","last_became_active":"2024-04-21T08:36:23.985696+0000","last_became_peered":"2024-04-21T08:36:23.985696+0000","last_unstale":"2024-04-21T08:38:55.104848+0000","last_undegraded":"2024-04-21T08:38:55.104848+0000","last_fullsized":"2024-04-21T08:38:55.104848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:58:40.556642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.995747+0000","last_change":"2024-04-21T08:36:28.011621+0000","last_active":"2024-04-21T08:38:54.995747+0000","last_peered":"2024-04-21T08:38:54.995747+0000","last_clean":"2024-04-21T08:38:54.995747+0000","last_became_active":"2024-04-21T08:36:28.011136+0000","last_became_peered":"2024-04-21T08:36:28.011136+0000","last_unstale":"2024-04-21T08:38:54.995747+0000","last_undegraded":"2024-04-21T08:38:54.995747+0000","last_fullsized":"2024-04-21T08:38:54.995747+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:47.623445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006842+0000","last_change":"2024-04-21T08:36:23.474019+0000","last_active":"2024-04-21T08:38:55.006842+0000","last_peered":"2024-04-21T08:38:55.006842+0000","last_clean":"2024-04-21T08:38:55.006842+0000","last_became_active":"2024-04-21T08:36:23.473690+0000","last_became_peered":"2024-04-21T08:36:23.473690+0000","last_unstale":"2024-04-21T08:38:55.006842+0000","last_undegraded":"2024-04-21T08:38:55.006842+0000","last_fullsized":"2024-04-21T08:38:55.006842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:38:32.913795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004613+0000","last_change":"2024-04-21T08:37:15.121455+0000","last_active":"2024-04-21T08:38:55.004613+0000","last_peered":"2024-04-21T08:38:55.004613+0000","last_clean":"2024-04-21T08:38:55.004613+0000","last_became_active":"2024-04-21T08:37:15.121113+0000","last_became_peered":"2024-04-21T08:37:15.121113+0000","last_unstale":"2024-04-21T08:38:55.004613+0000","last_undegraded":"2024-04-21T08:38:55.004613+0000","last_fullsized":"2024-04-21T08:38:55.004613+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:39:36.525495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006512+0000","last_change":"2024-04-21T08:38:15.413849+0000","last_active":"2024-04-21T08:38:55.006512+0000","last_peered":"2024-04-21T08:38:55.006512+0000","last_clean":"2024-04-21T08:38:55.006512+0000","last_became_active":"2024-04-21T08:38:15.413361+0000","last_became_peered":"2024-04-21T08:38:15.413361+0000","last_unstale":"2024-04-21T08:38:55.006512+0000","last_undegraded":"2024-04-21T08:38:55.006512+0000","last_fullsized":"2024-04-21T08:38:55.006512+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:16:49.197800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008173+0000","last_change":"2024-04-21T08:36:23.465825+0000","last_active":"2024-04-21T08:38:55.008173+0000","last_peered":"2024-04-21T08:38:55.008173+0000","last_clean":"2024-04-21T08:38:55.008173+0000","last_became_active":"2024-04-21T08:36:23.465525+0000","last_became_peered":"2024-04-21T08:36:23.465525+0000","last_unstale":"2024-04-21T08:38:55.008173+0000","last_undegraded":"2024-04-21T08:38:55.008173+0000","last_fullsized":"2024-04-21T08:38:55.008173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:59:50.502210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.105005+0000","last_change":"2024-04-21T08:36:23.987602+0000","last_active":"2024-04-21T08:38:55.105005+0000","last_peered":"2024-04-21T08:38:55.105005+0000","last_clean":"2024-04-21T08:38:55.105005+0000","last_became_active":"2024-04-21T08:36:23.987298+0000","last_became_peered":"2024-04-21T08:36:23.987298+0000","last_unstale":"2024-04-21T08:38:55.105005+0000","last_undegraded":"2024-04-21T08:38:55.105005+0000","last_fullsized":"2024-04-21T08:38:55.105005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:06.647676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.995946+0000","last_change":"2024-04-21T08:36:28.005935+0000","last_active":"2024-04-21T08:38:54.995946+0000","last_peered":"2024-04-21T08:38:54.995946+0000","last_clean":"2024-04-21T08:38:54.995946+0000","last_became_active":"2024-04-21T08:36:28.005512+0000","last_became_peered":"2024-04-21T08:36:28.005512+0000","last_unstale":"2024-04-21T08:38:54.995946+0000","last_undegraded":"2024-04-21T08:38:54.995946+0000","last_fullsized":"2024-04-21T08:38:54.995946+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:23:20.298524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"95'43","reported_seq":353,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:39:00.080992+0000","last_change":"2024-04-21T08:36:23.467728+0000","last_active":"2024-04-21T08:39:00.080992+0000","last_peered":"2024-04-21T08:39:00.080992+0000","last_clean":"2024-04-21T08:39:00.080992+0000","last_became_active":"2024-04-21T08:36:23.467430+0000","last_became_peered":"2024-04-21T08:36:23.467430+0000","last_unstale":"2024-04-21T08:39:00.080992+0000","last_undegraded":"2024-04-21T08:39:00.080992+0000","last_fullsized":"2024-04-21T08:39:00.080992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:47:18.900911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70068,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":276,"num_read_kb":208,"num_write":45,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890735+0000","last_change":"2024-04-21T08:36:23.989116+0000","last_active":"2024-04-21T08:38:54.890735+0000","last_peered":"2024-04-21T08:38:54.890735+0000","last_clean":"2024-04-21T08:38:54.890735+0000","last_became_active":"2024-04-21T08:36:23.988758+0000","last_became_peered":"2024-04-21T08:36:23.988758+0000","last_unstale":"2024-04-21T08:38:54.890735+0000","last_undegraded":"2024-04-21T08:38:54.890735+0000","last_fullsized":"2024-04-21T08:38:54.890735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:56:25.969905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008450+0000","last_change":"2024-04-21T08:36:28.007370+0000","last_active":"2024-04-21T08:38:55.008450+0000","last_peered":"2024-04-21T08:38:55.008450+0000","last_clean":"2024-04-21T08:38:55.008450+0000","last_became_active":"2024-04-21T08:36:28.006972+0000","last_became_peered":"2024-04-21T08:36:28.006972+0000","last_unstale":"2024-04-21T08:38:55.008450+0000","last_undegraded":"2024-04-21T08:38:55.008450+0000","last_fullsized":"2024-04-21T08:38:55.008450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:18:14.926058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104708+0000","last_change":"2024-04-21T08:36:23.472658+0000","last_active":"2024-04-21T08:38:55.104708+0000","last_peered":"2024-04-21T08:38:55.104708+0000","last_clean":"2024-04-21T08:38:55.104708+0000","last_became_active":"2024-04-21T08:36:23.472318+0000","last_became_peered":"2024-04-21T08:36:23.472318+0000","last_unstale":"2024-04-21T08:38:55.104708+0000","last_undegraded":"2024-04-21T08:38:55.104708+0000","last_fullsized":"2024-04-21T08:38:55.104708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:28:27.497605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076501+0000","last_change":"2024-04-21T08:38:15.338892+0000","last_active":"2024-04-21T08:38:55.076501+0000","last_peered":"2024-04-21T08:38:55.076501+0000","last_clean":"2024-04-21T08:38:55.076501+0000","last_became_active":"2024-04-21T08:38:15.338396+0000","last_became_peered":"2024-04-21T08:38:15.338396+0000","last_unstale":"2024-04-21T08:38:55.076501+0000","last_undegraded":"2024-04-21T08:38:55.076501+0000","last_fullsized":"2024-04-21T08:38:55.076501+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:19.670635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005342+0000","last_change":"2024-04-21T08:36:28.019186+0000","last_active":"2024-04-21T08:38:55.005342+0000","last_peered":"2024-04-21T08:38:55.005342+0000","last_clean":"2024-04-21T08:38:55.005342+0000","last_became_active":"2024-04-21T08:36:28.018774+0000","last_became_peered":"2024-04-21T08:36:28.018774+0000","last_unstale":"2024-04-21T08:38:55.005342+0000","last_undegraded":"2024-04-21T08:38:55.005342+0000","last_fullsized":"2024-04-21T08:38:55.005342+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:35:33.548656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008059+0000","last_change":"2024-04-21T08:36:23.472633+0000","last_active":"2024-04-21T08:38:55.008059+0000","last_peered":"2024-04-21T08:38:55.008059+0000","last_clean":"2024-04-21T08:38:55.008059+0000","last_became_active":"2024-04-21T08:36:23.472336+0000","last_became_peered":"2024-04-21T08:36:23.472336+0000","last_unstale":"2024-04-21T08:38:55.008059+0000","last_undegraded":"2024-04-21T08:38:55.008059+0000","last_fullsized":"2024-04-21T08:38:55.008059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:49.463492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006193+0000","last_change":"2024-04-21T08:36:23.994332+0000","last_active":"2024-04-21T08:38:55.006193+0000","last_peered":"2024-04-21T08:38:55.006193+0000","last_clean":"2024-04-21T08:38:55.006193+0000","last_became_active":"2024-04-21T08:36:23.993936+0000","last_became_peered":"2024-04-21T08:36:23.993936+0000","last_unstale":"2024-04-21T08:38:55.006193+0000","last_undegraded":"2024-04-21T08:38:55.006193+0000","last_fullsized":"2024-04-21T08:38:55.006193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:26:35.330559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.996887+0000","last_change":"2024-04-21T08:36:28.012549+0000","last_active":"2024-04-21T08:38:54.996887+0000","last_peered":"2024-04-21T08:38:54.996887+0000","last_clean":"2024-04-21T08:38:54.996887+0000","last_became_active":"2024-04-21T08:36:28.012151+0000","last_became_peered":"2024-04-21T08:36:28.012151+0000","last_unstale":"2024-04-21T08:38:54.996887+0000","last_undegraded":"2024-04-21T08:38:54.996887+0000","last_fullsized":"2024-04-21T08:38:54.996887+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:55.680399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'8","reported_seq":178,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:59.081251+0000","last_change":"2024-04-21T08:36:23.464586+0000","last_active":"2024-04-21T08:38:59.081251+0000","last_peered":"2024-04-21T08:38:59.081251+0000","last_clean":"2024-04-21T08:38:59.081251+0000","last_became_active":"2024-04-21T08:36:23.464281+0000","last_became_peered":"2024-04-21T08:36:23.464281+0000","last_unstale":"2024-04-21T08:38:59.081251+0000","last_undegraded":"2024-04-21T08:38:59.081251+0000","last_fullsized":"2024-04-21T08:38:59.081251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:04:58.577283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":137,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006132+0000","last_change":"2024-04-21T08:36:23.989604+0000","last_active":"2024-04-21T08:38:55.006132+0000","last_peered":"2024-04-21T08:38:55.006132+0000","last_clean":"2024-04-21T08:38:55.006132+0000","last_became_active":"2024-04-21T08:36:23.989257+0000","last_became_peered":"2024-04-21T08:36:23.989257+0000","last_unstale":"2024-04-21T08:38:55.006132+0000","last_undegraded":"2024-04-21T08:38:55.006132+0000","last_fullsized":"2024-04-21T08:38:55.006132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:45.077943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077854+0000","last_change":"2024-04-21T08:36:28.007959+0000","last_active":"2024-04-21T08:38:55.077854+0000","last_peered":"2024-04-21T08:38:55.077854+0000","last_clean":"2024-04-21T08:38:55.077854+0000","last_became_active":"2024-04-21T08:36:28.007571+0000","last_became_peered":"2024-04-21T08:36:28.007571+0000","last_unstale":"2024-04-21T08:38:55.077854+0000","last_undegraded":"2024-04-21T08:38:55.077854+0000","last_fullsized":"2024-04-21T08:38:55.077854+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:17:34.007013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890204+0000","last_change":"2024-04-21T08:36:23.475054+0000","last_active":"2024-04-21T08:38:54.890204+0000","last_peered":"2024-04-21T08:38:54.890204+0000","last_clean":"2024-04-21T08:38:54.890204+0000","last_became_active":"2024-04-21T08:36:23.474606+0000","last_became_peered":"2024-04-21T08:36:23.474606+0000","last_unstale":"2024-04-21T08:38:54.890204+0000","last_undegraded":"2024-04-21T08:38:54.890204+0000","last_fullsized":"2024-04-21T08:38:54.890204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:22.559174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007909+0000","last_change":"2024-04-21T08:36:23.995980+0000","last_active":"2024-04-21T08:38:55.007909+0000","last_peered":"2024-04-21T08:38:55.007909+0000","last_clean":"2024-04-21T08:38:55.007909+0000","last_became_active":"2024-04-21T08:36:23.995673+0000","last_became_peered":"2024-04-21T08:36:23.995673+0000","last_unstale":"2024-04-21T08:38:55.007909+0000","last_undegraded":"2024-04-21T08:38:55.007909+0000","last_fullsized":"2024-04-21T08:38:55.007909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:11:49.884384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007831+0000","last_change":"2024-04-21T08:36:28.018792+0000","last_active":"2024-04-21T08:38:55.007831+0000","last_peered":"2024-04-21T08:38:55.007831+0000","last_clean":"2024-04-21T08:38:55.007831+0000","last_became_active":"2024-04-21T08:36:28.018373+0000","last_became_peered":"2024-04-21T08:36:28.018373+0000","last_unstale":"2024-04-21T08:38:55.007831+0000","last_undegraded":"2024-04-21T08:38:55.007831+0000","last_fullsized":"2024-04-21T08:38:55.007831+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:11.857661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"95'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890811+0000","last_change":"2024-04-21T08:36:23.469787+0000","last_active":"2024-04-21T08:38:54.890811+0000","last_peered":"2024-04-21T08:38:54.890811+0000","last_clean":"2024-04-21T08:38:54.890811+0000","last_became_active":"2024-04-21T08:36:23.469462+0000","last_became_peered":"2024-04-21T08:36:23.469462+0000","last_unstale":"2024-04-21T08:38:54.890811+0000","last_undegraded":"2024-04-21T08:38:54.890811+0000","last_fullsized":"2024-04-21T08:38:54.890811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:33:25.849528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005561+0000","last_change":"2024-04-21T08:36:23.988492+0000","last_active":"2024-04-21T08:38:55.005561+0000","last_peered":"2024-04-21T08:38:55.005561+0000","last_clean":"2024-04-21T08:38:55.005561+0000","last_became_active":"2024-04-21T08:36:23.988133+0000","last_became_peered":"2024-04-21T08:36:23.988133+0000","last_unstale":"2024-04-21T08:38:55.005561+0000","last_undegraded":"2024-04-21T08:38:55.005561+0000","last_fullsized":"2024-04-21T08:38:55.005561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:01:25.320281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890332+0000","last_change":"2024-04-21T08:38:15.415043+0000","last_active":"2024-04-21T08:38:54.890332+0000","last_peered":"2024-04-21T08:38:54.890332+0000","last_clean":"2024-04-21T08:38:54.890332+0000","last_became_active":"2024-04-21T08:38:15.414460+0000","last_became_peered":"2024-04-21T08:38:15.414460+0000","last_unstale":"2024-04-21T08:38:54.890332+0000","last_undegraded":"2024-04-21T08:38:54.890332+0000","last_fullsized":"2024-04-21T08:38:54.890332+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:50:17.202423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.996393+0000","last_change":"2024-04-21T08:37:15.219428+0000","last_active":"2024-04-21T08:38:54.996393+0000","last_peered":"2024-04-21T08:38:54.996393+0000","last_clean":"2024-04-21T08:38:54.996393+0000","last_became_active":"2024-04-21T08:37:15.218923+0000","last_became_peered":"2024-04-21T08:37:15.218923+0000","last_unstale":"2024-04-21T08:38:54.996393+0000","last_undegraded":"2024-04-21T08:38:54.996393+0000","last_fullsized":"2024-04-21T08:38:54.996393+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:07:02.781701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888120+0000","last_change":"2024-04-21T08:36:23.985910+0000","last_active":"2024-04-21T08:38:54.888120+0000","last_peered":"2024-04-21T08:38:54.888120+0000","last_clean":"2024-04-21T08:38:54.888120+0000","last_became_active":"2024-04-21T08:36:23.985605+0000","last_became_peered":"2024-04-21T08:36:23.985605+0000","last_unstale":"2024-04-21T08:38:54.888120+0000","last_undegraded":"2024-04-21T08:38:54.888120+0000","last_fullsized":"2024-04-21T08:38:54.888120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:37:09.367005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077277+0000","last_change":"2024-04-21T08:36:28.015256+0000","last_active":"2024-04-21T08:38:55.077277+0000","last_peered":"2024-04-21T08:38:55.077277+0000","last_clean":"2024-04-21T08:38:55.077277+0000","last_became_active":"2024-04-21T08:36:28.014849+0000","last_became_peered":"2024-04-21T08:36:28.014849+0000","last_unstale":"2024-04-21T08:38:55.077277+0000","last_undegraded":"2024-04-21T08:38:55.077277+0000","last_fullsized":"2024-04-21T08:38:55.077277+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:22:05.466299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"95'6","reported_seq":171,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:59.080426+0000","last_change":"2024-04-21T08:36:23.474966+0000","last_active":"2024-04-21T08:38:59.080426+0000","last_peered":"2024-04-21T08:38:59.080426+0000","last_clean":"2024-04-21T08:38:59.080426+0000","last_became_active":"2024-04-21T08:36:23.474374+0000","last_became_peered":"2024-04-21T08:36:23.474374+0000","last_unstale":"2024-04-21T08:38:59.080426+0000","last_undegraded":"2024-04-21T08:38:59.080426+0000","last_fullsized":"2024-04-21T08:38:59.080426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:57:20.237966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.888138+0000","last_change":"2024-04-21T08:37:15.118887+0000","last_active":"2024-04-21T08:38:54.888138+0000","last_peered":"2024-04-21T08:38:54.888138+0000","last_clean":"2024-04-21T08:38:54.888138+0000","last_became_active":"2024-04-21T08:37:15.118510+0000","last_became_peered":"2024-04-21T08:37:15.118510+0000","last_unstale":"2024-04-21T08:38:54.888138+0000","last_undegraded":"2024-04-21T08:38:54.888138+0000","last_fullsized":"2024-04-21T08:38:54.888138+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:59:27.209083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077841+0000","last_change":"2024-04-21T08:36:28.013169+0000","last_active":"2024-04-21T08:38:55.077841+0000","last_peered":"2024-04-21T08:38:55.077841+0000","last_clean":"2024-04-21T08:38:55.077841+0000","last_became_active":"2024-04-21T08:36:28.012791+0000","last_became_peered":"2024-04-21T08:36:28.012791+0000","last_unstale":"2024-04-21T08:38:55.077841+0000","last_undegraded":"2024-04-21T08:38:55.077841+0000","last_fullsized":"2024-04-21T08:38:55.077841+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:42:56.063159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007983+0000","last_change":"2024-04-21T08:36:23.995209+0000","last_active":"2024-04-21T08:38:55.007983+0000","last_peered":"2024-04-21T08:38:55.007983+0000","last_clean":"2024-04-21T08:38:55.007983+0000","last_became_active":"2024-04-21T08:36:23.994866+0000","last_became_peered":"2024-04-21T08:36:23.994866+0000","last_unstale":"2024-04-21T08:38:55.007983+0000","last_undegraded":"2024-04-21T08:38:55.007983+0000","last_fullsized":"2024-04-21T08:38:55.007983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:33:45.582150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'7","reported_seq":173,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:59.080838+0000","last_change":"2024-04-21T08:36:23.471474+0000","last_active":"2024-04-21T08:38:59.080838+0000","last_peered":"2024-04-21T08:38:59.080838+0000","last_clean":"2024-04-21T08:38:59.080838+0000","last_became_active":"2024-04-21T08:36:23.471179+0000","last_became_peered":"2024-04-21T08:36:23.471179+0000","last_unstale":"2024-04-21T08:38:59.080838+0000","last_undegraded":"2024-04-21T08:38:59.080838+0000","last_fullsized":"2024-04-21T08:38:59.080838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:17.290051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":132,"num_read_kb":132,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006553+0000","last_change":"2024-04-21T08:38:15.344332+0000","last_active":"2024-04-21T08:38:55.006553+0000","last_peered":"2024-04-21T08:38:55.006553+0000","last_clean":"2024-04-21T08:38:55.006553+0000","last_became_active":"2024-04-21T08:38:15.339914+0000","last_became_peered":"2024-04-21T08:38:15.339914+0000","last_unstale":"2024-04-21T08:38:55.006553+0000","last_undegraded":"2024-04-21T08:38:55.006553+0000","last_fullsized":"2024-04-21T08:38:55.006553+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:39:24.049323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886832+0000","last_change":"2024-04-21T08:36:23.990272+0000","last_active":"2024-04-21T08:38:54.886832+0000","last_peered":"2024-04-21T08:38:54.886832+0000","last_clean":"2024-04-21T08:38:54.886832+0000","last_became_active":"2024-04-21T08:36:23.989940+0000","last_became_peered":"2024-04-21T08:36:23.989940+0000","last_unstale":"2024-04-21T08:38:54.886832+0000","last_undegraded":"2024-04-21T08:38:54.886832+0000","last_fullsized":"2024-04-21T08:38:54.886832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:27:50.705167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006322+0000","last_change":"2024-04-21T08:36:23.474162+0000","last_active":"2024-04-21T08:38:55.006322+0000","last_peered":"2024-04-21T08:38:55.006322+0000","last_clean":"2024-04-21T08:38:55.006322+0000","last_became_active":"2024-04-21T08:36:23.473802+0000","last_became_peered":"2024-04-21T08:36:23.473802+0000","last_unstale":"2024-04-21T08:38:55.006322+0000","last_undegraded":"2024-04-21T08:38:55.006322+0000","last_fullsized":"2024-04-21T08:38:55.006322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:32:41.918431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"95'148","reported_seq":410,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:57.010247+0000","last_change":"2024-04-21T08:35:10.062130+0000","last_active":"2024-04-21T08:38:57.010247+0000","last_peered":"2024-04-21T08:38:57.010247+0000","last_clean":"2024-04-21T08:38:57.010247+0000","last_became_active":"2024-04-21T08:35:10.061727+0000","last_became_peered":"2024-04-21T08:35:10.061727+0000","last_unstale":"2024-04-21T08:38:57.010247+0000","last_undegraded":"2024-04-21T08:38:57.010247+0000","last_fullsized":"2024-04-21T08:38:57.010247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:31:40.950939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:31:40.950939+0000","last_clean_scrub_stamp":"2024-04-21T08:31:40.950939+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:35:52.360167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.008148+0000","last_change":"2024-04-21T08:38:15.415007+0000","last_active":"2024-04-21T08:38:55.008148+0000","last_peered":"2024-04-21T08:38:55.008148+0000","last_clean":"2024-04-21T08:38:55.008148+0000","last_became_active":"2024-04-21T08:38:15.414478+0000","last_became_peered":"2024-04-21T08:38:15.414478+0000","last_unstale":"2024-04-21T08:38:55.008148+0000","last_undegraded":"2024-04-21T08:38:55.008148+0000","last_fullsized":"2024-04-21T08:38:55.008148+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:27.703964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.075651+0000","last_change":"2024-04-21T08:36:23.984332+0000","last_active":"2024-04-21T08:38:55.075651+0000","last_peered":"2024-04-21T08:38:55.075651+0000","last_clean":"2024-04-21T08:38:55.075651+0000","last_became_active":"2024-04-21T08:36:23.984021+0000","last_became_peered":"2024-04-21T08:36:23.984021+0000","last_unstale":"2024-04-21T08:38:55.075651+0000","last_undegraded":"2024-04-21T08:38:55.075651+0000","last_fullsized":"2024-04-21T08:38:55.075651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:05:21.090633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005972+0000","last_change":"2024-04-21T08:36:23.467522+0000","last_active":"2024-04-21T08:38:55.005972+0000","last_peered":"2024-04-21T08:38:55.005972+0000","last_clean":"2024-04-21T08:38:55.005972+0000","last_became_active":"2024-04-21T08:36:23.466379+0000","last_became_peered":"2024-04-21T08:36:23.466379+0000","last_unstale":"2024-04-21T08:38:55.005972+0000","last_undegraded":"2024-04-21T08:38:55.005972+0000","last_fullsized":"2024-04-21T08:38:55.005972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:04.185314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005193+0000","last_change":"2024-04-21T08:36:28.015120+0000","last_active":"2024-04-21T08:38:55.005193+0000","last_peered":"2024-04-21T08:38:55.005193+0000","last_clean":"2024-04-21T08:38:55.005193+0000","last_became_active":"2024-04-21T08:36:28.014701+0000","last_became_peered":"2024-04-21T08:36:28.014701+0000","last_unstale":"2024-04-21T08:38:55.005193+0000","last_undegraded":"2024-04-21T08:38:55.005193+0000","last_fullsized":"2024-04-21T08:38:55.005193+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:34:23.127622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890642+0000","last_change":"2024-04-21T08:36:23.475335+0000","last_active":"2024-04-21T08:38:54.890642+0000","last_peered":"2024-04-21T08:38:54.890642+0000","last_clean":"2024-04-21T08:38:54.890642+0000","last_became_active":"2024-04-21T08:36:23.474739+0000","last_became_peered":"2024-04-21T08:36:23.474739+0000","last_unstale":"2024-04-21T08:38:54.890642+0000","last_undegraded":"2024-04-21T08:38:54.890642+0000","last_fullsized":"2024-04-21T08:38:54.890642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:36.119649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077395+0000","last_change":"2024-04-21T08:36:23.977229+0000","last_active":"2024-04-21T08:38:55.077395+0000","last_peered":"2024-04-21T08:38:55.077395+0000","last_clean":"2024-04-21T08:38:55.077395+0000","last_became_active":"2024-04-21T08:36:23.976893+0000","last_became_peered":"2024-04-21T08:36:23.976893+0000","last_unstale":"2024-04-21T08:38:55.077395+0000","last_undegraded":"2024-04-21T08:38:55.077395+0000","last_fullsized":"2024-04-21T08:38:55.077395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:31:54.226274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886835+0000","last_change":"2024-04-21T08:36:28.018581+0000","last_active":"2024-04-21T08:38:54.886835+0000","last_peered":"2024-04-21T08:38:54.886835+0000","last_clean":"2024-04-21T08:38:54.886835+0000","last_became_active":"2024-04-21T08:36:28.018021+0000","last_became_peered":"2024-04-21T08:36:28.018021+0000","last_unstale":"2024-04-21T08:38:54.886835+0000","last_undegraded":"2024-04-21T08:38:54.886835+0000","last_fullsized":"2024-04-21T08:38:54.886835+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:29:00.355532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'73","reported_seq":160,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:39:00.890509+0000","last_change":"2024-04-21T08:36:23.473339+0000","last_active":"2024-04-21T08:39:00.890509+0000","last_peered":"2024-04-21T08:39:00.890509+0000","last_clean":"2024-04-21T08:39:00.890509+0000","last_became_active":"2024-04-21T08:36:23.472994+0000","last_became_peered":"2024-04-21T08:36:23.472994+0000","last_unstale":"2024-04-21T08:39:00.890509+0000","last_undegraded":"2024-04-21T08:39:00.890509+0000","last_fullsized":"2024-04-21T08:39:00.890509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:42.762273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":182,"num_read_kb":160,"num_write":129,"num_write_kb":1156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887378+0000","last_change":"2024-04-21T08:36:23.987354+0000","last_active":"2024-04-21T08:38:54.887378+0000","last_peered":"2024-04-21T08:38:54.887378+0000","last_clean":"2024-04-21T08:38:54.887378+0000","last_became_active":"2024-04-21T08:36:23.987032+0000","last_became_peered":"2024-04-21T08:36:23.987032+0000","last_unstale":"2024-04-21T08:38:54.887378+0000","last_undegraded":"2024-04-21T08:38:54.887378+0000","last_fullsized":"2024-04-21T08:38:54.887378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:01:05.164367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104455+0000","last_change":"2024-04-21T08:36:28.002582+0000","last_active":"2024-04-21T08:38:55.104455+0000","last_peered":"2024-04-21T08:38:55.104455+0000","last_clean":"2024-04-21T08:38:55.104455+0000","last_became_active":"2024-04-21T08:36:28.002133+0000","last_became_peered":"2024-04-21T08:36:28.002133+0000","last_unstale":"2024-04-21T08:38:55.104455+0000","last_undegraded":"2024-04-21T08:38:55.104455+0000","last_fullsized":"2024-04-21T08:38:55.104455+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:30:00.268015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007058+0000","last_change":"2024-04-21T08:36:23.462840+0000","last_active":"2024-04-21T08:38:55.007058+0000","last_peered":"2024-04-21T08:38:55.007058+0000","last_clean":"2024-04-21T08:38:55.007058+0000","last_became_active":"2024-04-21T08:36:23.462391+0000","last_became_peered":"2024-04-21T08:36:23.462391+0000","last_unstale":"2024-04-21T08:38:55.007058+0000","last_undegraded":"2024-04-21T08:38:55.007058+0000","last_fullsized":"2024-04-21T08:38:55.007058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:33.230388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.004380+0000","last_change":"2024-04-21T08:36:23.994751+0000","last_active":"2024-04-21T08:38:55.004380+0000","last_peered":"2024-04-21T08:38:55.004380+0000","last_clean":"2024-04-21T08:38:55.004380+0000","last_became_active":"2024-04-21T08:36:23.994384+0000","last_became_peered":"2024-04-21T08:36:23.994384+0000","last_unstale":"2024-04-21T08:38:55.004380+0000","last_undegraded":"2024-04-21T08:38:55.004380+0000","last_fullsized":"2024-04-21T08:38:55.004380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:18.956893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006981+0000","last_change":"2024-04-21T08:36:28.004004+0000","last_active":"2024-04-21T08:38:55.006981+0000","last_peered":"2024-04-21T08:38:55.006981+0000","last_clean":"2024-04-21T08:38:55.006981+0000","last_became_active":"2024-04-21T08:36:28.003547+0000","last_became_peered":"2024-04-21T08:36:28.003547+0000","last_unstale":"2024-04-21T08:38:55.006981+0000","last_undegraded":"2024-04-21T08:38:55.006981+0000","last_fullsized":"2024-04-21T08:38:55.006981+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:21:40.215428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104510+0000","last_change":"2024-04-21T08:37:15.222118+0000","last_active":"2024-04-21T08:38:55.104510+0000","last_peered":"2024-04-21T08:38:55.104510+0000","last_clean":"2024-04-21T08:38:55.104510+0000","last_became_active":"2024-04-21T08:37:15.219173+0000","last_became_peered":"2024-04-21T08:37:15.219173+0000","last_unstale":"2024-04-21T08:38:55.104510+0000","last_undegraded":"2024-04-21T08:38:55.104510+0000","last_fullsized":"2024-04-21T08:38:55.104510+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:12.931076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890299+0000","last_change":"2024-04-21T08:36:23.988020+0000","last_active":"2024-04-21T08:38:54.890299+0000","last_peered":"2024-04-21T08:38:54.890299+0000","last_clean":"2024-04-21T08:38:54.890299+0000","last_became_active":"2024-04-21T08:36:23.987630+0000","last_became_peered":"2024-04-21T08:36:23.987630+0000","last_unstale":"2024-04-21T08:38:54.890299+0000","last_undegraded":"2024-04-21T08:38:54.890299+0000","last_fullsized":"2024-04-21T08:38:54.890299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:06.519001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.997084+0000","last_change":"2024-04-21T08:36:28.006984+0000","last_active":"2024-04-21T08:38:54.997084+0000","last_peered":"2024-04-21T08:38:54.997084+0000","last_clean":"2024-04-21T08:38:54.997084+0000","last_became_active":"2024-04-21T08:36:28.006548+0000","last_became_peered":"2024-04-21T08:36:28.006548+0000","last_unstale":"2024-04-21T08:38:54.997084+0000","last_undegraded":"2024-04-21T08:38:54.997084+0000","last_fullsized":"2024-04-21T08:38:54.997084+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:25:50.870658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886907+0000","last_change":"2024-04-21T08:36:23.476266+0000","last_active":"2024-04-21T08:38:54.886907+0000","last_peered":"2024-04-21T08:38:54.886907+0000","last_clean":"2024-04-21T08:38:54.886907+0000","last_became_active":"2024-04-21T08:36:23.475948+0000","last_became_peered":"2024-04-21T08:36:23.475948+0000","last_unstale":"2024-04-21T08:38:54.886907+0000","last_undegraded":"2024-04-21T08:38:54.886907+0000","last_fullsized":"2024-04-21T08:38:54.886907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:41.529376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005217+0000","last_change":"2024-04-21T08:36:23.981942+0000","last_active":"2024-04-21T08:38:55.005217+0000","last_peered":"2024-04-21T08:38:55.005217+0000","last_clean":"2024-04-21T08:38:55.005217+0000","last_became_active":"2024-04-21T08:36:23.981643+0000","last_became_peered":"2024-04-21T08:36:23.981643+0000","last_unstale":"2024-04-21T08:38:55.005217+0000","last_undegraded":"2024-04-21T08:38:55.005217+0000","last_fullsized":"2024-04-21T08:38:55.005217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:26:24.722695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006307+0000","last_change":"2024-04-21T08:36:28.006980+0000","last_active":"2024-04-21T08:38:55.006307+0000","last_peered":"2024-04-21T08:38:55.006307+0000","last_clean":"2024-04-21T08:38:55.006307+0000","last_became_active":"2024-04-21T08:36:28.006530+0000","last_became_peered":"2024-04-21T08:36:28.006530+0000","last_unstale":"2024-04-21T08:38:55.006307+0000","last_undegraded":"2024-04-21T08:38:55.006307+0000","last_fullsized":"2024-04-21T08:38:55.006307+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:05:57.465612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"95'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104627+0000","last_change":"2024-04-21T08:36:23.473253+0000","last_active":"2024-04-21T08:38:55.104627+0000","last_peered":"2024-04-21T08:38:55.104627+0000","last_clean":"2024-04-21T08:38:55.104627+0000","last_became_active":"2024-04-21T08:36:23.472942+0000","last_became_peered":"2024-04-21T08:36:23.472942+0000","last_unstale":"2024-04-21T08:38:55.104627+0000","last_undegraded":"2024-04-21T08:38:55.104627+0000","last_fullsized":"2024-04-21T08:38:55.104627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:32:04.894388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887248+0000","last_change":"2024-04-21T08:36:23.988355+0000","last_active":"2024-04-21T08:38:54.887248+0000","last_peered":"2024-04-21T08:38:54.887248+0000","last_clean":"2024-04-21T08:38:54.887248+0000","last_became_active":"2024-04-21T08:36:23.988049+0000","last_became_peered":"2024-04-21T08:36:23.988049+0000","last_unstale":"2024-04-21T08:38:54.887248+0000","last_undegraded":"2024-04-21T08:38:54.887248+0000","last_fullsized":"2024-04-21T08:38:54.887248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:57.348698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006797+0000","last_change":"2024-04-21T08:36:28.007343+0000","last_active":"2024-04-21T08:38:55.006797+0000","last_peered":"2024-04-21T08:38:55.006797+0000","last_clean":"2024-04-21T08:38:55.006797+0000","last_became_active":"2024-04-21T08:36:28.006719+0000","last_became_peered":"2024-04-21T08:36:28.006719+0000","last_unstale":"2024-04-21T08:38:55.006797+0000","last_undegraded":"2024-04-21T08:38:55.006797+0000","last_fullsized":"2024-04-21T08:38:55.006797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:02:25.734584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.890843+0000","last_change":"2024-04-21T08:36:23.472416+0000","last_active":"2024-04-21T08:38:54.890843+0000","last_peered":"2024-04-21T08:38:54.890843+0000","last_clean":"2024-04-21T08:38:54.890843+0000","last_became_active":"2024-04-21T08:36:23.471500+0000","last_became_peered":"2024-04-21T08:36:23.471500+0000","last_unstale":"2024-04-21T08:38:54.890843+0000","last_undegraded":"2024-04-21T08:38:54.890843+0000","last_fullsized":"2024-04-21T08:38:54.890843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:39.706432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006128+0000","last_change":"2024-04-21T08:36:23.990653+0000","last_active":"2024-04-21T08:38:55.006128+0000","last_peered":"2024-04-21T08:38:55.006128+0000","last_clean":"2024-04-21T08:38:55.006128+0000","last_became_active":"2024-04-21T08:36:23.990324+0000","last_became_peered":"2024-04-21T08:36:23.990324+0000","last_unstale":"2024-04-21T08:38:55.006128+0000","last_undegraded":"2024-04-21T08:38:55.006128+0000","last_fullsized":"2024-04-21T08:38:55.006128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:08:59.417850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104492+0000","last_change":"2024-04-21T08:36:28.007956+0000","last_active":"2024-04-21T08:38:55.104492+0000","last_peered":"2024-04-21T08:38:55.104492+0000","last_clean":"2024-04-21T08:38:55.104492+0000","last_became_active":"2024-04-21T08:36:28.007475+0000","last_became_peered":"2024-04-21T08:36:28.007475+0000","last_unstale":"2024-04-21T08:38:55.104492+0000","last_undegraded":"2024-04-21T08:38:55.104492+0000","last_fullsized":"2024-04-21T08:38:55.104492+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:54.039359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'8","reported_seq":175,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:59.164772+0000","last_change":"2024-04-21T08:36:23.476887+0000","last_active":"2024-04-21T08:38:59.164772+0000","last_peered":"2024-04-21T08:38:59.164772+0000","last_clean":"2024-04-21T08:38:59.164772+0000","last_became_active":"2024-04-21T08:36:23.476574+0000","last_became_peered":"2024-04-21T08:36:23.476574+0000","last_unstale":"2024-04-21T08:38:59.164772+0000","last_undegraded":"2024-04-21T08:38:59.164772+0000","last_fullsized":"2024-04-21T08:38:59.164772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:37:47.692358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2010,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006973+0000","last_change":"2024-04-21T08:36:23.982845+0000","last_active":"2024-04-21T08:38:55.006973+0000","last_peered":"2024-04-21T08:38:55.006973+0000","last_clean":"2024-04-21T08:38:55.006973+0000","last_became_active":"2024-04-21T08:36:23.982491+0000","last_became_peered":"2024-04-21T08:36:23.982491+0000","last_unstale":"2024-04-21T08:38:55.006973+0000","last_undegraded":"2024-04-21T08:38:55.006973+0000","last_fullsized":"2024-04-21T08:38:55.006973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:48:46.413896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.002896+0000","last_change":"2024-04-21T08:36:28.010442+0000","last_active":"2024-04-21T08:38:55.002896+0000","last_peered":"2024-04-21T08:38:55.002896+0000","last_clean":"2024-04-21T08:38:55.002896+0000","last_became_active":"2024-04-21T08:36:28.010046+0000","last_became_peered":"2024-04-21T08:36:28.010046+0000","last_unstale":"2024-04-21T08:38:55.002896+0000","last_undegraded":"2024-04-21T08:38:55.002896+0000","last_fullsized":"2024-04-21T08:38:55.002896+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:18:36.382802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"95'59","reported_seq":111,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:39:00.885741+0000","last_change":"2024-04-21T08:36:23.473545+0000","last_active":"2024-04-21T08:39:00.885741+0000","last_peered":"2024-04-21T08:39:00.885741+0000","last_clean":"2024-04-21T08:39:00.885741+0000","last_became_active":"2024-04-21T08:36:23.473222+0000","last_became_peered":"2024-04-21T08:36:23.473222+0000","last_unstale":"2024-04-21T08:39:00.885741+0000","last_undegraded":"2024-04-21T08:39:00.885741+0000","last_fullsized":"2024-04-21T08:39:00.885741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:47:53.947187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":20,"num_write":68,"num_write_kb":807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006600+0000","last_change":"2024-04-21T08:36:23.990959+0000","last_active":"2024-04-21T08:38:55.006600+0000","last_peered":"2024-04-21T08:38:55.006600+0000","last_clean":"2024-04-21T08:38:55.006600+0000","last_became_active":"2024-04-21T08:36:23.990322+0000","last_became_peered":"2024-04-21T08:36:23.990322+0000","last_unstale":"2024-04-21T08:38:55.006600+0000","last_undegraded":"2024-04-21T08:38:55.006600+0000","last_fullsized":"2024-04-21T08:38:55.006600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:12:20.859224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007717+0000","last_change":"2024-04-21T08:36:28.006682+0000","last_active":"2024-04-21T08:38:55.007717+0000","last_peered":"2024-04-21T08:38:55.007717+0000","last_clean":"2024-04-21T08:38:55.007717+0000","last_became_active":"2024-04-21T08:36:28.006229+0000","last_became_peered":"2024-04-21T08:36:28.006229+0000","last_unstale":"2024-04-21T08:38:55.007717+0000","last_undegraded":"2024-04-21T08:38:55.007717+0000","last_fullsized":"2024-04-21T08:38:55.007717+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:26.976315+0000","last_clean_scrub_stamp":"2024-04-21T08:36:26.976315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:30:12.723848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077921+0000","last_change":"2024-04-21T08:36:23.473592+0000","last_active":"2024-04-21T08:38:55.077921+0000","last_peered":"2024-04-21T08:38:55.077921+0000","last_clean":"2024-04-21T08:38:55.077921+0000","last_became_active":"2024-04-21T08:36:23.473283+0000","last_became_peered":"2024-04-21T08:36:23.473283+0000","last_unstale":"2024-04-21T08:38:55.077921+0000","last_undegraded":"2024-04-21T08:38:55.077921+0000","last_fullsized":"2024-04-21T08:38:55.077921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:22.442649+0000","last_clean_scrub_stamp":"2024-04-21T08:36:22.442649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:10.425249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.006895+0000","last_change":"2024-04-21T08:36:23.985052+0000","last_active":"2024-04-21T08:38:55.006895+0000","last_peered":"2024-04-21T08:38:55.006895+0000","last_clean":"2024-04-21T08:38:55.006895+0000","last_became_active":"2024-04-21T08:36:23.984742+0000","last_became_peered":"2024-04-21T08:36:23.984742+0000","last_unstale":"2024-04-21T08:38:55.006895+0000","last_undegraded":"2024-04-21T08:38:55.006895+0000","last_fullsized":"2024-04-21T08:38:55.006895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:08:06.082974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005201+0000","last_change":"2024-04-21T08:36:23.980905+0000","last_active":"2024-04-21T08:38:55.005201+0000","last_peered":"2024-04-21T08:38:55.005201+0000","last_clean":"2024-04-21T08:38:55.005201+0000","last_became_active":"2024-04-21T08:36:23.980590+0000","last_became_peered":"2024-04-21T08:36:23.980590+0000","last_unstale":"2024-04-21T08:38:55.005201+0000","last_undegraded":"2024-04-21T08:38:55.005201+0000","last_fullsized":"2024-04-21T08:38:55.005201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:54:59.886014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104725+0000","last_change":"2024-04-21T08:36:23.980883+0000","last_active":"2024-04-21T08:38:55.104725+0000","last_peered":"2024-04-21T08:38:55.104725+0000","last_clean":"2024-04-21T08:38:55.104725+0000","last_became_active":"2024-04-21T08:36:23.980562+0000","last_became_peered":"2024-04-21T08:36:23.980562+0000","last_unstale":"2024-04-21T08:38:55.104725+0000","last_undegraded":"2024-04-21T08:38:55.104725+0000","last_fullsized":"2024-04-21T08:38:55.104725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:14:51.868438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104692+0000","last_change":"2024-04-21T08:36:23.991571+0000","last_active":"2024-04-21T08:38:55.104692+0000","last_peered":"2024-04-21T08:38:55.104692+0000","last_clean":"2024-04-21T08:38:55.104692+0000","last_became_active":"2024-04-21T08:36:23.991273+0000","last_became_peered":"2024-04-21T08:36:23.991273+0000","last_unstale":"2024-04-21T08:38:55.104692+0000","last_undegraded":"2024-04-21T08:38:55.104692+0000","last_fullsized":"2024-04-21T08:38:55.104692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:08:23.895038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.003835+0000","last_change":"2024-04-21T08:36:23.994729+0000","last_active":"2024-04-21T08:38:55.003835+0000","last_peered":"2024-04-21T08:38:55.003835+0000","last_clean":"2024-04-21T08:38:55.003835+0000","last_became_active":"2024-04-21T08:36:23.994405+0000","last_became_peered":"2024-04-21T08:36:23.994405+0000","last_unstale":"2024-04-21T08:38:55.003835+0000","last_undegraded":"2024-04-21T08:38:55.003835+0000","last_fullsized":"2024-04-21T08:38:55.003835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:42.463225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.077385+0000","last_change":"2024-04-21T08:36:23.995421+0000","last_active":"2024-04-21T08:38:55.077385+0000","last_peered":"2024-04-21T08:38:55.077385+0000","last_clean":"2024-04-21T08:38:55.077385+0000","last_became_active":"2024-04-21T08:36:23.993674+0000","last_became_peered":"2024-04-21T08:36:23.993674+0000","last_unstale":"2024-04-21T08:38:55.077385+0000","last_undegraded":"2024-04-21T08:38:55.077385+0000","last_fullsized":"2024-04-21T08:38:55.077385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:31.013874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005504+0000","last_change":"2024-04-21T08:36:23.986418+0000","last_active":"2024-04-21T08:38:55.005504+0000","last_peered":"2024-04-21T08:38:55.005504+0000","last_clean":"2024-04-21T08:38:55.005504+0000","last_became_active":"2024-04-21T08:36:23.986095+0000","last_became_peered":"2024-04-21T08:36:23.986095+0000","last_unstale":"2024-04-21T08:38:55.005504+0000","last_undegraded":"2024-04-21T08:38:55.005504+0000","last_fullsized":"2024-04-21T08:38:55.005504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:31:56.456218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104918+0000","last_change":"2024-04-21T08:36:23.982353+0000","last_active":"2024-04-21T08:38:55.104918+0000","last_peered":"2024-04-21T08:38:55.104918+0000","last_clean":"2024-04-21T08:38:55.104918+0000","last_became_active":"2024-04-21T08:36:23.982035+0000","last_became_peered":"2024-04-21T08:36:23.982035+0000","last_unstale":"2024-04-21T08:38:55.104918+0000","last_undegraded":"2024-04-21T08:38:55.104918+0000","last_fullsized":"2024-04-21T08:38:55.104918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:42:44.598407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.887832+0000","last_change":"2024-04-21T08:36:23.996371+0000","last_active":"2024-04-21T08:38:54.887832+0000","last_peered":"2024-04-21T08:38:54.887832+0000","last_clean":"2024-04-21T08:38:54.887832+0000","last_became_active":"2024-04-21T08:36:23.995802+0000","last_became_peered":"2024-04-21T08:36:23.995802+0000","last_unstale":"2024-04-21T08:38:54.887832+0000","last_undegraded":"2024-04-21T08:38:54.887832+0000","last_fullsized":"2024-04-21T08:38:54.887832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:52:08.571567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.007826+0000","last_change":"2024-04-21T08:36:23.987275+0000","last_active":"2024-04-21T08:38:55.007826+0000","last_peered":"2024-04-21T08:38:55.007826+0000","last_clean":"2024-04-21T08:38:55.007826+0000","last_became_active":"2024-04-21T08:36:23.986967+0000","last_became_peered":"2024-04-21T08:36:23.986967+0000","last_unstale":"2024-04-21T08:38:55.007826+0000","last_undegraded":"2024-04-21T08:38:55.007826+0000","last_fullsized":"2024-04-21T08:38:55.007826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:50.432502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:54.886991+0000","last_change":"2024-04-21T08:36:23.993046+0000","last_active":"2024-04-21T08:38:54.886991+0000","last_peered":"2024-04-21T08:38:54.886991+0000","last_clean":"2024-04-21T08:38:54.886991+0000","last_became_active":"2024-04-21T08:36:23.992693+0000","last_became_peered":"2024-04-21T08:36:23.992693+0000","last_unstale":"2024-04-21T08:38:54.886991+0000","last_undegraded":"2024-04-21T08:38:54.886991+0000","last_fullsized":"2024-04-21T08:38:54.886991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:12:06.287585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.104706+0000","last_change":"2024-04-21T08:36:23.992968+0000","last_active":"2024-04-21T08:38:55.104706+0000","last_peered":"2024-04-21T08:38:55.104706+0000","last_clean":"2024-04-21T08:38:55.104706+0000","last_became_active":"2024-04-21T08:36:23.992614+0000","last_became_peered":"2024-04-21T08:36:23.992614+0000","last_unstale":"2024-04-21T08:38:55.104706+0000","last_undegraded":"2024-04-21T08:38:55.104706+0000","last_fullsized":"2024-04-21T08:38:55.104706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:10.015323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.076758+0000","last_change":"2024-04-21T08:36:23.981169+0000","last_active":"2024-04-21T08:38:55.076758+0000","last_peered":"2024-04-21T08:38:55.076758+0000","last_clean":"2024-04-21T08:38:55.076758+0000","last_became_active":"2024-04-21T08:36:23.980849+0000","last_became_peered":"2024-04-21T08:36:23.980849+0000","last_unstale":"2024-04-21T08:38:55.076758+0000","last_undegraded":"2024-04-21T08:38:55.076758+0000","last_fullsized":"2024-04-21T08:38:55.076758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:13:49.907370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-21T08:38:55.005358+0000","last_change":"2024-04-21T08:36:23.993998+0000","last_active":"2024-04-21T08:38:55.005358+0000","last_peered":"2024-04-21T08:38:55.005358+0000","last_clean":"2024-04-21T08:38:55.005358+0000","last_became_active":"2024-04-21T08:36:23.993664+0000","last_became_peered":"2024-04-21T08:36:23.993664+0000","last_unstale":"2024-04-21T08:38:55.005358+0000","last_undegraded":"2024-04-21T08:38:55.005358+0000","last_fullsized":"2024-04-21T08:38:55.005358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.447797+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.447797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:31:49.076820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":801015,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1558,"num_read_kb":947,"num_write":445,"num_write_kb":2130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2403045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":353,"ondisk_log_size":353,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351020,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1093632,"data_stored":535779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612658,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37220,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707932,"statfs":{"total":95995035648,"available":95956922368,"internally_reserved":0,"allocated":1929216,"data_stored":1324346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776183,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36820,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708332,"statfs":{"total":95995035648,"available":95957331968,"internally_reserved":0,"allocated":1515520,"data_stored":932181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037820,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":932559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299456,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1712128,"data_stored":1133590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462981,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1093632,"data_stored":530764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1167360,"data_stored":601862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1118208,"data_stored":532669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1187840,"data_stored":603454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1054897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920848,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":535370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724620,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32984,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712168,"statfs":{"total":95995035648,"available":95961260032,"internally_reserved":0,"allocated":1851392,"data_stored":1255362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":331975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":400959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:39:06.346 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:39:06.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.485+0000 7f706e515640 1 -- 172.21.15.43:0/656829161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7068069620 msgr2=0x7f7068069a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:06.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.485+0000 7f706e515640 1 --2- 172.21.15.43:0/656829161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7068069620 0x7f7068069a60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f705c009900 tx=0x7f705c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:06.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.486+0000 7f706e515640 1 -- 172.21.15.43:0/656829161 shutdown_connections 2024-04-21T08:39:06.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.486+0000 7f706e515640 1 --2- 172.21.15.43:0/656829161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f706810e810 0x7f7068110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.486+0000 7f706e515640 1 --2- 172.21.15.43:0/656829161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7068069620 0x7f7068069a60 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.486+0000 7f706e515640 1 --2- 172.21.15.43:0/656829161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7068102e80 0x7f70680690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.486+0000 7f706e515640 1 -- 172.21.15.43:0/656829161 >> 172.21.15.43:0/656829161 conn(0x7f7068077e30 msgr2=0x7f7068078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:06.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.486+0000 7f706e515640 1 -- 172.21.15.43:0/656829161 shutdown_connections 2024-04-21T08:39:06.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.486+0000 7f706e515640 1 -- 172.21.15.43:0/656829161 wait complete. 2024-04-21T08:39:06.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.486+0000 7f706e515640 1 Processor -- start 2024-04-21T08:39:06.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.486+0000 7f706e515640 1 -- start start 2024-04-21T08:39:06.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.487+0000 7f706e515640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7068069620 0x7f70681a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:06.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.487+0000 7f706e515640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7068102e80 0x7f70681a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:06.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.487+0000 7f706e515640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f706810e810 0x7f706819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:06.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.487+0000 7f706e515640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f70681143a0 con 0x7f7068069620 2024-04-21T08:39:06.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.487+0000 7f706e515640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7068114220 con 0x7f706810e810 2024-04-21T08:39:06.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.487+0000 7f706e515640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7068114520 con 0x7f7068102e80 2024-04-21T08:39:06.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.487+0000 7f7067fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7068069620 0x7f70681a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:06.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.487+0000 7f7067fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7068069620 0x7f70681a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39128/0 (socket says 172.21.15.43:39128) 2024-04-21T08:39:06.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.487+0000 7f7067fff640 1 -- 172.21.15.43:0/528332187 learned_addr learned my addr 172.21.15.43:0/528332187 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:06.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.487+0000 7f70677fe640 1 --2- 172.21.15.43:0/528332187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7068102e80 0x7f70681a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:06.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.487+0000 7f706cab2640 1 --2- 172.21.15.43:0/528332187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f706810e810 0x7f706819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:06.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.487+0000 7f7067fff640 1 -- 172.21.15.43:0/528332187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7068102e80 msgr2=0x7f70681a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:06.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.487+0000 7f7067fff640 1 --2- 172.21.15.43:0/528332187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7068102e80 0x7f70681a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.487+0000 7f7067fff640 1 -- 172.21.15.43:0/528332187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f706810e810 msgr2=0x7f706819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:06.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.487+0000 7f7067fff640 1 --2- 172.21.15.43:0/528332187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f706810e810 0x7f706819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.487+0000 7f7067fff640 1 -- 172.21.15.43:0/528332187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f706819af60 con 0x7f7068069620 2024-04-21T08:39:06.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.487+0000 7f7067fff640 1 --2- 172.21.15.43:0/528332187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7068069620 0x7f70681a05e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f705800c890 tx=0x7f705800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:06.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.487+0000 7f70657fa640 1 -- 172.21.15.43:0/528332187 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058004220 con 0x7f7068069620 2024-04-21T08:39:06.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.487+0000 7f70657fa640 1 -- 172.21.15.43:0/528332187 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70580043c0 con 0x7f7068069620 2024-04-21T08:39:06.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.487+0000 7f706e515640 1 -- 172.21.15.43:0/528332187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f706819b250 con 0x7f7068069620 2024-04-21T08:39:06.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.487+0000 7f70657fa640 1 -- 172.21.15.43:0/528332187 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058004a00 con 0x7f7068069620 2024-04-21T08:39:06.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.488+0000 7f706e515640 1 -- 172.21.15.43:0/528332187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70681a7560 con 0x7f7068069620 2024-04-21T08:39:06.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.488+0000 7f706e515640 1 -- 172.21.15.43:0/528332187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f706819b590 con 0x7f7068069620 2024-04-21T08:39:06.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.490+0000 7f70657fa640 1 -- 172.21.15.43:0/528332187 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f70580079b0 con 0x7f7068069620 2024-04-21T08:39:06.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.491+0000 7f70657fa640 1 --2- 172.21.15.43:0/528332187 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f70500a3db0 0x7f70500a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:06.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.492+0000 7f70677fe640 1 --2- 172.21.15.43:0/528332187 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f70500a3db0 0x7f70500a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:06.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.492+0000 7f70657fa640 1 -- 172.21.15.43:0/528332187 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f70580d0ec0 con 0x7f7068069620 2024-04-21T08:39:06.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.492+0000 7f70677fe640 1 --2- 172.21.15.43:0/528332187 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f70500a3db0 0x7f70500a6270 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f705c006010 tx=0x7f705c005e40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:06.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.495+0000 7f70657fa640 1 -- 172.21.15.43:0/528332187 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7058014030 con 0x7f7068069620 2024-04-21T08:39:06.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.724+0000 7f706e515640 1 -- 172.21.15.43:0/528332187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7068101430 con 0x7f7068069620 2024-04-21T08:39:06.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.725+0000 7f70657fa640 1 -- 172.21.15.43:0/528332187 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7f705809d020 con 0x7f7068069620 2024-04-21T08:39:06.724 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:39:06.724 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:39:06.724 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:39:06.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.727+0000 7f706e515640 1 -- 172.21.15.43:0/528332187 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f70500a3db0 msgr2=0x7f70500a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:06.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.727+0000 7f706e515640 1 --2- 172.21.15.43:0/528332187 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f70500a3db0 0x7f70500a6270 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f705c006010 tx=0x7f705c005e40 comp rx=0 tx=0).stop 2024-04-21T08:39:06.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.727+0000 7f706e515640 1 -- 172.21.15.43:0/528332187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7068069620 msgr2=0x7f70681a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:06.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.727+0000 7f706e515640 1 --2- 172.21.15.43:0/528332187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7068069620 0x7f70681a05e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f705800c890 tx=0x7f705800cd50 comp rx=0 tx=0).stop 2024-04-21T08:39:06.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.727+0000 7f706e515640 1 -- 172.21.15.43:0/528332187 shutdown_connections 2024-04-21T08:39:06.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.727+0000 7f706e515640 1 --2- 172.21.15.43:0/528332187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f706810e810 0x7f706819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.727+0000 7f706e515640 1 --2- 172.21.15.43:0/528332187 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f70500a3db0 0x7f70500a6270 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.727+0000 7f706e515640 1 --2- 172.21.15.43:0/528332187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7068102e80 0x7f70681a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.727+0000 7f706e515640 1 --2- 172.21.15.43:0/528332187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7068069620 0x7f70681a05e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.727+0000 7f706e515640 1 -- 172.21.15.43:0/528332187 >> 172.21.15.43:0/528332187 conn(0x7f7068077e30 msgr2=0x7f706810f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:06.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.727+0000 7f706e515640 1 -- 172.21.15.43:0/528332187 shutdown_connections 2024-04-21T08:39:06.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.727+0000 7f706e515640 1 -- 172.21.15.43:0/528332187 wait complete. 2024-04-21T08:39:06.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:06 smithi043 ceph-mon[27978]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 73 KiB/s wr, 39 op/s 2024-04-21T08:39:06.738 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:39:06.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:06 smithi112 ceph-mon[31085]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 73 KiB/s wr, 39 op/s 2024-04-21T08:39:06.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.842+0000 7fb2f7375640 1 -- 172.21.15.43:0/3904144622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2f0102b50 msgr2=0x7fb2f0106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:06.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.842+0000 7fb2f7375640 1 --2- 172.21.15.43:0/3904144622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2f0102b50 0x7fb2f0106b90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009900 tx=0x7fb2e002bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:06.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.842+0000 7fb2f7375640 1 -- 172.21.15.43:0/3904144622 shutdown_connections 2024-04-21T08:39:06.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.842+0000 7fb2f7375640 1 --2- 172.21.15.43:0/3904144622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2f0107270 0x7fb2f010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.842+0000 7fb2f7375640 1 --2- 172.21.15.43:0/3904144622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2f0102b50 0x7fb2f0106b90 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.842+0000 7fb2f7375640 1 --2- 172.21.15.43:0/3904144622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f01021a0 0x7fb2f0102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.842+0000 7fb2f7375640 1 -- 172.21.15.43:0/3904144622 >> 172.21.15.43:0/3904144622 conn(0x7fb2f00793a0 msgr2=0x7fb2f00797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:06.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.842+0000 7fb2f7375640 1 -- 172.21.15.43:0/3904144622 shutdown_connections 2024-04-21T08:39:06.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.842+0000 7fb2f7375640 1 -- 172.21.15.43:0/3904144622 wait complete. 2024-04-21T08:39:06.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.843+0000 7fb2f7375640 1 Processor -- start 2024-04-21T08:39:06.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.843+0000 7fb2f7375640 1 -- start start 2024-04-21T08:39:06.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.843+0000 7fb2f7375640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2f01021a0 0x7fb2f0074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:06.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.843+0000 7fb2f7375640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2f0107270 0x7fb2f0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:06.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.843+0000 7fb2f7375640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f0074cd0 0x7fb2f01a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:06.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.843+0000 7fb2f7375640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f0112200 con 0x7fb2f0107270 2024-04-21T08:39:06.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.843+0000 7fb2f7375640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f0112080 con 0x7fb2f0074cd0 2024-04-21T08:39:06.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.843+0000 7fb2f7375640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f0112380 con 0x7fb2f01021a0 2024-04-21T08:39:06.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.843+0000 7fb2f4910640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2f0107270 0x7fb2f0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:06.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.843+0000 7fb2f4910640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2f0107270 0x7fb2f0074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49970/0 (socket says 172.21.15.43:49970) 2024-04-21T08:39:06.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.843+0000 7fb2f4910640 1 -- 172.21.15.43:0/3678809050 learned_addr learned my addr 172.21.15.43:0/3678809050 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:06.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.843+0000 7fb2f5111640 1 --2- 172.21.15.43:0/3678809050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2f01021a0 0x7fb2f0074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:06.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.843+0000 7fb2f5912640 1 --2- 172.21.15.43:0/3678809050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f0074cd0 0x7fb2f01a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:06.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.843+0000 7fb2f4910640 1 -- 172.21.15.43:0/3678809050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2f01021a0 msgr2=0x7fb2f0074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:06.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.843+0000 7fb2f4910640 1 --2- 172.21.15.43:0/3678809050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2f01021a0 0x7fb2f0074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.843+0000 7fb2f4910640 1 -- 172.21.15.43:0/3678809050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f0074cd0 msgr2=0x7fb2f01a68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:06.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.843+0000 7fb2f4910640 1 --2- 172.21.15.43:0/3678809050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f0074cd0 0x7fb2f01a68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:06.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.844+0000 7fb2f4910640 1 -- 172.21.15.43:0/3678809050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f01a6fd0 con 0x7fb2f0107270 2024-04-21T08:39:06.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.844+0000 7fb2f5111640 1 --2- 172.21.15.43:0/3678809050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2f01021a0 0x7fb2f0074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:06.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.844+0000 7fb2f4910640 1 --2- 172.21.15.43:0/3678809050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2f0107270 0x7fb2f0074790 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0000c00 tx=0x7fb2e002c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:06.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.844+0000 7fb2e67fc640 1 -- 172.21.15.43:0/3678809050 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0043020 con 0x7fb2f0107270 2024-04-21T08:39:06.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.844+0000 7fb2e67fc640 1 -- 172.21.15.43:0/3678809050 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2e002cc60 con 0x7fb2f0107270 2024-04-21T08:39:06.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.844+0000 7fb2f7375640 1 -- 172.21.15.43:0/3678809050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f01a7260 con 0x7fb2f0107270 2024-04-21T08:39:06.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.844+0000 7fb2e67fc640 1 -- 172.21.15.43:0/3678809050 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0003c30 con 0x7fb2f0107270 2024-04-21T08:39:06.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.844+0000 7fb2f7375640 1 -- 172.21.15.43:0/3678809050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f01a7800 con 0x7fb2f0107270 2024-04-21T08:39:06.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.845+0000 7fb2f7375640 1 -- 172.21.15.43:0/3678809050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2f0069230 con 0x7fb2f0107270 2024-04-21T08:39:06.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.847+0000 7fb2e67fc640 1 -- 172.21.15.43:0/3678809050 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fb2e0046020 con 0x7fb2f0107270 2024-04-21T08:39:06.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.847+0000 7fb2e67fc640 1 --2- 172.21.15.43:0/3678809050 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb2dc0a3e60 0x7fb2dc0a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:06.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.847+0000 7fb2f5111640 1 --2- 172.21.15.43:0/3678809050 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb2dc0a3e60 0x7fb2dc0a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:06.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.848+0000 7fb2e67fc640 1 -- 172.21.15.43:0/3678809050 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fb2e00f3260 con 0x7fb2f0107270 2024-04-21T08:39:06.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.848+0000 7fb2f5111640 1 --2- 172.21.15.43:0/3678809050 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb2dc0a3e60 0x7fb2dc0a6320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec005fd0 tx=0x7fb2ec00b5f0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:06.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:06.851+0000 7fb2e67fc640 1 -- 172.21.15.43:0/3678809050 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb2e004d020 con 0x7fb2f0107270 2024-04-21T08:39:06.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:06 smithi175 ceph-mon[31920]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 73 KiB/s wr, 39 op/s 2024-04-21T08:39:07.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.042+0000 7fb2f7375640 1 -- 172.21.15.43:0/3678809050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb2f010a4a0 con 0x7fb2f0107270 2024-04-21T08:39:07.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.044+0000 7fb2e67fc640 1 -- 172.21.15.43:0/3678809050 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+25233 (secure 0 0 0) 0x7fb2e003e450 con 0x7fb2f0107270 2024-04-21T08:39:07.043 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:39:07.043 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":95,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:38:54.881943+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/126475132":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/4288951239":"2024-04-22T08:38:54.881882+0000","172.21.15.43:6801/1577030772":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/2382670190":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/1174411284":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/1577030772":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:39:07.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.046+0000 7fb2f7375640 1 -- 172.21.15.43:0/3678809050 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb2dc0a3e60 msgr2=0x7fb2dc0a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.046+0000 7fb2f7375640 1 --2- 172.21.15.43:0/3678809050 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb2dc0a3e60 0x7fb2dc0a6320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec005fd0 tx=0x7fb2ec00b5f0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.046+0000 7fb2f7375640 1 -- 172.21.15.43:0/3678809050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2f0107270 msgr2=0x7fb2f0074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.046+0000 7fb2f7375640 1 --2- 172.21.15.43:0/3678809050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2f0107270 0x7fb2f0074790 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0000c00 tx=0x7fb2e002c860 comp rx=0 tx=0).stop 2024-04-21T08:39:07.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.046+0000 7fb2f7375640 1 -- 172.21.15.43:0/3678809050 shutdown_connections 2024-04-21T08:39:07.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.046+0000 7fb2f7375640 1 --2- 172.21.15.43:0/3678809050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f0074cd0 0x7fb2f01a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.046+0000 7fb2f7375640 1 --2- 172.21.15.43:0/3678809050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2f0107270 0x7fb2f0074790 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.046+0000 7fb2f7375640 1 --2- 172.21.15.43:0/3678809050 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb2dc0a3e60 0x7fb2dc0a6320 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.046+0000 7fb2f7375640 1 --2- 172.21.15.43:0/3678809050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2f01021a0 0x7fb2f0074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.046+0000 7fb2f7375640 1 -- 172.21.15.43:0/3678809050 >> 172.21.15.43:0/3678809050 conn(0x7fb2f00793a0 msgr2=0x7fb2f00fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:07.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.046+0000 7fb2f7375640 1 -- 172.21.15.43:0/3678809050 shutdown_connections 2024-04-21T08:39:07.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.046+0000 7fb2f7375640 1 -- 172.21.15.43:0/3678809050 wait complete. 2024-04-21T08:39:07.056 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:39.981405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:22.034751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:22.780795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:25.984261+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:39:07.057 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:39:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.163+0000 7ff9967b6640 1 -- 172.21.15.43:0/1311241426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff990101a50 msgr2=0x7ff990105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.163+0000 7ff9967b6640 1 --2- 172.21.15.43:0/1311241426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff990101a50 0x7ff990105a90 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff978009900 tx=0x7ff97802bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.163+0000 7ff9967b6640 1 -- 172.21.15.43:0/1311241426 shutdown_connections 2024-04-21T08:39:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.163+0000 7ff9967b6640 1 --2- 172.21.15.43:0/1311241426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff990106170 0x7ff99010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.163+0000 7ff9967b6640 1 --2- 172.21.15.43:0/1311241426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff990101a50 0x7ff990105a90 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.163+0000 7ff9967b6640 1 --2- 172.21.15.43:0/1311241426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9901010a0 0x7ff990101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.163+0000 7ff9967b6640 1 -- 172.21.15.43:0/1311241426 >> 172.21.15.43:0/1311241426 conn(0x7ff990077eb0 msgr2=0x7ff9900782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.163+0000 7ff9967b6640 1 -- 172.21.15.43:0/1311241426 shutdown_connections 2024-04-21T08:39:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.163+0000 7ff9967b6640 1 -- 172.21.15.43:0/1311241426 wait complete. 2024-04-21T08:39:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.164+0000 7ff9967b6640 1 Processor -- start 2024-04-21T08:39:07.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.164+0000 7ff9967b6640 1 -- start start 2024-04-21T08:39:07.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.164+0000 7ff9967b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9901010a0 0x7ff990198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.164+0000 7ff9967b6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff990106170 0x7ff990198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.164+0000 7ff9967b6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff99019d010 0x7ff99019d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.164+0000 7ff9967b6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff9901121f0 con 0x7ff99019d010 2024-04-21T08:39:07.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.164+0000 7ff9967b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff990112070 con 0x7ff9901010a0 2024-04-21T08:39:07.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.164+0000 7ff9967b6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff990112370 con 0x7ff990106170 2024-04-21T08:39:07.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.164+0000 7ff994d53640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff99019d010 0x7ff99019d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.164+0000 7ff994d53640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff99019d010 0x7ff99019d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49988/0 (socket says 172.21.15.43:49988) 2024-04-21T08:39:07.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.164+0000 7ff994d53640 1 -- 172.21.15.43:0/1887568483 learned_addr learned my addr 172.21.15.43:0/1887568483 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:07.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.165+0000 7ff98f7fe640 1 --2- 172.21.15.43:0/1887568483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff990106170 0x7ff990198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.165+0000 7ff98ffff640 1 --2- 172.21.15.43:0/1887568483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9901010a0 0x7ff990198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.165+0000 7ff994d53640 1 -- 172.21.15.43:0/1887568483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff990106170 msgr2=0x7ff990198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.165+0000 7ff994d53640 1 --2- 172.21.15.43:0/1887568483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff990106170 0x7ff990198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.165+0000 7ff994d53640 1 -- 172.21.15.43:0/1887568483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9901010a0 msgr2=0x7ff990198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.165+0000 7ff994d53640 1 --2- 172.21.15.43:0/1887568483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9901010a0 0x7ff990198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.165+0000 7ff994d53640 1 -- 172.21.15.43:0/1887568483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99019db00 con 0x7ff99019d010 2024-04-21T08:39:07.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.165+0000 7ff98f7fe640 1 --2- 172.21.15.43:0/1887568483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff990106170 0x7ff990198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:07.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.165+0000 7ff98ffff640 1 --2- 172.21.15.43:0/1887568483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9901010a0 0x7ff990198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:07.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.165+0000 7ff994d53640 1 --2- 172.21.15.43:0/1887568483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff99019d010 0x7ff99019d4c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff98800cad0 tx=0x7ff98800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:07.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.165+0000 7ff98d7fa640 1 -- 172.21.15.43:0/1887568483 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98800ae20 con 0x7ff99019d010 2024-04-21T08:39:07.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.165+0000 7ff98d7fa640 1 -- 172.21.15.43:0/1887568483 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff988005020 con 0x7ff99019d010 2024-04-21T08:39:07.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.165+0000 7ff9967b6640 1 -- 172.21.15.43:0/1887568483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff99019ddf0 con 0x7ff99019d010 2024-04-21T08:39:07.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.165+0000 7ff98d7fa640 1 -- 172.21.15.43:0/1887568483 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98800f6b0 con 0x7ff99019d010 2024-04-21T08:39:07.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.165+0000 7ff9967b6640 1 -- 172.21.15.43:0/1887568483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9901a55b0 con 0x7ff99019d010 2024-04-21T08:39:07.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.166+0000 7ff9967b6640 1 -- 172.21.15.43:0/1887568483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9900691c0 con 0x7ff99019d010 2024-04-21T08:39:07.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.167+0000 7ff98d7fa640 1 -- 172.21.15.43:0/1887568483 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7ff9880051d0 con 0x7ff99019d010 2024-04-21T08:39:07.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.168+0000 7ff98d7fa640 1 --2- 172.21.15.43:0/1887568483 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff97c0a3db0 0x7ff97c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.168+0000 7ff98d7fa640 1 -- 172.21.15.43:0/1887568483 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7ff9880d0e00 con 0x7ff99019d010 2024-04-21T08:39:07.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.168+0000 7ff98ffff640 1 --2- 172.21.15.43:0/1887568483 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff97c0a3db0 0x7ff97c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.168+0000 7ff98ffff640 1 --2- 172.21.15.43:0/1887568483 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff97c0a3db0 0x7ff97c0a6270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff9800058c0 tx=0x7ff980005d30 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:07.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.172+0000 7ff98d7fa640 1 -- 172.21.15.43:0/1887568483 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff988014030 con 0x7ff99019d010 2024-04-21T08:39:07.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.361+0000 7ff9967b6640 1 -- 172.21.15.43:0/1887568483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff990101a50 con 0x7ff99019d010 2024-04-21T08:39:07.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.362+0000 7ff98d7fa640 1 -- 172.21.15.43:0/1887568483 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7ff98809cf60 con 0x7ff99019d010 2024-04-21T08:39:07.361 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-21T08:39:07.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.364+0000 7ff9967b6640 1 -- 172.21.15.43:0/1887568483 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff97c0a3db0 msgr2=0x7ff97c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.364+0000 7ff9967b6640 1 --2- 172.21.15.43:0/1887568483 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff97c0a3db0 0x7ff97c0a6270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff9800058c0 tx=0x7ff980005d30 comp rx=0 tx=0).stop 2024-04-21T08:39:07.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.364+0000 7ff9967b6640 1 -- 172.21.15.43:0/1887568483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff99019d010 msgr2=0x7ff99019d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.364+0000 7ff9967b6640 1 --2- 172.21.15.43:0/1887568483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff99019d010 0x7ff99019d4c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff98800cad0 tx=0x7ff98800cf90 comp rx=0 tx=0).stop 2024-04-21T08:39:07.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.364+0000 7ff9967b6640 1 -- 172.21.15.43:0/1887568483 shutdown_connections 2024-04-21T08:39:07.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.364+0000 7ff9967b6640 1 --2- 172.21.15.43:0/1887568483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff99019d010 0x7ff99019d4c0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.364+0000 7ff9967b6640 1 --2- 172.21.15.43:0/1887568483 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff97c0a3db0 0x7ff97c0a6270 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.364+0000 7ff9967b6640 1 --2- 172.21.15.43:0/1887568483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff990106170 0x7ff990198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.364+0000 7ff9967b6640 1 --2- 172.21.15.43:0/1887568483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9901010a0 0x7ff990198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.364+0000 7ff9967b6640 1 -- 172.21.15.43:0/1887568483 >> 172.21.15.43:0/1887568483 conn(0x7ff990077eb0 msgr2=0x7ff990107340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:07.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.364+0000 7ff9967b6640 1 -- 172.21.15.43:0/1887568483 shutdown_connections 2024-04-21T08:39:07.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.364+0000 7ff9967b6640 1 -- 172.21.15.43:0/1887568483 wait complete. 2024-04-21T08:39:07.374 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:39:07.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.480+0000 7f7ea9bef640 1 -- 172.21.15.43:0/805786066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ea41031d0 msgr2=0x7f7ea41035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.480+0000 7f7ea9bef640 1 --2- 172.21.15.43:0/805786066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ea41031d0 0x7f7ea41035b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009900 tx=0x7f7e8c02bd60 comp rx=0 tx=0).stop 2024-04-21T08:39:07.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.480+0000 7f7ea9bef640 1 -- 172.21.15.43:0/805786066 shutdown_connections 2024-04-21T08:39:07.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.480+0000 7f7ea9bef640 1 --2- 172.21.15.43:0/805786066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ea410c510 0x7f7ea410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.480+0000 7f7ea9bef640 1 --2- 172.21.15.43:0/805786066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea4103af0 0x7f7ea410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.480+0000 7f7ea9bef640 1 --2- 172.21.15.43:0/805786066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ea41031d0 0x7f7ea41035b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.480+0000 7f7ea9bef640 1 -- 172.21.15.43:0/805786066 >> 172.21.15.43:0/805786066 conn(0x7f7ea4077f50 msgr2=0x7f7ea4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:07.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.480+0000 7f7ea9bef640 1 -- 172.21.15.43:0/805786066 shutdown_connections 2024-04-21T08:39:07.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.480+0000 7f7ea9bef640 1 -- 172.21.15.43:0/805786066 wait complete. 2024-04-21T08:39:07.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.481+0000 7f7ea9bef640 1 Processor -- start 2024-04-21T08:39:07.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.481+0000 7f7ea9bef640 1 -- start start 2024-04-21T08:39:07.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.481+0000 7f7ea9bef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea41031d0 0x7f7ea4198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.481+0000 7f7ea9bef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ea4103af0 0x7f7ea4198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.481+0000 7f7ea9bef640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ea410c510 0x7f7ea419d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.481+0000 7f7ea9bef640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea41121a0 con 0x7f7ea4103af0 2024-04-21T08:39:07.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.481+0000 7f7ea9bef640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea4112020 con 0x7f7ea41031d0 2024-04-21T08:39:07.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.481+0000 7f7ea9bef640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea4112320 con 0x7f7ea410c510 2024-04-21T08:39:07.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.482+0000 7f7ea2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ea4103af0 0x7f7ea4198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.482+0000 7f7ea2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ea4103af0 0x7f7ea4198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50006/0 (socket says 172.21.15.43:50006) 2024-04-21T08:39:07.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.482+0000 7f7ea2ffd640 1 -- 172.21.15.43:0/3192376997 learned_addr learned my addr 172.21.15.43:0/3192376997 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:07.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.482+0000 7f7ea37fe640 1 --2- 172.21.15.43:0/3192376997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea41031d0 0x7f7ea4198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.482+0000 7f7ea3fff640 1 --2- 172.21.15.43:0/3192376997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ea410c510 0x7f7ea419d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.482+0000 7f7ea2ffd640 1 -- 172.21.15.43:0/3192376997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ea410c510 msgr2=0x7f7ea419d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.482+0000 7f7ea2ffd640 1 --2- 172.21.15.43:0/3192376997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ea410c510 0x7f7ea419d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.482+0000 7f7ea2ffd640 1 -- 172.21.15.43:0/3192376997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea41031d0 msgr2=0x7f7ea4198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.482+0000 7f7ea2ffd640 1 --2- 172.21.15.43:0/3192376997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea41031d0 0x7f7ea4198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.482+0000 7f7ea2ffd640 1 -- 172.21.15.43:0/3192376997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea419d730 con 0x7f7ea4103af0 2024-04-21T08:39:07.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.482+0000 7f7ea2ffd640 1 --2- 172.21.15.43:0/3192376997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ea4103af0 0x7f7ea4198c80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400d950 tx=0x7f7e9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:07.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.484+0000 7f7ea0ff9640 1 -- 172.21.15.43:0/3192376997 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94017070 con 0x7f7ea4103af0 2024-04-21T08:39:07.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.484+0000 7f7ea0ff9640 1 -- 172.21.15.43:0/3192376997 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e94009c30 con 0x7f7ea4103af0 2024-04-21T08:39:07.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.484+0000 7f7ea9bef640 1 -- 172.21.15.43:0/3192376997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea419da20 con 0x7f7ea4103af0 2024-04-21T08:39:07.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.484+0000 7f7ea0ff9640 1 -- 172.21.15.43:0/3192376997 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9400f620 con 0x7f7ea4103af0 2024-04-21T08:39:07.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.484+0000 7f7ea9bef640 1 -- 172.21.15.43:0/3192376997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea41a52b0 con 0x7f7ea4103af0 2024-04-21T08:39:07.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.485+0000 7f7ea9bef640 1 -- 172.21.15.43:0/3192376997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea4068d70 con 0x7f7ea4103af0 2024-04-21T08:39:07.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.486+0000 7f7ea0ff9640 1 -- 172.21.15.43:0/3192376997 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f7e94010460 con 0x7f7ea4103af0 2024-04-21T08:39:07.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.487+0000 7f7ea0ff9640 1 --2- 172.21.15.43:0/3192376997 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7e900a3db0 0x7f7e900a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.487+0000 7f7ea0ff9640 1 -- 172.21.15.43:0/3192376997 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f7e940d0980 con 0x7f7ea4103af0 2024-04-21T08:39:07.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.487+0000 7f7ea37fe640 1 --2- 172.21.15.43:0/3192376997 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7e900a3db0 0x7f7e900a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.487+0000 7f7ea37fe640 1 --2- 172.21.15.43:0/3192376997 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7e900a3db0 0x7f7e900a6270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c006010 tx=0x7f7e8c005e40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:07.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.491+0000 7f7ea0ff9640 1 -- 172.21.15.43:0/3192376997 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e94009de0 con 0x7f7ea4103af0 2024-04-21T08:39:07.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.677+0000 7f7ea9bef640 1 -- 172.21.15.43:0/3192376997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7ea41076c0 con 0x7f7ea4103af0 2024-04-21T08:39:07.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.678+0000 7f7ea0ff9640 1 -- 172.21.15.43:0/3192376997 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7f7e9409c430 con 0x7f7ea4103af0 2024-04-21T08:39:07.677 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:39:07.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.680+0000 7f7ea9bef640 1 -- 172.21.15.43:0/3192376997 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7e900a3db0 msgr2=0x7f7e900a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.680+0000 7f7ea9bef640 1 --2- 172.21.15.43:0/3192376997 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7e900a3db0 0x7f7e900a6270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c006010 tx=0x7f7e8c005e40 comp rx=0 tx=0).stop 2024-04-21T08:39:07.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.680+0000 7f7ea9bef640 1 -- 172.21.15.43:0/3192376997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ea4103af0 msgr2=0x7f7ea4198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.680+0000 7f7ea9bef640 1 --2- 172.21.15.43:0/3192376997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ea4103af0 0x7f7ea4198c80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400d950 tx=0x7f7e9400de10 comp rx=0 tx=0).stop 2024-04-21T08:39:07.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.680+0000 7f7ea9bef640 1 -- 172.21.15.43:0/3192376997 shutdown_connections 2024-04-21T08:39:07.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.680+0000 7f7ea9bef640 1 --2- 172.21.15.43:0/3192376997 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7e900a3db0 0x7f7e900a6270 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.680+0000 7f7ea9bef640 1 --2- 172.21.15.43:0/3192376997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ea410c510 0x7f7ea419d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.680+0000 7f7ea9bef640 1 --2- 172.21.15.43:0/3192376997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ea4103af0 0x7f7ea4198c80 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.680+0000 7f7ea9bef640 1 --2- 172.21.15.43:0/3192376997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ea41031d0 0x7f7ea4198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.680+0000 7f7ea9bef640 1 -- 172.21.15.43:0/3192376997 >> 172.21.15.43:0/3192376997 conn(0x7f7ea4077f50 msgr2=0x7f7ea40fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:07.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.680+0000 7f7ea9bef640 1 -- 172.21.15.43:0/3192376997 shutdown_connections 2024-04-21T08:39:07.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.680+0000 7f7ea9bef640 1 -- 172.21.15.43:0/3192376997 wait complete. 2024-04-21T08:39:07.689 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:07 smithi043 ceph-mon[27978]: from='client.15864 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:39:07.689 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:07 smithi043 ceph-mon[27978]: from='client.15870 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:39:07.689 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:07 smithi043 ceph-mon[27978]: from='client.15876 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:39:07.689 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:07 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/528332187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:07.689 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:07 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3678809050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:39:07.689 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:07 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1887568483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:39:07.690 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:39:07.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:07 smithi112 ceph-mon[31085]: from='client.15864 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:39:07.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:07 smithi112 ceph-mon[31085]: from='client.15870 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:39:07.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:07 smithi112 ceph-mon[31085]: from='client.15876 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:39:07.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:07 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/528332187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:07.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:07 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3678809050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:39:07.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:07 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1887568483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:39:07.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.796+0000 7ff400d3d640 1 -- 172.21.15.43:0/2559768739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3fc107270 msgr2=0x7ff3fc10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.796+0000 7ff400d3d640 1 --2- 172.21.15.43:0/2559768739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3fc107270 0x7ff3fc10ee40 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009ca0 tx=0x7ff3f402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.797+0000 7ff400d3d640 1 -- 172.21.15.43:0/2559768739 shutdown_connections 2024-04-21T08:39:07.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.797+0000 7ff400d3d640 1 --2- 172.21.15.43:0/2559768739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3fc107270 0x7ff3fc10ee40 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.797+0000 7ff400d3d640 1 --2- 172.21.15.43:0/2559768739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3fc06b1f0 0x7ff3fc106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.797+0000 7ff400d3d640 1 --2- 172.21.15.43:0/2559768739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3fc06a840 0x7ff3fc06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.797+0000 7ff400d3d640 1 -- 172.21.15.43:0/2559768739 >> 172.21.15.43:0/2559768739 conn(0x7ff3fc077eb0 msgr2=0x7ff3fc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:07.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.797+0000 7ff400d3d640 1 -- 172.21.15.43:0/2559768739 shutdown_connections 2024-04-21T08:39:07.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.797+0000 7ff400d3d640 1 -- 172.21.15.43:0/2559768739 wait complete. 2024-04-21T08:39:07.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.797+0000 7ff400d3d640 1 Processor -- start 2024-04-21T08:39:07.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.797+0000 7ff400d3d640 1 -- start start 2024-04-21T08:39:07.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.797+0000 7ff400d3d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3fc06a840 0x7ff3fc1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.797+0000 7ff400d3d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3fc107270 0x7ff3fc198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.797+0000 7ff400d3d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3fc19cf90 0x7ff3fc19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.798+0000 7ff400d3d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc1121d0 con 0x7ff3fc19cf90 2024-04-21T08:39:07.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.798+0000 7ff400d3d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc112050 con 0x7ff3fc107270 2024-04-21T08:39:07.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.798+0000 7ff400d3d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc112350 con 0x7ff3fc06a840 2024-04-21T08:39:07.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.798+0000 7ff3fad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3fc19cf90 0x7ff3fc19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.798+0000 7ff3fad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3fc19cf90 0x7ff3fc19d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50026/0 (socket says 172.21.15.43:50026) 2024-04-21T08:39:07.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.798+0000 7ff3fad9d640 1 -- 172.21.15.43:0/1072782904 learned_addr learned my addr 172.21.15.43:0/1072782904 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:07.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.798+0000 7ff3f9d9b640 1 --2- 172.21.15.43:0/1072782904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3fc107270 0x7ff3fc198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.798+0000 7ff3fa59c640 1 --2- 172.21.15.43:0/1072782904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3fc06a840 0x7ff3fc1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.798+0000 7ff3fad9d640 1 -- 172.21.15.43:0/1072782904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3fc06a840 msgr2=0x7ff3fc1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.798+0000 7ff3fad9d640 1 --2- 172.21.15.43:0/1072782904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3fc06a840 0x7ff3fc1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.798+0000 7ff3fad9d640 1 -- 172.21.15.43:0/1072782904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3fc107270 msgr2=0x7ff3fc198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.798+0000 7ff3fad9d640 1 --2- 172.21.15.43:0/1072782904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3fc107270 0x7ff3fc198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.798+0000 7ff3fad9d640 1 -- 172.21.15.43:0/1072782904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3fc19da80 con 0x7ff3fc19cf90 2024-04-21T08:39:07.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.798+0000 7ff3fa59c640 1 --2- 172.21.15.43:0/1072782904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3fc06a840 0x7ff3fc1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:07.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.798+0000 7ff3f9d9b640 1 --2- 172.21.15.43:0/1072782904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3fc107270 0x7ff3fc198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:07.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.798+0000 7ff3fad9d640 1 --2- 172.21.15.43:0/1072782904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3fc19cf90 0x7ff3fc19d440 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4005e30 tx=0x7ff3f4034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:07.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.798+0000 7ff3eb7fe640 1 -- 172.21.15.43:0/1072782904 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4039040 con 0x7ff3fc19cf90 2024-04-21T08:39:07.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.798+0000 7ff3eb7fe640 1 -- 172.21.15.43:0/1072782904 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3f4044070 con 0x7ff3fc19cf90 2024-04-21T08:39:07.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.798+0000 7ff400d3d640 1 -- 172.21.15.43:0/1072782904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3fc19dd10 con 0x7ff3fc19cf90 2024-04-21T08:39:07.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.798+0000 7ff3eb7fe640 1 -- 172.21.15.43:0/1072782904 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4003e60 con 0x7ff3fc19cf90 2024-04-21T08:39:07.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.798+0000 7ff400d3d640 1 -- 172.21.15.43:0/1072782904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3fc1a54d0 con 0x7ff3fc19cf90 2024-04-21T08:39:07.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.799+0000 7ff400d3d640 1 -- 172.21.15.43:0/1072782904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3fc0691c0 con 0x7ff3fc19cf90 2024-04-21T08:39:07.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.800+0000 7ff3eb7fe640 1 -- 172.21.15.43:0/1072782904 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7ff3f40324c0 con 0x7ff3fc19cf90 2024-04-21T08:39:07.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.801+0000 7ff3eb7fe640 1 --2- 172.21.15.43:0/1072782904 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff3e40a3db0 0x7ff3e40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.801+0000 7ff3fa59c640 1 --2- 172.21.15.43:0/1072782904 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff3e40a3db0 0x7ff3e40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.802+0000 7ff3eb7fe640 1 -- 172.21.15.43:0/1072782904 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7ff3f40f2fe0 con 0x7ff3fc19cf90 2024-04-21T08:39:07.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.802+0000 7ff3fa59c640 1 --2- 172.21.15.43:0/1072782904 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff3e40a3db0 0x7ff3e40a6270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0058c0 tx=0x7ff3ec005850 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:07.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.806+0000 7ff3eb7fe640 1 -- 172.21.15.43:0/1072782904 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff3f40bf230 con 0x7ff3fc19cf90 2024-04-21T08:39:07.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:07 smithi175 ceph-mon[31920]: from='client.15864 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:39:07.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:07 smithi175 ceph-mon[31920]: from='client.15870 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:39:07.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:07 smithi175 ceph-mon[31920]: from='client.15876 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:39:07.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:07 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/528332187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:07.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:07 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3678809050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:39:07.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:07 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1887568483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:39:07.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.992+0000 7ff400d3d640 1 -- 172.21.15.43:0/1072782904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff3fc06b1f0 con 0x7ff3fc19cf90 2024-04-21T08:39:07.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.992+0000 7ff3eb7fe640 1 -- 172.21.15.43:0/1072782904 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7ff3f40beb30 con 0x7ff3fc19cf90 2024-04-21T08:39:07.991 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-21T08:39:07.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.994+0000 7ff400d3d640 1 -- 172.21.15.43:0/1072782904 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff3e40a3db0 msgr2=0x7ff3e40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.994+0000 7ff400d3d640 1 --2- 172.21.15.43:0/1072782904 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff3e40a3db0 0x7ff3e40a6270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0058c0 tx=0x7ff3ec005850 comp rx=0 tx=0).stop 2024-04-21T08:39:07.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.994+0000 7ff400d3d640 1 -- 172.21.15.43:0/1072782904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3fc19cf90 msgr2=0x7ff3fc19d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.994+0000 7ff400d3d640 1 --2- 172.21.15.43:0/1072782904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3fc19cf90 0x7ff3fc19d440 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4005e30 tx=0x7ff3f4034760 comp rx=0 tx=0).stop 2024-04-21T08:39:07.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.994+0000 7ff400d3d640 1 -- 172.21.15.43:0/1072782904 shutdown_connections 2024-04-21T08:39:07.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.994+0000 7ff400d3d640 1 --2- 172.21.15.43:0/1072782904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3fc19cf90 0x7ff3fc19d440 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.994+0000 7ff400d3d640 1 --2- 172.21.15.43:0/1072782904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3fc107270 0x7ff3fc198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.994+0000 7ff400d3d640 1 --2- 172.21.15.43:0/1072782904 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff3e40a3db0 0x7ff3e40a6270 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.994+0000 7ff400d3d640 1 --2- 172.21.15.43:0/1072782904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3fc06a840 0x7ff3fc1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.994+0000 7ff400d3d640 1 -- 172.21.15.43:0/1072782904 >> 172.21.15.43:0/1072782904 conn(0x7ff3fc077eb0 msgr2=0x7ff3fc100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:07.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.994+0000 7ff400d3d640 1 -- 172.21.15.43:0/1072782904 shutdown_connections 2024-04-21T08:39:07.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:07.994+0000 7ff400d3d640 1 -- 172.21.15.43:0/1072782904 wait complete. 2024-04-21T08:39:08.004 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:39:08.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.113+0000 7ff25fe4e640 1 -- 172.21.15.43:0/120906098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff25806a7c0 msgr2=0x7ff25806aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:08.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.113+0000 7ff25fe4e640 1 --2- 172.21.15.43:0/120906098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff25806a7c0 0x7ff25806aba0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff254009900 tx=0x7ff25402bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:08.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.113+0000 7ff25fe4e640 1 -- 172.21.15.43:0/120906098 shutdown_connections 2024-04-21T08:39:08.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.113+0000 7ff25fe4e640 1 --2- 172.21.15.43:0/120906098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff258107280 0x7ff25810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.113+0000 7ff25fe4e640 1 --2- 172.21.15.43:0/120906098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff25806b170 0x7ff258106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.113+0000 7ff25fe4e640 1 --2- 172.21.15.43:0/120906098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff25806a7c0 0x7ff25806aba0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.113+0000 7ff25fe4e640 1 -- 172.21.15.43:0/120906098 >> 172.21.15.43:0/120906098 conn(0x7ff258076c50 msgr2=0x7ff258077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.113+0000 7ff25fe4e640 1 -- 172.21.15.43:0/120906098 shutdown_connections 2024-04-21T08:39:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.113+0000 7ff25fe4e640 1 -- 172.21.15.43:0/120906098 wait complete. 2024-04-21T08:39:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.114+0000 7ff25fe4e640 1 Processor -- start 2024-04-21T08:39:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.114+0000 7ff25fe4e640 1 -- start start 2024-04-21T08:39:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.114+0000 7ff25fe4e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff25806a7c0 0x7ff2581986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.114+0000 7ff25fe4e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff258107280 0x7ff258198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.114+0000 7ff25fe4e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff25819cfa0 0x7ff25819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.114+0000 7ff25fe4e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff2581121c0 con 0x7ff25819cfa0 2024-04-21T08:39:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.114+0000 7ff25fe4e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff258112040 con 0x7ff258107280 2024-04-21T08:39:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.114+0000 7ff25fe4e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff258112340 con 0x7ff25806a7c0 2024-04-21T08:39:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.114+0000 7ff25e3eb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff25819cfa0 0x7ff25819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:08.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.114+0000 7ff25e3eb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff25819cfa0 0x7ff25819d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50046/0 (socket says 172.21.15.43:50046) 2024-04-21T08:39:08.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.114+0000 7ff25e3eb640 1 -- 172.21.15.43:0/717659751 learned_addr learned my addr 172.21.15.43:0/717659751 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:08.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.114+0000 7ff25d3e9640 1 --2- 172.21.15.43:0/717659751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff258107280 0x7ff258198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:08.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.114+0000 7ff25dbea640 1 --2- 172.21.15.43:0/717659751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff25806a7c0 0x7ff2581986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:08.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.114+0000 7ff25e3eb640 1 -- 172.21.15.43:0/717659751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff25806a7c0 msgr2=0x7ff2581986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:08.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.114+0000 7ff25e3eb640 1 --2- 172.21.15.43:0/717659751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff25806a7c0 0x7ff2581986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.114+0000 7ff25e3eb640 1 -- 172.21.15.43:0/717659751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff258107280 msgr2=0x7ff258198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:08.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.114+0000 7ff25e3eb640 1 --2- 172.21.15.43:0/717659751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff258107280 0x7ff258198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.114+0000 7ff25e3eb640 1 -- 172.21.15.43:0/717659751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25819da90 con 0x7ff25819cfa0 2024-04-21T08:39:08.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.114+0000 7ff25d3e9640 1 --2- 172.21.15.43:0/717659751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff258107280 0x7ff258198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:08.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.114+0000 7ff25e3eb640 1 --2- 172.21.15.43:0/717659751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff25819cfa0 0x7ff25819d450 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff25000cad0 tx=0x7ff25000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:08.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.115+0000 7ff24effd640 1 -- 172.21.15.43:0/717659751 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25000ae20 con 0x7ff25819cfa0 2024-04-21T08:39:08.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.115+0000 7ff24effd640 1 -- 172.21.15.43:0/717659751 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff250005020 con 0x7ff25819cfa0 2024-04-21T08:39:08.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.115+0000 7ff25fe4e640 1 -- 172.21.15.43:0/717659751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff25819dd80 con 0x7ff25819cfa0 2024-04-21T08:39:08.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.115+0000 7ff24effd640 1 -- 172.21.15.43:0/717659751 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25000f6b0 con 0x7ff25819cfa0 2024-04-21T08:39:08.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.115+0000 7ff25fe4e640 1 -- 172.21.15.43:0/717659751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2581a5570 con 0x7ff25819cfa0 2024-04-21T08:39:08.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.116+0000 7ff25fe4e640 1 -- 172.21.15.43:0/717659751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff258103680 con 0x7ff25819cfa0 2024-04-21T08:39:08.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.117+0000 7ff24effd640 1 -- 172.21.15.43:0/717659751 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7ff2500051d0 con 0x7ff25819cfa0 2024-04-21T08:39:08.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.117+0000 7ff24effd640 1 --2- 172.21.15.43:0/717659751 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff2440a3e30 0x7ff2440a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:08.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.117+0000 7ff25dbea640 1 --2- 172.21.15.43:0/717659751 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff2440a3e30 0x7ff2440a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:08.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.118+0000 7ff24effd640 1 -- 172.21.15.43:0/717659751 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7ff2500d0e70 con 0x7ff25819cfa0 2024-04-21T08:39:08.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.118+0000 7ff25dbea640 1 --2- 172.21.15.43:0/717659751 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff2440a3e30 0x7ff2440a62f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff254000c00 tx=0x7ff254004120 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:08.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.122+0000 7ff24effd640 1 -- 172.21.15.43:0/717659751 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff250014030 con 0x7ff25819cfa0 2024-04-21T08:39:08.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.310+0000 7ff25fe4e640 1 -- 172.21.15.43:0/717659751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff258199a30 con 0x7ff25819cfa0 2024-04-21T08:39:08.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.310+0000 7ff24effd640 1 -- 172.21.15.43:0/717659751 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7ff25009cfd0 con 0x7ff25819cfa0 2024-04-21T08:39:08.310 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:39:08.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.312+0000 7ff25fe4e640 1 -- 172.21.15.43:0/717659751 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff2440a3e30 msgr2=0x7ff2440a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:08.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.312+0000 7ff25fe4e640 1 --2- 172.21.15.43:0/717659751 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff2440a3e30 0x7ff2440a62f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff254000c00 tx=0x7ff254004120 comp rx=0 tx=0).stop 2024-04-21T08:39:08.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.312+0000 7ff25fe4e640 1 -- 172.21.15.43:0/717659751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff25819cfa0 msgr2=0x7ff25819d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:08.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.312+0000 7ff25fe4e640 1 --2- 172.21.15.43:0/717659751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff25819cfa0 0x7ff25819d450 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff25000cad0 tx=0x7ff25000cf90 comp rx=0 tx=0).stop 2024-04-21T08:39:08.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.312+0000 7ff25fe4e640 1 -- 172.21.15.43:0/717659751 shutdown_connections 2024-04-21T08:39:08.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.313+0000 7ff25fe4e640 1 --2- 172.21.15.43:0/717659751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff25819cfa0 0x7ff25819d450 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.313+0000 7ff25fe4e640 1 --2- 172.21.15.43:0/717659751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff258107280 0x7ff258198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.313+0000 7ff25fe4e640 1 --2- 172.21.15.43:0/717659751 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff2440a3e30 0x7ff2440a62f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.313+0000 7ff25fe4e640 1 --2- 172.21.15.43:0/717659751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff25806a7c0 0x7ff2581986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.313+0000 7ff25fe4e640 1 -- 172.21.15.43:0/717659751 >> 172.21.15.43:0/717659751 conn(0x7ff258076c50 msgr2=0x7ff2580ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:08.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.313+0000 7ff25fe4e640 1 -- 172.21.15.43:0/717659751 shutdown_connections 2024-04-21T08:39:08.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.313+0000 7ff25fe4e640 1 -- 172.21.15.43:0/717659751 wait complete. 2024-04-21T08:39:08.323 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:39:08.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.426+0000 7ff51b425640 1 -- 172.21.15.43:0/1251483816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff514106f10 msgr2=0x7ff5141072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:08.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.426+0000 7ff51b425640 1 --2- 172.21.15.43:0/1251483816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff514106f10 0x7ff5141072f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff510009900 tx=0x7ff51002bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:08.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.426+0000 7ff51b425640 1 -- 172.21.15.43:0/1251483816 shutdown_connections 2024-04-21T08:39:08.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.426+0000 7ff51b425640 1 --2- 172.21.15.43:0/1251483816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff514101ad0 0x7ff51410dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.426+0000 7ff51b425640 1 --2- 172.21.15.43:0/1251483816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff514101130 0x7ff514101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.426+0000 7ff51b425640 1 --2- 172.21.15.43:0/1251483816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff514106f10 0x7ff5141072f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.426+0000 7ff51b425640 1 -- 172.21.15.43:0/1251483816 >> 172.21.15.43:0/1251483816 conn(0x7ff514077e30 msgr2=0x7ff514078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:08.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.426+0000 7ff51b425640 1 -- 172.21.15.43:0/1251483816 shutdown_connections 2024-04-21T08:39:08.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.426+0000 7ff51b425640 1 -- 172.21.15.43:0/1251483816 wait complete. 2024-04-21T08:39:08.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.427+0000 7ff51b425640 1 Processor -- start 2024-04-21T08:39:08.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.427+0000 7ff51b425640 1 -- start start 2024-04-21T08:39:08.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.427+0000 7ff51b425640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff514101130 0x7ff5141986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:08.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.427+0000 7ff51b425640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff514101ad0 0x7ff514198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:08.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.427+0000 7ff51b425640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff514106f10 0x7ff51419cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:08.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.427+0000 7ff51b425640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141110e0 con 0x7ff514101ad0 2024-04-21T08:39:08.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.427+0000 7ff51b425640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff514110f60 con 0x7ff514106f10 2024-04-21T08:39:08.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.427+0000 7ff51b425640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff514111260 con 0x7ff514101130 2024-04-21T08:39:08.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.427+0000 7ff5189c0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff514101ad0 0x7ff514198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:08.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.427+0000 7ff5189c0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff514101ad0 0x7ff514198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50060/0 (socket says 172.21.15.43:50060) 2024-04-21T08:39:08.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.427+0000 7ff5189c0640 1 -- 172.21.15.43:0/1024173076 learned_addr learned my addr 172.21.15.43:0/1024173076 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:08.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.427+0000 7ff5191c1640 1 --2- 172.21.15.43:0/1024173076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff514101130 0x7ff5141986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:08.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.428+0000 7ff5199c2640 1 --2- 172.21.15.43:0/1024173076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff514106f10 0x7ff51419cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:08.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.428+0000 7ff5189c0640 1 -- 172.21.15.43:0/1024173076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff514101130 msgr2=0x7ff5141986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:08.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.428+0000 7ff5189c0640 1 --2- 172.21.15.43:0/1024173076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff514101130 0x7ff5141986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.428+0000 7ff5189c0640 1 -- 172.21.15.43:0/1024173076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff514106f10 msgr2=0x7ff51419cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:08.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.428+0000 7ff5189c0640 1 --2- 172.21.15.43:0/1024173076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff514106f10 0x7ff51419cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.428+0000 7ff5189c0640 1 -- 172.21.15.43:0/1024173076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51419d6e0 con 0x7ff514101ad0 2024-04-21T08:39:08.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.428+0000 7ff5199c2640 1 --2- 172.21.15.43:0/1024173076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff514106f10 0x7ff51419cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:08.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.428+0000 7ff5189c0640 1 --2- 172.21.15.43:0/1024173076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff514101ad0 0x7ff514198c30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff50400c970 tx=0x7ff50400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:08.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.428+0000 7ff50a7fc640 1 -- 172.21.15.43:0/1024173076 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504007bb0 con 0x7ff514101ad0 2024-04-21T08:39:08.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.428+0000 7ff50a7fc640 1 -- 172.21.15.43:0/1024173076 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff504007d50 con 0x7ff514101ad0 2024-04-21T08:39:08.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.428+0000 7ff51b425640 1 -- 172.21.15.43:0/1024173076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff51419d9d0 con 0x7ff514101ad0 2024-04-21T08:39:08.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.428+0000 7ff50a7fc640 1 -- 172.21.15.43:0/1024173076 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50400f450 con 0x7ff514101ad0 2024-04-21T08:39:08.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.428+0000 7ff51b425640 1 -- 172.21.15.43:0/1024173076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5141a5260 con 0x7ff514101ad0 2024-04-21T08:39:08.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.429+0000 7ff51b425640 1 -- 172.21.15.43:0/1024173076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff514069230 con 0x7ff514101ad0 2024-04-21T08:39:08.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.430+0000 7ff50a7fc640 1 -- 172.21.15.43:0/1024173076 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7ff504004f00 con 0x7ff514101ad0 2024-04-21T08:39:08.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.431+0000 7ff50a7fc640 1 --2- 172.21.15.43:0/1024173076 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff5000a3db0 0x7ff5000a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:08.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.431+0000 7ff5191c1640 1 --2- 172.21.15.43:0/1024173076 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff5000a3db0 0x7ff5000a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:08.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.431+0000 7ff50a7fc640 1 -- 172.21.15.43:0/1024173076 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7ff5040d0ec0 con 0x7ff514101ad0 2024-04-21T08:39:08.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.431+0000 7ff5191c1640 1 --2- 172.21.15.43:0/1024173076 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff5000a3db0 0x7ff5000a6270 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff510009230 tx=0x7ff510005fd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:08.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.434+0000 7ff50a7fc640 1 -- 172.21.15.43:0/1024173076 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff504014030 con 0x7ff514101ad0 2024-04-21T08:39:08.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.661+0000 7ff51b425640 1 -- 172.21.15.43:0/1024173076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff514199a40 con 0x7ff514101ad0 2024-04-21T08:39:08.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.662+0000 7ff50a7fc640 1 -- 172.21.15.43:0/1024173076 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7ff514199a40 con 0x7ff514101ad0 2024-04-21T08:39:08.661 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:39:08.662 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:39:08.662 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:39:08.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.664+0000 7ff51b425640 1 -- 172.21.15.43:0/1024173076 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff5000a3db0 msgr2=0x7ff5000a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:08.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.664+0000 7ff51b425640 1 --2- 172.21.15.43:0/1024173076 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff5000a3db0 0x7ff5000a6270 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff510009230 tx=0x7ff510005fd0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.664+0000 7ff51b425640 1 -- 172.21.15.43:0/1024173076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff514101ad0 msgr2=0x7ff514198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:08.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.664+0000 7ff51b425640 1 --2- 172.21.15.43:0/1024173076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff514101ad0 0x7ff514198c30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff50400c970 tx=0x7ff50400ce30 comp rx=0 tx=0).stop 2024-04-21T08:39:08.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.664+0000 7ff51b425640 1 -- 172.21.15.43:0/1024173076 shutdown_connections 2024-04-21T08:39:08.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.664+0000 7ff51b425640 1 --2- 172.21.15.43:0/1024173076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff514106f10 0x7ff51419cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.664+0000 7ff51b425640 1 --2- 172.21.15.43:0/1024173076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff514101ad0 0x7ff514198c30 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.664+0000 7ff51b425640 1 --2- 172.21.15.43:0/1024173076 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff5000a3db0 0x7ff5000a6270 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.664+0000 7ff51b425640 1 --2- 172.21.15.43:0/1024173076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff514101130 0x7ff5141986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.664+0000 7ff51b425640 1 -- 172.21.15.43:0/1024173076 >> 172.21.15.43:0/1024173076 conn(0x7ff514077e30 msgr2=0x7ff51406a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:08.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.665+0000 7ff51b425640 1 -- 172.21.15.43:0/1024173076 shutdown_connections 2024-04-21T08:39:08.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.665+0000 7ff51b425640 1 -- 172.21.15.43:0/1024173076 wait complete. 2024-04-21T08:39:08.674 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:08 smithi043 ceph-mon[27978]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 55 op/s 2024-04-21T08:39:08.674 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:08 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3192376997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:39:08.674 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:08 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1072782904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:39:08.674 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:08 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/717659751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:39:08.675 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:39:08.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.778+0000 7f844349a640 1 -- 172.21.15.43:0/1997804231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f843c10e900 msgr2=0x7f843c110cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:08.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.778+0000 7f844349a640 1 --2- 172.21.15.43:0/1997804231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f843c10e900 0x7f843c110cf0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8434009ca0 tx=0x7f843402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.778+0000 7f844349a640 1 -- 172.21.15.43:0/1997804231 shutdown_connections 2024-04-21T08:39:08.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.778+0000 7f844349a640 1 --2- 172.21.15.43:0/1997804231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f843c10e900 0x7f843c110cf0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.778+0000 7f844349a640 1 --2- 172.21.15.43:0/1997804231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f843c101670 0x7f843c10e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.778+0000 7f844349a640 1 --2- 172.21.15.43:0/1997804231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f843c100d50 0x7f843c101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.778+0000 7f844349a640 1 -- 172.21.15.43:0/1997804231 >> 172.21.15.43:0/1997804231 conn(0x7f843c0771d0 msgr2=0x7f843c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:08.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.778+0000 7f844349a640 1 -- 172.21.15.43:0/1997804231 shutdown_connections 2024-04-21T08:39:08.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.778+0000 7f844349a640 1 -- 172.21.15.43:0/1997804231 wait complete. 2024-04-21T08:39:08.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.779+0000 7f844349a640 1 Processor -- start 2024-04-21T08:39:08.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.779+0000 7f844349a640 1 -- start start 2024-04-21T08:39:08.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.779+0000 7f844349a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f843c100d50 0x7f843c1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:08.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.779+0000 7f844349a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f843c101670 0x7f843c1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:08.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.779+0000 7f844349a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f843c10e900 0x7f843c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:08.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.779+0000 7f844349a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f843c1143e0 con 0x7f843c10e900 2024-04-21T08:39:08.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.779+0000 7f844349a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f843c114260 con 0x7f843c101670 2024-04-21T08:39:08.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.779+0000 7f844349a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f843c114560 con 0x7f843c100d50 2024-04-21T08:39:08.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.779+0000 7f8441a37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f843c10e900 0x7f843c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:08.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.779+0000 7f8441a37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f843c10e900 0x7f843c19a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50076/0 (socket says 172.21.15.43:50076) 2024-04-21T08:39:08.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.779+0000 7f8441a37640 1 -- 172.21.15.43:0/3464755153 learned_addr learned my addr 172.21.15.43:0/3464755153 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:08.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.779+0000 7f8440a35640 1 --2- 172.21.15.43:0/3464755153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f843c101670 0x7f843c1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:08.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.779+0000 7f8441236640 1 --2- 172.21.15.43:0/3464755153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f843c100d50 0x7f843c1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:08.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.780+0000 7f8441a37640 1 -- 172.21.15.43:0/3464755153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f843c100d50 msgr2=0x7f843c1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:08.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.780+0000 7f8441a37640 1 --2- 172.21.15.43:0/3464755153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f843c100d50 0x7f843c1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.780+0000 7f8441a37640 1 -- 172.21.15.43:0/3464755153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f843c101670 msgr2=0x7f843c1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:08.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.780+0000 7f8441a37640 1 --2- 172.21.15.43:0/3464755153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f843c101670 0x7f843c1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.780+0000 7f8441a37640 1 -- 172.21.15.43:0/3464755153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843c19afc0 con 0x7f843c10e900 2024-04-21T08:39:08.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.780+0000 7f8440a35640 1 --2- 172.21.15.43:0/3464755153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f843c101670 0x7f843c1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:08.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.780+0000 7f8441a37640 1 --2- 172.21.15.43:0/3464755153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f843c10e900 0x7f843c19a700 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8434004af0 tx=0x7f84340346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:08.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.780+0000 7f84327fc640 1 -- 172.21.15.43:0/3464755153 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8434039040 con 0x7f843c10e900 2024-04-21T08:39:08.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.780+0000 7f84327fc640 1 -- 172.21.15.43:0/3464755153 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8434044070 con 0x7f843c10e900 2024-04-21T08:39:08.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.780+0000 7f844349a640 1 -- 172.21.15.43:0/3464755153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843c19b250 con 0x7f843c10e900 2024-04-21T08:39:08.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.780+0000 7f84327fc640 1 -- 172.21.15.43:0/3464755153 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84340042a0 con 0x7f843c10e900 2024-04-21T08:39:08.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.780+0000 7f844349a640 1 -- 172.21.15.43:0/3464755153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f843c1a73e0 con 0x7f843c10e900 2024-04-21T08:39:08.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.781+0000 7f844349a640 1 -- 172.21.15.43:0/3464755153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f843c06ae60 con 0x7f843c10e900 2024-04-21T08:39:08.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.782+0000 7f84327fc640 1 -- 172.21.15.43:0/3464755153 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f8434004440 con 0x7f843c10e900 2024-04-21T08:39:08.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.783+0000 7f84327fc640 1 --2- 172.21.15.43:0/3464755153 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84280a3db0 0x7f84280a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:08.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.783+0000 7f8441236640 1 --2- 172.21.15.43:0/3464755153 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84280a3db0 0x7f84280a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:08.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.783+0000 7f84327fc640 1 -- 172.21.15.43:0/3464755153 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f84340f3a70 con 0x7f843c10e900 2024-04-21T08:39:08.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.783+0000 7f8441236640 1 --2- 172.21.15.43:0/3464755153 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84280a3db0 0x7f84280a6270 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8438009b60 tx=0x7f8438005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:08.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.786+0000 7f84327fc640 1 -- 172.21.15.43:0/3464755153 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f84340bfd10 con 0x7f843c10e900 2024-04-21T08:39:08.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:08 smithi112 ceph-mon[31085]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 55 op/s 2024-04-21T08:39:08.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:08 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3192376997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:39:08.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:08 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1072782904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:39:08.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:08 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/717659751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:39:08.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:08 smithi175 ceph-mon[31920]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 55 op/s 2024-04-21T08:39:08.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:08 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3192376997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:39:08.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:08 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1072782904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:39:08.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:08 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/717659751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:39:08.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.973+0000 7f844349a640 1 -- 172.21.15.43:0/3464755153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f843c109a90 con 0x7f843c10e900 2024-04-21T08:39:08.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.974+0000 7f84327fc640 1 -- 172.21.15.43:0/3464755153 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+25233 (secure 0 0 0) 0x7f8434051e10 con 0x7f843c10e900 2024-04-21T08:39:08.974 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:39:08.974 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":95,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:38:54.881943+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/126475132":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/4288951239":"2024-04-22T08:38:54.881882+0000","172.21.15.43:6801/1577030772":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/2382670190":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/1174411284":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/1577030772":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:39:08.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.976+0000 7f844349a640 1 -- 172.21.15.43:0/3464755153 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84280a3db0 msgr2=0x7f84280a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:08.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.976+0000 7f844349a640 1 --2- 172.21.15.43:0/3464755153 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84280a3db0 0x7f84280a6270 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8438009b60 tx=0x7f8438005f90 comp rx=0 tx=0).stop 2024-04-21T08:39:08.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.976+0000 7f844349a640 1 -- 172.21.15.43:0/3464755153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f843c10e900 msgr2=0x7f843c19a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:08.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.976+0000 7f844349a640 1 --2- 172.21.15.43:0/3464755153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f843c10e900 0x7f843c19a700 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8434004af0 tx=0x7f84340346f0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.976+0000 7f844349a640 1 -- 172.21.15.43:0/3464755153 shutdown_connections 2024-04-21T08:39:08.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.976+0000 7f844349a640 1 --2- 172.21.15.43:0/3464755153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f843c10e900 0x7f843c19a700 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.976+0000 7f844349a640 1 --2- 172.21.15.43:0/3464755153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f843c101670 0x7f843c1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.976+0000 7f844349a640 1 --2- 172.21.15.43:0/3464755153 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84280a3db0 0x7f84280a6270 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.976+0000 7f844349a640 1 --2- 172.21.15.43:0/3464755153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f843c100d50 0x7f843c1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:08.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.976+0000 7f844349a640 1 -- 172.21.15.43:0/3464755153 >> 172.21.15.43:0/3464755153 conn(0x7f843c0771d0 msgr2=0x7f843c10ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:08.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.976+0000 7f844349a640 1 -- 172.21.15.43:0/3464755153 shutdown_connections 2024-04-21T08:39:08.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:08.976+0000 7f844349a640 1 -- 172.21.15.43:0/3464755153 wait complete. 2024-04-21T08:39:08.987 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with random seed 613782 2024-04-21T08:39:08.987 DEBUG:tasks.quiescer:Yielding 2024-04-21T08:39:08.987 INFO:teuthology.run_tasks:Running task workunit... 2024-04-21T08:39:09.003 INFO:tasks.workunit:Pulling workunits from ref 4f6bc5273643aaf5660909e85400647a8951c10b 2024-04-21T08:39:09.003 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-21T08:39:09.004 INFO:tasks.workunit:timeout=3h 2024-04-21T08:39:09.004 INFO:tasks.workunit:cleanup=True 2024-04-21T08:39:09.004 DEBUG:teuthology.orchestra.run.smithi043:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-21T08:39:09.004 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:39:09.020 INFO:teuthology.orchestra.run.smithi043.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-21T08:39:09.020 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-21T08:39:09.020 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-21T08:39:09.021 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:39:09.021 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-21T08:39:09.021 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-04-21 08:36:52.007696062 +0000 2024-04-21T08:39:09.021 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-04-21 08:36:52.007696062 +0000 2024-04-21T08:39:09.021 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-04-21 08:38:52.529428208 +0000 2024-04-21T08:39:09.021 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-04-21 08:36:52.007696062 +0000 2024-04-21T08:39:09.021 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-21T08:39:09.021 DEBUG:teuthology.orchestra.run.smithi043:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-21T08:39:09.093 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/leonid-s-usov/ceph-ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 4f6bc5273643aaf5660909e85400647a8951c10b 2024-04-21T08:39:09.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.110+0000 7f458c1e2640 1 -- 172.21.15.43:0/69464398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4584102190 msgr2=0x7f4584102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.110+0000 7f458c1e2640 1 --2- 172.21.15.43:0/69464398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4584102190 0x7f4584102570 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4580009900 tx=0x7f458002bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:09.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.110+0000 7f458c1e2640 1 -- 172.21.15.43:0/69464398 shutdown_connections 2024-04-21T08:39:09.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.110+0000 7f458c1e2640 1 --2- 172.21.15.43:0/69464398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4584107260 0x7f458410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.110+0000 7f458c1e2640 1 --2- 172.21.15.43:0/69464398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4584102b40 0x7f4584106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.110+0000 7f458c1e2640 1 --2- 172.21.15.43:0/69464398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4584102190 0x7f4584102570 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.110+0000 7f458c1e2640 1 -- 172.21.15.43:0/69464398 >> 172.21.15.43:0/69464398 conn(0x7f45840793e0 msgr2=0x7f45840797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.110+0000 7f458c1e2640 1 -- 172.21.15.43:0/69464398 shutdown_connections 2024-04-21T08:39:09.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.110+0000 7f458c1e2640 1 -- 172.21.15.43:0/69464398 wait complete. 2024-04-21T08:39:09.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.111+0000 7f458c1e2640 1 Processor -- start 2024-04-21T08:39:09.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.111+0000 7f458c1e2640 1 -- start start 2024-04-21T08:39:09.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.111+0000 7f458c1e2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4584102190 0x7f45841986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.111+0000 7f458c1e2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4584107260 0x7f4584198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.111+0000 7f458c1e2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f458419cfa0 0x7f458419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.111+0000 7f458c1e2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4584112220 con 0x7f4584107260 2024-04-21T08:39:09.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.111+0000 7f458c1e2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f45841120a0 con 0x7f458419cfa0 2024-04-21T08:39:09.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.111+0000 7f458c1e2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f45841123a0 con 0x7f4584102190 2024-04-21T08:39:09.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.111+0000 7f458977d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4584107260 0x7f4584198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.111+0000 7f458977d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4584107260 0x7f4584198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50098/0 (socket says 172.21.15.43:50098) 2024-04-21T08:39:09.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.111+0000 7f458977d640 1 -- 172.21.15.43:0/2739804306 learned_addr learned my addr 172.21.15.43:0/2739804306 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:09.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.111+0000 7f458a77f640 1 --2- 172.21.15.43:0/2739804306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f458419cfa0 0x7f458419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.111+0000 7f4589f7e640 1 --2- 172.21.15.43:0/2739804306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4584102190 0x7f45841986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.111+0000 7f458977d640 1 -- 172.21.15.43:0/2739804306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4584102190 msgr2=0x7f45841986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.111+0000 7f458977d640 1 --2- 172.21.15.43:0/2739804306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4584102190 0x7f45841986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.111+0000 7f458977d640 1 -- 172.21.15.43:0/2739804306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f458419cfa0 msgr2=0x7f458419d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.111+0000 7f458977d640 1 --2- 172.21.15.43:0/2739804306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f458419cfa0 0x7f458419d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.111+0000 7f458977d640 1 -- 172.21.15.43:0/2739804306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458419da90 con 0x7f4584107260 2024-04-21T08:39:09.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.111+0000 7f458977d640 1 --2- 172.21.15.43:0/2739804306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4584107260 0x7f4584198c20 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f457400c970 tx=0x7f457400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.112+0000 7f457affd640 1 -- 172.21.15.43:0/2739804306 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4574014070 con 0x7f4584107260 2024-04-21T08:39:09.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.112+0000 7f458c1e2640 1 -- 172.21.15.43:0/2739804306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458419dd80 con 0x7f4584107260 2024-04-21T08:39:09.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.112+0000 7f457affd640 1 -- 172.21.15.43:0/2739804306 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4574004440 con 0x7f4584107260 2024-04-21T08:39:09.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.112+0000 7f457affd640 1 -- 172.21.15.43:0/2739804306 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4574003e50 con 0x7f4584107260 2024-04-21T08:39:09.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.112+0000 7f458c1e2640 1 -- 172.21.15.43:0/2739804306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45841a55c0 con 0x7f4584107260 2024-04-21T08:39:09.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.113+0000 7f458c1e2640 1 -- 172.21.15.43:0/2739804306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4584069230 con 0x7f4584107260 2024-04-21T08:39:09.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.114+0000 7f457affd640 1 -- 172.21.15.43:0/2739804306 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f4574017070 con 0x7f4584107260 2024-04-21T08:39:09.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.114+0000 7f457affd640 1 --2- 172.21.15.43:0/2739804306 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f45700a3f50 0x7f45700a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.114+0000 7f4589f7e640 1 --2- 172.21.15.43:0/2739804306 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f45700a3f50 0x7f45700a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.115+0000 7f457affd640 1 -- 172.21.15.43:0/2739804306 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f45740d09b0 con 0x7f4584107260 2024-04-21T08:39:09.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.115+0000 7f4589f7e640 1 --2- 172.21.15.43:0/2739804306 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f45700a3f50 0x7f45700a6410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4580006010 tx=0x7f4580000bc0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.119+0000 7f457affd640 1 -- 172.21.15.43:0/2739804306 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f457401e030 con 0x7f4584107260 2024-04-21T08:39:09.125 INFO:tasks.workunit.client.0.smithi043.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-21T08:39:09.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.347+0000 7f458c1e2640 1 -- 172.21.15.43:0/2739804306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4584102b40 con 0x7f4584107260 2024-04-21T08:39:09.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.348+0000 7f457affd640 1 -- 172.21.15.43:0/2739804306 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7f457400de20 con 0x7f4584107260 2024-04-21T08:39:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:39:09.348 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:39:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:39:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.350+0000 7f458c1e2640 1 -- 172.21.15.43:0/2739804306 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f45700a3f50 msgr2=0x7f45700a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.350+0000 7f458c1e2640 1 --2- 172.21.15.43:0/2739804306 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f45700a3f50 0x7f45700a6410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4580006010 tx=0x7f4580000bc0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.350+0000 7f458c1e2640 1 -- 172.21.15.43:0/2739804306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4584107260 msgr2=0x7f4584198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.350+0000 7f458c1e2640 1 --2- 172.21.15.43:0/2739804306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4584107260 0x7f4584198c20 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f457400c970 tx=0x7f457400ce30 comp rx=0 tx=0).stop 2024-04-21T08:39:09.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.350+0000 7f458c1e2640 1 -- 172.21.15.43:0/2739804306 shutdown_connections 2024-04-21T08:39:09.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.350+0000 7f458c1e2640 1 --2- 172.21.15.43:0/2739804306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f458419cfa0 0x7f458419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.350+0000 7f458c1e2640 1 --2- 172.21.15.43:0/2739804306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4584107260 0x7f4584198c20 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.350+0000 7f458c1e2640 1 --2- 172.21.15.43:0/2739804306 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f45700a3f50 0x7f45700a6410 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.350+0000 7f458c1e2640 1 --2- 172.21.15.43:0/2739804306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4584102190 0x7f45841986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.350+0000 7f458c1e2640 1 -- 172.21.15.43:0/2739804306 >> 172.21.15.43:0/2739804306 conn(0x7f45840793e0 msgr2=0x7f45840ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.350+0000 7f458c1e2640 1 -- 172.21.15.43:0/2739804306 shutdown_connections 2024-04-21T08:39:09.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.350+0000 7f458c1e2640 1 -- 172.21.15.43:0/2739804306 wait complete. 2024-04-21T08:39:09.361 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-21T08:36:23.821253+0000', 'modified': '2024-04-21T08:38:53.219677+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24527, 'mds_1': 24491, 'mds_2': 24487, 'mds_3': 24425, 'mds_4': 24412}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14634': {'gid': 14634, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.43:6839/931821965', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6838', 'nonce': 931821965}, {'type': 'v1', 'addr': '172.21.15.43:6839', 'nonce': 931821965}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14652': {'gid': 14652, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.43:6841/623279035', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6840', 'nonce': 623279035}, {'type': 'v1', 'addr': '172.21.15.43:6841', 'nonce': 623279035}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24412': {'gid': 24412, 'name': 'l', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.175:6833/187945366', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6832', 'nonce': 187945366}, {'type': 'v1', 'addr': '172.21.15.175:6833', 'nonce': 187945366}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.112:6833/1516781885', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6832', 'nonce': 1516781885}, {'type': 'v1', 'addr': '172.21.15.112:6833', 'nonce': 1516781885}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6835/466532477', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6834', 'nonce': 466532477}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 466532477}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.112:6835/3159749962', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6834', 'nonce': 3159749962}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 3159749962}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6837/3198693848', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6836', 'nonce': 3198693848}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 3198693848}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.175:6839/3664054463', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 3664054463}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 3664054463}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.112:6837/3054508488', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6836', 'nonce': 3054508488}, {'type': 'v1', 'addr': '172.21.15.112:6837', 'nonce': 3054508488}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24527': {'gid': 24527, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6839/2143575300', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6838', 'nonce': 2143575300}, {'type': 'v1', 'addr': '172.21.15.112:6839', 'nonce': 2143575300}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24527, 'qdb_cluster': [24412, 24425, 24487, 24491, 24527]} 2024-04-21T08:39:09.361 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-21T08:39:09.361 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-04-21T08:39:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.506+0000 7f1bab975640 1 -- 172.21.15.43:0/796223325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba4107270 msgr2=0x7f1ba410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.506+0000 7f1bab975640 1 --2- 172.21.15.43:0/796223325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba4107270 0x7f1ba410ee40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009ca0 tx=0x7f1b9c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.506+0000 7f1bab975640 1 -- 172.21.15.43:0/796223325 shutdown_connections 2024-04-21T08:39:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.506+0000 7f1bab975640 1 --2- 172.21.15.43:0/796223325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba4107270 0x7f1ba410ee40 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.506+0000 7f1bab975640 1 --2- 172.21.15.43:0/796223325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba406b1f0 0x7f1ba4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.506+0000 7f1bab975640 1 --2- 172.21.15.43:0/796223325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ba406a840 0x7f1ba406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.506+0000 7f1bab975640 1 -- 172.21.15.43:0/796223325 >> 172.21.15.43:0/796223325 conn(0x7f1ba4077eb0 msgr2=0x7f1ba40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.506+0000 7f1bab975640 1 -- 172.21.15.43:0/796223325 shutdown_connections 2024-04-21T08:39:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.506+0000 7f1bab975640 1 -- 172.21.15.43:0/796223325 wait complete. 2024-04-21T08:39:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.507+0000 7f1bab975640 1 Processor -- start 2024-04-21T08:39:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.507+0000 7f1bab975640 1 -- start start 2024-04-21T08:39:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.507+0000 7f1bab975640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba406a840 0x7f1ba4198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.507+0000 7f1bab975640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba4107270 0x7f1ba4198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.507+0000 7f1bab975640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ba419cf10 0x7f1ba419d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.507+0000 7f1bab975640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba41121b0 con 0x7f1ba4107270 2024-04-21T08:39:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.507+0000 7f1bab975640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba4112030 con 0x7f1ba406a840 2024-04-21T08:39:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.507+0000 7f1bab975640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba4112330 con 0x7f1ba419cf10 2024-04-21T08:39:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.507+0000 7f1ba8f10640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba4107270 0x7f1ba4198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.507+0000 7f1ba8f10640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba4107270 0x7f1ba4198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50128/0 (socket says 172.21.15.43:50128) 2024-04-21T08:39:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.507+0000 7f1ba8f10640 1 -- 172.21.15.43:0/2583329349 learned_addr learned my addr 172.21.15.43:0/2583329349 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.507+0000 7f1ba9711640 1 --2- 172.21.15.43:0/2583329349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba406a840 0x7f1ba4198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.507+0000 7f1ba9f12640 1 --2- 172.21.15.43:0/2583329349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ba419cf10 0x7f1ba419d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.507+0000 7f1ba8f10640 1 -- 172.21.15.43:0/2583329349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ba419cf10 msgr2=0x7f1ba419d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.507+0000 7f1ba8f10640 1 --2- 172.21.15.43:0/2583329349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ba419cf10 0x7f1ba419d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.507+0000 7f1ba8f10640 1 -- 172.21.15.43:0/2583329349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba406a840 msgr2=0x7f1ba4198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.507+0000 7f1ba8f10640 1 --2- 172.21.15.43:0/2583329349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba406a840 0x7f1ba4198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.507+0000 7f1ba8f10640 1 -- 172.21.15.43:0/2583329349 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba419da00 con 0x7f1ba4107270 2024-04-21T08:39:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.507+0000 7f1ba8f10640 1 --2- 172.21.15.43:0/2583329349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba4107270 0x7f1ba4198b90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1b9400e9c0 tx=0x7f1b9400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.508+0000 7f1b9a7fc640 1 -- 172.21.15.43:0/2583329349 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9400ccc0 con 0x7f1ba4107270 2024-04-21T08:39:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.508+0000 7f1b9a7fc640 1 -- 172.21.15.43:0/2583329349 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b940054d0 con 0x7f1ba4107270 2024-04-21T08:39:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.508+0000 7f1bab975640 1 -- 172.21.15.43:0/2583329349 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba419dcf0 con 0x7f1ba4107270 2024-04-21T08:39:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.508+0000 7f1b9a7fc640 1 -- 172.21.15.43:0/2583329349 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b94010430 con 0x7f1ba4107270 2024-04-21T08:39:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.508+0000 7f1bab975640 1 -- 172.21.15.43:0/2583329349 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba41a5530 con 0x7f1ba4107270 2024-04-21T08:39:09.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.510+0000 7f1b9a7fc640 1 -- 172.21.15.43:0/2583329349 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f1b94004f20 con 0x7f1ba4107270 2024-04-21T08:39:09.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.510+0000 7f1bae565740 1 Processor -- start 2024-04-21T08:39:09.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.510+0000 7f1bae565740 1 -- start start 2024-04-21T08:39:09.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.510+0000 7f1bae565740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280a57b0 0x55dc280a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.510+0000 7f1bae565740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc280a6160 0x55dc280ae710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.510+0000 7f1bae565740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc280aece0 0x55dc280b1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.510+0000 7f1bae565740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dc28009b10 con 0x55dc280a57b0 2024-04-21T08:39:09.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.510+0000 7f1bae565740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dc27fa7240 con 0x55dc280aece0 2024-04-21T08:39:09.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.510+0000 7f1bae565740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dc28005ad0 con 0x55dc280a6160 2024-04-21T08:39:09.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.510+0000 7f1b9a7fc640 1 --2- 172.21.15.43:0/2583329349 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1b900a3e30 0x7f1b900a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.511+0000 7f1ba9f12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280a57b0 0x55dc280a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.511+0000 7f1ba9f12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280a57b0 0x55dc280a5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50134/0 (socket says 172.21.15.43:50134) 2024-04-21T08:39:09.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.511+0000 7f1ba9f12640 1 -- 172.21.15.43:0/1888053355 learned_addr learned my addr 172.21.15.43:0/1888053355 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:09.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.511+0000 7f1ba8f10640 1 --2- 172.21.15.43:0/1888053355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc280aece0 0x55dc280b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.511+0000 7f1ba9f12640 1 --2- 172.21.15.43:0/2583329349 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1b900a3e30 0x7f1b900a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.511+0000 7f1ba9711640 1 --2- 172.21.15.43:0/1888053355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc280a6160 0x55dc280ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.511+0000 7f1b9a7fc640 1 -- 172.21.15.43:0/2583329349 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f1b940d1270 con 0x7f1ba4107270 2024-04-21T08:39:09.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.511+0000 7f1ba8f10640 1 -- 172.21.15.43:0/1888053355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc280a6160 msgr2=0x55dc280ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.511+0000 7f1ba8f10640 1 --2- 172.21.15.43:0/1888053355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc280a6160 0x55dc280ae710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.511+0000 7f1ba8f10640 1 -- 172.21.15.43:0/1888053355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280a57b0 msgr2=0x55dc280a5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.511+0000 7f1ba8f10640 1 --2- 172.21.15.43:0/1888053355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280a57b0 0x55dc280a5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.511+0000 7f1ba8f10640 1 -- 172.21.15.43:0/1888053355 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc27d1c2d0 con 0x55dc280aece0 2024-04-21T08:39:09.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.511+0000 7f1ba9f12640 1 --2- 172.21.15.43:0/1888053355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280a57b0 0x55dc280a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:09.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.511+0000 7f1ba8f10640 1 --2- 172.21.15.43:0/1888053355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc280aece0 0x55dc280b1130 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1b940ce530 tx=0x7f1b9401bb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e033e68f5e17e71 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.511+0000 7f1ba9f12640 1 --2- 172.21.15.43:0/2583329349 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1b900a3e30 0x7f1b900a62f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c02c630 tx=0x7f1b9c0095c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.511+0000 7f1b8f7fe640 1 -- 172.21.15.43:0/1888053355 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9409caa0 con 0x55dc280aece0 2024-04-21T08:39:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.511+0000 7f1b8f7fe640 1 -- 172.21.15.43:0/1888053355 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b9409d3d0 con 0x55dc280aece0 2024-04-21T08:39:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.511+0000 7f1b8f7fe640 1 -- 172.21.15.43:0/1888053355 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9400ce60 con 0x55dc280aece0 2024-04-21T08:39:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.512+0000 7f1bae565740 1 -- 172.21.15.43:0/1888053355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc280aece0 msgr2=0x55dc280b1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.512+0000 7f1bae565740 1 --2- 172.21.15.43:0/1888053355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc280aece0 0x55dc280b1130 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1b940ce530 tx=0x7f1b9401bb30 comp rx=0 tx=0).stop 2024-04-21T08:39:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.512+0000 7f1bae565740 1 -- 172.21.15.43:0/1888053355 shutdown_connections 2024-04-21T08:39:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.512+0000 7f1bae565740 1 --2- 172.21.15.43:0/1888053355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc280aece0 0x55dc280b1130 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.512+0000 7f1bae565740 1 --2- 172.21.15.43:0/1888053355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc280a6160 0x55dc280ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.512+0000 7f1bae565740 1 --2- 172.21.15.43:0/1888053355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280a57b0 0x55dc280a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.512+0000 7f1bae565740 1 -- 172.21.15.43:0/1888053355 >> 172.21.15.43:0/1888053355 conn(0x55dc2809fa60 msgr2=0x55dc2809fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.512+0000 7f1bae565740 1 -- 172.21.15.43:0/1888053355 shutdown_connections 2024-04-21T08:39:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.512+0000 7f1bae565740 1 -- 172.21.15.43:0/1888053355 wait complete. 2024-04-21T08:39:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.513+0000 7f1bae565740 1 -- start start 2024-04-21T08:39:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.513+0000 7f1bae565740 1 Processor -- start 2024-04-21T08:39:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.513+0000 7f1bae565740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc280a57b0 0x55dc280abc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.513+0000 7f1bae565740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280a6160 0x55dc280ac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.513+0000 7f1bae565740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc280aece0 0x55dc280ac700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.513+0000 7f1bae565740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dc280b1c50 con 0x55dc280a6160 2024-04-21T08:39:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.513+0000 7f1bae565740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dc27f92190 con 0x55dc280a57b0 2024-04-21T08:39:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.513+0000 7f1bae565740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dc280b1dd0 con 0x55dc280aece0 2024-04-21T08:39:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.513+0000 7f1bae565740 10 client.0 _ll_register_callbacks cb 0x55dc27f8d8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.513+0000 7f1ba8f10640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280a6160 0x55dc280ac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.513+0000 7f1ba8f10640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280a6160 0x55dc280ac1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50140/0 (socket says 172.21.15.43:50140) 2024-04-21T08:39:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.513+0000 7f1ba8f10640 1 -- 172.21.15.43:0/2774782138 learned_addr learned my addr 172.21.15.43:0/2774782138 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.513+0000 7f1ba9711640 1 --2- 172.21.15.43:0/2774782138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc280a57b0 0x55dc280abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.513+0000 7f1ba9f12640 1 --2- 172.21.15.43:0/2774782138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc280aece0 0x55dc280ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.513+0000 7f1ba8f10640 1 -- 172.21.15.43:0/2774782138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc280aece0 msgr2=0x55dc280ac700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.513+0000 7f1ba8f10640 1 --2- 172.21.15.43:0/2774782138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc280aece0 0x55dc280ac700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.513+0000 7f1ba8f10640 1 -- 172.21.15.43:0/2774782138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc280a57b0 msgr2=0x55dc280abc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.513+0000 7f1ba8f10640 1 --2- 172.21.15.43:0/2774782138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc280a57b0 0x55dc280abc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.513+0000 7f1ba8f10640 1 -- 172.21.15.43:0/2774782138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc27de66b0 con 0x55dc280a6160 2024-04-21T08:39:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.513+0000 7f1ba9711640 1 --2- 172.21.15.43:0/2774782138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc280a57b0 0x55dc280abc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.514+0000 7f1ba8f10640 1 --2- 172.21.15.43:0/2774782138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280a6160 0x55dc280ac1c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1b940ce590 tx=0x7f1b940a59b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64be8cdaa15cfcb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.514+0000 7f1b8cff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.514+0000 7f1b8cff9640 1 -- 172.21.15.43:0/2774782138 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b940a2210 con 0x55dc280a6160 2024-04-21T08:39:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.514+0000 7f1b8cff9640 1 -- 172.21.15.43:0/2774782138 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f1b940a24f0 con 0x55dc280a6160 2024-04-21T08:39:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.514+0000 7f1bae565740 1 -- 172.21.15.43:0/2774782138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b9409d3d0 con 0x55dc280a6160 2024-04-21T08:39:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.514+0000 7f1b8cff9640 1 -- 172.21.15.43:0/2774782138 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1b9409c630 con 0x55dc280a6160 2024-04-21T08:39:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.514+0000 7f1bae565740 10 client.15942 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.514+0000 7f1bae565740 1 -- 172.21.15.43:0/2774782138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc280bdf20 con 0x55dc280a6160 2024-04-21T08:39:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.514+0000 7f1b8cff9640 1 -- 172.21.15.43:0/2774782138 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b9409d3d0 con 0x55dc280a6160 2024-04-21T08:39:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.514+0000 7f1b8cff9640 1 -- 172.21.15.43:0/2774782138 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1b940ce7e0 con 0x55dc280a6160 2024-04-21T08:39:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.514+0000 7f1bae565740 10 client.15942 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.514+0000 7f1bae565740 10 client.15942 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-21T08:39:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.514+0000 7f1bae565740 20 client.15942 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.514+0000 7f1bae565740 1 --2- 172.21.15.43:0/2774782138 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55dc280c00c0 0x55dc280c1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.514+0000 7f1bae565740 4 client.15942 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.514+0000 7f1bae565740 1 -- 172.21.15.43:0/2774782138 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc28017530 con 0x55dc280c00c0 2024-04-21T08:39:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.515+0000 7f1ba9711640 1 --2- 172.21.15.43:0/2774782138 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55dc280c00c0 0x55dc280c1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.515+0000 7f1ba9711640 1 --2- 172.21.15.43:0/2774782138 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55dc280c00c0 0x55dc280c1530 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c2696dd3a5b0518b server_cookie=5d3ac59f861f24d5 in_seq=0 out_seq=0 2024-04-21T08:39:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.515+0000 7f1b8cff9640 10 client.15942 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:39:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.516+0000 7f1b8cff9640 1 -- 172.21.15.43:0/2774782138 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc28017530 con 0x55dc280c00c0 2024-04-21T08:39:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.516+0000 7f1b8cff9640 10 client.15942 handle_command_reply: tid=0 2024-04-21T08:39:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.516+0000 7f1bae565740 1 client.15942 shutdown 2024-04-21T08:39:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.516+0000 7f1bae565740 1 -- 172.21.15.43:0/2774782138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280a6160 msgr2=0x55dc280ac1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.516+0000 7f1bae565740 1 --2- 172.21.15.43:0/2774782138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280a6160 0x55dc280ac1c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1b940ce590 tx=0x7f1b940a59b0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.516+0000 7f1bae565740 1 -- 172.21.15.43:0/2774782138 shutdown_connections 2024-04-21T08:39:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.516+0000 7f1bae565740 1 --2- 172.21.15.43:0/2774782138 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55dc280c00c0 0x55dc280c1530 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.516+0000 7f1bae565740 1 --2- 172.21.15.43:0/2774782138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc280aece0 0x55dc280ac700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.516+0000 7f1bae565740 1 --2- 172.21.15.43:0/2774782138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280a6160 0x55dc280ac1c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.516+0000 7f1bae565740 1 --2- 172.21.15.43:0/2774782138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc280a57b0 0x55dc280abc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.516+0000 7f1bae565740 1 -- 172.21.15.43:0/2774782138 >> 172.21.15.43:0/2774782138 conn(0x55dc2809fa60 msgr2=0x55dc280a1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.516+0000 7f1b8cff9640 0 client.15942 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:39:09.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.516+0000 7f1bae565740 1 -- 172.21.15.43:0/2774782138 shutdown_connections 2024-04-21T08:39:09.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.517+0000 7f1bae565740 1 -- 172.21.15.43:0/2774782138 wait complete. 2024-04-21T08:39:09.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.517+0000 7f1bae565740 20 client.15942 trim_cache size 0 max 16384 2024-04-21T08:39:09.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.535+0000 7f1bae565740 1 Processor -- start 2024-04-21T08:39:09.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.535+0000 7f1bae565740 1 -- start start 2024-04-21T08:39:09.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.535+0000 7f1bae565740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280a57b0 0x55dc280ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.535+0000 7f1bae565740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc280a6160 0x55dc280bb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.535+0000 7f1bae565740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc280aece0 0x55dc280a8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.535+0000 7f1bae565740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dc280bf870 con 0x55dc280a57b0 2024-04-21T08:39:09.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.535+0000 7f1bae565740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dc280bff40 con 0x55dc280aece0 2024-04-21T08:39:09.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.535+0000 7f1bae565740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dc280c0610 con 0x55dc280a6160 2024-04-21T08:39:09.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.535+0000 7f1ba9711640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280a57b0 0x55dc280ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.535+0000 7f1ba9711640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280a57b0 0x55dc280ae3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50144/0 (socket says 172.21.15.43:50144) 2024-04-21T08:39:09.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.535+0000 7f1ba9711640 1 -- 172.21.15.43:0/151254405 learned_addr learned my addr 172.21.15.43:0/151254405 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:09.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.535+0000 7f1ba9711640 1 -- 172.21.15.43:0/151254405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc280a6160 msgr2=0x55dc280bb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.535+0000 7f1ba9711640 1 --2- 172.21.15.43:0/151254405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc280a6160 0x55dc280bb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.535+0000 7f1ba9f12640 1 --2- 172.21.15.43:0/151254405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc280aece0 0x55dc280a8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.535+0000 7f1ba9711640 1 -- 172.21.15.43:0/151254405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc280aece0 msgr2=0x55dc280a8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.535+0000 7f1ba9711640 1 --2- 172.21.15.43:0/151254405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc280aece0 0x55dc280a8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.535+0000 7f1ba9711640 1 -- 172.21.15.43:0/151254405 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc280a4b10 con 0x55dc280a57b0 2024-04-21T08:39:09.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.535+0000 7f1ba9f12640 1 --2- 172.21.15.43:0/151254405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc280aece0 0x55dc280a8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:09.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.536+0000 7f1ba9711640 1 --2- 172.21.15.43:0/151254405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280a57b0 0x55dc280ae3b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x55dc280a5280 tx=0x7f1ba0010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158ca7673452b22d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.536+0000 7f1b8effd640 1 -- 172.21.15.43:0/151254405 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0010e60 con 0x55dc280a57b0 2024-04-21T08:39:09.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.536+0000 7f1b8effd640 1 -- 172.21.15.43:0/151254405 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ba0023030 con 0x55dc280a57b0 2024-04-21T08:39:09.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.536+0000 7f1b8effd640 1 -- 172.21.15.43:0/151254405 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba001f350 con 0x55dc280a57b0 2024-04-21T08:39:09.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.536+0000 7f1bae565740 1 -- 172.21.15.43:0/151254405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280a57b0 msgr2=0x55dc280ae3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.536+0000 7f1bae565740 1 --2- 172.21.15.43:0/151254405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280a57b0 0x55dc280ae3b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x55dc280a5280 tx=0x7f1ba0010800 comp rx=0 tx=0).stop 2024-04-21T08:39:09.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.536+0000 7f1bae565740 1 -- 172.21.15.43:0/151254405 shutdown_connections 2024-04-21T08:39:09.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.536+0000 7f1bae565740 1 --2- 172.21.15.43:0/151254405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc280aece0 0x55dc280a8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.536+0000 7f1bae565740 1 --2- 172.21.15.43:0/151254405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc280a6160 0x55dc280bb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.536+0000 7f1bae565740 1 --2- 172.21.15.43:0/151254405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280a57b0 0x55dc280ae3b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.536+0000 7f1bae565740 1 -- 172.21.15.43:0/151254405 >> 172.21.15.43:0/151254405 conn(0x55dc2809fa60 msgr2=0x55dc2809a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.537+0000 7f1bae565740 1 -- 172.21.15.43:0/151254405 shutdown_connections 2024-04-21T08:39:09.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.537+0000 7f1bae565740 1 -- 172.21.15.43:0/151254405 wait complete. 2024-04-21T08:39:09.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.537+0000 7f1bae565740 1 -- start start 2024-04-21T08:39:09.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.537+0000 7f1bae565740 1 Processor -- start 2024-04-21T08:39:09.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.537+0000 7f1bae565740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc280a57b0 0x55dc280c3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.537+0000 7f1bae565740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc280a6160 0x55dc280c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.537+0000 7f1bae565740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280aece0 0x55dc280c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.537+0000 7f1bae565740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dc280c13b0 con 0x55dc280aece0 2024-04-21T08:39:09.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.537+0000 7f1bae565740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dc280c0ce0 con 0x55dc280a57b0 2024-04-21T08:39:09.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.537+0000 7f1bae565740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dc280c1a80 con 0x55dc280a6160 2024-04-21T08:39:09.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.537+0000 7f1bae565740 10 client.0 _ll_register_callbacks cb 0x7f1b9c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:09.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.537+0000 7f1ba9f12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280aece0 0x55dc280c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.537+0000 7f1ba9f12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280aece0 0x55dc280c7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50158/0 (socket says 172.21.15.43:50158) 2024-04-21T08:39:09.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.537+0000 7f1ba9f12640 1 -- 172.21.15.43:0/2795310696 learned_addr learned my addr 172.21.15.43:0/2795310696 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:09.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.538+0000 7f1ba9711640 1 --2- 172.21.15.43:0/2795310696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc280a57b0 0x55dc280c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.538+0000 7f1ba8f10640 1 --2- 172.21.15.43:0/2795310696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc280a6160 0x55dc280c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.538+0000 7f1ba9f12640 1 -- 172.21.15.43:0/2795310696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc280a6160 msgr2=0x55dc280c3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.538+0000 7f1ba9f12640 1 --2- 172.21.15.43:0/2795310696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc280a6160 0x55dc280c3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.538+0000 7f1ba9f12640 1 -- 172.21.15.43:0/2795310696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc280a57b0 msgr2=0x55dc280c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.538+0000 7f1ba9f12640 1 --2- 172.21.15.43:0/2795310696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc280a57b0 0x55dc280c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.538+0000 7f1ba9f12640 1 -- 172.21.15.43:0/2795310696 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc280c4340 con 0x55dc280aece0 2024-04-21T08:39:09.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.538+0000 7f1ba9f12640 1 --2- 172.21.15.43:0/2795310696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280aece0 0x55dc280c7ce0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c003bd0 tx=0x7f1b9c004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad76a7695bfa201e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.538+0000 7f1b8cff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:09.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.538+0000 7f1b8cff9640 1 -- 172.21.15.43:0/2795310696 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c039070 con 0x55dc280aece0 2024-04-21T08:39:09.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.538+0000 7f1b8cff9640 1 -- 172.21.15.43:0/2795310696 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f1b9c004450 con 0x55dc280aece0 2024-04-21T08:39:09.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.538+0000 7f1bae565740 1 -- 172.21.15.43:0/2795310696 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ba0023030 con 0x55dc280aece0 2024-04-21T08:39:09.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.538+0000 7f1b8cff9640 1 -- 172.21.15.43:0/2795310696 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1b9c004650 con 0x55dc280aece0 2024-04-21T08:39:09.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.538+0000 7f1bae565740 10 client.15948 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:09.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.538+0000 7f1bae565740 1 -- 172.21.15.43:0/2795310696 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc281265e0 con 0x55dc280aece0 2024-04-21T08:39:09.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.538+0000 7f1b8cff9640 1 -- 172.21.15.43:0/2795310696 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ba0023030 con 0x55dc280aece0 2024-04-21T08:39:09.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.539+0000 7f1b8cff9640 1 -- 172.21.15.43:0/2795310696 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1b9c034440 con 0x55dc280aece0 2024-04-21T08:39:09.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.539+0000 7f1bae565740 10 client.15948 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:09.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.539+0000 7f1bae565740 10 client.15948 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-21T08:39:09.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.539+0000 7f1bae565740 20 client.15948 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:09.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.539+0000 7f1bae565740 1 --2- 172.21.15.43:0/2795310696 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55dc2812b280 0x55dc28129f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.539+0000 7f1bae565740 4 client.15948 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:39:09.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.539+0000 7f1bae565740 1 -- 172.21.15.43:0/2795310696 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "status"}) -- 0x55dc280113a0 con 0x55dc2812b280 2024-04-21T08:39:09.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.539+0000 7f1ba9711640 1 --2- 172.21.15.43:0/2795310696 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55dc2812b280 0x55dc28129f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.539+0000 7f1b8cff9640 10 client.15948 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:39:09.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.539+0000 7f1ba9711640 1 --2- 172.21.15.43:0/2795310696 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55dc2812b280 0x55dc28129f90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d219d0f571676f77 server_cookie=e07e640391ba6bec in_seq=0 out_seq=0 2024-04-21T08:39:09.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.540+0000 7f1b8cff9640 1 -- 172.21.15.43:0/2795310696 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55dc280113a0 con 0x55dc2812b280 2024-04-21T08:39:09.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.540+0000 7f1b8cff9640 10 client.15948 handle_command_reply: tid=0 2024-04-21T08:39:09.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.540+0000 7f1bae565740 1 client.15948 shutdown 2024-04-21T08:39:09.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.540+0000 7f1bae565740 1 -- 172.21.15.43:0/2795310696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280aece0 msgr2=0x55dc280c7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.540+0000 7f1bae565740 1 --2- 172.21.15.43:0/2795310696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280aece0 0x55dc280c7ce0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c003bd0 tx=0x7f1b9c004a60 comp rx=0 tx=0).stop 2024-04-21T08:39:09.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.540+0000 7f1bae565740 1 -- 172.21.15.43:0/2795310696 shutdown_connections 2024-04-21T08:39:09.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.540+0000 7f1bae565740 1 --2- 172.21.15.43:0/2795310696 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55dc2812b280 0x55dc28129f90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.540+0000 7f1bae565740 1 --2- 172.21.15.43:0/2795310696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc280aece0 0x55dc280c7ce0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.540+0000 7f1b8cff9640 0 client.15948 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:39:09.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.540+0000 7f1bae565740 1 --2- 172.21.15.43:0/2795310696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc280a6160 0x55dc280c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.540+0000 7f1bae565740 1 --2- 172.21.15.43:0/2795310696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc280a57b0 0x55dc280c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.540+0000 7f1bae565740 1 -- 172.21.15.43:0/2795310696 >> 172.21.15.43:0/2795310696 conn(0x55dc2809fa60 msgr2=0x55dc280abc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.540+0000 7f1bae565740 1 -- 172.21.15.43:0/2795310696 shutdown_connections 2024-04-21T08:39:09.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.540+0000 7f1bae565740 1 -- 172.21.15.43:0/2795310696 wait complete. 2024-04-21T08:39:09.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.540+0000 7f1bae565740 20 client.15948 trim_cache size 0 max 16384 2024-04-21T08:39:09.546 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:39:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "d57a7d98-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:39:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 4, 2024-04-21T08:39:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24412, 2024-04-21T08:39:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-21T08:39:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-21T08:39:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-21T08:39:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 138.64616931200001, 2024-04-21T08:39:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 28, 2024-04-21T08:39:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 95, 2024-04-21T08:39:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 94, 2024-04-21T08:39:09.547 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 203.03595108799999 2024-04-21T08:39:09.547 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:39:09.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.540+0000 7f1bab975640 1 -- 172.21.15.43:0/2583329349 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1b900a3e30 msgr2=0x7f1b900a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.540+0000 7f1bab975640 1 --2- 172.21.15.43:0/2583329349 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1b900a3e30 0x7f1b900a62f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c02c630 tx=0x7f1b9c0095c0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.540+0000 7f1bab975640 1 -- 172.21.15.43:0/2583329349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba4107270 msgr2=0x7f1ba4198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.540+0000 7f1bab975640 1 --2- 172.21.15.43:0/2583329349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba4107270 0x7f1ba4198b90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1b9400e9c0 tx=0x7f1b9400edc0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.540+0000 7f1bab975640 1 -- 172.21.15.43:0/2583329349 shutdown_connections 2024-04-21T08:39:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.540+0000 7f1bab975640 1 --2- 172.21.15.43:0/2583329349 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1b900a3e30 0x7f1b900a62f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.540+0000 7f1bab975640 1 --2- 172.21.15.43:0/2583329349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ba419cf10 0x7f1ba419d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.540+0000 7f1bab975640 1 --2- 172.21.15.43:0/2583329349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba4107270 0x7f1ba4198b90 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.540+0000 7f1bab975640 1 --2- 172.21.15.43:0/2583329349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba406a840 0x7f1ba4198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.541+0000 7f1bab975640 1 -- 172.21.15.43:0/2583329349 >> 172.21.15.43:0/2583329349 conn(0x7f1ba4077eb0 msgr2=0x7f1ba40fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.541+0000 7f1bab975640 1 -- 172.21.15.43:0/2583329349 shutdown_connections 2024-04-21T08:39:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.541+0000 7f1bab975640 1 -- 172.21.15.43:0/2583329349 wait complete. 2024-04-21T08:39:09.550 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-21T08:39:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.654+0000 7f8571584640 1 -- 172.21.15.43:0/2844358577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f856c1031d0 msgr2=0x7f856c1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.655+0000 7f8571584640 1 --2- 172.21.15.43:0/2844358577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f856c1031d0 0x7f856c1035b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f855c009900 tx=0x7f855c02bd60 comp rx=0 tx=0).stop 2024-04-21T08:39:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.655+0000 7f8571584640 1 -- 172.21.15.43:0/2844358577 shutdown_connections 2024-04-21T08:39:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.655+0000 7f8571584640 1 --2- 172.21.15.43:0/2844358577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f856c10c510 0x7f856c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.655+0000 7f8571584640 1 --2- 172.21.15.43:0/2844358577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f856c103af0 0x7f856c10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.655+0000 7f8571584640 1 --2- 172.21.15.43:0/2844358577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f856c1031d0 0x7f856c1035b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.655+0000 7f8571584640 1 -- 172.21.15.43:0/2844358577 >> 172.21.15.43:0/2844358577 conn(0x7f856c077f50 msgr2=0x7f856c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.655+0000 7f8571584640 1 -- 172.21.15.43:0/2844358577 shutdown_connections 2024-04-21T08:39:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.655+0000 7f8571584640 1 -- 172.21.15.43:0/2844358577 wait complete. 2024-04-21T08:39:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.655+0000 7f8571584640 1 Processor -- start 2024-04-21T08:39:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.655+0000 7f8571584640 1 -- start start 2024-04-21T08:39:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.656+0000 7f8571584640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f856c1031d0 0x7f856c19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.656+0000 7f8571584640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f856c103af0 0x7f856c19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.656+0000 7f8571584640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f856c10c510 0x7f856c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.656+0000 7f8571584640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f856c112180 con 0x7f856c10c510 2024-04-21T08:39:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.656+0000 7f8571584640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f856c112000 con 0x7f856c103af0 2024-04-21T08:39:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.656+0000 7f8571584640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f856c112300 con 0x7f856c1031d0 2024-04-21T08:39:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.656+0000 7f856b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f856c10c510 0x7f856c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.656+0000 7f856b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f856c10c510 0x7f856c1985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50174/0 (socket says 172.21.15.43:50174) 2024-04-21T08:39:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.656+0000 7f856b7fe640 1 -- 172.21.15.43:0/3175396667 learned_addr learned my addr 172.21.15.43:0/3175396667 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.656+0000 7f856affd640 1 --2- 172.21.15.43:0/3175396667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f856c1031d0 0x7f856c19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.656+0000 7f856a7fc640 1 --2- 172.21.15.43:0/3175396667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f856c103af0 0x7f856c19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.656+0000 7f856b7fe640 1 -- 172.21.15.43:0/3175396667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f856c1031d0 msgr2=0x7f856c19e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.656+0000 7f856b7fe640 1 --2- 172.21.15.43:0/3175396667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f856c1031d0 0x7f856c19e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.656+0000 7f856b7fe640 1 -- 172.21.15.43:0/3175396667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f856c103af0 msgr2=0x7f856c19e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.656+0000 7f856b7fe640 1 --2- 172.21.15.43:0/3175396667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f856c103af0 0x7f856c19e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.656+0000 7f856b7fe640 1 -- 172.21.15.43:0/3175396667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856c198d20 con 0x7f856c10c510 2024-04-21T08:39:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.656+0000 7f856a7fc640 1 --2- 172.21.15.43:0/3175396667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f856c103af0 0x7f856c19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.656+0000 7f856b7fe640 1 --2- 172.21.15.43:0/3175396667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f856c10c510 0x7f856c1985a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f856400cad0 tx=0x7f856400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.656+0000 7f8553fff640 1 -- 172.21.15.43:0/3175396667 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856400ae20 con 0x7f856c10c510 2024-04-21T08:39:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.657+0000 7f8553fff640 1 -- 172.21.15.43:0/3175396667 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8564005020 con 0x7f856c10c510 2024-04-21T08:39:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.657+0000 7f8571584640 1 -- 172.21.15.43:0/3175396667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f856c199010 con 0x7f856c10c510 2024-04-21T08:39:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.657+0000 7f8553fff640 1 -- 172.21.15.43:0/3175396667 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856400f6b0 con 0x7f856c10c510 2024-04-21T08:39:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.657+0000 7f8571584640 1 -- 172.21.15.43:0/3175396667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f856c1a52b0 con 0x7f856c10c510 2024-04-21T08:39:09.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.659+0000 7f8553fff640 1 -- 172.21.15.43:0/3175396667 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f85640051d0 con 0x7f856c10c510 2024-04-21T08:39:09.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.659+0000 7f857417a740 1 Processor -- start 2024-04-21T08:39:09.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.659+0000 7f857417a740 1 -- start start 2024-04-21T08:39:09.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.659+0000 7f857417a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561401af3690 0x561401af3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.659+0000 7f857417a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401af4040 0x561401afcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.659+0000 7f857417a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561401afd140 0x561401afd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.659+0000 7f857417a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561401a57b10 con 0x561401af4040 2024-04-21T08:39:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.659+0000 7f857417a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5614019f5240 con 0x561401af3690 2024-04-21T08:39:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.659+0000 7f857417a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561401a53ad0 con 0x561401afd140 2024-04-21T08:39:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.659+0000 7f856affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401af4040 0x561401afcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.659+0000 7f856affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401af4040 0x561401afcb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50178/0 (socket says 172.21.15.43:50178) 2024-04-21T08:39:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.659+0000 7f856affd640 1 -- 172.21.15.43:0/3777798811 learned_addr learned my addr 172.21.15.43:0/3777798811 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.660+0000 7f8553fff640 1 --2- 172.21.15.43:0/3175396667 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f85540a3db0 0x7f85540a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.660+0000 7f856a7fc640 1 --2- 172.21.15.43:0/3777798811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561401afd140 0x561401afd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.660+0000 7f856b7fe640 1 --2- 172.21.15.43:0/3175396667 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f85540a3db0 0x7f85540a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.660+0000 7f856a7fc640 1 --2- 172.21.15.43:0/3777798811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561401af3690 0x561401af3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.660+0000 7f856affd640 1 -- 172.21.15.43:0/3777798811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561401afd140 msgr2=0x561401afd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.660+0000 7f856affd640 1 --2- 172.21.15.43:0/3777798811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561401afd140 0x561401afd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.660+0000 7f856affd640 1 -- 172.21.15.43:0/3777798811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561401af3690 msgr2=0x561401af3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.660+0000 7f856affd640 1 --2- 172.21.15.43:0/3777798811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561401af3690 0x561401af3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.660+0000 7f856affd640 1 -- 172.21.15.43:0/3777798811 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56140176a2d0 con 0x561401af4040 2024-04-21T08:39:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.660+0000 7f856a7fc640 1 --2- 172.21.15.43:0/3777798811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561401af3690 0x561401af3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.660+0000 7f8553fff640 1 -- 172.21.15.43:0/3175396667 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f85640d0ec0 con 0x7f856c10c510 2024-04-21T08:39:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.660+0000 7f856affd640 1 --2- 172.21.15.43:0/3777798811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401af4040 0x561401afcb70 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f855c009900 tx=0x7f855c02c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f098baddb3dd261 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.660+0000 7f856b7fe640 1 --2- 172.21.15.43:0/3175396667 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f85540a3db0 0x7f85540a6270 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f85640ce180 tx=0x7f856406b270 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.660+0000 7f85517fa640 1 -- 172.21.15.43:0/3777798811 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c004340 con 0x561401af4040 2024-04-21T08:39:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.660+0000 7f85517fa640 1 -- 172.21.15.43:0/3777798811 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f855c02cdc0 con 0x561401af4040 2024-04-21T08:39:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.660+0000 7f85517fa640 1 -- 172.21.15.43:0/3777798811 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c004920 con 0x561401af4040 2024-04-21T08:39:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.661+0000 7f857417a740 1 -- 172.21.15.43:0/3777798811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401af4040 msgr2=0x561401afcb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.661+0000 7f857417a740 1 --2- 172.21.15.43:0/3777798811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401af4040 0x561401afcb70 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f855c009900 tx=0x7f855c02c780 comp rx=0 tx=0).stop 2024-04-21T08:39:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.661+0000 7f857417a740 1 -- 172.21.15.43:0/3777798811 shutdown_connections 2024-04-21T08:39:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.661+0000 7f857417a740 1 --2- 172.21.15.43:0/3777798811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561401afd140 0x561401afd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.661+0000 7f857417a740 1 --2- 172.21.15.43:0/3777798811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401af4040 0x561401afcb70 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.661+0000 7f857417a740 1 --2- 172.21.15.43:0/3777798811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561401af3690 0x561401af3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.661+0000 7f857417a740 1 -- 172.21.15.43:0/3777798811 >> 172.21.15.43:0/3777798811 conn(0x561401aeda60 msgr2=0x561401aede40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.661+0000 7f857417a740 1 -- 172.21.15.43:0/3777798811 shutdown_connections 2024-04-21T08:39:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.661+0000 7f857417a740 1 -- 172.21.15.43:0/3777798811 wait complete. 2024-04-21T08:39:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.661+0000 7f857417a740 1 -- start start 2024-04-21T08:39:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.661+0000 7f857417a740 1 Processor -- start 2024-04-21T08:39:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.662+0000 7f857417a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561401af3690 0x561401b00bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.662+0000 7f857417a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561401af4040 0x561401b01100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.662+0000 7f857417a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401afd140 0x561401b01640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.662+0000 7f857417a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561401b04020 con 0x561401afd140 2024-04-21T08:39:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.662+0000 7f857417a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5614019e0190 con 0x561401af4040 2024-04-21T08:39:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.662+0000 7f857417a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561401b041a0 con 0x561401af3690 2024-04-21T08:39:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.662+0000 7f857417a740 10 client.0 _ll_register_callbacks cb 0x5614019db8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.662+0000 7f856a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401afd140 0x561401b01640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.662+0000 7f856a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401afd140 0x561401b01640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50192/0 (socket says 172.21.15.43:50192) 2024-04-21T08:39:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.662+0000 7f856a7fc640 1 -- 172.21.15.43:0/2614808766 learned_addr learned my addr 172.21.15.43:0/2614808766 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.662+0000 7f856a7fc640 1 --2- 172.21.15.43:0/2614808766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561401af3690 0x561401b00bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.662+0000 7f856a7fc640 1 -- 172.21.15.43:0/2614808766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561401af3690 msgr2=0x561401b00bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.662+0000 7f856a7fc640 1 --2- 172.21.15.43:0/2614808766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561401af3690 0x561401b00bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.662+0000 7f856a7fc640 1 -- 172.21.15.43:0/2614808766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561401af4040 msgr2=0x561401b01100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.662+0000 7f856a7fc640 1 --2- 172.21.15.43:0/2614808766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561401af4040 0x561401b01100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.662+0000 7f856a7fc640 1 -- 172.21.15.43:0/2614808766 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614018346b0 con 0x561401afd140 2024-04-21T08:39:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.662+0000 7f856a7fc640 1 --2- 172.21.15.43:0/2614808766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401afd140 0x561401b01640 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f856000be10 tx=0x7f856000b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a169d6b2d17819db server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.662+0000 7f854eff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.662+0000 7f854eff5640 1 -- 172.21.15.43:0/2614808766 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85600142d0 con 0x561401afd140 2024-04-21T08:39:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.662+0000 7f854eff5640 1 -- 172.21.15.43:0/2614808766 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f8560004020 con 0x561401afd140 2024-04-21T08:39:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.662+0000 7f857417a740 1 -- 172.21.15.43:0/2614808766 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f855c02cdc0 con 0x561401afd140 2024-04-21T08:39:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.662+0000 7f854eff5640 1 -- 172.21.15.43:0/2614808766 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8560011e50 con 0x561401afd140 2024-04-21T08:39:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.662+0000 7f857417a740 10 client.15960 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.662+0000 7f857417a740 1 -- 172.21.15.43:0/2614808766 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561401b10260 con 0x561401afd140 2024-04-21T08:39:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.663+0000 7f854eff5640 1 -- 172.21.15.43:0/2614808766 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f855c02cdc0 con 0x561401afd140 2024-04-21T08:39:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.663+0000 7f854eff5640 1 -- 172.21.15.43:0/2614808766 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f856001e060 con 0x561401afd140 2024-04-21T08:39:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.663+0000 7f857417a740 10 client.15960 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.663+0000 7f857417a740 10 client.15960 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-21T08:39:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.663+0000 7f857417a740 20 client.15960 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.663+0000 7f857417a740 1 --2- 172.21.15.43:0/2614808766 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x561401b12930 0x561401b13da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.663+0000 7f857417a740 4 client.15960 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:09.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.663+0000 7f857417a740 1 -- 172.21.15.43:0/2614808766 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561401a65530 con 0x561401b12930 2024-04-21T08:39:09.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.663+0000 7f856affd640 1 --2- 172.21.15.43:0/2614808766 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x561401b12930 0x561401b13da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.664+0000 7f856affd640 1 --2- 172.21.15.43:0/2614808766 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x561401b12930 0x561401b13da0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4505bb7b62996ff3 server_cookie=1295be12ff8cff35 in_seq=0 out_seq=0 2024-04-21T08:39:09.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.664+0000 7f854eff5640 10 client.15960 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:39:09.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.664+0000 7f854eff5640 1 -- 172.21.15.43:0/2614808766 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561401a65530 con 0x561401b12930 2024-04-21T08:39:09.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.664+0000 7f854eff5640 10 client.15960 handle_command_reply: tid=0 2024-04-21T08:39:09.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.664+0000 7f857417a740 1 client.15960 shutdown 2024-04-21T08:39:09.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.665+0000 7f857417a740 1 -- 172.21.15.43:0/2614808766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401afd140 msgr2=0x561401b01640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.665+0000 7f857417a740 1 --2- 172.21.15.43:0/2614808766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401afd140 0x561401b01640 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f856000be10 tx=0x7f856000b7f0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.665+0000 7f857417a740 1 -- 172.21.15.43:0/2614808766 shutdown_connections 2024-04-21T08:39:09.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.665+0000 7f857417a740 1 --2- 172.21.15.43:0/2614808766 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x561401b12930 0x561401b13da0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.665+0000 7f857417a740 1 --2- 172.21.15.43:0/2614808766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401afd140 0x561401b01640 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.665+0000 7f857417a740 1 --2- 172.21.15.43:0/2614808766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561401af4040 0x561401b01100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.665+0000 7f854eff5640 0 client.15960 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:39:09.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.665+0000 7f857417a740 1 --2- 172.21.15.43:0/2614808766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561401af3690 0x561401b00bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.665+0000 7f857417a740 1 -- 172.21.15.43:0/2614808766 >> 172.21.15.43:0/2614808766 conn(0x561401aeda60 msgr2=0x561401aefbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.665+0000 7f857417a740 1 -- 172.21.15.43:0/2614808766 shutdown_connections 2024-04-21T08:39:09.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.665+0000 7f857417a740 1 -- 172.21.15.43:0/2614808766 wait complete. 2024-04-21T08:39:09.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.665+0000 7f857417a740 20 client.15960 trim_cache size 0 max 16384 2024-04-21T08:39:09.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.683+0000 7f857417a740 1 Processor -- start 2024-04-21T08:39:09.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.683+0000 7f857417a740 1 -- start start 2024-04-21T08:39:09.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.683+0000 7f857417a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561401af3690 0x561401b032f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.683+0000 7f857417a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561401af4040 0x561401afe530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.683+0000 7f857417a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401afd140 0x561401afbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.683+0000 7f857417a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561401b120e0 con 0x561401afd140 2024-04-21T08:39:09.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.683+0000 7f857417a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561401b127b0 con 0x561401af3690 2024-04-21T08:39:09.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.683+0000 7f857417a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561401b12e80 con 0x561401af4040 2024-04-21T08:39:09.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.683+0000 7f856a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401afd140 0x561401afbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.683+0000 7f856a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401afd140 0x561401afbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50198/0 (socket says 172.21.15.43:50198) 2024-04-21T08:39:09.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.683+0000 7f856a7fc640 1 -- 172.21.15.43:0/3389148633 learned_addr learned my addr 172.21.15.43:0/3389148633 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:09.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.683+0000 7f856a7fc640 1 -- 172.21.15.43:0/3389148633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561401af4040 msgr2=0x561401afe530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.683+0000 7f856a7fc640 1 --2- 172.21.15.43:0/3389148633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561401af4040 0x561401afe530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.683+0000 7f856a7fc640 1 -- 172.21.15.43:0/3389148633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561401af3690 msgr2=0x561401b032f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.683+0000 7f856a7fc640 1 --2- 172.21.15.43:0/3389148633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561401af3690 0x561401b032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.683+0000 7f856a7fc640 1 -- 172.21.15.43:0/3389148633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561401b01fb0 con 0x561401afd140 2024-04-21T08:39:09.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.684+0000 7f856a7fc640 1 --2- 172.21.15.43:0/3389148633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401afd140 0x561401afbaa0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8560014940 tx=0x7f856000b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d25f365d885ed09 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.684+0000 7f8550ff9640 1 -- 172.21.15.43:0/3389148633 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560011ca0 con 0x561401afd140 2024-04-21T08:39:09.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.684+0000 7f8550ff9640 1 -- 172.21.15.43:0/3389148633 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f856000fae0 con 0x561401afd140 2024-04-21T08:39:09.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.684+0000 7f8550ff9640 1 -- 172.21.15.43:0/3389148633 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560012990 con 0x561401afd140 2024-04-21T08:39:09.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.684+0000 7f857417a740 1 -- 172.21.15.43:0/3389148633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401afd140 msgr2=0x561401afbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.684+0000 7f857417a740 1 --2- 172.21.15.43:0/3389148633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401afd140 0x561401afbaa0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8560014940 tx=0x7f856000b3c0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.684+0000 7f857417a740 1 -- 172.21.15.43:0/3389148633 shutdown_connections 2024-04-21T08:39:09.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.684+0000 7f857417a740 1 --2- 172.21.15.43:0/3389148633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401afd140 0x561401afbaa0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.684+0000 7f857417a740 1 --2- 172.21.15.43:0/3389148633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561401af4040 0x561401afe530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.684+0000 7f857417a740 1 --2- 172.21.15.43:0/3389148633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561401af3690 0x561401b032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.684+0000 7f857417a740 1 -- 172.21.15.43:0/3389148633 >> 172.21.15.43:0/3389148633 conn(0x561401aeda60 msgr2=0x561401ae8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.684+0000 7f857417a740 1 -- 172.21.15.43:0/3389148633 shutdown_connections 2024-04-21T08:39:09.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.684+0000 7f857417a740 1 -- 172.21.15.43:0/3389148633 wait complete. 2024-04-21T08:39:09.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.685+0000 7f857417a740 1 -- start start 2024-04-21T08:39:09.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.685+0000 7f857417a740 1 Processor -- start 2024-04-21T08:39:09.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.685+0000 7f857417a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401af3690 0x561401b1c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.685+0000 7f857417a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561401af4040 0x561401afd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.685+0000 7f857417a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561401afd140 0x561401afde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.685+0000 7f857417a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561401b13c20 con 0x561401af3690 2024-04-21T08:39:09.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.685+0000 7f857417a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561401b13550 con 0x561401afd140 2024-04-21T08:39:09.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.685+0000 7f857417a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561401b142f0 con 0x561401af4040 2024-04-21T08:39:09.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.685+0000 7f857417a740 10 client.0 _ll_register_callbacks cb 0x7f856000b6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:09.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.685+0000 7f856a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401af3690 0x561401b1c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.685+0000 7f856a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401af3690 0x561401b1c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50208/0 (socket says 172.21.15.43:50208) 2024-04-21T08:39:09.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.685+0000 7f856a7fc640 1 -- 172.21.15.43:0/3206542864 learned_addr learned my addr 172.21.15.43:0/3206542864 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:09.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.685+0000 7f856affd640 1 --2- 172.21.15.43:0/3206542864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561401af4040 0x561401afd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.685+0000 7f856a7fc640 1 --2- 172.21.15.43:0/3206542864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561401afd140 0x561401afde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.685+0000 7f856a7fc640 1 -- 172.21.15.43:0/3206542864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561401af4040 msgr2=0x561401afd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.685+0000 7f856a7fc640 1 --2- 172.21.15.43:0/3206542864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561401af4040 0x561401afd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.685+0000 7f856a7fc640 1 -- 172.21.15.43:0/3206542864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561401afd140 msgr2=0x561401afde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.685+0000 7f856a7fc640 1 --2- 172.21.15.43:0/3206542864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561401afd140 0x561401afde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.686+0000 7f856a7fc640 1 -- 172.21.15.43:0/3206542864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561401afe6a0 con 0x561401af3690 2024-04-21T08:39:09.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.686+0000 7f856a7fc640 1 --2- 172.21.15.43:0/3206542864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401af3690 0x561401b1c4e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8560004960 tx=0x7f8560005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b44de97dfe5225 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.686+0000 7f854eff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:09.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.686+0000 7f854eff5640 1 -- 172.21.15.43:0/3206542864 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560014280 con 0x561401af3690 2024-04-21T08:39:09.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.686+0000 7f854eff5640 1 -- 172.21.15.43:0/3206542864 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f8560004040 con 0x561401af3690 2024-04-21T08:39:09.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.686+0000 7f857417a740 1 -- 172.21.15.43:0/3206542864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f856000fae0 con 0x561401af3690 2024-04-21T08:39:09.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.686+0000 7f854eff5640 1 -- 172.21.15.43:0/3206542864 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8560004240 con 0x561401af3690 2024-04-21T08:39:09.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.686+0000 7f857417a740 10 client.15966 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:09.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.686+0000 7f857417a740 1 -- 172.21.15.43:0/3206542864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561401b788e0 con 0x561401af3690 2024-04-21T08:39:09.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.686+0000 7f854eff5640 1 -- 172.21.15.43:0/3206542864 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f856000fae0 con 0x561401af3690 2024-04-21T08:39:09.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.686+0000 7f854eff5640 1 -- 172.21.15.43:0/3206542864 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f856001e070 con 0x561401af3690 2024-04-21T08:39:09.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.686+0000 7f857417a740 10 client.15966 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:09.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.686+0000 7f857417a740 10 client.15966 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-21T08:39:09.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.686+0000 7f857417a740 20 client.15966 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:09.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.686+0000 7f857417a740 1 --2- 172.21.15.43:0/3206542864 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x561401b7d230 0x561401b7bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.686+0000 7f857417a740 4 client.15966 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:39:09.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.686+0000 7f857417a740 1 -- 172.21.15.43:0/3206542864 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "status"}) -- 0x561401a5f3a0 con 0x561401b7d230 2024-04-21T08:39:09.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.687+0000 7f856affd640 1 --2- 172.21.15.43:0/3206542864 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x561401b7d230 0x561401b7bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.687+0000 7f856affd640 1 --2- 172.21.15.43:0/3206542864 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x561401b7d230 0x561401b7bf40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b65640f4582eac66 server_cookie=1a9eb243e718b09d in_seq=0 out_seq=0 2024-04-21T08:39:09.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.687+0000 7f854eff5640 10 client.15966 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:39:09.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.687+0000 7f854eff5640 1 -- 172.21.15.43:0/3206542864 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561401a5f3a0 con 0x561401b7d230 2024-04-21T08:39:09.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.687+0000 7f854eff5640 10 client.15966 handle_command_reply: tid=0 2024-04-21T08:39:09.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.687+0000 7f857417a740 1 client.15966 shutdown 2024-04-21T08:39:09.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.688+0000 7f857417a740 1 -- 172.21.15.43:0/3206542864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401af3690 msgr2=0x561401b1c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.688+0000 7f857417a740 1 --2- 172.21.15.43:0/3206542864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401af3690 0x561401b1c4e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8560004960 tx=0x7f8560005970 comp rx=0 tx=0).stop 2024-04-21T08:39:09.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.688+0000 7f857417a740 1 -- 172.21.15.43:0/3206542864 shutdown_connections 2024-04-21T08:39:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.688+0000 7f857417a740 1 --2- 172.21.15.43:0/3206542864 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x561401b7d230 0x561401b7bf40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.688+0000 7f857417a740 1 --2- 172.21.15.43:0/3206542864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561401afd140 0x561401afde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.688+0000 7f857417a740 1 --2- 172.21.15.43:0/3206542864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561401af4040 0x561401afd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.688+0000 7f857417a740 1 --2- 172.21.15.43:0/3206542864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561401af3690 0x561401b1c4e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.688+0000 7f857417a740 1 -- 172.21.15.43:0/3206542864 >> 172.21.15.43:0/3206542864 conn(0x561401aeda60 msgr2=0x561401afbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.688+0000 7f854eff5640 0 client.15966 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:39:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.688+0000 7f857417a740 1 -- 172.21.15.43:0/3206542864 shutdown_connections 2024-04-21T08:39:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.688+0000 7f857417a740 1 -- 172.21.15.43:0/3206542864 wait complete. 2024-04-21T08:39:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.688+0000 7f857417a740 20 client.15966 trim_cache size 0 max 16384 2024-04-21T08:39:09.693 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:39:09.693 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "d57a7d98-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:39:09.694 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 3, 2024-04-21T08:39:09.694 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24425, 2024-04-21T08:39:09.694 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-21T08:39:09.694 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-21T08:39:09.694 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-21T08:39:09.694 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 139.80634244699999, 2024-04-21T08:39:09.694 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 28, 2024-04-21T08:39:09.694 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 95, 2024-04-21T08:39:09.694 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 94, 2024-04-21T08:39:09.694 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 199.970103862 2024-04-21T08:39:09.694 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:39:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.688+0000 7f8571584640 1 -- 172.21.15.43:0/3175396667 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f85540a3db0 msgr2=0x7f85540a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.688+0000 7f8571584640 1 --2- 172.21.15.43:0/3175396667 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f85540a3db0 0x7f85540a6270 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f85640ce180 tx=0x7f856406b270 comp rx=0 tx=0).stop 2024-04-21T08:39:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.688+0000 7f8571584640 1 -- 172.21.15.43:0/3175396667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f856c10c510 msgr2=0x7f856c1985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.688+0000 7f8571584640 1 --2- 172.21.15.43:0/3175396667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f856c10c510 0x7f856c1985a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f856400cad0 tx=0x7f856400cf90 comp rx=0 tx=0).stop 2024-04-21T08:39:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.688+0000 7f8571584640 1 -- 172.21.15.43:0/3175396667 shutdown_connections 2024-04-21T08:39:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.688+0000 7f8571584640 1 --2- 172.21.15.43:0/3175396667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f856c10c510 0x7f856c1985a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.688+0000 7f8571584640 1 --2- 172.21.15.43:0/3175396667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f856c103af0 0x7f856c19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.688+0000 7f8571584640 1 --2- 172.21.15.43:0/3175396667 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f85540a3db0 0x7f85540a6270 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.688+0000 7f8571584640 1 --2- 172.21.15.43:0/3175396667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f856c1031d0 0x7f856c19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.688+0000 7f8571584640 1 -- 172.21.15.43:0/3175396667 >> 172.21.15.43:0/3175396667 conn(0x7f856c077f50 msgr2=0x7f856c0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.689+0000 7f8571584640 1 -- 172.21.15.43:0/3175396667 shutdown_connections 2024-04-21T08:39:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.689+0000 7f8571584640 1 -- 172.21.15.43:0/3175396667 wait complete. 2024-04-21T08:39:09.697 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:09 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1024173076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:09.697 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:09 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3464755153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:39:09.697 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:09 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2739804306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:09.698 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-21T08:39:09.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:09 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1024173076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:09.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:09 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3464755153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:39:09.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:09 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2739804306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:09.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.802+0000 7f4cdc6ba640 1 -- 172.21.15.43:0/4277594927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cd406b1f0 msgr2=0x7f4cd4106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.802+0000 7f4cdc6ba640 1 --2- 172.21.15.43:0/4277594927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cd406b1f0 0x7f4cd4106b90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009900 tx=0x7f4cc402bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.802+0000 7f4cdc6ba640 1 -- 172.21.15.43:0/4277594927 shutdown_connections 2024-04-21T08:39:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.802+0000 7f4cdc6ba640 1 --2- 172.21.15.43:0/4277594927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cd4107270 0x7f4cd410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.802+0000 7f4cdc6ba640 1 --2- 172.21.15.43:0/4277594927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cd406b1f0 0x7f4cd4106b90 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.802+0000 7f4cdc6ba640 1 --2- 172.21.15.43:0/4277594927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd406a840 0x7f4cd406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.802+0000 7f4cdc6ba640 1 -- 172.21.15.43:0/4277594927 >> 172.21.15.43:0/4277594927 conn(0x7f4cd4077eb0 msgr2=0x7f4cd40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.802+0000 7f4cdc6ba640 1 -- 172.21.15.43:0/4277594927 shutdown_connections 2024-04-21T08:39:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.802+0000 7f4cdc6ba640 1 -- 172.21.15.43:0/4277594927 wait complete. 2024-04-21T08:39:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.803+0000 7f4cdc6ba640 1 Processor -- start 2024-04-21T08:39:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.803+0000 7f4cdc6ba640 1 -- start start 2024-04-21T08:39:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.803+0000 7f4cdc6ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd406a840 0x7f4cd4198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.803+0000 7f4cdc6ba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cd4107270 0x7f4cd4198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.803+0000 7f4cdc6ba640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cd419cf50 0x7f4cd419d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.803+0000 7f4cdc6ba640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd4112190 con 0x7f4cd4107270 2024-04-21T08:39:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.803+0000 7f4cdc6ba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd4112010 con 0x7f4cd406a840 2024-04-21T08:39:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.803+0000 7f4cdc6ba640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd4112310 con 0x7f4cd419cf50 2024-04-21T08:39:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.803+0000 7f4cd9c55640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cd4107270 0x7f4cd4198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.803+0000 7f4cd9c55640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cd4107270 0x7f4cd4198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50234/0 (socket says 172.21.15.43:50234) 2024-04-21T08:39:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.803+0000 7f4cd9c55640 1 -- 172.21.15.43:0/1506663379 learned_addr learned my addr 172.21.15.43:0/1506663379 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.803+0000 7f4cda456640 1 --2- 172.21.15.43:0/1506663379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd406a840 0x7f4cd4198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.803+0000 7f4cd9c55640 1 -- 172.21.15.43:0/1506663379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cd419cf50 msgr2=0x7f4cd419d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.803+0000 7f4cd9c55640 1 --2- 172.21.15.43:0/1506663379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cd419cf50 0x7f4cd419d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.803+0000 7f4cd9c55640 1 -- 172.21.15.43:0/1506663379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd406a840 msgr2=0x7f4cd4198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.803+0000 7f4cd9c55640 1 --2- 172.21.15.43:0/1506663379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd406a840 0x7f4cd4198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.803+0000 7f4cd9c55640 1 -- 172.21.15.43:0/1506663379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd419da40 con 0x7f4cd4107270 2024-04-21T08:39:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.803+0000 7f4cda456640 1 --2- 172.21.15.43:0/1506663379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd406a840 0x7f4cd4198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.804+0000 7f4cd9c55640 1 --2- 172.21.15.43:0/1506663379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cd4107270 0x7f4cd4198bd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4036040 tx=0x7f4cc402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.804+0000 7f4ccb7fe640 1 -- 172.21.15.43:0/1506663379 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4043020 con 0x7f4cd4107270 2024-04-21T08:39:09.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.804+0000 7f4ccb7fe640 1 -- 172.21.15.43:0/1506663379 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cc402cbe0 con 0x7f4cd4107270 2024-04-21T08:39:09.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.804+0000 7f4cdc6ba640 1 -- 172.21.15.43:0/1506663379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd419dcd0 con 0x7f4cd4107270 2024-04-21T08:39:09.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.804+0000 7f4ccb7fe640 1 -- 172.21.15.43:0/1506663379 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4003bf0 con 0x7f4cd4107270 2024-04-21T08:39:09.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.804+0000 7f4cdc6ba640 1 -- 172.21.15.43:0/1506663379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd41a54f0 con 0x7f4cd4107270 2024-04-21T08:39:09.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.806+0000 7f4ccb7fe640 1 -- 172.21.15.43:0/1506663379 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f4cc4046020 con 0x7f4cd4107270 2024-04-21T08:39:09.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.806+0000 7f4cdf388740 1 Processor -- start 2024-04-21T08:39:09.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.807+0000 7f4cdf388740 1 -- start start 2024-04-21T08:39:09.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.807+0000 7f4cdf388740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56219ee997b0 0x56219ee99b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.807+0000 7f4cdf388740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee9a160 0x56219eea2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.807+0000 7f4cdf388740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56219eea2ce0 0x56219eea5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.807+0000 7f4cdf388740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56219edfdb10 con 0x56219ee9a160 2024-04-21T08:39:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.807+0000 7f4cdf388740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56219ed9b240 con 0x56219ee997b0 2024-04-21T08:39:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.807+0000 7f4cdf388740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56219edf9ad0 con 0x56219eea2ce0 2024-04-21T08:39:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.807+0000 7f4cda456640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee9a160 0x56219eea2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.807+0000 7f4cda456640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee9a160 0x56219eea2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50240/0 (socket says 172.21.15.43:50240) 2024-04-21T08:39:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.807+0000 7f4cda456640 1 -- 172.21.15.43:0/2246833166 learned_addr learned my addr 172.21.15.43:0/2246833166 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.807+0000 7f4ccb7fe640 1 --2- 172.21.15.43:0/1506663379 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4cc00a3db0 0x7f4cc00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.807+0000 7f4cd9c55640 1 --2- 172.21.15.43:0/2246833166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56219eea2ce0 0x56219eea5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.807+0000 7f4cdac57640 1 --2- 172.21.15.43:0/2246833166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56219ee997b0 0x56219ee99b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.807+0000 7f4cda456640 1 -- 172.21.15.43:0/2246833166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56219eea2ce0 msgr2=0x56219eea5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.807+0000 7f4cda456640 1 --2- 172.21.15.43:0/2246833166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56219eea2ce0 0x56219eea5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.807+0000 7f4cda456640 1 -- 172.21.15.43:0/2246833166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56219ee997b0 msgr2=0x56219ee99b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.807+0000 7f4cda456640 1 --2- 172.21.15.43:0/2246833166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56219ee997b0 0x56219ee99b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.807+0000 7f4cdac57640 1 --2- 172.21.15.43:0/1506663379 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4cc00a3db0 0x7f4cc00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.807+0000 7f4cda456640 1 -- 172.21.15.43:0/2246833166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56219eb102d0 con 0x56219ee9a160 2024-04-21T08:39:09.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.807+0000 7f4ccb7fe640 1 -- 172.21.15.43:0/1506663379 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f4cc40f2fc0 con 0x7f4cd4107270 2024-04-21T08:39:09.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.807+0000 7f4cda456640 1 --2- 172.21.15.43:0/2246833166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee9a160 0x56219eea2710 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000bd40 tx=0x7f4cd000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32173b0eddf07604 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.808+0000 7f4cc8ff9640 1 -- 172.21.15.43:0/2246833166 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0015070 con 0x56219ee9a160 2024-04-21T08:39:09.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.808+0000 7f4cdac57640 1 --2- 172.21.15.43:0/1506663379 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4cc00a3db0 0x7f4cc00a6270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009480 tx=0x7f4ccc005e90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.808+0000 7f4cc8ff9640 1 -- 172.21.15.43:0/2246833166 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cd000be30 con 0x56219ee9a160 2024-04-21T08:39:09.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.808+0000 7f4cc8ff9640 1 -- 172.21.15.43:0/2246833166 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0011900 con 0x56219ee9a160 2024-04-21T08:39:09.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.808+0000 7f4cdf388740 1 -- 172.21.15.43:0/2246833166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee9a160 msgr2=0x56219eea2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.808+0000 7f4cdf388740 1 --2- 172.21.15.43:0/2246833166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee9a160 0x56219eea2710 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000bd40 tx=0x7f4cd000f730 comp rx=0 tx=0).stop 2024-04-21T08:39:09.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.808+0000 7f4cdf388740 1 -- 172.21.15.43:0/2246833166 shutdown_connections 2024-04-21T08:39:09.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.808+0000 7f4cdf388740 1 --2- 172.21.15.43:0/2246833166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56219eea2ce0 0x56219eea5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.808+0000 7f4cdf388740 1 --2- 172.21.15.43:0/2246833166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee9a160 0x56219eea2710 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.808+0000 7f4cdf388740 1 --2- 172.21.15.43:0/2246833166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56219ee997b0 0x56219ee99b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.808+0000 7f4cdf388740 1 -- 172.21.15.43:0/2246833166 >> 172.21.15.43:0/2246833166 conn(0x56219ee93a60 msgr2=0x56219ee93e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.809+0000 7f4cdf388740 1 -- 172.21.15.43:0/2246833166 shutdown_connections 2024-04-21T08:39:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.809+0000 7f4cdf388740 1 -- 172.21.15.43:0/2246833166 wait complete. 2024-04-21T08:39:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.809+0000 7f4cdf388740 1 -- start start 2024-04-21T08:39:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.809+0000 7f4cdf388740 1 Processor -- start 2024-04-21T08:39:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.809+0000 7f4cdf388740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee997b0 0x56219ee9fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.809+0000 7f4cdf388740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56219ee9a160 0x56219eea01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.809+0000 7f4cdf388740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56219eea2ce0 0x56219eea0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.809+0000 7f4cdf388740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56219eea5c50 con 0x56219ee997b0 2024-04-21T08:39:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.809+0000 7f4cdf388740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56219ed86190 con 0x56219eea2ce0 2024-04-21T08:39:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.809+0000 7f4cdf388740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56219eea5dd0 con 0x56219ee9a160 2024-04-21T08:39:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.809+0000 7f4cdf388740 10 client.0 _ll_register_callbacks cb 0x56219ed818b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.809+0000 7f4cda456640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee997b0 0x56219ee9fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.809+0000 7f4cda456640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee997b0 0x56219ee9fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50254/0 (socket says 172.21.15.43:50254) 2024-04-21T08:39:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.809+0000 7f4cda456640 1 -- 172.21.15.43:0/101147800 learned_addr learned my addr 172.21.15.43:0/101147800 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.809+0000 7f4cdac57640 1 --2- 172.21.15.43:0/101147800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56219eea2ce0 0x56219eea0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.810+0000 7f4cd9c55640 1 --2- 172.21.15.43:0/101147800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56219ee9a160 0x56219eea01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.810+0000 7f4cda456640 1 -- 172.21.15.43:0/101147800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56219ee9a160 msgr2=0x56219eea01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.810+0000 7f4cda456640 1 --2- 172.21.15.43:0/101147800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56219ee9a160 0x56219eea01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.810+0000 7f4cda456640 1 -- 172.21.15.43:0/101147800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56219eea2ce0 msgr2=0x56219eea0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.810+0000 7f4cda456640 1 --2- 172.21.15.43:0/101147800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56219eea2ce0 0x56219eea0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.810+0000 7f4cda456640 1 -- 172.21.15.43:0/101147800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56219ebda6b0 con 0x56219ee997b0 2024-04-21T08:39:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.810+0000 7f4cdac57640 1 --2- 172.21.15.43:0/101147800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56219eea2ce0 0x56219eea0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.810+0000 7f4cd9c55640 1 --2- 172.21.15.43:0/101147800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56219ee9a160 0x56219eea01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.810+0000 7f4cda456640 1 --2- 172.21.15.43:0/101147800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee997b0 0x56219ee9fc90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00058b0 tx=0x7f4cd001aec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db716700f680badf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.810+0000 7f4cbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.810+0000 7f4cbdffb640 1 -- 172.21.15.43:0/101147800 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0015070 con 0x56219ee997b0 2024-04-21T08:39:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.810+0000 7f4cbdffb640 1 -- 172.21.15.43:0/101147800 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f4cd0005410 con 0x56219ee997b0 2024-04-21T08:39:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.810+0000 7f4cdf388740 1 -- 172.21.15.43:0/101147800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cd000be30 con 0x56219ee997b0 2024-04-21T08:39:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.810+0000 7f4cbdffb640 1 -- 172.21.15.43:0/101147800 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4cd0005610 con 0x56219ee997b0 2024-04-21T08:39:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.810+0000 7f4cdf388740 10 client.15978 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.810+0000 7f4cdf388740 1 -- 172.21.15.43:0/101147800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56219eeb1fa0 con 0x56219ee997b0 2024-04-21T08:39:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.810+0000 7f4cbdffb640 1 -- 172.21.15.43:0/101147800 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cd000be30 con 0x56219ee997b0 2024-04-21T08:39:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.811+0000 7f4cbdffb640 1 -- 172.21.15.43:0/101147800 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4cd0004a60 con 0x56219ee997b0 2024-04-21T08:39:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.811+0000 7f4cdf388740 10 client.15978 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.811+0000 7f4cdf388740 10 client.15978 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T08:39:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.811+0000 7f4cdf388740 20 client.15978 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.811+0000 7f4cdf388740 1 --2- 172.21.15.43:0/101147800 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x56219eeb4440 0x56219eeb58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.811+0000 7f4cdf388740 4 client.15978 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.811+0000 7f4cdf388740 1 -- 172.21.15.43:0/101147800 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56219ee0b530 con 0x56219eeb4440 2024-04-21T08:39:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.811+0000 7f4cd9c55640 1 --2- 172.21.15.43:0/101147800 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x56219eeb4440 0x56219eeb58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.811+0000 7f4cd9c55640 1 --2- 172.21.15.43:0/101147800 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x56219eeb4440 0x56219eeb58b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d8b18c877e3f646e server_cookie=535fe891adbd0aad in_seq=0 out_seq=0 2024-04-21T08:39:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.811+0000 7f4cbdffb640 10 client.15978 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:39:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.812+0000 7f4cbdffb640 1 -- 172.21.15.43:0/101147800 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56219ee0b530 con 0x56219eeb4440 2024-04-21T08:39:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.812+0000 7f4cbdffb640 10 client.15978 handle_command_reply: tid=0 2024-04-21T08:39:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.812+0000 7f4cdf388740 1 client.15978 shutdown 2024-04-21T08:39:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.812+0000 7f4cdf388740 1 -- 172.21.15.43:0/101147800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee997b0 msgr2=0x56219ee9fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.812+0000 7f4cdf388740 1 --2- 172.21.15.43:0/101147800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee997b0 0x56219ee9fc90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00058b0 tx=0x7f4cd001aec0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.812+0000 7f4cdf388740 1 -- 172.21.15.43:0/101147800 shutdown_connections 2024-04-21T08:39:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.812+0000 7f4cdf388740 1 --2- 172.21.15.43:0/101147800 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x56219eeb4440 0x56219eeb58b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.812+0000 7f4cdf388740 1 --2- 172.21.15.43:0/101147800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56219eea2ce0 0x56219eea0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.812+0000 7f4cdf388740 1 --2- 172.21.15.43:0/101147800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56219ee9a160 0x56219eea01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.812+0000 7f4cbdffb640 0 client.15978 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:39:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.812+0000 7f4cdf388740 1 --2- 172.21.15.43:0/101147800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee997b0 0x56219ee9fc90 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.812+0000 7f4cdf388740 1 -- 172.21.15.43:0/101147800 >> 172.21.15.43:0/101147800 conn(0x56219ee93a60 msgr2=0x56219ee95bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.812+0000 7f4cdf388740 1 -- 172.21.15.43:0/101147800 shutdown_connections 2024-04-21T08:39:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.812+0000 7f4cdf388740 1 -- 172.21.15.43:0/101147800 wait complete. 2024-04-21T08:39:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.812+0000 7f4cdf388740 20 client.15978 trim_cache size 0 max 16384 2024-04-21T08:39:09.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.831+0000 7f4cdf388740 1 Processor -- start 2024-04-21T08:39:09.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.831+0000 7f4cdf388740 1 -- start start 2024-04-21T08:39:09.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.831+0000 7f4cdf388740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee93a60 0x56219eea23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.831+0000 7f4cdf388740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56219ee997b0 0x56219eeb3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.831+0000 7f4cdf388740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56219ee9a160 0x56219eea0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.831+0000 7f4cdf388740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56219eeb3bf0 con 0x56219ee93a60 2024-04-21T08:39:09.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.831+0000 7f4cdf388740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56219eeb42c0 con 0x56219ee9a160 2024-04-21T08:39:09.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.831+0000 7f4cdf388740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56219eeb4990 con 0x56219ee997b0 2024-04-21T08:39:09.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.831+0000 7f4cda456640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee93a60 0x56219eea23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.831+0000 7f4cda456640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee93a60 0x56219eea23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50270/0 (socket says 172.21.15.43:50270) 2024-04-21T08:39:09.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.831+0000 7f4cda456640 1 -- 172.21.15.43:0/108114051 learned_addr learned my addr 172.21.15.43:0/108114051 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:09.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.831+0000 7f4cdac57640 1 --2- 172.21.15.43:0/108114051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56219ee9a160 0x56219eea0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.831+0000 7f4cd9c55640 1 --2- 172.21.15.43:0/108114051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56219ee997b0 0x56219eeb3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.831+0000 7f4cda456640 1 -- 172.21.15.43:0/108114051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56219ee997b0 msgr2=0x56219eeb3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.831+0000 7f4cda456640 1 --2- 172.21.15.43:0/108114051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56219ee997b0 0x56219eeb3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.831+0000 7f4cda456640 1 -- 172.21.15.43:0/108114051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56219ee9a160 msgr2=0x56219eea0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.831+0000 7f4cda456640 1 --2- 172.21.15.43:0/108114051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56219ee9a160 0x56219eea0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.831+0000 7f4cda456640 1 -- 172.21.15.43:0/108114051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56219eeb5510 con 0x56219ee93a60 2024-04-21T08:39:09.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.831+0000 7f4cd9c55640 1 --2- 172.21.15.43:0/108114051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56219ee997b0 0x56219eeb3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:09.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.831+0000 7f4cdac57640 1 --2- 172.21.15.43:0/108114051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56219ee9a160 0x56219eea0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:09.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.831+0000 7f4cda456640 1 --2- 172.21.15.43:0/108114051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee93a60 0x56219eea23c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0000c00 tx=0x7f4cd0005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484279dee7993f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.832+0000 7f4cbffff640 1 -- 172.21.15.43:0/108114051 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0015070 con 0x56219ee93a60 2024-04-21T08:39:09.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.832+0000 7f4cbffff640 1 -- 172.21.15.43:0/108114051 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cd001ace0 con 0x56219ee93a60 2024-04-21T08:39:09.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.832+0000 7f4cbffff640 1 -- 172.21.15.43:0/108114051 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0004990 con 0x56219ee93a60 2024-04-21T08:39:09.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.832+0000 7f4cdf388740 1 -- 172.21.15.43:0/108114051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee93a60 msgr2=0x56219eea23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.832+0000 7f4cdf388740 1 --2- 172.21.15.43:0/108114051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee93a60 0x56219eea23c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0000c00 tx=0x7f4cd0005b90 comp rx=0 tx=0).stop 2024-04-21T08:39:09.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.832+0000 7f4cdf388740 1 -- 172.21.15.43:0/108114051 shutdown_connections 2024-04-21T08:39:09.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.832+0000 7f4cdf388740 1 --2- 172.21.15.43:0/108114051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56219ee9a160 0x56219eea0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.832+0000 7f4cdf388740 1 --2- 172.21.15.43:0/108114051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56219ee997b0 0x56219eeb3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.832+0000 7f4cdf388740 1 --2- 172.21.15.43:0/108114051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee93a60 0x56219eea23c0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.832+0000 7f4cdf388740 1 -- 172.21.15.43:0/108114051 >> 172.21.15.43:0/108114051 conn(0x56219ee9a8b0 msgr2=0x56219ee8e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.832+0000 7f4cdf388740 1 -- 172.21.15.43:0/108114051 shutdown_connections 2024-04-21T08:39:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.832+0000 7f4cdf388740 1 -- 172.21.15.43:0/108114051 wait complete. 2024-04-21T08:39:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.833+0000 7f4cdf388740 1 -- start start 2024-04-21T08:39:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.833+0000 7f4cdf388740 1 Processor -- start 2024-04-21T08:39:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.833+0000 7f4cdf388740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56219ee93a60 0x56219eebe020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.833+0000 7f4cdf388740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee997b0 0x56219eeb7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.833+0000 7f4cdf388740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56219ee9a160 0x56219eeb79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.833+0000 7f4cdf388740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56219eeb5730 con 0x56219ee997b0 2024-04-21T08:39:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.833+0000 7f4cdf388740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56219eeb5060 con 0x56219ee93a60 2024-04-21T08:39:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.833+0000 7f4cdf388740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56219eeb5e00 con 0x56219ee9a160 2024-04-21T08:39:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.833+0000 7f4cdf388740 10 client.0 _ll_register_callbacks cb 0x7f4cd000bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.833+0000 7f4cd9c55640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee997b0 0x56219eeb7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.833+0000 7f4cd9c55640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee997b0 0x56219eeb7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50286/0 (socket says 172.21.15.43:50286) 2024-04-21T08:39:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.833+0000 7f4cd9c55640 1 -- 172.21.15.43:0/3830277679 learned_addr learned my addr 172.21.15.43:0/3830277679 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.833+0000 7f4cda456640 1 --2- 172.21.15.43:0/3830277679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56219ee93a60 0x56219eebe020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.833+0000 7f4cdac57640 1 --2- 172.21.15.43:0/3830277679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56219ee9a160 0x56219eeb79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.833+0000 7f4cd9c55640 1 -- 172.21.15.43:0/3830277679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56219ee9a160 msgr2=0x56219eeb79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.833+0000 7f4cd9c55640 1 --2- 172.21.15.43:0/3830277679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56219ee9a160 0x56219eeb79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.833+0000 7f4cd9c55640 1 -- 172.21.15.43:0/3830277679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56219ee93a60 msgr2=0x56219eebe020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.833+0000 7f4cd9c55640 1 --2- 172.21.15.43:0/3830277679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56219ee93a60 0x56219eebe020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.833+0000 7f4cd9c55640 1 -- 172.21.15.43:0/3830277679 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56219eebbb30 con 0x56219ee997b0 2024-04-21T08:39:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.833+0000 7f4cd9c55640 1 --2- 172.21.15.43:0/3830277679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee997b0 0x56219eeb7370 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40bb4a0 tx=0x7f4cc40bad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9522e431f6ddf0f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.833+0000 7f4cbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.834+0000 7f4cbdffb640 1 -- 172.21.15.43:0/3830277679 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc40d7030 con 0x56219ee997b0 2024-04-21T08:39:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.834+0000 7f4cbdffb640 1 -- 172.21.15.43:0/3830277679 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f4cc40bcdd0 con 0x56219ee997b0 2024-04-21T08:39:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.834+0000 7f4cdf388740 1 -- 172.21.15.43:0/3830277679 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cd001ace0 con 0x56219ee997b0 2024-04-21T08:39:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.834+0000 7f4cbdffb640 1 -- 172.21.15.43:0/3830277679 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4cc40c2e80 con 0x56219ee997b0 2024-04-21T08:39:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.834+0000 7f4cdf388740 10 client.15984 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.834+0000 7f4cdf388740 1 -- 172.21.15.43:0/3830277679 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56219ef1a540 con 0x56219ee997b0 2024-04-21T08:39:09.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.834+0000 7f4cbdffb640 1 -- 172.21.15.43:0/3830277679 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cd001ace0 con 0x56219ee997b0 2024-04-21T08:39:09.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.834+0000 7f4cbdffb640 1 -- 172.21.15.43:0/3830277679 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4cc40da020 con 0x56219ee997b0 2024-04-21T08:39:09.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.834+0000 7f4cdf388740 10 client.15984 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:09.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.834+0000 7f4cdf388740 10 client.15984 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T08:39:09.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.834+0000 7f4cdf388740 20 client.15984 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:09.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.834+0000 7f4cdf388740 1 --2- 172.21.15.43:0/3830277679 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x56219eea2ce0 0x56219ef1dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.834+0000 7f4cdf388740 4 client.15984 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:39:09.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.834+0000 7f4cdf388740 1 -- 172.21.15.43:0/3830277679 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "status"}) -- 0x56219ee053a0 con 0x56219eea2ce0 2024-04-21T08:39:09.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.834+0000 7f4cda456640 1 --2- 172.21.15.43:0/3830277679 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x56219eea2ce0 0x56219ef1dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.835+0000 7f4cda456640 1 --2- 172.21.15.43:0/3830277679 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x56219eea2ce0 0x56219ef1dee0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=935d4125d14da9b0 server_cookie=acff51262856a5a0 in_seq=0 out_seq=0 2024-04-21T08:39:09.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.835+0000 7f4cbdffb640 10 client.15984 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:39:09.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.835+0000 7f4cbdffb640 1 -- 172.21.15.43:0/3830277679 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56219ee053a0 con 0x56219eea2ce0 2024-04-21T08:39:09.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.835+0000 7f4cbdffb640 10 client.15984 handle_command_reply: tid=0 2024-04-21T08:39:09.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.835+0000 7f4cdf388740 1 client.15984 shutdown 2024-04-21T08:39:09.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.835+0000 7f4cdf388740 1 -- 172.21.15.43:0/3830277679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee997b0 msgr2=0x56219eeb7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.835+0000 7f4cdf388740 1 --2- 172.21.15.43:0/3830277679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee997b0 0x56219eeb7370 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40bb4a0 tx=0x7f4cc40bad70 comp rx=0 tx=0).stop 2024-04-21T08:39:09.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.835+0000 7f4cdf388740 1 -- 172.21.15.43:0/3830277679 shutdown_connections 2024-04-21T08:39:09.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.835+0000 7f4cdf388740 1 --2- 172.21.15.43:0/3830277679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56219ee9a160 0x56219eeb79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.835+0000 7f4cdf388740 1 --2- 172.21.15.43:0/3830277679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56219ee997b0 0x56219eeb7370 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.835+0000 7f4cdf388740 1 --2- 172.21.15.43:0/3830277679 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x56219eea2ce0 0x56219ef1dee0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.835+0000 7f4cdf388740 1 --2- 172.21.15.43:0/3830277679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56219ee93a60 0x56219eebe020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.835+0000 7f4cdf388740 1 -- 172.21.15.43:0/3830277679 >> 172.21.15.43:0/3830277679 conn(0x56219ee9a8b0 msgr2=0x56219eeb3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.835+0000 7f4cbdffb640 0 client.15984 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:39:09.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.835+0000 7f4cdf388740 1 -- 172.21.15.43:0/3830277679 shutdown_connections 2024-04-21T08:39:09.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.835+0000 7f4cdf388740 1 -- 172.21.15.43:0/3830277679 wait complete. 2024-04-21T08:39:09.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.835+0000 7f4cdf388740 20 client.15984 trim_cache size 0 max 16384 2024-04-21T08:39:09.841 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:39:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "d57a7d98-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:39:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 2, 2024-04-21T08:39:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24487, 2024-04-21T08:39:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-21T08:39:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-21T08:39:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-21T08:39:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 140.77520935800001, 2024-04-21T08:39:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 28, 2024-04-21T08:39:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 95, 2024-04-21T08:39:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 94, 2024-04-21T08:39:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 179.56335841200001 2024-04-21T08:39:09.842 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:39:09.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.836+0000 7f4cdc6ba640 1 -- 172.21.15.43:0/1506663379 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4cc00a3db0 msgr2=0x7f4cc00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.836+0000 7f4cdc6ba640 1 --2- 172.21.15.43:0/1506663379 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4cc00a3db0 0x7f4cc00a6270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009480 tx=0x7f4ccc005e90 comp rx=0 tx=0).stop 2024-04-21T08:39:09.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.836+0000 7f4cdc6ba640 1 -- 172.21.15.43:0/1506663379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cd4107270 msgr2=0x7f4cd4198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.836+0000 7f4cdc6ba640 1 --2- 172.21.15.43:0/1506663379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cd4107270 0x7f4cd4198bd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4036040 tx=0x7f4cc402c7e0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.836+0000 7f4cdc6ba640 1 -- 172.21.15.43:0/1506663379 shutdown_connections 2024-04-21T08:39:09.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.836+0000 7f4cdc6ba640 1 --2- 172.21.15.43:0/1506663379 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4cc00a3db0 0x7f4cc00a6270 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.836+0000 7f4cdc6ba640 1 --2- 172.21.15.43:0/1506663379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cd419cf50 0x7f4cd419d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.836+0000 7f4cdc6ba640 1 --2- 172.21.15.43:0/1506663379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cd4107270 0x7f4cd4198bd0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.836+0000 7f4cdc6ba640 1 --2- 172.21.15.43:0/1506663379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd406a840 0x7f4cd4198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.836+0000 7f4cdc6ba640 1 -- 172.21.15.43:0/1506663379 >> 172.21.15.43:0/1506663379 conn(0x7f4cd4077eb0 msgr2=0x7f4cd4100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.836+0000 7f4cdc6ba640 1 -- 172.21.15.43:0/1506663379 shutdown_connections 2024-04-21T08:39:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.836+0000 7f4cdc6ba640 1 -- 172.21.15.43:0/1506663379 wait complete. 2024-04-21T08:39:09.846 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-21T08:39:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.950+0000 7fda5ca81640 1 -- 172.21.15.43:0/3855477821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda5806b1f0 msgr2=0x7fda58106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.950+0000 7fda5ca81640 1 --2- 172.21.15.43:0/3855477821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda5806b1f0 0x7fda58106b90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fda40009900 tx=0x7fda4002bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.950+0000 7fda5ca81640 1 -- 172.21.15.43:0/3855477821 shutdown_connections 2024-04-21T08:39:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.950+0000 7fda5ca81640 1 --2- 172.21.15.43:0/3855477821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda58107270 0x7fda5810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.950+0000 7fda5ca81640 1 --2- 172.21.15.43:0/3855477821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda5806b1f0 0x7fda58106b90 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.950+0000 7fda5ca81640 1 --2- 172.21.15.43:0/3855477821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda5806a840 0x7fda5806ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.950+0000 7fda5ca81640 1 -- 172.21.15.43:0/3855477821 >> 172.21.15.43:0/3855477821 conn(0x7fda58077eb0 msgr2=0x7fda580782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.950+0000 7fda5ca81640 1 -- 172.21.15.43:0/3855477821 shutdown_connections 2024-04-21T08:39:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.950+0000 7fda5ca81640 1 -- 172.21.15.43:0/3855477821 wait complete. 2024-04-21T08:39:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.950+0000 7fda5ca81640 1 Processor -- start 2024-04-21T08:39:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.950+0000 7fda5ca81640 1 -- start start 2024-04-21T08:39:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.951+0000 7fda5ca81640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda5806a840 0x7fda581986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.951+0000 7fda5ca81640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda58107270 0x7fda58198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.951+0000 7fda5ca81640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda5819cf90 0x7fda5819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.951+0000 7fda5ca81640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fda581121b0 con 0x7fda58107270 2024-04-21T08:39:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.951+0000 7fda5ca81640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fda58112030 con 0x7fda5819cf90 2024-04-21T08:39:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.951+0000 7fda5ca81640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fda58112330 con 0x7fda5806a840 2024-04-21T08:39:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.951+0000 7fda4ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda58107270 0x7fda58198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.951+0000 7fda4ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda58107270 0x7fda58198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50304/0 (socket says 172.21.15.43:50304) 2024-04-21T08:39:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.951+0000 7fda4ffff640 1 -- 172.21.15.43:0/1757564646 learned_addr learned my addr 172.21.15.43:0/1757564646 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.951+0000 7fda5659c640 1 --2- 172.21.15.43:0/1757564646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda5806a840 0x7fda581986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.951+0000 7fda56d9d640 1 --2- 172.21.15.43:0/1757564646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda5819cf90 0x7fda5819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.951+0000 7fda4ffff640 1 -- 172.21.15.43:0/1757564646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda5806a840 msgr2=0x7fda581986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.951+0000 7fda4ffff640 1 --2- 172.21.15.43:0/1757564646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda5806a840 0x7fda581986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.951+0000 7fda4ffff640 1 -- 172.21.15.43:0/1757564646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda5819cf90 msgr2=0x7fda5819d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.951+0000 7fda4ffff640 1 --2- 172.21.15.43:0/1757564646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda5819cf90 0x7fda5819d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.951+0000 7fda4ffff640 1 -- 172.21.15.43:0/1757564646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda5819da80 con 0x7fda58107270 2024-04-21T08:39:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.951+0000 7fda5659c640 1 --2- 172.21.15.43:0/1757564646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda5806a840 0x7fda581986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.951+0000 7fda4ffff640 1 --2- 172.21.15.43:0/1757564646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda58107270 0x7fda58198c10 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fda40036040 tx=0x7fda400045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.951+0000 7fda4f7fe640 1 -- 172.21.15.43:0/1757564646 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda40043020 con 0x7fda58107270 2024-04-21T08:39:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.951+0000 7fda4f7fe640 1 -- 172.21.15.43:0/1757564646 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda4002cb70 con 0x7fda58107270 2024-04-21T08:39:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.951+0000 7fda4f7fe640 1 -- 172.21.15.43:0/1757564646 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda40003bf0 con 0x7fda58107270 2024-04-21T08:39:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.952+0000 7fda5ca81640 1 -- 172.21.15.43:0/1757564646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda5819dd10 con 0x7fda58107270 2024-04-21T08:39:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.952+0000 7fda5ca81640 1 -- 172.21.15.43:0/1757564646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda581a54e0 con 0x7fda58107270 2024-04-21T08:39:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.954+0000 7fda4f7fe640 1 -- 172.21.15.43:0/1757564646 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fda40046020 con 0x7fda58107270 2024-04-21T08:39:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.954+0000 7fda5f674740 1 Processor -- start 2024-04-21T08:39:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.954+0000 7fda5f674740 1 -- start start 2024-04-21T08:39:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.955+0000 7fda5f674740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55751bffe7b0 0x55751bffeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.955+0000 7fda5f674740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55751bfff160 0x55751c007710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.955+0000 7fda5f674740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751c007ce0 0x55751c00a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.955+0000 7fda5f674740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55751bf62b10 con 0x55751c007ce0 2024-04-21T08:39:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.955+0000 7fda5f674740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55751bf00240 con 0x55751bffe7b0 2024-04-21T08:39:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.955+0000 7fda5f674740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55751bf5ead0 con 0x55751bfff160 2024-04-21T08:39:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.955+0000 7fda4f7fe640 1 --2- 172.21.15.43:0/1757564646 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fda440a3db0 0x7fda440a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.955+0000 7fda4ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751c007ce0 0x55751c00a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.955+0000 7fda4ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751c007ce0 0x55751c00a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50316/0 (socket says 172.21.15.43:50316) 2024-04-21T08:39:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.955+0000 7fda4ffff640 1 -- 172.21.15.43:0/1254014181 learned_addr learned my addr 172.21.15.43:0/1254014181 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.955+0000 7fda5659c640 1 --2- 172.21.15.43:0/1254014181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55751bfff160 0x55751c007710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.955+0000 7fda56d9d640 1 --2- 172.21.15.43:0/1757564646 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fda440a3db0 0x7fda440a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.955+0000 7fda56d9d640 1 --2- 172.21.15.43:0/1254014181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55751bffe7b0 0x55751bffeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.955+0000 7fda4f7fe640 1 -- 172.21.15.43:0/1757564646 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fda400f2ed0 con 0x7fda58107270 2024-04-21T08:39:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.955+0000 7fda4ffff640 1 -- 172.21.15.43:0/1254014181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55751bfff160 msgr2=0x55751c007710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.955+0000 7fda4ffff640 1 --2- 172.21.15.43:0/1254014181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55751bfff160 0x55751c007710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.955+0000 7fda4ffff640 1 -- 172.21.15.43:0/1254014181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55751bffe7b0 msgr2=0x55751bffeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.955+0000 7fda4ffff640 1 --2- 172.21.15.43:0/1254014181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55751bffe7b0 0x55751bffeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.955+0000 7fda4ffff640 1 -- 172.21.15.43:0/1254014181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55751bc752d0 con 0x55751c007ce0 2024-04-21T08:39:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.955+0000 7fda4ffff640 1 --2- 172.21.15.43:0/1254014181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751c007ce0 0x55751c00a130 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fda400f0330 tx=0x7fda400bae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59061e55dd78445 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.955+0000 7fda56d9d640 1 --2- 172.21.15.43:0/1757564646 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fda440a3db0 0x7fda440a6270 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fda50012980 tx=0x7fda5000abc0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.956+0000 7fda4cff9640 1 -- 172.21.15.43:0/1254014181 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda400df0b0 con 0x55751c007ce0 2024-04-21T08:39:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.956+0000 7fda4cff9640 1 -- 172.21.15.43:0/1254014181 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda400bf140 con 0x55751c007ce0 2024-04-21T08:39:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.956+0000 7fda4cff9640 1 -- 172.21.15.43:0/1254014181 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda400054d0 con 0x55751c007ce0 2024-04-21T08:39:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.956+0000 7fda5f674740 1 -- 172.21.15.43:0/1254014181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751c007ce0 msgr2=0x55751c00a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.956+0000 7fda5f674740 1 --2- 172.21.15.43:0/1254014181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751c007ce0 0x55751c00a130 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fda400f0330 tx=0x7fda400bae10 comp rx=0 tx=0).stop 2024-04-21T08:39:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.956+0000 7fda5f674740 1 -- 172.21.15.43:0/1254014181 shutdown_connections 2024-04-21T08:39:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.956+0000 7fda5f674740 1 --2- 172.21.15.43:0/1254014181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751c007ce0 0x55751c00a130 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.956+0000 7fda5f674740 1 --2- 172.21.15.43:0/1254014181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55751bfff160 0x55751c007710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.956+0000 7fda5f674740 1 --2- 172.21.15.43:0/1254014181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55751bffe7b0 0x55751bffeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.956+0000 7fda5f674740 1 -- 172.21.15.43:0/1254014181 >> 172.21.15.43:0/1254014181 conn(0x55751bff8a60 msgr2=0x55751bff8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.956+0000 7fda5f674740 1 -- 172.21.15.43:0/1254014181 shutdown_connections 2024-04-21T08:39:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.957+0000 7fda5f674740 1 -- 172.21.15.43:0/1254014181 wait complete. 2024-04-21T08:39:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.957+0000 7fda5f674740 1 -- start start 2024-04-21T08:39:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.957+0000 7fda5f674740 1 Processor -- start 2024-04-21T08:39:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.957+0000 7fda5f674740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751bffe7b0 0x55751c004cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.957+0000 7fda5f674740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55751bfff160 0x55751c0051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.957+0000 7fda5f674740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55751c007ce0 0x55751c005730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.957+0000 7fda5f674740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55751c00aa10 con 0x55751bffe7b0 2024-04-21T08:39:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.957+0000 7fda5f674740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55751beeb190 con 0x55751bfff160 2024-04-21T08:39:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.957+0000 7fda5f674740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55751c00ab90 con 0x55751c007ce0 2024-04-21T08:39:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.957+0000 7fda5659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751bffe7b0 0x55751c004cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.957+0000 7fda5659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751bffe7b0 0x55751c004cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50332/0 (socket says 172.21.15.43:50332) 2024-04-21T08:39:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.957+0000 7fda5659c640 1 -- 172.21.15.43:0/3780505533 learned_addr learned my addr 172.21.15.43:0/3780505533 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.957+0000 7fda5f674740 10 client.0 _ll_register_callbacks cb 0x55751bee68b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.957+0000 7fda4ffff640 1 --2- 172.21.15.43:0/3780505533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55751bfff160 0x55751c0051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.957+0000 7fda56d9d640 1 --2- 172.21.15.43:0/3780505533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55751c007ce0 0x55751c005730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.958+0000 7fda5659c640 1 -- 172.21.15.43:0/3780505533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55751c007ce0 msgr2=0x55751c005730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.958+0000 7fda5659c640 1 --2- 172.21.15.43:0/3780505533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55751c007ce0 0x55751c005730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.958+0000 7fda5659c640 1 -- 172.21.15.43:0/3780505533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55751bfff160 msgr2=0x55751c0051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.958+0000 7fda5659c640 1 --2- 172.21.15.43:0/3780505533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55751bfff160 0x55751c0051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.958+0000 7fda5659c640 1 -- 172.21.15.43:0/3780505533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55751bd3f6b0 con 0x55751bffe7b0 2024-04-21T08:39:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.958+0000 7fda5659c640 1 --2- 172.21.15.43:0/3780505533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751bffe7b0 0x55751c004cb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fda48012700 tx=0x7fda48012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449f3fbed3a8358d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.958+0000 7fda3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.958+0000 7fda3dffb640 1 -- 172.21.15.43:0/3780505533 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda48011bb0 con 0x55751bffe7b0 2024-04-21T08:39:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.958+0000 7fda3dffb640 1 -- 172.21.15.43:0/3780505533 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fda48011d50 con 0x55751bffe7b0 2024-04-21T08:39:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.958+0000 7fda5f674740 1 -- 172.21.15.43:0/3780505533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda400bf140 con 0x55751bffe7b0 2024-04-21T08:39:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.958+0000 7fda3dffb640 1 -- 172.21.15.43:0/3780505533 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fda4800f460 con 0x55751bffe7b0 2024-04-21T08:39:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.958+0000 7fda5f674740 10 client.15996 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.958+0000 7fda5f674740 1 -- 172.21.15.43:0/3780505533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55751c016d50 con 0x55751bffe7b0 2024-04-21T08:39:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.958+0000 7fda3dffb640 1 -- 172.21.15.43:0/3780505533 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda400bf140 con 0x55751bffe7b0 2024-04-21T08:39:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.958+0000 7fda3dffb640 1 -- 172.21.15.43:0/3780505533 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fda4801c750 con 0x55751bffe7b0 2024-04-21T08:39:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.958+0000 7fda5f674740 10 client.15996 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.958+0000 7fda5f674740 10 client.15996 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-21T08:39:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.959+0000 7fda5f674740 20 client.15996 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.959+0000 7fda5f674740 1 --2- 172.21.15.43:0/3780505533 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55751c019430 0x55751c01a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.959+0000 7fda5f674740 4 client.15996 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.959+0000 7fda5f674740 1 -- 172.21.15.43:0/3780505533 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55751bf70530 con 0x55751c019430 2024-04-21T08:39:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.959+0000 7fda4ffff640 1 --2- 172.21.15.43:0/3780505533 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55751c019430 0x55751c01a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.959+0000 7fda3dffb640 10 client.15996 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:39:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.959+0000 7fda4ffff640 1 --2- 172.21.15.43:0/3780505533 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55751c019430 0x55751c01a8a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4967a3ef40da01fa server_cookie=724c2212ff734ffa in_seq=0 out_seq=0 2024-04-21T08:39:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.960+0000 7fda3dffb640 1 -- 172.21.15.43:0/3780505533 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55751bf70530 con 0x55751c019430 2024-04-21T08:39:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.960+0000 7fda3dffb640 10 client.15996 handle_command_reply: tid=0 2024-04-21T08:39:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.960+0000 7fda5f674740 1 client.15996 shutdown 2024-04-21T08:39:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.960+0000 7fda5f674740 1 -- 172.21.15.43:0/3780505533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751bffe7b0 msgr2=0x55751c004cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.960+0000 7fda5f674740 1 --2- 172.21.15.43:0/3780505533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751bffe7b0 0x55751c004cb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fda48012700 tx=0x7fda48012bc0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.960+0000 7fda5f674740 1 -- 172.21.15.43:0/3780505533 shutdown_connections 2024-04-21T08:39:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.960+0000 7fda5f674740 1 --2- 172.21.15.43:0/3780505533 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55751c019430 0x55751c01a8a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.960+0000 7fda5f674740 1 --2- 172.21.15.43:0/3780505533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55751c007ce0 0x55751c005730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.960+0000 7fda5f674740 1 --2- 172.21.15.43:0/3780505533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55751bfff160 0x55751c0051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.960+0000 7fda3dffb640 0 client.15996 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:39:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.960+0000 7fda5f674740 1 --2- 172.21.15.43:0/3780505533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751bffe7b0 0x55751c004cb0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.960+0000 7fda5f674740 1 -- 172.21.15.43:0/3780505533 >> 172.21.15.43:0/3780505533 conn(0x55751bff8a60 msgr2=0x55751bffabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.960+0000 7fda5f674740 1 -- 172.21.15.43:0/3780505533 shutdown_connections 2024-04-21T08:39:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.960+0000 7fda5f674740 1 -- 172.21.15.43:0/3780505533 wait complete. 2024-04-21T08:39:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.960+0000 7fda5f674740 20 client.15996 trim_cache size 0 max 16384 2024-04-21T08:39:09.967 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:09 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1024173076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:09.967 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:09 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3464755153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:39:09.968 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:09 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2739804306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:09.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.978+0000 7fda5f674740 1 Processor -- start 2024-04-21T08:39:09.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.979+0000 7fda5f674740 1 -- start start 2024-04-21T08:39:09.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.979+0000 7fda5f674740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55751bffe7b0 0x55751c0073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.979+0000 7fda5f674740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55751bfff160 0x55751bff3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.979+0000 7fda5f674740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751c007ce0 0x55751c005550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.979+0000 7fda5f674740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55751c018be0 con 0x55751c007ce0 2024-04-21T08:39:09.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.979+0000 7fda5f674740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55751c0192b0 con 0x55751bfff160 2024-04-21T08:39:09.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.979+0000 7fda5f674740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55751c019980 con 0x55751bffe7b0 2024-04-21T08:39:09.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.979+0000 7fda56d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751c007ce0 0x55751c005550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.979+0000 7fda56d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751c007ce0 0x55751c005550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50342/0 (socket says 172.21.15.43:50342) 2024-04-21T08:39:09.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.979+0000 7fda56d9d640 1 -- 172.21.15.43:0/596416870 learned_addr learned my addr 172.21.15.43:0/596416870 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:09.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.979+0000 7fda5659c640 1 --2- 172.21.15.43:0/596416870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55751bffe7b0 0x55751c0073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.979+0000 7fda56d9d640 1 -- 172.21.15.43:0/596416870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55751bffe7b0 msgr2=0x55751c0073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.979+0000 7fda56d9d640 1 --2- 172.21.15.43:0/596416870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55751bffe7b0 0x55751c0073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.979+0000 7fda56d9d640 1 -- 172.21.15.43:0/596416870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55751bfff160 msgr2=0x55751bff3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.979+0000 7fda56d9d640 1 --2- 172.21.15.43:0/596416870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55751bfff160 0x55751bff3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.979+0000 7fda56d9d640 1 -- 172.21.15.43:0/596416870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55751c0008a0 con 0x55751c007ce0 2024-04-21T08:39:09.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.979+0000 7fda5659c640 1 --2- 172.21.15.43:0/596416870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55751bffe7b0 0x55751c0073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:09.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.979+0000 7fda56d9d640 1 --2- 172.21.15.43:0/596416870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751c007ce0 0x55751c005550 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fda50012fc0 tx=0x7fda5001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2ea22c767b0a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.979+0000 7fda3ffff640 1 -- 172.21.15.43:0/596416870 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda50004b20 con 0x55751c007ce0 2024-04-21T08:39:09.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.980+0000 7fda3ffff640 1 -- 172.21.15.43:0/596416870 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda5001dd70 con 0x55751c007ce0 2024-04-21T08:39:09.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.980+0000 7fda3ffff640 1 -- 172.21.15.43:0/596416870 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda500056d0 con 0x55751c007ce0 2024-04-21T08:39:09.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.980+0000 7fda5f674740 1 -- 172.21.15.43:0/596416870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751c007ce0 msgr2=0x55751c005550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.980+0000 7fda5f674740 1 --2- 172.21.15.43:0/596416870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751c007ce0 0x55751c005550 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fda50012fc0 tx=0x7fda5001a000 comp rx=0 tx=0).stop 2024-04-21T08:39:09.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.980+0000 7fda5f674740 1 -- 172.21.15.43:0/596416870 shutdown_connections 2024-04-21T08:39:09.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.980+0000 7fda5f674740 1 --2- 172.21.15.43:0/596416870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751c007ce0 0x55751c005550 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.980+0000 7fda5f674740 1 --2- 172.21.15.43:0/596416870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55751bfff160 0x55751bff3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.980+0000 7fda5f674740 1 --2- 172.21.15.43:0/596416870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55751bffe7b0 0x55751c0073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.980+0000 7fda5f674740 1 -- 172.21.15.43:0/596416870 >> 172.21.15.43:0/596416870 conn(0x55751bff8a60 msgr2=0x55751bff8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.980+0000 7fda5f674740 1 -- 172.21.15.43:0/596416870 shutdown_connections 2024-04-21T08:39:09.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.980+0000 7fda5f674740 1 -- 172.21.15.43:0/596416870 wait complete. 2024-04-21T08:39:09.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.980+0000 7fda5f674740 1 -- start start 2024-04-21T08:39:09.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.980+0000 7fda5f674740 1 Processor -- start 2024-04-21T08:39:09.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.981+0000 7fda5f674740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55751bffe7b0 0x55751c022f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.981+0000 7fda5f674740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751bfff160 0x55751c0016f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.981+0000 7fda5f674740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55751c007ce0 0x55751c001c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.981+0000 7fda5f674740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55751c01a720 con 0x55751bfff160 2024-04-21T08:39:09.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.981+0000 7fda5f674740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55751c01a050 con 0x55751c007ce0 2024-04-21T08:39:09.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.981+0000 7fda5f674740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55751c01adf0 con 0x55751bffe7b0 2024-04-21T08:39:09.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.981+0000 7fda5f674740 10 client.0 _ll_register_callbacks cb 0x7fda48009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:09.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.981+0000 7fda4ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751bfff160 0x55751c0016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.981+0000 7fda5659c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55751bffe7b0 0x55751c022f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.981+0000 7fda4ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751bfff160 0x55751c0016f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50350/0 (socket says 172.21.15.43:50350) 2024-04-21T08:39:09.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.981+0000 7fda5659c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55751bffe7b0 0x55751c022f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.43:41882/0 (socket says 172.21.15.43:41882) 2024-04-21T08:39:09.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.981+0000 7fda4ffff640 1 -- 172.21.15.43:0/3570943243 learned_addr learned my addr 172.21.15.43:0/3570943243 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:09.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.981+0000 7fda56d9d640 1 --2- 172.21.15.43:0/3570943243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55751c007ce0 0x55751c001c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.981+0000 7fda4ffff640 1 -- 172.21.15.43:0/3570943243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55751bffe7b0 msgr2=0x55751c022f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.981+0000 7fda4ffff640 1 --2- 172.21.15.43:0/3570943243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55751bffe7b0 0x55751c022f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.981+0000 7fda4ffff640 1 -- 172.21.15.43:0/3570943243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55751c007ce0 msgr2=0x55751c001c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:09.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.981+0000 7fda4ffff640 1 --2- 172.21.15.43:0/3570943243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55751c007ce0 0x55751c001c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.981+0000 7fda4ffff640 1 -- 172.21.15.43:0/3570943243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55751c002410 con 0x55751bfff160 2024-04-21T08:39:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.981+0000 7fda4ffff640 1 --2- 172.21.15.43:0/3570943243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751bfff160 0x55751c0016f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fda40005c20 tx=0x7fda400036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8021d69e5aa0882 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.981+0000 7fda3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.981+0000 7fda3dffb640 1 -- 172.21.15.43:0/3570943243 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda400bf140 con 0x55751bfff160 2024-04-21T08:39:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.981+0000 7fda3dffb640 1 -- 172.21.15.43:0/3570943243 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fda400054d0 con 0x55751bfff160 2024-04-21T08:39:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.982+0000 7fda5f674740 1 -- 172.21.15.43:0/3570943243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda5001dd70 con 0x55751bfff160 2024-04-21T08:39:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.982+0000 7fda3dffb640 1 -- 172.21.15.43:0/3570943243 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fda4003c350 con 0x55751bfff160 2024-04-21T08:39:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.982+0000 7fda5f674740 10 client.16002 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.982+0000 7fda5f674740 1 -- 172.21.15.43:0/3570943243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55751c07f3e0 con 0x55751bfff160 2024-04-21T08:39:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.982+0000 7fda3dffb640 1 -- 172.21.15.43:0/3570943243 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda5001dd70 con 0x55751bfff160 2024-04-21T08:39:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.982+0000 7fda3dffb640 1 -- 172.21.15.43:0/3570943243 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fda400bb0b0 con 0x55751bfff160 2024-04-21T08:39:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.982+0000 7fda5f674740 10 client.16002 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.982+0000 7fda5f674740 10 client.16002 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-21T08:39:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.982+0000 7fda5f674740 20 client.16002 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.982+0000 7fda5f674740 1 --2- 172.21.15.43:0/3570943243 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55751c083d10 0x55751c082a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.982+0000 7fda5f674740 4 client.16002 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:39:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.982+0000 7fda5f674740 1 -- 172.21.15.43:0/3570943243 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "status"}) -- 0x55751bf6a3a0 con 0x55751c083d10 2024-04-21T08:39:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.982+0000 7fda5659c640 1 --2- 172.21.15.43:0/3570943243 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55751c083d10 0x55751c082a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.983+0000 7fda5659c640 1 --2- 172.21.15.43:0/3570943243 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55751c083d10 0x55751c082a20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d43d3e10b07a0e1c server_cookie=9bf0b73ec15ca0b6 in_seq=0 out_seq=0 2024-04-21T08:39:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.983+0000 7fda3dffb640 10 client.16002 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:39:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.983+0000 7fda3dffb640 1 -- 172.21.15.43:0/3570943243 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55751bf6a3a0 con 0x55751c083d10 2024-04-21T08:39:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.983+0000 7fda3dffb640 10 client.16002 handle_command_reply: tid=0 2024-04-21T08:39:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.983+0000 7fda5f674740 1 client.16002 shutdown 2024-04-21T08:39:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.983+0000 7fda5f674740 1 -- 172.21.15.43:0/3570943243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751bfff160 msgr2=0x55751c0016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.983+0000 7fda5f674740 1 --2- 172.21.15.43:0/3570943243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751bfff160 0x55751c0016f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fda40005c20 tx=0x7fda400036f0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.983+0000 7fda5f674740 1 -- 172.21.15.43:0/3570943243 shutdown_connections 2024-04-21T08:39:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.983+0000 7fda5f674740 1 --2- 172.21.15.43:0/3570943243 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55751c083d10 0x55751c082a20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.983+0000 7fda5f674740 1 --2- 172.21.15.43:0/3570943243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55751c007ce0 0x55751c001c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.983+0000 7fda3dffb640 0 client.16002 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:39:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.983+0000 7fda5f674740 1 --2- 172.21.15.43:0/3570943243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55751bfff160 0x55751c0016f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.983+0000 7fda5f674740 1 --2- 172.21.15.43:0/3570943243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55751bffe7b0 0x55751c022f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.983+0000 7fda5f674740 1 -- 172.21.15.43:0/3570943243 >> 172.21.15.43:0/3570943243 conn(0x55751bff8a60 msgr2=0x55751bff3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.983+0000 7fda5f674740 1 -- 172.21.15.43:0/3570943243 shutdown_connections 2024-04-21T08:39:09.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.983+0000 7fda5f674740 1 -- 172.21.15.43:0/3570943243 wait complete. 2024-04-21T08:39:09.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.983+0000 7fda5f674740 20 client.16002 trim_cache size 0 max 16384 2024-04-21T08:39:09.989 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:39:09.989 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "d57a7d98-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:39:09.989 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 1, 2024-04-21T08:39:09.989 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24491, 2024-04-21T08:39:09.989 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-21T08:39:09.990 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-21T08:39:09.990 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-21T08:39:09.990 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 141.92984740200001, 2024-04-21T08:39:09.990 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 28, 2024-04-21T08:39:09.990 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 95, 2024-04-21T08:39:09.990 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 94, 2024-04-21T08:39:09.990 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 182.692029531 2024-04-21T08:39:09.990 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:39:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.984+0000 7fda5ca81640 1 -- 172.21.15.43:0/1757564646 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fda440a3db0 msgr2=0x7fda440a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.984+0000 7fda5ca81640 1 --2- 172.21.15.43:0/1757564646 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fda440a3db0 0x7fda440a6270 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fda50012980 tx=0x7fda5000abc0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.984+0000 7fda5ca81640 1 -- 172.21.15.43:0/1757564646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda58107270 msgr2=0x7fda58198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.984+0000 7fda5ca81640 1 --2- 172.21.15.43:0/1757564646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda58107270 0x7fda58198c10 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fda40036040 tx=0x7fda400045e0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.984+0000 7fda5ca81640 1 -- 172.21.15.43:0/1757564646 shutdown_connections 2024-04-21T08:39:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.984+0000 7fda5ca81640 1 --2- 172.21.15.43:0/1757564646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda5819cf90 0x7fda5819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.984+0000 7fda5ca81640 1 --2- 172.21.15.43:0/1757564646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda58107270 0x7fda58198c10 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.984+0000 7fda5ca81640 1 --2- 172.21.15.43:0/1757564646 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fda440a3db0 0x7fda440a6270 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.984+0000 7fda5ca81640 1 --2- 172.21.15.43:0/1757564646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda5806a840 0x7fda581986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.984+0000 7fda5ca81640 1 -- 172.21.15.43:0/1757564646 >> 172.21.15.43:0/1757564646 conn(0x7fda58077eb0 msgr2=0x7fda58108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:09.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.984+0000 7fda5ca81640 1 -- 172.21.15.43:0/1757564646 shutdown_connections 2024-04-21T08:39:09.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:09.984+0000 7fda5ca81640 1 -- 172.21.15.43:0/1757564646 wait complete. 2024-04-21T08:39:09.994 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-21T08:39:10.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.098+0000 7fd2e1b8f640 1 -- 172.21.15.43:0/3189669964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2dc06b1f0 msgr2=0x7fd2dc106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:10.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.098+0000 7fd2e1b8f640 1 --2- 172.21.15.43:0/3189669964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2dc06b1f0 0x7fd2dc106b90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0009900 tx=0x7fd2d002bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:10.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.098+0000 7fd2e1b8f640 1 -- 172.21.15.43:0/3189669964 shutdown_connections 2024-04-21T08:39:10.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.098+0000 7fd2e1b8f640 1 --2- 172.21.15.43:0/3189669964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2dc107270 0x7fd2dc10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.098+0000 7fd2e1b8f640 1 --2- 172.21.15.43:0/3189669964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2dc06b1f0 0x7fd2dc106b90 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.098+0000 7fd2e1b8f640 1 --2- 172.21.15.43:0/3189669964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2dc06a840 0x7fd2dc06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.098+0000 7fd2e1b8f640 1 -- 172.21.15.43:0/3189669964 >> 172.21.15.43:0/3189669964 conn(0x7fd2dc077eb0 msgr2=0x7fd2dc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:10.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.098+0000 7fd2e1b8f640 1 -- 172.21.15.43:0/3189669964 shutdown_connections 2024-04-21T08:39:10.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.098+0000 7fd2e1b8f640 1 -- 172.21.15.43:0/3189669964 wait complete. 2024-04-21T08:39:10.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.099+0000 7fd2e1b8f640 1 Processor -- start 2024-04-21T08:39:10.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.099+0000 7fd2e1b8f640 1 -- start start 2024-04-21T08:39:10.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.099+0000 7fd2e1b8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2dc06a840 0x7fd2dc198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:10.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.099+0000 7fd2e1b8f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2dc107270 0x7fd2dc198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:10.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.099+0000 7fd2e1b8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2dc19cff0 0x7fd2dc19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:10.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.099+0000 7fd2e1b8f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd2dc1121b0 con 0x7fd2dc19cff0 2024-04-21T08:39:10.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.099+0000 7fd2e1b8f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd2dc112030 con 0x7fd2dc06a840 2024-04-21T08:39:10.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.099+0000 7fd2e1b8f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd2dc112330 con 0x7fd2dc107270 2024-04-21T08:39:10.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.099+0000 7fd2dbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2dc19cff0 0x7fd2dc19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:10.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.099+0000 7fd2dbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2dc19cff0 0x7fd2dc19d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50366/0 (socket says 172.21.15.43:50366) 2024-04-21T08:39:10.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.099+0000 7fd2dbfff640 1 -- 172.21.15.43:0/1619964770 learned_addr learned my addr 172.21.15.43:0/1619964770 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:10.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.100+0000 7fd2daffd640 1 --2- 172.21.15.43:0/1619964770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2dc107270 0x7fd2dc198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:10.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.100+0000 7fd2db7fe640 1 --2- 172.21.15.43:0/1619964770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2dc06a840 0x7fd2dc198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:10.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.100+0000 7fd2dbfff640 1 -- 172.21.15.43:0/1619964770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2dc107270 msgr2=0x7fd2dc198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:10.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.100+0000 7fd2dbfff640 1 --2- 172.21.15.43:0/1619964770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2dc107270 0x7fd2dc198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.100+0000 7fd2dbfff640 1 -- 172.21.15.43:0/1619964770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2dc06a840 msgr2=0x7fd2dc198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:10.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.100+0000 7fd2dbfff640 1 --2- 172.21.15.43:0/1619964770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2dc06a840 0x7fd2dc198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.100+0000 7fd2dbfff640 1 -- 172.21.15.43:0/1619964770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2dc19dae0 con 0x7fd2dc19cff0 2024-04-21T08:39:10.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.100+0000 7fd2db7fe640 1 --2- 172.21.15.43:0/1619964770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2dc06a840 0x7fd2dc198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:10.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.100+0000 7fd2dbfff640 1 --2- 172.21.15.43:0/1619964770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2dc19cff0 0x7fd2dc19d4a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009fd0 tx=0x7fd2d400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:10.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.100+0000 7fd2d8ff9640 1 -- 172.21.15.43:0/1619964770 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4017070 con 0x7fd2dc19cff0 2024-04-21T08:39:10.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.100+0000 7fd2e1b8f640 1 -- 172.21.15.43:0/1619964770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2dc19ddd0 con 0x7fd2dc19cff0 2024-04-21T08:39:10.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.100+0000 7fd2d8ff9640 1 -- 172.21.15.43:0/1619964770 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2d4003e20 con 0x7fd2dc19cff0 2024-04-21T08:39:10.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.100+0000 7fd2d8ff9640 1 -- 172.21.15.43:0/1619964770 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4010460 con 0x7fd2dc19cff0 2024-04-21T08:39:10.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.100+0000 7fd2e1b8f640 1 -- 172.21.15.43:0/1619964770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2dc1a5540 con 0x7fd2dc19cff0 2024-04-21T08:39:10.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.103+0000 7fd2d8ff9640 1 -- 172.21.15.43:0/1619964770 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fd2d4003970 con 0x7fd2dc19cff0 2024-04-21T08:39:10.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.103+0000 7fd2e477c740 1 Processor -- start 2024-04-21T08:39:10.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.103+0000 7fd2e477c740 1 -- start start 2024-04-21T08:39:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.103+0000 7fd2e477c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec6e7b0 0x55afeec6eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.103+0000 7fd2e477c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afeec6f160 0x55afeec77710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.103+0000 7fd2e477c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55afeec77ce0 0x55afeec7a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.103+0000 7fd2e477c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55afeebd2b10 con 0x55afeec6e7b0 2024-04-21T08:39:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.103+0000 7fd2e477c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55afeeb70240 con 0x55afeec6f160 2024-04-21T08:39:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.103+0000 7fd2e477c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55afeebcead0 con 0x55afeec77ce0 2024-04-21T08:39:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.103+0000 7fd2daffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec6e7b0 0x55afeec6eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.103+0000 7fd2daffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec6e7b0 0x55afeec6eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50368/0 (socket says 172.21.15.43:50368) 2024-04-21T08:39:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.103+0000 7fd2daffd640 1 -- 172.21.15.43:0/1039141634 learned_addr learned my addr 172.21.15.43:0/1039141634 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.104+0000 7fd2db7fe640 1 --2- 172.21.15.43:0/1039141634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afeec6f160 0x55afeec77710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.104+0000 7fd2daffd640 1 -- 172.21.15.43:0/1039141634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55afeec77ce0 msgr2=0x55afeec7a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.104+0000 7fd2daffd640 1 --2- 172.21.15.43:0/1039141634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55afeec77ce0 0x55afeec7a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.104+0000 7fd2d8ff9640 1 --2- 172.21.15.43:0/1619964770 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd2c40a3db0 0x7fd2c40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.104+0000 7fd2daffd640 1 -- 172.21.15.43:0/1039141634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afeec6f160 msgr2=0x55afeec77710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.104+0000 7fd2daffd640 1 --2- 172.21.15.43:0/1039141634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afeec6f160 0x55afeec77710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.104+0000 7fd2daffd640 1 -- 172.21.15.43:0/1039141634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afee8e52d0 con 0x55afeec6e7b0 2024-04-21T08:39:10.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.104+0000 7fd2db7fe640 1 --2- 172.21.15.43:0/1039141634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afeec6f160 0x55afeec77710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:10.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.104+0000 7fd2dbfff640 1 --2- 172.21.15.43:0/1619964770 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd2c40a3db0 0x7fd2c40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:10.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.104+0000 7fd2d8ff9640 1 -- 172.21.15.43:0/1619964770 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd2d40d0b60 con 0x7fd2dc19cff0 2024-04-21T08:39:10.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.104+0000 7fd2daffd640 1 --2- 172.21.15.43:0/1039141634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec6e7b0 0x55afeec6eb90 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc06a420 tx=0x7fd2d002c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f17739f3d2e0b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:10.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.104+0000 7fd2c1ffb640 1 -- 172.21.15.43:0/1039141634 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d00042f0 con 0x55afeec6e7b0 2024-04-21T08:39:10.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.104+0000 7fd2c1ffb640 1 -- 172.21.15.43:0/1039141634 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2d0045070 con 0x55afeec6e7b0 2024-04-21T08:39:10.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.104+0000 7fd2c1ffb640 1 -- 172.21.15.43:0/1039141634 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d00046c0 con 0x55afeec6e7b0 2024-04-21T08:39:10.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.105+0000 7fd2e477c740 1 -- 172.21.15.43:0/1039141634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec6e7b0 msgr2=0x55afeec6eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:10.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.105+0000 7fd2dbfff640 1 --2- 172.21.15.43:0/1619964770 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd2c40a3db0 0x7fd2c40a6270 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40cde80 tx=0x7fd2d406e000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:10.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.105+0000 7fd2e477c740 1 --2- 172.21.15.43:0/1039141634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec6e7b0 0x55afeec6eb90 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc06a420 tx=0x7fd2d002c5d0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.105+0000 7fd2e477c740 1 -- 172.21.15.43:0/1039141634 shutdown_connections 2024-04-21T08:39:10.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.105+0000 7fd2e477c740 1 --2- 172.21.15.43:0/1039141634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55afeec77ce0 0x55afeec7a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.105+0000 7fd2e477c740 1 --2- 172.21.15.43:0/1039141634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afeec6f160 0x55afeec77710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.105+0000 7fd2e477c740 1 --2- 172.21.15.43:0/1039141634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec6e7b0 0x55afeec6eb90 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.105+0000 7fd2e477c740 1 -- 172.21.15.43:0/1039141634 >> 172.21.15.43:0/1039141634 conn(0x55afeec68a60 msgr2=0x55afeec68e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:10.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.105+0000 7fd2e477c740 1 -- 172.21.15.43:0/1039141634 shutdown_connections 2024-04-21T08:39:10.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.105+0000 7fd2e477c740 1 -- 172.21.15.43:0/1039141634 wait complete. 2024-04-21T08:39:10.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.105+0000 7fd2e477c740 1 -- start start 2024-04-21T08:39:10.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.106+0000 7fd2e477c740 1 Processor -- start 2024-04-21T08:39:10.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.106+0000 7fd2e477c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afeec6e7b0 0x55afeec74c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:10.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.106+0000 7fd2e477c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55afeec6f160 0x55afeec751b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:10.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.106+0000 7fd2e477c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec77ce0 0x55afeec756f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:10.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.106+0000 7fd2e477c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55afeec7abf0 con 0x55afeec77ce0 2024-04-21T08:39:10.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.106+0000 7fd2e477c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55afeeb5b190 con 0x55afeec6e7b0 2024-04-21T08:39:10.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.106+0000 7fd2e477c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55afeec7ad70 con 0x55afeec6f160 2024-04-21T08:39:10.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.106+0000 7fd2e477c740 10 client.0 _ll_register_callbacks cb 0x55afeeb568b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:10.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.106+0000 7fd2daffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec77ce0 0x55afeec756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:10.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.106+0000 7fd2daffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec77ce0 0x55afeec756f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50370/0 (socket says 172.21.15.43:50370) 2024-04-21T08:39:10.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.106+0000 7fd2daffd640 1 -- 172.21.15.43:0/3406701036 learned_addr learned my addr 172.21.15.43:0/3406701036 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:10.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.106+0000 7fd2daffd640 1 --2- 172.21.15.43:0/3406701036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afeec6e7b0 0x55afeec74c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:10.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.106+0000 7fd2db7fe640 1 --2- 172.21.15.43:0/3406701036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55afeec6f160 0x55afeec751b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:10.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.106+0000 7fd2daffd640 1 -- 172.21.15.43:0/3406701036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55afeec6f160 msgr2=0x55afeec751b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:10.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.106+0000 7fd2daffd640 1 --2- 172.21.15.43:0/3406701036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55afeec6f160 0x55afeec751b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.106+0000 7fd2daffd640 1 -- 172.21.15.43:0/3406701036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afeec6e7b0 msgr2=0x55afeec74c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:10.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.106+0000 7fd2daffd640 1 --2- 172.21.15.43:0/3406701036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afeec6e7b0 0x55afeec74c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.106+0000 7fd2daffd640 1 -- 172.21.15.43:0/3406701036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afee9af6b0 con 0x55afeec77ce0 2024-04-21T08:39:10.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.107+0000 7fd2daffd640 1 --2- 172.21.15.43:0/3406701036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec77ce0 0x55afeec756f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0004610 tx=0x7fd2d002c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e34aacfaea230c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:10.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.107+0000 7fd2baffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:10.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.107+0000 7fd2baffd640 1 -- 172.21.15.43:0/3406701036 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d00048c0 con 0x55afeec77ce0 2024-04-21T08:39:10.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.107+0000 7fd2baffd640 1 -- 172.21.15.43:0/3406701036 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd2d0004a60 con 0x55afeec77ce0 2024-04-21T08:39:10.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.107+0000 7fd2e477c740 1 -- 172.21.15.43:0/3406701036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2d0045070 con 0x55afeec77ce0 2024-04-21T08:39:10.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.107+0000 7fd2baffd640 1 -- 172.21.15.43:0/3406701036 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd2d005d050 con 0x55afeec77ce0 2024-04-21T08:39:10.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.107+0000 7fd2e477c740 10 client.16014 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:10.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.107+0000 7fd2e477c740 1 -- 172.21.15.43:0/3406701036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55afeec86e40 con 0x55afeec77ce0 2024-04-21T08:39:10.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.107+0000 7fd2baffd640 1 -- 172.21.15.43:0/3406701036 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2d0045070 con 0x55afeec77ce0 2024-04-21T08:39:10.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.107+0000 7fd2baffd640 1 -- 172.21.15.43:0/3406701036 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd2d0003d10 con 0x55afeec77ce0 2024-04-21T08:39:10.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.107+0000 7fd2e477c740 10 client.16014 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:10.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.107+0000 7fd2e477c740 10 client.16014 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T08:39:10.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.107+0000 7fd2e477c740 20 client.16014 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:10.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.107+0000 7fd2e477c740 1 --2- 172.21.15.43:0/3406701036 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55afeec88ff0 0x55afeec8a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:10.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.107+0000 7fd2e477c740 4 client.16014 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:10.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.107+0000 7fd2e477c740 1 -- 172.21.15.43:0/3406701036 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afeebe0530 con 0x55afeec88ff0 2024-04-21T08:39:10.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.108+0000 7fd2db7fe640 1 --2- 172.21.15.43:0/3406701036 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55afeec88ff0 0x55afeec8a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:10.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.108+0000 7fd2baffd640 10 client.16014 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:10.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.108+0000 7fd2db7fe640 1 --2- 172.21.15.43:0/3406701036 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55afeec88ff0 0x55afeec8a460 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b5cf46b59f2fd7a server_cookie=8a75ffd4e8084c24 in_seq=0 out_seq=0 2024-04-21T08:39:10.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.109+0000 7fd2baffd640 1 -- 172.21.15.43:0/3406701036 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55afeebe0530 con 0x55afeec88ff0 2024-04-21T08:39:10.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.109+0000 7fd2baffd640 10 client.16014 handle_command_reply: tid=0 2024-04-21T08:39:10.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.109+0000 7fd2e477c740 1 client.16014 shutdown 2024-04-21T08:39:10.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.109+0000 7fd2e477c740 1 -- 172.21.15.43:0/3406701036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec77ce0 msgr2=0x55afeec756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:10.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.109+0000 7fd2e477c740 1 --2- 172.21.15.43:0/3406701036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec77ce0 0x55afeec756f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0004610 tx=0x7fd2d002c740 comp rx=0 tx=0).stop 2024-04-21T08:39:10.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.109+0000 7fd2e477c740 1 -- 172.21.15.43:0/3406701036 shutdown_connections 2024-04-21T08:39:10.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.109+0000 7fd2e477c740 1 --2- 172.21.15.43:0/3406701036 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55afeec88ff0 0x55afeec8a460 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.109+0000 7fd2e477c740 1 --2- 172.21.15.43:0/3406701036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec77ce0 0x55afeec756f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.109+0000 7fd2e477c740 1 --2- 172.21.15.43:0/3406701036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55afeec6f160 0x55afeec751b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.109+0000 7fd2baffd640 0 client.16014 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:10.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.109+0000 7fd2e477c740 1 --2- 172.21.15.43:0/3406701036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afeec6e7b0 0x55afeec74c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.109+0000 7fd2e477c740 1 -- 172.21.15.43:0/3406701036 >> 172.21.15.43:0/3406701036 conn(0x55afeec68a60 msgr2=0x55afeec6abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:10.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.109+0000 7fd2e477c740 1 -- 172.21.15.43:0/3406701036 shutdown_connections 2024-04-21T08:39:10.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.109+0000 7fd2e477c740 1 -- 172.21.15.43:0/3406701036 wait complete. 2024-04-21T08:39:10.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.109+0000 7fd2e477c740 20 client.16014 trim_cache size 0 max 16384 2024-04-21T08:39:10.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.127+0000 7fd2e477c740 1 Processor -- start 2024-04-21T08:39:10.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.127+0000 7fd2e477c740 1 -- start start 2024-04-21T08:39:10.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.127+0000 7fd2e477c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afeec6e7b0 0x55afeec773a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:10.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.127+0000 7fd2e477c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec6f160 0x55afeec88920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:10.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.127+0000 7fd2e477c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55afeec77ce0 0x55afeec68e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:10.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.127+0000 7fd2e477c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55afeec887a0 con 0x55afeec6f160 2024-04-21T08:39:10.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.127+0000 7fd2e477c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55afeec88e70 con 0x55afeec6e7b0 2024-04-21T08:39:10.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.127+0000 7fd2e477c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55afeec89540 con 0x55afeec77ce0 2024-04-21T08:39:10.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.128+0000 7fd2db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec6f160 0x55afeec88920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:10.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.128+0000 7fd2db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec6f160 0x55afeec88920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50374/0 (socket says 172.21.15.43:50374) 2024-04-21T08:39:10.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.128+0000 7fd2db7fe640 1 -- 172.21.15.43:0/1277438505 learned_addr learned my addr 172.21.15.43:0/1277438505 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:10.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.128+0000 7fd2daffd640 1 --2- 172.21.15.43:0/1277438505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afeec6e7b0 0x55afeec773a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:10.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.128+0000 7fd2daffd640 1 --2- 172.21.15.43:0/1277438505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55afeec77ce0 0x55afeec68e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.128+0000 7fd2db7fe640 1 -- 172.21.15.43:0/1277438505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55afeec77ce0 msgr2=0x55afeec68e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.128+0000 7fd2db7fe640 1 --2- 172.21.15.43:0/1277438505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55afeec77ce0 0x55afeec68e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.128+0000 7fd2db7fe640 1 -- 172.21.15.43:0/1277438505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afeec6e7b0 msgr2=0x55afeec773a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.128+0000 7fd2db7fe640 1 --2- 172.21.15.43:0/1277438505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afeec6e7b0 0x55afeec773a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.128+0000 7fd2db7fe640 1 -- 172.21.15.43:0/1277438505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afeec6ef00 con 0x55afeec6f160 2024-04-21T08:39:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.128+0000 7fd2daffd640 1 --2- 172.21.15.43:0/1277438505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afeec6e7b0 0x55afeec773a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.128+0000 7fd2db7fe640 1 --2- 172.21.15.43:0/1277438505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec6f160 0x55afeec88920 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc004700 tx=0x7fd2cc014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21de37ce00cb3196 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.128+0000 7fd2c17fa640 1 -- 172.21.15.43:0/1277438505 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc0140d0 con 0x55afeec6f160 2024-04-21T08:39:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.128+0000 7fd2c17fa640 1 -- 172.21.15.43:0/1277438505 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2cc014270 con 0x55afeec6f160 2024-04-21T08:39:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.128+0000 7fd2c17fa640 1 -- 172.21.15.43:0/1277438505 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc0129c0 con 0x55afeec6f160 2024-04-21T08:39:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.129+0000 7fd2e477c740 1 -- 172.21.15.43:0/1277438505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec6f160 msgr2=0x55afeec88920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.129+0000 7fd2e477c740 1 --2- 172.21.15.43:0/1277438505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec6f160 0x55afeec88920 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc004700 tx=0x7fd2cc014840 comp rx=0 tx=0).stop 2024-04-21T08:39:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.129+0000 7fd2e477c740 1 -- 172.21.15.43:0/1277438505 shutdown_connections 2024-04-21T08:39:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.129+0000 7fd2e477c740 1 --2- 172.21.15.43:0/1277438505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55afeec77ce0 0x55afeec68e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.129+0000 7fd2e477c740 1 --2- 172.21.15.43:0/1277438505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec6f160 0x55afeec88920 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.129+0000 7fd2e477c740 1 --2- 172.21.15.43:0/1277438505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afeec6e7b0 0x55afeec773a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.129+0000 7fd2e477c740 1 -- 172.21.15.43:0/1277438505 >> 172.21.15.43:0/1277438505 conn(0x55afeec68a60 msgr2=0x55afeec63800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.129+0000 7fd2e477c740 1 -- 172.21.15.43:0/1277438505 shutdown_connections 2024-04-21T08:39:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.129+0000 7fd2e477c740 1 -- 172.21.15.43:0/1277438505 wait complete. 2024-04-21T08:39:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.129+0000 7fd2e477c740 1 -- start start 2024-04-21T08:39:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.129+0000 7fd2e477c740 1 Processor -- start 2024-04-21T08:39:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.129+0000 7fd2e477c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec6e7b0 0x55afeec71b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.129+0000 7fd2e477c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afeec6f160 0x55afeec72050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.129+0000 7fd2e477c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55afeec77ce0 0x55afeec72590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.130+0000 7fd2e477c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55afeec8a2e0 con 0x55afeec6e7b0 2024-04-21T08:39:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.130+0000 7fd2e477c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55afeec89c10 con 0x55afeec6f160 2024-04-21T08:39:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.130+0000 7fd2e477c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55afeec8a9b0 con 0x55afeec77ce0 2024-04-21T08:39:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.130+0000 7fd2e477c740 10 client.0 _ll_register_callbacks cb 0x7fd2d0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.130+0000 7fd2daffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec6e7b0 0x55afeec71b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.130+0000 7fd2daffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec6e7b0 0x55afeec71b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50384/0 (socket says 172.21.15.43:50384) 2024-04-21T08:39:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.130+0000 7fd2daffd640 1 -- 172.21.15.43:0/1334091221 learned_addr learned my addr 172.21.15.43:0/1334091221 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.130+0000 7fd2db7fe640 1 --2- 172.21.15.43:0/1334091221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afeec6f160 0x55afeec72050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.130+0000 7fd2daffd640 1 --2- 172.21.15.43:0/1334091221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55afeec77ce0 0x55afeec72590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.130+0000 7fd2daffd640 1 -- 172.21.15.43:0/1334091221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55afeec77ce0 msgr2=0x55afeec72590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.130+0000 7fd2daffd640 1 --2- 172.21.15.43:0/1334091221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55afeec77ce0 0x55afeec72590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.130+0000 7fd2daffd640 1 -- 172.21.15.43:0/1334091221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afeec6f160 msgr2=0x55afeec72050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.130+0000 7fd2daffd640 1 --2- 172.21.15.43:0/1334091221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afeec6f160 0x55afeec72050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.130+0000 7fd2daffd640 1 -- 172.21.15.43:0/1334091221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afeec90d10 con 0x55afeec6e7b0 2024-04-21T08:39:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.130+0000 7fd2db7fe640 1 --2- 172.21.15.43:0/1334091221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afeec6f160 0x55afeec72050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.130+0000 7fd2daffd640 1 --2- 172.21.15.43:0/1334091221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec6e7b0 0x55afeec71b10 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0006010 tx=0x7fd2d002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd04e0036223479 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.130+0000 7fd2baffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.130+0000 7fd2baffd640 1 -- 172.21.15.43:0/1334091221 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0009520 con 0x55afeec6e7b0 2024-04-21T08:39:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.130+0000 7fd2e477c740 1 -- 172.21.15.43:0/1334091221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2cc014270 con 0x55afeec6e7b0 2024-04-21T08:39:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.130+0000 7fd2baffd640 1 -- 172.21.15.43:0/1334091221 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd2d00051a0 con 0x55afeec6e7b0 2024-04-21T08:39:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.130+0000 7fd2baffd640 1 -- 172.21.15.43:0/1334091221 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd2d0040b50 con 0x55afeec6e7b0 2024-04-21T08:39:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.130+0000 7fd2e477c740 10 client.16020 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.130+0000 7fd2e477c740 1 -- 172.21.15.43:0/1334091221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55afeecef4d0 con 0x55afeec6e7b0 2024-04-21T08:39:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.131+0000 7fd2baffd640 1 -- 172.21.15.43:0/1334091221 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2cc014270 con 0x55afeec6e7b0 2024-04-21T08:39:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.131+0000 7fd2baffd640 1 -- 172.21.15.43:0/1334091221 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd2d00035f0 con 0x55afeec6e7b0 2024-04-21T08:39:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.131+0000 7fd2e477c740 10 client.16020 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.131+0000 7fd2e477c740 10 client.16020 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T08:39:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.131+0000 7fd2e477c740 20 client.16020 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.131+0000 7fd2e477c740 1 --2- 172.21.15.43:0/1334091221 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55afeecf3fa0 0x55afeecf2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.131+0000 7fd2e477c740 4 client.16020 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:39:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.131+0000 7fd2e477c740 1 -- 172.21.15.43:0/1334091221 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "status"}) -- 0x55afeebda3a0 con 0x55afeecf3fa0 2024-04-21T08:39:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.131+0000 7fd2db7fe640 1 --2- 172.21.15.43:0/1334091221 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55afeecf3fa0 0x55afeecf2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.131+0000 7fd2baffd640 10 client.16020 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.131+0000 7fd2db7fe640 1 --2- 172.21.15.43:0/1334091221 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55afeecf3fa0 0x55afeecf2cb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=616ca5bf423a5263 server_cookie=624e0411d6a3cea3 in_seq=0 out_seq=0 2024-04-21T08:39:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.132+0000 7fd2baffd640 1 -- 172.21.15.43:0/1334091221 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55afeebda3a0 con 0x55afeecf3fa0 2024-04-21T08:39:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.132+0000 7fd2baffd640 10 client.16020 handle_command_reply: tid=0 2024-04-21T08:39:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.132+0000 7fd2e477c740 1 client.16020 shutdown 2024-04-21T08:39:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.132+0000 7fd2e477c740 1 -- 172.21.15.43:0/1334091221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec6e7b0 msgr2=0x55afeec71b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.132+0000 7fd2e477c740 1 --2- 172.21.15.43:0/1334091221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec6e7b0 0x55afeec71b10 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0006010 tx=0x7fd2d002bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.132+0000 7fd2e477c740 1 -- 172.21.15.43:0/1334091221 shutdown_connections 2024-04-21T08:39:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.132+0000 7fd2e477c740 1 --2- 172.21.15.43:0/1334091221 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55afeecf3fa0 0x55afeecf2cb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.132+0000 7fd2e477c740 1 --2- 172.21.15.43:0/1334091221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55afeec77ce0 0x55afeec72590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.132+0000 7fd2e477c740 1 --2- 172.21.15.43:0/1334091221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55afeec6f160 0x55afeec72050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.132+0000 7fd2baffd640 0 client.16020 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.132+0000 7fd2e477c740 1 --2- 172.21.15.43:0/1334091221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55afeec6e7b0 0x55afeec71b10 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.132+0000 7fd2e477c740 1 -- 172.21.15.43:0/1334091221 >> 172.21.15.43:0/1334091221 conn(0x55afeec68a60 msgr2=0x55afeec68e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.132+0000 7fd2e477c740 1 -- 172.21.15.43:0/1334091221 shutdown_connections 2024-04-21T08:39:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.132+0000 7fd2e477c740 1 -- 172.21.15.43:0/1334091221 wait complete. 2024-04-21T08:39:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.132+0000 7fd2e477c740 20 client.16020 trim_cache size 0 max 16384 2024-04-21T08:39:10.138 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:39:10.138 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "d57a7d98-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:39:10.138 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-04-21T08:39:10.138 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24527, 2024-04-21T08:39:10.138 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-21T08:39:10.138 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-21T08:39:10.138 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-21T08:39:10.138 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 165.67706009400001, 2024-04-21T08:39:10.138 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 28, 2024-04-21T08:39:10.138 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 95, 2024-04-21T08:39:10.139 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 94, 2024-04-21T08:39:10.139 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 170.952109675 2024-04-21T08:39:10.139 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:39:10.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.133+0000 7fd2e1b8f640 1 -- 172.21.15.43:0/1619964770 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd2c40a3db0 msgr2=0x7fd2c40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.133+0000 7fd2e1b8f640 1 --2- 172.21.15.43:0/1619964770 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd2c40a3db0 0x7fd2c40a6270 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40cde80 tx=0x7fd2d406e000 comp rx=0 tx=0).stop 2024-04-21T08:39:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.133+0000 7fd2e1b8f640 1 -- 172.21.15.43:0/1619964770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2dc19cff0 msgr2=0x7fd2dc19d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.133+0000 7fd2e1b8f640 1 --2- 172.21.15.43:0/1619964770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2dc19cff0 0x7fd2dc19d4a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009fd0 tx=0x7fd2d400b260 comp rx=0 tx=0).stop 2024-04-21T08:39:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.133+0000 7fd2e1b8f640 1 -- 172.21.15.43:0/1619964770 shutdown_connections 2024-04-21T08:39:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.133+0000 7fd2e1b8f640 1 --2- 172.21.15.43:0/1619964770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2dc19cff0 0x7fd2dc19d4a0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.133+0000 7fd2e1b8f640 1 --2- 172.21.15.43:0/1619964770 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd2c40a3db0 0x7fd2c40a6270 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.133+0000 7fd2e1b8f640 1 --2- 172.21.15.43:0/1619964770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2dc107270 0x7fd2dc198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.133+0000 7fd2e1b8f640 1 --2- 172.21.15.43:0/1619964770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2dc06a840 0x7fd2dc198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.133+0000 7fd2e1b8f640 1 -- 172.21.15.43:0/1619964770 >> 172.21.15.43:0/1619964770 conn(0x7fd2dc077eb0 msgr2=0x7fd2dc0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.133+0000 7fd2e1b8f640 1 -- 172.21.15.43:0/1619964770 shutdown_connections 2024-04-21T08:39:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:10.133+0000 7fd2e1b8f640 1 -- 172.21.15.43:0/1619964770 wait complete. 2024-04-21T08:39:10.143 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 120 sec 2024-04-21T08:39:10.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:10 smithi112 ceph-mon[31085]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-04-21T08:39:10.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:10 smithi043 ceph-mon[27978]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-04-21T08:39:10.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:10 smithi175 ceph-mon[31920]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-04-21T08:39:12.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:12 smithi112 ceph-mon[31085]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 51 op/s 2024-04-21T08:39:12.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:12 smithi043 ceph-mon[27978]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 51 op/s 2024-04-21T08:39:12.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:12 smithi175 ceph-mon[31920]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 51 op/s 2024-04-21T08:39:14.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:14 smithi112 ceph-mon[31085]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 50 KiB/s wr, 43 op/s 2024-04-21T08:39:14.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:14 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:39:14.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:14 smithi043 ceph-mon[27978]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 50 KiB/s wr, 43 op/s 2024-04-21T08:39:14.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:14 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:39:14.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:14 smithi175 ceph-mon[31920]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 50 KiB/s wr, 43 op/s 2024-04-21T08:39:14.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:14 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:39:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.229+0000 7f653f606640 1 -- 172.21.15.43:0/3850488520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6538106170 msgr2=0x7f653810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.229+0000 7f653f606640 1 --2- 172.21.15.43:0/3850488520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6538106170 0x7f653810dd40 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6530009ca0 tx=0x7f653002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.229+0000 7f653f606640 1 -- 172.21.15.43:0/3850488520 shutdown_connections 2024-04-21T08:39:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.229+0000 7f653f606640 1 --2- 172.21.15.43:0/3850488520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6538106170 0x7f653810dd40 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.229+0000 7f653f606640 1 --2- 172.21.15.43:0/3850488520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6538101aa0 0x7f6538105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.229+0000 7f653f606640 1 --2- 172.21.15.43:0/3850488520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65381010f0 0x7f65381014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.229+0000 7f653f606640 1 -- 172.21.15.43:0/3850488520 >> 172.21.15.43:0/3850488520 conn(0x7f6538077fd0 msgr2=0x7f65380783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.229+0000 7f653f606640 1 -- 172.21.15.43:0/3850488520 shutdown_connections 2024-04-21T08:39:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.229+0000 7f653f606640 1 -- 172.21.15.43:0/3850488520 wait complete. 2024-04-21T08:39:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.230+0000 7f653f606640 1 Processor -- start 2024-04-21T08:39:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.230+0000 7f653f606640 1 -- start start 2024-04-21T08:39:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.230+0000 7f653f606640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65381010f0 0x7f6538198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.230+0000 7f653f606640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6538106170 0x7f6538198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.230+0000 7f653f606640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f653819cf30 0x7f653819d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.230+0000 7f653f606640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f65381110e0 con 0x7f653819cf30 2024-04-21T08:39:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.230+0000 7f653f606640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6538110f60 con 0x7f65381010f0 2024-04-21T08:39:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.230+0000 7f653f606640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6538111260 con 0x7f6538106170 2024-04-21T08:39:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.230+0000 7f653dba3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f653819cf30 0x7f653819d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.230+0000 7f653dba3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f653819cf30 0x7f653819d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50396/0 (socket says 172.21.15.43:50396) 2024-04-21T08:39:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.230+0000 7f653dba3640 1 -- 172.21.15.43:0/920834145 learned_addr learned my addr 172.21.15.43:0/920834145 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.231+0000 7f653d3a2640 1 --2- 172.21.15.43:0/920834145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65381010f0 0x7f6538198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.231+0000 7f653cba1640 1 --2- 172.21.15.43:0/920834145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6538106170 0x7f6538198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.231+0000 7f653dba3640 1 -- 172.21.15.43:0/920834145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6538106170 msgr2=0x7f6538198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.231+0000 7f653dba3640 1 --2- 172.21.15.43:0/920834145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6538106170 0x7f6538198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.231+0000 7f653dba3640 1 -- 172.21.15.43:0/920834145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65381010f0 msgr2=0x7f6538198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.231+0000 7f653dba3640 1 --2- 172.21.15.43:0/920834145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65381010f0 0x7f6538198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.231+0000 7f653dba3640 1 -- 172.21.15.43:0/920834145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653819da20 con 0x7f653819cf30 2024-04-21T08:39:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.231+0000 7f653d3a2640 1 --2- 172.21.15.43:0/920834145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65381010f0 0x7f6538198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.231+0000 7f653dba3640 1 --2- 172.21.15.43:0/920834145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f653819cf30 0x7f653819d3e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f6530005e30 tx=0x7f6530034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.231+0000 7f652e7fc640 1 -- 172.21.15.43:0/920834145 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530041020 con 0x7f653819cf30 2024-04-21T08:39:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.231+0000 7f652e7fc640 1 -- 172.21.15.43:0/920834145 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6530032970 con 0x7f653819cf30 2024-04-21T08:39:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.231+0000 7f653f606640 1 -- 172.21.15.43:0/920834145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f653819dcb0 con 0x7f653819cf30 2024-04-21T08:39:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.231+0000 7f652e7fc640 1 -- 172.21.15.43:0/920834145 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530039040 con 0x7f653819cf30 2024-04-21T08:39:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.231+0000 7f653f606640 1 -- 172.21.15.43:0/920834145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65381a54d0 con 0x7f653819cf30 2024-04-21T08:39:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.234+0000 7f652e7fc640 1 -- 172.21.15.43:0/920834145 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f6530044020 con 0x7f653819cf30 2024-04-21T08:39:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.234+0000 7f65421fc740 1 Processor -- start 2024-04-21T08:39:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.234+0000 7f65421fc740 1 -- start start 2024-04-21T08:39:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.234+0000 7f65421fc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59b8d0 0x55cb0f59a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.234+0000 7f65421fc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb0f59ab60 0x55cb0f59afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.234+0000 7f65421fc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb0f5a40e0 0x55cb0f5a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.234+0000 7f65421fc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cb0f4ff8d0 con 0x55cb0f59b8d0 2024-04-21T08:39:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.234+0000 7f65421fc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cb0f49ce80 con 0x55cb0f59ab60 2024-04-21T08:39:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.234+0000 7f65421fc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55cb0f4fb890 con 0x55cb0f5a40e0 2024-04-21T08:39:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.234+0000 7f653cba1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59b8d0 0x55cb0f59a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.234+0000 7f653cba1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59b8d0 0x55cb0f59a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50402/0 (socket says 172.21.15.43:50402) 2024-04-21T08:39:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.234+0000 7f653cba1640 1 -- 172.21.15.43:0/2989329535 learned_addr learned my addr 172.21.15.43:0/2989329535 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.235+0000 7f652e7fc640 1 --2- 172.21.15.43:0/920834145 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f65240a3ee0 0x7f65240a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.235+0000 7f653cba1640 1 -- 172.21.15.43:0/2989329535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb0f5a40e0 msgr2=0x55cb0f5a64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.235+0000 7f653cba1640 1 --2- 172.21.15.43:0/2989329535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb0f5a40e0 0x55cb0f5a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.235+0000 7f653cba1640 1 -- 172.21.15.43:0/2989329535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb0f59ab60 msgr2=0x55cb0f59afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.235+0000 7f653d3a2640 1 --2- 172.21.15.43:0/2989329535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb0f59ab60 0x55cb0f59afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.235+0000 7f653cba1640 1 --2- 172.21.15.43:0/2989329535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb0f59ab60 0x55cb0f59afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.235+0000 7f653cba1640 1 -- 172.21.15.43:0/2989329535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb0f4be320 con 0x55cb0f59b8d0 2024-04-21T08:39:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.235+0000 7f653dba3640 1 --2- 172.21.15.43:0/920834145 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f65240a3ee0 0x7f65240a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.235+0000 7f653cba1640 1 --2- 172.21.15.43:0/2989329535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59b8d0 0x55cb0f59a620 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6528009b40 tx=0x7f6528010310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6160b3c9a1f5c0e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.235+0000 7f652e7fc640 1 -- 172.21.15.43:0/920834145 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f65300f2a40 con 0x7f653819cf30 2024-04-21T08:39:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.235+0000 7f65237fe640 1 -- 172.21.15.43:0/2989329535 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528016070 con 0x55cb0f59b8d0 2024-04-21T08:39:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.235+0000 7f65237fe640 1 -- 172.21.15.43:0/2989329535 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6528010ce0 con 0x55cb0f59b8d0 2024-04-21T08:39:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.235+0000 7f65237fe640 1 -- 172.21.15.43:0/2989329535 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65280115e0 con 0x55cb0f59b8d0 2024-04-21T08:39:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.236+0000 7f65421fc740 1 -- 172.21.15.43:0/2989329535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59b8d0 msgr2=0x55cb0f59a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.236+0000 7f65421fc740 1 --2- 172.21.15.43:0/2989329535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59b8d0 0x55cb0f59a620 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6528009b40 tx=0x7f6528010310 comp rx=0 tx=0).stop 2024-04-21T08:39:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.236+0000 7f653dba3640 1 --2- 172.21.15.43:0/920834145 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f65240a3ee0 0x7f65240a63a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f65300efe70 tx=0x7f6530088bb0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.236+0000 7f65421fc740 1 -- 172.21.15.43:0/2989329535 shutdown_connections 2024-04-21T08:39:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.236+0000 7f65421fc740 1 --2- 172.21.15.43:0/2989329535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb0f5a40e0 0x55cb0f5a64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.236+0000 7f65421fc740 1 --2- 172.21.15.43:0/2989329535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb0f59ab60 0x55cb0f59afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.236+0000 7f65421fc740 1 --2- 172.21.15.43:0/2989329535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59b8d0 0x55cb0f59a620 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.236+0000 7f65421fc740 1 -- 172.21.15.43:0/2989329535 >> 172.21.15.43:0/2989329535 conn(0x55cb0f561c90 msgr2=0x55cb0f595850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.236+0000 7f65421fc740 1 -- 172.21.15.43:0/2989329535 shutdown_connections 2024-04-21T08:39:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.236+0000 7f65421fc740 1 -- 172.21.15.43:0/2989329535 wait complete. 2024-04-21T08:39:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.236+0000 7f65421fc740 1 -- start start 2024-04-21T08:39:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.236+0000 7f65421fc740 1 Processor -- start 2024-04-21T08:39:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.237+0000 7f65421fc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59ab60 0x55cb0f5a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.237+0000 7f65421fc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb0f59b8d0 0x55cb0f5a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.237+0000 7f65421fc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb0f5a40e0 0x55cb0f5a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.237+0000 7f65421fc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cb0f5a6e10 con 0x55cb0f59ab60 2024-04-21T08:39:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.237+0000 7f65421fc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cb0f487eb0 con 0x55cb0f59b8d0 2024-04-21T08:39:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.237+0000 7f65421fc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55cb0f5a6f90 con 0x55cb0f5a40e0 2024-04-21T08:39:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.237+0000 7f65421fc740 10 client.0 _ll_register_callbacks cb 0x55cb0f4b4b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.237+0000 7f653cba1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59ab60 0x55cb0f5a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.237+0000 7f653cba1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59ab60 0x55cb0f5a3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50410/0 (socket says 172.21.15.43:50410) 2024-04-21T08:39:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.237+0000 7f653cba1640 1 -- 172.21.15.43:0/277337456 learned_addr learned my addr 172.21.15.43:0/277337456 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.237+0000 7f653d3a2640 1 --2- 172.21.15.43:0/277337456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb0f59b8d0 0x55cb0f5a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.237+0000 7f653cba1640 1 --2- 172.21.15.43:0/277337456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb0f5a40e0 0x55cb0f5a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.237+0000 7f653cba1640 1 -- 172.21.15.43:0/277337456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb0f5a40e0 msgr2=0x55cb0f5a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.237+0000 7f653cba1640 1 --2- 172.21.15.43:0/277337456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb0f5a40e0 0x55cb0f5a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.237+0000 7f653cba1640 1 -- 172.21.15.43:0/277337456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb0f59b8d0 msgr2=0x55cb0f5a0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.237+0000 7f653cba1640 1 --2- 172.21.15.43:0/277337456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb0f59b8d0 0x55cb0f5a0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.237+0000 7f653cba1640 1 -- 172.21.15.43:0/277337456 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb0f212210 con 0x55cb0f59ab60 2024-04-21T08:39:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.237+0000 7f653cba1640 1 --2- 172.21.15.43:0/277337456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59ab60 0x55cb0f5a3b00 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6528010fd0 tx=0x7f6528005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c28da996bd0752 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.237+0000 7f6520ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.238+0000 7f6520ff9640 1 -- 172.21.15.43:0/277337456 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528016030 con 0x55cb0f59ab60 2024-04-21T08:39:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.238+0000 7f6520ff9640 1 -- 172.21.15.43:0/277337456 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f6528004460 con 0x55cb0f59ab60 2024-04-21T08:39:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.238+0000 7f65421fc740 1 -- 172.21.15.43:0/277337456 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6528010ce0 con 0x55cb0f59ab60 2024-04-21T08:39:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.238+0000 7f6520ff9640 1 -- 172.21.15.43:0/277337456 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6528032050 con 0x55cb0f59ab60 2024-04-21T08:39:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.238+0000 7f65421fc740 10 client.16032 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.238+0000 7f65421fc740 1 -- 172.21.15.43:0/277337456 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cb0f2de620 con 0x55cb0f59ab60 2024-04-21T08:39:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.238+0000 7f6520ff9640 1 -- 172.21.15.43:0/277337456 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6528010ce0 con 0x55cb0f59ab60 2024-04-21T08:39:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.238+0000 7f6520ff9640 1 -- 172.21.15.43:0/277337456 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f65280037b0 con 0x55cb0f59ab60 2024-04-21T08:39:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.238+0000 7f65421fc740 10 client.16032 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.238+0000 7f65421fc740 10 client.16032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.238+0000 7f65421fc740 20 client.16032 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.238+0000 7f65421fc740 1 --2- 172.21.15.43:0/277337456 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cb0f5b4090 0x55cb0f5b6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.238+0000 7f65421fc740 4 client.16032 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.238+0000 7f65421fc740 1 -- 172.21.15.43:0/277337456 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb0f513880 con 0x55cb0f5b4090 2024-04-21T08:39:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.239+0000 7f653d3a2640 1 --2- 172.21.15.43:0/277337456 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cb0f5b4090 0x55cb0f5b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.239+0000 7f653d3a2640 1 --2- 172.21.15.43:0/277337456 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cb0f5b4090 0x55cb0f5b6480 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4806e8515ae1eae4 server_cookie=f372784572a62071 in_seq=0 out_seq=0 2024-04-21T08:39:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.239+0000 7f6520ff9640 10 client.16032 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.240+0000 7f6520ff9640 1 -- 172.21.15.43:0/277337456 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb0f513880 con 0x55cb0f5b4090 2024-04-21T08:39:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.240+0000 7f6520ff9640 10 client.16032 handle_command_reply: tid=0 2024-04-21T08:39:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.240+0000 7f65421fc740 1 client.16032 shutdown 2024-04-21T08:39:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.240+0000 7f65421fc740 1 -- 172.21.15.43:0/277337456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59ab60 msgr2=0x55cb0f5a3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.240+0000 7f65421fc740 1 --2- 172.21.15.43:0/277337456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59ab60 0x55cb0f5a3b00 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6528010fd0 tx=0x7f6528005f90 comp rx=0 tx=0).stop 2024-04-21T08:39:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.240+0000 7f65421fc740 1 -- 172.21.15.43:0/277337456 shutdown_connections 2024-04-21T08:39:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.240+0000 7f65421fc740 1 --2- 172.21.15.43:0/277337456 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cb0f5b4090 0x55cb0f5b6480 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.240+0000 7f65421fc740 1 --2- 172.21.15.43:0/277337456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb0f5a40e0 0x55cb0f5a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.240+0000 7f65421fc740 1 --2- 172.21.15.43:0/277337456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb0f59b8d0 0x55cb0f5a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.240+0000 7f65421fc740 1 --2- 172.21.15.43:0/277337456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59ab60 0x55cb0f5a3b00 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.240+0000 7f6520ff9640 0 client.16032 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.240+0000 7f65421fc740 1 -- 172.21.15.43:0/277337456 >> 172.21.15.43:0/277337456 conn(0x55cb0f561c90 msgr2=0x55cb0f5975a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.240+0000 7f65421fc740 1 -- 172.21.15.43:0/277337456 shutdown_connections 2024-04-21T08:39:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.240+0000 7f65421fc740 1 -- 172.21.15.43:0/277337456 wait complete. 2024-04-21T08:39:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.240+0000 7f65421fc740 20 client.16032 trim_cache size 0 max 16384 2024-04-21T08:39:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.255+0000 7f65421fc740 1 Processor -- start 2024-04-21T08:39:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.255+0000 7f65421fc740 1 -- start start 2024-04-21T08:39:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.255+0000 7f65421fc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb0f59ab60 0x55cb0f5afd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.255+0000 7f65421fc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59b8d0 0x55cb0f59fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.255+0000 7f65421fc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb0f5a40e0 0x55cb0f5a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.255+0000 7f65421fc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f65381af3c0 con 0x55cb0f59b8d0 2024-04-21T08:39:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.255+0000 7f65421fc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f65381b0f00 con 0x55cb0f5a40e0 2024-04-21T08:39:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.255+0000 7f65421fc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65381afa90 con 0x55cb0f59ab60 2024-04-21T08:39:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.255+0000 7f653d3a2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59b8d0 0x55cb0f59fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.255+0000 7f653d3a2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59b8d0 0x55cb0f59fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50414/0 (socket says 172.21.15.43:50414) 2024-04-21T08:39:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.255+0000 7f653d3a2640 1 -- 172.21.15.43:0/2666267670 learned_addr learned my addr 172.21.15.43:0/2666267670 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.255+0000 7f653cba1640 1 --2- 172.21.15.43:0/2666267670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb0f5a40e0 0x55cb0f5a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.255+0000 7f653cba1640 1 --2- 172.21.15.43:0/2666267670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb0f59ab60 0x55cb0f5afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.256+0000 7f653d3a2640 1 -- 172.21.15.43:0/2666267670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb0f59ab60 msgr2=0x55cb0f5afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.256+0000 7f653d3a2640 1 --2- 172.21.15.43:0/2666267670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb0f59ab60 0x55cb0f5afd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.256+0000 7f653d3a2640 1 -- 172.21.15.43:0/2666267670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb0f5a40e0 msgr2=0x55cb0f5a35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.256+0000 7f653d3a2640 1 --2- 172.21.15.43:0/2666267670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb0f5a40e0 0x55cb0f5a35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.256+0000 7f653d3a2640 1 -- 172.21.15.43:0/2666267670 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb0f59a990 con 0x55cb0f59b8d0 2024-04-21T08:39:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.256+0000 7f653cba1640 1 --2- 172.21.15.43:0/2666267670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb0f5a40e0 0x55cb0f5a35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.256+0000 7f653d3a2640 1 --2- 172.21.15.43:0/2666267670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59b8d0 0x55cb0f59fe70 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x55cb0f59c030 tx=0x7f653400cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf38d11bf9ad2ce server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.256+0000 7f6522ffd640 1 -- 172.21.15.43:0/2666267670 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653400ec20 con 0x55cb0f59b8d0 2024-04-21T08:39:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.256+0000 7f6522ffd640 1 -- 172.21.15.43:0/2666267670 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f653401c8e0 con 0x55cb0f59b8d0 2024-04-21T08:39:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.256+0000 7f6522ffd640 1 -- 172.21.15.43:0/2666267670 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653400fd00 con 0x55cb0f59b8d0 2024-04-21T08:39:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.257+0000 7f65421fc740 1 -- 172.21.15.43:0/2666267670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59b8d0 msgr2=0x55cb0f59fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.257+0000 7f65421fc740 1 --2- 172.21.15.43:0/2666267670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59b8d0 0x55cb0f59fe70 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x55cb0f59c030 tx=0x7f653400cfd0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.257+0000 7f65421fc740 1 -- 172.21.15.43:0/2666267670 shutdown_connections 2024-04-21T08:39:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.257+0000 7f65421fc740 1 --2- 172.21.15.43:0/2666267670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb0f5a40e0 0x55cb0f5a35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.257+0000 7f65421fc740 1 --2- 172.21.15.43:0/2666267670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59b8d0 0x55cb0f59fe70 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.257+0000 7f65421fc740 1 --2- 172.21.15.43:0/2666267670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb0f59ab60 0x55cb0f5afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.257+0000 7f65421fc740 1 -- 172.21.15.43:0/2666267670 >> 172.21.15.43:0/2666267670 conn(0x55cb0f561c90 msgr2=0x55cb0f59f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.257+0000 7f65421fc740 1 -- 172.21.15.43:0/2666267670 shutdown_connections 2024-04-21T08:39:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.257+0000 7f65421fc740 1 -- 172.21.15.43:0/2666267670 wait complete. 2024-04-21T08:39:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.257+0000 7f65421fc740 1 -- start start 2024-04-21T08:39:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.257+0000 7f65421fc740 1 Processor -- start 2024-04-21T08:39:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.258+0000 7f65421fc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59ab60 0x55cb0f5ac390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.258+0000 7f65421fc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb0f59b8d0 0x55cb0f5a7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.258+0000 7f65421fc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb0f5a40e0 0x55cb0f5a7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.258+0000 7f65421fc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f65381b0830 con 0x55cb0f59ab60 2024-04-21T08:39:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.258+0000 7f65421fc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f65381b0160 con 0x55cb0f5a40e0 2024-04-21T08:39:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.258+0000 7f65421fc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65381ad3a0 con 0x55cb0f59b8d0 2024-04-21T08:39:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.258+0000 7f65421fc740 10 client.0 _ll_register_callbacks cb 0x7f6528009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.258+0000 7f653cba1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59ab60 0x55cb0f5ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.258+0000 7f653cba1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59ab60 0x55cb0f5ac390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50424/0 (socket says 172.21.15.43:50424) 2024-04-21T08:39:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.258+0000 7f653cba1640 1 -- 172.21.15.43:0/3252301059 learned_addr learned my addr 172.21.15.43:0/3252301059 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.258+0000 7f653d3a2640 1 --2- 172.21.15.43:0/3252301059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb0f59b8d0 0x55cb0f5a7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.258+0000 7f653cba1640 1 -- 172.21.15.43:0/3252301059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb0f59b8d0 msgr2=0x55cb0f5a7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.258+0000 7f653cba1640 1 --2- 172.21.15.43:0/3252301059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb0f59b8d0 0x55cb0f5a7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.258+0000 7f653cba1640 1 -- 172.21.15.43:0/3252301059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb0f5a40e0 msgr2=0x55cb0f5a7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.258+0000 7f653cba1640 1 --2- 172.21.15.43:0/3252301059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb0f5a40e0 0x55cb0f5a7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.258+0000 7f653cba1640 1 -- 172.21.15.43:0/3252301059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb0f5b67f0 con 0x55cb0f59ab60 2024-04-21T08:39:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.258+0000 7f653cba1640 1 --2- 172.21.15.43:0/3252301059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59ab60 0x55cb0f5ac390 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f65280092d0 tx=0x7f6528009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a3e300cc8b919c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.258+0000 7f6520ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.258+0000 7f6520ff9640 1 -- 172.21.15.43:0/3252301059 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528016070 con 0x55cb0f59ab60 2024-04-21T08:39:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.258+0000 7f6520ff9640 1 -- 172.21.15.43:0/3252301059 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f6528010bf0 con 0x55cb0f59ab60 2024-04-21T08:39:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.258+0000 7f65421fc740 1 -- 172.21.15.43:0/3252301059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f653401c8e0 con 0x55cb0f59ab60 2024-04-21T08:39:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.259+0000 7f6520ff9640 1 -- 172.21.15.43:0/3252301059 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6528043050 con 0x55cb0f59ab60 2024-04-21T08:39:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.259+0000 7f65421fc740 10 client.16038 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.259+0000 7f65421fc740 1 -- 172.21.15.43:0/3252301059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cb0f619ec0 con 0x55cb0f59ab60 2024-04-21T08:39:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.259+0000 7f6520ff9640 1 -- 172.21.15.43:0/3252301059 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f653401c8e0 con 0x55cb0f59ab60 2024-04-21T08:39:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.259+0000 7f6520ff9640 1 -- 172.21.15.43:0/3252301059 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f65280128a0 con 0x55cb0f59ab60 2024-04-21T08:39:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.259+0000 7f65421fc740 10 client.16038 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.259+0000 7f65421fc740 10 client.16038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.259+0000 7f65421fc740 20 client.16038 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.259+0000 7f65421fc740 1 --2- 172.21.15.43:0/3252301059 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cb0f61a5b0 0x55cb0f61c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.259+0000 7f65421fc740 4 client.16038 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:39:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.259+0000 7f65421fc740 1 -- 172.21.15.43:0/3252301059 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cb0f49f3d0 con 0x55cb0f61a5b0 2024-04-21T08:39:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.259+0000 7f653d3a2640 1 --2- 172.21.15.43:0/3252301059 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cb0f61a5b0 0x55cb0f61c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.260+0000 7f653d3a2640 1 --2- 172.21.15.43:0/3252301059 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cb0f61a5b0 0x55cb0f61c9d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b82c868105b5ae65 server_cookie=282d11d4ad27916f in_seq=0 out_seq=0 2024-04-21T08:39:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.260+0000 7f6520ff9640 10 client.16038 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.272+0000 7f6520ff9640 1 -- 172.21.15.43:0/3252301059 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cb0f49f3d0 con 0x55cb0f61a5b0 2024-04-21T08:39:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.272+0000 7f6520ff9640 10 client.16038 handle_command_reply: tid=0 2024-04-21T08:39:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.272+0000 7f65421fc740 1 client.16038 shutdown 2024-04-21T08:39:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.272+0000 7f65421fc740 1 -- 172.21.15.43:0/3252301059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59ab60 msgr2=0x55cb0f5ac390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.272+0000 7f65421fc740 1 --2- 172.21.15.43:0/3252301059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59ab60 0x55cb0f5ac390 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f65280092d0 tx=0x7f6528009300 comp rx=0 tx=0).stop 2024-04-21T08:39:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.272+0000 7f65421fc740 1 -- 172.21.15.43:0/3252301059 shutdown_connections 2024-04-21T08:39:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.272+0000 7f65421fc740 1 --2- 172.21.15.43:0/3252301059 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cb0f61a5b0 0x55cb0f61c9d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.272+0000 7f65421fc740 1 --2- 172.21.15.43:0/3252301059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb0f5a40e0 0x55cb0f5a7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.272+0000 7f6520ff9640 0 client.16038 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.272+0000 7f65421fc740 1 --2- 172.21.15.43:0/3252301059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb0f59b8d0 0x55cb0f5a7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.272+0000 7f65421fc740 1 --2- 172.21.15.43:0/3252301059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb0f59ab60 0x55cb0f5ac390 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.272+0000 7f65421fc740 1 -- 172.21.15.43:0/3252301059 >> 172.21.15.43:0/3252301059 conn(0x55cb0f561c90 msgr2=0x55cb0f59fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.272+0000 7f65421fc740 1 -- 172.21.15.43:0/3252301059 shutdown_connections 2024-04-21T08:39:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.272+0000 7f65421fc740 1 -- 172.21.15.43:0/3252301059 wait complete. 2024-04-21T08:39:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.273+0000 7f65421fc740 20 client.16038 trim_cache size 0 max 16384 2024-04-21T08:39:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:39:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:39:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:39:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:39:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.273+0000 7f653f606640 1 -- 172.21.15.43:0/920834145 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f65240a3ee0 msgr2=0x7f65240a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.273+0000 7f653f606640 1 --2- 172.21.15.43:0/920834145 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f65240a3ee0 0x7f65240a63a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f65300efe70 tx=0x7f6530088bb0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.273+0000 7f653f606640 1 -- 172.21.15.43:0/920834145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f653819cf30 msgr2=0x7f653819d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.273+0000 7f653f606640 1 --2- 172.21.15.43:0/920834145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f653819cf30 0x7f653819d3e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f6530005e30 tx=0x7f6530034670 comp rx=0 tx=0).stop 2024-04-21T08:39:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.273+0000 7f653f606640 1 -- 172.21.15.43:0/920834145 shutdown_connections 2024-04-21T08:39:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.273+0000 7f653f606640 1 --2- 172.21.15.43:0/920834145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f653819cf30 0x7f653819d3e0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.273+0000 7f653f606640 1 --2- 172.21.15.43:0/920834145 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f65240a3ee0 0x7f65240a63a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.273+0000 7f653f606640 1 --2- 172.21.15.43:0/920834145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6538106170 0x7f6538198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.273+0000 7f653f606640 1 --2- 172.21.15.43:0/920834145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65381010f0 0x7f6538198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.273+0000 7f653f606640 1 -- 172.21.15.43:0/920834145 >> 172.21.15.43:0/920834145 conn(0x7f6538077fd0 msgr2=0x7f65380698e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.273+0000 7f653f606640 1 -- 172.21.15.43:0/920834145 shutdown_connections 2024-04-21T08:39:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:15.273+0000 7f653f606640 1 -- 172.21.15.43:0/920834145 wait complete. 2024-04-21T08:39:16.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:16 smithi112 ceph-mon[31085]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-04-21T08:39:16.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:16 smithi043 ceph-mon[27978]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-04-21T08:39:16.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:16 smithi175 ceph-mon[31920]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-04-21T08:39:18.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:18 smithi112 ceph-mon[31085]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 KiB/s wr, 34 op/s 2024-04-21T08:39:18.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:18 smithi043 ceph-mon[27978]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 KiB/s wr, 34 op/s 2024-04-21T08:39:18.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:18 smithi175 ceph-mon[31920]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 KiB/s wr, 34 op/s 2024-04-21T08:39:20.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:20 smithi112 ceph-mon[31085]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-04-21T08:39:20.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:20 smithi043 ceph-mon[27978]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-04-21T08:39:20.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:20 smithi175 ceph-mon[31920]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-04-21T08:39:22.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:22 smithi112 ceph-mon[31085]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2024-04-21T08:39:22.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:22 smithi043 ceph-mon[27978]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2024-04-21T08:39:22.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:22 smithi175 ceph-mon[31920]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2024-04-21T08:39:24.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:24 smithi112 ceph-mon[31085]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.2 KiB/s wr, 17 op/s 2024-04-21T08:39:24.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:24 smithi043 ceph-mon[27978]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.2 KiB/s wr, 17 op/s 2024-04-21T08:39:24.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:24 smithi175 ceph-mon[31920]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.2 KiB/s wr, 17 op/s 2024-04-21T08:39:26.766 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:26 smithi175 ceph-mon[31920]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-21T08:39:26.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:26 smithi112 ceph-mon[31085]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-21T08:39:26.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:26 smithi043 ceph-mon[27978]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-21T08:39:28.362 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.362+0000 7f7678ec1640 1 -- 172.21.15.43:0/3992403630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7674106170 msgr2=0x7f767410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.362+0000 7f7678ec1640 1 --2- 172.21.15.43:0/3992403630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7674106170 0x7f767410dd40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f766c009ca0 tx=0x7f766c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.362+0000 7f7678ec1640 1 -- 172.21.15.43:0/3992403630 shutdown_connections 2024-04-21T08:39:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.362+0000 7f7678ec1640 1 --2- 172.21.15.43:0/3992403630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7674106170 0x7f767410dd40 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.362+0000 7f7678ec1640 1 --2- 172.21.15.43:0/3992403630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7674101aa0 0x7f7674105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.362+0000 7f7678ec1640 1 --2- 172.21.15.43:0/3992403630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76741010f0 0x7f76741014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.362+0000 7f7678ec1640 1 -- 172.21.15.43:0/3992403630 >> 172.21.15.43:0/3992403630 conn(0x7f7674077fd0 msgr2=0x7f76740783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.362+0000 7f7678ec1640 1 -- 172.21.15.43:0/3992403630 shutdown_connections 2024-04-21T08:39:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.362+0000 7f7678ec1640 1 -- 172.21.15.43:0/3992403630 wait complete. 2024-04-21T08:39:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.363+0000 7f7678ec1640 1 Processor -- start 2024-04-21T08:39:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.363+0000 7f7678ec1640 1 -- start start 2024-04-21T08:39:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.363+0000 7f7678ec1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76741010f0 0x7f76741986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.363+0000 7f7678ec1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7674106170 0x7f7674198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.363+0000 7f7678ec1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f767419cee0 0x7f767419d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.363+0000 7f7678ec1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f76741110e0 con 0x7f76741010f0 2024-04-21T08:39:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.363+0000 7f7678ec1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7674110f60 con 0x7f7674106170 2024-04-21T08:39:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.363+0000 7f7678ec1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7674111260 con 0x7f767419cee0 2024-04-21T08:39:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.363+0000 7f767259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76741010f0 0x7f76741986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.363+0000 7f767259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76741010f0 0x7f76741986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48780/0 (socket says 172.21.15.43:48780) 2024-04-21T08:39:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.363+0000 7f767259c640 1 -- 172.21.15.43:0/1910257518 learned_addr learned my addr 172.21.15.43:0/1910257518 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.363+0000 7f7672d9d640 1 --2- 172.21.15.43:0/1910257518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f767419cee0 0x7f767419d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.363+0000 7f7671d9b640 1 --2- 172.21.15.43:0/1910257518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7674106170 0x7f7674198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.364+0000 7f767259c640 1 -- 172.21.15.43:0/1910257518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f767419cee0 msgr2=0x7f767419d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.364+0000 7f767259c640 1 --2- 172.21.15.43:0/1910257518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f767419cee0 0x7f767419d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.364+0000 7f767259c640 1 -- 172.21.15.43:0/1910257518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7674106170 msgr2=0x7f7674198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.364+0000 7f767259c640 1 --2- 172.21.15.43:0/1910257518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7674106170 0x7f7674198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.364+0000 7f767259c640 1 -- 172.21.15.43:0/1910257518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767419d9d0 con 0x7f76741010f0 2024-04-21T08:39:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.364+0000 7f767259c640 1 --2- 172.21.15.43:0/1910257518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76741010f0 0x7f76741986c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f766400e9f0 tx=0x7f766400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.364+0000 7f76637fe640 1 -- 172.21.15.43:0/1910257518 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766400cd80 con 0x7f76741010f0 2024-04-21T08:39:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.364+0000 7f7678ec1640 1 -- 172.21.15.43:0/1910257518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f767419dcc0 con 0x7f76741010f0 2024-04-21T08:39:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.364+0000 7f76637fe640 1 -- 172.21.15.43:0/1910257518 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76640054d0 con 0x7f76741010f0 2024-04-21T08:39:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.364+0000 7f76637fe640 1 -- 172.21.15.43:0/1910257518 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7664010690 con 0x7f76741010f0 2024-04-21T08:39:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.364+0000 7f7678ec1640 1 -- 172.21.15.43:0/1910257518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76741a5460 con 0x7f76741010f0 2024-04-21T08:39:28.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.366+0000 7f76637fe640 1 -- 172.21.15.43:0/1910257518 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f7664004e90 con 0x7f76741010f0 2024-04-21T08:39:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.367+0000 7f76637fe640 1 --2- 172.21.15.43:0/1910257518 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f765c0a3e60 0x7f765c0a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.367+0000 7f7671d9b640 1 --2- 172.21.15.43:0/1910257518 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f765c0a3e60 0x7f765c0a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.367+0000 7f76637fe640 1 -- 172.21.15.43:0/1910257518 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f76640d1220 con 0x7f76741010f0 2024-04-21T08:39:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.367+0000 7f7671d9b640 1 --2- 172.21.15.43:0/1910257518 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f765c0a3e60 0x7f765c0a6320 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7674100cc0 tx=0x7f7668005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.367+0000 7f767bb92740 1 Processor -- start 2024-04-21T08:39:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.368+0000 7f767bb92740 1 -- start start 2024-04-21T08:39:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.368+0000 7f767bb92740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7297f78d0 0x55c7297f6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.368+0000 7f767bb92740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7297f6b60 0x55c7297f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.368+0000 7f767bb92740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7298000e0 0x55c7298024d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.368+0000 7f767bb92740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c72975b8d0 con 0x55c7297f78d0 2024-04-21T08:39:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.368+0000 7f767bb92740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c7296f8e80 con 0x55c7298000e0 2024-04-21T08:39:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.368+0000 7f767bb92740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c729757890 con 0x55c7297f6b60 2024-04-21T08:39:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.368+0000 7f767259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7297f78d0 0x55c7297f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.368+0000 7f767259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7297f78d0 0x55c7297f6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48786/0 (socket says 172.21.15.43:48786) 2024-04-21T08:39:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.368+0000 7f767259c640 1 -- 172.21.15.43:0/1564936292 learned_addr learned my addr 172.21.15.43:0/1564936292 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.368+0000 7f7671d9b640 1 --2- 172.21.15.43:0/1564936292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7297f6b60 0x55c7297f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.368+0000 7f7672d9d640 1 --2- 172.21.15.43:0/1564936292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7298000e0 0x55c7298024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.368+0000 7f767259c640 1 -- 172.21.15.43:0/1564936292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7297f6b60 msgr2=0x55c7297f6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.368+0000 7f767259c640 1 --2- 172.21.15.43:0/1564936292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7297f6b60 0x55c7297f6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.368+0000 7f767259c640 1 -- 172.21.15.43:0/1564936292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7298000e0 msgr2=0x55c7298024d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.368+0000 7f767259c640 1 --2- 172.21.15.43:0/1564936292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7298000e0 0x55c7298024d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.368+0000 7f767259c640 1 -- 172.21.15.43:0/1564936292 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c72971a320 con 0x55c7297f78d0 2024-04-21T08:39:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.368+0000 7f7672d9d640 1 --2- 172.21.15.43:0/1564936292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7298000e0 0x55c7298024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.369+0000 7f767259c640 1 --2- 172.21.15.43:0/1564936292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7297f78d0 0x55c7297f6620 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f76640ceba0 tx=0x7f766409c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e569d5fe791cd486 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.369+0000 7f7660ff9640 1 -- 172.21.15.43:0/1564936292 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76640b6070 con 0x55c7297f78d0 2024-04-21T08:39:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.369+0000 7f7660ff9640 1 -- 172.21.15.43:0/1564936292 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f766409cc80 con 0x55c7297f78d0 2024-04-21T08:39:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.369+0000 7f7660ff9640 1 -- 172.21.15.43:0/1564936292 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76640ce790 con 0x55c7297f78d0 2024-04-21T08:39:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.370+0000 7f767bb92740 1 -- 172.21.15.43:0/1564936292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7297f78d0 msgr2=0x55c7297f6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.370+0000 7f767bb92740 1 --2- 172.21.15.43:0/1564936292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7297f78d0 0x55c7297f6620 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f76640ceba0 tx=0x7f766409c5e0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.370+0000 7f767bb92740 1 -- 172.21.15.43:0/1564936292 shutdown_connections 2024-04-21T08:39:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.370+0000 7f767bb92740 1 --2- 172.21.15.43:0/1564936292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7298000e0 0x55c7298024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.370+0000 7f767bb92740 1 --2- 172.21.15.43:0/1564936292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7297f6b60 0x55c7297f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.370+0000 7f767bb92740 1 --2- 172.21.15.43:0/1564936292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7297f78d0 0x55c7297f6620 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.370+0000 7f767bb92740 1 -- 172.21.15.43:0/1564936292 >> 172.21.15.43:0/1564936292 conn(0x55c7297bdc90 msgr2=0x55c7297f1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.370+0000 7f767bb92740 1 -- 172.21.15.43:0/1564936292 shutdown_connections 2024-04-21T08:39:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.370+0000 7f767bb92740 1 -- 172.21.15.43:0/1564936292 wait complete. 2024-04-21T08:39:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.370+0000 7f767bb92740 1 -- start start 2024-04-21T08:39:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.370+0000 7f767bb92740 1 Processor -- start 2024-04-21T08:39:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.370+0000 7f767bb92740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7297f6b60 0x55c7297ffb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.370+0000 7f767bb92740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7297f78d0 0x55c7297fcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.370+0000 7f767bb92740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7298000e0 0x55c7297fd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.370+0000 7f767bb92740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c729802e10 con 0x55c7298000e0 2024-04-21T08:39:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.370+0000 7f767bb92740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c7296e3eb0 con 0x55c7297f78d0 2024-04-21T08:39:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.370+0000 7f767bb92740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c729802f90 con 0x55c7297f6b60 2024-04-21T08:39:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.371+0000 7f767bb92740 10 client.0 _ll_register_callbacks cb 0x55c729710b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.371+0000 7f7672d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7298000e0 0x55c7297fd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.371+0000 7f7672d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7298000e0 0x55c7297fd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48800/0 (socket says 172.21.15.43:48800) 2024-04-21T08:39:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.371+0000 7f7672d9d640 1 -- 172.21.15.43:0/1712147418 learned_addr learned my addr 172.21.15.43:0/1712147418 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.371+0000 7f767259c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7297f6b60 0x55c7297ffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.371+0000 7f7671d9b640 1 --2- 172.21.15.43:0/1712147418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7297f78d0 0x55c7297fcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.371+0000 7f7672d9d640 1 -- 172.21.15.43:0/1712147418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7297f6b60 msgr2=0x55c7297ffb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.371+0000 7f7672d9d640 1 --2- 172.21.15.43:0/1712147418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7297f6b60 0x55c7297ffb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.371+0000 7f7672d9d640 1 -- 172.21.15.43:0/1712147418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7297f78d0 msgr2=0x55c7297fcf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.371+0000 7f7672d9d640 1 --2- 172.21.15.43:0/1712147418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7297f78d0 0x55c7297fcf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.371+0000 7f7672d9d640 1 -- 172.21.15.43:0/1712147418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c72946e210 con 0x55c7298000e0 2024-04-21T08:39:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.371+0000 7f767259c640 1 --2- 172.21.15.43:0/1712147418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7297f6b60 0x55c7297ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.371+0000 7f7672d9d640 1 --2- 172.21.15.43:0/1712147418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7298000e0 0x55c7297fd600 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f766c02c700 tx=0x7f766c032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade48e25b8d106e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.371+0000 7f7659ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.371+0000 7f7659ffb640 1 -- 172.21.15.43:0/1712147418 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c043070 con 0x55c7298000e0 2024-04-21T08:39:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.371+0000 7f7659ffb640 1 -- 172.21.15.43:0/1712147418 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f766c0326c0 con 0x55c7298000e0 2024-04-21T08:39:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.371+0000 7f767bb92740 1 -- 172.21.15.43:0/1712147418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f766409cc80 con 0x55c7298000e0 2024-04-21T08:39:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.372+0000 7f7659ffb640 1 -- 172.21.15.43:0/1712147418 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f766c04f050 con 0x55c7298000e0 2024-04-21T08:39:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.372+0000 7f767bb92740 10 client.16050 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.372+0000 7f767bb92740 1 -- 172.21.15.43:0/1712147418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c72953a620 con 0x55c7298000e0 2024-04-21T08:39:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.372+0000 7f7659ffb640 1 -- 172.21.15.43:0/1712147418 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f766409cc80 con 0x55c7298000e0 2024-04-21T08:39:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.372+0000 7f7659ffb640 1 -- 172.21.15.43:0/1712147418 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f766c0041d0 con 0x55c7298000e0 2024-04-21T08:39:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.372+0000 7f767bb92740 10 client.16050 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.372+0000 7f767bb92740 10 client.16050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.372+0000 7f767bb92740 20 client.16050 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.372+0000 7f767bb92740 1 --2- 172.21.15.43:0/1712147418 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c729810140 0x55c729812530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.372+0000 7f767bb92740 4 client.16050 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.372+0000 7f767bb92740 1 -- 172.21.15.43:0/1712147418 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c72976f880 con 0x55c729810140 2024-04-21T08:39:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.372+0000 7f767259c640 1 --2- 172.21.15.43:0/1712147418 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c729810140 0x55c729812530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.373+0000 7f7659ffb640 10 client.16050 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.373+0000 7f767259c640 1 --2- 172.21.15.43:0/1712147418 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c729810140 0x55c729812530 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=752bcb7947c9be0e server_cookie=4429840772914d2f in_seq=0 out_seq=0 2024-04-21T08:39:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.373+0000 7f7659ffb640 1 -- 172.21.15.43:0/1712147418 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c72976f880 con 0x55c729810140 2024-04-21T08:39:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.373+0000 7f7659ffb640 10 client.16050 handle_command_reply: tid=0 2024-04-21T08:39:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.373+0000 7f767bb92740 1 client.16050 shutdown 2024-04-21T08:39:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.373+0000 7f767bb92740 1 -- 172.21.15.43:0/1712147418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7298000e0 msgr2=0x55c7297fd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.373+0000 7f767bb92740 1 --2- 172.21.15.43:0/1712147418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7298000e0 0x55c7297fd600 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f766c02c700 tx=0x7f766c032dd0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.374+0000 7f767bb92740 1 -- 172.21.15.43:0/1712147418 shutdown_connections 2024-04-21T08:39:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.374+0000 7f767bb92740 1 --2- 172.21.15.43:0/1712147418 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c729810140 0x55c729812530 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.374+0000 7f767bb92740 1 --2- 172.21.15.43:0/1712147418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7298000e0 0x55c7297fd600 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.374+0000 7f767bb92740 1 --2- 172.21.15.43:0/1712147418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7297f78d0 0x55c7297fcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.374+0000 7f767bb92740 1 --2- 172.21.15.43:0/1712147418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7297f6b60 0x55c7297ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.374+0000 7f7659ffb640 0 client.16050 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.374+0000 7f767bb92740 1 -- 172.21.15.43:0/1712147418 >> 172.21.15.43:0/1712147418 conn(0x55c7297bdc90 msgr2=0x55c7297f35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.374+0000 7f767bb92740 1 -- 172.21.15.43:0/1712147418 shutdown_connections 2024-04-21T08:39:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.374+0000 7f767bb92740 1 -- 172.21.15.43:0/1712147418 wait complete. 2024-04-21T08:39:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.374+0000 7f767bb92740 20 client.16050 trim_cache size 0 max 16384 2024-04-21T08:39:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.389+0000 7f767bb92740 1 Processor -- start 2024-04-21T08:39:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.389+0000 7f767bb92740 1 -- start start 2024-04-21T08:39:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.389+0000 7f767bb92740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7297f6b60 0x55c7297fceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.389+0000 7f767bb92740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7297f78d0 0x55c72980bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.389+0000 7f767bb92740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7298000e0 0x55c7297fbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.389+0000 7f767bb92740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f76741af350 con 0x55c7297f6b60 2024-04-21T08:39:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.389+0000 7f767bb92740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f76741b0e90 con 0x55c7297f78d0 2024-04-21T08:39:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.389+0000 7f767bb92740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f76741afa20 con 0x55c7298000e0 2024-04-21T08:39:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.389+0000 7f767259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7297f6b60 0x55c7297fceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.389+0000 7f767259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7297f6b60 0x55c7297fceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48802/0 (socket says 172.21.15.43:48802) 2024-04-21T08:39:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.389+0000 7f767259c640 1 -- 172.21.15.43:0/1862364427 learned_addr learned my addr 172.21.15.43:0/1862364427 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.389+0000 7f7672d9d640 1 --2- 172.21.15.43:0/1862364427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7298000e0 0x55c7297fbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.389+0000 7f7671d9b640 1 --2- 172.21.15.43:0/1862364427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7297f78d0 0x55c72980bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.390+0000 7f767259c640 1 -- 172.21.15.43:0/1862364427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7298000e0 msgr2=0x55c7297fbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.390+0000 7f767259c640 1 --2- 172.21.15.43:0/1862364427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7298000e0 0x55c7297fbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.390+0000 7f767259c640 1 -- 172.21.15.43:0/1862364427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7297f78d0 msgr2=0x55c72980bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.390+0000 7f767259c640 1 --2- 172.21.15.43:0/1862364427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7297f78d0 0x55c72980bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.390+0000 7f767259c640 1 -- 172.21.15.43:0/1862364427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7297f3b50 con 0x55c7297f6b60 2024-04-21T08:39:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.390+0000 7f767259c640 1 --2- 172.21.15.43:0/1862364427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7297f6b60 0x55c7297fceb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f766400c6a0 tx=0x7f7664004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40e6bb00d498a78 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.390+0000 7f765bfff640 1 -- 172.21.15.43:0/1862364427 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76640b6040 con 0x55c7297f6b60 2024-04-21T08:39:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.390+0000 7f765bfff640 1 -- 172.21.15.43:0/1862364427 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f766409cc80 con 0x55c7297f6b60 2024-04-21T08:39:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.390+0000 7f765bfff640 1 -- 172.21.15.43:0/1862364427 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76640a58f0 con 0x55c7297f6b60 2024-04-21T08:39:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.391+0000 7f767bb92740 1 -- 172.21.15.43:0/1862364427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7297f6b60 msgr2=0x55c7297fceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.391+0000 7f767bb92740 1 --2- 172.21.15.43:0/1862364427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7297f6b60 0x55c7297fceb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f766400c6a0 tx=0x7f7664004a90 comp rx=0 tx=0).stop 2024-04-21T08:39:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.391+0000 7f767bb92740 1 -- 172.21.15.43:0/1862364427 shutdown_connections 2024-04-21T08:39:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.391+0000 7f767bb92740 1 --2- 172.21.15.43:0/1862364427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7298000e0 0x55c7297fbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.391+0000 7f767bb92740 1 --2- 172.21.15.43:0/1862364427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7297f78d0 0x55c72980bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.391+0000 7f767bb92740 1 --2- 172.21.15.43:0/1862364427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7297f6b60 0x55c7297fceb0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.391+0000 7f767bb92740 1 -- 172.21.15.43:0/1862364427 >> 172.21.15.43:0/1862364427 conn(0x55c7297bdc90 msgr2=0x55c7297fc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.391+0000 7f767bb92740 1 -- 172.21.15.43:0/1862364427 shutdown_connections 2024-04-21T08:39:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.391+0000 7f767bb92740 1 -- 172.21.15.43:0/1862364427 wait complete. 2024-04-21T08:39:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.391+0000 7f767bb92740 1 -- start start 2024-04-21T08:39:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.391+0000 7f767bb92740 1 Processor -- start 2024-04-21T08:39:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.392+0000 7f767bb92740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7297f6b60 0x55c729808700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.392+0000 7f767bb92740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7297f78d0 0x55c7297fa190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.392+0000 7f767bb92740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7298000e0 0x55c7297fa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.392+0000 7f767bb92740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f76741b07c0 con 0x55c7297f6b60 2024-04-21T08:39:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.392+0000 7f767bb92740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f76741b00f0 con 0x55c7298000e0 2024-04-21T08:39:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.392+0000 7f767bb92740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f76741ad330 con 0x55c7297f78d0 2024-04-21T08:39:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.392+0000 7f767bb92740 10 client.0 _ll_register_callbacks cb 0x7f7668005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.392+0000 7f767259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7297f6b60 0x55c729808700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.392+0000 7f767259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7297f6b60 0x55c729808700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48816/0 (socket says 172.21.15.43:48816) 2024-04-21T08:39:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.392+0000 7f767259c640 1 -- 172.21.15.43:0/2854926896 learned_addr learned my addr 172.21.15.43:0/2854926896 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.392+0000 7f7671d9b640 1 --2- 172.21.15.43:0/2854926896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7297f78d0 0x55c7297fa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.392+0000 7f7672d9d640 1 --2- 172.21.15.43:0/2854926896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7298000e0 0x55c7297fa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.392+0000 7f767259c640 1 -- 172.21.15.43:0/2854926896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7297f78d0 msgr2=0x55c7297fa190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.392+0000 7f767259c640 1 --2- 172.21.15.43:0/2854926896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7297f78d0 0x55c7297fa190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.392+0000 7f767259c640 1 -- 172.21.15.43:0/2854926896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7298000e0 msgr2=0x55c7297fa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.392+0000 7f767259c640 1 --2- 172.21.15.43:0/2854926896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7298000e0 0x55c7297fa730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.392+0000 7f767259c640 1 -- 172.21.15.43:0/2854926896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7297faf60 con 0x55c7297f6b60 2024-04-21T08:39:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.392+0000 7f7672d9d640 1 --2- 172.21.15.43:0/2854926896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7298000e0 0x55c7297fa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.392+0000 7f767259c640 1 --2- 172.21.15.43:0/2854926896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7297f6b60 0x55c729808700 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f766409af30 tx=0x7f766409c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7419bc0bbaac3ece server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.392+0000 7f7659ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.392+0000 7f7659ffb640 1 -- 172.21.15.43:0/2854926896 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76640b6040 con 0x55c7297f6b60 2024-04-21T08:39:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.392+0000 7f7659ffb640 1 -- 172.21.15.43:0/2854926896 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f76640ce790 con 0x55c7297f6b60 2024-04-21T08:39:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.392+0000 7f767bb92740 1 -- 172.21.15.43:0/2854926896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f766409cc80 con 0x55c7297f6b60 2024-04-21T08:39:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.393+0000 7f7659ffb640 1 -- 172.21.15.43:0/2854926896 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f76640b61e0 con 0x55c7297f6b60 2024-04-21T08:39:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.393+0000 7f767bb92740 10 client.16056 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.393+0000 7f767bb92740 1 -- 172.21.15.43:0/2854926896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c729876770 con 0x55c7297f6b60 2024-04-21T08:39:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.393+0000 7f7659ffb640 1 -- 172.21.15.43:0/2854926896 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f766409cc80 con 0x55c7297f6b60 2024-04-21T08:39:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.393+0000 7f7659ffb640 1 -- 172.21.15.43:0/2854926896 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f76640bd0d0 con 0x55c7297f6b60 2024-04-21T08:39:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.393+0000 7f767bb92740 10 client.16056 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.393+0000 7f767bb92740 10 client.16056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.393+0000 7f767bb92740 20 client.16056 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.393+0000 7f767bb92740 1 --2- 172.21.15.43:0/2854926896 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c7298769f0 0x55c729878de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.393+0000 7f767bb92740 4 client.16056 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:39:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.393+0000 7f767bb92740 1 -- 172.21.15.43:0/2854926896 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c7296fb3d0 con 0x55c7298769f0 2024-04-21T08:39:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.393+0000 7f7671d9b640 1 --2- 172.21.15.43:0/2854926896 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c7298769f0 0x55c729878de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.394+0000 7f7659ffb640 10 client.16056 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.394+0000 7f7671d9b640 1 --2- 172.21.15.43:0/2854926896 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c7298769f0 0x55c729878de0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69870ced9ca8f9a4 server_cookie=5232c0783b2aa73f in_seq=0 out_seq=0 2024-04-21T08:39:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.397+0000 7f7659ffb640 1 -- 172.21.15.43:0/2854926896 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c7296fb3d0 con 0x55c7298769f0 2024-04-21T08:39:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.397+0000 7f7659ffb640 10 client.16056 handle_command_reply: tid=0 2024-04-21T08:39:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.397+0000 7f767bb92740 1 client.16056 shutdown 2024-04-21T08:39:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.397+0000 7f767bb92740 1 -- 172.21.15.43:0/2854926896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7297f6b60 msgr2=0x55c729808700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.397+0000 7f767bb92740 1 --2- 172.21.15.43:0/2854926896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7297f6b60 0x55c729808700 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f766409af30 tx=0x7f766409c710 comp rx=0 tx=0).stop 2024-04-21T08:39:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.397+0000 7f767bb92740 1 -- 172.21.15.43:0/2854926896 shutdown_connections 2024-04-21T08:39:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.397+0000 7f767bb92740 1 --2- 172.21.15.43:0/2854926896 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c7298769f0 0x55c729878de0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.397+0000 7f7659ffb640 0 client.16056 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.397+0000 7f767bb92740 1 --2- 172.21.15.43:0/2854926896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7298000e0 0x55c7297fa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.397+0000 7f767bb92740 1 --2- 172.21.15.43:0/2854926896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7297f78d0 0x55c7297fa190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.397+0000 7f767bb92740 1 --2- 172.21.15.43:0/2854926896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c7297f6b60 0x55c729808700 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.397+0000 7f767bb92740 1 -- 172.21.15.43:0/2854926896 >> 172.21.15.43:0/2854926896 conn(0x55c7297bdc90 msgr2=0x55c7297f29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.397+0000 7f767bb92740 1 -- 172.21.15.43:0/2854926896 shutdown_connections 2024-04-21T08:39:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.397+0000 7f767bb92740 1 -- 172.21.15.43:0/2854926896 wait complete. 2024-04-21T08:39:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.397+0000 7f767bb92740 20 client.16056 trim_cache size 0 max 16384 2024-04-21T08:39:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:39:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:39:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:39:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:39:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.398+0000 7f7678ec1640 1 -- 172.21.15.43:0/1910257518 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f765c0a3e60 msgr2=0x7f765c0a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.398+0000 7f7678ec1640 1 --2- 172.21.15.43:0/1910257518 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f765c0a3e60 0x7f765c0a6320 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7674100cc0 tx=0x7f7668005f90 comp rx=0 tx=0).stop 2024-04-21T08:39:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.398+0000 7f7678ec1640 1 -- 172.21.15.43:0/1910257518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76741010f0 msgr2=0x7f76741986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.398+0000 7f7678ec1640 1 --2- 172.21.15.43:0/1910257518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76741010f0 0x7f76741986c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f766400e9f0 tx=0x7f766400edf0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.398+0000 7f7678ec1640 1 -- 172.21.15.43:0/1910257518 shutdown_connections 2024-04-21T08:39:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.398+0000 7f7678ec1640 1 --2- 172.21.15.43:0/1910257518 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f765c0a3e60 0x7f765c0a6320 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.398+0000 7f7678ec1640 1 --2- 172.21.15.43:0/1910257518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f767419cee0 0x7f767419d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.398+0000 7f7678ec1640 1 --2- 172.21.15.43:0/1910257518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7674106170 0x7f7674198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.398+0000 7f7678ec1640 1 --2- 172.21.15.43:0/1910257518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76741010f0 0x7f76741986c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.398+0000 7f7678ec1640 1 -- 172.21.15.43:0/1910257518 >> 172.21.15.43:0/1910257518 conn(0x7f7674077fd0 msgr2=0x7f76740698e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.398+0000 7f7678ec1640 1 -- 172.21.15.43:0/1910257518 shutdown_connections 2024-04-21T08:39:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:28.398+0000 7f7678ec1640 1 -- 172.21.15.43:0/1910257518 wait complete. 2024-04-21T08:39:28.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:28 smithi112 ceph-mon[31085]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 17 op/s 2024-04-21T08:39:28.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:28 smithi043 ceph-mon[27978]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 17 op/s 2024-04-21T08:39:28.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:28 smithi175 ceph-mon[31920]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 17 op/s 2024-04-21T08:39:30.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:30 smithi112 ceph-mon[31085]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:39:30.805 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:39:30.805 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:39:30.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:30 smithi043 ceph-mon[27978]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:39:30.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:30 smithi175 ceph-mon[31920]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:39:30.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.967+0000 7fa4a796d640 1 -- 172.21.15.43:0/3880576776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4a0101a30 msgr2=0x7fa4a0105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:30.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.967+0000 7fa4a796d640 1 --2- 172.21.15.43:0/3880576776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4a0101a30 0x7fa4a0105a70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa490009860 tx=0x7fa49002bb70 comp rx=0 tx=0).stop 2024-04-21T08:39:30.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.967+0000 7fa4a796d640 1 -- 172.21.15.43:0/3880576776 shutdown_connections 2024-04-21T08:39:30.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.967+0000 7fa4a796d640 1 --2- 172.21.15.43:0/3880576776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a0106150 0x7fa4a010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:30.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.967+0000 7fa4a796d640 1 --2- 172.21.15.43:0/3880576776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4a0101a30 0x7fa4a0105a70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:30.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.967+0000 7fa4a796d640 1 --2- 172.21.15.43:0/3880576776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a0101080 0x7fa4a0101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:30.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.967+0000 7fa4a796d640 1 -- 172.21.15.43:0/3880576776 >> 172.21.15.43:0/3880576776 conn(0x7fa4a0078000 msgr2=0x7fa4a0078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:30.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.967+0000 7fa4a796d640 1 -- 172.21.15.43:0/3880576776 shutdown_connections 2024-04-21T08:39:30.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.967+0000 7fa4a796d640 1 -- 172.21.15.43:0/3880576776 wait complete. 2024-04-21T08:39:30.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.968+0000 7fa4a796d640 1 Processor -- start 2024-04-21T08:39:30.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.968+0000 7fa4a796d640 1 -- start start 2024-04-21T08:39:30.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.968+0000 7fa4a796d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4a0101080 0x7fa4a01984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:30.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.968+0000 7fa4a796d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a0106150 0x7fa4a0198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:30.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.968+0000 7fa4a796d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a019cd90 0x7fa4a019d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:30.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.968+0000 7fa4a796d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a0111040 con 0x7fa4a019cd90 2024-04-21T08:39:30.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.968+0000 7fa4a796d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a0110ec0 con 0x7fa4a0106150 2024-04-21T08:39:30.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.968+0000 7fa4a796d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a01111c0 con 0x7fa4a0101080 2024-04-21T08:39:30.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.968+0000 7fa4a5f0a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a019cd90 0x7fa4a019d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:30.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.968+0000 7fa4a5f0a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a019cd90 0x7fa4a019d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48838/0 (socket says 172.21.15.43:48838) 2024-04-21T08:39:30.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.968+0000 7fa4a5f0a640 1 -- 172.21.15.43:0/3861097119 learned_addr learned my addr 172.21.15.43:0/3861097119 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:30.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.969+0000 7fa4a4f08640 1 --2- 172.21.15.43:0/3861097119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a0106150 0x7fa4a0198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:30.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.969+0000 7fa4a5709640 1 --2- 172.21.15.43:0/3861097119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4a0101080 0x7fa4a01984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:30.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.969+0000 7fa4a5f0a640 1 -- 172.21.15.43:0/3861097119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4a0101080 msgr2=0x7fa4a01984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:30.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.969+0000 7fa4a5f0a640 1 --2- 172.21.15.43:0/3861097119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4a0101080 0x7fa4a01984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:30.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.969+0000 7fa4a5f0a640 1 -- 172.21.15.43:0/3861097119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a0106150 msgr2=0x7fa4a0198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:30.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.969+0000 7fa4a5f0a640 1 --2- 172.21.15.43:0/3861097119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a0106150 0x7fa4a0198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:30.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.969+0000 7fa4a5f0a640 1 -- 172.21.15.43:0/3861097119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a019d880 con 0x7fa4a019cd90 2024-04-21T08:39:30.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.969+0000 7fa4a4f08640 1 --2- 172.21.15.43:0/3861097119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a0106150 0x7fa4a0198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:30.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.969+0000 7fa4a5709640 1 --2- 172.21.15.43:0/3861097119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4a0101080 0x7fa4a01984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:30.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.969+0000 7fa4a5f0a640 1 --2- 172.21.15.43:0/3861097119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a019cd90 0x7fa4a019d240 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa49800eb10 tx=0x7fa49800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:30.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.969+0000 7fa4967fc640 1 -- 172.21.15.43:0/3861097119 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498004b20 con 0x7fa4a019cd90 2024-04-21T08:39:30.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.969+0000 7fa4967fc640 1 -- 172.21.15.43:0/3861097119 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa498004cc0 con 0x7fa4a019cd90 2024-04-21T08:39:30.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.969+0000 7fa4a796d640 1 -- 172.21.15.43:0/3861097119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a019db70 con 0x7fa4a019cd90 2024-04-21T08:39:30.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.970+0000 7fa4967fc640 1 -- 172.21.15.43:0/3861097119 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498010670 con 0x7fa4a019cd90 2024-04-21T08:39:30.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.970+0000 7fa4a796d640 1 -- 172.21.15.43:0/3861097119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a00fee40 con 0x7fa4a019cd90 2024-04-21T08:39:30.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.973+0000 7fa4967fc640 1 -- 172.21.15.43:0/3861097119 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fa498005000 con 0x7fa4a019cd90 2024-04-21T08:39:30.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.974+0000 7fa4967fc640 1 --2- 172.21.15.43:0/3861097119 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa48c0a3e30 0x7fa48c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:30.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.974+0000 7fa4aa633740 1 Processor -- start 2024-04-21T08:39:30.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.974+0000 7fa4a5709640 1 --2- 172.21.15.43:0/3861097119 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa48c0a3e30 0x7fa48c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:30.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.974+0000 7fa4aa633740 1 -- start start 2024-04-21T08:39:30.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.975+0000 7fa4aa633740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562971ea7f60 0x562971ea8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:30.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.975+0000 7fa4aa633740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562971ea8910 0x562971eb9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:30.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.975+0000 7fa4aa633740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971eba290 0x562971ebc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:30.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.975+0000 7fa4aa633740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562971e0d1e0 con 0x562971eba290 2024-04-21T08:39:30.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.975+0000 7fa4aa633740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562971daa390 con 0x562971ea7f60 2024-04-21T08:39:30.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.975+0000 7fa4aa633740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562971e091a0 con 0x562971ea8910 2024-04-21T08:39:30.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.975+0000 7fa4a5f0a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971eba290 0x562971ebc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:30.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.975+0000 7fa4a5f0a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971eba290 0x562971ebc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48844/0 (socket says 172.21.15.43:48844) 2024-04-21T08:39:30.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.975+0000 7fa4a5f0a640 1 -- 172.21.15.43:0/778118227 learned_addr learned my addr 172.21.15.43:0/778118227 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:30.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.975+0000 7fa4a4f08640 1 --2- 172.21.15.43:0/778118227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562971ea8910 0x562971eb9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:30.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.975+0000 7fa4a5709640 1 --2- 172.21.15.43:0/3861097119 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa48c0a3e30 0x7fa48c0a62f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0096c0 tx=0x7fa49c009260 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:30.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.975+0000 7fa4a5709640 1 --2- 172.21.15.43:0/778118227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562971ea7f60 0x562971ea8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:30.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.975+0000 7fa4967fc640 1 -- 172.21.15.43:0/3861097119 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fa4980d11e0 con 0x7fa4a019cd90 2024-04-21T08:39:30.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.975+0000 7fa4a5f0a640 1 -- 172.21.15.43:0/778118227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562971ea8910 msgr2=0x562971eb9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:30.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.975+0000 7fa4a5f0a640 1 --2- 172.21.15.43:0/778118227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562971ea8910 0x562971eb9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:30.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.975+0000 7fa4a5f0a640 1 -- 172.21.15.43:0/778118227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562971ea7f60 msgr2=0x562971ea8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:30.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.975+0000 7fa4a5f0a640 1 --2- 172.21.15.43:0/778118227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562971ea7f60 0x562971ea8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:30.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.975+0000 7fa4a5f0a640 1 -- 172.21.15.43:0/778118227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562971be97d0 con 0x562971eba290 2024-04-21T08:39:30.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.975+0000 7fa4a4f08640 1 --2- 172.21.15.43:0/778118227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562971ea8910 0x562971eb9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:30.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.976+0000 7fa4a5f0a640 1 --2- 172.21.15.43:0/778118227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971eba290 0x562971ebc680 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa49801c000 tx=0x7fa4980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f52e33b383de5f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:30.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.976+0000 7fa48b7fe640 1 -- 172.21.15.43:0/778118227 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49809c5a0 con 0x562971eba290 2024-04-21T08:39:30.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.976+0000 7fa48b7fe640 1 -- 172.21.15.43:0/778118227 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa498087040 con 0x562971eba290 2024-04-21T08:39:30.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.976+0000 7fa48b7fe640 1 -- 172.21.15.43:0/778118227 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4980120f0 con 0x562971eba290 2024-04-21T08:39:30.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.977+0000 7fa4aa633740 1 -- 172.21.15.43:0/778118227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971eba290 msgr2=0x562971ebc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:30.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.977+0000 7fa4aa633740 1 --2- 172.21.15.43:0/778118227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971eba290 0x562971ebc680 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa49801c000 tx=0x7fa4980057f0 comp rx=0 tx=0).stop 2024-04-21T08:39:30.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.977+0000 7fa4aa633740 1 -- 172.21.15.43:0/778118227 shutdown_connections 2024-04-21T08:39:30.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.977+0000 7fa4aa633740 1 --2- 172.21.15.43:0/778118227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971eba290 0x562971ebc680 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:30.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.977+0000 7fa4aa633740 1 --2- 172.21.15.43:0/778118227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562971ea8910 0x562971eb9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:30.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.977+0000 7fa4aa633740 1 --2- 172.21.15.43:0/778118227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562971ea7f60 0x562971ea8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:30.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.977+0000 7fa4aa633740 1 -- 172.21.15.43:0/778118227 >> 172.21.15.43:0/778118227 conn(0x562971ea2cb0 msgr2=0x562971ea3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:30.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.977+0000 7fa4aa633740 1 -- 172.21.15.43:0/778118227 shutdown_connections 2024-04-21T08:39:30.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.977+0000 7fa4aa633740 1 -- 172.21.15.43:0/778118227 wait complete. 2024-04-21T08:39:30.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.978+0000 7fa4aa633740 1 -- start start 2024-04-21T08:39:30.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.978+0000 7fa4aa633740 1 Processor -- start 2024-04-21T08:39:30.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.978+0000 7fa4aa633740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971ea7f60 0x562971eb3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:30.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.978+0000 7fa4aa633740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562971ea8910 0x562971eb9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:30.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.978+0000 7fa4aa633740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562971eba290 0x562971eb4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:30.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.978+0000 7fa4aa633740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562971ea98e0 con 0x562971ea7f60 2024-04-21T08:39:30.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.978+0000 7fa4aa633740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562971dc62f0 con 0x562971ea8910 2024-04-21T08:39:30.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.978+0000 7fa4a5709640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971ea7f60 0x562971eb3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:30.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.978+0000 7fa4aa633740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562971ea9a60 con 0x562971eba290 2024-04-21T08:39:30.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.978+0000 7fa4a5709640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971ea7f60 0x562971eb3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48858/0 (socket says 172.21.15.43:48858) 2024-04-21T08:39:30.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.978+0000 7fa4a5709640 1 -- 172.21.15.43:0/2329012751 learned_addr learned my addr 172.21.15.43:0/2329012751 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:30.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.978+0000 7fa4aa633740 10 client.0 _ll_register_callbacks cb 0x562971dd3580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:30.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.979+0000 7fa4a4f08640 1 --2- 172.21.15.43:0/2329012751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562971ea8910 0x562971eb9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:30.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.979+0000 7fa4a5f0a640 1 --2- 172.21.15.43:0/2329012751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562971eba290 0x562971eb4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:30.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.979+0000 7fa4a5709640 1 -- 172.21.15.43:0/2329012751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562971eba290 msgr2=0x562971eb4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:30.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.979+0000 7fa4a5709640 1 --2- 172.21.15.43:0/2329012751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562971eba290 0x562971eb4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:30.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.979+0000 7fa4a5709640 1 -- 172.21.15.43:0/2329012751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562971ea8910 msgr2=0x562971eb9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:30.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.979+0000 7fa4a5709640 1 --2- 172.21.15.43:0/2329012751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562971ea8910 0x562971eb9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:30.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.979+0000 7fa4a5709640 1 -- 172.21.15.43:0/2329012751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562971b1f3f0 con 0x562971ea7f60 2024-04-21T08:39:30.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.979+0000 7fa4a5709640 1 --2- 172.21.15.43:0/2329012751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971ea7f60 0x562971eb3f30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00fbf0 tx=0x7fa49c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caee12277c1ed690 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:30.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.979+0000 7fa488ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:30.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.979+0000 7fa488ff9640 1 -- 172.21.15.43:0/2329012751 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c02c060 con 0x562971ea7f60 2024-04-21T08:39:30.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.980+0000 7fa488ff9640 1 -- 172.21.15.43:0/2329012751 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fa49c0273f0 con 0x562971ea7f60 2024-04-21T08:39:30.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.980+0000 7fa4aa633740 1 -- 172.21.15.43:0/2329012751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa498087040 con 0x562971ea7f60 2024-04-21T08:39:30.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.980+0000 7fa488ff9640 1 -- 172.21.15.43:0/2329012751 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa49c005020 con 0x562971ea7f60 2024-04-21T08:39:30.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.980+0000 7fa4aa633740 10 client.16068 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:30.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.980+0000 7fa4aa633740 1 -- 172.21.15.43:0/2329012751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562971adf0f0 con 0x562971ea7f60 2024-04-21T08:39:30.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.980+0000 7fa488ff9640 1 -- 172.21.15.43:0/2329012751 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa498087040 con 0x562971ea7f60 2024-04-21T08:39:30.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.980+0000 7fa488ff9640 1 -- 172.21.15.43:0/2329012751 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa49c026070 con 0x562971ea7f60 2024-04-21T08:39:30.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.980+0000 7fa4aa633740 10 client.16068 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:30.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.980+0000 7fa4aa633740 10 client.16068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:30.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.980+0000 7fa4aa633740 20 client.16068 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:30.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.980+0000 7fa4aa633740 1 --2- 172.21.15.43:0/2329012751 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562971ed28d0 0x562971ed4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:30.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.980+0000 7fa4aa633740 4 client.16068 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:30.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.980+0000 7fa4aa633740 1 -- 172.21.15.43:0/2329012751 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562971e1ac00 con 0x562971ed28d0 2024-04-21T08:39:30.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.981+0000 7fa4a4f08640 1 --2- 172.21.15.43:0/2329012751 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562971ed28d0 0x562971ed4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:30.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.981+0000 7fa488ff9640 10 client.16068 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:30.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.981+0000 7fa4a4f08640 1 --2- 172.21.15.43:0/2329012751 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562971ed28d0 0x562971ed4cc0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1ee49e11eb2e5dd server_cookie=929ab3378a6f9677 in_seq=0 out_seq=0 2024-04-21T08:39:30.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.982+0000 7fa488ff9640 1 -- 172.21.15.43:0/2329012751 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562971e1ac00 con 0x562971ed28d0 2024-04-21T08:39:30.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.982+0000 7fa488ff9640 10 client.16068 handle_command_reply: tid=0 2024-04-21T08:39:30.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.982+0000 7fa4aa633740 1 client.16068 shutdown 2024-04-21T08:39:30.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.982+0000 7fa4aa633740 1 -- 172.21.15.43:0/2329012751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971ea7f60 msgr2=0x562971eb3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:30.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.982+0000 7fa4aa633740 1 --2- 172.21.15.43:0/2329012751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971ea7f60 0x562971eb3f30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00fbf0 tx=0x7fa49c013040 comp rx=0 tx=0).stop 2024-04-21T08:39:30.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.982+0000 7fa4aa633740 1 -- 172.21.15.43:0/2329012751 shutdown_connections 2024-04-21T08:39:30.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.982+0000 7fa4aa633740 1 --2- 172.21.15.43:0/2329012751 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562971ed28d0 0x562971ed4cc0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:30.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.982+0000 7fa4aa633740 1 --2- 172.21.15.43:0/2329012751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562971eba290 0x562971eb4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:30.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.982+0000 7fa488ff9640 0 client.16068 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:30.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.982+0000 7fa4aa633740 1 --2- 172.21.15.43:0/2329012751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562971ea8910 0x562971eb9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:30.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.982+0000 7fa4aa633740 1 --2- 172.21.15.43:0/2329012751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971ea7f60 0x562971eb3f30 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:30.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.982+0000 7fa4aa633740 1 -- 172.21.15.43:0/2329012751 >> 172.21.15.43:0/2329012751 conn(0x562971ea2cb0 msgr2=0x562971ea4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:30.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.982+0000 7fa4aa633740 1 -- 172.21.15.43:0/2329012751 shutdown_connections 2024-04-21T08:39:30.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.983+0000 7fa4aa633740 1 -- 172.21.15.43:0/2329012751 wait complete. 2024-04-21T08:39:30.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:30.983+0000 7fa4aa633740 20 client.16068 trim_cache size 0 max 16384 2024-04-21T08:39:31.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.018+0000 7fa4aa633740 1 Processor -- start 2024-04-21T08:39:31.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.018+0000 7fa4aa633740 1 -- start start 2024-04-21T08:39:31.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.018+0000 7fa4aa633740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562971ea7f60 0x562971eb9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.018+0000 7fa4aa633740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562971ea8910 0x562971eda570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.018+0000 7fa4aa633740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971eba290 0x562971edaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.018+0000 7fa4aa633740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a01ad070 con 0x562971eba290 2024-04-21T08:39:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.018+0000 7fa4aa633740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a01aebb0 con 0x562971ea8910 2024-04-21T08:39:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.018+0000 7fa4aa633740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a01ad740 con 0x562971ea7f60 2024-04-21T08:39:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.018+0000 7fa4a5f0a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971eba290 0x562971edaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.018+0000 7fa4a5f0a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971eba290 0x562971edaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48868/0 (socket says 172.21.15.43:48868) 2024-04-21T08:39:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.018+0000 7fa4a5f0a640 1 -- 172.21.15.43:0/1153673722 learned_addr learned my addr 172.21.15.43:0/1153673722 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.019+0000 7fa4a5709640 1 --2- 172.21.15.43:0/1153673722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562971ea7f60 0x562971eb9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.019+0000 7fa4a4f08640 1 --2- 172.21.15.43:0/1153673722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562971ea8910 0x562971eda570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.019+0000 7fa4a5f0a640 1 -- 172.21.15.43:0/1153673722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562971ea7f60 msgr2=0x562971eb9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.019+0000 7fa4a5f0a640 1 --2- 172.21.15.43:0/1153673722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562971ea7f60 0x562971eb9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.019+0000 7fa4a5f0a640 1 -- 172.21.15.43:0/1153673722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562971ea8910 msgr2=0x562971eda570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.019+0000 7fa4a5f0a640 1 --2- 172.21.15.43:0/1153673722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562971ea8910 0x562971eda570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.019+0000 7fa4a5f0a640 1 -- 172.21.15.43:0/1153673722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562971ead880 con 0x562971eba290 2024-04-21T08:39:31.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.019+0000 7fa4a4f08640 1 --2- 172.21.15.43:0/1153673722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562971ea8910 0x562971eda570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:31.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.019+0000 7fa4a5f0a640 1 --2- 172.21.15.43:0/1153673722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971eba290 0x562971edaab0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fa4980049d0 tx=0x7fa49800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adb13bd9e6eae98 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:31.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.019+0000 7fa48affd640 1 -- 172.21.15.43:0/1153673722 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4980b5070 con 0x562971eba290 2024-04-21T08:39:31.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.019+0000 7fa48affd640 1 -- 172.21.15.43:0/1153673722 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa498087040 con 0x562971eba290 2024-04-21T08:39:31.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.019+0000 7fa48affd640 1 -- 172.21.15.43:0/1153673722 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49809cc40 con 0x562971eba290 2024-04-21T08:39:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.020+0000 7fa4aa633740 1 -- 172.21.15.43:0/1153673722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971eba290 msgr2=0x562971edaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.020+0000 7fa4aa633740 1 --2- 172.21.15.43:0/1153673722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971eba290 0x562971edaab0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fa4980049d0 tx=0x7fa49800c950 comp rx=0 tx=0).stop 2024-04-21T08:39:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.020+0000 7fa4aa633740 1 -- 172.21.15.43:0/1153673722 shutdown_connections 2024-04-21T08:39:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.020+0000 7fa4aa633740 1 --2- 172.21.15.43:0/1153673722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971eba290 0x562971edaab0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.020+0000 7fa4aa633740 1 --2- 172.21.15.43:0/1153673722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562971ea8910 0x562971eda570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.020+0000 7fa4aa633740 1 --2- 172.21.15.43:0/1153673722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562971ea7f60 0x562971eb9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.020+0000 7fa4aa633740 1 -- 172.21.15.43:0/1153673722 >> 172.21.15.43:0/1153673722 conn(0x562971ea2cb0 msgr2=0x562971eadc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.020+0000 7fa4aa633740 1 -- 172.21.15.43:0/1153673722 shutdown_connections 2024-04-21T08:39:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.020+0000 7fa4aa633740 1 -- 172.21.15.43:0/1153673722 wait complete. 2024-04-21T08:39:31.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.021+0000 7fa4aa633740 1 -- start start 2024-04-21T08:39:31.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.021+0000 7fa4aa633740 1 Processor -- start 2024-04-21T08:39:31.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.021+0000 7fa4aa633740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562971ea7f60 0x562971ee19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.022+0000 7fa4aa633740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971ea8910 0x562971ee1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.022+0000 7fa4aa633740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562971eba290 0x562971f378c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.022+0000 7fa4a4f08640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971ea8910 0x562971ee1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.022+0000 7fa4a4f08640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971ea8910 0x562971ee1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48876/0 (socket says 172.21.15.43:48876) 2024-04-21T08:39:31.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.022+0000 7fa4a5709640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562971ea7f60 0x562971ee19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.022+0000 7fa4a4f08640 1 -- 172.21.15.43:0/3954115868 learned_addr learned my addr 172.21.15.43:0/3954115868 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:31.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.022+0000 7fa4aa633740 1 -- 172.21.15.43:0/3954115868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a01ae4e0 con 0x562971ea8910 2024-04-21T08:39:31.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.022+0000 7fa4aa633740 1 -- 172.21.15.43:0/3954115868 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a01ade10 con 0x562971ea7f60 2024-04-21T08:39:31.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.022+0000 7fa4aa633740 1 -- 172.21.15.43:0/3954115868 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a01ab050 con 0x562971eba290 2024-04-21T08:39:31.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.022+0000 7fa4a5f0a640 1 --2- 172.21.15.43:0/3954115868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562971eba290 0x562971f378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.022+0000 7fa4a5709640 1 -- 172.21.15.43:0/3954115868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562971eba290 msgr2=0x562971f378c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.022+0000 7fa4a5709640 1 --2- 172.21.15.43:0/3954115868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562971eba290 0x562971f378c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.022+0000 7fa4a5709640 1 -- 172.21.15.43:0/3954115868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971ea8910 msgr2=0x562971ee1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.022+0000 7fa4a5709640 1 --2- 172.21.15.43:0/3954115868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971ea8910 0x562971ee1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.022+0000 7fa4aa633740 10 client.0 _ll_register_callbacks cb 0x7fa49c009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.022+0000 7fa4a5709640 1 -- 172.21.15.43:0/3954115868 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562971f37e00 con 0x562971ea7f60 2024-04-21T08:39:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.022+0000 7fa4a5f0a640 1 --2- 172.21.15.43:0/3954115868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562971eba290 0x562971f378c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.022+0000 7fa4a4f08640 1 --2- 172.21.15.43:0/3954115868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971ea8910 0x562971ee1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.023+0000 7fa4aa633740 1 -- 172.21.15.43:0/3954115868 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa498087040 con 0x562971ea7f60 2024-04-21T08:39:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.023+0000 7fa4a5709640 1 --2- 172.21.15.43:0/3954115868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562971ea7f60 0x562971ee19e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa49c016040 tx=0x7fa49c027c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5393861fa6483fee server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.023+0000 7fa488ff9640 10 client.25577 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-21T08:39:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.023+0000 7fa488ff9640 1 -- 172.21.15.43:0/3954115868 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c02c030 con 0x562971ea7f60 2024-04-21T08:39:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.023+0000 7fa488ff9640 1 -- 172.21.15.43:0/3954115868 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fa49c004020 con 0x562971ea7f60 2024-04-21T08:39:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.023+0000 7fa488ff9640 1 -- 172.21.15.43:0/3954115868 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa49c03d030 con 0x562971ea7f60 2024-04-21T08:39:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.023+0000 7fa4aa633740 10 client.25577 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:31.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.023+0000 7fa4aa633740 1 -- 172.21.15.43:0/3954115868 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({fsmap=28}) -- 0x562971f38dc0 con 0x562971ea7f60 2024-04-21T08:39:31.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.024+0000 7fa488ff9640 1 -- 172.21.15.43:0/3954115868 <== mon.2 v2:172.21.15.112:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa49c026040 con 0x562971ea7f60 2024-04-21T08:39:31.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.024+0000 7fa4aa633740 10 client.25577 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:31.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.024+0000 7fa4aa633740 10 client.25577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:31.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.024+0000 7fa4aa633740 20 client.25577 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:31.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.024+0000 7fa4aa633740 1 --2- 172.21.15.43:0/3954115868 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562971f393e0 0x562971f3b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.024+0000 7fa4aa633740 4 client.25577 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:39:31.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.024+0000 7fa4aa633740 1 -- 172.21.15.43:0/3954115868 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562971e14a70 con 0x562971f393e0 2024-04-21T08:39:31.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.024+0000 7fa4a4f08640 1 --2- 172.21.15.43:0/3954115868 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562971f393e0 0x562971f3b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.025+0000 7fa4a4f08640 1 --2- 172.21.15.43:0/3954115868 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562971f393e0 0x562971f3b860 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be72f5d2610896e4 server_cookie=e380f68dc17568e6 in_seq=0 out_seq=0 2024-04-21T08:39:31.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.025+0000 7fa488ff9640 10 client.25577 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:31.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.025+0000 7fa488ff9640 1 -- 172.21.15.43:0/3954115868 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562971e14a70 con 0x562971f393e0 2024-04-21T08:39:31.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.025+0000 7fa488ff9640 10 client.25577 handle_command_reply: tid=0 2024-04-21T08:39:31.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.025+0000 7fa4aa633740 1 client.25577 shutdown 2024-04-21T08:39:31.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.025+0000 7fa4aa633740 1 -- 172.21.15.43:0/3954115868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562971ea7f60 msgr2=0x562971ee19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.025+0000 7fa4aa633740 1 --2- 172.21.15.43:0/3954115868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562971ea7f60 0x562971ee19e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa49c016040 tx=0x7fa49c027c40 comp rx=0 tx=0).stop 2024-04-21T08:39:31.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.025+0000 7fa4aa633740 1 -- 172.21.15.43:0/3954115868 shutdown_connections 2024-04-21T08:39:31.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.025+0000 7fa4aa633740 1 --2- 172.21.15.43:0/3954115868 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562971f393e0 0x562971f3b860 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.025+0000 7fa488ff9640 0 client.25577 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:31.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.025+0000 7fa4aa633740 1 --2- 172.21.15.43:0/3954115868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562971eba290 0x562971f378c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.025+0000 7fa4aa633740 1 --2- 172.21.15.43:0/3954115868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562971ea8910 0x562971ee1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.025+0000 7fa4aa633740 1 --2- 172.21.15.43:0/3954115868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562971ea7f60 0x562971ee19e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.025+0000 7fa4aa633740 1 -- 172.21.15.43:0/3954115868 >> 172.21.15.43:0/3954115868 conn(0x562971ea2cb0 msgr2=0x562971ea40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:31.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.025+0000 7fa4aa633740 1 -- 172.21.15.43:0/3954115868 shutdown_connections 2024-04-21T08:39:31.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.025+0000 7fa4aa633740 1 -- 172.21.15.43:0/3954115868 wait complete. 2024-04-21T08:39:31.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.026+0000 7fa4aa633740 20 client.25577 trim_cache size 0 max 16384 2024-04-21T08:39:31.031 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:39:31.031 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:39:31.031 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:39:31.031 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:39:31.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.026+0000 7fa4a796d640 1 -- 172.21.15.43:0/3861097119 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa48c0a3e30 msgr2=0x7fa48c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.026+0000 7fa4a796d640 1 --2- 172.21.15.43:0/3861097119 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa48c0a3e30 0x7fa48c0a62f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0096c0 tx=0x7fa49c009260 comp rx=0 tx=0).stop 2024-04-21T08:39:31.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.026+0000 7fa4a796d640 1 -- 172.21.15.43:0/3861097119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a019cd90 msgr2=0x7fa4a019d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.026+0000 7fa4a796d640 1 --2- 172.21.15.43:0/3861097119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a019cd90 0x7fa4a019d240 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa49800eb10 tx=0x7fa49800c2d0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.026+0000 7fa4a796d640 1 -- 172.21.15.43:0/3861097119 shutdown_connections 2024-04-21T08:39:31.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.026+0000 7fa4a796d640 1 --2- 172.21.15.43:0/3861097119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a019cd90 0x7fa4a019d240 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.026+0000 7fa4a796d640 1 --2- 172.21.15.43:0/3861097119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a0106150 0x7fa4a0198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.026+0000 7fa4a796d640 1 --2- 172.21.15.43:0/3861097119 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa48c0a3e30 0x7fa48c0a62f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.026+0000 7fa4a796d640 1 --2- 172.21.15.43:0/3861097119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4a0101080 0x7fa4a01984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.026+0000 7fa4a796d640 1 -- 172.21.15.43:0/3861097119 >> 172.21.15.43:0/3861097119 conn(0x7fa4a0078000 msgr2=0x7fa4a0069280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.027+0000 7fa4a796d640 1 -- 172.21.15.43:0/3861097119 shutdown_connections 2024-04-21T08:39:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.027+0000 7fa4a796d640 1 -- 172.21.15.43:0/3861097119 wait complete. 2024-04-21T08:39:31.040 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:39:31.041 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:39:31.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.187+0000 7fa32de25640 1 -- 172.21.15.43:0/2501842126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa328102190 msgr2=0x7fa328102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.187+0000 7fa32de25640 1 --2- 172.21.15.43:0/2501842126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa328102190 0x7fa328102570 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa318009900 tx=0x7fa31802bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:31.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.187+0000 7fa32de25640 1 -- 172.21.15.43:0/2501842126 shutdown_connections 2024-04-21T08:39:31.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.187+0000 7fa32de25640 1 --2- 172.21.15.43:0/2501842126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa328107260 0x7fa32810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.187+0000 7fa32de25640 1 --2- 172.21.15.43:0/2501842126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa328102b40 0x7fa328106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.187+0000 7fa32de25640 1 --2- 172.21.15.43:0/2501842126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa328102190 0x7fa328102570 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.187+0000 7fa32de25640 1 -- 172.21.15.43:0/2501842126 >> 172.21.15.43:0/2501842126 conn(0x7fa3280793e0 msgr2=0x7fa3280797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:31.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.187+0000 7fa32de25640 1 -- 172.21.15.43:0/2501842126 shutdown_connections 2024-04-21T08:39:31.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.187+0000 7fa32de25640 1 -- 172.21.15.43:0/2501842126 wait complete. 2024-04-21T08:39:31.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.187+0000 7fa32de25640 1 Processor -- start 2024-04-21T08:39:31.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.188+0000 7fa32de25640 1 -- start start 2024-04-21T08:39:31.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.188+0000 7fa32de25640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa328102190 0x7fa328198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.188+0000 7fa32de25640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa328107260 0x7fa328198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.188+0000 7fa32de25640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa32819cfd0 0x7fa32819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.188+0000 7fa32de25640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281121f0 con 0x7fa328102190 2024-04-21T08:39:31.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.188+0000 7fa32de25640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa328112070 con 0x7fa328107260 2024-04-21T08:39:31.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.188+0000 7fa32de25640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa328112370 con 0x7fa32819cfd0 2024-04-21T08:39:31.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.188+0000 7fa3277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa328102190 0x7fa328198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.188+0000 7fa3277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa328102190 0x7fa328198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48892/0 (socket says 172.21.15.43:48892) 2024-04-21T08:39:31.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.188+0000 7fa3277fe640 1 -- 172.21.15.43:0/876387357 learned_addr learned my addr 172.21.15.43:0/876387357 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:31.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.188+0000 7fa326ffd640 1 --2- 172.21.15.43:0/876387357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa328107260 0x7fa328198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.188+0000 7fa327fff640 1 --2- 172.21.15.43:0/876387357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa32819cfd0 0x7fa32819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.188+0000 7fa3277fe640 1 -- 172.21.15.43:0/876387357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa32819cfd0 msgr2=0x7fa32819d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.188+0000 7fa3277fe640 1 --2- 172.21.15.43:0/876387357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa32819cfd0 0x7fa32819d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.188+0000 7fa3277fe640 1 -- 172.21.15.43:0/876387357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa328107260 msgr2=0x7fa328198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.188+0000 7fa3277fe640 1 --2- 172.21.15.43:0/876387357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa328107260 0x7fa328198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.188+0000 7fa3277fe640 1 -- 172.21.15.43:0/876387357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32819dac0 con 0x7fa328102190 2024-04-21T08:39:31.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.188+0000 7fa326ffd640 1 --2- 172.21.15.43:0/876387357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa328107260 0x7fa328198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:31.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.189+0000 7fa3277fe640 1 --2- 172.21.15.43:0/876387357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa328102190 0x7fa328198710 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa318000c00 tx=0x7fa31802c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:31.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.189+0000 7fa324ff9640 1 -- 172.21.15.43:0/876387357 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318043020 con 0x7fa328102190 2024-04-21T08:39:31.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.189+0000 7fa324ff9640 1 -- 172.21.15.43:0/876387357 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa31802cb30 con 0x7fa328102190 2024-04-21T08:39:31.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.189+0000 7fa32de25640 1 -- 172.21.15.43:0/876387357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa32819dd50 con 0x7fa328102190 2024-04-21T08:39:31.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.189+0000 7fa32de25640 1 -- 172.21.15.43:0/876387357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3281a55a0 con 0x7fa328102190 2024-04-21T08:39:31.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.189+0000 7fa324ff9640 1 -- 172.21.15.43:0/876387357 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318003a50 con 0x7fa328102190 2024-04-21T08:39:31.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.191+0000 7fa32de25640 1 -- 172.21.15.43:0/876387357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa328068de0 con 0x7fa328102190 2024-04-21T08:39:31.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.192+0000 7fa324ff9640 1 -- 172.21.15.43:0/876387357 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fa318046020 con 0x7fa328102190 2024-04-21T08:39:31.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.192+0000 7fa324ff9640 1 --2- 172.21.15.43:0/876387357 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa3100a3e30 0x7fa3100a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.192+0000 7fa324ff9640 1 -- 172.21.15.43:0/876387357 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fa3180f2c70 con 0x7fa328102190 2024-04-21T08:39:31.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.192+0000 7fa326ffd640 1 --2- 172.21.15.43:0/876387357 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa3100a3e30 0x7fa3100a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.193+0000 7fa326ffd640 1 --2- 172.21.15.43:0/876387357 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa3100a3e30 0x7fa3100a62f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa328101820 tx=0x7fa31c005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:31.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.197+0000 7fa324ff9640 1 -- 172.21.15.43:0/876387357 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa31804d020 con 0x7fa328102190 2024-04-21T08:39:31.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.431+0000 7fa32de25640 1 -- 172.21.15.43:0/876387357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa32810a490 con 0x7fa328102190 2024-04-21T08:39:31.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.431+0000 7fa324ff9640 1 -- 172.21.15.43:0/876387357 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7fa3180bef70 con 0x7fa328102190 2024-04-21T08:39:31.432 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:39:31.432 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:39:31.432 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:39:31.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.434+0000 7fa32de25640 1 -- 172.21.15.43:0/876387357 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa3100a3e30 msgr2=0x7fa3100a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.434+0000 7fa32de25640 1 --2- 172.21.15.43:0/876387357 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa3100a3e30 0x7fa3100a62f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa328101820 tx=0x7fa31c005f90 comp rx=0 tx=0).stop 2024-04-21T08:39:31.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.434+0000 7fa32de25640 1 -- 172.21.15.43:0/876387357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa328102190 msgr2=0x7fa328198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.434+0000 7fa32de25640 1 --2- 172.21.15.43:0/876387357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa328102190 0x7fa328198710 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa318000c00 tx=0x7fa31802c630 comp rx=0 tx=0).stop 2024-04-21T08:39:31.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.434+0000 7fa32de25640 1 -- 172.21.15.43:0/876387357 shutdown_connections 2024-04-21T08:39:31.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.434+0000 7fa32de25640 1 --2- 172.21.15.43:0/876387357 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa3100a3e30 0x7fa3100a62f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.434+0000 7fa32de25640 1 --2- 172.21.15.43:0/876387357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa32819cfd0 0x7fa32819d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.434+0000 7fa32de25640 1 --2- 172.21.15.43:0/876387357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa328107260 0x7fa328198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.434+0000 7fa32de25640 1 --2- 172.21.15.43:0/876387357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa328102190 0x7fa328198710 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.434+0000 7fa32de25640 1 -- 172.21.15.43:0/876387357 >> 172.21.15.43:0/876387357 conn(0x7fa3280793e0 msgr2=0x7fa3280fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:31.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.434+0000 7fa32de25640 1 -- 172.21.15.43:0/876387357 shutdown_connections 2024-04-21T08:39:31.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.434+0000 7fa32de25640 1 -- 172.21.15.43:0/876387357 wait complete. 2024-04-21T08:39:31.446 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:39:31.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.551+0000 7f75aa367640 1 -- 172.21.15.43:0/437662264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75a410c510 msgr2=0x7f75a410e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.551+0000 7f75aa367640 1 --2- 172.21.15.43:0/437662264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75a410c510 0x7f75a410e9b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f759c009ca0 tx=0x7f759c02c230 comp rx=0 tx=0).stop 2024-04-21T08:39:31.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.551+0000 7f75aa367640 1 -- 172.21.15.43:0/437662264 shutdown_connections 2024-04-21T08:39:31.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.551+0000 7f75aa367640 1 --2- 172.21.15.43:0/437662264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75a410c510 0x7f75a410e9b0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.551+0000 7f75aa367640 1 --2- 172.21.15.43:0/437662264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a4103af0 0x7f75a410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.551+0000 7f75aa367640 1 --2- 172.21.15.43:0/437662264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75a41031d0 0x7f75a41035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.551+0000 7f75aa367640 1 -- 172.21.15.43:0/437662264 >> 172.21.15.43:0/437662264 conn(0x7f75a4077f50 msgr2=0x7f75a4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:31.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.551+0000 7f75aa367640 1 -- 172.21.15.43:0/437662264 shutdown_connections 2024-04-21T08:39:31.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.551+0000 7f75aa367640 1 -- 172.21.15.43:0/437662264 wait complete. 2024-04-21T08:39:31.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.552+0000 7f75aa367640 1 Processor -- start 2024-04-21T08:39:31.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.552+0000 7f75aa367640 1 -- start start 2024-04-21T08:39:31.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.552+0000 7f75aa367640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a41031d0 0x7f75a41a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.552+0000 7f75aa367640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75a4103af0 0x7f75a41a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.552+0000 7f75aa367640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75a410c510 0x7f75a419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.552+0000 7f75aa367640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75a41121a0 con 0x7f75a410c510 2024-04-21T08:39:31.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.552+0000 7f75aa367640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75a4112020 con 0x7f75a4103af0 2024-04-21T08:39:31.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.552+0000 7f75aa367640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75a4112320 con 0x7f75a41031d0 2024-04-21T08:39:31.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.553+0000 7f75a8904640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75a410c510 0x7f75a419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.553+0000 7f75a8904640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75a410c510 0x7f75a419a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48912/0 (socket says 172.21.15.43:48912) 2024-04-21T08:39:31.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.553+0000 7f75a8904640 1 -- 172.21.15.43:0/529832265 learned_addr learned my addr 172.21.15.43:0/529832265 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:31.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.553+0000 7f75a3fff640 1 --2- 172.21.15.43:0/529832265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a41031d0 0x7f75a41a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.553+0000 7f75a37fe640 1 --2- 172.21.15.43:0/529832265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75a4103af0 0x7f75a41a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.553+0000 7f75a8904640 1 -- 172.21.15.43:0/529832265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a41031d0 msgr2=0x7f75a41a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.553+0000 7f75a8904640 1 --2- 172.21.15.43:0/529832265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a41031d0 0x7f75a41a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.553+0000 7f75a8904640 1 -- 172.21.15.43:0/529832265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75a4103af0 msgr2=0x7f75a41a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.553+0000 7f75a8904640 1 --2- 172.21.15.43:0/529832265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75a4103af0 0x7f75a41a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.553+0000 7f75a8904640 1 -- 172.21.15.43:0/529832265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a419b000 con 0x7f75a410c510 2024-04-21T08:39:31.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.553+0000 7f75a37fe640 1 --2- 172.21.15.43:0/529832265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75a4103af0 0x7f75a41a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:31.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.553+0000 7f75a8904640 1 --2- 172.21.15.43:0/529832265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75a410c510 0x7f75a419a740 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f759c02c840 tx=0x7f759c035ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:31.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.554+0000 7f75a17fa640 1 -- 172.21.15.43:0/529832265 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c039040 con 0x7f75a410c510 2024-04-21T08:39:31.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.554+0000 7f75a17fa640 1 -- 172.21.15.43:0/529832265 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f759c044070 con 0x7f75a410c510 2024-04-21T08:39:31.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.554+0000 7f75a17fa640 1 -- 172.21.15.43:0/529832265 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c032880 con 0x7f75a410c510 2024-04-21T08:39:31.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.554+0000 7f75aa367640 1 -- 172.21.15.43:0/529832265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75a419b290 con 0x7f75a410c510 2024-04-21T08:39:31.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.554+0000 7f75aa367640 1 -- 172.21.15.43:0/529832265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75a41a7420 con 0x7f75a410c510 2024-04-21T08:39:31.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.556+0000 7f75a17fa640 1 -- 172.21.15.43:0/529832265 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f759c032a20 con 0x7f75a410c510 2024-04-21T08:39:31.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.556+0000 7f75ad03a740 1 Processor -- start 2024-04-21T08:39:31.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.557+0000 7f75ad03a740 1 -- start start 2024-04-21T08:39:31.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.557+0000 7f75ad03a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557504ec54d0 0x557504ec58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.557+0000 7f75ad03a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec5e80 0x557504ece430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.557+0000 7f75ad03a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557504ecea00 0x557504ed0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.557+0000 7f75ad03a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557504e2a1f0 con 0x557504ec5e80 2024-04-21T08:39:31.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.557+0000 7f75ad03a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557504dc74b0 con 0x557504ecea00 2024-04-21T08:39:31.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.557+0000 7f75ad03a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557504e261b0 con 0x557504ec54d0 2024-04-21T08:39:31.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.557+0000 7f75a3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec5e80 0x557504ece430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.557+0000 7f75a3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec5e80 0x557504ece430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48926/0 (socket says 172.21.15.43:48926) 2024-04-21T08:39:31.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.557+0000 7f75a17fa640 1 --2- 172.21.15.43:0/529832265 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f758c0a3e30 0x7f758c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.557+0000 7f75a3fff640 1 -- 172.21.15.43:0/2376293552 learned_addr learned my addr 172.21.15.43:0/2376293552 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:31.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.557+0000 7f75a37fe640 1 --2- 172.21.15.43:0/2376293552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557504ecea00 0x557504ed0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.557+0000 7f75a37fe640 1 --2- 172.21.15.43:0/2376293552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557504ec54d0 0x557504ec58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.557+0000 7f75a8904640 1 --2- 172.21.15.43:0/529832265 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f758c0a3e30 0x7f758c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.557+0000 7f75a17fa640 1 -- 172.21.15.43:0/529832265 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f759c0f30a0 con 0x7f75a410c510 2024-04-21T08:39:31.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.557+0000 7f75a37fe640 1 -- 172.21.15.43:0/2376293552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557504ecea00 msgr2=0x557504ed0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.557+0000 7f75a37fe640 1 --2- 172.21.15.43:0/2376293552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557504ecea00 0x557504ed0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.558+0000 7f75a37fe640 1 -- 172.21.15.43:0/2376293552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec5e80 msgr2=0x557504ece430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.558+0000 7f75a37fe640 1 --2- 172.21.15.43:0/2376293552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec5e80 0x557504ece430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.558+0000 7f75a37fe640 1 -- 172.21.15.43:0/2376293552 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557504db91b0 con 0x557504ec54d0 2024-04-21T08:39:31.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.558+0000 7f75a37fe640 1 --2- 172.21.15.43:0/2376293552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557504ec54d0 0x557504ec58b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f75a419bf90 tx=0x7f7598005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d922d84598d5c61 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:31.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.558+0000 7f75a8904640 1 --2- 172.21.15.43:0/529832265 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f758c0a3e30 0x7f758c0a62f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f759c0f04d0 tx=0x7f759c091700 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:31.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.558+0000 7f758a7fc640 1 -- 172.21.15.43:0/2376293552 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75980142d0 con 0x557504ec54d0 2024-04-21T08:39:31.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.558+0000 7f758a7fc640 1 -- 172.21.15.43:0/2376293552 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75980054d0 con 0x557504ec54d0 2024-04-21T08:39:31.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.558+0000 7f758a7fc640 1 -- 172.21.15.43:0/2376293552 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75980117e0 con 0x557504ec54d0 2024-04-21T08:39:31.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.559+0000 7f75ad03a740 1 -- 172.21.15.43:0/2376293552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557504ec54d0 msgr2=0x557504ec58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.559+0000 7f75ad03a740 1 --2- 172.21.15.43:0/2376293552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557504ec54d0 0x557504ec58b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f75a419bf90 tx=0x7f7598005850 comp rx=0 tx=0).stop 2024-04-21T08:39:31.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.559+0000 7f75ad03a740 1 -- 172.21.15.43:0/2376293552 shutdown_connections 2024-04-21T08:39:31.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.559+0000 7f75ad03a740 1 --2- 172.21.15.43:0/2376293552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557504ecea00 0x557504ed0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.559+0000 7f75ad03a740 1 --2- 172.21.15.43:0/2376293552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec5e80 0x557504ece430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.559+0000 7f75ad03a740 1 --2- 172.21.15.43:0/2376293552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557504ec54d0 0x557504ec58b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.559+0000 7f75ad03a740 1 -- 172.21.15.43:0/2376293552 >> 172.21.15.43:0/2376293552 conn(0x557504ebfba0 msgr2=0x557504ebff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:31.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.559+0000 7f75ad03a740 1 -- 172.21.15.43:0/2376293552 shutdown_connections 2024-04-21T08:39:31.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.559+0000 7f75ad03a740 1 -- 172.21.15.43:0/2376293552 wait complete. 2024-04-21T08:39:31.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.559+0000 7f75ad03a740 1 -- start start 2024-04-21T08:39:31.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.560+0000 7f75ad03a740 1 Processor -- start 2024-04-21T08:39:31.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.560+0000 7f75ad03a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec54d0 0x557504ece2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.560+0000 7f75ad03a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557504ec5e80 0x557504ecb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.560+0000 7f75ad03a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557504ecea00 0x557504ecbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.560+0000 7f75ad03a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557504ed1970 con 0x557504ec54d0 2024-04-21T08:39:31.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.560+0000 7f75a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec54d0 0x557504ece2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.560+0000 7f75ad03a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557504de3500 con 0x557504ec5e80 2024-04-21T08:39:31.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.560+0000 7f75ad03a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557504ed1af0 con 0x557504ecea00 2024-04-21T08:39:31.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.560+0000 7f75a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec54d0 0x557504ece2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48934/0 (socket says 172.21.15.43:48934) 2024-04-21T08:39:31.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.560+0000 7f75a37fe640 1 -- 172.21.15.43:0/1025915869 learned_addr learned my addr 172.21.15.43:0/1025915869 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:31.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.560+0000 7f75a37fe640 1 --2- 172.21.15.43:0/1025915869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557504ecea00 0x557504ecbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.560+0000 7f75ad03a740 10 client.0 _ll_register_callbacks cb 0x557504db1b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:31.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.561+0000 7f75a37fe640 1 -- 172.21.15.43:0/1025915869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557504ecea00 msgr2=0x557504ecbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.561+0000 7f75a37fe640 1 --2- 172.21.15.43:0/1025915869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557504ecea00 0x557504ecbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.561+0000 7f75a37fe640 1 -- 172.21.15.43:0/1025915869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557504ec5e80 msgr2=0x557504ecb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.561+0000 7f75a37fe640 1 --2- 172.21.15.43:0/1025915869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557504ec5e80 0x557504ecb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.561+0000 7f75a37fe640 1 -- 172.21.15.43:0/1025915869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557504b3c3f0 con 0x557504ec54d0 2024-04-21T08:39:31.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.561+0000 7f75a37fe640 1 --2- 172.21.15.43:0/1025915869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec54d0 0x557504ece2e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f759800c4d0 tx=0x7f7598005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad46373104b54a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:31.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.561+0000 7f75837fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:31.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.561+0000 7f75837fe640 1 -- 172.21.15.43:0/1025915869 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7598005230 con 0x557504ec54d0 2024-04-21T08:39:31.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.561+0000 7f75837fe640 1 -- 172.21.15.43:0/1025915869 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f75980117e0 con 0x557504ec54d0 2024-04-21T08:39:31.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.561+0000 7f75ad03a740 1 -- 172.21.15.43:0/1025915869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75980054d0 con 0x557504ec54d0 2024-04-21T08:39:31.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.561+0000 7f75837fe640 1 -- 172.21.15.43:0/1025915869 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7598036050 con 0x557504ec54d0 2024-04-21T08:39:31.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.561+0000 7f75ad03a740 10 client.16092 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:31.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.562+0000 7f75ad03a740 1 -- 172.21.15.43:0/1025915869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557504c067d0 con 0x557504ec54d0 2024-04-21T08:39:31.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.562+0000 7f75837fe640 1 -- 172.21.15.43:0/1025915869 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75980054d0 con 0x557504ec54d0 2024-04-21T08:39:31.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.562+0000 7f75837fe640 1 -- 172.21.15.43:0/1025915869 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f759801f600 con 0x557504ec54d0 2024-04-21T08:39:31.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.562+0000 7f75ad03a740 10 client.16092 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:31.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.562+0000 7f75ad03a740 10 client.16092 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:39:31.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.562+0000 7f75ad03a740 20 client.16092 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:31.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.562+0000 7f75ad03a740 1 --2- 172.21.15.43:0/1025915869 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x557504edeaa0 0x557504ee0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.562+0000 7f75ad03a740 4 client.16092 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:31.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.562+0000 7f75ad03a740 1 -- 172.21.15.43:0/1025915869 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557504dc9930 con 0x557504edeaa0 2024-04-21T08:39:31.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.562+0000 7f75a3fff640 1 --2- 172.21.15.43:0/1025915869 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x557504edeaa0 0x557504ee0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.563+0000 7f75a3fff640 1 --2- 172.21.15.43:0/1025915869 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x557504edeaa0 0x557504ee0e90 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=536fc4f5d01a2d4b server_cookie=268c6824fde2bb3 in_seq=0 out_seq=0 2024-04-21T08:39:31.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.563+0000 7f75837fe640 10 client.16092 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:39:31.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.564+0000 7f75837fe640 1 -- 172.21.15.43:0/1025915869 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557504dc9930 con 0x557504edeaa0 2024-04-21T08:39:31.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.564+0000 7f75837fe640 10 client.16092 handle_command_reply: tid=0 2024-04-21T08:39:31.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.564+0000 7f75ad03a740 1 client.16092 shutdown 2024-04-21T08:39:31.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.564+0000 7f75ad03a740 1 -- 172.21.15.43:0/1025915869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec54d0 msgr2=0x557504ece2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.564+0000 7f75ad03a740 1 --2- 172.21.15.43:0/1025915869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec54d0 0x557504ece2e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f759800c4d0 tx=0x7f7598005fb0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.564+0000 7f75ad03a740 1 -- 172.21.15.43:0/1025915869 shutdown_connections 2024-04-21T08:39:31.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.564+0000 7f75ad03a740 1 --2- 172.21.15.43:0/1025915869 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x557504edeaa0 0x557504ee0e90 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.564+0000 7f75ad03a740 1 --2- 172.21.15.43:0/1025915869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557504ecea00 0x557504ecbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.564+0000 7f75ad03a740 1 --2- 172.21.15.43:0/1025915869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557504ec5e80 0x557504ecb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.564+0000 7f75ad03a740 1 --2- 172.21.15.43:0/1025915869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec54d0 0x557504ece2e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.564+0000 7f75ad03a740 1 -- 172.21.15.43:0/1025915869 >> 172.21.15.43:0/1025915869 conn(0x557504ebfba0 msgr2=0x557504ec18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:31.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.564+0000 7f75837fe640 0 client.16092 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:39:31.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.564+0000 7f75ad03a740 1 -- 172.21.15.43:0/1025915869 shutdown_connections 2024-04-21T08:39:31.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.564+0000 7f75ad03a740 1 -- 172.21.15.43:0/1025915869 wait complete. 2024-04-21T08:39:31.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.564+0000 7f75ad03a740 20 client.16092 trim_cache size 0 max 16384 2024-04-21T08:39:31.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.590+0000 7f75ad03a740 1 Processor -- start 2024-04-21T08:39:31.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.590+0000 7f75ad03a740 1 -- start start 2024-04-21T08:39:31.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.590+0000 7f75ad03a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec54d0 0x557504eda890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.590+0000 7f75ad03a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557504ec5e80 0x557504eca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.590+0000 7f75ad03a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557504ecea00 0x557504ebea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.590+0000 7f75ad03a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75a41b16b0 con 0x557504ec54d0 2024-04-21T08:39:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.590+0000 7f75ad03a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75a41b31f0 con 0x557504ec5e80 2024-04-21T08:39:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.590+0000 7f75ad03a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75a41b1d80 con 0x557504ecea00 2024-04-21T08:39:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.590+0000 7f75a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec54d0 0x557504eda890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.590+0000 7f75a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec54d0 0x557504eda890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48940/0 (socket says 172.21.15.43:48940) 2024-04-21T08:39:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.590+0000 7f75a37fe640 1 -- 172.21.15.43:0/2697870261 learned_addr learned my addr 172.21.15.43:0/2697870261 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.591+0000 7f75a3fff640 1 --2- 172.21.15.43:0/2697870261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557504ec5e80 0x557504eca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.591+0000 7f75a37fe640 1 -- 172.21.15.43:0/2697870261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557504ecea00 msgr2=0x557504ebea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.591+0000 7f75a37fe640 1 --2- 172.21.15.43:0/2697870261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557504ecea00 0x557504ebea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.591+0000 7f75a37fe640 1 -- 172.21.15.43:0/2697870261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557504ec5e80 msgr2=0x557504eca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.591+0000 7f75a37fe640 1 --2- 172.21.15.43:0/2697870261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557504ec5e80 0x557504eca7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.591+0000 7f75a37fe640 1 -- 172.21.15.43:0/2697870261 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557504ec9bc0 con 0x557504ec54d0 2024-04-21T08:39:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.591+0000 7f75a37fe640 1 --2- 172.21.15.43:0/2697870261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec54d0 0x557504eda890 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7598005c70 tx=0x7f759800c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d2a6a9282e015a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.591+0000 7f7589ffb640 1 -- 172.21.15.43:0/2697870261 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759800c790 con 0x557504ec54d0 2024-04-21T08:39:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.591+0000 7f7589ffb640 1 -- 172.21.15.43:0/2697870261 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f759800ce20 con 0x557504ec54d0 2024-04-21T08:39:31.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.591+0000 7f7589ffb640 1 -- 172.21.15.43:0/2697870261 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75980048e0 con 0x557504ec54d0 2024-04-21T08:39:31.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.592+0000 7f75ad03a740 1 -- 172.21.15.43:0/2697870261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec54d0 msgr2=0x557504eda890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.592+0000 7f75ad03a740 1 --2- 172.21.15.43:0/2697870261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec54d0 0x557504eda890 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7598005c70 tx=0x7f759800c560 comp rx=0 tx=0).stop 2024-04-21T08:39:31.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.592+0000 7f75ad03a740 1 -- 172.21.15.43:0/2697870261 shutdown_connections 2024-04-21T08:39:31.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.592+0000 7f75ad03a740 1 --2- 172.21.15.43:0/2697870261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557504ecea00 0x557504ebea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.592+0000 7f75ad03a740 1 --2- 172.21.15.43:0/2697870261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557504ec5e80 0x557504eca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.592+0000 7f75ad03a740 1 --2- 172.21.15.43:0/2697870261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec54d0 0x557504eda890 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.592+0000 7f75ad03a740 1 -- 172.21.15.43:0/2697870261 >> 172.21.15.43:0/2697870261 conn(0x557504ebfba0 msgr2=0x557504ebe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:31.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.592+0000 7f75ad03a740 1 -- 172.21.15.43:0/2697870261 shutdown_connections 2024-04-21T08:39:31.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.592+0000 7f75ad03a740 1 -- 172.21.15.43:0/2697870261 wait complete. 2024-04-21T08:39:31.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.593+0000 7f75ad03a740 1 -- start start 2024-04-21T08:39:31.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.593+0000 7f75ad03a740 1 Processor -- start 2024-04-21T08:39:31.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.594+0000 7f75ad03a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec54d0 0x557504ed2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.594+0000 7f75ad03a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557504ec5e80 0x557504ed25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.594+0000 7f75a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec54d0 0x557504ed2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.594+0000 7f75a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec54d0 0x557504ed2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48954/0 (socket says 172.21.15.43:48954) 2024-04-21T08:39:31.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.594+0000 7f75ad03a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557504ecea00 0x557504f3cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.594+0000 7f75ad03a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75a41b2b20 con 0x557504ec54d0 2024-04-21T08:39:31.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.594+0000 7f75a37fe640 1 -- 172.21.15.43:0/686231143 learned_addr learned my addr 172.21.15.43:0/686231143 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:31.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.594+0000 7f75ad03a740 1 -- 172.21.15.43:0/686231143 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75a41b2450 con 0x557504ec5e80 2024-04-21T08:39:31.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.594+0000 7f75ad03a740 1 -- 172.21.15.43:0/686231143 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75a41af690 con 0x557504ecea00 2024-04-21T08:39:31.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.594+0000 7f75ad03a740 10 client.0 _ll_register_callbacks cb 0x7f7598014840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:31.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.594+0000 7f75a3fff640 1 --2- 172.21.15.43:0/686231143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557504ec5e80 0x557504ed25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.594+0000 7f75a37fe640 1 -- 172.21.15.43:0/686231143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557504ecea00 msgr2=0x557504f3cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.594+0000 7f75a37fe640 1 --2- 172.21.15.43:0/686231143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557504ecea00 0x557504f3cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.594+0000 7f75a37fe640 1 -- 172.21.15.43:0/686231143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557504ec5e80 msgr2=0x557504ed25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.594+0000 7f75a37fe640 1 --2- 172.21.15.43:0/686231143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557504ec5e80 0x557504ed25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.594+0000 7f75a37fe640 1 -- 172.21.15.43:0/686231143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557504f3d3d0 con 0x557504ec54d0 2024-04-21T08:39:31.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.594+0000 7f75a37fe640 1 --2- 172.21.15.43:0/686231143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec54d0 0x557504ed2070 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7598006010 tx=0x7f759800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4343c0cd1d6c8cb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:31.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.594+0000 7f75837fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:31.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.594+0000 7f75837fe640 1 -- 172.21.15.43:0/686231143 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7598048060 con 0x557504ec54d0 2024-04-21T08:39:31.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.595+0000 7f75837fe640 1 -- 172.21.15.43:0/686231143 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f759801f470 con 0x557504ec54d0 2024-04-21T08:39:31.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.595+0000 7f75ad03a740 1 -- 172.21.15.43:0/686231143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f759800ce20 con 0x557504ec54d0 2024-04-21T08:39:31.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.595+0000 7f75837fe640 1 -- 172.21.15.43:0/686231143 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7598005230 con 0x557504ec54d0 2024-04-21T08:39:31.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.595+0000 7f75ad03a740 10 client.16098 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:31.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.595+0000 7f75ad03a740 1 -- 172.21.15.43:0/686231143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557504f44760 con 0x557504ec54d0 2024-04-21T08:39:31.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.595+0000 7f75837fe640 1 -- 172.21.15.43:0/686231143 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f759800ce20 con 0x557504ec54d0 2024-04-21T08:39:31.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.595+0000 7f75837fe640 1 -- 172.21.15.43:0/686231143 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7598011b20 con 0x557504ec54d0 2024-04-21T08:39:31.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.595+0000 7f75ad03a740 10 client.16098 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:31.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.595+0000 7f75ad03a740 10 client.16098 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:39:31.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.595+0000 7f75ad03a740 20 client.16098 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:31.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.595+0000 7f75ad03a740 1 --2- 172.21.15.43:0/686231143 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x557504f44bd0 0x557504f46fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.595+0000 7f75ad03a740 4 client.16098 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:39:31.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.595+0000 7f75ad03a740 1 -- 172.21.15.43:0/686231143 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557504e37c10 con 0x557504f44bd0 2024-04-21T08:39:31.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.596+0000 7f75a3fff640 1 --2- 172.21.15.43:0/686231143 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x557504f44bd0 0x557504f46fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.596+0000 7f75837fe640 10 client.16098 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:39:31.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.596+0000 7f75a3fff640 1 --2- 172.21.15.43:0/686231143 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x557504f44bd0 0x557504f46fc0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2d21fcaef021492e server_cookie=894b7c88d08e3154 in_seq=0 out_seq=0 2024-04-21T08:39:31.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.597+0000 7f75837fe640 1 -- 172.21.15.43:0/686231143 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557504e37c10 con 0x557504f44bd0 2024-04-21T08:39:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.597+0000 7f75837fe640 10 client.16098 handle_command_reply: tid=0 2024-04-21T08:39:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.597+0000 7f75ad03a740 1 client.16098 shutdown 2024-04-21T08:39:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.597+0000 7f75ad03a740 1 -- 172.21.15.43:0/686231143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec54d0 msgr2=0x557504ed2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.597+0000 7f75ad03a740 1 --2- 172.21.15.43:0/686231143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec54d0 0x557504ed2070 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7598006010 tx=0x7f759800cbc0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.597+0000 7f75ad03a740 1 -- 172.21.15.43:0/686231143 shutdown_connections 2024-04-21T08:39:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.597+0000 7f75ad03a740 1 --2- 172.21.15.43:0/686231143 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x557504f44bd0 0x557504f46fc0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.597+0000 7f75ad03a740 1 --2- 172.21.15.43:0/686231143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557504ecea00 0x557504f3cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.597+0000 7f75ad03a740 1 --2- 172.21.15.43:0/686231143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557504ec5e80 0x557504ed25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.597+0000 7f75ad03a740 1 --2- 172.21.15.43:0/686231143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557504ec54d0 0x557504ed2070 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.597+0000 7f75ad03a740 1 -- 172.21.15.43:0/686231143 >> 172.21.15.43:0/686231143 conn(0x557504ebfba0 msgr2=0x557504ebec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.597+0000 7f75837fe640 0 client.16098 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:39:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.597+0000 7f75ad03a740 1 -- 172.21.15.43:0/686231143 shutdown_connections 2024-04-21T08:39:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.597+0000 7f75ad03a740 1 -- 172.21.15.43:0/686231143 wait complete. 2024-04-21T08:39:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.597+0000 7f75ad03a740 20 client.16098 trim_cache size 0 max 16384 2024-04-21T08:39:31.602 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:39:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.598+0000 7f75aa367640 1 -- 172.21.15.43:0/529832265 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f758c0a3e30 msgr2=0x7f758c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.598+0000 7f75aa367640 1 --2- 172.21.15.43:0/529832265 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f758c0a3e30 0x7f758c0a62f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f759c0f04d0 tx=0x7f759c091700 comp rx=0 tx=0).stop 2024-04-21T08:39:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.598+0000 7f75aa367640 1 -- 172.21.15.43:0/529832265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75a410c510 msgr2=0x7f75a419a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.598+0000 7f75aa367640 1 --2- 172.21.15.43:0/529832265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75a410c510 0x7f75a419a740 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f759c02c840 tx=0x7f759c035ea0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.598+0000 7f75aa367640 1 -- 172.21.15.43:0/529832265 shutdown_connections 2024-04-21T08:39:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.598+0000 7f75aa367640 1 --2- 172.21.15.43:0/529832265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75a410c510 0x7f75a419a740 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.598+0000 7f75aa367640 1 --2- 172.21.15.43:0/529832265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75a4103af0 0x7f75a41a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.598+0000 7f75aa367640 1 --2- 172.21.15.43:0/529832265 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f758c0a3e30 0x7f758c0a62f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.598+0000 7f75aa367640 1 --2- 172.21.15.43:0/529832265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75a41031d0 0x7f75a41a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.598+0000 7f75aa367640 1 -- 172.21.15.43:0/529832265 >> 172.21.15.43:0/529832265 conn(0x7f75a4077f50 msgr2=0x7f75a4108480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.598+0000 7f75aa367640 1 -- 172.21.15.43:0/529832265 shutdown_connections 2024-04-21T08:39:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.598+0000 7f75aa367640 1 -- 172.21.15.43:0/529832265 wait complete. 2024-04-21T08:39:31.612 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:39:31.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.719+0000 7f9ea86a5640 1 -- 172.21.15.43:0/3315736874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ea0106160 msgr2=0x7f9ea010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.719+0000 7f9ea86a5640 1 --2- 172.21.15.43:0/3315736874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ea0106160 0x7f9ea010dd30 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009ca0 tx=0x7f9e9802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.719+0000 7f9ea86a5640 1 -- 172.21.15.43:0/3315736874 shutdown_connections 2024-04-21T08:39:31.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.719+0000 7f9ea86a5640 1 --2- 172.21.15.43:0/3315736874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ea0106160 0x7f9ea010dd30 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.719+0000 7f9ea86a5640 1 --2- 172.21.15.43:0/3315736874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ea0101a40 0x7f9ea0105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.719+0000 7f9ea86a5640 1 --2- 172.21.15.43:0/3315736874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ea0101090 0x7f9ea0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.719+0000 7f9ea86a5640 1 -- 172.21.15.43:0/3315736874 >> 172.21.15.43:0/3315736874 conn(0x7f9ea0077f50 msgr2=0x7f9ea0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:31.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.719+0000 7f9ea86a5640 1 -- 172.21.15.43:0/3315736874 shutdown_connections 2024-04-21T08:39:31.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.719+0000 7f9ea86a5640 1 -- 172.21.15.43:0/3315736874 wait complete. 2024-04-21T08:39:31.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.720+0000 7f9ea86a5640 1 Processor -- start 2024-04-21T08:39:31.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.720+0000 7f9ea86a5640 1 -- start start 2024-04-21T08:39:31.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.720+0000 7f9ea86a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ea0101090 0x7f9ea01986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.720+0000 7f9ea86a5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ea0106160 0x7f9ea0198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.720+0000 7f9ea86a5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ea019cf80 0x7f9ea019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.720+0000 7f9ea86a5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea01121e0 con 0x7f9ea0106160 2024-04-21T08:39:31.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.720+0000 7f9ea86a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea0112060 con 0x7f9ea0101090 2024-04-21T08:39:31.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.720+0000 7f9ea86a5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea0112360 con 0x7f9ea019cf80 2024-04-21T08:39:31.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.720+0000 7f9ea5c40640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ea0106160 0x7f9ea0198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.720+0000 7f9ea5c40640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ea0106160 0x7f9ea0198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48978/0 (socket says 172.21.15.43:48978) 2024-04-21T08:39:31.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.720+0000 7f9ea5c40640 1 -- 172.21.15.43:0/2910126527 learned_addr learned my addr 172.21.15.43:0/2910126527 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.720+0000 7f9ea6c42640 1 --2- 172.21.15.43:0/2910126527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ea019cf80 0x7f9ea019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.720+0000 7f9ea6441640 1 --2- 172.21.15.43:0/2910126527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ea0101090 0x7f9ea01986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.720+0000 7f9ea5c40640 1 -- 172.21.15.43:0/2910126527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ea019cf80 msgr2=0x7f9ea019d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.720+0000 7f9ea5c40640 1 --2- 172.21.15.43:0/2910126527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ea019cf80 0x7f9ea019d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.720+0000 7f9ea5c40640 1 -- 172.21.15.43:0/2910126527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ea0101090 msgr2=0x7f9ea01986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.720+0000 7f9ea5c40640 1 --2- 172.21.15.43:0/2910126527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ea0101090 0x7f9ea01986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.720+0000 7f9ea5c40640 1 -- 172.21.15.43:0/2910126527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea019da70 con 0x7f9ea0106160 2024-04-21T08:39:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.721+0000 7f9ea5c40640 1 --2- 172.21.15.43:0/2910126527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ea0106160 0x7f9ea0198c00 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000e9c0 tx=0x7f9e9000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.721+0000 7f9e977fe640 1 -- 172.21.15.43:0/2910126527 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9000cd80 con 0x7f9ea0106160 2024-04-21T08:39:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.721+0000 7f9ea86a5640 1 -- 172.21.15.43:0/2910126527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea019dd60 con 0x7f9ea0106160 2024-04-21T08:39:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.721+0000 7f9e977fe640 1 -- 172.21.15.43:0/2910126527 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e900054d0 con 0x7f9ea0106160 2024-04-21T08:39:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.721+0000 7f9e977fe640 1 -- 172.21.15.43:0/2910126527 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90010430 con 0x7f9ea0106160 2024-04-21T08:39:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.721+0000 7f9ea86a5640 1 -- 172.21.15.43:0/2910126527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea010e8e0 con 0x7f9ea0106160 2024-04-21T08:39:31.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.724+0000 7f9eab375740 1 Processor -- start 2024-04-21T08:39:31.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.724+0000 7f9e977fe640 1 -- 172.21.15.43:0/2910126527 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f9e90004fb0 con 0x7f9ea0106160 2024-04-21T08:39:31.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.724+0000 7f9eab375740 1 -- start start 2024-04-21T08:39:31.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.725+0000 7f9eab375740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc7a862470 0x55bc7a862850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.725+0000 7f9eab375740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc7a862e20 0x55bc7a873c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.725+0000 7f9eab375740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a874210 0x55bc7a876620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.725+0000 7f9eab375740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bc7a7c71f0 con 0x55bc7a874210 2024-04-21T08:39:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.725+0000 7f9eab375740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bc7a7644b0 con 0x55bc7a862470 2024-04-21T08:39:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.725+0000 7f9eab375740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bc7a7c31b0 con 0x55bc7a862e20 2024-04-21T08:39:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.725+0000 7f9ea5c40640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a874210 0x55bc7a876620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.725+0000 7f9ea5c40640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a874210 0x55bc7a876620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48990/0 (socket says 172.21.15.43:48990) 2024-04-21T08:39:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.725+0000 7f9ea5c40640 1 -- 172.21.15.43:0/1410475577 learned_addr learned my addr 172.21.15.43:0/1410475577 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.725+0000 7f9ea6441640 1 --2- 172.21.15.43:0/1410475577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc7a862e20 0x55bc7a873c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.725+0000 7f9ea6c42640 1 --2- 172.21.15.43:0/1410475577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc7a862470 0x55bc7a862850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.725+0000 7f9ea5c40640 1 -- 172.21.15.43:0/1410475577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc7a862e20 msgr2=0x55bc7a873c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.725+0000 7f9ea5c40640 1 --2- 172.21.15.43:0/1410475577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc7a862e20 0x55bc7a873c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.725+0000 7f9ea5c40640 1 -- 172.21.15.43:0/1410475577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc7a862470 msgr2=0x55bc7a862850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.725+0000 7f9ea5c40640 1 --2- 172.21.15.43:0/1410475577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc7a862470 0x55bc7a862850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.725+0000 7f9ea5c40640 1 -- 172.21.15.43:0/1410475577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc7a7561b0 con 0x55bc7a874210 2024-04-21T08:39:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.725+0000 7f9ea6441640 1 --2- 172.21.15.43:0/1410475577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc7a862e20 0x55bc7a873c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.725+0000 7f9e977fe640 1 --2- 172.21.15.43:0/2910126527 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9e8c0a3db0 0x7f9e8c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.726+0000 7f9ea6c42640 1 --2- 172.21.15.43:0/2910126527 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9e8c0a3db0 0x7f9e8c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.726+0000 7f9ea5c40640 1 --2- 172.21.15.43:0/1410475577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a874210 0x55bc7a876620 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9e900ce330 tx=0x7f9e900d91a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d0270ac028fecd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.726+0000 7f9e977fe640 1 -- 172.21.15.43:0/2910126527 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f9e900d1160 con 0x7f9ea0106160 2024-04-21T08:39:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.726+0000 7f9e94ff9640 1 -- 172.21.15.43:0/1410475577 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9009c930 con 0x55bc7a874210 2024-04-21T08:39:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.726+0000 7f9e94ff9640 1 -- 172.21.15.43:0/1410475577 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e900046c0 con 0x55bc7a874210 2024-04-21T08:39:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.726+0000 7f9e94ff9640 1 -- 172.21.15.43:0/1410475577 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9009c4c0 con 0x55bc7a874210 2024-04-21T08:39:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.727+0000 7f9eab375740 1 -- 172.21.15.43:0/1410475577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a874210 msgr2=0x55bc7a876620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.727+0000 7f9ea6c42640 1 --2- 172.21.15.43:0/2910126527 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9e8c0a3db0 0x7f9e8c0a6270 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9e98004af0 tx=0x7f9e98005c50 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.727+0000 7f9eab375740 1 --2- 172.21.15.43:0/1410475577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a874210 0x55bc7a876620 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9e900ce330 tx=0x7f9e900d91a0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.727+0000 7f9eab375740 1 -- 172.21.15.43:0/1410475577 shutdown_connections 2024-04-21T08:39:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.727+0000 7f9eab375740 1 --2- 172.21.15.43:0/1410475577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a874210 0x55bc7a876620 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.727+0000 7f9eab375740 1 --2- 172.21.15.43:0/1410475577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc7a862e20 0x55bc7a873c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.727+0000 7f9eab375740 1 --2- 172.21.15.43:0/1410475577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc7a862470 0x55bc7a862850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.727+0000 7f9eab375740 1 -- 172.21.15.43:0/1410475577 >> 172.21.15.43:0/1410475577 conn(0x55bc7a85cba0 msgr2=0x55bc7a85cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.727+0000 7f9eab375740 1 -- 172.21.15.43:0/1410475577 shutdown_connections 2024-04-21T08:39:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.727+0000 7f9eab375740 1 -- 172.21.15.43:0/1410475577 wait complete. 2024-04-21T08:39:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.728+0000 7f9eab375740 1 -- start start 2024-04-21T08:39:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.728+0000 7f9eab375740 1 Processor -- start 2024-04-21T08:39:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.728+0000 7f9eab375740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a862470 0x55bc7a86db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.728+0000 7f9eab375740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc7a862e20 0x55bc7a86e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.728+0000 7f9eab375740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc7a874210 0x55bc7a873860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.728+0000 7f9eab375740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bc7a8637d0 con 0x55bc7a862470 2024-04-21T08:39:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.728+0000 7f9eab375740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bc7a780500 con 0x55bc7a862e20 2024-04-21T08:39:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.728+0000 7f9eab375740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bc7a863950 con 0x55bc7a874210 2024-04-21T08:39:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.728+0000 7f9ea6441640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a862470 0x55bc7a86db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.728+0000 7f9eab375740 10 client.0 _ll_register_callbacks cb 0x55bc7a74eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.728+0000 7f9ea6441640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a862470 0x55bc7a86db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48998/0 (socket says 172.21.15.43:48998) 2024-04-21T08:39:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.728+0000 7f9ea6441640 1 -- 172.21.15.43:0/3260037252 learned_addr learned my addr 172.21.15.43:0/3260037252 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.728+0000 7f9ea5c40640 1 --2- 172.21.15.43:0/3260037252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc7a862e20 0x55bc7a86e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.728+0000 7f9ea6c42640 1 --2- 172.21.15.43:0/3260037252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc7a874210 0x55bc7a873860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.729+0000 7f9ea6441640 1 -- 172.21.15.43:0/3260037252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc7a874210 msgr2=0x55bc7a873860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.729+0000 7f9ea6441640 1 --2- 172.21.15.43:0/3260037252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc7a874210 0x55bc7a873860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.729+0000 7f9ea6441640 1 -- 172.21.15.43:0/3260037252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc7a862e20 msgr2=0x55bc7a86e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.729+0000 7f9ea6441640 1 --2- 172.21.15.43:0/3260037252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc7a862e20 0x55bc7a86e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.729+0000 7f9ea6441640 1 -- 172.21.15.43:0/3260037252 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc7a4d93f0 con 0x55bc7a862470 2024-04-21T08:39:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.729+0000 7f9ea5c40640 1 --2- 172.21.15.43:0/3260037252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc7a862e20 0x55bc7a86e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.729+0000 7f9ea6441640 1 --2- 172.21.15.43:0/3260037252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a862470 0x55bc7a86db40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c012700 tx=0x7f9e9c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaccc4f5a7fe5272 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.729+0000 7f9e89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.729+0000 7f9e89ffb640 1 -- 172.21.15.43:0/3260037252 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c004030 con 0x55bc7a862470 2024-04-21T08:39:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.729+0000 7f9e89ffb640 1 -- 172.21.15.43:0/3260037252 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f9e9c0041d0 con 0x55bc7a862470 2024-04-21T08:39:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.729+0000 7f9eab375740 1 -- 172.21.15.43:0/3260037252 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bc7a766930 con 0x55bc7a862470 2024-04-21T08:39:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.729+0000 7f9e89ffb640 1 -- 172.21.15.43:0/3260037252 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9e9c025050 con 0x55bc7a862470 2024-04-21T08:39:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.729+0000 7f9eab375740 10 client.16110 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.729+0000 7f9eab375740 1 -- 172.21.15.43:0/3260037252 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc7a5a37d0 con 0x55bc7a862470 2024-04-21T08:39:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.729+0000 7f9e89ffb640 1 -- 172.21.15.43:0/3260037252 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e9c004410 con 0x55bc7a862470 2024-04-21T08:39:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.730+0000 7f9e89ffb640 1 -- 172.21.15.43:0/3260037252 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9e9c003b90 con 0x55bc7a862470 2024-04-21T08:39:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.730+0000 7f9eab375740 10 client.16110 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.730+0000 7f9eab375740 10 client.16110 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:39:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.730+0000 7f9eab375740 20 client.16110 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.730+0000 7f9eab375740 1 --2- 172.21.15.43:0/3260037252 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55bc7a88ccb0 0x55bc7a88f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.730+0000 7f9eab375740 4 client.16110 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.730+0000 7f9eab375740 1 -- 172.21.15.43:0/3260037252 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc7a766930 con 0x55bc7a88ccb0 2024-04-21T08:39:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.730+0000 7f9ea5c40640 1 --2- 172.21.15.43:0/3260037252 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55bc7a88ccb0 0x55bc7a88f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.731+0000 7f9e89ffb640 10 client.16110 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:39:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.731+0000 7f9ea5c40640 1 --2- 172.21.15.43:0/3260037252 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55bc7a88ccb0 0x55bc7a88f0e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9874d161d8ec1441 server_cookie=635fda9646919f89 in_seq=0 out_seq=0 2024-04-21T08:39:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.731+0000 7f9e89ffb640 1 -- 172.21.15.43:0/3260037252 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bc7a766930 con 0x55bc7a88ccb0 2024-04-21T08:39:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.731+0000 7f9e89ffb640 10 client.16110 handle_command_reply: tid=0 2024-04-21T08:39:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.731+0000 7f9eab375740 1 client.16110 shutdown 2024-04-21T08:39:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.731+0000 7f9eab375740 1 -- 172.21.15.43:0/3260037252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a862470 msgr2=0x55bc7a86db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.731+0000 7f9eab375740 1 --2- 172.21.15.43:0/3260037252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a862470 0x55bc7a86db40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c012700 tx=0x7f9e9c012bc0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.732+0000 7f9eab375740 1 -- 172.21.15.43:0/3260037252 shutdown_connections 2024-04-21T08:39:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.732+0000 7f9eab375740 1 --2- 172.21.15.43:0/3260037252 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55bc7a88ccb0 0x55bc7a88f0e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.732+0000 7f9eab375740 1 --2- 172.21.15.43:0/3260037252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc7a874210 0x55bc7a873860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.732+0000 7f9e89ffb640 0 client.16110 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:39:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.732+0000 7f9eab375740 1 --2- 172.21.15.43:0/3260037252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc7a862e20 0x55bc7a86e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.732+0000 7f9eab375740 1 --2- 172.21.15.43:0/3260037252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a862470 0x55bc7a86db40 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.732+0000 7f9eab375740 1 -- 172.21.15.43:0/3260037252 >> 172.21.15.43:0/3260037252 conn(0x55bc7a85cba0 msgr2=0x55bc7a85e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:31.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.732+0000 7f9eab375740 1 -- 172.21.15.43:0/3260037252 shutdown_connections 2024-04-21T08:39:31.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.732+0000 7f9eab375740 1 -- 172.21.15.43:0/3260037252 wait complete. 2024-04-21T08:39:31.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.732+0000 7f9eab375740 20 client.16110 trim_cache size 0 max 16384 2024-04-21T08:39:31.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.758+0000 7f9eab375740 1 Processor -- start 2024-04-21T08:39:31.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.758+0000 7f9eab375740 1 -- start start 2024-04-21T08:39:31.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.758+0000 7f9eab375740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc7a862470 0x55bc7a8737d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.758+0000 7f9eab375740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a862e20 0x55bc7a8949b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.758+0000 7f9eab375740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc7a874210 0x55bc7a894ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.758+0000 7f9ea5c40640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a862e20 0x55bc7a8949b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.758+0000 7f9eab375740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea01ad2e0 con 0x55bc7a862e20 2024-04-21T08:39:31.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.758+0000 7f9ea5c40640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a862e20 0x55bc7a8949b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49000/0 (socket says 172.21.15.43:49000) 2024-04-21T08:39:31.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.758+0000 7f9eab375740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea01aee20 con 0x55bc7a874210 2024-04-21T08:39:31.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.758+0000 7f9eab375740 1 -- 172.21.15.43:0/3173402713 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea01ad9b0 con 0x55bc7a862470 2024-04-21T08:39:31.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.758+0000 7f9ea5c40640 1 -- 172.21.15.43:0/3173402713 learned_addr learned my addr 172.21.15.43:0/3173402713 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:31.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.758+0000 7f9ea6441640 1 --2- 172.21.15.43:0/3173402713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc7a862470 0x55bc7a8737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.759+0000 7f9ea6c42640 1 --2- 172.21.15.43:0/3173402713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc7a874210 0x55bc7a894ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.759+0000 7f9ea5c40640 1 -- 172.21.15.43:0/3173402713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc7a862470 msgr2=0x55bc7a8737d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.759+0000 7f9ea5c40640 1 --2- 172.21.15.43:0/3173402713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc7a862470 0x55bc7a8737d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.759+0000 7f9ea5c40640 1 -- 172.21.15.43:0/3173402713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc7a874210 msgr2=0x55bc7a894ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.759+0000 7f9ea5c40640 1 --2- 172.21.15.43:0/3173402713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc7a874210 0x55bc7a894ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.759+0000 7f9ea5c40640 1 -- 172.21.15.43:0/3173402713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc7a862bc0 con 0x55bc7a862e20 2024-04-21T08:39:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.759+0000 7f9ea5c40640 1 --2- 172.21.15.43:0/3173402713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a862e20 0x55bc7a8949b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9e900057a0 tx=0x7f9e90005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8f1dc3fb9dae85 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.759+0000 7f9e8bfff640 1 -- 172.21.15.43:0/3173402713 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9009c6a0 con 0x55bc7a862e20 2024-04-21T08:39:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.759+0000 7f9e8bfff640 1 -- 172.21.15.43:0/3173402713 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e9009d260 con 0x55bc7a862e20 2024-04-21T08:39:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.759+0000 7f9e8bfff640 1 -- 172.21.15.43:0/3173402713 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e900a57d0 con 0x55bc7a862e20 2024-04-21T08:39:31.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.760+0000 7f9eab375740 1 -- 172.21.15.43:0/3173402713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a862e20 msgr2=0x55bc7a8949b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.760+0000 7f9eab375740 1 --2- 172.21.15.43:0/3173402713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a862e20 0x55bc7a8949b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9e900057a0 tx=0x7f9e90005880 comp rx=0 tx=0).stop 2024-04-21T08:39:31.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.760+0000 7f9eab375740 1 -- 172.21.15.43:0/3173402713 shutdown_connections 2024-04-21T08:39:31.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.760+0000 7f9eab375740 1 --2- 172.21.15.43:0/3173402713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc7a874210 0x55bc7a894ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.760+0000 7f9eab375740 1 --2- 172.21.15.43:0/3173402713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a862e20 0x55bc7a8949b0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.760+0000 7f9eab375740 1 --2- 172.21.15.43:0/3173402713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc7a862470 0x55bc7a8737d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.760+0000 7f9eab375740 1 -- 172.21.15.43:0/3173402713 >> 172.21.15.43:0/3173402713 conn(0x55bc7a85cba0 msgr2=0x55bc7a85e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:31.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.760+0000 7f9eab375740 1 -- 172.21.15.43:0/3173402713 shutdown_connections 2024-04-21T08:39:31.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.760+0000 7f9eab375740 1 -- 172.21.15.43:0/3173402713 wait complete. 2024-04-21T08:39:31.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.760+0000 7f9eab375740 1 -- start start 2024-04-21T08:39:31.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.760+0000 7f9eab375740 1 Processor -- start 2024-04-21T08:39:31.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.760+0000 7f9eab375740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a862470 0x55bc7a89dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.760+0000 7f9eab375740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc7a862e20 0x55bc7a89e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.760+0000 7f9eab375740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc7a874210 0x55bc7a8f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.760+0000 7f9eab375740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea01ae750 con 0x55bc7a862470 2024-04-21T08:39:31.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.760+0000 7f9eab375740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea01ae080 con 0x55bc7a874210 2024-04-21T08:39:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.760+0000 7f9eab375740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea01ab2c0 con 0x55bc7a862e20 2024-04-21T08:39:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.761+0000 7f9eab375740 10 client.0 _ll_register_callbacks cb 0x7f9e9c009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.761+0000 7f9ea6441640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a862470 0x55bc7a89dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.761+0000 7f9ea5c40640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc7a862e20 0x55bc7a89e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.761+0000 7f9ea6441640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a862470 0x55bc7a89dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49002/0 (socket says 172.21.15.43:49002) 2024-04-21T08:39:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.761+0000 7f9ea5c40640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc7a862e20 0x55bc7a89e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.43:44226/0 (socket says 172.21.15.43:44226) 2024-04-21T08:39:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.761+0000 7f9ea6441640 1 -- 172.21.15.43:0/2885560233 learned_addr learned my addr 172.21.15.43:0/2885560233 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.761+0000 7f9ea6c42640 1 --2- 172.21.15.43:0/2885560233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc7a874210 0x55bc7a8f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.761+0000 7f9ea5c40640 1 -- 172.21.15.43:0/2885560233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc7a874210 msgr2=0x55bc7a8f1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.761+0000 7f9ea5c40640 1 --2- 172.21.15.43:0/2885560233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc7a874210 0x55bc7a8f1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.761+0000 7f9ea5c40640 1 -- 172.21.15.43:0/2885560233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a862470 msgr2=0x55bc7a89dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.761+0000 7f9ea5c40640 1 --2- 172.21.15.43:0/2885560233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a862470 0x55bc7a89dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.761+0000 7f9ea5c40640 1 -- 172.21.15.43:0/2885560233 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc7a89eba0 con 0x55bc7a862e20 2024-04-21T08:39:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.761+0000 7f9ea6441640 1 --2- 172.21.15.43:0/2885560233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a862470 0x55bc7a89dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.761+0000 7f9ea6c42640 1 --2- 172.21.15.43:0/2885560233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc7a874210 0x55bc7a8f1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.761+0000 7f9ea5c40640 1 --2- 172.21.15.43:0/2885560233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc7a862e20 0x55bc7a89e2a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x55bc7a766a00 tx=0x7f9e9000efd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f859ff14578f0a4f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.761+0000 7f9e89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-04-21T08:39:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.761+0000 7f9e89ffb640 1 -- 172.21.15.43:0/2885560233 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9009c6a0 con 0x55bc7a862e20 2024-04-21T08:39:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.762+0000 7f9e89ffb640 1 -- 172.21.15.43:0/2885560233 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f9e900bd0b0 con 0x55bc7a862e20 2024-04-21T08:39:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.762+0000 7f9eab375740 1 -- 172.21.15.43:0/2885560233 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e9009d260 con 0x55bc7a862e20 2024-04-21T08:39:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.762+0000 7f9e89ffb640 1 -- 172.21.15.43:0/2885560233 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9e9009c930 con 0x55bc7a862e20 2024-04-21T08:39:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.762+0000 7f9eab375740 10 client.25528 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.762+0000 7f9eab375740 1 -- 172.21.15.43:0/2885560233 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc7a8f3380 con 0x55bc7a862e20 2024-04-21T08:39:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.762+0000 7f9e89ffb640 1 -- 172.21.15.43:0/2885560233 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e9009d260 con 0x55bc7a862e20 2024-04-21T08:39:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.762+0000 7f9e89ffb640 1 -- 172.21.15.43:0/2885560233 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9e9009cdc0 con 0x55bc7a862e20 2024-04-21T08:39:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.762+0000 7f9eab375740 10 client.25528 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.762+0000 7f9eab375740 10 client.25528 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:39:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.762+0000 7f9eab375740 20 client.25528 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.762+0000 7f9eab375740 1 --2- 172.21.15.43:0/2885560233 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55bc7a88ccb0 0x55bc7a8f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.762+0000 7f9eab375740 4 client.25528 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:39:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.762+0000 7f9eab375740 1 -- 172.21.15.43:0/2885560233 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc7a7d4c10 con 0x55bc7a88ccb0 2024-04-21T08:39:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.763+0000 7f9ea6441640 1 --2- 172.21.15.43:0/2885560233 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55bc7a88ccb0 0x55bc7a8f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.763+0000 7f9ea6441640 1 --2- 172.21.15.43:0/2885560233 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55bc7a88ccb0 0x55bc7a8f55b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1da81ada1855f2bc server_cookie=6138be7023dc0858 in_seq=0 out_seq=0 2024-04-21T08:39:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.763+0000 7f9e89ffb640 10 client.25528 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:39:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.763+0000 7f9e89ffb640 1 -- 172.21.15.43:0/2885560233 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc7a7d4c10 con 0x55bc7a88ccb0 2024-04-21T08:39:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.763+0000 7f9e89ffb640 10 client.25528 handle_command_reply: tid=0 2024-04-21T08:39:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.763+0000 7f9eab375740 1 client.25528 shutdown 2024-04-21T08:39:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.763+0000 7f9eab375740 1 -- 172.21.15.43:0/2885560233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc7a862e20 msgr2=0x55bc7a89e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.763+0000 7f9eab375740 1 --2- 172.21.15.43:0/2885560233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc7a862e20 0x55bc7a89e2a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x55bc7a766a00 tx=0x7f9e9000efd0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.763+0000 7f9eab375740 1 -- 172.21.15.43:0/2885560233 shutdown_connections 2024-04-21T08:39:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.763+0000 7f9eab375740 1 --2- 172.21.15.43:0/2885560233 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55bc7a88ccb0 0x55bc7a8f55b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.763+0000 7f9e89ffb640 0 client.25528 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:39:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.763+0000 7f9eab375740 1 --2- 172.21.15.43:0/2885560233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc7a874210 0x55bc7a8f1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.763+0000 7f9eab375740 1 --2- 172.21.15.43:0/2885560233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc7a862e20 0x55bc7a89e2a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.763+0000 7f9eab375740 1 --2- 172.21.15.43:0/2885560233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc7a862470 0x55bc7a89dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.763+0000 7f9eab375740 1 -- 172.21.15.43:0/2885560233 >> 172.21.15.43:0/2885560233 conn(0x55bc7a85cba0 msgr2=0x55bc7a863ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:31.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.763+0000 7f9eab375740 1 -- 172.21.15.43:0/2885560233 shutdown_connections 2024-04-21T08:39:31.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.764+0000 7f9eab375740 1 -- 172.21.15.43:0/2885560233 wait complete. 2024-04-21T08:39:31.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.764+0000 7f9eab375740 20 client.25528 trim_cache size 0 max 16384 2024-04-21T08:39:31.770 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:39:31.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.764+0000 7f9ea86a5640 1 -- 172.21.15.43:0/2910126527 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9e8c0a3db0 msgr2=0x7f9e8c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.764+0000 7f9ea86a5640 1 --2- 172.21.15.43:0/2910126527 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9e8c0a3db0 0x7f9e8c0a6270 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9e98004af0 tx=0x7f9e98005c50 comp rx=0 tx=0).stop 2024-04-21T08:39:31.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.764+0000 7f9ea86a5640 1 -- 172.21.15.43:0/2910126527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ea0106160 msgr2=0x7f9ea0198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.764+0000 7f9ea86a5640 1 --2- 172.21.15.43:0/2910126527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ea0106160 0x7f9ea0198c00 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000e9c0 tx=0x7f9e9000ee80 comp rx=0 tx=0).stop 2024-04-21T08:39:31.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.764+0000 7f9ea86a5640 1 -- 172.21.15.43:0/2910126527 shutdown_connections 2024-04-21T08:39:31.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.764+0000 7f9ea86a5640 1 --2- 172.21.15.43:0/2910126527 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9e8c0a3db0 0x7f9e8c0a6270 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.764+0000 7f9ea86a5640 1 --2- 172.21.15.43:0/2910126527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ea019cf80 0x7f9ea019d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.764+0000 7f9ea86a5640 1 --2- 172.21.15.43:0/2910126527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ea0106160 0x7f9ea0198c00 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.764+0000 7f9ea86a5640 1 --2- 172.21.15.43:0/2910126527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ea0101090 0x7f9ea01986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.764+0000 7f9ea86a5640 1 -- 172.21.15.43:0/2910126527 >> 172.21.15.43:0/2910126527 conn(0x7f9ea0077f50 msgr2=0x7f9ea00fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:31.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.764+0000 7f9ea86a5640 1 -- 172.21.15.43:0/2910126527 shutdown_connections 2024-04-21T08:39:31.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.765+0000 7f9ea86a5640 1 -- 172.21.15.43:0/2910126527 wait complete. 2024-04-21T08:39:31.777 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:31 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/876387357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:31.779 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:39:31.793 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:31 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/876387357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:31.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.885+0000 7f60173d5640 1 -- 172.21.15.43:0/1466341486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6010103af0 msgr2=0x7f601010bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.885+0000 7f60173d5640 1 --2- 172.21.15.43:0/1466341486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6010103af0 0x7f601010bfd0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6000009900 tx=0x7f600002bd60 comp rx=0 tx=0).stop 2024-04-21T08:39:31.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.885+0000 7f60173d5640 1 -- 172.21.15.43:0/1466341486 shutdown_connections 2024-04-21T08:39:31.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.885+0000 7f60173d5640 1 --2- 172.21.15.43:0/1466341486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f601010c510 0x7f601010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.885+0000 7f60173d5640 1 --2- 172.21.15.43:0/1466341486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6010103af0 0x7f601010bfd0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.885+0000 7f60173d5640 1 --2- 172.21.15.43:0/1466341486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60101031d0 0x7f60101035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.885+0000 7f60173d5640 1 -- 172.21.15.43:0/1466341486 >> 172.21.15.43:0/1466341486 conn(0x7f6010077f50 msgr2=0x7f6010078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:31.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.885+0000 7f60173d5640 1 -- 172.21.15.43:0/1466341486 shutdown_connections 2024-04-21T08:39:31.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.885+0000 7f60173d5640 1 -- 172.21.15.43:0/1466341486 wait complete. 2024-04-21T08:39:31.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.886+0000 7f60173d5640 1 Processor -- start 2024-04-21T08:39:31.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.886+0000 7f60173d5640 1 -- start start 2024-04-21T08:39:31.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.886+0000 7f60173d5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60101031d0 0x7f6010198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.886+0000 7f60173d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6010103af0 0x7f6010198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.886+0000 7f60173d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f601010c510 0x7f601019d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.886+0000 7f60173d5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f60101121a0 con 0x7f601010c510 2024-04-21T08:39:31.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.886+0000 7f60173d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6010112020 con 0x7f6010103af0 2024-04-21T08:39:31.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.886+0000 7f60173d5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6010112320 con 0x7f60101031d0 2024-04-21T08:39:31.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.886+0000 7f6015972640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f601010c510 0x7f601019d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.886+0000 7f6015972640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f601010c510 0x7f601019d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49032/0 (socket says 172.21.15.43:49032) 2024-04-21T08:39:31.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.886+0000 7f6015972640 1 -- 172.21.15.43:0/952069429 learned_addr learned my addr 172.21.15.43:0/952069429 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:31.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.886+0000 7f6015171640 1 --2- 172.21.15.43:0/952069429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60101031d0 0x7f6010198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.887+0000 7f6014970640 1 --2- 172.21.15.43:0/952069429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6010103af0 0x7f6010198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.887+0000 7f6015972640 1 -- 172.21.15.43:0/952069429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60101031d0 msgr2=0x7f6010198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.887+0000 7f6015972640 1 --2- 172.21.15.43:0/952069429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60101031d0 0x7f6010198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.887+0000 7f6015972640 1 -- 172.21.15.43:0/952069429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6010103af0 msgr2=0x7f6010198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.887+0000 7f6015972640 1 --2- 172.21.15.43:0/952069429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6010103af0 0x7f6010198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.887+0000 7f6015972640 1 -- 172.21.15.43:0/952069429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601019d730 con 0x7f601010c510 2024-04-21T08:39:31.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.887+0000 7f6015972640 1 --2- 172.21.15.43:0/952069429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f601010c510 0x7f601019d000 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f600800eaa0 tx=0x7f600800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:31.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.887+0000 7f60067fc640 1 -- 172.21.15.43:0/952069429 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600800cdb0 con 0x7f601010c510 2024-04-21T08:39:31.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.887+0000 7f60067fc640 1 -- 172.21.15.43:0/952069429 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6008005020 con 0x7f601010c510 2024-04-21T08:39:31.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.887+0000 7f60067fc640 1 -- 172.21.15.43:0/952069429 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60080116d0 con 0x7f601010c510 2024-04-21T08:39:31.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.887+0000 7f60173d5640 1 -- 172.21.15.43:0/952069429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601019da20 con 0x7f601010c510 2024-04-21T08:39:31.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.887+0000 7f60173d5640 1 -- 172.21.15.43:0/952069429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60101a52b0 con 0x7f601010c510 2024-04-21T08:39:31.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.890+0000 7f60067fc640 1 -- 172.21.15.43:0/952069429 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f60080051d0 con 0x7f601010c510 2024-04-21T08:39:31.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.890+0000 7f60067fc640 1 --2- 172.21.15.43:0/952069429 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5ffc0a3e30 0x7f5ffc0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.890+0000 7f6015171640 1 --2- 172.21.15.43:0/952069429 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5ffc0a3e30 0x7f5ffc0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.890+0000 7f60067fc640 1 -- 172.21.15.43:0/952069429 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f60080d0dc0 con 0x7f601010c510 2024-04-21T08:39:31.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.891+0000 7f6019fc6740 1 Processor -- start 2024-04-21T08:39:31.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.891+0000 7f6019fc6740 1 -- start start 2024-04-21T08:39:31.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.891+0000 7f6015171640 1 --2- 172.21.15.43:0/952069429 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5ffc0a3e30 0x7f5ffc0a62f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f600c005fd0 tx=0x7f600c00b390 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:31.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.891+0000 7f6019fc6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016985d3b0 0x56016985d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.891+0000 7f6019fc6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016985dd60 0x560169866890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.891+0000 7f6019fc6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560169866e60 0x5601698672c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.891+0000 7f6019fc6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5601697c21f0 con 0x56016985dd60 2024-04-21T08:39:31.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.891+0000 7f6019fc6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56016975f4b0 con 0x56016985d3b0 2024-04-21T08:39:31.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.891+0000 7f6019fc6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5601697be1b0 con 0x560169866e60 2024-04-21T08:39:31.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.891+0000 7f6014970640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016985dd60 0x560169866890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.891+0000 7f6014970640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016985dd60 0x560169866890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49048/0 (socket says 172.21.15.43:49048) 2024-04-21T08:39:31.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.892+0000 7f6014970640 1 -- 172.21.15.43:0/1572388656 learned_addr learned my addr 172.21.15.43:0/1572388656 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:31.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.892+0000 7f6015171640 1 --2- 172.21.15.43:0/1572388656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016985d3b0 0x56016985d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.892+0000 7f6015972640 1 --2- 172.21.15.43:0/1572388656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560169866e60 0x5601698672c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.892+0000 7f6014970640 1 -- 172.21.15.43:0/1572388656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560169866e60 msgr2=0x5601698672c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.892+0000 7f6014970640 1 --2- 172.21.15.43:0/1572388656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560169866e60 0x5601698672c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.892+0000 7f6014970640 1 -- 172.21.15.43:0/1572388656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016985d3b0 msgr2=0x56016985d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.892+0000 7f6014970640 1 --2- 172.21.15.43:0/1572388656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016985d3b0 0x56016985d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.892+0000 7f6014970640 1 -- 172.21.15.43:0/1572388656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601697511b0 con 0x56016985dd60 2024-04-21T08:39:31.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.892+0000 7f6015171640 1 --2- 172.21.15.43:0/1572388656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016985d3b0 0x56016985d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:31.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.892+0000 7f6014970640 1 --2- 172.21.15.43:0/1572388656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016985dd60 0x560169866890 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6000009900 tx=0x7f600002c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f285bdd70ad1b00d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:31.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.892+0000 7f5ffb7fe640 1 -- 172.21.15.43:0/1572388656 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000004180 con 0x56016985dd60 2024-04-21T08:39:31.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.892+0000 7f5ffb7fe640 1 -- 172.21.15.43:0/1572388656 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f600002cdc0 con 0x56016985dd60 2024-04-21T08:39:31.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.892+0000 7f5ffb7fe640 1 -- 172.21.15.43:0/1572388656 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60000046c0 con 0x56016985dd60 2024-04-21T08:39:31.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.893+0000 7f6019fc6740 1 -- 172.21.15.43:0/1572388656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016985dd60 msgr2=0x560169866890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.893+0000 7f6019fc6740 1 --2- 172.21.15.43:0/1572388656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016985dd60 0x560169866890 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6000009900 tx=0x7f600002c7f0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.893+0000 7f6019fc6740 1 -- 172.21.15.43:0/1572388656 shutdown_connections 2024-04-21T08:39:31.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.893+0000 7f6019fc6740 1 --2- 172.21.15.43:0/1572388656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560169866e60 0x5601698672c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.893+0000 7f6019fc6740 1 --2- 172.21.15.43:0/1572388656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016985dd60 0x560169866890 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.893+0000 7f6019fc6740 1 --2- 172.21.15.43:0/1572388656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016985d3b0 0x56016985d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.893+0000 7f6019fc6740 1 -- 172.21.15.43:0/1572388656 >> 172.21.15.43:0/1572388656 conn(0x560169857ba0 msgr2=0x560169857f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:31.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.893+0000 7f6019fc6740 1 -- 172.21.15.43:0/1572388656 shutdown_connections 2024-04-21T08:39:31.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.894+0000 7f6019fc6740 1 -- 172.21.15.43:0/1572388656 wait complete. 2024-04-21T08:39:31.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.894+0000 7f6019fc6740 1 -- start start 2024-04-21T08:39:31.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.894+0000 7f6019fc6740 1 Processor -- start 2024-04-21T08:39:31.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.894+0000 7f6019fc6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016985d3b0 0x56016986d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.894+0000 7f6019fc6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016985dd60 0x56016986a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.895+0000 7f6019fc6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560169866e60 0x56016986aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.895+0000 7f6019fc6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56016986dd40 con 0x560169866e60 2024-04-21T08:39:31.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.895+0000 7f6019fc6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56016977b500 con 0x56016985dd60 2024-04-21T08:39:31.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.895+0000 7f6019fc6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56016986dec0 con 0x56016985d3b0 2024-04-21T08:39:31.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.895+0000 7f6019fc6740 10 client.0 _ll_register_callbacks cb 0x560169749b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:31.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.895+0000 7f6015972640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560169866e60 0x56016986aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.895+0000 7f6015972640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560169866e60 0x56016986aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49054/0 (socket says 172.21.15.43:49054) 2024-04-21T08:39:31.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.895+0000 7f6015972640 1 -- 172.21.15.43:0/3122915306 learned_addr learned my addr 172.21.15.43:0/3122915306 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:31.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.895+0000 7f6015171640 1 --2- 172.21.15.43:0/3122915306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016985d3b0 0x56016986d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.895+0000 7f6014970640 1 --2- 172.21.15.43:0/3122915306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016985dd60 0x56016986a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.895+0000 7f6015972640 1 -- 172.21.15.43:0/3122915306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016985d3b0 msgr2=0x56016986d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.895+0000 7f6015972640 1 --2- 172.21.15.43:0/3122915306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016985d3b0 0x56016986d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.895+0000 7f6015972640 1 -- 172.21.15.43:0/3122915306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016985dd60 msgr2=0x56016986a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.895+0000 7f6015972640 1 --2- 172.21.15.43:0/3122915306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016985dd60 0x56016986a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.895+0000 7f6015972640 1 -- 172.21.15.43:0/3122915306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601694d43f0 con 0x560169866e60 2024-04-21T08:39:31.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.895+0000 7f6015972640 1 --2- 172.21.15.43:0/3122915306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560169866e60 0x56016986aaf0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f60080ce160 tx=0x7f60080a0bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=710c8da39a1d57f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:31.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.895+0000 7f5ff8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:31.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.895+0000 7f5ff8ff9640 1 -- 172.21.15.43:0/3122915306 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600801f040 con 0x560169866e60 2024-04-21T08:39:31.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.896+0000 7f6019fc6740 1 -- 172.21.15.43:0/3122915306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f600002cdc0 con 0x560169866e60 2024-04-21T08:39:31.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.896+0000 7f5ff8ff9640 1 -- 172.21.15.43:0/3122915306 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f60080870b0 con 0x560169866e60 2024-04-21T08:39:31.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.896+0000 7f5ff8ff9640 1 -- 172.21.15.43:0/3122915306 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f60080ce530 con 0x560169866e60 2024-04-21T08:39:31.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.896+0000 7f6019fc6740 10 client.16128 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:31.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.896+0000 7f6019fc6740 1 -- 172.21.15.43:0/3122915306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56016959e7d0 con 0x560169866e60 2024-04-21T08:39:31.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.896+0000 7f5ff8ff9640 1 -- 172.21.15.43:0/3122915306 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f600002cdc0 con 0x560169866e60 2024-04-21T08:39:31.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.896+0000 7f5ff8ff9640 1 -- 172.21.15.43:0/3122915306 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f600800f040 con 0x560169866e60 2024-04-21T08:39:31.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.896+0000 7f6019fc6740 10 client.16128 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:31.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.896+0000 7f6019fc6740 10 client.16128 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:39:31.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.896+0000 7f6019fc6740 20 client.16128 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:31.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.896+0000 7f6019fc6740 1 --2- 172.21.15.43:0/3122915306 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x56016987b030 0x56016987d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.896+0000 7f6019fc6740 4 client.16128 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:31.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.896+0000 7f6019fc6740 1 -- 172.21.15.43:0/3122915306 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560169761930 con 0x56016987b030 2024-04-21T08:39:31.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.896+0000 7f6015171640 1 --2- 172.21.15.43:0/3122915306 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x56016987b030 0x56016987d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.897+0000 7f6015171640 1 --2- 172.21.15.43:0/3122915306 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x56016987b030 0x56016987d450 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=867158635d008b10 server_cookie=36b1c01b44b055da in_seq=0 out_seq=0 2024-04-21T08:39:31.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.897+0000 7f5ff8ff9640 10 client.16128 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:39:31.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.898+0000 7f5ff8ff9640 1 -- 172.21.15.43:0/3122915306 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560169761930 con 0x56016987b030 2024-04-21T08:39:31.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.898+0000 7f5ff8ff9640 10 client.16128 handle_command_reply: tid=0 2024-04-21T08:39:31.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.898+0000 7f6019fc6740 1 client.16128 shutdown 2024-04-21T08:39:31.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.898+0000 7f6019fc6740 1 -- 172.21.15.43:0/3122915306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560169866e60 msgr2=0x56016986aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.898+0000 7f6019fc6740 1 --2- 172.21.15.43:0/3122915306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560169866e60 0x56016986aaf0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f60080ce160 tx=0x7f60080a0bd0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.898+0000 7f6019fc6740 1 -- 172.21.15.43:0/3122915306 shutdown_connections 2024-04-21T08:39:31.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.898+0000 7f6019fc6740 1 --2- 172.21.15.43:0/3122915306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560169866e60 0x56016986aaf0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.898+0000 7f6019fc6740 1 --2- 172.21.15.43:0/3122915306 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x56016987b030 0x56016987d450 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.898+0000 7f6019fc6740 1 --2- 172.21.15.43:0/3122915306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016985dd60 0x56016986a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.898+0000 7f6019fc6740 1 --2- 172.21.15.43:0/3122915306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016985d3b0 0x56016986d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.898+0000 7f6019fc6740 1 -- 172.21.15.43:0/3122915306 >> 172.21.15.43:0/3122915306 conn(0x560169857ba0 msgr2=0x5601698598f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:31.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.898+0000 7f5ff8ff9640 0 client.16128 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:39:31.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.898+0000 7f6019fc6740 1 -- 172.21.15.43:0/3122915306 shutdown_connections 2024-04-21T08:39:31.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.898+0000 7f6019fc6740 1 -- 172.21.15.43:0/3122915306 wait complete. 2024-04-21T08:39:31.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.898+0000 7f6019fc6740 20 client.16128 trim_cache size 0 max 16384 2024-04-21T08:39:31.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.912+0000 7f6019fc6740 1 Processor -- start 2024-04-21T08:39:31.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.912+0000 7f6019fc6740 1 -- start start 2024-04-21T08:39:31.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.912+0000 7f6019fc6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016985d3b0 0x560169876c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.912+0000 7f6019fc6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016985dd60 0x560169865800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.912+0000 7f6019fc6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560169866e60 0x56016986a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.912+0000 7f6019fc6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f60101ad150 con 0x56016985dd60 2024-04-21T08:39:31.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.912+0000 7f6019fc6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f60101aec90 con 0x56016985d3b0 2024-04-21T08:39:31.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.912+0000 7f6019fc6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f60101ad820 con 0x560169866e60 2024-04-21T08:39:31.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.912+0000 7f6014970640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016985dd60 0x560169865800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.912+0000 7f6014970640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016985dd60 0x560169865800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49062/0 (socket says 172.21.15.43:49062) 2024-04-21T08:39:31.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.912+0000 7f6014970640 1 -- 172.21.15.43:0/3530143039 learned_addr learned my addr 172.21.15.43:0/3530143039 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:31.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.912+0000 7f6015171640 1 --2- 172.21.15.43:0/3530143039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016985d3b0 0x560169876c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.912+0000 7f6015972640 1 --2- 172.21.15.43:0/3530143039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560169866e60 0x56016986a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.913+0000 7f6014970640 1 -- 172.21.15.43:0/3530143039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560169866e60 msgr2=0x56016986a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.913+0000 7f6014970640 1 --2- 172.21.15.43:0/3530143039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560169866e60 0x56016986a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.913+0000 7f6014970640 1 -- 172.21.15.43:0/3530143039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016985d3b0 msgr2=0x560169876c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.913+0000 7f6014970640 1 --2- 172.21.15.43:0/3530143039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016985d3b0 0x560169876c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.913+0000 7f6014970640 1 -- 172.21.15.43:0/3530143039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560169853470 con 0x56016985dd60 2024-04-21T08:39:31.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.913+0000 7f6015171640 1 --2- 172.21.15.43:0/3530143039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016985d3b0 0x560169876c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:31.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.913+0000 7f6015972640 1 --2- 172.21.15.43:0/3530143039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560169866e60 0x56016986a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:31.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.913+0000 7f6014970640 1 --2- 172.21.15.43:0/3530143039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016985dd60 0x560169865800 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f6000005380 tx=0x7f6000005450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e50fb77942790f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:31.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.913+0000 7f5ffaffd640 1 -- 172.21.15.43:0/3530143039 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600002c910 con 0x56016985dd60 2024-04-21T08:39:31.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.913+0000 7f5ffaffd640 1 -- 172.21.15.43:0/3530143039 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6000034440 con 0x56016985dd60 2024-04-21T08:39:31.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.913+0000 7f5ffaffd640 1 -- 172.21.15.43:0/3530143039 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600003f690 con 0x56016985dd60 2024-04-21T08:39:31.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.914+0000 7f6019fc6740 1 -- 172.21.15.43:0/3530143039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016985dd60 msgr2=0x560169865800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.914+0000 7f6019fc6740 1 --2- 172.21.15.43:0/3530143039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016985dd60 0x560169865800 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f6000005380 tx=0x7f6000005450 comp rx=0 tx=0).stop 2024-04-21T08:39:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.914+0000 7f6019fc6740 1 -- 172.21.15.43:0/3530143039 shutdown_connections 2024-04-21T08:39:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.914+0000 7f6019fc6740 1 --2- 172.21.15.43:0/3530143039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560169866e60 0x56016986a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.914+0000 7f6019fc6740 1 --2- 172.21.15.43:0/3530143039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016985dd60 0x560169865800 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.914+0000 7f6019fc6740 1 --2- 172.21.15.43:0/3530143039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016985d3b0 0x560169876c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.914+0000 7f6019fc6740 1 -- 172.21.15.43:0/3530143039 >> 172.21.15.43:0/3530143039 conn(0x560169857ba0 msgr2=0x560169859830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.914+0000 7f6019fc6740 1 -- 172.21.15.43:0/3530143039 shutdown_connections 2024-04-21T08:39:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.914+0000 7f6019fc6740 1 -- 172.21.15.43:0/3530143039 wait complete. 2024-04-21T08:39:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.915+0000 7f6019fc6740 1 -- start start 2024-04-21T08:39:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.915+0000 7f6019fc6740 1 Processor -- start 2024-04-21T08:39:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.915+0000 7f6019fc6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016985d3b0 0x5601698733e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.915+0000 7f6019fc6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016985dd60 0x56016986bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.915+0000 7f6019fc6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560169866e60 0x56016986c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.915+0000 7f6019fc6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f60101ae5c0 con 0x560169866e60 2024-04-21T08:39:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.915+0000 7f6019fc6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f60101adef0 con 0x56016985d3b0 2024-04-21T08:39:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.915+0000 7f6019fc6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f60101ab130 con 0x56016985dd60 2024-04-21T08:39:31.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.915+0000 7f6019fc6740 10 client.0 _ll_register_callbacks cb 0x7f600809c2a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:31.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.915+0000 7f6015972640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560169866e60 0x56016986c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.915+0000 7f6015972640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560169866e60 0x56016986c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49064/0 (socket says 172.21.15.43:49064) 2024-04-21T08:39:31.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.915+0000 7f6015972640 1 -- 172.21.15.43:0/1779101913 learned_addr learned my addr 172.21.15.43:0/1779101913 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:31.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.915+0000 7f6015171640 1 --2- 172.21.15.43:0/1779101913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016985d3b0 0x5601698733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.916+0000 7f6014970640 1 --2- 172.21.15.43:0/1779101913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016985dd60 0x56016986bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.916+0000 7f6015972640 1 -- 172.21.15.43:0/1779101913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016985dd60 msgr2=0x56016986bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.916+0000 7f6015972640 1 --2- 172.21.15.43:0/1779101913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016985dd60 0x56016986bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.916+0000 7f6015972640 1 -- 172.21.15.43:0/1779101913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016985d3b0 msgr2=0x5601698733e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:31.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.916+0000 7f6015972640 1 --2- 172.21.15.43:0/1779101913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016985d3b0 0x5601698733e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.916+0000 7f6015972640 1 -- 172.21.15.43:0/1779101913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56016986c880 con 0x560169866e60 2024-04-21T08:39:31.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.916+0000 7f6015972640 1 --2- 172.21.15.43:0/1779101913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560169866e60 0x56016986c240 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f60080992a0 tx=0x7f600809c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a7c32b6360684a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:31.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.916+0000 7f5ff8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:31.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.916+0000 7f5ff8ff9640 1 -- 172.21.15.43:0/1779101913 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600801f040 con 0x560169866e60 2024-04-21T08:39:31.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.916+0000 7f6019fc6740 1 -- 172.21.15.43:0/1779101913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6000034440 con 0x560169866e60 2024-04-21T08:39:31.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.916+0000 7f5ff8ff9640 1 -- 172.21.15.43:0/1779101913 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f600809bf30 con 0x560169866e60 2024-04-21T08:39:31.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.916+0000 7f5ff8ff9640 1 -- 172.21.15.43:0/1779101913 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6008098b70 con 0x560169866e60 2024-04-21T08:39:31.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.916+0000 7f6019fc6740 10 client.16134 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:31.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.916+0000 7f6019fc6740 1 -- 172.21.15.43:0/1779101913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5601698e1430 con 0x560169866e60 2024-04-21T08:39:31.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.917+0000 7f5ff8ff9640 1 -- 172.21.15.43:0/1779101913 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6000034440 con 0x560169866e60 2024-04-21T08:39:31.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.917+0000 7f5ff8ff9640 1 -- 172.21.15.43:0/1779101913 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f60080a1d60 con 0x560169866e60 2024-04-21T08:39:31.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.917+0000 7f6019fc6740 10 client.16134 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:31.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.917+0000 7f6019fc6740 10 client.16134 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:39:31.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.917+0000 7f6019fc6740 20 client.16134 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:31.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.917+0000 7f6019fc6740 1 --2- 172.21.15.43:0/1779101913 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5601698e19f0 0x5601698e3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:31.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.917+0000 7f6019fc6740 4 client.16134 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:39:31.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.917+0000 7f6019fc6740 1 -- 172.21.15.43:0/1779101913 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5601697cfc10 con 0x5601698e19f0 2024-04-21T08:39:31.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.917+0000 7f6015171640 1 --2- 172.21.15.43:0/1779101913 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5601698e19f0 0x5601698e3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.917+0000 7f5ff8ff9640 10 client.16134 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:39:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.917+0000 7f6015171640 1 --2- 172.21.15.43:0/1779101913 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5601698e19f0 0x5601698e3e10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4abb3397e5850531 server_cookie=f7724a3303055bb2 in_seq=0 out_seq=0 2024-04-21T08:39:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.918+0000 7f5ff8ff9640 1 -- 172.21.15.43:0/1779101913 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5601697cfc10 con 0x5601698e19f0 2024-04-21T08:39:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.918+0000 7f5ff8ff9640 10 client.16134 handle_command_reply: tid=0 2024-04-21T08:39:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.918+0000 7f6019fc6740 1 client.16134 shutdown 2024-04-21T08:39:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.918+0000 7f6019fc6740 1 -- 172.21.15.43:0/1779101913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560169866e60 msgr2=0x56016986c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.918+0000 7f6019fc6740 1 --2- 172.21.15.43:0/1779101913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560169866e60 0x56016986c240 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f60080992a0 tx=0x7f600809c230 comp rx=0 tx=0).stop 2024-04-21T08:39:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.918+0000 7f6019fc6740 1 -- 172.21.15.43:0/1779101913 shutdown_connections 2024-04-21T08:39:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.918+0000 7f6019fc6740 1 --2- 172.21.15.43:0/1779101913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560169866e60 0x56016986c240 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.918+0000 7f6019fc6740 1 --2- 172.21.15.43:0/1779101913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016985dd60 0x56016986bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.918+0000 7f6019fc6740 1 --2- 172.21.15.43:0/1779101913 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5601698e19f0 0x5601698e3e10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.918+0000 7f6019fc6740 1 --2- 172.21.15.43:0/1779101913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016985d3b0 0x5601698733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.918+0000 7f5ff8ff9640 0 client.16134 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:39:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.918+0000 7f6019fc6740 1 -- 172.21.15.43:0/1779101913 >> 172.21.15.43:0/1779101913 conn(0x560169857ba0 msgr2=0x560169856370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.918+0000 7f6019fc6740 1 -- 172.21.15.43:0/1779101913 shutdown_connections 2024-04-21T08:39:31.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.918+0000 7f6019fc6740 1 -- 172.21.15.43:0/1779101913 wait complete. 2024-04-21T08:39:31.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.918+0000 7f6019fc6740 20 client.16134 trim_cache size 0 max 16384 2024-04-21T08:39:31.924 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:39:31.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.919+0000 7f60173d5640 1 -- 172.21.15.43:0/952069429 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5ffc0a3e30 msgr2=0x7f5ffc0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.919+0000 7f60173d5640 1 --2- 172.21.15.43:0/952069429 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5ffc0a3e30 0x7f5ffc0a62f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f600c005fd0 tx=0x7f600c00b390 comp rx=0 tx=0).stop 2024-04-21T08:39:31.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.919+0000 7f60173d5640 1 -- 172.21.15.43:0/952069429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f601010c510 msgr2=0x7f601019d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:31.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.919+0000 7f60173d5640 1 --2- 172.21.15.43:0/952069429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f601010c510 0x7f601019d000 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f600800eaa0 tx=0x7f600800ef60 comp rx=0 tx=0).stop 2024-04-21T08:39:31.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.919+0000 7f60173d5640 1 -- 172.21.15.43:0/952069429 shutdown_connections 2024-04-21T08:39:31.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.919+0000 7f60173d5640 1 --2- 172.21.15.43:0/952069429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f601010c510 0x7f601019d000 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.919+0000 7f60173d5640 1 --2- 172.21.15.43:0/952069429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6010103af0 0x7f6010198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.919+0000 7f60173d5640 1 --2- 172.21.15.43:0/952069429 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5ffc0a3e30 0x7f5ffc0a62f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.919+0000 7f60173d5640 1 --2- 172.21.15.43:0/952069429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60101031d0 0x7f6010198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:31.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.919+0000 7f60173d5640 1 -- 172.21.15.43:0/952069429 >> 172.21.15.43:0/952069429 conn(0x7f6010077f50 msgr2=0x7f60100fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:31.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.919+0000 7f60173d5640 1 -- 172.21.15.43:0/952069429 shutdown_connections 2024-04-21T08:39:31.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:31.919+0000 7f60173d5640 1 -- 172.21.15.43:0/952069429 wait complete. 2024-04-21T08:39:31.930 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:39:31.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:31 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/876387357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:32.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.048+0000 7fc3f05ef640 1 -- 172.21.15.43:0/2774171792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3e81031d0 msgr2=0x7fc3e81035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:32.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.048+0000 7fc3f05ef640 1 --2- 172.21.15.43:0/2774171792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3e81031d0 0x7fc3e81035b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009900 tx=0x7fc3e402bd60 comp rx=0 tx=0).stop 2024-04-21T08:39:32.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.048+0000 7fc3f05ef640 1 -- 172.21.15.43:0/2774171792 shutdown_connections 2024-04-21T08:39:32.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.048+0000 7fc3f05ef640 1 --2- 172.21.15.43:0/2774171792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e810c510 0x7fc3e810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.048+0000 7fc3f05ef640 1 --2- 172.21.15.43:0/2774171792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3e8103af0 0x7fc3e810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.048+0000 7fc3f05ef640 1 --2- 172.21.15.43:0/2774171792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3e81031d0 0x7fc3e81035b0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.048+0000 7fc3f05ef640 1 -- 172.21.15.43:0/2774171792 >> 172.21.15.43:0/2774171792 conn(0x7fc3e8077f50 msgr2=0x7fc3e8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:32.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.048+0000 7fc3f05ef640 1 -- 172.21.15.43:0/2774171792 shutdown_connections 2024-04-21T08:39:32.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.048+0000 7fc3f05ef640 1 -- 172.21.15.43:0/2774171792 wait complete. 2024-04-21T08:39:32.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.049+0000 7fc3f05ef640 1 Processor -- start 2024-04-21T08:39:32.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.049+0000 7fc3f05ef640 1 -- start start 2024-04-21T08:39:32.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.049+0000 7fc3f05ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3e81031d0 0x7fc3e8100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.049+0000 7fc3f05ef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e8103af0 0x7fc3e8101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.049+0000 7fc3f05ef640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3e810c510 0x7fc3e81018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.049+0000 7fc3f05ef640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e81121f0 con 0x7fc3e81031d0 2024-04-21T08:39:32.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.049+0000 7fc3f05ef640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8112070 con 0x7fc3e8103af0 2024-04-21T08:39:32.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.049+0000 7fc3f05ef640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8112370 con 0x7fc3e810c510 2024-04-21T08:39:32.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.049+0000 7fc3ee38b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3e81031d0 0x7fc3e8100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.049+0000 7fc3ee38b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3e81031d0 0x7fc3e8100e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49090/0 (socket says 172.21.15.43:49090) 2024-04-21T08:39:32.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.049+0000 7fc3ee38b640 1 -- 172.21.15.43:0/3734685309 learned_addr learned my addr 172.21.15.43:0/3734685309 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:32.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.050+0000 7fc3edb8a640 1 --2- 172.21.15.43:0/3734685309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e8103af0 0x7fc3e8101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.050+0000 7fc3eeb8c640 1 --2- 172.21.15.43:0/3734685309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3e810c510 0x7fc3e81018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.050+0000 7fc3ee38b640 1 -- 172.21.15.43:0/3734685309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3e810c510 msgr2=0x7fc3e81018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:32.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.050+0000 7fc3ee38b640 1 --2- 172.21.15.43:0/3734685309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3e810c510 0x7fc3e81018a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.050+0000 7fc3ee38b640 1 -- 172.21.15.43:0/3734685309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e8103af0 msgr2=0x7fc3e8101360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:32.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.050+0000 7fc3ee38b640 1 --2- 172.21.15.43:0/3734685309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e8103af0 0x7fc3e8101360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.050+0000 7fc3ee38b640 1 -- 172.21.15.43:0/3734685309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e81a6d40 con 0x7fc3e81031d0 2024-04-21T08:39:32.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.050+0000 7fc3ee38b640 1 --2- 172.21.15.43:0/3734685309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3e81031d0 0x7fc3e8100e20 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4000c00 tx=0x7fc3e402d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:32.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.050+0000 7fc3df7fe640 1 -- 172.21.15.43:0/3734685309 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4043020 con 0x7fc3e81031d0 2024-04-21T08:39:32.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.050+0000 7fc3f05ef640 1 -- 172.21.15.43:0/3734685309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e81a6fd0 con 0x7fc3e81031d0 2024-04-21T08:39:32.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.050+0000 7fc3f05ef640 1 -- 172.21.15.43:0/3734685309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e81a74e0 con 0x7fc3e81031d0 2024-04-21T08:39:32.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.050+0000 7fc3df7fe640 1 -- 172.21.15.43:0/3734685309 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3e402dde0 con 0x7fc3e81031d0 2024-04-21T08:39:32.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.050+0000 7fc3df7fe640 1 -- 172.21.15.43:0/3734685309 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4003b80 con 0x7fc3e81031d0 2024-04-21T08:39:32.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.054+0000 7fc3df7fe640 1 -- 172.21.15.43:0/3734685309 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fc3e4046020 con 0x7fc3e81031d0 2024-04-21T08:39:32.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.054+0000 7fc3f31d9740 1 Processor -- start 2024-04-21T08:39:32.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.054+0000 7fc3f31d9740 1 -- start start 2024-04-21T08:39:32.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.054+0000 7fc3f31d9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567aae7b4d0 0x5567aae7b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.054+0000 7fc3f31d9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567aae7be80 0x5567aae84430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.055+0000 7fc3f31d9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae84a00 0x5567aae86e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.055+0000 7fc3f31d9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5567aade01f0 con 0x5567aae84a00 2024-04-21T08:39:32.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.055+0000 7fc3f31d9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5567aad7d4b0 con 0x5567aae7b4d0 2024-04-21T08:39:32.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.055+0000 7fc3f31d9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5567aaddc1b0 con 0x5567aae7be80 2024-04-21T08:39:32.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.055+0000 7fc3edb8a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae84a00 0x5567aae86e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.055+0000 7fc3edb8a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae84a00 0x5567aae86e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49096/0 (socket says 172.21.15.43:49096) 2024-04-21T08:39:32.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.055+0000 7fc3edb8a640 1 -- 172.21.15.43:0/3627379367 learned_addr learned my addr 172.21.15.43:0/3627379367 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:32.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.055+0000 7fc3ee38b640 1 --2- 172.21.15.43:0/3627379367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567aae7be80 0x5567aae84430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.055+0000 7fc3eeb8c640 1 --2- 172.21.15.43:0/3627379367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567aae7b4d0 0x5567aae7b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.055+0000 7fc3edb8a640 1 -- 172.21.15.43:0/3627379367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567aae7be80 msgr2=0x5567aae84430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:32.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.055+0000 7fc3edb8a640 1 --2- 172.21.15.43:0/3627379367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567aae7be80 0x5567aae84430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.055+0000 7fc3edb8a640 1 -- 172.21.15.43:0/3627379367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567aae7b4d0 msgr2=0x5567aae7b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:32.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.055+0000 7fc3edb8a640 1 --2- 172.21.15.43:0/3627379367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567aae7b4d0 0x5567aae7b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.055+0000 7fc3edb8a640 1 -- 172.21.15.43:0/3627379367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567aad6f1b0 con 0x5567aae84a00 2024-04-21T08:39:32.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.055+0000 7fc3edb8a640 1 --2- 172.21.15.43:0/3627379367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae84a00 0x5567aae86e50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc3d800b740 tx=0x7fc3d800bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeaa7162354cacae server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:32.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.055+0000 7fc3df7fe640 1 --2- 172.21.15.43:0/3734685309 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc3d40a3e30 0x7fc3d40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.056+0000 7fc3eeb8c640 1 --2- 172.21.15.43:0/3734685309 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc3d40a3e30 0x7fc3d40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.056+0000 7fc3dcbfe640 1 -- 172.21.15.43:0/3627379367 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8003dd0 con 0x5567aae84a00 2024-04-21T08:39:32.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.056+0000 7fc3dcbfe640 1 -- 172.21.15.43:0/3627379367 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3d8015070 con 0x5567aae84a00 2024-04-21T08:39:32.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.056+0000 7fc3dcbfe640 1 -- 172.21.15.43:0/3627379367 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8004d90 con 0x5567aae84a00 2024-04-21T08:39:32.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.056+0000 7fc3df7fe640 1 -- 172.21.15.43:0/3734685309 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc3e40f2cf0 con 0x7fc3e81031d0 2024-04-21T08:39:32.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.057+0000 7fc3f31d9740 1 -- 172.21.15.43:0/3627379367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae84a00 msgr2=0x5567aae86e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:32.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.057+0000 7fc3f31d9740 1 --2- 172.21.15.43:0/3627379367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae84a00 0x5567aae86e50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc3d800b740 tx=0x7fc3d800bc00 comp rx=0 tx=0).stop 2024-04-21T08:39:32.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.057+0000 7fc3eeb8c640 1 --2- 172.21.15.43:0/3734685309 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc3d40a3e30 0x7fc3d40a62f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0005fd0 tx=0x7fc3e0009760 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:32.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.057+0000 7fc3f31d9740 1 -- 172.21.15.43:0/3627379367 shutdown_connections 2024-04-21T08:39:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.057+0000 7fc3f31d9740 1 --2- 172.21.15.43:0/3627379367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae84a00 0x5567aae86e50 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.057+0000 7fc3f31d9740 1 --2- 172.21.15.43:0/3627379367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567aae7be80 0x5567aae84430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.057+0000 7fc3f31d9740 1 --2- 172.21.15.43:0/3627379367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567aae7b4d0 0x5567aae7b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.057+0000 7fc3f31d9740 1 -- 172.21.15.43:0/3627379367 >> 172.21.15.43:0/3627379367 conn(0x5567aae75ba0 msgr2=0x5567aae75f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.057+0000 7fc3f31d9740 1 -- 172.21.15.43:0/3627379367 shutdown_connections 2024-04-21T08:39:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.057+0000 7fc3f31d9740 1 -- 172.21.15.43:0/3627379367 wait complete. 2024-04-21T08:39:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.057+0000 7fc3f31d9740 1 -- start start 2024-04-21T08:39:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.057+0000 7fc3f31d9740 1 Processor -- start 2024-04-21T08:39:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.058+0000 7fc3f31d9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae7b4d0 0x5567aae84270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.058+0000 7fc3f31d9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567aae7be80 0x5567aae81600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.058+0000 7fc3f31d9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567aae84a00 0x5567aae81b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.058+0000 7fc3f31d9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5567aae87970 con 0x5567aae7b4d0 2024-04-21T08:39:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.058+0000 7fc3f31d9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5567aad99500 con 0x5567aae84a00 2024-04-21T08:39:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.058+0000 7fc3ee38b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae7b4d0 0x5567aae84270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.058+0000 7fc3f31d9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5567aae87af0 con 0x5567aae7be80 2024-04-21T08:39:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.058+0000 7fc3ee38b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae7b4d0 0x5567aae84270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49108/0 (socket says 172.21.15.43:49108) 2024-04-21T08:39:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.058+0000 7fc3ee38b640 1 -- 172.21.15.43:0/126346590 learned_addr learned my addr 172.21.15.43:0/126346590 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.058+0000 7fc3edb8a640 1 --2- 172.21.15.43:0/126346590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567aae7be80 0x5567aae81600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.058+0000 7fc3f31d9740 10 client.0 _ll_register_callbacks cb 0x5567aad67b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.058+0000 7fc3eeb8c640 1 --2- 172.21.15.43:0/126346590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567aae84a00 0x5567aae81b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.058+0000 7fc3ee38b640 1 -- 172.21.15.43:0/126346590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567aae7be80 msgr2=0x5567aae81600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.058+0000 7fc3ee38b640 1 --2- 172.21.15.43:0/126346590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567aae7be80 0x5567aae81600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.058+0000 7fc3ee38b640 1 -- 172.21.15.43:0/126346590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567aae84a00 msgr2=0x5567aae81b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.058+0000 7fc3ee38b640 1 --2- 172.21.15.43:0/126346590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567aae84a00 0x5567aae81b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.058+0000 7fc3ee38b640 1 -- 172.21.15.43:0/126346590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567aaaf23f0 con 0x5567aae7b4d0 2024-04-21T08:39:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.058+0000 7fc3eeb8c640 1 --2- 172.21.15.43:0/126346590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567aae84a00 0x5567aae81b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.059+0000 7fc3ee38b640 1 --2- 172.21.15.43:0/126346590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae7b4d0 0x5567aae84270 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc3e403e040 tx=0x7fc3e4005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4f18c4888b0b2c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.059+0000 7fc3d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:32.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.059+0000 7fc3d1ffb640 1 -- 172.21.15.43:0/126346590 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e40bab00 con 0x5567aae7b4d0 2024-04-21T08:39:32.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.059+0000 7fc3d1ffb640 1 -- 172.21.15.43:0/126346590 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc3e40f0260 con 0x5567aae7b4d0 2024-04-21T08:39:32.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.059+0000 7fc3f31d9740 1 -- 172.21.15.43:0/126346590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3d8015070 con 0x5567aae7b4d0 2024-04-21T08:39:32.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.059+0000 7fc3d1ffb640 1 -- 172.21.15.43:0/126346590 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc3e40be1c0 con 0x5567aae7b4d0 2024-04-21T08:39:32.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.059+0000 7fc3f31d9740 10 client.16146 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:32.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.059+0000 7fc3f31d9740 1 -- 172.21.15.43:0/126346590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5567aabbc7d0 con 0x5567aae7b4d0 2024-04-21T08:39:32.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.059+0000 7fc3d1ffb640 1 -- 172.21.15.43:0/126346590 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3d8015070 con 0x5567aae7b4d0 2024-04-21T08:39:32.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.060+0000 7fc3d1ffb640 1 -- 172.21.15.43:0/126346590 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc3e40c7040 con 0x5567aae7b4d0 2024-04-21T08:39:32.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.060+0000 7fc3f31d9740 10 client.16146 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:32.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.060+0000 7fc3f31d9740 10 client.16146 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:39:32.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.060+0000 7fc3f31d9740 20 client.16146 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:32.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.060+0000 7fc3f31d9740 1 --2- 172.21.15.43:0/126346590 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5567aae94b40 0x5567aae97000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.060+0000 7fc3f31d9740 4 client.16146 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:32.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.060+0000 7fc3f31d9740 1 -- 172.21.15.43:0/126346590 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567aad7f930 con 0x5567aae94b40 2024-04-21T08:39:32.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.060+0000 7fc3edb8a640 1 --2- 172.21.15.43:0/126346590 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5567aae94b40 0x5567aae97000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.061+0000 7fc3edb8a640 1 --2- 172.21.15.43:0/126346590 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5567aae94b40 0x5567aae97000 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d05c17005a674ddc server_cookie=1089d38a4f175827 in_seq=0 out_seq=0 2024-04-21T08:39:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.061+0000 7fc3d1ffb640 10 client.16146 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:39:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.061+0000 7fc3d1ffb640 1 -- 172.21.15.43:0/126346590 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5567aad7f930 con 0x5567aae94b40 2024-04-21T08:39:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.061+0000 7fc3d1ffb640 10 client.16146 handle_command_reply: tid=0 2024-04-21T08:39:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.061+0000 7fc3f31d9740 1 client.16146 shutdown 2024-04-21T08:39:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.061+0000 7fc3f31d9740 1 -- 172.21.15.43:0/126346590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae7b4d0 msgr2=0x5567aae84270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.061+0000 7fc3f31d9740 1 --2- 172.21.15.43:0/126346590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae7b4d0 0x5567aae84270 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc3e403e040 tx=0x7fc3e4005c90 comp rx=0 tx=0).stop 2024-04-21T08:39:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.062+0000 7fc3f31d9740 1 -- 172.21.15.43:0/126346590 shutdown_connections 2024-04-21T08:39:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.062+0000 7fc3f31d9740 1 --2- 172.21.15.43:0/126346590 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5567aae94b40 0x5567aae97000 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.062+0000 7fc3f31d9740 1 --2- 172.21.15.43:0/126346590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567aae84a00 0x5567aae81b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.062+0000 7fc3d1ffb640 0 client.16146 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:39:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.062+0000 7fc3f31d9740 1 --2- 172.21.15.43:0/126346590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567aae7be80 0x5567aae81600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.062+0000 7fc3f31d9740 1 --2- 172.21.15.43:0/126346590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae7b4d0 0x5567aae84270 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.062+0000 7fc3f31d9740 1 -- 172.21.15.43:0/126346590 >> 172.21.15.43:0/126346590 conn(0x5567aae75ba0 msgr2=0x5567aae778f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:32.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.062+0000 7fc3f31d9740 1 -- 172.21.15.43:0/126346590 shutdown_connections 2024-04-21T08:39:32.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.062+0000 7fc3f31d9740 1 -- 172.21.15.43:0/126346590 wait complete. 2024-04-21T08:39:32.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.062+0000 7fc3f31d9740 20 client.16146 trim_cache size 0 max 16384 2024-04-21T08:39:32.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.076+0000 7fc3f31d9740 1 Processor -- start 2024-04-21T08:39:32.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.076+0000 7fc3f31d9740 1 -- start start 2024-04-21T08:39:32.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.076+0000 7fc3f31d9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567aae7b4d0 0x5567aae90830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.076+0000 7fc3f31d9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae7be80 0x5567aae83d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.076+0000 7fc3f31d9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567aae84a00 0x5567aae84270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.076+0000 7fc3f31d9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc00fff0 con 0x5567aae7be80 2024-04-21T08:39:32.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.076+0000 7fc3f31d9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc011b30 con 0x5567aae7b4d0 2024-04-21T08:39:32.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.076+0000 7fc3f31d9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc0106c0 con 0x5567aae84a00 2024-04-21T08:39:32.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.076+0000 7fc3edb8a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae7be80 0x5567aae83d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.076+0000 7fc3edb8a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae7be80 0x5567aae83d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49110/0 (socket says 172.21.15.43:49110) 2024-04-21T08:39:32.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.076+0000 7fc3edb8a640 1 -- 172.21.15.43:0/96091015 learned_addr learned my addr 172.21.15.43:0/96091015 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:32.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.076+0000 7fc3ee38b640 1 --2- 172.21.15.43:0/96091015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567aae7b4d0 0x5567aae90830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.076+0000 7fc3edb8a640 1 -- 172.21.15.43:0/96091015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567aae84a00 msgr2=0x5567aae84270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:32.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.076+0000 7fc3edb8a640 1 --2- 172.21.15.43:0/96091015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567aae84a00 0x5567aae84270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.076+0000 7fc3edb8a640 1 -- 172.21.15.43:0/96091015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567aae7b4d0 msgr2=0x5567aae90830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:32.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.076+0000 7fc3edb8a640 1 --2- 172.21.15.43:0/96091015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567aae7b4d0 0x5567aae90830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.076+0000 7fc3edb8a640 1 -- 172.21.15.43:0/96091015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567aae759d0 con 0x5567aae7be80 2024-04-21T08:39:32.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.076+0000 7fc3ee38b640 1 --2- 172.21.15.43:0/96091015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567aae7b4d0 0x5567aae90830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:32.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.077+0000 7fc3edb8a640 1 --2- 172.21.15.43:0/96091015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae7be80 0x5567aae83d30 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80048c0 tx=0x7fc3d80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368c3d7187fb928f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:32.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.077+0000 7fc3d3fff640 1 -- 172.21.15.43:0/96091015 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8030070 con 0x5567aae7be80 2024-04-21T08:39:32.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.077+0000 7fc3d3fff640 1 -- 172.21.15.43:0/96091015 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3d8015040 con 0x5567aae7be80 2024-04-21T08:39:32.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.077+0000 7fc3d3fff640 1 -- 172.21.15.43:0/96091015 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d802a800 con 0x5567aae7be80 2024-04-21T08:39:32.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.077+0000 7fc3f31d9740 1 -- 172.21.15.43:0/96091015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae7be80 msgr2=0x5567aae83d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:32.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.077+0000 7fc3f31d9740 1 --2- 172.21.15.43:0/96091015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae7be80 0x5567aae83d30 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80048c0 tx=0x7fc3d80045e0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.078+0000 7fc3f31d9740 1 -- 172.21.15.43:0/96091015 shutdown_connections 2024-04-21T08:39:32.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.078+0000 7fc3f31d9740 1 --2- 172.21.15.43:0/96091015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567aae84a00 0x5567aae84270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.078+0000 7fc3f31d9740 1 --2- 172.21.15.43:0/96091015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae7be80 0x5567aae83d30 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.078+0000 7fc3f31d9740 1 --2- 172.21.15.43:0/96091015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567aae7b4d0 0x5567aae90830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.078+0000 7fc3f31d9740 1 -- 172.21.15.43:0/96091015 >> 172.21.15.43:0/96091015 conn(0x5567aae75ba0 msgr2=0x5567aae74c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:32.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.078+0000 7fc3f31d9740 1 -- 172.21.15.43:0/96091015 shutdown_connections 2024-04-21T08:39:32.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.078+0000 7fc3f31d9740 1 -- 172.21.15.43:0/96091015 wait complete. 2024-04-21T08:39:32.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.078+0000 7fc3f31d9740 1 -- start start 2024-04-21T08:39:32.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.078+0000 7fc3f31d9740 1 Processor -- start 2024-04-21T08:39:32.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.078+0000 7fc3f31d9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae7b4d0 0x5567aae8d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.078+0000 7fc3f31d9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567aae7be80 0x5567aae885c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.079+0000 7fc3f31d9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567aae84a00 0x5567aae88bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.079+0000 7fc3f31d9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc011460 con 0x5567aae7b4d0 2024-04-21T08:39:32.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.079+0000 7fc3f31d9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc010d90 con 0x5567aae84a00 2024-04-21T08:39:32.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.079+0000 7fc3f31d9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc00dfd0 con 0x5567aae7be80 2024-04-21T08:39:32.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.079+0000 7fc3f31d9740 10 client.0 _ll_register_callbacks cb 0x7fc3e40f6510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:32.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.079+0000 7fc3ee38b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae7b4d0 0x5567aae8d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.079+0000 7fc3ee38b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae7b4d0 0x5567aae8d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49114/0 (socket says 172.21.15.43:49114) 2024-04-21T08:39:32.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.079+0000 7fc3ee38b640 1 -- 172.21.15.43:0/3373903315 learned_addr learned my addr 172.21.15.43:0/3373903315 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:32.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.079+0000 7fc3eeb8c640 1 --2- 172.21.15.43:0/3373903315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567aae84a00 0x5567aae88bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.079+0000 7fc3edb8a640 1 --2- 172.21.15.43:0/3373903315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567aae7be80 0x5567aae885c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.079+0000 7fc3ee38b640 1 -- 172.21.15.43:0/3373903315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567aae7be80 msgr2=0x5567aae885c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:32.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.079+0000 7fc3ee38b640 1 --2- 172.21.15.43:0/3373903315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567aae7be80 0x5567aae885c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.079+0000 7fc3ee38b640 1 -- 172.21.15.43:0/3373903315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567aae84a00 msgr2=0x5567aae88bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:32.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.079+0000 7fc3ee38b640 1 --2- 172.21.15.43:0/3373903315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567aae84a00 0x5567aae88bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.079+0000 7fc3ee38b640 1 -- 172.21.15.43:0/3373903315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567aae86df0 con 0x5567aae7b4d0 2024-04-21T08:39:32.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.079+0000 7fc3edb8a640 1 --2- 172.21.15.43:0/3373903315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567aae7be80 0x5567aae885c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:32.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.079+0000 7fc3ee38b640 1 --2- 172.21.15.43:0/3373903315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae7b4d0 0x5567aae8d200 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc3e40bb1e0 tx=0x7fc3e4005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d440c79c00e1f14d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:32.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.079+0000 7fc3d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:32.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.080+0000 7fc3d1ffb640 1 -- 172.21.15.43:0/3373903315 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e40be3a0 con 0x5567aae7b4d0 2024-04-21T08:39:32.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.080+0000 7fc3d1ffb640 1 -- 172.21.15.43:0/3373903315 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc3e40f03a0 con 0x5567aae7b4d0 2024-04-21T08:39:32.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.080+0000 7fc3f31d9740 1 -- 172.21.15.43:0/3373903315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3d8015040 con 0x5567aae7b4d0 2024-04-21T08:39:32.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.080+0000 7fc3d1ffb640 1 -- 172.21.15.43:0/3373903315 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc3e40be630 con 0x5567aae7b4d0 2024-04-21T08:39:32.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.080+0000 7fc3f31d9740 10 client.16152 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:32.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.080+0000 7fc3f31d9740 1 -- 172.21.15.43:0/3373903315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5567aae86df0 con 0x5567aae7b4d0 2024-04-21T08:39:32.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.080+0000 7fc3d1ffb640 1 -- 172.21.15.43:0/3373903315 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3d8015040 con 0x5567aae7b4d0 2024-04-21T08:39:32.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.080+0000 7fc3d1ffb640 1 -- 172.21.15.43:0/3373903315 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc3e40ba5b0 con 0x5567aae7b4d0 2024-04-21T08:39:32.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.080+0000 7fc3f31d9740 10 client.16152 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:32.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.080+0000 7fc3f31d9740 10 client.16152 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:39:32.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.080+0000 7fc3f31d9740 20 client.16152 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:32.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.080+0000 7fc3f31d9740 1 --2- 172.21.15.43:0/3373903315 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5567aaefb380 0x5567aaefd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.080+0000 7fc3f31d9740 4 client.16152 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:39:32.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.080+0000 7fc3f31d9740 1 -- 172.21.15.43:0/3373903315 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5567aadedc10 con 0x5567aaefb380 2024-04-21T08:39:32.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.081+0000 7fc3edb8a640 1 --2- 172.21.15.43:0/3373903315 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5567aaefb380 0x5567aaefd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.081+0000 7fc3edb8a640 1 --2- 172.21.15.43:0/3373903315 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5567aaefb380 0x5567aaefd770 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc39f84592d16b18 server_cookie=43d1ceda7780e6d3 in_seq=0 out_seq=0 2024-04-21T08:39:32.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.081+0000 7fc3d1ffb640 10 client.16152 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:39:32.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.081+0000 7fc3d1ffb640 1 -- 172.21.15.43:0/3373903315 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5567aadedc10 con 0x5567aaefb380 2024-04-21T08:39:32.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.081+0000 7fc3d1ffb640 10 client.16152 handle_command_reply: tid=0 2024-04-21T08:39:32.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.081+0000 7fc3f31d9740 1 client.16152 shutdown 2024-04-21T08:39:32.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.081+0000 7fc3f31d9740 1 -- 172.21.15.43:0/3373903315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae7b4d0 msgr2=0x5567aae8d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:32.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.081+0000 7fc3f31d9740 1 --2- 172.21.15.43:0/3373903315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae7b4d0 0x5567aae8d200 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc3e40bb1e0 tx=0x7fc3e4005e20 comp rx=0 tx=0).stop 2024-04-21T08:39:32.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.081+0000 7fc3f31d9740 1 -- 172.21.15.43:0/3373903315 shutdown_connections 2024-04-21T08:39:32.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.081+0000 7fc3f31d9740 1 --2- 172.21.15.43:0/3373903315 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5567aaefb380 0x5567aaefd770 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.081+0000 7fc3f31d9740 1 --2- 172.21.15.43:0/3373903315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567aae84a00 0x5567aae88bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.081+0000 7fc3f31d9740 1 --2- 172.21.15.43:0/3373903315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5567aae7be80 0x5567aae885c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.081+0000 7fc3f31d9740 1 --2- 172.21.15.43:0/3373903315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567aae7b4d0 0x5567aae8d200 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.081+0000 7fc3f31d9740 1 -- 172.21.15.43:0/3373903315 >> 172.21.15.43:0/3373903315 conn(0x5567aae75ba0 msgr2=0x5567aae74370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:32.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.081+0000 7fc3d1ffb640 0 client.16152 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:39:32.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.082+0000 7fc3f31d9740 1 -- 172.21.15.43:0/3373903315 shutdown_connections 2024-04-21T08:39:32.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.082+0000 7fc3f31d9740 1 -- 172.21.15.43:0/3373903315 wait complete. 2024-04-21T08:39:32.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.082+0000 7fc3f31d9740 20 client.16152 trim_cache size 0 max 16384 2024-04-21T08:39:32.087 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:39:32.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.082+0000 7fc3f05ef640 1 -- 172.21.15.43:0/3734685309 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc3d40a3e30 msgr2=0x7fc3d40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:32.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.082+0000 7fc3f05ef640 1 --2- 172.21.15.43:0/3734685309 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc3d40a3e30 0x7fc3d40a62f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0005fd0 tx=0x7fc3e0009760 comp rx=0 tx=0).stop 2024-04-21T08:39:32.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.082+0000 7fc3f05ef640 1 -- 172.21.15.43:0/3734685309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3e81031d0 msgr2=0x7fc3e8100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:32.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.082+0000 7fc3f05ef640 1 --2- 172.21.15.43:0/3734685309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3e81031d0 0x7fc3e8100e20 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4000c00 tx=0x7fc3e402d920 comp rx=0 tx=0).stop 2024-04-21T08:39:32.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.082+0000 7fc3f05ef640 1 -- 172.21.15.43:0/3734685309 shutdown_connections 2024-04-21T08:39:32.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.082+0000 7fc3f05ef640 1 --2- 172.21.15.43:0/3734685309 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc3d40a3e30 0x7fc3d40a62f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.082+0000 7fc3f05ef640 1 --2- 172.21.15.43:0/3734685309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3e810c510 0x7fc3e81018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.082+0000 7fc3f05ef640 1 --2- 172.21.15.43:0/3734685309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e8103af0 0x7fc3e8101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.082+0000 7fc3f05ef640 1 --2- 172.21.15.43:0/3734685309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3e81031d0 0x7fc3e8100e20 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.082+0000 7fc3f05ef640 1 -- 172.21.15.43:0/3734685309 >> 172.21.15.43:0/3734685309 conn(0x7fc3e8077f50 msgr2=0x7fc3e80ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:32.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.082+0000 7fc3f05ef640 1 -- 172.21.15.43:0/3734685309 shutdown_connections 2024-04-21T08:39:32.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.082+0000 7fc3f05ef640 1 -- 172.21.15.43:0/3734685309 wait complete. 2024-04-21T08:39:32.097 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:39:32.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.202+0000 7f0063f60640 1 -- 172.21.15.43:0/827020153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005c103af0 msgr2=0x7f005c10bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:32.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.202+0000 7f0063f60640 1 --2- 172.21.15.43:0/827020153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005c103af0 0x7f005c10bfd0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f004c009900 tx=0x7f004c02bd60 comp rx=0 tx=0).stop 2024-04-21T08:39:32.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.202+0000 7f0063f60640 1 -- 172.21.15.43:0/827020153 shutdown_connections 2024-04-21T08:39:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.202+0000 7f0063f60640 1 --2- 172.21.15.43:0/827020153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f005c10c510 0x7f005c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.202+0000 7f0063f60640 1 --2- 172.21.15.43:0/827020153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005c103af0 0x7f005c10bfd0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.202+0000 7f0063f60640 1 --2- 172.21.15.43:0/827020153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f005c1031d0 0x7f005c1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.202+0000 7f0063f60640 1 -- 172.21.15.43:0/827020153 >> 172.21.15.43:0/827020153 conn(0x7f005c077f50 msgr2=0x7f005c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.202+0000 7f0063f60640 1 -- 172.21.15.43:0/827020153 shutdown_connections 2024-04-21T08:39:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.203+0000 7f0063f60640 1 -- 172.21.15.43:0/827020153 wait complete. 2024-04-21T08:39:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.203+0000 7f0063f60640 1 Processor -- start 2024-04-21T08:39:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.203+0000 7f0063f60640 1 -- start start 2024-04-21T08:39:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.203+0000 7f0063f60640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005c1031d0 0x7f005c1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.203+0000 7f0063f60640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f005c103af0 0x7f005c1a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.203+0000 7f0063f60640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f005c10c510 0x7f005c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.203+0000 7f0063f60640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f005c1121a0 con 0x7f005c1031d0 2024-04-21T08:39:32.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.203+0000 7f0063f60640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f005c112020 con 0x7f005c10c510 2024-04-21T08:39:32.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.203+0000 7f0063f60640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f005c112320 con 0x7f005c103af0 2024-04-21T08:39:32.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.203+0000 7f0061cfc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005c1031d0 0x7f005c1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.203+0000 7f0061cfc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005c1031d0 0x7f005c1a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49138/0 (socket says 172.21.15.43:49138) 2024-04-21T08:39:32.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.203+0000 7f0061cfc640 1 -- 172.21.15.43:0/2939696340 learned_addr learned my addr 172.21.15.43:0/2939696340 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:32.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.204+0000 7f00624fd640 1 --2- 172.21.15.43:0/2939696340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f005c10c510 0x7f005c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.204+0000 7f0061cfc640 1 -- 172.21.15.43:0/2939696340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f005c103af0 msgr2=0x7f005c1a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:32.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.204+0000 7f0061cfc640 1 --2- 172.21.15.43:0/2939696340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f005c103af0 0x7f005c1a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.204+0000 7f0061cfc640 1 -- 172.21.15.43:0/2939696340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f005c10c510 msgr2=0x7f005c19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:32.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.204+0000 7f0061cfc640 1 --2- 172.21.15.43:0/2939696340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f005c10c510 0x7f005c19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.204+0000 7f0061cfc640 1 -- 172.21.15.43:0/2939696340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005c19af50 con 0x7f005c1031d0 2024-04-21T08:39:32.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.204+0000 7f00624fd640 1 --2- 172.21.15.43:0/2939696340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f005c10c510 0x7f005c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:32.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.204+0000 7f0061cfc640 1 --2- 172.21.15.43:0/2939696340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005c1031d0 0x7f005c1a0610 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f005800d950 tx=0x7f005800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:32.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.204+0000 7f0052ffd640 1 -- 172.21.15.43:0/2939696340 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058017030 con 0x7f005c1031d0 2024-04-21T08:39:32.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.204+0000 7f0063f60640 1 -- 172.21.15.43:0/2939696340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005c19b240 con 0x7f005c1031d0 2024-04-21T08:39:32.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.204+0000 7f0063f60640 1 -- 172.21.15.43:0/2939696340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f005c1a7460 con 0x7f005c1031d0 2024-04-21T08:39:32.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.204+0000 7f0052ffd640 1 -- 172.21.15.43:0/2939696340 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0058004020 con 0x7f005c1031d0 2024-04-21T08:39:32.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.204+0000 7f0052ffd640 1 -- 172.21.15.43:0/2939696340 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00580047a0 con 0x7f005c1031d0 2024-04-21T08:39:32.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.207+0000 7f0066c32740 1 Processor -- start 2024-04-21T08:39:32.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.207+0000 7f0066c32740 1 -- start start 2024-04-21T08:39:32.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.207+0000 7f0066c32740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec89dcf4d0 0x55ec89dcf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.207+0000 7f0066c32740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec89dcfe80 0x55ec89dd8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.207+0000 7f0066c32740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec89dd8a00 0x55ec89ddae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.207+0000 7f0066c32740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ec89d341f0 con 0x55ec89dcfe80 2024-04-21T08:39:32.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.207+0000 7f0066c32740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ec89cd14b0 con 0x55ec89dcf4d0 2024-04-21T08:39:32.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.207+0000 7f0066c32740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ec89d301b0 con 0x55ec89dd8a00 2024-04-21T08:39:32.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.208+0000 7f00614fb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec89dd8a00 0x55ec89ddae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.208+0000 7f00614fb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec89dd8a00 0x55ec89ddae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.43:44378/0 (socket says 172.21.15.43:44378) 2024-04-21T08:39:32.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.208+0000 7f00614fb640 1 -- 172.21.15.43:0/4129924973 learned_addr learned my addr 172.21.15.43:0/4129924973 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:32.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.208+0000 7f00624fd640 1 --2- 172.21.15.43:0/4129924973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec89dcf4d0 0x55ec89dcf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.208+0000 7f00614fb640 1 -- 172.21.15.43:0/4129924973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec89dcf4d0 msgr2=0x55ec89dcf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:32.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.208+0000 7f00614fb640 1 --2- 172.21.15.43:0/4129924973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec89dcf4d0 0x55ec89dcf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.208+0000 7f00614fb640 1 -- 172.21.15.43:0/4129924973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec89dcfe80 msgr2=0x55ec89dd8430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T08:39:32.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.208+0000 7f00614fb640 1 --2- 172.21.15.43:0/4129924973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec89dcfe80 0x55ec89dd8430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.208+0000 7f00614fb640 1 -- 172.21.15.43:0/4129924973 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec89cc31b0 con 0x55ec89dd8a00 2024-04-21T08:39:32.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.208+0000 7f00624fd640 1 --2- 172.21.15.43:0/4129924973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec89dcf4d0 0x55ec89dcf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:32.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.208+0000 7f00614fb640 1 --2- 172.21.15.43:0/4129924973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec89dd8a00 0x55ec89ddae50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f004c02c2a0 tx=0x7f004c0095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfcb7c4b316fb450 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:32.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.209+0000 7f0047fff640 1 -- 172.21.15.43:0/4129924973 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c041020 con 0x55ec89dd8a00 2024-04-21T08:39:32.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.209+0000 7f0047fff640 1 -- 172.21.15.43:0/4129924973 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f004c02cdc0 con 0x55ec89dd8a00 2024-04-21T08:39:32.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.209+0000 7f0047fff640 1 -- 172.21.15.43:0/4129924973 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c005490 con 0x55ec89dd8a00 2024-04-21T08:39:32.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.209+0000 7f0052ffd640 1 -- 172.21.15.43:0/2939696340 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f0058009a30 con 0x7f005c1031d0 2024-04-21T08:39:32.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.210+0000 7f0066c32740 1 -- 172.21.15.43:0/4129924973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec89dd8a00 msgr2=0x55ec89ddae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:32.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.210+0000 7f0066c32740 1 --2- 172.21.15.43:0/4129924973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec89dd8a00 0x55ec89ddae50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f004c02c2a0 tx=0x7f004c0095e0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.210+0000 7f0066c32740 1 -- 172.21.15.43:0/4129924973 shutdown_connections 2024-04-21T08:39:32.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.210+0000 7f0066c32740 1 --2- 172.21.15.43:0/4129924973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec89dd8a00 0x55ec89ddae50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.210+0000 7f0066c32740 1 --2- 172.21.15.43:0/4129924973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec89dcfe80 0x55ec89dd8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.210+0000 7f0066c32740 1 --2- 172.21.15.43:0/4129924973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec89dcf4d0 0x55ec89dcf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.210+0000 7f0066c32740 1 -- 172.21.15.43:0/4129924973 >> 172.21.15.43:0/4129924973 conn(0x55ec89dc9ba0 msgr2=0x55ec89dc9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:32.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.210+0000 7f0066c32740 1 -- 172.21.15.43:0/4129924973 shutdown_connections 2024-04-21T08:39:32.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.210+0000 7f0052ffd640 1 --2- 172.21.15.43:0/2939696340 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f00480a3d40 0x7f00480a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.210+0000 7f0066c32740 1 -- 172.21.15.43:0/4129924973 wait complete. 2024-04-21T08:39:32.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.210+0000 7f0052ffd640 1 -- 172.21.15.43:0/2939696340 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f00580d0850 con 0x7f005c1031d0 2024-04-21T08:39:32.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.210+0000 7f00614fb640 1 --2- 172.21.15.43:0/2939696340 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f00480a3d40 0x7f00480a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.210+0000 7f0066c32740 1 -- start start 2024-04-21T08:39:32.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.211+0000 7f0066c32740 1 Processor -- start 2024-04-21T08:39:32.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.211+0000 7f0066c32740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec89dcf4d0 0x55ec89dd8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.211+0000 7f0066c32740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec89dcfe80 0x55ec89dd55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.211+0000 7f0066c32740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec89dd8a00 0x55ec89dd5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.211+0000 7f0066c32740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ec89ddb910 con 0x55ec89dcfe80 2024-04-21T08:39:32.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.211+0000 7f0066c32740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ec89ced500 con 0x55ec89dd8a00 2024-04-21T08:39:32.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.211+0000 7f00614fb640 1 --2- 172.21.15.43:0/2939696340 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f00480a3d40 0x7f00480a6200 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f004c02c2a0 tx=0x7f004c005fd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:32.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.211+0000 7f0066c32740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ec89ddba90 con 0x55ec89dcf4d0 2024-04-21T08:39:32.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.211+0000 7f0066c32740 10 client.0 _ll_register_callbacks cb 0x55ec89cbbb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:32.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.211+0000 7f00624fd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec89dd8a00 0x55ec89dd5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.211+0000 7f00614fb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec89dcfe80 0x55ec89dd55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.211+0000 7f00624fd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec89dd8a00 0x55ec89dd5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:40022/0 (socket says 172.21.15.43:40022) 2024-04-21T08:39:32.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.211+0000 7f00624fd640 1 -- 172.21.15.43:0/663412851 learned_addr learned my addr 172.21.15.43:0/663412851 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:32.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.211+0000 7f0061cfc640 1 --2- 172.21.15.43:0/663412851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec89dcf4d0 0x55ec89dd8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.212+0000 7f00624fd640 1 -- 172.21.15.43:0/663412851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec89dcf4d0 msgr2=0x55ec89dd8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:32.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.212+0000 7f00624fd640 1 --2- 172.21.15.43:0/663412851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec89dcf4d0 0x55ec89dd8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.212+0000 7f00624fd640 1 -- 172.21.15.43:0/663412851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec89dcfe80 msgr2=0x55ec89dd55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.212+0000 7f00624fd640 1 --2- 172.21.15.43:0/663412851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec89dcfe80 0x55ec89dd55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.212+0000 7f00624fd640 1 -- 172.21.15.43:0/663412851 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec89a463f0 con 0x55ec89dd8a00 2024-04-21T08:39:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.212+0000 7f00614fb640 1 --2- 172.21.15.43:0/663412851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec89dcfe80 0x55ec89dd55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.212+0000 7f00624fd640 1 --2- 172.21.15.43:0/663412851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec89dd8a00 0x55ec89dd5b30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0054011d10 tx=0x7f00540107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e27850a7081351e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.212+0000 7f00457fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-21T08:39:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.212+0000 7f00457fa640 1 -- 172.21.15.43:0/663412851 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005401f060 con 0x55ec89dd8a00 2024-04-21T08:39:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.212+0000 7f0066c32740 1 -- 172.21.15.43:0/663412851 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f004c02cdc0 con 0x55ec89dd8a00 2024-04-21T08:39:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.212+0000 7f00457fa640 1 -- 172.21.15.43:0/663412851 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f0054017070 con 0x55ec89dd8a00 2024-04-21T08:39:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.212+0000 7f00457fa640 1 -- 172.21.15.43:0/663412851 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0054026050 con 0x55ec89dd8a00 2024-04-21T08:39:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.212+0000 7f0066c32740 10 client.25661 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.212+0000 7f0066c32740 1 -- 172.21.15.43:0/663412851 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ec89b107d0 con 0x55ec89dd8a00 2024-04-21T08:39:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.213+0000 7f00457fa640 1 -- 172.21.15.43:0/663412851 <== mon.2 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f004c02cdc0 con 0x55ec89dd8a00 2024-04-21T08:39:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.213+0000 7f00457fa640 1 -- 172.21.15.43:0/663412851 <== mon.2 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f005400e450 con 0x55ec89dd8a00 2024-04-21T08:39:32.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.213+0000 7f0066c32740 10 client.25661 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:32.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.213+0000 7f0066c32740 10 client.25661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:32.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.213+0000 7f0066c32740 20 client.25661 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:32.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.213+0000 7f0066c32740 1 --2- 172.21.15.43:0/663412851 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ec89de8b60 0x55ec89deaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.213+0000 7f0066c32740 4 client.25661 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:32.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.213+0000 7f0066c32740 1 -- 172.21.15.43:0/663412851 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec89cd3930 con 0x55ec89de8b60 2024-04-21T08:39:32.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.213+0000 7f0061cfc640 1 --2- 172.21.15.43:0/663412851 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ec89de8b60 0x55ec89deaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.214+0000 7f00457fa640 10 client.25661 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:32.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.214+0000 7f0061cfc640 1 --2- 172.21.15.43:0/663412851 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ec89de8b60 0x55ec89deaf80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87fcd8cc32096600 server_cookie=4cf7f51fa21d4f27 in_seq=0 out_seq=0 2024-04-21T08:39:32.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.214+0000 7f00457fa640 1 -- 172.21.15.43:0/663412851 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ec89cd3930 con 0x55ec89de8b60 2024-04-21T08:39:32.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.214+0000 7f00457fa640 10 client.25661 handle_command_reply: tid=0 2024-04-21T08:39:32.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.214+0000 7f0066c32740 1 client.25661 shutdown 2024-04-21T08:39:32.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.214+0000 7f0066c32740 1 -- 172.21.15.43:0/663412851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec89dd8a00 msgr2=0x55ec89dd5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:32.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.214+0000 7f0066c32740 1 --2- 172.21.15.43:0/663412851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec89dd8a00 0x55ec89dd5b30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0054011d10 tx=0x7f00540107a0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.215+0000 7f0066c32740 1 -- 172.21.15.43:0/663412851 shutdown_connections 2024-04-21T08:39:32.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.215+0000 7f0066c32740 1 --2- 172.21.15.43:0/663412851 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ec89de8b60 0x55ec89deaf80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.215+0000 7f0066c32740 1 --2- 172.21.15.43:0/663412851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec89dd8a00 0x55ec89dd5b30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.215+0000 7f0066c32740 1 --2- 172.21.15.43:0/663412851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec89dcfe80 0x55ec89dd55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.215+0000 7f0066c32740 1 --2- 172.21.15.43:0/663412851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec89dcf4d0 0x55ec89dd8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.215+0000 7f00457fa640 0 client.25661 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:32.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.215+0000 7f0066c32740 1 -- 172.21.15.43:0/663412851 >> 172.21.15.43:0/663412851 conn(0x55ec89dc9ba0 msgr2=0x55ec89dcb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:32.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.215+0000 7f0066c32740 1 -- 172.21.15.43:0/663412851 shutdown_connections 2024-04-21T08:39:32.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.215+0000 7f0066c32740 1 -- 172.21.15.43:0/663412851 wait complete. 2024-04-21T08:39:32.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.215+0000 7f0066c32740 20 client.25661 trim_cache size 0 max 16384 2024-04-21T08:39:32.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.229+0000 7f0066c32740 1 Processor -- start 2024-04-21T08:39:32.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.229+0000 7f0066c32740 1 -- start start 2024-04-21T08:39:32.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.229+0000 7f0066c32740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec89dcf4d0 0x55ec89de4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.229+0000 7f0066c32740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec89dcfe80 0x55ec89dd7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.229+0000 7f0066c32740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec89dd8a00 0x55ec89dd8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.229+0000 7f0066c32740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f005c1b1700 con 0x55ec89dd8a00 2024-04-21T08:39:32.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.229+0000 7f0066c32740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f005c1b3240 con 0x55ec89dcf4d0 2024-04-21T08:39:32.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.229+0000 7f0066c32740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f005c1b1dd0 con 0x55ec89dcfe80 2024-04-21T08:39:32.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.229+0000 7f00624fd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec89dd8a00 0x55ec89dd8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.229+0000 7f0061cfc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec89dcf4d0 0x55ec89de4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.229+0000 7f00624fd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec89dd8a00 0x55ec89dd8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49144/0 (socket says 172.21.15.43:49144) 2024-04-21T08:39:32.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.229+0000 7f00624fd640 1 -- 172.21.15.43:0/1062035120 learned_addr learned my addr 172.21.15.43:0/1062035120 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:32.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.229+0000 7f00614fb640 1 --2- 172.21.15.43:0/1062035120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec89dcfe80 0x55ec89dd7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.230+0000 7f00624fd640 1 -- 172.21.15.43:0/1062035120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec89dcfe80 msgr2=0x55ec89dd7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:32.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.230+0000 7f00624fd640 1 --2- 172.21.15.43:0/1062035120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec89dcfe80 0x55ec89dd7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.230+0000 7f00624fd640 1 -- 172.21.15.43:0/1062035120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec89dcf4d0 msgr2=0x55ec89de4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:32.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.230+0000 7f00624fd640 1 --2- 172.21.15.43:0/1062035120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec89dcf4d0 0x55ec89de4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.230+0000 7f00624fd640 1 -- 172.21.15.43:0/1062035120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec89de7760 con 0x55ec89dd8a00 2024-04-21T08:39:32.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.230+0000 7f0061cfc640 1 --2- 172.21.15.43:0/1062035120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec89dcf4d0 0x55ec89de4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:32.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.230+0000 7f00624fd640 1 --2- 172.21.15.43:0/1062035120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec89dd8a00 0x55ec89dd8260 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f005400ea10 tx=0x7f0054009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f079116b76ec512 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:32.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.230+0000 7f00477fe640 1 -- 172.21.15.43:0/1062035120 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005402f070 con 0x55ec89dd8a00 2024-04-21T08:39:32.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.230+0000 7f00477fe640 1 -- 172.21.15.43:0/1062035120 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f005400ea90 con 0x55ec89dd8a00 2024-04-21T08:39:32.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.230+0000 7f00477fe640 1 -- 172.21.15.43:0/1062035120 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054005040 con 0x55ec89dd8a00 2024-04-21T08:39:32.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.231+0000 7f0066c32740 1 -- 172.21.15.43:0/1062035120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec89dd8a00 msgr2=0x55ec89dd8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:32.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.231+0000 7f0066c32740 1 --2- 172.21.15.43:0/1062035120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec89dd8a00 0x55ec89dd8260 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f005400ea10 tx=0x7f0054009f50 comp rx=0 tx=0).stop 2024-04-21T08:39:32.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.231+0000 7f0066c32740 1 -- 172.21.15.43:0/1062035120 shutdown_connections 2024-04-21T08:39:32.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.231+0000 7f0066c32740 1 --2- 172.21.15.43:0/1062035120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec89dd8a00 0x55ec89dd8260 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.231+0000 7f0066c32740 1 --2- 172.21.15.43:0/1062035120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec89dcfe80 0x55ec89dd7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.231+0000 7f0066c32740 1 --2- 172.21.15.43:0/1062035120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec89dcf4d0 0x55ec89de4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.231+0000 7f0066c32740 1 -- 172.21.15.43:0/1062035120 >> 172.21.15.43:0/1062035120 conn(0x55ec89dc9ba0 msgr2=0x55ec89dc8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:32.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.231+0000 7f0066c32740 1 -- 172.21.15.43:0/1062035120 shutdown_connections 2024-04-21T08:39:32.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.231+0000 7f0066c32740 1 -- 172.21.15.43:0/1062035120 wait complete. 2024-04-21T08:39:32.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.231+0000 7f0066c32740 1 -- start start 2024-04-21T08:39:32.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.231+0000 7f0066c32740 1 Processor -- start 2024-04-21T08:39:32.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.232+0000 7f0066c32740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec89dcf4d0 0x55ec89de0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.232+0000 7f0066c32740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec89dcfe80 0x55ec89de1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.232+0000 7f0066c32740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec89dd8a00 0x55ec89dddce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.232+0000 7f0066c32740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f005c1b2b70 con 0x55ec89dcf4d0 2024-04-21T08:39:32.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.232+0000 7f0066c32740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f005c1b24a0 con 0x55ec89dcfe80 2024-04-21T08:39:32.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.232+0000 7f0066c32740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f005c1af6e0 con 0x55ec89dd8a00 2024-04-21T08:39:32.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.232+0000 7f0061cfc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec89dcf4d0 0x55ec89de0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.232+0000 7f0061cfc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec89dcf4d0 0x55ec89de0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49160/0 (socket says 172.21.15.43:49160) 2024-04-21T08:39:32.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.232+0000 7f0061cfc640 1 -- 172.21.15.43:0/3738347102 learned_addr learned my addr 172.21.15.43:0/3738347102 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:32.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.232+0000 7f0066c32740 10 client.0 _ll_register_callbacks cb 0x7f0054011920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:32.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.232+0000 7f00614fb640 1 --2- 172.21.15.43:0/3738347102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec89dcfe80 0x55ec89de1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.232+0000 7f00624fd640 1 --2- 172.21.15.43:0/3738347102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec89dd8a00 0x55ec89dddce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.232+0000 7f0061cfc640 1 -- 172.21.15.43:0/3738347102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec89dd8a00 msgr2=0x55ec89dddce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:32.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.232+0000 7f0061cfc640 1 --2- 172.21.15.43:0/3738347102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec89dd8a00 0x55ec89dddce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.232+0000 7f0061cfc640 1 -- 172.21.15.43:0/3738347102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec89dcfe80 msgr2=0x55ec89de1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:32.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.232+0000 7f0061cfc640 1 --2- 172.21.15.43:0/3738347102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec89dcfe80 0x55ec89de1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.232+0000 7f0061cfc640 1 -- 172.21.15.43:0/3738347102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec89ddadf0 con 0x55ec89dcf4d0 2024-04-21T08:39:32.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.233+0000 7f00457fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:32.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.233+0000 7f0061cfc640 1 --2- 172.21.15.43:0/3738347102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec89dcf4d0 0x55ec89de0d10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0058098710 tx=0x7f005809ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec57bbbfb673086b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:32.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.233+0000 7f00457fa640 1 -- 172.21.15.43:0/3738347102 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00580b4020 con 0x55ec89dcf4d0 2024-04-21T08:39:32.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.233+0000 7f0066c32740 1 -- 172.21.15.43:0/3738347102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f005400ea90 con 0x55ec89dcf4d0 2024-04-21T08:39:32.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.233+0000 7f00457fa640 1 -- 172.21.15.43:0/3738347102 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f00580041d0 con 0x55ec89dcf4d0 2024-04-21T08:39:32.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.233+0000 7f00457fa640 1 -- 172.21.15.43:0/3738347102 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f005807d020 con 0x55ec89dcf4d0 2024-04-21T08:39:32.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.233+0000 7f0066c32740 10 client.16167 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:32.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.233+0000 7f0066c32740 1 -- 172.21.15.43:0/3738347102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ec89e58080 con 0x55ec89dcf4d0 2024-04-21T08:39:32.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.234+0000 7f00457fa640 1 -- 172.21.15.43:0/3738347102 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ec89cd3930 con 0x55ec89dcf4d0 2024-04-21T08:39:32.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.234+0000 7f00457fa640 1 -- 172.21.15.43:0/3738347102 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f00580cddc0 con 0x55ec89dcf4d0 2024-04-21T08:39:32.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.234+0000 7f0066c32740 10 client.16167 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:32.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.234+0000 7f0066c32740 10 client.16167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:32.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.234+0000 7f0066c32740 20 client.16167 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:32.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.234+0000 7f0066c32740 1 --2- 172.21.15.43:0/3738347102 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ec89e587f0 0x55ec89e5ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:32.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.234+0000 7f0066c32740 4 client.16167 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:39:32.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.234+0000 7f0066c32740 1 -- 172.21.15.43:0/3738347102 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f005400ea90 con 0x55ec89e587f0 2024-04-21T08:39:32.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.234+0000 7f00614fb640 1 --2- 172.21.15.43:0/3738347102 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ec89e587f0 0x55ec89e5ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:32.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.235+0000 7f00457fa640 10 client.16167 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:32.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.235+0000 7f00614fb640 1 --2- 172.21.15.43:0/3738347102 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ec89e587f0 0x55ec89e5ac10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=131e3c8dffbb455 server_cookie=dbfc35a826602e59 in_seq=0 out_seq=0 2024-04-21T08:39:32.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.235+0000 7f00457fa640 1 -- 172.21.15.43:0/3738347102 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f005400ea90 con 0x55ec89e587f0 2024-04-21T08:39:32.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.235+0000 7f00457fa640 10 client.16167 handle_command_reply: tid=0 2024-04-21T08:39:32.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.235+0000 7f0066c32740 1 client.16167 shutdown 2024-04-21T08:39:32.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.235+0000 7f0066c32740 1 -- 172.21.15.43:0/3738347102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec89dcf4d0 msgr2=0x55ec89de0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:32.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.235+0000 7f0066c32740 1 --2- 172.21.15.43:0/3738347102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec89dcf4d0 0x55ec89de0d10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0058098710 tx=0x7f005809ca70 comp rx=0 tx=0).stop 2024-04-21T08:39:32.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.235+0000 7f0066c32740 1 -- 172.21.15.43:0/3738347102 shutdown_connections 2024-04-21T08:39:32.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.235+0000 7f0066c32740 1 --2- 172.21.15.43:0/3738347102 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ec89e587f0 0x55ec89e5ac10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.235+0000 7f0066c32740 1 --2- 172.21.15.43:0/3738347102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec89dd8a00 0x55ec89dddce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.235+0000 7f00457fa640 0 client.16167 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:32.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.235+0000 7f0066c32740 1 --2- 172.21.15.43:0/3738347102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec89dcfe80 0x55ec89de1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.235+0000 7f0066c32740 1 --2- 172.21.15.43:0/3738347102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec89dcf4d0 0x55ec89de0d10 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.235+0000 7f0066c32740 1 -- 172.21.15.43:0/3738347102 >> 172.21.15.43:0/3738347102 conn(0x55ec89dc9ba0 msgr2=0x55ec89dc8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:32.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.236+0000 7f0066c32740 1 -- 172.21.15.43:0/3738347102 shutdown_connections 2024-04-21T08:39:32.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.236+0000 7f0066c32740 1 -- 172.21.15.43:0/3738347102 wait complete. 2024-04-21T08:39:32.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.236+0000 7f0066c32740 20 client.16167 trim_cache size 0 max 16384 2024-04-21T08:39:32.242 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:39:32.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.236+0000 7f0063f60640 1 -- 172.21.15.43:0/2939696340 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f00480a3d40 msgr2=0x7f00480a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:32.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.236+0000 7f0063f60640 1 --2- 172.21.15.43:0/2939696340 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f00480a3d40 0x7f00480a6200 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f004c02c2a0 tx=0x7f004c005fd0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.236+0000 7f0063f60640 1 -- 172.21.15.43:0/2939696340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005c1031d0 msgr2=0x7f005c1a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:32.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.236+0000 7f0063f60640 1 --2- 172.21.15.43:0/2939696340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005c1031d0 0x7f005c1a0610 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f005800d950 tx=0x7f005800de10 comp rx=0 tx=0).stop 2024-04-21T08:39:32.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.236+0000 7f0063f60640 1 -- 172.21.15.43:0/2939696340 shutdown_connections 2024-04-21T08:39:32.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.236+0000 7f0063f60640 1 --2- 172.21.15.43:0/2939696340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f005c10c510 0x7f005c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.236+0000 7f0063f60640 1 --2- 172.21.15.43:0/2939696340 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f00480a3d40 0x7f00480a6200 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.236+0000 7f0063f60640 1 --2- 172.21.15.43:0/2939696340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f005c103af0 0x7f005c1a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.236+0000 7f0063f60640 1 --2- 172.21.15.43:0/2939696340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005c1031d0 0x7f005c1a0610 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:32.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.236+0000 7f0063f60640 1 -- 172.21.15.43:0/2939696340 >> 172.21.15.43:0/2939696340 conn(0x7f005c077f50 msgr2=0x7f005c108480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:32.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.236+0000 7f0063f60640 1 -- 172.21.15.43:0/2939696340 shutdown_connections 2024-04-21T08:39:32.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:32.236+0000 7f0063f60640 1 -- 172.21.15.43:0/2939696340 wait complete. 2024-04-21T08:39:32.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:32 smithi112 ceph-mon[31085]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-21T08:39:32.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:32 smithi043 ceph-mon[27978]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-21T08:39:32.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:32 smithi175 ceph-mon[31920]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-21T08:39:33.248 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:39:33.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:39:33.248 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:39:33.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.358+0000 7fc3c4975640 1 -- 172.21.15.43:0/3386559098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3c0103af0 msgr2=0x7fc3c010bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.358+0000 7fc3c4975640 1 --2- 172.21.15.43:0/3386559098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3c0103af0 0x7fc3c010bfd0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4009900 tx=0x7fc3b402bd60 comp rx=0 tx=0).stop 2024-04-21T08:39:33.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.358+0000 7fc3c4975640 1 -- 172.21.15.43:0/3386559098 shutdown_connections 2024-04-21T08:39:33.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.358+0000 7fc3c4975640 1 --2- 172.21.15.43:0/3386559098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3c010c510 0x7fc3c010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.358+0000 7fc3c4975640 1 --2- 172.21.15.43:0/3386559098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3c0103af0 0x7fc3c010bfd0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.358+0000 7fc3c4975640 1 --2- 172.21.15.43:0/3386559098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3c01031d0 0x7fc3c01035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.358+0000 7fc3c4975640 1 -- 172.21.15.43:0/3386559098 >> 172.21.15.43:0/3386559098 conn(0x7fc3c0077f50 msgr2=0x7fc3c0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:33.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.358+0000 7fc3c4975640 1 -- 172.21.15.43:0/3386559098 shutdown_connections 2024-04-21T08:39:33.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.358+0000 7fc3c4975640 1 -- 172.21.15.43:0/3386559098 wait complete. 2024-04-21T08:39:33.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.359+0000 7fc3c4975640 1 Processor -- start 2024-04-21T08:39:33.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.359+0000 7fc3c4975640 1 -- start start 2024-04-21T08:39:33.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.360+0000 7fc3c4975640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3c01031d0 0x7fc3c0198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.360+0000 7fc3c4975640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3c0103af0 0x7fc3c0198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.360+0000 7fc3c4975640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3c010c510 0x7fc3c019d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.360+0000 7fc3c4975640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c01121d0 con 0x7fc3c01031d0 2024-04-21T08:39:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.360+0000 7fc3c4975640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c0112050 con 0x7fc3c010c510 2024-04-21T08:39:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.360+0000 7fc3be59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3c01031d0 0x7fc3c0198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.360+0000 7fc3c4975640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c0112350 con 0x7fc3c0103af0 2024-04-21T08:39:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.360+0000 7fc3be59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3c01031d0 0x7fc3c0198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49178/0 (socket says 172.21.15.43:49178) 2024-04-21T08:39:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.360+0000 7fc3be59c640 1 -- 172.21.15.43:0/139914066 learned_addr learned my addr 172.21.15.43:0/139914066 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.360+0000 7fc3bdd9b640 1 --2- 172.21.15.43:0/139914066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3c0103af0 0x7fc3c0198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.360+0000 7fc3bed9d640 1 --2- 172.21.15.43:0/139914066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3c010c510 0x7fc3c019d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.360+0000 7fc3be59c640 1 -- 172.21.15.43:0/139914066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3c0103af0 msgr2=0x7fc3c0198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.360+0000 7fc3be59c640 1 --2- 172.21.15.43:0/139914066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3c0103af0 0x7fc3c0198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.360+0000 7fc3be59c640 1 -- 172.21.15.43:0/139914066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3c010c510 msgr2=0x7fc3c019d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.360+0000 7fc3be59c640 1 --2- 172.21.15.43:0/139914066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3c010c510 0x7fc3c019d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.360+0000 7fc3be59c640 1 -- 172.21.15.43:0/139914066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c019d730 con 0x7fc3c01031d0 2024-04-21T08:39:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.360+0000 7fc3be59c640 1 --2- 172.21.15.43:0/139914066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3c01031d0 0x7fc3c0198790 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc3b000d950 tx=0x7fc3b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.361+0000 7fc3af7fe640 1 -- 172.21.15.43:0/139914066 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0017030 con 0x7fc3c01031d0 2024-04-21T08:39:33.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.361+0000 7fc3c4975640 1 -- 172.21.15.43:0/139914066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c019da20 con 0x7fc3c01031d0 2024-04-21T08:39:33.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.361+0000 7fc3af7fe640 1 -- 172.21.15.43:0/139914066 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3b0004020 con 0x7fc3c01031d0 2024-04-21T08:39:33.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.361+0000 7fc3af7fe640 1 -- 172.21.15.43:0/139914066 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b00047a0 con 0x7fc3c01031d0 2024-04-21T08:39:33.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.361+0000 7fc3c4975640 1 -- 172.21.15.43:0/139914066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c01a52b0 con 0x7fc3c01031d0 2024-04-21T08:39:33.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.363+0000 7fc3af7fe640 1 -- 172.21.15.43:0/139914066 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fc3b0009a30 con 0x7fc3c01031d0 2024-04-21T08:39:33.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.364+0000 7fc3af7fe640 1 --2- 172.21.15.43:0/139914066 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc3a80a81c0 0x7fc3a80aa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.364+0000 7fc3bdd9b640 1 --2- 172.21.15.43:0/139914066 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc3a80a81c0 0x7fc3a80aa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.365+0000 7fc3c7567740 1 Processor -- start 2024-04-21T08:39:33.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.365+0000 7fc3af7fe640 1 -- 172.21.15.43:0/139914066 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc3b00d0720 con 0x7fc3c01031d0 2024-04-21T08:39:33.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.365+0000 7fc3bdd9b640 1 --2- 172.21.15.43:0/139914066 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc3a80a81c0 0x7fc3a80aa680 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4005ab0 tx=0x7fc3b4005a20 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:33.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.365+0000 7fc3c7567740 1 -- start start 2024-04-21T08:39:33.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.365+0000 7fc3c7567740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd4ea148e0 0x55fd4ea14cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.365+0000 7fc3c7567740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd4ea15290 0x55fd4ea1d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.365+0000 7fc3c7567740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea1de10 0x55fd4ea20260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.365+0000 7fc3c7567740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fd4e978f00 con 0x55fd4ea1de10 2024-04-21T08:39:33.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.365+0000 7fc3c7567740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fd4e916590 con 0x55fd4ea15290 2024-04-21T08:39:33.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.365+0000 7fc3c7567740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fd4e974ec0 con 0x55fd4ea148e0 2024-04-21T08:39:33.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.366+0000 7fc3bed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea1de10 0x55fd4ea20260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.366+0000 7fc3bed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea1de10 0x55fd4ea20260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49190/0 (socket says 172.21.15.43:49190) 2024-04-21T08:39:33.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.366+0000 7fc3bed9d640 1 -- 172.21.15.43:0/3587827779 learned_addr learned my addr 172.21.15.43:0/3587827779 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:33.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.366+0000 7fc3be59c640 1 --2- 172.21.15.43:0/3587827779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd4ea148e0 0x55fd4ea14cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.366+0000 7fc3bdd9b640 1 --2- 172.21.15.43:0/3587827779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd4ea15290 0x55fd4ea1d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.366+0000 7fc3bed9d640 1 -- 172.21.15.43:0/3587827779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd4ea148e0 msgr2=0x55fd4ea14cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:33.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.366+0000 7fc3bed9d640 1 --2- 172.21.15.43:0/3587827779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd4ea148e0 0x55fd4ea14cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.366+0000 7fc3bed9d640 1 -- 172.21.15.43:0/3587827779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd4ea15290 msgr2=0x55fd4ea1d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:33.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.366+0000 7fc3bed9d640 1 --2- 172.21.15.43:0/3587827779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd4ea15290 0x55fd4ea1d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.366+0000 7fc3bed9d640 1 -- 172.21.15.43:0/3587827779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd4e7537e0 con 0x55fd4ea1de10 2024-04-21T08:39:33.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.366+0000 7fc3bed9d640 1 --2- 172.21.15.43:0/3587827779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea1de10 0x55fd4ea20260 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009f50 tx=0x7fc3b80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5155d86c7264ec server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:33.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.366+0000 7fc3acff9640 1 -- 172.21.15.43:0/3587827779 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8004d20 con 0x55fd4ea1de10 2024-04-21T08:39:33.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.367+0000 7fc3acff9640 1 -- 172.21.15.43:0/3587827779 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3b8004050 con 0x55fd4ea1de10 2024-04-21T08:39:33.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.367+0000 7fc3acff9640 1 -- 172.21.15.43:0/3587827779 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8005280 con 0x55fd4ea1de10 2024-04-21T08:39:33.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.367+0000 7fc3c7567740 1 -- 172.21.15.43:0/3587827779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea1de10 msgr2=0x55fd4ea20260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.367+0000 7fc3c7567740 1 --2- 172.21.15.43:0/3587827779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea1de10 0x55fd4ea20260 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009f50 tx=0x7fc3b80118d0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.368+0000 7fc3c7567740 1 -- 172.21.15.43:0/3587827779 shutdown_connections 2024-04-21T08:39:33.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.368+0000 7fc3c7567740 1 --2- 172.21.15.43:0/3587827779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea1de10 0x55fd4ea20260 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.368+0000 7fc3c7567740 1 --2- 172.21.15.43:0/3587827779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd4ea15290 0x55fd4ea1d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.368+0000 7fc3c7567740 1 --2- 172.21.15.43:0/3587827779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd4ea148e0 0x55fd4ea14cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.368+0000 7fc3c7567740 1 -- 172.21.15.43:0/3587827779 >> 172.21.15.43:0/3587827779 conn(0x55fd4ea0efb0 msgr2=0x55fd4ea0f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.368+0000 7fc3c7567740 1 -- 172.21.15.43:0/3587827779 shutdown_connections 2024-04-21T08:39:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.368+0000 7fc3c7567740 1 -- 172.21.15.43:0/3587827779 wait complete. 2024-04-21T08:39:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.368+0000 7fc3c7567740 1 -- start start 2024-04-21T08:39:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.368+0000 7fc3c7567740 1 Processor -- start 2024-04-21T08:39:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.368+0000 7fc3c7567740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea148e0 0x55fd4ea1ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.368+0000 7fc3c7567740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd4ea15290 0x55fd4ea1b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.368+0000 7fc3c7567740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd4ea1de10 0x55fd4ea1b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.368+0000 7fc3c7567740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fd4ea20d80 con 0x55fd4ea148e0 2024-04-21T08:39:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.368+0000 7fc3c7567740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fd4e932580 con 0x55fd4ea1de10 2024-04-21T08:39:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.368+0000 7fc3c7567740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fd4ea20f00 con 0x55fd4ea15290 2024-04-21T08:39:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.369+0000 7fc3c7567740 10 client.0 _ll_register_callbacks cb 0x55fd4e938650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.369+0000 7fc3be59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea148e0 0x55fd4ea1ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.369+0000 7fc3be59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea148e0 0x55fd4ea1ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49198/0 (socket says 172.21.15.43:49198) 2024-04-21T08:39:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.369+0000 7fc3be59c640 1 -- 172.21.15.43:0/1283133583 learned_addr learned my addr 172.21.15.43:0/1283133583 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.369+0000 7fc3bdd9b640 1 --2- 172.21.15.43:0/1283133583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd4ea15290 0x55fd4ea1b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.369+0000 7fc3bed9d640 1 --2- 172.21.15.43:0/1283133583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd4ea1de10 0x55fd4ea1b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.369+0000 7fc3be59c640 1 -- 172.21.15.43:0/1283133583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd4ea15290 msgr2=0x55fd4ea1b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.369+0000 7fc3be59c640 1 --2- 172.21.15.43:0/1283133583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd4ea15290 0x55fd4ea1b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.369+0000 7fc3be59c640 1 -- 172.21.15.43:0/1283133583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd4ea1de10 msgr2=0x55fd4ea1b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.369+0000 7fc3be59c640 1 --2- 172.21.15.43:0/1283133583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd4ea1de10 0x55fd4ea1b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.369+0000 7fc3be59c640 1 -- 172.21.15.43:0/1283133583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd4e7b3f70 con 0x55fd4ea148e0 2024-04-21T08:39:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.369+0000 7fc3bdd9b640 1 --2- 172.21.15.43:0/1283133583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd4ea15290 0x55fd4ea1b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.369+0000 7fc3be59c640 1 --2- 172.21.15.43:0/1283133583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea148e0 0x55fd4ea1ad70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc3b00985e0 tx=0x7fc3b009c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560f68ac4305e63d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.369+0000 7fc3a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.369+0000 7fc3a5ffb640 1 -- 172.21.15.43:0/1283133583 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b00a01b0 con 0x55fd4ea148e0 2024-04-21T08:39:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.369+0000 7fc3a5ffb640 1 -- 172.21.15.43:0/1283133583 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc3b00987f0 con 0x55fd4ea148e0 2024-04-21T08:39:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.369+0000 7fc3c7567740 1 -- 172.21.15.43:0/1283133583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3b8004050 con 0x55fd4ea148e0 2024-04-21T08:39:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.370+0000 7fc3a5ffb640 1 -- 172.21.15.43:0/1283133583 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc3b00cdc90 con 0x55fd4ea148e0 2024-04-21T08:39:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.370+0000 7fc3c7567740 10 client.16179 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.370+0000 7fc3c7567740 1 -- 172.21.15.43:0/1283133583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fd4e64b170 con 0x55fd4ea148e0 2024-04-21T08:39:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.370+0000 7fc3a5ffb640 1 -- 172.21.15.43:0/1283133583 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3b00cde30 con 0x55fd4ea148e0 2024-04-21T08:39:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.370+0000 7fc3a5ffb640 1 -- 172.21.15.43:0/1283133583 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc3b0097ed0 con 0x55fd4ea148e0 2024-04-21T08:39:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.370+0000 7fc3c7567740 10 client.16179 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.370+0000 7fc3c7567740 10 client.16179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.370+0000 7fc3c7567740 20 client.16179 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.370+0000 7fc3c7567740 1 --2- 172.21.15.43:0/1283133583 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fd4ea2e2c0 0x55fd4ea306f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.370+0000 7fc3c7567740 4 client.16179 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.370+0000 7fc3c7567740 1 -- 172.21.15.43:0/1283133583 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc3b8004050 con 0x55fd4ea2e2c0 2024-04-21T08:39:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.370+0000 7fc3bdd9b640 1 --2- 172.21.15.43:0/1283133583 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fd4ea2e2c0 0x55fd4ea306f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.371+0000 7fc3a5ffb640 10 client.16179 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.371+0000 7fc3bdd9b640 1 --2- 172.21.15.43:0/1283133583 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fd4ea2e2c0 0x55fd4ea306f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=863ec7296e1223be server_cookie=421965f71a8628cc in_seq=0 out_seq=0 2024-04-21T08:39:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.372+0000 7fc3a5ffb640 1 -- 172.21.15.43:0/1283133583 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fc3b8004050 con 0x55fd4ea2e2c0 2024-04-21T08:39:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.372+0000 7fc3a5ffb640 10 client.16179 handle_command_reply: tid=0 2024-04-21T08:39:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.372+0000 7fc3c7567740 1 client.16179 shutdown 2024-04-21T08:39:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.372+0000 7fc3c7567740 1 -- 172.21.15.43:0/1283133583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea148e0 msgr2=0x55fd4ea1ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.372+0000 7fc3c7567740 1 --2- 172.21.15.43:0/1283133583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea148e0 0x55fd4ea1ad70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc3b00985e0 tx=0x7fc3b009c9c0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.372+0000 7fc3c7567740 1 -- 172.21.15.43:0/1283133583 shutdown_connections 2024-04-21T08:39:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.372+0000 7fc3c7567740 1 --2- 172.21.15.43:0/1283133583 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fd4ea2e2c0 0x55fd4ea306f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.372+0000 7fc3c7567740 1 --2- 172.21.15.43:0/1283133583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd4ea1de10 0x55fd4ea1b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.372+0000 7fc3c7567740 1 --2- 172.21.15.43:0/1283133583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd4ea15290 0x55fd4ea1b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.372+0000 7fc3a5ffb640 0 client.16179 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.372+0000 7fc3c7567740 1 --2- 172.21.15.43:0/1283133583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea148e0 0x55fd4ea1ad70 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.372+0000 7fc3c7567740 1 -- 172.21.15.43:0/1283133583 >> 172.21.15.43:0/1283133583 conn(0x55fd4ea0efb0 msgr2=0x55fd4ea10d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:33.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.372+0000 7fc3c7567740 1 -- 172.21.15.43:0/1283133583 shutdown_connections 2024-04-21T08:39:33.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.372+0000 7fc3c7567740 1 -- 172.21.15.43:0/1283133583 wait complete. 2024-04-21T08:39:33.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.372+0000 7fc3c7567740 20 client.16179 trim_cache size 0 max 16384 2024-04-21T08:39:33.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.406+0000 7fc3c7567740 1 Processor -- start 2024-04-21T08:39:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.406+0000 7fc3c7567740 1 -- start start 2024-04-21T08:39:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.407+0000 7fc3c7567740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd4ea148e0 0x55fd4ea1d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.407+0000 7fc3c7567740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd4ea15290 0x55fd4ea29c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.407+0000 7fc3c7567740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea1de10 0x55fd4ea10b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.407+0000 7fc3c7567740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c01ad340 con 0x55fd4ea1de10 2024-04-21T08:39:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.407+0000 7fc3c7567740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c01aee80 con 0x55fd4ea148e0 2024-04-21T08:39:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.407+0000 7fc3c7567740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c01ada10 con 0x55fd4ea15290 2024-04-21T08:39:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.407+0000 7fc3bed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea1de10 0x55fd4ea10b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.407+0000 7fc3bed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea1de10 0x55fd4ea10b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49206/0 (socket says 172.21.15.43:49206) 2024-04-21T08:39:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.407+0000 7fc3bed9d640 1 -- 172.21.15.43:0/2534361141 learned_addr learned my addr 172.21.15.43:0/2534361141 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.407+0000 7fc3be59c640 1 --2- 172.21.15.43:0/2534361141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd4ea148e0 0x55fd4ea1d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.407+0000 7fc3bdd9b640 1 --2- 172.21.15.43:0/2534361141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd4ea15290 0x55fd4ea29c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.407+0000 7fc3bed9d640 1 -- 172.21.15.43:0/2534361141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd4ea15290 msgr2=0x55fd4ea29c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:33.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.407+0000 7fc3bed9d640 1 --2- 172.21.15.43:0/2534361141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd4ea15290 0x55fd4ea29c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.407+0000 7fc3bed9d640 1 -- 172.21.15.43:0/2534361141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd4ea148e0 msgr2=0x55fd4ea1d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:33.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.407+0000 7fc3bed9d640 1 --2- 172.21.15.43:0/2534361141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd4ea148e0 0x55fd4ea1d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.407+0000 7fc3bed9d640 1 -- 172.21.15.43:0/2534361141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd4ea144a0 con 0x55fd4ea1de10 2024-04-21T08:39:33.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.407+0000 7fc3bdd9b640 1 --2- 172.21.15.43:0/2534361141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd4ea15290 0x55fd4ea29c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:33.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.408+0000 7fc3bed9d640 1 --2- 172.21.15.43:0/2534361141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea1de10 0x55fd4ea10b10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009f50 tx=0x7fc3b8009cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdaa2d47085fcef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:33.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.408+0000 7fc3a7fff640 1 -- 172.21.15.43:0/2534361141 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8003b70 con 0x55fd4ea1de10 2024-04-21T08:39:33.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.408+0000 7fc3a7fff640 1 -- 172.21.15.43:0/2534361141 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3b8003d10 con 0x55fd4ea1de10 2024-04-21T08:39:33.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.408+0000 7fc3a7fff640 1 -- 172.21.15.43:0/2534361141 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8025830 con 0x55fd4ea1de10 2024-04-21T08:39:33.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.409+0000 7fc3c7567740 1 -- 172.21.15.43:0/2534361141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea1de10 msgr2=0x55fd4ea10b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.409+0000 7fc3c7567740 1 --2- 172.21.15.43:0/2534361141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea1de10 0x55fd4ea10b10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009f50 tx=0x7fc3b8009cc0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.409+0000 7fc3c7567740 1 -- 172.21.15.43:0/2534361141 shutdown_connections 2024-04-21T08:39:33.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.409+0000 7fc3c7567740 1 --2- 172.21.15.43:0/2534361141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea1de10 0x55fd4ea10b10 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.409+0000 7fc3c7567740 1 --2- 172.21.15.43:0/2534361141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd4ea15290 0x55fd4ea29c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.409+0000 7fc3c7567740 1 --2- 172.21.15.43:0/2534361141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd4ea148e0 0x55fd4ea1d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.409+0000 7fc3c7567740 1 -- 172.21.15.43:0/2534361141 >> 172.21.15.43:0/2534361141 conn(0x55fd4ea0efb0 msgr2=0x55fd4ea10400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:33.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.409+0000 7fc3c7567740 1 -- 172.21.15.43:0/2534361141 shutdown_connections 2024-04-21T08:39:33.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.409+0000 7fc3c7567740 1 -- 172.21.15.43:0/2534361141 wait complete. 2024-04-21T08:39:33.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.410+0000 7fc3c7567740 1 -- start start 2024-04-21T08:39:33.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.410+0000 7fc3c7567740 1 Processor -- start 2024-04-21T08:39:33.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.411+0000 7fc3c7567740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd4ea148e0 0x55fd4ea17db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.411+0000 7fc3c7567740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd4ea15290 0x55fd4ea182f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.411+0000 7fc3c7567740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea1de10 0x55fd4ea93850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.411+0000 7fc3be59c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd4ea148e0 0x55fd4ea17db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.411+0000 7fc3be59c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd4ea148e0 0x55fd4ea17db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.43:44450/0 (socket says 172.21.15.43:44450) 2024-04-21T08:39:33.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.411+0000 7fc3bdd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd4ea15290 0x55fd4ea182f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.411+0000 7fc3bed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea1de10 0x55fd4ea93850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.411+0000 7fc3bdd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd4ea15290 0x55fd4ea182f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:40096/0 (socket says 172.21.15.43:40096) 2024-04-21T08:39:33.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.411+0000 7fc3bed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea1de10 0x55fd4ea93850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49214/0 (socket says 172.21.15.43:49214) 2024-04-21T08:39:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.411+0000 7fc3be59c640 1 -- 172.21.15.43:0/1511430166 learned_addr learned my addr 172.21.15.43:0/1511430166 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.411+0000 7fc3c7567740 1 -- 172.21.15.43:0/1511430166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c01ae7b0 con 0x55fd4ea1de10 2024-04-21T08:39:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.411+0000 7fc3c7567740 1 -- 172.21.15.43:0/1511430166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c01ae0e0 con 0x55fd4ea15290 2024-04-21T08:39:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.411+0000 7fc3c7567740 1 -- 172.21.15.43:0/1511430166 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c01ab320 con 0x55fd4ea148e0 2024-04-21T08:39:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.411+0000 7fc3c7567740 10 client.0 _ll_register_callbacks cb 0x7fc3b009bcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.412+0000 7fc3be59c640 1 -- 172.21.15.43:0/1511430166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd4ea15290 msgr2=0x55fd4ea182f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.412+0000 7fc3be59c640 1 --2- 172.21.15.43:0/1511430166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd4ea15290 0x55fd4ea182f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.412+0000 7fc3be59c640 1 -- 172.21.15.43:0/1511430166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea1de10 msgr2=0x55fd4ea93850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.412+0000 7fc3be59c640 1 --2- 172.21.15.43:0/1511430166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea1de10 0x55fd4ea93850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.412+0000 7fc3be59c640 1 -- 172.21.15.43:0/1511430166 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd4ea93d90 con 0x55fd4ea148e0 2024-04-21T08:39:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.412+0000 7fc3be59c640 1 --2- 172.21.15.43:0/1511430166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd4ea148e0 0x55fd4ea17db0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc3b00a06f0 tx=0x7fc3b0005ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85cddae90488d725 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.412+0000 7fc3a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-04-21T08:39:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.412+0000 7fc3a5ffb640 1 -- 172.21.15.43:0/1511430166 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b00987f0 con 0x55fd4ea148e0 2024-04-21T08:39:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.412+0000 7fc3a5ffb640 1 -- 172.21.15.43:0/1511430166 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc3b009b180 con 0x55fd4ea148e0 2024-04-21T08:39:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.412+0000 7fc3c7567740 1 -- 172.21.15.43:0/1511430166 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3b8003d10 con 0x55fd4ea148e0 2024-04-21T08:39:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.412+0000 7fc3a5ffb640 1 -- 172.21.15.43:0/1511430166 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc3b0098990 con 0x55fd4ea148e0 2024-04-21T08:39:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.412+0000 7fc3c7567740 10 client.25594 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.412+0000 7fc3c7567740 1 -- 172.21.15.43:0/1511430166 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fd4ea94eb0 con 0x55fd4ea148e0 2024-04-21T08:39:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.413+0000 7fc3a5ffb640 1 -- 172.21.15.43:0/1511430166 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3b009b380 con 0x55fd4ea148e0 2024-04-21T08:39:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.413+0000 7fc3a5ffb640 1 -- 172.21.15.43:0/1511430166 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc3b00d02e0 con 0x55fd4ea148e0 2024-04-21T08:39:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.413+0000 7fc3c7567740 10 client.25594 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.413+0000 7fc3c7567740 10 client.25594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.413+0000 7fc3c7567740 20 client.25594 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.413+0000 7fc3c7567740 1 --2- 172.21.15.43:0/1511430166 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fd4ea95490 0x55fd4ea97880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.413+0000 7fc3c7567740 4 client.25594 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:39:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.413+0000 7fc3c7567740 1 -- 172.21.15.43:0/1511430166 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fc3b8003d10 con 0x55fd4ea95490 2024-04-21T08:39:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.413+0000 7fc3bdd9b640 1 --2- 172.21.15.43:0/1511430166 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fd4ea95490 0x55fd4ea97880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.414+0000 7fc3a5ffb640 10 client.25594 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:33.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.414+0000 7fc3bdd9b640 1 --2- 172.21.15.43:0/1511430166 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fd4ea95490 0x55fd4ea97880 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd4cfea0de917107 server_cookie=4efba21b98b24610 in_seq=0 out_seq=0 2024-04-21T08:39:33.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.415+0000 7fc3a5ffb640 1 -- 172.21.15.43:0/1511430166 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fc3b8003d10 con 0x55fd4ea95490 2024-04-21T08:39:33.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.415+0000 7fc3a5ffb640 10 client.25594 handle_command_reply: tid=0 2024-04-21T08:39:33.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.415+0000 7fc3c7567740 1 client.25594 shutdown 2024-04-21T08:39:33.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.415+0000 7fc3c7567740 1 -- 172.21.15.43:0/1511430166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd4ea148e0 msgr2=0x55fd4ea17db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.415+0000 7fc3c7567740 1 --2- 172.21.15.43:0/1511430166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd4ea148e0 0x55fd4ea17db0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc3b00a06f0 tx=0x7fc3b0005ab0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.415+0000 7fc3c7567740 1 -- 172.21.15.43:0/1511430166 shutdown_connections 2024-04-21T08:39:33.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.415+0000 7fc3c7567740 1 --2- 172.21.15.43:0/1511430166 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fd4ea95490 0x55fd4ea97880 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.415+0000 7fc3c7567740 1 --2- 172.21.15.43:0/1511430166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd4ea1de10 0x55fd4ea93850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.415+0000 7fc3c7567740 1 --2- 172.21.15.43:0/1511430166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd4ea15290 0x55fd4ea182f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.415+0000 7fc3c7567740 1 --2- 172.21.15.43:0/1511430166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd4ea148e0 0x55fd4ea17db0 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc3b00a06f0 tx=0x7fc3b0005ab0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.415+0000 7fc3c7567740 1 -- 172.21.15.43:0/1511430166 >> 172.21.15.43:0/1511430166 conn(0x55fd4ea0efb0 msgr2=0x55fd4ea10c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:33.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.415+0000 7fc3a5ffb640 0 client.25594 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:33.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.416+0000 7fc3c7567740 1 -- 172.21.15.43:0/1511430166 shutdown_connections 2024-04-21T08:39:33.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.416+0000 7fc3c7567740 1 -- 172.21.15.43:0/1511430166 wait complete. 2024-04-21T08:39:33.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.416+0000 7fc3c7567740 20 client.25594 trim_cache size 0 max 16384 2024-04-21T08:39:33.420 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:39:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:39:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "6624c4a7-a3a1-4451-a91a-240e5fd6bb71", 2024-04-21T08:39:33.420 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:39:33.420 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:39:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.416+0000 7fc3c4975640 1 -- 172.21.15.43:0/139914066 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc3a80a81c0 msgr2=0x7fc3a80aa680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.416+0000 7fc3c4975640 1 --2- 172.21.15.43:0/139914066 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc3a80a81c0 0x7fc3a80aa680 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4005ab0 tx=0x7fc3b4005a20 comp rx=0 tx=0).stop 2024-04-21T08:39:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.416+0000 7fc3c4975640 1 -- 172.21.15.43:0/139914066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3c01031d0 msgr2=0x7fc3c0198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.416+0000 7fc3c4975640 1 --2- 172.21.15.43:0/139914066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3c01031d0 0x7fc3c0198790 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc3b000d950 tx=0x7fc3b000de10 comp rx=0 tx=0).stop 2024-04-21T08:39:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.416+0000 7fc3c4975640 1 -- 172.21.15.43:0/139914066 shutdown_connections 2024-04-21T08:39:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.416+0000 7fc3c4975640 1 --2- 172.21.15.43:0/139914066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3c010c510 0x7fc3c019d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.416+0000 7fc3c4975640 1 --2- 172.21.15.43:0/139914066 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc3a80a81c0 0x7fc3a80aa680 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.416+0000 7fc3c4975640 1 --2- 172.21.15.43:0/139914066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3c0103af0 0x7fc3c0198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.416+0000 7fc3c4975640 1 --2- 172.21.15.43:0/139914066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3c01031d0 0x7fc3c0198790 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.416+0000 7fc3c4975640 1 -- 172.21.15.43:0/139914066 >> 172.21.15.43:0/139914066 conn(0x7fc3c0077f50 msgr2=0x7fc3c00fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.417+0000 7fc3c4975640 1 -- 172.21.15.43:0/139914066 shutdown_connections 2024-04-21T08:39:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.417+0000 7fc3c4975640 1 -- 172.21.15.43:0/139914066 wait complete. 2024-04-21T08:39:33.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:39:33.432 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:39:33.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.540+0000 7fb7b4a11640 1 -- 172.21.15.43:0/2306538259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7b010c500 msgr2=0x7fb7b010e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.540+0000 7fb7b4a11640 1 --2- 172.21.15.43:0/2306538259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7b010c500 0x7fb7b010e9a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009ca0 tx=0x7fb7a802c230 comp rx=0 tx=0).stop 2024-04-21T08:39:33.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.540+0000 7fb7b4a11640 1 -- 172.21.15.43:0/2306538259 shutdown_connections 2024-04-21T08:39:33.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.540+0000 7fb7b4a11640 1 --2- 172.21.15.43:0/2306538259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7b010c500 0x7fb7b010e9a0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.540+0000 7fb7b4a11640 1 --2- 172.21.15.43:0/2306538259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7b0103ae0 0x7fb7b010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.540+0000 7fb7b4a11640 1 --2- 172.21.15.43:0/2306538259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7b01031c0 0x7fb7b01035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.540+0000 7fb7b4a11640 1 -- 172.21.15.43:0/2306538259 >> 172.21.15.43:0/2306538259 conn(0x7fb7b0078000 msgr2=0x7fb7b0078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:33.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.540+0000 7fb7b4a11640 1 -- 172.21.15.43:0/2306538259 shutdown_connections 2024-04-21T08:39:33.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.540+0000 7fb7b4a11640 1 -- 172.21.15.43:0/2306538259 wait complete. 2024-04-21T08:39:33.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.541+0000 7fb7b4a11640 1 Processor -- start 2024-04-21T08:39:33.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.541+0000 7fb7b4a11640 1 -- start start 2024-04-21T08:39:33.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.541+0000 7fb7b4a11640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7b01031c0 0x7fb7b0198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.541+0000 7fb7b4a11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7b0103ae0 0x7fb7b0198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.541+0000 7fb7b4a11640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7b010c500 0x7fb7b019cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.542+0000 7fb7b4a11640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b0112190 con 0x7fb7b0103ae0 2024-04-21T08:39:33.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.542+0000 7fb7b4a11640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b0112010 con 0x7fb7b010c500 2024-04-21T08:39:33.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.542+0000 7fb7b4a11640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b0112310 con 0x7fb7b01031c0 2024-04-21T08:39:33.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.542+0000 7fb7add9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7b0103ae0 0x7fb7b0198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.542+0000 7fb7add9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7b0103ae0 0x7fb7b0198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49232/0 (socket says 172.21.15.43:49232) 2024-04-21T08:39:33.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.542+0000 7fb7add9b640 1 -- 172.21.15.43:0/1794556210 learned_addr learned my addr 172.21.15.43:0/1794556210 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:33.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.542+0000 7fb7ae59c640 1 --2- 172.21.15.43:0/1794556210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7b01031c0 0x7fb7b0198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.542+0000 7fb7aed9d640 1 --2- 172.21.15.43:0/1794556210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7b010c500 0x7fb7b019cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.542+0000 7fb7add9b640 1 -- 172.21.15.43:0/1794556210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7b01031c0 msgr2=0x7fb7b0198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.542+0000 7fb7add9b640 1 --2- 172.21.15.43:0/1794556210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7b01031c0 0x7fb7b0198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.542+0000 7fb7add9b640 1 -- 172.21.15.43:0/1794556210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7b010c500 msgr2=0x7fb7b019cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.542+0000 7fb7add9b640 1 --2- 172.21.15.43:0/1794556210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7b010c500 0x7fb7b019cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.542+0000 7fb7add9b640 1 -- 172.21.15.43:0/1794556210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b019d720 con 0x7fb7b0103ae0 2024-04-21T08:39:33.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.542+0000 7fb7aed9d640 1 --2- 172.21.15.43:0/1794556210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7b010c500 0x7fb7b019cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:33.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.542+0000 7fb7add9b640 1 --2- 172.21.15.43:0/1794556210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7b0103ae0 0x7fb7b0198c70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400e9f0 tx=0x7fb7a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:33.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.543+0000 7fb79f7fe640 1 -- 172.21.15.43:0/1794556210 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a400cd80 con 0x7fb7b0103ae0 2024-04-21T08:39:33.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.543+0000 7fb79f7fe640 1 -- 172.21.15.43:0/1794556210 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7a40108e0 con 0x7fb7b0103ae0 2024-04-21T08:39:33.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.543+0000 7fb7b4a11640 1 -- 172.21.15.43:0/1794556210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b019da10 con 0x7fb7b0103ae0 2024-04-21T08:39:33.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.543+0000 7fb79f7fe640 1 -- 172.21.15.43:0/1794556210 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4018980 con 0x7fb7b0103ae0 2024-04-21T08:39:33.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.543+0000 7fb7b4a11640 1 -- 172.21.15.43:0/1794556210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b01a52a0 con 0x7fb7b0103ae0 2024-04-21T08:39:33.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.546+0000 7fb7b7601740 1 Processor -- start 2024-04-21T08:39:33.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.546+0000 7fb7b7601740 1 -- start start 2024-04-21T08:39:33.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.546+0000 7fb7b7601740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d94860dc20 0x55d94860e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.546+0000 7fb7b7601740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d94860e5d0 0x55d9485fa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.546+0000 7fb7b7601740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9485fad80 0x55d9485fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.546+0000 7fb7b7601740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d9485651e0 con 0x55d9485fad80 2024-04-21T08:39:33.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.546+0000 7fb7b7601740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d948502390 con 0x55d94860dc20 2024-04-21T08:39:33.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.546+0000 7fb7b7601740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d9485611a0 con 0x55d94860e5d0 2024-04-21T08:39:33.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.546+0000 7fb79f7fe640 1 -- 172.21.15.43:0/1794556210 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fb7a4018b20 con 0x7fb7b0103ae0 2024-04-21T08:39:33.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.547+0000 7fb7aed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d94860dc20 0x55d94860e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.547+0000 7fb7aed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d94860dc20 0x55d94860e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:40114/0 (socket says 172.21.15.43:40114) 2024-04-21T08:39:33.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.547+0000 7fb7aed9d640 1 -- 172.21.15.43:0/3778217001 learned_addr learned my addr 172.21.15.43:0/3778217001 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:33.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.547+0000 7fb7ae59c640 1 --2- 172.21.15.43:0/3778217001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d94860e5d0 0x55d9485fa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.547+0000 7fb7add9b640 1 --2- 172.21.15.43:0/3778217001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9485fad80 0x55d9485fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.547+0000 7fb7aed9d640 1 -- 172.21.15.43:0/3778217001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d94860e5d0 msgr2=0x55d9485fa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:33.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.547+0000 7fb7aed9d640 1 --2- 172.21.15.43:0/3778217001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d94860e5d0 0x55d9485fa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.547+0000 7fb7aed9d640 1 -- 172.21.15.43:0/3778217001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9485fad80 msgr2=0x55d9485fb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:33.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.547+0000 7fb7aed9d640 1 --2- 172.21.15.43:0/3778217001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9485fad80 0x55d9485fb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.547+0000 7fb7aed9d640 1 -- 172.21.15.43:0/3778217001 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9483417d0 con 0x55d94860dc20 2024-04-21T08:39:33.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.547+0000 7fb7add9b640 1 --2- 172.21.15.43:0/3778217001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9485fad80 0x55d9485fb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.547+0000 7fb7aed9d640 1 --2- 172.21.15.43:0/3778217001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d94860dc20 0x55d94860e000 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8004af0 tx=0x7fb7a8009930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=910e24172a9093fb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.548+0000 7fb79f7fe640 1 --2- 172.21.15.43:0/1794556210 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb7980a3e30 0x7fb7980a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.548+0000 7fb79cff9640 1 -- 172.21.15.43:0/3778217001 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8039050 con 0x55d94860dc20 2024-04-21T08:39:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.548+0000 7fb79f7fe640 1 -- 172.21.15.43:0/1794556210 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fb7a40d1140 con 0x7fb7b0103ae0 2024-04-21T08:39:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.548+0000 7fb79cff9640 1 -- 172.21.15.43:0/3778217001 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7a8045070 con 0x55d94860dc20 2024-04-21T08:39:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.548+0000 7fb79cff9640 1 -- 172.21.15.43:0/3778217001 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8032640 con 0x55d94860dc20 2024-04-21T08:39:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.548+0000 7fb7ae59c640 1 --2- 172.21.15.43:0/1794556210 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb7980a3e30 0x7fb7980a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.549+0000 7fb7b7601740 1 -- 172.21.15.43:0/3778217001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d94860dc20 msgr2=0x55d94860e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.549+0000 7fb7ae59c640 1 --2- 172.21.15.43:0/1794556210 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb7980a3e30 0x7fb7980a62f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55d948609400 tx=0x7fb7a000b4d0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.549+0000 7fb7b7601740 1 --2- 172.21.15.43:0/3778217001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d94860dc20 0x55d94860e000 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8004af0 tx=0x7fb7a8009930 comp rx=0 tx=0).stop 2024-04-21T08:39:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.549+0000 7fb7b7601740 1 -- 172.21.15.43:0/3778217001 shutdown_connections 2024-04-21T08:39:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.549+0000 7fb7b7601740 1 --2- 172.21.15.43:0/3778217001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9485fad80 0x55d9485fb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.549+0000 7fb7b7601740 1 --2- 172.21.15.43:0/3778217001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d94860e5d0 0x55d9485fa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.549+0000 7fb7b7601740 1 --2- 172.21.15.43:0/3778217001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d94860dc20 0x55d94860e000 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.549+0000 7fb7b7601740 1 -- 172.21.15.43:0/3778217001 >> 172.21.15.43:0/3778217001 conn(0x55d9485fd220 msgr2=0x55d9485fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.549+0000 7fb7b7601740 1 -- 172.21.15.43:0/3778217001 shutdown_connections 2024-04-21T08:39:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.549+0000 7fb7b7601740 1 -- 172.21.15.43:0/3778217001 wait complete. 2024-04-21T08:39:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.549+0000 7fb7b7601740 1 -- start start 2024-04-21T08:39:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.550+0000 7fb7b7601740 1 Processor -- start 2024-04-21T08:39:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.550+0000 7fb7b7601740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9485fad80 0x55d948611bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.550+0000 7fb7b7601740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d94860dc20 0x55d948612100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.550+0000 7fb7b7601740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d94860e5d0 0x55d948612640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.550+0000 7fb7b7601740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d948615050 con 0x55d94860dc20 2024-04-21T08:39:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.550+0000 7fb7b7601740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d94851e2f0 con 0x55d94860e5d0 2024-04-21T08:39:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.550+0000 7fb7b7601740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d9486151d0 con 0x55d9485fad80 2024-04-21T08:39:33.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.550+0000 7fb7b7601740 10 client.0 _ll_register_callbacks cb 0x55d94852b580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:33.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.550+0000 7fb7add9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d94860dc20 0x55d948612100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.550+0000 7fb7aed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d94860e5d0 0x55d948612640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.550+0000 7fb7add9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d94860dc20 0x55d948612100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49242/0 (socket says 172.21.15.43:49242) 2024-04-21T08:39:33.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.550+0000 7fb7add9b640 1 -- 172.21.15.43:0/2354804327 learned_addr learned my addr 172.21.15.43:0/2354804327 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:33.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.550+0000 7fb7ae59c640 1 --2- 172.21.15.43:0/2354804327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9485fad80 0x55d948611bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.551+0000 7fb7aed9d640 1 -- 172.21.15.43:0/2354804327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9485fad80 msgr2=0x55d948611bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:33.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.551+0000 7fb7aed9d640 1 --2- 172.21.15.43:0/2354804327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9485fad80 0x55d948611bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.551+0000 7fb7aed9d640 1 -- 172.21.15.43:0/2354804327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d94860dc20 msgr2=0x55d948612100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:33.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.551+0000 7fb7aed9d640 1 --2- 172.21.15.43:0/2354804327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d94860dc20 0x55d948612100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.551+0000 7fb7aed9d640 1 -- 172.21.15.43:0/2354804327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9482773f0 con 0x55d94860e5d0 2024-04-21T08:39:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.551+0000 7fb7add9b640 1 --2- 172.21.15.43:0/2354804327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d94860dc20 0x55d948612100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.551+0000 7fb7aed9d640 1 --2- 172.21.15.43:0/2354804327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d94860e5d0 0x55d948612640 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009bf0 tx=0x7fb7a8004410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c2ca230bbe6b630 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.551+0000 7fb795ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-21T08:39:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.551+0000 7fb795ffb640 1 -- 172.21.15.43:0/2354804327 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8003b70 con 0x55d94860e5d0 2024-04-21T08:39:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.551+0000 7fb7b7601740 1 -- 172.21.15.43:0/2354804327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7a8045070 con 0x55d94860e5d0 2024-04-21T08:39:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.551+0000 7fb795ffb640 1 -- 172.21.15.43:0/2354804327 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fb7a8003d10 con 0x55d94860e5d0 2024-04-21T08:39:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.552+0000 7fb795ffb640 1 -- 172.21.15.43:0/2354804327 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb7a805c050 con 0x55d94860e5d0 2024-04-21T08:39:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.552+0000 7fb7b7601740 10 client.25694 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.552+0000 7fb7b7601740 1 -- 172.21.15.43:0/2354804327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d9482370f0 con 0x55d94860e5d0 2024-04-21T08:39:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.552+0000 7fb795ffb640 1 -- 172.21.15.43:0/2354804327 <== mon.2 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7a8045070 con 0x55d94860e5d0 2024-04-21T08:39:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.552+0000 7fb795ffb640 1 -- 172.21.15.43:0/2354804327 <== mon.2 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb7a8040450 con 0x55d94860e5d0 2024-04-21T08:39:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.552+0000 7fb7b7601740 10 client.25694 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.552+0000 7fb7b7601740 10 client.25694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:33.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.552+0000 7fb7b7601740 20 client.25694 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:33.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.552+0000 7fb7b7601740 1 --2- 172.21.15.43:0/2354804327 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d9486221a0 0x55d948624590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.552+0000 7fb7b7601740 4 client.25694 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:33.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.552+0000 7fb7b7601740 1 -- 172.21.15.43:0/2354804327 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d948572c00 con 0x55d9486221a0 2024-04-21T08:39:33.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.553+0000 7fb7ae59c640 1 --2- 172.21.15.43:0/2354804327 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d9486221a0 0x55d948624590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.553+0000 7fb7ae59c640 1 --2- 172.21.15.43:0/2354804327 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d9486221a0 0x55d948624590 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a68445d351f9dffc server_cookie=32fccd0099fb3b3e in_seq=0 out_seq=0 2024-04-21T08:39:33.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.553+0000 7fb795ffb640 10 client.25694 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:33.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.554+0000 7fb795ffb640 1 -- 172.21.15.43:0/2354804327 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d948572c00 con 0x55d9486221a0 2024-04-21T08:39:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.554+0000 7fb795ffb640 10 client.25694 handle_command_reply: tid=0 2024-04-21T08:39:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.554+0000 7fb7b7601740 1 client.25694 shutdown 2024-04-21T08:39:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.554+0000 7fb7b7601740 1 -- 172.21.15.43:0/2354804327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d94860e5d0 msgr2=0x55d948612640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.554+0000 7fb7b7601740 1 --2- 172.21.15.43:0/2354804327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d94860e5d0 0x55d948612640 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009bf0 tx=0x7fb7a8004410 comp rx=0 tx=0).stop 2024-04-21T08:39:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.554+0000 7fb7b7601740 1 -- 172.21.15.43:0/2354804327 shutdown_connections 2024-04-21T08:39:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.554+0000 7fb7b7601740 1 --2- 172.21.15.43:0/2354804327 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d9486221a0 0x55d948624590 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.554+0000 7fb7b7601740 1 --2- 172.21.15.43:0/2354804327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d94860e5d0 0x55d948612640 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.554+0000 7fb7b7601740 1 --2- 172.21.15.43:0/2354804327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d94860dc20 0x55d948612100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.554+0000 7fb7b7601740 1 --2- 172.21.15.43:0/2354804327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9485fad80 0x55d948611bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.554+0000 7fb7b7601740 1 -- 172.21.15.43:0/2354804327 >> 172.21.15.43:0/2354804327 conn(0x55d9485fd220 msgr2=0x55d9485fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.554+0000 7fb795ffb640 0 client.25694 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.555+0000 7fb7b7601740 1 -- 172.21.15.43:0/2354804327 shutdown_connections 2024-04-21T08:39:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.555+0000 7fb7b7601740 1 -- 172.21.15.43:0/2354804327 wait complete. 2024-04-21T08:39:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.555+0000 7fb7b7601740 20 client.25694 trim_cache size 0 max 16384 2024-04-21T08:39:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.573+0000 7fb7b7601740 1 Processor -- start 2024-04-21T08:39:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.573+0000 7fb7b7601740 1 -- start start 2024-04-21T08:39:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.573+0000 7fb7b7601740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9485fad80 0x55d9486142f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.573+0000 7fb7b7601740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d94860dc20 0x55d9485f9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.573+0000 7fb7b7601740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d94860e5d0 0x55d94861d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.573+0000 7fb7b7601740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b01ad190 con 0x55d9485fad80 2024-04-21T08:39:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.573+0000 7fb7b7601740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b01aecd0 con 0x55d94860dc20 2024-04-21T08:39:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.573+0000 7fb7b7601740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b01ad860 con 0x55d94860e5d0 2024-04-21T08:39:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.573+0000 7fb7ae59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9485fad80 0x55d9486142f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.573+0000 7fb7ae59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9485fad80 0x55d9486142f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49254/0 (socket says 172.21.15.43:49254) 2024-04-21T08:39:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.573+0000 7fb7ae59c640 1 -- 172.21.15.43:0/1402393065 learned_addr learned my addr 172.21.15.43:0/1402393065 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.573+0000 7fb7add9b640 1 --2- 172.21.15.43:0/1402393065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d94860dc20 0x55d9485f9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.573+0000 7fb7aed9d640 1 --2- 172.21.15.43:0/1402393065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d94860e5d0 0x55d94861d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.573+0000 7fb7ae59c640 1 -- 172.21.15.43:0/1402393065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d94860e5d0 msgr2=0x55d94861d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.573+0000 7fb7ae59c640 1 --2- 172.21.15.43:0/1402393065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d94860e5d0 0x55d94861d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.573+0000 7fb7ae59c640 1 -- 172.21.15.43:0/1402393065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d94860dc20 msgr2=0x55d9485f9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.573+0000 7fb7ae59c640 1 --2- 172.21.15.43:0/1402393065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d94860dc20 0x55d9485f9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.573+0000 7fb7ae59c640 1 -- 172.21.15.43:0/1402393065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9485faad0 con 0x55d9485fad80 2024-04-21T08:39:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.573+0000 7fb7aed9d640 1 --2- 172.21.15.43:0/1402393065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d94860e5d0 0x55d94861d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.573+0000 7fb7add9b640 1 --2- 172.21.15.43:0/1402393065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d94860dc20 0x55d9485f9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.574+0000 7fb7ae59c640 1 --2- 172.21.15.43:0/1402393065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9485fad80 0x55d9486142f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000b840 tx=0x7fb7a0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64f800981833b27 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.574+0000 7fb797fff640 1 -- 172.21.15.43:0/1402393065 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a002e070 con 0x55d9485fad80 2024-04-21T08:39:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.574+0000 7fb797fff640 1 -- 172.21.15.43:0/1402393065 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7a0028840 con 0x55d9485fad80 2024-04-21T08:39:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.574+0000 7fb797fff640 1 -- 172.21.15.43:0/1402393065 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a001f3f0 con 0x55d9485fad80 2024-04-21T08:39:33.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.575+0000 7fb7b7601740 1 -- 172.21.15.43:0/1402393065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9485fad80 msgr2=0x55d9486142f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.575+0000 7fb7b7601740 1 --2- 172.21.15.43:0/1402393065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9485fad80 0x55d9486142f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000b840 tx=0x7fb7a0015000 comp rx=0 tx=0).stop 2024-04-21T08:39:33.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.575+0000 7fb7b7601740 1 -- 172.21.15.43:0/1402393065 shutdown_connections 2024-04-21T08:39:33.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.575+0000 7fb7b7601740 1 --2- 172.21.15.43:0/1402393065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d94860e5d0 0x55d94861d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.575+0000 7fb7b7601740 1 --2- 172.21.15.43:0/1402393065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d94860dc20 0x55d9485f9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.575+0000 7fb7b7601740 1 --2- 172.21.15.43:0/1402393065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9485fad80 0x55d9486142f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.575+0000 7fb7b7601740 1 -- 172.21.15.43:0/1402393065 >> 172.21.15.43:0/1402393065 conn(0x55d9485fd220 msgr2=0x55d9485f94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:33.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.575+0000 7fb7b7601740 1 -- 172.21.15.43:0/1402393065 shutdown_connections 2024-04-21T08:39:33.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.575+0000 7fb7b7601740 1 -- 172.21.15.43:0/1402393065 wait complete. 2024-04-21T08:39:33.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.575+0000 7fb7b7601740 1 -- start start 2024-04-21T08:39:33.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.575+0000 7fb7b7601740 1 Processor -- start 2024-04-21T08:39:33.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.576+0000 7fb7b7601740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9485fad80 0x55d9486271e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.576+0000 7fb7b7601740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d94860dc20 0x55d948627720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.576+0000 7fb7b7601740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d94860e5d0 0x55d948624960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.576+0000 7fb7b7601740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b01ae600 con 0x55d94860e5d0 2024-04-21T08:39:33.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.576+0000 7fb7b7601740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b01adf30 con 0x55d94860dc20 2024-04-21T08:39:33.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.576+0000 7fb7b7601740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b01ab170 con 0x55d9485fad80 2024-04-21T08:39:33.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.576+0000 7fb7aed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d94860e5d0 0x55d948624960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.576+0000 7fb7aed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d94860e5d0 0x55d948624960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49256/0 (socket says 172.21.15.43:49256) 2024-04-21T08:39:33.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.576+0000 7fb7aed9d640 1 -- 172.21.15.43:0/455684287 learned_addr learned my addr 172.21.15.43:0/455684287 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:33.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.576+0000 7fb7b7601740 10 client.0 _ll_register_callbacks cb 0x7fb7a000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:33.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.576+0000 7fb7ae59c640 1 --2- 172.21.15.43:0/455684287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9485fad80 0x55d9486271e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.576+0000 7fb7add9b640 1 --2- 172.21.15.43:0/455684287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d94860dc20 0x55d948627720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.576+0000 7fb7aed9d640 1 -- 172.21.15.43:0/455684287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9485fad80 msgr2=0x55d9486271e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:33.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.576+0000 7fb7aed9d640 1 --2- 172.21.15.43:0/455684287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9485fad80 0x55d9486271e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.576+0000 7fb7aed9d640 1 -- 172.21.15.43:0/455684287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d94860dc20 msgr2=0x55d948627720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:33.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.576+0000 7fb7aed9d640 1 --2- 172.21.15.43:0/455684287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d94860dc20 0x55d948627720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.576+0000 7fb7aed9d640 1 -- 172.21.15.43:0/455684287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d948625160 con 0x55d94860e5d0 2024-04-21T08:39:33.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.577+0000 7fb7aed9d640 1 --2- 172.21.15.43:0/455684287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d94860e5d0 0x55d948624960 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8000f80 tx=0x7fb7a8032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7564775e585777b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:33.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.577+0000 7fb795ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:33.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.577+0000 7fb795ffb640 1 -- 172.21.15.43:0/455684287 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8065060 con 0x55d94860e5d0 2024-04-21T08:39:33.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.577+0000 7fb795ffb640 1 -- 172.21.15.43:0/455684287 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fb7a8040a90 con 0x55d94860e5d0 2024-04-21T08:39:33.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.577+0000 7fb7b7601740 1 -- 172.21.15.43:0/455684287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7a0028840 con 0x55d94860e5d0 2024-04-21T08:39:33.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.577+0000 7fb795ffb640 1 -- 172.21.15.43:0/455684287 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb7a8039040 con 0x55d94860e5d0 2024-04-21T08:39:33.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.577+0000 7fb7b7601740 10 client.16203 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:33.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.577+0000 7fb7b7601740 1 -- 172.21.15.43:0/455684287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d948688030 con 0x55d94860e5d0 2024-04-21T08:39:33.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.577+0000 7fb795ffb640 1 -- 172.21.15.43:0/455684287 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7a0028840 con 0x55d94860e5d0 2024-04-21T08:39:33.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.578+0000 7fb795ffb640 1 -- 172.21.15.43:0/455684287 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb7a80326d0 con 0x55d94860e5d0 2024-04-21T08:39:33.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.578+0000 7fb7b7601740 10 client.16203 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:33.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.578+0000 7fb7b7601740 10 client.16203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:33.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.578+0000 7fb7b7601740 20 client.16203 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:33.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.578+0000 7fb7b7601740 1 --2- 172.21.15.43:0/455684287 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d948688720 0x55d94868ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.578+0000 7fb7b7601740 4 client.16203 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:39:33.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.578+0000 7fb7b7601740 1 -- 172.21.15.43:0/455684287 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d94856ca70 con 0x55d948688720 2024-04-21T08:39:33.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.578+0000 7fb7ae59c640 1 --2- 172.21.15.43:0/455684287 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d948688720 0x55d94868ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.578+0000 7fb795ffb640 10 client.16203 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:33.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.578+0000 7fb7ae59c640 1 --2- 172.21.15.43:0/455684287 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d948688720 0x55d94868ab40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62eb67038dc43f39 server_cookie=14a00c4d5c8f8db6 in_seq=0 out_seq=0 2024-04-21T08:39:33.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.579+0000 7fb795ffb640 1 -- 172.21.15.43:0/455684287 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d94856ca70 con 0x55d948688720 2024-04-21T08:39:33.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.579+0000 7fb795ffb640 10 client.16203 handle_command_reply: tid=0 2024-04-21T08:39:33.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.579+0000 7fb7b7601740 1 client.16203 shutdown 2024-04-21T08:39:33.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.579+0000 7fb7b7601740 1 -- 172.21.15.43:0/455684287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d94860e5d0 msgr2=0x55d948624960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.579+0000 7fb7b7601740 1 --2- 172.21.15.43:0/455684287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d94860e5d0 0x55d948624960 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8000f80 tx=0x7fb7a8032570 comp rx=0 tx=0).stop 2024-04-21T08:39:33.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.579+0000 7fb7b7601740 1 -- 172.21.15.43:0/455684287 shutdown_connections 2024-04-21T08:39:33.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.579+0000 7fb7b7601740 1 --2- 172.21.15.43:0/455684287 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d948688720 0x55d94868ab40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.579+0000 7fb7b7601740 1 --2- 172.21.15.43:0/455684287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d94860e5d0 0x55d948624960 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.579+0000 7fb7b7601740 1 --2- 172.21.15.43:0/455684287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d94860dc20 0x55d948627720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.579+0000 7fb7b7601740 1 --2- 172.21.15.43:0/455684287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d9485fad80 0x55d9486271e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.579+0000 7fb7b7601740 1 -- 172.21.15.43:0/455684287 >> 172.21.15.43:0/455684287 conn(0x55d9485fd220 msgr2=0x55d9485f9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:33.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.579+0000 7fb795ffb640 0 client.16203 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:33.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.579+0000 7fb7b7601740 1 -- 172.21.15.43:0/455684287 shutdown_connections 2024-04-21T08:39:33.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.579+0000 7fb7b7601740 1 -- 172.21.15.43:0/455684287 wait complete. 2024-04-21T08:39:33.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.579+0000 7fb7b7601740 20 client.16203 trim_cache size 0 max 16384 2024-04-21T08:39:33.585 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:39:33.585 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:39:33.585 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-21T08:39:33.585 INFO:teuthology.orchestra.run.smithi043.stdout: "6624c4a7-a3a1-4451-a91a-240e5fd6bb71": { 2024-04-21T08:39:33.585 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-21T08:39:33.585 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "6624c4a7-a3a1-4451-a91a-240e5fd6bb71", 2024-04-21T08:39:33.585 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-21T08:39:33.585 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:39:33.586 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:39:33.586 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:39:33.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.580+0000 7fb7b4a11640 1 -- 172.21.15.43:0/1794556210 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb7980a3e30 msgr2=0x7fb7980a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.580+0000 7fb7b4a11640 1 --2- 172.21.15.43:0/1794556210 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb7980a3e30 0x7fb7980a62f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55d948609400 tx=0x7fb7a000b4d0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.580+0000 7fb7b4a11640 1 -- 172.21.15.43:0/1794556210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7b0103ae0 msgr2=0x7fb7b0198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.580+0000 7fb7b4a11640 1 --2- 172.21.15.43:0/1794556210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7b0103ae0 0x7fb7b0198c70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400e9f0 tx=0x7fb7a400eeb0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.580+0000 7fb7b4a11640 1 -- 172.21.15.43:0/1794556210 shutdown_connections 2024-04-21T08:39:33.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.580+0000 7fb7b4a11640 1 --2- 172.21.15.43:0/1794556210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7b010c500 0x7fb7b019cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.580+0000 7fb7b4a11640 1 --2- 172.21.15.43:0/1794556210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7b0103ae0 0x7fb7b0198c70 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.580+0000 7fb7b4a11640 1 --2- 172.21.15.43:0/1794556210 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb7980a3e30 0x7fb7980a62f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.580+0000 7fb7b4a11640 1 --2- 172.21.15.43:0/1794556210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7b01031c0 0x7fb7b0198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.580+0000 7fb7b4a11640 1 -- 172.21.15.43:0/1794556210 >> 172.21.15.43:0/1794556210 conn(0x7fb7b0078000 msgr2=0x7fb7b00ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:33.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.580+0000 7fb7b4a11640 1 -- 172.21.15.43:0/1794556210 shutdown_connections 2024-04-21T08:39:33.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:39:33.580+0000 7fb7b4a11640 1 -- 172.21.15.43:0/1794556210 wait complete. 2024-04-21T08:39:33.594 INFO:tasks.cephfs.filesystem:scrub status for tag:6624c4a7-a3a1-4451-a91a-240e5fd6bb71 - {'path': '/', 'tag': '6624c4a7-a3a1-4451-a91a-240e5fd6bb71', 'options': 'recursive,force'} 2024-04-21T08:39:34.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:34 smithi112 ceph-mon[31085]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-21T08:39:34.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:34 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:39:34.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:34 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:39:34.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:34 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:39:34.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:34 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:39:34.796 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:34 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:39:34.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:34 smithi043 ceph-mon[27978]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-21T08:39:34.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:34 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:39:34.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:34 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:39:34.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:34 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:39:34.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:34 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:39:34.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:34 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:39:34.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:34 smithi175 ceph-mon[31920]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-21T08:39:34.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:34 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:39:34.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:34 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:39:34.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:34 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:39:34.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:34 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:39:34.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:34 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:39:36.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:36 smithi112 ceph-mon[31085]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-21T08:39:36.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:36 smithi043 ceph-mon[27978]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-21T08:39:36.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:36 smithi175 ceph-mon[31920]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-21T08:39:38.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:38 smithi112 ceph-mon[31085]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-21T08:39:38.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:38 smithi043 ceph-mon[27978]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-21T08:39:38.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:38 smithi175 ceph-mon[31920]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-21T08:39:40.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:40 smithi112 ceph-mon[31085]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-21T08:39:40.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:40 smithi043 ceph-mon[27978]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-21T08:39:40.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:40 smithi175 ceph-mon[31920]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-21T08:39:41.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.513+0000 7fb4a59ce640 1 -- 172.21.15.43:0/2021580376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a0103b00 msgr2=0x7fb4a010bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:41.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.513+0000 7fb4a59ce640 1 --2- 172.21.15.43:0/2021580376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a0103b00 0x7fb4a010bfe0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb494009900 tx=0x7fb49402bd60 comp rx=0 tx=0).stop 2024-04-21T08:39:41.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.513+0000 7fb4a59ce640 1 -- 172.21.15.43:0/2021580376 shutdown_connections 2024-04-21T08:39:41.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.513+0000 7fb4a59ce640 1 --2- 172.21.15.43:0/2021580376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4a010c520 0x7fb4a010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.513+0000 7fb4a59ce640 1 --2- 172.21.15.43:0/2021580376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a0103b00 0x7fb4a010bfe0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.513+0000 7fb4a59ce640 1 --2- 172.21.15.43:0/2021580376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4a01031e0 0x7fb4a01035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.513+0000 7fb4a59ce640 1 -- 172.21.15.43:0/2021580376 >> 172.21.15.43:0/2021580376 conn(0x7fb4a0077fd0 msgr2=0x7fb4a00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.513+0000 7fb4a59ce640 1 -- 172.21.15.43:0/2021580376 shutdown_connections 2024-04-21T08:39:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.513+0000 7fb4a59ce640 1 -- 172.21.15.43:0/2021580376 wait complete. 2024-04-21T08:39:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.514+0000 7fb4a59ce640 1 Processor -- start 2024-04-21T08:39:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.514+0000 7fb4a59ce640 1 -- start start 2024-04-21T08:39:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.514+0000 7fb4a59ce640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4a01031e0 0x7fb4a01a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.514+0000 7fb4a59ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4a0103b00 0x7fb4a01a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.514+0000 7fb4a59ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a010c520 0x7fb4a019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.514+0000 7fb4a59ce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0112190 con 0x7fb4a010c520 2024-04-21T08:39:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.514+0000 7fb4a59ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0112010 con 0x7fb4a0103b00 2024-04-21T08:39:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.514+0000 7fb4a59ce640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0112310 con 0x7fb4a01031e0 2024-04-21T08:39:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.515+0000 7fb49f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a010c520 0x7fb4a019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.515+0000 7fb49f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a010c520 0x7fb4a019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37896/0 (socket says 172.21.15.43:37896) 2024-04-21T08:39:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.515+0000 7fb49f7fe640 1 -- 172.21.15.43:0/180003052 learned_addr learned my addr 172.21.15.43:0/180003052 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.515+0000 7fb49e7fc640 1 --2- 172.21.15.43:0/180003052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4a0103b00 0x7fb4a01a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.515+0000 7fb49effd640 1 --2- 172.21.15.43:0/180003052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4a01031e0 0x7fb4a01a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.515+0000 7fb49f7fe640 1 -- 172.21.15.43:0/180003052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4a01031e0 msgr2=0x7fb4a01a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.515+0000 7fb49f7fe640 1 --2- 172.21.15.43:0/180003052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4a01031e0 0x7fb4a01a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.515+0000 7fb49f7fe640 1 -- 172.21.15.43:0/180003052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4a0103b00 msgr2=0x7fb4a01a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.515+0000 7fb49f7fe640 1 --2- 172.21.15.43:0/180003052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4a0103b00 0x7fb4a01a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.515+0000 7fb49f7fe640 1 -- 172.21.15.43:0/180003052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a019af10 con 0x7fb4a010c520 2024-04-21T08:39:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.515+0000 7fb49e7fc640 1 --2- 172.21.15.43:0/180003052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4a0103b00 0x7fb4a01a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.515+0000 7fb49effd640 1 --2- 172.21.15.43:0/180003052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4a01031e0 0x7fb4a01a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.515+0000 7fb49f7fe640 1 --2- 172.21.15.43:0/180003052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a010c520 0x7fb4a019a650 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fb49800cce0 tx=0x7fb49800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:41.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.515+0000 7fb4a49cc640 1 -- 172.21.15.43:0/180003052 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49800f450 con 0x7fb4a010c520 2024-04-21T08:39:41.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.516+0000 7fb4a59ce640 1 -- 172.21.15.43:0/180003052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a019b200 con 0x7fb4a010c520 2024-04-21T08:39:41.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.516+0000 7fb4a49cc640 1 -- 172.21.15.43:0/180003052 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb49800faa0 con 0x7fb4a010c520 2024-04-21T08:39:41.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.516+0000 7fb4a49cc640 1 -- 172.21.15.43:0/180003052 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498018b70 con 0x7fb4a010c520 2024-04-21T08:39:41.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.516+0000 7fb4a59ce640 1 -- 172.21.15.43:0/180003052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a01a7510 con 0x7fb4a010c520 2024-04-21T08:39:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.519+0000 7fb4a85be740 1 Processor -- start 2024-04-21T08:39:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.519+0000 7fb4a49cc640 1 -- 172.21.15.43:0/180003052 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fb498018d10 con 0x7fb4a010c520 2024-04-21T08:39:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.519+0000 7fb4a85be740 1 -- start start 2024-04-21T08:39:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.519+0000 7fb4a85be740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04e8d0 0x55f07d04d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.519+0000 7fb4a85be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f07d04db60 0x55f07d04dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.519+0000 7fb4a85be740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f07d0570e0 0x55f07d0594d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.519+0000 7fb4a85be740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f07cfb28d0 con 0x55f07d04e8d0 2024-04-21T08:39:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.519+0000 7fb4a85be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f07cf4fe80 con 0x55f07d04db60 2024-04-21T08:39:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.519+0000 7fb4a85be740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f07cfae890 con 0x55f07d0570e0 2024-04-21T08:39:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.520+0000 7fb49e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04e8d0 0x55f07d04d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.520+0000 7fb49e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04e8d0 0x55f07d04d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37902/0 (socket says 172.21.15.43:37902) 2024-04-21T08:39:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.520+0000 7fb49e7fc640 1 -- 172.21.15.43:0/710372906 learned_addr learned my addr 172.21.15.43:0/710372906 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.520+0000 7fb49effd640 1 --2- 172.21.15.43:0/710372906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f07d04db60 0x55f07d04dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.520+0000 7fb4a49cc640 1 --2- 172.21.15.43:0/180003052 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb4880a3ee0 0x7fb4880a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.520+0000 7fb49e7fc640 1 --2- 172.21.15.43:0/710372906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f07d0570e0 0x55f07d0594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.520+0000 7fb49e7fc640 1 -- 172.21.15.43:0/710372906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f07d0570e0 msgr2=0x55f07d0594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.520+0000 7fb49e7fc640 1 --2- 172.21.15.43:0/710372906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f07d0570e0 0x55f07d0594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.520+0000 7fb49f7fe640 1 --2- 172.21.15.43:0/180003052 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb4880a3ee0 0x7fb4880a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.520+0000 7fb49e7fc640 1 -- 172.21.15.43:0/710372906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f07d04db60 msgr2=0x55f07d04dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.520+0000 7fb49e7fc640 1 --2- 172.21.15.43:0/710372906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f07d04db60 0x55f07d04dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.520+0000 7fb49e7fc640 1 -- 172.21.15.43:0/710372906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f07cf71320 con 0x55f07d04e8d0 2024-04-21T08:39:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.520+0000 7fb4a49cc640 1 -- 172.21.15.43:0/180003052 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fb4980d10e0 con 0x7fb4a010c520 2024-04-21T08:39:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.520+0000 7fb49e7fc640 1 --2- 172.21.15.43:0/710372906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04e8d0 0x55f07d04d620 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb494009900 tx=0x7fb494009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23d6865522e8fcd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.521+0000 7fb485ffb640 1 -- 172.21.15.43:0/710372906 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb494049070 con 0x55f07d04e8d0 2024-04-21T08:39:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.521+0000 7fb485ffb640 1 -- 172.21.15.43:0/710372906 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4940044b0 con 0x55f07d04e8d0 2024-04-21T08:39:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.521+0000 7fb485ffb640 1 -- 172.21.15.43:0/710372906 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb494003e30 con 0x55f07d04e8d0 2024-04-21T08:39:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.522+0000 7fb4a85be740 1 -- 172.21.15.43:0/710372906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04e8d0 msgr2=0x55f07d04d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.522+0000 7fb4a85be740 1 --2- 172.21.15.43:0/710372906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04e8d0 0x55f07d04d620 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb494009900 tx=0x7fb494009560 comp rx=0 tx=0).stop 2024-04-21T08:39:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.522+0000 7fb49f7fe640 1 --2- 172.21.15.43:0/180003052 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb4880a3ee0 0x7fb4880a63a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb4980ce400 tx=0x7fb498091fd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.522+0000 7fb4a85be740 1 -- 172.21.15.43:0/710372906 shutdown_connections 2024-04-21T08:39:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.522+0000 7fb4a85be740 1 --2- 172.21.15.43:0/710372906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f07d0570e0 0x55f07d0594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.522+0000 7fb4a85be740 1 --2- 172.21.15.43:0/710372906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f07d04db60 0x55f07d04dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.522+0000 7fb4a85be740 1 --2- 172.21.15.43:0/710372906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04e8d0 0x55f07d04d620 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.522+0000 7fb4a85be740 1 -- 172.21.15.43:0/710372906 >> 172.21.15.43:0/710372906 conn(0x55f07d014c90 msgr2=0x55f07d048850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.522+0000 7fb4a85be740 1 -- 172.21.15.43:0/710372906 shutdown_connections 2024-04-21T08:39:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.522+0000 7fb4a85be740 1 -- 172.21.15.43:0/710372906 wait complete. 2024-04-21T08:39:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.522+0000 7fb4a85be740 1 -- start start 2024-04-21T08:39:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.522+0000 7fb4a85be740 1 Processor -- start 2024-04-21T08:39:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.523+0000 7fb4a85be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f07d04db60 0x55f07d056ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.523+0000 7fb4a85be740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04e8d0 0x55f07d053e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.523+0000 7fb4a85be740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f07d0570e0 0x55f07d054460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.523+0000 7fb4a85be740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f07d059db0 con 0x55f07d04e8d0 2024-04-21T08:39:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.523+0000 7fb4a85be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f07cf3aeb0 con 0x55f07d04db60 2024-04-21T08:39:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.523+0000 7fb4a85be740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f07d059f30 con 0x55f07d0570e0 2024-04-21T08:39:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.523+0000 7fb4a85be740 10 client.0 _ll_register_callbacks cb 0x55f07cf67b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.523+0000 7fb49effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04e8d0 0x55f07d053e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.523+0000 7fb49effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04e8d0 0x55f07d053e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37906/0 (socket says 172.21.15.43:37906) 2024-04-21T08:39:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.523+0000 7fb49effd640 1 -- 172.21.15.43:0/718123250 learned_addr learned my addr 172.21.15.43:0/718123250 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.523+0000 7fb49e7fc640 1 --2- 172.21.15.43:0/718123250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f07d0570e0 0x55f07d054460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.523+0000 7fb49e7fc640 1 --2- 172.21.15.43:0/718123250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f07d04db60 0x55f07d056ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.523+0000 7fb49effd640 1 -- 172.21.15.43:0/718123250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f07d0570e0 msgr2=0x55f07d054460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.523+0000 7fb49effd640 1 --2- 172.21.15.43:0/718123250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f07d0570e0 0x55f07d054460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.523+0000 7fb49effd640 1 -- 172.21.15.43:0/718123250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f07d04db60 msgr2=0x55f07d056ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.523+0000 7fb49effd640 1 --2- 172.21.15.43:0/718123250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f07d04db60 0x55f07d056ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.523+0000 7fb49effd640 1 -- 172.21.15.43:0/718123250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f07ccc5210 con 0x55f07d04e8d0 2024-04-21T08:39:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.524+0000 7fb49effd640 1 --2- 172.21.15.43:0/718123250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04e8d0 0x55f07d053e50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb490013710 tx=0x7fb490013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2e22d1752a04ea server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.524+0000 7fb47effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.524+0000 7fb47effd640 1 -- 172.21.15.43:0/718123250 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490011d50 con 0x55f07d04e8d0 2024-04-21T08:39:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.524+0000 7fb47effd640 1 -- 172.21.15.43:0/718123250 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fb49001b3e0 con 0x55f07d04e8d0 2024-04-21T08:39:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.524+0000 7fb4a85be740 1 -- 172.21.15.43:0/718123250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4940044b0 con 0x55f07d04e8d0 2024-04-21T08:39:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.524+0000 7fb47effd640 1 -- 172.21.15.43:0/718123250 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb49001b5e0 con 0x55f07d04e8d0 2024-04-21T08:39:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.524+0000 7fb4a85be740 10 client.16215 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.524+0000 7fb4a85be740 1 -- 172.21.15.43:0/718123250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f07cd91620 con 0x55f07d04e8d0 2024-04-21T08:39:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.524+0000 7fb47effd640 1 -- 172.21.15.43:0/718123250 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4940044b0 con 0x55f07d04e8d0 2024-04-21T08:39:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.525+0000 7fb47effd640 1 -- 172.21.15.43:0/718123250 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb49001a070 con 0x55f07d04e8d0 2024-04-21T08:39:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.525+0000 7fb4a85be740 10 client.16215 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.525+0000 7fb4a85be740 10 client.16215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.525+0000 7fb4a85be740 20 client.16215 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.525+0000 7fb4a85be740 1 --2- 172.21.15.43:0/718123250 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f07d067160 0x55f07d069590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.525+0000 7fb4a85be740 4 client.16215 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.525+0000 7fb4a85be740 1 -- 172.21.15.43:0/718123250 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f07cfc6880 con 0x55f07d067160 2024-04-21T08:39:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.525+0000 7fb49e7fc640 1 --2- 172.21.15.43:0/718123250 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f07d067160 0x55f07d069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.525+0000 7fb49e7fc640 1 --2- 172.21.15.43:0/718123250 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f07d067160 0x55f07d069590 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8cee16a292920fc server_cookie=5f46780119c8dd43 in_seq=0 out_seq=0 2024-04-21T08:39:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.525+0000 7fb47effd640 10 client.16215 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.526+0000 7fb47effd640 1 -- 172.21.15.43:0/718123250 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f07cfc6880 con 0x55f07d067160 2024-04-21T08:39:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.526+0000 7fb47effd640 10 client.16215 handle_command_reply: tid=0 2024-04-21T08:39:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.526+0000 7fb4a85be740 1 client.16215 shutdown 2024-04-21T08:39:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.526+0000 7fb4a85be740 1 -- 172.21.15.43:0/718123250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04e8d0 msgr2=0x55f07d053e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.526+0000 7fb4a85be740 1 --2- 172.21.15.43:0/718123250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04e8d0 0x55f07d053e50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb490013710 tx=0x7fb490013bd0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.526+0000 7fb4a85be740 1 -- 172.21.15.43:0/718123250 shutdown_connections 2024-04-21T08:39:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.526+0000 7fb4a85be740 1 --2- 172.21.15.43:0/718123250 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f07d067160 0x55f07d069590 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.526+0000 7fb4a85be740 1 --2- 172.21.15.43:0/718123250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f07d0570e0 0x55f07d054460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.526+0000 7fb4a85be740 1 --2- 172.21.15.43:0/718123250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04e8d0 0x55f07d053e50 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.526+0000 7fb47effd640 0 client.16215 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.526+0000 7fb4a85be740 1 --2- 172.21.15.43:0/718123250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f07d04db60 0x55f07d056ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.526+0000 7fb4a85be740 1 -- 172.21.15.43:0/718123250 >> 172.21.15.43:0/718123250 conn(0x55f07d014c90 msgr2=0x55f07d04a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.527+0000 7fb4a85be740 1 -- 172.21.15.43:0/718123250 shutdown_connections 2024-04-21T08:39:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.527+0000 7fb4a85be740 1 -- 172.21.15.43:0/718123250 wait complete. 2024-04-21T08:39:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.527+0000 7fb4a85be740 20 client.16215 trim_cache size 0 max 16384 2024-04-21T08:39:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.542+0000 7fb4a85be740 1 Processor -- start 2024-04-21T08:39:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.542+0000 7fb4a85be740 1 -- start start 2024-04-21T08:39:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.542+0000 7fb4a85be740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04db60 0x55f07d062ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.542+0000 7fb4a85be740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f07d04e8d0 0x55f07d052e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.542+0000 7fb4a85be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f07d0570e0 0x55f07d056580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:41.550 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.542+0000 7fb4a85be740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01b13d0 con 0x55f07d04db60 2024-04-21T08:39:41.550 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.542+0000 7fb4a85be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01b2f10 con 0x55f07d0570e0 2024-04-21T08:39:41.550 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.542+0000 7fb4a85be740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01b1aa0 con 0x55f07d04e8d0 2024-04-21T08:39:41.550 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.543+0000 7fb49e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04db60 0x55f07d062ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:41.550 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.543+0000 7fb49e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04db60 0x55f07d062ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37920/0 (socket says 172.21.15.43:37920) 2024-04-21T08:39:41.550 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.543+0000 7fb49e7fc640 1 -- 172.21.15.43:0/4110214528 learned_addr learned my addr 172.21.15.43:0/4110214528 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:41.550 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.543+0000 7fb49effd640 1 --2- 172.21.15.43:0/4110214528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f07d04e8d0 0x55f07d052e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.543+0000 7fb49e7fc640 1 -- 172.21.15.43:0/4110214528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f07d04e8d0 msgr2=0x55f07d052e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.543+0000 7fb49e7fc640 1 --2- 172.21.15.43:0/4110214528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f07d04e8d0 0x55f07d052e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.543+0000 7fb49e7fc640 1 -- 172.21.15.43:0/4110214528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f07d0570e0 msgr2=0x55f07d056580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.543+0000 7fb49e7fc640 1 --2- 172.21.15.43:0/4110214528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f07d0570e0 0x55f07d056580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.543+0000 7fb49e7fc640 1 -- 172.21.15.43:0/4110214528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f07d04ce30 con 0x55f07d04db60 2024-04-21T08:39:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.543+0000 7fb49e7fc640 1 --2- 172.21.15.43:0/4110214528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04db60 0x55f07d062ce0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb494009900 tx=0x7fb494003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43221578d8d829ca server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.543+0000 7fb4857fa640 1 -- 172.21.15.43:0/4110214528 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb494049070 con 0x55f07d04db60 2024-04-21T08:39:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.543+0000 7fb4857fa640 1 -- 172.21.15.43:0/4110214528 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb49404a040 con 0x55f07d04db60 2024-04-21T08:39:41.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.543+0000 7fb4857fa640 1 -- 172.21.15.43:0/4110214528 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb494042760 con 0x55f07d04db60 2024-04-21T08:39:41.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.544+0000 7fb4a85be740 1 -- 172.21.15.43:0/4110214528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04db60 msgr2=0x55f07d062ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:41.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.544+0000 7fb4a85be740 1 --2- 172.21.15.43:0/4110214528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04db60 0x55f07d062ce0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb494009900 tx=0x7fb494003a10 comp rx=0 tx=0).stop 2024-04-21T08:39:41.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.544+0000 7fb4a85be740 1 -- 172.21.15.43:0/4110214528 shutdown_connections 2024-04-21T08:39:41.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.544+0000 7fb4a85be740 1 --2- 172.21.15.43:0/4110214528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f07d0570e0 0x55f07d056580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.544+0000 7fb4a85be740 1 --2- 172.21.15.43:0/4110214528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f07d04e8d0 0x55f07d052e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.544+0000 7fb4a85be740 1 --2- 172.21.15.43:0/4110214528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04db60 0x55f07d062ce0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.544+0000 7fb4a85be740 1 -- 172.21.15.43:0/4110214528 >> 172.21.15.43:0/4110214528 conn(0x55f07d014c90 msgr2=0x55f07d052740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:41.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.544+0000 7fb4a85be740 1 -- 172.21.15.43:0/4110214528 shutdown_connections 2024-04-21T08:39:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.544+0000 7fb4a85be740 1 -- 172.21.15.43:0/4110214528 wait complete. 2024-04-21T08:39:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.545+0000 7fb4a85be740 1 -- start start 2024-04-21T08:39:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.545+0000 7fb4a85be740 1 Processor -- start 2024-04-21T08:39:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.545+0000 7fb4a85be740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04db60 0x55f07d05f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.545+0000 7fb4a85be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f07d04e8d0 0x55f07d0507a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.545+0000 7fb4a85be740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f07d0570e0 0x55f07d050ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.545+0000 7fb4a85be740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01b2840 con 0x55f07d04db60 2024-04-21T08:39:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.545+0000 7fb4a85be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01b2170 con 0x55f07d04e8d0 2024-04-21T08:39:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.545+0000 7fb4a85be740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01af3b0 con 0x55f07d0570e0 2024-04-21T08:39:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.545+0000 7fb4a85be740 10 client.0 _ll_register_callbacks cb 0x7fb490009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.545+0000 7fb49e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04db60 0x55f07d05f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.545+0000 7fb49e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04db60 0x55f07d05f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37936/0 (socket says 172.21.15.43:37936) 2024-04-21T08:39:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.545+0000 7fb49e7fc640 1 -- 172.21.15.43:0/877221043 learned_addr learned my addr 172.21.15.43:0/877221043 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.545+0000 7fb49effd640 1 --2- 172.21.15.43:0/877221043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f07d04e8d0 0x55f07d0507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.546+0000 7fb49e7fc640 1 --2- 172.21.15.43:0/877221043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f07d0570e0 0x55f07d050ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.546+0000 7fb49e7fc640 1 -- 172.21.15.43:0/877221043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f07d0570e0 msgr2=0x55f07d050ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.546+0000 7fb49e7fc640 1 --2- 172.21.15.43:0/877221043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f07d0570e0 0x55f07d050ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.546+0000 7fb49e7fc640 1 -- 172.21.15.43:0/877221043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f07d04e8d0 msgr2=0x55f07d0507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.546+0000 7fb49e7fc640 1 --2- 172.21.15.43:0/877221043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f07d04e8d0 0x55f07d0507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.546+0000 7fb49e7fc640 1 -- 172.21.15.43:0/877221043 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f07d0513a0 con 0x55f07d04db60 2024-04-21T08:39:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.546+0000 7fb49e7fc640 1 --2- 172.21.15.43:0/877221043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04db60 0x55f07d05f4c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb494004650 tx=0x7fb494004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ba24acb0958a3d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.546+0000 7fb47effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.546+0000 7fb47effd640 1 -- 172.21.15.43:0/877221043 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb494049030 con 0x55f07d04db60 2024-04-21T08:39:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.546+0000 7fb47effd640 1 -- 172.21.15.43:0/877221043 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fb49402c910 con 0x55f07d04db60 2024-04-21T08:39:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.546+0000 7fb4a85be740 1 -- 172.21.15.43:0/877221043 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb49404a040 con 0x55f07d04db60 2024-04-21T08:39:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.546+0000 7fb47effd640 1 -- 172.21.15.43:0/877221043 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb49402cb10 con 0x55f07d04db60 2024-04-21T08:39:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.546+0000 7fb4a85be740 10 client.16221 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.546+0000 7fb4a85be740 1 -- 172.21.15.43:0/877221043 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f07d0cd5d0 con 0x55f07d04db60 2024-04-21T08:39:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.547+0000 7fb47effd640 1 -- 172.21.15.43:0/877221043 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb49404a040 con 0x55f07d04db60 2024-04-21T08:39:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.547+0000 7fb47effd640 1 -- 172.21.15.43:0/877221043 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb4940426a0 con 0x55f07d04db60 2024-04-21T08:39:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.547+0000 7fb4a85be740 10 client.16221 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.547+0000 7fb4a85be740 10 client.16221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.547+0000 7fb4a85be740 20 client.16221 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.547+0000 7fb4a85be740 1 --2- 172.21.15.43:0/877221043 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f07d0cd800 0x55f07d0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.547+0000 7fb4a85be740 4 client.16221 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:39:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.547+0000 7fb4a85be740 1 -- 172.21.15.43:0/877221043 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f07cf523d0 con 0x55f07d0cd800 2024-04-21T08:39:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.547+0000 7fb49effd640 1 --2- 172.21.15.43:0/877221043 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f07d0cd800 0x55f07d0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.548+0000 7fb47effd640 10 client.16221 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.548+0000 7fb49effd640 1 --2- 172.21.15.43:0/877221043 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f07d0cd800 0x55f07d0cfbf0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eb8dbaa99d9789a server_cookie=1b01639a045ef5de in_seq=0 out_seq=0 2024-04-21T08:39:41.556 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.551+0000 7fb47effd640 1 -- 172.21.15.43:0/877221043 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f07cf523d0 con 0x55f07d0cd800 2024-04-21T08:39:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.551+0000 7fb47effd640 10 client.16221 handle_command_reply: tid=0 2024-04-21T08:39:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.551+0000 7fb4a85be740 1 client.16221 shutdown 2024-04-21T08:39:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.551+0000 7fb4a85be740 1 -- 172.21.15.43:0/877221043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04db60 msgr2=0x55f07d05f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.551+0000 7fb4a85be740 1 --2- 172.21.15.43:0/877221043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04db60 0x55f07d05f4c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb494004650 tx=0x7fb494004550 comp rx=0 tx=0).stop 2024-04-21T08:39:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.552+0000 7fb4a85be740 1 -- 172.21.15.43:0/877221043 shutdown_connections 2024-04-21T08:39:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.552+0000 7fb4a85be740 1 --2- 172.21.15.43:0/877221043 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f07d0cd800 0x55f07d0cfbf0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.552+0000 7fb4a85be740 1 --2- 172.21.15.43:0/877221043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f07d0570e0 0x55f07d050ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.552+0000 7fb4a85be740 1 --2- 172.21.15.43:0/877221043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f07d04e8d0 0x55f07d0507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.552+0000 7fb47effd640 0 client.16221 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.552+0000 7fb4a85be740 1 --2- 172.21.15.43:0/877221043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f07d04db60 0x55f07d05f4c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.552+0000 7fb4a85be740 1 -- 172.21.15.43:0/877221043 >> 172.21.15.43:0/877221043 conn(0x55f07d014c90 msgr2=0x55f07d049c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.552+0000 7fb4a85be740 1 -- 172.21.15.43:0/877221043 shutdown_connections 2024-04-21T08:39:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.552+0000 7fb4a85be740 1 -- 172.21.15.43:0/877221043 wait complete. 2024-04-21T08:39:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.552+0000 7fb4a85be740 20 client.16221 trim_cache size 0 max 16384 2024-04-21T08:39:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:39:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:39:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:39:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:39:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.552+0000 7fb4a59ce640 1 -- 172.21.15.43:0/180003052 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb4880a3ee0 msgr2=0x7fb4880a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.552+0000 7fb4a59ce640 1 --2- 172.21.15.43:0/180003052 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb4880a3ee0 0x7fb4880a63a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb4980ce400 tx=0x7fb498091fd0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.552+0000 7fb4a59ce640 1 -- 172.21.15.43:0/180003052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a010c520 msgr2=0x7fb4a019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.552+0000 7fb4a59ce640 1 --2- 172.21.15.43:0/180003052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a010c520 0x7fb4a019a650 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fb49800cce0 tx=0x7fb49800a620 comp rx=0 tx=0).stop 2024-04-21T08:39:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.552+0000 7fb4a59ce640 1 -- 172.21.15.43:0/180003052 shutdown_connections 2024-04-21T08:39:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.552+0000 7fb4a59ce640 1 --2- 172.21.15.43:0/180003052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a010c520 0x7fb4a019a650 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.552+0000 7fb4a59ce640 1 --2- 172.21.15.43:0/180003052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4a0103b00 0x7fb4a01a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.552+0000 7fb4a59ce640 1 --2- 172.21.15.43:0/180003052 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb4880a3ee0 0x7fb4880a63a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.552+0000 7fb4a59ce640 1 --2- 172.21.15.43:0/180003052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4a01031e0 0x7fb4a01a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.552+0000 7fb4a59ce640 1 -- 172.21.15.43:0/180003052 >> 172.21.15.43:0/180003052 conn(0x7fb4a0077fd0 msgr2=0x7fb4a00fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.553+0000 7fb4a59ce640 1 -- 172.21.15.43:0/180003052 shutdown_connections 2024-04-21T08:39:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:41.553+0000 7fb4a59ce640 1 -- 172.21.15.43:0/180003052 wait complete. 2024-04-21T08:39:42.793 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:42 smithi112 ceph-mon[31085]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-21T08:39:42.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:42 smithi043 ceph-mon[27978]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-21T08:39:42.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:42 smithi175 ceph-mon[31920]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-21T08:39:44.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:44 smithi112 ceph-mon[31085]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-21T08:39:44.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:44 smithi043 ceph-mon[27978]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-21T08:39:44.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:44 smithi175 ceph-mon[31920]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-21T08:39:46.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:46 smithi112 ceph-mon[31085]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-21T08:39:46.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:46 smithi043 ceph-mon[27978]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-21T08:39:46.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:46 smithi175 ceph-mon[31920]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-21T08:39:48.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:48 smithi112 ceph-mon[31085]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-21T08:39:48.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:48 smithi043 ceph-mon[27978]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-21T08:39:48.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:48 smithi175 ceph-mon[31920]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-21T08:39:49.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:49 smithi043 ceph-mon[27978]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-21T08:39:49.963 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:49 smithi175 ceph-mon[31920]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-21T08:39:50.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:49 smithi112 ceph-mon[31085]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-21T08:39:52.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:51 smithi112 ceph-mon[31085]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-04-21T08:39:52.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:51 smithi043 ceph-mon[27978]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-04-21T08:39:52.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:51 smithi175 ceph-mon[31920]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-04-21T08:39:54.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:53 smithi112 ceph-mon[31085]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-21T08:39:54.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:53 smithi043 ceph-mon[27978]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-21T08:39:54.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:53 smithi175 ceph-mon[31920]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-21T08:39:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.648+0000 7f3001ba4640 1 -- 172.21.15.43:0/4163405014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ffc10c520 msgr2=0x7f2ffc10e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.648+0000 7f3001ba4640 1 --2- 172.21.15.43:0/4163405014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ffc10c520 0x7f2ffc10e9c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009ca0 tx=0x7f2ff402c230 comp rx=0 tx=0).stop 2024-04-21T08:39:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.649+0000 7f3001ba4640 1 -- 172.21.15.43:0/4163405014 shutdown_connections 2024-04-21T08:39:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.649+0000 7f3001ba4640 1 --2- 172.21.15.43:0/4163405014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ffc10c520 0x7f2ffc10e9c0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.649+0000 7f3001ba4640 1 --2- 172.21.15.43:0/4163405014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ffc103b00 0x7f2ffc10bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.649+0000 7f3001ba4640 1 --2- 172.21.15.43:0/4163405014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc1031e0 0x7f2ffc1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.649+0000 7f3001ba4640 1 -- 172.21.15.43:0/4163405014 >> 172.21.15.43:0/4163405014 conn(0x7f2ffc077fd0 msgr2=0x7f2ffc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.649+0000 7f3001ba4640 1 -- 172.21.15.43:0/4163405014 shutdown_connections 2024-04-21T08:39:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.649+0000 7f3001ba4640 1 -- 172.21.15.43:0/4163405014 wait complete. 2024-04-21T08:39:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.649+0000 7f3001ba4640 1 Processor -- start 2024-04-21T08:39:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.649+0000 7f3001ba4640 1 -- start start 2024-04-21T08:39:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.650+0000 7f3001ba4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ffc1031e0 0x7f2ffc198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.650+0000 7f3001ba4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc103b00 0x7f2ffc198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.650+0000 7f3001ba4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ffc10c520 0x7f2ffc19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.650+0000 7f3001ba4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc112190 con 0x7f2ffc1031e0 2024-04-21T08:39:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.650+0000 7f3001ba4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc112010 con 0x7f2ffc103b00 2024-04-21T08:39:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.650+0000 7f3001ba4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc112310 con 0x7f2ffc10c520 2024-04-21T08:39:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.650+0000 7f2ffb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ffc1031e0 0x7f2ffc198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.650+0000 7f2ffb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ffc1031e0 0x7f2ffc198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47994/0 (socket says 172.21.15.43:47994) 2024-04-21T08:39:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.650+0000 7f2ffb7fe640 1 -- 172.21.15.43:0/4267145037 learned_addr learned my addr 172.21.15.43:0/4267145037 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.650+0000 7f2ffaffd640 1 --2- 172.21.15.43:0/4267145037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc103b00 0x7f2ffc198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.650+0000 7f2ffbfff640 1 --2- 172.21.15.43:0/4267145037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ffc10c520 0x7f2ffc19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.650+0000 7f2ffb7fe640 1 -- 172.21.15.43:0/4267145037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ffc10c520 msgr2=0x7f2ffc19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.650+0000 7f2ffb7fe640 1 --2- 172.21.15.43:0/4267145037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ffc10c520 0x7f2ffc19cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.650+0000 7f2ffb7fe640 1 -- 172.21.15.43:0/4267145037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc103b00 msgr2=0x7f2ffc198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.650+0000 7f2ffb7fe640 1 --2- 172.21.15.43:0/4267145037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc103b00 0x7f2ffc198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.650+0000 7f2ffb7fe640 1 -- 172.21.15.43:0/4267145037 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ffc19d640 con 0x7f2ffc1031e0 2024-04-21T08:39:54.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.650+0000 7f2ffaffd640 1 --2- 172.21.15.43:0/4267145037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc103b00 0x7f2ffc198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:54.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.650+0000 7f2ffb7fe640 1 --2- 172.21.15.43:0/4267145037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ffc1031e0 0x7f2ffc198650 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00e9f0 tx=0x7f2fec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:54.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.650+0000 7f2ff8ff9640 1 -- 172.21.15.43:0/4267145037 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec00cd80 con 0x7f2ffc1031e0 2024-04-21T08:39:54.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.650+0000 7f3001ba4640 1 -- 172.21.15.43:0/4267145037 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ffc19d930 con 0x7f2ffc1031e0 2024-04-21T08:39:54.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.650+0000 7f2ff8ff9640 1 -- 172.21.15.43:0/4267145037 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fec0108e0 con 0x7f2ffc1031e0 2024-04-21T08:39:54.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.650+0000 7f2ff8ff9640 1 -- 172.21.15.43:0/4267145037 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec018980 con 0x7f2ffc1031e0 2024-04-21T08:39:54.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.650+0000 7f3001ba4640 1 -- 172.21.15.43:0/4267145037 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ffc1a51c0 con 0x7f2ffc1031e0 2024-04-21T08:39:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.653+0000 7f2ff8ff9640 1 -- 172.21.15.43:0/4267145037 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f2fec010430 con 0x7f2ffc1031e0 2024-04-21T08:39:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.653+0000 7f3004798740 1 Processor -- start 2024-04-21T08:39:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.653+0000 7f3004798740 1 -- start start 2024-04-21T08:39:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.654+0000 7f3004798740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a94588d0 0x5619a9457620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.654+0000 7f3004798740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a9457b60 0x5619a9457fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.654+0000 7f3004798740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619a94610e0 0x5619a94634d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.654+0000 7f3004798740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5619a93bc8d0 con 0x5619a9457b60 2024-04-21T08:39:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.654+0000 7f3004798740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5619a9359e80 con 0x5619a94588d0 2024-04-21T08:39:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.654+0000 7f3004798740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5619a93b8890 con 0x5619a94610e0 2024-04-21T08:39:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.654+0000 7f2ff8ff9640 1 --2- 172.21.15.43:0/4267145037 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2fe40a3e60 0x7f2fe40a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.654+0000 7f2ffb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a9457b60 0x5619a9457fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.654+0000 7f2ffb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a9457b60 0x5619a9457fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48000/0 (socket says 172.21.15.43:48000) 2024-04-21T08:39:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.654+0000 7f2ffb7fe640 1 -- 172.21.15.43:0/3286748809 learned_addr learned my addr 172.21.15.43:0/3286748809 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.654+0000 7f2ffbfff640 1 --2- 172.21.15.43:0/3286748809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a94588d0 0x5619a9457620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.654+0000 7f2ffbfff640 1 --2- 172.21.15.43:0/4267145037 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2fe40a3e60 0x7f2fe40a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.654+0000 7f2ffaffd640 1 --2- 172.21.15.43:0/3286748809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619a94610e0 0x5619a94634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.654+0000 7f2ff8ff9640 1 -- 172.21.15.43:0/4267145037 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f2fec0d11b0 con 0x7f2ffc1031e0 2024-04-21T08:39:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.654+0000 7f2ffb7fe640 1 -- 172.21.15.43:0/3286748809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619a94610e0 msgr2=0x5619a94634d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.654+0000 7f2ffb7fe640 1 --2- 172.21.15.43:0/3286748809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619a94610e0 0x5619a94634d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.654+0000 7f2ffb7fe640 1 -- 172.21.15.43:0/3286748809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a94588d0 msgr2=0x5619a9457620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.654+0000 7f2ffb7fe640 1 --2- 172.21.15.43:0/3286748809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a94588d0 0x5619a9457620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.654+0000 7f2ffb7fe640 1 -- 172.21.15.43:0/3286748809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619a937b320 con 0x5619a9457b60 2024-04-21T08:39:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.654+0000 7f2ffbfff640 1 --2- 172.21.15.43:0/3286748809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a94588d0 0x5619a9457620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.654+0000 7f2ffb7fe640 1 --2- 172.21.15.43:0/3286748809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a9457b60 0x5619a9457fe0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2fec0ce3f0 tx=0x7f2fec01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb0b27a746aa5d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.655+0000 7f2ffbfff640 1 --2- 172.21.15.43:0/4267145037 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2fe40a3e60 0x7f2fe40a6320 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2ff40325e0 tx=0x7f2ff40095c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.655+0000 7f2fe1ffb640 1 -- 172.21.15.43:0/3286748809 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec026040 con 0x5619a9457b60 2024-04-21T08:39:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.655+0000 7f2fe1ffb640 1 -- 172.21.15.43:0/3286748809 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fec0a0fa0 con 0x5619a9457b60 2024-04-21T08:39:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.655+0000 7f2fe1ffb640 1 -- 172.21.15.43:0/3286748809 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec0a53d0 con 0x5619a9457b60 2024-04-21T08:39:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.655+0000 7f3004798740 1 -- 172.21.15.43:0/3286748809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a9457b60 msgr2=0x5619a9457fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.655+0000 7f3004798740 1 --2- 172.21.15.43:0/3286748809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a9457b60 0x5619a9457fe0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2fec0ce3f0 tx=0x7f2fec01f000 comp rx=0 tx=0).stop 2024-04-21T08:39:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.656+0000 7f3004798740 1 -- 172.21.15.43:0/3286748809 shutdown_connections 2024-04-21T08:39:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.656+0000 7f3004798740 1 --2- 172.21.15.43:0/3286748809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619a94610e0 0x5619a94634d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.656+0000 7f3004798740 1 --2- 172.21.15.43:0/3286748809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a9457b60 0x5619a9457fe0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.656+0000 7f3004798740 1 --2- 172.21.15.43:0/3286748809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a94588d0 0x5619a9457620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.656+0000 7f3004798740 1 -- 172.21.15.43:0/3286748809 >> 172.21.15.43:0/3286748809 conn(0x5619a941ec90 msgr2=0x5619a9452850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.656+0000 7f3004798740 1 -- 172.21.15.43:0/3286748809 shutdown_connections 2024-04-21T08:39:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.656+0000 7f3004798740 1 -- 172.21.15.43:0/3286748809 wait complete. 2024-04-21T08:39:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.656+0000 7f3004798740 1 -- start start 2024-04-21T08:39:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.656+0000 7f3004798740 1 Processor -- start 2024-04-21T08:39:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.657+0000 7f3004798740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a9457b60 0x5619a9460aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.657+0000 7f3004798740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a94588d0 0x5619a945de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.657+0000 7f3004798740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619a94610e0 0x5619a945e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.657+0000 7f3004798740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5619a9463db0 con 0x5619a94588d0 2024-04-21T08:39:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.657+0000 7f3004798740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5619a9344eb0 con 0x5619a9457b60 2024-04-21T08:39:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.657+0000 7f3004798740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5619a9463f30 con 0x5619a94610e0 2024-04-21T08:39:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.657+0000 7f3004798740 10 client.0 _ll_register_callbacks cb 0x5619a9371b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.657+0000 7f2ffaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a94588d0 0x5619a945de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.657+0000 7f2ffaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a94588d0 0x5619a945de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48004/0 (socket says 172.21.15.43:48004) 2024-04-21T08:39:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.657+0000 7f2ffaffd640 1 -- 172.21.15.43:0/3289452666 learned_addr learned my addr 172.21.15.43:0/3289452666 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.657+0000 7f2ffb7fe640 1 --2- 172.21.15.43:0/3289452666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a9457b60 0x5619a9460aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.657+0000 7f2ffbfff640 1 --2- 172.21.15.43:0/3289452666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619a94610e0 0x5619a945e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.657+0000 7f2ffaffd640 1 -- 172.21.15.43:0/3289452666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619a94610e0 msgr2=0x5619a945e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.657+0000 7f2ffaffd640 1 --2- 172.21.15.43:0/3289452666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619a94610e0 0x5619a945e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.657+0000 7f2ffaffd640 1 -- 172.21.15.43:0/3289452666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a9457b60 msgr2=0x5619a9460aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.657+0000 7f2ffaffd640 1 --2- 172.21.15.43:0/3289452666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a9457b60 0x5619a9460aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.657+0000 7f2ffaffd640 1 -- 172.21.15.43:0/3289452666 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619a90cf210 con 0x5619a94588d0 2024-04-21T08:39:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.657+0000 7f2ffbfff640 1 --2- 172.21.15.43:0/3289452666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619a94610e0 0x5619a945e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.657+0000 7f2ffaffd640 1 --2- 172.21.15.43:0/3289452666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a94588d0 0x5619a945de30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000bf40 tx=0x7f2ff00128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8208dbee1f48045 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.657+0000 7f2fdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.658+0000 7f2fdaffd640 1 -- 172.21.15.43:0/3289452666 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff00035f0 con 0x5619a94588d0 2024-04-21T08:39:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.658+0000 7f2fdaffd640 1 -- 172.21.15.43:0/3289452666 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f2ff0003790 con 0x5619a94588d0 2024-04-21T08:39:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.658+0000 7f3004798740 1 -- 172.21.15.43:0/3289452666 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fec0a0fa0 con 0x5619a94588d0 2024-04-21T08:39:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.658+0000 7f2fdaffd640 1 -- 172.21.15.43:0/3289452666 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2ff0003990 con 0x5619a94588d0 2024-04-21T08:39:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.658+0000 7f3004798740 10 client.16233 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.658+0000 7f3004798740 1 -- 172.21.15.43:0/3289452666 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5619a919b620 con 0x5619a94588d0 2024-04-21T08:39:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.658+0000 7f2fdaffd640 1 -- 172.21.15.43:0/3289452666 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fec0a0fa0 con 0x5619a94588d0 2024-04-21T08:39:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.658+0000 7f2fdaffd640 1 -- 172.21.15.43:0/3289452666 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2ff0019070 con 0x5619a94588d0 2024-04-21T08:39:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.658+0000 7f3004798740 10 client.16233 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.658+0000 7f3004798740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.658+0000 7f3004798740 20 client.16233 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.658+0000 7f3004798740 1 --2- 172.21.15.43:0/3289452666 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5619a9471260 0x5619a9473690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.658+0000 7f3004798740 4 client.16233 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.658+0000 7f3004798740 1 -- 172.21.15.43:0/3289452666 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619a93d0880 con 0x5619a9471260 2024-04-21T08:39:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.658+0000 7f2ffb7fe640 1 --2- 172.21.15.43:0/3289452666 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5619a9471260 0x5619a9473690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.659+0000 7f2ffb7fe640 1 --2- 172.21.15.43:0/3289452666 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5619a9471260 0x5619a9473690 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5fe646ec8f09cf5 server_cookie=caf28718453fb599 in_seq=0 out_seq=0 2024-04-21T08:39:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.659+0000 7f2fdaffd640 10 client.16233 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.660+0000 7f2fdaffd640 1 -- 172.21.15.43:0/3289452666 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5619a93d0880 con 0x5619a9471260 2024-04-21T08:39:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.660+0000 7f2fdaffd640 10 client.16233 handle_command_reply: tid=0 2024-04-21T08:39:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.660+0000 7f3004798740 1 client.16233 shutdown 2024-04-21T08:39:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.660+0000 7f3004798740 1 -- 172.21.15.43:0/3289452666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a94588d0 msgr2=0x5619a945de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.660+0000 7f3004798740 1 --2- 172.21.15.43:0/3289452666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a94588d0 0x5619a945de30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000bf40 tx=0x7f2ff00128e0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.660+0000 7f3004798740 1 -- 172.21.15.43:0/3289452666 shutdown_connections 2024-04-21T08:39:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.660+0000 7f3004798740 1 --2- 172.21.15.43:0/3289452666 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5619a9471260 0x5619a9473690 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.660+0000 7f3004798740 1 --2- 172.21.15.43:0/3289452666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619a94610e0 0x5619a945e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.660+0000 7f2fdaffd640 0 client.16233 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.660+0000 7f3004798740 1 --2- 172.21.15.43:0/3289452666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a94588d0 0x5619a945de30 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.660+0000 7f3004798740 1 --2- 172.21.15.43:0/3289452666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a9457b60 0x5619a9460aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.660+0000 7f3004798740 1 -- 172.21.15.43:0/3289452666 >> 172.21.15.43:0/3289452666 conn(0x5619a941ec90 msgr2=0x5619a94545a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.660+0000 7f3004798740 1 -- 172.21.15.43:0/3289452666 shutdown_connections 2024-04-21T08:39:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.660+0000 7f3004798740 1 -- 172.21.15.43:0/3289452666 wait complete. 2024-04-21T08:39:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.660+0000 7f3004798740 20 client.16233 trim_cache size 0 max 16384 2024-04-21T08:39:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.675+0000 7f3004798740 1 Processor -- start 2024-04-21T08:39:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.675+0000 7f3004798740 1 -- start start 2024-04-21T08:39:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.675+0000 7f3004798740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a9457b60 0x5619a946cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.675+0000 7f3004798740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619a94588d0 0x5619a9460560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.675+0000 7f3004798740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a94610e0 0x5619a9460aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.675+0000 7f3004798740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1af3e0 con 0x5619a9457b60 2024-04-21T08:39:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.675+0000 7f3004798740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1b0f20 con 0x5619a94610e0 2024-04-21T08:39:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.675+0000 7f3004798740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1afab0 con 0x5619a94588d0 2024-04-21T08:39:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.675+0000 7f2ffb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a9457b60 0x5619a946cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.675+0000 7f2ffb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a9457b60 0x5619a946cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48016/0 (socket says 172.21.15.43:48016) 2024-04-21T08:39:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.675+0000 7f2ffb7fe640 1 -- 172.21.15.43:0/486582174 learned_addr learned my addr 172.21.15.43:0/486582174 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.675+0000 7f2ffbfff640 1 --2- 172.21.15.43:0/486582174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a94610e0 0x5619a9460aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.675+0000 7f2ffaffd640 1 --2- 172.21.15.43:0/486582174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619a94588d0 0x5619a9460560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.675+0000 7f2ffb7fe640 1 -- 172.21.15.43:0/486582174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619a94588d0 msgr2=0x5619a9460560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.675+0000 7f2ffb7fe640 1 --2- 172.21.15.43:0/486582174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619a94588d0 0x5619a9460560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.675+0000 7f2ffb7fe640 1 -- 172.21.15.43:0/486582174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a94610e0 msgr2=0x5619a9460aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.675+0000 7f2ffb7fe640 1 --2- 172.21.15.43:0/486582174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a94610e0 0x5619a9460aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.675+0000 7f2ffb7fe640 1 -- 172.21.15.43:0/486582174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619a9456e30 con 0x5619a9457b60 2024-04-21T08:39:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.675+0000 7f2ffaffd640 1 --2- 172.21.15.43:0/486582174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619a94588d0 0x5619a9460560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.675+0000 7f2ffbfff640 1 --2- 172.21.15.43:0/486582174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a94610e0 0x5619a9460aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.676+0000 7f2ffb7fe640 1 --2- 172.21.15.43:0/486582174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a9457b60 0x5619a946cd10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2fec09cb80 tx=0x7f2fec09d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5b47e728310dd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.676+0000 7f2fe17fa640 1 -- 172.21.15.43:0/486582174 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec099360 con 0x5619a9457b60 2024-04-21T08:39:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.676+0000 7f2fe17fa640 1 -- 172.21.15.43:0/486582174 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fec010a90 con 0x5619a9457b60 2024-04-21T08:39:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.676+0000 7f2fe17fa640 1 -- 172.21.15.43:0/486582174 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec018b20 con 0x5619a9457b60 2024-04-21T08:39:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.676+0000 7f3004798740 1 -- 172.21.15.43:0/486582174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a9457b60 msgr2=0x5619a946cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.676+0000 7f3004798740 1 --2- 172.21.15.43:0/486582174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a9457b60 0x5619a946cd10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2fec09cb80 tx=0x7f2fec09d430 comp rx=0 tx=0).stop 2024-04-21T08:39:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.677+0000 7f3004798740 1 -- 172.21.15.43:0/486582174 shutdown_connections 2024-04-21T08:39:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.677+0000 7f3004798740 1 --2- 172.21.15.43:0/486582174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a94610e0 0x5619a9460aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.677+0000 7f3004798740 1 --2- 172.21.15.43:0/486582174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619a94588d0 0x5619a9460560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.677+0000 7f3004798740 1 --2- 172.21.15.43:0/486582174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a9457b60 0x5619a946cd10 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.677+0000 7f3004798740 1 -- 172.21.15.43:0/486582174 >> 172.21.15.43:0/486582174 conn(0x5619a941ec90 msgr2=0x5619a9459060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.677+0000 7f3004798740 1 -- 172.21.15.43:0/486582174 shutdown_connections 2024-04-21T08:39:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.677+0000 7f3004798740 1 -- 172.21.15.43:0/486582174 wait complete. 2024-04-21T08:39:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.677+0000 7f3004798740 1 -- start start 2024-04-21T08:39:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.677+0000 7f3004798740 1 Processor -- start 2024-04-21T08:39:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.677+0000 7f3004798740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a9457b60 0x5619a9461600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.677+0000 7f3004798740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a94588d0 0x5619a9461b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.677+0000 7f3004798740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619a94610e0 0x5619a9468220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.677+0000 7f3004798740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1b0850 con 0x5619a94588d0 2024-04-21T08:39:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.677+0000 7f3004798740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1b0180 con 0x5619a9457b60 2024-04-21T08:39:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.677+0000 7f3004798740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1ad3c0 con 0x5619a94610e0 2024-04-21T08:39:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.677+0000 7f3004798740 10 client.0 _ll_register_callbacks cb 0x7f2ff000bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.677+0000 7f2ffaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a94588d0 0x5619a9461b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.677+0000 7f2ffaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a94588d0 0x5619a9461b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48028/0 (socket says 172.21.15.43:48028) 2024-04-21T08:39:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.678+0000 7f2ffaffd640 1 -- 172.21.15.43:0/1725667398 learned_addr learned my addr 172.21.15.43:0/1725667398 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:39:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.678+0000 7f2ffbfff640 1 --2- 172.21.15.43:0/1725667398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619a94610e0 0x5619a9468220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.678+0000 7f2ffb7fe640 1 --2- 172.21.15.43:0/1725667398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a9457b60 0x5619a9461600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.678+0000 7f2ffaffd640 1 -- 172.21.15.43:0/1725667398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619a94610e0 msgr2=0x5619a9468220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.678+0000 7f2ffaffd640 1 --2- 172.21.15.43:0/1725667398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619a94610e0 0x5619a9468220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.678+0000 7f2ffaffd640 1 -- 172.21.15.43:0/1725667398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a9457b60 msgr2=0x5619a9461600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.678+0000 7f2ffaffd640 1 --2- 172.21.15.43:0/1725667398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a9457b60 0x5619a9461600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.678+0000 7f2ffaffd640 1 -- 172.21.15.43:0/1725667398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619a9462430 con 0x5619a94588d0 2024-04-21T08:39:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.678+0000 7f2ffb7fe640 1 --2- 172.21.15.43:0/1725667398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a9457b60 0x5619a9461600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.678+0000 7f2ffbfff640 1 --2- 172.21.15.43:0/1725667398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619a94610e0 0x5619a9468220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.678+0000 7f2ffaffd640 1 --2- 172.21.15.43:0/1725667398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a94588d0 0x5619a9461b40 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0000c00 tx=0x7f2ff0005860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d420452454b91421 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.678+0000 7f2fdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:39:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.678+0000 7f2fdaffd640 1 -- 172.21.15.43:0/1725667398 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0003d30 con 0x5619a94588d0 2024-04-21T08:39:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.678+0000 7f2fdaffd640 1 -- 172.21.15.43:0/1725667398 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f2ff0005040 con 0x5619a94588d0 2024-04-21T08:39:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.678+0000 7f3004798740 1 -- 172.21.15.43:0/1725667398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fec010a90 con 0x5619a94588d0 2024-04-21T08:39:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.678+0000 7f2fdaffd640 1 -- 172.21.15.43:0/1725667398 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2ff0037050 con 0x5619a94588d0 2024-04-21T08:39:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.678+0000 7f3004798740 10 client.16239 fetch_fsmap learned FSMap version 28 2024-04-21T08:39:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.678+0000 7f3004798740 1 -- 172.21.15.43:0/1725667398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5619a9469840 con 0x5619a94588d0 2024-04-21T08:39:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.678+0000 7f2fdaffd640 1 -- 172.21.15.43:0/1725667398 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fec010a90 con 0x5619a94588d0 2024-04-21T08:39:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.679+0000 7f2fdaffd640 1 -- 172.21.15.43:0/1725667398 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2ff0019070 con 0x5619a94588d0 2024-04-21T08:39:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.679+0000 7f3004798740 10 client.16239 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:39:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.679+0000 7f3004798740 10 client.16239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.679+0000 7f3004798740 20 client.16239 populate_metadata read hostname 'smithi043' 2024-04-21T08:39:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.679+0000 7f3004798740 1 --2- 172.21.15.43:0/1725667398 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5619a94d7b30 0x5619a94d9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.679+0000 7f3004798740 4 client.16239 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:39:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.679+0000 7f3004798740 1 -- 172.21.15.43:0/1725667398 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5619a935c3d0 con 0x5619a94d7b30 2024-04-21T08:39:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.679+0000 7f2ffb7fe640 1 --2- 172.21.15.43:0/1725667398 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5619a94d7b30 0x5619a94d9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.679+0000 7f2fdaffd640 10 client.16239 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.679+0000 7f2ffb7fe640 1 --2- 172.21.15.43:0/1725667398 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5619a94d7b30 0x5619a94d9f50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c55d6f65dd067ec6 server_cookie=897021b29baa6d9e in_seq=0 out_seq=0 2024-04-21T08:39:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.687+0000 7f2fdaffd640 1 -- 172.21.15.43:0/1725667398 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5619a935c3d0 con 0x5619a94d7b30 2024-04-21T08:39:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.687+0000 7f2fdaffd640 10 client.16239 handle_command_reply: tid=0 2024-04-21T08:39:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.687+0000 7f3004798740 1 client.16239 shutdown 2024-04-21T08:39:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.687+0000 7f3004798740 1 -- 172.21.15.43:0/1725667398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a94588d0 msgr2=0x5619a9461b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.687+0000 7f3004798740 1 --2- 172.21.15.43:0/1725667398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a94588d0 0x5619a9461b40 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0000c00 tx=0x7f2ff0005860 comp rx=0 tx=0).stop 2024-04-21T08:39:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.687+0000 7f3004798740 1 -- 172.21.15.43:0/1725667398 shutdown_connections 2024-04-21T08:39:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.687+0000 7f3004798740 1 --2- 172.21.15.43:0/1725667398 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5619a94d7b30 0x5619a94d9f50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.687+0000 7f2fdaffd640 0 client.16239 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:39:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.687+0000 7f3004798740 1 --2- 172.21.15.43:0/1725667398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619a94610e0 0x5619a9468220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.687+0000 7f3004798740 1 --2- 172.21.15.43:0/1725667398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619a94588d0 0x5619a9461b40 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.687+0000 7f3004798740 1 --2- 172.21.15.43:0/1725667398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a9457b60 0x5619a9461600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.687+0000 7f3004798740 1 -- 172.21.15.43:0/1725667398 >> 172.21.15.43:0/1725667398 conn(0x5619a941ec90 msgr2=0x5619a945c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.687+0000 7f3004798740 1 -- 172.21.15.43:0/1725667398 shutdown_connections 2024-04-21T08:39:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.687+0000 7f3004798740 1 -- 172.21.15.43:0/1725667398 wait complete. 2024-04-21T08:39:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.687+0000 7f3004798740 20 client.16239 trim_cache size 0 max 16384 2024-04-21T08:39:54.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:39:54.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:39:54.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:39:54.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:39:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.688+0000 7f3001ba4640 1 -- 172.21.15.43:0/4267145037 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2fe40a3e60 msgr2=0x7f2fe40a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.688+0000 7f3001ba4640 1 --2- 172.21.15.43:0/4267145037 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2fe40a3e60 0x7f2fe40a6320 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2ff40325e0 tx=0x7f2ff40095c0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.688+0000 7f3001ba4640 1 -- 172.21.15.43:0/4267145037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ffc1031e0 msgr2=0x7f2ffc198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.688+0000 7f3001ba4640 1 --2- 172.21.15.43:0/4267145037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ffc1031e0 0x7f2ffc198650 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00e9f0 tx=0x7f2fec00eeb0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.688+0000 7f3001ba4640 1 -- 172.21.15.43:0/4267145037 shutdown_connections 2024-04-21T08:39:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.688+0000 7f3001ba4640 1 --2- 172.21.15.43:0/4267145037 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2fe40a3e60 0x7f2fe40a6320 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.688+0000 7f3001ba4640 1 --2- 172.21.15.43:0/4267145037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ffc10c520 0x7f2ffc19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.688+0000 7f3001ba4640 1 --2- 172.21.15.43:0/4267145037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc103b00 0x7f2ffc198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.688+0000 7f3001ba4640 1 --2- 172.21.15.43:0/4267145037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ffc1031e0 0x7f2ffc198650 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.688+0000 7f3001ba4640 1 -- 172.21.15.43:0/4267145037 >> 172.21.15.43:0/4267145037 conn(0x7f2ffc077fd0 msgr2=0x7f2ffc0fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.688+0000 7f3001ba4640 1 -- 172.21.15.43:0/4267145037 shutdown_connections 2024-04-21T08:39:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:39:54.688+0000 7f3001ba4640 1 -- 172.21.15.43:0/4267145037 wait complete. 2024-04-21T08:39:55.568 INFO:tasks.workunit.client.0.smithi043.stderr:Note: switching to '4f6bc5273643aaf5660909e85400647a8951c10b'. 2024-04-21T08:39:55.568 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-04-21T08:39:55.568 INFO:tasks.workunit.client.0.smithi043.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-21T08:39:55.568 INFO:tasks.workunit.client.0.smithi043.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-21T08:39:55.568 INFO:tasks.workunit.client.0.smithi043.stderr:state without impacting any branches by switching back to a branch. 2024-04-21T08:39:55.569 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-04-21T08:39:55.569 INFO:tasks.workunit.client.0.smithi043.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-21T08:39:55.569 INFO:tasks.workunit.client.0.smithi043.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-21T08:39:55.569 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-04-21T08:39:55.569 INFO:tasks.workunit.client.0.smithi043.stderr: git switch -c 2024-04-21T08:39:55.569 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-04-21T08:39:55.569 INFO:tasks.workunit.client.0.smithi043.stderr:Or undo this operation with: 2024-04-21T08:39:55.569 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-04-21T08:39:55.569 INFO:tasks.workunit.client.0.smithi043.stderr: git switch - 2024-04-21T08:39:55.569 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-04-21T08:39:55.569 INFO:tasks.workunit.client.0.smithi043.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-21T08:39:55.569 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-04-21T08:39:55.569 INFO:tasks.workunit.client.0.smithi043.stderr:HEAD is now at 4f6bc527364 use ubuntu for fs/workload tests 2024-04-21T08:39:55.572 DEBUG:teuthology.orchestra.run.smithi043:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-21T08:39:55.602 INFO:tasks.workunit.client.0.smithi043.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-21T08:39:55.604 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-21T08:39:55.604 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-21T08:39:56.251 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-21T08:39:56.267 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:55 smithi175 ceph-mon[31920]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-21T08:39:56.293 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:55 smithi112 ceph-mon[31085]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-21T08:39:56.304 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-21T08:39:56.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:55 smithi043 ceph-mon[27978]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-21T08:39:56.345 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-21T08:39:56.347 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-21T08:39:56.347 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-21T08:39:56.387 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-21T08:39:56.390 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-21T08:39:56.390 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-21T08:39:56.447 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-04-21T08:39:56.448 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-04-21T08:39:56.449 DEBUG:teuthology.orchestra.run.smithi043:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4f6bc5273643aaf5660909e85400647a8951c10b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-21T08:39:56.508 INFO:tasks.workunit.client.0.smithi043.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-21T08:39:56.509 INFO:tasks.workunit.client.0.smithi043.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-04-21T08:39:56.509 INFO:tasks.workunit.client.0.smithi043.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-04-21T08:39:58.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:57 smithi112 ceph-mon[31085]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2024-04-21T08:39:58.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:57 smithi043 ceph-mon[27978]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2024-04-21T08:39:58.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:57 smithi175 ceph-mon[31920]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2024-04-21T08:40:00.274 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:59 smithi112 ceph-mon[31085]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-21T08:40:00.275 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:39:59 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:40:00.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:59 smithi175 ceph-mon[31920]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-21T08:40:00.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:39:59 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:40:00.306 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:59 smithi043 ceph-mon[27978]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-21T08:40:00.306 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:39:59 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:40:00.561 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:40:00 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:40:00.339+0000 7fa459303700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:40:00.561 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:40:00 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:40:00.339+0000 7fa459303700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:40:00.561 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:40:00 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:40:00.339+0000 7fa459303700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:40:00.561 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:40:00 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:40:00.339+0000 7fa459303700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:40:00.562 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:40:00 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:40:00.339+0000 7fa459303700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:40:01.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:00 smithi112 ceph-mon[31085]: overall HEALTH_OK 2024-04-21T08:40:01.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:00 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:40:01.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:00 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:40:01.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:00 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:40:01.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:00 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:40:01.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:00 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:40:01.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:00 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:40:01.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:00 smithi043 ceph-mon[27978]: overall HEALTH_OK 2024-04-21T08:40:01.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:00 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:40:01.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:00 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:40:01.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:00 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:40:01.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:00 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:40:01.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:00 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:40:01.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:00 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:40:01.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:00 smithi175 ceph-mon[31920]: overall HEALTH_OK 2024-04-21T08:40:01.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:00 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:40:01.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:00 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:40:01.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:00 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:40:01.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:00 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:40:01.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:00 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:40:01.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:00 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:40:02.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:01 smithi112 ceph-mon[31085]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 KiB/s wr, 31 op/s 2024-04-21T08:40:02.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:01 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:02.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:01 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:40:02.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:01 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:40:02.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:01 smithi112 ceph-mon[31085]: mgrmap e25: x(active, since 66s), standbys: z, y 2024-04-21T08:40:02.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:01 smithi043 ceph-mon[27978]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 KiB/s wr, 31 op/s 2024-04-21T08:40:02.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:01 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:02.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:01 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:40:02.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:01 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:40:02.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:01 smithi043 ceph-mon[27978]: mgrmap e25: x(active, since 66s), standbys: z, y 2024-04-21T08:40:02.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:01 smithi175 ceph-mon[31920]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 KiB/s wr, 31 op/s 2024-04-21T08:40:02.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:01 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:02.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:01 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:40:02.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:01 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:40:02.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:01 smithi175 ceph-mon[31920]: mgrmap e25: x(active, since 66s), standbys: z, y 2024-04-21T08:40:03.595 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:40:03.595 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:40:03.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.716+0000 7ff93e715640 1 -- 172.21.15.43:0/1507018607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff93810c500 msgr2=0x7ff93810e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.716+0000 7ff93e715640 1 --2- 172.21.15.43:0/1507018607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff93810c500 0x7ff93810e9a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff930009ca0 tx=0x7ff93002c230 comp rx=0 tx=0).stop 2024-04-21T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.716+0000 7ff93e715640 1 -- 172.21.15.43:0/1507018607 shutdown_connections 2024-04-21T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.716+0000 7ff93e715640 1 --2- 172.21.15.43:0/1507018607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff93810c500 0x7ff93810e9a0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.716+0000 7ff93e715640 1 --2- 172.21.15.43:0/1507018607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff938103ae0 0x7ff93810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.716+0000 7ff93e715640 1 --2- 172.21.15.43:0/1507018607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9381031c0 0x7ff9381035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.716+0000 7ff93e715640 1 -- 172.21.15.43:0/1507018607 >> 172.21.15.43:0/1507018607 conn(0x7ff938078000 msgr2=0x7ff938078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.716+0000 7ff93e715640 1 -- 172.21.15.43:0/1507018607 shutdown_connections 2024-04-21T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.716+0000 7ff93e715640 1 -- 172.21.15.43:0/1507018607 wait complete. 2024-04-21T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.716+0000 7ff93e715640 1 Processor -- start 2024-04-21T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.716+0000 7ff93e715640 1 -- start start 2024-04-21T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.717+0000 7ff93e715640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9381031c0 0x7ff9381a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.717+0000 7ff93e715640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff938103ae0 0x7ff9381a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.717+0000 7ff93e715640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff93810c500 0x7ff93819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.717+0000 7ff93e715640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff938112190 con 0x7ff9381031c0 2024-04-21T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.717+0000 7ff93e715640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff938112010 con 0x7ff93810c500 2024-04-21T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.717+0000 7ff93e715640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff938112310 con 0x7ff938103ae0 2024-04-21T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.717+0000 7ff937fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9381031c0 0x7ff9381a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.717+0000 7ff937fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9381031c0 0x7ff9381a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48870/0 (socket says 172.21.15.43:48870) 2024-04-21T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.717+0000 7ff937fff640 1 -- 172.21.15.43:0/3784006478 learned_addr learned my addr 172.21.15.43:0/3784006478 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.717+0000 7ff93ccb2640 1 --2- 172.21.15.43:0/3784006478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff93810c500 0x7ff93819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.717+0000 7ff9377fe640 1 --2- 172.21.15.43:0/3784006478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff938103ae0 0x7ff9381a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.717+0000 7ff937fff640 1 -- 172.21.15.43:0/3784006478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff938103ae0 msgr2=0x7ff9381a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.717+0000 7ff937fff640 1 --2- 172.21.15.43:0/3784006478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff938103ae0 0x7ff9381a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.717+0000 7ff937fff640 1 -- 172.21.15.43:0/3784006478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff93810c500 msgr2=0x7ff93819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.717+0000 7ff937fff640 1 --2- 172.21.15.43:0/3784006478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff93810c500 0x7ff93819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.717+0000 7ff937fff640 1 -- 172.21.15.43:0/3784006478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff93819aff0 con 0x7ff9381031c0 2024-04-21T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.717+0000 7ff937fff640 1 --2- 172.21.15.43:0/3784006478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9381031c0 0x7ff9381a0670 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff92800c970 tx=0x7ff92800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.718+0000 7ff9357fa640 1 -- 172.21.15.43:0/3784006478 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff928007bb0 con 0x7ff9381031c0 2024-04-21T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.718+0000 7ff9357fa640 1 -- 172.21.15.43:0/3784006478 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff928007d50 con 0x7ff9381031c0 2024-04-21T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.718+0000 7ff93e715640 1 -- 172.21.15.43:0/3784006478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff93819b2e0 con 0x7ff9381031c0 2024-04-21T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.718+0000 7ff9357fa640 1 -- 172.21.15.43:0/3784006478 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92800f450 con 0x7ff9381031c0 2024-04-21T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.718+0000 7ff93e715640 1 -- 172.21.15.43:0/3784006478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9381a75f0 con 0x7ff9381031c0 2024-04-21T08:40:03.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.720+0000 7ff9357fa640 1 -- 172.21.15.43:0/3784006478 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7ff928004f20 con 0x7ff9381031c0 2024-04-21T08:40:03.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.720+0000 7ff9413dd740 1 Processor -- start 2024-04-21T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.720+0000 7ff9413dd740 1 -- start start 2024-04-21T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.720+0000 7ff9413dd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b848580 0x556a7b848960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.720+0000 7ff9413dd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a7b848ea0 0x556a7b851540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.720+0000 7ff9413dd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a7b851a80 0x556a7b853e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.721+0000 7ff9413dd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556a7b7ad1e0 con 0x556a7b848580 2024-04-21T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.721+0000 7ff9413dd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556a7b74a390 con 0x556a7b851a80 2024-04-21T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.721+0000 7ff9413dd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556a7b7a91a0 con 0x556a7b848ea0 2024-04-21T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.721+0000 7ff93ccb2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b848580 0x556a7b848960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.721+0000 7ff93ccb2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b848580 0x556a7b848960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48878/0 (socket says 172.21.15.43:48878) 2024-04-21T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.721+0000 7ff93ccb2640 1 -- 172.21.15.43:0/538735346 learned_addr learned my addr 172.21.15.43:0/538735346 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.721+0000 7ff9357fa640 1 --2- 172.21.15.43:0/3784006478 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff9240a3ec0 0x7ff9240a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.721+0000 7ff937fff640 1 --2- 172.21.15.43:0/538735346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a7b848ea0 0x556a7b851540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.721+0000 7ff9377fe640 1 --2- 172.21.15.43:0/538735346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a7b851a80 0x556a7b853e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.721+0000 7ff93ccb2640 1 -- 172.21.15.43:0/538735346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a7b848ea0 msgr2=0x556a7b851540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.721+0000 7ff93ccb2640 1 --2- 172.21.15.43:0/538735346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a7b848ea0 0x556a7b851540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.721+0000 7ff93ccb2640 1 -- 172.21.15.43:0/538735346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a7b851a80 msgr2=0x556a7b853e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.721+0000 7ff93ccb2640 1 --2- 172.21.15.43:0/538735346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a7b851a80 0x556a7b853e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.721+0000 7ff93ccb2640 1 -- 172.21.15.43:0/538735346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a7b5897d0 con 0x556a7b848580 2024-04-21T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.721+0000 7ff9377fe640 1 --2- 172.21.15.43:0/538735346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a7b851a80 0x556a7b853e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.721+0000 7ff9357fa640 1 -- 172.21.15.43:0/3784006478 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7ff9280d0c30 con 0x7ff9381031c0 2024-04-21T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.721+0000 7ff93ccb2640 1 --2- 172.21.15.43:0/3784006478 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff9240a3ec0 0x7ff9240a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.721+0000 7ff93ccb2640 1 --2- 172.21.15.43:0/538735346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b848580 0x556a7b848960 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff930009ca0 tx=0x7ff930032400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61156e220790b34 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.721+0000 7ff92e7fc640 1 -- 172.21.15.43:0/538735346 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930039040 con 0x556a7b848580 2024-04-21T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.722+0000 7ff92e7fc640 1 -- 172.21.15.43:0/538735346 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff930004710 con 0x556a7b848580 2024-04-21T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.722+0000 7ff92e7fc640 1 -- 172.21.15.43:0/538735346 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930034720 con 0x556a7b848580 2024-04-21T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.722+0000 7ff9413dd740 1 -- 172.21.15.43:0/538735346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b848580 msgr2=0x556a7b848960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.722+0000 7ff9413dd740 1 --2- 172.21.15.43:0/538735346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b848580 0x556a7b848960 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff930009ca0 tx=0x7ff930032400 comp rx=0 tx=0).stop 2024-04-21T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.722+0000 7ff93ccb2640 1 --2- 172.21.15.43:0/3784006478 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff9240a3ec0 0x7ff9240a6380 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff93004b000 tx=0x7ff930044000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.722+0000 7ff9413dd740 1 -- 172.21.15.43:0/538735346 shutdown_connections 2024-04-21T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.722+0000 7ff9413dd740 1 --2- 172.21.15.43:0/538735346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a7b851a80 0x556a7b853e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.722+0000 7ff9413dd740 1 --2- 172.21.15.43:0/538735346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a7b848ea0 0x556a7b851540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.722+0000 7ff9413dd740 1 --2- 172.21.15.43:0/538735346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b848580 0x556a7b848960 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.722+0000 7ff9413dd740 1 -- 172.21.15.43:0/538735346 >> 172.21.15.43:0/538735346 conn(0x556a7b842cb0 msgr2=0x556a7b843090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.722+0000 7ff9413dd740 1 -- 172.21.15.43:0/538735346 shutdown_connections 2024-04-21T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.722+0000 7ff9413dd740 1 -- 172.21.15.43:0/538735346 wait complete. 2024-04-21T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.723+0000 7ff9413dd740 1 -- start start 2024-04-21T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.723+0000 7ff9413dd740 1 Processor -- start 2024-04-21T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.723+0000 7ff9413dd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b848580 0x556a7b84eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.723+0000 7ff9413dd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a7b848ea0 0x556a7b84eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.723+0000 7ff9413dd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a7b851a80 0x556a7b84f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.723+0000 7ff9413dd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556a7b854870 con 0x556a7b848580 2024-04-21T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.723+0000 7ff9413dd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556a7b7662f0 con 0x556a7b848ea0 2024-04-21T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.723+0000 7ff9413dd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556a7b8549f0 con 0x556a7b851a80 2024-04-21T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.723+0000 7ff9413dd740 10 client.0 _ll_register_callbacks cb 0x556a7b773580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.723+0000 7ff937fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b848580 0x556a7b84eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.723+0000 7ff937fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b848580 0x556a7b84eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48888/0 (socket says 172.21.15.43:48888) 2024-04-21T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.723+0000 7ff937fff640 1 -- 172.21.15.43:0/292335844 learned_addr learned my addr 172.21.15.43:0/292335844 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.724+0000 7ff93ccb2640 1 --2- 172.21.15.43:0/292335844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a7b851a80 0x556a7b84f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.724+0000 7ff937fff640 1 -- 172.21.15.43:0/292335844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a7b851a80 msgr2=0x556a7b84f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.724+0000 7ff937fff640 1 --2- 172.21.15.43:0/292335844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a7b851a80 0x556a7b84f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.724+0000 7ff9377fe640 1 --2- 172.21.15.43:0/292335844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a7b848ea0 0x556a7b84eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.724+0000 7ff937fff640 1 -- 172.21.15.43:0/292335844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a7b848ea0 msgr2=0x556a7b84eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.724+0000 7ff937fff640 1 --2- 172.21.15.43:0/292335844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a7b848ea0 0x556a7b84eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.724+0000 7ff937fff640 1 -- 172.21.15.43:0/292335844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a7b4bf3f0 con 0x556a7b848580 2024-04-21T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.724+0000 7ff937fff640 1 --2- 172.21.15.43:0/292335844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b848580 0x556a7b84eab0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff92809c250 tx=0x7ff9280055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c50cd9d88946d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.724+0000 7ff9177fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.724+0000 7ff9177fe640 1 -- 172.21.15.43:0/292335844 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92801d070 con 0x556a7b848580 2024-04-21T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.724+0000 7ff9177fe640 1 -- 172.21.15.43:0/292335844 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7ff9280a70c0 con 0x556a7b848580 2024-04-21T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.724+0000 7ff9413dd740 1 -- 172.21.15.43:0/292335844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff930004710 con 0x556a7b848580 2024-04-21T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.724+0000 7ff9177fe640 1 -- 172.21.15.43:0/292335844 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff9280a0900 con 0x556a7b848580 2024-04-21T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.724+0000 7ff9413dd740 10 client.16257 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.724+0000 7ff9413dd740 1 -- 172.21.15.43:0/292335844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556a7b47f0f0 con 0x556a7b848580 2024-04-21T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.724+0000 7ff9177fe640 1 -- 172.21.15.43:0/292335844 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff930004710 con 0x556a7b848580 2024-04-21T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.724+0000 7ff9177fe640 1 -- 172.21.15.43:0/292335844 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff92809b9a0 con 0x556a7b848580 2024-04-21T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.725+0000 7ff9413dd740 10 client.16257 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.725+0000 7ff9413dd740 10 client.16257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.725+0000 7ff9413dd740 20 client.16257 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.725+0000 7ff9413dd740 1 --2- 172.21.15.43:0/292335844 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556a7b861960 0x556a7b863dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.725+0000 7ff9413dd740 4 client.16257 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.725+0000 7ff9413dd740 1 -- 172.21.15.43:0/292335844 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a7b7bac00 con 0x556a7b861960 2024-04-21T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.725+0000 7ff9377fe640 1 --2- 172.21.15.43:0/292335844 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556a7b861960 0x556a7b863dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.725+0000 7ff9377fe640 1 --2- 172.21.15.43:0/292335844 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556a7b861960 0x556a7b863dd0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90bf691d4a20db0c server_cookie=3648a90bdf82bc73 in_seq=0 out_seq=0 2024-04-21T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.725+0000 7ff9177fe640 10 client.16257 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.726+0000 7ff9177fe640 1 -- 172.21.15.43:0/292335844 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556a7b7bac00 con 0x556a7b861960 2024-04-21T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.726+0000 7ff9177fe640 10 client.16257 handle_command_reply: tid=0 2024-04-21T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.726+0000 7ff9413dd740 1 client.16257 shutdown 2024-04-21T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.726+0000 7ff9413dd740 1 -- 172.21.15.43:0/292335844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b848580 msgr2=0x556a7b84eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.726+0000 7ff9413dd740 1 --2- 172.21.15.43:0/292335844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b848580 0x556a7b84eab0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff92809c250 tx=0x7ff9280055f0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.726+0000 7ff9413dd740 1 -- 172.21.15.43:0/292335844 shutdown_connections 2024-04-21T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.726+0000 7ff9413dd740 1 --2- 172.21.15.43:0/292335844 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556a7b861960 0x556a7b863dd0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.726+0000 7ff9413dd740 1 --2- 172.21.15.43:0/292335844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a7b851a80 0x556a7b84f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.726+0000 7ff9413dd740 1 --2- 172.21.15.43:0/292335844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a7b848ea0 0x556a7b84eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.726+0000 7ff9413dd740 1 --2- 172.21.15.43:0/292335844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b848580 0x556a7b84eab0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.726+0000 7ff9177fe640 0 client.16257 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:03.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.726+0000 7ff9413dd740 1 -- 172.21.15.43:0/292335844 >> 172.21.15.43:0/292335844 conn(0x556a7b842cb0 msgr2=0x556a7b844a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:03.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.726+0000 7ff9413dd740 1 -- 172.21.15.43:0/292335844 shutdown_connections 2024-04-21T08:40:03.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.726+0000 7ff9413dd740 1 -- 172.21.15.43:0/292335844 wait complete. 2024-04-21T08:40:03.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.726+0000 7ff9413dd740 20 client.16257 trim_cache size 0 max 16384 2024-04-21T08:40:03.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.744+0000 7ff9413dd740 1 Processor -- start 2024-04-21T08:40:03.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.744+0000 7ff9413dd740 1 -- start start 2024-04-21T08:40:03.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.744+0000 7ff9413dd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a7b848580 0x556a7b8511e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:03.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.744+0000 7ff9413dd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b848ea0 0x556a7b85d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:03.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.744+0000 7ff9413dd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a7b851a80 0x556a7b84dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:03.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.744+0000 7ff9413dd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff9381b15f0 con 0x556a7b848ea0 2024-04-21T08:40:03.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.744+0000 7ff9413dd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff9381b3130 con 0x556a7b848580 2024-04-21T08:40:03.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.744+0000 7ff9413dd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff9381b1cc0 con 0x556a7b851a80 2024-04-21T08:40:03.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.744+0000 7ff9377fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b848ea0 0x556a7b85d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.744+0000 7ff9377fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b848ea0 0x556a7b85d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48896/0 (socket says 172.21.15.43:48896) 2024-04-21T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.744+0000 7ff9377fe640 1 -- 172.21.15.43:0/3901441693 learned_addr learned my addr 172.21.15.43:0/3901441693 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.745+0000 7ff9377fe640 1 -- 172.21.15.43:0/3901441693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a7b851a80 msgr2=0x556a7b84dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.745+0000 7ff9377fe640 1 --2- 172.21.15.43:0/3901441693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a7b851a80 0x556a7b84dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.745+0000 7ff937fff640 1 --2- 172.21.15.43:0/3901441693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a7b848580 0x556a7b8511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.745+0000 7ff9377fe640 1 -- 172.21.15.43:0/3901441693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a7b848580 msgr2=0x556a7b8511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.745+0000 7ff9377fe640 1 --2- 172.21.15.43:0/3901441693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a7b848580 0x556a7b8511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.745+0000 7ff9377fe640 1 -- 172.21.15.43:0/3901441693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a7b848cd0 con 0x556a7b848ea0 2024-04-21T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.745+0000 7ff937fff640 1 --2- 172.21.15.43:0/3901441693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a7b848580 0x556a7b8511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.745+0000 7ff9377fe640 1 --2- 172.21.15.43:0/3901441693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b848ea0 0x556a7b85d760 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x556a7b84e310 tx=0x7ff920005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787db28326b1af0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.745+0000 7ff92dffb640 1 -- 172.21.15.43:0/3901441693 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920022070 con 0x556a7b848ea0 2024-04-21T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.745+0000 7ff92dffb640 1 -- 172.21.15.43:0/3901441693 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff920012d00 con 0x556a7b848ea0 2024-04-21T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.745+0000 7ff92dffb640 1 -- 172.21.15.43:0/3901441693 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920003850 con 0x556a7b848ea0 2024-04-21T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.745+0000 7ff9413dd740 1 -- 172.21.15.43:0/3901441693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b848ea0 msgr2=0x556a7b85d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.745+0000 7ff9413dd740 1 --2- 172.21.15.43:0/3901441693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b848ea0 0x556a7b85d760 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x556a7b84e310 tx=0x7ff920005d10 comp rx=0 tx=0).stop 2024-04-21T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.746+0000 7ff9413dd740 1 -- 172.21.15.43:0/3901441693 shutdown_connections 2024-04-21T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.746+0000 7ff9413dd740 1 --2- 172.21.15.43:0/3901441693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a7b851a80 0x556a7b84dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.746+0000 7ff9413dd740 1 --2- 172.21.15.43:0/3901441693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b848ea0 0x556a7b85d760 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.746+0000 7ff9413dd740 1 --2- 172.21.15.43:0/3901441693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a7b848580 0x556a7b8511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.746+0000 7ff9413dd740 1 -- 172.21.15.43:0/3901441693 >> 172.21.15.43:0/3901441693 conn(0x556a7b842cb0 msgr2=0x556a7b844080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.746+0000 7ff9413dd740 1 -- 172.21.15.43:0/3901441693 shutdown_connections 2024-04-21T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.746+0000 7ff9413dd740 1 -- 172.21.15.43:0/3901441693 wait complete. 2024-04-21T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.746+0000 7ff9413dd740 1 -- start start 2024-04-21T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.746+0000 7ff9413dd740 1 Processor -- start 2024-04-21T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.746+0000 7ff9413dd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a7b848580 0x556a7b85a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.746+0000 7ff9413dd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a7b848ea0 0x556a7b8554b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.746+0000 7ff9413dd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b851a80 0x556a7b855ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.746+0000 7ff9413dd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff9381b2a60 con 0x556a7b851a80 2024-04-21T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.746+0000 7ff9413dd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff9381b2390 con 0x556a7b848ea0 2024-04-21T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.746+0000 7ff9413dd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff9381af5d0 con 0x556a7b848580 2024-04-21T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.746+0000 7ff9413dd740 10 client.0 _ll_register_callbacks cb 0x7ff92800c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.747+0000 7ff93ccb2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b851a80 0x556a7b855ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.747+0000 7ff93ccb2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b851a80 0x556a7b855ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48900/0 (socket says 172.21.15.43:48900) 2024-04-21T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.747+0000 7ff93ccb2640 1 -- 172.21.15.43:0/949144023 learned_addr learned my addr 172.21.15.43:0/949144023 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.747+0000 7ff937fff640 1 --2- 172.21.15.43:0/949144023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a7b848580 0x556a7b85a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.747+0000 7ff9377fe640 1 --2- 172.21.15.43:0/949144023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a7b848ea0 0x556a7b8554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.747+0000 7ff93ccb2640 1 -- 172.21.15.43:0/949144023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a7b848580 msgr2=0x556a7b85a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.747+0000 7ff93ccb2640 1 --2- 172.21.15.43:0/949144023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a7b848580 0x556a7b85a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.747+0000 7ff93ccb2640 1 -- 172.21.15.43:0/949144023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a7b848ea0 msgr2=0x556a7b8554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.747+0000 7ff93ccb2640 1 --2- 172.21.15.43:0/949144023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a7b848ea0 0x556a7b8554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.747+0000 7ff93ccb2640 1 -- 172.21.15.43:0/949144023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a7b8562c0 con 0x556a7b851a80 2024-04-21T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.747+0000 7ff93ccb2640 1 --2- 172.21.15.43:0/949144023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b851a80 0x556a7b855ac0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff930009430 tx=0x7ff930034820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122be02434618ab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.747+0000 7ff9177fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.747+0000 7ff9177fe640 1 -- 172.21.15.43:0/949144023 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930004490 con 0x556a7b851a80 2024-04-21T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.747+0000 7ff9177fe640 1 -- 172.21.15.43:0/949144023 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7ff930004630 con 0x556a7b851a80 2024-04-21T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.747+0000 7ff9413dd740 1 -- 172.21.15.43:0/949144023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff920012d00 con 0x556a7b851a80 2024-04-21T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.747+0000 7ff9177fe640 1 -- 172.21.15.43:0/949144023 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff930062050 con 0x556a7b851a80 2024-04-21T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.747+0000 7ff9413dd740 10 client.16263 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.747+0000 7ff9413dd740 1 -- 172.21.15.43:0/949144023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556a7b8c7f90 con 0x556a7b851a80 2024-04-21T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.748+0000 7ff9177fe640 1 -- 172.21.15.43:0/949144023 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff920012d00 con 0x556a7b851a80 2024-04-21T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.748+0000 7ff9177fe640 1 -- 172.21.15.43:0/949144023 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff930053040 con 0x556a7b851a80 2024-04-21T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.748+0000 7ff9413dd740 10 client.16263 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.748+0000 7ff9413dd740 10 client.16263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.748+0000 7ff9413dd740 20 client.16263 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.748+0000 7ff9413dd740 1 --2- 172.21.15.43:0/949144023 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556a7b8c83a0 0x556a7b8ca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.748+0000 7ff9413dd740 4 client.16263 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.748+0000 7ff9413dd740 1 -- 172.21.15.43:0/949144023 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556a7b7b4a70 con 0x556a7b8c83a0 2024-04-21T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.748+0000 7ff937fff640 1 --2- 172.21.15.43:0/949144023 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556a7b8c83a0 0x556a7b8ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.748+0000 7ff9177fe640 10 client.16263 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.748+0000 7ff937fff640 1 --2- 172.21.15.43:0/949144023 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556a7b8c83a0 0x556a7b8ca790 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9462e15dfdeaa2ec server_cookie=ae26d307b538967e in_seq=0 out_seq=0 2024-04-21T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.749+0000 7ff9177fe640 1 -- 172.21.15.43:0/949144023 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556a7b7b4a70 con 0x556a7b8c83a0 2024-04-21T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.749+0000 7ff9177fe640 10 client.16263 handle_command_reply: tid=0 2024-04-21T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.749+0000 7ff9413dd740 1 client.16263 shutdown 2024-04-21T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.749+0000 7ff9413dd740 1 -- 172.21.15.43:0/949144023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b851a80 msgr2=0x556a7b855ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.749+0000 7ff9413dd740 1 --2- 172.21.15.43:0/949144023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b851a80 0x556a7b855ac0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff930009430 tx=0x7ff930034820 comp rx=0 tx=0).stop 2024-04-21T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.749+0000 7ff9413dd740 1 -- 172.21.15.43:0/949144023 shutdown_connections 2024-04-21T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.749+0000 7ff9413dd740 1 --2- 172.21.15.43:0/949144023 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556a7b8c83a0 0x556a7b8ca790 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.749+0000 7ff9177fe640 0 client.16263 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.749+0000 7ff9413dd740 1 --2- 172.21.15.43:0/949144023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a7b851a80 0x556a7b855ac0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.749+0000 7ff9413dd740 1 --2- 172.21.15.43:0/949144023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a7b848ea0 0x556a7b8554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.749+0000 7ff9413dd740 1 --2- 172.21.15.43:0/949144023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556a7b848580 0x556a7b85a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.749+0000 7ff9413dd740 1 -- 172.21.15.43:0/949144023 >> 172.21.15.43:0/949144023 conn(0x556a7b842cb0 msgr2=0x556a7b84f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:03.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.749+0000 7ff9413dd740 1 -- 172.21.15.43:0/949144023 shutdown_connections 2024-04-21T08:40:03.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.749+0000 7ff9413dd740 1 -- 172.21.15.43:0/949144023 wait complete. 2024-04-21T08:40:03.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.749+0000 7ff9413dd740 20 client.16263 trim_cache size 0 max 16384 2024-04-21T08:40:03.756 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:40:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:40:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:40:03.756 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:40:03.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.749+0000 7ff93e715640 1 -- 172.21.15.43:0/3784006478 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff9240a3ec0 msgr2=0x7ff9240a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:03.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.749+0000 7ff93e715640 1 --2- 172.21.15.43:0/3784006478 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff9240a3ec0 0x7ff9240a6380 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff93004b000 tx=0x7ff930044000 comp rx=0 tx=0).stop 2024-04-21T08:40:03.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.749+0000 7ff93e715640 1 -- 172.21.15.43:0/3784006478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9381031c0 msgr2=0x7ff9381a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:03.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.749+0000 7ff93e715640 1 --2- 172.21.15.43:0/3784006478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9381031c0 0x7ff9381a0670 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff92800c970 tx=0x7ff92800ce30 comp rx=0 tx=0).stop 2024-04-21T08:40:03.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.750+0000 7ff93e715640 1 -- 172.21.15.43:0/3784006478 shutdown_connections 2024-04-21T08:40:03.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.750+0000 7ff93e715640 1 --2- 172.21.15.43:0/3784006478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff93810c500 0x7ff93819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.750+0000 7ff93e715640 1 --2- 172.21.15.43:0/3784006478 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff9240a3ec0 0x7ff9240a6380 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.750+0000 7ff93e715640 1 --2- 172.21.15.43:0/3784006478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff938103ae0 0x7ff9381a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.750+0000 7ff93e715640 1 --2- 172.21.15.43:0/3784006478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9381031c0 0x7ff9381a0670 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.750+0000 7ff93e715640 1 -- 172.21.15.43:0/3784006478 >> 172.21.15.43:0/3784006478 conn(0x7ff938078000 msgr2=0x7ff938108470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:03.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.750+0000 7ff93e715640 1 -- 172.21.15.43:0/3784006478 shutdown_connections 2024-04-21T08:40:03.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.750+0000 7ff93e715640 1 -- 172.21.15.43:0/3784006478 wait complete. 2024-04-21T08:40:03.760 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:40:03.760 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:40:03.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.863+0000 7f0f23515640 1 -- 172.21.15.43:0/1484420818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f1c1031d0 msgr2=0x7f0f1c1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:03.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.863+0000 7f0f23515640 1 --2- 172.21.15.43:0/1484420818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f1c1031d0 0x7f0f1c1035b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009900 tx=0x7f0f1802bcd0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.863+0000 7f0f23515640 1 -- 172.21.15.43:0/1484420818 shutdown_connections 2024-04-21T08:40:03.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.863+0000 7f0f23515640 1 --2- 172.21.15.43:0/1484420818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f1c10c5a0 0x7f0f1c10e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.863+0000 7f0f23515640 1 --2- 172.21.15.43:0/1484420818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f1c103af0 0x7f0f1c10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.863+0000 7f0f23515640 1 --2- 172.21.15.43:0/1484420818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f1c1031d0 0x7f0f1c1035b0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.863+0000 7f0f23515640 1 -- 172.21.15.43:0/1484420818 >> 172.21.15.43:0/1484420818 conn(0x7f0f1c0771d0 msgr2=0x7f0f1c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.863+0000 7f0f23515640 1 -- 172.21.15.43:0/1484420818 shutdown_connections 2024-04-21T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.863+0000 7f0f23515640 1 -- 172.21.15.43:0/1484420818 wait complete. 2024-04-21T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.864+0000 7f0f23515640 1 Processor -- start 2024-04-21T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.864+0000 7f0f23515640 1 -- start start 2024-04-21T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.864+0000 7f0f23515640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f1c1031d0 0x7f0f1c100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.864+0000 7f0f23515640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f1c103af0 0x7f0f1c101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.864+0000 7f0f23515640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f1c10c5a0 0x7f0f1c1018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.864+0000 7f0f23515640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c1121f0 con 0x7f0f1c10c5a0 2024-04-21T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.864+0000 7f0f23515640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c112070 con 0x7f0f1c103af0 2024-04-21T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.864+0000 7f0f23515640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c112370 con 0x7f0f1c1031d0 2024-04-21T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.864+0000 7f0f21ab2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f1c10c5a0 0x7f0f1c1018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.864+0000 7f0f21ab2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f1c10c5a0 0x7f0f1c1018a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48928/0 (socket says 172.21.15.43:48928) 2024-04-21T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.864+0000 7f0f21ab2640 1 -- 172.21.15.43:0/4132344479 learned_addr learned my addr 172.21.15.43:0/4132344479 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.864+0000 7f0f20ab0640 1 --2- 172.21.15.43:0/4132344479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f1c103af0 0x7f0f1c101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.864+0000 7f0f212b1640 1 --2- 172.21.15.43:0/4132344479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f1c1031d0 0x7f0f1c100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.864+0000 7f0f21ab2640 1 -- 172.21.15.43:0/4132344479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f1c1031d0 msgr2=0x7f0f1c100e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.864+0000 7f0f21ab2640 1 --2- 172.21.15.43:0/4132344479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f1c1031d0 0x7f0f1c100e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.864+0000 7f0f21ab2640 1 -- 172.21.15.43:0/4132344479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f1c103af0 msgr2=0x7f0f1c101360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.864+0000 7f0f21ab2640 1 --2- 172.21.15.43:0/4132344479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f1c103af0 0x7f0f1c101360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.864+0000 7f0f21ab2640 1 -- 172.21.15.43:0/4132344479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f1c1a6c90 con 0x7f0f1c10c5a0 2024-04-21T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.864+0000 7f0f212b1640 1 --2- 172.21.15.43:0/4132344479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f1c1031d0 0x7f0f1c100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.864+0000 7f0f21ab2640 1 --2- 172.21.15.43:0/4132344479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f1c10c5a0 0x7f0f1c1018a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400e9d0 tx=0x7f0f1400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.865+0000 7f0f127fc640 1 -- 172.21.15.43:0/4132344479 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1400cd80 con 0x7f0f1c10c5a0 2024-04-21T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.865+0000 7f0f127fc640 1 -- 172.21.15.43:0/4132344479 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f14005020 con 0x7f0f1c10c5a0 2024-04-21T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.865+0000 7f0f23515640 1 -- 172.21.15.43:0/4132344479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f1c1a6f80 con 0x7f0f1c10c5a0 2024-04-21T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.865+0000 7f0f127fc640 1 -- 172.21.15.43:0/4132344479 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f140116d0 con 0x7f0f1c10c5a0 2024-04-21T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.865+0000 7f0f23515640 1 -- 172.21.15.43:0/4132344479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f1c1a74f0 con 0x7f0f1c10c5a0 2024-04-21T08:40:03.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.866+0000 7f0f23515640 1 -- 172.21.15.43:0/4132344479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f1c069230 con 0x7f0f1c10c5a0 2024-04-21T08:40:03.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.867+0000 7f0f127fc640 1 -- 172.21.15.43:0/4132344479 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f0f140051d0 con 0x7f0f1c10c5a0 2024-04-21T08:40:03.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.868+0000 7f0f127fc640 1 --2- 172.21.15.43:0/4132344479 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0f080a3ec0 0x7f0f080a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:03.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.868+0000 7f0f212b1640 1 --2- 172.21.15.43:0/4132344479 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0f080a3ec0 0x7f0f080a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:03.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.868+0000 7f0f127fc640 1 -- 172.21.15.43:0/4132344479 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f0f140d12e0 con 0x7f0f1c10c5a0 2024-04-21T08:40:03.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.868+0000 7f0f212b1640 1 --2- 172.21.15.43:0/4132344479 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0f080a3ec0 0x7f0f080a6380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0f18005ab0 tx=0x7f0f18005a20 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:03.873+0000 7f0f127fc640 1 -- 172.21.15.43:0/4132344479 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f14016030 con 0x7f0f1c10c5a0 2024-04-21T08:40:04.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.103+0000 7f0f23515640 1 -- 172.21.15.43:0/4132344479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f1c102350 con 0x7f0f1c10c5a0 2024-04-21T08:40:04.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.104+0000 7f0f127fc640 1 -- 172.21.15.43:0/4132344479 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7f0f140bbe20 con 0x7f0f1c10c5a0 2024-04-21T08:40:04.103 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:40:04.103 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:40:04.104 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:40:04.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.106+0000 7f0f23515640 1 -- 172.21.15.43:0/4132344479 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0f080a3ec0 msgr2=0x7f0f080a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.106+0000 7f0f23515640 1 --2- 172.21.15.43:0/4132344479 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0f080a3ec0 0x7f0f080a6380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0f18005ab0 tx=0x7f0f18005a20 comp rx=0 tx=0).stop 2024-04-21T08:40:04.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.106+0000 7f0f23515640 1 -- 172.21.15.43:0/4132344479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f1c10c5a0 msgr2=0x7f0f1c1018a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.106+0000 7f0f23515640 1 --2- 172.21.15.43:0/4132344479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f1c10c5a0 0x7f0f1c1018a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400e9d0 tx=0x7f0f1400ee90 comp rx=0 tx=0).stop 2024-04-21T08:40:04.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.106+0000 7f0f23515640 1 -- 172.21.15.43:0/4132344479 shutdown_connections 2024-04-21T08:40:04.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.106+0000 7f0f23515640 1 --2- 172.21.15.43:0/4132344479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f1c10c5a0 0x7f0f1c1018a0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.106+0000 7f0f23515640 1 --2- 172.21.15.43:0/4132344479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f1c103af0 0x7f0f1c101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.106+0000 7f0f23515640 1 --2- 172.21.15.43:0/4132344479 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0f080a3ec0 0x7f0f080a6380 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.106+0000 7f0f23515640 1 --2- 172.21.15.43:0/4132344479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f1c1031d0 0x7f0f1c100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.106+0000 7f0f23515640 1 -- 172.21.15.43:0/4132344479 >> 172.21.15.43:0/4132344479 conn(0x7f0f1c0771d0 msgr2=0x7f0f1c0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.106+0000 7f0f23515640 1 -- 172.21.15.43:0/4132344479 shutdown_connections 2024-04-21T08:40:04.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.106+0000 7f0f23515640 1 -- 172.21.15.43:0/4132344479 wait complete. 2024-04-21T08:40:04.116 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:40:04.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.222+0000 7f3401014640 1 -- 172.21.15.43:0/2927845492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33fc101a50 msgr2=0x7f33fc105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.222+0000 7f3401014640 1 --2- 172.21.15.43:0/2927845492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33fc101a50 0x7f33fc105a90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f33e4009900 tx=0x7f33e402bc30 comp rx=0 tx=0).stop 2024-04-21T08:40:04.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.222+0000 7f3401014640 1 -- 172.21.15.43:0/2927845492 shutdown_connections 2024-04-21T08:40:04.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.222+0000 7f3401014640 1 --2- 172.21.15.43:0/2927845492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33fc106170 0x7f33fc10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.222+0000 7f3401014640 1 --2- 172.21.15.43:0/2927845492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33fc101a50 0x7f33fc105a90 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.222+0000 7f3401014640 1 --2- 172.21.15.43:0/2927845492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33fc1010a0 0x7f33fc101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.222+0000 7f3401014640 1 -- 172.21.15.43:0/2927845492 >> 172.21.15.43:0/2927845492 conn(0x7f33fc077eb0 msgr2=0x7f33fc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.222+0000 7f3401014640 1 -- 172.21.15.43:0/2927845492 shutdown_connections 2024-04-21T08:40:04.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.222+0000 7f3401014640 1 -- 172.21.15.43:0/2927845492 wait complete. 2024-04-21T08:40:04.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.223+0000 7f3401014640 1 Processor -- start 2024-04-21T08:40:04.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.223+0000 7f3401014640 1 -- start start 2024-04-21T08:40:04.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.223+0000 7f3401014640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33fc1010a0 0x7f33fc1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.223+0000 7f3401014640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33fc106170 0x7f33fc198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.223+0000 7f3401014640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33fc19cfb0 0x7f33fc19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.223+0000 7f3401014640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1110b0 con 0x7f33fc106170 2024-04-21T08:40:04.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.223+0000 7f3401014640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc110f30 con 0x7f33fc19cfb0 2024-04-21T08:40:04.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.223+0000 7f3401014640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc111230 con 0x7f33fc1010a0 2024-04-21T08:40:04.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.223+0000 7f33f3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33fc106170 0x7f33fc198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.223+0000 7f33f3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33fc106170 0x7f33fc198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48932/0 (socket says 172.21.15.43:48932) 2024-04-21T08:40:04.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.223+0000 7f33f3fff640 1 -- 172.21.15.43:0/638907083 learned_addr learned my addr 172.21.15.43:0/638907083 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.223+0000 7f33fb59e640 1 --2- 172.21.15.43:0/638907083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33fc19cfb0 0x7f33fc19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.224+0000 7f33fad9d640 1 --2- 172.21.15.43:0/638907083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33fc1010a0 0x7f33fc1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.224+0000 7f33f3fff640 1 -- 172.21.15.43:0/638907083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33fc1010a0 msgr2=0x7f33fc1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.224+0000 7f33f3fff640 1 --2- 172.21.15.43:0/638907083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33fc1010a0 0x7f33fc1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.224+0000 7f33f3fff640 1 -- 172.21.15.43:0/638907083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33fc19cfb0 msgr2=0x7f33fc19d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.224+0000 7f33f3fff640 1 --2- 172.21.15.43:0/638907083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33fc19cfb0 0x7f33fc19d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.224+0000 7f33f3fff640 1 -- 172.21.15.43:0/638907083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33fc19daa0 con 0x7f33fc106170 2024-04-21T08:40:04.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.224+0000 7f33fb59e640 1 --2- 172.21.15.43:0/638907083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33fc19cfb0 0x7f33fc19d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:04.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.224+0000 7f33f3fff640 1 --2- 172.21.15.43:0/638907083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33fc106170 0x7f33fc198c30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f33e4036040 tx=0x7f33e402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.224+0000 7f33f8d99640 1 -- 172.21.15.43:0/638907083 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e4043020 con 0x7f33fc106170 2024-04-21T08:40:04.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.224+0000 7f3401014640 1 -- 172.21.15.43:0/638907083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33fc19dd30 con 0x7f33fc106170 2024-04-21T08:40:04.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.224+0000 7f33f8d99640 1 -- 172.21.15.43:0/638907083 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33e402cbe0 con 0x7f33fc106170 2024-04-21T08:40:04.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.224+0000 7f33f8d99640 1 -- 172.21.15.43:0/638907083 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e4003bf0 con 0x7f33fc106170 2024-04-21T08:40:04.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.224+0000 7f3401014640 1 -- 172.21.15.43:0/638907083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33fc1a5550 con 0x7f33fc106170 2024-04-21T08:40:04.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.227+0000 7f33f8d99640 1 -- 172.21.15.43:0/638907083 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f33e4046020 con 0x7f33fc106170 2024-04-21T08:40:04.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.227+0000 7f3403c0a740 1 Processor -- start 2024-04-21T08:40:04.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.227+0000 7f3403c0a740 1 -- start start 2024-04-21T08:40:04.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.227+0000 7f3403c0a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5607571404d0 0x5607571408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.227+0000 7f3403c0a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560757140e80 0x560757149430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.227+0000 7f3403c0a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560757149a00 0x56075714be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.228+0000 7f3403c0a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5607570a51f0 con 0x560757149a00 2024-04-21T08:40:04.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.228+0000 7f3403c0a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5607570424b0 con 0x5607571404d0 2024-04-21T08:40:04.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.228+0000 7f3403c0a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5607570a11b0 con 0x560757140e80 2024-04-21T08:40:04.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.228+0000 7f33f3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560757149a00 0x56075714be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.228+0000 7f33f3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560757149a00 0x56075714be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48942/0 (socket says 172.21.15.43:48942) 2024-04-21T08:40:04.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.228+0000 7f33f3fff640 1 -- 172.21.15.43:0/2056286182 learned_addr learned my addr 172.21.15.43:0/2056286182 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.228+0000 7f33fb59e640 1 --2- 172.21.15.43:0/2056286182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5607571404d0 0x5607571408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.228+0000 7f33fad9d640 1 --2- 172.21.15.43:0/2056286182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560757140e80 0x560757149430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.228+0000 7f33f8d99640 1 --2- 172.21.15.43:0/638907083 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f33e80a3e40 0x7f33e80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.228+0000 7f33f3fff640 1 -- 172.21.15.43:0/2056286182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560757140e80 msgr2=0x560757149430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.228+0000 7f33f3fff640 1 --2- 172.21.15.43:0/2056286182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560757140e80 0x560757149430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.228+0000 7f33f3fff640 1 -- 172.21.15.43:0/2056286182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5607571404d0 msgr2=0x5607571408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.228+0000 7f33f3fff640 1 --2- 172.21.15.43:0/2056286182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5607571404d0 0x5607571408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.228+0000 7f33f3fff640 1 -- 172.21.15.43:0/2056286182 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607570341b0 con 0x560757149a00 2024-04-21T08:40:04.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.228+0000 7f33fb59e640 1 --2- 172.21.15.43:0/638907083 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f33e80a3e40 0x7f33e80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.228+0000 7f33f8d99640 1 -- 172.21.15.43:0/638907083 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f33e40f30d0 con 0x7f33fc106170 2024-04-21T08:40:04.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.228+0000 7f33f3fff640 1 --2- 172.21.15.43:0/2056286182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560757149a00 0x56075714be50 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f33e40f0320 tx=0x7f33e40915f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec9ab33d8e7dcbc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.229+0000 7f33f17fa640 1 -- 172.21.15.43:0/2056286182 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e40be640 con 0x560757149a00 2024-04-21T08:40:04.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.229+0000 7f33f17fa640 1 -- 172.21.15.43:0/2056286182 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33e40bf200 con 0x560757149a00 2024-04-21T08:40:04.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.229+0000 7f33fb59e640 1 --2- 172.21.15.43:0/638907083 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f33e80a3e40 0x7f33e80a6300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f33f4005cc0 tx=0x7f33f4004a40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.229+0000 7f33f17fa640 1 -- 172.21.15.43:0/2056286182 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e40f6d50 con 0x560757149a00 2024-04-21T08:40:04.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.229+0000 7f3403c0a740 1 -- 172.21.15.43:0/2056286182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560757149a00 msgr2=0x56075714be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.229+0000 7f3403c0a740 1 --2- 172.21.15.43:0/2056286182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560757149a00 0x56075714be50 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f33e40f0320 tx=0x7f33e40915f0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.229+0000 7f3403c0a740 1 -- 172.21.15.43:0/2056286182 shutdown_connections 2024-04-21T08:40:04.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.229+0000 7f3403c0a740 1 --2- 172.21.15.43:0/2056286182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560757149a00 0x56075714be50 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.229+0000 7f3403c0a740 1 --2- 172.21.15.43:0/2056286182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560757140e80 0x560757149430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.229+0000 7f3403c0a740 1 --2- 172.21.15.43:0/2056286182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5607571404d0 0x5607571408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.229+0000 7f3403c0a740 1 -- 172.21.15.43:0/2056286182 >> 172.21.15.43:0/2056286182 conn(0x56075713aba0 msgr2=0x56075713af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.230+0000 7f3403c0a740 1 -- 172.21.15.43:0/2056286182 shutdown_connections 2024-04-21T08:40:04.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.230+0000 7f3403c0a740 1 -- 172.21.15.43:0/2056286182 wait complete. 2024-04-21T08:40:04.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.230+0000 7f3403c0a740 1 -- start start 2024-04-21T08:40:04.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.230+0000 7f3403c0a740 1 Processor -- start 2024-04-21T08:40:04.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.230+0000 7f3403c0a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607571404d0 0x560757149260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.230+0000 7f3403c0a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560757140e80 0x5607571465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.230+0000 7f3403c0a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560757149a00 0x560757146b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.230+0000 7f3403c0a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56075714c730 con 0x5607571404d0 2024-04-21T08:40:04.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.230+0000 7f3403c0a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56075705e500 con 0x560757149a00 2024-04-21T08:40:04.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.230+0000 7f3403c0a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56075714c8b0 con 0x560757140e80 2024-04-21T08:40:04.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.230+0000 7f33fad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607571404d0 0x560757149260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.230+0000 7f3403c0a740 10 client.0 _ll_register_callbacks cb 0x56075702cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:04.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.230+0000 7f33fad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607571404d0 0x560757149260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48950/0 (socket says 172.21.15.43:48950) 2024-04-21T08:40:04.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.230+0000 7f33fad9d640 1 -- 172.21.15.43:0/3358866482 learned_addr learned my addr 172.21.15.43:0/3358866482 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.230+0000 7f33f3fff640 1 --2- 172.21.15.43:0/3358866482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560757140e80 0x5607571465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.231+0000 7f33fb59e640 1 --2- 172.21.15.43:0/3358866482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560757149a00 0x560757146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.231+0000 7f33fad9d640 1 -- 172.21.15.43:0/3358866482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560757140e80 msgr2=0x5607571465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.231+0000 7f33fad9d640 1 --2- 172.21.15.43:0/3358866482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560757140e80 0x5607571465f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.231+0000 7f33fad9d640 1 -- 172.21.15.43:0/3358866482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560757149a00 msgr2=0x560757146b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.231+0000 7f33fad9d640 1 --2- 172.21.15.43:0/3358866482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560757149a00 0x560757146b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.231+0000 7f33fad9d640 1 -- 172.21.15.43:0/3358866482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560756db73f0 con 0x5607571404d0 2024-04-21T08:40:04.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.231+0000 7f33fad9d640 1 --2- 172.21.15.43:0/3358866482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607571404d0 0x560757149260 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f33ec010420 tx=0x7f33ec0108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b2ba3150f43f6a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.231+0000 7f33e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:04.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.231+0000 7f33e27fc640 1 -- 172.21.15.43:0/3358866482 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec016070 con 0x5607571404d0 2024-04-21T08:40:04.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.231+0000 7f33e27fc640 1 -- 172.21.15.43:0/3358866482 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f33ec005220 con 0x5607571404d0 2024-04-21T08:40:04.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.231+0000 7f3403c0a740 1 -- 172.21.15.43:0/3358866482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33e40bf200 con 0x5607571404d0 2024-04-21T08:40:04.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.231+0000 7f33e27fc640 1 -- 172.21.15.43:0/3358866482 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f33ec005420 con 0x5607571404d0 2024-04-21T08:40:04.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.231+0000 7f3403c0a740 10 client.16281 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:04.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.231+0000 7f3403c0a740 1 -- 172.21.15.43:0/3358866482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560756e817d0 con 0x5607571404d0 2024-04-21T08:40:04.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.231+0000 7f33e27fc640 1 -- 172.21.15.43:0/3358866482 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33e40bf200 con 0x5607571404d0 2024-04-21T08:40:04.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.231+0000 7f33e27fc640 1 -- 172.21.15.43:0/3358866482 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f33ec00f040 con 0x5607571404d0 2024-04-21T08:40:04.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.231+0000 7f3403c0a740 10 client.16281 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:04.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.231+0000 7f3403c0a740 10 client.16281 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:40:04.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.231+0000 7f3403c0a740 20 client.16281 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:04.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.232+0000 7f3403c0a740 1 --2- 172.21.15.43:0/3358866482 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x560757159910 0x56075715bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.232+0000 7f3403c0a740 4 client.16281 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:04.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.232+0000 7f3403c0a740 1 -- 172.21.15.43:0/3358866482 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560757044930 con 0x560757159910 2024-04-21T08:40:04.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.232+0000 7f33f3fff640 1 --2- 172.21.15.43:0/3358866482 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x560757159910 0x56075715bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.232+0000 7f33f3fff640 1 --2- 172.21.15.43:0/3358866482 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x560757159910 0x56075715bd60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=26565da046e09adf server_cookie=9d7d823c3fef0814 in_seq=0 out_seq=0 2024-04-21T08:40:04.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.232+0000 7f33e27fc640 10 client.16281 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:40:04.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.233+0000 7f33e27fc640 1 -- 172.21.15.43:0/3358866482 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560757044930 con 0x560757159910 2024-04-21T08:40:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.233+0000 7f33e27fc640 10 client.16281 handle_command_reply: tid=0 2024-04-21T08:40:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.233+0000 7f3403c0a740 1 client.16281 shutdown 2024-04-21T08:40:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.233+0000 7f3403c0a740 1 -- 172.21.15.43:0/3358866482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607571404d0 msgr2=0x560757149260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.233+0000 7f3403c0a740 1 --2- 172.21.15.43:0/3358866482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607571404d0 0x560757149260 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f33ec010420 tx=0x7f33ec0108e0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.233+0000 7f3403c0a740 1 -- 172.21.15.43:0/3358866482 shutdown_connections 2024-04-21T08:40:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.233+0000 7f3403c0a740 1 --2- 172.21.15.43:0/3358866482 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x560757159910 0x56075715bd60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.233+0000 7f3403c0a740 1 --2- 172.21.15.43:0/3358866482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560757149a00 0x560757146b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.233+0000 7f33e27fc640 0 client.16281 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:40:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.233+0000 7f3403c0a740 1 --2- 172.21.15.43:0/3358866482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560757140e80 0x5607571465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.233+0000 7f3403c0a740 1 --2- 172.21.15.43:0/3358866482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607571404d0 0x560757149260 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.233+0000 7f3403c0a740 1 -- 172.21.15.43:0/3358866482 >> 172.21.15.43:0/3358866482 conn(0x56075713aba0 msgr2=0x56075713c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.233+0000 7f3403c0a740 1 -- 172.21.15.43:0/3358866482 shutdown_connections 2024-04-21T08:40:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.233+0000 7f3403c0a740 1 -- 172.21.15.43:0/3358866482 wait complete. 2024-04-21T08:40:04.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.234+0000 7f3403c0a740 20 client.16281 trim_cache size 0 max 16384 2024-04-21T08:40:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.247+0000 7f3403c0a740 1 Processor -- start 2024-04-21T08:40:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.247+0000 7f3403c0a740 1 -- start start 2024-04-21T08:40:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.247+0000 7f3403c0a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607571404d0 0x560757139e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.247+0000 7f3403c0a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560757140e80 0x560757155530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.247+0000 7f3403c0a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560757149a00 0x560757148d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.247+0000 7f3403c0a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1af360 con 0x5607571404d0 2024-04-21T08:40:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.247+0000 7f3403c0a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1b0ea0 con 0x560757140e80 2024-04-21T08:40:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.247+0000 7f3403c0a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1afa30 con 0x560757149a00 2024-04-21T08:40:04.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.247+0000 7f33fad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607571404d0 0x560757139e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.247+0000 7f33fad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607571404d0 0x560757139e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48966/0 (socket says 172.21.15.43:48966) 2024-04-21T08:40:04.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.247+0000 7f33fad9d640 1 -- 172.21.15.43:0/1783479011 learned_addr learned my addr 172.21.15.43:0/1783479011 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.247+0000 7f33fb59e640 1 --2- 172.21.15.43:0/1783479011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560757149a00 0x560757148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.247+0000 7f33fad9d640 1 -- 172.21.15.43:0/1783479011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560757149a00 msgr2=0x560757148d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.247+0000 7f33fad9d640 1 --2- 172.21.15.43:0/1783479011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560757149a00 0x560757148d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.247+0000 7f33f3fff640 1 --2- 172.21.15.43:0/1783479011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560757140e80 0x560757155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.247+0000 7f33fad9d640 1 -- 172.21.15.43:0/1783479011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560757140e80 msgr2=0x560757155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.248+0000 7f33fad9d640 1 --2- 172.21.15.43:0/1783479011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560757140e80 0x560757155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.248+0000 7f33fad9d640 1 -- 172.21.15.43:0/1783479011 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56075714c2b0 con 0x5607571404d0 2024-04-21T08:40:04.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.248+0000 7f33fb59e640 1 --2- 172.21.15.43:0/1783479011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560757149a00 0x560757148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:04.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.248+0000 7f33f3fff640 1 --2- 172.21.15.43:0/1783479011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560757140e80 0x560757155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:04.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.248+0000 7f33fad9d640 1 --2- 172.21.15.43:0/1783479011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607571404d0 0x560757139e40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f33ec0056c0 tx=0x7f33ec005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89177fdaa79b90 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.248+0000 7f33f0ff9640 1 -- 172.21.15.43:0/1783479011 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec02b070 con 0x5607571404d0 2024-04-21T08:40:04.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.248+0000 7f33f0ff9640 1 -- 172.21.15.43:0/1783479011 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33ec012a40 con 0x5607571404d0 2024-04-21T08:40:04.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.248+0000 7f33f0ff9640 1 -- 172.21.15.43:0/1783479011 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec016040 con 0x5607571404d0 2024-04-21T08:40:04.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.249+0000 7f3403c0a740 1 -- 172.21.15.43:0/1783479011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607571404d0 msgr2=0x560757139e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.249+0000 7f3403c0a740 1 --2- 172.21.15.43:0/1783479011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607571404d0 0x560757139e40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f33ec0056c0 tx=0x7f33ec005900 comp rx=0 tx=0).stop 2024-04-21T08:40:04.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.249+0000 7f3403c0a740 1 -- 172.21.15.43:0/1783479011 shutdown_connections 2024-04-21T08:40:04.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.249+0000 7f3403c0a740 1 --2- 172.21.15.43:0/1783479011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560757149a00 0x560757148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.249+0000 7f3403c0a740 1 --2- 172.21.15.43:0/1783479011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560757140e80 0x560757155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.249+0000 7f3403c0a740 1 --2- 172.21.15.43:0/1783479011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607571404d0 0x560757139e40 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.249+0000 7f3403c0a740 1 -- 172.21.15.43:0/1783479011 >> 172.21.15.43:0/1783479011 conn(0x56075713aba0 msgr2=0x560757141af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.249+0000 7f3403c0a740 1 -- 172.21.15.43:0/1783479011 shutdown_connections 2024-04-21T08:40:04.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.249+0000 7f3403c0a740 1 -- 172.21.15.43:0/1783479011 wait complete. 2024-04-21T08:40:04.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.249+0000 7f3403c0a740 1 -- start start 2024-04-21T08:40:04.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.249+0000 7f3403c0a740 1 Processor -- start 2024-04-21T08:40:04.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.249+0000 7f3403c0a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607571404d0 0x560757151d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.249+0000 7f3403c0a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560757140e80 0x56075714d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.249+0000 7f3403c0a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560757149a00 0x56075714d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.249+0000 7f3403c0a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1b07d0 con 0x560757149a00 2024-04-21T08:40:04.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.249+0000 7f3403c0a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1b0100 con 0x560757140e80 2024-04-21T08:40:04.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.249+0000 7f3403c0a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1ad340 con 0x5607571404d0 2024-04-21T08:40:04.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.249+0000 7f3403c0a740 10 client.0 _ll_register_callbacks cb 0x7f33ec009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:04.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.249+0000 7f33fb59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560757149a00 0x56075714d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.249+0000 7f33fb59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560757149a00 0x56075714d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48968/0 (socket says 172.21.15.43:48968) 2024-04-21T08:40:04.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.249+0000 7f33fb59e640 1 -- 172.21.15.43:0/3104181076 learned_addr learned my addr 172.21.15.43:0/3104181076 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.250+0000 7f33f3fff640 1 --2- 172.21.15.43:0/3104181076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560757140e80 0x56075714d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.250+0000 7f33fad9d640 1 --2- 172.21.15.43:0/3104181076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607571404d0 0x560757151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.250+0000 7f33fb59e640 1 -- 172.21.15.43:0/3104181076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607571404d0 msgr2=0x560757151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.250+0000 7f33fb59e640 1 --2- 172.21.15.43:0/3104181076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607571404d0 0x560757151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.250+0000 7f33fb59e640 1 -- 172.21.15.43:0/3104181076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560757140e80 msgr2=0x56075714d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.250+0000 7f33fb59e640 1 --2- 172.21.15.43:0/3104181076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560757140e80 0x56075714d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.250+0000 7f33fb59e640 1 -- 172.21.15.43:0/3104181076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56075714de40 con 0x560757149a00 2024-04-21T08:40:04.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.250+0000 7f33fad9d640 1 --2- 172.21.15.43:0/3104181076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607571404d0 0x560757151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:04.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.250+0000 7f33f3fff640 1 --2- 172.21.15.43:0/3104181076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560757140e80 0x56075714d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:04.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.250+0000 7f33fb59e640 1 --2- 172.21.15.43:0/3104181076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560757149a00 0x56075714d640 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f33f401dee0 tx=0x7f33f4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310256ffdd32db92 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.250+0000 7f33e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:04.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.250+0000 7f33e27fc640 1 -- 172.21.15.43:0/3104181076 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f401b440 con 0x560757149a00 2024-04-21T08:40:04.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.250+0000 7f3403c0a740 1 -- 172.21.15.43:0/3104181076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33ec012a40 con 0x560757149a00 2024-04-21T08:40:04.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.250+0000 7f33e27fc640 1 -- 172.21.15.43:0/3104181076 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f33f401b5e0 con 0x560757149a00 2024-04-21T08:40:04.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.250+0000 7f33e27fc640 1 -- 172.21.15.43:0/3104181076 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f33f401b7e0 con 0x560757149a00 2024-04-21T08:40:04.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.250+0000 7f3403c0a740 10 client.16287 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:04.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.250+0000 7f3403c0a740 1 -- 172.21.15.43:0/3104181076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5607571bfe50 con 0x560757149a00 2024-04-21T08:40:04.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.250+0000 7f33e27fc640 1 -- 172.21.15.43:0/3104181076 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33ec012a40 con 0x560757149a00 2024-04-21T08:40:04.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.251+0000 7f33e27fc640 1 -- 172.21.15.43:0/3104181076 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f33f4027860 con 0x560757149a00 2024-04-21T08:40:04.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.251+0000 7f3403c0a740 10 client.16287 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:04.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.251+0000 7f3403c0a740 10 client.16287 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:40:04.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.251+0000 7f3403c0a740 20 client.16287 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:04.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.251+0000 7f3403c0a740 1 --2- 172.21.15.43:0/3104181076 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5607571c0570 0x5607571c2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.251+0000 7f3403c0a740 4 client.16287 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:40:04.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.251+0000 7f3403c0a740 1 -- 172.21.15.43:0/3104181076 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5607570b2c10 con 0x5607571c0570 2024-04-21T08:40:04.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.251+0000 7f33fad9d640 1 --2- 172.21.15.43:0/3104181076 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5607571c0570 0x5607571c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.251+0000 7f33e27fc640 10 client.16287 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:40:04.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.251+0000 7f33fad9d640 1 --2- 172.21.15.43:0/3104181076 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5607571c0570 0x5607571c2960 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f5d66f25cae0614a server_cookie=e5a6880599c1dfca in_seq=0 out_seq=0 2024-04-21T08:40:04.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.252+0000 7f33e27fc640 1 -- 172.21.15.43:0/3104181076 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5607570b2c10 con 0x5607571c0570 2024-04-21T08:40:04.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.252+0000 7f33e27fc640 10 client.16287 handle_command_reply: tid=0 2024-04-21T08:40:04.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.252+0000 7f3403c0a740 1 client.16287 shutdown 2024-04-21T08:40:04.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.252+0000 7f3403c0a740 1 -- 172.21.15.43:0/3104181076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560757149a00 msgr2=0x56075714d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.252+0000 7f3403c0a740 1 --2- 172.21.15.43:0/3104181076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560757149a00 0x56075714d640 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f33f401dee0 tx=0x7f33f4004ab0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.252+0000 7f3403c0a740 1 -- 172.21.15.43:0/3104181076 shutdown_connections 2024-04-21T08:40:04.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.252+0000 7f3403c0a740 1 --2- 172.21.15.43:0/3104181076 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5607571c0570 0x5607571c2960 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.252+0000 7f3403c0a740 1 --2- 172.21.15.43:0/3104181076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560757149a00 0x56075714d640 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.252+0000 7f33e27fc640 0 client.16287 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:40:04.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.252+0000 7f3403c0a740 1 --2- 172.21.15.43:0/3104181076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560757140e80 0x56075714d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.252+0000 7f3403c0a740 1 --2- 172.21.15.43:0/3104181076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607571404d0 0x560757151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.252+0000 7f3403c0a740 1 -- 172.21.15.43:0/3104181076 >> 172.21.15.43:0/3104181076 conn(0x56075713aba0 msgr2=0x560757145e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.252+0000 7f3403c0a740 1 -- 172.21.15.43:0/3104181076 shutdown_connections 2024-04-21T08:40:04.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.252+0000 7f3403c0a740 1 -- 172.21.15.43:0/3104181076 wait complete. 2024-04-21T08:40:04.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.252+0000 7f3403c0a740 20 client.16287 trim_cache size 0 max 16384 2024-04-21T08:40:04.259 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:40:04.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.252+0000 7f3401014640 1 -- 172.21.15.43:0/638907083 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f33e80a3e40 msgr2=0x7f33e80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.252+0000 7f3401014640 1 --2- 172.21.15.43:0/638907083 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f33e80a3e40 0x7f33e80a6300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f33f4005cc0 tx=0x7f33f4004a40 comp rx=0 tx=0).stop 2024-04-21T08:40:04.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.253+0000 7f3401014640 1 -- 172.21.15.43:0/638907083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33fc106170 msgr2=0x7f33fc198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.253+0000 7f3401014640 1 --2- 172.21.15.43:0/638907083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33fc106170 0x7f33fc198c30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f33e4036040 tx=0x7f33e402c7e0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.253+0000 7f3401014640 1 -- 172.21.15.43:0/638907083 shutdown_connections 2024-04-21T08:40:04.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.253+0000 7f3401014640 1 --2- 172.21.15.43:0/638907083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33fc19cfb0 0x7f33fc19d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.253+0000 7f3401014640 1 --2- 172.21.15.43:0/638907083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33fc106170 0x7f33fc198c30 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.253+0000 7f3401014640 1 --2- 172.21.15.43:0/638907083 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f33e80a3e40 0x7f33e80a6300 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.253+0000 7f3401014640 1 --2- 172.21.15.43:0/638907083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33fc1010a0 0x7f33fc1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.253+0000 7f3401014640 1 -- 172.21.15.43:0/638907083 >> 172.21.15.43:0/638907083 conn(0x7f33fc077eb0 msgr2=0x7f33fc06a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.253+0000 7f3401014640 1 -- 172.21.15.43:0/638907083 shutdown_connections 2024-04-21T08:40:04.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.253+0000 7f3401014640 1 -- 172.21.15.43:0/638907083 wait complete. 2024-04-21T08:40:04.262 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:03 smithi043 ceph-mon[27978]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 27 op/s 2024-04-21T08:40:04.263 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:40:04.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:03 smithi112 ceph-mon[31085]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 27 op/s 2024-04-21T08:40:04.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.368+0000 7f1d578e9640 1 -- 172.21.15.43:0/1484612214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d50106170 msgr2=0x7f1d5010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.368+0000 7f1d578e9640 1 --2- 172.21.15.43:0/1484612214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d50106170 0x7f1d5010dd40 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c009ca0 tx=0x7f1d4c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.368+0000 7f1d578e9640 1 -- 172.21.15.43:0/1484612214 shutdown_connections 2024-04-21T08:40:04.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.368+0000 7f1d578e9640 1 --2- 172.21.15.43:0/1484612214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d50106170 0x7f1d5010dd40 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.368+0000 7f1d578e9640 1 --2- 172.21.15.43:0/1484612214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d50101aa0 0x7f1d50105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.368+0000 7f1d578e9640 1 --2- 172.21.15.43:0/1484612214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d501010f0 0x7f1d501014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.368+0000 7f1d578e9640 1 -- 172.21.15.43:0/1484612214 >> 172.21.15.43:0/1484612214 conn(0x7f1d50077fb0 msgr2=0x7f1d500783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.368+0000 7f1d578e9640 1 -- 172.21.15.43:0/1484612214 shutdown_connections 2024-04-21T08:40:04.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.368+0000 7f1d578e9640 1 -- 172.21.15.43:0/1484612214 wait complete. 2024-04-21T08:40:04.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.368+0000 7f1d578e9640 1 Processor -- start 2024-04-21T08:40:04.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.368+0000 7f1d578e9640 1 -- start start 2024-04-21T08:40:04.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.368+0000 7f1d578e9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d501010f0 0x7f1d5010e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.368+0000 7f1d578e9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d50106170 0x7f1d5010ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.369+0000 7f1d578e9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d5010f150 0x7f1d501a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.369+0000 7f1d578e9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1d501121f0 con 0x7f1d501010f0 2024-04-21T08:40:04.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.369+0000 7f1d578e9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1d50112070 con 0x7f1d50106170 2024-04-21T08:40:04.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.369+0000 7f1d578e9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1d50112370 con 0x7f1d5010f150 2024-04-21T08:40:04.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.369+0000 7f1d55685640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d501010f0 0x7f1d5010e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.369+0000 7f1d55685640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d501010f0 0x7f1d5010e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48980/0 (socket says 172.21.15.43:48980) 2024-04-21T08:40:04.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.369+0000 7f1d55685640 1 -- 172.21.15.43:0/3684388537 learned_addr learned my addr 172.21.15.43:0/3684388537 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.369+0000 7f1d54e84640 1 --2- 172.21.15.43:0/3684388537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d50106170 0x7f1d5010ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.369+0000 7f1d55e86640 1 --2- 172.21.15.43:0/3684388537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d5010f150 0x7f1d501a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.369+0000 7f1d55685640 1 -- 172.21.15.43:0/3684388537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d5010f150 msgr2=0x7f1d501a6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.369+0000 7f1d55685640 1 --2- 172.21.15.43:0/3684388537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d5010f150 0x7f1d501a6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.369+0000 7f1d55685640 1 -- 172.21.15.43:0/3684388537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d50106170 msgr2=0x7f1d5010ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.369+0000 7f1d55685640 1 --2- 172.21.15.43:0/3684388537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d50106170 0x7f1d5010ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.369+0000 7f1d55685640 1 -- 172.21.15.43:0/3684388537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d501a6f00 con 0x7f1d501010f0 2024-04-21T08:40:04.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.369+0000 7f1d54e84640 1 --2- 172.21.15.43:0/3684388537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d50106170 0x7f1d5010ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:04.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.369+0000 7f1d55e86640 1 --2- 172.21.15.43:0/3684388537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d5010f150 0x7f1d501a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:04.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.369+0000 7f1d55685640 1 --2- 172.21.15.43:0/3684388537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d501010f0 0x7f1d5010e6d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800d950 tx=0x7f1d4800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.369+0000 7f1d467fc640 1 -- 172.21.15.43:0/3684388537 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48017030 con 0x7f1d501010f0 2024-04-21T08:40:04.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.369+0000 7f1d467fc640 1 -- 172.21.15.43:0/3684388537 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d48004020 con 0x7f1d501010f0 2024-04-21T08:40:04.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.369+0000 7f1d578e9640 1 -- 172.21.15.43:0/3684388537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d501a71f0 con 0x7f1d501010f0 2024-04-21T08:40:04.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.369+0000 7f1d467fc640 1 -- 172.21.15.43:0/3684388537 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d480047a0 con 0x7f1d501010f0 2024-04-21T08:40:04.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.369+0000 7f1d578e9640 1 -- 172.21.15.43:0/3684388537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d501a7740 con 0x7f1d501010f0 2024-04-21T08:40:04.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.372+0000 7f1d467fc640 1 -- 172.21.15.43:0/3684388537 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f1d48009a30 con 0x7f1d501010f0 2024-04-21T08:40:04.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.372+0000 7f1d5a5b0740 1 Processor -- start 2024-04-21T08:40:04.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.372+0000 7f1d5a5b0740 1 -- start start 2024-04-21T08:40:04.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.372+0000 7f1d467fc640 1 --2- 172.21.15.43:0/3684388537 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1d3c0a3e40 0x7f1d3c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.372+0000 7f1d5a5b0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562bd28644d0 0x562bd28648b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.372+0000 7f1d5a5b0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd2864e80 0x562bd286d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.372+0000 7f1d5a5b0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562bd286da00 0x562bd286fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.372+0000 7f1d5a5b0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562bd27c91f0 con 0x562bd2864e80 2024-04-21T08:40:04.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.372+0000 7f1d5a5b0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562bd27664b0 con 0x562bd286da00 2024-04-21T08:40:04.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.372+0000 7f1d5a5b0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562bd27c51b0 con 0x562bd28644d0 2024-04-21T08:40:04.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.373+0000 7f1d55e86640 1 --2- 172.21.15.43:0/3684388537 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1d3c0a3e40 0x7f1d3c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.373+0000 7f1d54e84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd2864e80 0x562bd286d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.373+0000 7f1d54e84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd2864e80 0x562bd286d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48988/0 (socket says 172.21.15.43:48988) 2024-04-21T08:40:04.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.373+0000 7f1d54e84640 1 -- 172.21.15.43:0/2636469165 learned_addr learned my addr 172.21.15.43:0/2636469165 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.373+0000 7f1d467fc640 1 -- 172.21.15.43:0/3684388537 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f1d480d0970 con 0x7f1d501010f0 2024-04-21T08:40:04.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.373+0000 7f1d55685640 1 --2- 172.21.15.43:0/2636469165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562bd28644d0 0x562bd28648b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.373+0000 7f1d55e86640 1 --2- 172.21.15.43:0/2636469165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562bd286da00 0x562bd286fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.373+0000 7f1d54e84640 1 -- 172.21.15.43:0/2636469165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562bd28644d0 msgr2=0x562bd28648b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.373+0000 7f1d54e84640 1 --2- 172.21.15.43:0/2636469165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562bd28644d0 0x562bd28648b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.373+0000 7f1d54e84640 1 -- 172.21.15.43:0/2636469165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562bd286da00 msgr2=0x562bd286fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.373+0000 7f1d54e84640 1 --2- 172.21.15.43:0/2636469165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562bd286da00 0x562bd286fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.373+0000 7f1d54e84640 1 -- 172.21.15.43:0/2636469165 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bd27581b0 con 0x562bd2864e80 2024-04-21T08:40:04.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.373+0000 7f1d55e86640 1 --2- 172.21.15.43:0/3684388537 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1d3c0a3e40 0x7f1d3c0a6300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c005e30 tx=0x7f1d4c02ceb0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.373+0000 7f1d54e84640 1 --2- 172.21.15.43:0/2636469165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd2864e80 0x562bd286d430 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1d40008b80 tx=0x7f1d4000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3d872103f2a9aa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.373+0000 7f1d3b7fe640 1 -- 172.21.15.43:0/2636469165 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40014070 con 0x562bd2864e80 2024-04-21T08:40:04.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.373+0000 7f1d3b7fe640 1 -- 172.21.15.43:0/2636469165 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d400047a0 con 0x562bd2864e80 2024-04-21T08:40:04.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.373+0000 7f1d3b7fe640 1 -- 172.21.15.43:0/2636469165 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40010690 con 0x562bd2864e80 2024-04-21T08:40:04.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.374+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/2636469165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd2864e80 msgr2=0x562bd286d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.374+0000 7f1d5a5b0740 1 --2- 172.21.15.43:0/2636469165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd2864e80 0x562bd286d430 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1d40008b80 tx=0x7f1d4000e720 comp rx=0 tx=0).stop 2024-04-21T08:40:04.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.374+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/2636469165 shutdown_connections 2024-04-21T08:40:04.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.374+0000 7f1d5a5b0740 1 --2- 172.21.15.43:0/2636469165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562bd286da00 0x562bd286fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.374+0000 7f1d5a5b0740 1 --2- 172.21.15.43:0/2636469165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd2864e80 0x562bd286d430 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.374+0000 7f1d5a5b0740 1 --2- 172.21.15.43:0/2636469165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562bd28644d0 0x562bd28648b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.374+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/2636469165 >> 172.21.15.43:0/2636469165 conn(0x562bd285eba0 msgr2=0x562bd285ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.374+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/2636469165 shutdown_connections 2024-04-21T08:40:04.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.374+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/2636469165 wait complete. 2024-04-21T08:40:04.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.374+0000 7f1d5a5b0740 1 -- start start 2024-04-21T08:40:04.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.374+0000 7f1d5a5b0740 1 Processor -- start 2024-04-21T08:40:04.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.375+0000 7f1d5a5b0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd28644d0 0x562bd286d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.375+0000 7f1d5a5b0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562bd2864e80 0x562bd286a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.375+0000 7f1d5a5b0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562bd286da00 0x562bd286ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.375+0000 7f1d5a5b0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562bd2870970 con 0x562bd28644d0 2024-04-21T08:40:04.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.375+0000 7f1d5a5b0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562bd2782500 con 0x562bd2864e80 2024-04-21T08:40:04.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.375+0000 7f1d5a5b0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562bd2870af0 con 0x562bd286da00 2024-04-21T08:40:04.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.375+0000 7f1d5a5b0740 10 client.0 _ll_register_callbacks cb 0x562bd2750b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:04.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.375+0000 7f1d55685640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd28644d0 0x562bd286d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.375+0000 7f1d55685640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd28644d0 0x562bd286d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49004/0 (socket says 172.21.15.43:49004) 2024-04-21T08:40:04.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.375+0000 7f1d55685640 1 -- 172.21.15.43:0/2885065442 learned_addr learned my addr 172.21.15.43:0/2885065442 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.375+0000 7f1d55e86640 1 --2- 172.21.15.43:0/2885065442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562bd286da00 0x562bd286ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.375+0000 7f1d55685640 1 -- 172.21.15.43:0/2885065442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562bd286da00 msgr2=0x562bd286ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.375+0000 7f1d55685640 1 --2- 172.21.15.43:0/2885065442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562bd286da00 0x562bd286ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.375+0000 7f1d54e84640 1 --2- 172.21.15.43:0/2885065442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562bd2864e80 0x562bd286a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.375+0000 7f1d55685640 1 -- 172.21.15.43:0/2885065442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562bd2864e80 msgr2=0x562bd286a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.375+0000 7f1d55685640 1 --2- 172.21.15.43:0/2885065442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562bd2864e80 0x562bd286a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.375+0000 7f1d55685640 1 -- 172.21.15.43:0/2885065442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bd24db3f0 con 0x562bd28644d0 2024-04-21T08:40:04.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.375+0000 7f1d55e86640 1 --2- 172.21.15.43:0/2885065442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562bd286da00 0x562bd286ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:04.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.375+0000 7f1d55685640 1 --2- 172.21.15.43:0/2885065442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd28644d0 0x562bd286d320 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1d48098750 tx=0x7f1d4809bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23083d7b57d85dc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.375+0000 7f1d38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:04.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.375+0000 7f1d38ff9640 1 -- 172.21.15.43:0/2885065442 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d480a06a0 con 0x562bd28644d0 2024-04-21T08:40:04.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.376+0000 7f1d38ff9640 1 -- 172.21.15.43:0/2885065442 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f1d4809a5f0 con 0x562bd28644d0 2024-04-21T08:40:04.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.376+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/2885065442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d400047a0 con 0x562bd28644d0 2024-04-21T08:40:04.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.376+0000 7f1d38ff9640 1 -- 172.21.15.43:0/2885065442 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1d480a4f60 con 0x562bd28644d0 2024-04-21T08:40:04.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.376+0000 7f1d5a5b0740 10 client.16299 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:04.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.376+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/2885065442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562bd25a57d0 con 0x562bd28644d0 2024-04-21T08:40:04.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.376+0000 7f1d38ff9640 1 -- 172.21.15.43:0/2885065442 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d400047a0 con 0x562bd28644d0 2024-04-21T08:40:04.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.376+0000 7f1d38ff9640 1 -- 172.21.15.43:0/2885065442 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1d480cde50 con 0x562bd28644d0 2024-04-21T08:40:04.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.376+0000 7f1d5a5b0740 10 client.16299 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:04.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.376+0000 7f1d5a5b0740 10 client.16299 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:40:04.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.376+0000 7f1d5a5b0740 20 client.16299 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:04.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.376+0000 7f1d5a5b0740 1 --2- 172.21.15.43:0/2885065442 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x562bd287dbb0 0x562bd2880000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.376+0000 7f1d5a5b0740 4 client.16299 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:04.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.376+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/2885065442 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bd2768930 con 0x562bd287dbb0 2024-04-21T08:40:04.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.376+0000 7f1d54e84640 1 --2- 172.21.15.43:0/2885065442 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x562bd287dbb0 0x562bd2880000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.377+0000 7f1d38ff9640 10 client.16299 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:40:04.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.377+0000 7f1d54e84640 1 --2- 172.21.15.43:0/2885065442 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x562bd287dbb0 0x562bd2880000 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=47296a6ac36e91e0 server_cookie=fc2bb5bf42a75ca8 in_seq=0 out_seq=0 2024-04-21T08:40:04.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.378+0000 7f1d38ff9640 1 -- 172.21.15.43:0/2885065442 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562bd2768930 con 0x562bd287dbb0 2024-04-21T08:40:04.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.378+0000 7f1d38ff9640 10 client.16299 handle_command_reply: tid=0 2024-04-21T08:40:04.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.378+0000 7f1d5a5b0740 1 client.16299 shutdown 2024-04-21T08:40:04.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.378+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/2885065442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd28644d0 msgr2=0x562bd286d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.378+0000 7f1d5a5b0740 1 --2- 172.21.15.43:0/2885065442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd28644d0 0x562bd286d320 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1d48098750 tx=0x7f1d4809bd10 comp rx=0 tx=0).stop 2024-04-21T08:40:04.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.378+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/2885065442 shutdown_connections 2024-04-21T08:40:04.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.378+0000 7f1d5a5b0740 1 --2- 172.21.15.43:0/2885065442 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x562bd287dbb0 0x562bd2880000 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.378+0000 7f1d5a5b0740 1 --2- 172.21.15.43:0/2885065442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562bd286da00 0x562bd286ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.378+0000 7f1d5a5b0740 1 --2- 172.21.15.43:0/2885065442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562bd2864e80 0x562bd286a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.378+0000 7f1d38ff9640 0 client.16299 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:40:04.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.378+0000 7f1d5a5b0740 1 --2- 172.21.15.43:0/2885065442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd28644d0 0x562bd286d320 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.378+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/2885065442 >> 172.21.15.43:0/2885065442 conn(0x562bd285eba0 msgr2=0x562bd28608f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.378+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/2885065442 shutdown_connections 2024-04-21T08:40:04.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.378+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/2885065442 wait complete. 2024-04-21T08:40:04.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.378+0000 7f1d5a5b0740 20 client.16299 trim_cache size 0 max 16384 2024-04-21T08:40:04.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.391+0000 7f1d5a5b0740 1 Processor -- start 2024-04-21T08:40:04.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.391+0000 7f1d5a5b0740 1 -- start start 2024-04-21T08:40:04.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.391+0000 7f1d5a5b0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd28644d0 0x562bd2879830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.392+0000 7f1d5a5b0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562bd2864e80 0x562bd286cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.392+0000 7f1d5a5b0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562bd286da00 0x562bd286d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.392+0000 7f1d5a5b0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1d501b15a0 con 0x562bd28644d0 2024-04-21T08:40:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.392+0000 7f1d5a5b0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1d501b30e0 con 0x562bd2864e80 2024-04-21T08:40:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.392+0000 7f1d5a5b0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1d501b1c70 con 0x562bd286da00 2024-04-21T08:40:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.392+0000 7f1d55685640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd28644d0 0x562bd2879830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.392+0000 7f1d55685640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd28644d0 0x562bd2879830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49012/0 (socket says 172.21.15.43:49012) 2024-04-21T08:40:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.392+0000 7f1d55685640 1 -- 172.21.15.43:0/336274429 learned_addr learned my addr 172.21.15.43:0/336274429 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.392+0000 7f1d55e86640 1 --2- 172.21.15.43:0/336274429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562bd286da00 0x562bd286d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.392+0000 7f1d55685640 1 -- 172.21.15.43:0/336274429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562bd286da00 msgr2=0x562bd286d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.392+0000 7f1d55685640 1 --2- 172.21.15.43:0/336274429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562bd286da00 0x562bd286d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.392+0000 7f1d55685640 1 -- 172.21.15.43:0/336274429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562bd2864e80 msgr2=0x562bd286cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.392+0000 7f1d55685640 1 --2- 172.21.15.43:0/336274429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562bd2864e80 0x562bd286cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.392+0000 7f1d55685640 1 -- 172.21.15.43:0/336274429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bd286afc0 con 0x562bd28644d0 2024-04-21T08:40:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.392+0000 7f1d55e86640 1 --2- 172.21.15.43:0/336274429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562bd286da00 0x562bd286d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.392+0000 7f1d55685640 1 --2- 172.21.15.43:0/336274429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd28644d0 0x562bd2879830 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1d4809bff0 tx=0x7f1d48000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129efb30bc7f03eb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.392+0000 7f1d3affd640 1 -- 172.21.15.43:0/336274429 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48014070 con 0x562bd28644d0 2024-04-21T08:40:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.392+0000 7f1d3affd640 1 -- 172.21.15.43:0/336274429 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d480a06a0 con 0x562bd28644d0 2024-04-21T08:40:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.392+0000 7f1d3affd640 1 -- 172.21.15.43:0/336274429 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48096e60 con 0x562bd28644d0 2024-04-21T08:40:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.393+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/336274429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd28644d0 msgr2=0x562bd2879830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.393+0000 7f1d5a5b0740 1 --2- 172.21.15.43:0/336274429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd28644d0 0x562bd2879830 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1d4809bff0 tx=0x7f1d48000bc0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.393+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/336274429 shutdown_connections 2024-04-21T08:40:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.393+0000 7f1d5a5b0740 1 --2- 172.21.15.43:0/336274429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562bd286da00 0x562bd286d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.393+0000 7f1d5a5b0740 1 --2- 172.21.15.43:0/336274429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562bd2864e80 0x562bd286cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.393+0000 7f1d5a5b0740 1 --2- 172.21.15.43:0/336274429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd28644d0 0x562bd2879830 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.393+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/336274429 >> 172.21.15.43:0/336274429 conn(0x562bd285eba0 msgr2=0x562bd2860810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.393+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/336274429 shutdown_connections 2024-04-21T08:40:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.393+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/336274429 wait complete. 2024-04-21T08:40:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.393+0000 7f1d5a5b0740 1 -- start start 2024-04-21T08:40:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.393+0000 7f1d5a5b0740 1 Processor -- start 2024-04-21T08:40:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.394+0000 7f1d5a5b0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562bd28644d0 0x562bd2876010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.394+0000 7f1d5a5b0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562bd2864e80 0x562bd2866fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.394+0000 7f1d5a5b0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd286da00 0x562bd28674e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.394+0000 7f1d5a5b0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1d501b2a10 con 0x562bd286da00 2024-04-21T08:40:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.394+0000 7f1d5a5b0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1d501b2340 con 0x562bd2864e80 2024-04-21T08:40:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.394+0000 7f1d5a5b0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1d501af580 con 0x562bd28644d0 2024-04-21T08:40:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.394+0000 7f1d5a5b0740 10 client.0 _ll_register_callbacks cb 0x7f1d4809be30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.394+0000 7f1d55e86640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd286da00 0x562bd28674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.394+0000 7f1d55e86640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd286da00 0x562bd28674e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49016/0 (socket says 172.21.15.43:49016) 2024-04-21T08:40:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.394+0000 7f1d55e86640 1 -- 172.21.15.43:0/3681737462 learned_addr learned my addr 172.21.15.43:0/3681737462 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.394+0000 7f1d55685640 1 --2- 172.21.15.43:0/3681737462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562bd28644d0 0x562bd2876010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.394+0000 7f1d54e84640 1 --2- 172.21.15.43:0/3681737462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562bd2864e80 0x562bd2866fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.394+0000 7f1d55e86640 1 -- 172.21.15.43:0/3681737462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562bd28644d0 msgr2=0x562bd2876010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.394+0000 7f1d55e86640 1 --2- 172.21.15.43:0/3681737462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562bd28644d0 0x562bd2876010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.394+0000 7f1d55e86640 1 -- 172.21.15.43:0/3681737462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562bd2864e80 msgr2=0x562bd2866fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.394+0000 7f1d55e86640 1 --2- 172.21.15.43:0/3681737462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562bd2864e80 0x562bd2866fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.394+0000 7f1d55e86640 1 -- 172.21.15.43:0/3681737462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bd2867ba0 con 0x562bd286da00 2024-04-21T08:40:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.394+0000 7f1d55e86640 1 --2- 172.21.15.43:0/3681737462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd286da00 0x562bd28674e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c037040 tx=0x7f1d4c02e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a01cfd5b1e154a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.394+0000 7f1d38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.394+0000 7f1d38ff9640 1 -- 172.21.15.43:0/3681737462 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c049040 con 0x562bd286da00 2024-04-21T08:40:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.395+0000 7f1d38ff9640 1 -- 172.21.15.43:0/3681737462 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f1d4c004040 con 0x562bd286da00 2024-04-21T08:40:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.395+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/3681737462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d480a06a0 con 0x562bd286da00 2024-04-21T08:40:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.395+0000 7f1d38ff9640 1 -- 172.21.15.43:0/3681737462 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1d4c05b050 con 0x562bd286da00 2024-04-21T08:40:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.395+0000 7f1d5a5b0740 10 client.16305 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.395+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/3681737462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562bd28e3f60 con 0x562bd286da00 2024-04-21T08:40:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.395+0000 7f1d38ff9640 1 -- 172.21.15.43:0/3681737462 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d480a06a0 con 0x562bd286da00 2024-04-21T08:40:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.395+0000 7f1d38ff9640 1 -- 172.21.15.43:0/3681737462 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1d4c034740 con 0x562bd286da00 2024-04-21T08:40:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.395+0000 7f1d5a5b0740 10 client.16305 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.395+0000 7f1d5a5b0740 10 client.16305 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:40:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.395+0000 7f1d5a5b0740 20 client.16305 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.395+0000 7f1d5a5b0740 1 --2- 172.21.15.43:0/3681737462 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x562bd28e4730 0x562bd28e6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.395+0000 7f1d5a5b0740 4 client.16305 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:40:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.395+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/3681737462 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562bd27d6c10 con 0x562bd28e4730 2024-04-21T08:40:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.395+0000 7f1d55685640 1 --2- 172.21.15.43:0/3681737462 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x562bd28e4730 0x562bd28e6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.396+0000 7f1d38ff9640 10 client.16305 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:40:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.396+0000 7f1d55685640 1 --2- 172.21.15.43:0/3681737462 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x562bd28e4730 0x562bd28e6b50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bf592be0f13da381 server_cookie=ed8b134ed518c7a2 in_seq=0 out_seq=0 2024-04-21T08:40:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.396+0000 7f1d38ff9640 1 -- 172.21.15.43:0/3681737462 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562bd27d6c10 con 0x562bd28e4730 2024-04-21T08:40:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.396+0000 7f1d38ff9640 10 client.16305 handle_command_reply: tid=0 2024-04-21T08:40:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.396+0000 7f1d5a5b0740 1 client.16305 shutdown 2024-04-21T08:40:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.396+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/3681737462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd286da00 msgr2=0x562bd28674e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.396+0000 7f1d5a5b0740 1 --2- 172.21.15.43:0/3681737462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd286da00 0x562bd28674e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c037040 tx=0x7f1d4c02e050 comp rx=0 tx=0).stop 2024-04-21T08:40:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.396+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/3681737462 shutdown_connections 2024-04-21T08:40:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.396+0000 7f1d5a5b0740 1 --2- 172.21.15.43:0/3681737462 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x562bd28e4730 0x562bd28e6b50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.396+0000 7f1d5a5b0740 1 --2- 172.21.15.43:0/3681737462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562bd286da00 0x562bd28674e0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.396+0000 7f1d5a5b0740 1 --2- 172.21.15.43:0/3681737462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562bd2864e80 0x562bd2866fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.396+0000 7f1d38ff9640 0 client.16305 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:40:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.396+0000 7f1d5a5b0740 1 --2- 172.21.15.43:0/3681737462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562bd28644d0 0x562bd2876010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.396+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/3681737462 >> 172.21.15.43:0/3681737462 conn(0x562bd285eba0 msgr2=0x562bd285d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.396+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/3681737462 shutdown_connections 2024-04-21T08:40:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.396+0000 7f1d5a5b0740 1 -- 172.21.15.43:0/3681737462 wait complete. 2024-04-21T08:40:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.396+0000 7f1d5a5b0740 20 client.16305 trim_cache size 0 max 16384 2024-04-21T08:40:04.403 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:40:04.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.397+0000 7f1d578e9640 1 -- 172.21.15.43:0/3684388537 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1d3c0a3e40 msgr2=0x7f1d3c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.397+0000 7f1d578e9640 1 --2- 172.21.15.43:0/3684388537 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1d3c0a3e40 0x7f1d3c0a6300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c005e30 tx=0x7f1d4c02ceb0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.397+0000 7f1d578e9640 1 -- 172.21.15.43:0/3684388537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d501010f0 msgr2=0x7f1d5010e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.397+0000 7f1d578e9640 1 --2- 172.21.15.43:0/3684388537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d501010f0 0x7f1d5010e6d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800d950 tx=0x7f1d4800de10 comp rx=0 tx=0).stop 2024-04-21T08:40:04.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.397+0000 7f1d578e9640 1 -- 172.21.15.43:0/3684388537 shutdown_connections 2024-04-21T08:40:04.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.397+0000 7f1d578e9640 1 --2- 172.21.15.43:0/3684388537 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1d3c0a3e40 0x7f1d3c0a6300 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.397+0000 7f1d578e9640 1 --2- 172.21.15.43:0/3684388537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d5010f150 0x7f1d501a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.397+0000 7f1d578e9640 1 --2- 172.21.15.43:0/3684388537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d50106170 0x7f1d5010ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.397+0000 7f1d578e9640 1 --2- 172.21.15.43:0/3684388537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d501010f0 0x7f1d5010e6d0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.397+0000 7f1d578e9640 1 -- 172.21.15.43:0/3684388537 >> 172.21.15.43:0/3684388537 conn(0x7f1d50077fb0 msgr2=0x7f1d500feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.397+0000 7f1d578e9640 1 -- 172.21.15.43:0/3684388537 shutdown_connections 2024-04-21T08:40:04.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.397+0000 7f1d578e9640 1 -- 172.21.15.43:0/3684388537 wait complete. 2024-04-21T08:40:04.407 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:40:04.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:03 smithi175 ceph-mon[31920]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 27 op/s 2024-04-21T08:40:04.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.512+0000 7f160542e640 1 -- 172.21.15.43:0/2187901038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1600102b40 msgr2=0x7f1600106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.512+0000 7f160542e640 1 --2- 172.21.15.43:0/2187901038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1600102b40 0x7f1600106b80 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009900 tx=0x7f15e802bc30 comp rx=0 tx=0).stop 2024-04-21T08:40:04.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.513+0000 7f160542e640 1 -- 172.21.15.43:0/2187901038 shutdown_connections 2024-04-21T08:40:04.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.513+0000 7f160542e640 1 --2- 172.21.15.43:0/2187901038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1600107260 0x7f160010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.513+0000 7f160542e640 1 --2- 172.21.15.43:0/2187901038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1600102b40 0x7f1600106b80 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.513+0000 7f160542e640 1 --2- 172.21.15.43:0/2187901038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1600102190 0x7f1600102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.513+0000 7f160542e640 1 -- 172.21.15.43:0/2187901038 >> 172.21.15.43:0/2187901038 conn(0x7f16000fd990 msgr2=0x7f16000ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.513+0000 7f160542e640 1 -- 172.21.15.43:0/2187901038 shutdown_connections 2024-04-21T08:40:04.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.513+0000 7f160542e640 1 -- 172.21.15.43:0/2187901038 wait complete. 2024-04-21T08:40:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.513+0000 7f160542e640 1 Processor -- start 2024-04-21T08:40:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.513+0000 7f160542e640 1 -- start start 2024-04-21T08:40:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.514+0000 7f160542e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1600102190 0x7f160019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.514+0000 7f160542e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1600107260 0x7f160019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.514+0000 7f160542e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16001985c0 0x7f1600198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.514+0000 7f160542e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f16001121a0 con 0x7f1600102190 2024-04-21T08:40:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.514+0000 7f160542e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1600112020 con 0x7f1600107260 2024-04-21T08:40:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.514+0000 7f160542e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1600112320 con 0x7f16001985c0 2024-04-21T08:40:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.514+0000 7f15feffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1600102190 0x7f160019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.514+0000 7f15feffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1600102190 0x7f160019e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49028/0 (socket says 172.21.15.43:49028) 2024-04-21T08:40:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.514+0000 7f15feffd640 1 -- 172.21.15.43:0/3182617315 learned_addr learned my addr 172.21.15.43:0/3182617315 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.514+0000 7f15fe7fc640 1 --2- 172.21.15.43:0/3182617315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1600107260 0x7f160019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.514+0000 7f15ff7fe640 1 --2- 172.21.15.43:0/3182617315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16001985c0 0x7f1600198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.514+0000 7f15feffd640 1 -- 172.21.15.43:0/3182617315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16001985c0 msgr2=0x7f1600198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.514+0000 7f15feffd640 1 --2- 172.21.15.43:0/3182617315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16001985c0 0x7f1600198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.514+0000 7f15feffd640 1 -- 172.21.15.43:0/3182617315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1600107260 msgr2=0x7f160019e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.514+0000 7f15feffd640 1 --2- 172.21.15.43:0/3182617315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1600107260 0x7f160019e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.514+0000 7f15feffd640 1 -- 172.21.15.43:0/3182617315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16001990e0 con 0x7f1600102190 2024-04-21T08:40:04.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.514+0000 7f15feffd640 1 --2- 172.21.15.43:0/3182617315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1600102190 0x7f160019e410 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f15f800d950 tx=0x7f15f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.514+0000 7f15f7fff640 1 -- 172.21.15.43:0/3182617315 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f8017070 con 0x7f1600102190 2024-04-21T08:40:04.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.514+0000 7f15f7fff640 1 -- 172.21.15.43:0/3182617315 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15f8009c30 con 0x7f1600102190 2024-04-21T08:40:04.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.514+0000 7f160542e640 1 -- 172.21.15.43:0/3182617315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16001993d0 con 0x7f1600102190 2024-04-21T08:40:04.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.514+0000 7f15f7fff640 1 -- 172.21.15.43:0/3182617315 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f800f620 con 0x7f1600102190 2024-04-21T08:40:04.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.514+0000 7f160542e640 1 -- 172.21.15.43:0/3182617315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16001a5620 con 0x7f1600102190 2024-04-21T08:40:04.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.517+0000 7f15f7fff640 1 -- 172.21.15.43:0/3182617315 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f15f8010460 con 0x7f1600102190 2024-04-21T08:40:04.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.517+0000 7f1608017740 1 Processor -- start 2024-04-21T08:40:04.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.517+0000 7f1608017740 1 -- start start 2024-04-21T08:40:04.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.517+0000 7f1608017740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ed2c4c3b0 0x559ed2c4c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.517+0000 7f1608017740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559ed2c4cd60 0x559ed2c55890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.517+0000 7f1608017740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c55e60 0x559ed2c562c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.517+0000 7f1608017740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559ed2bb11f0 con 0x559ed2c55e60 2024-04-21T08:40:04.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.517+0000 7f1608017740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559ed2b4e4b0 con 0x559ed2c4c3b0 2024-04-21T08:40:04.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.517+0000 7f1608017740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559ed2bad1b0 con 0x559ed2c4cd60 2024-04-21T08:40:04.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.517+0000 7f15f7fff640 1 --2- 172.21.15.43:0/3182617315 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f15ec0a3e40 0x7f15ec0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.517+0000 7f15fe7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c55e60 0x559ed2c562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.517+0000 7f15fe7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c55e60 0x559ed2c562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49042/0 (socket says 172.21.15.43:49042) 2024-04-21T08:40:04.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.517+0000 7f15fe7fc640 1 -- 172.21.15.43:0/2248943455 learned_addr learned my addr 172.21.15.43:0/2248943455 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.517+0000 7f15ff7fe640 1 --2- 172.21.15.43:0/3182617315 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f15ec0a3e40 0x7f15ec0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.518+0000 7f15ff7fe640 1 --2- 172.21.15.43:0/2248943455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ed2c4c3b0 0x559ed2c4c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.518+0000 7f15f7fff640 1 -- 172.21.15.43:0/3182617315 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f15f80d0a10 con 0x7f1600102190 2024-04-21T08:40:04.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.518+0000 7f15feffd640 1 --2- 172.21.15.43:0/2248943455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559ed2c4cd60 0x559ed2c55890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.518+0000 7f15fe7fc640 1 -- 172.21.15.43:0/2248943455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559ed2c4cd60 msgr2=0x559ed2c55890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.518+0000 7f15fe7fc640 1 --2- 172.21.15.43:0/2248943455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559ed2c4cd60 0x559ed2c55890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.518+0000 7f15fe7fc640 1 -- 172.21.15.43:0/2248943455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ed2c4c3b0 msgr2=0x559ed2c4c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.518+0000 7f15fe7fc640 1 --2- 172.21.15.43:0/2248943455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ed2c4c3b0 0x559ed2c4c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.518+0000 7f15fe7fc640 1 -- 172.21.15.43:0/2248943455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ed2b401b0 con 0x559ed2c55e60 2024-04-21T08:40:04.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.518+0000 7f15feffd640 1 --2- 172.21.15.43:0/2248943455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559ed2c4cd60 0x559ed2c55890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:04.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.518+0000 7f15fe7fc640 1 --2- 172.21.15.43:0/2248943455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c55e60 0x559ed2c562c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009900 tx=0x7f15e802c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d7720ce5a50ccb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.518+0000 7f15ff7fe640 1 --2- 172.21.15.43:0/3182617315 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f15ec0a3e40 0x7f15ec0a6300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f15f0009fd0 tx=0x7f15f0004ab0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.518+0000 7f15f57fa640 1 -- 172.21.15.43:0/2248943455 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8045020 con 0x559ed2c55e60 2024-04-21T08:40:04.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.518+0000 7f15f57fa640 1 -- 172.21.15.43:0/2248943455 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15e802c7e0 con 0x559ed2c55e60 2024-04-21T08:40:04.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.518+0000 7f15f57fa640 1 -- 172.21.15.43:0/2248943455 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e80046c0 con 0x559ed2c55e60 2024-04-21T08:40:04.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.519+0000 7f1608017740 1 -- 172.21.15.43:0/2248943455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c55e60 msgr2=0x559ed2c562c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.519+0000 7f1608017740 1 --2- 172.21.15.43:0/2248943455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c55e60 0x559ed2c562c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009900 tx=0x7f15e802c610 comp rx=0 tx=0).stop 2024-04-21T08:40:04.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.519+0000 7f1608017740 1 -- 172.21.15.43:0/2248943455 shutdown_connections 2024-04-21T08:40:04.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.519+0000 7f1608017740 1 --2- 172.21.15.43:0/2248943455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c55e60 0x559ed2c562c0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.519+0000 7f1608017740 1 --2- 172.21.15.43:0/2248943455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559ed2c4cd60 0x559ed2c55890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.519+0000 7f1608017740 1 --2- 172.21.15.43:0/2248943455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ed2c4c3b0 0x559ed2c4c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.519+0000 7f1608017740 1 -- 172.21.15.43:0/2248943455 >> 172.21.15.43:0/2248943455 conn(0x559ed2c46ba0 msgr2=0x559ed2c46f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.519+0000 7f1608017740 1 -- 172.21.15.43:0/2248943455 shutdown_connections 2024-04-21T08:40:04.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.519+0000 7f1608017740 1 -- 172.21.15.43:0/2248943455 wait complete. 2024-04-21T08:40:04.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.519+0000 7f1608017740 1 -- start start 2024-04-21T08:40:04.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.519+0000 7f1608017740 1 Processor -- start 2024-04-21T08:40:04.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.520+0000 7f1608017740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c4c3b0 0x559ed2c5c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.520+0000 7f1608017740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ed2c4cd60 0x559ed2c59590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.520+0000 7f1608017740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559ed2c55e60 0x559ed2c59ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.520+0000 7f1608017740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559ed2c5cd40 con 0x559ed2c4c3b0 2024-04-21T08:40:04.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.520+0000 7f1608017740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559ed2b6a500 con 0x559ed2c4cd60 2024-04-21T08:40:04.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.520+0000 7f1608017740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559ed2c5cec0 con 0x559ed2c55e60 2024-04-21T08:40:04.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.520+0000 7f1608017740 10 client.0 _ll_register_callbacks cb 0x559ed2b38b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:04.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.520+0000 7f15feffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c4c3b0 0x559ed2c5c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.520+0000 7f15feffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c4c3b0 0x559ed2c5c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49044/0 (socket says 172.21.15.43:49044) 2024-04-21T08:40:04.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.520+0000 7f15feffd640 1 -- 172.21.15.43:0/2963643993 learned_addr learned my addr 172.21.15.43:0/2963643993 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.520+0000 7f15fe7fc640 1 --2- 172.21.15.43:0/2963643993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ed2c4cd60 0x559ed2c59590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.520+0000 7f15ff7fe640 1 --2- 172.21.15.43:0/2963643993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559ed2c55e60 0x559ed2c59ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.520+0000 7f15feffd640 1 -- 172.21.15.43:0/2963643993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559ed2c55e60 msgr2=0x559ed2c59ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.520+0000 7f15feffd640 1 --2- 172.21.15.43:0/2963643993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559ed2c55e60 0x559ed2c59ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.520+0000 7f15feffd640 1 -- 172.21.15.43:0/2963643993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ed2c4cd60 msgr2=0x559ed2c59590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.520+0000 7f15feffd640 1 --2- 172.21.15.43:0/2963643993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ed2c4cd60 0x559ed2c59590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.520+0000 7f15feffd640 1 -- 172.21.15.43:0/2963643993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ed28c33f0 con 0x559ed2c4c3b0 2024-04-21T08:40:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.520+0000 7f15ff7fe640 1 --2- 172.21.15.43:0/2963643993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559ed2c55e60 0x559ed2c59ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.520+0000 7f15feffd640 1 --2- 172.21.15.43:0/2963643993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c4c3b0 0x559ed2c5c200 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f15f8098e10 tx=0x7f15f809bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fdfc55818ac6bf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.520+0000 7f15e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.520+0000 7f15e67fc640 1 -- 172.21.15.43:0/2963643993 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f8020040 con 0x559ed2c4c3b0 2024-04-21T08:40:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.520+0000 7f15e67fc640 1 -- 172.21.15.43:0/2963643993 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f15f80a4d70 con 0x559ed2c4c3b0 2024-04-21T08:40:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.520+0000 7f1608017740 1 -- 172.21.15.43:0/2963643993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15e802c7e0 con 0x559ed2c4c3b0 2024-04-21T08:40:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.521+0000 7f15e67fc640 1 -- 172.21.15.43:0/2963643993 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f15f809c440 con 0x559ed2c4c3b0 2024-04-21T08:40:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.521+0000 7f1608017740 10 client.16317 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.521+0000 7f1608017740 1 -- 172.21.15.43:0/2963643993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559ed298d7d0 con 0x559ed2c4c3b0 2024-04-21T08:40:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.521+0000 7f15e67fc640 1 -- 172.21.15.43:0/2963643993 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15e802c7e0 con 0x559ed2c4c3b0 2024-04-21T08:40:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.521+0000 7f15e67fc640 1 -- 172.21.15.43:0/2963643993 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f15f8098190 con 0x559ed2c4c3b0 2024-04-21T08:40:04.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.521+0000 7f1608017740 10 client.16317 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:04.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.521+0000 7f1608017740 10 client.16317 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:40:04.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.521+0000 7f1608017740 20 client.16317 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:04.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.521+0000 7f1608017740 1 --2- 172.21.15.43:0/2963643993 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x559ed2c69f60 0x559ed2c6c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.521+0000 7f1608017740 4 client.16317 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:04.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.521+0000 7f1608017740 1 -- 172.21.15.43:0/2963643993 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ed2b50930 con 0x559ed2c69f60 2024-04-21T08:40:04.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.521+0000 7f15fe7fc640 1 --2- 172.21.15.43:0/2963643993 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x559ed2c69f60 0x559ed2c6c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.522+0000 7f15fe7fc640 1 --2- 172.21.15.43:0/2963643993 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x559ed2c69f60 0x559ed2c6c390 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6a5e3ce2f8a5454 server_cookie=870fb43d13c7a6b7 in_seq=0 out_seq=0 2024-04-21T08:40:04.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.522+0000 7f15e67fc640 10 client.16317 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:40:04.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.523+0000 7f15e67fc640 1 -- 172.21.15.43:0/2963643993 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559ed2b50930 con 0x559ed2c69f60 2024-04-21T08:40:04.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.523+0000 7f15e67fc640 10 client.16317 handle_command_reply: tid=0 2024-04-21T08:40:04.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.523+0000 7f1608017740 1 client.16317 shutdown 2024-04-21T08:40:04.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.523+0000 7f1608017740 1 -- 172.21.15.43:0/2963643993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c4c3b0 msgr2=0x559ed2c5c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.523+0000 7f1608017740 1 --2- 172.21.15.43:0/2963643993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c4c3b0 0x559ed2c5c200 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f15f8098e10 tx=0x7f15f809bf80 comp rx=0 tx=0).stop 2024-04-21T08:40:04.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.523+0000 7f1608017740 1 -- 172.21.15.43:0/2963643993 shutdown_connections 2024-04-21T08:40:04.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.523+0000 7f1608017740 1 --2- 172.21.15.43:0/2963643993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559ed2c55e60 0x559ed2c59ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.523+0000 7f1608017740 1 --2- 172.21.15.43:0/2963643993 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x559ed2c69f60 0x559ed2c6c390 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.523+0000 7f1608017740 1 --2- 172.21.15.43:0/2963643993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ed2c4cd60 0x559ed2c59590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.523+0000 7f15e67fc640 0 client.16317 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:40:04.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.523+0000 7f1608017740 1 --2- 172.21.15.43:0/2963643993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c4c3b0 0x559ed2c5c200 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.523+0000 7f1608017740 1 -- 172.21.15.43:0/2963643993 >> 172.21.15.43:0/2963643993 conn(0x559ed2c46ba0 msgr2=0x559ed2c488f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.523+0000 7f1608017740 1 -- 172.21.15.43:0/2963643993 shutdown_connections 2024-04-21T08:40:04.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.523+0000 7f1608017740 1 -- 172.21.15.43:0/2963643993 wait complete. 2024-04-21T08:40:04.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.523+0000 7f1608017740 20 client.16317 trim_cache size 0 max 16384 2024-04-21T08:40:04.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.536+0000 7f1608017740 1 Processor -- start 2024-04-21T08:40:04.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.536+0000 7f1608017740 1 -- start start 2024-04-21T08:40:04.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.537+0000 7f1608017740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559ed2c4c3b0 0x559ed2c65c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.537+0000 7f1608017740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c4cd60 0x559ed2c5bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.537+0000 7f1608017740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ed2c55e60 0x559ed2c5c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.537+0000 7f1608017740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f16001ad230 con 0x559ed2c4cd60 2024-04-21T08:40:04.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.537+0000 7f1608017740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16001aed70 con 0x559ed2c55e60 2024-04-21T08:40:04.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.537+0000 7f1608017740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f16001ad900 con 0x559ed2c4c3b0 2024-04-21T08:40:04.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.537+0000 7f15fe7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c4cd60 0x559ed2c5bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.537+0000 7f15fe7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c4cd60 0x559ed2c5bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49052/0 (socket says 172.21.15.43:49052) 2024-04-21T08:40:04.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.537+0000 7f15fe7fc640 1 -- 172.21.15.43:0/1483759064 learned_addr learned my addr 172.21.15.43:0/1483759064 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.537+0000 7f15feffd640 1 --2- 172.21.15.43:0/1483759064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559ed2c4c3b0 0x559ed2c65c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.537+0000 7f15fe7fc640 1 -- 172.21.15.43:0/1483759064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559ed2c4c3b0 msgr2=0x559ed2c65c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.537+0000 7f15fe7fc640 1 --2- 172.21.15.43:0/1483759064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559ed2c4c3b0 0x559ed2c65c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.537+0000 7f15fe7fc640 1 -- 172.21.15.43:0/1483759064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ed2c55e60 msgr2=0x559ed2c5c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.537+0000 7f15fe7fc640 1 --2- 172.21.15.43:0/1483759064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ed2c55e60 0x559ed2c5c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.537+0000 7f15fe7fc640 1 -- 172.21.15.43:0/1483759064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ed2c466d0 con 0x559ed2c4cd60 2024-04-21T08:40:04.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.537+0000 7f15fe7fc640 1 --2- 172.21.15.43:0/1483759064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c4cd60 0x559ed2c5bcc0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f15e8005310 tx=0x7f15e802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329604cfe3d48b50 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.537+0000 7f15f4ff9640 1 -- 172.21.15.43:0/1483759064 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8045040 con 0x559ed2c4cd60 2024-04-21T08:40:04.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.537+0000 7f15f4ff9640 1 -- 172.21.15.43:0/1483759064 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15e802c7e0 con 0x559ed2c4cd60 2024-04-21T08:40:04.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.537+0000 7f15f4ff9640 1 -- 172.21.15.43:0/1483759064 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8003b20 con 0x559ed2c4cd60 2024-04-21T08:40:04.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.538+0000 7f1608017740 1 -- 172.21.15.43:0/1483759064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c4cd60 msgr2=0x559ed2c5bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.538+0000 7f1608017740 1 --2- 172.21.15.43:0/1483759064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c4cd60 0x559ed2c5bcc0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f15e8005310 tx=0x7f15e802bc30 comp rx=0 tx=0).stop 2024-04-21T08:40:04.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.538+0000 7f1608017740 1 -- 172.21.15.43:0/1483759064 shutdown_connections 2024-04-21T08:40:04.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.538+0000 7f1608017740 1 --2- 172.21.15.43:0/1483759064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ed2c55e60 0x559ed2c5c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.538+0000 7f1608017740 1 --2- 172.21.15.43:0/1483759064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c4cd60 0x559ed2c5bcc0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.538+0000 7f1608017740 1 --2- 172.21.15.43:0/1483759064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559ed2c4c3b0 0x559ed2c65c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.538+0000 7f1608017740 1 -- 172.21.15.43:0/1483759064 >> 172.21.15.43:0/1483759064 conn(0x559ed2c46ba0 msgr2=0x559ed2c45c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.538+0000 7f1608017740 1 -- 172.21.15.43:0/1483759064 shutdown_connections 2024-04-21T08:40:04.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.538+0000 7f1608017740 1 -- 172.21.15.43:0/1483759064 wait complete. 2024-04-21T08:40:04.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.538+0000 7f1608017740 1 -- start start 2024-04-21T08:40:04.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.538+0000 7f1608017740 1 Processor -- start 2024-04-21T08:40:04.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.539+0000 7f1608017740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559ed2c4c3b0 0x559ed2c62570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.539+0000 7f1608017740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ed2c4cd60 0x559ed2c5b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.539+0000 7f1608017740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c55e60 0x559ed2c5b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.539+0000 7f1608017740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f16001ae6a0 con 0x559ed2c55e60 2024-04-21T08:40:04.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.539+0000 7f1608017740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16001adfd0 con 0x559ed2c4cd60 2024-04-21T08:40:04.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.539+0000 7f1608017740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f16001ab210 con 0x559ed2c4c3b0 2024-04-21T08:40:04.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.539+0000 7f1608017740 10 client.0 _ll_register_callbacks cb 0x7f15f80987b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:04.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.539+0000 7f15ff7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c55e60 0x559ed2c5b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.539+0000 7f15ff7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c55e60 0x559ed2c5b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49060/0 (socket says 172.21.15.43:49060) 2024-04-21T08:40:04.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.539+0000 7f15ff7fe640 1 -- 172.21.15.43:0/1822590427 learned_addr learned my addr 172.21.15.43:0/1822590427 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.539+0000 7f15feffd640 1 --2- 172.21.15.43:0/1822590427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559ed2c4c3b0 0x559ed2c62570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.539+0000 7f15fe7fc640 1 --2- 172.21.15.43:0/1822590427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ed2c4cd60 0x559ed2c5b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.539+0000 7f15ff7fe640 1 -- 172.21.15.43:0/1822590427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559ed2c4c3b0 msgr2=0x559ed2c62570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.539+0000 7f15ff7fe640 1 --2- 172.21.15.43:0/1822590427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559ed2c4c3b0 0x559ed2c62570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.539+0000 7f15ff7fe640 1 -- 172.21.15.43:0/1822590427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ed2c4cd60 msgr2=0x559ed2c5b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.539+0000 7f15ff7fe640 1 --2- 172.21.15.43:0/1822590427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ed2c4cd60 0x559ed2c5b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.539+0000 7f15ff7fe640 1 -- 172.21.15.43:0/1822590427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ed2c5be20 con 0x559ed2c55e60 2024-04-21T08:40:04.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.539+0000 7f15feffd640 1 --2- 172.21.15.43:0/1822590427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559ed2c4c3b0 0x559ed2c62570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:04.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.539+0000 7f15ff7fe640 1 --2- 172.21.15.43:0/1822590427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c55e60 0x559ed2c5b5a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f15f0003c20 tx=0x7f15f0009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac52fc4a474083f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.539+0000 7f15e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:04.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.539+0000 7f15e67fc640 1 -- 172.21.15.43:0/1822590427 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0003e60 con 0x559ed2c55e60 2024-04-21T08:40:04.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.539+0000 7f15e67fc640 1 -- 172.21.15.43:0/1822590427 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f15f00045b0 con 0x559ed2c55e60 2024-04-21T08:40:04.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.539+0000 7f1608017740 1 -- 172.21.15.43:0/1822590427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15e802c7e0 con 0x559ed2c55e60 2024-04-21T08:40:04.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.540+0000 7f15e67fc640 1 -- 172.21.15.43:0/1822590427 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f15f0033050 con 0x559ed2c55e60 2024-04-21T08:40:04.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.540+0000 7f1608017740 10 client.16323 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:04.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.540+0000 7f1608017740 1 -- 172.21.15.43:0/1822590427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559ed2cd0590 con 0x559ed2c55e60 2024-04-21T08:40:04.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.540+0000 7f15e67fc640 1 -- 172.21.15.43:0/1822590427 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15e802c7e0 con 0x559ed2c55e60 2024-04-21T08:40:04.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.540+0000 7f15e67fc640 1 -- 172.21.15.43:0/1822590427 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f15f0029030 con 0x559ed2c55e60 2024-04-21T08:40:04.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.540+0000 7f1608017740 10 client.16323 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:04.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.540+0000 7f1608017740 10 client.16323 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:40:04.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.540+0000 7f1608017740 20 client.16323 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:04.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.540+0000 7f1608017740 1 --2- 172.21.15.43:0/1822590427 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x559ed2cd0d70 0x559ed2cd3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.540+0000 7f1608017740 4 client.16323 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:40:04.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.540+0000 7f1608017740 1 -- 172.21.15.43:0/1822590427 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ed2c59e60 con 0x559ed2cd0d70 2024-04-21T08:40:04.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.540+0000 7f15feffd640 1 --2- 172.21.15.43:0/1822590427 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x559ed2cd0d70 0x559ed2cd3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.541+0000 7f15e67fc640 10 client.16323 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:40:04.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.541+0000 7f15feffd640 1 --2- 172.21.15.43:0/1822590427 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x559ed2cd0d70 0x559ed2cd3190 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1cf733155e992414 server_cookie=fa20f1a0a92f0e3f in_seq=0 out_seq=0 2024-04-21T08:40:04.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.541+0000 7f15e67fc640 1 -- 172.21.15.43:0/1822590427 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ed2c59e60 con 0x559ed2cd0d70 2024-04-21T08:40:04.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.541+0000 7f15e67fc640 10 client.16323 handle_command_reply: tid=0 2024-04-21T08:40:04.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.541+0000 7f1608017740 1 client.16323 shutdown 2024-04-21T08:40:04.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.541+0000 7f1608017740 1 -- 172.21.15.43:0/1822590427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c55e60 msgr2=0x559ed2c5b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.541+0000 7f1608017740 1 --2- 172.21.15.43:0/1822590427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c55e60 0x559ed2c5b5a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f15f0003c20 tx=0x7f15f0009a90 comp rx=0 tx=0).stop 2024-04-21T08:40:04.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.541+0000 7f1608017740 1 -- 172.21.15.43:0/1822590427 shutdown_connections 2024-04-21T08:40:04.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.541+0000 7f1608017740 1 --2- 172.21.15.43:0/1822590427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ed2c55e60 0x559ed2c5b5a0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.541+0000 7f1608017740 1 --2- 172.21.15.43:0/1822590427 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x559ed2cd0d70 0x559ed2cd3190 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.541+0000 7f1608017740 1 --2- 172.21.15.43:0/1822590427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ed2c4cd60 0x559ed2c5b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.541+0000 7f1608017740 1 --2- 172.21.15.43:0/1822590427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559ed2c4c3b0 0x559ed2c62570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.541+0000 7f15e67fc640 0 client.16323 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:40:04.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.541+0000 7f1608017740 1 -- 172.21.15.43:0/1822590427 >> 172.21.15.43:0/1822590427 conn(0x559ed2c46ba0 msgr2=0x559ed2c45370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.541+0000 7f1608017740 1 -- 172.21.15.43:0/1822590427 shutdown_connections 2024-04-21T08:40:04.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.541+0000 7f1608017740 1 -- 172.21.15.43:0/1822590427 wait complete. 2024-04-21T08:40:04.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.541+0000 7f1608017740 20 client.16323 trim_cache size 0 max 16384 2024-04-21T08:40:04.547 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:40:04.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.542+0000 7f160542e640 1 -- 172.21.15.43:0/3182617315 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f15ec0a3e40 msgr2=0x7f15ec0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.542+0000 7f160542e640 1 --2- 172.21.15.43:0/3182617315 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f15ec0a3e40 0x7f15ec0a6300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f15f0009fd0 tx=0x7f15f0004ab0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.542+0000 7f160542e640 1 -- 172.21.15.43:0/3182617315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1600102190 msgr2=0x7f160019e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.542+0000 7f160542e640 1 --2- 172.21.15.43:0/3182617315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1600102190 0x7f160019e410 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f15f800d950 tx=0x7f15f800de10 comp rx=0 tx=0).stop 2024-04-21T08:40:04.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.542+0000 7f160542e640 1 -- 172.21.15.43:0/3182617315 shutdown_connections 2024-04-21T08:40:04.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.542+0000 7f160542e640 1 --2- 172.21.15.43:0/3182617315 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f15ec0a3e40 0x7f15ec0a6300 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.542+0000 7f160542e640 1 --2- 172.21.15.43:0/3182617315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16001985c0 0x7f1600198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.542+0000 7f160542e640 1 --2- 172.21.15.43:0/3182617315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1600107260 0x7f160019e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.542+0000 7f160542e640 1 --2- 172.21.15.43:0/3182617315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1600102190 0x7f160019e410 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.542+0000 7f160542e640 1 -- 172.21.15.43:0/3182617315 >> 172.21.15.43:0/3182617315 conn(0x7f16000fd990 msgr2=0x7f16000fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.542+0000 7f160542e640 1 -- 172.21.15.43:0/3182617315 shutdown_connections 2024-04-21T08:40:04.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.542+0000 7f160542e640 1 -- 172.21.15.43:0/3182617315 wait complete. 2024-04-21T08:40:04.552 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:40:04.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.657+0000 7f5ea7dd5640 1 -- 172.21.15.43:0/217182840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ea0107270 msgr2=0x7f5ea010ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.657+0000 7f5ea7dd5640 1 --2- 172.21.15.43:0/217182840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ea0107270 0x7f5ea010ee40 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009ca0 tx=0x7f5e9802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.657+0000 7f5ea7dd5640 1 -- 172.21.15.43:0/217182840 shutdown_connections 2024-04-21T08:40:04.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.657+0000 7f5ea7dd5640 1 --2- 172.21.15.43:0/217182840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ea0107270 0x7f5ea010ee40 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.657+0000 7f5ea7dd5640 1 --2- 172.21.15.43:0/217182840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ea006b1f0 0x7f5ea0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.657+0000 7f5ea7dd5640 1 --2- 172.21.15.43:0/217182840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ea006a840 0x7f5ea006ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.657+0000 7f5ea7dd5640 1 -- 172.21.15.43:0/217182840 >> 172.21.15.43:0/217182840 conn(0x7f5ea0077eb0 msgr2=0x7f5ea00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.657+0000 7f5ea7dd5640 1 -- 172.21.15.43:0/217182840 shutdown_connections 2024-04-21T08:40:04.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.658+0000 7f5ea7dd5640 1 -- 172.21.15.43:0/217182840 wait complete. 2024-04-21T08:40:04.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.658+0000 7f5ea7dd5640 1 Processor -- start 2024-04-21T08:40:04.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.658+0000 7f5ea7dd5640 1 -- start start 2024-04-21T08:40:04.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.658+0000 7f5ea7dd5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ea006a840 0x7f5ea01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.658+0000 7f5ea7dd5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ea0107270 0x7f5ea0198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.658+0000 7f5ea7dd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ea019cf40 0x7f5ea019d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.658+0000 7f5ea7dd5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea01121b0 con 0x7f5ea006a840 2024-04-21T08:40:04.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.658+0000 7f5ea7dd5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea0112030 con 0x7f5ea019cf40 2024-04-21T08:40:04.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.658+0000 7f5ea7dd5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea0112330 con 0x7f5ea0107270 2024-04-21T08:40:04.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.658+0000 7f5ea5b71640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ea006a840 0x7f5ea01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.658+0000 7f5ea5b71640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ea006a840 0x7f5ea01986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49070/0 (socket says 172.21.15.43:49070) 2024-04-21T08:40:04.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.658+0000 7f5ea5b71640 1 -- 172.21.15.43:0/3075071868 learned_addr learned my addr 172.21.15.43:0/3075071868 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.659+0000 7f5ea5370640 1 --2- 172.21.15.43:0/3075071868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ea0107270 0x7f5ea0198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.659+0000 7f5ea6372640 1 --2- 172.21.15.43:0/3075071868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ea019cf40 0x7f5ea019d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.659+0000 7f5ea5b71640 1 -- 172.21.15.43:0/3075071868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ea0107270 msgr2=0x7f5ea0198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.659+0000 7f5ea5b71640 1 --2- 172.21.15.43:0/3075071868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ea0107270 0x7f5ea0198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.659+0000 7f5ea5b71640 1 -- 172.21.15.43:0/3075071868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ea019cf40 msgr2=0x7f5ea019d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.659+0000 7f5ea5b71640 1 --2- 172.21.15.43:0/3075071868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ea019cf40 0x7f5ea019d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.659+0000 7f5ea5b71640 1 -- 172.21.15.43:0/3075071868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea019da30 con 0x7f5ea006a840 2024-04-21T08:40:04.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.659+0000 7f5ea5b71640 1 --2- 172.21.15.43:0/3075071868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ea006a840 0x7f5ea01986d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00d950 tx=0x7f5e9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.659+0000 7f5e96ffd640 1 -- 172.21.15.43:0/3075071868 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c017070 con 0x7f5ea006a840 2024-04-21T08:40:04.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.659+0000 7f5ea7dd5640 1 -- 172.21.15.43:0/3075071868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea019dd20 con 0x7f5ea006a840 2024-04-21T08:40:04.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.659+0000 7f5e96ffd640 1 -- 172.21.15.43:0/3075071868 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e9c009c30 con 0x7f5ea006a840 2024-04-21T08:40:04.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.659+0000 7f5e96ffd640 1 -- 172.21.15.43:0/3075071868 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c00f620 con 0x7f5ea006a840 2024-04-21T08:40:04.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.659+0000 7f5ea7dd5640 1 -- 172.21.15.43:0/3075071868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea01a5560 con 0x7f5ea006a840 2024-04-21T08:40:04.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.662+0000 7f5e96ffd640 1 -- 172.21.15.43:0/3075071868 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f5e9c010460 con 0x7f5ea006a840 2024-04-21T08:40:04.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.662+0000 7f5eaa9c6740 1 Processor -- start 2024-04-21T08:40:04.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.662+0000 7f5eaa9c6740 1 -- start start 2024-04-21T08:40:04.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.663+0000 7f5eaa9c6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00efe94d0 0x55e00efe98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.663+0000 7f5eaa9c6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e00efe9e80 0x55e00eff2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.663+0000 7f5eaa9c6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e00eff2a00 0x55e00eff4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.663+0000 7f5eaa9c6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e00ef4e1f0 con 0x55e00efe94d0 2024-04-21T08:40:04.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.663+0000 7f5eaa9c6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e00eeeb4b0 con 0x55e00eff2a00 2024-04-21T08:40:04.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.663+0000 7f5eaa9c6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e00ef4a1b0 con 0x55e00efe9e80 2024-04-21T08:40:04.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.663+0000 7f5ea6372640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00efe94d0 0x55e00efe98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.663+0000 7f5ea6372640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00efe94d0 0x55e00efe98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49080/0 (socket says 172.21.15.43:49080) 2024-04-21T08:40:04.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.663+0000 7f5ea6372640 1 -- 172.21.15.43:0/1180564385 learned_addr learned my addr 172.21.15.43:0/1180564385 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.663+0000 7f5e96ffd640 1 --2- 172.21.15.43:0/3075071868 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5e8c0a3e40 0x7f5e8c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.663+0000 7f5ea5370640 1 --2- 172.21.15.43:0/1180564385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e00eff2a00 0x55e00eff4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.663+0000 7f5ea5b71640 1 --2- 172.21.15.43:0/1180564385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e00efe9e80 0x55e00eff2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.663+0000 7f5ea6372640 1 --2- 172.21.15.43:0/3075071868 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5e8c0a3e40 0x7f5e8c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.663+0000 7f5e96ffd640 1 -- 172.21.15.43:0/3075071868 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f5e9c0d0b70 con 0x7f5ea006a840 2024-04-21T08:40:04.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.663+0000 7f5ea6372640 1 -- 172.21.15.43:0/1180564385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e00efe9e80 msgr2=0x55e00eff2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.663+0000 7f5ea6372640 1 --2- 172.21.15.43:0/1180564385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e00efe9e80 0x55e00eff2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.663+0000 7f5ea6372640 1 -- 172.21.15.43:0/1180564385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e00eff2a00 msgr2=0x55e00eff4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.663+0000 7f5ea6372640 1 --2- 172.21.15.43:0/1180564385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e00eff2a00 0x55e00eff4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.663+0000 7f5ea6372640 1 -- 172.21.15.43:0/1180564385 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e00eedd1b0 con 0x55e00efe94d0 2024-04-21T08:40:04.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.663+0000 7f5ea6372640 1 --2- 172.21.15.43:0/1180564385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00efe94d0 0x55e00efe98b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5e98004af0 tx=0x7f5e98032e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0d971b104432a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.664+0000 7f5e8bfff640 1 -- 172.21.15.43:0/1180564385 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e98003b70 con 0x55e00efe94d0 2024-04-21T08:40:04.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.664+0000 7f5e8bfff640 1 -- 172.21.15.43:0/1180564385 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e98003d10 con 0x55e00efe94d0 2024-04-21T08:40:04.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.664+0000 7f5e8bfff640 1 -- 172.21.15.43:0/1180564385 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e98034900 con 0x55e00efe94d0 2024-04-21T08:40:04.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.664+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/1180564385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00efe94d0 msgr2=0x55e00efe98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.664+0000 7f5ea6372640 1 --2- 172.21.15.43:0/3075071868 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5e8c0a3e40 0x7f5e8c0a6300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5e9804c000 tx=0x7f5e98048000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.664+0000 7f5eaa9c6740 1 --2- 172.21.15.43:0/1180564385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00efe94d0 0x55e00efe98b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5e98004af0 tx=0x7f5e98032e50 comp rx=0 tx=0).stop 2024-04-21T08:40:04.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.664+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/1180564385 shutdown_connections 2024-04-21T08:40:04.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.664+0000 7f5eaa9c6740 1 --2- 172.21.15.43:0/1180564385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e00eff2a00 0x55e00eff4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.664+0000 7f5eaa9c6740 1 --2- 172.21.15.43:0/1180564385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e00efe9e80 0x55e00eff2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.664+0000 7f5eaa9c6740 1 --2- 172.21.15.43:0/1180564385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00efe94d0 0x55e00efe98b0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.664+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/1180564385 >> 172.21.15.43:0/1180564385 conn(0x55e00efe3ba0 msgr2=0x55e00efe3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.665+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/1180564385 shutdown_connections 2024-04-21T08:40:04.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.665+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/1180564385 wait complete. 2024-04-21T08:40:04.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.665+0000 7f5eaa9c6740 1 -- start start 2024-04-21T08:40:04.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.665+0000 7f5eaa9c6740 1 Processor -- start 2024-04-21T08:40:04.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.665+0000 7f5eaa9c6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00efe94d0 0x55e00eff22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.665+0000 7f5eaa9c6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e00efe9e80 0x55e00efef630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.665+0000 7f5eaa9c6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e00eff2a00 0x55e00efefb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.665+0000 7f5eaa9c6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e00eff5970 con 0x55e00efe94d0 2024-04-21T08:40:04.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.665+0000 7f5eaa9c6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e00ef07500 con 0x55e00efe9e80 2024-04-21T08:40:04.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.665+0000 7f5eaa9c6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e00eff5af0 con 0x55e00eff2a00 2024-04-21T08:40:04.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.665+0000 7f5ea5b71640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00efe94d0 0x55e00eff22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.665+0000 7f5eaa9c6740 10 client.0 _ll_register_callbacks cb 0x55e00eed5b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:04.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.665+0000 7f5ea5b71640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00efe94d0 0x55e00eff22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49084/0 (socket says 172.21.15.43:49084) 2024-04-21T08:40:04.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.665+0000 7f5ea5b71640 1 -- 172.21.15.43:0/1502075384 learned_addr learned my addr 172.21.15.43:0/1502075384 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.666+0000 7f5ea5370640 1 --2- 172.21.15.43:0/1502075384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e00efe9e80 0x55e00efef630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.666+0000 7f5ea5b71640 1 -- 172.21.15.43:0/1502075384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e00eff2a00 msgr2=0x55e00efefb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.666+0000 7f5ea5b71640 1 --2- 172.21.15.43:0/1502075384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e00eff2a00 0x55e00efefb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.666+0000 7f5ea5b71640 1 -- 172.21.15.43:0/1502075384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e00efe9e80 msgr2=0x55e00efef630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.666+0000 7f5ea5b71640 1 --2- 172.21.15.43:0/1502075384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e00efe9e80 0x55e00efef630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.666+0000 7f5ea5b71640 1 -- 172.21.15.43:0/1502075384 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e00ec603f0 con 0x55e00efe94d0 2024-04-21T08:40:04.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.666+0000 7f5ea5370640 1 --2- 172.21.15.43:0/1502075384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e00efe9e80 0x55e00efef630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:04.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.666+0000 7f5ea5b71640 1 --2- 172.21.15.43:0/1502075384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00efe94d0 0x55e00eff22a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c09c480 tx=0x7f5e9c066f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34b458e185d2e45 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.666+0000 7f5e897fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:04.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.666+0000 7f5e897fa640 1 -- 172.21.15.43:0/1502075384 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c020040 con 0x55e00efe94d0 2024-04-21T08:40:04.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.666+0000 7f5e897fa640 1 -- 172.21.15.43:0/1502075384 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f5e9c0a4ed0 con 0x55e00efe94d0 2024-04-21T08:40:04.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.666+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/1502075384 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e98003d10 con 0x55e00efe94d0 2024-04-21T08:40:04.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.666+0000 7f5e897fa640 1 -- 172.21.15.43:0/1502075384 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5e9c067000 con 0x55e00efe94d0 2024-04-21T08:40:04.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.666+0000 7f5eaa9c6740 10 client.16335 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:04.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.666+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/1502075384 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e00ed2a7d0 con 0x55e00efe94d0 2024-04-21T08:40:04.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.666+0000 7f5e897fa640 1 -- 172.21.15.43:0/1502075384 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e98003d10 con 0x55e00efe94d0 2024-04-21T08:40:04.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.667+0000 7f5e897fa640 1 -- 172.21.15.43:0/1502075384 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5e9c0ce050 con 0x55e00efe94d0 2024-04-21T08:40:04.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.667+0000 7f5eaa9c6740 10 client.16335 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:04.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.667+0000 7f5eaa9c6740 10 client.16335 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:40:04.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.667+0000 7f5eaa9c6740 20 client.16335 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:04.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.667+0000 7f5eaa9c6740 1 --2- 172.21.15.43:0/1502075384 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55e00f002b50 0x55e00f004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.667+0000 7f5eaa9c6740 4 client.16335 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:04.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.667+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/1502075384 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e00eeed930 con 0x55e00f002b50 2024-04-21T08:40:04.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.667+0000 7f5ea5370640 1 --2- 172.21.15.43:0/1502075384 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55e00f002b50 0x55e00f004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.667+0000 7f5e897fa640 10 client.16335 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:40:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.667+0000 7f5ea5370640 1 --2- 172.21.15.43:0/1502075384 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55e00f002b50 0x55e00f004fc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=687a1f58b1590c7e server_cookie=b6b2ab933d74dbbb in_seq=0 out_seq=0 2024-04-21T08:40:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.668+0000 7f5e897fa640 1 -- 172.21.15.43:0/1502075384 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e00eeed930 con 0x55e00f002b50 2024-04-21T08:40:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.668+0000 7f5e897fa640 10 client.16335 handle_command_reply: tid=0 2024-04-21T08:40:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.668+0000 7f5eaa9c6740 1 client.16335 shutdown 2024-04-21T08:40:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.668+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/1502075384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00efe94d0 msgr2=0x55e00eff22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.668+0000 7f5eaa9c6740 1 --2- 172.21.15.43:0/1502075384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00efe94d0 0x55e00eff22a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c09c480 tx=0x7f5e9c066f90 comp rx=0 tx=0).stop 2024-04-21T08:40:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.668+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/1502075384 shutdown_connections 2024-04-21T08:40:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.668+0000 7f5eaa9c6740 1 --2- 172.21.15.43:0/1502075384 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55e00f002b50 0x55e00f004fc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.668+0000 7f5eaa9c6740 1 --2- 172.21.15.43:0/1502075384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e00eff2a00 0x55e00efefb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.668+0000 7f5eaa9c6740 1 --2- 172.21.15.43:0/1502075384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e00efe9e80 0x55e00efef630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.668+0000 7f5e897fa640 0 client.16335 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:40:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.668+0000 7f5eaa9c6740 1 --2- 172.21.15.43:0/1502075384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00efe94d0 0x55e00eff22a0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.668+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/1502075384 >> 172.21.15.43:0/1502075384 conn(0x55e00efe3ba0 msgr2=0x55e00efe58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.669+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/1502075384 shutdown_connections 2024-04-21T08:40:04.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.669+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/1502075384 wait complete. 2024-04-21T08:40:04.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.669+0000 7f5eaa9c6740 20 client.16335 trim_cache size 0 max 16384 2024-04-21T08:40:04.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.682+0000 7f5eaa9c6740 1 Processor -- start 2024-04-21T08:40:04.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.682+0000 7f5eaa9c6740 1 -- start start 2024-04-21T08:40:04.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.682+0000 7f5eaa9c6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e00efe94d0 0x55e00effe830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.682+0000 7f5eaa9c6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e00efe9e80 0x55e00eff1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.682+0000 7f5eaa9c6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00eff2a00 0x55e00eff22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.682+0000 7f5eaa9c6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea01af2a0 con 0x55e00eff2a00 2024-04-21T08:40:04.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.682+0000 7f5eaa9c6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea01b0de0 con 0x55e00efe9e80 2024-04-21T08:40:04.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.682+0000 7f5eaa9c6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea01af970 con 0x55e00efe94d0 2024-04-21T08:40:04.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.682+0000 7f5ea6372640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00eff2a00 0x55e00eff22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.682+0000 7f5ea6372640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00eff2a00 0x55e00eff22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49088/0 (socket says 172.21.15.43:49088) 2024-04-21T08:40:04.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.682+0000 7f5ea6372640 1 -- 172.21.15.43:0/2527351421 learned_addr learned my addr 172.21.15.43:0/2527351421 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.683+0000 7f5ea6372640 1 -- 172.21.15.43:0/2527351421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e00efe94d0 msgr2=0x55e00effe830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.683+0000 7f5ea6372640 1 --2- 172.21.15.43:0/2527351421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e00efe94d0 0x55e00effe830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.683+0000 7f5ea6372640 1 -- 172.21.15.43:0/2527351421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e00efe9e80 msgr2=0x55e00eff1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.683+0000 7f5ea6372640 1 --2- 172.21.15.43:0/2527351421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e00efe9e80 0x55e00eff1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.683+0000 7f5ea6372640 1 -- 172.21.15.43:0/2527351421 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e00efefed0 con 0x55e00eff2a00 2024-04-21T08:40:04.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.683+0000 7f5ea6372640 1 --2- 172.21.15.43:0/2527351421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00eff2a00 0x55e00eff22a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009ca0 tx=0x7f5e9802c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207392158529df4d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.683+0000 7f5e8b7fe640 1 -- 172.21.15.43:0/2527351421 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e98039040 con 0x55e00eff2a00 2024-04-21T08:40:04.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.683+0000 7f5e8b7fe640 1 -- 172.21.15.43:0/2527351421 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e980348f0 con 0x55e00eff2a00 2024-04-21T08:40:04.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.683+0000 7f5e8b7fe640 1 -- 172.21.15.43:0/2527351421 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e98004040 con 0x55e00eff2a00 2024-04-21T08:40:04.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.684+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/2527351421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00eff2a00 msgr2=0x55e00eff22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.684+0000 7f5eaa9c6740 1 --2- 172.21.15.43:0/2527351421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00eff2a00 0x55e00eff22a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009ca0 tx=0x7f5e9802c770 comp rx=0 tx=0).stop 2024-04-21T08:40:04.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.684+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/2527351421 shutdown_connections 2024-04-21T08:40:04.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.684+0000 7f5eaa9c6740 1 --2- 172.21.15.43:0/2527351421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00eff2a00 0x55e00eff22a0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.684+0000 7f5eaa9c6740 1 --2- 172.21.15.43:0/2527351421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e00efe9e80 0x55e00eff1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.684+0000 7f5eaa9c6740 1 --2- 172.21.15.43:0/2527351421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e00efe94d0 0x55e00effe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.684+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/2527351421 >> 172.21.15.43:0/2527351421 conn(0x55e00efe3ba0 msgr2=0x55e00efe2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.684+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/2527351421 shutdown_connections 2024-04-21T08:40:04.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.684+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/2527351421 wait complete. 2024-04-21T08:40:04.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.684+0000 7f5eaa9c6740 1 -- start start 2024-04-21T08:40:04.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.684+0000 7f5eaa9c6740 1 Processor -- start 2024-04-21T08:40:04.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.684+0000 7f5eaa9c6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00efe94d0 0x55e00effb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.684+0000 7f5eaa9c6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e00efe9e80 0x55e00eff6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.684+0000 7f5eaa9c6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e00eff2a00 0x55e00eff6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.684+0000 7f5eaa9c6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea01b0710 con 0x55e00efe94d0 2024-04-21T08:40:04.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.684+0000 7f5eaa9c6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea01b0040 con 0x55e00eff2a00 2024-04-21T08:40:04.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.684+0000 7f5eaa9c6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea01ad280 con 0x55e00efe9e80 2024-04-21T08:40:04.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.684+0000 7f5ea5b71640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00efe94d0 0x55e00effb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.684+0000 7f5eaa9c6740 10 client.0 _ll_register_callbacks cb 0x7f5e9c098860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:04.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.685+0000 7f5ea5b71640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00efe94d0 0x55e00effb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49100/0 (socket says 172.21.15.43:49100) 2024-04-21T08:40:04.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.685+0000 7f5ea5b71640 1 -- 172.21.15.43:0/1021220423 learned_addr learned my addr 172.21.15.43:0/1021220423 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.685+0000 7f5ea6372640 1 --2- 172.21.15.43:0/1021220423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e00eff2a00 0x55e00eff6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.685+0000 7f5ea5b71640 1 -- 172.21.15.43:0/1021220423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e00efe9e80 msgr2=0x55e00eff6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.685+0000 7f5ea5370640 1 --2- 172.21.15.43:0/1021220423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e00efe9e80 0x55e00eff6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.685+0000 7f5ea5b71640 1 --2- 172.21.15.43:0/1021220423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e00efe9e80 0x55e00eff6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.685+0000 7f5ea5b71640 1 -- 172.21.15.43:0/1021220423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e00eff2a00 msgr2=0x55e00eff6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.685+0000 7f5ea5b71640 1 --2- 172.21.15.43:0/1021220423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e00eff2a00 0x55e00eff6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.685+0000 7f5ea5b71640 1 -- 172.21.15.43:0/1021220423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e00eff72b0 con 0x55e00efe94d0 2024-04-21T08:40:04.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.685+0000 7f5ea5b71640 1 --2- 172.21.15.43:0/1021220423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00efe94d0 0x55e00effb190 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c000c00 tx=0x7f5e9c098240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc350b780c6a125 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.685+0000 7f5e897fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:04.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.685+0000 7f5e897fa640 1 -- 172.21.15.43:0/1021220423 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c020070 con 0x55e00efe94d0 2024-04-21T08:40:04.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.685+0000 7f5e897fa640 1 -- 172.21.15.43:0/1021220423 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f5e9c0a4ed0 con 0x55e00efe94d0 2024-04-21T08:40:04.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.685+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/1021220423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e980348f0 con 0x55e00efe94d0 2024-04-21T08:40:04.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.685+0000 7f5e897fa640 1 -- 172.21.15.43:0/1021220423 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5e9c0cdcb0 con 0x55e00efe94d0 2024-04-21T08:40:04.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.685+0000 7f5eaa9c6740 10 client.16341 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.685+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/1021220423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e00f0691a0 con 0x55e00efe94d0 2024-04-21T08:40:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.685+0000 7f5e897fa640 1 -- 172.21.15.43:0/1021220423 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e980348f0 con 0x55e00efe94d0 2024-04-21T08:40:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.686+0000 7f5e897fa640 1 -- 172.21.15.43:0/1021220423 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5e9c010750 con 0x55e00efe94d0 2024-04-21T08:40:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.686+0000 7f5eaa9c6740 10 client.16341 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.686+0000 7f5eaa9c6740 10 client.16341 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:40:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.686+0000 7f5eaa9c6740 20 client.16341 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.686+0000 7f5eaa9c6740 1 --2- 172.21.15.43:0/1021220423 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55e00f069860 0x55e00f06bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.686+0000 7f5eaa9c6740 4 client.16341 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:40:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.686+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/1021220423 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e00ef5bc10 con 0x55e00f069860 2024-04-21T08:40:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.686+0000 7f5ea5370640 1 --2- 172.21.15.43:0/1021220423 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55e00f069860 0x55e00f06bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.686+0000 7f5e897fa640 10 client.16341 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:40:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.686+0000 7f5ea5370640 1 --2- 172.21.15.43:0/1021220423 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55e00f069860 0x55e00f06bc50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e826e63d032deeb server_cookie=76b9d0e7679e9b72 in_seq=0 out_seq=0 2024-04-21T08:40:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.686+0000 7f5e897fa640 1 -- 172.21.15.43:0/1021220423 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e00ef5bc10 con 0x55e00f069860 2024-04-21T08:40:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.686+0000 7f5e897fa640 10 client.16341 handle_command_reply: tid=0 2024-04-21T08:40:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.686+0000 7f5eaa9c6740 1 client.16341 shutdown 2024-04-21T08:40:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.687+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/1021220423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00efe94d0 msgr2=0x55e00effb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.687+0000 7f5eaa9c6740 1 --2- 172.21.15.43:0/1021220423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00efe94d0 0x55e00effb190 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c000c00 tx=0x7f5e9c098240 comp rx=0 tx=0).stop 2024-04-21T08:40:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.687+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/1021220423 shutdown_connections 2024-04-21T08:40:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.687+0000 7f5eaa9c6740 1 --2- 172.21.15.43:0/1021220423 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55e00f069860 0x55e00f06bc50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.687+0000 7f5eaa9c6740 1 --2- 172.21.15.43:0/1021220423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e00eff2a00 0x55e00eff6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.687+0000 7f5e897fa640 0 client.16341 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:40:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.687+0000 7f5eaa9c6740 1 --2- 172.21.15.43:0/1021220423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e00efe9e80 0x55e00eff6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.687+0000 7f5eaa9c6740 1 --2- 172.21.15.43:0/1021220423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e00efe94d0 0x55e00effb190 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.687+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/1021220423 >> 172.21.15.43:0/1021220423 conn(0x55e00efe3ba0 msgr2=0x55e00efe4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.687+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/1021220423 shutdown_connections 2024-04-21T08:40:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.687+0000 7f5eaa9c6740 1 -- 172.21.15.43:0/1021220423 wait complete. 2024-04-21T08:40:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.687+0000 7f5eaa9c6740 20 client.16341 trim_cache size 0 max 16384 2024-04-21T08:40:04.693 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:40:04.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.687+0000 7f5ea7dd5640 1 -- 172.21.15.43:0/3075071868 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5e8c0a3e40 msgr2=0x7f5e8c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.687+0000 7f5ea7dd5640 1 --2- 172.21.15.43:0/3075071868 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5e8c0a3e40 0x7f5e8c0a6300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5e9804c000 tx=0x7f5e98048000 comp rx=0 tx=0).stop 2024-04-21T08:40:04.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.687+0000 7f5ea7dd5640 1 -- 172.21.15.43:0/3075071868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ea006a840 msgr2=0x7f5ea01986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.687+0000 7f5ea7dd5640 1 --2- 172.21.15.43:0/3075071868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ea006a840 0x7f5ea01986d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00d950 tx=0x7f5e9c00de10 comp rx=0 tx=0).stop 2024-04-21T08:40:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.687+0000 7f5ea7dd5640 1 -- 172.21.15.43:0/3075071868 shutdown_connections 2024-04-21T08:40:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.687+0000 7f5ea7dd5640 1 --2- 172.21.15.43:0/3075071868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ea019cf40 0x7f5ea019d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.687+0000 7f5ea7dd5640 1 --2- 172.21.15.43:0/3075071868 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5e8c0a3e40 0x7f5e8c0a6300 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.687+0000 7f5ea7dd5640 1 --2- 172.21.15.43:0/3075071868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ea0107270 0x7f5ea0198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.687+0000 7f5ea7dd5640 1 --2- 172.21.15.43:0/3075071868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ea006a840 0x7f5ea01986d0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.687+0000 7f5ea7dd5640 1 -- 172.21.15.43:0/3075071868 >> 172.21.15.43:0/3075071868 conn(0x7f5ea0077eb0 msgr2=0x7f5ea00fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.687+0000 7f5ea7dd5640 1 -- 172.21.15.43:0/3075071868 shutdown_connections 2024-04-21T08:40:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.688+0000 7f5ea7dd5640 1 -- 172.21.15.43:0/3075071868 wait complete. 2024-04-21T08:40:04.697 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:40:04.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.802+0000 7f31032b3640 1 -- 172.21.15.43:0/139381044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30fc101670 msgr2=0x7f30fc10e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.802+0000 7f31032b3640 1 --2- 172.21.15.43:0/139381044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30fc101670 0x7f30fc10e3c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f30ec009900 tx=0x7f30ec02bc30 comp rx=0 tx=0).stop 2024-04-21T08:40:04.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.803+0000 7f31032b3640 1 -- 172.21.15.43:0/139381044 shutdown_connections 2024-04-21T08:40:04.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.803+0000 7f31032b3640 1 --2- 172.21.15.43:0/139381044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30fc10e900 0x7f30fc110cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.803+0000 7f31032b3640 1 --2- 172.21.15.43:0/139381044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30fc101670 0x7f30fc10e3c0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.803+0000 7f31032b3640 1 --2- 172.21.15.43:0/139381044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30fc100d50 0x7f30fc101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.803+0000 7f31032b3640 1 -- 172.21.15.43:0/139381044 >> 172.21.15.43:0/139381044 conn(0x7f30fc077f50 msgr2=0x7f30fc078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.803+0000 7f31032b3640 1 -- 172.21.15.43:0/139381044 shutdown_connections 2024-04-21T08:40:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.803+0000 7f31032b3640 1 -- 172.21.15.43:0/139381044 wait complete. 2024-04-21T08:40:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.803+0000 7f31032b3640 1 Processor -- start 2024-04-21T08:40:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.803+0000 7f31032b3640 1 -- start start 2024-04-21T08:40:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.803+0000 7f31032b3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30fc100d50 0x7f30fc1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.803+0000 7f31032b3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30fc101670 0x7f30fc1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.803+0000 7f31032b3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30fc10e900 0x7f30fc19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.803+0000 7f31032b3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1143d0 con 0x7f30fc10e900 2024-04-21T08:40:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.803+0000 7f31032b3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc114250 con 0x7f30fc101670 2024-04-21T08:40:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.803+0000 7f31032b3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc114550 con 0x7f30fc100d50 2024-04-21T08:40:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.804+0000 7f3101850640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30fc10e900 0x7f30fc19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.804+0000 7f3101850640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30fc10e900 0x7f30fc19a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49126/0 (socket says 172.21.15.43:49126) 2024-04-21T08:40:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.804+0000 7f3101850640 1 -- 172.21.15.43:0/318449200 learned_addr learned my addr 172.21.15.43:0/318449200 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.804+0000 7f310084e640 1 --2- 172.21.15.43:0/318449200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30fc101670 0x7f30fc1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.804+0000 7f310104f640 1 --2- 172.21.15.43:0/318449200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30fc100d50 0x7f30fc1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.804+0000 7f3101850640 1 -- 172.21.15.43:0/318449200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30fc100d50 msgr2=0x7f30fc1a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.804+0000 7f3101850640 1 --2- 172.21.15.43:0/318449200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30fc100d50 0x7f30fc1a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.804+0000 7f3101850640 1 -- 172.21.15.43:0/318449200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30fc101670 msgr2=0x7f30fc1a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.804+0000 7f3101850640 1 --2- 172.21.15.43:0/318449200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30fc101670 0x7f30fc1a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.804+0000 7f3101850640 1 -- 172.21.15.43:0/318449200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30fc19b020 con 0x7f30fc10e900 2024-04-21T08:40:04.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.804+0000 7f3101850640 1 --2- 172.21.15.43:0/318449200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30fc10e900 0x7f30fc19a850 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f30f400e9d0 tx=0x7f30f400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.804+0000 7f30f27fc640 1 -- 172.21.15.43:0/318449200 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f400cd80 con 0x7f30fc10e900 2024-04-21T08:40:04.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.804+0000 7f30f27fc640 1 -- 172.21.15.43:0/318449200 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30f4005020 con 0x7f30fc10e900 2024-04-21T08:40:04.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.804+0000 7f31032b3640 1 -- 172.21.15.43:0/318449200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30fc19b310 con 0x7f30fc10e900 2024-04-21T08:40:04.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.804+0000 7f30f27fc640 1 -- 172.21.15.43:0/318449200 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f40116d0 con 0x7f30fc10e900 2024-04-21T08:40:04.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.804+0000 7f31032b3640 1 -- 172.21.15.43:0/318449200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30fc1a7530 con 0x7f30fc10e900 2024-04-21T08:40:04.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.807+0000 7f30f27fc640 1 -- 172.21.15.43:0/318449200 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f30f40051d0 con 0x7f30fc10e900 2024-04-21T08:40:04.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.807+0000 7f3105f7c740 1 Processor -- start 2024-04-21T08:40:04.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.807+0000 7f3105f7c740 1 -- start start 2024-04-21T08:40:04.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.807+0000 7f3105f7c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f02744d0 0x55e9f02748b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.807+0000 7f3105f7c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9f0274e80 0x55e9f027d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.807+0000 7f3105f7c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9f027da00 0x55e9f027fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.807+0000 7f3105f7c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e9f01d91f0 con 0x55e9f02744d0 2024-04-21T08:40:04.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.807+0000 7f3105f7c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e9f01764b0 con 0x55e9f0274e80 2024-04-21T08:40:04.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.807+0000 7f3105f7c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e9f01d51b0 con 0x55e9f027da00 2024-04-21T08:40:04.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.807+0000 7f30f27fc640 1 --2- 172.21.15.43:0/318449200 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f30e80a3e40 0x7f30e80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.807+0000 7f310084e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f02744d0 0x55e9f02748b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.807+0000 7f310084e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f02744d0 0x55e9f02748b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49140/0 (socket says 172.21.15.43:49140) 2024-04-21T08:40:04.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.808+0000 7f310084e640 1 -- 172.21.15.43:0/1286734303 learned_addr learned my addr 172.21.15.43:0/1286734303 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.808+0000 7f3101850640 1 --2- 172.21.15.43:0/318449200 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f30e80a3e40 0x7f30e80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.808+0000 7f30f27fc640 1 -- 172.21.15.43:0/318449200 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f30f40d1330 con 0x7f30fc10e900 2024-04-21T08:40:04.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.808+0000 7f310104f640 1 --2- 172.21.15.43:0/1286734303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9f0274e80 0x55e9f027d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.808+0000 7f310084e640 1 --2- 172.21.15.43:0/1286734303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9f027da00 0x55e9f027fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.808+0000 7f310084e640 1 -- 172.21.15.43:0/1286734303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9f027da00 msgr2=0x55e9f027fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.808+0000 7f310084e640 1 --2- 172.21.15.43:0/1286734303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9f027da00 0x55e9f027fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.808+0000 7f310084e640 1 -- 172.21.15.43:0/1286734303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9f0274e80 msgr2=0x55e9f027d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.808+0000 7f310084e640 1 --2- 172.21.15.43:0/1286734303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9f0274e80 0x55e9f027d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.808+0000 7f310084e640 1 -- 172.21.15.43:0/1286734303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9f01681b0 con 0x55e9f02744d0 2024-04-21T08:40:04.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.808+0000 7f3101850640 1 --2- 172.21.15.43:0/318449200 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f30e80a3e40 0x7f30e80a6300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f30f40ce590 tx=0x7f30f4098910 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.808+0000 7f310084e640 1 --2- 172.21.15.43:0/1286734303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f02744d0 0x55e9f02748b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f30ec02c170 tx=0x7f30ec005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7060c5ae0122e9d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.808+0000 7f30e77fe640 1 -- 172.21.15.43:0/1286734303 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec02cb50 con 0x55e9f02744d0 2024-04-21T08:40:04.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.808+0000 7f30e77fe640 1 -- 172.21.15.43:0/1286734303 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30ec048070 con 0x55e9f02744d0 2024-04-21T08:40:04.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.808+0000 7f30e77fe640 1 -- 172.21.15.43:0/1286734303 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec0048c0 con 0x55e9f02744d0 2024-04-21T08:40:04.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.809+0000 7f3105f7c740 1 -- 172.21.15.43:0/1286734303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f02744d0 msgr2=0x55e9f02748b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.809+0000 7f3105f7c740 1 --2- 172.21.15.43:0/1286734303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f02744d0 0x55e9f02748b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f30ec02c170 tx=0x7f30ec005b50 comp rx=0 tx=0).stop 2024-04-21T08:40:04.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.809+0000 7f3105f7c740 1 -- 172.21.15.43:0/1286734303 shutdown_connections 2024-04-21T08:40:04.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.809+0000 7f3105f7c740 1 --2- 172.21.15.43:0/1286734303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9f027da00 0x55e9f027fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.809+0000 7f3105f7c740 1 --2- 172.21.15.43:0/1286734303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9f0274e80 0x55e9f027d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.809+0000 7f3105f7c740 1 --2- 172.21.15.43:0/1286734303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f02744d0 0x55e9f02748b0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.809+0000 7f3105f7c740 1 -- 172.21.15.43:0/1286734303 >> 172.21.15.43:0/1286734303 conn(0x55e9f026eba0 msgr2=0x55e9f026ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.809+0000 7f3105f7c740 1 -- 172.21.15.43:0/1286734303 shutdown_connections 2024-04-21T08:40:04.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.809+0000 7f3105f7c740 1 -- 172.21.15.43:0/1286734303 wait complete. 2024-04-21T08:40:04.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.809+0000 7f3105f7c740 1 -- start start 2024-04-21T08:40:04.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.809+0000 7f3105f7c740 1 Processor -- start 2024-04-21T08:40:04.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.810+0000 7f3105f7c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9f02744d0 0x55e9f027d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.810+0000 7f3105f7c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f0274e80 0x55e9f027a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.810+0000 7f3105f7c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9f027da00 0x55e9f027acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.810+0000 7f3105f7c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e9f0280910 con 0x55e9f0274e80 2024-04-21T08:40:04.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.810+0000 7f3105f7c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e9f0192500 con 0x55e9f027da00 2024-04-21T08:40:04.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.810+0000 7f3105f7c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e9f0280a90 con 0x55e9f02744d0 2024-04-21T08:40:04.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.810+0000 7f3105f7c740 10 client.0 _ll_register_callbacks cb 0x55e9f0160b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:04.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.810+0000 7f310104f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f0274e80 0x55e9f027a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.810+0000 7f310104f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f0274e80 0x55e9f027a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49156/0 (socket says 172.21.15.43:49156) 2024-04-21T08:40:04.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.810+0000 7f310104f640 1 -- 172.21.15.43:0/614304805 learned_addr learned my addr 172.21.15.43:0/614304805 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.810+0000 7f310084e640 1 --2- 172.21.15.43:0/614304805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9f027da00 0x55e9f027acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.810+0000 7f310104f640 1 -- 172.21.15.43:0/614304805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9f02744d0 msgr2=0x55e9f027d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.810+0000 7f310104f640 1 --2- 172.21.15.43:0/614304805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9f02744d0 0x55e9f027d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.810+0000 7f310104f640 1 -- 172.21.15.43:0/614304805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9f027da00 msgr2=0x55e9f027acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.810+0000 7f310104f640 1 --2- 172.21.15.43:0/614304805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9f027da00 0x55e9f027acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.810+0000 7f310104f640 1 -- 172.21.15.43:0/614304805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9efeeb3f0 con 0x55e9f0274e80 2024-04-21T08:40:04.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.810+0000 7f310084e640 1 --2- 172.21.15.43:0/614304805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9f027da00 0x55e9f027acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:04.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.810+0000 7f310104f640 1 --2- 172.21.15.43:0/614304805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f0274e80 0x55e9f027a6c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f30f8013710 tx=0x7f30f8013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b51f4f5560acd24 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.811+0000 7f30e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:04.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.811+0000 7f30e4ff9640 1 -- 172.21.15.43:0/614304805 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8020060 con 0x55e9f0274e80 2024-04-21T08:40:04.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.811+0000 7f30e4ff9640 1 -- 172.21.15.43:0/614304805 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f30f8011dd0 con 0x55e9f0274e80 2024-04-21T08:40:04.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.811+0000 7f3105f7c740 1 -- 172.21.15.43:0/614304805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30ec048070 con 0x55e9f0274e80 2024-04-21T08:40:04.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.811+0000 7f30e4ff9640 1 -- 172.21.15.43:0/614304805 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f30f8027050 con 0x55e9f0274e80 2024-04-21T08:40:04.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.811+0000 7f3105f7c740 10 client.16353 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:04.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.811+0000 7f3105f7c740 1 -- 172.21.15.43:0/614304805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e9effb57d0 con 0x55e9f0274e80 2024-04-21T08:40:04.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.811+0000 7f30e4ff9640 1 -- 172.21.15.43:0/614304805 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30ec048070 con 0x55e9f0274e80 2024-04-21T08:40:04.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.811+0000 7f30e4ff9640 1 -- 172.21.15.43:0/614304805 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f30f801a030 con 0x55e9f0274e80 2024-04-21T08:40:04.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.811+0000 7f3105f7c740 10 client.16353 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:04.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.811+0000 7f3105f7c740 10 client.16353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:04.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.811+0000 7f3105f7c740 20 client.16353 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:04.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.811+0000 7f3105f7c740 1 --2- 172.21.15.43:0/614304805 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e9f028dcb0 0x55e9f0290130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.811+0000 7f3105f7c740 4 client.16353 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:04.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.811+0000 7f3105f7c740 1 -- 172.21.15.43:0/614304805 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9f0178930 con 0x55e9f028dcb0 2024-04-21T08:40:04.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.812+0000 7f310084e640 1 --2- 172.21.15.43:0/614304805 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e9f028dcb0 0x55e9f0290130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.812+0000 7f310084e640 1 --2- 172.21.15.43:0/614304805 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e9f028dcb0 0x55e9f0290130 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cef7ef9eb360831 server_cookie=9028feeea6574b in_seq=0 out_seq=0 2024-04-21T08:40:04.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.812+0000 7f30e4ff9640 10 client.16353 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:04.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.813+0000 7f30e4ff9640 1 -- 172.21.15.43:0/614304805 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e9f0178930 con 0x55e9f028dcb0 2024-04-21T08:40:04.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.813+0000 7f30e4ff9640 10 client.16353 handle_command_reply: tid=0 2024-04-21T08:40:04.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.813+0000 7f3105f7c740 1 client.16353 shutdown 2024-04-21T08:40:04.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.813+0000 7f3105f7c740 1 -- 172.21.15.43:0/614304805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f0274e80 msgr2=0x55e9f027a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.813+0000 7f3105f7c740 1 --2- 172.21.15.43:0/614304805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f0274e80 0x55e9f027a6c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f30f8013710 tx=0x7f30f8013bd0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.813+0000 7f3105f7c740 1 -- 172.21.15.43:0/614304805 shutdown_connections 2024-04-21T08:40:04.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.813+0000 7f3105f7c740 1 --2- 172.21.15.43:0/614304805 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e9f028dcb0 0x55e9f0290130 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.813+0000 7f3105f7c740 1 --2- 172.21.15.43:0/614304805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9f027da00 0x55e9f027acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.813+0000 7f3105f7c740 1 --2- 172.21.15.43:0/614304805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f0274e80 0x55e9f027a6c0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.813+0000 7f3105f7c740 1 --2- 172.21.15.43:0/614304805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9f02744d0 0x55e9f027d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.813+0000 7f30e4ff9640 0 client.16353 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:04.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.813+0000 7f3105f7c740 1 -- 172.21.15.43:0/614304805 >> 172.21.15.43:0/614304805 conn(0x55e9f026eba0 msgr2=0x55e9f02708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.813+0000 7f3105f7c740 1 -- 172.21.15.43:0/614304805 shutdown_connections 2024-04-21T08:40:04.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.813+0000 7f3105f7c740 1 -- 172.21.15.43:0/614304805 wait complete. 2024-04-21T08:40:04.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.813+0000 7f3105f7c740 20 client.16353 trim_cache size 0 max 16384 2024-04-21T08:40:04.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.827+0000 7f3105f7c740 1 Processor -- start 2024-04-21T08:40:04.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.827+0000 7f3105f7c740 1 -- start start 2024-04-21T08:40:04.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.827+0000 7f3105f7c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9f02744d0 0x55e9f0289800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.827+0000 7f3105f7c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f0274e80 0x55e9f0275ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.827+0000 7f3105f7c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9f027da00 0x55e9f027cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.827+0000 7f3105f7c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1b1650 con 0x55e9f0274e80 2024-04-21T08:40:04.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.827+0000 7f3105f7c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1b3190 con 0x55e9f02744d0 2024-04-21T08:40:04.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.827+0000 7f3105f7c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1b1d20 con 0x55e9f027da00 2024-04-21T08:40:04.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.827+0000 7f310104f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f0274e80 0x55e9f0275ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.827+0000 7f310104f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f0274e80 0x55e9f0275ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49158/0 (socket says 172.21.15.43:49158) 2024-04-21T08:40:04.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.827+0000 7f310104f640 1 -- 172.21.15.43:0/3429040512 learned_addr learned my addr 172.21.15.43:0/3429040512 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.827+0000 7f310084e640 1 --2- 172.21.15.43:0/3429040512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9f02744d0 0x55e9f0289800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.827+0000 7f310104f640 1 -- 172.21.15.43:0/3429040512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9f027da00 msgr2=0x55e9f027cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.827+0000 7f310104f640 1 --2- 172.21.15.43:0/3429040512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9f027da00 0x55e9f027cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.827+0000 7f310104f640 1 -- 172.21.15.43:0/3429040512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9f02744d0 msgr2=0x55e9f0289800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.827+0000 7f310104f640 1 --2- 172.21.15.43:0/3429040512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9f02744d0 0x55e9f0289800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.827+0000 7f310104f640 1 -- 172.21.15.43:0/3429040512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9f027b040 con 0x55e9f0274e80 2024-04-21T08:40:04.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.827+0000 7f310104f640 1 --2- 172.21.15.43:0/3429040512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f0274e80 0x55e9f0275ea0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f30f800e610 tx=0x7f30f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5387a5690edbe7a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.827+0000 7f30e6ffd640 1 -- 172.21.15.43:0/3429040512 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8020020 con 0x55e9f0274e80 2024-04-21T08:40:04.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.828+0000 7f30e6ffd640 1 -- 172.21.15.43:0/3429040512 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30f800ee30 con 0x55e9f0274e80 2024-04-21T08:40:04.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.828+0000 7f30e6ffd640 1 -- 172.21.15.43:0/3429040512 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8003850 con 0x55e9f0274e80 2024-04-21T08:40:04.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.828+0000 7f3105f7c740 1 -- 172.21.15.43:0/3429040512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f0274e80 msgr2=0x55e9f0275ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.828+0000 7f3105f7c740 1 --2- 172.21.15.43:0/3429040512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f0274e80 0x55e9f0275ea0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f30f800e610 tx=0x7f30f800e4b0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.828+0000 7f3105f7c740 1 -- 172.21.15.43:0/3429040512 shutdown_connections 2024-04-21T08:40:04.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.828+0000 7f3105f7c740 1 --2- 172.21.15.43:0/3429040512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9f027da00 0x55e9f027cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.828+0000 7f3105f7c740 1 --2- 172.21.15.43:0/3429040512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f0274e80 0x55e9f0275ea0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.828+0000 7f3105f7c740 1 --2- 172.21.15.43:0/3429040512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9f02744d0 0x55e9f0289800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.828+0000 7f3105f7c740 1 -- 172.21.15.43:0/3429040512 >> 172.21.15.43:0/3429040512 conn(0x55e9f026eba0 msgr2=0x55e9f0270840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.828+0000 7f3105f7c740 1 -- 172.21.15.43:0/3429040512 shutdown_connections 2024-04-21T08:40:04.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.828+0000 7f3105f7c740 1 -- 172.21.15.43:0/3429040512 wait complete. 2024-04-21T08:40:04.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.828+0000 7f3105f7c740 1 -- start start 2024-04-21T08:40:04.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.829+0000 7f3105f7c740 1 Processor -- start 2024-04-21T08:40:04.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.829+0000 7f3105f7c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9f02744d0 0x55e9f0285fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.829+0000 7f3105f7c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9f0274e80 0x55e9f0281350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.829+0000 7f3105f7c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f027da00 0x55e9f0281890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.829+0000 7f3105f7c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1b2ac0 con 0x55e9f027da00 2024-04-21T08:40:04.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.829+0000 7f3105f7c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1b23f0 con 0x55e9f02744d0 2024-04-21T08:40:04.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.829+0000 7f3105f7c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1af630 con 0x55e9f0274e80 2024-04-21T08:40:04.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.829+0000 7f3105f7c740 10 client.0 _ll_register_callbacks cb 0x7f30f8009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:04.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.829+0000 7f310084e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f027da00 0x55e9f0281890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.829+0000 7f310084e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f027da00 0x55e9f0281890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49166/0 (socket says 172.21.15.43:49166) 2024-04-21T08:40:04.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.829+0000 7f310084e640 1 -- 172.21.15.43:0/342402208 learned_addr learned my addr 172.21.15.43:0/342402208 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:04.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.829+0000 7f310084e640 1 --2- 172.21.15.43:0/342402208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9f02744d0 0x55e9f0285fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.829+0000 7f310104f640 1 --2- 172.21.15.43:0/342402208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9f0274e80 0x55e9f0281350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.829+0000 7f310084e640 1 -- 172.21.15.43:0/342402208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9f0274e80 msgr2=0x55e9f0281350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.829+0000 7f310084e640 1 --2- 172.21.15.43:0/342402208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9f0274e80 0x55e9f0281350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.829+0000 7f310084e640 1 -- 172.21.15.43:0/342402208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9f02744d0 msgr2=0x55e9f0285fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:04.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.829+0000 7f310084e640 1 --2- 172.21.15.43:0/342402208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9f02744d0 0x55e9f0285fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.829+0000 7f310084e640 1 -- 172.21.15.43:0/342402208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9f0281f80 con 0x55e9f027da00 2024-04-21T08:40:04.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.829+0000 7f310104f640 1 --2- 172.21.15.43:0/342402208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9f0274e80 0x55e9f0281350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:04.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.830+0000 7f310084e640 1 --2- 172.21.15.43:0/342402208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f027da00 0x55e9f0281890 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f30ec036040 tx=0x7f30ec02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7f2ab7259fb468 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:04.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.830+0000 7f30e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:04.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.830+0000 7f30e4ff9640 1 -- 172.21.15.43:0/342402208 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec053330 con 0x55e9f027da00 2024-04-21T08:40:04.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.830+0000 7f30e4ff9640 1 -- 172.21.15.43:0/342402208 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f30ec0534d0 con 0x55e9f027da00 2024-04-21T08:40:04.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.830+0000 7f3105f7c740 1 -- 172.21.15.43:0/342402208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30f800ee30 con 0x55e9f027da00 2024-04-21T08:40:04.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.830+0000 7f30e4ff9640 1 -- 172.21.15.43:0/342402208 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f30ec0536d0 con 0x55e9f027da00 2024-04-21T08:40:04.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.830+0000 7f3105f7c740 10 client.16359 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:04.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.830+0000 7f3105f7c740 1 -- 172.21.15.43:0/342402208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e9f02f4170 con 0x55e9f027da00 2024-04-21T08:40:04.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.830+0000 7f30e4ff9640 1 -- 172.21.15.43:0/342402208 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30f800ee30 con 0x55e9f027da00 2024-04-21T08:40:04.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.830+0000 7f30e4ff9640 1 -- 172.21.15.43:0/342402208 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f30ec040040 con 0x55e9f027da00 2024-04-21T08:40:04.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.830+0000 7f3105f7c740 10 client.16359 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:04.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.830+0000 7f3105f7c740 10 client.16359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:04.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.830+0000 7f3105f7c740 20 client.16359 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:04.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.830+0000 7f3105f7c740 1 --2- 172.21.15.43:0/342402208 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e9f02f43f0 0x55e9f02f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:04.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.830+0000 7f3105f7c740 4 client.16359 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:40:04.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.830+0000 7f3105f7c740 1 -- 172.21.15.43:0/342402208 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9f01e6c10 con 0x55e9f02f43f0 2024-04-21T08:40:04.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.831+0000 7f310104f640 1 --2- 172.21.15.43:0/342402208 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e9f02f43f0 0x55e9f02f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:04.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.831+0000 7f30e4ff9640 10 client.16359 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:04.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.831+0000 7f310104f640 1 --2- 172.21.15.43:0/342402208 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e9f02f43f0 0x55e9f02f67e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fb9b479930ab697 server_cookie=847401bf266f919e in_seq=0 out_seq=0 2024-04-21T08:40:04.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.831+0000 7f30e4ff9640 1 -- 172.21.15.43:0/342402208 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9f01e6c10 con 0x55e9f02f43f0 2024-04-21T08:40:04.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.831+0000 7f30e4ff9640 10 client.16359 handle_command_reply: tid=0 2024-04-21T08:40:04.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.831+0000 7f3105f7c740 1 client.16359 shutdown 2024-04-21T08:40:04.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.831+0000 7f3105f7c740 1 -- 172.21.15.43:0/342402208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f027da00 msgr2=0x55e9f0281890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.831+0000 7f3105f7c740 1 --2- 172.21.15.43:0/342402208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f027da00 0x55e9f0281890 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f30ec036040 tx=0x7f30ec02bc30 comp rx=0 tx=0).stop 2024-04-21T08:40:04.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.831+0000 7f3105f7c740 1 -- 172.21.15.43:0/342402208 shutdown_connections 2024-04-21T08:40:04.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.831+0000 7f3105f7c740 1 --2- 172.21.15.43:0/342402208 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e9f02f43f0 0x55e9f02f67e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.831+0000 7f3105f7c740 1 --2- 172.21.15.43:0/342402208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9f027da00 0x55e9f0281890 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.831+0000 7f3105f7c740 1 --2- 172.21.15.43:0/342402208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9f0274e80 0x55e9f0281350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.831+0000 7f30e4ff9640 0 client.16359 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:04.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.831+0000 7f3105f7c740 1 --2- 172.21.15.43:0/342402208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9f02744d0 0x55e9f0285fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.831+0000 7f3105f7c740 1 -- 172.21.15.43:0/342402208 >> 172.21.15.43:0/342402208 conn(0x55e9f026eba0 msgr2=0x55e9f026ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.831+0000 7f3105f7c740 1 -- 172.21.15.43:0/342402208 shutdown_connections 2024-04-21T08:40:04.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.831+0000 7f3105f7c740 1 -- 172.21.15.43:0/342402208 wait complete. 2024-04-21T08:40:04.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.831+0000 7f3105f7c740 20 client.16359 trim_cache size 0 max 16384 2024-04-21T08:40:04.837 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:40:04.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.832+0000 7f31032b3640 1 -- 172.21.15.43:0/318449200 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f30e80a3e40 msgr2=0x7f30e80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.832+0000 7f31032b3640 1 --2- 172.21.15.43:0/318449200 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f30e80a3e40 0x7f30e80a6300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f30f40ce590 tx=0x7f30f4098910 comp rx=0 tx=0).stop 2024-04-21T08:40:04.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.832+0000 7f31032b3640 1 -- 172.21.15.43:0/318449200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30fc10e900 msgr2=0x7f30fc19a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:04.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.832+0000 7f31032b3640 1 --2- 172.21.15.43:0/318449200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30fc10e900 0x7f30fc19a850 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f30f400e9d0 tx=0x7f30f400ee90 comp rx=0 tx=0).stop 2024-04-21T08:40:04.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.832+0000 7f31032b3640 1 -- 172.21.15.43:0/318449200 shutdown_connections 2024-04-21T08:40:04.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.832+0000 7f31032b3640 1 --2- 172.21.15.43:0/318449200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30fc10e900 0x7f30fc19a850 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.832+0000 7f31032b3640 1 --2- 172.21.15.43:0/318449200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30fc101670 0x7f30fc1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.832+0000 7f31032b3640 1 --2- 172.21.15.43:0/318449200 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f30e80a3e40 0x7f30e80a6300 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.832+0000 7f31032b3640 1 --2- 172.21.15.43:0/318449200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30fc100d50 0x7f30fc1a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:04.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.832+0000 7f31032b3640 1 -- 172.21.15.43:0/318449200 >> 172.21.15.43:0/318449200 conn(0x7f30fc077f50 msgr2=0x7f30fc110090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:04.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.832+0000 7f31032b3640 1 -- 172.21.15.43:0/318449200 shutdown_connections 2024-04-21T08:40:04.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:04.832+0000 7f31032b3640 1 -- 172.21.15.43:0/318449200 wait complete. 2024-04-21T08:40:05.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:04 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/4132344479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:40:05.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:04 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/4132344479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:40:05.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:04 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/4132344479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:40:05.842 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:40:05.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:40:05.843 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:40:05.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.964+0000 7f05bce55640 1 -- 172.21.15.43:0/1313363712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05b8102b40 msgr2=0x7f05b8106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:05.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.964+0000 7f05bce55640 1 --2- 172.21.15.43:0/1313363712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05b8102b40 0x7f05b8106b80 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f05a0009900 tx=0x7f05a002bc30 comp rx=0 tx=0).stop 2024-04-21T08:40:05.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.964+0000 7f05bce55640 1 -- 172.21.15.43:0/1313363712 shutdown_connections 2024-04-21T08:40:05.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.964+0000 7f05bce55640 1 --2- 172.21.15.43:0/1313363712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05b8107260 0x7f05b810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:05.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.964+0000 7f05bce55640 1 --2- 172.21.15.43:0/1313363712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05b8102b40 0x7f05b8106b80 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:05.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.964+0000 7f05bce55640 1 --2- 172.21.15.43:0/1313363712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05b8102190 0x7f05b8102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:05.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.964+0000 7f05bce55640 1 -- 172.21.15.43:0/1313363712 >> 172.21.15.43:0/1313363712 conn(0x7f05b80fd990 msgr2=0x7f05b80ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:05.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.964+0000 7f05bce55640 1 -- 172.21.15.43:0/1313363712 shutdown_connections 2024-04-21T08:40:05.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.964+0000 7f05bce55640 1 -- 172.21.15.43:0/1313363712 wait complete. 2024-04-21T08:40:05.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.965+0000 7f05bce55640 1 Processor -- start 2024-04-21T08:40:05.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.965+0000 7f05bce55640 1 -- start start 2024-04-21T08:40:05.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.965+0000 7f05bce55640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05b8102190 0x7f05b819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:05.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.965+0000 7f05bce55640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05b8107260 0x7f05b819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:05.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.965+0000 7f05bce55640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05b81985a0 0x7f05b8198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:05.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.965+0000 7f05bce55640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81121a0 con 0x7f05b81985a0 2024-04-21T08:40:05.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.965+0000 7f05bce55640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f05b8112020 con 0x7f05b8102190 2024-04-21T08:40:05.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.965+0000 7f05bce55640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05b8112320 con 0x7f05b8107260 2024-04-21T08:40:05.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.966+0000 7f05b6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05b81985a0 0x7f05b8198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:05.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.966+0000 7f05b6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05b81985a0 0x7f05b8198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49174/0 (socket says 172.21.15.43:49174) 2024-04-21T08:40:05.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.966+0000 7f05b6d9d640 1 -- 172.21.15.43:0/2644396117 learned_addr learned my addr 172.21.15.43:0/2644396117 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:05.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.966+0000 7f05b659c640 1 --2- 172.21.15.43:0/2644396117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05b8102190 0x7f05b819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:05.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.966+0000 7f05affff640 1 --2- 172.21.15.43:0/2644396117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05b8107260 0x7f05b819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:05.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.966+0000 7f05b6d9d640 1 -- 172.21.15.43:0/2644396117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05b8107260 msgr2=0x7f05b819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:05.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.966+0000 7f05b6d9d640 1 --2- 172.21.15.43:0/2644396117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05b8107260 0x7f05b819e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:05.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.966+0000 7f05b6d9d640 1 -- 172.21.15.43:0/2644396117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05b8102190 msgr2=0x7f05b819e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:05.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.966+0000 7f05b6d9d640 1 --2- 172.21.15.43:0/2644396117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05b8102190 0x7f05b819e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:05.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.966+0000 7f05b6d9d640 1 -- 172.21.15.43:0/2644396117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b81990c0 con 0x7f05b81985a0 2024-04-21T08:40:05.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.966+0000 7f05b659c640 1 --2- 172.21.15.43:0/2644396117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05b8102190 0x7f05b819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:05.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.966+0000 7f05b6d9d640 1 --2- 172.21.15.43:0/2644396117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05b81985a0 0x7f05b8198a50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f05b000eb10 tx=0x7f05b000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:05.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.966+0000 7f05af7fe640 1 -- 172.21.15.43:0/2644396117 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0004b20 con 0x7f05b81985a0 2024-04-21T08:40:05.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.966+0000 7f05af7fe640 1 -- 172.21.15.43:0/2644396117 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05b0004cc0 con 0x7f05b81985a0 2024-04-21T08:40:05.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.966+0000 7f05bce55640 1 -- 172.21.15.43:0/2644396117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b81993b0 con 0x7f05b81985a0 2024-04-21T08:40:05.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.966+0000 7f05af7fe640 1 -- 172.21.15.43:0/2644396117 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0010670 con 0x7f05b81985a0 2024-04-21T08:40:05.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.966+0000 7f05bce55640 1 -- 172.21.15.43:0/2644396117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b81a55b0 con 0x7f05b81985a0 2024-04-21T08:40:05.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.969+0000 7f05bfa48740 1 Processor -- start 2024-04-21T08:40:05.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.969+0000 7f05af7fe640 1 -- 172.21.15.43:0/2644396117 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f05b0005020 con 0x7f05b81985a0 2024-04-21T08:40:05.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.969+0000 7f05bfa48740 1 -- start start 2024-04-21T08:40:05.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.969+0000 7f05bfa48740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555aa25b68e0 0x555aa25b6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:05.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.969+0000 7f05bfa48740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25b7290 0x555aa25bf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:05.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.969+0000 7f05bfa48740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aa25bfe10 0x555aa25c2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:05.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.969+0000 7f05bfa48740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555aa251af00 con 0x555aa25b7290 2024-04-21T08:40:05.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.969+0000 7f05bfa48740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555aa24b8590 con 0x555aa25b68e0 2024-04-21T08:40:05.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.969+0000 7f05bfa48740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555aa2516ec0 con 0x555aa25bfe10 2024-04-21T08:40:05.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.970+0000 7f05b659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25b7290 0x555aa25bf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:05.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.970+0000 7f05b659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25b7290 0x555aa25bf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49180/0 (socket says 172.21.15.43:49180) 2024-04-21T08:40:05.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.970+0000 7f05b659c640 1 -- 172.21.15.43:0/1955899694 learned_addr learned my addr 172.21.15.43:0/1955899694 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:05.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.970+0000 7f05affff640 1 --2- 172.21.15.43:0/1955899694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555aa25b68e0 0x555aa25b6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:05.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.970+0000 7f05af7fe640 1 --2- 172.21.15.43:0/2644396117 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f05a40a3e40 0x7f05a40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:05.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.970+0000 7f05affff640 1 --2- 172.21.15.43:0/1955899694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aa25bfe10 0x555aa25c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:05.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.970+0000 7f05b659c640 1 -- 172.21.15.43:0/1955899694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aa25bfe10 msgr2=0x555aa25c2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:05.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.970+0000 7f05b659c640 1 --2- 172.21.15.43:0/1955899694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aa25bfe10 0x555aa25c2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:05.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.970+0000 7f05b659c640 1 -- 172.21.15.43:0/1955899694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555aa25b68e0 msgr2=0x555aa25b6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:05.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.970+0000 7f05b659c640 1 --2- 172.21.15.43:0/1955899694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555aa25b68e0 0x555aa25b6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:05.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.970+0000 7f05b6d9d640 1 --2- 172.21.15.43:0/2644396117 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f05a40a3e40 0x7f05a40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:05.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.970+0000 7f05b659c640 1 -- 172.21.15.43:0/1955899694 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aa22f57e0 con 0x555aa25b7290 2024-04-21T08:40:05.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.970+0000 7f05af7fe640 1 -- 172.21.15.43:0/2644396117 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f05b00d1430 con 0x7f05b81985a0 2024-04-21T08:40:05.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.970+0000 7f05b659c640 1 --2- 172.21.15.43:0/1955899694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25b7290 0x555aa25bf840 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009d40 tx=0x7f05a80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9ef62ad1ece325 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:05.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.970+0000 7f05acff9640 1 -- 172.21.15.43:0/1955899694 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a8012650 con 0x555aa25b7290 2024-04-21T08:40:05.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.970+0000 7f05acff9640 1 -- 172.21.15.43:0/1955899694 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05a8009e30 con 0x555aa25b7290 2024-04-21T08:40:05.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.970+0000 7f05acff9640 1 -- 172.21.15.43:0/1955899694 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a800fbd0 con 0x555aa25b7290 2024-04-21T08:40:05.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.970+0000 7f05b6d9d640 1 --2- 172.21.15.43:0/2644396117 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f05a40a3e40 0x7f05a40a6300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f05b00ce690 tx=0x7f05b00990d0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:05.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.971+0000 7f05bfa48740 1 -- 172.21.15.43:0/1955899694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25b7290 msgr2=0x555aa25bf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:05.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.971+0000 7f05bfa48740 1 --2- 172.21.15.43:0/1955899694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25b7290 0x555aa25bf840 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009d40 tx=0x7f05a80117b0 comp rx=0 tx=0).stop 2024-04-21T08:40:05.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.971+0000 7f05bfa48740 1 -- 172.21.15.43:0/1955899694 shutdown_connections 2024-04-21T08:40:05.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.971+0000 7f05bfa48740 1 --2- 172.21.15.43:0/1955899694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aa25bfe10 0x555aa25c2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:05.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.971+0000 7f05bfa48740 1 --2- 172.21.15.43:0/1955899694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25b7290 0x555aa25bf840 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:05.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.971+0000 7f05bfa48740 1 --2- 172.21.15.43:0/1955899694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555aa25b68e0 0x555aa25b6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:05.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.971+0000 7f05bfa48740 1 -- 172.21.15.43:0/1955899694 >> 172.21.15.43:0/1955899694 conn(0x555aa25b0fb0 msgr2=0x555aa25b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:05.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.971+0000 7f05bfa48740 1 -- 172.21.15.43:0/1955899694 shutdown_connections 2024-04-21T08:40:05.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.971+0000 7f05bfa48740 1 -- 172.21.15.43:0/1955899694 wait complete. 2024-04-21T08:40:05.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.972+0000 7f05bfa48740 1 -- start start 2024-04-21T08:40:05.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.972+0000 7f05bfa48740 1 Processor -- start 2024-04-21T08:40:05.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.972+0000 7f05bfa48740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aa25b68e0 0x555aa25bce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:05.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.972+0000 7f05bfa48740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25b7290 0x555aa25bd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:05.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.972+0000 7f05bfa48740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555aa25bfe10 0x555aa25bd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:05.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.972+0000 7f05bfa48740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555aa25c2d80 con 0x555aa25b7290 2024-04-21T08:40:05.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.972+0000 7f05bfa48740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555aa24d4580 con 0x555aa25bfe10 2024-04-21T08:40:05.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.972+0000 7f05bfa48740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555aa25c2f00 con 0x555aa25b68e0 2024-04-21T08:40:05.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.972+0000 7f05bfa48740 10 client.0 _ll_register_callbacks cb 0x555aa24da650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:05.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.972+0000 7f05b659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25b7290 0x555aa25bd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:05.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.972+0000 7f05b659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25b7290 0x555aa25bd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49188/0 (socket says 172.21.15.43:49188) 2024-04-21T08:40:05.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.972+0000 7f05b659c640 1 -- 172.21.15.43:0/2527157989 learned_addr learned my addr 172.21.15.43:0/2527157989 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:05.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.972+0000 7f05affff640 1 --2- 172.21.15.43:0/2527157989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555aa25bfe10 0x555aa25bd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:05.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.972+0000 7f05b659c640 1 -- 172.21.15.43:0/2527157989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aa25b68e0 msgr2=0x555aa25bce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:05.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.972+0000 7f05b659c640 1 --2- 172.21.15.43:0/2527157989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aa25b68e0 0x555aa25bce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:05.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.972+0000 7f05b659c640 1 -- 172.21.15.43:0/2527157989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555aa25bfe10 msgr2=0x555aa25bd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:05.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.972+0000 7f05b659c640 1 --2- 172.21.15.43:0/2527157989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555aa25bfe10 0x555aa25bd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:05.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.972+0000 7f05b659c640 1 -- 172.21.15.43:0/2527157989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aa2355f70 con 0x555aa25b7290 2024-04-21T08:40:05.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.972+0000 7f05affff640 1 --2- 172.21.15.43:0/2527157989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555aa25bfe10 0x555aa25bd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:05.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.973+0000 7f05b659c640 1 --2- 172.21.15.43:0/2527157989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25b7290 0x555aa25bd340 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f05a800f770 tx=0x7f05a80058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32c190c3b941ca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:05.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.973+0000 7f059dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:05.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.973+0000 7f059dffb640 1 -- 172.21.15.43:0/2527157989 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a8028020 con 0x555aa25b7290 2024-04-21T08:40:05.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.973+0000 7f059dffb640 1 -- 172.21.15.43:0/2527157989 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f05a8005240 con 0x555aa25b7290 2024-04-21T08:40:05.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.973+0000 7f05bfa48740 1 -- 172.21.15.43:0/2527157989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05a8009e30 con 0x555aa25b7290 2024-04-21T08:40:05.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.973+0000 7f059dffb640 1 -- 172.21.15.43:0/2527157989 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f05a8033050 con 0x555aa25b7290 2024-04-21T08:40:05.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.973+0000 7f05bfa48740 10 client.16371 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:05.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.973+0000 7f05bfa48740 1 -- 172.21.15.43:0/2527157989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555aa21ed170 con 0x555aa25b7290 2024-04-21T08:40:05.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.973+0000 7f059dffb640 1 -- 172.21.15.43:0/2527157989 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05a8009e30 con 0x555aa25b7290 2024-04-21T08:40:05.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.973+0000 7f059dffb640 1 -- 172.21.15.43:0/2527157989 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f05a80124c0 con 0x555aa25b7290 2024-04-21T08:40:05.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.973+0000 7f05bfa48740 10 client.16371 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:05.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.973+0000 7f05bfa48740 10 client.16371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:05.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.973+0000 7f05bfa48740 20 client.16371 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:05.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.973+0000 7f05bfa48740 1 --2- 172.21.15.43:0/2527157989 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555aa25d0300 0x555aa25d26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:05.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.973+0000 7f05bfa48740 4 client.16371 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:05.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.973+0000 7f05bfa48740 1 -- 172.21.15.43:0/2527157989 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555aa2528920 con 0x555aa25d0300 2024-04-21T08:40:05.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.974+0000 7f05affff640 1 --2- 172.21.15.43:0/2527157989 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555aa25d0300 0x555aa25d26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:05.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.974+0000 7f05affff640 1 --2- 172.21.15.43:0/2527157989 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555aa25d0300 0x555aa25d26f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=590630dbe6cc1582 server_cookie=cda2bde1b838ab1a in_seq=0 out_seq=0 2024-04-21T08:40:05.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.974+0000 7f059dffb640 10 client.16371 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:05.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.975+0000 7f059dffb640 1 -- 172.21.15.43:0/2527157989 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555aa2528920 con 0x555aa25d0300 2024-04-21T08:40:05.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.975+0000 7f059dffb640 10 client.16371 handle_command_reply: tid=0 2024-04-21T08:40:05.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.975+0000 7f05bfa48740 1 client.16371 shutdown 2024-04-21T08:40:05.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.975+0000 7f05bfa48740 1 -- 172.21.15.43:0/2527157989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25b7290 msgr2=0x555aa25bd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:05.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.975+0000 7f05bfa48740 1 --2- 172.21.15.43:0/2527157989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25b7290 0x555aa25bd340 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f05a800f770 tx=0x7f05a80058d0 comp rx=0 tx=0).stop 2024-04-21T08:40:05.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.975+0000 7f05bfa48740 1 -- 172.21.15.43:0/2527157989 shutdown_connections 2024-04-21T08:40:05.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.975+0000 7f05bfa48740 1 --2- 172.21.15.43:0/2527157989 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555aa25d0300 0x555aa25d26f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:05.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.975+0000 7f05bfa48740 1 --2- 172.21.15.43:0/2527157989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555aa25bfe10 0x555aa25bd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:05.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.975+0000 7f05bfa48740 1 --2- 172.21.15.43:0/2527157989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25b7290 0x555aa25bd340 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:05.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.975+0000 7f059dffb640 0 client.16371 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:05.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.975+0000 7f05bfa48740 1 --2- 172.21.15.43:0/2527157989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aa25b68e0 0x555aa25bce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:05.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.975+0000 7f05bfa48740 1 -- 172.21.15.43:0/2527157989 >> 172.21.15.43:0/2527157989 conn(0x555aa25b0fb0 msgr2=0x555aa25b2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:05.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.975+0000 7f05bfa48740 1 -- 172.21.15.43:0/2527157989 shutdown_connections 2024-04-21T08:40:05.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.975+0000 7f05bfa48740 1 -- 172.21.15.43:0/2527157989 wait complete. 2024-04-21T08:40:05.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.975+0000 7f05bfa48740 20 client.16371 trim_cache size 0 max 16384 2024-04-21T08:40:05.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.993+0000 7f05bfa48740 1 Processor -- start 2024-04-21T08:40:05.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.993+0000 7f05bfa48740 1 -- start start 2024-04-21T08:40:05.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.993+0000 7f05bfa48740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25b68e0 0x555aa25bf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:05.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.993+0000 7f05bfa48740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555aa25b7290 0x555aa25cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:05.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.993+0000 7f05bfa48740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aa25bfe10 0x555aa25bbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:05.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.994+0000 7f05bfa48740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81af490 con 0x555aa25b68e0 2024-04-21T08:40:05.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.994+0000 7f05bfa48740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81b0fd0 con 0x555aa25b7290 2024-04-21T08:40:05.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.994+0000 7f05bfa48740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81afb60 con 0x555aa25bfe10 2024-04-21T08:40:05.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.994+0000 7f05affff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25b68e0 0x555aa25bf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:05.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.994+0000 7f05b659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555aa25b7290 0x555aa25cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:05.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.994+0000 7f05affff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25b68e0 0x555aa25bf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49192/0 (socket says 172.21.15.43:49192) 2024-04-21T08:40:05.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.994+0000 7f05affff640 1 -- 172.21.15.43:0/511291503 learned_addr learned my addr 172.21.15.43:0/511291503 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:05.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.994+0000 7f05affff640 1 --2- 172.21.15.43:0/511291503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aa25bfe10 0x555aa25bbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:05.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.994+0000 7f05affff640 1 -- 172.21.15.43:0/511291503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aa25bfe10 msgr2=0x555aa25bbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:05.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.994+0000 7f05affff640 1 --2- 172.21.15.43:0/511291503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aa25bfe10 0x555aa25bbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:05.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.994+0000 7f05affff640 1 -- 172.21.15.43:0/511291503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555aa25b7290 msgr2=0x555aa25cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:05.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.994+0000 7f05affff640 1 --2- 172.21.15.43:0/511291503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555aa25b7290 0x555aa25cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:05.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.994+0000 7f05affff640 1 -- 172.21.15.43:0/511291503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aa25b15c0 con 0x555aa25b68e0 2024-04-21T08:40:05.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.994+0000 7f05b659c640 1 --2- 172.21.15.43:0/511291503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555aa25b7290 0x555aa25cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:40:05.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.994+0000 7f05affff640 1 --2- 172.21.15.43:0/511291503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25b68e0 0x555aa25bf530 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f05b806b5d0 tx=0x7f05a002c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30dff1beb99d6f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:05.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.996+0000 7f059ffff640 1 -- 172.21.15.43:0/511291503 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a004f020 con 0x555aa25b68e0 2024-04-21T08:40:05.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.996+0000 7f059ffff640 1 -- 172.21.15.43:0/511291503 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05a002cc90 con 0x555aa25b68e0 2024-04-21T08:40:05.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.996+0000 7f059ffff640 1 -- 172.21.15.43:0/511291503 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a0043450 con 0x555aa25b68e0 2024-04-21T08:40:05.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.997+0000 7f05bfa48740 1 -- 172.21.15.43:0/511291503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25b68e0 msgr2=0x555aa25bf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:05.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.997+0000 7f05bfa48740 1 --2- 172.21.15.43:0/511291503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25b68e0 0x555aa25bf530 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f05b806b5d0 tx=0x7f05a002c610 comp rx=0 tx=0).stop 2024-04-21T08:40:05.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.997+0000 7f05bfa48740 1 -- 172.21.15.43:0/511291503 shutdown_connections 2024-04-21T08:40:05.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.997+0000 7f05bfa48740 1 --2- 172.21.15.43:0/511291503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aa25bfe10 0x555aa25bbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:05.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.997+0000 7f05bfa48740 1 --2- 172.21.15.43:0/511291503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555aa25b7290 0x555aa25cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:05.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.997+0000 7f05bfa48740 1 --2- 172.21.15.43:0/511291503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25b68e0 0x555aa25bf530 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:05.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.997+0000 7f05bfa48740 1 -- 172.21.15.43:0/511291503 >> 172.21.15.43:0/511291503 conn(0x555aa25b0fb0 msgr2=0x555aa25b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:05.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.997+0000 7f05bfa48740 1 -- 172.21.15.43:0/511291503 shutdown_connections 2024-04-21T08:40:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.997+0000 7f05bfa48740 1 -- 172.21.15.43:0/511291503 wait complete. 2024-04-21T08:40:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.997+0000 7f05bfa48740 1 -- start start 2024-04-21T08:40:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.997+0000 7f05bfa48740 1 Processor -- start 2024-04-21T08:40:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.998+0000 7f05bfa48740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aa25b68e0 0x555aa25c8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.998+0000 7f05bfa48740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555aa25b7290 0x555aa25c8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.998+0000 7f05bfa48740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25bfe10 0x555aa25c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.998+0000 7f05bfa48740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81b0900 con 0x555aa25bfe10 2024-04-21T08:40:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.998+0000 7f05bfa48740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81b0230 con 0x555aa25b7290 2024-04-21T08:40:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.998+0000 7f05bfa48740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81ad470 con 0x555aa25b68e0 2024-04-21T08:40:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.998+0000 7f05bfa48740 10 client.0 _ll_register_callbacks cb 0x7f05a8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.998+0000 7f05affff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25bfe10 0x555aa25c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.998+0000 7f05affff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25bfe10 0x555aa25c5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49208/0 (socket says 172.21.15.43:49208) 2024-04-21T08:40:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.998+0000 7f05affff640 1 -- 172.21.15.43:0/792469586 learned_addr learned my addr 172.21.15.43:0/792469586 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:06.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.998+0000 7f05affff640 1 --2- 172.21.15.43:0/792469586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aa25b68e0 0x555aa25c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:06.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.998+0000 7f05b659c640 1 --2- 172.21.15.43:0/792469586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555aa25b7290 0x555aa25c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:06.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.998+0000 7f05affff640 1 -- 172.21.15.43:0/792469586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aa25b68e0 msgr2=0x555aa25c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:06.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.998+0000 7f05affff640 1 --2- 172.21.15.43:0/792469586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aa25b68e0 0x555aa25c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.998+0000 7f05affff640 1 -- 172.21.15.43:0/792469586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555aa25b7290 msgr2=0x555aa25c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:06.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.998+0000 7f05affff640 1 --2- 172.21.15.43:0/792469586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555aa25b7290 0x555aa25c8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.998+0000 7f05affff640 1 -- 172.21.15.43:0/792469586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aa25c59a0 con 0x555aa25bfe10 2024-04-21T08:40:06.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.998+0000 7f05affff640 1 --2- 172.21.15.43:0/792469586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25bfe10 0x555aa25c5120 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f05a002c7e0 tx=0x7f05a0005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53823b76c81471a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:06.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.998+0000 7f059dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:06.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.999+0000 7f059dffb640 1 -- 172.21.15.43:0/792469586 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a0059020 con 0x555aa25bfe10 2024-04-21T08:40:06.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.999+0000 7f059dffb640 1 -- 172.21.15.43:0/792469586 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f05a00035f0 con 0x555aa25bfe10 2024-04-21T08:40:06.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.999+0000 7f05bfa48740 1 -- 172.21.15.43:0/792469586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05a002cc90 con 0x555aa25bfe10 2024-04-21T08:40:06.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.999+0000 7f059dffb640 1 -- 172.21.15.43:0/792469586 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f05a004f070 con 0x555aa25bfe10 2024-04-21T08:40:06.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.999+0000 7f05bfa48740 10 client.16377 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:06.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.999+0000 7f05bfa48740 1 -- 172.21.15.43:0/792469586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555aa26371c0 con 0x555aa25bfe10 2024-04-21T08:40:06.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.999+0000 7f059dffb640 1 -- 172.21.15.43:0/792469586 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05a002cc90 con 0x555aa25bfe10 2024-04-21T08:40:06.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.999+0000 7f059dffb640 1 -- 172.21.15.43:0/792469586 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f05a0034440 con 0x555aa25bfe10 2024-04-21T08:40:06.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.999+0000 7f05bfa48740 10 client.16377 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:06.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.999+0000 7f05bfa48740 10 client.16377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:06.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.999+0000 7f05bfa48740 20 client.16377 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:06.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.999+0000 7f05bfa48740 1 --2- 172.21.15.43:0/792469586 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555aa26377d0 0x555aa2639bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:06.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.999+0000 7f05bfa48740 4 client.16377 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:40:06.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.999+0000 7f05bfa48740 1 -- 172.21.15.43:0/792469586 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555aa2522790 con 0x555aa26377d0 2024-04-21T08:40:06.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:05.999+0000 7f05b659c640 1 --2- 172.21.15.43:0/792469586 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555aa26377d0 0x555aa2639bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:06.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.000+0000 7f059dffb640 10 client.16377 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:06.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.000+0000 7f05b659c640 1 --2- 172.21.15.43:0/792469586 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555aa26377d0 0x555aa2639bf0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94128081451dcd5d server_cookie=6cd3243680921320 in_seq=0 out_seq=0 2024-04-21T08:40:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.001+0000 7f059dffb640 1 -- 172.21.15.43:0/792469586 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555aa2522790 con 0x555aa26377d0 2024-04-21T08:40:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.001+0000 7f059dffb640 10 client.16377 handle_command_reply: tid=0 2024-04-21T08:40:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.001+0000 7f05bfa48740 1 client.16377 shutdown 2024-04-21T08:40:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.001+0000 7f05bfa48740 1 -- 172.21.15.43:0/792469586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25bfe10 msgr2=0x555aa25c5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.001+0000 7f05bfa48740 1 --2- 172.21.15.43:0/792469586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25bfe10 0x555aa25c5120 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f05a002c7e0 tx=0x7f05a0005850 comp rx=0 tx=0).stop 2024-04-21T08:40:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.001+0000 7f05bfa48740 1 -- 172.21.15.43:0/792469586 shutdown_connections 2024-04-21T08:40:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.001+0000 7f05bfa48740 1 --2- 172.21.15.43:0/792469586 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555aa26377d0 0x555aa2639bf0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.001+0000 7f05bfa48740 1 --2- 172.21.15.43:0/792469586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555aa25bfe10 0x555aa25c5120 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.001+0000 7f05bfa48740 1 --2- 172.21.15.43:0/792469586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555aa25b7290 0x555aa25c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.001+0000 7f05bfa48740 1 --2- 172.21.15.43:0/792469586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555aa25b68e0 0x555aa25c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.001+0000 7f05bfa48740 1 -- 172.21.15.43:0/792469586 >> 172.21.15.43:0/792469586 conn(0x555aa25b0fb0 msgr2=0x555aa25af780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.001+0000 7f059dffb640 0 client.16377 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.001+0000 7f05bfa48740 1 -- 172.21.15.43:0/792469586 shutdown_connections 2024-04-21T08:40:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.001+0000 7f05bfa48740 1 -- 172.21.15.43:0/792469586 wait complete. 2024-04-21T08:40:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.001+0000 7f05bfa48740 20 client.16377 trim_cache size 0 max 16384 2024-04-21T08:40:06.005 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:40:06.005 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:40:06.005 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "55e6a3ed-235c-4dee-aba8-7823654c4d0a", 2024-04-21T08:40:06.005 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:40:06.005 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:40:06.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.002+0000 7f05bce55640 1 -- 172.21.15.43:0/2644396117 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f05a40a3e40 msgr2=0x7f05a40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:06.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.002+0000 7f05bce55640 1 --2- 172.21.15.43:0/2644396117 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f05a40a3e40 0x7f05a40a6300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f05b00ce690 tx=0x7f05b00990d0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.002+0000 7f05bce55640 1 -- 172.21.15.43:0/2644396117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05b81985a0 msgr2=0x7f05b8198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:06.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.002+0000 7f05bce55640 1 --2- 172.21.15.43:0/2644396117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05b81985a0 0x7f05b8198a50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f05b000eb10 tx=0x7f05b000c2d0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.002+0000 7f05bce55640 1 -- 172.21.15.43:0/2644396117 shutdown_connections 2024-04-21T08:40:06.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.002+0000 7f05bce55640 1 --2- 172.21.15.43:0/2644396117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05b81985a0 0x7f05b8198a50 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.002+0000 7f05bce55640 1 --2- 172.21.15.43:0/2644396117 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f05a40a3e40 0x7f05a40a6300 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.002+0000 7f05bce55640 1 --2- 172.21.15.43:0/2644396117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05b8107260 0x7f05b819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.002+0000 7f05bce55640 1 --2- 172.21.15.43:0/2644396117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05b8102190 0x7f05b819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.002+0000 7f05bce55640 1 -- 172.21.15.43:0/2644396117 >> 172.21.15.43:0/2644396117 conn(0x7f05b80fd990 msgr2=0x7f05b80fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:06.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.002+0000 7f05bce55640 1 -- 172.21.15.43:0/2644396117 shutdown_connections 2024-04-21T08:40:06.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.002+0000 7f05bce55640 1 -- 172.21.15.43:0/2644396117 wait complete. 2024-04-21T08:40:06.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:40:06.013 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:40:06.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.118+0000 7f540b746640 1 -- 172.21.15.43:0/929869220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5404103ae0 msgr2=0x7f540410bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:06.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.118+0000 7f540b746640 1 --2- 172.21.15.43:0/929869220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5404103ae0 0x7f540410bfc0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f53f4009900 tx=0x7f53f402bd60 comp rx=0 tx=0).stop 2024-04-21T08:40:06.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.118+0000 7f540b746640 1 -- 172.21.15.43:0/929869220 shutdown_connections 2024-04-21T08:40:06.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.118+0000 7f540b746640 1 --2- 172.21.15.43:0/929869220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f540410c500 0x7f540410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.118+0000 7f540b746640 1 --2- 172.21.15.43:0/929869220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5404103ae0 0x7f540410bfc0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.118+0000 7f540b746640 1 --2- 172.21.15.43:0/929869220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54041031c0 0x7f54041035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.118+0000 7f540b746640 1 -- 172.21.15.43:0/929869220 >> 172.21.15.43:0/929869220 conn(0x7f5404078000 msgr2=0x7f5404078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:06.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.118+0000 7f540b746640 1 -- 172.21.15.43:0/929869220 shutdown_connections 2024-04-21T08:40:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.118+0000 7f540b746640 1 -- 172.21.15.43:0/929869220 wait complete. 2024-04-21T08:40:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.118+0000 7f540b746640 1 Processor -- start 2024-04-21T08:40:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.118+0000 7f540b746640 1 -- start start 2024-04-21T08:40:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.119+0000 7f540b746640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54041031c0 0x7f54041986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.119+0000 7f540b746640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5404103ae0 0x7f5404198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.119+0000 7f540b746640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f540410c500 0x7f540419cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.119+0000 7f540b746640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5404112190 con 0x7f5404103ae0 2024-04-21T08:40:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.119+0000 7f540b746640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5404112010 con 0x7f540410c500 2024-04-21T08:40:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.119+0000 7f540b746640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5404112310 con 0x7f54041031c0 2024-04-21T08:40:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.119+0000 7f5408ce1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5404103ae0 0x7f5404198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.119+0000 7f5408ce1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5404103ae0 0x7f5404198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49228/0 (socket says 172.21.15.43:49228) 2024-04-21T08:40:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.119+0000 7f5408ce1640 1 -- 172.21.15.43:0/638672782 learned_addr learned my addr 172.21.15.43:0/638672782 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.119+0000 7f5409ce3640 1 --2- 172.21.15.43:0/638672782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f540410c500 0x7f540419cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.119+0000 7f5408ce1640 1 -- 172.21.15.43:0/638672782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54041031c0 msgr2=0x7f54041986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.119+0000 7f5408ce1640 1 --2- 172.21.15.43:0/638672782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54041031c0 0x7f54041986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.119+0000 7f5408ce1640 1 -- 172.21.15.43:0/638672782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f540410c500 msgr2=0x7f540419cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.119+0000 7f5408ce1640 1 --2- 172.21.15.43:0/638672782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f540410c500 0x7f540419cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.119+0000 7f5408ce1640 1 -- 172.21.15.43:0/638672782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540419d6c0 con 0x7f5404103ae0 2024-04-21T08:40:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.119+0000 7f5409ce3640 1 --2- 172.21.15.43:0/638672782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f540410c500 0x7f540419cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.119+0000 7f5408ce1640 1 --2- 172.21.15.43:0/638672782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5404103ae0 0x7f5404198c10 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f53f4005940 tx=0x7f53f40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.119+0000 7f53fa7fc640 1 -- 172.21.15.43:0/638672782 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4043020 con 0x7f5404103ae0 2024-04-21T08:40:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.119+0000 7f53fa7fc640 1 -- 172.21.15.43:0/638672782 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53f402dc30 con 0x7f5404103ae0 2024-04-21T08:40:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.119+0000 7f53fa7fc640 1 -- 172.21.15.43:0/638672782 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4003a10 con 0x7f5404103ae0 2024-04-21T08:40:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.119+0000 7f540b746640 1 -- 172.21.15.43:0/638672782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540419d950 con 0x7f5404103ae0 2024-04-21T08:40:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.119+0000 7f540b746640 1 -- 172.21.15.43:0/638672782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54041a5240 con 0x7f5404103ae0 2024-04-21T08:40:06.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.122+0000 7f53fa7fc640 1 -- 172.21.15.43:0/638672782 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f53f4046020 con 0x7f5404103ae0 2024-04-21T08:40:06.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.122+0000 7f540e40c740 1 Processor -- start 2024-04-21T08:40:06.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.122+0000 7f540e40c740 1 -- start start 2024-04-21T08:40:06.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.122+0000 7f540e40c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36580 0x5609f3e36960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:06.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.122+0000 7f540e40c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609f3e36ea0 0x5609f3e3f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:06.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.122+0000 7f540e40c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609f3e3fa80 0x5609f3e41e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:06.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.122+0000 7f540e40c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5609f3d9b1e0 con 0x5609f3e36580 2024-04-21T08:40:06.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.122+0000 7f540e40c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5609f3d38390 con 0x5609f3e36ea0 2024-04-21T08:40:06.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.122+0000 7f540e40c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5609f3d971a0 con 0x5609f3e3fa80 2024-04-21T08:40:06.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.123+0000 7f5409ce3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36580 0x5609f3e36960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:06.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.123+0000 7f53fa7fc640 1 --2- 172.21.15.43:0/638672782 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f53f00a3ec0 0x7f53f00a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:06.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.123+0000 7f5409ce3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36580 0x5609f3e36960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49242/0 (socket says 172.21.15.43:49242) 2024-04-21T08:40:06.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.123+0000 7f5409ce3640 1 -- 172.21.15.43:0/1744513922 learned_addr learned my addr 172.21.15.43:0/1744513922 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:06.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.123+0000 7f5409ce3640 1 --2- 172.21.15.43:0/638672782 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f53f00a3ec0 0x7f53f00a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:06.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.123+0000 7f53fa7fc640 1 -- 172.21.15.43:0/638672782 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f53f40f2e90 con 0x7f5404103ae0 2024-04-21T08:40:06.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.123+0000 7f5408ce1640 1 --2- 172.21.15.43:0/1744513922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609f3e3fa80 0x5609f3e41e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:06.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.123+0000 7f54094e2640 1 --2- 172.21.15.43:0/1744513922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609f3e36ea0 0x5609f3e3f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:06.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.123+0000 7f5409ce3640 1 -- 172.21.15.43:0/1744513922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609f3e3fa80 msgr2=0x5609f3e41e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:06.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.123+0000 7f5409ce3640 1 --2- 172.21.15.43:0/1744513922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609f3e3fa80 0x5609f3e41e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.123+0000 7f5409ce3640 1 -- 172.21.15.43:0/1744513922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609f3e36ea0 msgr2=0x5609f3e3f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:06.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.123+0000 7f5409ce3640 1 --2- 172.21.15.43:0/1744513922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609f3e36ea0 0x5609f3e3f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.123+0000 7f5409ce3640 1 -- 172.21.15.43:0/1744513922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609f3b777d0 con 0x5609f3e36580 2024-04-21T08:40:06.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.123+0000 7f54094e2640 1 --2- 172.21.15.43:0/1744513922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609f3e36ea0 0x5609f3e3f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:06.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.123+0000 7f5409ce3640 1 --2- 172.21.15.43:0/1744513922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36580 0x5609f3e36960 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f53fc012dc0 tx=0x7f53fc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e6a7421046df16 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:06.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.123+0000 7f5409ce3640 1 --2- 172.21.15.43:0/638672782 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f53f00a3ec0 0x7f53f00a6380 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f53fc010c40 tx=0x7f53fc01d000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:06.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.123+0000 7f53ef7fe640 1 -- 172.21.15.43:0/1744513922 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc022070 con 0x5609f3e36580 2024-04-21T08:40:06.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.123+0000 7f53ef7fe640 1 -- 172.21.15.43:0/1744513922 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53fc009dc0 con 0x5609f3e36580 2024-04-21T08:40:06.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.123+0000 7f53ef7fe640 1 -- 172.21.15.43:0/1744513922 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc01eb30 con 0x5609f3e36580 2024-04-21T08:40:06.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.124+0000 7f540e40c740 1 -- 172.21.15.43:0/1744513922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36580 msgr2=0x5609f3e36960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:06.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.124+0000 7f540e40c740 1 --2- 172.21.15.43:0/1744513922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36580 0x5609f3e36960 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f53fc012dc0 tx=0x7f53fc010740 comp rx=0 tx=0).stop 2024-04-21T08:40:06.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.124+0000 7f540e40c740 1 -- 172.21.15.43:0/1744513922 shutdown_connections 2024-04-21T08:40:06.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.124+0000 7f540e40c740 1 --2- 172.21.15.43:0/1744513922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609f3e3fa80 0x5609f3e41e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.124+0000 7f540e40c740 1 --2- 172.21.15.43:0/1744513922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609f3e36ea0 0x5609f3e3f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.124+0000 7f540e40c740 1 --2- 172.21.15.43:0/1744513922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36580 0x5609f3e36960 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.124+0000 7f540e40c740 1 -- 172.21.15.43:0/1744513922 >> 172.21.15.43:0/1744513922 conn(0x5609f3e30cb0 msgr2=0x5609f3e31090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:06.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.124+0000 7f540e40c740 1 -- 172.21.15.43:0/1744513922 shutdown_connections 2024-04-21T08:40:06.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.124+0000 7f540e40c740 1 -- 172.21.15.43:0/1744513922 wait complete. 2024-04-21T08:40:06.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.125+0000 7f540e40c740 1 -- start start 2024-04-21T08:40:06.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.125+0000 7f540e40c740 1 Processor -- start 2024-04-21T08:40:06.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.125+0000 7f540e40c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36580 0x5609f3e3cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:06.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.125+0000 7f540e40c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609f3e36ea0 0x5609f3e3d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:06.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.125+0000 7f540e40c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609f3e3fa80 0x5609f3e3d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:06.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.125+0000 7f540e40c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5609f3e427b0 con 0x5609f3e36580 2024-04-21T08:40:06.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.125+0000 7f540e40c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5609f3d542f0 con 0x5609f3e3fa80 2024-04-21T08:40:06.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.125+0000 7f540e40c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5609f3e42930 con 0x5609f3e36ea0 2024-04-21T08:40:06.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.125+0000 7f540e40c740 10 client.0 _ll_register_callbacks cb 0x5609f3d61580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:06.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.125+0000 7f54094e2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36580 0x5609f3e3cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:06.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.125+0000 7f54094e2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36580 0x5609f3e3cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49252/0 (socket says 172.21.15.43:49252) 2024-04-21T08:40:06.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.125+0000 7f54094e2640 1 -- 172.21.15.43:0/1230291518 learned_addr learned my addr 172.21.15.43:0/1230291518 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:06.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.125+0000 7f5409ce3640 1 --2- 172.21.15.43:0/1230291518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609f3e3fa80 0x5609f3e3d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:06.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.125+0000 7f5408ce1640 1 --2- 172.21.15.43:0/1230291518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609f3e36ea0 0x5609f3e3d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:06.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.125+0000 7f54094e2640 1 -- 172.21.15.43:0/1230291518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609f3e36ea0 msgr2=0x5609f3e3d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:06.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.125+0000 7f54094e2640 1 --2- 172.21.15.43:0/1230291518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609f3e36ea0 0x5609f3e3d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.125+0000 7f54094e2640 1 -- 172.21.15.43:0/1230291518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609f3e3fa80 msgr2=0x5609f3e3d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:06.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.125+0000 7f54094e2640 1 --2- 172.21.15.43:0/1230291518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609f3e3fa80 0x5609f3e3d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.125+0000 7f54094e2640 1 -- 172.21.15.43:0/1230291518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609f3aad3f0 con 0x5609f3e36580 2024-04-21T08:40:06.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.125+0000 7f5409ce3640 1 --2- 172.21.15.43:0/1230291518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609f3e3fa80 0x5609f3e3d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:06.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.126+0000 7f54094e2640 1 --2- 172.21.15.43:0/1230291518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36580 0x5609f3e3cb20 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f540000de00 tx=0x7f540000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c04937bb0ccdd4d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:06.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.126+0000 7f53ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:06.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.126+0000 7f53ecff9640 1 -- 172.21.15.43:0/1230291518 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5400015070 con 0x5609f3e36580 2024-04-21T08:40:06.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.126+0000 7f53ecff9640 1 -- 172.21.15.43:0/1230291518 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f5400011850 con 0x5609f3e36580 2024-04-21T08:40:06.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.126+0000 7f540e40c740 1 -- 172.21.15.43:0/1230291518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53fc009dc0 con 0x5609f3e36580 2024-04-21T08:40:06.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.126+0000 7f53ecff9640 1 -- 172.21.15.43:0/1230291518 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5400011a50 con 0x5609f3e36580 2024-04-21T08:40:06.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.126+0000 7f540e40c740 10 client.16389 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:06.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.126+0000 7f540e40c740 1 -- 172.21.15.43:0/1230291518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5609f3a6d0f0 con 0x5609f3e36580 2024-04-21T08:40:06.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.126+0000 7f53ecff9640 1 -- 172.21.15.43:0/1230291518 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53fc009dc0 con 0x5609f3e36580 2024-04-21T08:40:06.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.126+0000 7f53ecff9640 1 -- 172.21.15.43:0/1230291518 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5400010c20 con 0x5609f3e36580 2024-04-21T08:40:06.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.126+0000 7f540e40c740 10 client.16389 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:06.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.126+0000 7f540e40c740 10 client.16389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:06.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.126+0000 7f540e40c740 20 client.16389 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:06.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.126+0000 7f540e40c740 1 --2- 172.21.15.43:0/1230291518 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5609f3e4fa70 0x5609f3e51ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:06.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.126+0000 7f540e40c740 4 client.16389 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:06.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.126+0000 7f540e40c740 1 -- 172.21.15.43:0/1230291518 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609f3da8c00 con 0x5609f3e4fa70 2024-04-21T08:40:06.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.127+0000 7f5408ce1640 1 --2- 172.21.15.43:0/1230291518 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5609f3e4fa70 0x5609f3e51ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:06.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.127+0000 7f5408ce1640 1 --2- 172.21.15.43:0/1230291518 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5609f3e4fa70 0x5609f3e51ec0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9910c33952c143ac server_cookie=f249844727f45c2f in_seq=0 out_seq=0 2024-04-21T08:40:06.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.127+0000 7f53ecff9640 10 client.16389 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:06.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.128+0000 7f53ecff9640 1 -- 172.21.15.43:0/1230291518 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5609f3da8c00 con 0x5609f3e4fa70 2024-04-21T08:40:06.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.128+0000 7f53ecff9640 10 client.16389 handle_command_reply: tid=0 2024-04-21T08:40:06.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.128+0000 7f540e40c740 1 client.16389 shutdown 2024-04-21T08:40:06.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.128+0000 7f540e40c740 1 -- 172.21.15.43:0/1230291518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36580 msgr2=0x5609f3e3cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:06.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.128+0000 7f540e40c740 1 --2- 172.21.15.43:0/1230291518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36580 0x5609f3e3cb20 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f540000de00 tx=0x7f540000f730 comp rx=0 tx=0).stop 2024-04-21T08:40:06.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.128+0000 7f540e40c740 1 -- 172.21.15.43:0/1230291518 shutdown_connections 2024-04-21T08:40:06.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.128+0000 7f540e40c740 1 --2- 172.21.15.43:0/1230291518 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5609f3e4fa70 0x5609f3e51ec0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.128+0000 7f540e40c740 1 --2- 172.21.15.43:0/1230291518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609f3e3fa80 0x5609f3e3d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.128+0000 7f540e40c740 1 --2- 172.21.15.43:0/1230291518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609f3e36ea0 0x5609f3e3d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.128+0000 7f53ecff9640 0 client.16389 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:06.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.128+0000 7f540e40c740 1 --2- 172.21.15.43:0/1230291518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36580 0x5609f3e3cb20 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.128+0000 7f540e40c740 1 -- 172.21.15.43:0/1230291518 >> 172.21.15.43:0/1230291518 conn(0x5609f3e30cb0 msgr2=0x5609f3e32a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:06.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.128+0000 7f540e40c740 1 -- 172.21.15.43:0/1230291518 shutdown_connections 2024-04-21T08:40:06.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.128+0000 7f540e40c740 1 -- 172.21.15.43:0/1230291518 wait complete. 2024-04-21T08:40:06.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.128+0000 7f540e40c740 20 client.16389 trim_cache size 0 max 16384 2024-04-21T08:40:06.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.146+0000 7f540e40c740 1 Processor -- start 2024-04-21T08:40:06.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.146+0000 7f540e40c740 1 -- start start 2024-04-21T08:40:06.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.146+0000 7f540e40c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609f3e36580 0x5609f3e3f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:06.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.146+0000 7f540e40c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36ea0 0x5609f3e4b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:06.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.146+0000 7f540e40c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609f3e3fa80 0x5609f3e3bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:06.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.146+0000 7f540e40c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f54041af4d0 con 0x5609f3e36ea0 2024-04-21T08:40:06.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.146+0000 7f540e40c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54041b1010 con 0x5609f3e3fa80 2024-04-21T08:40:06.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.146+0000 7f540e40c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54041afba0 con 0x5609f3e36580 2024-04-21T08:40:06.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.146+0000 7f5408ce1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36ea0 0x5609f3e4b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:06.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.146+0000 7f5408ce1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36ea0 0x5609f3e4b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49258/0 (socket says 172.21.15.43:49258) 2024-04-21T08:40:06.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.146+0000 7f5408ce1640 1 -- 172.21.15.43:0/1257116656 learned_addr learned my addr 172.21.15.43:0/1257116656 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:06.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.146+0000 7f5409ce3640 1 --2- 172.21.15.43:0/1257116656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609f3e3fa80 0x5609f3e3bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:06.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.146+0000 7f54094e2640 1 --2- 172.21.15.43:0/1257116656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609f3e36580 0x5609f3e3f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:06.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.146+0000 7f5408ce1640 1 -- 172.21.15.43:0/1257116656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609f3e36580 msgr2=0x5609f3e3f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:06.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.146+0000 7f5408ce1640 1 --2- 172.21.15.43:0/1257116656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609f3e36580 0x5609f3e3f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.146+0000 7f5408ce1640 1 -- 172.21.15.43:0/1257116656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609f3e3fa80 msgr2=0x5609f3e3bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:06.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.146+0000 7f5408ce1640 1 --2- 172.21.15.43:0/1257116656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609f3e3fa80 0x5609f3e3bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.146+0000 7f5408ce1640 1 -- 172.21.15.43:0/1257116656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609f3e36cd0 con 0x5609f3e36ea0 2024-04-21T08:40:06.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.146+0000 7f5409ce3640 1 --2- 172.21.15.43:0/1257116656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609f3e3fa80 0x5609f3e3bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:06.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.147+0000 7f5408ce1640 1 --2- 172.21.15.43:0/1257116656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36ea0 0x5609f3e4b670 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f53f4006010 tx=0x7f53f40f2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3d6576e0ad92be server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:06.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.147+0000 7f53eeffd640 1 -- 172.21.15.43:0/1257116656 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f40bc9c0 con 0x5609f3e36ea0 2024-04-21T08:40:06.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.147+0000 7f53eeffd640 1 -- 172.21.15.43:0/1257116656 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53f40bcc20 con 0x5609f3e36ea0 2024-04-21T08:40:06.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.147+0000 7f53eeffd640 1 -- 172.21.15.43:0/1257116656 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f40c7360 con 0x5609f3e36ea0 2024-04-21T08:40:06.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.147+0000 7f540e40c740 1 -- 172.21.15.43:0/1257116656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36ea0 msgr2=0x5609f3e4b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:06.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.147+0000 7f540e40c740 1 --2- 172.21.15.43:0/1257116656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36ea0 0x5609f3e4b670 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f53f4006010 tx=0x7f53f40f2040 comp rx=0 tx=0).stop 2024-04-21T08:40:06.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.147+0000 7f540e40c740 1 -- 172.21.15.43:0/1257116656 shutdown_connections 2024-04-21T08:40:06.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.147+0000 7f540e40c740 1 --2- 172.21.15.43:0/1257116656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609f3e3fa80 0x5609f3e3bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.147+0000 7f540e40c740 1 --2- 172.21.15.43:0/1257116656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36ea0 0x5609f3e4b670 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.147+0000 7f540e40c740 1 --2- 172.21.15.43:0/1257116656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609f3e36580 0x5609f3e3f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.147+0000 7f540e40c740 1 -- 172.21.15.43:0/1257116656 >> 172.21.15.43:0/1257116656 conn(0x5609f3e30cb0 msgr2=0x5609f3e3cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:06.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.147+0000 7f540e40c740 1 -- 172.21.15.43:0/1257116656 shutdown_connections 2024-04-21T08:40:06.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.147+0000 7f540e40c740 1 -- 172.21.15.43:0/1257116656 wait complete. 2024-04-21T08:40:06.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.148+0000 7f540e40c740 1 -- start start 2024-04-21T08:40:06.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.148+0000 7f540e40c740 1 Processor -- start 2024-04-21T08:40:06.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.148+0000 7f540e40c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36580 0x5609f3e480c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:06.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.148+0000 7f540e40c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609f3e36ea0 0x5609f3e43430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:06.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.148+0000 7f540e40c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609f3e3fa80 0x5609f3e43b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:06.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.148+0000 7f540e40c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f54041b0940 con 0x5609f3e36580 2024-04-21T08:40:06.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.148+0000 7f540e40c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54041b0270 con 0x5609f3e3fa80 2024-04-21T08:40:06.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.148+0000 7f540e40c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54041ad4b0 con 0x5609f3e36ea0 2024-04-21T08:40:06.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.148+0000 7f540e40c740 10 client.0 _ll_register_callbacks cb 0x7f540000d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:06.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.148+0000 7f54094e2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36580 0x5609f3e480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:06.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.148+0000 7f54094e2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36580 0x5609f3e480c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49262/0 (socket says 172.21.15.43:49262) 2024-04-21T08:40:06.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.148+0000 7f54094e2640 1 -- 172.21.15.43:0/2138829672 learned_addr learned my addr 172.21.15.43:0/2138829672 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:06.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.148+0000 7f5409ce3640 1 --2- 172.21.15.43:0/2138829672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609f3e3fa80 0x5609f3e43b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:06.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.148+0000 7f5408ce1640 1 --2- 172.21.15.43:0/2138829672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609f3e36ea0 0x5609f3e43430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:06.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.149+0000 7f54094e2640 1 -- 172.21.15.43:0/2138829672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609f3e36ea0 msgr2=0x5609f3e43430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:06.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.149+0000 7f54094e2640 1 --2- 172.21.15.43:0/2138829672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609f3e36ea0 0x5609f3e43430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.149+0000 7f54094e2640 1 -- 172.21.15.43:0/2138829672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609f3e3fa80 msgr2=0x5609f3e43b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:06.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.149+0000 7f54094e2640 1 --2- 172.21.15.43:0/2138829672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609f3e3fa80 0x5609f3e43b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.149+0000 7f54094e2640 1 -- 172.21.15.43:0/2138829672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609f3eb5000 con 0x5609f3e36580 2024-04-21T08:40:06.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.149+0000 7f54094e2640 1 --2- 172.21.15.43:0/2138829672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36580 0x5609f3e480c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5400000c00 tx=0x7f540000d720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4b9844dac70d5a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:06.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.149+0000 7f53ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:06.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.149+0000 7f53ecff9640 1 -- 172.21.15.43:0/2138829672 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540002c070 con 0x5609f3e36580 2024-04-21T08:40:06.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.149+0000 7f53ecff9640 1 -- 172.21.15.43:0/2138829672 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f5400011850 con 0x5609f3e36580 2024-04-21T08:40:06.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.149+0000 7f540e40c740 1 -- 172.21.15.43:0/2138829672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53f40bcc20 con 0x5609f3e36580 2024-04-21T08:40:06.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.149+0000 7f53ecff9640 1 -- 172.21.15.43:0/2138829672 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f540003a050 con 0x5609f3e36580 2024-04-21T08:40:06.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.149+0000 7f540e40c740 10 client.16395 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:06.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.149+0000 7f540e40c740 1 -- 172.21.15.43:0/2138829672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5609f3eb6210 con 0x5609f3e36580 2024-04-21T08:40:06.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.149+0000 7f53ecff9640 1 -- 172.21.15.43:0/2138829672 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53f40bcc20 con 0x5609f3e36580 2024-04-21T08:40:06.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.149+0000 7f53ecff9640 1 -- 172.21.15.43:0/2138829672 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f54000035f0 con 0x5609f3e36580 2024-04-21T08:40:06.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.150+0000 7f540e40c740 10 client.16395 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:06.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.150+0000 7f540e40c740 10 client.16395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:06.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.150+0000 7f540e40c740 20 client.16395 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:06.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.150+0000 7f540e40c740 1 --2- 172.21.15.43:0/2138829672 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5609f3eb6960 0x5609f3eb8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:06.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.150+0000 7f540e40c740 4 client.16395 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:40:06.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.150+0000 7f540e40c740 1 -- 172.21.15.43:0/2138829672 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5609f3da2a70 con 0x5609f3eb6960 2024-04-21T08:40:06.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.150+0000 7f5408ce1640 1 --2- 172.21.15.43:0/2138829672 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5609f3eb6960 0x5609f3eb8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:06.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.150+0000 7f5408ce1640 1 --2- 172.21.15.43:0/2138829672 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5609f3eb6960 0x5609f3eb8d80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=913f3d6258b38681 server_cookie=3ca18820af5afe08 in_seq=0 out_seq=0 2024-04-21T08:40:06.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.150+0000 7f53ecff9640 10 client.16395 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:06.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f53ecff9640 1 -- 172.21.15.43:0/2138829672 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5609f3da2a70 con 0x5609f3eb6960 2024-04-21T08:40:06.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f53ecff9640 10 client.16395 handle_command_reply: tid=0 2024-04-21T08:40:06.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f540e40c740 1 client.16395 shutdown 2024-04-21T08:40:06.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f540e40c740 1 -- 172.21.15.43:0/2138829672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36580 msgr2=0x5609f3e480c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:06.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f540e40c740 1 --2- 172.21.15.43:0/2138829672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36580 0x5609f3e480c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5400000c00 tx=0x7f540000d720 comp rx=0 tx=0).stop 2024-04-21T08:40:06.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f540e40c740 1 -- 172.21.15.43:0/2138829672 shutdown_connections 2024-04-21T08:40:06.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f540e40c740 1 --2- 172.21.15.43:0/2138829672 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5609f3eb6960 0x5609f3eb8d80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f540e40c740 1 --2- 172.21.15.43:0/2138829672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609f3e3fa80 0x5609f3e43b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f540e40c740 1 --2- 172.21.15.43:0/2138829672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609f3e36ea0 0x5609f3e43430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f540e40c740 1 --2- 172.21.15.43:0/2138829672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609f3e36580 0x5609f3e480c0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f53ecff9640 0 client.16395 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:06.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f540e40c740 1 -- 172.21.15.43:0/2138829672 >> 172.21.15.43:0/2138829672 conn(0x5609f3e30cb0 msgr2=0x5609f3e32820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:06.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f540e40c740 1 -- 172.21.15.43:0/2138829672 shutdown_connections 2024-04-21T08:40:06.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f540e40c740 1 -- 172.21.15.43:0/2138829672 wait complete. 2024-04-21T08:40:06.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f540e40c740 20 client.16395 trim_cache size 0 max 16384 2024-04-21T08:40:06.157 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:40:06.157 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:40:06.158 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-21T08:40:06.158 INFO:teuthology.orchestra.run.smithi043.stdout: "55e6a3ed-235c-4dee-aba8-7823654c4d0a": { 2024-04-21T08:40:06.158 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-21T08:40:06.158 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "55e6a3ed-235c-4dee-aba8-7823654c4d0a", 2024-04-21T08:40:06.158 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-21T08:40:06.158 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:40:06.158 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:40:06.158 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:40:06.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f540b746640 1 -- 172.21.15.43:0/638672782 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f53f00a3ec0 msgr2=0x7f53f00a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:06.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f540b746640 1 --2- 172.21.15.43:0/638672782 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f53f00a3ec0 0x7f53f00a6380 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f53fc010c40 tx=0x7f53fc01d000 comp rx=0 tx=0).stop 2024-04-21T08:40:06.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f540b746640 1 -- 172.21.15.43:0/638672782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5404103ae0 msgr2=0x7f5404198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:06.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f540b746640 1 --2- 172.21.15.43:0/638672782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5404103ae0 0x7f5404198c10 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f53f4005940 tx=0x7f53f40045e0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f540b746640 1 -- 172.21.15.43:0/638672782 shutdown_connections 2024-04-21T08:40:06.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f540b746640 1 --2- 172.21.15.43:0/638672782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f540410c500 0x7f540419cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f540b746640 1 --2- 172.21.15.43:0/638672782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5404103ae0 0x7f5404198c10 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f540b746640 1 --2- 172.21.15.43:0/638672782 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f53f00a3ec0 0x7f53f00a6380 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f540b746640 1 --2- 172.21.15.43:0/638672782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54041031c0 0x7f54041986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:06.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.151+0000 7f540b746640 1 -- 172.21.15.43:0/638672782 >> 172.21.15.43:0/638672782 conn(0x7f5404078000 msgr2=0x7f54040fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:06.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.152+0000 7f540b746640 1 -- 172.21.15.43:0/638672782 shutdown_connections 2024-04-21T08:40:06.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:06.152+0000 7f540b746640 1 -- 172.21.15.43:0/638672782 wait complete. 2024-04-21T08:40:06.160 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:05 smithi043 ceph-mon[27978]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 27 op/s 2024-04-21T08:40:06.161 INFO:tasks.cephfs.filesystem:scrub status for tag:55e6a3ed-235c-4dee-aba8-7823654c4d0a - {'path': '/', 'tag': '55e6a3ed-235c-4dee-aba8-7823654c4d0a', 'options': 'recursive,force'} 2024-04-21T08:40:06.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:05 smithi112 ceph-mon[31085]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 27 op/s 2024-04-21T08:40:06.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:05 smithi175 ceph-mon[31920]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 27 op/s 2024-04-21T08:40:07.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:07 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:40:07.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:07 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:40:07.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:07 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:40:07.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:07 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:40:07.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:07 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:40:07.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:07 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:40:07.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:07 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:40:07.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:07 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:40:07.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:07 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:40:07.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:07 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:40:07.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:07 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:40:07.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:07 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:40:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.784+0000 7f9d7038f640 1 -- 172.21.15.43:0/3466186599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d6806a780 msgr2=0x7f9d6806abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.784+0000 7f9d7038f640 1 --2- 172.21.15.43:0/3466186599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d6806a780 0x7f9d6806abe0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009930 tx=0x7f9d5802bc30 comp rx=0 tx=0).stop 2024-04-21T08:40:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.784+0000 7f9d7038f640 1 -- 172.21.15.43:0/3466186599 shutdown_connections 2024-04-21T08:40:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.784+0000 7f9d7038f640 1 --2- 172.21.15.43:0/3466186599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d6806b120 0x7f9d6810edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.784+0000 7f9d7038f640 1 --2- 172.21.15.43:0/3466186599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d6806a780 0x7f9d6806abe0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.784+0000 7f9d7038f640 1 --2- 172.21.15.43:0/3466186599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d68107fe0 0x7f9d681083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.784+0000 7f9d7038f640 1 -- 172.21.15.43:0/3466186599 >> 172.21.15.43:0/3466186599 conn(0x7f9d68076c50 msgr2=0x7f9d68077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.784+0000 7f9d7038f640 1 -- 172.21.15.43:0/3466186599 shutdown_connections 2024-04-21T08:40:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.784+0000 7f9d7038f640 1 -- 172.21.15.43:0/3466186599 wait complete. 2024-04-21T08:40:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.784+0000 7f9d7038f640 1 Processor -- start 2024-04-21T08:40:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.784+0000 7f9d7038f640 1 -- start start 2024-04-21T08:40:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.785+0000 7f9d7038f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d6806a780 0x7f9d68101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.785+0000 7f9d7038f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d6806b120 0x7f9d681017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.785+0000 7f9d7038f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d68107fe0 0x7f9d68101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.785+0000 7f9d7038f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9d681121f0 con 0x7f9d6806a780 2024-04-21T08:40:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.785+0000 7f9d7038f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9d68112070 con 0x7f9d68107fe0 2024-04-21T08:40:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.785+0000 7f9d7038f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9d68112370 con 0x7f9d6806b120 2024-04-21T08:40:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.785+0000 7f9d6e12b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d6806a780 0x7f9d68101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.785+0000 7f9d6e12b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d6806a780 0x7f9d68101270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56598/0 (socket says 172.21.15.43:56598) 2024-04-21T08:40:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.785+0000 7f9d6e12b640 1 -- 172.21.15.43:0/1939349863 learned_addr learned my addr 172.21.15.43:0/1939349863 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.785+0000 7f9d6d92a640 1 --2- 172.21.15.43:0/1939349863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d6806b120 0x7f9d681017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.785+0000 7f9d6e92c640 1 --2- 172.21.15.43:0/1939349863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d68107fe0 0x7f9d68101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.785+0000 7f9d6e12b640 1 -- 172.21.15.43:0/1939349863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d6806b120 msgr2=0x7f9d681017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.785+0000 7f9d6e12b640 1 --2- 172.21.15.43:0/1939349863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d6806b120 0x7f9d681017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.785+0000 7f9d6e12b640 1 -- 172.21.15.43:0/1939349863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d68107fe0 msgr2=0x7f9d68101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.785+0000 7f9d6e12b640 1 --2- 172.21.15.43:0/1939349863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d68107fe0 0x7f9d68101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.785+0000 7f9d6e12b640 1 -- 172.21.15.43:0/1939349863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d681a6b90 con 0x7f9d6806a780 2024-04-21T08:40:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.785+0000 7f9d6e12b640 1 --2- 172.21.15.43:0/1939349863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d6806a780 0x7f9d68101270 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400b4a0 tx=0x7f9d6400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.785+0000 7f9d5f7fe640 1 -- 172.21.15.43:0/1939349863 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64014070 con 0x7f9d6806a780 2024-04-21T08:40:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.786+0000 7f9d5f7fe640 1 -- 172.21.15.43:0/1939349863 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d64010ac0 con 0x7f9d6806a780 2024-04-21T08:40:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.786+0000 7f9d5f7fe640 1 -- 172.21.15.43:0/1939349863 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64017c30 con 0x7f9d6806a780 2024-04-21T08:40:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.786+0000 7f9d7038f640 1 -- 172.21.15.43:0/1939349863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d681a6e80 con 0x7f9d6806a780 2024-04-21T08:40:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.786+0000 7f9d7038f640 1 -- 172.21.15.43:0/1939349863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d681a72f0 con 0x7f9d6806a780 2024-04-21T08:40:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.789+0000 7f9d5f7fe640 1 -- 172.21.15.43:0/1939349863 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f9d64017070 con 0x7f9d6806a780 2024-04-21T08:40:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.789+0000 7f9d72f80740 1 Processor -- start 2024-04-21T08:40:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.789+0000 7f9d72f80740 1 -- start start 2024-04-21T08:40:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.789+0000 7f9d72f80740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564cb65eb8d0 0x564cb65ea620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.789+0000 7f9d72f80740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eab60 0x564cb65eafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.789+0000 7f9d72f80740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564cb65f40e0 0x564cb65f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.789+0000 7f9d72f80740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564cb654f8d0 con 0x564cb65eab60 2024-04-21T08:40:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.789+0000 7f9d72f80740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564cb64ece80 con 0x564cb65eb8d0 2024-04-21T08:40:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.789+0000 7f9d72f80740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564cb654b890 con 0x564cb65f40e0 2024-04-21T08:40:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.789+0000 7f9d6e12b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eab60 0x564cb65eafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.789+0000 7f9d6e12b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eab60 0x564cb65eafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56610/0 (socket says 172.21.15.43:56610) 2024-04-21T08:40:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.789+0000 7f9d6e12b640 1 -- 172.21.15.43:0/3021815234 learned_addr learned my addr 172.21.15.43:0/3021815234 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.789+0000 7f9d6e92c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564cb65eb8d0 0x564cb65ea620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.790+0000 7f9d6d92a640 1 --2- 172.21.15.43:0/3021815234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564cb65f40e0 0x564cb65f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.790+0000 7f9d5f7fe640 1 --2- 172.21.15.43:0/1939349863 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9d540a3f10 0x7f9d540a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.790+0000 7f9d6e92c640 1 -- 172.21.15.43:0/3021815234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564cb65f40e0 msgr2=0x564cb65f64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.790+0000 7f9d6e92c640 1 --2- 172.21.15.43:0/3021815234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564cb65f40e0 0x564cb65f64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.790+0000 7f9d6e92c640 1 -- 172.21.15.43:0/3021815234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eab60 msgr2=0x564cb65eafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.790+0000 7f9d6e92c640 1 --2- 172.21.15.43:0/3021815234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eab60 0x564cb65eafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.790+0000 7f9d6e92c640 1 -- 172.21.15.43:0/3021815234 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cb650e320 con 0x564cb65eb8d0 2024-04-21T08:40:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.790+0000 7f9d6e12b640 1 --2- 172.21.15.43:0/3021815234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eab60 0x564cb65eafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:40:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.790+0000 7f9d6e92c640 1 --2- 172.21.15.43:0/1939349863 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9d540a3f10 0x7f9d540a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.790+0000 7f9d5f7fe640 1 -- 172.21.15.43:0/1939349863 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f9d640cffd0 con 0x7f9d6806a780 2024-04-21T08:40:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.790+0000 7f9d6e92c640 1 --2- 172.21.15.43:0/3021815234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564cb65eb8d0 0x564cb65ea620 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9d60010440 tx=0x7f9d60010900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=527fe7834c7291bc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.790+0000 7f9d6e92c640 1 --2- 172.21.15.43:0/1939349863 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9d540a3f10 0x7f9d540a63d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9d60021000 tx=0x7f9d6001e000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.790+0000 7f9d5cbfe640 1 -- 172.21.15.43:0/3021815234 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d600134c0 con 0x564cb65eb8d0 2024-04-21T08:40:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.790+0000 7f9d5cbfe640 1 -- 172.21.15.43:0/3021815234 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d60003e20 con 0x564cb65eb8d0 2024-04-21T08:40:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.790+0000 7f9d5cbfe640 1 -- 172.21.15.43:0/3021815234 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6001d400 con 0x564cb65eb8d0 2024-04-21T08:40:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.791+0000 7f9d72f80740 1 -- 172.21.15.43:0/3021815234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564cb65eb8d0 msgr2=0x564cb65ea620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.791+0000 7f9d72f80740 1 --2- 172.21.15.43:0/3021815234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564cb65eb8d0 0x564cb65ea620 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9d60010440 tx=0x7f9d60010900 comp rx=0 tx=0).stop 2024-04-21T08:40:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.791+0000 7f9d72f80740 1 -- 172.21.15.43:0/3021815234 shutdown_connections 2024-04-21T08:40:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.791+0000 7f9d72f80740 1 --2- 172.21.15.43:0/3021815234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564cb65f40e0 0x564cb65f64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.791+0000 7f9d72f80740 1 --2- 172.21.15.43:0/3021815234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eab60 0x564cb65eafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.791+0000 7f9d72f80740 1 --2- 172.21.15.43:0/3021815234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564cb65eb8d0 0x564cb65ea620 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.791+0000 7f9d72f80740 1 -- 172.21.15.43:0/3021815234 >> 172.21.15.43:0/3021815234 conn(0x564cb65b1c90 msgr2=0x564cb65e5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.791+0000 7f9d72f80740 1 -- 172.21.15.43:0/3021815234 shutdown_connections 2024-04-21T08:40:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.791+0000 7f9d72f80740 1 -- 172.21.15.43:0/3021815234 wait complete. 2024-04-21T08:40:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.791+0000 7f9d72f80740 1 -- start start 2024-04-21T08:40:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.792+0000 7f9d72f80740 1 Processor -- start 2024-04-21T08:40:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.792+0000 7f9d72f80740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564cb65eab60 0x564cb65f3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.792+0000 7f9d72f80740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eb8d0 0x564cb65f0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.792+0000 7f9d72f80740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564cb65f40e0 0x564cb65f1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.792+0000 7f9d72f80740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564cb65f6e10 con 0x564cb65eb8d0 2024-04-21T08:40:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.792+0000 7f9d72f80740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564cb64d7eb0 con 0x564cb65eab60 2024-04-21T08:40:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.792+0000 7f9d72f80740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564cb65f6f90 con 0x564cb65f40e0 2024-04-21T08:40:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.792+0000 7f9d72f80740 10 client.0 _ll_register_callbacks cb 0x564cb6504b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.792+0000 7f9d6d92a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eb8d0 0x564cb65f0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.792+0000 7f9d6d92a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eb8d0 0x564cb65f0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56622/0 (socket says 172.21.15.43:56622) 2024-04-21T08:40:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.792+0000 7f9d6d92a640 1 -- 172.21.15.43:0/978365950 learned_addr learned my addr 172.21.15.43:0/978365950 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.792+0000 7f9d6e12b640 1 --2- 172.21.15.43:0/978365950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564cb65eab60 0x564cb65f3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.792+0000 7f9d6e92c640 1 --2- 172.21.15.43:0/978365950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564cb65f40e0 0x564cb65f1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.792+0000 7f9d6d92a640 1 -- 172.21.15.43:0/978365950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564cb65f40e0 msgr2=0x564cb65f1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.792+0000 7f9d6d92a640 1 --2- 172.21.15.43:0/978365950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564cb65f40e0 0x564cb65f1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.792+0000 7f9d6d92a640 1 -- 172.21.15.43:0/978365950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564cb65eab60 msgr2=0x564cb65f3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.792+0000 7f9d6d92a640 1 --2- 172.21.15.43:0/978365950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564cb65eab60 0x564cb65f3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.792+0000 7f9d6d92a640 1 -- 172.21.15.43:0/978365950 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cb6262210 con 0x564cb65eb8d0 2024-04-21T08:40:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.792+0000 7f9d6e92c640 1 --2- 172.21.15.43:0/978365950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564cb65f40e0 0x564cb65f1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.792+0000 7f9d6d92a640 1 --2- 172.21.15.43:0/978365950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eb8d0 0x564cb65f0e60 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009930 tx=0x7f9d5802c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fde442cf3231e59 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.792+0000 7f9d51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.793+0000 7f9d51ffb640 1 -- 172.21.15.43:0/978365950 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58005030 con 0x564cb65eb8d0 2024-04-21T08:40:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.793+0000 7f9d51ffb640 1 -- 172.21.15.43:0/978365950 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f9d580051d0 con 0x564cb65eb8d0 2024-04-21T08:40:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.793+0000 7f9d72f80740 1 -- 172.21.15.43:0/978365950 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d60003e20 con 0x564cb65eb8d0 2024-04-21T08:40:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.793+0000 7f9d51ffb640 1 -- 172.21.15.43:0/978365950 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9d580053d0 con 0x564cb65eb8d0 2024-04-21T08:40:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.793+0000 7f9d72f80740 10 client.16407 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.793+0000 7f9d72f80740 1 -- 172.21.15.43:0/978365950 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564cb632e620 con 0x564cb65eb8d0 2024-04-21T08:40:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.793+0000 7f9d51ffb640 1 -- 172.21.15.43:0/978365950 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d60003e20 con 0x564cb65eb8d0 2024-04-21T08:40:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.793+0000 7f9d51ffb640 1 -- 172.21.15.43:0/978365950 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9d58003710 con 0x564cb65eb8d0 2024-04-21T08:40:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.793+0000 7f9d72f80740 10 client.16407 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.793+0000 7f9d72f80740 10 client.16407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.793+0000 7f9d72f80740 20 client.16407 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.793+0000 7f9d72f80740 1 --2- 172.21.15.43:0/978365950 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564cb66041d0 0x564cb6606600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.793+0000 7f9d72f80740 4 client.16407 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.793+0000 7f9d72f80740 1 -- 172.21.15.43:0/978365950 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cb6563880 con 0x564cb66041d0 2024-04-21T08:40:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.793+0000 7f9d6e12b640 1 --2- 172.21.15.43:0/978365950 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564cb66041d0 0x564cb6606600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.794+0000 7f9d6e12b640 1 --2- 172.21.15.43:0/978365950 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564cb66041d0 0x564cb6606600 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e1c52397244fab9 server_cookie=aa8163b6ff7033fc in_seq=0 out_seq=0 2024-04-21T08:40:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.794+0000 7f9d51ffb640 10 client.16407 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.795+0000 7f9d51ffb640 1 -- 172.21.15.43:0/978365950 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564cb6563880 con 0x564cb66041d0 2024-04-21T08:40:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.795+0000 7f9d51ffb640 10 client.16407 handle_command_reply: tid=0 2024-04-21T08:40:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.795+0000 7f9d72f80740 1 client.16407 shutdown 2024-04-21T08:40:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.795+0000 7f9d72f80740 1 -- 172.21.15.43:0/978365950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eb8d0 msgr2=0x564cb65f0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.795+0000 7f9d72f80740 1 --2- 172.21.15.43:0/978365950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eb8d0 0x564cb65f0e60 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009930 tx=0x7f9d5802c9f0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.795+0000 7f9d72f80740 1 -- 172.21.15.43:0/978365950 shutdown_connections 2024-04-21T08:40:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.795+0000 7f9d72f80740 1 --2- 172.21.15.43:0/978365950 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564cb66041d0 0x564cb6606600 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.795+0000 7f9d72f80740 1 --2- 172.21.15.43:0/978365950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564cb65f40e0 0x564cb65f1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.795+0000 7f9d72f80740 1 --2- 172.21.15.43:0/978365950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eb8d0 0x564cb65f0e60 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.795+0000 7f9d51ffb640 0 client.16407 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.795+0000 7f9d72f80740 1 --2- 172.21.15.43:0/978365950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564cb65eab60 0x564cb65f3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.795+0000 7f9d72f80740 1 -- 172.21.15.43:0/978365950 >> 172.21.15.43:0/978365950 conn(0x564cb65b1c90 msgr2=0x564cb65e75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.795+0000 7f9d72f80740 1 -- 172.21.15.43:0/978365950 shutdown_connections 2024-04-21T08:40:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.795+0000 7f9d72f80740 1 -- 172.21.15.43:0/978365950 wait complete. 2024-04-21T08:40:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.795+0000 7f9d72f80740 20 client.16407 trim_cache size 0 max 16384 2024-04-21T08:40:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.810+0000 7f9d72f80740 1 Processor -- start 2024-04-21T08:40:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.810+0000 7f9d72f80740 1 -- start start 2024-04-21T08:40:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.810+0000 7f9d72f80740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564cb65b1c90 0x564cb65ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.810+0000 7f9d72f80740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eab60 0x564cb65f3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.810+0000 7f9d72f80740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564cb65eb8d0 0x564cb65f3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.810+0000 7f9d72f80740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9d681b16f0 con 0x564cb65eab60 2024-04-21T08:40:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.810+0000 7f9d72f80740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9d681b3230 con 0x564cb65b1c90 2024-04-21T08:40:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.810+0000 7f9d72f80740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9d681b1dc0 con 0x564cb65eb8d0 2024-04-21T08:40:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.810+0000 7f9d6d92a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eab60 0x564cb65f3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.810+0000 7f9d6d92a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eab60 0x564cb65f3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56626/0 (socket says 172.21.15.43:56626) 2024-04-21T08:40:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.810+0000 7f9d6d92a640 1 -- 172.21.15.43:0/2402592780 learned_addr learned my addr 172.21.15.43:0/2402592780 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.810+0000 7f9d6e12b640 1 --2- 172.21.15.43:0/2402592780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564cb65b1c90 0x564cb65ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.810+0000 7f9d6d92a640 1 -- 172.21.15.43:0/2402592780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564cb65eb8d0 msgr2=0x564cb65f3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.810+0000 7f9d6e92c640 1 --2- 172.21.15.43:0/2402592780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564cb65eb8d0 0x564cb65f3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.810+0000 7f9d6d92a640 1 --2- 172.21.15.43:0/2402592780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564cb65eb8d0 0x564cb65f3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.810+0000 7f9d6d92a640 1 -- 172.21.15.43:0/2402592780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564cb65b1c90 msgr2=0x564cb65ffd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.810+0000 7f9d6d92a640 1 --2- 172.21.15.43:0/2402592780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564cb65b1c90 0x564cb65ffd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.810+0000 7f9d6d92a640 1 -- 172.21.15.43:0/2402592780 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cb65e0f30 con 0x564cb65eab60 2024-04-21T08:40:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.810+0000 7f9d6e92c640 1 --2- 172.21.15.43:0/2402592780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564cb65eb8d0 0x564cb65f3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.811+0000 7f9d6d92a640 1 --2- 172.21.15.43:0/2402592780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eab60 0x564cb65f3590 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9d58003d20 tx=0x7f9d58009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5ed6988f3b1681 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.811+0000 7f9d53fff640 1 -- 172.21.15.43:0/2402592780 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58000e60 con 0x564cb65eab60 2024-04-21T08:40:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.811+0000 7f9d53fff640 1 -- 172.21.15.43:0/2402592780 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d58005850 con 0x564cb65eab60 2024-04-21T08:40:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.811+0000 7f9d53fff640 1 -- 172.21.15.43:0/2402592780 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58034a90 con 0x564cb65eab60 2024-04-21T08:40:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.811+0000 7f9d72f80740 1 -- 172.21.15.43:0/2402592780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eab60 msgr2=0x564cb65f3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.811+0000 7f9d72f80740 1 --2- 172.21.15.43:0/2402592780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eab60 0x564cb65f3590 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9d58003d20 tx=0x7f9d58009610 comp rx=0 tx=0).stop 2024-04-21T08:40:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.811+0000 7f9d72f80740 1 -- 172.21.15.43:0/2402592780 shutdown_connections 2024-04-21T08:40:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.811+0000 7f9d72f80740 1 --2- 172.21.15.43:0/2402592780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564cb65eb8d0 0x564cb65f3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.811+0000 7f9d72f80740 1 --2- 172.21.15.43:0/2402592780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eab60 0x564cb65f3590 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.811+0000 7f9d72f80740 1 --2- 172.21.15.43:0/2402592780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564cb65b1c90 0x564cb65ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.811+0000 7f9d72f80740 1 -- 172.21.15.43:0/2402592780 >> 172.21.15.43:0/2402592780 conn(0x564cb65ea6c0 msgr2=0x564cb65ef730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.811+0000 7f9d72f80740 1 -- 172.21.15.43:0/2402592780 shutdown_connections 2024-04-21T08:40:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.811+0000 7f9d72f80740 1 -- 172.21.15.43:0/2402592780 wait complete. 2024-04-21T08:40:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.812+0000 7f9d72f80740 1 -- start start 2024-04-21T08:40:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.812+0000 7f9d72f80740 1 Processor -- start 2024-04-21T08:40:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.812+0000 7f9d72f80740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564cb65b1c90 0x564cb65f8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.812+0000 7f9d72f80740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eab60 0x564cb65f8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.812+0000 7f9d72f80740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564cb65eb8d0 0x564cb6669780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.812+0000 7f9d72f80740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9d681b2b60 con 0x564cb65eab60 2024-04-21T08:40:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.812+0000 7f9d72f80740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9d681b2490 con 0x564cb65b1c90 2024-04-21T08:40:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.812+0000 7f9d72f80740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9d681af6d0 con 0x564cb65eb8d0 2024-04-21T08:40:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.812+0000 7f9d72f80740 10 client.0 _ll_register_callbacks cb 0x7f9d58009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.812+0000 7f9d6d92a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eab60 0x564cb65f8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.812+0000 7f9d6d92a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eab60 0x564cb65f8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56640/0 (socket says 172.21.15.43:56640) 2024-04-21T08:40:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.812+0000 7f9d6d92a640 1 -- 172.21.15.43:0/4212265939 learned_addr learned my addr 172.21.15.43:0/4212265939 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.812+0000 7f9d6e92c640 1 --2- 172.21.15.43:0/4212265939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564cb65eb8d0 0x564cb6669780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.812+0000 7f9d6e12b640 1 --2- 172.21.15.43:0/4212265939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564cb65b1c90 0x564cb65f8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.813+0000 7f9d6d92a640 1 -- 172.21.15.43:0/4212265939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564cb65eb8d0 msgr2=0x564cb6669780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.813+0000 7f9d6d92a640 1 --2- 172.21.15.43:0/4212265939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564cb65eb8d0 0x564cb6669780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.813+0000 7f9d6d92a640 1 -- 172.21.15.43:0/4212265939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564cb65b1c90 msgr2=0x564cb65f8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.813+0000 7f9d6d92a640 1 --2- 172.21.15.43:0/4212265939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564cb65b1c90 0x564cb65f8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.813+0000 7f9d6d92a640 1 -- 172.21.15.43:0/4212265939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cb65f17e0 con 0x564cb65eab60 2024-04-21T08:40:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.813+0000 7f9d6e12b640 1 --2- 172.21.15.43:0/4212265939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564cb65b1c90 0x564cb65f8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.813+0000 7f9d6d92a640 1 --2- 172.21.15.43:0/4212265939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eab60 0x564cb65f8940 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9d58003bb0 tx=0x7f9d5802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1ae24b6d93c168 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.813+0000 7f9d51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.813+0000 7f9d51ffb640 1 -- 172.21.15.43:0/4212265939 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d580349d0 con 0x564cb65eab60 2024-04-21T08:40:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.813+0000 7f9d72f80740 1 -- 172.21.15.43:0/4212265939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d58005850 con 0x564cb65eab60 2024-04-21T08:40:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.813+0000 7f9d51ffb640 1 -- 172.21.15.43:0/4212265939 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f9d58034b70 con 0x564cb65eab60 2024-04-21T08:40:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.813+0000 7f9d51ffb640 1 -- 172.21.15.43:0/4212265939 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9d58034d70 con 0x564cb65eab60 2024-04-21T08:40:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.813+0000 7f9d72f80740 10 client.16413 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.813+0000 7f9d72f80740 1 -- 172.21.15.43:0/4212265939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564cb666ae00 con 0x564cb65eab60 2024-04-21T08:40:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.813+0000 7f9d51ffb640 1 -- 172.21.15.43:0/4212265939 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d58005850 con 0x564cb65eab60 2024-04-21T08:40:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.813+0000 7f9d51ffb640 1 -- 172.21.15.43:0/4212265939 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9d58040420 con 0x564cb65eab60 2024-04-21T08:40:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.813+0000 7f9d72f80740 10 client.16413 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.813+0000 7f9d72f80740 10 client.16413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.813+0000 7f9d72f80740 20 client.16413 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.813+0000 7f9d72f80740 1 --2- 172.21.15.43:0/4212265939 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564cb65f40e0 0x564cb666d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.814+0000 7f9d72f80740 4 client.16413 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:40:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.814+0000 7f9d72f80740 1 -- 172.21.15.43:0/4212265939 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564cb64ef3d0 con 0x564cb65f40e0 2024-04-21T08:40:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.814+0000 7f9d6e12b640 1 --2- 172.21.15.43:0/4212265939 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564cb65f40e0 0x564cb666d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.814+0000 7f9d6e12b640 1 --2- 172.21.15.43:0/4212265939 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564cb65f40e0 0x564cb666d320 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d570394c9fc62109 server_cookie=7f2a57c66d29bd90 in_seq=0 out_seq=0 2024-04-21T08:40:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.814+0000 7f9d51ffb640 10 client.16413 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.826+0000 7f9d51ffb640 1 -- 172.21.15.43:0/4212265939 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564cb64ef3d0 con 0x564cb65f40e0 2024-04-21T08:40:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.826+0000 7f9d51ffb640 10 client.16413 handle_command_reply: tid=0 2024-04-21T08:40:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.826+0000 7f9d72f80740 1 client.16413 shutdown 2024-04-21T08:40:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.826+0000 7f9d72f80740 1 -- 172.21.15.43:0/4212265939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eab60 msgr2=0x564cb65f8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.826+0000 7f9d72f80740 1 --2- 172.21.15.43:0/4212265939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eab60 0x564cb65f8940 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9d58003bb0 tx=0x7f9d5802bc30 comp rx=0 tx=0).stop 2024-04-21T08:40:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.826+0000 7f9d72f80740 1 -- 172.21.15.43:0/4212265939 shutdown_connections 2024-04-21T08:40:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.826+0000 7f9d72f80740 1 --2- 172.21.15.43:0/4212265939 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564cb65f40e0 0x564cb666d320 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.826+0000 7f9d72f80740 1 --2- 172.21.15.43:0/4212265939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564cb65eb8d0 0x564cb6669780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.826+0000 7f9d51ffb640 0 client.16413 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.826+0000 7f9d72f80740 1 --2- 172.21.15.43:0/4212265939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564cb65eab60 0x564cb65f8940 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.826+0000 7f9d72f80740 1 --2- 172.21.15.43:0/4212265939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564cb65b1c90 0x564cb65f8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.826+0000 7f9d72f80740 1 -- 172.21.15.43:0/4212265939 >> 172.21.15.43:0/4212265939 conn(0x564cb65ea6c0 msgr2=0x564cb65e6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.826+0000 7f9d72f80740 1 -- 172.21.15.43:0/4212265939 shutdown_connections 2024-04-21T08:40:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.826+0000 7f9d72f80740 1 -- 172.21.15.43:0/4212265939 wait complete. 2024-04-21T08:40:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.826+0000 7f9d72f80740 20 client.16413 trim_cache size 0 max 16384 2024-04-21T08:40:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:40:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:40:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:40:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:40:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.827+0000 7f9d7038f640 1 -- 172.21.15.43:0/1939349863 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9d540a3f10 msgr2=0x7f9d540a63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.827+0000 7f9d7038f640 1 --2- 172.21.15.43:0/1939349863 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9d540a3f10 0x7f9d540a63d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9d60021000 tx=0x7f9d6001e000 comp rx=0 tx=0).stop 2024-04-21T08:40:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.827+0000 7f9d7038f640 1 -- 172.21.15.43:0/1939349863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d6806a780 msgr2=0x7f9d68101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.827+0000 7f9d7038f640 1 --2- 172.21.15.43:0/1939349863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d6806a780 0x7f9d68101270 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400b4a0 tx=0x7f9d6400b960 comp rx=0 tx=0).stop 2024-04-21T08:40:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.827+0000 7f9d7038f640 1 -- 172.21.15.43:0/1939349863 shutdown_connections 2024-04-21T08:40:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.827+0000 7f9d7038f640 1 --2- 172.21.15.43:0/1939349863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d68107fe0 0x7f9d68101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.827+0000 7f9d7038f640 1 --2- 172.21.15.43:0/1939349863 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9d540a3f10 0x7f9d540a63d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.827+0000 7f9d7038f640 1 --2- 172.21.15.43:0/1939349863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d6806b120 0x7f9d681017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.827+0000 7f9d7038f640 1 --2- 172.21.15.43:0/1939349863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d6806a780 0x7f9d68101270 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.827+0000 7f9d7038f640 1 -- 172.21.15.43:0/1939349863 >> 172.21.15.43:0/1939349863 conn(0x7f9d68076c50 msgr2=0x7f9d680ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.827+0000 7f9d7038f640 1 -- 172.21.15.43:0/1939349863 shutdown_connections 2024-04-21T08:40:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:07.827+0000 7f9d7038f640 1 -- 172.21.15.43:0/1939349863 wait complete. 2024-04-21T08:40:08.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:08 smithi112 ceph-mon[31085]: pgmap v39: 129 pgs: 129 active+clean; 1.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 36 KiB/s wr, 34 op/s 2024-04-21T08:40:08.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:08 smithi043 ceph-mon[27978]: pgmap v39: 129 pgs: 129 active+clean; 1.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 36 KiB/s wr, 34 op/s 2024-04-21T08:40:08.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:08 smithi175 ceph-mon[31920]: pgmap v39: 129 pgs: 129 active+clean; 1.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 36 KiB/s wr, 34 op/s 2024-04-21T08:40:10.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:10 smithi112 ceph-mon[31085]: pgmap v40: 129 pgs: 129 active+clean; 1.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 36 KiB/s wr, 28 op/s 2024-04-21T08:40:10.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:10 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:40:10.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:10 smithi043 ceph-mon[27978]: pgmap v40: 129 pgs: 129 active+clean; 1.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 36 KiB/s wr, 28 op/s 2024-04-21T08:40:10.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:10 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:40:10.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:10 smithi175 ceph-mon[31920]: pgmap v40: 129 pgs: 129 active+clean; 1.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 36 KiB/s wr, 28 op/s 2024-04-21T08:40:10.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:10 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:40:12.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:12 smithi112 ceph-mon[31085]: pgmap v41: 129 pgs: 129 active+clean; 1.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 46 KiB/s wr, 38 op/s 2024-04-21T08:40:12.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:12 smithi043 ceph-mon[27978]: pgmap v41: 129 pgs: 129 active+clean; 1.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 46 KiB/s wr, 38 op/s 2024-04-21T08:40:12.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:12 smithi175 ceph-mon[31920]: pgmap v41: 129 pgs: 129 active+clean; 1.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 46 KiB/s wr, 38 op/s 2024-04-21T08:40:14.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:14 smithi112 ceph-mon[31085]: pgmap v42: 129 pgs: 129 active+clean; 1.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-04-21T08:40:14.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:14 smithi043 ceph-mon[27978]: pgmap v42: 129 pgs: 129 active+clean; 1.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-04-21T08:40:14.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:14 smithi175 ceph-mon[31920]: pgmap v42: 129 pgs: 129 active+clean; 1.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-04-21T08:40:16.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:15 smithi175 ceph-mon[31920]: pgmap v43: 129 pgs: 129 active+clean; 1.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-04-21T08:40:16.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:15 smithi112 ceph-mon[31085]: pgmap v43: 129 pgs: 129 active+clean; 1.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-04-21T08:40:16.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:15 smithi043 ceph-mon[27978]: pgmap v43: 129 pgs: 129 active+clean; 1.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-04-21T08:40:18.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:17 smithi175 ceph-mon[31920]: pgmap v44: 129 pgs: 129 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-04-21T08:40:18.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:17 smithi112 ceph-mon[31085]: pgmap v44: 129 pgs: 129 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-04-21T08:40:18.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:17 smithi043 ceph-mon[27978]: pgmap v44: 129 pgs: 129 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-04-21T08:40:20.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:19 smithi175 ceph-mon[31920]: pgmap v45: 129 pgs: 129 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-21T08:40:20.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:19 smithi112 ceph-mon[31085]: pgmap v45: 129 pgs: 129 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-21T08:40:20.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:19 smithi043 ceph-mon[27978]: pgmap v45: 129 pgs: 129 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-21T08:40:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.927+0000 7f70e96e4640 1 -- 172.21.15.43:0/406142649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70e41010e0 msgr2=0x7f70e41014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.927+0000 7f70e96e4640 1 --2- 172.21.15.43:0/406142649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70e41010e0 0x7f70e41014c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f70cc009900 tx=0x7f70cc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:40:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.927+0000 7f70e96e4640 1 -- 172.21.15.43:0/406142649 shutdown_connections 2024-04-21T08:40:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.927+0000 7f70e96e4640 1 --2- 172.21.15.43:0/406142649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70e41061b0 0x7f70e410dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.927+0000 7f70e96e4640 1 --2- 172.21.15.43:0/406142649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70e4101a90 0x7f70e4105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.927+0000 7f70e96e4640 1 --2- 172.21.15.43:0/406142649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70e41010e0 0x7f70e41014c0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.927+0000 7f70e96e4640 1 -- 172.21.15.43:0/406142649 >> 172.21.15.43:0/406142649 conn(0x7f70e40771d0 msgr2=0x7f70e40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.927+0000 7f70e96e4640 1 -- 172.21.15.43:0/406142649 shutdown_connections 2024-04-21T08:40:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.927+0000 7f70e96e4640 1 -- 172.21.15.43:0/406142649 wait complete. 2024-04-21T08:40:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.928+0000 7f70e96e4640 1 Processor -- start 2024-04-21T08:40:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.928+0000 7f70e96e4640 1 -- start start 2024-04-21T08:40:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.928+0000 7f70e96e4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70e41010e0 0x7f70e41986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.928+0000 7f70e96e4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70e41061b0 0x7f70e4198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.928+0000 7f70e96e4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70e419cf70 0x7f70e419d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.928+0000 7f70e96e4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f70e4111120 con 0x7f70e41061b0 2024-04-21T08:40:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.928+0000 7f70e96e4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f70e4110fa0 con 0x7f70e41010e0 2024-04-21T08:40:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.928+0000 7f70e96e4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f70e41112a0 con 0x7f70e419cf70 2024-04-21T08:40:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.928+0000 7f70e27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70e41061b0 0x7f70e4198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.928+0000 7f70e27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70e41061b0 0x7f70e4198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39928/0 (socket says 172.21.15.43:39928) 2024-04-21T08:40:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.928+0000 7f70e27fc640 1 -- 172.21.15.43:0/1285775924 learned_addr learned my addr 172.21.15.43:0/1285775924 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.929+0000 7f70e37fe640 1 --2- 172.21.15.43:0/1285775924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70e419cf70 0x7f70e419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.929+0000 7f70e2ffd640 1 --2- 172.21.15.43:0/1285775924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70e41010e0 0x7f70e41986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.929+0000 7f70e27fc640 1 -- 172.21.15.43:0/1285775924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70e419cf70 msgr2=0x7f70e419d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.929+0000 7f70e27fc640 1 --2- 172.21.15.43:0/1285775924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70e419cf70 0x7f70e419d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.929+0000 7f70e27fc640 1 -- 172.21.15.43:0/1285775924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70e41010e0 msgr2=0x7f70e41986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.929+0000 7f70e27fc640 1 --2- 172.21.15.43:0/1285775924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70e41010e0 0x7f70e41986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.929+0000 7f70e27fc640 1 -- 172.21.15.43:0/1285775924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e419da60 con 0x7f70e41061b0 2024-04-21T08:40:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.929+0000 7f70e2ffd640 1 --2- 172.21.15.43:0/1285775924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70e41010e0 0x7f70e41986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.929+0000 7f70e27fc640 1 --2- 172.21.15.43:0/1285775924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70e41061b0 0x7f70e4198bf0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f70d400d770 tx=0x7f70d400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.929+0000 7f70dbfff640 1 -- 172.21.15.43:0/1285775924 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d4017070 con 0x7f70e41061b0 2024-04-21T08:40:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.929+0000 7f70e96e4640 1 -- 172.21.15.43:0/1285775924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e419dd50 con 0x7f70e41061b0 2024-04-21T08:40:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.929+0000 7f70dbfff640 1 -- 172.21.15.43:0/1285775924 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70d4009c30 con 0x7f70e41061b0 2024-04-21T08:40:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.929+0000 7f70dbfff640 1 -- 172.21.15.43:0/1285775924 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d400f620 con 0x7f70e41061b0 2024-04-21T08:40:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.929+0000 7f70e96e4640 1 -- 172.21.15.43:0/1285775924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e41a5540 con 0x7f70e41061b0 2024-04-21T08:40:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.932+0000 7f70dbfff640 1 -- 172.21.15.43:0/1285775924 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f70d401a070 con 0x7f70e41061b0 2024-04-21T08:40:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.932+0000 7f70ec3b3740 1 Processor -- start 2024-04-21T08:40:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.932+0000 7f70ec3b3740 1 -- start start 2024-04-21T08:40:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.932+0000 7f70ec3b3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164c66a8d0 0x56164c669620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.932+0000 7f70ec3b3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164c669b60 0x56164c669fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.932+0000 7f70ec3b3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c6730e0 0x56164c6754d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.932+0000 7f70ec3b3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56164c5ce8d0 con 0x56164c6730e0 2024-04-21T08:40:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.932+0000 7f70ec3b3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56164c56be80 con 0x56164c66a8d0 2024-04-21T08:40:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.932+0000 7f70ec3b3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56164c5ca890 con 0x56164c669b60 2024-04-21T08:40:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.933+0000 7f70e27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c6730e0 0x56164c6754d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.933+0000 7f70e27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c6730e0 0x56164c6754d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39930/0 (socket says 172.21.15.43:39930) 2024-04-21T08:40:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.933+0000 7f70e37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164c66a8d0 0x56164c669620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.933+0000 7f70e27fc640 1 -- 172.21.15.43:0/2916681062 learned_addr learned my addr 172.21.15.43:0/2916681062 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.933+0000 7f70dbfff640 1 --2- 172.21.15.43:0/1285775924 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f70d00a3ee0 0x7f70d00a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.933+0000 7f70e2ffd640 1 --2- 172.21.15.43:0/2916681062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164c669b60 0x56164c669fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.933+0000 7f70e37fe640 1 --2- 172.21.15.43:0/1285775924 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f70d00a3ee0 0x7f70d00a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.933+0000 7f70dbfff640 1 -- 172.21.15.43:0/1285775924 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f70d40d0aa0 con 0x7f70e41061b0 2024-04-21T08:40:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.933+0000 7f70e27fc640 1 -- 172.21.15.43:0/2916681062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164c669b60 msgr2=0x56164c669fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.933+0000 7f70e27fc640 1 --2- 172.21.15.43:0/2916681062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164c669b60 0x56164c669fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.933+0000 7f70e27fc640 1 -- 172.21.15.43:0/2916681062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164c66a8d0 msgr2=0x56164c669620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.933+0000 7f70e27fc640 1 --2- 172.21.15.43:0/2916681062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164c66a8d0 0x56164c669620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.933+0000 7f70e27fc640 1 -- 172.21.15.43:0/2916681062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56164c58d320 con 0x56164c6730e0 2024-04-21T08:40:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.933+0000 7f70e37fe640 1 --2- 172.21.15.43:0/2916681062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164c66a8d0 0x56164c669620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:40:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.933+0000 7f70e2ffd640 1 --2- 172.21.15.43:0/2916681062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164c669b60 0x56164c669fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:40:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.933+0000 7f70e27fc640 1 --2- 172.21.15.43:0/2916681062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c6730e0 0x56164c6754d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f70d40cddb0 tx=0x7f70d400df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef2195ed50800 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.934+0000 7f70e37fe640 1 --2- 172.21.15.43:0/1285775924 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f70d00a3ee0 0x7f70d00a63a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f70dc005c40 tx=0x7f70dc004ab0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:20.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.936+0000 7f70d97fa640 1 -- 172.21.15.43:0/2916681062 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d40d6070 con 0x56164c6730e0 2024-04-21T08:40:20.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.936+0000 7f70d97fa640 1 -- 172.21.15.43:0/2916681062 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70d4009de0 con 0x56164c6730e0 2024-04-21T08:40:20.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.936+0000 7f70d97fa640 1 -- 172.21.15.43:0/2916681062 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d4021040 con 0x56164c6730e0 2024-04-21T08:40:20.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.936+0000 7f70ec3b3740 1 -- 172.21.15.43:0/2916681062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c6730e0 msgr2=0x56164c6754d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:20.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.936+0000 7f70ec3b3740 1 --2- 172.21.15.43:0/2916681062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c6730e0 0x56164c6754d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f70d40cddb0 tx=0x7f70d400df10 comp rx=0 tx=0).stop 2024-04-21T08:40:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.937+0000 7f70ec3b3740 1 -- 172.21.15.43:0/2916681062 shutdown_connections 2024-04-21T08:40:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.937+0000 7f70ec3b3740 1 --2- 172.21.15.43:0/2916681062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c6730e0 0x56164c6754d0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.937+0000 7f70ec3b3740 1 --2- 172.21.15.43:0/2916681062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164c669b60 0x56164c669fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.937+0000 7f70ec3b3740 1 --2- 172.21.15.43:0/2916681062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164c66a8d0 0x56164c669620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.937+0000 7f70ec3b3740 1 -- 172.21.15.43:0/2916681062 >> 172.21.15.43:0/2916681062 conn(0x56164c630c90 msgr2=0x56164c664850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.937+0000 7f70ec3b3740 1 -- 172.21.15.43:0/2916681062 shutdown_connections 2024-04-21T08:40:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.937+0000 7f70ec3b3740 1 -- 172.21.15.43:0/2916681062 wait complete. 2024-04-21T08:40:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.937+0000 7f70ec3b3740 1 -- start start 2024-04-21T08:40:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.937+0000 7f70ec3b3740 1 Processor -- start 2024-04-21T08:40:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.937+0000 7f70ec3b3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c669b60 0x56164c672aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.937+0000 7f70ec3b3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164c66a8d0 0x56164c66fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.937+0000 7f70ec3b3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164c6730e0 0x56164c670440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.937+0000 7f70ec3b3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56164c675d50 con 0x56164c669b60 2024-04-21T08:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.937+0000 7f70ec3b3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56164c556eb0 con 0x56164c6730e0 2024-04-21T08:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.937+0000 7f70ec3b3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56164c675ed0 con 0x56164c66a8d0 2024-04-21T08:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.937+0000 7f70ec3b3740 10 client.0 _ll_register_callbacks cb 0x56164c583b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.937+0000 7f70e2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c669b60 0x56164c672aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.938+0000 7f70e2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c669b60 0x56164c672aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39938/0 (socket says 172.21.15.43:39938) 2024-04-21T08:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.938+0000 7f70e2ffd640 1 -- 172.21.15.43:0/555300302 learned_addr learned my addr 172.21.15.43:0/555300302 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.938+0000 7f70e37fe640 1 --2- 172.21.15.43:0/555300302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164c6730e0 0x56164c670440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.938+0000 7f70e27fc640 1 --2- 172.21.15.43:0/555300302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164c66a8d0 0x56164c66fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.938+0000 7f70e2ffd640 1 -- 172.21.15.43:0/555300302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164c66a8d0 msgr2=0x56164c66fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.938+0000 7f70e2ffd640 1 --2- 172.21.15.43:0/555300302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164c66a8d0 0x56164c66fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.938+0000 7f70e2ffd640 1 -- 172.21.15.43:0/555300302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164c6730e0 msgr2=0x56164c670440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.938+0000 7f70e2ffd640 1 --2- 172.21.15.43:0/555300302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164c6730e0 0x56164c670440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.938+0000 7f70e2ffd640 1 -- 172.21.15.43:0/555300302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56164c2e1210 con 0x56164c669b60 2024-04-21T08:40:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.938+0000 7f70e2ffd640 1 --2- 172.21.15.43:0/555300302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c669b60 0x56164c672aa0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f70cc009230 tx=0x7f70cc02ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=786059d499d378a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.938+0000 7f70ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.938+0000 7f70ca7fc640 1 -- 172.21.15.43:0/555300302 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc02c5d0 con 0x56164c669b60 2024-04-21T08:40:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.938+0000 7f70ca7fc640 1 -- 172.21.15.43:0/555300302 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f70cc005030 con 0x56164c669b60 2024-04-21T08:40:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.938+0000 7f70ec3b3740 1 -- 172.21.15.43:0/555300302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70d4009de0 con 0x56164c669b60 2024-04-21T08:40:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.938+0000 7f70ca7fc640 1 -- 172.21.15.43:0/555300302 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f70cc04f050 con 0x56164c669b60 2024-04-21T08:40:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.938+0000 7f70ec3b3740 10 client.16425 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.938+0000 7f70ec3b3740 1 -- 172.21.15.43:0/555300302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56164c3ad620 con 0x56164c669b60 2024-04-21T08:40:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.939+0000 7f70ca7fc640 1 -- 172.21.15.43:0/555300302 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70d4009de0 con 0x56164c669b60 2024-04-21T08:40:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.939+0000 7f70ca7fc640 1 -- 172.21.15.43:0/555300302 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f70cc0035f0 con 0x56164c669b60 2024-04-21T08:40:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.939+0000 7f70ec3b3740 10 client.16425 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.939+0000 7f70ec3b3740 10 client.16425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.939+0000 7f70ec3b3740 20 client.16425 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.939+0000 7f70ec3b3740 1 --2- 172.21.15.43:0/555300302 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56164c683120 0x56164c685510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.939+0000 7f70ec3b3740 4 client.16425 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.939+0000 7f70ec3b3740 1 -- 172.21.15.43:0/555300302 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56164c5e2880 con 0x56164c683120 2024-04-21T08:40:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.939+0000 7f70e27fc640 1 --2- 172.21.15.43:0/555300302 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56164c683120 0x56164c685510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.940+0000 7f70e27fc640 1 --2- 172.21.15.43:0/555300302 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56164c683120 0x56164c685510 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28f6a1888bf43846 server_cookie=7ea4b176e26cfb8c in_seq=0 out_seq=0 2024-04-21T08:40:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.940+0000 7f70ca7fc640 10 client.16425 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.940+0000 7f70ca7fc640 1 -- 172.21.15.43:0/555300302 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56164c5e2880 con 0x56164c683120 2024-04-21T08:40:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.940+0000 7f70ca7fc640 10 client.16425 handle_command_reply: tid=0 2024-04-21T08:40:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.941+0000 7f70ec3b3740 1 client.16425 shutdown 2024-04-21T08:40:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.941+0000 7f70ec3b3740 1 -- 172.21.15.43:0/555300302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c669b60 msgr2=0x56164c672aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.941+0000 7f70ec3b3740 1 --2- 172.21.15.43:0/555300302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c669b60 0x56164c672aa0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f70cc009230 tx=0x7f70cc02ca10 comp rx=0 tx=0).stop 2024-04-21T08:40:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.941+0000 7f70ec3b3740 1 -- 172.21.15.43:0/555300302 shutdown_connections 2024-04-21T08:40:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.941+0000 7f70ec3b3740 1 --2- 172.21.15.43:0/555300302 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56164c683120 0x56164c685510 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.941+0000 7f70ec3b3740 1 --2- 172.21.15.43:0/555300302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164c6730e0 0x56164c670440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.941+0000 7f70ec3b3740 1 --2- 172.21.15.43:0/555300302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164c66a8d0 0x56164c66fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.941+0000 7f70ca7fc640 0 client.16425 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.941+0000 7f70ec3b3740 1 --2- 172.21.15.43:0/555300302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c669b60 0x56164c672aa0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.941+0000 7f70ec3b3740 1 -- 172.21.15.43:0/555300302 >> 172.21.15.43:0/555300302 conn(0x56164c630c90 msgr2=0x56164c6665a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.941+0000 7f70ec3b3740 1 -- 172.21.15.43:0/555300302 shutdown_connections 2024-04-21T08:40:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.941+0000 7f70ec3b3740 1 -- 172.21.15.43:0/555300302 wait complete. 2024-04-21T08:40:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.941+0000 7f70ec3b3740 20 client.16425 trim_cache size 0 max 16384 2024-04-21T08:40:20.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.956+0000 7f70ec3b3740 1 Processor -- start 2024-04-21T08:40:20.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.956+0000 7f70ec3b3740 1 -- start start 2024-04-21T08:40:20.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.956+0000 7f70ec3b3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164c669b60 0x56164c67ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:20.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.956+0000 7f70ec3b3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164c66a8d0 0x56164c66ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.956+0000 7f70ec3b3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c6730e0 0x56164c672560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.956+0000 7f70ec3b3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f70e41af3c0 con 0x56164c6730e0 2024-04-21T08:40:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.956+0000 7f70ec3b3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f70e41b0f00 con 0x56164c669b60 2024-04-21T08:40:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.956+0000 7f70ec3b3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f70e41afa90 con 0x56164c66a8d0 2024-04-21T08:40:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.957+0000 7f70e37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c6730e0 0x56164c672560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.957+0000 7f70e37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c6730e0 0x56164c672560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39944/0 (socket says 172.21.15.43:39944) 2024-04-21T08:40:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.957+0000 7f70e37fe640 1 -- 172.21.15.43:0/1781729911 learned_addr learned my addr 172.21.15.43:0/1781729911 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.957+0000 7f70e27fc640 1 --2- 172.21.15.43:0/1781729911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164c66a8d0 0x56164c66ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.957+0000 7f70e2ffd640 1 --2- 172.21.15.43:0/1781729911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164c669b60 0x56164c67ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.957+0000 7f70e37fe640 1 -- 172.21.15.43:0/1781729911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164c66a8d0 msgr2=0x56164c66ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.957+0000 7f70e37fe640 1 --2- 172.21.15.43:0/1781729911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164c66a8d0 0x56164c66ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.957+0000 7f70e37fe640 1 -- 172.21.15.43:0/1781729911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164c669b60 msgr2=0x56164c67ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.957+0000 7f70e37fe640 1 --2- 172.21.15.43:0/1781729911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164c669b60 0x56164c67ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.957+0000 7f70e37fe640 1 -- 172.21.15.43:0/1781729911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56164c668e30 con 0x56164c6730e0 2024-04-21T08:40:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.957+0000 7f70e2ffd640 1 --2- 172.21.15.43:0/1781729911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164c669b60 0x56164c67ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.957+0000 7f70e37fe640 1 --2- 172.21.15.43:0/1781729911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c6730e0 0x56164c672560 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f70dc013470 tx=0x7f70dc0099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9f3314d129eaba server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.957+0000 7f70d8ff9640 1 -- 172.21.15.43:0/1781729911 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc026070 con 0x56164c6730e0 2024-04-21T08:40:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.957+0000 7f70d8ff9640 1 -- 172.21.15.43:0/1781729911 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70dc0134a0 con 0x56164c6730e0 2024-04-21T08:40:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.957+0000 7f70d8ff9640 1 -- 172.21.15.43:0/1781729911 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc019040 con 0x56164c6730e0 2024-04-21T08:40:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.958+0000 7f70ec3b3740 1 -- 172.21.15.43:0/1781729911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c6730e0 msgr2=0x56164c672560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.958+0000 7f70ec3b3740 1 --2- 172.21.15.43:0/1781729911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c6730e0 0x56164c672560 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f70dc013470 tx=0x7f70dc0099b0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.958+0000 7f70ec3b3740 1 -- 172.21.15.43:0/1781729911 shutdown_connections 2024-04-21T08:40:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.958+0000 7f70ec3b3740 1 --2- 172.21.15.43:0/1781729911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c6730e0 0x56164c672560 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.958+0000 7f70ec3b3740 1 --2- 172.21.15.43:0/1781729911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164c66a8d0 0x56164c66ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.958+0000 7f70ec3b3740 1 --2- 172.21.15.43:0/1781729911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164c669b60 0x56164c67ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.958+0000 7f70ec3b3740 1 -- 172.21.15.43:0/1781729911 >> 172.21.15.43:0/1781729911 conn(0x56164c630c90 msgr2=0x56164c66e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.958+0000 7f70ec3b3740 1 -- 172.21.15.43:0/1781729911 shutdown_connections 2024-04-21T08:40:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.958+0000 7f70ec3b3740 1 -- 172.21.15.43:0/1781729911 wait complete. 2024-04-21T08:40:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.958+0000 7f70ec3b3740 1 -- start start 2024-04-21T08:40:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.959+0000 7f70ec3b3740 1 Processor -- start 2024-04-21T08:40:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.959+0000 7f70ec3b3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164c669b60 0x56164c66ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.959+0000 7f70ec3b3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c66a8d0 0x56164c66cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.959+0000 7f70ec3b3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164c6730e0 0x56164c66d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.959+0000 7f70ec3b3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f70e41b0830 con 0x56164c66a8d0 2024-04-21T08:40:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.959+0000 7f70ec3b3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f70e41b0160 con 0x56164c6730e0 2024-04-21T08:40:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.959+0000 7f70ec3b3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f70e41ad3a0 con 0x56164c669b60 2024-04-21T08:40:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.959+0000 7f70ec3b3740 10 client.0 _ll_register_callbacks cb 0x7f70cc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.959+0000 7f70e27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c66a8d0 0x56164c66cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.959+0000 7f70e27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c66a8d0 0x56164c66cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39956/0 (socket says 172.21.15.43:39956) 2024-04-21T08:40:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.959+0000 7f70e27fc640 1 -- 172.21.15.43:0/427063438 learned_addr learned my addr 172.21.15.43:0/427063438 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.959+0000 7f70e2ffd640 1 --2- 172.21.15.43:0/427063438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164c669b60 0x56164c66ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.959+0000 7f70e37fe640 1 --2- 172.21.15.43:0/427063438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164c6730e0 0x56164c66d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.959+0000 7f70e27fc640 1 -- 172.21.15.43:0/427063438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164c669b60 msgr2=0x56164c66ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.959+0000 7f70e27fc640 1 --2- 172.21.15.43:0/427063438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164c669b60 0x56164c66ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.959+0000 7f70e27fc640 1 -- 172.21.15.43:0/427063438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164c6730e0 msgr2=0x56164c66d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.959+0000 7f70e27fc640 1 --2- 172.21.15.43:0/427063438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164c6730e0 0x56164c66d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.959+0000 7f70e27fc640 1 -- 172.21.15.43:0/427063438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56164c65ff30 con 0x56164c66a8d0 2024-04-21T08:40:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.959+0000 7f70e2ffd640 1 --2- 172.21.15.43:0/427063438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164c669b60 0x56164c66ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.959+0000 7f70e27fc640 1 --2- 172.21.15.43:0/427063438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c66a8d0 0x56164c66cf80 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f70d40ce420 tx=0x7f70d409be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26e6f5076e51293 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.959+0000 7f70ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.959+0000 7f70ca7fc640 1 -- 172.21.15.43:0/427063438 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d40d6040 con 0x56164c66a8d0 2024-04-21T08:40:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.960+0000 7f70ca7fc640 1 -- 172.21.15.43:0/427063438 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f70d4098a90 con 0x56164c66a8d0 2024-04-21T08:40:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.960+0000 7f70ec3b3740 1 -- 172.21.15.43:0/427063438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70dc0134a0 con 0x56164c66a8d0 2024-04-21T08:40:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.960+0000 7f70ca7fc640 1 -- 172.21.15.43:0/427063438 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f70d4021040 con 0x56164c66a8d0 2024-04-21T08:40:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.960+0000 7f70ec3b3740 10 client.16431 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.960+0000 7f70ec3b3740 1 -- 172.21.15.43:0/427063438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56164c6e9690 con 0x56164c66a8d0 2024-04-21T08:40:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.960+0000 7f70ca7fc640 1 -- 172.21.15.43:0/427063438 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70dc0134a0 con 0x56164c66a8d0 2024-04-21T08:40:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.960+0000 7f70ca7fc640 1 -- 172.21.15.43:0/427063438 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f70d40cdf80 con 0x56164c66a8d0 2024-04-21T08:40:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.960+0000 7f70ec3b3740 10 client.16431 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.960+0000 7f70ec3b3740 10 client.16431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.960+0000 7f70ec3b3740 20 client.16431 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.960+0000 7f70ec3b3740 1 --2- 172.21.15.43:0/427063438 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56164c6e99d0 0x56164c6ebdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.960+0000 7f70ec3b3740 4 client.16431 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:40:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.960+0000 7f70ec3b3740 1 -- 172.21.15.43:0/427063438 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56164c56e3d0 con 0x56164c6e99d0 2024-04-21T08:40:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.960+0000 7f70e2ffd640 1 --2- 172.21.15.43:0/427063438 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56164c6e99d0 0x56164c6ebdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.961+0000 7f70ca7fc640 10 client.16431 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.961+0000 7f70e2ffd640 1 --2- 172.21.15.43:0/427063438 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56164c6e99d0 0x56164c6ebdc0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39ae9aed85c10f9e server_cookie=4e63d190bd18dc46 in_seq=0 out_seq=0 2024-04-21T08:40:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.974+0000 7f70ca7fc640 1 -- 172.21.15.43:0/427063438 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56164c56e3d0 con 0x56164c6e99d0 2024-04-21T08:40:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.974+0000 7f70ca7fc640 10 client.16431 handle_command_reply: tid=0 2024-04-21T08:40:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.974+0000 7f70ec3b3740 1 client.16431 shutdown 2024-04-21T08:40:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.974+0000 7f70ec3b3740 1 -- 172.21.15.43:0/427063438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c66a8d0 msgr2=0x56164c66cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.974+0000 7f70ec3b3740 1 --2- 172.21.15.43:0/427063438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c66a8d0 0x56164c66cf80 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f70d40ce420 tx=0x7f70d409be80 comp rx=0 tx=0).stop 2024-04-21T08:40:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.974+0000 7f70ec3b3740 1 -- 172.21.15.43:0/427063438 shutdown_connections 2024-04-21T08:40:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.974+0000 7f70ec3b3740 1 --2- 172.21.15.43:0/427063438 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56164c6e99d0 0x56164c6ebdc0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.974+0000 7f70ec3b3740 1 --2- 172.21.15.43:0/427063438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164c6730e0 0x56164c66d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.974+0000 7f70ec3b3740 1 --2- 172.21.15.43:0/427063438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56164c66a8d0 0x56164c66cf80 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.974+0000 7f70ec3b3740 1 --2- 172.21.15.43:0/427063438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56164c669b60 0x56164c66ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.974+0000 7f70ca7fc640 0 client.16431 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.974+0000 7f70ec3b3740 1 -- 172.21.15.43:0/427063438 >> 172.21.15.43:0/427063438 conn(0x56164c630c90 msgr2=0x56164c663040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.974+0000 7f70ec3b3740 1 -- 172.21.15.43:0/427063438 shutdown_connections 2024-04-21T08:40:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.974+0000 7f70ec3b3740 1 -- 172.21.15.43:0/427063438 wait complete. 2024-04-21T08:40:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.975+0000 7f70ec3b3740 20 client.16431 trim_cache size 0 max 16384 2024-04-21T08:40:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:40:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:40:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:40:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:40:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.975+0000 7f70e96e4640 1 -- 172.21.15.43:0/1285775924 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f70d00a3ee0 msgr2=0x7f70d00a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.975+0000 7f70e96e4640 1 --2- 172.21.15.43:0/1285775924 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f70d00a3ee0 0x7f70d00a63a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f70dc005c40 tx=0x7f70dc004ab0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.975+0000 7f70e96e4640 1 -- 172.21.15.43:0/1285775924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70e41061b0 msgr2=0x7f70e4198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.975+0000 7f70e96e4640 1 --2- 172.21.15.43:0/1285775924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70e41061b0 0x7f70e4198bf0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f70d400d770 tx=0x7f70d400dc30 comp rx=0 tx=0).stop 2024-04-21T08:40:20.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.975+0000 7f70e96e4640 1 -- 172.21.15.43:0/1285775924 shutdown_connections 2024-04-21T08:40:20.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.975+0000 7f70e96e4640 1 --2- 172.21.15.43:0/1285775924 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f70d00a3ee0 0x7f70d00a63a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.975+0000 7f70e96e4640 1 --2- 172.21.15.43:0/1285775924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70e419cf70 0x7f70e419d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.975+0000 7f70e96e4640 1 --2- 172.21.15.43:0/1285775924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70e41061b0 0x7f70e4198bf0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.975+0000 7f70e96e4640 1 --2- 172.21.15.43:0/1285775924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70e41010e0 0x7f70e41986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:20.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.975+0000 7f70e96e4640 1 -- 172.21.15.43:0/1285775924 >> 172.21.15.43:0/1285775924 conn(0x7f70e40771d0 msgr2=0x7f70e406aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:20.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.975+0000 7f70e96e4640 1 -- 172.21.15.43:0/1285775924 shutdown_connections 2024-04-21T08:40:20.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:20.975+0000 7f70e96e4640 1 -- 172.21.15.43:0/1285775924 wait complete. 2024-04-21T08:40:22.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:21 smithi175 ceph-mon[31920]: pgmap v46: 129 pgs: 129 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 23 op/s 2024-04-21T08:40:22.293 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:21 smithi112 ceph-mon[31085]: pgmap v46: 129 pgs: 129 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 23 op/s 2024-04-21T08:40:22.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:21 smithi043 ceph-mon[27978]: pgmap v46: 129 pgs: 129 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 23 op/s 2024-04-21T08:40:24.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:23 smithi175 ceph-mon[31920]: pgmap v47: 129 pgs: 129 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-21T08:40:24.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:23 smithi112 ceph-mon[31085]: pgmap v47: 129 pgs: 129 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-21T08:40:24.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:23 smithi043 ceph-mon[27978]: pgmap v47: 129 pgs: 129 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-21T08:40:26.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:25 smithi175 ceph-mon[31920]: pgmap v48: 129 pgs: 129 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-21T08:40:26.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:25 smithi112 ceph-mon[31085]: pgmap v48: 129 pgs: 129 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-21T08:40:26.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:25 smithi043 ceph-mon[27978]: pgmap v48: 129 pgs: 129 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-21T08:40:28.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:27 smithi175 ceph-mon[31920]: pgmap v49: 129 pgs: 129 active+clean; 1.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-04-21T08:40:28.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:27 smithi175 ceph-mon[31920]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-21T08:40:28.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:27 smithi112 ceph-mon[31085]: pgmap v49: 129 pgs: 129 active+clean; 1.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-04-21T08:40:28.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:27 smithi112 ceph-mon[31085]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-21T08:40:28.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:27 smithi043 ceph-mon[27978]: pgmap v49: 129 pgs: 129 active+clean; 1.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-04-21T08:40:28.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:27 smithi043 ceph-mon[27978]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-21T08:40:30.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:29 smithi175 ceph-mon[31920]: pgmap v50: 129 pgs: 129 active+clean; 1.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-04-21T08:40:30.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:29 smithi112 ceph-mon[31085]: pgmap v50: 129 pgs: 129 active+clean; 1.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-04-21T08:40:30.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:29 smithi043 ceph-mon[27978]: pgmap v50: 129 pgs: 129 active+clean; 1.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-04-21T08:40:32.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:31 smithi175 ceph-mon[31920]: pgmap v51: 129 pgs: 129 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-04-21T08:40:32.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:31 smithi112 ceph-mon[31085]: pgmap v51: 129 pgs: 129 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-04-21T08:40:32.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:31 smithi043 ceph-mon[27978]: pgmap v51: 129 pgs: 129 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-04-21T08:40:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.076+0000 7f655406a640 1 -- 172.21.15.43:0/1170964407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f654c101910 msgr2=0x7f654c105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.076+0000 7f655406a640 1 --2- 172.21.15.43:0/1170964407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f654c101910 0x7f654c105dd0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f653c009930 tx=0x7f653c02bcd0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.076+0000 7f655406a640 1 -- 172.21.15.43:0/1170964407 shutdown_connections 2024-04-21T08:40:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.076+0000 7f655406a640 1 --2- 172.21.15.43:0/1170964407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f654c1063f0 0x7f654c1068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.076+0000 7f655406a640 1 --2- 172.21.15.43:0/1170964407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f654c101910 0x7f654c105dd0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.076+0000 7f655406a640 1 --2- 172.21.15.43:0/1170964407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f654c100ff0 0x7f654c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.076+0000 7f655406a640 1 -- 172.21.15.43:0/1170964407 >> 172.21.15.43:0/1170964407 conn(0x7f654c0771d0 msgr2=0x7f654c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.076+0000 7f655406a640 1 -- 172.21.15.43:0/1170964407 shutdown_connections 2024-04-21T08:40:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.076+0000 7f655406a640 1 -- 172.21.15.43:0/1170964407 wait complete. 2024-04-21T08:40:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.077+0000 7f655406a640 1 Processor -- start 2024-04-21T08:40:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.077+0000 7f655406a640 1 -- start start 2024-04-21T08:40:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.077+0000 7f655406a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f654c100ff0 0x7f654c1a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.077+0000 7f655406a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f654c101910 0x7f654c1a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.077+0000 7f655406a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f654c1063f0 0x7f654c19eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.077+0000 7f655406a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f654c1187d0 con 0x7f654c1063f0 2024-04-21T08:40:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.077+0000 7f655406a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f654c118650 con 0x7f654c101910 2024-04-21T08:40:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.077+0000 7f655406a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f654c118950 con 0x7f654c100ff0 2024-04-21T08:40:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.077+0000 7f6552607640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f654c1063f0 0x7f654c19eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.077+0000 7f6552607640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f654c1063f0 0x7f654c19eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56558/0 (socket says 172.21.15.43:56558) 2024-04-21T08:40:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.077+0000 7f6552607640 1 -- 172.21.15.43:0/3793803945 learned_addr learned my addr 172.21.15.43:0/3793803945 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.077+0000 7f6551e06640 1 --2- 172.21.15.43:0/3793803945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f654c100ff0 0x7f654c1a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.077+0000 7f6551605640 1 --2- 172.21.15.43:0/3793803945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f654c101910 0x7f654c1a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.077+0000 7f6552607640 1 -- 172.21.15.43:0/3793803945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f654c100ff0 msgr2=0x7f654c1a4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.077+0000 7f6552607640 1 --2- 172.21.15.43:0/3793803945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f654c100ff0 0x7f654c1a4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.077+0000 7f6552607640 1 -- 172.21.15.43:0/3793803945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f654c101910 msgr2=0x7f654c1a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.077+0000 7f6552607640 1 --2- 172.21.15.43:0/3793803945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f654c101910 0x7f654c1a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.077+0000 7f6552607640 1 -- 172.21.15.43:0/3793803945 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654c19f230 con 0x7f654c1063f0 2024-04-21T08:40:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.078+0000 7f6552607640 1 --2- 172.21.15.43:0/3793803945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f654c1063f0 0x7f654c19eab0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6544009fd0 tx=0x7f654400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.078+0000 7f6542ffd640 1 -- 172.21.15.43:0/3793803945 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6544017070 con 0x7f654c1063f0 2024-04-21T08:40:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.078+0000 7f6542ffd640 1 -- 172.21.15.43:0/3793803945 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6544003e20 con 0x7f654c1063f0 2024-04-21T08:40:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.078+0000 7f655406a640 1 -- 172.21.15.43:0/3793803945 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f654c19f520 con 0x7f654c1063f0 2024-04-21T08:40:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.078+0000 7f6542ffd640 1 -- 172.21.15.43:0/3793803945 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6544010460 con 0x7f654c1063f0 2024-04-21T08:40:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.078+0000 7f655406a640 1 -- 172.21.15.43:0/3793803945 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f654c1ab7c0 con 0x7f654c1063f0 2024-04-21T08:40:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.080+0000 7f6542ffd640 1 -- 172.21.15.43:0/3793803945 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6544003970 con 0x7f654c1063f0 2024-04-21T08:40:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.081+0000 7f6556c5d740 1 Processor -- start 2024-04-21T08:40:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.081+0000 7f6556c5d740 1 -- start start 2024-04-21T08:40:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.081+0000 7f6556c5d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a729428d0 0x562a72941620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.081+0000 7f6556c5d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a72941b60 0x562a72941fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.081+0000 7f6556c5d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a7294b0e0 0x562a7294d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.081+0000 7f6556c5d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562a728a68d0 con 0x562a7294b0e0 2024-04-21T08:40:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.081+0000 7f6556c5d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562a72843e80 con 0x562a72941b60 2024-04-21T08:40:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.081+0000 7f6556c5d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562a728a2890 con 0x562a729428d0 2024-04-21T08:40:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.081+0000 7f6542ffd640 1 --2- 172.21.15.43:0/3793803945 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f65380a3dc0 0x7f65380a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.081+0000 7f6551605640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a7294b0e0 0x562a7294d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.081+0000 7f6551605640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a7294b0e0 0x562a7294d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56562/0 (socket says 172.21.15.43:56562) 2024-04-21T08:40:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.081+0000 7f6551605640 1 -- 172.21.15.43:0/346899382 learned_addr learned my addr 172.21.15.43:0/346899382 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.081+0000 7f6552607640 1 --2- 172.21.15.43:0/3793803945 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f65380a3dc0 0x7f65380a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.081+0000 7f6551605640 1 --2- 172.21.15.43:0/346899382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a729428d0 0x562a72941620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.082+0000 7f6542ffd640 1 -- 172.21.15.43:0/3793803945 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f65440d08c0 con 0x7f654c1063f0 2024-04-21T08:40:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.082+0000 7f6551e06640 1 --2- 172.21.15.43:0/346899382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a72941b60 0x562a72941fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.082+0000 7f6551605640 1 -- 172.21.15.43:0/346899382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a729428d0 msgr2=0x562a72941620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.082+0000 7f6551605640 1 --2- 172.21.15.43:0/346899382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a729428d0 0x562a72941620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.082+0000 7f6551605640 1 -- 172.21.15.43:0/346899382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a72941b60 msgr2=0x562a72941fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.082+0000 7f6551605640 1 --2- 172.21.15.43:0/346899382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a72941b60 0x562a72941fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.082+0000 7f6551605640 1 -- 172.21.15.43:0/346899382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a72865320 con 0x562a7294b0e0 2024-04-21T08:40:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.082+0000 7f6552607640 1 --2- 172.21.15.43:0/3793803945 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f65380a3dc0 0x7f65380a6280 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f654409c470 tx=0x7f654406df20 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.082+0000 7f6551605640 1 --2- 172.21.15.43:0/346899382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a7294b0e0 0x562a7294d4d0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f653c005c40 tx=0x7f653c005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491bcccb5bd6a6ea server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.082+0000 7f6537fff640 1 -- 172.21.15.43:0/346899382 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c02c880 con 0x562a7294b0e0 2024-04-21T08:40:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.082+0000 7f6537fff640 1 -- 172.21.15.43:0/346899382 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f653c003aa0 con 0x562a7294b0e0 2024-04-21T08:40:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.082+0000 7f6537fff640 1 -- 172.21.15.43:0/346899382 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c0438b0 con 0x562a7294b0e0 2024-04-21T08:40:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.083+0000 7f6556c5d740 1 -- 172.21.15.43:0/346899382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a7294b0e0 msgr2=0x562a7294d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.083+0000 7f6556c5d740 1 --2- 172.21.15.43:0/346899382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a7294b0e0 0x562a7294d4d0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f653c005c40 tx=0x7f653c005d90 comp rx=0 tx=0).stop 2024-04-21T08:40:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.083+0000 7f6556c5d740 1 -- 172.21.15.43:0/346899382 shutdown_connections 2024-04-21T08:40:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.083+0000 7f6556c5d740 1 --2- 172.21.15.43:0/346899382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a7294b0e0 0x562a7294d4d0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.083+0000 7f6556c5d740 1 --2- 172.21.15.43:0/346899382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a72941b60 0x562a72941fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.083+0000 7f6556c5d740 1 --2- 172.21.15.43:0/346899382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a729428d0 0x562a72941620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.083+0000 7f6556c5d740 1 -- 172.21.15.43:0/346899382 >> 172.21.15.43:0/346899382 conn(0x562a72908c90 msgr2=0x562a7293c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.083+0000 7f6556c5d740 1 -- 172.21.15.43:0/346899382 shutdown_connections 2024-04-21T08:40:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.083+0000 7f6556c5d740 1 -- 172.21.15.43:0/346899382 wait complete. 2024-04-21T08:40:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.083+0000 7f6556c5d740 1 -- start start 2024-04-21T08:40:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.084+0000 7f6556c5d740 1 Processor -- start 2024-04-21T08:40:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.084+0000 7f6556c5d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a72941b60 0x562a7294aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.084+0000 7f6556c5d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a729428d0 0x562a72947de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.084+0000 7f6556c5d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a7294b0e0 0x562a72948340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.084+0000 7f6556c5d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562a7294dd50 con 0x562a72941b60 2024-04-21T08:40:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.084+0000 7f6556c5d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562a7282eeb0 con 0x562a729428d0 2024-04-21T08:40:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.084+0000 7f6556c5d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562a7294ded0 con 0x562a7294b0e0 2024-04-21T08:40:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.084+0000 7f6556c5d740 10 client.0 _ll_register_callbacks cb 0x562a7285bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.084+0000 7f6551605640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a72941b60 0x562a7294aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.084+0000 7f6551605640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a72941b60 0x562a7294aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56564/0 (socket says 172.21.15.43:56564) 2024-04-21T08:40:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.084+0000 7f6551605640 1 -- 172.21.15.43:0/2038794052 learned_addr learned my addr 172.21.15.43:0/2038794052 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.084+0000 7f6551e06640 1 --2- 172.21.15.43:0/2038794052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a729428d0 0x562a72947de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.084+0000 7f6551605640 1 -- 172.21.15.43:0/2038794052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a7294b0e0 msgr2=0x562a72948340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.084+0000 7f6551605640 1 --2- 172.21.15.43:0/2038794052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a7294b0e0 0x562a72948340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.084+0000 7f6551605640 1 -- 172.21.15.43:0/2038794052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a729428d0 msgr2=0x562a72947de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.084+0000 7f6551605640 1 --2- 172.21.15.43:0/2038794052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a729428d0 0x562a72947de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.084+0000 7f6551605640 1 -- 172.21.15.43:0/2038794052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a725b9210 con 0x562a72941b60 2024-04-21T08:40:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.085+0000 7f6551605640 1 --2- 172.21.15.43:0/2038794052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a72941b60 0x562a7294aa50 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f653c009900 tx=0x7f653c004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23d6c69743f1d9c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.085+0000 7f65357fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.085+0000 7f65357fa640 1 -- 172.21.15.43:0/2038794052 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c02ca80 con 0x562a72941b60 2024-04-21T08:40:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.085+0000 7f65357fa640 1 -- 172.21.15.43:0/2038794052 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f653c02cc20 con 0x562a72941b60 2024-04-21T08:40:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.085+0000 7f6556c5d740 1 -- 172.21.15.43:0/2038794052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f653c003aa0 con 0x562a72941b60 2024-04-21T08:40:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.085+0000 7f65357fa640 1 -- 172.21.15.43:0/2038794052 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f653c005850 con 0x562a72941b60 2024-04-21T08:40:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.085+0000 7f6556c5d740 10 client.16443 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.085+0000 7f6556c5d740 1 -- 172.21.15.43:0/2038794052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562a72685620 con 0x562a72941b60 2024-04-21T08:40:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.085+0000 7f65357fa640 1 -- 172.21.15.43:0/2038794052 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f653c003aa0 con 0x562a72941b60 2024-04-21T08:40:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.085+0000 7f65357fa640 1 -- 172.21.15.43:0/2038794052 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f653c0344d0 con 0x562a72941b60 2024-04-21T08:40:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.085+0000 7f6556c5d740 10 client.16443 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.085+0000 7f6556c5d740 10 client.16443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.086+0000 7f6556c5d740 20 client.16443 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.086+0000 7f6556c5d740 1 --2- 172.21.15.43:0/2038794052 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562a7295aee0 0x562a7295d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.086+0000 7f6556c5d740 4 client.16443 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.086+0000 7f6556c5d740 1 -- 172.21.15.43:0/2038794052 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a728ba880 con 0x562a7295aee0 2024-04-21T08:40:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.086+0000 7f6551e06640 1 --2- 172.21.15.43:0/2038794052 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562a7295aee0 0x562a7295d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.086+0000 7f6551e06640 1 --2- 172.21.15.43:0/2038794052 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562a7295aee0 0x562a7295d2d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc482ec71cd10e9d server_cookie=3e5feceb9df80380 in_seq=0 out_seq=0 2024-04-21T08:40:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.086+0000 7f65357fa640 10 client.16443 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.087+0000 7f65357fa640 1 -- 172.21.15.43:0/2038794052 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562a728ba880 con 0x562a7295aee0 2024-04-21T08:40:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.087+0000 7f65357fa640 10 client.16443 handle_command_reply: tid=0 2024-04-21T08:40:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.087+0000 7f6556c5d740 1 client.16443 shutdown 2024-04-21T08:40:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.087+0000 7f6556c5d740 1 -- 172.21.15.43:0/2038794052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a72941b60 msgr2=0x562a7294aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.087+0000 7f6556c5d740 1 --2- 172.21.15.43:0/2038794052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a72941b60 0x562a7294aa50 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f653c009900 tx=0x7f653c004540 comp rx=0 tx=0).stop 2024-04-21T08:40:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.087+0000 7f6556c5d740 1 -- 172.21.15.43:0/2038794052 shutdown_connections 2024-04-21T08:40:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.087+0000 7f6556c5d740 1 --2- 172.21.15.43:0/2038794052 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562a7295aee0 0x562a7295d2d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.087+0000 7f6556c5d740 1 --2- 172.21.15.43:0/2038794052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a7294b0e0 0x562a72948340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.087+0000 7f6556c5d740 1 --2- 172.21.15.43:0/2038794052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a729428d0 0x562a72947de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.087+0000 7f65357fa640 0 client.16443 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.087+0000 7f6556c5d740 1 --2- 172.21.15.43:0/2038794052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a72941b60 0x562a7294aa50 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.087+0000 7f6556c5d740 1 -- 172.21.15.43:0/2038794052 >> 172.21.15.43:0/2038794052 conn(0x562a72908c90 msgr2=0x562a7293e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.088+0000 7f6556c5d740 1 -- 172.21.15.43:0/2038794052 shutdown_connections 2024-04-21T08:40:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.088+0000 7f6556c5d740 1 -- 172.21.15.43:0/2038794052 wait complete. 2024-04-21T08:40:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.088+0000 7f6556c5d740 20 client.16443 trim_cache size 0 max 16384 2024-04-21T08:40:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.103+0000 7f6556c5d740 1 Processor -- start 2024-04-21T08:40:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.103+0000 7f6556c5d740 1 -- start start 2024-04-21T08:40:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.103+0000 7f6556c5d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a72941b60 0x562a72956cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.103+0000 7f6556c5d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a729428d0 0x562a7294a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.103+0000 7f6556c5d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a7294b0e0 0x562a7294aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.103+0000 7f6556c5d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f654c1b58e0 con 0x562a7294b0e0 2024-04-21T08:40:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.104+0000 7f6556c5d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f654c1b7420 con 0x562a729428d0 2024-04-21T08:40:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.104+0000 7f6556c5d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f654c1b5fb0 con 0x562a72941b60 2024-04-21T08:40:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.104+0000 7f6551605640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a7294b0e0 0x562a7294aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.104+0000 7f6551605640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a7294b0e0 0x562a7294aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56570/0 (socket says 172.21.15.43:56570) 2024-04-21T08:40:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.104+0000 7f6551605640 1 -- 172.21.15.43:0/2660987330 learned_addr learned my addr 172.21.15.43:0/2660987330 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.104+0000 7f6551e06640 1 --2- 172.21.15.43:0/2660987330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a729428d0 0x562a7294a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.104+0000 7f6551605640 1 --2- 172.21.15.43:0/2660987330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a72941b60 0x562a72956cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.104+0000 7f6551605640 1 -- 172.21.15.43:0/2660987330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a72941b60 msgr2=0x562a72956cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.104+0000 7f6551605640 1 --2- 172.21.15.43:0/2660987330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a72941b60 0x562a72956cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.104+0000 7f6551605640 1 -- 172.21.15.43:0/2660987330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a729428d0 msgr2=0x562a7294a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.104+0000 7f6551605640 1 --2- 172.21.15.43:0/2660987330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a729428d0 0x562a7294a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.104+0000 7f6551605640 1 -- 172.21.15.43:0/2660987330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a72940e30 con 0x562a7294b0e0 2024-04-21T08:40:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.104+0000 7f6551605640 1 --2- 172.21.15.43:0/2660987330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a7294b0e0 0x562a7294aa50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f653c005bb0 tx=0x7f653c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f7a555ab34558c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.104+0000 7f65377fe640 1 -- 172.21.15.43:0/2660987330 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c005850 con 0x562a7294b0e0 2024-04-21T08:40:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.104+0000 7f65377fe640 1 -- 172.21.15.43:0/2660987330 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f653c003a30 con 0x562a7294b0e0 2024-04-21T08:40:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.104+0000 7f65377fe640 1 -- 172.21.15.43:0/2660987330 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c034440 con 0x562a7294b0e0 2024-04-21T08:40:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.105+0000 7f6556c5d740 1 -- 172.21.15.43:0/2660987330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a7294b0e0 msgr2=0x562a7294aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.105+0000 7f6556c5d740 1 --2- 172.21.15.43:0/2660987330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a7294b0e0 0x562a7294aa50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f653c005bb0 tx=0x7f653c005fd0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.105+0000 7f6556c5d740 1 -- 172.21.15.43:0/2660987330 shutdown_connections 2024-04-21T08:40:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.105+0000 7f6556c5d740 1 --2- 172.21.15.43:0/2660987330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a7294b0e0 0x562a7294aa50 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.105+0000 7f6556c5d740 1 --2- 172.21.15.43:0/2660987330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a729428d0 0x562a7294a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.105+0000 7f6556c5d740 1 --2- 172.21.15.43:0/2660987330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a72941b60 0x562a72956cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.105+0000 7f6556c5d740 1 -- 172.21.15.43:0/2660987330 >> 172.21.15.43:0/2660987330 conn(0x562a72908c90 msgr2=0x562a7293c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.105+0000 7f6556c5d740 1 -- 172.21.15.43:0/2660987330 shutdown_connections 2024-04-21T08:40:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.105+0000 7f6556c5d740 1 -- 172.21.15.43:0/2660987330 wait complete. 2024-04-21T08:40:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.105+0000 7f6556c5d740 1 -- start start 2024-04-21T08:40:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.106+0000 7f6556c5d740 1 Processor -- start 2024-04-21T08:40:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.106+0000 7f6556c5d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a72941b60 0x562a729531c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.106+0000 7f6556c5d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a729428d0 0x562a72953700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.106+0000 7f6556c5d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a7294b0e0 0x562a72950190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.106+0000 7f6556c5d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f654c1b6d50 con 0x562a72941b60 2024-04-21T08:40:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.106+0000 7f6556c5d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f654c1b6680 con 0x562a7294b0e0 2024-04-21T08:40:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.106+0000 7f6556c5d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f654c1b38c0 con 0x562a729428d0 2024-04-21T08:40:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.106+0000 7f6556c5d740 10 client.0 _ll_register_callbacks cb 0x7f654800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.106+0000 7f6551605640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a72941b60 0x562a729531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.106+0000 7f6551605640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a72941b60 0x562a729531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56576/0 (socket says 172.21.15.43:56576) 2024-04-21T08:40:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.106+0000 7f6551605640 1 -- 172.21.15.43:0/661251615 learned_addr learned my addr 172.21.15.43:0/661251615 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.106+0000 7f6551605640 1 --2- 172.21.15.43:0/661251615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a7294b0e0 0x562a72950190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.106+0000 7f6551e06640 1 --2- 172.21.15.43:0/661251615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a729428d0 0x562a72953700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.107+0000 7f6551605640 1 -- 172.21.15.43:0/661251615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a729428d0 msgr2=0x562a72953700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.107+0000 7f6551605640 1 --2- 172.21.15.43:0/661251615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a729428d0 0x562a72953700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.107+0000 7f6551605640 1 -- 172.21.15.43:0/661251615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a7294b0e0 msgr2=0x562a72950190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.107+0000 7f6551605640 1 --2- 172.21.15.43:0/661251615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a7294b0e0 0x562a72950190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.107+0000 7f6551605640 1 -- 172.21.15.43:0/661251615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a729507d0 con 0x562a72941b60 2024-04-21T08:40:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.107+0000 7f6551605640 1 --2- 172.21.15.43:0/661251615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a72941b60 0x562a729531c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f653c000c00 tx=0x7f653c02ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9309e4ec913af85 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.107+0000 7f65357fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.107+0000 7f65357fa640 1 -- 172.21.15.43:0/661251615 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c034650 con 0x562a72941b60 2024-04-21T08:40:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.107+0000 7f65357fa640 1 -- 172.21.15.43:0/661251615 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f653c043450 con 0x562a72941b60 2024-04-21T08:40:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.107+0000 7f6556c5d740 1 -- 172.21.15.43:0/661251615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f653c003a30 con 0x562a72941b60 2024-04-21T08:40:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.107+0000 7f65357fa640 1 -- 172.21.15.43:0/661251615 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f653c043650 con 0x562a72941b60 2024-04-21T08:40:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.107+0000 7f6556c5d740 10 client.16449 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.107+0000 7f6556c5d740 1 -- 172.21.15.43:0/661251615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562a729c1a40 con 0x562a72941b60 2024-04-21T08:40:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.107+0000 7f65357fa640 1 -- 172.21.15.43:0/661251615 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f653c003a30 con 0x562a72941b60 2024-04-21T08:40:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.107+0000 7f65357fa640 1 -- 172.21.15.43:0/661251615 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f653c005020 con 0x562a72941b60 2024-04-21T08:40:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.107+0000 7f6556c5d740 10 client.16449 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.107+0000 7f6556c5d740 10 client.16449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.107+0000 7f6556c5d740 20 client.16449 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.108+0000 7f6556c5d740 1 --2- 172.21.15.43:0/661251615 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562a729c1db0 0x562a729c41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.108+0000 7f6556c5d740 4 client.16449 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:40:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.108+0000 7f6556c5d740 1 -- 172.21.15.43:0/661251615 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a728463d0 con 0x562a729c1db0 2024-04-21T08:40:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.108+0000 7f6551e06640 1 --2- 172.21.15.43:0/661251615 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562a729c1db0 0x562a729c41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.108+0000 7f6551e06640 1 --2- 172.21.15.43:0/661251615 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562a729c1db0 0x562a729c41d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bebcc929bd8574fb server_cookie=bf0b83f09ddb47d2 in_seq=0 out_seq=0 2024-04-21T08:40:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.108+0000 7f65357fa640 10 client.16449 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.123+0000 7f65357fa640 1 -- 172.21.15.43:0/661251615 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562a728463d0 con 0x562a729c1db0 2024-04-21T08:40:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.123+0000 7f65357fa640 10 client.16449 handle_command_reply: tid=0 2024-04-21T08:40:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.123+0000 7f6556c5d740 1 client.16449 shutdown 2024-04-21T08:40:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.123+0000 7f6556c5d740 1 -- 172.21.15.43:0/661251615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a72941b60 msgr2=0x562a729531c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.123+0000 7f6556c5d740 1 --2- 172.21.15.43:0/661251615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a72941b60 0x562a729531c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f653c000c00 tx=0x7f653c02ccc0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.123+0000 7f6556c5d740 1 -- 172.21.15.43:0/661251615 shutdown_connections 2024-04-21T08:40:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.123+0000 7f6556c5d740 1 --2- 172.21.15.43:0/661251615 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562a729c1db0 0x562a729c41d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.123+0000 7f6556c5d740 1 --2- 172.21.15.43:0/661251615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a7294b0e0 0x562a72950190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.123+0000 7f6556c5d740 1 --2- 172.21.15.43:0/661251615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a729428d0 0x562a72953700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.123+0000 7f6556c5d740 1 --2- 172.21.15.43:0/661251615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a72941b60 0x562a729531c0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.123+0000 7f65357fa640 0 client.16449 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.123+0000 7f6556c5d740 1 -- 172.21.15.43:0/661251615 >> 172.21.15.43:0/661251615 conn(0x562a72908c90 msgr2=0x562a7293b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.123+0000 7f6556c5d740 1 -- 172.21.15.43:0/661251615 shutdown_connections 2024-04-21T08:40:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.123+0000 7f6556c5d740 1 -- 172.21.15.43:0/661251615 wait complete. 2024-04-21T08:40:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.123+0000 7f6556c5d740 20 client.16449 trim_cache size 0 max 16384 2024-04-21T08:40:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:40:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:40:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:40:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:40:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.124+0000 7f655406a640 1 -- 172.21.15.43:0/3793803945 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f65380a3dc0 msgr2=0x7f65380a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.124+0000 7f655406a640 1 --2- 172.21.15.43:0/3793803945 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f65380a3dc0 0x7f65380a6280 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f654409c470 tx=0x7f654406df20 comp rx=0 tx=0).stop 2024-04-21T08:40:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.124+0000 7f655406a640 1 -- 172.21.15.43:0/3793803945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f654c1063f0 msgr2=0x7f654c19eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.124+0000 7f655406a640 1 --2- 172.21.15.43:0/3793803945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f654c1063f0 0x7f654c19eab0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6544009fd0 tx=0x7f654400b260 comp rx=0 tx=0).stop 2024-04-21T08:40:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.124+0000 7f655406a640 1 -- 172.21.15.43:0/3793803945 shutdown_connections 2024-04-21T08:40:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.124+0000 7f655406a640 1 --2- 172.21.15.43:0/3793803945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f654c1063f0 0x7f654c19eab0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.124+0000 7f655406a640 1 --2- 172.21.15.43:0/3793803945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f654c101910 0x7f654c1a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.124+0000 7f655406a640 1 --2- 172.21.15.43:0/3793803945 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f65380a3dc0 0x7f65380a6280 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.124+0000 7f655406a640 1 --2- 172.21.15.43:0/3793803945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f654c100ff0 0x7f654c1a4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.124+0000 7f655406a640 1 -- 172.21.15.43:0/3793803945 >> 172.21.15.43:0/3793803945 conn(0x7f654c0771d0 msgr2=0x7f654c103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.124+0000 7f655406a640 1 -- 172.21.15.43:0/3793803945 shutdown_connections 2024-04-21T08:40:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:34.124+0000 7f655406a640 1 -- 172.21.15.43:0/3793803945 wait complete. 2024-04-21T08:40:34.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:33 smithi175 ceph-mon[31920]: pgmap v52: 129 pgs: 129 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-21T08:40:34.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:33 smithi112 ceph-mon[31085]: pgmap v52: 129 pgs: 129 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-21T08:40:34.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:33 smithi043 ceph-mon[27978]: pgmap v52: 129 pgs: 129 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-21T08:40:36.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:40:36.162 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:40:36.213 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:35 smithi175 ceph-mon[31920]: pgmap v53: 129 pgs: 129 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-21T08:40:36.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.282+0000 7fd1a3111640 1 -- 172.21.15.43:0/2630135262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19c1021a0 msgr2=0x7fd19c102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.282+0000 7fd1a3111640 1 --2- 172.21.15.43:0/2630135262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19c1021a0 0x7fd19c102580 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd194009ca0 tx=0x7fd19402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.282+0000 7fd1a3111640 1 -- 172.21.15.43:0/2630135262 shutdown_connections 2024-04-21T08:40:36.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.282+0000 7fd1a3111640 1 --2- 172.21.15.43:0/2630135262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd19c107270 0x7fd19c10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.282+0000 7fd1a3111640 1 --2- 172.21.15.43:0/2630135262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19c102b50 0x7fd19c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.282+0000 7fd1a3111640 1 --2- 172.21.15.43:0/2630135262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19c1021a0 0x7fd19c102580 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.282+0000 7fd1a3111640 1 -- 172.21.15.43:0/2630135262 >> 172.21.15.43:0/2630135262 conn(0x7fd19c0793e0 msgr2=0x7fd19c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:36.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.282+0000 7fd1a3111640 1 -- 172.21.15.43:0/2630135262 shutdown_connections 2024-04-21T08:40:36.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.282+0000 7fd1a3111640 1 -- 172.21.15.43:0/2630135262 wait complete. 2024-04-21T08:40:36.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.283+0000 7fd1a3111640 1 Processor -- start 2024-04-21T08:40:36.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.283+0000 7fd1a3111640 1 -- start start 2024-04-21T08:40:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.283+0000 7fd1a3111640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd19c1021a0 0x7fd19c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.283+0000 7fd1a3111640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19c107270 0x7fd19c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.283+0000 7fd1a3111640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19c19cfb0 0x7fd19c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.283+0000 7fd1a3111640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1121d0 con 0x7fd19c107270 2024-04-21T08:40:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.283+0000 7fd1a3111640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c112050 con 0x7fd19c1021a0 2024-04-21T08:40:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.283+0000 7fd1a3111640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c112350 con 0x7fd19c19cfb0 2024-04-21T08:40:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.283+0000 7fd19bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19c107270 0x7fd19c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.283+0000 7fd19bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19c107270 0x7fd19c198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56604/0 (socket says 172.21.15.43:56604) 2024-04-21T08:40:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.283+0000 7fd19bfff640 1 -- 172.21.15.43:0/1034128534 learned_addr learned my addr 172.21.15.43:0/1034128534 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.284+0000 7fd1a0ead640 1 --2- 172.21.15.43:0/1034128534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd19c1021a0 0x7fd19c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.284+0000 7fd1a16ae640 1 --2- 172.21.15.43:0/1034128534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19c19cfb0 0x7fd19c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.284+0000 7fd19bfff640 1 -- 172.21.15.43:0/1034128534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19c19cfb0 msgr2=0x7fd19c19d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.284+0000 7fd19bfff640 1 --2- 172.21.15.43:0/1034128534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19c19cfb0 0x7fd19c19d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.284+0000 7fd19bfff640 1 -- 172.21.15.43:0/1034128534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd19c1021a0 msgr2=0x7fd19c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.284+0000 7fd19bfff640 1 --2- 172.21.15.43:0/1034128534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd19c1021a0 0x7fd19c1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.284+0000 7fd19bfff640 1 -- 172.21.15.43:0/1034128534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19c19daa0 con 0x7fd19c107270 2024-04-21T08:40:36.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.284+0000 7fd19bfff640 1 --2- 172.21.15.43:0/1034128534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19c107270 0x7fd19c198c30 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd18400c970 tx=0x7fd18400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:36.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.284+0000 7fd199ffb640 1 -- 172.21.15.43:0/1034128534 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184007bb0 con 0x7fd19c107270 2024-04-21T08:40:36.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.284+0000 7fd199ffb640 1 -- 172.21.15.43:0/1034128534 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd184007d50 con 0x7fd19c107270 2024-04-21T08:40:36.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.284+0000 7fd1a3111640 1 -- 172.21.15.43:0/1034128534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19c19dd90 con 0x7fd19c107270 2024-04-21T08:40:36.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.284+0000 7fd199ffb640 1 -- 172.21.15.43:0/1034128534 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184018730 con 0x7fd19c107270 2024-04-21T08:40:36.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.284+0000 7fd1a3111640 1 -- 172.21.15.43:0/1034128534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd19c1a5530 con 0x7fd19c107270 2024-04-21T08:40:36.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.286+0000 7fd199ffb640 1 -- 172.21.15.43:0/1034128534 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd18400f990 con 0x7fd19c107270 2024-04-21T08:40:36.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.287+0000 7fd1a5dd7740 1 Processor -- start 2024-04-21T08:40:36.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.287+0000 7fd1a5dd7740 1 -- start start 2024-04-21T08:40:36.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.287+0000 7fd199ffb640 1 --2- 172.21.15.43:0/1034128534 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd1880a3d20 0x7fd1880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.287+0000 7fd1a5dd7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06e8230 0x55eeb06e8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.287+0000 7fd1a5dd7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeb06e8b50 0x55eeb06f5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.287+0000 7fd1a16ae640 1 --2- 172.21.15.43:0/1034128534 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd1880a3d20 0x7fd1880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.287+0000 7fd199ffb640 1 -- 172.21.15.43:0/1034128534 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd1840d0d30 con 0x7fd19c107270 2024-04-21T08:40:36.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.287+0000 7fd1a5dd7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeb06f5e90 0x55eeb06f8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.287+0000 7fd1a5dd7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55eeb064d1e0 con 0x55eeb06e8230 2024-04-21T08:40:36.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.287+0000 7fd1a5dd7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55eeb05ea390 con 0x55eeb06e8b50 2024-04-21T08:40:36.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.287+0000 7fd1a5dd7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55eeb06491a0 con 0x55eeb06f5e90 2024-04-21T08:40:36.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.287+0000 7fd1a0ead640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06e8230 0x55eeb06e8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.287+0000 7fd1a0ead640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06e8230 0x55eeb06e8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56610/0 (socket says 172.21.15.43:56610) 2024-04-21T08:40:36.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.287+0000 7fd1a0ead640 1 -- 172.21.15.43:0/2481551356 learned_addr learned my addr 172.21.15.43:0/2481551356 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:36.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.288+0000 7fd19bfff640 1 --2- 172.21.15.43:0/2481551356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeb06e8b50 0x55eeb06f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.288+0000 7fd1a16ae640 1 --2- 172.21.15.43:0/2481551356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeb06f5e90 0x55eeb06f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.288+0000 7fd1a0ead640 1 -- 172.21.15.43:0/2481551356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeb06f5e90 msgr2=0x55eeb06f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:36.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.288+0000 7fd1a0ead640 1 --2- 172.21.15.43:0/2481551356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeb06f5e90 0x55eeb06f8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.288+0000 7fd1a0ead640 1 -- 172.21.15.43:0/2481551356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeb06e8b50 msgr2=0x55eeb06f5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:36.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.288+0000 7fd1a0ead640 1 --2- 172.21.15.43:0/2481551356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeb06e8b50 0x55eeb06f5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.288+0000 7fd1a0ead640 1 -- 172.21.15.43:0/2481551356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeb04297d0 con 0x55eeb06e8230 2024-04-21T08:40:36.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.288+0000 7fd1a16ae640 1 --2- 172.21.15.43:0/2481551356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeb06f5e90 0x55eeb06f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:36.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.288+0000 7fd19bfff640 1 --2- 172.21.15.43:0/2481551356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeb06e8b50 0x55eeb06f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:36.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.288+0000 7fd1a16ae640 1 --2- 172.21.15.43:0/1034128534 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd1880a3d20 0x7fd1880a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd18c005fd0 tx=0x7fd18c00b510 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:36.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.288+0000 7fd1a0ead640 1 --2- 172.21.15.43:0/2481551356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06e8230 0x55eeb06e8610 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd194009ca0 tx=0x7fd1940099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2016e5747e0953be server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:36.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.288+0000 7fd192ffd640 1 -- 172.21.15.43:0/2481551356 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194043020 con 0x55eeb06e8230 2024-04-21T08:40:36.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.288+0000 7fd192ffd640 1 -- 172.21.15.43:0/2481551356 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd194032970 con 0x55eeb06e8230 2024-04-21T08:40:36.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.288+0000 7fd192ffd640 1 -- 172.21.15.43:0/2481551356 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194039040 con 0x55eeb06e8230 2024-04-21T08:40:36.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.289+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/2481551356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06e8230 msgr2=0x55eeb06e8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.289+0000 7fd1a5dd7740 1 --2- 172.21.15.43:0/2481551356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06e8230 0x55eeb06e8610 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd194009ca0 tx=0x7fd1940099b0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.289+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/2481551356 shutdown_connections 2024-04-21T08:40:36.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.289+0000 7fd1a5dd7740 1 --2- 172.21.15.43:0/2481551356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeb06f5e90 0x55eeb06f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.289+0000 7fd1a5dd7740 1 --2- 172.21.15.43:0/2481551356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeb06e8b50 0x55eeb06f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.289+0000 7fd1a5dd7740 1 --2- 172.21.15.43:0/2481551356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06e8230 0x55eeb06e8610 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.289+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/2481551356 >> 172.21.15.43:0/2481551356 conn(0x55eeb06e2cb0 msgr2=0x55eeb06e3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:36.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.289+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/2481551356 shutdown_connections 2024-04-21T08:40:36.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.289+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/2481551356 wait complete. 2024-04-21T08:40:36.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.289+0000 7fd1a5dd7740 1 -- start start 2024-04-21T08:40:36.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.289+0000 7fd1a5dd7740 1 Processor -- start 2024-04-21T08:40:36.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.290+0000 7fd1a5dd7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06e8230 0x55eeb06f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.290+0000 7fd1a5dd7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeb06e8b50 0x55eeb06f3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.290+0000 7fd1a5dd7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeb06f5e90 0x55eeb06f3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.290+0000 7fd1a5dd7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55eeb06f8c70 con 0x55eeb06e8230 2024-04-21T08:40:36.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.290+0000 7fd1a5dd7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55eeb06062f0 con 0x55eeb06e8b50 2024-04-21T08:40:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.290+0000 7fd1a5dd7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55eeb06f8df0 con 0x55eeb06f5e90 2024-04-21T08:40:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.290+0000 7fd1a5dd7740 10 client.0 _ll_register_callbacks cb 0x55eeb0613580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.290+0000 7fd1a0ead640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06e8230 0x55eeb06f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.290+0000 7fd1a0ead640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06e8230 0x55eeb06f2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56628/0 (socket says 172.21.15.43:56628) 2024-04-21T08:40:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.290+0000 7fd1a0ead640 1 -- 172.21.15.43:0/2824858170 learned_addr learned my addr 172.21.15.43:0/2824858170 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.290+0000 7fd1a16ae640 1 --2- 172.21.15.43:0/2824858170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeb06f5e90 0x55eeb06f3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.290+0000 7fd19bfff640 1 --2- 172.21.15.43:0/2824858170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeb06e8b50 0x55eeb06f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.290+0000 7fd1a0ead640 1 -- 172.21.15.43:0/2824858170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeb06f5e90 msgr2=0x55eeb06f3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.290+0000 7fd1a0ead640 1 --2- 172.21.15.43:0/2824858170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeb06f5e90 0x55eeb06f3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.290+0000 7fd1a0ead640 1 -- 172.21.15.43:0/2824858170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeb06e8b50 msgr2=0x55eeb06f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.290+0000 7fd1a0ead640 1 --2- 172.21.15.43:0/2824858170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeb06e8b50 0x55eeb06f3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.290+0000 7fd1a0ead640 1 -- 172.21.15.43:0/2824858170 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeb035f3f0 con 0x55eeb06e8230 2024-04-21T08:40:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.290+0000 7fd1a0ead640 1 --2- 172.21.15.43:0/2824858170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06e8230 0x55eeb06f2ef0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd194006010 tx=0x7fd194032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a94b7ef097155 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.290+0000 7fd17bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.291+0000 7fd17bfff640 1 -- 172.21.15.43:0/2824858170 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19404c070 con 0x55eeb06e8230 2024-04-21T08:40:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.291+0000 7fd17bfff640 1 -- 172.21.15.43:0/2824858170 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd1940326c0 con 0x55eeb06e8230 2024-04-21T08:40:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.291+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/2824858170 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd194032970 con 0x55eeb06e8230 2024-04-21T08:40:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.291+0000 7fd17bfff640 1 -- 172.21.15.43:0/2824858170 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd194058050 con 0x55eeb06e8230 2024-04-21T08:40:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.291+0000 7fd1a5dd7740 10 client.16461 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.291+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/2824858170 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eeb031f0f0 con 0x55eeb06e8230 2024-04-21T08:40:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.291+0000 7fd17bfff640 1 -- 172.21.15.43:0/2824858170 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd194032970 con 0x55eeb06e8230 2024-04-21T08:40:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.291+0000 7fd17bfff640 1 -- 172.21.15.43:0/2824858170 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd194048570 con 0x55eeb06e8230 2024-04-21T08:40:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.291+0000 7fd1a5dd7740 10 client.16461 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.291+0000 7fd1a5dd7740 10 client.16461 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.291+0000 7fd1a5dd7740 20 client.16461 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.291+0000 7fd1a5dd7740 1 --2- 172.21.15.43:0/2824858170 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55eeb0705d10 0x55eeb0708100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.291+0000 7fd1a5dd7740 4 client.16461 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.291+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/2824858170 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eeb065ac00 con 0x55eeb0705d10 2024-04-21T08:40:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.292+0000 7fd19bfff640 1 --2- 172.21.15.43:0/2824858170 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55eeb0705d10 0x55eeb0708100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.292+0000 7fd19bfff640 1 --2- 172.21.15.43:0/2824858170 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55eeb0705d10 0x55eeb0708100 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=934a9681be966a35 server_cookie=f1e93e1048322c8c in_seq=0 out_seq=0 2024-04-21T08:40:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.292+0000 7fd17bfff640 10 client.16461 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.293+0000 7fd17bfff640 1 -- 172.21.15.43:0/2824858170 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eeb065ac00 con 0x55eeb0705d10 2024-04-21T08:40:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.293+0000 7fd17bfff640 10 client.16461 handle_command_reply: tid=0 2024-04-21T08:40:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.293+0000 7fd1a5dd7740 1 client.16461 shutdown 2024-04-21T08:40:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.293+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/2824858170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06e8230 msgr2=0x55eeb06f2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.293+0000 7fd1a5dd7740 1 --2- 172.21.15.43:0/2824858170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06e8230 0x55eeb06f2ef0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd194006010 tx=0x7fd194032fc0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.293+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/2824858170 shutdown_connections 2024-04-21T08:40:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.293+0000 7fd1a5dd7740 1 --2- 172.21.15.43:0/2824858170 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55eeb0705d10 0x55eeb0708100 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.293+0000 7fd1a5dd7740 1 --2- 172.21.15.43:0/2824858170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeb06f5e90 0x55eeb06f3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.293+0000 7fd1a5dd7740 1 --2- 172.21.15.43:0/2824858170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeb06e8b50 0x55eeb06f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.293+0000 7fd1a5dd7740 1 --2- 172.21.15.43:0/2824858170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06e8230 0x55eeb06f2ef0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.293+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/2824858170 >> 172.21.15.43:0/2824858170 conn(0x55eeb06e2cb0 msgr2=0x55eeb06e4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.293+0000 7fd17bfff640 0 client.16461 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.293+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/2824858170 shutdown_connections 2024-04-21T08:40:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.293+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/2824858170 wait complete. 2024-04-21T08:40:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.293+0000 7fd1a5dd7740 20 client.16461 trim_cache size 0 max 16384 2024-04-21T08:40:36.299 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:35 smithi112 ceph-mon[31085]: pgmap v53: 129 pgs: 129 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-21T08:40:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.311+0000 7fd1a5dd7740 1 Processor -- start 2024-04-21T08:40:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.311+0000 7fd1a5dd7740 1 -- start start 2024-04-21T08:40:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.311+0000 7fd1a5dd7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeb06e8230 0x55eeb06f5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.311+0000 7fd1a5dd7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeb06e8b50 0x55eeb0701ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.311+0000 7fd1a5dd7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06f5e90 0x55eeb070d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.311+0000 7fd1a5dd7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1ad0d0 con 0x55eeb06f5e90 2024-04-21T08:40:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.311+0000 7fd1a5dd7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1aec10 con 0x55eeb06e8230 2024-04-21T08:40:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.311+0000 7fd1a5dd7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1ad7a0 con 0x55eeb06e8b50 2024-04-21T08:40:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.311+0000 7fd1a16ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06f5e90 0x55eeb070d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.311+0000 7fd1a16ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06f5e90 0x55eeb070d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56630/0 (socket says 172.21.15.43:56630) 2024-04-21T08:40:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.311+0000 7fd1a16ae640 1 -- 172.21.15.43:0/1451056121 learned_addr learned my addr 172.21.15.43:0/1451056121 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.311+0000 7fd1a0ead640 1 --2- 172.21.15.43:0/1451056121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeb06e8230 0x55eeb06f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.311+0000 7fd19bfff640 1 --2- 172.21.15.43:0/1451056121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeb06e8b50 0x55eeb0701ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.311+0000 7fd1a16ae640 1 -- 172.21.15.43:0/1451056121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeb06e8b50 msgr2=0x55eeb0701ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.311+0000 7fd1a16ae640 1 --2- 172.21.15.43:0/1451056121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeb06e8b50 0x55eeb0701ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.311+0000 7fd1a16ae640 1 -- 172.21.15.43:0/1451056121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeb06e8230 msgr2=0x55eeb06f5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.311+0000 7fd1a16ae640 1 --2- 172.21.15.43:0/1451056121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeb06e8230 0x55eeb06f5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.311+0000 7fd1a16ae640 1 -- 172.21.15.43:0/1451056121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeb06e8980 con 0x55eeb06f5e90 2024-04-21T08:40:36.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.312+0000 7fd1a16ae640 1 --2- 172.21.15.43:0/1451056121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06f5e90 0x55eeb070d700 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd18c011f70 tx=0x7fd18c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe1229363c836a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:36.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.312+0000 7fd1927fc640 1 -- 172.21.15.43:0/1451056121 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c029670 con 0x55eeb06f5e90 2024-04-21T08:40:36.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.312+0000 7fd1927fc640 1 -- 172.21.15.43:0/1451056121 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd18c024070 con 0x55eeb06f5e90 2024-04-21T08:40:36.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.312+0000 7fd1927fc640 1 -- 172.21.15.43:0/1451056121 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c020ab0 con 0x55eeb06f5e90 2024-04-21T08:40:36.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:35 smithi043 ceph-mon[27978]: pgmap v53: 129 pgs: 129 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-21T08:40:36.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.313+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/1451056121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06f5e90 msgr2=0x55eeb070d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.313+0000 7fd1a5dd7740 1 --2- 172.21.15.43:0/1451056121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06f5e90 0x55eeb070d700 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd18c011f70 tx=0x7fd18c014000 comp rx=0 tx=0).stop 2024-04-21T08:40:36.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.313+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/1451056121 shutdown_connections 2024-04-21T08:40:36.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.313+0000 7fd1a5dd7740 1 --2- 172.21.15.43:0/1451056121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06f5e90 0x55eeb070d700 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.313+0000 7fd1a5dd7740 1 --2- 172.21.15.43:0/1451056121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeb06e8b50 0x55eeb0701ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.313+0000 7fd1a5dd7740 1 --2- 172.21.15.43:0/1451056121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeb06e8230 0x55eeb06f5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.313+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/1451056121 >> 172.21.15.43:0/1451056121 conn(0x55eeb06e2cb0 msgr2=0x55eeb06e4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:36.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.313+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/1451056121 shutdown_connections 2024-04-21T08:40:36.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.313+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/1451056121 wait complete. 2024-04-21T08:40:36.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.313+0000 7fd1a5dd7740 1 -- start start 2024-04-21T08:40:36.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.313+0000 7fd1a5dd7740 1 Processor -- start 2024-04-21T08:40:36.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.314+0000 7fd1a5dd7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeb06e8230 0x55eeb06fa220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.314+0000 7fd1a5dd7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeb06e8b50 0x55eeb06fa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.314+0000 7fd1a5dd7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06f5e90 0x55eeb076af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.314+0000 7fd1a5dd7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1ae540 con 0x55eeb06f5e90 2024-04-21T08:40:36.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.314+0000 7fd1a5dd7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1ade70 con 0x55eeb06e8230 2024-04-21T08:40:36.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.314+0000 7fd1a5dd7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1ab0b0 con 0x55eeb06e8b50 2024-04-21T08:40:36.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.314+0000 7fd1a5dd7740 10 client.0 _ll_register_callbacks cb 0x7fd194009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:36.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.314+0000 7fd1a16ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06f5e90 0x55eeb076af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.314+0000 7fd1a16ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06f5e90 0x55eeb076af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56634/0 (socket says 172.21.15.43:56634) 2024-04-21T08:40:36.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.314+0000 7fd1a16ae640 1 -- 172.21.15.43:0/1115878851 learned_addr learned my addr 172.21.15.43:0/1115878851 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:36.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.314+0000 7fd19bfff640 1 --2- 172.21.15.43:0/1115878851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeb06e8b50 0x55eeb06fa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.314+0000 7fd1a0ead640 1 --2- 172.21.15.43:0/1115878851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeb06e8230 0x55eeb06fa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.314+0000 7fd1a16ae640 1 -- 172.21.15.43:0/1115878851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeb06e8b50 msgr2=0x55eeb06fa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:36.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.314+0000 7fd1a16ae640 1 --2- 172.21.15.43:0/1115878851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeb06e8b50 0x55eeb06fa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.314+0000 7fd1a16ae640 1 -- 172.21.15.43:0/1115878851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeb06e8230 msgr2=0x55eeb06fa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:36.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.314+0000 7fd1a16ae640 1 --2- 172.21.15.43:0/1115878851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeb06e8230 0x55eeb06fa220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.314+0000 7fd1a16ae640 1 -- 172.21.15.43:0/1115878851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeb076b4e0 con 0x55eeb06f5e90 2024-04-21T08:40:36.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.314+0000 7fd19bfff640 1 --2- 172.21.15.43:0/1115878851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeb06e8b50 0x55eeb06fa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:36.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.314+0000 7fd1a16ae640 1 --2- 172.21.15.43:0/1115878851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06f5e90 0x55eeb076af70 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00b830 tx=0x7fd18c012f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6877d3cb843851e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.314+0000 7fd17bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.314+0000 7fd17bfff640 1 -- 172.21.15.43:0/1115878851 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c0209c0 con 0x55eeb06f5e90 2024-04-21T08:40:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.314+0000 7fd17bfff640 1 -- 172.21.15.43:0/1115878851 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd18c020b60 con 0x55eeb06f5e90 2024-04-21T08:40:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.314+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/1115878851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd18c024070 con 0x55eeb06f5e90 2024-04-21T08:40:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.315+0000 7fd17bfff640 1 -- 172.21.15.43:0/1115878851 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd18c020d60 con 0x55eeb06f5e90 2024-04-21T08:40:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.315+0000 7fd1a5dd7740 10 client.16467 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.315+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/1115878851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eeb076c7f0 con 0x55eeb06f5e90 2024-04-21T08:40:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.315+0000 7fd17bfff640 1 -- 172.21.15.43:0/1115878851 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd18c024070 con 0x55eeb06f5e90 2024-04-21T08:40:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.315+0000 7fd17bfff640 1 -- 172.21.15.43:0/1115878851 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd18c035460 con 0x55eeb06f5e90 2024-04-21T08:40:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.315+0000 7fd1a5dd7740 10 client.16467 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.315+0000 7fd1a5dd7740 10 client.16467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.315+0000 7fd1a5dd7740 20 client.16467 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.315+0000 7fd1a5dd7740 1 --2- 172.21.15.43:0/1115878851 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55eeb076ce50 0x55eeb076f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.315+0000 7fd1a5dd7740 4 client.16467 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:40:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.315+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/1115878851 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eeb0654a70 con 0x55eeb076ce50 2024-04-21T08:40:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.315+0000 7fd1a0ead640 1 --2- 172.21.15.43:0/1115878851 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55eeb076ce50 0x55eeb076f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.316+0000 7fd1a0ead640 1 --2- 172.21.15.43:0/1115878851 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55eeb076ce50 0x55eeb076f270 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cf9a9e4821c5f1b server_cookie=ceeb203d2179ed5e in_seq=0 out_seq=0 2024-04-21T08:40:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.316+0000 7fd17bfff640 10 client.16467 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.316+0000 7fd17bfff640 1 -- 172.21.15.43:0/1115878851 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55eeb0654a70 con 0x55eeb076ce50 2024-04-21T08:40:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.316+0000 7fd17bfff640 10 client.16467 handle_command_reply: tid=0 2024-04-21T08:40:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.316+0000 7fd1a5dd7740 1 client.16467 shutdown 2024-04-21T08:40:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.316+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/1115878851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06f5e90 msgr2=0x55eeb076af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.316+0000 7fd1a5dd7740 1 --2- 172.21.15.43:0/1115878851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06f5e90 0x55eeb076af70 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00b830 tx=0x7fd18c012f70 comp rx=0 tx=0).stop 2024-04-21T08:40:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.316+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/1115878851 shutdown_connections 2024-04-21T08:40:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.316+0000 7fd1a5dd7740 1 --2- 172.21.15.43:0/1115878851 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55eeb076ce50 0x55eeb076f270 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.316+0000 7fd1a5dd7740 1 --2- 172.21.15.43:0/1115878851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeb06f5e90 0x55eeb076af70 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.316+0000 7fd1a5dd7740 1 --2- 172.21.15.43:0/1115878851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeb06e8b50 0x55eeb06fa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.316+0000 7fd17bfff640 0 client.16467 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.316+0000 7fd1a5dd7740 1 --2- 172.21.15.43:0/1115878851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeb06e8230 0x55eeb06fa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.316+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/1115878851 >> 172.21.15.43:0/1115878851 conn(0x55eeb06e2cb0 msgr2=0x55eeb06e14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:36.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.316+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/1115878851 shutdown_connections 2024-04-21T08:40:36.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.316+0000 7fd1a5dd7740 1 -- 172.21.15.43:0/1115878851 wait complete. 2024-04-21T08:40:36.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.316+0000 7fd1a5dd7740 20 client.16467 trim_cache size 0 max 16384 2024-04-21T08:40:36.322 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:40:36.322 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:40:36.322 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:40:36.322 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:40:36.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.317+0000 7fd1a3111640 1 -- 172.21.15.43:0/1034128534 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd1880a3d20 msgr2=0x7fd1880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.317+0000 7fd1a3111640 1 --2- 172.21.15.43:0/1034128534 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd1880a3d20 0x7fd1880a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd18c005fd0 tx=0x7fd18c00b510 comp rx=0 tx=0).stop 2024-04-21T08:40:36.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.317+0000 7fd1a3111640 1 -- 172.21.15.43:0/1034128534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19c107270 msgr2=0x7fd19c198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.317+0000 7fd1a3111640 1 --2- 172.21.15.43:0/1034128534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19c107270 0x7fd19c198c30 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd18400c970 tx=0x7fd18400ce30 comp rx=0 tx=0).stop 2024-04-21T08:40:36.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.317+0000 7fd1a3111640 1 -- 172.21.15.43:0/1034128534 shutdown_connections 2024-04-21T08:40:36.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.317+0000 7fd1a3111640 1 --2- 172.21.15.43:0/1034128534 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd1880a3d20 0x7fd1880a61e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.317+0000 7fd1a3111640 1 --2- 172.21.15.43:0/1034128534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19c19cfb0 0x7fd19c19d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.317+0000 7fd1a3111640 1 --2- 172.21.15.43:0/1034128534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19c107270 0x7fd19c198c30 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.317+0000 7fd1a3111640 1 --2- 172.21.15.43:0/1034128534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd19c1021a0 0x7fd19c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.317+0000 7fd1a3111640 1 -- 172.21.15.43:0/1034128534 >> 172.21.15.43:0/1034128534 conn(0x7fd19c0793e0 msgr2=0x7fd19c0ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:36.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.317+0000 7fd1a3111640 1 -- 172.21.15.43:0/1034128534 shutdown_connections 2024-04-21T08:40:36.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.317+0000 7fd1a3111640 1 -- 172.21.15.43:0/1034128534 wait complete. 2024-04-21T08:40:36.327 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:40:36.327 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:40:36.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.431+0000 7f05933c7640 1 -- 172.21.15.43:0/2323332885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f058c101b60 msgr2=0x7f058c10dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.431+0000 7f05933c7640 1 --2- 172.21.15.43:0/2323332885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f058c101b60 0x7f058c10dc90 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f0584009930 tx=0x7f058402bc30 comp rx=0 tx=0).stop 2024-04-21T08:40:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.431+0000 7f05933c7640 1 -- 172.21.15.43:0/2323332885 shutdown_connections 2024-04-21T08:40:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.431+0000 7f05933c7640 1 --2- 172.21.15.43:0/2323332885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f058c101b60 0x7f058c10dc90 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.431+0000 7f05933c7640 1 --2- 172.21.15.43:0/2323332885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f058c1011a0 0x7f058c101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.431+0000 7f05933c7640 1 --2- 172.21.15.43:0/2323332885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f058c105e30 0x7f058c106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.431+0000 7f05933c7640 1 -- 172.21.15.43:0/2323332885 >> 172.21.15.43:0/2323332885 conn(0x7f058c077e30 msgr2=0x7f058c078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.431+0000 7f05933c7640 1 -- 172.21.15.43:0/2323332885 shutdown_connections 2024-04-21T08:40:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.431+0000 7f05933c7640 1 -- 172.21.15.43:0/2323332885 wait complete. 2024-04-21T08:40:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.432+0000 7f05933c7640 1 Processor -- start 2024-04-21T08:40:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.432+0000 7f05933c7640 1 -- start start 2024-04-21T08:40:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.432+0000 7f05933c7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f058c1011a0 0x7f058c0feae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.432+0000 7f05933c7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f058c101b60 0x7f058c0ff020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.432+0000 7f05933c7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f058c105e30 0x7f058c0ff560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.432+0000 7f05933c7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f058c1110c0 con 0x7f058c105e30 2024-04-21T08:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.432+0000 7f05933c7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f058c110f40 con 0x7f058c101b60 2024-04-21T08:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.432+0000 7f05933c7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f058c111240 con 0x7f058c1011a0 2024-04-21T08:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.432+0000 7f0591964640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f058c105e30 0x7f058c0ff560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.432+0000 7f0591964640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f058c105e30 0x7f058c0ff560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56650/0 (socket says 172.21.15.43:56650) 2024-04-21T08:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.432+0000 7f0591964640 1 -- 172.21.15.43:0/3919334917 learned_addr learned my addr 172.21.15.43:0/3919334917 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.432+0000 7f0590962640 1 --2- 172.21.15.43:0/3919334917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f058c101b60 0x7f058c0ff020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.432+0000 7f0591163640 1 --2- 172.21.15.43:0/3919334917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f058c1011a0 0x7f058c0feae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.432+0000 7f0591964640 1 -- 172.21.15.43:0/3919334917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f058c1011a0 msgr2=0x7f058c0feae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.432+0000 7f0591964640 1 --2- 172.21.15.43:0/3919334917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f058c1011a0 0x7f058c0feae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.432+0000 7f0591964640 1 -- 172.21.15.43:0/3919334917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f058c101b60 msgr2=0x7f058c0ff020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.432+0000 7f0591964640 1 --2- 172.21.15.43:0/3919334917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f058c101b60 0x7f058c0ff020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.432+0000 7f0591964640 1 -- 172.21.15.43:0/3919334917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f058c1a6ae0 con 0x7f058c105e30 2024-04-21T08:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.432+0000 7f0590962640 1 --2- 172.21.15.43:0/3919334917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f058c101b60 0x7f058c0ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.433+0000 7f0591964640 1 --2- 172.21.15.43:0/3919334917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f058c105e30 0x7f058c0ff560 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f0584005940 tx=0x7f058402c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.433+0000 7f05827fc640 1 -- 172.21.15.43:0/3919334917 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0584043020 con 0x7f058c105e30 2024-04-21T08:40:36.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.433+0000 7f05827fc640 1 -- 172.21.15.43:0/3919334917 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05840043d0 con 0x7f058c105e30 2024-04-21T08:40:36.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.433+0000 7f05933c7640 1 -- 172.21.15.43:0/3919334917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f058c1a6d70 con 0x7f058c105e30 2024-04-21T08:40:36.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.433+0000 7f05827fc640 1 -- 172.21.15.43:0/3919334917 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0584003b50 con 0x7f058c105e30 2024-04-21T08:40:36.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.433+0000 7f05933c7640 1 -- 172.21.15.43:0/3919334917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f058c1a72c0 con 0x7f058c105e30 2024-04-21T08:40:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.434+0000 7f05933c7640 1 -- 172.21.15.43:0/3919334917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f058c068de0 con 0x7f058c105e30 2024-04-21T08:40:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.435+0000 7f05827fc640 1 -- 172.21.15.43:0/3919334917 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0584046020 con 0x7f058c105e30 2024-04-21T08:40:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.436+0000 7f05827fc640 1 --2- 172.21.15.43:0/3919334917 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f05780a3d50 0x7f05780a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.436+0000 7f0591163640 1 --2- 172.21.15.43:0/3919334917 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f05780a3d50 0x7f05780a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.436+0000 7f05827fc640 1 -- 172.21.15.43:0/3919334917 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f05840f3170 con 0x7f058c105e30 2024-04-21T08:40:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.436+0000 7f0591163640 1 --2- 172.21.15.43:0/3919334917 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f05780a3d50 0x7f05780a6210 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0588007d50 tx=0x7f0588007730 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:36.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.439+0000 7f05827fc640 1 -- 172.21.15.43:0/3919334917 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f058404d020 con 0x7f058c105e30 2024-04-21T08:40:36.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.667+0000 7f05933c7640 1 -- 172.21.15.43:0/3919334917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f058c10a360 con 0x7f058c105e30 2024-04-21T08:40:36.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.668+0000 7f05827fc640 1 -- 172.21.15.43:0/3919334917 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7f05840bf550 con 0x7f058c105e30 2024-04-21T08:40:36.667 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:40:36.667 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:40:36.667 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:40:36.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.670+0000 7f05933c7640 1 -- 172.21.15.43:0/3919334917 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f05780a3d50 msgr2=0x7f05780a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.670+0000 7f05933c7640 1 --2- 172.21.15.43:0/3919334917 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f05780a3d50 0x7f05780a6210 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0588007d50 tx=0x7f0588007730 comp rx=0 tx=0).stop 2024-04-21T08:40:36.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.670+0000 7f05933c7640 1 -- 172.21.15.43:0/3919334917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f058c105e30 msgr2=0x7f058c0ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.670+0000 7f05933c7640 1 --2- 172.21.15.43:0/3919334917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f058c105e30 0x7f058c0ff560 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f0584005940 tx=0x7f058402c710 comp rx=0 tx=0).stop 2024-04-21T08:40:36.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.670+0000 7f05933c7640 1 -- 172.21.15.43:0/3919334917 shutdown_connections 2024-04-21T08:40:36.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.670+0000 7f05933c7640 1 --2- 172.21.15.43:0/3919334917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f058c105e30 0x7f058c0ff560 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.670+0000 7f05933c7640 1 --2- 172.21.15.43:0/3919334917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f058c101b60 0x7f058c0ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.670+0000 7f05933c7640 1 --2- 172.21.15.43:0/3919334917 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f05780a3d50 0x7f05780a6210 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.670+0000 7f05933c7640 1 --2- 172.21.15.43:0/3919334917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f058c1011a0 0x7f058c0feae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.670+0000 7f05933c7640 1 -- 172.21.15.43:0/3919334917 >> 172.21.15.43:0/3919334917 conn(0x7f058c077e30 msgr2=0x7f058c069360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:36.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.670+0000 7f05933c7640 1 -- 172.21.15.43:0/3919334917 shutdown_connections 2024-04-21T08:40:36.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.670+0000 7f05933c7640 1 -- 172.21.15.43:0/3919334917 wait complete. 2024-04-21T08:40:36.680 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:40:36.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.786+0000 7ff4fd9f4640 1 -- 172.21.15.43:0/3845344232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4f8101a50 msgr2=0x7ff4f8105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.786+0000 7ff4fd9f4640 1 --2- 172.21.15.43:0/3845344232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4f8101a50 0x7ff4f8105a90 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0009900 tx=0x7ff4e002bc30 comp rx=0 tx=0).stop 2024-04-21T08:40:36.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.786+0000 7ff4fd9f4640 1 -- 172.21.15.43:0/3845344232 shutdown_connections 2024-04-21T08:40:36.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.786+0000 7ff4fd9f4640 1 --2- 172.21.15.43:0/3845344232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4f8106170 0x7ff4f810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.786+0000 7ff4fd9f4640 1 --2- 172.21.15.43:0/3845344232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4f8101a50 0x7ff4f8105a90 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.786+0000 7ff4fd9f4640 1 --2- 172.21.15.43:0/3845344232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4f81010a0 0x7ff4f8101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.786+0000 7ff4fd9f4640 1 -- 172.21.15.43:0/3845344232 >> 172.21.15.43:0/3845344232 conn(0x7ff4f8077eb0 msgr2=0x7ff4f80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:36.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.786+0000 7ff4fd9f4640 1 -- 172.21.15.43:0/3845344232 shutdown_connections 2024-04-21T08:40:36.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.786+0000 7ff4fd9f4640 1 -- 172.21.15.43:0/3845344232 wait complete. 2024-04-21T08:40:36.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.787+0000 7ff4fd9f4640 1 Processor -- start 2024-04-21T08:40:36.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.787+0000 7ff4fd9f4640 1 -- start start 2024-04-21T08:40:36.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.787+0000 7ff4fd9f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4f81010a0 0x7ff4f81a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.787+0000 7ff4fd9f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4f8106170 0x7ff4f81a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.787+0000 7ff4fd9f4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4f819a710 0x7ff4f819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.787+0000 7ff4fd9f4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f81110b0 con 0x7ff4f81010a0 2024-04-21T08:40:36.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.787+0000 7ff4fd9f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f8110f30 con 0x7ff4f8106170 2024-04-21T08:40:36.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.787+0000 7ff4fd9f4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f8111230 con 0x7ff4f819a710 2024-04-21T08:40:36.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.787+0000 7ff4f6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4f81010a0 0x7ff4f81a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.787+0000 7ff4f6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4f81010a0 0x7ff4f81a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50022/0 (socket says 172.21.15.43:50022) 2024-04-21T08:40:36.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.787+0000 7ff4f6ffd640 1 -- 172.21.15.43:0/2323438472 learned_addr learned my addr 172.21.15.43:0/2323438472 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:36.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.787+0000 7ff4effff640 1 --2- 172.21.15.43:0/2323438472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4f8106170 0x7ff4f81a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.787+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/2323438472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4f819a710 0x7ff4f819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.787+0000 7ff4f6ffd640 1 -- 172.21.15.43:0/2323438472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4f819a710 msgr2=0x7ff4f819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.787+0000 7ff4f6ffd640 1 --2- 172.21.15.43:0/2323438472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4f819a710 0x7ff4f819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.787+0000 7ff4f6ffd640 1 -- 172.21.15.43:0/2323438472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4f8106170 msgr2=0x7ff4f81a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.787+0000 7ff4f6ffd640 1 --2- 172.21.15.43:0/2323438472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4f8106170 0x7ff4f81a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.787+0000 7ff4f6ffd640 1 -- 172.21.15.43:0/2323438472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f819b310 con 0x7ff4f81010a0 2024-04-21T08:40:36.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.787+0000 7ff4effff640 1 --2- 172.21.15.43:0/2323438472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4f8106170 0x7ff4f81a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:36.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.787+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/2323438472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4f819a710 0x7ff4f819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:36.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.788+0000 7ff4f6ffd640 1 --2- 172.21.15.43:0/2323438472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4f81010a0 0x7ff4f81a0650 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff4e800c970 tx=0x7ff4e800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:36.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.788+0000 7ff4fc9f2640 1 -- 172.21.15.43:0/2323438472 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8007bb0 con 0x7ff4f81010a0 2024-04-21T08:40:36.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.788+0000 7ff4fc9f2640 1 -- 172.21.15.43:0/2323438472 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4e8007d50 con 0x7ff4f81010a0 2024-04-21T08:40:36.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.788+0000 7ff4fd9f4640 1 -- 172.21.15.43:0/2323438472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f81a73f0 con 0x7ff4f81010a0 2024-04-21T08:40:36.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.788+0000 7ff4fc9f2640 1 -- 172.21.15.43:0/2323438472 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e800f450 con 0x7ff4f81010a0 2024-04-21T08:40:36.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.788+0000 7ff4fd9f4640 1 -- 172.21.15.43:0/2323438472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f81a77a0 con 0x7ff4f81010a0 2024-04-21T08:40:36.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.790+0000 7ff4fc9f2640 1 -- 172.21.15.43:0/2323438472 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff4e8004f20 con 0x7ff4f81010a0 2024-04-21T08:40:36.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.790+0000 7ff5005eb740 1 Processor -- start 2024-04-21T08:40:36.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.790+0000 7ff5005eb740 1 -- start start 2024-04-21T08:40:36.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.791+0000 7ff5005eb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0ee44d0 0x5648e0ee48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.791+0000 7ff4fc9f2640 1 --2- 172.21.15.43:0/2323438472 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff4e40a3d20 0x7ff4e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.791+0000 7ff5005eb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648e0ee4e80 0x5648e0eed430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.791+0000 7ff5005eb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5648e0eeda00 0x5648e0eefe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.791+0000 7ff5005eb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5648e0e491f0 con 0x5648e0ee44d0 2024-04-21T08:40:36.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.791+0000 7ff5005eb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5648e0de64b0 con 0x5648e0eeda00 2024-04-21T08:40:36.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.791+0000 7ff5005eb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5648e0e451b0 con 0x5648e0ee4e80 2024-04-21T08:40:36.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.791+0000 7ff4f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0ee44d0 0x5648e0ee48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.791+0000 7ff4f6ffd640 1 --2- 172.21.15.43:0/2323438472 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff4e40a3d20 0x7ff4e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.791+0000 7ff4f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0ee44d0 0x5648e0ee48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50034/0 (socket says 172.21.15.43:50034) 2024-04-21T08:40:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.791+0000 7ff4f77fe640 1 -- 172.21.15.43:0/176178144 learned_addr learned my addr 172.21.15.43:0/176178144 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.791+0000 7ff4fc9f2640 1 -- 172.21.15.43:0/2323438472 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7ff4e80d0a40 con 0x7ff4f81010a0 2024-04-21T08:40:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.791+0000 7ff4effff640 1 --2- 172.21.15.43:0/176178144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648e0ee4e80 0x5648e0eed430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.791+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/176178144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5648e0eeda00 0x5648e0eefe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.791+0000 7ff4f77fe640 1 -- 172.21.15.43:0/176178144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648e0ee4e80 msgr2=0x5648e0eed430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.791+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/176178144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648e0ee4e80 0x5648e0eed430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.791+0000 7ff4f77fe640 1 -- 172.21.15.43:0/176178144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5648e0eeda00 msgr2=0x5648e0eefe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.791+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/176178144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5648e0eeda00 0x5648e0eefe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.791+0000 7ff4f77fe640 1 -- 172.21.15.43:0/176178144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648e0dd81b0 con 0x5648e0ee44d0 2024-04-21T08:40:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.791+0000 7ff4effff640 1 --2- 172.21.15.43:0/176178144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648e0ee4e80 0x5648e0eed430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.791+0000 7ff4f6ffd640 1 --2- 172.21.15.43:0/2323438472 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff4e40a3d20 0x7ff4e40a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff4e809b470 tx=0x7ff4e8004980 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.791+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/176178144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0ee44d0 0x5648e0ee48b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0008f20 tx=0x7ff4f00087c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d4f4dcd2dbe1ca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.791+0000 7ff4edffb640 1 -- 172.21.15.43:0/176178144 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0004b20 con 0x5648e0ee44d0 2024-04-21T08:40:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.792+0000 7ff4edffb640 1 -- 172.21.15.43:0/176178144 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4f000fd10 con 0x5648e0ee44d0 2024-04-21T08:40:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.792+0000 7ff4edffb640 1 -- 172.21.15.43:0/176178144 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f00053a0 con 0x5648e0ee44d0 2024-04-21T08:40:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.792+0000 7ff5005eb740 1 -- 172.21.15.43:0/176178144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0ee44d0 msgr2=0x5648e0ee48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.792+0000 7ff5005eb740 1 --2- 172.21.15.43:0/176178144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0ee44d0 0x5648e0ee48b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0008f20 tx=0x7ff4f00087c0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.792+0000 7ff5005eb740 1 -- 172.21.15.43:0/176178144 shutdown_connections 2024-04-21T08:40:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.792+0000 7ff5005eb740 1 --2- 172.21.15.43:0/176178144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5648e0eeda00 0x5648e0eefe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.792+0000 7ff5005eb740 1 --2- 172.21.15.43:0/176178144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648e0ee4e80 0x5648e0eed430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.792+0000 7ff5005eb740 1 --2- 172.21.15.43:0/176178144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0ee44d0 0x5648e0ee48b0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.792+0000 7ff5005eb740 1 -- 172.21.15.43:0/176178144 >> 172.21.15.43:0/176178144 conn(0x5648e0edeba0 msgr2=0x5648e0edef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.792+0000 7ff5005eb740 1 -- 172.21.15.43:0/176178144 shutdown_connections 2024-04-21T08:40:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.792+0000 7ff5005eb740 1 -- 172.21.15.43:0/176178144 wait complete. 2024-04-21T08:40:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.793+0000 7ff5005eb740 1 -- start start 2024-04-21T08:40:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.793+0000 7ff5005eb740 1 Processor -- start 2024-04-21T08:40:36.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.793+0000 7ff5005eb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0ee44d0 0x5648e0eed2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.793+0000 7ff5005eb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5648e0ee4e80 0x5648e0eea650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.793+0000 7ff5005eb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648e0eeda00 0x5648e0eeabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.793+0000 7ff5005eb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5648e0ef0970 con 0x5648e0ee44d0 2024-04-21T08:40:36.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.793+0000 7ff5005eb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5648e0e02500 con 0x5648e0ee4e80 2024-04-21T08:40:36.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.793+0000 7ff5005eb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5648e0ef0af0 con 0x5648e0eeda00 2024-04-21T08:40:36.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.793+0000 7ff5005eb740 10 client.0 _ll_register_callbacks cb 0x5648e0dd0b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:36.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.793+0000 7ff4f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0ee44d0 0x5648e0eed2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.793+0000 7ff4f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0ee44d0 0x5648e0eed2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50048/0 (socket says 172.21.15.43:50048) 2024-04-21T08:40:36.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.793+0000 7ff4f77fe640 1 -- 172.21.15.43:0/3136174520 learned_addr learned my addr 172.21.15.43:0/3136174520 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:36.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.793+0000 7ff4effff640 1 --2- 172.21.15.43:0/3136174520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5648e0ee4e80 0x5648e0eea650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.793+0000 7ff4f77fe640 1 -- 172.21.15.43:0/3136174520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648e0eeda00 msgr2=0x5648e0eeabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:36.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.793+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/3136174520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648e0eeda00 0x5648e0eeabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.793+0000 7ff4f77fe640 1 -- 172.21.15.43:0/3136174520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5648e0ee4e80 msgr2=0x5648e0eea650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:36.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.793+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/3136174520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5648e0ee4e80 0x5648e0eea650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.793+0000 7ff4f77fe640 1 -- 172.21.15.43:0/3136174520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648e0b5b3f0 con 0x5648e0ee44d0 2024-04-21T08:40:36.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.794+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/3136174520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0ee44d0 0x5648e0eed2c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0005f60 tx=0x7ff4f0014590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb03b0a8927aaaf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:36.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.794+0000 7ff4d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:36.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.794+0000 7ff4d6ffd640 1 -- 172.21.15.43:0/3136174520 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0024070 con 0x5648e0ee44d0 2024-04-21T08:40:36.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.794+0000 7ff4d6ffd640 1 -- 172.21.15.43:0/3136174520 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7ff4f000fa70 con 0x5648e0ee44d0 2024-04-21T08:40:36.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.794+0000 7ff5005eb740 1 -- 172.21.15.43:0/3136174520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4f000fd10 con 0x5648e0ee44d0 2024-04-21T08:40:36.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.794+0000 7ff4d6ffd640 1 -- 172.21.15.43:0/3136174520 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff4f0030050 con 0x5648e0ee44d0 2024-04-21T08:40:36.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.794+0000 7ff5005eb740 10 client.16485 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:36.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.794+0000 7ff5005eb740 1 -- 172.21.15.43:0/3136174520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5648e0c257d0 con 0x5648e0ee44d0 2024-04-21T08:40:36.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.794+0000 7ff4d6ffd640 1 -- 172.21.15.43:0/3136174520 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4f000fd10 con 0x5648e0ee44d0 2024-04-21T08:40:36.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.794+0000 7ff4d6ffd640 1 -- 172.21.15.43:0/3136174520 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff4f0012450 con 0x5648e0ee44d0 2024-04-21T08:40:36.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.794+0000 7ff5005eb740 10 client.16485 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.794+0000 7ff5005eb740 10 client.16485 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:40:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.795+0000 7ff5005eb740 20 client.16485 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.795+0000 7ff5005eb740 1 --2- 172.21.15.43:0/3136174520 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5648e0efdac0 0x5648e0effeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.795+0000 7ff5005eb740 4 client.16485 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.795+0000 7ff5005eb740 1 -- 172.21.15.43:0/3136174520 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648e0de8930 con 0x5648e0efdac0 2024-04-21T08:40:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.795+0000 7ff4effff640 1 --2- 172.21.15.43:0/3136174520 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5648e0efdac0 0x5648e0effeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.795+0000 7ff4d6ffd640 10 client.16485 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:40:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.795+0000 7ff4effff640 1 --2- 172.21.15.43:0/3136174520 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5648e0efdac0 0x5648e0effeb0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9780a5611fd2d7fc server_cookie=5a459dd737e649fd in_seq=0 out_seq=0 2024-04-21T08:40:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.796+0000 7ff4d6ffd640 1 -- 172.21.15.43:0/3136174520 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5648e0de8930 con 0x5648e0efdac0 2024-04-21T08:40:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.796+0000 7ff4d6ffd640 10 client.16485 handle_command_reply: tid=0 2024-04-21T08:40:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.796+0000 7ff5005eb740 1 client.16485 shutdown 2024-04-21T08:40:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.796+0000 7ff5005eb740 1 -- 172.21.15.43:0/3136174520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0ee44d0 msgr2=0x5648e0eed2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.796+0000 7ff5005eb740 1 --2- 172.21.15.43:0/3136174520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0ee44d0 0x5648e0eed2c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0005f60 tx=0x7ff4f0014590 comp rx=0 tx=0).stop 2024-04-21T08:40:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.796+0000 7ff5005eb740 1 -- 172.21.15.43:0/3136174520 shutdown_connections 2024-04-21T08:40:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.796+0000 7ff5005eb740 1 --2- 172.21.15.43:0/3136174520 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5648e0efdac0 0x5648e0effeb0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.796+0000 7ff5005eb740 1 --2- 172.21.15.43:0/3136174520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648e0eeda00 0x5648e0eeabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.796+0000 7ff5005eb740 1 --2- 172.21.15.43:0/3136174520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5648e0ee4e80 0x5648e0eea650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.796+0000 7ff4d6ffd640 0 client.16485 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:40:36.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.796+0000 7ff5005eb740 1 --2- 172.21.15.43:0/3136174520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0ee44d0 0x5648e0eed2c0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.796+0000 7ff5005eb740 1 -- 172.21.15.43:0/3136174520 >> 172.21.15.43:0/3136174520 conn(0x5648e0edeba0 msgr2=0x5648e0ee08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:36.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.797+0000 7ff5005eb740 1 -- 172.21.15.43:0/3136174520 shutdown_connections 2024-04-21T08:40:36.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.797+0000 7ff5005eb740 1 -- 172.21.15.43:0/3136174520 wait complete. 2024-04-21T08:40:36.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.797+0000 7ff5005eb740 20 client.16485 trim_cache size 0 max 16384 2024-04-21T08:40:36.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.810+0000 7ff5005eb740 1 Processor -- start 2024-04-21T08:40:36.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.810+0000 7ff5005eb740 1 -- start start 2024-04-21T08:40:36.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.810+0000 7ff5005eb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648e0ee44d0 0x5648e0ef9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.810+0000 7ff5005eb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5648e0ee4e80 0x5648e0ee97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.810+0000 7ff5005eb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0eeda00 0x5648e0ee06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.810+0000 7ff5005eb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f81b15e0 con 0x5648e0eeda00 2024-04-21T08:40:36.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.810+0000 7ff5005eb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f81b3120 con 0x5648e0ee4e80 2024-04-21T08:40:36.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.810+0000 7ff5005eb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f81b1cb0 con 0x5648e0ee44d0 2024-04-21T08:40:36.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.810+0000 7ff4f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0eeda00 0x5648e0ee06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.810+0000 7ff4f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0eeda00 0x5648e0ee06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50060/0 (socket says 172.21.15.43:50060) 2024-04-21T08:40:36.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.810+0000 7ff4f77fe640 1 -- 172.21.15.43:0/189838535 learned_addr learned my addr 172.21.15.43:0/189838535 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:36.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.810+0000 7ff4effff640 1 --2- 172.21.15.43:0/189838535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5648e0ee4e80 0x5648e0ee97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.810+0000 7ff4f77fe640 1 -- 172.21.15.43:0/189838535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648e0ee44d0 msgr2=0x5648e0ef9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:36.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.810+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/189838535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648e0ee44d0 0x5648e0ef9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.810+0000 7ff4f77fe640 1 -- 172.21.15.43:0/189838535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5648e0ee4e80 msgr2=0x5648e0ee97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:36.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.810+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/189838535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5648e0ee4e80 0x5648e0ee97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.810+0000 7ff4f77fe640 1 -- 172.21.15.43:0/189838535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648e0eeaf20 con 0x5648e0eeda00 2024-04-21T08:40:36.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.810+0000 7ff4effff640 1 --2- 172.21.15.43:0/189838535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5648e0ee4e80 0x5648e0ee97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:36.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.811+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/189838535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0eeda00 0x5648e0ee06e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0004f70 tx=0x7ff4f0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49da745ba2b1c74d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:36.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.811+0000 7ff4ed7fa640 1 -- 172.21.15.43:0/189838535 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0024030 con 0x5648e0eeda00 2024-04-21T08:40:36.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.811+0000 7ff4ed7fa640 1 -- 172.21.15.43:0/189838535 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4f0012bc0 con 0x5648e0eeda00 2024-04-21T08:40:36.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.811+0000 7ff4ed7fa640 1 -- 172.21.15.43:0/189838535 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f00052c0 con 0x5648e0eeda00 2024-04-21T08:40:36.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.811+0000 7ff5005eb740 1 -- 172.21.15.43:0/189838535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0eeda00 msgr2=0x5648e0ee06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.811+0000 7ff5005eb740 1 --2- 172.21.15.43:0/189838535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0eeda00 0x5648e0ee06e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0004f70 tx=0x7ff4f0000f40 comp rx=0 tx=0).stop 2024-04-21T08:40:36.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.811+0000 7ff5005eb740 1 -- 172.21.15.43:0/189838535 shutdown_connections 2024-04-21T08:40:36.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.811+0000 7ff5005eb740 1 --2- 172.21.15.43:0/189838535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0eeda00 0x5648e0ee06e0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.811+0000 7ff5005eb740 1 --2- 172.21.15.43:0/189838535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5648e0ee4e80 0x5648e0ee97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.811+0000 7ff5005eb740 1 --2- 172.21.15.43:0/189838535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648e0ee44d0 0x5648e0ef9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.811+0000 7ff5005eb740 1 -- 172.21.15.43:0/189838535 >> 172.21.15.43:0/189838535 conn(0x5648e0edeba0 msgr2=0x5648e0edffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:36.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.811+0000 7ff5005eb740 1 -- 172.21.15.43:0/189838535 shutdown_connections 2024-04-21T08:40:36.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.811+0000 7ff5005eb740 1 -- 172.21.15.43:0/189838535 wait complete. 2024-04-21T08:40:36.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.812+0000 7ff5005eb740 1 -- start start 2024-04-21T08:40:36.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.812+0000 7ff5005eb740 1 Processor -- start 2024-04-21T08:40:36.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.812+0000 7ff5005eb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5648e0ee44d0 0x5648e0ef5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.812+0000 7ff5005eb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648e0ee4e80 0x5648e0ef6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.812+0000 7ff5005eb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0eeda00 0x5648e0ee6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.812+0000 7ff5005eb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f81b2a50 con 0x5648e0eeda00 2024-04-21T08:40:36.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.812+0000 7ff5005eb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f81b2380 con 0x5648e0ee44d0 2024-04-21T08:40:36.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.812+0000 7ff5005eb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f81af5c0 con 0x5648e0ee4e80 2024-04-21T08:40:36.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.812+0000 7ff5005eb740 10 client.0 _ll_register_callbacks cb 0x7ff4f0008f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:36.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.812+0000 7ff4f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0eeda00 0x5648e0ee6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.812+0000 7ff4f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0eeda00 0x5648e0ee6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50074/0 (socket says 172.21.15.43:50074) 2024-04-21T08:40:36.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.812+0000 7ff4f77fe640 1 -- 172.21.15.43:0/1916623680 learned_addr learned my addr 172.21.15.43:0/1916623680 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:36.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.812+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/1916623680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5648e0ee44d0 0x5648e0ef5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.812+0000 7ff4effff640 1 --2- 172.21.15.43:0/1916623680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648e0ee4e80 0x5648e0ef6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.813+0000 7ff4f77fe640 1 -- 172.21.15.43:0/1916623680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648e0ee4e80 msgr2=0x5648e0ef6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:36.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.813+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/1916623680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648e0ee4e80 0x5648e0ef6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.813+0000 7ff4f77fe640 1 -- 172.21.15.43:0/1916623680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5648e0ee44d0 msgr2=0x5648e0ef5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:36.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.813+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/1916623680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5648e0ee44d0 0x5648e0ef5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.813+0000 7ff4f77fe640 1 -- 172.21.15.43:0/1916623680 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648e0ee74a0 con 0x5648e0eeda00 2024-04-21T08:40:36.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.813+0000 7ff4effff640 1 --2- 172.21.15.43:0/1916623680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648e0ee4e80 0x5648e0ef6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:36.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.813+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/1916623680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0eeda00 0x5648e0ee6c90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0004df0 tx=0x7ff4f0008740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1f45f1f5d8e4c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:36.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.813+0000 7ff4d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:36.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.813+0000 7ff4d6ffd640 1 -- 172.21.15.43:0/1916623680 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0024040 con 0x5648e0eeda00 2024-04-21T08:40:36.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.813+0000 7ff4d6ffd640 1 -- 172.21.15.43:0/1916623680 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7ff4f00083b0 con 0x5648e0eeda00 2024-04-21T08:40:36.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.813+0000 7ff5005eb740 1 -- 172.21.15.43:0/1916623680 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4f0012bc0 con 0x5648e0eeda00 2024-04-21T08:40:36.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.813+0000 7ff4d6ffd640 1 -- 172.21.15.43:0/1916623680 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff4f000fc80 con 0x5648e0eeda00 2024-04-21T08:40:36.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.813+0000 7ff5005eb740 10 client.16491 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:36.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.813+0000 7ff5005eb740 1 -- 172.21.15.43:0/1916623680 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5648e0f637e0 con 0x5648e0eeda00 2024-04-21T08:40:36.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.813+0000 7ff4d6ffd640 1 -- 172.21.15.43:0/1916623680 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4f0012bc0 con 0x5648e0eeda00 2024-04-21T08:40:36.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.813+0000 7ff4d6ffd640 1 -- 172.21.15.43:0/1916623680 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff4f0041060 con 0x5648e0eeda00 2024-04-21T08:40:36.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.813+0000 7ff5005eb740 10 client.16491 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:36.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.813+0000 7ff5005eb740 10 client.16491 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:40:36.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.813+0000 7ff5005eb740 20 client.16491 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:36.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.813+0000 7ff5005eb740 1 --2- 172.21.15.43:0/1916623680 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5648e0f63ca0 0x5648e0f66090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.813+0000 7ff5005eb740 4 client.16491 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:40:36.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.813+0000 7ff5005eb740 1 -- 172.21.15.43:0/1916623680 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5648e0e56c10 con 0x5648e0f63ca0 2024-04-21T08:40:36.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.814+0000 7ff4effff640 1 --2- 172.21.15.43:0/1916623680 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5648e0f63ca0 0x5648e0f66090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.814+0000 7ff4effff640 1 --2- 172.21.15.43:0/1916623680 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5648e0f63ca0 0x5648e0f66090 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b87d1ab958d9708d server_cookie=75c12f2be4fce7c0 in_seq=0 out_seq=0 2024-04-21T08:40:36.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.814+0000 7ff4d6ffd640 10 client.16491 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:40:36.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.814+0000 7ff4d6ffd640 1 -- 172.21.15.43:0/1916623680 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5648e0e56c10 con 0x5648e0f63ca0 2024-04-21T08:40:36.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.814+0000 7ff4d6ffd640 10 client.16491 handle_command_reply: tid=0 2024-04-21T08:40:36.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.814+0000 7ff5005eb740 1 client.16491 shutdown 2024-04-21T08:40:36.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.815+0000 7ff5005eb740 1 -- 172.21.15.43:0/1916623680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0eeda00 msgr2=0x5648e0ee6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.815+0000 7ff5005eb740 1 --2- 172.21.15.43:0/1916623680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0eeda00 0x5648e0ee6c90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0004df0 tx=0x7ff4f0008740 comp rx=0 tx=0).stop 2024-04-21T08:40:36.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.815+0000 7ff5005eb740 1 -- 172.21.15.43:0/1916623680 shutdown_connections 2024-04-21T08:40:36.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.815+0000 7ff5005eb740 1 --2- 172.21.15.43:0/1916623680 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5648e0f63ca0 0x5648e0f66090 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.815+0000 7ff5005eb740 1 --2- 172.21.15.43:0/1916623680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648e0eeda00 0x5648e0ee6c90 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.815+0000 7ff5005eb740 1 --2- 172.21.15.43:0/1916623680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648e0ee4e80 0x5648e0ef6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.815+0000 7ff4d6ffd640 0 client.16491 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:40:36.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.815+0000 7ff5005eb740 1 --2- 172.21.15.43:0/1916623680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5648e0ee44d0 0x5648e0ef5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.815+0000 7ff5005eb740 1 -- 172.21.15.43:0/1916623680 >> 172.21.15.43:0/1916623680 conn(0x5648e0edeba0 msgr2=0x5648e0edd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:36.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.815+0000 7ff5005eb740 1 -- 172.21.15.43:0/1916623680 shutdown_connections 2024-04-21T08:40:36.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.815+0000 7ff5005eb740 1 -- 172.21.15.43:0/1916623680 wait complete. 2024-04-21T08:40:36.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.815+0000 7ff5005eb740 20 client.16491 trim_cache size 0 max 16384 2024-04-21T08:40:36.820 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:40:36.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.815+0000 7ff4fd9f4640 1 -- 172.21.15.43:0/2323438472 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff4e40a3d20 msgr2=0x7ff4e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.815+0000 7ff4fd9f4640 1 --2- 172.21.15.43:0/2323438472 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff4e40a3d20 0x7ff4e40a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff4e809b470 tx=0x7ff4e8004980 comp rx=0 tx=0).stop 2024-04-21T08:40:36.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.815+0000 7ff4fd9f4640 1 -- 172.21.15.43:0/2323438472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4f81010a0 msgr2=0x7ff4f81a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.815+0000 7ff4fd9f4640 1 --2- 172.21.15.43:0/2323438472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4f81010a0 0x7ff4f81a0650 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff4e800c970 tx=0x7ff4e800ce30 comp rx=0 tx=0).stop 2024-04-21T08:40:36.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.815+0000 7ff4fd9f4640 1 -- 172.21.15.43:0/2323438472 shutdown_connections 2024-04-21T08:40:36.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.815+0000 7ff4fd9f4640 1 --2- 172.21.15.43:0/2323438472 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff4e40a3d20 0x7ff4e40a61e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.815+0000 7ff4fd9f4640 1 --2- 172.21.15.43:0/2323438472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4f819a710 0x7ff4f819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.815+0000 7ff4fd9f4640 1 --2- 172.21.15.43:0/2323438472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4f8106170 0x7ff4f81a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.815+0000 7ff4fd9f4640 1 --2- 172.21.15.43:0/2323438472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4f81010a0 0x7ff4f81a0650 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.815+0000 7ff4fd9f4640 1 -- 172.21.15.43:0/2323438472 >> 172.21.15.43:0/2323438472 conn(0x7ff4f8077eb0 msgr2=0x7ff4f810c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.815+0000 7ff4fd9f4640 1 -- 172.21.15.43:0/2323438472 shutdown_connections 2024-04-21T08:40:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.815+0000 7ff4fd9f4640 1 -- 172.21.15.43:0/2323438472 wait complete. 2024-04-21T08:40:36.825 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:40:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.931+0000 7f4b94bb1640 1 -- 172.21.15.43:0/2502661595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b90106160 msgr2=0x7f4b9010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.931+0000 7f4b94bb1640 1 --2- 172.21.15.43:0/2502661595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b90106160 0x7f4b9010dd30 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009ca0 tx=0x7f4b8802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.932+0000 7f4b94bb1640 1 -- 172.21.15.43:0/2502661595 shutdown_connections 2024-04-21T08:40:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.932+0000 7f4b94bb1640 1 --2- 172.21.15.43:0/2502661595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b90106160 0x7f4b9010dd30 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.932+0000 7f4b94bb1640 1 --2- 172.21.15.43:0/2502661595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b90101a40 0x7f4b90105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.932+0000 7f4b94bb1640 1 --2- 172.21.15.43:0/2502661595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b90101090 0x7f4b90101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.932+0000 7f4b94bb1640 1 -- 172.21.15.43:0/2502661595 >> 172.21.15.43:0/2502661595 conn(0x7f4b90077f50 msgr2=0x7f4b90078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.932+0000 7f4b94bb1640 1 -- 172.21.15.43:0/2502661595 shutdown_connections 2024-04-21T08:40:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.932+0000 7f4b94bb1640 1 -- 172.21.15.43:0/2502661595 wait complete. 2024-04-21T08:40:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.932+0000 7f4b94bb1640 1 Processor -- start 2024-04-21T08:40:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.932+0000 7f4b94bb1640 1 -- start start 2024-04-21T08:40:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.932+0000 7f4b94bb1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b90101090 0x7f4b901995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.932+0000 7f4b94bb1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b90106160 0x7f4b90199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.932+0000 7f4b94bb1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b9019a070 0x7f4b901a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.933+0000 7f4b94bb1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4b90111120 con 0x7f4b9019a070 2024-04-21T08:40:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.933+0000 7f4b94bb1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4b90110fa0 con 0x7f4b90101090 2024-04-21T08:40:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.933+0000 7f4b94bb1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4b901112a0 con 0x7f4b90106160 2024-04-21T08:40:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.933+0000 7f4b8ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b9019a070 0x7f4b901a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.933+0000 7f4b8ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b9019a070 0x7f4b901a4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50090/0 (socket says 172.21.15.43:50090) 2024-04-21T08:40:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.933+0000 7f4b8ed9d640 1 -- 172.21.15.43:0/3271588721 learned_addr learned my addr 172.21.15.43:0/3271588721 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.933+0000 7f4b87fff640 1 --2- 172.21.15.43:0/3271588721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b90106160 0x7f4b90199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.933+0000 7f4b8e59c640 1 --2- 172.21.15.43:0/3271588721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b90101090 0x7f4b901995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.933+0000 7f4b8ed9d640 1 -- 172.21.15.43:0/3271588721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b90106160 msgr2=0x7f4b90199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.933+0000 7f4b8ed9d640 1 --2- 172.21.15.43:0/3271588721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b90106160 0x7f4b90199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.933+0000 7f4b8ed9d640 1 -- 172.21.15.43:0/3271588721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b90101090 msgr2=0x7f4b901995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.933+0000 7f4b8ed9d640 1 --2- 172.21.15.43:0/3271588721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b90101090 0x7f4b901995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.933+0000 7f4b8ed9d640 1 -- 172.21.15.43:0/3271588721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b901a4e20 con 0x7f4b9019a070 2024-04-21T08:40:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.933+0000 7f4b8ed9d640 1 --2- 172.21.15.43:0/3271588721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b9019a070 0x7f4b901a4620 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4b88005e30 tx=0x7f4b88034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.933+0000 7f4b877fe640 1 -- 172.21.15.43:0/3271588721 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b88039040 con 0x7f4b9019a070 2024-04-21T08:40:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.933+0000 7f4b94bb1640 1 -- 172.21.15.43:0/3271588721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b901a50b0 con 0x7f4b9019a070 2024-04-21T08:40:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.933+0000 7f4b877fe640 1 -- 172.21.15.43:0/3271588721 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b88044070 con 0x7f4b9019a070 2024-04-21T08:40:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.933+0000 7f4b877fe640 1 -- 172.21.15.43:0/3271588721 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b88003e60 con 0x7f4b9019a070 2024-04-21T08:40:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.933+0000 7f4b94bb1640 1 -- 172.21.15.43:0/3271588721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b901a55a0 con 0x7f4b9019a070 2024-04-21T08:40:36.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.935+0000 7f4b877fe640 1 -- 172.21.15.43:0/3271588721 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4b880324c0 con 0x7f4b9019a070 2024-04-21T08:40:36.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.936+0000 7f4b9779b740 1 Processor -- start 2024-04-21T08:40:36.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.936+0000 7f4b9779b740 1 -- start start 2024-04-21T08:40:36.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.936+0000 7f4b9779b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc04d304d0 0x55bc04d308b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.936+0000 7f4b9779b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d30e80 0x55bc04d39430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.936+0000 7f4b9779b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc04d39a00 0x55bc04d3be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.936+0000 7f4b9779b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bc04c951f0 con 0x55bc04d30e80 2024-04-21T08:40:36.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.936+0000 7f4b9779b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bc04c324b0 con 0x55bc04d304d0 2024-04-21T08:40:36.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.936+0000 7f4b9779b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bc04c911b0 con 0x55bc04d39a00 2024-04-21T08:40:36.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.936+0000 7f4b877fe640 1 --2- 172.21.15.43:0/3271588721 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4b7c0a3d20 0x7f4b7c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.937+0000 7f4b8e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d30e80 0x55bc04d39430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.937+0000 7f4b8ed9d640 1 --2- 172.21.15.43:0/3271588721 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4b7c0a3d20 0x7f4b7c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.937+0000 7f4b8e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d30e80 0x55bc04d39430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50098/0 (socket says 172.21.15.43:50098) 2024-04-21T08:40:36.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.937+0000 7f4b8e59c640 1 -- 172.21.15.43:0/3118081861 learned_addr learned my addr 172.21.15.43:0/3118081861 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:36.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.937+0000 7f4b87fff640 1 --2- 172.21.15.43:0/3118081861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc04d39a00 0x55bc04d3be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.937+0000 7f4b87fff640 1 --2- 172.21.15.43:0/3118081861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc04d304d0 0x55bc04d308b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.937+0000 7f4b877fe640 1 -- 172.21.15.43:0/3271588721 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f4b880f2eb0 con 0x7f4b9019a070 2024-04-21T08:40:36.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.937+0000 7f4b8e59c640 1 -- 172.21.15.43:0/3118081861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc04d39a00 msgr2=0x55bc04d3be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:36.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.937+0000 7f4b8e59c640 1 --2- 172.21.15.43:0/3118081861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc04d39a00 0x55bc04d3be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.937+0000 7f4b8e59c640 1 -- 172.21.15.43:0/3118081861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc04d304d0 msgr2=0x55bc04d308b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:36.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.937+0000 7f4b8e59c640 1 --2- 172.21.15.43:0/3118081861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc04d304d0 0x55bc04d308b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.937+0000 7f4b8e59c640 1 -- 172.21.15.43:0/3118081861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc04c241b0 con 0x55bc04d30e80 2024-04-21T08:40:36.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.937+0000 7f4b87fff640 1 --2- 172.21.15.43:0/3118081861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc04d304d0 0x55bc04d308b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:36.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.937+0000 7f4b8ed9d640 1 --2- 172.21.15.43:0/3271588721 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4b7c0a3d20 0x7f4b7c0a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4b880be830 tx=0x7f4b88091620 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:36.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.937+0000 7f4b8e59c640 1 --2- 172.21.15.43:0/3118081861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d30e80 0x55bc04d39430 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000bc90 tx=0x7f4b8000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c6527b558c70d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:36.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.937+0000 7f4b84ff9640 1 -- 172.21.15.43:0/3118081861 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80015070 con 0x55bc04d30e80 2024-04-21T08:40:36.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.937+0000 7f4b84ff9640 1 -- 172.21.15.43:0/3118081861 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b800118f0 con 0x55bc04d30e80 2024-04-21T08:40:36.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.937+0000 7f4b84ff9640 1 -- 172.21.15.43:0/3118081861 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b800107d0 con 0x55bc04d30e80 2024-04-21T08:40:36.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.938+0000 7f4b9779b740 1 -- 172.21.15.43:0/3118081861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d30e80 msgr2=0x55bc04d39430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.938+0000 7f4b9779b740 1 --2- 172.21.15.43:0/3118081861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d30e80 0x55bc04d39430 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000bc90 tx=0x7f4b8000f730 comp rx=0 tx=0).stop 2024-04-21T08:40:36.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.938+0000 7f4b9779b740 1 -- 172.21.15.43:0/3118081861 shutdown_connections 2024-04-21T08:40:36.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.938+0000 7f4b9779b740 1 --2- 172.21.15.43:0/3118081861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc04d39a00 0x55bc04d3be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.938+0000 7f4b9779b740 1 --2- 172.21.15.43:0/3118081861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d30e80 0x55bc04d39430 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.938+0000 7f4b9779b740 1 --2- 172.21.15.43:0/3118081861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc04d304d0 0x55bc04d308b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.938+0000 7f4b9779b740 1 -- 172.21.15.43:0/3118081861 >> 172.21.15.43:0/3118081861 conn(0x55bc04d2aba0 msgr2=0x55bc04d2af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:36.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.938+0000 7f4b9779b740 1 -- 172.21.15.43:0/3118081861 shutdown_connections 2024-04-21T08:40:36.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.938+0000 7f4b9779b740 1 -- 172.21.15.43:0/3118081861 wait complete. 2024-04-21T08:40:36.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.939+0000 7f4b9779b740 1 -- start start 2024-04-21T08:40:36.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.939+0000 7f4b9779b740 1 Processor -- start 2024-04-21T08:40:36.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.939+0000 7f4b9779b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d304d0 0x55bc04d392f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.939+0000 7f4b9779b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc04d30e80 0x55bc04d36680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.939+0000 7f4b9779b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc04d39a00 0x55bc04d36c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.939+0000 7f4b9779b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bc04d3c970 con 0x55bc04d304d0 2024-04-21T08:40:36.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.939+0000 7f4b9779b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bc04c4e500 con 0x55bc04d30e80 2024-04-21T08:40:36.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.939+0000 7f4b9779b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bc04d3caf0 con 0x55bc04d39a00 2024-04-21T08:40:36.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.939+0000 7f4b87fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d304d0 0x55bc04d392f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.939+0000 7f4b9779b740 10 client.0 _ll_register_callbacks cb 0x55bc04c1cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:36.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.939+0000 7f4b87fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d304d0 0x55bc04d392f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50108/0 (socket says 172.21.15.43:50108) 2024-04-21T08:40:36.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.939+0000 7f4b87fff640 1 -- 172.21.15.43:0/357666062 learned_addr learned my addr 172.21.15.43:0/357666062 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:36.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.939+0000 7f4b8e59c640 1 --2- 172.21.15.43:0/357666062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc04d30e80 0x55bc04d36680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.939+0000 7f4b87fff640 1 --2- 172.21.15.43:0/357666062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc04d39a00 0x55bc04d36c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.939+0000 7f4b87fff640 1 -- 172.21.15.43:0/357666062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc04d39a00 msgr2=0x55bc04d36c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:36.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.939+0000 7f4b87fff640 1 --2- 172.21.15.43:0/357666062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc04d39a00 0x55bc04d36c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.939+0000 7f4b87fff640 1 -- 172.21.15.43:0/357666062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc04d30e80 msgr2=0x55bc04d36680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:36.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.939+0000 7f4b87fff640 1 --2- 172.21.15.43:0/357666062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc04d30e80 0x55bc04d36680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.939+0000 7f4b87fff640 1 -- 172.21.15.43:0/357666062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc049a73f0 con 0x55bc04d304d0 2024-04-21T08:40:36.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.939+0000 7f4b87fff640 1 --2- 172.21.15.43:0/357666062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d304d0 0x55bc04d392f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800afd0 tx=0x7f4b780047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5485e7caf48b266 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:36.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.939+0000 7f4b75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:36.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.940+0000 7f4b75ffb640 1 -- 172.21.15.43:0/357666062 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78014720 con 0x55bc04d304d0 2024-04-21T08:40:36.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.940+0000 7f4b75ffb640 1 -- 172.21.15.43:0/357666062 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f4b780148c0 con 0x55bc04d304d0 2024-04-21T08:40:36.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.940+0000 7f4b9779b740 1 -- 172.21.15.43:0/357666062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b800118f0 con 0x55bc04d304d0 2024-04-21T08:40:36.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.940+0000 7f4b75ffb640 1 -- 172.21.15.43:0/357666062 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4b78028050 con 0x55bc04d304d0 2024-04-21T08:40:36.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.940+0000 7f4b9779b740 10 client.16503 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:36.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.940+0000 7f4b9779b740 1 -- 172.21.15.43:0/357666062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc04a717d0 con 0x55bc04d304d0 2024-04-21T08:40:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.940+0000 7f4b75ffb640 1 -- 172.21.15.43:0/357666062 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b800118f0 con 0x55bc04d304d0 2024-04-21T08:40:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.940+0000 7f4b75ffb640 1 -- 172.21.15.43:0/357666062 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4b78015040 con 0x55bc04d304d0 2024-04-21T08:40:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.940+0000 7f4b9779b740 10 client.16503 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.940+0000 7f4b9779b740 10 client.16503 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:40:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.940+0000 7f4b9779b740 20 client.16503 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.940+0000 7f4b9779b740 1 --2- 172.21.15.43:0/357666062 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55bc04d49c50 0x55bc04d4c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.940+0000 7f4b9779b740 4 client.16503 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.940+0000 7f4b9779b740 1 -- 172.21.15.43:0/357666062 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc04c34930 con 0x55bc04d49c50 2024-04-21T08:40:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.941+0000 7f4b8e59c640 1 --2- 172.21.15.43:0/357666062 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55bc04d49c50 0x55bc04d4c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.941+0000 7f4b8e59c640 1 --2- 172.21.15.43:0/357666062 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55bc04d49c50 0x55bc04d4c060 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7f261840bf6e1075 server_cookie=c0c4d4b671c1f4d4 in_seq=0 out_seq=0 2024-04-21T08:40:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.941+0000 7f4b75ffb640 10 client.16503 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:40:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.942+0000 7f4b75ffb640 1 -- 172.21.15.43:0/357666062 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bc04c34930 con 0x55bc04d49c50 2024-04-21T08:40:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.942+0000 7f4b75ffb640 10 client.16503 handle_command_reply: tid=0 2024-04-21T08:40:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.942+0000 7f4b9779b740 1 client.16503 shutdown 2024-04-21T08:40:36.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.942+0000 7f4b9779b740 1 -- 172.21.15.43:0/357666062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d304d0 msgr2=0x55bc04d392f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.942+0000 7f4b9779b740 1 --2- 172.21.15.43:0/357666062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d304d0 0x55bc04d392f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800afd0 tx=0x7f4b780047d0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.942+0000 7f4b9779b740 1 -- 172.21.15.43:0/357666062 shutdown_connections 2024-04-21T08:40:36.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.942+0000 7f4b9779b740 1 --2- 172.21.15.43:0/357666062 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55bc04d49c50 0x55bc04d4c060 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.942+0000 7f4b9779b740 1 --2- 172.21.15.43:0/357666062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc04d39a00 0x55bc04d36c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.942+0000 7f4b9779b740 1 --2- 172.21.15.43:0/357666062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc04d30e80 0x55bc04d36680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.942+0000 7f4b75ffb640 0 client.16503 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:40:36.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.942+0000 7f4b9779b740 1 --2- 172.21.15.43:0/357666062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d304d0 0x55bc04d392f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.942+0000 7f4b9779b740 1 -- 172.21.15.43:0/357666062 >> 172.21.15.43:0/357666062 conn(0x55bc04d2aba0 msgr2=0x55bc04d2c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:36.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.942+0000 7f4b9779b740 1 -- 172.21.15.43:0/357666062 shutdown_connections 2024-04-21T08:40:36.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.942+0000 7f4b9779b740 1 -- 172.21.15.43:0/357666062 wait complete. 2024-04-21T08:40:36.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.942+0000 7f4b9779b740 20 client.16503 trim_cache size 0 max 16384 2024-04-21T08:40:36.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.955+0000 7f4b9779b740 1 Processor -- start 2024-04-21T08:40:36.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.955+0000 7f4b9779b740 1 -- start start 2024-04-21T08:40:36.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.956+0000 7f4b9779b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d304d0 0x55bc04d45830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.956+0000 7f4b9779b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc04d30e80 0x55bc04d38db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.956+0000 7f4b9779b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc04d39a00 0x55bc04d392f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.956+0000 7f4b9779b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4b901af460 con 0x55bc04d304d0 2024-04-21T08:40:36.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.956+0000 7f4b9779b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4b901b0fa0 con 0x55bc04d39a00 2024-04-21T08:40:36.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.956+0000 7f4b9779b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4b901afb30 con 0x55bc04d30e80 2024-04-21T08:40:36.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.956+0000 7f4b87fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d304d0 0x55bc04d45830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.956+0000 7f4b87fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d304d0 0x55bc04d45830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50124/0 (socket says 172.21.15.43:50124) 2024-04-21T08:40:36.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.956+0000 7f4b87fff640 1 -- 172.21.15.43:0/3086501422 learned_addr learned my addr 172.21.15.43:0/3086501422 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:36.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.956+0000 7f4b87fff640 1 -- 172.21.15.43:0/3086501422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc04d30e80 msgr2=0x55bc04d38db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:36.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.956+0000 7f4b87fff640 1 --2- 172.21.15.43:0/3086501422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc04d30e80 0x55bc04d38db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.956+0000 7f4b87fff640 1 -- 172.21.15.43:0/3086501422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc04d39a00 msgr2=0x55bc04d392f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:36.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.956+0000 7f4b87fff640 1 --2- 172.21.15.43:0/3086501422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc04d39a00 0x55bc04d392f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.956+0000 7f4b87fff640 1 -- 172.21.15.43:0/3086501422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc04d36f70 con 0x55bc04d304d0 2024-04-21T08:40:36.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.956+0000 7f4b87fff640 1 --2- 172.21.15.43:0/3086501422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d304d0 0x55bc04d45830 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800a880 tx=0x7f4b780069d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec3ca28950dd940 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:36.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.956+0000 7f4b77fff640 1 -- 172.21.15.43:0/3086501422 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78030020 con 0x55bc04d304d0 2024-04-21T08:40:36.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.956+0000 7f4b77fff640 1 -- 172.21.15.43:0/3086501422 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b7801f070 con 0x55bc04d304d0 2024-04-21T08:40:36.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.956+0000 7f4b77fff640 1 -- 172.21.15.43:0/3086501422 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b780168b0 con 0x55bc04d304d0 2024-04-21T08:40:36.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.957+0000 7f4b9779b740 1 -- 172.21.15.43:0/3086501422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d304d0 msgr2=0x55bc04d45830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.957+0000 7f4b9779b740 1 --2- 172.21.15.43:0/3086501422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d304d0 0x55bc04d45830 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800a880 tx=0x7f4b780069d0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.957+0000 7f4b9779b740 1 -- 172.21.15.43:0/3086501422 shutdown_connections 2024-04-21T08:40:36.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.957+0000 7f4b9779b740 1 --2- 172.21.15.43:0/3086501422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc04d39a00 0x55bc04d392f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.957+0000 7f4b9779b740 1 --2- 172.21.15.43:0/3086501422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc04d30e80 0x55bc04d38db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.957+0000 7f4b9779b740 1 --2- 172.21.15.43:0/3086501422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d304d0 0x55bc04d45830 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.957+0000 7f4b9779b740 1 -- 172.21.15.43:0/3086501422 >> 172.21.15.43:0/3086501422 conn(0x55bc04d2aba0 msgr2=0x55bc04d29c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:36.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.957+0000 7f4b9779b740 1 -- 172.21.15.43:0/3086501422 shutdown_connections 2024-04-21T08:40:36.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.957+0000 7f4b9779b740 1 -- 172.21.15.43:0/3086501422 wait complete. 2024-04-21T08:40:36.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.957+0000 7f4b9779b740 1 -- start start 2024-04-21T08:40:36.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.957+0000 7f4b9779b740 1 Processor -- start 2024-04-21T08:40:36.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.958+0000 7f4b9779b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d304d0 0x55bc04d41f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.958+0000 7f4b9779b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc04d30e80 0x55bc04d32ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.958+0000 7f4b9779b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc04d39a00 0x55bc04d33430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.958+0000 7f4b9779b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4b901b08d0 con 0x55bc04d304d0 2024-04-21T08:40:36.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.958+0000 7f4b9779b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4b901b0200 con 0x55bc04d30e80 2024-04-21T08:40:36.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.958+0000 7f4b9779b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4b901ad440 con 0x55bc04d39a00 2024-04-21T08:40:36.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.958+0000 7f4b9779b740 10 client.0 _ll_register_callbacks cb 0x7f4b7800ae90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:36.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.958+0000 7f4b87fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d304d0 0x55bc04d41f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.958+0000 7f4b87fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d304d0 0x55bc04d41f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50140/0 (socket says 172.21.15.43:50140) 2024-04-21T08:40:36.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.958+0000 7f4b87fff640 1 -- 172.21.15.43:0/3966968616 learned_addr learned my addr 172.21.15.43:0/3966968616 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:36.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.958+0000 7f4b87fff640 1 --2- 172.21.15.43:0/3966968616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc04d39a00 0x55bc04d33430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.958+0000 7f4b8e59c640 1 --2- 172.21.15.43:0/3966968616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc04d30e80 0x55bc04d32ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.958+0000 7f4b87fff640 1 -- 172.21.15.43:0/3966968616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc04d39a00 msgr2=0x55bc04d33430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:36.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.958+0000 7f4b87fff640 1 --2- 172.21.15.43:0/3966968616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc04d39a00 0x55bc04d33430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.958+0000 7f4b87fff640 1 -- 172.21.15.43:0/3966968616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc04d30e80 msgr2=0x55bc04d32ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:36.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.958+0000 7f4b87fff640 1 --2- 172.21.15.43:0/3966968616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc04d30e80 0x55bc04d32ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.958+0000 7f4b87fff640 1 -- 172.21.15.43:0/3966968616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc04d339a0 con 0x55bc04d304d0 2024-04-21T08:40:36.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.958+0000 7f4b87fff640 1 --2- 172.21.15.43:0/3966968616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d304d0 0x55bc04d41f30 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800ae60 tx=0x7f4b78017fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ee2c82a5a2fb9c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:36.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.958+0000 7f4b75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:36.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.958+0000 7f4b75ffb640 1 -- 172.21.15.43:0/3966968616 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7803a070 con 0x55bc04d304d0 2024-04-21T08:40:36.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.958+0000 7f4b75ffb640 1 -- 172.21.15.43:0/3966968616 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f4b78014b30 con 0x55bc04d304d0 2024-04-21T08:40:36.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.958+0000 7f4b9779b740 1 -- 172.21.15.43:0/3966968616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b7801f070 con 0x55bc04d304d0 2024-04-21T08:40:36.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.959+0000 7f4b75ffb640 1 -- 172.21.15.43:0/3966968616 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4b78030030 con 0x55bc04d304d0 2024-04-21T08:40:36.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.959+0000 7f4b9779b740 10 client.16509 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:36.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.959+0000 7f4b9779b740 1 -- 172.21.15.43:0/3966968616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc04daff20 con 0x55bc04d304d0 2024-04-21T08:40:36.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.959+0000 7f4b75ffb640 1 -- 172.21.15.43:0/3966968616 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b7801f070 con 0x55bc04d304d0 2024-04-21T08:40:36.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.959+0000 7f4b75ffb640 1 -- 172.21.15.43:0/3966968616 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4b7800f070 con 0x55bc04d304d0 2024-04-21T08:40:36.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.959+0000 7f4b9779b740 10 client.16509 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:36.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.959+0000 7f4b9779b740 10 client.16509 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:40:36.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.959+0000 7f4b9779b740 20 client.16509 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:36.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.959+0000 7f4b9779b740 1 --2- 172.21.15.43:0/3966968616 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55bc04db04d0 0x55bc04db28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:36.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.959+0000 7f4b9779b740 4 client.16509 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:40:36.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.959+0000 7f4b9779b740 1 -- 172.21.15.43:0/3966968616 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc04ca2c10 con 0x55bc04db04d0 2024-04-21T08:40:36.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.959+0000 7f4b8e59c640 1 --2- 172.21.15.43:0/3966968616 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55bc04db04d0 0x55bc04db28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:36.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.960+0000 7f4b75ffb640 10 client.16509 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:40:36.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.960+0000 7f4b8e59c640 1 --2- 172.21.15.43:0/3966968616 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55bc04db04d0 0x55bc04db28f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a7eb718f052db9c5 server_cookie=e91b33c0cd5c3863 in_seq=0 out_seq=0 2024-04-21T08:40:36.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.960+0000 7f4b75ffb640 1 -- 172.21.15.43:0/3966968616 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc04ca2c10 con 0x55bc04db04d0 2024-04-21T08:40:36.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.960+0000 7f4b75ffb640 10 client.16509 handle_command_reply: tid=0 2024-04-21T08:40:36.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.960+0000 7f4b9779b740 1 client.16509 shutdown 2024-04-21T08:40:36.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.960+0000 7f4b9779b740 1 -- 172.21.15.43:0/3966968616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d304d0 msgr2=0x55bc04d41f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.960+0000 7f4b9779b740 1 --2- 172.21.15.43:0/3966968616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d304d0 0x55bc04d41f30 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800ae60 tx=0x7f4b78017fd0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.960+0000 7f4b9779b740 1 -- 172.21.15.43:0/3966968616 shutdown_connections 2024-04-21T08:40:36.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.960+0000 7f4b9779b740 1 --2- 172.21.15.43:0/3966968616 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55bc04db04d0 0x55bc04db28f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.960+0000 7f4b9779b740 1 --2- 172.21.15.43:0/3966968616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bc04d39a00 0x55bc04d33430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.960+0000 7f4b75ffb640 0 client.16509 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:40:36.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.960+0000 7f4b9779b740 1 --2- 172.21.15.43:0/3966968616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc04d30e80 0x55bc04d32ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.960+0000 7f4b9779b740 1 --2- 172.21.15.43:0/3966968616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc04d304d0 0x55bc04d41f30 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.960+0000 7f4b9779b740 1 -- 172.21.15.43:0/3966968616 >> 172.21.15.43:0/3966968616 conn(0x55bc04d2aba0 msgr2=0x55bc04d29370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:36.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.960+0000 7f4b9779b740 1 -- 172.21.15.43:0/3966968616 shutdown_connections 2024-04-21T08:40:36.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.960+0000 7f4b9779b740 1 -- 172.21.15.43:0/3966968616 wait complete. 2024-04-21T08:40:36.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.960+0000 7f4b9779b740 20 client.16509 trim_cache size 0 max 16384 2024-04-21T08:40:36.966 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:40:36.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.961+0000 7f4b94bb1640 1 -- 172.21.15.43:0/3271588721 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4b7c0a3d20 msgr2=0x7f4b7c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.961+0000 7f4b94bb1640 1 --2- 172.21.15.43:0/3271588721 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4b7c0a3d20 0x7f4b7c0a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4b880be830 tx=0x7f4b88091620 comp rx=0 tx=0).stop 2024-04-21T08:40:36.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.961+0000 7f4b94bb1640 1 -- 172.21.15.43:0/3271588721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b9019a070 msgr2=0x7f4b901a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.961+0000 7f4b94bb1640 1 --2- 172.21.15.43:0/3271588721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b9019a070 0x7f4b901a4620 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4b88005e30 tx=0x7f4b88034760 comp rx=0 tx=0).stop 2024-04-21T08:40:36.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.961+0000 7f4b94bb1640 1 -- 172.21.15.43:0/3271588721 shutdown_connections 2024-04-21T08:40:36.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.961+0000 7f4b94bb1640 1 --2- 172.21.15.43:0/3271588721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b9019a070 0x7f4b901a4620 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.961+0000 7f4b94bb1640 1 --2- 172.21.15.43:0/3271588721 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4b7c0a3d20 0x7f4b7c0a61e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.961+0000 7f4b94bb1640 1 --2- 172.21.15.43:0/3271588721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b90106160 0x7f4b90199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.961+0000 7f4b94bb1640 1 --2- 172.21.15.43:0/3271588721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b90101090 0x7f4b901995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.961+0000 7f4b94bb1640 1 -- 172.21.15.43:0/3271588721 >> 172.21.15.43:0/3271588721 conn(0x7f4b90077f50 msgr2=0x7f4b900ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:36.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.961+0000 7f4b94bb1640 1 -- 172.21.15.43:0/3271588721 shutdown_connections 2024-04-21T08:40:36.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:36.961+0000 7f4b94bb1640 1 -- 172.21.15.43:0/3271588721 wait complete. 2024-04-21T08:40:36.973 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:40:37.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.078+0000 7f97b4a55640 1 -- 172.21.15.43:0/3852358950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b006a840 msgr2=0x7f97b006ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.078+0000 7f97b4a55640 1 --2- 172.21.15.43:0/3852358950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b006a840 0x7f97b006ac20 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009900 tx=0x7f97a002bc30 comp rx=0 tx=0).stop 2024-04-21T08:40:37.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.078+0000 7f97b4a55640 1 -- 172.21.15.43:0/3852358950 shutdown_connections 2024-04-21T08:40:37.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.078+0000 7f97b4a55640 1 --2- 172.21.15.43:0/3852358950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b0107270 0x7f97b010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.078+0000 7f97b4a55640 1 --2- 172.21.15.43:0/3852358950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97b006b1f0 0x7f97b0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.078+0000 7f97b4a55640 1 --2- 172.21.15.43:0/3852358950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b006a840 0x7f97b006ac20 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.078+0000 7f97b4a55640 1 -- 172.21.15.43:0/3852358950 >> 172.21.15.43:0/3852358950 conn(0x7f97b0077eb0 msgr2=0x7f97b00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:37.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.078+0000 7f97b4a55640 1 -- 172.21.15.43:0/3852358950 shutdown_connections 2024-04-21T08:40:37.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.078+0000 7f97b4a55640 1 -- 172.21.15.43:0/3852358950 wait complete. 2024-04-21T08:40:37.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.079+0000 7f97b4a55640 1 Processor -- start 2024-04-21T08:40:37.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.079+0000 7f97b4a55640 1 -- start start 2024-04-21T08:40:37.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.079+0000 7f97b4a55640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97b006a840 0x7f97b01986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.079+0000 7f97b4a55640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b0107270 0x7f97b0198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.079+0000 7f97b4a55640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b019cf60 0x7f97b019d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.079+0000 7f97b4a55640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f97b01121b0 con 0x7f97b019cf60 2024-04-21T08:40:37.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.079+0000 7f97b4a55640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f97b0112030 con 0x7f97b0107270 2024-04-21T08:40:37.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.079+0000 7f97b4a55640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f97b0112330 con 0x7f97b006a840 2024-04-21T08:40:37.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.079+0000 7f97aed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b019cf60 0x7f97b019d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.079+0000 7f97aed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b019cf60 0x7f97b019d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50156/0 (socket says 172.21.15.43:50156) 2024-04-21T08:40:37.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.079+0000 7f97aed9d640 1 -- 172.21.15.43:0/722483613 learned_addr learned my addr 172.21.15.43:0/722483613 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:37.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.079+0000 7f97ae59c640 1 --2- 172.21.15.43:0/722483613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97b006a840 0x7f97b01986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.079+0000 7f97add9b640 1 --2- 172.21.15.43:0/722483613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b0107270 0x7f97b0198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.080+0000 7f97aed9d640 1 -- 172.21.15.43:0/722483613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97b006a840 msgr2=0x7f97b01986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.080+0000 7f97aed9d640 1 --2- 172.21.15.43:0/722483613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97b006a840 0x7f97b01986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.080+0000 7f97aed9d640 1 -- 172.21.15.43:0/722483613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b0107270 msgr2=0x7f97b0198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.080+0000 7f97aed9d640 1 --2- 172.21.15.43:0/722483613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b0107270 0x7f97b0198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.080+0000 7f97aed9d640 1 -- 172.21.15.43:0/722483613 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b019da50 con 0x7f97b019cf60 2024-04-21T08:40:37.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.080+0000 7f97add9b640 1 --2- 172.21.15.43:0/722483613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b0107270 0x7f97b0198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:37.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.080+0000 7f97aed9d640 1 --2- 172.21.15.43:0/722483613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b019cf60 0x7f97b019d410 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f97a8009fd0 tx=0x7f97a800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:37.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.080+0000 7f979f7fe640 1 -- 172.21.15.43:0/722483613 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a8017070 con 0x7f97b019cf60 2024-04-21T08:40:37.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.080+0000 7f979f7fe640 1 -- 172.21.15.43:0/722483613 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97a8003e20 con 0x7f97b019cf60 2024-04-21T08:40:37.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.080+0000 7f97b4a55640 1 -- 172.21.15.43:0/722483613 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b019dd40 con 0x7f97b019cf60 2024-04-21T08:40:37.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.080+0000 7f979f7fe640 1 -- 172.21.15.43:0/722483613 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a8010460 con 0x7f97b019cf60 2024-04-21T08:40:37.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.080+0000 7f97b4a55640 1 -- 172.21.15.43:0/722483613 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b01a54e0 con 0x7f97b019cf60 2024-04-21T08:40:37.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.083+0000 7f979f7fe640 1 -- 172.21.15.43:0/722483613 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f97a8003970 con 0x7f97b019cf60 2024-04-21T08:40:37.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.083+0000 7f97b7648740 1 Processor -- start 2024-04-21T08:40:37.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.083+0000 7f97b7648740 1 -- start start 2024-04-21T08:40:37.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.083+0000 7f97b7648740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a6e80f250 0x558a6e80f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.083+0000 7f97b7648740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a6e80fc00 0x558a6e81c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.083+0000 7f97b7648740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e81ce10 0x558a6e81f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.083+0000 7f97b7648740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558a6e7741f0 con 0x558a6e81ce10 2024-04-21T08:40:37.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.083+0000 7f97b7648740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558a6e7114b0 con 0x558a6e80fc00 2024-04-21T08:40:37.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.083+0000 7f97b7648740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558a6e7701b0 con 0x558a6e80f250 2024-04-21T08:40:37.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.084+0000 7f97add9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e81ce10 0x558a6e81f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.084+0000 7f97add9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e81ce10 0x558a6e81f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50164/0 (socket says 172.21.15.43:50164) 2024-04-21T08:40:37.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.084+0000 7f97add9b640 1 -- 172.21.15.43:0/3539641934 learned_addr learned my addr 172.21.15.43:0/3539641934 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:37.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.084+0000 7f979f7fe640 1 --2- 172.21.15.43:0/722483613 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f97980a3d20 0x7f97980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.084+0000 7f97ae59c640 1 --2- 172.21.15.43:0/3539641934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a6e80fc00 0x558a6e81c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.084+0000 7f97aed9d640 1 --2- 172.21.15.43:0/722483613 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f97980a3d20 0x7f97980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.084+0000 7f97add9b640 1 --2- 172.21.15.43:0/3539641934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a6e80f250 0x558a6e80f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.084+0000 7f97add9b640 1 -- 172.21.15.43:0/3539641934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a6e80f250 msgr2=0x558a6e80f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:37.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.084+0000 7f97add9b640 1 --2- 172.21.15.43:0/3539641934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a6e80f250 0x558a6e80f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.084+0000 7f97add9b640 1 -- 172.21.15.43:0/3539641934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a6e80fc00 msgr2=0x558a6e81c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:37.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.084+0000 7f97add9b640 1 --2- 172.21.15.43:0/3539641934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a6e80fc00 0x558a6e81c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.084+0000 7f97add9b640 1 -- 172.21.15.43:0/3539641934 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a6e7031b0 con 0x558a6e81ce10 2024-04-21T08:40:37.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.084+0000 7f979f7fe640 1 -- 172.21.15.43:0/722483613 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f97a80d0970 con 0x7f97b019cf60 2024-04-21T08:40:37.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.084+0000 7f97add9b640 1 --2- 172.21.15.43:0/3539641934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e81ce10 0x558a6e81f220 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f97a400dfc0 tx=0x7f97a400d600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c9abf6574aeb005 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:37.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.084+0000 7f97aed9d640 1 --2- 172.21.15.43:0/722483613 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f97980a3d20 0x7f97980a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f97a80925d0 tx=0x7f97a8098800 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:37.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.084+0000 7f979cff9640 1 -- 172.21.15.43:0/3539641934 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a4005220 con 0x558a6e81ce10 2024-04-21T08:40:37.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.084+0000 7f979cff9640 1 -- 172.21.15.43:0/3539641934 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97a4004c50 con 0x558a6e81ce10 2024-04-21T08:40:37.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.084+0000 7f979cff9640 1 -- 172.21.15.43:0/3539641934 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a4012850 con 0x558a6e81ce10 2024-04-21T08:40:37.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.085+0000 7f97b7648740 1 -- 172.21.15.43:0/3539641934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e81ce10 msgr2=0x558a6e81f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.085+0000 7f97b7648740 1 --2- 172.21.15.43:0/3539641934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e81ce10 0x558a6e81f220 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f97a400dfc0 tx=0x7f97a400d600 comp rx=0 tx=0).stop 2024-04-21T08:40:37.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.085+0000 7f97b7648740 1 -- 172.21.15.43:0/3539641934 shutdown_connections 2024-04-21T08:40:37.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.085+0000 7f97b7648740 1 --2- 172.21.15.43:0/3539641934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e81ce10 0x558a6e81f220 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.085+0000 7f97b7648740 1 --2- 172.21.15.43:0/3539641934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a6e80fc00 0x558a6e81c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.085+0000 7f97b7648740 1 --2- 172.21.15.43:0/3539641934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a6e80f250 0x558a6e80f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.085+0000 7f97b7648740 1 -- 172.21.15.43:0/3539641934 >> 172.21.15.43:0/3539641934 conn(0x558a6e809ba0 msgr2=0x558a6e809f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:37.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.085+0000 7f97b7648740 1 -- 172.21.15.43:0/3539641934 shutdown_connections 2024-04-21T08:40:37.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.085+0000 7f97b7648740 1 -- 172.21.15.43:0/3539641934 wait complete. 2024-04-21T08:40:37.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.086+0000 7f97b7648740 1 -- start start 2024-04-21T08:40:37.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.086+0000 7f97b7648740 1 Processor -- start 2024-04-21T08:40:37.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.086+0000 7f97b7648740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a6e80f250 0x558a6e81c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.086+0000 7f97b7648740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a6e80fc00 0x558a6e819ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.086+0000 7f97b7648740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e81ce10 0x558a6e81a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.086+0000 7f97b7648740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558a6e81fd40 con 0x558a6e81ce10 2024-04-21T08:40:37.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.086+0000 7f97b7648740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558a6e72d500 con 0x558a6e80f250 2024-04-21T08:40:37.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.086+0000 7f97b7648740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558a6e81fec0 con 0x558a6e80fc00 2024-04-21T08:40:37.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.086+0000 7f97b7648740 10 client.0 _ll_register_callbacks cb 0x558a6e6fbb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:37.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.086+0000 7f97add9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e81ce10 0x558a6e81a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.086+0000 7f97add9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e81ce10 0x558a6e81a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50174/0 (socket says 172.21.15.43:50174) 2024-04-21T08:40:37.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.086+0000 7f97add9b640 1 -- 172.21.15.43:0/1467723469 learned_addr learned my addr 172.21.15.43:0/1467723469 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:37.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.086+0000 7f97add9b640 1 --2- 172.21.15.43:0/1467723469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a6e80f250 0x558a6e81c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.086+0000 7f97ae59c640 1 --2- 172.21.15.43:0/1467723469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a6e80fc00 0x558a6e819ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.086+0000 7f97add9b640 1 -- 172.21.15.43:0/1467723469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a6e80fc00 msgr2=0x558a6e819ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:37.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.086+0000 7f97add9b640 1 --2- 172.21.15.43:0/1467723469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a6e80fc00 0x558a6e819ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.086+0000 7f97add9b640 1 -- 172.21.15.43:0/1467723469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a6e80f250 msgr2=0x558a6e81c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:37.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.086+0000 7f97add9b640 1 --2- 172.21.15.43:0/1467723469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a6e80f250 0x558a6e81c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.086+0000 7f97add9b640 1 -- 172.21.15.43:0/1467723469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a6e4863f0 con 0x558a6e81ce10 2024-04-21T08:40:37.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.087+0000 7f97add9b640 1 --2- 172.21.15.43:0/1467723469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e81ce10 0x558a6e81a0f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f97a4005660 tx=0x7f97a4016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259ce0e8267064f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:37.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.087+0000 7f9795ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:37.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.087+0000 7f9795ffb640 1 -- 172.21.15.43:0/1467723469 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a4004fa0 con 0x558a6e81ce10 2024-04-21T08:40:37.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.087+0000 7f97b7648740 1 -- 172.21.15.43:0/1467723469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97a4004c50 con 0x558a6e81ce10 2024-04-21T08:40:37.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.087+0000 7f9795ffb640 1 -- 172.21.15.43:0/1467723469 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f97a4005140 con 0x558a6e81ce10 2024-04-21T08:40:37.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.087+0000 7f9795ffb640 1 -- 172.21.15.43:0/1467723469 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f97a4032050 con 0x558a6e81ce10 2024-04-21T08:40:37.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.087+0000 7f97b7648740 10 client.16521 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:37.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.087+0000 7f97b7648740 1 -- 172.21.15.43:0/1467723469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558a6e5507d0 con 0x558a6e81ce10 2024-04-21T08:40:37.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.087+0000 7f9795ffb640 1 -- 172.21.15.43:0/1467723469 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97a4004c50 con 0x558a6e81ce10 2024-04-21T08:40:37.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.087+0000 7f9795ffb640 1 -- 172.21.15.43:0/1467723469 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f97a402a020 con 0x558a6e81ce10 2024-04-21T08:40:37.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.087+0000 7f97b7648740 10 client.16521 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:37.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.087+0000 7f97b7648740 10 client.16521 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:40:37.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.087+0000 7f97b7648740 20 client.16521 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:37.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.087+0000 7f97b7648740 1 --2- 172.21.15.43:0/1467723469 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x558a6e82d040 0x558a6e82f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.087+0000 7f97b7648740 4 client.16521 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:37.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.087+0000 7f97b7648740 1 -- 172.21.15.43:0/1467723469 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a6e713930 con 0x558a6e82d040 2024-04-21T08:40:37.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.088+0000 7f97ae59c640 1 --2- 172.21.15.43:0/1467723469 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x558a6e82d040 0x558a6e82f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.088+0000 7f97ae59c640 1 --2- 172.21.15.43:0/1467723469 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x558a6e82d040 0x558a6e82f430 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=48903388440ece57 server_cookie=45ff59ab66b3bad3 in_seq=0 out_seq=0 2024-04-21T08:40:37.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.088+0000 7f9795ffb640 10 client.16521 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:40:37.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.089+0000 7f9795ffb640 1 -- 172.21.15.43:0/1467723469 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558a6e713930 con 0x558a6e82d040 2024-04-21T08:40:37.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.089+0000 7f9795ffb640 10 client.16521 handle_command_reply: tid=0 2024-04-21T08:40:37.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.089+0000 7f97b7648740 1 client.16521 shutdown 2024-04-21T08:40:37.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.089+0000 7f97b7648740 1 -- 172.21.15.43:0/1467723469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e81ce10 msgr2=0x558a6e81a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.089+0000 7f97b7648740 1 --2- 172.21.15.43:0/1467723469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e81ce10 0x558a6e81a0f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f97a4005660 tx=0x7f97a4016040 comp rx=0 tx=0).stop 2024-04-21T08:40:37.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.089+0000 7f97b7648740 1 -- 172.21.15.43:0/1467723469 shutdown_connections 2024-04-21T08:40:37.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.089+0000 7f97b7648740 1 --2- 172.21.15.43:0/1467723469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e81ce10 0x558a6e81a0f0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.089+0000 7f97b7648740 1 --2- 172.21.15.43:0/1467723469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a6e80fc00 0x558a6e819ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.089+0000 7f97b7648740 1 --2- 172.21.15.43:0/1467723469 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x558a6e82d040 0x558a6e82f430 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.089+0000 7f97b7648740 1 --2- 172.21.15.43:0/1467723469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a6e80f250 0x558a6e81c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.089+0000 7f97b7648740 1 -- 172.21.15.43:0/1467723469 >> 172.21.15.43:0/1467723469 conn(0x558a6e809ba0 msgr2=0x558a6e80b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:37.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.089+0000 7f9795ffb640 0 client.16521 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:40:37.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.089+0000 7f97b7648740 1 -- 172.21.15.43:0/1467723469 shutdown_connections 2024-04-21T08:40:37.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.090+0000 7f97b7648740 1 -- 172.21.15.43:0/1467723469 wait complete. 2024-04-21T08:40:37.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.090+0000 7f97b7648740 20 client.16521 trim_cache size 0 max 16384 2024-04-21T08:40:37.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.103+0000 7f97b7648740 1 Processor -- start 2024-04-21T08:40:37.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.103+0000 7f97b7648740 1 -- start start 2024-04-21T08:40:37.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.103+0000 7f97b7648740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e80f250 0x558a6e828c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.103+0000 7f97b7648740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a6e80fc00 0x558a6e818d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.103+0000 7f97b7648740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a6e81ce10 0x558a6e81c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.103+0000 7f97b7648740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f97b01ad130 con 0x558a6e80f250 2024-04-21T08:40:37.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.103+0000 7f97b7648740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f97b01aec70 con 0x558a6e80fc00 2024-04-21T08:40:37.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.103+0000 7f97b7648740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f97b01ad800 con 0x558a6e81ce10 2024-04-21T08:40:37.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.103+0000 7f97add9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e80f250 0x558a6e828c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.103+0000 7f97add9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e80f250 0x558a6e828c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50188/0 (socket says 172.21.15.43:50188) 2024-04-21T08:40:37.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.103+0000 7f97add9b640 1 -- 172.21.15.43:0/4021435936 learned_addr learned my addr 172.21.15.43:0/4021435936 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:37.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.103+0000 7f97ae59c640 1 --2- 172.21.15.43:0/4021435936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a6e80fc00 0x558a6e818d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.104+0000 7f97add9b640 1 -- 172.21.15.43:0/4021435936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a6e81ce10 msgr2=0x558a6e81c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:37.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.104+0000 7f97add9b640 1 --2- 172.21.15.43:0/4021435936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a6e81ce10 0x558a6e81c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.104+0000 7f97add9b640 1 -- 172.21.15.43:0/4021435936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a6e80fc00 msgr2=0x558a6e818d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:37.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.104+0000 7f97add9b640 1 --2- 172.21.15.43:0/4021435936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a6e80fc00 0x558a6e818d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.104+0000 7f97add9b640 1 -- 172.21.15.43:0/4021435936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a6e80e560 con 0x558a6e80f250 2024-04-21T08:40:37.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.104+0000 7f97add9b640 1 --2- 172.21.15.43:0/4021435936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e80f250 0x558a6e828c40 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f97a4030040 tx=0x7f97a4012f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c874df2bc02afe server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:37.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.104+0000 7f9797fff640 1 -- 172.21.15.43:0/4021435936 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a4005be0 con 0x558a6e80f250 2024-04-21T08:40:37.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.104+0000 7f9797fff640 1 -- 172.21.15.43:0/4021435936 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97a400db50 con 0x558a6e80f250 2024-04-21T08:40:37.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.104+0000 7f9797fff640 1 -- 172.21.15.43:0/4021435936 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a4004660 con 0x558a6e80f250 2024-04-21T08:40:37.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.104+0000 7f97b7648740 1 -- 172.21.15.43:0/4021435936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e80f250 msgr2=0x558a6e828c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.104+0000 7f97b7648740 1 --2- 172.21.15.43:0/4021435936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e80f250 0x558a6e828c40 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f97a4030040 tx=0x7f97a4012f10 comp rx=0 tx=0).stop 2024-04-21T08:40:37.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.104+0000 7f97b7648740 1 -- 172.21.15.43:0/4021435936 shutdown_connections 2024-04-21T08:40:37.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.104+0000 7f97b7648740 1 --2- 172.21.15.43:0/4021435936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a6e81ce10 0x558a6e81c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.104+0000 7f97b7648740 1 --2- 172.21.15.43:0/4021435936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a6e80fc00 0x558a6e818d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.104+0000 7f97b7648740 1 --2- 172.21.15.43:0/4021435936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e80f250 0x558a6e828c40 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.104+0000 7f97b7648740 1 -- 172.21.15.43:0/4021435936 >> 172.21.15.43:0/4021435936 conn(0x558a6e809ba0 msgr2=0x558a6e815070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:37.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.105+0000 7f97b7648740 1 -- 172.21.15.43:0/4021435936 shutdown_connections 2024-04-21T08:40:37.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.105+0000 7f97b7648740 1 -- 172.21.15.43:0/4021435936 wait complete. 2024-04-21T08:40:37.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.105+0000 7f97b7648740 1 -- start start 2024-04-21T08:40:37.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.105+0000 7f97b7648740 1 Processor -- start 2024-04-21T08:40:37.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.105+0000 7f97b7648740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e80f250 0x558a6e825150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.105+0000 7f97b7648740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a6e80fc00 0x558a6e825690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.105+0000 7f97b7648740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a6e81ce10 0x558a6e815ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.105+0000 7f97b7648740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f97b01ae5a0 con 0x558a6e80f250 2024-04-21T08:40:37.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.105+0000 7f97b7648740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f97b01aded0 con 0x558a6e80fc00 2024-04-21T08:40:37.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.105+0000 7f97b7648740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f97b01ab110 con 0x558a6e81ce10 2024-04-21T08:40:37.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.105+0000 7f97b7648740 10 client.0 _ll_register_callbacks cb 0x7f97a4004e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:37.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.105+0000 7f97add9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e80f250 0x558a6e825150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.105+0000 7f97add9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e80f250 0x558a6e825150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50196/0 (socket says 172.21.15.43:50196) 2024-04-21T08:40:37.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.105+0000 7f97add9b640 1 -- 172.21.15.43:0/2234888721 learned_addr learned my addr 172.21.15.43:0/2234888721 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:37.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.105+0000 7f97ae59c640 1 --2- 172.21.15.43:0/2234888721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a6e80fc00 0x558a6e825690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.105+0000 7f97add9b640 1 -- 172.21.15.43:0/2234888721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a6e81ce10 msgr2=0x558a6e815ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:37.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.105+0000 7f97add9b640 1 --2- 172.21.15.43:0/2234888721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a6e81ce10 0x558a6e815ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.105+0000 7f97add9b640 1 -- 172.21.15.43:0/2234888721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a6e80fc00 msgr2=0x558a6e825690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:37.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.105+0000 7f97add9b640 1 --2- 172.21.15.43:0/2234888721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a6e80fc00 0x558a6e825690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.105+0000 7f97add9b640 1 -- 172.21.15.43:0/2234888721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a6e8166f0 con 0x558a6e80f250 2024-04-21T08:40:37.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.106+0000 7f97add9b640 1 --2- 172.21.15.43:0/2234888721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e80f250 0x558a6e825150 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f97a4014440 tx=0x7f97a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07ec4ac634ce830 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:37.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.106+0000 7f9795ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:37.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.106+0000 7f9795ffb640 1 -- 172.21.15.43:0/2234888721 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a403f060 con 0x558a6e80f250 2024-04-21T08:40:37.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.106+0000 7f9795ffb640 1 -- 172.21.15.43:0/2234888721 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f97a4004ef0 con 0x558a6e80f250 2024-04-21T08:40:37.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.106+0000 7f97b7648740 1 -- 172.21.15.43:0/2234888721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97a400db50 con 0x558a6e80f250 2024-04-21T08:40:37.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.106+0000 7f9795ffb640 1 -- 172.21.15.43:0/2234888721 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f97a4046050 con 0x558a6e80f250 2024-04-21T08:40:37.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.106+0000 7f97b7648740 10 client.16527 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:37.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.106+0000 7f97b7648740 1 -- 172.21.15.43:0/2234888721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558a6e892de0 con 0x558a6e80f250 2024-04-21T08:40:37.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.106+0000 7f9795ffb640 1 -- 172.21.15.43:0/2234888721 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97a400db50 con 0x558a6e80f250 2024-04-21T08:40:37.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.106+0000 7f9795ffb640 1 -- 172.21.15.43:0/2234888721 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f97a402a040 con 0x558a6e80f250 2024-04-21T08:40:37.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.106+0000 7f97b7648740 10 client.16527 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:37.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.106+0000 7f97b7648740 10 client.16527 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:40:37.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.106+0000 7f97b7648740 20 client.16527 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:37.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.106+0000 7f97b7648740 1 --2- 172.21.15.43:0/2234888721 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x558a6e8933e0 0x558a6e895800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.106+0000 7f97b7648740 4 client.16527 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:40:37.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.106+0000 7f97b7648740 1 -- 172.21.15.43:0/2234888721 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a6e781c10 con 0x558a6e8933e0 2024-04-21T08:40:37.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.107+0000 7f97ae59c640 1 --2- 172.21.15.43:0/2234888721 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x558a6e8933e0 0x558a6e895800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.107+0000 7f9795ffb640 10 client.16527 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:40:37.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.107+0000 7f97ae59c640 1 --2- 172.21.15.43:0/2234888721 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x558a6e8933e0 0x558a6e895800 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6bea116c789545cd server_cookie=5542347c86003f8a in_seq=0 out_seq=0 2024-04-21T08:40:37.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.107+0000 7f9795ffb640 1 -- 172.21.15.43:0/2234888721 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a6e781c10 con 0x558a6e8933e0 2024-04-21T08:40:37.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.107+0000 7f9795ffb640 10 client.16527 handle_command_reply: tid=0 2024-04-21T08:40:37.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.107+0000 7f97b7648740 1 client.16527 shutdown 2024-04-21T08:40:37.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.107+0000 7f97b7648740 1 -- 172.21.15.43:0/2234888721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e80f250 msgr2=0x558a6e825150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.107+0000 7f97b7648740 1 --2- 172.21.15.43:0/2234888721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e80f250 0x558a6e825150 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f97a4014440 tx=0x7f97a4005830 comp rx=0 tx=0).stop 2024-04-21T08:40:37.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.107+0000 7f97b7648740 1 -- 172.21.15.43:0/2234888721 shutdown_connections 2024-04-21T08:40:37.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.107+0000 7f97b7648740 1 --2- 172.21.15.43:0/2234888721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558a6e81ce10 0x558a6e815ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.107+0000 7f97b7648740 1 --2- 172.21.15.43:0/2234888721 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x558a6e8933e0 0x558a6e895800 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.107+0000 7f97b7648740 1 --2- 172.21.15.43:0/2234888721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a6e80fc00 0x558a6e825690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.107+0000 7f97b7648740 1 --2- 172.21.15.43:0/2234888721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a6e80f250 0x558a6e825150 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.107+0000 7f9795ffb640 0 client.16527 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:40:37.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.107+0000 7f97b7648740 1 -- 172.21.15.43:0/2234888721 >> 172.21.15.43:0/2234888721 conn(0x558a6e809ba0 msgr2=0x558a6e80af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:37.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.107+0000 7f97b7648740 1 -- 172.21.15.43:0/2234888721 shutdown_connections 2024-04-21T08:40:37.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.107+0000 7f97b7648740 1 -- 172.21.15.43:0/2234888721 wait complete. 2024-04-21T08:40:37.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.107+0000 7f97b7648740 20 client.16527 trim_cache size 0 max 16384 2024-04-21T08:40:37.114 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:40:37.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.108+0000 7f97b4a55640 1 -- 172.21.15.43:0/722483613 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f97980a3d20 msgr2=0x7f97980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.108+0000 7f97b4a55640 1 --2- 172.21.15.43:0/722483613 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f97980a3d20 0x7f97980a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f97a80925d0 tx=0x7f97a8098800 comp rx=0 tx=0).stop 2024-04-21T08:40:37.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.108+0000 7f97b4a55640 1 -- 172.21.15.43:0/722483613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b019cf60 msgr2=0x7f97b019d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.108+0000 7f97b4a55640 1 --2- 172.21.15.43:0/722483613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b019cf60 0x7f97b019d410 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f97a8009fd0 tx=0x7f97a800b260 comp rx=0 tx=0).stop 2024-04-21T08:40:37.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.108+0000 7f97b4a55640 1 -- 172.21.15.43:0/722483613 shutdown_connections 2024-04-21T08:40:37.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.108+0000 7f97b4a55640 1 --2- 172.21.15.43:0/722483613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b019cf60 0x7f97b019d410 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.108+0000 7f97b4a55640 1 --2- 172.21.15.43:0/722483613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b0107270 0x7f97b0198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.108+0000 7f97b4a55640 1 --2- 172.21.15.43:0/722483613 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f97980a3d20 0x7f97980a61e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.108+0000 7f97b4a55640 1 --2- 172.21.15.43:0/722483613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97b006a840 0x7f97b01986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.108+0000 7f97b4a55640 1 -- 172.21.15.43:0/722483613 >> 172.21.15.43:0/722483613 conn(0x7f97b0077eb0 msgr2=0x7f97b0100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:37.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.108+0000 7f97b4a55640 1 -- 172.21.15.43:0/722483613 shutdown_connections 2024-04-21T08:40:37.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.108+0000 7f97b4a55640 1 -- 172.21.15.43:0/722483613 wait complete. 2024-04-21T08:40:37.118 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:40:37.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.222+0000 7f6af538e640 1 -- 172.21.15.43:0/4158127018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6af0103af0 msgr2=0x7f6af010bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.222+0000 7f6af538e640 1 --2- 172.21.15.43:0/4158127018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6af0103af0 0x7f6af010bfd0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009900 tx=0x7f6ae402bd60 comp rx=0 tx=0).stop 2024-04-21T08:40:37.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.223+0000 7f6af538e640 1 -- 172.21.15.43:0/4158127018 shutdown_connections 2024-04-21T08:40:37.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.223+0000 7f6af538e640 1 --2- 172.21.15.43:0/4158127018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6af010c510 0x7f6af010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.223+0000 7f6af538e640 1 --2- 172.21.15.43:0/4158127018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6af0103af0 0x7f6af010bfd0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.223+0000 7f6af538e640 1 --2- 172.21.15.43:0/4158127018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6af01031d0 0x7f6af01035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.223+0000 7f6af538e640 1 -- 172.21.15.43:0/4158127018 >> 172.21.15.43:0/4158127018 conn(0x7f6af0077f50 msgr2=0x7f6af0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:37.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.223+0000 7f6af538e640 1 -- 172.21.15.43:0/4158127018 shutdown_connections 2024-04-21T08:40:37.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.223+0000 7f6af538e640 1 -- 172.21.15.43:0/4158127018 wait complete. 2024-04-21T08:40:37.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.223+0000 7f6af538e640 1 Processor -- start 2024-04-21T08:40:37.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.223+0000 7f6af538e640 1 -- start start 2024-04-21T08:40:37.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.223+0000 7f6af538e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6af01031d0 0x7f6af0100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.223+0000 7f6af538e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6af0103af0 0x7f6af0101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.224+0000 7f6af538e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6af010c510 0x7f6af01018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.224+0000 7f6af538e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01121a0 con 0x7f6af010c510 2024-04-21T08:40:37.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.224+0000 7f6af538e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6af0112020 con 0x7f6af01031d0 2024-04-21T08:40:37.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.224+0000 7f6af538e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6af0112320 con 0x7f6af0103af0 2024-04-21T08:40:37.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.224+0000 7f6aef7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6af010c510 0x7f6af01018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.224+0000 7f6aef7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6af010c510 0x7f6af01018a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50214/0 (socket says 172.21.15.43:50214) 2024-04-21T08:40:37.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.224+0000 7f6aef7fe640 1 -- 172.21.15.43:0/3929214107 learned_addr learned my addr 172.21.15.43:0/3929214107 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:37.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.224+0000 7f6aeeffd640 1 --2- 172.21.15.43:0/3929214107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6af01031d0 0x7f6af0100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.224+0000 7f6aee7fc640 1 --2- 172.21.15.43:0/3929214107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6af0103af0 0x7f6af0101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.224+0000 7f6aef7fe640 1 -- 172.21.15.43:0/3929214107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6af0103af0 msgr2=0x7f6af0101360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.224+0000 7f6aef7fe640 1 --2- 172.21.15.43:0/3929214107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6af0103af0 0x7f6af0101360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.224+0000 7f6aef7fe640 1 -- 172.21.15.43:0/3929214107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6af01031d0 msgr2=0x7f6af0100e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.224+0000 7f6aef7fe640 1 --2- 172.21.15.43:0/3929214107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6af01031d0 0x7f6af0100e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.224+0000 7f6aef7fe640 1 -- 172.21.15.43:0/3929214107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af01a6c40 con 0x7f6af010c510 2024-04-21T08:40:37.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.224+0000 7f6aef7fe640 1 --2- 172.21.15.43:0/3929214107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6af010c510 0x7f6af01018a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800d750 tx=0x7f6ae800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:37.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.224+0000 7f6ad7fff640 1 -- 172.21.15.43:0/3929214107 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8017070 con 0x7f6af010c510 2024-04-21T08:40:37.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.224+0000 7f6ad7fff640 1 -- 172.21.15.43:0/3929214107 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ae80054d0 con 0x7f6af010c510 2024-04-21T08:40:37.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.224+0000 7f6af538e640 1 -- 172.21.15.43:0/3929214107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af01a6f30 con 0x7f6af010c510 2024-04-21T08:40:37.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.224+0000 7f6ad7fff640 1 -- 172.21.15.43:0/3929214107 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8004c60 con 0x7f6af010c510 2024-04-21T08:40:37.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.224+0000 7f6af538e640 1 -- 172.21.15.43:0/3929214107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af01a7400 con 0x7f6af010c510 2024-04-21T08:40:37.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.227+0000 7f6ad7fff640 1 -- 172.21.15.43:0/3929214107 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6ae8009bd0 con 0x7f6af010c510 2024-04-21T08:40:37.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.227+0000 7f6af7f77740 1 Processor -- start 2024-04-21T08:40:37.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.227+0000 7f6af7f77740 1 -- start start 2024-04-21T08:40:37.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.227+0000 7f6af7f77740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568777044d0 0x5568777048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.227+0000 7f6af7f77740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556877704e80 0x55687770d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.227+0000 7f6af7f77740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55687770da00 0x55687770fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.227+0000 7f6af7f77740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5568776691f0 con 0x5568777044d0 2024-04-21T08:40:37.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.227+0000 7f6af7f77740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5568776064b0 con 0x556877704e80 2024-04-21T08:40:37.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.227+0000 7f6af7f77740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5568776651b0 con 0x55687770da00 2024-04-21T08:40:37.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.227+0000 7f6ad7fff640 1 --2- 172.21.15.43:0/3929214107 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6ad80a3d20 0x7f6ad80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.227+0000 7f6aee7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568777044d0 0x5568777048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.227+0000 7f6aee7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568777044d0 0x5568777048b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50228/0 (socket says 172.21.15.43:50228) 2024-04-21T08:40:37.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.227+0000 7f6aee7fc640 1 -- 172.21.15.43:0/3756588910 learned_addr learned my addr 172.21.15.43:0/3756588910 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:37.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.227+0000 7f6aef7fe640 1 --2- 172.21.15.43:0/3929214107 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6ad80a3d20 0x7f6ad80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.227+0000 7f6aeeffd640 1 --2- 172.21.15.43:0/3756588910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556877704e80 0x55687770d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.228+0000 7f6ad7fff640 1 -- 172.21.15.43:0/3929214107 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f6ae80d0460 con 0x7f6af010c510 2024-04-21T08:40:37.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.228+0000 7f6aee7fc640 1 -- 172.21.15.43:0/3756588910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55687770da00 msgr2=0x55687770fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:37.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.228+0000 7f6aee7fc640 1 --2- 172.21.15.43:0/3756588910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55687770da00 0x55687770fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.228+0000 7f6aee7fc640 1 -- 172.21.15.43:0/3756588910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556877704e80 msgr2=0x55687770d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:37.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.228+0000 7f6aee7fc640 1 --2- 172.21.15.43:0/3756588910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556877704e80 0x55687770d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.228+0000 7f6aee7fc640 1 -- 172.21.15.43:0/3756588910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568775f81b0 con 0x5568777044d0 2024-04-21T08:40:37.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.228+0000 7f6aee7fc640 1 --2- 172.21.15.43:0/3756588910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568777044d0 0x5568777048b0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6ae402c2a0 tx=0x7f6ae4009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0c640548c52e7e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:37.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.228+0000 7f6aef7fe640 1 --2- 172.21.15.43:0/3929214107 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6ad80a3d20 0x7f6ad80a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80cd890 tx=0x7f6ae806ebd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:37.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.228+0000 7f6ad57fa640 1 -- 172.21.15.43:0/3756588910 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4004180 con 0x5568777044d0 2024-04-21T08:40:37.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.228+0000 7f6ad57fa640 1 -- 172.21.15.43:0/3756588910 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ae402cdc0 con 0x5568777044d0 2024-04-21T08:40:37.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.228+0000 7f6ad57fa640 1 -- 172.21.15.43:0/3756588910 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae40046c0 con 0x5568777044d0 2024-04-21T08:40:37.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.229+0000 7f6af7f77740 1 -- 172.21.15.43:0/3756588910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568777044d0 msgr2=0x5568777048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.229+0000 7f6af7f77740 1 --2- 172.21.15.43:0/3756588910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568777044d0 0x5568777048b0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6ae402c2a0 tx=0x7f6ae4009670 comp rx=0 tx=0).stop 2024-04-21T08:40:37.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.229+0000 7f6af7f77740 1 -- 172.21.15.43:0/3756588910 shutdown_connections 2024-04-21T08:40:37.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.229+0000 7f6af7f77740 1 --2- 172.21.15.43:0/3756588910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55687770da00 0x55687770fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.229+0000 7f6af7f77740 1 --2- 172.21.15.43:0/3756588910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556877704e80 0x55687770d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.229+0000 7f6af7f77740 1 --2- 172.21.15.43:0/3756588910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568777044d0 0x5568777048b0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.229+0000 7f6af7f77740 1 -- 172.21.15.43:0/3756588910 >> 172.21.15.43:0/3756588910 conn(0x5568776feba0 msgr2=0x5568776fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:37.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.229+0000 7f6af7f77740 1 -- 172.21.15.43:0/3756588910 shutdown_connections 2024-04-21T08:40:37.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.229+0000 7f6af7f77740 1 -- 172.21.15.43:0/3756588910 wait complete. 2024-04-21T08:40:37.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.229+0000 7f6af7f77740 1 -- start start 2024-04-21T08:40:37.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.229+0000 7f6af7f77740 1 Processor -- start 2024-04-21T08:40:37.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.230+0000 7f6af7f77740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568777044d0 0x55687770d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.230+0000 7f6af7f77740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556877704e80 0x55687770a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.230+0000 7f6af7f77740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687770da00 0x55687770acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.230+0000 7f6af7f77740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556877710910 con 0x55687770da00 2024-04-21T08:40:37.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.230+0000 7f6af7f77740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556877622500 con 0x5568777044d0 2024-04-21T08:40:37.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.230+0000 7f6af7f77740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556877710a90 con 0x556877704e80 2024-04-21T08:40:37.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.230+0000 7f6af7f77740 10 client.0 _ll_register_callbacks cb 0x5568775f0b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:37.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.230+0000 7f6aee7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687770da00 0x55687770acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.230+0000 7f6aee7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687770da00 0x55687770acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50236/0 (socket says 172.21.15.43:50236) 2024-04-21T08:40:37.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.230+0000 7f6aee7fc640 1 -- 172.21.15.43:0/919171785 learned_addr learned my addr 172.21.15.43:0/919171785 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:37.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.230+0000 7f6aeeffd640 1 --2- 172.21.15.43:0/919171785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556877704e80 0x55687770a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.230+0000 7f6aee7fc640 1 --2- 172.21.15.43:0/919171785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568777044d0 0x55687770d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.230+0000 7f6aee7fc640 1 -- 172.21.15.43:0/919171785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556877704e80 msgr2=0x55687770a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:37.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.230+0000 7f6aee7fc640 1 --2- 172.21.15.43:0/919171785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556877704e80 0x55687770a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.230+0000 7f6aee7fc640 1 -- 172.21.15.43:0/919171785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568777044d0 msgr2=0x55687770d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:37.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.230+0000 7f6aee7fc640 1 --2- 172.21.15.43:0/919171785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568777044d0 0x55687770d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.230+0000 7f6aee7fc640 1 -- 172.21.15.43:0/919171785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55687737b3f0 con 0x55687770da00 2024-04-21T08:40:37.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.230+0000 7f6aee7fc640 1 --2- 172.21.15.43:0/919171785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687770da00 0x55687770acd0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4005c40 tx=0x7f6ae4005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e30e2a95e1ffac5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:37.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.230+0000 7f6ad2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:37.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.230+0000 7f6ad2ff5640 1 -- 172.21.15.43:0/919171785 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4004390 con 0x55687770da00 2024-04-21T08:40:37.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.230+0000 7f6ad2ff5640 1 -- 172.21.15.43:0/919171785 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f6ae402c910 con 0x55687770da00 2024-04-21T08:40:37.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.230+0000 7f6af7f77740 1 -- 172.21.15.43:0/919171785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ae402cdc0 con 0x55687770da00 2024-04-21T08:40:37.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.231+0000 7f6ad2ff5640 1 -- 172.21.15.43:0/919171785 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6ae405a050 con 0x55687770da00 2024-04-21T08:40:37.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.231+0000 7f6af7f77740 10 client.16539 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:37.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.231+0000 7f6af7f77740 1 -- 172.21.15.43:0/919171785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5568774457d0 con 0x55687770da00 2024-04-21T08:40:37.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.231+0000 7f6ad2ff5640 1 -- 172.21.15.43:0/919171785 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ae402cdc0 con 0x55687770da00 2024-04-21T08:40:37.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.231+0000 7f6ad2ff5640 1 -- 172.21.15.43:0/919171785 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6ae4003710 con 0x55687770da00 2024-04-21T08:40:37.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.231+0000 7f6af7f77740 10 client.16539 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:37.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.231+0000 7f6af7f77740 10 client.16539 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:40:37.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.231+0000 7f6af7f77740 20 client.16539 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:37.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.231+0000 7f6af7f77740 1 --2- 172.21.15.43:0/919171785 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55687771da90 0x55687771fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.231+0000 7f6af7f77740 4 client.16539 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:37.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.231+0000 7f6af7f77740 1 -- 172.21.15.43:0/919171785 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556877608930 con 0x55687771da90 2024-04-21T08:40:37.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.231+0000 7f6aeeffd640 1 --2- 172.21.15.43:0/919171785 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55687771da90 0x55687771fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.232+0000 7f6aeeffd640 1 --2- 172.21.15.43:0/919171785 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55687771da90 0x55687771fe80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d56ef04e770d0d9c server_cookie=ecdb08fe8cee9a28 in_seq=0 out_seq=0 2024-04-21T08:40:37.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.232+0000 7f6ad2ff5640 10 client.16539 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:40:37.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.232+0000 7f6ad2ff5640 1 -- 172.21.15.43:0/919171785 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556877608930 con 0x55687771da90 2024-04-21T08:40:37.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.232+0000 7f6ad2ff5640 10 client.16539 handle_command_reply: tid=0 2024-04-21T08:40:37.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.232+0000 7f6af7f77740 1 client.16539 shutdown 2024-04-21T08:40:37.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.233+0000 7f6af7f77740 1 -- 172.21.15.43:0/919171785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687770da00 msgr2=0x55687770acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.233+0000 7f6af7f77740 1 --2- 172.21.15.43:0/919171785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687770da00 0x55687770acd0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4005c40 tx=0x7f6ae4005250 comp rx=0 tx=0).stop 2024-04-21T08:40:37.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.233+0000 7f6af7f77740 1 -- 172.21.15.43:0/919171785 shutdown_connections 2024-04-21T08:40:37.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.233+0000 7f6af7f77740 1 --2- 172.21.15.43:0/919171785 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55687771da90 0x55687771fe80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.233+0000 7f6af7f77740 1 --2- 172.21.15.43:0/919171785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687770da00 0x55687770acd0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.233+0000 7f6ad2ff5640 0 client.16539 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:40:37.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.233+0000 7f6af7f77740 1 --2- 172.21.15.43:0/919171785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556877704e80 0x55687770a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.233+0000 7f6af7f77740 1 --2- 172.21.15.43:0/919171785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568777044d0 0x55687770d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.233+0000 7f6af7f77740 1 -- 172.21.15.43:0/919171785 >> 172.21.15.43:0/919171785 conn(0x5568776feba0 msgr2=0x5568777008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:37.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.233+0000 7f6af7f77740 1 -- 172.21.15.43:0/919171785 shutdown_connections 2024-04-21T08:40:37.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.233+0000 7f6af7f77740 1 -- 172.21.15.43:0/919171785 wait complete. 2024-04-21T08:40:37.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.233+0000 7f6af7f77740 20 client.16539 trim_cache size 0 max 16384 2024-04-21T08:40:37.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.246+0000 7f6af7f77740 1 Processor -- start 2024-04-21T08:40:37.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.246+0000 7f6af7f77740 1 -- start start 2024-04-21T08:40:37.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.246+0000 7f6af7f77740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568777044d0 0x55687770cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.246+0000 7f6af7f77740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556877704e80 0x55687770d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.246+0000 7f6af7f77740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55687770da00 0x556877722ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.246+0000 7f6af7f77740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01b1480 con 0x5568777044d0 2024-04-21T08:40:37.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.246+0000 7f6af7f77740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01b2fc0 con 0x55687770da00 2024-04-21T08:40:37.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.246+0000 7f6af7f77740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01b1b50 con 0x556877704e80 2024-04-21T08:40:37.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.246+0000 7f6aee7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568777044d0 0x55687770cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.246+0000 7f6aee7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568777044d0 0x55687770cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50246/0 (socket says 172.21.15.43:50246) 2024-04-21T08:40:37.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.246+0000 7f6aee7fc640 1 -- 172.21.15.43:0/3104939809 learned_addr learned my addr 172.21.15.43:0/3104939809 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:37.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.246+0000 7f6aee7fc640 1 -- 172.21.15.43:0/3104939809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556877704e80 msgr2=0x55687770d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:37.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.246+0000 7f6aee7fc640 1 --2- 172.21.15.43:0/3104939809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556877704e80 0x55687770d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.247+0000 7f6aee7fc640 1 -- 172.21.15.43:0/3104939809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55687770da00 msgr2=0x556877722ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:37.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.247+0000 7f6aee7fc640 1 --2- 172.21.15.43:0/3104939809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55687770da00 0x556877722ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.247+0000 7f6aee7fc640 1 -- 172.21.15.43:0/3104939809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556877703560 con 0x5568777044d0 2024-04-21T08:40:37.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.247+0000 7f6aee7fc640 1 --2- 172.21.15.43:0/3104939809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568777044d0 0x55687770cdf0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4000c00 tx=0x7f6ae4005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6322f7008c825594 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:37.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.247+0000 7f6ad4ff9640 1 -- 172.21.15.43:0/3104939809 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae40052c0 con 0x5568777044d0 2024-04-21T08:40:37.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.247+0000 7f6ad4ff9640 1 -- 172.21.15.43:0/3104939809 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ae4004390 con 0x5568777044d0 2024-04-21T08:40:37.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.247+0000 7f6ad4ff9640 1 -- 172.21.15.43:0/3104939809 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae402cd10 con 0x5568777044d0 2024-04-21T08:40:37.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.247+0000 7f6af7f77740 1 -- 172.21.15.43:0/3104939809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568777044d0 msgr2=0x55687770cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.247+0000 7f6af7f77740 1 --2- 172.21.15.43:0/3104939809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568777044d0 0x55687770cdf0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4000c00 tx=0x7f6ae4005e50 comp rx=0 tx=0).stop 2024-04-21T08:40:37.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.247+0000 7f6af7f77740 1 -- 172.21.15.43:0/3104939809 shutdown_connections 2024-04-21T08:40:37.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.247+0000 7f6af7f77740 1 --2- 172.21.15.43:0/3104939809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55687770da00 0x556877722ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.247+0000 7f6af7f77740 1 --2- 172.21.15.43:0/3104939809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556877704e80 0x55687770d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.247+0000 7f6af7f77740 1 --2- 172.21.15.43:0/3104939809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568777044d0 0x55687770cdf0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.247+0000 7f6af7f77740 1 -- 172.21.15.43:0/3104939809 >> 172.21.15.43:0/3104939809 conn(0x5568776feba0 msgr2=0x556877700840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:37.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.247+0000 7f6af7f77740 1 -- 172.21.15.43:0/3104939809 shutdown_connections 2024-04-21T08:40:37.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.247+0000 7f6af7f77740 1 -- 172.21.15.43:0/3104939809 wait complete. 2024-04-21T08:40:37.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.248+0000 7f6af7f77740 1 -- start start 2024-04-21T08:40:37.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.248+0000 7f6af7f77740 1 Processor -- start 2024-04-21T08:40:37.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.248+0000 7f6af7f77740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568777044d0 0x556877720850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.248+0000 7f6af7f77740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556877704e80 0x556877720d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.248+0000 7f6af7f77740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55687770da00 0x5568777212d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.248+0000 7f6af7f77740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01b28f0 con 0x556877704e80 2024-04-21T08:40:37.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.248+0000 7f6af7f77740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01b2220 con 0x5568777044d0 2024-04-21T08:40:37.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.248+0000 7f6af7f77740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01af460 con 0x55687770da00 2024-04-21T08:40:37.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.248+0000 7f6af7f77740 10 client.0 _ll_register_callbacks cb 0x7f6ae40046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:37.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.248+0000 7f6aeeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556877704e80 0x556877720d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.248+0000 7f6aeeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556877704e80 0x556877720d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50250/0 (socket says 172.21.15.43:50250) 2024-04-21T08:40:37.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.248+0000 7f6aeeffd640 1 -- 172.21.15.43:0/1569353902 learned_addr learned my addr 172.21.15.43:0/1569353902 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:37.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.248+0000 7f6aeeffd640 1 -- 172.21.15.43:0/1569353902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55687770da00 msgr2=0x5568777212d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:37.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.248+0000 7f6aee7fc640 1 --2- 172.21.15.43:0/1569353902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568777044d0 0x556877720850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.248+0000 7f6aeeffd640 1 --2- 172.21.15.43:0/1569353902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55687770da00 0x5568777212d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.248+0000 7f6aeeffd640 1 -- 172.21.15.43:0/1569353902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568777044d0 msgr2=0x556877720850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:37.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.248+0000 7f6aeeffd640 1 --2- 172.21.15.43:0/1569353902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568777044d0 0x556877720850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.248+0000 7f6aeeffd640 1 -- 172.21.15.43:0/1569353902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556877782c70 con 0x556877704e80 2024-04-21T08:40:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.249+0000 7f6aeeffd640 1 --2- 172.21.15.43:0/1569353902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556877704e80 0x556877720d90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009b00 tx=0x7f6ae00117c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a038eccf71be33f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.249+0000 7f6ad2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.249+0000 7f6ad2ff5640 1 -- 172.21.15.43:0/1569353902 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0022070 con 0x556877704e80 2024-04-21T08:40:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.249+0000 7f6ad2ff5640 1 -- 172.21.15.43:0/1569353902 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f6ae001e840 con 0x556877704e80 2024-04-21T08:40:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.249+0000 7f6af7f77740 1 -- 172.21.15.43:0/1569353902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ae4004390 con 0x556877704e80 2024-04-21T08:40:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.249+0000 7f6ad2ff5640 1 -- 172.21.15.43:0/1569353902 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6ae002e050 con 0x556877704e80 2024-04-21T08:40:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.249+0000 7f6af7f77740 10 client.16545 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.249+0000 7f6af7f77740 1 -- 172.21.15.43:0/1569353902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556877783f40 con 0x556877704e80 2024-04-21T08:40:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.249+0000 7f6ad2ff5640 1 -- 172.21.15.43:0/1569353902 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ae4004390 con 0x556877704e80 2024-04-21T08:40:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.249+0000 7f6ad2ff5640 1 -- 172.21.15.43:0/1569353902 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6ae00266e0 con 0x556877704e80 2024-04-21T08:40:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.249+0000 7f6af7f77740 10 client.16545 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.249+0000 7f6af7f77740 10 client.16545 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:40:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.249+0000 7f6af7f77740 20 client.16545 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:37.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.249+0000 7f6af7f77740 1 --2- 172.21.15.43:0/1569353902 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5568777846a0 0x556877786ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.249+0000 7f6af7f77740 4 client.16545 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:40:37.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.249+0000 7f6af7f77740 1 -- 172.21.15.43:0/1569353902 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556877676c10 con 0x5568777846a0 2024-04-21T08:40:37.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.250+0000 7f6aee7fc640 1 --2- 172.21.15.43:0/1569353902 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5568777846a0 0x556877786ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.250+0000 7f6aee7fc640 1 --2- 172.21.15.43:0/1569353902 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5568777846a0 0x556877786ac0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=deb8a93c5ffe55ef server_cookie=148d7c26576ded5d in_seq=0 out_seq=0 2024-04-21T08:40:37.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.250+0000 7f6ad2ff5640 10 client.16545 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:40:37.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.250+0000 7f6ad2ff5640 1 -- 172.21.15.43:0/1569353902 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556877676c10 con 0x5568777846a0 2024-04-21T08:40:37.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.250+0000 7f6ad2ff5640 10 client.16545 handle_command_reply: tid=0 2024-04-21T08:40:37.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.250+0000 7f6af7f77740 1 client.16545 shutdown 2024-04-21T08:40:37.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.250+0000 7f6af7f77740 1 -- 172.21.15.43:0/1569353902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556877704e80 msgr2=0x556877720d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.250+0000 7f6af7f77740 1 --2- 172.21.15.43:0/1569353902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556877704e80 0x556877720d90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009b00 tx=0x7f6ae00117c0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.250+0000 7f6af7f77740 1 -- 172.21.15.43:0/1569353902 shutdown_connections 2024-04-21T08:40:37.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.250+0000 7f6af7f77740 1 --2- 172.21.15.43:0/1569353902 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5568777846a0 0x556877786ac0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.250+0000 7f6af7f77740 1 --2- 172.21.15.43:0/1569353902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55687770da00 0x5568777212d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.250+0000 7f6ad2ff5640 0 client.16545 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:40:37.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.250+0000 7f6af7f77740 1 --2- 172.21.15.43:0/1569353902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556877704e80 0x556877720d90 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.250+0000 7f6af7f77740 1 --2- 172.21.15.43:0/1569353902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568777044d0 0x556877720850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.250+0000 7f6af7f77740 1 -- 172.21.15.43:0/1569353902 >> 172.21.15.43:0/1569353902 conn(0x5568776feba0 msgr2=0x5568776ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:37.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.250+0000 7f6af7f77740 1 -- 172.21.15.43:0/1569353902 shutdown_connections 2024-04-21T08:40:37.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.250+0000 7f6af7f77740 1 -- 172.21.15.43:0/1569353902 wait complete. 2024-04-21T08:40:37.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.250+0000 7f6af7f77740 20 client.16545 trim_cache size 0 max 16384 2024-04-21T08:40:37.257 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:40:37.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.251+0000 7f6af538e640 1 -- 172.21.15.43:0/3929214107 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6ad80a3d20 msgr2=0x7f6ad80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.251+0000 7f6af538e640 1 --2- 172.21.15.43:0/3929214107 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6ad80a3d20 0x7f6ad80a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80cd890 tx=0x7f6ae806ebd0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.251+0000 7f6af538e640 1 -- 172.21.15.43:0/3929214107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6af010c510 msgr2=0x7f6af01018a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.251+0000 7f6af538e640 1 --2- 172.21.15.43:0/3929214107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6af010c510 0x7f6af01018a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800d750 tx=0x7f6ae800dc10 comp rx=0 tx=0).stop 2024-04-21T08:40:37.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.251+0000 7f6af538e640 1 -- 172.21.15.43:0/3929214107 shutdown_connections 2024-04-21T08:40:37.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.251+0000 7f6af538e640 1 --2- 172.21.15.43:0/3929214107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6af010c510 0x7f6af01018a0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.251+0000 7f6af538e640 1 --2- 172.21.15.43:0/3929214107 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6ad80a3d20 0x7f6ad80a61e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.251+0000 7f6af538e640 1 --2- 172.21.15.43:0/3929214107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6af0103af0 0x7f6af0101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.251+0000 7f6af538e640 1 --2- 172.21.15.43:0/3929214107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6af01031d0 0x7f6af0100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.251+0000 7f6af538e640 1 -- 172.21.15.43:0/3929214107 >> 172.21.15.43:0/3929214107 conn(0x7f6af0077f50 msgr2=0x7f6af00fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:37.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.251+0000 7f6af538e640 1 -- 172.21.15.43:0/3929214107 shutdown_connections 2024-04-21T08:40:37.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.251+0000 7f6af538e640 1 -- 172.21.15.43:0/3929214107 wait complete. 2024-04-21T08:40:37.260 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:36 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3919334917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:40:37.261 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:40:37.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:36 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3919334917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:40:37.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.366+0000 7f186ef2e640 1 -- 172.21.15.43:0/2661897283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1868107280 msgr2=0x7f186810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.366+0000 7f186ef2e640 1 --2- 172.21.15.43:0/2661897283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1868107280 0x7f186810ee30 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1860009ca0 tx=0x7f186002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.366+0000 7f186ef2e640 1 -- 172.21.15.43:0/2661897283 shutdown_connections 2024-04-21T08:40:37.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.366+0000 7f186ef2e640 1 --2- 172.21.15.43:0/2661897283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1868107280 0x7f186810ee30 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.366+0000 7f186ef2e640 1 --2- 172.21.15.43:0/2661897283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f186806b170 0x7f1868106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.366+0000 7f186ef2e640 1 --2- 172.21.15.43:0/2661897283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f186806a7c0 0x7f186806aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.366+0000 7f186ef2e640 1 -- 172.21.15.43:0/2661897283 >> 172.21.15.43:0/2661897283 conn(0x7f1868076c50 msgr2=0x7f1868077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:37.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.366+0000 7f186ef2e640 1 -- 172.21.15.43:0/2661897283 shutdown_connections 2024-04-21T08:40:37.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.366+0000 7f186ef2e640 1 -- 172.21.15.43:0/2661897283 wait complete. 2024-04-21T08:40:37.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.367+0000 7f186ef2e640 1 Processor -- start 2024-04-21T08:40:37.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.367+0000 7f186ef2e640 1 -- start start 2024-04-21T08:40:37.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.367+0000 7f186ef2e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f186806a7c0 0x7f18681986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.367+0000 7f186ef2e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1868107280 0x7f1868198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.367+0000 7f186ef2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f186819cf60 0x7f186819d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.367+0000 7f186ef2e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f18681121e0 con 0x7f186819cf60 2024-04-21T08:40:37.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.367+0000 7f186ef2e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1868112060 con 0x7f1868107280 2024-04-21T08:40:37.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.367+0000 7f186ef2e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1868112360 con 0x7f186806a7c0 2024-04-21T08:40:37.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.367+0000 7f186d4cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f186819cf60 0x7f186819d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.367+0000 7f186d4cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f186819cf60 0x7f186819d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50272/0 (socket says 172.21.15.43:50272) 2024-04-21T08:40:37.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.367+0000 7f186d4cb640 1 -- 172.21.15.43:0/1604944759 learned_addr learned my addr 172.21.15.43:0/1604944759 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:37.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.367+0000 7f186ccca640 1 --2- 172.21.15.43:0/1604944759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f186806a7c0 0x7f18681986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.367+0000 7f186d4cb640 1 -- 172.21.15.43:0/1604944759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f186806a7c0 msgr2=0x7f18681986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.367+0000 7f185ffff640 1 --2- 172.21.15.43:0/1604944759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1868107280 0x7f1868198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.367+0000 7f186d4cb640 1 --2- 172.21.15.43:0/1604944759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f186806a7c0 0x7f18681986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.367+0000 7f186d4cb640 1 -- 172.21.15.43:0/1604944759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1868107280 msgr2=0x7f1868198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.368+0000 7f186d4cb640 1 --2- 172.21.15.43:0/1604944759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1868107280 0x7f1868198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.368+0000 7f186d4cb640 1 -- 172.21.15.43:0/1604944759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186819da50 con 0x7f186819cf60 2024-04-21T08:40:37.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.368+0000 7f186d4cb640 1 --2- 172.21.15.43:0/1604944759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f186819cf60 0x7f186819d410 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1860005e30 tx=0x7f1860034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:37.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.368+0000 7f185dffb640 1 -- 172.21.15.43:0/1604944759 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860041020 con 0x7f186819cf60 2024-04-21T08:40:37.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.368+0000 7f186ef2e640 1 -- 172.21.15.43:0/1604944759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f186819dce0 con 0x7f186819cf60 2024-04-21T08:40:37.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.368+0000 7f185dffb640 1 -- 172.21.15.43:0/1604944759 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1860032970 con 0x7f186819cf60 2024-04-21T08:40:37.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.368+0000 7f185dffb640 1 -- 172.21.15.43:0/1604944759 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860039040 con 0x7f186819cf60 2024-04-21T08:40:37.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.368+0000 7f186ef2e640 1 -- 172.21.15.43:0/1604944759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18681a54b0 con 0x7f186819cf60 2024-04-21T08:40:37.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.370+0000 7f185dffb640 1 -- 172.21.15.43:0/1604944759 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1860044020 con 0x7f186819cf60 2024-04-21T08:40:37.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.371+0000 7f1871bf3740 1 Processor -- start 2024-04-21T08:40:37.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.371+0000 7f1871bf3740 1 -- start start 2024-04-21T08:40:37.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.371+0000 7f185dffb640 1 --2- 172.21.15.43:0/1604944759 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f18500a3d20 0x7f18500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.371+0000 7f1871bf3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca5792250 0x55fca5792630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.371+0000 7f1871bf3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca5792c00 0x55fca579f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.371+0000 7f1871bf3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca579fe10 0x55fca57a2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.371+0000 7f1871bf3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fca56f71f0 con 0x55fca579fe10 2024-04-21T08:40:37.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.371+0000 7f1871bf3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fca56944b0 con 0x55fca5792c00 2024-04-21T08:40:37.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.371+0000 7f1871bf3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fca56f31b0 con 0x55fca5792250 2024-04-21T08:40:37.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.371+0000 7f185dffb640 1 -- 172.21.15.43:0/1604944759 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f18600f28a0 con 0x7f186819cf60 2024-04-21T08:40:37.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.371+0000 7f186d4cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca579fe10 0x55fca57a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.371+0000 7f185ffff640 1 --2- 172.21.15.43:0/1604944759 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f18500a3d20 0x7f18500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.371+0000 7f186d4cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca579fe10 0x55fca57a2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50276/0 (socket says 172.21.15.43:50276) 2024-04-21T08:40:37.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.371+0000 7f186d4cb640 1 -- 172.21.15.43:0/2294273626 learned_addr learned my addr 172.21.15.43:0/2294273626 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:37.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.371+0000 7f186ccca640 1 --2- 172.21.15.43:0/2294273626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca5792250 0x55fca5792630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.371+0000 7f185ffff640 1 --2- 172.21.15.43:0/2294273626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca5792c00 0x55fca579f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.371+0000 7f186d4cb640 1 -- 172.21.15.43:0/2294273626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca5792250 msgr2=0x55fca5792630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:37.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.371+0000 7f186d4cb640 1 --2- 172.21.15.43:0/2294273626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca5792250 0x55fca5792630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.371+0000 7f186d4cb640 1 -- 172.21.15.43:0/2294273626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca5792c00 msgr2=0x55fca579f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:37.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.371+0000 7f186d4cb640 1 --2- 172.21.15.43:0/2294273626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca5792c00 0x55fca579f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.371+0000 7f186d4cb640 1 -- 172.21.15.43:0/2294273626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca56861b0 con 0x55fca579fe10 2024-04-21T08:40:37.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.371+0000 7f185ffff640 1 --2- 172.21.15.43:0/2294273626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca5792c00 0x55fca579f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:37.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.372+0000 7f185ffff640 1 --2- 172.21.15.43:0/1604944759 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f18500a3d20 0x7f18500a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1858005fd0 tx=0x7f1858012760 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:37.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.372+0000 7f186d4cb640 1 --2- 172.21.15.43:0/2294273626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca579fe10 0x55fca57a2220 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f18600ba9e0 tx=0x7f186002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df79217e952b216e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:37.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.372+0000 7f184effd640 1 -- 172.21.15.43:0/2294273626 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18600bdee0 con 0x55fca579fe10 2024-04-21T08:40:37.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.372+0000 7f184effd640 1 -- 172.21.15.43:0/2294273626 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18600bec80 con 0x55fca579fe10 2024-04-21T08:40:37.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.372+0000 7f184effd640 1 -- 172.21.15.43:0/2294273626 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186009ba80 con 0x55fca579fe10 2024-04-21T08:40:37.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.372+0000 7f1871bf3740 1 -- 172.21.15.43:0/2294273626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca579fe10 msgr2=0x55fca57a2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.372+0000 7f1871bf3740 1 --2- 172.21.15.43:0/2294273626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca579fe10 0x55fca57a2220 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f18600ba9e0 tx=0x7f186002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.372+0000 7f1871bf3740 1 -- 172.21.15.43:0/2294273626 shutdown_connections 2024-04-21T08:40:37.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.372+0000 7f1871bf3740 1 --2- 172.21.15.43:0/2294273626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca579fe10 0x55fca57a2220 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.372+0000 7f1871bf3740 1 --2- 172.21.15.43:0/2294273626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca5792c00 0x55fca579f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.372+0000 7f1871bf3740 1 --2- 172.21.15.43:0/2294273626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca5792250 0x55fca5792630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.372+0000 7f1871bf3740 1 -- 172.21.15.43:0/2294273626 >> 172.21.15.43:0/2294273626 conn(0x55fca578cba0 msgr2=0x55fca578cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:37.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.373+0000 7f1871bf3740 1 -- 172.21.15.43:0/2294273626 shutdown_connections 2024-04-21T08:40:37.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.373+0000 7f1871bf3740 1 -- 172.21.15.43:0/2294273626 wait complete. 2024-04-21T08:40:37.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.373+0000 7f1871bf3740 1 -- start start 2024-04-21T08:40:37.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.373+0000 7f1871bf3740 1 Processor -- start 2024-04-21T08:40:37.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.373+0000 7f1871bf3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca5792250 0x55fca579f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.373+0000 7f1871bf3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca5792c00 0x55fca579c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.373+0000 7f1871bf3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca579fe10 0x55fca579cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.373+0000 7f1871bf3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fca56f71f0 con 0x55fca579fe10 2024-04-21T08:40:37.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.373+0000 7f1871bf3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fca56b0500 con 0x55fca5792c00 2024-04-21T08:40:37.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.373+0000 7f1871bf3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fca57a2b00 con 0x55fca5792250 2024-04-21T08:40:37.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.373+0000 7f1871bf3740 10 client.0 _ll_register_callbacks cb 0x55fca567eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:37.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.373+0000 7f186d4cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca579fe10 0x55fca579cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.373+0000 7f186d4cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca579fe10 0x55fca579cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50282/0 (socket says 172.21.15.43:50282) 2024-04-21T08:40:37.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.373+0000 7f186d4cb640 1 -- 172.21.15.43:0/2336590870 learned_addr learned my addr 172.21.15.43:0/2336590870 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:37.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.374+0000 7f186ccca640 1 --2- 172.21.15.43:0/2336590870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca5792250 0x55fca579f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.374+0000 7f185ffff640 1 --2- 172.21.15.43:0/2336590870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca5792c00 0x55fca579c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.374+0000 7f186d4cb640 1 -- 172.21.15.43:0/2336590870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca5792250 msgr2=0x55fca579f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:37.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.374+0000 7f186d4cb640 1 --2- 172.21.15.43:0/2336590870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca5792250 0x55fca579f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.374+0000 7f186d4cb640 1 -- 172.21.15.43:0/2336590870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca5792c00 msgr2=0x55fca579c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:37.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.374+0000 7f186d4cb640 1 --2- 172.21.15.43:0/2336590870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca5792c00 0x55fca579c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.374+0000 7f186d4cb640 1 -- 172.21.15.43:0/2336590870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca54093f0 con 0x55fca579fe10 2024-04-21T08:40:37.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.374+0000 7f186d4cb640 1 --2- 172.21.15.43:0/2336590870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca579fe10 0x55fca579cf20 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f18600efde0 tx=0x7f18600f00b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272627a4d7514e81 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:37.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.374+0000 7f1847fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:37.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.374+0000 7f1847fff640 1 -- 172.21.15.43:0/2336590870 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18600d7030 con 0x55fca579fe10 2024-04-21T08:40:37.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.374+0000 7f1847fff640 1 -- 172.21.15.43:0/2336590870 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f186003d3f0 con 0x55fca579fe10 2024-04-21T08:40:37.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.374+0000 7f1871bf3740 1 -- 172.21.15.43:0/2336590870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18600bec80 con 0x55fca579fe10 2024-04-21T08:40:37.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.374+0000 7f1847fff640 1 -- 172.21.15.43:0/2336590870 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f186009bc60 con 0x55fca579fe10 2024-04-21T08:40:37.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.374+0000 7f1871bf3740 10 client.16557 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:37.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.374+0000 7f1871bf3740 1 -- 172.21.15.43:0/2336590870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fca54d37d0 con 0x55fca579fe10 2024-04-21T08:40:37.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.374+0000 7f1847fff640 1 -- 172.21.15.43:0/2336590870 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18600bec80 con 0x55fca579fe10 2024-04-21T08:40:37.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.375+0000 7f1847fff640 1 -- 172.21.15.43:0/2336590870 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1860099060 con 0x55fca579fe10 2024-04-21T08:40:37.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.375+0000 7f1871bf3740 10 client.16557 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:37.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.375+0000 7f1871bf3740 10 client.16557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:37.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.375+0000 7f1871bf3740 20 client.16557 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:37.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.375+0000 7f1871bf3740 1 --2- 172.21.15.43:0/2336590870 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fca57af990 0x55fca57b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.375+0000 7f1871bf3740 4 client.16557 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:37.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.375+0000 7f1871bf3740 1 -- 172.21.15.43:0/2336590870 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fca5696930 con 0x55fca57af990 2024-04-21T08:40:37.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.375+0000 7f186ccca640 1 --2- 172.21.15.43:0/2336590870 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fca57af990 0x55fca57b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.375+0000 7f186ccca640 1 --2- 172.21.15.43:0/2336590870 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fca57af990 0x55fca57b1d80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ee2a84589a1ee93 server_cookie=a81f3b709476454 in_seq=0 out_seq=0 2024-04-21T08:40:37.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.375+0000 7f1847fff640 10 client.16557 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:37.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.376+0000 7f1847fff640 1 -- 172.21.15.43:0/2336590870 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fca5696930 con 0x55fca57af990 2024-04-21T08:40:37.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.376+0000 7f1847fff640 10 client.16557 handle_command_reply: tid=0 2024-04-21T08:40:37.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.376+0000 7f1871bf3740 1 client.16557 shutdown 2024-04-21T08:40:37.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.376+0000 7f1871bf3740 1 -- 172.21.15.43:0/2336590870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca579fe10 msgr2=0x55fca579cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.376+0000 7f1871bf3740 1 --2- 172.21.15.43:0/2336590870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca579fe10 0x55fca579cf20 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f18600efde0 tx=0x7f18600f00b0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.376+0000 7f1871bf3740 1 -- 172.21.15.43:0/2336590870 shutdown_connections 2024-04-21T08:40:37.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.376+0000 7f1871bf3740 1 --2- 172.21.15.43:0/2336590870 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fca57af990 0x55fca57b1d80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.376+0000 7f1871bf3740 1 --2- 172.21.15.43:0/2336590870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca579fe10 0x55fca579cf20 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.376+0000 7f1871bf3740 1 --2- 172.21.15.43:0/2336590870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca5792c00 0x55fca579c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.376+0000 7f1847fff640 0 client.16557 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:37.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.376+0000 7f1871bf3740 1 --2- 172.21.15.43:0/2336590870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca5792250 0x55fca579f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.376+0000 7f1871bf3740 1 -- 172.21.15.43:0/2336590870 >> 172.21.15.43:0/2336590870 conn(0x55fca578cba0 msgr2=0x55fca578e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:37.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.377+0000 7f1871bf3740 1 -- 172.21.15.43:0/2336590870 shutdown_connections 2024-04-21T08:40:37.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.377+0000 7f1871bf3740 1 -- 172.21.15.43:0/2336590870 wait complete. 2024-04-21T08:40:37.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.377+0000 7f1871bf3740 20 client.16557 trim_cache size 0 max 16384 2024-04-21T08:40:37.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.390+0000 7f1871bf3740 1 Processor -- start 2024-04-21T08:40:37.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.390+0000 7f1871bf3740 1 -- start start 2024-04-21T08:40:37.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.390+0000 7f1871bf3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca5792250 0x55fca57ab880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.390+0000 7f1871bf3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca5792c00 0x55fca579f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.390+0000 7f1871bf3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca579fe10 0x55fca579f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.390+0000 7f1871bf3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f18681ad180 con 0x55fca5792250 2024-04-21T08:40:37.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.390+0000 7f1871bf3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f18681aecc0 con 0x55fca5792c00 2024-04-21T08:40:37.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.390+0000 7f1871bf3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18681ad850 con 0x55fca579fe10 2024-04-21T08:40:37.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.390+0000 7f186ccca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca5792250 0x55fca57ab880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.390+0000 7f186ccca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca5792250 0x55fca57ab880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50286/0 (socket says 172.21.15.43:50286) 2024-04-21T08:40:37.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.390+0000 7f186ccca640 1 -- 172.21.15.43:0/3930417282 learned_addr learned my addr 172.21.15.43:0/3930417282 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:37.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.390+0000 7f185ffff640 1 --2- 172.21.15.43:0/3930417282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca5792c00 0x55fca579f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.390+0000 7f186ccca640 1 -- 172.21.15.43:0/3930417282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca579fe10 msgr2=0x55fca579f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:37.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.390+0000 7f186ccca640 1 --2- 172.21.15.43:0/3930417282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca579fe10 0x55fca579f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.390+0000 7f186ccca640 1 -- 172.21.15.43:0/3930417282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca5792c00 msgr2=0x55fca579f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:37.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.390+0000 7f186ccca640 1 --2- 172.21.15.43:0/3930417282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca5792c00 0x55fca579f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.390+0000 7f186ccca640 1 -- 172.21.15.43:0/3930417282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca57929a0 con 0x55fca5792250 2024-04-21T08:40:37.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.390+0000 7f185ffff640 1 --2- 172.21.15.43:0/3930417282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca5792c00 0x55fca579f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:37.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.391+0000 7f186ccca640 1 --2- 172.21.15.43:0/3930417282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca5792250 0x55fca57ab880 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x55fca57b2550 tx=0x7f1864005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c32a9b1d3a3a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:37.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.391+0000 7f184e7fc640 1 -- 172.21.15.43:0/3930417282 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186401f250 con 0x55fca5792250 2024-04-21T08:40:37.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.391+0000 7f184e7fc640 1 -- 172.21.15.43:0/3930417282 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f186401f8a0 con 0x55fca5792250 2024-04-21T08:40:37.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.391+0000 7f184e7fc640 1 -- 172.21.15.43:0/3930417282 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186401edd0 con 0x55fca5792250 2024-04-21T08:40:37.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.391+0000 7f1871bf3740 1 -- 172.21.15.43:0/3930417282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca5792250 msgr2=0x55fca57ab880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.391+0000 7f1871bf3740 1 --2- 172.21.15.43:0/3930417282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca5792250 0x55fca57ab880 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x55fca57b2550 tx=0x7f1864005f90 comp rx=0 tx=0).stop 2024-04-21T08:40:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.391+0000 7f1871bf3740 1 -- 172.21.15.43:0/3930417282 shutdown_connections 2024-04-21T08:40:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.391+0000 7f1871bf3740 1 --2- 172.21.15.43:0/3930417282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca579fe10 0x55fca579f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.391+0000 7f1871bf3740 1 --2- 172.21.15.43:0/3930417282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca5792c00 0x55fca579f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.391+0000 7f1871bf3740 1 --2- 172.21.15.43:0/3930417282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca5792250 0x55fca57ab880 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.391+0000 7f1871bf3740 1 -- 172.21.15.43:0/3930417282 >> 172.21.15.43:0/3930417282 conn(0x55fca578cba0 msgr2=0x55fca578e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.391+0000 7f1871bf3740 1 -- 172.21.15.43:0/3930417282 shutdown_connections 2024-04-21T08:40:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.391+0000 7f1871bf3740 1 -- 172.21.15.43:0/3930417282 wait complete. 2024-04-21T08:40:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.392+0000 7f1871bf3740 1 -- start start 2024-04-21T08:40:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.392+0000 7f1871bf3740 1 Processor -- start 2024-04-21T08:40:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.392+0000 7f1871bf3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca5792250 0x55fca57a8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.392+0000 7f1871bf3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca5792c00 0x55fca57993e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.392+0000 7f1871bf3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca579fe10 0x55fca5799920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.392+0000 7f1871bf3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f18681ae5f0 con 0x55fca5792250 2024-04-21T08:40:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.392+0000 7f1871bf3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f18681adf20 con 0x55fca5792c00 2024-04-21T08:40:37.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.392+0000 7f1871bf3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18681ab160 con 0x55fca579fe10 2024-04-21T08:40:37.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.392+0000 7f1871bf3740 10 client.0 _ll_register_callbacks cb 0x7f1860009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:37.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.392+0000 7f186ccca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca5792250 0x55fca57a8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.392+0000 7f186ccca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca5792250 0x55fca57a8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50288/0 (socket says 172.21.15.43:50288) 2024-04-21T08:40:37.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.392+0000 7f186ccca640 1 -- 172.21.15.43:0/465313116 learned_addr learned my addr 172.21.15.43:0/465313116 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:37.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.393+0000 7f185ffff640 1 --2- 172.21.15.43:0/465313116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca5792c00 0x55fca57993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.393+0000 7f186d4cb640 1 --2- 172.21.15.43:0/465313116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca579fe10 0x55fca5799920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.393+0000 7f186ccca640 1 -- 172.21.15.43:0/465313116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca579fe10 msgr2=0x55fca5799920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:37.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.393+0000 7f186ccca640 1 --2- 172.21.15.43:0/465313116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca579fe10 0x55fca5799920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.393+0000 7f186ccca640 1 -- 172.21.15.43:0/465313116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca5792c00 msgr2=0x55fca57993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:37.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.393+0000 7f186ccca640 1 --2- 172.21.15.43:0/465313116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca5792c00 0x55fca57993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.393+0000 7f186ccca640 1 -- 172.21.15.43:0/465313116 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca579a150 con 0x55fca5792250 2024-04-21T08:40:37.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.393+0000 7f186ccca640 1 --2- 172.21.15.43:0/465313116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca5792250 0x55fca57a8160 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f186401ece0 tx=0x7f186401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac438a865b6b9c45 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:37.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.393+0000 7f1847fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:37.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.393+0000 7f1847fff640 1 -- 172.21.15.43:0/465313116 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186401e420 con 0x55fca5792250 2024-04-21T08:40:37.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.393+0000 7f1847fff640 1 -- 172.21.15.43:0/465313116 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f186401e5c0 con 0x55fca5792250 2024-04-21T08:40:37.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.393+0000 7f1871bf3740 1 -- 172.21.15.43:0/465313116 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f186401f8a0 con 0x55fca5792250 2024-04-21T08:40:37.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.393+0000 7f1847fff640 1 -- 172.21.15.43:0/465313116 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f186401e7c0 con 0x55fca5792250 2024-04-21T08:40:37.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.393+0000 7f1871bf3740 10 client.16563 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:37.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.393+0000 7f1871bf3740 1 -- 172.21.15.43:0/465313116 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fca5815d20 con 0x55fca5792250 2024-04-21T08:40:37.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.393+0000 7f1847fff640 1 -- 172.21.15.43:0/465313116 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f186401f8a0 con 0x55fca5792250 2024-04-21T08:40:37.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.394+0000 7f1847fff640 1 -- 172.21.15.43:0/465313116 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f186401f410 con 0x55fca5792250 2024-04-21T08:40:37.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.394+0000 7f1871bf3740 10 client.16563 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:37.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.394+0000 7f1871bf3740 10 client.16563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:37.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.394+0000 7f1871bf3740 20 client.16563 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:37.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.394+0000 7f1871bf3740 1 --2- 172.21.15.43:0/465313116 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fca58163d0 0x55fca58187f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:37.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.394+0000 7f1871bf3740 4 client.16563 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:40:37.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.394+0000 7f1871bf3740 1 -- 172.21.15.43:0/465313116 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fca5704c10 con 0x55fca58163d0 2024-04-21T08:40:37.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.394+0000 7f185ffff640 1 --2- 172.21.15.43:0/465313116 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fca58163d0 0x55fca58187f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:37.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.394+0000 7f185ffff640 1 --2- 172.21.15.43:0/465313116 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fca58163d0 0x55fca58187f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a08d7c06cf83847 server_cookie=803eb4fd28bfd5a in_seq=0 out_seq=0 2024-04-21T08:40:37.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.394+0000 7f1847fff640 10 client.16563 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:37.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.394+0000 7f1847fff640 1 -- 172.21.15.43:0/465313116 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fca5704c10 con 0x55fca58163d0 2024-04-21T08:40:37.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.394+0000 7f1847fff640 10 client.16563 handle_command_reply: tid=0 2024-04-21T08:40:37.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.394+0000 7f1871bf3740 1 client.16563 shutdown 2024-04-21T08:40:37.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.395+0000 7f1871bf3740 1 -- 172.21.15.43:0/465313116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca5792250 msgr2=0x55fca57a8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.395+0000 7f1871bf3740 1 --2- 172.21.15.43:0/465313116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca5792250 0x55fca57a8160 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f186401ece0 tx=0x7f186401ed10 comp rx=0 tx=0).stop 2024-04-21T08:40:37.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.395+0000 7f1871bf3740 1 -- 172.21.15.43:0/465313116 shutdown_connections 2024-04-21T08:40:37.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.395+0000 7f1871bf3740 1 --2- 172.21.15.43:0/465313116 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fca58163d0 0x55fca58187f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.395+0000 7f1871bf3740 1 --2- 172.21.15.43:0/465313116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fca579fe10 0x55fca5799920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.395+0000 7f1847fff640 0 client.16563 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:37.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.395+0000 7f1871bf3740 1 --2- 172.21.15.43:0/465313116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca5792c00 0x55fca57993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.395+0000 7f1871bf3740 1 --2- 172.21.15.43:0/465313116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fca5792250 0x55fca57a8160 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.395+0000 7f1871bf3740 1 -- 172.21.15.43:0/465313116 >> 172.21.15.43:0/465313116 conn(0x55fca578cba0 msgr2=0x55fca578df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:37.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.395+0000 7f1871bf3740 1 -- 172.21.15.43:0/465313116 shutdown_connections 2024-04-21T08:40:37.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.395+0000 7f1871bf3740 1 -- 172.21.15.43:0/465313116 wait complete. 2024-04-21T08:40:37.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.395+0000 7f1871bf3740 20 client.16563 trim_cache size 0 max 16384 2024-04-21T08:40:37.401 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:40:37.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.395+0000 7f186ef2e640 1 -- 172.21.15.43:0/1604944759 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f18500a3d20 msgr2=0x7f18500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.395+0000 7f186ef2e640 1 --2- 172.21.15.43:0/1604944759 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f18500a3d20 0x7f18500a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1858005fd0 tx=0x7f1858012760 comp rx=0 tx=0).stop 2024-04-21T08:40:37.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.395+0000 7f186ef2e640 1 -- 172.21.15.43:0/1604944759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f186819cf60 msgr2=0x7f186819d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:37.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.395+0000 7f186ef2e640 1 --2- 172.21.15.43:0/1604944759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f186819cf60 0x7f186819d410 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1860005e30 tx=0x7f1860034670 comp rx=0 tx=0).stop 2024-04-21T08:40:37.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.395+0000 7f186ef2e640 1 -- 172.21.15.43:0/1604944759 shutdown_connections 2024-04-21T08:40:37.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.395+0000 7f186ef2e640 1 --2- 172.21.15.43:0/1604944759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f186819cf60 0x7f186819d410 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.395+0000 7f186ef2e640 1 --2- 172.21.15.43:0/1604944759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1868107280 0x7f1868198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.395+0000 7f186ef2e640 1 --2- 172.21.15.43:0/1604944759 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f18500a3d20 0x7f18500a61e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.395+0000 7f186ef2e640 1 --2- 172.21.15.43:0/1604944759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f186806a7c0 0x7f18681986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:37.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.395+0000 7f186ef2e640 1 -- 172.21.15.43:0/1604944759 >> 172.21.15.43:0/1604944759 conn(0x7f1868076c50 msgr2=0x7f18680ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:37.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.396+0000 7f186ef2e640 1 -- 172.21.15.43:0/1604944759 shutdown_connections 2024-04-21T08:40:37.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:37.396+0000 7f186ef2e640 1 -- 172.21.15.43:0/1604944759 wait complete. 2024-04-21T08:40:37.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:36 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3919334917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:40:38.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:37 smithi112 ceph-mon[31085]: pgmap v54: 129 pgs: 129 active+clean; 2.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 26 op/s 2024-04-21T08:40:38.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:37 smithi043 ceph-mon[27978]: pgmap v54: 129 pgs: 129 active+clean; 2.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 26 op/s 2024-04-21T08:40:38.406 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:40:38.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:40:38.406 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:40:38.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:37 smithi175 ceph-mon[31920]: pgmap v54: 129 pgs: 129 active+clean; 2.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 26 op/s 2024-04-21T08:40:38.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.526+0000 7f3f67e81640 1 -- 172.21.15.43:0/2413111915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f60106170 msgr2=0x7f3f6010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:38.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.526+0000 7f3f67e81640 1 --2- 172.21.15.43:0/2413111915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f60106170 0x7f3f6010dd40 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009ca0 tx=0x7f3f5802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.526+0000 7f3f67e81640 1 -- 172.21.15.43:0/2413111915 shutdown_connections 2024-04-21T08:40:38.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.526+0000 7f3f67e81640 1 --2- 172.21.15.43:0/2413111915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f60106170 0x7f3f6010dd40 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.526+0000 7f3f67e81640 1 --2- 172.21.15.43:0/2413111915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f60101a50 0x7f3f60105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.526+0000 7f3f67e81640 1 --2- 172.21.15.43:0/2413111915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f601010a0 0x7f3f60101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.526+0000 7f3f67e81640 1 -- 172.21.15.43:0/2413111915 >> 172.21.15.43:0/2413111915 conn(0x7f3f60077eb0 msgr2=0x7f3f600782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:38.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.526+0000 7f3f67e81640 1 -- 172.21.15.43:0/2413111915 shutdown_connections 2024-04-21T08:40:38.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.526+0000 7f3f67e81640 1 -- 172.21.15.43:0/2413111915 wait complete. 2024-04-21T08:40:38.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.527+0000 7f3f67e81640 1 Processor -- start 2024-04-21T08:40:38.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.527+0000 7f3f67e81640 1 -- start start 2024-04-21T08:40:38.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.527+0000 7f3f67e81640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f601010a0 0x7f3f60199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.527+0000 7f3f67e81640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f60106170 0x7f3f60199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.527+0000 7f3f67e81640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f60199ff0 0x7f3f601a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.527+0000 7f3f67e81640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3f601110e0 con 0x7f3f60106170 2024-04-21T08:40:38.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.527+0000 7f3f67e81640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f60110f60 con 0x7f3f60199ff0 2024-04-21T08:40:38.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.527+0000 7f3f67e81640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3f60111260 con 0x7f3f601010a0 2024-04-21T08:40:38.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.527+0000 7f3f6541c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f60106170 0x7f3f60199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.527+0000 7f3f6541c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f60106170 0x7f3f60199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50306/0 (socket says 172.21.15.43:50306) 2024-04-21T08:40:38.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.527+0000 7f3f6541c640 1 -- 172.21.15.43:0/1106035666 learned_addr learned my addr 172.21.15.43:0/1106035666 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:38.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.528+0000 7f3f65c1d640 1 --2- 172.21.15.43:0/1106035666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f601010a0 0x7f3f60199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.528+0000 7f3f6641e640 1 --2- 172.21.15.43:0/1106035666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f60199ff0 0x7f3f601a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.528+0000 7f3f6541c640 1 -- 172.21.15.43:0/1106035666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f601010a0 msgr2=0x7f3f60199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:38.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.528+0000 7f3f6541c640 1 --2- 172.21.15.43:0/1106035666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f601010a0 0x7f3f60199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.528+0000 7f3f6541c640 1 -- 172.21.15.43:0/1106035666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f60199ff0 msgr2=0x7f3f601a45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:38.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.528+0000 7f3f6541c640 1 --2- 172.21.15.43:0/1106035666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f60199ff0 0x7f3f601a45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.528+0000 7f3f6541c640 1 -- 172.21.15.43:0/1106035666 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f601a4da0 con 0x7f3f60106170 2024-04-21T08:40:38.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.528+0000 7f3f65c1d640 1 --2- 172.21.15.43:0/1106035666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f601010a0 0x7f3f60199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:38.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.528+0000 7f3f6641e640 1 --2- 172.21.15.43:0/1106035666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f60199ff0 0x7f3f601a45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:38.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.528+0000 7f3f6541c640 1 --2- 172.21.15.43:0/1106035666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f60106170 0x7f3f60199a60 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000d770 tx=0x7f3f5000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:38.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.528+0000 7f3f56ffd640 1 -- 172.21.15.43:0/1106035666 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f50017070 con 0x7f3f60106170 2024-04-21T08:40:38.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.528+0000 7f3f56ffd640 1 -- 172.21.15.43:0/1106035666 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f50009c30 con 0x7f3f60106170 2024-04-21T08:40:38.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.528+0000 7f3f67e81640 1 -- 172.21.15.43:0/1106035666 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f601a5090 con 0x7f3f60106170 2024-04-21T08:40:38.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.528+0000 7f3f56ffd640 1 -- 172.21.15.43:0/1106035666 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5000f620 con 0x7f3f60106170 2024-04-21T08:40:38.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.528+0000 7f3f67e81640 1 -- 172.21.15.43:0/1106035666 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f601a55e0 con 0x7f3f60106170 2024-04-21T08:40:38.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.531+0000 7f3f56ffd640 1 -- 172.21.15.43:0/1106035666 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3f5001a070 con 0x7f3f60106170 2024-04-21T08:40:38.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.531+0000 7f3f6aa79740 1 Processor -- start 2024-04-21T08:40:38.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.531+0000 7f3f6aa79740 1 -- start start 2024-04-21T08:40:38.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.531+0000 7f3f6aa79740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efc8e0 0x564119efccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.531+0000 7f3f6aa79740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564119efd290 0x564119f05840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.531+0000 7f3f6aa79740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564119f05e10 0x564119f08260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.531+0000 7f3f6aa79740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564119e60f00 con 0x564119efc8e0 2024-04-21T08:40:38.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.531+0000 7f3f6aa79740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564119dfe590 con 0x564119efd290 2024-04-21T08:40:38.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.531+0000 7f3f6aa79740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564119e5cec0 con 0x564119f05e10 2024-04-21T08:40:38.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.531+0000 7f3f6641e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efc8e0 0x564119efccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.531+0000 7f3f65c1d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564119efd290 0x564119f05840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.531+0000 7f3f6641e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efc8e0 0x564119efccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50310/0 (socket says 172.21.15.43:50310) 2024-04-21T08:40:38.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.531+0000 7f3f6641e640 1 -- 172.21.15.43:0/697441716 learned_addr learned my addr 172.21.15.43:0/697441716 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:38.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.532+0000 7f3f6541c640 1 --2- 172.21.15.43:0/697441716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564119f05e10 0x564119f08260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.532+0000 7f3f56ffd640 1 --2- 172.21.15.43:0/1106035666 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3f4c0a3d20 0x7f3f4c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.532+0000 7f3f65c1d640 1 -- 172.21.15.43:0/697441716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564119f05e10 msgr2=0x564119f08260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:38.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.532+0000 7f3f65c1d640 1 --2- 172.21.15.43:0/697441716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564119f05e10 0x564119f08260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.532+0000 7f3f65c1d640 1 -- 172.21.15.43:0/697441716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efc8e0 msgr2=0x564119efccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:38.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.532+0000 7f3f65c1d640 1 --2- 172.21.15.43:0/697441716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efc8e0 0x564119efccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.532+0000 7f3f65c1d640 1 -- 172.21.15.43:0/697441716 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564119c3b7e0 con 0x564119efd290 2024-04-21T08:40:38.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.532+0000 7f3f6641e640 1 --2- 172.21.15.43:0/697441716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efc8e0 0x564119efccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:40:38.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.532+0000 7f3f6541c640 1 --2- 172.21.15.43:0/697441716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564119f05e10 0x564119f08260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:40:38.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.532+0000 7f3f6641e640 1 --2- 172.21.15.43:0/1106035666 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3f4c0a3d20 0x7f3f4c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.532+0000 7f3f56ffd640 1 -- 172.21.15.43:0/1106035666 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f3f500d0760 con 0x7f3f60106170 2024-04-21T08:40:38.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.532+0000 7f3f65c1d640 1 --2- 172.21.15.43:0/697441716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564119efd290 0x564119f05840 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00bc80 tx=0x7f3f5c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57acc112bebde906 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:38.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.532+0000 7f3f4bfff640 1 -- 172.21.15.43:0/697441716 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c015070 con 0x564119efd290 2024-04-21T08:40:38.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.532+0000 7f3f6641e640 1 --2- 172.21.15.43:0/1106035666 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3f4c0a3d20 0x7f3f4c0a61e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009430 tx=0x7f3f5802e040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:38.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.532+0000 7f3f4bfff640 1 -- 172.21.15.43:0/697441716 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f5c0118f0 con 0x564119efd290 2024-04-21T08:40:38.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.532+0000 7f3f4bfff640 1 -- 172.21.15.43:0/697441716 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c010870 con 0x564119efd290 2024-04-21T08:40:38.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.533+0000 7f3f6aa79740 1 -- 172.21.15.43:0/697441716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564119efd290 msgr2=0x564119f05840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:38.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.533+0000 7f3f6aa79740 1 --2- 172.21.15.43:0/697441716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564119efd290 0x564119f05840 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00bc80 tx=0x7f3f5c00f730 comp rx=0 tx=0).stop 2024-04-21T08:40:38.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.533+0000 7f3f6aa79740 1 -- 172.21.15.43:0/697441716 shutdown_connections 2024-04-21T08:40:38.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.533+0000 7f3f6aa79740 1 --2- 172.21.15.43:0/697441716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564119f05e10 0x564119f08260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.533+0000 7f3f6aa79740 1 --2- 172.21.15.43:0/697441716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564119efd290 0x564119f05840 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.533+0000 7f3f6aa79740 1 --2- 172.21.15.43:0/697441716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efc8e0 0x564119efccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.533+0000 7f3f6aa79740 1 -- 172.21.15.43:0/697441716 >> 172.21.15.43:0/697441716 conn(0x564119ef6fb0 msgr2=0x564119ef7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:38.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.533+0000 7f3f6aa79740 1 -- 172.21.15.43:0/697441716 shutdown_connections 2024-04-21T08:40:38.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.533+0000 7f3f6aa79740 1 -- 172.21.15.43:0/697441716 wait complete. 2024-04-21T08:40:38.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.534+0000 7f3f6aa79740 1 -- start start 2024-04-21T08:40:38.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.534+0000 7f3f6aa79740 1 Processor -- start 2024-04-21T08:40:38.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.534+0000 7f3f6aa79740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564119efc8e0 0x564119f02e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.534+0000 7f3f6aa79740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efd290 0x564119f03340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.534+0000 7f3f6aa79740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564119f05e10 0x564119f03880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.534+0000 7f3f6aa79740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564119f08d80 con 0x564119efd290 2024-04-21T08:40:38.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.534+0000 7f3f6aa79740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564119e1a580 con 0x564119f05e10 2024-04-21T08:40:38.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.534+0000 7f3f6aa79740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564119f08f00 con 0x564119efc8e0 2024-04-21T08:40:38.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.534+0000 7f3f6aa79740 10 client.0 _ll_register_callbacks cb 0x564119e20650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:38.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.534+0000 7f3f6541c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efd290 0x564119f03340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.534+0000 7f3f6541c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efd290 0x564119f03340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50312/0 (socket says 172.21.15.43:50312) 2024-04-21T08:40:38.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.534+0000 7f3f6541c640 1 -- 172.21.15.43:0/41287990 learned_addr learned my addr 172.21.15.43:0/41287990 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:38.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.535+0000 7f3f65c1d640 1 --2- 172.21.15.43:0/41287990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564119efc8e0 0x564119f02e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.535+0000 7f3f6641e640 1 --2- 172.21.15.43:0/41287990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564119f05e10 0x564119f03880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.535+0000 7f3f6541c640 1 -- 172.21.15.43:0/41287990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564119efc8e0 msgr2=0x564119f02e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:38.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.535+0000 7f3f6541c640 1 --2- 172.21.15.43:0/41287990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564119efc8e0 0x564119f02e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.535+0000 7f3f6541c640 1 -- 172.21.15.43:0/41287990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564119f05e10 msgr2=0x564119f03880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:38.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.535+0000 7f3f6541c640 1 --2- 172.21.15.43:0/41287990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564119f05e10 0x564119f03880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.535+0000 7f3f6541c640 1 -- 172.21.15.43:0/41287990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564119c9bf70 con 0x564119efd290 2024-04-21T08:40:38.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.535+0000 7f3f6641e640 1 --2- 172.21.15.43:0/41287990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564119f05e10 0x564119f03880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:38.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.535+0000 7f3f6541c640 1 --2- 172.21.15.43:0/41287990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efd290 0x564119f03340 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009610 tx=0x7f3f5009c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2031bee6c3496c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:38.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.535+0000 7f3f497fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:38.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.535+0000 7f3f497fa640 1 -- 172.21.15.43:0/41287990 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f50021070 con 0x564119efd290 2024-04-21T08:40:38.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.535+0000 7f3f497fa640 1 -- 172.21.15.43:0/41287990 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f3f500d02e0 con 0x564119efd290 2024-04-21T08:40:38.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.535+0000 7f3f6aa79740 1 -- 172.21.15.43:0/41287990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f5c0118f0 con 0x564119efd290 2024-04-21T08:40:38.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.535+0000 7f3f497fa640 1 -- 172.21.15.43:0/41287990 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3f5009bc90 con 0x564119efd290 2024-04-21T08:40:38.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.535+0000 7f3f6aa79740 10 client.16575 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:38.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.535+0000 7f3f6aa79740 1 -- 172.21.15.43:0/41287990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564119b33170 con 0x564119efd290 2024-04-21T08:40:38.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.535+0000 7f3f497fa640 1 -- 172.21.15.43:0/41287990 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f5c0118f0 con 0x564119efd290 2024-04-21T08:40:38.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.536+0000 7f3f497fa640 1 -- 172.21.15.43:0/41287990 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3f500a0160 con 0x564119efd290 2024-04-21T08:40:38.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.536+0000 7f3f6aa79740 10 client.16575 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:38.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.536+0000 7f3f6aa79740 10 client.16575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:38.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.536+0000 7f3f6aa79740 20 client.16575 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:38.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.536+0000 7f3f6aa79740 1 --2- 172.21.15.43:0/41287990 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564119f16460 0x564119f188b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.536+0000 7f3f6aa79740 4 client.16575 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:38.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.536+0000 7f3f6aa79740 1 -- 172.21.15.43:0/41287990 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564119e6e920 con 0x564119f16460 2024-04-21T08:40:38.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.536+0000 7f3f65c1d640 1 --2- 172.21.15.43:0/41287990 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564119f16460 0x564119f188b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.536+0000 7f3f65c1d640 1 --2- 172.21.15.43:0/41287990 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564119f16460 0x564119f188b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb6c26f7a5374b31 server_cookie=a9aaa7b3df285142 in_seq=0 out_seq=0 2024-04-21T08:40:38.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.536+0000 7f3f497fa640 10 client.16575 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:38.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.537+0000 7f3f497fa640 1 -- 172.21.15.43:0/41287990 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564119e6e920 con 0x564119f16460 2024-04-21T08:40:38.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.537+0000 7f3f497fa640 10 client.16575 handle_command_reply: tid=0 2024-04-21T08:40:38.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.537+0000 7f3f6aa79740 1 client.16575 shutdown 2024-04-21T08:40:38.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.537+0000 7f3f6aa79740 1 -- 172.21.15.43:0/41287990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efd290 msgr2=0x564119f03340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:38.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.537+0000 7f3f6aa79740 1 --2- 172.21.15.43:0/41287990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efd290 0x564119f03340 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009610 tx=0x7f3f5009c100 comp rx=0 tx=0).stop 2024-04-21T08:40:38.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.537+0000 7f3f6aa79740 1 -- 172.21.15.43:0/41287990 shutdown_connections 2024-04-21T08:40:38.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.537+0000 7f3f6aa79740 1 --2- 172.21.15.43:0/41287990 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564119f16460 0x564119f188b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.537+0000 7f3f6aa79740 1 --2- 172.21.15.43:0/41287990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564119f05e10 0x564119f03880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.537+0000 7f3f6aa79740 1 --2- 172.21.15.43:0/41287990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efd290 0x564119f03340 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.537+0000 7f3f6aa79740 1 --2- 172.21.15.43:0/41287990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564119efc8e0 0x564119f02e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.537+0000 7f3f497fa640 0 client.16575 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:38.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.537+0000 7f3f6aa79740 1 -- 172.21.15.43:0/41287990 >> 172.21.15.43:0/41287990 conn(0x564119ef6fb0 msgr2=0x564119ef8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:38.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.537+0000 7f3f6aa79740 1 -- 172.21.15.43:0/41287990 shutdown_connections 2024-04-21T08:40:38.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.537+0000 7f3f6aa79740 1 -- 172.21.15.43:0/41287990 wait complete. 2024-04-21T08:40:38.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.538+0000 7f3f6aa79740 20 client.16575 trim_cache size 0 max 16384 2024-04-21T08:40:38.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.555+0000 7f3f6aa79740 1 Processor -- start 2024-04-21T08:40:38.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.555+0000 7f3f6aa79740 1 -- start start 2024-04-21T08:40:38.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.555+0000 7f3f6aa79740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564119ef6fb0 0x564119f11c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.555+0000 7f3f6aa79740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564119efc8e0 0x564119f034d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.555+0000 7f3f6aa79740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efd290 0x564119f1e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.555+0000 7f3f6aa79740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3f601af3a0 con 0x564119efd290 2024-04-21T08:40:38.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.555+0000 7f3f6aa79740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f601b0ee0 con 0x564119efc8e0 2024-04-21T08:40:38.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.555+0000 7f3f6aa79740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3f601afa70 con 0x564119ef6fb0 2024-04-21T08:40:38.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.556+0000 7f3f6641e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efd290 0x564119f1e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.556+0000 7f3f6641e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efd290 0x564119f1e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50328/0 (socket says 172.21.15.43:50328) 2024-04-21T08:40:38.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.556+0000 7f3f6641e640 1 -- 172.21.15.43:0/1517766458 learned_addr learned my addr 172.21.15.43:0/1517766458 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:38.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.556+0000 7f3f65c1d640 1 --2- 172.21.15.43:0/1517766458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564119ef6fb0 0x564119f11c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.556+0000 7f3f6541c640 1 --2- 172.21.15.43:0/1517766458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564119efc8e0 0x564119f034d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.556+0000 7f3f6641e640 1 -- 172.21.15.43:0/1517766458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564119ef6fb0 msgr2=0x564119f11c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:38.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.556+0000 7f3f6641e640 1 --2- 172.21.15.43:0/1517766458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564119ef6fb0 0x564119f11c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.556+0000 7f3f6641e640 1 -- 172.21.15.43:0/1517766458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564119efc8e0 msgr2=0x564119f034d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:38.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.556+0000 7f3f6641e640 1 --2- 172.21.15.43:0/1517766458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564119efc8e0 0x564119f034d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.556+0000 7f3f6641e640 1 -- 172.21.15.43:0/1517766458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564119f14fb0 con 0x564119efd290 2024-04-21T08:40:38.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.556+0000 7f3f6541c640 1 --2- 172.21.15.43:0/1517766458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564119efc8e0 0x564119f034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:38.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.556+0000 7f3f6641e640 1 --2- 172.21.15.43:0/1517766458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efd290 0x564119f1e180 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3f5802c700 tx=0x7f3f58040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f4c5aee7ec744a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:38.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.556+0000 7f3f4b7fe640 1 -- 172.21.15.43:0/1517766458 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f58039040 con 0x564119efd290 2024-04-21T08:40:38.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.556+0000 7f3f4b7fe640 1 -- 172.21.15.43:0/1517766458 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f58052070 con 0x564119efd290 2024-04-21T08:40:38.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.556+0000 7f3f4b7fe640 1 -- 172.21.15.43:0/1517766458 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f580346a0 con 0x564119efd290 2024-04-21T08:40:38.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.557+0000 7f3f6aa79740 1 -- 172.21.15.43:0/1517766458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efd290 msgr2=0x564119f1e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:38.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.557+0000 7f3f6aa79740 1 --2- 172.21.15.43:0/1517766458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efd290 0x564119f1e180 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3f5802c700 tx=0x7f3f58040000 comp rx=0 tx=0).stop 2024-04-21T08:40:38.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.557+0000 7f3f6aa79740 1 -- 172.21.15.43:0/1517766458 shutdown_connections 2024-04-21T08:40:38.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.557+0000 7f3f6aa79740 1 --2- 172.21.15.43:0/1517766458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efd290 0x564119f1e180 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.557+0000 7f3f6aa79740 1 --2- 172.21.15.43:0/1517766458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564119efc8e0 0x564119f034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.557+0000 7f3f6aa79740 1 --2- 172.21.15.43:0/1517766458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564119ef6fb0 0x564119f11c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.557+0000 7f3f6aa79740 1 -- 172.21.15.43:0/1517766458 >> 172.21.15.43:0/1517766458 conn(0x564119f01a40 msgr2=0x564119f02dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:38.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.557+0000 7f3f6aa79740 1 -- 172.21.15.43:0/1517766458 shutdown_connections 2024-04-21T08:40:38.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.557+0000 7f3f6aa79740 1 -- 172.21.15.43:0/1517766458 wait complete. 2024-04-21T08:40:38.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.557+0000 7f3f6aa79740 1 -- start start 2024-04-21T08:40:38.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.557+0000 7f3f6aa79740 1 Processor -- start 2024-04-21T08:40:38.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.558+0000 7f3f6aa79740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564119ef6fb0 0x564119f0ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.558+0000 7f3f6aa79740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564119efc8e0 0x564119f0b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.558+0000 7f3f6aa79740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efd290 0x564119f0b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.558+0000 7f3f6aa79740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3f601b0810 con 0x564119efd290 2024-04-21T08:40:38.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.558+0000 7f3f6aa79740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f601b0140 con 0x564119ef6fb0 2024-04-21T08:40:38.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.558+0000 7f3f6aa79740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3f601ad380 con 0x564119efc8e0 2024-04-21T08:40:38.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.558+0000 7f3f6aa79740 10 client.0 _ll_register_callbacks cb 0x7f3f500d6240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:38.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.558+0000 7f3f6641e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efd290 0x564119f0b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.558+0000 7f3f6641e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efd290 0x564119f0b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50340/0 (socket says 172.21.15.43:50340) 2024-04-21T08:40:38.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.558+0000 7f3f6641e640 1 -- 172.21.15.43:0/2678101955 learned_addr learned my addr 172.21.15.43:0/2678101955 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:38.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.558+0000 7f3f65c1d640 1 --2- 172.21.15.43:0/2678101955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564119ef6fb0 0x564119f0ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.558+0000 7f3f6541c640 1 --2- 172.21.15.43:0/2678101955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564119efc8e0 0x564119f0b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.558+0000 7f3f6641e640 1 -- 172.21.15.43:0/2678101955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564119efc8e0 msgr2=0x564119f0b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:38.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.558+0000 7f3f6641e640 1 --2- 172.21.15.43:0/2678101955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564119efc8e0 0x564119f0b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.558+0000 7f3f6641e640 1 -- 172.21.15.43:0/2678101955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564119ef6fb0 msgr2=0x564119f0ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:38.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.558+0000 7f3f6641e640 1 --2- 172.21.15.43:0/2678101955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564119ef6fb0 0x564119f0ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.558+0000 7f3f6641e640 1 -- 172.21.15.43:0/2678101955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564119f80e90 con 0x564119efd290 2024-04-21T08:40:38.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.558+0000 7f3f6541c640 1 --2- 172.21.15.43:0/2678101955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564119efc8e0 0x564119f0b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:38.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.558+0000 7f3f6641e640 1 --2- 172.21.15.43:0/2678101955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efd290 0x564119f0b680 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f3f58032af0 tx=0x7f3f5804da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c85e703a3ece69a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:38.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.558+0000 7f3f497fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:38.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.559+0000 7f3f497fa640 1 -- 172.21.15.43:0/2678101955 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5805a070 con 0x564119efd290 2024-04-21T08:40:38.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.559+0000 7f3f497fa640 1 -- 172.21.15.43:0/2678101955 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f3f58034640 con 0x564119efd290 2024-04-21T08:40:38.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.559+0000 7f3f6aa79740 1 -- 172.21.15.43:0/2678101955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f58052070 con 0x564119efd290 2024-04-21T08:40:38.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.559+0000 7f3f497fa640 1 -- 172.21.15.43:0/2678101955 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3f58067050 con 0x564119efd290 2024-04-21T08:40:38.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.559+0000 7f3f6aa79740 10 client.16581 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:38.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.559+0000 7f3f6aa79740 1 -- 172.21.15.43:0/2678101955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564119f82120 con 0x564119efd290 2024-04-21T08:40:38.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.559+0000 7f3f497fa640 1 -- 172.21.15.43:0/2678101955 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f58052070 con 0x564119efd290 2024-04-21T08:40:38.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.559+0000 7f3f497fa640 1 -- 172.21.15.43:0/2678101955 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3f5803f040 con 0x564119efd290 2024-04-21T08:40:38.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.559+0000 7f3f6aa79740 10 client.16581 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:38.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.559+0000 7f3f6aa79740 10 client.16581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:38.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.559+0000 7f3f6aa79740 20 client.16581 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:38.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.559+0000 7f3f6aa79740 1 --2- 172.21.15.43:0/2678101955 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564119f05e10 0x564119f847a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.559+0000 7f3f6aa79740 4 client.16581 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:40:38.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.559+0000 7f3f6aa79740 1 -- 172.21.15.43:0/2678101955 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564119e68790 con 0x564119f05e10 2024-04-21T08:40:38.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.559+0000 7f3f65c1d640 1 --2- 172.21.15.43:0/2678101955 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564119f05e10 0x564119f847a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.560+0000 7f3f65c1d640 1 --2- 172.21.15.43:0/2678101955 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564119f05e10 0x564119f847a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fa4b895f0138610 server_cookie=df133fd4d53de7e1 in_seq=0 out_seq=0 2024-04-21T08:40:38.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.560+0000 7f3f497fa640 10 client.16581 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:38.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.561+0000 7f3f497fa640 1 -- 172.21.15.43:0/2678101955 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564119e68790 con 0x564119f05e10 2024-04-21T08:40:38.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.561+0000 7f3f497fa640 10 client.16581 handle_command_reply: tid=0 2024-04-21T08:40:38.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.561+0000 7f3f6aa79740 1 client.16581 shutdown 2024-04-21T08:40:38.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.561+0000 7f3f6aa79740 1 -- 172.21.15.43:0/2678101955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efd290 msgr2=0x564119f0b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:38.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.561+0000 7f3f6aa79740 1 --2- 172.21.15.43:0/2678101955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efd290 0x564119f0b680 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f3f58032af0 tx=0x7f3f5804da30 comp rx=0 tx=0).stop 2024-04-21T08:40:38.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.561+0000 7f3f6aa79740 1 -- 172.21.15.43:0/2678101955 shutdown_connections 2024-04-21T08:40:38.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.561+0000 7f3f6aa79740 1 --2- 172.21.15.43:0/2678101955 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564119f05e10 0x564119f847a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.561+0000 7f3f6aa79740 1 --2- 172.21.15.43:0/2678101955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564119efd290 0x564119f0b680 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.561+0000 7f3f6aa79740 1 --2- 172.21.15.43:0/2678101955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564119efc8e0 0x564119f0b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.561+0000 7f3f6aa79740 1 --2- 172.21.15.43:0/2678101955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564119ef6fb0 0x564119f0ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.561+0000 7f3f497fa640 0 client.16581 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:38.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.561+0000 7f3f6aa79740 1 -- 172.21.15.43:0/2678101955 >> 172.21.15.43:0/2678101955 conn(0x564119f01a40 msgr2=0x564119f03650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:38.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.561+0000 7f3f6aa79740 1 -- 172.21.15.43:0/2678101955 shutdown_connections 2024-04-21T08:40:38.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.561+0000 7f3f6aa79740 1 -- 172.21.15.43:0/2678101955 wait complete. 2024-04-21T08:40:38.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.562+0000 7f3f6aa79740 20 client.16581 trim_cache size 0 max 16384 2024-04-21T08:40:38.566 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:40:38.566 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:40:38.566 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "fa293520-d003-4312-ac07-088ca778bf93", 2024-04-21T08:40:38.566 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:40:38.566 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:40:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.562+0000 7f3f67e81640 1 -- 172.21.15.43:0/1106035666 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3f4c0a3d20 msgr2=0x7f3f4c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.562+0000 7f3f67e81640 1 --2- 172.21.15.43:0/1106035666 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3f4c0a3d20 0x7f3f4c0a61e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009430 tx=0x7f3f5802e040 comp rx=0 tx=0).stop 2024-04-21T08:40:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.562+0000 7f3f67e81640 1 -- 172.21.15.43:0/1106035666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f60106170 msgr2=0x7f3f60199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.562+0000 7f3f67e81640 1 --2- 172.21.15.43:0/1106035666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f60106170 0x7f3f60199a60 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000d770 tx=0x7f3f5000dc30 comp rx=0 tx=0).stop 2024-04-21T08:40:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.562+0000 7f3f67e81640 1 -- 172.21.15.43:0/1106035666 shutdown_connections 2024-04-21T08:40:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.562+0000 7f3f67e81640 1 --2- 172.21.15.43:0/1106035666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f60199ff0 0x7f3f601a45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.562+0000 7f3f67e81640 1 --2- 172.21.15.43:0/1106035666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f60106170 0x7f3f60199a60 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.562+0000 7f3f67e81640 1 --2- 172.21.15.43:0/1106035666 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3f4c0a3d20 0x7f3f4c0a61e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.562+0000 7f3f67e81640 1 --2- 172.21.15.43:0/1106035666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f601010a0 0x7f3f60199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.562+0000 7f3f67e81640 1 -- 172.21.15.43:0/1106035666 >> 172.21.15.43:0/1106035666 conn(0x7f3f60077eb0 msgr2=0x7f3f600fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:38.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.562+0000 7f3f67e81640 1 -- 172.21.15.43:0/1106035666 shutdown_connections 2024-04-21T08:40:38.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.562+0000 7f3f67e81640 1 -- 172.21.15.43:0/1106035666 wait complete. 2024-04-21T08:40:38.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:40:38.572 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:40:38.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.678+0000 7f1a34ec0640 1 -- 172.21.15.43:0/368663590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a30106150 msgr2=0x7f1a3010dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:38.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.678+0000 7f1a34ec0640 1 --2- 172.21.15.43:0/368663590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a30106150 0x7f1a3010dd20 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009900 tx=0x7f1a2802bc30 comp rx=0 tx=0).stop 2024-04-21T08:40:38.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.678+0000 7f1a34ec0640 1 -- 172.21.15.43:0/368663590 shutdown_connections 2024-04-21T08:40:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.678+0000 7f1a34ec0640 1 --2- 172.21.15.43:0/368663590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a30106150 0x7f1a3010dd20 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.678+0000 7f1a34ec0640 1 --2- 172.21.15.43:0/368663590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a30101a30 0x7f1a30105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.678+0000 7f1a34ec0640 1 --2- 172.21.15.43:0/368663590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a30101080 0x7f1a30101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.678+0000 7f1a34ec0640 1 -- 172.21.15.43:0/368663590 >> 172.21.15.43:0/368663590 conn(0x7f1a30078000 msgr2=0x7f1a30078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.678+0000 7f1a34ec0640 1 -- 172.21.15.43:0/368663590 shutdown_connections 2024-04-21T08:40:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.678+0000 7f1a34ec0640 1 -- 172.21.15.43:0/368663590 wait complete. 2024-04-21T08:40:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.678+0000 7f1a34ec0640 1 Processor -- start 2024-04-21T08:40:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.679+0000 7f1a34ec0640 1 -- start start 2024-04-21T08:40:38.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.679+0000 7f1a34ec0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a30101080 0x7f1a30198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.679+0000 7f1a34ec0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a30106150 0x7f1a30198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.679+0000 7f1a34ec0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a3019cf50 0x7f1a3019d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.679+0000 7f1a34ec0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301121d0 con 0x7f1a3019cf50 2024-04-21T08:40:38.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.679+0000 7f1a34ec0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a30112050 con 0x7f1a30101080 2024-04-21T08:40:38.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.679+0000 7f1a34ec0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1a30112350 con 0x7f1a30106150 2024-04-21T08:40:38.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.679+0000 7f1a2ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a3019cf50 0x7f1a3019d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.679+0000 7f1a2ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a3019cf50 0x7f1a3019d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50362/0 (socket says 172.21.15.43:50362) 2024-04-21T08:40:38.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.679+0000 7f1a2ed9d640 1 -- 172.21.15.43:0/2820114487 learned_addr learned my addr 172.21.15.43:0/2820114487 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:38.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.679+0000 7f1a2e59c640 1 --2- 172.21.15.43:0/2820114487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a30101080 0x7f1a30198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.679+0000 7f1a2dd9b640 1 --2- 172.21.15.43:0/2820114487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a30106150 0x7f1a30198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.679+0000 7f1a2ed9d640 1 -- 172.21.15.43:0/2820114487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a30106150 msgr2=0x7f1a30198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:38.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.679+0000 7f1a2ed9d640 1 --2- 172.21.15.43:0/2820114487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a30106150 0x7f1a30198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.679+0000 7f1a2ed9d640 1 -- 172.21.15.43:0/2820114487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a30101080 msgr2=0x7f1a30198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:38.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.679+0000 7f1a2ed9d640 1 --2- 172.21.15.43:0/2820114487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a30101080 0x7f1a30198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.679+0000 7f1a2ed9d640 1 -- 172.21.15.43:0/2820114487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3019da40 con 0x7f1a3019cf50 2024-04-21T08:40:38.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.679+0000 7f1a2ed9d640 1 --2- 172.21.15.43:0/2820114487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a3019cf50 0x7f1a3019d400 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009230 tx=0x7f1a280045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:38.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.680+0000 7f1a1f7fe640 1 -- 172.21.15.43:0/2820114487 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28043020 con 0x7f1a3019cf50 2024-04-21T08:40:38.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.680+0000 7f1a1f7fe640 1 -- 172.21.15.43:0/2820114487 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a2802cb70 con 0x7f1a3019cf50 2024-04-21T08:40:38.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.680+0000 7f1a34ec0640 1 -- 172.21.15.43:0/2820114487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a3019dcd0 con 0x7f1a3019cf50 2024-04-21T08:40:38.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.680+0000 7f1a1f7fe640 1 -- 172.21.15.43:0/2820114487 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28003ac0 con 0x7f1a3019cf50 2024-04-21T08:40:38.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.680+0000 7f1a34ec0640 1 -- 172.21.15.43:0/2820114487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a301a5520 con 0x7f1a3019cf50 2024-04-21T08:40:38.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.682+0000 7f1a1f7fe640 1 -- 172.21.15.43:0/2820114487 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1a28046020 con 0x7f1a3019cf50 2024-04-21T08:40:38.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.682+0000 7f1a37b81740 1 Processor -- start 2024-04-21T08:40:38.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.682+0000 7f1a37b81740 1 -- start start 2024-04-21T08:40:38.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.682+0000 7f1a37b81740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cfe093580 0x555cfe093960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.682+0000 7f1a37b81740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cfe093ea0 0x555cfe09c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.682+0000 7f1a37b81740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe09ca80 0x555cfe09ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.682+0000 7f1a37b81740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555cfdff81e0 con 0x555cfe09ca80 2024-04-21T08:40:38.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.682+0000 7f1a37b81740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555cfdf95390 con 0x555cfe093ea0 2024-04-21T08:40:38.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.682+0000 7f1a37b81740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555cfdff41a0 con 0x555cfe093580 2024-04-21T08:40:38.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.682+0000 7f1a1f7fe640 1 --2- 172.21.15.43:0/2820114487 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1a180a3da0 0x7f1a180a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.683+0000 7f1a2dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe09ca80 0x555cfe09ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.683+0000 7f1a2dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe09ca80 0x555cfe09ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50374/0 (socket says 172.21.15.43:50374) 2024-04-21T08:40:38.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.683+0000 7f1a2dd9b640 1 -- 172.21.15.43:0/1045117835 learned_addr learned my addr 172.21.15.43:0/1045117835 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:38.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.683+0000 7f1a2ed9d640 1 --2- 172.21.15.43:0/2820114487 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1a180a3da0 0x7f1a180a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.683+0000 7f1a2e59c640 1 --2- 172.21.15.43:0/1045117835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cfe093ea0 0x555cfe09c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.683+0000 7f1a2dd9b640 1 --2- 172.21.15.43:0/1045117835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cfe093580 0x555cfe093960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.683+0000 7f1a1f7fe640 1 -- 172.21.15.43:0/2820114487 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f1a280f2dd0 con 0x7f1a3019cf50 2024-04-21T08:40:38.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.683+0000 7f1a2dd9b640 1 -- 172.21.15.43:0/1045117835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cfe093580 msgr2=0x555cfe093960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:38.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.683+0000 7f1a2dd9b640 1 --2- 172.21.15.43:0/1045117835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cfe093580 0x555cfe093960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.683+0000 7f1a2dd9b640 1 -- 172.21.15.43:0/1045117835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cfe093ea0 msgr2=0x555cfe09c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:38.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.683+0000 7f1a2dd9b640 1 --2- 172.21.15.43:0/1045117835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cfe093ea0 0x555cfe09c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.683+0000 7f1a2dd9b640 1 -- 172.21.15.43:0/1045117835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cfddd47d0 con 0x555cfe09ca80 2024-04-21T08:40:38.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.683+0000 7f1a2dd9b640 1 --2- 172.21.15.43:0/1045117835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe09ca80 0x555cfe09ee70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1a240134d0 tx=0x7f1a2400c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f46279fb0901f0c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:38.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.683+0000 7f1a2ed9d640 1 --2- 172.21.15.43:0/2820114487 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1a180a3da0 0x7f1a180a6260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1a280f0170 tx=0x7f1a28090040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:38.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.683+0000 7f1a1cff9640 1 -- 172.21.15.43:0/1045117835 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a24005020 con 0x555cfe09ca80 2024-04-21T08:40:38.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.683+0000 7f1a1cff9640 1 -- 172.21.15.43:0/1045117835 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a240047a0 con 0x555cfe09ca80 2024-04-21T08:40:38.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.683+0000 7f1a1cff9640 1 -- 172.21.15.43:0/1045117835 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2400f6e0 con 0x555cfe09ca80 2024-04-21T08:40:38.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.684+0000 7f1a37b81740 1 -- 172.21.15.43:0/1045117835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe09ca80 msgr2=0x555cfe09ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:38.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.684+0000 7f1a37b81740 1 --2- 172.21.15.43:0/1045117835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe09ca80 0x555cfe09ee70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1a240134d0 tx=0x7f1a2400c610 comp rx=0 tx=0).stop 2024-04-21T08:40:38.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.684+0000 7f1a37b81740 1 -- 172.21.15.43:0/1045117835 shutdown_connections 2024-04-21T08:40:38.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.684+0000 7f1a37b81740 1 --2- 172.21.15.43:0/1045117835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe09ca80 0x555cfe09ee70 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.684+0000 7f1a37b81740 1 --2- 172.21.15.43:0/1045117835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cfe093ea0 0x555cfe09c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.684+0000 7f1a37b81740 1 --2- 172.21.15.43:0/1045117835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cfe093580 0x555cfe093960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.684+0000 7f1a37b81740 1 -- 172.21.15.43:0/1045117835 >> 172.21.15.43:0/1045117835 conn(0x555cfe08dcb0 msgr2=0x555cfe08e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:38.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.684+0000 7f1a37b81740 1 -- 172.21.15.43:0/1045117835 shutdown_connections 2024-04-21T08:40:38.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.684+0000 7f1a37b81740 1 -- 172.21.15.43:0/1045117835 wait complete. 2024-04-21T08:40:38.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.685+0000 7f1a37b81740 1 -- start start 2024-04-21T08:40:38.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.685+0000 7f1a37b81740 1 Processor -- start 2024-04-21T08:40:38.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.685+0000 7f1a37b81740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cfe093580 0x555cfe099aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.685+0000 7f1a37b81740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe093ea0 0x555cfe099fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.685+0000 7f1a37b81740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cfe09ca80 0x555cfe09a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.685+0000 7f1a37b81740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555cfe09f870 con 0x555cfe093ea0 2024-04-21T08:40:38.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.685+0000 7f1a37b81740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555cfdfb12f0 con 0x555cfe093580 2024-04-21T08:40:38.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.685+0000 7f1a37b81740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555cfe09f9f0 con 0x555cfe09ca80 2024-04-21T08:40:38.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.685+0000 7f1a37b81740 10 client.0 _ll_register_callbacks cb 0x555cfdfbe580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:38.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.685+0000 7f1a2e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe093ea0 0x555cfe099fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.685+0000 7f1a2e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe093ea0 0x555cfe099fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50382/0 (socket says 172.21.15.43:50382) 2024-04-21T08:40:38.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.685+0000 7f1a2e59c640 1 -- 172.21.15.43:0/2438021079 learned_addr learned my addr 172.21.15.43:0/2438021079 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:38.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.685+0000 7f1a2dd9b640 1 --2- 172.21.15.43:0/2438021079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cfe093580 0x555cfe099aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.685+0000 7f1a2dd9b640 1 --2- 172.21.15.43:0/2438021079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cfe09ca80 0x555cfe09a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.685+0000 7f1a2e59c640 1 -- 172.21.15.43:0/2438021079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cfe09ca80 msgr2=0x555cfe09a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:38.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.685+0000 7f1a2e59c640 1 --2- 172.21.15.43:0/2438021079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cfe09ca80 0x555cfe09a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.685+0000 7f1a2e59c640 1 -- 172.21.15.43:0/2438021079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cfe093580 msgr2=0x555cfe099aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:38.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.685+0000 7f1a2e59c640 1 --2- 172.21.15.43:0/2438021079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cfe093580 0x555cfe099aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.685+0000 7f1a2e59c640 1 -- 172.21.15.43:0/2438021079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cfdd0a3f0 con 0x555cfe093ea0 2024-04-21T08:40:38.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.685+0000 7f1a2dd9b640 1 --2- 172.21.15.43:0/2438021079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cfe09ca80 0x555cfe09a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:38.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.685+0000 7f1a2e59c640 1 --2- 172.21.15.43:0/2438021079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe093ea0 0x555cfe099fe0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1a20003f30 tx=0x7f1a200044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab8b03bbb9492f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:38.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.686+0000 7f1a15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:38.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.686+0000 7f1a15ffb640 1 -- 172.21.15.43:0/2438021079 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20012a80 con 0x555cfe093ea0 2024-04-21T08:40:38.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.686+0000 7f1a15ffb640 1 -- 172.21.15.43:0/2438021079 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f1a20012c20 con 0x555cfe093ea0 2024-04-21T08:40:38.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.686+0000 7f1a37b81740 1 -- 172.21.15.43:0/2438021079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a240047a0 con 0x555cfe093ea0 2024-04-21T08:40:38.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.686+0000 7f1a15ffb640 1 -- 172.21.15.43:0/2438021079 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1a20012e20 con 0x555cfe093ea0 2024-04-21T08:40:38.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.686+0000 7f1a37b81740 10 client.16593 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:38.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.686+0000 7f1a37b81740 1 -- 172.21.15.43:0/2438021079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555cfdcca0f0 con 0x555cfe093ea0 2024-04-21T08:40:38.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.686+0000 7f1a15ffb640 1 -- 172.21.15.43:0/2438021079 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a240047a0 con 0x555cfe093ea0 2024-04-21T08:40:38.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.686+0000 7f1a15ffb640 1 -- 172.21.15.43:0/2438021079 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1a20019070 con 0x555cfe093ea0 2024-04-21T08:40:38.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.686+0000 7f1a37b81740 10 client.16593 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:38.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.686+0000 7f1a37b81740 10 client.16593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:38.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.686+0000 7f1a37b81740 20 client.16593 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:38.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.686+0000 7f1a37b81740 1 --2- 172.21.15.43:0/2438021079 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555cfe0acb30 0x555cfe0aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.686+0000 7f1a37b81740 4 client.16593 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:38.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.686+0000 7f1a37b81740 1 -- 172.21.15.43:0/2438021079 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cfe005c00 con 0x555cfe0acb30 2024-04-21T08:40:38.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.687+0000 7f1a2dd9b640 1 --2- 172.21.15.43:0/2438021079 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555cfe0acb30 0x555cfe0aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.687+0000 7f1a15ffb640 10 client.16593 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:38.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.687+0000 7f1a2dd9b640 1 --2- 172.21.15.43:0/2438021079 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555cfe0acb30 0x555cfe0aefb0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a6d86a0cf8ec14e server_cookie=e1fe21c85ec2d5c2 in_seq=0 out_seq=0 2024-04-21T08:40:38.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.688+0000 7f1a15ffb640 1 -- 172.21.15.43:0/2438021079 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555cfe005c00 con 0x555cfe0acb30 2024-04-21T08:40:38.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.688+0000 7f1a15ffb640 10 client.16593 handle_command_reply: tid=0 2024-04-21T08:40:38.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.688+0000 7f1a37b81740 1 client.16593 shutdown 2024-04-21T08:40:38.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.688+0000 7f1a37b81740 1 -- 172.21.15.43:0/2438021079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe093ea0 msgr2=0x555cfe099fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:38.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.688+0000 7f1a37b81740 1 --2- 172.21.15.43:0/2438021079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe093ea0 0x555cfe099fe0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1a20003f30 tx=0x7f1a200044d0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.688+0000 7f1a37b81740 1 -- 172.21.15.43:0/2438021079 shutdown_connections 2024-04-21T08:40:38.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.688+0000 7f1a37b81740 1 --2- 172.21.15.43:0/2438021079 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555cfe0acb30 0x555cfe0aefb0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.688+0000 7f1a37b81740 1 --2- 172.21.15.43:0/2438021079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cfe09ca80 0x555cfe09a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.688+0000 7f1a37b81740 1 --2- 172.21.15.43:0/2438021079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe093ea0 0x555cfe099fe0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.688+0000 7f1a15ffb640 0 client.16593 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:38.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.688+0000 7f1a37b81740 1 --2- 172.21.15.43:0/2438021079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cfe093580 0x555cfe099aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.688+0000 7f1a37b81740 1 -- 172.21.15.43:0/2438021079 >> 172.21.15.43:0/2438021079 conn(0x555cfe08dcb0 msgr2=0x555cfe08fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:38.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.688+0000 7f1a37b81740 1 -- 172.21.15.43:0/2438021079 shutdown_connections 2024-04-21T08:40:38.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.688+0000 7f1a37b81740 1 -- 172.21.15.43:0/2438021079 wait complete. 2024-04-21T08:40:38.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.688+0000 7f1a37b81740 20 client.16593 trim_cache size 0 max 16384 2024-04-21T08:40:38.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.706+0000 7f1a37b81740 1 Processor -- start 2024-04-21T08:40:38.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.706+0000 7f1a37b81740 1 -- start start 2024-04-21T08:40:38.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.706+0000 7f1a37b81740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe093580 0x555cfe09c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.706+0000 7f1a37b81740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cfe093ea0 0x555cfe0a8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.706+0000 7f1a37b81740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cfe09ca80 0x555cfe08f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.706+0000 7f1a37b81740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301af360 con 0x555cfe093580 2024-04-21T08:40:38.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.706+0000 7f1a37b81740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301b0ea0 con 0x555cfe09ca80 2024-04-21T08:40:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.706+0000 7f1a37b81740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301afa30 con 0x555cfe093ea0 2024-04-21T08:40:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.706+0000 7f1a2dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe093580 0x555cfe09c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.706+0000 7f1a2dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe093580 0x555cfe09c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50384/0 (socket says 172.21.15.43:50384) 2024-04-21T08:40:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.706+0000 7f1a2dd9b640 1 -- 172.21.15.43:0/773146378 learned_addr learned my addr 172.21.15.43:0/773146378 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.706+0000 7f1a2e59c640 1 --2- 172.21.15.43:0/773146378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cfe093ea0 0x555cfe0a8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.706+0000 7f1a2dd9b640 1 -- 172.21.15.43:0/773146378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cfe093ea0 msgr2=0x555cfe0a8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.706+0000 7f1a2dd9b640 1 --2- 172.21.15.43:0/773146378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cfe093ea0 0x555cfe0a8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.706+0000 7f1a2dd9b640 1 -- 172.21.15.43:0/773146378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cfe09ca80 msgr2=0x555cfe08f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.706+0000 7f1a2dd9b640 1 --2- 172.21.15.43:0/773146378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cfe09ca80 0x555cfe08f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.706+0000 7f1a2dd9b640 1 -- 172.21.15.43:0/773146378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cfe08eee0 con 0x555cfe093580 2024-04-21T08:40:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.707+0000 7f1a2dd9b640 1 --2- 172.21.15.43:0/773146378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe093580 0x555cfe09c1d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f1a24013fd0 tx=0x7f1a2400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd57a5d11532f278 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.707+0000 7f1a17fff640 1 -- 172.21.15.43:0/773146378 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a24004020 con 0x555cfe093580 2024-04-21T08:40:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.707+0000 7f1a17fff640 1 -- 172.21.15.43:0/773146378 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a24031070 con 0x555cfe093580 2024-04-21T08:40:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.707+0000 7f1a17fff640 1 -- 172.21.15.43:0/773146378 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2401f9b0 con 0x555cfe093580 2024-04-21T08:40:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.707+0000 7f1a37b81740 1 -- 172.21.15.43:0/773146378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe093580 msgr2=0x555cfe09c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:38.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.707+0000 7f1a37b81740 1 --2- 172.21.15.43:0/773146378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe093580 0x555cfe09c1d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f1a24013fd0 tx=0x7f1a2400cbe0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.707+0000 7f1a37b81740 1 -- 172.21.15.43:0/773146378 shutdown_connections 2024-04-21T08:40:38.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.707+0000 7f1a37b81740 1 --2- 172.21.15.43:0/773146378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cfe09ca80 0x555cfe08f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.707+0000 7f1a37b81740 1 --2- 172.21.15.43:0/773146378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cfe093ea0 0x555cfe0a8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.707+0000 7f1a37b81740 1 --2- 172.21.15.43:0/773146378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe093580 0x555cfe09c1d0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.707+0000 7f1a37b81740 1 -- 172.21.15.43:0/773146378 >> 172.21.15.43:0/773146378 conn(0x555cfe08dcb0 msgr2=0x555cfe08f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:38.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.707+0000 7f1a37b81740 1 -- 172.21.15.43:0/773146378 shutdown_connections 2024-04-21T08:40:38.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.707+0000 7f1a37b81740 1 -- 172.21.15.43:0/773146378 wait complete. 2024-04-21T08:40:38.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.708+0000 7f1a37b81740 1 -- start start 2024-04-21T08:40:38.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.708+0000 7f1a37b81740 1 Processor -- start 2024-04-21T08:40:38.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.708+0000 7f1a37b81740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe093580 0x555cfe0a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.708+0000 7f1a37b81740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cfe093ea0 0x555cfe0a1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.708+0000 7f1a37b81740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cfe09ca80 0x555cfe10c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.708+0000 7f1a37b81740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301b07d0 con 0x555cfe093580 2024-04-21T08:40:38.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.708+0000 7f1a37b81740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301b0100 con 0x555cfe09ca80 2024-04-21T08:40:38.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.708+0000 7f1a37b81740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301ad340 con 0x555cfe093ea0 2024-04-21T08:40:38.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.708+0000 7f1a37b81740 10 client.0 _ll_register_callbacks cb 0x7f1a20009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:38.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.708+0000 7f1a2dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe093580 0x555cfe0a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.708+0000 7f1a2dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe093580 0x555cfe0a0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50386/0 (socket says 172.21.15.43:50386) 2024-04-21T08:40:38.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.708+0000 7f1a2dd9b640 1 -- 172.21.15.43:0/1114503258 learned_addr learned my addr 172.21.15.43:0/1114503258 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:38.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.708+0000 7f1a2e59c640 1 --2- 172.21.15.43:0/1114503258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cfe093ea0 0x555cfe0a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.708+0000 7f1a2dd9b640 1 -- 172.21.15.43:0/1114503258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cfe093ea0 msgr2=0x555cfe0a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:38.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.708+0000 7f1a2dd9b640 1 --2- 172.21.15.43:0/1114503258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cfe093ea0 0x555cfe0a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.708+0000 7f1a2dd9b640 1 -- 172.21.15.43:0/1114503258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cfe09ca80 msgr2=0x555cfe10c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:38.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.708+0000 7f1a2dd9b640 1 --2- 172.21.15.43:0/1114503258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cfe09ca80 0x555cfe10c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.708+0000 7f1a2dd9b640 1 -- 172.21.15.43:0/1114503258 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cfe10c7b0 con 0x555cfe093580 2024-04-21T08:40:38.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.708+0000 7f1a2dd9b640 1 --2- 172.21.15.43:0/1114503258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe093580 0x555cfe0a0f10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f1a24013d60 tx=0x7f1a2400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f627adc87569a416 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:38.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.708+0000 7f1a15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:38.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.709+0000 7f1a15ffb640 1 -- 172.21.15.43:0/1114503258 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2402d4a0 con 0x555cfe093580 2024-04-21T08:40:38.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.709+0000 7f1a15ffb640 1 -- 172.21.15.43:0/1114503258 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f1a2402d640 con 0x555cfe093580 2024-04-21T08:40:38.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.709+0000 7f1a37b81740 1 -- 172.21.15.43:0/1114503258 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a24031070 con 0x555cfe093580 2024-04-21T08:40:38.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.709+0000 7f1a15ffb640 1 -- 172.21.15.43:0/1114503258 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1a24043050 con 0x555cfe093580 2024-04-21T08:40:38.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.709+0000 7f1a37b81740 10 client.16599 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:38.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.709+0000 7f1a37b81740 1 -- 172.21.15.43:0/1114503258 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555cfe113b30 con 0x555cfe093580 2024-04-21T08:40:38.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.709+0000 7f1a15ffb640 1 -- 172.21.15.43:0/1114503258 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a24031070 con 0x555cfe093580 2024-04-21T08:40:38.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.709+0000 7f1a15ffb640 1 -- 172.21.15.43:0/1114503258 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1a2401fc50 con 0x555cfe093580 2024-04-21T08:40:38.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.709+0000 7f1a37b81740 10 client.16599 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:38.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.709+0000 7f1a37b81740 10 client.16599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:38.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.709+0000 7f1a37b81740 20 client.16599 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:38.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.709+0000 7f1a37b81740 1 --2- 172.21.15.43:0/1114503258 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555cfe113f50 0x555cfe116340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:38.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.709+0000 7f1a37b81740 4 client.16599 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:40:38.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.709+0000 7f1a37b81740 1 -- 172.21.15.43:0/1114503258 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555cfdfffa70 con 0x555cfe113f50 2024-04-21T08:40:38.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.709+0000 7f1a2e59c640 1 --2- 172.21.15.43:0/1114503258 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555cfe113f50 0x555cfe116340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:38.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.710+0000 7f1a15ffb640 10 client.16599 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:38.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.710+0000 7f1a2e59c640 1 --2- 172.21.15.43:0/1114503258 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555cfe113f50 0x555cfe116340 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1ffcc56f984eda2 server_cookie=33ca1d0c654a8389 in_seq=0 out_seq=0 2024-04-21T08:40:38.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.710+0000 7f1a15ffb640 1 -- 172.21.15.43:0/1114503258 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555cfdfffa70 con 0x555cfe113f50 2024-04-21T08:40:38.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.710+0000 7f1a15ffb640 10 client.16599 handle_command_reply: tid=0 2024-04-21T08:40:38.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.710+0000 7f1a37b81740 1 client.16599 shutdown 2024-04-21T08:40:38.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.710+0000 7f1a37b81740 1 -- 172.21.15.43:0/1114503258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe093580 msgr2=0x555cfe0a0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:38.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.710+0000 7f1a37b81740 1 --2- 172.21.15.43:0/1114503258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe093580 0x555cfe0a0f10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f1a24013d60 tx=0x7f1a2400c8d0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.710+0000 7f1a37b81740 1 -- 172.21.15.43:0/1114503258 shutdown_connections 2024-04-21T08:40:38.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.710+0000 7f1a37b81740 1 --2- 172.21.15.43:0/1114503258 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555cfe113f50 0x555cfe116340 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.710+0000 7f1a37b81740 1 --2- 172.21.15.43:0/1114503258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cfe09ca80 0x555cfe10c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.710+0000 7f1a37b81740 1 --2- 172.21.15.43:0/1114503258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cfe093ea0 0x555cfe0a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.710+0000 7f1a15ffb640 0 client.16599 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:38.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.710+0000 7f1a37b81740 1 --2- 172.21.15.43:0/1114503258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cfe093580 0x555cfe0a0f10 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.710+0000 7f1a37b81740 1 -- 172.21.15.43:0/1114503258 >> 172.21.15.43:0/1114503258 conn(0x555cfe08dcb0 msgr2=0x555cfe09a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:38.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.710+0000 7f1a37b81740 1 -- 172.21.15.43:0/1114503258 shutdown_connections 2024-04-21T08:40:38.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.710+0000 7f1a37b81740 1 -- 172.21.15.43:0/1114503258 wait complete. 2024-04-21T08:40:38.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.710+0000 7f1a37b81740 20 client.16599 trim_cache size 0 max 16384 2024-04-21T08:40:38.716 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:40:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:40:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-21T08:40:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: "fa293520-d003-4312-ac07-088ca778bf93": { 2024-04-21T08:40:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-21T08:40:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "fa293520-d003-4312-ac07-088ca778bf93", 2024-04-21T08:40:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-21T08:40:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:40:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:40:38.717 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:40:38.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.711+0000 7f1a34ec0640 1 -- 172.21.15.43:0/2820114487 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1a180a3da0 msgr2=0x7f1a180a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:38.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.711+0000 7f1a34ec0640 1 --2- 172.21.15.43:0/2820114487 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1a180a3da0 0x7f1a180a6260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1a280f0170 tx=0x7f1a28090040 comp rx=0 tx=0).stop 2024-04-21T08:40:38.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.711+0000 7f1a34ec0640 1 -- 172.21.15.43:0/2820114487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a3019cf50 msgr2=0x7f1a3019d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:38.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.711+0000 7f1a34ec0640 1 --2- 172.21.15.43:0/2820114487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a3019cf50 0x7f1a3019d400 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009230 tx=0x7f1a280045e0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.711+0000 7f1a34ec0640 1 -- 172.21.15.43:0/2820114487 shutdown_connections 2024-04-21T08:40:38.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.711+0000 7f1a34ec0640 1 --2- 172.21.15.43:0/2820114487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a3019cf50 0x7f1a3019d400 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.711+0000 7f1a34ec0640 1 --2- 172.21.15.43:0/2820114487 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1a180a3da0 0x7f1a180a6260 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.711+0000 7f1a34ec0640 1 --2- 172.21.15.43:0/2820114487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a30106150 0x7f1a30198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.711+0000 7f1a34ec0640 1 --2- 172.21.15.43:0/2820114487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a30101080 0x7f1a30198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:38.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.711+0000 7f1a34ec0640 1 -- 172.21.15.43:0/2820114487 >> 172.21.15.43:0/2820114487 conn(0x7f1a30078000 msgr2=0x7f1a300ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:38.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.711+0000 7f1a34ec0640 1 -- 172.21.15.43:0/2820114487 shutdown_connections 2024-04-21T08:40:38.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:40:38.711+0000 7f1a34ec0640 1 -- 172.21.15.43:0/2820114487 wait complete. 2024-04-21T08:40:38.721 INFO:tasks.cephfs.filesystem:scrub status for tag:fa293520-d003-4312-ac07-088ca778bf93 - {'path': '/', 'tag': 'fa293520-d003-4312-ac07-088ca778bf93', 'options': 'recursive,force'} 2024-04-21T08:40:40.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:39 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:40:40.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:39 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:40:40.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:39 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:40:40.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:39 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:40:40.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:39 smithi112 ceph-mon[31085]: pgmap v55: 129 pgs: 129 active+clean; 2.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-04-21T08:40:40.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:39 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:40:40.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:39 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:40:40.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:39 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:40:40.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:39 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:40:40.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:39 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:40:40.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:39 smithi043 ceph-mon[27978]: pgmap v55: 129 pgs: 129 active+clean; 2.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-04-21T08:40:40.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:39 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:40:40.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:39 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:40:40.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:39 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:40:40.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:39 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:40:40.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:39 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:40:40.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:39 smithi175 ceph-mon[31920]: pgmap v55: 129 pgs: 129 active+clean; 2.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-04-21T08:40:40.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:39 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:40:42.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:41 smithi112 ceph-mon[31085]: pgmap v56: 129 pgs: 129 active+clean; 2.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-04-21T08:40:42.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:41 smithi043 ceph-mon[27978]: pgmap v56: 129 pgs: 129 active+clean; 2.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-04-21T08:40:42.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:41 smithi175 ceph-mon[31920]: pgmap v56: 129 pgs: 129 active+clean; 2.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-04-21T08:40:44.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:43 smithi112 ceph-mon[31085]: pgmap v57: 129 pgs: 129 active+clean; 2.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-21T08:40:44.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:43 smithi043 ceph-mon[27978]: pgmap v57: 129 pgs: 129 active+clean; 2.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-21T08:40:44.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:43 smithi175 ceph-mon[31920]: pgmap v57: 129 pgs: 129 active+clean; 2.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-21T08:40:46.293 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:45 smithi112 ceph-mon[31085]: pgmap v58: 129 pgs: 129 active+clean; 2.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-21T08:40:46.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:45 smithi043 ceph-mon[27978]: pgmap v58: 129 pgs: 129 active+clean; 2.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-21T08:40:46.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:45 smithi175 ceph-mon[31920]: pgmap v58: 129 pgs: 129 active+clean; 2.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-21T08:40:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.224+0000 7fc61c8a0640 1 -- 172.21.15.43:0/2810004041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc618101a90 msgr2=0x7fc618105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.224+0000 7fc61c8a0640 1 --2- 172.21.15.43:0/2810004041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc618101a90 0x7fc618105ad0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc600009900 tx=0x7fc60002bc30 comp rx=0 tx=0).stop 2024-04-21T08:40:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.224+0000 7fc61c8a0640 1 -- 172.21.15.43:0/2810004041 shutdown_connections 2024-04-21T08:40:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.224+0000 7fc61c8a0640 1 --2- 172.21.15.43:0/2810004041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6181061b0 0x7fc61810dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.224+0000 7fc61c8a0640 1 --2- 172.21.15.43:0/2810004041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc618101a90 0x7fc618105ad0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.224+0000 7fc61c8a0640 1 --2- 172.21.15.43:0/2810004041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6181010e0 0x7fc6181014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.224+0000 7fc61c8a0640 1 -- 172.21.15.43:0/2810004041 >> 172.21.15.43:0/2810004041 conn(0x7fc6180771d0 msgr2=0x7fc6180775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.224+0000 7fc61c8a0640 1 -- 172.21.15.43:0/2810004041 shutdown_connections 2024-04-21T08:40:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.224+0000 7fc61c8a0640 1 -- 172.21.15.43:0/2810004041 wait complete. 2024-04-21T08:40:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.225+0000 7fc61c8a0640 1 Processor -- start 2024-04-21T08:40:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.225+0000 7fc61c8a0640 1 -- start start 2024-04-21T08:40:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.225+0000 7fc61c8a0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6181010e0 0x7fc618198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.225+0000 7fc61c8a0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6181061b0 0x7fc618198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.225+0000 7fc61c8a0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc61819cf00 0x7fc61819d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.225+0000 7fc61c8a0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc6181110f0 con 0x7fc6181061b0 2024-04-21T08:40:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.225+0000 7fc61c8a0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc618110f70 con 0x7fc61819cf00 2024-04-21T08:40:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.225+0000 7fc61c8a0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc618111270 con 0x7fc6181010e0 2024-04-21T08:40:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.225+0000 7fc60ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6181061b0 0x7fc618198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.225+0000 7fc60ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6181061b0 0x7fc618198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49092/0 (socket says 172.21.15.43:49092) 2024-04-21T08:40:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.225+0000 7fc60ffff640 1 -- 172.21.15.43:0/505771972 learned_addr learned my addr 172.21.15.43:0/505771972 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.225+0000 7fc61659c640 1 --2- 172.21.15.43:0/505771972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6181010e0 0x7fc618198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.225+0000 7fc616d9d640 1 --2- 172.21.15.43:0/505771972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc61819cf00 0x7fc61819d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.225+0000 7fc60ffff640 1 -- 172.21.15.43:0/505771972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6181010e0 msgr2=0x7fc618198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.225+0000 7fc60ffff640 1 --2- 172.21.15.43:0/505771972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6181010e0 0x7fc618198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.225+0000 7fc60ffff640 1 -- 172.21.15.43:0/505771972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc61819cf00 msgr2=0x7fc61819d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.225+0000 7fc60ffff640 1 --2- 172.21.15.43:0/505771972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc61819cf00 0x7fc61819d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.225+0000 7fc60ffff640 1 -- 172.21.15.43:0/505771972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61819d9f0 con 0x7fc6181061b0 2024-04-21T08:40:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.226+0000 7fc60ffff640 1 --2- 172.21.15.43:0/505771972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6181061b0 0x7fc618198b80 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc600036040 tx=0x7fc6000045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.226+0000 7fc60f7fe640 1 -- 172.21.15.43:0/505771972 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600043020 con 0x7fc6181061b0 2024-04-21T08:40:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.226+0000 7fc60f7fe640 1 -- 172.21.15.43:0/505771972 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc60002cb70 con 0x7fc6181061b0 2024-04-21T08:40:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.226+0000 7fc60f7fe640 1 -- 172.21.15.43:0/505771972 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600003bf0 con 0x7fc6181061b0 2024-04-21T08:40:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.226+0000 7fc61c8a0640 1 -- 172.21.15.43:0/505771972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61819dc80 con 0x7fc6181061b0 2024-04-21T08:40:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.226+0000 7fc61c8a0640 1 -- 172.21.15.43:0/505771972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6181a5450 con 0x7fc6181061b0 2024-04-21T08:40:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.228+0000 7fc60f7fe640 1 -- 172.21.15.43:0/505771972 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc600046020 con 0x7fc6181061b0 2024-04-21T08:40:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.229+0000 7fc61f568740 1 Processor -- start 2024-04-21T08:40:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.229+0000 7fc61f568740 1 -- start start 2024-04-21T08:40:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.229+0000 7fc61f568740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3d8d0 0x55e5e2f3c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.229+0000 7fc61f568740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5e2f3cb60 0x55e5e2f3cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.229+0000 7fc61f568740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e2f4a4f0 0x55e5e2f4c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.229+0000 7fc61f568740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e5e2ea18d0 con 0x55e5e2f3d8d0 2024-04-21T08:40:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.229+0000 7fc61f568740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e5e2e3ee80 con 0x55e5e2f4a4f0 2024-04-21T08:40:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.229+0000 7fc61f568740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e5e2e9d890 con 0x55e5e2f3cb60 2024-04-21T08:40:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.229+0000 7fc60f7fe640 1 --2- 172.21.15.43:0/505771972 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc6040a3dc0 0x7fc6040a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.229+0000 7fc616d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3d8d0 0x55e5e2f3c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.229+0000 7fc616d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3d8d0 0x55e5e2f3c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49108/0 (socket says 172.21.15.43:49108) 2024-04-21T08:40:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.229+0000 7fc616d9d640 1 -- 172.21.15.43:0/543879887 learned_addr learned my addr 172.21.15.43:0/543879887 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.229+0000 7fc616d9d640 1 --2- 172.21.15.43:0/505771972 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc6040a3dc0 0x7fc6040a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.230+0000 7fc60ffff640 1 --2- 172.21.15.43:0/543879887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e2f4a4f0 0x55e5e2f4c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.230+0000 7fc61659c640 1 --2- 172.21.15.43:0/543879887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5e2f3cb60 0x55e5e2f3cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.230+0000 7fc60f7fe640 1 -- 172.21.15.43:0/505771972 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc6000f2bd0 con 0x7fc6181061b0 2024-04-21T08:40:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.230+0000 7fc61659c640 1 -- 172.21.15.43:0/543879887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e2f4a4f0 msgr2=0x55e5e2f4c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.230+0000 7fc61659c640 1 --2- 172.21.15.43:0/543879887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e2f4a4f0 0x55e5e2f4c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.230+0000 7fc61659c640 1 -- 172.21.15.43:0/543879887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3d8d0 msgr2=0x55e5e2f3c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.230+0000 7fc61659c640 1 --2- 172.21.15.43:0/543879887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3d8d0 0x55e5e2f3c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.230+0000 7fc61659c640 1 -- 172.21.15.43:0/543879887 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5e2e60320 con 0x55e5e2f3cb60 2024-04-21T08:40:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.230+0000 7fc616d9d640 1 --2- 172.21.15.43:0/505771972 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc6040a3dc0 0x7fc6040a6280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc6100092f0 tx=0x7fc610009b90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.230+0000 7fc61659c640 1 --2- 172.21.15.43:0/543879887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5e2f3cb60 0x55e5e2f3cfe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc608009c20 tx=0x7fc608011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9be61489cd44c6a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.230+0000 7fc60cff9640 1 -- 172.21.15.43:0/543879887 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608004050 con 0x55e5e2f3cb60 2024-04-21T08:40:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.231+0000 7fc60cff9640 1 -- 172.21.15.43:0/543879887 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6080054d0 con 0x55e5e2f3cb60 2024-04-21T08:40:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.231+0000 7fc60cff9640 1 -- 172.21.15.43:0/543879887 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608004d90 con 0x55e5e2f3cb60 2024-04-21T08:40:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.231+0000 7fc61f568740 1 -- 172.21.15.43:0/543879887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5e2f3cb60 msgr2=0x55e5e2f3cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.231+0000 7fc61f568740 1 --2- 172.21.15.43:0/543879887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5e2f3cb60 0x55e5e2f3cfe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc608009c20 tx=0x7fc608011810 comp rx=0 tx=0).stop 2024-04-21T08:40:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.231+0000 7fc61f568740 1 -- 172.21.15.43:0/543879887 shutdown_connections 2024-04-21T08:40:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.231+0000 7fc61f568740 1 --2- 172.21.15.43:0/543879887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e2f4a4f0 0x55e5e2f4c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.231+0000 7fc61f568740 1 --2- 172.21.15.43:0/543879887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5e2f3cb60 0x55e5e2f3cfe0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.231+0000 7fc61f568740 1 --2- 172.21.15.43:0/543879887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3d8d0 0x55e5e2f3c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.231+0000 7fc61f568740 1 -- 172.21.15.43:0/543879887 >> 172.21.15.43:0/543879887 conn(0x55e5e2f03c90 msgr2=0x55e5e2f37850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.232+0000 7fc61f568740 1 -- 172.21.15.43:0/543879887 shutdown_connections 2024-04-21T08:40:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.232+0000 7fc61f568740 1 -- 172.21.15.43:0/543879887 wait complete. 2024-04-21T08:40:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.232+0000 7fc61f568740 1 -- start start 2024-04-21T08:40:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.232+0000 7fc61f568740 1 Processor -- start 2024-04-21T08:40:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.232+0000 7fc61f568740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3cb60 0x55e5e2f49e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.232+0000 7fc61f568740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e2f3d8d0 0x55e5e2f471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.232+0000 7fc61f568740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5e2f4a4f0 0x55e5e2f476f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.232+0000 7fc61f568740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e5e2f4d220 con 0x55e5e2f3cb60 2024-04-21T08:40:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.232+0000 7fc61f568740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e5e2e29eb0 con 0x55e5e2f3d8d0 2024-04-21T08:40:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.232+0000 7fc61f568740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e5e2f4d3a0 con 0x55e5e2f4a4f0 2024-04-21T08:40:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.232+0000 7fc61f568740 10 client.0 _ll_register_callbacks cb 0x55e5e2e56b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.232+0000 7fc61659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3cb60 0x55e5e2f49e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.232+0000 7fc61659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3cb60 0x55e5e2f49e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49122/0 (socket says 172.21.15.43:49122) 2024-04-21T08:40:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.232+0000 7fc61659c640 1 -- 172.21.15.43:0/2319812602 learned_addr learned my addr 172.21.15.43:0/2319812602 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.232+0000 7fc60ffff640 1 --2- 172.21.15.43:0/2319812602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e2f3d8d0 0x55e5e2f471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.232+0000 7fc616d9d640 1 --2- 172.21.15.43:0/2319812602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5e2f4a4f0 0x55e5e2f476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.233+0000 7fc61659c640 1 -- 172.21.15.43:0/2319812602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5e2f4a4f0 msgr2=0x55e5e2f476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.233+0000 7fc61659c640 1 --2- 172.21.15.43:0/2319812602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5e2f4a4f0 0x55e5e2f476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.233+0000 7fc61659c640 1 -- 172.21.15.43:0/2319812602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e2f3d8d0 msgr2=0x55e5e2f471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.233+0000 7fc61659c640 1 --2- 172.21.15.43:0/2319812602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e2f3d8d0 0x55e5e2f471b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.233+0000 7fc61659c640 1 -- 172.21.15.43:0/2319812602 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5e2bb4210 con 0x55e5e2f3cb60 2024-04-21T08:40:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.233+0000 7fc61659c640 1 --2- 172.21.15.43:0/2319812602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3cb60 0x55e5e2f49e20 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fc608004a70 tx=0x7fc608004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01ee361f74c464c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.233+0000 7fc5fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.233+0000 7fc5fdffb640 1 -- 172.21.15.43:0/2319812602 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6080046c0 con 0x55e5e2f3cb60 2024-04-21T08:40:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.233+0000 7fc61f568740 1 -- 172.21.15.43:0/2319812602 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6080054d0 con 0x55e5e2f3cb60 2024-04-21T08:40:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.233+0000 7fc5fdffb640 1 -- 172.21.15.43:0/2319812602 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc6080051c0 con 0x55e5e2f3cb60 2024-04-21T08:40:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.233+0000 7fc5fdffb640 1 -- 172.21.15.43:0/2319812602 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc608032050 con 0x55e5e2f3cb60 2024-04-21T08:40:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.233+0000 7fc61f568740 10 client.16611 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.233+0000 7fc61f568740 1 -- 172.21.15.43:0/2319812602 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e5e2c80620 con 0x55e5e2f3cb60 2024-04-21T08:40:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.234+0000 7fc5fdffb640 1 -- 172.21.15.43:0/2319812602 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6080054d0 con 0x55e5e2f3cb60 2024-04-21T08:40:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.234+0000 7fc5fdffb640 1 -- 172.21.15.43:0/2319812602 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc608018070 con 0x55e5e2f3cb60 2024-04-21T08:40:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.234+0000 7fc61f568740 10 client.16611 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.234+0000 7fc61f568740 10 client.16611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.234+0000 7fc61f568740 20 client.16611 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.234+0000 7fc61f568740 1 --2- 172.21.15.43:0/2319812602 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e5e2f5a3f0 0x55e5e2f5c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.234+0000 7fc61f568740 4 client.16611 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.234+0000 7fc61f568740 1 -- 172.21.15.43:0/2319812602 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5e2eb5880 con 0x55e5e2f5a3f0 2024-04-21T08:40:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.234+0000 7fc60ffff640 1 --2- 172.21.15.43:0/2319812602 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e5e2f5a3f0 0x55e5e2f5c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.234+0000 7fc60ffff640 1 --2- 172.21.15.43:0/2319812602 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e5e2f5a3f0 0x55e5e2f5c7e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0adb9e3985a7c64 server_cookie=c89464626a46ca35 in_seq=0 out_seq=0 2024-04-21T08:40:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.234+0000 7fc5fdffb640 10 client.16611 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.235+0000 7fc5fdffb640 1 -- 172.21.15.43:0/2319812602 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e5e2eb5880 con 0x55e5e2f5a3f0 2024-04-21T08:40:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.235+0000 7fc5fdffb640 10 client.16611 handle_command_reply: tid=0 2024-04-21T08:40:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.235+0000 7fc61f568740 1 client.16611 shutdown 2024-04-21T08:40:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.235+0000 7fc61f568740 1 -- 172.21.15.43:0/2319812602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3cb60 msgr2=0x55e5e2f49e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.235+0000 7fc61f568740 1 --2- 172.21.15.43:0/2319812602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3cb60 0x55e5e2f49e20 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fc608004a70 tx=0x7fc608004dc0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.235+0000 7fc61f568740 1 -- 172.21.15.43:0/2319812602 shutdown_connections 2024-04-21T08:40:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.235+0000 7fc61f568740 1 --2- 172.21.15.43:0/2319812602 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e5e2f5a3f0 0x55e5e2f5c7e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.235+0000 7fc61f568740 1 --2- 172.21.15.43:0/2319812602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5e2f4a4f0 0x55e5e2f476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.235+0000 7fc61f568740 1 --2- 172.21.15.43:0/2319812602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e2f3d8d0 0x55e5e2f471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.235+0000 7fc5fdffb640 0 client.16611 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.235+0000 7fc61f568740 1 --2- 172.21.15.43:0/2319812602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3cb60 0x55e5e2f49e20 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.235+0000 7fc61f568740 1 -- 172.21.15.43:0/2319812602 >> 172.21.15.43:0/2319812602 conn(0x55e5e2f03c90 msgr2=0x55e5e2f395a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.235+0000 7fc61f568740 1 -- 172.21.15.43:0/2319812602 shutdown_connections 2024-04-21T08:40:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.235+0000 7fc61f568740 1 -- 172.21.15.43:0/2319812602 wait complete. 2024-04-21T08:40:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.236+0000 7fc61f568740 20 client.16611 trim_cache size 0 max 16384 2024-04-21T08:40:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.250+0000 7fc61f568740 1 Processor -- start 2024-04-21T08:40:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.251+0000 7fc61f568740 1 -- start start 2024-04-21T08:40:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.251+0000 7fc61f568740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3cb60 0x55e5e2f56120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.251+0000 7fc61f568740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5e2f3d8d0 0x55e5e2f475d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.251+0000 7fc61f568740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e2f4a4f0 0x55e5e2f498e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.251+0000 7fc61f568740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc6181ad1a0 con 0x55e5e2f3cb60 2024-04-21T08:40:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.251+0000 7fc61f568740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc6181aece0 con 0x55e5e2f4a4f0 2024-04-21T08:40:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.251+0000 7fc61f568740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc6181ad870 con 0x55e5e2f3d8d0 2024-04-21T08:40:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.251+0000 7fc61659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3cb60 0x55e5e2f56120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.251+0000 7fc61659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3cb60 0x55e5e2f56120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49132/0 (socket says 172.21.15.43:49132) 2024-04-21T08:40:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.251+0000 7fc61659c640 1 -- 172.21.15.43:0/1185637252 learned_addr learned my addr 172.21.15.43:0/1185637252 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.251+0000 7fc60ffff640 1 --2- 172.21.15.43:0/1185637252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5e2f3d8d0 0x55e5e2f475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.251+0000 7fc616d9d640 1 --2- 172.21.15.43:0/1185637252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e2f4a4f0 0x55e5e2f498e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.251+0000 7fc61659c640 1 -- 172.21.15.43:0/1185637252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5e2f3d8d0 msgr2=0x55e5e2f475d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.251+0000 7fc61659c640 1 --2- 172.21.15.43:0/1185637252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5e2f3d8d0 0x55e5e2f475d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.251+0000 7fc61659c640 1 -- 172.21.15.43:0/1185637252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e2f4a4f0 msgr2=0x55e5e2f498e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.251+0000 7fc61659c640 1 --2- 172.21.15.43:0/1185637252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e2f4a4f0 0x55e5e2f498e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.251+0000 7fc61659c640 1 -- 172.21.15.43:0/1185637252 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5e2f4d100 con 0x55e5e2f3cb60 2024-04-21T08:40:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.251+0000 7fc616d9d640 1 --2- 172.21.15.43:0/1185637252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e2f4a4f0 0x55e5e2f498e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.251+0000 7fc61659c640 1 --2- 172.21.15.43:0/1185637252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3cb60 0x55e5e2f56120 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc608004ae0 tx=0x7fc608011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc3456012f3e488 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.251+0000 7fc5fffff640 1 -- 172.21.15.43:0/1185637252 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60803b070 con 0x55e5e2f3cb60 2024-04-21T08:40:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.251+0000 7fc5fffff640 1 -- 172.21.15.43:0/1185637252 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc608029070 con 0x55e5e2f3cb60 2024-04-21T08:40:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.251+0000 7fc5fffff640 1 -- 172.21.15.43:0/1185637252 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6080051c0 con 0x55e5e2f3cb60 2024-04-21T08:40:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.252+0000 7fc61f568740 1 -- 172.21.15.43:0/1185637252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3cb60 msgr2=0x55e5e2f56120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.252+0000 7fc61f568740 1 --2- 172.21.15.43:0/1185637252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3cb60 0x55e5e2f56120 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc608004ae0 tx=0x7fc608011810 comp rx=0 tx=0).stop 2024-04-21T08:40:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.252+0000 7fc61f568740 1 -- 172.21.15.43:0/1185637252 shutdown_connections 2024-04-21T08:40:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.252+0000 7fc61f568740 1 --2- 172.21.15.43:0/1185637252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e2f4a4f0 0x55e5e2f498e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.252+0000 7fc61f568740 1 --2- 172.21.15.43:0/1185637252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5e2f3d8d0 0x55e5e2f475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.252+0000 7fc61f568740 1 --2- 172.21.15.43:0/1185637252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3cb60 0x55e5e2f56120 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.252+0000 7fc61f568740 1 -- 172.21.15.43:0/1185637252 >> 172.21.15.43:0/1185637252 conn(0x55e5e2f03c90 msgr2=0x55e5e2f45aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.252+0000 7fc61f568740 1 -- 172.21.15.43:0/1185637252 shutdown_connections 2024-04-21T08:40:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.252+0000 7fc61f568740 1 -- 172.21.15.43:0/1185637252 wait complete. 2024-04-21T08:40:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.252+0000 7fc61f568740 1 -- start start 2024-04-21T08:40:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.252+0000 7fc61f568740 1 Processor -- start 2024-04-21T08:40:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.253+0000 7fc61f568740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5e2f3cb60 0x55e5e2f52820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.253+0000 7fc61f568740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3d8d0 0x55e5e2f43a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.253+0000 7fc61f568740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e2f4a4f0 0x55e5e2f43f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.253+0000 7fc61f568740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc6181ae610 con 0x55e5e2f3d8d0 2024-04-21T08:40:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.253+0000 7fc61f568740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc6181adf40 con 0x55e5e2f4a4f0 2024-04-21T08:40:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.253+0000 7fc61f568740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc6181ab180 con 0x55e5e2f3cb60 2024-04-21T08:40:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.253+0000 7fc61f568740 10 client.0 _ll_register_callbacks cb 0x7fc6100123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.253+0000 7fc60ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3d8d0 0x55e5e2f43a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.253+0000 7fc60ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3d8d0 0x55e5e2f43a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49142/0 (socket says 172.21.15.43:49142) 2024-04-21T08:40:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.253+0000 7fc60ffff640 1 -- 172.21.15.43:0/1057067184 learned_addr learned my addr 172.21.15.43:0/1057067184 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:40:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.253+0000 7fc61659c640 1 --2- 172.21.15.43:0/1057067184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5e2f3cb60 0x55e5e2f52820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.253+0000 7fc616d9d640 1 --2- 172.21.15.43:0/1057067184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e2f4a4f0 0x55e5e2f43f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.253+0000 7fc60ffff640 1 -- 172.21.15.43:0/1057067184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5e2f3cb60 msgr2=0x55e5e2f52820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.253+0000 7fc60ffff640 1 --2- 172.21.15.43:0/1057067184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5e2f3cb60 0x55e5e2f52820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.253+0000 7fc60ffff640 1 -- 172.21.15.43:0/1057067184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e2f4a4f0 msgr2=0x55e5e2f43f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.253+0000 7fc60ffff640 1 --2- 172.21.15.43:0/1057067184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e2f4a4f0 0x55e5e2f43f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.253+0000 7fc60ffff640 1 -- 172.21.15.43:0/1057067184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5e2f444f0 con 0x55e5e2f3d8d0 2024-04-21T08:40:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.253+0000 7fc616d9d640 1 --2- 172.21.15.43:0/1057067184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e2f4a4f0 0x55e5e2f43f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.253+0000 7fc60ffff640 1 --2- 172.21.15.43:0/1057067184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3d8d0 0x55e5e2f43a40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc6000f0110 tx=0x7fc6000b3d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bac8cc76beb7844 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.253+0000 7fc5fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:40:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.253+0000 7fc5fdffb640 1 -- 172.21.15.43:0/1057067184 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6000c7520 con 0x55e5e2f3d8d0 2024-04-21T08:40:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.253+0000 7fc5fdffb640 1 -- 172.21.15.43:0/1057067184 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc6000bcbb0 con 0x55e5e2f3d8d0 2024-04-21T08:40:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.253+0000 7fc61f568740 1 -- 172.21.15.43:0/1057067184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc608029070 con 0x55e5e2f3d8d0 2024-04-21T08:40:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.254+0000 7fc5fdffb640 1 -- 172.21.15.43:0/1057067184 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc6000be8b0 con 0x55e5e2f3d8d0 2024-04-21T08:40:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.254+0000 7fc61f568740 10 client.16617 fetch_fsmap learned FSMap version 28 2024-04-21T08:40:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.254+0000 7fc61f568740 1 -- 172.21.15.43:0/1057067184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e5e2fc0500 con 0x55e5e2f3d8d0 2024-04-21T08:40:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.254+0000 7fc5fdffb640 1 -- 172.21.15.43:0/1057067184 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc608029070 con 0x55e5e2f3d8d0 2024-04-21T08:40:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.254+0000 7fc5fdffb640 1 -- 172.21.15.43:0/1057067184 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc6000baf20 con 0x55e5e2f3d8d0 2024-04-21T08:40:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.254+0000 7fc61f568740 10 client.16617 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:40:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.254+0000 7fc61f568740 10 client.16617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.254+0000 7fc61f568740 20 client.16617 populate_metadata read hostname 'smithi043' 2024-04-21T08:40:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.254+0000 7fc61f568740 1 --2- 172.21.15.43:0/1057067184 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e5e2fc0b80 0x55e5e2fc2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.254+0000 7fc61f568740 4 client.16617 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:40:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.254+0000 7fc61f568740 1 -- 172.21.15.43:0/1057067184 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e5e2e413d0 con 0x55e5e2fc0b80 2024-04-21T08:40:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.254+0000 7fc61659c640 1 --2- 172.21.15.43:0/1057067184 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e5e2fc0b80 0x55e5e2fc2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.255+0000 7fc61659c640 1 --2- 172.21.15.43:0/1057067184 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e5e2fc0b80 0x55e5e2fc2f70 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa73d495ab480866 server_cookie=cfcd36be9458b435 in_seq=0 out_seq=0 2024-04-21T08:40:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.255+0000 7fc5fdffb640 10 client.16617 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.269+0000 7fc5fdffb640 1 -- 172.21.15.43:0/1057067184 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e5e2e413d0 con 0x55e5e2fc0b80 2024-04-21T08:40:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.269+0000 7fc5fdffb640 10 client.16617 handle_command_reply: tid=0 2024-04-21T08:40:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.269+0000 7fc61f568740 1 client.16617 shutdown 2024-04-21T08:40:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.269+0000 7fc61f568740 1 -- 172.21.15.43:0/1057067184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3d8d0 msgr2=0x55e5e2f43a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.269+0000 7fc61f568740 1 --2- 172.21.15.43:0/1057067184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3d8d0 0x55e5e2f43a40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc6000f0110 tx=0x7fc6000b3d40 comp rx=0 tx=0).stop 2024-04-21T08:40:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.269+0000 7fc61f568740 1 -- 172.21.15.43:0/1057067184 shutdown_connections 2024-04-21T08:40:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.269+0000 7fc61f568740 1 --2- 172.21.15.43:0/1057067184 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e5e2fc0b80 0x55e5e2fc2f70 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.269+0000 7fc61f568740 1 --2- 172.21.15.43:0/1057067184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e5e2f4a4f0 0x55e5e2f43f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.269+0000 7fc61f568740 1 --2- 172.21.15.43:0/1057067184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5e2f3d8d0 0x55e5e2f43a40 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.269+0000 7fc61f568740 1 --2- 172.21.15.43:0/1057067184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5e2f3cb60 0x55e5e2f52820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.269+0000 7fc5fdffb640 0 client.16617 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:40:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.269+0000 7fc61f568740 1 -- 172.21.15.43:0/1057067184 >> 172.21.15.43:0/1057067184 conn(0x55e5e2f03c90 msgr2=0x55e5e2f36040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.269+0000 7fc61f568740 1 -- 172.21.15.43:0/1057067184 shutdown_connections 2024-04-21T08:40:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.269+0000 7fc61f568740 1 -- 172.21.15.43:0/1057067184 wait complete. 2024-04-21T08:40:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.269+0000 7fc61f568740 20 client.16617 trim_cache size 0 max 16384 2024-04-21T08:40:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:40:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:40:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:40:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:40:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.270+0000 7fc61c8a0640 1 -- 172.21.15.43:0/505771972 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc6040a3dc0 msgr2=0x7fc6040a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.270+0000 7fc61c8a0640 1 --2- 172.21.15.43:0/505771972 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc6040a3dc0 0x7fc6040a6280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc6100092f0 tx=0x7fc610009b90 comp rx=0 tx=0).stop 2024-04-21T08:40:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.270+0000 7fc61c8a0640 1 -- 172.21.15.43:0/505771972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6181061b0 msgr2=0x7fc618198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.270+0000 7fc61c8a0640 1 --2- 172.21.15.43:0/505771972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6181061b0 0x7fc618198b80 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc600036040 tx=0x7fc6000045e0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.270+0000 7fc61c8a0640 1 -- 172.21.15.43:0/505771972 shutdown_connections 2024-04-21T08:40:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.270+0000 7fc61c8a0640 1 --2- 172.21.15.43:0/505771972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc61819cf00 0x7fc61819d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.270+0000 7fc61c8a0640 1 --2- 172.21.15.43:0/505771972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6181061b0 0x7fc618198b80 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.270+0000 7fc61c8a0640 1 --2- 172.21.15.43:0/505771972 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc6040a3dc0 0x7fc6040a6280 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.270+0000 7fc61c8a0640 1 --2- 172.21.15.43:0/505771972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6181010e0 0x7fc618198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.270+0000 7fc61c8a0640 1 -- 172.21.15.43:0/505771972 >> 172.21.15.43:0/505771972 conn(0x7fc6180771d0 msgr2=0x7fc61806a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.270+0000 7fc61c8a0640 1 -- 172.21.15.43:0/505771972 shutdown_connections 2024-04-21T08:40:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:40:47.270+0000 7fc61c8a0640 1 -- 172.21.15.43:0/505771972 wait complete. 2024-04-21T08:40:48.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:47 smithi112 ceph-mon[31085]: pgmap v59: 129 pgs: 129 active+clean; 2.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 23 op/s 2024-04-21T08:40:48.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:47 smithi043 ceph-mon[27978]: pgmap v59: 129 pgs: 129 active+clean; 2.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 23 op/s 2024-04-21T08:40:48.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:47 smithi175 ceph-mon[31920]: pgmap v59: 129 pgs: 129 active+clean; 2.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 23 op/s 2024-04-21T08:40:50.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:49 smithi112 ceph-mon[31085]: pgmap v60: 129 pgs: 129 active+clean; 2.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-21T08:40:50.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:49 smithi043 ceph-mon[27978]: pgmap v60: 129 pgs: 129 active+clean; 2.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-21T08:40:50.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:49 smithi175 ceph-mon[31920]: pgmap v60: 129 pgs: 129 active+clean; 2.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-21T08:40:52.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:51 smithi112 ceph-mon[31085]: pgmap v61: 129 pgs: 129 active+clean; 2.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-21T08:40:52.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:51 smithi043 ceph-mon[27978]: pgmap v61: 129 pgs: 129 active+clean; 2.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-21T08:40:52.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:51 smithi175 ceph-mon[31920]: pgmap v61: 129 pgs: 129 active+clean; 2.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-21T08:40:54.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:53 smithi112 ceph-mon[31085]: pgmap v62: 129 pgs: 129 active+clean; 2.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-21T08:40:54.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:53 smithi043 ceph-mon[27978]: pgmap v62: 129 pgs: 129 active+clean; 2.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-21T08:40:54.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:53 smithi175 ceph-mon[31920]: pgmap v62: 129 pgs: 129 active+clean; 2.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-21T08:40:56.266 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:55 smithi175 ceph-mon[31920]: pgmap v63: 129 pgs: 129 active+clean; 2.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-21T08:40:56.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:55 smithi112 ceph-mon[31085]: pgmap v63: 129 pgs: 129 active+clean; 2.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-21T08:40:56.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:55 smithi043 ceph-mon[27978]: pgmap v63: 129 pgs: 129 active+clean; 2.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-21T08:40:58.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:57 smithi112 ceph-mon[31085]: pgmap v64: 129 pgs: 129 active+clean; 2.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-21T08:40:58.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:57 smithi043 ceph-mon[27978]: pgmap v64: 129 pgs: 129 active+clean; 2.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-21T08:40:58.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:57 smithi175 ceph-mon[31920]: pgmap v64: 129 pgs: 129 active+clean; 2.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-21T08:41:00.002 INFO:tasks.workunit.client.0.smithi043.stdout:1/600: open fd = 3 2024-04-21T08:41:00.002 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.002 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.002 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.002 INFO:tasks.workunit.client.0.smithi043.stdout:2/600: open fd = 3 2024-04-21T08:41:00.002 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.002 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.002 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.002 INFO:tasks.workunit.client.0.smithi043.stdout:3/600: open fd = 3 2024-04-21T08:41:00.003 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.003 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.003 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.003 INFO:tasks.workunit.client.0.smithi043.stdout:4/600: open fd = 3 2024-04-21T08:41:00.003 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.003 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.003 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.003 INFO:tasks.workunit.client.0.smithi043.stdout:5/600: open fd = 3 2024-04-21T08:41:00.003 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.003 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.003 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.003 INFO:tasks.workunit.client.0.smithi043.stdout:6/600: open fd = 3 2024-04-21T08:41:00.003 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.004 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.004 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.004 INFO:tasks.workunit.client.0.smithi043.stdout:7/600: open fd = 3 2024-04-21T08:41:00.004 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.004 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.004 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.004 INFO:tasks.workunit.client.0.smithi043.stdout:8/600: open fd = 3 2024-04-21T08:41:00.004 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.004 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.004 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.004 INFO:tasks.workunit.client.0.smithi043.stdout:9/600: open fd = 3 2024-04-21T08:41:00.004 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.004 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.005 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.005 INFO:tasks.workunit.client.0.smithi043.stdout:10/600: open fd = 3 2024-04-21T08:41:00.005 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.005 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.005 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.005 INFO:tasks.workunit.client.0.smithi043.stdout:11/600: open fd = 3 2024-04-21T08:41:00.005 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.005 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.005 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.005 INFO:tasks.workunit.client.0.smithi043.stdout:12/600: open fd = 3 2024-04-21T08:41:00.005 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.005 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.006 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.006 INFO:tasks.workunit.client.0.smithi043.stdout:13/600: open fd = 3 2024-04-21T08:41:00.006 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.006 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.006 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.006 INFO:tasks.workunit.client.0.smithi043.stdout:14/600: open fd = 3 2024-04-21T08:41:00.006 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.006 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.006 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.006 INFO:tasks.workunit.client.0.smithi043.stdout:15/600: open fd = 3 2024-04-21T08:41:00.006 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.006 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.007 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.007 INFO:tasks.workunit.client.0.smithi043.stdout:16/600: open fd = 3 2024-04-21T08:41:00.007 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.007 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.007 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.007 INFO:tasks.workunit.client.0.smithi043.stdout:17/600: open fd = 3 2024-04-21T08:41:00.007 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.007 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.007 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.007 INFO:tasks.workunit.client.0.smithi043.stdout:18/600: open fd = 3 2024-04-21T08:41:00.007 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.007 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.008 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.008 INFO:tasks.workunit.client.0.smithi043.stdout:19/600: open fd = 3 2024-04-21T08:41:00.008 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.008 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.008 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.008 INFO:tasks.workunit.client.0.smithi043.stdout:20/600: open fd = 3 2024-04-21T08:41:00.008 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.008 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.008 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.008 INFO:tasks.workunit.client.0.smithi043.stdout:21/600: open fd = 3 2024-04-21T08:41:00.008 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.008 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.009 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.009 INFO:tasks.workunit.client.0.smithi043.stdout:22/600: open fd = 3 2024-04-21T08:41:00.009 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.009 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.009 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.009 INFO:tasks.workunit.client.0.smithi043.stdout:23/600: open fd = 3 2024-04-21T08:41:00.009 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.009 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.009 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.010 INFO:tasks.workunit.client.0.smithi043.stdout:24/600: open fd = 3 2024-04-21T08:41:00.010 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.010 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.010 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.010 INFO:tasks.workunit.client.0.smithi043.stdout:25/600: open fd = 3 2024-04-21T08:41:00.010 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.010 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.010 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.010 INFO:tasks.workunit.client.0.smithi043.stdout:26/600: open fd = 3 2024-04-21T08:41:00.010 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.010 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.011 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.011 INFO:tasks.workunit.client.0.smithi043.stdout:27/600: open fd = 3 2024-04-21T08:41:00.011 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.011 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.011 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.011 INFO:tasks.workunit.client.0.smithi043.stdout:28/600: open fd = 3 2024-04-21T08:41:00.011 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.011 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.011 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.011 INFO:tasks.workunit.client.0.smithi043.stdout:29/600: open fd = 3 2024-04-21T08:41:00.011 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.011 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.011 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.011 INFO:tasks.workunit.client.0.smithi043.stdout:30/600: open fd = 3 2024-04-21T08:41:00.012 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.012 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.012 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.012 INFO:tasks.workunit.client.0.smithi043.stdout:31/600: open fd = 3 2024-04-21T08:41:00.012 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.012 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.012 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.012 INFO:tasks.workunit.client.0.smithi043.stdout:32/600: open fd = 3 2024-04-21T08:41:00.012 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.012 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.012 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.012 INFO:tasks.workunit.client.0.smithi043.stdout:33/600: open fd = 3 2024-04-21T08:41:00.012 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.012 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.013 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.013 INFO:tasks.workunit.client.0.smithi043.stdout:34/600: open fd = 3 2024-04-21T08:41:00.013 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.013 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.013 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.013 INFO:tasks.workunit.client.0.smithi043.stdout:35/600: open fd = 3 2024-04-21T08:41:00.013 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.013 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.013 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.013 INFO:tasks.workunit.client.0.smithi043.stdout:36/600: open fd = 3 2024-04-21T08:41:00.013 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.013 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.013 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.014 INFO:tasks.workunit.client.0.smithi043.stdout:37/600: open fd = 3 2024-04-21T08:41:00.014 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.014 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.014 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.014 INFO:tasks.workunit.client.0.smithi043.stdout:38/600: open fd = 3 2024-04-21T08:41:00.014 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.014 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.014 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.014 INFO:tasks.workunit.client.0.smithi043.stdout:39/600: open fd = 3 2024-04-21T08:41:00.014 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.014 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.015 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.015 INFO:tasks.workunit.client.0.smithi043.stdout:40/600: open fd = 3 2024-04-21T08:41:00.015 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.015 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.015 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.015 INFO:tasks.workunit.client.0.smithi043.stdout:41/600: open fd = 3 2024-04-21T08:41:00.015 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.015 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.015 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.015 INFO:tasks.workunit.client.0.smithi043.stdout:42/600: open fd = 3 2024-04-21T08:41:00.015 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.015 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.015 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.015 INFO:tasks.workunit.client.0.smithi043.stdout:43/600: open fd = 3 2024-04-21T08:41:00.016 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.016 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.016 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.016 INFO:tasks.workunit.client.0.smithi043.stdout:44/600: open fd = 3 2024-04-21T08:41:00.016 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.016 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.016 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.016 INFO:tasks.workunit.client.0.smithi043.stdout:45/600: open fd = 3 2024-04-21T08:41:00.016 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.016 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.016 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.016 INFO:tasks.workunit.client.0.smithi043.stdout:46/600: open fd = 3 2024-04-21T08:41:00.016 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.017 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.017 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.017 INFO:tasks.workunit.client.0.smithi043.stdout:47/600: open fd = 3 2024-04-21T08:41:00.017 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.017 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.017 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.017 INFO:tasks.workunit.client.0.smithi043.stdout:48/600: open fd = 3 2024-04-21T08:41:00.017 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.017 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.017 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.017 INFO:tasks.workunit.client.0.smithi043.stdout:49/600: open fd = 3 2024-04-21T08:41:00.017 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.017 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.017 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.017 INFO:tasks.workunit.client.0.smithi043.stdout:50/600: open fd = 3 2024-04-21T08:41:00.018 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.018 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.018 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.018 INFO:tasks.workunit.client.0.smithi043.stdout:51/600: open fd = 3 2024-04-21T08:41:00.018 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.018 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.018 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.018 INFO:tasks.workunit.client.0.smithi043.stdout:52/600: open fd = 3 2024-04-21T08:41:00.018 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.018 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.018 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.018 INFO:tasks.workunit.client.0.smithi043.stdout:53/600: open fd = 3 2024-04-21T08:41:00.019 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.019 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.019 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.019 INFO:tasks.workunit.client.0.smithi043.stdout:54/600: open fd = 3 2024-04-21T08:41:00.019 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.019 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.019 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.019 INFO:tasks.workunit.client.0.smithi043.stdout:55/600: open fd = 3 2024-04-21T08:41:00.019 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.019 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.019 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.019 INFO:tasks.workunit.client.0.smithi043.stdout:56/600: open fd = 3 2024-04-21T08:41:00.019 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.019 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.019 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.020 INFO:tasks.workunit.client.0.smithi043.stdout:57/600: open fd = 3 2024-04-21T08:41:00.020 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.020 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.020 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.020 INFO:tasks.workunit.client.0.smithi043.stdout:58/600: open fd = 3 2024-04-21T08:41:00.020 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.020 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.020 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.020 INFO:tasks.workunit.client.0.smithi043.stdout:59/600: open fd = 3 2024-04-21T08:41:00.020 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.020 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.020 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.020 INFO:tasks.workunit.client.0.smithi043.stdout:60/600: open fd = 3 2024-04-21T08:41:00.021 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.021 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.021 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.021 INFO:tasks.workunit.client.0.smithi043.stdout:61/600: open fd = 3 2024-04-21T08:41:00.021 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.021 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.021 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.021 INFO:tasks.workunit.client.0.smithi043.stdout:62/600: open fd = 3 2024-04-21T08:41:00.021 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.021 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.021 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.021 INFO:tasks.workunit.client.0.smithi043.stdout:63/600: open fd = 3 2024-04-21T08:41:00.021 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.022 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:41:00.022 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:41:00.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:40:59 smithi112 ceph-mon[31085]: pgmap v65: 129 pgs: 129 active+clean; 2.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-21T08:41:00.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:40:59 smithi043 ceph-mon[27978]: pgmap v65: 129 pgs: 129 active+clean; 2.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-21T08:41:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.370+0000 7fa2fa29a640 1 -- 172.21.15.43:0/528924360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2f4102b80 msgr2=0x7fa2f4106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.370+0000 7fa2fa29a640 1 --2- 172.21.15.43:0/528924360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2f4102b80 0x7fa2f4106bc0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009900 tx=0x7fa2dc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.370+0000 7fa2fa29a640 1 -- 172.21.15.43:0/528924360 shutdown_connections 2024-04-21T08:41:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.370+0000 7fa2fa29a640 1 --2- 172.21.15.43:0/528924360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2f41072a0 0x7fa2f410ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.370+0000 7fa2fa29a640 1 --2- 172.21.15.43:0/528924360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2f4102b80 0x7fa2f4106bc0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.370+0000 7fa2fa29a640 1 --2- 172.21.15.43:0/528924360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2f41021d0 0x7fa2f41025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.370+0000 7fa2fa29a640 1 -- 172.21.15.43:0/528924360 >> 172.21.15.43:0/528924360 conn(0x7fa2f40793e0 msgr2=0x7fa2f40797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.370+0000 7fa2fa29a640 1 -- 172.21.15.43:0/528924360 shutdown_connections 2024-04-21T08:41:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.370+0000 7fa2fa29a640 1 -- 172.21.15.43:0/528924360 wait complete. 2024-04-21T08:41:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.371+0000 7fa2fa29a640 1 Processor -- start 2024-04-21T08:41:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.371+0000 7fa2fa29a640 1 -- start start 2024-04-21T08:41:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.371+0000 7fa2fa29a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2f41021d0 0x7fa2f41986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.371+0000 7fa2fa29a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2f41072a0 0x7fa2f4198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.371+0000 7fa2fa29a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2f419cf80 0x7fa2f419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.371+0000 7fa2fa29a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f41121c0 con 0x7fa2f419cf80 2024-04-21T08:41:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.371+0000 7fa2fa29a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f4112040 con 0x7fa2f41072a0 2024-04-21T08:41:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.371+0000 7fa2fa29a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f4112340 con 0x7fa2f41021d0 2024-04-21T08:41:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.371+0000 7fa2f8837640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2f419cf80 0x7fa2f419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.371+0000 7fa2f8837640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2f419cf80 0x7fa2f419d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47280/0 (socket says 172.21.15.43:47280) 2024-04-21T08:41:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.371+0000 7fa2f8837640 1 -- 172.21.15.43:0/1609597599 learned_addr learned my addr 172.21.15.43:0/1609597599 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.371+0000 7fa2f37fe640 1 --2- 172.21.15.43:0/1609597599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2f41072a0 0x7fa2f4198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.371+0000 7fa2f3fff640 1 --2- 172.21.15.43:0/1609597599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2f41021d0 0x7fa2f41986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.372+0000 7fa2f8837640 1 -- 172.21.15.43:0/1609597599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2f41021d0 msgr2=0x7fa2f41986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.372+0000 7fa2f8837640 1 --2- 172.21.15.43:0/1609597599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2f41021d0 0x7fa2f41986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.372+0000 7fa2f8837640 1 -- 172.21.15.43:0/1609597599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2f41072a0 msgr2=0x7fa2f4198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.372+0000 7fa2f8837640 1 --2- 172.21.15.43:0/1609597599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2f41072a0 0x7fa2f4198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.372+0000 7fa2f8837640 1 -- 172.21.15.43:0/1609597599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f419da70 con 0x7fa2f419cf80 2024-04-21T08:41:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.372+0000 7fa2f8837640 1 --2- 172.21.15.43:0/1609597599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2f419cf80 0x7fa2f419d430 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec00cab0 tx=0x7fa2ec00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.372+0000 7fa2f17fa640 1 -- 172.21.15.43:0/1609597599 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec00adc0 con 0x7fa2f419cf80 2024-04-21T08:41:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.372+0000 7fa2f17fa640 1 -- 172.21.15.43:0/1609597599 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2ec00f900 con 0x7fa2f419cf80 2024-04-21T08:41:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.372+0000 7fa2fa29a640 1 -- 172.21.15.43:0/1609597599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2f419dd60 con 0x7fa2f419cf80 2024-04-21T08:41:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.372+0000 7fa2f17fa640 1 -- 172.21.15.43:0/1609597599 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec0189a0 con 0x7fa2f419cf80 2024-04-21T08:41:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.372+0000 7fa2fa29a640 1 -- 172.21.15.43:0/1609597599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2f41a54d0 con 0x7fa2f419cf80 2024-04-21T08:41:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.375+0000 7fa2f17fa640 1 -- 172.21.15.43:0/1609597599 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa2ec00f450 con 0x7fa2f419cf80 2024-04-21T08:41:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.375+0000 7fa2fcf6e740 1 Processor -- start 2024-04-21T08:41:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.375+0000 7fa2fcf6e740 1 -- start start 2024-04-21T08:41:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.375+0000 7fa2fcf6e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555684def8d0 0x555684dee620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.375+0000 7fa2fcf6e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555684deeb60 0x555684deefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.375+0000 7fa2fcf6e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684df80e0 0x555684dfa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.375+0000 7fa2fcf6e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555684d538d0 con 0x555684df80e0 2024-04-21T08:41:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.375+0000 7fa2fcf6e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555684cf0e80 con 0x555684def8d0 2024-04-21T08:41:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.375+0000 7fa2fcf6e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555684d4f890 con 0x555684deeb60 2024-04-21T08:41:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.375+0000 7fa2f17fa640 1 --2- 172.21.15.43:0/1609597599 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa2e00a3dc0 0x7fa2e00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.375+0000 7fa2f37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684df80e0 0x555684dfa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.376+0000 7fa2f8837640 1 --2- 172.21.15.43:0/1609597599 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa2e00a3dc0 0x7fa2e00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.376+0000 7fa2f37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684df80e0 0x555684dfa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47284/0 (socket says 172.21.15.43:47284) 2024-04-21T08:41:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.376+0000 7fa2f37fe640 1 -- 172.21.15.43:0/587458908 learned_addr learned my addr 172.21.15.43:0/587458908 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.376+0000 7fa2f17fa640 1 -- 172.21.15.43:0/1609597599 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fa2ec0d0d60 con 0x7fa2f419cf80 2024-04-21T08:41:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.376+0000 7fa2f3fff640 1 --2- 172.21.15.43:0/587458908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555684deeb60 0x555684deefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.376+0000 7fa2f37fe640 1 --2- 172.21.15.43:0/587458908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555684def8d0 0x555684dee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.376+0000 7fa2f3fff640 1 -- 172.21.15.43:0/587458908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555684def8d0 msgr2=0x555684dee620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.376+0000 7fa2f3fff640 1 --2- 172.21.15.43:0/587458908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555684def8d0 0x555684dee620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.376+0000 7fa2f3fff640 1 -- 172.21.15.43:0/587458908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684df80e0 msgr2=0x555684dfa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.376+0000 7fa2f3fff640 1 --2- 172.21.15.43:0/587458908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684df80e0 0x555684dfa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.376+0000 7fa2f3fff640 1 -- 172.21.15.43:0/587458908 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555684d12320 con 0x555684deeb60 2024-04-21T08:41:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.376+0000 7fa2f37fe640 1 --2- 172.21.15.43:0/587458908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684df80e0 0x555684dfa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:41:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.376+0000 7fa2f8837640 1 --2- 172.21.15.43:0/1609597599 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa2e00a3dc0 0x7fa2e00a6280 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0ce6e0 tx=0x7fa2ec01baf0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.376+0000 7fa2f3fff640 1 --2- 172.21.15.43:0/587458908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555684deeb60 0x555684deefe0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4005fd0 tx=0x7fa2e40107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40790a6e9354d82c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.376+0000 7fa2ea7fc640 1 -- 172.21.15.43:0/587458908 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e401c480 con 0x555684deeb60 2024-04-21T08:41:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.376+0000 7fa2ea7fc640 1 -- 172.21.15.43:0/587458908 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2e401cad0 con 0x555684deeb60 2024-04-21T08:41:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.376+0000 7fa2ea7fc640 1 -- 172.21.15.43:0/587458908 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4011a80 con 0x555684deeb60 2024-04-21T08:41:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.377+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/587458908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555684deeb60 msgr2=0x555684deefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.377+0000 7fa2fcf6e740 1 --2- 172.21.15.43:0/587458908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555684deeb60 0x555684deefe0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4005fd0 tx=0x7fa2e40107a0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.377+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/587458908 shutdown_connections 2024-04-21T08:41:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.377+0000 7fa2fcf6e740 1 --2- 172.21.15.43:0/587458908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684df80e0 0x555684dfa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.377+0000 7fa2fcf6e740 1 --2- 172.21.15.43:0/587458908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555684deeb60 0x555684deefe0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.377+0000 7fa2fcf6e740 1 --2- 172.21.15.43:0/587458908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555684def8d0 0x555684dee620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.377+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/587458908 >> 172.21.15.43:0/587458908 conn(0x555684db5c90 msgr2=0x555684de9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.377+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/587458908 shutdown_connections 2024-04-21T08:41:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.377+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/587458908 wait complete. 2024-04-21T08:41:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.378+0000 7fa2fcf6e740 1 -- start start 2024-04-21T08:41:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.378+0000 7fa2fcf6e740 1 Processor -- start 2024-04-21T08:41:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.378+0000 7fa2fcf6e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555684deeb60 0x555684df7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.378+0000 7fa2fcf6e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555684def8d0 0x555684df4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.378+0000 7fa2fcf6e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684df80e0 0x555684df5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.378+0000 7fa2fcf6e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555684dfae10 con 0x555684df80e0 2024-04-21T08:41:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.378+0000 7fa2fcf6e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555684cdbeb0 con 0x555684deeb60 2024-04-21T08:41:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.378+0000 7fa2fcf6e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555684dfaf90 con 0x555684def8d0 2024-04-21T08:41:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.378+0000 7fa2fcf6e740 10 client.0 _ll_register_callbacks cb 0x555684d08b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.378+0000 7fa2f37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684df80e0 0x555684df5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.378+0000 7fa2f37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684df80e0 0x555684df5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47300/0 (socket says 172.21.15.43:47300) 2024-04-21T08:41:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.378+0000 7fa2f37fe640 1 -- 172.21.15.43:0/4079422570 learned_addr learned my addr 172.21.15.43:0/4079422570 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.378+0000 7fa2f37fe640 1 --2- 172.21.15.43:0/4079422570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555684deeb60 0x555684df7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.378+0000 7fa2f3fff640 1 --2- 172.21.15.43:0/4079422570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555684def8d0 0x555684df4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.379+0000 7fa2f37fe640 1 -- 172.21.15.43:0/4079422570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555684def8d0 msgr2=0x555684df4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.379+0000 7fa2f37fe640 1 --2- 172.21.15.43:0/4079422570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555684def8d0 0x555684df4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.379+0000 7fa2f37fe640 1 -- 172.21.15.43:0/4079422570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555684deeb60 msgr2=0x555684df7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.379+0000 7fa2f37fe640 1 --2- 172.21.15.43:0/4079422570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555684deeb60 0x555684df7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.379+0000 7fa2f37fe640 1 -- 172.21.15.43:0/4079422570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555684a66210 con 0x555684df80e0 2024-04-21T08:41:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.379+0000 7fa2f37fe640 1 --2- 172.21.15.43:0/4079422570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684df80e0 0x555684df5470 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009230 tx=0x7fa2dc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d136a0f36f7aba server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.379+0000 7fa2d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.379+0000 7fa2d37fe640 1 -- 172.21.15.43:0/4079422570 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc0046c0 con 0x555684df80e0 2024-04-21T08:41:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.379+0000 7fa2d37fe640 1 -- 172.21.15.43:0/4079422570 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fa2dc004860 con 0x555684df80e0 2024-04-21T08:41:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.379+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/4079422570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2e401cad0 con 0x555684df80e0 2024-04-21T08:41:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.379+0000 7fa2d37fe640 1 -- 172.21.15.43:0/4079422570 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa2dc056050 con 0x555684df80e0 2024-04-21T08:41:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.379+0000 7fa2fcf6e740 10 client.16629 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.379+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/4079422570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555684b32620 con 0x555684df80e0 2024-04-21T08:41:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.379+0000 7fa2d37fe640 1 -- 172.21.15.43:0/4079422570 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2e401cad0 con 0x555684df80e0 2024-04-21T08:41:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.380+0000 7fa2d37fe640 1 -- 172.21.15.43:0/4079422570 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa2dc003aa0 con 0x555684df80e0 2024-04-21T08:41:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.380+0000 7fa2fcf6e740 10 client.16629 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.380+0000 7fa2fcf6e740 10 client.16629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.380+0000 7fa2fcf6e740 20 client.16629 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.380+0000 7fa2fcf6e740 1 --2- 172.21.15.43:0/4079422570 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555684e080e0 0x555684e0a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.380+0000 7fa2fcf6e740 4 client.16629 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.380+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/4079422570 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555684d67880 con 0x555684e080e0 2024-04-21T08:41:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.380+0000 7fa2f3fff640 1 --2- 172.21.15.43:0/4079422570 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555684e080e0 0x555684e0a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.380+0000 7fa2d37fe640 10 client.16629 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.380+0000 7fa2f3fff640 1 --2- 172.21.15.43:0/4079422570 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555684e080e0 0x555684e0a4d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0c03b47b5ea1fc8 server_cookie=507ccb56a02163a9 in_seq=0 out_seq=0 2024-04-21T08:41:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.381+0000 7fa2d37fe640 1 -- 172.21.15.43:0/4079422570 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555684d67880 con 0x555684e080e0 2024-04-21T08:41:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.381+0000 7fa2d37fe640 10 client.16629 handle_command_reply: tid=0 2024-04-21T08:41:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.381+0000 7fa2fcf6e740 1 client.16629 shutdown 2024-04-21T08:41:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.382+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/4079422570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684df80e0 msgr2=0x555684df5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.382+0000 7fa2fcf6e740 1 --2- 172.21.15.43:0/4079422570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684df80e0 0x555684df5470 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009230 tx=0x7fa2dc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.382+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/4079422570 shutdown_connections 2024-04-21T08:41:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.382+0000 7fa2fcf6e740 1 --2- 172.21.15.43:0/4079422570 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555684e080e0 0x555684e0a4d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.382+0000 7fa2fcf6e740 1 --2- 172.21.15.43:0/4079422570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684df80e0 0x555684df5470 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.382+0000 7fa2fcf6e740 1 --2- 172.21.15.43:0/4079422570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555684def8d0 0x555684df4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.382+0000 7fa2d37fe640 0 client.16629 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.382+0000 7fa2fcf6e740 1 --2- 172.21.15.43:0/4079422570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555684deeb60 0x555684df7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.382+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/4079422570 >> 172.21.15.43:0/4079422570 conn(0x555684db5c90 msgr2=0x555684deb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.382+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/4079422570 shutdown_connections 2024-04-21T08:41:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.382+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/4079422570 wait complete. 2024-04-21T08:41:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.382+0000 7fa2fcf6e740 20 client.16629 trim_cache size 0 max 16384 2024-04-21T08:41:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.397+0000 7fa2fcf6e740 1 Processor -- start 2024-04-21T08:41:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.397+0000 7fa2fcf6e740 1 -- start start 2024-04-21T08:41:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.397+0000 7fa2fcf6e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684deeb60 0x555684e03d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.397+0000 7fa2fcf6e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555684def8d0 0x555684df7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.397+0000 7fa2fcf6e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555684df80e0 0x555684df7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.397+0000 7fa2fcf6e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f41af3a0 con 0x555684deeb60 2024-04-21T08:41:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.397+0000 7fa2fcf6e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f41b0ee0 con 0x555684def8d0 2024-04-21T08:41:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.397+0000 7fa2fcf6e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f41afa70 con 0x555684df80e0 2024-04-21T08:41:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.397+0000 7fa2f37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684deeb60 0x555684e03d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.397+0000 7fa2f37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684deeb60 0x555684e03d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47306/0 (socket says 172.21.15.43:47306) 2024-04-21T08:41:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.397+0000 7fa2f37fe640 1 -- 172.21.15.43:0/2807355184 learned_addr learned my addr 172.21.15.43:0/2807355184 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.397+0000 7fa2f37fe640 1 -- 172.21.15.43:0/2807355184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555684df80e0 msgr2=0x555684df7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.397+0000 7fa2f3fff640 1 --2- 172.21.15.43:0/2807355184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555684def8d0 0x555684df7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.397+0000 7fa2f37fe640 1 --2- 172.21.15.43:0/2807355184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555684df80e0 0x555684df7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.397+0000 7fa2f37fe640 1 -- 172.21.15.43:0/2807355184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555684def8d0 msgr2=0x555684df7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.397+0000 7fa2f37fe640 1 --2- 172.21.15.43:0/2807355184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555684def8d0 0x555684df7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.397+0000 7fa2f37fe640 1 -- 172.21.15.43:0/2807355184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555684dfa840 con 0x555684deeb60 2024-04-21T08:41:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.397+0000 7fa2f3fff640 1 --2- 172.21.15.43:0/2807355184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555684def8d0 0x555684df7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.398+0000 7fa2f37fe640 1 --2- 172.21.15.43:0/2807355184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684deeb60 0x555684e03d10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc0096c0 tx=0x7fa2dc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96dcf0b52b0cdb42 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.398+0000 7fa2e9ffb640 1 -- 172.21.15.43:0/2807355184 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc0043b0 con 0x555684deeb60 2024-04-21T08:41:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.398+0000 7fa2e9ffb640 1 -- 172.21.15.43:0/2807355184 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2dc053070 con 0x555684deeb60 2024-04-21T08:41:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.398+0000 7fa2e9ffb640 1 -- 172.21.15.43:0/2807355184 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc003850 con 0x555684deeb60 2024-04-21T08:41:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.398+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/2807355184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684deeb60 msgr2=0x555684e03d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.398+0000 7fa2fcf6e740 1 --2- 172.21.15.43:0/2807355184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684deeb60 0x555684e03d10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc0096c0 tx=0x7fa2dc009260 comp rx=0 tx=0).stop 2024-04-21T08:41:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.398+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/2807355184 shutdown_connections 2024-04-21T08:41:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.398+0000 7fa2fcf6e740 1 --2- 172.21.15.43:0/2807355184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555684df80e0 0x555684df7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.398+0000 7fa2fcf6e740 1 --2- 172.21.15.43:0/2807355184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555684def8d0 0x555684df7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.398+0000 7fa2fcf6e740 1 --2- 172.21.15.43:0/2807355184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684deeb60 0x555684e03d10 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.398+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/2807355184 >> 172.21.15.43:0/2807355184 conn(0x555684db5c90 msgr2=0x555684de9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.398+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/2807355184 shutdown_connections 2024-04-21T08:41:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.398+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/2807355184 wait complete. 2024-04-21T08:41:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.399+0000 7fa2fcf6e740 1 -- start start 2024-04-21T08:41:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.399+0000 7fa2fcf6e740 1 Processor -- start 2024-04-21T08:41:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.399+0000 7fa2fcf6e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555684deeb60 0x555684df84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.399+0000 7fa2fcf6e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684def8d0 0x555684df8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.399+0000 7fa2fcf6e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555684df80e0 0x555684df8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.399+0000 7fa2fcf6e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f41b0810 con 0x555684def8d0 2024-04-21T08:41:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.399+0000 7fa2fcf6e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f41b0140 con 0x555684df80e0 2024-04-21T08:41:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.399+0000 7fa2fcf6e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f41ad380 con 0x555684deeb60 2024-04-21T08:41:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.399+0000 7fa2fcf6e740 10 client.0 _ll_register_callbacks cb 0x7fa2dc005040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.399+0000 7fa2f3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684def8d0 0x555684df8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.399+0000 7fa2f3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684def8d0 0x555684df8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47318/0 (socket says 172.21.15.43:47318) 2024-04-21T08:41:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.399+0000 7fa2f3fff640 1 -- 172.21.15.43:0/2122007970 learned_addr learned my addr 172.21.15.43:0/2122007970 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.400+0000 7fa2f37fe640 1 --2- 172.21.15.43:0/2122007970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555684deeb60 0x555684df84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.400+0000 7fa2f37fe640 1 --2- 172.21.15.43:0/2122007970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555684df80e0 0x555684df8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.400+0000 7fa2f3fff640 1 -- 172.21.15.43:0/2122007970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555684deeb60 msgr2=0x555684df84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.400+0000 7fa2f3fff640 1 --2- 172.21.15.43:0/2122007970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555684deeb60 0x555684df84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.400+0000 7fa2f3fff640 1 -- 172.21.15.43:0/2122007970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555684df80e0 msgr2=0x555684df8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.400+0000 7fa2f3fff640 1 --2- 172.21.15.43:0/2122007970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555684df80e0 0x555684df8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.400+0000 7fa2f3fff640 1 -- 172.21.15.43:0/2122007970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555684dff490 con 0x555684def8d0 2024-04-21T08:41:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.400+0000 7fa2f37fe640 1 --2- 172.21.15.43:0/2122007970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555684df80e0 0x555684df8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.400+0000 7fa2f3fff640 1 --2- 172.21.15.43:0/2122007970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684def8d0 0x555684df8a00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4005fd0 tx=0x7fa2e400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382db38ba0174765 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.400+0000 7fa2d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.400+0000 7fa2d37fe640 1 -- 172.21.15.43:0/2122007970 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4005230 con 0x555684def8d0 2024-04-21T08:41:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.400+0000 7fa2d37fe640 1 -- 172.21.15.43:0/2122007970 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fa2e40053d0 con 0x555684def8d0 2024-04-21T08:41:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.400+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/2122007970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2dc053070 con 0x555684def8d0 2024-04-21T08:41:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.400+0000 7fa2d37fe640 1 -- 172.21.15.43:0/2122007970 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa2e4036050 con 0x555684def8d0 2024-04-21T08:41:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.400+0000 7fa2fcf6e740 10 client.16635 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.400+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/2122007970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555684e00740 con 0x555684def8d0 2024-04-21T08:41:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.400+0000 7fa2d37fe640 1 -- 172.21.15.43:0/2122007970 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2dc053070 con 0x555684def8d0 2024-04-21T08:41:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.401+0000 7fa2d37fe640 1 -- 172.21.15.43:0/2122007970 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa2e4004660 con 0x555684def8d0 2024-04-21T08:41:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.401+0000 7fa2fcf6e740 10 client.16635 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.401+0000 7fa2fcf6e740 10 client.16635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.401+0000 7fa2fcf6e740 20 client.16635 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.401+0000 7fa2fcf6e740 1 --2- 172.21.15.43:0/2122007970 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555684e72a10 0x555684e74e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.401+0000 7fa2fcf6e740 4 client.16635 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:41:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.401+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/2122007970 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555684cf33d0 con 0x555684e72a10 2024-04-21T08:41:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.401+0000 7fa2f37fe640 1 --2- 172.21.15.43:0/2122007970 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555684e72a10 0x555684e74e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.401+0000 7fa2f37fe640 1 --2- 172.21.15.43:0/2122007970 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555684e72a10 0x555684e74e00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42f80365c56f3721 server_cookie=9b7c6b9d1d0b1f10 in_seq=0 out_seq=0 2024-04-21T08:41:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.401+0000 7fa2d37fe640 10 client.16635 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.416+0000 7fa2d37fe640 1 -- 172.21.15.43:0/2122007970 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555684cf33d0 con 0x555684e72a10 2024-04-21T08:41:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.416+0000 7fa2d37fe640 10 client.16635 handle_command_reply: tid=0 2024-04-21T08:41:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.416+0000 7fa2fcf6e740 1 client.16635 shutdown 2024-04-21T08:41:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.416+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/2122007970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684def8d0 msgr2=0x555684df8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.416+0000 7fa2fcf6e740 1 --2- 172.21.15.43:0/2122007970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684def8d0 0x555684df8a00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4005fd0 tx=0x7fa2e400b480 comp rx=0 tx=0).stop 2024-04-21T08:41:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.416+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/2122007970 shutdown_connections 2024-04-21T08:41:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.416+0000 7fa2fcf6e740 1 --2- 172.21.15.43:0/2122007970 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555684e72a10 0x555684e74e00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.416+0000 7fa2fcf6e740 1 --2- 172.21.15.43:0/2122007970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555684df80e0 0x555684df8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.416+0000 7fa2fcf6e740 1 --2- 172.21.15.43:0/2122007970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555684def8d0 0x555684df8a00 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.416+0000 7fa2d37fe640 0 client.16635 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.416+0000 7fa2fcf6e740 1 --2- 172.21.15.43:0/2122007970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555684deeb60 0x555684df84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.416+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/2122007970 >> 172.21.15.43:0/2122007970 conn(0x555684db5c90 msgr2=0x555684df36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.416+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/2122007970 shutdown_connections 2024-04-21T08:41:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.416+0000 7fa2fcf6e740 1 -- 172.21.15.43:0/2122007970 wait complete. 2024-04-21T08:41:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.416+0000 7fa2fcf6e740 20 client.16635 trim_cache size 0 max 16384 2024-04-21T08:41:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:41:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:41:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:41:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:41:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.417+0000 7fa2fa29a640 1 -- 172.21.15.43:0/1609597599 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa2e00a3dc0 msgr2=0x7fa2e00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.417+0000 7fa2fa29a640 1 --2- 172.21.15.43:0/1609597599 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa2e00a3dc0 0x7fa2e00a6280 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0ce6e0 tx=0x7fa2ec01baf0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.417+0000 7fa2fa29a640 1 -- 172.21.15.43:0/1609597599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2f419cf80 msgr2=0x7fa2f419d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.417+0000 7fa2fa29a640 1 --2- 172.21.15.43:0/1609597599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2f419cf80 0x7fa2f419d430 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec00cab0 tx=0x7fa2ec00cf70 comp rx=0 tx=0).stop 2024-04-21T08:41:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.417+0000 7fa2fa29a640 1 -- 172.21.15.43:0/1609597599 shutdown_connections 2024-04-21T08:41:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.417+0000 7fa2fa29a640 1 --2- 172.21.15.43:0/1609597599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2f419cf80 0x7fa2f419d430 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.417+0000 7fa2fa29a640 1 --2- 172.21.15.43:0/1609597599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2f41072a0 0x7fa2f4198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.417+0000 7fa2fa29a640 1 --2- 172.21.15.43:0/1609597599 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa2e00a3dc0 0x7fa2e00a6280 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.417+0000 7fa2fa29a640 1 --2- 172.21.15.43:0/1609597599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2f41021d0 0x7fa2f41986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.417+0000 7fa2fa29a640 1 -- 172.21.15.43:0/1609597599 >> 172.21.15.43:0/1609597599 conn(0x7fa2f40793e0 msgr2=0x7fa2f40ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.417+0000 7fa2fa29a640 1 -- 172.21.15.43:0/1609597599 shutdown_connections 2024-04-21T08:41:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:00.417+0000 7fa2fa29a640 1 -- 172.21.15.43:0/1609597599 wait complete. 2024-04-21T08:41:00.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:40:59 smithi175 ceph-mon[31920]: pgmap v65: 129 pgs: 129 active+clean; 2.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-21T08:41:02.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:02 smithi112 ceph-mon[31085]: pgmap v66: 129 pgs: 129 active+clean; 2.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 KiB/s wr, 39 op/s 2024-04-21T08:41:02.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:02 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:41:02.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:02 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:02.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:02 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:41:02.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:02 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:41:02.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:02 smithi043 ceph-mon[27978]: pgmap v66: 129 pgs: 129 active+clean; 2.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 KiB/s wr, 39 op/s 2024-04-21T08:41:02.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:02 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:41:02.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:02 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:02.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:02 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:41:02.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:02 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:41:02.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:02 smithi175 ceph-mon[31920]: pgmap v66: 129 pgs: 129 active+clean; 2.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 KiB/s wr, 39 op/s 2024-04-21T08:41:02.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:02 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:41:02.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:02 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:02.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:02 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:41:02.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:02 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:41:04.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:04 smithi112 ceph-mon[31085]: pgmap v67: 129 pgs: 129 active+clean; 2.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 40 KiB/s wr, 30 op/s 2024-04-21T08:41:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:04 smithi043 ceph-mon[27978]: pgmap v67: 129 pgs: 129 active+clean; 2.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 40 KiB/s wr, 30 op/s 2024-04-21T08:41:04.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:04 smithi175 ceph-mon[31920]: pgmap v67: 129 pgs: 129 active+clean; 2.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 40 KiB/s wr, 30 op/s 2024-04-21T08:41:06.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:06 smithi112 ceph-mon[31085]: pgmap v68: 129 pgs: 129 active+clean; 2.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 40 KiB/s wr, 30 op/s 2024-04-21T08:41:06.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:06 smithi043 ceph-mon[27978]: pgmap v68: 129 pgs: 129 active+clean; 2.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 40 KiB/s wr, 30 op/s 2024-04-21T08:41:06.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:06 smithi175 ceph-mon[31920]: pgmap v68: 129 pgs: 129 active+clean; 2.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 40 KiB/s wr, 30 op/s 2024-04-21T08:41:08.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:08 smithi112 ceph-mon[31085]: pgmap v69: 129 pgs: 129 active+clean; 2.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 50 KiB/s wr, 37 op/s 2024-04-21T08:41:08.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:08 smithi043 ceph-mon[27978]: pgmap v69: 129 pgs: 129 active+clean; 2.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 50 KiB/s wr, 37 op/s 2024-04-21T08:41:08.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:08 smithi175 ceph-mon[31920]: pgmap v69: 129 pgs: 129 active+clean; 2.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 50 KiB/s wr, 37 op/s 2024-04-21T08:41:08.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:41:08.721 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:41:08.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.840+0000 7f0cdbb15640 1 -- 172.21.15.43:0/3973886414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd406a5b0 msgr2=0x7f0cd406a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:08.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.840+0000 7f0cdbb15640 1 --2- 172.21.15.43:0/3973886414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd406a5b0 0x7f0cd406a990 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009930 tx=0x7f0cd002bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:08.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.841+0000 7f0cdbb15640 1 -- 172.21.15.43:0/3973886414 shutdown_connections 2024-04-21T08:41:08.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.841+0000 7f0cdbb15640 1 --2- 172.21.15.43:0/3973886414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cd410ec50 0x7f0cd4111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.841+0000 7f0cdbb15640 1 --2- 172.21.15.43:0/3973886414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cd406aed0 0x7f0cd410e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.841+0000 7f0cdbb15640 1 --2- 172.21.15.43:0/3973886414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd406a5b0 0x7f0cd406a990 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.841+0000 7f0cdbb15640 1 -- 172.21.15.43:0/3973886414 >> 172.21.15.43:0/3973886414 conn(0x7f0cd4077c40 msgr2=0x7f0cd4078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:08.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.841+0000 7f0cdbb15640 1 -- 172.21.15.43:0/3973886414 shutdown_connections 2024-04-21T08:41:08.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.841+0000 7f0cdbb15640 1 -- 172.21.15.43:0/3973886414 wait complete. 2024-04-21T08:41:08.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.841+0000 7f0cdbb15640 1 Processor -- start 2024-04-21T08:41:08.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.841+0000 7f0cdbb15640 1 -- start start 2024-04-21T08:41:08.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.842+0000 7f0cdbb15640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cd406a5b0 0x7f0cd41a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:08.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.842+0000 7f0cdbb15640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd406aed0 0x7f0cd41a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:08.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.842+0000 7f0cdbb15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cd410ec50 0x7f0cd419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:08.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.842+0000 7f0cdbb15640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd41143d0 con 0x7f0cd406aed0 2024-04-21T08:41:08.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.842+0000 7f0cdbb15640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd4114250 con 0x7f0cd410ec50 2024-04-21T08:41:08.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.842+0000 7f0cdbb15640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd4114550 con 0x7f0cd406a5b0 2024-04-21T08:41:08.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.842+0000 7f0cd90b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd406aed0 0x7f0cd41a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:08.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.842+0000 7f0cd90b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd406aed0 0x7f0cd41a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55628/0 (socket says 172.21.15.43:55628) 2024-04-21T08:41:08.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.842+0000 7f0cd90b0640 1 -- 172.21.15.43:0/2681905176 learned_addr learned my addr 172.21.15.43:0/2681905176 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:08.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.842+0000 7f0cd98b1640 1 --2- 172.21.15.43:0/2681905176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cd406a5b0 0x7f0cd41a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:08.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.842+0000 7f0cda0b2640 1 --2- 172.21.15.43:0/2681905176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cd410ec50 0x7f0cd419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:08.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.842+0000 7f0cd90b0640 1 -- 172.21.15.43:0/2681905176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cd406a5b0 msgr2=0x7f0cd41a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:08.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.842+0000 7f0cd90b0640 1 --2- 172.21.15.43:0/2681905176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cd406a5b0 0x7f0cd41a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.842+0000 7f0cd90b0640 1 -- 172.21.15.43:0/2681905176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cd410ec50 msgr2=0x7f0cd419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:08.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.842+0000 7f0cd90b0640 1 --2- 172.21.15.43:0/2681905176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cd410ec50 0x7f0cd419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.842+0000 7f0cd90b0640 1 -- 172.21.15.43:0/2681905176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd419af90 con 0x7f0cd406aed0 2024-04-21T08:41:08.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.842+0000 7f0cd90b0640 1 --2- 172.21.15.43:0/2681905176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd406aed0 0x7f0cd41a0b50 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400c890 tx=0x7f0cc400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:08.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.843+0000 7f0ccaffd640 1 -- 172.21.15.43:0/2681905176 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4004220 con 0x7f0cd406aed0 2024-04-21T08:41:08.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.843+0000 7f0ccaffd640 1 -- 172.21.15.43:0/2681905176 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cc40043c0 con 0x7f0cd406aed0 2024-04-21T08:41:08.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.843+0000 7f0cdbb15640 1 -- 172.21.15.43:0/2681905176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd419b280 con 0x7f0cd406aed0 2024-04-21T08:41:08.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.843+0000 7f0ccaffd640 1 -- 172.21.15.43:0/2681905176 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4004a00 con 0x7f0cd406aed0 2024-04-21T08:41:08.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.843+0000 7f0cdbb15640 1 -- 172.21.15.43:0/2681905176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd41a7590 con 0x7f0cd406aed0 2024-04-21T08:41:08.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.845+0000 7f0ccaffd640 1 -- 172.21.15.43:0/2681905176 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0cc40079b0 con 0x7f0cd406aed0 2024-04-21T08:41:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.845+0000 7f0cde7de740 1 Processor -- start 2024-04-21T08:41:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.846+0000 7f0cde7de740 1 -- start start 2024-04-21T08:41:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.846+0000 7f0cde7de740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574a29f1580 0x5574a29f1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.846+0000 7f0cde7de740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574a29f1ea0 0x5574a29fa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.846+0000 7f0cde7de740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29faa80 0x5574a29fce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.846+0000 7f0cde7de740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5574a29561e0 con 0x5574a29faa80 2024-04-21T08:41:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.846+0000 7f0cde7de740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5574a28f3390 con 0x5574a29f1ea0 2024-04-21T08:41:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.846+0000 7f0cde7de740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5574a29521a0 con 0x5574a29f1580 2024-04-21T08:41:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.846+0000 7f0ccaffd640 1 --2- 172.21.15.43:0/2681905176 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0cc00a3da0 0x7f0cc00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.846+0000 7f0cd90b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29faa80 0x5574a29fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.846+0000 7f0cd90b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29faa80 0x5574a29fce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55640/0 (socket says 172.21.15.43:55640) 2024-04-21T08:41:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.846+0000 7f0cd90b0640 1 -- 172.21.15.43:0/35452775 learned_addr learned my addr 172.21.15.43:0/35452775 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.846+0000 7f0cda0b2640 1 --2- 172.21.15.43:0/35452775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574a29f1580 0x5574a29f1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.846+0000 7f0cd98b1640 1 --2- 172.21.15.43:0/35452775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574a29f1ea0 0x5574a29fa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.846+0000 7f0cda0b2640 1 --2- 172.21.15.43:0/2681905176 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0cc00a3da0 0x7f0cc00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.847+0000 7f0ccaffd640 1 -- 172.21.15.43:0/2681905176 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f0cc40d08a0 con 0x7f0cd406aed0 2024-04-21T08:41:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.847+0000 7f0cda0b2640 1 -- 172.21.15.43:0/35452775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574a29f1ea0 msgr2=0x5574a29fa540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.847+0000 7f0cda0b2640 1 --2- 172.21.15.43:0/35452775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574a29f1ea0 0x5574a29fa540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.847+0000 7f0cda0b2640 1 -- 172.21.15.43:0/35452775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29faa80 msgr2=0x5574a29fce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.847+0000 7f0cda0b2640 1 --2- 172.21.15.43:0/35452775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29faa80 0x5574a29fce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.847+0000 7f0cda0b2640 1 -- 172.21.15.43:0/35452775 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574a27327d0 con 0x5574a29f1580 2024-04-21T08:41:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.847+0000 7f0cd90b0640 1 --2- 172.21.15.43:0/35452775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29faa80 0x5574a29fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:41:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.847+0000 7f0cda0b2640 1 --2- 172.21.15.43:0/35452775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574a29f1580 0x5574a29f1960 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc005fd0 tx=0x7f0ccc012da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43de8e6c0137bb6f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.847+0000 7f0cda0b2640 1 --2- 172.21.15.43:0/2681905176 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0cc00a3da0 0x7f0cc00a6260 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc01b000 tx=0x7f0ccc01d000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.847+0000 7f0cbffff640 1 -- 172.21.15.43:0/35452775 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc022070 con 0x5574a29f1580 2024-04-21T08:41:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.847+0000 7f0cbffff640 1 -- 172.21.15.43:0/35452775 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ccc0095e0 con 0x5574a29f1580 2024-04-21T08:41:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.847+0000 7f0cbffff640 1 -- 172.21.15.43:0/35452775 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc01dd30 con 0x5574a29f1580 2024-04-21T08:41:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.848+0000 7f0cde7de740 1 -- 172.21.15.43:0/35452775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574a29f1580 msgr2=0x5574a29f1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:08.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.848+0000 7f0cde7de740 1 --2- 172.21.15.43:0/35452775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574a29f1580 0x5574a29f1960 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc005fd0 tx=0x7f0ccc012da0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.848+0000 7f0cde7de740 1 -- 172.21.15.43:0/35452775 shutdown_connections 2024-04-21T08:41:08.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.848+0000 7f0cde7de740 1 --2- 172.21.15.43:0/35452775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29faa80 0x5574a29fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.848+0000 7f0cde7de740 1 --2- 172.21.15.43:0/35452775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574a29f1ea0 0x5574a29fa540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.848+0000 7f0cde7de740 1 --2- 172.21.15.43:0/35452775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574a29f1580 0x5574a29f1960 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.848+0000 7f0cde7de740 1 -- 172.21.15.43:0/35452775 >> 172.21.15.43:0/35452775 conn(0x5574a29ebcb0 msgr2=0x5574a29ec090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:08.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.848+0000 7f0cde7de740 1 -- 172.21.15.43:0/35452775 shutdown_connections 2024-04-21T08:41:08.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.848+0000 7f0cde7de740 1 -- 172.21.15.43:0/35452775 wait complete. 2024-04-21T08:41:08.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.848+0000 7f0cde7de740 1 -- start start 2024-04-21T08:41:08.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.849+0000 7f0cde7de740 1 Processor -- start 2024-04-21T08:41:08.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.849+0000 7f0cde7de740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574a29f1580 0x5574a29f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:08.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.849+0000 7f0cde7de740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574a29f1ea0 0x5574a29f8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:08.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.849+0000 7f0cde7de740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29faa80 0x5574a29f8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:08.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.849+0000 7f0cde7de740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5574a29fd810 con 0x5574a29faa80 2024-04-21T08:41:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.849+0000 7f0cde7de740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5574a290f2f0 con 0x5574a29f1ea0 2024-04-21T08:41:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.849+0000 7f0cde7de740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5574a29fd990 con 0x5574a29f1580 2024-04-21T08:41:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.849+0000 7f0cde7de740 10 client.0 _ll_register_callbacks cb 0x5574a291c580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.849+0000 7f0cda0b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29faa80 0x5574a29f8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.849+0000 7f0cda0b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29faa80 0x5574a29f8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55650/0 (socket says 172.21.15.43:55650) 2024-04-21T08:41:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.849+0000 7f0cda0b2640 1 -- 172.21.15.43:0/4124306349 learned_addr learned my addr 172.21.15.43:0/4124306349 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.849+0000 7f0cd98b1640 1 --2- 172.21.15.43:0/4124306349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574a29f1580 0x5574a29f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.849+0000 7f0cd90b0640 1 --2- 172.21.15.43:0/4124306349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574a29f1ea0 0x5574a29f8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.849+0000 7f0cda0b2640 1 -- 172.21.15.43:0/4124306349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574a29f1580 msgr2=0x5574a29f7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.849+0000 7f0cda0b2640 1 --2- 172.21.15.43:0/4124306349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574a29f1580 0x5574a29f7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.849+0000 7f0cda0b2640 1 -- 172.21.15.43:0/4124306349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574a29f1ea0 msgr2=0x5574a29f8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.849+0000 7f0cda0b2640 1 --2- 172.21.15.43:0/4124306349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574a29f1ea0 0x5574a29f8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.849+0000 7f0cda0b2640 1 -- 172.21.15.43:0/4124306349 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574a26683f0 con 0x5574a29faa80 2024-04-21T08:41:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.849+0000 7f0cd98b1640 1 --2- 172.21.15.43:0/4124306349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574a29f1580 0x5574a29f7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:41:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.850+0000 7f0cda0b2640 1 --2- 172.21.15.43:0/4124306349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29faa80 0x5574a29f8570 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009f30 tx=0x7f0ccc005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f899438049dc15c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:08.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.850+0000 7f0cbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:08.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.850+0000 7f0cbd7fa640 1 -- 172.21.15.43:0/4124306349 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc022040 con 0x5574a29faa80 2024-04-21T08:41:08.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.850+0000 7f0cbd7fa640 1 -- 172.21.15.43:0/4124306349 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f0ccc01c7f0 con 0x5574a29faa80 2024-04-21T08:41:08.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.850+0000 7f0cde7de740 1 -- 172.21.15.43:0/4124306349 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ccc0095e0 con 0x5574a29faa80 2024-04-21T08:41:08.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.850+0000 7f0cbd7fa640 1 -- 172.21.15.43:0/4124306349 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0ccc01c9f0 con 0x5574a29faa80 2024-04-21T08:41:08.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.850+0000 7f0cde7de740 10 client.16647 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:08.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.850+0000 7f0cde7de740 1 -- 172.21.15.43:0/4124306349 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5574a26280f0 con 0x5574a29faa80 2024-04-21T08:41:08.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.850+0000 7f0cbd7fa640 1 -- 172.21.15.43:0/4124306349 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ccc0095e0 con 0x5574a29faa80 2024-04-21T08:41:08.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.850+0000 7f0cbd7fa640 1 -- 172.21.15.43:0/4124306349 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0ccc030bf0 con 0x5574a29faa80 2024-04-21T08:41:08.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.851+0000 7f0cde7de740 10 client.16647 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:08.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.851+0000 7f0cde7de740 10 client.16647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:08.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.851+0000 7f0cde7de740 20 client.16647 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:08.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.851+0000 7f0cde7de740 1 --2- 172.21.15.43:0/4124306349 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5574a2a0aa60 0x5574a2a0ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:08.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.851+0000 7f0cde7de740 4 client.16647 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:08.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.851+0000 7f0cde7de740 1 -- 172.21.15.43:0/4124306349 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574a2963c00 con 0x5574a2a0aa60 2024-04-21T08:41:08.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.851+0000 7f0cd98b1640 1 --2- 172.21.15.43:0/4124306349 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5574a2a0aa60 0x5574a2a0ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:08.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.851+0000 7f0cbd7fa640 10 client.16647 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:08.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.851+0000 7f0cd98b1640 1 --2- 172.21.15.43:0/4124306349 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5574a2a0aa60 0x5574a2a0ce50 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fae76eb0748e2405 server_cookie=ea5ad60439c9abae in_seq=0 out_seq=0 2024-04-21T08:41:08.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.852+0000 7f0cbd7fa640 1 -- 172.21.15.43:0/4124306349 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5574a2963c00 con 0x5574a2a0aa60 2024-04-21T08:41:08.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.852+0000 7f0cbd7fa640 10 client.16647 handle_command_reply: tid=0 2024-04-21T08:41:08.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.852+0000 7f0cde7de740 1 client.16647 shutdown 2024-04-21T08:41:08.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.852+0000 7f0cde7de740 1 -- 172.21.15.43:0/4124306349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29faa80 msgr2=0x5574a29f8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:08.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.852+0000 7f0cde7de740 1 --2- 172.21.15.43:0/4124306349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29faa80 0x5574a29f8570 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009f30 tx=0x7f0ccc005af0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.852+0000 7f0cde7de740 1 -- 172.21.15.43:0/4124306349 shutdown_connections 2024-04-21T08:41:08.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.852+0000 7f0cde7de740 1 --2- 172.21.15.43:0/4124306349 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5574a2a0aa60 0x5574a2a0ce50 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.852+0000 7f0cde7de740 1 --2- 172.21.15.43:0/4124306349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29faa80 0x5574a29f8570 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.852+0000 7f0cde7de740 1 --2- 172.21.15.43:0/4124306349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574a29f1ea0 0x5574a29f8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.852+0000 7f0cbd7fa640 0 client.16647 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:08.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.852+0000 7f0cde7de740 1 --2- 172.21.15.43:0/4124306349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574a29f1580 0x5574a29f7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.852+0000 7f0cde7de740 1 -- 172.21.15.43:0/4124306349 >> 172.21.15.43:0/4124306349 conn(0x5574a29ebcb0 msgr2=0x5574a29eda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:08.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.852+0000 7f0cde7de740 1 -- 172.21.15.43:0/4124306349 shutdown_connections 2024-04-21T08:41:08.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.852+0000 7f0cde7de740 1 -- 172.21.15.43:0/4124306349 wait complete. 2024-04-21T08:41:08.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.853+0000 7f0cde7de740 20 client.16647 trim_cache size 0 max 16384 2024-04-21T08:41:08.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.870+0000 7f0cde7de740 1 Processor -- start 2024-04-21T08:41:08.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.870+0000 7f0cde7de740 1 -- start start 2024-04-21T08:41:08.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.870+0000 7f0cde7de740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29f1580 0x5574a2a066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:08.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.870+0000 7f0cde7de740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574a29f1ea0 0x5574a29f73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:08.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.871+0000 7f0cde7de740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574a29faa80 0x5574a29ed970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:08.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.871+0000 7f0cde7de740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd41b15a0 con 0x5574a29f1580 2024-04-21T08:41:08.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.871+0000 7f0cde7de740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd41b30e0 con 0x5574a29faa80 2024-04-21T08:41:08.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.871+0000 7f0cde7de740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd41b1c70 con 0x5574a29f1ea0 2024-04-21T08:41:08.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.871+0000 7f0cd98b1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29f1580 0x5574a2a066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:08.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.871+0000 7f0cd98b1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29f1580 0x5574a2a066d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55660/0 (socket says 172.21.15.43:55660) 2024-04-21T08:41:08.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.871+0000 7f0cd98b1640 1 -- 172.21.15.43:0/1240131017 learned_addr learned my addr 172.21.15.43:0/1240131017 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:08.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.871+0000 7f0cd90b0640 1 --2- 172.21.15.43:0/1240131017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574a29f1ea0 0x5574a29f73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:08.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.871+0000 7f0cda0b2640 1 --2- 172.21.15.43:0/1240131017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574a29faa80 0x5574a29ed970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:08.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.871+0000 7f0cd98b1640 1 -- 172.21.15.43:0/1240131017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574a29f1ea0 msgr2=0x5574a29f73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:08.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.871+0000 7f0cd98b1640 1 --2- 172.21.15.43:0/1240131017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574a29f1ea0 0x5574a29f73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.871+0000 7f0cd98b1640 1 -- 172.21.15.43:0/1240131017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574a29faa80 msgr2=0x5574a29ed970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:08.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.871+0000 7f0cd98b1640 1 --2- 172.21.15.43:0/1240131017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574a29faa80 0x5574a29ed970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.871+0000 7f0cd98b1640 1 -- 172.21.15.43:0/1240131017 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574a29fd750 con 0x5574a29f1580 2024-04-21T08:41:08.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.871+0000 7f0cd90b0640 1 --2- 172.21.15.43:0/1240131017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574a29f1ea0 0x5574a29f73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:08.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.871+0000 7f0cda0b2640 1 --2- 172.21.15.43:0/1240131017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574a29faa80 0x5574a29ed970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:08.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.871+0000 7f0cd98b1640 1 --2- 172.21.15.43:0/1240131017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29f1580 0x5574a2a066d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009850 tx=0x7f0cd0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0498a14d6ee0f36 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:08.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.871+0000 7f0cbf7fe640 1 -- 172.21.15.43:0/1240131017 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd004bbd0 con 0x5574a29f1580 2024-04-21T08:41:08.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.871+0000 7f0cbf7fe640 1 -- 172.21.15.43:0/1240131017 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cd0050070 con 0x5574a29f1580 2024-04-21T08:41:08.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.871+0000 7f0cbf7fe640 1 -- 172.21.15.43:0/1240131017 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd002cca0 con 0x5574a29f1580 2024-04-21T08:41:08.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.872+0000 7f0cde7de740 1 -- 172.21.15.43:0/1240131017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29f1580 msgr2=0x5574a2a066d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:08.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.872+0000 7f0cde7de740 1 --2- 172.21.15.43:0/1240131017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29f1580 0x5574a2a066d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009850 tx=0x7f0cd0005fd0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.872+0000 7f0cde7de740 1 -- 172.21.15.43:0/1240131017 shutdown_connections 2024-04-21T08:41:08.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.872+0000 7f0cde7de740 1 --2- 172.21.15.43:0/1240131017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574a29faa80 0x5574a29ed970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.872+0000 7f0cde7de740 1 --2- 172.21.15.43:0/1240131017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574a29f1ea0 0x5574a29f73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.872+0000 7f0cde7de740 1 --2- 172.21.15.43:0/1240131017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29f1580 0x5574a2a066d0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.872+0000 7f0cde7de740 1 -- 172.21.15.43:0/1240131017 >> 172.21.15.43:0/1240131017 conn(0x5574a29ebcb0 msgr2=0x5574a29f7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:08.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.872+0000 7f0cde7de740 1 -- 172.21.15.43:0/1240131017 shutdown_connections 2024-04-21T08:41:08.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.872+0000 7f0cde7de740 1 -- 172.21.15.43:0/1240131017 wait complete. 2024-04-21T08:41:08.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.872+0000 7f0cde7de740 1 -- start start 2024-04-21T08:41:08.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.872+0000 7f0cde7de740 1 Processor -- start 2024-04-21T08:41:08.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.873+0000 7f0cde7de740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574a29f1580 0x5574a2a02b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:08.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.873+0000 7f0cde7de740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574a29f1ea0 0x5574a2a030d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:08.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.873+0000 7f0cde7de740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29faa80 0x5574a29fe060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:08.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.873+0000 7f0cde7de740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd41b2a10 con 0x5574a29faa80 2024-04-21T08:41:08.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.873+0000 7f0cde7de740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd41b2340 con 0x5574a29f1580 2024-04-21T08:41:08.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.873+0000 7f0cde7de740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd41af580 con 0x5574a29f1ea0 2024-04-21T08:41:08.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.873+0000 7f0cde7de740 10 client.0 _ll_register_callbacks cb 0x7f0ccc020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:08.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.873+0000 7f0cda0b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29faa80 0x5574a29fe060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:08.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.873+0000 7f0cda0b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29faa80 0x5574a29fe060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55674/0 (socket says 172.21.15.43:55674) 2024-04-21T08:41:08.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.873+0000 7f0cda0b2640 1 -- 172.21.15.43:0/3954526800 learned_addr learned my addr 172.21.15.43:0/3954526800 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:08.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.873+0000 7f0cd98b1640 1 --2- 172.21.15.43:0/3954526800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574a29f1580 0x5574a2a02b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:08.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.873+0000 7f0cd90b0640 1 --2- 172.21.15.43:0/3954526800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574a29f1ea0 0x5574a2a030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:08.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.873+0000 7f0cda0b2640 1 -- 172.21.15.43:0/3954526800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574a29f1ea0 msgr2=0x5574a2a030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:08.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.873+0000 7f0cda0b2640 1 --2- 172.21.15.43:0/3954526800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574a29f1ea0 0x5574a2a030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.873+0000 7f0cda0b2640 1 -- 172.21.15.43:0/3954526800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574a29f1580 msgr2=0x5574a2a02b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:08.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.873+0000 7f0cda0b2640 1 --2- 172.21.15.43:0/3954526800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574a29f1580 0x5574a2a02b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.873+0000 7f0cda0b2640 1 -- 172.21.15.43:0/3954526800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574a29fe890 con 0x5574a29faa80 2024-04-21T08:41:08.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.873+0000 7f0cd90b0640 1 --2- 172.21.15.43:0/3954526800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574a29f1ea0 0x5574a2a030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:08.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.873+0000 7f0cda0b2640 1 --2- 172.21.15.43:0/3954526800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29faa80 0x5574a29fe060 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc027580 tx=0x7f0ccc0127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1283ed74dd0cad server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:08.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.873+0000 7f0cbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:08.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.873+0000 7f0cbd7fa640 1 -- 172.21.15.43:0/3954526800 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc042060 con 0x5574a29faa80 2024-04-21T08:41:08.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.873+0000 7f0cbd7fa640 1 -- 172.21.15.43:0/3954526800 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f0ccc01d400 con 0x5574a29faa80 2024-04-21T08:41:08.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.873+0000 7f0cde7de740 1 -- 172.21.15.43:0/3954526800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cd0050070 con 0x5574a29faa80 2024-04-21T08:41:08.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.874+0000 7f0cbd7fa640 1 -- 172.21.15.43:0/3954526800 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0ccc022030 con 0x5574a29faa80 2024-04-21T08:41:08.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.874+0000 7f0cde7de740 10 client.16653 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:08.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.874+0000 7f0cde7de740 1 -- 172.21.15.43:0/3954526800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5574a2a709b0 con 0x5574a29faa80 2024-04-21T08:41:08.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.874+0000 7f0cbd7fa640 1 -- 172.21.15.43:0/3954526800 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cd0050070 con 0x5574a29faa80 2024-04-21T08:41:08.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.874+0000 7f0cbd7fa640 1 -- 172.21.15.43:0/3954526800 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0ccc027930 con 0x5574a29faa80 2024-04-21T08:41:08.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.874+0000 7f0cde7de740 10 client.16653 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:08.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.874+0000 7f0cde7de740 10 client.16653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:08.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.874+0000 7f0cde7de740 20 client.16653 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:08.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.874+0000 7f0cde7de740 1 --2- 172.21.15.43:0/3954526800 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5574a2a71100 0x5574a2a73520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:08.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.874+0000 7f0cde7de740 4 client.16653 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:41:08.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.874+0000 7f0cde7de740 1 -- 172.21.15.43:0/3954526800 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5574a295da70 con 0x5574a2a71100 2024-04-21T08:41:08.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.874+0000 7f0cd98b1640 1 --2- 172.21.15.43:0/3954526800 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5574a2a71100 0x5574a2a73520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:08.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.875+0000 7f0cbd7fa640 10 client.16653 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:08.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.875+0000 7f0cd98b1640 1 --2- 172.21.15.43:0/3954526800 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5574a2a71100 0x5574a2a73520 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b4545f654d5c791 server_cookie=70acbae7f5947ae2 in_seq=0 out_seq=0 2024-04-21T08:41:08.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.875+0000 7f0cbd7fa640 1 -- 172.21.15.43:0/3954526800 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5574a295da70 con 0x5574a2a71100 2024-04-21T08:41:08.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.875+0000 7f0cbd7fa640 10 client.16653 handle_command_reply: tid=0 2024-04-21T08:41:08.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.875+0000 7f0cde7de740 1 client.16653 shutdown 2024-04-21T08:41:08.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.875+0000 7f0cde7de740 1 -- 172.21.15.43:0/3954526800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29faa80 msgr2=0x5574a29fe060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:08.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.875+0000 7f0cde7de740 1 --2- 172.21.15.43:0/3954526800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29faa80 0x5574a29fe060 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc027580 tx=0x7f0ccc0127c0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.875+0000 7f0cde7de740 1 -- 172.21.15.43:0/3954526800 shutdown_connections 2024-04-21T08:41:08.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.875+0000 7f0cde7de740 1 --2- 172.21.15.43:0/3954526800 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5574a2a71100 0x5574a2a73520 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.875+0000 7f0cde7de740 1 --2- 172.21.15.43:0/3954526800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574a29faa80 0x5574a29fe060 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.875+0000 7f0cbd7fa640 0 client.16653 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:08.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.875+0000 7f0cde7de740 1 --2- 172.21.15.43:0/3954526800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574a29f1ea0 0x5574a2a030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.875+0000 7f0cde7de740 1 --2- 172.21.15.43:0/3954526800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574a29f1580 0x5574a2a02b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.875+0000 7f0cde7de740 1 -- 172.21.15.43:0/3954526800 >> 172.21.15.43:0/3954526800 conn(0x5574a29ebcb0 msgr2=0x5574a29ead70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:08.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.875+0000 7f0cde7de740 1 -- 172.21.15.43:0/3954526800 shutdown_connections 2024-04-21T08:41:08.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.875+0000 7f0cde7de740 1 -- 172.21.15.43:0/3954526800 wait complete. 2024-04-21T08:41:08.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.875+0000 7f0cde7de740 20 client.16653 trim_cache size 0 max 16384 2024-04-21T08:41:08.884 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:41:08.884 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:41:08.884 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:41:08.884 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:41:08.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.876+0000 7f0cdbb15640 1 -- 172.21.15.43:0/2681905176 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0cc00a3da0 msgr2=0x7f0cc00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:08.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.876+0000 7f0cdbb15640 1 --2- 172.21.15.43:0/2681905176 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0cc00a3da0 0x7f0cc00a6260 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc01b000 tx=0x7f0ccc01d000 comp rx=0 tx=0).stop 2024-04-21T08:41:08.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.876+0000 7f0cdbb15640 1 -- 172.21.15.43:0/2681905176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd406aed0 msgr2=0x7f0cd41a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:08.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.876+0000 7f0cdbb15640 1 --2- 172.21.15.43:0/2681905176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd406aed0 0x7f0cd41a0b50 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400c890 tx=0x7f0cc400cd50 comp rx=0 tx=0).stop 2024-04-21T08:41:08.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.876+0000 7f0cdbb15640 1 -- 172.21.15.43:0/2681905176 shutdown_connections 2024-04-21T08:41:08.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.876+0000 7f0cdbb15640 1 --2- 172.21.15.43:0/2681905176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cd410ec50 0x7f0cd419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.876+0000 7f0cdbb15640 1 --2- 172.21.15.43:0/2681905176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd406aed0 0x7f0cd41a0b50 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.876+0000 7f0cdbb15640 1 --2- 172.21.15.43:0/2681905176 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0cc00a3da0 0x7f0cc00a6260 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.876+0000 7f0cdbb15640 1 --2- 172.21.15.43:0/2681905176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cd406a5b0 0x7f0cd41a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.876+0000 7f0cdbb15640 1 -- 172.21.15.43:0/2681905176 >> 172.21.15.43:0/2681905176 conn(0x7f0cd4077c40 msgr2=0x7f0cd4110170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:08.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.876+0000 7f0cdbb15640 1 -- 172.21.15.43:0/2681905176 shutdown_connections 2024-04-21T08:41:08.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.876+0000 7f0cdbb15640 1 -- 172.21.15.43:0/2681905176 wait complete. 2024-04-21T08:41:08.887 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:41:08.888 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:41:08.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.990+0000 7f4e1db3d640 1 -- 172.21.15.43:0/2120454689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e18102b50 msgr2=0x7f4e18106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:08.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.990+0000 7f4e1db3d640 1 --2- 172.21.15.43:0/2120454689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e18102b50 0x7f4e18106b90 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c009900 tx=0x7f4e0c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:08.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.990+0000 7f4e1db3d640 1 -- 172.21.15.43:0/2120454689 shutdown_connections 2024-04-21T08:41:08.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.990+0000 7f4e1db3d640 1 --2- 172.21.15.43:0/2120454689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e18107270 0x7f4e1810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.990+0000 7f4e1db3d640 1 --2- 172.21.15.43:0/2120454689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e18102b50 0x7f4e18106b90 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.990+0000 7f4e1db3d640 1 --2- 172.21.15.43:0/2120454689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e181021a0 0x7f4e18102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.990+0000 7f4e1db3d640 1 -- 172.21.15.43:0/2120454689 >> 172.21.15.43:0/2120454689 conn(0x7f4e180793a0 msgr2=0x7f4e180797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:08.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.990+0000 7f4e1db3d640 1 -- 172.21.15.43:0/2120454689 shutdown_connections 2024-04-21T08:41:08.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.990+0000 7f4e1db3d640 1 -- 172.21.15.43:0/2120454689 wait complete. 2024-04-21T08:41:08.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.991+0000 7f4e1db3d640 1 Processor -- start 2024-04-21T08:41:08.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.991+0000 7f4e1db3d640 1 -- start start 2024-04-21T08:41:08.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.991+0000 7f4e1db3d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e181021a0 0x7f4e18074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:08.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.991+0000 7f4e1db3d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e18107270 0x7f4e18074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:08.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.991+0000 7f4e1db3d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e18074cd0 0x7f4e181a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:08.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.991+0000 7f4e1db3d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4e181121b0 con 0x7f4e18074cd0 2024-04-21T08:41:08.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.991+0000 7f4e1db3d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4e18112030 con 0x7f4e18107270 2024-04-21T08:41:08.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.991+0000 7f4e1db3d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4e18112330 con 0x7f4e181021a0 2024-04-21T08:41:08.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.991+0000 7f4e17fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e18074cd0 0x7f4e181a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:08.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.991+0000 7f4e17fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e18074cd0 0x7f4e181a6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55692/0 (socket says 172.21.15.43:55692) 2024-04-21T08:41:08.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.991+0000 7f4e17fff640 1 -- 172.21.15.43:0/1988056097 learned_addr learned my addr 172.21.15.43:0/1988056097 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:08.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.991+0000 7f4e16ffd640 1 --2- 172.21.15.43:0/1988056097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e18107270 0x7f4e18074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:08.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.991+0000 7f4e177fe640 1 --2- 172.21.15.43:0/1988056097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e181021a0 0x7f4e18074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:08.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.991+0000 7f4e17fff640 1 -- 172.21.15.43:0/1988056097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e181021a0 msgr2=0x7f4e18074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:08.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.991+0000 7f4e17fff640 1 --2- 172.21.15.43:0/1988056097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e181021a0 0x7f4e18074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.991+0000 7f4e17fff640 1 -- 172.21.15.43:0/1988056097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e18107270 msgr2=0x7f4e18074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:08.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.991+0000 7f4e17fff640 1 --2- 172.21.15.43:0/1988056097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e18107270 0x7f4e18074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:08.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.991+0000 7f4e17fff640 1 -- 172.21.15.43:0/1988056097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e181a7040 con 0x7f4e18074cd0 2024-04-21T08:41:08.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.992+0000 7f4e17fff640 1 --2- 172.21.15.43:0/1988056097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e18074cd0 0x7f4e181a6940 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000cad0 tx=0x7f4e1000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:08.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.992+0000 7f4e14ff9640 1 -- 172.21.15.43:0/1988056097 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1000ae20 con 0x7f4e18074cd0 2024-04-21T08:41:08.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.992+0000 7f4e14ff9640 1 -- 172.21.15.43:0/1988056097 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e10005020 con 0x7f4e18074cd0 2024-04-21T08:41:08.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.992+0000 7f4e1db3d640 1 -- 172.21.15.43:0/1988056097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e181a7330 con 0x7f4e18074cd0 2024-04-21T08:41:08.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.992+0000 7f4e14ff9640 1 -- 172.21.15.43:0/1988056097 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1000f6b0 con 0x7f4e18074cd0 2024-04-21T08:41:08.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.992+0000 7f4e1db3d640 1 -- 172.21.15.43:0/1988056097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e181a7850 con 0x7f4e18074cd0 2024-04-21T08:41:08.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.993+0000 7f4e1db3d640 1 -- 172.21.15.43:0/1988056097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e18069230 con 0x7f4e18074cd0 2024-04-21T08:41:08.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.994+0000 7f4e14ff9640 1 -- 172.21.15.43:0/1988056097 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4e100051d0 con 0x7f4e18074cd0 2024-04-21T08:41:08.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.995+0000 7f4e14ff9640 1 --2- 172.21.15.43:0/1988056097 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4e000a3d50 0x7f4e000a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:08.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.995+0000 7f4e177fe640 1 --2- 172.21.15.43:0/1988056097 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4e000a3d50 0x7f4e000a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:08.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.995+0000 7f4e14ff9640 1 -- 172.21.15.43:0/1988056097 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f4e100d0f30 con 0x7f4e18074cd0 2024-04-21T08:41:08.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.995+0000 7f4e177fe640 1 --2- 172.21.15.43:0/1988056097 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4e000a3d50 0x7f4e000a6210 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4e08005fd0 tx=0x7f4e0800b590 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:08.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:08.999+0000 7f4e14ff9640 1 -- 172.21.15.43:0/1988056097 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e100ffff0 con 0x7f4e18074cd0 2024-04-21T08:41:09.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.226+0000 7f4e1db3d640 1 -- 172.21.15.43:0/1988056097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e18102b50 con 0x7f4e18074cd0 2024-04-21T08:41:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.227+0000 7f4e14ff9640 1 -- 172.21.15.43:0/1988056097 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7f4e10014030 con 0x7f4e18074cd0 2024-04-21T08:41:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:41:09.227 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:41:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:41:09.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.228+0000 7f4e1db3d640 1 -- 172.21.15.43:0/1988056097 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4e000a3d50 msgr2=0x7f4e000a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.228+0000 7f4e1db3d640 1 --2- 172.21.15.43:0/1988056097 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4e000a3d50 0x7f4e000a6210 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4e08005fd0 tx=0x7f4e0800b590 comp rx=0 tx=0).stop 2024-04-21T08:41:09.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.228+0000 7f4e1db3d640 1 -- 172.21.15.43:0/1988056097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e18074cd0 msgr2=0x7f4e181a6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.228+0000 7f4e1db3d640 1 --2- 172.21.15.43:0/1988056097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e18074cd0 0x7f4e181a6940 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000cad0 tx=0x7f4e1000cf90 comp rx=0 tx=0).stop 2024-04-21T08:41:09.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.229+0000 7f4e1db3d640 1 -- 172.21.15.43:0/1988056097 shutdown_connections 2024-04-21T08:41:09.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.229+0000 7f4e1db3d640 1 --2- 172.21.15.43:0/1988056097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e18074cd0 0x7f4e181a6940 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.229+0000 7f4e1db3d640 1 --2- 172.21.15.43:0/1988056097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e18107270 0x7f4e18074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.229+0000 7f4e1db3d640 1 --2- 172.21.15.43:0/1988056097 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4e000a3d50 0x7f4e000a6210 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.229+0000 7f4e1db3d640 1 --2- 172.21.15.43:0/1988056097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e181021a0 0x7f4e18074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.229+0000 7f4e1db3d640 1 -- 172.21.15.43:0/1988056097 >> 172.21.15.43:0/1988056097 conn(0x7f4e180793a0 msgr2=0x7f4e180fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.229+0000 7f4e1db3d640 1 -- 172.21.15.43:0/1988056097 shutdown_connections 2024-04-21T08:41:09.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.229+0000 7f4e1db3d640 1 -- 172.21.15.43:0/1988056097 wait complete. 2024-04-21T08:41:09.240 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:41:09.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.344+0000 7fd91bf32640 1 -- 172.21.15.43:0/3923855174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd914103b00 msgr2=0x7fd91410bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.344+0000 7fd91bf32640 1 --2- 172.21.15.43:0/3923855174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd914103b00 0x7fd91410bfe0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd904009900 tx=0x7fd90402bd60 comp rx=0 tx=0).stop 2024-04-21T08:41:09.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.344+0000 7fd91bf32640 1 -- 172.21.15.43:0/3923855174 shutdown_connections 2024-04-21T08:41:09.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.344+0000 7fd91bf32640 1 --2- 172.21.15.43:0/3923855174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd91410c520 0x7fd91410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.344+0000 7fd91bf32640 1 --2- 172.21.15.43:0/3923855174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd914103b00 0x7fd91410bfe0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.344+0000 7fd91bf32640 1 --2- 172.21.15.43:0/3923855174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9141031e0 0x7fd9141035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.344+0000 7fd91bf32640 1 -- 172.21.15.43:0/3923855174 >> 172.21.15.43:0/3923855174 conn(0x7fd9140771d0 msgr2=0x7fd9140775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.344+0000 7fd91bf32640 1 -- 172.21.15.43:0/3923855174 shutdown_connections 2024-04-21T08:41:09.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.344+0000 7fd91bf32640 1 -- 172.21.15.43:0/3923855174 wait complete. 2024-04-21T08:41:09.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.345+0000 7fd91bf32640 1 Processor -- start 2024-04-21T08:41:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.345+0000 7fd91bf32640 1 -- start start 2024-04-21T08:41:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.345+0000 7fd91bf32640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9141031e0 0x7fd914100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.345+0000 7fd91bf32640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd914103b00 0x7fd914101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.345+0000 7fd91bf32640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91410c520 0x7fd9141018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.345+0000 7fd91bf32640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9141121b0 con 0x7fd91410c520 2024-04-21T08:41:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.345+0000 7fd91bf32640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd914112030 con 0x7fd9141031e0 2024-04-21T08:41:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.345+0000 7fd91bf32640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd914112330 con 0x7fd914103b00 2024-04-21T08:41:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.345+0000 7fd91a4cf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91410c520 0x7fd9141018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.345+0000 7fd91a4cf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91410c520 0x7fd9141018b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55704/0 (socket says 172.21.15.43:55704) 2024-04-21T08:41:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.345+0000 7fd91a4cf640 1 -- 172.21.15.43:0/1707869418 learned_addr learned my addr 172.21.15.43:0/1707869418 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.345+0000 7fd919cce640 1 --2- 172.21.15.43:0/1707869418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9141031e0 0x7fd914100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.346+0000 7fd9194cd640 1 --2- 172.21.15.43:0/1707869418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd914103b00 0x7fd914101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.346+0000 7fd91a4cf640 1 -- 172.21.15.43:0/1707869418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd914103b00 msgr2=0x7fd914101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.346+0000 7fd91a4cf640 1 --2- 172.21.15.43:0/1707869418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd914103b00 0x7fd914101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.346+0000 7fd91a4cf640 1 -- 172.21.15.43:0/1707869418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9141031e0 msgr2=0x7fd914100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.346+0000 7fd91a4cf640 1 --2- 172.21.15.43:0/1707869418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9141031e0 0x7fd914100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.346+0000 7fd91a4cf640 1 -- 172.21.15.43:0/1707869418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9141a6c70 con 0x7fd91410c520 2024-04-21T08:41:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.346+0000 7fd919cce640 1 --2- 172.21.15.43:0/1707869418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9141031e0 0x7fd914100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.346+0000 7fd91a4cf640 1 --2- 172.21.15.43:0/1707869418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91410c520 0x7fd9141018b0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00eaa0 tx=0x7fd90c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.346+0000 7fd90affd640 1 -- 172.21.15.43:0/1707869418 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c00cdb0 con 0x7fd91410c520 2024-04-21T08:41:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.346+0000 7fd90affd640 1 -- 172.21.15.43:0/1707869418 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd90c005020 con 0x7fd91410c520 2024-04-21T08:41:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.346+0000 7fd91bf32640 1 -- 172.21.15.43:0/1707869418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9141a6f60 con 0x7fd91410c520 2024-04-21T08:41:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.346+0000 7fd90affd640 1 -- 172.21.15.43:0/1707869418 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c0116d0 con 0x7fd91410c520 2024-04-21T08:41:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.346+0000 7fd91bf32640 1 -- 172.21.15.43:0/1707869418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9141a74d0 con 0x7fd91410c520 2024-04-21T08:41:09.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.348+0000 7fd90affd640 1 -- 172.21.15.43:0/1707869418 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd90c0051d0 con 0x7fd91410c520 2024-04-21T08:41:09.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.348+0000 7fd91ebff740 1 Processor -- start 2024-04-21T08:41:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.349+0000 7fd91ebff740 1 -- start start 2024-04-21T08:41:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.349+0000 7fd91ebff740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a71338b4d0 0x55a71338b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.349+0000 7fd91ebff740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71338be80 0x55a713394430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.349+0000 7fd91ebff740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a713394a00 0x55a713396e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.349+0000 7fd91ebff740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a7132f01f0 con 0x55a71338b4d0 2024-04-21T08:41:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.349+0000 7fd91ebff740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a71328d4b0 con 0x55a71338be80 2024-04-21T08:41:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.349+0000 7fd91ebff740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a7132ec1b0 con 0x55a713394a00 2024-04-21T08:41:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.349+0000 7fd9194cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a71338b4d0 0x55a71338b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.349+0000 7fd9194cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a71338b4d0 0x55a71338b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55708/0 (socket says 172.21.15.43:55708) 2024-04-21T08:41:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.349+0000 7fd9194cd640 1 -- 172.21.15.43:0/3620322423 learned_addr learned my addr 172.21.15.43:0/3620322423 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.349+0000 7fd919cce640 1 --2- 172.21.15.43:0/3620322423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71338be80 0x55a713394430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.349+0000 7fd90affd640 1 --2- 172.21.15.43:0/1707869418 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd9000a3da0 0x7fd9000a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.349+0000 7fd9194cd640 1 --2- 172.21.15.43:0/3620322423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a713394a00 0x55a713396e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.349+0000 7fd9194cd640 1 -- 172.21.15.43:0/3620322423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a713394a00 msgr2=0x55a713396e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.349+0000 7fd9194cd640 1 --2- 172.21.15.43:0/3620322423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a713394a00 0x55a713396e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.349+0000 7fd91a4cf640 1 --2- 172.21.15.43:0/1707869418 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd9000a3da0 0x7fd9000a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.349+0000 7fd9194cd640 1 -- 172.21.15.43:0/3620322423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71338be80 msgr2=0x55a713394430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.349+0000 7fd9194cd640 1 --2- 172.21.15.43:0/3620322423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71338be80 0x55a713394430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.349+0000 7fd9194cd640 1 -- 172.21.15.43:0/3620322423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a71327f1b0 con 0x55a71338b4d0 2024-04-21T08:41:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.349+0000 7fd90affd640 1 -- 172.21.15.43:0/1707869418 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd90c0d0d30 con 0x7fd91410c520 2024-04-21T08:41:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.349+0000 7fd9194cd640 1 --2- 172.21.15.43:0/3620322423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a71338b4d0 0x55a71338b8b0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fd904005ab0 tx=0x7fd904005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd591273d05fde2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.350+0000 7fd8fffff640 1 -- 172.21.15.43:0/3620322423 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904047070 con 0x55a71338b4d0 2024-04-21T08:41:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.350+0000 7fd91a4cf640 1 --2- 172.21.15.43:0/1707869418 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd9000a3da0 0x7fd9000a6260 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd90c0ce6b0 tx=0x7fd90c08e040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.350+0000 7fd8fffff640 1 -- 172.21.15.43:0/3620322423 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd90402cdc0 con 0x55a71338b4d0 2024-04-21T08:41:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.350+0000 7fd8fffff640 1 -- 172.21.15.43:0/3620322423 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9040048c0 con 0x55a71338b4d0 2024-04-21T08:41:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.350+0000 7fd91ebff740 1 -- 172.21.15.43:0/3620322423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a71338b4d0 msgr2=0x55a71338b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.350+0000 7fd91ebff740 1 --2- 172.21.15.43:0/3620322423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a71338b4d0 0x55a71338b8b0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fd904005ab0 tx=0x7fd904005a40 comp rx=0 tx=0).stop 2024-04-21T08:41:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.350+0000 7fd91ebff740 1 -- 172.21.15.43:0/3620322423 shutdown_connections 2024-04-21T08:41:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.350+0000 7fd91ebff740 1 --2- 172.21.15.43:0/3620322423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a713394a00 0x55a713396e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.350+0000 7fd91ebff740 1 --2- 172.21.15.43:0/3620322423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71338be80 0x55a713394430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.350+0000 7fd91ebff740 1 --2- 172.21.15.43:0/3620322423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a71338b4d0 0x55a71338b8b0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.350+0000 7fd91ebff740 1 -- 172.21.15.43:0/3620322423 >> 172.21.15.43:0/3620322423 conn(0x55a713385ba0 msgr2=0x55a713385f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.351+0000 7fd91ebff740 1 -- 172.21.15.43:0/3620322423 shutdown_connections 2024-04-21T08:41:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.351+0000 7fd91ebff740 1 -- 172.21.15.43:0/3620322423 wait complete. 2024-04-21T08:41:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.351+0000 7fd91ebff740 1 -- start start 2024-04-21T08:41:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.351+0000 7fd91ebff740 1 Processor -- start 2024-04-21T08:41:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.351+0000 7fd91ebff740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71338b4d0 0x55a7133943a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.351+0000 7fd91ebff740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a71338be80 0x55a713391730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.351+0000 7fd91ebff740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a713394a00 0x55a713391e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.351+0000 7fd91ebff740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a713397910 con 0x55a713394a00 2024-04-21T08:41:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.351+0000 7fd91ebff740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a7132a9500 con 0x55a71338b4d0 2024-04-21T08:41:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.351+0000 7fd91ebff740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a713397a90 con 0x55a71338be80 2024-04-21T08:41:09.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.351+0000 7fd91ebff740 10 client.0 _ll_register_callbacks cb 0x55a713277b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:09.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.352+0000 7fd9194cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a713394a00 0x55a713391e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.352+0000 7fd9194cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a713394a00 0x55a713391e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55714/0 (socket says 172.21.15.43:55714) 2024-04-21T08:41:09.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.352+0000 7fd9194cd640 1 -- 172.21.15.43:0/3369745077 learned_addr learned my addr 172.21.15.43:0/3369745077 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.352+0000 7fd9194cd640 1 --2- 172.21.15.43:0/3369745077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71338b4d0 0x55a7133943a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.352+0000 7fd919cce640 1 --2- 172.21.15.43:0/3369745077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a71338be80 0x55a713391730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.352+0000 7fd9194cd640 1 -- 172.21.15.43:0/3369745077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a71338be80 msgr2=0x55a713391730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.352+0000 7fd9194cd640 1 --2- 172.21.15.43:0/3369745077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a71338be80 0x55a713391730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.352+0000 7fd9194cd640 1 -- 172.21.15.43:0/3369745077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71338b4d0 msgr2=0x55a7133943a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.352+0000 7fd9194cd640 1 --2- 172.21.15.43:0/3369745077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71338b4d0 0x55a7133943a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.352+0000 7fd9194cd640 1 -- 172.21.15.43:0/3369745077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7130023f0 con 0x55a713394a00 2024-04-21T08:41:09.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.352+0000 7fd9194cd640 1 --2- 172.21.15.43:0/3369745077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a713394a00 0x55a713391e70 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fd90402cd50 tx=0x7fd9040058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56266eb091c1af7c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.352+0000 7fd8fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:09.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.352+0000 7fd8fd7fa640 1 -- 172.21.15.43:0/3369745077 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904047040 con 0x55a713394a00 2024-04-21T08:41:09.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.352+0000 7fd8fd7fa640 1 -- 172.21.15.43:0/3369745077 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd9040046c0 con 0x55a713394a00 2024-04-21T08:41:09.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.352+0000 7fd91ebff740 1 -- 172.21.15.43:0/3369745077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd90402cdc0 con 0x55a713394a00 2024-04-21T08:41:09.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.352+0000 7fd8fd7fa640 1 -- 172.21.15.43:0/3369745077 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd904058050 con 0x55a713394a00 2024-04-21T08:41:09.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.352+0000 7fd91ebff740 10 client.16671 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:09.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.352+0000 7fd91ebff740 1 -- 172.21.15.43:0/3369745077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a7130cc7d0 con 0x55a713394a00 2024-04-21T08:41:09.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.353+0000 7fd8fd7fa640 1 -- 172.21.15.43:0/3369745077 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd90402cdc0 con 0x55a713394a00 2024-04-21T08:41:09.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.353+0000 7fd8fd7fa640 1 -- 172.21.15.43:0/3369745077 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd90404d070 con 0x55a713394a00 2024-04-21T08:41:09.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.353+0000 7fd91ebff740 10 client.16671 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:09.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.353+0000 7fd91ebff740 10 client.16671 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:41:09.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.353+0000 7fd91ebff740 20 client.16671 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:09.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.353+0000 7fd91ebff740 1 --2- 172.21.15.43:0/3369745077 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55a7133a4ae0 0x55a7133a6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.353+0000 7fd91ebff740 4 client.16671 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:09.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.353+0000 7fd91ebff740 1 -- 172.21.15.43:0/3369745077 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a71328f930 con 0x55a7133a4ae0 2024-04-21T08:41:09.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.353+0000 7fd919cce640 1 --2- 172.21.15.43:0/3369745077 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55a7133a4ae0 0x55a7133a6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.353+0000 7fd919cce640 1 --2- 172.21.15.43:0/3369745077 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55a7133a4ae0 0x55a7133a6ed0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e943905f006c19de server_cookie=1fab9901f6e6a2b7 in_seq=0 out_seq=0 2024-04-21T08:41:09.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.353+0000 7fd8fd7fa640 10 client.16671 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:41:09.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.354+0000 7fd8fd7fa640 1 -- 172.21.15.43:0/3369745077 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a71328f930 con 0x55a7133a4ae0 2024-04-21T08:41:09.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.354+0000 7fd8fd7fa640 10 client.16671 handle_command_reply: tid=0 2024-04-21T08:41:09.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.354+0000 7fd91ebff740 1 client.16671 shutdown 2024-04-21T08:41:09.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.354+0000 7fd91ebff740 1 -- 172.21.15.43:0/3369745077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a713394a00 msgr2=0x55a713391e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.354+0000 7fd91ebff740 1 --2- 172.21.15.43:0/3369745077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a713394a00 0x55a713391e70 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fd90402cd50 tx=0x7fd9040058b0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.354+0000 7fd91ebff740 1 -- 172.21.15.43:0/3369745077 shutdown_connections 2024-04-21T08:41:09.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.354+0000 7fd91ebff740 1 --2- 172.21.15.43:0/3369745077 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55a7133a4ae0 0x55a7133a6ed0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.354+0000 7fd91ebff740 1 --2- 172.21.15.43:0/3369745077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a713394a00 0x55a713391e70 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.354+0000 7fd91ebff740 1 --2- 172.21.15.43:0/3369745077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a71338be80 0x55a713391730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.354+0000 7fd8fd7fa640 0 client.16671 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:41:09.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.354+0000 7fd91ebff740 1 --2- 172.21.15.43:0/3369745077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71338b4d0 0x55a7133943a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.354+0000 7fd91ebff740 1 -- 172.21.15.43:0/3369745077 >> 172.21.15.43:0/3369745077 conn(0x55a713385ba0 msgr2=0x55a7133878f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.354+0000 7fd91ebff740 1 -- 172.21.15.43:0/3369745077 shutdown_connections 2024-04-21T08:41:09.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.355+0000 7fd91ebff740 1 -- 172.21.15.43:0/3369745077 wait complete. 2024-04-21T08:41:09.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.355+0000 7fd91ebff740 20 client.16671 trim_cache size 0 max 16384 2024-04-21T08:41:09.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.368+0000 7fd91ebff740 1 Processor -- start 2024-04-21T08:41:09.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.368+0000 7fd91ebff740 1 -- start start 2024-04-21T08:41:09.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.368+0000 7fd91ebff740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71338b4d0 0x55a7133a0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.368+0000 7fd91ebff740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a71338be80 0x55a7133a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.368+0000 7fd91ebff740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a713394a00 0x55a7133aa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.368+0000 7fd91ebff740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9141b16c0 con 0x55a713394a00 2024-04-21T08:41:09.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.368+0000 7fd91ebff740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9141b3200 con 0x55a71338b4d0 2024-04-21T08:41:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.368+0000 7fd91ebff740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd9141b1d90 con 0x55a71338be80 2024-04-21T08:41:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.368+0000 7fd9194cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a713394a00 0x55a7133aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.368+0000 7fd9194cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a713394a00 0x55a7133aa430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55724/0 (socket says 172.21.15.43:55724) 2024-04-21T08:41:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.368+0000 7fd9194cd640 1 -- 172.21.15.43:0/1238817770 learned_addr learned my addr 172.21.15.43:0/1238817770 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.368+0000 7fd9194cd640 1 --2- 172.21.15.43:0/1238817770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71338b4d0 0x55a7133a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.368+0000 7fd9194cd640 1 -- 172.21.15.43:0/1238817770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a71338be80 msgr2=0x55a7133a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.368+0000 7fd9194cd640 1 --2- 172.21.15.43:0/1238817770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a71338be80 0x55a7133a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.368+0000 7fd9194cd640 1 -- 172.21.15.43:0/1238817770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71338b4d0 msgr2=0x55a7133a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.368+0000 7fd9194cd640 1 --2- 172.21.15.43:0/1238817770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71338b4d0 0x55a7133a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.368+0000 7fd9194cd640 1 -- 172.21.15.43:0/1238817770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7133856d0 con 0x55a713394a00 2024-04-21T08:41:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.369+0000 7fd9194cd640 1 --2- 172.21.15.43:0/1238817770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a713394a00 0x55a7133aa430 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd904001eb0 tx=0x7fd90402bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395681b6d762e203 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.369+0000 7fd8ff7fe640 1 -- 172.21.15.43:0/1238817770 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904047070 con 0x55a713394a00 2024-04-21T08:41:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.369+0000 7fd8ff7fe640 1 -- 172.21.15.43:0/1238817770 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd904009520 con 0x55a713394a00 2024-04-21T08:41:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.369+0000 7fd8ff7fe640 1 -- 172.21.15.43:0/1238817770 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904052080 con 0x55a713394a00 2024-04-21T08:41:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.369+0000 7fd91ebff740 1 -- 172.21.15.43:0/1238817770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a713394a00 msgr2=0x55a7133aa430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.369+0000 7fd91ebff740 1 --2- 172.21.15.43:0/1238817770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a713394a00 0x55a7133aa430 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd904001eb0 tx=0x7fd90402bd60 comp rx=0 tx=0).stop 2024-04-21T08:41:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.369+0000 7fd91ebff740 1 -- 172.21.15.43:0/1238817770 shutdown_connections 2024-04-21T08:41:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.369+0000 7fd91ebff740 1 --2- 172.21.15.43:0/1238817770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a713394a00 0x55a7133aa430 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.369+0000 7fd91ebff740 1 --2- 172.21.15.43:0/1238817770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a71338be80 0x55a7133a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.369+0000 7fd91ebff740 1 --2- 172.21.15.43:0/1238817770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71338b4d0 0x55a7133a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.369+0000 7fd91ebff740 1 -- 172.21.15.43:0/1238817770 >> 172.21.15.43:0/1238817770 conn(0x55a713385ba0 msgr2=0x55a71339fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.369+0000 7fd91ebff740 1 -- 172.21.15.43:0/1238817770 shutdown_connections 2024-04-21T08:41:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.369+0000 7fd91ebff740 1 -- 172.21.15.43:0/1238817770 wait complete. 2024-04-21T08:41:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.370+0000 7fd91ebff740 1 -- start start 2024-04-21T08:41:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.370+0000 7fd91ebff740 1 Processor -- start 2024-04-21T08:41:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.370+0000 7fd91ebff740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a71338b4d0 0x55a7133a7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.370+0000 7fd91ebff740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a71338be80 0x55a7133a7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.370+0000 7fd91ebff740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a713394a00 0x55a7133a82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.370+0000 7fd91ebff740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9141b2b30 con 0x55a71338b4d0 2024-04-21T08:41:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.370+0000 7fd91ebff740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9141b2460 con 0x55a713394a00 2024-04-21T08:41:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.370+0000 7fd91ebff740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd9141af6a0 con 0x55a71338be80 2024-04-21T08:41:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.370+0000 7fd91ebff740 10 client.0 _ll_register_callbacks cb 0x7fd90402c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.370+0000 7fd9194cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a71338b4d0 0x55a7133a7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.370+0000 7fd9194cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a71338b4d0 0x55a7133a7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55732/0 (socket says 172.21.15.43:55732) 2024-04-21T08:41:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.370+0000 7fd9194cd640 1 -- 172.21.15.43:0/947758996 learned_addr learned my addr 172.21.15.43:0/947758996 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.371+0000 7fd9194cd640 1 -- 172.21.15.43:0/947758996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a71338be80 msgr2=0x55a7133a7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.371+0000 7fd9194cd640 1 --2- 172.21.15.43:0/947758996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a71338be80 0x55a7133a7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.371+0000 7fd9194cd640 1 -- 172.21.15.43:0/947758996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a713394a00 msgr2=0x55a7133a82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.371+0000 7fd9194cd640 1 --2- 172.21.15.43:0/947758996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a713394a00 0x55a7133a82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.371+0000 7fd9194cd640 1 -- 172.21.15.43:0/947758996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7134124d0 con 0x55a71338b4d0 2024-04-21T08:41:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.371+0000 7fd9194cd640 1 --2- 172.21.15.43:0/947758996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a71338b4d0 0x55a7133a7830 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd9040043a0 tx=0x7fd90402c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cea062e3cd004aa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.371+0000 7fd8fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.371+0000 7fd8fd7fa640 1 -- 172.21.15.43:0/947758996 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904061060 con 0x55a71338b4d0 2024-04-21T08:41:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.371+0000 7fd8fd7fa640 1 -- 172.21.15.43:0/947758996 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd904003d50 con 0x55a71338b4d0 2024-04-21T08:41:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.371+0000 7fd91ebff740 1 -- 172.21.15.43:0/947758996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd904009520 con 0x55a71338b4d0 2024-04-21T08:41:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.371+0000 7fd8fd7fa640 1 -- 172.21.15.43:0/947758996 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd904068050 con 0x55a71338b4d0 2024-04-21T08:41:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.371+0000 7fd91ebff740 10 client.16677 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.371+0000 7fd91ebff740 1 -- 172.21.15.43:0/947758996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a7134136b0 con 0x55a71338b4d0 2024-04-21T08:41:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.371+0000 7fd8fd7fa640 1 -- 172.21.15.43:0/947758996 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd904009520 con 0x55a71338b4d0 2024-04-21T08:41:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.371+0000 7fd8fd7fa640 1 -- 172.21.15.43:0/947758996 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd90404d040 con 0x55a71338b4d0 2024-04-21T08:41:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.372+0000 7fd91ebff740 10 client.16677 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.372+0000 7fd91ebff740 10 client.16677 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:41:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.372+0000 7fd91ebff740 20 client.16677 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.372+0000 7fd91ebff740 1 --2- 172.21.15.43:0/947758996 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55a713413c40 0x55a713416060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.372+0000 7fd91ebff740 4 client.16677 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:41:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.372+0000 7fd91ebff740 1 -- 172.21.15.43:0/947758996 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a7132fdc10 con 0x55a713413c40 2024-04-21T08:41:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.372+0000 7fd919cce640 1 --2- 172.21.15.43:0/947758996 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55a713413c40 0x55a713416060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.372+0000 7fd919cce640 1 --2- 172.21.15.43:0/947758996 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55a713413c40 0x55a713416060 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3761d1129043a2f4 server_cookie=bfa05f8e75472ea in_seq=0 out_seq=0 2024-04-21T08:41:09.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.372+0000 7fd8fd7fa640 10 client.16677 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:41:09.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.373+0000 7fd8fd7fa640 1 -- 172.21.15.43:0/947758996 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a7132fdc10 con 0x55a713413c40 2024-04-21T08:41:09.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.373+0000 7fd8fd7fa640 10 client.16677 handle_command_reply: tid=0 2024-04-21T08:41:09.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.373+0000 7fd91ebff740 1 client.16677 shutdown 2024-04-21T08:41:09.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.373+0000 7fd91ebff740 1 -- 172.21.15.43:0/947758996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a71338b4d0 msgr2=0x55a7133a7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.373+0000 7fd91ebff740 1 --2- 172.21.15.43:0/947758996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a71338b4d0 0x55a7133a7830 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd9040043a0 tx=0x7fd90402c3e0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.373+0000 7fd91ebff740 1 -- 172.21.15.43:0/947758996 shutdown_connections 2024-04-21T08:41:09.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.373+0000 7fd91ebff740 1 --2- 172.21.15.43:0/947758996 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55a713413c40 0x55a713416060 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.373+0000 7fd91ebff740 1 --2- 172.21.15.43:0/947758996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a713394a00 0x55a7133a82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.373+0000 7fd91ebff740 1 --2- 172.21.15.43:0/947758996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a71338be80 0x55a7133a7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.373+0000 7fd8fd7fa640 0 client.16677 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:41:09.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.373+0000 7fd91ebff740 1 --2- 172.21.15.43:0/947758996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a71338b4d0 0x55a7133a7830 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.373+0000 7fd91ebff740 1 -- 172.21.15.43:0/947758996 >> 172.21.15.43:0/947758996 conn(0x55a713385ba0 msgr2=0x55a713384c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.373+0000 7fd91ebff740 1 -- 172.21.15.43:0/947758996 shutdown_connections 2024-04-21T08:41:09.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.373+0000 7fd91ebff740 1 -- 172.21.15.43:0/947758996 wait complete. 2024-04-21T08:41:09.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.373+0000 7fd91ebff740 20 client.16677 trim_cache size 0 max 16384 2024-04-21T08:41:09.379 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:41:09.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.373+0000 7fd91bf32640 1 -- 172.21.15.43:0/1707869418 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd9000a3da0 msgr2=0x7fd9000a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.373+0000 7fd91bf32640 1 --2- 172.21.15.43:0/1707869418 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd9000a3da0 0x7fd9000a6260 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd90c0ce6b0 tx=0x7fd90c08e040 comp rx=0 tx=0).stop 2024-04-21T08:41:09.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.373+0000 7fd91bf32640 1 -- 172.21.15.43:0/1707869418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91410c520 msgr2=0x7fd9141018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.373+0000 7fd91bf32640 1 --2- 172.21.15.43:0/1707869418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91410c520 0x7fd9141018b0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00eaa0 tx=0x7fd90c00ef60 comp rx=0 tx=0).stop 2024-04-21T08:41:09.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.374+0000 7fd91bf32640 1 -- 172.21.15.43:0/1707869418 shutdown_connections 2024-04-21T08:41:09.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.374+0000 7fd91bf32640 1 --2- 172.21.15.43:0/1707869418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91410c520 0x7fd9141018b0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.374+0000 7fd91bf32640 1 --2- 172.21.15.43:0/1707869418 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd9000a3da0 0x7fd9000a6260 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.374+0000 7fd91bf32640 1 --2- 172.21.15.43:0/1707869418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd914103b00 0x7fd914101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.374+0000 7fd91bf32640 1 --2- 172.21.15.43:0/1707869418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9141031e0 0x7fd914100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.374+0000 7fd91bf32640 1 -- 172.21.15.43:0/1707869418 >> 172.21.15.43:0/1707869418 conn(0x7fd9140771d0 msgr2=0x7fd9140fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.374+0000 7fd91bf32640 1 -- 172.21.15.43:0/1707869418 shutdown_connections 2024-04-21T08:41:09.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.374+0000 7fd91bf32640 1 -- 172.21.15.43:0/1707869418 wait complete. 2024-04-21T08:41:09.384 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:41:09.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.489+0000 7f5463196640 1 -- 172.21.15.43:0/3330974603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f545c101a50 msgr2=0x7f545c105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.489+0000 7f5463196640 1 --2- 172.21.15.43:0/3330974603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f545c101a50 0x7f545c105a90 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f5448009900 tx=0x7f544802bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:09.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.489+0000 7f5463196640 1 -- 172.21.15.43:0/3330974603 shutdown_connections 2024-04-21T08:41:09.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.489+0000 7f5463196640 1 --2- 172.21.15.43:0/3330974603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f545c106170 0x7f545c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.489+0000 7f5463196640 1 --2- 172.21.15.43:0/3330974603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f545c101a50 0x7f545c105a90 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.489+0000 7f5463196640 1 --2- 172.21.15.43:0/3330974603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f545c1010a0 0x7f545c101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.489+0000 7f5463196640 1 -- 172.21.15.43:0/3330974603 >> 172.21.15.43:0/3330974603 conn(0x7f545c077eb0 msgr2=0x7f545c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.489+0000 7f5463196640 1 -- 172.21.15.43:0/3330974603 shutdown_connections 2024-04-21T08:41:09.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.489+0000 7f5463196640 1 -- 172.21.15.43:0/3330974603 wait complete. 2024-04-21T08:41:09.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.490+0000 7f5463196640 1 Processor -- start 2024-04-21T08:41:09.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.490+0000 7f5463196640 1 -- start start 2024-04-21T08:41:09.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.490+0000 7f5463196640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f545c1010a0 0x7f545c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.490+0000 7f5463196640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f545c106170 0x7f545c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.490+0000 7f5463196640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f545c19d010 0x7f545c19d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.490+0000 7f5463196640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f545c1121f0 con 0x7f545c106170 2024-04-21T08:41:09.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.490+0000 7f5463196640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f545c112070 con 0x7f545c1010a0 2024-04-21T08:41:09.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.490+0000 7f5463196640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f545c112370 con 0x7f545c19d010 2024-04-21T08:41:09.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.490+0000 7f545bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f545c106170 0x7f545c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.490+0000 7f545bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f545c106170 0x7f545c198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55748/0 (socket says 172.21.15.43:55748) 2024-04-21T08:41:09.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.490+0000 7f545bfff640 1 -- 172.21.15.43:0/216609391 learned_addr learned my addr 172.21.15.43:0/216609391 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.490+0000 7f5460f32640 1 --2- 172.21.15.43:0/216609391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f545c1010a0 0x7f545c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.490+0000 7f5461733640 1 --2- 172.21.15.43:0/216609391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f545c19d010 0x7f545c19d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.490+0000 7f545bfff640 1 -- 172.21.15.43:0/216609391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f545c19d010 msgr2=0x7f545c19d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.490+0000 7f545bfff640 1 --2- 172.21.15.43:0/216609391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f545c19d010 0x7f545c19d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.490+0000 7f545bfff640 1 -- 172.21.15.43:0/216609391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f545c1010a0 msgr2=0x7f545c198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.490+0000 7f545bfff640 1 --2- 172.21.15.43:0/216609391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f545c1010a0 0x7f545c198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.490+0000 7f545bfff640 1 -- 172.21.15.43:0/216609391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545c19db00 con 0x7f545c106170 2024-04-21T08:41:09.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.490+0000 7f545bfff640 1 --2- 172.21.15.43:0/216609391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f545c106170 0x7f545c198c90 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f5448036040 tx=0x7f54480045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.491+0000 7f5459ffb640 1 -- 172.21.15.43:0/216609391 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448043020 con 0x7f545c106170 2024-04-21T08:41:09.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.491+0000 7f5463196640 1 -- 172.21.15.43:0/216609391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f545c19dd90 con 0x7f545c106170 2024-04-21T08:41:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.491+0000 7f5459ffb640 1 -- 172.21.15.43:0/216609391 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f544802ca70 con 0x7f545c106170 2024-04-21T08:41:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.491+0000 7f5459ffb640 1 -- 172.21.15.43:0/216609391 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448003ac0 con 0x7f545c106170 2024-04-21T08:41:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.491+0000 7f5463196640 1 -- 172.21.15.43:0/216609391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f545c1a5590 con 0x7f545c106170 2024-04-21T08:41:09.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.493+0000 7f5459ffb640 1 -- 172.21.15.43:0/216609391 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5448046020 con 0x7f545c106170 2024-04-21T08:41:09.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.493+0000 7f5465d85740 1 Processor -- start 2024-04-21T08:41:09.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.493+0000 7f5465d85740 1 -- start start 2024-04-21T08:41:09.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.493+0000 7f5465d85740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a144d0 0x55f436a148b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.493+0000 7f5465d85740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f436a14e80 0x55f436a1d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.493+0000 7f5465d85740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f436a1da00 0x55f436a1fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.493+0000 7f5465d85740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f4369791f0 con 0x55f436a144d0 2024-04-21T08:41:09.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.493+0000 7f5465d85740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f4369164b0 con 0x55f436a14e80 2024-04-21T08:41:09.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.493+0000 7f5465d85740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f4369751b0 con 0x55f436a1da00 2024-04-21T08:41:09.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.494+0000 7f5461733640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a144d0 0x55f436a148b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.494+0000 7f5461733640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a144d0 0x55f436a148b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55756/0 (socket says 172.21.15.43:55756) 2024-04-21T08:41:09.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.494+0000 7f5461733640 1 -- 172.21.15.43:0/100087682 learned_addr learned my addr 172.21.15.43:0/100087682 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.494+0000 7f5459ffb640 1 --2- 172.21.15.43:0/216609391 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f544c0a3d20 0x7f544c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.494+0000 7f5460f32640 1 --2- 172.21.15.43:0/100087682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f436a14e80 0x55f436a1d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.494+0000 7f545bfff640 1 --2- 172.21.15.43:0/100087682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f436a1da00 0x55f436a1fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.494+0000 7f5461733640 1 --2- 172.21.15.43:0/216609391 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f544c0a3d20 0x7f544c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.494+0000 7f5459ffb640 1 -- 172.21.15.43:0/216609391 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f54480f2c50 con 0x7f545c106170 2024-04-21T08:41:09.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.494+0000 7f5461733640 1 -- 172.21.15.43:0/100087682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f436a1da00 msgr2=0x55f436a1fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.494+0000 7f5461733640 1 --2- 172.21.15.43:0/100087682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f436a1da00 0x55f436a1fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.494+0000 7f5461733640 1 -- 172.21.15.43:0/100087682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f436a14e80 msgr2=0x55f436a1d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.494+0000 7f5461733640 1 --2- 172.21.15.43:0/100087682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f436a14e80 0x55f436a1d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.494+0000 7f5461733640 1 -- 172.21.15.43:0/100087682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4369081b0 con 0x55f436a144d0 2024-04-21T08:41:09.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.494+0000 7f5461733640 1 --2- 172.21.15.43:0/100087682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a144d0 0x55f436a148b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f5454012d30 tx=0x7f545400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a1bf4a11a6de6d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.494+0000 7f5461733640 1 --2- 172.21.15.43:0/216609391 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f544c0a3d20 0x7f544c0a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f545400ec20 tx=0x7f5454020000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.494+0000 7f5446ffd640 1 -- 172.21.15.43:0/100087682 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5454012490 con 0x55f436a144d0 2024-04-21T08:41:09.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.495+0000 7f5446ffd640 1 -- 172.21.15.43:0/100087682 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5454012880 con 0x55f436a144d0 2024-04-21T08:41:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.495+0000 7f5446ffd640 1 -- 172.21.15.43:0/100087682 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5454010b30 con 0x55f436a144d0 2024-04-21T08:41:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.495+0000 7f5465d85740 1 -- 172.21.15.43:0/100087682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a144d0 msgr2=0x55f436a148b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.495+0000 7f5465d85740 1 --2- 172.21.15.43:0/100087682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a144d0 0x55f436a148b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f5454012d30 tx=0x7f545400e720 comp rx=0 tx=0).stop 2024-04-21T08:41:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.495+0000 7f5465d85740 1 -- 172.21.15.43:0/100087682 shutdown_connections 2024-04-21T08:41:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.495+0000 7f5465d85740 1 --2- 172.21.15.43:0/100087682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f436a1da00 0x55f436a1fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.495+0000 7f5465d85740 1 --2- 172.21.15.43:0/100087682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f436a14e80 0x55f436a1d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.495+0000 7f5465d85740 1 --2- 172.21.15.43:0/100087682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a144d0 0x55f436a148b0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.495+0000 7f5465d85740 1 -- 172.21.15.43:0/100087682 >> 172.21.15.43:0/100087682 conn(0x55f436a0eba0 msgr2=0x55f436a0ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.495+0000 7f5465d85740 1 -- 172.21.15.43:0/100087682 shutdown_connections 2024-04-21T08:41:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.495+0000 7f5465d85740 1 -- 172.21.15.43:0/100087682 wait complete. 2024-04-21T08:41:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.496+0000 7f5465d85740 1 -- start start 2024-04-21T08:41:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.496+0000 7f5465d85740 1 Processor -- start 2024-04-21T08:41:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.496+0000 7f5465d85740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a144d0 0x55f436a1d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.496+0000 7f5465d85740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f436a14e80 0x55f436a1a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.496+0000 7f5465d85740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f436a1da00 0x55f436a1ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.496+0000 7f5465d85740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f436a20910 con 0x55f436a144d0 2024-04-21T08:41:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.496+0000 7f5465d85740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f436932500 con 0x55f436a1da00 2024-04-21T08:41:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.496+0000 7f5465d85740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f436a20a90 con 0x55f436a14e80 2024-04-21T08:41:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.496+0000 7f5465d85740 10 client.0 _ll_register_callbacks cb 0x55f436900b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.496+0000 7f5460f32640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a144d0 0x55f436a1d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.496+0000 7f5460f32640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a144d0 0x55f436a1d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55760/0 (socket says 172.21.15.43:55760) 2024-04-21T08:41:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.496+0000 7f5460f32640 1 -- 172.21.15.43:0/821401422 learned_addr learned my addr 172.21.15.43:0/821401422 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.496+0000 7f5461733640 1 --2- 172.21.15.43:0/821401422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f436a1da00 0x55f436a1ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.496+0000 7f545bfff640 1 --2- 172.21.15.43:0/821401422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f436a14e80 0x55f436a1a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.497+0000 7f5460f32640 1 -- 172.21.15.43:0/821401422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f436a14e80 msgr2=0x55f436a1a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.497+0000 7f5460f32640 1 --2- 172.21.15.43:0/821401422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f436a14e80 0x55f436a1a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.497+0000 7f5460f32640 1 -- 172.21.15.43:0/821401422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f436a1da00 msgr2=0x55f436a1ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.497+0000 7f5460f32640 1 --2- 172.21.15.43:0/821401422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f436a1da00 0x55f436a1ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.497+0000 7f5460f32640 1 -- 172.21.15.43:0/821401422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f43668b3f0 con 0x55f436a144d0 2024-04-21T08:41:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.497+0000 7f5460f32640 1 --2- 172.21.15.43:0/821401422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a144d0 0x55f436a1d310 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f5450009a50 tx=0x7f5450009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92f426696a973b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.497+0000 7f543bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.497+0000 7f543bfff640 1 -- 172.21.15.43:0/821401422 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450023060 con 0x55f436a144d0 2024-04-21T08:41:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.497+0000 7f543bfff640 1 -- 172.21.15.43:0/821401422 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f545001e400 con 0x55f436a144d0 2024-04-21T08:41:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.497+0000 7f5465d85740 1 -- 172.21.15.43:0/821401422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5454012880 con 0x55f436a144d0 2024-04-21T08:41:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.497+0000 7f543bfff640 1 -- 172.21.15.43:0/821401422 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f545002a050 con 0x55f436a144d0 2024-04-21T08:41:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.497+0000 7f5465d85740 10 client.16689 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.497+0000 7f5465d85740 1 -- 172.21.15.43:0/821401422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f4367557d0 con 0x55f436a144d0 2024-04-21T08:41:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.497+0000 7f543bfff640 1 -- 172.21.15.43:0/821401422 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5454012880 con 0x55f436a144d0 2024-04-21T08:41:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.497+0000 7f543bfff640 1 -- 172.21.15.43:0/821401422 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f545000d340 con 0x55f436a144d0 2024-04-21T08:41:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.497+0000 7f5465d85740 10 client.16689 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.497+0000 7f5465d85740 10 client.16689 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:41:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.497+0000 7f5465d85740 20 client.16689 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.497+0000 7f5465d85740 1 --2- 172.21.15.43:0/821401422 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55f436a2dc90 0x55f436a300c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.497+0000 7f5465d85740 4 client.16689 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.497+0000 7f5465d85740 1 -- 172.21.15.43:0/821401422 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f436918930 con 0x55f436a2dc90 2024-04-21T08:41:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.498+0000 7f545bfff640 1 --2- 172.21.15.43:0/821401422 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55f436a2dc90 0x55f436a300c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.498+0000 7f543bfff640 10 client.16689 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:41:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.498+0000 7f545bfff640 1 --2- 172.21.15.43:0/821401422 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55f436a2dc90 0x55f436a300c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=28108dc88d5741da server_cookie=9184d15307845bcd in_seq=0 out_seq=0 2024-04-21T08:41:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.499+0000 7f543bfff640 1 -- 172.21.15.43:0/821401422 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f436918930 con 0x55f436a2dc90 2024-04-21T08:41:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.499+0000 7f543bfff640 10 client.16689 handle_command_reply: tid=0 2024-04-21T08:41:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.499+0000 7f5465d85740 1 client.16689 shutdown 2024-04-21T08:41:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.499+0000 7f5465d85740 1 -- 172.21.15.43:0/821401422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a144d0 msgr2=0x55f436a1d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.499+0000 7f5465d85740 1 --2- 172.21.15.43:0/821401422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a144d0 0x55f436a1d310 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f5450009a50 tx=0x7f5450009f10 comp rx=0 tx=0).stop 2024-04-21T08:41:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.499+0000 7f5465d85740 1 -- 172.21.15.43:0/821401422 shutdown_connections 2024-04-21T08:41:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.499+0000 7f5465d85740 1 --2- 172.21.15.43:0/821401422 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55f436a2dc90 0x55f436a300c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.499+0000 7f5465d85740 1 --2- 172.21.15.43:0/821401422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f436a1da00 0x55f436a1ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.499+0000 7f5465d85740 1 --2- 172.21.15.43:0/821401422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f436a14e80 0x55f436a1a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.499+0000 7f5465d85740 1 --2- 172.21.15.43:0/821401422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a144d0 0x55f436a1d310 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.499+0000 7f543bfff640 0 client.16689 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:41:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.499+0000 7f5465d85740 1 -- 172.21.15.43:0/821401422 >> 172.21.15.43:0/821401422 conn(0x55f436a0eba0 msgr2=0x55f436a108f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.499+0000 7f5465d85740 1 -- 172.21.15.43:0/821401422 shutdown_connections 2024-04-21T08:41:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.499+0000 7f5465d85740 1 -- 172.21.15.43:0/821401422 wait complete. 2024-04-21T08:41:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.499+0000 7f5465d85740 20 client.16689 trim_cache size 0 max 16384 2024-04-21T08:41:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.513+0000 7f5465d85740 1 Processor -- start 2024-04-21T08:41:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.513+0000 7f5465d85740 1 -- start start 2024-04-21T08:41:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.513+0000 7f5465d85740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f436a144d0 0x55f436a29800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.513+0000 7f5465d85740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a14e80 0x55f436a1cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.513+0000 7f5465d85740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f436a1da00 0x55f436a1d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.513+0000 7f5465d85740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f545c1ad1c0 con 0x55f436a14e80 2024-04-21T08:41:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.513+0000 7f5465d85740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f545c1aed00 con 0x55f436a1da00 2024-04-21T08:41:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.513+0000 7f5465d85740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f545c1ad890 con 0x55f436a144d0 2024-04-21T08:41:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.513+0000 7f545bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a14e80 0x55f436a1cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.513+0000 7f545bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a14e80 0x55f436a1cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55776/0 (socket says 172.21.15.43:55776) 2024-04-21T08:41:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.513+0000 7f545bfff640 1 -- 172.21.15.43:0/3362347860 learned_addr learned my addr 172.21.15.43:0/3362347860 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.513+0000 7f5461733640 1 --2- 172.21.15.43:0/3362347860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f436a1da00 0x55f436a1d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.513+0000 7f5460f32640 1 --2- 172.21.15.43:0/3362347860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f436a144d0 0x55f436a29800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.513+0000 7f545bfff640 1 -- 172.21.15.43:0/3362347860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f436a144d0 msgr2=0x55f436a29800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.513+0000 7f545bfff640 1 --2- 172.21.15.43:0/3362347860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f436a144d0 0x55f436a29800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.513+0000 7f545bfff640 1 -- 172.21.15.43:0/3362347860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f436a1da00 msgr2=0x55f436a1d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.513+0000 7f545bfff640 1 --2- 172.21.15.43:0/3362347860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f436a1da00 0x55f436a1d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.513+0000 7f545bfff640 1 -- 172.21.15.43:0/3362347860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f436a1afb0 con 0x55f436a14e80 2024-04-21T08:41:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.513+0000 7f5460f32640 1 --2- 172.21.15.43:0/3362347860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f436a144d0 0x55f436a29800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.513+0000 7f5461733640 1 --2- 172.21.15.43:0/3362347860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f436a1da00 0x55f436a1d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.513+0000 7f545bfff640 1 --2- 172.21.15.43:0/3362347860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a14e80 0x55f436a1cdd0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f54480be860 tx=0x7f54480df040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2b20ab4c325351 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.514+0000 7f54467fc640 1 -- 172.21.15.43:0/3362347860 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54480befd0 con 0x55f436a14e80 2024-04-21T08:41:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.514+0000 7f54467fc640 1 -- 172.21.15.43:0/3362347860 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5448082040 con 0x55f436a14e80 2024-04-21T08:41:09.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.514+0000 7f54467fc640 1 -- 172.21.15.43:0/3362347860 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54480bcc60 con 0x55f436a14e80 2024-04-21T08:41:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.514+0000 7f5465d85740 1 -- 172.21.15.43:0/3362347860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a14e80 msgr2=0x55f436a1cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.514+0000 7f5465d85740 1 --2- 172.21.15.43:0/3362347860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a14e80 0x55f436a1cdd0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f54480be860 tx=0x7f54480df040 comp rx=0 tx=0).stop 2024-04-21T08:41:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.514+0000 7f5465d85740 1 -- 172.21.15.43:0/3362347860 shutdown_connections 2024-04-21T08:41:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.514+0000 7f5465d85740 1 --2- 172.21.15.43:0/3362347860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f436a1da00 0x55f436a1d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.514+0000 7f5465d85740 1 --2- 172.21.15.43:0/3362347860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a14e80 0x55f436a1cdd0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.514+0000 7f5465d85740 1 --2- 172.21.15.43:0/3362347860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f436a144d0 0x55f436a29800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.514+0000 7f5465d85740 1 -- 172.21.15.43:0/3362347860 >> 172.21.15.43:0/3362347860 conn(0x55f436a0eba0 msgr2=0x55f436a1a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.514+0000 7f5465d85740 1 -- 172.21.15.43:0/3362347860 shutdown_connections 2024-04-21T08:41:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.514+0000 7f5465d85740 1 -- 172.21.15.43:0/3362347860 wait complete. 2024-04-21T08:41:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.515+0000 7f5465d85740 1 -- start start 2024-04-21T08:41:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.515+0000 7f5465d85740 1 Processor -- start 2024-04-21T08:41:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.515+0000 7f5465d85740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f436a144d0 0x55f436a25d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.515+0000 7f5465d85740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a14e80 0x55f436a26290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.515+0000 7f5465d85740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f436a1da00 0x55f436a21480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.515+0000 7f5465d85740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f545c1ae630 con 0x55f436a14e80 2024-04-21T08:41:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.515+0000 7f5465d85740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f545c1adf60 con 0x55f436a1da00 2024-04-21T08:41:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.515+0000 7f5465d85740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f545c1ab1a0 con 0x55f436a144d0 2024-04-21T08:41:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.515+0000 7f5465d85740 10 client.0 _ll_register_callbacks cb 0x7f5450013e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.515+0000 7f545bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a14e80 0x55f436a26290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.515+0000 7f545bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a14e80 0x55f436a26290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55792/0 (socket says 172.21.15.43:55792) 2024-04-21T08:41:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.515+0000 7f545bfff640 1 -- 172.21.15.43:0/3908988266 learned_addr learned my addr 172.21.15.43:0/3908988266 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.515+0000 7f5461733640 1 --2- 172.21.15.43:0/3908988266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f436a1da00 0x55f436a21480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.515+0000 7f5460f32640 1 --2- 172.21.15.43:0/3908988266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f436a144d0 0x55f436a25d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.515+0000 7f545bfff640 1 -- 172.21.15.43:0/3908988266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f436a144d0 msgr2=0x55f436a25d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.515+0000 7f545bfff640 1 --2- 172.21.15.43:0/3908988266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f436a144d0 0x55f436a25d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.515+0000 7f545bfff640 1 -- 172.21.15.43:0/3908988266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f436a1da00 msgr2=0x55f436a21480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.515+0000 7f545bfff640 1 --2- 172.21.15.43:0/3908988266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f436a1da00 0x55f436a21480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.515+0000 7f545bfff640 1 -- 172.21.15.43:0/3908988266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f436a21cb0 con 0x55f436a14e80 2024-04-21T08:41:09.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.515+0000 7f5460f32640 1 --2- 172.21.15.43:0/3908988266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f436a144d0 0x55f436a25d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:09.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.516+0000 7f545bfff640 1 --2- 172.21.15.43:0/3908988266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a14e80 0x55f436a26290 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f54480bad30 tx=0x7f5448009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7220369f794839c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.516+0000 7f543bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:09.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.516+0000 7f543bfff640 1 -- 172.21.15.43:0/3908988266 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54480c7540 con 0x55f436a14e80 2024-04-21T08:41:09.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.516+0000 7f543bfff640 1 -- 172.21.15.43:0/3908988266 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f54480053d0 con 0x55f436a14e80 2024-04-21T08:41:09.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.516+0000 7f5465d85740 1 -- 172.21.15.43:0/3908988266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5448082040 con 0x55f436a14e80 2024-04-21T08:41:09.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.516+0000 7f543bfff640 1 -- 172.21.15.43:0/3908988266 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f54480befd0 con 0x55f436a14e80 2024-04-21T08:41:09.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.516+0000 7f5465d85740 10 client.16695 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:09.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.516+0000 7f5465d85740 1 -- 172.21.15.43:0/3908988266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f436a93dd0 con 0x55f436a14e80 2024-04-21T08:41:09.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.516+0000 7f543bfff640 1 -- 172.21.15.43:0/3908988266 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54480db070 con 0x55f436a14e80 2024-04-21T08:41:09.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.516+0000 7f543bfff640 1 -- 172.21.15.43:0/3908988266 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f54480a90b0 con 0x55f436a14e80 2024-04-21T08:41:09.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.516+0000 7f5465d85740 10 client.16695 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:09.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.516+0000 7f5465d85740 10 client.16695 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:41:09.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.516+0000 7f5465d85740 20 client.16695 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:09.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.516+0000 7f5465d85740 1 --2- 172.21.15.43:0/3908988266 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55f436a943e0 0x55f436a967d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.516+0000 7f5465d85740 4 client.16695 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:41:09.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.516+0000 7f5465d85740 1 -- 172.21.15.43:0/3908988266 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f436986c10 con 0x55f436a943e0 2024-04-21T08:41:09.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.517+0000 7f5460f32640 1 --2- 172.21.15.43:0/3908988266 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55f436a943e0 0x55f436a967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.517+0000 7f5460f32640 1 --2- 172.21.15.43:0/3908988266 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55f436a943e0 0x55f436a967d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=497f49f0e1a304a7 server_cookie=59057db60aebda85 in_seq=0 out_seq=0 2024-04-21T08:41:09.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.517+0000 7f543bfff640 10 client.16695 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:41:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.517+0000 7f543bfff640 1 -- 172.21.15.43:0/3908988266 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f436986c10 con 0x55f436a943e0 2024-04-21T08:41:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.517+0000 7f543bfff640 10 client.16695 handle_command_reply: tid=0 2024-04-21T08:41:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.517+0000 7f5465d85740 1 client.16695 shutdown 2024-04-21T08:41:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.517+0000 7f5465d85740 1 -- 172.21.15.43:0/3908988266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a14e80 msgr2=0x55f436a26290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.517+0000 7f5465d85740 1 --2- 172.21.15.43:0/3908988266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a14e80 0x55f436a26290 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f54480bad30 tx=0x7f5448009680 comp rx=0 tx=0).stop 2024-04-21T08:41:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.517+0000 7f5465d85740 1 -- 172.21.15.43:0/3908988266 shutdown_connections 2024-04-21T08:41:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.517+0000 7f5465d85740 1 --2- 172.21.15.43:0/3908988266 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55f436a943e0 0x55f436a967d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.517+0000 7f5465d85740 1 --2- 172.21.15.43:0/3908988266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f436a1da00 0x55f436a21480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.517+0000 7f5465d85740 1 --2- 172.21.15.43:0/3908988266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f436a14e80 0x55f436a26290 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.517+0000 7f543bfff640 0 client.16695 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:41:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.517+0000 7f5465d85740 1 --2- 172.21.15.43:0/3908988266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f436a144d0 0x55f436a25d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.517+0000 7f5465d85740 1 -- 172.21.15.43:0/3908988266 >> 172.21.15.43:0/3908988266 conn(0x55f436a0eba0 msgr2=0x55f436a0ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.517+0000 7f5465d85740 1 -- 172.21.15.43:0/3908988266 shutdown_connections 2024-04-21T08:41:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.517+0000 7f5465d85740 1 -- 172.21.15.43:0/3908988266 wait complete. 2024-04-21T08:41:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.518+0000 7f5465d85740 20 client.16695 trim_cache size 0 max 16384 2024-04-21T08:41:09.526 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:41:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.518+0000 7f5463196640 1 -- 172.21.15.43:0/216609391 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f544c0a3d20 msgr2=0x7f544c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.518+0000 7f5463196640 1 --2- 172.21.15.43:0/216609391 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f544c0a3d20 0x7f544c0a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f545400ec20 tx=0x7f5454020000 comp rx=0 tx=0).stop 2024-04-21T08:41:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.518+0000 7f5463196640 1 -- 172.21.15.43:0/216609391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f545c106170 msgr2=0x7f545c198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.518+0000 7f5463196640 1 --2- 172.21.15.43:0/216609391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f545c106170 0x7f545c198c90 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f5448036040 tx=0x7f54480045e0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.518+0000 7f5463196640 1 -- 172.21.15.43:0/216609391 shutdown_connections 2024-04-21T08:41:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.518+0000 7f5463196640 1 --2- 172.21.15.43:0/216609391 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f544c0a3d20 0x7f544c0a61e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.518+0000 7f5463196640 1 --2- 172.21.15.43:0/216609391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f545c19d010 0x7f545c19d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.518+0000 7f5463196640 1 --2- 172.21.15.43:0/216609391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f545c106170 0x7f545c198c90 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.518+0000 7f5463196640 1 --2- 172.21.15.43:0/216609391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f545c1010a0 0x7f545c198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.518+0000 7f5463196640 1 -- 172.21.15.43:0/216609391 >> 172.21.15.43:0/216609391 conn(0x7f545c077eb0 msgr2=0x7f545c107340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.518+0000 7f5463196640 1 -- 172.21.15.43:0/216609391 shutdown_connections 2024-04-21T08:41:09.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.518+0000 7f5463196640 1 -- 172.21.15.43:0/216609391 wait complete. 2024-04-21T08:41:09.529 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:41:09.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.633+0000 7ff5d888e640 1 -- 172.21.15.43:0/4286314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5d4102b40 msgr2=0x7ff5d4106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.633+0000 7ff5d888e640 1 --2- 172.21.15.43:0/4286314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5d4102b40 0x7ff5d4106b80 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009900 tx=0x7ff5bc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:09.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.633+0000 7ff5d888e640 1 -- 172.21.15.43:0/4286314 shutdown_connections 2024-04-21T08:41:09.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.633+0000 7ff5d888e640 1 --2- 172.21.15.43:0/4286314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5d4107260 0x7ff5d410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.633+0000 7ff5d888e640 1 --2- 172.21.15.43:0/4286314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5d4102b40 0x7ff5d4106b80 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.633+0000 7ff5d888e640 1 --2- 172.21.15.43:0/4286314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d4102190 0x7ff5d4102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.633+0000 7ff5d888e640 1 -- 172.21.15.43:0/4286314 >> 172.21.15.43:0/4286314 conn(0x7ff5d40fd990 msgr2=0x7ff5d40ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.633+0000 7ff5d888e640 1 -- 172.21.15.43:0/4286314 shutdown_connections 2024-04-21T08:41:09.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.633+0000 7ff5d888e640 1 -- 172.21.15.43:0/4286314 wait complete. 2024-04-21T08:41:09.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.634+0000 7ff5d888e640 1 Processor -- start 2024-04-21T08:41:09.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.634+0000 7ff5d888e640 1 -- start start 2024-04-21T08:41:09.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.634+0000 7ff5d888e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5d4102190 0x7ff5d419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.634+0000 7ff5d888e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5d4107260 0x7ff5d419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.634+0000 7ff5d888e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d41985a0 0x7ff5d4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.634+0000 7ff5d888e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d41121a0 con 0x7ff5d4107260 2024-04-21T08:41:09.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.634+0000 7ff5d888e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d4112020 con 0x7ff5d41985a0 2024-04-21T08:41:09.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.634+0000 7ff5d888e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d4112320 con 0x7ff5d4102190 2024-04-21T08:41:09.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.634+0000 7ff5cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5d4107260 0x7ff5d419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.634+0000 7ff5cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5d4107260 0x7ff5d419e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55806/0 (socket says 172.21.15.43:55806) 2024-04-21T08:41:09.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.634+0000 7ff5cbfff640 1 -- 172.21.15.43:0/343143066 learned_addr learned my addr 172.21.15.43:0/343143066 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.634+0000 7ff5d259c640 1 --2- 172.21.15.43:0/343143066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5d4102190 0x7ff5d419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.635+0000 7ff5d2d9d640 1 --2- 172.21.15.43:0/343143066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d41985a0 0x7ff5d4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.635+0000 7ff5cbfff640 1 -- 172.21.15.43:0/343143066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5d4102190 msgr2=0x7ff5d419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.635+0000 7ff5cbfff640 1 --2- 172.21.15.43:0/343143066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5d4102190 0x7ff5d419e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.635+0000 7ff5cbfff640 1 -- 172.21.15.43:0/343143066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d41985a0 msgr2=0x7ff5d4198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.635+0000 7ff5cbfff640 1 --2- 172.21.15.43:0/343143066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d41985a0 0x7ff5d4198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.635+0000 7ff5cbfff640 1 -- 172.21.15.43:0/343143066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d41990c0 con 0x7ff5d4107260 2024-04-21T08:41:09.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.635+0000 7ff5d2d9d640 1 --2- 172.21.15.43:0/343143066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d41985a0 0x7ff5d4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:09.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.635+0000 7ff5cbfff640 1 --2- 172.21.15.43:0/343143066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5d4107260 0x7ff5d419e930 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc036040 tx=0x7ff5bc02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.635+0000 7ff5cb7fe640 1 -- 172.21.15.43:0/343143066 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc043020 con 0x7ff5d4107260 2024-04-21T08:41:09.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.635+0000 7ff5cb7fe640 1 -- 172.21.15.43:0/343143066 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5bc02cbe0 con 0x7ff5d4107260 2024-04-21T08:41:09.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.635+0000 7ff5d888e640 1 -- 172.21.15.43:0/343143066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d4199350 con 0x7ff5d4107260 2024-04-21T08:41:09.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.635+0000 7ff5cb7fe640 1 -- 172.21.15.43:0/343143066 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc003bf0 con 0x7ff5d4107260 2024-04-21T08:41:09.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.635+0000 7ff5d888e640 1 -- 172.21.15.43:0/343143066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d41a55a0 con 0x7ff5d4107260 2024-04-21T08:41:09.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.637+0000 7ff5cb7fe640 1 -- 172.21.15.43:0/343143066 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff5bc046020 con 0x7ff5d4107260 2024-04-21T08:41:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.638+0000 7ff5db47d740 1 Processor -- start 2024-04-21T08:41:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.638+0000 7ff5db47d740 1 -- start start 2024-04-21T08:41:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.638+0000 7ff5cb7fe640 1 --2- 172.21.15.43:0/343143066 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff5c00a3d20 0x7ff5c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.638+0000 7ff5db47d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc0e4d0 0x55589bc0e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.638+0000 7ff5db47d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55589bc0ee80 0x55589bc17430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.638+0000 7ff5db47d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55589bc17a00 0x55589bc19e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.638+0000 7ff5db47d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55589bb731f0 con 0x55589bc0e4d0 2024-04-21T08:41:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.638+0000 7ff5db47d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55589bb104b0 con 0x55589bc17a00 2024-04-21T08:41:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.638+0000 7ff5db47d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55589bb6f1b0 con 0x55589bc0ee80 2024-04-21T08:41:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.638+0000 7ff5d2d9d640 1 --2- 172.21.15.43:0/343143066 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff5c00a3d20 0x7ff5c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.638+0000 7ff5d259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc0e4d0 0x55589bc0e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.638+0000 7ff5d259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc0e4d0 0x55589bc0e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55822/0 (socket says 172.21.15.43:55822) 2024-04-21T08:41:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.638+0000 7ff5d259c640 1 -- 172.21.15.43:0/1650735091 learned_addr learned my addr 172.21.15.43:0/1650735091 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.638+0000 7ff5cb7fe640 1 -- 172.21.15.43:0/343143066 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7ff5bc0f2d60 con 0x7ff5d4107260 2024-04-21T08:41:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.638+0000 7ff5d2d9d640 1 --2- 172.21.15.43:0/1650735091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55589bc17a00 0x55589bc19e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.638+0000 7ff5cbfff640 1 --2- 172.21.15.43:0/1650735091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55589bc0ee80 0x55589bc17430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.638+0000 7ff5d259c640 1 -- 172.21.15.43:0/1650735091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55589bc0ee80 msgr2=0x55589bc17430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.639+0000 7ff5d259c640 1 --2- 172.21.15.43:0/1650735091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55589bc0ee80 0x55589bc17430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.639+0000 7ff5d259c640 1 -- 172.21.15.43:0/1650735091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55589bc17a00 msgr2=0x55589bc19e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.639+0000 7ff5d259c640 1 --2- 172.21.15.43:0/1650735091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55589bc17a00 0x55589bc19e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.639+0000 7ff5d259c640 1 -- 172.21.15.43:0/1650735091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55589bb021b0 con 0x55589bc0e4d0 2024-04-21T08:41:09.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.639+0000 7ff5d259c640 1 --2- 172.21.15.43:0/1650735091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc0e4d0 0x55589bc0e8b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff5c400bd10 tx=0x7ff5c40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd927e32c2f069fb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.639+0000 7ff5d2d9d640 1 --2- 172.21.15.43:0/343143066 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff5c00a3d20 0x7ff5c00a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc005cc0 tx=0x7ff5cc0127c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.639+0000 7ff5c8ff9640 1 -- 172.21.15.43:0/1650735091 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4004030 con 0x55589bc0e4d0 2024-04-21T08:41:09.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.639+0000 7ff5c8ff9640 1 -- 172.21.15.43:0/1650735091 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5c40035f0 con 0x55589bc0e4d0 2024-04-21T08:41:09.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.639+0000 7ff5c8ff9640 1 -- 172.21.15.43:0/1650735091 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4011640 con 0x55589bc0e4d0 2024-04-21T08:41:09.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.640+0000 7ff5db47d740 1 -- 172.21.15.43:0/1650735091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc0e4d0 msgr2=0x55589bc0e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.640+0000 7ff5db47d740 1 --2- 172.21.15.43:0/1650735091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc0e4d0 0x55589bc0e8b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff5c400bd10 tx=0x7ff5c40107a0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.640+0000 7ff5db47d740 1 -- 172.21.15.43:0/1650735091 shutdown_connections 2024-04-21T08:41:09.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.640+0000 7ff5db47d740 1 --2- 172.21.15.43:0/1650735091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55589bc17a00 0x55589bc19e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.640+0000 7ff5db47d740 1 --2- 172.21.15.43:0/1650735091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55589bc0ee80 0x55589bc17430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.640+0000 7ff5db47d740 1 --2- 172.21.15.43:0/1650735091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc0e4d0 0x55589bc0e8b0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.640+0000 7ff5db47d740 1 -- 172.21.15.43:0/1650735091 >> 172.21.15.43:0/1650735091 conn(0x55589bc08ba0 msgr2=0x55589bc08f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.640+0000 7ff5db47d740 1 -- 172.21.15.43:0/1650735091 shutdown_connections 2024-04-21T08:41:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.640+0000 7ff5db47d740 1 -- 172.21.15.43:0/1650735091 wait complete. 2024-04-21T08:41:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.640+0000 7ff5db47d740 1 -- start start 2024-04-21T08:41:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.640+0000 7ff5db47d740 1 Processor -- start 2024-04-21T08:41:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.640+0000 7ff5db47d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55589bc0e4d0 0x55589bc172d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.640+0000 7ff5db47d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55589bc0ee80 0x55589bc14660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.640+0000 7ff5db47d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc17a00 0x55589bc14bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.640+0000 7ff5db47d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55589bc1a970 con 0x55589bc17a00 2024-04-21T08:41:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.640+0000 7ff5db47d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55589bb2c500 con 0x55589bc0ee80 2024-04-21T08:41:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.640+0000 7ff5db47d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55589bc1aaf0 con 0x55589bc0e4d0 2024-04-21T08:41:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.640+0000 7ff5db47d740 10 client.0 _ll_register_callbacks cb 0x55589bafab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.641+0000 7ff5d2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc17a00 0x55589bc14bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.641+0000 7ff5d2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc17a00 0x55589bc14bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55824/0 (socket says 172.21.15.43:55824) 2024-04-21T08:41:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.641+0000 7ff5d2d9d640 1 -- 172.21.15.43:0/1171693141 learned_addr learned my addr 172.21.15.43:0/1171693141 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.641+0000 7ff5cbfff640 1 --2- 172.21.15.43:0/1171693141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55589bc0ee80 0x55589bc14660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.641+0000 7ff5d259c640 1 --2- 172.21.15.43:0/1171693141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55589bc0e4d0 0x55589bc172d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.641+0000 7ff5d2d9d640 1 -- 172.21.15.43:0/1171693141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55589bc0e4d0 msgr2=0x55589bc172d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.641+0000 7ff5d2d9d640 1 --2- 172.21.15.43:0/1171693141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55589bc0e4d0 0x55589bc172d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.641+0000 7ff5d2d9d640 1 -- 172.21.15.43:0/1171693141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55589bc0ee80 msgr2=0x55589bc14660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.641+0000 7ff5d2d9d640 1 --2- 172.21.15.43:0/1171693141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55589bc0ee80 0x55589bc14660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.641+0000 7ff5d2d9d640 1 -- 172.21.15.43:0/1171693141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55589b8853f0 con 0x55589bc17a00 2024-04-21T08:41:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.641+0000 7ff5d2d9d640 1 --2- 172.21.15.43:0/1171693141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc17a00 0x55589bc14bc0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc010e30 tx=0x7ff5cc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f51483ce59ede1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.641+0000 7ff5b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.641+0000 7ff5b9ffb640 1 -- 172.21.15.43:0/1171693141 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc022030 con 0x55589bc17a00 2024-04-21T08:41:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.641+0000 7ff5b9ffb640 1 -- 172.21.15.43:0/1171693141 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7ff5cc004b20 con 0x55589bc17a00 2024-04-21T08:41:09.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.641+0000 7ff5db47d740 1 -- 172.21.15.43:0/1171693141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5c40035f0 con 0x55589bc17a00 2024-04-21T08:41:09.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.641+0000 7ff5b9ffb640 1 -- 172.21.15.43:0/1171693141 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff5cc02e050 con 0x55589bc17a00 2024-04-21T08:41:09.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.641+0000 7ff5db47d740 10 client.16707 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:09.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.641+0000 7ff5db47d740 1 -- 172.21.15.43:0/1171693141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55589b94f7d0 con 0x55589bc17a00 2024-04-21T08:41:09.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.642+0000 7ff5b9ffb640 1 -- 172.21.15.43:0/1171693141 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5c40035f0 con 0x55589bc17a00 2024-04-21T08:41:09.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.642+0000 7ff5b9ffb640 1 -- 172.21.15.43:0/1171693141 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff5cc01e410 con 0x55589bc17a00 2024-04-21T08:41:09.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.642+0000 7ff5db47d740 10 client.16707 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:09.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.642+0000 7ff5db47d740 10 client.16707 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:41:09.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.642+0000 7ff5db47d740 20 client.16707 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:09.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.642+0000 7ff5db47d740 1 --2- 172.21.15.43:0/1171693141 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55589bc27b70 0x55589bc29fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.642+0000 7ff5db47d740 4 client.16707 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:09.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.642+0000 7ff5db47d740 1 -- 172.21.15.43:0/1171693141 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55589bb12930 con 0x55589bc27b70 2024-04-21T08:41:09.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.642+0000 7ff5d259c640 1 --2- 172.21.15.43:0/1171693141 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55589bc27b70 0x55589bc29fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.643+0000 7ff5b9ffb640 10 client.16707 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:41:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.643+0000 7ff5d259c640 1 --2- 172.21.15.43:0/1171693141 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55589bc27b70 0x55589bc29fa0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47f830694e8509c5 server_cookie=773cfc5eacc75304 in_seq=0 out_seq=0 2024-04-21T08:41:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.643+0000 7ff5b9ffb640 1 -- 172.21.15.43:0/1171693141 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55589bb12930 con 0x55589bc27b70 2024-04-21T08:41:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.644+0000 7ff5b9ffb640 10 client.16707 handle_command_reply: tid=0 2024-04-21T08:41:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.644+0000 7ff5db47d740 1 client.16707 shutdown 2024-04-21T08:41:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.644+0000 7ff5db47d740 1 -- 172.21.15.43:0/1171693141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc17a00 msgr2=0x55589bc14bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.644+0000 7ff5db47d740 1 --2- 172.21.15.43:0/1171693141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc17a00 0x55589bc14bc0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc010e30 tx=0x7ff5cc004ab0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.644+0000 7ff5db47d740 1 -- 172.21.15.43:0/1171693141 shutdown_connections 2024-04-21T08:41:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.644+0000 7ff5db47d740 1 --2- 172.21.15.43:0/1171693141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc17a00 0x55589bc14bc0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.644+0000 7ff5db47d740 1 --2- 172.21.15.43:0/1171693141 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55589bc27b70 0x55589bc29fa0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.644+0000 7ff5db47d740 1 --2- 172.21.15.43:0/1171693141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55589bc0ee80 0x55589bc14660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.644+0000 7ff5db47d740 1 --2- 172.21.15.43:0/1171693141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55589bc0e4d0 0x55589bc172d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.644+0000 7ff5b9ffb640 0 client.16707 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:41:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.644+0000 7ff5db47d740 1 -- 172.21.15.43:0/1171693141 >> 172.21.15.43:0/1171693141 conn(0x55589bc08ba0 msgr2=0x55589bc0a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.644+0000 7ff5db47d740 1 -- 172.21.15.43:0/1171693141 shutdown_connections 2024-04-21T08:41:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.644+0000 7ff5db47d740 1 -- 172.21.15.43:0/1171693141 wait complete. 2024-04-21T08:41:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.644+0000 7ff5db47d740 20 client.16707 trim_cache size 0 max 16384 2024-04-21T08:41:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.657+0000 7ff5db47d740 1 Processor -- start 2024-04-21T08:41:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.657+0000 7ff5db47d740 1 -- start start 2024-04-21T08:41:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.658+0000 7ff5db47d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55589bc0e4d0 0x55589bc23830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.658+0000 7ff5db47d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc0ee80 0x55589bc20120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.658+0000 7ff5db47d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55589bc17a00 0x55589bc07a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.658+0000 7ff5db47d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d41af520 con 0x55589bc0ee80 2024-04-21T08:41:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.658+0000 7ff5db47d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d41b1060 con 0x55589bc17a00 2024-04-21T08:41:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.658+0000 7ff5db47d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d41afbf0 con 0x55589bc0e4d0 2024-04-21T08:41:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.658+0000 7ff5cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc0ee80 0x55589bc20120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.658+0000 7ff5cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc0ee80 0x55589bc20120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55826/0 (socket says 172.21.15.43:55826) 2024-04-21T08:41:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.658+0000 7ff5cbfff640 1 -- 172.21.15.43:0/2211431943 learned_addr learned my addr 172.21.15.43:0/2211431943 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.658+0000 7ff5d259c640 1 --2- 172.21.15.43:0/2211431943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55589bc0e4d0 0x55589bc23830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.658+0000 7ff5cbfff640 1 -- 172.21.15.43:0/2211431943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55589bc0e4d0 msgr2=0x55589bc23830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.658+0000 7ff5cbfff640 1 --2- 172.21.15.43:0/2211431943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55589bc0e4d0 0x55589bc23830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.658+0000 7ff5d2d9d640 1 --2- 172.21.15.43:0/2211431943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55589bc17a00 0x55589bc07a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.658+0000 7ff5cbfff640 1 -- 172.21.15.43:0/2211431943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55589bc17a00 msgr2=0x55589bc07a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.658+0000 7ff5cbfff640 1 --2- 172.21.15.43:0/2211431943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55589bc17a00 0x55589bc07a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.658+0000 7ff5cbfff640 1 -- 172.21.15.43:0/2211431943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55589bc12bb0 con 0x55589bc0ee80 2024-04-21T08:41:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.658+0000 7ff5d2d9d640 1 --2- 172.21.15.43:0/2211431943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55589bc17a00 0x55589bc07a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.658+0000 7ff5cbfff640 1 --2- 172.21.15.43:0/2211431943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc0ee80 0x55589bc20120 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc0bae40 tx=0x7ff5bc0df040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48675c66d22a79fe server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.658+0000 7ff5bbfff640 1 -- 172.21.15.43:0/2211431943 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc0bf0e0 con 0x55589bc0ee80 2024-04-21T08:41:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.658+0000 7ff5bbfff640 1 -- 172.21.15.43:0/2211431943 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5bc089780 con 0x55589bc0ee80 2024-04-21T08:41:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.658+0000 7ff5bbfff640 1 -- 172.21.15.43:0/2211431943 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc0bb050 con 0x55589bc0ee80 2024-04-21T08:41:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.659+0000 7ff5db47d740 1 -- 172.21.15.43:0/2211431943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc0ee80 msgr2=0x55589bc20120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.659+0000 7ff5db47d740 1 --2- 172.21.15.43:0/2211431943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc0ee80 0x55589bc20120 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc0bae40 tx=0x7ff5bc0df040 comp rx=0 tx=0).stop 2024-04-21T08:41:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.659+0000 7ff5db47d740 1 -- 172.21.15.43:0/2211431943 shutdown_connections 2024-04-21T08:41:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.659+0000 7ff5db47d740 1 --2- 172.21.15.43:0/2211431943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55589bc17a00 0x55589bc07a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.659+0000 7ff5db47d740 1 --2- 172.21.15.43:0/2211431943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc0ee80 0x55589bc20120 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.659+0000 7ff5db47d740 1 --2- 172.21.15.43:0/2211431943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55589bc0e4d0 0x55589bc23830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.659+0000 7ff5db47d740 1 -- 172.21.15.43:0/2211431943 >> 172.21.15.43:0/2211431943 conn(0x55589bc08ba0 msgr2=0x55589bc07370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.659+0000 7ff5db47d740 1 -- 172.21.15.43:0/2211431943 shutdown_connections 2024-04-21T08:41:09.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.659+0000 7ff5db47d740 1 -- 172.21.15.43:0/2211431943 wait complete. 2024-04-21T08:41:09.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.659+0000 7ff5db47d740 1 -- start start 2024-04-21T08:41:09.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.659+0000 7ff5db47d740 1 Processor -- start 2024-04-21T08:41:09.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.659+0000 7ff5db47d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55589bc0e4d0 0x55589bc169c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.659+0000 7ff5db47d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55589bc0ee80 0x55589bc16f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.660+0000 7ff5db47d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc17a00 0x55589bc897d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.660+0000 7ff5db47d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d41b0990 con 0x55589bc17a00 2024-04-21T08:41:09.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.660+0000 7ff5db47d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d41b02c0 con 0x55589bc0e4d0 2024-04-21T08:41:09.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.660+0000 7ff5db47d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d41ad500 con 0x55589bc0ee80 2024-04-21T08:41:09.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.660+0000 7ff5db47d740 10 client.0 _ll_register_callbacks cb 0x7ff5cc010b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:09.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.660+0000 7ff5d2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc17a00 0x55589bc897d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.660+0000 7ff5d2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc17a00 0x55589bc897d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55828/0 (socket says 172.21.15.43:55828) 2024-04-21T08:41:09.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.660+0000 7ff5d2d9d640 1 -- 172.21.15.43:0/530106632 learned_addr learned my addr 172.21.15.43:0/530106632 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.660+0000 7ff5cbfff640 1 --2- 172.21.15.43:0/530106632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55589bc0ee80 0x55589bc16f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.660+0000 7ff5d259c640 1 --2- 172.21.15.43:0/530106632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55589bc0e4d0 0x55589bc169c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.660+0000 7ff5d2d9d640 1 -- 172.21.15.43:0/530106632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55589bc0ee80 msgr2=0x55589bc16f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.660+0000 7ff5d2d9d640 1 --2- 172.21.15.43:0/530106632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55589bc0ee80 0x55589bc16f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.660+0000 7ff5d2d9d640 1 -- 172.21.15.43:0/530106632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55589bc0e4d0 msgr2=0x55589bc169c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.660+0000 7ff5d2d9d640 1 --2- 172.21.15.43:0/530106632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55589bc0e4d0 0x55589bc169c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.660+0000 7ff5d2d9d640 1 -- 172.21.15.43:0/530106632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55589bc89e40 con 0x55589bc17a00 2024-04-21T08:41:09.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.660+0000 7ff5d2d9d640 1 --2- 172.21.15.43:0/530106632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc17a00 0x55589bc897d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc004a80 tx=0x7ff5cc00ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c66acfb40351f2c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.660+0000 7ff5b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:09.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.660+0000 7ff5b9ffb640 1 -- 172.21.15.43:0/530106632 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc022040 con 0x55589bc17a00 2024-04-21T08:41:09.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.660+0000 7ff5b9ffb640 1 -- 172.21.15.43:0/530106632 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7ff5cc01e9d0 con 0x55589bc17a00 2024-04-21T08:41:09.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.660+0000 7ff5db47d740 1 -- 172.21.15.43:0/530106632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5bc089780 con 0x55589bc17a00 2024-04-21T08:41:09.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.660+0000 7ff5b9ffb640 1 -- 172.21.15.43:0/530106632 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff5cc01ebd0 con 0x55589bc17a00 2024-04-21T08:41:09.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.661+0000 7ff5db47d740 10 client.16713 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:09.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.661+0000 7ff5db47d740 1 -- 172.21.15.43:0/530106632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55589bc923b0 con 0x55589bc17a00 2024-04-21T08:41:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.661+0000 7ff5b9ffb640 1 -- 172.21.15.43:0/530106632 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5bc089780 con 0x55589bc17a00 2024-04-21T08:41:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.661+0000 7ff5b9ffb640 1 -- 172.21.15.43:0/530106632 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff5cc03a030 con 0x55589bc17a00 2024-04-21T08:41:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.661+0000 7ff5db47d740 10 client.16713 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.661+0000 7ff5db47d740 10 client.16713 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:41:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.661+0000 7ff5db47d740 20 client.16713 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.661+0000 7ff5db47d740 1 --2- 172.21.15.43:0/530106632 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55589bc92b80 0x55589bc94fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.661+0000 7ff5db47d740 4 client.16713 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:41:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.661+0000 7ff5db47d740 1 -- 172.21.15.43:0/530106632 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55589bb80c10 con 0x55589bc92b80 2024-04-21T08:41:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.661+0000 7ff5d259c640 1 --2- 172.21.15.43:0/530106632 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55589bc92b80 0x55589bc94fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.661+0000 7ff5b9ffb640 10 client.16713 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:41:09.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.661+0000 7ff5d259c640 1 --2- 172.21.15.43:0/530106632 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55589bc92b80 0x55589bc94fa0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f4c75900c42f0a82 server_cookie=e00ec81d54b06758 in_seq=0 out_seq=0 2024-04-21T08:41:09.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.662+0000 7ff5b9ffb640 1 -- 172.21.15.43:0/530106632 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55589bb80c10 con 0x55589bc92b80 2024-04-21T08:41:09.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.662+0000 7ff5b9ffb640 10 client.16713 handle_command_reply: tid=0 2024-04-21T08:41:09.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.662+0000 7ff5db47d740 1 client.16713 shutdown 2024-04-21T08:41:09.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.662+0000 7ff5db47d740 1 -- 172.21.15.43:0/530106632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc17a00 msgr2=0x55589bc897d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.662+0000 7ff5db47d740 1 --2- 172.21.15.43:0/530106632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc17a00 0x55589bc897d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc004a80 tx=0x7ff5cc00ad00 comp rx=0 tx=0).stop 2024-04-21T08:41:09.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.662+0000 7ff5db47d740 1 -- 172.21.15.43:0/530106632 shutdown_connections 2024-04-21T08:41:09.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.662+0000 7ff5db47d740 1 --2- 172.21.15.43:0/530106632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55589bc17a00 0x55589bc897d0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.662+0000 7ff5db47d740 1 --2- 172.21.15.43:0/530106632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55589bc0ee80 0x55589bc16f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.662+0000 7ff5db47d740 1 --2- 172.21.15.43:0/530106632 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55589bc92b80 0x55589bc94fa0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.662+0000 7ff5db47d740 1 --2- 172.21.15.43:0/530106632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55589bc0e4d0 0x55589bc169c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.662+0000 7ff5b9ffb640 0 client.16713 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:41:09.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.662+0000 7ff5db47d740 1 -- 172.21.15.43:0/530106632 >> 172.21.15.43:0/530106632 conn(0x55589bc08ba0 msgr2=0x55589bc07c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.662+0000 7ff5db47d740 1 -- 172.21.15.43:0/530106632 shutdown_connections 2024-04-21T08:41:09.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.662+0000 7ff5db47d740 1 -- 172.21.15.43:0/530106632 wait complete. 2024-04-21T08:41:09.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.662+0000 7ff5db47d740 20 client.16713 trim_cache size 0 max 16384 2024-04-21T08:41:09.678 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:41:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.662+0000 7ff5d888e640 1 -- 172.21.15.43:0/343143066 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff5c00a3d20 msgr2=0x7ff5c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.662+0000 7ff5d888e640 1 --2- 172.21.15.43:0/343143066 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff5c00a3d20 0x7ff5c00a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc005cc0 tx=0x7ff5cc0127c0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.662+0000 7ff5d888e640 1 -- 172.21.15.43:0/343143066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5d4107260 msgr2=0x7ff5d419e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.662+0000 7ff5d888e640 1 --2- 172.21.15.43:0/343143066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5d4107260 0x7ff5d419e930 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc036040 tx=0x7ff5bc02c7e0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.662+0000 7ff5d888e640 1 -- 172.21.15.43:0/343143066 shutdown_connections 2024-04-21T08:41:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.662+0000 7ff5d888e640 1 --2- 172.21.15.43:0/343143066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d41985a0 0x7ff5d4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.662+0000 7ff5d888e640 1 --2- 172.21.15.43:0/343143066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5d4107260 0x7ff5d419e930 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.662+0000 7ff5d888e640 1 --2- 172.21.15.43:0/343143066 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff5c00a3d20 0x7ff5c00a61e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.662+0000 7ff5d888e640 1 --2- 172.21.15.43:0/343143066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5d4102190 0x7ff5d419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.663+0000 7ff5d888e640 1 -- 172.21.15.43:0/343143066 >> 172.21.15.43:0/343143066 conn(0x7ff5d40fd990 msgr2=0x7ff5d40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.663+0000 7ff5d888e640 1 -- 172.21.15.43:0/343143066 shutdown_connections 2024-04-21T08:41:09.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.663+0000 7ff5d888e640 1 -- 172.21.15.43:0/343143066 wait complete. 2024-04-21T08:41:09.682 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:41:09.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.787+0000 7f14ef2a8640 1 -- 172.21.15.43:0/3737675680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14e8107270 msgr2=0x7f14e810ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.787+0000 7f14ef2a8640 1 --2- 172.21.15.43:0/3737675680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14e8107270 0x7f14e810ee40 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009ca0 tx=0x7f14e002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.787+0000 7f14ef2a8640 1 -- 172.21.15.43:0/3737675680 shutdown_connections 2024-04-21T08:41:09.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.787+0000 7f14ef2a8640 1 --2- 172.21.15.43:0/3737675680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14e8107270 0x7f14e810ee40 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.787+0000 7f14ef2a8640 1 --2- 172.21.15.43:0/3737675680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14e806b1f0 0x7f14e8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.787+0000 7f14ef2a8640 1 --2- 172.21.15.43:0/3737675680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14e806a840 0x7f14e806ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.787+0000 7f14ef2a8640 1 -- 172.21.15.43:0/3737675680 >> 172.21.15.43:0/3737675680 conn(0x7f14e8077eb0 msgr2=0x7f14e80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.787+0000 7f14ef2a8640 1 -- 172.21.15.43:0/3737675680 shutdown_connections 2024-04-21T08:41:09.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.787+0000 7f14ef2a8640 1 -- 172.21.15.43:0/3737675680 wait complete. 2024-04-21T08:41:09.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.788+0000 7f14ef2a8640 1 Processor -- start 2024-04-21T08:41:09.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.788+0000 7f14ef2a8640 1 -- start start 2024-04-21T08:41:09.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.788+0000 7f14ef2a8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14e806a840 0x7f14e81986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.788+0000 7f14ef2a8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14e8107270 0x7f14e8198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.788+0000 7f14ef2a8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14e819cf90 0x7f14e819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.788+0000 7f14ef2a8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14e81121d0 con 0x7f14e8107270 2024-04-21T08:41:09.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.788+0000 7f14ef2a8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14e8112050 con 0x7f14e806a840 2024-04-21T08:41:09.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.788+0000 7f14ef2a8640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f14e8112350 con 0x7f14e819cf90 2024-04-21T08:41:09.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.788+0000 7f14ec843640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14e8107270 0x7f14e8198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.788+0000 7f14ec843640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14e8107270 0x7f14e8198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55856/0 (socket says 172.21.15.43:55856) 2024-04-21T08:41:09.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.788+0000 7f14ec843640 1 -- 172.21.15.43:0/290514522 learned_addr learned my addr 172.21.15.43:0/290514522 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.788+0000 7f14ed044640 1 --2- 172.21.15.43:0/290514522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14e806a840 0x7f14e81986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.789+0000 7f14ed845640 1 --2- 172.21.15.43:0/290514522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14e819cf90 0x7f14e819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.789+0000 7f14ec843640 1 -- 172.21.15.43:0/290514522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14e819cf90 msgr2=0x7f14e819d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.789+0000 7f14ec843640 1 --2- 172.21.15.43:0/290514522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14e819cf90 0x7f14e819d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.789+0000 7f14ec843640 1 -- 172.21.15.43:0/290514522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14e806a840 msgr2=0x7f14e81986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.789+0000 7f14ec843640 1 --2- 172.21.15.43:0/290514522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14e806a840 0x7f14e81986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.789+0000 7f14ec843640 1 -- 172.21.15.43:0/290514522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e819da80 con 0x7f14e8107270 2024-04-21T08:41:09.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.789+0000 7f14ed044640 1 --2- 172.21.15.43:0/290514522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14e806a840 0x7f14e81986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:09.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.789+0000 7f14ec843640 1 --2- 172.21.15.43:0/290514522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14e8107270 0x7f14e8198c10 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f14d800c970 tx=0x7f14d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.789+0000 7f14de7fc640 1 -- 172.21.15.43:0/290514522 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8007bb0 con 0x7f14e8107270 2024-04-21T08:41:09.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.789+0000 7f14ef2a8640 1 -- 172.21.15.43:0/290514522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e819dd70 con 0x7f14e8107270 2024-04-21T08:41:09.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.789+0000 7f14de7fc640 1 -- 172.21.15.43:0/290514522 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14d8007d50 con 0x7f14e8107270 2024-04-21T08:41:09.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.789+0000 7f14de7fc640 1 -- 172.21.15.43:0/290514522 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8018730 con 0x7f14e8107270 2024-04-21T08:41:09.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.789+0000 7f14ef2a8640 1 -- 172.21.15.43:0/290514522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e81a5510 con 0x7f14e8107270 2024-04-21T08:41:09.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.792+0000 7f14de7fc640 1 -- 172.21.15.43:0/290514522 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f14d800f990 con 0x7f14e8107270 2024-04-21T08:41:09.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.792+0000 7f14f1f7a740 1 Processor -- start 2024-04-21T08:41:09.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.792+0000 7f14f1f7a740 1 -- start start 2024-04-21T08:41:09.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.792+0000 7f14f1f7a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56439cf484d0 0x56439cf488b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.792+0000 7f14f1f7a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf48e80 0x56439cf51430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.793+0000 7f14f1f7a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56439cf51a00 0x56439cf53e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.793+0000 7f14f1f7a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56439cead1f0 con 0x56439cf48e80 2024-04-21T08:41:09.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.793+0000 7f14f1f7a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56439ce4a4b0 con 0x56439cf484d0 2024-04-21T08:41:09.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.793+0000 7f14f1f7a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56439cea91b0 con 0x56439cf51a00 2024-04-21T08:41:09.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.793+0000 7f14ed044640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf48e80 0x56439cf51430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.793+0000 7f14ed044640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf48e80 0x56439cf51430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55864/0 (socket says 172.21.15.43:55864) 2024-04-21T08:41:09.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.793+0000 7f14ed044640 1 -- 172.21.15.43:0/1123019127 learned_addr learned my addr 172.21.15.43:0/1123019127 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.793+0000 7f14ec843640 1 --2- 172.21.15.43:0/1123019127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56439cf51a00 0x56439cf53e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.793+0000 7f14de7fc640 1 --2- 172.21.15.43:0/290514522 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f14d40a3d20 0x7f14d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.793+0000 7f14ed845640 1 --2- 172.21.15.43:0/1123019127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56439cf484d0 0x56439cf488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.793+0000 7f14ed845640 1 --2- 172.21.15.43:0/290514522 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f14d40a3d20 0x7f14d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.793+0000 7f14de7fc640 1 -- 172.21.15.43:0/290514522 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f14d80d0cd0 con 0x7f14e8107270 2024-04-21T08:41:09.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.793+0000 7f14ed044640 1 -- 172.21.15.43:0/1123019127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56439cf51a00 msgr2=0x56439cf53e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.793+0000 7f14ed044640 1 --2- 172.21.15.43:0/1123019127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56439cf51a00 0x56439cf53e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.793+0000 7f14ed044640 1 -- 172.21.15.43:0/1123019127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56439cf484d0 msgr2=0x56439cf488b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.793+0000 7f14ed044640 1 --2- 172.21.15.43:0/1123019127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56439cf484d0 0x56439cf488b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.793+0000 7f14ed044640 1 -- 172.21.15.43:0/1123019127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56439ce3c1b0 con 0x56439cf48e80 2024-04-21T08:41:09.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.793+0000 7f14ed044640 1 --2- 172.21.15.43:0/1123019127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf48e80 0x56439cf51430 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009c50 tx=0x7f14e400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1e22c860836a79 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.793+0000 7f14ed845640 1 --2- 172.21.15.43:0/290514522 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f14d40a3d20 0x7f14d40a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f14e002c630 tx=0x7f14e00095c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.793+0000 7f14d37fe640 1 -- 172.21.15.43:0/1123019127 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4016070 con 0x56439cf48e80 2024-04-21T08:41:09.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.794+0000 7f14d37fe640 1 -- 172.21.15.43:0/1123019127 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14e40054d0 con 0x56439cf48e80 2024-04-21T08:41:09.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.794+0000 7f14d37fe640 1 -- 172.21.15.43:0/1123019127 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4011470 con 0x56439cf48e80 2024-04-21T08:41:09.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.794+0000 7f14f1f7a740 1 -- 172.21.15.43:0/1123019127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf48e80 msgr2=0x56439cf51430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.794+0000 7f14f1f7a740 1 --2- 172.21.15.43:0/1123019127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf48e80 0x56439cf51430 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009c50 tx=0x7f14e400e6c0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.794+0000 7f14f1f7a740 1 -- 172.21.15.43:0/1123019127 shutdown_connections 2024-04-21T08:41:09.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.794+0000 7f14f1f7a740 1 --2- 172.21.15.43:0/1123019127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56439cf51a00 0x56439cf53e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.794+0000 7f14f1f7a740 1 --2- 172.21.15.43:0/1123019127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf48e80 0x56439cf51430 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.794+0000 7f14f1f7a740 1 --2- 172.21.15.43:0/1123019127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56439cf484d0 0x56439cf488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.794+0000 7f14f1f7a740 1 -- 172.21.15.43:0/1123019127 >> 172.21.15.43:0/1123019127 conn(0x56439cf42ba0 msgr2=0x56439cf42f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.794+0000 7f14f1f7a740 1 -- 172.21.15.43:0/1123019127 shutdown_connections 2024-04-21T08:41:09.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.794+0000 7f14f1f7a740 1 -- 172.21.15.43:0/1123019127 wait complete. 2024-04-21T08:41:09.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.795+0000 7f14f1f7a740 1 -- start start 2024-04-21T08:41:09.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.795+0000 7f14f1f7a740 1 Processor -- start 2024-04-21T08:41:09.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.795+0000 7f14f1f7a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56439cf484d0 0x56439cf512e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.795+0000 7f14f1f7a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf48e80 0x56439cf4e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.795+0000 7f14f1f7a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56439cf51a00 0x56439cf4ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.795+0000 7f14f1f7a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56439cf54970 con 0x56439cf48e80 2024-04-21T08:41:09.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.795+0000 7f14f1f7a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56439ce66500 con 0x56439cf484d0 2024-04-21T08:41:09.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.795+0000 7f14f1f7a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56439cf54af0 con 0x56439cf51a00 2024-04-21T08:41:09.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.795+0000 7f14ec843640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf48e80 0x56439cf4e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.795+0000 7f14f1f7a740 10 client.0 _ll_register_callbacks cb 0x56439ce34b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:09.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.795+0000 7f14ec843640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf48e80 0x56439cf4e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55878/0 (socket says 172.21.15.43:55878) 2024-04-21T08:41:09.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.795+0000 7f14ec843640 1 -- 172.21.15.43:0/2271846330 learned_addr learned my addr 172.21.15.43:0/2271846330 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.795+0000 7f14ed044640 1 --2- 172.21.15.43:0/2271846330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56439cf484d0 0x56439cf512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.795+0000 7f14ed845640 1 --2- 172.21.15.43:0/2271846330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56439cf51a00 0x56439cf4ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.796+0000 7f14ec843640 1 -- 172.21.15.43:0/2271846330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56439cf51a00 msgr2=0x56439cf4ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.796+0000 7f14ec843640 1 --2- 172.21.15.43:0/2271846330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56439cf51a00 0x56439cf4ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.796+0000 7f14ec843640 1 -- 172.21.15.43:0/2271846330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56439cf484d0 msgr2=0x56439cf512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.796+0000 7f14ec843640 1 --2- 172.21.15.43:0/2271846330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56439cf484d0 0x56439cf512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.796+0000 7f14ec843640 1 -- 172.21.15.43:0/2271846330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56439cbbf3f0 con 0x56439cf48e80 2024-04-21T08:41:09.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.796+0000 7f14ed044640 1 --2- 172.21.15.43:0/2271846330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56439cf484d0 0x56439cf512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:09.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.796+0000 7f14ed845640 1 --2- 172.21.15.43:0/2271846330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56439cf51a00 0x56439cf4ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:09.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.796+0000 7f14ec843640 1 --2- 172.21.15.43:0/2271846330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf48e80 0x56439cf4e670 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f14d8017000 tx=0x7f14d80d0ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18b60e124dfe758 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.796+0000 7f14d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:09.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.796+0000 7f14d0ff9640 1 -- 172.21.15.43:0/2271846330 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d809ddd0 con 0x56439cf48e80 2024-04-21T08:41:09.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.796+0000 7f14d0ff9640 1 -- 172.21.15.43:0/2271846330 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f14d80870b0 con 0x56439cf48e80 2024-04-21T08:41:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.796+0000 7f14f1f7a740 1 -- 172.21.15.43:0/2271846330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14e40054d0 con 0x56439cf48e80 2024-04-21T08:41:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.796+0000 7f14d0ff9640 1 -- 172.21.15.43:0/2271846330 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14d80a0c50 con 0x56439cf48e80 2024-04-21T08:41:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.796+0000 7f14f1f7a740 10 client.16725 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.796+0000 7f14f1f7a740 1 -- 172.21.15.43:0/2271846330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56439cc897d0 con 0x56439cf48e80 2024-04-21T08:41:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.796+0000 7f14d0ff9640 1 -- 172.21.15.43:0/2271846330 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14e40054d0 con 0x56439cf48e80 2024-04-21T08:41:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.796+0000 7f14d0ff9640 1 -- 172.21.15.43:0/2271846330 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f14d8020030 con 0x56439cf48e80 2024-04-21T08:41:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.796+0000 7f14f1f7a740 10 client.16725 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.796+0000 7f14f1f7a740 10 client.16725 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:41:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.797+0000 7f14f1f7a740 20 client.16725 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.797+0000 7f14f1f7a740 1 --2- 172.21.15.43:0/2271846330 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x56439cf61b50 0x56439cf63f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.797+0000 7f14f1f7a740 4 client.16725 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.797+0000 7f14f1f7a740 1 -- 172.21.15.43:0/2271846330 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56439ce4c930 con 0x56439cf61b50 2024-04-21T08:41:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.797+0000 7f14ed044640 1 --2- 172.21.15.43:0/2271846330 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x56439cf61b50 0x56439cf63f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.797+0000 7f14d0ff9640 10 client.16725 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:41:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.797+0000 7f14ed044640 1 --2- 172.21.15.43:0/2271846330 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x56439cf61b50 0x56439cf63f60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3e51bdc5d7be73d server_cookie=aac10c2fa50c9092 in_seq=0 out_seq=0 2024-04-21T08:41:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.798+0000 7f14d0ff9640 1 -- 172.21.15.43:0/2271846330 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56439ce4c930 con 0x56439cf61b50 2024-04-21T08:41:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.798+0000 7f14d0ff9640 10 client.16725 handle_command_reply: tid=0 2024-04-21T08:41:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.798+0000 7f14f1f7a740 1 client.16725 shutdown 2024-04-21T08:41:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.798+0000 7f14f1f7a740 1 -- 172.21.15.43:0/2271846330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf48e80 msgr2=0x56439cf4e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.798+0000 7f14f1f7a740 1 --2- 172.21.15.43:0/2271846330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf48e80 0x56439cf4e670 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f14d8017000 tx=0x7f14d80d0ed0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.798+0000 7f14f1f7a740 1 -- 172.21.15.43:0/2271846330 shutdown_connections 2024-04-21T08:41:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.798+0000 7f14f1f7a740 1 --2- 172.21.15.43:0/2271846330 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x56439cf61b50 0x56439cf63f60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.798+0000 7f14f1f7a740 1 --2- 172.21.15.43:0/2271846330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56439cf51a00 0x56439cf4ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.798+0000 7f14d0ff9640 0 client.16725 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:41:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.798+0000 7f14f1f7a740 1 --2- 172.21.15.43:0/2271846330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf48e80 0x56439cf4e670 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.798+0000 7f14f1f7a740 1 --2- 172.21.15.43:0/2271846330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56439cf484d0 0x56439cf512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.798+0000 7f14f1f7a740 1 -- 172.21.15.43:0/2271846330 >> 172.21.15.43:0/2271846330 conn(0x56439cf42ba0 msgr2=0x56439cf448f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.798+0000 7f14f1f7a740 1 -- 172.21.15.43:0/2271846330 shutdown_connections 2024-04-21T08:41:09.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.798+0000 7f14f1f7a740 1 -- 172.21.15.43:0/2271846330 wait complete. 2024-04-21T08:41:09.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.799+0000 7f14f1f7a740 20 client.16725 trim_cache size 0 max 16384 2024-04-21T08:41:09.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.812+0000 7f14f1f7a740 1 Processor -- start 2024-04-21T08:41:09.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.812+0000 7f14f1f7a740 1 -- start start 2024-04-21T08:41:09.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.812+0000 7f14f1f7a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56439cf484d0 0x56439cf5d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.812+0000 7f14f1f7a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56439cf48e80 0x56439cf41a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.812+0000 7f14f1f7a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf51a00 0x56439cf50da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.812+0000 7f14f1f7a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14e81af450 con 0x56439cf51a00 2024-04-21T08:41:09.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.812+0000 7f14f1f7a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14e81b0f90 con 0x56439cf48e80 2024-04-21T08:41:09.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.812+0000 7f14f1f7a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f14e81afb20 con 0x56439cf484d0 2024-04-21T08:41:09.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.812+0000 7f14ed845640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf51a00 0x56439cf50da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.812+0000 7f14ed845640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf51a00 0x56439cf50da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55892/0 (socket says 172.21.15.43:55892) 2024-04-21T08:41:09.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.812+0000 7f14ed845640 1 -- 172.21.15.43:0/3436691146 learned_addr learned my addr 172.21.15.43:0/3436691146 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.812+0000 7f14ed044640 1 --2- 172.21.15.43:0/3436691146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56439cf484d0 0x56439cf5d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.813+0000 7f14ec843640 1 --2- 172.21.15.43:0/3436691146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56439cf48e80 0x56439cf41a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.813+0000 7f14ed845640 1 -- 172.21.15.43:0/3436691146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56439cf484d0 msgr2=0x56439cf5d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.813+0000 7f14ed845640 1 --2- 172.21.15.43:0/3436691146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56439cf484d0 0x56439cf5d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.813+0000 7f14ed845640 1 -- 172.21.15.43:0/3436691146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56439cf48e80 msgr2=0x56439cf41a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.813+0000 7f14ed845640 1 --2- 172.21.15.43:0/3436691146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56439cf48e80 0x56439cf41a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.813+0000 7f14ed845640 1 -- 172.21.15.43:0/3436691146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56439cf48c20 con 0x56439cf51a00 2024-04-21T08:41:09.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.813+0000 7f14ed044640 1 --2- 172.21.15.43:0/3436691146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56439cf484d0 0x56439cf5d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:09.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.813+0000 7f14ed845640 1 --2- 172.21.15.43:0/3436691146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf51a00 0x56439cf50da0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009ca0 tx=0x7f14e003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77a8a0fde153415 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.813+0000 7f14d2ffd640 1 -- 172.21.15.43:0/3436691146 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e0039070 con 0x56439cf51a00 2024-04-21T08:41:09.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.813+0000 7f14d2ffd640 1 -- 172.21.15.43:0/3436691146 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14e0053070 con 0x56439cf51a00 2024-04-21T08:41:09.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.813+0000 7f14d2ffd640 1 -- 172.21.15.43:0/3436691146 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e004e6a0 con 0x56439cf51a00 2024-04-21T08:41:09.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.814+0000 7f14f1f7a740 1 -- 172.21.15.43:0/3436691146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf51a00 msgr2=0x56439cf50da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.814+0000 7f14f1f7a740 1 --2- 172.21.15.43:0/3436691146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf51a00 0x56439cf50da0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009ca0 tx=0x7f14e003e000 comp rx=0 tx=0).stop 2024-04-21T08:41:09.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.814+0000 7f14f1f7a740 1 -- 172.21.15.43:0/3436691146 shutdown_connections 2024-04-21T08:41:09.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.814+0000 7f14f1f7a740 1 --2- 172.21.15.43:0/3436691146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf51a00 0x56439cf50da0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.814+0000 7f14f1f7a740 1 --2- 172.21.15.43:0/3436691146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56439cf48e80 0x56439cf41a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.814+0000 7f14f1f7a740 1 --2- 172.21.15.43:0/3436691146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56439cf484d0 0x56439cf5d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.814+0000 7f14f1f7a740 1 -- 172.21.15.43:0/3436691146 >> 172.21.15.43:0/3436691146 conn(0x56439cf42ba0 msgr2=0x56439cf41370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.814+0000 7f14f1f7a740 1 -- 172.21.15.43:0/3436691146 shutdown_connections 2024-04-21T08:41:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.814+0000 7f14f1f7a740 1 -- 172.21.15.43:0/3436691146 wait complete. 2024-04-21T08:41:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.814+0000 7f14f1f7a740 1 -- start start 2024-04-21T08:41:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.814+0000 7f14f1f7a740 1 Processor -- start 2024-04-21T08:41:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.814+0000 7f14f1f7a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56439cf484d0 0x56439cf55950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14f1f7a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf48e80 0x56439cf55e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14f1f7a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56439cf51a00 0x56439cf563d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14f1f7a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14e81b08c0 con 0x56439cf48e80 2024-04-21T08:41:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14f1f7a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14e81b01f0 con 0x56439cf51a00 2024-04-21T08:41:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14f1f7a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f14e81ad430 con 0x56439cf484d0 2024-04-21T08:41:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14f1f7a740 10 client.0 _ll_register_callbacks cb 0x7f14d800a480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14ec843640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf48e80 0x56439cf55e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14ec843640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf48e80 0x56439cf55e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55906/0 (socket says 172.21.15.43:55906) 2024-04-21T08:41:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14ec843640 1 -- 172.21.15.43:0/171097472 learned_addr learned my addr 172.21.15.43:0/171097472 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14ed845640 1 --2- 172.21.15.43:0/171097472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56439cf51a00 0x56439cf563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14ed044640 1 --2- 172.21.15.43:0/171097472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56439cf484d0 0x56439cf55950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14ec843640 1 -- 172.21.15.43:0/171097472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56439cf484d0 msgr2=0x56439cf55950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14ec843640 1 --2- 172.21.15.43:0/171097472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56439cf484d0 0x56439cf55950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14ec843640 1 -- 172.21.15.43:0/171097472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56439cf51a00 msgr2=0x56439cf563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14ec843640 1 --2- 172.21.15.43:0/171097472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56439cf51a00 0x56439cf563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14ec843640 1 -- 172.21.15.43:0/171097472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56439cfc72a0 con 0x56439cf48e80 2024-04-21T08:41:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14ed845640 1 --2- 172.21.15.43:0/171097472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56439cf51a00 0x56439cf563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14ed044640 1 --2- 172.21.15.43:0/171097472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56439cf484d0 0x56439cf55950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14ec843640 1 --2- 172.21.15.43:0/171097472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf48e80 0x56439cf55e90 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x56439cf4fc40 tx=0x7f14d80d1060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3e5c2c47342a49 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14d0ff9640 1 -- 172.21.15.43:0/171097472 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d809c2b0 con 0x56439cf48e80 2024-04-21T08:41:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14d0ff9640 1 -- 172.21.15.43:0/171097472 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f14d80870b0 con 0x56439cf48e80 2024-04-21T08:41:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14f1f7a740 1 -- 172.21.15.43:0/171097472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14e0053070 con 0x56439cf48e80 2024-04-21T08:41:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14d0ff9640 1 -- 172.21.15.43:0/171097472 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14d80986a0 con 0x56439cf48e80 2024-04-21T08:41:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14f1f7a740 10 client.16731 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.815+0000 7f14f1f7a740 1 -- 172.21.15.43:0/171097472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56439cfc8560 con 0x56439cf48e80 2024-04-21T08:41:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.816+0000 7f14d0ff9640 1 -- 172.21.15.43:0/171097472 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14e0053070 con 0x56439cf48e80 2024-04-21T08:41:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.816+0000 7f14d0ff9640 1 -- 172.21.15.43:0/171097472 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f14d8020070 con 0x56439cf48e80 2024-04-21T08:41:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.816+0000 7f14f1f7a740 10 client.16731 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.816+0000 7f14f1f7a740 10 client.16731 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:41:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.816+0000 7f14f1f7a740 20 client.16731 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.816+0000 7f14f1f7a740 1 --2- 172.21.15.43:0/171097472 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x56439cfc8a90 0x56439cfcae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.816+0000 7f14f1f7a740 4 client.16731 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:41:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.816+0000 7f14f1f7a740 1 -- 172.21.15.43:0/171097472 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56439cebac10 con 0x56439cfc8a90 2024-04-21T08:41:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.816+0000 7f14ed044640 1 --2- 172.21.15.43:0/171097472 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x56439cfc8a90 0x56439cfcae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.816+0000 7f14ed044640 1 --2- 172.21.15.43:0/171097472 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x56439cfc8a90 0x56439cfcae80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=997daf992f32b7dc server_cookie=d49563d06545f651 in_seq=0 out_seq=0 2024-04-21T08:41:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.816+0000 7f14d0ff9640 10 client.16731 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:41:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14d0ff9640 1 -- 172.21.15.43:0/171097472 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56439cebac10 con 0x56439cfc8a90 2024-04-21T08:41:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14d0ff9640 10 client.16731 handle_command_reply: tid=0 2024-04-21T08:41:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14f1f7a740 1 client.16731 shutdown 2024-04-21T08:41:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14f1f7a740 1 -- 172.21.15.43:0/171097472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf48e80 msgr2=0x56439cf55e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14f1f7a740 1 --2- 172.21.15.43:0/171097472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf48e80 0x56439cf55e90 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x56439cf4fc40 tx=0x7f14d80d1060 comp rx=0 tx=0).stop 2024-04-21T08:41:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14f1f7a740 1 -- 172.21.15.43:0/171097472 shutdown_connections 2024-04-21T08:41:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14f1f7a740 1 --2- 172.21.15.43:0/171097472 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x56439cfc8a90 0x56439cfcae80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14f1f7a740 1 --2- 172.21.15.43:0/171097472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56439cf51a00 0x56439cf563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14f1f7a740 1 --2- 172.21.15.43:0/171097472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56439cf48e80 0x56439cf55e90 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14d0ff9640 0 client.16731 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:41:09.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14f1f7a740 1 --2- 172.21.15.43:0/171097472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56439cf484d0 0x56439cf55950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14f1f7a740 1 -- 172.21.15.43:0/171097472 >> 172.21.15.43:0/171097472 conn(0x56439cf42ba0 msgr2=0x56439cf41c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14f1f7a740 1 -- 172.21.15.43:0/171097472 shutdown_connections 2024-04-21T08:41:09.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14f1f7a740 1 -- 172.21.15.43:0/171097472 wait complete. 2024-04-21T08:41:09.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14f1f7a740 20 client.16731 trim_cache size 0 max 16384 2024-04-21T08:41:09.828 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:41:09.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14ef2a8640 1 -- 172.21.15.43:0/290514522 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f14d40a3d20 msgr2=0x7f14d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14ef2a8640 1 --2- 172.21.15.43:0/290514522 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f14d40a3d20 0x7f14d40a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f14e002c630 tx=0x7f14e00095c0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14ef2a8640 1 -- 172.21.15.43:0/290514522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14e8107270 msgr2=0x7f14e8198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14ef2a8640 1 --2- 172.21.15.43:0/290514522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14e8107270 0x7f14e8198c10 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f14d800c970 tx=0x7f14d800ce30 comp rx=0 tx=0).stop 2024-04-21T08:41:09.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14ef2a8640 1 -- 172.21.15.43:0/290514522 shutdown_connections 2024-04-21T08:41:09.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14ef2a8640 1 --2- 172.21.15.43:0/290514522 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f14d40a3d20 0x7f14d40a61e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14ef2a8640 1 --2- 172.21.15.43:0/290514522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14e819cf90 0x7f14e819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14ef2a8640 1 --2- 172.21.15.43:0/290514522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14e8107270 0x7f14e8198c10 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14ef2a8640 1 --2- 172.21.15.43:0/290514522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14e806a840 0x7f14e81986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.817+0000 7f14ef2a8640 1 -- 172.21.15.43:0/290514522 >> 172.21.15.43:0/290514522 conn(0x7f14e8077eb0 msgr2=0x7f14e8100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.818+0000 7f14ef2a8640 1 -- 172.21.15.43:0/290514522 shutdown_connections 2024-04-21T08:41:09.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.818+0000 7f14ef2a8640 1 -- 172.21.15.43:0/290514522 wait complete. 2024-04-21T08:41:09.831 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:41:09.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.935+0000 7f74dbbef640 1 -- 172.21.15.43:0/1764870402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74d4103af0 msgr2=0x7f74d410bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.935+0000 7f74dbbef640 1 --2- 172.21.15.43:0/1764870402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74d4103af0 0x7f74d410bfd0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f74c4009900 tx=0x7f74c402bd60 comp rx=0 tx=0).stop 2024-04-21T08:41:09.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.936+0000 7f74dbbef640 1 -- 172.21.15.43:0/1764870402 shutdown_connections 2024-04-21T08:41:09.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.936+0000 7f74dbbef640 1 --2- 172.21.15.43:0/1764870402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74d410c510 0x7f74d410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.936+0000 7f74dbbef640 1 --2- 172.21.15.43:0/1764870402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74d4103af0 0x7f74d410bfd0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.936+0000 7f74dbbef640 1 --2- 172.21.15.43:0/1764870402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74d41031d0 0x7f74d41035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.936+0000 7f74dbbef640 1 -- 172.21.15.43:0/1764870402 >> 172.21.15.43:0/1764870402 conn(0x7f74d4077f50 msgr2=0x7f74d4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.936+0000 7f74dbbef640 1 -- 172.21.15.43:0/1764870402 shutdown_connections 2024-04-21T08:41:09.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.936+0000 7f74dbbef640 1 -- 172.21.15.43:0/1764870402 wait complete. 2024-04-21T08:41:09.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.936+0000 7f74dbbef640 1 Processor -- start 2024-04-21T08:41:09.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.936+0000 7f74dbbef640 1 -- start start 2024-04-21T08:41:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.936+0000 7f74dbbef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74d41031d0 0x7f74d4198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.936+0000 7f74dbbef640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74d4103af0 0x7f74d4198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.936+0000 7f74dbbef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74d410c510 0x7f74d419cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.936+0000 7f74dbbef640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f74d4112180 con 0x7f74d410c510 2024-04-21T08:41:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.936+0000 7f74dbbef640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f74d4112000 con 0x7f74d41031d0 2024-04-21T08:41:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.936+0000 7f74dbbef640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74d4112300 con 0x7f74d4103af0 2024-04-21T08:41:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.937+0000 7f74da18c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74d410c510 0x7f74d419cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.937+0000 7f74da18c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74d410c510 0x7f74d419cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55934/0 (socket says 172.21.15.43:55934) 2024-04-21T08:41:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.937+0000 7f74da18c640 1 -- 172.21.15.43:0/4155896076 learned_addr learned my addr 172.21.15.43:0/4155896076 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.937+0000 7f74d998b640 1 --2- 172.21.15.43:0/4155896076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74d41031d0 0x7f74d4198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.937+0000 7f74da18c640 1 -- 172.21.15.43:0/4155896076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74d4103af0 msgr2=0x7f74d4198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.937+0000 7f74da18c640 1 --2- 172.21.15.43:0/4155896076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74d4103af0 0x7f74d4198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.937+0000 7f74da18c640 1 -- 172.21.15.43:0/4155896076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74d41031d0 msgr2=0x7f74d4198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.937+0000 7f74da18c640 1 --2- 172.21.15.43:0/4155896076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74d41031d0 0x7f74d4198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.937+0000 7f74da18c640 1 -- 172.21.15.43:0/4155896076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d419d610 con 0x7f74d410c510 2024-04-21T08:41:09.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.937+0000 7f74da18c640 1 --2- 172.21.15.43:0/4155896076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74d410c510 0x7f74d419cee0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f74cc00cce0 tx=0x7f74cc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.937+0000 7f74caffd640 1 -- 172.21.15.43:0/4155896076 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc00f450 con 0x7f74d410c510 2024-04-21T08:41:09.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.937+0000 7f74caffd640 1 -- 172.21.15.43:0/4155896076 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74cc00faa0 con 0x7f74d410c510 2024-04-21T08:41:09.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.937+0000 7f74caffd640 1 -- 172.21.15.43:0/4155896076 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc018b70 con 0x7f74d410c510 2024-04-21T08:41:09.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.937+0000 7f74dbbef640 1 -- 172.21.15.43:0/4155896076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d419d900 con 0x7f74d410c510 2024-04-21T08:41:09.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.937+0000 7f74dbbef640 1 -- 172.21.15.43:0/4155896076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d41a5190 con 0x7f74d410c510 2024-04-21T08:41:09.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.939+0000 7f74caffd640 1 -- 172.21.15.43:0/4155896076 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f74cc018d10 con 0x7f74d410c510 2024-04-21T08:41:09.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.940+0000 7f74de7da740 1 Processor -- start 2024-04-21T08:41:09.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.940+0000 7f74de7da740 1 -- start start 2024-04-21T08:41:09.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.940+0000 7f74de7da740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1ec3b0 0x55daad1ec790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.940+0000 7f74de7da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55daad1ecd60 0x55daad1f5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.940+0000 7f74de7da740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55daad1f5e60 0x55daad1f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.940+0000 7f74de7da740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55daad1511f0 con 0x55daad1ec3b0 2024-04-21T08:41:09.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.940+0000 7f74de7da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55daad0ee4b0 con 0x55daad1ecd60 2024-04-21T08:41:09.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.940+0000 7f74de7da740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55daad14d1b0 con 0x55daad1f5e60 2024-04-21T08:41:09.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.940+0000 7f74caffd640 1 --2- 172.21.15.43:0/4155896076 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f74c00a3da0 0x7f74c00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.940+0000 7f74d918a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1ec3b0 0x55daad1ec790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.941+0000 7f74d918a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1ec3b0 0x55daad1ec790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55938/0 (socket says 172.21.15.43:55938) 2024-04-21T08:41:09.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.941+0000 7f74d918a640 1 -- 172.21.15.43:0/3594565926 learned_addr learned my addr 172.21.15.43:0/3594565926 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.941+0000 7f74da18c640 1 --2- 172.21.15.43:0/4155896076 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f74c00a3da0 0x7f74c00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.941+0000 7f74caffd640 1 -- 172.21.15.43:0/4155896076 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f74cc0d1050 con 0x7f74d410c510 2024-04-21T08:41:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.941+0000 7f74d998b640 1 --2- 172.21.15.43:0/3594565926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55daad1ecd60 0x55daad1f5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.941+0000 7f74d918a640 1 --2- 172.21.15.43:0/3594565926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55daad1f5e60 0x55daad1f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.941+0000 7f74d918a640 1 -- 172.21.15.43:0/3594565926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55daad1f5e60 msgr2=0x55daad1f62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.941+0000 7f74d918a640 1 --2- 172.21.15.43:0/3594565926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55daad1f5e60 0x55daad1f62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.941+0000 7f74d918a640 1 -- 172.21.15.43:0/3594565926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55daad1ecd60 msgr2=0x55daad1f5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.941+0000 7f74d918a640 1 --2- 172.21.15.43:0/3594565926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55daad1ecd60 0x55daad1f5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.941+0000 7f74d918a640 1 -- 172.21.15.43:0/3594565926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daad0e01b0 con 0x55daad1ec3b0 2024-04-21T08:41:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.941+0000 7f74d918a640 1 --2- 172.21.15.43:0/3594565926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1ec3b0 0x55daad1ec790 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f74c4036040 tx=0x7f74c4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ceed2eb92fa28bd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.941+0000 7f74da18c640 1 --2- 172.21.15.43:0/4155896076 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f74c00a3da0 0x7f74c00a6260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f74cc09cb50 tx=0x7f74cc00ff50 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.941+0000 7f74bffff640 1 -- 172.21.15.43:0/3594565926 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4047070 con 0x55daad1ec3b0 2024-04-21T08:41:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.941+0000 7f74bffff640 1 -- 172.21.15.43:0/3594565926 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c402cdc0 con 0x55daad1ec3b0 2024-04-21T08:41:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.941+0000 7f74bffff640 1 -- 172.21.15.43:0/3594565926 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4004920 con 0x55daad1ec3b0 2024-04-21T08:41:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.942+0000 7f74de7da740 1 -- 172.21.15.43:0/3594565926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1ec3b0 msgr2=0x55daad1ec790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.942+0000 7f74de7da740 1 --2- 172.21.15.43:0/3594565926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1ec3b0 0x55daad1ec790 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f74c4036040 tx=0x7f74c4005d50 comp rx=0 tx=0).stop 2024-04-21T08:41:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.942+0000 7f74de7da740 1 -- 172.21.15.43:0/3594565926 shutdown_connections 2024-04-21T08:41:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.942+0000 7f74de7da740 1 --2- 172.21.15.43:0/3594565926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55daad1f5e60 0x55daad1f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.942+0000 7f74de7da740 1 --2- 172.21.15.43:0/3594565926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55daad1ecd60 0x55daad1f5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.942+0000 7f74de7da740 1 --2- 172.21.15.43:0/3594565926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1ec3b0 0x55daad1ec790 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.942+0000 7f74de7da740 1 -- 172.21.15.43:0/3594565926 >> 172.21.15.43:0/3594565926 conn(0x55daad1e6ba0 msgr2=0x55daad1e6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.942+0000 7f74de7da740 1 -- 172.21.15.43:0/3594565926 shutdown_connections 2024-04-21T08:41:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.942+0000 7f74de7da740 1 -- 172.21.15.43:0/3594565926 wait complete. 2024-04-21T08:41:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.943+0000 7f74de7da740 1 -- start start 2024-04-21T08:41:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.943+0000 7f74de7da740 1 Processor -- start 2024-04-21T08:41:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.943+0000 7f74de7da740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1ec3b0 0x55daad1fc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.943+0000 7f74de7da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55daad1ecd60 0x55daad1f9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.943+0000 7f74de7da740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55daad1f5e60 0x55daad1f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.943+0000 7f74de7da740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55daad1fcd40 con 0x55daad1ec3b0 2024-04-21T08:41:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.943+0000 7f74de7da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55daad10a500 con 0x55daad1ecd60 2024-04-21T08:41:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.943+0000 7f74de7da740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55daad1fcec0 con 0x55daad1f5e60 2024-04-21T08:41:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.943+0000 7f74d918a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1ec3b0 0x55daad1fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.943+0000 7f74de7da740 10 client.0 _ll_register_callbacks cb 0x55daad0d8b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.943+0000 7f74d918a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1ec3b0 0x55daad1fc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55946/0 (socket says 172.21.15.43:55946) 2024-04-21T08:41:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.943+0000 7f74d918a640 1 -- 172.21.15.43:0/3386222676 learned_addr learned my addr 172.21.15.43:0/3386222676 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.943+0000 7f74d998b640 1 --2- 172.21.15.43:0/3386222676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55daad1ecd60 0x55daad1f9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.943+0000 7f74d918a640 1 -- 172.21.15.43:0/3386222676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55daad1f5e60 msgr2=0x55daad1f9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.943+0000 7f74d918a640 1 --2- 172.21.15.43:0/3386222676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55daad1f5e60 0x55daad1f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.943+0000 7f74d918a640 1 -- 172.21.15.43:0/3386222676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55daad1ecd60 msgr2=0x55daad1f9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.943+0000 7f74d918a640 1 --2- 172.21.15.43:0/3386222676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55daad1ecd60 0x55daad1f9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.943+0000 7f74d918a640 1 -- 172.21.15.43:0/3386222676 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daace633f0 con 0x55daad1ec3b0 2024-04-21T08:41:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.944+0000 7f74d918a640 1 --2- 172.21.15.43:0/3386222676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1ec3b0 0x55daad1fc290 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f74c4004650 tx=0x7f74c4009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa267a5adfbeab07 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.944+0000 7f74bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.944+0000 7f74bd7fa640 1 -- 172.21.15.43:0/3386222676 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4047040 con 0x55daad1ec3b0 2024-04-21T08:41:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.944+0000 7f74bd7fa640 1 -- 172.21.15.43:0/3386222676 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f74c40035f0 con 0x55daad1ec3b0 2024-04-21T08:41:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.944+0000 7f74de7da740 1 -- 172.21.15.43:0/3386222676 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74c402cdc0 con 0x55daad1ec3b0 2024-04-21T08:41:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.944+0000 7f74bd7fa640 1 -- 172.21.15.43:0/3386222676 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f74c402cbe0 con 0x55daad1ec3b0 2024-04-21T08:41:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.944+0000 7f74de7da740 10 client.16743 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.944+0000 7f74de7da740 1 -- 172.21.15.43:0/3386222676 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55daacf2d7d0 con 0x55daad1ec3b0 2024-04-21T08:41:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.944+0000 7f74bd7fa640 1 -- 172.21.15.43:0/3386222676 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c402cdc0 con 0x55daad1ec3b0 2024-04-21T08:41:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.944+0000 7f74bd7fa640 1 -- 172.21.15.43:0/3386222676 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f74c404a070 con 0x55daad1ec3b0 2024-04-21T08:41:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.944+0000 7f74de7da740 10 client.16743 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.944+0000 7f74de7da740 10 client.16743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.944+0000 7f74de7da740 20 client.16743 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.944+0000 7f74de7da740 1 --2- 172.21.15.43:0/3386222676 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55daad209e70 0x55daad20c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.944+0000 7f74de7da740 4 client.16743 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:09.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.944+0000 7f74de7da740 1 -- 172.21.15.43:0/3386222676 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55daad0f0930 con 0x55daad209e70 2024-04-21T08:41:09.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.945+0000 7f74d998b640 1 --2- 172.21.15.43:0/3386222676 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55daad209e70 0x55daad20c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.945+0000 7f74d998b640 1 --2- 172.21.15.43:0/3386222676 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55daad209e70 0x55daad20c260 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad3b89ee0acd67ae server_cookie=f2c4ba09d814b00c in_seq=0 out_seq=0 2024-04-21T08:41:09.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.945+0000 7f74bd7fa640 10 client.16743 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:09.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.946+0000 7f74bd7fa640 1 -- 172.21.15.43:0/3386222676 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55daad0f0930 con 0x55daad209e70 2024-04-21T08:41:09.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.946+0000 7f74bd7fa640 10 client.16743 handle_command_reply: tid=0 2024-04-21T08:41:09.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.946+0000 7f74de7da740 1 client.16743 shutdown 2024-04-21T08:41:09.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.946+0000 7f74de7da740 1 -- 172.21.15.43:0/3386222676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1ec3b0 msgr2=0x55daad1fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.946+0000 7f74de7da740 1 --2- 172.21.15.43:0/3386222676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1ec3b0 0x55daad1fc290 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f74c4004650 tx=0x7f74c4009560 comp rx=0 tx=0).stop 2024-04-21T08:41:09.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.946+0000 7f74de7da740 1 -- 172.21.15.43:0/3386222676 shutdown_connections 2024-04-21T08:41:09.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.946+0000 7f74de7da740 1 --2- 172.21.15.43:0/3386222676 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55daad209e70 0x55daad20c260 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.946+0000 7f74de7da740 1 --2- 172.21.15.43:0/3386222676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55daad1f5e60 0x55daad1f9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.946+0000 7f74de7da740 1 --2- 172.21.15.43:0/3386222676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55daad1ecd60 0x55daad1f9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.946+0000 7f74de7da740 1 --2- 172.21.15.43:0/3386222676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1ec3b0 0x55daad1fc290 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.946+0000 7f74bd7fa640 0 client.16743 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:09.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.946+0000 7f74de7da740 1 -- 172.21.15.43:0/3386222676 >> 172.21.15.43:0/3386222676 conn(0x55daad1e6ba0 msgr2=0x55daad1e88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.946+0000 7f74de7da740 1 -- 172.21.15.43:0/3386222676 shutdown_connections 2024-04-21T08:41:09.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.946+0000 7f74de7da740 1 -- 172.21.15.43:0/3386222676 wait complete. 2024-04-21T08:41:09.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.946+0000 7f74de7da740 20 client.16743 trim_cache size 0 max 16384 2024-04-21T08:41:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.959+0000 7f74de7da740 1 Processor -- start 2024-04-21T08:41:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.960+0000 7f74de7da740 1 -- start start 2024-04-21T08:41:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.960+0000 7f74de7da740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1ec3b0 0x55daad1f7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.960+0000 7f74de7da740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55daad1ecd60 0x55daad1fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.960+0000 7f74de7da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55daad1f5e60 0x55daad1fc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.960+0000 7f74de7da740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41ad0e0 con 0x55daad1ec3b0 2024-04-21T08:41:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.960+0000 7f74de7da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41aec20 con 0x55daad1f5e60 2024-04-21T08:41:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.960+0000 7f74de7da740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41ad7b0 con 0x55daad1ecd60 2024-04-21T08:41:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.960+0000 7f74d918a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1ec3b0 0x55daad1f7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.960+0000 7f74d918a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1ec3b0 0x55daad1f7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55956/0 (socket says 172.21.15.43:55956) 2024-04-21T08:41:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.960+0000 7f74d918a640 1 -- 172.21.15.43:0/1838872301 learned_addr learned my addr 172.21.15.43:0/1838872301 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.960+0000 7f74d998b640 1 --2- 172.21.15.43:0/1838872301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55daad1ecd60 0x55daad1fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.960+0000 7f74d918a640 1 -- 172.21.15.43:0/1838872301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55daad1ecd60 msgr2=0x55daad1fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.960+0000 7f74d918a640 1 --2- 172.21.15.43:0/1838872301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55daad1ecd60 0x55daad1fbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.960+0000 7f74d918a640 1 -- 172.21.15.43:0/1838872301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55daad1f5e60 msgr2=0x55daad1fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.960+0000 7f74d918a640 1 --2- 172.21.15.43:0/1838872301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55daad1f5e60 0x55daad1fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.960+0000 7f74d918a640 1 -- 172.21.15.43:0/1838872301 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daad1e66d0 con 0x55daad1ec3b0 2024-04-21T08:41:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.960+0000 7f74d998b640 1 --2- 172.21.15.43:0/1838872301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55daad1ecd60 0x55daad1fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.960+0000 7f74d918a640 1 --2- 172.21.15.43:0/1838872301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1ec3b0 0x55daad1f7250 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f74c4042890 tx=0x7f74c402ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f81197b8b6e588 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.960+0000 7f74bf7fe640 1 -- 172.21.15.43:0/1838872301 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4047040 con 0x55daad1ec3b0 2024-04-21T08:41:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.960+0000 7f74bf7fe640 1 -- 172.21.15.43:0/1838872301 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c4009780 con 0x55daad1ec3b0 2024-04-21T08:41:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.960+0000 7f74bf7fe640 1 -- 172.21.15.43:0/1838872301 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4034b50 con 0x55daad1ec3b0 2024-04-21T08:41:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.961+0000 7f74de7da740 1 -- 172.21.15.43:0/1838872301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1ec3b0 msgr2=0x55daad1f7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.961+0000 7f74de7da740 1 --2- 172.21.15.43:0/1838872301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1ec3b0 0x55daad1f7250 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f74c4042890 tx=0x7f74c402ca00 comp rx=0 tx=0).stop 2024-04-21T08:41:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.961+0000 7f74de7da740 1 -- 172.21.15.43:0/1838872301 shutdown_connections 2024-04-21T08:41:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.961+0000 7f74de7da740 1 --2- 172.21.15.43:0/1838872301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55daad1f5e60 0x55daad1fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.961+0000 7f74de7da740 1 --2- 172.21.15.43:0/1838872301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55daad1ecd60 0x55daad1fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.961+0000 7f74de7da740 1 --2- 172.21.15.43:0/1838872301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1ec3b0 0x55daad1f7250 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.961+0000 7f74de7da740 1 -- 172.21.15.43:0/1838872301 >> 172.21.15.43:0/1838872301 conn(0x55daad1e6ba0 msgr2=0x55daad2059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.961+0000 7f74de7da740 1 -- 172.21.15.43:0/1838872301 shutdown_connections 2024-04-21T08:41:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.961+0000 7f74de7da740 1 -- 172.21.15.43:0/1838872301 wait complete. 2024-04-21T08:41:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.961+0000 7f74de7da740 1 -- start start 2024-04-21T08:41:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.961+0000 7f74de7da740 1 Processor -- start 2024-04-21T08:41:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.962+0000 7f74de7da740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55daad1ec3b0 0x55daad20f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.962+0000 7f74de7da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55daad1ecd60 0x55daad20c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.962+0000 7f74de7da740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1f5e60 0x55daad20ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.962+0000 7f74de7da740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41ae550 con 0x55daad1f5e60 2024-04-21T08:41:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.962+0000 7f74de7da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41ade80 con 0x55daad1ecd60 2024-04-21T08:41:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.962+0000 7f74de7da740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41ab0c0 con 0x55daad1ec3b0 2024-04-21T08:41:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.962+0000 7f74de7da740 10 client.0 _ll_register_callbacks cb 0x7f74c4005690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.962+0000 7f74d918a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1f5e60 0x55daad20ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.962+0000 7f74d918a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1f5e60 0x55daad20ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55960/0 (socket says 172.21.15.43:55960) 2024-04-21T08:41:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.962+0000 7f74d998b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55daad1ecd60 0x55daad20c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.962+0000 7f74d918a640 1 -- 172.21.15.43:0/2589459750 learned_addr learned my addr 172.21.15.43:0/2589459750 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.962+0000 7f74d918a640 1 --2- 172.21.15.43:0/2589459750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55daad1ec3b0 0x55daad20f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.962+0000 7f74d918a640 1 -- 172.21.15.43:0/2589459750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55daad1ec3b0 msgr2=0x55daad20f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.962+0000 7f74d918a640 1 --2- 172.21.15.43:0/2589459750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55daad1ec3b0 0x55daad20f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.962+0000 7f74d918a640 1 -- 172.21.15.43:0/2589459750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55daad1ecd60 msgr2=0x55daad20c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.962+0000 7f74d918a640 1 --2- 172.21.15.43:0/2589459750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55daad1ecd60 0x55daad20c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.962+0000 7f74d918a640 1 -- 172.21.15.43:0/2589459750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daad20d660 con 0x55daad1f5e60 2024-04-21T08:41:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.962+0000 7f74d998b640 1 --2- 172.21.15.43:0/2589459750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55daad1ecd60 0x55daad20c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:41:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.963+0000 7f74d918a640 1 --2- 172.21.15.43:0/2589459750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1f5e60 0x55daad20ce60 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f74c4042e50 tx=0x7f74c4005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2a487d5353a6de server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.963+0000 7f74bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.963+0000 7f74bd7fa640 1 -- 172.21.15.43:0/2589459750 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4047040 con 0x55daad1f5e60 2024-04-21T08:41:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.963+0000 7f74bd7fa640 1 -- 172.21.15.43:0/2589459750 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f74c4060040 con 0x55daad1f5e60 2024-04-21T08:41:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.963+0000 7f74de7da740 1 -- 172.21.15.43:0/2589459750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74c4009780 con 0x55daad1f5e60 2024-04-21T08:41:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.963+0000 7f74bd7fa640 1 -- 172.21.15.43:0/2589459750 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f74c4034500 con 0x55daad1f5e60 2024-04-21T08:41:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.963+0000 7f74de7da740 10 client.16749 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.963+0000 7f74de7da740 1 -- 172.21.15.43:0/2589459750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55daad26ff00 con 0x55daad1f5e60 2024-04-21T08:41:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.963+0000 7f74bd7fa640 1 -- 172.21.15.43:0/2589459750 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c4009780 con 0x55daad1f5e60 2024-04-21T08:41:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.963+0000 7f74bd7fa640 1 -- 172.21.15.43:0/2589459750 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f74c4063030 con 0x55daad1f5e60 2024-04-21T08:41:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.963+0000 7f74de7da740 10 client.16749 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.963+0000 7f74de7da740 10 client.16749 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.963+0000 7f74de7da740 20 client.16749 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.963+0000 7f74de7da740 1 --2- 172.21.15.43:0/2589459750 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55daad270370 0x55daad272760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.963+0000 7f74de7da740 4 client.16749 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:41:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.963+0000 7f74de7da740 1 -- 172.21.15.43:0/2589459750 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55daad15ec10 con 0x55daad270370 2024-04-21T08:41:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.963+0000 7f74d998b640 1 --2- 172.21.15.43:0/2589459750 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55daad270370 0x55daad272760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.964+0000 7f74bd7fa640 10 client.16749 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.964+0000 7f74d998b640 1 --2- 172.21.15.43:0/2589459750 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55daad270370 0x55daad272760 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6396261c9ee94822 server_cookie=2535de925277b5ab in_seq=0 out_seq=0 2024-04-21T08:41:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.964+0000 7f74bd7fa640 1 -- 172.21.15.43:0/2589459750 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55daad15ec10 con 0x55daad270370 2024-04-21T08:41:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.964+0000 7f74bd7fa640 10 client.16749 handle_command_reply: tid=0 2024-04-21T08:41:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.964+0000 7f74de7da740 1 client.16749 shutdown 2024-04-21T08:41:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.964+0000 7f74de7da740 1 -- 172.21.15.43:0/2589459750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1f5e60 msgr2=0x55daad20ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.964+0000 7f74de7da740 1 --2- 172.21.15.43:0/2589459750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1f5e60 0x55daad20ce60 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f74c4042e50 tx=0x7f74c4005e50 comp rx=0 tx=0).stop 2024-04-21T08:41:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.964+0000 7f74de7da740 1 -- 172.21.15.43:0/2589459750 shutdown_connections 2024-04-21T08:41:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.964+0000 7f74de7da740 1 --2- 172.21.15.43:0/2589459750 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55daad270370 0x55daad272760 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.964+0000 7f74de7da740 1 --2- 172.21.15.43:0/2589459750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daad1f5e60 0x55daad20ce60 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.964+0000 7f74de7da740 1 --2- 172.21.15.43:0/2589459750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55daad1ecd60 0x55daad20c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.964+0000 7f74de7da740 1 --2- 172.21.15.43:0/2589459750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55daad1ec3b0 0x55daad20f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.964+0000 7f74bd7fa640 0 client.16749 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.964+0000 7f74de7da740 1 -- 172.21.15.43:0/2589459750 >> 172.21.15.43:0/2589459750 conn(0x55daad1e6ba0 msgr2=0x55daad1e5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.964+0000 7f74de7da740 1 -- 172.21.15.43:0/2589459750 shutdown_connections 2024-04-21T08:41:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.964+0000 7f74de7da740 1 -- 172.21.15.43:0/2589459750 wait complete. 2024-04-21T08:41:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.964+0000 7f74de7da740 20 client.16749 trim_cache size 0 max 16384 2024-04-21T08:41:09.972 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:41:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.965+0000 7f74dbbef640 1 -- 172.21.15.43:0/4155896076 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f74c00a3da0 msgr2=0x7f74c00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.965+0000 7f74dbbef640 1 --2- 172.21.15.43:0/4155896076 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f74c00a3da0 0x7f74c00a6260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f74cc09cb50 tx=0x7f74cc00ff50 comp rx=0 tx=0).stop 2024-04-21T08:41:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.965+0000 7f74dbbef640 1 -- 172.21.15.43:0/4155896076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74d410c510 msgr2=0x7f74d419cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.965+0000 7f74dbbef640 1 --2- 172.21.15.43:0/4155896076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74d410c510 0x7f74d419cee0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f74cc00cce0 tx=0x7f74cc00a620 comp rx=0 tx=0).stop 2024-04-21T08:41:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.965+0000 7f74dbbef640 1 -- 172.21.15.43:0/4155896076 shutdown_connections 2024-04-21T08:41:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.965+0000 7f74dbbef640 1 --2- 172.21.15.43:0/4155896076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74d410c510 0x7f74d419cee0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.965+0000 7f74dbbef640 1 --2- 172.21.15.43:0/4155896076 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f74c00a3da0 0x7f74c00a6260 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.965+0000 7f74dbbef640 1 --2- 172.21.15.43:0/4155896076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74d4103af0 0x7f74d4198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.965+0000 7f74dbbef640 1 --2- 172.21.15.43:0/4155896076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74d41031d0 0x7f74d4198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:09.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.965+0000 7f74dbbef640 1 -- 172.21.15.43:0/4155896076 >> 172.21.15.43:0/4155896076 conn(0x7f74d4077f50 msgr2=0x7f74d40fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:09.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.965+0000 7f74dbbef640 1 -- 172.21.15.43:0/4155896076 shutdown_connections 2024-04-21T08:41:09.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:09.965+0000 7f74dbbef640 1 -- 172.21.15.43:0/4155896076 wait complete. 2024-04-21T08:41:10.143 INFO:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-04-21T08:41:10.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:41:10.166 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:10 smithi043 ceph-mon[27978]: pgmap v70: 129 pgs: 129 active+clean; 2.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 43 KiB/s wr, 30 op/s 2024-04-21T08:41:10.166 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:10 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1988056097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:10.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.247+0000 7feb26eba640 1 -- 172.21.15.43:0/746216240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb200690e0 msgr2=0x7feb20069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:10.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.247+0000 7feb26eba640 1 --2- 172.21.15.43:0/746216240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb200690e0 0x7feb20069560 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7feb08009900 tx=0x7feb0802bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:10.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.247+0000 7feb26eba640 1 -- 172.21.15.43:0/746216240 shutdown_connections 2024-04-21T08:41:10.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.247+0000 7feb26eba640 1 --2- 172.21.15.43:0/746216240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb20069aa0 0x7feb2010ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.247+0000 7feb26eba640 1 --2- 172.21.15.43:0/746216240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb200690e0 0x7feb20069560 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.247+0000 7feb26eba640 1 --2- 172.21.15.43:0/746216240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb20106f50 0x7feb20107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.247+0000 7feb26eba640 1 -- 172.21.15.43:0/746216240 >> 172.21.15.43:0/746216240 conn(0x7feb20077e30 msgr2=0x7feb20078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:10.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.247+0000 7feb26eba640 1 -- 172.21.15.43:0/746216240 shutdown_connections 2024-04-21T08:41:10.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.247+0000 7feb26eba640 1 -- 172.21.15.43:0/746216240 wait complete. 2024-04-21T08:41:10.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.248+0000 7feb26eba640 1 Processor -- start 2024-04-21T08:41:10.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.248+0000 7feb26eba640 1 -- start start 2024-04-21T08:41:10.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.248+0000 7feb26eba640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb200690e0 0x7feb20101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.248+0000 7feb26eba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb20069aa0 0x7feb201017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.248+0000 7feb26eba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb20106f50 0x7feb20101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.248+0000 7feb26eba640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feb20112170 con 0x7feb20106f50 2024-04-21T08:41:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.248+0000 7feb26eba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb20111ff0 con 0x7feb20069aa0 2024-04-21T08:41:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.248+0000 7feb26eba640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feb201122f0 con 0x7feb200690e0 2024-04-21T08:41:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.248+0000 7feb25457640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb20106f50 0x7feb20101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.248+0000 7feb25457640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb20106f50 0x7feb20101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55970/0 (socket says 172.21.15.43:55970) 2024-04-21T08:41:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.248+0000 7feb25457640 1 -- 172.21.15.43:0/4120450557 learned_addr learned my addr 172.21.15.43:0/4120450557 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.248+0000 7feb1ffff640 1 --2- 172.21.15.43:0/4120450557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb20069aa0 0x7feb201017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.248+0000 7feb24c56640 1 --2- 172.21.15.43:0/4120450557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb200690e0 0x7feb20101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.248+0000 7feb25457640 1 -- 172.21.15.43:0/4120450557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb200690e0 msgr2=0x7feb20101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.248+0000 7feb25457640 1 --2- 172.21.15.43:0/4120450557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb200690e0 0x7feb20101270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.248+0000 7feb25457640 1 -- 172.21.15.43:0/4120450557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb20069aa0 msgr2=0x7feb201017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.248+0000 7feb25457640 1 --2- 172.21.15.43:0/4120450557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb20069aa0 0x7feb201017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.248+0000 7feb25457640 1 -- 172.21.15.43:0/4120450557 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb201a6ad0 con 0x7feb20106f50 2024-04-21T08:41:10.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.248+0000 7feb1ffff640 1 --2- 172.21.15.43:0/4120450557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb20069aa0 0x7feb201017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:10.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.249+0000 7feb25457640 1 --2- 172.21.15.43:0/4120450557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb20106f50 0x7feb20101cf0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7feb1000d750 tx=0x7feb1000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:10.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.249+0000 7feb1dffb640 1 -- 172.21.15.43:0/4120450557 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10017070 con 0x7feb20106f50 2024-04-21T08:41:10.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.249+0000 7feb1dffb640 1 -- 172.21.15.43:0/4120450557 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb100054d0 con 0x7feb20106f50 2024-04-21T08:41:10.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.249+0000 7feb26eba640 1 -- 172.21.15.43:0/4120450557 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb201a6dc0 con 0x7feb20106f50 2024-04-21T08:41:10.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.249+0000 7feb1dffb640 1 -- 172.21.15.43:0/4120450557 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10004c60 con 0x7feb20106f50 2024-04-21T08:41:10.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.249+0000 7feb26eba640 1 -- 172.21.15.43:0/4120450557 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb201a7330 con 0x7feb20106f50 2024-04-21T08:41:10.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.250+0000 7feb26eba640 1 -- 172.21.15.43:0/4120450557 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb20068de0 con 0x7feb20106f50 2024-04-21T08:41:10.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.251+0000 7feb1dffb640 1 -- 172.21.15.43:0/4120450557 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feb10009bd0 con 0x7feb20106f50 2024-04-21T08:41:10.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.253+0000 7feb1dffb640 1 --2- 172.21.15.43:0/4120450557 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7feb0c0a3da0 0x7feb0c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:10.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.253+0000 7feb24c56640 1 --2- 172.21.15.43:0/4120450557 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7feb0c0a3da0 0x7feb0c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:10.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.253+0000 7feb1dffb640 1 -- 172.21.15.43:0/4120450557 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7feb100d0600 con 0x7feb20106f50 2024-04-21T08:41:10.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.253+0000 7feb24c56640 1 --2- 172.21.15.43:0/4120450557 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7feb0c0a3da0 0x7feb0c0a6260 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feb18005fd0 tx=0x7feb18008230 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:10.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.256+0000 7feb1dffb640 1 -- 172.21.15.43:0/4120450557 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb1009c870 con 0x7feb20106f50 2024-04-21T08:41:10.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:10 smithi112 ceph-mon[31085]: pgmap v70: 129 pgs: 129 active+clean; 2.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 43 KiB/s wr, 30 op/s 2024-04-21T08:41:10.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:10 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1988056097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:10.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:10 smithi175 ceph-mon[31920]: pgmap v70: 129 pgs: 129 active+clean; 2.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 43 KiB/s wr, 30 op/s 2024-04-21T08:41:10.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:10 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1988056097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:10.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.485+0000 7feb26eba640 1 -- 172.21.15.43:0/4120450557 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb201a7560 con 0x7feb20106f50 2024-04-21T08:41:10.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.485+0000 7feb1dffb640 1 -- 172.21.15.43:0/4120450557 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7feb1009c170 con 0x7feb20106f50 2024-04-21T08:41:10.486 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:41:10.486 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:41:10.486 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:41:10.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.488+0000 7feb26eba640 1 -- 172.21.15.43:0/4120450557 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7feb0c0a3da0 msgr2=0x7feb0c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:10.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.488+0000 7feb26eba640 1 --2- 172.21.15.43:0/4120450557 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7feb0c0a3da0 0x7feb0c0a6260 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feb18005fd0 tx=0x7feb18008230 comp rx=0 tx=0).stop 2024-04-21T08:41:10.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.488+0000 7feb26eba640 1 -- 172.21.15.43:0/4120450557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb20106f50 msgr2=0x7feb20101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:10.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.488+0000 7feb26eba640 1 --2- 172.21.15.43:0/4120450557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb20106f50 0x7feb20101cf0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7feb1000d750 tx=0x7feb1000dc10 comp rx=0 tx=0).stop 2024-04-21T08:41:10.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.488+0000 7feb26eba640 1 -- 172.21.15.43:0/4120450557 shutdown_connections 2024-04-21T08:41:10.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.488+0000 7feb26eba640 1 --2- 172.21.15.43:0/4120450557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb20106f50 0x7feb20101cf0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.488+0000 7feb26eba640 1 --2- 172.21.15.43:0/4120450557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb20069aa0 0x7feb201017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.488+0000 7feb26eba640 1 --2- 172.21.15.43:0/4120450557 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7feb0c0a3da0 0x7feb0c0a6260 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.488+0000 7feb26eba640 1 --2- 172.21.15.43:0/4120450557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb200690e0 0x7feb20101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.488+0000 7feb26eba640 1 -- 172.21.15.43:0/4120450557 >> 172.21.15.43:0/4120450557 conn(0x7feb20077e30 msgr2=0x7feb200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:10.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.488+0000 7feb26eba640 1 -- 172.21.15.43:0/4120450557 shutdown_connections 2024-04-21T08:41:10.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.488+0000 7feb26eba640 1 -- 172.21.15.43:0/4120450557 wait complete. 2024-04-21T08:41:10.499 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db / --timeout 30 --expiration 84.5 --await' 2024-04-21T08:41:10.500 DEBUG:teuthology.orchestra.run.smithi043:> timeout 45 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24527 quiesce db / --timeout 30 --expiration 84.5 --await 2024-04-21T08:41:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.651+0000 7eff88f0e640 1 -- 172.21.15.43:0/2998361566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff841010a0 msgr2=0x7eff84101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.651+0000 7eff88f0e640 1 --2- 172.21.15.43:0/2998361566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff841010a0 0x7eff84101480 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7eff74009900 tx=0x7eff7402bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.651+0000 7eff88f0e640 1 -- 172.21.15.43:0/2998361566 shutdown_connections 2024-04-21T08:41:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.651+0000 7eff88f0e640 1 --2- 172.21.15.43:0/2998361566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff84106170 0x7eff8410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.651+0000 7eff88f0e640 1 --2- 172.21.15.43:0/2998361566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff84101a50 0x7eff84105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.651+0000 7eff88f0e640 1 --2- 172.21.15.43:0/2998361566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff841010a0 0x7eff84101480 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.651+0000 7eff88f0e640 1 -- 172.21.15.43:0/2998361566 >> 172.21.15.43:0/2998361566 conn(0x7eff84077f50 msgr2=0x7eff84078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.651+0000 7eff88f0e640 1 -- 172.21.15.43:0/2998361566 shutdown_connections 2024-04-21T08:41:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.651+0000 7eff88f0e640 1 -- 172.21.15.43:0/2998361566 wait complete. 2024-04-21T08:41:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.651+0000 7eff88f0e640 1 Processor -- start 2024-04-21T08:41:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.652+0000 7eff88f0e640 1 -- start start 2024-04-21T08:41:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.652+0000 7eff88f0e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff841010a0 0x7eff84199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.652+0000 7eff88f0e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff84106170 0x7eff84199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.652+0000 7eff88f0e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff8419a050 0x7eff841a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.652+0000 7eff88f0e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7eff841110d0 con 0x7eff8419a050 2024-04-21T08:41:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.652+0000 7eff88f0e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff84110f50 con 0x7eff84106170 2024-04-21T08:41:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.652+0000 7eff88f0e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7eff84111250 con 0x7eff841010a0 2024-04-21T08:41:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.652+0000 7eff82d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff8419a050 0x7eff841a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.652+0000 7eff82d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff8419a050 0x7eff841a45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55988/0 (socket says 172.21.15.43:55988) 2024-04-21T08:41:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.652+0000 7eff82d9d640 1 -- 172.21.15.43:0/1719177811 learned_addr learned my addr 172.21.15.43:0/1719177811 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:10.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.652+0000 7eff8259c640 1 --2- 172.21.15.43:0/1719177811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff841010a0 0x7eff84199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:10.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.652+0000 7eff81d9b640 1 --2- 172.21.15.43:0/1719177811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff84106170 0x7eff84199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:10.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.652+0000 7eff82d9d640 1 -- 172.21.15.43:0/1719177811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff841010a0 msgr2=0x7eff84199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:10.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.652+0000 7eff82d9d640 1 --2- 172.21.15.43:0/1719177811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff841010a0 0x7eff84199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.652+0000 7eff82d9d640 1 -- 172.21.15.43:0/1719177811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff84106170 msgr2=0x7eff84199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:10.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.652+0000 7eff82d9d640 1 --2- 172.21.15.43:0/1719177811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff84106170 0x7eff84199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.652+0000 7eff82d9d640 1 -- 172.21.15.43:0/1719177811 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff841a4dd0 con 0x7eff8419a050 2024-04-21T08:41:10.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.652+0000 7eff82d9d640 1 --2- 172.21.15.43:0/1719177811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff8419a050 0x7eff841a45b0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00eaa0 tx=0x7eff7c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:10.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.653+0000 7eff737fe640 1 -- 172.21.15.43:0/1719177811 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c00cdb0 con 0x7eff8419a050 2024-04-21T08:41:10.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.653+0000 7eff737fe640 1 -- 172.21.15.43:0/1719177811 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff7c005020 con 0x7eff8419a050 2024-04-21T08:41:10.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.653+0000 7eff88f0e640 1 -- 172.21.15.43:0/1719177811 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff841a50c0 con 0x7eff8419a050 2024-04-21T08:41:10.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.653+0000 7eff737fe640 1 -- 172.21.15.43:0/1719177811 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c0116d0 con 0x7eff8419a050 2024-04-21T08:41:10.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.653+0000 7eff88f0e640 1 -- 172.21.15.43:0/1719177811 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff841a55c0 con 0x7eff8419a050 2024-04-21T08:41:10.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.656+0000 7eff737fe640 1 -- 172.21.15.43:0/1719177811 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7eff7c0051d0 con 0x7eff8419a050 2024-04-21T08:41:10.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.656+0000 7eff8bbdc740 1 Processor -- start 2024-04-21T08:41:10.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.656+0000 7eff8bbdc740 1 -- start start 2024-04-21T08:41:10.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.656+0000 7eff8bbdc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557d5032c6f0 0x557d5032cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:10.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.656+0000 7eff8bbdc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d5032d010 0x557d50339d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.656+0000 7eff8bbdc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5033a2d0 0x557d5033c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.656+0000 7eff8bbdc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557d5028d0b0 con 0x557d5033a2d0 2024-04-21T08:41:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.656+0000 7eff8bbdc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557d5023e680 con 0x557d5032d010 2024-04-21T08:41:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.656+0000 7eff8bbdc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557d5024ac60 con 0x557d5032c6f0 2024-04-21T08:41:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.656+0000 7eff81d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5033a2d0 0x557d5033c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.656+0000 7eff8259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d5032d010 0x557d50339d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.656+0000 7eff81d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5033a2d0 0x557d5033c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55994/0 (socket says 172.21.15.43:55994) 2024-04-21T08:41:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.656+0000 7eff81d9b640 1 -- 172.21.15.43:0/3694464351 learned_addr learned my addr 172.21.15.43:0/3694464351 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.656+0000 7eff737fe640 1 --2- 172.21.15.43:0/1719177811 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7eff6c0a3d20 0x7eff6c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.656+0000 7eff81d9b640 1 --2- 172.21.15.43:0/3694464351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557d5032c6f0 0x557d5032cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.657+0000 7eff82d9d640 1 --2- 172.21.15.43:0/1719177811 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7eff6c0a3d20 0x7eff6c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.657+0000 7eff8259c640 1 -- 172.21.15.43:0/3694464351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557d5032c6f0 msgr2=0x557d5032cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.657+0000 7eff8259c640 1 --2- 172.21.15.43:0/3694464351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557d5032c6f0 0x557d5032cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.657+0000 7eff8259c640 1 -- 172.21.15.43:0/3694464351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5033a2d0 msgr2=0x557d5033c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.657+0000 7eff8259c640 1 --2- 172.21.15.43:0/3694464351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5033a2d0 0x557d5033c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.657+0000 7eff8259c640 1 -- 172.21.15.43:0/3694464351 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d5006fc90 con 0x557d5032d010 2024-04-21T08:41:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.657+0000 7eff737fe640 1 -- 172.21.15.43:0/1719177811 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7eff7c0d0cc0 con 0x7eff8419a050 2024-04-21T08:41:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.657+0000 7eff8259c640 1 --2- 172.21.15.43:0/3694464351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d5032d010 0x557d50339d90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7eff74000c00 tx=0x7eff7402c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=601a2c382c331b02 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.657+0000 7eff82d9d640 1 --2- 172.21.15.43:0/1719177811 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7eff6c0a3d20 0x7eff6c0a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7eff7c0ce0f0 tx=0x7eff7c063040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.657+0000 7eff70ff9640 1 -- 172.21.15.43:0/3694464351 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74045020 con 0x557d5032d010 2024-04-21T08:41:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.657+0000 7eff70ff9640 1 -- 172.21.15.43:0/3694464351 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff7402c7e0 con 0x557d5032d010 2024-04-21T08:41:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.657+0000 7eff70ff9640 1 -- 172.21.15.43:0/3694464351 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff740046c0 con 0x557d5032d010 2024-04-21T08:41:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.658+0000 7eff8bbdc740 1 -- 172.21.15.43:0/3694464351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d5032d010 msgr2=0x557d50339d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.658+0000 7eff8bbdc740 1 --2- 172.21.15.43:0/3694464351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d5032d010 0x557d50339d90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7eff74000c00 tx=0x7eff7402c6c0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.658+0000 7eff8bbdc740 1 -- 172.21.15.43:0/3694464351 shutdown_connections 2024-04-21T08:41:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.658+0000 7eff8bbdc740 1 --2- 172.21.15.43:0/3694464351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5033a2d0 0x557d5033c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.658+0000 7eff8bbdc740 1 --2- 172.21.15.43:0/3694464351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d5032d010 0x557d50339d90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.658+0000 7eff8bbdc740 1 --2- 172.21.15.43:0/3694464351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557d5032c6f0 0x557d5032cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.658+0000 7eff8bbdc740 1 -- 172.21.15.43:0/3694464351 >> 172.21.15.43:0/3694464351 conn(0x557d503270f0 msgr2=0x557d503274d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.658+0000 7eff8bbdc740 1 -- 172.21.15.43:0/3694464351 shutdown_connections 2024-04-21T08:41:10.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.658+0000 7eff8bbdc740 1 -- 172.21.15.43:0/3694464351 wait complete. 2024-04-21T08:41:10.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.658+0000 7eff8bbdc740 1 -- start start 2024-04-21T08:41:10.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.659+0000 7eff8bbdc740 1 Processor -- start 2024-04-21T08:41:10.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.659+0000 7eff8bbdc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d5032c6f0 0x557d50337330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:10.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.659+0000 7eff8bbdc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5032d010 0x557d50337870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:10.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.659+0000 7eff8bbdc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557d5033a2d0 0x557d50337db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:10.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.659+0000 7eff8bbdc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557d50290400 con 0x557d5032d010 2024-04-21T08:41:10.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.659+0000 7eff8bbdc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557d5024c840 con 0x557d5032c6f0 2024-04-21T08:41:10.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.659+0000 7eff8bbdc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557d50295720 con 0x557d5033a2d0 2024-04-21T08:41:10.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.659+0000 7eff8bbdc740 10 client.0 _ll_register_callbacks cb 0x557d5029dea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:10.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.659+0000 7eff8259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5032d010 0x557d50337870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:10.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.659+0000 7eff8259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5032d010 0x557d50337870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56006/0 (socket says 172.21.15.43:56006) 2024-04-21T08:41:10.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.659+0000 7eff8259c640 1 -- 172.21.15.43:0/3169120822 learned_addr learned my addr 172.21.15.43:0/3169120822 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.659+0000 7eff81d9b640 1 --2- 172.21.15.43:0/3169120822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d5032c6f0 0x557d50337330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.659+0000 7eff81d9b640 1 --2- 172.21.15.43:0/3169120822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557d5033a2d0 0x557d50337db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.659+0000 7eff8259c640 1 -- 172.21.15.43:0/3169120822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557d5033a2d0 msgr2=0x557d50337db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.659+0000 7eff8259c640 1 --2- 172.21.15.43:0/3169120822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557d5033a2d0 0x557d50337db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.659+0000 7eff8259c640 1 -- 172.21.15.43:0/3169120822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d5032c6f0 msgr2=0x557d50337330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.659+0000 7eff8259c640 1 --2- 172.21.15.43:0/3169120822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d5032c6f0 0x557d50337330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.659+0000 7eff8259c640 1 -- 172.21.15.43:0/3169120822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d500d0420 con 0x557d5032d010 2024-04-21T08:41:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.659+0000 7eff81d9b640 1 --2- 172.21.15.43:0/3169120822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557d5033a2d0 0x557d50337db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.659+0000 7eff8259c640 1 --2- 172.21.15.43:0/3169120822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5032d010 0x557d50337870 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7eff7402c170 tx=0x7eff7402cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be17fc90b9300518 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.659+0000 7eff69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.660+0000 7eff69ffb640 1 -- 172.21.15.43:0/3169120822 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff740035f0 con 0x557d5032d010 2024-04-21T08:41:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.660+0000 7eff69ffb640 1 -- 172.21.15.43:0/3169120822 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7eff74003790 con 0x557d5032d010 2024-04-21T08:41:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.660+0000 7eff8bbdc740 1 -- 172.21.15.43:0/3169120822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff7402c7e0 con 0x557d5032d010 2024-04-21T08:41:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.660+0000 7eff69ffb640 1 -- 172.21.15.43:0/3169120822 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7eff7405c050 con 0x557d5032d010 2024-04-21T08:41:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.660+0000 7eff8bbdc740 10 client.16767 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.660+0000 7eff8bbdc740 1 -- 172.21.15.43:0/3169120822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557d4ff63580 con 0x557d5032d010 2024-04-21T08:41:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.660+0000 7eff69ffb640 1 -- 172.21.15.43:0/3169120822 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff7402c7e0 con 0x557d5032d010 2024-04-21T08:41:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.660+0000 7eff69ffb640 1 -- 172.21.15.43:0/3169120822 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7eff74004c60 con 0x557d5032d010 2024-04-21T08:41:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.660+0000 7eff8bbdc740 10 client.16767 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.660+0000 7eff8bbdc740 10 client.16767 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-21T08:41:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.660+0000 7eff8bbdc740 20 client.16767 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.660+0000 7eff8bbdc740 1 --2- 172.21.15.43:0/3169120822 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557d50349e30 0x557d5034c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.660+0000 7eff8bbdc740 4 client.16767 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.660+0000 7eff8bbdc740 1 -- 172.21.15.43:0/3169120822 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d5029eb10 con 0x557d50349e30 2024-04-21T08:41:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.660+0000 7eff81d9b640 1 --2- 172.21.15.43:0/3169120822 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557d50349e30 0x557d5034c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.661+0000 7eff81d9b640 1 --2- 172.21.15.43:0/3169120822 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557d50349e30 0x557d5034c220 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62aa88f94964b06d server_cookie=a16c2dd704bcf7eb in_seq=0 out_seq=0 2024-04-21T08:41:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.661+0000 7eff69ffb640 10 client.16767 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.661+0000 7eff69ffb640 1 -- 172.21.15.43:0/3169120822 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557d5029eb10 con 0x557d50349e30 2024-04-21T08:41:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.661+0000 7eff69ffb640 10 client.16767 handle_command_reply: tid=0 2024-04-21T08:41:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.662+0000 7eff8bbdc740 1 client.16767 shutdown 2024-04-21T08:41:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.662+0000 7eff8bbdc740 1 -- 172.21.15.43:0/3169120822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5032d010 msgr2=0x557d50337870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.662+0000 7eff8bbdc740 1 --2- 172.21.15.43:0/3169120822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5032d010 0x557d50337870 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7eff7402c170 tx=0x7eff7402cbc0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.662+0000 7eff8bbdc740 1 -- 172.21.15.43:0/3169120822 shutdown_connections 2024-04-21T08:41:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.662+0000 7eff8bbdc740 1 --2- 172.21.15.43:0/3169120822 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557d50349e30 0x557d5034c220 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.662+0000 7eff8bbdc740 1 --2- 172.21.15.43:0/3169120822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557d5033a2d0 0x557d50337db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.662+0000 7eff8bbdc740 1 --2- 172.21.15.43:0/3169120822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5032d010 0x557d50337870 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.662+0000 7eff8bbdc740 1 --2- 172.21.15.43:0/3169120822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d5032c6f0 0x557d50337330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.662+0000 7eff69ffb640 0 client.16767 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.662+0000 7eff8bbdc740 1 -- 172.21.15.43:0/3169120822 >> 172.21.15.43:0/3169120822 conn(0x557d503270f0 msgr2=0x557d50328e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.662+0000 7eff8bbdc740 1 -- 172.21.15.43:0/3169120822 shutdown_connections 2024-04-21T08:41:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.662+0000 7eff8bbdc740 1 -- 172.21.15.43:0/3169120822 wait complete. 2024-04-21T08:41:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.662+0000 7eff8bbdc740 20 client.16767 trim_cache size 0 max 16384 2024-04-21T08:41:10.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff8bbdc740 1 Processor -- start 2024-04-21T08:41:10.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff8bbdc740 1 -- start start 2024-04-21T08:41:10.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff8bbdc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557d5032c6f0 0x557d50345c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:10.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff8bbdc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5032d010 0x557d50326050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:10.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff8bbdc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d5033a2d0 0x557d50326590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:10.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff8bbdc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7eff841aeca0 con 0x557d5032d010 2024-04-21T08:41:10.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff8bbdc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff841ad830 con 0x557d5033a2d0 2024-04-21T08:41:10.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff8bbdc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7eff841adf00 con 0x557d5032c6f0 2024-04-21T08:41:10.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff8259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5032d010 0x557d50326050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:10.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff8259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5032d010 0x557d50326050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56018/0 (socket says 172.21.15.43:56018) 2024-04-21T08:41:10.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff8259c640 1 -- 172.21.15.43:0/1631823996 learned_addr learned my addr 172.21.15.43:0/1631823996 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:10.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff81d9b640 1 --2- 172.21.15.43:0/1631823996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d5033a2d0 0x557d50326590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:10.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff8259c640 1 -- 172.21.15.43:0/1631823996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557d5032c6f0 msgr2=0x557d50345c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:10.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff8259c640 1 --2- 172.21.15.43:0/1631823996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557d5032c6f0 0x557d50345c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff8259c640 1 -- 172.21.15.43:0/1631823996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d5033a2d0 msgr2=0x557d50326590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:10.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff8259c640 1 --2- 172.21.15.43:0/1631823996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d5033a2d0 0x557d50326590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff8259c640 1 -- 172.21.15.43:0/1631823996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d5032c320 con 0x557d5032d010 2024-04-21T08:41:10.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff81d9b640 1 --2- 172.21.15.43:0/1631823996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d5033a2d0 0x557d50326590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:10.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff8259c640 1 --2- 172.21.15.43:0/1631823996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5032d010 0x557d50326050 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7eff74009590 tx=0x7eff740056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4031c44a06105bb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:10.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff6bfff640 1 -- 172.21.15.43:0/1631823996 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7404e020 con 0x557d5032d010 2024-04-21T08:41:10.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff6bfff640 1 -- 172.21.15.43:0/1631823996 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff7402c7e0 con 0x557d5032d010 2024-04-21T08:41:10.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff6bfff640 1 -- 172.21.15.43:0/1631823996 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74045040 con 0x557d5032d010 2024-04-21T08:41:10.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff8bbdc740 1 -- 172.21.15.43:0/1631823996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5032d010 msgr2=0x557d50326050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:10.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff8bbdc740 1 --2- 172.21.15.43:0/1631823996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5032d010 0x557d50326050 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7eff74009590 tx=0x7eff740056e0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff8bbdc740 1 -- 172.21.15.43:0/1631823996 shutdown_connections 2024-04-21T08:41:10.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff8bbdc740 1 --2- 172.21.15.43:0/1631823996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d5033a2d0 0x557d50326590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff8bbdc740 1 --2- 172.21.15.43:0/1631823996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5032d010 0x557d50326050 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff8bbdc740 1 --2- 172.21.15.43:0/1631823996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557d5032c6f0 0x557d50345c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.680+0000 7eff8bbdc740 1 -- 172.21.15.43:0/1631823996 >> 172.21.15.43:0/1631823996 conn(0x557d503270f0 msgr2=0x557d50325940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:10.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.681+0000 7eff8bbdc740 1 -- 172.21.15.43:0/1631823996 shutdown_connections 2024-04-21T08:41:10.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.681+0000 7eff8bbdc740 1 -- 172.21.15.43:0/1631823996 wait complete. 2024-04-21T08:41:10.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.681+0000 7eff8bbdc740 1 -- start start 2024-04-21T08:41:10.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.681+0000 7eff8bbdc740 1 Processor -- start 2024-04-21T08:41:10.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.681+0000 7eff8bbdc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5032c6f0 0x557d50342330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:10.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.681+0000 7eff8bbdc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557d5032d010 0x557d5033f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:10.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.681+0000 7eff8bbdc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d5033a2d0 0x557d5033f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:10.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.681+0000 7eff8bbdc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7eff841ab140 con 0x557d5032c6f0 2024-04-21T08:41:10.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.681+0000 7eff8bbdc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff841ae5d0 con 0x557d5033a2d0 2024-04-21T08:41:10.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.681+0000 7eff8bbdc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7eff841ab7b0 con 0x557d5032d010 2024-04-21T08:41:10.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.681+0000 7eff8bbdc740 10 client.0 _ll_register_callbacks cb 0x7eff7800bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:10.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.681+0000 7eff81d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5032c6f0 0x557d50342330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:10.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.681+0000 7eff81d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5032c6f0 0x557d50342330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56024/0 (socket says 172.21.15.43:56024) 2024-04-21T08:41:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.681+0000 7eff81d9b640 1 -- 172.21.15.43:0/3936477873 learned_addr learned my addr 172.21.15.43:0/3936477873 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.681+0000 7eff8259c640 1 --2- 172.21.15.43:0/3936477873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557d5032d010 0x557d5033f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.681+0000 7eff81d9b640 1 -- 172.21.15.43:0/3936477873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557d5032d010 msgr2=0x557d5033f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.681+0000 7eff81d9b640 1 --2- 172.21.15.43:0/3936477873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557d5032d010 0x557d5033f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.681+0000 7eff81d9b640 1 -- 172.21.15.43:0/3936477873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d5033a2d0 msgr2=0x557d5033f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.681+0000 7eff81d9b640 1 --2- 172.21.15.43:0/3936477873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d5033a2d0 0x557d5033f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.681+0000 7eff81d9b640 1 -- 172.21.15.43:0/3936477873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d5033fc90 con 0x557d5032c6f0 2024-04-21T08:41:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.682+0000 7eff81d9b640 1 --2- 172.21.15.43:0/3936477873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5032c6f0 0x557d50342330 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x557d50328280 tx=0x7eff78005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf961d81d2b95a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.682+0000 7eff69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.682+0000 7eff69ffb640 1 -- 172.21.15.43:0/3936477873 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff78004b80 con 0x557d5032c6f0 2024-04-21T08:41:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.682+0000 7eff69ffb640 1 -- 172.21.15.43:0/3936477873 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7eff78004d20 con 0x557d5032c6f0 2024-04-21T08:41:10.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.682+0000 7eff8bbdc740 1 -- 172.21.15.43:0/3936477873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff7402c7e0 con 0x557d5032c6f0 2024-04-21T08:41:10.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.682+0000 7eff69ffb640 1 -- 172.21.15.43:0/3936477873 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7eff780230e0 con 0x557d5032c6f0 2024-04-21T08:41:10.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.682+0000 7eff8bbdc740 10 client.16773 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:10.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.682+0000 7eff8bbdc740 1 -- 172.21.15.43:0/3936477873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557d503b0eb0 con 0x557d5032c6f0 2024-04-21T08:41:10.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.682+0000 7eff69ffb640 1 -- 172.21.15.43:0/3936477873 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff7402c7e0 con 0x557d5032c6f0 2024-04-21T08:41:10.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.682+0000 7eff69ffb640 1 -- 172.21.15.43:0/3936477873 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7eff78012960 con 0x557d5032c6f0 2024-04-21T08:41:10.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.682+0000 7eff8bbdc740 10 client.16773 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:10.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.682+0000 7eff8bbdc740 10 client.16773 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-21T08:41:10.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.682+0000 7eff8bbdc740 20 client.16773 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:10.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.682+0000 7eff8bbdc740 1 --2- 172.21.15.43:0/3936477873 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557d503b16a0 0x557d503b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:10.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.682+0000 7eff8bbdc740 4 client.16773 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 30.0, "expiration": 84.5, "await": true}] 2024-04-21T08:41:10.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.682+0000 7eff8bbdc740 1 -- 172.21.15.43:0/3936477873 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 30.0, "expiration": 84.5, "await": true}) -- 0x557d50298980 con 0x557d503b16a0 2024-04-21T08:41:10.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.683+0000 7eff8259c640 1 --2- 172.21.15.43:0/3936477873 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557d503b16a0 0x557d503b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:10.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.683+0000 7eff69ffb640 10 client.16773 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:10.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.683+0000 7eff8259c640 1 --2- 172.21.15.43:0/3936477873 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557d503b16a0 0x557d503b3ac0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eeb2de448789983 server_cookie=6507c7a9af6d235f in_seq=0 out_seq=0 2024-04-21T08:41:10.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff69ffb640 1 -- 172.21.15.43:0/3936477873 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x557d50298980 con 0x557d503b16a0 2024-04-21T08:41:10.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff69ffb640 10 client.16773 handle_command_reply: tid=0 2024-04-21T08:41:10.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff8bbdc740 1 client.16773 shutdown 2024-04-21T08:41:10.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff8bbdc740 1 -- 172.21.15.43:0/3936477873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5032c6f0 msgr2=0x557d50342330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:10.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff8bbdc740 1 --2- 172.21.15.43:0/3936477873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5032c6f0 0x557d50342330 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x557d50328280 tx=0x7eff78005410 comp rx=0 tx=0).stop 2024-04-21T08:41:10.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff8bbdc740 1 -- 172.21.15.43:0/3936477873 shutdown_connections 2024-04-21T08:41:10.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff8bbdc740 1 --2- 172.21.15.43:0/3936477873 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557d503b16a0 0x557d503b3ac0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff8bbdc740 1 --2- 172.21.15.43:0/3936477873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d5033a2d0 0x557d5033f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff8bbdc740 1 --2- 172.21.15.43:0/3936477873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557d5032d010 0x557d5033f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff69ffb640 0 client.16773 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:10.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff8bbdc740 1 --2- 172.21.15.43:0/3936477873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557d5032c6f0 0x557d50342330 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff8bbdc740 1 -- 172.21.15.43:0/3936477873 >> 172.21.15.43:0/3936477873 conn(0x557d503270f0 msgr2=0x557d503261d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:10.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff8bbdc740 1 -- 172.21.15.43:0/3936477873 shutdown_connections 2024-04-21T08:41:10.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff8bbdc740 1 -- 172.21.15.43:0/3936477873 wait complete. 2024-04-21T08:41:10.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff8bbdc740 20 client.16773 trim_cache size 0 max 16384 2024-04-21T08:41:10.700 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:41:10.700 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 28, 2024-04-21T08:41:10.700 INFO:teuthology.orchestra.run.smithi043.stdout: "leader": 24527, 2024-04-21T08:41:10.700 INFO:teuthology.orchestra.run.smithi043.stdout: "set_version": 2, 2024-04-21T08:41:10.700 INFO:teuthology.orchestra.run.smithi043.stdout: "sets": { 2024-04-21T08:41:10.700 INFO:teuthology.orchestra.run.smithi043.stdout: "52e10dcd": { 2024-04-21T08:41:10.700 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 2, 2024-04-21T08:41:10.700 INFO:teuthology.orchestra.run.smithi043.stdout: "age_ref": 0.0, 2024-04-21T08:41:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-04-21T08:41:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "QUIESCED", 2024-04-21T08:41:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-04-21T08:41:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:41:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: "timeout": 30.0, 2024-04-21T08:41:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: "expiration": 84.5, 2024-04-21T08:41:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: "members": { 2024-04-21T08:41:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: "file:/": { 2024-04-21T08:41:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: "excluded": false, 2024-04-21T08:41:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-04-21T08:41:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "QUIESCED", 2024-04-21T08:41:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-04-21T08:41:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:41:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:41:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:41:10.702 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:41:10.702 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:41:10.702 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:41:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff88f0e640 1 -- 172.21.15.43:0/1719177811 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7eff6c0a3d20 msgr2=0x7eff6c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff88f0e640 1 --2- 172.21.15.43:0/1719177811 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7eff6c0a3d20 0x7eff6c0a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7eff7c0ce0f0 tx=0x7eff7c063040 comp rx=0 tx=0).stop 2024-04-21T08:41:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff88f0e640 1 -- 172.21.15.43:0/1719177811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff8419a050 msgr2=0x7eff841a45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff88f0e640 1 --2- 172.21.15.43:0/1719177811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff8419a050 0x7eff841a45b0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00eaa0 tx=0x7eff7c00ef60 comp rx=0 tx=0).stop 2024-04-21T08:41:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff88f0e640 1 -- 172.21.15.43:0/1719177811 shutdown_connections 2024-04-21T08:41:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff88f0e640 1 --2- 172.21.15.43:0/1719177811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff8419a050 0x7eff841a45b0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff88f0e640 1 --2- 172.21.15.43:0/1719177811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff84106170 0x7eff84199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff88f0e640 1 --2- 172.21.15.43:0/1719177811 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7eff6c0a3d20 0x7eff6c0a61e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff88f0e640 1 --2- 172.21.15.43:0/1719177811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff841010a0 0x7eff84199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff88f0e640 1 -- 172.21.15.43:0/1719177811 >> 172.21.15.43:0/1719177811 conn(0x7eff84077f50 msgr2=0x7eff840ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff88f0e640 1 -- 172.21.15.43:0/1719177811 shutdown_connections 2024-04-21T08:41:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:10.699+0000 7eff88f0e640 1 -- 172.21.15.43:0/1719177811 wait complete. 2024-04-21T08:41:10.710 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced, set_id: 52e10dcd, quiesce duration: 24.5 2024-04-21T08:41:10.976 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:41:10.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:41:10.977 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:41:11.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.087+0000 7ff2fdd85640 1 -- 172.21.15.43:0/3217484183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2f8106170 msgr2=0x7ff2f810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:11.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.087+0000 7ff2fdd85640 1 --2- 172.21.15.43:0/3217484183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2f8106170 0x7ff2f810dd40 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009ca0 tx=0x7ff2f002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.087+0000 7ff2fdd85640 1 -- 172.21.15.43:0/3217484183 shutdown_connections 2024-04-21T08:41:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.087+0000 7ff2fdd85640 1 --2- 172.21.15.43:0/3217484183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2f8106170 0x7ff2f810dd40 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.087+0000 7ff2fdd85640 1 --2- 172.21.15.43:0/3217484183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2f8101a50 0x7ff2f8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.087+0000 7ff2fdd85640 1 --2- 172.21.15.43:0/3217484183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2f81010a0 0x7ff2f8101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.087+0000 7ff2fdd85640 1 -- 172.21.15.43:0/3217484183 >> 172.21.15.43:0/3217484183 conn(0x7ff2f8077eb0 msgr2=0x7ff2f80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.087+0000 7ff2fdd85640 1 -- 172.21.15.43:0/3217484183 shutdown_connections 2024-04-21T08:41:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.087+0000 7ff2fdd85640 1 -- 172.21.15.43:0/3217484183 wait complete. 2024-04-21T08:41:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.088+0000 7ff2fdd85640 1 Processor -- start 2024-04-21T08:41:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.088+0000 7ff2fdd85640 1 -- start start 2024-04-21T08:41:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.088+0000 7ff2fdd85640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2f81010a0 0x7ff2f81a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.088+0000 7ff2fdd85640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2f8106170 0x7ff2f81a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.088+0000 7ff2fdd85640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2f819a6f0 0x7ff2f819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.088+0000 7ff2fdd85640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f81110c0 con 0x7ff2f81010a0 2024-04-21T08:41:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.088+0000 7ff2fdd85640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f8110f40 con 0x7ff2f819a6f0 2024-04-21T08:41:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.088+0000 7ff2fdd85640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f8111240 con 0x7ff2f8106170 2024-04-21T08:41:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.088+0000 7ff2f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2f81010a0 0x7ff2f81a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.088+0000 7ff2f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2f81010a0 0x7ff2f81a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56052/0 (socket says 172.21.15.43:56052) 2024-04-21T08:41:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.088+0000 7ff2f77fe640 1 -- 172.21.15.43:0/2618869879 learned_addr learned my addr 172.21.15.43:0/2618869879 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.088+0000 7ff2f7fff640 1 --2- 172.21.15.43:0/2618869879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2f819a6f0 0x7ff2f819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.088+0000 7ff2f6ffd640 1 --2- 172.21.15.43:0/2618869879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2f8106170 0x7ff2f81a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.088+0000 7ff2f77fe640 1 -- 172.21.15.43:0/2618869879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2f8106170 msgr2=0x7ff2f81a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.088+0000 7ff2f77fe640 1 --2- 172.21.15.43:0/2618869879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2f8106170 0x7ff2f81a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.088+0000 7ff2f77fe640 1 -- 172.21.15.43:0/2618869879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2f819a6f0 msgr2=0x7ff2f819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.088+0000 7ff2f77fe640 1 --2- 172.21.15.43:0/2618869879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2f819a6f0 0x7ff2f819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.088+0000 7ff2f77fe640 1 -- 172.21.15.43:0/2618869879 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f819b2f0 con 0x7ff2f81010a0 2024-04-21T08:41:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.088+0000 7ff2f7fff640 1 --2- 172.21.15.43:0/2618869879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2f819a6f0 0x7ff2f819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.089+0000 7ff2f77fe640 1 --2- 172.21.15.43:0/2618869879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2f81010a0 0x7ff2f81a0630 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800c890 tx=0x7ff2e800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:11.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.089+0000 7ff2f4ff9640 1 -- 172.21.15.43:0/2618869879 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8004220 con 0x7ff2f81010a0 2024-04-21T08:41:11.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.089+0000 7ff2f4ff9640 1 -- 172.21.15.43:0/2618869879 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2e80043c0 con 0x7ff2f81010a0 2024-04-21T08:41:11.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.089+0000 7ff2fdd85640 1 -- 172.21.15.43:0/2618869879 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f81a73d0 con 0x7ff2f81010a0 2024-04-21T08:41:11.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.089+0000 7ff2f4ff9640 1 -- 172.21.15.43:0/2618869879 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8004a00 con 0x7ff2f81010a0 2024-04-21T08:41:11.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.089+0000 7ff2fdd85640 1 -- 172.21.15.43:0/2618869879 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f81a7830 con 0x7ff2f81010a0 2024-04-21T08:41:11.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.091+0000 7ff2f4ff9640 1 -- 172.21.15.43:0/2618869879 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff2e80079b0 con 0x7ff2f81010a0 2024-04-21T08:41:11.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.091+0000 7ff300971740 1 Processor -- start 2024-04-21T08:41:11.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.092+0000 7ff300971740 1 -- start start 2024-04-21T08:41:11.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.092+0000 7ff300971740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d8218e0 0x55c88d821cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.092+0000 7ff300971740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c88d822290 0x55c88d82a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.092+0000 7ff300971740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c88d82ae10 0x55c88d82d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.092+0000 7ff300971740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c88d785f00 con 0x55c88d8218e0 2024-04-21T08:41:11.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.092+0000 7ff300971740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c88d723590 con 0x55c88d822290 2024-04-21T08:41:11.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.092+0000 7ff300971740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c88d781ec0 con 0x55c88d82ae10 2024-04-21T08:41:11.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.092+0000 7ff2f4ff9640 1 --2- 172.21.15.43:0/2618869879 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff2e00a3da0 0x7ff2e00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.092+0000 7ff2f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d8218e0 0x55c88d821cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.092+0000 7ff2f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d8218e0 0x55c88d821cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56060/0 (socket says 172.21.15.43:56060) 2024-04-21T08:41:11.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.092+0000 7ff2f7fff640 1 -- 172.21.15.43:0/579724408 learned_addr learned my addr 172.21.15.43:0/579724408 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:11.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.092+0000 7ff2f6ffd640 1 --2- 172.21.15.43:0/579724408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c88d82ae10 0x55c88d82d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.092+0000 7ff2f77fe640 1 --2- 172.21.15.43:0/579724408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c88d822290 0x55c88d82a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.092+0000 7ff2f7fff640 1 --2- 172.21.15.43:0/2618869879 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff2e00a3da0 0x7ff2e00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.092+0000 7ff2f4ff9640 1 -- 172.21.15.43:0/2618869879 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7ff2e80d09b0 con 0x7ff2f81010a0 2024-04-21T08:41:11.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.092+0000 7ff2f7fff640 1 -- 172.21.15.43:0/579724408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c88d82ae10 msgr2=0x55c88d82d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:11.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.092+0000 7ff2f7fff640 1 --2- 172.21.15.43:0/579724408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c88d82ae10 0x55c88d82d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.092+0000 7ff2f7fff640 1 -- 172.21.15.43:0/579724408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c88d822290 msgr2=0x55c88d82a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:11.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.092+0000 7ff2f7fff640 1 --2- 172.21.15.43:0/579724408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c88d822290 0x55c88d82a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.092+0000 7ff2f7fff640 1 -- 172.21.15.43:0/579724408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c88d5607e0 con 0x55c88d8218e0 2024-04-21T08:41:11.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.092+0000 7ff2f77fe640 1 --2- 172.21.15.43:0/579724408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c88d822290 0x55c88d82a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:11.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.092+0000 7ff2f7fff640 1 --2- 172.21.15.43:0/579724408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d8218e0 0x55c88d821cc0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0036000 tx=0x7ff2f002cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53957768a5505a0a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:11.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.093+0000 7ff2ddffb640 1 -- 172.21.15.43:0/579724408 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0039040 con 0x55c88d8218e0 2024-04-21T08:41:11.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.093+0000 7ff2f7fff640 1 --2- 172.21.15.43:0/2618869879 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff2e00a3da0 0x7ff2e00a6260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0005fe0 tx=0x7ff2f0046000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:11.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.093+0000 7ff2ddffb640 1 -- 172.21.15.43:0/579724408 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2f002cca0 con 0x55c88d8218e0 2024-04-21T08:41:11.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.093+0000 7ff2ddffb640 1 -- 172.21.15.43:0/579724408 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0034440 con 0x55c88d8218e0 2024-04-21T08:41:11.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.093+0000 7ff300971740 1 -- 172.21.15.43:0/579724408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d8218e0 msgr2=0x55c88d821cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:11.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.093+0000 7ff300971740 1 --2- 172.21.15.43:0/579724408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d8218e0 0x55c88d821cc0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0036000 tx=0x7ff2f002cf90 comp rx=0 tx=0).stop 2024-04-21T08:41:11.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.093+0000 7ff300971740 1 -- 172.21.15.43:0/579724408 shutdown_connections 2024-04-21T08:41:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.093+0000 7ff300971740 1 --2- 172.21.15.43:0/579724408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c88d82ae10 0x55c88d82d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.093+0000 7ff300971740 1 --2- 172.21.15.43:0/579724408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c88d822290 0x55c88d82a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.093+0000 7ff300971740 1 --2- 172.21.15.43:0/579724408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d8218e0 0x55c88d821cc0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.093+0000 7ff300971740 1 -- 172.21.15.43:0/579724408 >> 172.21.15.43:0/579724408 conn(0x55c88d81bfb0 msgr2=0x55c88d81c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.093+0000 7ff300971740 1 -- 172.21.15.43:0/579724408 shutdown_connections 2024-04-21T08:41:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.094+0000 7ff300971740 1 -- 172.21.15.43:0/579724408 wait complete. 2024-04-21T08:41:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.094+0000 7ff300971740 1 -- start start 2024-04-21T08:41:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.094+0000 7ff300971740 1 Processor -- start 2024-04-21T08:41:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.094+0000 7ff300971740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d8218e0 0x55c88d827d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.094+0000 7ff300971740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c88d822290 0x55c88d8282b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.094+0000 7ff300971740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c88d82ae10 0x55c88d8287f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.094+0000 7ff300971740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c88d82dcc0 con 0x55c88d8218e0 2024-04-21T08:41:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.094+0000 7ff300971740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c88d73f580 con 0x55c88d822290 2024-04-21T08:41:11.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.094+0000 7ff300971740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c88d82de40 con 0x55c88d82ae10 2024-04-21T08:41:11.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.094+0000 7ff300971740 10 client.0 _ll_register_callbacks cb 0x55c88d745650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:11.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.094+0000 7ff2f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d8218e0 0x55c88d827d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.094+0000 7ff2f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d8218e0 0x55c88d827d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56066/0 (socket says 172.21.15.43:56066) 2024-04-21T08:41:11.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.094+0000 7ff2f77fe640 1 -- 172.21.15.43:0/1149143588 learned_addr learned my addr 172.21.15.43:0/1149143588 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:11.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.095+0000 7ff2f7fff640 1 --2- 172.21.15.43:0/1149143588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c88d82ae10 0x55c88d8287f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.095+0000 7ff2f6ffd640 1 --2- 172.21.15.43:0/1149143588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c88d822290 0x55c88d8282b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.095+0000 7ff2f77fe640 1 -- 172.21.15.43:0/1149143588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c88d82ae10 msgr2=0x55c88d8287f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:11.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.095+0000 7ff2f77fe640 1 --2- 172.21.15.43:0/1149143588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c88d82ae10 0x55c88d8287f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.095+0000 7ff2f77fe640 1 -- 172.21.15.43:0/1149143588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c88d822290 msgr2=0x55c88d8282b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:11.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.095+0000 7ff2f77fe640 1 --2- 172.21.15.43:0/1149143588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c88d822290 0x55c88d8282b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.095+0000 7ff2f77fe640 1 -- 172.21.15.43:0/1149143588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c88d5c0f70 con 0x55c88d8218e0 2024-04-21T08:41:11.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.095+0000 7ff2f6ffd640 1 --2- 172.21.15.43:0/1149143588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c88d822290 0x55c88d8282b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:11.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.095+0000 7ff2f7fff640 1 --2- 172.21.15.43:0/1149143588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c88d82ae10 0x55c88d8287f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:11.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.095+0000 7ff2f77fe640 1 --2- 172.21.15.43:0/1149143588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d8218e0 0x55c88d827d70 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8098980 tx=0x7ff2e801ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d7f5a67121f33c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:11.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.095+0000 7ff2d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:11.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.095+0000 7ff2d6ffd640 1 -- 172.21.15.43:0/1149143588 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8005290 con 0x55c88d8218e0 2024-04-21T08:41:11.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.095+0000 7ff2d6ffd640 1 -- 172.21.15.43:0/1149143588 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7ff2e80cdf50 con 0x55c88d8218e0 2024-04-21T08:41:11.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.095+0000 7ff300971740 1 -- 172.21.15.43:0/1149143588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2f002cca0 con 0x55c88d8218e0 2024-04-21T08:41:11.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.095+0000 7ff2d6ffd640 1 -- 172.21.15.43:0/1149143588 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff2e80ce150 con 0x55c88d8218e0 2024-04-21T08:41:11.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.095+0000 7ff300971740 10 client.16785 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:11.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.095+0000 7ff300971740 1 -- 172.21.15.43:0/1149143588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c88d458170 con 0x55c88d8218e0 2024-04-21T08:41:11.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.095+0000 7ff2d6ffd640 1 -- 172.21.15.43:0/1149143588 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2f002cca0 con 0x55c88d8218e0 2024-04-21T08:41:11.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.095+0000 7ff2d6ffd640 1 -- 172.21.15.43:0/1149143588 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff2e801cbb0 con 0x55c88d8218e0 2024-04-21T08:41:11.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.096+0000 7ff300971740 10 client.16785 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:11.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.096+0000 7ff300971740 10 client.16785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:11.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.096+0000 7ff300971740 20 client.16785 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:11.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.096+0000 7ff300971740 1 --2- 172.21.15.43:0/1149143588 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c88d83b250 0x55c88d83d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.096+0000 7ff300971740 4 client.16785 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:11.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.096+0000 7ff300971740 1 -- 172.21.15.43:0/1149143588 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c88d793920 con 0x55c88d83b250 2024-04-21T08:41:11.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.096+0000 7ff2f6ffd640 1 --2- 172.21.15.43:0/1149143588 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c88d83b250 0x55c88d83d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.096+0000 7ff2d6ffd640 10 client.16785 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:11.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.096+0000 7ff2f6ffd640 1 --2- 172.21.15.43:0/1149143588 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c88d83b250 0x55c88d83d6c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76a45a6765c6da5c server_cookie=660aa0176efffe35 in_seq=0 out_seq=0 2024-04-21T08:41:11.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.097+0000 7ff2d6ffd640 1 -- 172.21.15.43:0/1149143588 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c88d793920 con 0x55c88d83b250 2024-04-21T08:41:11.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.097+0000 7ff2d6ffd640 10 client.16785 handle_command_reply: tid=0 2024-04-21T08:41:11.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.097+0000 7ff300971740 1 client.16785 shutdown 2024-04-21T08:41:11.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.097+0000 7ff300971740 1 -- 172.21.15.43:0/1149143588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d8218e0 msgr2=0x55c88d827d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:11.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.097+0000 7ff300971740 1 --2- 172.21.15.43:0/1149143588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d8218e0 0x55c88d827d70 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8098980 tx=0x7ff2e801ec90 comp rx=0 tx=0).stop 2024-04-21T08:41:11.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.097+0000 7ff300971740 1 -- 172.21.15.43:0/1149143588 shutdown_connections 2024-04-21T08:41:11.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.097+0000 7ff300971740 1 --2- 172.21.15.43:0/1149143588 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c88d83b250 0x55c88d83d6c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.097+0000 7ff300971740 1 --2- 172.21.15.43:0/1149143588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c88d82ae10 0x55c88d8287f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.097+0000 7ff300971740 1 --2- 172.21.15.43:0/1149143588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c88d822290 0x55c88d8282b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.097+0000 7ff2d6ffd640 0 client.16785 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:11.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.097+0000 7ff300971740 1 --2- 172.21.15.43:0/1149143588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d8218e0 0x55c88d827d70 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.097+0000 7ff300971740 1 -- 172.21.15.43:0/1149143588 >> 172.21.15.43:0/1149143588 conn(0x55c88d81bfb0 msgr2=0x55c88d81dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:11.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.097+0000 7ff300971740 1 -- 172.21.15.43:0/1149143588 shutdown_connections 2024-04-21T08:41:11.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.097+0000 7ff300971740 1 -- 172.21.15.43:0/1149143588 wait complete. 2024-04-21T08:41:11.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.097+0000 7ff300971740 20 client.16785 trim_cache size 0 max 16384 2024-04-21T08:41:11.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.115+0000 7ff300971740 1 Processor -- start 2024-04-21T08:41:11.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.115+0000 7ff300971740 1 -- start start 2024-04-21T08:41:11.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.115+0000 7ff300971740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c88d8218e0 0x55c88d82a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.115+0000 7ff300971740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c88d822290 0x55c88d836b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.115+0000 7ff300971740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d82ae10 0x55c88d828440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.115+0000 7ff300971740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f81b16c0 con 0x55c88d82ae10 2024-04-21T08:41:11.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.115+0000 7ff300971740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f81b3200 con 0x55c88d822290 2024-04-21T08:41:11.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.115+0000 7ff300971740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f81b1d90 con 0x55c88d8218e0 2024-04-21T08:41:11.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.115+0000 7ff2f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d82ae10 0x55c88d828440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.115+0000 7ff2f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d82ae10 0x55c88d828440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56080/0 (socket says 172.21.15.43:56080) 2024-04-21T08:41:11.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.115+0000 7ff2f7fff640 1 -- 172.21.15.43:0/1032127915 learned_addr learned my addr 172.21.15.43:0/1032127915 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:11.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.115+0000 7ff2f77fe640 1 --2- 172.21.15.43:0/1032127915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c88d8218e0 0x55c88d82a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.115+0000 7ff2f7fff640 1 -- 172.21.15.43:0/1032127915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c88d8218e0 msgr2=0x55c88d82a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:11.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.115+0000 7ff2f6ffd640 1 --2- 172.21.15.43:0/1032127915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c88d822290 0x55c88d836b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.115+0000 7ff2f7fff640 1 --2- 172.21.15.43:0/1032127915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c88d8218e0 0x55c88d82a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.116+0000 7ff2f7fff640 1 -- 172.21.15.43:0/1032127915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c88d822290 msgr2=0x55c88d836b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:11.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.116+0000 7ff2f7fff640 1 --2- 172.21.15.43:0/1032127915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c88d822290 0x55c88d836b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.116+0000 7ff2f7fff640 1 -- 172.21.15.43:0/1032127915 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c88d8262c0 con 0x55c88d82ae10 2024-04-21T08:41:11.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.116+0000 7ff2f77fe640 1 --2- 172.21.15.43:0/1032127915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c88d8218e0 0x55c88d82a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:11.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.116+0000 7ff2f6ffd640 1 --2- 172.21.15.43:0/1032127915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c88d822290 0x55c88d836b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:11.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.116+0000 7ff2f7fff640 1 --2- 172.21.15.43:0/1032127915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d82ae10 0x55c88d828440 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff2f004d040 tx=0x7ff2f0032f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9942282d27b14e67 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:11.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.116+0000 7ff2dd7fa640 1 -- 172.21.15.43:0/1032127915 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0039040 con 0x55c88d82ae10 2024-04-21T08:41:11.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.116+0000 7ff2dd7fa640 1 -- 172.21.15.43:0/1032127915 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2f002cca0 con 0x55c88d82ae10 2024-04-21T08:41:11.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.116+0000 7ff2dd7fa640 1 -- 172.21.15.43:0/1032127915 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0034c30 con 0x55c88d82ae10 2024-04-21T08:41:11.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.116+0000 7ff300971740 1 -- 172.21.15.43:0/1032127915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d82ae10 msgr2=0x55c88d828440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:11.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.116+0000 7ff300971740 1 --2- 172.21.15.43:0/1032127915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d82ae10 0x55c88d828440 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff2f004d040 tx=0x7ff2f0032f50 comp rx=0 tx=0).stop 2024-04-21T08:41:11.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.117+0000 7ff300971740 1 -- 172.21.15.43:0/1032127915 shutdown_connections 2024-04-21T08:41:11.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.117+0000 7ff300971740 1 --2- 172.21.15.43:0/1032127915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d82ae10 0x55c88d828440 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.117+0000 7ff300971740 1 --2- 172.21.15.43:0/1032127915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c88d822290 0x55c88d836b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.117+0000 7ff300971740 1 --2- 172.21.15.43:0/1032127915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c88d8218e0 0x55c88d82a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.117+0000 7ff300971740 1 -- 172.21.15.43:0/1032127915 >> 172.21.15.43:0/1032127915 conn(0x55c88d81bfb0 msgr2=0x55c88d827d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:11.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.117+0000 7ff300971740 1 -- 172.21.15.43:0/1032127915 shutdown_connections 2024-04-21T08:41:11.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.117+0000 7ff300971740 1 -- 172.21.15.43:0/1032127915 wait complete. 2024-04-21T08:41:11.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.117+0000 7ff300971740 1 -- start start 2024-04-21T08:41:11.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.117+0000 7ff300971740 1 Processor -- start 2024-04-21T08:41:11.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.117+0000 7ff300971740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c88d8218e0 0x55c88d82f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.117+0000 7ff300971740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c88d822290 0x55c88d82f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.117+0000 7ff300971740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d82ae10 0x55c88d89a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.117+0000 7ff300971740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f81b2b30 con 0x55c88d82ae10 2024-04-21T08:41:11.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.117+0000 7ff300971740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f81b2460 con 0x55c88d822290 2024-04-21T08:41:11.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.117+0000 7ff300971740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f81af6a0 con 0x55c88d8218e0 2024-04-21T08:41:11.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.117+0000 7ff300971740 10 client.0 _ll_register_callbacks cb 0x7ff2e8098800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:11.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.117+0000 7ff2f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d82ae10 0x55c88d89a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.117+0000 7ff2f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d82ae10 0x55c88d89a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56088/0 (socket says 172.21.15.43:56088) 2024-04-21T08:41:11.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.117+0000 7ff2f7fff640 1 -- 172.21.15.43:0/2012608111 learned_addr learned my addr 172.21.15.43:0/2012608111 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:11.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.118+0000 7ff2f6ffd640 1 --2- 172.21.15.43:0/2012608111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c88d822290 0x55c88d82f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.118+0000 7ff2f77fe640 1 --2- 172.21.15.43:0/2012608111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c88d8218e0 0x55c88d82f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.118+0000 7ff2f7fff640 1 -- 172.21.15.43:0/2012608111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c88d8218e0 msgr2=0x55c88d82f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.118+0000 7ff2f7fff640 1 --2- 172.21.15.43:0/2012608111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c88d8218e0 0x55c88d82f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.118+0000 7ff2f7fff640 1 -- 172.21.15.43:0/2012608111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c88d822290 msgr2=0x55c88d82f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.118+0000 7ff2f7fff640 1 --2- 172.21.15.43:0/2012608111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c88d822290 0x55c88d82f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.118+0000 7ff2f7fff640 1 -- 172.21.15.43:0/2012608111 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c88d89adc0 con 0x55c88d82ae10 2024-04-21T08:41:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.118+0000 7ff2f7fff640 1 --2- 172.21.15.43:0/2012608111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d82ae10 0x55c88d89a7d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7ff2f002c630 tx=0x7ff2f0003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809dce1cee187935 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.118+0000 7ff2d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.118+0000 7ff2d6ffd640 1 -- 172.21.15.43:0/2012608111 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0039040 con 0x55c88d82ae10 2024-04-21T08:41:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.118+0000 7ff2d6ffd640 1 -- 172.21.15.43:0/2012608111 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7ff2f0034970 con 0x55c88d82ae10 2024-04-21T08:41:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.118+0000 7ff300971740 1 -- 172.21.15.43:0/2012608111 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2f002cca0 con 0x55c88d82ae10 2024-04-21T08:41:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.118+0000 7ff2d6ffd640 1 -- 172.21.15.43:0/2012608111 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff2f0034b70 con 0x55c88d82ae10 2024-04-21T08:41:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.118+0000 7ff300971740 10 client.16791 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.118+0000 7ff300971740 1 -- 172.21.15.43:0/2012608111 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c88d8a20f0 con 0x55c88d82ae10 2024-04-21T08:41:11.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.118+0000 7ff2d6ffd640 1 -- 172.21.15.43:0/2012608111 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2f002cca0 con 0x55c88d82ae10 2024-04-21T08:41:11.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.119+0000 7ff2d6ffd640 1 -- 172.21.15.43:0/2012608111 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff2f0055680 con 0x55c88d82ae10 2024-04-21T08:41:11.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.119+0000 7ff300971740 10 client.16791 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:11.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.119+0000 7ff300971740 10 client.16791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:11.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.119+0000 7ff300971740 20 client.16791 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:11.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.119+0000 7ff300971740 1 --2- 172.21.15.43:0/2012608111 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c88d8a2370 0x55c88d8a4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.119+0000 7ff300971740 4 client.16791 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:41:11.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.119+0000 7ff300971740 1 -- 172.21.15.43:0/2012608111 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c88d78d790 con 0x55c88d8a2370 2024-04-21T08:41:11.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.119+0000 7ff2f77fe640 1 --2- 172.21.15.43:0/2012608111 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c88d8a2370 0x55c88d8a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.119+0000 7ff2d6ffd640 10 client.16791 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:11.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.119+0000 7ff2f77fe640 1 --2- 172.21.15.43:0/2012608111 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c88d8a2370 0x55c88d8a4760 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dcbcebc67b49834 server_cookie=2d2f2a9d32715a07 in_seq=0 out_seq=0 2024-04-21T08:41:11.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.120+0000 7ff2d6ffd640 1 -- 172.21.15.43:0/2012608111 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c88d78d790 con 0x55c88d8a2370 2024-04-21T08:41:11.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.120+0000 7ff2d6ffd640 10 client.16791 handle_command_reply: tid=0 2024-04-21T08:41:11.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.120+0000 7ff300971740 1 client.16791 shutdown 2024-04-21T08:41:11.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.121+0000 7ff300971740 1 -- 172.21.15.43:0/2012608111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d82ae10 msgr2=0x55c88d89a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:11.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.121+0000 7ff300971740 1 --2- 172.21.15.43:0/2012608111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d82ae10 0x55c88d89a7d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7ff2f002c630 tx=0x7ff2f0003ef0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.121+0000 7ff300971740 1 -- 172.21.15.43:0/2012608111 shutdown_connections 2024-04-21T08:41:11.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.121+0000 7ff300971740 1 --2- 172.21.15.43:0/2012608111 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c88d8a2370 0x55c88d8a4760 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.121+0000 7ff300971740 1 --2- 172.21.15.43:0/2012608111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c88d82ae10 0x55c88d89a7d0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.121+0000 7ff300971740 1 --2- 172.21.15.43:0/2012608111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c88d822290 0x55c88d82f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.121+0000 7ff300971740 1 --2- 172.21.15.43:0/2012608111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c88d8218e0 0x55c88d82f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.121+0000 7ff300971740 1 -- 172.21.15.43:0/2012608111 >> 172.21.15.43:0/2012608111 conn(0x55c88d81bfb0 msgr2=0x55c88d81db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:11.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.121+0000 7ff2d6ffd640 0 client.16791 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:11.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.121+0000 7ff300971740 1 -- 172.21.15.43:0/2012608111 shutdown_connections 2024-04-21T08:41:11.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.121+0000 7ff300971740 1 -- 172.21.15.43:0/2012608111 wait complete. 2024-04-21T08:41:11.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.121+0000 7ff300971740 20 client.16791 trim_cache size 0 max 16384 2024-04-21T08:41:11.128 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:41:11.128 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:41:11.128 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "d13cfb66-432c-4d9e-a40e-f50a5ce32d56", 2024-04-21T08:41:11.128 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:41:11.128 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:41:11.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.121+0000 7ff2fdd85640 1 -- 172.21.15.43:0/2618869879 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff2e00a3da0 msgr2=0x7ff2e00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:11.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.121+0000 7ff2fdd85640 1 --2- 172.21.15.43:0/2618869879 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff2e00a3da0 0x7ff2e00a6260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0005fe0 tx=0x7ff2f0046000 comp rx=0 tx=0).stop 2024-04-21T08:41:11.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.121+0000 7ff2fdd85640 1 -- 172.21.15.43:0/2618869879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2f81010a0 msgr2=0x7ff2f81a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:11.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.121+0000 7ff2fdd85640 1 --2- 172.21.15.43:0/2618869879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2f81010a0 0x7ff2f81a0630 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800c890 tx=0x7ff2e800cd50 comp rx=0 tx=0).stop 2024-04-21T08:41:11.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.121+0000 7ff2fdd85640 1 -- 172.21.15.43:0/2618869879 shutdown_connections 2024-04-21T08:41:11.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.121+0000 7ff2fdd85640 1 --2- 172.21.15.43:0/2618869879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2f819a6f0 0x7ff2f819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.121+0000 7ff2fdd85640 1 --2- 172.21.15.43:0/2618869879 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff2e00a3da0 0x7ff2e00a6260 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.121+0000 7ff2fdd85640 1 --2- 172.21.15.43:0/2618869879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2f8106170 0x7ff2f81a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.121+0000 7ff2fdd85640 1 --2- 172.21.15.43:0/2618869879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2f81010a0 0x7ff2f81a0630 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.121+0000 7ff2fdd85640 1 -- 172.21.15.43:0/2618869879 >> 172.21.15.43:0/2618869879 conn(0x7ff2f8077eb0 msgr2=0x7ff2f810c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:11.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.122+0000 7ff2fdd85640 1 -- 172.21.15.43:0/2618869879 shutdown_connections 2024-04-21T08:41:11.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.122+0000 7ff2fdd85640 1 -- 172.21.15.43:0/2618869879 wait complete. 2024-04-21T08:41:11.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:41:11.133 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:41:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.237+0000 7fa919c55640 1 -- 172.21.15.43:0/3408720209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9141021a0 msgr2=0x7fa914102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.237+0000 7fa919c55640 1 --2- 172.21.15.43:0/3408720209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9141021a0 0x7fa914102580 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa908009900 tx=0x7fa90802bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.237+0000 7fa919c55640 1 -- 172.21.15.43:0/3408720209 shutdown_connections 2024-04-21T08:41:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.237+0000 7fa919c55640 1 --2- 172.21.15.43:0/3408720209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa914107270 0x7fa91410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.237+0000 7fa919c55640 1 --2- 172.21.15.43:0/3408720209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa914102b50 0x7fa914106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.237+0000 7fa919c55640 1 --2- 172.21.15.43:0/3408720209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9141021a0 0x7fa914102580 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.237+0000 7fa919c55640 1 -- 172.21.15.43:0/3408720209 >> 172.21.15.43:0/3408720209 conn(0x7fa9140fda40 msgr2=0x7fa9140ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.237+0000 7fa919c55640 1 -- 172.21.15.43:0/3408720209 shutdown_connections 2024-04-21T08:41:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.237+0000 7fa919c55640 1 -- 172.21.15.43:0/3408720209 wait complete. 2024-04-21T08:41:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.238+0000 7fa919c55640 1 Processor -- start 2024-04-21T08:41:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.238+0000 7fa919c55640 1 -- start start 2024-04-21T08:41:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.238+0000 7fa919c55640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9141021a0 0x7fa914198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.238+0000 7fa919c55640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa914107270 0x7fa914198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.238+0000 7fa919c55640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa91419d020 0x7fa91419d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.238+0000 7fa919c55640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa9141121e0 con 0x7fa91419d020 2024-04-21T08:41:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.238+0000 7fa919c55640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa914112060 con 0x7fa9141021a0 2024-04-21T08:41:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.238+0000 7fa919c55640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa914112360 con 0x7fa914107270 2024-04-21T08:41:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.239+0000 7fa913fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa91419d020 0x7fa91419d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.239+0000 7fa913fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa91419d020 0x7fa91419d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56098/0 (socket says 172.21.15.43:56098) 2024-04-21T08:41:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.239+0000 7fa913fff640 1 -- 172.21.15.43:0/1609159799 learned_addr learned my addr 172.21.15.43:0/1609159799 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.239+0000 7fa9137fe640 1 --2- 172.21.15.43:0/1609159799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9141021a0 0x7fa914198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.239+0000 7fa912ffd640 1 --2- 172.21.15.43:0/1609159799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa914107270 0x7fa914198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.239+0000 7fa913fff640 1 -- 172.21.15.43:0/1609159799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa914107270 msgr2=0x7fa914198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.239+0000 7fa913fff640 1 --2- 172.21.15.43:0/1609159799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa914107270 0x7fa914198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.239+0000 7fa913fff640 1 -- 172.21.15.43:0/1609159799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9141021a0 msgr2=0x7fa914198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.239+0000 7fa913fff640 1 --2- 172.21.15.43:0/1609159799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9141021a0 0x7fa914198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.239+0000 7fa913fff640 1 -- 172.21.15.43:0/1609159799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91419db10 con 0x7fa91419d020 2024-04-21T08:41:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.239+0000 7fa913fff640 1 --2- 172.21.15.43:0/1609159799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa91419d020 0x7fa91419d4d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00eaa0 tx=0x7fa90c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.239+0000 7fa910ff9640 1 -- 172.21.15.43:0/1609159799 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c00cdb0 con 0x7fa91419d020 2024-04-21T08:41:11.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.239+0000 7fa910ff9640 1 -- 172.21.15.43:0/1609159799 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa90c005020 con 0x7fa91419d020 2024-04-21T08:41:11.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.239+0000 7fa919c55640 1 -- 172.21.15.43:0/1609159799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91419de00 con 0x7fa91419d020 2024-04-21T08:41:11.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.239+0000 7fa910ff9640 1 -- 172.21.15.43:0/1609159799 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c0116d0 con 0x7fa91419d020 2024-04-21T08:41:11.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.239+0000 7fa919c55640 1 -- 172.21.15.43:0/1609159799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9141a55f0 con 0x7fa91419d020 2024-04-21T08:41:11.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.241+0000 7fa910ff9640 1 -- 172.21.15.43:0/1609159799 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa90c0051d0 con 0x7fa91419d020 2024-04-21T08:41:11.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.242+0000 7fa91c849740 1 Processor -- start 2024-04-21T08:41:11.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.242+0000 7fa91c849740 1 -- start start 2024-04-21T08:41:11.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.242+0000 7fa91c849740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f46e272b0 0x563f46e27690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.242+0000 7fa91c849740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e27bd0 0x563f46e34950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.242+0000 7fa91c849740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f46e34e90 0x563f46e37280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.242+0000 7fa91c849740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563f46d8c1e0 con 0x563f46e27bd0 2024-04-21T08:41:11.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.242+0000 7fa91c849740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563f46d29390 con 0x563f46e34e90 2024-04-21T08:41:11.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.242+0000 7fa91c849740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563f46d881a0 con 0x563f46e272b0 2024-04-21T08:41:11.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.242+0000 7fa910ff9640 1 --2- 172.21.15.43:0/1609159799 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa8fc0a3d20 0x7fa8fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.242+0000 7fa9137fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e27bd0 0x563f46e34950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.242+0000 7fa9137fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e27bd0 0x563f46e34950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56106/0 (socket says 172.21.15.43:56106) 2024-04-21T08:41:11.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.242+0000 7fa9137fe640 1 -- 172.21.15.43:0/3651720435 learned_addr learned my addr 172.21.15.43:0/3651720435 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:11.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.242+0000 7fa912ffd640 1 --2- 172.21.15.43:0/3651720435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f46e34e90 0x563f46e37280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.242+0000 7fa913fff640 1 --2- 172.21.15.43:0/1609159799 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa8fc0a3d20 0x7fa8fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.242+0000 7fa912ffd640 1 --2- 172.21.15.43:0/3651720435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f46e272b0 0x563f46e27690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.242+0000 7fa910ff9640 1 -- 172.21.15.43:0/1609159799 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fa90c0d0d10 con 0x7fa91419d020 2024-04-21T08:41:11.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.242+0000 7fa9137fe640 1 -- 172.21.15.43:0/3651720435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f46e272b0 msgr2=0x563f46e27690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:11.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.242+0000 7fa9137fe640 1 --2- 172.21.15.43:0/3651720435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f46e272b0 0x563f46e27690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.242+0000 7fa9137fe640 1 -- 172.21.15.43:0/3651720435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f46e34e90 msgr2=0x563f46e37280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:11.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.242+0000 7fa9137fe640 1 --2- 172.21.15.43:0/3651720435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f46e34e90 0x563f46e37280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.242+0000 7fa9137fe640 1 -- 172.21.15.43:0/3651720435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f46b687d0 con 0x563f46e27bd0 2024-04-21T08:41:11.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.242+0000 7fa912ffd640 1 --2- 172.21.15.43:0/3651720435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f46e272b0 0x563f46e27690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:11.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.243+0000 7fa9137fe640 1 --2- 172.21.15.43:0/3651720435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e27bd0 0x563f46e34950 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fa908009900 tx=0x7fa90802c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8945e56072cb581 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:11.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.243+0000 7fa913fff640 1 --2- 172.21.15.43:0/1609159799 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa8fc0a3d20 0x7fa8fc0a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0ce140 tx=0x7fa90c087040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:11.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.243+0000 7fa8f9ffb640 1 -- 172.21.15.43:0/3651720435 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908045020 con 0x563f46e27bd0 2024-04-21T08:41:11.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.243+0000 7fa8f9ffb640 1 -- 172.21.15.43:0/3651720435 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa90802c7e0 con 0x563f46e27bd0 2024-04-21T08:41:11.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.243+0000 7fa8f9ffb640 1 -- 172.21.15.43:0/3651720435 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9080046c0 con 0x563f46e27bd0 2024-04-21T08:41:11.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.243+0000 7fa91c849740 1 -- 172.21.15.43:0/3651720435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e27bd0 msgr2=0x563f46e34950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:11.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.243+0000 7fa91c849740 1 --2- 172.21.15.43:0/3651720435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e27bd0 0x563f46e34950 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fa908009900 tx=0x7fa90802c610 comp rx=0 tx=0).stop 2024-04-21T08:41:11.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.244+0000 7fa91c849740 1 -- 172.21.15.43:0/3651720435 shutdown_connections 2024-04-21T08:41:11.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.244+0000 7fa91c849740 1 --2- 172.21.15.43:0/3651720435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f46e34e90 0x563f46e37280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.244+0000 7fa91c849740 1 --2- 172.21.15.43:0/3651720435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e27bd0 0x563f46e34950 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.244+0000 7fa91c849740 1 --2- 172.21.15.43:0/3651720435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f46e272b0 0x563f46e27690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.244+0000 7fa91c849740 1 -- 172.21.15.43:0/3651720435 >> 172.21.15.43:0/3651720435 conn(0x563f46e21cb0 msgr2=0x563f46e22090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:11.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.244+0000 7fa91c849740 1 -- 172.21.15.43:0/3651720435 shutdown_connections 2024-04-21T08:41:11.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.244+0000 7fa91c849740 1 -- 172.21.15.43:0/3651720435 wait complete. 2024-04-21T08:41:11.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.244+0000 7fa91c849740 1 -- start start 2024-04-21T08:41:11.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.244+0000 7fa91c849740 1 Processor -- start 2024-04-21T08:41:11.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.244+0000 7fa91c849740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e272b0 0x563f46e31e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.244+0000 7fa91c849740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f46e27bd0 0x563f46e32390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.244+0000 7fa91c849740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f46e34e90 0x563f46e328d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.244+0000 7fa91c849740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563f46e37bf0 con 0x563f46e272b0 2024-04-21T08:41:11.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.244+0000 7fa91c849740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563f46d452f0 con 0x563f46e34e90 2024-04-21T08:41:11.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.244+0000 7fa91c849740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563f46e37d70 con 0x563f46e27bd0 2024-04-21T08:41:11.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.245+0000 7fa91c849740 10 client.0 _ll_register_callbacks cb 0x563f46d52580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:11.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.245+0000 7fa912ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e272b0 0x563f46e31e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.245+0000 7fa912ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e272b0 0x563f46e31e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56118/0 (socket says 172.21.15.43:56118) 2024-04-21T08:41:11.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.245+0000 7fa912ffd640 1 -- 172.21.15.43:0/1117297223 learned_addr learned my addr 172.21.15.43:0/1117297223 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:11.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.245+0000 7fa9137fe640 1 --2- 172.21.15.43:0/1117297223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f46e27bd0 0x563f46e32390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.245+0000 7fa912ffd640 1 --2- 172.21.15.43:0/1117297223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f46e34e90 0x563f46e328d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.245+0000 7fa912ffd640 1 -- 172.21.15.43:0/1117297223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f46e27bd0 msgr2=0x563f46e32390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:11.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.245+0000 7fa912ffd640 1 --2- 172.21.15.43:0/1117297223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f46e27bd0 0x563f46e32390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.245+0000 7fa912ffd640 1 -- 172.21.15.43:0/1117297223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f46e34e90 msgr2=0x563f46e328d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:11.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.245+0000 7fa912ffd640 1 --2- 172.21.15.43:0/1117297223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f46e34e90 0x563f46e328d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.245+0000 7fa912ffd640 1 -- 172.21.15.43:0/1117297223 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f46a9e3f0 con 0x563f46e272b0 2024-04-21T08:41:11.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.245+0000 7fa912ffd640 1 --2- 172.21.15.43:0/1117297223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e272b0 0x563f46e31e50 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fa904011c90 tx=0x7fa9040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8ca4bca2f5cc9a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:11.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.245+0000 7fa8f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:11.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.245+0000 7fa8f2ffd640 1 -- 172.21.15.43:0/1117297223 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa904011e40 con 0x563f46e272b0 2024-04-21T08:41:11.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.245+0000 7fa8f2ffd640 1 -- 172.21.15.43:0/1117297223 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fa904021450 con 0x563f46e272b0 2024-04-21T08:41:11.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.245+0000 7fa91c849740 1 -- 172.21.15.43:0/1117297223 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa90802c7e0 con 0x563f46e272b0 2024-04-21T08:41:11.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.245+0000 7fa8f2ffd640 1 -- 172.21.15.43:0/1117297223 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa90402a050 con 0x563f46e272b0 2024-04-21T08:41:11.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.245+0000 7fa91c849740 10 client.16803 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:11.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.245+0000 7fa91c849740 1 -- 172.21.15.43:0/1117297223 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563f46a5e0f0 con 0x563f46e272b0 2024-04-21T08:41:11.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.246+0000 7fa8f2ffd640 1 -- 172.21.15.43:0/1117297223 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa90802c7e0 con 0x563f46e272b0 2024-04-21T08:41:11.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.246+0000 7fa8f2ffd640 1 -- 172.21.15.43:0/1117297223 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa90400f680 con 0x563f46e272b0 2024-04-21T08:41:11.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.246+0000 7fa91c849740 10 client.16803 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:11.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.246+0000 7fa91c849740 10 client.16803 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:11.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.246+0000 7fa91c849740 20 client.16803 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:11.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.246+0000 7fa91c849740 1 --2- 172.21.15.43:0/1117297223 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563f46e44f10 0x563f46e47320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.246+0000 7fa91c849740 4 client.16803 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:11.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.246+0000 7fa91c849740 1 -- 172.21.15.43:0/1117297223 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f46d99c00 con 0x563f46e44f10 2024-04-21T08:41:11.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.246+0000 7fa9137fe640 1 --2- 172.21.15.43:0/1117297223 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563f46e44f10 0x563f46e47320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.246+0000 7fa9137fe640 1 --2- 172.21.15.43:0/1117297223 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563f46e44f10 0x563f46e47320 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66f50b06277af396 server_cookie=a1bb31b08a8c619b in_seq=0 out_seq=0 2024-04-21T08:41:11.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.246+0000 7fa8f2ffd640 10 client.16803 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:11.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.247+0000 7fa8f2ffd640 1 -- 172.21.15.43:0/1117297223 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f46d99c00 con 0x563f46e44f10 2024-04-21T08:41:11.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.247+0000 7fa8f2ffd640 10 client.16803 handle_command_reply: tid=0 2024-04-21T08:41:11.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.247+0000 7fa91c849740 1 client.16803 shutdown 2024-04-21T08:41:11.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.247+0000 7fa91c849740 1 -- 172.21.15.43:0/1117297223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e272b0 msgr2=0x563f46e31e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:11.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.247+0000 7fa91c849740 1 --2- 172.21.15.43:0/1117297223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e272b0 0x563f46e31e50 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fa904011c90 tx=0x7fa9040046c0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.247+0000 7fa91c849740 1 -- 172.21.15.43:0/1117297223 shutdown_connections 2024-04-21T08:41:11.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.247+0000 7fa91c849740 1 --2- 172.21.15.43:0/1117297223 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563f46e44f10 0x563f46e47320 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.247+0000 7fa91c849740 1 --2- 172.21.15.43:0/1117297223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f46e34e90 0x563f46e328d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.247+0000 7fa8f2ffd640 0 client.16803 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:11.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.247+0000 7fa91c849740 1 --2- 172.21.15.43:0/1117297223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f46e27bd0 0x563f46e32390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.247+0000 7fa91c849740 1 --2- 172.21.15.43:0/1117297223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e272b0 0x563f46e31e50 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.247+0000 7fa91c849740 1 -- 172.21.15.43:0/1117297223 >> 172.21.15.43:0/1117297223 conn(0x563f46e21cb0 msgr2=0x563f46e23a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:11.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.247+0000 7fa91c849740 1 -- 172.21.15.43:0/1117297223 shutdown_connections 2024-04-21T08:41:11.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.247+0000 7fa91c849740 1 -- 172.21.15.43:0/1117297223 wait complete. 2024-04-21T08:41:11.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.248+0000 7fa91c849740 20 client.16803 trim_cache size 0 max 16384 2024-04-21T08:41:11.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.265+0000 7fa91c849740 1 Processor -- start 2024-04-21T08:41:11.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.265+0000 7fa91c849740 1 -- start start 2024-04-21T08:41:11.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.265+0000 7fa91c849740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f46e272b0 0x563f46e30d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.265+0000 7fa91c849740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e27bd0 0x563f46e40b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.265+0000 7fa91c849740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f46e34e90 0x563f46e4cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.265+0000 7fa91c849740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa9141ad1b0 con 0x563f46e27bd0 2024-04-21T08:41:11.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.265+0000 7fa91c849740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9141aecf0 con 0x563f46e272b0 2024-04-21T08:41:11.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.265+0000 7fa91c849740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9141ad880 con 0x563f46e34e90 2024-04-21T08:41:11.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.266+0000 7fa9137fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e27bd0 0x563f46e40b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.266+0000 7fa912ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f46e272b0 0x563f46e30d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.266+0000 7fa9137fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e27bd0 0x563f46e40b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56126/0 (socket says 172.21.15.43:56126) 2024-04-21T08:41:11.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.266+0000 7fa9137fe640 1 -- 172.21.15.43:0/3996192932 learned_addr learned my addr 172.21.15.43:0/3996192932 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.266+0000 7fa912ffd640 1 --2- 172.21.15.43:0/3996192932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f46e34e90 0x563f46e4cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.266+0000 7fa9137fe640 1 -- 172.21.15.43:0/3996192932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f46e34e90 msgr2=0x563f46e4cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.266+0000 7fa9137fe640 1 --2- 172.21.15.43:0/3996192932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f46e34e90 0x563f46e4cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.266+0000 7fa9137fe640 1 -- 172.21.15.43:0/3996192932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f46e272b0 msgr2=0x563f46e30d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.266+0000 7fa9137fe640 1 --2- 172.21.15.43:0/3996192932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f46e272b0 0x563f46e30d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.266+0000 7fa9137fe640 1 -- 172.21.15.43:0/3996192932 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f46e43a60 con 0x563f46e27bd0 2024-04-21T08:41:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.266+0000 7fa912ffd640 1 --2- 172.21.15.43:0/3996192932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f46e34e90 0x563f46e4cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.266+0000 7fa9137fe640 1 --2- 172.21.15.43:0/3996192932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e27bd0 0x563f46e40b50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa908009900 tx=0x7fa908005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310e313b714c6351 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.266+0000 7fa8f97fa640 1 -- 172.21.15.43:0/3996192932 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908045040 con 0x563f46e27bd0 2024-04-21T08:41:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.266+0000 7fa8f97fa640 1 -- 172.21.15.43:0/3996192932 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa90804e070 con 0x563f46e27bd0 2024-04-21T08:41:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.266+0000 7fa8f97fa640 1 -- 172.21.15.43:0/3996192932 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908003ab0 con 0x563f46e27bd0 2024-04-21T08:41:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.267+0000 7fa91c849740 1 -- 172.21.15.43:0/3996192932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e27bd0 msgr2=0x563f46e40b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.267+0000 7fa91c849740 1 --2- 172.21.15.43:0/3996192932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e27bd0 0x563f46e40b50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa908009900 tx=0x7fa908005fd0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.267+0000 7fa91c849740 1 -- 172.21.15.43:0/3996192932 shutdown_connections 2024-04-21T08:41:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.267+0000 7fa91c849740 1 --2- 172.21.15.43:0/3996192932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f46e34e90 0x563f46e4cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.267+0000 7fa91c849740 1 --2- 172.21.15.43:0/3996192932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e27bd0 0x563f46e40b50 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.267+0000 7fa91c849740 1 --2- 172.21.15.43:0/3996192932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f46e272b0 0x563f46e30d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.267+0000 7fa91c849740 1 -- 172.21.15.43:0/3996192932 >> 172.21.15.43:0/3996192932 conn(0x563f46e21cb0 msgr2=0x563f46e20d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.267+0000 7fa91c849740 1 -- 172.21.15.43:0/3996192932 shutdown_connections 2024-04-21T08:41:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.267+0000 7fa91c849740 1 -- 172.21.15.43:0/3996192932 wait complete. 2024-04-21T08:41:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.267+0000 7fa91c849740 1 -- start start 2024-04-21T08:41:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.267+0000 7fa91c849740 1 Processor -- start 2024-04-21T08:41:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.268+0000 7fa91c849740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f46e272b0 0x563f46e393b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.268+0000 7fa91c849740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e27bd0 0x563f46e398f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.268+0000 7fa91c849740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f46e34e90 0x563f46ea44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.268+0000 7fa91c849740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa9141ae620 con 0x563f46e27bd0 2024-04-21T08:41:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.268+0000 7fa91c849740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9141adf50 con 0x563f46e272b0 2024-04-21T08:41:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.268+0000 7fa91c849740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9141ab190 con 0x563f46e34e90 2024-04-21T08:41:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.268+0000 7fa91c849740 10 client.0 _ll_register_callbacks cb 0x7fa904005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.268+0000 7fa9137fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e27bd0 0x563f46e398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.268+0000 7fa9137fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e27bd0 0x563f46e398f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56130/0 (socket says 172.21.15.43:56130) 2024-04-21T08:41:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.268+0000 7fa9137fe640 1 -- 172.21.15.43:0/3062663585 learned_addr learned my addr 172.21.15.43:0/3062663585 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.268+0000 7fa912ffd640 1 --2- 172.21.15.43:0/3062663585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f46e272b0 0x563f46e393b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.268+0000 7fa912ffd640 1 --2- 172.21.15.43:0/3062663585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f46e34e90 0x563f46ea44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.268+0000 7fa9137fe640 1 -- 172.21.15.43:0/3062663585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f46e34e90 msgr2=0x563f46ea44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.268+0000 7fa9137fe640 1 --2- 172.21.15.43:0/3062663585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f46e34e90 0x563f46ea44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.268+0000 7fa9137fe640 1 -- 172.21.15.43:0/3062663585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f46e272b0 msgr2=0x563f46e393b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.268+0000 7fa9137fe640 1 --2- 172.21.15.43:0/3062663585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f46e272b0 0x563f46e393b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.268+0000 7fa9137fe640 1 -- 172.21.15.43:0/3062663585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f46e21ae0 con 0x563f46e27bd0 2024-04-21T08:41:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.268+0000 7fa9137fe640 1 --2- 172.21.15.43:0/3062663585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e27bd0 0x563f46e398f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fa90802c170 tx=0x7fa908000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92cb784e8d9ed2c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.268+0000 7fa8f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:11.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.268+0000 7fa8f2ffd640 1 -- 172.21.15.43:0/3062663585 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908045040 con 0x563f46e27bd0 2024-04-21T08:41:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.268+0000 7fa8f2ffd640 1 -- 172.21.15.43:0/3062663585 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fa908041680 con 0x563f46e27bd0 2024-04-21T08:41:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.268+0000 7fa91c849740 1 -- 172.21.15.43:0/3062663585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa90804e070 con 0x563f46e27bd0 2024-04-21T08:41:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.269+0000 7fa8f2ffd640 1 -- 172.21.15.43:0/3062663585 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa908041880 con 0x563f46e27bd0 2024-04-21T08:41:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.269+0000 7fa91c849740 10 client.16809 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.269+0000 7fa91c849740 1 -- 172.21.15.43:0/3062663585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563f46eabd40 con 0x563f46e27bd0 2024-04-21T08:41:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.269+0000 7fa8f2ffd640 1 -- 172.21.15.43:0/3062663585 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa90804e070 con 0x563f46e27bd0 2024-04-21T08:41:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.269+0000 7fa8f2ffd640 1 -- 172.21.15.43:0/3062663585 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa908034960 con 0x563f46e27bd0 2024-04-21T08:41:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.269+0000 7fa91c849740 10 client.16809 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.269+0000 7fa91c849740 10 client.16809 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.269+0000 7fa91c849740 20 client.16809 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.269+0000 7fa91c849740 1 --2- 172.21.15.43:0/3062663585 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563f46eac070 0x563f46eae460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.269+0000 7fa91c849740 4 client.16809 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:41:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.269+0000 7fa91c849740 1 -- 172.21.15.43:0/3062663585 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563f46d93a70 con 0x563f46eac070 2024-04-21T08:41:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.269+0000 7fa912ffd640 1 --2- 172.21.15.43:0/3062663585 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563f46eac070 0x563f46eae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.269+0000 7fa912ffd640 1 --2- 172.21.15.43:0/3062663585 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563f46eac070 0x563f46eae460 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aec0673978730390 server_cookie=4f4e614d0873df34 in_seq=0 out_seq=0 2024-04-21T08:41:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.269+0000 7fa8f2ffd640 10 client.16809 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.270+0000 7fa8f2ffd640 1 -- 172.21.15.43:0/3062663585 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563f46d93a70 con 0x563f46eac070 2024-04-21T08:41:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.270+0000 7fa8f2ffd640 10 client.16809 handle_command_reply: tid=0 2024-04-21T08:41:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.270+0000 7fa91c849740 1 client.16809 shutdown 2024-04-21T08:41:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.270+0000 7fa91c849740 1 -- 172.21.15.43:0/3062663585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e27bd0 msgr2=0x563f46e398f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.270+0000 7fa91c849740 1 --2- 172.21.15.43:0/3062663585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e27bd0 0x563f46e398f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fa90802c170 tx=0x7fa908000bc0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.270+0000 7fa91c849740 1 -- 172.21.15.43:0/3062663585 shutdown_connections 2024-04-21T08:41:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.270+0000 7fa91c849740 1 --2- 172.21.15.43:0/3062663585 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563f46eac070 0x563f46eae460 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.270+0000 7fa91c849740 1 --2- 172.21.15.43:0/3062663585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563f46e34e90 0x563f46ea44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.270+0000 7fa91c849740 1 --2- 172.21.15.43:0/3062663585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f46e27bd0 0x563f46e398f0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.270+0000 7fa91c849740 1 --2- 172.21.15.43:0/3062663585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f46e272b0 0x563f46e393b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.270+0000 7fa8f2ffd640 0 client.16809 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.270+0000 7fa91c849740 1 -- 172.21.15.43:0/3062663585 >> 172.21.15.43:0/3062663585 conn(0x563f46e21cb0 msgr2=0x563f46e31e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.270+0000 7fa91c849740 1 -- 172.21.15.43:0/3062663585 shutdown_connections 2024-04-21T08:41:11.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.270+0000 7fa91c849740 1 -- 172.21.15.43:0/3062663585 wait complete. 2024-04-21T08:41:11.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.270+0000 7fa91c849740 20 client.16809 trim_cache size 0 max 16384 2024-04-21T08:41:11.277 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:41:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:41:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-21T08:41:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "d13cfb66-432c-4d9e-a40e-f50a5ce32d56": { 2024-04-21T08:41:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-21T08:41:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "d13cfb66-432c-4d9e-a40e-f50a5ce32d56", 2024-04-21T08:41:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-21T08:41:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:41:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:41:11.278 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:41:11.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.271+0000 7fa919c55640 1 -- 172.21.15.43:0/1609159799 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa8fc0a3d20 msgr2=0x7fa8fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:11.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.271+0000 7fa919c55640 1 --2- 172.21.15.43:0/1609159799 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa8fc0a3d20 0x7fa8fc0a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0ce140 tx=0x7fa90c087040 comp rx=0 tx=0).stop 2024-04-21T08:41:11.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.271+0000 7fa919c55640 1 -- 172.21.15.43:0/1609159799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa91419d020 msgr2=0x7fa91419d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:11.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.271+0000 7fa919c55640 1 --2- 172.21.15.43:0/1609159799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa91419d020 0x7fa91419d4d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00eaa0 tx=0x7fa90c00ef60 comp rx=0 tx=0).stop 2024-04-21T08:41:11.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.271+0000 7fa919c55640 1 -- 172.21.15.43:0/1609159799 shutdown_connections 2024-04-21T08:41:11.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.271+0000 7fa919c55640 1 --2- 172.21.15.43:0/1609159799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa91419d020 0x7fa91419d4d0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.271+0000 7fa919c55640 1 --2- 172.21.15.43:0/1609159799 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa8fc0a3d20 0x7fa8fc0a61e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.271+0000 7fa919c55640 1 --2- 172.21.15.43:0/1609159799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa914107270 0x7fa914198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.271+0000 7fa919c55640 1 --2- 172.21.15.43:0/1609159799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9141021a0 0x7fa914198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:11.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.271+0000 7fa919c55640 1 -- 172.21.15.43:0/1609159799 >> 172.21.15.43:0/1609159799 conn(0x7fa9140fda40 msgr2=0x7fa9140fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:11.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.271+0000 7fa919c55640 1 -- 172.21.15.43:0/1609159799 shutdown_connections 2024-04-21T08:41:11.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:11.271+0000 7fa919c55640 1 -- 172.21.15.43:0/1609159799 wait complete. 2024-04-21T08:41:11.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:11 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/4120450557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:11.281 INFO:tasks.cephfs.filesystem:scrub status for tag:d13cfb66-432c-4d9e-a40e-f50a5ce32d56 - {'path': '/', 'tag': 'd13cfb66-432c-4d9e-a40e-f50a5ce32d56', 'options': 'recursive,force'} 2024-04-21T08:41:11.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:11 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/4120450557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:11.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:11 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/4120450557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:12.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:12 smithi112 ceph-mon[31085]: pgmap v71: 129 pgs: 129 active+clean; 2.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-04-21T08:41:12.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:12 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:41:12.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:12 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:41:12.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:12 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:41:12.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:12 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:41:12.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:12 smithi043 ceph-mon[27978]: pgmap v71: 129 pgs: 129 active+clean; 2.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-04-21T08:41:12.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:12 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:41:12.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:12 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:41:12.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:12 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:41:12.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:12 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:41:12.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:12 smithi175 ceph-mon[31920]: pgmap v71: 129 pgs: 129 active+clean; 2.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-04-21T08:41:12.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:12 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:41:12.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:12 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:41:12.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:12 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:41:12.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:12 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:41:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.520+0000 7ff005a3f640 1 -- 172.21.15.43:0/2546861667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff000102b80 msgr2=0x7ff000106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.520+0000 7ff005a3f640 1 --2- 172.21.15.43:0/2546861667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff000102b80 0x7ff000106bc0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7feff4009900 tx=0x7feff402bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.520+0000 7ff005a3f640 1 -- 172.21.15.43:0/2546861667 shutdown_connections 2024-04-21T08:41:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.520+0000 7ff005a3f640 1 --2- 172.21.15.43:0/2546861667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0001072a0 0x7ff00010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.520+0000 7ff005a3f640 1 --2- 172.21.15.43:0/2546861667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff000102b80 0x7ff000106bc0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.520+0000 7ff005a3f640 1 --2- 172.21.15.43:0/2546861667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0001021d0 0x7ff0001025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.520+0000 7ff005a3f640 1 -- 172.21.15.43:0/2546861667 >> 172.21.15.43:0/2546861667 conn(0x7ff0000793e0 msgr2=0x7ff0000797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.520+0000 7ff005a3f640 1 -- 172.21.15.43:0/2546861667 shutdown_connections 2024-04-21T08:41:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.520+0000 7ff005a3f640 1 -- 172.21.15.43:0/2546861667 wait complete. 2024-04-21T08:41:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.521+0000 7ff005a3f640 1 Processor -- start 2024-04-21T08:41:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.521+0000 7ff005a3f640 1 -- start start 2024-04-21T08:41:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.521+0000 7ff005a3f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0001021d0 0x7ff000198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.521+0000 7ff005a3f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0001072a0 0x7ff000198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.521+0000 7ff005a3f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00019ced0 0x7ff00019d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.521+0000 7ff005a3f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff000112210 con 0x7ff0001072a0 2024-04-21T08:41:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.521+0000 7ff005a3f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff000112090 con 0x7ff00019ced0 2024-04-21T08:41:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.521+0000 7ff005a3f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff000112390 con 0x7ff0001021d0 2024-04-21T08:41:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.521+0000 7feffe7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0001072a0 0x7ff000198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.521+0000 7feffe7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0001072a0 0x7ff000198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56160/0 (socket says 172.21.15.43:56160) 2024-04-21T08:41:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.521+0000 7feffe7fc640 1 -- 172.21.15.43:0/3865986738 learned_addr learned my addr 172.21.15.43:0/3865986738 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.521+0000 7feffeffd640 1 --2- 172.21.15.43:0/3865986738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0001021d0 0x7ff000198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.521+0000 7fefff7fe640 1 --2- 172.21.15.43:0/3865986738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00019ced0 0x7ff00019d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.522+0000 7feffe7fc640 1 -- 172.21.15.43:0/3865986738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0001021d0 msgr2=0x7ff000198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.522+0000 7feffe7fc640 1 --2- 172.21.15.43:0/3865986738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0001021d0 0x7ff000198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.522+0000 7feffe7fc640 1 -- 172.21.15.43:0/3865986738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00019ced0 msgr2=0x7ff00019d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.522+0000 7feffe7fc640 1 --2- 172.21.15.43:0/3865986738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00019ced0 0x7ff00019d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.522+0000 7feffe7fc640 1 -- 172.21.15.43:0/3865986738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00019d9c0 con 0x7ff0001072a0 2024-04-21T08:41:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.522+0000 7feffe7fc640 1 --2- 172.21.15.43:0/3865986738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0001072a0 0x7ff000198b50 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7feff4000c00 tx=0x7feff40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.522+0000 7ff004a3d640 1 -- 172.21.15.43:0/3865986738 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4043020 con 0x7ff0001072a0 2024-04-21T08:41:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.522+0000 7ff004a3d640 1 -- 172.21.15.43:0/3865986738 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feff402cb70 con 0x7ff0001072a0 2024-04-21T08:41:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.522+0000 7ff005a3f640 1 -- 172.21.15.43:0/3865986738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00019dc50 con 0x7ff0001072a0 2024-04-21T08:41:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.522+0000 7ff004a3d640 1 -- 172.21.15.43:0/3865986738 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4003b80 con 0x7ff0001072a0 2024-04-21T08:41:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.522+0000 7ff005a3f640 1 -- 172.21.15.43:0/3865986738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0001a5470 con 0x7ff0001072a0 2024-04-21T08:41:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.525+0000 7ff004a3d640 1 -- 172.21.15.43:0/3865986738 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feff4046020 con 0x7ff0001072a0 2024-04-21T08:41:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.525+0000 7ff00862f740 1 Processor -- start 2024-04-21T08:41:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.525+0000 7ff00862f740 1 -- start start 2024-04-21T08:41:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.525+0000 7ff00862f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561060edb8d0 0x561060eda620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.525+0000 7ff00862f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561060edab60 0x561060edafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.525+0000 7ff00862f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060ee40e0 0x561060ee64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.525+0000 7ff00862f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561060e3f8d0 con 0x561060ee40e0 2024-04-21T08:41:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.525+0000 7ff00862f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561060ddce80 con 0x561060edab60 2024-04-21T08:41:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.525+0000 7ff00862f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561060e3b890 con 0x561060edb8d0 2024-04-21T08:41:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.525+0000 7feffe7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060ee40e0 0x561060ee64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.525+0000 7feffe7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060ee40e0 0x561060ee64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56174/0 (socket says 172.21.15.43:56174) 2024-04-21T08:41:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.525+0000 7feffe7fc640 1 -- 172.21.15.43:0/3784033813 learned_addr learned my addr 172.21.15.43:0/3784033813 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.526+0000 7feffeffd640 1 --2- 172.21.15.43:0/3784033813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561060edab60 0x561060edafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.526+0000 7fefff7fe640 1 --2- 172.21.15.43:0/3784033813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561060edb8d0 0x561060eda620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.526+0000 7ff004a3d640 1 --2- 172.21.15.43:0/3865986738 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fefe80a3dc0 0x7fefe80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.526+0000 7feffe7fc640 1 -- 172.21.15.43:0/3784033813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561060edb8d0 msgr2=0x561060eda620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.526+0000 7feffe7fc640 1 --2- 172.21.15.43:0/3784033813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561060edb8d0 0x561060eda620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.526+0000 7feffe7fc640 1 -- 172.21.15.43:0/3784033813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561060edab60 msgr2=0x561060edafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.526+0000 7feffe7fc640 1 --2- 172.21.15.43:0/3784033813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561060edab60 0x561060edafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.526+0000 7feffe7fc640 1 -- 172.21.15.43:0/3784033813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561060dfe320 con 0x561060ee40e0 2024-04-21T08:41:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.526+0000 7fefff7fe640 1 --2- 172.21.15.43:0/3784033813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561060edb8d0 0x561060eda620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.526+0000 7feffeffd640 1 --2- 172.21.15.43:0/3784033813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561060edab60 0x561060edafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:41:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.526+0000 7fefff7fe640 1 --2- 172.21.15.43:0/3865986738 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fefe80a3dc0 0x7fefe80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.526+0000 7ff004a3d640 1 -- 172.21.15.43:0/3865986738 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7feff40f2b20 con 0x7ff0001072a0 2024-04-21T08:41:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.526+0000 7feffe7fc640 1 --2- 172.21.15.43:0/3784033813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060ee40e0 0x561060ee64d0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7feff40eff20 tx=0x7feff404f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0765af4f2c8f95 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.526+0000 7fefe5ffb640 1 -- 172.21.15.43:0/3784033813 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff40a7e60 con 0x561060ee40e0 2024-04-21T08:41:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.526+0000 7fefe5ffb640 1 -- 172.21.15.43:0/3784033813 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feff40f7040 con 0x561060ee40e0 2024-04-21T08:41:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.526+0000 7fefe5ffb640 1 -- 172.21.15.43:0/3784033813 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff40be340 con 0x561060ee40e0 2024-04-21T08:41:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.526+0000 7fefff7fe640 1 --2- 172.21.15.43:0/3865986738 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fefe80a3dc0 0x7fefe80a6280 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7feff8007f70 tx=0x7feff8004a40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.527+0000 7ff00862f740 1 -- 172.21.15.43:0/3784033813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060ee40e0 msgr2=0x561060ee64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.527+0000 7ff00862f740 1 --2- 172.21.15.43:0/3784033813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060ee40e0 0x561060ee64d0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7feff40eff20 tx=0x7feff404f000 comp rx=0 tx=0).stop 2024-04-21T08:41:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.527+0000 7ff00862f740 1 -- 172.21.15.43:0/3784033813 shutdown_connections 2024-04-21T08:41:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.527+0000 7ff00862f740 1 --2- 172.21.15.43:0/3784033813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060ee40e0 0x561060ee64d0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.527+0000 7ff00862f740 1 --2- 172.21.15.43:0/3784033813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561060edab60 0x561060edafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.527+0000 7ff00862f740 1 --2- 172.21.15.43:0/3784033813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561060edb8d0 0x561060eda620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.527+0000 7ff00862f740 1 -- 172.21.15.43:0/3784033813 >> 172.21.15.43:0/3784033813 conn(0x561060ea1c90 msgr2=0x561060ed5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.527+0000 7ff00862f740 1 -- 172.21.15.43:0/3784033813 shutdown_connections 2024-04-21T08:41:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.527+0000 7ff00862f740 1 -- 172.21.15.43:0/3784033813 wait complete. 2024-04-21T08:41:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.527+0000 7ff00862f740 1 -- start start 2024-04-21T08:41:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.528+0000 7ff00862f740 1 Processor -- start 2024-04-21T08:41:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.528+0000 7ff00862f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561060edab60 0x561060ee3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.528+0000 7ff00862f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561060edb8d0 0x561060ee0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.528+0000 7ff00862f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060ee40e0 0x561060ee12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.528+0000 7ff00862f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561060ee6bd0 con 0x561060ee40e0 2024-04-21T08:41:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.528+0000 7ff00862f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561060dc7eb0 con 0x561060edb8d0 2024-04-21T08:41:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.528+0000 7ff00862f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561060ee6d50 con 0x561060edab60 2024-04-21T08:41:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.528+0000 7ff00862f740 10 client.0 _ll_register_callbacks cb 0x561060df4b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.528+0000 7fefff7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060ee40e0 0x561060ee12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.528+0000 7fefff7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060ee40e0 0x561060ee12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56190/0 (socket says 172.21.15.43:56190) 2024-04-21T08:41:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.528+0000 7fefff7fe640 1 -- 172.21.15.43:0/2533016794 learned_addr learned my addr 172.21.15.43:0/2533016794 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.528+0000 7feffe7fc640 1 --2- 172.21.15.43:0/2533016794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561060edb8d0 0x561060ee0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.528+0000 7feffeffd640 1 --2- 172.21.15.43:0/2533016794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561060edab60 0x561060ee3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.529+0000 7fefff7fe640 1 -- 172.21.15.43:0/2533016794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561060edab60 msgr2=0x561060ee3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.529+0000 7fefff7fe640 1 --2- 172.21.15.43:0/2533016794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561060edab60 0x561060ee3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.529+0000 7fefff7fe640 1 -- 172.21.15.43:0/2533016794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561060edb8d0 msgr2=0x561060ee0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.529+0000 7fefff7fe640 1 --2- 172.21.15.43:0/2533016794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561060edb8d0 0x561060ee0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.529+0000 7fefff7fe640 1 -- 172.21.15.43:0/2533016794 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561060b52210 con 0x561060ee40e0 2024-04-21T08:41:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.529+0000 7feffeffd640 1 --2- 172.21.15.43:0/2533016794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561060edab60 0x561060ee3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.529+0000 7fefff7fe640 1 --2- 172.21.15.43:0/2533016794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060ee40e0 0x561060ee12f0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7feff8012e20 tx=0x7feff8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696a1f26c69f6900 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.529+0000 7fefdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.529+0000 7fefdeffd640 1 -- 172.21.15.43:0/2533016794 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff8027670 con 0x561060ee40e0 2024-04-21T08:41:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.529+0000 7fefdeffd640 1 -- 172.21.15.43:0/2533016794 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7feff8027810 con 0x561060ee40e0 2024-04-21T08:41:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.529+0000 7ff00862f740 1 -- 172.21.15.43:0/2533016794 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feff40f7040 con 0x561060ee40e0 2024-04-21T08:41:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.529+0000 7fefdeffd640 1 -- 172.21.15.43:0/2533016794 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feff8027a10 con 0x561060ee40e0 2024-04-21T08:41:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.529+0000 7ff00862f740 10 client.16821 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.529+0000 7ff00862f740 1 -- 172.21.15.43:0/2533016794 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561060c1e620 con 0x561060ee40e0 2024-04-21T08:41:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.529+0000 7fefdeffd640 1 -- 172.21.15.43:0/2533016794 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feff40f7040 con 0x561060ee40e0 2024-04-21T08:41:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.530+0000 7fefdeffd640 1 -- 172.21.15.43:0/2533016794 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feff801b440 con 0x561060ee40e0 2024-04-21T08:41:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.530+0000 7ff00862f740 10 client.16821 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.530+0000 7ff00862f740 10 client.16821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.530+0000 7ff00862f740 20 client.16821 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.530+0000 7ff00862f740 1 --2- 172.21.15.43:0/2533016794 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561060ef3e30 0x561060ef6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.530+0000 7ff00862f740 4 client.16821 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.530+0000 7ff00862f740 1 -- 172.21.15.43:0/2533016794 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561060e53880 con 0x561060ef3e30 2024-04-21T08:41:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.530+0000 7feffeffd640 1 --2- 172.21.15.43:0/2533016794 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561060ef3e30 0x561060ef6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.530+0000 7feffeffd640 1 --2- 172.21.15.43:0/2533016794 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561060ef3e30 0x561060ef6220 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ebfae09f977ce15 server_cookie=13bc548393b00e68 in_seq=0 out_seq=0 2024-04-21T08:41:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.530+0000 7fefdeffd640 10 client.16821 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.531+0000 7fefdeffd640 1 -- 172.21.15.43:0/2533016794 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561060e53880 con 0x561060ef3e30 2024-04-21T08:41:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.531+0000 7fefdeffd640 10 client.16821 handle_command_reply: tid=0 2024-04-21T08:41:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.531+0000 7ff00862f740 1 client.16821 shutdown 2024-04-21T08:41:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.531+0000 7ff00862f740 1 -- 172.21.15.43:0/2533016794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060ee40e0 msgr2=0x561060ee12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.531+0000 7ff00862f740 1 --2- 172.21.15.43:0/2533016794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060ee40e0 0x561060ee12f0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7feff8012e20 tx=0x7feff8004ab0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.531+0000 7ff00862f740 1 -- 172.21.15.43:0/2533016794 shutdown_connections 2024-04-21T08:41:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.532+0000 7ff00862f740 1 --2- 172.21.15.43:0/2533016794 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561060ef3e30 0x561060ef6220 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.532+0000 7ff00862f740 1 --2- 172.21.15.43:0/2533016794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060ee40e0 0x561060ee12f0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.532+0000 7ff00862f740 1 --2- 172.21.15.43:0/2533016794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561060edb8d0 0x561060ee0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.532+0000 7fefdeffd640 0 client.16821 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.532+0000 7ff00862f740 1 --2- 172.21.15.43:0/2533016794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561060edab60 0x561060ee3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.532+0000 7ff00862f740 1 -- 172.21.15.43:0/2533016794 >> 172.21.15.43:0/2533016794 conn(0x561060ea1c90 msgr2=0x561060ed75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.532+0000 7ff00862f740 1 -- 172.21.15.43:0/2533016794 shutdown_connections 2024-04-21T08:41:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.532+0000 7ff00862f740 1 -- 172.21.15.43:0/2533016794 wait complete. 2024-04-21T08:41:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.532+0000 7ff00862f740 20 client.16821 trim_cache size 0 max 16384 2024-04-21T08:41:13.556 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.547+0000 7ff00862f740 1 Processor -- start 2024-04-21T08:41:13.556 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.547+0000 7ff00862f740 1 -- start start 2024-04-21T08:41:13.556 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.547+0000 7ff00862f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561060ea1c90 0x561060eefa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:13.556 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.547+0000 7ff00862f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060edab60 0x561060eda0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:13.556 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.547+0000 7ff00862f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561060edb8d0 0x561060ee34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:13.556 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.547+0000 7ff00862f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff0001af260 con 0x561060edab60 2024-04-21T08:41:13.556 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.547+0000 7ff00862f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff0001b0da0 con 0x561060edb8d0 2024-04-21T08:41:13.556 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.547+0000 7ff00862f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff0001af930 con 0x561060ea1c90 2024-04-21T08:41:13.556 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.547+0000 7feffe7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060edab60 0x561060eda0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:13.556 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.547+0000 7feffe7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060edab60 0x561060eda0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56200/0 (socket says 172.21.15.43:56200) 2024-04-21T08:41:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.547+0000 7feffe7fc640 1 -- 172.21.15.43:0/2576436828 learned_addr learned my addr 172.21.15.43:0/2576436828 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.547+0000 7fefff7fe640 1 --2- 172.21.15.43:0/2576436828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561060edb8d0 0x561060ee34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.547+0000 7feffe7fc640 1 -- 172.21.15.43:0/2576436828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561060ea1c90 msgr2=0x561060eefa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.547+0000 7feffe7fc640 1 --2- 172.21.15.43:0/2576436828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561060ea1c90 0x561060eefa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.547+0000 7feffe7fc640 1 -- 172.21.15.43:0/2576436828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561060edb8d0 msgr2=0x561060ee34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.547+0000 7feffe7fc640 1 --2- 172.21.15.43:0/2576436828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561060edb8d0 0x561060ee34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.547+0000 7feffe7fc640 1 -- 172.21.15.43:0/2576436828 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561060ef6850 con 0x561060edab60 2024-04-21T08:41:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.548+0000 7feffe7fc640 1 --2- 172.21.15.43:0/2576436828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060edab60 0x561060eda0a0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7feff40f04a0 tx=0x7feff4041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c309984c4b409d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.548+0000 7fefe57fa640 1 -- 172.21.15.43:0/2576436828 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff40be160 con 0x561060edab60 2024-04-21T08:41:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.548+0000 7fefe57fa640 1 -- 172.21.15.43:0/2576436828 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feff40f6090 con 0x561060edab60 2024-04-21T08:41:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.548+0000 7fefe57fa640 1 -- 172.21.15.43:0/2576436828 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff40c2bb0 con 0x561060edab60 2024-04-21T08:41:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.548+0000 7ff00862f740 1 -- 172.21.15.43:0/2576436828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060edab60 msgr2=0x561060eda0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.548+0000 7ff00862f740 1 --2- 172.21.15.43:0/2576436828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060edab60 0x561060eda0a0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7feff40f04a0 tx=0x7feff4041040 comp rx=0 tx=0).stop 2024-04-21T08:41:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.548+0000 7ff00862f740 1 -- 172.21.15.43:0/2576436828 shutdown_connections 2024-04-21T08:41:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.548+0000 7ff00862f740 1 --2- 172.21.15.43:0/2576436828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561060edb8d0 0x561060ee34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.548+0000 7ff00862f740 1 --2- 172.21.15.43:0/2576436828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060edab60 0x561060eda0a0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.548+0000 7ff00862f740 1 --2- 172.21.15.43:0/2576436828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561060ea1c90 0x561060eefa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.548+0000 7ff00862f740 1 -- 172.21.15.43:0/2576436828 >> 172.21.15.43:0/2576436828 conn(0x561060ee1450 msgr2=0x561060edf680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.548+0000 7ff00862f740 1 -- 172.21.15.43:0/2576436828 shutdown_connections 2024-04-21T08:41:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.548+0000 7ff00862f740 1 -- 172.21.15.43:0/2576436828 wait complete. 2024-04-21T08:41:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.549+0000 7ff00862f740 1 -- start start 2024-04-21T08:41:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.549+0000 7ff00862f740 1 Processor -- start 2024-04-21T08:41:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.549+0000 7ff00862f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561060ea1c90 0x561060eddb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.549+0000 7ff00862f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060edab60 0x561060ede0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.549+0000 7ff00862f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561060edb8d0 0x561060ede610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.549+0000 7ff00862f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff0001b06d0 con 0x561060edab60 2024-04-21T08:41:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.549+0000 7ff00862f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff0001b0000 con 0x561060ea1c90 2024-04-21T08:41:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.549+0000 7ff00862f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff0001ad240 con 0x561060edb8d0 2024-04-21T08:41:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.549+0000 7ff00862f740 10 client.0 _ll_register_callbacks cb 0x7feff8007850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.549+0000 7feffe7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060edab60 0x561060ede0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.549+0000 7feffe7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060edab60 0x561060ede0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56214/0 (socket says 172.21.15.43:56214) 2024-04-21T08:41:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.549+0000 7feffe7fc640 1 -- 172.21.15.43:0/4064885523 learned_addr learned my addr 172.21.15.43:0/4064885523 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.549+0000 7feffeffd640 1 --2- 172.21.15.43:0/4064885523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561060ea1c90 0x561060eddb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.550+0000 7fefff7fe640 1 --2- 172.21.15.43:0/4064885523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561060edb8d0 0x561060ede610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.550+0000 7feffe7fc640 1 -- 172.21.15.43:0/4064885523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561060edb8d0 msgr2=0x561060ede610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.550+0000 7feffe7fc640 1 --2- 172.21.15.43:0/4064885523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561060edb8d0 0x561060ede610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.550+0000 7feffe7fc640 1 -- 172.21.15.43:0/4064885523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561060ea1c90 msgr2=0x561060eddb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.550+0000 7feffe7fc640 1 --2- 172.21.15.43:0/4064885523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561060ea1c90 0x561060eddb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.550+0000 7feffe7fc640 1 -- 172.21.15.43:0/4064885523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561060f59480 con 0x561060edab60 2024-04-21T08:41:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.550+0000 7fefff7fe640 1 --2- 172.21.15.43:0/4064885523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561060edb8d0 0x561060ede610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.550+0000 7feffe7fc640 1 --2- 172.21.15.43:0/4064885523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060edab60 0x561060ede0d0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7feff40bb1d0 tx=0x7feff4003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537f2b4535e86ed6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.550+0000 7fefdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.550+0000 7fefdeffd640 1 -- 172.21.15.43:0/4064885523 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff40be160 con 0x561060edab60 2024-04-21T08:41:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.550+0000 7fefdeffd640 1 -- 172.21.15.43:0/4064885523 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7feff4003d20 con 0x561060edab60 2024-04-21T08:41:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.550+0000 7ff00862f740 1 -- 172.21.15.43:0/4064885523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feff40f6090 con 0x561060edab60 2024-04-21T08:41:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.550+0000 7fefdeffd640 1 -- 172.21.15.43:0/4064885523 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feff40b3020 con 0x561060edab60 2024-04-21T08:41:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.550+0000 7ff00862f740 10 client.16827 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.550+0000 7ff00862f740 1 -- 172.21.15.43:0/4064885523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561060f5a770 con 0x561060edab60 2024-04-21T08:41:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.550+0000 7fefdeffd640 1 -- 172.21.15.43:0/4064885523 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feff40f6090 con 0x561060edab60 2024-04-21T08:41:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.551+0000 7fefdeffd640 1 -- 172.21.15.43:0/4064885523 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feff4005490 con 0x561060edab60 2024-04-21T08:41:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.551+0000 7ff00862f740 10 client.16827 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.551+0000 7ff00862f740 10 client.16827 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.551+0000 7ff00862f740 20 client.16827 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.551+0000 7ff00862f740 1 --2- 172.21.15.43:0/4064885523 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561060ee40e0 0x561060f5cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.551+0000 7ff00862f740 4 client.16827 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:41:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.551+0000 7ff00862f740 1 -- 172.21.15.43:0/4064885523 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561060ddf3d0 con 0x561060ee40e0 2024-04-21T08:41:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.551+0000 7feffeffd640 1 --2- 172.21.15.43:0/4064885523 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561060ee40e0 0x561060f5cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.551+0000 7feffeffd640 1 --2- 172.21.15.43:0/4064885523 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561060ee40e0 0x561060f5cb90 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40ef7cced2190865 server_cookie=fcb1e52693b3079d in_seq=0 out_seq=0 2024-04-21T08:41:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.551+0000 7fefdeffd640 10 client.16827 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.566+0000 7fefdeffd640 1 -- 172.21.15.43:0/4064885523 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561060ddf3d0 con 0x561060ee40e0 2024-04-21T08:41:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.566+0000 7fefdeffd640 10 client.16827 handle_command_reply: tid=0 2024-04-21T08:41:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.566+0000 7ff00862f740 1 client.16827 shutdown 2024-04-21T08:41:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.566+0000 7ff00862f740 1 -- 172.21.15.43:0/4064885523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060edab60 msgr2=0x561060ede0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.566+0000 7ff00862f740 1 --2- 172.21.15.43:0/4064885523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060edab60 0x561060ede0d0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7feff40bb1d0 tx=0x7feff4003680 comp rx=0 tx=0).stop 2024-04-21T08:41:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.566+0000 7ff00862f740 1 -- 172.21.15.43:0/4064885523 shutdown_connections 2024-04-21T08:41:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.566+0000 7ff00862f740 1 --2- 172.21.15.43:0/4064885523 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561060ee40e0 0x561060f5cb90 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.566+0000 7ff00862f740 1 --2- 172.21.15.43:0/4064885523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561060edb8d0 0x561060ede610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.566+0000 7ff00862f740 1 --2- 172.21.15.43:0/4064885523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561060edab60 0x561060ede0d0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.566+0000 7ff00862f740 1 --2- 172.21.15.43:0/4064885523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561060ea1c90 0x561060eddb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.566+0000 7fefdeffd640 0 client.16827 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.566+0000 7ff00862f740 1 -- 172.21.15.43:0/4064885523 >> 172.21.15.43:0/4064885523 conn(0x561060ee1450 msgr2=0x561060ed74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.566+0000 7ff00862f740 1 -- 172.21.15.43:0/4064885523 shutdown_connections 2024-04-21T08:41:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.566+0000 7ff00862f740 1 -- 172.21.15.43:0/4064885523 wait complete. 2024-04-21T08:41:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.567+0000 7ff00862f740 20 client.16827 trim_cache size 0 max 16384 2024-04-21T08:41:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:41:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:41:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:41:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:41:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.567+0000 7ff005a3f640 1 -- 172.21.15.43:0/3865986738 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fefe80a3dc0 msgr2=0x7fefe80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.567+0000 7ff005a3f640 1 --2- 172.21.15.43:0/3865986738 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fefe80a3dc0 0x7fefe80a6280 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7feff8007f70 tx=0x7feff8004a40 comp rx=0 tx=0).stop 2024-04-21T08:41:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.567+0000 7ff005a3f640 1 -- 172.21.15.43:0/3865986738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0001072a0 msgr2=0x7ff000198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.567+0000 7ff005a3f640 1 --2- 172.21.15.43:0/3865986738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0001072a0 0x7ff000198b50 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7feff4000c00 tx=0x7feff40045e0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.567+0000 7ff005a3f640 1 -- 172.21.15.43:0/3865986738 shutdown_connections 2024-04-21T08:41:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.567+0000 7ff005a3f640 1 --2- 172.21.15.43:0/3865986738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00019ced0 0x7ff00019d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.567+0000 7ff005a3f640 1 --2- 172.21.15.43:0/3865986738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0001072a0 0x7ff000198b50 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.567+0000 7ff005a3f640 1 --2- 172.21.15.43:0/3865986738 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fefe80a3dc0 0x7fefe80a6280 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.567+0000 7ff005a3f640 1 --2- 172.21.15.43:0/3865986738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0001021d0 0x7ff000198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.567+0000 7ff005a3f640 1 -- 172.21.15.43:0/3865986738 >> 172.21.15.43:0/3865986738 conn(0x7ff0000793e0 msgr2=0x7ff0000ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.567+0000 7ff005a3f640 1 -- 172.21.15.43:0/3865986738 shutdown_connections 2024-04-21T08:41:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:13.567+0000 7ff005a3f640 1 -- 172.21.15.43:0/3865986738 wait complete. 2024-04-21T08:41:14.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:14 smithi112 ceph-mon[31085]: pgmap v72: 129 pgs: 129 active+clean; 2.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-21T08:41:14.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:14 smithi043 ceph-mon[27978]: pgmap v72: 129 pgs: 129 active+clean; 2.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-21T08:41:14.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:14 smithi175 ceph-mon[31920]: pgmap v72: 129 pgs: 129 active+clean; 2.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-21T08:41:15.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:15 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:41:15.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:15 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:41:15.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:15 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:41:16.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:16 smithi112 ceph-mon[31085]: pgmap v73: 129 pgs: 129 active+clean; 2.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-21T08:41:16.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:16 smithi043 ceph-mon[27978]: pgmap v73: 129 pgs: 129 active+clean; 2.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-21T08:41:16.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:16 smithi175 ceph-mon[31920]: pgmap v73: 129 pgs: 129 active+clean; 2.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-21T08:41:18.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:18 smithi112 ceph-mon[31085]: pgmap v74: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 21 op/s 2024-04-21T08:41:18.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:18 smithi043 ceph-mon[27978]: pgmap v74: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 21 op/s 2024-04-21T08:41:18.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:18 smithi175 ceph-mon[31920]: pgmap v74: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 21 op/s 2024-04-21T08:41:20.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:20 smithi112 ceph-mon[31085]: pgmap v75: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-04-21T08:41:20.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:20 smithi043 ceph-mon[27978]: pgmap v75: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-04-21T08:41:20.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:20 smithi175 ceph-mon[31920]: pgmap v75: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-04-21T08:41:22.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:22 smithi112 ceph-mon[31085]: pgmap v76: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-21T08:41:22.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:22 smithi043 ceph-mon[27978]: pgmap v76: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-21T08:41:22.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:22 smithi175 ceph-mon[31920]: pgmap v76: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-21T08:41:24.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:24 smithi112 ceph-mon[31085]: pgmap v77: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-04-21T08:41:24.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:24 smithi043 ceph-mon[27978]: pgmap v77: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-04-21T08:41:24.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:24 smithi175 ceph-mon[31920]: pgmap v77: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-04-21T08:41:26.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:26 smithi043 ceph-mon[27978]: pgmap v78: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-04-21T08:41:26.369 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:26 smithi112 ceph-mon[31085]: pgmap v78: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-04-21T08:41:26.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:26 smithi175 ceph-mon[31920]: pgmap v78: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-04-21T08:41:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.667+0000 7f2307aa0640 1 -- 172.21.15.43:0/3558529862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23001016a0 msgr2=0x7f230010e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.667+0000 7f2307aa0640 1 --2- 172.21.15.43:0/3558529862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23001016a0 0x7f230010e3f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f22f4009900 tx=0x7f22f402bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.667+0000 7f2307aa0640 1 -- 172.21.15.43:0/3558529862 shutdown_connections 2024-04-21T08:41:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.667+0000 7f2307aa0640 1 --2- 172.21.15.43:0/3558529862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f230010e930 0x7f2300110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.667+0000 7f2307aa0640 1 --2- 172.21.15.43:0/3558529862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23001016a0 0x7f230010e3f0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.667+0000 7f2307aa0640 1 --2- 172.21.15.43:0/3558529862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2300100d80 0x7f2300101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.667+0000 7f2307aa0640 1 -- 172.21.15.43:0/3558529862 >> 172.21.15.43:0/3558529862 conn(0x7f2300077fd0 msgr2=0x7f23000783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.667+0000 7f2307aa0640 1 -- 172.21.15.43:0/3558529862 shutdown_connections 2024-04-21T08:41:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.667+0000 7f2307aa0640 1 -- 172.21.15.43:0/3558529862 wait complete. 2024-04-21T08:41:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.668+0000 7f2307aa0640 1 Processor -- start 2024-04-21T08:41:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.668+0000 7f2307aa0640 1 -- start start 2024-04-21T08:41:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.668+0000 7f2307aa0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2300100d80 0x7f23001a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.668+0000 7f2307aa0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23001016a0 0x7f23001a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.668+0000 7f2307aa0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f230010e930 0x7f230019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.668+0000 7f2307aa0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2300114430 con 0x7f2300100d80 2024-04-21T08:41:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.668+0000 7f2307aa0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f23001142b0 con 0x7f230010e930 2024-04-21T08:41:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.668+0000 7f2307aa0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23001145b0 con 0x7f23001016a0 2024-04-21T08:41:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.668+0000 7f230583c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2300100d80 0x7f23001a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.668+0000 7f230583c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2300100d80 0x7f23001a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49838/0 (socket says 172.21.15.43:49838) 2024-04-21T08:41:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.668+0000 7f230583c640 1 -- 172.21.15.43:0/3495931306 learned_addr learned my addr 172.21.15.43:0/3495931306 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.668+0000 7f230603d640 1 --2- 172.21.15.43:0/3495931306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f230010e930 0x7f230019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.668+0000 7f230503b640 1 --2- 172.21.15.43:0/3495931306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23001016a0 0x7f23001a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.668+0000 7f230583c640 1 -- 172.21.15.43:0/3495931306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23001016a0 msgr2=0x7f23001a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.668+0000 7f230583c640 1 --2- 172.21.15.43:0/3495931306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23001016a0 0x7f23001a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.668+0000 7f230583c640 1 -- 172.21.15.43:0/3495931306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f230010e930 msgr2=0x7f230019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.668+0000 7f230583c640 1 --2- 172.21.15.43:0/3495931306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f230010e930 0x7f230019a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.668+0000 7f230583c640 1 -- 172.21.15.43:0/3495931306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f230019b030 con 0x7f2300100d80 2024-04-21T08:41:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.669+0000 7f230583c640 1 --2- 172.21.15.43:0/3495931306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2300100d80 0x7f23001a06b0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f22f000ea80 tx=0x7f22f000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.669+0000 7f22feffd640 1 -- 172.21.15.43:0/3495931306 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f000ce40 con 0x7f2300100d80 2024-04-21T08:41:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.669+0000 7f22feffd640 1 -- 172.21.15.43:0/3495931306 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22f0005020 con 0x7f2300100d80 2024-04-21T08:41:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.669+0000 7f2307aa0640 1 -- 172.21.15.43:0/3495931306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f230019b320 con 0x7f2300100d80 2024-04-21T08:41:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.669+0000 7f22feffd640 1 -- 172.21.15.43:0/3495931306 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f00116d0 con 0x7f2300100d80 2024-04-21T08:41:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.669+0000 7f2307aa0640 1 -- 172.21.15.43:0/3495931306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23001a7630 con 0x7f2300100d80 2024-04-21T08:41:26.678 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.671+0000 7f22feffd640 1 -- 172.21.15.43:0/3495931306 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f22f00054d0 con 0x7f2300100d80 2024-04-21T08:41:26.678 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.672+0000 7f230a768740 1 Processor -- start 2024-04-21T08:41:26.678 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.672+0000 7f230a768740 1 -- start start 2024-04-21T08:41:26.678 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.672+0000 7f230a768740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650597748d0 0x565059773620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:26.678 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.672+0000 7f230a768740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565059773b60 0x565059773fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:26.678 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.672+0000 7f22feffd640 1 --2- 172.21.15.43:0/3495931306 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f22e80a3e40 0x7f22e80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:26.678 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.672+0000 7f230a768740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56505977d0e0 0x56505977f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:26.678 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.672+0000 7f230a768740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5650596d88d0 con 0x565059773b60 2024-04-21T08:41:26.678 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.672+0000 7f230a768740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x565059675e80 con 0x5650597748d0 2024-04-21T08:41:26.678 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.672+0000 7f230a768740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5650596d4890 con 0x56505977d0e0 2024-04-21T08:41:26.678 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.673+0000 7f230583c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565059773b60 0x565059773fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.673+0000 7f230603d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650597748d0 0x565059773620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.673+0000 7f230503b640 1 --2- 172.21.15.43:0/3495931306 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f22e80a3e40 0x7f22e80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.673+0000 7f230603d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650597748d0 0x565059773620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:48572/0 (socket says 172.21.15.43:48572) 2024-04-21T08:41:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.673+0000 7f230583c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565059773b60 0x565059773fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49842/0 (socket says 172.21.15.43:49842) 2024-04-21T08:41:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.673+0000 7f230603d640 1 -- 172.21.15.43:0/1494865441 learned_addr learned my addr 172.21.15.43:0/1494865441 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.673+0000 7f22feffd640 1 -- 172.21.15.43:0/3495931306 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f22f00d0dd0 con 0x7f2300100d80 2024-04-21T08:41:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.673+0000 7f230603d640 1 --2- 172.21.15.43:0/1494865441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56505977d0e0 0x56505977f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.673+0000 7f230603d640 1 -- 172.21.15.43:0/1494865441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56505977d0e0 msgr2=0x56505977f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.673+0000 7f230603d640 1 --2- 172.21.15.43:0/1494865441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56505977d0e0 0x56505977f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.673+0000 7f230603d640 1 -- 172.21.15.43:0/1494865441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565059773b60 msgr2=0x565059773fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.673+0000 7f230603d640 1 --2- 172.21.15.43:0/1494865441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565059773b60 0x565059773fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.673+0000 7f230603d640 1 -- 172.21.15.43:0/1494865441 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565059697320 con 0x5650597748d0 2024-04-21T08:41:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.673+0000 7f230603d640 1 --2- 172.21.15.43:0/1494865441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650597748d0 0x565059773620 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009f30 tx=0x7f22f8005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80469d64b6f63bac server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.673+0000 7f230503b640 1 --2- 172.21.15.43:0/3495931306 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f22e80a3e40 0x7f22e80a6300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f22f4005ab0 tx=0x7f22f4005a20 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.673+0000 7f22e7fff640 1 -- 172.21.15.43:0/1494865441 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f801e420 con 0x5650597748d0 2024-04-21T08:41:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.673+0000 7f22e7fff640 1 -- 172.21.15.43:0/1494865441 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22f800f870 con 0x5650597748d0 2024-04-21T08:41:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.673+0000 7f22e7fff640 1 -- 172.21.15.43:0/1494865441 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f800e6f0 con 0x5650597748d0 2024-04-21T08:41:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.674+0000 7f230a768740 1 -- 172.21.15.43:0/1494865441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650597748d0 msgr2=0x565059773620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.674+0000 7f230a768740 1 --2- 172.21.15.43:0/1494865441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650597748d0 0x565059773620 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009f30 tx=0x7f22f8005850 comp rx=0 tx=0).stop 2024-04-21T08:41:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.674+0000 7f230a768740 1 -- 172.21.15.43:0/1494865441 shutdown_connections 2024-04-21T08:41:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.674+0000 7f230a768740 1 --2- 172.21.15.43:0/1494865441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56505977d0e0 0x56505977f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.674+0000 7f230a768740 1 --2- 172.21.15.43:0/1494865441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565059773b60 0x565059773fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.674+0000 7f230a768740 1 --2- 172.21.15.43:0/1494865441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650597748d0 0x565059773620 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.674+0000 7f230a768740 1 -- 172.21.15.43:0/1494865441 >> 172.21.15.43:0/1494865441 conn(0x56505973ac90 msgr2=0x56505976e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.674+0000 7f230a768740 1 -- 172.21.15.43:0/1494865441 shutdown_connections 2024-04-21T08:41:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.674+0000 7f230a768740 1 -- 172.21.15.43:0/1494865441 wait complete. 2024-04-21T08:41:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.675+0000 7f230a768740 1 -- start start 2024-04-21T08:41:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.675+0000 7f230a768740 1 Processor -- start 2024-04-21T08:41:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.675+0000 7f230a768740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565059773b60 0x56505977cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.675+0000 7f230a768740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650597748d0 0x565059779e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.675+0000 7f230a768740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56505977d0e0 0x56505977a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.675+0000 7f230a768740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56505977fe10 con 0x5650597748d0 2024-04-21T08:41:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.675+0000 7f230a768740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x565059660eb0 con 0x565059773b60 2024-04-21T08:41:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.675+0000 7f230a768740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56505977ff90 con 0x56505977d0e0 2024-04-21T08:41:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.675+0000 7f230a768740 10 client.0 _ll_register_callbacks cb 0x56505968db40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.675+0000 7f230503b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650597748d0 0x565059779e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.675+0000 7f230503b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650597748d0 0x565059779e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49850/0 (socket says 172.21.15.43:49850) 2024-04-21T08:41:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.675+0000 7f230503b640 1 -- 172.21.15.43:0/1516536479 learned_addr learned my addr 172.21.15.43:0/1516536479 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.675+0000 7f230583c640 1 --2- 172.21.15.43:0/1516536479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565059773b60 0x56505977cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.676+0000 7f230603d640 1 --2- 172.21.15.43:0/1516536479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56505977d0e0 0x56505977a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.676+0000 7f230583c640 1 -- 172.21.15.43:0/1516536479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56505977d0e0 msgr2=0x56505977a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.676+0000 7f230583c640 1 --2- 172.21.15.43:0/1516536479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56505977d0e0 0x56505977a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.676+0000 7f230583c640 1 -- 172.21.15.43:0/1516536479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650597748d0 msgr2=0x565059779e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.676+0000 7f230583c640 1 --2- 172.21.15.43:0/1516536479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650597748d0 0x565059779e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.676+0000 7f230583c640 1 -- 172.21.15.43:0/1516536479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650593eb210 con 0x565059773b60 2024-04-21T08:41:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.676+0000 7f230503b640 1 --2- 172.21.15.43:0/1516536479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650597748d0 0x565059779e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:41:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.676+0000 7f230583c640 1 --2- 172.21.15.43:0/1516536479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565059773b60 0x56505977cac0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f22f00ce0f0 tx=0x7f22f006c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd6cab79dd2cd55a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.676+0000 7f22e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-21T08:41:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.676+0000 7f22e57fa640 1 -- 172.21.15.43:0/1516536479 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f001f040 con 0x565059773b60 2024-04-21T08:41:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.676+0000 7f230a768740 1 -- 172.21.15.43:0/1516536479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22f800f870 con 0x565059773b60 2024-04-21T08:41:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.676+0000 7f22e57fa640 1 -- 172.21.15.43:0/1516536479 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f22f00a5120 con 0x565059773b60 2024-04-21T08:41:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.676+0000 7f22e57fa640 1 -- 172.21.15.43:0/1516536479 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f22f00ce570 con 0x565059773b60 2024-04-21T08:41:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.676+0000 7f230a768740 10 client.26186 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.676+0000 7f230a768740 1 -- 172.21.15.43:0/1516536479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5650594b7620 con 0x565059773b60 2024-04-21T08:41:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.676+0000 7f22e57fa640 1 -- 172.21.15.43:0/1516536479 <== mon.2 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22f800f870 con 0x565059773b60 2024-04-21T08:41:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.677+0000 7f22e57fa640 1 -- 172.21.15.43:0/1516536479 <== mon.2 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f22f000f040 con 0x565059773b60 2024-04-21T08:41:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.677+0000 7f230a768740 10 client.26186 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.677+0000 7f230a768740 10 client.26186 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.677+0000 7f230a768740 20 client.26186 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.677+0000 7f230a768740 1 --2- 172.21.15.43:0/1516536479 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56505978d190 0x56505978f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.677+0000 7f230a768740 4 client.26186 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.677+0000 7f230a768740 1 -- 172.21.15.43:0/1516536479 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650596ec880 con 0x56505978d190 2024-04-21T08:41:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.677+0000 7f230503b640 1 --2- 172.21.15.43:0/1516536479 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56505978d190 0x56505978f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.677+0000 7f230503b640 1 --2- 172.21.15.43:0/1516536479 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56505978d190 0x56505978f580 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8b5c500485cac3e server_cookie=fa84b024d081accd in_seq=0 out_seq=0 2024-04-21T08:41:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.677+0000 7f22e57fa640 10 client.26186 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.678+0000 7f22e57fa640 1 -- 172.21.15.43:0/1516536479 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5650596ec880 con 0x56505978d190 2024-04-21T08:41:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.678+0000 7f22e57fa640 10 client.26186 handle_command_reply: tid=0 2024-04-21T08:41:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.678+0000 7f230a768740 1 client.26186 shutdown 2024-04-21T08:41:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.678+0000 7f230a768740 1 -- 172.21.15.43:0/1516536479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565059773b60 msgr2=0x56505977cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.678+0000 7f230a768740 1 --2- 172.21.15.43:0/1516536479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565059773b60 0x56505977cac0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f22f00ce0f0 tx=0x7f22f006c2d0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.678+0000 7f230a768740 1 -- 172.21.15.43:0/1516536479 shutdown_connections 2024-04-21T08:41:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.678+0000 7f230a768740 1 --2- 172.21.15.43:0/1516536479 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56505978d190 0x56505978f580 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.678+0000 7f230a768740 1 --2- 172.21.15.43:0/1516536479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56505977d0e0 0x56505977a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.678+0000 7f230a768740 1 --2- 172.21.15.43:0/1516536479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650597748d0 0x565059779e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.678+0000 7f22e57fa640 0 client.26186 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.678+0000 7f230a768740 1 --2- 172.21.15.43:0/1516536479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565059773b60 0x56505977cac0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.678+0000 7f230a768740 1 -- 172.21.15.43:0/1516536479 >> 172.21.15.43:0/1516536479 conn(0x56505973ac90 msgr2=0x5650597705a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.678+0000 7f230a768740 1 -- 172.21.15.43:0/1516536479 shutdown_connections 2024-04-21T08:41:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.678+0000 7f230a768740 1 -- 172.21.15.43:0/1516536479 wait complete. 2024-04-21T08:41:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.678+0000 7f230a768740 20 client.26186 trim_cache size 0 max 16384 2024-04-21T08:41:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.694+0000 7f230a768740 1 Processor -- start 2024-04-21T08:41:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.694+0000 7f230a768740 1 -- start start 2024-04-21T08:41:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.694+0000 7f230a768740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565059773b60 0x565059788d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.694+0000 7f230a768740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650597748d0 0x565059778e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.694+0000 7f230a768740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56505977d0e0 0x56505977c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.694+0000 7f230a768740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f23001b1610 con 0x56505977d0e0 2024-04-21T08:41:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.694+0000 7f230a768740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f23001b3150 con 0x565059773b60 2024-04-21T08:41:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.694+0000 7f230a768740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23001b1ce0 con 0x5650597748d0 2024-04-21T08:41:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.694+0000 7f230603d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56505977d0e0 0x56505977c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.694+0000 7f230603d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56505977d0e0 0x56505977c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49856/0 (socket says 172.21.15.43:49856) 2024-04-21T08:41:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.694+0000 7f230603d640 1 -- 172.21.15.43:0/2843495124 learned_addr learned my addr 172.21.15.43:0/2843495124 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.694+0000 7f230583c640 1 --2- 172.21.15.43:0/2843495124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565059773b60 0x565059788d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.694+0000 7f230503b640 1 --2- 172.21.15.43:0/2843495124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650597748d0 0x565059778e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.694+0000 7f230603d640 1 -- 172.21.15.43:0/2843495124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650597748d0 msgr2=0x565059778e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.694+0000 7f230603d640 1 --2- 172.21.15.43:0/2843495124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650597748d0 0x565059778e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.694+0000 7f230603d640 1 -- 172.21.15.43:0/2843495124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565059773b60 msgr2=0x565059788d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.694+0000 7f230603d640 1 --2- 172.21.15.43:0/2843495124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565059773b60 0x565059788d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.694+0000 7f230603d640 1 -- 172.21.15.43:0/2843495124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56505978f8f0 con 0x56505977d0e0 2024-04-21T08:41:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.694+0000 7f230503b640 1 --2- 172.21.15.43:0/2843495124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650597748d0 0x565059778e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.694+0000 7f230603d640 1 --2- 172.21.15.43:0/2843495124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56505977d0e0 0x56505977c580 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009fd0 tx=0x7f22f8009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85bbda3ac8fb5921 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.695+0000 7f22e77fe640 1 -- 172.21.15.43:0/2843495124 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f80053c0 con 0x56505977d0e0 2024-04-21T08:41:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.695+0000 7f22e77fe640 1 -- 172.21.15.43:0/2843495124 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22f8022030 con 0x56505977d0e0 2024-04-21T08:41:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.695+0000 7f22e77fe640 1 -- 172.21.15.43:0/2843495124 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f801e680 con 0x56505977d0e0 2024-04-21T08:41:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.695+0000 7f230a768740 1 -- 172.21.15.43:0/2843495124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56505977d0e0 msgr2=0x56505977c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.695+0000 7f230a768740 1 --2- 172.21.15.43:0/2843495124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56505977d0e0 0x56505977c580 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009fd0 tx=0x7f22f8009620 comp rx=0 tx=0).stop 2024-04-21T08:41:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.695+0000 7f230a768740 1 -- 172.21.15.43:0/2843495124 shutdown_connections 2024-04-21T08:41:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.695+0000 7f230a768740 1 --2- 172.21.15.43:0/2843495124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56505977d0e0 0x56505977c580 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.695+0000 7f230a768740 1 --2- 172.21.15.43:0/2843495124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650597748d0 0x565059778e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.695+0000 7f230a768740 1 --2- 172.21.15.43:0/2843495124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565059773b60 0x565059788d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.695+0000 7f230a768740 1 -- 172.21.15.43:0/2843495124 >> 172.21.15.43:0/2843495124 conn(0x56505973ac90 msgr2=0x565059778740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.695+0000 7f230a768740 1 -- 172.21.15.43:0/2843495124 shutdown_connections 2024-04-21T08:41:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.695+0000 7f230a768740 1 -- 172.21.15.43:0/2843495124 wait complete. 2024-04-21T08:41:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.696+0000 7f230a768740 1 -- start start 2024-04-21T08:41:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.696+0000 7f230a768740 1 Processor -- start 2024-04-21T08:41:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.696+0000 7f230a768740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565059773b60 0x565059785410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.696+0000 7f230a768740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650597748d0 0x5650597764f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.696+0000 7f230a768740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56505977d0e0 0x565059776a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.696+0000 7f230a768740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f23001b2a80 con 0x56505977d0e0 2024-04-21T08:41:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.696+0000 7f230a768740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f23001b23b0 con 0x565059773b60 2024-04-21T08:41:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.696+0000 7f230a768740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23001af5f0 con 0x5650597748d0 2024-04-21T08:41:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.696+0000 7f230a768740 10 client.0 _ll_register_callbacks cb 0x7f22f0098d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.696+0000 7f230603d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56505977d0e0 0x565059776a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.696+0000 7f230583c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565059773b60 0x565059785410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.696+0000 7f230583c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565059773b60 0x565059785410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:48596/0 (socket says 172.21.15.43:48596) 2024-04-21T08:41:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.696+0000 7f230603d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56505977d0e0 0x565059776a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49866/0 (socket says 172.21.15.43:49866) 2024-04-21T08:41:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.696+0000 7f230583c640 1 -- 172.21.15.43:0/87593087 learned_addr learned my addr 172.21.15.43:0/87593087 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.697+0000 7f230503b640 1 --2- 172.21.15.43:0/87593087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650597748d0 0x5650597764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.697+0000 7f230583c640 1 -- 172.21.15.43:0/87593087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650597748d0 msgr2=0x5650597764f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.697+0000 7f230583c640 1 --2- 172.21.15.43:0/87593087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650597748d0 0x5650597764f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.697+0000 7f230583c640 1 -- 172.21.15.43:0/87593087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56505977d0e0 msgr2=0x565059776a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.697+0000 7f230583c640 1 --2- 172.21.15.43:0/87593087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56505977d0e0 0x565059776a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.697+0000 7f230583c640 1 -- 172.21.15.43:0/87593087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565059776f70 con 0x565059773b60 2024-04-21T08:41:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.697+0000 7f230603d640 1 --2- 172.21.15.43:0/87593087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56505977d0e0 0x565059776a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:41:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.697+0000 7f230583c640 1 --2- 172.21.15.43:0/87593087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565059773b60 0x565059785410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f22f009aca0 tx=0x7f22f009c0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d21a47f32e6d8d3e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.697+0000 7f22e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-21T08:41:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.697+0000 7f22e57fa640 1 -- 172.21.15.43:0/87593087 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f009c530 con 0x565059773b60 2024-04-21T08:41:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.697+0000 7f22e57fa640 1 -- 172.21.15.43:0/87593087 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f22f00872c0 con 0x565059773b60 2024-04-21T08:41:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.697+0000 7f230a768740 1 -- 172.21.15.43:0/87593087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22f8022030 con 0x565059773b60 2024-04-21T08:41:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.697+0000 7f22e57fa640 1 -- 172.21.15.43:0/87593087 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f22f009c9f0 con 0x565059773b60 2024-04-21T08:41:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.697+0000 7f230a768740 10 client.26192 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.697+0000 7f230a768740 1 -- 172.21.15.43:0/87593087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5650597f3350 con 0x565059773b60 2024-04-21T08:41:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.697+0000 7f22e57fa640 1 -- 172.21.15.43:0/87593087 <== mon.2 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22f8022030 con 0x565059773b60 2024-04-21T08:41:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.697+0000 7f22e57fa640 1 -- 172.21.15.43:0/87593087 <== mon.2 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f22f009b8b0 con 0x565059773b60 2024-04-21T08:41:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.697+0000 7f230a768740 10 client.26192 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.697+0000 7f230a768740 10 client.26192 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.697+0000 7f230a768740 20 client.26192 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.697+0000 7f230a768740 1 --2- 172.21.15.43:0/87593087 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5650597f38c0 0x5650597f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.698+0000 7f230a768740 4 client.26192 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:41:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.698+0000 7f230a768740 1 -- 172.21.15.43:0/87593087 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5650596783d0 con 0x5650597f38c0 2024-04-21T08:41:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.698+0000 7f230503b640 1 --2- 172.21.15.43:0/87593087 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5650597f38c0 0x5650597f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.698+0000 7f230503b640 1 --2- 172.21.15.43:0/87593087 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5650597f38c0 0x5650597f5ce0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37c91bd88962929a server_cookie=169683630cbdfc70 in_seq=0 out_seq=0 2024-04-21T08:41:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.698+0000 7f22e57fa640 10 client.26192 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.708+0000 7f22e57fa640 1 -- 172.21.15.43:0/87593087 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5650596783d0 con 0x5650597f38c0 2024-04-21T08:41:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.708+0000 7f22e57fa640 10 client.26192 handle_command_reply: tid=0 2024-04-21T08:41:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.708+0000 7f230a768740 1 client.26192 shutdown 2024-04-21T08:41:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.708+0000 7f230a768740 1 -- 172.21.15.43:0/87593087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565059773b60 msgr2=0x565059785410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.708+0000 7f230a768740 1 --2- 172.21.15.43:0/87593087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565059773b60 0x565059785410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f22f009aca0 tx=0x7f22f009c0f0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.708+0000 7f230a768740 1 -- 172.21.15.43:0/87593087 shutdown_connections 2024-04-21T08:41:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.708+0000 7f230a768740 1 --2- 172.21.15.43:0/87593087 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5650597f38c0 0x5650597f5ce0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.708+0000 7f230a768740 1 --2- 172.21.15.43:0/87593087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56505977d0e0 0x565059776a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.708+0000 7f230a768740 1 --2- 172.21.15.43:0/87593087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5650597748d0 0x5650597764f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.708+0000 7f230a768740 1 --2- 172.21.15.43:0/87593087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565059773b60 0x565059785410 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.708+0000 7f22e57fa640 0 client.26192 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.708+0000 7f230a768740 1 -- 172.21.15.43:0/87593087 >> 172.21.15.43:0/87593087 conn(0x56505973ac90 msgr2=0x565059778fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.708+0000 7f230a768740 1 -- 172.21.15.43:0/87593087 shutdown_connections 2024-04-21T08:41:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.708+0000 7f230a768740 1 -- 172.21.15.43:0/87593087 wait complete. 2024-04-21T08:41:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.709+0000 7f230a768740 20 client.26192 trim_cache size 0 max 16384 2024-04-21T08:41:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:41:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:41:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:41:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:41:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.709+0000 7f2307aa0640 1 -- 172.21.15.43:0/3495931306 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f22e80a3e40 msgr2=0x7f22e80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.709+0000 7f2307aa0640 1 --2- 172.21.15.43:0/3495931306 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f22e80a3e40 0x7f22e80a6300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f22f4005ab0 tx=0x7f22f4005a20 comp rx=0 tx=0).stop 2024-04-21T08:41:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.709+0000 7f2307aa0640 1 -- 172.21.15.43:0/3495931306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2300100d80 msgr2=0x7f23001a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.709+0000 7f2307aa0640 1 --2- 172.21.15.43:0/3495931306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2300100d80 0x7f23001a06b0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f22f000ea80 tx=0x7f22f000ef40 comp rx=0 tx=0).stop 2024-04-21T08:41:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.709+0000 7f2307aa0640 1 -- 172.21.15.43:0/3495931306 shutdown_connections 2024-04-21T08:41:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.709+0000 7f2307aa0640 1 --2- 172.21.15.43:0/3495931306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f230010e930 0x7f230019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.709+0000 7f2307aa0640 1 --2- 172.21.15.43:0/3495931306 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f22e80a3e40 0x7f22e80a6300 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.709+0000 7f2307aa0640 1 --2- 172.21.15.43:0/3495931306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23001016a0 0x7f23001a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.709+0000 7f2307aa0640 1 --2- 172.21.15.43:0/3495931306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2300100d80 0x7f23001a06b0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.709+0000 7f2307aa0640 1 -- 172.21.15.43:0/3495931306 >> 172.21.15.43:0/3495931306 conn(0x7f2300077fd0 msgr2=0x7f230010fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.709+0000 7f2307aa0640 1 -- 172.21.15.43:0/3495931306 shutdown_connections 2024-04-21T08:41:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:26.709+0000 7f2307aa0640 1 -- 172.21.15.43:0/3495931306 wait complete. 2024-04-21T08:41:28.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:28 smithi043 ceph-mon[27978]: pgmap v79: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 17 op/s 2024-04-21T08:41:28.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:28 smithi175 ceph-mon[31920]: pgmap v79: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 17 op/s 2024-04-21T08:41:28.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:28 smithi112 ceph-mon[31085]: pgmap v79: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 17 op/s 2024-04-21T08:41:30.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:30 smithi043 ceph-mon[27978]: pgmap v80: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-21T08:41:30.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:30 smithi175 ceph-mon[31920]: pgmap v80: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-21T08:41:30.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:30 smithi112 ceph-mon[31085]: pgmap v80: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-21T08:41:32.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:32 smithi043 ceph-mon[27978]: pgmap v81: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-04-21T08:41:32.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:32 smithi175 ceph-mon[31920]: pgmap v81: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-04-21T08:41:32.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:32 smithi112 ceph-mon[31085]: pgmap v81: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-04-21T08:41:34.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:34 smithi043 ceph-mon[27978]: pgmap v82: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-04-21T08:41:34.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:34 smithi175 ceph-mon[31920]: pgmap v82: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-04-21T08:41:34.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:34 smithi112 ceph-mon[31085]: pgmap v82: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-04-21T08:41:35.211 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:41:35.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.327+0000 7f2e66c9a640 1 -- 172.21.15.43:0/3657726421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e601021f0 msgr2=0x7f2e60105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.327+0000 7f2e66c9a640 1 --2- 172.21.15.43:0/3657726421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e601021f0 0x7f2e60105a70 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009900 tx=0x7f2e4c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:35.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.327+0000 7f2e66c9a640 1 -- 172.21.15.43:0/3657726421 shutdown_connections 2024-04-21T08:41:35.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.327+0000 7f2e66c9a640 1 --2- 172.21.15.43:0/3657726421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e60106150 0x7f2e6010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.327+0000 7f2e66c9a640 1 --2- 172.21.15.43:0/3657726421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e601021f0 0x7f2e60105a70 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.327+0000 7f2e66c9a640 1 --2- 172.21.15.43:0/3657726421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e60101840 0x7f2e60101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.327+0000 7f2e66c9a640 1 -- 172.21.15.43:0/3657726421 >> 172.21.15.43:0/3657726421 conn(0x7f2e60077450 msgr2=0x7f2e600fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:35.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.327+0000 7f2e66c9a640 1 -- 172.21.15.43:0/3657726421 shutdown_connections 2024-04-21T08:41:35.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.327+0000 7f2e66c9a640 1 -- 172.21.15.43:0/3657726421 wait complete. 2024-04-21T08:41:35.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.328+0000 7f2e66c9a640 1 Processor -- start 2024-04-21T08:41:35.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.328+0000 7f2e66c9a640 1 -- start start 2024-04-21T08:41:35.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.328+0000 7f2e66c9a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e60101840 0x7f2e6019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.328+0000 7f2e66c9a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e60106150 0x7f2e6019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.328+0000 7f2e66c9a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e601985c0 0x7f2e60198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.328+0000 7f2e66c9a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2e60111090 con 0x7f2e601985c0 2024-04-21T08:41:35.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.328+0000 7f2e66c9a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e60110f10 con 0x7f2e60106150 2024-04-21T08:41:35.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.328+0000 7f2e66c9a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2e60111210 con 0x7f2e60101840 2024-04-21T08:41:35.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.328+0000 7f2e65237640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e601985c0 0x7f2e60198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.328+0000 7f2e65237640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e601985c0 0x7f2e60198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49876/0 (socket says 172.21.15.43:49876) 2024-04-21T08:41:35.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.328+0000 7f2e65237640 1 -- 172.21.15.43:0/2759382336 learned_addr learned my addr 172.21.15.43:0/2759382336 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:35.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.329+0000 7f2e64a36640 1 --2- 172.21.15.43:0/2759382336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e60101840 0x7f2e6019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.329+0000 7f2e5ffff640 1 --2- 172.21.15.43:0/2759382336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e60106150 0x7f2e6019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.329+0000 7f2e65237640 1 -- 172.21.15.43:0/2759382336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e60101840 msgr2=0x7f2e6019e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.329+0000 7f2e65237640 1 --2- 172.21.15.43:0/2759382336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e60101840 0x7f2e6019e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.329+0000 7f2e65237640 1 -- 172.21.15.43:0/2759382336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e60106150 msgr2=0x7f2e6019e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.329+0000 7f2e65237640 1 --2- 172.21.15.43:0/2759382336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e60106150 0x7f2e6019e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.329+0000 7f2e65237640 1 -- 172.21.15.43:0/2759382336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e601990e0 con 0x7f2e601985c0 2024-04-21T08:41:35.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.329+0000 7f2e5ffff640 1 --2- 172.21.15.43:0/2759382336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e60106150 0x7f2e6019e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:35.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.329+0000 7f2e64a36640 1 --2- 172.21.15.43:0/2759382336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e60101840 0x7f2e6019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:35.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.329+0000 7f2e65237640 1 --2- 172.21.15.43:0/2759382336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e601985c0 0x7f2e60198a70 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009fd0 tx=0x7f2e5800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:35.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.329+0000 7f2e5dffb640 1 -- 172.21.15.43:0/2759382336 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e58017070 con 0x7f2e601985c0 2024-04-21T08:41:35.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.329+0000 7f2e5dffb640 1 -- 172.21.15.43:0/2759382336 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e58003e20 con 0x7f2e601985c0 2024-04-21T08:41:35.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.329+0000 7f2e66c9a640 1 -- 172.21.15.43:0/2759382336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e601993d0 con 0x7f2e601985c0 2024-04-21T08:41:35.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.329+0000 7f2e5dffb640 1 -- 172.21.15.43:0/2759382336 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e58010460 con 0x7f2e601985c0 2024-04-21T08:41:35.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.329+0000 7f2e66c9a640 1 -- 172.21.15.43:0/2759382336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e601a55d0 con 0x7f2e601985c0 2024-04-21T08:41:35.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.330+0000 7f2e66c9a640 1 -- 172.21.15.43:0/2759382336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e60199af0 con 0x7f2e601985c0 2024-04-21T08:41:35.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.331+0000 7f2e5dffb640 1 -- 172.21.15.43:0/2759382336 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2e58003970 con 0x7f2e601985c0 2024-04-21T08:41:35.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.332+0000 7f2e5dffb640 1 --2- 172.21.15.43:0/2759382336 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2e500a3d20 0x7f2e500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.333+0000 7f2e64a36640 1 --2- 172.21.15.43:0/2759382336 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2e500a3d20 0x7f2e500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.333+0000 7f2e5dffb640 1 -- 172.21.15.43:0/2759382336 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f2e580d09c0 con 0x7f2e601985c0 2024-04-21T08:41:35.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.333+0000 7f2e64a36640 1 --2- 172.21.15.43:0/2759382336 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2e500a3d20 0x7f2e500a61e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2e540058c0 tx=0x7f2e54005850 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:35.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.337+0000 7f2e5dffb640 1 -- 172.21.15.43:0/2759382336 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2e580d2e20 con 0x7f2e601985c0 2024-04-21T08:41:35.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.566+0000 7f2e66c9a640 1 -- 172.21.15.43:0/2759382336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e60102400 con 0x7f2e601985c0 2024-04-21T08:41:35.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.566+0000 7f2e5dffb640 1 -- 172.21.15.43:0/2759382336 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7f2e5809cc90 con 0x7f2e601985c0 2024-04-21T08:41:35.566 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:41:35.566 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:41:35.566 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:41:35.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.568+0000 7f2e66c9a640 1 -- 172.21.15.43:0/2759382336 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2e500a3d20 msgr2=0x7f2e500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.568+0000 7f2e66c9a640 1 --2- 172.21.15.43:0/2759382336 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2e500a3d20 0x7f2e500a61e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2e540058c0 tx=0x7f2e54005850 comp rx=0 tx=0).stop 2024-04-21T08:41:35.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.568+0000 7f2e66c9a640 1 -- 172.21.15.43:0/2759382336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e601985c0 msgr2=0x7f2e60198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.569+0000 7f2e66c9a640 1 --2- 172.21.15.43:0/2759382336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e601985c0 0x7f2e60198a70 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009fd0 tx=0x7f2e5800b260 comp rx=0 tx=0).stop 2024-04-21T08:41:35.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.569+0000 7f2e66c9a640 1 -- 172.21.15.43:0/2759382336 shutdown_connections 2024-04-21T08:41:35.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.569+0000 7f2e66c9a640 1 --2- 172.21.15.43:0/2759382336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e601985c0 0x7f2e60198a70 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.569+0000 7f2e66c9a640 1 --2- 172.21.15.43:0/2759382336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e60106150 0x7f2e6019e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.569+0000 7f2e66c9a640 1 --2- 172.21.15.43:0/2759382336 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2e500a3d20 0x7f2e500a61e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.569+0000 7f2e66c9a640 1 --2- 172.21.15.43:0/2759382336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e60101840 0x7f2e6019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.569+0000 7f2e66c9a640 1 -- 172.21.15.43:0/2759382336 >> 172.21.15.43:0/2759382336 conn(0x7f2e60077450 msgr2=0x7f2e601007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:35.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.569+0000 7f2e66c9a640 1 -- 172.21.15.43:0/2759382336 shutdown_connections 2024-04-21T08:41:35.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.569+0000 7f2e66c9a640 1 -- 172.21.15.43:0/2759382336 wait complete. 2024-04-21T08:41:35.579 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db --set-id 52e10dcd --release --await' 2024-04-21T08:41:35.579 DEBUG:teuthology.orchestra.run.smithi043:> timeout 45 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24527 quiesce db --set-id 52e10dcd --release --await 2024-04-21T08:41:35.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.689+0000 7faa54f2b640 1 -- 172.21.15.43:0/1878822420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa50101090 msgr2=0x7faa50101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.689+0000 7faa54f2b640 1 --2- 172.21.15.43:0/1878822420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa50101090 0x7faa50101470 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7faa40009900 tx=0x7faa4002bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:35.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.689+0000 7faa54f2b640 1 -- 172.21.15.43:0/1878822420 shutdown_connections 2024-04-21T08:41:35.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.689+0000 7faa54f2b640 1 --2- 172.21.15.43:0/1878822420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa50106160 0x7faa5010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.689+0000 7faa54f2b640 1 --2- 172.21.15.43:0/1878822420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa50101a40 0x7faa50105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.689+0000 7faa54f2b640 1 --2- 172.21.15.43:0/1878822420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa50101090 0x7faa50101470 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.689+0000 7faa54f2b640 1 -- 172.21.15.43:0/1878822420 >> 172.21.15.43:0/1878822420 conn(0x7faa50077f50 msgr2=0x7faa50078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.689+0000 7faa54f2b640 1 -- 172.21.15.43:0/1878822420 shutdown_connections 2024-04-21T08:41:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.689+0000 7faa54f2b640 1 -- 172.21.15.43:0/1878822420 wait complete. 2024-04-21T08:41:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.689+0000 7faa54f2b640 1 Processor -- start 2024-04-21T08:41:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.689+0000 7faa54f2b640 1 -- start start 2024-04-21T08:41:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.690+0000 7faa54f2b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa50101090 0x7faa50198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.690+0000 7faa54f2b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa50106160 0x7faa50198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.690+0000 7faa54f2b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa5019d020 0x7faa5019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.690+0000 7faa54f2b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa501121e0 con 0x7faa50106160 2024-04-21T08:41:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.690+0000 7faa54f2b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faa50112060 con 0x7faa50101090 2024-04-21T08:41:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.690+0000 7faa54f2b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faa50112360 con 0x7faa5019d020 2024-04-21T08:41:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.690+0000 7faa4dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa50106160 0x7faa50198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.690+0000 7faa4dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa50106160 0x7faa50198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49894/0 (socket says 172.21.15.43:49894) 2024-04-21T08:41:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.690+0000 7faa4dd9b640 1 -- 172.21.15.43:0/1550729701 learned_addr learned my addr 172.21.15.43:0/1550729701 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.690+0000 7faa4e59c640 1 --2- 172.21.15.43:0/1550729701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa50101090 0x7faa50198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.690+0000 7faa4ed9d640 1 --2- 172.21.15.43:0/1550729701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa5019d020 0x7faa5019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.690+0000 7faa4dd9b640 1 -- 172.21.15.43:0/1550729701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa5019d020 msgr2=0x7faa5019d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.690+0000 7faa4dd9b640 1 --2- 172.21.15.43:0/1550729701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa5019d020 0x7faa5019d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.690+0000 7faa4dd9b640 1 -- 172.21.15.43:0/1550729701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa50101090 msgr2=0x7faa50198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.690+0000 7faa4dd9b640 1 --2- 172.21.15.43:0/1550729701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa50101090 0x7faa50198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.690+0000 7faa4dd9b640 1 -- 172.21.15.43:0/1550729701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5019db10 con 0x7faa50106160 2024-04-21T08:41:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.690+0000 7faa4ed9d640 1 --2- 172.21.15.43:0/1550729701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa5019d020 0x7faa5019d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.690+0000 7faa4e59c640 1 --2- 172.21.15.43:0/1550729701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa50101090 0x7faa50198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.690+0000 7faa4dd9b640 1 --2- 172.21.15.43:0/1550729701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa50106160 0x7faa50198ca0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7faa4400c970 tx=0x7faa4400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.690+0000 7faa3f7fe640 1 -- 172.21.15.43:0/1550729701 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa44007bb0 con 0x7faa50106160 2024-04-21T08:41:35.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.690+0000 7faa3f7fe640 1 -- 172.21.15.43:0/1550729701 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa44007d50 con 0x7faa50106160 2024-04-21T08:41:35.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.690+0000 7faa54f2b640 1 -- 172.21.15.43:0/1550729701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa5019de00 con 0x7faa50106160 2024-04-21T08:41:35.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.690+0000 7faa3f7fe640 1 -- 172.21.15.43:0/1550729701 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4400f450 con 0x7faa50106160 2024-04-21T08:41:35.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.690+0000 7faa54f2b640 1 -- 172.21.15.43:0/1550729701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa501a55f0 con 0x7faa50106160 2024-04-21T08:41:35.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.693+0000 7faa3f7fe640 1 -- 172.21.15.43:0/1550729701 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faa44004f20 con 0x7faa50106160 2024-04-21T08:41:35.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.693+0000 7faa57bfa740 1 Processor -- start 2024-04-21T08:41:35.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.693+0000 7faa57bfa740 1 -- start start 2024-04-21T08:41:35.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.693+0000 7faa57bfa740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f03843310 0x559f038436f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.693+0000 7faa57bfa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843c30 0x559f0384c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.693+0000 7faa57bfa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f0384c810 0x559f0384ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.693+0000 7faa3f7fe640 1 --2- 172.21.15.43:0/1550729701 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faa380a3d20 0x7faa380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.693+0000 7faa57bfa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559f037a8500 con 0x559f03843c30 2024-04-21T08:41:35.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.693+0000 7faa57bfa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559f0374e6a0 con 0x559f0384c810 2024-04-21T08:41:35.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.693+0000 7faa57bfa740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559f037a44c0 con 0x559f03843310 2024-04-21T08:41:35.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.694+0000 7faa4e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843c30 0x559f0384c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.694+0000 7faa4e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843c30 0x559f0384c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49908/0 (socket says 172.21.15.43:49908) 2024-04-21T08:41:35.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.694+0000 7faa4e59c640 1 -- 172.21.15.43:0/3004793799 learned_addr learned my addr 172.21.15.43:0/3004793799 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:35.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.694+0000 7faa4ed9d640 1 --2- 172.21.15.43:0/1550729701 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faa380a3d20 0x7faa380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.694+0000 7faa3f7fe640 1 -- 172.21.15.43:0/1550729701 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7faa440d0cb0 con 0x7faa50106160 2024-04-21T08:41:35.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.694+0000 7faa4dd9b640 1 --2- 172.21.15.43:0/3004793799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f0384c810 0x559f0384ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.694+0000 7faa4ed9d640 1 --2- 172.21.15.43:0/3004793799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f03843310 0x559f038436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.694+0000 7faa4e59c640 1 -- 172.21.15.43:0/3004793799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f03843310 msgr2=0x559f038436f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:35.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.694+0000 7faa4e59c640 1 --2- 172.21.15.43:0/3004793799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f03843310 0x559f038436f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.694+0000 7faa4e59c640 1 -- 172.21.15.43:0/3004793799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f0384c810 msgr2=0x559f0384ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:35.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.694+0000 7faa4e59c640 1 --2- 172.21.15.43:0/3004793799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f0384c810 0x559f0384ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.694+0000 7faa4e59c640 1 -- 172.21.15.43:0/3004793799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f0376d5e0 con 0x559f03843c30 2024-04-21T08:41:35.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.694+0000 7faa4dd9b640 1 --2- 172.21.15.43:0/3004793799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f0384c810 0x559f0384ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:35.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.694+0000 7faa4ed9d640 1 --2- 172.21.15.43:0/1550729701 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faa380a3d20 0x7faa380a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faa48005c40 tx=0x7faa48009960 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:35.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.694+0000 7faa4e59c640 1 --2- 172.21.15.43:0/3004793799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843c30 0x559f0384c2d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7faa40000c00 tx=0x7faa40005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384c3c3348d87e6f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:35.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.694+0000 7faa3cff9640 1 -- 172.21.15.43:0/3004793799 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40044020 con 0x559f03843c30 2024-04-21T08:41:35.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.694+0000 7faa3cff9640 1 -- 172.21.15.43:0/3004793799 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa4002cc90 con 0x559f03843c30 2024-04-21T08:41:35.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.694+0000 7faa3cff9640 1 -- 172.21.15.43:0/3004793799 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa400046c0 con 0x559f03843c30 2024-04-21T08:41:35.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.695+0000 7faa57bfa740 1 -- 172.21.15.43:0/3004793799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843c30 msgr2=0x559f0384c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.695+0000 7faa57bfa740 1 --2- 172.21.15.43:0/3004793799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843c30 0x559f0384c2d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7faa40000c00 tx=0x7faa40005e80 comp rx=0 tx=0).stop 2024-04-21T08:41:35.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.695+0000 7faa57bfa740 1 -- 172.21.15.43:0/3004793799 shutdown_connections 2024-04-21T08:41:35.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.695+0000 7faa57bfa740 1 --2- 172.21.15.43:0/3004793799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f0384c810 0x559f0384ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.695+0000 7faa57bfa740 1 --2- 172.21.15.43:0/3004793799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843c30 0x559f0384c2d0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.695+0000 7faa57bfa740 1 --2- 172.21.15.43:0/3004793799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f03843310 0x559f038436f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.695+0000 7faa57bfa740 1 -- 172.21.15.43:0/3004793799 >> 172.21.15.43:0/3004793799 conn(0x559f0383da40 msgr2=0x559f0383de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:35.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.695+0000 7faa57bfa740 1 -- 172.21.15.43:0/3004793799 shutdown_connections 2024-04-21T08:41:35.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.695+0000 7faa57bfa740 1 -- 172.21.15.43:0/3004793799 wait complete. 2024-04-21T08:41:35.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.695+0000 7faa57bfa740 1 -- start start 2024-04-21T08:41:35.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.696+0000 7faa57bfa740 1 Processor -- start 2024-04-21T08:41:35.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.696+0000 7faa57bfa740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f03843310 0x559f0384c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.696+0000 7faa57bfa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843c30 0x559f03849620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.696+0000 7faa57bfa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f0384c810 0x559f03849d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.696+0000 7faa57bfa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559f0384f5a0 con 0x559f03843c30 2024-04-21T08:41:35.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.696+0000 7faa57bfa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559f03761a60 con 0x559f0384c810 2024-04-21T08:41:35.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.696+0000 7faa57bfa740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559f0384f720 con 0x559f03843310 2024-04-21T08:41:35.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.696+0000 7faa57bfa740 10 client.0 _ll_register_callbacks cb 0x559f034ba110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:35.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.696+0000 7faa4dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843c30 0x559f03849620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.696+0000 7faa4dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843c30 0x559f03849620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49924/0 (socket says 172.21.15.43:49924) 2024-04-21T08:41:35.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.696+0000 7faa4dd9b640 1 -- 172.21.15.43:0/764970063 learned_addr learned my addr 172.21.15.43:0/764970063 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:35.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.696+0000 7faa4e59c640 1 --2- 172.21.15.43:0/764970063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f03843310 0x559f0384c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.696+0000 7faa4ed9d640 1 --2- 172.21.15.43:0/764970063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f0384c810 0x559f03849d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.696+0000 7faa4dd9b640 1 -- 172.21.15.43:0/764970063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f03843310 msgr2=0x559f0384c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:35.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.696+0000 7faa4dd9b640 1 --2- 172.21.15.43:0/764970063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f03843310 0x559f0384c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.696+0000 7faa4dd9b640 1 -- 172.21.15.43:0/764970063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f0384c810 msgr2=0x559f03849d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:35.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.696+0000 7faa4dd9b640 1 --2- 172.21.15.43:0/764970063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f0384c810 0x559f03849d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.696+0000 7faa4dd9b640 1 -- 172.21.15.43:0/764970063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f034ba920 con 0x559f03843c30 2024-04-21T08:41:35.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.696+0000 7faa4dd9b640 1 --2- 172.21.15.43:0/764970063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843c30 0x559f03849620 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7faa440ce0e0 tx=0x7faa4409d0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d2977614627540 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:35.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.696+0000 7faa35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:35.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.697+0000 7faa35ffb640 1 -- 172.21.15.43:0/764970063 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4401d040 con 0x559f03843c30 2024-04-21T08:41:35.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.697+0000 7faa35ffb640 1 -- 172.21.15.43:0/764970063 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7faa44098ef0 con 0x559f03843c30 2024-04-21T08:41:35.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.697+0000 7faa57bfa740 1 -- 172.21.15.43:0/764970063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa4002cc90 con 0x559f03843c30 2024-04-21T08:41:35.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.697+0000 7faa35ffb640 1 -- 172.21.15.43:0/764970063 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faa440ba050 con 0x559f03843c30 2024-04-21T08:41:35.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.697+0000 7faa57bfa740 10 client.16863 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:35.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.697+0000 7faa57bfa740 1 -- 172.21.15.43:0/764970063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559f03586d30 con 0x559f03843c30 2024-04-21T08:41:35.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.697+0000 7faa35ffb640 1 -- 172.21.15.43:0/764970063 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa4002cc90 con 0x559f03843c30 2024-04-21T08:41:35.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.697+0000 7faa35ffb640 1 -- 172.21.15.43:0/764970063 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faa440a5060 con 0x559f03843c30 2024-04-21T08:41:35.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.697+0000 7faa57bfa740 10 client.16863 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:35.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.697+0000 7faa57bfa740 10 client.16863 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-21T08:41:35.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.697+0000 7faa57bfa740 20 client.16863 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:35.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.697+0000 7faa57bfa740 1 --2- 172.21.15.43:0/764970063 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559f0385c8d0 0x559f0385ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.697+0000 7faa57bfa740 4 client.16863 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:35.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.697+0000 7faa57bfa740 1 -- 172.21.15.43:0/764970063 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f0377ba10 con 0x559f0385c8d0 2024-04-21T08:41:35.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.697+0000 7faa4e59c640 1 --2- 172.21.15.43:0/764970063 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559f0385c8d0 0x559f0385ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.698+0000 7faa4e59c640 1 --2- 172.21.15.43:0/764970063 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559f0385c8d0 0x559f0385ed20 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da626ef57f3f23f6 server_cookie=d30e058c2ee7a772 in_seq=0 out_seq=0 2024-04-21T08:41:35.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.698+0000 7faa35ffb640 10 client.16863 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:35.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.698+0000 7faa35ffb640 1 -- 172.21.15.43:0/764970063 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f0377ba10 con 0x559f0385c8d0 2024-04-21T08:41:35.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.698+0000 7faa35ffb640 10 client.16863 handle_command_reply: tid=0 2024-04-21T08:41:35.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.699+0000 7faa57bfa740 1 client.16863 shutdown 2024-04-21T08:41:35.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.699+0000 7faa57bfa740 1 -- 172.21.15.43:0/764970063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843c30 msgr2=0x559f03849620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.699+0000 7faa57bfa740 1 --2- 172.21.15.43:0/764970063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843c30 0x559f03849620 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7faa440ce0e0 tx=0x7faa4409d0c0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.699+0000 7faa57bfa740 1 -- 172.21.15.43:0/764970063 shutdown_connections 2024-04-21T08:41:35.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.699+0000 7faa57bfa740 1 --2- 172.21.15.43:0/764970063 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559f0385c8d0 0x559f0385ed20 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.699+0000 7faa57bfa740 1 --2- 172.21.15.43:0/764970063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f0384c810 0x559f03849d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.699+0000 7faa57bfa740 1 --2- 172.21.15.43:0/764970063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843c30 0x559f03849620 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.699+0000 7faa35ffb640 0 client.16863 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:35.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.699+0000 7faa57bfa740 1 --2- 172.21.15.43:0/764970063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f03843310 0x559f0384c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.699+0000 7faa57bfa740 1 -- 172.21.15.43:0/764970063 >> 172.21.15.43:0/764970063 conn(0x559f0383da40 msgr2=0x559f0383f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:35.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.699+0000 7faa57bfa740 1 -- 172.21.15.43:0/764970063 shutdown_connections 2024-04-21T08:41:35.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.699+0000 7faa57bfa740 1 -- 172.21.15.43:0/764970063 wait complete. 2024-04-21T08:41:35.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.699+0000 7faa57bfa740 20 client.16863 trim_cache size 0 max 16384 2024-04-21T08:41:35.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.716+0000 7faa57bfa740 1 Processor -- start 2024-04-21T08:41:35.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.716+0000 7faa57bfa740 1 -- start start 2024-04-21T08:41:35.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.716+0000 7faa57bfa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843310 0x559f0384bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.716+0000 7faa57bfa740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f03843c30 0x559f0384c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.716+0000 7faa57bfa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f0384c810 0x559f038645f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.716+0000 7faa57bfa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa501af400 con 0x559f03843310 2024-04-21T08:41:35.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.716+0000 7faa57bfa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faa501b0f40 con 0x559f0384c810 2024-04-21T08:41:35.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.716+0000 7faa57bfa740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faa501afad0 con 0x559f03843c30 2024-04-21T08:41:35.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.716+0000 7faa4e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843310 0x559f0384bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.716+0000 7faa4e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843310 0x559f0384bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49932/0 (socket says 172.21.15.43:49932) 2024-04-21T08:41:35.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.716+0000 7faa4e59c640 1 -- 172.21.15.43:0/2857479473 learned_addr learned my addr 172.21.15.43:0/2857479473 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:35.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.716+0000 7faa4dd9b640 1 --2- 172.21.15.43:0/2857479473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f03843c30 0x559f0384c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.717+0000 7faa4ed9d640 1 --2- 172.21.15.43:0/2857479473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f0384c810 0x559f038645f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.717+0000 7faa4e59c640 1 -- 172.21.15.43:0/2857479473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f03843c30 msgr2=0x559f0384c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:35.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.717+0000 7faa4e59c640 1 --2- 172.21.15.43:0/2857479473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f03843c30 0x559f0384c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.717+0000 7faa4e59c640 1 -- 172.21.15.43:0/2857479473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f0384c810 msgr2=0x559f038645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:35.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.717+0000 7faa4e59c640 1 --2- 172.21.15.43:0/2857479473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f0384c810 0x559f038645f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.717+0000 7faa4e59c640 1 -- 172.21.15.43:0/2857479473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f03843a60 con 0x559f03843310 2024-04-21T08:41:35.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.717+0000 7faa4ed9d640 1 --2- 172.21.15.43:0/2857479473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f0384c810 0x559f038645f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:35.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.717+0000 7faa4e59c640 1 --2- 172.21.15.43:0/2857479473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843310 0x559f0384bd50 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7faa40045000 tx=0x7faa40005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e652bc06eddb46 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:35.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.717+0000 7faa37fff640 1 -- 172.21.15.43:0/2857479473 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40044040 con 0x559f03843310 2024-04-21T08:41:35.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.717+0000 7faa37fff640 1 -- 172.21.15.43:0/2857479473 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa400044a0 con 0x559f03843310 2024-04-21T08:41:35.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.717+0000 7faa37fff640 1 -- 172.21.15.43:0/2857479473 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa400407b0 con 0x559f03843310 2024-04-21T08:41:35.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.717+0000 7faa57bfa740 1 -- 172.21.15.43:0/2857479473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843310 msgr2=0x559f0384bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.717+0000 7faa57bfa740 1 --2- 172.21.15.43:0/2857479473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843310 0x559f0384bd50 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7faa40045000 tx=0x7faa40005fd0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.717+0000 7faa57bfa740 1 -- 172.21.15.43:0/2857479473 shutdown_connections 2024-04-21T08:41:35.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.717+0000 7faa57bfa740 1 --2- 172.21.15.43:0/2857479473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f0384c810 0x559f038645f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.717+0000 7faa57bfa740 1 --2- 172.21.15.43:0/2857479473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f03843c30 0x559f0384c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.717+0000 7faa57bfa740 1 --2- 172.21.15.43:0/2857479473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843310 0x559f0384bd50 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.717+0000 7faa57bfa740 1 -- 172.21.15.43:0/2857479473 >> 172.21.15.43:0/2857479473 conn(0x559f0383da40 msgr2=0x559f0383cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:35.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.718+0000 7faa57bfa740 1 -- 172.21.15.43:0/2857479473 shutdown_connections 2024-04-21T08:41:35.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.718+0000 7faa57bfa740 1 -- 172.21.15.43:0/2857479473 wait complete. 2024-04-21T08:41:35.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.718+0000 7faa57bfa740 1 -- start start 2024-04-21T08:41:35.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.718+0000 7faa57bfa740 1 Processor -- start 2024-04-21T08:41:35.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.718+0000 7faa57bfa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f03843310 0x559f03854d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.718+0000 7faa57bfa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843c30 0x559f0384f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.718+0000 7faa57bfa740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f0384c810 0x559f0384fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.718+0000 7faa57bfa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa501b0870 con 0x559f03843c30 2024-04-21T08:41:35.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.718+0000 7faa57bfa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faa501b01a0 con 0x559f03843310 2024-04-21T08:41:35.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.718+0000 7faa57bfa740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faa501ad3e0 con 0x559f0384c810 2024-04-21T08:41:35.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.718+0000 7faa57bfa740 10 client.0 _ll_register_callbacks cb 0x7faa44098950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:35.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.718+0000 7faa4dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843c30 0x559f0384f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.718+0000 7faa4dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843c30 0x559f0384f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49938/0 (socket says 172.21.15.43:49938) 2024-04-21T08:41:35.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.718+0000 7faa4dd9b640 1 -- 172.21.15.43:0/656712383 learned_addr learned my addr 172.21.15.43:0/656712383 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:35.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.718+0000 7faa4e59c640 1 --2- 172.21.15.43:0/656712383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f03843310 0x559f03854d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.718+0000 7faa4dd9b640 1 -- 172.21.15.43:0/656712383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f0384c810 msgr2=0x559f0384fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:35.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.718+0000 7faa4ed9d640 1 --2- 172.21.15.43:0/656712383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f0384c810 0x559f0384fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.718+0000 7faa4dd9b640 1 --2- 172.21.15.43:0/656712383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f0384c810 0x559f0384fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.718+0000 7faa4dd9b640 1 -- 172.21.15.43:0/656712383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f03843310 msgr2=0x559f03854d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:35.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.718+0000 7faa4dd9b640 1 --2- 172.21.15.43:0/656712383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f03843310 0x559f03854d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.719+0000 7faa4dd9b640 1 -- 172.21.15.43:0/656712383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f038506a0 con 0x559f03843c30 2024-04-21T08:41:35.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.719+0000 7faa4e59c640 1 --2- 172.21.15.43:0/656712383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f03843310 0x559f03854d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:35.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.719+0000 7faa4dd9b640 1 --2- 172.21.15.43:0/656712383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843c30 0x559f0384f930 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7faa440ce630 tx=0x7faa4409ad90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30ad7f34da7d253 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:35.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.719+0000 7faa35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:35.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.719+0000 7faa35ffb640 1 -- 172.21.15.43:0/656712383 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4401d030 con 0x559f03843c30 2024-04-21T08:41:35.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.719+0000 7faa35ffb640 1 -- 172.21.15.43:0/656712383 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7faa44098ef0 con 0x559f03843c30 2024-04-21T08:41:35.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.719+0000 7faa57bfa740 1 -- 172.21.15.43:0/656712383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa400044a0 con 0x559f03843c30 2024-04-21T08:41:35.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.719+0000 7faa35ffb640 1 -- 172.21.15.43:0/656712383 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faa4409c1e0 con 0x559f03843c30 2024-04-21T08:41:35.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.719+0000 7faa57bfa740 10 client.16869 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:35.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.719+0000 7faa57bfa740 1 -- 172.21.15.43:0/656712383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559f038c3da0 con 0x559f03843c30 2024-04-21T08:41:35.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.719+0000 7faa35ffb640 1 -- 172.21.15.43:0/656712383 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa400044a0 con 0x559f03843c30 2024-04-21T08:41:35.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.719+0000 7faa35ffb640 1 -- 172.21.15.43:0/656712383 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faa440870b0 con 0x559f03843c30 2024-04-21T08:41:35.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.719+0000 7faa57bfa740 10 client.16869 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:35.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.719+0000 7faa57bfa740 10 client.16869 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-21T08:41:35.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.719+0000 7faa57bfa740 20 client.16869 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:35.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.719+0000 7faa57bfa740 1 --2- 172.21.15.43:0/656712383 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559f038c4300 0x559f038c6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.719+0000 7faa57bfa740 4 client.16869 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "52e10dcd", "release": true, "await": true}] 2024-04-21T08:41:35.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.719+0000 7faa57bfa740 1 -- 172.21.15.43:0/656712383 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "quiesce db", "set_id": "52e10dcd", "release": true, "await": true}) -- 0x559f037b5f20 con 0x559f038c4300 2024-04-21T08:41:35.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.720+0000 7faa4e59c640 1 --2- 172.21.15.43:0/656712383 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559f038c4300 0x559f038c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.720+0000 7faa4e59c640 1 --2- 172.21.15.43:0/656712383 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559f038c4300 0x559f038c6720 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c1ddc7888751b7a server_cookie=ccc0831d32c11fe5 in_seq=0 out_seq=0 2024-04-21T08:41:35.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.720+0000 7faa35ffb640 10 client.16869 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:35.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.724+0000 7faa35ffb640 1 -- 172.21.15.43:0/656712383 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x559f037b5f20 con 0x559f038c4300 2024-04-21T08:41:35.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.724+0000 7faa35ffb640 10 client.16869 handle_command_reply: tid=0 2024-04-21T08:41:35.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.724+0000 7faa57bfa740 1 client.16869 shutdown 2024-04-21T08:41:35.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.724+0000 7faa57bfa740 1 -- 172.21.15.43:0/656712383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843c30 msgr2=0x559f0384f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.724+0000 7faa57bfa740 1 --2- 172.21.15.43:0/656712383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843c30 0x559f0384f930 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7faa440ce630 tx=0x7faa4409ad90 comp rx=0 tx=0).stop 2024-04-21T08:41:35.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.724+0000 7faa57bfa740 1 -- 172.21.15.43:0/656712383 shutdown_connections 2024-04-21T08:41:35.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.724+0000 7faa57bfa740 1 --2- 172.21.15.43:0/656712383 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559f038c4300 0x559f038c6720 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.724+0000 7faa57bfa740 1 --2- 172.21.15.43:0/656712383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f0384c810 0x559f0384fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.724+0000 7faa35ffb640 0 client.16869 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:35.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.724+0000 7faa57bfa740 1 --2- 172.21.15.43:0/656712383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559f03843c30 0x559f0384f930 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.724+0000 7faa57bfa740 1 --2- 172.21.15.43:0/656712383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f03843310 0x559f03854d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.724+0000 7faa57bfa740 1 -- 172.21.15.43:0/656712383 >> 172.21.15.43:0/656712383 conn(0x559f0383da40 msgr2=0x559f0383ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:35.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.724+0000 7faa57bfa740 1 -- 172.21.15.43:0/656712383 shutdown_connections 2024-04-21T08:41:35.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.724+0000 7faa57bfa740 1 -- 172.21.15.43:0/656712383 wait complete. 2024-04-21T08:41:35.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.724+0000 7faa57bfa740 20 client.16869 trim_cache size 0 max 16384 2024-04-21T08:41:35.727 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:41:35.727 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 28, 2024-04-21T08:41:35.728 INFO:teuthology.orchestra.run.smithi043.stdout: "leader": 24527, 2024-04-21T08:41:35.728 INFO:teuthology.orchestra.run.smithi043.stdout: "set_version": 4, 2024-04-21T08:41:35.728 INFO:teuthology.orchestra.run.smithi043.stdout: "sets": { 2024-04-21T08:41:35.728 INFO:teuthology.orchestra.run.smithi043.stdout: "52e10dcd": { 2024-04-21T08:41:35.728 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 4, 2024-04-21T08:41:35.728 INFO:teuthology.orchestra.run.smithi043.stdout: "age_ref": 0.0, 2024-04-21T08:41:35.728 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-04-21T08:41:35.728 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RELEASED", 2024-04-21T08:41:35.728 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-04-21T08:41:35.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:41:35.728 INFO:teuthology.orchestra.run.smithi043.stdout: "timeout": 30.0, 2024-04-21T08:41:35.728 INFO:teuthology.orchestra.run.smithi043.stdout: "expiration": 84.5, 2024-04-21T08:41:35.728 INFO:teuthology.orchestra.run.smithi043.stdout: "members": { 2024-04-21T08:41:35.728 INFO:teuthology.orchestra.run.smithi043.stdout: "file:/": { 2024-04-21T08:41:35.729 INFO:teuthology.orchestra.run.smithi043.stdout: "excluded": false, 2024-04-21T08:41:35.729 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-04-21T08:41:35.729 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RELEASED", 2024-04-21T08:41:35.729 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-04-21T08:41:35.729 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:41:35.729 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:41:35.729 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:41:35.729 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:41:35.729 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:41:35.729 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:41:35.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.725+0000 7faa54f2b640 1 -- 172.21.15.43:0/1550729701 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faa380a3d20 msgr2=0x7faa380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.725+0000 7faa54f2b640 1 --2- 172.21.15.43:0/1550729701 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faa380a3d20 0x7faa380a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faa48005c40 tx=0x7faa48009960 comp rx=0 tx=0).stop 2024-04-21T08:41:35.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.725+0000 7faa54f2b640 1 -- 172.21.15.43:0/1550729701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa50106160 msgr2=0x7faa50198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.725+0000 7faa54f2b640 1 --2- 172.21.15.43:0/1550729701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa50106160 0x7faa50198ca0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7faa4400c970 tx=0x7faa4400ce30 comp rx=0 tx=0).stop 2024-04-21T08:41:35.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.725+0000 7faa54f2b640 1 -- 172.21.15.43:0/1550729701 shutdown_connections 2024-04-21T08:41:35.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.725+0000 7faa54f2b640 1 --2- 172.21.15.43:0/1550729701 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faa380a3d20 0x7faa380a61e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.725+0000 7faa54f2b640 1 --2- 172.21.15.43:0/1550729701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa5019d020 0x7faa5019d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.725+0000 7faa54f2b640 1 --2- 172.21.15.43:0/1550729701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa50106160 0x7faa50198ca0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.725+0000 7faa54f2b640 1 --2- 172.21.15.43:0/1550729701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa50101090 0x7faa50198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.725+0000 7faa54f2b640 1 -- 172.21.15.43:0/1550729701 >> 172.21.15.43:0/1550729701 conn(0x7faa50077f50 msgr2=0x7faa500ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:35.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.725+0000 7faa54f2b640 1 -- 172.21.15.43:0/1550729701 shutdown_connections 2024-04-21T08:41:35.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:35.725+0000 7faa54f2b640 1 -- 172.21.15.43:0/1550729701 wait complete. 2024-04-21T08:41:35.735 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set_id: 52e10dcd, seconds elapsed: 25.6 2024-04-21T08:41:36.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:36 smithi043 ceph-mon[27978]: pgmap v83: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-04-21T08:41:36.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:36 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2759382336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:36.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:36 smithi175 ceph-mon[31920]: pgmap v83: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-04-21T08:41:36.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:36 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2759382336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:36.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:36 smithi112 ceph-mon[31085]: pgmap v83: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-04-21T08:41:36.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:36 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2759382336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:38.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:38 smithi175 ceph-mon[31920]: pgmap v84: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.0 KiB/s wr, 15 op/s 2024-04-21T08:41:38.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:38 smithi112 ceph-mon[31085]: pgmap v84: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.0 KiB/s wr, 15 op/s 2024-04-21T08:41:38.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:38 smithi043 ceph-mon[27978]: pgmap v84: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.0 KiB/s wr, 15 op/s 2024-04-21T08:41:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.809+0000 7f23ef8e4640 1 -- 172.21.15.43:0/1029356250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23e8107290 msgr2=0x7f23e810ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.809+0000 7f23ef8e4640 1 --2- 172.21.15.43:0/1029356250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23e8107290 0x7f23e810ee60 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009ca0 tx=0x7f23e002c100 comp rx=0 tx=0).stop 2024-04-21T08:41:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.809+0000 7f23ef8e4640 1 -- 172.21.15.43:0/1029356250 shutdown_connections 2024-04-21T08:41:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.809+0000 7f23ef8e4640 1 --2- 172.21.15.43:0/1029356250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23e8107290 0x7f23e810ee60 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.810+0000 7f23ef8e4640 1 --2- 172.21.15.43:0/1029356250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23e8102b70 0x7f23e8106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.810+0000 7f23ef8e4640 1 --2- 172.21.15.43:0/1029356250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23e81021c0 0x7f23e81025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.810+0000 7f23ef8e4640 1 -- 172.21.15.43:0/1029356250 >> 172.21.15.43:0/1029356250 conn(0x7f23e80fda20 msgr2=0x7f23e80ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.810+0000 7f23ef8e4640 1 -- 172.21.15.43:0/1029356250 shutdown_connections 2024-04-21T08:41:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.810+0000 7f23ef8e4640 1 -- 172.21.15.43:0/1029356250 wait complete. 2024-04-21T08:41:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.810+0000 7f23ef8e4640 1 Processor -- start 2024-04-21T08:41:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.810+0000 7f23ef8e4640 1 -- start start 2024-04-21T08:41:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.811+0000 7f23ef8e4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23e81021c0 0x7f23e8074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.811+0000 7f23ef8e4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23e8107290 0x7f23e8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.811+0000 7f23ef8e4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23e8074cd0 0x7f23e81a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.811+0000 7f23ef8e4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f23e8112200 con 0x7f23e8107290 2024-04-21T08:41:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.811+0000 7f23ef8e4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f23e8112080 con 0x7f23e8074cd0 2024-04-21T08:41:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.811+0000 7f23ef8e4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23e8112380 con 0x7f23e81021c0 2024-04-21T08:41:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.811+0000 7f23ece7f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23e8107290 0x7f23e8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.811+0000 7f23ece7f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23e8107290 0x7f23e8074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33886/0 (socket says 172.21.15.43:33886) 2024-04-21T08:41:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.811+0000 7f23ece7f640 1 -- 172.21.15.43:0/1289867459 learned_addr learned my addr 172.21.15.43:0/1289867459 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.811+0000 7f23ed680640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23e81021c0 0x7f23e8074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.811+0000 7f23ede81640 1 --2- 172.21.15.43:0/1289867459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23e8074cd0 0x7f23e81a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.811+0000 7f23ece7f640 1 -- 172.21.15.43:0/1289867459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23e81021c0 msgr2=0x7f23e8074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.811+0000 7f23ece7f640 1 --2- 172.21.15.43:0/1289867459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23e81021c0 0x7f23e8074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.811+0000 7f23ece7f640 1 -- 172.21.15.43:0/1289867459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23e8074cd0 msgr2=0x7f23e81a67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.811+0000 7f23ece7f640 1 --2- 172.21.15.43:0/1289867459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23e8074cd0 0x7f23e81a67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.811+0000 7f23ece7f640 1 -- 172.21.15.43:0/1289867459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e81a6eb0 con 0x7f23e8107290 2024-04-21T08:41:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.811+0000 7f23ece7f640 1 --2- 172.21.15.43:0/1289867459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23e8107290 0x7f23e8074790 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f23d800d740 tx=0x7f23d800db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.811+0000 7f23de7fc640 1 -- 172.21.15.43:0/1289867459 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d8017070 con 0x7f23e8107290 2024-04-21T08:41:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.812+0000 7f23ef8e4640 1 -- 172.21.15.43:0/1289867459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23e81a71a0 con 0x7f23e8107290 2024-04-21T08:41:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.812+0000 7f23de7fc640 1 -- 172.21.15.43:0/1289867459 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23d8009820 con 0x7f23e8107290 2024-04-21T08:41:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.812+0000 7f23de7fc640 1 -- 172.21.15.43:0/1289867459 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d8004ce0 con 0x7f23e8107290 2024-04-21T08:41:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.812+0000 7f23ef8e4640 1 -- 172.21.15.43:0/1289867459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23e81a76a0 con 0x7f23e8107290 2024-04-21T08:41:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.814+0000 7f23de7fc640 1 -- 172.21.15.43:0/1289867459 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f23d8005220 con 0x7f23e8107290 2024-04-21T08:41:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.815+0000 7f23f25b4740 1 Processor -- start 2024-04-21T08:41:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.815+0000 7f23f25b4740 1 -- start start 2024-04-21T08:41:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.815+0000 7f23f25b4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f228c9c8d0 0x55f228c9b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.815+0000 7f23f25b4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f228c9bb60 0x55f228c9bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.815+0000 7f23f25b4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228ca50e0 0x55f228ca74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.815+0000 7f23f25b4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f228c008d0 con 0x55f228ca50e0 2024-04-21T08:41:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.815+0000 7f23f25b4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f228b9de80 con 0x55f228c9c8d0 2024-04-21T08:41:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.815+0000 7f23f25b4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f228bfc890 con 0x55f228c9bb60 2024-04-21T08:41:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.815+0000 7f23de7fc640 1 --2- 172.21.15.43:0/1289867459 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f23d40a3df0 0x7f23d40a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.815+0000 7f23ece7f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228ca50e0 0x55f228ca74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.815+0000 7f23ece7f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228ca50e0 0x55f228ca74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33888/0 (socket says 172.21.15.43:33888) 2024-04-21T08:41:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.815+0000 7f23ece7f640 1 -- 172.21.15.43:0/2564018665 learned_addr learned my addr 172.21.15.43:0/2564018665 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.815+0000 7f23ede81640 1 --2- 172.21.15.43:0/1289867459 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f23d40a3df0 0x7f23d40a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.815+0000 7f23ede81640 1 --2- 172.21.15.43:0/2564018665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f228c9c8d0 0x55f228c9b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.816+0000 7f23ed680640 1 --2- 172.21.15.43:0/2564018665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f228c9bb60 0x55f228c9bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.816+0000 7f23de7fc640 1 -- 172.21.15.43:0/1289867459 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f23d80d0510 con 0x7f23e8107290 2024-04-21T08:41:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.816+0000 7f23ece7f640 1 -- 172.21.15.43:0/2564018665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f228c9bb60 msgr2=0x55f228c9bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.816+0000 7f23ece7f640 1 --2- 172.21.15.43:0/2564018665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f228c9bb60 0x55f228c9bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.816+0000 7f23ece7f640 1 -- 172.21.15.43:0/2564018665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f228c9c8d0 msgr2=0x55f228c9b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.816+0000 7f23ece7f640 1 --2- 172.21.15.43:0/2564018665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f228c9c8d0 0x55f228c9b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.816+0000 7f23ece7f640 1 -- 172.21.15.43:0/2564018665 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f228bbf320 con 0x55f228ca50e0 2024-04-21T08:41:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.816+0000 7f23ece7f640 1 --2- 172.21.15.43:0/2564018665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228ca50e0 0x55f228ca74d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f23d80cd800 tx=0x7f23d809bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649d0cae72e4a564 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.816+0000 7f23ede81640 1 --2- 172.21.15.43:0/1289867459 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f23d40a3df0 0x7f23d40a62b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f23e0032460 tx=0x7f23e002e040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.816+0000 7f23d37fe640 1 -- 172.21.15.43:0/2564018665 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d8020070 con 0x55f228ca50e0 2024-04-21T08:41:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.816+0000 7f23d37fe640 1 -- 172.21.15.43:0/2564018665 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23d809c7e0 con 0x55f228ca50e0 2024-04-21T08:41:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.816+0000 7f23d37fe640 1 -- 172.21.15.43:0/2564018665 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d80099d0 con 0x55f228ca50e0 2024-04-21T08:41:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.817+0000 7f23f25b4740 1 -- 172.21.15.43:0/2564018665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228ca50e0 msgr2=0x55f228ca74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.817+0000 7f23f25b4740 1 --2- 172.21.15.43:0/2564018665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228ca50e0 0x55f228ca74d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f23d80cd800 tx=0x7f23d809bb00 comp rx=0 tx=0).stop 2024-04-21T08:41:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.817+0000 7f23f25b4740 1 -- 172.21.15.43:0/2564018665 shutdown_connections 2024-04-21T08:41:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.817+0000 7f23f25b4740 1 --2- 172.21.15.43:0/2564018665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228ca50e0 0x55f228ca74d0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.817+0000 7f23f25b4740 1 --2- 172.21.15.43:0/2564018665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f228c9bb60 0x55f228c9bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.817+0000 7f23f25b4740 1 --2- 172.21.15.43:0/2564018665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f228c9c8d0 0x55f228c9b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.817+0000 7f23f25b4740 1 -- 172.21.15.43:0/2564018665 >> 172.21.15.43:0/2564018665 conn(0x55f228c62c90 msgr2=0x55f228c96850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.817+0000 7f23f25b4740 1 -- 172.21.15.43:0/2564018665 shutdown_connections 2024-04-21T08:41:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.817+0000 7f23f25b4740 1 -- 172.21.15.43:0/2564018665 wait complete. 2024-04-21T08:41:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.817+0000 7f23f25b4740 1 -- start start 2024-04-21T08:41:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.818+0000 7f23f25b4740 1 Processor -- start 2024-04-21T08:41:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.818+0000 7f23f25b4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f228c9bb60 0x55f228ca49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.818+0000 7f23f25b4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228c9c8d0 0x55f228ca1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.818+0000 7f23f25b4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f228ca50e0 0x55f228ca2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.818+0000 7f23f25b4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f228ca7db0 con 0x55f228c9c8d0 2024-04-21T08:41:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.818+0000 7f23f25b4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f228b88eb0 con 0x55f228c9bb60 2024-04-21T08:41:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.818+0000 7f23f25b4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f228ca7f30 con 0x55f228ca50e0 2024-04-21T08:41:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.818+0000 7f23f25b4740 10 client.0 _ll_register_callbacks cb 0x55f228bb5b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.818+0000 7f23ece7f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228c9c8d0 0x55f228ca1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.818+0000 7f23ece7f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228c9c8d0 0x55f228ca1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33892/0 (socket says 172.21.15.43:33892) 2024-04-21T08:41:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.818+0000 7f23ece7f640 1 -- 172.21.15.43:0/2634933380 learned_addr learned my addr 172.21.15.43:0/2634933380 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.818+0000 7f23ed680640 1 --2- 172.21.15.43:0/2634933380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f228c9bb60 0x55f228ca49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.818+0000 7f23ede81640 1 --2- 172.21.15.43:0/2634933380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f228ca50e0 0x55f228ca2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.819+0000 7f23ece7f640 1 -- 172.21.15.43:0/2634933380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f228ca50e0 msgr2=0x55f228ca2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.819+0000 7f23ece7f640 1 --2- 172.21.15.43:0/2634933380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f228ca50e0 0x55f228ca2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.819+0000 7f23ece7f640 1 -- 172.21.15.43:0/2634933380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f228c9bb60 msgr2=0x55f228ca49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.819+0000 7f23ece7f640 1 --2- 172.21.15.43:0/2634933380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f228c9bb60 0x55f228ca49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.819+0000 7f23ece7f640 1 -- 172.21.15.43:0/2634933380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f228913210 con 0x55f228c9c8d0 2024-04-21T08:41:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.819+0000 7f23ece7f640 1 --2- 172.21.15.43:0/2634933380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228c9c8d0 0x55f228ca1dc0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f23d809bde0 tx=0x7f23d8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ae626380c1b327 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.819+0000 7f23d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.819+0000 7f23d0ff9640 1 -- 172.21.15.43:0/2634933380 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d80b5020 con 0x55f228c9c8d0 2024-04-21T08:41:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.819+0000 7f23d0ff9640 1 -- 172.21.15.43:0/2634933380 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f23d80099d0 con 0x55f228c9c8d0 2024-04-21T08:41:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.819+0000 7f23f25b4740 1 -- 172.21.15.43:0/2634933380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23d809c7e0 con 0x55f228c9c8d0 2024-04-21T08:41:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.819+0000 7f23d0ff9640 1 -- 172.21.15.43:0/2634933380 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f23d8020040 con 0x55f228c9c8d0 2024-04-21T08:41:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.819+0000 7f23f25b4740 10 client.16881 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.819+0000 7f23f25b4740 1 -- 172.21.15.43:0/2634933380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f2289df620 con 0x55f228c9c8d0 2024-04-21T08:41:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.819+0000 7f23d0ff9640 1 -- 172.21.15.43:0/2634933380 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23d809c7e0 con 0x55f228c9c8d0 2024-04-21T08:41:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.819+0000 7f23d0ff9640 1 -- 172.21.15.43:0/2634933380 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f23d80cdb10 con 0x55f228c9c8d0 2024-04-21T08:41:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.820+0000 7f23f25b4740 10 client.16881 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.820+0000 7f23f25b4740 10 client.16881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.820+0000 7f23f25b4740 20 client.16881 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.820+0000 7f23f25b4740 1 --2- 172.21.15.43:0/2634933380 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f228cb4ec0 0x55f228cb72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.820+0000 7f23f25b4740 4 client.16881 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.820+0000 7f23f25b4740 1 -- 172.21.15.43:0/2634933380 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f228c14880 con 0x55f228cb4ec0 2024-04-21T08:41:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.820+0000 7f23ed680640 1 --2- 172.21.15.43:0/2634933380 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f228cb4ec0 0x55f228cb72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.820+0000 7f23ed680640 1 --2- 172.21.15.43:0/2634933380 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f228cb4ec0 0x55f228cb72b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7dfa96c7a5c3840 server_cookie=e617b22523062bd9 in_seq=0 out_seq=0 2024-04-21T08:41:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.820+0000 7f23d0ff9640 10 client.16881 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.821+0000 7f23d0ff9640 1 -- 172.21.15.43:0/2634933380 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f228c14880 con 0x55f228cb4ec0 2024-04-21T08:41:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.821+0000 7f23d0ff9640 10 client.16881 handle_command_reply: tid=0 2024-04-21T08:41:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.821+0000 7f23f25b4740 1 client.16881 shutdown 2024-04-21T08:41:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.821+0000 7f23f25b4740 1 -- 172.21.15.43:0/2634933380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228c9c8d0 msgr2=0x55f228ca1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.821+0000 7f23f25b4740 1 --2- 172.21.15.43:0/2634933380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228c9c8d0 0x55f228ca1dc0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f23d809bde0 tx=0x7f23d8005740 comp rx=0 tx=0).stop 2024-04-21T08:41:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.821+0000 7f23f25b4740 1 -- 172.21.15.43:0/2634933380 shutdown_connections 2024-04-21T08:41:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.821+0000 7f23f25b4740 1 --2- 172.21.15.43:0/2634933380 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f228cb4ec0 0x55f228cb72b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.821+0000 7f23f25b4740 1 --2- 172.21.15.43:0/2634933380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f228ca50e0 0x55f228ca2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.821+0000 7f23d0ff9640 0 client.16881 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.821+0000 7f23f25b4740 1 --2- 172.21.15.43:0/2634933380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228c9c8d0 0x55f228ca1dc0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.821+0000 7f23f25b4740 1 --2- 172.21.15.43:0/2634933380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f228c9bb60 0x55f228ca49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.821+0000 7f23f25b4740 1 -- 172.21.15.43:0/2634933380 >> 172.21.15.43:0/2634933380 conn(0x55f228c62c90 msgr2=0x55f228c985a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.822+0000 7f23f25b4740 1 -- 172.21.15.43:0/2634933380 shutdown_connections 2024-04-21T08:41:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.822+0000 7f23f25b4740 1 -- 172.21.15.43:0/2634933380 wait complete. 2024-04-21T08:41:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.822+0000 7f23f25b4740 20 client.16881 trim_cache size 0 max 16384 2024-04-21T08:41:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.837+0000 7f23f25b4740 1 Processor -- start 2024-04-21T08:41:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.837+0000 7f23f25b4740 1 -- start start 2024-04-21T08:41:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.837+0000 7f23f25b4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f228c62c90 0x55f228cb0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.837+0000 7f23f25b4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f228c9bb60 0x55f228ca1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.837+0000 7f23f25b4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228c9c8d0 0x55f228cbc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.837+0000 7f23f25b4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81b1550 con 0x55f228c9c8d0 2024-04-21T08:41:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.837+0000 7f23f25b4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81b3090 con 0x55f228c9bb60 2024-04-21T08:41:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.837+0000 7f23f25b4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81b1c20 con 0x55f228c62c90 2024-04-21T08:41:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.837+0000 7f23ede81640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228c9c8d0 0x55f228cbc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.837+0000 7f23ede81640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228c9c8d0 0x55f228cbc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33900/0 (socket says 172.21.15.43:33900) 2024-04-21T08:41:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.837+0000 7f23ede81640 1 -- 172.21.15.43:0/50929708 learned_addr learned my addr 172.21.15.43:0/50929708 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.837+0000 7f23ece7f640 1 --2- 172.21.15.43:0/50929708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f228c9bb60 0x55f228ca1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.837+0000 7f23ed680640 1 --2- 172.21.15.43:0/50929708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f228c62c90 0x55f228cb0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.837+0000 7f23ede81640 1 -- 172.21.15.43:0/50929708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f228c62c90 msgr2=0x55f228cb0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.837+0000 7f23ede81640 1 --2- 172.21.15.43:0/50929708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f228c62c90 0x55f228cb0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.837+0000 7f23ede81640 1 -- 172.21.15.43:0/50929708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f228c9bb60 msgr2=0x55f228ca1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.837+0000 7f23ede81640 1 --2- 172.21.15.43:0/50929708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f228c9bb60 0x55f228ca1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.837+0000 7f23ede81640 1 -- 172.21.15.43:0/50929708 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f228c9ae30 con 0x55f228c9c8d0 2024-04-21T08:41:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.837+0000 7f23ed680640 1 --2- 172.21.15.43:0/50929708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f228c62c90 0x55f228cb0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.838+0000 7f23ede81640 1 --2- 172.21.15.43:0/50929708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228c9c8d0 0x55f228cbc880 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009ca0 tx=0x7f23e0039000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8c760ea107400e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.838+0000 7f23d2ffd640 1 -- 172.21.15.43:0/50929708 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e004a050 con 0x55f228c9c8d0 2024-04-21T08:41:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.838+0000 7f23d2ffd640 1 -- 172.21.15.43:0/50929708 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23e0054070 con 0x55f228c9c8d0 2024-04-21T08:41:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.838+0000 7f23d2ffd640 1 -- 172.21.15.43:0/50929708 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0039070 con 0x55f228c9c8d0 2024-04-21T08:41:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.838+0000 7f23f25b4740 1 -- 172.21.15.43:0/50929708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228c9c8d0 msgr2=0x55f228cbc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.838+0000 7f23f25b4740 1 --2- 172.21.15.43:0/50929708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228c9c8d0 0x55f228cbc880 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009ca0 tx=0x7f23e0039000 comp rx=0 tx=0).stop 2024-04-21T08:41:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.839+0000 7f23f25b4740 1 -- 172.21.15.43:0/50929708 shutdown_connections 2024-04-21T08:41:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.839+0000 7f23f25b4740 1 --2- 172.21.15.43:0/50929708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228c9c8d0 0x55f228cbc880 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.839+0000 7f23f25b4740 1 --2- 172.21.15.43:0/50929708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f228c9bb60 0x55f228ca1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.839+0000 7f23f25b4740 1 --2- 172.21.15.43:0/50929708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f228c62c90 0x55f228cb0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.839+0000 7f23f25b4740 1 -- 172.21.15.43:0/50929708 >> 172.21.15.43:0/50929708 conn(0x55f228ca2460 msgr2=0x55f228c96380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.839+0000 7f23f25b4740 1 -- 172.21.15.43:0/50929708 shutdown_connections 2024-04-21T08:41:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.839+0000 7f23f25b4740 1 -- 172.21.15.43:0/50929708 wait complete. 2024-04-21T08:41:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.839+0000 7f23f25b4740 1 -- start start 2024-04-21T08:41:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.839+0000 7f23f25b4740 1 Processor -- start 2024-04-21T08:41:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.839+0000 7f23f25b4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228c62c90 0x55f228c9fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.839+0000 7f23f25b4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f228c9bb60 0x55f228cad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.839+0000 7f23f25b4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f228c9c8d0 0x55f228ca6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.839+0000 7f23f25b4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81b29c0 con 0x55f228c62c90 2024-04-21T08:41:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.839+0000 7f23f25b4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81b22f0 con 0x55f228c9bb60 2024-04-21T08:41:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.839+0000 7f23f25b4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81af530 con 0x55f228c9c8d0 2024-04-21T08:41:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.839+0000 7f23f25b4740 10 client.0 _ll_register_callbacks cb 0x7f23e0037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.840+0000 7f23ed680640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228c62c90 0x55f228c9fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.840+0000 7f23ed680640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228c62c90 0x55f228c9fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33916/0 (socket says 172.21.15.43:33916) 2024-04-21T08:41:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.840+0000 7f23ed680640 1 -- 172.21.15.43:0/1287995781 learned_addr learned my addr 172.21.15.43:0/1287995781 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.840+0000 7f23ece7f640 1 --2- 172.21.15.43:0/1287995781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f228c9bb60 0x55f228cad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.840+0000 7f23ede81640 1 --2- 172.21.15.43:0/1287995781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f228c9c8d0 0x55f228ca6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.840+0000 7f23ed680640 1 -- 172.21.15.43:0/1287995781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f228c9c8d0 msgr2=0x55f228ca6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.840+0000 7f23ed680640 1 --2- 172.21.15.43:0/1287995781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f228c9c8d0 0x55f228ca6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.840+0000 7f23ed680640 1 -- 172.21.15.43:0/1287995781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f228c9bb60 msgr2=0x55f228cad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.840+0000 7f23ed680640 1 --2- 172.21.15.43:0/1287995781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f228c9bb60 0x55f228cad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.840+0000 7f23ed680640 1 -- 172.21.15.43:0/1287995781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f228c91f30 con 0x55f228c62c90 2024-04-21T08:41:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.840+0000 7f23ed680640 1 --2- 172.21.15.43:0/1287995781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228c62c90 0x55f228c9fa10 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f23e4000c00 tx=0x7f23e4010af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb783b2fb70e7ff5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.840+0000 7f23d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.840+0000 7f23d0ff9640 1 -- 172.21.15.43:0/1287995781 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e40205c0 con 0x55f228c62c90 2024-04-21T08:41:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.840+0000 7f23d0ff9640 1 -- 172.21.15.43:0/1287995781 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f23e4020760 con 0x55f228c62c90 2024-04-21T08:41:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.840+0000 7f23f25b4740 1 -- 172.21.15.43:0/1287995781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23e0054070 con 0x55f228c62c90 2024-04-21T08:41:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.840+0000 7f23d0ff9640 1 -- 172.21.15.43:0/1287995781 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f23e4031050 con 0x55f228c62c90 2024-04-21T08:41:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.840+0000 7f23f25b4740 10 client.16887 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.840+0000 7f23f25b4740 1 -- 172.21.15.43:0/1287995781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f228d1b530 con 0x55f228c62c90 2024-04-21T08:41:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.841+0000 7f23d0ff9640 1 -- 172.21.15.43:0/1287995781 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23e0054070 con 0x55f228c62c90 2024-04-21T08:41:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.841+0000 7f23d0ff9640 1 -- 172.21.15.43:0/1287995781 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f23e40295e0 con 0x55f228c62c90 2024-04-21T08:41:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.841+0000 7f23f25b4740 10 client.16887 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.841+0000 7f23f25b4740 10 client.16887 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.841+0000 7f23f25b4740 20 client.16887 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.841+0000 7f23f25b4740 1 --2- 172.21.15.43:0/1287995781 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f228ca50e0 0x55f228d1dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.841+0000 7f23f25b4740 4 client.16887 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:41:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.841+0000 7f23f25b4740 1 -- 172.21.15.43:0/1287995781 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f228ba03d0 con 0x55f228ca50e0 2024-04-21T08:41:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.841+0000 7f23ece7f640 1 --2- 172.21.15.43:0/1287995781 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f228ca50e0 0x55f228d1dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.841+0000 7f23ece7f640 1 --2- 172.21.15.43:0/1287995781 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f228ca50e0 0x55f228d1dd10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b31d54dbc8e9e083 server_cookie=f0080636fd83b93f in_seq=0 out_seq=0 2024-04-21T08:41:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.841+0000 7f23d0ff9640 10 client.16887 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.850+0000 7f23d0ff9640 1 -- 172.21.15.43:0/1287995781 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f228ba03d0 con 0x55f228ca50e0 2024-04-21T08:41:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.850+0000 7f23d0ff9640 10 client.16887 handle_command_reply: tid=0 2024-04-21T08:41:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.850+0000 7f23f25b4740 1 client.16887 shutdown 2024-04-21T08:41:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.850+0000 7f23f25b4740 1 -- 172.21.15.43:0/1287995781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228c62c90 msgr2=0x55f228c9fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.850+0000 7f23f25b4740 1 --2- 172.21.15.43:0/1287995781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228c62c90 0x55f228c9fa10 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f23e4000c00 tx=0x7f23e4010af0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.850+0000 7f23f25b4740 1 -- 172.21.15.43:0/1287995781 shutdown_connections 2024-04-21T08:41:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.850+0000 7f23f25b4740 1 --2- 172.21.15.43:0/1287995781 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f228ca50e0 0x55f228d1dd10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.850+0000 7f23f25b4740 1 --2- 172.21.15.43:0/1287995781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f228c9c8d0 0x55f228ca6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.850+0000 7f23f25b4740 1 --2- 172.21.15.43:0/1287995781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f228c9bb60 0x55f228cad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.850+0000 7f23f25b4740 1 --2- 172.21.15.43:0/1287995781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f228c62c90 0x55f228c9fa10 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.850+0000 7f23d0ff9640 0 client.16887 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.850+0000 7f23f25b4740 1 -- 172.21.15.43:0/1287995781 >> 172.21.15.43:0/1287995781 conn(0x55f228ca2460 msgr2=0x55f228ca0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.851+0000 7f23f25b4740 1 -- 172.21.15.43:0/1287995781 shutdown_connections 2024-04-21T08:41:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.851+0000 7f23f25b4740 1 -- 172.21.15.43:0/1287995781 wait complete. 2024-04-21T08:41:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.851+0000 7f23f25b4740 20 client.16887 trim_cache size 0 max 16384 2024-04-21T08:41:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:41:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:41:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:41:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:41:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.851+0000 7f23ef8e4640 1 -- 172.21.15.43:0/1289867459 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f23d40a3df0 msgr2=0x7f23d40a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.851+0000 7f23ef8e4640 1 --2- 172.21.15.43:0/1289867459 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f23d40a3df0 0x7f23d40a62b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f23e0032460 tx=0x7f23e002e040 comp rx=0 tx=0).stop 2024-04-21T08:41:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.851+0000 7f23ef8e4640 1 -- 172.21.15.43:0/1289867459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23e8107290 msgr2=0x7f23e8074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.851+0000 7f23ef8e4640 1 --2- 172.21.15.43:0/1289867459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23e8107290 0x7f23e8074790 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f23d800d740 tx=0x7f23d800db40 comp rx=0 tx=0).stop 2024-04-21T08:41:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.851+0000 7f23ef8e4640 1 -- 172.21.15.43:0/1289867459 shutdown_connections 2024-04-21T08:41:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.851+0000 7f23ef8e4640 1 --2- 172.21.15.43:0/1289867459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23e8074cd0 0x7f23e81a67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.851+0000 7f23ef8e4640 1 --2- 172.21.15.43:0/1289867459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23e8107290 0x7f23e8074790 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.851+0000 7f23ef8e4640 1 --2- 172.21.15.43:0/1289867459 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f23d40a3df0 0x7f23d40a62b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.851+0000 7f23ef8e4640 1 --2- 172.21.15.43:0/1289867459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23e81021c0 0x7f23e8074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.851+0000 7f23ef8e4640 1 -- 172.21.15.43:0/1289867459 >> 172.21.15.43:0/1289867459 conn(0x7f23e80fda20 msgr2=0x7f23e80fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.851+0000 7f23ef8e4640 1 -- 172.21.15.43:0/1289867459 shutdown_connections 2024-04-21T08:41:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:39.851+0000 7f23ef8e4640 1 -- 172.21.15.43:0/1289867459 wait complete. 2024-04-21T08:41:40.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:40 smithi175 ceph-mon[31920]: pgmap v85: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-04-21T08:41:40.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:40 smithi112 ceph-mon[31085]: pgmap v85: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-04-21T08:41:40.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:40 smithi043 ceph-mon[27978]: pgmap v85: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-04-21T08:41:41.282 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:41:41.282 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:41:41.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.401+0000 7fd16eac8640 1 -- 172.21.15.43:0/3553024674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd168101a40 msgr2=0x7fd168105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:41.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.401+0000 7fd16eac8640 1 --2- 172.21.15.43:0/3553024674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd168101a40 0x7fd168105a80 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fd154009900 tx=0x7fd15402bc50 comp rx=0 tx=0).stop 2024-04-21T08:41:41.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.401+0000 7fd16eac8640 1 -- 172.21.15.43:0/3553024674 shutdown_connections 2024-04-21T08:41:41.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.401+0000 7fd16eac8640 1 --2- 172.21.15.43:0/3553024674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd168106160 0x7fd16810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.401+0000 7fd16eac8640 1 --2- 172.21.15.43:0/3553024674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd168101a40 0x7fd168105a80 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.401+0000 7fd16eac8640 1 --2- 172.21.15.43:0/3553024674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd168101090 0x7fd168101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.401+0000 7fd16eac8640 1 -- 172.21.15.43:0/3553024674 >> 172.21.15.43:0/3553024674 conn(0x7fd1680771d0 msgr2=0x7fd1680775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:41.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.401+0000 7fd16eac8640 1 -- 172.21.15.43:0/3553024674 shutdown_connections 2024-04-21T08:41:41.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.401+0000 7fd16eac8640 1 -- 172.21.15.43:0/3553024674 wait complete. 2024-04-21T08:41:41.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.402+0000 7fd16eac8640 1 Processor -- start 2024-04-21T08:41:41.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.402+0000 7fd16eac8640 1 -- start start 2024-04-21T08:41:41.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.402+0000 7fd16eac8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd168101090 0x7fd168198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.402+0000 7fd16eac8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd168106160 0x7fd168198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.402+0000 7fd16eac8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd16819cf50 0x7fd16819d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.402+0000 7fd16eac8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd1681110d0 con 0x7fd16819cf50 2024-04-21T08:41:41.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.402+0000 7fd16eac8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd168110f50 con 0x7fd168106160 2024-04-21T08:41:41.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.402+0000 7fd16eac8640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd168111250 con 0x7fd168101090 2024-04-21T08:41:41.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.402+0000 7fd16d065640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd16819cf50 0x7fd16819d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.402+0000 7fd16d065640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd16819cf50 0x7fd16819d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33928/0 (socket says 172.21.15.43:33928) 2024-04-21T08:41:41.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.402+0000 7fd16d065640 1 -- 172.21.15.43:0/914723753 learned_addr learned my addr 172.21.15.43:0/914723753 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:41.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.402+0000 7fd16c864640 1 --2- 172.21.15.43:0/914723753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd168101090 0x7fd168198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.402+0000 7fd167fff640 1 --2- 172.21.15.43:0/914723753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd168106160 0x7fd168198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.403+0000 7fd16d065640 1 -- 172.21.15.43:0/914723753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd168101090 msgr2=0x7fd168198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:41.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.403+0000 7fd16d065640 1 --2- 172.21.15.43:0/914723753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd168101090 0x7fd168198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.403+0000 7fd16d065640 1 -- 172.21.15.43:0/914723753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd168106160 msgr2=0x7fd168198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:41.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.403+0000 7fd16d065640 1 --2- 172.21.15.43:0/914723753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd168106160 0x7fd168198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.403+0000 7fd16d065640 1 -- 172.21.15.43:0/914723753 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16819da40 con 0x7fd16819cf50 2024-04-21T08:41:41.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.403+0000 7fd16d065640 1 --2- 172.21.15.43:0/914723753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd16819cf50 0x7fd16819d400 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fd160009fd0 tx=0x7fd16000def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:41.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.403+0000 7fd165ffb640 1 -- 172.21.15.43:0/914723753 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160017070 con 0x7fd16819cf50 2024-04-21T08:41:41.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.403+0000 7fd165ffb640 1 -- 172.21.15.43:0/914723753 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd160003e20 con 0x7fd16819cf50 2024-04-21T08:41:41.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.403+0000 7fd16eac8640 1 -- 172.21.15.43:0/914723753 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd16819dd30 con 0x7fd16819cf50 2024-04-21T08:41:41.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.403+0000 7fd165ffb640 1 -- 172.21.15.43:0/914723753 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160010460 con 0x7fd16819cf50 2024-04-21T08:41:41.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.403+0000 7fd16eac8640 1 -- 172.21.15.43:0/914723753 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1681a54f0 con 0x7fd16819cf50 2024-04-21T08:41:41.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.405+0000 7fd165ffb640 1 -- 172.21.15.43:0/914723753 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd160004020 con 0x7fd16819cf50 2024-04-21T08:41:41.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.406+0000 7fd171798740 1 Processor -- start 2024-04-21T08:41:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.406+0000 7fd171798740 1 -- start start 2024-04-21T08:41:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.406+0000 7fd171798740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e96cff390 0x556e96cff770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.406+0000 7fd171798740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96cffd40 0x556e96d0c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.406+0000 7fd171798740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e96d0ce90 0x556e96d0f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.406+0000 7fd171798740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556e96c641e0 con 0x556e96cffd40 2024-04-21T08:41:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.406+0000 7fd171798740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556e96c01390 con 0x556e96cff390 2024-04-21T08:41:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.406+0000 7fd171798740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556e96c601a0 con 0x556e96d0ce90 2024-04-21T08:41:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.406+0000 7fd165ffb640 1 --2- 172.21.15.43:0/914723753 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd1580a3da0 0x7fd1580a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.406+0000 7fd16c864640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96cffd40 0x556e96d0c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.406+0000 7fd167fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e96cff390 0x556e96cff770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.406+0000 7fd167fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e96cff390 0x556e96cff770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:56360/0 (socket says 172.21.15.43:56360) 2024-04-21T08:41:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.406+0000 7fd167fff640 1 -- 172.21.15.43:0/2154553262 learned_addr learned my addr 172.21.15.43:0/2154553262 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.406+0000 7fd16d065640 1 --2- 172.21.15.43:0/914723753 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd1580a3da0 0x7fd1580a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.406+0000 7fd167fff640 1 --2- 172.21.15.43:0/2154553262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e96d0ce90 0x556e96d0f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.406+0000 7fd165ffb640 1 -- 172.21.15.43:0/914723753 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd1600d0770 con 0x7fd16819cf50 2024-04-21T08:41:41.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.406+0000 7fd16c864640 1 -- 172.21.15.43:0/2154553262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e96d0ce90 msgr2=0x556e96d0f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:41.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.406+0000 7fd16c864640 1 --2- 172.21.15.43:0/2154553262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e96d0ce90 0x556e96d0f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.406+0000 7fd16c864640 1 -- 172.21.15.43:0/2154553262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e96cff390 msgr2=0x556e96cff770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:41.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.406+0000 7fd16c864640 1 --2- 172.21.15.43:0/2154553262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e96cff390 0x556e96cff770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.406+0000 7fd16c864640 1 -- 172.21.15.43:0/2154553262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e96a407d0 con 0x556e96cffd40 2024-04-21T08:41:41.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.407+0000 7fd16d065640 1 --2- 172.21.15.43:0/914723753 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd1580a3da0 0x7fd1580a6260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd1600ce0f0 tx=0x7fd16006dd80 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:41.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.407+0000 7fd16c864640 1 --2- 172.21.15.43:0/2154553262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96cffd40 0x556e96d0c950 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00abf0 tx=0x7fd15c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a7ecb270aa7d6f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:41.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.407+0000 7fd152ffd640 1 -- 172.21.15.43:0/2154553262 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c005220 con 0x556e96cffd40 2024-04-21T08:41:41.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.407+0000 7fd152ffd640 1 -- 172.21.15.43:0/2154553262 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd15c004c50 con 0x556e96cffd40 2024-04-21T08:41:41.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.407+0000 7fd152ffd640 1 -- 172.21.15.43:0/2154553262 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c0187c0 con 0x556e96cffd40 2024-04-21T08:41:41.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.408+0000 7fd171798740 1 -- 172.21.15.43:0/2154553262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96cffd40 msgr2=0x556e96d0c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:41.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.408+0000 7fd171798740 1 --2- 172.21.15.43:0/2154553262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96cffd40 0x556e96d0c950 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00abf0 tx=0x7fd15c00f790 comp rx=0 tx=0).stop 2024-04-21T08:41:41.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.408+0000 7fd171798740 1 -- 172.21.15.43:0/2154553262 shutdown_connections 2024-04-21T08:41:41.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.408+0000 7fd171798740 1 --2- 172.21.15.43:0/2154553262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e96d0ce90 0x556e96d0f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.408+0000 7fd171798740 1 --2- 172.21.15.43:0/2154553262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96cffd40 0x556e96d0c950 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.408+0000 7fd171798740 1 --2- 172.21.15.43:0/2154553262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e96cff390 0x556e96cff770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.408+0000 7fd171798740 1 -- 172.21.15.43:0/2154553262 >> 172.21.15.43:0/2154553262 conn(0x556e96cf9cb0 msgr2=0x556e96cfa090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:41.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.408+0000 7fd171798740 1 -- 172.21.15.43:0/2154553262 shutdown_connections 2024-04-21T08:41:41.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.408+0000 7fd171798740 1 -- 172.21.15.43:0/2154553262 wait complete. 2024-04-21T08:41:41.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.408+0000 7fd171798740 1 -- start start 2024-04-21T08:41:41.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.408+0000 7fd171798740 1 Processor -- start 2024-04-21T08:41:41.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.409+0000 7fd171798740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e96cff390 0x556e96d09ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.409+0000 7fd171798740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96cffd40 0x556e96d0a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.409+0000 7fd171798740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e96d0ce90 0x556e96d0a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.409+0000 7fd171798740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556e96d0fc50 con 0x556e96cffd40 2024-04-21T08:41:41.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.409+0000 7fd171798740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556e96c1d2f0 con 0x556e96cff390 2024-04-21T08:41:41.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.409+0000 7fd171798740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556e96d0fdd0 con 0x556e96d0ce90 2024-04-21T08:41:41.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.409+0000 7fd171798740 10 client.0 _ll_register_callbacks cb 0x556e96c2a580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:41.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.409+0000 7fd16c864640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96cffd40 0x556e96d0a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.409+0000 7fd16c864640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96cffd40 0x556e96d0a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33960/0 (socket says 172.21.15.43:33960) 2024-04-21T08:41:41.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.409+0000 7fd16c864640 1 -- 172.21.15.43:0/972925798 learned_addr learned my addr 172.21.15.43:0/972925798 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:41.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.409+0000 7fd167fff640 1 --2- 172.21.15.43:0/972925798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e96d0ce90 0x556e96d0a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.409+0000 7fd167fff640 1 --2- 172.21.15.43:0/972925798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e96cff390 0x556e96d09ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.409+0000 7fd16c864640 1 -- 172.21.15.43:0/972925798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e96d0ce90 msgr2=0x556e96d0a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:41.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.409+0000 7fd16c864640 1 --2- 172.21.15.43:0/972925798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e96d0ce90 0x556e96d0a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.409+0000 7fd16c864640 1 -- 172.21.15.43:0/972925798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e96cff390 msgr2=0x556e96d09ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:41.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.409+0000 7fd16c864640 1 --2- 172.21.15.43:0/972925798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e96cff390 0x556e96d09ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.409+0000 7fd16c864640 1 -- 172.21.15.43:0/972925798 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e969763f0 con 0x556e96cffd40 2024-04-21T08:41:41.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.409+0000 7fd167fff640 1 --2- 172.21.15.43:0/972925798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e96cff390 0x556e96d09ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:41.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.409+0000 7fd16c864640 1 --2- 172.21.15.43:0/972925798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96cffd40 0x556e96d0a430 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fd15c005720 tx=0x7fd15c00a910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b66292f12c6da3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:41.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.409+0000 7fd147fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:41.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.409+0000 7fd147fff640 1 -- 172.21.15.43:0/972925798 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c027070 con 0x556e96cffd40 2024-04-21T08:41:41.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.409+0000 7fd147fff640 1 -- 172.21.15.43:0/972925798 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd15c005010 con 0x556e96cffd40 2024-04-21T08:41:41.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.409+0000 7fd171798740 1 -- 172.21.15.43:0/972925798 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd15c004c50 con 0x556e96cffd40 2024-04-21T08:41:41.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.410+0000 7fd147fff640 1 -- 172.21.15.43:0/972925798 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd15c033050 con 0x556e96cffd40 2024-04-21T08:41:41.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.410+0000 7fd171798740 10 client.16899 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:41.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.410+0000 7fd171798740 1 -- 172.21.15.43:0/972925798 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556e969360f0 con 0x556e96cffd40 2024-04-21T08:41:41.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.410+0000 7fd147fff640 1 -- 172.21.15.43:0/972925798 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd15c004c50 con 0x556e96cffd40 2024-04-21T08:41:41.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.410+0000 7fd147fff640 1 -- 172.21.15.43:0/972925798 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd15c023430 con 0x556e96cffd40 2024-04-21T08:41:41.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.410+0000 7fd171798740 10 client.16899 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:41.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.410+0000 7fd171798740 10 client.16899 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:41.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.410+0000 7fd171798740 20 client.16899 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:41.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.410+0000 7fd171798740 1 --2- 172.21.15.43:0/972925798 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556e96d1ce10 0x556e96d1f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.410+0000 7fd171798740 4 client.16899 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:41.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.410+0000 7fd171798740 1 -- 172.21.15.43:0/972925798 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e96c71c00 con 0x556e96d1ce10 2024-04-21T08:41:41.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.410+0000 7fd167fff640 1 --2- 172.21.15.43:0/972925798 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556e96d1ce10 0x556e96d1f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.411+0000 7fd147fff640 10 client.16899 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:41.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.411+0000 7fd167fff640 1 --2- 172.21.15.43:0/972925798 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556e96d1ce10 0x556e96d1f200 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5158d99ffb0d224a server_cookie=f59be38a4483739b in_seq=0 out_seq=0 2024-04-21T08:41:41.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.411+0000 7fd147fff640 1 -- 172.21.15.43:0/972925798 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e96c71c00 con 0x556e96d1ce10 2024-04-21T08:41:41.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.411+0000 7fd147fff640 10 client.16899 handle_command_reply: tid=0 2024-04-21T08:41:41.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.412+0000 7fd171798740 1 client.16899 shutdown 2024-04-21T08:41:41.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.412+0000 7fd171798740 1 -- 172.21.15.43:0/972925798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96cffd40 msgr2=0x556e96d0a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:41.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.412+0000 7fd171798740 1 --2- 172.21.15.43:0/972925798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96cffd40 0x556e96d0a430 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fd15c005720 tx=0x7fd15c00a910 comp rx=0 tx=0).stop 2024-04-21T08:41:41.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.412+0000 7fd171798740 1 -- 172.21.15.43:0/972925798 shutdown_connections 2024-04-21T08:41:41.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.412+0000 7fd171798740 1 --2- 172.21.15.43:0/972925798 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556e96d1ce10 0x556e96d1f200 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.412+0000 7fd171798740 1 --2- 172.21.15.43:0/972925798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e96d0ce90 0x556e96d0a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.412+0000 7fd171798740 1 --2- 172.21.15.43:0/972925798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96cffd40 0x556e96d0a430 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.412+0000 7fd147fff640 0 client.16899 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:41.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.412+0000 7fd171798740 1 --2- 172.21.15.43:0/972925798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e96cff390 0x556e96d09ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.412+0000 7fd171798740 1 -- 172.21.15.43:0/972925798 >> 172.21.15.43:0/972925798 conn(0x556e96cf9cb0 msgr2=0x556e96cfba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:41.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.412+0000 7fd171798740 1 -- 172.21.15.43:0/972925798 shutdown_connections 2024-04-21T08:41:41.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.412+0000 7fd171798740 1 -- 172.21.15.43:0/972925798 wait complete. 2024-04-21T08:41:41.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.412+0000 7fd171798740 20 client.16899 trim_cache size 0 max 16384 2024-04-21T08:41:41.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.430+0000 7fd171798740 1 Processor -- start 2024-04-21T08:41:41.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.430+0000 7fd171798740 1 -- start start 2024-04-21T08:41:41.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.430+0000 7fd171798740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e96cff390 0x556e96d0c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.430+0000 7fd171798740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e96cffd40 0x556e96d08e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.430+0000 7fd171798740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96d0ce90 0x556e96d18b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.430+0000 7fd171798740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd1681ad1d0 con 0x556e96d0ce90 2024-04-21T08:41:41.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.430+0000 7fd171798740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd1681aed10 con 0x556e96cffd40 2024-04-21T08:41:41.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.430+0000 7fd171798740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd1681ad8a0 con 0x556e96cff390 2024-04-21T08:41:41.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.430+0000 7fd167fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96d0ce90 0x556e96d18b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.430+0000 7fd167fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96d0ce90 0x556e96d18b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33976/0 (socket says 172.21.15.43:33976) 2024-04-21T08:41:41.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.430+0000 7fd167fff640 1 -- 172.21.15.43:0/1054790945 learned_addr learned my addr 172.21.15.43:0/1054790945 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:41.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.430+0000 7fd16c864640 1 --2- 172.21.15.43:0/1054790945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e96cffd40 0x556e96d08e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.430+0000 7fd167fff640 1 --2- 172.21.15.43:0/1054790945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e96cff390 0x556e96d0c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.431+0000 7fd167fff640 1 -- 172.21.15.43:0/1054790945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e96cff390 msgr2=0x556e96d0c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:41.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.431+0000 7fd167fff640 1 --2- 172.21.15.43:0/1054790945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e96cff390 0x556e96d0c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.431+0000 7fd167fff640 1 -- 172.21.15.43:0/1054790945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e96cffd40 msgr2=0x556e96d08e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:41.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.431+0000 7fd167fff640 1 --2- 172.21.15.43:0/1054790945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e96cffd40 0x556e96d08e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.431+0000 7fd167fff640 1 -- 172.21.15.43:0/1054790945 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e96d0f280 con 0x556e96d0ce90 2024-04-21T08:41:41.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.431+0000 7fd167fff640 1 --2- 172.21.15.43:0/1054790945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96d0ce90 0x556e96d18b50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd15402c190 tx=0x7fd154005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23df24ef8e9dffb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:41.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.431+0000 7fd1527fc640 1 -- 172.21.15.43:0/1054790945 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15404e2b0 con 0x556e96d0ce90 2024-04-21T08:41:41.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.431+0000 7fd1527fc640 1 -- 172.21.15.43:0/1054790945 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd15404e900 con 0x556e96d0ce90 2024-04-21T08:41:41.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.431+0000 7fd1527fc640 1 -- 172.21.15.43:0/1054790945 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd154003db0 con 0x556e96d0ce90 2024-04-21T08:41:41.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.431+0000 7fd171798740 1 -- 172.21.15.43:0/1054790945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96d0ce90 msgr2=0x556e96d18b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:41.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.431+0000 7fd171798740 1 --2- 172.21.15.43:0/1054790945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96d0ce90 0x556e96d18b50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd15402c190 tx=0x7fd154005fd0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.431+0000 7fd171798740 1 -- 172.21.15.43:0/1054790945 shutdown_connections 2024-04-21T08:41:41.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.431+0000 7fd171798740 1 --2- 172.21.15.43:0/1054790945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96d0ce90 0x556e96d18b50 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.431+0000 7fd171798740 1 --2- 172.21.15.43:0/1054790945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e96cffd40 0x556e96d08e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.431+0000 7fd171798740 1 --2- 172.21.15.43:0/1054790945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e96cff390 0x556e96d0c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.431+0000 7fd171798740 1 -- 172.21.15.43:0/1054790945 >> 172.21.15.43:0/1054790945 conn(0x556e96cf9cb0 msgr2=0x556e96cf8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:41.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.432+0000 7fd171798740 1 -- 172.21.15.43:0/1054790945 shutdown_connections 2024-04-21T08:41:41.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.432+0000 7fd171798740 1 -- 172.21.15.43:0/1054790945 wait complete. 2024-04-21T08:41:41.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.432+0000 7fd171798740 1 -- start start 2024-04-21T08:41:41.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.432+0000 7fd171798740 1 Processor -- start 2024-04-21T08:41:41.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.432+0000 7fd171798740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e96cff390 0x556e96d10cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.432+0000 7fd171798740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96cffd40 0x556e96d11200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.432+0000 7fd171798740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e96d0ce90 0x556e96d11740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.432+0000 7fd171798740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd1681ae640 con 0x556e96cffd40 2024-04-21T08:41:41.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.432+0000 7fd171798740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd1681adf70 con 0x556e96d0ce90 2024-04-21T08:41:41.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.432+0000 7fd171798740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd1681ab1b0 con 0x556e96cff390 2024-04-21T08:41:41.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.432+0000 7fd171798740 10 client.0 _ll_register_callbacks cb 0x7fd15402c670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:41.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.432+0000 7fd16c864640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96cffd40 0x556e96d11200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.432+0000 7fd16c864640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96cffd40 0x556e96d11200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33978/0 (socket says 172.21.15.43:33978) 2024-04-21T08:41:41.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.432+0000 7fd16c864640 1 -- 172.21.15.43:0/47801675 learned_addr learned my addr 172.21.15.43:0/47801675 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:41.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.433+0000 7fd167fff640 1 --2- 172.21.15.43:0/47801675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e96cff390 0x556e96d10cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.433+0000 7fd167fff640 1 --2- 172.21.15.43:0/47801675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e96d0ce90 0x556e96d11740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.433+0000 7fd16c864640 1 -- 172.21.15.43:0/47801675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e96cff390 msgr2=0x556e96d10cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:41.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.433+0000 7fd16c864640 1 --2- 172.21.15.43:0/47801675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e96cff390 0x556e96d10cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.433+0000 7fd16c864640 1 -- 172.21.15.43:0/47801675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e96d0ce90 msgr2=0x556e96d11740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:41.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.433+0000 7fd16c864640 1 --2- 172.21.15.43:0/47801675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e96d0ce90 0x556e96d11740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.433+0000 7fd16c864640 1 -- 172.21.15.43:0/47801675 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e96d822f0 con 0x556e96cffd40 2024-04-21T08:41:41.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.433+0000 7fd16c864640 1 --2- 172.21.15.43:0/47801675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96cffd40 0x556e96d11200 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x556e96d09750 tx=0x7fd15c023aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70dffccd3f54557 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:41.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.433+0000 7fd147fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:41.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.433+0000 7fd147fff640 1 -- 172.21.15.43:0/47801675 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c03c060 con 0x556e96cffd40 2024-04-21T08:41:41.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.433+0000 7fd147fff640 1 -- 172.21.15.43:0/47801675 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd15c005010 con 0x556e96cffd40 2024-04-21T08:41:41.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.433+0000 7fd171798740 1 -- 172.21.15.43:0/47801675 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd15404e900 con 0x556e96cffd40 2024-04-21T08:41:41.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.433+0000 7fd147fff640 1 -- 172.21.15.43:0/47801675 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd15c043050 con 0x556e96cffd40 2024-04-21T08:41:41.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.433+0000 7fd171798740 10 client.16905 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:41.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.433+0000 7fd171798740 1 -- 172.21.15.43:0/47801675 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556e96d835e0 con 0x556e96cffd40 2024-04-21T08:41:41.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.433+0000 7fd147fff640 1 -- 172.21.15.43:0/47801675 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd15404e900 con 0x556e96cffd40 2024-04-21T08:41:41.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.434+0000 7fd147fff640 1 -- 172.21.15.43:0/47801675 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd15c02c5e0 con 0x556e96cffd40 2024-04-21T08:41:41.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.434+0000 7fd171798740 10 client.16905 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:41.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.434+0000 7fd171798740 10 client.16905 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:41.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.434+0000 7fd171798740 20 client.16905 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:41.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.434+0000 7fd171798740 1 --2- 172.21.15.43:0/47801675 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556e96d83bc0 0x556e96d85fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.434+0000 7fd171798740 4 client.16905 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:41:41.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.434+0000 7fd171798740 1 -- 172.21.15.43:0/47801675 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556e96c6ba70 con 0x556e96d83bc0 2024-04-21T08:41:41.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.434+0000 7fd167fff640 1 --2- 172.21.15.43:0/47801675 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556e96d83bc0 0x556e96d85fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.434+0000 7fd147fff640 10 client.16905 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:41.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.434+0000 7fd167fff640 1 --2- 172.21.15.43:0/47801675 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556e96d83bc0 0x556e96d85fe0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7240e604bc78c9e3 server_cookie=938e0041d1303bfa in_seq=0 out_seq=0 2024-04-21T08:41:41.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.435+0000 7fd147fff640 1 -- 172.21.15.43:0/47801675 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556e96c6ba70 con 0x556e96d83bc0 2024-04-21T08:41:41.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.435+0000 7fd147fff640 10 client.16905 handle_command_reply: tid=0 2024-04-21T08:41:41.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.435+0000 7fd171798740 1 client.16905 shutdown 2024-04-21T08:41:41.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.435+0000 7fd171798740 1 -- 172.21.15.43:0/47801675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96cffd40 msgr2=0x556e96d11200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:41.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.435+0000 7fd171798740 1 --2- 172.21.15.43:0/47801675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96cffd40 0x556e96d11200 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x556e96d09750 tx=0x7fd15c023aa0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.435+0000 7fd171798740 1 -- 172.21.15.43:0/47801675 shutdown_connections 2024-04-21T08:41:41.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.435+0000 7fd171798740 1 --2- 172.21.15.43:0/47801675 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556e96d83bc0 0x556e96d85fe0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.435+0000 7fd171798740 1 --2- 172.21.15.43:0/47801675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e96d0ce90 0x556e96d11740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.435+0000 7fd171798740 1 --2- 172.21.15.43:0/47801675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e96cffd40 0x556e96d11200 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.435+0000 7fd171798740 1 --2- 172.21.15.43:0/47801675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e96cff390 0x556e96d10cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.435+0000 7fd147fff640 0 client.16905 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:41.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.435+0000 7fd171798740 1 -- 172.21.15.43:0/47801675 >> 172.21.15.43:0/47801675 conn(0x556e96cf9cb0 msgr2=0x556e96d09e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:41.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.435+0000 7fd171798740 1 -- 172.21.15.43:0/47801675 shutdown_connections 2024-04-21T08:41:41.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.435+0000 7fd171798740 1 -- 172.21.15.43:0/47801675 wait complete. 2024-04-21T08:41:41.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.435+0000 7fd171798740 20 client.16905 trim_cache size 0 max 16384 2024-04-21T08:41:41.444 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:41:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:41:41.444 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:41:41.444 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:41:41.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.435+0000 7fd16eac8640 1 -- 172.21.15.43:0/914723753 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd1580a3da0 msgr2=0x7fd1580a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:41.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.435+0000 7fd16eac8640 1 --2- 172.21.15.43:0/914723753 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd1580a3da0 0x7fd1580a6260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd1600ce0f0 tx=0x7fd16006dd80 comp rx=0 tx=0).stop 2024-04-21T08:41:41.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.435+0000 7fd16eac8640 1 -- 172.21.15.43:0/914723753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd16819cf50 msgr2=0x7fd16819d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:41.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.435+0000 7fd16eac8640 1 --2- 172.21.15.43:0/914723753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd16819cf50 0x7fd16819d400 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fd160009fd0 tx=0x7fd16000def0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.436+0000 7fd16eac8640 1 -- 172.21.15.43:0/914723753 shutdown_connections 2024-04-21T08:41:41.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.436+0000 7fd16eac8640 1 --2- 172.21.15.43:0/914723753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd16819cf50 0x7fd16819d400 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.436+0000 7fd16eac8640 1 --2- 172.21.15.43:0/914723753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd168106160 0x7fd168198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.436+0000 7fd16eac8640 1 --2- 172.21.15.43:0/914723753 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd1580a3da0 0x7fd1580a6260 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.436+0000 7fd16eac8640 1 --2- 172.21.15.43:0/914723753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd168101090 0x7fd168198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.436+0000 7fd16eac8640 1 -- 172.21.15.43:0/914723753 >> 172.21.15.43:0/914723753 conn(0x7fd1680771d0 msgr2=0x7fd16806a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:41.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.436+0000 7fd16eac8640 1 -- 172.21.15.43:0/914723753 shutdown_connections 2024-04-21T08:41:41.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.436+0000 7fd16eac8640 1 -- 172.21.15.43:0/914723753 wait complete. 2024-04-21T08:41:41.447 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:41:41.447 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:41:41.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.550+0000 7f0b324d2640 1 -- 172.21.15.43:0/2053219965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2c106fb0 msgr2=0x7f0b2c107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:41.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.550+0000 7f0b324d2640 1 --2- 172.21.15.43:0/2053219965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2c106fb0 0x7f0b2c107390 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009900 tx=0x7f0b1c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:41.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.550+0000 7f0b324d2640 1 -- 172.21.15.43:0/2053219965 shutdown_connections 2024-04-21T08:41:41.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.550+0000 7f0b324d2640 1 --2- 172.21.15.43:0/2053219965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b2c101b60 0x7f0b2c10dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.550+0000 7f0b324d2640 1 --2- 172.21.15.43:0/2053219965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b2c1011a0 0x7f0b2c101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.550+0000 7f0b324d2640 1 --2- 172.21.15.43:0/2053219965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2c106fb0 0x7f0b2c107390 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.550+0000 7f0b324d2640 1 -- 172.21.15.43:0/2053219965 >> 172.21.15.43:0/2053219965 conn(0x7f0b2c077e30 msgr2=0x7f0b2c078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.550+0000 7f0b324d2640 1 -- 172.21.15.43:0/2053219965 shutdown_connections 2024-04-21T08:41:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.550+0000 7f0b324d2640 1 -- 172.21.15.43:0/2053219965 wait complete. 2024-04-21T08:41:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.551+0000 7f0b324d2640 1 Processor -- start 2024-04-21T08:41:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.551+0000 7f0b324d2640 1 -- start start 2024-04-21T08:41:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.551+0000 7f0b324d2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2c1011a0 0x7f0b2c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.551+0000 7f0b324d2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b2c101b60 0x7f0b2c198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.551+0000 7f0b324d2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b2c106fb0 0x7f0b2c19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.551+0000 7f0b324d2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c111050 con 0x7f0b2c1011a0 2024-04-21T08:41:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.551+0000 7f0b324d2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c110ed0 con 0x7f0b2c106fb0 2024-04-21T08:41:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.551+0000 7f0b324d2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c1111d0 con 0x7f0b2c101b60 2024-04-21T08:41:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.551+0000 7f0b2bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2c1011a0 0x7f0b2c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.551+0000 7f0b2bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2c1011a0 0x7f0b2c1986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33988/0 (socket says 172.21.15.43:33988) 2024-04-21T08:41:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.551+0000 7f0b2bfff640 1 -- 172.21.15.43:0/1840436474 learned_addr learned my addr 172.21.15.43:0/1840436474 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.552+0000 7f0b2b7fe640 1 --2- 172.21.15.43:0/1840436474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b2c101b60 0x7f0b2c198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.552+0000 7f0b30a6f640 1 --2- 172.21.15.43:0/1840436474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b2c106fb0 0x7f0b2c19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.552+0000 7f0b2bfff640 1 -- 172.21.15.43:0/1840436474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b2c101b60 msgr2=0x7f0b2c198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.552+0000 7f0b2bfff640 1 --2- 172.21.15.43:0/1840436474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b2c101b60 0x7f0b2c198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.552+0000 7f0b2bfff640 1 -- 172.21.15.43:0/1840436474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b2c106fb0 msgr2=0x7f0b2c19cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.552+0000 7f0b2bfff640 1 --2- 172.21.15.43:0/1840436474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b2c106fb0 0x7f0b2c19cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.552+0000 7f0b2bfff640 1 -- 172.21.15.43:0/1840436474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2c19d6b0 con 0x7f0b2c1011a0 2024-04-21T08:41:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.552+0000 7f0b2bfff640 1 --2- 172.21.15.43:0/1840436474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2c1011a0 0x7f0b2c1986c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c005940 tx=0x7f0b1c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.552+0000 7f0b297fa640 1 -- 172.21.15.43:0/1840436474 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c043020 con 0x7f0b2c1011a0 2024-04-21T08:41:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.552+0000 7f0b297fa640 1 -- 172.21.15.43:0/1840436474 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b1c02caf0 con 0x7f0b2c1011a0 2024-04-21T08:41:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.552+0000 7f0b324d2640 1 -- 172.21.15.43:0/1840436474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b2c19d940 con 0x7f0b2c1011a0 2024-04-21T08:41:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.552+0000 7f0b297fa640 1 -- 172.21.15.43:0/1840436474 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c003af0 con 0x7f0b2c1011a0 2024-04-21T08:41:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.552+0000 7f0b324d2640 1 -- 172.21.15.43:0/1840436474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b2c1a5230 con 0x7f0b2c1011a0 2024-04-21T08:41:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.553+0000 7f0b324d2640 1 -- 172.21.15.43:0/1840436474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b2c102550 con 0x7f0b2c1011a0 2024-04-21T08:41:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.554+0000 7f0b297fa640 1 -- 172.21.15.43:0/1840436474 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0b1c046020 con 0x7f0b2c1011a0 2024-04-21T08:41:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.555+0000 7f0b297fa640 1 --2- 172.21.15.43:0/1840436474 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0b180a3d20 0x7f0b180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.555+0000 7f0b297fa640 1 -- 172.21.15.43:0/1840436474 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f0b1c0f2d00 con 0x7f0b2c1011a0 2024-04-21T08:41:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.555+0000 7f0b2b7fe640 1 --2- 172.21.15.43:0/1840436474 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0b180a3d20 0x7f0b180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.555+0000 7f0b2b7fe640 1 --2- 172.21.15.43:0/1840436474 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0b180a3d20 0x7f0b180a61e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c100c60 tx=0x7f0b14005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:41.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.560+0000 7f0b297fa640 1 -- 172.21.15.43:0/1840436474 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0b1c04d020 con 0x7f0b2c1011a0 2024-04-21T08:41:41.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.785+0000 7f0b324d2640 1 -- 172.21.15.43:0/1840436474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b2c1a5400 con 0x7f0b2c1011a0 2024-04-21T08:41:41.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.786+0000 7f0b297fa640 1 -- 172.21.15.43:0/1840436474 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7f0b1c0bf0e0 con 0x7f0b2c1011a0 2024-04-21T08:41:41.785 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:41:41.785 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:41:41.785 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:41:41.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.788+0000 7f0b324d2640 1 -- 172.21.15.43:0/1840436474 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0b180a3d20 msgr2=0x7f0b180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:41.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.788+0000 7f0b324d2640 1 --2- 172.21.15.43:0/1840436474 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0b180a3d20 0x7f0b180a61e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c100c60 tx=0x7f0b14005f90 comp rx=0 tx=0).stop 2024-04-21T08:41:41.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.788+0000 7f0b324d2640 1 -- 172.21.15.43:0/1840436474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2c1011a0 msgr2=0x7f0b2c1986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:41.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.788+0000 7f0b324d2640 1 --2- 172.21.15.43:0/1840436474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2c1011a0 0x7f0b2c1986c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c005940 tx=0x7f0b1c0045e0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.788+0000 7f0b324d2640 1 -- 172.21.15.43:0/1840436474 shutdown_connections 2024-04-21T08:41:41.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.788+0000 7f0b324d2640 1 --2- 172.21.15.43:0/1840436474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b2c106fb0 0x7f0b2c19cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.788+0000 7f0b324d2640 1 --2- 172.21.15.43:0/1840436474 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0b180a3d20 0x7f0b180a61e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.788+0000 7f0b324d2640 1 --2- 172.21.15.43:0/1840436474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b2c101b60 0x7f0b2c198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.788+0000 7f0b324d2640 1 --2- 172.21.15.43:0/1840436474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2c1011a0 0x7f0b2c1986c0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.788+0000 7f0b324d2640 1 -- 172.21.15.43:0/1840436474 >> 172.21.15.43:0/1840436474 conn(0x7f0b2c077e30 msgr2=0x7f0b2c06a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:41.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.788+0000 7f0b324d2640 1 -- 172.21.15.43:0/1840436474 shutdown_connections 2024-04-21T08:41:41.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.788+0000 7f0b324d2640 1 -- 172.21.15.43:0/1840436474 wait complete. 2024-04-21T08:41:41.798 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:41:41.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.903+0000 7f335385c640 1 -- 172.21.15.43:0/711962170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f334c107260 msgr2=0x7f334c10ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:41.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.903+0000 7f335385c640 1 --2- 172.21.15.43:0/711962170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f334c107260 0x7f334c10ee30 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f3344009ca0 tx=0x7f334402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.903+0000 7f335385c640 1 -- 172.21.15.43:0/711962170 shutdown_connections 2024-04-21T08:41:41.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.903+0000 7f335385c640 1 --2- 172.21.15.43:0/711962170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f334c107260 0x7f334c10ee30 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.903+0000 7f335385c640 1 --2- 172.21.15.43:0/711962170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f334c102b40 0x7f334c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.904+0000 7f335385c640 1 --2- 172.21.15.43:0/711962170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f334c102190 0x7f334c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.904+0000 7f335385c640 1 -- 172.21.15.43:0/711962170 >> 172.21.15.43:0/711962170 conn(0x7f334c0793a0 msgr2=0x7f334c0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:41.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.904+0000 7f335385c640 1 -- 172.21.15.43:0/711962170 shutdown_connections 2024-04-21T08:41:41.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.904+0000 7f335385c640 1 -- 172.21.15.43:0/711962170 wait complete. 2024-04-21T08:41:41.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.904+0000 7f335385c640 1 Processor -- start 2024-04-21T08:41:41.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.904+0000 7f335385c640 1 -- start start 2024-04-21T08:41:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.904+0000 7f335385c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f334c102190 0x7f334c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.904+0000 7f335385c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f334c107260 0x7f334c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.904+0000 7f335385c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f334c19d010 0x7f334c19d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.904+0000 7f335385c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1121d0 con 0x7f334c102190 2024-04-21T08:41:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.904+0000 7f335385c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f334c112050 con 0x7f334c107260 2024-04-21T08:41:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.904+0000 7f335385c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f334c112350 con 0x7f334c19d010 2024-04-21T08:41:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.904+0000 7f33515f8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f334c102190 0x7f334c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.905+0000 7f33515f8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f334c102190 0x7f334c198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34004/0 (socket says 172.21.15.43:34004) 2024-04-21T08:41:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.905+0000 7f33515f8640 1 -- 172.21.15.43:0/2357764598 learned_addr learned my addr 172.21.15.43:0/2357764598 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.905+0000 7f3350df7640 1 --2- 172.21.15.43:0/2357764598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f334c107260 0x7f334c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.905+0000 7f33515f8640 1 -- 172.21.15.43:0/2357764598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f334c19d010 msgr2=0x7f334c19d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.905+0000 7f33515f8640 1 --2- 172.21.15.43:0/2357764598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f334c19d010 0x7f334c19d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.905+0000 7f33515f8640 1 -- 172.21.15.43:0/2357764598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f334c107260 msgr2=0x7f334c198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:41.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.905+0000 7f33515f8640 1 --2- 172.21.15.43:0/2357764598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f334c107260 0x7f334c198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.905+0000 7f33515f8640 1 -- 172.21.15.43:0/2357764598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334c19db00 con 0x7f334c102190 2024-04-21T08:41:41.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.905+0000 7f33515f8640 1 --2- 172.21.15.43:0/2357764598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f334c102190 0x7f334c198750 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f334800d770 tx=0x7f334800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:41.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.905+0000 7f33427fc640 1 -- 172.21.15.43:0/2357764598 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3348017070 con 0x7f334c102190 2024-04-21T08:41:41.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.905+0000 7f33427fc640 1 -- 172.21.15.43:0/2357764598 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33480054d0 con 0x7f334c102190 2024-04-21T08:41:41.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.905+0000 7f335385c640 1 -- 172.21.15.43:0/2357764598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f334c19ddf0 con 0x7f334c102190 2024-04-21T08:41:41.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.905+0000 7f33427fc640 1 -- 172.21.15.43:0/2357764598 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3348004c30 con 0x7f334c102190 2024-04-21T08:41:41.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.905+0000 7f335385c640 1 -- 172.21.15.43:0/2357764598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f334c1a5590 con 0x7f334c102190 2024-04-21T08:41:41.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.907+0000 7f33427fc640 1 -- 172.21.15.43:0/2357764598 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3348004e10 con 0x7f334c102190 2024-04-21T08:41:41.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.908+0000 7f3356451740 1 Processor -- start 2024-04-21T08:41:41.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.908+0000 7f3356451740 1 -- start start 2024-04-21T08:41:41.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.908+0000 7f3356451740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d56e4d0 0x56018d56e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.908+0000 7f3356451740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56018d56ee80 0x56018d577430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.908+0000 7f3356451740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56018d577a00 0x56018d579e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.909+0000 7f3356451740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56018d4d31f0 con 0x56018d56e4d0 2024-04-21T08:41:41.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.909+0000 7f3356451740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56018d4704b0 con 0x56018d56ee80 2024-04-21T08:41:41.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.909+0000 7f3356451740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56018d4cf1b0 con 0x56018d577a00 2024-04-21T08:41:41.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.909+0000 7f33427fc640 1 --2- 172.21.15.43:0/2357764598 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f33380a3d20 0x7f33380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.909+0000 7f3351df9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d56e4d0 0x56018d56e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.909+0000 7f3351df9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d56e4d0 0x56018d56e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34016/0 (socket says 172.21.15.43:34016) 2024-04-21T08:41:41.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.909+0000 7f3351df9640 1 -- 172.21.15.43:0/2397363597 learned_addr learned my addr 172.21.15.43:0/2397363597 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:41.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.909+0000 7f3351df9640 1 --2- 172.21.15.43:0/2357764598 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f33380a3d20 0x7f33380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.909+0000 7f33427fc640 1 -- 172.21.15.43:0/2357764598 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f33480d0700 con 0x7f334c102190 2024-04-21T08:41:41.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.909+0000 7f33515f8640 1 --2- 172.21.15.43:0/2397363597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56018d56ee80 0x56018d577430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.909+0000 7f3350df7640 1 --2- 172.21.15.43:0/2397363597 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56018d577a00 0x56018d579e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.909+0000 7f3351df9640 1 -- 172.21.15.43:0/2397363597 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56018d577a00 msgr2=0x56018d579e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.909+0000 7f3351df9640 1 --2- 172.21.15.43:0/2397363597 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56018d577a00 0x56018d579e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.909+0000 7f3351df9640 1 -- 172.21.15.43:0/2397363597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56018d56ee80 msgr2=0x56018d577430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.909+0000 7f3351df9640 1 --2- 172.21.15.43:0/2397363597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56018d56ee80 0x56018d577430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.909+0000 7f3351df9640 1 -- 172.21.15.43:0/2397363597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56018d4621b0 con 0x56018d56e4d0 2024-04-21T08:41:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.909+0000 7f3351df9640 1 --2- 172.21.15.43:0/2397363597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d56e4d0 0x56018d56e8b0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f3344004af0 tx=0x7f3344032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56cd073401f6175 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.909+0000 7f3351df9640 1 --2- 172.21.15.43:0/2357764598 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f33380a3d20 0x7f33380a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3344005cc0 tx=0x7f3344045000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.909+0000 7f33377fe640 1 -- 172.21.15.43:0/2397363597 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3344039040 con 0x56018d56e4d0 2024-04-21T08:41:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.910+0000 7f33377fe640 1 -- 172.21.15.43:0/2397363597 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f334404f070 con 0x56018d56e4d0 2024-04-21T08:41:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.910+0000 7f33377fe640 1 -- 172.21.15.43:0/2397363597 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3344034d30 con 0x56018d56e4d0 2024-04-21T08:41:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.910+0000 7f3356451740 1 -- 172.21.15.43:0/2397363597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d56e4d0 msgr2=0x56018d56e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.910+0000 7f3356451740 1 --2- 172.21.15.43:0/2397363597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d56e4d0 0x56018d56e8b0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f3344004af0 tx=0x7f3344032eb0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.910+0000 7f3356451740 1 -- 172.21.15.43:0/2397363597 shutdown_connections 2024-04-21T08:41:41.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.910+0000 7f3356451740 1 --2- 172.21.15.43:0/2397363597 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56018d577a00 0x56018d579e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.910+0000 7f3356451740 1 --2- 172.21.15.43:0/2397363597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56018d56ee80 0x56018d577430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.910+0000 7f3356451740 1 --2- 172.21.15.43:0/2397363597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d56e4d0 0x56018d56e8b0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.910+0000 7f3356451740 1 -- 172.21.15.43:0/2397363597 >> 172.21.15.43:0/2397363597 conn(0x56018d568ba0 msgr2=0x56018d568f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:41.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.910+0000 7f3356451740 1 -- 172.21.15.43:0/2397363597 shutdown_connections 2024-04-21T08:41:41.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.910+0000 7f3356451740 1 -- 172.21.15.43:0/2397363597 wait complete. 2024-04-21T08:41:41.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.911+0000 7f3356451740 1 -- start start 2024-04-21T08:41:41.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.911+0000 7f3356451740 1 Processor -- start 2024-04-21T08:41:41.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.911+0000 7f3356451740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d56e4d0 0x56018d5773d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.911+0000 7f3356451740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56018d56ee80 0x56018d574760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.911+0000 7f3356451740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56018d577a00 0x56018d574e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.911+0000 7f3356451740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56018d57a910 con 0x56018d56e4d0 2024-04-21T08:41:41.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.911+0000 7f3356451740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56018d48c500 con 0x56018d56ee80 2024-04-21T08:41:41.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.911+0000 7f3356451740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56018d57aa90 con 0x56018d577a00 2024-04-21T08:41:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.911+0000 7f3356451740 10 client.0 _ll_register_callbacks cb 0x56018d45ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.911+0000 7f33515f8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d56e4d0 0x56018d5773d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.911+0000 7f33515f8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d56e4d0 0x56018d5773d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34030/0 (socket says 172.21.15.43:34030) 2024-04-21T08:41:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.911+0000 7f33515f8640 1 -- 172.21.15.43:0/261810851 learned_addr learned my addr 172.21.15.43:0/261810851 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.911+0000 7f3350df7640 1 --2- 172.21.15.43:0/261810851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56018d56ee80 0x56018d574760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.911+0000 7f33515f8640 1 -- 172.21.15.43:0/261810851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56018d577a00 msgr2=0x56018d574e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.911+0000 7f33515f8640 1 --2- 172.21.15.43:0/261810851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56018d577a00 0x56018d574e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.911+0000 7f33515f8640 1 -- 172.21.15.43:0/261810851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56018d56ee80 msgr2=0x56018d574760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.911+0000 7f33515f8640 1 --2- 172.21.15.43:0/261810851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56018d56ee80 0x56018d574760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.912+0000 7f33515f8640 1 -- 172.21.15.43:0/261810851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56018d1e53f0 con 0x56018d56e4d0 2024-04-21T08:41:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.912+0000 7f3350df7640 1 --2- 172.21.15.43:0/261810851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56018d56ee80 0x56018d574760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.912+0000 7f33515f8640 1 --2- 172.21.15.43:0/261810851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d56e4d0 0x56018d5773d0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3348098c40 tx=0x7f3348005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5213e80be9b8fcf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.912+0000 7f3334ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.912+0000 7f3334ff9640 1 -- 172.21.15.43:0/261810851 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33480b4070 con 0x56018d56e4d0 2024-04-21T08:41:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.912+0000 7f3334ff9640 1 -- 172.21.15.43:0/261810851 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f33480988e0 con 0x56018d56e4d0 2024-04-21T08:41:41.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.912+0000 7f3356451740 1 -- 172.21.15.43:0/261810851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f334404f070 con 0x56018d56e4d0 2024-04-21T08:41:41.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.912+0000 7f3334ff9640 1 -- 172.21.15.43:0/261810851 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f33480cf040 con 0x56018d56e4d0 2024-04-21T08:41:41.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.912+0000 7f3356451740 10 client.16923 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:41.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.912+0000 7f3356451740 1 -- 172.21.15.43:0/261810851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56018d2af7d0 con 0x56018d56e4d0 2024-04-21T08:41:41.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.912+0000 7f3334ff9640 1 -- 172.21.15.43:0/261810851 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f334404f070 con 0x56018d56e4d0 2024-04-21T08:41:41.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.912+0000 7f3334ff9640 1 -- 172.21.15.43:0/261810851 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f33480a4a50 con 0x56018d56e4d0 2024-04-21T08:41:41.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.912+0000 7f3356451740 10 client.16923 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:41.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.912+0000 7f3356451740 10 client.16923 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:41:41.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.912+0000 7f3356451740 20 client.16923 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:41.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.912+0000 7f3356451740 1 --2- 172.21.15.43:0/261810851 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x56018d587d70 0x56018d58a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.912+0000 7f3356451740 4 client.16923 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:41.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.912+0000 7f3356451740 1 -- 172.21.15.43:0/261810851 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56018d472930 con 0x56018d587d70 2024-04-21T08:41:41.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.913+0000 7f3350df7640 1 --2- 172.21.15.43:0/261810851 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x56018d587d70 0x56018d58a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.913+0000 7f3350df7640 1 --2- 172.21.15.43:0/261810851 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x56018d587d70 0x56018d58a1c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=aa5d168a61741f7a server_cookie=159e69964fd3643 in_seq=0 out_seq=0 2024-04-21T08:41:41.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.913+0000 7f3334ff9640 10 client.16923 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:41:41.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.914+0000 7f3334ff9640 1 -- 172.21.15.43:0/261810851 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56018d472930 con 0x56018d587d70 2024-04-21T08:41:41.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.914+0000 7f3334ff9640 10 client.16923 handle_command_reply: tid=0 2024-04-21T08:41:41.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.914+0000 7f3356451740 1 client.16923 shutdown 2024-04-21T08:41:41.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.914+0000 7f3356451740 1 -- 172.21.15.43:0/261810851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d56e4d0 msgr2=0x56018d5773d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:41.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.914+0000 7f3356451740 1 --2- 172.21.15.43:0/261810851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d56e4d0 0x56018d5773d0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3348098c40 tx=0x7f3348005790 comp rx=0 tx=0).stop 2024-04-21T08:41:41.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.914+0000 7f3356451740 1 -- 172.21.15.43:0/261810851 shutdown_connections 2024-04-21T08:41:41.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.914+0000 7f3356451740 1 --2- 172.21.15.43:0/261810851 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x56018d587d70 0x56018d58a1c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.914+0000 7f3356451740 1 --2- 172.21.15.43:0/261810851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56018d577a00 0x56018d574e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.914+0000 7f3356451740 1 --2- 172.21.15.43:0/261810851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56018d56ee80 0x56018d574760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.914+0000 7f3334ff9640 0 client.16923 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:41:41.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.914+0000 7f3356451740 1 --2- 172.21.15.43:0/261810851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d56e4d0 0x56018d5773d0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.914+0000 7f3356451740 1 -- 172.21.15.43:0/261810851 >> 172.21.15.43:0/261810851 conn(0x56018d568ba0 msgr2=0x56018d56a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:41.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.915+0000 7f3356451740 1 -- 172.21.15.43:0/261810851 shutdown_connections 2024-04-21T08:41:41.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.915+0000 7f3356451740 1 -- 172.21.15.43:0/261810851 wait complete. 2024-04-21T08:41:41.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.915+0000 7f3356451740 20 client.16923 trim_cache size 0 max 16384 2024-04-21T08:41:41.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.928+0000 7f3356451740 1 Processor -- start 2024-04-21T08:41:41.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.928+0000 7f3356451740 1 -- start start 2024-04-21T08:41:41.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.928+0000 7f3356451740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d56e4d0 0x56018d5837d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.928+0000 7f3356451740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56018d56ee80 0x56018d576e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.928+0000 7f3356451740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56018d577a00 0x56018d5773d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.928+0000 7f3356451740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1af3b0 con 0x56018d56e4d0 2024-04-21T08:41:41.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.928+0000 7f3356451740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1b0ef0 con 0x56018d577a00 2024-04-21T08:41:41.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.928+0000 7f3356451740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1afa80 con 0x56018d56ee80 2024-04-21T08:41:41.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.928+0000 7f33515f8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d56e4d0 0x56018d5837d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.928+0000 7f33515f8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d56e4d0 0x56018d5837d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34040/0 (socket says 172.21.15.43:34040) 2024-04-21T08:41:41.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.928+0000 7f33515f8640 1 -- 172.21.15.43:0/3534724225 learned_addr learned my addr 172.21.15.43:0/3534724225 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:41.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.928+0000 7f3350df7640 1 --2- 172.21.15.43:0/3534724225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56018d56ee80 0x56018d576e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.929+0000 7f33515f8640 1 -- 172.21.15.43:0/3534724225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56018d56ee80 msgr2=0x56018d576e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:41.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.929+0000 7f3351df9640 1 --2- 172.21.15.43:0/3534724225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56018d577a00 0x56018d5773d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.929+0000 7f33515f8640 1 --2- 172.21.15.43:0/3534724225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56018d56ee80 0x56018d576e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.929+0000 7f33515f8640 1 -- 172.21.15.43:0/3534724225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56018d577a00 msgr2=0x56018d5773d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:41.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.929+0000 7f33515f8640 1 --2- 172.21.15.43:0/3534724225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56018d577a00 0x56018d5773d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.929+0000 7f33515f8640 1 -- 172.21.15.43:0/3534724225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56018d562fe0 con 0x56018d56e4d0 2024-04-21T08:41:41.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.929+0000 7f33515f8640 1 --2- 172.21.15.43:0/3534724225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d56e4d0 0x56018d5837d0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f3348009f40 tx=0x7f334800def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf99144d57a6ec56 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:41.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.929+0000 7f3336ffd640 1 -- 172.21.15.43:0/3534724225 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33480b4070 con 0x56018d56e4d0 2024-04-21T08:41:41.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.929+0000 7f3336ffd640 1 -- 172.21.15.43:0/3534724225 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33480a0170 con 0x56018d56e4d0 2024-04-21T08:41:41.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.929+0000 7f3336ffd640 1 -- 172.21.15.43:0/3534724225 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3348020040 con 0x56018d56e4d0 2024-04-21T08:41:41.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.929+0000 7f3356451740 1 -- 172.21.15.43:0/3534724225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d56e4d0 msgr2=0x56018d5837d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:41.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.929+0000 7f3356451740 1 --2- 172.21.15.43:0/3534724225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d56e4d0 0x56018d5837d0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f3348009f40 tx=0x7f334800def0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.930+0000 7f3356451740 1 -- 172.21.15.43:0/3534724225 shutdown_connections 2024-04-21T08:41:41.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.930+0000 7f3356451740 1 --2- 172.21.15.43:0/3534724225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56018d577a00 0x56018d5773d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.930+0000 7f3356451740 1 --2- 172.21.15.43:0/3534724225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56018d56ee80 0x56018d576e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.930+0000 7f3356451740 1 --2- 172.21.15.43:0/3534724225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d56e4d0 0x56018d5837d0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.930+0000 7f3356451740 1 -- 172.21.15.43:0/3534724225 >> 172.21.15.43:0/3534724225 conn(0x56018d568ba0 msgr2=0x56018d567c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:41.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.930+0000 7f3356451740 1 -- 172.21.15.43:0/3534724225 shutdown_connections 2024-04-21T08:41:41.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.930+0000 7f3356451740 1 -- 172.21.15.43:0/3534724225 wait complete. 2024-04-21T08:41:41.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.930+0000 7f3356451740 1 -- start start 2024-04-21T08:41:41.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.930+0000 7f3356451740 1 Processor -- start 2024-04-21T08:41:41.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.930+0000 7f3356451740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56018d56e4d0 0x56018d571850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.930+0000 7f3356451740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56018d56ee80 0x56018d571d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.930+0000 7f3356451740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d577a00 0x56018d5ed300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.930+0000 7f3356451740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1b0820 con 0x56018d577a00 2024-04-21T08:41:41.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.930+0000 7f3356451740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1b0150 con 0x56018d56e4d0 2024-04-21T08:41:41.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.930+0000 7f3356451740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1ad390 con 0x56018d56ee80 2024-04-21T08:41:41.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.930+0000 7f3356451740 10 client.0 _ll_register_callbacks cb 0x7f334809bdd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:41.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.930+0000 7f3351df9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d577a00 0x56018d5ed300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.930+0000 7f3351df9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d577a00 0x56018d5ed300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34050/0 (socket says 172.21.15.43:34050) 2024-04-21T08:41:41.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.930+0000 7f3351df9640 1 -- 172.21.15.43:0/1743818936 learned_addr learned my addr 172.21.15.43:0/1743818936 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:41.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.931+0000 7f3350df7640 1 --2- 172.21.15.43:0/1743818936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56018d56ee80 0x56018d571d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.931+0000 7f33515f8640 1 --2- 172.21.15.43:0/1743818936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56018d56e4d0 0x56018d571850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.931+0000 7f3351df9640 1 -- 172.21.15.43:0/1743818936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56018d56ee80 msgr2=0x56018d571d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:41.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.931+0000 7f3351df9640 1 --2- 172.21.15.43:0/1743818936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56018d56ee80 0x56018d571d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.931+0000 7f3351df9640 1 -- 172.21.15.43:0/1743818936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56018d56e4d0 msgr2=0x56018d571850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:41.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.931+0000 7f3351df9640 1 --2- 172.21.15.43:0/1743818936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56018d56e4d0 0x56018d571850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.931+0000 7f3351df9640 1 -- 172.21.15.43:0/1743818936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56018d574ad0 con 0x56018d577a00 2024-04-21T08:41:41.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.931+0000 7f33515f8640 1 --2- 172.21.15.43:0/1743818936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56018d56e4d0 0x56018d571850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:41.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.931+0000 7f3351df9640 1 --2- 172.21.15.43:0/1743818936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d577a00 0x56018d5ed300 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f334404c040 tx=0x7f3344005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548c9e1fe5917293 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:41.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.931+0000 7f3334ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:41.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.931+0000 7f3334ff9640 1 -- 172.21.15.43:0/1743818936 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3344058060 con 0x56018d577a00 2024-04-21T08:41:41.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.931+0000 7f3334ff9640 1 -- 172.21.15.43:0/1743818936 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f3344034710 con 0x56018d577a00 2024-04-21T08:41:41.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.931+0000 7f3356451740 1 -- 172.21.15.43:0/1743818936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33480a0170 con 0x56018d577a00 2024-04-21T08:41:41.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.931+0000 7f3334ff9640 1 -- 172.21.15.43:0/1743818936 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f334405f050 con 0x56018d577a00 2024-04-21T08:41:41.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.931+0000 7f3356451740 10 client.16929 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:41.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.931+0000 7f3356451740 1 -- 172.21.15.43:0/1743818936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56018d5868c0 con 0x56018d577a00 2024-04-21T08:41:41.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.932+0000 7f3334ff9640 1 -- 172.21.15.43:0/1743818936 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33480a0170 con 0x56018d577a00 2024-04-21T08:41:41.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.932+0000 7f3334ff9640 1 -- 172.21.15.43:0/1743818936 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f334403f050 con 0x56018d577a00 2024-04-21T08:41:41.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.932+0000 7f3356451740 10 client.16929 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:41.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.932+0000 7f3356451740 10 client.16929 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:41:41.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.932+0000 7f3356451740 20 client.16929 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:41.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.932+0000 7f3356451740 1 --2- 172.21.15.43:0/1743818936 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x56018d5eea10 0x56018d5f0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:41.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.932+0000 7f3356451740 4 client.16929 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:41:41.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.932+0000 7f3356451740 1 -- 172.21.15.43:0/1743818936 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56018d4e0c10 con 0x56018d5eea10 2024-04-21T08:41:41.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.932+0000 7f33515f8640 1 --2- 172.21.15.43:0/1743818936 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x56018d5eea10 0x56018d5f0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:41.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.932+0000 7f33515f8640 1 --2- 172.21.15.43:0/1743818936 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x56018d5eea10 0x56018d5f0e00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a63fdc593c39f472 server_cookie=34e37769afdc7c31 in_seq=0 out_seq=0 2024-04-21T08:41:41.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.932+0000 7f3334ff9640 10 client.16929 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:41:41.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.933+0000 7f3334ff9640 1 -- 172.21.15.43:0/1743818936 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56018d4e0c10 con 0x56018d5eea10 2024-04-21T08:41:41.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.933+0000 7f3334ff9640 10 client.16929 handle_command_reply: tid=0 2024-04-21T08:41:41.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.933+0000 7f3356451740 1 client.16929 shutdown 2024-04-21T08:41:41.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.933+0000 7f3356451740 1 -- 172.21.15.43:0/1743818936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d577a00 msgr2=0x56018d5ed300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:41.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.933+0000 7f3356451740 1 --2- 172.21.15.43:0/1743818936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d577a00 0x56018d5ed300 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f334404c040 tx=0x7f3344005d80 comp rx=0 tx=0).stop 2024-04-21T08:41:41.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.933+0000 7f3356451740 1 -- 172.21.15.43:0/1743818936 shutdown_connections 2024-04-21T08:41:41.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.933+0000 7f3356451740 1 --2- 172.21.15.43:0/1743818936 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x56018d5eea10 0x56018d5f0e00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.933+0000 7f3356451740 1 --2- 172.21.15.43:0/1743818936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56018d577a00 0x56018d5ed300 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.933+0000 7f3356451740 1 --2- 172.21.15.43:0/1743818936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56018d56ee80 0x56018d571d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.933+0000 7f3334ff9640 0 client.16929 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:41:41.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.933+0000 7f3356451740 1 --2- 172.21.15.43:0/1743818936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56018d56e4d0 0x56018d571850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.933+0000 7f3356451740 1 -- 172.21.15.43:0/1743818936 >> 172.21.15.43:0/1743818936 conn(0x56018d568ba0 msgr2=0x56018d569ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:41.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.933+0000 7f3356451740 1 -- 172.21.15.43:0/1743818936 shutdown_connections 2024-04-21T08:41:41.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.933+0000 7f3356451740 1 -- 172.21.15.43:0/1743818936 wait complete. 2024-04-21T08:41:41.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.933+0000 7f3356451740 20 client.16929 trim_cache size 0 max 16384 2024-04-21T08:41:41.939 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:41:41.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.933+0000 7f335385c640 1 -- 172.21.15.43:0/2357764598 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f33380a3d20 msgr2=0x7f33380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:41.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.933+0000 7f335385c640 1 --2- 172.21.15.43:0/2357764598 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f33380a3d20 0x7f33380a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3344005cc0 tx=0x7f3344045000 comp rx=0 tx=0).stop 2024-04-21T08:41:41.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.933+0000 7f335385c640 1 -- 172.21.15.43:0/2357764598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f334c102190 msgr2=0x7f334c198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:41.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.934+0000 7f335385c640 1 --2- 172.21.15.43:0/2357764598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f334c102190 0x7f334c198750 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f334800d770 tx=0x7f334800dc30 comp rx=0 tx=0).stop 2024-04-21T08:41:41.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.934+0000 7f335385c640 1 -- 172.21.15.43:0/2357764598 shutdown_connections 2024-04-21T08:41:41.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.934+0000 7f335385c640 1 --2- 172.21.15.43:0/2357764598 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f33380a3d20 0x7f33380a61e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.934+0000 7f335385c640 1 --2- 172.21.15.43:0/2357764598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f334c19d010 0x7f334c19d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.934+0000 7f335385c640 1 --2- 172.21.15.43:0/2357764598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f334c107260 0x7f334c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.934+0000 7f335385c640 1 --2- 172.21.15.43:0/2357764598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f334c102190 0x7f334c198750 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:41.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.934+0000 7f335385c640 1 -- 172.21.15.43:0/2357764598 >> 172.21.15.43:0/2357764598 conn(0x7f334c0793a0 msgr2=0x7f334c100030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:41.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.934+0000 7f335385c640 1 -- 172.21.15.43:0/2357764598 shutdown_connections 2024-04-21T08:41:41.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:41.934+0000 7f335385c640 1 -- 172.21.15.43:0/2357764598 wait complete. 2024-04-21T08:41:41.943 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:41:42.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.049+0000 7f8ec6443640 1 -- 172.21.15.43:0/4103334875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec0102190 msgr2=0x7f8ec0102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.049+0000 7f8ec6443640 1 --2- 172.21.15.43:0/4103334875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec0102190 0x7f8ec0102570 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0009900 tx=0x7f8eb002bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:42.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.049+0000 7f8ec6443640 1 -- 172.21.15.43:0/4103334875 shutdown_connections 2024-04-21T08:41:42.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.049+0000 7f8ec6443640 1 --2- 172.21.15.43:0/4103334875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec0107260 0x7f8ec010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.049+0000 7f8ec6443640 1 --2- 172.21.15.43:0/4103334875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ec0102b40 0x7f8ec0106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.049+0000 7f8ec6443640 1 --2- 172.21.15.43:0/4103334875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec0102190 0x7f8ec0102570 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.049+0000 7f8ec6443640 1 -- 172.21.15.43:0/4103334875 >> 172.21.15.43:0/4103334875 conn(0x7f8ec00fd990 msgr2=0x7f8ec00ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:42.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.049+0000 7f8ec6443640 1 -- 172.21.15.43:0/4103334875 shutdown_connections 2024-04-21T08:41:42.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.049+0000 7f8ec6443640 1 -- 172.21.15.43:0/4103334875 wait complete. 2024-04-21T08:41:42.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.050+0000 7f8ec6443640 1 Processor -- start 2024-04-21T08:41:42.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.050+0000 7f8ec6443640 1 -- start start 2024-04-21T08:41:42.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.050+0000 7f8ec6443640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ec0102190 0x7f8ec0074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.050+0000 7f8ec6443640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec0107260 0x7f8ec0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.050+0000 7f8ec6443640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec0074cd0 0x7f8ec01a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.050+0000 7f8ec6443640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec01121c0 con 0x7f8ec0074cd0 2024-04-21T08:41:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.050+0000 7f8ec6443640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec0112040 con 0x7f8ec0107260 2024-04-21T08:41:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.050+0000 7f8ec6443640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec0112340 con 0x7f8ec0102190 2024-04-21T08:41:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.050+0000 7f8ec49e0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec0074cd0 0x7f8ec01a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.050+0000 7f8ec49e0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec0074cd0 0x7f8ec01a68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34070/0 (socket says 172.21.15.43:34070) 2024-04-21T08:41:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.050+0000 7f8ec49e0640 1 -- 172.21.15.43:0/13921960 learned_addr learned my addr 172.21.15.43:0/13921960 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.050+0000 7f8ebf7fe640 1 --2- 172.21.15.43:0/13921960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec0107260 0x7f8ec0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.051+0000 7f8ec49e0640 1 -- 172.21.15.43:0/13921960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ec0102190 msgr2=0x7f8ec0074250 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:41:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.051+0000 7f8ec49e0640 1 --2- 172.21.15.43:0/13921960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ec0102190 0x7f8ec0074250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.051+0000 7f8ec49e0640 1 -- 172.21.15.43:0/13921960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec0107260 msgr2=0x7f8ec0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.051+0000 7f8ec49e0640 1 --2- 172.21.15.43:0/13921960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec0107260 0x7f8ec0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.051+0000 7f8ec49e0640 1 -- 172.21.15.43:0/13921960 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec01a6ff0 con 0x7f8ec0074cd0 2024-04-21T08:41:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.051+0000 7f8ebf7fe640 1 --2- 172.21.15.43:0/13921960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec0107260 0x7f8ec0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:42.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.051+0000 7f8ec49e0640 1 --2- 172.21.15.43:0/13921960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec0074cd0 0x7f8ec01a68f0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800cce0 tx=0x7f8eb800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:42.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.051+0000 7f8ebd7fa640 1 -- 172.21.15.43:0/13921960 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb800f450 con 0x7f8ec0074cd0 2024-04-21T08:41:42.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.051+0000 7f8ebd7fa640 1 -- 172.21.15.43:0/13921960 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eb800faa0 con 0x7f8ec0074cd0 2024-04-21T08:41:42.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.051+0000 7f8ec6443640 1 -- 172.21.15.43:0/13921960 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec01a72e0 con 0x7f8ec0074cd0 2024-04-21T08:41:42.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.051+0000 7f8ebd7fa640 1 -- 172.21.15.43:0/13921960 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8018b70 con 0x7f8ec0074cd0 2024-04-21T08:41:42.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.051+0000 7f8ec6443640 1 -- 172.21.15.43:0/13921960 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec01a77e0 con 0x7f8ec0074cd0 2024-04-21T08:41:42.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.053+0000 7f8ebd7fa640 1 -- 172.21.15.43:0/13921960 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8eb8018d10 con 0x7f8ec0074cd0 2024-04-21T08:41:42.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.053+0000 7f8ec9035740 1 Processor -- start 2024-04-21T08:41:42.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.053+0000 7f8ec9035740 1 -- start start 2024-04-21T08:41:42.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.054+0000 7f8ec9035740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55788ec9a4d0 0x55788ec9a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.054+0000 7f8ec9035740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788ec9ae80 0x55788eca3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.054+0000 7f8ec9035740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55788eca3a00 0x55788eca5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.054+0000 7f8ec9035740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55788ebff1f0 con 0x55788ec9ae80 2024-04-21T08:41:42.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.054+0000 7f8ec9035740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55788eb9c4b0 con 0x55788ec9a4d0 2024-04-21T08:41:42.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.054+0000 7f8ec9035740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55788ebfb1b0 con 0x55788eca3a00 2024-04-21T08:41:42.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.054+0000 7f8ebffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788ec9ae80 0x55788eca3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.054+0000 7f8ebffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788ec9ae80 0x55788eca3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34074/0 (socket says 172.21.15.43:34074) 2024-04-21T08:41:42.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.054+0000 7f8ebffff640 1 -- 172.21.15.43:0/1091538119 learned_addr learned my addr 172.21.15.43:0/1091538119 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:42.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.054+0000 7f8ebd7fa640 1 --2- 172.21.15.43:0/13921960 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8eac0a3d50 0x7f8eac0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.054+0000 7f8ebf7fe640 1 --2- 172.21.15.43:0/1091538119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55788ec9a4d0 0x55788ec9a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.054+0000 7f8ebffff640 1 -- 172.21.15.43:0/1091538119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55788eca3a00 msgr2=0x55788eca5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.054+0000 7f8ebffff640 1 --2- 172.21.15.43:0/1091538119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55788eca3a00 0x55788eca5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.054+0000 7f8ebffff640 1 -- 172.21.15.43:0/1091538119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55788ec9a4d0 msgr2=0x55788ec9a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.054+0000 7f8ebffff640 1 --2- 172.21.15.43:0/1091538119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55788ec9a4d0 0x55788ec9a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.054+0000 7f8ebffff640 1 -- 172.21.15.43:0/1091538119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55788eb8e1b0 con 0x55788ec9ae80 2024-04-21T08:41:42.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.054+0000 7f8ebf7fe640 1 --2- 172.21.15.43:0/1091538119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55788ec9a4d0 0x55788ec9a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:42.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.054+0000 7f8ec49e0640 1 --2- 172.21.15.43:0/13921960 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8eac0a3d50 0x7f8eac0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.054+0000 7f8ebd7fa640 1 -- 172.21.15.43:0/13921960 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f8eb80d1060 con 0x7f8ec0074cd0 2024-04-21T08:41:42.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.054+0000 7f8ebffff640 1 --2- 172.21.15.43:0/1091538119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788ec9ae80 0x55788eca3430 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8eb002c170 tx=0x7f8eb00095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb35a22def822eb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:42.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.054+0000 7f8eb67fc640 1 -- 172.21.15.43:0/1091538119 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0042020 con 0x55788ec9ae80 2024-04-21T08:41:42.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.054+0000 7f8ec49e0640 1 --2- 172.21.15.43:0/13921960 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8eac0a3d50 0x7f8eac0a6210 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8eb80ce9e0 tx=0x7f8eb8087760 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:42.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.055+0000 7f8eb67fc640 1 -- 172.21.15.43:0/1091538119 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eb002cc90 con 0x55788ec9ae80 2024-04-21T08:41:42.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.055+0000 7f8eb67fc640 1 -- 172.21.15.43:0/1091538119 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0003c30 con 0x55788ec9ae80 2024-04-21T08:41:42.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.055+0000 7f8ec9035740 1 -- 172.21.15.43:0/1091538119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788ec9ae80 msgr2=0x55788eca3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.055+0000 7f8ec9035740 1 --2- 172.21.15.43:0/1091538119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788ec9ae80 0x55788eca3430 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8eb002c170 tx=0x7f8eb00095a0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.055+0000 7f8ec9035740 1 -- 172.21.15.43:0/1091538119 shutdown_connections 2024-04-21T08:41:42.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.055+0000 7f8ec9035740 1 --2- 172.21.15.43:0/1091538119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55788eca3a00 0x55788eca5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.055+0000 7f8ec9035740 1 --2- 172.21.15.43:0/1091538119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788ec9ae80 0x55788eca3430 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.055+0000 7f8ec9035740 1 --2- 172.21.15.43:0/1091538119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55788ec9a4d0 0x55788ec9a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.055+0000 7f8ec9035740 1 -- 172.21.15.43:0/1091538119 >> 172.21.15.43:0/1091538119 conn(0x55788ec94ba0 msgr2=0x55788ec94f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:42.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.055+0000 7f8ec9035740 1 -- 172.21.15.43:0/1091538119 shutdown_connections 2024-04-21T08:41:42.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.055+0000 7f8ec9035740 1 -- 172.21.15.43:0/1091538119 wait complete. 2024-04-21T08:41:42.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.055+0000 7f8ec9035740 1 -- start start 2024-04-21T08:41:42.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.056+0000 7f8ec9035740 1 Processor -- start 2024-04-21T08:41:42.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.056+0000 7f8ec9035740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55788ec9a4d0 0x55788eca3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.056+0000 7f8ec9035740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788ec9ae80 0x55788eca06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.056+0000 7f8ec9035740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55788eca3a00 0x55788eca0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.056+0000 7f8ec9035740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55788eca6910 con 0x55788ec9ae80 2024-04-21T08:41:42.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.056+0000 7f8ec9035740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55788ebb8500 con 0x55788eca3a00 2024-04-21T08:41:42.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.056+0000 7f8ec9035740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55788eca6a90 con 0x55788ec9a4d0 2024-04-21T08:41:42.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.056+0000 7f8ec9035740 10 client.0 _ll_register_callbacks cb 0x55788eb86b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:42.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.056+0000 7f8ebffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788ec9ae80 0x55788eca06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.056+0000 7f8ebffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788ec9ae80 0x55788eca06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34082/0 (socket says 172.21.15.43:34082) 2024-04-21T08:41:42.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.056+0000 7f8ebffff640 1 -- 172.21.15.43:0/847827588 learned_addr learned my addr 172.21.15.43:0/847827588 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:42.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.056+0000 7f8ebf7fe640 1 --2- 172.21.15.43:0/847827588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55788eca3a00 0x55788eca0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.056+0000 7f8ebffff640 1 -- 172.21.15.43:0/847827588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55788ec9a4d0 msgr2=0x55788eca3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.056+0000 7f8ebffff640 1 --2- 172.21.15.43:0/847827588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55788ec9a4d0 0x55788eca3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.056+0000 7f8ebffff640 1 -- 172.21.15.43:0/847827588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55788eca3a00 msgr2=0x55788eca0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.056+0000 7f8ebffff640 1 --2- 172.21.15.43:0/847827588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55788eca3a00 0x55788eca0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.056+0000 7f8ebffff640 1 -- 172.21.15.43:0/847827588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55788e9113f0 con 0x55788ec9ae80 2024-04-21T08:41:42.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.056+0000 7f8ebf7fe640 1 --2- 172.21.15.43:0/847827588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55788eca3a00 0x55788eca0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:42.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.057+0000 7f8ebffff640 1 --2- 172.21.15.43:0/847827588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788ec9ae80 0x55788eca06e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0009900 tx=0x7f8eb0003870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a927e0fc4fffb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:42.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.057+0000 7f8e9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:42.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.057+0000 7f8e9f7fe640 1 -- 172.21.15.43:0/847827588 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0042030 con 0x55788ec9ae80 2024-04-21T08:41:42.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.057+0000 7f8e9f7fe640 1 -- 172.21.15.43:0/847827588 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f8eb0034a10 con 0x55788ec9ae80 2024-04-21T08:41:42.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.057+0000 7f8ec9035740 1 -- 172.21.15.43:0/847827588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8eb002cc90 con 0x55788ec9ae80 2024-04-21T08:41:42.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.057+0000 7f8e9f7fe640 1 -- 172.21.15.43:0/847827588 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8eb0034c10 con 0x55788ec9ae80 2024-04-21T08:41:42.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.057+0000 7f8ec9035740 10 client.16941 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:42.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.057+0000 7f8ec9035740 1 -- 172.21.15.43:0/847827588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55788e9db7d0 con 0x55788ec9ae80 2024-04-21T08:41:42.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.057+0000 7f8e9f7fe640 1 -- 172.21.15.43:0/847827588 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eb002cc90 con 0x55788ec9ae80 2024-04-21T08:41:42.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.057+0000 7f8e9f7fe640 1 -- 172.21.15.43:0/847827588 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8eb0047420 con 0x55788ec9ae80 2024-04-21T08:41:42.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.057+0000 7f8ec9035740 10 client.16941 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:42.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.057+0000 7f8ec9035740 10 client.16941 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:41:42.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.057+0000 7f8ec9035740 20 client.16941 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:42.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.057+0000 7f8ec9035740 1 --2- 172.21.15.43:0/847827588 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55788ecb39f0 0x55788ecb5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.057+0000 7f8ec9035740 4 client.16941 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:42.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.057+0000 7f8ec9035740 1 -- 172.21.15.43:0/847827588 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55788eb9e930 con 0x55788ecb39f0 2024-04-21T08:41:42.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.058+0000 7f8ebf7fe640 1 --2- 172.21.15.43:0/847827588 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55788ecb39f0 0x55788ecb5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.058+0000 7f8ebf7fe640 1 --2- 172.21.15.43:0/847827588 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55788ecb39f0 0x55788ecb5de0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d04eb9a17fc2b358 server_cookie=c7eb0d4c11c31986 in_seq=0 out_seq=0 2024-04-21T08:41:42.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.058+0000 7f8e9f7fe640 10 client.16941 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:41:42.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.059+0000 7f8e9f7fe640 1 -- 172.21.15.43:0/847827588 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55788eb9e930 con 0x55788ecb39f0 2024-04-21T08:41:42.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.059+0000 7f8e9f7fe640 10 client.16941 handle_command_reply: tid=0 2024-04-21T08:41:42.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.059+0000 7f8ec9035740 1 client.16941 shutdown 2024-04-21T08:41:42.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.059+0000 7f8ec9035740 1 -- 172.21.15.43:0/847827588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788ec9ae80 msgr2=0x55788eca06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.059+0000 7f8ec9035740 1 --2- 172.21.15.43:0/847827588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788ec9ae80 0x55788eca06e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0009900 tx=0x7f8eb0003870 comp rx=0 tx=0).stop 2024-04-21T08:41:42.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.059+0000 7f8ec9035740 1 -- 172.21.15.43:0/847827588 shutdown_connections 2024-04-21T08:41:42.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.059+0000 7f8ec9035740 1 --2- 172.21.15.43:0/847827588 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55788ecb39f0 0x55788ecb5de0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.059+0000 7f8ec9035740 1 --2- 172.21.15.43:0/847827588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55788eca3a00 0x55788eca0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.059+0000 7f8ec9035740 1 --2- 172.21.15.43:0/847827588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788ec9ae80 0x55788eca06e0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.059+0000 7f8ec9035740 1 --2- 172.21.15.43:0/847827588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55788ec9a4d0 0x55788eca3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.059+0000 7f8e9f7fe640 0 client.16941 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:41:42.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.059+0000 7f8ec9035740 1 -- 172.21.15.43:0/847827588 >> 172.21.15.43:0/847827588 conn(0x55788ec94ba0 msgr2=0x55788ec968f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:42.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.060+0000 7f8ec9035740 1 -- 172.21.15.43:0/847827588 shutdown_connections 2024-04-21T08:41:42.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.060+0000 7f8ec9035740 1 -- 172.21.15.43:0/847827588 wait complete. 2024-04-21T08:41:42.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.060+0000 7f8ec9035740 20 client.16941 trim_cache size 0 max 16384 2024-04-21T08:41:42.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.073+0000 7f8ec9035740 1 Processor -- start 2024-04-21T08:41:42.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.074+0000 7f8ec9035740 1 -- start start 2024-04-21T08:41:42.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.074+0000 7f8ec9035740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788ec9a4d0 0x55788ecaf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.074+0000 7f8ec9035740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55788ec9ae80 0x55788ec93a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.074+0000 7f8ec9035740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55788eca3a00 0x55788eca2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.074+0000 7f8ec9035740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec01b1580 con 0x55788ec9a4d0 2024-04-21T08:41:42.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.074+0000 7f8ec9035740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec01b30c0 con 0x55788ec9ae80 2024-04-21T08:41:42.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.074+0000 7f8ec9035740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec01b1c50 con 0x55788eca3a00 2024-04-21T08:41:42.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.074+0000 7f8ebf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788ec9a4d0 0x55788ecaf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.074+0000 7f8ebf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788ec9a4d0 0x55788ecaf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34090/0 (socket says 172.21.15.43:34090) 2024-04-21T08:41:42.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.074+0000 7f8ebf7fe640 1 -- 172.21.15.43:0/2139172863 learned_addr learned my addr 172.21.15.43:0/2139172863 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:42.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.074+0000 7f8ebffff640 1 --2- 172.21.15.43:0/2139172863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55788ec9ae80 0x55788ec93a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.074+0000 7f8ebf7fe640 1 -- 172.21.15.43:0/2139172863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55788eca3a00 msgr2=0x55788eca2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.074+0000 7f8ebf7fe640 1 --2- 172.21.15.43:0/2139172863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55788eca3a00 0x55788eca2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.074+0000 7f8ebf7fe640 1 -- 172.21.15.43:0/2139172863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55788ec9ae80 msgr2=0x55788ec93a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.074+0000 7f8ebf7fe640 1 --2- 172.21.15.43:0/2139172863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55788ec9ae80 0x55788ec93a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.074+0000 7f8ebf7fe640 1 -- 172.21.15.43:0/2139172863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55788eca1060 con 0x55788ec9a4d0 2024-04-21T08:41:42.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.074+0000 7f8ebf7fe640 1 --2- 172.21.15.43:0/2139172863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788ec9a4d0 0x55788ecaf7d0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800e010 tx=0x7f8ea8012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c336ce877a9c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:42.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.075+0000 7f8eb5ffb640 1 -- 172.21.15.43:0/2139172863 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea801d020 con 0x55788ec9a4d0 2024-04-21T08:41:42.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.075+0000 7f8eb5ffb640 1 -- 172.21.15.43:0/2139172863 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ea80196e0 con 0x55788ec9a4d0 2024-04-21T08:41:42.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.075+0000 7f8eb5ffb640 1 -- 172.21.15.43:0/2139172863 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8010aa0 con 0x55788ec9a4d0 2024-04-21T08:41:42.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.075+0000 7f8ec9035740 1 -- 172.21.15.43:0/2139172863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788ec9a4d0 msgr2=0x55788ecaf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.075+0000 7f8ec9035740 1 --2- 172.21.15.43:0/2139172863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788ec9a4d0 0x55788ecaf7d0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800e010 tx=0x7f8ea8012820 comp rx=0 tx=0).stop 2024-04-21T08:41:42.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.075+0000 7f8ec9035740 1 -- 172.21.15.43:0/2139172863 shutdown_connections 2024-04-21T08:41:42.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.075+0000 7f8ec9035740 1 --2- 172.21.15.43:0/2139172863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55788eca3a00 0x55788eca2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.075+0000 7f8ec9035740 1 --2- 172.21.15.43:0/2139172863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55788ec9ae80 0x55788ec93a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.075+0000 7f8ec9035740 1 --2- 172.21.15.43:0/2139172863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788ec9a4d0 0x55788ecaf7d0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.075+0000 7f8ec9035740 1 -- 172.21.15.43:0/2139172863 >> 172.21.15.43:0/2139172863 conn(0x55788ec94ba0 msgr2=0x55788ec93370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:42.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.075+0000 7f8ec9035740 1 -- 172.21.15.43:0/2139172863 shutdown_connections 2024-04-21T08:41:42.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.075+0000 7f8ec9035740 1 -- 172.21.15.43:0/2139172863 wait complete. 2024-04-21T08:41:42.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.076+0000 7f8ec9035740 1 -- start start 2024-04-21T08:41:42.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.076+0000 7f8ec9035740 1 Processor -- start 2024-04-21T08:41:42.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.076+0000 7f8ec9035740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55788ec9a4d0 0x55788ecabce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.076+0000 7f8ec9035740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55788ec9ae80 0x55788ecac220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.076+0000 7f8ec9035740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788eca3a00 0x55788ec9c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.076+0000 7f8ec9035740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec01b29f0 con 0x55788eca3a00 2024-04-21T08:41:42.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.076+0000 7f8ec9035740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec01b2320 con 0x55788ec9a4d0 2024-04-21T08:41:42.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.076+0000 7f8ec9035740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec01af560 con 0x55788ec9ae80 2024-04-21T08:41:42.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.076+0000 7f8ec9035740 10 client.0 _ll_register_callbacks cb 0x7f8eb002cbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:42.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.076+0000 7f8ebf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788eca3a00 0x55788ec9c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.076+0000 7f8ebf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788eca3a00 0x55788ec9c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34098/0 (socket says 172.21.15.43:34098) 2024-04-21T08:41:42.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.076+0000 7f8ebf7fe640 1 -- 172.21.15.43:0/2306420655 learned_addr learned my addr 172.21.15.43:0/2306420655 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:42.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.076+0000 7f8ebf7fe640 1 --2- 172.21.15.43:0/2306420655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55788ec9a4d0 0x55788ecabce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.077+0000 7f8ebffff640 1 --2- 172.21.15.43:0/2306420655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55788ec9ae80 0x55788ecac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.077+0000 7f8ebf7fe640 1 -- 172.21.15.43:0/2306420655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55788ec9ae80 msgr2=0x55788ecac220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.077+0000 7f8ebf7fe640 1 --2- 172.21.15.43:0/2306420655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55788ec9ae80 0x55788ecac220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.077+0000 7f8ebf7fe640 1 -- 172.21.15.43:0/2306420655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55788ec9a4d0 msgr2=0x55788ecabce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.077+0000 7f8ebf7fe640 1 --2- 172.21.15.43:0/2306420655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55788ec9a4d0 0x55788ecabce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.077+0000 7f8ebf7fe640 1 -- 172.21.15.43:0/2306420655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55788ec9d000 con 0x55788eca3a00 2024-04-21T08:41:42.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.077+0000 7f8ebf7fe640 1 --2- 172.21.15.43:0/2306420655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788eca3a00 0x55788ec9c990 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80106b0 tx=0x7f8ea800d7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c2207960bf5dd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:42.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.077+0000 7f8e9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:42.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.077+0000 7f8e9f7fe640 1 -- 172.21.15.43:0/2306420655 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea801d030 con 0x55788eca3a00 2024-04-21T08:41:42.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.077+0000 7f8e9f7fe640 1 -- 172.21.15.43:0/2306420655 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f8ea8012190 con 0x55788eca3a00 2024-04-21T08:41:42.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.077+0000 7f8ec9035740 1 -- 172.21.15.43:0/2306420655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ea80196e0 con 0x55788eca3a00 2024-04-21T08:41:42.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.077+0000 7f8e9f7fe640 1 -- 172.21.15.43:0/2306420655 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8ea8019230 con 0x55788eca3a00 2024-04-21T08:41:42.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.077+0000 7f8ec9035740 10 client.16947 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:42.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.077+0000 7f8ec9035740 1 -- 172.21.15.43:0/2306420655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55788ed19530 con 0x55788eca3a00 2024-04-21T08:41:42.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.077+0000 7f8e9f7fe640 1 -- 172.21.15.43:0/2306420655 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ea80196e0 con 0x55788eca3a00 2024-04-21T08:41:42.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.077+0000 7f8e9f7fe640 1 -- 172.21.15.43:0/2306420655 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8ea8032430 con 0x55788eca3a00 2024-04-21T08:41:42.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.077+0000 7f8ec9035740 10 client.16947 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:42.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.077+0000 7f8ec9035740 10 client.16947 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:41:42.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.077+0000 7f8ec9035740 20 client.16947 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:42.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.078+0000 7f8ec9035740 1 --2- 172.21.15.43:0/2306420655 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55788ed19ba0 0x55788ed1bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.078+0000 7f8ec9035740 4 client.16947 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:41:42.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.078+0000 7f8ec9035740 1 -- 172.21.15.43:0/2306420655 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55788ec0cc10 con 0x55788ed19ba0 2024-04-21T08:41:42.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.078+0000 7f8ebffff640 1 --2- 172.21.15.43:0/2306420655 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55788ed19ba0 0x55788ed1bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.078+0000 7f8ebffff640 1 --2- 172.21.15.43:0/2306420655 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55788ed19ba0 0x55788ed1bf90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=caf6f6500b7183ce server_cookie=1f26dfa9e5b106bf in_seq=0 out_seq=0 2024-04-21T08:41:42.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.078+0000 7f8e9f7fe640 10 client.16947 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:41:42.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.078+0000 7f8e9f7fe640 1 -- 172.21.15.43:0/2306420655 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55788ec0cc10 con 0x55788ed19ba0 2024-04-21T08:41:42.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.078+0000 7f8e9f7fe640 10 client.16947 handle_command_reply: tid=0 2024-04-21T08:41:42.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.078+0000 7f8ec9035740 1 client.16947 shutdown 2024-04-21T08:41:42.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.078+0000 7f8ec9035740 1 -- 172.21.15.43:0/2306420655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788eca3a00 msgr2=0x55788ec9c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.078+0000 7f8ec9035740 1 --2- 172.21.15.43:0/2306420655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788eca3a00 0x55788ec9c990 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80106b0 tx=0x7f8ea800d7d0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.078+0000 7f8ec9035740 1 -- 172.21.15.43:0/2306420655 shutdown_connections 2024-04-21T08:41:42.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.078+0000 7f8ec9035740 1 --2- 172.21.15.43:0/2306420655 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55788ed19ba0 0x55788ed1bf90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.079+0000 7f8ec9035740 1 --2- 172.21.15.43:0/2306420655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55788eca3a00 0x55788ec9c990 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.079+0000 7f8ec9035740 1 --2- 172.21.15.43:0/2306420655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55788ec9ae80 0x55788ecac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.079+0000 7f8e9f7fe640 0 client.16947 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:41:42.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.079+0000 7f8ec9035740 1 --2- 172.21.15.43:0/2306420655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55788ec9a4d0 0x55788ecabce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.079+0000 7f8ec9035740 1 -- 172.21.15.43:0/2306420655 >> 172.21.15.43:0/2306420655 conn(0x55788ec94ba0 msgr2=0x55788ec93c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:42.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.079+0000 7f8ec9035740 1 -- 172.21.15.43:0/2306420655 shutdown_connections 2024-04-21T08:41:42.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.079+0000 7f8ec9035740 1 -- 172.21.15.43:0/2306420655 wait complete. 2024-04-21T08:41:42.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.079+0000 7f8ec9035740 20 client.16947 trim_cache size 0 max 16384 2024-04-21T08:41:42.084 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:41:42.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.079+0000 7f8ec6443640 1 -- 172.21.15.43:0/13921960 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8eac0a3d50 msgr2=0x7f8eac0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.079+0000 7f8ec6443640 1 --2- 172.21.15.43:0/13921960 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8eac0a3d50 0x7f8eac0a6210 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8eb80ce9e0 tx=0x7f8eb8087760 comp rx=0 tx=0).stop 2024-04-21T08:41:42.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.079+0000 7f8ec6443640 1 -- 172.21.15.43:0/13921960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec0074cd0 msgr2=0x7f8ec01a68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.079+0000 7f8ec6443640 1 --2- 172.21.15.43:0/13921960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec0074cd0 0x7f8ec01a68f0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800cce0 tx=0x7f8eb800a620 comp rx=0 tx=0).stop 2024-04-21T08:41:42.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.079+0000 7f8ec6443640 1 -- 172.21.15.43:0/13921960 shutdown_connections 2024-04-21T08:41:42.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.079+0000 7f8ec6443640 1 --2- 172.21.15.43:0/13921960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ec0074cd0 0x7f8ec01a68f0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.079+0000 7f8ec6443640 1 --2- 172.21.15.43:0/13921960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec0107260 0x7f8ec0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.079+0000 7f8ec6443640 1 --2- 172.21.15.43:0/13921960 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8eac0a3d50 0x7f8eac0a6210 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.079+0000 7f8ec6443640 1 --2- 172.21.15.43:0/13921960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ec0102190 0x7f8ec0074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.079+0000 7f8ec6443640 1 -- 172.21.15.43:0/13921960 >> 172.21.15.43:0/13921960 conn(0x7f8ec00fd990 msgr2=0x7f8ec00fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:42.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.079+0000 7f8ec6443640 1 -- 172.21.15.43:0/13921960 shutdown_connections 2024-04-21T08:41:42.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.079+0000 7f8ec6443640 1 -- 172.21.15.43:0/13921960 wait complete. 2024-04-21T08:41:42.089 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:41:42.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.194+0000 7f7c41811640 1 -- 172.21.15.43:0/581241130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c3c101a40 msgr2=0x7f7c3c105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.194+0000 7f7c41811640 1 --2- 172.21.15.43:0/581241130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c3c101a40 0x7f7c3c105a80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009900 tx=0x7f7c3002bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:42.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.194+0000 7f7c41811640 1 -- 172.21.15.43:0/581241130 shutdown_connections 2024-04-21T08:41:42.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.194+0000 7f7c41811640 1 --2- 172.21.15.43:0/581241130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c3c106160 0x7f7c3c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.195+0000 7f7c41811640 1 --2- 172.21.15.43:0/581241130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c3c101a40 0x7f7c3c105a80 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.195+0000 7f7c41811640 1 --2- 172.21.15.43:0/581241130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c3c101090 0x7f7c3c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.195+0000 7f7c41811640 1 -- 172.21.15.43:0/581241130 >> 172.21.15.43:0/581241130 conn(0x7f7c3c077f50 msgr2=0x7f7c3c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:42.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.195+0000 7f7c41811640 1 -- 172.21.15.43:0/581241130 shutdown_connections 2024-04-21T08:41:42.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.195+0000 7f7c41811640 1 -- 172.21.15.43:0/581241130 wait complete. 2024-04-21T08:41:42.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.195+0000 7f7c41811640 1 Processor -- start 2024-04-21T08:41:42.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.195+0000 7f7c41811640 1 -- start start 2024-04-21T08:41:42.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.196+0000 7f7c41811640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c3c101090 0x7f7c3c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.196+0000 7f7c41811640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c3c106160 0x7f7c3c198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.196+0000 7f7c41811640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c3c19cfc0 0x7f7c3c19d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.196+0000 7f7c41811640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c1121e0 con 0x7f7c3c106160 2024-04-21T08:41:42.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.196+0000 7f7c41811640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c112060 con 0x7f7c3c19cfc0 2024-04-21T08:41:42.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.196+0000 7f7c41811640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c112360 con 0x7f7c3c101090 2024-04-21T08:41:42.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.196+0000 7f7c3a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c3c106160 0x7f7c3c198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.196+0000 7f7c3a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c3c106160 0x7f7c3c198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34120/0 (socket says 172.21.15.43:34120) 2024-04-21T08:41:42.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.196+0000 7f7c3a7fc640 1 -- 172.21.15.43:0/1038922547 learned_addr learned my addr 172.21.15.43:0/1038922547 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:42.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.196+0000 7f7c3b7fe640 1 --2- 172.21.15.43:0/1038922547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c3c19cfc0 0x7f7c3c19d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.196+0000 7f7c3a7fc640 1 -- 172.21.15.43:0/1038922547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c3c101090 msgr2=0x7f7c3c198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.196+0000 7f7c3a7fc640 1 --2- 172.21.15.43:0/1038922547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c3c101090 0x7f7c3c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.196+0000 7f7c3a7fc640 1 -- 172.21.15.43:0/1038922547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c3c19cfc0 msgr2=0x7f7c3c19d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.196+0000 7f7c3a7fc640 1 --2- 172.21.15.43:0/1038922547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c3c19cfc0 0x7f7c3c19d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.196+0000 7f7c3a7fc640 1 -- 172.21.15.43:0/1038922547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3c19dab0 con 0x7f7c3c106160 2024-04-21T08:41:42.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.196+0000 7f7c3a7fc640 1 --2- 172.21.15.43:0/1038922547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c3c106160 0x7f7c3c198c40 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f7c30036040 tx=0x7f7c3002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:42.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.196+0000 7f7c4080f640 1 -- 172.21.15.43:0/1038922547 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30043020 con 0x7f7c3c106160 2024-04-21T08:41:42.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.196+0000 7f7c4080f640 1 -- 172.21.15.43:0/1038922547 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c3002cbe0 con 0x7f7c3c106160 2024-04-21T08:41:42.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.196+0000 7f7c41811640 1 -- 172.21.15.43:0/1038922547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c3c19dd40 con 0x7f7c3c106160 2024-04-21T08:41:42.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.196+0000 7f7c4080f640 1 -- 172.21.15.43:0/1038922547 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30003bf0 con 0x7f7c3c106160 2024-04-21T08:41:42.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.196+0000 7f7c41811640 1 -- 172.21.15.43:0/1038922547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c3c1a5560 con 0x7f7c3c106160 2024-04-21T08:41:42.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.199+0000 7f7c4080f640 1 -- 172.21.15.43:0/1038922547 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7c30046020 con 0x7f7c3c106160 2024-04-21T08:41:42.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.199+0000 7f7c44404740 1 Processor -- start 2024-04-21T08:41:42.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.199+0000 7f7c44404740 1 -- start start 2024-04-21T08:41:42.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.199+0000 7f7c44404740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636608024d0 0x5636608028b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.199+0000 7f7c44404740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563660802e80 0x56366080b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.199+0000 7f7c44404740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56366080ba00 0x56366080de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.199+0000 7f7c44404740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5636607671f0 con 0x56366080ba00 2024-04-21T08:41:42.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.199+0000 7f7c44404740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5636607044b0 con 0x5636608024d0 2024-04-21T08:41:42.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.199+0000 7f7c44404740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5636607631b0 con 0x563660802e80 2024-04-21T08:41:42.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.199+0000 7f7c4080f640 1 --2- 172.21.15.43:0/1038922547 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7c240a3d20 0x7f7c240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.199+0000 7f7c3a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56366080ba00 0x56366080de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.199+0000 7f7c3a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56366080ba00 0x56366080de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34132/0 (socket says 172.21.15.43:34132) 2024-04-21T08:41:42.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.199+0000 7f7c3a7fc640 1 -- 172.21.15.43:0/3192044469 learned_addr learned my addr 172.21.15.43:0/3192044469 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:42.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.200+0000 7f7c3b7fe640 1 --2- 172.21.15.43:0/1038922547 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7c240a3d20 0x7f7c240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.200+0000 7f7c3b7fe640 1 --2- 172.21.15.43:0/3192044469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636608024d0 0x5636608028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.200+0000 7f7c4080f640 1 -- 172.21.15.43:0/1038922547 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f7c300f2dc0 con 0x7f7c3c106160 2024-04-21T08:41:42.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.200+0000 7f7c3affd640 1 --2- 172.21.15.43:0/3192044469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563660802e80 0x56366080b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.200+0000 7f7c3a7fc640 1 -- 172.21.15.43:0/3192044469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563660802e80 msgr2=0x56366080b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.200+0000 7f7c3a7fc640 1 --2- 172.21.15.43:0/3192044469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563660802e80 0x56366080b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.200+0000 7f7c3a7fc640 1 -- 172.21.15.43:0/3192044469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636608024d0 msgr2=0x5636608028b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.200+0000 7f7c3a7fc640 1 --2- 172.21.15.43:0/3192044469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636608024d0 0x5636608028b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.200+0000 7f7c3a7fc640 1 -- 172.21.15.43:0/3192044469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636606f61b0 con 0x56366080ba00 2024-04-21T08:41:42.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.200+0000 7f7c3b7fe640 1 --2- 172.21.15.43:0/1038922547 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7c240a3d20 0x7f7c240a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7c34007fd0 tx=0x7f7c34007c60 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:42.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.200+0000 7f7c3a7fc640 1 --2- 172.21.15.43:0/3192044469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56366080ba00 0x56366080de50 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f7c300bb470 tx=0x7f7c3002cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0d92cc4e0d1dd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:42.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.200+0000 7f7c21ffb640 1 -- 172.21.15.43:0/3192044469 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c300beaa0 con 0x56366080ba00 2024-04-21T08:41:42.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.200+0000 7f7c21ffb640 1 -- 172.21.15.43:0/3192044469 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c300c2e50 con 0x56366080ba00 2024-04-21T08:41:42.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.200+0000 7f7c21ffb640 1 -- 172.21.15.43:0/3192044469 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c300f03c0 con 0x56366080ba00 2024-04-21T08:41:42.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.201+0000 7f7c44404740 1 -- 172.21.15.43:0/3192044469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56366080ba00 msgr2=0x56366080de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.201+0000 7f7c44404740 1 --2- 172.21.15.43:0/3192044469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56366080ba00 0x56366080de50 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f7c300bb470 tx=0x7f7c3002cd90 comp rx=0 tx=0).stop 2024-04-21T08:41:42.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.201+0000 7f7c44404740 1 -- 172.21.15.43:0/3192044469 shutdown_connections 2024-04-21T08:41:42.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.201+0000 7f7c44404740 1 --2- 172.21.15.43:0/3192044469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56366080ba00 0x56366080de50 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.201+0000 7f7c44404740 1 --2- 172.21.15.43:0/3192044469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563660802e80 0x56366080b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.201+0000 7f7c44404740 1 --2- 172.21.15.43:0/3192044469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636608024d0 0x5636608028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.201+0000 7f7c44404740 1 -- 172.21.15.43:0/3192044469 >> 172.21.15.43:0/3192044469 conn(0x5636607fcba0 msgr2=0x5636607fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:42.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.201+0000 7f7c44404740 1 -- 172.21.15.43:0/3192044469 shutdown_connections 2024-04-21T08:41:42.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.201+0000 7f7c44404740 1 -- 172.21.15.43:0/3192044469 wait complete. 2024-04-21T08:41:42.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.201+0000 7f7c44404740 1 -- start start 2024-04-21T08:41:42.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.202+0000 7f7c44404740 1 Processor -- start 2024-04-21T08:41:42.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.202+0000 7f7c44404740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636608024d0 0x56366080b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.202+0000 7f7c44404740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563660802e80 0x563660808690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.202+0000 7f7c44404740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56366080ba00 0x563660808c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.202+0000 7f7c44404740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56366080e730 con 0x5636608024d0 2024-04-21T08:41:42.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.202+0000 7f7c44404740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563660720500 con 0x563660802e80 2024-04-21T08:41:42.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.202+0000 7f7c44404740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56366080e8b0 con 0x56366080ba00 2024-04-21T08:41:42.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.202+0000 7f7c44404740 10 client.0 _ll_register_callbacks cb 0x5636606eeb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:42.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.202+0000 7f7c3affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636608024d0 0x56366080b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.202+0000 7f7c3affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636608024d0 0x56366080b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34136/0 (socket says 172.21.15.43:34136) 2024-04-21T08:41:42.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.202+0000 7f7c3affd640 1 -- 172.21.15.43:0/1940392211 learned_addr learned my addr 172.21.15.43:0/1940392211 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:42.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.202+0000 7f7c3b7fe640 1 --2- 172.21.15.43:0/1940392211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56366080ba00 0x563660808c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.202+0000 7f7c3a7fc640 1 --2- 172.21.15.43:0/1940392211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563660802e80 0x563660808690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.202+0000 7f7c3affd640 1 -- 172.21.15.43:0/1940392211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56366080ba00 msgr2=0x563660808c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.202+0000 7f7c3affd640 1 --2- 172.21.15.43:0/1940392211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56366080ba00 0x563660808c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.202+0000 7f7c3affd640 1 -- 172.21.15.43:0/1940392211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563660802e80 msgr2=0x563660808690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.202+0000 7f7c3affd640 1 --2- 172.21.15.43:0/1940392211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563660802e80 0x563660808690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.202+0000 7f7c3affd640 1 -- 172.21.15.43:0/1940392211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636604793f0 con 0x5636608024d0 2024-04-21T08:41:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.202+0000 7f7c3b7fe640 1 --2- 172.21.15.43:0/1940392211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56366080ba00 0x563660808c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.202+0000 7f7c3a7fc640 1 --2- 172.21.15.43:0/1940392211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563660802e80 0x563660808690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.202+0000 7f7c3affd640 1 --2- 172.21.15.43:0/1940392211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636608024d0 0x56366080b300 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00dd80 tx=0x7f7c2c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55651677e9b944e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.202+0000 7f7c1affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.203+0000 7f7c1affd640 1 -- 172.21.15.43:0/1940392211 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c015070 con 0x5636608024d0 2024-04-21T08:41:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.203+0000 7f7c1affd640 1 -- 172.21.15.43:0/1940392211 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f7c2c005020 con 0x5636608024d0 2024-04-21T08:41:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.203+0000 7f7c44404740 1 -- 172.21.15.43:0/1940392211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c300c2e50 con 0x5636608024d0 2024-04-21T08:41:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.203+0000 7f7c1affd640 1 -- 172.21.15.43:0/1940392211 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7c2c022050 con 0x5636608024d0 2024-04-21T08:41:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.203+0000 7f7c44404740 10 client.16959 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.203+0000 7f7c44404740 1 -- 172.21.15.43:0/1940392211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5636605437d0 con 0x5636608024d0 2024-04-21T08:41:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.203+0000 7f7c1affd640 1 -- 172.21.15.43:0/1940392211 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c300c2e50 con 0x5636608024d0 2024-04-21T08:41:42.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.203+0000 7f7c1affd640 1 -- 172.21.15.43:0/1940392211 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7c2c011650 con 0x5636608024d0 2024-04-21T08:41:42.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.203+0000 7f7c44404740 10 client.16959 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:42.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.203+0000 7f7c44404740 10 client.16959 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:41:42.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.203+0000 7f7c44404740 20 client.16959 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:42.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.203+0000 7f7c44404740 1 --2- 172.21.15.43:0/1940392211 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x56366081b990 0x56366081dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.203+0000 7f7c44404740 4 client.16959 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:42.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.203+0000 7f7c44404740 1 -- 172.21.15.43:0/1940392211 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563660706930 con 0x56366081b990 2024-04-21T08:41:42.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.204+0000 7f7c3a7fc640 1 --2- 172.21.15.43:0/1940392211 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x56366081b990 0x56366081dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.204+0000 7f7c3a7fc640 1 --2- 172.21.15.43:0/1940392211 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x56366081b990 0x56366081dde0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=626358cbc7297a7d server_cookie=44f4ed0944196cee in_seq=0 out_seq=0 2024-04-21T08:41:42.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.204+0000 7f7c1affd640 10 client.16959 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:41:42.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.205+0000 7f7c1affd640 1 -- 172.21.15.43:0/1940392211 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563660706930 con 0x56366081b990 2024-04-21T08:41:42.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.205+0000 7f7c1affd640 10 client.16959 handle_command_reply: tid=0 2024-04-21T08:41:42.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.205+0000 7f7c44404740 1 client.16959 shutdown 2024-04-21T08:41:42.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.205+0000 7f7c44404740 1 -- 172.21.15.43:0/1940392211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636608024d0 msgr2=0x56366080b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.205+0000 7f7c44404740 1 --2- 172.21.15.43:0/1940392211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636608024d0 0x56366080b300 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00dd80 tx=0x7f7c2c00f730 comp rx=0 tx=0).stop 2024-04-21T08:41:42.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.205+0000 7f7c44404740 1 -- 172.21.15.43:0/1940392211 shutdown_connections 2024-04-21T08:41:42.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.205+0000 7f7c44404740 1 --2- 172.21.15.43:0/1940392211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56366080ba00 0x563660808c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.205+0000 7f7c44404740 1 --2- 172.21.15.43:0/1940392211 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x56366081b990 0x56366081dde0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.205+0000 7f7c44404740 1 --2- 172.21.15.43:0/1940392211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563660802e80 0x563660808690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.205+0000 7f7c44404740 1 --2- 172.21.15.43:0/1940392211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636608024d0 0x56366080b300 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.205+0000 7f7c1affd640 0 client.16959 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:41:42.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.205+0000 7f7c44404740 1 -- 172.21.15.43:0/1940392211 >> 172.21.15.43:0/1940392211 conn(0x5636607fcba0 msgr2=0x5636607fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:42.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.205+0000 7f7c44404740 1 -- 172.21.15.43:0/1940392211 shutdown_connections 2024-04-21T08:41:42.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.205+0000 7f7c44404740 1 -- 172.21.15.43:0/1940392211 wait complete. 2024-04-21T08:41:42.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.205+0000 7f7c44404740 20 client.16959 trim_cache size 0 max 16384 2024-04-21T08:41:42.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.219+0000 7f7c44404740 1 Processor -- start 2024-04-21T08:41:42.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.219+0000 7f7c44404740 1 -- start start 2024-04-21T08:41:42.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.219+0000 7f7c44404740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636608024d0 0x563660817530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.219+0000 7f7c44404740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563660802e80 0x5636607fba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.219+0000 7f7c44404740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56366080ba00 0x56366080adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.219+0000 7f7c44404740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c1af300 con 0x56366080ba00 2024-04-21T08:41:42.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.219+0000 7f7c44404740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c1b0e40 con 0x5636608024d0 2024-04-21T08:41:42.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.219+0000 7f7c44404740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c1af9d0 con 0x563660802e80 2024-04-21T08:41:42.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.219+0000 7f7c3b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56366080ba00 0x56366080adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.219+0000 7f7c3b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56366080ba00 0x56366080adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34142/0 (socket says 172.21.15.43:34142) 2024-04-21T08:41:42.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.219+0000 7f7c3b7fe640 1 -- 172.21.15.43:0/1000146171 learned_addr learned my addr 172.21.15.43:0/1000146171 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:42.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.219+0000 7f7c3a7fc640 1 --2- 172.21.15.43:0/1000146171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563660802e80 0x5636607fba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.219+0000 7f7c3affd640 1 --2- 172.21.15.43:0/1000146171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636608024d0 0x563660817530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.219+0000 7f7c3b7fe640 1 -- 172.21.15.43:0/1000146171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563660802e80 msgr2=0x5636607fba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.219+0000 7f7c3b7fe640 1 --2- 172.21.15.43:0/1000146171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563660802e80 0x5636607fba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.219+0000 7f7c3b7fe640 1 -- 172.21.15.43:0/1000146171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636608024d0 msgr2=0x563660817530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.219+0000 7f7c3b7fe640 1 --2- 172.21.15.43:0/1000146171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636608024d0 0x563660817530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.219+0000 7f7c3b7fe640 1 -- 172.21.15.43:0/1000146171 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56366080e160 con 0x56366080ba00 2024-04-21T08:41:42.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.219+0000 7f7c3affd640 1 --2- 172.21.15.43:0/1000146171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636608024d0 0x563660817530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:42.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.220+0000 7f7c3b7fe640 1 --2- 172.21.15.43:0/1000146171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56366080ba00 0x56366080adc0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7c34013270 tx=0x7f7c34004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea24bdf67bd7cf10 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:42.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.220+0000 7f7c217fa640 1 -- 172.21.15.43:0/1000146171 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34025070 con 0x56366080ba00 2024-04-21T08:41:42.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.220+0000 7f7c217fa640 1 -- 172.21.15.43:0/1000146171 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c34011bc0 con 0x56366080ba00 2024-04-21T08:41:42.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.220+0000 7f7c217fa640 1 -- 172.21.15.43:0/1000146171 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34021440 con 0x56366080ba00 2024-04-21T08:41:42.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.220+0000 7f7c44404740 1 -- 172.21.15.43:0/1000146171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56366080ba00 msgr2=0x56366080adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.220+0000 7f7c44404740 1 --2- 172.21.15.43:0/1000146171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56366080ba00 0x56366080adc0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7c34013270 tx=0x7f7c34004ab0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.220+0000 7f7c44404740 1 -- 172.21.15.43:0/1000146171 shutdown_connections 2024-04-21T08:41:42.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.220+0000 7f7c44404740 1 --2- 172.21.15.43:0/1000146171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56366080ba00 0x56366080adc0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.220+0000 7f7c44404740 1 --2- 172.21.15.43:0/1000146171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563660802e80 0x5636607fba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.220+0000 7f7c44404740 1 --2- 172.21.15.43:0/1000146171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636608024d0 0x563660817530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.220+0000 7f7c44404740 1 -- 172.21.15.43:0/1000146171 >> 172.21.15.43:0/1000146171 conn(0x5636607fcba0 msgr2=0x5636607fb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:42.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.220+0000 7f7c44404740 1 -- 172.21.15.43:0/1000146171 shutdown_connections 2024-04-21T08:41:42.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.220+0000 7f7c44404740 1 -- 172.21.15.43:0/1000146171 wait complete. 2024-04-21T08:41:42.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.221+0000 7f7c44404740 1 -- start start 2024-04-21T08:41:42.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.221+0000 7f7c44404740 1 Processor -- start 2024-04-21T08:41:42.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.221+0000 7f7c44404740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636608024d0 0x563660813b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.221+0000 7f7c44404740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563660802e80 0x56366080f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.221+0000 7f7c44404740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56366080ba00 0x56366080f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.221+0000 7f7c44404740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c1b0770 con 0x5636608024d0 2024-04-21T08:41:42.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.221+0000 7f7c44404740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c1b00a0 con 0x563660802e80 2024-04-21T08:41:42.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.221+0000 7f7c44404740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c1ad2e0 con 0x56366080ba00 2024-04-21T08:41:42.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.221+0000 7f7c44404740 10 client.0 _ll_register_callbacks cb 0x7f7c2c00d910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:42.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.221+0000 7f7c3affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636608024d0 0x563660813b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.221+0000 7f7c3affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636608024d0 0x563660813b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34156/0 (socket says 172.21.15.43:34156) 2024-04-21T08:41:42.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.221+0000 7f7c3affd640 1 -- 172.21.15.43:0/3149273011 learned_addr learned my addr 172.21.15.43:0/3149273011 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:42.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.221+0000 7f7c3a7fc640 1 --2- 172.21.15.43:0/3149273011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563660802e80 0x56366080f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.222+0000 7f7c3affd640 1 -- 172.21.15.43:0/3149273011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56366080ba00 msgr2=0x56366080f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.222+0000 7f7c3b7fe640 1 --2- 172.21.15.43:0/3149273011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56366080ba00 0x56366080f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.222+0000 7f7c3affd640 1 --2- 172.21.15.43:0/3149273011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56366080ba00 0x56366080f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.222+0000 7f7c3affd640 1 -- 172.21.15.43:0/3149273011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563660802e80 msgr2=0x56366080f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.222+0000 7f7c3affd640 1 --2- 172.21.15.43:0/3149273011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563660802e80 0x56366080f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.222+0000 7f7c3affd640 1 -- 172.21.15.43:0/3149273011 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56366080fa80 con 0x5636608024d0 2024-04-21T08:41:42.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.222+0000 7f7c3a7fc640 1 --2- 172.21.15.43:0/3149273011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563660802e80 0x56366080f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:42.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.222+0000 7f7c3affd640 1 --2- 172.21.15.43:0/3149273011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636608024d0 0x563660813b50 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c001eb0 tx=0x7f7c2c011880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dac22ebe9a19ec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:42.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.222+0000 7f7c1affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:42.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.222+0000 7f7c1affd640 1 -- 172.21.15.43:0/3149273011 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c01f040 con 0x5636608024d0 2024-04-21T08:41:42.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.222+0000 7f7c1affd640 1 -- 172.21.15.43:0/3149273011 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f7c2c004250 con 0x5636608024d0 2024-04-21T08:41:42.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.222+0000 7f7c44404740 1 -- 172.21.15.43:0/3149273011 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c34011bc0 con 0x5636608024d0 2024-04-21T08:41:42.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.222+0000 7f7c1affd640 1 -- 172.21.15.43:0/3149273011 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7c2c015040 con 0x5636608024d0 2024-04-21T08:41:42.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.222+0000 7f7c44404740 10 client.16965 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:42.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.222+0000 7f7c44404740 1 -- 172.21.15.43:0/3149273011 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563660881c40 con 0x5636608024d0 2024-04-21T08:41:42.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.222+0000 7f7c1affd640 1 -- 172.21.15.43:0/3149273011 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c34011bc0 con 0x5636608024d0 2024-04-21T08:41:42.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.222+0000 7f7c1affd640 1 -- 172.21.15.43:0/3149273011 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7c2c0194f0 con 0x5636608024d0 2024-04-21T08:41:42.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.222+0000 7f7c44404740 10 client.16965 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:42.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.222+0000 7f7c44404740 10 client.16965 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:41:42.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.222+0000 7f7c44404740 20 client.16965 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:42.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.222+0000 7f7c44404740 1 --2- 172.21.15.43:0/3149273011 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5636608823f0 0x563660884810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.222+0000 7f7c44404740 4 client.16965 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:41:42.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.222+0000 7f7c44404740 1 -- 172.21.15.43:0/3149273011 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563660774c10 con 0x5636608823f0 2024-04-21T08:41:42.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.223+0000 7f7c3a7fc640 1 --2- 172.21.15.43:0/3149273011 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5636608823f0 0x563660884810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.223+0000 7f7c3a7fc640 1 --2- 172.21.15.43:0/3149273011 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5636608823f0 0x563660884810 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b7a601d308f4f47 server_cookie=9d7f763dc05faa8 in_seq=0 out_seq=0 2024-04-21T08:41:42.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.223+0000 7f7c1affd640 10 client.16965 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:41:42.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c1affd640 1 -- 172.21.15.43:0/3149273011 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563660774c10 con 0x5636608823f0 2024-04-21T08:41:42.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c1affd640 10 client.16965 handle_command_reply: tid=0 2024-04-21T08:41:42.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c44404740 1 client.16965 shutdown 2024-04-21T08:41:42.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c44404740 1 -- 172.21.15.43:0/3149273011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636608024d0 msgr2=0x563660813b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c44404740 1 --2- 172.21.15.43:0/3149273011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636608024d0 0x563660813b50 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c001eb0 tx=0x7f7c2c011880 comp rx=0 tx=0).stop 2024-04-21T08:41:42.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c44404740 1 -- 172.21.15.43:0/3149273011 shutdown_connections 2024-04-21T08:41:42.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c44404740 1 --2- 172.21.15.43:0/3149273011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56366080ba00 0x56366080f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c44404740 1 --2- 172.21.15.43:0/3149273011 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5636608823f0 0x563660884810 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c44404740 1 --2- 172.21.15.43:0/3149273011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563660802e80 0x56366080f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c1affd640 0 client.16965 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:41:42.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c44404740 1 --2- 172.21.15.43:0/3149273011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5636608024d0 0x563660813b50 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c44404740 1 -- 172.21.15.43:0/3149273011 >> 172.21.15.43:0/3149273011 conn(0x5636607fcba0 msgr2=0x5636607fbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:42.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c44404740 1 -- 172.21.15.43:0/3149273011 shutdown_connections 2024-04-21T08:41:42.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c44404740 1 -- 172.21.15.43:0/3149273011 wait complete. 2024-04-21T08:41:42.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c44404740 20 client.16965 trim_cache size 0 max 16384 2024-04-21T08:41:42.229 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:41:42.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c41811640 1 -- 172.21.15.43:0/1038922547 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7c240a3d20 msgr2=0x7f7c240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c41811640 1 --2- 172.21.15.43:0/1038922547 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7c240a3d20 0x7f7c240a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7c34007fd0 tx=0x7f7c34007c60 comp rx=0 tx=0).stop 2024-04-21T08:41:42.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c41811640 1 -- 172.21.15.43:0/1038922547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c3c106160 msgr2=0x7f7c3c198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c41811640 1 --2- 172.21.15.43:0/1038922547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c3c106160 0x7f7c3c198c40 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f7c30036040 tx=0x7f7c3002c7e0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c41811640 1 -- 172.21.15.43:0/1038922547 shutdown_connections 2024-04-21T08:41:42.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c41811640 1 --2- 172.21.15.43:0/1038922547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c3c19cfc0 0x7f7c3c19d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c41811640 1 --2- 172.21.15.43:0/1038922547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c3c106160 0x7f7c3c198c40 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c41811640 1 --2- 172.21.15.43:0/1038922547 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7c240a3d20 0x7f7c240a61e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c41811640 1 --2- 172.21.15.43:0/1038922547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c3c101090 0x7f7c3c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.224+0000 7f7c41811640 1 -- 172.21.15.43:0/1038922547 >> 172.21.15.43:0/1038922547 conn(0x7f7c3c077f50 msgr2=0x7f7c3c0fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:42.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.225+0000 7f7c41811640 1 -- 172.21.15.43:0/1038922547 shutdown_connections 2024-04-21T08:41:42.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.225+0000 7f7c41811640 1 -- 172.21.15.43:0/1038922547 wait complete. 2024-04-21T08:41:42.234 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:41:42.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.340+0000 7f2dddde6640 1 -- 172.21.15.43:0/3774496243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2dd806a5d0 msgr2=0x7f2dd806a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.340+0000 7f2dddde6640 1 --2- 172.21.15.43:0/3774496243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2dd806a5d0 0x7f2dd806a9b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8009900 tx=0x7f2dc802bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:42.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.340+0000 7f2dddde6640 1 -- 172.21.15.43:0/3774496243 shutdown_connections 2024-04-21T08:41:42.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.340+0000 7f2dddde6640 1 --2- 172.21.15.43:0/3774496243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dd810e960 0x7f2dd8110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.340+0000 7f2dddde6640 1 --2- 172.21.15.43:0/3774496243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dd806aef0 0x7f2dd810e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.340+0000 7f2dddde6640 1 --2- 172.21.15.43:0/3774496243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2dd806a5d0 0x7f2dd806a9b0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.340+0000 7f2dddde6640 1 -- 172.21.15.43:0/3774496243 >> 172.21.15.43:0/3774496243 conn(0x7f2dd8077eb0 msgr2=0x7f2dd80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:42.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.340+0000 7f2dddde6640 1 -- 172.21.15.43:0/3774496243 shutdown_connections 2024-04-21T08:41:42.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.340+0000 7f2dddde6640 1 -- 172.21.15.43:0/3774496243 wait complete. 2024-04-21T08:41:42.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.341+0000 7f2dddde6640 1 Processor -- start 2024-04-21T08:41:42.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.341+0000 7f2dddde6640 1 -- start start 2024-04-21T08:41:42.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.341+0000 7f2dddde6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2dd806a5d0 0x7f2dd81a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.341+0000 7f2dddde6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dd806aef0 0x7f2dd81a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.341+0000 7f2dddde6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dd810e960 0x7f2dd819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.341+0000 7f2dddde6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd81143e0 con 0x7f2dd806a5d0 2024-04-21T08:41:42.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.341+0000 7f2dddde6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd8114260 con 0x7f2dd806aef0 2024-04-21T08:41:42.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.341+0000 7f2dddde6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd8114560 con 0x7f2dd810e960 2024-04-21T08:41:42.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.341+0000 7f2dd77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2dd806a5d0 0x7f2dd81a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.341+0000 7f2dd77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2dd806a5d0 0x7f2dd81a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34174/0 (socket says 172.21.15.43:34174) 2024-04-21T08:41:42.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.341+0000 7f2dd77fe640 1 -- 172.21.15.43:0/2657474342 learned_addr learned my addr 172.21.15.43:0/2657474342 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:42.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.341+0000 7f2dd6ffd640 1 --2- 172.21.15.43:0/2657474342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dd806aef0 0x7f2dd81a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.341+0000 7f2dd77fe640 1 -- 172.21.15.43:0/2657474342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dd810e960 msgr2=0x7f2dd819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.341+0000 7f2dd7fff640 1 --2- 172.21.15.43:0/2657474342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dd810e960 0x7f2dd819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.341+0000 7f2dd77fe640 1 --2- 172.21.15.43:0/2657474342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dd810e960 0x7f2dd819a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.342+0000 7f2dd77fe640 1 -- 172.21.15.43:0/2657474342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dd806aef0 msgr2=0x7f2dd81a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.342+0000 7f2dd77fe640 1 --2- 172.21.15.43:0/2657474342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dd806aef0 0x7f2dd81a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.342+0000 7f2dd77fe640 1 -- 172.21.15.43:0/2657474342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd819af50 con 0x7f2dd806a5d0 2024-04-21T08:41:42.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.342+0000 7f2dd77fe640 1 --2- 172.21.15.43:0/2657474342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2dd806a5d0 0x7f2dd81a0610 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8005940 tx=0x7f2dc80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:42.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.342+0000 7f2dd4ff9640 1 -- 172.21.15.43:0/2657474342 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8043020 con 0x7f2dd806a5d0 2024-04-21T08:41:42.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.342+0000 7f2dddde6640 1 -- 172.21.15.43:0/2657474342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd819b1e0 con 0x7f2dd806a5d0 2024-04-21T08:41:42.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.342+0000 7f2dd4ff9640 1 -- 172.21.15.43:0/2657474342 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dc802caf0 con 0x7f2dd806a5d0 2024-04-21T08:41:42.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.342+0000 7f2dd4ff9640 1 -- 172.21.15.43:0/2657474342 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8003af0 con 0x7f2dd806a5d0 2024-04-21T08:41:42.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.342+0000 7f2dddde6640 1 -- 172.21.15.43:0/2657474342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd81a7460 con 0x7f2dd806a5d0 2024-04-21T08:41:42.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.344+0000 7f2dd4ff9640 1 -- 172.21.15.43:0/2657474342 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2dc8046020 con 0x7f2dd806a5d0 2024-04-21T08:41:42.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.345+0000 7f2de09d2740 1 Processor -- start 2024-04-21T08:41:42.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.345+0000 7f2de09d2740 1 -- start start 2024-04-21T08:41:42.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.345+0000 7f2de09d2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581ac3dc4d0 0x5581ac3dc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.345+0000 7f2de09d2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 0x5581ac3e5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.345+0000 7f2de09d2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581ac3e5a00 0x5581ac3e7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.345+0000 7f2de09d2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5581ac3411f0 con 0x5581ac3dce80 2024-04-21T08:41:42.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.345+0000 7f2de09d2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5581ac2de4b0 con 0x5581ac3dc4d0 2024-04-21T08:41:42.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.345+0000 7f2de09d2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5581ac33d1b0 con 0x5581ac3e5a00 2024-04-21T08:41:42.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.345+0000 7f2dd77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 0x5581ac3e5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.345+0000 7f2dd7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581ac3dc4d0 0x5581ac3dc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.345+0000 7f2dd77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 0x5581ac3e5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34190/0 (socket says 172.21.15.43:34190) 2024-04-21T08:41:42.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.345+0000 7f2dd77fe640 1 -- 172.21.15.43:0/4257085111 learned_addr learned my addr 172.21.15.43:0/4257085111 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:42.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.345+0000 7f2dd6ffd640 1 --2- 172.21.15.43:0/4257085111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581ac3e5a00 0x5581ac3e7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.345+0000 7f2dd4ff9640 1 --2- 172.21.15.43:0/2657474342 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2dc00a3d20 0x7f2dc00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.345+0000 7f2dd7fff640 1 -- 172.21.15.43:0/4257085111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581ac3e5a00 msgr2=0x5581ac3e7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.345+0000 7f2dd7fff640 1 --2- 172.21.15.43:0/4257085111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581ac3e5a00 0x5581ac3e7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.345+0000 7f2dd7fff640 1 -- 172.21.15.43:0/4257085111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 msgr2=0x5581ac3e5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.345+0000 7f2dd7fff640 1 --2- 172.21.15.43:0/4257085111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 0x5581ac3e5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.345+0000 7f2dd7fff640 1 -- 172.21.15.43:0/4257085111 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581ac2d01b0 con 0x5581ac3dc4d0 2024-04-21T08:41:42.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.345+0000 7f2dd4ff9640 1 -- 172.21.15.43:0/2657474342 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f2dc80f2cf0 con 0x7f2dd806a5d0 2024-04-21T08:41:42.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.345+0000 7f2dd7fff640 1 --2- 172.21.15.43:0/2657474342 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2dc00a3d20 0x7f2dc00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.346+0000 7f2dd7fff640 1 --2- 172.21.15.43:0/4257085111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581ac3dc4d0 0x5581ac3dc8b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0005fd0 tx=0x7f2dd000f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fda32bedd0cd7b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:42.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.346+0000 7f2dbdffb640 1 -- 172.21.15.43:0/4257085111 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0023070 con 0x5581ac3dc4d0 2024-04-21T08:41:42.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.346+0000 7f2dbdffb640 1 -- 172.21.15.43:0/4257085111 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dd0003970 con 0x5581ac3dc4d0 2024-04-21T08:41:42.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.346+0000 7f2dbdffb640 1 -- 172.21.15.43:0/4257085111 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd001b640 con 0x5581ac3dc4d0 2024-04-21T08:41:42.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.347+0000 7f2de09d2740 1 -- 172.21.15.43:0/4257085111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581ac3dc4d0 msgr2=0x5581ac3dc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.347+0000 7f2dd7fff640 1 --2- 172.21.15.43:0/2657474342 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2dc00a3d20 0x7f2dc00a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0020000 tx=0x7f2dd001d000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:42.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.347+0000 7f2de09d2740 1 --2- 172.21.15.43:0/4257085111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581ac3dc4d0 0x5581ac3dc8b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0005fd0 tx=0x7f2dd000f7f0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.347+0000 7f2de09d2740 1 -- 172.21.15.43:0/4257085111 shutdown_connections 2024-04-21T08:41:42.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.347+0000 7f2de09d2740 1 --2- 172.21.15.43:0/4257085111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581ac3e5a00 0x5581ac3e7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.347+0000 7f2de09d2740 1 --2- 172.21.15.43:0/4257085111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 0x5581ac3e5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.347+0000 7f2de09d2740 1 --2- 172.21.15.43:0/4257085111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581ac3dc4d0 0x5581ac3dc8b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.347+0000 7f2de09d2740 1 -- 172.21.15.43:0/4257085111 >> 172.21.15.43:0/4257085111 conn(0x5581ac3d6ba0 msgr2=0x5581ac3d6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:42.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.347+0000 7f2de09d2740 1 -- 172.21.15.43:0/4257085111 shutdown_connections 2024-04-21T08:41:42.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.347+0000 7f2de09d2740 1 -- 172.21.15.43:0/4257085111 wait complete. 2024-04-21T08:41:42.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.347+0000 7f2de09d2740 1 -- start start 2024-04-21T08:41:42.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.347+0000 7f2de09d2740 1 Processor -- start 2024-04-21T08:41:42.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.347+0000 7f2de09d2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581ac3dc4d0 0x5581ac3e5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.347+0000 7f2de09d2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 0x5581ac3e2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.347+0000 7f2de09d2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581ac3e5a00 0x5581ac3e2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.347+0000 7f2de09d2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5581ac3e8970 con 0x5581ac3dce80 2024-04-21T08:41:42.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.347+0000 7f2de09d2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5581ac2fa500 con 0x5581ac3e5a00 2024-04-21T08:41:42.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.347+0000 7f2de09d2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5581ac3e8af0 con 0x5581ac3dc4d0 2024-04-21T08:41:42.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.348+0000 7f2de09d2740 10 client.0 _ll_register_callbacks cb 0x5581ac2c8b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:42.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.348+0000 7f2dd6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 0x5581ac3e2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.348+0000 7f2dd6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 0x5581ac3e2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34200/0 (socket says 172.21.15.43:34200) 2024-04-21T08:41:42.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.348+0000 7f2dd6ffd640 1 -- 172.21.15.43:0/1855407002 learned_addr learned my addr 172.21.15.43:0/1855407002 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:42.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.348+0000 7f2dd7fff640 1 --2- 172.21.15.43:0/1855407002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581ac3e5a00 0x5581ac3e2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.348+0000 7f2dd77fe640 1 --2- 172.21.15.43:0/1855407002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581ac3dc4d0 0x5581ac3e5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.348+0000 7f2dd6ffd640 1 -- 172.21.15.43:0/1855407002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581ac3dc4d0 msgr2=0x5581ac3e5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.348+0000 7f2dd6ffd640 1 --2- 172.21.15.43:0/1855407002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581ac3dc4d0 0x5581ac3e5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.348+0000 7f2dd6ffd640 1 -- 172.21.15.43:0/1855407002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581ac3e5a00 msgr2=0x5581ac3e2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.348+0000 7f2dd6ffd640 1 --2- 172.21.15.43:0/1855407002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581ac3e5a00 0x5581ac3e2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.348+0000 7f2dd6ffd640 1 -- 172.21.15.43:0/1855407002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581ac0533f0 con 0x5581ac3dce80 2024-04-21T08:41:42.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.348+0000 7f2dd6ffd640 1 --2- 172.21.15.43:0/1855407002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 0x5581ac3e2600 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00ba60 tx=0x7f2dcc00bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c71a2005ee4e53 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:42.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.348+0000 7f2db6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:42.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.348+0000 7f2db6ffd640 1 -- 172.21.15.43:0/1855407002 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc005020 con 0x5581ac3dce80 2024-04-21T08:41:42.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.348+0000 7f2db6ffd640 1 -- 172.21.15.43:0/1855407002 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f2dcc0051c0 con 0x5581ac3dce80 2024-04-21T08:41:42.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.348+0000 7f2de09d2740 1 -- 172.21.15.43:0/1855407002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5581ac2e0930 con 0x5581ac3dce80 2024-04-21T08:41:42.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.348+0000 7f2db6ffd640 1 -- 172.21.15.43:0/1855407002 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2dcc005400 con 0x5581ac3dce80 2024-04-21T08:41:42.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.348+0000 7f2de09d2740 10 client.16977 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:42.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.348+0000 7f2de09d2740 1 -- 172.21.15.43:0/1855407002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5581ac11d7d0 con 0x5581ac3dce80 2024-04-21T08:41:42.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.349+0000 7f2db6ffd640 1 -- 172.21.15.43:0/1855407002 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5581ac2e0930 con 0x5581ac3dce80 2024-04-21T08:41:42.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.349+0000 7f2db6ffd640 1 -- 172.21.15.43:0/1855407002 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2dcc01b600 con 0x5581ac3dce80 2024-04-21T08:41:42.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.349+0000 7f2de09d2740 10 client.16977 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:42.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.349+0000 7f2de09d2740 10 client.16977 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:41:42.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.349+0000 7f2de09d2740 20 client.16977 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:42.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.349+0000 7f2de09d2740 1 --2- 172.21.15.43:0/1855407002 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5581ac3f5c70 0x5581ac3f8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.349+0000 7f2de09d2740 4 client.16977 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:42.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.349+0000 7f2de09d2740 1 -- 172.21.15.43:0/1855407002 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581ac34ec10 con 0x5581ac3f5c70 2024-04-21T08:41:42.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.349+0000 7f2dd77fe640 1 --2- 172.21.15.43:0/1855407002 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5581ac3f5c70 0x5581ac3f8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.349+0000 7f2dd77fe640 1 --2- 172.21.15.43:0/1855407002 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5581ac3f5c70 0x5581ac3f8090 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d30215cb0fdb27b server_cookie=9ff4a97bef75e176 in_seq=0 out_seq=0 2024-04-21T08:41:42.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.349+0000 7f2db6ffd640 10 client.16977 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:41:42.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.350+0000 7f2db6ffd640 1 -- 172.21.15.43:0/1855407002 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5581ac34ec10 con 0x5581ac3f5c70 2024-04-21T08:41:42.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.350+0000 7f2db6ffd640 10 client.16977 handle_command_reply: tid=0 2024-04-21T08:41:42.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.350+0000 7f2de09d2740 1 client.16977 shutdown 2024-04-21T08:41:42.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.350+0000 7f2de09d2740 1 -- 172.21.15.43:0/1855407002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 msgr2=0x5581ac3e2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.350+0000 7f2de09d2740 1 --2- 172.21.15.43:0/1855407002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 0x5581ac3e2600 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00ba60 tx=0x7f2dcc00bf20 comp rx=0 tx=0).stop 2024-04-21T08:41:42.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.350+0000 7f2de09d2740 1 -- 172.21.15.43:0/1855407002 shutdown_connections 2024-04-21T08:41:42.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.350+0000 7f2de09d2740 1 --2- 172.21.15.43:0/1855407002 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5581ac3f5c70 0x5581ac3f8090 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.350+0000 7f2de09d2740 1 --2- 172.21.15.43:0/1855407002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581ac3e5a00 0x5581ac3e2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.350+0000 7f2db6ffd640 0 client.16977 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:41:42.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.350+0000 7f2de09d2740 1 --2- 172.21.15.43:0/1855407002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 0x5581ac3e2600 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.350+0000 7f2de09d2740 1 --2- 172.21.15.43:0/1855407002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581ac3dc4d0 0x5581ac3e5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.350+0000 7f2de09d2740 1 -- 172.21.15.43:0/1855407002 >> 172.21.15.43:0/1855407002 conn(0x5581ac3d6ba0 msgr2=0x5581ac3d88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:42.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.350+0000 7f2de09d2740 1 -- 172.21.15.43:0/1855407002 shutdown_connections 2024-04-21T08:41:42.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.350+0000 7f2de09d2740 1 -- 172.21.15.43:0/1855407002 wait complete. 2024-04-21T08:41:42.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.351+0000 7f2de09d2740 20 client.16977 trim_cache size 0 max 16384 2024-04-21T08:41:42.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.364+0000 7f2de09d2740 1 Processor -- start 2024-04-21T08:41:42.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.364+0000 7f2de09d2740 1 -- start start 2024-04-21T08:41:42.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.364+0000 7f2de09d2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581ac3d6ba0 0x5581ac3f1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.364+0000 7f2de09d2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581ac3dc4d0 0x5581ac3ddae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.364+0000 7f2de09d2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 0x5581ac3e4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.364+0000 7f2de09d2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd81b1610 con 0x5581ac3dce80 2024-04-21T08:41:42.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.364+0000 7f2de09d2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd81b3150 con 0x5581ac3d6ba0 2024-04-21T08:41:42.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.364+0000 7f2de09d2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd81b1ce0 con 0x5581ac3dc4d0 2024-04-21T08:41:42.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.364+0000 7f2dd7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 0x5581ac3e4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.364+0000 7f2dd7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 0x5581ac3e4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34204/0 (socket says 172.21.15.43:34204) 2024-04-21T08:41:42.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.364+0000 7f2dd7fff640 1 -- 172.21.15.43:0/2753749835 learned_addr learned my addr 172.21.15.43:0/2753749835 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:42.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.364+0000 7f2dd77fe640 1 --2- 172.21.15.43:0/2753749835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581ac3d6ba0 0x5581ac3f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.364+0000 7f2dd6ffd640 1 --2- 172.21.15.43:0/2753749835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581ac3dc4d0 0x5581ac3ddae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.364+0000 7f2dd7fff640 1 -- 172.21.15.43:0/2753749835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581ac3dc4d0 msgr2=0x5581ac3ddae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.364+0000 7f2dd7fff640 1 --2- 172.21.15.43:0/2753749835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581ac3dc4d0 0x5581ac3ddae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.364+0000 7f2dd7fff640 1 -- 172.21.15.43:0/2753749835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581ac3d6ba0 msgr2=0x5581ac3f1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.364+0000 7f2dd7fff640 1 --2- 172.21.15.43:0/2753749835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581ac3d6ba0 0x5581ac3f1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.364+0000 7f2dd7fff640 1 -- 172.21.15.43:0/2753749835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581ac3e0b50 con 0x5581ac3dce80 2024-04-21T08:41:42.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.365+0000 7f2dd7fff640 1 --2- 172.21.15.43:0/2753749835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 0x5581ac3e4d30 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f2dd00094c0 tx=0x7f2dd0005b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a2cbbe2354ce3f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:42.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.365+0000 7f2dbd7fa640 1 -- 172.21.15.43:0/2753749835 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0023040 con 0x5581ac3dce80 2024-04-21T08:41:42.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.365+0000 7f2dbd7fa640 1 -- 172.21.15.43:0/2753749835 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dd0004920 con 0x5581ac3dce80 2024-04-21T08:41:42.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.365+0000 7f2dbd7fa640 1 -- 172.21.15.43:0/2753749835 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd001d6b0 con 0x5581ac3dce80 2024-04-21T08:41:42.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.365+0000 7f2de09d2740 1 -- 172.21.15.43:0/2753749835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 msgr2=0x5581ac3e4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.365+0000 7f2de09d2740 1 --2- 172.21.15.43:0/2753749835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 0x5581ac3e4d30 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f2dd00094c0 tx=0x7f2dd0005b30 comp rx=0 tx=0).stop 2024-04-21T08:41:42.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.365+0000 7f2de09d2740 1 -- 172.21.15.43:0/2753749835 shutdown_connections 2024-04-21T08:41:42.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.365+0000 7f2de09d2740 1 --2- 172.21.15.43:0/2753749835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 0x5581ac3e4d30 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.365+0000 7f2de09d2740 1 --2- 172.21.15.43:0/2753749835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581ac3dc4d0 0x5581ac3ddae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.365+0000 7f2de09d2740 1 --2- 172.21.15.43:0/2753749835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581ac3d6ba0 0x5581ac3f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.365+0000 7f2de09d2740 1 -- 172.21.15.43:0/2753749835 >> 172.21.15.43:0/2753749835 conn(0x5581ac3e2ca0 msgr2=0x5581ac3dd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:42.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.365+0000 7f2de09d2740 1 -- 172.21.15.43:0/2753749835 shutdown_connections 2024-04-21T08:41:42.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.365+0000 7f2de09d2740 1 -- 172.21.15.43:0/2753749835 wait complete. 2024-04-21T08:41:42.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.366+0000 7f2de09d2740 1 -- start start 2024-04-21T08:41:42.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.366+0000 7f2de09d2740 1 Processor -- start 2024-04-21T08:41:42.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.366+0000 7f2de09d2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581ac3d6ba0 0x5581ac3ee230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.366+0000 7f2de09d2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581ac3dc4d0 0x5581ac3e95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.366+0000 7f2de09d2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 0x5581ac3e9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.366+0000 7f2de09d2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd81b2a80 con 0x5581ac3dce80 2024-04-21T08:41:42.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.366+0000 7f2de09d2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd81b23b0 con 0x5581ac3d6ba0 2024-04-21T08:41:42.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.366+0000 7f2de09d2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd81af5f0 con 0x5581ac3dc4d0 2024-04-21T08:41:42.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.366+0000 7f2de09d2740 10 client.0 _ll_register_callbacks cb 0x7f2dcc00b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:42.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.366+0000 7f2dd7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 0x5581ac3e9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.366+0000 7f2dd7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 0x5581ac3e9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34210/0 (socket says 172.21.15.43:34210) 2024-04-21T08:41:42.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.366+0000 7f2dd7fff640 1 -- 172.21.15.43:0/1736206056 learned_addr learned my addr 172.21.15.43:0/1736206056 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:42.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.366+0000 7f2dd77fe640 1 --2- 172.21.15.43:0/1736206056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581ac3d6ba0 0x5581ac3ee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.366+0000 7f2dd6ffd640 1 --2- 172.21.15.43:0/1736206056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581ac3dc4d0 0x5581ac3e95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.367+0000 7f2dd7fff640 1 -- 172.21.15.43:0/1736206056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581ac3dc4d0 msgr2=0x5581ac3e95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.367+0000 7f2dd7fff640 1 --2- 172.21.15.43:0/1736206056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581ac3dc4d0 0x5581ac3e95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.367+0000 7f2dd7fff640 1 -- 172.21.15.43:0/1736206056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581ac3d6ba0 msgr2=0x5581ac3ee230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.367+0000 7f2dd7fff640 1 --2- 172.21.15.43:0/1736206056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581ac3d6ba0 0x5581ac3ee230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.367+0000 7f2dd7fff640 1 -- 172.21.15.43:0/1736206056 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581ac3ea3e0 con 0x5581ac3dce80 2024-04-21T08:41:42.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.367+0000 7f2dd77fe640 1 --2- 172.21.15.43:0/1736206056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581ac3d6ba0 0x5581ac3ee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:42.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.367+0000 7f2dd7fff640 1 --2- 172.21.15.43:0/1736206056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 0x5581ac3e9bb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f2dd000f830 tx=0x7f2dd0009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307fd4816eb266fb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:42.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.367+0000 7f2db6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:42.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.367+0000 7f2db6ffd640 1 -- 172.21.15.43:0/1736206056 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0035020 con 0x5581ac3dce80 2024-04-21T08:41:42.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.367+0000 7f2db6ffd640 1 -- 172.21.15.43:0/1736206056 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f2dd001b960 con 0x5581ac3dce80 2024-04-21T08:41:42.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.367+0000 7f2de09d2740 1 -- 172.21.15.43:0/1736206056 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dd0004920 con 0x5581ac3dce80 2024-04-21T08:41:42.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.367+0000 7f2db6ffd640 1 -- 172.21.15.43:0/1736206056 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2dd0023070 con 0x5581ac3dce80 2024-04-21T08:41:42.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.367+0000 7f2de09d2740 10 client.16983 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:42.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.367+0000 7f2de09d2740 1 -- 172.21.15.43:0/1736206056 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5581ac45c320 con 0x5581ac3dce80 2024-04-21T08:41:42.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.367+0000 7f2db6ffd640 1 -- 172.21.15.43:0/1736206056 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dd0004920 con 0x5581ac3dce80 2024-04-21T08:41:42.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.367+0000 7f2db6ffd640 1 -- 172.21.15.43:0/1736206056 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2dd0027440 con 0x5581ac3dce80 2024-04-21T08:41:42.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.367+0000 7f2de09d2740 10 client.16983 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:42.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.367+0000 7f2de09d2740 10 client.16983 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:41:42.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.367+0000 7f2de09d2740 20 client.16983 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:42.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.368+0000 7f2de09d2740 1 --2- 172.21.15.43:0/1736206056 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5581ac3e5a00 0x5581ac45e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.368+0000 7f2de09d2740 4 client.16983 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:41:42.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.368+0000 7f2de09d2740 1 -- 172.21.15.43:0/1736206056 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581ac348a80 con 0x5581ac3e5a00 2024-04-21T08:41:42.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.368+0000 7f2dd77fe640 1 --2- 172.21.15.43:0/1736206056 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5581ac3e5a00 0x5581ac45e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.368+0000 7f2db6ffd640 10 client.16983 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:41:42.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.368+0000 7f2dd77fe640 1 --2- 172.21.15.43:0/1736206056 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5581ac3e5a00 0x5581ac45e840 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=71a1e4cd30e2c70f server_cookie=2b46ddfd461551d2 in_seq=0 out_seq=0 2024-04-21T08:41:42.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.368+0000 7f2db6ffd640 1 -- 172.21.15.43:0/1736206056 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581ac348a80 con 0x5581ac3e5a00 2024-04-21T08:41:42.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.368+0000 7f2db6ffd640 10 client.16983 handle_command_reply: tid=0 2024-04-21T08:41:42.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.368+0000 7f2de09d2740 1 client.16983 shutdown 2024-04-21T08:41:42.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.369+0000 7f2de09d2740 1 -- 172.21.15.43:0/1736206056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 msgr2=0x5581ac3e9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.369+0000 7f2de09d2740 1 --2- 172.21.15.43:0/1736206056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 0x5581ac3e9bb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f2dd000f830 tx=0x7f2dd0009f20 comp rx=0 tx=0).stop 2024-04-21T08:41:42.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.369+0000 7f2de09d2740 1 -- 172.21.15.43:0/1736206056 shutdown_connections 2024-04-21T08:41:42.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.369+0000 7f2de09d2740 1 --2- 172.21.15.43:0/1736206056 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5581ac3e5a00 0x5581ac45e840 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.369+0000 7f2de09d2740 1 --2- 172.21.15.43:0/1736206056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5581ac3dce80 0x5581ac3e9bb0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.369+0000 7f2db6ffd640 0 client.16983 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:41:42.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.369+0000 7f2de09d2740 1 --2- 172.21.15.43:0/1736206056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581ac3dc4d0 0x5581ac3e95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.369+0000 7f2de09d2740 1 --2- 172.21.15.43:0/1736206056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581ac3d6ba0 0x5581ac3ee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.369+0000 7f2de09d2740 1 -- 172.21.15.43:0/1736206056 >> 172.21.15.43:0/1736206056 conn(0x5581ac3e2ca0 msgr2=0x5581ac3d5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:42.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.369+0000 7f2de09d2740 1 -- 172.21.15.43:0/1736206056 shutdown_connections 2024-04-21T08:41:42.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.369+0000 7f2de09d2740 1 -- 172.21.15.43:0/1736206056 wait complete. 2024-04-21T08:41:42.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.369+0000 7f2de09d2740 20 client.16983 trim_cache size 0 max 16384 2024-04-21T08:41:42.376 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:41:42.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.369+0000 7f2dddde6640 1 -- 172.21.15.43:0/2657474342 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2dc00a3d20 msgr2=0x7f2dc00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.369+0000 7f2dddde6640 1 --2- 172.21.15.43:0/2657474342 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2dc00a3d20 0x7f2dc00a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0020000 tx=0x7f2dd001d000 comp rx=0 tx=0).stop 2024-04-21T08:41:42.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.369+0000 7f2dddde6640 1 -- 172.21.15.43:0/2657474342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2dd806a5d0 msgr2=0x7f2dd81a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.369+0000 7f2dddde6640 1 --2- 172.21.15.43:0/2657474342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2dd806a5d0 0x7f2dd81a0610 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8005940 tx=0x7f2dc80045e0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.369+0000 7f2dddde6640 1 -- 172.21.15.43:0/2657474342 shutdown_connections 2024-04-21T08:41:42.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.369+0000 7f2dddde6640 1 --2- 172.21.15.43:0/2657474342 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2dc00a3d20 0x7f2dc00a61e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.369+0000 7f2dddde6640 1 --2- 172.21.15.43:0/2657474342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dd810e960 0x7f2dd819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.369+0000 7f2dddde6640 1 --2- 172.21.15.43:0/2657474342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dd806aef0 0x7f2dd81a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.369+0000 7f2dddde6640 1 --2- 172.21.15.43:0/2657474342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2dd806a5d0 0x7f2dd81a0610 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.369+0000 7f2dddde6640 1 -- 172.21.15.43:0/2657474342 >> 172.21.15.43:0/2657474342 conn(0x7f2dd8077eb0 msgr2=0x7f2dd810fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:42.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.369+0000 7f2dddde6640 1 -- 172.21.15.43:0/2657474342 shutdown_connections 2024-04-21T08:41:42.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.369+0000 7f2dddde6640 1 -- 172.21.15.43:0/2657474342 wait complete. 2024-04-21T08:41:42.378 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:42 smithi043 ceph-mon[27978]: pgmap v86: 129 pgs: 129 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-21T08:41:42.378 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:42 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1840436474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:42.379 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:41:42.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:42 smithi175 ceph-mon[31920]: pgmap v86: 129 pgs: 129 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-21T08:41:42.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:42 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1840436474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:42.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.484+0000 7fd0d308c640 1 -- 172.21.15.43:0/1218586999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0cc107270 msgr2=0x7fd0cc10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.484+0000 7fd0d308c640 1 --2- 172.21.15.43:0/1218586999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0cc107270 0x7fd0cc10ee40 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4009ca0 tx=0x7fd0c402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.484+0000 7fd0d308c640 1 -- 172.21.15.43:0/1218586999 shutdown_connections 2024-04-21T08:41:42.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.484+0000 7fd0d308c640 1 --2- 172.21.15.43:0/1218586999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0cc107270 0x7fd0cc10ee40 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.484+0000 7fd0d308c640 1 --2- 172.21.15.43:0/1218586999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0cc06b1f0 0x7fd0cc106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.484+0000 7fd0d308c640 1 --2- 172.21.15.43:0/1218586999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0cc06a840 0x7fd0cc06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.484+0000 7fd0d308c640 1 -- 172.21.15.43:0/1218586999 >> 172.21.15.43:0/1218586999 conn(0x7fd0cc077eb0 msgr2=0x7fd0cc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:42.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.484+0000 7fd0d308c640 1 -- 172.21.15.43:0/1218586999 shutdown_connections 2024-04-21T08:41:42.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.484+0000 7fd0d308c640 1 -- 172.21.15.43:0/1218586999 wait complete. 2024-04-21T08:41:42.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.485+0000 7fd0d308c640 1 Processor -- start 2024-04-21T08:41:42.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.485+0000 7fd0d308c640 1 -- start start 2024-04-21T08:41:42.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.485+0000 7fd0d308c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0cc06a840 0x7fd0cc1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.485+0000 7fd0d308c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0cc107270 0x7fd0cc198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.485+0000 7fd0d308c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0cc19cf40 0x7fd0cc19d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.485+0000 7fd0d308c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc112230 con 0x7fd0cc107270 2024-04-21T08:41:42.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.485+0000 7fd0d308c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1120b0 con 0x7fd0cc06a840 2024-04-21T08:41:42.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.485+0000 7fd0d308c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1123b0 con 0x7fd0cc19cf40 2024-04-21T08:41:42.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.485+0000 7fd0cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0cc107270 0x7fd0cc198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.485+0000 7fd0cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0cc107270 0x7fd0cc198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34234/0 (socket says 172.21.15.43:34234) 2024-04-21T08:41:42.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.485+0000 7fd0cbfff640 1 -- 172.21.15.43:0/481137326 learned_addr learned my addr 172.21.15.43:0/481137326 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:42.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.485+0000 7fd0d1629640 1 --2- 172.21.15.43:0/481137326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0cc19cf40 0x7fd0cc19d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.485+0000 7fd0d0e28640 1 --2- 172.21.15.43:0/481137326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0cc06a840 0x7fd0cc1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.486+0000 7fd0cbfff640 1 -- 172.21.15.43:0/481137326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0cc19cf40 msgr2=0x7fd0cc19d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.486+0000 7fd0cbfff640 1 --2- 172.21.15.43:0/481137326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0cc19cf40 0x7fd0cc19d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.486+0000 7fd0cbfff640 1 -- 172.21.15.43:0/481137326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0cc06a840 msgr2=0x7fd0cc1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.486+0000 7fd0cbfff640 1 --2- 172.21.15.43:0/481137326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0cc06a840 0x7fd0cc1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.486+0000 7fd0cbfff640 1 -- 172.21.15.43:0/481137326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0cc19da30 con 0x7fd0cc107270 2024-04-21T08:41:42.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.486+0000 7fd0cbfff640 1 --2- 172.21.15.43:0/481137326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0cc107270 0x7fd0cc198c10 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000c970 tx=0x7fd0c000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:42.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.486+0000 7fd0c9ffb640 1 -- 172.21.15.43:0/481137326 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0014070 con 0x7fd0cc107270 2024-04-21T08:41:42.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.486+0000 7fd0c9ffb640 1 -- 172.21.15.43:0/481137326 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0c0004440 con 0x7fd0cc107270 2024-04-21T08:41:42.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.486+0000 7fd0d308c640 1 -- 172.21.15.43:0/481137326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0cc19dd20 con 0x7fd0cc107270 2024-04-21T08:41:42.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.486+0000 7fd0c9ffb640 1 -- 172.21.15.43:0/481137326 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0003d30 con 0x7fd0cc107270 2024-04-21T08:41:42.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.486+0000 7fd0d308c640 1 -- 172.21.15.43:0/481137326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0cc1a5560 con 0x7fd0cc107270 2024-04-21T08:41:42.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.488+0000 7fd0c9ffb640 1 -- 172.21.15.43:0/481137326 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd0c0017070 con 0x7fd0cc107270 2024-04-21T08:41:42.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.489+0000 7fd0d5c82740 1 Processor -- start 2024-04-21T08:41:42.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.489+0000 7fd0d5c82740 1 -- start start 2024-04-21T08:41:42.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.489+0000 7fd0c9ffb640 1 --2- 172.21.15.43:0/481137326 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd0b40a3da0 0x7fd0b40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.489+0000 7fd0d5c82740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4e4d0 0x56022ab4e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.489+0000 7fd0d5c82740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56022ab4ee80 0x56022ab57430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.489+0000 7fd0d5c82740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56022ab57a00 0x56022ab59e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.489+0000 7fd0d5c82740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56022aab31f0 con 0x56022ab4e4d0 2024-04-21T08:41:42.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.489+0000 7fd0d5c82740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56022aa504b0 con 0x56022ab57a00 2024-04-21T08:41:42.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.489+0000 7fd0d5c82740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56022aaaf1b0 con 0x56022ab4ee80 2024-04-21T08:41:42.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.489+0000 7fd0d1629640 1 --2- 172.21.15.43:0/481137326 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd0b40a3da0 0x7fd0b40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.489+0000 7fd0c9ffb640 1 -- 172.21.15.43:0/481137326 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd0c00d0920 con 0x7fd0cc107270 2024-04-21T08:41:42.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.489+0000 7fd0d0e28640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4e4d0 0x56022ab4e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.489+0000 7fd0d0e28640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4e4d0 0x56022ab4e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34246/0 (socket says 172.21.15.43:34246) 2024-04-21T08:41:42.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.489+0000 7fd0d0e28640 1 -- 172.21.15.43:0/3134740856 learned_addr learned my addr 172.21.15.43:0/3134740856 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:42.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.489+0000 7fd0cbfff640 1 --2- 172.21.15.43:0/3134740856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56022ab4ee80 0x56022ab57430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.489+0000 7fd0d0e28640 1 -- 172.21.15.43:0/3134740856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56022ab4ee80 msgr2=0x56022ab57430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.489+0000 7fd0d0e28640 1 --2- 172.21.15.43:0/3134740856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56022ab4ee80 0x56022ab57430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.489+0000 7fd0d1629640 1 --2- 172.21.15.43:0/3134740856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56022ab57a00 0x56022ab59e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.489+0000 7fd0d0e28640 1 -- 172.21.15.43:0/3134740856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56022ab57a00 msgr2=0x56022ab59e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.489+0000 7fd0d0e28640 1 --2- 172.21.15.43:0/3134740856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56022ab57a00 0x56022ab59e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.489+0000 7fd0d0e28640 1 -- 172.21.15.43:0/3134740856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56022aa421b0 con 0x56022ab4e4d0 2024-04-21T08:41:42.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.489+0000 7fd0cbfff640 1 --2- 172.21.15.43:0/3134740856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56022ab4ee80 0x56022ab57430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:42.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.489+0000 7fd0d1629640 1 --2- 172.21.15.43:0/3134740856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56022ab57a00 0x56022ab59e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:42.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.490+0000 7fd0d1629640 1 --2- 172.21.15.43:0/481137326 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd0b40a3da0 0x7fd0b40a6260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd0c40098d0 tx=0x7fd0c402cca0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.490+0000 7fd0d0e28640 1 --2- 172.21.15.43:0/3134740856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4e4d0 0x56022ab4e8b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009a70 tx=0x7fd0bc009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9d9b06bfe52f25 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.490+0000 7fd0b2ffd640 1 -- 172.21.15.43:0/3134740856 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc017070 con 0x56022ab4e4d0 2024-04-21T08:41:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.490+0000 7fd0b2ffd640 1 -- 172.21.15.43:0/3134740856 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0bc013470 con 0x56022ab4e4d0 2024-04-21T08:41:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.490+0000 7fd0b2ffd640 1 -- 172.21.15.43:0/3134740856 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc0125d0 con 0x56022ab4e4d0 2024-04-21T08:41:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.490+0000 7fd0d5c82740 1 -- 172.21.15.43:0/3134740856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4e4d0 msgr2=0x56022ab4e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.490+0000 7fd0d5c82740 1 --2- 172.21.15.43:0/3134740856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4e4d0 0x56022ab4e8b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009a70 tx=0x7fd0bc009f30 comp rx=0 tx=0).stop 2024-04-21T08:41:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.491+0000 7fd0d5c82740 1 -- 172.21.15.43:0/3134740856 shutdown_connections 2024-04-21T08:41:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.491+0000 7fd0d5c82740 1 --2- 172.21.15.43:0/3134740856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56022ab57a00 0x56022ab59e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.491+0000 7fd0d5c82740 1 --2- 172.21.15.43:0/3134740856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56022ab4ee80 0x56022ab57430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.491+0000 7fd0d5c82740 1 --2- 172.21.15.43:0/3134740856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4e4d0 0x56022ab4e8b0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.491+0000 7fd0d5c82740 1 -- 172.21.15.43:0/3134740856 >> 172.21.15.43:0/3134740856 conn(0x56022ab48ba0 msgr2=0x56022ab48f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.491+0000 7fd0d5c82740 1 -- 172.21.15.43:0/3134740856 shutdown_connections 2024-04-21T08:41:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.491+0000 7fd0d5c82740 1 -- 172.21.15.43:0/3134740856 wait complete. 2024-04-21T08:41:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.491+0000 7fd0d5c82740 1 -- start start 2024-04-21T08:41:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.491+0000 7fd0d5c82740 1 Processor -- start 2024-04-21T08:41:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.491+0000 7fd0d5c82740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56022ab4e4d0 0x56022ab57310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.491+0000 7fd0d5c82740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4ee80 0x56022ab546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.491+0000 7fd0d5c82740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56022ab57a00 0x56022ab54c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.491+0000 7fd0d5c82740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56022ab5a970 con 0x56022ab4ee80 2024-04-21T08:41:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.491+0000 7fd0d5c82740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56022aa6c500 con 0x56022ab57a00 2024-04-21T08:41:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.491+0000 7fd0d5c82740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56022ab5aaf0 con 0x56022ab4e4d0 2024-04-21T08:41:42.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.491+0000 7fd0d5c82740 10 client.0 _ll_register_callbacks cb 0x56022aa3ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:42.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.491+0000 7fd0cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4ee80 0x56022ab546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.492+0000 7fd0cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4ee80 0x56022ab546a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34252/0 (socket says 172.21.15.43:34252) 2024-04-21T08:41:42.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.492+0000 7fd0cbfff640 1 -- 172.21.15.43:0/708460804 learned_addr learned my addr 172.21.15.43:0/708460804 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:42.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.492+0000 7fd0d0e28640 1 --2- 172.21.15.43:0/708460804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56022ab4e4d0 0x56022ab57310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.492+0000 7fd0d1629640 1 --2- 172.21.15.43:0/708460804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56022ab57a00 0x56022ab54c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.492+0000 7fd0cbfff640 1 -- 172.21.15.43:0/708460804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56022ab4e4d0 msgr2=0x56022ab57310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.492+0000 7fd0cbfff640 1 --2- 172.21.15.43:0/708460804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56022ab4e4d0 0x56022ab57310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.492+0000 7fd0cbfff640 1 -- 172.21.15.43:0/708460804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56022ab57a00 msgr2=0x56022ab54c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.492+0000 7fd0cbfff640 1 --2- 172.21.15.43:0/708460804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56022ab57a00 0x56022ab54c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.492+0000 7fd0cbfff640 1 -- 172.21.15.43:0/708460804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56022a7c53f0 con 0x56022ab4ee80 2024-04-21T08:41:42.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.492+0000 7fd0d1629640 1 --2- 172.21.15.43:0/708460804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56022ab57a00 0x56022ab54c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.492+0000 7fd0d0e28640 1 --2- 172.21.15.43:0/708460804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56022ab4e4d0 0x56022ab57310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.492+0000 7fd0cbfff640 1 --2- 172.21.15.43:0/708460804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4ee80 0x56022ab546a0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00988f0 tx=0x7fd0c0007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d1b995009a3f79 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.492+0000 7fd0abfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.492+0000 7fd0abfff640 1 -- 172.21.15.43:0/708460804 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c009c490 con 0x56022ab4ee80 2024-04-21T08:41:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.492+0000 7fd0abfff640 1 -- 172.21.15.43:0/708460804 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd0c00a4c70 con 0x56022ab4ee80 2024-04-21T08:41:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.492+0000 7fd0d5c82740 1 -- 172.21.15.43:0/708460804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0bc013470 con 0x56022ab4ee80 2024-04-21T08:41:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.492+0000 7fd0abfff640 1 -- 172.21.15.43:0/708460804 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd0c00a0840 con 0x56022ab4ee80 2024-04-21T08:41:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.492+0000 7fd0d5c82740 10 client.16995 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.492+0000 7fd0d5c82740 1 -- 172.21.15.43:0/708460804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56022a88f7d0 con 0x56022ab4ee80 2024-04-21T08:41:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.492+0000 7fd0abfff640 1 -- 172.21.15.43:0/708460804 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0bc013470 con 0x56022ab4ee80 2024-04-21T08:41:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.493+0000 7fd0abfff640 1 -- 172.21.15.43:0/708460804 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd0c009b8e0 con 0x56022ab4ee80 2024-04-21T08:41:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.493+0000 7fd0d5c82740 10 client.16995 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.493+0000 7fd0d5c82740 10 client.16995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.493+0000 7fd0d5c82740 20 client.16995 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.493+0000 7fd0d5c82740 1 --2- 172.21.15.43:0/708460804 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56022ab67ad0 0x56022ab69f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.493+0000 7fd0d5c82740 4 client.16995 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.493+0000 7fd0d5c82740 1 -- 172.21.15.43:0/708460804 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56022aa52930 con 0x56022ab67ad0 2024-04-21T08:41:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.493+0000 7fd0d0e28640 1 --2- 172.21.15.43:0/708460804 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56022ab67ad0 0x56022ab69f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.493+0000 7fd0abfff640 10 client.16995 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.493+0000 7fd0d0e28640 1 --2- 172.21.15.43:0/708460804 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56022ab67ad0 0x56022ab69f20 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb39319be2eefeec server_cookie=423811d555567117 in_seq=0 out_seq=0 2024-04-21T08:41:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.494+0000 7fd0abfff640 1 -- 172.21.15.43:0/708460804 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56022aa52930 con 0x56022ab67ad0 2024-04-21T08:41:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.494+0000 7fd0abfff640 10 client.16995 handle_command_reply: tid=0 2024-04-21T08:41:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.494+0000 7fd0d5c82740 1 client.16995 shutdown 2024-04-21T08:41:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.494+0000 7fd0d5c82740 1 -- 172.21.15.43:0/708460804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4ee80 msgr2=0x56022ab546a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.494+0000 7fd0d5c82740 1 --2- 172.21.15.43:0/708460804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4ee80 0x56022ab546a0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00988f0 tx=0x7fd0c0007ec0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.494+0000 7fd0d5c82740 1 -- 172.21.15.43:0/708460804 shutdown_connections 2024-04-21T08:41:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.494+0000 7fd0d5c82740 1 --2- 172.21.15.43:0/708460804 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56022ab67ad0 0x56022ab69f20 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.494+0000 7fd0d5c82740 1 --2- 172.21.15.43:0/708460804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56022ab57a00 0x56022ab54c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.494+0000 7fd0d5c82740 1 --2- 172.21.15.43:0/708460804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4ee80 0x56022ab546a0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.494+0000 7fd0abfff640 0 client.16995 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:42.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.494+0000 7fd0d5c82740 1 --2- 172.21.15.43:0/708460804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56022ab4e4d0 0x56022ab57310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.494+0000 7fd0d5c82740 1 -- 172.21.15.43:0/708460804 >> 172.21.15.43:0/708460804 conn(0x56022ab48ba0 msgr2=0x56022ab4a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:42.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.494+0000 7fd0d5c82740 1 -- 172.21.15.43:0/708460804 shutdown_connections 2024-04-21T08:41:42.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.495+0000 7fd0d5c82740 1 -- 172.21.15.43:0/708460804 wait complete. 2024-04-21T08:41:42.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.495+0000 7fd0d5c82740 20 client.16995 trim_cache size 0 max 16384 2024-04-21T08:41:42.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.508+0000 7fd0d5c82740 1 Processor -- start 2024-04-21T08:41:42.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.508+0000 7fd0d5c82740 1 -- start start 2024-04-21T08:41:42.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.508+0000 7fd0d5c82740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56022ab4e4d0 0x56022ab63830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.508+0000 7fd0d5c82740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4ee80 0x56022ab4a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.508+0000 7fd0d5c82740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56022ab57a00 0x56022ab56dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.508+0000 7fd0d5c82740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1af370 con 0x56022ab4ee80 2024-04-21T08:41:42.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.508+0000 7fd0d5c82740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1b0eb0 con 0x56022ab4e4d0 2024-04-21T08:41:42.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.508+0000 7fd0d5c82740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1afa40 con 0x56022ab57a00 2024-04-21T08:41:42.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.508+0000 7fd0cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4ee80 0x56022ab4a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.508+0000 7fd0cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4ee80 0x56022ab4a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34258/0 (socket says 172.21.15.43:34258) 2024-04-21T08:41:42.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.508+0000 7fd0cbfff640 1 -- 172.21.15.43:0/1722617579 learned_addr learned my addr 172.21.15.43:0/1722617579 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:42.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.508+0000 7fd0d0e28640 1 --2- 172.21.15.43:0/1722617579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56022ab4e4d0 0x56022ab63830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.508+0000 7fd0d1629640 1 --2- 172.21.15.43:0/1722617579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56022ab57a00 0x56022ab56dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.508+0000 7fd0cbfff640 1 -- 172.21.15.43:0/1722617579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56022ab57a00 msgr2=0x56022ab56dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.508+0000 7fd0cbfff640 1 --2- 172.21.15.43:0/1722617579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56022ab57a00 0x56022ab56dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.508+0000 7fd0cbfff640 1 -- 172.21.15.43:0/1722617579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56022ab4e4d0 msgr2=0x56022ab63830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.508+0000 7fd0cbfff640 1 --2- 172.21.15.43:0/1722617579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56022ab4e4d0 0x56022ab63830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.508+0000 7fd0cbfff640 1 -- 172.21.15.43:0/1722617579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56022ab5a180 con 0x56022ab4ee80 2024-04-21T08:41:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.509+0000 7fd0cbfff640 1 --2- 172.21.15.43:0/1722617579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4ee80 0x56022ab4a860 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd0c009c420 tx=0x7fd0c00045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea97226b26565ca server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.509+0000 7fd0b27fc640 1 -- 172.21.15.43:0/1722617579 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0098730 con 0x56022ab4ee80 2024-04-21T08:41:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.509+0000 7fd0b27fc640 1 -- 172.21.15.43:0/1722617579 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0c009c490 con 0x56022ab4ee80 2024-04-21T08:41:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.509+0000 7fd0b27fc640 1 -- 172.21.15.43:0/1722617579 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00a4c70 con 0x56022ab4ee80 2024-04-21T08:41:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.509+0000 7fd0d5c82740 1 -- 172.21.15.43:0/1722617579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4ee80 msgr2=0x56022ab4a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.509+0000 7fd0d5c82740 1 --2- 172.21.15.43:0/1722617579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4ee80 0x56022ab4a860 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd0c009c420 tx=0x7fd0c00045f0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.509+0000 7fd0d5c82740 1 -- 172.21.15.43:0/1722617579 shutdown_connections 2024-04-21T08:41:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.509+0000 7fd0d5c82740 1 --2- 172.21.15.43:0/1722617579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56022ab57a00 0x56022ab56dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.509+0000 7fd0d5c82740 1 --2- 172.21.15.43:0/1722617579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4ee80 0x56022ab4a860 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.509+0000 7fd0d5c82740 1 --2- 172.21.15.43:0/1722617579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56022ab4e4d0 0x56022ab63830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.509+0000 7fd0d5c82740 1 -- 172.21.15.43:0/1722617579 >> 172.21.15.43:0/1722617579 conn(0x56022ab48ba0 msgr2=0x56022ab48f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.509+0000 7fd0d5c82740 1 -- 172.21.15.43:0/1722617579 shutdown_connections 2024-04-21T08:41:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.509+0000 7fd0d5c82740 1 -- 172.21.15.43:0/1722617579 wait complete. 2024-04-21T08:41:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.510+0000 7fd0d5c82740 1 -- start start 2024-04-21T08:41:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.510+0000 7fd0d5c82740 1 Processor -- start 2024-04-21T08:41:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.510+0000 7fd0d5c82740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56022ab4e4d0 0x56022ab5fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.510+0000 7fd0d5c82740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4ee80 0x56022ab5b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.510+0000 7fd0d5c82740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56022ab57a00 0x56022ab5b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.510+0000 7fd0d5c82740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1b07e0 con 0x56022ab4ee80 2024-04-21T08:41:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.510+0000 7fd0d5c82740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1b0110 con 0x56022ab57a00 2024-04-21T08:41:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.510+0000 7fd0d5c82740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1ad350 con 0x56022ab4e4d0 2024-04-21T08:41:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.510+0000 7fd0d5c82740 10 client.0 _ll_register_callbacks cb 0x7fd0c009c060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.510+0000 7fd0cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4ee80 0x56022ab5b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.510+0000 7fd0cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4ee80 0x56022ab5b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34266/0 (socket says 172.21.15.43:34266) 2024-04-21T08:41:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.510+0000 7fd0cbfff640 1 -- 172.21.15.43:0/2326084409 learned_addr learned my addr 172.21.15.43:0/2326084409 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.510+0000 7fd0d0e28640 1 --2- 172.21.15.43:0/2326084409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56022ab4e4d0 0x56022ab5fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.510+0000 7fd0d1629640 1 --2- 172.21.15.43:0/2326084409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56022ab57a00 0x56022ab5b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.511+0000 7fd0cbfff640 1 -- 172.21.15.43:0/2326084409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56022ab4e4d0 msgr2=0x56022ab5fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.511+0000 7fd0cbfff640 1 --2- 172.21.15.43:0/2326084409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56022ab4e4d0 0x56022ab5fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.511+0000 7fd0cbfff640 1 -- 172.21.15.43:0/2326084409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56022ab57a00 msgr2=0x56022ab5b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:42.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.511+0000 7fd0cbfff640 1 --2- 172.21.15.43:0/2326084409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56022ab57a00 0x56022ab5b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.511+0000 7fd0cbfff640 1 -- 172.21.15.43:0/2326084409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56022ab5bd40 con 0x56022ab4ee80 2024-04-21T08:41:42.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.511+0000 7fd0d0e28640 1 --2- 172.21.15.43:0/2326084409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56022ab4e4d0 0x56022ab5fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:42.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.511+0000 7fd0cbfff640 1 --2- 172.21.15.43:0/2326084409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4ee80 0x56022ab5b2c0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00988f0 tx=0x7fd0c009bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3257e2e1868482d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:42.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.511+0000 7fd0abfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:42.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.511+0000 7fd0abfff640 1 -- 172.21.15.43:0/2326084409 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0084060 con 0x56022ab4ee80 2024-04-21T08:41:42.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.511+0000 7fd0abfff640 1 -- 172.21.15.43:0/2326084409 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd0c00d0410 con 0x56022ab4ee80 2024-04-21T08:41:42.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.511+0000 7fd0d5c82740 1 -- 172.21.15.43:0/2326084409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0c009c490 con 0x56022ab4ee80 2024-04-21T08:41:42.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.511+0000 7fd0abfff640 1 -- 172.21.15.43:0/2326084409 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd0c0098730 con 0x56022ab4ee80 2024-04-21T08:41:42.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.511+0000 7fd0d5c82740 10 client.17001 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:42.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.511+0000 7fd0d5c82740 1 -- 172.21.15.43:0/2326084409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56022abd2180 con 0x56022ab4ee80 2024-04-21T08:41:42.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.511+0000 7fd0abfff640 1 -- 172.21.15.43:0/2326084409 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0c009c490 con 0x56022ab4ee80 2024-04-21T08:41:42.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.511+0000 7fd0abfff640 1 -- 172.21.15.43:0/2326084409 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd0c00a19d0 con 0x56022ab4ee80 2024-04-21T08:41:42.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.511+0000 7fd0d5c82740 10 client.17001 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:42.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.511+0000 7fd0d5c82740 10 client.17001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:42.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.511+0000 7fd0d5c82740 20 client.17001 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:42.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.511+0000 7fd0d5c82740 1 --2- 172.21.15.43:0/2326084409 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56022abd26b0 0x56022abd4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:42.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.511+0000 7fd0d5c82740 4 client.17001 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:41:42.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.511+0000 7fd0d5c82740 1 -- 172.21.15.43:0/2326084409 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56022aac0c10 con 0x56022abd26b0 2024-04-21T08:41:42.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.512+0000 7fd0d0e28640 1 --2- 172.21.15.43:0/2326084409 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56022abd26b0 0x56022abd4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:42.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.512+0000 7fd0d0e28640 1 --2- 172.21.15.43:0/2326084409 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56022abd26b0 0x56022abd4ad0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83bd4417ee9007f7 server_cookie=e03da62edddf8d17 in_seq=0 out_seq=0 2024-04-21T08:41:42.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.512+0000 7fd0abfff640 10 client.17001 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:42.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.512+0000 7fd0abfff640 1 -- 172.21.15.43:0/2326084409 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56022aac0c10 con 0x56022abd26b0 2024-04-21T08:41:42.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.512+0000 7fd0abfff640 10 client.17001 handle_command_reply: tid=0 2024-04-21T08:41:42.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.512+0000 7fd0d5c82740 1 client.17001 shutdown 2024-04-21T08:41:42.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.513+0000 7fd0d5c82740 1 -- 172.21.15.43:0/2326084409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4ee80 msgr2=0x56022ab5b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.513+0000 7fd0d5c82740 1 --2- 172.21.15.43:0/2326084409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4ee80 0x56022ab5b2c0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00988f0 tx=0x7fd0c009bdf0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.513+0000 7fd0d5c82740 1 -- 172.21.15.43:0/2326084409 shutdown_connections 2024-04-21T08:41:42.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.513+0000 7fd0d5c82740 1 --2- 172.21.15.43:0/2326084409 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56022abd26b0 0x56022abd4ad0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.513+0000 7fd0d5c82740 1 --2- 172.21.15.43:0/2326084409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56022ab57a00 0x56022ab5b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.513+0000 7fd0abfff640 0 client.17001 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:42.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.513+0000 7fd0d5c82740 1 --2- 172.21.15.43:0/2326084409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56022ab4ee80 0x56022ab5b2c0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.513+0000 7fd0d5c82740 1 --2- 172.21.15.43:0/2326084409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56022ab4e4d0 0x56022ab5fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.513+0000 7fd0d5c82740 1 -- 172.21.15.43:0/2326084409 >> 172.21.15.43:0/2326084409 conn(0x56022ab48ba0 msgr2=0x56022ab47c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:42.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.513+0000 7fd0d5c82740 1 -- 172.21.15.43:0/2326084409 shutdown_connections 2024-04-21T08:41:42.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.513+0000 7fd0d5c82740 1 -- 172.21.15.43:0/2326084409 wait complete. 2024-04-21T08:41:42.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.513+0000 7fd0d5c82740 20 client.17001 trim_cache size 0 max 16384 2024-04-21T08:41:42.520 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:41:42.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.513+0000 7fd0d308c640 1 -- 172.21.15.43:0/481137326 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd0b40a3da0 msgr2=0x7fd0b40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.513+0000 7fd0d308c640 1 --2- 172.21.15.43:0/481137326 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd0b40a3da0 0x7fd0b40a6260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd0c40098d0 tx=0x7fd0c402cca0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.513+0000 7fd0d308c640 1 -- 172.21.15.43:0/481137326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0cc107270 msgr2=0x7fd0cc198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.513+0000 7fd0d308c640 1 --2- 172.21.15.43:0/481137326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0cc107270 0x7fd0cc198c10 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000c970 tx=0x7fd0c000cd70 comp rx=0 tx=0).stop 2024-04-21T08:41:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.513+0000 7fd0d308c640 1 -- 172.21.15.43:0/481137326 shutdown_connections 2024-04-21T08:41:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.513+0000 7fd0d308c640 1 --2- 172.21.15.43:0/481137326 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd0b40a3da0 0x7fd0b40a6260 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.513+0000 7fd0d308c640 1 --2- 172.21.15.43:0/481137326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0cc19cf40 0x7fd0cc19d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.513+0000 7fd0d308c640 1 --2- 172.21.15.43:0/481137326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0cc107270 0x7fd0cc198c10 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.513+0000 7fd0d308c640 1 --2- 172.21.15.43:0/481137326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0cc06a840 0x7fd0cc1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.513+0000 7fd0d308c640 1 -- 172.21.15.43:0/481137326 >> 172.21.15.43:0/481137326 conn(0x7fd0cc077eb0 msgr2=0x7fd0cc0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.513+0000 7fd0d308c640 1 -- 172.21.15.43:0/481137326 shutdown_connections 2024-04-21T08:41:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:42.513+0000 7fd0d308c640 1 -- 172.21.15.43:0/481137326 wait complete. 2024-04-21T08:41:42.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:42 smithi112 ceph-mon[31085]: pgmap v86: 129 pgs: 129 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-21T08:41:42.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:42 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1840436474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:43.525 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:41:43.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:41:43.525 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:41:43.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.646+0000 7f2030c06640 1 -- 172.21.15.43:0/3297536280 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202c101a40 msgr2=0x7f202c105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:43.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.646+0000 7f2030c06640 1 --2- 172.21.15.43:0/3297536280 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202c101a40 0x7f202c105a80 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f2020009900 tx=0x7f202002bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:43.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.646+0000 7f2030c06640 1 -- 172.21.15.43:0/3297536280 shutdown_connections 2024-04-21T08:41:43.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.646+0000 7f2030c06640 1 --2- 172.21.15.43:0/3297536280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f202c106160 0x7f202c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.646+0000 7f2030c06640 1 --2- 172.21.15.43:0/3297536280 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202c101a40 0x7f202c105a80 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.646+0000 7f2030c06640 1 --2- 172.21.15.43:0/3297536280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f202c101090 0x7f202c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.646+0000 7f2030c06640 1 -- 172.21.15.43:0/3297536280 >> 172.21.15.43:0/3297536280 conn(0x7f202c077f50 msgr2=0x7f202c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:43.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.646+0000 7f2030c06640 1 -- 172.21.15.43:0/3297536280 shutdown_connections 2024-04-21T08:41:43.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.646+0000 7f2030c06640 1 -- 172.21.15.43:0/3297536280 wait complete. 2024-04-21T08:41:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.647+0000 7f2030c06640 1 Processor -- start 2024-04-21T08:41:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.647+0000 7f2030c06640 1 -- start start 2024-04-21T08:41:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.647+0000 7f2030c06640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f202c101090 0x7f202c199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.647+0000 7f2030c06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202c106160 0x7f202c199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.647+0000 7f2030c06640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f202c19a010 0x7f202c1a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.647+0000 7f2030c06640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f202c111100 con 0x7f202c106160 2024-04-21T08:41:43.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.647+0000 7f2030c06640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f202c110f80 con 0x7f202c101090 2024-04-21T08:41:43.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.647+0000 7f2030c06640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f202c111280 con 0x7f202c19a010 2024-04-21T08:41:43.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.647+0000 7f2029d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202c106160 0x7f202c199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.647+0000 7f2029d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202c106160 0x7f202c199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34286/0 (socket says 172.21.15.43:34286) 2024-04-21T08:41:43.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.647+0000 7f2029d9b640 1 -- 172.21.15.43:0/4268237057 learned_addr learned my addr 172.21.15.43:0/4268237057 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:43.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.647+0000 7f202a59c640 1 --2- 172.21.15.43:0/4268237057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f202c101090 0x7f202c199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.647+0000 7f202ad9d640 1 --2- 172.21.15.43:0/4268237057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f202c19a010 0x7f202c1a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.647+0000 7f2029d9b640 1 -- 172.21.15.43:0/4268237057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f202c19a010 msgr2=0x7f202c1a45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:43.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.647+0000 7f2029d9b640 1 --2- 172.21.15.43:0/4268237057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f202c19a010 0x7f202c1a45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.647+0000 7f2029d9b640 1 -- 172.21.15.43:0/4268237057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f202c101090 msgr2=0x7f202c199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:43.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.647+0000 7f2029d9b640 1 --2- 172.21.15.43:0/4268237057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f202c101090 0x7f202c199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.647+0000 7f2029d9b640 1 -- 172.21.15.43:0/4268237057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202c1a4d90 con 0x7f202c106160 2024-04-21T08:41:43.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.647+0000 7f202ad9d640 1 --2- 172.21.15.43:0/4268237057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f202c19a010 0x7f202c1a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:43.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.648+0000 7f2029d9b640 1 --2- 172.21.15.43:0/4268237057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202c106160 0x7f202c199a80 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f2020005ec0 tx=0x7f202002ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:43.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.648+0000 7f201b7fe640 1 -- 172.21.15.43:0/4268237057 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2020043020 con 0x7f202c106160 2024-04-21T08:41:43.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.648+0000 7f201b7fe640 1 -- 172.21.15.43:0/4268237057 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20200052b0 con 0x7f202c106160 2024-04-21T08:41:43.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.648+0000 7f2030c06640 1 -- 172.21.15.43:0/4268237057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f202c1a5020 con 0x7f202c106160 2024-04-21T08:41:43.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.648+0000 7f201b7fe640 1 -- 172.21.15.43:0/4268237057 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2020003c60 con 0x7f202c106160 2024-04-21T08:41:43.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.648+0000 7f2030c06640 1 -- 172.21.15.43:0/4268237057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f202c1a5570 con 0x7f202c106160 2024-04-21T08:41:43.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.650+0000 7f201b7fe640 1 -- 172.21.15.43:0/4268237057 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2020046020 con 0x7f202c106160 2024-04-21T08:41:43.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.650+0000 7f20337f4740 1 Processor -- start 2024-04-21T08:41:43.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.651+0000 7f20337f4740 1 -- start start 2024-04-21T08:41:43.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.651+0000 7f20337f4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56335e5067c0 0x56335e506ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.651+0000 7f20337f4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335e507170 0x56335e50fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.651+0000 7f201b7fe640 1 --2- 172.21.15.43:0/4268237057 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f20140a3d20 0x7f20140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.651+0000 7f20337f4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e510270 0x56335e5106d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.651+0000 7f20337f4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56335e46af00 con 0x56335e510270 2024-04-21T08:41:43.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.651+0000 7f20337f4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56335e408590 con 0x56335e507170 2024-04-21T08:41:43.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.651+0000 7f20337f4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56335e466ec0 con 0x56335e5067c0 2024-04-21T08:41:43.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.651+0000 7f2029d9b640 1 --2- 172.21.15.43:0/4268237057 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f20140a3d20 0x7f20140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.651+0000 7f202ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e510270 0x56335e5106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.651+0000 7f202ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e510270 0x56335e5106d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34300/0 (socket says 172.21.15.43:34300) 2024-04-21T08:41:43.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.651+0000 7f202ad9d640 1 -- 172.21.15.43:0/3081413133 learned_addr learned my addr 172.21.15.43:0/3081413133 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:43.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.651+0000 7f202ad9d640 1 --2- 172.21.15.43:0/3081413133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56335e5067c0 0x56335e506ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.651+0000 7f202a59c640 1 --2- 172.21.15.43:0/3081413133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335e507170 0x56335e50fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.651+0000 7f201b7fe640 1 -- 172.21.15.43:0/4268237057 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f20200f2e70 con 0x7f202c106160 2024-04-21T08:41:43.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.651+0000 7f202ad9d640 1 -- 172.21.15.43:0/3081413133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56335e5067c0 msgr2=0x56335e506ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:43.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.651+0000 7f202ad9d640 1 --2- 172.21.15.43:0/3081413133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56335e5067c0 0x56335e506ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.651+0000 7f202ad9d640 1 -- 172.21.15.43:0/3081413133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335e507170 msgr2=0x56335e50fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:43.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.651+0000 7f202ad9d640 1 --2- 172.21.15.43:0/3081413133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335e507170 0x56335e50fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.652+0000 7f202ad9d640 1 -- 172.21.15.43:0/3081413133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56335e2457e0 con 0x56335e510270 2024-04-21T08:41:43.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.652+0000 7f2029d9b640 1 --2- 172.21.15.43:0/4268237057 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f20140a3d20 0x7f20140a61e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f202004fbc0 tx=0x7f2020090530 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:43.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.652+0000 7f202ad9d640 1 --2- 172.21.15.43:0/3081413133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e510270 0x56335e5106d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f201c00acb0 tx=0x7f201c009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17590a275027eb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:43.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.652+0000 7f2018ff9640 1 -- 172.21.15.43:0/3081413133 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c013040 con 0x56335e510270 2024-04-21T08:41:43.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.652+0000 7f2018ff9640 1 -- 172.21.15.43:0/3081413133 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f201c01a430 con 0x56335e510270 2024-04-21T08:41:43.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.652+0000 7f2018ff9640 1 -- 172.21.15.43:0/3081413133 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c019640 con 0x56335e510270 2024-04-21T08:41:43.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.653+0000 7f20337f4740 1 -- 172.21.15.43:0/3081413133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e510270 msgr2=0x56335e5106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:43.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.653+0000 7f20337f4740 1 --2- 172.21.15.43:0/3081413133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e510270 0x56335e5106d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f201c00acb0 tx=0x7f201c009850 comp rx=0 tx=0).stop 2024-04-21T08:41:43.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.653+0000 7f20337f4740 1 -- 172.21.15.43:0/3081413133 shutdown_connections 2024-04-21T08:41:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.653+0000 7f20337f4740 1 --2- 172.21.15.43:0/3081413133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e510270 0x56335e5106d0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.653+0000 7f20337f4740 1 --2- 172.21.15.43:0/3081413133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335e507170 0x56335e50fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.653+0000 7f20337f4740 1 --2- 172.21.15.43:0/3081413133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56335e5067c0 0x56335e506ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.653+0000 7f20337f4740 1 -- 172.21.15.43:0/3081413133 >> 172.21.15.43:0/3081413133 conn(0x56335e500fb0 msgr2=0x56335e501390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.653+0000 7f20337f4740 1 -- 172.21.15.43:0/3081413133 shutdown_connections 2024-04-21T08:41:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.653+0000 7f20337f4740 1 -- 172.21.15.43:0/3081413133 wait complete. 2024-04-21T08:41:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.653+0000 7f20337f4740 1 -- start start 2024-04-21T08:41:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.653+0000 7f20337f4740 1 Processor -- start 2024-04-21T08:41:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.653+0000 7f20337f4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e5067c0 0x56335e513d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.653+0000 7f20337f4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335e507170 0x56335e514240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.653+0000 7f20337f4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56335e510270 0x56335e514780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.653+0000 7f20337f4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56335e517150 con 0x56335e5067c0 2024-04-21T08:41:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.653+0000 7f20337f4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56335e424580 con 0x56335e507170 2024-04-21T08:41:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.653+0000 7f20337f4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56335e5172d0 con 0x56335e510270 2024-04-21T08:41:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.653+0000 7f20337f4740 10 client.0 _ll_register_callbacks cb 0x56335e42a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:43.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.654+0000 7f202ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e5067c0 0x56335e513d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.654+0000 7f202ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e5067c0 0x56335e513d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34306/0 (socket says 172.21.15.43:34306) 2024-04-21T08:41:43.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.654+0000 7f202ad9d640 1 -- 172.21.15.43:0/4100954633 learned_addr learned my addr 172.21.15.43:0/4100954633 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:43.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.654+0000 7f202a59c640 1 --2- 172.21.15.43:0/4100954633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335e507170 0x56335e514240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.654+0000 7f202ad9d640 1 --2- 172.21.15.43:0/4100954633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56335e510270 0x56335e514780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.654+0000 7f202ad9d640 1 -- 172.21.15.43:0/4100954633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56335e510270 msgr2=0x56335e514780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:43.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.654+0000 7f202ad9d640 1 --2- 172.21.15.43:0/4100954633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56335e510270 0x56335e514780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.654+0000 7f202ad9d640 1 -- 172.21.15.43:0/4100954633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335e507170 msgr2=0x56335e514240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:43.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.654+0000 7f202ad9d640 1 --2- 172.21.15.43:0/4100954633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335e507170 0x56335e514240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.654+0000 7f202ad9d640 1 -- 172.21.15.43:0/4100954633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56335e2a5f70 con 0x56335e5067c0 2024-04-21T08:41:43.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.654+0000 7f202ad9d640 1 --2- 172.21.15.43:0/4100954633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e5067c0 0x56335e513d00 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f201c00afd0 tx=0x7f201c01ae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabd34b4ecad3bc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:43.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.654+0000 7f2011ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:43.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.654+0000 7f2011ffb640 1 -- 172.21.15.43:0/4100954633 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c02d060 con 0x56335e5067c0 2024-04-21T08:41:43.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.654+0000 7f2011ffb640 1 -- 172.21.15.43:0/4100954633 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f201c01a7f0 con 0x56335e5067c0 2024-04-21T08:41:43.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.654+0000 7f20337f4740 1 -- 172.21.15.43:0/4100954633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f201c01a430 con 0x56335e5067c0 2024-04-21T08:41:43.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.654+0000 7f2011ffb640 1 -- 172.21.15.43:0/4100954633 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f201c014a80 con 0x56335e5067c0 2024-04-21T08:41:43.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.654+0000 7f20337f4740 10 client.17013 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:43.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.654+0000 7f20337f4740 1 -- 172.21.15.43:0/4100954633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56335e13d170 con 0x56335e5067c0 2024-04-21T08:41:43.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.655+0000 7f2011ffb640 1 -- 172.21.15.43:0/4100954633 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f201c01a430 con 0x56335e5067c0 2024-04-21T08:41:43.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.655+0000 7f2011ffb640 1 -- 172.21.15.43:0/4100954633 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f201c026040 con 0x56335e5067c0 2024-04-21T08:41:43.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.655+0000 7f20337f4740 10 client.17013 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:43.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.655+0000 7f20337f4740 10 client.17013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:43.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.655+0000 7f20337f4740 20 client.17013 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:43.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.655+0000 7f20337f4740 1 --2- 172.21.15.43:0/4100954633 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56335e524610 0x56335e526a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.655+0000 7f20337f4740 4 client.17013 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:43.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.655+0000 7f20337f4740 1 -- 172.21.15.43:0/4100954633 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56335e478920 con 0x56335e524610 2024-04-21T08:41:43.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.655+0000 7f202a59c640 1 --2- 172.21.15.43:0/4100954633 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56335e524610 0x56335e526a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.656+0000 7f202a59c640 1 --2- 172.21.15.43:0/4100954633 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56335e524610 0x56335e526a00 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ceacdeac3408f4f server_cookie=b888d5a95ce649f9 in_seq=0 out_seq=0 2024-04-21T08:41:43.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.656+0000 7f2011ffb640 10 client.17013 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:43.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.656+0000 7f2011ffb640 1 -- 172.21.15.43:0/4100954633 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56335e478920 con 0x56335e524610 2024-04-21T08:41:43.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.656+0000 7f2011ffb640 10 client.17013 handle_command_reply: tid=0 2024-04-21T08:41:43.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.656+0000 7f20337f4740 1 client.17013 shutdown 2024-04-21T08:41:43.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.657+0000 7f20337f4740 1 -- 172.21.15.43:0/4100954633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e5067c0 msgr2=0x56335e513d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:43.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.657+0000 7f20337f4740 1 --2- 172.21.15.43:0/4100954633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e5067c0 0x56335e513d00 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f201c00afd0 tx=0x7f201c01ae10 comp rx=0 tx=0).stop 2024-04-21T08:41:43.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.657+0000 7f20337f4740 1 -- 172.21.15.43:0/4100954633 shutdown_connections 2024-04-21T08:41:43.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.657+0000 7f20337f4740 1 --2- 172.21.15.43:0/4100954633 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56335e524610 0x56335e526a00 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.657+0000 7f20337f4740 1 --2- 172.21.15.43:0/4100954633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56335e510270 0x56335e514780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.657+0000 7f20337f4740 1 --2- 172.21.15.43:0/4100954633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335e507170 0x56335e514240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.657+0000 7f20337f4740 1 --2- 172.21.15.43:0/4100954633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e5067c0 0x56335e513d00 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.657+0000 7f20337f4740 1 -- 172.21.15.43:0/4100954633 >> 172.21.15.43:0/4100954633 conn(0x56335e500fb0 msgr2=0x56335e502d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:43.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.657+0000 7f2011ffb640 0 client.17013 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:43.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.657+0000 7f20337f4740 1 -- 172.21.15.43:0/4100954633 shutdown_connections 2024-04-21T08:41:43.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.657+0000 7f20337f4740 1 -- 172.21.15.43:0/4100954633 wait complete. 2024-04-21T08:41:43.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.657+0000 7f20337f4740 20 client.17013 trim_cache size 0 max 16384 2024-04-21T08:41:43.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.674+0000 7f20337f4740 1 Processor -- start 2024-04-21T08:41:43.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.674+0000 7f20337f4740 1 -- start start 2024-04-21T08:41:43.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.675+0000 7f20337f4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56335e5067c0 0x56335e516430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.675+0000 7f20337f4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335e507170 0x56335e520080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.675+0000 7f20337f4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e510270 0x56335e511680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.675+0000 7f20337f4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f202c1ad160 con 0x56335e510270 2024-04-21T08:41:43.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.675+0000 7f20337f4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f202c1aeca0 con 0x56335e507170 2024-04-21T08:41:43.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.675+0000 7f20337f4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f202c1ad830 con 0x56335e5067c0 2024-04-21T08:41:43.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.675+0000 7f202ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e510270 0x56335e511680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.675+0000 7f202ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e510270 0x56335e511680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34318/0 (socket says 172.21.15.43:34318) 2024-04-21T08:41:43.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.675+0000 7f202ad9d640 1 -- 172.21.15.43:0/2258659958 learned_addr learned my addr 172.21.15.43:0/2258659958 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:43.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.675+0000 7f202a59c640 1 --2- 172.21.15.43:0/2258659958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335e507170 0x56335e520080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.675+0000 7f202ad9d640 1 -- 172.21.15.43:0/2258659958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56335e5067c0 msgr2=0x56335e516430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:43.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.675+0000 7f202ad9d640 1 --2- 172.21.15.43:0/2258659958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56335e5067c0 0x56335e516430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.675+0000 7f202ad9d640 1 -- 172.21.15.43:0/2258659958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335e507170 msgr2=0x56335e520080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:43.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.675+0000 7f202ad9d640 1 --2- 172.21.15.43:0/2258659958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335e507170 0x56335e520080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.675+0000 7f202ad9d640 1 -- 172.21.15.43:0/2258659958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56335e506f10 con 0x56335e510270 2024-04-21T08:41:43.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.675+0000 7f202a59c640 1 --2- 172.21.15.43:0/2258659958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335e507170 0x56335e520080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:43.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.675+0000 7f202ad9d640 1 --2- 172.21.15.43:0/2258659958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e510270 0x56335e511680 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f201c02b040 tx=0x7f201c028f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8377d43a523f5929 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:43.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.675+0000 7f2013fff640 1 -- 172.21.15.43:0/2258659958 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c02d070 con 0x56335e510270 2024-04-21T08:41:43.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.676+0000 7f2013fff640 1 -- 172.21.15.43:0/2258659958 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f201c014c20 con 0x56335e510270 2024-04-21T08:41:43.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.676+0000 7f2013fff640 1 -- 172.21.15.43:0/2258659958 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c0286c0 con 0x56335e510270 2024-04-21T08:41:43.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.676+0000 7f20337f4740 1 -- 172.21.15.43:0/2258659958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e510270 msgr2=0x56335e511680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:43.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.676+0000 7f20337f4740 1 --2- 172.21.15.43:0/2258659958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e510270 0x56335e511680 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f201c02b040 tx=0x7f201c028f10 comp rx=0 tx=0).stop 2024-04-21T08:41:43.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.676+0000 7f20337f4740 1 -- 172.21.15.43:0/2258659958 shutdown_connections 2024-04-21T08:41:43.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.676+0000 7f20337f4740 1 --2- 172.21.15.43:0/2258659958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e510270 0x56335e511680 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.676+0000 7f20337f4740 1 --2- 172.21.15.43:0/2258659958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335e507170 0x56335e520080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.676+0000 7f20337f4740 1 --2- 172.21.15.43:0/2258659958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56335e5067c0 0x56335e516430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.676+0000 7f20337f4740 1 -- 172.21.15.43:0/2258659958 >> 172.21.15.43:0/2258659958 conn(0x56335e500fb0 msgr2=0x56335e512d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:43.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.676+0000 7f20337f4740 1 -- 172.21.15.43:0/2258659958 shutdown_connections 2024-04-21T08:41:43.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.676+0000 7f20337f4740 1 -- 172.21.15.43:0/2258659958 wait complete. 2024-04-21T08:41:43.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.676+0000 7f20337f4740 1 -- start start 2024-04-21T08:41:43.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.676+0000 7f20337f4740 1 Processor -- start 2024-04-21T08:41:43.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.677+0000 7f20337f4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335e5067c0 0x56335e51c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.677+0000 7f20337f4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56335e507170 0x56335e51cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.677+0000 7f20337f4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e510270 0x56335e50cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.677+0000 7f20337f4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f202c1ae5d0 con 0x56335e510270 2024-04-21T08:41:43.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.677+0000 7f20337f4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f202c1adf00 con 0x56335e5067c0 2024-04-21T08:41:43.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.677+0000 7f20337f4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f202c1ab140 con 0x56335e507170 2024-04-21T08:41:43.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.677+0000 7f20337f4740 10 client.0 _ll_register_callbacks cb 0x7f202400a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:43.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.677+0000 7f202ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e510270 0x56335e50cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.677+0000 7f202ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e510270 0x56335e50cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34328/0 (socket says 172.21.15.43:34328) 2024-04-21T08:41:43.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.677+0000 7f202ad9d640 1 -- 172.21.15.43:0/4208059466 learned_addr learned my addr 172.21.15.43:0/4208059466 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:43.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.677+0000 7f202ad9d640 1 --2- 172.21.15.43:0/4208059466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335e5067c0 0x56335e51c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.677+0000 7f202a59c640 1 --2- 172.21.15.43:0/4208059466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56335e507170 0x56335e51cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.677+0000 7f202ad9d640 1 -- 172.21.15.43:0/4208059466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56335e507170 msgr2=0x56335e51cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:43.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.677+0000 7f202ad9d640 1 --2- 172.21.15.43:0/4208059466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56335e507170 0x56335e51cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.677+0000 7f202ad9d640 1 -- 172.21.15.43:0/4208059466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335e5067c0 msgr2=0x56335e51c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:43.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.677+0000 7f202ad9d640 1 --2- 172.21.15.43:0/4208059466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335e5067c0 0x56335e51c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.677+0000 7f202ad9d640 1 -- 172.21.15.43:0/4208059466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56335e50d5b0 con 0x56335e510270 2024-04-21T08:41:43.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.677+0000 7f202ad9d640 1 --2- 172.21.15.43:0/4208059466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e510270 0x56335e50cdb0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f201c028c20 tx=0x7f201c007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49dffc3af8d3f66 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:43.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.677+0000 7f2011ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:43.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.677+0000 7f2011ffb640 1 -- 172.21.15.43:0/4208059466 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c02d070 con 0x56335e510270 2024-04-21T08:41:43.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.677+0000 7f2011ffb640 1 -- 172.21.15.43:0/4208059466 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f201c00a4f0 con 0x56335e510270 2024-04-21T08:41:43.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.677+0000 7f20337f4740 1 -- 172.21.15.43:0/4208059466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f201c014c20 con 0x56335e510270 2024-04-21T08:41:43.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.678+0000 7f2011ffb640 1 -- 172.21.15.43:0/4208059466 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f201c01a430 con 0x56335e510270 2024-04-21T08:41:43.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.678+0000 7f20337f4740 10 client.17019 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:43.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.678+0000 7f20337f4740 1 -- 172.21.15.43:0/4208059466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56335e58a960 con 0x56335e510270 2024-04-21T08:41:43.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.678+0000 7f2011ffb640 1 -- 172.21.15.43:0/4208059466 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f201c014c20 con 0x56335e510270 2024-04-21T08:41:43.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.678+0000 7f2011ffb640 1 -- 172.21.15.43:0/4208059466 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f201c025830 con 0x56335e510270 2024-04-21T08:41:43.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.678+0000 7f20337f4740 10 client.17019 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:43.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.678+0000 7f20337f4740 10 client.17019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:43.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.678+0000 7f20337f4740 20 client.17019 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:43.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.678+0000 7f20337f4740 1 --2- 172.21.15.43:0/4208059466 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56335e58ac50 0x56335e58d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.678+0000 7f20337f4740 4 client.17019 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:41:43.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.678+0000 7f20337f4740 1 -- 172.21.15.43:0/4208059466 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56335e472790 con 0x56335e58ac50 2024-04-21T08:41:43.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.678+0000 7f202a59c640 1 --2- 172.21.15.43:0/4208059466 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56335e58ac50 0x56335e58d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.679+0000 7f202a59c640 1 --2- 172.21.15.43:0/4208059466 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56335e58ac50 0x56335e58d040 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=488dfb23a08f13c8 server_cookie=36039f25777cec0b in_seq=0 out_seq=0 2024-04-21T08:41:43.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.679+0000 7f2011ffb640 10 client.17019 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:43.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.680+0000 7f2011ffb640 1 -- 172.21.15.43:0/4208059466 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56335e472790 con 0x56335e58ac50 2024-04-21T08:41:43.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.680+0000 7f2011ffb640 10 client.17019 handle_command_reply: tid=0 2024-04-21T08:41:43.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.680+0000 7f20337f4740 1 client.17019 shutdown 2024-04-21T08:41:43.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.680+0000 7f20337f4740 1 -- 172.21.15.43:0/4208059466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e510270 msgr2=0x56335e50cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:43.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.680+0000 7f20337f4740 1 --2- 172.21.15.43:0/4208059466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e510270 0x56335e50cdb0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f201c028c20 tx=0x7f201c007000 comp rx=0 tx=0).stop 2024-04-21T08:41:43.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.680+0000 7f20337f4740 1 -- 172.21.15.43:0/4208059466 shutdown_connections 2024-04-21T08:41:43.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.680+0000 7f20337f4740 1 --2- 172.21.15.43:0/4208059466 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56335e58ac50 0x56335e58d040 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.680+0000 7f20337f4740 1 --2- 172.21.15.43:0/4208059466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335e510270 0x56335e50cdb0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.680+0000 7f20337f4740 1 --2- 172.21.15.43:0/4208059466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56335e507170 0x56335e51cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.680+0000 7f20337f4740 1 --2- 172.21.15.43:0/4208059466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56335e5067c0 0x56335e51c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.680+0000 7f20337f4740 1 -- 172.21.15.43:0/4208059466 >> 172.21.15.43:0/4208059466 conn(0x56335e500fb0 msgr2=0x56335e513cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:43.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.680+0000 7f2011ffb640 0 client.17019 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:43.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.680+0000 7f20337f4740 1 -- 172.21.15.43:0/4208059466 shutdown_connections 2024-04-21T08:41:43.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.680+0000 7f20337f4740 1 -- 172.21.15.43:0/4208059466 wait complete. 2024-04-21T08:41:43.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.680+0000 7f20337f4740 20 client.17019 trim_cache size 0 max 16384 2024-04-21T08:41:43.687 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:41:43.687 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:41:43.687 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "7e48e4aa-7bff-4abe-b100-44c8949935f2", 2024-04-21T08:41:43.687 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:41:43.687 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:41:43.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.681+0000 7f2030c06640 1 -- 172.21.15.43:0/4268237057 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f20140a3d20 msgr2=0x7f20140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:43.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.681+0000 7f2030c06640 1 --2- 172.21.15.43:0/4268237057 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f20140a3d20 0x7f20140a61e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f202004fbc0 tx=0x7f2020090530 comp rx=0 tx=0).stop 2024-04-21T08:41:43.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.681+0000 7f2030c06640 1 -- 172.21.15.43:0/4268237057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202c106160 msgr2=0x7f202c199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:43.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.681+0000 7f2030c06640 1 --2- 172.21.15.43:0/4268237057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202c106160 0x7f202c199a80 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f2020005ec0 tx=0x7f202002ca10 comp rx=0 tx=0).stop 2024-04-21T08:41:43.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.681+0000 7f2030c06640 1 -- 172.21.15.43:0/4268237057 shutdown_connections 2024-04-21T08:41:43.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.681+0000 7f2030c06640 1 --2- 172.21.15.43:0/4268237057 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f20140a3d20 0x7f20140a61e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.681+0000 7f2030c06640 1 --2- 172.21.15.43:0/4268237057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f202c19a010 0x7f202c1a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.681+0000 7f2030c06640 1 --2- 172.21.15.43:0/4268237057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202c106160 0x7f202c199a80 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.681+0000 7f2030c06640 1 --2- 172.21.15.43:0/4268237057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f202c101090 0x7f202c199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.681+0000 7f2030c06640 1 -- 172.21.15.43:0/4268237057 >> 172.21.15.43:0/4268237057 conn(0x7f202c077f50 msgr2=0x7f202c0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:43.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.681+0000 7f2030c06640 1 -- 172.21.15.43:0/4268237057 shutdown_connections 2024-04-21T08:41:43.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.681+0000 7f2030c06640 1 -- 172.21.15.43:0/4268237057 wait complete. 2024-04-21T08:41:43.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:41:43.691 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:41:43.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.797+0000 7f7c79b11640 1 -- 172.21.15.43:0/2860169139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c74101080 msgr2=0x7f7c74101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:43.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.797+0000 7f7c79b11640 1 --2- 172.21.15.43:0/2860169139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c74101080 0x7f7c74101460 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009900 tx=0x7f7c6402bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:43.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.797+0000 7f7c79b11640 1 -- 172.21.15.43:0/2860169139 shutdown_connections 2024-04-21T08:41:43.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.797+0000 7f7c79b11640 1 --2- 172.21.15.43:0/2860169139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c74106150 0x7f7c7410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.797+0000 7f7c79b11640 1 --2- 172.21.15.43:0/2860169139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c74101a30 0x7f7c74105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.797+0000 7f7c79b11640 1 --2- 172.21.15.43:0/2860169139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c74101080 0x7f7c74101460 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.797+0000 7f7c79b11640 1 -- 172.21.15.43:0/2860169139 >> 172.21.15.43:0/2860169139 conn(0x7f7c74078000 msgr2=0x7f7c74078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:43.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.797+0000 7f7c79b11640 1 -- 172.21.15.43:0/2860169139 shutdown_connections 2024-04-21T08:41:43.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.797+0000 7f7c79b11640 1 -- 172.21.15.43:0/2860169139 wait complete. 2024-04-21T08:41:43.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.797+0000 7f7c79b11640 1 Processor -- start 2024-04-21T08:41:43.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.797+0000 7f7c79b11640 1 -- start start 2024-04-21T08:41:43.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.798+0000 7f7c79b11640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c74101080 0x7f7c74198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.798+0000 7f7c79b11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c74106150 0x7f7c74198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.798+0000 7f7c79b11640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c7419d000 0x7f7c7419d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.798+0000 7f7c79b11640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7c741110b0 con 0x7f7c74106150 2024-04-21T08:41:43.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.798+0000 7f7c79b11640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7c74110f30 con 0x7f7c7419d000 2024-04-21T08:41:43.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.798+0000 7f7c79b11640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7c74111230 con 0x7f7c74101080 2024-04-21T08:41:43.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.798+0000 7f7c72ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c74106150 0x7f7c74198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.798+0000 7f7c72ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c74106150 0x7f7c74198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34346/0 (socket says 172.21.15.43:34346) 2024-04-21T08:41:43.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.798+0000 7f7c72ffd640 1 -- 172.21.15.43:0/3264109023 learned_addr learned my addr 172.21.15.43:0/3264109023 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:43.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.798+0000 7f7c737fe640 1 --2- 172.21.15.43:0/3264109023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c74101080 0x7f7c74198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.798+0000 7f7c73fff640 1 --2- 172.21.15.43:0/3264109023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c7419d000 0x7f7c7419d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.798+0000 7f7c72ffd640 1 -- 172.21.15.43:0/3264109023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c74101080 msgr2=0x7f7c74198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:43.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.798+0000 7f7c72ffd640 1 --2- 172.21.15.43:0/3264109023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c74101080 0x7f7c74198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.798+0000 7f7c72ffd640 1 -- 172.21.15.43:0/3264109023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c7419d000 msgr2=0x7f7c7419d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:43.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.798+0000 7f7c72ffd640 1 --2- 172.21.15.43:0/3264109023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c7419d000 0x7f7c7419d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.798+0000 7f7c72ffd640 1 -- 172.21.15.43:0/3264109023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c7419daf0 con 0x7f7c74106150 2024-04-21T08:41:43.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.798+0000 7f7c737fe640 1 --2- 172.21.15.43:0/3264109023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c74101080 0x7f7c74198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:43.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.798+0000 7f7c72ffd640 1 --2- 172.21.15.43:0/3264109023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c74106150 0x7f7c74198c80 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800e9f0 tx=0x7f7c6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:43.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.798+0000 7f7c70ff9640 1 -- 172.21.15.43:0/3264109023 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6800ccc0 con 0x7f7c74106150 2024-04-21T08:41:43.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.798+0000 7f7c70ff9640 1 -- 172.21.15.43:0/3264109023 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c680054d0 con 0x7f7c74106150 2024-04-21T08:41:43.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.798+0000 7f7c79b11640 1 -- 172.21.15.43:0/3264109023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c7419dde0 con 0x7f7c74106150 2024-04-21T08:41:43.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.798+0000 7f7c70ff9640 1 -- 172.21.15.43:0/3264109023 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c68010430 con 0x7f7c74106150 2024-04-21T08:41:43.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.798+0000 7f7c79b11640 1 -- 172.21.15.43:0/3264109023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c741a5580 con 0x7f7c74106150 2024-04-21T08:41:43.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.801+0000 7f7c70ff9640 1 -- 172.21.15.43:0/3264109023 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7c680105d0 con 0x7f7c74106150 2024-04-21T08:41:43.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.801+0000 7f7c7c7d6740 1 Processor -- start 2024-04-21T08:41:43.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.801+0000 7f7c7c7d6740 1 -- start start 2024-04-21T08:41:43.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.801+0000 7f7c7c7d6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56239a133990 0x56239a133d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.801+0000 7f7c7c7d6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56239a1342b0 0x56239a13c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.801+0000 7f7c7c7d6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a13ce90 0x56239a13f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.801+0000 7f7c7c7d6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56239a0941e0 con 0x56239a13ce90 2024-04-21T08:41:43.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.801+0000 7f7c7c7d6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56239a031390 con 0x56239a1342b0 2024-04-21T08:41:43.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.801+0000 7f7c7c7d6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56239a0901a0 con 0x56239a133990 2024-04-21T08:41:43.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.802+0000 7f7c72ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a13ce90 0x56239a13f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.802+0000 7f7c72ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a13ce90 0x56239a13f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34352/0 (socket says 172.21.15.43:34352) 2024-04-21T08:41:43.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.802+0000 7f7c72ffd640 1 -- 172.21.15.43:0/1246986580 learned_addr learned my addr 172.21.15.43:0/1246986580 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:43.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.802+0000 7f7c737fe640 1 --2- 172.21.15.43:0/1246986580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56239a1342b0 0x56239a13c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.802+0000 7f7c70ff9640 1 --2- 172.21.15.43:0/3264109023 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7c5c0a3d20 0x7f7c5c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.802+0000 7f7c73fff640 1 --2- 172.21.15.43:0/1246986580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56239a133990 0x56239a133d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.802+0000 7f7c72ffd640 1 -- 172.21.15.43:0/1246986580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56239a133990 msgr2=0x56239a133d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:43.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.802+0000 7f7c72ffd640 1 --2- 172.21.15.43:0/1246986580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56239a133990 0x56239a133d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.802+0000 7f7c72ffd640 1 -- 172.21.15.43:0/1246986580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56239a1342b0 msgr2=0x56239a13c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:43.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.802+0000 7f7c72ffd640 1 --2- 172.21.15.43:0/1246986580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56239a1342b0 0x56239a13c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.802+0000 7f7c72ffd640 1 -- 172.21.15.43:0/1246986580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562399e707d0 con 0x56239a13ce90 2024-04-21T08:41:43.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.802+0000 7f7c73fff640 1 --2- 172.21.15.43:0/1246986580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56239a133990 0x56239a133d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:43.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.802+0000 7f7c73fff640 1 --2- 172.21.15.43:0/3264109023 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7c5c0a3d20 0x7f7c5c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.802+0000 7f7c70ff9640 1 -- 172.21.15.43:0/3264109023 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f7c680d0fc0 con 0x7f7c74106150 2024-04-21T08:41:43.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.802+0000 7f7c72ffd640 1 --2- 172.21.15.43:0/1246986580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a13ce90 0x56239a13f280 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f7c680ce2b0 tx=0x7f7c68067190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91113eb3c251631 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:43.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.802+0000 7f7c59ffb640 1 -- 172.21.15.43:0/1246986580 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6809c6d0 con 0x56239a13ce90 2024-04-21T08:41:43.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.802+0000 7f7c73fff640 1 --2- 172.21.15.43:0/3264109023 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7c5c0a3d20 0x7f7c5c0a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c005fd0 tx=0x7f7c6c004a40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:43.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.802+0000 7f7c59ffb640 1 -- 172.21.15.43:0/1246986580 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c68005680 con 0x56239a13ce90 2024-04-21T08:41:43.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.803+0000 7f7c59ffb640 1 -- 172.21.15.43:0/1246986580 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6807f9c0 con 0x56239a13ce90 2024-04-21T08:41:43.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.803+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/1246986580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a13ce90 msgr2=0x56239a13f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:43.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.803+0000 7f7c7c7d6740 1 --2- 172.21.15.43:0/1246986580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a13ce90 0x56239a13f280 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f7c680ce2b0 tx=0x7f7c68067190 comp rx=0 tx=0).stop 2024-04-21T08:41:43.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.803+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/1246986580 shutdown_connections 2024-04-21T08:41:43.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.803+0000 7f7c7c7d6740 1 --2- 172.21.15.43:0/1246986580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a13ce90 0x56239a13f280 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.803+0000 7f7c7c7d6740 1 --2- 172.21.15.43:0/1246986580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56239a1342b0 0x56239a13c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.803+0000 7f7c7c7d6740 1 --2- 172.21.15.43:0/1246986580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56239a133990 0x56239a133d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.803+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/1246986580 >> 172.21.15.43:0/1246986580 conn(0x56239a12e0c0 msgr2=0x56239a12e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:43.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.803+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/1246986580 shutdown_connections 2024-04-21T08:41:43.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.803+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/1246986580 wait complete. 2024-04-21T08:41:43.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.803+0000 7f7c7c7d6740 1 -- start start 2024-04-21T08:41:43.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.804+0000 7f7c7c7d6740 1 Processor -- start 2024-04-21T08:41:43.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.804+0000 7f7c7c7d6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a133990 0x56239a139ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.804+0000 7f7c7c7d6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56239a1342b0 0x56239a13a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.804+0000 7f7c7c7d6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56239a13ce90 0x56239a13a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.804+0000 7f7c7c7d6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56239a13fc20 con 0x56239a133990 2024-04-21T08:41:43.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.804+0000 7f7c7c7d6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56239a04d2f0 con 0x56239a1342b0 2024-04-21T08:41:43.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.804+0000 7f7c7c7d6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56239a13fda0 con 0x56239a13ce90 2024-04-21T08:41:43.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.804+0000 7f7c7c7d6740 10 client.0 _ll_register_callbacks cb 0x56239a05a580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:43.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.804+0000 7f7c737fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a133990 0x56239a139ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.804+0000 7f7c737fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a133990 0x56239a139ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34354/0 (socket says 172.21.15.43:34354) 2024-04-21T08:41:43.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.804+0000 7f7c737fe640 1 -- 172.21.15.43:0/2094458137 learned_addr learned my addr 172.21.15.43:0/2094458137 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:43.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.804+0000 7f7c72ffd640 1 --2- 172.21.15.43:0/2094458137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56239a1342b0 0x56239a13a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.804+0000 7f7c737fe640 1 -- 172.21.15.43:0/2094458137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56239a13ce90 msgr2=0x56239a13a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:43.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.804+0000 7f7c737fe640 1 --2- 172.21.15.43:0/2094458137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56239a13ce90 0x56239a13a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.804+0000 7f7c737fe640 1 -- 172.21.15.43:0/2094458137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56239a1342b0 msgr2=0x56239a13a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:43.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.804+0000 7f7c737fe640 1 --2- 172.21.15.43:0/2094458137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56239a1342b0 0x56239a13a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.804+0000 7f7c737fe640 1 -- 172.21.15.43:0/2094458137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562399da63f0 con 0x56239a133990 2024-04-21T08:41:43.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.805+0000 7f7c737fe640 1 --2- 172.21.15.43:0/2094458137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a133990 0x56239a139ea0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009900 tx=0x7f7c6402ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d474f9f545bc4be2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:43.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.805+0000 7f7c52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:43.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.805+0000 7f7c52ffd640 1 -- 172.21.15.43:0/2094458137 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64005d50 con 0x56239a133990 2024-04-21T08:41:43.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.805+0000 7f7c52ffd640 1 -- 172.21.15.43:0/2094458137 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f7c64005450 con 0x56239a133990 2024-04-21T08:41:43.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.805+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/2094458137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c68005680 con 0x56239a133990 2024-04-21T08:41:43.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.805+0000 7f7c52ffd640 1 -- 172.21.15.43:0/2094458137 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7c6402cc80 con 0x56239a133990 2024-04-21T08:41:43.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.805+0000 7f7c7c7d6740 10 client.17031 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:43.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.805+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/2094458137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562399d660f0 con 0x56239a133990 2024-04-21T08:41:43.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.805+0000 7f7c52ffd640 1 -- 172.21.15.43:0/2094458137 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c68005680 con 0x56239a133990 2024-04-21T08:41:43.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.805+0000 7f7c52ffd640 1 -- 172.21.15.43:0/2094458137 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7c64003680 con 0x56239a133990 2024-04-21T08:41:43.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.805+0000 7f7c7c7d6740 10 client.17031 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:43.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.805+0000 7f7c7c7d6740 10 client.17031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:43.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.805+0000 7f7c7c7d6740 20 client.17031 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:43.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.805+0000 7f7c7c7d6740 1 --2- 172.21.15.43:0/2094458137 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56239a14cd70 0x56239a14f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.805+0000 7f7c7c7d6740 4 client.17031 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:43.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.805+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/2094458137 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56239a0a1c00 con 0x56239a14cd70 2024-04-21T08:41:43.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.806+0000 7f7c72ffd640 1 --2- 172.21.15.43:0/2094458137 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56239a14cd70 0x56239a14f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.806+0000 7f7c52ffd640 10 client.17031 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:43.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.806+0000 7f7c72ffd640 1 --2- 172.21.15.43:0/2094458137 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56239a14cd70 0x56239a14f1c0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4a3ca43e7dbb814 server_cookie=95557f18eac2ecf3 in_seq=0 out_seq=0 2024-04-21T08:41:43.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.807+0000 7f7c52ffd640 1 -- 172.21.15.43:0/2094458137 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56239a0a1c00 con 0x56239a14cd70 2024-04-21T08:41:43.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.807+0000 7f7c52ffd640 10 client.17031 handle_command_reply: tid=0 2024-04-21T08:41:43.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.807+0000 7f7c7c7d6740 1 client.17031 shutdown 2024-04-21T08:41:43.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.807+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/2094458137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a133990 msgr2=0x56239a139ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:43.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.807+0000 7f7c7c7d6740 1 --2- 172.21.15.43:0/2094458137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a133990 0x56239a139ea0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009900 tx=0x7f7c6402ca90 comp rx=0 tx=0).stop 2024-04-21T08:41:43.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.807+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/2094458137 shutdown_connections 2024-04-21T08:41:43.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.807+0000 7f7c7c7d6740 1 --2- 172.21.15.43:0/2094458137 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56239a14cd70 0x56239a14f1c0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.807+0000 7f7c7c7d6740 1 --2- 172.21.15.43:0/2094458137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56239a13ce90 0x56239a13a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.807+0000 7f7c7c7d6740 1 --2- 172.21.15.43:0/2094458137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56239a1342b0 0x56239a13a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.807+0000 7f7c52ffd640 0 client.17031 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:43.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.807+0000 7f7c7c7d6740 1 --2- 172.21.15.43:0/2094458137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a133990 0x56239a139ea0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.807+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/2094458137 >> 172.21.15.43:0/2094458137 conn(0x56239a12e0c0 msgr2=0x56239a12fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:43.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.807+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/2094458137 shutdown_connections 2024-04-21T08:41:43.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.807+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/2094458137 wait complete. 2024-04-21T08:41:43.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.807+0000 7f7c7c7d6740 20 client.17031 trim_cache size 0 max 16384 2024-04-21T08:41:43.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.825+0000 7f7c7c7d6740 1 Processor -- start 2024-04-21T08:41:43.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.825+0000 7f7c7c7d6740 1 -- start start 2024-04-21T08:41:43.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.825+0000 7f7c7c7d6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56239a133990 0x56239a13c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.825+0000 7f7c7c7d6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56239a1342b0 0x56239a138de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.825+0000 7f7c7c7d6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a13ce90 0x56239a148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.825+0000 7f7c7c7d6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7c741ad290 con 0x56239a13ce90 2024-04-21T08:41:43.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.825+0000 7f7c7c7d6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7c741aedd0 con 0x56239a1342b0 2024-04-21T08:41:43.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.825+0000 7f7c7c7d6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7c741ad960 con 0x56239a133990 2024-04-21T08:41:43.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.825+0000 7f7c73fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a13ce90 0x56239a148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.825+0000 7f7c73fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a13ce90 0x56239a148ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34362/0 (socket says 172.21.15.43:34362) 2024-04-21T08:41:43.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.825+0000 7f7c73fff640 1 -- 172.21.15.43:0/1832704494 learned_addr learned my addr 172.21.15.43:0/1832704494 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:43.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.825+0000 7f7c72ffd640 1 --2- 172.21.15.43:0/1832704494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56239a1342b0 0x56239a138de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.825+0000 7f7c73fff640 1 -- 172.21.15.43:0/1832704494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56239a133990 msgr2=0x56239a13c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:43.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.825+0000 7f7c73fff640 1 --2- 172.21.15.43:0/1832704494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56239a133990 0x56239a13c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.825+0000 7f7c73fff640 1 -- 172.21.15.43:0/1832704494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56239a1342b0 msgr2=0x56239a138de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:43.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.825+0000 7f7c73fff640 1 --2- 172.21.15.43:0/1832704494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56239a1342b0 0x56239a138de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.825+0000 7f7c73fff640 1 -- 172.21.15.43:0/1832704494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56239a1383f0 con 0x56239a13ce90 2024-04-21T08:41:43.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.825+0000 7f7c73fff640 1 --2- 172.21.15.43:0/1832704494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a13ce90 0x56239a148ae0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c01e7a0 tx=0x7f7c6c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9689242bb472c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:43.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.826+0000 7f7c597fa640 1 -- 172.21.15.43:0/1832704494 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c01e030 con 0x56239a13ce90 2024-04-21T08:41:43.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.826+0000 7f7c597fa640 1 -- 172.21.15.43:0/1832704494 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c6c01e1d0 con 0x56239a13ce90 2024-04-21T08:41:43.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.826+0000 7f7c597fa640 1 -- 172.21.15.43:0/1832704494 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c01d740 con 0x56239a13ce90 2024-04-21T08:41:43.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.826+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/1832704494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a13ce90 msgr2=0x56239a148ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:43.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.826+0000 7f7c7c7d6740 1 --2- 172.21.15.43:0/1832704494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a13ce90 0x56239a148ae0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c01e7a0 tx=0x7f7c6c009500 comp rx=0 tx=0).stop 2024-04-21T08:41:43.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.826+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/1832704494 shutdown_connections 2024-04-21T08:41:43.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.826+0000 7f7c7c7d6740 1 --2- 172.21.15.43:0/1832704494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a13ce90 0x56239a148ae0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.826+0000 7f7c7c7d6740 1 --2- 172.21.15.43:0/1832704494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56239a1342b0 0x56239a138de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.826+0000 7f7c7c7d6740 1 --2- 172.21.15.43:0/1832704494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56239a133990 0x56239a13c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.826+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/1832704494 >> 172.21.15.43:0/1832704494 conn(0x56239a12e0c0 msgr2=0x56239a128d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:43.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.826+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/1832704494 shutdown_connections 2024-04-21T08:41:43.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.826+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/1832704494 wait complete. 2024-04-21T08:41:43.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.827+0000 7f7c7c7d6740 1 -- start start 2024-04-21T08:41:43.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.827+0000 7f7c7c7d6740 1 Processor -- start 2024-04-21T08:41:43.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.827+0000 7f7c7c7d6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56239a133990 0x56239a141210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.827+0000 7f7c7c7d6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56239a1342b0 0x56239a141750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.827+0000 7f7c7c7d6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a13ce90 0x56239a1b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.827+0000 7f7c7c7d6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7c741ae700 con 0x56239a13ce90 2024-04-21T08:41:43.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.827+0000 7f7c7c7d6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7c741ae030 con 0x56239a1342b0 2024-04-21T08:41:43.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.827+0000 7f7c7c7d6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7c741ab270 con 0x56239a133990 2024-04-21T08:41:43.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.827+0000 7f7c7c7d6740 10 client.0 _ll_register_callbacks cb 0x7f7c68005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:43.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.827+0000 7f7c73fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a13ce90 0x56239a1b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.827+0000 7f7c73fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a13ce90 0x56239a1b2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34368/0 (socket says 172.21.15.43:34368) 2024-04-21T08:41:43.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.827+0000 7f7c73fff640 1 -- 172.21.15.43:0/3028680756 learned_addr learned my addr 172.21.15.43:0/3028680756 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:43.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.827+0000 7f7c72ffd640 1 --2- 172.21.15.43:0/3028680756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56239a1342b0 0x56239a141750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.827+0000 7f7c737fe640 1 --2- 172.21.15.43:0/3028680756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56239a133990 0x56239a141210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.827+0000 7f7c73fff640 1 -- 172.21.15.43:0/3028680756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56239a133990 msgr2=0x56239a141210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:43.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.827+0000 7f7c73fff640 1 --2- 172.21.15.43:0/3028680756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56239a133990 0x56239a141210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.827+0000 7f7c73fff640 1 -- 172.21.15.43:0/3028680756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56239a1342b0 msgr2=0x56239a141750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:43.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.827+0000 7f7c73fff640 1 --2- 172.21.15.43:0/3028680756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56239a1342b0 0x56239a141750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.827+0000 7f7c73fff640 1 -- 172.21.15.43:0/3028680756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56239a1b2910 con 0x56239a13ce90 2024-04-21T08:41:43.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.827+0000 7f7c737fe640 1 --2- 172.21.15.43:0/3028680756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56239a133990 0x56239a141210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:43.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.828+0000 7f7c73fff640 1 --2- 172.21.15.43:0/3028680756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a13ce90 0x56239a1b2300 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c018040 tx=0x7f7c6c01e460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6629f4c48062ff57 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:43.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.828+0000 7f7c52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:43.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.828+0000 7f7c52ffd640 1 -- 172.21.15.43:0/3028680756 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c0299a0 con 0x56239a13ce90 2024-04-21T08:41:43.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.828+0000 7f7c52ffd640 1 -- 172.21.15.43:0/3028680756 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f7c6c029b40 con 0x56239a13ce90 2024-04-21T08:41:43.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.828+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/3028680756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c6c01e1d0 con 0x56239a13ce90 2024-04-21T08:41:43.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.828+0000 7f7c52ffd640 1 -- 172.21.15.43:0/3028680756 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7c6c005040 con 0x56239a13ce90 2024-04-21T08:41:43.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.828+0000 7f7c7c7d6740 10 client.17037 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:43.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.828+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/3028680756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56239a1b3c20 con 0x56239a13ce90 2024-04-21T08:41:43.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.828+0000 7f7c52ffd640 1 -- 172.21.15.43:0/3028680756 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c6c01e1d0 con 0x56239a13ce90 2024-04-21T08:41:43.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.828+0000 7f7c52ffd640 1 -- 172.21.15.43:0/3028680756 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7c6c030070 con 0x56239a13ce90 2024-04-21T08:41:43.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.828+0000 7f7c7c7d6740 10 client.17037 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:43.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.828+0000 7f7c7c7d6740 10 client.17037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:43.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.828+0000 7f7c7c7d6740 20 client.17037 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:43.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.828+0000 7f7c7c7d6740 1 --2- 172.21.15.43:0/3028680756 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56239a1b4260 0x56239a1b6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:43.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.828+0000 7f7c7c7d6740 4 client.17037 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:41:43.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.828+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/3028680756 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56239a09ba70 con 0x56239a1b4260 2024-04-21T08:41:43.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.829+0000 7f7c737fe640 1 --2- 172.21.15.43:0/3028680756 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56239a1b4260 0x56239a1b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:43.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.829+0000 7f7c737fe640 1 --2- 172.21.15.43:0/3028680756 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56239a1b4260 0x56239a1b6680 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=303f13bcb955dec6 server_cookie=4515ce6020ea1536 in_seq=0 out_seq=0 2024-04-21T08:41:43.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.829+0000 7f7c52ffd640 10 client.17037 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:43.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.829+0000 7f7c52ffd640 1 -- 172.21.15.43:0/3028680756 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56239a09ba70 con 0x56239a1b4260 2024-04-21T08:41:43.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.829+0000 7f7c52ffd640 10 client.17037 handle_command_reply: tid=0 2024-04-21T08:41:43.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.829+0000 7f7c7c7d6740 1 client.17037 shutdown 2024-04-21T08:41:43.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.829+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/3028680756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a13ce90 msgr2=0x56239a1b2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:43.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.829+0000 7f7c7c7d6740 1 --2- 172.21.15.43:0/3028680756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a13ce90 0x56239a1b2300 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c018040 tx=0x7f7c6c01e460 comp rx=0 tx=0).stop 2024-04-21T08:41:43.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.830+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/3028680756 shutdown_connections 2024-04-21T08:41:43.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.830+0000 7f7c7c7d6740 1 --2- 172.21.15.43:0/3028680756 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56239a1b4260 0x56239a1b6680 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.830+0000 7f7c7c7d6740 1 --2- 172.21.15.43:0/3028680756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56239a13ce90 0x56239a1b2300 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.830+0000 7f7c7c7d6740 1 --2- 172.21.15.43:0/3028680756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56239a1342b0 0x56239a141750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.830+0000 7f7c52ffd640 0 client.17037 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:43.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.830+0000 7f7c7c7d6740 1 --2- 172.21.15.43:0/3028680756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56239a133990 0x56239a141210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.830+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/3028680756 >> 172.21.15.43:0/3028680756 conn(0x56239a12e0c0 msgr2=0x56239a12f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:43.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.830+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/3028680756 shutdown_connections 2024-04-21T08:41:43.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.830+0000 7f7c7c7d6740 1 -- 172.21.15.43:0/3028680756 wait complete. 2024-04-21T08:41:43.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.830+0000 7f7c7c7d6740 20 client.17037 trim_cache size 0 max 16384 2024-04-21T08:41:43.836 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:41:43.836 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:41:43.836 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-21T08:41:43.836 INFO:teuthology.orchestra.run.smithi043.stdout: "7e48e4aa-7bff-4abe-b100-44c8949935f2": { 2024-04-21T08:41:43.836 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-21T08:41:43.836 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "7e48e4aa-7bff-4abe-b100-44c8949935f2", 2024-04-21T08:41:43.836 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-21T08:41:43.836 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:41:43.836 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:41:43.836 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:41:43.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.830+0000 7f7c79b11640 1 -- 172.21.15.43:0/3264109023 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7c5c0a3d20 msgr2=0x7f7c5c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:43.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.830+0000 7f7c79b11640 1 --2- 172.21.15.43:0/3264109023 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7c5c0a3d20 0x7f7c5c0a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c005fd0 tx=0x7f7c6c004a40 comp rx=0 tx=0).stop 2024-04-21T08:41:43.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.830+0000 7f7c79b11640 1 -- 172.21.15.43:0/3264109023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c74106150 msgr2=0x7f7c74198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:43.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.830+0000 7f7c79b11640 1 --2- 172.21.15.43:0/3264109023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c74106150 0x7f7c74198c80 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800e9f0 tx=0x7f7c6800eeb0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.830+0000 7f7c79b11640 1 -- 172.21.15.43:0/3264109023 shutdown_connections 2024-04-21T08:41:43.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.830+0000 7f7c79b11640 1 --2- 172.21.15.43:0/3264109023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c7419d000 0x7f7c7419d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.830+0000 7f7c79b11640 1 --2- 172.21.15.43:0/3264109023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c74106150 0x7f7c74198c80 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.830+0000 7f7c79b11640 1 --2- 172.21.15.43:0/3264109023 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7c5c0a3d20 0x7f7c5c0a61e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.830+0000 7f7c79b11640 1 --2- 172.21.15.43:0/3264109023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c74101080 0x7f7c74198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:43.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.830+0000 7f7c79b11640 1 -- 172.21.15.43:0/3264109023 >> 172.21.15.43:0/3264109023 conn(0x7f7c74078000 msgr2=0x7f7c740694e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:43.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.830+0000 7f7c79b11640 1 -- 172.21.15.43:0/3264109023 shutdown_connections 2024-04-21T08:41:43.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:41:43.830+0000 7f7c79b11640 1 -- 172.21.15.43:0/3264109023 wait complete. 2024-04-21T08:41:43.840 INFO:tasks.cephfs.filesystem:scrub status for tag:7e48e4aa-7bff-4abe-b100-44c8949935f2 - {'path': '/', 'tag': '7e48e4aa-7bff-4abe-b100-44c8949935f2', 'options': 'recursive,force'} 2024-04-21T08:41:44.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:44 smithi043 ceph-mon[27978]: pgmap v87: 129 pgs: 129 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-21T08:41:44.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:44 smithi175 ceph-mon[31920]: pgmap v87: 129 pgs: 129 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-21T08:41:44.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:44 smithi112 ceph-mon[31085]: pgmap v87: 129 pgs: 129 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-21T08:41:45.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:45 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:41:45.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:45 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:41:45.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:45 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:41:45.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:45 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:41:45.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:45 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:41:45.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:45 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:41:45.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:45 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:41:45.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:45 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:41:45.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:45 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:41:45.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:45 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:41:45.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:45 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:41:45.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:45 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:41:45.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:45 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:41:45.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:45 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:41:45.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:45 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:41:46.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:46 smithi175 ceph-mon[31920]: pgmap v88: 129 pgs: 129 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-21T08:41:46.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:46 smithi112 ceph-mon[31085]: pgmap v88: 129 pgs: 129 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-21T08:41:46.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:46 smithi043 ceph-mon[27978]: pgmap v88: 129 pgs: 129 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-21T08:41:48.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:48 smithi175 ceph-mon[31920]: pgmap v89: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-21T08:41:48.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:48 smithi112 ceph-mon[31085]: pgmap v89: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-21T08:41:48.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:48 smithi043 ceph-mon[27978]: pgmap v89: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-21T08:41:50.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:50 smithi175 ceph-mon[31920]: pgmap v90: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-21T08:41:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:50 smithi112 ceph-mon[31085]: pgmap v90: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-21T08:41:50.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:50 smithi043 ceph-mon[27978]: pgmap v90: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-21T08:41:52.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:52 smithi175 ceph-mon[31920]: pgmap v91: 129 pgs: 129 active+clean; 3.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-04-21T08:41:52.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:52 smithi112 ceph-mon[31085]: pgmap v91: 129 pgs: 129 active+clean; 3.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-04-21T08:41:52.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:52 smithi043 ceph-mon[27978]: pgmap v91: 129 pgs: 129 active+clean; 3.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-04-21T08:41:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.952+0000 7f6f72681640 1 -- 172.21.15.43:0/1186884049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f6c101aa0 msgr2=0x7f6f6c105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.952+0000 7f6f72681640 1 --2- 172.21.15.43:0/1186884049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f6c101aa0 0x7f6f6c105a90 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f6f60009900 tx=0x7f6f6002bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.952+0000 7f6f72681640 1 -- 172.21.15.43:0/1186884049 shutdown_connections 2024-04-21T08:41:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.952+0000 7f6f72681640 1 --2- 172.21.15.43:0/1186884049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f6c106170 0x7f6f6c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.952+0000 7f6f72681640 1 --2- 172.21.15.43:0/1186884049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f6c101aa0 0x7f6f6c105a90 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.952+0000 7f6f72681640 1 --2- 172.21.15.43:0/1186884049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f6c1010f0 0x7f6f6c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.952+0000 7f6f72681640 1 -- 172.21.15.43:0/1186884049 >> 172.21.15.43:0/1186884049 conn(0x7f6f6c077fd0 msgr2=0x7f6f6c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.952+0000 7f6f72681640 1 -- 172.21.15.43:0/1186884049 shutdown_connections 2024-04-21T08:41:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.952+0000 7f6f72681640 1 -- 172.21.15.43:0/1186884049 wait complete. 2024-04-21T08:41:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.953+0000 7f6f72681640 1 Processor -- start 2024-04-21T08:41:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.953+0000 7f6f72681640 1 -- start start 2024-04-21T08:41:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.953+0000 7f6f72681640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f6c1010f0 0x7f6f6c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.953+0000 7f6f72681640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f6c106170 0x7f6f6c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.953+0000 7f6f72681640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f6c19cff0 0x7f6f6c19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.953+0000 7f6f72681640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c111100 con 0x7f6f6c1010f0 2024-04-21T08:41:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.953+0000 7f6f72681640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c110f80 con 0x7f6f6c19cff0 2024-04-21T08:41:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.953+0000 7f6f72681640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c111280 con 0x7f6f6c106170 2024-04-21T08:41:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.953+0000 7f6f6bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f6c1010f0 0x7f6f6c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.954+0000 7f6f6bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f6c1010f0 0x7f6f6c198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39398/0 (socket says 172.21.15.43:39398) 2024-04-21T08:41:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.954+0000 7f6f6bfff640 1 -- 172.21.15.43:0/4120400016 learned_addr learned my addr 172.21.15.43:0/4120400016 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.954+0000 7f6f70c1e640 1 --2- 172.21.15.43:0/4120400016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f6c19cff0 0x7f6f6c19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.954+0000 7f6f6b7fe640 1 --2- 172.21.15.43:0/4120400016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f6c106170 0x7f6f6c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.954+0000 7f6f6bfff640 1 -- 172.21.15.43:0/4120400016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f6c106170 msgr2=0x7f6f6c198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.954+0000 7f6f6bfff640 1 --2- 172.21.15.43:0/4120400016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f6c106170 0x7f6f6c198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.954+0000 7f6f6bfff640 1 -- 172.21.15.43:0/4120400016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f6c19cff0 msgr2=0x7f6f6c19d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.954+0000 7f6f6bfff640 1 --2- 172.21.15.43:0/4120400016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f6c19cff0 0x7f6f6c19d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.954+0000 7f6f6bfff640 1 -- 172.21.15.43:0/4120400016 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6c19dae0 con 0x7f6f6c1010f0 2024-04-21T08:41:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.954+0000 7f6f6b7fe640 1 --2- 172.21.15.43:0/4120400016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f6c106170 0x7f6f6c198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.954+0000 7f6f6bfff640 1 --2- 172.21.15.43:0/4120400016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f6c1010f0 0x7f6f6c198730 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00d770 tx=0x7f6f5c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.954+0000 7f6f697fa640 1 -- 172.21.15.43:0/4120400016 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c017070 con 0x7f6f6c1010f0 2024-04-21T08:41:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.954+0000 7f6f72681640 1 -- 172.21.15.43:0/4120400016 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f6c19ddd0 con 0x7f6f6c1010f0 2024-04-21T08:41:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.954+0000 7f6f697fa640 1 -- 172.21.15.43:0/4120400016 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f5c0054d0 con 0x7f6f6c1010f0 2024-04-21T08:41:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.954+0000 7f6f697fa640 1 -- 172.21.15.43:0/4120400016 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c004c30 con 0x7f6f6c1010f0 2024-04-21T08:41:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.954+0000 7f6f72681640 1 -- 172.21.15.43:0/4120400016 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f6c1a55c0 con 0x7f6f6c1010f0 2024-04-21T08:41:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.957+0000 7f6f697fa640 1 -- 172.21.15.43:0/4120400016 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6f5c004e10 con 0x7f6f6c1010f0 2024-04-21T08:41:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.957+0000 7f6f7527a740 1 Processor -- start 2024-04-21T08:41:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.957+0000 7f6f7527a740 1 -- start start 2024-04-21T08:41:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.957+0000 7f6f7527a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8a51fe8d0 0x55e8a51fd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.957+0000 7f6f7527a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8a51fdb60 0x55e8a51fdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.957+0000 7f6f7527a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a520b4f0 0x55e8a520d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.957+0000 7f6f7527a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e8a51628d0 con 0x55e8a520b4f0 2024-04-21T08:41:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.957+0000 7f6f7527a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e8a50ffe80 con 0x55e8a51fdb60 2024-04-21T08:41:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.957+0000 7f6f7527a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e8a515e890 con 0x55e8a51fe8d0 2024-04-21T08:41:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.957+0000 7f6f6b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a520b4f0 0x55e8a520d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.957+0000 7f6f697fa640 1 --2- 172.21.15.43:0/4120400016 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6f540a3dc0 0x7f6f540a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.957+0000 7f6f6b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a520b4f0 0x55e8a520d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39412/0 (socket says 172.21.15.43:39412) 2024-04-21T08:41:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.957+0000 7f6f6b7fe640 1 -- 172.21.15.43:0/652321494 learned_addr learned my addr 172.21.15.43:0/652321494 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.957+0000 7f6f6bfff640 1 --2- 172.21.15.43:0/652321494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8a51fdb60 0x55e8a51fdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.958+0000 7f6f70c1e640 1 --2- 172.21.15.43:0/4120400016 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6f540a3dc0 0x7f6f540a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.958+0000 7f6f697fa640 1 -- 172.21.15.43:0/4120400016 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f6f5c0d0750 con 0x7f6f6c1010f0 2024-04-21T08:41:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.958+0000 7f6f70c1e640 1 --2- 172.21.15.43:0/652321494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8a51fe8d0 0x55e8a51fd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.958+0000 7f6f6b7fe640 1 -- 172.21.15.43:0/652321494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8a51fe8d0 msgr2=0x55e8a51fd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.958+0000 7f6f6b7fe640 1 --2- 172.21.15.43:0/652321494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8a51fe8d0 0x55e8a51fd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.958+0000 7f6f6b7fe640 1 -- 172.21.15.43:0/652321494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8a51fdb60 msgr2=0x55e8a51fdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.958+0000 7f6f6b7fe640 1 --2- 172.21.15.43:0/652321494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8a51fdb60 0x55e8a51fdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.958+0000 7f6f6b7fe640 1 -- 172.21.15.43:0/652321494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8a5121320 con 0x55e8a520b4f0 2024-04-21T08:41:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.958+0000 7f6f6bfff640 1 --2- 172.21.15.43:0/652321494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8a51fdb60 0x55e8a51fdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.958+0000 7f6f70c1e640 1 --2- 172.21.15.43:0/652321494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8a51fe8d0 0x55e8a51fd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.958+0000 7f6f6b7fe640 1 --2- 172.21.15.43:0/652321494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a520b4f0 0x55e8a520d8e0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f6f60005ab0 tx=0x7f6f6002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598fe4bb85a5732a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.958+0000 7f6f70c1e640 1 --2- 172.21.15.43:0/4120400016 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6f540a3dc0 0x7f6f540a6280 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6f64005fd0 tx=0x7f6f64009330 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.958+0000 7f6f527fc640 1 -- 172.21.15.43:0/652321494 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60043020 con 0x55e8a520b4f0 2024-04-21T08:41:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.958+0000 7f6f527fc640 1 -- 172.21.15.43:0/652321494 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f6002c7e0 con 0x55e8a520b4f0 2024-04-21T08:41:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.958+0000 7f6f527fc640 1 -- 172.21.15.43:0/652321494 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f600046c0 con 0x55e8a520b4f0 2024-04-21T08:41:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.959+0000 7f6f7527a740 1 -- 172.21.15.43:0/652321494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a520b4f0 msgr2=0x55e8a520d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.959+0000 7f6f7527a740 1 --2- 172.21.15.43:0/652321494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a520b4f0 0x55e8a520d8e0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f6f60005ab0 tx=0x7f6f6002c650 comp rx=0 tx=0).stop 2024-04-21T08:41:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.959+0000 7f6f7527a740 1 -- 172.21.15.43:0/652321494 shutdown_connections 2024-04-21T08:41:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.959+0000 7f6f7527a740 1 --2- 172.21.15.43:0/652321494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a520b4f0 0x55e8a520d8e0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.959+0000 7f6f7527a740 1 --2- 172.21.15.43:0/652321494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8a51fdb60 0x55e8a51fdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.959+0000 7f6f7527a740 1 --2- 172.21.15.43:0/652321494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8a51fe8d0 0x55e8a51fd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.959+0000 7f6f7527a740 1 -- 172.21.15.43:0/652321494 >> 172.21.15.43:0/652321494 conn(0x55e8a51c4c90 msgr2=0x55e8a51f8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.959+0000 7f6f7527a740 1 -- 172.21.15.43:0/652321494 shutdown_connections 2024-04-21T08:41:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.959+0000 7f6f7527a740 1 -- 172.21.15.43:0/652321494 wait complete. 2024-04-21T08:41:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.959+0000 7f6f7527a740 1 -- start start 2024-04-21T08:41:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.960+0000 7f6f7527a740 1 Processor -- start 2024-04-21T08:41:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.960+0000 7f6f7527a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a51fdb60 0x55e8a520ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.960+0000 7f6f7527a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8a51fe8d0 0x55e8a52081b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.960+0000 7f6f7527a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8a520b4f0 0x55e8a52086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.960+0000 7f6f7527a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e8a520e1c0 con 0x55e8a51fdb60 2024-04-21T08:41:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.960+0000 7f6f7527a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e8a50eaeb0 con 0x55e8a520b4f0 2024-04-21T08:41:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.960+0000 7f6f7527a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e8a520e340 con 0x55e8a51fe8d0 2024-04-21T08:41:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.960+0000 7f6f7527a740 10 client.0 _ll_register_callbacks cb 0x55e8a5117b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.960+0000 7f6f6bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a51fdb60 0x55e8a520ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.960+0000 7f6f6bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a51fdb60 0x55e8a520ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39418/0 (socket says 172.21.15.43:39418) 2024-04-21T08:41:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.960+0000 7f6f6bfff640 1 -- 172.21.15.43:0/615566748 learned_addr learned my addr 172.21.15.43:0/615566748 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.960+0000 7f6f6b7fe640 1 --2- 172.21.15.43:0/615566748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8a51fe8d0 0x55e8a52081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.960+0000 7f6f70c1e640 1 --2- 172.21.15.43:0/615566748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8a520b4f0 0x55e8a52086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.961+0000 7f6f6bfff640 1 -- 172.21.15.43:0/615566748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8a51fe8d0 msgr2=0x55e8a52081b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.961+0000 7f6f6bfff640 1 --2- 172.21.15.43:0/615566748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8a51fe8d0 0x55e8a52081b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.961+0000 7f6f6bfff640 1 -- 172.21.15.43:0/615566748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8a520b4f0 msgr2=0x55e8a52086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.961+0000 7f6f6bfff640 1 --2- 172.21.15.43:0/615566748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8a520b4f0 0x55e8a52086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.961+0000 7f6f6bfff640 1 -- 172.21.15.43:0/615566748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8a4e75210 con 0x55e8a51fdb60 2024-04-21T08:41:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.961+0000 7f6f70c1e640 1 --2- 172.21.15.43:0/615566748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8a520b4f0 0x55e8a52086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.961+0000 7f6f6bfff640 1 --2- 172.21.15.43:0/615566748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a51fdb60 0x55e8a520ae20 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0ce0d0 tx=0x7f6f5c06acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3995f745e2e30a9a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.961+0000 7f6f4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.961+0000 7f6f4b7fe640 1 -- 172.21.15.43:0/615566748 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c020040 con 0x55e8a51fdb60 2024-04-21T08:41:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.961+0000 7f6f4b7fe640 1 -- 172.21.15.43:0/615566748 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f6f5c0cdd50 con 0x55e8a51fdb60 2024-04-21T08:41:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.961+0000 7f6f7527a740 1 -- 172.21.15.43:0/615566748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f6002c7e0 con 0x55e8a51fdb60 2024-04-21T08:41:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.961+0000 7f6f4b7fe640 1 -- 172.21.15.43:0/615566748 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6f5c09c320 con 0x55e8a51fdb60 2024-04-21T08:41:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.961+0000 7f6f7527a740 10 client.17049 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.961+0000 7f6f7527a740 1 -- 172.21.15.43:0/615566748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e8a4f41620 con 0x55e8a51fdb60 2024-04-21T08:41:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.961+0000 7f6f4b7fe640 1 -- 172.21.15.43:0/615566748 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f6002c7e0 con 0x55e8a51fdb60 2024-04-21T08:41:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.961+0000 7f6f4b7fe640 1 -- 172.21.15.43:0/615566748 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6f5c0d0320 con 0x55e8a51fdb60 2024-04-21T08:41:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.962+0000 7f6f7527a740 10 client.17049 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.962+0000 7f6f7527a740 10 client.17049 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.962+0000 7f6f7527a740 20 client.17049 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.962+0000 7f6f7527a740 1 --2- 172.21.15.43:0/615566748 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e8a521b3e0 0x55e8a521d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.962+0000 7f6f7527a740 4 client.17049 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.962+0000 7f6f7527a740 1 -- 172.21.15.43:0/615566748 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8a5176880 con 0x55e8a521b3e0 2024-04-21T08:41:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.962+0000 7f6f6b7fe640 1 --2- 172.21.15.43:0/615566748 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e8a521b3e0 0x55e8a521d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.962+0000 7f6f6b7fe640 1 --2- 172.21.15.43:0/615566748 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e8a521b3e0 0x55e8a521d830 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab4158e3f1631ac3 server_cookie=f267464bead700c0 in_seq=0 out_seq=0 2024-04-21T08:41:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.962+0000 7f6f4b7fe640 10 client.17049 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.963+0000 7f6f4b7fe640 1 -- 172.21.15.43:0/615566748 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e8a5176880 con 0x55e8a521b3e0 2024-04-21T08:41:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.963+0000 7f6f4b7fe640 10 client.17049 handle_command_reply: tid=0 2024-04-21T08:41:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.963+0000 7f6f7527a740 1 client.17049 shutdown 2024-04-21T08:41:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.963+0000 7f6f7527a740 1 -- 172.21.15.43:0/615566748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a51fdb60 msgr2=0x55e8a520ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.963+0000 7f6f7527a740 1 --2- 172.21.15.43:0/615566748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a51fdb60 0x55e8a520ae20 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0ce0d0 tx=0x7f6f5c06acd0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.963+0000 7f6f7527a740 1 -- 172.21.15.43:0/615566748 shutdown_connections 2024-04-21T08:41:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.963+0000 7f6f7527a740 1 --2- 172.21.15.43:0/615566748 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e8a521b3e0 0x55e8a521d830 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.963+0000 7f6f7527a740 1 --2- 172.21.15.43:0/615566748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8a520b4f0 0x55e8a52086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.963+0000 7f6f7527a740 1 --2- 172.21.15.43:0/615566748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8a51fe8d0 0x55e8a52081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.963+0000 7f6f4b7fe640 0 client.17049 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.963+0000 7f6f7527a740 1 --2- 172.21.15.43:0/615566748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a51fdb60 0x55e8a520ae20 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.963+0000 7f6f7527a740 1 -- 172.21.15.43:0/615566748 >> 172.21.15.43:0/615566748 conn(0x55e8a51c4c90 msgr2=0x55e8a51fa5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.963+0000 7f6f7527a740 1 -- 172.21.15.43:0/615566748 shutdown_connections 2024-04-21T08:41:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.963+0000 7f6f7527a740 1 -- 172.21.15.43:0/615566748 wait complete. 2024-04-21T08:41:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.964+0000 7f6f7527a740 20 client.17049 trim_cache size 0 max 16384 2024-04-21T08:41:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.979+0000 7f6f7527a740 1 Processor -- start 2024-04-21T08:41:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.979+0000 7f6f7527a740 1 -- start start 2024-04-21T08:41:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.979+0000 7f6f7527a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8a51c4c90 0x55e8a52170f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.979+0000 7f6f7527a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a51fdb60 0x55e8a52076f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.979+0000 7f6f7527a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8a51fe8d0 0x55e8a520a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.979+0000 7f6f7527a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c1ad100 con 0x55e8a51fdb60 2024-04-21T08:41:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.979+0000 7f6f7527a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c1aec40 con 0x55e8a51fe8d0 2024-04-21T08:41:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.979+0000 7f6f7527a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c1ad7d0 con 0x55e8a51c4c90 2024-04-21T08:41:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.979+0000 7f6f6b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a51fdb60 0x55e8a52076f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.979+0000 7f6f6b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a51fdb60 0x55e8a52076f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39430/0 (socket says 172.21.15.43:39430) 2024-04-21T08:41:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.979+0000 7f6f6b7fe640 1 -- 172.21.15.43:0/2956256743 learned_addr learned my addr 172.21.15.43:0/2956256743 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.979+0000 7f6f70c1e640 1 --2- 172.21.15.43:0/2956256743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8a51fe8d0 0x55e8a520a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.979+0000 7f6f6bfff640 1 --2- 172.21.15.43:0/2956256743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8a51c4c90 0x55e8a52170f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.979+0000 7f6f6b7fe640 1 -- 172.21.15.43:0/2956256743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8a51c4c90 msgr2=0x55e8a52170f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.979+0000 7f6f6b7fe640 1 --2- 172.21.15.43:0/2956256743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8a51c4c90 0x55e8a52170f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.979+0000 7f6f6b7fe640 1 -- 172.21.15.43:0/2956256743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8a51fe8d0 msgr2=0x55e8a520a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.979+0000 7f6f6b7fe640 1 --2- 172.21.15.43:0/2956256743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8a51fe8d0 0x55e8a520a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.979+0000 7f6f6b7fe640 1 -- 172.21.15.43:0/2956256743 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8a51fce30 con 0x55e8a51fdb60 2024-04-21T08:41:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.979+0000 7f6f6b7fe640 1 --2- 172.21.15.43:0/2956256743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a51fdb60 0x55e8a52076f0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f6f6002c210 tx=0x7f6f6002c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb27c73b74a7551 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.980+0000 7f6f51ffb640 1 -- 172.21.15.43:0/2956256743 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60043040 con 0x55e8a51fdb60 2024-04-21T08:41:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.980+0000 7f6f51ffb640 1 -- 172.21.15.43:0/2956256743 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f6002c7e0 con 0x55e8a51fdb60 2024-04-21T08:41:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.980+0000 7f6f51ffb640 1 -- 172.21.15.43:0/2956256743 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60003af0 con 0x55e8a51fdb60 2024-04-21T08:41:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.980+0000 7f6f7527a740 1 -- 172.21.15.43:0/2956256743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a51fdb60 msgr2=0x55e8a52076f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.980+0000 7f6f7527a740 1 --2- 172.21.15.43:0/2956256743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a51fdb60 0x55e8a52076f0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f6f6002c210 tx=0x7f6f6002c740 comp rx=0 tx=0).stop 2024-04-21T08:41:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.980+0000 7f6f7527a740 1 -- 172.21.15.43:0/2956256743 shutdown_connections 2024-04-21T08:41:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.980+0000 7f6f7527a740 1 --2- 172.21.15.43:0/2956256743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8a51fe8d0 0x55e8a520a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.980+0000 7f6f7527a740 1 --2- 172.21.15.43:0/2956256743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a51fdb60 0x55e8a52076f0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.980+0000 7f6f7527a740 1 --2- 172.21.15.43:0/2956256743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8a51c4c90 0x55e8a52170f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.980+0000 7f6f7527a740 1 -- 172.21.15.43:0/2956256743 >> 172.21.15.43:0/2956256743 conn(0x55e8a5208850 msgr2=0x55e8a5206a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.980+0000 7f6f7527a740 1 -- 172.21.15.43:0/2956256743 shutdown_connections 2024-04-21T08:41:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.980+0000 7f6f7527a740 1 -- 172.21.15.43:0/2956256743 wait complete. 2024-04-21T08:41:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.981+0000 7f6f7527a740 1 -- start start 2024-04-21T08:41:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.981+0000 7f6f7527a740 1 Processor -- start 2024-04-21T08:41:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.981+0000 7f6f7527a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8a51c4c90 0x55e8a5204e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.981+0000 7f6f7527a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8a51fdb60 0x55e8a52053a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.981+0000 7f6f7527a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a51fe8d0 0x55e8a52058e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.981+0000 7f6f7527a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c1ae570 con 0x55e8a51fe8d0 2024-04-21T08:41:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.981+0000 7f6f7527a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c1adea0 con 0x55e8a51fdb60 2024-04-21T08:41:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.981+0000 7f6f7527a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c1ab0e0 con 0x55e8a51c4c90 2024-04-21T08:41:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.981+0000 7f6f7527a740 10 client.0 _ll_register_callbacks cb 0x7f6f5c0985c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.981+0000 7f6f70c1e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a51fe8d0 0x55e8a52058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.981+0000 7f6f70c1e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a51fe8d0 0x55e8a52058e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39442/0 (socket says 172.21.15.43:39442) 2024-04-21T08:41:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.981+0000 7f6f70c1e640 1 -- 172.21.15.43:0/445130476 learned_addr learned my addr 172.21.15.43:0/445130476 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:41:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.981+0000 7f6f6bfff640 1 --2- 172.21.15.43:0/445130476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8a51c4c90 0x55e8a5204e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.981+0000 7f6f6b7fe640 1 --2- 172.21.15.43:0/445130476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8a51fdb60 0x55e8a52053a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.982+0000 7f6f70c1e640 1 -- 172.21.15.43:0/445130476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8a51c4c90 msgr2=0x55e8a5204e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.982+0000 7f6f70c1e640 1 --2- 172.21.15.43:0/445130476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8a51c4c90 0x55e8a5204e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.982+0000 7f6f70c1e640 1 -- 172.21.15.43:0/445130476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8a51fdb60 msgr2=0x55e8a52053a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.982+0000 7f6f70c1e640 1 --2- 172.21.15.43:0/445130476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8a51fdb60 0x55e8a52053a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.982+0000 7f6f70c1e640 1 -- 172.21.15.43:0/445130476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8a52809d0 con 0x55e8a51fe8d0 2024-04-21T08:41:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.982+0000 7f6f70c1e640 1 --2- 172.21.15.43:0/445130476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a51fe8d0 0x55e8a52058e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f6f64012b40 tx=0x7f6f64004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd19f17ef0169adf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.982+0000 7f6f4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:41:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.982+0000 7f6f4b7fe640 1 -- 172.21.15.43:0/445130476 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f64003b70 con 0x55e8a51fe8d0 2024-04-21T08:41:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.982+0000 7f6f4b7fe640 1 -- 172.21.15.43:0/445130476 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f6f64003d10 con 0x55e8a51fe8d0 2024-04-21T08:41:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.982+0000 7f6f7527a740 1 -- 172.21.15.43:0/445130476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f6002c7e0 con 0x55e8a51fe8d0 2024-04-21T08:41:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.982+0000 7f6f4b7fe640 1 -- 172.21.15.43:0/445130476 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6f64031050 con 0x55e8a51fe8d0 2024-04-21T08:41:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.982+0000 7f6f7527a740 10 client.17055 fetch_fsmap learned FSMap version 28 2024-04-21T08:41:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.982+0000 7f6f7527a740 1 -- 172.21.15.43:0/445130476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e8a5281d20 con 0x55e8a51fe8d0 2024-04-21T08:41:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.982+0000 7f6f4b7fe640 1 -- 172.21.15.43:0/445130476 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f6002c7e0 con 0x55e8a51fe8d0 2024-04-21T08:41:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.982+0000 7f6f4b7fe640 1 -- 172.21.15.43:0/445130476 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6f64019030 con 0x55e8a51fe8d0 2024-04-21T08:41:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.983+0000 7f6f7527a740 10 client.17055 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:41:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.983+0000 7f6f7527a740 10 client.17055 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.983+0000 7f6f7527a740 20 client.17055 populate_metadata read hostname 'smithi043' 2024-04-21T08:41:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.983+0000 7f6f7527a740 1 --2- 172.21.15.43:0/445130476 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e8a520b4f0 0x55e8a5284450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.983+0000 7f6f7527a740 4 client.17055 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:41:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.983+0000 7f6f7527a740 1 -- 172.21.15.43:0/445130476 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e8a51023d0 con 0x55e8a520b4f0 2024-04-21T08:41:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.983+0000 7f6f6bfff640 1 --2- 172.21.15.43:0/445130476 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e8a520b4f0 0x55e8a5284450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.983+0000 7f6f4b7fe640 10 client.17055 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.983+0000 7f6f6bfff640 1 --2- 172.21.15.43:0/445130476 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e8a520b4f0 0x55e8a5284450 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e018903d58ef72d server_cookie=683528cd53406564 in_seq=0 out_seq=0 2024-04-21T08:41:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.995+0000 7f6f4b7fe640 1 -- 172.21.15.43:0/445130476 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e8a51023d0 con 0x55e8a520b4f0 2024-04-21T08:41:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.995+0000 7f6f4b7fe640 10 client.17055 handle_command_reply: tid=0 2024-04-21T08:41:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.995+0000 7f6f7527a740 1 client.17055 shutdown 2024-04-21T08:41:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.995+0000 7f6f7527a740 1 -- 172.21.15.43:0/445130476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a51fe8d0 msgr2=0x55e8a52058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.995+0000 7f6f7527a740 1 --2- 172.21.15.43:0/445130476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a51fe8d0 0x55e8a52058e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f6f64012b40 tx=0x7f6f64004ab0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.995+0000 7f6f7527a740 1 -- 172.21.15.43:0/445130476 shutdown_connections 2024-04-21T08:41:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.995+0000 7f6f7527a740 1 --2- 172.21.15.43:0/445130476 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e8a520b4f0 0x55e8a5284450 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.995+0000 7f6f7527a740 1 --2- 172.21.15.43:0/445130476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e8a51fe8d0 0x55e8a52058e0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.995+0000 7f6f7527a740 1 --2- 172.21.15.43:0/445130476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e8a51fdb60 0x55e8a52053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.995+0000 7f6f4b7fe640 0 client.17055 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:41:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.995+0000 7f6f7527a740 1 --2- 172.21.15.43:0/445130476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8a51c4c90 0x55e8a5204e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.995+0000 7f6f7527a740 1 -- 172.21.15.43:0/445130476 >> 172.21.15.43:0/445130476 conn(0x55e8a5208850 msgr2=0x55e8a51fa4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.995+0000 7f6f7527a740 1 -- 172.21.15.43:0/445130476 shutdown_connections 2024-04-21T08:41:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.995+0000 7f6f7527a740 1 -- 172.21.15.43:0/445130476 wait complete. 2024-04-21T08:41:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.995+0000 7f6f7527a740 20 client.17055 trim_cache size 0 max 16384 2024-04-21T08:41:52.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:41:52.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:41:52.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:41:52.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:41:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.996+0000 7f6f72681640 1 -- 172.21.15.43:0/4120400016 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6f540a3dc0 msgr2=0x7f6f540a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.996+0000 7f6f72681640 1 --2- 172.21.15.43:0/4120400016 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6f540a3dc0 0x7f6f540a6280 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6f64005fd0 tx=0x7f6f64009330 comp rx=0 tx=0).stop 2024-04-21T08:41:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.996+0000 7f6f72681640 1 -- 172.21.15.43:0/4120400016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f6c1010f0 msgr2=0x7f6f6c198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.996+0000 7f6f72681640 1 --2- 172.21.15.43:0/4120400016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f6c1010f0 0x7f6f6c198730 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00d770 tx=0x7f6f5c00dc30 comp rx=0 tx=0).stop 2024-04-21T08:41:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.996+0000 7f6f72681640 1 -- 172.21.15.43:0/4120400016 shutdown_connections 2024-04-21T08:41:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.996+0000 7f6f72681640 1 --2- 172.21.15.43:0/4120400016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f6c19cff0 0x7f6f6c19d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.996+0000 7f6f72681640 1 --2- 172.21.15.43:0/4120400016 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6f540a3dc0 0x7f6f540a6280 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.996+0000 7f6f72681640 1 --2- 172.21.15.43:0/4120400016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f6c106170 0x7f6f6c198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.996+0000 7f6f72681640 1 --2- 172.21.15.43:0/4120400016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f6c1010f0 0x7f6f6c198730 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.996+0000 7f6f72681640 1 -- 172.21.15.43:0/4120400016 >> 172.21.15.43:0/4120400016 conn(0x7f6f6c077fd0 msgr2=0x7f6f6c069910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.996+0000 7f6f72681640 1 -- 172.21.15.43:0/4120400016 shutdown_connections 2024-04-21T08:41:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:41:52.996+0000 7f6f72681640 1 -- 172.21.15.43:0/4120400016 wait complete. 2024-04-21T08:41:54.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:54 smithi175 ceph-mon[31920]: pgmap v92: 129 pgs: 129 active+clean; 3.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-21T08:41:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:54 smithi112 ceph-mon[31085]: pgmap v92: 129 pgs: 129 active+clean; 3.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-21T08:41:54.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:54 smithi043 ceph-mon[27978]: pgmap v92: 129 pgs: 129 active+clean; 3.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-21T08:41:56.345 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:56 smithi112 ceph-mon[31085]: pgmap v93: 129 pgs: 129 active+clean; 3.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-21T08:41:56.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:56 smithi175 ceph-mon[31920]: pgmap v93: 129 pgs: 129 active+clean; 3.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-21T08:41:56.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:56 smithi043 ceph-mon[27978]: pgmap v93: 129 pgs: 129 active+clean; 3.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-21T08:41:58.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:41:58 smithi175 ceph-mon[31920]: pgmap v94: 129 pgs: 129 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 23 op/s 2024-04-21T08:41:58.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:41:58 smithi112 ceph-mon[31085]: pgmap v94: 129 pgs: 129 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 23 op/s 2024-04-21T08:41:58.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:41:58 smithi043 ceph-mon[27978]: pgmap v94: 129 pgs: 129 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 23 op/s 2024-04-21T08:42:00.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:00 smithi175 ceph-mon[31920]: pgmap v95: 129 pgs: 129 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-04-21T08:42:00.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:00 smithi112 ceph-mon[31085]: pgmap v95: 129 pgs: 129 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-04-21T08:42:00.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:00 smithi043 ceph-mon[27978]: pgmap v95: 129 pgs: 129 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-04-21T08:42:02.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:02 smithi175 ceph-mon[31920]: pgmap v96: 129 pgs: 129 active+clean; 3.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 53 KiB/s wr, 36 op/s 2024-04-21T08:42:02.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:02 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:42:02.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:02 smithi112 ceph-mon[31085]: pgmap v96: 129 pgs: 129 active+clean; 3.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 53 KiB/s wr, 36 op/s 2024-04-21T08:42:02.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:02 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:42:02.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:02 smithi043 ceph-mon[27978]: pgmap v96: 129 pgs: 129 active+clean; 3.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 53 KiB/s wr, 36 op/s 2024-04-21T08:42:02.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:02 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:42:03.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:03 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:03.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:03 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:42:03.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:03 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:42:03.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:03 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:03.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:03 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:42:03.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:03 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:42:03.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:03 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:03.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:03 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:42:03.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:03 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:42:04.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:04 smithi175 ceph-mon[31920]: pgmap v97: 129 pgs: 129 active+clean; 3.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 43 KiB/s wr, 30 op/s 2024-04-21T08:42:04.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:04 smithi112 ceph-mon[31085]: pgmap v97: 129 pgs: 129 active+clean; 3.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 43 KiB/s wr, 30 op/s 2024-04-21T08:42:04.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:04 smithi043 ceph-mon[27978]: pgmap v97: 129 pgs: 129 active+clean; 3.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 43 KiB/s wr, 30 op/s 2024-04-21T08:42:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.096+0000 7fc09bed2640 1 -- 172.21.15.43:0/3950840784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc094101070 msgr2=0x7fc094106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.096+0000 7fc09bed2640 1 --2- 172.21.15.43:0/3950840784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc094101070 0x7fc094106ba0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fc084009900 tx=0x7fc08402bc30 comp rx=0 tx=0).stop 2024-04-21T08:42:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.096+0000 7fc09bed2640 1 -- 172.21.15.43:0/3950840784 shutdown_connections 2024-04-21T08:42:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.096+0000 7fc09bed2640 1 --2- 172.21.15.43:0/3950840784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc094107280 0x7fc09410ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.096+0000 7fc09bed2640 1 --2- 172.21.15.43:0/3950840784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc094101070 0x7fc094106ba0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.096+0000 7fc09bed2640 1 --2- 172.21.15.43:0/3950840784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0941006c0 0x7fc094100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.096+0000 7fc09bed2640 1 -- 172.21.15.43:0/3950840784 >> 172.21.15.43:0/3950840784 conn(0x7fc094077390 msgr2=0x7fc0940fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.096+0000 7fc09bed2640 1 -- 172.21.15.43:0/3950840784 shutdown_connections 2024-04-21T08:42:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.096+0000 7fc09bed2640 1 -- 172.21.15.43:0/3950840784 wait complete. 2024-04-21T08:42:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.097+0000 7fc09bed2640 1 Processor -- start 2024-04-21T08:42:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.097+0000 7fc09bed2640 1 -- start start 2024-04-21T08:42:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.097+0000 7fc09bed2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0941006c0 0x7fc094198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.097+0000 7fc09bed2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc094107280 0x7fc094198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.097+0000 7fc09bed2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc09419cf30 0x7fc09419d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.097+0000 7fc09bed2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc094112210 con 0x7fc094107280 2024-04-21T08:42:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.097+0000 7fc09bed2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc094112090 con 0x7fc09419cf30 2024-04-21T08:42:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.097+0000 7fc09bed2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc094112390 con 0x7fc0941006c0 2024-04-21T08:42:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.097+0000 7fc09946d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc094107280 0x7fc094198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.097+0000 7fc09946d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc094107280 0x7fc094198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48768/0 (socket says 172.21.15.43:48768) 2024-04-21T08:42:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.097+0000 7fc09946d640 1 -- 172.21.15.43:0/2619159142 learned_addr learned my addr 172.21.15.43:0/2619159142 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.097+0000 7fc09a46f640 1 --2- 172.21.15.43:0/2619159142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc09419cf30 0x7fc09419d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.097+0000 7fc099c6e640 1 --2- 172.21.15.43:0/2619159142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0941006c0 0x7fc094198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.097+0000 7fc09946d640 1 -- 172.21.15.43:0/2619159142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0941006c0 msgr2=0x7fc094198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.098+0000 7fc09946d640 1 --2- 172.21.15.43:0/2619159142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0941006c0 0x7fc094198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.098+0000 7fc09946d640 1 -- 172.21.15.43:0/2619159142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc09419cf30 msgr2=0x7fc09419d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.098+0000 7fc09946d640 1 --2- 172.21.15.43:0/2619159142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc09419cf30 0x7fc09419d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.098+0000 7fc09946d640 1 -- 172.21.15.43:0/2619159142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc09419da20 con 0x7fc094107280 2024-04-21T08:42:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.098+0000 7fc09a46f640 1 --2- 172.21.15.43:0/2619159142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc09419cf30 0x7fc09419d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.098+0000 7fc09946d640 1 --2- 172.21.15.43:0/2619159142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc094107280 0x7fc094198bb0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fc084000c00 tx=0x7fc08402c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.098+0000 7fc08affd640 1 -- 172.21.15.43:0/2619159142 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc084043020 con 0x7fc094107280 2024-04-21T08:42:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.098+0000 7fc08affd640 1 -- 172.21.15.43:0/2619159142 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc08402cc60 con 0x7fc094107280 2024-04-21T08:42:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.098+0000 7fc09bed2640 1 -- 172.21.15.43:0/2619159142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc09419dcb0 con 0x7fc094107280 2024-04-21T08:42:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.098+0000 7fc08affd640 1 -- 172.21.15.43:0/2619159142 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc084003ca0 con 0x7fc094107280 2024-04-21T08:42:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.098+0000 7fc09bed2640 1 -- 172.21.15.43:0/2619159142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0941a54d0 con 0x7fc094107280 2024-04-21T08:42:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.100+0000 7fc08affd640 1 -- 172.21.15.43:0/2619159142 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc084046020 con 0x7fc094107280 2024-04-21T08:42:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.101+0000 7fc09eba4740 1 Processor -- start 2024-04-21T08:42:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.101+0000 7fc09eba4740 1 -- start start 2024-04-21T08:42:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.101+0000 7fc09eba4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c837a1c8d0 0x55c837a1b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.101+0000 7fc09eba4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c837a1bb60 0x55c837a1bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.101+0000 7fc09eba4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a250e0 0x55c837a274d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.101+0000 7fc09eba4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c8379808d0 con 0x55c837a250e0 2024-04-21T08:42:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.101+0000 7fc09eba4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c83791de80 con 0x55c837a1bb60 2024-04-21T08:42:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.101+0000 7fc09eba4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c83797c890 con 0x55c837a1c8d0 2024-04-21T08:42:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.101+0000 7fc09946d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a250e0 0x55c837a274d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.101+0000 7fc09946d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a250e0 0x55c837a274d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48770/0 (socket says 172.21.15.43:48770) 2024-04-21T08:42:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.101+0000 7fc09946d640 1 -- 172.21.15.43:0/2413679122 learned_addr learned my addr 172.21.15.43:0/2413679122 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.101+0000 7fc09a46f640 1 --2- 172.21.15.43:0/2413679122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c837a1c8d0 0x55c837a1b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.101+0000 7fc099c6e640 1 --2- 172.21.15.43:0/2413679122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c837a1bb60 0x55c837a1bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.101+0000 7fc08affd640 1 --2- 172.21.15.43:0/2619159142 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc0800a3dc0 0x7fc0800a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.102+0000 7fc09946d640 1 -- 172.21.15.43:0/2413679122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c837a1c8d0 msgr2=0x55c837a1b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.102+0000 7fc09946d640 1 --2- 172.21.15.43:0/2413679122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c837a1c8d0 0x55c837a1b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.102+0000 7fc09946d640 1 -- 172.21.15.43:0/2413679122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c837a1bb60 msgr2=0x55c837a1bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.102+0000 7fc09946d640 1 --2- 172.21.15.43:0/2413679122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c837a1bb60 0x55c837a1bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.102+0000 7fc08affd640 1 -- 172.21.15.43:0/2619159142 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc0840f2d90 con 0x7fc094107280 2024-04-21T08:42:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.102+0000 7fc09946d640 1 -- 172.21.15.43:0/2413679122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c83793f320 con 0x55c837a250e0 2024-04-21T08:42:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.102+0000 7fc09a46f640 1 --2- 172.21.15.43:0/2413679122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c837a1c8d0 0x55c837a1b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:42:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.102+0000 7fc09a46f640 1 --2- 172.21.15.43:0/2619159142 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc0800a3dc0 0x7fc0800a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.102+0000 7fc09946d640 1 --2- 172.21.15.43:0/2413679122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a250e0 0x55c837a274d0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc0840be770 tx=0x7fc084009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bdb8800040e0049 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.102+0000 7fc07ffff640 1 -- 172.21.15.43:0/2413679122 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0840be5b0 con 0x55c837a250e0 2024-04-21T08:42:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.102+0000 7fc07ffff640 1 -- 172.21.15.43:0/2413679122 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc084003e40 con 0x55c837a250e0 2024-04-21T08:42:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.102+0000 7fc07ffff640 1 -- 172.21.15.43:0/2413679122 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0840df0f0 con 0x55c837a250e0 2024-04-21T08:42:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.103+0000 7fc09eba4740 1 -- 172.21.15.43:0/2413679122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a250e0 msgr2=0x55c837a274d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.103+0000 7fc09eba4740 1 --2- 172.21.15.43:0/2413679122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a250e0 0x55c837a274d0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc0840be770 tx=0x7fc084009260 comp rx=0 tx=0).stop 2024-04-21T08:42:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.103+0000 7fc09a46f640 1 --2- 172.21.15.43:0/2619159142 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc0800a3dc0 0x7fc0800a6280 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc08c005fd0 tx=0x7fc08c004a40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.103+0000 7fc09eba4740 1 -- 172.21.15.43:0/2413679122 shutdown_connections 2024-04-21T08:42:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.103+0000 7fc09eba4740 1 --2- 172.21.15.43:0/2413679122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a250e0 0x55c837a274d0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.103+0000 7fc09eba4740 1 --2- 172.21.15.43:0/2413679122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c837a1bb60 0x55c837a1bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.103+0000 7fc09eba4740 1 --2- 172.21.15.43:0/2413679122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c837a1c8d0 0x55c837a1b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.103+0000 7fc09eba4740 1 -- 172.21.15.43:0/2413679122 >> 172.21.15.43:0/2413679122 conn(0x55c8379e2c90 msgr2=0x55c837a16850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.103+0000 7fc09eba4740 1 -- 172.21.15.43:0/2413679122 shutdown_connections 2024-04-21T08:42:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.103+0000 7fc09eba4740 1 -- 172.21.15.43:0/2413679122 wait complete. 2024-04-21T08:42:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.103+0000 7fc09eba4740 1 -- start start 2024-04-21T08:42:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.104+0000 7fc09eba4740 1 Processor -- start 2024-04-21T08:42:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.104+0000 7fc09eba4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c837a1bb60 0x55c837a24aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.104+0000 7fc09eba4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a1c8d0 0x55c837a21e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.104+0000 7fc09eba4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c837a250e0 0x55c837a22440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.104+0000 7fc09eba4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c837a27bd0 con 0x55c837a1c8d0 2024-04-21T08:42:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.104+0000 7fc09eba4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c837908eb0 con 0x55c837a1bb60 2024-04-21T08:42:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.104+0000 7fc09eba4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c837a27d50 con 0x55c837a250e0 2024-04-21T08:42:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.104+0000 7fc09eba4740 10 client.0 _ll_register_callbacks cb 0x55c837935b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.104+0000 7fc09946d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a1c8d0 0x55c837a21e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.104+0000 7fc09946d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a1c8d0 0x55c837a21e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48776/0 (socket says 172.21.15.43:48776) 2024-04-21T08:42:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.104+0000 7fc09946d640 1 -- 172.21.15.43:0/162694238 learned_addr learned my addr 172.21.15.43:0/162694238 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.104+0000 7fc099c6e640 1 --2- 172.21.15.43:0/162694238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c837a1bb60 0x55c837a24aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.105+0000 7fc09946d640 1 -- 172.21.15.43:0/162694238 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c837a250e0 msgr2=0x55c837a22440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.105+0000 7fc09946d640 1 --2- 172.21.15.43:0/162694238 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c837a250e0 0x55c837a22440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.105+0000 7fc09946d640 1 -- 172.21.15.43:0/162694238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c837a1bb60 msgr2=0x55c837a24aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.105+0000 7fc09946d640 1 --2- 172.21.15.43:0/162694238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c837a1bb60 0x55c837a24aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.105+0000 7fc09946d640 1 -- 172.21.15.43:0/162694238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c837693210 con 0x55c837a1c8d0 2024-04-21T08:42:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.105+0000 7fc09946d640 1 --2- 172.21.15.43:0/162694238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a1c8d0 0x55c837a21e30 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc0840f0710 tx=0x7fc0840f3100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af379c6c1b31a5b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.105+0000 7fc07d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.105+0000 7fc07d7fa640 1 -- 172.21.15.43:0/162694238 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0840be3d0 con 0x55c837a1c8d0 2024-04-21T08:42:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.105+0000 7fc07d7fa640 1 -- 172.21.15.43:0/162694238 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc0840c7460 con 0x55c837a1c8d0 2024-04-21T08:42:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.105+0000 7fc09eba4740 1 -- 172.21.15.43:0/162694238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c837994880 con 0x55c837a1c8d0 2024-04-21T08:42:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.105+0000 7fc07d7fa640 1 -- 172.21.15.43:0/162694238 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc0840035f0 con 0x55c837a1c8d0 2024-04-21T08:42:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.105+0000 7fc09eba4740 10 client.17067 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.105+0000 7fc09eba4740 1 -- 172.21.15.43:0/162694238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c83775f620 con 0x55c837a1c8d0 2024-04-21T08:42:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.105+0000 7fc07d7fa640 1 -- 172.21.15.43:0/162694238 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c837994880 con 0x55c837a1c8d0 2024-04-21T08:42:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.105+0000 7fc07d7fa640 1 -- 172.21.15.43:0/162694238 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc0840f63a0 con 0x55c837a1c8d0 2024-04-21T08:42:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.105+0000 7fc09eba4740 10 client.17067 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.105+0000 7fc09eba4740 10 client.17067 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.106+0000 7fc09eba4740 20 client.17067 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.106+0000 7fc09eba4740 1 --2- 172.21.15.43:0/162694238 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c837a34ca0 0x55c837a37090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.106+0000 7fc09eba4740 4 client.17067 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.106+0000 7fc09eba4740 1 -- 172.21.15.43:0/162694238 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8379203d0 con 0x55c837a34ca0 2024-04-21T08:42:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.106+0000 7fc099c6e640 1 --2- 172.21.15.43:0/162694238 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c837a34ca0 0x55c837a37090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.106+0000 7fc099c6e640 1 --2- 172.21.15.43:0/162694238 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c837a34ca0 0x55c837a37090 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6087f5de7a1f576e server_cookie=178ce9376a132a5a in_seq=0 out_seq=0 2024-04-21T08:42:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.106+0000 7fc07d7fa640 10 client.17067 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.107+0000 7fc07d7fa640 1 -- 172.21.15.43:0/162694238 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c8379203d0 con 0x55c837a34ca0 2024-04-21T08:42:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.107+0000 7fc07d7fa640 10 client.17067 handle_command_reply: tid=0 2024-04-21T08:42:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.107+0000 7fc09eba4740 1 client.17067 shutdown 2024-04-21T08:42:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.107+0000 7fc09eba4740 1 -- 172.21.15.43:0/162694238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a1c8d0 msgr2=0x55c837a21e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.107+0000 7fc09eba4740 1 --2- 172.21.15.43:0/162694238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a1c8d0 0x55c837a21e30 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc0840f0710 tx=0x7fc0840f3100 comp rx=0 tx=0).stop 2024-04-21T08:42:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.107+0000 7fc09eba4740 1 -- 172.21.15.43:0/162694238 shutdown_connections 2024-04-21T08:42:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.107+0000 7fc09eba4740 1 --2- 172.21.15.43:0/162694238 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c837a34ca0 0x55c837a37090 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.107+0000 7fc09eba4740 1 --2- 172.21.15.43:0/162694238 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c837a250e0 0x55c837a22440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.107+0000 7fc09eba4740 1 --2- 172.21.15.43:0/162694238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a1c8d0 0x55c837a21e30 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.107+0000 7fc07d7fa640 0 client.17067 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.107+0000 7fc09eba4740 1 --2- 172.21.15.43:0/162694238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c837a1bb60 0x55c837a24aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.107+0000 7fc09eba4740 1 -- 172.21.15.43:0/162694238 >> 172.21.15.43:0/162694238 conn(0x55c8379e2c90 msgr2=0x55c837a185a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.107+0000 7fc09eba4740 1 -- 172.21.15.43:0/162694238 shutdown_connections 2024-04-21T08:42:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.107+0000 7fc09eba4740 1 -- 172.21.15.43:0/162694238 wait complete. 2024-04-21T08:42:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.108+0000 7fc09eba4740 20 client.17067 trim_cache size 0 max 16384 2024-04-21T08:42:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.123+0000 7fc09eba4740 1 Processor -- start 2024-04-21T08:42:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.123+0000 7fc09eba4740 1 -- start start 2024-04-21T08:42:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.123+0000 7fc09eba4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c8379e2c90 0x55c837a15750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.123+0000 7fc09eba4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c837a1bb60 0x55c837a24560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.123+0000 7fc09eba4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a1c8d0 0x55c837a24aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.123+0000 7fc09eba4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc0941af290 con 0x55c837a1c8d0 2024-04-21T08:42:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.123+0000 7fc09eba4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0941b0dd0 con 0x55c8379e2c90 2024-04-21T08:42:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.123+0000 7fc09eba4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc0941af960 con 0x55c837a1bb60 2024-04-21T08:42:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.124+0000 7fc09a46f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a1c8d0 0x55c837a24aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.124+0000 7fc09a46f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a1c8d0 0x55c837a24aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48792/0 (socket says 172.21.15.43:48792) 2024-04-21T08:42:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.124+0000 7fc09a46f640 1 -- 172.21.15.43:0/1169929570 learned_addr learned my addr 172.21.15.43:0/1169929570 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.124+0000 7fc09946d640 1 --2- 172.21.15.43:0/1169929570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c837a1bb60 0x55c837a24560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.124+0000 7fc099c6e640 1 --2- 172.21.15.43:0/1169929570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c8379e2c90 0x55c837a15750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.124+0000 7fc09a46f640 1 -- 172.21.15.43:0/1169929570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c837a1bb60 msgr2=0x55c837a24560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.124+0000 7fc09a46f640 1 --2- 172.21.15.43:0/1169929570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c837a1bb60 0x55c837a24560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.124+0000 7fc09a46f640 1 -- 172.21.15.43:0/1169929570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c8379e2c90 msgr2=0x55c837a15750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.124+0000 7fc09a46f640 1 --2- 172.21.15.43:0/1169929570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c8379e2c90 0x55c837a15750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.124+0000 7fc09a46f640 1 -- 172.21.15.43:0/1169929570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c837a376e0 con 0x55c837a1c8d0 2024-04-21T08:42:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.124+0000 7fc09a46f640 1 --2- 172.21.15.43:0/1169929570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a1c8d0 0x55c837a24aa0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc08c010730 tx=0x7fc08c015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d894252869e25c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.124+0000 7fc07f7fe640 1 -- 172.21.15.43:0/1169929570 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c01d0d0 con 0x55c837a1c8d0 2024-04-21T08:42:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.124+0000 7fc07f7fe640 1 -- 172.21.15.43:0/1169929570 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc08c01d270 con 0x55c837a1c8d0 2024-04-21T08:42:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.124+0000 7fc07f7fe640 1 -- 172.21.15.43:0/1169929570 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c01b7a0 con 0x55c837a1c8d0 2024-04-21T08:42:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.125+0000 7fc09eba4740 1 -- 172.21.15.43:0/1169929570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a1c8d0 msgr2=0x55c837a24aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.125+0000 7fc09eba4740 1 --2- 172.21.15.43:0/1169929570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a1c8d0 0x55c837a24aa0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc08c010730 tx=0x7fc08c015040 comp rx=0 tx=0).stop 2024-04-21T08:42:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.125+0000 7fc09eba4740 1 -- 172.21.15.43:0/1169929570 shutdown_connections 2024-04-21T08:42:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.125+0000 7fc09eba4740 1 --2- 172.21.15.43:0/1169929570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a1c8d0 0x55c837a24aa0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.125+0000 7fc09eba4740 1 --2- 172.21.15.43:0/1169929570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c837a1bb60 0x55c837a24560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.125+0000 7fc09eba4740 1 --2- 172.21.15.43:0/1169929570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c8379e2c90 0x55c837a15750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.125+0000 7fc09eba4740 1 -- 172.21.15.43:0/1169929570 >> 172.21.15.43:0/1169929570 conn(0x55c837a27840 msgr2=0x55c837a15040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.125+0000 7fc09eba4740 1 -- 172.21.15.43:0/1169929570 shutdown_connections 2024-04-21T08:42:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.125+0000 7fc09eba4740 1 -- 172.21.15.43:0/1169929570 wait complete. 2024-04-21T08:42:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.125+0000 7fc09eba4740 1 -- start start 2024-04-21T08:42:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.125+0000 7fc09eba4740 1 Processor -- start 2024-04-21T08:42:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.126+0000 7fc09eba4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8379e2c90 0x55c837a2ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.126+0000 7fc09eba4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c837a1bb60 0x55c837a2d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.126+0000 7fc09eba4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a1c8d0 0x55c837a29130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.126+0000 7fc09eba4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc0941b0700 con 0x55c837a1c8d0 2024-04-21T08:42:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.126+0000 7fc09eba4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0941b0030 con 0x55c837a1bb60 2024-04-21T08:42:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.126+0000 7fc09eba4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc0941ad270 con 0x55c8379e2c90 2024-04-21T08:42:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.126+0000 7fc09eba4740 10 client.0 _ll_register_callbacks cb 0x7fc09000b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.126+0000 7fc09a46f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a1c8d0 0x55c837a29130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.126+0000 7fc09a46f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a1c8d0 0x55c837a29130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48804/0 (socket says 172.21.15.43:48804) 2024-04-21T08:42:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.126+0000 7fc09a46f640 1 -- 172.21.15.43:0/669737590 learned_addr learned my addr 172.21.15.43:0/669737590 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.126+0000 7fc099c6e640 1 --2- 172.21.15.43:0/669737590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8379e2c90 0x55c837a2ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.126+0000 7fc09946d640 1 --2- 172.21.15.43:0/669737590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c837a1bb60 0x55c837a2d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.126+0000 7fc09a46f640 1 -- 172.21.15.43:0/669737590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8379e2c90 msgr2=0x55c837a2ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.126+0000 7fc09a46f640 1 --2- 172.21.15.43:0/669737590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8379e2c90 0x55c837a2ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.126+0000 7fc09a46f640 1 -- 172.21.15.43:0/669737590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c837a1bb60 msgr2=0x55c837a2d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.126+0000 7fc09a46f640 1 --2- 172.21.15.43:0/669737590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c837a1bb60 0x55c837a2d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.126+0000 7fc09a46f640 1 -- 172.21.15.43:0/669737590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c837a1ae30 con 0x55c837a1c8d0 2024-04-21T08:42:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.126+0000 7fc09946d640 1 --2- 172.21.15.43:0/669737590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c837a1bb60 0x55c837a2d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.127+0000 7fc09a46f640 1 --2- 172.21.15.43:0/669737590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a1c8d0 0x55c837a29130 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0132a0 tx=0x7fc08c0132d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618717e35247bce6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.127+0000 7fc07d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.127+0000 7fc07d7fa640 1 -- 172.21.15.43:0/669737590 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c034060 con 0x55c837a1c8d0 2024-04-21T08:42:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.127+0000 7fc07d7fa640 1 -- 172.21.15.43:0/669737590 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc08c023040 con 0x55c837a1c8d0 2024-04-21T08:42:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.127+0000 7fc09eba4740 1 -- 172.21.15.43:0/669737590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc08c01d270 con 0x55c837a1c8d0 2024-04-21T08:42:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.127+0000 7fc07d7fa640 1 -- 172.21.15.43:0/669737590 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc08c03b050 con 0x55c837a1c8d0 2024-04-21T08:42:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.127+0000 7fc09eba4740 10 client.17073 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.127+0000 7fc09eba4740 1 -- 172.21.15.43:0/669737590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c837a9ba80 con 0x55c837a1c8d0 2024-04-21T08:42:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.127+0000 7fc07d7fa640 1 -- 172.21.15.43:0/669737590 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc08c01d270 con 0x55c837a1c8d0 2024-04-21T08:42:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.127+0000 7fc07d7fa640 1 -- 172.21.15.43:0/669737590 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc08c00f040 con 0x55c837a1c8d0 2024-04-21T08:42:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.127+0000 7fc09eba4740 10 client.17073 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.127+0000 7fc09eba4740 10 client.17073 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.127+0000 7fc09eba4740 20 client.17073 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.127+0000 7fc09eba4740 1 --2- 172.21.15.43:0/669737590 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c837a250e0 0x55c837a9e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.127+0000 7fc09eba4740 4 client.17073 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:42:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.127+0000 7fc09eba4740 1 -- 172.21.15.43:0/669737590 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c83798e2f0 con 0x55c837a250e0 2024-04-21T08:42:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.128+0000 7fc099c6e640 1 --2- 172.21.15.43:0/669737590 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c837a250e0 0x55c837a9e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.128+0000 7fc099c6e640 1 --2- 172.21.15.43:0/669737590 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c837a250e0 0x55c837a9e160 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97ff2d86841163d5 server_cookie=6bf6caa17f08ee2c in_seq=0 out_seq=0 2024-04-21T08:42:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.128+0000 7fc07d7fa640 10 client.17073 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.143+0000 7fc07d7fa640 1 -- 172.21.15.43:0/669737590 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c83798e2f0 con 0x55c837a250e0 2024-04-21T08:42:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.143+0000 7fc07d7fa640 10 client.17073 handle_command_reply: tid=0 2024-04-21T08:42:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.143+0000 7fc09eba4740 1 client.17073 shutdown 2024-04-21T08:42:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.143+0000 7fc09eba4740 1 -- 172.21.15.43:0/669737590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a1c8d0 msgr2=0x55c837a29130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.143+0000 7fc09eba4740 1 --2- 172.21.15.43:0/669737590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a1c8d0 0x55c837a29130 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0132a0 tx=0x7fc08c0132d0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.143+0000 7fc09eba4740 1 -- 172.21.15.43:0/669737590 shutdown_connections 2024-04-21T08:42:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.143+0000 7fc09eba4740 1 --2- 172.21.15.43:0/669737590 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c837a250e0 0x55c837a9e160 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.143+0000 7fc09eba4740 1 --2- 172.21.15.43:0/669737590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c837a1c8d0 0x55c837a29130 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.143+0000 7fc09eba4740 1 --2- 172.21.15.43:0/669737590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c837a1bb60 0x55c837a2d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.143+0000 7fc09eba4740 1 --2- 172.21.15.43:0/669737590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c8379e2c90 0x55c837a2ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.143+0000 7fc07d7fa640 0 client.17073 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.143+0000 7fc09eba4740 1 -- 172.21.15.43:0/669737590 >> 172.21.15.43:0/669737590 conn(0x55c837a27840 msgr2=0x55c837a158d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.143+0000 7fc09eba4740 1 -- 172.21.15.43:0/669737590 shutdown_connections 2024-04-21T08:42:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.143+0000 7fc09eba4740 1 -- 172.21.15.43:0/669737590 wait complete. 2024-04-21T08:42:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.143+0000 7fc09eba4740 20 client.17073 trim_cache size 0 max 16384 2024-04-21T08:42:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:42:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:42:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:42:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:42:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.143+0000 7fc09bed2640 1 -- 172.21.15.43:0/2619159142 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc0800a3dc0 msgr2=0x7fc0800a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.143+0000 7fc09bed2640 1 --2- 172.21.15.43:0/2619159142 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc0800a3dc0 0x7fc0800a6280 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc08c005fd0 tx=0x7fc08c004a40 comp rx=0 tx=0).stop 2024-04-21T08:42:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.143+0000 7fc09bed2640 1 -- 172.21.15.43:0/2619159142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc094107280 msgr2=0x7fc094198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.143+0000 7fc09bed2640 1 --2- 172.21.15.43:0/2619159142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc094107280 0x7fc094198bb0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fc084000c00 tx=0x7fc08402c860 comp rx=0 tx=0).stop 2024-04-21T08:42:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.144+0000 7fc09bed2640 1 -- 172.21.15.43:0/2619159142 shutdown_connections 2024-04-21T08:42:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.144+0000 7fc09bed2640 1 --2- 172.21.15.43:0/2619159142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc09419cf30 0x7fc09419d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.144+0000 7fc09bed2640 1 --2- 172.21.15.43:0/2619159142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc094107280 0x7fc094198bb0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.144+0000 7fc09bed2640 1 --2- 172.21.15.43:0/2619159142 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc0800a3dc0 0x7fc0800a6280 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.144+0000 7fc09bed2640 1 --2- 172.21.15.43:0/2619159142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0941006c0 0x7fc094198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.144+0000 7fc09bed2640 1 -- 172.21.15.43:0/2619159142 >> 172.21.15.43:0/2619159142 conn(0x7fc094077390 msgr2=0x7fc094105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.144+0000 7fc09bed2640 1 -- 172.21.15.43:0/2619159142 shutdown_connections 2024-04-21T08:42:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:06.144+0000 7fc09bed2640 1 -- 172.21.15.43:0/2619159142 wait complete. 2024-04-21T08:42:06.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:06 smithi043 ceph-mon[27978]: pgmap v98: 129 pgs: 129 active+clean; 3.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 43 KiB/s wr, 30 op/s 2024-04-21T08:42:06.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:06 smithi175 ceph-mon[31920]: pgmap v98: 129 pgs: 129 active+clean; 3.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 43 KiB/s wr, 30 op/s 2024-04-21T08:42:06.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:06 smithi112 ceph-mon[31085]: pgmap v98: 129 pgs: 129 active+clean; 3.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 43 KiB/s wr, 30 op/s 2024-04-21T08:42:08.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:08 smithi175 ceph-mon[31920]: pgmap v99: 129 pgs: 129 active+clean; 3.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 55 KiB/s wr, 36 op/s 2024-04-21T08:42:08.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:08 smithi112 ceph-mon[31085]: pgmap v99: 129 pgs: 129 active+clean; 3.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 55 KiB/s wr, 36 op/s 2024-04-21T08:42:08.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:08 smithi043 ceph-mon[27978]: pgmap v99: 129 pgs: 129 active+clean; 3.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 55 KiB/s wr, 36 op/s 2024-04-21T08:42:10.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:10 smithi175 ceph-mon[31920]: pgmap v100: 129 pgs: 129 active+clean; 3.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 KiB/s wr, 27 op/s 2024-04-21T08:42:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:10 smithi112 ceph-mon[31085]: pgmap v100: 129 pgs: 129 active+clean; 3.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 KiB/s wr, 27 op/s 2024-04-21T08:42:10.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:10 smithi043 ceph-mon[27978]: pgmap v100: 129 pgs: 129 active+clean; 3.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 KiB/s wr, 27 op/s 2024-04-21T08:42:12.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:12 smithi175 ceph-mon[31920]: pgmap v101: 129 pgs: 129 active+clean; 3.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 58 KiB/s wr, 36 op/s 2024-04-21T08:42:12.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:12 smithi112 ceph-mon[31085]: pgmap v101: 129 pgs: 129 active+clean; 3.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 58 KiB/s wr, 36 op/s 2024-04-21T08:42:12.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:12 smithi043 ceph-mon[27978]: pgmap v101: 129 pgs: 129 active+clean; 3.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 58 KiB/s wr, 36 op/s 2024-04-21T08:42:13.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:42:13.842 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:42:13.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.959+0000 7fe416fef640 1 -- 172.21.15.43:0/2381445117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4101021a0 msgr2=0x7fe410102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:13.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.959+0000 7fe416fef640 1 --2- 172.21.15.43:0/2381445117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4101021a0 0x7fe410102580 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe400009900 tx=0x7fe40002bc50 comp rx=0 tx=0).stop 2024-04-21T08:42:13.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.959+0000 7fe416fef640 1 -- 172.21.15.43:0/2381445117 shutdown_connections 2024-04-21T08:42:13.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.959+0000 7fe416fef640 1 --2- 172.21.15.43:0/2381445117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe410107270 0x7fe41010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.959+0000 7fe416fef640 1 --2- 172.21.15.43:0/2381445117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe410102b50 0x7fe410106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.959+0000 7fe416fef640 1 --2- 172.21.15.43:0/2381445117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4101021a0 0x7fe410102580 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.959+0000 7fe416fef640 1 -- 172.21.15.43:0/2381445117 >> 172.21.15.43:0/2381445117 conn(0x7fe4100793e0 msgr2=0x7fe4100797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:13.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.959+0000 7fe416fef640 1 -- 172.21.15.43:0/2381445117 shutdown_connections 2024-04-21T08:42:13.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.959+0000 7fe416fef640 1 -- 172.21.15.43:0/2381445117 wait complete. 2024-04-21T08:42:13.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.960+0000 7fe416fef640 1 Processor -- start 2024-04-21T08:42:13.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.960+0000 7fe416fef640 1 -- start start 2024-04-21T08:42:13.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.960+0000 7fe416fef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4101021a0 0x7fe410074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:13.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.960+0000 7fe416fef640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe410107270 0x7fe410074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:13.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.960+0000 7fe416fef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe410074cd0 0x7fe4101a6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:13.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.960+0000 7fe416fef640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe410112250 con 0x7fe4101021a0 2024-04-21T08:42:13.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.960+0000 7fe416fef640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101120d0 con 0x7fe410074cd0 2024-04-21T08:42:13.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.960+0000 7fe416fef640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101123d0 con 0x7fe410107270 2024-04-21T08:42:13.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.960+0000 7fe414d8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4101021a0 0x7fe410074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:13.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.960+0000 7fe414d8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4101021a0 0x7fe410074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36556/0 (socket says 172.21.15.43:36556) 2024-04-21T08:42:13.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.960+0000 7fe414d8b640 1 -- 172.21.15.43:0/3455118788 learned_addr learned my addr 172.21.15.43:0/3455118788 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:13.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.960+0000 7fe41558c640 1 --2- 172.21.15.43:0/3455118788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe410074cd0 0x7fe4101a6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:13.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.960+0000 7fe414d8b640 1 -- 172.21.15.43:0/3455118788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe410107270 msgr2=0x7fe410074790 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:42:13.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.960+0000 7fe414d8b640 1 --2- 172.21.15.43:0/3455118788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe410107270 0x7fe410074790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.960+0000 7fe414d8b640 1 -- 172.21.15.43:0/3455118788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe410074cd0 msgr2=0x7fe4101a6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:13.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.960+0000 7fe414d8b640 1 --2- 172.21.15.43:0/3455118788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe410074cd0 0x7fe4101a6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.960+0000 7fe414d8b640 1 -- 172.21.15.43:0/3455118788 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4101a7080 con 0x7fe4101021a0 2024-04-21T08:42:13.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.961+0000 7fe414d8b640 1 --2- 172.21.15.43:0/3455118788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4101021a0 0x7fe410074250 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe400005ec0 tx=0x7fe40002c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:13.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.961+0000 7fe40dffb640 1 -- 172.21.15.43:0/3455118788 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400043020 con 0x7fe4101021a0 2024-04-21T08:42:13.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.961+0000 7fe40dffb640 1 -- 172.21.15.43:0/3455118788 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe40002cc80 con 0x7fe4101021a0 2024-04-21T08:42:13.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.961+0000 7fe416fef640 1 -- 172.21.15.43:0/3455118788 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4101a7310 con 0x7fe4101021a0 2024-04-21T08:42:13.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.961+0000 7fe40dffb640 1 -- 172.21.15.43:0/3455118788 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400003c30 con 0x7fe4101021a0 2024-04-21T08:42:13.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.961+0000 7fe416fef640 1 -- 172.21.15.43:0/3455118788 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4101a78b0 con 0x7fe4101021a0 2024-04-21T08:42:13.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.964+0000 7fe40dffb640 1 -- 172.21.15.43:0/3455118788 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe400046020 con 0x7fe4101021a0 2024-04-21T08:42:13.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.964+0000 7fe419bd6740 1 Processor -- start 2024-04-21T08:42:13.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.964+0000 7fe419bd6740 1 -- start start 2024-04-21T08:42:13.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.964+0000 7fe419bd6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d0669e580 0x556d0669e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:13.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.964+0000 7fe419bd6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d0669eea0 0x556d066a7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:13.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.965+0000 7fe419bd6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d066a7a80 0x556d066a9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:13.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.965+0000 7fe419bd6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556d066031e0 con 0x556d0669eea0 2024-04-21T08:42:13.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.965+0000 7fe419bd6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556d065a0390 con 0x556d066a7a80 2024-04-21T08:42:13.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.965+0000 7fe419bd6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556d065ff1a0 con 0x556d0669e580 2024-04-21T08:42:13.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.965+0000 7fe414d8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d0669eea0 0x556d066a7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:13.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.965+0000 7fe414d8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d0669eea0 0x556d066a7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36560/0 (socket says 172.21.15.43:36560) 2024-04-21T08:42:13.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.965+0000 7fe414d8b640 1 -- 172.21.15.43:0/735235254 learned_addr learned my addr 172.21.15.43:0/735235254 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:13.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.965+0000 7fe40dffb640 1 --2- 172.21.15.43:0/3455118788 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe3fc0a3dd0 0x7fe3fc0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:13.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.965+0000 7fe40ffff640 1 --2- 172.21.15.43:0/735235254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d066a7a80 0x556d066a9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:13.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.965+0000 7fe41558c640 1 --2- 172.21.15.43:0/735235254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d0669e580 0x556d0669e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:13.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.965+0000 7fe41558c640 1 --2- 172.21.15.43:0/3455118788 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe3fc0a3dd0 0x7fe3fc0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:13.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.965+0000 7fe40dffb640 1 -- 172.21.15.43:0/3455118788 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fe4000f2dc0 con 0x7fe4101021a0 2024-04-21T08:42:13.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.965+0000 7fe414d8b640 1 -- 172.21.15.43:0/735235254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d0669e580 msgr2=0x556d0669e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:13.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.965+0000 7fe414d8b640 1 --2- 172.21.15.43:0/735235254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d0669e580 0x556d0669e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.965+0000 7fe414d8b640 1 -- 172.21.15.43:0/735235254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d066a7a80 msgr2=0x556d066a9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:13.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.965+0000 7fe414d8b640 1 --2- 172.21.15.43:0/735235254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d066a7a80 0x556d066a9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.965+0000 7fe414d8b640 1 -- 172.21.15.43:0/735235254 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d063df7d0 con 0x556d0669eea0 2024-04-21T08:42:13.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.965+0000 7fe414d8b640 1 --2- 172.21.15.43:0/735235254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d0669eea0 0x556d066a7540 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fe4000be7a0 tx=0x7fe40002bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c8c0554876d254 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:13.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.965+0000 7fe41558c640 1 --2- 172.21.15.43:0/3455118788 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe3fc0a3dd0 0x7fe3fc0a6290 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe408009fd0 tx=0x7fe4080093d0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:13.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.966+0000 7fe406ffd640 1 -- 172.21.15.43:0/735235254 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4000be5e0 con 0x556d0669eea0 2024-04-21T08:42:13.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.966+0000 7fe406ffd640 1 -- 172.21.15.43:0/735235254 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4000bf1a0 con 0x556d0669eea0 2024-04-21T08:42:13.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.966+0000 7fe406ffd640 1 -- 172.21.15.43:0/735235254 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4000be400 con 0x556d0669eea0 2024-04-21T08:42:13.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.966+0000 7fe419bd6740 1 -- 172.21.15.43:0/735235254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d0669eea0 msgr2=0x556d066a7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:13.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.966+0000 7fe419bd6740 1 --2- 172.21.15.43:0/735235254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d0669eea0 0x556d066a7540 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fe4000be7a0 tx=0x7fe40002bc50 comp rx=0 tx=0).stop 2024-04-21T08:42:13.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.966+0000 7fe419bd6740 1 -- 172.21.15.43:0/735235254 shutdown_connections 2024-04-21T08:42:13.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.966+0000 7fe419bd6740 1 --2- 172.21.15.43:0/735235254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d066a7a80 0x556d066a9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.966+0000 7fe419bd6740 1 --2- 172.21.15.43:0/735235254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d0669eea0 0x556d066a7540 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.966+0000 7fe419bd6740 1 --2- 172.21.15.43:0/735235254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d0669e580 0x556d0669e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.966+0000 7fe419bd6740 1 -- 172.21.15.43:0/735235254 >> 172.21.15.43:0/735235254 conn(0x556d06698cb0 msgr2=0x556d06699090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:13.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.966+0000 7fe419bd6740 1 -- 172.21.15.43:0/735235254 shutdown_connections 2024-04-21T08:42:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.966+0000 7fe419bd6740 1 -- 172.21.15.43:0/735235254 wait complete. 2024-04-21T08:42:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.967+0000 7fe419bd6740 1 -- start start 2024-04-21T08:42:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.967+0000 7fe419bd6740 1 Processor -- start 2024-04-21T08:42:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.967+0000 7fe419bd6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d0669e580 0x556d066a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.967+0000 7fe419bd6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d0669eea0 0x556d066a5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.967+0000 7fe419bd6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d066a7a80 0x556d066a5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.967+0000 7fe419bd6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556d066031e0 con 0x556d066a7a80 2024-04-21T08:42:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.967+0000 7fe419bd6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556d065bc2f0 con 0x556d0669e580 2024-04-21T08:42:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.967+0000 7fe419bd6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556d066aa630 con 0x556d0669eea0 2024-04-21T08:42:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.967+0000 7fe419bd6740 10 client.0 _ll_register_callbacks cb 0x556d065c9580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.967+0000 7fe41558c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d066a7a80 0x556d066a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.967+0000 7fe41558c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d066a7a80 0x556d066a5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36562/0 (socket says 172.21.15.43:36562) 2024-04-21T08:42:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.967+0000 7fe41558c640 1 -- 172.21.15.43:0/927205854 learned_addr learned my addr 172.21.15.43:0/927205854 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.968+0000 7fe414d8b640 1 --2- 172.21.15.43:0/927205854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d0669e580 0x556d066a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.968+0000 7fe41558c640 1 -- 172.21.15.43:0/927205854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d0669eea0 msgr2=0x556d066a5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.968+0000 7fe41558c640 1 --2- 172.21.15.43:0/927205854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d0669eea0 0x556d066a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.968+0000 7fe41558c640 1 -- 172.21.15.43:0/927205854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d0669e580 msgr2=0x556d066a4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.968+0000 7fe41558c640 1 --2- 172.21.15.43:0/927205854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d0669e580 0x556d066a4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.968+0000 7fe41558c640 1 -- 172.21.15.43:0/927205854 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d063153f0 con 0x556d066a7a80 2024-04-21T08:42:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.968+0000 7fe41558c640 1 --2- 172.21.15.43:0/927205854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d066a7a80 0x556d066a5570 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fe408009da0 tx=0x7fe408004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c422309c30fd23 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.968+0000 7fe3effff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.968+0000 7fe3effff640 1 -- 172.21.15.43:0/927205854 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe408029060 con 0x556d066a7a80 2024-04-21T08:42:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.968+0000 7fe3effff640 1 -- 172.21.15.43:0/927205854 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fe40801b890 con 0x556d066a7a80 2024-04-21T08:42:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.968+0000 7fe419bd6740 1 -- 172.21.15.43:0/927205854 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4000bf1a0 con 0x556d066a7a80 2024-04-21T08:42:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.968+0000 7fe3effff640 1 -- 172.21.15.43:0/927205854 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe408030050 con 0x556d066a7a80 2024-04-21T08:42:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.968+0000 7fe419bd6740 10 client.17085 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.968+0000 7fe419bd6740 1 -- 172.21.15.43:0/927205854 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556d062d50f0 con 0x556d066a7a80 2024-04-21T08:42:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.969+0000 7fe3effff640 1 -- 172.21.15.43:0/927205854 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4000bf1a0 con 0x556d066a7a80 2024-04-21T08:42:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.969+0000 7fe3effff640 1 -- 172.21.15.43:0/927205854 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe408024ad0 con 0x556d066a7a80 2024-04-21T08:42:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.969+0000 7fe419bd6740 10 client.17085 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.969+0000 7fe419bd6740 10 client.17085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.969+0000 7fe419bd6740 20 client.17085 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.969+0000 7fe419bd6740 1 --2- 172.21.15.43:0/927205854 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556d066b77a0 0x556d066b9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.969+0000 7fe419bd6740 4 client.17085 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.969+0000 7fe419bd6740 1 -- 172.21.15.43:0/927205854 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d06610c00 con 0x556d066b77a0 2024-04-21T08:42:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.969+0000 7fe414d8b640 1 --2- 172.21.15.43:0/927205854 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556d066b77a0 0x556d066b9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.969+0000 7fe414d8b640 1 --2- 172.21.15.43:0/927205854 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556d066b77a0 0x556d066b9c30 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3db36600a338d04 server_cookie=127f99fca4b0971 in_seq=0 out_seq=0 2024-04-21T08:42:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.969+0000 7fe3effff640 10 client.17085 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:13.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.970+0000 7fe3effff640 1 -- 172.21.15.43:0/927205854 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556d06610c00 con 0x556d066b77a0 2024-04-21T08:42:13.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.970+0000 7fe3effff640 10 client.17085 handle_command_reply: tid=0 2024-04-21T08:42:13.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.970+0000 7fe419bd6740 1 client.17085 shutdown 2024-04-21T08:42:13.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.970+0000 7fe419bd6740 1 -- 172.21.15.43:0/927205854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d066a7a80 msgr2=0x556d066a5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:13.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.970+0000 7fe419bd6740 1 --2- 172.21.15.43:0/927205854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d066a7a80 0x556d066a5570 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fe408009da0 tx=0x7fe408004ad0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.970+0000 7fe419bd6740 1 -- 172.21.15.43:0/927205854 shutdown_connections 2024-04-21T08:42:13.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.970+0000 7fe419bd6740 1 --2- 172.21.15.43:0/927205854 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556d066b77a0 0x556d066b9c30 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.970+0000 7fe419bd6740 1 --2- 172.21.15.43:0/927205854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d066a7a80 0x556d066a5570 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.970+0000 7fe3effff640 0 client.17085 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:13.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.970+0000 7fe419bd6740 1 --2- 172.21.15.43:0/927205854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d0669eea0 0x556d066a5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.970+0000 7fe419bd6740 1 --2- 172.21.15.43:0/927205854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d0669e580 0x556d066a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.970+0000 7fe419bd6740 1 -- 172.21.15.43:0/927205854 >> 172.21.15.43:0/927205854 conn(0x556d06698cb0 msgr2=0x556d0669aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:13.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.971+0000 7fe419bd6740 1 -- 172.21.15.43:0/927205854 shutdown_connections 2024-04-21T08:42:13.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.971+0000 7fe419bd6740 1 -- 172.21.15.43:0/927205854 wait complete. 2024-04-21T08:42:13.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.971+0000 7fe419bd6740 20 client.17085 trim_cache size 0 max 16384 2024-04-21T08:42:13.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.988+0000 7fe419bd6740 1 Processor -- start 2024-04-21T08:42:13.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.988+0000 7fe419bd6740 1 -- start start 2024-04-21T08:42:13.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.988+0000 7fe419bd6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d0669e580 0x556d066a7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:13.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.989+0000 7fe419bd6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d0669eea0 0x556d066b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:13.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.989+0000 7fe419bd6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d066a7a80 0x556d066a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:13.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.989+0000 7fe419bd6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101b1700 con 0x556d0669eea0 2024-04-21T08:42:13.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.989+0000 7fe419bd6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101b3240 con 0x556d066a7a80 2024-04-21T08:42:13.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.989+0000 7fe419bd6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101b1dd0 con 0x556d0669e580 2024-04-21T08:42:13.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.989+0000 7fe40ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d0669eea0 0x556d066b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:13.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.989+0000 7fe40ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d0669eea0 0x556d066b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36578/0 (socket says 172.21.15.43:36578) 2024-04-21T08:42:13.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.989+0000 7fe40ffff640 1 -- 172.21.15.43:0/3835352047 learned_addr learned my addr 172.21.15.43:0/3835352047 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:13.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.989+0000 7fe41558c640 1 --2- 172.21.15.43:0/3835352047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d066a7a80 0x556d066a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:13.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.989+0000 7fe40ffff640 1 -- 172.21.15.43:0/3835352047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d0669e580 msgr2=0x556d066a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:13.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.989+0000 7fe40ffff640 1 --2- 172.21.15.43:0/3835352047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d0669e580 0x556d066a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.989+0000 7fe40ffff640 1 -- 172.21.15.43:0/3835352047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d066a7a80 msgr2=0x556d066a3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:13.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.989+0000 7fe40ffff640 1 --2- 172.21.15.43:0/3835352047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d066a7a80 0x556d066a3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.989+0000 7fe40ffff640 1 -- 172.21.15.43:0/3835352047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d0669ecd0 con 0x556d0669eea0 2024-04-21T08:42:13.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.989+0000 7fe40ffff640 1 --2- 172.21.15.43:0/3835352047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d0669eea0 0x556d066b3370 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe41006b670 tx=0x7fe3f800ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b8fd54dd1c0f74 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:13.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.989+0000 7fe4067fc640 1 -- 172.21.15.43:0/3835352047 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f800fdd0 con 0x556d0669eea0 2024-04-21T08:42:13.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.989+0000 7fe4067fc640 1 -- 172.21.15.43:0/3835352047 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3f80054d0 con 0x556d0669eea0 2024-04-21T08:42:13.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.989+0000 7fe4067fc640 1 -- 172.21.15.43:0/3835352047 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f80186a0 con 0x556d0669eea0 2024-04-21T08:42:13.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.990+0000 7fe419bd6740 1 -- 172.21.15.43:0/3835352047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d0669eea0 msgr2=0x556d066b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:13.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.990+0000 7fe419bd6740 1 --2- 172.21.15.43:0/3835352047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d0669eea0 0x556d066b3370 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe41006b670 tx=0x7fe3f800ae50 comp rx=0 tx=0).stop 2024-04-21T08:42:13.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.990+0000 7fe419bd6740 1 -- 172.21.15.43:0/3835352047 shutdown_connections 2024-04-21T08:42:13.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.990+0000 7fe419bd6740 1 --2- 172.21.15.43:0/3835352047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d066a7a80 0x556d066a3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.990+0000 7fe419bd6740 1 --2- 172.21.15.43:0/3835352047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d0669eea0 0x556d066b3370 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.990+0000 7fe419bd6740 1 --2- 172.21.15.43:0/3835352047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d0669e580 0x556d066a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.990+0000 7fe419bd6740 1 -- 172.21.15.43:0/3835352047 >> 172.21.15.43:0/3835352047 conn(0x556d06698cb0 msgr2=0x556d066974e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:13.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.990+0000 7fe419bd6740 1 -- 172.21.15.43:0/3835352047 shutdown_connections 2024-04-21T08:42:13.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.990+0000 7fe419bd6740 1 -- 172.21.15.43:0/3835352047 wait complete. 2024-04-21T08:42:13.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.990+0000 7fe419bd6740 1 -- start start 2024-04-21T08:42:13.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.990+0000 7fe419bd6740 1 Processor -- start 2024-04-21T08:42:13.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.991+0000 7fe419bd6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d0669e580 0x556d066ab190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:13.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.991+0000 7fe419bd6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d0669eea0 0x556d066ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:13.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.991+0000 7fe419bd6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d066a7a80 0x556d066abc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:13.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.991+0000 7fe419bd6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101b2b70 con 0x556d0669eea0 2024-04-21T08:42:13.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.991+0000 7fe419bd6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101b24a0 con 0x556d066a7a80 2024-04-21T08:42:13.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.991+0000 7fe419bd6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101af6e0 con 0x556d0669e580 2024-04-21T08:42:13.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.991+0000 7fe419bd6740 10 client.0 _ll_register_callbacks cb 0x7fe40800aca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:13.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.991+0000 7fe40ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d0669eea0 0x556d066ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:13.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.991+0000 7fe40ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d0669eea0 0x556d066ab6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36580/0 (socket says 172.21.15.43:36580) 2024-04-21T08:42:13.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.991+0000 7fe40ffff640 1 -- 172.21.15.43:0/3636988145 learned_addr learned my addr 172.21.15.43:0/3636988145 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:13.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.991+0000 7fe414d8b640 1 --2- 172.21.15.43:0/3636988145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d0669e580 0x556d066ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:13.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.991+0000 7fe41558c640 1 --2- 172.21.15.43:0/3636988145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d066a7a80 0x556d066abc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:13.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.991+0000 7fe40ffff640 1 -- 172.21.15.43:0/3636988145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d0669e580 msgr2=0x556d066ab190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:13.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.991+0000 7fe40ffff640 1 --2- 172.21.15.43:0/3636988145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d0669e580 0x556d066ab190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.991+0000 7fe40ffff640 1 -- 172.21.15.43:0/3636988145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d066a7a80 msgr2=0x556d066abc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:13.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.991+0000 7fe40ffff640 1 --2- 172.21.15.43:0/3636988145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d066a7a80 0x556d066abc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.991+0000 7fe40ffff640 1 -- 172.21.15.43:0/3636988145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d0671cdb0 con 0x556d0669eea0 2024-04-21T08:42:13.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.991+0000 7fe414d8b640 1 --2- 172.21.15.43:0/3636988145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d0669e580 0x556d066ab190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:42:13.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.991+0000 7fe40ffff640 1 --2- 172.21.15.43:0/3636988145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d0669eea0 0x556d066ab6d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8004cf0 tx=0x7fe3f8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4ec9e63419ce78 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:13.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.991+0000 7fe3effff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:13.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.992+0000 7fe3effff640 1 -- 172.21.15.43:0/3636988145 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8027070 con 0x556d0669eea0 2024-04-21T08:42:13.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.992+0000 7fe3effff640 1 -- 172.21.15.43:0/3636988145 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fe3f8004f50 con 0x556d0669eea0 2024-04-21T08:42:13.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.992+0000 7fe419bd6740 1 -- 172.21.15.43:0/3636988145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3f80054d0 con 0x556d0669eea0 2024-04-21T08:42:13.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.992+0000 7fe3effff640 1 -- 172.21.15.43:0/3636988145 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe3f8032050 con 0x556d0669eea0 2024-04-21T08:42:13.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.992+0000 7fe419bd6740 10 client.17091 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:13.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.992+0000 7fe419bd6740 1 -- 172.21.15.43:0/3636988145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556d0671dfa0 con 0x556d0669eea0 2024-04-21T08:42:13.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.992+0000 7fe3effff640 1 -- 172.21.15.43:0/3636988145 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3f80054d0 con 0x556d0669eea0 2024-04-21T08:42:13.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.992+0000 7fe3effff640 1 -- 172.21.15.43:0/3636988145 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe3f8003890 con 0x556d0669eea0 2024-04-21T08:42:13.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.992+0000 7fe419bd6740 10 client.17091 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:13.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.992+0000 7fe419bd6740 10 client.17091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:13.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.992+0000 7fe419bd6740 20 client.17091 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:13.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.992+0000 7fe419bd6740 1 --2- 172.21.15.43:0/3636988145 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556d0671e6f0 0x556d06720b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:13.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.992+0000 7fe419bd6740 4 client.17091 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:42:13.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.992+0000 7fe419bd6740 1 -- 172.21.15.43:0/3636988145 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556d0660aa70 con 0x556d0671e6f0 2024-04-21T08:42:13.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.993+0000 7fe414d8b640 1 --2- 172.21.15.43:0/3636988145 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556d0671e6f0 0x556d06720b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:13.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.993+0000 7fe3effff640 10 client.17091 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:13.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.993+0000 7fe414d8b640 1 --2- 172.21.15.43:0/3636988145 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556d0671e6f0 0x556d06720b10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3719f6bc36c483cf server_cookie=48d9bc8bfa6d75cb in_seq=0 out_seq=0 2024-04-21T08:42:13.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.993+0000 7fe3effff640 1 -- 172.21.15.43:0/3636988145 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556d0660aa70 con 0x556d0671e6f0 2024-04-21T08:42:13.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.993+0000 7fe3effff640 10 client.17091 handle_command_reply: tid=0 2024-04-21T08:42:13.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.993+0000 7fe419bd6740 1 client.17091 shutdown 2024-04-21T08:42:13.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.993+0000 7fe419bd6740 1 -- 172.21.15.43:0/3636988145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d0669eea0 msgr2=0x556d066ab6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:13.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.993+0000 7fe419bd6740 1 --2- 172.21.15.43:0/3636988145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d0669eea0 0x556d066ab6d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8004cf0 tx=0x7fe3f8004af0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.993+0000 7fe419bd6740 1 -- 172.21.15.43:0/3636988145 shutdown_connections 2024-04-21T08:42:13.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.993+0000 7fe419bd6740 1 --2- 172.21.15.43:0/3636988145 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556d0671e6f0 0x556d06720b10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.993+0000 7fe419bd6740 1 --2- 172.21.15.43:0/3636988145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d066a7a80 0x556d066abc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.993+0000 7fe3effff640 0 client.17091 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:13.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.993+0000 7fe419bd6740 1 --2- 172.21.15.43:0/3636988145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d0669eea0 0x556d066ab6d0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.993+0000 7fe419bd6740 1 --2- 172.21.15.43:0/3636988145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556d0669e580 0x556d066ab190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:13.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.993+0000 7fe419bd6740 1 -- 172.21.15.43:0/3636988145 >> 172.21.15.43:0/3636988145 conn(0x556d06698cb0 msgr2=0x556d066a4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:13.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.993+0000 7fe419bd6740 1 -- 172.21.15.43:0/3636988145 shutdown_connections 2024-04-21T08:42:13.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.993+0000 7fe419bd6740 1 -- 172.21.15.43:0/3636988145 wait complete. 2024-04-21T08:42:13.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.994+0000 7fe419bd6740 20 client.17091 trim_cache size 0 max 16384 2024-04-21T08:42:14.000 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:42:14.000 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:42:14.000 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:42:14.000 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:42:14.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.994+0000 7fe416fef640 1 -- 172.21.15.43:0/3455118788 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe3fc0a3dd0 msgr2=0x7fe3fc0a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.994+0000 7fe416fef640 1 --2- 172.21.15.43:0/3455118788 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe3fc0a3dd0 0x7fe3fc0a6290 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe408009fd0 tx=0x7fe4080093d0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.994+0000 7fe416fef640 1 -- 172.21.15.43:0/3455118788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4101021a0 msgr2=0x7fe410074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.994+0000 7fe416fef640 1 --2- 172.21.15.43:0/3455118788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4101021a0 0x7fe410074250 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe400005ec0 tx=0x7fe40002c880 comp rx=0 tx=0).stop 2024-04-21T08:42:14.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.994+0000 7fe416fef640 1 -- 172.21.15.43:0/3455118788 shutdown_connections 2024-04-21T08:42:14.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.994+0000 7fe416fef640 1 --2- 172.21.15.43:0/3455118788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe410074cd0 0x7fe4101a6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.994+0000 7fe416fef640 1 --2- 172.21.15.43:0/3455118788 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe3fc0a3dd0 0x7fe3fc0a6290 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.994+0000 7fe416fef640 1 --2- 172.21.15.43:0/3455118788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe410107270 0x7fe410074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.994+0000 7fe416fef640 1 --2- 172.21.15.43:0/3455118788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4101021a0 0x7fe410074250 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.994+0000 7fe416fef640 1 -- 172.21.15.43:0/3455118788 >> 172.21.15.43:0/3455118788 conn(0x7fe4100793e0 msgr2=0x7fe4100fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.994+0000 7fe416fef640 1 -- 172.21.15.43:0/3455118788 shutdown_connections 2024-04-21T08:42:14.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:13.994+0000 7fe416fef640 1 -- 172.21.15.43:0/3455118788 wait complete. 2024-04-21T08:42:14.005 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:42:14.006 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:42:14.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.108+0000 7fe1a6fce640 1 -- 172.21.15.43:0/3265209826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1a0101a60 msgr2=0x7fe1a0105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.108+0000 7fe1a6fce640 1 --2- 172.21.15.43:0/3265209826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1a0101a60 0x7fe1a0105aa0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fe18c009900 tx=0x7fe18c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:42:14.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.108+0000 7fe1a6fce640 1 -- 172.21.15.43:0/3265209826 shutdown_connections 2024-04-21T08:42:14.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.108+0000 7fe1a6fce640 1 --2- 172.21.15.43:0/3265209826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1a0106180 0x7fe1a010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.108+0000 7fe1a6fce640 1 --2- 172.21.15.43:0/3265209826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1a0101a60 0x7fe1a0105aa0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.108+0000 7fe1a6fce640 1 --2- 172.21.15.43:0/3265209826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1a01010b0 0x7fe1a0101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.108+0000 7fe1a6fce640 1 -- 172.21.15.43:0/3265209826 >> 172.21.15.43:0/3265209826 conn(0x7fe1a00771d0 msgr2=0x7fe1a00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.108+0000 7fe1a6fce640 1 -- 172.21.15.43:0/3265209826 shutdown_connections 2024-04-21T08:42:14.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.108+0000 7fe1a6fce640 1 -- 172.21.15.43:0/3265209826 wait complete. 2024-04-21T08:42:14.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.109+0000 7fe1a6fce640 1 Processor -- start 2024-04-21T08:42:14.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.109+0000 7fe1a6fce640 1 -- start start 2024-04-21T08:42:14.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.109+0000 7fe1a6fce640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1a01010b0 0x7fe1a0199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.109+0000 7fe1a6fce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1a0106180 0x7fe1a0199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.109+0000 7fe1a6fce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1a019a050 0x7fe1a01a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.109+0000 7fe1a6fce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a01110c0 con 0x7fe1a0106180 2024-04-21T08:42:14.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.109+0000 7fe1a6fce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a0110f40 con 0x7fe1a019a050 2024-04-21T08:42:14.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.109+0000 7fe1a6fce640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a0111240 con 0x7fe1a01010b0 2024-04-21T08:42:14.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.109+0000 7fe19ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1a0106180 0x7fe1a0199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.109+0000 7fe19ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1a0106180 0x7fe1a0199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36598/0 (socket says 172.21.15.43:36598) 2024-04-21T08:42:14.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.109+0000 7fe19ffff640 1 -- 172.21.15.43:0/4232235096 learned_addr learned my addr 172.21.15.43:0/4232235096 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:14.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.109+0000 7fe1a4d6a640 1 --2- 172.21.15.43:0/4232235096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1a01010b0 0x7fe1a0199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.109+0000 7fe1a556b640 1 --2- 172.21.15.43:0/4232235096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1a019a050 0x7fe1a01a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.109+0000 7fe19ffff640 1 -- 172.21.15.43:0/4232235096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1a01010b0 msgr2=0x7fe1a0199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.109+0000 7fe19ffff640 1 --2- 172.21.15.43:0/4232235096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1a01010b0 0x7fe1a0199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.109+0000 7fe19ffff640 1 -- 172.21.15.43:0/4232235096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1a019a050 msgr2=0x7fe1a01a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.109+0000 7fe19ffff640 1 --2- 172.21.15.43:0/4232235096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1a019a050 0x7fe1a01a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.110+0000 7fe19ffff640 1 -- 172.21.15.43:0/4232235096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a01a4dd0 con 0x7fe1a0106180 2024-04-21T08:42:14.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.110+0000 7fe1a556b640 1 --2- 172.21.15.43:0/4232235096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1a019a050 0x7fe1a01a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:14.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.110+0000 7fe19ffff640 1 --2- 172.21.15.43:0/4232235096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1a0106180 0x7fe1a0199ac0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fe18c036040 tx=0x7fe18c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.110+0000 7fe19dffb640 1 -- 172.21.15.43:0/4232235096 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c043020 con 0x7fe1a0106180 2024-04-21T08:42:14.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.110+0000 7fe19dffb640 1 -- 172.21.15.43:0/4232235096 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe18c02ca70 con 0x7fe1a0106180 2024-04-21T08:42:14.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.110+0000 7fe1a6fce640 1 -- 172.21.15.43:0/4232235096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a01a5060 con 0x7fe1a0106180 2024-04-21T08:42:14.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.110+0000 7fe19dffb640 1 -- 172.21.15.43:0/4232235096 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c003ac0 con 0x7fe1a0106180 2024-04-21T08:42:14.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.110+0000 7fe1a6fce640 1 -- 172.21.15.43:0/4232235096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a01a5600 con 0x7fe1a0106180 2024-04-21T08:42:14.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.111+0000 7fe1a6fce640 1 -- 172.21.15.43:0/4232235096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1a0069230 con 0x7fe1a0106180 2024-04-21T08:42:14.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.112+0000 7fe19dffb640 1 -- 172.21.15.43:0/4232235096 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe18c046020 con 0x7fe1a0106180 2024-04-21T08:42:14.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.113+0000 7fe19dffb640 1 --2- 172.21.15.43:0/4232235096 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe1900a3da0 0x7fe1900a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.113+0000 7fe19dffb640 1 -- 172.21.15.43:0/4232235096 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fe18c0f3090 con 0x7fe1a0106180 2024-04-21T08:42:14.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.113+0000 7fe1a4d6a640 1 --2- 172.21.15.43:0/4232235096 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe1900a3da0 0x7fe1900a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.114+0000 7fe1a4d6a640 1 --2- 172.21.15.43:0/4232235096 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe1900a3da0 0x7fe1900a6260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe194009b60 tx=0x7fe194005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.117+0000 7fe19dffb640 1 -- 172.21.15.43:0/4232235096 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe18c04d020 con 0x7fe1a0106180 2024-04-21T08:42:14.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.343+0000 7fe1a6fce640 1 -- 172.21.15.43:0/4232235096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1a01093b0 con 0x7fe1a0106180 2024-04-21T08:42:14.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.344+0000 7fe19dffb640 1 -- 172.21.15.43:0/4232235096 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7fe18c02cc20 con 0x7fe1a0106180 2024-04-21T08:42:14.344 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:42:14.344 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:42:14.345 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:42:14.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.346+0000 7fe1a6fce640 1 -- 172.21.15.43:0/4232235096 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe1900a3da0 msgr2=0x7fe1900a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.346+0000 7fe1a6fce640 1 --2- 172.21.15.43:0/4232235096 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe1900a3da0 0x7fe1900a6260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe194009b60 tx=0x7fe194005f90 comp rx=0 tx=0).stop 2024-04-21T08:42:14.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.346+0000 7fe1a6fce640 1 -- 172.21.15.43:0/4232235096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1a0106180 msgr2=0x7fe1a0199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.346+0000 7fe1a6fce640 1 --2- 172.21.15.43:0/4232235096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1a0106180 0x7fe1a0199ac0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fe18c036040 tx=0x7fe18c0045e0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.346+0000 7fe1a6fce640 1 -- 172.21.15.43:0/4232235096 shutdown_connections 2024-04-21T08:42:14.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.346+0000 7fe1a6fce640 1 --2- 172.21.15.43:0/4232235096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1a019a050 0x7fe1a01a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.346+0000 7fe1a6fce640 1 --2- 172.21.15.43:0/4232235096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1a0106180 0x7fe1a0199ac0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.346+0000 7fe1a6fce640 1 --2- 172.21.15.43:0/4232235096 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe1900a3da0 0x7fe1900a6260 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.346+0000 7fe1a6fce640 1 --2- 172.21.15.43:0/4232235096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1a01010b0 0x7fe1a0199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.346+0000 7fe1a6fce640 1 -- 172.21.15.43:0/4232235096 >> 172.21.15.43:0/4232235096 conn(0x7fe1a00771d0 msgr2=0x7fe1a00fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.346+0000 7fe1a6fce640 1 -- 172.21.15.43:0/4232235096 shutdown_connections 2024-04-21T08:42:14.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.346+0000 7fe1a6fce640 1 -- 172.21.15.43:0/4232235096 wait complete. 2024-04-21T08:42:14.357 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:42:14.378 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:14 smithi043 ceph-mon[27978]: pgmap v102: 129 pgs: 129 active+clean; 3.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-04-21T08:42:14.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.462+0000 7fc035831640 1 -- 172.21.15.43:0/1963953032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc030107260 msgr2=0x7fc03010ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.462+0000 7fc035831640 1 --2- 172.21.15.43:0/1963953032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc030107260 0x7fc03010ee30 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc028009ca0 tx=0x7fc02802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.462+0000 7fc035831640 1 -- 172.21.15.43:0/1963953032 shutdown_connections 2024-04-21T08:42:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.462+0000 7fc035831640 1 --2- 172.21.15.43:0/1963953032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc030107260 0x7fc03010ee30 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.462+0000 7fc035831640 1 --2- 172.21.15.43:0/1963953032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc030102b40 0x7fc030106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.462+0000 7fc035831640 1 --2- 172.21.15.43:0/1963953032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc030102190 0x7fc030102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.462+0000 7fc035831640 1 -- 172.21.15.43:0/1963953032 >> 172.21.15.43:0/1963953032 conn(0x7fc0300fd990 msgr2=0x7fc0300ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.462+0000 7fc035831640 1 -- 172.21.15.43:0/1963953032 shutdown_connections 2024-04-21T08:42:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.462+0000 7fc035831640 1 -- 172.21.15.43:0/1963953032 wait complete. 2024-04-21T08:42:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.463+0000 7fc035831640 1 Processor -- start 2024-04-21T08:42:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.463+0000 7fc035831640 1 -- start start 2024-04-21T08:42:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.463+0000 7fc035831640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc030102190 0x7fc030198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.463+0000 7fc035831640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc030107260 0x7fc030198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.463+0000 7fc035831640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03019cf30 0x7fc03019d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.463+0000 7fc035831640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc0301121d0 con 0x7fc03019cf30 2024-04-21T08:42:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.463+0000 7fc035831640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc030112050 con 0x7fc030107260 2024-04-21T08:42:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.463+0000 7fc035831640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc030112350 con 0x7fc030102190 2024-04-21T08:42:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.463+0000 7fc02f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03019cf30 0x7fc03019d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.463+0000 7fc02f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03019cf30 0x7fc03019d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36606/0 (socket says 172.21.15.43:36606) 2024-04-21T08:42:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.463+0000 7fc02f7fe640 1 -- 172.21.15.43:0/3645145653 learned_addr learned my addr 172.21.15.43:0/3645145653 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.463+0000 7fc027fff640 1 --2- 172.21.15.43:0/3645145653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc030107260 0x7fc030198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.463+0000 7fc02effd640 1 --2- 172.21.15.43:0/3645145653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc030102190 0x7fc030198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.463+0000 7fc02f7fe640 1 -- 172.21.15.43:0/3645145653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc030102190 msgr2=0x7fc030198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.463+0000 7fc02f7fe640 1 --2- 172.21.15.43:0/3645145653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc030102190 0x7fc030198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.463+0000 7fc02f7fe640 1 -- 172.21.15.43:0/3645145653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc030107260 msgr2=0x7fc030198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.463+0000 7fc02f7fe640 1 --2- 172.21.15.43:0/3645145653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc030107260 0x7fc030198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.463+0000 7fc02f7fe640 1 -- 172.21.15.43:0/3645145653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03019da20 con 0x7fc03019cf30 2024-04-21T08:42:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.464+0000 7fc02f7fe640 1 --2- 172.21.15.43:0/3645145653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03019cf30 0x7fc03019d3e0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc028005e30 tx=0x7fc028034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.464+0000 7fc03482f640 1 -- 172.21.15.43:0/3645145653 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028041020 con 0x7fc03019cf30 2024-04-21T08:42:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.464+0000 7fc03482f640 1 -- 172.21.15.43:0/3645145653 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc028032970 con 0x7fc03019cf30 2024-04-21T08:42:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.464+0000 7fc035831640 1 -- 172.21.15.43:0/3645145653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc03019dcb0 con 0x7fc03019cf30 2024-04-21T08:42:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.464+0000 7fc03482f640 1 -- 172.21.15.43:0/3645145653 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028039040 con 0x7fc03019cf30 2024-04-21T08:42:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.464+0000 7fc035831640 1 -- 172.21.15.43:0/3645145653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0301a54d0 con 0x7fc03019cf30 2024-04-21T08:42:14.468 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:14 smithi175 ceph-mon[31920]: pgmap v102: 129 pgs: 129 active+clean; 3.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-04-21T08:42:14.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.466+0000 7fc03482f640 1 -- 172.21.15.43:0/3645145653 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc028044020 con 0x7fc03019cf30 2024-04-21T08:42:14.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.467+0000 7fc038424740 1 Processor -- start 2024-04-21T08:42:14.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.467+0000 7fc038424740 1 -- start start 2024-04-21T08:42:14.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.467+0000 7fc038424740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604869424d0 0x5604869428b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.467+0000 7fc038424740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560486942e80 0x56048694b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.467+0000 7fc038424740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56048694ba00 0x56048694de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.467+0000 7fc038424740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5604868a71f0 con 0x56048694ba00 2024-04-21T08:42:14.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.467+0000 7fc038424740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5604868444b0 con 0x5604869424d0 2024-04-21T08:42:14.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.467+0000 7fc038424740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5604868a31b0 con 0x560486942e80 2024-04-21T08:42:14.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.467+0000 7fc03482f640 1 --2- 172.21.15.43:0/3645145653 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc01c0a3da0 0x7fc01c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.467+0000 7fc027fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56048694ba00 0x56048694de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.467+0000 7fc02f7fe640 1 --2- 172.21.15.43:0/3645145653 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc01c0a3da0 0x7fc01c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.467+0000 7fc02effd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560486942e80 0x56048694b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.467+0000 7fc027fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56048694ba00 0x56048694de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36618/0 (socket says 172.21.15.43:36618) 2024-04-21T08:42:14.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.467+0000 7fc027fff640 1 -- 172.21.15.43:0/1693380695 learned_addr learned my addr 172.21.15.43:0/1693380695 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:14.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.467+0000 7fc027fff640 1 --2- 172.21.15.43:0/1693380695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604869424d0 0x5604869428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.467+0000 7fc03482f640 1 -- 172.21.15.43:0/3645145653 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc0280f27a0 con 0x7fc03019cf30 2024-04-21T08:42:14.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.467+0000 7fc02effd640 1 -- 172.21.15.43:0/1693380695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604869424d0 msgr2=0x5604869428b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.467+0000 7fc02effd640 1 --2- 172.21.15.43:0/1693380695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604869424d0 0x5604869428b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.468+0000 7fc02effd640 1 -- 172.21.15.43:0/1693380695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56048694ba00 msgr2=0x56048694de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.468+0000 7fc02effd640 1 --2- 172.21.15.43:0/1693380695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56048694ba00 0x56048694de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.468+0000 7fc02effd640 1 -- 172.21.15.43:0/1693380695 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604868361b0 con 0x560486942e80 2024-04-21T08:42:14.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.468+0000 7fc027fff640 1 --2- 172.21.15.43:0/1693380695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56048694ba00 0x56048694de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:42:14.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.468+0000 7fc02f7fe640 1 --2- 172.21.15.43:0/3645145653 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc01c0a3da0 0x7fc01c0a6260 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc0280be3b0 tx=0x7fc0280b2f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.468+0000 7fc02effd640 1 --2- 172.21.15.43:0/1693380695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560486942e80 0x56048694b430 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc02000abf0 tx=0x7fc02000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da90639210b3eee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.468+0000 7fc025ffb640 1 -- 172.21.15.43:0/1693380695 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc020005220 con 0x560486942e80 2024-04-21T08:42:14.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.468+0000 7fc025ffb640 1 -- 172.21.15.43:0/1693380695 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0200047a0 con 0x560486942e80 2024-04-21T08:42:14.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.468+0000 7fc025ffb640 1 -- 172.21.15.43:0/1693380695 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0200186a0 con 0x560486942e80 2024-04-21T08:42:14.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.469+0000 7fc038424740 1 -- 172.21.15.43:0/1693380695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560486942e80 msgr2=0x56048694b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.469+0000 7fc038424740 1 --2- 172.21.15.43:0/1693380695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560486942e80 0x56048694b430 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc02000abf0 tx=0x7fc02000f790 comp rx=0 tx=0).stop 2024-04-21T08:42:14.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.469+0000 7fc038424740 1 -- 172.21.15.43:0/1693380695 shutdown_connections 2024-04-21T08:42:14.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.469+0000 7fc038424740 1 --2- 172.21.15.43:0/1693380695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56048694ba00 0x56048694de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.469+0000 7fc038424740 1 --2- 172.21.15.43:0/1693380695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560486942e80 0x56048694b430 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.469+0000 7fc038424740 1 --2- 172.21.15.43:0/1693380695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604869424d0 0x5604869428b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.469+0000 7fc038424740 1 -- 172.21.15.43:0/1693380695 >> 172.21.15.43:0/1693380695 conn(0x56048693cba0 msgr2=0x56048693cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.469+0000 7fc038424740 1 -- 172.21.15.43:0/1693380695 shutdown_connections 2024-04-21T08:42:14.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.469+0000 7fc038424740 1 -- 172.21.15.43:0/1693380695 wait complete. 2024-04-21T08:42:14.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.469+0000 7fc038424740 1 -- start start 2024-04-21T08:42:14.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.469+0000 7fc038424740 1 Processor -- start 2024-04-21T08:42:14.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.469+0000 7fc038424740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604869424d0 0x56048694b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.469+0000 7fc038424740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560486942e80 0x560486948690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.470+0000 7fc038424740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56048694ba00 0x560486948c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.470+0000 7fc038424740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56048694e970 con 0x560486942e80 2024-04-21T08:42:14.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.470+0000 7fc038424740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560486860500 con 0x56048694ba00 2024-04-21T08:42:14.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.470+0000 7fc038424740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56048694eaf0 con 0x5604869424d0 2024-04-21T08:42:14.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.470+0000 7fc038424740 10 client.0 _ll_register_callbacks cb 0x56048682eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:14.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.470+0000 7fc02effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560486942e80 0x560486948690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.470+0000 7fc02effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560486942e80 0x560486948690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36622/0 (socket says 172.21.15.43:36622) 2024-04-21T08:42:14.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.470+0000 7fc02effd640 1 -- 172.21.15.43:0/2799261672 learned_addr learned my addr 172.21.15.43:0/2799261672 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:14.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.470+0000 7fc027fff640 1 --2- 172.21.15.43:0/2799261672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604869424d0 0x56048694b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.470+0000 7fc027fff640 1 --2- 172.21.15.43:0/2799261672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56048694ba00 0x560486948c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.470+0000 7fc02effd640 1 -- 172.21.15.43:0/2799261672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604869424d0 msgr2=0x56048694b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.470+0000 7fc02effd640 1 --2- 172.21.15.43:0/2799261672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604869424d0 0x56048694b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.470+0000 7fc02effd640 1 -- 172.21.15.43:0/2799261672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56048694ba00 msgr2=0x560486948c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.470+0000 7fc02effd640 1 --2- 172.21.15.43:0/2799261672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56048694ba00 0x560486948c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.470+0000 7fc02effd640 1 -- 172.21.15.43:0/2799261672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604865b93f0 con 0x560486942e80 2024-04-21T08:42:14.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.470+0000 7fc027fff640 1 --2- 172.21.15.43:0/2799261672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604869424d0 0x56048694b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:14.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.470+0000 7fc02effd640 1 --2- 172.21.15.43:0/2799261672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560486942e80 0x560486948690 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc020019000 tx=0x7fc0200051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f01ae1592935ee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.470+0000 7fc00effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:14.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.470+0000 7fc00effd640 1 -- 172.21.15.43:0/2799261672 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc020027070 con 0x560486942e80 2024-04-21T08:42:14.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.470+0000 7fc00effd640 1 -- 172.21.15.43:0/2799261672 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc020005600 con 0x560486942e80 2024-04-21T08:42:14.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.470+0000 7fc038424740 1 -- 172.21.15.43:0/2799261672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0200047a0 con 0x560486942e80 2024-04-21T08:42:14.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.470+0000 7fc00effd640 1 -- 172.21.15.43:0/2799261672 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc020033050 con 0x560486942e80 2024-04-21T08:42:14.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.470+0000 7fc038424740 10 client.17109 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:14.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.470+0000 7fc038424740 1 -- 172.21.15.43:0/2799261672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5604866837d0 con 0x560486942e80 2024-04-21T08:42:14.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.471+0000 7fc00effd640 1 -- 172.21.15.43:0/2799261672 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0200047a0 con 0x560486942e80 2024-04-21T08:42:14.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.471+0000 7fc00effd640 1 -- 172.21.15.43:0/2799261672 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc020023430 con 0x560486942e80 2024-04-21T08:42:14.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.471+0000 7fc038424740 10 client.17109 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:14.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.471+0000 7fc038424740 10 client.17109 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:42:14.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.471+0000 7fc038424740 20 client.17109 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:14.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.471+0000 7fc038424740 1 --2- 172.21.15.43:0/2799261672 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x56048695bb10 0x56048695df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.471+0000 7fc038424740 4 client.17109 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:14.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.471+0000 7fc038424740 1 -- 172.21.15.43:0/2799261672 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560486846930 con 0x56048695bb10 2024-04-21T08:42:14.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.471+0000 7fc027fff640 1 --2- 172.21.15.43:0/2799261672 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x56048695bb10 0x56048695df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.471+0000 7fc027fff640 1 --2- 172.21.15.43:0/2799261672 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x56048695bb10 0x56048695df00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fea5f101064f4647 server_cookie=8e7f9bdefda8892c in_seq=0 out_seq=0 2024-04-21T08:42:14.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.471+0000 7fc00effd640 10 client.17109 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:42:14.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.472+0000 7fc00effd640 1 -- 172.21.15.43:0/2799261672 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560486846930 con 0x56048695bb10 2024-04-21T08:42:14.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.472+0000 7fc00effd640 10 client.17109 handle_command_reply: tid=0 2024-04-21T08:42:14.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.472+0000 7fc038424740 1 client.17109 shutdown 2024-04-21T08:42:14.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.472+0000 7fc038424740 1 -- 172.21.15.43:0/2799261672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560486942e80 msgr2=0x560486948690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.472+0000 7fc038424740 1 --2- 172.21.15.43:0/2799261672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560486942e80 0x560486948690 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc020019000 tx=0x7fc0200051d0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.472+0000 7fc038424740 1 -- 172.21.15.43:0/2799261672 shutdown_connections 2024-04-21T08:42:14.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.472+0000 7fc038424740 1 --2- 172.21.15.43:0/2799261672 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x56048695bb10 0x56048695df00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.472+0000 7fc038424740 1 --2- 172.21.15.43:0/2799261672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56048694ba00 0x560486948c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.472+0000 7fc038424740 1 --2- 172.21.15.43:0/2799261672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560486942e80 0x560486948690 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.472+0000 7fc00effd640 0 client.17109 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:42:14.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.472+0000 7fc038424740 1 --2- 172.21.15.43:0/2799261672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604869424d0 0x56048694b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.472+0000 7fc038424740 1 -- 172.21.15.43:0/2799261672 >> 172.21.15.43:0/2799261672 conn(0x56048693cba0 msgr2=0x56048693e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.472+0000 7fc038424740 1 -- 172.21.15.43:0/2799261672 shutdown_connections 2024-04-21T08:42:14.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.472+0000 7fc038424740 1 -- 172.21.15.43:0/2799261672 wait complete. 2024-04-21T08:42:14.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.473+0000 7fc038424740 20 client.17109 trim_cache size 0 max 16384 2024-04-21T08:42:14.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.486+0000 7fc038424740 1 Processor -- start 2024-04-21T08:42:14.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.486+0000 7fc038424740 1 -- start start 2024-04-21T08:42:14.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.486+0000 7fc038424740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604869424d0 0x560486957860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.486+0000 7fc038424740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560486942e80 0x560486943ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.486+0000 7fc038424740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56048694ba00 0x56048694adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.486+0000 7fc038424740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc0301af310 con 0x5604869424d0 2024-04-21T08:42:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.486+0000 7fc038424740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0301b0e50 con 0x56048694ba00 2024-04-21T08:42:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.486+0000 7fc038424740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc0301af9e0 con 0x560486942e80 2024-04-21T08:42:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.486+0000 7fc027fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604869424d0 0x560486957860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.486+0000 7fc027fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604869424d0 0x560486957860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36632/0 (socket says 172.21.15.43:36632) 2024-04-21T08:42:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.486+0000 7fc027fff640 1 -- 172.21.15.43:0/3344375368 learned_addr learned my addr 172.21.15.43:0/3344375368 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.486+0000 7fc02effd640 1 --2- 172.21.15.43:0/3344375368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560486942e80 0x560486943ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.486+0000 7fc027fff640 1 -- 172.21.15.43:0/3344375368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560486942e80 msgr2=0x560486943ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.486+0000 7fc027fff640 1 --2- 172.21.15.43:0/3344375368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560486942e80 0x560486943ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.486+0000 7fc027fff640 1 -- 172.21.15.43:0/3344375368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56048694ba00 msgr2=0x56048694adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.486+0000 7fc027fff640 1 --2- 172.21.15.43:0/3344375368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56048694ba00 0x56048694adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.486+0000 7fc027fff640 1 -- 172.21.15.43:0/3344375368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560486942be0 con 0x5604869424d0 2024-04-21T08:42:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.486+0000 7fc02effd640 1 --2- 172.21.15.43:0/3344375368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560486942e80 0x560486943ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.487+0000 7fc027fff640 1 --2- 172.21.15.43:0/3344375368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604869424d0 0x560486957860 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fc018005850 tx=0x7fc01800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60408aa3599b2717 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.487+0000 7fc0257fa640 1 -- 172.21.15.43:0/3344375368 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0180044d0 con 0x5604869424d0 2024-04-21T08:42:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.487+0000 7fc0257fa640 1 -- 172.21.15.43:0/3344375368 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc018013e90 con 0x5604869424d0 2024-04-21T08:42:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.487+0000 7fc0257fa640 1 -- 172.21.15.43:0/3344375368 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018022800 con 0x5604869424d0 2024-04-21T08:42:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.487+0000 7fc038424740 1 -- 172.21.15.43:0/3344375368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604869424d0 msgr2=0x560486957860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.487+0000 7fc038424740 1 --2- 172.21.15.43:0/3344375368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604869424d0 0x560486957860 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fc018005850 tx=0x7fc01800b390 comp rx=0 tx=0).stop 2024-04-21T08:42:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.487+0000 7fc038424740 1 -- 172.21.15.43:0/3344375368 shutdown_connections 2024-04-21T08:42:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.487+0000 7fc038424740 1 --2- 172.21.15.43:0/3344375368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56048694ba00 0x56048694adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.487+0000 7fc038424740 1 --2- 172.21.15.43:0/3344375368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560486942e80 0x560486943ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.487+0000 7fc038424740 1 --2- 172.21.15.43:0/3344375368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604869424d0 0x560486957860 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.487+0000 7fc038424740 1 -- 172.21.15.43:0/3344375368 >> 172.21.15.43:0/3344375368 conn(0x56048693cba0 msgr2=0x5604869433d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.487+0000 7fc038424740 1 -- 172.21.15.43:0/3344375368 shutdown_connections 2024-04-21T08:42:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.487+0000 7fc038424740 1 -- 172.21.15.43:0/3344375368 wait complete. 2024-04-21T08:42:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.488+0000 7fc038424740 1 -- start start 2024-04-21T08:42:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.488+0000 7fc038424740 1 Processor -- start 2024-04-21T08:42:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.488+0000 7fc038424740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604869424d0 0x560486954070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.488+0000 7fc038424740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560486942e80 0x56048694f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.488+0000 7fc038424740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56048694ba00 0x56048694f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.488+0000 7fc038424740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc0301b0780 con 0x560486942e80 2024-04-21T08:42:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.488+0000 7fc038424740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0301b00b0 con 0x5604869424d0 2024-04-21T08:42:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.488+0000 7fc038424740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc0301ad2f0 con 0x56048694ba00 2024-04-21T08:42:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.488+0000 7fc038424740 10 client.0 _ll_register_callbacks cb 0x7fc02000aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.488+0000 7fc02effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560486942e80 0x56048694f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.488+0000 7fc02effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560486942e80 0x56048694f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36646/0 (socket says 172.21.15.43:36646) 2024-04-21T08:42:14.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.488+0000 7fc02effd640 1 -- 172.21.15.43:0/310725827 learned_addr learned my addr 172.21.15.43:0/310725827 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:14.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.488+0000 7fc027fff640 1 --2- 172.21.15.43:0/310725827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56048694ba00 0x56048694f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.488+0000 7fc027fff640 1 --2- 172.21.15.43:0/310725827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604869424d0 0x560486954070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.489+0000 7fc02effd640 1 -- 172.21.15.43:0/310725827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56048694ba00 msgr2=0x56048694f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.489+0000 7fc02effd640 1 --2- 172.21.15.43:0/310725827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56048694ba00 0x56048694f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.489+0000 7fc02effd640 1 -- 172.21.15.43:0/310725827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604869424d0 msgr2=0x560486954070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.489+0000 7fc02effd640 1 --2- 172.21.15.43:0/310725827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604869424d0 0x560486954070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.489+0000 7fc02effd640 1 -- 172.21.15.43:0/310725827 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560486950170 con 0x560486942e80 2024-04-21T08:42:14.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.489+0000 7fc027fff640 1 --2- 172.21.15.43:0/310725827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604869424d0 0x560486954070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:14.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.489+0000 7fc02effd640 1 --2- 172.21.15.43:0/310725827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560486942e80 0x56048694f430 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fc0200051a0 tx=0x7fc020004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f30b629714f9ba server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.489+0000 7fc00effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.489+0000 7fc00effd640 1 -- 172.21.15.43:0/310725827 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02003b070 con 0x560486942e80 2024-04-21T08:42:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.489+0000 7fc00effd640 1 -- 172.21.15.43:0/310725827 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc02000a630 con 0x560486942e80 2024-04-21T08:42:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.489+0000 7fc038424740 1 -- 172.21.15.43:0/310725827 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc018013e90 con 0x560486942e80 2024-04-21T08:42:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.489+0000 7fc00effd640 1 -- 172.21.15.43:0/310725827 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc020047050 con 0x560486942e80 2024-04-21T08:42:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.489+0000 7fc038424740 10 client.17115 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.489+0000 7fc038424740 1 -- 172.21.15.43:0/310725827 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5604869c1d40 con 0x560486942e80 2024-04-21T08:42:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.489+0000 7fc00effd640 1 -- 172.21.15.43:0/310725827 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc018013e90 con 0x560486942e80 2024-04-21T08:42:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.489+0000 7fc00effd640 1 -- 172.21.15.43:0/310725827 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc020023b90 con 0x560486942e80 2024-04-21T08:42:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.489+0000 7fc038424740 10 client.17115 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.489+0000 7fc038424740 10 client.17115 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:42:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.489+0000 7fc038424740 20 client.17115 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.489+0000 7fc038424740 1 --2- 172.21.15.43:0/310725827 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5604869c23a0 0x5604869c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.489+0000 7fc038424740 4 client.17115 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:42:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.489+0000 7fc038424740 1 -- 172.21.15.43:0/310725827 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5604868b4c10 con 0x5604869c23a0 2024-04-21T08:42:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.490+0000 7fc027fff640 1 --2- 172.21.15.43:0/310725827 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5604869c23a0 0x5604869c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.490+0000 7fc027fff640 1 --2- 172.21.15.43:0/310725827 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5604869c23a0 0x5604869c47c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a54c79d36a74e6bc server_cookie=517d00c48588207c in_seq=0 out_seq=0 2024-04-21T08:42:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.490+0000 7fc00effd640 10 client.17115 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:42:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.490+0000 7fc00effd640 1 -- 172.21.15.43:0/310725827 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5604868b4c10 con 0x5604869c23a0 2024-04-21T08:42:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.490+0000 7fc00effd640 10 client.17115 handle_command_reply: tid=0 2024-04-21T08:42:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.490+0000 7fc038424740 1 client.17115 shutdown 2024-04-21T08:42:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.490+0000 7fc038424740 1 -- 172.21.15.43:0/310725827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560486942e80 msgr2=0x56048694f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.490+0000 7fc038424740 1 --2- 172.21.15.43:0/310725827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560486942e80 0x56048694f430 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fc0200051a0 tx=0x7fc020004e90 comp rx=0 tx=0).stop 2024-04-21T08:42:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.490+0000 7fc038424740 1 -- 172.21.15.43:0/310725827 shutdown_connections 2024-04-21T08:42:14.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.490+0000 7fc038424740 1 --2- 172.21.15.43:0/310725827 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5604869c23a0 0x5604869c47c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.491+0000 7fc038424740 1 --2- 172.21.15.43:0/310725827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56048694ba00 0x56048694f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.491+0000 7fc00effd640 0 client.17115 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:42:14.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.491+0000 7fc038424740 1 --2- 172.21.15.43:0/310725827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560486942e80 0x56048694f430 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.491+0000 7fc038424740 1 --2- 172.21.15.43:0/310725827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604869424d0 0x560486954070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.491+0000 7fc038424740 1 -- 172.21.15.43:0/310725827 >> 172.21.15.43:0/310725827 conn(0x56048693cba0 msgr2=0x56048693e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.491+0000 7fc038424740 1 -- 172.21.15.43:0/310725827 shutdown_connections 2024-04-21T08:42:14.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.491+0000 7fc038424740 1 -- 172.21.15.43:0/310725827 wait complete. 2024-04-21T08:42:14.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.491+0000 7fc038424740 20 client.17115 trim_cache size 0 max 16384 2024-04-21T08:42:14.500 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:42:14.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.491+0000 7fc035831640 1 -- 172.21.15.43:0/3645145653 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc01c0a3da0 msgr2=0x7fc01c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.491+0000 7fc035831640 1 --2- 172.21.15.43:0/3645145653 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc01c0a3da0 0x7fc01c0a6260 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc0280be3b0 tx=0x7fc0280b2f90 comp rx=0 tx=0).stop 2024-04-21T08:42:14.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.491+0000 7fc035831640 1 -- 172.21.15.43:0/3645145653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03019cf30 msgr2=0x7fc03019d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.491+0000 7fc035831640 1 --2- 172.21.15.43:0/3645145653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03019cf30 0x7fc03019d3e0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc028005e30 tx=0x7fc028034670 comp rx=0 tx=0).stop 2024-04-21T08:42:14.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.491+0000 7fc035831640 1 -- 172.21.15.43:0/3645145653 shutdown_connections 2024-04-21T08:42:14.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.491+0000 7fc035831640 1 --2- 172.21.15.43:0/3645145653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03019cf30 0x7fc03019d3e0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.491+0000 7fc035831640 1 --2- 172.21.15.43:0/3645145653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc030107260 0x7fc030198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.491+0000 7fc035831640 1 --2- 172.21.15.43:0/3645145653 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc01c0a3da0 0x7fc01c0a6260 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.491+0000 7fc035831640 1 --2- 172.21.15.43:0/3645145653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc030102190 0x7fc030198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.491+0000 7fc035831640 1 -- 172.21.15.43:0/3645145653 >> 172.21.15.43:0/3645145653 conn(0x7fc0300fd990 msgr2=0x7fc0300ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.491+0000 7fc035831640 1 -- 172.21.15.43:0/3645145653 shutdown_connections 2024-04-21T08:42:14.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.491+0000 7fc035831640 1 -- 172.21.15.43:0/3645145653 wait complete. 2024-04-21T08:42:14.503 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:42:14.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:14 smithi112 ceph-mon[31085]: pgmap v102: 129 pgs: 129 active+clean; 3.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-04-21T08:42:14.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.647+0000 7f583c166640 1 -- 172.21.15.43:0/3214070945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f583410c510 msgr2=0x7f583410e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.647+0000 7f583c166640 1 --2- 172.21.15.43:0/3214070945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f583410c510 0x7f583410e9b0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f582c009ca0 tx=0x7f582c02c230 comp rx=0 tx=0).stop 2024-04-21T08:42:14.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.647+0000 7f583c166640 1 -- 172.21.15.43:0/3214070945 shutdown_connections 2024-04-21T08:42:14.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.647+0000 7f583c166640 1 --2- 172.21.15.43:0/3214070945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f583410c510 0x7f583410e9b0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.647+0000 7f583c166640 1 --2- 172.21.15.43:0/3214070945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5834103af0 0x7f583410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.647+0000 7f583c166640 1 --2- 172.21.15.43:0/3214070945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58341031d0 0x7f58341035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.647+0000 7f583c166640 1 -- 172.21.15.43:0/3214070945 >> 172.21.15.43:0/3214070945 conn(0x7f5834077f50 msgr2=0x7f5834078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.647+0000 7f583c166640 1 -- 172.21.15.43:0/3214070945 shutdown_connections 2024-04-21T08:42:14.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.647+0000 7f583c166640 1 -- 172.21.15.43:0/3214070945 wait complete. 2024-04-21T08:42:14.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.648+0000 7f583c166640 1 Processor -- start 2024-04-21T08:42:14.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.648+0000 7f583c166640 1 -- start start 2024-04-21T08:42:14.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.648+0000 7f583c166640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58341031d0 0x7f58341986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.648+0000 7f583c166640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5834103af0 0x7f5834198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.648+0000 7f583c166640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f583410c510 0x7f583419cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.648+0000 7f583c166640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5834112180 con 0x7f583410c510 2024-04-21T08:42:14.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.648+0000 7f583c166640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5834112000 con 0x7f5834103af0 2024-04-21T08:42:14.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.648+0000 7f583c166640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5834112300 con 0x7f58341031d0 2024-04-21T08:42:14.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.648+0000 7f583a703640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f583410c510 0x7f583419cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.648+0000 7f583a703640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f583410c510 0x7f583419cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36660/0 (socket says 172.21.15.43:36660) 2024-04-21T08:42:14.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.648+0000 7f583a703640 1 -- 172.21.15.43:0/760059902 learned_addr learned my addr 172.21.15.43:0/760059902 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:14.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.648+0000 7f5839f02640 1 --2- 172.21.15.43:0/760059902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58341031d0 0x7f58341986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.648+0000 7f5839701640 1 --2- 172.21.15.43:0/760059902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5834103af0 0x7f5834198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.648+0000 7f583a703640 1 -- 172.21.15.43:0/760059902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58341031d0 msgr2=0x7f58341986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.648+0000 7f583a703640 1 --2- 172.21.15.43:0/760059902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58341031d0 0x7f58341986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.648+0000 7f583a703640 1 -- 172.21.15.43:0/760059902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5834103af0 msgr2=0x7f5834198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.648+0000 7f583a703640 1 --2- 172.21.15.43:0/760059902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5834103af0 0x7f5834198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.648+0000 7f583a703640 1 -- 172.21.15.43:0/760059902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583419d6d0 con 0x7f583410c510 2024-04-21T08:42:14.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.648+0000 7f5839701640 1 --2- 172.21.15.43:0/760059902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5834103af0 0x7f5834198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:14.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.648+0000 7f583a703640 1 --2- 172.21.15.43:0/760059902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f583410c510 0x7f583419cfa0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f582c037040 tx=0x7f582c035e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.649+0000 7f5826ffd640 1 -- 172.21.15.43:0/760059902 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c039040 con 0x7f583410c510 2024-04-21T08:42:14.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.649+0000 7f583c166640 1 -- 172.21.15.43:0/760059902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f583419d960 con 0x7f583410c510 2024-04-21T08:42:14.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.649+0000 7f5826ffd640 1 -- 172.21.15.43:0/760059902 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f582c044070 con 0x7f583410c510 2024-04-21T08:42:14.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.649+0000 7f5826ffd640 1 -- 172.21.15.43:0/760059902 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c0328a0 con 0x7f583410c510 2024-04-21T08:42:14.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.649+0000 7f583c166640 1 -- 172.21.15.43:0/760059902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58341a5250 con 0x7f583410c510 2024-04-21T08:42:14.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.651+0000 7f5826ffd640 1 -- 172.21.15.43:0/760059902 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f582c032a40 con 0x7f583410c510 2024-04-21T08:42:14.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.652+0000 7f583ee31740 1 Processor -- start 2024-04-21T08:42:14.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.652+0000 7f583ee31740 1 -- start start 2024-04-21T08:42:14.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.652+0000 7f583ee31740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56549b46e4d0 0x56549b46e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.652+0000 7f583ee31740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b46ee80 0x56549b477430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.652+0000 7f583ee31740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56549b477a00 0x56549b479e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.652+0000 7f583ee31740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56549b3d31f0 con 0x56549b46ee80 2024-04-21T08:42:14.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.652+0000 7f583ee31740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56549b3704b0 con 0x56549b477a00 2024-04-21T08:42:14.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.652+0000 7f583ee31740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56549b3cf1b0 con 0x56549b46e4d0 2024-04-21T08:42:14.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.652+0000 7f5839f02640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b46ee80 0x56549b477430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.652+0000 7f5839f02640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b46ee80 0x56549b477430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36672/0 (socket says 172.21.15.43:36672) 2024-04-21T08:42:14.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.652+0000 7f5839f02640 1 -- 172.21.15.43:0/3751336874 learned_addr learned my addr 172.21.15.43:0/3751336874 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:14.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.652+0000 7f5826ffd640 1 --2- 172.21.15.43:0/760059902 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f58200a3d20 0x7f58200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.652+0000 7f5839701640 1 --2- 172.21.15.43:0/3751336874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56549b46e4d0 0x56549b46e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.652+0000 7f5839f02640 1 -- 172.21.15.43:0/3751336874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56549b46e4d0 msgr2=0x56549b46e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.652+0000 7f5839f02640 1 --2- 172.21.15.43:0/3751336874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56549b46e4d0 0x56549b46e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.652+0000 7f5839701640 1 --2- 172.21.15.43:0/3751336874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56549b477a00 0x56549b479e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.652+0000 7f5839f02640 1 -- 172.21.15.43:0/3751336874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56549b477a00 msgr2=0x56549b479e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.652+0000 7f5839f02640 1 --2- 172.21.15.43:0/3751336874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56549b477a00 0x56549b479e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.652+0000 7f5839f02640 1 -- 172.21.15.43:0/3751336874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56549b3621b0 con 0x56549b46ee80 2024-04-21T08:42:14.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.652+0000 7f583a703640 1 --2- 172.21.15.43:0/760059902 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f58200a3d20 0x7f58200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.652+0000 7f5839701640 1 --2- 172.21.15.43:0/3751336874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56549b477a00 0x56549b479e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:14.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.653+0000 7f5826ffd640 1 -- 172.21.15.43:0/760059902 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f582c0f2fe0 con 0x7f583410c510 2024-04-21T08:42:14.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.653+0000 7f5839f02640 1 --2- 172.21.15.43:0/3751336874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b46ee80 0x56549b477430 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f5830009c80 tx=0x7f58300117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7259abbaf0fb766 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.653+0000 7f581ffff640 1 -- 172.21.15.43:0/3751336874 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5830012450 con 0x56549b46ee80 2024-04-21T08:42:14.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.653+0000 7f581ffff640 1 -- 172.21.15.43:0/3751336874 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5830012aa0 con 0x56549b46ee80 2024-04-21T08:42:14.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.653+0000 7f581ffff640 1 -- 172.21.15.43:0/3751336874 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583000fae0 con 0x56549b46ee80 2024-04-21T08:42:14.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.654+0000 7f583ee31740 1 -- 172.21.15.43:0/3751336874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b46ee80 msgr2=0x56549b477430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.654+0000 7f583ee31740 1 --2- 172.21.15.43:0/3751336874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b46ee80 0x56549b477430 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f5830009c80 tx=0x7f58300117b0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.654+0000 7f583a703640 1 --2- 172.21.15.43:0/760059902 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f58200a3d20 0x7f58200a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f582c0bb120 tx=0x7f582c0095c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.654+0000 7f583ee31740 1 -- 172.21.15.43:0/3751336874 shutdown_connections 2024-04-21T08:42:14.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.654+0000 7f583ee31740 1 --2- 172.21.15.43:0/3751336874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56549b477a00 0x56549b479e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.654+0000 7f583ee31740 1 --2- 172.21.15.43:0/3751336874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b46ee80 0x56549b477430 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.654+0000 7f583ee31740 1 --2- 172.21.15.43:0/3751336874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56549b46e4d0 0x56549b46e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.654+0000 7f583ee31740 1 -- 172.21.15.43:0/3751336874 >> 172.21.15.43:0/3751336874 conn(0x56549b468ba0 msgr2=0x56549b468f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.654+0000 7f583ee31740 1 -- 172.21.15.43:0/3751336874 shutdown_connections 2024-04-21T08:42:14.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.654+0000 7f583ee31740 1 -- 172.21.15.43:0/3751336874 wait complete. 2024-04-21T08:42:14.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.654+0000 7f583ee31740 1 -- start start 2024-04-21T08:42:14.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.654+0000 7f583ee31740 1 Processor -- start 2024-04-21T08:42:14.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.654+0000 7f583ee31740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56549b46e4d0 0x56549b477290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.655+0000 7f583ee31740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56549b46ee80 0x56549b474620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.655+0000 7f583ee31740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b477a00 0x56549b474b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.655+0000 7f583ee31740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56549b47a970 con 0x56549b477a00 2024-04-21T08:42:14.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.655+0000 7f583ee31740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56549b38c500 con 0x56549b46e4d0 2024-04-21T08:42:14.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.655+0000 7f583ee31740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56549b47aaf0 con 0x56549b46ee80 2024-04-21T08:42:14.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.655+0000 7f583ee31740 10 client.0 _ll_register_callbacks cb 0x56549b35ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:14.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.655+0000 7f5839701640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b477a00 0x56549b474b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.655+0000 7f5839701640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b477a00 0x56549b474b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36674/0 (socket says 172.21.15.43:36674) 2024-04-21T08:42:14.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.655+0000 7f5839701640 1 -- 172.21.15.43:0/3371927042 learned_addr learned my addr 172.21.15.43:0/3371927042 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:14.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.655+0000 7f5839f02640 1 --2- 172.21.15.43:0/3371927042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56549b46ee80 0x56549b474620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.655+0000 7f5839701640 1 --2- 172.21.15.43:0/3371927042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56549b46e4d0 0x56549b477290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.655+0000 7f5839701640 1 -- 172.21.15.43:0/3371927042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56549b46ee80 msgr2=0x56549b474620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.655+0000 7f5839701640 1 --2- 172.21.15.43:0/3371927042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56549b46ee80 0x56549b474620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.655+0000 7f5839701640 1 -- 172.21.15.43:0/3371927042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56549b46e4d0 msgr2=0x56549b477290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.655+0000 7f5839701640 1 --2- 172.21.15.43:0/3371927042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56549b46e4d0 0x56549b477290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.655+0000 7f5839701640 1 -- 172.21.15.43:0/3371927042 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56549b0e53f0 con 0x56549b477a00 2024-04-21T08:42:14.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.655+0000 7f5839701640 1 --2- 172.21.15.43:0/3371927042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b477a00 0x56549b474b60 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f5828013b70 tx=0x7f58280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc806e46f591a831 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.655+0000 7f581d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:14.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.655+0000 7f581d7fa640 1 -- 172.21.15.43:0/3371927042 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5828004220 con 0x56549b477a00 2024-04-21T08:42:14.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.655+0000 7f581d7fa640 1 -- 172.21.15.43:0/3371927042 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f58280043c0 con 0x56549b477a00 2024-04-21T08:42:14.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.655+0000 7f583ee31740 1 -- 172.21.15.43:0/3371927042 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5830012aa0 con 0x56549b477a00 2024-04-21T08:42:14.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.656+0000 7f581d7fa640 1 -- 172.21.15.43:0/3371927042 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5828026050 con 0x56549b477a00 2024-04-21T08:42:14.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.656+0000 7f583ee31740 10 client.17127 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:14.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.656+0000 7f583ee31740 1 -- 172.21.15.43:0/3371927042 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56549b1af7d0 con 0x56549b477a00 2024-04-21T08:42:14.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.656+0000 7f581d7fa640 1 -- 172.21.15.43:0/3371927042 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5830012aa0 con 0x56549b477a00 2024-04-21T08:42:14.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.656+0000 7f581d7fa640 1 -- 172.21.15.43:0/3371927042 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f582800f040 con 0x56549b477a00 2024-04-21T08:42:14.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.656+0000 7f583ee31740 10 client.17127 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:14.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.656+0000 7f583ee31740 10 client.17127 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:42:14.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.656+0000 7f583ee31740 20 client.17127 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:14.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.656+0000 7f583ee31740 1 --2- 172.21.15.43:0/3371927042 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56549b487b00 0x56549b489f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.656+0000 7f583ee31740 4 client.17127 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:14.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.656+0000 7f583ee31740 1 -- 172.21.15.43:0/3371927042 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56549b372930 con 0x56549b487b00 2024-04-21T08:42:14.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.656+0000 7f5839f02640 1 --2- 172.21.15.43:0/3371927042 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56549b487b00 0x56549b489f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.657+0000 7f5839f02640 1 --2- 172.21.15.43:0/3371927042 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56549b487b00 0x56549b489f10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a3c26ea2d4d3d7a3 server_cookie=30a580e9c0cfb6d7 in_seq=0 out_seq=0 2024-04-21T08:42:14.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.657+0000 7f581d7fa640 10 client.17127 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:42:14.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.658+0000 7f581d7fa640 1 -- 172.21.15.43:0/3371927042 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56549b372930 con 0x56549b487b00 2024-04-21T08:42:14.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.658+0000 7f581d7fa640 10 client.17127 handle_command_reply: tid=0 2024-04-21T08:42:14.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.658+0000 7f583ee31740 1 client.17127 shutdown 2024-04-21T08:42:14.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.658+0000 7f583ee31740 1 -- 172.21.15.43:0/3371927042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b477a00 msgr2=0x56549b474b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.658+0000 7f583ee31740 1 --2- 172.21.15.43:0/3371927042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b477a00 0x56549b474b60 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f5828013b70 tx=0x7f58280046c0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.658+0000 7f583ee31740 1 -- 172.21.15.43:0/3371927042 shutdown_connections 2024-04-21T08:42:14.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.658+0000 7f583ee31740 1 --2- 172.21.15.43:0/3371927042 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56549b487b00 0x56549b489f10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.658+0000 7f583ee31740 1 --2- 172.21.15.43:0/3371927042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b477a00 0x56549b474b60 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.658+0000 7f581d7fa640 0 client.17127 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:42:14.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.658+0000 7f583ee31740 1 --2- 172.21.15.43:0/3371927042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56549b46ee80 0x56549b474620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.658+0000 7f583ee31740 1 --2- 172.21.15.43:0/3371927042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56549b46e4d0 0x56549b477290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.658+0000 7f583ee31740 1 -- 172.21.15.43:0/3371927042 >> 172.21.15.43:0/3371927042 conn(0x56549b468ba0 msgr2=0x56549b46a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.658+0000 7f583ee31740 1 -- 172.21.15.43:0/3371927042 shutdown_connections 2024-04-21T08:42:14.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.658+0000 7f583ee31740 1 -- 172.21.15.43:0/3371927042 wait complete. 2024-04-21T08:42:14.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.658+0000 7f583ee31740 20 client.17127 trim_cache size 0 max 16384 2024-04-21T08:42:14.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.671+0000 7f583ee31740 1 Processor -- start 2024-04-21T08:42:14.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.671+0000 7f583ee31740 1 -- start start 2024-04-21T08:42:14.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.672+0000 7f583ee31740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56549b46e4d0 0x56549b473f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.672+0000 7f583ee31740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56549b46ee80 0x56549b480180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.672+0000 7f583ee31740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b477a00 0x56549b467b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.672+0000 7f583ee31740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f58341af4f0 con 0x56549b477a00 2024-04-21T08:42:14.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.672+0000 7f583ee31740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f58341b1030 con 0x56549b46e4d0 2024-04-21T08:42:14.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.672+0000 7f583ee31740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58341afbc0 con 0x56549b46ee80 2024-04-21T08:42:14.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.672+0000 7f5839701640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b477a00 0x56549b467b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.672+0000 7f5839701640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b477a00 0x56549b467b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36688/0 (socket says 172.21.15.43:36688) 2024-04-21T08:42:14.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.672+0000 7f5839701640 1 -- 172.21.15.43:0/1501774145 learned_addr learned my addr 172.21.15.43:0/1501774145 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:14.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.672+0000 7f5839f02640 1 --2- 172.21.15.43:0/1501774145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56549b46ee80 0x56549b480180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.672+0000 7f5839701640 1 --2- 172.21.15.43:0/1501774145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56549b46e4d0 0x56549b473f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.672+0000 7f5839701640 1 -- 172.21.15.43:0/1501774145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56549b46ee80 msgr2=0x56549b480180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.672+0000 7f5839701640 1 --2- 172.21.15.43:0/1501774145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56549b46ee80 0x56549b480180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.672+0000 7f5839701640 1 -- 172.21.15.43:0/1501774145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56549b46e4d0 msgr2=0x56549b473f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.672+0000 7f5839701640 1 --2- 172.21.15.43:0/1501774145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56549b46e4d0 0x56549b473f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.672+0000 7f5839701640 1 -- 172.21.15.43:0/1501774145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56549b472b70 con 0x56549b477a00 2024-04-21T08:42:14.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.672+0000 7f5839701640 1 --2- 172.21.15.43:0/1501774145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b477a00 0x56549b467b40 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x56549b472e00 tx=0x7f5828005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12faf47f67a8407a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.672+0000 7f581f7fe640 1 -- 172.21.15.43:0/1501774145 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582800e7a0 con 0x56549b477a00 2024-04-21T08:42:14.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.672+0000 7f581f7fe640 1 -- 172.21.15.43:0/1501774145 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5828005520 con 0x56549b477a00 2024-04-21T08:42:14.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.672+0000 7f581f7fe640 1 -- 172.21.15.43:0/1501774145 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5828004990 con 0x56549b477a00 2024-04-21T08:42:14.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.673+0000 7f583ee31740 1 -- 172.21.15.43:0/1501774145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b477a00 msgr2=0x56549b467b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.673+0000 7f583ee31740 1 --2- 172.21.15.43:0/1501774145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b477a00 0x56549b467b40 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x56549b472e00 tx=0x7f5828005c90 comp rx=0 tx=0).stop 2024-04-21T08:42:14.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.673+0000 7f583ee31740 1 -- 172.21.15.43:0/1501774145 shutdown_connections 2024-04-21T08:42:14.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.673+0000 7f583ee31740 1 --2- 172.21.15.43:0/1501774145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b477a00 0x56549b467b40 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.673+0000 7f583ee31740 1 --2- 172.21.15.43:0/1501774145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56549b46ee80 0x56549b480180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.673+0000 7f583ee31740 1 --2- 172.21.15.43:0/1501774145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56549b46e4d0 0x56549b473f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.673+0000 7f583ee31740 1 -- 172.21.15.43:0/1501774145 >> 172.21.15.43:0/1501774145 conn(0x56549b468ba0 msgr2=0x56549b46a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.673+0000 7f583ee31740 1 -- 172.21.15.43:0/1501774145 shutdown_connections 2024-04-21T08:42:14.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.673+0000 7f583ee31740 1 -- 172.21.15.43:0/1501774145 wait complete. 2024-04-21T08:42:14.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.673+0000 7f583ee31740 1 -- start start 2024-04-21T08:42:14.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.673+0000 7f583ee31740 1 Processor -- start 2024-04-21T08:42:14.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.674+0000 7f583ee31740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b46e4d0 0x56549b4ea7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.674+0000 7f583ee31740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56549b46ee80 0x56549b4ead00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.674+0000 7f583ee31740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56549b477a00 0x56549b476390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.674+0000 7f583ee31740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f58341b0960 con 0x56549b46e4d0 2024-04-21T08:42:14.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.674+0000 7f583ee31740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f58341b0290 con 0x56549b477a00 2024-04-21T08:42:14.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.674+0000 7f583ee31740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58341ad4d0 con 0x56549b46ee80 2024-04-21T08:42:14.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.674+0000 7f583ee31740 10 client.0 _ll_register_callbacks cb 0x7f5828013700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:14.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.674+0000 7f5839701640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b46e4d0 0x56549b4ea7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.674+0000 7f5839701640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b46e4d0 0x56549b4ea7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36704/0 (socket says 172.21.15.43:36704) 2024-04-21T08:42:14.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.674+0000 7f5839701640 1 -- 172.21.15.43:0/2021405374 learned_addr learned my addr 172.21.15.43:0/2021405374 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:14.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.674+0000 7f5839f02640 1 --2- 172.21.15.43:0/2021405374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56549b46ee80 0x56549b4ead00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.674+0000 7f5839701640 1 -- 172.21.15.43:0/2021405374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56549b46ee80 msgr2=0x56549b4ead00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.674+0000 7f5839701640 1 --2- 172.21.15.43:0/2021405374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56549b46ee80 0x56549b4ead00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.674+0000 7f5839701640 1 -- 172.21.15.43:0/2021405374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56549b477a00 msgr2=0x56549b476390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.674+0000 7f5839701640 1 --2- 172.21.15.43:0/2021405374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56549b477a00 0x56549b476390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.674+0000 7f5839701640 1 -- 172.21.15.43:0/2021405374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56549b476a80 con 0x56549b46e4d0 2024-04-21T08:42:14.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.674+0000 7f5839701640 1 --2- 172.21.15.43:0/2021405374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b46e4d0 0x56549b4ea7c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f5828011d40 tx=0x7f582801e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876bedac5fd40e2b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.674+0000 7f581d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:14.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.675+0000 7f581d7fa640 1 -- 172.21.15.43:0/2021405374 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582800e7a0 con 0x56549b46e4d0 2024-04-21T08:42:14.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.675+0000 7f581d7fa640 1 -- 172.21.15.43:0/2021405374 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f582801e3a0 con 0x56549b46e4d0 2024-04-21T08:42:14.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.675+0000 7f583ee31740 1 -- 172.21.15.43:0/2021405374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5828005520 con 0x56549b46e4d0 2024-04-21T08:42:14.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.675+0000 7f581d7fa640 1 -- 172.21.15.43:0/2021405374 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f582801ece0 con 0x56549b46e4d0 2024-04-21T08:42:14.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.675+0000 7f583ee31740 10 client.17133 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:14.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.675+0000 7f583ee31740 1 -- 172.21.15.43:0/2021405374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56549b4f1fc0 con 0x56549b46e4d0 2024-04-21T08:42:14.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.675+0000 7f581d7fa640 1 -- 172.21.15.43:0/2021405374 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5828005520 con 0x56549b46e4d0 2024-04-21T08:42:14.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.675+0000 7f581d7fa640 1 -- 172.21.15.43:0/2021405374 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5828034070 con 0x56549b46e4d0 2024-04-21T08:42:14.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.675+0000 7f583ee31740 10 client.17133 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:14.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.675+0000 7f583ee31740 10 client.17133 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:42:14.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.675+0000 7f583ee31740 20 client.17133 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:14.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.675+0000 7f583ee31740 1 --2- 172.21.15.43:0/2021405374 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56549b4f25d0 0x56549b4f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.675+0000 7f583ee31740 4 client.17133 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:42:14.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.675+0000 7f583ee31740 1 -- 172.21.15.43:0/2021405374 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56549b3e0c10 con 0x56549b4f25d0 2024-04-21T08:42:14.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.675+0000 7f5839f02640 1 --2- 172.21.15.43:0/2021405374 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56549b4f25d0 0x56549b4f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.676+0000 7f581d7fa640 10 client.17133 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:42:14.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.676+0000 7f5839f02640 1 --2- 172.21.15.43:0/2021405374 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56549b4f25d0 0x56549b4f49c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e95fbdc9f3cefbe9 server_cookie=1ba71a3200b58678 in_seq=0 out_seq=0 2024-04-21T08:42:14.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.676+0000 7f581d7fa640 1 -- 172.21.15.43:0/2021405374 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56549b3e0c10 con 0x56549b4f25d0 2024-04-21T08:42:14.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.676+0000 7f581d7fa640 10 client.17133 handle_command_reply: tid=0 2024-04-21T08:42:14.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.676+0000 7f583ee31740 1 client.17133 shutdown 2024-04-21T08:42:14.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.676+0000 7f583ee31740 1 -- 172.21.15.43:0/2021405374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b46e4d0 msgr2=0x56549b4ea7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.676+0000 7f583ee31740 1 --2- 172.21.15.43:0/2021405374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b46e4d0 0x56549b4ea7c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f5828011d40 tx=0x7f582801e880 comp rx=0 tx=0).stop 2024-04-21T08:42:14.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.676+0000 7f583ee31740 1 -- 172.21.15.43:0/2021405374 shutdown_connections 2024-04-21T08:42:14.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.676+0000 7f583ee31740 1 --2- 172.21.15.43:0/2021405374 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56549b4f25d0 0x56549b4f49c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.676+0000 7f583ee31740 1 --2- 172.21.15.43:0/2021405374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56549b477a00 0x56549b476390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.676+0000 7f581d7fa640 0 client.17133 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:42:14.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.676+0000 7f583ee31740 1 --2- 172.21.15.43:0/2021405374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56549b46ee80 0x56549b4ead00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.676+0000 7f583ee31740 1 --2- 172.21.15.43:0/2021405374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56549b46e4d0 0x56549b4ea7c0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.676+0000 7f583ee31740 1 -- 172.21.15.43:0/2021405374 >> 172.21.15.43:0/2021405374 conn(0x56549b468ba0 msgr2=0x56549b46fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.676+0000 7f583ee31740 1 -- 172.21.15.43:0/2021405374 shutdown_connections 2024-04-21T08:42:14.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.676+0000 7f583ee31740 1 -- 172.21.15.43:0/2021405374 wait complete. 2024-04-21T08:42:14.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.676+0000 7f583ee31740 20 client.17133 trim_cache size 0 max 16384 2024-04-21T08:42:14.683 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:42:14.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.677+0000 7f583c166640 1 -- 172.21.15.43:0/760059902 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f58200a3d20 msgr2=0x7f58200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.677+0000 7f583c166640 1 --2- 172.21.15.43:0/760059902 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f58200a3d20 0x7f58200a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f582c0bb120 tx=0x7f582c0095c0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.677+0000 7f583c166640 1 -- 172.21.15.43:0/760059902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f583410c510 msgr2=0x7f583419cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.677+0000 7f583c166640 1 --2- 172.21.15.43:0/760059902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f583410c510 0x7f583419cfa0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f582c037040 tx=0x7f582c035e20 comp rx=0 tx=0).stop 2024-04-21T08:42:14.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.677+0000 7f583c166640 1 -- 172.21.15.43:0/760059902 shutdown_connections 2024-04-21T08:42:14.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.677+0000 7f583c166640 1 --2- 172.21.15.43:0/760059902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f583410c510 0x7f583419cfa0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.677+0000 7f583c166640 1 --2- 172.21.15.43:0/760059902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5834103af0 0x7f5834198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.677+0000 7f583c166640 1 --2- 172.21.15.43:0/760059902 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f58200a3d20 0x7f58200a61e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.677+0000 7f583c166640 1 --2- 172.21.15.43:0/760059902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58341031d0 0x7f58341986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.677+0000 7f583c166640 1 -- 172.21.15.43:0/760059902 >> 172.21.15.43:0/760059902 conn(0x7f5834077f50 msgr2=0x7f58340fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.677+0000 7f583c166640 1 -- 172.21.15.43:0/760059902 shutdown_connections 2024-04-21T08:42:14.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.677+0000 7f583c166640 1 -- 172.21.15.43:0/760059902 wait complete. 2024-04-21T08:42:14.688 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:42:14.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.792+0000 7fb49b5f4640 1 -- 172.21.15.43:0/2295510879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb494102b40 msgr2=0x7fb494106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.792+0000 7fb49b5f4640 1 --2- 172.21.15.43:0/2295510879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb494102b40 0x7fb494106b80 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb484009900 tx=0x7fb48402bc30 comp rx=0 tx=0).stop 2024-04-21T08:42:14.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.792+0000 7fb49b5f4640 1 -- 172.21.15.43:0/2295510879 shutdown_connections 2024-04-21T08:42:14.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.792+0000 7fb49b5f4640 1 --2- 172.21.15.43:0/2295510879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb494107260 0x7fb49410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.792+0000 7fb49b5f4640 1 --2- 172.21.15.43:0/2295510879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb494102b40 0x7fb494106b80 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.792+0000 7fb49b5f4640 1 --2- 172.21.15.43:0/2295510879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb494102190 0x7fb494102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.792+0000 7fb49b5f4640 1 -- 172.21.15.43:0/2295510879 >> 172.21.15.43:0/2295510879 conn(0x7fb4940fd990 msgr2=0x7fb4940ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.792+0000 7fb49b5f4640 1 -- 172.21.15.43:0/2295510879 shutdown_connections 2024-04-21T08:42:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.792+0000 7fb49b5f4640 1 -- 172.21.15.43:0/2295510879 wait complete. 2024-04-21T08:42:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.793+0000 7fb49b5f4640 1 Processor -- start 2024-04-21T08:42:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.793+0000 7fb49b5f4640 1 -- start start 2024-04-21T08:42:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.793+0000 7fb49b5f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb494102190 0x7fb494074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.793+0000 7fb49b5f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb494107260 0x7fb494074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.793+0000 7fb49b5f4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb494074cd0 0x7fb4941a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.793+0000 7fb49b5f4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb494112180 con 0x7fb494107260 2024-04-21T08:42:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.793+0000 7fb49b5f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb494112000 con 0x7fb494102190 2024-04-21T08:42:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.793+0000 7fb49b5f4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb494112300 con 0x7fb494074cd0 2024-04-21T08:42:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.793+0000 7fb498b8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb494107260 0x7fb494074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.793+0000 7fb498b8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb494107260 0x7fb494074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36724/0 (socket says 172.21.15.43:36724) 2024-04-21T08:42:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.793+0000 7fb498b8f640 1 -- 172.21.15.43:0/3759222577 learned_addr learned my addr 172.21.15.43:0/3759222577 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.793+0000 7fb499390640 1 --2- 172.21.15.43:0/3759222577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb494102190 0x7fb494074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.793+0000 7fb499b91640 1 --2- 172.21.15.43:0/3759222577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb494074cd0 0x7fb4941a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.794+0000 7fb498b8f640 1 -- 172.21.15.43:0/3759222577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb494074cd0 msgr2=0x7fb4941a6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.794+0000 7fb498b8f640 1 --2- 172.21.15.43:0/3759222577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb494074cd0 0x7fb4941a6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.794+0000 7fb498b8f640 1 -- 172.21.15.43:0/3759222577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb494102190 msgr2=0x7fb494074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.794+0000 7fb498b8f640 1 --2- 172.21.15.43:0/3759222577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb494102190 0x7fb494074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.794+0000 7fb498b8f640 1 -- 172.21.15.43:0/3759222577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4941a7030 con 0x7fb494107260 2024-04-21T08:42:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.794+0000 7fb498b8f640 1 --2- 172.21.15.43:0/3759222577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb494107260 0x7fb494074790 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb484036040 tx=0x7fb48402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.794+0000 7fb48a7fc640 1 -- 172.21.15.43:0/3759222577 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb484043020 con 0x7fb494107260 2024-04-21T08:42:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.794+0000 7fb49b5f4640 1 -- 172.21.15.43:0/3759222577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4941a72c0 con 0x7fb494107260 2024-04-21T08:42:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.794+0000 7fb48a7fc640 1 -- 172.21.15.43:0/3759222577 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb48402cbe0 con 0x7fb494107260 2024-04-21T08:42:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.794+0000 7fb48a7fc640 1 -- 172.21.15.43:0/3759222577 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb484003bf0 con 0x7fb494107260 2024-04-21T08:42:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.794+0000 7fb49b5f4640 1 -- 172.21.15.43:0/3759222577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4941a7860 con 0x7fb494107260 2024-04-21T08:42:14.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.796+0000 7fb48a7fc640 1 -- 172.21.15.43:0/3759222577 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb484046020 con 0x7fb494107260 2024-04-21T08:42:14.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.797+0000 7fb49e1e9740 1 Processor -- start 2024-04-21T08:42:14.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.797+0000 7fb49e1e9740 1 -- start start 2024-04-21T08:42:14.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.797+0000 7fb49e1e9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587d38454d0 0x5587d38458b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.797+0000 7fb49e1e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d3845e80 0x5587d384e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.797+0000 7fb49e1e9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d384ea00 0x5587d3850e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.797+0000 7fb49e1e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5587d37aa1f0 con 0x5587d3845e80 2024-04-21T08:42:14.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.797+0000 7fb49e1e9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5587d37474b0 con 0x5587d384ea00 2024-04-21T08:42:14.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.797+0000 7fb49e1e9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5587d37a61b0 con 0x5587d38454d0 2024-04-21T08:42:14.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.797+0000 7fb48a7fc640 1 --2- 172.21.15.43:0/3759222577 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb4800a3d50 0x7fb4800a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.797+0000 7fb499390640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d3845e80 0x5587d384e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.797+0000 7fb499390640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d3845e80 0x5587d384e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36740/0 (socket says 172.21.15.43:36740) 2024-04-21T08:42:14.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.797+0000 7fb499390640 1 -- 172.21.15.43:0/3609478589 learned_addr learned my addr 172.21.15.43:0/3609478589 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:14.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.797+0000 7fb499b91640 1 --2- 172.21.15.43:0/3759222577 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb4800a3d50 0x7fb4800a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.797+0000 7fb499b91640 1 --2- 172.21.15.43:0/3609478589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587d38454d0 0x5587d38458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.797+0000 7fb48a7fc640 1 -- 172.21.15.43:0/3759222577 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fb4840f2cd0 con 0x7fb494107260 2024-04-21T08:42:14.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.797+0000 7fb498b8f640 1 --2- 172.21.15.43:0/3609478589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d384ea00 0x5587d3850e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.798+0000 7fb499390640 1 -- 172.21.15.43:0/3609478589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587d38454d0 msgr2=0x5587d38458b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.798+0000 7fb499390640 1 --2- 172.21.15.43:0/3609478589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587d38454d0 0x5587d38458b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.798+0000 7fb499390640 1 -- 172.21.15.43:0/3609478589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d384ea00 msgr2=0x5587d3850e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.798+0000 7fb499390640 1 --2- 172.21.15.43:0/3609478589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d384ea00 0x5587d3850e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.798+0000 7fb499390640 1 -- 172.21.15.43:0/3609478589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587d37391b0 con 0x5587d3845e80 2024-04-21T08:42:14.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.798+0000 7fb498b8f640 1 --2- 172.21.15.43:0/3609478589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d384ea00 0x5587d3850e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:14.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.798+0000 7fb499b91640 1 --2- 172.21.15.43:0/3759222577 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb4800a3d50 0x7fb4800a6210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb48c005c40 tx=0x7fb48c009c90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.798+0000 7fb499390640 1 --2- 172.21.15.43:0/3609478589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d3845e80 0x5587d384e430 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb49000bd10 tx=0x7fb4900107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82d232db60f2d59 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.798+0000 7fb47f7fe640 1 -- 172.21.15.43:0/3609478589 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490004030 con 0x5587d3845e80 2024-04-21T08:42:14.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.798+0000 7fb47f7fe640 1 -- 172.21.15.43:0/3609478589 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4900035f0 con 0x5587d3845e80 2024-04-21T08:42:14.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.798+0000 7fb47f7fe640 1 -- 172.21.15.43:0/3609478589 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4900116a0 con 0x5587d3845e80 2024-04-21T08:42:14.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.799+0000 7fb49e1e9740 1 -- 172.21.15.43:0/3609478589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d3845e80 msgr2=0x5587d384e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.799+0000 7fb49e1e9740 1 --2- 172.21.15.43:0/3609478589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d3845e80 0x5587d384e430 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb49000bd10 tx=0x7fb4900107a0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.799+0000 7fb49e1e9740 1 -- 172.21.15.43:0/3609478589 shutdown_connections 2024-04-21T08:42:14.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.799+0000 7fb49e1e9740 1 --2- 172.21.15.43:0/3609478589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d384ea00 0x5587d3850e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.799+0000 7fb49e1e9740 1 --2- 172.21.15.43:0/3609478589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d3845e80 0x5587d384e430 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.799+0000 7fb49e1e9740 1 --2- 172.21.15.43:0/3609478589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587d38454d0 0x5587d38458b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.799+0000 7fb49e1e9740 1 -- 172.21.15.43:0/3609478589 >> 172.21.15.43:0/3609478589 conn(0x5587d383fba0 msgr2=0x5587d383ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.799+0000 7fb49e1e9740 1 -- 172.21.15.43:0/3609478589 shutdown_connections 2024-04-21T08:42:14.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.799+0000 7fb49e1e9740 1 -- 172.21.15.43:0/3609478589 wait complete. 2024-04-21T08:42:14.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.799+0000 7fb49e1e9740 1 -- start start 2024-04-21T08:42:14.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.799+0000 7fb49e1e9740 1 Processor -- start 2024-04-21T08:42:14.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.800+0000 7fb49e1e9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587d38454d0 0x5587d384e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.800+0000 7fb49e1e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d3845e80 0x5587d384b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.800+0000 7fb49e1e9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d384ea00 0x5587d384bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.800+0000 7fb49e1e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5587d3851970 con 0x5587d3845e80 2024-04-21T08:42:14.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.800+0000 7fb49e1e9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5587d3763500 con 0x5587d384ea00 2024-04-21T08:42:14.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.800+0000 7fb49e1e9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5587d3851af0 con 0x5587d38454d0 2024-04-21T08:42:14.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.800+0000 7fb49e1e9740 10 client.0 _ll_register_callbacks cb 0x5587d3731b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:14.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.800+0000 7fb498b8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d3845e80 0x5587d384b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.800+0000 7fb498b8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d3845e80 0x5587d384b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36754/0 (socket says 172.21.15.43:36754) 2024-04-21T08:42:14.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.800+0000 7fb498b8f640 1 -- 172.21.15.43:0/1998454621 learned_addr learned my addr 172.21.15.43:0/1998454621 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:14.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.800+0000 7fb499390640 1 --2- 172.21.15.43:0/1998454621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587d38454d0 0x5587d384e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.800+0000 7fb499b91640 1 --2- 172.21.15.43:0/1998454621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d384ea00 0x5587d384bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.800+0000 7fb498b8f640 1 -- 172.21.15.43:0/1998454621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587d38454d0 msgr2=0x5587d384e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.800+0000 7fb498b8f640 1 --2- 172.21.15.43:0/1998454621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587d38454d0 0x5587d384e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.800+0000 7fb498b8f640 1 -- 172.21.15.43:0/1998454621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d384ea00 msgr2=0x5587d384bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.800+0000 7fb498b8f640 1 --2- 172.21.15.43:0/1998454621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d384ea00 0x5587d384bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.800+0000 7fb498b8f640 1 -- 172.21.15.43:0/1998454621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587d34bc3f0 con 0x5587d3845e80 2024-04-21T08:42:14.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.800+0000 7fb499b91640 1 --2- 172.21.15.43:0/1998454621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d384ea00 0x5587d384bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:14.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.800+0000 7fb498b8f640 1 --2- 172.21.15.43:0/1998454621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d3845e80 0x5587d384b680 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb4840bb380 tx=0x7fb4840be840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7319291d25f4d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.800+0000 7fb47cff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:14.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.800+0000 7fb47cff9640 1 -- 172.21.15.43:0/1998454621 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4840c7620 con 0x5587d3845e80 2024-04-21T08:42:14.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.800+0000 7fb47cff9640 1 -- 172.21.15.43:0/1998454621 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fb4840bb020 con 0x5587d3845e80 2024-04-21T08:42:14.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.801+0000 7fb49e1e9740 1 -- 172.21.15.43:0/1998454621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4900035f0 con 0x5587d3845e80 2024-04-21T08:42:14.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.801+0000 7fb47cff9640 1 -- 172.21.15.43:0/1998454621 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb4840be9b0 con 0x5587d3845e80 2024-04-21T08:42:14.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.801+0000 7fb49e1e9740 10 client.17145 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:14.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.801+0000 7fb49e1e9740 1 -- 172.21.15.43:0/1998454621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5587d35867d0 con 0x5587d3845e80 2024-04-21T08:42:14.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.801+0000 7fb47cff9640 1 -- 172.21.15.43:0/1998454621 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4900035f0 con 0x5587d3845e80 2024-04-21T08:42:14.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.801+0000 7fb47cff9640 1 -- 172.21.15.43:0/1998454621 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb4840bfce0 con 0x5587d3845e80 2024-04-21T08:42:14.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.801+0000 7fb49e1e9740 10 client.17145 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:14.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.801+0000 7fb49e1e9740 10 client.17145 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:42:14.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.801+0000 7fb49e1e9740 20 client.17145 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:14.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.801+0000 7fb49e1e9740 1 --2- 172.21.15.43:0/1998454621 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5587d385ec30 0x5587d3861060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.801+0000 7fb49e1e9740 4 client.17145 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:14.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.801+0000 7fb49e1e9740 1 -- 172.21.15.43:0/1998454621 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587d3749930 con 0x5587d385ec30 2024-04-21T08:42:14.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.801+0000 7fb499390640 1 --2- 172.21.15.43:0/1998454621 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5587d385ec30 0x5587d3861060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.802+0000 7fb499390640 1 --2- 172.21.15.43:0/1998454621 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5587d385ec30 0x5587d3861060 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f53786e8cc1a0805 server_cookie=28fd3ca56f3daab in_seq=0 out_seq=0 2024-04-21T08:42:14.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.802+0000 7fb47cff9640 10 client.17145 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:42:14.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.803+0000 7fb47cff9640 1 -- 172.21.15.43:0/1998454621 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5587d3749930 con 0x5587d385ec30 2024-04-21T08:42:14.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.803+0000 7fb47cff9640 10 client.17145 handle_command_reply: tid=0 2024-04-21T08:42:14.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.803+0000 7fb49e1e9740 1 client.17145 shutdown 2024-04-21T08:42:14.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.803+0000 7fb49e1e9740 1 -- 172.21.15.43:0/1998454621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d3845e80 msgr2=0x5587d384b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.803+0000 7fb49e1e9740 1 --2- 172.21.15.43:0/1998454621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d3845e80 0x5587d384b680 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb4840bb380 tx=0x7fb4840be840 comp rx=0 tx=0).stop 2024-04-21T08:42:14.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.803+0000 7fb49e1e9740 1 -- 172.21.15.43:0/1998454621 shutdown_connections 2024-04-21T08:42:14.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.803+0000 7fb49e1e9740 1 --2- 172.21.15.43:0/1998454621 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5587d385ec30 0x5587d3861060 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.803+0000 7fb49e1e9740 1 --2- 172.21.15.43:0/1998454621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d384ea00 0x5587d384bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.803+0000 7fb47cff9640 0 client.17145 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:42:14.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.803+0000 7fb49e1e9740 1 --2- 172.21.15.43:0/1998454621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d3845e80 0x5587d384b680 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.803+0000 7fb49e1e9740 1 --2- 172.21.15.43:0/1998454621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587d38454d0 0x5587d384e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.803+0000 7fb49e1e9740 1 -- 172.21.15.43:0/1998454621 >> 172.21.15.43:0/1998454621 conn(0x5587d383fba0 msgr2=0x5587d38418f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.803+0000 7fb49e1e9740 1 -- 172.21.15.43:0/1998454621 shutdown_connections 2024-04-21T08:42:14.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.803+0000 7fb49e1e9740 1 -- 172.21.15.43:0/1998454621 wait complete. 2024-04-21T08:42:14.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.803+0000 7fb49e1e9740 20 client.17145 trim_cache size 0 max 16384 2024-04-21T08:42:14.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.816+0000 7fb49e1e9740 1 Processor -- start 2024-04-21T08:42:14.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.816+0000 7fb49e1e9740 1 -- start start 2024-04-21T08:42:14.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.816+0000 7fb49e1e9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d38454d0 0x5587d385a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.817+0000 7fb49e1e9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587d3845e80 0x5587d383ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.817+0000 7fb49e1e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d384ea00 0x5587d384ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.817+0000 7fb49e1e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb4941b16e0 con 0x5587d384ea00 2024-04-21T08:42:14.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.817+0000 7fb49e1e9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb4941b3220 con 0x5587d38454d0 2024-04-21T08:42:14.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.817+0000 7fb49e1e9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb4941b1db0 con 0x5587d3845e80 2024-04-21T08:42:14.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.817+0000 7fb499b91640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d384ea00 0x5587d384ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.817+0000 7fb499b91640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d384ea00 0x5587d384ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36760/0 (socket says 172.21.15.43:36760) 2024-04-21T08:42:14.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.817+0000 7fb499b91640 1 -- 172.21.15.43:0/556683785 learned_addr learned my addr 172.21.15.43:0/556683785 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:14.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.817+0000 7fb498b8f640 1 --2- 172.21.15.43:0/556683785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587d3845e80 0x5587d383ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.817+0000 7fb499390640 1 --2- 172.21.15.43:0/556683785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d38454d0 0x5587d385a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.817+0000 7fb499b91640 1 -- 172.21.15.43:0/556683785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587d3845e80 msgr2=0x5587d383ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.817+0000 7fb499b91640 1 --2- 172.21.15.43:0/556683785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587d3845e80 0x5587d383ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.817+0000 7fb499b91640 1 -- 172.21.15.43:0/556683785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d38454d0 msgr2=0x5587d385a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.817+0000 7fb499b91640 1 --2- 172.21.15.43:0/556683785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d38454d0 0x5587d385a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.817+0000 7fb499b91640 1 -- 172.21.15.43:0/556683785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587d384a630 con 0x5587d384ea00 2024-04-21T08:42:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.817+0000 7fb498b8f640 1 --2- 172.21.15.43:0/556683785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587d3845e80 0x5587d383ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.817+0000 7fb499b91640 1 --2- 172.21.15.43:0/556683785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d384ea00 0x5587d384ddb0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb48c009780 tx=0x7fb48c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3345b17bfbf879bc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.817+0000 7fb47effd640 1 -- 172.21.15.43:0/556683785 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c01e970 con 0x5587d384ea00 2024-04-21T08:42:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.817+0000 7fb47effd640 1 -- 172.21.15.43:0/556683785 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb48c02b480 con 0x5587d384ea00 2024-04-21T08:42:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.817+0000 7fb47effd640 1 -- 172.21.15.43:0/556683785 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c020660 con 0x5587d384ea00 2024-04-21T08:42:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.818+0000 7fb49e1e9740 1 -- 172.21.15.43:0/556683785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d384ea00 msgr2=0x5587d384ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.818+0000 7fb49e1e9740 1 --2- 172.21.15.43:0/556683785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d384ea00 0x5587d384ddb0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb48c009780 tx=0x7fb48c004ab0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.818+0000 7fb49e1e9740 1 -- 172.21.15.43:0/556683785 shutdown_connections 2024-04-21T08:42:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.818+0000 7fb49e1e9740 1 --2- 172.21.15.43:0/556683785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d384ea00 0x5587d384ddb0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.818+0000 7fb49e1e9740 1 --2- 172.21.15.43:0/556683785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587d3845e80 0x5587d383ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.818+0000 7fb49e1e9740 1 --2- 172.21.15.43:0/556683785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d38454d0 0x5587d385a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.818+0000 7fb49e1e9740 1 -- 172.21.15.43:0/556683785 >> 172.21.15.43:0/556683785 conn(0x5587d383fba0 msgr2=0x5587d383ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.818+0000 7fb49e1e9740 1 -- 172.21.15.43:0/556683785 shutdown_connections 2024-04-21T08:42:14.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.818+0000 7fb49e1e9740 1 -- 172.21.15.43:0/556683785 wait complete. 2024-04-21T08:42:14.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.818+0000 7fb49e1e9740 1 -- start start 2024-04-21T08:42:14.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.818+0000 7fb49e1e9740 1 Processor -- start 2024-04-21T08:42:14.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.819+0000 7fb49e1e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d38454d0 0x5587d38482b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.819+0000 7fb49e1e9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d3845e80 0x5587d38487f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.819+0000 7fb49e1e9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587d384ea00 0x5587d3848d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.819+0000 7fb49e1e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb4941b2b50 con 0x5587d38454d0 2024-04-21T08:42:14.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.819+0000 7fb49e1e9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb4941b2480 con 0x5587d3845e80 2024-04-21T08:42:14.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.819+0000 7fb49e1e9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb4941af6c0 con 0x5587d384ea00 2024-04-21T08:42:14.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.819+0000 7fb49e1e9740 10 client.0 _ll_register_callbacks cb 0x7fb4840df0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:14.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.819+0000 7fb499390640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d38454d0 0x5587d38482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.819+0000 7fb499390640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d38454d0 0x5587d38482b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36770/0 (socket says 172.21.15.43:36770) 2024-04-21T08:42:14.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.819+0000 7fb499390640 1 -- 172.21.15.43:0/889094870 learned_addr learned my addr 172.21.15.43:0/889094870 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:14.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.819+0000 7fb498b8f640 1 --2- 172.21.15.43:0/889094870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d3845e80 0x5587d38487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.819+0000 7fb499b91640 1 --2- 172.21.15.43:0/889094870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587d384ea00 0x5587d3848d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.819+0000 7fb499390640 1 -- 172.21.15.43:0/889094870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587d384ea00 msgr2=0x5587d3848d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.819+0000 7fb499390640 1 --2- 172.21.15.43:0/889094870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587d384ea00 0x5587d3848d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.819+0000 7fb499390640 1 -- 172.21.15.43:0/889094870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d3845e80 msgr2=0x5587d38487f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.819+0000 7fb499390640 1 --2- 172.21.15.43:0/889094870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d3845e80 0x5587d38487f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.819+0000 7fb499390640 1 -- 172.21.15.43:0/889094870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587d3845c20 con 0x5587d38454d0 2024-04-21T08:42:14.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.819+0000 7fb499b91640 1 --2- 172.21.15.43:0/889094870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587d384ea00 0x5587d3848d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:14.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.819+0000 7fb498b8f640 1 --2- 172.21.15.43:0/889094870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d3845e80 0x5587d38487f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:14.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.819+0000 7fb499390640 1 --2- 172.21.15.43:0/889094870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d38454d0 0x5587d38482b0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fb49000bd10 tx=0x7fb490005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d190ca8e2b849e7f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.819+0000 7fb47cff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:14.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.820+0000 7fb47cff9640 1 -- 172.21.15.43:0/889094870 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49000b7f0 con 0x5587d38454d0 2024-04-21T08:42:14.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.820+0000 7fb47cff9640 1 -- 172.21.15.43:0/889094870 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fb490005250 con 0x5587d38454d0 2024-04-21T08:42:14.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.820+0000 7fb49e1e9740 1 -- 172.21.15.43:0/889094870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb48c02b480 con 0x5587d38454d0 2024-04-21T08:42:14.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.820+0000 7fb47cff9640 1 -- 172.21.15.43:0/889094870 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb490038050 con 0x5587d38454d0 2024-04-21T08:42:14.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.820+0000 7fb49e1e9740 10 client.17151 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:14.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.820+0000 7fb49e1e9740 1 -- 172.21.15.43:0/889094870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5587d38c5250 con 0x5587d38454d0 2024-04-21T08:42:14.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.820+0000 7fb47cff9640 1 -- 172.21.15.43:0/889094870 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb48c02b480 con 0x5587d38454d0 2024-04-21T08:42:14.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.820+0000 7fb47cff9640 1 -- 172.21.15.43:0/889094870 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb4900290f0 con 0x5587d38454d0 2024-04-21T08:42:14.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.820+0000 7fb49e1e9740 10 client.17151 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:14.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.820+0000 7fb49e1e9740 10 client.17151 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:42:14.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.820+0000 7fb49e1e9740 20 client.17151 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:14.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.820+0000 7fb49e1e9740 1 --2- 172.21.15.43:0/889094870 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5587d38c5750 0x5587d38c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.820+0000 7fb49e1e9740 4 client.17151 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:42:14.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.820+0000 7fb49e1e9740 1 -- 172.21.15.43:0/889094870 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5587d37b7c10 con 0x5587d38c5750 2024-04-21T08:42:14.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.820+0000 7fb498b8f640 1 --2- 172.21.15.43:0/889094870 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5587d38c5750 0x5587d38c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.821+0000 7fb498b8f640 1 --2- 172.21.15.43:0/889094870 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5587d38c5750 0x5587d38c7b70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee58f6136ba6e2c2 server_cookie=e5e7a08d7c208f5d in_seq=0 out_seq=0 2024-04-21T08:42:14.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.821+0000 7fb47cff9640 10 client.17151 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:42:14.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.821+0000 7fb47cff9640 1 -- 172.21.15.43:0/889094870 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5587d37b7c10 con 0x5587d38c5750 2024-04-21T08:42:14.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.821+0000 7fb47cff9640 10 client.17151 handle_command_reply: tid=0 2024-04-21T08:42:14.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.821+0000 7fb49e1e9740 1 client.17151 shutdown 2024-04-21T08:42:14.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.821+0000 7fb49e1e9740 1 -- 172.21.15.43:0/889094870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d38454d0 msgr2=0x5587d38482b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.821+0000 7fb49e1e9740 1 --2- 172.21.15.43:0/889094870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d38454d0 0x5587d38482b0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fb49000bd10 tx=0x7fb490005bd0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.821+0000 7fb49e1e9740 1 -- 172.21.15.43:0/889094870 shutdown_connections 2024-04-21T08:42:14.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.821+0000 7fb49e1e9740 1 --2- 172.21.15.43:0/889094870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587d384ea00 0x5587d3848d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.821+0000 7fb49e1e9740 1 --2- 172.21.15.43:0/889094870 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5587d38c5750 0x5587d38c7b70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.821+0000 7fb49e1e9740 1 --2- 172.21.15.43:0/889094870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d3845e80 0x5587d38487f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.821+0000 7fb49e1e9740 1 --2- 172.21.15.43:0/889094870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587d38454d0 0x5587d38482b0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.821+0000 7fb47cff9640 0 client.17151 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:42:14.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.821+0000 7fb49e1e9740 1 -- 172.21.15.43:0/889094870 >> 172.21.15.43:0/889094870 conn(0x5587d383fba0 msgr2=0x5587d3846b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.821+0000 7fb49e1e9740 1 -- 172.21.15.43:0/889094870 shutdown_connections 2024-04-21T08:42:14.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.821+0000 7fb49e1e9740 1 -- 172.21.15.43:0/889094870 wait complete. 2024-04-21T08:42:14.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.821+0000 7fb49e1e9740 20 client.17151 trim_cache size 0 max 16384 2024-04-21T08:42:14.828 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:42:14.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.822+0000 7fb49b5f4640 1 -- 172.21.15.43:0/3759222577 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb4800a3d50 msgr2=0x7fb4800a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.822+0000 7fb49b5f4640 1 --2- 172.21.15.43:0/3759222577 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb4800a3d50 0x7fb4800a6210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb48c005c40 tx=0x7fb48c009c90 comp rx=0 tx=0).stop 2024-04-21T08:42:14.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.822+0000 7fb49b5f4640 1 -- 172.21.15.43:0/3759222577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb494107260 msgr2=0x7fb494074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.822+0000 7fb49b5f4640 1 --2- 172.21.15.43:0/3759222577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb494107260 0x7fb494074790 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb484036040 tx=0x7fb48402c7e0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.822+0000 7fb49b5f4640 1 -- 172.21.15.43:0/3759222577 shutdown_connections 2024-04-21T08:42:14.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.822+0000 7fb49b5f4640 1 --2- 172.21.15.43:0/3759222577 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb4800a3d50 0x7fb4800a6210 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.822+0000 7fb49b5f4640 1 --2- 172.21.15.43:0/3759222577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb494074cd0 0x7fb4941a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.822+0000 7fb49b5f4640 1 --2- 172.21.15.43:0/3759222577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb494107260 0x7fb494074790 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.822+0000 7fb49b5f4640 1 --2- 172.21.15.43:0/3759222577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb494102190 0x7fb494074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.822+0000 7fb49b5f4640 1 -- 172.21.15.43:0/3759222577 >> 172.21.15.43:0/3759222577 conn(0x7fb4940fd990 msgr2=0x7fb4940fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.822+0000 7fb49b5f4640 1 -- 172.21.15.43:0/3759222577 shutdown_connections 2024-04-21T08:42:14.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.822+0000 7fb49b5f4640 1 -- 172.21.15.43:0/3759222577 wait complete. 2024-04-21T08:42:14.836 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:42:14.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.940+0000 7f66d65b2640 1 -- 172.21.15.43:0/3763790072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66d0101aa0 msgr2=0x7f66d0105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.940+0000 7f66d65b2640 1 --2- 172.21.15.43:0/3763790072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66d0101aa0 0x7f66d0105a90 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009900 tx=0x7f66c002bc30 comp rx=0 tx=0).stop 2024-04-21T08:42:14.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.940+0000 7f66d65b2640 1 -- 172.21.15.43:0/3763790072 shutdown_connections 2024-04-21T08:42:14.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.940+0000 7f66d65b2640 1 --2- 172.21.15.43:0/3763790072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66d0106170 0x7f66d010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.940+0000 7f66d65b2640 1 --2- 172.21.15.43:0/3763790072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66d0101aa0 0x7f66d0105a90 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.940+0000 7f66d65b2640 1 --2- 172.21.15.43:0/3763790072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66d01010f0 0x7f66d01014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.940+0000 7f66d65b2640 1 -- 172.21.15.43:0/3763790072 >> 172.21.15.43:0/3763790072 conn(0x7f66d0077fb0 msgr2=0x7f66d00783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.940+0000 7f66d65b2640 1 -- 172.21.15.43:0/3763790072 shutdown_connections 2024-04-21T08:42:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.940+0000 7f66d65b2640 1 -- 172.21.15.43:0/3763790072 wait complete. 2024-04-21T08:42:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.940+0000 7f66d65b2640 1 Processor -- start 2024-04-21T08:42:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.940+0000 7f66d65b2640 1 -- start start 2024-04-21T08:42:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.941+0000 7f66d65b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66d01010f0 0x7f66d019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.941+0000 7f66d65b2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66d0106170 0x7f66d019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.941+0000 7f66d65b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66d01985a0 0x7f66d0198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.941+0000 7f66d65b2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f66d0111090 con 0x7f66d01985a0 2024-04-21T08:42:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.941+0000 7f66d65b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66d0110f10 con 0x7f66d01010f0 2024-04-21T08:42:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.941+0000 7f66d65b2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f66d0111210 con 0x7f66d0106170 2024-04-21T08:42:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.941+0000 7f66d4b4f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66d01985a0 0x7f66d0198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.941+0000 7f66d4b4f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66d01985a0 0x7f66d0198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36800/0 (socket says 172.21.15.43:36800) 2024-04-21T08:42:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.941+0000 7f66d4b4f640 1 -- 172.21.15.43:0/1586539567 learned_addr learned my addr 172.21.15.43:0/1586539567 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.941+0000 7f66cffff640 1 --2- 172.21.15.43:0/1586539567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66d01010f0 0x7f66d019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.941+0000 7f66cf7fe640 1 --2- 172.21.15.43:0/1586539567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66d0106170 0x7f66d019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.941+0000 7f66d4b4f640 1 -- 172.21.15.43:0/1586539567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66d0106170 msgr2=0x7f66d019e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.941+0000 7f66d4b4f640 1 --2- 172.21.15.43:0/1586539567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66d0106170 0x7f66d019e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.941+0000 7f66d4b4f640 1 -- 172.21.15.43:0/1586539567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66d01010f0 msgr2=0x7f66d019e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.941+0000 7f66d4b4f640 1 --2- 172.21.15.43:0/1586539567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66d01010f0 0x7f66d019e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.941+0000 7f66d4b4f640 1 -- 172.21.15.43:0/1586539567 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d01990c0 con 0x7f66d01985a0 2024-04-21T08:42:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.941+0000 7f66cffff640 1 --2- 172.21.15.43:0/1586539567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66d01010f0 0x7f66d019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.941+0000 7f66cf7fe640 1 --2- 172.21.15.43:0/1586539567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66d0106170 0x7f66d019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.941+0000 7f66d4b4f640 1 --2- 172.21.15.43:0/1586539567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66d01985a0 0x7f66d0198a50 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f66c800da40 tx=0x7f66c800df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.941+0000 7f66cd7fa640 1 -- 172.21.15.43:0/1586539567 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8017070 con 0x7f66d01985a0 2024-04-21T08:42:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.941+0000 7f66cd7fa640 1 -- 172.21.15.43:0/1586539567 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66c8003e20 con 0x7f66d01985a0 2024-04-21T08:42:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.941+0000 7f66d65b2640 1 -- 172.21.15.43:0/1586539567 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d01993b0 con 0x7f66d01985a0 2024-04-21T08:42:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.941+0000 7f66cd7fa640 1 -- 172.21.15.43:0/1586539567 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8010460 con 0x7f66d01985a0 2024-04-21T08:42:14.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.942+0000 7f66d65b2640 1 -- 172.21.15.43:0/1586539567 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d01a55b0 con 0x7f66d01985a0 2024-04-21T08:42:14.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.944+0000 7f66cd7fa640 1 -- 172.21.15.43:0/1586539567 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f66c8004020 con 0x7f66d01985a0 2024-04-21T08:42:14.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.944+0000 7f66d91a0740 1 Processor -- start 2024-04-21T08:42:14.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.944+0000 7f66d91a0740 1 -- start start 2024-04-21T08:42:14.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.944+0000 7f66cd7fa640 1 --2- 172.21.15.43:0/1586539567 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f66b80a3d20 0x7f66b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.944+0000 7f66d91a0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b9e1412250 0x55b9e1412630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.944+0000 7f66d91a0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e1412c00 0x55b9e141f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.944+0000 7f66d91a0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9e141fe10 0x55b9e1422220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.944+0000 7f66d91a0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b9e13771f0 con 0x55b9e1412c00 2024-04-21T08:42:14.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.944+0000 7f66d91a0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b9e13144b0 con 0x55b9e141fe10 2024-04-21T08:42:14.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.944+0000 7f66d91a0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b9e13731b0 con 0x55b9e1412250 2024-04-21T08:42:14.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.945+0000 7f66cf7fe640 1 --2- 172.21.15.43:0/1586539567 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f66b80a3d20 0x7f66b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.945+0000 7f66cd7fa640 1 -- 172.21.15.43:0/1586539567 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f66c80d08d0 con 0x7f66d01985a0 2024-04-21T08:42:14.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.945+0000 7f66cf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e1412c00 0x55b9e141f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.945+0000 7f66cf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e1412c00 0x55b9e141f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36804/0 (socket says 172.21.15.43:36804) 2024-04-21T08:42:14.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.945+0000 7f66cf7fe640 1 -- 172.21.15.43:0/2192449555 learned_addr learned my addr 172.21.15.43:0/2192449555 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:14.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.945+0000 7f66cffff640 1 --2- 172.21.15.43:0/2192449555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b9e1412250 0x55b9e1412630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.945+0000 7f66d4b4f640 1 --2- 172.21.15.43:0/2192449555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9e141fe10 0x55b9e1422220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.945+0000 7f66cf7fe640 1 -- 172.21.15.43:0/2192449555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b9e1412250 msgr2=0x55b9e1412630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.945+0000 7f66cf7fe640 1 --2- 172.21.15.43:0/2192449555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b9e1412250 0x55b9e1412630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.945+0000 7f66cf7fe640 1 -- 172.21.15.43:0/2192449555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9e141fe10 msgr2=0x55b9e1422220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.945+0000 7f66cf7fe640 1 --2- 172.21.15.43:0/2192449555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9e141fe10 0x55b9e1422220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.945+0000 7f66cf7fe640 1 -- 172.21.15.43:0/2192449555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9e13061b0 con 0x55b9e1412c00 2024-04-21T08:42:14.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.945+0000 7f66d4b4f640 1 --2- 172.21.15.43:0/2192449555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9e141fe10 0x55b9e1422220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:42:14.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.945+0000 7f66cf7fe640 1 --2- 172.21.15.43:0/1586539567 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f66b80a3d20 0x7f66b80a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f66c0005ec0 tx=0x7f66c0004040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.945+0000 7f66cf7fe640 1 --2- 172.21.15.43:0/2192449555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e1412c00 0x55b9e141f840 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f66d0068e50 tx=0x7f66c0046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f6816ed380b26f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.945+0000 7f66b67fc640 1 -- 172.21.15.43:0/2192449555 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0052070 con 0x55b9e1412c00 2024-04-21T08:42:14.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.945+0000 7f66b67fc640 1 -- 172.21.15.43:0/2192449555 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66c002cac0 con 0x55b9e1412c00 2024-04-21T08:42:14.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.945+0000 7f66b67fc640 1 -- 172.21.15.43:0/2192449555 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c00037f0 con 0x55b9e1412c00 2024-04-21T08:42:14.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.946+0000 7f66d91a0740 1 -- 172.21.15.43:0/2192449555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e1412c00 msgr2=0x55b9e141f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.946+0000 7f66d91a0740 1 --2- 172.21.15.43:0/2192449555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e1412c00 0x55b9e141f840 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f66d0068e50 tx=0x7f66c0046000 comp rx=0 tx=0).stop 2024-04-21T08:42:14.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.946+0000 7f66d91a0740 1 -- 172.21.15.43:0/2192449555 shutdown_connections 2024-04-21T08:42:14.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.946+0000 7f66d91a0740 1 --2- 172.21.15.43:0/2192449555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9e141fe10 0x55b9e1422220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.946+0000 7f66d91a0740 1 --2- 172.21.15.43:0/2192449555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e1412c00 0x55b9e141f840 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.946+0000 7f66d91a0740 1 --2- 172.21.15.43:0/2192449555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b9e1412250 0x55b9e1412630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.946+0000 7f66d91a0740 1 -- 172.21.15.43:0/2192449555 >> 172.21.15.43:0/2192449555 conn(0x55b9e140cba0 msgr2=0x55b9e140cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.946+0000 7f66d91a0740 1 -- 172.21.15.43:0/2192449555 shutdown_connections 2024-04-21T08:42:14.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.946+0000 7f66d91a0740 1 -- 172.21.15.43:0/2192449555 wait complete. 2024-04-21T08:42:14.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.946+0000 7f66d91a0740 1 -- start start 2024-04-21T08:42:14.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.947+0000 7f66d91a0740 1 Processor -- start 2024-04-21T08:42:14.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.947+0000 7f66d91a0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9e1412250 0x55b9e141f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.947+0000 7f66d91a0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b9e1412c00 0x55b9e141cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.947+0000 7f66d91a0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e141fe10 0x55b9e141d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.947+0000 7f66d91a0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b9e1422ce0 con 0x55b9e141fe10 2024-04-21T08:42:14.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.947+0000 7f66d91a0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b9e1330500 con 0x55b9e1412250 2024-04-21T08:42:14.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.947+0000 7f66d91a0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b9e1422e60 con 0x55b9e1412c00 2024-04-21T08:42:14.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.947+0000 7f66d91a0740 10 client.0 _ll_register_callbacks cb 0x55b9e12feb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:14.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.947+0000 7f66d4b4f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e141fe10 0x55b9e141d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.947+0000 7f66d4b4f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e141fe10 0x55b9e141d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36818/0 (socket says 172.21.15.43:36818) 2024-04-21T08:42:14.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.947+0000 7f66d4b4f640 1 -- 172.21.15.43:0/242440104 learned_addr learned my addr 172.21.15.43:0/242440104 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:14.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.947+0000 7f66cf7fe640 1 --2- 172.21.15.43:0/242440104 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b9e1412c00 0x55b9e141cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.947+0000 7f66cffff640 1 --2- 172.21.15.43:0/242440104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9e1412250 0x55b9e141f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.947+0000 7f66d4b4f640 1 -- 172.21.15.43:0/242440104 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b9e1412c00 msgr2=0x55b9e141cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.947+0000 7f66d4b4f640 1 --2- 172.21.15.43:0/242440104 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b9e1412c00 0x55b9e141cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.947+0000 7f66d4b4f640 1 -- 172.21.15.43:0/242440104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9e1412250 msgr2=0x55b9e141f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.947+0000 7f66d4b4f640 1 --2- 172.21.15.43:0/242440104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9e1412250 0x55b9e141f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.947+0000 7f66d4b4f640 1 -- 172.21.15.43:0/242440104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9e10893f0 con 0x55b9e141fe10 2024-04-21T08:42:14.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.947+0000 7f66d4b4f640 1 --2- 172.21.15.43:0/242440104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e141fe10 0x55b9e141d0f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f66c80ce250 tx=0x7f66c809bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e0851bdf0530e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.947+0000 7f66af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.948+0000 7f66af7fe640 1 -- 172.21.15.43:0/242440104 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8020050 con 0x55b9e141fe10 2024-04-21T08:42:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.948+0000 7f66af7fe640 1 -- 172.21.15.43:0/242440104 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f66c80cded0 con 0x55b9e141fe10 2024-04-21T08:42:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.948+0000 7f66d91a0740 1 -- 172.21.15.43:0/242440104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66c002cac0 con 0x55b9e141fe10 2024-04-21T08:42:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.948+0000 7f66af7fe640 1 -- 172.21.15.43:0/242440104 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f66c809cba0 con 0x55b9e141fe10 2024-04-21T08:42:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.948+0000 7f66d91a0740 10 client.17163 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.948+0000 7f66d91a0740 1 -- 172.21.15.43:0/242440104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b9e11537d0 con 0x55b9e141fe10 2024-04-21T08:42:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.948+0000 7f66af7fe640 1 -- 172.21.15.43:0/242440104 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66c002cac0 con 0x55b9e141fe10 2024-04-21T08:42:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.948+0000 7f66af7fe640 1 -- 172.21.15.43:0/242440104 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f66c8098b10 con 0x55b9e141fe10 2024-04-21T08:42:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.948+0000 7f66d91a0740 10 client.17163 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.948+0000 7f66d91a0740 10 client.17163 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:42:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.948+0000 7f66d91a0740 20 client.17163 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.948+0000 7f66d91a0740 1 --2- 172.21.15.43:0/242440104 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55b9e142ff30 0x55b9e1432360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.948+0000 7f66d91a0740 4 client.17163 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.948+0000 7f66d91a0740 1 -- 172.21.15.43:0/242440104 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9e1316930 con 0x55b9e142ff30 2024-04-21T08:42:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.948+0000 7f66cffff640 1 --2- 172.21.15.43:0/242440104 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55b9e142ff30 0x55b9e1432360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.949+0000 7f66af7fe640 10 client.17163 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:42:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.949+0000 7f66cffff640 1 --2- 172.21.15.43:0/242440104 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55b9e142ff30 0x55b9e1432360 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=884e7b820fe590f0 server_cookie=1d7e148d8af6faf4 in_seq=0 out_seq=0 2024-04-21T08:42:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.949+0000 7f66af7fe640 1 -- 172.21.15.43:0/242440104 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b9e1316930 con 0x55b9e142ff30 2024-04-21T08:42:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.949+0000 7f66af7fe640 10 client.17163 handle_command_reply: tid=0 2024-04-21T08:42:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.949+0000 7f66d91a0740 1 client.17163 shutdown 2024-04-21T08:42:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.950+0000 7f66d91a0740 1 -- 172.21.15.43:0/242440104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e141fe10 msgr2=0x55b9e141d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.950+0000 7f66d91a0740 1 --2- 172.21.15.43:0/242440104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e141fe10 0x55b9e141d0f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f66c80ce250 tx=0x7f66c809bec0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.950+0000 7f66d91a0740 1 -- 172.21.15.43:0/242440104 shutdown_connections 2024-04-21T08:42:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.950+0000 7f66d91a0740 1 --2- 172.21.15.43:0/242440104 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55b9e142ff30 0x55b9e1432360 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.950+0000 7f66d91a0740 1 --2- 172.21.15.43:0/242440104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e141fe10 0x55b9e141d0f0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.950+0000 7f66d91a0740 1 --2- 172.21.15.43:0/242440104 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b9e1412c00 0x55b9e141cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.950+0000 7f66d91a0740 1 --2- 172.21.15.43:0/242440104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9e1412250 0x55b9e141f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.950+0000 7f66d91a0740 1 -- 172.21.15.43:0/242440104 >> 172.21.15.43:0/242440104 conn(0x55b9e140cba0 msgr2=0x55b9e140e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.950+0000 7f66af7fe640 0 client.17163 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:42:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.950+0000 7f66d91a0740 1 -- 172.21.15.43:0/242440104 shutdown_connections 2024-04-21T08:42:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.950+0000 7f66d91a0740 1 -- 172.21.15.43:0/242440104 wait complete. 2024-04-21T08:42:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.950+0000 7f66d91a0740 20 client.17163 trim_cache size 0 max 16384 2024-04-21T08:42:14.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.963+0000 7f66d91a0740 1 Processor -- start 2024-04-21T08:42:14.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.963+0000 7f66d91a0740 1 -- start start 2024-04-21T08:42:14.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.963+0000 7f66d91a0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b9e1412250 0x55b9e142bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.963+0000 7f66d91a0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e1412c00 0x55b9e141f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.963+0000 7f66d91a0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9e141fe10 0x55b9e141f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.963+0000 7f66d91a0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f66d01ad240 con 0x55b9e1412c00 2024-04-21T08:42:14.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.963+0000 7f66d91a0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66d01aed80 con 0x55b9e141fe10 2024-04-21T08:42:14.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.963+0000 7f66d91a0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f66d01ad910 con 0x55b9e1412250 2024-04-21T08:42:14.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.964+0000 7f66cf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e1412c00 0x55b9e141f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.964+0000 7f66cf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e1412c00 0x55b9e141f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36828/0 (socket says 172.21.15.43:36828) 2024-04-21T08:42:14.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.964+0000 7f66cf7fe640 1 -- 172.21.15.43:0/2352502767 learned_addr learned my addr 172.21.15.43:0/2352502767 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:14.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.964+0000 7f66cf7fe640 1 -- 172.21.15.43:0/2352502767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b9e1412250 msgr2=0x55b9e142bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.964+0000 7f66cffff640 1 --2- 172.21.15.43:0/2352502767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b9e1412250 0x55b9e142bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.964+0000 7f66cf7fe640 1 --2- 172.21.15.43:0/2352502767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b9e1412250 0x55b9e142bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.964+0000 7f66d4b4f640 1 --2- 172.21.15.43:0/2352502767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9e141fe10 0x55b9e141f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.964+0000 7f66cf7fe640 1 -- 172.21.15.43:0/2352502767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9e141fe10 msgr2=0x55b9e141f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.964+0000 7f66cf7fe640 1 --2- 172.21.15.43:0/2352502767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9e141fe10 0x55b9e141f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.964+0000 7f66cf7fe640 1 -- 172.21.15.43:0/2352502767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9e1411560 con 0x55b9e1412c00 2024-04-21T08:42:14.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.964+0000 7f66cffff640 1 --2- 172.21.15.43:0/2352502767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b9e1412250 0x55b9e142bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:14.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.964+0000 7f66cf7fe640 1 --2- 172.21.15.43:0/2352502767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e1412c00 0x55b9e141f210 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f66c0050040 tx=0x7f66c00045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994f92e8092f97dd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.964+0000 7f66b5ffb640 1 -- 172.21.15.43:0/2352502767 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0052040 con 0x55b9e1412c00 2024-04-21T08:42:14.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.964+0000 7f66b5ffb640 1 -- 172.21.15.43:0/2352502767 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66c0053040 con 0x55b9e1412c00 2024-04-21T08:42:14.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.964+0000 7f66b5ffb640 1 -- 172.21.15.43:0/2352502767 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0004db0 con 0x55b9e1412c00 2024-04-21T08:42:14.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.965+0000 7f66d91a0740 1 -- 172.21.15.43:0/2352502767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e1412c00 msgr2=0x55b9e141f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.965+0000 7f66d91a0740 1 --2- 172.21.15.43:0/2352502767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e1412c00 0x55b9e141f210 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f66c0050040 tx=0x7f66c00045b0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.965+0000 7f66d91a0740 1 -- 172.21.15.43:0/2352502767 shutdown_connections 2024-04-21T08:42:14.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.965+0000 7f66d91a0740 1 --2- 172.21.15.43:0/2352502767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9e141fe10 0x55b9e141f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.965+0000 7f66d91a0740 1 --2- 172.21.15.43:0/2352502767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e1412c00 0x55b9e141f210 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.965+0000 7f66d91a0740 1 --2- 172.21.15.43:0/2352502767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b9e1412250 0x55b9e142bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.965+0000 7f66d91a0740 1 -- 172.21.15.43:0/2352502767 >> 172.21.15.43:0/2352502767 conn(0x55b9e140cba0 msgr2=0x55b9e140bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.965+0000 7f66d91a0740 1 -- 172.21.15.43:0/2352502767 shutdown_connections 2024-04-21T08:42:14.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.965+0000 7f66d91a0740 1 -- 172.21.15.43:0/2352502767 wait complete. 2024-04-21T08:42:14.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.965+0000 7f66d91a0740 1 -- start start 2024-04-21T08:42:14.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.965+0000 7f66d91a0740 1 Processor -- start 2024-04-21T08:42:14.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.966+0000 7f66d91a0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e1412250 0x55b9e1428640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.966+0000 7f66d91a0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9e1412c00 0x55b9e14195f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.966+0000 7f66d91a0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b9e141fe10 0x55b9e1419d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.966+0000 7f66d91a0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f66d01ae6b0 con 0x55b9e1412250 2024-04-21T08:42:14.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.966+0000 7f66d91a0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66d01adfe0 con 0x55b9e1412c00 2024-04-21T08:42:14.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.966+0000 7f66d91a0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f66d01ab220 con 0x55b9e141fe10 2024-04-21T08:42:14.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.966+0000 7f66d91a0740 10 client.0 _ll_register_callbacks cb 0x7f66c809bfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:14.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.966+0000 7f66cffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e1412250 0x55b9e1428640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.966+0000 7f66cffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e1412250 0x55b9e1428640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36832/0 (socket says 172.21.15.43:36832) 2024-04-21T08:42:14.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.966+0000 7f66cffff640 1 -- 172.21.15.43:0/4154317704 learned_addr learned my addr 172.21.15.43:0/4154317704 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:14.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.966+0000 7f66d4b4f640 1 --2- 172.21.15.43:0/4154317704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b9e141fe10 0x55b9e1419d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.966+0000 7f66cf7fe640 1 --2- 172.21.15.43:0/4154317704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9e1412c00 0x55b9e14195f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.966+0000 7f66cffff640 1 -- 172.21.15.43:0/4154317704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b9e141fe10 msgr2=0x55b9e1419d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.966+0000 7f66cffff640 1 --2- 172.21.15.43:0/4154317704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b9e141fe10 0x55b9e1419d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.966+0000 7f66cffff640 1 -- 172.21.15.43:0/4154317704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9e1412c00 msgr2=0x55b9e14195f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:14.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.966+0000 7f66cffff640 1 --2- 172.21.15.43:0/4154317704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9e1412c00 0x55b9e14195f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.966+0000 7f66cffff640 1 -- 172.21.15.43:0/4154317704 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9e140dd40 con 0x55b9e1412250 2024-04-21T08:42:14.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.966+0000 7f66cffff640 1 --2- 172.21.15.43:0/4154317704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e1412250 0x55b9e1428640 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f66c4000c00 tx=0x7f66c4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42a61fac466898d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:14.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.966+0000 7f66af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:14.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.966+0000 7f66af7fe640 1 -- 172.21.15.43:0/4154317704 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c4023060 con 0x55b9e1412250 2024-04-21T08:42:14.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.966+0000 7f66af7fe640 1 -- 172.21.15.43:0/4154317704 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f66c401d430 con 0x55b9e1412250 2024-04-21T08:42:14.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.966+0000 7f66d91a0740 1 -- 172.21.15.43:0/4154317704 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66c0053040 con 0x55b9e1412250 2024-04-21T08:42:14.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.967+0000 7f66af7fe640 1 -- 172.21.15.43:0/4154317704 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f66c402a050 con 0x55b9e1412250 2024-04-21T08:42:14.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.967+0000 7f66d91a0740 10 client.17169 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:14.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.967+0000 7f66d91a0740 1 -- 172.21.15.43:0/4154317704 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b9e14963e0 con 0x55b9e1412250 2024-04-21T08:42:14.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.967+0000 7f66af7fe640 1 -- 172.21.15.43:0/4154317704 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66c0053040 con 0x55b9e1412250 2024-04-21T08:42:14.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.967+0000 7f66af7fe640 1 -- 172.21.15.43:0/4154317704 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f66c40104f0 con 0x55b9e1412250 2024-04-21T08:42:14.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.967+0000 7f66d91a0740 10 client.17169 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:14.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.967+0000 7f66d91a0740 10 client.17169 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:42:14.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.967+0000 7f66d91a0740 20 client.17169 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:14.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.967+0000 7f66d91a0740 1 --2- 172.21.15.43:0/4154317704 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55b9e1496b10 0x55b9e1498f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:14.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.967+0000 7f66d91a0740 4 client.17169 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:42:14.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.967+0000 7f66d91a0740 1 -- 172.21.15.43:0/4154317704 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b9e1384c10 con 0x55b9e1496b10 2024-04-21T08:42:14.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.967+0000 7f66cf7fe640 1 --2- 172.21.15.43:0/4154317704 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55b9e1496b10 0x55b9e1498f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:14.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.968+0000 7f66af7fe640 10 client.17169 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:42:14.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.968+0000 7f66cf7fe640 1 --2- 172.21.15.43:0/4154317704 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55b9e1496b10 0x55b9e1498f30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dadb582e018a7847 server_cookie=edd58c4306849a76 in_seq=0 out_seq=0 2024-04-21T08:42:14.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.968+0000 7f66af7fe640 1 -- 172.21.15.43:0/4154317704 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b9e1384c10 con 0x55b9e1496b10 2024-04-21T08:42:14.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.968+0000 7f66af7fe640 10 client.17169 handle_command_reply: tid=0 2024-04-21T08:42:14.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.968+0000 7f66d91a0740 1 client.17169 shutdown 2024-04-21T08:42:14.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.968+0000 7f66d91a0740 1 -- 172.21.15.43:0/4154317704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e1412250 msgr2=0x55b9e1428640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.968+0000 7f66d91a0740 1 --2- 172.21.15.43:0/4154317704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e1412250 0x55b9e1428640 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f66c4000c00 tx=0x7f66c4011b00 comp rx=0 tx=0).stop 2024-04-21T08:42:14.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.968+0000 7f66d91a0740 1 -- 172.21.15.43:0/4154317704 shutdown_connections 2024-04-21T08:42:14.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.968+0000 7f66d91a0740 1 --2- 172.21.15.43:0/4154317704 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55b9e1496b10 0x55b9e1498f30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.968+0000 7f66d91a0740 1 --2- 172.21.15.43:0/4154317704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b9e141fe10 0x55b9e1419d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.968+0000 7f66af7fe640 0 client.17169 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:42:14.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.968+0000 7f66d91a0740 1 --2- 172.21.15.43:0/4154317704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b9e1412c00 0x55b9e14195f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.968+0000 7f66d91a0740 1 --2- 172.21.15.43:0/4154317704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9e1412250 0x55b9e1428640 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.968+0000 7f66d91a0740 1 -- 172.21.15.43:0/4154317704 >> 172.21.15.43:0/4154317704 conn(0x55b9e140cba0 msgr2=0x55b9e140df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.968+0000 7f66d91a0740 1 -- 172.21.15.43:0/4154317704 shutdown_connections 2024-04-21T08:42:14.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.968+0000 7f66d91a0740 1 -- 172.21.15.43:0/4154317704 wait complete. 2024-04-21T08:42:14.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.968+0000 7f66d91a0740 20 client.17169 trim_cache size 0 max 16384 2024-04-21T08:42:14.975 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:42:14.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.969+0000 7f66d65b2640 1 -- 172.21.15.43:0/1586539567 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f66b80a3d20 msgr2=0x7f66b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.969+0000 7f66d65b2640 1 --2- 172.21.15.43:0/1586539567 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f66b80a3d20 0x7f66b80a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f66c0005ec0 tx=0x7f66c0004040 comp rx=0 tx=0).stop 2024-04-21T08:42:14.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.969+0000 7f66d65b2640 1 -- 172.21.15.43:0/1586539567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66d01985a0 msgr2=0x7f66d0198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:14.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.969+0000 7f66d65b2640 1 --2- 172.21.15.43:0/1586539567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66d01985a0 0x7f66d0198a50 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f66c800da40 tx=0x7f66c800df00 comp rx=0 tx=0).stop 2024-04-21T08:42:14.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.969+0000 7f66d65b2640 1 -- 172.21.15.43:0/1586539567 shutdown_connections 2024-04-21T08:42:14.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.969+0000 7f66d65b2640 1 --2- 172.21.15.43:0/1586539567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66d01985a0 0x7f66d0198a50 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.969+0000 7f66d65b2640 1 --2- 172.21.15.43:0/1586539567 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f66b80a3d20 0x7f66b80a61e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.969+0000 7f66d65b2640 1 --2- 172.21.15.43:0/1586539567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66d0106170 0x7f66d019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.969+0000 7f66d65b2640 1 --2- 172.21.15.43:0/1586539567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66d01010f0 0x7f66d019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:14.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.969+0000 7f66d65b2640 1 -- 172.21.15.43:0/1586539567 >> 172.21.15.43:0/1586539567 conn(0x7f66d0077fb0 msgr2=0x7f66d006a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:14.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.969+0000 7f66d65b2640 1 -- 172.21.15.43:0/1586539567 shutdown_connections 2024-04-21T08:42:14.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:14.969+0000 7f66d65b2640 1 -- 172.21.15.43:0/1586539567 wait complete. 2024-04-21T08:42:14.980 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:42:15.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.084+0000 7f29bca06640 1 -- 172.21.15.43:0/322336519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29b8102b40 msgr2=0x7f29b8106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:15.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.084+0000 7f29bca06640 1 --2- 172.21.15.43:0/322336519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29b8102b40 0x7f29b8106b80 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f29ac009900 tx=0x7f29ac02bc30 comp rx=0 tx=0).stop 2024-04-21T08:42:15.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.084+0000 7f29bca06640 1 -- 172.21.15.43:0/322336519 shutdown_connections 2024-04-21T08:42:15.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.084+0000 7f29bca06640 1 --2- 172.21.15.43:0/322336519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b8107260 0x7f29b810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.084+0000 7f29bca06640 1 --2- 172.21.15.43:0/322336519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29b8102b40 0x7f29b8106b80 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.084+0000 7f29bca06640 1 --2- 172.21.15.43:0/322336519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29b8102190 0x7f29b8102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.084+0000 7f29bca06640 1 -- 172.21.15.43:0/322336519 >> 172.21.15.43:0/322336519 conn(0x7f29b80fd990 msgr2=0x7f29b80ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:15.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.085+0000 7f29bca06640 1 -- 172.21.15.43:0/322336519 shutdown_connections 2024-04-21T08:42:15.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.085+0000 7f29bca06640 1 -- 172.21.15.43:0/322336519 wait complete. 2024-04-21T08:42:15.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.085+0000 7f29bca06640 1 Processor -- start 2024-04-21T08:42:15.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.085+0000 7f29bca06640 1 -- start start 2024-04-21T08:42:15.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.085+0000 7f29bca06640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b8102190 0x7f29b8074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:15.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.086+0000 7f29bca06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29b8107260 0x7f29b8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:15.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.086+0000 7f29bca06640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29b8074cd0 0x7f29b81a6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:15.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.086+0000 7f29bca06640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f29b81121a0 con 0x7f29b8107260 2024-04-21T08:42:15.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.086+0000 7f29bca06640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f29b8112020 con 0x7f29b8102190 2024-04-21T08:42:15.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.086+0000 7f29bca06640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29b8112320 con 0x7f29b8074cd0 2024-04-21T08:42:15.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.086+0000 7f29b5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29b8107260 0x7f29b8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:15.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.086+0000 7f29b5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29b8107260 0x7f29b8074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36844/0 (socket says 172.21.15.43:36844) 2024-04-21T08:42:15.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.086+0000 7f29b5d9b640 1 -- 172.21.15.43:0/1537052705 learned_addr learned my addr 172.21.15.43:0/1537052705 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:15.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.086+0000 7f29b659c640 1 --2- 172.21.15.43:0/1537052705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b8102190 0x7f29b8074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:15.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.086+0000 7f29b5d9b640 1 -- 172.21.15.43:0/1537052705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29b8074cd0 msgr2=0x7f29b81a6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:15.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.086+0000 7f29b5d9b640 1 --2- 172.21.15.43:0/1537052705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29b8074cd0 0x7f29b81a6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.086+0000 7f29b5d9b640 1 -- 172.21.15.43:0/1537052705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b8102190 msgr2=0x7f29b8074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:15.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.086+0000 7f29b5d9b640 1 --2- 172.21.15.43:0/1537052705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b8102190 0x7f29b8074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.086+0000 7f29b5d9b640 1 -- 172.21.15.43:0/1537052705 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b81a7060 con 0x7f29b8107260 2024-04-21T08:42:15.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.086+0000 7f29b659c640 1 --2- 172.21.15.43:0/1537052705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b8102190 0x7f29b8074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:15.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.086+0000 7f29b5d9b640 1 --2- 172.21.15.43:0/1537052705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29b8107260 0x7f29b8074790 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f29ac036040 tx=0x7f29ac02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:15.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.086+0000 7f29a77fe640 1 -- 172.21.15.43:0/1537052705 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac043020 con 0x7f29b8107260 2024-04-21T08:42:15.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.086+0000 7f29bca06640 1 -- 172.21.15.43:0/1537052705 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b81a72f0 con 0x7f29b8107260 2024-04-21T08:42:15.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.086+0000 7f29a77fe640 1 -- 172.21.15.43:0/1537052705 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29ac02cbe0 con 0x7f29b8107260 2024-04-21T08:42:15.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.086+0000 7f29a77fe640 1 -- 172.21.15.43:0/1537052705 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac003bf0 con 0x7f29b8107260 2024-04-21T08:42:15.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.086+0000 7f29bca06640 1 -- 172.21.15.43:0/1537052705 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b81a7890 con 0x7f29b8107260 2024-04-21T08:42:15.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.089+0000 7f29a77fe640 1 -- 172.21.15.43:0/1537052705 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f29ac046020 con 0x7f29b8107260 2024-04-21T08:42:15.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.089+0000 7f29bf5f5740 1 Processor -- start 2024-04-21T08:42:15.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.089+0000 7f29bf5f5740 1 -- start start 2024-04-21T08:42:15.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.089+0000 7f29bf5f5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dbf95bf4d0 0x55dbf95bf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:15.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.089+0000 7f29bf5f5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dbf95bfe80 0x55dbf95c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:15.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.090+0000 7f29bf5f5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95c8a00 0x55dbf95cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:15.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.090+0000 7f29bf5f5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dbf95241f0 con 0x55dbf95c8a00 2024-04-21T08:42:15.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.090+0000 7f29bf5f5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dbf94c14b0 con 0x55dbf95bfe80 2024-04-21T08:42:15.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.090+0000 7f29bf5f5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dbf95201b0 con 0x55dbf95bf4d0 2024-04-21T08:42:15.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.090+0000 7f29a77fe640 1 --2- 172.21.15.43:0/1537052705 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f29a00a3d50 0x7f29a00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:15.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.090+0000 7f29b5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95c8a00 0x55dbf95cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:15.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.090+0000 7f29b5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95c8a00 0x55dbf95cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36856/0 (socket says 172.21.15.43:36856) 2024-04-21T08:42:15.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.090+0000 7f29b5d9b640 1 -- 172.21.15.43:0/3463408397 learned_addr learned my addr 172.21.15.43:0/3463408397 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:15.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.090+0000 7f29b6d9d640 1 --2- 172.21.15.43:0/3463408397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dbf95bf4d0 0x55dbf95bf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:15.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.090+0000 7f29b659c640 1 --2- 172.21.15.43:0/3463408397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dbf95bfe80 0x55dbf95c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:15.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.090+0000 7f29b6d9d640 1 --2- 172.21.15.43:0/1537052705 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f29a00a3d50 0x7f29a00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:15.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.090+0000 7f29a77fe640 1 -- 172.21.15.43:0/1537052705 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f29ac0f2d70 con 0x7f29b8107260 2024-04-21T08:42:15.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.090+0000 7f29b5d9b640 1 -- 172.21.15.43:0/3463408397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dbf95bf4d0 msgr2=0x55dbf95bf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:15.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.090+0000 7f29b5d9b640 1 --2- 172.21.15.43:0/3463408397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dbf95bf4d0 0x55dbf95bf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.090+0000 7f29b5d9b640 1 -- 172.21.15.43:0/3463408397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dbf95bfe80 msgr2=0x55dbf95c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:15.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.090+0000 7f29b5d9b640 1 --2- 172.21.15.43:0/3463408397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dbf95bfe80 0x55dbf95c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.090+0000 7f29b5d9b640 1 -- 172.21.15.43:0/3463408397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf94b31b0 con 0x55dbf95c8a00 2024-04-21T08:42:15.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.090+0000 7f29b659c640 1 --2- 172.21.15.43:0/3463408397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dbf95bfe80 0x55dbf95c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:42:15.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.090+0000 7f29b5d9b640 1 --2- 172.21.15.43:0/3463408397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95c8a00 0x55dbf95cae50 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f29ac005ec0 tx=0x7f29ac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f70cd3b8fc46a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:15.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.090+0000 7f29b6d9d640 1 --2- 172.21.15.43:0/1537052705 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f29a00a3d50 0x7f29a00a6210 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f29b0005fd0 tx=0x7f29b00094b0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:15.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.090+0000 7f29a4ff9640 1 -- 172.21.15.43:0/3463408397 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac0be590 con 0x55dbf95c8a00 2024-04-21T08:42:15.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.091+0000 7f29a4ff9640 1 -- 172.21.15.43:0/3463408397 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29ac0bf150 con 0x55dbf95c8a00 2024-04-21T08:42:15.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.091+0000 7f29a4ff9640 1 -- 172.21.15.43:0/3463408397 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac0be3b0 con 0x55dbf95c8a00 2024-04-21T08:42:15.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.091+0000 7f29bf5f5740 1 -- 172.21.15.43:0/3463408397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95c8a00 msgr2=0x55dbf95cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:15.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.091+0000 7f29bf5f5740 1 --2- 172.21.15.43:0/3463408397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95c8a00 0x55dbf95cae50 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f29ac005ec0 tx=0x7f29ac000bc0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.091+0000 7f29bf5f5740 1 -- 172.21.15.43:0/3463408397 shutdown_connections 2024-04-21T08:42:15.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.091+0000 7f29bf5f5740 1 --2- 172.21.15.43:0/3463408397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95c8a00 0x55dbf95cae50 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.091+0000 7f29bf5f5740 1 --2- 172.21.15.43:0/3463408397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dbf95bfe80 0x55dbf95c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.091+0000 7f29bf5f5740 1 --2- 172.21.15.43:0/3463408397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dbf95bf4d0 0x55dbf95bf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.091+0000 7f29bf5f5740 1 -- 172.21.15.43:0/3463408397 >> 172.21.15.43:0/3463408397 conn(0x55dbf95b9ba0 msgr2=0x55dbf95b9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:15.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.091+0000 7f29bf5f5740 1 -- 172.21.15.43:0/3463408397 shutdown_connections 2024-04-21T08:42:15.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.091+0000 7f29bf5f5740 1 -- 172.21.15.43:0/3463408397 wait complete. 2024-04-21T08:42:15.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.092+0000 7f29bf5f5740 1 -- start start 2024-04-21T08:42:15.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.092+0000 7f29bf5f5740 1 Processor -- start 2024-04-21T08:42:15.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.092+0000 7f29bf5f5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dbf95bf4d0 0x55dbf95c8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:15.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.092+0000 7f29bf5f5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dbf95bfe80 0x55dbf95c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:15.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.092+0000 7f29bf5f5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95c8a00 0x55dbf95c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:15.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.092+0000 7f29bf5f5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dbf95cb730 con 0x55dbf95c8a00 2024-04-21T08:42:15.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.092+0000 7f29bf5f5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dbf94dd500 con 0x55dbf95bfe80 2024-04-21T08:42:15.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.092+0000 7f29bf5f5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dbf95cb8b0 con 0x55dbf95bf4d0 2024-04-21T08:42:15.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.092+0000 7f29bf5f5740 10 client.0 _ll_register_callbacks cb 0x55dbf94abb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:15.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.092+0000 7f29b6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95c8a00 0x55dbf95c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:15.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.092+0000 7f29b6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95c8a00 0x55dbf95c5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36860/0 (socket says 172.21.15.43:36860) 2024-04-21T08:42:15.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.092+0000 7f29b6d9d640 1 -- 172.21.15.43:0/2524417782 learned_addr learned my addr 172.21.15.43:0/2524417782 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:15.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.092+0000 7f29b5d9b640 1 --2- 172.21.15.43:0/2524417782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dbf95bfe80 0x55dbf95c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:15.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.092+0000 7f29b659c640 1 --2- 172.21.15.43:0/2524417782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dbf95bf4d0 0x55dbf95c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:15.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.092+0000 7f29b6d9d640 1 -- 172.21.15.43:0/2524417782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dbf95bf4d0 msgr2=0x55dbf95c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:15.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.092+0000 7f29b6d9d640 1 --2- 172.21.15.43:0/2524417782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dbf95bf4d0 0x55dbf95c8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.092+0000 7f29b6d9d640 1 -- 172.21.15.43:0/2524417782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dbf95bfe80 msgr2=0x55dbf95c56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:15.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.092+0000 7f29b6d9d640 1 --2- 172.21.15.43:0/2524417782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dbf95bfe80 0x55dbf95c56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.092+0000 7f29b6d9d640 1 -- 172.21.15.43:0/2524417782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf92363f0 con 0x55dbf95c8a00 2024-04-21T08:42:15.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.093+0000 7f29b6d9d640 1 --2- 172.21.15.43:0/2524417782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95c8a00 0x55dbf95c5c40 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f29b0003ae0 tx=0x7f29b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317b8ac085baa8b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:15.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.093+0000 7f299dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:15.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.093+0000 7f299dffb640 1 -- 172.21.15.43:0/2524417782 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b0023030 con 0x55dbf95c8a00 2024-04-21T08:42:15.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.093+0000 7f299dffb640 1 -- 172.21.15.43:0/2524417782 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f29b0004240 con 0x55dbf95c8a00 2024-04-21T08:42:15.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.093+0000 7f29bf5f5740 1 -- 172.21.15.43:0/2524417782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29ac0bf150 con 0x55dbf95c8a00 2024-04-21T08:42:15.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.093+0000 7f299dffb640 1 -- 172.21.15.43:0/2524417782 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f29b0030050 con 0x55dbf95c8a00 2024-04-21T08:42:15.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.093+0000 7f29bf5f5740 10 client.17181 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:15.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.093+0000 7f29bf5f5740 1 -- 172.21.15.43:0/2524417782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dbf93007d0 con 0x55dbf95c8a00 2024-04-21T08:42:15.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.093+0000 7f299dffb640 1 -- 172.21.15.43:0/2524417782 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29ac0bf150 con 0x55dbf95c8a00 2024-04-21T08:42:15.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.093+0000 7f299dffb640 1 -- 172.21.15.43:0/2524417782 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f29b001fac0 con 0x55dbf95c8a00 2024-04-21T08:42:15.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.093+0000 7f29bf5f5740 10 client.17181 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:15.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.093+0000 7f29bf5f5740 10 client.17181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:15.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.093+0000 7f29bf5f5740 20 client.17181 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:15.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.093+0000 7f29bf5f5740 1 --2- 172.21.15.43:0/2524417782 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55dbf95d88e0 0x55dbf95dad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:15.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.093+0000 7f29bf5f5740 4 client.17181 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:15.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.093+0000 7f29bf5f5740 1 -- 172.21.15.43:0/2524417782 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbf94c3930 con 0x55dbf95d88e0 2024-04-21T08:42:15.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.094+0000 7f29b659c640 1 --2- 172.21.15.43:0/2524417782 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55dbf95d88e0 0x55dbf95dad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:15.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.094+0000 7f29b659c640 1 --2- 172.21.15.43:0/2524417782 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55dbf95d88e0 0x55dbf95dad10 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40c97a6053ddc8cd server_cookie=cbce136ddd864957 in_seq=0 out_seq=0 2024-04-21T08:42:15.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.094+0000 7f299dffb640 10 client.17181 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:15.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.095+0000 7f299dffb640 1 -- 172.21.15.43:0/2524417782 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dbf94c3930 con 0x55dbf95d88e0 2024-04-21T08:42:15.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.095+0000 7f299dffb640 10 client.17181 handle_command_reply: tid=0 2024-04-21T08:42:15.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.095+0000 7f29bf5f5740 1 client.17181 shutdown 2024-04-21T08:42:15.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.095+0000 7f29bf5f5740 1 -- 172.21.15.43:0/2524417782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95c8a00 msgr2=0x55dbf95c5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.095+0000 7f29bf5f5740 1 --2- 172.21.15.43:0/2524417782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95c8a00 0x55dbf95c5c40 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f29b0003ae0 tx=0x7f29b0004a40 comp rx=0 tx=0).stop 2024-04-21T08:42:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.095+0000 7f29bf5f5740 1 -- 172.21.15.43:0/2524417782 shutdown_connections 2024-04-21T08:42:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.095+0000 7f29bf5f5740 1 --2- 172.21.15.43:0/2524417782 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55dbf95d88e0 0x55dbf95dad10 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.095+0000 7f29bf5f5740 1 --2- 172.21.15.43:0/2524417782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95c8a00 0x55dbf95c5c40 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.095+0000 7f299dffb640 0 client.17181 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.095+0000 7f29bf5f5740 1 --2- 172.21.15.43:0/2524417782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dbf95bfe80 0x55dbf95c56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.095+0000 7f29bf5f5740 1 --2- 172.21.15.43:0/2524417782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dbf95bf4d0 0x55dbf95c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.095+0000 7f29bf5f5740 1 -- 172.21.15.43:0/2524417782 >> 172.21.15.43:0/2524417782 conn(0x55dbf95b9ba0 msgr2=0x55dbf95bb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.095+0000 7f29bf5f5740 1 -- 172.21.15.43:0/2524417782 shutdown_connections 2024-04-21T08:42:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.095+0000 7f29bf5f5740 1 -- 172.21.15.43:0/2524417782 wait complete. 2024-04-21T08:42:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.095+0000 7f29bf5f5740 20 client.17181 trim_cache size 0 max 16384 2024-04-21T08:42:15.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.108+0000 7f29bf5f5740 1 Processor -- start 2024-04-21T08:42:15.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.108+0000 7f29bf5f5740 1 -- start start 2024-04-21T08:42:15.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.109+0000 7f29bf5f5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dbf95bf4d0 0x55dbf95d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:15.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.109+0000 7f29bf5f5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dbf95bfe80 0x55dbf95c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:15.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.109+0000 7f29bf5f5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95c8a00 0x55dbf95c8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.109+0000 7f29bf5f5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f29b81b16b0 con 0x55dbf95c8a00 2024-04-21T08:42:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.109+0000 7f29bf5f5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f29b81b31f0 con 0x55dbf95bfe80 2024-04-21T08:42:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.109+0000 7f29bf5f5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29b81b1d80 con 0x55dbf95bf4d0 2024-04-21T08:42:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.109+0000 7f29b6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95c8a00 0x55dbf95c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.109+0000 7f29b6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95c8a00 0x55dbf95c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36876/0 (socket says 172.21.15.43:36876) 2024-04-21T08:42:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.109+0000 7f29b6d9d640 1 -- 172.21.15.43:0/4223945573 learned_addr learned my addr 172.21.15.43:0/4223945573 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.109+0000 7f29b6d9d640 1 -- 172.21.15.43:0/4223945573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dbf95bf4d0 msgr2=0x55dbf95d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.109+0000 7f29b659c640 1 --2- 172.21.15.43:0/4223945573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dbf95bf4d0 0x55dbf95d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.109+0000 7f29b6d9d640 1 --2- 172.21.15.43:0/4223945573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dbf95bf4d0 0x55dbf95d4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.109+0000 7f29b6d9d640 1 -- 172.21.15.43:0/4223945573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dbf95bfe80 msgr2=0x55dbf95c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.109+0000 7f29b6d9d640 1 --2- 172.21.15.43:0/4223945573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dbf95bfe80 0x55dbf95c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.109+0000 7f29b6d9d640 1 -- 172.21.15.43:0/4223945573 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf95c3bf0 con 0x55dbf95c8a00 2024-04-21T08:42:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.109+0000 7f29b659c640 1 --2- 172.21.15.43:0/4223945573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dbf95bf4d0 0x55dbf95d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.109+0000 7f29b6d9d640 1 --2- 172.21.15.43:0/4223945573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95c8a00 0x55dbf95c8310 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f29b0014010 tx=0x7f29b001ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6f44629c2b9036 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.109+0000 7f299ffff640 1 -- 172.21.15.43:0/4223945573 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b0023030 con 0x55dbf95c8a00 2024-04-21T08:42:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.109+0000 7f299ffff640 1 -- 172.21.15.43:0/4223945573 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29b000fe80 con 0x55dbf95c8a00 2024-04-21T08:42:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.109+0000 7f299ffff640 1 -- 172.21.15.43:0/4223945573 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b002aa40 con 0x55dbf95c8a00 2024-04-21T08:42:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.110+0000 7f29bf5f5740 1 -- 172.21.15.43:0/4223945573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95c8a00 msgr2=0x55dbf95c8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.110+0000 7f29bf5f5740 1 --2- 172.21.15.43:0/4223945573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95c8a00 0x55dbf95c8310 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f29b0014010 tx=0x7f29b001ff90 comp rx=0 tx=0).stop 2024-04-21T08:42:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.110+0000 7f29bf5f5740 1 -- 172.21.15.43:0/4223945573 shutdown_connections 2024-04-21T08:42:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.110+0000 7f29bf5f5740 1 --2- 172.21.15.43:0/4223945573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95c8a00 0x55dbf95c8310 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.110+0000 7f29bf5f5740 1 --2- 172.21.15.43:0/4223945573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dbf95bfe80 0x55dbf95c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.110+0000 7f29bf5f5740 1 --2- 172.21.15.43:0/4223945573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dbf95bf4d0 0x55dbf95d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.110+0000 7f29bf5f5740 1 -- 172.21.15.43:0/4223945573 >> 172.21.15.43:0/4223945573 conn(0x55dbf95b9ba0 msgr2=0x55dbf95bb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.110+0000 7f29bf5f5740 1 -- 172.21.15.43:0/4223945573 shutdown_connections 2024-04-21T08:42:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.110+0000 7f29bf5f5740 1 -- 172.21.15.43:0/4223945573 wait complete. 2024-04-21T08:42:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.110+0000 7f29bf5f5740 1 -- start start 2024-04-21T08:42:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.110+0000 7f29bf5f5740 1 Processor -- start 2024-04-21T08:42:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.110+0000 7f29bf5f5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dbf95bf4d0 0x55dbf95cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:15.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f29bf5f5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95bfe80 0x55dbf95ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:15.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f29bf5f5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dbf95c8a00 0x55dbf95cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:15.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f29bf5f5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f29b81b2b20 con 0x55dbf95bfe80 2024-04-21T08:42:15.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f29bf5f5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f29b81b2450 con 0x55dbf95c8a00 2024-04-21T08:42:15.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f29bf5f5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29b81af690 con 0x55dbf95bf4d0 2024-04-21T08:42:15.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f29bf5f5740 10 client.0 _ll_register_callbacks cb 0x7f29b0009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:15.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f29b5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95bfe80 0x55dbf95ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:15.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f29b5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95bfe80 0x55dbf95ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36890/0 (socket says 172.21.15.43:36890) 2024-04-21T08:42:15.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f29b5d9b640 1 -- 172.21.15.43:0/99068885 learned_addr learned my addr 172.21.15.43:0/99068885 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:15.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f29b659c640 1 --2- 172.21.15.43:0/99068885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dbf95bf4d0 0x55dbf95cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:15.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f29b5d9b640 1 -- 172.21.15.43:0/99068885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dbf95bf4d0 msgr2=0x55dbf95cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:15.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f29b5d9b640 1 --2- 172.21.15.43:0/99068885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dbf95bf4d0 0x55dbf95cc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f29b5d9b640 1 -- 172.21.15.43:0/99068885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dbf95c8a00 msgr2=0x55dbf95cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:15.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f29b6d9d640 1 --2- 172.21.15.43:0/99068885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dbf95c8a00 0x55dbf95cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:15.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f29b5d9b640 1 --2- 172.21.15.43:0/99068885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dbf95c8a00 0x55dbf95cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f29b5d9b640 1 -- 172.21.15.43:0/99068885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf963e0f0 con 0x55dbf95bfe80 2024-04-21T08:42:15.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f29b659c640 1 --2- 172.21.15.43:0/99068885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dbf95bf4d0 0x55dbf95cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:15.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f29b6d9d640 1 --2- 172.21.15.43:0/99068885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dbf95c8a00 0x55dbf95cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:15.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f29b5d9b640 1 --2- 172.21.15.43:0/99068885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95bfe80 0x55dbf95ccc40 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0096c0 tx=0x7f29ac0df190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfeddb18f1e29669 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:15.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f299dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:15.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f299dffb640 1 -- 172.21.15.43:0/99068885 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac0be590 con 0x55dbf95bfe80 2024-04-21T08:42:15.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f299dffb640 1 -- 172.21.15.43:0/99068885 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f29ac0054d0 con 0x55dbf95bfe80 2024-04-21T08:42:15.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f29bf5f5740 1 -- 172.21.15.43:0/99068885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dbf9531c10 con 0x55dbf95bfe80 2024-04-21T08:42:15.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f299dffb640 1 -- 172.21.15.43:0/99068885 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f29ac0bf150 con 0x55dbf95bfe80 2024-04-21T08:42:15.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f29bf5f5740 10 client.17187 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:15.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.111+0000 7f29bf5f5740 1 -- 172.21.15.43:0/99068885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dbf963f340 con 0x55dbf95bfe80 2024-04-21T08:42:15.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.112+0000 7f299dffb640 1 -- 172.21.15.43:0/99068885 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55dbf9531c10 con 0x55dbf95bfe80 2024-04-21T08:42:15.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.112+0000 7f299dffb640 1 -- 172.21.15.43:0/99068885 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f29ac0f02e0 con 0x55dbf95bfe80 2024-04-21T08:42:15.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.112+0000 7f29bf5f5740 10 client.17187 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:15.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.112+0000 7f29bf5f5740 10 client.17187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:15.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.112+0000 7f29bf5f5740 20 client.17187 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:15.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.112+0000 7f29bf5f5740 1 --2- 172.21.15.43:0/99068885 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55dbf963f970 0x55dbf9641d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:15.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.112+0000 7f29bf5f5740 4 client.17187 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:42:15.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.112+0000 7f29bf5f5740 1 -- 172.21.15.43:0/99068885 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dbf952ba80 con 0x55dbf963f970 2024-04-21T08:42:15.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.112+0000 7f29b659c640 1 --2- 172.21.15.43:0/99068885 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55dbf963f970 0x55dbf9641d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:15.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.112+0000 7f299dffb640 10 client.17187 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:15.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.112+0000 7f29b659c640 1 --2- 172.21.15.43:0/99068885 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55dbf963f970 0x55dbf9641d60 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b604a85fff8493a5 server_cookie=97f3c2e3a9336951 in_seq=0 out_seq=0 2024-04-21T08:42:15.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f299dffb640 1 -- 172.21.15.43:0/99068885 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dbf952ba80 con 0x55dbf963f970 2024-04-21T08:42:15.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f299dffb640 10 client.17187 handle_command_reply: tid=0 2024-04-21T08:42:15.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f29bf5f5740 1 client.17187 shutdown 2024-04-21T08:42:15.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f29bf5f5740 1 -- 172.21.15.43:0/99068885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95bfe80 msgr2=0x55dbf95ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:15.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f29bf5f5740 1 --2- 172.21.15.43:0/99068885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95bfe80 0x55dbf95ccc40 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0096c0 tx=0x7f29ac0df190 comp rx=0 tx=0).stop 2024-04-21T08:42:15.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f29bf5f5740 1 -- 172.21.15.43:0/99068885 shutdown_connections 2024-04-21T08:42:15.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f29bf5f5740 1 --2- 172.21.15.43:0/99068885 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55dbf963f970 0x55dbf9641d60 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f29bf5f5740 1 --2- 172.21.15.43:0/99068885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dbf95c8a00 0x55dbf95cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f299dffb640 0 client.17187 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:15.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f29bf5f5740 1 --2- 172.21.15.43:0/99068885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dbf95bfe80 0x55dbf95ccc40 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f29bf5f5740 1 --2- 172.21.15.43:0/99068885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dbf95bf4d0 0x55dbf95cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f29bf5f5740 1 -- 172.21.15.43:0/99068885 >> 172.21.15.43:0/99068885 conn(0x55dbf95b9ba0 msgr2=0x55dbf95bafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:15.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f29bf5f5740 1 -- 172.21.15.43:0/99068885 shutdown_connections 2024-04-21T08:42:15.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f29bf5f5740 1 -- 172.21.15.43:0/99068885 wait complete. 2024-04-21T08:42:15.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f29bf5f5740 20 client.17187 trim_cache size 0 max 16384 2024-04-21T08:42:15.120 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:42:15.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f29bca06640 1 -- 172.21.15.43:0/1537052705 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f29a00a3d50 msgr2=0x7f29a00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:15.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f29bca06640 1 --2- 172.21.15.43:0/1537052705 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f29a00a3d50 0x7f29a00a6210 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f29b0005fd0 tx=0x7f29b00094b0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f29bca06640 1 -- 172.21.15.43:0/1537052705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29b8107260 msgr2=0x7f29b8074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:15.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f29bca06640 1 --2- 172.21.15.43:0/1537052705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29b8107260 0x7f29b8074790 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f29ac036040 tx=0x7f29ac02c7e0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f29bca06640 1 -- 172.21.15.43:0/1537052705 shutdown_connections 2024-04-21T08:42:15.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f29bca06640 1 --2- 172.21.15.43:0/1537052705 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f29a00a3d50 0x7f29a00a6210 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f29bca06640 1 --2- 172.21.15.43:0/1537052705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29b8074cd0 0x7f29b81a6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f29bca06640 1 --2- 172.21.15.43:0/1537052705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29b8107260 0x7f29b8074790 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f29bca06640 1 --2- 172.21.15.43:0/1537052705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b8102190 0x7f29b8074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:15.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f29bca06640 1 -- 172.21.15.43:0/1537052705 >> 172.21.15.43:0/1537052705 conn(0x7f29b80fd990 msgr2=0x7f29b80fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:15.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.113+0000 7f29bca06640 1 -- 172.21.15.43:0/1537052705 shutdown_connections 2024-04-21T08:42:15.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:15.114+0000 7f29bca06640 1 -- 172.21.15.43:0/1537052705 wait complete. 2024-04-21T08:42:15.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:15 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/4232235096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:42:15.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:15 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/4232235096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:42:15.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:15 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/4232235096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:42:16.125 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:42:16.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:42:16.126 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:42:16.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.245+0000 7f454015f640 1 -- 172.21.15.43:0/1188620409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4538101670 msgr2=0x7f453810e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:16.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.245+0000 7f454015f640 1 --2- 172.21.15.43:0/1188620409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4538101670 0x7f453810e3c0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f4528009900 tx=0x7f452802bc30 comp rx=0 tx=0).stop 2024-04-21T08:42:16.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.245+0000 7f454015f640 1 -- 172.21.15.43:0/1188620409 shutdown_connections 2024-04-21T08:42:16.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.245+0000 7f454015f640 1 --2- 172.21.15.43:0/1188620409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f453810e900 0x7f4538110cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.245+0000 7f454015f640 1 --2- 172.21.15.43:0/1188620409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4538101670 0x7f453810e3c0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.245+0000 7f454015f640 1 --2- 172.21.15.43:0/1188620409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4538100d50 0x7f4538101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.245+0000 7f454015f640 1 -- 172.21.15.43:0/1188620409 >> 172.21.15.43:0/1188620409 conn(0x7f4538077f50 msgr2=0x7f4538078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:16.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.246+0000 7f454015f640 1 -- 172.21.15.43:0/1188620409 shutdown_connections 2024-04-21T08:42:16.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.246+0000 7f454015f640 1 -- 172.21.15.43:0/1188620409 wait complete. 2024-04-21T08:42:16.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.246+0000 7f454015f640 1 Processor -- start 2024-04-21T08:42:16.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.246+0000 7f454015f640 1 -- start start 2024-04-21T08:42:16.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.246+0000 7f454015f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4538100d50 0x7f45381a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.246+0000 7f454015f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4538101670 0x7f45381a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.246+0000 7f454015f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f453810e900 0x7f453819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.246+0000 7f454015f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4538114430 con 0x7f453810e900 2024-04-21T08:42:16.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.246+0000 7f454015f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f45381142b0 con 0x7f4538101670 2024-04-21T08:42:16.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.246+0000 7f454015f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f45381145b0 con 0x7f4538100d50 2024-04-21T08:42:16.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.247+0000 7f453e6fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f453810e900 0x7f453819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.247+0000 7f453e6fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f453810e900 0x7f453819a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36910/0 (socket says 172.21.15.43:36910) 2024-04-21T08:42:16.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.247+0000 7f453e6fc640 1 -- 172.21.15.43:0/761136609 learned_addr learned my addr 172.21.15.43:0/761136609 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:16.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.247+0000 7f453d6fa640 1 --2- 172.21.15.43:0/761136609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4538101670 0x7f45381a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.247+0000 7f453e6fc640 1 -- 172.21.15.43:0/761136609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4538100d50 msgr2=0x7f45381a0680 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:42:16.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.247+0000 7f453e6fc640 1 --2- 172.21.15.43:0/761136609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4538100d50 0x7f45381a0680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.247+0000 7f453e6fc640 1 -- 172.21.15.43:0/761136609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4538101670 msgr2=0x7f45381a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:16.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.247+0000 7f453e6fc640 1 --2- 172.21.15.43:0/761136609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4538101670 0x7f45381a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.247+0000 7f453e6fc640 1 -- 172.21.15.43:0/761136609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453819afc0 con 0x7f453810e900 2024-04-21T08:42:16.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.247+0000 7f453e6fc640 1 --2- 172.21.15.43:0/761136609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f453810e900 0x7f453819a7f0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f453000dac0 tx=0x7f453000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:16.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.247+0000 7f452effd640 1 -- 172.21.15.43:0/761136609 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530017070 con 0x7f453810e900 2024-04-21T08:42:16.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.247+0000 7f454015f640 1 -- 172.21.15.43:0/761136609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f453819b2b0 con 0x7f453810e900 2024-04-21T08:42:16.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.247+0000 7f452effd640 1 -- 172.21.15.43:0/761136609 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f453000be30 con 0x7f453810e900 2024-04-21T08:42:16.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.247+0000 7f452effd640 1 -- 172.21.15.43:0/761136609 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453000f820 con 0x7f453810e900 2024-04-21T08:42:16.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.247+0000 7f454015f640 1 -- 172.21.15.43:0/761136609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45381a74d0 con 0x7f453810e900 2024-04-21T08:42:16.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.250+0000 7f452effd640 1 -- 172.21.15.43:0/761136609 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4530010ac0 con 0x7f453810e900 2024-04-21T08:42:16.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.250+0000 7f4542e2a740 1 Processor -- start 2024-04-21T08:42:16.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.250+0000 7f4542e2a740 1 -- start start 2024-04-21T08:42:16.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.250+0000 7f4542e2a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd08e0 0x55d536fd0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.250+0000 7f4542e2a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d536fd1290 0x55d536fd9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.250+0000 7f4542e2a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d536fd9e10 0x55d536fdc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.250+0000 7f4542e2a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d536f34f00 con 0x55d536fd08e0 2024-04-21T08:42:16.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.250+0000 7f4542e2a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d536ed2590 con 0x55d536fd1290 2024-04-21T08:42:16.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.250+0000 7f4542e2a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d536f30ec0 con 0x55d536fd9e10 2024-04-21T08:42:16.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.250+0000 7f452effd640 1 --2- 172.21.15.43:0/761136609 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f45240a3d20 0x7f45240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.250+0000 7f453d6fa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd08e0 0x55d536fd0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.250+0000 7f453d6fa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd08e0 0x55d536fd0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36918/0 (socket says 172.21.15.43:36918) 2024-04-21T08:42:16.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.250+0000 7f453d6fa640 1 -- 172.21.15.43:0/2787304113 learned_addr learned my addr 172.21.15.43:0/2787304113 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:16.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.251+0000 7f453e6fc640 1 --2- 172.21.15.43:0/761136609 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f45240a3d20 0x7f45240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.251+0000 7f453defb640 1 --2- 172.21.15.43:0/2787304113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d536fd1290 0x55d536fd9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.251+0000 7f452effd640 1 -- 172.21.15.43:0/761136609 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f45300d0b70 con 0x7f453810e900 2024-04-21T08:42:16.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.251+0000 7f453d6fa640 1 -- 172.21.15.43:0/2787304113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d536fd9e10 msgr2=0x55d536fdc260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:16.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.251+0000 7f453d6fa640 1 --2- 172.21.15.43:0/2787304113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d536fd9e10 0x55d536fdc260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.251+0000 7f453d6fa640 1 -- 172.21.15.43:0/2787304113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d536fd1290 msgr2=0x55d536fd9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:16.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.251+0000 7f453d6fa640 1 --2- 172.21.15.43:0/2787304113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d536fd1290 0x55d536fd9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.251+0000 7f453d6fa640 1 -- 172.21.15.43:0/2787304113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d536d0f7e0 con 0x55d536fd08e0 2024-04-21T08:42:16.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.251+0000 7f453d6fa640 1 --2- 172.21.15.43:0/2787304113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd08e0 0x55d536fd0cc0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f4528009900 tx=0x7f452802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15cd357e4fe349f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:16.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.251+0000 7f453e6fc640 1 --2- 172.21.15.43:0/761136609 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f45240a3d20 0x7f45240a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f453009c3e0 tx=0x7f453006bfc0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:16.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.251+0000 7f4523fff640 1 -- 172.21.15.43:0/2787304113 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452802c950 con 0x55d536fd08e0 2024-04-21T08:42:16.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.251+0000 7f4523fff640 1 -- 172.21.15.43:0/2787304113 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4528046070 con 0x55d536fd08e0 2024-04-21T08:42:16.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.251+0000 7f4523fff640 1 -- 172.21.15.43:0/2787304113 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45280046c0 con 0x55d536fd08e0 2024-04-21T08:42:16.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.252+0000 7f4542e2a740 1 -- 172.21.15.43:0/2787304113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd08e0 msgr2=0x55d536fd0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:16.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.252+0000 7f4542e2a740 1 --2- 172.21.15.43:0/2787304113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd08e0 0x55d536fd0cc0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f4528009900 tx=0x7f452802c7e0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.252+0000 7f4542e2a740 1 -- 172.21.15.43:0/2787304113 shutdown_connections 2024-04-21T08:42:16.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.252+0000 7f4542e2a740 1 --2- 172.21.15.43:0/2787304113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d536fd9e10 0x55d536fdc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.252+0000 7f4542e2a740 1 --2- 172.21.15.43:0/2787304113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d536fd1290 0x55d536fd9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.252+0000 7f4542e2a740 1 --2- 172.21.15.43:0/2787304113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd08e0 0x55d536fd0cc0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.252+0000 7f4542e2a740 1 -- 172.21.15.43:0/2787304113 >> 172.21.15.43:0/2787304113 conn(0x55d536fcafb0 msgr2=0x55d536fcb390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:16.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.252+0000 7f4542e2a740 1 -- 172.21.15.43:0/2787304113 shutdown_connections 2024-04-21T08:42:16.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.252+0000 7f4542e2a740 1 -- 172.21.15.43:0/2787304113 wait complete. 2024-04-21T08:42:16.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.252+0000 7f4542e2a740 1 -- start start 2024-04-21T08:42:16.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.252+0000 7f4542e2a740 1 Processor -- start 2024-04-21T08:42:16.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.253+0000 7f4542e2a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d536fd08e0 0x55d536fd6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.253+0000 7f4542e2a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd1290 0x55d536fd7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.253+0000 7f4542e2a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d536fd9e10 0x55d536fd7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.253+0000 7f4542e2a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d536fdcd20 con 0x55d536fd1290 2024-04-21T08:42:16.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.253+0000 7f4542e2a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d536eee580 con 0x55d536fd9e10 2024-04-21T08:42:16.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.253+0000 7f4542e2a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d536fdcea0 con 0x55d536fd08e0 2024-04-21T08:42:16.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.253+0000 7f4542e2a740 10 client.0 _ll_register_callbacks cb 0x55d536ef4650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:16.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.253+0000 7f453defb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd1290 0x55d536fd7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.253+0000 7f453defb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd1290 0x55d536fd7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36930/0 (socket says 172.21.15.43:36930) 2024-04-21T08:42:16.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.253+0000 7f453defb640 1 -- 172.21.15.43:0/3247607389 learned_addr learned my addr 172.21.15.43:0/3247607389 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:16.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.253+0000 7f453d6fa640 1 --2- 172.21.15.43:0/3247607389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d536fd9e10 0x55d536fd7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.253+0000 7f453d6fa640 1 --2- 172.21.15.43:0/3247607389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d536fd08e0 0x55d536fd6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.253+0000 7f453defb640 1 -- 172.21.15.43:0/3247607389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d536fd08e0 msgr2=0x55d536fd6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:16.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.253+0000 7f453defb640 1 --2- 172.21.15.43:0/3247607389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d536fd08e0 0x55d536fd6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.253+0000 7f453defb640 1 -- 172.21.15.43:0/3247607389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d536fd9e10 msgr2=0x55d536fd7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:16.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.253+0000 7f453defb640 1 --2- 172.21.15.43:0/3247607389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d536fd9e10 0x55d536fd7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.253+0000 7f453defb640 1 -- 172.21.15.43:0/3247607389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d536d6ff70 con 0x55d536fd1290 2024-04-21T08:42:16.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.253+0000 7f453d6fa640 1 --2- 172.21.15.43:0/3247607389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d536fd08e0 0x55d536fd6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:16.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.254+0000 7f453defb640 1 --2- 172.21.15.43:0/3247607389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd1290 0x55d536fd7320 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f4534006aa0 tx=0x7f453400d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b906c417f268603 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:16.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.254+0000 7f45217fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:16.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.254+0000 7f45217fa640 1 -- 172.21.15.43:0/3247607389 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534014430 con 0x55d536fd1290 2024-04-21T08:42:16.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.254+0000 7f45217fa640 1 -- 172.21.15.43:0/3247607389 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f45340145d0 con 0x55d536fd1290 2024-04-21T08:42:16.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.254+0000 7f4542e2a740 1 -- 172.21.15.43:0/3247607389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4528046070 con 0x55d536fd1290 2024-04-21T08:42:16.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.254+0000 7f45217fa640 1 -- 172.21.15.43:0/3247607389 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4534022050 con 0x55d536fd1290 2024-04-21T08:42:16.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.254+0000 7f4542e2a740 10 client.17199 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:16.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.254+0000 7f4542e2a740 1 -- 172.21.15.43:0/3247607389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d536c07170 con 0x55d536fd1290 2024-04-21T08:42:16.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.254+0000 7f45217fa640 1 -- 172.21.15.43:0/3247607389 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4528046070 con 0x55d536fd1290 2024-04-21T08:42:16.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.254+0000 7f45217fa640 1 -- 172.21.15.43:0/3247607389 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4534013070 con 0x55d536fd1290 2024-04-21T08:42:16.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.254+0000 7f4542e2a740 10 client.17199 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:16.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.254+0000 7f4542e2a740 10 client.17199 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:16.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.254+0000 7f4542e2a740 20 client.17199 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:16.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.254+0000 7f4542e2a740 1 --2- 172.21.15.43:0/3247607389 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d536fea430 0x55d536fec870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.254+0000 7f4542e2a740 4 client.17199 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:16.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.254+0000 7f4542e2a740 1 -- 172.21.15.43:0/3247607389 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d536f42920 con 0x55d536fea430 2024-04-21T08:42:16.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.255+0000 7f453d6fa640 1 --2- 172.21.15.43:0/3247607389 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d536fea430 0x55d536fec870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.255+0000 7f453d6fa640 1 --2- 172.21.15.43:0/3247607389 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d536fea430 0x55d536fec870 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eebe5bca6c28e31 server_cookie=2ae11e1a977af271 in_seq=0 out_seq=0 2024-04-21T08:42:16.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.255+0000 7f45217fa640 10 client.17199 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:16.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.256+0000 7f45217fa640 1 -- 172.21.15.43:0/3247607389 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d536f42920 con 0x55d536fea430 2024-04-21T08:42:16.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.256+0000 7f45217fa640 10 client.17199 handle_command_reply: tid=0 2024-04-21T08:42:16.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.256+0000 7f4542e2a740 1 client.17199 shutdown 2024-04-21T08:42:16.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.256+0000 7f4542e2a740 1 -- 172.21.15.43:0/3247607389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd1290 msgr2=0x55d536fd7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:16.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.256+0000 7f4542e2a740 1 --2- 172.21.15.43:0/3247607389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd1290 0x55d536fd7320 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f4534006aa0 tx=0x7f453400d5f0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.256+0000 7f4542e2a740 1 -- 172.21.15.43:0/3247607389 shutdown_connections 2024-04-21T08:42:16.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.256+0000 7f4542e2a740 1 --2- 172.21.15.43:0/3247607389 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d536fea430 0x55d536fec870 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.256+0000 7f4542e2a740 1 --2- 172.21.15.43:0/3247607389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d536fd9e10 0x55d536fd7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.256+0000 7f4542e2a740 1 --2- 172.21.15.43:0/3247607389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd1290 0x55d536fd7320 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.256+0000 7f45217fa640 0 client.17199 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:16.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.256+0000 7f4542e2a740 1 --2- 172.21.15.43:0/3247607389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d536fd08e0 0x55d536fd6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.256+0000 7f4542e2a740 1 -- 172.21.15.43:0/3247607389 >> 172.21.15.43:0/3247607389 conn(0x55d536fcafb0 msgr2=0x55d536fccd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:16.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.256+0000 7f4542e2a740 1 -- 172.21.15.43:0/3247607389 shutdown_connections 2024-04-21T08:42:16.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.256+0000 7f4542e2a740 1 -- 172.21.15.43:0/3247607389 wait complete. 2024-04-21T08:42:16.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.256+0000 7f4542e2a740 20 client.17199 trim_cache size 0 max 16384 2024-04-21T08:42:16.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.274+0000 7f4542e2a740 1 Processor -- start 2024-04-21T08:42:16.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.274+0000 7f4542e2a740 1 -- start start 2024-04-21T08:42:16.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.274+0000 7f4542e2a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d536fd08e0 0x55d536fd9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.274+0000 7f4542e2a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d536fd1290 0x55d536fd18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.274+0000 7f4542e2a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd9e10 0x55d536fe5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.274+0000 7f4542e2a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f45381b1530 con 0x55d536fd9e10 2024-04-21T08:42:16.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.274+0000 7f4542e2a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f45381b3070 con 0x55d536fd08e0 2024-04-21T08:42:16.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.274+0000 7f4542e2a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f45381b1c00 con 0x55d536fd1290 2024-04-21T08:42:16.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.275+0000 7f453d6fa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd9e10 0x55d536fe5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.275+0000 7f453d6fa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd9e10 0x55d536fe5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36942/0 (socket says 172.21.15.43:36942) 2024-04-21T08:42:16.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.275+0000 7f453d6fa640 1 -- 172.21.15.43:0/3137584724 learned_addr learned my addr 172.21.15.43:0/3137584724 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:16.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.275+0000 7f453d6fa640 1 --2- 172.21.15.43:0/3137584724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d536fd08e0 0x55d536fd9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.275+0000 7f453defb640 1 --2- 172.21.15.43:0/3137584724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d536fd1290 0x55d536fd18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.275+0000 7f453d6fa640 1 -- 172.21.15.43:0/3137584724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d536fd1290 msgr2=0x55d536fd18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:16.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.275+0000 7f453d6fa640 1 --2- 172.21.15.43:0/3137584724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d536fd1290 0x55d536fd18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.275+0000 7f453d6fa640 1 -- 172.21.15.43:0/3137584724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d536fd08e0 msgr2=0x55d536fd9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:16.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.275+0000 7f453d6fa640 1 --2- 172.21.15.43:0/3137584724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d536fd08e0 0x55d536fd9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.275+0000 7f453d6fa640 1 -- 172.21.15.43:0/3137584724 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d536fca790 con 0x55d536fd9e10 2024-04-21T08:42:16.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.275+0000 7f453d6fa640 1 --2- 172.21.15.43:0/3137584724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd9e10 0x55d536fe5be0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f4528009900 tx=0x7f452802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2696f4e067b57c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:16.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.275+0000 7f45237fe640 1 -- 172.21.15.43:0/3137584724 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452802cb50 con 0x55d536fd9e10 2024-04-21T08:42:16.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.275+0000 7f45237fe640 1 -- 172.21.15.43:0/3137584724 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4528046040 con 0x55d536fd9e10 2024-04-21T08:42:16.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.275+0000 7f45237fe640 1 -- 172.21.15.43:0/3137584724 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452804e470 con 0x55d536fd9e10 2024-04-21T08:42:16.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.276+0000 7f4542e2a740 1 -- 172.21.15.43:0/3137584724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd9e10 msgr2=0x55d536fe5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:16.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.276+0000 7f4542e2a740 1 --2- 172.21.15.43:0/3137584724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd9e10 0x55d536fe5be0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f4528009900 tx=0x7f452802c7e0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.276+0000 7f4542e2a740 1 -- 172.21.15.43:0/3137584724 shutdown_connections 2024-04-21T08:42:16.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.276+0000 7f4542e2a740 1 --2- 172.21.15.43:0/3137584724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd9e10 0x55d536fe5be0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.276+0000 7f4542e2a740 1 --2- 172.21.15.43:0/3137584724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d536fd1290 0x55d536fd18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.276+0000 7f4542e2a740 1 --2- 172.21.15.43:0/3137584724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d536fd08e0 0x55d536fd9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.276+0000 7f4542e2a740 1 -- 172.21.15.43:0/3137584724 >> 172.21.15.43:0/3137584724 conn(0x55d536fcafb0 msgr2=0x55d536fd6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:16.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.276+0000 7f4542e2a740 1 -- 172.21.15.43:0/3137584724 shutdown_connections 2024-04-21T08:42:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.276+0000 7f4542e2a740 1 -- 172.21.15.43:0/3137584724 wait complete. 2024-04-21T08:42:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.276+0000 7f4542e2a740 1 -- start start 2024-04-21T08:42:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.276+0000 7f4542e2a740 1 Processor -- start 2024-04-21T08:42:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.277+0000 7f4542e2a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d536fd08e0 0x55d536fe20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.277+0000 7f4542e2a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d536fd1290 0x55d536fe2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.277+0000 7f4542e2a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd9e10 0x55d536fdf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.277+0000 7f4542e2a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f45381b29a0 con 0x55d536fd9e10 2024-04-21T08:42:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.277+0000 7f4542e2a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f45381b22d0 con 0x55d536fd08e0 2024-04-21T08:42:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.277+0000 7f4542e2a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f45381af510 con 0x55d536fd1290 2024-04-21T08:42:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.277+0000 7f4542e2a740 10 client.0 _ll_register_callbacks cb 0x7f453400b0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.277+0000 7f453d6fa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd9e10 0x55d536fdf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.277+0000 7f453d6fa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd9e10 0x55d536fdf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36956/0 (socket says 172.21.15.43:36956) 2024-04-21T08:42:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.277+0000 7f453d6fa640 1 -- 172.21.15.43:0/3192382543 learned_addr learned my addr 172.21.15.43:0/3192382543 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.277+0000 7f453defb640 1 --2- 172.21.15.43:0/3192382543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d536fd1290 0x55d536fe2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.277+0000 7f453d6fa640 1 --2- 172.21.15.43:0/3192382543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d536fd08e0 0x55d536fe20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.277+0000 7f453d6fa640 1 -- 172.21.15.43:0/3192382543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d536fd1290 msgr2=0x55d536fe2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:16.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.277+0000 7f453d6fa640 1 --2- 172.21.15.43:0/3192382543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d536fd1290 0x55d536fe2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.277+0000 7f453d6fa640 1 -- 172.21.15.43:0/3192382543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d536fd08e0 msgr2=0x55d536fe20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:16.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.277+0000 7f453d6fa640 1 --2- 172.21.15.43:0/3192382543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d536fd08e0 0x55d536fe20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.277+0000 7f453d6fa640 1 -- 172.21.15.43:0/3192382543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d536fdf950 con 0x55d536fd9e10 2024-04-21T08:42:16.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.277+0000 7f453defb640 1 --2- 172.21.15.43:0/3192382543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d536fd1290 0x55d536fe2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:16.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.277+0000 7f453d6fa640 1 --2- 172.21.15.43:0/3192382543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd9e10 0x55d536fdf120 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f452802c170 tx=0x7f452802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a8ba32b211741d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:16.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.277+0000 7f45217fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:16.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.277+0000 7f45217fa640 1 -- 172.21.15.43:0/3192382543 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4528004bd0 con 0x55d536fd9e10 2024-04-21T08:42:16.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.277+0000 7f45217fa640 1 -- 172.21.15.43:0/3192382543 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f4528004d70 con 0x55d536fd9e10 2024-04-21T08:42:16.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.277+0000 7f4542e2a740 1 -- 172.21.15.43:0/3192382543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4528046040 con 0x55d536fd9e10 2024-04-21T08:42:16.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.278+0000 7f45217fa640 1 -- 172.21.15.43:0/3192382543 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f452804e0f0 con 0x55d536fd9e10 2024-04-21T08:42:16.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.278+0000 7f4542e2a740 10 client.17205 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:16.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.278+0000 7f4542e2a740 1 -- 172.21.15.43:0/3192382543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d537051580 con 0x55d536fd9e10 2024-04-21T08:42:16.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.278+0000 7f45217fa640 1 -- 172.21.15.43:0/3192382543 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4528046040 con 0x55d536fd9e10 2024-04-21T08:42:16.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.278+0000 7f45217fa640 1 -- 172.21.15.43:0/3192382543 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4528060030 con 0x55d536fd9e10 2024-04-21T08:42:16.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.278+0000 7f4542e2a740 10 client.17205 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:16.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.278+0000 7f4542e2a740 10 client.17205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:16.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.278+0000 7f4542e2a740 20 client.17205 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:16.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.278+0000 7f4542e2a740 1 --2- 172.21.15.43:0/3192382543 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d5370517a0 0x55d537053b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.278+0000 7f4542e2a740 4 client.17205 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:42:16.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.278+0000 7f4542e2a740 1 -- 172.21.15.43:0/3192382543 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d536f3c790 con 0x55d5370517a0 2024-04-21T08:42:16.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.278+0000 7f453defb640 1 --2- 172.21.15.43:0/3192382543 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d5370517a0 0x55d537053b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.279+0000 7f453defb640 1 --2- 172.21.15.43:0/3192382543 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d5370517a0 0x55d537053b90 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c0c2091702c8f0a server_cookie=f1bdda420794e8e in_seq=0 out_seq=0 2024-04-21T08:42:16.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.279+0000 7f45217fa640 10 client.17205 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:16.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.280+0000 7f45217fa640 1 -- 172.21.15.43:0/3192382543 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d536f3c790 con 0x55d5370517a0 2024-04-21T08:42:16.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.280+0000 7f45217fa640 10 client.17205 handle_command_reply: tid=0 2024-04-21T08:42:16.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.280+0000 7f4542e2a740 1 client.17205 shutdown 2024-04-21T08:42:16.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.280+0000 7f4542e2a740 1 -- 172.21.15.43:0/3192382543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd9e10 msgr2=0x55d536fdf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:16.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.280+0000 7f4542e2a740 1 --2- 172.21.15.43:0/3192382543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd9e10 0x55d536fdf120 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f452802c170 tx=0x7f452802bc30 comp rx=0 tx=0).stop 2024-04-21T08:42:16.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.280+0000 7f4542e2a740 1 -- 172.21.15.43:0/3192382543 shutdown_connections 2024-04-21T08:42:16.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.280+0000 7f4542e2a740 1 --2- 172.21.15.43:0/3192382543 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d5370517a0 0x55d537053b90 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.280+0000 7f4542e2a740 1 --2- 172.21.15.43:0/3192382543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d536fd9e10 0x55d536fdf120 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.280+0000 7f4542e2a740 1 --2- 172.21.15.43:0/3192382543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d536fd1290 0x55d536fe2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.280+0000 7f45217fa640 0 client.17205 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:16.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.280+0000 7f4542e2a740 1 --2- 172.21.15.43:0/3192382543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d536fd08e0 0x55d536fe20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.280+0000 7f4542e2a740 1 -- 172.21.15.43:0/3192382543 >> 172.21.15.43:0/3192382543 conn(0x55d536fcafb0 msgr2=0x55d536fd75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:16.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.280+0000 7f4542e2a740 1 -- 172.21.15.43:0/3192382543 shutdown_connections 2024-04-21T08:42:16.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.280+0000 7f4542e2a740 1 -- 172.21.15.43:0/3192382543 wait complete. 2024-04-21T08:42:16.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.280+0000 7f4542e2a740 20 client.17205 trim_cache size 0 max 16384 2024-04-21T08:42:16.287 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:42:16.287 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:42:16.287 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "59e0dfdf-6ef4-4aae-a2b4-48edda687e1d", 2024-04-21T08:42:16.287 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:42:16.287 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:42:16.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.281+0000 7f454015f640 1 -- 172.21.15.43:0/761136609 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f45240a3d20 msgr2=0x7f45240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:16.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.281+0000 7f454015f640 1 --2- 172.21.15.43:0/761136609 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f45240a3d20 0x7f45240a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f453009c3e0 tx=0x7f453006bfc0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.281+0000 7f454015f640 1 -- 172.21.15.43:0/761136609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f453810e900 msgr2=0x7f453819a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:16.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.281+0000 7f454015f640 1 --2- 172.21.15.43:0/761136609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f453810e900 0x7f453819a7f0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f453000dac0 tx=0x7f453000df80 comp rx=0 tx=0).stop 2024-04-21T08:42:16.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.281+0000 7f454015f640 1 -- 172.21.15.43:0/761136609 shutdown_connections 2024-04-21T08:42:16.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.281+0000 7f454015f640 1 --2- 172.21.15.43:0/761136609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f453810e900 0x7f453819a7f0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.281+0000 7f454015f640 1 --2- 172.21.15.43:0/761136609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4538101670 0x7f45381a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.281+0000 7f454015f640 1 --2- 172.21.15.43:0/761136609 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f45240a3d20 0x7f45240a61e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.281+0000 7f454015f640 1 --2- 172.21.15.43:0/761136609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4538100d50 0x7f45381a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.281+0000 7f454015f640 1 -- 172.21.15.43:0/761136609 >> 172.21.15.43:0/761136609 conn(0x7f4538077f50 msgr2=0x7f4538110000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:16.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.281+0000 7f454015f640 1 -- 172.21.15.43:0/761136609 shutdown_connections 2024-04-21T08:42:16.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.281+0000 7f454015f640 1 -- 172.21.15.43:0/761136609 wait complete. 2024-04-21T08:42:16.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:42:16.292 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:42:16.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.397+0000 7f245ccd2640 1 -- 172.21.15.43:0/1206061917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2458105dd0 msgr2=0x7f24581061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:16.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.397+0000 7f245ccd2640 1 --2- 172.21.15.43:0/1206061917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2458105dd0 0x7f24581061b0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f2448009900 tx=0x7f244802bc30 comp rx=0 tx=0).stop 2024-04-21T08:42:16.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.397+0000 7f245ccd2640 1 -- 172.21.15.43:0/1206061917 shutdown_connections 2024-04-21T08:42:16.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.397+0000 7f245ccd2640 1 --2- 172.21.15.43:0/1206061917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2458101a40 0x7f245810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.397+0000 7f245ccd2640 1 --2- 172.21.15.43:0/1206061917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24581010a0 0x7f2458101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.397+0000 7f245ccd2640 1 --2- 172.21.15.43:0/1206061917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2458105dd0 0x7f24581061b0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.397+0000 7f245ccd2640 1 -- 172.21.15.43:0/1206061917 >> 172.21.15.43:0/1206061917 conn(0x7f2458077c40 msgr2=0x7f2458078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:16.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.397+0000 7f245ccd2640 1 -- 172.21.15.43:0/1206061917 shutdown_connections 2024-04-21T08:42:16.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.397+0000 7f245ccd2640 1 -- 172.21.15.43:0/1206061917 wait complete. 2024-04-21T08:42:16.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.398+0000 7f245ccd2640 1 Processor -- start 2024-04-21T08:42:16.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.398+0000 7f245ccd2640 1 -- start start 2024-04-21T08:42:16.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.398+0000 7f245ccd2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24581010a0 0x7f2458199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.398+0000 7f245ccd2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2458101a40 0x7f2458199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.398+0000 7f245ccd2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2458105dd0 0x7f24581a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.398+0000 7f245ccd2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24581111a0 con 0x7f2458101a40 2024-04-21T08:42:16.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.398+0000 7f245ccd2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2458111020 con 0x7f24581010a0 2024-04-21T08:42:16.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.398+0000 7f245ccd2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2458111320 con 0x7f2458105dd0 2024-04-21T08:42:16.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.398+0000 7f244ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2458101a40 0x7f2458199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.398+0000 7f244ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2458101a40 0x7f2458199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36970/0 (socket says 172.21.15.43:36970) 2024-04-21T08:42:16.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.398+0000 7f244ffff640 1 -- 172.21.15.43:0/3441515330 learned_addr learned my addr 172.21.15.43:0/3441515330 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:16.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.398+0000 7f245659c640 1 --2- 172.21.15.43:0/3441515330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24581010a0 0x7f2458199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.399+0000 7f2456d9d640 1 --2- 172.21.15.43:0/3441515330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2458105dd0 0x7f24581a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.399+0000 7f244ffff640 1 -- 172.21.15.43:0/3441515330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2458105dd0 msgr2=0x7f24581a45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:16.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.399+0000 7f244ffff640 1 --2- 172.21.15.43:0/3441515330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2458105dd0 0x7f24581a45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.399+0000 7f244ffff640 1 -- 172.21.15.43:0/3441515330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24581010a0 msgr2=0x7f2458199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:16.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.399+0000 7f244ffff640 1 --2- 172.21.15.43:0/3441515330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24581010a0 0x7f2458199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.399+0000 7f244ffff640 1 -- 172.21.15.43:0/3441515330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24581a4b20 con 0x7f2458101a40 2024-04-21T08:42:16.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.399+0000 7f244ffff640 1 --2- 172.21.15.43:0/3441515330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2458101a40 0x7f2458199aa0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f244000c920 tx=0x7f244000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:16.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.399+0000 7f244f7fe640 1 -- 172.21.15.43:0/3441515330 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2440007c70 con 0x7f2458101a40 2024-04-21T08:42:16.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.399+0000 7f244f7fe640 1 -- 172.21.15.43:0/3441515330 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2440007e10 con 0x7f2458101a40 2024-04-21T08:42:16.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.399+0000 7f245ccd2640 1 -- 172.21.15.43:0/3441515330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24581a4cf0 con 0x7f2458101a40 2024-04-21T08:42:16.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.399+0000 7f244f7fe640 1 -- 172.21.15.43:0/3441515330 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2440018820 con 0x7f2458101a40 2024-04-21T08:42:16.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.399+0000 7f245ccd2640 1 -- 172.21.15.43:0/3441515330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24581a51b0 con 0x7f2458101a40 2024-04-21T08:42:16.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.401+0000 7f244f7fe640 1 -- 172.21.15.43:0/3441515330 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f244000f450 con 0x7f2458101a40 2024-04-21T08:42:16.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.401+0000 7f245f9a3740 1 Processor -- start 2024-04-21T08:42:16.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.402+0000 7f245f9a3740 1 -- start start 2024-04-21T08:42:16.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.402+0000 7f245f9a3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a56f0b580 0x560a56f0b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.402+0000 7f245f9a3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f0bea0 0x560a56f14540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.402+0000 7f245f9a3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a56f14a80 0x560a56f16e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.402+0000 7f244f7fe640 1 --2- 172.21.15.43:0/3441515330 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f24440a3d20 0x7f24440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.402+0000 7f245f9a3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560a56e701e0 con 0x560a56f0bea0 2024-04-21T08:42:16.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.402+0000 7f245f9a3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560a56e0d390 con 0x560a56f14a80 2024-04-21T08:42:16.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.402+0000 7f245f9a3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560a56e6c1a0 con 0x560a56f0b580 2024-04-21T08:42:16.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.402+0000 7f245659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f0bea0 0x560a56f14540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.402+0000 7f245659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f0bea0 0x560a56f14540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36976/0 (socket says 172.21.15.43:36976) 2024-04-21T08:42:16.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.402+0000 7f244ffff640 1 --2- 172.21.15.43:0/3441515330 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f24440a3d20 0x7f24440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.402+0000 7f245659c640 1 -- 172.21.15.43:0/1393394801 learned_addr learned my addr 172.21.15.43:0/1393394801 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:16.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.402+0000 7f2456d9d640 1 --2- 172.21.15.43:0/1393394801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a56f0b580 0x560a56f0b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.402+0000 7f244f7fe640 1 -- 172.21.15.43:0/3441515330 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f24400d0d60 con 0x7f2458101a40 2024-04-21T08:42:16.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.402+0000 7f2456d9d640 1 --2- 172.21.15.43:0/1393394801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a56f14a80 0x560a56f16e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.402+0000 7f245659c640 1 -- 172.21.15.43:0/1393394801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a56f0b580 msgr2=0x560a56f0b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:16.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.402+0000 7f245659c640 1 --2- 172.21.15.43:0/1393394801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a56f0b580 0x560a56f0b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.402+0000 7f245659c640 1 -- 172.21.15.43:0/1393394801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a56f14a80 msgr2=0x560a56f16e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:16.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.402+0000 7f245659c640 1 --2- 172.21.15.43:0/1393394801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a56f14a80 0x560a56f16e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.402+0000 7f245659c640 1 -- 172.21.15.43:0/1393394801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a56c4c7d0 con 0x560a56f0bea0 2024-04-21T08:42:16.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.402+0000 7f2456d9d640 1 --2- 172.21.15.43:0/1393394801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a56f14a80 0x560a56f16e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:16.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.403+0000 7f245659c640 1 --2- 172.21.15.43:0/1393394801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f0bea0 0x560a56f14540 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f2448009230 tx=0x7f244802c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3218ac8f9b7006aa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:16.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.403+0000 7f244ffff640 1 --2- 172.21.15.43:0/3441515330 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f24440a3d20 0x7f24440a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f24400992a0 tx=0x7f24400bd040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:16.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.403+0000 7f244cff9640 1 -- 172.21.15.43:0/1393394801 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2448045020 con 0x560a56f0bea0 2024-04-21T08:42:16.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.403+0000 7f244cff9640 1 -- 172.21.15.43:0/1393394801 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f244802c7e0 con 0x560a56f0bea0 2024-04-21T08:42:16.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.403+0000 7f244cff9640 1 -- 172.21.15.43:0/1393394801 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24480046c0 con 0x560a56f0bea0 2024-04-21T08:42:16.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.404+0000 7f245f9a3740 1 -- 172.21.15.43:0/1393394801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f0bea0 msgr2=0x560a56f14540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:16.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.404+0000 7f245f9a3740 1 --2- 172.21.15.43:0/1393394801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f0bea0 0x560a56f14540 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f2448009230 tx=0x7f244802c720 comp rx=0 tx=0).stop 2024-04-21T08:42:16.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.404+0000 7f245f9a3740 1 -- 172.21.15.43:0/1393394801 shutdown_connections 2024-04-21T08:42:16.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.404+0000 7f245f9a3740 1 --2- 172.21.15.43:0/1393394801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a56f14a80 0x560a56f16e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.404+0000 7f245f9a3740 1 --2- 172.21.15.43:0/1393394801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f0bea0 0x560a56f14540 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.404+0000 7f245f9a3740 1 --2- 172.21.15.43:0/1393394801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a56f0b580 0x560a56f0b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.404+0000 7f245f9a3740 1 -- 172.21.15.43:0/1393394801 >> 172.21.15.43:0/1393394801 conn(0x560a56f05cb0 msgr2=0x560a56f06090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:16.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.404+0000 7f245f9a3740 1 -- 172.21.15.43:0/1393394801 shutdown_connections 2024-04-21T08:42:16.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.404+0000 7f245f9a3740 1 -- 172.21.15.43:0/1393394801 wait complete. 2024-04-21T08:42:16.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.404+0000 7f245f9a3740 1 -- start start 2024-04-21T08:42:16.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.404+0000 7f245f9a3740 1 Processor -- start 2024-04-21T08:42:16.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.404+0000 7f245f9a3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a56f0b580 0x560a56f11ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.404+0000 7f245f9a3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a56f0bea0 0x560a56f11ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.404+0000 7f245f9a3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f14a80 0x560a56f12530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.404+0000 7f245f9a3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560a56f17810 con 0x560a56f14a80 2024-04-21T08:42:16.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.404+0000 7f245f9a3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560a56e292f0 con 0x560a56f0b580 2024-04-21T08:42:16.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.404+0000 7f245f9a3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560a56f17990 con 0x560a56f0bea0 2024-04-21T08:42:16.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.404+0000 7f245f9a3740 10 client.0 _ll_register_callbacks cb 0x560a56e36580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:16.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.404+0000 7f2456d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f14a80 0x560a56f12530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.404+0000 7f2456d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f14a80 0x560a56f12530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36984/0 (socket says 172.21.15.43:36984) 2024-04-21T08:42:16.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.405+0000 7f2456d9d640 1 -- 172.21.15.43:0/617773592 learned_addr learned my addr 172.21.15.43:0/617773592 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:16.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.405+0000 7f2456d9d640 1 --2- 172.21.15.43:0/617773592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a56f0b580 0x560a56f11ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.405+0000 7f245659c640 1 --2- 172.21.15.43:0/617773592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a56f0bea0 0x560a56f11ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.405+0000 7f2456d9d640 1 -- 172.21.15.43:0/617773592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a56f0bea0 msgr2=0x560a56f11ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:16.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.405+0000 7f2456d9d640 1 --2- 172.21.15.43:0/617773592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a56f0bea0 0x560a56f11ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.405+0000 7f2456d9d640 1 -- 172.21.15.43:0/617773592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a56f0b580 msgr2=0x560a56f11ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:16.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.405+0000 7f2456d9d640 1 --2- 172.21.15.43:0/617773592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a56f0b580 0x560a56f11ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.405+0000 7f2456d9d640 1 -- 172.21.15.43:0/617773592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a56b823f0 con 0x560a56f14a80 2024-04-21T08:42:16.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.405+0000 7f2456d9d640 1 --2- 172.21.15.43:0/617773592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f14a80 0x560a56f12530 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f245000f410 tx=0x7f2450009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2c3fc73f70535a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:16.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.405+0000 7f243dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:16.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.405+0000 7f243dffb640 1 -- 172.21.15.43:0/617773592 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24500115e0 con 0x560a56f14a80 2024-04-21T08:42:16.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.405+0000 7f243dffb640 1 -- 172.21.15.43:0/617773592 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f2450011780 con 0x560a56f14a80 2024-04-21T08:42:16.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.405+0000 7f245f9a3740 1 -- 172.21.15.43:0/617773592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f244802c7e0 con 0x560a56f14a80 2024-04-21T08:42:16.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.405+0000 7f243dffb640 1 -- 172.21.15.43:0/617773592 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2450027050 con 0x560a56f14a80 2024-04-21T08:42:16.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.405+0000 7f245f9a3740 10 client.17217 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:16.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.405+0000 7f245f9a3740 1 -- 172.21.15.43:0/617773592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560a56b420f0 con 0x560a56f14a80 2024-04-21T08:42:16.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.406+0000 7f243dffb640 1 -- 172.21.15.43:0/617773592 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f244802c7e0 con 0x560a56f14a80 2024-04-21T08:42:16.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.406+0000 7f243dffb640 1 -- 172.21.15.43:0/617773592 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f245000f040 con 0x560a56f14a80 2024-04-21T08:42:16.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.406+0000 7f245f9a3740 10 client.17217 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:16.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.406+0000 7f245f9a3740 10 client.17217 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:16.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.406+0000 7f245f9a3740 20 client.17217 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:16.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.406+0000 7f245f9a3740 1 --2- 172.21.15.43:0/617773592 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560a56f24a10 0x560a56f26e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.406+0000 7f245f9a3740 4 client.17217 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:16.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.406+0000 7f245f9a3740 1 -- 172.21.15.43:0/617773592 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a56e7dc00 con 0x560a56f24a10 2024-04-21T08:42:16.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.406+0000 7f245659c640 1 --2- 172.21.15.43:0/617773592 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560a56f24a10 0x560a56f26e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.406+0000 7f245659c640 1 --2- 172.21.15.43:0/617773592 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560a56f24a10 0x560a56f26e40 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a94f269c71b6218 server_cookie=b38b7ae4463738e9 in_seq=0 out_seq=0 2024-04-21T08:42:16.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.406+0000 7f243dffb640 10 client.17217 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:16.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.407+0000 7f243dffb640 1 -- 172.21.15.43:0/617773592 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a56e7dc00 con 0x560a56f24a10 2024-04-21T08:42:16.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.407+0000 7f243dffb640 10 client.17217 handle_command_reply: tid=0 2024-04-21T08:42:16.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.407+0000 7f245f9a3740 1 client.17217 shutdown 2024-04-21T08:42:16.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.407+0000 7f245f9a3740 1 -- 172.21.15.43:0/617773592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f14a80 msgr2=0x560a56f12530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:16.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.407+0000 7f245f9a3740 1 --2- 172.21.15.43:0/617773592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f14a80 0x560a56f12530 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f245000f410 tx=0x7f2450009c40 comp rx=0 tx=0).stop 2024-04-21T08:42:16.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.407+0000 7f245f9a3740 1 -- 172.21.15.43:0/617773592 shutdown_connections 2024-04-21T08:42:16.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.407+0000 7f245f9a3740 1 --2- 172.21.15.43:0/617773592 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560a56f24a10 0x560a56f26e40 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.407+0000 7f245f9a3740 1 --2- 172.21.15.43:0/617773592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f14a80 0x560a56f12530 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.407+0000 7f245f9a3740 1 --2- 172.21.15.43:0/617773592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a56f0bea0 0x560a56f11ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.407+0000 7f243dffb640 0 client.17217 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:16.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.407+0000 7f245f9a3740 1 --2- 172.21.15.43:0/617773592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a56f0b580 0x560a56f11ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.407+0000 7f245f9a3740 1 -- 172.21.15.43:0/617773592 >> 172.21.15.43:0/617773592 conn(0x560a56f05cb0 msgr2=0x560a56f07a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:16.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.407+0000 7f245f9a3740 1 -- 172.21.15.43:0/617773592 shutdown_connections 2024-04-21T08:42:16.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.408+0000 7f245f9a3740 1 -- 172.21.15.43:0/617773592 wait complete. 2024-04-21T08:42:16.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.408+0000 7f245f9a3740 20 client.17217 trim_cache size 0 max 16384 2024-04-21T08:42:16.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.425+0000 7f245f9a3740 1 Processor -- start 2024-04-21T08:42:16.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.425+0000 7f245f9a3740 1 -- start start 2024-04-21T08:42:16.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.425+0000 7f245f9a3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a56f0b580 0x560a56f141e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.425+0000 7f245f9a3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a56f0bea0 0x560a56f206d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.425+0000 7f245f9a3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f14a80 0x560a56f04bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.425+0000 7f245f9a3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24581af330 con 0x560a56f14a80 2024-04-21T08:42:16.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.425+0000 7f245f9a3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24581b0e70 con 0x560a56f0b580 2024-04-21T08:42:16.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.425+0000 7f245f9a3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f24581afa00 con 0x560a56f0bea0 2024-04-21T08:42:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.425+0000 7f2456d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f14a80 0x560a56f04bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.425+0000 7f2456d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f14a80 0x560a56f04bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36994/0 (socket says 172.21.15.43:36994) 2024-04-21T08:42:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.425+0000 7f2456d9d640 1 -- 172.21.15.43:0/2585299336 learned_addr learned my addr 172.21.15.43:0/2585299336 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.426+0000 7f2456d9d640 1 -- 172.21.15.43:0/2585299336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a56f0bea0 msgr2=0x560a56f206d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.426+0000 7f2456d9d640 1 --2- 172.21.15.43:0/2585299336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a56f0bea0 0x560a56f206d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.426+0000 7f2456d9d640 1 -- 172.21.15.43:0/2585299336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a56f0b580 msgr2=0x560a56f141e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.426+0000 7f2456d9d640 1 --2- 172.21.15.43:0/2585299336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a56f0b580 0x560a56f141e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.426+0000 7f2456d9d640 1 -- 172.21.15.43:0/2585299336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a56f0bcd0 con 0x560a56f14a80 2024-04-21T08:42:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.426+0000 7f2456d9d640 1 --2- 172.21.15.43:0/2585299336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f14a80 0x560a56f04bf0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f2450009d20 tx=0x7f2450022df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a5d5ecc06978e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.426+0000 7f243ffff640 1 -- 172.21.15.43:0/2585299336 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24500225f0 con 0x560a56f14a80 2024-04-21T08:42:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.426+0000 7f243ffff640 1 -- 172.21.15.43:0/2585299336 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24500094a0 con 0x560a56f14a80 2024-04-21T08:42:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.426+0000 7f243ffff640 1 -- 172.21.15.43:0/2585299336 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245002da80 con 0x560a56f14a80 2024-04-21T08:42:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.426+0000 7f245f9a3740 1 -- 172.21.15.43:0/2585299336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f14a80 msgr2=0x560a56f04bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.426+0000 7f245f9a3740 1 --2- 172.21.15.43:0/2585299336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f14a80 0x560a56f04bf0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f2450009d20 tx=0x7f2450022df0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.426+0000 7f245f9a3740 1 -- 172.21.15.43:0/2585299336 shutdown_connections 2024-04-21T08:42:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.426+0000 7f245f9a3740 1 --2- 172.21.15.43:0/2585299336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f14a80 0x560a56f04bf0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.427+0000 7f245f9a3740 1 --2- 172.21.15.43:0/2585299336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a56f0bea0 0x560a56f206d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.427+0000 7f245f9a3740 1 --2- 172.21.15.43:0/2585299336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a56f0b580 0x560a56f141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.427+0000 7f245f9a3740 1 -- 172.21.15.43:0/2585299336 >> 172.21.15.43:0/2585299336 conn(0x560a56f05cb0 msgr2=0x560a56f044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.427+0000 7f245f9a3740 1 -- 172.21.15.43:0/2585299336 shutdown_connections 2024-04-21T08:42:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.427+0000 7f245f9a3740 1 -- 172.21.15.43:0/2585299336 wait complete. 2024-04-21T08:42:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.427+0000 7f245f9a3740 1 -- start start 2024-04-21T08:42:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.427+0000 7f245f9a3740 1 Processor -- start 2024-04-21T08:42:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.427+0000 7f245f9a3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a56f0b580 0x560a56f0e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.427+0000 7f245f9a3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a56f0bea0 0x560a56f0ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.427+0000 7f245f9a3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f14a80 0x560a56f0f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.427+0000 7f245f9a3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24581b07a0 con 0x560a56f14a80 2024-04-21T08:42:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.427+0000 7f245f9a3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24581b00d0 con 0x560a56f0b580 2024-04-21T08:42:16.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.427+0000 7f245f9a3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f24581ad310 con 0x560a56f0bea0 2024-04-21T08:42:16.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.427+0000 7f245f9a3740 10 client.0 _ll_register_callbacks cb 0x7f2450009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:16.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.427+0000 7f2456d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f14a80 0x560a56f0f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.427+0000 7f2456d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f14a80 0x560a56f0f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36996/0 (socket says 172.21.15.43:36996) 2024-04-21T08:42:16.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.427+0000 7f2456d9d640 1 -- 172.21.15.43:0/414104089 learned_addr learned my addr 172.21.15.43:0/414104089 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:16.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.428+0000 7f2456d9d640 1 --2- 172.21.15.43:0/414104089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a56f0b580 0x560a56f0e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.428+0000 7f245659c640 1 --2- 172.21.15.43:0/414104089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a56f0bea0 0x560a56f0ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.428+0000 7f2456d9d640 1 -- 172.21.15.43:0/414104089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a56f0bea0 msgr2=0x560a56f0ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:16.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.428+0000 7f2456d9d640 1 --2- 172.21.15.43:0/414104089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a56f0bea0 0x560a56f0ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.428+0000 7f2456d9d640 1 -- 172.21.15.43:0/414104089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a56f0b580 msgr2=0x560a56f0e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:16.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.428+0000 7f2456d9d640 1 --2- 172.21.15.43:0/414104089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a56f0b580 0x560a56f0e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.428+0000 7f2456d9d640 1 -- 172.21.15.43:0/414104089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a56f89f80 con 0x560a56f14a80 2024-04-21T08:42:16.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.428+0000 7f2456d9d640 1 --2- 172.21.15.43:0/414104089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f14a80 0x560a56f0f180 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f2450005fd0 tx=0x7f245002d6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836f5c897a8d6e9c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:16.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.428+0000 7f243dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:16.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.428+0000 7f243dffb640 1 -- 172.21.15.43:0/414104089 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245002d4c0 con 0x560a56f14a80 2024-04-21T08:42:16.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.428+0000 7f243dffb640 1 -- 172.21.15.43:0/414104089 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f24500223f0 con 0x560a56f14a80 2024-04-21T08:42:16.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.428+0000 7f245f9a3740 1 -- 172.21.15.43:0/414104089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24500094a0 con 0x560a56f14a80 2024-04-21T08:42:16.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.428+0000 7f243dffb640 1 -- 172.21.15.43:0/414104089 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2450022610 con 0x560a56f14a80 2024-04-21T08:42:16.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.428+0000 7f245f9a3740 10 client.17223 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:16.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.428+0000 7f245f9a3740 1 -- 172.21.15.43:0/414104089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560a56f8b1d0 con 0x560a56f14a80 2024-04-21T08:42:16.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.428+0000 7f243dffb640 1 -- 172.21.15.43:0/414104089 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24500094a0 con 0x560a56f14a80 2024-04-21T08:42:16.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.428+0000 7f243dffb640 1 -- 172.21.15.43:0/414104089 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2450038060 con 0x560a56f14a80 2024-04-21T08:42:16.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.428+0000 7f245f9a3740 10 client.17223 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:16.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.428+0000 7f245f9a3740 10 client.17223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:16.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.428+0000 7f245f9a3740 20 client.17223 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:16.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.429+0000 7f245f9a3740 1 --2- 172.21.15.43:0/414104089 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560a56f24a10 0x560a56f8d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:16.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.429+0000 7f245f9a3740 4 client.17223 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:42:16.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.429+0000 7f245f9a3740 1 -- 172.21.15.43:0/414104089 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560a56e77a70 con 0x560a56f24a10 2024-04-21T08:42:16.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.429+0000 7f245659c640 1 --2- 172.21.15.43:0/414104089 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560a56f24a10 0x560a56f8d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:16.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.429+0000 7f245659c640 1 --2- 172.21.15.43:0/414104089 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560a56f24a10 0x560a56f8d840 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e54f294bcb2cc06 server_cookie=89d8f3a55d667748 in_seq=0 out_seq=0 2024-04-21T08:42:16.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.429+0000 7f243dffb640 10 client.17223 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:16.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.429+0000 7f243dffb640 1 -- 172.21.15.43:0/414104089 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560a56e77a70 con 0x560a56f24a10 2024-04-21T08:42:16.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.429+0000 7f243dffb640 10 client.17223 handle_command_reply: tid=0 2024-04-21T08:42:16.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.429+0000 7f245f9a3740 1 client.17223 shutdown 2024-04-21T08:42:16.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.429+0000 7f245f9a3740 1 -- 172.21.15.43:0/414104089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f14a80 msgr2=0x560a56f0f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:16.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.429+0000 7f245f9a3740 1 --2- 172.21.15.43:0/414104089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f14a80 0x560a56f0f180 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f2450005fd0 tx=0x7f245002d6c0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.429+0000 7f245f9a3740 1 -- 172.21.15.43:0/414104089 shutdown_connections 2024-04-21T08:42:16.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.429+0000 7f245f9a3740 1 --2- 172.21.15.43:0/414104089 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560a56f24a10 0x560a56f8d840 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.429+0000 7f245f9a3740 1 --2- 172.21.15.43:0/414104089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a56f14a80 0x560a56f0f180 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.430+0000 7f243dffb640 0 client.17223 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:16.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.430+0000 7f245f9a3740 1 --2- 172.21.15.43:0/414104089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a56f0bea0 0x560a56f0ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.430+0000 7f245f9a3740 1 --2- 172.21.15.43:0/414104089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a56f0b580 0x560a56f0e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.430+0000 7f245f9a3740 1 -- 172.21.15.43:0/414104089 >> 172.21.15.43:0/414104089 conn(0x560a56f05cb0 msgr2=0x560a56f12300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:16.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.430+0000 7f245f9a3740 1 -- 172.21.15.43:0/414104089 shutdown_connections 2024-04-21T08:42:16.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.430+0000 7f245f9a3740 1 -- 172.21.15.43:0/414104089 wait complete. 2024-04-21T08:42:16.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.430+0000 7f245f9a3740 20 client.17223 trim_cache size 0 max 16384 2024-04-21T08:42:16.437 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:42:16.437 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:42:16.437 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-21T08:42:16.438 INFO:teuthology.orchestra.run.smithi043.stdout: "59e0dfdf-6ef4-4aae-a2b4-48edda687e1d": { 2024-04-21T08:42:16.438 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-21T08:42:16.438 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "59e0dfdf-6ef4-4aae-a2b4-48edda687e1d", 2024-04-21T08:42:16.438 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-21T08:42:16.438 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:42:16.438 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:42:16.438 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:42:16.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.430+0000 7f245ccd2640 1 -- 172.21.15.43:0/3441515330 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f24440a3d20 msgr2=0x7f24440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:16.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.430+0000 7f245ccd2640 1 --2- 172.21.15.43:0/3441515330 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f24440a3d20 0x7f24440a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f24400992a0 tx=0x7f24400bd040 comp rx=0 tx=0).stop 2024-04-21T08:42:16.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.430+0000 7f245ccd2640 1 -- 172.21.15.43:0/3441515330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2458101a40 msgr2=0x7f2458199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:16.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.430+0000 7f245ccd2640 1 --2- 172.21.15.43:0/3441515330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2458101a40 0x7f2458199aa0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f244000c920 tx=0x7f244000cde0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.430+0000 7f245ccd2640 1 -- 172.21.15.43:0/3441515330 shutdown_connections 2024-04-21T08:42:16.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.430+0000 7f245ccd2640 1 --2- 172.21.15.43:0/3441515330 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f24440a3d20 0x7f24440a61e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.430+0000 7f245ccd2640 1 --2- 172.21.15.43:0/3441515330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2458105dd0 0x7f24581a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.430+0000 7f245ccd2640 1 --2- 172.21.15.43:0/3441515330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2458101a40 0x7f2458199aa0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.430+0000 7f245ccd2640 1 --2- 172.21.15.43:0/3441515330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24581010a0 0x7f2458199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:16.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.430+0000 7f245ccd2640 1 -- 172.21.15.43:0/3441515330 >> 172.21.15.43:0/3441515330 conn(0x7f2458077c40 msgr2=0x7f24580fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:16.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.430+0000 7f245ccd2640 1 -- 172.21.15.43:0/3441515330 shutdown_connections 2024-04-21T08:42:16.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:16.430+0000 7f245ccd2640 1 -- 172.21.15.43:0/3441515330 wait complete. 2024-04-21T08:42:16.440 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:16 smithi043 ceph-mon[27978]: pgmap v103: 129 pgs: 129 active+clean; 3.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-04-21T08:42:16.441 INFO:tasks.cephfs.filesystem:scrub status for tag:59e0dfdf-6ef4-4aae-a2b4-48edda687e1d - {'path': '/', 'tag': '59e0dfdf-6ef4-4aae-a2b4-48edda687e1d', 'options': 'recursive,force'} 2024-04-21T08:42:16.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:16 smithi175 ceph-mon[31920]: pgmap v103: 129 pgs: 129 active+clean; 3.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-04-21T08:42:16.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:16 smithi112 ceph-mon[31085]: pgmap v103: 129 pgs: 129 active+clean; 3.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-04-21T08:42:18.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:18 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:42:18.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:18 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:42:18.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:18 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:42:18.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:18 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:42:18.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:18 smithi175 ceph-mon[31920]: pgmap v104: 129 pgs: 129 active+clean; 3.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 22 op/s 2024-04-21T08:42:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:18 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:42:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:18 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:42:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:18 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:42:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:18 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:42:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:18 smithi112 ceph-mon[31085]: pgmap v104: 129 pgs: 129 active+clean; 3.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 22 op/s 2024-04-21T08:42:18.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:18 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:42:18.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:18 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:42:18.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:18 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:42:18.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:18 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:42:18.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:18 smithi043 ceph-mon[27978]: pgmap v104: 129 pgs: 129 active+clean; 3.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 22 op/s 2024-04-21T08:42:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.243+0000 7f86fddb2640 1 -- 172.21.15.43:0/2666743571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f81031e0 msgr2=0x7f86f81035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.243+0000 7f86fddb2640 1 --2- 172.21.15.43:0/2666743571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f81031e0 0x7f86f81035c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009900 tx=0x7f86e802bd60 comp rx=0 tx=0).stop 2024-04-21T08:42:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.243+0000 7f86fddb2640 1 -- 172.21.15.43:0/2666743571 shutdown_connections 2024-04-21T08:42:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.243+0000 7f86fddb2640 1 --2- 172.21.15.43:0/2666743571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86f810c520 0x7f86f810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.243+0000 7f86fddb2640 1 --2- 172.21.15.43:0/2666743571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86f8103b00 0x7f86f810bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.243+0000 7f86fddb2640 1 --2- 172.21.15.43:0/2666743571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f81031e0 0x7f86f81035c0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.243+0000 7f86fddb2640 1 -- 172.21.15.43:0/2666743571 >> 172.21.15.43:0/2666743571 conn(0x7f86f8077fd0 msgr2=0x7f86f80783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.243+0000 7f86fddb2640 1 -- 172.21.15.43:0/2666743571 shutdown_connections 2024-04-21T08:42:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.243+0000 7f86fddb2640 1 -- 172.21.15.43:0/2666743571 wait complete. 2024-04-21T08:42:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.244+0000 7f86fddb2640 1 Processor -- start 2024-04-21T08:42:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.244+0000 7f86fddb2640 1 -- start start 2024-04-21T08:42:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.244+0000 7f86fddb2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86f81031e0 0x7f86f8198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.244+0000 7f86fddb2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86f8103b00 0x7f86f8198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.244+0000 7f86fddb2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f810c520 0x7f86f819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.244+0000 7f86fddb2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f86f81121b0 con 0x7f86f810c520 2024-04-21T08:42:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.244+0000 7f86fddb2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f86f8112030 con 0x7f86f8103b00 2024-04-21T08:42:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.244+0000 7f86fddb2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f86f8112330 con 0x7f86f81031e0 2024-04-21T08:42:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.244+0000 7f86f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f810c520 0x7f86f819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.244+0000 7f86f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f810c520 0x7f86f819cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47738/0 (socket says 172.21.15.43:47738) 2024-04-21T08:42:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.244+0000 7f86f7fff640 1 -- 172.21.15.43:0/1429746996 learned_addr learned my addr 172.21.15.43:0/1429746996 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.244+0000 7f86f6ffd640 1 --2- 172.21.15.43:0/1429746996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86f8103b00 0x7f86f8198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.245+0000 7f86f77fe640 1 --2- 172.21.15.43:0/1429746996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86f81031e0 0x7f86f8198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.245+0000 7f86f7fff640 1 -- 172.21.15.43:0/1429746996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86f81031e0 msgr2=0x7f86f8198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.245+0000 7f86f7fff640 1 --2- 172.21.15.43:0/1429746996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86f81031e0 0x7f86f8198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.245+0000 7f86f7fff640 1 -- 172.21.15.43:0/1429746996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86f8103b00 msgr2=0x7f86f8198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.245+0000 7f86f7fff640 1 --2- 172.21.15.43:0/1429746996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86f8103b00 0x7f86f8198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.245+0000 7f86f7fff640 1 -- 172.21.15.43:0/1429746996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f819d630 con 0x7f86f810c520 2024-04-21T08:42:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.245+0000 7f86f7fff640 1 --2- 172.21.15.43:0/1429746996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f810c520 0x7f86f819cf00 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f86f000eaa0 tx=0x7f86f000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.245+0000 7f86f4ff9640 1 -- 172.21.15.43:0/1429746996 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0004d20 con 0x7f86f810c520 2024-04-21T08:42:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.245+0000 7f86f4ff9640 1 -- 172.21.15.43:0/1429746996 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f0005020 con 0x7f86f810c520 2024-04-21T08:42:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.245+0000 7f86f4ff9640 1 -- 172.21.15.43:0/1429746996 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f00116b0 con 0x7f86f810c520 2024-04-21T08:42:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.245+0000 7f86fddb2640 1 -- 172.21.15.43:0/1429746996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f819d920 con 0x7f86f810c520 2024-04-21T08:42:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.245+0000 7f86fddb2640 1 -- 172.21.15.43:0/1429746996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f81a51b0 con 0x7f86f810c520 2024-04-21T08:42:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.247+0000 7f86f4ff9640 1 -- 172.21.15.43:0/1429746996 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f86f00051d0 con 0x7f86f810c520 2024-04-21T08:42:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.248+0000 7f87009a4740 1 Processor -- start 2024-04-21T08:42:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.248+0000 7f87009a4740 1 -- start start 2024-04-21T08:42:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.248+0000 7f87009a4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638608328d0 0x563860831620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.248+0000 7f87009a4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860831b60 0x563860831fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.248+0000 7f87009a4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56386083f4f0 0x5638608418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.248+0000 7f87009a4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5638607968d0 con 0x56386083f4f0 2024-04-21T08:42:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.248+0000 7f87009a4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563860733e80 con 0x563860831b60 2024-04-21T08:42:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.248+0000 7f87009a4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563860792890 con 0x5638608328d0 2024-04-21T08:42:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.248+0000 7f86f4ff9640 1 --2- 172.21.15.43:0/1429746996 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f86e00a3e40 0x7f86e00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.248+0000 7f86f6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56386083f4f0 0x5638608418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.248+0000 7f86f6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56386083f4f0 0x5638608418e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47754/0 (socket says 172.21.15.43:47754) 2024-04-21T08:42:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.248+0000 7f86f6ffd640 1 -- 172.21.15.43:0/736041099 learned_addr learned my addr 172.21.15.43:0/736041099 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.248+0000 7f86f77fe640 1 --2- 172.21.15.43:0/736041099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860831b60 0x563860831fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.248+0000 7f86f7fff640 1 --2- 172.21.15.43:0/1429746996 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f86e00a3e40 0x7f86e00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.249+0000 7f86f6ffd640 1 --2- 172.21.15.43:0/736041099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638608328d0 0x563860831620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.249+0000 7f86f4ff9640 1 -- 172.21.15.43:0/1429746996 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f86f00d0cd0 con 0x7f86f810c520 2024-04-21T08:42:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.249+0000 7f86f6ffd640 1 -- 172.21.15.43:0/736041099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638608328d0 msgr2=0x563860831620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.249+0000 7f86f6ffd640 1 --2- 172.21.15.43:0/736041099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638608328d0 0x563860831620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.249+0000 7f86f6ffd640 1 -- 172.21.15.43:0/736041099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860831b60 msgr2=0x563860831fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.249+0000 7f86f6ffd640 1 --2- 172.21.15.43:0/736041099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860831b60 0x563860831fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.249+0000 7f86f6ffd640 1 -- 172.21.15.43:0/736041099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563860755320 con 0x56386083f4f0 2024-04-21T08:42:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.249+0000 7f86f77fe640 1 --2- 172.21.15.43:0/736041099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860831b60 0x563860831fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:42:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.249+0000 7f86f6ffd640 1 --2- 172.21.15.43:0/736041099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56386083f4f0 0x5638608418e0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f86ec011560 tx=0x7f86ec00a810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1dd71873b1ef1e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.249+0000 7f86f7fff640 1 --2- 172.21.15.43:0/1429746996 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f86e00a3e40 0x7f86e00a6300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f86f00ce100 tx=0x7f86f0087040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.249+0000 7f86ddffb640 1 -- 172.21.15.43:0/736041099 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec005220 con 0x56386083f4f0 2024-04-21T08:42:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.249+0000 7f86ddffb640 1 -- 172.21.15.43:0/736041099 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86ec004c50 con 0x56386083f4f0 2024-04-21T08:42:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.249+0000 7f86ddffb640 1 -- 172.21.15.43:0/736041099 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec00fc10 con 0x56386083f4f0 2024-04-21T08:42:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.250+0000 7f87009a4740 1 -- 172.21.15.43:0/736041099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56386083f4f0 msgr2=0x5638608418e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.250+0000 7f87009a4740 1 --2- 172.21.15.43:0/736041099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56386083f4f0 0x5638608418e0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f86ec011560 tx=0x7f86ec00a810 comp rx=0 tx=0).stop 2024-04-21T08:42:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.250+0000 7f87009a4740 1 -- 172.21.15.43:0/736041099 shutdown_connections 2024-04-21T08:42:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.250+0000 7f87009a4740 1 --2- 172.21.15.43:0/736041099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56386083f4f0 0x5638608418e0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.250+0000 7f87009a4740 1 --2- 172.21.15.43:0/736041099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860831b60 0x563860831fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.250+0000 7f87009a4740 1 --2- 172.21.15.43:0/736041099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638608328d0 0x563860831620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.250+0000 7f87009a4740 1 -- 172.21.15.43:0/736041099 >> 172.21.15.43:0/736041099 conn(0x5638607f8c90 msgr2=0x56386082c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.250+0000 7f87009a4740 1 -- 172.21.15.43:0/736041099 shutdown_connections 2024-04-21T08:42:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.250+0000 7f87009a4740 1 -- 172.21.15.43:0/736041099 wait complete. 2024-04-21T08:42:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.250+0000 7f87009a4740 1 -- start start 2024-04-21T08:42:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.251+0000 7f87009a4740 1 Processor -- start 2024-04-21T08:42:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.251+0000 7f87009a4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563860831b60 0x56386083ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.251+0000 7f87009a4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638608328d0 0x56386083c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.251+0000 7f87009a4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56386083f4f0 0x56386083c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.251+0000 7f87009a4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563860842220 con 0x56386083f4f0 2024-04-21T08:42:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.251+0000 7f87009a4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56386071eeb0 con 0x5638608328d0 2024-04-21T08:42:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.251+0000 7f87009a4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5638608423a0 con 0x563860831b60 2024-04-21T08:42:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.251+0000 7f87009a4740 10 client.0 _ll_register_callbacks cb 0x56386074bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.251+0000 7f86f6ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563860831b60 0x56386083ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.251+0000 7f86f77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638608328d0 0x56386083c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.251+0000 7f86f6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56386083f4f0 0x56386083c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.251+0000 7f86f77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638608328d0 0x56386083c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:42868/0 (socket says 172.21.15.43:42868) 2024-04-21T08:42:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.251+0000 7f86f77fe640 1 -- 172.21.15.43:0/435140599 learned_addr learned my addr 172.21.15.43:0/435140599 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.251+0000 7f86f6ffd640 1 -- 172.21.15.43:0/435140599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563860831b60 msgr2=0x56386083ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.251+0000 7f86f6ffd640 1 --2- 172.21.15.43:0/435140599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563860831b60 0x56386083ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.251+0000 7f86f6ffd640 1 -- 172.21.15.43:0/435140599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638608328d0 msgr2=0x56386083c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.251+0000 7f86f6ffd640 1 --2- 172.21.15.43:0/435140599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638608328d0 0x56386083c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.251+0000 7f86f6ffd640 1 -- 172.21.15.43:0/435140599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638604a9210 con 0x56386083f4f0 2024-04-21T08:42:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.252+0000 7f86f6ffd640 1 --2- 172.21.15.43:0/435140599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56386083f4f0 0x56386083c8b0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f86ec011eb0 tx=0x7f86ec00a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78385ec418cb7437 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.252+0000 7f86d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.252+0000 7f86d6ffd640 1 -- 172.21.15.43:0/435140599 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec0048d0 con 0x56386083f4f0 2024-04-21T08:42:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.252+0000 7f87009a4740 1 -- 172.21.15.43:0/435140599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86ec004c50 con 0x56386083f4f0 2024-04-21T08:42:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.252+0000 7f86d6ffd640 1 -- 172.21.15.43:0/435140599 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f86ec005000 con 0x56386083f4f0 2024-04-21T08:42:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.252+0000 7f86d6ffd640 1 -- 172.21.15.43:0/435140599 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f86ec004a70 con 0x56386083f4f0 2024-04-21T08:42:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.252+0000 7f87009a4740 10 client.17235 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.252+0000 7f87009a4740 1 -- 172.21.15.43:0/435140599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563860575620 con 0x56386083f4f0 2024-04-21T08:42:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.252+0000 7f86d6ffd640 1 -- 172.21.15.43:0/435140599 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86ec004c50 con 0x56386083f4f0 2024-04-21T08:42:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.252+0000 7f86d6ffd640 1 -- 172.21.15.43:0/435140599 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f86ec027070 con 0x56386083f4f0 2024-04-21T08:42:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.252+0000 7f87009a4740 10 client.17235 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.252+0000 7f87009a4740 10 client.17235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.252+0000 7f87009a4740 20 client.17235 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.252+0000 7f87009a4740 1 --2- 172.21.15.43:0/435140599 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56386084f510 0x563860851900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.252+0000 7f87009a4740 4 client.17235 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.252+0000 7f87009a4740 1 -- 172.21.15.43:0/435140599 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638607aa880 con 0x56386084f510 2024-04-21T08:42:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.253+0000 7f86f77fe640 1 --2- 172.21.15.43:0/435140599 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56386084f510 0x563860851900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.253+0000 7f86d6ffd640 10 client.17235 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.253+0000 7f86f77fe640 1 --2- 172.21.15.43:0/435140599 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56386084f510 0x563860851900 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=917192f7a90b82c6 server_cookie=672356ee65b4de7b in_seq=0 out_seq=0 2024-04-21T08:42:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.254+0000 7f86d6ffd640 1 -- 172.21.15.43:0/435140599 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5638607aa880 con 0x56386084f510 2024-04-21T08:42:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.254+0000 7f86d6ffd640 10 client.17235 handle_command_reply: tid=0 2024-04-21T08:42:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.254+0000 7f87009a4740 1 client.17235 shutdown 2024-04-21T08:42:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.254+0000 7f87009a4740 1 -- 172.21.15.43:0/435140599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56386083f4f0 msgr2=0x56386083c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.254+0000 7f87009a4740 1 --2- 172.21.15.43:0/435140599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56386083f4f0 0x56386083c8b0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f86ec011eb0 tx=0x7f86ec00a2d0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.254+0000 7f87009a4740 1 -- 172.21.15.43:0/435140599 shutdown_connections 2024-04-21T08:42:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.254+0000 7f87009a4740 1 --2- 172.21.15.43:0/435140599 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56386084f510 0x563860851900 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.254+0000 7f87009a4740 1 --2- 172.21.15.43:0/435140599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56386083f4f0 0x56386083c8b0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.254+0000 7f86d6ffd640 0 client.17235 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.254+0000 7f87009a4740 1 --2- 172.21.15.43:0/435140599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638608328d0 0x56386083c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.254+0000 7f87009a4740 1 --2- 172.21.15.43:0/435140599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563860831b60 0x56386083ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.254+0000 7f87009a4740 1 -- 172.21.15.43:0/435140599 >> 172.21.15.43:0/435140599 conn(0x5638607f8c90 msgr2=0x56386082e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.254+0000 7f87009a4740 1 -- 172.21.15.43:0/435140599 shutdown_connections 2024-04-21T08:42:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.254+0000 7f87009a4740 1 -- 172.21.15.43:0/435140599 wait complete. 2024-04-21T08:42:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.254+0000 7f87009a4740 20 client.17235 trim_cache size 0 max 16384 2024-04-21T08:42:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.269+0000 7f87009a4740 1 Processor -- start 2024-04-21T08:42:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.269+0000 7f87009a4740 1 -- start start 2024-04-21T08:42:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.270+0000 7f87009a4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563860831b60 0x56386084b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.270+0000 7f87009a4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638608328d0 0x56386083e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.270+0000 7f87009a4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56386083f4f0 0x56386083ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.270+0000 7f87009a4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f86f81ad230 con 0x56386083f4f0 2024-04-21T08:42:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.270+0000 7f87009a4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f86f81aed70 con 0x5638608328d0 2024-04-21T08:42:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.270+0000 7f87009a4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f86f81ad900 con 0x563860831b60 2024-04-21T08:42:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.270+0000 7f86f6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56386083f4f0 0x56386083ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.270+0000 7f86f6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56386083f4f0 0x56386083ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47780/0 (socket says 172.21.15.43:47780) 2024-04-21T08:42:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.270+0000 7f86f6ffd640 1 -- 172.21.15.43:0/2043291035 learned_addr learned my addr 172.21.15.43:0/2043291035 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.270+0000 7f86f77fe640 1 --2- 172.21.15.43:0/2043291035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638608328d0 0x56386083e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.270+0000 7f86f6ffd640 1 --2- 172.21.15.43:0/2043291035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563860831b60 0x56386084b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.270+0000 7f86f6ffd640 1 -- 172.21.15.43:0/2043291035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563860831b60 msgr2=0x56386084b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.270+0000 7f86f6ffd640 1 --2- 172.21.15.43:0/2043291035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563860831b60 0x56386084b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.270+0000 7f86f6ffd640 1 -- 172.21.15.43:0/2043291035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638608328d0 msgr2=0x56386083e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.270+0000 7f86f6ffd640 1 --2- 172.21.15.43:0/2043291035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638608328d0 0x56386083e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.270+0000 7f86f6ffd640 1 -- 172.21.15.43:0/2043291035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563860830e30 con 0x56386083f4f0 2024-04-21T08:42:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.270+0000 7f86f6ffd640 1 --2- 172.21.15.43:0/2043291035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56386083f4f0 0x56386083ef10 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00a990 tx=0x7f86ec00a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37781013f8ef9dfa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.270+0000 7f86dd7fa640 1 -- 172.21.15.43:0/2043291035 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec0048d0 con 0x56386083f4f0 2024-04-21T08:42:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.270+0000 7f86dd7fa640 1 -- 172.21.15.43:0/2043291035 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86ec00a630 con 0x56386083f4f0 2024-04-21T08:42:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.270+0000 7f86dd7fa640 1 -- 172.21.15.43:0/2043291035 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec004fc0 con 0x56386083f4f0 2024-04-21T08:42:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.271+0000 7f87009a4740 1 -- 172.21.15.43:0/2043291035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56386083f4f0 msgr2=0x56386083ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.271+0000 7f87009a4740 1 --2- 172.21.15.43:0/2043291035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56386083f4f0 0x56386083ef10 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00a990 tx=0x7f86ec00a400 comp rx=0 tx=0).stop 2024-04-21T08:42:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.271+0000 7f87009a4740 1 -- 172.21.15.43:0/2043291035 shutdown_connections 2024-04-21T08:42:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.271+0000 7f87009a4740 1 --2- 172.21.15.43:0/2043291035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56386083f4f0 0x56386083ef10 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.271+0000 7f87009a4740 1 --2- 172.21.15.43:0/2043291035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638608328d0 0x56386083e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.271+0000 7f87009a4740 1 --2- 172.21.15.43:0/2043291035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563860831b60 0x56386084b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.271+0000 7f87009a4740 1 -- 172.21.15.43:0/2043291035 >> 172.21.15.43:0/2043291035 conn(0x5638607f8c90 msgr2=0x56386082c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.271+0000 7f87009a4740 1 -- 172.21.15.43:0/2043291035 shutdown_connections 2024-04-21T08:42:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.271+0000 7f87009a4740 1 -- 172.21.15.43:0/2043291035 wait complete. 2024-04-21T08:42:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.271+0000 7f87009a4740 1 -- start start 2024-04-21T08:42:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.272+0000 7f87009a4740 1 Processor -- start 2024-04-21T08:42:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.272+0000 7f87009a4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860831b60 0x56386083f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.272+0000 7f87009a4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638608328d0 0x56386083feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.272+0000 7f87009a4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56386083f4f0 0x563860846630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.272+0000 7f87009a4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f86f81ae6a0 con 0x5638608328d0 2024-04-21T08:42:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.272+0000 7f87009a4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f86f81adfd0 con 0x563860831b60 2024-04-21T08:42:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.272+0000 7f87009a4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f86f81ab210 con 0x56386083f4f0 2024-04-21T08:42:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.272+0000 7f87009a4740 10 client.0 _ll_register_callbacks cb 0x7f86e8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.272+0000 7f86f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638608328d0 0x56386083feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.272+0000 7f86f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638608328d0 0x56386083feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47794/0 (socket says 172.21.15.43:47794) 2024-04-21T08:42:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.272+0000 7f86f77fe640 1 -- 172.21.15.43:0/2735480652 learned_addr learned my addr 172.21.15.43:0/2735480652 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.272+0000 7f86f6ffd640 1 --2- 172.21.15.43:0/2735480652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860831b60 0x56386083f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.272+0000 7f86f6ffd640 1 --2- 172.21.15.43:0/2735480652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56386083f4f0 0x563860846630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.272+0000 7f86f77fe640 1 -- 172.21.15.43:0/2735480652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56386083f4f0 msgr2=0x563860846630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.272+0000 7f86f77fe640 1 --2- 172.21.15.43:0/2735480652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56386083f4f0 0x563860846630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.272+0000 7f86f77fe640 1 -- 172.21.15.43:0/2735480652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860831b60 msgr2=0x56386083f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.272+0000 7f86f77fe640 1 --2- 172.21.15.43:0/2735480652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860831b60 0x56386083f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.272+0000 7f86f77fe640 1 -- 172.21.15.43:0/2735480652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563860840740 con 0x5638608328d0 2024-04-21T08:42:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.272+0000 7f86f6ffd640 1 --2- 172.21.15.43:0/2735480652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56386083f4f0 0x563860846630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.273+0000 7f86f77fe640 1 --2- 172.21.15.43:0/2735480652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638608328d0 0x56386083feb0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f86e8005ab0 tx=0x7f86e80043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0d742ffc5ae6a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.273+0000 7f86d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.273+0000 7f86d6ffd640 1 -- 172.21.15.43:0/2735480652 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e804f060 con 0x5638608328d0 2024-04-21T08:42:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.273+0000 7f86d6ffd640 1 -- 172.21.15.43:0/2735480652 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f86e802cd20 con 0x5638608328d0 2024-04-21T08:42:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.273+0000 7f87009a4740 1 -- 172.21.15.43:0/2735480652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86ec00a630 con 0x5638608328d0 2024-04-21T08:42:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.273+0000 7f86d6ffd640 1 -- 172.21.15.43:0/2735480652 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f86e8056050 con 0x5638608328d0 2024-04-21T08:42:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.273+0000 7f87009a4740 10 client.17241 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.273+0000 7f87009a4740 1 -- 172.21.15.43:0/2735480652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563860847d40 con 0x5638608328d0 2024-04-21T08:42:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.273+0000 7f86d6ffd640 1 -- 172.21.15.43:0/2735480652 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86ec00a630 con 0x5638608328d0 2024-04-21T08:42:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.273+0000 7f86d6ffd640 1 -- 172.21.15.43:0/2735480652 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f86e8049070 con 0x5638608328d0 2024-04-21T08:42:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.273+0000 7f87009a4740 10 client.17241 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.273+0000 7f87009a4740 10 client.17241 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.273+0000 7f87009a4740 20 client.17241 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.273+0000 7f87009a4740 1 --2- 172.21.15.43:0/2735480652 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5638608b5be0 0x5638608b8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.273+0000 7f87009a4740 4 client.17241 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:42:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.273+0000 7f87009a4740 1 -- 172.21.15.43:0/2735480652 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5638607363d0 con 0x5638608b5be0 2024-04-21T08:42:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.274+0000 7f86f6ffd640 1 --2- 172.21.15.43:0/2735480652 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5638608b5be0 0x5638608b8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.274+0000 7f86f6ffd640 1 --2- 172.21.15.43:0/2735480652 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5638608b5be0 0x5638608b8000 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=373e410ed34a374d server_cookie=bf941f452cd46c4b in_seq=0 out_seq=0 2024-04-21T08:42:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.274+0000 7f86d6ffd640 10 client.17241 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.288+0000 7f86d6ffd640 1 -- 172.21.15.43:0/2735480652 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5638607363d0 con 0x5638608b5be0 2024-04-21T08:42:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.288+0000 7f86d6ffd640 10 client.17241 handle_command_reply: tid=0 2024-04-21T08:42:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.289+0000 7f87009a4740 1 client.17241 shutdown 2024-04-21T08:42:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.289+0000 7f87009a4740 1 -- 172.21.15.43:0/2735480652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638608328d0 msgr2=0x56386083feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.289+0000 7f87009a4740 1 --2- 172.21.15.43:0/2735480652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638608328d0 0x56386083feb0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f86e8005ab0 tx=0x7f86e80043e0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.289+0000 7f87009a4740 1 -- 172.21.15.43:0/2735480652 shutdown_connections 2024-04-21T08:42:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.289+0000 7f87009a4740 1 --2- 172.21.15.43:0/2735480652 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5638608b5be0 0x5638608b8000 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.289+0000 7f87009a4740 1 --2- 172.21.15.43:0/2735480652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56386083f4f0 0x563860846630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.289+0000 7f87009a4740 1 --2- 172.21.15.43:0/2735480652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638608328d0 0x56386083feb0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.289+0000 7f86d6ffd640 0 client.17241 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.289+0000 7f87009a4740 1 --2- 172.21.15.43:0/2735480652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563860831b60 0x56386083f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.289+0000 7f87009a4740 1 -- 172.21.15.43:0/2735480652 >> 172.21.15.43:0/2735480652 conn(0x5638607f8c90 msgr2=0x56386083aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.289+0000 7f87009a4740 1 -- 172.21.15.43:0/2735480652 shutdown_connections 2024-04-21T08:42:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.289+0000 7f87009a4740 1 -- 172.21.15.43:0/2735480652 wait complete. 2024-04-21T08:42:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.289+0000 7f87009a4740 20 client.17241 trim_cache size 0 max 16384 2024-04-21T08:42:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:42:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:42:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:42:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:42:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.290+0000 7f86fddb2640 1 -- 172.21.15.43:0/1429746996 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f86e00a3e40 msgr2=0x7f86e00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.290+0000 7f86fddb2640 1 --2- 172.21.15.43:0/1429746996 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f86e00a3e40 0x7f86e00a6300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f86f00ce100 tx=0x7f86f0087040 comp rx=0 tx=0).stop 2024-04-21T08:42:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.290+0000 7f86fddb2640 1 -- 172.21.15.43:0/1429746996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f810c520 msgr2=0x7f86f819cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.290+0000 7f86fddb2640 1 --2- 172.21.15.43:0/1429746996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f810c520 0x7f86f819cf00 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f86f000eaa0 tx=0x7f86f000eea0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.290+0000 7f86fddb2640 1 -- 172.21.15.43:0/1429746996 shutdown_connections 2024-04-21T08:42:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.290+0000 7f86fddb2640 1 --2- 172.21.15.43:0/1429746996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f810c520 0x7f86f819cf00 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.290+0000 7f86fddb2640 1 --2- 172.21.15.43:0/1429746996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86f8103b00 0x7f86f8198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.290+0000 7f86fddb2640 1 --2- 172.21.15.43:0/1429746996 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f86e00a3e40 0x7f86e00a6300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.290+0000 7f86fddb2640 1 --2- 172.21.15.43:0/1429746996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86f81031e0 0x7f86f8198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.290+0000 7f86fddb2640 1 -- 172.21.15.43:0/1429746996 >> 172.21.15.43:0/1429746996 conn(0x7f86f8077fd0 msgr2=0x7f86f80fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.290+0000 7f86fddb2640 1 -- 172.21.15.43:0/1429746996 shutdown_connections 2024-04-21T08:42:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:19.290+0000 7f86fddb2640 1 -- 172.21.15.43:0/1429746996 wait complete. 2024-04-21T08:42:20.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:20 smithi175 ceph-mon[31920]: pgmap v105: 129 pgs: 129 active+clean; 3.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-21T08:42:20.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:20 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:42:20.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:20 smithi112 ceph-mon[31085]: pgmap v105: 129 pgs: 129 active+clean; 3.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-21T08:42:20.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:20 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:42:20.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:20 smithi043 ceph-mon[27978]: pgmap v105: 129 pgs: 129 active+clean; 3.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-21T08:42:20.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:20 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:42:22.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:22 smithi175 ceph-mon[31920]: pgmap v106: 129 pgs: 129 active+clean; 3.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 25 op/s 2024-04-21T08:42:22.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:22 smithi112 ceph-mon[31085]: pgmap v106: 129 pgs: 129 active+clean; 3.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 25 op/s 2024-04-21T08:42:22.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:22 smithi043 ceph-mon[27978]: pgmap v106: 129 pgs: 129 active+clean; 3.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 25 op/s 2024-04-21T08:42:24.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:24 smithi175 ceph-mon[31920]: pgmap v107: 129 pgs: 129 active+clean; 3.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 16 op/s 2024-04-21T08:42:24.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:24 smithi112 ceph-mon[31085]: pgmap v107: 129 pgs: 129 active+clean; 3.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 16 op/s 2024-04-21T08:42:24.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:24 smithi043 ceph-mon[27978]: pgmap v107: 129 pgs: 129 active+clean; 3.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 16 op/s 2024-04-21T08:42:26.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:26 smithi175 ceph-mon[31920]: pgmap v108: 129 pgs: 129 active+clean; 3.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 16 op/s 2024-04-21T08:42:26.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:26 smithi112 ceph-mon[31085]: pgmap v108: 129 pgs: 129 active+clean; 3.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 16 op/s 2024-04-21T08:42:26.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:26 smithi043 ceph-mon[27978]: pgmap v108: 129 pgs: 129 active+clean; 3.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 16 op/s 2024-04-21T08:42:28.151 INFO:tasks.workunit.client.0.smithi043.stdout:64/600: open fd = 3 2024-04-21T08:42:28.151 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.151 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.151 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.151 INFO:tasks.workunit.client.0.smithi043.stdout:65/600: open fd = 3 2024-04-21T08:42:28.151 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.151 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.151 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.151 INFO:tasks.workunit.client.0.smithi043.stdout:66/600: open fd = 3 2024-04-21T08:42:28.151 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.151 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.152 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.152 INFO:tasks.workunit.client.0.smithi043.stdout:67/600: open fd = 3 2024-04-21T08:42:28.152 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.152 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.152 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.152 INFO:tasks.workunit.client.0.smithi043.stdout:68/600: open fd = 3 2024-04-21T08:42:28.152 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.152 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.152 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.152 INFO:tasks.workunit.client.0.smithi043.stdout:69/600: open fd = 3 2024-04-21T08:42:28.152 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.152 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.152 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.152 INFO:tasks.workunit.client.0.smithi043.stdout:70/600: open fd = 3 2024-04-21T08:42:28.153 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.153 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.153 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.153 INFO:tasks.workunit.client.0.smithi043.stdout:71/600: open fd = 3 2024-04-21T08:42:28.153 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.153 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.153 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.153 INFO:tasks.workunit.client.0.smithi043.stdout:72/600: open fd = 3 2024-04-21T08:42:28.153 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.153 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.153 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.153 INFO:tasks.workunit.client.0.smithi043.stdout:73/600: open fd = 3 2024-04-21T08:42:28.154 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.154 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.154 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.154 INFO:tasks.workunit.client.0.smithi043.stdout:74/600: open fd = 3 2024-04-21T08:42:28.154 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.154 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.154 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.154 INFO:tasks.workunit.client.0.smithi043.stdout:75/600: open fd = 3 2024-04-21T08:42:28.154 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.154 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.154 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.154 INFO:tasks.workunit.client.0.smithi043.stdout:76/600: open fd = 3 2024-04-21T08:42:28.154 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.154 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.155 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.155 INFO:tasks.workunit.client.0.smithi043.stdout:77/600: open fd = 3 2024-04-21T08:42:28.155 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.155 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.155 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.155 INFO:tasks.workunit.client.0.smithi043.stdout:78/600: open fd = 3 2024-04-21T08:42:28.155 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.155 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.155 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.155 INFO:tasks.workunit.client.0.smithi043.stdout:79/600: open fd = 3 2024-04-21T08:42:28.155 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.155 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.155 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.155 INFO:tasks.workunit.client.0.smithi043.stdout:80/600: open fd = 3 2024-04-21T08:42:28.156 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.156 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.156 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.156 INFO:tasks.workunit.client.0.smithi043.stdout:81/600: open fd = 3 2024-04-21T08:42:28.156 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.156 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.156 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.156 INFO:tasks.workunit.client.0.smithi043.stdout:82/600: open fd = 3 2024-04-21T08:42:28.156 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.156 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.156 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.156 INFO:tasks.workunit.client.0.smithi043.stdout:83/600: open fd = 3 2024-04-21T08:42:28.156 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.156 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.157 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.157 INFO:tasks.workunit.client.0.smithi043.stdout:84/600: open fd = 3 2024-04-21T08:42:28.157 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.157 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.157 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.157 INFO:tasks.workunit.client.0.smithi043.stdout:85/600: open fd = 3 2024-04-21T08:42:28.157 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.157 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.157 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.157 INFO:tasks.workunit.client.0.smithi043.stdout:86/600: open fd = 3 2024-04-21T08:42:28.157 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.157 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.157 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.157 INFO:tasks.workunit.client.0.smithi043.stdout:87/600: open fd = 3 2024-04-21T08:42:28.158 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.158 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.158 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.158 INFO:tasks.workunit.client.0.smithi043.stdout:88/600: open fd = 3 2024-04-21T08:42:28.158 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.158 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.158 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.158 INFO:tasks.workunit.client.0.smithi043.stdout:89/600: open fd = 3 2024-04-21T08:42:28.158 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.158 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.158 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.158 INFO:tasks.workunit.client.0.smithi043.stdout:90/600: open fd = 3 2024-04-21T08:42:28.158 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.158 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.159 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.159 INFO:tasks.workunit.client.0.smithi043.stdout:91/600: open fd = 3 2024-04-21T08:42:28.159 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.159 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.159 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.159 INFO:tasks.workunit.client.0.smithi043.stdout:92/600: open fd = 3 2024-04-21T08:42:28.159 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.159 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.159 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.159 INFO:tasks.workunit.client.0.smithi043.stdout:93/600: open fd = 3 2024-04-21T08:42:28.159 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.159 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.159 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.159 INFO:tasks.workunit.client.0.smithi043.stdout:94/600: open fd = 3 2024-04-21T08:42:28.160 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.160 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.160 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.160 INFO:tasks.workunit.client.0.smithi043.stdout:95/600: open fd = 3 2024-04-21T08:42:28.160 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.160 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.160 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.160 INFO:tasks.workunit.client.0.smithi043.stdout:96/600: open fd = 3 2024-04-21T08:42:28.160 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.160 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.160 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.160 INFO:tasks.workunit.client.0.smithi043.stdout:97/600: open fd = 3 2024-04-21T08:42:28.160 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.160 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.160 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.161 INFO:tasks.workunit.client.0.smithi043.stdout:98/600: open fd = 3 2024-04-21T08:42:28.161 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.161 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.161 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.161 INFO:tasks.workunit.client.0.smithi043.stdout:99/600: open fd = 3 2024-04-21T08:42:28.161 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.161 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.161 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.161 INFO:tasks.workunit.client.0.smithi043.stdout:100/600: open fd = 3 2024-04-21T08:42:28.161 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.161 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.161 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.161 INFO:tasks.workunit.client.0.smithi043.stdout:101/600: open fd = 3 2024-04-21T08:42:28.161 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.161 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.162 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.162 INFO:tasks.workunit.client.0.smithi043.stdout:102/600: open fd = 3 2024-04-21T08:42:28.162 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.162 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.162 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.162 INFO:tasks.workunit.client.0.smithi043.stdout:103/600: open fd = 3 2024-04-21T08:42:28.162 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.162 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.162 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.162 INFO:tasks.workunit.client.0.smithi043.stdout:104/600: open fd = 3 2024-04-21T08:42:28.162 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.162 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.162 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.162 INFO:tasks.workunit.client.0.smithi043.stdout:105/600: open fd = 3 2024-04-21T08:42:28.163 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.163 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.163 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.163 INFO:tasks.workunit.client.0.smithi043.stdout:106/600: open fd = 3 2024-04-21T08:42:28.163 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.163 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.163 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.163 INFO:tasks.workunit.client.0.smithi043.stdout:107/600: open fd = 3 2024-04-21T08:42:28.163 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.163 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.163 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.163 INFO:tasks.workunit.client.0.smithi043.stdout:108/600: open fd = 3 2024-04-21T08:42:28.163 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.163 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.163 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.163 INFO:tasks.workunit.client.0.smithi043.stdout:109/600: open fd = 3 2024-04-21T08:42:28.163 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.164 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.164 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.164 INFO:tasks.workunit.client.0.smithi043.stdout:110/600: open fd = 3 2024-04-21T08:42:28.164 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.164 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.164 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.164 INFO:tasks.workunit.client.0.smithi043.stdout:111/600: open fd = 3 2024-04-21T08:42:28.164 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.164 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.164 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.164 INFO:tasks.workunit.client.0.smithi043.stdout:112/600: open fd = 3 2024-04-21T08:42:28.164 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.164 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.164 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.164 INFO:tasks.workunit.client.0.smithi043.stdout:113/600: open fd = 3 2024-04-21T08:42:28.165 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.165 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.165 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.165 INFO:tasks.workunit.client.0.smithi043.stdout:114/600: open fd = 3 2024-04-21T08:42:28.165 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.165 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.165 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.165 INFO:tasks.workunit.client.0.smithi043.stdout:115/600: open fd = 3 2024-04-21T08:42:28.165 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.165 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.165 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.165 INFO:tasks.workunit.client.0.smithi043.stdout:116/600: open fd = 3 2024-04-21T08:42:28.165 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.165 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.166 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.166 INFO:tasks.workunit.client.0.smithi043.stdout:117/600: open fd = 3 2024-04-21T08:42:28.166 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.166 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.166 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.166 INFO:tasks.workunit.client.0.smithi043.stdout:118/600: open fd = 3 2024-04-21T08:42:28.166 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.166 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.166 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.166 INFO:tasks.workunit.client.0.smithi043.stdout:119/600: open fd = 3 2024-04-21T08:42:28.166 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.166 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.166 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.166 INFO:tasks.workunit.client.0.smithi043.stdout:120/600: open fd = 3 2024-04-21T08:42:28.167 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.167 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.167 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.167 INFO:tasks.workunit.client.0.smithi043.stdout:121/600: open fd = 3 2024-04-21T08:42:28.167 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.167 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.167 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.167 INFO:tasks.workunit.client.0.smithi043.stdout:122/600: open fd = 3 2024-04-21T08:42:28.167 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.167 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.167 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.167 INFO:tasks.workunit.client.0.smithi043.stdout:123/600: open fd = 3 2024-04-21T08:42:28.167 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.167 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.167 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.168 INFO:tasks.workunit.client.0.smithi043.stdout:124/600: open fd = 3 2024-04-21T08:42:28.168 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.168 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.168 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.168 INFO:tasks.workunit.client.0.smithi043.stdout:125/600: open fd = 3 2024-04-21T08:42:28.168 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.168 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.168 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:42:28.168 INFO:tasks.workunit.client.0.smithi043.stdout:126/600: open fd = 3 2024-04-21T08:42:28.168 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:42:28.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:28 smithi175 ceph-mon[31920]: pgmap v109: 129 pgs: 129 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 23 op/s 2024-04-21T08:42:28.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:28 smithi112 ceph-mon[31085]: pgmap v109: 129 pgs: 129 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 23 op/s 2024-04-21T08:42:28.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:28 smithi043 ceph-mon[27978]: pgmap v109: 129 pgs: 129 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 23 op/s 2024-04-21T08:42:30.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:30 smithi175 ceph-mon[31920]: pgmap v110: 129 pgs: 129 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 16 op/s 2024-04-21T08:42:30.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:30 smithi112 ceph-mon[31085]: pgmap v110: 129 pgs: 129 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 16 op/s 2024-04-21T08:42:30.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:30 smithi043 ceph-mon[27978]: pgmap v110: 129 pgs: 129 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 16 op/s 2024-04-21T08:42:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.390+0000 7f1a584d2640 1 -- 172.21.15.43:0/4128496026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a501021d0 msgr2=0x7f1a501025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.390+0000 7f1a584d2640 1 --2- 172.21.15.43:0/4128496026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a501021d0 0x7f1a501025b0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009900 tx=0x7f1a4c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:42:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.390+0000 7f1a584d2640 1 -- 172.21.15.43:0/4128496026 shutdown_connections 2024-04-21T08:42:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.390+0000 7f1a584d2640 1 --2- 172.21.15.43:0/4128496026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a501072a0 0x7f1a5010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.390+0000 7f1a584d2640 1 --2- 172.21.15.43:0/4128496026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a50102b80 0x7f1a50106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.390+0000 7f1a584d2640 1 --2- 172.21.15.43:0/4128496026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a501021d0 0x7f1a501025b0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.390+0000 7f1a584d2640 1 -- 172.21.15.43:0/4128496026 >> 172.21.15.43:0/4128496026 conn(0x7f1a500793e0 msgr2=0x7f1a500797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.390+0000 7f1a584d2640 1 -- 172.21.15.43:0/4128496026 shutdown_connections 2024-04-21T08:42:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.390+0000 7f1a584d2640 1 -- 172.21.15.43:0/4128496026 wait complete. 2024-04-21T08:42:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.390+0000 7f1a584d2640 1 Processor -- start 2024-04-21T08:42:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.390+0000 7f1a584d2640 1 -- start start 2024-04-21T08:42:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.391+0000 7f1a584d2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a501021d0 0x7f1a501986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.391+0000 7f1a584d2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a501072a0 0x7f1a50198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.391+0000 7f1a584d2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a5019cfa0 0x7f1a5019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.391+0000 7f1a584d2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a50112200 con 0x7f1a5019cfa0 2024-04-21T08:42:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.391+0000 7f1a584d2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a50112080 con 0x7f1a501072a0 2024-04-21T08:42:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.391+0000 7f1a584d2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1a50112380 con 0x7f1a501021d0 2024-04-21T08:42:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.391+0000 7f1a56a6f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a5019cfa0 0x7f1a5019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.391+0000 7f1a56a6f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a5019cfa0 0x7f1a5019d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56912/0 (socket says 172.21.15.43:56912) 2024-04-21T08:42:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.391+0000 7f1a56a6f640 1 -- 172.21.15.43:0/1754406357 learned_addr learned my addr 172.21.15.43:0/1754406357 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.391+0000 7f1a5626e640 1 --2- 172.21.15.43:0/1754406357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a501021d0 0x7f1a501986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.391+0000 7f1a55a6d640 1 --2- 172.21.15.43:0/1754406357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a501072a0 0x7f1a50198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.391+0000 7f1a56a6f640 1 -- 172.21.15.43:0/1754406357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a501021d0 msgr2=0x7f1a501986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.391+0000 7f1a56a6f640 1 --2- 172.21.15.43:0/1754406357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a501021d0 0x7f1a501986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.391+0000 7f1a56a6f640 1 -- 172.21.15.43:0/1754406357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a501072a0 msgr2=0x7f1a50198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.391+0000 7f1a56a6f640 1 --2- 172.21.15.43:0/1754406357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a501072a0 0x7f1a50198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.391+0000 7f1a56a6f640 1 -- 172.21.15.43:0/1754406357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5019da90 con 0x7f1a5019cfa0 2024-04-21T08:42:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.391+0000 7f1a56a6f640 1 --2- 172.21.15.43:0/1754406357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a5019cfa0 0x7f1a5019d450 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800da40 tx=0x7f1a4800df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.392+0000 7f1a477fe640 1 -- 172.21.15.43:0/1754406357 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a48017070 con 0x7f1a5019cfa0 2024-04-21T08:42:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.392+0000 7f1a584d2640 1 -- 172.21.15.43:0/1754406357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a5019dd80 con 0x7f1a5019cfa0 2024-04-21T08:42:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.392+0000 7f1a477fe640 1 -- 172.21.15.43:0/1754406357 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a48003e20 con 0x7f1a5019cfa0 2024-04-21T08:42:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.392+0000 7f1a477fe640 1 -- 172.21.15.43:0/1754406357 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a48010460 con 0x7f1a5019cfa0 2024-04-21T08:42:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.392+0000 7f1a584d2640 1 -- 172.21.15.43:0/1754406357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a501a5520 con 0x7f1a5019cfa0 2024-04-21T08:42:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.394+0000 7f1a477fe640 1 -- 172.21.15.43:0/1754406357 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1a48004020 con 0x7f1a5019cfa0 2024-04-21T08:42:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.394+0000 7f1a5b1a6740 1 Processor -- start 2024-04-21T08:42:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.395+0000 7f1a5b1a6740 1 -- start start 2024-04-21T08:42:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.395+0000 7f1a5b1a6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dcd8d0 0x555cd4dcc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.395+0000 7f1a5b1a6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cd4dccb60 0x555cd4dccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.395+0000 7f1a5b1a6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cd4dda4f0 0x555cd4ddc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.395+0000 7f1a5b1a6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555cd4d318d0 con 0x555cd4dcd8d0 2024-04-21T08:42:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.395+0000 7f1a5b1a6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555cd4ccee80 con 0x555cd4dccb60 2024-04-21T08:42:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.395+0000 7f1a5b1a6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555cd4d2d890 con 0x555cd4dda4f0 2024-04-21T08:42:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.395+0000 7f1a477fe640 1 --2- 172.21.15.43:0/1754406357 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1a3c0a3dc0 0x7f1a3c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.395+0000 7f1a55a6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dcd8d0 0x555cd4dcc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.395+0000 7f1a55a6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dcd8d0 0x555cd4dcc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56916/0 (socket says 172.21.15.43:56916) 2024-04-21T08:42:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.395+0000 7f1a55a6d640 1 -- 172.21.15.43:0/2709135768 learned_addr learned my addr 172.21.15.43:0/2709135768 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.395+0000 7f1a5626e640 1 --2- 172.21.15.43:0/2709135768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cd4dccb60 0x555cd4dccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.395+0000 7f1a56a6f640 1 --2- 172.21.15.43:0/1754406357 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1a3c0a3dc0 0x7f1a3c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.395+0000 7f1a55a6d640 1 --2- 172.21.15.43:0/2709135768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cd4dda4f0 0x555cd4ddc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.395+0000 7f1a477fe640 1 -- 172.21.15.43:0/1754406357 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f1a480d07d0 con 0x7f1a5019cfa0 2024-04-21T08:42:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.395+0000 7f1a55a6d640 1 -- 172.21.15.43:0/2709135768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cd4dda4f0 msgr2=0x555cd4ddc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.395+0000 7f1a55a6d640 1 --2- 172.21.15.43:0/2709135768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cd4dda4f0 0x555cd4ddc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.395+0000 7f1a55a6d640 1 -- 172.21.15.43:0/2709135768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cd4dccb60 msgr2=0x555cd4dccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.395+0000 7f1a55a6d640 1 --2- 172.21.15.43:0/2709135768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cd4dccb60 0x555cd4dccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.396+0000 7f1a55a6d640 1 -- 172.21.15.43:0/2709135768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cd4cf0320 con 0x555cd4dcd8d0 2024-04-21T08:42:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.396+0000 7f1a55a6d640 1 --2- 172.21.15.43:0/2709135768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dcd8d0 0x555cd4dcc620 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000afd0 tx=0x7f1a4000a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780f9712aed227e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.396+0000 7f1a56a6f640 1 --2- 172.21.15.43:0/1754406357 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1a3c0a3dc0 0x7f1a3c0a6280 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1a4809c0f0 tx=0x7f1a4806de30 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.396+0000 7f1a44bfe640 1 -- 172.21.15.43:0/2709135768 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40019070 con 0x555cd4dcd8d0 2024-04-21T08:42:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.396+0000 7f1a44bfe640 1 -- 172.21.15.43:0/2709135768 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a400054d0 con 0x555cd4dcd8d0 2024-04-21T08:42:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.396+0000 7f1a44bfe640 1 -- 172.21.15.43:0/2709135768 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40004c30 con 0x555cd4dcd8d0 2024-04-21T08:42:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.397+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/2709135768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dcd8d0 msgr2=0x555cd4dcc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.397+0000 7f1a5b1a6740 1 --2- 172.21.15.43:0/2709135768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dcd8d0 0x555cd4dcc620 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000afd0 tx=0x7f1a4000a740 comp rx=0 tx=0).stop 2024-04-21T08:42:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.397+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/2709135768 shutdown_connections 2024-04-21T08:42:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.397+0000 7f1a5b1a6740 1 --2- 172.21.15.43:0/2709135768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cd4dda4f0 0x555cd4ddc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.397+0000 7f1a5b1a6740 1 --2- 172.21.15.43:0/2709135768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cd4dccb60 0x555cd4dccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.397+0000 7f1a5b1a6740 1 --2- 172.21.15.43:0/2709135768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dcd8d0 0x555cd4dcc620 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.397+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/2709135768 >> 172.21.15.43:0/2709135768 conn(0x555cd4d93c90 msgr2=0x555cd4dc7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.397+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/2709135768 shutdown_connections 2024-04-21T08:42:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.397+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/2709135768 wait complete. 2024-04-21T08:42:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.397+0000 7f1a5b1a6740 1 -- start start 2024-04-21T08:42:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.397+0000 7f1a5b1a6740 1 Processor -- start 2024-04-21T08:42:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.398+0000 7f1a5b1a6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cd4dccb60 0x555cd4dd9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.398+0000 7f1a5b1a6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cd4dcd8d0 0x555cd4dd71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.398+0000 7f1a5b1a6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dda4f0 0x555cd4dd7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.398+0000 7f1a5b1a6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555cd4ddd220 con 0x555cd4dda4f0 2024-04-21T08:42:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.398+0000 7f1a5b1a6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555cd4cb9eb0 con 0x555cd4dcd8d0 2024-04-21T08:42:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.398+0000 7f1a5b1a6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555cd4ddd3a0 con 0x555cd4dccb60 2024-04-21T08:42:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.398+0000 7f1a5b1a6740 10 client.0 _ll_register_callbacks cb 0x555cd4ce6b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.398+0000 7f1a55a6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dda4f0 0x555cd4dd7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.398+0000 7f1a55a6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dda4f0 0x555cd4dd7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56924/0 (socket says 172.21.15.43:56924) 2024-04-21T08:42:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.398+0000 7f1a55a6d640 1 -- 172.21.15.43:0/3351130058 learned_addr learned my addr 172.21.15.43:0/3351130058 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.398+0000 7f1a5626e640 1 --2- 172.21.15.43:0/3351130058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cd4dcd8d0 0x555cd4dd71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.398+0000 7f1a55a6d640 1 --2- 172.21.15.43:0/3351130058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cd4dccb60 0x555cd4dd9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.398+0000 7f1a55a6d640 1 -- 172.21.15.43:0/3351130058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cd4dccb60 msgr2=0x555cd4dd9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.398+0000 7f1a55a6d640 1 --2- 172.21.15.43:0/3351130058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cd4dccb60 0x555cd4dd9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.398+0000 7f1a55a6d640 1 -- 172.21.15.43:0/3351130058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cd4dcd8d0 msgr2=0x555cd4dd71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.398+0000 7f1a55a6d640 1 --2- 172.21.15.43:0/3351130058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cd4dcd8d0 0x555cd4dd71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.398+0000 7f1a55a6d640 1 -- 172.21.15.43:0/3351130058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cd4a44210 con 0x555cd4dda4f0 2024-04-21T08:42:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.398+0000 7f1a55a6d640 1 --2- 172.21.15.43:0/3351130058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dda4f0 0x555cd4dd7730 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1a40005e70 tx=0x7f1a40005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc847795077f42c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.398+0000 7f1a39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.398+0000 7f1a39ffb640 1 -- 172.21.15.43:0/3351130058 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40019040 con 0x555cd4dda4f0 2024-04-21T08:42:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.399+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/3351130058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a400054d0 con 0x555cd4dda4f0 2024-04-21T08:42:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.399+0000 7f1a39ffb640 1 -- 172.21.15.43:0/3351130058 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f1a40005250 con 0x555cd4dda4f0 2024-04-21T08:42:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.399+0000 7f1a39ffb640 1 -- 172.21.15.43:0/3351130058 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1a400116b0 con 0x555cd4dda4f0 2024-04-21T08:42:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.399+0000 7f1a5b1a6740 10 client.17253 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.399+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/3351130058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555cd4b10620 con 0x555cd4dda4f0 2024-04-21T08:42:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.399+0000 7f1a39ffb640 1 -- 172.21.15.43:0/3351130058 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a400054d0 con 0x555cd4dda4f0 2024-04-21T08:42:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.399+0000 7f1a39ffb640 1 -- 172.21.15.43:0/3351130058 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1a4002b620 con 0x555cd4dda4f0 2024-04-21T08:42:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.399+0000 7f1a5b1a6740 10 client.17253 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.399+0000 7f1a5b1a6740 10 client.17253 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.399+0000 7f1a5b1a6740 20 client.17253 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.399+0000 7f1a5b1a6740 1 --2- 172.21.15.43:0/3351130058 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555cd4dea4c0 0x555cd4dec8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.399+0000 7f1a5b1a6740 4 client.17253 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.399+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/3351130058 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cd4d45880 con 0x555cd4dea4c0 2024-04-21T08:42:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.399+0000 7f1a5626e640 1 --2- 172.21.15.43:0/3351130058 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555cd4dea4c0 0x555cd4dec8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.400+0000 7f1a5626e640 1 --2- 172.21.15.43:0/3351130058 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555cd4dea4c0 0x555cd4dec8b0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79a5ca03004efb3 server_cookie=27e0673d5bb2bad5 in_seq=0 out_seq=0 2024-04-21T08:42:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.400+0000 7f1a39ffb640 10 client.17253 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:32.406 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.401+0000 7f1a39ffb640 1 -- 172.21.15.43:0/3351130058 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555cd4d45880 con 0x555cd4dea4c0 2024-04-21T08:42:32.406 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.401+0000 7f1a39ffb640 10 client.17253 handle_command_reply: tid=0 2024-04-21T08:42:32.406 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.401+0000 7f1a5b1a6740 1 client.17253 shutdown 2024-04-21T08:42:32.406 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.401+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/3351130058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dda4f0 msgr2=0x555cd4dd7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.401+0000 7f1a5b1a6740 1 --2- 172.21.15.43:0/3351130058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dda4f0 0x555cd4dd7730 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1a40005e70 tx=0x7f1a40005da0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.401+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/3351130058 shutdown_connections 2024-04-21T08:42:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.401+0000 7f1a5b1a6740 1 --2- 172.21.15.43:0/3351130058 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555cd4dea4c0 0x555cd4dec8b0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.401+0000 7f1a5b1a6740 1 --2- 172.21.15.43:0/3351130058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dda4f0 0x555cd4dd7730 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.401+0000 7f1a5b1a6740 1 --2- 172.21.15.43:0/3351130058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cd4dcd8d0 0x555cd4dd71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.401+0000 7f1a39ffb640 0 client.17253 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.401+0000 7f1a5b1a6740 1 --2- 172.21.15.43:0/3351130058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cd4dccb60 0x555cd4dd9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.401+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/3351130058 >> 172.21.15.43:0/3351130058 conn(0x555cd4d93c90 msgr2=0x555cd4dc95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.401+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/3351130058 shutdown_connections 2024-04-21T08:42:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.401+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/3351130058 wait complete. 2024-04-21T08:42:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.401+0000 7f1a5b1a6740 20 client.17253 trim_cache size 0 max 16384 2024-04-21T08:42:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.416+0000 7f1a5b1a6740 1 Processor -- start 2024-04-21T08:42:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.416+0000 7f1a5b1a6740 1 -- start start 2024-04-21T08:42:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.416+0000 7f1a5b1a6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cd4dccb60 0x555cd4de6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.416+0000 7f1a5b1a6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cd4dcd8d0 0x555cd4dcc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.416+0000 7f1a5b1a6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dda4f0 0x555cd4dd9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.416+0000 7f1a5b1a6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a501ad0c0 con 0x555cd4dda4f0 2024-04-21T08:42:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.416+0000 7f1a5b1a6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a501aec00 con 0x555cd4dcd8d0 2024-04-21T08:42:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.416+0000 7f1a5b1a6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1a501ad790 con 0x555cd4dccb60 2024-04-21T08:42:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.416+0000 7f1a55a6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dda4f0 0x555cd4dd9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.416+0000 7f1a55a6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dda4f0 0x555cd4dd9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56930/0 (socket says 172.21.15.43:56930) 2024-04-21T08:42:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.416+0000 7f1a55a6d640 1 -- 172.21.15.43:0/4039184475 learned_addr learned my addr 172.21.15.43:0/4039184475 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.417+0000 7f1a5626e640 1 --2- 172.21.15.43:0/4039184475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cd4dcd8d0 0x555cd4dcc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.417+0000 7f1a55a6d640 1 --2- 172.21.15.43:0/4039184475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cd4dccb60 0x555cd4de6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.417+0000 7f1a55a6d640 1 -- 172.21.15.43:0/4039184475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cd4dccb60 msgr2=0x555cd4de6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.417+0000 7f1a55a6d640 1 --2- 172.21.15.43:0/4039184475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cd4dccb60 0x555cd4de6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.417+0000 7f1a55a6d640 1 -- 172.21.15.43:0/4039184475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cd4dcd8d0 msgr2=0x555cd4dcc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.417+0000 7f1a55a6d640 1 --2- 172.21.15.43:0/4039184475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cd4dcd8d0 0x555cd4dcc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.417+0000 7f1a55a6d640 1 -- 172.21.15.43:0/4039184475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cd4dcbe30 con 0x555cd4dda4f0 2024-04-21T08:42:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.417+0000 7f1a5626e640 1 --2- 172.21.15.43:0/4039184475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cd4dcd8d0 0x555cd4dcc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.417+0000 7f1a55a6d640 1 --2- 172.21.15.43:0/4039184475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dda4f0 0x555cd4dd9900 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f1a400058c0 tx=0x7f1a4000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3b5d92600ec746 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.417+0000 7f1a3bfff640 1 -- 172.21.15.43:0/4039184475 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40019070 con 0x555cd4dda4f0 2024-04-21T08:42:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.417+0000 7f1a3bfff640 1 -- 172.21.15.43:0/4039184475 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a40004d30 con 0x555cd4dda4f0 2024-04-21T08:42:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.417+0000 7f1a3bfff640 1 -- 172.21.15.43:0/4039184475 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4002b680 con 0x555cd4dda4f0 2024-04-21T08:42:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.418+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/4039184475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dda4f0 msgr2=0x555cd4dd9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.418+0000 7f1a5b1a6740 1 --2- 172.21.15.43:0/4039184475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dda4f0 0x555cd4dd9900 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f1a400058c0 tx=0x7f1a4000a340 comp rx=0 tx=0).stop 2024-04-21T08:42:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.418+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/4039184475 shutdown_connections 2024-04-21T08:42:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.418+0000 7f1a5b1a6740 1 --2- 172.21.15.43:0/4039184475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dda4f0 0x555cd4dd9900 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.418+0000 7f1a5b1a6740 1 --2- 172.21.15.43:0/4039184475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cd4dcd8d0 0x555cd4dcc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.418+0000 7f1a5b1a6740 1 --2- 172.21.15.43:0/4039184475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cd4dccb60 0x555cd4de6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.418+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/4039184475 >> 172.21.15.43:0/4039184475 conn(0x555cd4d93c90 msgr2=0x555cd4dd5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.418+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/4039184475 shutdown_connections 2024-04-21T08:42:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.418+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/4039184475 wait complete. 2024-04-21T08:42:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.418+0000 7f1a5b1a6740 1 -- start start 2024-04-21T08:42:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.418+0000 7f1a5b1a6740 1 Processor -- start 2024-04-21T08:42:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.418+0000 7f1a5b1a6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dccb60 0x555cd4de2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.418+0000 7f1a5b1a6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cd4dcd8d0 0x555cd4ddde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.418+0000 7f1a5b1a6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cd4dda4f0 0x555cd4dde410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.418+0000 7f1a5b1a6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a501ae530 con 0x555cd4dccb60 2024-04-21T08:42:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.418+0000 7f1a5b1a6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a501ade60 con 0x555cd4dcd8d0 2024-04-21T08:42:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.418+0000 7f1a5b1a6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1a501ab0a0 con 0x555cd4dda4f0 2024-04-21T08:42:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.418+0000 7f1a5b1a6740 10 client.0 _ll_register_callbacks cb 0x7f1a4c005e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.418+0000 7f1a55a6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dccb60 0x555cd4de2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.418+0000 7f1a55a6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dccb60 0x555cd4de2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56932/0 (socket says 172.21.15.43:56932) 2024-04-21T08:42:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.418+0000 7f1a55a6d640 1 -- 172.21.15.43:0/796147715 learned_addr learned my addr 172.21.15.43:0/796147715 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:32.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.419+0000 7f1a5626e640 1 --2- 172.21.15.43:0/796147715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cd4dcd8d0 0x555cd4ddde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:32.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.419+0000 7f1a55a6d640 1 -- 172.21.15.43:0/796147715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cd4dda4f0 msgr2=0x555cd4dde410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:32.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.419+0000 7f1a55a6d640 1 --2- 172.21.15.43:0/796147715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cd4dda4f0 0x555cd4dde410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.419+0000 7f1a55a6d640 1 -- 172.21.15.43:0/796147715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cd4dcd8d0 msgr2=0x555cd4ddde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:32.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.419+0000 7f1a55a6d640 1 --2- 172.21.15.43:0/796147715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cd4dcd8d0 0x555cd4ddde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.419+0000 7f1a55a6d640 1 -- 172.21.15.43:0/796147715 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cd4dc2f30 con 0x555cd4dccb60 2024-04-21T08:42:32.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.419+0000 7f1a55a6d640 1 --2- 172.21.15.43:0/796147715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dccb60 0x555cd4de2ab0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f1a40004d00 tx=0x7f1a4000ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d926240bc97bfeb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:32.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.419+0000 7f1a39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:32.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.419+0000 7f1a39ffb640 1 -- 172.21.15.43:0/796147715 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40019070 con 0x555cd4dccb60 2024-04-21T08:42:32.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.419+0000 7f1a39ffb640 1 -- 172.21.15.43:0/796147715 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f1a4002b420 con 0x555cd4dccb60 2024-04-21T08:42:32.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.419+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/796147715 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a40004d30 con 0x555cd4dccb60 2024-04-21T08:42:32.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.419+0000 7f1a39ffb640 1 -- 172.21.15.43:0/796147715 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1a4002bdc0 con 0x555cd4dccb60 2024-04-21T08:42:32.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.419+0000 7f1a5b1a6740 10 client.17259 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:32.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.419+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/796147715 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555cd4e505c0 con 0x555cd4dccb60 2024-04-21T08:42:32.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.420+0000 7f1a39ffb640 1 -- 172.21.15.43:0/796147715 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a40004d30 con 0x555cd4dccb60 2024-04-21T08:42:32.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.420+0000 7f1a39ffb640 1 -- 172.21.15.43:0/796147715 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1a40011b50 con 0x555cd4dccb60 2024-04-21T08:42:32.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.420+0000 7f1a5b1a6740 10 client.17259 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:32.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.420+0000 7f1a5b1a6740 10 client.17259 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:32.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.420+0000 7f1a5b1a6740 20 client.17259 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:32.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.420+0000 7f1a5b1a6740 1 --2- 172.21.15.43:0/796147715 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555cd4e50c20 0x555cd4e53010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:32.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.420+0000 7f1a5b1a6740 4 client.17259 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:42:32.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.420+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/796147715 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555cd4cd13d0 con 0x555cd4e50c20 2024-04-21T08:42:32.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.420+0000 7f1a5626e640 1 --2- 172.21.15.43:0/796147715 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555cd4e50c20 0x555cd4e53010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:32.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.420+0000 7f1a5626e640 1 --2- 172.21.15.43:0/796147715 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555cd4e50c20 0x555cd4e53010 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ee636df3ca1a30c server_cookie=7e08b1a5e16d1582 in_seq=0 out_seq=0 2024-04-21T08:42:32.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.420+0000 7f1a39ffb640 10 client.17259 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a39ffb640 1 -- 172.21.15.43:0/796147715 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555cd4cd13d0 con 0x555cd4e50c20 2024-04-21T08:42:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a39ffb640 10 client.17259 handle_command_reply: tid=0 2024-04-21T08:42:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a5b1a6740 1 client.17259 shutdown 2024-04-21T08:42:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/796147715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dccb60 msgr2=0x555cd4de2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a5b1a6740 1 --2- 172.21.15.43:0/796147715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dccb60 0x555cd4de2ab0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f1a40004d00 tx=0x7f1a4000ad50 comp rx=0 tx=0).stop 2024-04-21T08:42:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/796147715 shutdown_connections 2024-04-21T08:42:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a5b1a6740 1 --2- 172.21.15.43:0/796147715 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555cd4e50c20 0x555cd4e53010 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a5b1a6740 1 --2- 172.21.15.43:0/796147715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555cd4dda4f0 0x555cd4dde410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a5b1a6740 1 --2- 172.21.15.43:0/796147715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555cd4dcd8d0 0x555cd4ddde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a39ffb640 0 client.17259 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a5b1a6740 1 --2- 172.21.15.43:0/796147715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555cd4dccb60 0x555cd4de2ab0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/796147715 >> 172.21.15.43:0/796147715 conn(0x555cd4d93c90 msgr2=0x555cd4dc6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/796147715 shutdown_connections 2024-04-21T08:42:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a5b1a6740 1 -- 172.21.15.43:0/796147715 wait complete. 2024-04-21T08:42:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a5b1a6740 20 client.17259 trim_cache size 0 max 16384 2024-04-21T08:42:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:42:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:42:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:42:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:42:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a584d2640 1 -- 172.21.15.43:0/1754406357 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1a3c0a3dc0 msgr2=0x7f1a3c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a584d2640 1 --2- 172.21.15.43:0/1754406357 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1a3c0a3dc0 0x7f1a3c0a6280 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1a4809c0f0 tx=0x7f1a4806de30 comp rx=0 tx=0).stop 2024-04-21T08:42:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a584d2640 1 -- 172.21.15.43:0/1754406357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a5019cfa0 msgr2=0x7f1a5019d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a584d2640 1 --2- 172.21.15.43:0/1754406357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a5019cfa0 0x7f1a5019d450 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800da40 tx=0x7f1a4800df00 comp rx=0 tx=0).stop 2024-04-21T08:42:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a584d2640 1 -- 172.21.15.43:0/1754406357 shutdown_connections 2024-04-21T08:42:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a584d2640 1 --2- 172.21.15.43:0/1754406357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a5019cfa0 0x7f1a5019d450 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a584d2640 1 --2- 172.21.15.43:0/1754406357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a501072a0 0x7f1a50198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a584d2640 1 --2- 172.21.15.43:0/1754406357 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1a3c0a3dc0 0x7f1a3c0a6280 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a584d2640 1 --2- 172.21.15.43:0/1754406357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a501021d0 0x7f1a501986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.437+0000 7f1a584d2640 1 -- 172.21.15.43:0/1754406357 >> 172.21.15.43:0/1754406357 conn(0x7f1a500793e0 msgr2=0x7f1a500ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.438+0000 7f1a584d2640 1 -- 172.21.15.43:0/1754406357 shutdown_connections 2024-04-21T08:42:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:32.438+0000 7f1a584d2640 1 -- 172.21.15.43:0/1754406357 wait complete. 2024-04-21T08:42:32.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:32 smithi175 ceph-mon[31920]: pgmap v111: 129 pgs: 129 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 23 op/s 2024-04-21T08:42:32.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:32 smithi112 ceph-mon[31085]: pgmap v111: 129 pgs: 129 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 23 op/s 2024-04-21T08:42:32.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:32 smithi043 ceph-mon[27978]: pgmap v111: 129 pgs: 129 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 23 op/s 2024-04-21T08:42:34.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:34 smithi175 ceph-mon[31920]: pgmap v112: 129 pgs: 129 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-04-21T08:42:34.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:34 smithi112 ceph-mon[31085]: pgmap v112: 129 pgs: 129 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-04-21T08:42:34.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:34 smithi043 ceph-mon[27978]: pgmap v112: 129 pgs: 129 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-04-21T08:42:36.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:36 smithi175 ceph-mon[31920]: pgmap v113: 129 pgs: 129 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-04-21T08:42:36.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:36 smithi112 ceph-mon[31085]: pgmap v113: 129 pgs: 129 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-04-21T08:42:36.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:36 smithi043 ceph-mon[27978]: pgmap v113: 129 pgs: 129 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-04-21T08:42:38.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:38 smithi175 ceph-mon[31920]: pgmap v114: 129 pgs: 129 active+clean; 4.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 22 op/s 2024-04-21T08:42:38.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:38 smithi112 ceph-mon[31085]: pgmap v114: 129 pgs: 129 active+clean; 4.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 22 op/s 2024-04-21T08:42:38.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:38 smithi043 ceph-mon[27978]: pgmap v114: 129 pgs: 129 active+clean; 4.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 22 op/s 2024-04-21T08:42:40.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:40 smithi175 ceph-mon[31920]: pgmap v115: 129 pgs: 129 active+clean; 4.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-04-21T08:42:40.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:40 smithi112 ceph-mon[31085]: pgmap v115: 129 pgs: 129 active+clean; 4.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-04-21T08:42:40.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:40 smithi043 ceph-mon[27978]: pgmap v115: 129 pgs: 129 active+clean; 4.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-04-21T08:42:42.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:42 smithi175 ceph-mon[31920]: pgmap v116: 129 pgs: 129 active+clean; 4.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 22 op/s 2024-04-21T08:42:42.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:42 smithi112 ceph-mon[31085]: pgmap v116: 129 pgs: 129 active+clean; 4.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 22 op/s 2024-04-21T08:42:42.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:42 smithi043 ceph-mon[27978]: pgmap v116: 129 pgs: 129 active+clean; 4.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 22 op/s 2024-04-21T08:42:44.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:44 smithi175 ceph-mon[31920]: pgmap v117: 129 pgs: 129 active+clean; 4.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-04-21T08:42:44.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:44 smithi112 ceph-mon[31085]: pgmap v117: 129 pgs: 129 active+clean; 4.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-04-21T08:42:44.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:44 smithi043 ceph-mon[27978]: pgmap v117: 129 pgs: 129 active+clean; 4.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-04-21T08:42:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.537+0000 7fd23d1a4640 1 -- 172.21.15.43:0/162653216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd238101a90 msgr2=0x7fd238105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.537+0000 7fd23d1a4640 1 --2- 172.21.15.43:0/162653216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd238101a90 0x7fd238105ad0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fd220009900 tx=0x7fd22002bc30 comp rx=0 tx=0).stop 2024-04-21T08:42:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.538+0000 7fd23d1a4640 1 -- 172.21.15.43:0/162653216 shutdown_connections 2024-04-21T08:42:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.538+0000 7fd23d1a4640 1 --2- 172.21.15.43:0/162653216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2381061b0 0x7fd23810dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.538+0000 7fd23d1a4640 1 --2- 172.21.15.43:0/162653216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd238101a90 0x7fd238105ad0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.538+0000 7fd23d1a4640 1 --2- 172.21.15.43:0/162653216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2381010e0 0x7fd2381014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.538+0000 7fd23d1a4640 1 -- 172.21.15.43:0/162653216 >> 172.21.15.43:0/162653216 conn(0x7fd2380771d0 msgr2=0x7fd2380775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.538+0000 7fd23d1a4640 1 -- 172.21.15.43:0/162653216 shutdown_connections 2024-04-21T08:42:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.538+0000 7fd23d1a4640 1 -- 172.21.15.43:0/162653216 wait complete. 2024-04-21T08:42:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.538+0000 7fd23d1a4640 1 Processor -- start 2024-04-21T08:42:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.539+0000 7fd23d1a4640 1 -- start start 2024-04-21T08:42:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.539+0000 7fd23d1a4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2381010e0 0x7fd238198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.539+0000 7fd23d1a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2381061b0 0x7fd238198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.539+0000 7fd23d1a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd23819cf00 0x7fd23819d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.539+0000 7fd23d1a4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd2381110f0 con 0x7fd23819cf00 2024-04-21T08:42:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.539+0000 7fd23d1a4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd238110f70 con 0x7fd2381061b0 2024-04-21T08:42:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.539+0000 7fd23d1a4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd238111270 con 0x7fd2381010e0 2024-04-21T08:42:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.539+0000 7fd23759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd23819cf00 0x7fd23819d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.539+0000 7fd23759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd23819cf00 0x7fd23819d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44416/0 (socket says 172.21.15.43:44416) 2024-04-21T08:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.539+0000 7fd23759e640 1 -- 172.21.15.43:0/2417390379 learned_addr learned my addr 172.21.15.43:0/2417390379 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.539+0000 7fd22e59c640 1 --2- 172.21.15.43:0/2417390379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2381061b0 0x7fd238198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.539+0000 7fd236d9d640 1 --2- 172.21.15.43:0/2417390379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2381010e0 0x7fd238198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.540+0000 7fd23759e640 1 -- 172.21.15.43:0/2417390379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2381010e0 msgr2=0x7fd238198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.540+0000 7fd23759e640 1 --2- 172.21.15.43:0/2417390379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2381010e0 0x7fd238198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.540+0000 7fd23759e640 1 -- 172.21.15.43:0/2417390379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2381061b0 msgr2=0x7fd238198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.540+0000 7fd23759e640 1 --2- 172.21.15.43:0/2417390379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2381061b0 0x7fd238198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.540+0000 7fd23759e640 1 -- 172.21.15.43:0/2417390379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd23819d9f0 con 0x7fd23819cf00 2024-04-21T08:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.540+0000 7fd23759e640 1 --2- 172.21.15.43:0/2417390379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd23819cf00 0x7fd23819d3b0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd23000eb10 tx=0x7fd23000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.540+0000 7fd234d99640 1 -- 172.21.15.43:0/2417390379 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd230004b20 con 0x7fd23819cf00 2024-04-21T08:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.540+0000 7fd234d99640 1 -- 172.21.15.43:0/2417390379 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd230004cc0 con 0x7fd23819cf00 2024-04-21T08:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.540+0000 7fd23d1a4640 1 -- 172.21.15.43:0/2417390379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd23819dce0 con 0x7fd23819cf00 2024-04-21T08:42:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.540+0000 7fd234d99640 1 -- 172.21.15.43:0/2417390379 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd230010670 con 0x7fd23819cf00 2024-04-21T08:42:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.540+0000 7fd23d1a4640 1 -- 172.21.15.43:0/2417390379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2381a54a0 con 0x7fd23819cf00 2024-04-21T08:42:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.542+0000 7fd234d99640 1 -- 172.21.15.43:0/2417390379 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd230005020 con 0x7fd23819cf00 2024-04-21T08:42:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.543+0000 7fd23fd97740 1 Processor -- start 2024-04-21T08:42:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.543+0000 7fd23fd97740 1 -- start start 2024-04-21T08:42:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.543+0000 7fd234d99640 1 --2- 172.21.15.43:0/2417390379 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd2240a3dc0 0x7fd2240a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.543+0000 7fd23fd97740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcaa414ce0 0x55dcaa413a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.543+0000 7fd23fd97740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa413f70 0x55dcaa4143f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.543+0000 7fd23fd97740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcaa41d4f0 0x55dcaa41f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.543+0000 7fd23fd97740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dcaa3748d0 con 0x55dcaa413f70 2024-04-21T08:42:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.543+0000 7fd23fd97740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dcaa311e80 con 0x55dcaa41d4f0 2024-04-21T08:42:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.543+0000 7fd23fd97740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dcaa370890 con 0x55dcaa414ce0 2024-04-21T08:42:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.543+0000 7fd22e59c640 1 --2- 172.21.15.43:0/2417390379 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd2240a3dc0 0x7fd2240a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.544+0000 7fd23759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcaa41d4f0 0x55dcaa41f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.544+0000 7fd22e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa413f70 0x55dcaa4143f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.544+0000 7fd23759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcaa41d4f0 0x55dcaa41f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:34344/0 (socket says 172.21.15.43:34344) 2024-04-21T08:42:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.544+0000 7fd23759e640 1 -- 172.21.15.43:0/1652351246 learned_addr learned my addr 172.21.15.43:0/1652351246 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.544+0000 7fd234d99640 1 -- 172.21.15.43:0/2417390379 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd2300d10b0 con 0x7fd23819cf00 2024-04-21T08:42:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.544+0000 7fd236d9d640 1 --2- 172.21.15.43:0/1652351246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcaa414ce0 0x55dcaa413a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.544+0000 7fd23759e640 1 -- 172.21.15.43:0/1652351246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcaa414ce0 msgr2=0x55dcaa413a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.544+0000 7fd23759e640 1 --2- 172.21.15.43:0/1652351246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcaa414ce0 0x55dcaa413a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.544+0000 7fd23759e640 1 -- 172.21.15.43:0/1652351246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa413f70 msgr2=0x55dcaa4143f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.544+0000 7fd23759e640 1 --2- 172.21.15.43:0/1652351246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa413f70 0x55dcaa4143f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.544+0000 7fd23759e640 1 -- 172.21.15.43:0/1652351246 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcaa333320 con 0x55dcaa41d4f0 2024-04-21T08:42:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.544+0000 7fd236d9d640 1 --2- 172.21.15.43:0/1652351246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcaa414ce0 0x55dcaa413a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.544+0000 7fd22e59c640 1 --2- 172.21.15.43:0/2417390379 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd2240a3dc0 0x7fd2240a6280 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd220009230 tx=0x7fd22002c6c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.544+0000 7fd23759e640 1 --2- 172.21.15.43:0/1652351246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcaa41d4f0 0x55dcaa41f8e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd2300049d0 tx=0x7fd230000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=315031adca9b1c2e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.544+0000 7fd22d59a640 1 -- 172.21.15.43:0/1652351246 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23009c5e0 con 0x55dcaa41d4f0 2024-04-21T08:42:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.544+0000 7fd22d59a640 1 -- 172.21.15.43:0/1652351246 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd23009d380 con 0x55dcaa41d4f0 2024-04-21T08:42:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.544+0000 7fd22d59a640 1 -- 172.21.15.43:0/1652351246 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2300b6400 con 0x55dcaa41d4f0 2024-04-21T08:42:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.545+0000 7fd23fd97740 1 -- 172.21.15.43:0/1652351246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcaa41d4f0 msgr2=0x55dcaa41f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.545+0000 7fd23fd97740 1 --2- 172.21.15.43:0/1652351246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcaa41d4f0 0x55dcaa41f8e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd2300049d0 tx=0x7fd230000f40 comp rx=0 tx=0).stop 2024-04-21T08:42:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.545+0000 7fd23fd97740 1 -- 172.21.15.43:0/1652351246 shutdown_connections 2024-04-21T08:42:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.545+0000 7fd23fd97740 1 --2- 172.21.15.43:0/1652351246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcaa41d4f0 0x55dcaa41f8e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.545+0000 7fd23fd97740 1 --2- 172.21.15.43:0/1652351246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa413f70 0x55dcaa4143f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.545+0000 7fd23fd97740 1 --2- 172.21.15.43:0/1652351246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcaa414ce0 0x55dcaa413a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.545+0000 7fd23fd97740 1 -- 172.21.15.43:0/1652351246 >> 172.21.15.43:0/1652351246 conn(0x55dcaa3d6c90 msgr2=0x55dcaa40ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.545+0000 7fd23fd97740 1 -- 172.21.15.43:0/1652351246 shutdown_connections 2024-04-21T08:42:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.545+0000 7fd23fd97740 1 -- 172.21.15.43:0/1652351246 wait complete. 2024-04-21T08:42:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.546+0000 7fd23fd97740 1 -- start start 2024-04-21T08:42:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.546+0000 7fd23fd97740 1 Processor -- start 2024-04-21T08:42:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.546+0000 7fd23fd97740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcaa413f70 0x55dcaa41cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.546+0000 7fd23fd97740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcaa414ce0 0x55dcaa41a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.546+0000 7fd23fd97740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa41d4f0 0x55dcaa41a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.546+0000 7fd23fd97740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dcaa420220 con 0x55dcaa41d4f0 2024-04-21T08:42:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.546+0000 7fd23fd97740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dcaa2fceb0 con 0x55dcaa414ce0 2024-04-21T08:42:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.546+0000 7fd23fd97740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dcaa4203a0 con 0x55dcaa413f70 2024-04-21T08:42:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.546+0000 7fd23fd97740 10 client.0 _ll_register_callbacks cb 0x55dcaa329b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.546+0000 7fd23759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa41d4f0 0x55dcaa41a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.546+0000 7fd23759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa41d4f0 0x55dcaa41a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44420/0 (socket says 172.21.15.43:44420) 2024-04-21T08:42:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.546+0000 7fd23759e640 1 -- 172.21.15.43:0/599394422 learned_addr learned my addr 172.21.15.43:0/599394422 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.546+0000 7fd22e59c640 1 --2- 172.21.15.43:0/599394422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcaa414ce0 0x55dcaa41a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.546+0000 7fd236d9d640 1 --2- 172.21.15.43:0/599394422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcaa413f70 0x55dcaa41cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.546+0000 7fd23759e640 1 -- 172.21.15.43:0/599394422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcaa413f70 msgr2=0x55dcaa41cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.546+0000 7fd23759e640 1 --2- 172.21.15.43:0/599394422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcaa413f70 0x55dcaa41cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.546+0000 7fd23759e640 1 -- 172.21.15.43:0/599394422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcaa414ce0 msgr2=0x55dcaa41a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.546+0000 7fd23759e640 1 --2- 172.21.15.43:0/599394422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcaa414ce0 0x55dcaa41a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.547+0000 7fd23759e640 1 -- 172.21.15.43:0/599394422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcaa087210 con 0x55dcaa41d4f0 2024-04-21T08:42:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.547+0000 7fd23759e640 1 --2- 172.21.15.43:0/599394422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa41d4f0 0x55dcaa41a690 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fd23009cc10 tx=0x7fd2300b1950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc55914f7a1b21fa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.547+0000 7fd21e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.547+0000 7fd21e7fc640 1 -- 172.21.15.43:0/599394422 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2300b65e0 con 0x55dcaa41d4f0 2024-04-21T08:42:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.547+0000 7fd23fd97740 1 -- 172.21.15.43:0/599394422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd23009d380 con 0x55dcaa41d4f0 2024-04-21T08:42:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.547+0000 7fd21e7fc640 1 -- 172.21.15.43:0/599394422 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd23009cee0 con 0x55dcaa41d4f0 2024-04-21T08:42:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.547+0000 7fd21e7fc640 1 -- 172.21.15.43:0/599394422 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd23009c5e0 con 0x55dcaa41d4f0 2024-04-21T08:42:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.547+0000 7fd23fd97740 10 client.17271 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.547+0000 7fd23fd97740 1 -- 172.21.15.43:0/599394422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dcaa153620 con 0x55dcaa41d4f0 2024-04-21T08:42:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.547+0000 7fd21e7fc640 1 -- 172.21.15.43:0/599394422 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd23009d380 con 0x55dcaa41d4f0 2024-04-21T08:42:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.547+0000 7fd21e7fc640 1 -- 172.21.15.43:0/599394422 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd2300a5460 con 0x55dcaa41d4f0 2024-04-21T08:42:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.547+0000 7fd23fd97740 10 client.17271 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.547+0000 7fd23fd97740 10 client.17271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.548+0000 7fd23fd97740 20 client.17271 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.548+0000 7fd23fd97740 1 --2- 172.21.15.43:0/599394422 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55dcaa42d2c0 0x55dcaa42f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.548+0000 7fd23fd97740 4 client.17271 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.548+0000 7fd23fd97740 1 -- 172.21.15.43:0/599394422 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcaa388880 con 0x55dcaa42d2c0 2024-04-21T08:42:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.548+0000 7fd236d9d640 1 --2- 172.21.15.43:0/599394422 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55dcaa42d2c0 0x55dcaa42f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.548+0000 7fd236d9d640 1 --2- 172.21.15.43:0/599394422 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55dcaa42d2c0 0x55dcaa42f6b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc67c22fce602472 server_cookie=7dcfbd73ac00e8d4 in_seq=0 out_seq=0 2024-04-21T08:42:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.548+0000 7fd21e7fc640 10 client.17271 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.549+0000 7fd21e7fc640 1 -- 172.21.15.43:0/599394422 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dcaa388880 con 0x55dcaa42d2c0 2024-04-21T08:42:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.549+0000 7fd21e7fc640 10 client.17271 handle_command_reply: tid=0 2024-04-21T08:42:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.549+0000 7fd23fd97740 1 client.17271 shutdown 2024-04-21T08:42:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.549+0000 7fd23fd97740 1 -- 172.21.15.43:0/599394422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa41d4f0 msgr2=0x55dcaa41a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.549+0000 7fd23fd97740 1 --2- 172.21.15.43:0/599394422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa41d4f0 0x55dcaa41a690 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fd23009cc10 tx=0x7fd2300b1950 comp rx=0 tx=0).stop 2024-04-21T08:42:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.549+0000 7fd23fd97740 1 -- 172.21.15.43:0/599394422 shutdown_connections 2024-04-21T08:42:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.549+0000 7fd23fd97740 1 --2- 172.21.15.43:0/599394422 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55dcaa42d2c0 0x55dcaa42f6b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.549+0000 7fd23fd97740 1 --2- 172.21.15.43:0/599394422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa41d4f0 0x55dcaa41a690 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.549+0000 7fd23fd97740 1 --2- 172.21.15.43:0/599394422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcaa414ce0 0x55dcaa41a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.549+0000 7fd21e7fc640 0 client.17271 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.549+0000 7fd23fd97740 1 --2- 172.21.15.43:0/599394422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcaa413f70 0x55dcaa41cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.549+0000 7fd23fd97740 1 -- 172.21.15.43:0/599394422 >> 172.21.15.43:0/599394422 conn(0x55dcaa3d6c90 msgr2=0x55dcaa4109b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.550+0000 7fd23fd97740 1 -- 172.21.15.43:0/599394422 shutdown_connections 2024-04-21T08:42:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.550+0000 7fd23fd97740 1 -- 172.21.15.43:0/599394422 wait complete. 2024-04-21T08:42:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.550+0000 7fd23fd97740 20 client.17271 trim_cache size 0 max 16384 2024-04-21T08:42:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.565+0000 7fd23fd97740 1 Processor -- start 2024-04-21T08:42:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.565+0000 7fd23fd97740 1 -- start start 2024-04-21T08:42:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.565+0000 7fd23fd97740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcaa413f70 0x55dcaa4192b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.565+0000 7fd23fd97740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcaa414ce0 0x55dcaa4107c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.565+0000 7fd23fd97740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa41d4f0 0x55dcaa4173b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.565+0000 7fd23fd97740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd2381ad050 con 0x55dcaa41d4f0 2024-04-21T08:42:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.565+0000 7fd23fd97740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd2381aeb90 con 0x55dcaa414ce0 2024-04-21T08:42:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.565+0000 7fd23fd97740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd2381ad720 con 0x55dcaa413f70 2024-04-21T08:42:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.565+0000 7fd23759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa41d4f0 0x55dcaa4173b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.565+0000 7fd23759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa41d4f0 0x55dcaa4173b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44426/0 (socket says 172.21.15.43:44426) 2024-04-21T08:42:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.565+0000 7fd23759e640 1 -- 172.21.15.43:0/145982630 learned_addr learned my addr 172.21.15.43:0/145982630 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.565+0000 7fd22e59c640 1 --2- 172.21.15.43:0/145982630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcaa414ce0 0x55dcaa4107c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.565+0000 7fd236d9d640 1 --2- 172.21.15.43:0/145982630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcaa413f70 0x55dcaa4192b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.565+0000 7fd23759e640 1 -- 172.21.15.43:0/145982630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcaa413f70 msgr2=0x55dcaa4192b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.565+0000 7fd23759e640 1 --2- 172.21.15.43:0/145982630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcaa413f70 0x55dcaa4192b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.566+0000 7fd23759e640 1 -- 172.21.15.43:0/145982630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcaa414ce0 msgr2=0x55dcaa4107c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.566+0000 7fd23759e640 1 --2- 172.21.15.43:0/145982630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcaa414ce0 0x55dcaa4107c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.566+0000 7fd23759e640 1 -- 172.21.15.43:0/145982630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcaa405f30 con 0x55dcaa41d4f0 2024-04-21T08:42:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.566+0000 7fd23759e640 1 --2- 172.21.15.43:0/145982630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa41d4f0 0x55dcaa4173b0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd230099650 tx=0x7fd23000efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d5036b1a531700 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.566+0000 7fd22cd99640 1 -- 172.21.15.43:0/145982630 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd230098f20 con 0x55dcaa41d4f0 2024-04-21T08:42:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.566+0000 7fd22cd99640 1 -- 172.21.15.43:0/145982630 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd23009d380 con 0x55dcaa41d4f0 2024-04-21T08:42:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.566+0000 7fd22cd99640 1 -- 172.21.15.43:0/145982630 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23009cee0 con 0x55dcaa41d4f0 2024-04-21T08:42:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.566+0000 7fd23fd97740 1 -- 172.21.15.43:0/145982630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa41d4f0 msgr2=0x55dcaa4173b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.566+0000 7fd23fd97740 1 --2- 172.21.15.43:0/145982630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa41d4f0 0x55dcaa4173b0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd230099650 tx=0x7fd23000efd0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.567+0000 7fd23fd97740 1 -- 172.21.15.43:0/145982630 shutdown_connections 2024-04-21T08:42:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.567+0000 7fd23fd97740 1 --2- 172.21.15.43:0/145982630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa41d4f0 0x55dcaa4173b0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.567+0000 7fd23fd97740 1 --2- 172.21.15.43:0/145982630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcaa414ce0 0x55dcaa4107c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.567+0000 7fd23fd97740 1 --2- 172.21.15.43:0/145982630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcaa413f70 0x55dcaa4192b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.567+0000 7fd23fd97740 1 -- 172.21.15.43:0/145982630 >> 172.21.15.43:0/145982630 conn(0x55dcaa3d6c90 msgr2=0x55dcaa4100b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.567+0000 7fd23fd97740 1 -- 172.21.15.43:0/145982630 shutdown_connections 2024-04-21T08:42:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.567+0000 7fd23fd97740 1 -- 172.21.15.43:0/145982630 wait complete. 2024-04-21T08:42:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.567+0000 7fd23fd97740 1 -- start start 2024-04-21T08:42:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.567+0000 7fd23fd97740 1 Processor -- start 2024-04-21T08:42:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.567+0000 7fd23fd97740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa413f70 0x55dcaa4301b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.567+0000 7fd23fd97740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcaa414ce0 0x55dcaa4306f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.567+0000 7fd23fd97740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcaa41d4f0 0x55dcaa430c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.567+0000 7fd23fd97740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd2381ae4c0 con 0x55dcaa413f70 2024-04-21T08:42:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.567+0000 7fd23fd97740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd2381addf0 con 0x55dcaa41d4f0 2024-04-21T08:42:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.567+0000 7fd23fd97740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd2381ab030 con 0x55dcaa414ce0 2024-04-21T08:42:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.567+0000 7fd236d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa413f70 0x55dcaa4301b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.567+0000 7fd23fd97740 10 client.0 _ll_register_callbacks cb 0x7fd230004e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.568+0000 7fd236d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa413f70 0x55dcaa4301b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44442/0 (socket says 172.21.15.43:44442) 2024-04-21T08:42:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.568+0000 7fd236d9d640 1 -- 172.21.15.43:0/1785475186 learned_addr learned my addr 172.21.15.43:0/1785475186 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.568+0000 7fd236d9d640 1 -- 172.21.15.43:0/1785475186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcaa414ce0 msgr2=0x55dcaa4306f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.568+0000 7fd236d9d640 1 --2- 172.21.15.43:0/1785475186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcaa414ce0 0x55dcaa4306f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.568+0000 7fd236d9d640 1 -- 172.21.15.43:0/1785475186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcaa41d4f0 msgr2=0x55dcaa430c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.568+0000 7fd23759e640 1 --2- 172.21.15.43:0/1785475186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcaa41d4f0 0x55dcaa430c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.568+0000 7fd236d9d640 1 --2- 172.21.15.43:0/1785475186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcaa41d4f0 0x55dcaa430c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.568+0000 7fd236d9d640 1 -- 172.21.15.43:0/1785475186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcaa4924f0 con 0x55dcaa413f70 2024-04-21T08:42:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.568+0000 7fd236d9d640 1 --2- 172.21.15.43:0/1785475186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa413f70 0x55dcaa4301b0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd228009f50 tx=0x7fd2280128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f15b775993f5cc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.568+0000 7fd21e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.568+0000 7fd21e7fc640 1 -- 172.21.15.43:0/1785475186 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd228004250 con 0x55dcaa413f70 2024-04-21T08:42:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.568+0000 7fd21e7fc640 1 -- 172.21.15.43:0/1785475186 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd2280043f0 con 0x55dcaa413f70 2024-04-21T08:42:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.568+0000 7fd23fd97740 1 -- 172.21.15.43:0/1785475186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd23009d380 con 0x55dcaa413f70 2024-04-21T08:42:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.568+0000 7fd21e7fc640 1 -- 172.21.15.43:0/1785475186 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd22802f050 con 0x55dcaa413f70 2024-04-21T08:42:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.568+0000 7fd23fd97740 10 client.17277 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.568+0000 7fd23fd97740 1 -- 172.21.15.43:0/1785475186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dcaa493840 con 0x55dcaa413f70 2024-04-21T08:42:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.569+0000 7fd21e7fc640 1 -- 172.21.15.43:0/1785475186 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd23009d380 con 0x55dcaa413f70 2024-04-21T08:42:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.569+0000 7fd21e7fc640 1 -- 172.21.15.43:0/1785475186 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd228003950 con 0x55dcaa413f70 2024-04-21T08:42:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.569+0000 7fd23fd97740 10 client.17277 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.569+0000 7fd23fd97740 10 client.17277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.569+0000 7fd23fd97740 20 client.17277 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.569+0000 7fd23fd97740 1 --2- 172.21.15.43:0/1785475186 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55dcaa493fc0 0x55dcaa4963e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.569+0000 7fd23fd97740 4 client.17277 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:42:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.569+0000 7fd23fd97740 1 -- 172.21.15.43:0/1785475186 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dcaa3143d0 con 0x55dcaa493fc0 2024-04-21T08:42:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.569+0000 7fd22e59c640 1 --2- 172.21.15.43:0/1785475186 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55dcaa493fc0 0x55dcaa4963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.569+0000 7fd22e59c640 1 --2- 172.21.15.43:0/1785475186 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55dcaa493fc0 0x55dcaa4963e0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fd7966a264a6d03 server_cookie=b7290e6bf4423849 in_seq=0 out_seq=0 2024-04-21T08:42:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.569+0000 7fd21e7fc640 10 client.17277 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.585+0000 7fd21e7fc640 1 -- 172.21.15.43:0/1785475186 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dcaa3143d0 con 0x55dcaa493fc0 2024-04-21T08:42:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.585+0000 7fd21e7fc640 10 client.17277 handle_command_reply: tid=0 2024-04-21T08:42:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.585+0000 7fd23fd97740 1 client.17277 shutdown 2024-04-21T08:42:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.585+0000 7fd23fd97740 1 -- 172.21.15.43:0/1785475186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa413f70 msgr2=0x55dcaa4301b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.585+0000 7fd23fd97740 1 --2- 172.21.15.43:0/1785475186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa413f70 0x55dcaa4301b0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd228009f50 tx=0x7fd2280128e0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.585+0000 7fd23fd97740 1 -- 172.21.15.43:0/1785475186 shutdown_connections 2024-04-21T08:42:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.585+0000 7fd23fd97740 1 --2- 172.21.15.43:0/1785475186 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55dcaa493fc0 0x55dcaa4963e0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.585+0000 7fd23fd97740 1 --2- 172.21.15.43:0/1785475186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dcaa41d4f0 0x55dcaa430c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.585+0000 7fd23fd97740 1 --2- 172.21.15.43:0/1785475186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcaa414ce0 0x55dcaa4306f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.585+0000 7fd23fd97740 1 --2- 172.21.15.43:0/1785475186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcaa413f70 0x55dcaa4301b0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.585+0000 7fd21e7fc640 0 client.17277 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.585+0000 7fd23fd97740 1 -- 172.21.15.43:0/1785475186 >> 172.21.15.43:0/1785475186 conn(0x55dcaa3d6c90 msgr2=0x55dcaa428630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.585+0000 7fd23fd97740 1 -- 172.21.15.43:0/1785475186 shutdown_connections 2024-04-21T08:42:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.585+0000 7fd23fd97740 1 -- 172.21.15.43:0/1785475186 wait complete. 2024-04-21T08:42:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.586+0000 7fd23fd97740 20 client.17277 trim_cache size 0 max 16384 2024-04-21T08:42:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:42:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:42:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:42:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:42:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.586+0000 7fd23d1a4640 1 -- 172.21.15.43:0/2417390379 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd2240a3dc0 msgr2=0x7fd2240a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.586+0000 7fd23d1a4640 1 --2- 172.21.15.43:0/2417390379 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd2240a3dc0 0x7fd2240a6280 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd220009230 tx=0x7fd22002c6c0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.586+0000 7fd23d1a4640 1 -- 172.21.15.43:0/2417390379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd23819cf00 msgr2=0x7fd23819d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.586+0000 7fd23d1a4640 1 --2- 172.21.15.43:0/2417390379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd23819cf00 0x7fd23819d3b0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd23000eb10 tx=0x7fd23000c2d0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.586+0000 7fd23d1a4640 1 -- 172.21.15.43:0/2417390379 shutdown_connections 2024-04-21T08:42:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.586+0000 7fd23d1a4640 1 --2- 172.21.15.43:0/2417390379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd23819cf00 0x7fd23819d3b0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.586+0000 7fd23d1a4640 1 --2- 172.21.15.43:0/2417390379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2381061b0 0x7fd238198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.586+0000 7fd23d1a4640 1 --2- 172.21.15.43:0/2417390379 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd2240a3dc0 0x7fd2240a6280 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.586+0000 7fd23d1a4640 1 --2- 172.21.15.43:0/2417390379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2381010e0 0x7fd238198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.586+0000 7fd23d1a4640 1 -- 172.21.15.43:0/2417390379 >> 172.21.15.43:0/2417390379 conn(0x7fd2380771d0 msgr2=0x7fd23806a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.586+0000 7fd23d1a4640 1 -- 172.21.15.43:0/2417390379 shutdown_connections 2024-04-21T08:42:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:45.586+0000 7fd23d1a4640 1 -- 172.21.15.43:0/2417390379 wait complete. 2024-04-21T08:42:46.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:42:46.442 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:42:46.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:46 smithi175 ceph-mon[31920]: pgmap v118: 129 pgs: 129 active+clean; 4.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-04-21T08:42:46.469 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:46 smithi043 ceph-mon[27978]: pgmap v118: 129 pgs: 129 active+clean; 4.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-04-21T08:42:46.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:46 smithi112 ceph-mon[31085]: pgmap v118: 129 pgs: 129 active+clean; 4.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-04-21T08:42:46.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.561+0000 7f5367c06640 1 -- 172.21.15.43:0/819603181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5360101a40 msgr2=0x7f536010dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:46.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.561+0000 7f5367c06640 1 --2- 172.21.15.43:0/819603181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5360101a40 0x7f536010dca0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f5358009ca0 tx=0x7f535802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.561+0000 7f5367c06640 1 -- 172.21.15.43:0/819603181 shutdown_connections 2024-04-21T08:42:46.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.561+0000 7f5367c06640 1 --2- 172.21.15.43:0/819603181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5360101a40 0x7f536010dca0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.561+0000 7f5367c06640 1 --2- 172.21.15.43:0/819603181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53601010a0 0x7f5360101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.561+0000 7f5367c06640 1 --2- 172.21.15.43:0/819603181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5360105dd0 0x7f53601061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.561+0000 7f5367c06640 1 -- 172.21.15.43:0/819603181 >> 172.21.15.43:0/819603181 conn(0x7f5360077c40 msgr2=0x7f5360078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:46.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.561+0000 7f5367c06640 1 -- 172.21.15.43:0/819603181 shutdown_connections 2024-04-21T08:42:46.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.561+0000 7f5367c06640 1 -- 172.21.15.43:0/819603181 wait complete. 2024-04-21T08:42:46.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.562+0000 7f5367c06640 1 Processor -- start 2024-04-21T08:42:46.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.562+0000 7f5367c06640 1 -- start start 2024-04-21T08:42:46.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.562+0000 7f5367c06640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53601010a0 0x7f5360198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:46.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.562+0000 7f5367c06640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5360101a40 0x7f5360198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:46.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.562+0000 7f5367c06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5360105dd0 0x7f536019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:46.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.562+0000 7f5367c06640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f53601122c0 con 0x7f5360105dd0 2024-04-21T08:42:46.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.562+0000 7f5367c06640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5360112140 con 0x7f53601010a0 2024-04-21T08:42:46.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.562+0000 7f5367c06640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5360112440 con 0x7f5360101a40 2024-04-21T08:42:46.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.562+0000 7f53661a3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5360105dd0 0x7f536019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:46.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.562+0000 7f53661a3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5360105dd0 0x7f536019d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44462/0 (socket says 172.21.15.43:44462) 2024-04-21T08:42:46.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.562+0000 7f53661a3640 1 -- 172.21.15.43:0/301439041 learned_addr learned my addr 172.21.15.43:0/301439041 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:46.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.562+0000 7f53659a2640 1 --2- 172.21.15.43:0/301439041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53601010a0 0x7f5360198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:46.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.563+0000 7f53651a1640 1 --2- 172.21.15.43:0/301439041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5360101a40 0x7f5360198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:46.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.563+0000 7f53661a3640 1 -- 172.21.15.43:0/301439041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5360101a40 msgr2=0x7f5360198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:46.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.563+0000 7f53661a3640 1 --2- 172.21.15.43:0/301439041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5360101a40 0x7f5360198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.563+0000 7f53661a3640 1 -- 172.21.15.43:0/301439041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53601010a0 msgr2=0x7f5360198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:46.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.563+0000 7f53661a3640 1 --2- 172.21.15.43:0/301439041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53601010a0 0x7f5360198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.563+0000 7f53661a3640 1 -- 172.21.15.43:0/301439041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536019d750 con 0x7f5360105dd0 2024-04-21T08:42:46.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.563+0000 7f53661a3640 1 --2- 172.21.15.43:0/301439041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5360105dd0 0x7f536019d020 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f5358005e30 tx=0x7f5358034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:46.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.563+0000 7f5356ffd640 1 -- 172.21.15.43:0/301439041 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358039040 con 0x7f5360105dd0 2024-04-21T08:42:46.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.563+0000 7f5356ffd640 1 -- 172.21.15.43:0/301439041 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5358044070 con 0x7f5360105dd0 2024-04-21T08:42:46.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.563+0000 7f5367c06640 1 -- 172.21.15.43:0/301439041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f536019d9e0 con 0x7f5360105dd0 2024-04-21T08:42:46.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.563+0000 7f5356ffd640 1 -- 172.21.15.43:0/301439041 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358003e60 con 0x7f5360105dd0 2024-04-21T08:42:46.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.563+0000 7f5367c06640 1 -- 172.21.15.43:0/301439041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53601a52d0 con 0x7f5360105dd0 2024-04-21T08:42:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.565+0000 7f5356ffd640 1 -- 172.21.15.43:0/301439041 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f53580324c0 con 0x7f5360105dd0 2024-04-21T08:42:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.566+0000 7f536a7f5740 1 Processor -- start 2024-04-21T08:42:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.566+0000 7f536a7f5740 1 -- start start 2024-04-21T08:42:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.566+0000 7f536a7f5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f3b75c580 0x555f3b75c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.566+0000 7f536a7f5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f3b75cea0 0x555f3b765540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.566+0000 7f536a7f5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b765a80 0x555f3b767e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.566+0000 7f536a7f5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555f3b6c11e0 con 0x555f3b765a80 2024-04-21T08:42:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.566+0000 7f536a7f5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555f3b65e390 con 0x555f3b75c580 2024-04-21T08:42:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.566+0000 7f536a7f5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555f3b6bd1a0 con 0x555f3b75cea0 2024-04-21T08:42:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.566+0000 7f5356ffd640 1 --2- 172.21.15.43:0/301439041 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f534c0a3da0 0x7f534c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.567+0000 7f53651a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b765a80 0x555f3b767e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.567+0000 7f53651a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b765a80 0x555f3b767e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44478/0 (socket says 172.21.15.43:44478) 2024-04-21T08:42:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.567+0000 7f53651a1640 1 -- 172.21.15.43:0/1629096654 learned_addr learned my addr 172.21.15.43:0/1629096654 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.567+0000 7f53661a3640 1 --2- 172.21.15.43:0/301439041 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f534c0a3da0 0x7f534c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.567+0000 7f53659a2640 1 --2- 172.21.15.43:0/1629096654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f3b75cea0 0x555f3b765540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.567+0000 7f53651a1640 1 --2- 172.21.15.43:0/1629096654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f3b75c580 0x555f3b75c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.567+0000 7f5356ffd640 1 -- 172.21.15.43:0/301439041 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f53580f2ce0 con 0x7f5360105dd0 2024-04-21T08:42:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.567+0000 7f53651a1640 1 -- 172.21.15.43:0/1629096654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f3b75cea0 msgr2=0x555f3b765540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.567+0000 7f53651a1640 1 --2- 172.21.15.43:0/1629096654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f3b75cea0 0x555f3b765540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.567+0000 7f53651a1640 1 -- 172.21.15.43:0/1629096654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f3b75c580 msgr2=0x555f3b75c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.567+0000 7f53651a1640 1 --2- 172.21.15.43:0/1629096654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f3b75c580 0x555f3b75c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.567+0000 7f53651a1640 1 -- 172.21.15.43:0/1629096654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f3b49d7d0 con 0x555f3b765a80 2024-04-21T08:42:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.567+0000 7f53661a3640 1 --2- 172.21.15.43:0/301439041 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f534c0a3da0 0x7f534c0a6260 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f53580f0220 tx=0x7f53580095c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.567+0000 7f53651a1640 1 --2- 172.21.15.43:0/1629096654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b765a80 0x555f3b767e70 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f53500149c0 tx=0x7f535000c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c4b50bf384e53b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.568+0000 7f534bfff640 1 -- 172.21.15.43:0/1629096654 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53500142d0 con 0x555f3b765a80 2024-04-21T08:42:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.568+0000 7f534bfff640 1 -- 172.21.15.43:0/1629096654 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53500054d0 con 0x555f3b765a80 2024-04-21T08:42:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.568+0000 7f534bfff640 1 -- 172.21.15.43:0/1629096654 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535000f760 con 0x555f3b765a80 2024-04-21T08:42:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.568+0000 7f536a7f5740 1 -- 172.21.15.43:0/1629096654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b765a80 msgr2=0x555f3b767e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.568+0000 7f536a7f5740 1 --2- 172.21.15.43:0/1629096654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b765a80 0x555f3b767e70 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f53500149c0 tx=0x7f535000c4d0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.568+0000 7f536a7f5740 1 -- 172.21.15.43:0/1629096654 shutdown_connections 2024-04-21T08:42:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.568+0000 7f536a7f5740 1 --2- 172.21.15.43:0/1629096654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b765a80 0x555f3b767e70 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.568+0000 7f536a7f5740 1 --2- 172.21.15.43:0/1629096654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f3b75cea0 0x555f3b765540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.568+0000 7f536a7f5740 1 --2- 172.21.15.43:0/1629096654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f3b75c580 0x555f3b75c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.568+0000 7f536a7f5740 1 -- 172.21.15.43:0/1629096654 >> 172.21.15.43:0/1629096654 conn(0x555f3b756cb0 msgr2=0x555f3b757090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.568+0000 7f536a7f5740 1 -- 172.21.15.43:0/1629096654 shutdown_connections 2024-04-21T08:42:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.568+0000 7f536a7f5740 1 -- 172.21.15.43:0/1629096654 wait complete. 2024-04-21T08:42:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.569+0000 7f536a7f5740 1 -- start start 2024-04-21T08:42:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.569+0000 7f536a7f5740 1 Processor -- start 2024-04-21T08:42:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.569+0000 7f536a7f5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b75c580 0x555f3b762aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.569+0000 7f536a7f5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f3b75cea0 0x555f3b762fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.569+0000 7f536a7f5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f3b765a80 0x555f3b763520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.569+0000 7f536a7f5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555f3b768870 con 0x555f3b75c580 2024-04-21T08:42:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.569+0000 7f536a7f5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555f3b67a2f0 con 0x555f3b765a80 2024-04-21T08:42:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.569+0000 7f536a7f5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555f3b7689f0 con 0x555f3b75cea0 2024-04-21T08:42:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.569+0000 7f536a7f5740 10 client.0 _ll_register_callbacks cb 0x555f3b687580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.569+0000 7f53651a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b75c580 0x555f3b762aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.569+0000 7f53651a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b75c580 0x555f3b762aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44490/0 (socket says 172.21.15.43:44490) 2024-04-21T08:42:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.569+0000 7f53651a1640 1 -- 172.21.15.43:0/3786988684 learned_addr learned my addr 172.21.15.43:0/3786988684 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.570+0000 7f53659a2640 1 --2- 172.21.15.43:0/3786988684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f3b75cea0 0x555f3b762fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.570+0000 7f53651a1640 1 -- 172.21.15.43:0/3786988684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f3b75cea0 msgr2=0x555f3b762fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.570+0000 7f53651a1640 1 --2- 172.21.15.43:0/3786988684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f3b75cea0 0x555f3b762fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.570+0000 7f53651a1640 1 -- 172.21.15.43:0/3786988684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f3b765a80 msgr2=0x555f3b763520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.570+0000 7f53651a1640 1 --2- 172.21.15.43:0/3786988684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f3b765a80 0x555f3b763520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.570+0000 7f53651a1640 1 -- 172.21.15.43:0/3786988684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f3b3d33f0 con 0x555f3b75c580 2024-04-21T08:42:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.570+0000 7f53651a1640 1 --2- 172.21.15.43:0/3786988684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b75c580 0x555f3b762aa0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f5350004ae0 tx=0x7f5350004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12697d334dc431 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.570+0000 7f53497fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.570+0000 7f53497fa640 1 -- 172.21.15.43:0/3786988684 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5350027020 con 0x555f3b75c580 2024-04-21T08:42:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.570+0000 7f53497fa640 1 -- 172.21.15.43:0/3786988684 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f5350005000 con 0x555f3b75c580 2024-04-21T08:42:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.570+0000 7f536a7f5740 1 -- 172.21.15.43:0/3786988684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53500054d0 con 0x555f3b75c580 2024-04-21T08:42:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.570+0000 7f53497fa640 1 -- 172.21.15.43:0/3786988684 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f53500142d0 con 0x555f3b75c580 2024-04-21T08:42:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.570+0000 7f536a7f5740 10 client.17289 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.570+0000 7f536a7f5740 1 -- 172.21.15.43:0/3786988684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555f3b3930f0 con 0x555f3b75c580 2024-04-21T08:42:46.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.571+0000 7f53497fa640 1 -- 172.21.15.43:0/3786988684 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53500054d0 con 0x555f3b75c580 2024-04-21T08:42:46.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.571+0000 7f53497fa640 1 -- 172.21.15.43:0/3786988684 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f53500124e0 con 0x555f3b75c580 2024-04-21T08:42:46.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.571+0000 7f536a7f5740 10 client.17289 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:46.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.571+0000 7f536a7f5740 10 client.17289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:46.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.571+0000 7f536a7f5740 20 client.17289 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:46.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.571+0000 7f536a7f5740 1 --2- 172.21.15.43:0/3786988684 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555f3b775940 0x555f3b777d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:46.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.571+0000 7f536a7f5740 4 client.17289 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:46.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.571+0000 7f536a7f5740 1 -- 172.21.15.43:0/3786988684 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f3b6cec00 con 0x555f3b775940 2024-04-21T08:42:46.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.571+0000 7f53659a2640 1 --2- 172.21.15.43:0/3786988684 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555f3b775940 0x555f3b777d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:46.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.571+0000 7f53659a2640 1 --2- 172.21.15.43:0/3786988684 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555f3b775940 0x555f3b777d30 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f9dc3b6fec1ec0a server_cookie=b3f61bc7f55e2599 in_seq=0 out_seq=0 2024-04-21T08:42:46.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.572+0000 7f53497fa640 10 client.17289 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:46.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.572+0000 7f53497fa640 1 -- 172.21.15.43:0/3786988684 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f3b6cec00 con 0x555f3b775940 2024-04-21T08:42:46.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.572+0000 7f53497fa640 10 client.17289 handle_command_reply: tid=0 2024-04-21T08:42:46.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.572+0000 7f536a7f5740 1 client.17289 shutdown 2024-04-21T08:42:46.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.572+0000 7f536a7f5740 1 -- 172.21.15.43:0/3786988684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b75c580 msgr2=0x555f3b762aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:46.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.572+0000 7f536a7f5740 1 --2- 172.21.15.43:0/3786988684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b75c580 0x555f3b762aa0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f5350004ae0 tx=0x7f5350004bb0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.573+0000 7f536a7f5740 1 -- 172.21.15.43:0/3786988684 shutdown_connections 2024-04-21T08:42:46.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.573+0000 7f536a7f5740 1 --2- 172.21.15.43:0/3786988684 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555f3b775940 0x555f3b777d30 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.573+0000 7f536a7f5740 1 --2- 172.21.15.43:0/3786988684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f3b765a80 0x555f3b763520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.573+0000 7f536a7f5740 1 --2- 172.21.15.43:0/3786988684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f3b75cea0 0x555f3b762fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.573+0000 7f53497fa640 0 client.17289 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:46.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.573+0000 7f536a7f5740 1 --2- 172.21.15.43:0/3786988684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b75c580 0x555f3b762aa0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.573+0000 7f536a7f5740 1 -- 172.21.15.43:0/3786988684 >> 172.21.15.43:0/3786988684 conn(0x555f3b756cb0 msgr2=0x555f3b758a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:46.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.573+0000 7f536a7f5740 1 -- 172.21.15.43:0/3786988684 shutdown_connections 2024-04-21T08:42:46.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.573+0000 7f536a7f5740 1 -- 172.21.15.43:0/3786988684 wait complete. 2024-04-21T08:42:46.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.573+0000 7f536a7f5740 20 client.17289 trim_cache size 0 max 16384 2024-04-21T08:42:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.591+0000 7f536a7f5740 1 Processor -- start 2024-04-21T08:42:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.591+0000 7f536a7f5740 1 -- start start 2024-04-21T08:42:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.591+0000 7f536a7f5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f3b75c580 0x555f3b7651d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.591+0000 7f536a7f5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f3b75cea0 0x555f3b771790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.591+0000 7f536a7f5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b765a80 0x555f3b758810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.591+0000 7f536a7f5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f53601af360 con 0x555f3b765a80 2024-04-21T08:42:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.591+0000 7f536a7f5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f53601b0ea0 con 0x555f3b75cea0 2024-04-21T08:42:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.591+0000 7f536a7f5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f53601afa30 con 0x555f3b75c580 2024-04-21T08:42:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.591+0000 7f53651a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b765a80 0x555f3b758810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.591+0000 7f53651a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b765a80 0x555f3b758810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44494/0 (socket says 172.21.15.43:44494) 2024-04-21T08:42:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.591+0000 7f53651a1640 1 -- 172.21.15.43:0/3030327778 learned_addr learned my addr 172.21.15.43:0/3030327778 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.591+0000 7f53659a2640 1 --2- 172.21.15.43:0/3030327778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f3b75cea0 0x555f3b771790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.591+0000 7f53651a1640 1 -- 172.21.15.43:0/3030327778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f3b75c580 msgr2=0x555f3b7651d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.591+0000 7f53651a1640 1 --2- 172.21.15.43:0/3030327778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f3b75c580 0x555f3b7651d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.591+0000 7f53651a1640 1 -- 172.21.15.43:0/3030327778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f3b75cea0 msgr2=0x555f3b771790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.591+0000 7f53651a1640 1 --2- 172.21.15.43:0/3030327778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f3b75cea0 0x555f3b771790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.591+0000 7f53651a1640 1 -- 172.21.15.43:0/3030327778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f3b768400 con 0x555f3b765a80 2024-04-21T08:42:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.591+0000 7f53659a2640 1 --2- 172.21.15.43:0/3030327778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f3b75cea0 0x555f3b771790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.591+0000 7f53651a1640 1 --2- 172.21.15.43:0/3030327778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b765a80 0x555f3b758810 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f5350012900 tx=0x7f53500148c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a05034f96270d02 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.591+0000 7f534b7fe640 1 -- 172.21.15.43:0/3030327778 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5350027040 con 0x555f3b765a80 2024-04-21T08:42:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.592+0000 7f534b7fe640 1 -- 172.21.15.43:0/3030327778 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53500048d0 con 0x555f3b765a80 2024-04-21T08:42:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.592+0000 7f534b7fe640 1 -- 172.21.15.43:0/3030327778 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5350012b70 con 0x555f3b765a80 2024-04-21T08:42:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.592+0000 7f536a7f5740 1 -- 172.21.15.43:0/3030327778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b765a80 msgr2=0x555f3b758810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.592+0000 7f536a7f5740 1 --2- 172.21.15.43:0/3030327778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b765a80 0x555f3b758810 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f5350012900 tx=0x7f53500148c0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.592+0000 7f536a7f5740 1 -- 172.21.15.43:0/3030327778 shutdown_connections 2024-04-21T08:42:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.592+0000 7f536a7f5740 1 --2- 172.21.15.43:0/3030327778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b765a80 0x555f3b758810 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.592+0000 7f536a7f5740 1 --2- 172.21.15.43:0/3030327778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f3b75cea0 0x555f3b771790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.592+0000 7f536a7f5740 1 --2- 172.21.15.43:0/3030327778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f3b75c580 0x555f3b7651d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.592+0000 7f536a7f5740 1 -- 172.21.15.43:0/3030327778 >> 172.21.15.43:0/3030327778 conn(0x555f3b756cb0 msgr2=0x555f3b758100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.592+0000 7f536a7f5740 1 -- 172.21.15.43:0/3030327778 shutdown_connections 2024-04-21T08:42:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.592+0000 7f536a7f5740 1 -- 172.21.15.43:0/3030327778 wait complete. 2024-04-21T08:42:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.593+0000 7f536a7f5740 1 -- start start 2024-04-21T08:42:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.593+0000 7f536a7f5740 1 Processor -- start 2024-04-21T08:42:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.593+0000 7f536a7f5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f3b75c580 0x555f3b76dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.593+0000 7f536a7f5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f3b75cea0 0x555f3b76e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.593+0000 7f536a7f5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b765a80 0x555f3b769220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.593+0000 7f536a7f5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f53601b07d0 con 0x555f3b765a80 2024-04-21T08:42:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.593+0000 7f536a7f5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f53601b0100 con 0x555f3b75c580 2024-04-21T08:42:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.593+0000 7f536a7f5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f53601ad340 con 0x555f3b75cea0 2024-04-21T08:42:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.593+0000 7f536a7f5740 10 client.0 _ll_register_callbacks cb 0x7f535000cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.593+0000 7f53651a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b765a80 0x555f3b769220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.593+0000 7f53651a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b765a80 0x555f3b769220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44496/0 (socket says 172.21.15.43:44496) 2024-04-21T08:42:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.593+0000 7f53651a1640 1 -- 172.21.15.43:0/3164906880 learned_addr learned my addr 172.21.15.43:0/3164906880 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.593+0000 7f53651a1640 1 --2- 172.21.15.43:0/3164906880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f3b75c580 0x555f3b76dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.593+0000 7f53659a2640 1 --2- 172.21.15.43:0/3164906880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f3b75cea0 0x555f3b76e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.594+0000 7f53651a1640 1 -- 172.21.15.43:0/3164906880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f3b75cea0 msgr2=0x555f3b76e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.594+0000 7f53651a1640 1 --2- 172.21.15.43:0/3164906880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f3b75cea0 0x555f3b76e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.594+0000 7f53651a1640 1 -- 172.21.15.43:0/3164906880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f3b75c580 msgr2=0x555f3b76dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.594+0000 7f53651a1640 1 --2- 172.21.15.43:0/3164906880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f3b75c580 0x555f3b76dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.594+0000 7f53651a1640 1 -- 172.21.15.43:0/3164906880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f3b769a20 con 0x555f3b765a80 2024-04-21T08:42:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.594+0000 7f53651a1640 1 --2- 172.21.15.43:0/3164906880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b765a80 0x555f3b769220 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f5350005c20 tx=0x7f5350005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d551857260e3ff2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.594+0000 7f53497fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.594+0000 7f53497fa640 1 -- 172.21.15.43:0/3164906880 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535004b060 con 0x555f3b765a80 2024-04-21T08:42:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.594+0000 7f53497fa640 1 -- 172.21.15.43:0/3164906880 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f5350012450 con 0x555f3b765a80 2024-04-21T08:42:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.594+0000 7f536a7f5740 1 -- 172.21.15.43:0/3164906880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53500048d0 con 0x555f3b765a80 2024-04-21T08:42:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.594+0000 7f53497fa640 1 -- 172.21.15.43:0/3164906880 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5350027040 con 0x555f3b765a80 2024-04-21T08:42:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.594+0000 7f536a7f5740 10 client.17295 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.594+0000 7f536a7f5740 1 -- 172.21.15.43:0/3164906880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555f3b7db780 con 0x555f3b765a80 2024-04-21T08:42:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.594+0000 7f53497fa640 1 -- 172.21.15.43:0/3164906880 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53500048d0 con 0x555f3b765a80 2024-04-21T08:42:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.594+0000 7f53497fa640 1 -- 172.21.15.43:0/3164906880 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f535000fc70 con 0x555f3b765a80 2024-04-21T08:42:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.594+0000 7f536a7f5740 10 client.17295 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.594+0000 7f536a7f5740 10 client.17295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.594+0000 7f536a7f5740 20 client.17295 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.594+0000 7f536a7f5740 1 --2- 172.21.15.43:0/3164906880 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555f3b7dbc00 0x555f3b7ddff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.595+0000 7f536a7f5740 4 client.17295 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:42:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.595+0000 7f536a7f5740 1 -- 172.21.15.43:0/3164906880 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555f3b6c8a70 con 0x555f3b7dbc00 2024-04-21T08:42:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.595+0000 7f53659a2640 1 --2- 172.21.15.43:0/3164906880 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555f3b7dbc00 0x555f3b7ddff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.595+0000 7f53659a2640 1 --2- 172.21.15.43:0/3164906880 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555f3b7dbc00 0x555f3b7ddff0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e54dc05111f7f64 server_cookie=d1971a4ce8297c9 in_seq=0 out_seq=0 2024-04-21T08:42:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.595+0000 7f53497fa640 10 client.17295 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.596+0000 7f53497fa640 1 -- 172.21.15.43:0/3164906880 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555f3b6c8a70 con 0x555f3b7dbc00 2024-04-21T08:42:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.596+0000 7f53497fa640 10 client.17295 handle_command_reply: tid=0 2024-04-21T08:42:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.596+0000 7f536a7f5740 1 client.17295 shutdown 2024-04-21T08:42:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.596+0000 7f536a7f5740 1 -- 172.21.15.43:0/3164906880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b765a80 msgr2=0x555f3b769220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.596+0000 7f536a7f5740 1 --2- 172.21.15.43:0/3164906880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b765a80 0x555f3b769220 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f5350005c20 tx=0x7f5350005fa0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.596+0000 7f536a7f5740 1 -- 172.21.15.43:0/3164906880 shutdown_connections 2024-04-21T08:42:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.596+0000 7f536a7f5740 1 --2- 172.21.15.43:0/3164906880 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555f3b7dbc00 0x555f3b7ddff0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.596+0000 7f536a7f5740 1 --2- 172.21.15.43:0/3164906880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f3b765a80 0x555f3b769220 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.596+0000 7f536a7f5740 1 --2- 172.21.15.43:0/3164906880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f3b75cea0 0x555f3b76e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.596+0000 7f536a7f5740 1 --2- 172.21.15.43:0/3164906880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f3b75c580 0x555f3b76dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.596+0000 7f53497fa640 0 client.17295 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.596+0000 7f536a7f5740 1 -- 172.21.15.43:0/3164906880 >> 172.21.15.43:0/3164906880 conn(0x555f3b756cb0 msgr2=0x555f3b755d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.596+0000 7f536a7f5740 1 -- 172.21.15.43:0/3164906880 shutdown_connections 2024-04-21T08:42:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.596+0000 7f536a7f5740 1 -- 172.21.15.43:0/3164906880 wait complete. 2024-04-21T08:42:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.596+0000 7f536a7f5740 20 client.17295 trim_cache size 0 max 16384 2024-04-21T08:42:46.603 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:42:46.603 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:42:46.603 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:42:46.603 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:42:46.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.596+0000 7f5367c06640 1 -- 172.21.15.43:0/301439041 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f534c0a3da0 msgr2=0x7f534c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:46.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.596+0000 7f5367c06640 1 --2- 172.21.15.43:0/301439041 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f534c0a3da0 0x7f534c0a6260 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f53580f0220 tx=0x7f53580095c0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.596+0000 7f5367c06640 1 -- 172.21.15.43:0/301439041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5360105dd0 msgr2=0x7f536019d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.596+0000 7f5367c06640 1 --2- 172.21.15.43:0/301439041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5360105dd0 0x7f536019d020 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f5358005e30 tx=0x7f5358034760 comp rx=0 tx=0).stop 2024-04-21T08:42:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.597+0000 7f5367c06640 1 -- 172.21.15.43:0/301439041 shutdown_connections 2024-04-21T08:42:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.597+0000 7f5367c06640 1 --2- 172.21.15.43:0/301439041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5360105dd0 0x7f536019d020 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.597+0000 7f5367c06640 1 --2- 172.21.15.43:0/301439041 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f534c0a3da0 0x7f534c0a6260 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.597+0000 7f5367c06640 1 --2- 172.21.15.43:0/301439041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5360101a40 0x7f5360198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.597+0000 7f5367c06640 1 --2- 172.21.15.43:0/301439041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53601010a0 0x7f5360198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.597+0000 7f5367c06640 1 -- 172.21.15.43:0/301439041 >> 172.21.15.43:0/301439041 conn(0x7f5360077c40 msgr2=0x7f53600fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.597+0000 7f5367c06640 1 -- 172.21.15.43:0/301439041 shutdown_connections 2024-04-21T08:42:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.597+0000 7f5367c06640 1 -- 172.21.15.43:0/301439041 wait complete. 2024-04-21T08:42:46.607 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:42:46.607 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:42:46.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.751+0000 7feff20fd640 1 -- 172.21.15.43:0/512139151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefec06a8b0 msgr2=0x7fefec06ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:46.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.751+0000 7feff20fd640 1 --2- 172.21.15.43:0/512139151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefec06a8b0 0x7fefec06ac90 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fefdc009900 tx=0x7fefdc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:42:46.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.751+0000 7feff20fd640 1 -- 172.21.15.43:0/512139151 shutdown_connections 2024-04-21T08:42:46.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.751+0000 7feff20fd640 1 --2- 172.21.15.43:0/512139151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefec1072c0 0x7fefec10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.751+0000 7feff20fd640 1 --2- 172.21.15.43:0/512139151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefec06b260 0x7fefec106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.751+0000 7feff20fd640 1 --2- 172.21.15.43:0/512139151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefec06a8b0 0x7fefec06ac90 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.751+0000 7feff20fd640 1 -- 172.21.15.43:0/512139151 >> 172.21.15.43:0/512139151 conn(0x7fefec077e30 msgr2=0x7fefec078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:46.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.751+0000 7feff20fd640 1 -- 172.21.15.43:0/512139151 shutdown_connections 2024-04-21T08:42:46.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.751+0000 7feff20fd640 1 -- 172.21.15.43:0/512139151 wait complete. 2024-04-21T08:42:46.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.752+0000 7feff20fd640 1 Processor -- start 2024-04-21T08:42:46.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.752+0000 7feff20fd640 1 -- start start 2024-04-21T08:42:46.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.752+0000 7feff20fd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefec06a8b0 0x7fefec198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:46.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.752+0000 7feff20fd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefec1072c0 0x7fefec198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:46.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.752+0000 7feff20fd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefec19cfd0 0x7fefec19d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:46.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.752+0000 7feff20fd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fefec1121d0 con 0x7fefec06a8b0 2024-04-21T08:42:46.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.752+0000 7feff20fd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fefec112050 con 0x7fefec1072c0 2024-04-21T08:42:46.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.752+0000 7feff20fd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fefec112350 con 0x7fefec19cfd0 2024-04-21T08:42:46.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.752+0000 7fefeb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefec06a8b0 0x7fefec198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:46.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.752+0000 7fefeb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefec06a8b0 0x7fefec198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34290/0 (socket says 172.21.15.43:34290) 2024-04-21T08:42:46.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.752+0000 7fefeb7fe640 1 -- 172.21.15.43:0/2115373615 learned_addr learned my addr 172.21.15.43:0/2115373615 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:46.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.752+0000 7fefeaffd640 1 --2- 172.21.15.43:0/2115373615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefec1072c0 0x7fefec198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:46.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.752+0000 7fefeb7fe640 1 -- 172.21.15.43:0/2115373615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefec19cfd0 msgr2=0x7fefec19d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:46.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.753+0000 7fefeb7fe640 1 --2- 172.21.15.43:0/2115373615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefec19cfd0 0x7fefec19d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.753+0000 7fefeb7fe640 1 -- 172.21.15.43:0/2115373615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefec1072c0 msgr2=0x7fefec198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:46.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.753+0000 7fefeb7fe640 1 --2- 172.21.15.43:0/2115373615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefec1072c0 0x7fefec198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.753+0000 7fefeb7fe640 1 -- 172.21.15.43:0/2115373615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefec19dac0 con 0x7fefec06a8b0 2024-04-21T08:42:46.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.753+0000 7fefeaffd640 1 --2- 172.21.15.43:0/2115373615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefec1072c0 0x7fefec198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:46.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.753+0000 7fefeb7fe640 1 --2- 172.21.15.43:0/2115373615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefec06a8b0 0x7fefec198710 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fefdc005940 tx=0x7fefdc02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:46.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.753+0000 7fefe8ff9640 1 -- 172.21.15.43:0/2115373615 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc043020 con 0x7fefec06a8b0 2024-04-21T08:42:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.753+0000 7fefe8ff9640 1 -- 172.21.15.43:0/2115373615 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefdc02cbe0 con 0x7fefec06a8b0 2024-04-21T08:42:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.753+0000 7feff20fd640 1 -- 172.21.15.43:0/2115373615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefec19dd50 con 0x7fefec06a8b0 2024-04-21T08:42:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.753+0000 7fefe8ff9640 1 -- 172.21.15.43:0/2115373615 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc003bf0 con 0x7fefec06a8b0 2024-04-21T08:42:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.753+0000 7feff20fd640 1 -- 172.21.15.43:0/2115373615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefec1a55a0 con 0x7fefec06a8b0 2024-04-21T08:42:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.754+0000 7feff20fd640 1 -- 172.21.15.43:0/2115373615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefec069230 con 0x7fefec06a8b0 2024-04-21T08:42:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.755+0000 7fefe8ff9640 1 -- 172.21.15.43:0/2115373615 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fefdc046020 con 0x7fefec06a8b0 2024-04-21T08:42:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.756+0000 7fefe8ff9640 1 --2- 172.21.15.43:0/2115373615 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fefd40a3da0 0x7fefd40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.756+0000 7fefe8ff9640 1 -- 172.21.15.43:0/2115373615 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fefdc0f33a0 con 0x7fefec06a8b0 2024-04-21T08:42:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.756+0000 7fefeaffd640 1 --2- 172.21.15.43:0/2115373615 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fefd40a3da0 0x7fefd40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.756+0000 7fefeaffd640 1 --2- 172.21.15.43:0/2115373615 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fefd40a3da0 0x7fefd40a6260 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fefec100d30 tx=0x7fefe000a5c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:46.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.760+0000 7fefe8ff9640 1 -- 172.21.15.43:0/2115373615 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fefdc04d020 con 0x7fefec06a8b0 2024-04-21T08:42:46.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.986+0000 7feff20fd640 1 -- 172.21.15.43:0/2115373615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefec199a60 con 0x7fefec06a8b0 2024-04-21T08:42:46.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.986+0000 7fefe8ff9640 1 -- 172.21.15.43:0/2115373615 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7fefdc0bf780 con 0x7fefec06a8b0 2024-04-21T08:42:46.986 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:42:46.986 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:42:46.986 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:42:46.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.988+0000 7feff20fd640 1 -- 172.21.15.43:0/2115373615 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fefd40a3da0 msgr2=0x7fefd40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:46.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.988+0000 7feff20fd640 1 --2- 172.21.15.43:0/2115373615 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fefd40a3da0 0x7fefd40a6260 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fefec100d30 tx=0x7fefe000a5c0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.989+0000 7feff20fd640 1 -- 172.21.15.43:0/2115373615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefec06a8b0 msgr2=0x7fefec198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:46.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.989+0000 7feff20fd640 1 --2- 172.21.15.43:0/2115373615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefec06a8b0 0x7fefec198710 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fefdc005940 tx=0x7fefdc02c7e0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.989+0000 7feff20fd640 1 -- 172.21.15.43:0/2115373615 shutdown_connections 2024-04-21T08:42:46.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.989+0000 7feff20fd640 1 --2- 172.21.15.43:0/2115373615 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fefd40a3da0 0x7fefd40a6260 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.989+0000 7feff20fd640 1 --2- 172.21.15.43:0/2115373615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefec19cfd0 0x7fefec19d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.989+0000 7feff20fd640 1 --2- 172.21.15.43:0/2115373615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefec1072c0 0x7fefec198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.989+0000 7feff20fd640 1 --2- 172.21.15.43:0/2115373615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefec06a8b0 0x7fefec198710 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.989+0000 7feff20fd640 1 -- 172.21.15.43:0/2115373615 >> 172.21.15.43:0/2115373615 conn(0x7fefec077e30 msgr2=0x7fefec0ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.989+0000 7feff20fd640 1 -- 172.21.15.43:0/2115373615 shutdown_connections 2024-04-21T08:42:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:46.989+0000 7feff20fd640 1 -- 172.21.15.43:0/2115373615 wait complete. 2024-04-21T08:42:46.999 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:42:47.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.104+0000 7fc725dfb640 1 -- 172.21.15.43:0/3717547600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc720102b40 msgr2=0x7fc720106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.104+0000 7fc725dfb640 1 --2- 172.21.15.43:0/3717547600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc720102b40 0x7fc720106b80 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fc714009900 tx=0x7fc71402bc30 comp rx=0 tx=0).stop 2024-04-21T08:42:47.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.104+0000 7fc725dfb640 1 -- 172.21.15.43:0/3717547600 shutdown_connections 2024-04-21T08:42:47.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.104+0000 7fc725dfb640 1 --2- 172.21.15.43:0/3717547600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc720107260 0x7fc72010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.104+0000 7fc725dfb640 1 --2- 172.21.15.43:0/3717547600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc720102b40 0x7fc720106b80 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.104+0000 7fc725dfb640 1 --2- 172.21.15.43:0/3717547600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc720102190 0x7fc720102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.104+0000 7fc725dfb640 1 -- 172.21.15.43:0/3717547600 >> 172.21.15.43:0/3717547600 conn(0x7fc7200fd990 msgr2=0x7fc7200ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.104+0000 7fc725dfb640 1 -- 172.21.15.43:0/3717547600 shutdown_connections 2024-04-21T08:42:47.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.104+0000 7fc725dfb640 1 -- 172.21.15.43:0/3717547600 wait complete. 2024-04-21T08:42:47.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.104+0000 7fc725dfb640 1 Processor -- start 2024-04-21T08:42:47.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.105+0000 7fc725dfb640 1 -- start start 2024-04-21T08:42:47.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.105+0000 7fc725dfb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc720102190 0x7fc72019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.105+0000 7fc725dfb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc720107260 0x7fc72019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.105+0000 7fc725dfb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7201985b0 0x7fc720198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.105+0000 7fc725dfb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc7201121f0 con 0x7fc7201985b0 2024-04-21T08:42:47.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.105+0000 7fc725dfb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc720112070 con 0x7fc720107260 2024-04-21T08:42:47.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.105+0000 7fc725dfb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc720112370 con 0x7fc720102190 2024-04-21T08:42:47.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.105+0000 7fc71ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7201985b0 0x7fc720198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.105+0000 7fc71ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7201985b0 0x7fc720198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34306/0 (socket says 172.21.15.43:34306) 2024-04-21T08:42:47.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.105+0000 7fc71ffff640 1 -- 172.21.15.43:0/2160601857 learned_addr learned my addr 172.21.15.43:0/2160601857 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.105+0000 7fc71effd640 1 --2- 172.21.15.43:0/2160601857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc720107260 0x7fc72019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.105+0000 7fc71f7fe640 1 --2- 172.21.15.43:0/2160601857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc720102190 0x7fc72019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.105+0000 7fc71ffff640 1 -- 172.21.15.43:0/2160601857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc720102190 msgr2=0x7fc72019e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.105+0000 7fc71ffff640 1 --2- 172.21.15.43:0/2160601857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc720102190 0x7fc72019e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.105+0000 7fc71ffff640 1 -- 172.21.15.43:0/2160601857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc720107260 msgr2=0x7fc72019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.105+0000 7fc71ffff640 1 --2- 172.21.15.43:0/2160601857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc720107260 0x7fc72019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.105+0000 7fc71ffff640 1 -- 172.21.15.43:0/2160601857 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7201990d0 con 0x7fc7201985b0 2024-04-21T08:42:47.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.105+0000 7fc71f7fe640 1 --2- 172.21.15.43:0/2160601857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc720102190 0x7fc72019e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:47.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.105+0000 7fc71ffff640 1 --2- 172.21.15.43:0/2160601857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7201985b0 0x7fc720198a60 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fc71800d750 tx=0x7fc71800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.106+0000 7fc71cff9640 1 -- 172.21.15.43:0/2160601857 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc718017070 con 0x7fc7201985b0 2024-04-21T08:42:47.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.106+0000 7fc71cff9640 1 -- 172.21.15.43:0/2160601857 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7180054d0 con 0x7fc7201985b0 2024-04-21T08:42:47.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.106+0000 7fc725dfb640 1 -- 172.21.15.43:0/2160601857 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7201993c0 con 0x7fc7201985b0 2024-04-21T08:42:47.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.106+0000 7fc71cff9640 1 -- 172.21.15.43:0/2160601857 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc718004c60 con 0x7fc7201985b0 2024-04-21T08:42:47.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.106+0000 7fc725dfb640 1 -- 172.21.15.43:0/2160601857 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7201a55c0 con 0x7fc7201985b0 2024-04-21T08:42:47.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.108+0000 7fc71cff9640 1 -- 172.21.15.43:0/2160601857 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc718009bd0 con 0x7fc7201985b0 2024-04-21T08:42:47.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.108+0000 7fc7289e2740 1 Processor -- start 2024-04-21T08:42:47.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.108+0000 7fc7289e2740 1 -- start start 2024-04-21T08:42:47.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.108+0000 7fc7289e2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b86b0a4d0 0x555b86b0a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.108+0000 7fc7289e2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b0ae80 0x555b86b13430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.108+0000 7fc7289e2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b86b13a00 0x555b86b15e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.108+0000 7fc7289e2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555b86a6f1f0 con 0x555b86b0ae80 2024-04-21T08:42:47.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.108+0000 7fc7289e2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555b86a0c4b0 con 0x555b86b13a00 2024-04-21T08:42:47.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.108+0000 7fc7289e2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555b86a6b1b0 con 0x555b86b0a4d0 2024-04-21T08:42:47.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.108+0000 7fc71cff9640 1 --2- 172.21.15.43:0/2160601857 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc7080a3d20 0x7fc7080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.108+0000 7fc71f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b0ae80 0x555b86b13430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.108+0000 7fc71f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b0ae80 0x555b86b13430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34310/0 (socket says 172.21.15.43:34310) 2024-04-21T08:42:47.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.108+0000 7fc71f7fe640 1 -- 172.21.15.43:0/2070496602 learned_addr learned my addr 172.21.15.43:0/2070496602 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.109+0000 7fc71ffff640 1 --2- 172.21.15.43:0/2160601857 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc7080a3d20 0x7fc7080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.109+0000 7fc71effd640 1 --2- 172.21.15.43:0/2070496602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b86b0a4d0 0x555b86b0a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.109+0000 7fc71effd640 1 --2- 172.21.15.43:0/2070496602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b86b13a00 0x555b86b15e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.109+0000 7fc71f7fe640 1 -- 172.21.15.43:0/2070496602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b86b0a4d0 msgr2=0x555b86b0a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.109+0000 7fc71f7fe640 1 --2- 172.21.15.43:0/2070496602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b86b0a4d0 0x555b86b0a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.109+0000 7fc71f7fe640 1 -- 172.21.15.43:0/2070496602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b86b13a00 msgr2=0x555b86b15e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.109+0000 7fc71f7fe640 1 --2- 172.21.15.43:0/2070496602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b86b13a00 0x555b86b15e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.109+0000 7fc71cff9640 1 -- 172.21.15.43:0/2160601857 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc7180d0490 con 0x7fc7201985b0 2024-04-21T08:42:47.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.109+0000 7fc71f7fe640 1 -- 172.21.15.43:0/2070496602 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b869fe1b0 con 0x555b86b0ae80 2024-04-21T08:42:47.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.109+0000 7fc71effd640 1 --2- 172.21.15.43:0/2070496602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b86b13a00 0x555b86b15e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:47.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.109+0000 7fc71ffff640 1 --2- 172.21.15.43:0/2160601857 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc7080a3d20 0x7fc7080a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc7180cd960 tx=0x7fc71806db40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.109+0000 7fc71f7fe640 1 --2- 172.21.15.43:0/2070496602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b0ae80 0x555b86b13430 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fc710008e70 tx=0x7fc71000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f6b7ca275ed809 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.109+0000 7fc705ffb640 1 -- 172.21.15.43:0/2070496602 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71000f830 con 0x555b86b0ae80 2024-04-21T08:42:47.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.109+0000 7fc705ffb640 1 -- 172.21.15.43:0/2070496602 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc710018480 con 0x555b86b0ae80 2024-04-21T08:42:47.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.109+0000 7fc705ffb640 1 -- 172.21.15.43:0/2070496602 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71000e5b0 con 0x555b86b0ae80 2024-04-21T08:42:47.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.110+0000 7fc7289e2740 1 -- 172.21.15.43:0/2070496602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b0ae80 msgr2=0x555b86b13430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.110+0000 7fc7289e2740 1 --2- 172.21.15.43:0/2070496602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b0ae80 0x555b86b13430 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fc710008e70 tx=0x7fc71000d710 comp rx=0 tx=0).stop 2024-04-21T08:42:47.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.110+0000 7fc7289e2740 1 -- 172.21.15.43:0/2070496602 shutdown_connections 2024-04-21T08:42:47.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.110+0000 7fc7289e2740 1 --2- 172.21.15.43:0/2070496602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b86b13a00 0x555b86b15e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.110+0000 7fc7289e2740 1 --2- 172.21.15.43:0/2070496602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b0ae80 0x555b86b13430 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.110+0000 7fc7289e2740 1 --2- 172.21.15.43:0/2070496602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b86b0a4d0 0x555b86b0a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.110+0000 7fc7289e2740 1 -- 172.21.15.43:0/2070496602 >> 172.21.15.43:0/2070496602 conn(0x555b86b04ba0 msgr2=0x555b86b04f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.110+0000 7fc7289e2740 1 -- 172.21.15.43:0/2070496602 shutdown_connections 2024-04-21T08:42:47.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.110+0000 7fc7289e2740 1 -- 172.21.15.43:0/2070496602 wait complete. 2024-04-21T08:42:47.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.110+0000 7fc7289e2740 1 -- start start 2024-04-21T08:42:47.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.111+0000 7fc7289e2740 1 Processor -- start 2024-04-21T08:42:47.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.111+0000 7fc7289e2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b86b0a4d0 0x555b86b13380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.111+0000 7fc7289e2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b86b0ae80 0x555b86b10710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.111+0000 7fc7289e2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b13a00 0x555b86b10d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.111+0000 7fc7289e2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555b86b16970 con 0x555b86b13a00 2024-04-21T08:42:47.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.111+0000 7fc7289e2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555b86a28500 con 0x555b86b0a4d0 2024-04-21T08:42:47.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.111+0000 7fc7289e2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555b86b16af0 con 0x555b86b0ae80 2024-04-21T08:42:47.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.111+0000 7fc7289e2740 10 client.0 _ll_register_callbacks cb 0x555b869f6b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:47.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.111+0000 7fc71effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b13a00 0x555b86b10d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.111+0000 7fc71effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b13a00 0x555b86b10d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34314/0 (socket says 172.21.15.43:34314) 2024-04-21T08:42:47.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.111+0000 7fc71effd640 1 -- 172.21.15.43:0/1698907626 learned_addr learned my addr 172.21.15.43:0/1698907626 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.111+0000 7fc71f7fe640 1 --2- 172.21.15.43:0/1698907626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b86b0ae80 0x555b86b10710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.111+0000 7fc71effd640 1 --2- 172.21.15.43:0/1698907626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b86b0a4d0 0x555b86b13380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.111+0000 7fc71effd640 1 -- 172.21.15.43:0/1698907626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b86b0ae80 msgr2=0x555b86b10710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.111+0000 7fc71effd640 1 --2- 172.21.15.43:0/1698907626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b86b0ae80 0x555b86b10710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.111+0000 7fc71effd640 1 -- 172.21.15.43:0/1698907626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b86b0a4d0 msgr2=0x555b86b13380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.111+0000 7fc71effd640 1 --2- 172.21.15.43:0/1698907626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b86b0a4d0 0x555b86b13380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.111+0000 7fc71effd640 1 -- 172.21.15.43:0/1698907626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b867813f0 con 0x555b86b13a00 2024-04-21T08:42:47.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.111+0000 7fc71effd640 1 --2- 172.21.15.43:0/1698907626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b13a00 0x555b86b10d20 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fc714005ec0 tx=0x7fc714009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3dc6063c32580a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.111+0000 7fc6feffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:47.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.112+0000 7fc6feffd640 1 -- 172.21.15.43:0/1698907626 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7140053d0 con 0x555b86b13a00 2024-04-21T08:42:47.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.112+0000 7fc6feffd640 1 -- 172.21.15.43:0/1698907626 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc714005570 con 0x555b86b13a00 2024-04-21T08:42:47.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.112+0000 7fc7289e2740 1 -- 172.21.15.43:0/1698907626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc710018480 con 0x555b86b13a00 2024-04-21T08:42:47.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.112+0000 7fc6feffd640 1 -- 172.21.15.43:0/1698907626 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc714054050 con 0x555b86b13a00 2024-04-21T08:42:47.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.112+0000 7fc7289e2740 10 client.17313 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:47.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.112+0000 7fc7289e2740 1 -- 172.21.15.43:0/1698907626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555b8684b7d0 con 0x555b86b13a00 2024-04-21T08:42:47.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.112+0000 7fc6feffd640 1 -- 172.21.15.43:0/1698907626 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc710018480 con 0x555b86b13a00 2024-04-21T08:42:47.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.112+0000 7fc6feffd640 1 -- 172.21.15.43:0/1698907626 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc7140038f0 con 0x555b86b13a00 2024-04-21T08:42:47.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.112+0000 7fc7289e2740 10 client.17313 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:47.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.112+0000 7fc7289e2740 10 client.17313 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:42:47.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.112+0000 7fc7289e2740 20 client.17313 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:47.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.112+0000 7fc7289e2740 1 --2- 172.21.15.43:0/1698907626 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x555b86b23ba0 0x555b86b25f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.112+0000 7fc7289e2740 4 client.17313 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:47.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.112+0000 7fc7289e2740 1 -- 172.21.15.43:0/1698907626 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b86a0e930 con 0x555b86b23ba0 2024-04-21T08:42:47.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.112+0000 7fc71f7fe640 1 --2- 172.21.15.43:0/1698907626 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x555b86b23ba0 0x555b86b25f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.113+0000 7fc6feffd640 10 client.17313 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:42:47.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.113+0000 7fc71f7fe640 1 --2- 172.21.15.43:0/1698907626 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x555b86b23ba0 0x555b86b25f90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9c59d8772bc2b58 server_cookie=8af335949ebad47f in_seq=0 out_seq=0 2024-04-21T08:42:47.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.114+0000 7fc6feffd640 1 -- 172.21.15.43:0/1698907626 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555b86a0e930 con 0x555b86b23ba0 2024-04-21T08:42:47.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.114+0000 7fc6feffd640 10 client.17313 handle_command_reply: tid=0 2024-04-21T08:42:47.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.114+0000 7fc7289e2740 1 client.17313 shutdown 2024-04-21T08:42:47.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.114+0000 7fc7289e2740 1 -- 172.21.15.43:0/1698907626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b13a00 msgr2=0x555b86b10d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.114+0000 7fc7289e2740 1 --2- 172.21.15.43:0/1698907626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b13a00 0x555b86b10d20 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fc714005ec0 tx=0x7fc714009260 comp rx=0 tx=0).stop 2024-04-21T08:42:47.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.114+0000 7fc7289e2740 1 -- 172.21.15.43:0/1698907626 shutdown_connections 2024-04-21T08:42:47.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.114+0000 7fc7289e2740 1 --2- 172.21.15.43:0/1698907626 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x555b86b23ba0 0x555b86b25f90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.114+0000 7fc7289e2740 1 --2- 172.21.15.43:0/1698907626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b13a00 0x555b86b10d20 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.114+0000 7fc7289e2740 1 --2- 172.21.15.43:0/1698907626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b86b0ae80 0x555b86b10710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.114+0000 7fc7289e2740 1 --2- 172.21.15.43:0/1698907626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b86b0a4d0 0x555b86b13380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.114+0000 7fc7289e2740 1 -- 172.21.15.43:0/1698907626 >> 172.21.15.43:0/1698907626 conn(0x555b86b04ba0 msgr2=0x555b86b068f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.114+0000 7fc6feffd640 0 client.17313 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:42:47.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.114+0000 7fc7289e2740 1 -- 172.21.15.43:0/1698907626 shutdown_connections 2024-04-21T08:42:47.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.114+0000 7fc7289e2740 1 -- 172.21.15.43:0/1698907626 wait complete. 2024-04-21T08:42:47.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.114+0000 7fc7289e2740 20 client.17313 trim_cache size 0 max 16384 2024-04-21T08:42:47.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.127+0000 7fc7289e2740 1 Processor -- start 2024-04-21T08:42:47.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.127+0000 7fc7289e2740 1 -- start start 2024-04-21T08:42:47.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.127+0000 7fc7289e2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b86b0a4d0 0x555b86b1f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.127+0000 7fc7289e2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b86b0ae80 0x555b86b12e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.127+0000 7fc7289e2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b13a00 0x555b86b13380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.127+0000 7fc7289e2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc7201af390 con 0x555b86b13a00 2024-04-21T08:42:47.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.127+0000 7fc7289e2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc7201b0ed0 con 0x555b86b0ae80 2024-04-21T08:42:47.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.127+0000 7fc7289e2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc7201afa60 con 0x555b86b0a4d0 2024-04-21T08:42:47.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.128+0000 7fc71effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b13a00 0x555b86b13380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.128+0000 7fc71effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b13a00 0x555b86b13380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34324/0 (socket says 172.21.15.43:34324) 2024-04-21T08:42:47.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.128+0000 7fc71effd640 1 -- 172.21.15.43:0/1314514185 learned_addr learned my addr 172.21.15.43:0/1314514185 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.128+0000 7fc71f7fe640 1 --2- 172.21.15.43:0/1314514185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b86b0ae80 0x555b86b12e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.128+0000 7fc71effd640 1 -- 172.21.15.43:0/1314514185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b86b0a4d0 msgr2=0x555b86b1f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.128+0000 7fc71effd640 1 --2- 172.21.15.43:0/1314514185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b86b0a4d0 0x555b86b1f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.128+0000 7fc71effd640 1 -- 172.21.15.43:0/1314514185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b86b0ae80 msgr2=0x555b86b12e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.128+0000 7fc71effd640 1 --2- 172.21.15.43:0/1314514185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b86b0ae80 0x555b86b12e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.128+0000 7fc71effd640 1 -- 172.21.15.43:0/1314514185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b86b26300 con 0x555b86b13a00 2024-04-21T08:42:47.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.128+0000 7fc71f7fe640 1 --2- 172.21.15.43:0/1314514185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b86b0ae80 0x555b86b12e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:47.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.128+0000 7fc71effd640 1 --2- 172.21.15.43:0/1314514185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b13a00 0x555b86b13380 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fc714009850 tx=0x7fc71402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e187de43b80b213 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.128+0000 7fc7057fa640 1 -- 172.21.15.43:0/1314514185 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc714051020 con 0x555b86b13a00 2024-04-21T08:42:47.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.128+0000 7fc7057fa640 1 -- 172.21.15.43:0/1314514185 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc714042440 con 0x555b86b13a00 2024-04-21T08:42:47.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.128+0000 7fc7057fa640 1 -- 172.21.15.43:0/1314514185 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7140429e0 con 0x555b86b13a00 2024-04-21T08:42:47.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.129+0000 7fc7289e2740 1 -- 172.21.15.43:0/1314514185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b13a00 msgr2=0x555b86b13380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.129+0000 7fc7289e2740 1 --2- 172.21.15.43:0/1314514185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b13a00 0x555b86b13380 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fc714009850 tx=0x7fc71402bc30 comp rx=0 tx=0).stop 2024-04-21T08:42:47.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.129+0000 7fc7289e2740 1 -- 172.21.15.43:0/1314514185 shutdown_connections 2024-04-21T08:42:47.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.129+0000 7fc7289e2740 1 --2- 172.21.15.43:0/1314514185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b13a00 0x555b86b13380 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.129+0000 7fc7289e2740 1 --2- 172.21.15.43:0/1314514185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b86b0ae80 0x555b86b12e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.129+0000 7fc7289e2740 1 --2- 172.21.15.43:0/1314514185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b86b0a4d0 0x555b86b1f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.129+0000 7fc7289e2740 1 -- 172.21.15.43:0/1314514185 >> 172.21.15.43:0/1314514185 conn(0x555b86b04ba0 msgr2=0x555b86b03c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.129+0000 7fc7289e2740 1 -- 172.21.15.43:0/1314514185 shutdown_connections 2024-04-21T08:42:47.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.129+0000 7fc7289e2740 1 -- 172.21.15.43:0/1314514185 wait complete. 2024-04-21T08:42:47.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.129+0000 7fc7289e2740 1 -- start start 2024-04-21T08:42:47.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.129+0000 7fc7289e2740 1 Processor -- start 2024-04-21T08:42:47.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc7289e2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b86b0a4d0 0x555b86b1c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc7289e2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b86b0ae80 0x555b86b17480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc7289e2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b13a00 0x555b86b179c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc7289e2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc7201b0800 con 0x555b86b13a00 2024-04-21T08:42:47.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc7289e2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc7201b0130 con 0x555b86b0a4d0 2024-04-21T08:42:47.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc7289e2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc7201ad370 con 0x555b86b0ae80 2024-04-21T08:42:47.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc7289e2740 10 client.0 _ll_register_callbacks cb 0x7fc71402caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:47.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc71effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b13a00 0x555b86b179c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc71effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b13a00 0x555b86b179c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34326/0 (socket says 172.21.15.43:34326) 2024-04-21T08:42:47.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc71effd640 1 -- 172.21.15.43:0/2142392525 learned_addr learned my addr 172.21.15.43:0/2142392525 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc71effd640 1 --2- 172.21.15.43:0/2142392525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b86b0a4d0 0x555b86b1c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc71f7fe640 1 --2- 172.21.15.43:0/2142392525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b86b0ae80 0x555b86b17480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc71effd640 1 -- 172.21.15.43:0/2142392525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b86b0ae80 msgr2=0x555b86b17480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc71effd640 1 --2- 172.21.15.43:0/2142392525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b86b0ae80 0x555b86b17480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc71effd640 1 -- 172.21.15.43:0/2142392525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b86b0a4d0 msgr2=0x555b86b1c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc71effd640 1 --2- 172.21.15.43:0/2142392525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b86b0a4d0 0x555b86b1c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc71effd640 1 -- 172.21.15.43:0/2142392525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b86b181a0 con 0x555b86b13a00 2024-04-21T08:42:47.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc71f7fe640 1 --2- 172.21.15.43:0/2142392525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b86b0ae80 0x555b86b17480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:47.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc71effd640 1 --2- 172.21.15.43:0/2142392525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b13a00 0x555b86b179c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fc714009900 tx=0x7fc71402c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244c55078d941333 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc6feffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:47.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc6feffd640 1 -- 172.21.15.43:0/2142392525 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc714051040 con 0x555b86b13a00 2024-04-21T08:42:47.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc6feffd640 1 -- 172.21.15.43:0/2142392525 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc7140055d0 con 0x555b86b13a00 2024-04-21T08:42:47.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc7289e2740 1 -- 172.21.15.43:0/2142392525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc714042440 con 0x555b86b13a00 2024-04-21T08:42:47.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc6feffd640 1 -- 172.21.15.43:0/2142392525 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc714042900 con 0x555b86b13a00 2024-04-21T08:42:47.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.130+0000 7fc7289e2740 10 client.17319 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:47.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.131+0000 7fc7289e2740 1 -- 172.21.15.43:0/2142392525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555b86b8a000 con 0x555b86b13a00 2024-04-21T08:42:47.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.131+0000 7fc6feffd640 1 -- 172.21.15.43:0/2142392525 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc714042440 con 0x555b86b13a00 2024-04-21T08:42:47.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.131+0000 7fc6feffd640 1 -- 172.21.15.43:0/2142392525 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc714004750 con 0x555b86b13a00 2024-04-21T08:42:47.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.131+0000 7fc7289e2740 10 client.17319 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:47.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.131+0000 7fc7289e2740 10 client.17319 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:42:47.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.131+0000 7fc7289e2740 20 client.17319 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:47.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.131+0000 7fc7289e2740 1 --2- 172.21.15.43:0/2142392525 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x555b86b8a660 0x555b86b8ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.131+0000 7fc7289e2740 4 client.17319 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:42:47.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.131+0000 7fc7289e2740 1 -- 172.21.15.43:0/2142392525 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b86a7cc10 con 0x555b86b8a660 2024-04-21T08:42:47.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.131+0000 7fc71f7fe640 1 --2- 172.21.15.43:0/2142392525 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x555b86b8a660 0x555b86b8ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.132+0000 7fc6feffd640 10 client.17319 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:42:47.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.132+0000 7fc71f7fe640 1 --2- 172.21.15.43:0/2142392525 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x555b86b8a660 0x555b86b8ca50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f1f4241fdf129b92 server_cookie=28d0596269f7eb23 in_seq=0 out_seq=0 2024-04-21T08:42:47.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.132+0000 7fc6feffd640 1 -- 172.21.15.43:0/2142392525 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b86a7cc10 con 0x555b86b8a660 2024-04-21T08:42:47.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.132+0000 7fc6feffd640 10 client.17319 handle_command_reply: tid=0 2024-04-21T08:42:47.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.132+0000 7fc7289e2740 1 client.17319 shutdown 2024-04-21T08:42:47.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.132+0000 7fc7289e2740 1 -- 172.21.15.43:0/2142392525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b13a00 msgr2=0x555b86b179c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.132+0000 7fc7289e2740 1 --2- 172.21.15.43:0/2142392525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b13a00 0x555b86b179c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fc714009900 tx=0x7fc71402c2b0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.132+0000 7fc7289e2740 1 -- 172.21.15.43:0/2142392525 shutdown_connections 2024-04-21T08:42:47.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.132+0000 7fc7289e2740 1 --2- 172.21.15.43:0/2142392525 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x555b86b8a660 0x555b86b8ca50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.132+0000 7fc7289e2740 1 --2- 172.21.15.43:0/2142392525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b86b13a00 0x555b86b179c0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.132+0000 7fc7289e2740 1 --2- 172.21.15.43:0/2142392525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b86b0ae80 0x555b86b17480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.132+0000 7fc6feffd640 0 client.17319 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:42:47.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.132+0000 7fc7289e2740 1 --2- 172.21.15.43:0/2142392525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b86b0a4d0 0x555b86b1c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.132+0000 7fc7289e2740 1 -- 172.21.15.43:0/2142392525 >> 172.21.15.43:0/2142392525 conn(0x555b86b04ba0 msgr2=0x555b86b03370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.132+0000 7fc7289e2740 1 -- 172.21.15.43:0/2142392525 shutdown_connections 2024-04-21T08:42:47.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.132+0000 7fc7289e2740 1 -- 172.21.15.43:0/2142392525 wait complete. 2024-04-21T08:42:47.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.132+0000 7fc7289e2740 20 client.17319 trim_cache size 0 max 16384 2024-04-21T08:42:47.139 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:42:47.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.133+0000 7fc725dfb640 1 -- 172.21.15.43:0/2160601857 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc7080a3d20 msgr2=0x7fc7080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.133+0000 7fc725dfb640 1 --2- 172.21.15.43:0/2160601857 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc7080a3d20 0x7fc7080a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc7180cd960 tx=0x7fc71806db40 comp rx=0 tx=0).stop 2024-04-21T08:42:47.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.133+0000 7fc725dfb640 1 -- 172.21.15.43:0/2160601857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7201985b0 msgr2=0x7fc720198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.133+0000 7fc725dfb640 1 --2- 172.21.15.43:0/2160601857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7201985b0 0x7fc720198a60 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fc71800d750 tx=0x7fc71800dc10 comp rx=0 tx=0).stop 2024-04-21T08:42:47.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.133+0000 7fc725dfb640 1 -- 172.21.15.43:0/2160601857 shutdown_connections 2024-04-21T08:42:47.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.133+0000 7fc725dfb640 1 --2- 172.21.15.43:0/2160601857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7201985b0 0x7fc720198a60 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.133+0000 7fc725dfb640 1 --2- 172.21.15.43:0/2160601857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc720107260 0x7fc72019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.133+0000 7fc725dfb640 1 --2- 172.21.15.43:0/2160601857 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc7080a3d20 0x7fc7080a61e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.133+0000 7fc725dfb640 1 --2- 172.21.15.43:0/2160601857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc720102190 0x7fc72019e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.133+0000 7fc725dfb640 1 -- 172.21.15.43:0/2160601857 >> 172.21.15.43:0/2160601857 conn(0x7fc7200fd990 msgr2=0x7fc7200fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.133+0000 7fc725dfb640 1 -- 172.21.15.43:0/2160601857 shutdown_connections 2024-04-21T08:42:47.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.133+0000 7fc725dfb640 1 -- 172.21.15.43:0/2160601857 wait complete. 2024-04-21T08:42:47.143 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:42:47.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.247+0000 7f3661474640 1 -- 172.21.15.43:0/3902930873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365c102190 msgr2=0x7f365c102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.247+0000 7f3661474640 1 --2- 172.21.15.43:0/3902930873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365c102190 0x7f365c102570 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f3654009900 tx=0x7f365402bc30 comp rx=0 tx=0).stop 2024-04-21T08:42:47.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.247+0000 7f3661474640 1 -- 172.21.15.43:0/3902930873 shutdown_connections 2024-04-21T08:42:47.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.247+0000 7f3661474640 1 --2- 172.21.15.43:0/3902930873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f365c107260 0x7f365c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.247+0000 7f3661474640 1 --2- 172.21.15.43:0/3902930873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f365c102b40 0x7f365c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.247+0000 7f3661474640 1 --2- 172.21.15.43:0/3902930873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365c102190 0x7f365c102570 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.247+0000 7f3661474640 1 -- 172.21.15.43:0/3902930873 >> 172.21.15.43:0/3902930873 conn(0x7f365c0793a0 msgr2=0x7f365c0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.247+0000 7f3661474640 1 -- 172.21.15.43:0/3902930873 shutdown_connections 2024-04-21T08:42:47.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.247+0000 7f3661474640 1 -- 172.21.15.43:0/3902930873 wait complete. 2024-04-21T08:42:47.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.248+0000 7f3661474640 1 Processor -- start 2024-04-21T08:42:47.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.248+0000 7f3661474640 1 -- start start 2024-04-21T08:42:47.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.248+0000 7f3661474640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f365c102190 0x7f365c19e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.248+0000 7f3661474640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f365c107260 0x7f365c19e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.248+0000 7f3661474640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365c1985c0 0x7f365c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.248+0000 7f3661474640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f365c112220 con 0x7f365c1985c0 2024-04-21T08:42:47.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.248+0000 7f3661474640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1120a0 con 0x7f365c102190 2024-04-21T08:42:47.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.248+0000 7f3661474640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1123a0 con 0x7f365c107260 2024-04-21T08:42:47.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.248+0000 7f365b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365c1985c0 0x7f365c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.248+0000 7f365b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365c1985c0 0x7f365c198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34350/0 (socket says 172.21.15.43:34350) 2024-04-21T08:42:47.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.248+0000 7f365b7fe640 1 -- 172.21.15.43:0/3491097668 learned_addr learned my addr 172.21.15.43:0/3491097668 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.249+0000 7f365affd640 1 --2- 172.21.15.43:0/3491097668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f365c102190 0x7f365c19e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.249+0000 7f365a7fc640 1 --2- 172.21.15.43:0/3491097668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f365c107260 0x7f365c19e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.249+0000 7f365b7fe640 1 -- 172.21.15.43:0/3491097668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f365c107260 msgr2=0x7f365c19e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.249+0000 7f365b7fe640 1 --2- 172.21.15.43:0/3491097668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f365c107260 0x7f365c19e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.249+0000 7f365b7fe640 1 -- 172.21.15.43:0/3491097668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f365c102190 msgr2=0x7f365c19e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.249+0000 7f365b7fe640 1 --2- 172.21.15.43:0/3491097668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f365c102190 0x7f365c19e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.249+0000 7f365b7fe640 1 -- 172.21.15.43:0/3491097668 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365c1990e0 con 0x7f365c1985c0 2024-04-21T08:42:47.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.249+0000 7f365affd640 1 --2- 172.21.15.43:0/3491097668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f365c102190 0x7f365c19e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:47.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.249+0000 7f365b7fe640 1 --2- 172.21.15.43:0/3491097668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365c1985c0 0x7f365c198a70 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f364c00cad0 tx=0x7f364c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.249+0000 7f3653fff640 1 -- 172.21.15.43:0/3491097668 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c00ae20 con 0x7f365c1985c0 2024-04-21T08:42:47.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.249+0000 7f3653fff640 1 -- 172.21.15.43:0/3491097668 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f364c005020 con 0x7f365c1985c0 2024-04-21T08:42:47.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.249+0000 7f3661474640 1 -- 172.21.15.43:0/3491097668 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f365c1993d0 con 0x7f365c1985c0 2024-04-21T08:42:47.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.249+0000 7f3653fff640 1 -- 172.21.15.43:0/3491097668 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c00f6b0 con 0x7f365c1985c0 2024-04-21T08:42:47.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.249+0000 7f3661474640 1 -- 172.21.15.43:0/3491097668 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f365c1a5620 con 0x7f365c1985c0 2024-04-21T08:42:47.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.251+0000 7f3653fff640 1 -- 172.21.15.43:0/3491097668 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f364c0051d0 con 0x7f365c1985c0 2024-04-21T08:42:47.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.252+0000 7f366406d740 1 Processor -- start 2024-04-21T08:42:47.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.252+0000 7f366406d740 1 -- start start 2024-04-21T08:42:47.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.252+0000 7f366406d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643e50b04d0 0x5643e50b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.252+0000 7f366406d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b0e80 0x5643e50b9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.252+0000 7f366406d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643e50b9a00 0x5643e50bbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.252+0000 7f366406d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5643e50151f0 con 0x5643e50b0e80 2024-04-21T08:42:47.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.252+0000 7f366406d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5643e4fb24b0 con 0x5643e50b9a00 2024-04-21T08:42:47.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.252+0000 7f366406d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5643e50111b0 con 0x5643e50b04d0 2024-04-21T08:42:47.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.252+0000 7f365affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b0e80 0x5643e50b9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.252+0000 7f365affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b0e80 0x5643e50b9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34360/0 (socket says 172.21.15.43:34360) 2024-04-21T08:42:47.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.252+0000 7f365affd640 1 -- 172.21.15.43:0/2834477187 learned_addr learned my addr 172.21.15.43:0/2834477187 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.252+0000 7f3653fff640 1 --2- 172.21.15.43:0/3491097668 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f36480a3d20 0x7f36480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.252+0000 7f365a7fc640 1 --2- 172.21.15.43:0/2834477187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643e50b9a00 0x5643e50bbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.253+0000 7f365affd640 1 -- 172.21.15.43:0/2834477187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643e50b04d0 msgr2=0x5643e50b08b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:47.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.253+0000 7f365affd640 1 --2- 172.21.15.43:0/2834477187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643e50b04d0 0x5643e50b08b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.253+0000 7f365affd640 1 -- 172.21.15.43:0/2834477187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643e50b9a00 msgr2=0x5643e50bbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.253+0000 7f365b7fe640 1 --2- 172.21.15.43:0/3491097668 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f36480a3d20 0x7f36480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.253+0000 7f365affd640 1 --2- 172.21.15.43:0/2834477187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643e50b9a00 0x5643e50bbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.253+0000 7f365affd640 1 -- 172.21.15.43:0/2834477187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643e4fa41b0 con 0x5643e50b0e80 2024-04-21T08:42:47.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.253+0000 7f3653fff640 1 -- 172.21.15.43:0/3491097668 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f364c0d0c60 con 0x7f365c1985c0 2024-04-21T08:42:47.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.253+0000 7f365affd640 1 --2- 172.21.15.43:0/2834477187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b0e80 0x5643e50b9430 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f3654009900 tx=0x7f365402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15aa439063f562d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.253+0000 7f36517fa640 1 -- 172.21.15.43:0/2834477187 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654044020 con 0x5643e50b0e80 2024-04-21T08:42:47.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.253+0000 7f365b7fe640 1 --2- 172.21.15.43:0/3491097668 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f36480a3d20 0x7f36480a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f364c09c530 tx=0x7f364c06f320 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.253+0000 7f36517fa640 1 -- 172.21.15.43:0/2834477187 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f365402cc90 con 0x5643e50b0e80 2024-04-21T08:42:47.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.253+0000 7f36517fa640 1 -- 172.21.15.43:0/2834477187 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36540046c0 con 0x5643e50b0e80 2024-04-21T08:42:47.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.254+0000 7f366406d740 1 -- 172.21.15.43:0/2834477187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b0e80 msgr2=0x5643e50b9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.254+0000 7f366406d740 1 --2- 172.21.15.43:0/2834477187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b0e80 0x5643e50b9430 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f3654009900 tx=0x7f365402c650 comp rx=0 tx=0).stop 2024-04-21T08:42:47.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.254+0000 7f366406d740 1 -- 172.21.15.43:0/2834477187 shutdown_connections 2024-04-21T08:42:47.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.254+0000 7f366406d740 1 --2- 172.21.15.43:0/2834477187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643e50b9a00 0x5643e50bbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.254+0000 7f366406d740 1 --2- 172.21.15.43:0/2834477187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b0e80 0x5643e50b9430 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.254+0000 7f366406d740 1 --2- 172.21.15.43:0/2834477187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643e50b04d0 0x5643e50b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.254+0000 7f366406d740 1 -- 172.21.15.43:0/2834477187 >> 172.21.15.43:0/2834477187 conn(0x5643e50aaba0 msgr2=0x5643e50aaf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.254+0000 7f366406d740 1 -- 172.21.15.43:0/2834477187 shutdown_connections 2024-04-21T08:42:47.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.254+0000 7f366406d740 1 -- 172.21.15.43:0/2834477187 wait complete. 2024-04-21T08:42:47.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.254+0000 7f366406d740 1 -- start start 2024-04-21T08:42:47.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.254+0000 7f366406d740 1 Processor -- start 2024-04-21T08:42:47.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.255+0000 7f366406d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643e50b04d0 0x5643e50b92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.255+0000 7f366406d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b0e80 0x5643e50b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.255+0000 7f366406d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643e50b9a00 0x5643e50b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.255+0000 7f366406d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5643e50bc910 con 0x5643e50b0e80 2024-04-21T08:42:47.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.255+0000 7f366406d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5643e4fce500 con 0x5643e50b04d0 2024-04-21T08:42:47.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.255+0000 7f366406d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5643e50bca90 con 0x5643e50b9a00 2024-04-21T08:42:47.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.255+0000 7f366406d740 10 client.0 _ll_register_callbacks cb 0x5643e4f9cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:47.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.255+0000 7f365affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b0e80 0x5643e50b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.255+0000 7f365affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b0e80 0x5643e50b6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34374/0 (socket says 172.21.15.43:34374) 2024-04-21T08:42:47.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.255+0000 7f365affd640 1 -- 172.21.15.43:0/3492658398 learned_addr learned my addr 172.21.15.43:0/3492658398 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.255+0000 7f365a7fc640 1 --2- 172.21.15.43:0/3492658398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643e50b04d0 0x5643e50b92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.255+0000 7f365a7fc640 1 --2- 172.21.15.43:0/3492658398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643e50b9a00 0x5643e50b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.255+0000 7f365affd640 1 -- 172.21.15.43:0/3492658398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643e50b9a00 msgr2=0x5643e50b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.255+0000 7f365affd640 1 --2- 172.21.15.43:0/3492658398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643e50b9a00 0x5643e50b6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.255+0000 7f365affd640 1 -- 172.21.15.43:0/3492658398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643e50b04d0 msgr2=0x5643e50b92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.255+0000 7f365affd640 1 --2- 172.21.15.43:0/3492658398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643e50b04d0 0x5643e50b92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.255+0000 7f365affd640 1 -- 172.21.15.43:0/3492658398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643e4d273f0 con 0x5643e50b0e80 2024-04-21T08:42:47.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.255+0000 7f365a7fc640 1 --2- 172.21.15.43:0/3492658398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643e50b9a00 0x5643e50b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:47.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.255+0000 7f365affd640 1 --2- 172.21.15.43:0/3492658398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b0e80 0x5643e50b6630 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f365402c240 tx=0x7f3654005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dcfb89f6986636e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.255+0000 7f36427fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:47.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.255+0000 7f36427fc640 1 -- 172.21.15.43:0/3492658398 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365404f070 con 0x5643e50b0e80 2024-04-21T08:42:47.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.255+0000 7f36427fc640 1 -- 172.21.15.43:0/3492658398 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f36540037f0 con 0x5643e50b0e80 2024-04-21T08:42:47.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.256+0000 7f366406d740 1 -- 172.21.15.43:0/3492658398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f365402cc90 con 0x5643e50b0e80 2024-04-21T08:42:47.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.256+0000 7f36427fc640 1 -- 172.21.15.43:0/3492658398 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f365405b050 con 0x5643e50b0e80 2024-04-21T08:42:47.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.256+0000 7f366406d740 10 client.17331 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:47.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.256+0000 7f366406d740 1 -- 172.21.15.43:0/3492658398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5643e4df17d0 con 0x5643e50b0e80 2024-04-21T08:42:47.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.256+0000 7f36427fc640 1 -- 172.21.15.43:0/3492658398 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f365402cc90 con 0x5643e50b0e80 2024-04-21T08:42:47.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.256+0000 7f36427fc640 1 -- 172.21.15.43:0/3492658398 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3654040a40 con 0x5643e50b0e80 2024-04-21T08:42:47.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.256+0000 7f366406d740 10 client.17331 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.256+0000 7f366406d740 10 client.17331 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:42:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.256+0000 7f366406d740 20 client.17331 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.256+0000 7f366406d740 1 --2- 172.21.15.43:0/3492658398 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5643e50c9a10 0x5643e50cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.256+0000 7f366406d740 4 client.17331 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.256+0000 7f366406d740 1 -- 172.21.15.43:0/3492658398 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643e4fb4930 con 0x5643e50c9a10 2024-04-21T08:42:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.256+0000 7f365a7fc640 1 --2- 172.21.15.43:0/3492658398 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5643e50c9a10 0x5643e50cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.257+0000 7f36427fc640 10 client.17331 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:42:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.257+0000 7f365a7fc640 1 --2- 172.21.15.43:0/3492658398 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5643e50c9a10 0x5643e50cbe00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ca1dbfe3a5ce4212 server_cookie=bafb6aa7344357e8 in_seq=0 out_seq=0 2024-04-21T08:42:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.258+0000 7f36427fc640 1 -- 172.21.15.43:0/3492658398 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5643e4fb4930 con 0x5643e50c9a10 2024-04-21T08:42:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.258+0000 7f36427fc640 10 client.17331 handle_command_reply: tid=0 2024-04-21T08:42:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.258+0000 7f366406d740 1 client.17331 shutdown 2024-04-21T08:42:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.258+0000 7f366406d740 1 -- 172.21.15.43:0/3492658398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b0e80 msgr2=0x5643e50b6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.258+0000 7f366406d740 1 --2- 172.21.15.43:0/3492658398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b0e80 0x5643e50b6630 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f365402c240 tx=0x7f3654005410 comp rx=0 tx=0).stop 2024-04-21T08:42:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.258+0000 7f366406d740 1 -- 172.21.15.43:0/3492658398 shutdown_connections 2024-04-21T08:42:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.258+0000 7f366406d740 1 --2- 172.21.15.43:0/3492658398 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5643e50c9a10 0x5643e50cbe00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.258+0000 7f366406d740 1 --2- 172.21.15.43:0/3492658398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643e50b9a00 0x5643e50b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.258+0000 7f36427fc640 0 client.17331 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:42:47.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.258+0000 7f366406d740 1 --2- 172.21.15.43:0/3492658398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b0e80 0x5643e50b6630 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.258+0000 7f366406d740 1 --2- 172.21.15.43:0/3492658398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643e50b04d0 0x5643e50b92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.258+0000 7f366406d740 1 -- 172.21.15.43:0/3492658398 >> 172.21.15.43:0/3492658398 conn(0x5643e50aaba0 msgr2=0x5643e50ac8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.258+0000 7f366406d740 1 -- 172.21.15.43:0/3492658398 shutdown_connections 2024-04-21T08:42:47.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.258+0000 7f366406d740 1 -- 172.21.15.43:0/3492658398 wait complete. 2024-04-21T08:42:47.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.258+0000 7f366406d740 20 client.17331 trim_cache size 0 max 16384 2024-04-21T08:42:47.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.271+0000 7f366406d740 1 Processor -- start 2024-04-21T08:42:47.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.271+0000 7f366406d740 1 -- start start 2024-04-21T08:42:47.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.272+0000 7f366406d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b04d0 0x5643e50c5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.272+0000 7f366406d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643e50b0e80 0x5643e50c20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.272+0000 7f366406d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643e50b9a00 0x5643e50b5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.272+0000 7f366406d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1af530 con 0x5643e50b04d0 2024-04-21T08:42:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.272+0000 7f366406d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1b1070 con 0x5643e50b9a00 2024-04-21T08:42:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.272+0000 7f366406d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1afc00 con 0x5643e50b0e80 2024-04-21T08:42:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.272+0000 7f365a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b04d0 0x5643e50c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.272+0000 7f365a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b04d0 0x5643e50c5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34380/0 (socket says 172.21.15.43:34380) 2024-04-21T08:42:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.272+0000 7f365a7fc640 1 -- 172.21.15.43:0/3574990583 learned_addr learned my addr 172.21.15.43:0/3574990583 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.272+0000 7f365affd640 1 --2- 172.21.15.43:0/3574990583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643e50b0e80 0x5643e50c20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.272+0000 7f365a7fc640 1 -- 172.21.15.43:0/3574990583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643e50b0e80 msgr2=0x5643e50c20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.272+0000 7f365a7fc640 1 --2- 172.21.15.43:0/3574990583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643e50b0e80 0x5643e50c20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.272+0000 7f365a7fc640 1 -- 172.21.15.43:0/3574990583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643e50b9a00 msgr2=0x5643e50b5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.272+0000 7f365a7fc640 1 --2- 172.21.15.43:0/3574990583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643e50b9a00 0x5643e50b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.272+0000 7f365a7fc640 1 -- 172.21.15.43:0/3574990583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643e50b4b80 con 0x5643e50b04d0 2024-04-21T08:42:47.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.272+0000 7f365affd640 1 --2- 172.21.15.43:0/3574990583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643e50b0e80 0x5643e50c20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:47.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.272+0000 7f365a7fc640 1 --2- 172.21.15.43:0/3574990583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b04d0 0x5643e50c5800 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f3644001eb0 tx=0x7f3644013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565e3f77b1ba1fe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.272+0000 7f3650ff9640 1 -- 172.21.15.43:0/3574990583 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364401e470 con 0x5643e50b04d0 2024-04-21T08:42:47.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.272+0000 7f3650ff9640 1 -- 172.21.15.43:0/3574990583 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3644011440 con 0x5643e50b04d0 2024-04-21T08:42:47.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.272+0000 7f3650ff9640 1 -- 172.21.15.43:0/3574990583 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364400f680 con 0x5643e50b04d0 2024-04-21T08:42:47.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.273+0000 7f366406d740 1 -- 172.21.15.43:0/3574990583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b04d0 msgr2=0x5643e50c5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.273+0000 7f366406d740 1 --2- 172.21.15.43:0/3574990583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b04d0 0x5643e50c5800 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f3644001eb0 tx=0x7f3644013830 comp rx=0 tx=0).stop 2024-04-21T08:42:47.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.273+0000 7f366406d740 1 -- 172.21.15.43:0/3574990583 shutdown_connections 2024-04-21T08:42:47.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.273+0000 7f366406d740 1 --2- 172.21.15.43:0/3574990583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643e50b9a00 0x5643e50b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.273+0000 7f366406d740 1 --2- 172.21.15.43:0/3574990583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643e50b0e80 0x5643e50c20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.273+0000 7f366406d740 1 --2- 172.21.15.43:0/3574990583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b04d0 0x5643e50c5800 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.273+0000 7f366406d740 1 -- 172.21.15.43:0/3574990583 >> 172.21.15.43:0/3574990583 conn(0x5643e50aaba0 msgr2=0x5643e50a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.273+0000 7f366406d740 1 -- 172.21.15.43:0/3574990583 shutdown_connections 2024-04-21T08:42:47.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.273+0000 7f366406d740 1 -- 172.21.15.43:0/3574990583 wait complete. 2024-04-21T08:42:47.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.273+0000 7f366406d740 1 -- start start 2024-04-21T08:42:47.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.273+0000 7f366406d740 1 Processor -- start 2024-04-21T08:42:47.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.274+0000 7f366406d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b04d0 0x5643e512c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.274+0000 7f366406d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643e50b0e80 0x5643e512ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.274+0000 7f366406d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643e50b9a00 0x5643e50b8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.274+0000 7f366406d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1b09a0 con 0x5643e50b04d0 2024-04-21T08:42:47.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.274+0000 7f366406d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1b02d0 con 0x5643e50b9a00 2024-04-21T08:42:47.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.274+0000 7f366406d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1ad510 con 0x5643e50b0e80 2024-04-21T08:42:47.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.274+0000 7f366406d740 10 client.0 _ll_register_callbacks cb 0x7f365402cbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:47.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.274+0000 7f365a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b04d0 0x5643e512c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.274+0000 7f365a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b04d0 0x5643e512c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34382/0 (socket says 172.21.15.43:34382) 2024-04-21T08:42:47.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.274+0000 7f365a7fc640 1 -- 172.21.15.43:0/2244353870 learned_addr learned my addr 172.21.15.43:0/2244353870 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.274+0000 7f365affd640 1 --2- 172.21.15.43:0/2244353870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643e50b0e80 0x5643e512ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.274+0000 7f365a7fc640 1 --2- 172.21.15.43:0/2244353870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643e50b9a00 0x5643e50b8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.274+0000 7f365a7fc640 1 -- 172.21.15.43:0/2244353870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643e50b0e80 msgr2=0x5643e512ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.274+0000 7f365a7fc640 1 --2- 172.21.15.43:0/2244353870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643e50b0e80 0x5643e512ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.274+0000 7f365a7fc640 1 -- 172.21.15.43:0/2244353870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643e50b9a00 msgr2=0x5643e50b8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.274+0000 7f365a7fc640 1 --2- 172.21.15.43:0/2244353870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643e50b9a00 0x5643e50b8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.274+0000 7f365a7fc640 1 -- 172.21.15.43:0/2244353870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643e50b8b80 con 0x5643e50b04d0 2024-04-21T08:42:47.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.274+0000 7f365a7fc640 1 --2- 172.21.15.43:0/2244353870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b04d0 0x5643e512c530 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f36440057a0 tx=0x7f3644009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82230dbd8e4c93 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.274+0000 7f36427fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:47.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.274+0000 7f36427fc640 1 -- 172.21.15.43:0/2244353870 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36440117f0 con 0x5643e50b04d0 2024-04-21T08:42:47.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.274+0000 7f36427fc640 1 -- 172.21.15.43:0/2244353870 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f3644011990 con 0x5643e50b04d0 2024-04-21T08:42:47.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.274+0000 7f366406d740 1 -- 172.21.15.43:0/2244353870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3644011440 con 0x5643e50b04d0 2024-04-21T08:42:47.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.275+0000 7f36427fc640 1 -- 172.21.15.43:0/2244353870 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3644005030 con 0x5643e50b04d0 2024-04-21T08:42:47.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.275+0000 7f366406d740 10 client.17337 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:47.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.275+0000 7f366406d740 1 -- 172.21.15.43:0/2244353870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5643e5133ca0 con 0x5643e50b04d0 2024-04-21T08:42:47.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.275+0000 7f36427fc640 1 -- 172.21.15.43:0/2244353870 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3644011440 con 0x5643e50b04d0 2024-04-21T08:42:47.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.275+0000 7f36427fc640 1 -- 172.21.15.43:0/2244353870 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3644004cc0 con 0x5643e50b04d0 2024-04-21T08:42:47.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.275+0000 7f366406d740 10 client.17337 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:47.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.275+0000 7f366406d740 10 client.17337 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:42:47.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.275+0000 7f366406d740 20 client.17337 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:47.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.275+0000 7f366406d740 1 --2- 172.21.15.43:0/2244353870 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5643e50c9a10 0x5643e5136340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.275+0000 7f366406d740 4 client.17337 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:42:47.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.275+0000 7f366406d740 1 -- 172.21.15.43:0/2244353870 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5643e5022c10 con 0x5643e50c9a10 2024-04-21T08:42:47.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.275+0000 7f365affd640 1 --2- 172.21.15.43:0/2244353870 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5643e50c9a10 0x5643e5136340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.276+0000 7f365affd640 1 --2- 172.21.15.43:0/2244353870 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5643e50c9a10 0x5643e5136340 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a774eb023c10ad89 server_cookie=495e74908af4f520 in_seq=0 out_seq=0 2024-04-21T08:42:47.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.276+0000 7f36427fc640 10 client.17337 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:42:47.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.276+0000 7f36427fc640 1 -- 172.21.15.43:0/2244353870 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5643e5022c10 con 0x5643e50c9a10 2024-04-21T08:42:47.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.276+0000 7f36427fc640 10 client.17337 handle_command_reply: tid=0 2024-04-21T08:42:47.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.276+0000 7f366406d740 1 client.17337 shutdown 2024-04-21T08:42:47.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.276+0000 7f366406d740 1 -- 172.21.15.43:0/2244353870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b04d0 msgr2=0x5643e512c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.276+0000 7f366406d740 1 --2- 172.21.15.43:0/2244353870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b04d0 0x5643e512c530 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f36440057a0 tx=0x7f3644009920 comp rx=0 tx=0).stop 2024-04-21T08:42:47.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.276+0000 7f366406d740 1 -- 172.21.15.43:0/2244353870 shutdown_connections 2024-04-21T08:42:47.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.276+0000 7f366406d740 1 --2- 172.21.15.43:0/2244353870 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5643e50c9a10 0x5643e5136340 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.276+0000 7f366406d740 1 --2- 172.21.15.43:0/2244353870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643e50b9a00 0x5643e50b8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.276+0000 7f36427fc640 0 client.17337 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:42:47.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.276+0000 7f366406d740 1 --2- 172.21.15.43:0/2244353870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5643e50b0e80 0x5643e512ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.276+0000 7f366406d740 1 --2- 172.21.15.43:0/2244353870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643e50b04d0 0x5643e512c530 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.276+0000 7f366406d740 1 -- 172.21.15.43:0/2244353870 >> 172.21.15.43:0/2244353870 conn(0x5643e50aaba0 msgr2=0x5643e50b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.276+0000 7f366406d740 1 -- 172.21.15.43:0/2244353870 shutdown_connections 2024-04-21T08:42:47.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.276+0000 7f366406d740 1 -- 172.21.15.43:0/2244353870 wait complete. 2024-04-21T08:42:47.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.276+0000 7f366406d740 20 client.17337 trim_cache size 0 max 16384 2024-04-21T08:42:47.282 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:42:47.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.277+0000 7f3661474640 1 -- 172.21.15.43:0/3491097668 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f36480a3d20 msgr2=0x7f36480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.277+0000 7f3661474640 1 --2- 172.21.15.43:0/3491097668 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f36480a3d20 0x7f36480a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f364c09c530 tx=0x7f364c06f320 comp rx=0 tx=0).stop 2024-04-21T08:42:47.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.277+0000 7f3661474640 1 -- 172.21.15.43:0/3491097668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365c1985c0 msgr2=0x7f365c198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.277+0000 7f3661474640 1 --2- 172.21.15.43:0/3491097668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365c1985c0 0x7f365c198a70 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f364c00cad0 tx=0x7f364c00cf90 comp rx=0 tx=0).stop 2024-04-21T08:42:47.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.277+0000 7f3661474640 1 -- 172.21.15.43:0/3491097668 shutdown_connections 2024-04-21T08:42:47.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.277+0000 7f3661474640 1 --2- 172.21.15.43:0/3491097668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365c1985c0 0x7f365c198a70 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.277+0000 7f3661474640 1 --2- 172.21.15.43:0/3491097668 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f36480a3d20 0x7f36480a61e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.277+0000 7f3661474640 1 --2- 172.21.15.43:0/3491097668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f365c107260 0x7f365c19e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.277+0000 7f3661474640 1 --2- 172.21.15.43:0/3491097668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f365c102190 0x7f365c19e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.277+0000 7f3661474640 1 -- 172.21.15.43:0/3491097668 >> 172.21.15.43:0/3491097668 conn(0x7f365c0793a0 msgr2=0x7f365c0fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.277+0000 7f3661474640 1 -- 172.21.15.43:0/3491097668 shutdown_connections 2024-04-21T08:42:47.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.277+0000 7f3661474640 1 -- 172.21.15.43:0/3491097668 wait complete. 2024-04-21T08:42:47.286 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:47 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2115373615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:42:47.287 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:42:47.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.392+0000 7f9a873bd640 1 -- 172.21.15.43:0/1282612177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a80107260 msgr2=0x7f9a8010ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.392+0000 7f9a873bd640 1 --2- 172.21.15.43:0/1282612177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a80107260 0x7f9a8010ee30 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009ca0 tx=0x7f9a7802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.392+0000 7f9a873bd640 1 -- 172.21.15.43:0/1282612177 shutdown_connections 2024-04-21T08:42:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.392+0000 7f9a873bd640 1 --2- 172.21.15.43:0/1282612177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a80107260 0x7f9a8010ee30 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.392+0000 7f9a873bd640 1 --2- 172.21.15.43:0/1282612177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a80102b40 0x7f9a80106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.392+0000 7f9a873bd640 1 --2- 172.21.15.43:0/1282612177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a80102190 0x7f9a80102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.392+0000 7f9a873bd640 1 -- 172.21.15.43:0/1282612177 >> 172.21.15.43:0/1282612177 conn(0x7f9a800793a0 msgr2=0x7f9a800797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.392+0000 7f9a873bd640 1 -- 172.21.15.43:0/1282612177 shutdown_connections 2024-04-21T08:42:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.392+0000 7f9a873bd640 1 -- 172.21.15.43:0/1282612177 wait complete. 2024-04-21T08:42:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.393+0000 7f9a873bd640 1 Processor -- start 2024-04-21T08:42:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.393+0000 7f9a873bd640 1 -- start start 2024-04-21T08:42:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.393+0000 7f9a873bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a80102190 0x7f9a8019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.393+0000 7f9a873bd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a80107260 0x7f9a8019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.393+0000 7f9a873bd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a801985a0 0x7f9a80198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.393+0000 7f9a873bd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9a80112220 con 0x7f9a80107260 2024-04-21T08:42:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.393+0000 7f9a873bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9a801120a0 con 0x7f9a80102190 2024-04-21T08:42:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.393+0000 7f9a873bd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a801123a0 con 0x7f9a801985a0 2024-04-21T08:42:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.393+0000 7f9a84958640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a80107260 0x7f9a8019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.393+0000 7f9a85159640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a80102190 0x7f9a8019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.393+0000 7f9a84958640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a80107260 0x7f9a8019e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34412/0 (socket says 172.21.15.43:34412) 2024-04-21T08:42:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.393+0000 7f9a84958640 1 -- 172.21.15.43:0/2460938385 learned_addr learned my addr 172.21.15.43:0/2460938385 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.393+0000 7f9a8595a640 1 --2- 172.21.15.43:0/2460938385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a801985a0 0x7f9a80198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.393+0000 7f9a84958640 1 -- 172.21.15.43:0/2460938385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a801985a0 msgr2=0x7f9a80198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.393+0000 7f9a84958640 1 --2- 172.21.15.43:0/2460938385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a801985a0 0x7f9a80198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.393+0000 7f9a84958640 1 -- 172.21.15.43:0/2460938385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a80102190 msgr2=0x7f9a8019e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.393+0000 7f9a84958640 1 --2- 172.21.15.43:0/2460938385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a80102190 0x7f9a8019e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.393+0000 7f9a84958640 1 -- 172.21.15.43:0/2460938385 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a801990c0 con 0x7f9a80107260 2024-04-21T08:42:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.393+0000 7f9a85159640 1 --2- 172.21.15.43:0/2460938385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a80102190 0x7f9a8019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:42:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.393+0000 7f9a84958640 1 --2- 172.21.15.43:0/2460938385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a80107260 0x7f9a8019e930 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000b450 tx=0x7f9a7000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.394+0000 7f9a767fc640 1 -- 172.21.15.43:0/2460938385 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a70015020 con 0x7f9a80107260 2024-04-21T08:42:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.394+0000 7f9a767fc640 1 -- 172.21.15.43:0/2460938385 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a70011d40 con 0x7f9a80107260 2024-04-21T08:42:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.394+0000 7f9a873bd640 1 -- 172.21.15.43:0/2460938385 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a801993b0 con 0x7f9a80107260 2024-04-21T08:42:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.394+0000 7f9a767fc640 1 -- 172.21.15.43:0/2460938385 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7000f340 con 0x7f9a80107260 2024-04-21T08:42:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.394+0000 7f9a873bd640 1 -- 172.21.15.43:0/2460938385 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a801a55b0 con 0x7f9a80107260 2024-04-21T08:42:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.396+0000 7f9a767fc640 1 -- 172.21.15.43:0/2460938385 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9a70018020 con 0x7f9a80107260 2024-04-21T08:42:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.396+0000 7f9a89fb5740 1 Processor -- start 2024-04-21T08:42:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.396+0000 7f9a89fb5740 1 -- start start 2024-04-21T08:42:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.397+0000 7f9a89fb5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad7204d0 0x5642ad7208b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.397+0000 7f9a89fb5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642ad720e80 0x5642ad729430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.397+0000 7f9a89fb5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642ad729a00 0x5642ad72be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.397+0000 7f9a89fb5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5642ad6851f0 con 0x5642ad7204d0 2024-04-21T08:42:47.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.397+0000 7f9a89fb5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5642ad6224b0 con 0x5642ad729a00 2024-04-21T08:42:47.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.397+0000 7f9a89fb5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5642ad6811b0 con 0x5642ad720e80 2024-04-21T08:42:47.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.397+0000 7f9a767fc640 1 --2- 172.21.15.43:0/2460938385 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9a6c0a3d20 0x7f9a6c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.397+0000 7f9a8595a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad7204d0 0x5642ad7208b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.397+0000 7f9a8595a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad7204d0 0x5642ad7208b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34414/0 (socket says 172.21.15.43:34414) 2024-04-21T08:42:47.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.397+0000 7f9a8595a640 1 -- 172.21.15.43:0/3035290064 learned_addr learned my addr 172.21.15.43:0/3035290064 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.397+0000 7f9a767fc640 1 -- 172.21.15.43:0/2460938385 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f9a700d0630 con 0x7f9a80107260 2024-04-21T08:42:47.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.397+0000 7f9a8595a640 1 --2- 172.21.15.43:0/2460938385 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9a6c0a3d20 0x7f9a6c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.397+0000 7f9a85159640 1 --2- 172.21.15.43:0/3035290064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642ad720e80 0x5642ad729430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.397+0000 7f9a84958640 1 --2- 172.21.15.43:0/3035290064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642ad729a00 0x5642ad72be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.397+0000 7f9a8595a640 1 -- 172.21.15.43:0/3035290064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642ad720e80 msgr2=0x5642ad729430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.397+0000 7f9a8595a640 1 --2- 172.21.15.43:0/3035290064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642ad720e80 0x5642ad729430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.397+0000 7f9a8595a640 1 -- 172.21.15.43:0/3035290064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642ad729a00 msgr2=0x5642ad72be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.397+0000 7f9a8595a640 1 --2- 172.21.15.43:0/3035290064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642ad729a00 0x5642ad72be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.397+0000 7f9a8595a640 1 -- 172.21.15.43:0/3035290064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642ad6141b0 con 0x5642ad7204d0 2024-04-21T08:42:47.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.398+0000 7f9a8595a640 1 --2- 172.21.15.43:0/3035290064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad7204d0 0x5642ad7208b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009ca0 tx=0x7f9a78032650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41b47dd25ac3687 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.398+0000 7f9a8595a640 1 --2- 172.21.15.43:0/2460938385 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9a6c0a3d20 0x7f9a6c0a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9a78036000 tx=0x7f9a78047000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.398+0000 7f9a6b7fe640 1 -- 172.21.15.43:0/3035290064 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a78039040 con 0x5642ad7204d0 2024-04-21T08:42:47.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.398+0000 7f9a6b7fe640 1 -- 172.21.15.43:0/3035290064 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a78051070 con 0x5642ad7204d0 2024-04-21T08:42:47.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.398+0000 7f9a6b7fe640 1 -- 172.21.15.43:0/3035290064 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a780044c0 con 0x5642ad7204d0 2024-04-21T08:42:47.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.398+0000 7f9a89fb5740 1 -- 172.21.15.43:0/3035290064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad7204d0 msgr2=0x5642ad7208b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.398+0000 7f9a89fb5740 1 --2- 172.21.15.43:0/3035290064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad7204d0 0x5642ad7208b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009ca0 tx=0x7f9a78032650 comp rx=0 tx=0).stop 2024-04-21T08:42:47.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.398+0000 7f9a89fb5740 1 -- 172.21.15.43:0/3035290064 shutdown_connections 2024-04-21T08:42:47.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.398+0000 7f9a89fb5740 1 --2- 172.21.15.43:0/3035290064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642ad729a00 0x5642ad72be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.398+0000 7f9a89fb5740 1 --2- 172.21.15.43:0/3035290064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642ad720e80 0x5642ad729430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.398+0000 7f9a89fb5740 1 --2- 172.21.15.43:0/3035290064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad7204d0 0x5642ad7208b0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.398+0000 7f9a89fb5740 1 -- 172.21.15.43:0/3035290064 >> 172.21.15.43:0/3035290064 conn(0x5642ad71aba0 msgr2=0x5642ad71af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.398+0000 7f9a89fb5740 1 -- 172.21.15.43:0/3035290064 shutdown_connections 2024-04-21T08:42:47.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.399+0000 7f9a89fb5740 1 -- 172.21.15.43:0/3035290064 wait complete. 2024-04-21T08:42:47.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.399+0000 7f9a89fb5740 1 -- start start 2024-04-21T08:42:47.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.399+0000 7f9a89fb5740 1 Processor -- start 2024-04-21T08:42:47.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.399+0000 7f9a89fb5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642ad7204d0 0x5642ad7293a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.399+0000 7f9a89fb5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642ad720e80 0x5642ad726730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.399+0000 7f9a89fb5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad729a00 0x5642ad726e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.399+0000 7f9a89fb5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5642ad72c910 con 0x5642ad729a00 2024-04-21T08:42:47.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.399+0000 7f9a89fb5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5642ad63e500 con 0x5642ad720e80 2024-04-21T08:42:47.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.399+0000 7f9a89fb5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5642ad72ca90 con 0x5642ad7204d0 2024-04-21T08:42:47.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.399+0000 7f9a89fb5740 10 client.0 _ll_register_callbacks cb 0x5642ad60cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:47.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.399+0000 7f9a8595a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad729a00 0x5642ad726e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.399+0000 7f9a8595a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad729a00 0x5642ad726e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34426/0 (socket says 172.21.15.43:34426) 2024-04-21T08:42:47.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.399+0000 7f9a8595a640 1 -- 172.21.15.43:0/1448301360 learned_addr learned my addr 172.21.15.43:0/1448301360 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.400+0000 7f9a84958640 1 --2- 172.21.15.43:0/1448301360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642ad720e80 0x5642ad726730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.400+0000 7f9a85159640 1 --2- 172.21.15.43:0/1448301360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642ad7204d0 0x5642ad7293a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.400+0000 7f9a8595a640 1 -- 172.21.15.43:0/1448301360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642ad7204d0 msgr2=0x5642ad7293a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.400+0000 7f9a8595a640 1 --2- 172.21.15.43:0/1448301360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642ad7204d0 0x5642ad7293a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.400+0000 7f9a8595a640 1 -- 172.21.15.43:0/1448301360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642ad720e80 msgr2=0x5642ad726730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.400+0000 7f9a8595a640 1 --2- 172.21.15.43:0/1448301360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642ad720e80 0x5642ad726730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.400+0000 7f9a8595a640 1 -- 172.21.15.43:0/1448301360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642ad3973f0 con 0x5642ad729a00 2024-04-21T08:42:47.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.400+0000 7f9a84958640 1 --2- 172.21.15.43:0/1448301360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642ad720e80 0x5642ad726730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:47.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.400+0000 7f9a8595a640 1 --2- 172.21.15.43:0/1448301360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad729a00 0x5642ad726e70 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009940 tx=0x7f9a78032ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c314c1a00348eb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.400+0000 7f9a68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:47.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.400+0000 7f9a68ff9640 1 -- 172.21.15.43:0/1448301360 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a78039040 con 0x5642ad729a00 2024-04-21T08:42:47.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.400+0000 7f9a68ff9640 1 -- 172.21.15.43:0/1448301360 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f9a7802c770 con 0x5642ad729a00 2024-04-21T08:42:47.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.400+0000 7f9a89fb5740 1 -- 172.21.15.43:0/1448301360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a78051070 con 0x5642ad729a00 2024-04-21T08:42:47.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.400+0000 7f9a68ff9640 1 -- 172.21.15.43:0/1448301360 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9a78061050 con 0x5642ad729a00 2024-04-21T08:42:47.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.400+0000 7f9a89fb5740 10 client.17349 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:47.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.400+0000 7f9a89fb5740 1 -- 172.21.15.43:0/1448301360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5642ad4617d0 con 0x5642ad729a00 2024-04-21T08:42:47.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.400+0000 7f9a68ff9640 1 -- 172.21.15.43:0/1448301360 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a78051070 con 0x5642ad729a00 2024-04-21T08:42:47.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.401+0000 7f9a68ff9640 1 -- 172.21.15.43:0/1448301360 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9a78005040 con 0x5642ad729a00 2024-04-21T08:42:47.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.401+0000 7f9a89fb5740 10 client.17349 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:47.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.401+0000 7f9a89fb5740 10 client.17349 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:42:47.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.401+0000 7f9a89fb5740 20 client.17349 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:47.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.401+0000 7f9a89fb5740 1 --2- 172.21.15.43:0/1448301360 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5642ad739ab0 0x5642ad73bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.401+0000 7f9a89fb5740 4 client.17349 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:47.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.401+0000 7f9a89fb5740 1 -- 172.21.15.43:0/1448301360 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642ad624930 con 0x5642ad739ab0 2024-04-21T08:42:47.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.401+0000 7f9a85159640 1 --2- 172.21.15.43:0/1448301360 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5642ad739ab0 0x5642ad73bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.401+0000 7f9a68ff9640 10 client.17349 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:42:47.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.401+0000 7f9a85159640 1 --2- 172.21.15.43:0/1448301360 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5642ad739ab0 0x5642ad73bea0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=41fec651732210b4 server_cookie=d80680a045b6bdf9 in_seq=0 out_seq=0 2024-04-21T08:42:47.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.402+0000 7f9a68ff9640 1 -- 172.21.15.43:0/1448301360 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5642ad624930 con 0x5642ad739ab0 2024-04-21T08:42:47.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.402+0000 7f9a68ff9640 10 client.17349 handle_command_reply: tid=0 2024-04-21T08:42:47.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.402+0000 7f9a89fb5740 1 client.17349 shutdown 2024-04-21T08:42:47.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.402+0000 7f9a89fb5740 1 -- 172.21.15.43:0/1448301360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad729a00 msgr2=0x5642ad726e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.402+0000 7f9a89fb5740 1 --2- 172.21.15.43:0/1448301360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad729a00 0x5642ad726e70 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009940 tx=0x7f9a78032ab0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.402+0000 7f9a89fb5740 1 -- 172.21.15.43:0/1448301360 shutdown_connections 2024-04-21T08:42:47.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.402+0000 7f9a89fb5740 1 --2- 172.21.15.43:0/1448301360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad729a00 0x5642ad726e70 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.403+0000 7f9a89fb5740 1 --2- 172.21.15.43:0/1448301360 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5642ad739ab0 0x5642ad73bea0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.403+0000 7f9a89fb5740 1 --2- 172.21.15.43:0/1448301360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642ad720e80 0x5642ad726730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.403+0000 7f9a68ff9640 0 client.17349 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:42:47.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.403+0000 7f9a89fb5740 1 --2- 172.21.15.43:0/1448301360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642ad7204d0 0x5642ad7293a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.403+0000 7f9a89fb5740 1 -- 172.21.15.43:0/1448301360 >> 172.21.15.43:0/1448301360 conn(0x5642ad71aba0 msgr2=0x5642ad71c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.403+0000 7f9a89fb5740 1 -- 172.21.15.43:0/1448301360 shutdown_connections 2024-04-21T08:42:47.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.403+0000 7f9a89fb5740 1 -- 172.21.15.43:0/1448301360 wait complete. 2024-04-21T08:42:47.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.403+0000 7f9a89fb5740 20 client.17349 trim_cache size 0 max 16384 2024-04-21T08:42:47.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.417+0000 7f9a89fb5740 1 Processor -- start 2024-04-21T08:42:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.417+0000 7f9a89fb5740 1 -- start start 2024-04-21T08:42:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.417+0000 7f9a89fb5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642ad7204d0 0x5642ad71ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.417+0000 7f9a89fb5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642ad720e80 0x5642ad7320c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.417+0000 7f9a89fb5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad729a00 0x5642ad741490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.417+0000 7f9a89fb5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9a801af4a0 con 0x5642ad729a00 2024-04-21T08:42:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.417+0000 7f9a89fb5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9a801b0fe0 con 0x5642ad720e80 2024-04-21T08:42:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.417+0000 7f9a89fb5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a801afb70 con 0x5642ad7204d0 2024-04-21T08:42:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.417+0000 7f9a8595a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad729a00 0x5642ad741490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.417+0000 7f9a8595a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad729a00 0x5642ad741490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34436/0 (socket says 172.21.15.43:34436) 2024-04-21T08:42:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.417+0000 7f9a8595a640 1 -- 172.21.15.43:0/386182633 learned_addr learned my addr 172.21.15.43:0/386182633 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.417+0000 7f9a84958640 1 --2- 172.21.15.43:0/386182633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642ad720e80 0x5642ad7320c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.417+0000 7f9a8595a640 1 -- 172.21.15.43:0/386182633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642ad7204d0 msgr2=0x5642ad71ca40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.417+0000 7f9a8595a640 1 --2- 172.21.15.43:0/386182633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642ad7204d0 0x5642ad71ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.417+0000 7f9a8595a640 1 -- 172.21.15.43:0/386182633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642ad720e80 msgr2=0x5642ad7320c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.417+0000 7f9a8595a640 1 --2- 172.21.15.43:0/386182633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642ad720e80 0x5642ad7320c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.417+0000 7f9a8595a640 1 -- 172.21.15.43:0/386182633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642ad71a310 con 0x5642ad729a00 2024-04-21T08:42:47.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.417+0000 7f9a84958640 1 --2- 172.21.15.43:0/386182633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642ad720e80 0x5642ad7320c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:47.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.417+0000 7f9a8595a640 1 --2- 172.21.15.43:0/386182633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad729a00 0x5642ad741490 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f9a78005440 tx=0x7f9a78009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3c3df2871fcee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.417+0000 7f9a6affd640 1 -- 172.21.15.43:0/386182633 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a78039070 con 0x5642ad729a00 2024-04-21T08:42:47.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.417+0000 7f9a6affd640 1 -- 172.21.15.43:0/386182633 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a78051040 con 0x5642ad729a00 2024-04-21T08:42:47.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.417+0000 7f9a6affd640 1 -- 172.21.15.43:0/386182633 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a780059e0 con 0x5642ad729a00 2024-04-21T08:42:47.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.418+0000 7f9a89fb5740 1 -- 172.21.15.43:0/386182633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad729a00 msgr2=0x5642ad741490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.418+0000 7f9a89fb5740 1 --2- 172.21.15.43:0/386182633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad729a00 0x5642ad741490 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f9a78005440 tx=0x7f9a78009c60 comp rx=0 tx=0).stop 2024-04-21T08:42:47.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.418+0000 7f9a89fb5740 1 -- 172.21.15.43:0/386182633 shutdown_connections 2024-04-21T08:42:47.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.418+0000 7f9a89fb5740 1 --2- 172.21.15.43:0/386182633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad729a00 0x5642ad741490 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.418+0000 7f9a89fb5740 1 --2- 172.21.15.43:0/386182633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642ad720e80 0x5642ad7320c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.418+0000 7f9a89fb5740 1 --2- 172.21.15.43:0/386182633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642ad7204d0 0x5642ad71ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.418+0000 7f9a89fb5740 1 -- 172.21.15.43:0/386182633 >> 172.21.15.43:0/386182633 conn(0x5642ad71aba0 msgr2=0x5642ad719c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.418+0000 7f9a89fb5740 1 -- 172.21.15.43:0/386182633 shutdown_connections 2024-04-21T08:42:47.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.418+0000 7f9a89fb5740 1 -- 172.21.15.43:0/386182633 wait complete. 2024-04-21T08:42:47.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.418+0000 7f9a89fb5740 1 -- start start 2024-04-21T08:42:47.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.418+0000 7f9a89fb5740 1 Processor -- start 2024-04-21T08:42:47.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.419+0000 7f9a89fb5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642ad7204d0 0x5642ad797eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.419+0000 7f9a89fb5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad720e80 0x5642ad7983f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.419+0000 7f9a89fb5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642ad729a00 0x5642ad79c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.419+0000 7f9a89fb5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9a801b0910 con 0x5642ad720e80 2024-04-21T08:42:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.419+0000 7f9a89fb5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9a801b0240 con 0x5642ad729a00 2024-04-21T08:42:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.419+0000 7f9a89fb5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a801ad480 con 0x5642ad7204d0 2024-04-21T08:42:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.419+0000 7f9a89fb5740 10 client.0 _ll_register_callbacks cb 0x7f9a7c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.419+0000 7f9a84958640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad720e80 0x5642ad7983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.419+0000 7f9a84958640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad720e80 0x5642ad7983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34452/0 (socket says 172.21.15.43:34452) 2024-04-21T08:42:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.419+0000 7f9a84958640 1 -- 172.21.15.43:0/2744565825 learned_addr learned my addr 172.21.15.43:0/2744565825 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.419+0000 7f9a8595a640 1 --2- 172.21.15.43:0/2744565825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642ad729a00 0x5642ad79c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.419+0000 7f9a84958640 1 -- 172.21.15.43:0/2744565825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642ad7204d0 msgr2=0x5642ad797eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.419+0000 7f9a85159640 1 --2- 172.21.15.43:0/2744565825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642ad7204d0 0x5642ad797eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.419+0000 7f9a84958640 1 --2- 172.21.15.43:0/2744565825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642ad7204d0 0x5642ad797eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.419+0000 7f9a84958640 1 -- 172.21.15.43:0/2744565825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642ad729a00 msgr2=0x5642ad79c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.419+0000 7f9a84958640 1 --2- 172.21.15.43:0/2744565825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642ad729a00 0x5642ad79c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.419+0000 7f9a84958640 1 -- 172.21.15.43:0/2744565825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642ad79c8f0 con 0x5642ad720e80 2024-04-21T08:42:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.419+0000 7f9a8595a640 1 --2- 172.21.15.43:0/2744565825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642ad729a00 0x5642ad79c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.419+0000 7f9a84958640 1 --2- 172.21.15.43:0/2744565825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad720e80 0x5642ad7983f0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f9a700cdac0 tx=0x7f9a7006bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39966d469fa42ef server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.419+0000 7f9a68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.420+0000 7f9a68ff9640 1 -- 172.21.15.43:0/2744565825 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a700a0660 con 0x5642ad720e80 2024-04-21T08:42:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.420+0000 7f9a68ff9640 1 -- 172.21.15.43:0/2744565825 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f9a700cdbd0 con 0x5642ad720e80 2024-04-21T08:42:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.420+0000 7f9a89fb5740 1 -- 172.21.15.43:0/2744565825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a78051040 con 0x5642ad720e80 2024-04-21T08:42:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.420+0000 7f9a68ff9640 1 -- 172.21.15.43:0/2744565825 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9a700cddd0 con 0x5642ad720e80 2024-04-21T08:42:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.420+0000 7f9a89fb5740 10 client.17355 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.420+0000 7f9a89fb5740 1 -- 172.21.15.43:0/2744565825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5642ad7a4e90 con 0x5642ad720e80 2024-04-21T08:42:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.420+0000 7f9a68ff9640 1 -- 172.21.15.43:0/2744565825 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a78051040 con 0x5642ad720e80 2024-04-21T08:42:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.420+0000 7f9a68ff9640 1 -- 172.21.15.43:0/2744565825 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9a700d0190 con 0x5642ad720e80 2024-04-21T08:42:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.420+0000 7f9a89fb5740 10 client.17355 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.420+0000 7f9a89fb5740 10 client.17355 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:42:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.420+0000 7f9a89fb5740 20 client.17355 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:47.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.420+0000 7f9a89fb5740 1 --2- 172.21.15.43:0/2744565825 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5642ad7a5610 0x5642ad7a7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.420+0000 7f9a89fb5740 4 client.17355 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:42:47.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.420+0000 7f9a89fb5740 1 -- 172.21.15.43:0/2744565825 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5642ad736480 con 0x5642ad7a5610 2024-04-21T08:42:47.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.420+0000 7f9a85159640 1 --2- 172.21.15.43:0/2744565825 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5642ad7a5610 0x5642ad7a7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.421+0000 7f9a85159640 1 --2- 172.21.15.43:0/2744565825 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5642ad7a5610 0x5642ad7a7a30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b568a871fe75ce39 server_cookie=657b9bc696734b38 in_seq=0 out_seq=0 2024-04-21T08:42:47.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.421+0000 7f9a68ff9640 10 client.17355 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:42:47.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.421+0000 7f9a68ff9640 1 -- 172.21.15.43:0/2744565825 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5642ad736480 con 0x5642ad7a5610 2024-04-21T08:42:47.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.421+0000 7f9a68ff9640 10 client.17355 handle_command_reply: tid=0 2024-04-21T08:42:47.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.421+0000 7f9a89fb5740 1 client.17355 shutdown 2024-04-21T08:42:47.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.421+0000 7f9a89fb5740 1 -- 172.21.15.43:0/2744565825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad720e80 msgr2=0x5642ad7983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.421+0000 7f9a89fb5740 1 --2- 172.21.15.43:0/2744565825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad720e80 0x5642ad7983f0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f9a700cdac0 tx=0x7f9a7006bb90 comp rx=0 tx=0).stop 2024-04-21T08:42:47.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.421+0000 7f9a89fb5740 1 -- 172.21.15.43:0/2744565825 shutdown_connections 2024-04-21T08:42:47.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.421+0000 7f9a89fb5740 1 --2- 172.21.15.43:0/2744565825 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5642ad7a5610 0x5642ad7a7a30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.421+0000 7f9a89fb5740 1 --2- 172.21.15.43:0/2744565825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642ad729a00 0x5642ad79c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.421+0000 7f9a89fb5740 1 --2- 172.21.15.43:0/2744565825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5642ad720e80 0x5642ad7983f0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.421+0000 7f9a68ff9640 0 client.17355 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:42:47.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.421+0000 7f9a89fb5740 1 --2- 172.21.15.43:0/2744565825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642ad7204d0 0x5642ad797eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.421+0000 7f9a89fb5740 1 -- 172.21.15.43:0/2744565825 >> 172.21.15.43:0/2744565825 conn(0x5642ad71aba0 msgr2=0x5642ad7212b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.421+0000 7f9a89fb5740 1 -- 172.21.15.43:0/2744565825 shutdown_connections 2024-04-21T08:42:47.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.421+0000 7f9a89fb5740 1 -- 172.21.15.43:0/2744565825 wait complete. 2024-04-21T08:42:47.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.421+0000 7f9a89fb5740 20 client.17355 trim_cache size 0 max 16384 2024-04-21T08:42:47.427 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:42:47.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.422+0000 7f9a873bd640 1 -- 172.21.15.43:0/2460938385 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9a6c0a3d20 msgr2=0x7f9a6c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.422+0000 7f9a873bd640 1 --2- 172.21.15.43:0/2460938385 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9a6c0a3d20 0x7f9a6c0a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9a78036000 tx=0x7f9a78047000 comp rx=0 tx=0).stop 2024-04-21T08:42:47.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.422+0000 7f9a873bd640 1 -- 172.21.15.43:0/2460938385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a80107260 msgr2=0x7f9a8019e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.422+0000 7f9a873bd640 1 --2- 172.21.15.43:0/2460938385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a80107260 0x7f9a8019e930 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000b450 tx=0x7f9a7000b910 comp rx=0 tx=0).stop 2024-04-21T08:42:47.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.422+0000 7f9a873bd640 1 -- 172.21.15.43:0/2460938385 shutdown_connections 2024-04-21T08:42:47.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.422+0000 7f9a873bd640 1 --2- 172.21.15.43:0/2460938385 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9a6c0a3d20 0x7f9a6c0a61e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.422+0000 7f9a873bd640 1 --2- 172.21.15.43:0/2460938385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a801985a0 0x7f9a80198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.422+0000 7f9a873bd640 1 --2- 172.21.15.43:0/2460938385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a80107260 0x7f9a8019e930 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.422+0000 7f9a873bd640 1 --2- 172.21.15.43:0/2460938385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a80102190 0x7f9a8019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.422+0000 7f9a873bd640 1 -- 172.21.15.43:0/2460938385 >> 172.21.15.43:0/2460938385 conn(0x7f9a800793a0 msgr2=0x7f9a800fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.422+0000 7f9a873bd640 1 -- 172.21.15.43:0/2460938385 shutdown_connections 2024-04-21T08:42:47.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.422+0000 7f9a873bd640 1 -- 172.21.15.43:0/2460938385 wait complete. 2024-04-21T08:42:47.432 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:42:47.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:47 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2115373615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:42:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.537+0000 7f5381f85640 1 -- 172.21.15.43:0/1723368227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f537c1010f0 msgr2=0x7f537c1014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.537+0000 7f5381f85640 1 --2- 172.21.15.43:0/1723368227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f537c1010f0 0x7f537c1014d0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f536c009900 tx=0x7f536c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:42:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.538+0000 7f5381f85640 1 -- 172.21.15.43:0/1723368227 shutdown_connections 2024-04-21T08:42:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.538+0000 7f5381f85640 1 --2- 172.21.15.43:0/1723368227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f537c106170 0x7f537c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.538+0000 7f5381f85640 1 --2- 172.21.15.43:0/1723368227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f537c101aa0 0x7f537c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.538+0000 7f5381f85640 1 --2- 172.21.15.43:0/1723368227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f537c1010f0 0x7f537c1014d0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.538+0000 7f5381f85640 1 -- 172.21.15.43:0/1723368227 >> 172.21.15.43:0/1723368227 conn(0x7f537c077fb0 msgr2=0x7f537c0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.538+0000 7f5381f85640 1 -- 172.21.15.43:0/1723368227 shutdown_connections 2024-04-21T08:42:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.538+0000 7f5381f85640 1 -- 172.21.15.43:0/1723368227 wait complete. 2024-04-21T08:42:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.538+0000 7f5381f85640 1 Processor -- start 2024-04-21T08:42:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.538+0000 7f5381f85640 1 -- start start 2024-04-21T08:42:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.539+0000 7f5381f85640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f537c1010f0 0x7f537c19e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.539+0000 7f5381f85640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f537c106170 0x7f537c19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.539+0000 7f5381f85640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f537c1985e0 0x7f537c198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.539+0000 7f5381f85640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f537c112280 con 0x7f537c1010f0 2024-04-21T08:42:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.539+0000 7f5381f85640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f537c112100 con 0x7f537c1985e0 2024-04-21T08:42:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.539+0000 7f5381f85640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f537c112400 con 0x7f537c106170 2024-04-21T08:42:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.539+0000 7f537b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f537c1010f0 0x7f537c19e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.539+0000 7f537b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f537c1010f0 0x7f537c19e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34472/0 (socket says 172.21.15.43:34472) 2024-04-21T08:42:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.539+0000 7f537b7fe640 1 -- 172.21.15.43:0/1861793031 learned_addr learned my addr 172.21.15.43:0/1861793031 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.539+0000 7f537bfff640 1 --2- 172.21.15.43:0/1861793031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f537c1985e0 0x7f537c198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.539+0000 7f537affd640 1 --2- 172.21.15.43:0/1861793031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f537c106170 0x7f537c19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.539+0000 7f537b7fe640 1 -- 172.21.15.43:0/1861793031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f537c106170 msgr2=0x7f537c19e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.539+0000 7f537b7fe640 1 --2- 172.21.15.43:0/1861793031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f537c106170 0x7f537c19e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.539+0000 7f537b7fe640 1 -- 172.21.15.43:0/1861793031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f537c1985e0 msgr2=0x7f537c198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.539+0000 7f537b7fe640 1 --2- 172.21.15.43:0/1861793031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f537c1985e0 0x7f537c198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.539+0000 7f537b7fe640 1 -- 172.21.15.43:0/1861793031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537c199100 con 0x7f537c1010f0 2024-04-21T08:42:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.539+0000 7f537b7fe640 1 --2- 172.21.15.43:0/1861793031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f537c1010f0 0x7f537c19e430 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f536c000c00 tx=0x7f536c02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.539+0000 7f5378ff9640 1 -- 172.21.15.43:0/1861793031 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c043020 con 0x7f537c1010f0 2024-04-21T08:42:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.539+0000 7f5378ff9640 1 -- 172.21.15.43:0/1861793031 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536c02cbe0 con 0x7f537c1010f0 2024-04-21T08:42:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.539+0000 7f5381f85640 1 -- 172.21.15.43:0/1861793031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537c199390 con 0x7f537c1010f0 2024-04-21T08:42:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.539+0000 7f5378ff9640 1 -- 172.21.15.43:0/1861793031 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c003b80 con 0x7f537c1010f0 2024-04-21T08:42:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.540+0000 7f5381f85640 1 -- 172.21.15.43:0/1861793031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f537c1a55e0 con 0x7f537c1010f0 2024-04-21T08:42:47.548 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:47 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2115373615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:42:47.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.542+0000 7f5378ff9640 1 -- 172.21.15.43:0/1861793031 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f536c046020 con 0x7f537c1010f0 2024-04-21T08:42:47.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.542+0000 7f5384b71740 1 Processor -- start 2024-04-21T08:42:47.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.542+0000 7f5384b71740 1 -- start start 2024-04-21T08:42:47.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.542+0000 7f5378ff9640 1 --2- 172.21.15.43:0/1861793031 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f53640a3d20 0x7f53640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.542+0000 7f5384b71740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc40250 0x56512fc40630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.542+0000 7f5384b71740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56512fc40c00 0x56512fc4d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.542+0000 7f5384b71740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56512fc4de10 0x56512fc50220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.542+0000 7f5384b71740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56512fba51f0 con 0x56512fc40250 2024-04-21T08:42:47.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.542+0000 7f5384b71740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56512fb424b0 con 0x56512fc4de10 2024-04-21T08:42:47.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.542+0000 7f5384b71740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56512fba11b0 con 0x56512fc40c00 2024-04-21T08:42:47.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.542+0000 7f537b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc40250 0x56512fc40630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.542+0000 7f537bfff640 1 --2- 172.21.15.43:0/1861793031 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f53640a3d20 0x7f53640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.542+0000 7f537b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc40250 0x56512fc40630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34476/0 (socket says 172.21.15.43:34476) 2024-04-21T08:42:47.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.543+0000 7f537b7fe640 1 -- 172.21.15.43:0/1659176175 learned_addr learned my addr 172.21.15.43:0/1659176175 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.543+0000 7f5378ff9640 1 -- 172.21.15.43:0/1861793031 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f536c0f2bf0 con 0x7f537c1010f0 2024-04-21T08:42:47.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.543+0000 7f537bfff640 1 --2- 172.21.15.43:0/1659176175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56512fc4de10 0x56512fc50220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.543+0000 7f537affd640 1 --2- 172.21.15.43:0/1659176175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56512fc40c00 0x56512fc4d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.543+0000 7f537b7fe640 1 -- 172.21.15.43:0/1659176175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56512fc40c00 msgr2=0x56512fc4d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.543+0000 7f537b7fe640 1 --2- 172.21.15.43:0/1659176175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56512fc40c00 0x56512fc4d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.543+0000 7f537b7fe640 1 -- 172.21.15.43:0/1659176175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56512fc4de10 msgr2=0x56512fc50220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.543+0000 7f537b7fe640 1 --2- 172.21.15.43:0/1659176175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56512fc4de10 0x56512fc50220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.543+0000 7f537b7fe640 1 -- 172.21.15.43:0/1659176175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56512fb341b0 con 0x56512fc40250 2024-04-21T08:42:47.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.543+0000 7f537affd640 1 --2- 172.21.15.43:0/1659176175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56512fc40c00 0x56512fc4d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:47.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.543+0000 7f537bfff640 1 --2- 172.21.15.43:0/1659176175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56512fc4de10 0x56512fc50220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:42:47.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.543+0000 7f537b7fe640 1 --2- 172.21.15.43:0/1659176175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc40250 0x56512fc40630 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f536c005ec0 tx=0x7f536c005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0c468dd7a96c4c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.543+0000 7f537bfff640 1 --2- 172.21.15.43:0/1861793031 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f53640a3d20 0x7f53640a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5374009fd0 tx=0x7f5374004ab0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.543+0000 7f5361ffb640 1 -- 172.21.15.43:0/1659176175 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c0c3e10 con 0x56512fc40250 2024-04-21T08:42:47.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.543+0000 7f5361ffb640 1 -- 172.21.15.43:0/1659176175 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536c0befd0 con 0x56512fc40250 2024-04-21T08:42:47.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.543+0000 7f5361ffb640 1 -- 172.21.15.43:0/1659176175 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c0bfa00 con 0x56512fc40250 2024-04-21T08:42:47.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.544+0000 7f5384b71740 1 -- 172.21.15.43:0/1659176175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc40250 msgr2=0x56512fc40630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.544+0000 7f5384b71740 1 --2- 172.21.15.43:0/1659176175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc40250 0x56512fc40630 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f536c005ec0 tx=0x7f536c005e20 comp rx=0 tx=0).stop 2024-04-21T08:42:47.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.544+0000 7f5384b71740 1 -- 172.21.15.43:0/1659176175 shutdown_connections 2024-04-21T08:42:47.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.544+0000 7f5384b71740 1 --2- 172.21.15.43:0/1659176175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56512fc4de10 0x56512fc50220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.544+0000 7f5384b71740 1 --2- 172.21.15.43:0/1659176175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56512fc40c00 0x56512fc4d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.544+0000 7f5384b71740 1 --2- 172.21.15.43:0/1659176175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc40250 0x56512fc40630 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.544+0000 7f5384b71740 1 -- 172.21.15.43:0/1659176175 >> 172.21.15.43:0/1659176175 conn(0x56512fc3aba0 msgr2=0x56512fc3af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.544+0000 7f5384b71740 1 -- 172.21.15.43:0/1659176175 shutdown_connections 2024-04-21T08:42:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.544+0000 7f5384b71740 1 -- 172.21.15.43:0/1659176175 wait complete. 2024-04-21T08:42:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.544+0000 7f5384b71740 1 -- start start 2024-04-21T08:42:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.544+0000 7f5384b71740 1 Processor -- start 2024-04-21T08:42:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.545+0000 7f5384b71740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56512fc40250 0x56512fc4d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.545+0000 7f5384b71740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc40c00 0x56512fc4ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.545+0000 7f5384b71740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56512fc4de10 0x56512fc4b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.545+0000 7f5384b71740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56512fc50b00 con 0x56512fc40c00 2024-04-21T08:42:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.545+0000 7f5384b71740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56512fb5e500 con 0x56512fc40250 2024-04-21T08:42:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.545+0000 7f5384b71740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56512fc50c80 con 0x56512fc4de10 2024-04-21T08:42:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.545+0000 7f5384b71740 10 client.0 _ll_register_callbacks cb 0x56512fb2cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.545+0000 7f537affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc40c00 0x56512fc4ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.545+0000 7f537affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc40c00 0x56512fc4ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34484/0 (socket says 172.21.15.43:34484) 2024-04-21T08:42:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.545+0000 7f537affd640 1 -- 172.21.15.43:0/220817960 learned_addr learned my addr 172.21.15.43:0/220817960 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.545+0000 7f537b7fe640 1 --2- 172.21.15.43:0/220817960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56512fc40250 0x56512fc4d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.545+0000 7f537bfff640 1 --2- 172.21.15.43:0/220817960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56512fc4de10 0x56512fc4b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.545+0000 7f537affd640 1 -- 172.21.15.43:0/220817960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56512fc4de10 msgr2=0x56512fc4b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.545+0000 7f537affd640 1 --2- 172.21.15.43:0/220817960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56512fc4de10 0x56512fc4b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.545+0000 7f537affd640 1 -- 172.21.15.43:0/220817960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56512fc40250 msgr2=0x56512fc4d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.545+0000 7f537affd640 1 --2- 172.21.15.43:0/220817960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56512fc40250 0x56512fc4d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.545+0000 7f537affd640 1 -- 172.21.15.43:0/220817960 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56512f8b73f0 con 0x56512fc40c00 2024-04-21T08:42:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.545+0000 7f537bfff640 1 --2- 172.21.15.43:0/220817960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56512fc4de10 0x56512fc4b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.545+0000 7f537b7fe640 1 --2- 172.21.15.43:0/220817960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56512fc40250 0x56512fc4d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:42:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.545+0000 7f537affd640 1 --2- 172.21.15.43:0/220817960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc40c00 0x56512fc4ab10 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f5370009ed0 tx=0x7f5370011630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b161fb7c6a0c109 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.545+0000 7f535affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.546+0000 7f535affd640 1 -- 172.21.15.43:0/220817960 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5370019070 con 0x56512fc40c00 2024-04-21T08:42:47.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.546+0000 7f535affd640 1 -- 172.21.15.43:0/220817960 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f5370015870 con 0x56512fc40c00 2024-04-21T08:42:47.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.546+0000 7f5384b71740 1 -- 172.21.15.43:0/220817960 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f536c0befd0 con 0x56512fc40c00 2024-04-21T08:42:47.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.546+0000 7f535affd640 1 -- 172.21.15.43:0/220817960 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5370025050 con 0x56512fc40c00 2024-04-21T08:42:47.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.546+0000 7f5384b71740 10 client.17367 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:47.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.546+0000 7f5384b71740 1 -- 172.21.15.43:0/220817960 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56512f9817d0 con 0x56512fc40c00 2024-04-21T08:42:47.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.546+0000 7f535affd640 1 -- 172.21.15.43:0/220817960 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536c0befd0 con 0x56512fc40c00 2024-04-21T08:42:47.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.546+0000 7f535affd640 1 -- 172.21.15.43:0/220817960 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5370014a70 con 0x56512fc40c00 2024-04-21T08:42:47.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.546+0000 7f5384b71740 10 client.17367 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:47.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.546+0000 7f5384b71740 10 client.17367 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:42:47.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.546+0000 7f5384b71740 20 client.17367 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:47.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.546+0000 7f5384b71740 1 --2- 172.21.15.43:0/220817960 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x56512fc5dcc0 0x56512fc600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.546+0000 7f5384b71740 4 client.17367 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:47.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.546+0000 7f5384b71740 1 -- 172.21.15.43:0/220817960 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56512fb44930 con 0x56512fc5dcc0 2024-04-21T08:42:47.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.546+0000 7f537b7fe640 1 --2- 172.21.15.43:0/220817960 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x56512fc5dcc0 0x56512fc600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.547+0000 7f535affd640 10 client.17367 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:42:47.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.547+0000 7f537b7fe640 1 --2- 172.21.15.43:0/220817960 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x56512fc5dcc0 0x56512fc600d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9e3253757b43ae47 server_cookie=484f638e86e60376 in_seq=0 out_seq=0 2024-04-21T08:42:47.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.547+0000 7f535affd640 1 -- 172.21.15.43:0/220817960 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56512fb44930 con 0x56512fc5dcc0 2024-04-21T08:42:47.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.548+0000 7f535affd640 10 client.17367 handle_command_reply: tid=0 2024-04-21T08:42:47.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.548+0000 7f5384b71740 1 client.17367 shutdown 2024-04-21T08:42:47.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.548+0000 7f5384b71740 1 -- 172.21.15.43:0/220817960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc40c00 msgr2=0x56512fc4ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.548+0000 7f5384b71740 1 --2- 172.21.15.43:0/220817960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc40c00 0x56512fc4ab10 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f5370009ed0 tx=0x7f5370011630 comp rx=0 tx=0).stop 2024-04-21T08:42:47.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.548+0000 7f5384b71740 1 -- 172.21.15.43:0/220817960 shutdown_connections 2024-04-21T08:42:47.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.548+0000 7f5384b71740 1 --2- 172.21.15.43:0/220817960 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x56512fc5dcc0 0x56512fc600d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.548+0000 7f5384b71740 1 --2- 172.21.15.43:0/220817960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56512fc4de10 0x56512fc4b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.548+0000 7f535affd640 0 client.17367 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:42:47.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.548+0000 7f5384b71740 1 --2- 172.21.15.43:0/220817960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc40c00 0x56512fc4ab10 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.548+0000 7f5384b71740 1 --2- 172.21.15.43:0/220817960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56512fc40250 0x56512fc4d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.548+0000 7f5384b71740 1 -- 172.21.15.43:0/220817960 >> 172.21.15.43:0/220817960 conn(0x56512fc3aba0 msgr2=0x56512fc3c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.548+0000 7f5384b71740 1 -- 172.21.15.43:0/220817960 shutdown_connections 2024-04-21T08:42:47.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.548+0000 7f5384b71740 1 -- 172.21.15.43:0/220817960 wait complete. 2024-04-21T08:42:47.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.548+0000 7f5384b71740 20 client.17367 trim_cache size 0 max 16384 2024-04-21T08:42:47.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.561+0000 7f5384b71740 1 Processor -- start 2024-04-21T08:42:47.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.561+0000 7f5384b71740 1 -- start start 2024-04-21T08:42:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.562+0000 7f5384b71740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56512fc40250 0x56512fc59910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.562+0000 7f5384b71740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56512fc40c00 0x56512fc3c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.562+0000 7f5384b71740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc4de10 0x56512fc4d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.562+0000 7f5384b71740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f537c1ad320 con 0x56512fc4de10 2024-04-21T08:42:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.562+0000 7f5384b71740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f537c1aee60 con 0x56512fc40c00 2024-04-21T08:42:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.562+0000 7f5384b71740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f537c1ad9f0 con 0x56512fc40250 2024-04-21T08:42:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.562+0000 7f537bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc4de10 0x56512fc4d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.562+0000 7f537bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc4de10 0x56512fc4d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34490/0 (socket says 172.21.15.43:34490) 2024-04-21T08:42:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.562+0000 7f537bfff640 1 -- 172.21.15.43:0/598327662 learned_addr learned my addr 172.21.15.43:0/598327662 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.562+0000 7f537affd640 1 --2- 172.21.15.43:0/598327662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56512fc40c00 0x56512fc3c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.562+0000 7f537b7fe640 1 --2- 172.21.15.43:0/598327662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56512fc40250 0x56512fc59910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.562+0000 7f537bfff640 1 -- 172.21.15.43:0/598327662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56512fc40250 msgr2=0x56512fc59910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.562+0000 7f537bfff640 1 --2- 172.21.15.43:0/598327662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56512fc40250 0x56512fc59910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.562+0000 7f537bfff640 1 -- 172.21.15.43:0/598327662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56512fc40c00 msgr2=0x56512fc3c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.562+0000 7f537bfff640 1 --2- 172.21.15.43:0/598327662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56512fc40c00 0x56512fc3c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.562+0000 7f537bfff640 1 -- 172.21.15.43:0/598327662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56512fc4b490 con 0x56512fc4de10 2024-04-21T08:42:47.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.562+0000 7f537affd640 1 --2- 172.21.15.43:0/598327662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56512fc40c00 0x56512fc3c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:42:47.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.562+0000 7f537bfff640 1 --2- 172.21.15.43:0/598327662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc4de10 0x56512fc4d240 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f5374012d20 tx=0x7f537400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd85527281d9f4d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.562+0000 7f53617fa640 1 -- 172.21.15.43:0/598327662 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5374012d50 con 0x56512fc4de10 2024-04-21T08:42:47.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.563+0000 7f53617fa640 1 -- 172.21.15.43:0/598327662 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f537400fc60 con 0x56512fc4de10 2024-04-21T08:42:47.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.563+0000 7f53617fa640 1 -- 172.21.15.43:0/598327662 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53740293d0 con 0x56512fc4de10 2024-04-21T08:42:47.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.563+0000 7f5384b71740 1 -- 172.21.15.43:0/598327662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc4de10 msgr2=0x56512fc4d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.563+0000 7f5384b71740 1 --2- 172.21.15.43:0/598327662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc4de10 0x56512fc4d240 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f5374012d20 tx=0x7f537400c000 comp rx=0 tx=0).stop 2024-04-21T08:42:47.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.563+0000 7f5384b71740 1 -- 172.21.15.43:0/598327662 shutdown_connections 2024-04-21T08:42:47.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.563+0000 7f5384b71740 1 --2- 172.21.15.43:0/598327662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc4de10 0x56512fc4d240 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.563+0000 7f5384b71740 1 --2- 172.21.15.43:0/598327662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56512fc40c00 0x56512fc3c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.563+0000 7f5384b71740 1 --2- 172.21.15.43:0/598327662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56512fc40250 0x56512fc59910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.563+0000 7f5384b71740 1 -- 172.21.15.43:0/598327662 >> 172.21.15.43:0/598327662 conn(0x56512fc3aba0 msgr2=0x56512fc3af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.563+0000 7f5384b71740 1 -- 172.21.15.43:0/598327662 shutdown_connections 2024-04-21T08:42:47.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.563+0000 7f5384b71740 1 -- 172.21.15.43:0/598327662 wait complete. 2024-04-21T08:42:47.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.563+0000 7f5384b71740 1 -- start start 2024-04-21T08:42:47.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.564+0000 7f5384b71740 1 Processor -- start 2024-04-21T08:42:47.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.564+0000 7f5384b71740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56512fc40250 0x56512fc47840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.564+0000 7f5384b71740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56512fc40c00 0x56512fc47d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.564+0000 7f5384b71740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc4de10 0x56512fc482c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.564+0000 7f5384b71740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f537c1ae790 con 0x56512fc4de10 2024-04-21T08:42:47.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.564+0000 7f5384b71740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f537c1ae0c0 con 0x56512fc40c00 2024-04-21T08:42:47.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.564+0000 7f5384b71740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f537c1ab300 con 0x56512fc40250 2024-04-21T08:42:47.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.564+0000 7f5384b71740 10 client.0 _ll_register_callbacks cb 0x7f5370009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:47.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.564+0000 7f537bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc4de10 0x56512fc482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.564+0000 7f537bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc4de10 0x56512fc482c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34502/0 (socket says 172.21.15.43:34502) 2024-04-21T08:42:47.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.564+0000 7f537bfff640 1 -- 172.21.15.43:0/3295257145 learned_addr learned my addr 172.21.15.43:0/3295257145 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.564+0000 7f537b7fe640 1 --2- 172.21.15.43:0/3295257145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56512fc40250 0x56512fc47840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.564+0000 7f537affd640 1 --2- 172.21.15.43:0/3295257145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56512fc40c00 0x56512fc47d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.564+0000 7f537bfff640 1 -- 172.21.15.43:0/3295257145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56512fc40250 msgr2=0x56512fc47840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.564+0000 7f537bfff640 1 --2- 172.21.15.43:0/3295257145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56512fc40250 0x56512fc47840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.564+0000 7f537bfff640 1 -- 172.21.15.43:0/3295257145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56512fc40c00 msgr2=0x56512fc47d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.564+0000 7f537bfff640 1 --2- 172.21.15.43:0/3295257145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56512fc40c00 0x56512fc47d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.564+0000 7f537bfff640 1 -- 172.21.15.43:0/3295257145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56512fcc3240 con 0x56512fc4de10 2024-04-21T08:42:47.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.564+0000 7f537b7fe640 1 --2- 172.21.15.43:0/3295257145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56512fc40250 0x56512fc47840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:47.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.565+0000 7f537bfff640 1 --2- 172.21.15.43:0/3295257145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc4de10 0x56512fc482c0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f5374009590 tx=0x7f5374010f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ff15e799f2229b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.565+0000 7f535affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:47.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.565+0000 7f535affd640 1 -- 172.21.15.43:0/3295257145 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5374021c40 con 0x56512fc4de10 2024-04-21T08:42:47.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.565+0000 7f535affd640 1 -- 172.21.15.43:0/3295257145 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f537401b070 con 0x56512fc4de10 2024-04-21T08:42:47.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.565+0000 7f5384b71740 1 -- 172.21.15.43:0/3295257145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f537400fc60 con 0x56512fc4de10 2024-04-21T08:42:47.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.565+0000 7f535affd640 1 -- 172.21.15.43:0/3295257145 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f537403f050 con 0x56512fc4de10 2024-04-21T08:42:47.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.565+0000 7f5384b71740 10 client.17373 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:47.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.565+0000 7f5384b71740 1 -- 172.21.15.43:0/3295257145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56512fcc4520 con 0x56512fc4de10 2024-04-21T08:42:47.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.565+0000 7f535affd640 1 -- 172.21.15.43:0/3295257145 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f537400fc60 con 0x56512fc4de10 2024-04-21T08:42:47.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.565+0000 7f535affd640 1 -- 172.21.15.43:0/3295257145 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f537401e070 con 0x56512fc4de10 2024-04-21T08:42:47.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.565+0000 7f5384b71740 10 client.17373 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:47.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.565+0000 7f5384b71740 10 client.17373 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:42:47.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.565+0000 7f5384b71740 20 client.17373 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:47.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.565+0000 7f5384b71740 1 --2- 172.21.15.43:0/3295257145 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x56512fc5dcc0 0x56512fcc6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.565+0000 7f5384b71740 4 client.17373 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:42:47.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.565+0000 7f5384b71740 1 -- 172.21.15.43:0/3295257145 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56512fbb2c10 con 0x56512fc5dcc0 2024-04-21T08:42:47.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.566+0000 7f537b7fe640 1 --2- 172.21.15.43:0/3295257145 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x56512fc5dcc0 0x56512fcc6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.566+0000 7f535affd640 10 client.17373 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:42:47.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.566+0000 7f537b7fe640 1 --2- 172.21.15.43:0/3295257145 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x56512fc5dcc0 0x56512fcc6c70 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5dfbb8b2fa49edb server_cookie=b51f49495d8e92b9 in_seq=0 out_seq=0 2024-04-21T08:42:47.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.566+0000 7f535affd640 1 -- 172.21.15.43:0/3295257145 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56512fbb2c10 con 0x56512fc5dcc0 2024-04-21T08:42:47.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.566+0000 7f535affd640 10 client.17373 handle_command_reply: tid=0 2024-04-21T08:42:47.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.566+0000 7f5384b71740 1 client.17373 shutdown 2024-04-21T08:42:47.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.566+0000 7f5384b71740 1 -- 172.21.15.43:0/3295257145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc4de10 msgr2=0x56512fc482c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.566+0000 7f5384b71740 1 --2- 172.21.15.43:0/3295257145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc4de10 0x56512fc482c0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f5374009590 tx=0x7f5374010f60 comp rx=0 tx=0).stop 2024-04-21T08:42:47.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.567+0000 7f5384b71740 1 -- 172.21.15.43:0/3295257145 shutdown_connections 2024-04-21T08:42:47.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.567+0000 7f5384b71740 1 --2- 172.21.15.43:0/3295257145 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x56512fc5dcc0 0x56512fcc6c70 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.567+0000 7f5384b71740 1 --2- 172.21.15.43:0/3295257145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56512fc4de10 0x56512fc482c0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.567+0000 7f535affd640 0 client.17373 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:42:47.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.567+0000 7f5384b71740 1 --2- 172.21.15.43:0/3295257145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56512fc40c00 0x56512fc47d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.567+0000 7f5384b71740 1 --2- 172.21.15.43:0/3295257145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56512fc40250 0x56512fc47840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.567+0000 7f5384b71740 1 -- 172.21.15.43:0/3295257145 >> 172.21.15.43:0/3295257145 conn(0x56512fc3aba0 msgr2=0x56512fc39c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.567+0000 7f5384b71740 1 -- 172.21.15.43:0/3295257145 shutdown_connections 2024-04-21T08:42:47.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.567+0000 7f5384b71740 1 -- 172.21.15.43:0/3295257145 wait complete. 2024-04-21T08:42:47.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.567+0000 7f5384b71740 20 client.17373 trim_cache size 0 max 16384 2024-04-21T08:42:47.573 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:42:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.567+0000 7f5381f85640 1 -- 172.21.15.43:0/1861793031 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f53640a3d20 msgr2=0x7f53640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.567+0000 7f5381f85640 1 --2- 172.21.15.43:0/1861793031 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f53640a3d20 0x7f53640a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5374009fd0 tx=0x7f5374004ab0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.567+0000 7f5381f85640 1 -- 172.21.15.43:0/1861793031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f537c1010f0 msgr2=0x7f537c19e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.567+0000 7f5381f85640 1 --2- 172.21.15.43:0/1861793031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f537c1010f0 0x7f537c19e430 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f536c000c00 tx=0x7f536c02c7e0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.567+0000 7f5381f85640 1 -- 172.21.15.43:0/1861793031 shutdown_connections 2024-04-21T08:42:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.567+0000 7f5381f85640 1 --2- 172.21.15.43:0/1861793031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f537c1985e0 0x7f537c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.567+0000 7f5381f85640 1 --2- 172.21.15.43:0/1861793031 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f53640a3d20 0x7f53640a61e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.567+0000 7f5381f85640 1 --2- 172.21.15.43:0/1861793031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f537c106170 0x7f537c19e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.567+0000 7f5381f85640 1 --2- 172.21.15.43:0/1861793031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f537c1010f0 0x7f537c19e430 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.567+0000 7f5381f85640 1 -- 172.21.15.43:0/1861793031 >> 172.21.15.43:0/1861793031 conn(0x7f537c077fb0 msgr2=0x7f537c0ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.567+0000 7f5381f85640 1 -- 172.21.15.43:0/1861793031 shutdown_connections 2024-04-21T08:42:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.567+0000 7f5381f85640 1 -- 172.21.15.43:0/1861793031 wait complete. 2024-04-21T08:42:47.577 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:42:47.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.682+0000 7f3e1aea5640 1 -- 172.21.15.43:0/2696882539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e14102b40 msgr2=0x7f3e14106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.682+0000 7f3e1aea5640 1 --2- 172.21.15.43:0/2696882539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e14102b40 0x7f3e14106b80 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3e00009900 tx=0x7f3e0002bc30 comp rx=0 tx=0).stop 2024-04-21T08:42:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.682+0000 7f3e1aea5640 1 -- 172.21.15.43:0/2696882539 shutdown_connections 2024-04-21T08:42:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.682+0000 7f3e1aea5640 1 --2- 172.21.15.43:0/2696882539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e14107260 0x7f3e1410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.682+0000 7f3e1aea5640 1 --2- 172.21.15.43:0/2696882539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e14102b40 0x7f3e14106b80 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.682+0000 7f3e1aea5640 1 --2- 172.21.15.43:0/2696882539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e14102190 0x7f3e14102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.682+0000 7f3e1aea5640 1 -- 172.21.15.43:0/2696882539 >> 172.21.15.43:0/2696882539 conn(0x7f3e140fd990 msgr2=0x7f3e140ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.682+0000 7f3e1aea5640 1 -- 172.21.15.43:0/2696882539 shutdown_connections 2024-04-21T08:42:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.682+0000 7f3e1aea5640 1 -- 172.21.15.43:0/2696882539 wait complete. 2024-04-21T08:42:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.683+0000 7f3e1aea5640 1 Processor -- start 2024-04-21T08:42:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.683+0000 7f3e1aea5640 1 -- start start 2024-04-21T08:42:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.683+0000 7f3e1aea5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e14102190 0x7f3e14198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.683+0000 7f3e1aea5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e14107260 0x7f3e14198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.683+0000 7f3e1aea5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e1419d020 0x7f3e1419d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.683+0000 7f3e1aea5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3e14112180 con 0x7f3e14107260 2024-04-21T08:42:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.683+0000 7f3e1aea5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e14112000 con 0x7f3e1419d020 2024-04-21T08:42:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.683+0000 7f3e1aea5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3e14112300 con 0x7f3e14102190 2024-04-21T08:42:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.683+0000 7f3e13fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e14107260 0x7f3e14198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.683+0000 7f3e13fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e14107260 0x7f3e14198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34526/0 (socket says 172.21.15.43:34526) 2024-04-21T08:42:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.683+0000 7f3e13fff640 1 -- 172.21.15.43:0/3251326768 learned_addr learned my addr 172.21.15.43:0/3251326768 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.683+0000 7f3e19442640 1 --2- 172.21.15.43:0/3251326768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e1419d020 0x7f3e1419d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.683+0000 7f3e13fff640 1 -- 172.21.15.43:0/3251326768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e14102190 msgr2=0x7f3e14198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.683+0000 7f3e13fff640 1 --2- 172.21.15.43:0/3251326768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e14102190 0x7f3e14198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.683+0000 7f3e13fff640 1 -- 172.21.15.43:0/3251326768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e1419d020 msgr2=0x7f3e1419d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.683+0000 7f3e13fff640 1 --2- 172.21.15.43:0/3251326768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e1419d020 0x7f3e1419d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.683+0000 7f3e13fff640 1 -- 172.21.15.43:0/3251326768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1419db10 con 0x7f3e14107260 2024-04-21T08:42:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.684+0000 7f3e13fff640 1 --2- 172.21.15.43:0/3251326768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e14107260 0x7f3e14198ca0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f3e00036040 tx=0x7f3e000045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.684+0000 7f3e11ffb640 1 -- 172.21.15.43:0/3251326768 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e00043020 con 0x7f3e14107260 2024-04-21T08:42:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.684+0000 7f3e11ffb640 1 -- 172.21.15.43:0/3251326768 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e0002cb70 con 0x7f3e14107260 2024-04-21T08:42:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.684+0000 7f3e1aea5640 1 -- 172.21.15.43:0/3251326768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e1419dda0 con 0x7f3e14107260 2024-04-21T08:42:47.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.684+0000 7f3e11ffb640 1 -- 172.21.15.43:0/3251326768 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e00003bf0 con 0x7f3e14107260 2024-04-21T08:42:47.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.684+0000 7f3e1aea5640 1 -- 172.21.15.43:0/3251326768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e141a55c0 con 0x7f3e14107260 2024-04-21T08:42:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.686+0000 7f3e11ffb640 1 -- 172.21.15.43:0/3251326768 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3e00046020 con 0x7f3e14107260 2024-04-21T08:42:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.686+0000 7f3e1db74740 1 Processor -- start 2024-04-21T08:42:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.686+0000 7f3e1db74740 1 -- start start 2024-04-21T08:42:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.687+0000 7f3e1db74740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f85f718330 0x55f85f718710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.687+0000 7f3e1db74740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85f718ce0 0x55f85f721820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.687+0000 7f3e1db74740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f721e40 0x55f85f7222c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.687+0000 7f3e1db74740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f85f67d1f0 con 0x55f85f721e40 2024-04-21T08:42:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.687+0000 7f3e1db74740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f85f61a4b0 con 0x55f85f718ce0 2024-04-21T08:42:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.687+0000 7f3e1db74740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f85f6791b0 con 0x55f85f718330 2024-04-21T08:42:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.687+0000 7f3e11ffb640 1 --2- 172.21.15.43:0/3251326768 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3e040a3d20 0x7f3e040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.687+0000 7f3e13fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f721e40 0x55f85f7222c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.687+0000 7f3e13fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f721e40 0x55f85f7222c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34528/0 (socket says 172.21.15.43:34528) 2024-04-21T08:42:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.687+0000 7f3e13fff640 1 -- 172.21.15.43:0/2668210682 learned_addr learned my addr 172.21.15.43:0/2668210682 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.687+0000 7f3e19442640 1 --2- 172.21.15.43:0/2668210682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f85f718330 0x55f85f718710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.687+0000 7f3e19442640 1 --2- 172.21.15.43:0/3251326768 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3e040a3d20 0x7f3e040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.687+0000 7f3e18c41640 1 --2- 172.21.15.43:0/2668210682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85f718ce0 0x55f85f721820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.687+0000 7f3e11ffb640 1 -- 172.21.15.43:0/3251326768 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f3e000f2da0 con 0x7f3e14107260 2024-04-21T08:42:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.687+0000 7f3e13fff640 1 -- 172.21.15.43:0/2668210682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f85f718330 msgr2=0x55f85f718710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.687+0000 7f3e13fff640 1 --2- 172.21.15.43:0/2668210682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f85f718330 0x55f85f718710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.687+0000 7f3e13fff640 1 -- 172.21.15.43:0/2668210682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85f718ce0 msgr2=0x55f85f721820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.687+0000 7f3e13fff640 1 --2- 172.21.15.43:0/2668210682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85f718ce0 0x55f85f721820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.687+0000 7f3e13fff640 1 -- 172.21.15.43:0/2668210682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f85f60c1b0 con 0x55f85f721e40 2024-04-21T08:42:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.687+0000 7f3e18c41640 1 --2- 172.21.15.43:0/2668210682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85f718ce0 0x55f85f721820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.687+0000 7f3e13fff640 1 --2- 172.21.15.43:0/2668210682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f721e40 0x55f85f7222c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f3e000f0110 tx=0x7f3e0002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17aa68108a82e032 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.687+0000 7f3e19442640 1 --2- 172.21.15.43:0/3251326768 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3e040a3d20 0x7f3e040a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009fd0 tx=0x7f3e0c005bd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.688+0000 7f3dfeffd640 1 -- 172.21.15.43:0/2668210682 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e000be560 con 0x55f85f721e40 2024-04-21T08:42:47.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.688+0000 7f3dfeffd640 1 -- 172.21.15.43:0/2668210682 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e000bf120 con 0x55f85f721e40 2024-04-21T08:42:47.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.688+0000 7f3dfeffd640 1 -- 172.21.15.43:0/2668210682 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e000be380 con 0x55f85f721e40 2024-04-21T08:42:47.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.688+0000 7f3e1db74740 1 -- 172.21.15.43:0/2668210682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f721e40 msgr2=0x55f85f7222c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.688+0000 7f3e1db74740 1 --2- 172.21.15.43:0/2668210682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f721e40 0x55f85f7222c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f3e000f0110 tx=0x7f3e0002bc30 comp rx=0 tx=0).stop 2024-04-21T08:42:47.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.688+0000 7f3e1db74740 1 -- 172.21.15.43:0/2668210682 shutdown_connections 2024-04-21T08:42:47.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.688+0000 7f3e1db74740 1 --2- 172.21.15.43:0/2668210682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f721e40 0x55f85f7222c0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.688+0000 7f3e1db74740 1 --2- 172.21.15.43:0/2668210682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85f718ce0 0x55f85f721820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.688+0000 7f3e1db74740 1 --2- 172.21.15.43:0/2668210682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f85f718330 0x55f85f718710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.688+0000 7f3e1db74740 1 -- 172.21.15.43:0/2668210682 >> 172.21.15.43:0/2668210682 conn(0x55f85f712ba0 msgr2=0x55f85f712f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.688+0000 7f3e1db74740 1 -- 172.21.15.43:0/2668210682 shutdown_connections 2024-04-21T08:42:47.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.688+0000 7f3e1db74740 1 -- 172.21.15.43:0/2668210682 wait complete. 2024-04-21T08:42:47.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.689+0000 7f3e1db74740 1 -- start start 2024-04-21T08:42:47.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.689+0000 7f3e1db74740 1 Processor -- start 2024-04-21T08:42:47.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.689+0000 7f3e1db74740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f85f718330 0x55f85f7280b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.689+0000 7f3e1db74740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85f718ce0 0x55f85f725500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.689+0000 7f3e1db74740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f721e40 0x55f85f725a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.689+0000 7f3e1db74740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f85f728b80 con 0x55f85f721e40 2024-04-21T08:42:47.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.689+0000 7f3e1db74740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f85f636500 con 0x55f85f718ce0 2024-04-21T08:42:47.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.689+0000 7f3e1db74740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f85f728d00 con 0x55f85f718330 2024-04-21T08:42:47.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.689+0000 7f3e1db74740 10 client.0 _ll_register_callbacks cb 0x55f85f604b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:47.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.689+0000 7f3e19442640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f721e40 0x55f85f725a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.689+0000 7f3e19442640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f721e40 0x55f85f725a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34530/0 (socket says 172.21.15.43:34530) 2024-04-21T08:42:47.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.689+0000 7f3e19442640 1 -- 172.21.15.43:0/3230871325 learned_addr learned my addr 172.21.15.43:0/3230871325 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.689+0000 7f3e13fff640 1 --2- 172.21.15.43:0/3230871325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85f718ce0 0x55f85f725500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.689+0000 7f3e18c41640 1 --2- 172.21.15.43:0/3230871325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f85f718330 0x55f85f7280b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.690+0000 7f3e19442640 1 -- 172.21.15.43:0/3230871325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f85f718330 msgr2=0x55f85f7280b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.690+0000 7f3e19442640 1 --2- 172.21.15.43:0/3230871325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f85f718330 0x55f85f7280b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.690+0000 7f3e19442640 1 -- 172.21.15.43:0/3230871325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85f718ce0 msgr2=0x55f85f725500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.690+0000 7f3e19442640 1 --2- 172.21.15.43:0/3230871325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85f718ce0 0x55f85f725500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.690+0000 7f3e19442640 1 -- 172.21.15.43:0/3230871325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f85f38f3f0 con 0x55f85f721e40 2024-04-21T08:42:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.690+0000 7f3e19442640 1 --2- 172.21.15.43:0/3230871325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f721e40 0x55f85f725a40 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00db80 tx=0x7f3e0c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a13b8ada140d63 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.690+0000 7f3df3fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.690+0000 7f3df3fff640 1 -- 172.21.15.43:0/3230871325 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c025030 con 0x55f85f721e40 2024-04-21T08:42:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.690+0000 7f3df3fff640 1 -- 172.21.15.43:0/3230871325 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f3e0c01b070 con 0x55f85f721e40 2024-04-21T08:42:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.690+0000 7f3e1db74740 1 -- 172.21.15.43:0/3230871325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e000bf120 con 0x55f85f721e40 2024-04-21T08:42:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.690+0000 7f3df3fff640 1 -- 172.21.15.43:0/3230871325 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3e0c004b20 con 0x55f85f721e40 2024-04-21T08:42:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.690+0000 7f3e1db74740 10 client.17385 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.690+0000 7f3e1db74740 1 -- 172.21.15.43:0/3230871325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f85f4597d0 con 0x55f85f721e40 2024-04-21T08:42:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.690+0000 7f3df3fff640 1 -- 172.21.15.43:0/3230871325 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e000bf120 con 0x55f85f721e40 2024-04-21T08:42:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.690+0000 7f3df3fff640 1 -- 172.21.15.43:0/3230871325 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3e0c01e050 con 0x55f85f721e40 2024-04-21T08:42:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.690+0000 7f3e1db74740 10 client.17385 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.690+0000 7f3e1db74740 10 client.17385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.690+0000 7f3e1db74740 20 client.17385 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.690+0000 7f3e1db74740 1 --2- 172.21.15.43:0/3230871325 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f85f735c30 0x55f85f738050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.690+0000 7f3e1db74740 4 client.17385 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.690+0000 7f3e1db74740 1 -- 172.21.15.43:0/3230871325 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f85f61c930 con 0x55f85f735c30 2024-04-21T08:42:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.691+0000 7f3e18c41640 1 --2- 172.21.15.43:0/3230871325 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f85f735c30 0x55f85f738050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.691+0000 7f3df3fff640 10 client.17385 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.691+0000 7f3e18c41640 1 --2- 172.21.15.43:0/3230871325 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f85f735c30 0x55f85f738050 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14b846a0c9692472 server_cookie=ca20e89044884972 in_seq=0 out_seq=0 2024-04-21T08:42:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.692+0000 7f3df3fff640 1 -- 172.21.15.43:0/3230871325 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f85f61c930 con 0x55f85f735c30 2024-04-21T08:42:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.692+0000 7f3df3fff640 10 client.17385 handle_command_reply: tid=0 2024-04-21T08:42:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.692+0000 7f3e1db74740 1 client.17385 shutdown 2024-04-21T08:42:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.692+0000 7f3e1db74740 1 -- 172.21.15.43:0/3230871325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f721e40 msgr2=0x55f85f725a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.692+0000 7f3e1db74740 1 --2- 172.21.15.43:0/3230871325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f721e40 0x55f85f725a40 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00db80 tx=0x7f3e0c004ab0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.692+0000 7f3e1db74740 1 -- 172.21.15.43:0/3230871325 shutdown_connections 2024-04-21T08:42:47.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.692+0000 7f3e1db74740 1 --2- 172.21.15.43:0/3230871325 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f85f735c30 0x55f85f738050 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.692+0000 7f3e1db74740 1 --2- 172.21.15.43:0/3230871325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f721e40 0x55f85f725a40 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.692+0000 7f3e1db74740 1 --2- 172.21.15.43:0/3230871325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85f718ce0 0x55f85f725500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.692+0000 7f3df3fff640 0 client.17385 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:47.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.692+0000 7f3e1db74740 1 --2- 172.21.15.43:0/3230871325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f85f718330 0x55f85f7280b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.692+0000 7f3e1db74740 1 -- 172.21.15.43:0/3230871325 >> 172.21.15.43:0/3230871325 conn(0x55f85f712ba0 msgr2=0x55f85f7148f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.692+0000 7f3e1db74740 1 -- 172.21.15.43:0/3230871325 shutdown_connections 2024-04-21T08:42:47.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.692+0000 7f3e1db74740 1 -- 172.21.15.43:0/3230871325 wait complete. 2024-04-21T08:42:47.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.692+0000 7f3e1db74740 20 client.17385 trim_cache size 0 max 16384 2024-04-21T08:42:47.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.706+0000 7f3e1db74740 1 Processor -- start 2024-04-21T08:42:47.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.706+0000 7f3e1db74740 1 -- start start 2024-04-21T08:42:47.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.706+0000 7f3e1db74740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f718330 0x55f85f731a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.706+0000 7f3e1db74740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85f718ce0 0x55f85f725bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.706+0000 7f3e1db74740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f85f721e40 0x55f85f726bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.706+0000 7f3e1db74740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141ad2d0 con 0x55f85f718330 2024-04-21T08:42:47.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.706+0000 7f3e1db74740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141aee10 con 0x55f85f718ce0 2024-04-21T08:42:47.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.706+0000 7f3e1db74740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141ad9a0 con 0x55f85f721e40 2024-04-21T08:42:47.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.706+0000 7f3e18c41640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f718330 0x55f85f731a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.706+0000 7f3e18c41640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f718330 0x55f85f731a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34542/0 (socket says 172.21.15.43:34542) 2024-04-21T08:42:47.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.706+0000 7f3e18c41640 1 -- 172.21.15.43:0/2498036913 learned_addr learned my addr 172.21.15.43:0/2498036913 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.706+0000 7f3e13fff640 1 --2- 172.21.15.43:0/2498036913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85f718ce0 0x55f85f725bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.706+0000 7f3e19442640 1 --2- 172.21.15.43:0/2498036913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f85f721e40 0x55f85f726bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.706+0000 7f3e18c41640 1 -- 172.21.15.43:0/2498036913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f85f721e40 msgr2=0x55f85f726bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.706+0000 7f3e18c41640 1 --2- 172.21.15.43:0/2498036913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f85f721e40 0x55f85f726bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.706+0000 7f3e18c41640 1 -- 172.21.15.43:0/2498036913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85f718ce0 msgr2=0x55f85f725bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.706+0000 7f3e18c41640 1 --2- 172.21.15.43:0/2498036913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85f718ce0 0x55f85f725bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.706+0000 7f3e18c41640 1 -- 172.21.15.43:0/2498036913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f85f7383c0 con 0x55f85f718330 2024-04-21T08:42:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.706+0000 7f3e19442640 1 --2- 172.21.15.43:0/2498036913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f85f721e40 0x55f85f726bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.706+0000 7f3e13fff640 1 --2- 172.21.15.43:0/2498036913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85f718ce0 0x55f85f725bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.706+0000 7f3e18c41640 1 --2- 172.21.15.43:0/2498036913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f718330 0x55f85f731a40 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f3e08009c30 tx=0x7f3e080097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86da24799bbb0d7d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.706+0000 7f3dfe7fc640 1 -- 172.21.15.43:0/2498036913 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e080149e0 con 0x55f85f718330 2024-04-21T08:42:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.706+0000 7f3dfe7fc640 1 -- 172.21.15.43:0/2498036913 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e0801b330 con 0x55f85f718330 2024-04-21T08:42:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.706+0000 7f3dfe7fc640 1 -- 172.21.15.43:0/2498036913 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e08012850 con 0x55f85f718330 2024-04-21T08:42:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.707+0000 7f3e1db74740 1 -- 172.21.15.43:0/2498036913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f718330 msgr2=0x55f85f731a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.707+0000 7f3e1db74740 1 --2- 172.21.15.43:0/2498036913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f718330 0x55f85f731a40 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f3e08009c30 tx=0x7f3e080097d0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.707+0000 7f3e1db74740 1 -- 172.21.15.43:0/2498036913 shutdown_connections 2024-04-21T08:42:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.707+0000 7f3e1db74740 1 --2- 172.21.15.43:0/2498036913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f85f721e40 0x55f85f726bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.707+0000 7f3e1db74740 1 --2- 172.21.15.43:0/2498036913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85f718ce0 0x55f85f725bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.707+0000 7f3e1db74740 1 --2- 172.21.15.43:0/2498036913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f718330 0x55f85f731a40 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.707+0000 7f3e1db74740 1 -- 172.21.15.43:0/2498036913 >> 172.21.15.43:0/2498036913 conn(0x55f85f712ba0 msgr2=0x55f85f7254c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.707+0000 7f3e1db74740 1 -- 172.21.15.43:0/2498036913 shutdown_connections 2024-04-21T08:42:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.707+0000 7f3e1db74740 1 -- 172.21.15.43:0/2498036913 wait complete. 2024-04-21T08:42:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.707+0000 7f3e1db74740 1 -- start start 2024-04-21T08:42:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.707+0000 7f3e1db74740 1 Processor -- start 2024-04-21T08:42:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.708+0000 7f3e1db74740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f85f718330 0x55f85f73d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.708+0000 7f3e1db74740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f718ce0 0x55f85f73d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.708+0000 7f3e1db74740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85f721e40 0x55f85f79b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.708+0000 7f3e1db74740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141ae740 con 0x55f85f718ce0 2024-04-21T08:42:47.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.708+0000 7f3e1db74740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141ae070 con 0x55f85f721e40 2024-04-21T08:42:47.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.708+0000 7f3e1db74740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141ab2b0 con 0x55f85f718330 2024-04-21T08:42:47.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.708+0000 7f3e1db74740 10 client.0 _ll_register_callbacks cb 0x55f85f604b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:47.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.708+0000 7f3e13fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f718ce0 0x55f85f73d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.708+0000 7f3e13fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f718ce0 0x55f85f73d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34558/0 (socket says 172.21.15.43:34558) 2024-04-21T08:42:47.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.708+0000 7f3e13fff640 1 -- 172.21.15.43:0/2509058391 learned_addr learned my addr 172.21.15.43:0/2509058391 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:47.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.708+0000 7f3e18c41640 1 --2- 172.21.15.43:0/2509058391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f85f718330 0x55f85f73d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.708+0000 7f3e19442640 1 --2- 172.21.15.43:0/2509058391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85f721e40 0x55f85f79b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.708+0000 7f3e13fff640 1 -- 172.21.15.43:0/2509058391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f85f718330 msgr2=0x55f85f73d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.708+0000 7f3e13fff640 1 --2- 172.21.15.43:0/2509058391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f85f718330 0x55f85f73d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.708+0000 7f3e13fff640 1 -- 172.21.15.43:0/2509058391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85f721e40 msgr2=0x55f85f79b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:47.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.708+0000 7f3e13fff640 1 --2- 172.21.15.43:0/2509058391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85f721e40 0x55f85f79b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.708+0000 7f3e13fff640 1 -- 172.21.15.43:0/2509058391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f85f79b6d0 con 0x55f85f718ce0 2024-04-21T08:42:47.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.708+0000 7f3e13fff640 1 --2- 172.21.15.43:0/2509058391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f718ce0 0x55f85f73d750 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f3e000bb3f0 tx=0x7f3e000f0170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6814b676af99168 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:47.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.708+0000 7f3df3fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:47.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.709+0000 7f3df3fff640 1 -- 172.21.15.43:0/2509058391 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e000be380 con 0x55f85f718ce0 2024-04-21T08:42:47.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.709+0000 7f3df3fff640 1 -- 172.21.15.43:0/2509058391 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f3e000ddbe0 con 0x55f85f718ce0 2024-04-21T08:42:47.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.709+0000 7f3e1db74740 1 -- 172.21.15.43:0/2509058391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e0801b330 con 0x55f85f718ce0 2024-04-21T08:42:47.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.709+0000 7f3df3fff640 1 -- 172.21.15.43:0/2509058391 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3e000be560 con 0x55f85f718ce0 2024-04-21T08:42:47.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.709+0000 7f3e1db74740 10 client.17391 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:47.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.709+0000 7f3e1db74740 1 -- 172.21.15.43:0/2509058391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f85f79c7e0 con 0x55f85f718ce0 2024-04-21T08:42:47.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.709+0000 7f3df3fff640 1 -- 172.21.15.43:0/2509058391 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e0801b330 con 0x55f85f718ce0 2024-04-21T08:42:47.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.709+0000 7f3df3fff640 1 -- 172.21.15.43:0/2509058391 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3e000c7200 con 0x55f85f718ce0 2024-04-21T08:42:47.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.709+0000 7f3e1db74740 10 client.17391 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:47.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.709+0000 7f3e1db74740 10 client.17391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:47.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.709+0000 7f3e1db74740 20 client.17391 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:47.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.709+0000 7f3e1db74740 1 --2- 172.21.15.43:0/2509058391 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f85f79cb80 0x55f85f79ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:47.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.709+0000 7f3e1db74740 4 client.17391 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:42:47.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.709+0000 7f3e1db74740 1 -- 172.21.15.43:0/2509058391 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f85f68ac10 con 0x55f85f79cb80 2024-04-21T08:42:47.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.710+0000 7f3e18c41640 1 --2- 172.21.15.43:0/2509058391 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f85f79cb80 0x55f85f79ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:47.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.710+0000 7f3df3fff640 10 client.17391 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:47.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.710+0000 7f3e18c41640 1 --2- 172.21.15.43:0/2509058391 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f85f79cb80 0x55f85f79ef70 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81f3e4575e5ad6fd server_cookie=6d66fcb9d319f43b in_seq=0 out_seq=0 2024-04-21T08:42:47.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.710+0000 7f3df3fff640 1 -- 172.21.15.43:0/2509058391 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f85f68ac10 con 0x55f85f79cb80 2024-04-21T08:42:47.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.710+0000 7f3df3fff640 10 client.17391 handle_command_reply: tid=0 2024-04-21T08:42:47.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.710+0000 7f3e1db74740 1 client.17391 shutdown 2024-04-21T08:42:47.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.710+0000 7f3e1db74740 1 -- 172.21.15.43:0/2509058391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f718ce0 msgr2=0x55f85f73d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.710+0000 7f3e1db74740 1 --2- 172.21.15.43:0/2509058391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f718ce0 0x55f85f73d750 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f3e000bb3f0 tx=0x7f3e000f0170 comp rx=0 tx=0).stop 2024-04-21T08:42:47.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.710+0000 7f3e1db74740 1 -- 172.21.15.43:0/2509058391 shutdown_connections 2024-04-21T08:42:47.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.710+0000 7f3e1db74740 1 --2- 172.21.15.43:0/2509058391 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f85f79cb80 0x55f85f79ef70 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.710+0000 7f3e1db74740 1 --2- 172.21.15.43:0/2509058391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85f721e40 0x55f85f79b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.710+0000 7f3e1db74740 1 --2- 172.21.15.43:0/2509058391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f85f718ce0 0x55f85f73d750 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.710+0000 7f3df3fff640 0 client.17391 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:47.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.710+0000 7f3e1db74740 1 --2- 172.21.15.43:0/2509058391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f85f718330 0x55f85f73d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.710+0000 7f3e1db74740 1 -- 172.21.15.43:0/2509058391 >> 172.21.15.43:0/2509058391 conn(0x55f85f712ba0 msgr2=0x55f85f711370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.710+0000 7f3e1db74740 1 -- 172.21.15.43:0/2509058391 shutdown_connections 2024-04-21T08:42:47.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.710+0000 7f3e1db74740 1 -- 172.21.15.43:0/2509058391 wait complete. 2024-04-21T08:42:47.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.710+0000 7f3e1db74740 20 client.17391 trim_cache size 0 max 16384 2024-04-21T08:42:47.717 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:42:47.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.711+0000 7f3e1aea5640 1 -- 172.21.15.43:0/3251326768 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3e040a3d20 msgr2=0x7f3e040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.711+0000 7f3e1aea5640 1 --2- 172.21.15.43:0/3251326768 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3e040a3d20 0x7f3e040a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009fd0 tx=0x7f3e0c005bd0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.711+0000 7f3e1aea5640 1 -- 172.21.15.43:0/3251326768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e14107260 msgr2=0x7f3e14198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:47.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.711+0000 7f3e1aea5640 1 --2- 172.21.15.43:0/3251326768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e14107260 0x7f3e14198ca0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f3e00036040 tx=0x7f3e000045e0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.711+0000 7f3e1aea5640 1 -- 172.21.15.43:0/3251326768 shutdown_connections 2024-04-21T08:42:47.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.711+0000 7f3e1aea5640 1 --2- 172.21.15.43:0/3251326768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e1419d020 0x7f3e1419d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.711+0000 7f3e1aea5640 1 --2- 172.21.15.43:0/3251326768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e14107260 0x7f3e14198ca0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.711+0000 7f3e1aea5640 1 --2- 172.21.15.43:0/3251326768 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3e040a3d20 0x7f3e040a61e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.711+0000 7f3e1aea5640 1 --2- 172.21.15.43:0/3251326768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e14102190 0x7f3e14198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:47.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.711+0000 7f3e1aea5640 1 -- 172.21.15.43:0/3251326768 >> 172.21.15.43:0/3251326768 conn(0x7f3e140fd990 msgr2=0x7f3e140fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:47.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.711+0000 7f3e1aea5640 1 -- 172.21.15.43:0/3251326768 shutdown_connections 2024-04-21T08:42:47.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:47.711+0000 7f3e1aea5640 1 -- 172.21.15.43:0/3251326768 wait complete. 2024-04-21T08:42:48.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:48 smithi175 ceph-mon[31920]: pgmap v119: 129 pgs: 129 active+clean; 4.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-21T08:42:48.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:48 smithi112 ceph-mon[31085]: pgmap v119: 129 pgs: 129 active+clean; 4.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-21T08:42:48.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:48 smithi043 ceph-mon[27978]: pgmap v119: 129 pgs: 129 active+clean; 4.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-21T08:42:48.722 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:42:48.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:42:48.722 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:42:48.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.843+0000 7f3c40b4a640 1 -- 172.21.15.43:0/2385019094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c3c101a50 msgr2=0x7f3c3c105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:48.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.843+0000 7f3c40b4a640 1 --2- 172.21.15.43:0/2385019094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c3c101a50 0x7f3c3c105a90 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009900 tx=0x7f3c2402bc30 comp rx=0 tx=0).stop 2024-04-21T08:42:48.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.843+0000 7f3c40b4a640 1 -- 172.21.15.43:0/2385019094 shutdown_connections 2024-04-21T08:42:48.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.843+0000 7f3c40b4a640 1 --2- 172.21.15.43:0/2385019094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c3c106170 0x7f3c3c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.843+0000 7f3c40b4a640 1 --2- 172.21.15.43:0/2385019094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c3c101a50 0x7f3c3c105a90 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.843+0000 7f3c40b4a640 1 --2- 172.21.15.43:0/2385019094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c3c1010a0 0x7f3c3c101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.843+0000 7f3c40b4a640 1 -- 172.21.15.43:0/2385019094 >> 172.21.15.43:0/2385019094 conn(0x7f3c3c077eb0 msgr2=0x7f3c3c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:48.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.843+0000 7f3c40b4a640 1 -- 172.21.15.43:0/2385019094 shutdown_connections 2024-04-21T08:42:48.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.843+0000 7f3c40b4a640 1 -- 172.21.15.43:0/2385019094 wait complete. 2024-04-21T08:42:48.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.844+0000 7f3c40b4a640 1 Processor -- start 2024-04-21T08:42:48.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.844+0000 7f3c40b4a640 1 -- start start 2024-04-21T08:42:48.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.844+0000 7f3c40b4a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c3c1010a0 0x7f3c3c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:48.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.844+0000 7f3c40b4a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c3c106170 0x7f3c3c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:48.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.844+0000 7f3c40b4a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c3c19d010 0x7f3c3c19d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:48.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.844+0000 7f3c40b4a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c112210 con 0x7f3c3c19d010 2024-04-21T08:42:48.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.844+0000 7f3c40b4a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c112090 con 0x7f3c3c1010a0 2024-04-21T08:42:48.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.844+0000 7f3c40b4a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c112390 con 0x7f3c3c106170 2024-04-21T08:42:48.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.845+0000 7f3c3ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c3c19d010 0x7f3c3c19d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:48.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.845+0000 7f3c3ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c3c19d010 0x7f3c3c19d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34572/0 (socket says 172.21.15.43:34572) 2024-04-21T08:42:48.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.845+0000 7f3c3ad9d640 1 -- 172.21.15.43:0/826140685 learned_addr learned my addr 172.21.15.43:0/826140685 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:48.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.845+0000 7f3c3a59c640 1 --2- 172.21.15.43:0/826140685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c3c1010a0 0x7f3c3c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:48.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.845+0000 7f3c33fff640 1 --2- 172.21.15.43:0/826140685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c3c106170 0x7f3c3c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:48.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.845+0000 7f3c3ad9d640 1 -- 172.21.15.43:0/826140685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c3c106170 msgr2=0x7f3c3c198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:48.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.845+0000 7f3c3ad9d640 1 --2- 172.21.15.43:0/826140685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c3c106170 0x7f3c3c198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.845+0000 7f3c3ad9d640 1 -- 172.21.15.43:0/826140685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c3c1010a0 msgr2=0x7f3c3c198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:48.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.845+0000 7f3c3ad9d640 1 --2- 172.21.15.43:0/826140685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c3c1010a0 0x7f3c3c198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.845+0000 7f3c3ad9d640 1 -- 172.21.15.43:0/826140685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3c19db00 con 0x7f3c3c19d010 2024-04-21T08:42:48.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.845+0000 7f3c3ad9d640 1 --2- 172.21.15.43:0/826140685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c3c19d010 0x7f3c3c19d4c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400cce0 tx=0x7f3c3400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:48.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.845+0000 7f3c337fe640 1 -- 172.21.15.43:0/826140685 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3400f450 con 0x7f3c3c19d010 2024-04-21T08:42:48.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.845+0000 7f3c337fe640 1 -- 172.21.15.43:0/826140685 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c3400faa0 con 0x7f3c3c19d010 2024-04-21T08:42:48.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.845+0000 7f3c40b4a640 1 -- 172.21.15.43:0/826140685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c3c19ddf0 con 0x7f3c3c19d010 2024-04-21T08:42:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.845+0000 7f3c337fe640 1 -- 172.21.15.43:0/826140685 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34018a80 con 0x7f3c3c19d010 2024-04-21T08:42:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.845+0000 7f3c40b4a640 1 -- 172.21.15.43:0/826140685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c3c1a5560 con 0x7f3c3c19d010 2024-04-21T08:42:48.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.848+0000 7f3c337fe640 1 -- 172.21.15.43:0/826140685 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3c3400fc50 con 0x7f3c3c19d010 2024-04-21T08:42:48.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.848+0000 7f3c43810740 1 Processor -- start 2024-04-21T08:42:48.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.848+0000 7f3c43810740 1 -- start start 2024-04-21T08:42:48.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.848+0000 7f3c43810740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598ebd858e0 0x5598ebd85cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:48.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.848+0000 7f3c43810740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598ebd86290 0x5598ebd8e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:48.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.848+0000 7f3c43810740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd8ee10 0x5598ebd91260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:48.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.848+0000 7f3c43810740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5598ebce9f00 con 0x5598ebd8ee10 2024-04-21T08:42:48.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.848+0000 7f3c43810740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5598ebc87590 con 0x5598ebd86290 2024-04-21T08:42:48.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.848+0000 7f3c43810740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5598ebce5ec0 con 0x5598ebd858e0 2024-04-21T08:42:48.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.849+0000 7f3c33fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd8ee10 0x5598ebd91260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:48.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.849+0000 7f3c33fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd8ee10 0x5598ebd91260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34588/0 (socket says 172.21.15.43:34588) 2024-04-21T08:42:48.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.849+0000 7f3c33fff640 1 -- 172.21.15.43:0/1886427065 learned_addr learned my addr 172.21.15.43:0/1886427065 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:48.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.849+0000 7f3c3a59c640 1 --2- 172.21.15.43:0/1886427065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598ebd86290 0x5598ebd8e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:48.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.849+0000 7f3c337fe640 1 --2- 172.21.15.43:0/826140685 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3c280a3d20 0x7f3c280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:48.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.849+0000 7f3c33fff640 1 --2- 172.21.15.43:0/1886427065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598ebd858e0 0x5598ebd85cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:48.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.849+0000 7f3c33fff640 1 -- 172.21.15.43:0/1886427065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598ebd858e0 msgr2=0x5598ebd85cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:48.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.849+0000 7f3c33fff640 1 --2- 172.21.15.43:0/1886427065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598ebd858e0 0x5598ebd85cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.849+0000 7f3c33fff640 1 -- 172.21.15.43:0/1886427065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598ebd86290 msgr2=0x5598ebd8e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:48.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.849+0000 7f3c33fff640 1 --2- 172.21.15.43:0/1886427065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598ebd86290 0x5598ebd8e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.849+0000 7f3c3ad9d640 1 --2- 172.21.15.43:0/826140685 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3c280a3d20 0x7f3c280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:48.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.849+0000 7f3c33fff640 1 -- 172.21.15.43:0/1886427065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598ebac47e0 con 0x5598ebd8ee10 2024-04-21T08:42:48.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.849+0000 7f3c3a59c640 1 --2- 172.21.15.43:0/1886427065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598ebd86290 0x5598ebd8e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:42:48.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.849+0000 7f3c337fe640 1 -- 172.21.15.43:0/826140685 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f3c340d10b0 con 0x7f3c3c19d010 2024-04-21T08:42:48.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.849+0000 7f3c33fff640 1 --2- 172.21.15.43:0/1886427065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd8ee10 0x5598ebd91260 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f3c24005ec0 tx=0x7f3c24005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3924287c852c2766 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:48.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.849+0000 7f3c30ff9640 1 -- 172.21.15.43:0/1886427065 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2402c9e0 con 0x5598ebd8ee10 2024-04-21T08:42:48.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.849+0000 7f3c3ad9d640 1 --2- 172.21.15.43:0/826140685 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3c280a3d20 0x7f3c280a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3c340ce4e0 tx=0x7f3c3406e6c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:48.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.850+0000 7f3c30ff9640 1 -- 172.21.15.43:0/1886427065 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c24048070 con 0x5598ebd8ee10 2024-04-21T08:42:48.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.850+0000 7f3c30ff9640 1 -- 172.21.15.43:0/1886427065 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c24004a40 con 0x5598ebd8ee10 2024-04-21T08:42:48.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.850+0000 7f3c43810740 1 -- 172.21.15.43:0/1886427065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd8ee10 msgr2=0x5598ebd91260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:48.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.850+0000 7f3c43810740 1 --2- 172.21.15.43:0/1886427065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd8ee10 0x5598ebd91260 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f3c24005ec0 tx=0x7f3c24005e30 comp rx=0 tx=0).stop 2024-04-21T08:42:48.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.850+0000 7f3c43810740 1 -- 172.21.15.43:0/1886427065 shutdown_connections 2024-04-21T08:42:48.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.850+0000 7f3c43810740 1 --2- 172.21.15.43:0/1886427065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd8ee10 0x5598ebd91260 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.850+0000 7f3c43810740 1 --2- 172.21.15.43:0/1886427065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598ebd86290 0x5598ebd8e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.850+0000 7f3c43810740 1 --2- 172.21.15.43:0/1886427065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598ebd858e0 0x5598ebd85cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.850+0000 7f3c43810740 1 -- 172.21.15.43:0/1886427065 >> 172.21.15.43:0/1886427065 conn(0x5598ebd7ffb0 msgr2=0x5598ebd80390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:48.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.850+0000 7f3c43810740 1 -- 172.21.15.43:0/1886427065 shutdown_connections 2024-04-21T08:42:48.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.850+0000 7f3c43810740 1 -- 172.21.15.43:0/1886427065 wait complete. 2024-04-21T08:42:48.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.851+0000 7f3c43810740 1 -- start start 2024-04-21T08:42:48.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.851+0000 7f3c43810740 1 Processor -- start 2024-04-21T08:42:48.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.851+0000 7f3c43810740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598ebd858e0 0x5598ebd8bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:48.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.851+0000 7f3c43810740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd86290 0x5598ebd8c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:48.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.851+0000 7f3c43810740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598ebd8ee10 0x5598ebd8c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:48.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.851+0000 7f3c43810740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5598ebd91d20 con 0x5598ebd86290 2024-04-21T08:42:48.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.851+0000 7f3c43810740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5598ebca3580 con 0x5598ebd858e0 2024-04-21T08:42:48.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.851+0000 7f3c43810740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5598ebd91ea0 con 0x5598ebd8ee10 2024-04-21T08:42:48.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.851+0000 7f3c43810740 10 client.0 _ll_register_callbacks cb 0x5598ebca9650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:48.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.851+0000 7f3c3a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd86290 0x5598ebd8c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:48.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.851+0000 7f3c3a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd86290 0x5598ebd8c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34602/0 (socket says 172.21.15.43:34602) 2024-04-21T08:42:48.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.851+0000 7f3c3a59c640 1 -- 172.21.15.43:0/4203739260 learned_addr learned my addr 172.21.15.43:0/4203739260 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:48.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.851+0000 7f3c33fff640 1 --2- 172.21.15.43:0/4203739260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598ebd858e0 0x5598ebd8bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:48.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.851+0000 7f3c33fff640 1 --2- 172.21.15.43:0/4203739260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598ebd8ee10 0x5598ebd8c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:48.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.851+0000 7f3c3a59c640 1 -- 172.21.15.43:0/4203739260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598ebd8ee10 msgr2=0x5598ebd8c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:48.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.851+0000 7f3c3a59c640 1 --2- 172.21.15.43:0/4203739260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598ebd8ee10 0x5598ebd8c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.851+0000 7f3c3a59c640 1 -- 172.21.15.43:0/4203739260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598ebd858e0 msgr2=0x5598ebd8bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:48.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.851+0000 7f3c3a59c640 1 --2- 172.21.15.43:0/4203739260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598ebd858e0 0x5598ebd8bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.851+0000 7f3c3a59c640 1 -- 172.21.15.43:0/4203739260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598ebb24f70 con 0x5598ebd86290 2024-04-21T08:42:48.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.851+0000 7f3c33fff640 1 --2- 172.21.15.43:0/4203739260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598ebd8ee10 0x5598ebd8c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:48.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.852+0000 7f3c3a59c640 1 --2- 172.21.15.43:0/4203739260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd86290 0x5598ebd8c320 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c011570 tx=0x7f3c2c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d74c76805cda4fe server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:48.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.852+0000 7f3c21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:48.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.852+0000 7f3c21ffb640 1 -- 172.21.15.43:0/4203739260 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c013670 con 0x5598ebd86290 2024-04-21T08:42:48.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.852+0000 7f3c21ffb640 1 -- 172.21.15.43:0/4203739260 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f3c2c013810 con 0x5598ebd86290 2024-04-21T08:42:48.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.852+0000 7f3c43810740 1 -- 172.21.15.43:0/4203739260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c24048070 con 0x5598ebd86290 2024-04-21T08:42:48.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.852+0000 7f3c21ffb640 1 -- 172.21.15.43:0/4203739260 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c2c026050 con 0x5598ebd86290 2024-04-21T08:42:48.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.852+0000 7f3c43810740 10 client.17403 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:48.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.852+0000 7f3c43810740 1 -- 172.21.15.43:0/4203739260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5598eb9bc170 con 0x5598ebd86290 2024-04-21T08:42:48.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.852+0000 7f3c21ffb640 1 -- 172.21.15.43:0/4203739260 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c24048070 con 0x5598ebd86290 2024-04-21T08:42:48.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.852+0000 7f3c21ffb640 1 -- 172.21.15.43:0/4203739260 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3c2c014bb0 con 0x5598ebd86290 2024-04-21T08:42:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.852+0000 7f3c43810740 10 client.17403 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.852+0000 7f3c43810740 10 client.17403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.852+0000 7f3c43810740 20 client.17403 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.852+0000 7f3c43810740 1 --2- 172.21.15.43:0/4203739260 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5598ebd9f340 0x5598ebda1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.852+0000 7f3c43810740 4 client.17403 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.852+0000 7f3c43810740 1 -- 172.21.15.43:0/4203739260 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598ebcf7920 con 0x5598ebd9f340 2024-04-21T08:42:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.853+0000 7f3c33fff640 1 --2- 172.21.15.43:0/4203739260 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5598ebd9f340 0x5598ebda1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.853+0000 7f3c21ffb640 10 client.17403 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.853+0000 7f3c33fff640 1 --2- 172.21.15.43:0/4203739260 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5598ebd9f340 0x5598ebda1790 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c31b73f1f24770ed server_cookie=db849c48d051f9c3 in_seq=0 out_seq=0 2024-04-21T08:42:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.854+0000 7f3c21ffb640 1 -- 172.21.15.43:0/4203739260 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5598ebcf7920 con 0x5598ebd9f340 2024-04-21T08:42:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.854+0000 7f3c21ffb640 10 client.17403 handle_command_reply: tid=0 2024-04-21T08:42:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.854+0000 7f3c43810740 1 client.17403 shutdown 2024-04-21T08:42:48.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.854+0000 7f3c43810740 1 -- 172.21.15.43:0/4203739260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd86290 msgr2=0x5598ebd8c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:48.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.854+0000 7f3c43810740 1 --2- 172.21.15.43:0/4203739260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd86290 0x5598ebd8c320 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c011570 tx=0x7f3c2c011a30 comp rx=0 tx=0).stop 2024-04-21T08:42:48.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.854+0000 7f3c43810740 1 -- 172.21.15.43:0/4203739260 shutdown_connections 2024-04-21T08:42:48.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.854+0000 7f3c43810740 1 --2- 172.21.15.43:0/4203739260 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5598ebd9f340 0x5598ebda1790 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.854+0000 7f3c43810740 1 --2- 172.21.15.43:0/4203739260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598ebd8ee10 0x5598ebd8c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.854+0000 7f3c21ffb640 0 client.17403 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:48.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.854+0000 7f3c43810740 1 --2- 172.21.15.43:0/4203739260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd86290 0x5598ebd8c320 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.854+0000 7f3c43810740 1 --2- 172.21.15.43:0/4203739260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598ebd858e0 0x5598ebd8bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.854+0000 7f3c43810740 1 -- 172.21.15.43:0/4203739260 >> 172.21.15.43:0/4203739260 conn(0x5598ebd7ffb0 msgr2=0x5598ebd81d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:48.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.854+0000 7f3c43810740 1 -- 172.21.15.43:0/4203739260 shutdown_connections 2024-04-21T08:42:48.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.854+0000 7f3c43810740 1 -- 172.21.15.43:0/4203739260 wait complete. 2024-04-21T08:42:48.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.854+0000 7f3c43810740 20 client.17403 trim_cache size 0 max 16384 2024-04-21T08:42:48.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.872+0000 7f3c43810740 1 Processor -- start 2024-04-21T08:42:48.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.872+0000 7f3c43810740 1 -- start start 2024-04-21T08:42:48.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.872+0000 7f3c43810740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598ebd858e0 0x5598ebd8e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:48.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.872+0000 7f3c43810740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598ebd86290 0x5598ebd8ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:48.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.872+0000 7f3c43810740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd8ee10 0x5598ebd9abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:48.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.872+0000 7f3c43810740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c1af2e0 con 0x5598ebd8ee10 2024-04-21T08:42:48.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.872+0000 7f3c43810740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c1b0e20 con 0x5598ebd86290 2024-04-21T08:42:48.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.872+0000 7f3c43810740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c1af9b0 con 0x5598ebd858e0 2024-04-21T08:42:48.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.872+0000 7f3c33fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd8ee10 0x5598ebd9abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:48.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.872+0000 7f3c33fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd8ee10 0x5598ebd9abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34614/0 (socket says 172.21.15.43:34614) 2024-04-21T08:42:48.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.872+0000 7f3c33fff640 1 -- 172.21.15.43:0/3622747576 learned_addr learned my addr 172.21.15.43:0/3622747576 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:48.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.872+0000 7f3c3a59c640 1 --2- 172.21.15.43:0/3622747576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598ebd86290 0x5598ebd8ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:48.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.873+0000 7f3c33fff640 1 --2- 172.21.15.43:0/3622747576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598ebd858e0 0x5598ebd8e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:48.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.873+0000 7f3c33fff640 1 -- 172.21.15.43:0/3622747576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598ebd858e0 msgr2=0x5598ebd8e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:48.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.873+0000 7f3c33fff640 1 --2- 172.21.15.43:0/3622747576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598ebd858e0 0x5598ebd8e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.873+0000 7f3c33fff640 1 -- 172.21.15.43:0/3622747576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598ebd86290 msgr2=0x5598ebd8ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:48.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.873+0000 7f3c33fff640 1 --2- 172.21.15.43:0/3622747576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598ebd86290 0x5598ebd8ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.873+0000 7f3c33fff640 1 -- 172.21.15.43:0/3622747576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598ebda1e10 con 0x5598ebd8ee10 2024-04-21T08:42:48.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.873+0000 7f3c33fff640 1 --2- 172.21.15.43:0/3622747576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd8ee10 0x5598ebd9abe0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f3c24004730 tx=0x7f3c24004210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b1f5b6402dd399 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:48.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.873+0000 7f3c23fff640 1 -- 172.21.15.43:0/3622747576 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2402cca0 con 0x5598ebd8ee10 2024-04-21T08:42:48.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.873+0000 7f3c23fff640 1 -- 172.21.15.43:0/3622747576 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c24048070 con 0x5598ebd8ee10 2024-04-21T08:42:48.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.873+0000 7f3c23fff640 1 -- 172.21.15.43:0/3622747576 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c24043480 con 0x5598ebd8ee10 2024-04-21T08:42:48.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.874+0000 7f3c43810740 1 -- 172.21.15.43:0/3622747576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd8ee10 msgr2=0x5598ebd9abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:48.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.874+0000 7f3c43810740 1 --2- 172.21.15.43:0/3622747576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd8ee10 0x5598ebd9abe0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f3c24004730 tx=0x7f3c24004210 comp rx=0 tx=0).stop 2024-04-21T08:42:48.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.874+0000 7f3c43810740 1 -- 172.21.15.43:0/3622747576 shutdown_connections 2024-04-21T08:42:48.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.874+0000 7f3c43810740 1 --2- 172.21.15.43:0/3622747576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd8ee10 0x5598ebd9abe0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.874+0000 7f3c43810740 1 --2- 172.21.15.43:0/3622747576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598ebd86290 0x5598ebd8ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.874+0000 7f3c43810740 1 --2- 172.21.15.43:0/3622747576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598ebd858e0 0x5598ebd8e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.874+0000 7f3c43810740 1 -- 172.21.15.43:0/3622747576 >> 172.21.15.43:0/3622747576 conn(0x5598ebd7ffb0 msgr2=0x5598ebd81c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:48.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.874+0000 7f3c43810740 1 -- 172.21.15.43:0/3622747576 shutdown_connections 2024-04-21T08:42:48.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.874+0000 7f3c43810740 1 -- 172.21.15.43:0/3622747576 wait complete. 2024-04-21T08:42:48.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.874+0000 7f3c43810740 1 -- start start 2024-04-21T08:42:48.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.874+0000 7f3c43810740 1 Processor -- start 2024-04-21T08:42:48.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.874+0000 7f3c43810740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598ebd858e0 0x5598ebd93750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:48.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.874+0000 7f3c43810740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd86290 0x5598ebd93c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:48.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.874+0000 7f3c43810740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598ebd8ee10 0x5598ebdfd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:48.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.874+0000 7f3c43810740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c1b0750 con 0x5598ebd86290 2024-04-21T08:42:48.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.874+0000 7f3c43810740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c1b0080 con 0x5598ebd8ee10 2024-04-21T08:42:48.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.874+0000 7f3c43810740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c1ad2c0 con 0x5598ebd858e0 2024-04-21T08:42:48.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.874+0000 7f3c43810740 10 client.0 _ll_register_callbacks cb 0x7f3c2c0095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:48.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.874+0000 7f3c3a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd86290 0x5598ebd93c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:48.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.874+0000 7f3c3a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd86290 0x5598ebd93c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34624/0 (socket says 172.21.15.43:34624) 2024-04-21T08:42:48.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.874+0000 7f3c3a59c640 1 -- 172.21.15.43:0/799581896 learned_addr learned my addr 172.21.15.43:0/799581896 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:48.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.875+0000 7f3c33fff640 1 --2- 172.21.15.43:0/799581896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598ebd8ee10 0x5598ebdfd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:48.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.875+0000 7f3c33fff640 1 --2- 172.21.15.43:0/799581896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598ebd858e0 0x5598ebd93750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:48.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.875+0000 7f3c3a59c640 1 -- 172.21.15.43:0/799581896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598ebd858e0 msgr2=0x5598ebd93750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:48.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.875+0000 7f3c3a59c640 1 --2- 172.21.15.43:0/799581896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598ebd858e0 0x5598ebd93750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.875+0000 7f3c3a59c640 1 -- 172.21.15.43:0/799581896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598ebd8ee10 msgr2=0x5598ebdfd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:48.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.875+0000 7f3c3a59c640 1 --2- 172.21.15.43:0/799581896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598ebd8ee10 0x5598ebdfd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.875+0000 7f3c3a59c640 1 -- 172.21.15.43:0/799581896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598ebdfe0d0 con 0x5598ebd86290 2024-04-21T08:42:48.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.875+0000 7f3c33fff640 1 --2- 172.21.15.43:0/799581896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598ebd8ee10 0x5598ebdfd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:48.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.875+0000 7f3c3a59c640 1 --2- 172.21.15.43:0/799581896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd86290 0x5598ebd93c90 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c004810 tx=0x7f3c2c009210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9817d3482805a14 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:48.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.875+0000 7f3c21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:48.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.875+0000 7f3c21ffb640 1 -- 172.21.15.43:0/799581896 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c02e020 con 0x5598ebd86290 2024-04-21T08:42:48.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.875+0000 7f3c21ffb640 1 -- 172.21.15.43:0/799581896 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f3c2c013880 con 0x5598ebd86290 2024-04-21T08:42:48.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.875+0000 7f3c43810740 1 -- 172.21.15.43:0/799581896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c24048070 con 0x5598ebd86290 2024-04-21T08:42:48.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.875+0000 7f3c21ffb640 1 -- 172.21.15.43:0/799581896 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c2c039050 con 0x5598ebd86290 2024-04-21T08:42:48.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.875+0000 7f3c43810740 10 client.17409 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:48.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.875+0000 7f3c43810740 1 -- 172.21.15.43:0/799581896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5598ebe064a0 con 0x5598ebd86290 2024-04-21T08:42:48.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.875+0000 7f3c21ffb640 1 -- 172.21.15.43:0/799581896 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c24048070 con 0x5598ebd86290 2024-04-21T08:42:48.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.876+0000 7f3c21ffb640 1 -- 172.21.15.43:0/799581896 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3c2c014440 con 0x5598ebd86290 2024-04-21T08:42:48.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.876+0000 7f3c43810740 10 client.17409 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:48.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.876+0000 7f3c43810740 10 client.17409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:48.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.876+0000 7f3c43810740 20 client.17409 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:48.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.876+0000 7f3c43810740 1 --2- 172.21.15.43:0/799581896 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5598ebe06ca0 0x5598ebe090c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:48.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.876+0000 7f3c43810740 4 client.17409 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:42:48.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.876+0000 7f3c43810740 1 -- 172.21.15.43:0/799581896 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5598ebcf1790 con 0x5598ebe06ca0 2024-04-21T08:42:48.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.876+0000 7f3c33fff640 1 --2- 172.21.15.43:0/799581896 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5598ebe06ca0 0x5598ebe090c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:48.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.876+0000 7f3c21ffb640 10 client.17409 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:48.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.876+0000 7f3c33fff640 1 --2- 172.21.15.43:0/799581896 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5598ebe06ca0 0x5598ebe090c0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80fe3b1261b9990f server_cookie=4e0d5b5ce59e568b in_seq=0 out_seq=0 2024-04-21T08:42:48.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.877+0000 7f3c21ffb640 1 -- 172.21.15.43:0/799581896 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5598ebcf1790 con 0x5598ebe06ca0 2024-04-21T08:42:48.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.877+0000 7f3c21ffb640 10 client.17409 handle_command_reply: tid=0 2024-04-21T08:42:48.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.877+0000 7f3c43810740 1 client.17409 shutdown 2024-04-21T08:42:48.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.878+0000 7f3c43810740 1 -- 172.21.15.43:0/799581896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd86290 msgr2=0x5598ebd93c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:48.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.878+0000 7f3c43810740 1 --2- 172.21.15.43:0/799581896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd86290 0x5598ebd93c90 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c004810 tx=0x7f3c2c009210 comp rx=0 tx=0).stop 2024-04-21T08:42:48.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.878+0000 7f3c43810740 1 -- 172.21.15.43:0/799581896 shutdown_connections 2024-04-21T08:42:48.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.878+0000 7f3c43810740 1 --2- 172.21.15.43:0/799581896 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5598ebe06ca0 0x5598ebe090c0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.878+0000 7f3c43810740 1 --2- 172.21.15.43:0/799581896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598ebd8ee10 0x5598ebdfd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.878+0000 7f3c43810740 1 --2- 172.21.15.43:0/799581896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598ebd86290 0x5598ebd93c90 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.878+0000 7f3c43810740 1 --2- 172.21.15.43:0/799581896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5598ebd858e0 0x5598ebd93750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.878+0000 7f3c43810740 1 -- 172.21.15.43:0/799581896 >> 172.21.15.43:0/799581896 conn(0x5598ebd7ffb0 msgr2=0x5598ebd8bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:48.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.878+0000 7f3c21ffb640 0 client.17409 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:48.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.878+0000 7f3c43810740 1 -- 172.21.15.43:0/799581896 shutdown_connections 2024-04-21T08:42:48.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.878+0000 7f3c43810740 1 -- 172.21.15.43:0/799581896 wait complete. 2024-04-21T08:42:48.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.878+0000 7f3c43810740 20 client.17409 trim_cache size 0 max 16384 2024-04-21T08:42:48.884 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:42:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:42:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "e8967d17-0025-43c1-960d-f46a63aa6d13", 2024-04-21T08:42:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:42:48.885 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:42:48.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.878+0000 7f3c40b4a640 1 -- 172.21.15.43:0/826140685 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3c280a3d20 msgr2=0x7f3c280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:48.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.878+0000 7f3c40b4a640 1 --2- 172.21.15.43:0/826140685 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3c280a3d20 0x7f3c280a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3c340ce4e0 tx=0x7f3c3406e6c0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.878+0000 7f3c40b4a640 1 -- 172.21.15.43:0/826140685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c3c19d010 msgr2=0x7f3c3c19d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:48.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.878+0000 7f3c40b4a640 1 --2- 172.21.15.43:0/826140685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c3c19d010 0x7f3c3c19d4c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400cce0 tx=0x7f3c3400a620 comp rx=0 tx=0).stop 2024-04-21T08:42:48.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.878+0000 7f3c40b4a640 1 -- 172.21.15.43:0/826140685 shutdown_connections 2024-04-21T08:42:48.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.878+0000 7f3c40b4a640 1 --2- 172.21.15.43:0/826140685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c3c19d010 0x7f3c3c19d4c0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.878+0000 7f3c40b4a640 1 --2- 172.21.15.43:0/826140685 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3c280a3d20 0x7f3c280a61e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.878+0000 7f3c40b4a640 1 --2- 172.21.15.43:0/826140685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c3c106170 0x7f3c3c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.878+0000 7f3c40b4a640 1 --2- 172.21.15.43:0/826140685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c3c1010a0 0x7f3c3c198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.879+0000 7f3c40b4a640 1 -- 172.21.15.43:0/826140685 >> 172.21.15.43:0/826140685 conn(0x7f3c3c077eb0 msgr2=0x7f3c3c1005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:48.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.879+0000 7f3c40b4a640 1 -- 172.21.15.43:0/826140685 shutdown_connections 2024-04-21T08:42:48.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.879+0000 7f3c40b4a640 1 -- 172.21.15.43:0/826140685 wait complete. 2024-04-21T08:42:48.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:42:48.889 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:42:48.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.996+0000 7f47b00c1640 1 -- 172.21.15.43:0/3671895673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a8106150 msgr2=0x7f47a810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:48.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.996+0000 7f47b00c1640 1 --2- 172.21.15.43:0/3671895673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a8106150 0x7f47a810dd20 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009ca0 tx=0x7f47a002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.996+0000 7f47b00c1640 1 -- 172.21.15.43:0/3671895673 shutdown_connections 2024-04-21T08:42:48.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.996+0000 7f47b00c1640 1 --2- 172.21.15.43:0/3671895673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a8106150 0x7f47a810dd20 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.996+0000 7f47b00c1640 1 --2- 172.21.15.43:0/3671895673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47a8101a30 0x7f47a8105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.996+0000 7f47b00c1640 1 --2- 172.21.15.43:0/3671895673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47a8101080 0x7f47a8101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:48.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.996+0000 7f47b00c1640 1 -- 172.21.15.43:0/3671895673 >> 172.21.15.43:0/3671895673 conn(0x7f47a8078000 msgr2=0x7f47a8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.996+0000 7f47b00c1640 1 -- 172.21.15.43:0/3671895673 shutdown_connections 2024-04-21T08:42:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.996+0000 7f47b00c1640 1 -- 172.21.15.43:0/3671895673 wait complete. 2024-04-21T08:42:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.997+0000 7f47b00c1640 1 Processor -- start 2024-04-21T08:42:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.997+0000 7f47b00c1640 1 -- start start 2024-04-21T08:42:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.997+0000 7f47b00c1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47a8101080 0x7f47a8199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.997+0000 7f47b00c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a8106150 0x7f47a8199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.997+0000 7f47b00c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47a819a000 0x7f47a81a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.997+0000 7f47b00c1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81110c0 con 0x7f47a8106150 2024-04-21T08:42:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.997+0000 7f47b00c1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f47a8110f40 con 0x7f47a819a000 2024-04-21T08:42:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.997+0000 7f47b00c1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f47a8111240 con 0x7f47a8101080 2024-04-21T08:42:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.997+0000 7f47ad65c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a8106150 0x7f47a8199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.997+0000 7f47ad65c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a8106150 0x7f47a8199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34636/0 (socket says 172.21.15.43:34636) 2024-04-21T08:42:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.997+0000 7f47ad65c640 1 -- 172.21.15.43:0/527490885 learned_addr learned my addr 172.21.15.43:0/527490885 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.998+0000 7f47ae65e640 1 --2- 172.21.15.43:0/527490885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47a819a000 0x7f47a81a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.998+0000 7f47ad65c640 1 -- 172.21.15.43:0/527490885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47a8101080 msgr2=0x7f47a8199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.998+0000 7f47ad65c640 1 --2- 172.21.15.43:0/527490885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47a8101080 0x7f47a8199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.998+0000 7f47ad65c640 1 -- 172.21.15.43:0/527490885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47a819a000 msgr2=0x7f47a81a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.998+0000 7f47ad65c640 1 --2- 172.21.15.43:0/527490885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47a819a000 0x7f47a81a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.998+0000 7f47ad65c640 1 -- 172.21.15.43:0/527490885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a81a4d80 con 0x7f47a8106150 2024-04-21T08:42:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.998+0000 7f47ad65c640 1 --2- 172.21.15.43:0/527490885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a8106150 0x7f47a8199ac0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f479800d740 tx=0x7f479800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.998+0000 7f479effd640 1 -- 172.21.15.43:0/527490885 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798017070 con 0x7f47a8106150 2024-04-21T08:42:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.998+0000 7f479effd640 1 -- 172.21.15.43:0/527490885 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f479800de50 con 0x7f47a8106150 2024-04-21T08:42:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.998+0000 7f47b00c1640 1 -- 172.21.15.43:0/527490885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a81a5070 con 0x7f47a8106150 2024-04-21T08:42:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.998+0000 7f479effd640 1 -- 172.21.15.43:0/527490885 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798004ce0 con 0x7f47a8106150 2024-04-21T08:42:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:48.998+0000 7f47b00c1640 1 -- 172.21.15.43:0/527490885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a81a55c0 con 0x7f47a8106150 2024-04-21T08:42:49.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.000+0000 7f479effd640 1 -- 172.21.15.43:0/527490885 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4798004e80 con 0x7f47a8106150 2024-04-21T08:42:49.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.001+0000 7f47b2d94740 1 Processor -- start 2024-04-21T08:42:49.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.001+0000 7f47b2d94740 1 -- start start 2024-04-21T08:42:49.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.001+0000 7f47b2d94740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd8123580 0x559bd8123960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:49.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.001+0000 7f47b2d94740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123ea0 0x559bd812c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:49.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.001+0000 7f47b2d94740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bd812ca80 0x559bd812ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:49.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.001+0000 7f47b2d94740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559bd80881e0 con 0x559bd8123ea0 2024-04-21T08:42:49.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.001+0000 7f47b2d94740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559bd8025390 con 0x559bd812ca80 2024-04-21T08:42:49.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.001+0000 7f47b2d94740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559bd80841a0 con 0x559bd8123580 2024-04-21T08:42:49.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.001+0000 7f47ade5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123ea0 0x559bd812c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:49.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.001+0000 7f47ade5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123ea0 0x559bd812c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34650/0 (socket says 172.21.15.43:34650) 2024-04-21T08:42:49.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.001+0000 7f47ade5d640 1 -- 172.21.15.43:0/318763173 learned_addr learned my addr 172.21.15.43:0/318763173 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:49.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.001+0000 7f47ae65e640 1 --2- 172.21.15.43:0/318763173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd8123580 0x559bd8123960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:49.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.001+0000 7f47ad65c640 1 --2- 172.21.15.43:0/318763173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bd812ca80 0x559bd812ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:49.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.001+0000 7f479effd640 1 --2- 172.21.15.43:0/527490885 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f47940a3da0 0x7f47940a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:49.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.001+0000 7f47ade5d640 1 -- 172.21.15.43:0/318763173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd8123580 msgr2=0x559bd8123960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:49.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.001+0000 7f47ade5d640 1 --2- 172.21.15.43:0/318763173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd8123580 0x559bd8123960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.001+0000 7f47ade5d640 1 -- 172.21.15.43:0/318763173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bd812ca80 msgr2=0x559bd812ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:49.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.001+0000 7f47ae65e640 1 --2- 172.21.15.43:0/527490885 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f47940a3da0 0x7f47940a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:49.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.001+0000 7f47ade5d640 1 --2- 172.21.15.43:0/318763173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bd812ca80 0x559bd812ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.001+0000 7f47ade5d640 1 -- 172.21.15.43:0/318763173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bd7e647d0 con 0x559bd8123ea0 2024-04-21T08:42:49.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.001+0000 7f47ad65c640 1 --2- 172.21.15.43:0/318763173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bd812ca80 0x559bd812ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:49.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.002+0000 7f479effd640 1 -- 172.21.15.43:0/527490885 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f47980d0580 con 0x7f47a8106150 2024-04-21T08:42:49.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.002+0000 7f47ade5d640 1 --2- 172.21.15.43:0/318763173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123ea0 0x559bd812c540 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009c40 tx=0x7f47a400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3545980f02ee5b47 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:49.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.002+0000 7f47ae65e640 1 --2- 172.21.15.43:0/527490885 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f47940a3da0 0x7f47940a6260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f47a0005e30 tx=0x7f47a002c0f0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:49.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.002+0000 7f4793fff640 1 -- 172.21.15.43:0/318763173 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4005020 con 0x559bd8123ea0 2024-04-21T08:42:49.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.002+0000 7f4793fff640 1 -- 172.21.15.43:0/318763173 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a40047a0 con 0x559bd8123ea0 2024-04-21T08:42:49.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.002+0000 7f4793fff640 1 -- 172.21.15.43:0/318763173 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4010690 con 0x559bd8123ea0 2024-04-21T08:42:49.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.003+0000 7f47b2d94740 1 -- 172.21.15.43:0/318763173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123ea0 msgr2=0x559bd812c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:49.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.003+0000 7f47b2d94740 1 --2- 172.21.15.43:0/318763173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123ea0 0x559bd812c540 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009c40 tx=0x7f47a400e720 comp rx=0 tx=0).stop 2024-04-21T08:42:49.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.003+0000 7f47b2d94740 1 -- 172.21.15.43:0/318763173 shutdown_connections 2024-04-21T08:42:49.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.003+0000 7f47b2d94740 1 --2- 172.21.15.43:0/318763173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bd812ca80 0x559bd812ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.003+0000 7f47b2d94740 1 --2- 172.21.15.43:0/318763173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123ea0 0x559bd812c540 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.003+0000 7f47b2d94740 1 --2- 172.21.15.43:0/318763173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd8123580 0x559bd8123960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.003+0000 7f47b2d94740 1 -- 172.21.15.43:0/318763173 >> 172.21.15.43:0/318763173 conn(0x559bd811dcb0 msgr2=0x559bd811e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:49.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.003+0000 7f47b2d94740 1 -- 172.21.15.43:0/318763173 shutdown_connections 2024-04-21T08:42:49.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.003+0000 7f47b2d94740 1 -- 172.21.15.43:0/318763173 wait complete. 2024-04-21T08:42:49.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.003+0000 7f47b2d94740 1 -- start start 2024-04-21T08:42:49.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.003+0000 7f47b2d94740 1 Processor -- start 2024-04-21T08:42:49.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.004+0000 7f47b2d94740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123580 0x559bd8129a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:49.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.004+0000 7f47b2d94740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bd8123ea0 0x559bd8129fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:49.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.004+0000 7f47b2d94740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd812ca80 0x559bd812a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:49.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.004+0000 7f47b2d94740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559bd812f870 con 0x559bd8123580 2024-04-21T08:42:49.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.004+0000 7f47b2d94740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559bd80412f0 con 0x559bd8123ea0 2024-04-21T08:42:49.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.004+0000 7f47b2d94740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559bd812f9f0 con 0x559bd812ca80 2024-04-21T08:42:49.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.004+0000 7f47b2d94740 10 client.0 _ll_register_callbacks cb 0x559bd804e580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:49.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.004+0000 7f47ade5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123580 0x559bd8129a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:49.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.004+0000 7f47ade5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123580 0x559bd8129a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34664/0 (socket says 172.21.15.43:34664) 2024-04-21T08:42:49.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.004+0000 7f47ade5d640 1 -- 172.21.15.43:0/511376465 learned_addr learned my addr 172.21.15.43:0/511376465 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:49.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.004+0000 7f47ae65e640 1 --2- 172.21.15.43:0/511376465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd812ca80 0x559bd812a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:49.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.004+0000 7f47ad65c640 1 --2- 172.21.15.43:0/511376465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bd8123ea0 0x559bd8129fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:49.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.004+0000 7f47ade5d640 1 -- 172.21.15.43:0/511376465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd812ca80 msgr2=0x559bd812a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:49.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.004+0000 7f47ade5d640 1 --2- 172.21.15.43:0/511376465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd812ca80 0x559bd812a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.004+0000 7f47ade5d640 1 -- 172.21.15.43:0/511376465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bd8123ea0 msgr2=0x559bd8129fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:49.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.004+0000 7f47ade5d640 1 --2- 172.21.15.43:0/511376465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bd8123ea0 0x559bd8129fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.004+0000 7f47ade5d640 1 -- 172.21.15.43:0/511376465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bd7d9a3f0 con 0x559bd8123580 2024-04-21T08:42:49.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.004+0000 7f47ade5d640 1 --2- 172.21.15.43:0/511376465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123580 0x559bd8129a90 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009800 tx=0x7f47a4004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de68012d82b8149 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:49.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.004+0000 7f47917fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:49.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.004+0000 7f47917fa640 1 -- 172.21.15.43:0/511376465 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4026020 con 0x559bd8123580 2024-04-21T08:42:49.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.005+0000 7f47917fa640 1 -- 172.21.15.43:0/511376465 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f47a4005220 con 0x559bd8123580 2024-04-21T08:42:49.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.005+0000 7f47b2d94740 1 -- 172.21.15.43:0/511376465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47a40047a0 con 0x559bd8123580 2024-04-21T08:42:49.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.005+0000 7f47917fa640 1 -- 172.21.15.43:0/511376465 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f47a4031050 con 0x559bd8123580 2024-04-21T08:42:49.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.005+0000 7f47b2d94740 10 client.17421 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:49.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.005+0000 7f47b2d94740 1 -- 172.21.15.43:0/511376465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559bd7d5a0f0 con 0x559bd8123580 2024-04-21T08:42:49.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.005+0000 7f47917fa640 1 -- 172.21.15.43:0/511376465 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a40047a0 con 0x559bd8123580 2024-04-21T08:42:49.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.005+0000 7f47917fa640 1 -- 172.21.15.43:0/511376465 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f47a4010430 con 0x559bd8123580 2024-04-21T08:42:49.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.005+0000 7f47b2d94740 10 client.17421 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:49.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.005+0000 7f47b2d94740 10 client.17421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:49.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.005+0000 7f47b2d94740 20 client.17421 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:49.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.005+0000 7f47b2d94740 1 --2- 172.21.15.43:0/511376465 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559bd813ca30 0x559bd813ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:49.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.005+0000 7f47b2d94740 4 client.17421 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:49.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.005+0000 7f47b2d94740 1 -- 172.21.15.43:0/511376465 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bd8095c00 con 0x559bd813ca30 2024-04-21T08:42:49.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.005+0000 7f47ad65c640 1 --2- 172.21.15.43:0/511376465 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559bd813ca30 0x559bd813ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:49.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.006+0000 7f47ad65c640 1 --2- 172.21.15.43:0/511376465 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559bd813ca30 0x559bd813ee20 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b52f8f3bb61e4bf server_cookie=8e994c892a7c986a in_seq=0 out_seq=0 2024-04-21T08:42:49.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.006+0000 7f47917fa640 10 client.17421 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:49.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.006+0000 7f47917fa640 1 -- 172.21.15.43:0/511376465 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559bd8095c00 con 0x559bd813ca30 2024-04-21T08:42:49.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.006+0000 7f47917fa640 10 client.17421 handle_command_reply: tid=0 2024-04-21T08:42:49.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.007+0000 7f47b2d94740 1 client.17421 shutdown 2024-04-21T08:42:49.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.007+0000 7f47b2d94740 1 -- 172.21.15.43:0/511376465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123580 msgr2=0x559bd8129a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:49.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.007+0000 7f47b2d94740 1 --2- 172.21.15.43:0/511376465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123580 0x559bd8129a90 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009800 tx=0x7f47a4004f40 comp rx=0 tx=0).stop 2024-04-21T08:42:49.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.007+0000 7f47b2d94740 1 -- 172.21.15.43:0/511376465 shutdown_connections 2024-04-21T08:42:49.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.007+0000 7f47b2d94740 1 --2- 172.21.15.43:0/511376465 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559bd813ca30 0x559bd813ee20 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.007+0000 7f47b2d94740 1 --2- 172.21.15.43:0/511376465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd812ca80 0x559bd812a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.007+0000 7f47b2d94740 1 --2- 172.21.15.43:0/511376465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bd8123ea0 0x559bd8129fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.007+0000 7f47917fa640 0 client.17421 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:49.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.007+0000 7f47b2d94740 1 --2- 172.21.15.43:0/511376465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123580 0x559bd8129a90 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.007+0000 7f47b2d94740 1 -- 172.21.15.43:0/511376465 >> 172.21.15.43:0/511376465 conn(0x559bd811dcb0 msgr2=0x559bd811fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:49.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.007+0000 7f47b2d94740 1 -- 172.21.15.43:0/511376465 shutdown_connections 2024-04-21T08:42:49.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.007+0000 7f47b2d94740 1 -- 172.21.15.43:0/511376465 wait complete. 2024-04-21T08:42:49.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.007+0000 7f47b2d94740 20 client.17421 trim_cache size 0 max 16384 2024-04-21T08:42:49.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.024+0000 7f47b2d94740 1 Processor -- start 2024-04-21T08:42:49.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.024+0000 7f47b2d94740 1 -- start start 2024-04-21T08:42:49.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.025+0000 7f47b2d94740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123580 0x559bd8138730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:49.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.025+0000 7f47b2d94740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd8123ea0 0x559bd8129370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:49.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.025+0000 7f47b2d94740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bd812ca80 0x559bd812a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:49.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.025+0000 7f47b2d94740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81af420 con 0x559bd8123580 2024-04-21T08:42:49.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.025+0000 7f47b2d94740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81b0f60 con 0x559bd812ca80 2024-04-21T08:42:49.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.025+0000 7f47b2d94740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81afaf0 con 0x559bd8123ea0 2024-04-21T08:42:49.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.025+0000 7f47ade5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123580 0x559bd8138730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:49.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.025+0000 7f47ade5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123580 0x559bd8138730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34676/0 (socket says 172.21.15.43:34676) 2024-04-21T08:42:49.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.025+0000 7f47ade5d640 1 -- 172.21.15.43:0/2192167676 learned_addr learned my addr 172.21.15.43:0/2192167676 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:49.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.025+0000 7f47ae65e640 1 --2- 172.21.15.43:0/2192167676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bd812ca80 0x559bd812a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:49.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.025+0000 7f47ade5d640 1 -- 172.21.15.43:0/2192167676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd8123ea0 msgr2=0x559bd8129370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:49.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.025+0000 7f47ade5d640 1 --2- 172.21.15.43:0/2192167676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd8123ea0 0x559bd8129370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.025+0000 7f47ade5d640 1 -- 172.21.15.43:0/2192167676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bd812ca80 msgr2=0x559bd812a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:49.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.025+0000 7f47ade5d640 1 --2- 172.21.15.43:0/2192167676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bd812ca80 0x559bd812a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.025+0000 7f47ade5d640 1 -- 172.21.15.43:0/2192167676 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bd811d3e0 con 0x559bd8123580 2024-04-21T08:42:49.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.025+0000 7f47ae65e640 1 --2- 172.21.15.43:0/2192167676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bd812ca80 0x559bd812a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:49.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.025+0000 7f47ade5d640 1 --2- 172.21.15.43:0/2192167676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123580 0x559bd8138730 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f47a4005db0 tx=0x7f47a4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32ad43894fdfcfc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:49.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.025+0000 7f47937fe640 1 -- 172.21.15.43:0/2192167676 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4026050 con 0x559bd8123580 2024-04-21T08:42:49.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.025+0000 7f47937fe640 1 -- 172.21.15.43:0/2192167676 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a40047a0 con 0x559bd8123580 2024-04-21T08:42:49.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.025+0000 7f47937fe640 1 -- 172.21.15.43:0/2192167676 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a401d040 con 0x559bd8123580 2024-04-21T08:42:49.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.026+0000 7f47b2d94740 1 -- 172.21.15.43:0/2192167676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123580 msgr2=0x559bd8138730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:49.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.026+0000 7f47b2d94740 1 --2- 172.21.15.43:0/2192167676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123580 0x559bd8138730 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f47a4005db0 tx=0x7f47a4004600 comp rx=0 tx=0).stop 2024-04-21T08:42:49.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.026+0000 7f47b2d94740 1 -- 172.21.15.43:0/2192167676 shutdown_connections 2024-04-21T08:42:49.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.026+0000 7f47b2d94740 1 --2- 172.21.15.43:0/2192167676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bd812ca80 0x559bd812a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.026+0000 7f47b2d94740 1 --2- 172.21.15.43:0/2192167676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd8123ea0 0x559bd8129370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.026+0000 7f47b2d94740 1 --2- 172.21.15.43:0/2192167676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123580 0x559bd8138730 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.026+0000 7f47b2d94740 1 -- 172.21.15.43:0/2192167676 >> 172.21.15.43:0/2192167676 conn(0x559bd811dcb0 msgr2=0x559bd8129a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:49.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.026+0000 7f47b2d94740 1 -- 172.21.15.43:0/2192167676 shutdown_connections 2024-04-21T08:42:49.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.026+0000 7f47b2d94740 1 -- 172.21.15.43:0/2192167676 wait complete. 2024-04-21T08:42:49.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.026+0000 7f47b2d94740 1 -- start start 2024-04-21T08:42:49.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.026+0000 7f47b2d94740 1 Processor -- start 2024-04-21T08:42:49.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.027+0000 7f47b2d94740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123580 0x559bd8134f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:49.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.027+0000 7f47b2d94740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd8123ea0 0x559bd81263d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:49.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.027+0000 7f47b2d94740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bd812ca80 0x559bd8126910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:49.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.027+0000 7f47b2d94740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81b0890 con 0x559bd8123580 2024-04-21T08:42:49.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.027+0000 7f47b2d94740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81b01c0 con 0x559bd812ca80 2024-04-21T08:42:49.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.027+0000 7f47b2d94740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81ad400 con 0x559bd8123ea0 2024-04-21T08:42:49.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.027+0000 7f47b2d94740 10 client.0 _ll_register_callbacks cb 0x7f47a0037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:49.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.027+0000 7f47ade5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123580 0x559bd8134f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:49.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.027+0000 7f47ade5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123580 0x559bd8134f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34688/0 (socket says 172.21.15.43:34688) 2024-04-21T08:42:49.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.027+0000 7f47ade5d640 1 -- 172.21.15.43:0/1327803249 learned_addr learned my addr 172.21.15.43:0/1327803249 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:49.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.027+0000 7f47ad65c640 1 --2- 172.21.15.43:0/1327803249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd8123ea0 0x559bd81263d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:49.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.027+0000 7f47ae65e640 1 --2- 172.21.15.43:0/1327803249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bd812ca80 0x559bd8126910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:49.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.027+0000 7f47ade5d640 1 -- 172.21.15.43:0/1327803249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd8123ea0 msgr2=0x559bd81263d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:49.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.027+0000 7f47ade5d640 1 --2- 172.21.15.43:0/1327803249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd8123ea0 0x559bd81263d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.027+0000 7f47ade5d640 1 -- 172.21.15.43:0/1327803249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bd812ca80 msgr2=0x559bd8126910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:49.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.027+0000 7f47ade5d640 1 --2- 172.21.15.43:0/1327803249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bd812ca80 0x559bd8126910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.027+0000 7f47ade5d640 1 -- 172.21.15.43:0/1327803249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bd8127080 con 0x559bd8123580 2024-04-21T08:42:49.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.027+0000 7f47ae65e640 1 --2- 172.21.15.43:0/1327803249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bd812ca80 0x559bd8126910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:49.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.027+0000 7f47ade5d640 1 --2- 172.21.15.43:0/1327803249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123580 0x559bd8134f50 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f47a400e760 tx=0x7f47a4009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa206772e267e26 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:49.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.027+0000 7f47917fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:49.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.028+0000 7f47917fa640 1 -- 172.21.15.43:0/1327803249 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4026040 con 0x559bd8123580 2024-04-21T08:42:49.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.028+0000 7f47917fa640 1 -- 172.21.15.43:0/1327803249 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f47a401d070 con 0x559bd8123580 2024-04-21T08:42:49.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.028+0000 7f47b2d94740 1 -- 172.21.15.43:0/1327803249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47a40047a0 con 0x559bd8123580 2024-04-21T08:42:49.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.028+0000 7f47917fa640 1 -- 172.21.15.43:0/1327803249 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f47a4049050 con 0x559bd8123580 2024-04-21T08:42:49.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.028+0000 7f47b2d94740 10 client.17427 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:49.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.028+0000 7f47b2d94740 1 -- 172.21.15.43:0/1327803249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559bd81a2be0 con 0x559bd8123580 2024-04-21T08:42:49.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.028+0000 7f47917fa640 1 -- 172.21.15.43:0/1327803249 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a40047a0 con 0x559bd8123580 2024-04-21T08:42:49.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.028+0000 7f47917fa640 1 -- 172.21.15.43:0/1327803249 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f47a4040be0 con 0x559bd8123580 2024-04-21T08:42:49.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.028+0000 7f47b2d94740 10 client.17427 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:49.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.028+0000 7f47b2d94740 10 client.17427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:49.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.028+0000 7f47b2d94740 20 client.17427 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:49.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.028+0000 7f47b2d94740 1 --2- 172.21.15.43:0/1327803249 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559bd81a3240 0x559bd81a5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:49.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.028+0000 7f47b2d94740 4 client.17427 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:42:49.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.028+0000 7f47b2d94740 1 -- 172.21.15.43:0/1327803249 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559bd808fa70 con 0x559bd81a3240 2024-04-21T08:42:49.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.028+0000 7f47ad65c640 1 --2- 172.21.15.43:0/1327803249 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559bd81a3240 0x559bd81a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:49.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.029+0000 7f47917fa640 10 client.17427 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:49.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.029+0000 7f47ad65c640 1 --2- 172.21.15.43:0/1327803249 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559bd81a3240 0x559bd81a5660 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d606601ca109c9c3 server_cookie=738f623a46b81331 in_seq=0 out_seq=0 2024-04-21T08:42:49.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.029+0000 7f47917fa640 1 -- 172.21.15.43:0/1327803249 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559bd808fa70 con 0x559bd81a3240 2024-04-21T08:42:49.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.029+0000 7f47917fa640 10 client.17427 handle_command_reply: tid=0 2024-04-21T08:42:49.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.029+0000 7f47b2d94740 1 client.17427 shutdown 2024-04-21T08:42:49.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.029+0000 7f47b2d94740 1 -- 172.21.15.43:0/1327803249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123580 msgr2=0x559bd8134f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:49.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.029+0000 7f47b2d94740 1 --2- 172.21.15.43:0/1327803249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123580 0x559bd8134f50 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f47a400e760 tx=0x7f47a4009760 comp rx=0 tx=0).stop 2024-04-21T08:42:49.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.029+0000 7f47b2d94740 1 -- 172.21.15.43:0/1327803249 shutdown_connections 2024-04-21T08:42:49.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.029+0000 7f47b2d94740 1 --2- 172.21.15.43:0/1327803249 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559bd81a3240 0x559bd81a5660 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.029+0000 7f47b2d94740 1 --2- 172.21.15.43:0/1327803249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bd812ca80 0x559bd8126910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.029+0000 7f47b2d94740 1 --2- 172.21.15.43:0/1327803249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd8123ea0 0x559bd81263d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.029+0000 7f47917fa640 0 client.17427 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:49.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.029+0000 7f47b2d94740 1 --2- 172.21.15.43:0/1327803249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559bd8123580 0x559bd8134f50 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.029+0000 7f47b2d94740 1 -- 172.21.15.43:0/1327803249 >> 172.21.15.43:0/1327803249 conn(0x559bd811dcb0 msgr2=0x559bd812a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:49.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.029+0000 7f47b2d94740 1 -- 172.21.15.43:0/1327803249 shutdown_connections 2024-04-21T08:42:49.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.029+0000 7f47b2d94740 1 -- 172.21.15.43:0/1327803249 wait complete. 2024-04-21T08:42:49.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.029+0000 7f47b2d94740 20 client.17427 trim_cache size 0 max 16384 2024-04-21T08:42:49.037 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:42:49.037 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:42:49.037 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-21T08:42:49.037 INFO:teuthology.orchestra.run.smithi043.stdout: "e8967d17-0025-43c1-960d-f46a63aa6d13": { 2024-04-21T08:42:49.037 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-21T08:42:49.037 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "e8967d17-0025-43c1-960d-f46a63aa6d13", 2024-04-21T08:42:49.038 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-21T08:42:49.038 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:42:49.038 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:42:49.038 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:42:49.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.030+0000 7f47b00c1640 1 -- 172.21.15.43:0/527490885 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f47940a3da0 msgr2=0x7f47940a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:49.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.030+0000 7f47b00c1640 1 --2- 172.21.15.43:0/527490885 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f47940a3da0 0x7f47940a6260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f47a0005e30 tx=0x7f47a002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.030+0000 7f47b00c1640 1 -- 172.21.15.43:0/527490885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a8106150 msgr2=0x7f47a8199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:49.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.030+0000 7f47b00c1640 1 --2- 172.21.15.43:0/527490885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a8106150 0x7f47a8199ac0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f479800d740 tx=0x7f479800dc00 comp rx=0 tx=0).stop 2024-04-21T08:42:49.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.030+0000 7f47b00c1640 1 -- 172.21.15.43:0/527490885 shutdown_connections 2024-04-21T08:42:49.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.030+0000 7f47b00c1640 1 --2- 172.21.15.43:0/527490885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47a819a000 0x7f47a81a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.030+0000 7f47b00c1640 1 --2- 172.21.15.43:0/527490885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a8106150 0x7f47a8199ac0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.030+0000 7f47b00c1640 1 --2- 172.21.15.43:0/527490885 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f47940a3da0 0x7f47940a6260 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.030+0000 7f47b00c1640 1 --2- 172.21.15.43:0/527490885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47a8101080 0x7f47a8199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:49.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.030+0000 7f47b00c1640 1 -- 172.21.15.43:0/527490885 >> 172.21.15.43:0/527490885 conn(0x7f47a8078000 msgr2=0x7f47a80feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:49.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.030+0000 7f47b00c1640 1 -- 172.21.15.43:0/527490885 shutdown_connections 2024-04-21T08:42:49.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:42:49.030+0000 7f47b00c1640 1 -- 172.21.15.43:0/527490885 wait complete. 2024-04-21T08:42:49.041 INFO:tasks.cephfs.filesystem:scrub status for tag:e8967d17-0025-43c1-960d-f46a63aa6d13 - {'path': '/', 'tag': 'e8967d17-0025-43c1-960d-f46a63aa6d13', 'options': 'recursive,force'} 2024-04-21T08:42:50.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:50 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:42:50.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:50 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:42:50.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:50 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:42:50.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:50 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:42:50.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:50 smithi175 ceph-mon[31920]: pgmap v120: 129 pgs: 129 active+clean; 4.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-04-21T08:42:50.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:50 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:42:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:50 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:42:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:50 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:42:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:50 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:42:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:50 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:42:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:50 smithi112 ceph-mon[31085]: pgmap v120: 129 pgs: 129 active+clean; 4.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-04-21T08:42:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:50 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:42:50.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:50 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:42:50.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:50 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:42:50.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:50 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:42:50.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:50 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:42:50.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:50 smithi043 ceph-mon[27978]: pgmap v120: 129 pgs: 129 active+clean; 4.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-04-21T08:42:50.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:50 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:42:52.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:52 smithi175 ceph-mon[31920]: pgmap v121: 129 pgs: 129 active+clean; 4.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 23 op/s 2024-04-21T08:42:52.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:52 smithi112 ceph-mon[31085]: pgmap v121: 129 pgs: 129 active+clean; 4.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 23 op/s 2024-04-21T08:42:52.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:52 smithi043 ceph-mon[27978]: pgmap v121: 129 pgs: 129 active+clean; 4.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 23 op/s 2024-04-21T08:42:54.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:54 smithi175 ceph-mon[31920]: pgmap v122: 129 pgs: 129 active+clean; 4.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 16 op/s 2024-04-21T08:42:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:54 smithi112 ceph-mon[31085]: pgmap v122: 129 pgs: 129 active+clean; 4.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 16 op/s 2024-04-21T08:42:54.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:54 smithi043 ceph-mon[27978]: pgmap v122: 129 pgs: 129 active+clean; 4.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 16 op/s 2024-04-21T08:42:56.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:56 smithi175 ceph-mon[31920]: pgmap v123: 129 pgs: 129 active+clean; 4.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 16 op/s 2024-04-21T08:42:56.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:56 smithi112 ceph-mon[31085]: pgmap v123: 129 pgs: 129 active+clean; 4.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 16 op/s 2024-04-21T08:42:56.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:56 smithi043 ceph-mon[27978]: pgmap v123: 129 pgs: 129 active+clean; 4.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 16 op/s 2024-04-21T08:42:58.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:42:58 smithi175 ceph-mon[31920]: pgmap v124: 129 pgs: 129 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 23 op/s 2024-04-21T08:42:58.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:42:58 smithi112 ceph-mon[31085]: pgmap v124: 129 pgs: 129 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 23 op/s 2024-04-21T08:42:58.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:42:58 smithi043 ceph-mon[27978]: pgmap v124: 129 pgs: 129 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 23 op/s 2024-04-21T08:42:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.686+0000 7f4aa2670640 1 -- 172.21.15.43:0/503915974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a9c103200 msgr2=0x7f4a9c1035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.686+0000 7f4aa2670640 1 --2- 172.21.15.43:0/503915974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a9c103200 0x7f4a9c1035e0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c009900 tx=0x7f4a8c02bcd0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.686+0000 7f4aa2670640 1 -- 172.21.15.43:0/503915974 shutdown_connections 2024-04-21T08:42:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.686+0000 7f4aa2670640 1 --2- 172.21.15.43:0/503915974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a9c10c5d0 0x7f4a9c10ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.686+0000 7f4aa2670640 1 --2- 172.21.15.43:0/503915974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a9c103b20 0x7f4a9c10c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.686+0000 7f4aa2670640 1 --2- 172.21.15.43:0/503915974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a9c103200 0x7f4a9c1035e0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.686+0000 7f4aa2670640 1 -- 172.21.15.43:0/503915974 >> 172.21.15.43:0/503915974 conn(0x7f4a9c0771d0 msgr2=0x7f4a9c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.686+0000 7f4aa2670640 1 -- 172.21.15.43:0/503915974 shutdown_connections 2024-04-21T08:42:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.686+0000 7f4aa2670640 1 -- 172.21.15.43:0/503915974 wait complete. 2024-04-21T08:42:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.687+0000 7f4aa2670640 1 Processor -- start 2024-04-21T08:42:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.687+0000 7f4aa2670640 1 -- start start 2024-04-21T08:42:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.687+0000 7f4aa2670640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a9c103200 0x7f4a9c1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.687+0000 7f4aa2670640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a9c103b20 0x7f4a9c1a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.687+0000 7f4aa2670640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a9c10c5d0 0x7f4a9c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.687+0000 7f4aa2670640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c112200 con 0x7f4a9c10c5d0 2024-04-21T08:42:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.687+0000 7f4aa2670640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c112080 con 0x7f4a9c103200 2024-04-21T08:42:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.687+0000 7f4aa2670640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c112380 con 0x7f4a9c103b20 2024-04-21T08:42:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.687+0000 7f4aa0c0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a9c10c5d0 0x7f4a9c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.687+0000 7f4aa0c0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a9c10c5d0 0x7f4a9c19a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45898/0 (socket says 172.21.15.43:45898) 2024-04-21T08:42:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.687+0000 7f4aa0c0d640 1 -- 172.21.15.43:0/2395447959 learned_addr learned my addr 172.21.15.43:0/2395447959 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.688+0000 7f4a9b7fe640 1 --2- 172.21.15.43:0/2395447959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a9c103b20 0x7f4a9c1a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.688+0000 7f4a9bfff640 1 --2- 172.21.15.43:0/2395447959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a9c103200 0x7f4a9c1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.688+0000 7f4aa0c0d640 1 -- 172.21.15.43:0/2395447959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a9c103b20 msgr2=0x7f4a9c1a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.688+0000 7f4aa0c0d640 1 --2- 172.21.15.43:0/2395447959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a9c103b20 0x7f4a9c1a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.688+0000 7f4aa0c0d640 1 -- 172.21.15.43:0/2395447959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a9c103200 msgr2=0x7f4a9c1a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.688+0000 7f4aa0c0d640 1 --2- 172.21.15.43:0/2395447959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a9c103200 0x7f4a9c1a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.688+0000 7f4aa0c0d640 1 -- 172.21.15.43:0/2395447959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9c19b010 con 0x7f4a9c10c5d0 2024-04-21T08:42:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.688+0000 7f4a9bfff640 1 --2- 172.21.15.43:0/2395447959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a9c103200 0x7f4a9c1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.688+0000 7f4aa0c0d640 1 --2- 172.21.15.43:0/2395447959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a9c10c5d0 0x7f4a9c19a750 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400eb40 tx=0x7f4a9400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.688+0000 7f4a997fa640 1 -- 172.21.15.43:0/2395447959 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a94004b20 con 0x7f4a9c10c5d0 2024-04-21T08:42:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.688+0000 7f4a997fa640 1 -- 172.21.15.43:0/2395447959 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a94004cc0 con 0x7f4a9c10c5d0 2024-04-21T08:42:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.688+0000 7f4aa2670640 1 -- 172.21.15.43:0/2395447959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a9c19b300 con 0x7f4a9c10c5d0 2024-04-21T08:42:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.688+0000 7f4a997fa640 1 -- 172.21.15.43:0/2395447959 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a94010720 con 0x7f4a9c10c5d0 2024-04-21T08:42:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.688+0000 7f4aa2670640 1 -- 172.21.15.43:0/2395447959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a9c1a7610 con 0x7f4a9c10c5d0 2024-04-21T08:42:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.691+0000 7f4a997fa640 1 -- 172.21.15.43:0/2395447959 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4a94005560 con 0x7f4a9c10c5d0 2024-04-21T08:42:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.691+0000 7f4aa525b740 1 Processor -- start 2024-04-21T08:42:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.691+0000 7f4aa525b740 1 -- start start 2024-04-21T08:42:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.691+0000 7f4aa525b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad07a5d8d0 0x55ad07a5c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.691+0000 7f4aa525b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad07a5cb60 0x55ad07a5cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.691+0000 7f4aa525b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a660e0 0x55ad07a684d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.691+0000 7f4aa525b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ad079c18d0 con 0x55ad07a660e0 2024-04-21T08:42:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.691+0000 7f4aa525b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ad0795ee80 con 0x55ad07a5cb60 2024-04-21T08:42:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.691+0000 7f4aa525b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ad079bd890 con 0x55ad07a5d8d0 2024-04-21T08:42:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.691+0000 7f4a997fa640 1 --2- 172.21.15.43:0/2395447959 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4a840a3e40 0x7f4a840a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.692+0000 7f4a9b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a660e0 0x55ad07a684d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.692+0000 7f4a9b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a660e0 0x55ad07a684d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45914/0 (socket says 172.21.15.43:45914) 2024-04-21T08:42:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.692+0000 7f4a9b7fe640 1 -- 172.21.15.43:0/1664850393 learned_addr learned my addr 172.21.15.43:0/1664850393 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.692+0000 7f4aa0c0d640 1 --2- 172.21.15.43:0/2395447959 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4a840a3e40 0x7f4a840a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.692+0000 7f4a9bfff640 1 --2- 172.21.15.43:0/1664850393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad07a5cb60 0x55ad07a5cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.692+0000 7f4a9b7fe640 1 --2- 172.21.15.43:0/1664850393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad07a5d8d0 0x55ad07a5c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.692+0000 7f4a997fa640 1 -- 172.21.15.43:0/2395447959 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f4a940d1390 con 0x7f4a9c10c5d0 2024-04-21T08:42:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.692+0000 7f4a9b7fe640 1 -- 172.21.15.43:0/1664850393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad07a5d8d0 msgr2=0x55ad07a5c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.692+0000 7f4a9b7fe640 1 --2- 172.21.15.43:0/1664850393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad07a5d8d0 0x55ad07a5c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.692+0000 7f4a9b7fe640 1 -- 172.21.15.43:0/1664850393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad07a5cb60 msgr2=0x55ad07a5cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.692+0000 7f4a9b7fe640 1 --2- 172.21.15.43:0/1664850393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad07a5cb60 0x55ad07a5cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.692+0000 7f4a9b7fe640 1 -- 172.21.15.43:0/1664850393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad07980320 con 0x55ad07a660e0 2024-04-21T08:42:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.692+0000 7f4a9b7fe640 1 --2- 172.21.15.43:0/1664850393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a660e0 0x55ad07a684d0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f4a90013580 tx=0x7f4a9000c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37df53926e52dc75 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.692+0000 7f4aa0c0d640 1 --2- 172.21.15.43:0/2395447959 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4a840a3e40 0x7f4a840a6300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400eb10 tx=0x7f4a94005900 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.692+0000 7f4a827fc640 1 -- 172.21.15.43:0/1664850393 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90019030 con 0x55ad07a660e0 2024-04-21T08:42:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.692+0000 7f4a827fc640 1 -- 172.21.15.43:0/1664850393 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a900119d0 con 0x55ad07a660e0 2024-04-21T08:42:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.692+0000 7f4a827fc640 1 -- 172.21.15.43:0/1664850393 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9000fa40 con 0x55ad07a660e0 2024-04-21T08:42:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.693+0000 7f4aa525b740 1 -- 172.21.15.43:0/1664850393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a660e0 msgr2=0x55ad07a684d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.693+0000 7f4aa525b740 1 --2- 172.21.15.43:0/1664850393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a660e0 0x55ad07a684d0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f4a90013580 tx=0x7f4a9000c6b0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.693+0000 7f4aa525b740 1 -- 172.21.15.43:0/1664850393 shutdown_connections 2024-04-21T08:42:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.693+0000 7f4aa525b740 1 --2- 172.21.15.43:0/1664850393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a660e0 0x55ad07a684d0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.693+0000 7f4aa525b740 1 --2- 172.21.15.43:0/1664850393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad07a5cb60 0x55ad07a5cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.693+0000 7f4aa525b740 1 --2- 172.21.15.43:0/1664850393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad07a5d8d0 0x55ad07a5c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.693+0000 7f4aa525b740 1 -- 172.21.15.43:0/1664850393 >> 172.21.15.43:0/1664850393 conn(0x55ad07a23c90 msgr2=0x55ad07a57850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.693+0000 7f4aa525b740 1 -- 172.21.15.43:0/1664850393 shutdown_connections 2024-04-21T08:42:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.693+0000 7f4aa525b740 1 -- 172.21.15.43:0/1664850393 wait complete. 2024-04-21T08:42:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.693+0000 7f4aa525b740 1 -- start start 2024-04-21T08:42:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.694+0000 7f4aa525b740 1 Processor -- start 2024-04-21T08:42:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.694+0000 7f4aa525b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad07a5cb60 0x55ad07a65a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.694+0000 7f4aa525b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a5d8d0 0x55ad07a62e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.694+0000 7f4aa525b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad07a660e0 0x55ad07a633c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.694+0000 7f4aa525b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ad07a68e10 con 0x55ad07a5d8d0 2024-04-21T08:42:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.694+0000 7f4aa525b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ad07949eb0 con 0x55ad07a5cb60 2024-04-21T08:42:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.694+0000 7f4aa525b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ad07a68f90 con 0x55ad07a660e0 2024-04-21T08:42:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.694+0000 7f4aa525b740 10 client.0 _ll_register_callbacks cb 0x55ad07976b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.694+0000 7f4a9bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a5d8d0 0x55ad07a62e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.694+0000 7f4a9b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad07a5cb60 0x55ad07a65a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.694+0000 7f4a9bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a5d8d0 0x55ad07a62e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45924/0 (socket says 172.21.15.43:45924) 2024-04-21T08:42:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.694+0000 7f4a9bfff640 1 -- 172.21.15.43:0/162480245 learned_addr learned my addr 172.21.15.43:0/162480245 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.694+0000 7f4a9b7fe640 1 --2- 172.21.15.43:0/162480245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad07a5cb60 0x55ad07a65a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:33626/0 (socket says 172.21.15.43:33626) 2024-04-21T08:42:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.694+0000 7f4a9b7fe640 1 --2- 172.21.15.43:0/162480245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad07a660e0 0x55ad07a633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.695+0000 7f4a9b7fe640 1 -- 172.21.15.43:0/162480245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad07a660e0 msgr2=0x55ad07a633c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.695+0000 7f4a9b7fe640 1 --2- 172.21.15.43:0/162480245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad07a660e0 0x55ad07a633c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.695+0000 7f4a9b7fe640 1 -- 172.21.15.43:0/162480245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a5d8d0 msgr2=0x55ad07a62e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.695+0000 7f4a9b7fe640 1 --2- 172.21.15.43:0/162480245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a5d8d0 0x55ad07a62e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.695+0000 7f4a9b7fe640 1 -- 172.21.15.43:0/162480245 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad076d4210 con 0x55ad07a5cb60 2024-04-21T08:42:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.695+0000 7f4a9bfff640 1 --2- 172.21.15.43:0/162480245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a5d8d0 0x55ad07a62e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:42:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.695+0000 7f4a9b7fe640 1 --2- 172.21.15.43:0/162480245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad07a5cb60 0x55ad07a65a90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4a90001eb0 tx=0x7f4a90015040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7755be785f427afc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.695+0000 7f4a7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-21T08:42:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.695+0000 7f4a7b7fe640 1 -- 172.21.15.43:0/162480245 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90019070 con 0x55ad07a5cb60 2024-04-21T08:42:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.695+0000 7f4a7b7fe640 1 -- 172.21.15.43:0/162480245 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f4a90011750 con 0x55ad07a5cb60 2024-04-21T08:42:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.695+0000 7f4aa525b740 1 -- 172.21.15.43:0/162480245 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a900119d0 con 0x55ad07a5cb60 2024-04-21T08:42:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.695+0000 7f4a7b7fe640 1 -- 172.21.15.43:0/162480245 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a90031050 con 0x55ad07a5cb60 2024-04-21T08:42:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.695+0000 7f4aa525b740 10 client.26666 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.695+0000 7f4aa525b740 1 -- 172.21.15.43:0/162480245 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ad077a0620 con 0x55ad07a5cb60 2024-04-21T08:42:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.695+0000 7f4a7b7fe640 1 -- 172.21.15.43:0/162480245 <== mon.2 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a900119d0 con 0x55ad07a5cb60 2024-04-21T08:42:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.696+0000 7f4a7b7fe640 1 -- 172.21.15.43:0/162480245 <== mon.2 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4a900035f0 con 0x55ad07a5cb60 2024-04-21T08:42:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.696+0000 7f4aa525b740 10 client.26666 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.696+0000 7f4aa525b740 10 client.26666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.696+0000 7f4aa525b740 20 client.26666 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.696+0000 7f4aa525b740 1 --2- 172.21.15.43:0/162480245 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ad07a76060 0x55ad07a78450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.696+0000 7f4aa525b740 4 client.26666 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.696+0000 7f4aa525b740 1 -- 172.21.15.43:0/162480245 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad079d5880 con 0x55ad07a76060 2024-04-21T08:42:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.696+0000 7f4a9bfff640 1 --2- 172.21.15.43:0/162480245 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ad07a76060 0x55ad07a78450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.696+0000 7f4a9bfff640 1 --2- 172.21.15.43:0/162480245 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ad07a76060 0x55ad07a78450 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=920f71831f8f3a3b server_cookie=3b779e6ce6a652ce in_seq=0 out_seq=0 2024-04-21T08:42:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.696+0000 7f4a7b7fe640 10 client.26666 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.697+0000 7f4a7b7fe640 1 -- 172.21.15.43:0/162480245 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ad079d5880 con 0x55ad07a76060 2024-04-21T08:42:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.697+0000 7f4a7b7fe640 10 client.26666 handle_command_reply: tid=0 2024-04-21T08:42:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.697+0000 7f4aa525b740 1 client.26666 shutdown 2024-04-21T08:42:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.697+0000 7f4aa525b740 1 -- 172.21.15.43:0/162480245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad07a5cb60 msgr2=0x55ad07a65a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.697+0000 7f4aa525b740 1 --2- 172.21.15.43:0/162480245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad07a5cb60 0x55ad07a65a90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4a90001eb0 tx=0x7f4a90015040 comp rx=0 tx=0).stop 2024-04-21T08:42:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.697+0000 7f4aa525b740 1 -- 172.21.15.43:0/162480245 shutdown_connections 2024-04-21T08:42:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.697+0000 7f4aa525b740 1 --2- 172.21.15.43:0/162480245 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ad07a76060 0x55ad07a78450 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.697+0000 7f4aa525b740 1 --2- 172.21.15.43:0/162480245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad07a660e0 0x55ad07a633c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.697+0000 7f4aa525b740 1 --2- 172.21.15.43:0/162480245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a5d8d0 0x55ad07a62e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.697+0000 7f4a7b7fe640 0 client.26666 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.697+0000 7f4aa525b740 1 --2- 172.21.15.43:0/162480245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad07a5cb60 0x55ad07a65a90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.697+0000 7f4aa525b740 1 -- 172.21.15.43:0/162480245 >> 172.21.15.43:0/162480245 conn(0x55ad07a23c90 msgr2=0x55ad07a595a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.698+0000 7f4aa525b740 1 -- 172.21.15.43:0/162480245 shutdown_connections 2024-04-21T08:42:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.698+0000 7f4aa525b740 1 -- 172.21.15.43:0/162480245 wait complete. 2024-04-21T08:42:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.698+0000 7f4aa525b740 20 client.26666 trim_cache size 0 max 16384 2024-04-21T08:42:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.713+0000 7f4aa525b740 1 Processor -- start 2024-04-21T08:42:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.713+0000 7f4aa525b740 1 -- start start 2024-04-21T08:42:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.713+0000 7f4aa525b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad07a5cb60 0x55ad07a61e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.713+0000 7f4aa525b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad07a5d8d0 0x55ad07a65550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.713+0000 7f4aa525b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a660e0 0x55ad07a65a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.713+0000 7f4aa525b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c1b1580 con 0x55ad07a660e0 2024-04-21T08:42:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.713+0000 7f4aa525b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c1b30c0 con 0x55ad07a5cb60 2024-04-21T08:42:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.713+0000 7f4aa525b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c1b1c50 con 0x55ad07a5d8d0 2024-04-21T08:42:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.713+0000 7f4a9b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a660e0 0x55ad07a65a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.713+0000 7f4a9b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a660e0 0x55ad07a65a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45932/0 (socket says 172.21.15.43:45932) 2024-04-21T08:42:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.713+0000 7f4a9b7fe640 1 -- 172.21.15.43:0/2352624879 learned_addr learned my addr 172.21.15.43:0/2352624879 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.713+0000 7f4a9b7fe640 1 --2- 172.21.15.43:0/2352624879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad07a5cb60 0x55ad07a61e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.713+0000 7f4a9b7fe640 1 -- 172.21.15.43:0/2352624879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad07a5d8d0 msgr2=0x55ad07a65550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.713+0000 7f4a9b7fe640 1 --2- 172.21.15.43:0/2352624879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad07a5d8d0 0x55ad07a65550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.713+0000 7f4a9b7fe640 1 -- 172.21.15.43:0/2352624879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad07a5cb60 msgr2=0x55ad07a61e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.713+0000 7f4a9b7fe640 1 --2- 172.21.15.43:0/2352624879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad07a5cb60 0x55ad07a61e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.713+0000 7f4a9b7fe640 1 -- 172.21.15.43:0/2352624879 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad07a63730 con 0x55ad07a660e0 2024-04-21T08:42:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.713+0000 7f4a9b7fe640 1 --2- 172.21.15.43:0/2352624879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a660e0 0x55ad07a65a90 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x55ad07a57150 tx=0x7f4a90005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5804eef1fa24f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.714+0000 7f4a81ffb640 1 -- 172.21.15.43:0/2352624879 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90019040 con 0x55ad07a660e0 2024-04-21T08:42:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.714+0000 7f4a81ffb640 1 -- 172.21.15.43:0/2352624879 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a9000fe60 con 0x55ad07a660e0 2024-04-21T08:42:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.714+0000 7f4a81ffb640 1 -- 172.21.15.43:0/2352624879 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a900035f0 con 0x55ad07a660e0 2024-04-21T08:42:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.714+0000 7f4aa525b740 1 -- 172.21.15.43:0/2352624879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a660e0 msgr2=0x55ad07a65a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.714+0000 7f4aa525b740 1 --2- 172.21.15.43:0/2352624879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a660e0 0x55ad07a65a90 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x55ad07a57150 tx=0x7f4a90005970 comp rx=0 tx=0).stop 2024-04-21T08:42:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.714+0000 7f4aa525b740 1 -- 172.21.15.43:0/2352624879 shutdown_connections 2024-04-21T08:42:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.714+0000 7f4aa525b740 1 --2- 172.21.15.43:0/2352624879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a660e0 0x55ad07a65a90 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.714+0000 7f4aa525b740 1 --2- 172.21.15.43:0/2352624879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad07a5d8d0 0x55ad07a65550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.714+0000 7f4aa525b740 1 --2- 172.21.15.43:0/2352624879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad07a5cb60 0x55ad07a61e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.714+0000 7f4aa525b740 1 -- 172.21.15.43:0/2352624879 >> 172.21.15.43:0/2352624879 conn(0x55ad07a23c90 msgr2=0x55ad07a616f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.714+0000 7f4aa525b740 1 -- 172.21.15.43:0/2352624879 shutdown_connections 2024-04-21T08:42:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.714+0000 7f4aa525b740 1 -- 172.21.15.43:0/2352624879 wait complete. 2024-04-21T08:42:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.715+0000 7f4aa525b740 1 -- start start 2024-04-21T08:42:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.715+0000 7f4aa525b740 1 Processor -- start 2024-04-21T08:42:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.715+0000 7f4aa525b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad07a5cb60 0x55ad07a7b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.715+0000 7f4aa525b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad07a5d8d0 0x55ad07a76b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.715+0000 7f4aa525b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a660e0 0x55ad07a770c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.715+0000 7f4aa525b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c1b29f0 con 0x55ad07a660e0 2024-04-21T08:42:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.715+0000 7f4aa525b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c1b2320 con 0x55ad07a5d8d0 2024-04-21T08:42:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.715+0000 7f4aa525b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c1af560 con 0x55ad07a5cb60 2024-04-21T08:42:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.715+0000 7f4aa525b740 10 client.0 _ll_register_callbacks cb 0x7f4a90013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.715+0000 7f4a9b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a660e0 0x55ad07a770c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.715+0000 7f4a9b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a660e0 0x55ad07a770c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45936/0 (socket says 172.21.15.43:45936) 2024-04-21T08:42:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.715+0000 7f4a9b7fe640 1 -- 172.21.15.43:0/2458423911 learned_addr learned my addr 172.21.15.43:0/2458423911 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:42:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.715+0000 7f4a9bfff640 1 --2- 172.21.15.43:0/2458423911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad07a5d8d0 0x55ad07a76b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.715+0000 7f4a9b7fe640 1 -- 172.21.15.43:0/2458423911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad07a5cb60 msgr2=0x55ad07a7b810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.715+0000 7f4a9b7fe640 1 --2- 172.21.15.43:0/2458423911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad07a5cb60 0x55ad07a7b810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.715+0000 7f4a9b7fe640 1 -- 172.21.15.43:0/2458423911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad07a5d8d0 msgr2=0x55ad07a76b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.715+0000 7f4a9b7fe640 1 --2- 172.21.15.43:0/2458423911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad07a5d8d0 0x55ad07a76b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.716+0000 7f4a9b7fe640 1 -- 172.21.15.43:0/2458423911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad07a778d0 con 0x55ad07a660e0 2024-04-21T08:42:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.716+0000 7f4a9bfff640 1 --2- 172.21.15.43:0/2458423911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad07a5d8d0 0x55ad07a76b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.716+0000 7f4a9b7fe640 1 --2- 172.21.15.43:0/2458423911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a660e0 0x55ad07a770c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f4a90003b80 tx=0x7f4a9000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9a2984e5d16e4a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.716+0000 7f4a7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:42:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.716+0000 7f4a7b7fe640 1 -- 172.21.15.43:0/2458423911 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90019070 con 0x55ad07a660e0 2024-04-21T08:42:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.716+0000 7f4a7b7fe640 1 -- 172.21.15.43:0/2458423911 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f4a90011730 con 0x55ad07a660e0 2024-04-21T08:42:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.716+0000 7f4aa525b740 1 -- 172.21.15.43:0/2458423911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a9000fe60 con 0x55ad07a660e0 2024-04-21T08:42:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.716+0000 7f4a7b7fe640 1 -- 172.21.15.43:0/2458423911 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a90039040 con 0x55ad07a660e0 2024-04-21T08:42:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.716+0000 7f4aa525b740 10 client.17445 fetch_fsmap learned FSMap version 28 2024-04-21T08:42:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.716+0000 7f4aa525b740 1 -- 172.21.15.43:0/2458423911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ad07adc210 con 0x55ad07a660e0 2024-04-21T08:42:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.716+0000 7f4a7b7fe640 1 -- 172.21.15.43:0/2458423911 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a9000fe60 con 0x55ad07a660e0 2024-04-21T08:42:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.716+0000 7f4a7b7fe640 1 -- 172.21.15.43:0/2458423911 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4a90005020 con 0x55ad07a660e0 2024-04-21T08:42:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.716+0000 7f4aa525b740 10 client.17445 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:42:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.716+0000 7f4aa525b740 10 client.17445 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.716+0000 7f4aa525b740 20 client.17445 populate_metadata read hostname 'smithi043' 2024-04-21T08:42:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.716+0000 7f4aa525b740 1 --2- 172.21.15.43:0/2458423911 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ad07adc820 0x55ad07adec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.717+0000 7f4aa525b740 4 client.17445 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:42:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.717+0000 7f4aa525b740 1 -- 172.21.15.43:0/2458423911 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ad079613d0 con 0x55ad07adc820 2024-04-21T08:42:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.717+0000 7f4a9bfff640 1 --2- 172.21.15.43:0/2458423911 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ad07adc820 0x55ad07adec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.717+0000 7f4a9bfff640 1 --2- 172.21.15.43:0/2458423911 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ad07adc820 0x55ad07adec40 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4e045738c60f49c server_cookie=f4683f5b7e91efc9 in_seq=0 out_seq=0 2024-04-21T08:42:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.717+0000 7f4a7b7fe640 10 client.17445 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.733+0000 7f4a7b7fe640 1 -- 172.21.15.43:0/2458423911 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ad079613d0 con 0x55ad07adc820 2024-04-21T08:42:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.733+0000 7f4a7b7fe640 10 client.17445 handle_command_reply: tid=0 2024-04-21T08:42:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.733+0000 7f4aa525b740 1 client.17445 shutdown 2024-04-21T08:42:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.733+0000 7f4aa525b740 1 -- 172.21.15.43:0/2458423911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a660e0 msgr2=0x55ad07a770c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.733+0000 7f4aa525b740 1 --2- 172.21.15.43:0/2458423911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a660e0 0x55ad07a770c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f4a90003b80 tx=0x7f4a9000f670 comp rx=0 tx=0).stop 2024-04-21T08:42:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.733+0000 7f4aa525b740 1 -- 172.21.15.43:0/2458423911 shutdown_connections 2024-04-21T08:42:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.733+0000 7f4aa525b740 1 --2- 172.21.15.43:0/2458423911 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ad07adc820 0x55ad07adec40 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.733+0000 7f4aa525b740 1 --2- 172.21.15.43:0/2458423911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad07a660e0 0x55ad07a770c0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.733+0000 7f4aa525b740 1 --2- 172.21.15.43:0/2458423911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad07a5d8d0 0x55ad07a76b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.733+0000 7f4aa525b740 1 --2- 172.21.15.43:0/2458423911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ad07a5cb60 0x55ad07a7b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.733+0000 7f4a7b7fe640 0 client.17445 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:42:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.733+0000 7f4aa525b740 1 -- 172.21.15.43:0/2458423911 >> 172.21.15.43:0/2458423911 conn(0x55ad07a23c90 msgr2=0x55ad07a71250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.733+0000 7f4aa525b740 1 -- 172.21.15.43:0/2458423911 shutdown_connections 2024-04-21T08:42:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.733+0000 7f4aa525b740 1 -- 172.21.15.43:0/2458423911 wait complete. 2024-04-21T08:42:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.733+0000 7f4aa525b740 20 client.17445 trim_cache size 0 max 16384 2024-04-21T08:42:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:42:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:42:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:42:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:42:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.734+0000 7f4aa2670640 1 -- 172.21.15.43:0/2395447959 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4a840a3e40 msgr2=0x7f4a840a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.734+0000 7f4aa2670640 1 --2- 172.21.15.43:0/2395447959 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4a840a3e40 0x7f4a840a6300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400eb10 tx=0x7f4a94005900 comp rx=0 tx=0).stop 2024-04-21T08:42:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.734+0000 7f4aa2670640 1 -- 172.21.15.43:0/2395447959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a9c10c5d0 msgr2=0x7f4a9c19a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.734+0000 7f4aa2670640 1 --2- 172.21.15.43:0/2395447959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a9c10c5d0 0x7f4a9c19a750 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400eb40 tx=0x7f4a9400c2d0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.734+0000 7f4aa2670640 1 -- 172.21.15.43:0/2395447959 shutdown_connections 2024-04-21T08:42:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.734+0000 7f4aa2670640 1 --2- 172.21.15.43:0/2395447959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a9c10c5d0 0x7f4a9c19a750 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.734+0000 7f4aa2670640 1 --2- 172.21.15.43:0/2395447959 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4a840a3e40 0x7f4a840a6300 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.734+0000 7f4aa2670640 1 --2- 172.21.15.43:0/2395447959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a9c103b20 0x7f4a9c1a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.734+0000 7f4aa2670640 1 --2- 172.21.15.43:0/2395447959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a9c103200 0x7f4a9c1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.734+0000 7f4aa2670640 1 -- 172.21.15.43:0/2395447959 >> 172.21.15.43:0/2395447959 conn(0x7f4a9c0771d0 msgr2=0x7f4a9c0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.734+0000 7f4aa2670640 1 -- 172.21.15.43:0/2395447959 shutdown_connections 2024-04-21T08:42:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:42:58.734+0000 7f4aa2670640 1 -- 172.21.15.43:0/2395447959 wait complete. 2024-04-21T08:43:00.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:00 smithi175 ceph-mon[31920]: pgmap v125: 129 pgs: 129 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-21T08:43:00.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:00 smithi112 ceph-mon[31085]: pgmap v125: 129 pgs: 129 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-21T08:43:00.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:00 smithi043 ceph-mon[27978]: pgmap v125: 129 pgs: 129 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-21T08:43:02.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:02 smithi175 ceph-mon[31920]: pgmap v126: 129 pgs: 129 active+clean; 4.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 61 KiB/s wr, 38 op/s 2024-04-21T08:43:02.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:02 smithi112 ceph-mon[31085]: pgmap v126: 129 pgs: 129 active+clean; 4.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 61 KiB/s wr, 38 op/s 2024-04-21T08:43:02.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:02 smithi043 ceph-mon[27978]: pgmap v126: 129 pgs: 129 active+clean; 4.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 61 KiB/s wr, 38 op/s 2024-04-21T08:43:03.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:03 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:43:03.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:03 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:43:03.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:03 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:43:04.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:04 smithi175 ceph-mon[31920]: pgmap v127: 129 pgs: 129 active+clean; 4.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 50 KiB/s wr, 30 op/s 2024-04-21T08:43:04.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:04 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:04.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:04 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:43:04.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:04 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:43:04.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:04 smithi112 ceph-mon[31085]: pgmap v127: 129 pgs: 129 active+clean; 4.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 50 KiB/s wr, 30 op/s 2024-04-21T08:43:04.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:04 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:04.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:04 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:43:04.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:04 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:43:04.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:04 smithi043 ceph-mon[27978]: pgmap v127: 129 pgs: 129 active+clean; 4.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 50 KiB/s wr, 30 op/s 2024-04-21T08:43:04.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:04 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:04.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:04 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:43:04.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:04 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:43:06.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:06 smithi175 ceph-mon[31920]: pgmap v128: 129 pgs: 129 active+clean; 4.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 50 KiB/s wr, 30 op/s 2024-04-21T08:43:06.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:06 smithi112 ceph-mon[31085]: pgmap v128: 129 pgs: 129 active+clean; 4.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 50 KiB/s wr, 30 op/s 2024-04-21T08:43:06.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:06 smithi043 ceph-mon[27978]: pgmap v128: 129 pgs: 129 active+clean; 4.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 50 KiB/s wr, 30 op/s 2024-04-21T08:43:08.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:08 smithi175 ceph-mon[31920]: pgmap v129: 129 pgs: 129 active+clean; 5.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 64 KiB/s wr, 37 op/s 2024-04-21T08:43:08.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:08 smithi112 ceph-mon[31085]: pgmap v129: 129 pgs: 129 active+clean; 5.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 64 KiB/s wr, 37 op/s 2024-04-21T08:43:08.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:08 smithi043 ceph-mon[27978]: pgmap v129: 129 pgs: 129 active+clean; 5.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 64 KiB/s wr, 37 op/s 2024-04-21T08:43:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:10 smithi112 ceph-mon[31085]: pgmap v130: 129 pgs: 129 active+clean; 5.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-04-21T08:43:10.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:10 smithi043 ceph-mon[27978]: pgmap v130: 129 pgs: 129 active+clean; 5.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-04-21T08:43:10.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:10 smithi175 ceph-mon[31920]: pgmap v130: 129 pgs: 129 active+clean; 5.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-04-21T08:43:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.834+0000 7f10ff0a5640 1 -- 172.21.15.43:0/158207216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f81021c0 msgr2=0x7f10f81025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.834+0000 7f10ff0a5640 1 --2- 172.21.15.43:0/158207216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f81021c0 0x7f10f81025a0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f10ec009900 tx=0x7f10ec02bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.834+0000 7f10ff0a5640 1 -- 172.21.15.43:0/158207216 shutdown_connections 2024-04-21T08:43:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.834+0000 7f10ff0a5640 1 --2- 172.21.15.43:0/158207216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f8107290 0x7f10f810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.834+0000 7f10ff0a5640 1 --2- 172.21.15.43:0/158207216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10f8102b70 0x7f10f8106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.834+0000 7f10ff0a5640 1 --2- 172.21.15.43:0/158207216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f81021c0 0x7f10f81025a0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.834+0000 7f10ff0a5640 1 -- 172.21.15.43:0/158207216 >> 172.21.15.43:0/158207216 conn(0x7f10f80fda20 msgr2=0x7f10f80ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.835+0000 7f10ff0a5640 1 -- 172.21.15.43:0/158207216 shutdown_connections 2024-04-21T08:43:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.835+0000 7f10ff0a5640 1 -- 172.21.15.43:0/158207216 wait complete. 2024-04-21T08:43:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.835+0000 7f10ff0a5640 1 Processor -- start 2024-04-21T08:43:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.835+0000 7f10ff0a5640 1 -- start start 2024-04-21T08:43:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.835+0000 7f10ff0a5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f81021c0 0x7f10f819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.835+0000 7f10ff0a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10f8107290 0x7f10f819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.835+0000 7f10ff0a5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f81985e0 0x7f10f8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.835+0000 7f10ff0a5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f10f8112200 con 0x7f10f81985e0 2024-04-21T08:43:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.835+0000 7f10ff0a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f10f8112080 con 0x7f10f8107290 2024-04-21T08:43:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.835+0000 7f10ff0a5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f10f8112380 con 0x7f10f81021c0 2024-04-21T08:43:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.836+0000 7f10fd642640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f81985e0 0x7f10f8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.836+0000 7f10fd642640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f81985e0 0x7f10f8198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46114/0 (socket says 172.21.15.43:46114) 2024-04-21T08:43:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.836+0000 7f10fd642640 1 -- 172.21.15.43:0/1853720759 learned_addr learned my addr 172.21.15.43:0/1853720759 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.836+0000 7f10f7fff640 1 --2- 172.21.15.43:0/1853720759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10f8107290 0x7f10f819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.836+0000 7f10fce41640 1 --2- 172.21.15.43:0/1853720759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f81021c0 0x7f10f819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.836+0000 7f10fd642640 1 -- 172.21.15.43:0/1853720759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f81021c0 msgr2=0x7f10f819e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.836+0000 7f10fd642640 1 --2- 172.21.15.43:0/1853720759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f81021c0 0x7f10f819e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.836+0000 7f10fd642640 1 -- 172.21.15.43:0/1853720759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10f8107290 msgr2=0x7f10f819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.836+0000 7f10fd642640 1 --2- 172.21.15.43:0/1853720759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10f8107290 0x7f10f819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.836+0000 7f10fd642640 1 -- 172.21.15.43:0/1853720759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f8199100 con 0x7f10f81985e0 2024-04-21T08:43:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.836+0000 7f10fd642640 1 --2- 172.21.15.43:0/1853720759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f81985e0 0x7f10f8198a90 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f10f000cad0 tx=0x7f10f000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.836+0000 7f10f5ffb640 1 -- 172.21.15.43:0/1853720759 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f000ae20 con 0x7f10f81985e0 2024-04-21T08:43:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.836+0000 7f10f5ffb640 1 -- 172.21.15.43:0/1853720759 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10f0005020 con 0x7f10f81985e0 2024-04-21T08:43:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.836+0000 7f10ff0a5640 1 -- 172.21.15.43:0/1853720759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f81993f0 con 0x7f10f81985e0 2024-04-21T08:43:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.836+0000 7f10f5ffb640 1 -- 172.21.15.43:0/1853720759 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f000f6b0 con 0x7f10f81985e0 2024-04-21T08:43:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.837+0000 7f10ff0a5640 1 -- 172.21.15.43:0/1853720759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f81a55f0 con 0x7f10f81985e0 2024-04-21T08:43:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.840+0000 7f10f5ffb640 1 -- 172.21.15.43:0/1853720759 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f10f00051d0 con 0x7f10f81985e0 2024-04-21T08:43:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.840+0000 7f1101d75740 1 Processor -- start 2024-04-21T08:43:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.840+0000 7f1101d75740 1 -- start start 2024-04-21T08:43:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.840+0000 7f1101d75740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555822cd88d0 0x555822cd7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.840+0000 7f1101d75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822cd7b60 0x555822cd7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.840+0000 7f1101d75740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555822ce10e0 0x555822ce34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.840+0000 7f1101d75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555822c3c8d0 con 0x555822cd7b60 2024-04-21T08:43:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.840+0000 7f1101d75740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555822bd9e80 con 0x555822cd88d0 2024-04-21T08:43:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.840+0000 7f1101d75740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555822c38890 con 0x555822ce10e0 2024-04-21T08:43:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.840+0000 7f10f5ffb640 1 --2- 172.21.15.43:0/1853720759 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f10e80a3dc0 0x7f10e80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.840+0000 7f10fce41640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822cd7b60 0x555822cd7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.840+0000 7f10fce41640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822cd7b60 0x555822cd7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46130/0 (socket says 172.21.15.43:46130) 2024-04-21T08:43:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.840+0000 7f10fce41640 1 -- 172.21.15.43:0/517582459 learned_addr learned my addr 172.21.15.43:0/517582459 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.840+0000 7f10f7fff640 1 --2- 172.21.15.43:0/517582459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555822cd88d0 0x555822cd7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.840+0000 7f10f5ffb640 1 -- 172.21.15.43:0/1853720759 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f10f00d0c10 con 0x7f10f81985e0 2024-04-21T08:43:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.840+0000 7f10fd642640 1 --2- 172.21.15.43:0/1853720759 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f10e80a3dc0 0x7f10e80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.841+0000 7f10f7fff640 1 --2- 172.21.15.43:0/517582459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555822ce10e0 0x555822ce34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.841+0000 7f10fce41640 1 -- 172.21.15.43:0/517582459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555822ce10e0 msgr2=0x555822ce34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.841+0000 7f10fce41640 1 --2- 172.21.15.43:0/517582459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555822ce10e0 0x555822ce34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.841+0000 7f10fce41640 1 -- 172.21.15.43:0/517582459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555822cd88d0 msgr2=0x555822cd7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.841+0000 7f10fce41640 1 --2- 172.21.15.43:0/517582459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555822cd88d0 0x555822cd7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.841+0000 7f10fce41640 1 -- 172.21.15.43:0/517582459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555822bfb320 con 0x555822cd7b60 2024-04-21T08:43:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.841+0000 7f10f7fff640 1 --2- 172.21.15.43:0/517582459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555822cd88d0 0x555822cd7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:43:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.841+0000 7f10fd642640 1 --2- 172.21.15.43:0/1853720759 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f10e80a3dc0 0x7f10e80a6280 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f10f009c480 tx=0x7f10f0087040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.841+0000 7f10fce41640 1 --2- 172.21.15.43:0/517582459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822cd7b60 0x555822cd7fe0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f10ec009900 tx=0x7f10ec02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da567cbf5ee2780 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.841+0000 7f10e2ffd640 1 -- 172.21.15.43:0/517582459 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec044020 con 0x555822cd7b60 2024-04-21T08:43:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.841+0000 7f10e2ffd640 1 -- 172.21.15.43:0/517582459 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10ec02cc90 con 0x555822cd7b60 2024-04-21T08:43:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.841+0000 7f10e2ffd640 1 -- 172.21.15.43:0/517582459 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec0046c0 con 0x555822cd7b60 2024-04-21T08:43:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.842+0000 7f1101d75740 1 -- 172.21.15.43:0/517582459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822cd7b60 msgr2=0x555822cd7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.842+0000 7f1101d75740 1 --2- 172.21.15.43:0/517582459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822cd7b60 0x555822cd7fe0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f10ec009900 tx=0x7f10ec02c650 comp rx=0 tx=0).stop 2024-04-21T08:43:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.842+0000 7f1101d75740 1 -- 172.21.15.43:0/517582459 shutdown_connections 2024-04-21T08:43:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.842+0000 7f1101d75740 1 --2- 172.21.15.43:0/517582459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555822ce10e0 0x555822ce34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.842+0000 7f1101d75740 1 --2- 172.21.15.43:0/517582459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822cd7b60 0x555822cd7fe0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.842+0000 7f1101d75740 1 --2- 172.21.15.43:0/517582459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555822cd88d0 0x555822cd7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.842+0000 7f1101d75740 1 -- 172.21.15.43:0/517582459 >> 172.21.15.43:0/517582459 conn(0x555822c9ec90 msgr2=0x555822cd2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.842+0000 7f1101d75740 1 -- 172.21.15.43:0/517582459 shutdown_connections 2024-04-21T08:43:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.842+0000 7f1101d75740 1 -- 172.21.15.43:0/517582459 wait complete. 2024-04-21T08:43:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.842+0000 7f1101d75740 1 -- start start 2024-04-21T08:43:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.842+0000 7f1101d75740 1 Processor -- start 2024-04-21T08:43:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.843+0000 7f1101d75740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555822cd7b60 0x555822ce0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.843+0000 7f1101d75740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555822cd88d0 0x555822cdde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.843+0000 7f1101d75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822ce10e0 0x555822cde460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.843+0000 7f1101d75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555822ce3db0 con 0x555822ce10e0 2024-04-21T08:43:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.843+0000 7f1101d75740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555822bc4eb0 con 0x555822cd88d0 2024-04-21T08:43:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.843+0000 7f1101d75740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555822ce3f30 con 0x555822cd7b60 2024-04-21T08:43:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.843+0000 7f1101d75740 10 client.0 _ll_register_callbacks cb 0x555822bf1b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.843+0000 7f10f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822ce10e0 0x555822cde460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.843+0000 7f10f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822ce10e0 0x555822cde460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46146/0 (socket says 172.21.15.43:46146) 2024-04-21T08:43:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.843+0000 7f10f7fff640 1 -- 172.21.15.43:0/2494432496 learned_addr learned my addr 172.21.15.43:0/2494432496 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.843+0000 7f10fce41640 1 --2- 172.21.15.43:0/2494432496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555822cd88d0 0x555822cdde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.843+0000 7f10f7fff640 1 --2- 172.21.15.43:0/2494432496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555822cd7b60 0x555822ce0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.843+0000 7f10f7fff640 1 -- 172.21.15.43:0/2494432496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555822cd7b60 msgr2=0x555822ce0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.843+0000 7f10f7fff640 1 --2- 172.21.15.43:0/2494432496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555822cd7b60 0x555822ce0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.843+0000 7f10f7fff640 1 -- 172.21.15.43:0/2494432496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555822cd88d0 msgr2=0x555822cdde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.843+0000 7f10f7fff640 1 --2- 172.21.15.43:0/2494432496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555822cd88d0 0x555822cdde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.843+0000 7f10f7fff640 1 -- 172.21.15.43:0/2494432496 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55582294f210 con 0x555822ce10e0 2024-04-21T08:43:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.843+0000 7f10fce41640 1 --2- 172.21.15.43:0/2494432496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555822cd88d0 0x555822cdde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:43:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.844+0000 7f10f7fff640 1 --2- 172.21.15.43:0/2494432496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822ce10e0 0x555822cde460 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f10e4005d00 tx=0x7f10e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ec5eadb4c8df10 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.844+0000 7f10d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.844+0000 7f10d7fff640 1 -- 172.21.15.43:0/2494432496 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e4012030 con 0x555822ce10e0 2024-04-21T08:43:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.844+0000 7f10d7fff640 1 -- 172.21.15.43:0/2494432496 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f10e40121d0 con 0x555822ce10e0 2024-04-21T08:43:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.844+0000 7f1101d75740 1 -- 172.21.15.43:0/2494432496 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10ec02cc90 con 0x555822ce10e0 2024-04-21T08:43:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.844+0000 7f10d7fff640 1 -- 172.21.15.43:0/2494432496 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f10e4010b30 con 0x555822ce10e0 2024-04-21T08:43:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.844+0000 7f1101d75740 10 client.17457 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.844+0000 7f1101d75740 1 -- 172.21.15.43:0/2494432496 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555822a1b620 con 0x555822ce10e0 2024-04-21T08:43:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.844+0000 7f10d7fff640 1 -- 172.21.15.43:0/2494432496 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10ec02cc90 con 0x555822ce10e0 2024-04-21T08:43:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.844+0000 7f10d7fff640 1 -- 172.21.15.43:0/2494432496 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f10e4003710 con 0x555822ce10e0 2024-04-21T08:43:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.844+0000 7f1101d75740 10 client.17457 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.844+0000 7f1101d75740 10 client.17457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.844+0000 7f1101d75740 20 client.17457 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.844+0000 7f1101d75740 1 --2- 172.21.15.43:0/2494432496 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555822cf1160 0x555822cf3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.844+0000 7f1101d75740 4 client.17457 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.844+0000 7f1101d75740 1 -- 172.21.15.43:0/2494432496 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555822c50880 con 0x555822cf1160 2024-04-21T08:43:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.845+0000 7f10fce41640 1 --2- 172.21.15.43:0/2494432496 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555822cf1160 0x555822cf3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.845+0000 7f10fce41640 1 --2- 172.21.15.43:0/2494432496 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555822cf1160 0x555822cf3550 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cfda2daaba990e2 server_cookie=9df6cbd373a69336 in_seq=0 out_seq=0 2024-04-21T08:43:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.845+0000 7f10d7fff640 10 client.17457 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.846+0000 7f10d7fff640 1 -- 172.21.15.43:0/2494432496 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555822c50880 con 0x555822cf1160 2024-04-21T08:43:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.846+0000 7f10d7fff640 10 client.17457 handle_command_reply: tid=0 2024-04-21T08:43:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.846+0000 7f1101d75740 1 client.17457 shutdown 2024-04-21T08:43:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.846+0000 7f1101d75740 1 -- 172.21.15.43:0/2494432496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822ce10e0 msgr2=0x555822cde460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.846+0000 7f1101d75740 1 --2- 172.21.15.43:0/2494432496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822ce10e0 0x555822cde460 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f10e4005d00 tx=0x7f10e4004730 comp rx=0 tx=0).stop 2024-04-21T08:43:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.846+0000 7f1101d75740 1 -- 172.21.15.43:0/2494432496 shutdown_connections 2024-04-21T08:43:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.846+0000 7f1101d75740 1 --2- 172.21.15.43:0/2494432496 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555822cf1160 0x555822cf3550 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.846+0000 7f1101d75740 1 --2- 172.21.15.43:0/2494432496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822ce10e0 0x555822cde460 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.846+0000 7f1101d75740 1 --2- 172.21.15.43:0/2494432496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555822cd88d0 0x555822cdde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.846+0000 7f10d7fff640 0 client.17457 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.846+0000 7f1101d75740 1 --2- 172.21.15.43:0/2494432496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555822cd7b60 0x555822ce0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.846+0000 7f1101d75740 1 -- 172.21.15.43:0/2494432496 >> 172.21.15.43:0/2494432496 conn(0x555822c9ec90 msgr2=0x555822cd45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.846+0000 7f1101d75740 1 -- 172.21.15.43:0/2494432496 shutdown_connections 2024-04-21T08:43:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.846+0000 7f1101d75740 1 -- 172.21.15.43:0/2494432496 wait complete. 2024-04-21T08:43:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.846+0000 7f1101d75740 20 client.17457 trim_cache size 0 max 16384 2024-04-21T08:43:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.862+0000 7f1101d75740 1 Processor -- start 2024-04-21T08:43:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.862+0000 7f1101d75740 1 -- start start 2024-04-21T08:43:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.862+0000 7f1101d75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822cd7b60 0x555822cdd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.862+0000 7f1101d75740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555822cd88d0 0x555822cd43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.862+0000 7f1101d75740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555822ce10e0 0x555822ce0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.862+0000 7f1101d75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f10f81af430 con 0x555822cd7b60 2024-04-21T08:43:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.862+0000 7f1101d75740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f10f81b0f70 con 0x555822ce10e0 2024-04-21T08:43:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.862+0000 7f1101d75740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f10f81afb00 con 0x555822cd88d0 2024-04-21T08:43:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.862+0000 7f10f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822cd7b60 0x555822cdd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.862+0000 7f10f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822cd7b60 0x555822cdd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46152/0 (socket says 172.21.15.43:46152) 2024-04-21T08:43:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.862+0000 7f10f7fff640 1 -- 172.21.15.43:0/3793695375 learned_addr learned my addr 172.21.15.43:0/3793695375 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.862+0000 7f10fce41640 1 --2- 172.21.15.43:0/3793695375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555822cd88d0 0x555822cd43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.862+0000 7f10f7fff640 1 -- 172.21.15.43:0/3793695375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555822cd88d0 msgr2=0x555822cd43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.862+0000 7f10f7fff640 1 --2- 172.21.15.43:0/3793695375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555822cd88d0 0x555822cd43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.862+0000 7f10f7fff640 1 -- 172.21.15.43:0/3793695375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555822ce10e0 msgr2=0x555822ce0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.862+0000 7f10f7fff640 1 --2- 172.21.15.43:0/3793695375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555822ce10e0 0x555822ce0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.862+0000 7f10f7fff640 1 -- 172.21.15.43:0/3793695375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555822ce3940 con 0x555822cd7b60 2024-04-21T08:43:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.862+0000 7f10f7fff640 1 --2- 172.21.15.43:0/3793695375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822cd7b60 0x555822cdd730 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f10e400b470 tx=0x7f10e4012690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b802c6c19c85425f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.863+0000 7f10e27fc640 1 -- 172.21.15.43:0/3793695375 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e4005850 con 0x555822cd7b60 2024-04-21T08:43:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.863+0000 7f10e27fc640 1 -- 172.21.15.43:0/3793695375 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10e400b910 con 0x555822cd7b60 2024-04-21T08:43:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.863+0000 7f10e27fc640 1 -- 172.21.15.43:0/3793695375 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e4004480 con 0x555822cd7b60 2024-04-21T08:43:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.863+0000 7f1101d75740 1 -- 172.21.15.43:0/3793695375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822cd7b60 msgr2=0x555822cdd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.863+0000 7f1101d75740 1 --2- 172.21.15.43:0/3793695375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822cd7b60 0x555822cdd730 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f10e400b470 tx=0x7f10e4012690 comp rx=0 tx=0).stop 2024-04-21T08:43:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.863+0000 7f1101d75740 1 -- 172.21.15.43:0/3793695375 shutdown_connections 2024-04-21T08:43:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.863+0000 7f1101d75740 1 --2- 172.21.15.43:0/3793695375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555822ce10e0 0x555822ce0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.863+0000 7f1101d75740 1 --2- 172.21.15.43:0/3793695375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555822cd88d0 0x555822cd43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.863+0000 7f1101d75740 1 --2- 172.21.15.43:0/3793695375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822cd7b60 0x555822cdd730 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.863+0000 7f1101d75740 1 -- 172.21.15.43:0/3793695375 >> 172.21.15.43:0/3793695375 conn(0x555822c9ec90 msgr2=0x555822cd3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.863+0000 7f1101d75740 1 -- 172.21.15.43:0/3793695375 shutdown_connections 2024-04-21T08:43:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.863+0000 7f1101d75740 1 -- 172.21.15.43:0/3793695375 wait complete. 2024-04-21T08:43:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.864+0000 7f1101d75740 1 -- start start 2024-04-21T08:43:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.864+0000 7f1101d75740 1 Processor -- start 2024-04-21T08:43:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.864+0000 7f1101d75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822cd7b60 0x555822cdf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.864+0000 7f1101d75740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555822cd88d0 0x555822cdf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.864+0000 7f1101d75740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555822ce10e0 0x555822cdfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.864+0000 7f1101d75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f10f81b08a0 con 0x555822cd7b60 2024-04-21T08:43:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.864+0000 7f1101d75740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f10f81b01d0 con 0x555822ce10e0 2024-04-21T08:43:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.864+0000 7f1101d75740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f10f81ad410 con 0x555822cd88d0 2024-04-21T08:43:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.864+0000 7f1101d75740 10 client.0 _ll_register_callbacks cb 0x7f10e400be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.864+0000 7f10f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822cd7b60 0x555822cdf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.864+0000 7f10f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822cd7b60 0x555822cdf240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46168/0 (socket says 172.21.15.43:46168) 2024-04-21T08:43:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.864+0000 7f10f7fff640 1 -- 172.21.15.43:0/100925832 learned_addr learned my addr 172.21.15.43:0/100925832 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.864+0000 7f10fce41640 1 --2- 172.21.15.43:0/100925832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555822cd88d0 0x555822cdf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.865+0000 7f10f7fff640 1 -- 172.21.15.43:0/100925832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555822cd88d0 msgr2=0x555822cdf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.865+0000 7f10f7fff640 1 --2- 172.21.15.43:0/100925832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555822cd88d0 0x555822cdf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.865+0000 7f10f7fff640 1 -- 172.21.15.43:0/100925832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555822ce10e0 msgr2=0x555822cdfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.865+0000 7f10f7fff640 1 --2- 172.21.15.43:0/100925832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555822ce10e0 0x555822cdfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.865+0000 7f10f7fff640 1 -- 172.21.15.43:0/100925832 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555822ce8470 con 0x555822cd7b60 2024-04-21T08:43:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.865+0000 7f10fce41640 1 --2- 172.21.15.43:0/100925832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555822cd88d0 0x555822cdf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.865+0000 7f10f7fff640 1 --2- 172.21.15.43:0/100925832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822cd7b60 0x555822cdf240 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f10e400b6c0 tx=0x7f10e4003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5054ca551c0d5491 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.865+0000 7f10d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.865+0000 7f10d7fff640 1 -- 172.21.15.43:0/100925832 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e40038c0 con 0x555822cd7b60 2024-04-21T08:43:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.865+0000 7f1101d75740 1 -- 172.21.15.43:0/100925832 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10e400b910 con 0x555822cd7b60 2024-04-21T08:43:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.865+0000 7f10d7fff640 1 -- 172.21.15.43:0/100925832 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f10e4004000 con 0x555822cd7b60 2024-04-21T08:43:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.865+0000 7f10d7fff640 1 -- 172.21.15.43:0/100925832 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f10e4004200 con 0x555822cd7b60 2024-04-21T08:43:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.865+0000 7f1101d75740 10 client.17463 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.865+0000 7f1101d75740 1 -- 172.21.15.43:0/100925832 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555822ce9780 con 0x555822cd7b60 2024-04-21T08:43:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.865+0000 7f10d7fff640 1 -- 172.21.15.43:0/100925832 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10e400b910 con 0x555822cd7b60 2024-04-21T08:43:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.865+0000 7f10d7fff640 1 -- 172.21.15.43:0/100925832 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f10e402c070 con 0x555822cd7b60 2024-04-21T08:43:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.865+0000 7f1101d75740 10 client.17463 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.865+0000 7f1101d75740 10 client.17463 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.865+0000 7f1101d75740 20 client.17463 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.865+0000 7f1101d75740 1 --2- 172.21.15.43:0/100925832 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555822d57790 0x555822d59b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.865+0000 7f1101d75740 4 client.17463 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:43:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.865+0000 7f1101d75740 1 -- 172.21.15.43:0/100925832 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555822bdc3d0 con 0x555822d57790 2024-04-21T08:43:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.866+0000 7f10fce41640 1 --2- 172.21.15.43:0/100925832 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555822d57790 0x555822d59b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.866+0000 7f10d7fff640 10 client.17463 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.866+0000 7f10fce41640 1 --2- 172.21.15.43:0/100925832 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555822d57790 0x555822d59b80 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90198d5bbabb0e4d server_cookie=53402479e338628 in_seq=0 out_seq=0 2024-04-21T08:43:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.882+0000 7f10d7fff640 1 -- 172.21.15.43:0/100925832 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555822bdc3d0 con 0x555822d57790 2024-04-21T08:43:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.882+0000 7f10d7fff640 10 client.17463 handle_command_reply: tid=0 2024-04-21T08:43:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.882+0000 7f1101d75740 1 client.17463 shutdown 2024-04-21T08:43:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.882+0000 7f1101d75740 1 -- 172.21.15.43:0/100925832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822cd7b60 msgr2=0x555822cdf240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.882+0000 7f1101d75740 1 --2- 172.21.15.43:0/100925832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822cd7b60 0x555822cdf240 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f10e400b6c0 tx=0x7f10e4003ee0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.882+0000 7f1101d75740 1 -- 172.21.15.43:0/100925832 shutdown_connections 2024-04-21T08:43:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.882+0000 7f1101d75740 1 --2- 172.21.15.43:0/100925832 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555822d57790 0x555822d59b80 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.882+0000 7f1101d75740 1 --2- 172.21.15.43:0/100925832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555822ce10e0 0x555822cdfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.882+0000 7f10d7fff640 0 client.17463 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.882+0000 7f1101d75740 1 --2- 172.21.15.43:0/100925832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555822cd88d0 0x555822cdf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.882+0000 7f1101d75740 1 --2- 172.21.15.43:0/100925832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555822cd7b60 0x555822cdf240 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.882+0000 7f1101d75740 1 -- 172.21.15.43:0/100925832 >> 172.21.15.43:0/100925832 conn(0x555822c9ec90 msgr2=0x555822cd18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.883+0000 7f1101d75740 1 -- 172.21.15.43:0/100925832 shutdown_connections 2024-04-21T08:43:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.883+0000 7f1101d75740 1 -- 172.21.15.43:0/100925832 wait complete. 2024-04-21T08:43:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.883+0000 7f1101d75740 20 client.17463 trim_cache size 0 max 16384 2024-04-21T08:43:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:43:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:43:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:43:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:43:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.883+0000 7f10ff0a5640 1 -- 172.21.15.43:0/1853720759 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f10e80a3dc0 msgr2=0x7f10e80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.883+0000 7f10ff0a5640 1 --2- 172.21.15.43:0/1853720759 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f10e80a3dc0 0x7f10e80a6280 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f10f009c480 tx=0x7f10f0087040 comp rx=0 tx=0).stop 2024-04-21T08:43:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.883+0000 7f10ff0a5640 1 -- 172.21.15.43:0/1853720759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f81985e0 msgr2=0x7f10f8198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.883+0000 7f10ff0a5640 1 --2- 172.21.15.43:0/1853720759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f81985e0 0x7f10f8198a90 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f10f000cad0 tx=0x7f10f000cf90 comp rx=0 tx=0).stop 2024-04-21T08:43:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.883+0000 7f10ff0a5640 1 -- 172.21.15.43:0/1853720759 shutdown_connections 2024-04-21T08:43:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.883+0000 7f10ff0a5640 1 --2- 172.21.15.43:0/1853720759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f81985e0 0x7f10f8198a90 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.883+0000 7f10ff0a5640 1 --2- 172.21.15.43:0/1853720759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10f8107290 0x7f10f819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.883+0000 7f10ff0a5640 1 --2- 172.21.15.43:0/1853720759 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f10e80a3dc0 0x7f10e80a6280 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.883+0000 7f10ff0a5640 1 --2- 172.21.15.43:0/1853720759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f81021c0 0x7f10f819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.883+0000 7f10ff0a5640 1 -- 172.21.15.43:0/1853720759 >> 172.21.15.43:0/1853720759 conn(0x7f10f80fda20 msgr2=0x7f10f80ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.883+0000 7f10ff0a5640 1 -- 172.21.15.43:0/1853720759 shutdown_connections 2024-04-21T08:43:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:11.883+0000 7f10ff0a5640 1 -- 172.21.15.43:0/1853720759 wait complete. 2024-04-21T08:43:12.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:12 smithi112 ceph-mon[31085]: pgmap v131: 129 pgs: 129 active+clean; 5.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 65 KiB/s wr, 37 op/s 2024-04-21T08:43:12.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:12 smithi043 ceph-mon[27978]: pgmap v131: 129 pgs: 129 active+clean; 5.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 65 KiB/s wr, 37 op/s 2024-04-21T08:43:12.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:12 smithi175 ceph-mon[31920]: pgmap v131: 129 pgs: 129 active+clean; 5.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 65 KiB/s wr, 37 op/s 2024-04-21T08:43:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:14 smithi112 ceph-mon[31085]: pgmap v132: 129 pgs: 129 active+clean; 5.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-21T08:43:14.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:14 smithi043 ceph-mon[27978]: pgmap v132: 129 pgs: 129 active+clean; 5.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-21T08:43:14.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:14 smithi175 ceph-mon[31920]: pgmap v132: 129 pgs: 129 active+clean; 5.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-21T08:43:16.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:16 smithi112 ceph-mon[31085]: pgmap v133: 129 pgs: 129 active+clean; 5.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-21T08:43:16.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:16 smithi043 ceph-mon[27978]: pgmap v133: 129 pgs: 129 active+clean; 5.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-21T08:43:16.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:16 smithi175 ceph-mon[31920]: pgmap v133: 129 pgs: 129 active+clean; 5.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-21T08:43:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:18 smithi112 ceph-mon[31085]: pgmap v134: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-21T08:43:18.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:18 smithi043 ceph-mon[27978]: pgmap v134: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-21T08:43:18.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:18 smithi175 ceph-mon[31920]: pgmap v134: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-21T08:43:19.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:43:19.042 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:43:19.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.160+0000 7f3e8a3b2640 1 -- 172.21.15.43:0/58242362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e84101a30 msgr2=0x7f3e84105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.160+0000 7f3e8a3b2640 1 --2- 172.21.15.43:0/58242362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e84101a30 0x7f3e84105a70 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f3e78009900 tx=0x7f3e7802bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:19.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.160+0000 7f3e8a3b2640 1 -- 172.21.15.43:0/58242362 shutdown_connections 2024-04-21T08:43:19.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.160+0000 7f3e8a3b2640 1 --2- 172.21.15.43:0/58242362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e84106150 0x7f3e8410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.160+0000 7f3e8a3b2640 1 --2- 172.21.15.43:0/58242362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e84101a30 0x7f3e84105a70 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.160+0000 7f3e8a3b2640 1 --2- 172.21.15.43:0/58242362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e84101080 0x7f3e84101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.160+0000 7f3e8a3b2640 1 -- 172.21.15.43:0/58242362 >> 172.21.15.43:0/58242362 conn(0x7f3e84078000 msgr2=0x7f3e84078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.160+0000 7f3e8a3b2640 1 -- 172.21.15.43:0/58242362 shutdown_connections 2024-04-21T08:43:19.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.160+0000 7f3e8a3b2640 1 -- 172.21.15.43:0/58242362 wait complete. 2024-04-21T08:43:19.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.161+0000 7f3e8a3b2640 1 Processor -- start 2024-04-21T08:43:19.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.161+0000 7f3e8a3b2640 1 -- start start 2024-04-21T08:43:19.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.161+0000 7f3e8a3b2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e84101080 0x7f3e84198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.161+0000 7f3e8a3b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e84106150 0x7f3e84198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.161+0000 7f3e8a3b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e8419cff0 0x7f3e8419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.161+0000 7f3e8a3b2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3e84111090 con 0x7f3e84106150 2024-04-21T08:43:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.161+0000 7f3e8a3b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e84110f10 con 0x7f3e8419cff0 2024-04-21T08:43:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.161+0000 7f3e8a3b2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3e84111210 con 0x7f3e84101080 2024-04-21T08:43:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.161+0000 7f3e837fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e84106150 0x7f3e84198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.161+0000 7f3e837fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e84106150 0x7f3e84198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57658/0 (socket says 172.21.15.43:57658) 2024-04-21T08:43:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.161+0000 7f3e837fe640 1 -- 172.21.15.43:0/2861504979 learned_addr learned my addr 172.21.15.43:0/2861504979 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.161+0000 7f3e8894f640 1 --2- 172.21.15.43:0/2861504979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e8419cff0 0x7f3e8419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.161+0000 7f3e83fff640 1 --2- 172.21.15.43:0/2861504979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e84101080 0x7f3e84198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.162+0000 7f3e837fe640 1 -- 172.21.15.43:0/2861504979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e84101080 msgr2=0x7f3e84198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.162+0000 7f3e837fe640 1 --2- 172.21.15.43:0/2861504979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e84101080 0x7f3e84198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.162+0000 7f3e837fe640 1 -- 172.21.15.43:0/2861504979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e8419cff0 msgr2=0x7f3e8419d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.162+0000 7f3e837fe640 1 --2- 172.21.15.43:0/2861504979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e8419cff0 0x7f3e8419d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.162+0000 7f3e837fe640 1 -- 172.21.15.43:0/2861504979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e8419dae0 con 0x7f3e84106150 2024-04-21T08:43:19.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.162+0000 7f3e83fff640 1 --2- 172.21.15.43:0/2861504979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e84101080 0x7f3e84198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:19.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.162+0000 7f3e837fe640 1 --2- 172.21.15.43:0/2861504979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e84106150 0x7f3e84198c70 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f3e78036040 tx=0x7f3e7802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.162+0000 7f3e817fa640 1 -- 172.21.15.43:0/2861504979 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78043020 con 0x7f3e84106150 2024-04-21T08:43:19.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.162+0000 7f3e817fa640 1 -- 172.21.15.43:0/2861504979 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e7802cbe0 con 0x7f3e84106150 2024-04-21T08:43:19.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.162+0000 7f3e8a3b2640 1 -- 172.21.15.43:0/2861504979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e8419dd70 con 0x7f3e84106150 2024-04-21T08:43:19.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.162+0000 7f3e817fa640 1 -- 172.21.15.43:0/2861504979 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78003bf0 con 0x7f3e84106150 2024-04-21T08:43:19.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.162+0000 7f3e8a3b2640 1 -- 172.21.15.43:0/2861504979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e841a55c0 con 0x7f3e84106150 2024-04-21T08:43:19.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.164+0000 7f3e817fa640 1 -- 172.21.15.43:0/2861504979 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3e78046020 con 0x7f3e84106150 2024-04-21T08:43:19.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.165+0000 7f3e8cfa0740 1 Processor -- start 2024-04-21T08:43:19.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.165+0000 7f3e8cfa0740 1 -- start start 2024-04-21T08:43:19.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.165+0000 7f3e8cfa0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d826608580 0x55d826608960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.165+0000 7f3e8cfa0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d826608ea0 0x55d826611540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.165+0000 7f3e8cfa0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826611a80 0x55d826613e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.165+0000 7f3e8cfa0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d82656d1e0 con 0x55d826611a80 2024-04-21T08:43:19.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.165+0000 7f3e8cfa0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d82650a390 con 0x55d826608ea0 2024-04-21T08:43:19.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.165+0000 7f3e8cfa0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d8265691a0 con 0x55d826608580 2024-04-21T08:43:19.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.165+0000 7f3e817fa640 1 --2- 172.21.15.43:0/2861504979 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3e6c0a3da0 0x7f3e6c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.165+0000 7f3e837fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826611a80 0x55d826613e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.165+0000 7f3e83fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d826608ea0 0x55d826611540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.165+0000 7f3e837fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826611a80 0x55d826613e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57672/0 (socket says 172.21.15.43:57672) 2024-04-21T08:43:19.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.165+0000 7f3e837fe640 1 -- 172.21.15.43:0/2214357093 learned_addr learned my addr 172.21.15.43:0/2214357093 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:19.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.165+0000 7f3e8894f640 1 --2- 172.21.15.43:0/2214357093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d826608580 0x55d826608960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.165+0000 7f3e8894f640 1 --2- 172.21.15.43:0/2861504979 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3e6c0a3da0 0x7f3e6c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.166+0000 7f3e83fff640 1 -- 172.21.15.43:0/2214357093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d826608580 msgr2=0x55d826608960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.166+0000 7f3e83fff640 1 --2- 172.21.15.43:0/2214357093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d826608580 0x55d826608960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.166+0000 7f3e817fa640 1 -- 172.21.15.43:0/2861504979 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f3e780f2c80 con 0x7f3e84106150 2024-04-21T08:43:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.166+0000 7f3e83fff640 1 -- 172.21.15.43:0/2214357093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826611a80 msgr2=0x55d826613e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.166+0000 7f3e83fff640 1 --2- 172.21.15.43:0/2214357093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826611a80 0x55d826613e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.166+0000 7f3e83fff640 1 -- 172.21.15.43:0/2214357093 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8263497d0 con 0x55d826608ea0 2024-04-21T08:43:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.166+0000 7f3e83fff640 1 --2- 172.21.15.43:0/2214357093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d826608ea0 0x55d826611540 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400bc80 tx=0x7f3e7400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=399239b677ce125 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.166+0000 7f3e8894f640 1 --2- 172.21.15.43:0/2861504979 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3e6c0a3da0 0x7f3e6c0a6260 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009fd0 tx=0x7f3e7c0096a0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.166+0000 7f3e6a7fc640 1 -- 172.21.15.43:0/2214357093 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e74015070 con 0x55d826608ea0 2024-04-21T08:43:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.166+0000 7f3e6a7fc640 1 -- 172.21.15.43:0/2214357093 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e7401a910 con 0x55d826608ea0 2024-04-21T08:43:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.166+0000 7f3e6a7fc640 1 -- 172.21.15.43:0/2214357093 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e74011860 con 0x55d826608ea0 2024-04-21T08:43:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.167+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/2214357093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d826608ea0 msgr2=0x55d826611540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.167+0000 7f3e8cfa0740 1 --2- 172.21.15.43:0/2214357093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d826608ea0 0x55d826611540 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400bc80 tx=0x7f3e7400f730 comp rx=0 tx=0).stop 2024-04-21T08:43:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.167+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/2214357093 shutdown_connections 2024-04-21T08:43:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.167+0000 7f3e8cfa0740 1 --2- 172.21.15.43:0/2214357093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826611a80 0x55d826613e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.167+0000 7f3e8cfa0740 1 --2- 172.21.15.43:0/2214357093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d826608ea0 0x55d826611540 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.167+0000 7f3e8cfa0740 1 --2- 172.21.15.43:0/2214357093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d826608580 0x55d826608960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.167+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/2214357093 >> 172.21.15.43:0/2214357093 conn(0x55d826602cb0 msgr2=0x55d826603090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.167+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/2214357093 shutdown_connections 2024-04-21T08:43:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.167+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/2214357093 wait complete. 2024-04-21T08:43:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.167+0000 7f3e8cfa0740 1 -- start start 2024-04-21T08:43:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.167+0000 7f3e8cfa0740 1 Processor -- start 2024-04-21T08:43:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.168+0000 7f3e8cfa0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d826608580 0x55d82660eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.168+0000 7f3e8cfa0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826608ea0 0x55d82660f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.168+0000 7f3e8cfa0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d826611a80 0x55d82660f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.168+0000 7f3e8cfa0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d826614870 con 0x55d826608ea0 2024-04-21T08:43:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.168+0000 7f3e8cfa0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d8265262f0 con 0x55d826611a80 2024-04-21T08:43:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.168+0000 7f3e8cfa0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d8266149f0 con 0x55d826608580 2024-04-21T08:43:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.168+0000 7f3e8cfa0740 10 client.0 _ll_register_callbacks cb 0x55d826533580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.168+0000 7f3e837fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826608ea0 0x55d82660f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.168+0000 7f3e837fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826608ea0 0x55d82660f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57688/0 (socket says 172.21.15.43:57688) 2024-04-21T08:43:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.168+0000 7f3e837fe640 1 -- 172.21.15.43:0/333008013 learned_addr learned my addr 172.21.15.43:0/333008013 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.168+0000 7f3e8894f640 1 --2- 172.21.15.43:0/333008013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d826611a80 0x55d82660f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.168+0000 7f3e83fff640 1 --2- 172.21.15.43:0/333008013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d826608580 0x55d82660eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.168+0000 7f3e837fe640 1 -- 172.21.15.43:0/333008013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d826608580 msgr2=0x55d82660eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.168+0000 7f3e837fe640 1 --2- 172.21.15.43:0/333008013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d826608580 0x55d82660eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.168+0000 7f3e837fe640 1 -- 172.21.15.43:0/333008013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d826611a80 msgr2=0x55d82660f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.168+0000 7f3e837fe640 1 --2- 172.21.15.43:0/333008013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d826611a80 0x55d82660f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.168+0000 7f3e837fe640 1 -- 172.21.15.43:0/333008013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d82627f3f0 con 0x55d826608ea0 2024-04-21T08:43:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.168+0000 7f3e837fe640 1 --2- 172.21.15.43:0/333008013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826608ea0 0x55d82660f020 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f3e780badc0 tx=0x7f3e7802cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55339c5889edd913 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.168+0000 7f3e637fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.168+0000 7f3e637fe640 1 -- 172.21.15.43:0/333008013 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e780be960 con 0x55d826608ea0 2024-04-21T08:43:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.169+0000 7f3e637fe640 1 -- 172.21.15.43:0/333008013 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f3e780bafd0 con 0x55d826608ea0 2024-04-21T08:43:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.169+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/333008013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e7401a910 con 0x55d826608ea0 2024-04-21T08:43:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.169+0000 7f3e637fe640 1 -- 172.21.15.43:0/333008013 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3e780c2d10 con 0x55d826608ea0 2024-04-21T08:43:19.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.169+0000 7f3e8cfa0740 10 client.17475 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:19.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.169+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/333008013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d82623f0f0 con 0x55d826608ea0 2024-04-21T08:43:19.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.169+0000 7f3e637fe640 1 -- 172.21.15.43:0/333008013 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e7401a910 con 0x55d826608ea0 2024-04-21T08:43:19.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.169+0000 7f3e637fe640 1 -- 172.21.15.43:0/333008013 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3e780a90b0 con 0x55d826608ea0 2024-04-21T08:43:19.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.169+0000 7f3e8cfa0740 10 client.17475 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:19.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.169+0000 7f3e8cfa0740 10 client.17475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:19.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.169+0000 7f3e8cfa0740 20 client.17475 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:19.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.169+0000 7f3e8cfa0740 1 --2- 172.21.15.43:0/333008013 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d826625e90 0x55d8266282d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.169+0000 7f3e8cfa0740 4 client.17475 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:19.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.169+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/333008013 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d82657ac00 con 0x55d826625e90 2024-04-21T08:43:19.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.169+0000 7f3e83fff640 1 --2- 172.21.15.43:0/333008013 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d826625e90 0x55d8266282d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.170+0000 7f3e83fff640 1 --2- 172.21.15.43:0/333008013 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d826625e90 0x55d8266282d0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d064ed9362c28d63 server_cookie=4004c81df3ca394a in_seq=0 out_seq=0 2024-04-21T08:43:19.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.170+0000 7f3e637fe640 10 client.17475 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:19.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.171+0000 7f3e637fe640 1 -- 172.21.15.43:0/333008013 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d82657ac00 con 0x55d826625e90 2024-04-21T08:43:19.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.171+0000 7f3e637fe640 10 client.17475 handle_command_reply: tid=0 2024-04-21T08:43:19.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.171+0000 7f3e8cfa0740 1 client.17475 shutdown 2024-04-21T08:43:19.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.171+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/333008013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826608ea0 msgr2=0x55d82660f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.171+0000 7f3e8cfa0740 1 --2- 172.21.15.43:0/333008013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826608ea0 0x55d82660f020 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f3e780badc0 tx=0x7f3e7802cd90 comp rx=0 tx=0).stop 2024-04-21T08:43:19.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.171+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/333008013 shutdown_connections 2024-04-21T08:43:19.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.171+0000 7f3e8cfa0740 1 --2- 172.21.15.43:0/333008013 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d826625e90 0x55d8266282d0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.171+0000 7f3e8cfa0740 1 --2- 172.21.15.43:0/333008013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d826611a80 0x55d82660f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.171+0000 7f3e8cfa0740 1 --2- 172.21.15.43:0/333008013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826608ea0 0x55d82660f020 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.171+0000 7f3e8cfa0740 1 --2- 172.21.15.43:0/333008013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d826608580 0x55d82660eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.171+0000 7f3e637fe640 0 client.17475 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:19.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.171+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/333008013 >> 172.21.15.43:0/333008013 conn(0x55d826602cb0 msgr2=0x55d826604a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.171+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/333008013 shutdown_connections 2024-04-21T08:43:19.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.171+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/333008013 wait complete. 2024-04-21T08:43:19.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.171+0000 7f3e8cfa0740 20 client.17475 trim_cache size 0 max 16384 2024-04-21T08:43:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.189+0000 7f3e8cfa0740 1 Processor -- start 2024-04-21T08:43:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.189+0000 7f3e8cfa0740 1 -- start start 2024-04-21T08:43:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.189+0000 7f3e8cfa0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d826608580 0x55d826611210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.189+0000 7f3e8cfa0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d826608ea0 0x55d82660da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.189+0000 7f3e8cfa0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826611a80 0x55d826621bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.189+0000 7f3e8cfa0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3e841ad1c0 con 0x55d826611a80 2024-04-21T08:43:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.189+0000 7f3e8cfa0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e841aed00 con 0x55d826608580 2024-04-21T08:43:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.189+0000 7f3e8cfa0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3e841ad890 con 0x55d826608ea0 2024-04-21T08:43:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.189+0000 7f3e8894f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826611a80 0x55d826621bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.189+0000 7f3e8894f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826611a80 0x55d826621bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57702/0 (socket says 172.21.15.43:57702) 2024-04-21T08:43:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.189+0000 7f3e8894f640 1 -- 172.21.15.43:0/3342243900 learned_addr learned my addr 172.21.15.43:0/3342243900 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.190+0000 7f3e83fff640 1 --2- 172.21.15.43:0/3342243900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d826608580 0x55d826611210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.190+0000 7f3e837fe640 1 --2- 172.21.15.43:0/3342243900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d826608ea0 0x55d82660da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.190+0000 7f3e8894f640 1 -- 172.21.15.43:0/3342243900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d826608ea0 msgr2=0x55d82660da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.190+0000 7f3e8894f640 1 --2- 172.21.15.43:0/3342243900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d826608ea0 0x55d82660da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.190+0000 7f3e8894f640 1 -- 172.21.15.43:0/3342243900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d826608580 msgr2=0x55d826611210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.190+0000 7f3e8894f640 1 --2- 172.21.15.43:0/3342243900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d826608580 0x55d826611210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.190+0000 7f3e8894f640 1 -- 172.21.15.43:0/3342243900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d826603d30 con 0x55d826611a80 2024-04-21T08:43:19.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.190+0000 7f3e83fff640 1 --2- 172.21.15.43:0/3342243900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d826608580 0x55d826611210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:19.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.190+0000 7f3e837fe640 1 --2- 172.21.15.43:0/3342243900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d826608ea0 0x55d82660da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:19.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.190+0000 7f3e8894f640 1 --2- 172.21.15.43:0/3342243900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826611a80 0x55d826621bc0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c003aa0 tx=0x7f3e7c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cade50126750a8a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.190+0000 7f3e69ffb640 1 -- 172.21.15.43:0/3342243900 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c026070 con 0x55d826611a80 2024-04-21T08:43:19.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.190+0000 7f3e69ffb640 1 -- 172.21.15.43:0/3342243900 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e7c0044e0 con 0x55d826611a80 2024-04-21T08:43:19.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.190+0000 7f3e69ffb640 1 -- 172.21.15.43:0/3342243900 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c02ba30 con 0x55d826611a80 2024-04-21T08:43:19.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.191+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/3342243900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826611a80 msgr2=0x55d826621bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.191+0000 7f3e8cfa0740 1 --2- 172.21.15.43:0/3342243900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826611a80 0x55d826621bc0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c003aa0 tx=0x7f3e7c009920 comp rx=0 tx=0).stop 2024-04-21T08:43:19.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.191+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/3342243900 shutdown_connections 2024-04-21T08:43:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.191+0000 7f3e8cfa0740 1 --2- 172.21.15.43:0/3342243900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826611a80 0x55d826621bc0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.191+0000 7f3e8cfa0740 1 --2- 172.21.15.43:0/3342243900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d826608ea0 0x55d82660da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.191+0000 7f3e8cfa0740 1 --2- 172.21.15.43:0/3342243900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d826608580 0x55d826611210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.191+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/3342243900 >> 172.21.15.43:0/3342243900 conn(0x55d826602cb0 msgr2=0x55d826601d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.191+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/3342243900 shutdown_connections 2024-04-21T08:43:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.191+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/3342243900 wait complete. 2024-04-21T08:43:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.191+0000 7f3e8cfa0740 1 -- start start 2024-04-21T08:43:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.191+0000 7f3e8cfa0740 1 Processor -- start 2024-04-21T08:43:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e8cfa0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826608580 0x55d82661dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e8cfa0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d826608ea0 0x55d82661e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e8cfa0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d826611a80 0x55d82661a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e8cfa0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3e841ae630 con 0x55d826608580 2024-04-21T08:43:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e8cfa0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e841adf60 con 0x55d826608ea0 2024-04-21T08:43:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e8cfa0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3e841ab1a0 con 0x55d826611a80 2024-04-21T08:43:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e8cfa0740 10 client.0 _ll_register_callbacks cb 0x7f3e780057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e83fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826608580 0x55d82661dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e83fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826608580 0x55d82661dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57704/0 (socket says 172.21.15.43:57704) 2024-04-21T08:43:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e83fff640 1 -- 172.21.15.43:0/1914402489 learned_addr learned my addr 172.21.15.43:0/1914402489 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e837fe640 1 --2- 172.21.15.43:0/1914402489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d826608ea0 0x55d82661e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e8894f640 1 --2- 172.21.15.43:0/1914402489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d826611a80 0x55d82661a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e83fff640 1 -- 172.21.15.43:0/1914402489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d826611a80 msgr2=0x55d82661a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e83fff640 1 --2- 172.21.15.43:0/1914402489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d826611a80 0x55d82661a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e83fff640 1 -- 172.21.15.43:0/1914402489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d826608ea0 msgr2=0x55d82661e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e83fff640 1 --2- 172.21.15.43:0/1914402489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d826608ea0 0x55d82661e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e83fff640 1 -- 172.21.15.43:0/1914402489 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d82661aa00 con 0x55d826608580 2024-04-21T08:43:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e837fe640 1 --2- 172.21.15.43:0/1914402489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d826608ea0 0x55d82661e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e83fff640 1 --2- 172.21.15.43:0/1914402489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826608580 0x55d82661dd10 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f3e74006010 tx=0x7f3e7400ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae61b904e9efca82 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e637fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e637fe640 1 -- 172.21.15.43:0/1914402489 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e74015040 con 0x55d826608580 2024-04-21T08:43:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e637fe640 1 -- 172.21.15.43:0/1914402489 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f3e74005450 con 0x55d826608580 2024-04-21T08:43:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/1914402489 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e7c0044e0 con 0x55d826608580 2024-04-21T08:43:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e637fe640 1 -- 172.21.15.43:0/1914402489 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3e7400b760 con 0x55d826608580 2024-04-21T08:43:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e8cfa0740 10 client.17481 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.192+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/1914402489 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d82668ce00 con 0x55d826608580 2024-04-21T08:43:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.193+0000 7f3e637fe640 1 -- 172.21.15.43:0/1914402489 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e7c0044e0 con 0x55d826608580 2024-04-21T08:43:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.193+0000 7f3e637fe640 1 -- 172.21.15.43:0/1914402489 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3e74027340 con 0x55d826608580 2024-04-21T08:43:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.193+0000 7f3e8cfa0740 10 client.17481 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.193+0000 7f3e8cfa0740 10 client.17481 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.193+0000 7f3e8cfa0740 20 client.17481 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.193+0000 7f3e8cfa0740 1 --2- 172.21.15.43:0/1914402489 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d82668d3b0 0x55d82668f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.193+0000 7f3e8cfa0740 4 client.17481 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:43:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.193+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/1914402489 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d826574a70 con 0x55d82668d3b0 2024-04-21T08:43:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.193+0000 7f3e837fe640 1 --2- 172.21.15.43:0/1914402489 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d82668d3b0 0x55d82668f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.193+0000 7f3e837fe640 1 --2- 172.21.15.43:0/1914402489 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d82668d3b0 0x55d82668f7d0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5adececf483d5a4a server_cookie=8966c0104bfe0d1b in_seq=0 out_seq=0 2024-04-21T08:43:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.193+0000 7f3e637fe640 10 client.17481 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.194+0000 7f3e637fe640 1 -- 172.21.15.43:0/1914402489 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d826574a70 con 0x55d82668d3b0 2024-04-21T08:43:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.194+0000 7f3e637fe640 10 client.17481 handle_command_reply: tid=0 2024-04-21T08:43:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.194+0000 7f3e8cfa0740 1 client.17481 shutdown 2024-04-21T08:43:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.194+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/1914402489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826608580 msgr2=0x55d82661dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.194+0000 7f3e8cfa0740 1 --2- 172.21.15.43:0/1914402489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826608580 0x55d82661dd10 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f3e74006010 tx=0x7f3e7400ff60 comp rx=0 tx=0).stop 2024-04-21T08:43:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.194+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/1914402489 shutdown_connections 2024-04-21T08:43:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.194+0000 7f3e8cfa0740 1 --2- 172.21.15.43:0/1914402489 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d82668d3b0 0x55d82668f7d0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.194+0000 7f3e8cfa0740 1 --2- 172.21.15.43:0/1914402489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d826611a80 0x55d82661a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.194+0000 7f3e8cfa0740 1 --2- 172.21.15.43:0/1914402489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d826608ea0 0x55d82661e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.194+0000 7f3e8cfa0740 1 --2- 172.21.15.43:0/1914402489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d826608580 0x55d82661dd10 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.194+0000 7f3e637fe640 0 client.17481 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.194+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/1914402489 >> 172.21.15.43:0/1914402489 conn(0x55d826602cb0 msgr2=0x55d82660eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.194+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/1914402489 shutdown_connections 2024-04-21T08:43:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.194+0000 7f3e8cfa0740 1 -- 172.21.15.43:0/1914402489 wait complete. 2024-04-21T08:43:19.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.194+0000 7f3e8cfa0740 20 client.17481 trim_cache size 0 max 16384 2024-04-21T08:43:19.200 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:43:19.200 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:43:19.200 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:43:19.200 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:43:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.195+0000 7f3e8a3b2640 1 -- 172.21.15.43:0/2861504979 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3e6c0a3da0 msgr2=0x7f3e6c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.195+0000 7f3e8a3b2640 1 --2- 172.21.15.43:0/2861504979 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3e6c0a3da0 0x7f3e6c0a6260 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009fd0 tx=0x7f3e7c0096a0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.195+0000 7f3e8a3b2640 1 -- 172.21.15.43:0/2861504979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e84106150 msgr2=0x7f3e84198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.195+0000 7f3e8a3b2640 1 --2- 172.21.15.43:0/2861504979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e84106150 0x7f3e84198c70 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f3e78036040 tx=0x7f3e7802c7e0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.195+0000 7f3e8a3b2640 1 -- 172.21.15.43:0/2861504979 shutdown_connections 2024-04-21T08:43:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.195+0000 7f3e8a3b2640 1 --2- 172.21.15.43:0/2861504979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e8419cff0 0x7f3e8419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.195+0000 7f3e8a3b2640 1 --2- 172.21.15.43:0/2861504979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e84106150 0x7f3e84198c70 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.195+0000 7f3e8a3b2640 1 --2- 172.21.15.43:0/2861504979 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3e6c0a3da0 0x7f3e6c0a6260 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.195+0000 7f3e8a3b2640 1 --2- 172.21.15.43:0/2861504979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e84101080 0x7f3e84198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.195+0000 7f3e8a3b2640 1 -- 172.21.15.43:0/2861504979 >> 172.21.15.43:0/2861504979 conn(0x7f3e84078000 msgr2=0x7f3e84069540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.195+0000 7f3e8a3b2640 1 -- 172.21.15.43:0/2861504979 shutdown_connections 2024-04-21T08:43:19.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.195+0000 7f3e8a3b2640 1 -- 172.21.15.43:0/2861504979 wait complete. 2024-04-21T08:43:19.205 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:43:19.205 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:43:19.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.309+0000 7f2c34293640 1 -- 172.21.15.43:0/4246801504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c2c106180 msgr2=0x7f2c2c10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.309+0000 7f2c34293640 1 --2- 172.21.15.43:0/4246801504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c2c106180 0x7f2c2c10dd50 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009900 tx=0x7f2c2402bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:19.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.309+0000 7f2c34293640 1 -- 172.21.15.43:0/4246801504 shutdown_connections 2024-04-21T08:43:19.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.309+0000 7f2c34293640 1 --2- 172.21.15.43:0/4246801504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c2c106180 0x7f2c2c10dd50 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.309+0000 7f2c34293640 1 --2- 172.21.15.43:0/4246801504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c2c101a60 0x7f2c2c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.309+0000 7f2c34293640 1 --2- 172.21.15.43:0/4246801504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c2c1010b0 0x7f2c2c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.309+0000 7f2c34293640 1 -- 172.21.15.43:0/4246801504 >> 172.21.15.43:0/4246801504 conn(0x7f2c2c0771d0 msgr2=0x7f2c2c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.309+0000 7f2c34293640 1 -- 172.21.15.43:0/4246801504 shutdown_connections 2024-04-21T08:43:19.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.309+0000 7f2c34293640 1 -- 172.21.15.43:0/4246801504 wait complete. 2024-04-21T08:43:19.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.310+0000 7f2c34293640 1 Processor -- start 2024-04-21T08:43:19.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.310+0000 7f2c34293640 1 -- start start 2024-04-21T08:43:19.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.310+0000 7f2c34293640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c2c1010b0 0x7f2c2c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.310+0000 7f2c34293640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c2c106180 0x7f2c2c198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.310+0000 7f2c34293640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c2c19cf50 0x7f2c2c19d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.310+0000 7f2c34293640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c112200 con 0x7f2c2c106180 2024-04-21T08:43:19.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.310+0000 7f2c34293640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c112080 con 0x7f2c2c1010b0 2024-04-21T08:43:19.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.310+0000 7f2c34293640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c112380 con 0x7f2c2c19cf50 2024-04-21T08:43:19.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.310+0000 7f2c3182e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c2c106180 0x7f2c2c198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.310+0000 7f2c3182e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c2c106180 0x7f2c2c198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57722/0 (socket says 172.21.15.43:57722) 2024-04-21T08:43:19.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.310+0000 7f2c3182e640 1 -- 172.21.15.43:0/2994004231 learned_addr learned my addr 172.21.15.43:0/2994004231 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:19.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.311+0000 7f2c3202f640 1 --2- 172.21.15.43:0/2994004231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c2c1010b0 0x7f2c2c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.311+0000 7f2c32830640 1 --2- 172.21.15.43:0/2994004231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c2c19cf50 0x7f2c2c19d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.311+0000 7f2c3182e640 1 -- 172.21.15.43:0/2994004231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c2c19cf50 msgr2=0x7f2c2c19d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.311+0000 7f2c3182e640 1 --2- 172.21.15.43:0/2994004231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c2c19cf50 0x7f2c2c19d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.311+0000 7f2c3182e640 1 -- 172.21.15.43:0/2994004231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c2c1010b0 msgr2=0x7f2c2c1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.311+0000 7f2c3182e640 1 --2- 172.21.15.43:0/2994004231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c2c1010b0 0x7f2c2c1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.311+0000 7f2c3182e640 1 -- 172.21.15.43:0/2994004231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2c19da40 con 0x7f2c2c106180 2024-04-21T08:43:19.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.311+0000 7f2c3202f640 1 --2- 172.21.15.43:0/2994004231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c2c1010b0 0x7f2c2c1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:19.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.311+0000 7f2c3182e640 1 --2- 172.21.15.43:0/2994004231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c2c106180 0x7f2c2c198c20 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000d740 tx=0x7f2c2000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.311+0000 7f2c2b7fe640 1 -- 172.21.15.43:0/2994004231 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20017070 con 0x7f2c2c106180 2024-04-21T08:43:19.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.311+0000 7f2c34293640 1 -- 172.21.15.43:0/2994004231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c2c19dd30 con 0x7f2c2c106180 2024-04-21T08:43:19.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.311+0000 7f2c2b7fe640 1 -- 172.21.15.43:0/2994004231 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c2000de50 con 0x7f2c2c106180 2024-04-21T08:43:19.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.311+0000 7f2c2b7fe640 1 -- 172.21.15.43:0/2994004231 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20004ce0 con 0x7f2c2c106180 2024-04-21T08:43:19.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.311+0000 7f2c34293640 1 -- 172.21.15.43:0/2994004231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c2c1a54d0 con 0x7f2c2c106180 2024-04-21T08:43:19.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.312+0000 7f2c34293640 1 -- 172.21.15.43:0/2994004231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c2c069230 con 0x7f2c2c106180 2024-04-21T08:43:19.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.313+0000 7f2c2b7fe640 1 -- 172.21.15.43:0/2994004231 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2c20004e80 con 0x7f2c2c106180 2024-04-21T08:43:19.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.314+0000 7f2c2b7fe640 1 --2- 172.21.15.43:0/2994004231 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2c140a3d20 0x7f2c140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.314+0000 7f2c3202f640 1 --2- 172.21.15.43:0/2994004231 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2c140a3d20 0x7f2c140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.314+0000 7f2c2b7fe640 1 -- 172.21.15.43:0/2994004231 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f2c200d05a0 con 0x7f2c2c106180 2024-04-21T08:43:19.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.314+0000 7f2c3202f640 1 --2- 172.21.15.43:0/2994004231 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2c140a3d20 0x7f2c140a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c005fd0 tx=0x7f2c1c009970 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.318+0000 7f2c2b7fe640 1 -- 172.21.15.43:0/2994004231 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c2009c870 con 0x7f2c2c106180 2024-04-21T08:43:19.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.545+0000 7f2c34293640 1 -- 172.21.15.43:0/2994004231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c2c101a60 con 0x7f2c2c106180 2024-04-21T08:43:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.546+0000 7f2c2b7fe640 1 -- 172.21.15.43:0/2994004231 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7f2c2009c170 con 0x7f2c2c106180 2024-04-21T08:43:19.545 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:43:19.545 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:43:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:43:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.548+0000 7f2c34293640 1 -- 172.21.15.43:0/2994004231 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2c140a3d20 msgr2=0x7f2c140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.548+0000 7f2c34293640 1 --2- 172.21.15.43:0/2994004231 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2c140a3d20 0x7f2c140a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c005fd0 tx=0x7f2c1c009970 comp rx=0 tx=0).stop 2024-04-21T08:43:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.548+0000 7f2c34293640 1 -- 172.21.15.43:0/2994004231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c2c106180 msgr2=0x7f2c2c198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.548+0000 7f2c34293640 1 --2- 172.21.15.43:0/2994004231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c2c106180 0x7f2c2c198c20 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000d740 tx=0x7f2c2000dc00 comp rx=0 tx=0).stop 2024-04-21T08:43:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.548+0000 7f2c34293640 1 -- 172.21.15.43:0/2994004231 shutdown_connections 2024-04-21T08:43:19.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.548+0000 7f2c34293640 1 --2- 172.21.15.43:0/2994004231 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2c140a3d20 0x7f2c140a61e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.548+0000 7f2c34293640 1 --2- 172.21.15.43:0/2994004231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c2c19cf50 0x7f2c2c19d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.548+0000 7f2c34293640 1 --2- 172.21.15.43:0/2994004231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c2c106180 0x7f2c2c198c20 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.548+0000 7f2c34293640 1 --2- 172.21.15.43:0/2994004231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c2c1010b0 0x7f2c2c1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.548+0000 7f2c34293640 1 -- 172.21.15.43:0/2994004231 >> 172.21.15.43:0/2994004231 conn(0x7f2c2c0771d0 msgr2=0x7f2c2c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.548+0000 7f2c34293640 1 -- 172.21.15.43:0/2994004231 shutdown_connections 2024-04-21T08:43:19.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.548+0000 7f2c34293640 1 -- 172.21.15.43:0/2994004231 wait complete. 2024-04-21T08:43:19.558 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:43:19.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.663+0000 7f1db16d2640 1 -- 172.21.15.43:0/421257375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dac1021c0 msgr2=0x7f1dac1025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.663+0000 7f1db16d2640 1 --2- 172.21.15.43:0/421257375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dac1021c0 0x7f1dac1025a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009ca0 tx=0x7f1d9c02c120 comp rx=0 tx=0).stop 2024-04-21T08:43:19.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.663+0000 7f1db16d2640 1 -- 172.21.15.43:0/421257375 shutdown_connections 2024-04-21T08:43:19.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.663+0000 7f1db16d2640 1 --2- 172.21.15.43:0/421257375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dac107290 0x7f1dac10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.663+0000 7f1db16d2640 1 --2- 172.21.15.43:0/421257375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dac102b70 0x7f1dac106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.663+0000 7f1db16d2640 1 --2- 172.21.15.43:0/421257375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dac1021c0 0x7f1dac1025a0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.663+0000 7f1db16d2640 1 -- 172.21.15.43:0/421257375 >> 172.21.15.43:0/421257375 conn(0x7f1dac0793e0 msgr2=0x7f1dac0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.663+0000 7f1db16d2640 1 -- 172.21.15.43:0/421257375 shutdown_connections 2024-04-21T08:43:19.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.664+0000 7f1db16d2640 1 -- 172.21.15.43:0/421257375 wait complete. 2024-04-21T08:43:19.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.664+0000 7f1db16d2640 1 Processor -- start 2024-04-21T08:43:19.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.664+0000 7f1db16d2640 1 -- start start 2024-04-21T08:43:19.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.664+0000 7f1db16d2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dac1021c0 0x7f1dac19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.664+0000 7f1db16d2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dac107290 0x7f1dac19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.664+0000 7f1db16d2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dac198670 0x7f1dac198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.664+0000 7f1db16d2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac112270 con 0x7f1dac198670 2024-04-21T08:43:19.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.664+0000 7f1db16d2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac1120f0 con 0x7f1dac1021c0 2024-04-21T08:43:19.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.665+0000 7f1db16d2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac1123f0 con 0x7f1dac107290 2024-04-21T08:43:19.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.665+0000 7f1dab7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dac198670 0x7f1dac198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.665+0000 7f1dab7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dac198670 0x7f1dac198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57744/0 (socket says 172.21.15.43:57744) 2024-04-21T08:43:19.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.665+0000 7f1dab7fe640 1 -- 172.21.15.43:0/2785485943 learned_addr learned my addr 172.21.15.43:0/2785485943 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:19.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.665+0000 7f1daaffd640 1 --2- 172.21.15.43:0/2785485943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dac1021c0 0x7f1dac19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.665+0000 7f1dab7fe640 1 -- 172.21.15.43:0/2785485943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dac107290 msgr2=0x7f1dac19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.665+0000 7f1dab7fe640 1 --2- 172.21.15.43:0/2785485943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dac107290 0x7f1dac19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.665+0000 7f1dab7fe640 1 -- 172.21.15.43:0/2785485943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dac1021c0 msgr2=0x7f1dac19e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.665+0000 7f1dab7fe640 1 --2- 172.21.15.43:0/2785485943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dac1021c0 0x7f1dac19e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.665+0000 7f1dab7fe640 1 -- 172.21.15.43:0/2785485943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dac199270 con 0x7f1dac198670 2024-04-21T08:43:19.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.665+0000 7f1dab7fe640 1 --2- 172.21.15.43:0/2785485943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dac198670 0x7f1dac198af0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f1da400b410 tx=0x7f1da400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.665+0000 7f1d93fff640 1 -- 172.21.15.43:0/2785485943 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4015020 con 0x7f1dac198670 2024-04-21T08:43:19.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.665+0000 7f1d93fff640 1 -- 172.21.15.43:0/2785485943 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1da4011ce0 con 0x7f1dac198670 2024-04-21T08:43:19.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.665+0000 7f1d93fff640 1 -- 172.21.15.43:0/2785485943 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4010d50 con 0x7f1dac198670 2024-04-21T08:43:19.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.665+0000 7f1db16d2640 1 -- 172.21.15.43:0/2785485943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dac1a5350 con 0x7f1dac198670 2024-04-21T08:43:19.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.665+0000 7f1db16d2640 1 -- 172.21.15.43:0/2785485943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dac1a5700 con 0x7f1dac198670 2024-04-21T08:43:19.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.667+0000 7f1d93fff640 1 -- 172.21.15.43:0/2785485943 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1da4018020 con 0x7f1dac198670 2024-04-21T08:43:19.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.668+0000 7f1db439e740 1 Processor -- start 2024-04-21T08:43:19.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.668+0000 7f1db439e740 1 -- start start 2024-04-21T08:43:19.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.668+0000 7f1db439e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c78bb653b0 0x55c78bb65790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.668+0000 7f1db439e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c78bb65d60 0x55c78bb6e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.668+0000 7f1db439e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb6ee60 0x55c78bb6f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.668+0000 7f1db439e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c78baca1f0 con 0x55c78bb6ee60 2024-04-21T08:43:19.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.668+0000 7f1db439e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c78ba674b0 con 0x55c78bb653b0 2024-04-21T08:43:19.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.668+0000 7f1db439e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c78bac61b0 con 0x55c78bb65d60 2024-04-21T08:43:19.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.668+0000 7f1daa7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb6ee60 0x55c78bb6f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.668+0000 7f1daa7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb6ee60 0x55c78bb6f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57748/0 (socket says 172.21.15.43:57748) 2024-04-21T08:43:19.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.668+0000 7f1daa7fc640 1 -- 172.21.15.43:0/220436246 learned_addr learned my addr 172.21.15.43:0/220436246 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:19.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.668+0000 7f1d93fff640 1 --2- 172.21.15.43:0/2785485943 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1d940a3da0 0x7f1d940a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.668+0000 7f1daa7fc640 1 --2- 172.21.15.43:0/220436246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c78bb653b0 0x55c78bb65790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.668+0000 7f1daaffd640 1 --2- 172.21.15.43:0/220436246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c78bb65d60 0x55c78bb6e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.668+0000 7f1dab7fe640 1 --2- 172.21.15.43:0/2785485943 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1d940a3da0 0x7f1d940a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.668+0000 7f1daa7fc640 1 -- 172.21.15.43:0/220436246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c78bb65d60 msgr2=0x55c78bb6e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.668+0000 7f1daa7fc640 1 --2- 172.21.15.43:0/220436246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c78bb65d60 0x55c78bb6e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.668+0000 7f1daa7fc640 1 -- 172.21.15.43:0/220436246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c78bb653b0 msgr2=0x55c78bb65790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.668+0000 7f1daa7fc640 1 --2- 172.21.15.43:0/220436246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c78bb653b0 0x55c78bb65790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.668+0000 7f1daa7fc640 1 -- 172.21.15.43:0/220436246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c78ba591b0 con 0x55c78bb6ee60 2024-04-21T08:43:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.668+0000 7f1d93fff640 1 -- 172.21.15.43:0/2785485943 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f1da40d02f0 con 0x7f1dac198670 2024-04-21T08:43:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.669+0000 7f1daa7fc640 1 --2- 172.21.15.43:0/220436246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb6ee60 0x55c78bb6f2c0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f1da0013720 tx=0x7f1da000c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2ea8ff99f83a9d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.669+0000 7f1d917fa640 1 -- 172.21.15.43:0/220436246 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da0004220 con 0x55c78bb6ee60 2024-04-21T08:43:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.669+0000 7f1d917fa640 1 -- 172.21.15.43:0/220436246 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1da00043c0 con 0x55c78bb6ee60 2024-04-21T08:43:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.669+0000 7f1dab7fe640 1 --2- 172.21.15.43:0/2785485943 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1d940a3da0 0x7f1d940a6260 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1da409bc70 tx=0x7f1da4009f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.669+0000 7f1d917fa640 1 -- 172.21.15.43:0/220436246 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da00098d0 con 0x55c78bb6ee60 2024-04-21T08:43:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.670+0000 7f1db439e740 1 -- 172.21.15.43:0/220436246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb6ee60 msgr2=0x55c78bb6f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.670+0000 7f1db439e740 1 --2- 172.21.15.43:0/220436246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb6ee60 0x55c78bb6f2c0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f1da0013720 tx=0x7f1da000c270 comp rx=0 tx=0).stop 2024-04-21T08:43:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.670+0000 7f1db439e740 1 -- 172.21.15.43:0/220436246 shutdown_connections 2024-04-21T08:43:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.670+0000 7f1db439e740 1 --2- 172.21.15.43:0/220436246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb6ee60 0x55c78bb6f2c0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.670+0000 7f1db439e740 1 --2- 172.21.15.43:0/220436246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c78bb65d60 0x55c78bb6e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.670+0000 7f1db439e740 1 --2- 172.21.15.43:0/220436246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c78bb653b0 0x55c78bb65790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.670+0000 7f1db439e740 1 -- 172.21.15.43:0/220436246 >> 172.21.15.43:0/220436246 conn(0x55c78bb5fba0 msgr2=0x55c78bb5ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.670+0000 7f1db439e740 1 -- 172.21.15.43:0/220436246 shutdown_connections 2024-04-21T08:43:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.670+0000 7f1db439e740 1 -- 172.21.15.43:0/220436246 wait complete. 2024-04-21T08:43:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.670+0000 7f1db439e740 1 -- start start 2024-04-21T08:43:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.670+0000 7f1db439e740 1 Processor -- start 2024-04-21T08:43:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.670+0000 7f1db439e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c78bb653b0 0x55c78bb75290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.670+0000 7f1db439e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c78bb65d60 0x55c78bb72620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.670+0000 7f1db439e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb6ee60 0x55c78bb72b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.670+0000 7f1db439e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c78bb75da0 con 0x55c78bb6ee60 2024-04-21T08:43:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.670+0000 7f1db439e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c78ba83500 con 0x55c78bb653b0 2024-04-21T08:43:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.670+0000 7f1db439e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c78bb75f20 con 0x55c78bb65d60 2024-04-21T08:43:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.670+0000 7f1db439e740 10 client.0 _ll_register_callbacks cb 0x55c78ba51b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.670+0000 7f1daa7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb6ee60 0x55c78bb72b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.670+0000 7f1daa7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb6ee60 0x55c78bb72b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57752/0 (socket says 172.21.15.43:57752) 2024-04-21T08:43:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.670+0000 7f1daa7fc640 1 -- 172.21.15.43:0/2466732158 learned_addr learned my addr 172.21.15.43:0/2466732158 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.671+0000 7f1daa7fc640 1 --2- 172.21.15.43:0/2466732158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c78bb653b0 0x55c78bb75290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.671+0000 7f1daa7fc640 1 -- 172.21.15.43:0/2466732158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c78bb65d60 msgr2=0x55c78bb72620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.671+0000 7f1daa7fc640 1 --2- 172.21.15.43:0/2466732158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c78bb65d60 0x55c78bb72620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.671+0000 7f1daa7fc640 1 -- 172.21.15.43:0/2466732158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c78bb653b0 msgr2=0x55c78bb75290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.671+0000 7f1daa7fc640 1 --2- 172.21.15.43:0/2466732158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c78bb653b0 0x55c78bb75290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.671+0000 7f1daa7fc640 1 -- 172.21.15.43:0/2466732158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c78b7dc3f0 con 0x55c78bb6ee60 2024-04-21T08:43:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.671+0000 7f1daa7fc640 1 --2- 172.21.15.43:0/2466732158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb6ee60 0x55c78bb72b80 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f1da000cc20 tx=0x7f1da0003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c32c8e5ab7bcce59 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.671+0000 7f1d8eff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.671+0000 7f1d8eff5640 1 -- 172.21.15.43:0/2466732158 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da0013050 con 0x55c78bb6ee60 2024-04-21T08:43:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.671+0000 7f1db439e740 1 -- 172.21.15.43:0/2466732158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1da00043c0 con 0x55c78bb6ee60 2024-04-21T08:43:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.671+0000 7f1d8eff5640 1 -- 172.21.15.43:0/2466732158 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f1da00131f0 con 0x55c78bb6ee60 2024-04-21T08:43:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.671+0000 7f1d8eff5640 1 -- 172.21.15.43:0/2466732158 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1da002f050 con 0x55c78bb6ee60 2024-04-21T08:43:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.671+0000 7f1db439e740 10 client.17499 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.671+0000 7f1db439e740 1 -- 172.21.15.43:0/2466732158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c78b8a67d0 con 0x55c78bb6ee60 2024-04-21T08:43:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.671+0000 7f1d8eff5640 1 -- 172.21.15.43:0/2466732158 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1da00043c0 con 0x55c78bb6ee60 2024-04-21T08:43:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.672+0000 7f1d8eff5640 1 -- 172.21.15.43:0/2466732158 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1da000f040 con 0x55c78bb6ee60 2024-04-21T08:43:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.672+0000 7f1db439e740 10 client.17499 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.672+0000 7f1db439e740 10 client.17499 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:43:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.672+0000 7f1db439e740 20 client.17499 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.672+0000 7f1db439e740 1 --2- 172.21.15.43:0/2466732158 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55c78bb82ed0 0x55c78bb852c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.672+0000 7f1db439e740 4 client.17499 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.672+0000 7f1db439e740 1 -- 172.21.15.43:0/2466732158 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c78ba69930 con 0x55c78bb82ed0 2024-04-21T08:43:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.672+0000 7f1daaffd640 1 --2- 172.21.15.43:0/2466732158 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55c78bb82ed0 0x55c78bb852c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.672+0000 7f1d8eff5640 10 client.17499 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:43:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.672+0000 7f1daaffd640 1 --2- 172.21.15.43:0/2466732158 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55c78bb82ed0 0x55c78bb852c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=24fa5db6ce54237b server_cookie=69a1f35be85a2131 in_seq=0 out_seq=0 2024-04-21T08:43:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.673+0000 7f1d8eff5640 1 -- 172.21.15.43:0/2466732158 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c78ba69930 con 0x55c78bb82ed0 2024-04-21T08:43:19.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.673+0000 7f1d8eff5640 10 client.17499 handle_command_reply: tid=0 2024-04-21T08:43:19.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.673+0000 7f1db439e740 1 client.17499 shutdown 2024-04-21T08:43:19.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.673+0000 7f1db439e740 1 -- 172.21.15.43:0/2466732158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb6ee60 msgr2=0x55c78bb72b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.673+0000 7f1db439e740 1 --2- 172.21.15.43:0/2466732158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb6ee60 0x55c78bb72b80 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f1da000cc20 tx=0x7f1da0003e80 comp rx=0 tx=0).stop 2024-04-21T08:43:19.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.673+0000 7f1db439e740 1 -- 172.21.15.43:0/2466732158 shutdown_connections 2024-04-21T08:43:19.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.673+0000 7f1db439e740 1 --2- 172.21.15.43:0/2466732158 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55c78bb82ed0 0x55c78bb852c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.673+0000 7f1db439e740 1 --2- 172.21.15.43:0/2466732158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb6ee60 0x55c78bb72b80 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.673+0000 7f1d8eff5640 0 client.17499 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:43:19.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.673+0000 7f1db439e740 1 --2- 172.21.15.43:0/2466732158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c78bb65d60 0x55c78bb72620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.673+0000 7f1db439e740 1 --2- 172.21.15.43:0/2466732158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c78bb653b0 0x55c78bb75290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.673+0000 7f1db439e740 1 -- 172.21.15.43:0/2466732158 >> 172.21.15.43:0/2466732158 conn(0x55c78bb5fba0 msgr2=0x55c78bb618f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.673+0000 7f1db439e740 1 -- 172.21.15.43:0/2466732158 shutdown_connections 2024-04-21T08:43:19.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.673+0000 7f1db439e740 1 -- 172.21.15.43:0/2466732158 wait complete. 2024-04-21T08:43:19.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.674+0000 7f1db439e740 20 client.17499 trim_cache size 0 max 16384 2024-04-21T08:43:19.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.687+0000 7f1db439e740 1 Processor -- start 2024-04-21T08:43:19.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.687+0000 7f1db439e740 1 -- start start 2024-04-21T08:43:19.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.687+0000 7f1db439e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c78bb653b0 0x55c78bb71780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.687+0000 7f1db439e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb65d60 0x55c78bb7e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.687+0000 7f1db439e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c78bb6ee60 0x55c78bb74d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.687+0000 7f1db439e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac1ad330 con 0x55c78bb65d60 2024-04-21T08:43:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.687+0000 7f1db439e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac1aee70 con 0x55c78bb653b0 2024-04-21T08:43:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.687+0000 7f1db439e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac1ada00 con 0x55c78bb6ee60 2024-04-21T08:43:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.687+0000 7f1daaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb65d60 0x55c78bb7e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.687+0000 7f1daaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb65d60 0x55c78bb7e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57758/0 (socket says 172.21.15.43:57758) 2024-04-21T08:43:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.687+0000 7f1daaffd640 1 -- 172.21.15.43:0/3752993205 learned_addr learned my addr 172.21.15.43:0/3752993205 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.687+0000 7f1daa7fc640 1 --2- 172.21.15.43:0/3752993205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c78bb653b0 0x55c78bb71780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.687+0000 7f1daaffd640 1 -- 172.21.15.43:0/3752993205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c78bb6ee60 msgr2=0x55c78bb74d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.687+0000 7f1daaffd640 1 --2- 172.21.15.43:0/3752993205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c78bb6ee60 0x55c78bb74d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.688+0000 7f1daaffd640 1 -- 172.21.15.43:0/3752993205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c78bb653b0 msgr2=0x55c78bb71780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.688+0000 7f1daaffd640 1 --2- 172.21.15.43:0/3752993205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c78bb653b0 0x55c78bb71780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.688+0000 7f1daaffd640 1 -- 172.21.15.43:0/3752993205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c78bb5b470 con 0x55c78bb65d60 2024-04-21T08:43:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.688+0000 7f1daaffd640 1 --2- 172.21.15.43:0/3752993205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb65d60 0x55c78bb7e8c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009ca0 tx=0x7f1d9c009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23cd39cbdde67f58 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.688+0000 7f1d90ff9640 1 -- 172.21.15.43:0/3752993205 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c039040 con 0x55c78bb65d60 2024-04-21T08:43:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.688+0000 7f1d90ff9640 1 -- 172.21.15.43:0/3752993205 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d9c04b030 con 0x55c78bb65d60 2024-04-21T08:43:19.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.688+0000 7f1d90ff9640 1 -- 172.21.15.43:0/3752993205 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c0473d0 con 0x55c78bb65d60 2024-04-21T08:43:19.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.688+0000 7f1db439e740 1 -- 172.21.15.43:0/3752993205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb65d60 msgr2=0x55c78bb7e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.688+0000 7f1db439e740 1 --2- 172.21.15.43:0/3752993205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb65d60 0x55c78bb7e8c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009ca0 tx=0x7f1d9c009830 comp rx=0 tx=0).stop 2024-04-21T08:43:19.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.688+0000 7f1db439e740 1 -- 172.21.15.43:0/3752993205 shutdown_connections 2024-04-21T08:43:19.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.688+0000 7f1db439e740 1 --2- 172.21.15.43:0/3752993205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c78bb6ee60 0x55c78bb74d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.688+0000 7f1db439e740 1 --2- 172.21.15.43:0/3752993205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb65d60 0x55c78bb7e8c0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.688+0000 7f1db439e740 1 --2- 172.21.15.43:0/3752993205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c78bb653b0 0x55c78bb71780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.688+0000 7f1db439e740 1 -- 172.21.15.43:0/3752993205 >> 172.21.15.43:0/3752993205 conn(0x55c78bb5fba0 msgr2=0x55c78bb7e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.688+0000 7f1db439e740 1 -- 172.21.15.43:0/3752993205 shutdown_connections 2024-04-21T08:43:19.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.689+0000 7f1db439e740 1 -- 172.21.15.43:0/3752993205 wait complete. 2024-04-21T08:43:19.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.689+0000 7f1db439e740 1 -- start start 2024-04-21T08:43:19.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.689+0000 7f1db439e740 1 Processor -- start 2024-04-21T08:43:19.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.689+0000 7f1db439e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c78bb653b0 0x55c78bb887c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.689+0000 7f1db439e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb65d60 0x55c78bb83b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.689+0000 7f1db439e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c78bb6ee60 0x55c78bb84140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.689+0000 7f1db439e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac1ae7a0 con 0x55c78bb65d60 2024-04-21T08:43:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.689+0000 7f1db439e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac1ae0d0 con 0x55c78bb653b0 2024-04-21T08:43:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.689+0000 7f1db439e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac1ab310 con 0x55c78bb6ee60 2024-04-21T08:43:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.689+0000 7f1db439e740 10 client.0 _ll_register_callbacks cb 0x7f1da0013620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.689+0000 7f1daaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb65d60 0x55c78bb83b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.689+0000 7f1daaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb65d60 0x55c78bb83b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57772/0 (socket says 172.21.15.43:57772) 2024-04-21T08:43:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.689+0000 7f1daaffd640 1 -- 172.21.15.43:0/3324598558 learned_addr learned my addr 172.21.15.43:0/3324598558 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.689+0000 7f1daa7fc640 1 --2- 172.21.15.43:0/3324598558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c78bb653b0 0x55c78bb887c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.689+0000 7f1daaffd640 1 -- 172.21.15.43:0/3324598558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c78bb6ee60 msgr2=0x55c78bb84140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.689+0000 7f1daaffd640 1 --2- 172.21.15.43:0/3324598558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c78bb6ee60 0x55c78bb84140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.689+0000 7f1daaffd640 1 -- 172.21.15.43:0/3324598558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c78bb653b0 msgr2=0x55c78bb887c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.689+0000 7f1daaffd640 1 --2- 172.21.15.43:0/3324598558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c78bb653b0 0x55c78bb887c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.689+0000 7f1daaffd640 1 -- 172.21.15.43:0/3324598558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c78bb84940 con 0x55c78bb65d60 2024-04-21T08:43:19.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.689+0000 7f1daa7fc640 1 --2- 172.21.15.43:0/3324598558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c78bb653b0 0x55c78bb887c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:19.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.690+0000 7f1daaffd640 1 --2- 172.21.15.43:0/3324598558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb65d60 0x55c78bb83b30 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c04d000 tx=0x7f1d9c032e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c2f1c35810f4f97 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.690+0000 7f1d8eff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:19.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.690+0000 7f1d8eff5640 1 -- 172.21.15.43:0/3324598558 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c054070 con 0x55c78bb65d60 2024-04-21T08:43:19.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.690+0000 7f1d8eff5640 1 -- 172.21.15.43:0/3324598558 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f1d9c003c70 con 0x55c78bb65d60 2024-04-21T08:43:19.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.690+0000 7f1db439e740 1 -- 172.21.15.43:0/3324598558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d9c04b030 con 0x55c78bb65d60 2024-04-21T08:43:19.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.690+0000 7f1d8eff5640 1 -- 172.21.15.43:0/3324598558 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1d9c05f050 con 0x55c78bb65d60 2024-04-21T08:43:19.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.690+0000 7f1db439e740 10 client.17505 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:19.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.690+0000 7f1db439e740 1 -- 172.21.15.43:0/3324598558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c78bbe91d0 con 0x55c78bb65d60 2024-04-21T08:43:19.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.690+0000 7f1d8eff5640 1 -- 172.21.15.43:0/3324598558 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d9c04b030 con 0x55c78bb65d60 2024-04-21T08:43:19.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.690+0000 7f1d8eff5640 1 -- 172.21.15.43:0/3324598558 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1d9c02dce0 con 0x55c78bb65d60 2024-04-21T08:43:19.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.690+0000 7f1db439e740 10 client.17505 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:19.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.690+0000 7f1db439e740 10 client.17505 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:43:19.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.690+0000 7f1db439e740 20 client.17505 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:19.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.690+0000 7f1db439e740 1 --2- 172.21.15.43:0/3324598558 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55c78bb82ed0 0x55c78bbeb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.690+0000 7f1db439e740 4 client.17505 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:43:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.690+0000 7f1db439e740 1 -- 172.21.15.43:0/3324598558 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c78bad7c10 con 0x55c78bb82ed0 2024-04-21T08:43:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.691+0000 7f1daa7fc640 1 --2- 172.21.15.43:0/3324598558 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55c78bb82ed0 0x55c78bbeb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.691+0000 7f1d8eff5640 10 client.17505 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:43:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.691+0000 7f1daa7fc640 1 --2- 172.21.15.43:0/3324598558 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55c78bb82ed0 0x55c78bbeb980 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ca2ad4f7db5fd78d server_cookie=122953cfe5d5fe68 in_seq=0 out_seq=0 2024-04-21T08:43:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.691+0000 7f1d8eff5640 1 -- 172.21.15.43:0/3324598558 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c78bad7c10 con 0x55c78bb82ed0 2024-04-21T08:43:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.691+0000 7f1d8eff5640 10 client.17505 handle_command_reply: tid=0 2024-04-21T08:43:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.691+0000 7f1db439e740 1 client.17505 shutdown 2024-04-21T08:43:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.691+0000 7f1db439e740 1 -- 172.21.15.43:0/3324598558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb65d60 msgr2=0x55c78bb83b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.691+0000 7f1db439e740 1 --2- 172.21.15.43:0/3324598558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb65d60 0x55c78bb83b30 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c04d000 tx=0x7f1d9c032e70 comp rx=0 tx=0).stop 2024-04-21T08:43:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.691+0000 7f1db439e740 1 -- 172.21.15.43:0/3324598558 shutdown_connections 2024-04-21T08:43:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.691+0000 7f1db439e740 1 --2- 172.21.15.43:0/3324598558 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55c78bb82ed0 0x55c78bbeb980 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.691+0000 7f1db439e740 1 --2- 172.21.15.43:0/3324598558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c78bb6ee60 0x55c78bb84140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.691+0000 7f1db439e740 1 --2- 172.21.15.43:0/3324598558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c78bb65d60 0x55c78bb83b30 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.691+0000 7f1db439e740 1 --2- 172.21.15.43:0/3324598558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c78bb653b0 0x55c78bb887c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.691+0000 7f1d8eff5640 0 client.17505 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:43:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.691+0000 7f1db439e740 1 -- 172.21.15.43:0/3324598558 >> 172.21.15.43:0/3324598558 conn(0x55c78bb5fba0 msgr2=0x55c78bb5ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.691+0000 7f1db439e740 1 -- 172.21.15.43:0/3324598558 shutdown_connections 2024-04-21T08:43:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.691+0000 7f1db439e740 1 -- 172.21.15.43:0/3324598558 wait complete. 2024-04-21T08:43:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.692+0000 7f1db439e740 20 client.17505 trim_cache size 0 max 16384 2024-04-21T08:43:19.697 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:43:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.692+0000 7f1db16d2640 1 -- 172.21.15.43:0/2785485943 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1d940a3da0 msgr2=0x7f1d940a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.692+0000 7f1db16d2640 1 --2- 172.21.15.43:0/2785485943 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1d940a3da0 0x7f1d940a6260 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1da409bc70 tx=0x7f1da4009f90 comp rx=0 tx=0).stop 2024-04-21T08:43:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.692+0000 7f1db16d2640 1 -- 172.21.15.43:0/2785485943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dac198670 msgr2=0x7f1dac198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.692+0000 7f1db16d2640 1 --2- 172.21.15.43:0/2785485943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dac198670 0x7f1dac198af0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f1da400b410 tx=0x7f1da400b8d0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.692+0000 7f1db16d2640 1 -- 172.21.15.43:0/2785485943 shutdown_connections 2024-04-21T08:43:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.692+0000 7f1db16d2640 1 --2- 172.21.15.43:0/2785485943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dac198670 0x7f1dac198af0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.692+0000 7f1db16d2640 1 --2- 172.21.15.43:0/2785485943 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1d940a3da0 0x7f1d940a6260 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.692+0000 7f1db16d2640 1 --2- 172.21.15.43:0/2785485943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dac107290 0x7f1dac19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.692+0000 7f1db16d2640 1 --2- 172.21.15.43:0/2785485943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dac1021c0 0x7f1dac19e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.692+0000 7f1db16d2640 1 -- 172.21.15.43:0/2785485943 >> 172.21.15.43:0/2785485943 conn(0x7f1dac0793e0 msgr2=0x7f1dac108460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.692+0000 7f1db16d2640 1 -- 172.21.15.43:0/2785485943 shutdown_connections 2024-04-21T08:43:19.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.692+0000 7f1db16d2640 1 -- 172.21.15.43:0/2785485943 wait complete. 2024-04-21T08:43:19.702 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:43:19.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.807+0000 7fa97694b640 1 -- 172.21.15.43:0/125913867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa970101a50 msgr2=0x7fa970105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.807+0000 7fa97694b640 1 --2- 172.21.15.43:0/125913867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa970101a50 0x7fa970105a90 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa964009900 tx=0x7fa96402bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.807+0000 7fa97694b640 1 -- 172.21.15.43:0/125913867 shutdown_connections 2024-04-21T08:43:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.807+0000 7fa97694b640 1 --2- 172.21.15.43:0/125913867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa970106170 0x7fa97010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.807+0000 7fa97694b640 1 --2- 172.21.15.43:0/125913867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa970101a50 0x7fa970105a90 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.807+0000 7fa97694b640 1 --2- 172.21.15.43:0/125913867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9701010a0 0x7fa970101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.807+0000 7fa97694b640 1 -- 172.21.15.43:0/125913867 >> 172.21.15.43:0/125913867 conn(0x7fa970077eb0 msgr2=0x7fa9700782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.807+0000 7fa97694b640 1 -- 172.21.15.43:0/125913867 shutdown_connections 2024-04-21T08:43:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.807+0000 7fa97694b640 1 -- 172.21.15.43:0/125913867 wait complete. 2024-04-21T08:43:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.808+0000 7fa97694b640 1 Processor -- start 2024-04-21T08:43:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.808+0000 7fa97694b640 1 -- start start 2024-04-21T08:43:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.808+0000 7fa97694b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9701010a0 0x7fa97019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.808+0000 7fa97694b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa970106170 0x7fa97019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.808+0000 7fa97694b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa970198590 0x7fa970198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.808+0000 7fa97694b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa970111100 con 0x7fa970106170 2024-04-21T08:43:19.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.808+0000 7fa97694b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa970110f80 con 0x7fa9701010a0 2024-04-21T08:43:19.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.808+0000 7fa97694b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa970111280 con 0x7fa970198590 2024-04-21T08:43:19.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.808+0000 7fa96f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa970106170 0x7fa97019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.808+0000 7fa96f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa970106170 0x7fa97019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57788/0 (socket says 172.21.15.43:57788) 2024-04-21T08:43:19.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.808+0000 7fa96f7fe640 1 -- 172.21.15.43:0/473495189 learned_addr learned my addr 172.21.15.43:0/473495189 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:19.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.808+0000 7fa96ffff640 1 --2- 172.21.15.43:0/473495189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9701010a0 0x7fa97019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.809+0000 7fa96f7fe640 1 -- 172.21.15.43:0/473495189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa970198590 msgr2=0x7fa970198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.809+0000 7fa96f7fe640 1 --2- 172.21.15.43:0/473495189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa970198590 0x7fa970198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.809+0000 7fa96f7fe640 1 -- 172.21.15.43:0/473495189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9701010a0 msgr2=0x7fa97019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.809+0000 7fa96f7fe640 1 --2- 172.21.15.43:0/473495189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9701010a0 0x7fa97019e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.809+0000 7fa96f7fe640 1 -- 172.21.15.43:0/473495189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9701990b0 con 0x7fa970106170 2024-04-21T08:43:19.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.809+0000 7fa96ffff640 1 --2- 172.21.15.43:0/473495189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9701010a0 0x7fa97019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:19.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.809+0000 7fa96f7fe640 1 --2- 172.21.15.43:0/473495189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa970106170 0x7fa97019e920 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fa964000c00 tx=0x7fa96402c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.809+0000 7fa96d7fa640 1 -- 172.21.15.43:0/473495189 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964043020 con 0x7fa970106170 2024-04-21T08:43:19.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.809+0000 7fa97694b640 1 -- 172.21.15.43:0/473495189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa970199340 con 0x7fa970106170 2024-04-21T08:43:19.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.809+0000 7fa96d7fa640 1 -- 172.21.15.43:0/473495189 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa96402cc60 con 0x7fa970106170 2024-04-21T08:43:19.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.809+0000 7fa96d7fa640 1 -- 172.21.15.43:0/473495189 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964003ca0 con 0x7fa970106170 2024-04-21T08:43:19.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.809+0000 7fa97694b640 1 -- 172.21.15.43:0/473495189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9701a5590 con 0x7fa970106170 2024-04-21T08:43:19.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.811+0000 7fa96d7fa640 1 -- 172.21.15.43:0/473495189 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa964046020 con 0x7fa970106170 2024-04-21T08:43:19.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.812+0000 7fa979614740 1 Processor -- start 2024-04-21T08:43:19.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.812+0000 7fa979614740 1 -- start start 2024-04-21T08:43:19.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.812+0000 7fa979614740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395324f250 0x56395324f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.812+0000 7fa979614740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56395324fc00 0x56395325c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.812+0000 7fa979614740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56395325ce10 0x56395325f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.812+0000 7fa979614740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5639531b41f0 con 0x56395324f250 2024-04-21T08:43:19.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.812+0000 7fa979614740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5639531514b0 con 0x56395324fc00 2024-04-21T08:43:19.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.812+0000 7fa979614740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5639531b01b0 con 0x56395325ce10 2024-04-21T08:43:19.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.812+0000 7fa974ee8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395324f250 0x56395324f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.812+0000 7fa96d7fa640 1 --2- 172.21.15.43:0/473495189 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa9580a3d20 0x7fa9580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.812+0000 7fa974ee8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395324f250 0x56395324f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57796/0 (socket says 172.21.15.43:57796) 2024-04-21T08:43:19.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.812+0000 7fa974ee8640 1 -- 172.21.15.43:0/381408697 learned_addr learned my addr 172.21.15.43:0/381408697 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:19.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.812+0000 7fa96ffff640 1 --2- 172.21.15.43:0/381408697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56395324fc00 0x56395325c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.812+0000 7fa974ee8640 1 --2- 172.21.15.43:0/473495189 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa9580a3d20 0x7fa9580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.812+0000 7fa96d7fa640 1 -- 172.21.15.43:0/473495189 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fa9640f2d90 con 0x7fa970106170 2024-04-21T08:43:19.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.812+0000 7fa96f7fe640 1 --2- 172.21.15.43:0/381408697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56395325ce10 0x56395325f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.813+0000 7fa974ee8640 1 -- 172.21.15.43:0/381408697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56395325ce10 msgr2=0x56395325f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.813+0000 7fa974ee8640 1 --2- 172.21.15.43:0/381408697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56395325ce10 0x56395325f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.813+0000 7fa974ee8640 1 -- 172.21.15.43:0/381408697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56395324fc00 msgr2=0x56395325c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.813+0000 7fa974ee8640 1 --2- 172.21.15.43:0/381408697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56395324fc00 0x56395325c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.813+0000 7fa974ee8640 1 -- 172.21.15.43:0/381408697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639531431b0 con 0x56395324f250 2024-04-21T08:43:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.813+0000 7fa96ffff640 1 --2- 172.21.15.43:0/381408697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56395324fc00 0x56395325c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.813+0000 7fa974ee8640 1 --2- 172.21.15.43:0/381408697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395324f250 0x56395324f630 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fa968015890 tx=0x7fa968015d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2002830a09c8a306 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.813+0000 7fa9567fc640 1 -- 172.21.15.43:0/381408697 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968022070 con 0x56395324f250 2024-04-21T08:43:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.813+0000 7fa9567fc640 1 -- 172.21.15.43:0/381408697 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa968009e40 con 0x56395324f250 2024-04-21T08:43:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.813+0000 7fa9567fc640 1 -- 172.21.15.43:0/381408697 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968010c60 con 0x56395324f250 2024-04-21T08:43:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.814+0000 7fa974ee8640 1 --2- 172.21.15.43:0/473495189 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa9580a3d20 0x7fa9580a61e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa96801f000 tx=0x7fa96801b000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.814+0000 7fa979614740 1 -- 172.21.15.43:0/381408697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395324f250 msgr2=0x56395324f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.814+0000 7fa979614740 1 --2- 172.21.15.43:0/381408697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395324f250 0x56395324f630 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fa968015890 tx=0x7fa968015d50 comp rx=0 tx=0).stop 2024-04-21T08:43:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.814+0000 7fa979614740 1 -- 172.21.15.43:0/381408697 shutdown_connections 2024-04-21T08:43:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.814+0000 7fa979614740 1 --2- 172.21.15.43:0/381408697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56395325ce10 0x56395325f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.814+0000 7fa979614740 1 --2- 172.21.15.43:0/381408697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56395324fc00 0x56395325c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.814+0000 7fa979614740 1 --2- 172.21.15.43:0/381408697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395324f250 0x56395324f630 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.814+0000 7fa979614740 1 -- 172.21.15.43:0/381408697 >> 172.21.15.43:0/381408697 conn(0x563953249ba0 msgr2=0x563953249f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.814+0000 7fa979614740 1 -- 172.21.15.43:0/381408697 shutdown_connections 2024-04-21T08:43:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.814+0000 7fa979614740 1 -- 172.21.15.43:0/381408697 wait complete. 2024-04-21T08:43:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.814+0000 7fa979614740 1 -- start start 2024-04-21T08:43:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.814+0000 7fa979614740 1 Processor -- start 2024-04-21T08:43:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.814+0000 7fa979614740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56395324f250 0x56395325c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.815+0000 7fa979614740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56395324fc00 0x563953259ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.815+0000 7fa979614740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395325ce10 0x56395325a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.815+0000 7fa979614740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56395325fd40 con 0x56395325ce10 2024-04-21T08:43:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.815+0000 7fa979614740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56395316d500 con 0x56395324f250 2024-04-21T08:43:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.815+0000 7fa979614740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56395325fec0 con 0x56395324fc00 2024-04-21T08:43:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.815+0000 7fa979614740 10 client.0 _ll_register_callbacks cb 0x56395313bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.815+0000 7fa974ee8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395325ce10 0x56395325a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.815+0000 7fa974ee8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395325ce10 0x56395325a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57804/0 (socket says 172.21.15.43:57804) 2024-04-21T08:43:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.815+0000 7fa974ee8640 1 -- 172.21.15.43:0/1875745709 learned_addr learned my addr 172.21.15.43:0/1875745709 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.815+0000 7fa96ffff640 1 --2- 172.21.15.43:0/1875745709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56395324f250 0x56395325c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.815+0000 7fa96f7fe640 1 --2- 172.21.15.43:0/1875745709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56395324fc00 0x563953259ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.815+0000 7fa974ee8640 1 -- 172.21.15.43:0/1875745709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56395324fc00 msgr2=0x563953259ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.815+0000 7fa974ee8640 1 --2- 172.21.15.43:0/1875745709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56395324fc00 0x563953259ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.815+0000 7fa974ee8640 1 -- 172.21.15.43:0/1875745709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56395324f250 msgr2=0x56395325c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.815+0000 7fa974ee8640 1 --2- 172.21.15.43:0/1875745709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56395324f250 0x56395325c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.815+0000 7fa974ee8640 1 -- 172.21.15.43:0/1875745709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563952ec63f0 con 0x56395325ce10 2024-04-21T08:43:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.815+0000 7fa974ee8640 1 --2- 172.21.15.43:0/1875745709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395325ce10 0x56395325a030 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fa968010800 tx=0x7fa968010630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c4f06deff624db server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.815+0000 7fa94f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.815+0000 7fa94f7fe640 1 -- 172.21.15.43:0/1875745709 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968035060 con 0x56395325ce10 2024-04-21T08:43:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.815+0000 7fa94f7fe640 1 -- 172.21.15.43:0/1875745709 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fa968015040 con 0x56395325ce10 2024-04-21T08:43:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.815+0000 7fa979614740 1 -- 172.21.15.43:0/1875745709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563953153930 con 0x56395325ce10 2024-04-21T08:43:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.815+0000 7fa94f7fe640 1 -- 172.21.15.43:0/1875745709 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa96803c050 con 0x56395325ce10 2024-04-21T08:43:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.815+0000 7fa979614740 10 client.17517 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.815+0000 7fa979614740 1 -- 172.21.15.43:0/1875745709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563952f907d0 con 0x56395325ce10 2024-04-21T08:43:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.816+0000 7fa94f7fe640 1 -- 172.21.15.43:0/1875745709 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563953153930 con 0x56395325ce10 2024-04-21T08:43:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.816+0000 7fa94f7fe640 1 -- 172.21.15.43:0/1875745709 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa968030440 con 0x56395325ce10 2024-04-21T08:43:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.816+0000 7fa979614740 10 client.17517 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.816+0000 7fa979614740 10 client.17517 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:43:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.816+0000 7fa979614740 20 client.17517 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.816+0000 7fa979614740 1 --2- 172.21.15.43:0/1875745709 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56395326ce90 0x56395326f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.816+0000 7fa979614740 4 client.17517 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.816+0000 7fa979614740 1 -- 172.21.15.43:0/1875745709 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639531c1c10 con 0x56395326ce90 2024-04-21T08:43:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.816+0000 7fa96ffff640 1 --2- 172.21.15.43:0/1875745709 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56395326ce90 0x56395326f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.817+0000 7fa94f7fe640 10 client.17517 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:43:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.817+0000 7fa96ffff640 1 --2- 172.21.15.43:0/1875745709 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56395326ce90 0x56395326f280 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e104950ee39d3aa3 server_cookie=a38cddb5dc2eabea in_seq=0 out_seq=0 2024-04-21T08:43:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.817+0000 7fa94f7fe640 1 -- 172.21.15.43:0/1875745709 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5639531c1c10 con 0x56395326ce90 2024-04-21T08:43:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.817+0000 7fa94f7fe640 10 client.17517 handle_command_reply: tid=0 2024-04-21T08:43:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.817+0000 7fa979614740 1 client.17517 shutdown 2024-04-21T08:43:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.817+0000 7fa979614740 1 -- 172.21.15.43:0/1875745709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395325ce10 msgr2=0x56395325a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.817+0000 7fa979614740 1 --2- 172.21.15.43:0/1875745709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395325ce10 0x56395325a030 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fa968010800 tx=0x7fa968010630 comp rx=0 tx=0).stop 2024-04-21T08:43:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.817+0000 7fa979614740 1 -- 172.21.15.43:0/1875745709 shutdown_connections 2024-04-21T08:43:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.817+0000 7fa979614740 1 --2- 172.21.15.43:0/1875745709 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56395326ce90 0x56395326f280 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.818+0000 7fa979614740 1 --2- 172.21.15.43:0/1875745709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395325ce10 0x56395325a030 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.818+0000 7fa94f7fe640 0 client.17517 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:43:19.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.818+0000 7fa979614740 1 --2- 172.21.15.43:0/1875745709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56395324fc00 0x563953259ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.818+0000 7fa979614740 1 --2- 172.21.15.43:0/1875745709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56395324f250 0x56395325c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.818+0000 7fa979614740 1 -- 172.21.15.43:0/1875745709 >> 172.21.15.43:0/1875745709 conn(0x563953249ba0 msgr2=0x56395324b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.818+0000 7fa979614740 1 -- 172.21.15.43:0/1875745709 shutdown_connections 2024-04-21T08:43:19.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.818+0000 7fa979614740 1 -- 172.21.15.43:0/1875745709 wait complete. 2024-04-21T08:43:19.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.818+0000 7fa979614740 20 client.17517 trim_cache size 0 max 16384 2024-04-21T08:43:19.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.831+0000 7fa979614740 1 Processor -- start 2024-04-21T08:43:19.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.831+0000 7fa979614740 1 -- start start 2024-04-21T08:43:19.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.831+0000 7fa979614740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56395324f250 0x56395325a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.831+0000 7fa979614740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395324fc00 0x563953268c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.831+0000 7fa979614740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56395325ce10 0x563953274a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.831+0000 7fa979614740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa9701ad220 con 0x56395324fc00 2024-04-21T08:43:19.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.831+0000 7fa979614740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9701aed60 con 0x56395325ce10 2024-04-21T08:43:19.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.831+0000 7fa979614740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9701ad8f0 con 0x56395324f250 2024-04-21T08:43:19.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.831+0000 7fa96f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395324fc00 0x563953268c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.831+0000 7fa96f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395324fc00 0x563953268c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57816/0 (socket says 172.21.15.43:57816) 2024-04-21T08:43:19.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.831+0000 7fa96f7fe640 1 -- 172.21.15.43:0/3254852452 learned_addr learned my addr 172.21.15.43:0/3254852452 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:19.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.831+0000 7fa96f7fe640 1 -- 172.21.15.43:0/3254852452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56395324f250 msgr2=0x56395325a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.831+0000 7fa96f7fe640 1 --2- 172.21.15.43:0/3254852452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56395324f250 0x56395325a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.831+0000 7fa96f7fe640 1 -- 172.21.15.43:0/3254852452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56395325ce10 msgr2=0x563953274a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.831+0000 7fa974ee8640 1 --2- 172.21.15.43:0/3254852452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56395325ce10 0x563953274a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.831+0000 7fa96f7fe640 1 --2- 172.21.15.43:0/3254852452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56395325ce10 0x563953274a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.832+0000 7fa96f7fe640 1 -- 172.21.15.43:0/3254852452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56395324e560 con 0x56395324fc00 2024-04-21T08:43:19.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.832+0000 7fa96f7fe640 1 --2- 172.21.15.43:0/3254852452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395324fc00 0x563953268c40 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa9640be770 tx=0x7fa9640a9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c90897eb840500 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.832+0000 7fa955ffb640 1 -- 172.21.15.43:0/3254852452 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964003e40 con 0x56395324fc00 2024-04-21T08:43:19.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.832+0000 7fa955ffb640 1 -- 172.21.15.43:0/3254852452 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9640035f0 con 0x56395324fc00 2024-04-21T08:43:19.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.832+0000 7fa955ffb640 1 -- 172.21.15.43:0/3254852452 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9640bea70 con 0x56395324fc00 2024-04-21T08:43:19.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.832+0000 7fa979614740 1 -- 172.21.15.43:0/3254852452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395324fc00 msgr2=0x563953268c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.832+0000 7fa979614740 1 --2- 172.21.15.43:0/3254852452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395324fc00 0x563953268c40 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa9640be770 tx=0x7fa9640a9040 comp rx=0 tx=0).stop 2024-04-21T08:43:19.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.832+0000 7fa979614740 1 -- 172.21.15.43:0/3254852452 shutdown_connections 2024-04-21T08:43:19.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.832+0000 7fa979614740 1 --2- 172.21.15.43:0/3254852452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56395325ce10 0x563953274a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.832+0000 7fa979614740 1 --2- 172.21.15.43:0/3254852452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395324fc00 0x563953268c40 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.832+0000 7fa979614740 1 --2- 172.21.15.43:0/3254852452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56395324f250 0x56395325a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.832+0000 7fa979614740 1 -- 172.21.15.43:0/3254852452 >> 172.21.15.43:0/3254852452 conn(0x563953249ba0 msgr2=0x563953255070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.832+0000 7fa979614740 1 -- 172.21.15.43:0/3254852452 shutdown_connections 2024-04-21T08:43:19.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.832+0000 7fa979614740 1 -- 172.21.15.43:0/3254852452 wait complete. 2024-04-21T08:43:19.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.833+0000 7fa979614740 1 -- start start 2024-04-21T08:43:19.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.833+0000 7fa979614740 1 Processor -- start 2024-04-21T08:43:19.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.833+0000 7fa979614740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56395324f250 0x5639532603a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.833+0000 7fa979614740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56395324fc00 0x5639532608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.833+0000 7fa979614740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395325ce10 0x5639532cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.833+0000 7fa979614740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa9701ae690 con 0x56395325ce10 2024-04-21T08:43:19.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.833+0000 7fa979614740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9701adfc0 con 0x56395324fc00 2024-04-21T08:43:19.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.833+0000 7fa979614740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9701ab200 con 0x56395324f250 2024-04-21T08:43:19.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.833+0000 7fa979614740 10 client.0 _ll_register_callbacks cb 0x7fa968005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:19.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.833+0000 7fa974ee8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395325ce10 0x5639532cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.833+0000 7fa974ee8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395325ce10 0x5639532cb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57830/0 (socket says 172.21.15.43:57830) 2024-04-21T08:43:19.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.833+0000 7fa974ee8640 1 -- 172.21.15.43:0/3582717062 learned_addr learned my addr 172.21.15.43:0/3582717062 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:19.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.834+0000 7fa96f7fe640 1 --2- 172.21.15.43:0/3582717062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56395324fc00 0x5639532608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.834+0000 7fa974ee8640 1 -- 172.21.15.43:0/3582717062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56395324f250 msgr2=0x5639532603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.834+0000 7fa974ee8640 1 --2- 172.21.15.43:0/3582717062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56395324f250 0x5639532603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.834+0000 7fa974ee8640 1 -- 172.21.15.43:0/3582717062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56395324fc00 msgr2=0x5639532608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.834+0000 7fa974ee8640 1 --2- 172.21.15.43:0/3582717062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56395324fc00 0x5639532608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.834+0000 7fa974ee8640 1 -- 172.21.15.43:0/3582717062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639532cb880 con 0x56395325ce10 2024-04-21T08:43:19.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.834+0000 7fa96f7fe640 1 --2- 172.21.15.43:0/3582717062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56395324fc00 0x5639532608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:19.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.834+0000 7fa974ee8640 1 --2- 172.21.15.43:0/3582717062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395325ce10 0x5639532cb2c0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fa9680308b0 tx=0x7fa968009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d577d29408fb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.834+0000 7fa94f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:19.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.834+0000 7fa94f7fe640 1 -- 172.21.15.43:0/3582717062 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968045060 con 0x56395325ce10 2024-04-21T08:43:19.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.834+0000 7fa94f7fe640 1 -- 172.21.15.43:0/3582717062 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fa968030440 con 0x56395325ce10 2024-04-21T08:43:19.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.834+0000 7fa979614740 1 -- 172.21.15.43:0/3582717062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5639531bba80 con 0x56395325ce10 2024-04-21T08:43:19.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.834+0000 7fa94f7fe640 1 -- 172.21.15.43:0/3582717062 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa968035030 con 0x56395325ce10 2024-04-21T08:43:19.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.834+0000 7fa979614740 10 client.17523 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:19.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.834+0000 7fa979614740 1 -- 172.21.15.43:0/3582717062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5639532d2c30 con 0x56395325ce10 2024-04-21T08:43:19.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.834+0000 7fa94f7fe640 1 -- 172.21.15.43:0/3582717062 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5639531bba80 con 0x56395325ce10 2024-04-21T08:43:19.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.834+0000 7fa94f7fe640 1 -- 172.21.15.43:0/3582717062 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa96801d800 con 0x56395325ce10 2024-04-21T08:43:19.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.835+0000 7fa979614740 10 client.17523 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:19.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.835+0000 7fa979614740 10 client.17523 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:43:19.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.835+0000 7fa979614740 20 client.17523 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:19.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.835+0000 7fa979614740 1 --2- 172.21.15.43:0/3582717062 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5639532d3290 0x5639532d56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.835+0000 7fa979614740 4 client.17523 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:43:19.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.835+0000 7fa979614740 1 -- 172.21.15.43:0/3582717062 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5639531376e0 con 0x5639532d3290 2024-04-21T08:43:19.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.835+0000 7fa96ffff640 1 --2- 172.21.15.43:0/3582717062 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5639532d3290 0x5639532d56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.835+0000 7fa94f7fe640 10 client.17523 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:43:19.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.835+0000 7fa96ffff640 1 --2- 172.21.15.43:0/3582717062 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5639532d3290 0x5639532d56a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8c5e2228beb2f4bb server_cookie=d28f6ac24119cc32 in_seq=0 out_seq=0 2024-04-21T08:43:19.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.835+0000 7fa94f7fe640 1 -- 172.21.15.43:0/3582717062 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5639531376e0 con 0x5639532d3290 2024-04-21T08:43:19.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.835+0000 7fa94f7fe640 10 client.17523 handle_command_reply: tid=0 2024-04-21T08:43:19.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.835+0000 7fa979614740 1 client.17523 shutdown 2024-04-21T08:43:19.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.836+0000 7fa979614740 1 -- 172.21.15.43:0/3582717062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395325ce10 msgr2=0x5639532cb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.836+0000 7fa979614740 1 --2- 172.21.15.43:0/3582717062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395325ce10 0x5639532cb2c0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fa9680308b0 tx=0x7fa968009570 comp rx=0 tx=0).stop 2024-04-21T08:43:19.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.836+0000 7fa979614740 1 -- 172.21.15.43:0/3582717062 shutdown_connections 2024-04-21T08:43:19.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.836+0000 7fa979614740 1 --2- 172.21.15.43:0/3582717062 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5639532d3290 0x5639532d56a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.836+0000 7fa979614740 1 --2- 172.21.15.43:0/3582717062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56395325ce10 0x5639532cb2c0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.836+0000 7fa979614740 1 --2- 172.21.15.43:0/3582717062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56395324fc00 0x5639532608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.836+0000 7fa979614740 1 --2- 172.21.15.43:0/3582717062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56395324f250 0x5639532603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.836+0000 7fa94f7fe640 0 client.17523 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:43:19.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.836+0000 7fa979614740 1 -- 172.21.15.43:0/3582717062 >> 172.21.15.43:0/3582717062 conn(0x563953249ba0 msgr2=0x56395325b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.836+0000 7fa979614740 1 -- 172.21.15.43:0/3582717062 shutdown_connections 2024-04-21T08:43:19.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.836+0000 7fa979614740 1 -- 172.21.15.43:0/3582717062 wait complete. 2024-04-21T08:43:19.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.836+0000 7fa979614740 20 client.17523 trim_cache size 0 max 16384 2024-04-21T08:43:19.842 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:43:19.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.836+0000 7fa97694b640 1 -- 172.21.15.43:0/473495189 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa9580a3d20 msgr2=0x7fa9580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.836+0000 7fa97694b640 1 --2- 172.21.15.43:0/473495189 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa9580a3d20 0x7fa9580a61e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa96801f000 tx=0x7fa96801b000 comp rx=0 tx=0).stop 2024-04-21T08:43:19.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.836+0000 7fa97694b640 1 -- 172.21.15.43:0/473495189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa970106170 msgr2=0x7fa97019e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.836+0000 7fa97694b640 1 --2- 172.21.15.43:0/473495189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa970106170 0x7fa97019e920 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fa964000c00 tx=0x7fa96402c860 comp rx=0 tx=0).stop 2024-04-21T08:43:19.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.836+0000 7fa97694b640 1 -- 172.21.15.43:0/473495189 shutdown_connections 2024-04-21T08:43:19.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.836+0000 7fa97694b640 1 --2- 172.21.15.43:0/473495189 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa9580a3d20 0x7fa9580a61e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.836+0000 7fa97694b640 1 --2- 172.21.15.43:0/473495189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa970198590 0x7fa970198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.836+0000 7fa97694b640 1 --2- 172.21.15.43:0/473495189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa970106170 0x7fa97019e920 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.836+0000 7fa97694b640 1 --2- 172.21.15.43:0/473495189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9701010a0 0x7fa97019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.836+0000 7fa97694b640 1 -- 172.21.15.43:0/473495189 >> 172.21.15.43:0/473495189 conn(0x7fa970077eb0 msgr2=0x7fa970104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.836+0000 7fa97694b640 1 -- 172.21.15.43:0/473495189 shutdown_connections 2024-04-21T08:43:19.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.836+0000 7fa97694b640 1 -- 172.21.15.43:0/473495189 wait complete. 2024-04-21T08:43:19.848 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:43:19.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.953+0000 7f8343104640 1 -- 172.21.15.43:0/1974210064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f833c107260 msgr2=0x7f833c10ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.953+0000 7f8343104640 1 --2- 172.21.15.43:0/1974210064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f833c107260 0x7f833c10ee30 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f8334009ca0 tx=0x7f833402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.953+0000 7f8343104640 1 -- 172.21.15.43:0/1974210064 shutdown_connections 2024-04-21T08:43:19.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.953+0000 7f8343104640 1 --2- 172.21.15.43:0/1974210064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f833c107260 0x7f833c10ee30 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.953+0000 7f8343104640 1 --2- 172.21.15.43:0/1974210064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f833c102b40 0x7f833c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.953+0000 7f8343104640 1 --2- 172.21.15.43:0/1974210064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f833c102190 0x7f833c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.953+0000 7f8343104640 1 -- 172.21.15.43:0/1974210064 >> 172.21.15.43:0/1974210064 conn(0x7f833c0fd990 msgr2=0x7f833c0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.953+0000 7f8343104640 1 -- 172.21.15.43:0/1974210064 shutdown_connections 2024-04-21T08:43:19.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.953+0000 7f8343104640 1 -- 172.21.15.43:0/1974210064 wait complete. 2024-04-21T08:43:19.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.953+0000 7f8343104640 1 Processor -- start 2024-04-21T08:43:19.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.953+0000 7f8343104640 1 -- start start 2024-04-21T08:43:19.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.954+0000 7f8343104640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f833c102190 0x7f833c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.954+0000 7f8343104640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f833c107260 0x7f833c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.954+0000 7f8343104640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f833c19cff0 0x7f833c19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.954+0000 7f8343104640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f833c1121d0 con 0x7f833c102190 2024-04-21T08:43:19.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.954+0000 7f8343104640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f833c112050 con 0x7f833c19cff0 2024-04-21T08:43:19.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.954+0000 7f8343104640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f833c112350 con 0x7f833c107260 2024-04-21T08:43:19.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.954+0000 7f8340ea0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f833c102190 0x7f833c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.954+0000 7f8340ea0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f833c102190 0x7f833c198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57854/0 (socket says 172.21.15.43:57854) 2024-04-21T08:43:19.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.954+0000 7f8340ea0640 1 -- 172.21.15.43:0/1956191517 learned_addr learned my addr 172.21.15.43:0/1956191517 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:19.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.954+0000 7f833bfff640 1 --2- 172.21.15.43:0/1956191517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f833c107260 0x7f833c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.954+0000 7f83416a1640 1 --2- 172.21.15.43:0/1956191517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f833c19cff0 0x7f833c19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.954+0000 7f8340ea0640 1 -- 172.21.15.43:0/1956191517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f833c107260 msgr2=0x7f833c198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.954+0000 7f8340ea0640 1 --2- 172.21.15.43:0/1956191517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f833c107260 0x7f833c198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.954+0000 7f8340ea0640 1 -- 172.21.15.43:0/1956191517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f833c19cff0 msgr2=0x7f833c19d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.954+0000 7f8340ea0640 1 --2- 172.21.15.43:0/1956191517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f833c19cff0 0x7f833c19d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.954+0000 7f8340ea0640 1 -- 172.21.15.43:0/1956191517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833c19dae0 con 0x7f833c102190 2024-04-21T08:43:19.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.954+0000 7f8340ea0640 1 --2- 172.21.15.43:0/1956191517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f833c102190 0x7f833c198730 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f833000e9f0 tx=0x7f833000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.954+0000 7f8339ffb640 1 -- 172.21.15.43:0/1956191517 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833000cd80 con 0x7f833c102190 2024-04-21T08:43:19.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.954+0000 7f8339ffb640 1 -- 172.21.15.43:0/1956191517 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83300054d0 con 0x7f833c102190 2024-04-21T08:43:19.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.954+0000 7f8343104640 1 -- 172.21.15.43:0/1956191517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f833c19ddd0 con 0x7f833c102190 2024-04-21T08:43:19.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.954+0000 7f8339ffb640 1 -- 172.21.15.43:0/1956191517 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8330010690 con 0x7f833c102190 2024-04-21T08:43:19.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.955+0000 7f8343104640 1 -- 172.21.15.43:0/1956191517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f833c1a5570 con 0x7f833c102190 2024-04-21T08:43:19.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.957+0000 7f8339ffb640 1 -- 172.21.15.43:0/1956191517 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8330004e90 con 0x7f833c102190 2024-04-21T08:43:19.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.957+0000 7f8345dd1740 1 Processor -- start 2024-04-21T08:43:19.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.957+0000 7f8339ffb640 1 --2- 172.21.15.43:0/1956191517 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f832c0a3d20 0x7f832c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.958+0000 7f8345dd1740 1 -- start start 2024-04-21T08:43:19.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.958+0000 7f8339ffb640 1 -- 172.21.15.43:0/1956191517 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f83300d1130 con 0x7f833c102190 2024-04-21T08:43:19.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.958+0000 7f83416a1640 1 --2- 172.21.15.43:0/1956191517 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f832c0a3d20 0x7f832c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.958+0000 7f8345dd1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65cc54d0 0x55dc65cc58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.958+0000 7f8345dd1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc65cc5e80 0x55dc65cce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.958+0000 7f8345dd1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc65ccea00 0x55dc65cd0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.958+0000 7f8345dd1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dc65c2a1f0 con 0x55dc65cc54d0 2024-04-21T08:43:19.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.958+0000 7f8345dd1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dc65bc74b0 con 0x55dc65cc5e80 2024-04-21T08:43:19.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.958+0000 7f8345dd1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dc65c261b0 con 0x55dc65ccea00 2024-04-21T08:43:19.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.958+0000 7f8340ea0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65cc54d0 0x55dc65cc58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.958+0000 7f8340ea0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65cc54d0 0x55dc65cc58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57868/0 (socket says 172.21.15.43:57868) 2024-04-21T08:43:19.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.958+0000 7f8340ea0640 1 -- 172.21.15.43:0/3021786670 learned_addr learned my addr 172.21.15.43:0/3021786670 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:19.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.958+0000 7f833bfff640 1 --2- 172.21.15.43:0/3021786670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc65cc5e80 0x55dc65cce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.958+0000 7f83416a1640 1 --2- 172.21.15.43:0/3021786670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc65ccea00 0x55dc65cd0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.958+0000 7f8340ea0640 1 -- 172.21.15.43:0/3021786670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc65ccea00 msgr2=0x55dc65cd0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.958+0000 7f8340ea0640 1 --2- 172.21.15.43:0/3021786670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc65ccea00 0x55dc65cd0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.958+0000 7f8340ea0640 1 -- 172.21.15.43:0/3021786670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc65cc5e80 msgr2=0x55dc65cce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.958+0000 7f83416a1640 1 --2- 172.21.15.43:0/1956191517 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f832c0a3d20 0x7f832c0a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8334005cc0 tx=0x7f8334005c30 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.958+0000 7f8340ea0640 1 --2- 172.21.15.43:0/3021786670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc65cc5e80 0x55dc65cce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.958+0000 7f8340ea0640 1 -- 172.21.15.43:0/3021786670 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc65bb91b0 con 0x55dc65cc54d0 2024-04-21T08:43:19.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.958+0000 7f833bfff640 1 --2- 172.21.15.43:0/3021786670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc65cc5e80 0x55dc65cce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:43:19.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.958+0000 7f8340ea0640 1 --2- 172.21.15.43:0/3021786670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65cc54d0 0x55dc65cc58b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f8330099160 tx=0x7f833009c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224e64f80858555 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.959+0000 7f8326ffd640 1 -- 172.21.15.43:0/3021786670 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83300a10b0 con 0x55dc65cc54d0 2024-04-21T08:43:19.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.959+0000 7f8326ffd640 1 -- 172.21.15.43:0/3021786670 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83300a5970 con 0x55dc65cc54d0 2024-04-21T08:43:19.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.959+0000 7f8326ffd640 1 -- 172.21.15.43:0/3021786670 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8330099370 con 0x55dc65cc54d0 2024-04-21T08:43:19.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.959+0000 7f8345dd1740 1 -- 172.21.15.43:0/3021786670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65cc54d0 msgr2=0x55dc65cc58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.959+0000 7f8345dd1740 1 --2- 172.21.15.43:0/3021786670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65cc54d0 0x55dc65cc58b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f8330099160 tx=0x7f833009c720 comp rx=0 tx=0).stop 2024-04-21T08:43:19.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.959+0000 7f8345dd1740 1 -- 172.21.15.43:0/3021786670 shutdown_connections 2024-04-21T08:43:19.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.959+0000 7f8345dd1740 1 --2- 172.21.15.43:0/3021786670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc65ccea00 0x55dc65cd0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.959+0000 7f8345dd1740 1 --2- 172.21.15.43:0/3021786670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc65cc5e80 0x55dc65cce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.959+0000 7f8345dd1740 1 --2- 172.21.15.43:0/3021786670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65cc54d0 0x55dc65cc58b0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.959+0000 7f8345dd1740 1 -- 172.21.15.43:0/3021786670 >> 172.21.15.43:0/3021786670 conn(0x55dc65cbfba0 msgr2=0x55dc65cbff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.959+0000 7f8345dd1740 1 -- 172.21.15.43:0/3021786670 shutdown_connections 2024-04-21T08:43:19.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.959+0000 7f8345dd1740 1 -- 172.21.15.43:0/3021786670 wait complete. 2024-04-21T08:43:19.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.960+0000 7f8345dd1740 1 -- start start 2024-04-21T08:43:19.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.960+0000 7f8345dd1740 1 Processor -- start 2024-04-21T08:43:19.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.960+0000 7f8345dd1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc65cc54d0 0x55dc65cce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.960+0000 7f8345dd1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc65cc5e80 0x55dc65ccb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.960+0000 7f8345dd1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65ccea00 0x55dc65ccbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.960+0000 7f8345dd1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dc65cd1970 con 0x55dc65ccea00 2024-04-21T08:43:19.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.960+0000 7f8345dd1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dc65be3500 con 0x55dc65cc5e80 2024-04-21T08:43:19.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.960+0000 7f8345dd1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dc65cd1af0 con 0x55dc65cc54d0 2024-04-21T08:43:19.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.960+0000 7f8345dd1740 10 client.0 _ll_register_callbacks cb 0x55dc65bb1b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:19.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.960+0000 7f83416a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65ccea00 0x55dc65ccbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.960+0000 7f83416a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65ccea00 0x55dc65ccbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57874/0 (socket says 172.21.15.43:57874) 2024-04-21T08:43:19.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.960+0000 7f83416a1640 1 -- 172.21.15.43:0/120101711 learned_addr learned my addr 172.21.15.43:0/120101711 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:19.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.960+0000 7f833bfff640 1 --2- 172.21.15.43:0/120101711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc65cc5e80 0x55dc65ccb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.961+0000 7f8340ea0640 1 --2- 172.21.15.43:0/120101711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc65cc54d0 0x55dc65cce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.961+0000 7f83416a1640 1 -- 172.21.15.43:0/120101711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc65cc54d0 msgr2=0x55dc65cce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.961+0000 7f83416a1640 1 --2- 172.21.15.43:0/120101711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc65cc54d0 0x55dc65cce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.961+0000 7f83416a1640 1 -- 172.21.15.43:0/120101711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc65cc5e80 msgr2=0x55dc65ccb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.961+0000 7f83416a1640 1 --2- 172.21.15.43:0/120101711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc65cc5e80 0x55dc65ccb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.961+0000 7f83416a1640 1 -- 172.21.15.43:0/120101711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc6593c3f0 con 0x55dc65ccea00 2024-04-21T08:43:19.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.961+0000 7f83416a1640 1 --2- 172.21.15.43:0/120101711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65ccea00 0x55dc65ccbd30 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f8334037040 tx=0x7f833402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2fe7c84774b9730 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.961+0000 7f831bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.961+0000 7f831bfff640 1 -- 172.21.15.43:0/120101711 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833404f020 con 0x55dc65ccea00 2024-04-21T08:43:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.961+0000 7f831bfff640 1 -- 172.21.15.43:0/120101711 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f8334003d80 con 0x55dc65ccea00 2024-04-21T08:43:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.961+0000 7f8345dd1740 1 -- 172.21.15.43:0/120101711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83300a5970 con 0x55dc65ccea00 2024-04-21T08:43:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.961+0000 7f831bfff640 1 -- 172.21.15.43:0/120101711 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f833405a050 con 0x55dc65ccea00 2024-04-21T08:43:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.961+0000 7f8345dd1740 10 client.17535 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.961+0000 7f8345dd1740 1 -- 172.21.15.43:0/120101711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc65a067d0 con 0x55dc65ccea00 2024-04-21T08:43:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.961+0000 7f831bfff640 1 -- 172.21.15.43:0/120101711 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83300a5970 con 0x55dc65ccea00 2024-04-21T08:43:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.962+0000 7f831bfff640 1 -- 172.21.15.43:0/120101711 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f833403f770 con 0x55dc65ccea00 2024-04-21T08:43:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.962+0000 7f8345dd1740 10 client.17535 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.962+0000 7f8345dd1740 10 client.17535 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:43:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.962+0000 7f8345dd1740 20 client.17535 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.962+0000 7f8345dd1740 1 --2- 172.21.15.43:0/120101711 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55dc65cdece0 0x55dc65ce10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.962+0000 7f8345dd1740 4 client.17535 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.962+0000 7f8345dd1740 1 -- 172.21.15.43:0/120101711 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc65bc9930 con 0x55dc65cdece0 2024-04-21T08:43:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.962+0000 7f8340ea0640 1 --2- 172.21.15.43:0/120101711 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55dc65cdece0 0x55dc65ce10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.962+0000 7f8340ea0640 1 --2- 172.21.15.43:0/120101711 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55dc65cdece0 0x55dc65ce10f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c0d3ba2c157bab00 server_cookie=9bcf6df62f3d581f in_seq=0 out_seq=0 2024-04-21T08:43:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.962+0000 7f831bfff640 10 client.17535 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:43:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.963+0000 7f831bfff640 1 -- 172.21.15.43:0/120101711 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc65bc9930 con 0x55dc65cdece0 2024-04-21T08:43:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.963+0000 7f831bfff640 10 client.17535 handle_command_reply: tid=0 2024-04-21T08:43:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.963+0000 7f8345dd1740 1 client.17535 shutdown 2024-04-21T08:43:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.963+0000 7f8345dd1740 1 -- 172.21.15.43:0/120101711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65ccea00 msgr2=0x55dc65ccbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.963+0000 7f8345dd1740 1 --2- 172.21.15.43:0/120101711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65ccea00 0x55dc65ccbd30 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f8334037040 tx=0x7f833402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.963+0000 7f8345dd1740 1 -- 172.21.15.43:0/120101711 shutdown_connections 2024-04-21T08:43:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.963+0000 7f8345dd1740 1 --2- 172.21.15.43:0/120101711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65ccea00 0x55dc65ccbd30 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.963+0000 7f8345dd1740 1 --2- 172.21.15.43:0/120101711 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55dc65cdece0 0x55dc65ce10f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.964+0000 7f8345dd1740 1 --2- 172.21.15.43:0/120101711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc65cc5e80 0x55dc65ccb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.964+0000 7f8345dd1740 1 --2- 172.21.15.43:0/120101711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc65cc54d0 0x55dc65cce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.964+0000 7f8345dd1740 1 -- 172.21.15.43:0/120101711 >> 172.21.15.43:0/120101711 conn(0x55dc65cbfba0 msgr2=0x55dc65cc18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.964+0000 7f831bfff640 0 client.17535 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:43:19.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.964+0000 7f8345dd1740 1 -- 172.21.15.43:0/120101711 shutdown_connections 2024-04-21T08:43:19.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.964+0000 7f8345dd1740 1 -- 172.21.15.43:0/120101711 wait complete. 2024-04-21T08:43:19.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.964+0000 7f8345dd1740 20 client.17535 trim_cache size 0 max 16384 2024-04-21T08:43:19.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.977+0000 7f8345dd1740 1 Processor -- start 2024-04-21T08:43:19.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.977+0000 7f8345dd1740 1 -- start start 2024-04-21T08:43:19.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.977+0000 7f8345dd1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc65cc54d0 0x55dc65cda830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.977+0000 7f8345dd1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65cc5e80 0x55dc65cbec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.977+0000 7f8345dd1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc65ccea00 0x55dc65ccde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.977+0000 7f8345dd1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f833c1af4a0 con 0x55dc65cc5e80 2024-04-21T08:43:19.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.977+0000 7f8345dd1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f833c1b0fe0 con 0x55dc65ccea00 2024-04-21T08:43:19.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.977+0000 7f8345dd1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f833c1afb70 con 0x55dc65cc54d0 2024-04-21T08:43:19.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.977+0000 7f833bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65cc5e80 0x55dc65cbec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.977+0000 7f833bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65cc5e80 0x55dc65cbec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57882/0 (socket says 172.21.15.43:57882) 2024-04-21T08:43:19.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.977+0000 7f833bfff640 1 -- 172.21.15.43:0/506552453 learned_addr learned my addr 172.21.15.43:0/506552453 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:19.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.977+0000 7f8340ea0640 1 --2- 172.21.15.43:0/506552453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc65cc54d0 0x55dc65cda830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.977+0000 7f83416a1640 1 --2- 172.21.15.43:0/506552453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc65ccea00 0x55dc65ccde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.977+0000 7f833bfff640 1 -- 172.21.15.43:0/506552453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc65cc54d0 msgr2=0x55dc65cda830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.977+0000 7f833bfff640 1 --2- 172.21.15.43:0/506552453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc65cc54d0 0x55dc65cda830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.977+0000 7f833bfff640 1 -- 172.21.15.43:0/506552453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc65ccea00 msgr2=0x55dc65ccde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.977+0000 7f833bfff640 1 --2- 172.21.15.43:0/506552453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc65ccea00 0x55dc65ccde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.977+0000 7f833bfff640 1 -- 172.21.15.43:0/506552453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc65cca6d0 con 0x55dc65cc5e80 2024-04-21T08:43:19.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.978+0000 7f833bfff640 1 --2- 172.21.15.43:0/506552453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65cc5e80 0x55dc65cbec00 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f833c108880 tx=0x7f8328005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af758a14dda56a3d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.978+0000 7f83267fc640 1 -- 172.21.15.43:0/506552453 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832800f440 con 0x55dc65cc5e80 2024-04-21T08:43:19.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.978+0000 7f83267fc640 1 -- 172.21.15.43:0/506552453 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8328021070 con 0x55dc65cc5e80 2024-04-21T08:43:19.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.978+0000 7f83267fc640 1 -- 172.21.15.43:0/506552453 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832801c7c0 con 0x55dc65cc5e80 2024-04-21T08:43:19.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.978+0000 7f8345dd1740 1 -- 172.21.15.43:0/506552453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65cc5e80 msgr2=0x55dc65cbec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.978+0000 7f8345dd1740 1 --2- 172.21.15.43:0/506552453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65cc5e80 0x55dc65cbec00 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f833c108880 tx=0x7f8328005f90 comp rx=0 tx=0).stop 2024-04-21T08:43:19.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.978+0000 7f8345dd1740 1 -- 172.21.15.43:0/506552453 shutdown_connections 2024-04-21T08:43:19.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.978+0000 7f8345dd1740 1 --2- 172.21.15.43:0/506552453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc65ccea00 0x55dc65ccde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.978+0000 7f8345dd1740 1 --2- 172.21.15.43:0/506552453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65cc5e80 0x55dc65cbec00 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.978+0000 7f8345dd1740 1 --2- 172.21.15.43:0/506552453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc65cc54d0 0x55dc65cda830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.978+0000 7f8345dd1740 1 -- 172.21.15.43:0/506552453 >> 172.21.15.43:0/506552453 conn(0x55dc65cbfba0 msgr2=0x55dc65cbff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.978+0000 7f8345dd1740 1 -- 172.21.15.43:0/506552453 shutdown_connections 2024-04-21T08:43:19.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.978+0000 7f8345dd1740 1 -- 172.21.15.43:0/506552453 wait complete. 2024-04-21T08:43:19.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.979+0000 7f8345dd1740 1 -- start start 2024-04-21T08:43:19.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.979+0000 7f8345dd1740 1 Processor -- start 2024-04-21T08:43:19.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.979+0000 7f8345dd1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc65cc54d0 0x55dc65cc85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.979+0000 7f8345dd1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc65cc5e80 0x55dc65cc8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.979+0000 7f8345dd1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65ccea00 0x55dc65cc9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.979+0000 7f8345dd1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f833c1b0910 con 0x55dc65ccea00 2024-04-21T08:43:19.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.979+0000 7f8345dd1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f833c1b0240 con 0x55dc65cc54d0 2024-04-21T08:43:19.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.979+0000 7f8345dd1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f833c1ad480 con 0x55dc65cc5e80 2024-04-21T08:43:19.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.979+0000 7f8345dd1740 10 client.0 _ll_register_callbacks cb 0x7f8330005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:19.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.979+0000 7f83416a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65ccea00 0x55dc65cc9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.979+0000 7f83416a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65ccea00 0x55dc65cc9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57890/0 (socket says 172.21.15.43:57890) 2024-04-21T08:43:19.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.979+0000 7f83416a1640 1 -- 172.21.15.43:0/4180160203 learned_addr learned my addr 172.21.15.43:0/4180160203 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:19.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.980+0000 7f833bfff640 1 --2- 172.21.15.43:0/4180160203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc65cc5e80 0x55dc65cc8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.980+0000 7f8340ea0640 1 --2- 172.21.15.43:0/4180160203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc65cc54d0 0x55dc65cc85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.980+0000 7f83416a1640 1 -- 172.21.15.43:0/4180160203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc65cc5e80 msgr2=0x55dc65cc8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.980+0000 7f83416a1640 1 --2- 172.21.15.43:0/4180160203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc65cc5e80 0x55dc65cc8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.980+0000 7f83416a1640 1 -- 172.21.15.43:0/4180160203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc65cc54d0 msgr2=0x55dc65cc85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:19.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.980+0000 7f83416a1640 1 --2- 172.21.15.43:0/4180160203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc65cc54d0 0x55dc65cc85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.980+0000 7f83416a1640 1 -- 172.21.15.43:0/4180160203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc65d44480 con 0x55dc65ccea00 2024-04-21T08:43:19.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.980+0000 7f833bfff640 1 --2- 172.21.15.43:0/4180160203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc65cc5e80 0x55dc65cc8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:19.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.980+0000 7f83416a1640 1 --2- 172.21.15.43:0/4180160203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65ccea00 0x55dc65cc9070 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f8334009bf0 tx=0x7f8334040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=454f7e4f369e3d87 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:19.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.980+0000 7f831bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:19.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.980+0000 7f831bfff640 1 -- 172.21.15.43:0/4180160203 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833404f070 con 0x55dc65ccea00 2024-04-21T08:43:19.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.980+0000 7f831bfff640 1 -- 172.21.15.43:0/4180160203 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f8334034440 con 0x55dc65ccea00 2024-04-21T08:43:19.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.980+0000 7f8345dd1740 1 -- 172.21.15.43:0/4180160203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8328021070 con 0x55dc65ccea00 2024-04-21T08:43:19.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.980+0000 7f831bfff640 1 -- 172.21.15.43:0/4180160203 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8334046050 con 0x55dc65ccea00 2024-04-21T08:43:19.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.980+0000 7f8345dd1740 10 client.17541 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:19.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.980+0000 7f8345dd1740 1 -- 172.21.15.43:0/4180160203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc65d45740 con 0x55dc65ccea00 2024-04-21T08:43:19.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.980+0000 7f831bfff640 1 -- 172.21.15.43:0/4180160203 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8328021070 con 0x55dc65ccea00 2024-04-21T08:43:19.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.981+0000 7f831bfff640 1 -- 172.21.15.43:0/4180160203 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8334040520 con 0x55dc65ccea00 2024-04-21T08:43:19.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.981+0000 7f8345dd1740 10 client.17541 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:19.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.981+0000 7f8345dd1740 10 client.17541 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:43:19.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.981+0000 7f8345dd1740 20 client.17541 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:19.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.981+0000 7f8345dd1740 1 --2- 172.21.15.43:0/4180160203 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55dc65d45e50 0x55dc65d48270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:19.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.981+0000 7f8345dd1740 4 client.17541 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:43:19.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.981+0000 7f8345dd1740 1 -- 172.21.15.43:0/4180160203 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc65c37c10 con 0x55dc65d45e50 2024-04-21T08:43:19.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.981+0000 7f8340ea0640 1 --2- 172.21.15.43:0/4180160203 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55dc65d45e50 0x55dc65d48270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:19.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.981+0000 7f831bfff640 10 client.17541 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:43:19.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.981+0000 7f8340ea0640 1 --2- 172.21.15.43:0/4180160203 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55dc65d45e50 0x55dc65d48270 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7f4c3e42cfe81330 server_cookie=4429585a5c2ae487 in_seq=0 out_seq=0 2024-04-21T08:43:19.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.981+0000 7f831bfff640 1 -- 172.21.15.43:0/4180160203 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc65c37c10 con 0x55dc65d45e50 2024-04-21T08:43:19.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.981+0000 7f831bfff640 10 client.17541 handle_command_reply: tid=0 2024-04-21T08:43:19.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.981+0000 7f8345dd1740 1 client.17541 shutdown 2024-04-21T08:43:19.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.982+0000 7f8345dd1740 1 -- 172.21.15.43:0/4180160203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65ccea00 msgr2=0x55dc65cc9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.982+0000 7f8345dd1740 1 --2- 172.21.15.43:0/4180160203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65ccea00 0x55dc65cc9070 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f8334009bf0 tx=0x7f8334040000 comp rx=0 tx=0).stop 2024-04-21T08:43:19.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.982+0000 7f8345dd1740 1 -- 172.21.15.43:0/4180160203 shutdown_connections 2024-04-21T08:43:19.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.982+0000 7f8345dd1740 1 --2- 172.21.15.43:0/4180160203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc65ccea00 0x55dc65cc9070 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.982+0000 7f8345dd1740 1 --2- 172.21.15.43:0/4180160203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc65cc5e80 0x55dc65cc8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.982+0000 7f8345dd1740 1 --2- 172.21.15.43:0/4180160203 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55dc65d45e50 0x55dc65d48270 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.982+0000 7f8345dd1740 1 --2- 172.21.15.43:0/4180160203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc65cc54d0 0x55dc65cc85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.982+0000 7f831bfff640 0 client.17541 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:43:19.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.982+0000 7f8345dd1740 1 -- 172.21.15.43:0/4180160203 >> 172.21.15.43:0/4180160203 conn(0x55dc65cbfba0 msgr2=0x55dc65cc17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.982+0000 7f8345dd1740 1 -- 172.21.15.43:0/4180160203 shutdown_connections 2024-04-21T08:43:19.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.982+0000 7f8345dd1740 1 -- 172.21.15.43:0/4180160203 wait complete. 2024-04-21T08:43:19.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.982+0000 7f8345dd1740 20 client.17541 trim_cache size 0 max 16384 2024-04-21T08:43:19.988 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:43:19.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.982+0000 7f8343104640 1 -- 172.21.15.43:0/1956191517 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f832c0a3d20 msgr2=0x7f832c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.982+0000 7f8343104640 1 --2- 172.21.15.43:0/1956191517 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f832c0a3d20 0x7f832c0a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8334005cc0 tx=0x7f8334005c30 comp rx=0 tx=0).stop 2024-04-21T08:43:19.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.982+0000 7f8343104640 1 -- 172.21.15.43:0/1956191517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f833c102190 msgr2=0x7f833c198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:19.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.982+0000 7f8343104640 1 --2- 172.21.15.43:0/1956191517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f833c102190 0x7f833c198730 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f833000e9f0 tx=0x7f833000edf0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.982+0000 7f8343104640 1 -- 172.21.15.43:0/1956191517 shutdown_connections 2024-04-21T08:43:19.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.982+0000 7f8343104640 1 --2- 172.21.15.43:0/1956191517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f833c19cff0 0x7f833c19d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.982+0000 7f8343104640 1 --2- 172.21.15.43:0/1956191517 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f832c0a3d20 0x7f832c0a61e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.982+0000 7f8343104640 1 --2- 172.21.15.43:0/1956191517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f833c107260 0x7f833c198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.982+0000 7f8343104640 1 --2- 172.21.15.43:0/1956191517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f833c102190 0x7f833c198730 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:19.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.982+0000 7f8343104640 1 -- 172.21.15.43:0/1956191517 >> 172.21.15.43:0/1956191517 conn(0x7f833c0fd990 msgr2=0x7f833c0ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:19.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.982+0000 7f8343104640 1 -- 172.21.15.43:0/1956191517 shutdown_connections 2024-04-21T08:43:19.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:19.983+0000 7f8343104640 1 -- 172.21.15.43:0/1956191517 wait complete. 2024-04-21T08:43:19.993 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:43:20.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.099+0000 7f6d65a25640 1 -- 172.21.15.43:0/2450890985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d60102190 msgr2=0x7f6d60102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:20.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.099+0000 7f6d65a25640 1 --2- 172.21.15.43:0/2450890985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d60102190 0x7f6d60102570 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6d50009900 tx=0x7f6d5002bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:20.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.099+0000 7f6d65a25640 1 -- 172.21.15.43:0/2450890985 shutdown_connections 2024-04-21T08:43:20.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.099+0000 7f6d65a25640 1 --2- 172.21.15.43:0/2450890985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d60107260 0x7f6d6010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.099+0000 7f6d65a25640 1 --2- 172.21.15.43:0/2450890985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d60102b40 0x7f6d60106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.099+0000 7f6d65a25640 1 --2- 172.21.15.43:0/2450890985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d60102190 0x7f6d60102570 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.099+0000 7f6d65a25640 1 -- 172.21.15.43:0/2450890985 >> 172.21.15.43:0/2450890985 conn(0x7f6d600793a0 msgr2=0x7f6d600797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:20.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.099+0000 7f6d65a25640 1 -- 172.21.15.43:0/2450890985 shutdown_connections 2024-04-21T08:43:20.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.099+0000 7f6d65a25640 1 -- 172.21.15.43:0/2450890985 wait complete. 2024-04-21T08:43:20.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.100+0000 7f6d65a25640 1 Processor -- start 2024-04-21T08:43:20.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.100+0000 7f6d65a25640 1 -- start start 2024-04-21T08:43:20.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.100+0000 7f6d65a25640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d60102190 0x7f6d6019e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.100+0000 7f6d65a25640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d60107260 0x7f6d6019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.100+0000 7f6d65a25640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d60198600 0x7f6d60198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.100+0000 7f6d65a25640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6d60112220 con 0x7f6d60102190 2024-04-21T08:43:20.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.100+0000 7f6d65a25640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601120a0 con 0x7f6d60107260 2024-04-21T08:43:20.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.100+0000 7f6d65a25640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601123a0 con 0x7f6d60198600 2024-04-21T08:43:20.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.100+0000 7f6d5effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d60102190 0x7f6d6019e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.100+0000 7f6d5effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d60102190 0x7f6d6019e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57902/0 (socket says 172.21.15.43:57902) 2024-04-21T08:43:20.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.100+0000 7f6d5effd640 1 -- 172.21.15.43:0/207425654 learned_addr learned my addr 172.21.15.43:0/207425654 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:20.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.100+0000 7f6d57fff640 1 --2- 172.21.15.43:0/207425654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d60107260 0x7f6d6019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.100+0000 7f6d5f7fe640 1 --2- 172.21.15.43:0/207425654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d60198600 0x7f6d60198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.100+0000 7f6d5effd640 1 -- 172.21.15.43:0/207425654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d60198600 msgr2=0x7f6d60198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:20.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.100+0000 7f6d5effd640 1 --2- 172.21.15.43:0/207425654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d60198600 0x7f6d60198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.100+0000 7f6d5effd640 1 -- 172.21.15.43:0/207425654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d60107260 msgr2=0x7f6d6019e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:20.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.100+0000 7f6d5effd640 1 --2- 172.21.15.43:0/207425654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d60107260 0x7f6d6019e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.100+0000 7f6d5effd640 1 -- 172.21.15.43:0/207425654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d60199120 con 0x7f6d60102190 2024-04-21T08:43:20.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.100+0000 7f6d57fff640 1 --2- 172.21.15.43:0/207425654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d60107260 0x7f6d6019e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:20.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.101+0000 7f6d5effd640 1 --2- 172.21.15.43:0/207425654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d60102190 0x7f6d6019e450 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f6d50005940 tx=0x7f6d5002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:20.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.101+0000 7f6d64a23640 1 -- 172.21.15.43:0/207425654 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50043020 con 0x7f6d60102190 2024-04-21T08:43:20.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.101+0000 7f6d64a23640 1 -- 172.21.15.43:0/207425654 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d5002cbe0 con 0x7f6d60102190 2024-04-21T08:43:20.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.101+0000 7f6d65a25640 1 -- 172.21.15.43:0/207425654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d601993b0 con 0x7f6d60102190 2024-04-21T08:43:20.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.101+0000 7f6d64a23640 1 -- 172.21.15.43:0/207425654 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50003bf0 con 0x7f6d60102190 2024-04-21T08:43:20.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.101+0000 7f6d65a25640 1 -- 172.21.15.43:0/207425654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d601a5600 con 0x7f6d60102190 2024-04-21T08:43:20.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.103+0000 7f6d64a23640 1 -- 172.21.15.43:0/207425654 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6d50046020 con 0x7f6d60102190 2024-04-21T08:43:20.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.103+0000 7f6d68615740 1 Processor -- start 2024-04-21T08:43:20.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.103+0000 7f6d68615740 1 -- start start 2024-04-21T08:43:20.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.104+0000 7f6d64a23640 1 --2- 172.21.15.43:0/207425654 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6d4c0a3d20 0x7f6d4c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.104+0000 7f6d68615740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4dd7c04d0 0x55f4dd7c08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.104+0000 7f6d68615740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4dd7c0e80 0x55f4dd7c9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.104+0000 7f6d68615740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 0x55f4dd7cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.104+0000 7f6d68615740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f4dd7251f0 con 0x55f4dd7c9a00 2024-04-21T08:43:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.104+0000 7f6d68615740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f4dd6c24b0 con 0x55f4dd7c04d0 2024-04-21T08:43:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.104+0000 7f6d68615740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f4dd7211b0 con 0x55f4dd7c0e80 2024-04-21T08:43:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.104+0000 7f6d5f7fe640 1 --2- 172.21.15.43:0/207425654 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6d4c0a3d20 0x7f6d4c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.104+0000 7f6d64a23640 1 -- 172.21.15.43:0/207425654 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f6d500f2e10 con 0x7f6d60102190 2024-04-21T08:43:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.104+0000 7f6d5f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 0x55f4dd7cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.104+0000 7f6d57fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4dd7c0e80 0x55f4dd7c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.104+0000 7f6d5f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 0x55f4dd7cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57914/0 (socket says 172.21.15.43:57914) 2024-04-21T08:43:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.104+0000 7f6d5f7fe640 1 -- 172.21.15.43:0/2906479837 learned_addr learned my addr 172.21.15.43:0/2906479837 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.104+0000 7f6d5effd640 1 --2- 172.21.15.43:0/2906479837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4dd7c04d0 0x55f4dd7c08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.104+0000 7f6d5f7fe640 1 -- 172.21.15.43:0/2906479837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4dd7c0e80 msgr2=0x55f4dd7c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.104+0000 7f6d5f7fe640 1 --2- 172.21.15.43:0/2906479837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4dd7c0e80 0x55f4dd7c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.104+0000 7f6d5f7fe640 1 -- 172.21.15.43:0/2906479837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4dd7c04d0 msgr2=0x55f4dd7c08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.104+0000 7f6d5f7fe640 1 --2- 172.21.15.43:0/2906479837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4dd7c04d0 0x55f4dd7c08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.104+0000 7f6d5f7fe640 1 -- 172.21.15.43:0/2906479837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4dd6b41b0 con 0x55f4dd7c9a00 2024-04-21T08:43:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.104+0000 7f6d5f7fe640 1 --2- 172.21.15.43:0/207425654 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6d4c0a3d20 0x7f6d4c0a61e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6d58012be0 tx=0x7f6d58009db0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.104+0000 7f6d5f7fe640 1 --2- 172.21.15.43:0/2906479837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 0x55f4dd7cbe50 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f6d58010800 tx=0x7f6d5801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd8485b998e17f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.104+0000 7f6d55ffb640 1 -- 172.21.15.43:0/2906479837 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58004b20 con 0x55f4dd7c9a00 2024-04-21T08:43:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.105+0000 7f6d55ffb640 1 -- 172.21.15.43:0/2906479837 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d58004cc0 con 0x55f4dd7c9a00 2024-04-21T08:43:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.105+0000 7f6d55ffb640 1 -- 172.21.15.43:0/2906479837 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58005240 con 0x55f4dd7c9a00 2024-04-21T08:43:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.105+0000 7f6d68615740 1 -- 172.21.15.43:0/2906479837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 msgr2=0x55f4dd7cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.105+0000 7f6d68615740 1 --2- 172.21.15.43:0/2906479837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 0x55f4dd7cbe50 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f6d58010800 tx=0x7f6d5801c000 comp rx=0 tx=0).stop 2024-04-21T08:43:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.105+0000 7f6d68615740 1 -- 172.21.15.43:0/2906479837 shutdown_connections 2024-04-21T08:43:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.105+0000 7f6d68615740 1 --2- 172.21.15.43:0/2906479837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 0x55f4dd7cbe50 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.105+0000 7f6d68615740 1 --2- 172.21.15.43:0/2906479837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4dd7c0e80 0x55f4dd7c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.105+0000 7f6d68615740 1 --2- 172.21.15.43:0/2906479837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4dd7c04d0 0x55f4dd7c08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.105+0000 7f6d68615740 1 -- 172.21.15.43:0/2906479837 >> 172.21.15.43:0/2906479837 conn(0x55f4dd7baba0 msgr2=0x55f4dd7baf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.105+0000 7f6d68615740 1 -- 172.21.15.43:0/2906479837 shutdown_connections 2024-04-21T08:43:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.105+0000 7f6d68615740 1 -- 172.21.15.43:0/2906479837 wait complete. 2024-04-21T08:43:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.106+0000 7f6d68615740 1 -- start start 2024-04-21T08:43:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.106+0000 7f6d68615740 1 Processor -- start 2024-04-21T08:43:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.106+0000 7f6d68615740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4dd7c04d0 0x55f4dd7c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.106+0000 7f6d68615740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4dd7c0e80 0x55f4dd7c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.106+0000 7f6d68615740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 0x55f4dd7c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.106+0000 7f6d68615740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f4dd7cc910 con 0x55f4dd7c9a00 2024-04-21T08:43:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.106+0000 7f6d68615740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f4dd6de500 con 0x55f4dd7c0e80 2024-04-21T08:43:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.106+0000 7f6d68615740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f4dd7cca90 con 0x55f4dd7c04d0 2024-04-21T08:43:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.106+0000 7f6d68615740 10 client.0 _ll_register_callbacks cb 0x55f4dd6acb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.106+0000 7f6d5f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 0x55f4dd7c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.106+0000 7f6d5f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 0x55f4dd7c6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57916/0 (socket says 172.21.15.43:57916) 2024-04-21T08:43:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.106+0000 7f6d5f7fe640 1 -- 172.21.15.43:0/1288635152 learned_addr learned my addr 172.21.15.43:0/1288635152 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.106+0000 7f6d57fff640 1 --2- 172.21.15.43:0/1288635152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4dd7c0e80 0x55f4dd7c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.106+0000 7f6d5effd640 1 --2- 172.21.15.43:0/1288635152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4dd7c04d0 0x55f4dd7c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.106+0000 7f6d5f7fe640 1 -- 172.21.15.43:0/1288635152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4dd7c04d0 msgr2=0x55f4dd7c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.106+0000 7f6d5f7fe640 1 --2- 172.21.15.43:0/1288635152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4dd7c04d0 0x55f4dd7c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.107+0000 7f6d5f7fe640 1 -- 172.21.15.43:0/1288635152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4dd7c0e80 msgr2=0x55f4dd7c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.107+0000 7f6d5f7fe640 1 --2- 172.21.15.43:0/1288635152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4dd7c0e80 0x55f4dd7c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.107+0000 7f6d5f7fe640 1 -- 172.21.15.43:0/1288635152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4dd4373f0 con 0x55f4dd7c9a00 2024-04-21T08:43:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.107+0000 7f6d5effd640 1 --2- 172.21.15.43:0/1288635152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4dd7c04d0 0x55f4dd7c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.107+0000 7f6d57fff640 1 --2- 172.21.15.43:0/1288635152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4dd7c0e80 0x55f4dd7c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.107+0000 7f6d5f7fe640 1 --2- 172.21.15.43:0/1288635152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 0x55f4dd7c6c90 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6d58004c90 tx=0x7f6d580099a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23723d363458fb44 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.107+0000 7f6d3effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.107+0000 7f6d3effd640 1 -- 172.21.15.43:0/1288635152 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5802e020 con 0x55f4dd7c9a00 2024-04-21T08:43:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.107+0000 7f6d3effd640 1 -- 172.21.15.43:0/1288635152 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f6d5800c9b0 con 0x55f4dd7c9a00 2024-04-21T08:43:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.107+0000 7f6d68615740 1 -- 172.21.15.43:0/1288635152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d58004cc0 con 0x55f4dd7c9a00 2024-04-21T08:43:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.107+0000 7f6d3effd640 1 -- 172.21.15.43:0/1288635152 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6d58039050 con 0x55f4dd7c9a00 2024-04-21T08:43:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.107+0000 7f6d68615740 10 client.17553 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.107+0000 7f6d68615740 1 -- 172.21.15.43:0/1288635152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f4dd5017d0 con 0x55f4dd7c9a00 2024-04-21T08:43:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.107+0000 7f6d3effd640 1 -- 172.21.15.43:0/1288635152 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d58004cc0 con 0x55f4dd7c9a00 2024-04-21T08:43:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.107+0000 7f6d3effd640 1 -- 172.21.15.43:0/1288635152 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6d5801f070 con 0x55f4dd7c9a00 2024-04-21T08:43:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.107+0000 7f6d68615740 10 client.17553 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.107+0000 7f6d68615740 10 client.17553 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:43:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.107+0000 7f6d68615740 20 client.17553 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.107+0000 7f6d68615740 1 --2- 172.21.15.43:0/1288635152 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55f4dd7d9b10 0x55f4dd7dbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.108+0000 7f6d68615740 4 client.17553 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.108+0000 7f6d68615740 1 -- 172.21.15.43:0/1288635152 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4dd6c4930 con 0x55f4dd7d9b10 2024-04-21T08:43:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.108+0000 7f6d5effd640 1 --2- 172.21.15.43:0/1288635152 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55f4dd7d9b10 0x55f4dd7dbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.108+0000 7f6d5effd640 1 --2- 172.21.15.43:0/1288635152 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55f4dd7d9b10 0x55f4dd7dbf00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=149d879c91ee9f89 server_cookie=4c0712fa18c2bed9 in_seq=0 out_seq=0 2024-04-21T08:43:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.108+0000 7f6d3effd640 10 client.17553 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:43:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.109+0000 7f6d3effd640 1 -- 172.21.15.43:0/1288635152 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f4dd6c4930 con 0x55f4dd7d9b10 2024-04-21T08:43:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.109+0000 7f6d3effd640 10 client.17553 handle_command_reply: tid=0 2024-04-21T08:43:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.109+0000 7f6d68615740 1 client.17553 shutdown 2024-04-21T08:43:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.109+0000 7f6d68615740 1 -- 172.21.15.43:0/1288635152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 msgr2=0x55f4dd7c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.109+0000 7f6d68615740 1 --2- 172.21.15.43:0/1288635152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 0x55f4dd7c6c90 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6d58004c90 tx=0x7f6d580099a0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.109+0000 7f6d68615740 1 -- 172.21.15.43:0/1288635152 shutdown_connections 2024-04-21T08:43:20.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.109+0000 7f6d68615740 1 --2- 172.21.15.43:0/1288635152 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55f4dd7d9b10 0x55f4dd7dbf00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.109+0000 7f6d68615740 1 --2- 172.21.15.43:0/1288635152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 0x55f4dd7c6c90 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.109+0000 7f6d68615740 1 --2- 172.21.15.43:0/1288635152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4dd7c0e80 0x55f4dd7c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.109+0000 7f6d3effd640 0 client.17553 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:43:20.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.109+0000 7f6d68615740 1 --2- 172.21.15.43:0/1288635152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4dd7c04d0 0x55f4dd7c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.109+0000 7f6d68615740 1 -- 172.21.15.43:0/1288635152 >> 172.21.15.43:0/1288635152 conn(0x55f4dd7baba0 msgr2=0x55f4dd7bc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:20.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.109+0000 7f6d68615740 1 -- 172.21.15.43:0/1288635152 shutdown_connections 2024-04-21T08:43:20.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.109+0000 7f6d68615740 1 -- 172.21.15.43:0/1288635152 wait complete. 2024-04-21T08:43:20.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.109+0000 7f6d68615740 20 client.17553 trim_cache size 0 max 16384 2024-04-21T08:43:20.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.123+0000 7f6d68615740 1 Processor -- start 2024-04-21T08:43:20.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.123+0000 7f6d68615740 1 -- start start 2024-04-21T08:43:20.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.123+0000 7f6d68615740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4dd7c04d0 0x55f4dd7c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.123+0000 7f6d68615740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4dd7c0e80 0x55f4dd7b9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.123+0000 7f6d68615740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 0x55f4dd7c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.123+0000 7f6d68615740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601af4c0 con 0x55f4dd7c9a00 2024-04-21T08:43:20.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.123+0000 7f6d68615740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601b1000 con 0x55f4dd7c04d0 2024-04-21T08:43:20.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.123+0000 7f6d68615740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601afb90 con 0x55f4dd7c0e80 2024-04-21T08:43:20.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.123+0000 7f6d5f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 0x55f4dd7c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.123+0000 7f6d5f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 0x55f4dd7c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57920/0 (socket says 172.21.15.43:57920) 2024-04-21T08:43:20.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.123+0000 7f6d5f7fe640 1 -- 172.21.15.43:0/1471399291 learned_addr learned my addr 172.21.15.43:0/1471399291 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:20.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.123+0000 7f6d5effd640 1 --2- 172.21.15.43:0/1471399291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4dd7c04d0 0x55f4dd7c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.123+0000 7f6d57fff640 1 --2- 172.21.15.43:0/1471399291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4dd7c0e80 0x55f4dd7b9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.123+0000 7f6d5f7fe640 1 -- 172.21.15.43:0/1471399291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4dd7c0e80 msgr2=0x55f4dd7b9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.123+0000 7f6d5f7fe640 1 --2- 172.21.15.43:0/1471399291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4dd7c0e80 0x55f4dd7b9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.123+0000 7f6d5f7fe640 1 -- 172.21.15.43:0/1471399291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4dd7c04d0 msgr2=0x55f4dd7c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.123+0000 7f6d5f7fe640 1 --2- 172.21.15.43:0/1471399291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4dd7c04d0 0x55f4dd7c5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.123+0000 7f6d5f7fe640 1 -- 172.21.15.43:0/1471399291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4dd7bbdd0 con 0x55f4dd7c9a00 2024-04-21T08:43:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.123+0000 7f6d5f7fe640 1 --2- 172.21.15.43:0/1471399291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 0x55f4dd7c8e00 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009d00 tx=0x7f6d58004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746d8c493675e96b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.124+0000 7f6d557fa640 1 -- 172.21.15.43:0/1471399291 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5802e070 con 0x55f4dd7c9a00 2024-04-21T08:43:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.124+0000 7f6d557fa640 1 -- 172.21.15.43:0/1471399291 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d58020570 con 0x55f4dd7c9a00 2024-04-21T08:43:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.124+0000 7f6d557fa640 1 -- 172.21.15.43:0/1471399291 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58004fc0 con 0x55f4dd7c9a00 2024-04-21T08:43:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.124+0000 7f6d68615740 1 -- 172.21.15.43:0/1471399291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 msgr2=0x55f4dd7c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.124+0000 7f6d68615740 1 --2- 172.21.15.43:0/1471399291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 0x55f4dd7c8e00 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009d00 tx=0x7f6d58004e50 comp rx=0 tx=0).stop 2024-04-21T08:43:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.124+0000 7f6d68615740 1 -- 172.21.15.43:0/1471399291 shutdown_connections 2024-04-21T08:43:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.124+0000 7f6d68615740 1 --2- 172.21.15.43:0/1471399291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 0x55f4dd7c8e00 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.124+0000 7f6d68615740 1 --2- 172.21.15.43:0/1471399291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4dd7c0e80 0x55f4dd7b9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.124+0000 7f6d68615740 1 --2- 172.21.15.43:0/1471399291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4dd7c04d0 0x55f4dd7c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.124+0000 7f6d68615740 1 -- 172.21.15.43:0/1471399291 >> 172.21.15.43:0/1471399291 conn(0x55f4dd7baba0 msgr2=0x55f4dd7bc880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:20.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.124+0000 7f6d68615740 1 -- 172.21.15.43:0/1471399291 shutdown_connections 2024-04-21T08:43:20.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.124+0000 7f6d68615740 1 -- 172.21.15.43:0/1471399291 wait complete. 2024-04-21T08:43:20.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.124+0000 7f6d68615740 1 -- start start 2024-04-21T08:43:20.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.125+0000 7f6d68615740 1 Processor -- start 2024-04-21T08:43:20.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.125+0000 7f6d68615740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4dd7c04d0 0x55f4dd7d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.125+0000 7f6d68615740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4dd7c0e80 0x55f4dd7d1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.125+0000 7f6d68615740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 0x55f4dd7cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.125+0000 7f6d68615740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601b0930 con 0x55f4dd7c9a00 2024-04-21T08:43:20.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.125+0000 7f6d68615740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601b0260 con 0x55f4dd7c0e80 2024-04-21T08:43:20.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.125+0000 7f6d68615740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601ad4a0 con 0x55f4dd7c04d0 2024-04-21T08:43:20.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.125+0000 7f6d68615740 10 client.0 _ll_register_callbacks cb 0x7f6d58012530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:20.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.125+0000 7f6d5f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 0x55f4dd7cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.125+0000 7f6d5f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 0x55f4dd7cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57930/0 (socket says 172.21.15.43:57930) 2024-04-21T08:43:20.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.125+0000 7f6d5f7fe640 1 -- 172.21.15.43:0/1150739539 learned_addr learned my addr 172.21.15.43:0/1150739539 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:20.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.125+0000 7f6d5effd640 1 --2- 172.21.15.43:0/1150739539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4dd7c04d0 0x55f4dd7d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.125+0000 7f6d57fff640 1 --2- 172.21.15.43:0/1150739539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4dd7c0e80 0x55f4dd7d1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.125+0000 7f6d5f7fe640 1 -- 172.21.15.43:0/1150739539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4dd7c04d0 msgr2=0x55f4dd7d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:20.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.125+0000 7f6d5f7fe640 1 --2- 172.21.15.43:0/1150739539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4dd7c04d0 0x55f4dd7d1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.125+0000 7f6d5f7fe640 1 -- 172.21.15.43:0/1150739539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4dd7c0e80 msgr2=0x55f4dd7d1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:20.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.125+0000 7f6d5f7fe640 1 --2- 172.21.15.43:0/1150739539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4dd7c0e80 0x55f4dd7d1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.125+0000 7f6d5f7fe640 1 -- 172.21.15.43:0/1150739539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4dd7cd590 con 0x55f4dd7c9a00 2024-04-21T08:43:20.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.125+0000 7f6d5f7fe640 1 --2- 172.21.15.43:0/1150739539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 0x55f4dd7cce50 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f6d58005b40 tx=0x7f6d58009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a5cebddab9b740 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:20.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.125+0000 7f6d3effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:20.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.126+0000 7f6d3effd640 1 -- 172.21.15.43:0/1150739539 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5802e070 con 0x55f4dd7c9a00 2024-04-21T08:43:20.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.126+0000 7f6d3effd640 1 -- 172.21.15.43:0/1150739539 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f6d58020930 con 0x55f4dd7c9a00 2024-04-21T08:43:20.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.126+0000 7f6d68615740 1 -- 172.21.15.43:0/1150739539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d58020570 con 0x55f4dd7c9a00 2024-04-21T08:43:20.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.126+0000 7f6d3effd640 1 -- 172.21.15.43:0/1150739539 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6d5804f050 con 0x55f4dd7c9a00 2024-04-21T08:43:20.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.126+0000 7f6d68615740 10 client.17559 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:20.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.126+0000 7f6d68615740 1 -- 172.21.15.43:0/1150739539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f4dd83f950 con 0x55f4dd7c9a00 2024-04-21T08:43:20.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.126+0000 7f6d3effd640 1 -- 172.21.15.43:0/1150739539 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d58020570 con 0x55f4dd7c9a00 2024-04-21T08:43:20.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.126+0000 7f6d3effd640 1 -- 172.21.15.43:0/1150739539 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6d58005580 con 0x55f4dd7c9a00 2024-04-21T08:43:20.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.126+0000 7f6d68615740 10 client.17559 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:20.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.126+0000 7f6d68615740 10 client.17559 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:43:20.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.126+0000 7f6d68615740 20 client.17559 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:20.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.126+0000 7f6d68615740 1 --2- 172.21.15.43:0/1150739539 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55f4dd83ff30 0x55f4dd842350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.126+0000 7f6d68615740 4 client.17559 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:43:20.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.126+0000 7f6d68615740 1 -- 172.21.15.43:0/1150739539 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f4dd732c10 con 0x55f4dd83ff30 2024-04-21T08:43:20.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.126+0000 7f6d5effd640 1 --2- 172.21.15.43:0/1150739539 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55f4dd83ff30 0x55f4dd842350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.127+0000 7f6d3effd640 10 client.17559 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:43:20.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.127+0000 7f6d5effd640 1 --2- 172.21.15.43:0/1150739539 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55f4dd83ff30 0x55f4dd842350 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=572852ddd410ee2c server_cookie=7e56ce9d70d41bdd in_seq=0 out_seq=0 2024-04-21T08:43:20.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.127+0000 7f6d3effd640 1 -- 172.21.15.43:0/1150739539 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f4dd732c10 con 0x55f4dd83ff30 2024-04-21T08:43:20.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.127+0000 7f6d3effd640 10 client.17559 handle_command_reply: tid=0 2024-04-21T08:43:20.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.127+0000 7f6d68615740 1 client.17559 shutdown 2024-04-21T08:43:20.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.127+0000 7f6d68615740 1 -- 172.21.15.43:0/1150739539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 msgr2=0x55f4dd7cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:20.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.127+0000 7f6d68615740 1 --2- 172.21.15.43:0/1150739539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 0x55f4dd7cce50 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f6d58005b40 tx=0x7f6d58009c90 comp rx=0 tx=0).stop 2024-04-21T08:43:20.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.127+0000 7f6d68615740 1 -- 172.21.15.43:0/1150739539 shutdown_connections 2024-04-21T08:43:20.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.127+0000 7f6d68615740 1 --2- 172.21.15.43:0/1150739539 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55f4dd83ff30 0x55f4dd842350 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.127+0000 7f6d68615740 1 --2- 172.21.15.43:0/1150739539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4dd7c9a00 0x55f4dd7cce50 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.127+0000 7f6d68615740 1 --2- 172.21.15.43:0/1150739539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4dd7c0e80 0x55f4dd7d1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.127+0000 7f6d3effd640 0 client.17559 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:43:20.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.127+0000 7f6d68615740 1 --2- 172.21.15.43:0/1150739539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4dd7c04d0 0x55f4dd7d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.127+0000 7f6d68615740 1 -- 172.21.15.43:0/1150739539 >> 172.21.15.43:0/1150739539 conn(0x55f4dd7baba0 msgr2=0x55f4dd7bc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:20.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.127+0000 7f6d68615740 1 -- 172.21.15.43:0/1150739539 shutdown_connections 2024-04-21T08:43:20.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.127+0000 7f6d68615740 1 -- 172.21.15.43:0/1150739539 wait complete. 2024-04-21T08:43:20.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.127+0000 7f6d68615740 20 client.17559 trim_cache size 0 max 16384 2024-04-21T08:43:20.134 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:43:20.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.128+0000 7f6d65a25640 1 -- 172.21.15.43:0/207425654 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6d4c0a3d20 msgr2=0x7f6d4c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:20.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.128+0000 7f6d65a25640 1 --2- 172.21.15.43:0/207425654 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6d4c0a3d20 0x7f6d4c0a61e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6d58012be0 tx=0x7f6d58009db0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.128+0000 7f6d65a25640 1 -- 172.21.15.43:0/207425654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d60102190 msgr2=0x7f6d6019e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:20.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.128+0000 7f6d65a25640 1 --2- 172.21.15.43:0/207425654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d60102190 0x7f6d6019e450 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f6d50005940 tx=0x7f6d5002c7e0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.128+0000 7f6d65a25640 1 -- 172.21.15.43:0/207425654 shutdown_connections 2024-04-21T08:43:20.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.128+0000 7f6d65a25640 1 --2- 172.21.15.43:0/207425654 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6d4c0a3d20 0x7f6d4c0a61e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.128+0000 7f6d65a25640 1 --2- 172.21.15.43:0/207425654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d60198600 0x7f6d60198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.128+0000 7f6d65a25640 1 --2- 172.21.15.43:0/207425654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d60107260 0x7f6d6019e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.128+0000 7f6d65a25640 1 --2- 172.21.15.43:0/207425654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d60102190 0x7f6d6019e450 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.128+0000 7f6d65a25640 1 -- 172.21.15.43:0/207425654 >> 172.21.15.43:0/207425654 conn(0x7f6d600793a0 msgr2=0x7f6d60100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:20.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.128+0000 7f6d65a25640 1 -- 172.21.15.43:0/207425654 shutdown_connections 2024-04-21T08:43:20.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.128+0000 7f6d65a25640 1 -- 172.21.15.43:0/207425654 wait complete. 2024-04-21T08:43:20.138 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:43:20.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.243+0000 7f5fb4df4640 1 -- 172.21.15.43:0/1920438645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fb0106170 msgr2=0x7f5fb010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:20.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.243+0000 7f5fb4df4640 1 --2- 172.21.15.43:0/1920438645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fb0106170 0x7f5fb010dd40 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009ca0 tx=0x7f5fa802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.243+0000 7f5fb4df4640 1 -- 172.21.15.43:0/1920438645 shutdown_connections 2024-04-21T08:43:20.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.243+0000 7f5fb4df4640 1 --2- 172.21.15.43:0/1920438645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fb0106170 0x7f5fb010dd40 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.243+0000 7f5fb4df4640 1 --2- 172.21.15.43:0/1920438645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fb0101a50 0x7f5fb0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.243+0000 7f5fb4df4640 1 --2- 172.21.15.43:0/1920438645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fb01010a0 0x7f5fb0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.243+0000 7f5fb4df4640 1 -- 172.21.15.43:0/1920438645 >> 172.21.15.43:0/1920438645 conn(0x7f5fb0077eb0 msgr2=0x7f5fb00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:20.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.243+0000 7f5fb4df4640 1 -- 172.21.15.43:0/1920438645 shutdown_connections 2024-04-21T08:43:20.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.243+0000 7f5fb4df4640 1 -- 172.21.15.43:0/1920438645 wait complete. 2024-04-21T08:43:20.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.244+0000 7f5fb4df4640 1 Processor -- start 2024-04-21T08:43:20.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.244+0000 7f5fb4df4640 1 -- start start 2024-04-21T08:43:20.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.244+0000 7f5fb4df4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fb01010a0 0x7f5fb01994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.244+0000 7f5fb4df4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fb0106170 0x7f5fb01999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.244+0000 7f5fb4df4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fb0199f70 0x7f5fb01a4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.244+0000 7f5fb4df4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb01110e0 con 0x7f5fb0199f70 2024-04-21T08:43:20.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.244+0000 7f5fb4df4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb0110f60 con 0x7f5fb01010a0 2024-04-21T08:43:20.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.244+0000 7f5fb4df4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb0111260 con 0x7f5fb0106170 2024-04-21T08:43:20.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.244+0000 7f5faed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fb0199f70 0x7f5fb01a4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.244+0000 7f5faed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fb0199f70 0x7f5fb01a4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57954/0 (socket says 172.21.15.43:57954) 2024-04-21T08:43:20.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.244+0000 7f5faed9d640 1 -- 172.21.15.43:0/3387410874 learned_addr learned my addr 172.21.15.43:0/3387410874 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:20.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.245+0000 7f5fae59c640 1 --2- 172.21.15.43:0/3387410874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fb01010a0 0x7f5fb01994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.245+0000 7f5fadd9b640 1 --2- 172.21.15.43:0/3387410874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fb0106170 0x7f5fb01999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.245+0000 7f5faed9d640 1 -- 172.21.15.43:0/3387410874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fb0106170 msgr2=0x7f5fb01999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:20.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.245+0000 7f5faed9d640 1 --2- 172.21.15.43:0/3387410874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fb0106170 0x7f5fb01999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.245+0000 7f5faed9d640 1 -- 172.21.15.43:0/3387410874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fb01010a0 msgr2=0x7f5fb01994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:20.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.245+0000 7f5faed9d640 1 --2- 172.21.15.43:0/3387410874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fb01010a0 0x7f5fb01994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.245+0000 7f5faed9d640 1 -- 172.21.15.43:0/3387410874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb01a4d20 con 0x7f5fb0199f70 2024-04-21T08:43:20.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.245+0000 7f5fae59c640 1 --2- 172.21.15.43:0/3387410874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fb01010a0 0x7f5fb01994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:20.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.245+0000 7f5faed9d640 1 --2- 172.21.15.43:0/3387410874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fb0199f70 0x7f5fb01a4520 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80098d0 tx=0x7f5fa8035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:20.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.245+0000 7f5fa77fe640 1 -- 172.21.15.43:0/3387410874 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8039040 con 0x7f5fb0199f70 2024-04-21T08:43:20.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.245+0000 7f5fa77fe640 1 -- 172.21.15.43:0/3387410874 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fa8044070 con 0x7f5fb0199f70 2024-04-21T08:43:20.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.245+0000 7f5fb4df4640 1 -- 172.21.15.43:0/3387410874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb01a4fb0 con 0x7f5fb0199f70 2024-04-21T08:43:20.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.245+0000 7f5fa77fe640 1 -- 172.21.15.43:0/3387410874 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa80042a0 con 0x7f5fb0199f70 2024-04-21T08:43:20.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.245+0000 7f5fb4df4640 1 -- 172.21.15.43:0/3387410874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb01a5470 con 0x7f5fb0199f70 2024-04-21T08:43:20.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.247+0000 7f5fa77fe640 1 -- 172.21.15.43:0/3387410874 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5fa8004440 con 0x7f5fb0199f70 2024-04-21T08:43:20.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.248+0000 7f5fb79e5740 1 Processor -- start 2024-04-21T08:43:20.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.248+0000 7f5fb79e5740 1 -- start start 2024-04-21T08:43:20.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.248+0000 7f5fa77fe640 1 --2- 172.21.15.43:0/3387410874 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5f9c0a3d20 0x7f5f9c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.248+0000 7f5fb79e5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568135d93b0 0x5568135d9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.248+0000 7f5fb79e5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135d9d60 0x5568135e2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.248+0000 7f5fb79e5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568135e2e60 0x5568135e32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.248+0000 7f5fb79e5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55681353e1f0 con 0x5568135d9d60 2024-04-21T08:43:20.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.248+0000 7f5fa77fe640 1 -- 172.21.15.43:0/3387410874 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f5fa80f3120 con 0x7f5fb0199f70 2024-04-21T08:43:20.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.248+0000 7f5fb79e5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5568134db4b0 con 0x5568135d93b0 2024-04-21T08:43:20.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.248+0000 7f5fb79e5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55681353a1b0 con 0x5568135e2e60 2024-04-21T08:43:20.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.248+0000 7f5fadd9b640 1 --2- 172.21.15.43:0/3387410874 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5f9c0a3d20 0x7f5f9c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.248+0000 7f5fadd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135d9d60 0x5568135e2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.248+0000 7f5fadd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135d9d60 0x5568135e2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57960/0 (socket says 172.21.15.43:57960) 2024-04-21T08:43:20.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.248+0000 7f5fadd9b640 1 -- 172.21.15.43:0/3705486399 learned_addr learned my addr 172.21.15.43:0/3705486399 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:20.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.248+0000 7f5fae59c640 1 --2- 172.21.15.43:0/3705486399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568135d93b0 0x5568135d9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.248+0000 7f5faed9d640 1 --2- 172.21.15.43:0/3705486399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568135e2e60 0x5568135e32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.249+0000 7f5fadd9b640 1 -- 172.21.15.43:0/3705486399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568135e2e60 msgr2=0x5568135e32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:20.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.249+0000 7f5fadd9b640 1 --2- 172.21.15.43:0/3705486399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568135e2e60 0x5568135e32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.249+0000 7f5fadd9b640 1 -- 172.21.15.43:0/3705486399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568135d93b0 msgr2=0x5568135d9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:20.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.249+0000 7f5fadd9b640 1 --2- 172.21.15.43:0/3705486399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568135d93b0 0x5568135d9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.249+0000 7f5fadd9b640 1 -- 172.21.15.43:0/3705486399 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568134cd1b0 con 0x5568135d9d60 2024-04-21T08:43:20.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.249+0000 7f5fadd9b640 1 --2- 172.21.15.43:0/3387410874 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5f9c0a3d20 0x7f5f9c0a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5f980138f0 tx=0x7f5f9800bac0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:20.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.249+0000 7f5fadd9b640 1 --2- 172.21.15.43:0/3705486399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135d9d60 0x5568135e2890 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f5f98010c10 tx=0x7f5f9801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641d711701632496 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:20.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.249+0000 7f5fa4ff9640 1 -- 172.21.15.43:0/3705486399 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9800b3c0 con 0x5568135d9d60 2024-04-21T08:43:20.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.249+0000 7f5fa4ff9640 1 -- 172.21.15.43:0/3705486399 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f980130d0 con 0x5568135d9d60 2024-04-21T08:43:20.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.249+0000 7f5fa4ff9640 1 -- 172.21.15.43:0/3705486399 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f98005020 con 0x5568135d9d60 2024-04-21T08:43:20.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.250+0000 7f5fb79e5740 1 -- 172.21.15.43:0/3705486399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135d9d60 msgr2=0x5568135e2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:20.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.250+0000 7f5fb79e5740 1 --2- 172.21.15.43:0/3705486399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135d9d60 0x5568135e2890 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f5f98010c10 tx=0x7f5f9801a000 comp rx=0 tx=0).stop 2024-04-21T08:43:20.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.250+0000 7f5fb79e5740 1 -- 172.21.15.43:0/3705486399 shutdown_connections 2024-04-21T08:43:20.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.250+0000 7f5fb79e5740 1 --2- 172.21.15.43:0/3705486399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568135e2e60 0x5568135e32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.250+0000 7f5fb79e5740 1 --2- 172.21.15.43:0/3705486399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135d9d60 0x5568135e2890 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.250+0000 7f5fb79e5740 1 --2- 172.21.15.43:0/3705486399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568135d93b0 0x5568135d9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.250+0000 7f5fb79e5740 1 -- 172.21.15.43:0/3705486399 >> 172.21.15.43:0/3705486399 conn(0x5568135d3ba0 msgr2=0x5568135d3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:20.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.250+0000 7f5fb79e5740 1 -- 172.21.15.43:0/3705486399 shutdown_connections 2024-04-21T08:43:20.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.250+0000 7f5fb79e5740 1 -- 172.21.15.43:0/3705486399 wait complete. 2024-04-21T08:43:20.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.250+0000 7f5fb79e5740 1 -- start start 2024-04-21T08:43:20.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.250+0000 7f5fb79e5740 1 Processor -- start 2024-04-21T08:43:20.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.250+0000 7f5fb79e5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568135d93b0 0x5568135e92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.250+0000 7f5fb79e5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135d9d60 0x5568135e6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.250+0000 7f5fb79e5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568135e2e60 0x5568135e6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.250+0000 7f5fb79e5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5568135e9d40 con 0x5568135d9d60 2024-04-21T08:43:20.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.250+0000 7f5fb79e5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5568134f7500 con 0x5568135e2e60 2024-04-21T08:43:20.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.250+0000 7f5fb79e5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5568135e9ec0 con 0x5568135d93b0 2024-04-21T08:43:20.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.250+0000 7f5fb79e5740 10 client.0 _ll_register_callbacks cb 0x5568134c5b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:20.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.251+0000 7f5fadd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135d9d60 0x5568135e6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.251+0000 7f5fadd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135d9d60 0x5568135e6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57962/0 (socket says 172.21.15.43:57962) 2024-04-21T08:43:20.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.251+0000 7f5fadd9b640 1 -- 172.21.15.43:0/678654868 learned_addr learned my addr 172.21.15.43:0/678654868 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:20.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.251+0000 7f5faed9d640 1 --2- 172.21.15.43:0/678654868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568135e2e60 0x5568135e6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.251+0000 7f5fae59c640 1 --2- 172.21.15.43:0/678654868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568135d93b0 0x5568135e92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.251+0000 7f5fadd9b640 1 -- 172.21.15.43:0/678654868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568135d93b0 msgr2=0x5568135e92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:20.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.251+0000 7f5fadd9b640 1 --2- 172.21.15.43:0/678654868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568135d93b0 0x5568135e92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.251+0000 7f5fadd9b640 1 -- 172.21.15.43:0/678654868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568135e2e60 msgr2=0x5568135e6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:20.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.251+0000 7f5fadd9b640 1 --2- 172.21.15.43:0/678654868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568135e2e60 0x5568135e6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.251+0000 7f5fadd9b640 1 -- 172.21.15.43:0/678654868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568132503f0 con 0x5568135d9d60 2024-04-21T08:43:20.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.251+0000 7f5fadd9b640 1 --2- 172.21.15.43:0/678654868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135d9d60 0x5568135e6630 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f5f980107a0 tx=0x7f5f98013f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4e029023e18be server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:20.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.251+0000 7f5f95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:20.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.251+0000 7f5f95ffb640 1 -- 172.21.15.43:0/678654868 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9802b070 con 0x5568135d9d60 2024-04-21T08:43:20.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.251+0000 7f5f95ffb640 1 -- 172.21.15.43:0/678654868 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f5f98005250 con 0x5568135d9d60 2024-04-21T08:43:20.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.251+0000 7f5fb79e5740 1 -- 172.21.15.43:0/678654868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f980130d0 con 0x5568135d9d60 2024-04-21T08:43:20.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.251+0000 7f5f95ffb640 1 -- 172.21.15.43:0/678654868 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5f98037050 con 0x5568135d9d60 2024-04-21T08:43:20.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.251+0000 7f5fb79e5740 10 client.17571 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:20.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.251+0000 7f5fb79e5740 1 -- 172.21.15.43:0/678654868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55681331a7d0 con 0x5568135d9d60 2024-04-21T08:43:20.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.252+0000 7f5f95ffb640 1 -- 172.21.15.43:0/678654868 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f980130d0 con 0x5568135d9d60 2024-04-21T08:43:20.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.252+0000 7f5f95ffb640 1 -- 172.21.15.43:0/678654868 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5f98004cc0 con 0x5568135d9d60 2024-04-21T08:43:20.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.252+0000 7f5fb79e5740 10 client.17571 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:20.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.252+0000 7f5fb79e5740 10 client.17571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:20.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.252+0000 7f5fb79e5740 20 client.17571 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:20.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.252+0000 7f5fb79e5740 1 --2- 172.21.15.43:0/678654868 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5568135f6ef0 0x5568135f92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.252+0000 7f5fb79e5740 4 client.17571 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:20.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.252+0000 7f5fb79e5740 1 -- 172.21.15.43:0/678654868 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568134dd930 con 0x5568135f6ef0 2024-04-21T08:43:20.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.252+0000 7f5fae59c640 1 --2- 172.21.15.43:0/678654868 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5568135f6ef0 0x5568135f92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.253+0000 7f5f95ffb640 10 client.17571 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:20.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.253+0000 7f5fae59c640 1 --2- 172.21.15.43:0/678654868 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5568135f6ef0 0x5568135f92e0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c01799831a63be65 server_cookie=746aeec05f1744ce in_seq=0 out_seq=0 2024-04-21T08:43:20.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.253+0000 7f5f95ffb640 1 -- 172.21.15.43:0/678654868 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568134dd930 con 0x5568135f6ef0 2024-04-21T08:43:20.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.253+0000 7f5f95ffb640 10 client.17571 handle_command_reply: tid=0 2024-04-21T08:43:20.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.253+0000 7f5fb79e5740 1 client.17571 shutdown 2024-04-21T08:43:20.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.253+0000 7f5fb79e5740 1 -- 172.21.15.43:0/678654868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135d9d60 msgr2=0x5568135e6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:20.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.253+0000 7f5fb79e5740 1 --2- 172.21.15.43:0/678654868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135d9d60 0x5568135e6630 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f5f980107a0 tx=0x7f5f98013f50 comp rx=0 tx=0).stop 2024-04-21T08:43:20.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.253+0000 7f5fb79e5740 1 -- 172.21.15.43:0/678654868 shutdown_connections 2024-04-21T08:43:20.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.253+0000 7f5fb79e5740 1 --2- 172.21.15.43:0/678654868 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5568135f6ef0 0x5568135f92e0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.253+0000 7f5fb79e5740 1 --2- 172.21.15.43:0/678654868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568135e2e60 0x5568135e6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.253+0000 7f5f95ffb640 0 client.17571 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:20.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.253+0000 7f5fb79e5740 1 --2- 172.21.15.43:0/678654868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135d9d60 0x5568135e6630 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.253+0000 7f5fb79e5740 1 --2- 172.21.15.43:0/678654868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568135d93b0 0x5568135e92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.253+0000 7f5fb79e5740 1 -- 172.21.15.43:0/678654868 >> 172.21.15.43:0/678654868 conn(0x5568135d3ba0 msgr2=0x5568135d58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:20.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.254+0000 7f5fb79e5740 1 -- 172.21.15.43:0/678654868 shutdown_connections 2024-04-21T08:43:20.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.254+0000 7f5fb79e5740 1 -- 172.21.15.43:0/678654868 wait complete. 2024-04-21T08:43:20.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.254+0000 7f5fb79e5740 20 client.17571 trim_cache size 0 max 16384 2024-04-21T08:43:20.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.267+0000 7f5fb79e5740 1 Processor -- start 2024-04-21T08:43:20.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.267+0000 7f5fb79e5740 1 -- start start 2024-04-21T08:43:20.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.267+0000 7f5fb79e5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135d93b0 0x5568135e4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.267+0000 7f5fb79e5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568135d9d60 0x5568135e5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.267+0000 7f5fb79e5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568135e2e60 0x5568135d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.267+0000 7f5fb79e5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb01ad0e0 con 0x5568135d93b0 2024-04-21T08:43:20.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.267+0000 7f5fb79e5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb01aec20 con 0x5568135d9d60 2024-04-21T08:43:20.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.267+0000 7f5fb79e5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb01ad7b0 con 0x5568135e2e60 2024-04-21T08:43:20.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.267+0000 7f5fae59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135d93b0 0x5568135e4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.267+0000 7f5fae59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135d93b0 0x5568135e4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57964/0 (socket says 172.21.15.43:57964) 2024-04-21T08:43:20.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.267+0000 7f5fae59c640 1 -- 172.21.15.43:0/2945270725 learned_addr learned my addr 172.21.15.43:0/2945270725 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:20.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.267+0000 7f5faed9d640 1 --2- 172.21.15.43:0/2945270725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568135e2e60 0x5568135d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.267+0000 7f5fadd9b640 1 --2- 172.21.15.43:0/2945270725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568135d9d60 0x5568135e5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.267+0000 7f5fae59c640 1 -- 172.21.15.43:0/2945270725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568135e2e60 msgr2=0x5568135d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:20.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.267+0000 7f5fae59c640 1 --2- 172.21.15.43:0/2945270725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568135e2e60 0x5568135d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.267+0000 7f5fae59c640 1 -- 172.21.15.43:0/2945270725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568135d9d60 msgr2=0x5568135e5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:20.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.267+0000 7f5fae59c640 1 --2- 172.21.15.43:0/2945270725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568135d9d60 0x5568135e5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.267+0000 7f5fae59c640 1 -- 172.21.15.43:0/2945270725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568135e6f00 con 0x5568135d93b0 2024-04-21T08:43:20.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.267+0000 7f5fadd9b640 1 --2- 172.21.15.43:0/2945270725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568135d9d60 0x5568135e5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:20.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.267+0000 7f5faed9d640 1 --2- 172.21.15.43:0/2945270725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568135e2e60 0x5568135d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:20.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.268+0000 7f5fae59c640 1 --2- 172.21.15.43:0/2945270725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135d93b0 0x5568135e4250 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000bbd0 tx=0x7f5fa000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6d35430dd40569 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:20.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.268+0000 7f5f97fff640 1 -- 172.21.15.43:0/2945270725 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa001c6f0 con 0x5568135d93b0 2024-04-21T08:43:20.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.268+0000 7f5f97fff640 1 -- 172.21.15.43:0/2945270725 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fa001c890 con 0x5568135d93b0 2024-04-21T08:43:20.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.268+0000 7f5f97fff640 1 -- 172.21.15.43:0/2945270725 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa000e6a0 con 0x5568135d93b0 2024-04-21T08:43:20.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.268+0000 7f5fb79e5740 1 -- 172.21.15.43:0/2945270725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135d93b0 msgr2=0x5568135e4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:20.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.268+0000 7f5fb79e5740 1 --2- 172.21.15.43:0/2945270725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135d93b0 0x5568135e4250 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000bbd0 tx=0x7f5fa000f7f0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.268+0000 7f5fb79e5740 1 -- 172.21.15.43:0/2945270725 shutdown_connections 2024-04-21T08:43:20.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.268+0000 7f5fb79e5740 1 --2- 172.21.15.43:0/2945270725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568135e2e60 0x5568135d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.268+0000 7f5fb79e5740 1 --2- 172.21.15.43:0/2945270725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568135d9d60 0x5568135e5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.268+0000 7f5fb79e5740 1 --2- 172.21.15.43:0/2945270725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135d93b0 0x5568135e4250 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.268+0000 7f5fb79e5740 1 -- 172.21.15.43:0/2945270725 >> 172.21.15.43:0/2945270725 conn(0x5568135d3ba0 msgr2=0x5568135e4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:20.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.268+0000 7f5fb79e5740 1 -- 172.21.15.43:0/2945270725 shutdown_connections 2024-04-21T08:43:20.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.268+0000 7f5fb79e5740 1 -- 172.21.15.43:0/2945270725 wait complete. 2024-04-21T08:43:20.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.269+0000 7f5fb79e5740 1 -- start start 2024-04-21T08:43:20.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.269+0000 7f5fb79e5740 1 Processor -- start 2024-04-21T08:43:20.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.269+0000 7f5fb79e5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568135d93b0 0x5568135f9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.269+0000 7f5fb79e5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568135d9d60 0x5568135fa510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.269+0000 7f5fb79e5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135e2e60 0x5568135faa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.269+0000 7f5fb79e5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb01ae550 con 0x5568135e2e60 2024-04-21T08:43:20.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.269+0000 7f5fb79e5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb01ade80 con 0x5568135d9d60 2024-04-21T08:43:20.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.269+0000 7f5fb79e5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb01ab0c0 con 0x5568135d93b0 2024-04-21T08:43:20.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.269+0000 7f5fb79e5740 10 client.0 _ll_register_callbacks cb 0x7f5f9800be50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:20.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.269+0000 7f5faed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135e2e60 0x5568135faa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.269+0000 7f5faed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135e2e60 0x5568135faa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57978/0 (socket says 172.21.15.43:57978) 2024-04-21T08:43:20.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.269+0000 7f5faed9d640 1 -- 172.21.15.43:0/1821942617 learned_addr learned my addr 172.21.15.43:0/1821942617 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:20.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.269+0000 7f5fae59c640 1 --2- 172.21.15.43:0/1821942617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568135d93b0 0x5568135f9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.269+0000 7f5fadd9b640 1 --2- 172.21.15.43:0/1821942617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568135d9d60 0x5568135fa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.269+0000 7f5faed9d640 1 -- 172.21.15.43:0/1821942617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568135d93b0 msgr2=0x5568135f9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:20.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.269+0000 7f5faed9d640 1 --2- 172.21.15.43:0/1821942617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568135d93b0 0x5568135f9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.269+0000 7f5faed9d640 1 -- 172.21.15.43:0/1821942617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568135d9d60 msgr2=0x5568135fa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:20.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.269+0000 7f5faed9d640 1 --2- 172.21.15.43:0/1821942617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568135d9d60 0x5568135fa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.269+0000 7f5faed9d640 1 -- 172.21.15.43:0/1821942617 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55681365c370 con 0x5568135e2e60 2024-04-21T08:43:20.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.270+0000 7f5faed9d640 1 --2- 172.21.15.43:0/1821942617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135e2e60 0x5568135faa50 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8033000 tx=0x7f5fa80be8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ef018032db4205 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:20.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.270+0000 7f5f95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:20.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.270+0000 7f5f95ffb640 1 -- 172.21.15.43:0/1821942617 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa80326c0 con 0x5568135e2e60 2024-04-21T08:43:20.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.270+0000 7f5f95ffb640 1 -- 172.21.15.43:0/1821942617 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f5fa80f06c0 con 0x5568135e2e60 2024-04-21T08:43:20.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.270+0000 7f5fb79e5740 1 -- 172.21.15.43:0/1821942617 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fa001c890 con 0x5568135e2e60 2024-04-21T08:43:20.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.270+0000 7f5f95ffb640 1 -- 172.21.15.43:0/1821942617 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5fa80f08c0 con 0x5568135e2e60 2024-04-21T08:43:20.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.270+0000 7f5fb79e5740 10 client.17577 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:20.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.270+0000 7f5fb79e5740 1 -- 172.21.15.43:0/1821942617 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55681365d620 con 0x5568135e2e60 2024-04-21T08:43:20.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.270+0000 7f5f95ffb640 1 -- 172.21.15.43:0/1821942617 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fa001c890 con 0x5568135e2e60 2024-04-21T08:43:20.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.270+0000 7f5f95ffb640 1 -- 172.21.15.43:0/1821942617 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5fa803c030 con 0x5568135e2e60 2024-04-21T08:43:20.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.270+0000 7f5fb79e5740 10 client.17577 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:20.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.270+0000 7f5fb79e5740 10 client.17577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:20.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.270+0000 7f5fb79e5740 20 client.17577 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:20.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.270+0000 7f5fb79e5740 1 --2- 172.21.15.43:0/1821942617 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55681365dd40 0x556813660160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:20.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.270+0000 7f5fb79e5740 4 client.17577 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:43:20.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.270+0000 7f5fb79e5740 1 -- 172.21.15.43:0/1821942617 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55681354bc10 con 0x55681365dd40 2024-04-21T08:43:20.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.271+0000 7f5fae59c640 1 --2- 172.21.15.43:0/1821942617 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55681365dd40 0x556813660160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:20.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.271+0000 7f5f95ffb640 10 client.17577 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:20.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.271+0000 7f5fae59c640 1 --2- 172.21.15.43:0/1821942617 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55681365dd40 0x556813660160 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f130fcb9c980054e server_cookie=ec6ba86cee523b27 in_seq=0 out_seq=0 2024-04-21T08:43:20.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.271+0000 7f5f95ffb640 1 -- 172.21.15.43:0/1821942617 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55681354bc10 con 0x55681365dd40 2024-04-21T08:43:20.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.271+0000 7f5f95ffb640 10 client.17577 handle_command_reply: tid=0 2024-04-21T08:43:20.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.271+0000 7f5fb79e5740 1 client.17577 shutdown 2024-04-21T08:43:20.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.271+0000 7f5fb79e5740 1 -- 172.21.15.43:0/1821942617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135e2e60 msgr2=0x5568135faa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:20.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.271+0000 7f5fb79e5740 1 --2- 172.21.15.43:0/1821942617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135e2e60 0x5568135faa50 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8033000 tx=0x7f5fa80be8e0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.271+0000 7f5fb79e5740 1 -- 172.21.15.43:0/1821942617 shutdown_connections 2024-04-21T08:43:20.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.271+0000 7f5fb79e5740 1 --2- 172.21.15.43:0/1821942617 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55681365dd40 0x556813660160 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.271+0000 7f5fb79e5740 1 --2- 172.21.15.43:0/1821942617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568135e2e60 0x5568135faa50 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.271+0000 7f5f95ffb640 0 client.17577 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:20.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.271+0000 7f5fb79e5740 1 --2- 172.21.15.43:0/1821942617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568135d9d60 0x5568135fa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.271+0000 7f5fb79e5740 1 --2- 172.21.15.43:0/1821942617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568135d93b0 0x5568135f9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.271+0000 7f5fb79e5740 1 -- 172.21.15.43:0/1821942617 >> 172.21.15.43:0/1821942617 conn(0x5568135d3ba0 msgr2=0x5568135d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:20.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.271+0000 7f5fb79e5740 1 -- 172.21.15.43:0/1821942617 shutdown_connections 2024-04-21T08:43:20.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.271+0000 7f5fb79e5740 1 -- 172.21.15.43:0/1821942617 wait complete. 2024-04-21T08:43:20.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.271+0000 7f5fb79e5740 20 client.17577 trim_cache size 0 max 16384 2024-04-21T08:43:20.278 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:43:20.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.272+0000 7f5fb4df4640 1 -- 172.21.15.43:0/3387410874 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5f9c0a3d20 msgr2=0x7f5f9c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:20.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.272+0000 7f5fb4df4640 1 --2- 172.21.15.43:0/3387410874 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5f9c0a3d20 0x7f5f9c0a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5f980138f0 tx=0x7f5f9800bac0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.272+0000 7f5fb4df4640 1 -- 172.21.15.43:0/3387410874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fb0199f70 msgr2=0x7f5fb01a4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:20.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.272+0000 7f5fb4df4640 1 --2- 172.21.15.43:0/3387410874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fb0199f70 0x7f5fb01a4520 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80098d0 tx=0x7f5fa8035d60 comp rx=0 tx=0).stop 2024-04-21T08:43:20.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.272+0000 7f5fb4df4640 1 -- 172.21.15.43:0/3387410874 shutdown_connections 2024-04-21T08:43:20.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.272+0000 7f5fb4df4640 1 --2- 172.21.15.43:0/3387410874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fb0199f70 0x7f5fb01a4520 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.272+0000 7f5fb4df4640 1 --2- 172.21.15.43:0/3387410874 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5f9c0a3d20 0x7f5f9c0a61e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.272+0000 7f5fb4df4640 1 --2- 172.21.15.43:0/3387410874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fb0106170 0x7f5fb01999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.272+0000 7f5fb4df4640 1 --2- 172.21.15.43:0/3387410874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fb01010a0 0x7f5fb01994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:20.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.272+0000 7f5fb4df4640 1 -- 172.21.15.43:0/3387410874 >> 172.21.15.43:0/3387410874 conn(0x7f5fb0077eb0 msgr2=0x7f5fb00fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:20.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.272+0000 7f5fb4df4640 1 -- 172.21.15.43:0/3387410874 shutdown_connections 2024-04-21T08:43:20.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:20.272+0000 7f5fb4df4640 1 -- 172.21.15.43:0/3387410874 wait complete. 2024-04-21T08:43:20.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:20 smithi112 ceph-mon[31085]: pgmap v135: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-04-21T08:43:20.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:20 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2994004231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:43:20.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:20 smithi043 ceph-mon[27978]: pgmap v135: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-04-21T08:43:20.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:20 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2994004231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:43:20.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:20 smithi175 ceph-mon[31920]: pgmap v135: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-04-21T08:43:20.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:20 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2994004231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:43:21.283 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:43:21.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:43:21.283 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:43:21.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.404+0000 7f8fe34d2640 1 -- 172.21.15.43:0/2080228739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fdc10e960 msgr2=0x7f8fdc110d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.404+0000 7f8fe34d2640 1 --2- 172.21.15.43:0/2080228739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fdc10e960 0x7f8fdc110d50 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4009ca0 tx=0x7f8fd402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.404+0000 7f8fe34d2640 1 -- 172.21.15.43:0/2080228739 shutdown_connections 2024-04-21T08:43:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.404+0000 7f8fe34d2640 1 --2- 172.21.15.43:0/2080228739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fdc10e960 0x7f8fdc110d50 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.404+0000 7f8fe34d2640 1 --2- 172.21.15.43:0/2080228739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fdc06aef0 0x7f8fdc10e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.404+0000 7f8fe34d2640 1 --2- 172.21.15.43:0/2080228739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fdc06a5d0 0x7f8fdc06a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.404+0000 7f8fe34d2640 1 -- 172.21.15.43:0/2080228739 >> 172.21.15.43:0/2080228739 conn(0x7f8fdc077eb0 msgr2=0x7f8fdc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.404+0000 7f8fe34d2640 1 -- 172.21.15.43:0/2080228739 shutdown_connections 2024-04-21T08:43:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.404+0000 7f8fe34d2640 1 -- 172.21.15.43:0/2080228739 wait complete. 2024-04-21T08:43:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.405+0000 7f8fe34d2640 1 Processor -- start 2024-04-21T08:43:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.405+0000 7f8fe34d2640 1 -- start start 2024-04-21T08:43:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.405+0000 7f8fe34d2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fdc06a5d0 0x7f8fdc1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.405+0000 7f8fe34d2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fdc06aef0 0x7f8fdc1a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.405+0000 7f8fe34d2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fdc10e960 0x7f8fdc19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.405+0000 7f8fe34d2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8fdc114390 con 0x7f8fdc06aef0 2024-04-21T08:43:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.405+0000 7f8fe34d2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8fdc114210 con 0x7f8fdc10e960 2024-04-21T08:43:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.405+0000 7f8fe34d2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8fdc114510 con 0x7f8fdc06a5d0 2024-04-21T08:43:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.405+0000 7f8fe0a6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fdc06aef0 0x7f8fdc1a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.405+0000 7f8fe0a6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fdc06aef0 0x7f8fdc1a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58002/0 (socket says 172.21.15.43:58002) 2024-04-21T08:43:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.405+0000 7f8fe0a6d640 1 -- 172.21.15.43:0/2180575225 learned_addr learned my addr 172.21.15.43:0/2180575225 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.405+0000 7f8fe126e640 1 --2- 172.21.15.43:0/2180575225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fdc06a5d0 0x7f8fdc1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.405+0000 7f8fe1a6f640 1 --2- 172.21.15.43:0/2180575225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fdc10e960 0x7f8fdc19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.405+0000 7f8fe0a6d640 1 -- 172.21.15.43:0/2180575225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fdc06a5d0 msgr2=0x7f8fdc1a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.405+0000 7f8fe0a6d640 1 --2- 172.21.15.43:0/2180575225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fdc06a5d0 0x7f8fdc1a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.405+0000 7f8fe0a6d640 1 -- 172.21.15.43:0/2180575225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fdc10e960 msgr2=0x7f8fdc19a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.405+0000 7f8fe0a6d640 1 --2- 172.21.15.43:0/2180575225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fdc10e960 0x7f8fdc19a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.405+0000 7f8fe0a6d640 1 -- 172.21.15.43:0/2180575225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fdc19af90 con 0x7f8fdc06aef0 2024-04-21T08:43:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.405+0000 7f8fe126e640 1 --2- 172.21.15.43:0/2180575225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fdc06a5d0 0x7f8fdc1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:21.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.406+0000 7f8fe0a6d640 1 --2- 172.21.15.43:0/2180575225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fdc06aef0 0x7f8fdc1a0b50 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00e9f0 tx=0x7f8fcc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:21.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.406+0000 7f8fd27fc640 1 -- 172.21.15.43:0/2180575225 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc00cd80 con 0x7f8fdc06aef0 2024-04-21T08:43:21.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.406+0000 7f8fd27fc640 1 -- 172.21.15.43:0/2180575225 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fcc0108e0 con 0x7f8fdc06aef0 2024-04-21T08:43:21.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.406+0000 7f8fe34d2640 1 -- 172.21.15.43:0/2180575225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fdc19b280 con 0x7f8fdc06aef0 2024-04-21T08:43:21.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.406+0000 7f8fd27fc640 1 -- 172.21.15.43:0/2180575225 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc018980 con 0x7f8fdc06aef0 2024-04-21T08:43:21.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.406+0000 7f8fe34d2640 1 -- 172.21.15.43:0/2180575225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fdc1a7590 con 0x7f8fdc06aef0 2024-04-21T08:43:21.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.408+0000 7f8fd27fc640 1 -- 172.21.15.43:0/2180575225 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8fcc010430 con 0x7f8fdc06aef0 2024-04-21T08:43:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.409+0000 7f8fe61a1740 1 Processor -- start 2024-04-21T08:43:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.409+0000 7f8fe61a1740 1 -- start start 2024-04-21T08:43:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.409+0000 7f8fe61a1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cf4e708e0 0x556cf4e70cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.409+0000 7f8fe61a1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e71290 0x556cf4e79840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.409+0000 7f8fe61a1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cf4e79e10 0x556cf4e7c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.409+0000 7f8fe61a1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556cf4dd4f00 con 0x556cf4e71290 2024-04-21T08:43:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.409+0000 7f8fe61a1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556cf4d72590 con 0x556cf4e79e10 2024-04-21T08:43:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.409+0000 7f8fe61a1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556cf4dd0ec0 con 0x556cf4e708e0 2024-04-21T08:43:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.409+0000 7f8fe126e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e71290 0x556cf4e79840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.409+0000 7f8fe126e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e71290 0x556cf4e79840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58008/0 (socket says 172.21.15.43:58008) 2024-04-21T08:43:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.409+0000 7f8fe126e640 1 -- 172.21.15.43:0/2592849125 learned_addr learned my addr 172.21.15.43:0/2592849125 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.409+0000 7f8fe1a6f640 1 --2- 172.21.15.43:0/2592849125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cf4e708e0 0x556cf4e70cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.409+0000 7f8fe0a6d640 1 --2- 172.21.15.43:0/2592849125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cf4e79e10 0x556cf4e7c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.409+0000 7f8fd27fc640 1 --2- 172.21.15.43:0/2180575225 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8fc80a3da0 0x7f8fc80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.409+0000 7f8fe126e640 1 -- 172.21.15.43:0/2592849125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cf4e708e0 msgr2=0x556cf4e70cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.409+0000 7f8fe126e640 1 --2- 172.21.15.43:0/2592849125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cf4e708e0 0x556cf4e70cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.409+0000 7f8fe126e640 1 -- 172.21.15.43:0/2592849125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cf4e79e10 msgr2=0x556cf4e7c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.409+0000 7f8fe126e640 1 --2- 172.21.15.43:0/2592849125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cf4e79e10 0x556cf4e7c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.409+0000 7f8fe126e640 1 -- 172.21.15.43:0/2592849125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cf4baf7e0 con 0x556cf4e71290 2024-04-21T08:43:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.410+0000 7f8fe1a6f640 1 --2- 172.21.15.43:0/2180575225 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8fc80a3da0 0x7f8fc80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.410+0000 7f8fd27fc640 1 -- 172.21.15.43:0/2180575225 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f8fcc0d0f00 con 0x7f8fdc06aef0 2024-04-21T08:43:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.410+0000 7f8fe126e640 1 --2- 172.21.15.43:0/2592849125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e71290 0x556cf4e79840 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800bcc0 tx=0x7f8fd800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49edda24159a30f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.410+0000 7f8fc77fe640 1 -- 172.21.15.43:0/2592849125 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8005020 con 0x556cf4e71290 2024-04-21T08:43:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.410+0000 7f8fc77fe640 1 -- 172.21.15.43:0/2592849125 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fd80047a0 con 0x556cf4e71290 2024-04-21T08:43:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.410+0000 7f8fc77fe640 1 -- 172.21.15.43:0/2592849125 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8011680 con 0x556cf4e71290 2024-04-21T08:43:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.411+0000 7f8fe61a1740 1 -- 172.21.15.43:0/2592849125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e71290 msgr2=0x556cf4e79840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.411+0000 7f8fe61a1740 1 --2- 172.21.15.43:0/2592849125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e71290 0x556cf4e79840 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800bcc0 tx=0x7f8fd800f730 comp rx=0 tx=0).stop 2024-04-21T08:43:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.411+0000 7f8fe1a6f640 1 --2- 172.21.15.43:0/2180575225 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8fc80a3da0 0x7f8fc80a6260 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4006010 tx=0x7f8fd402e040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.411+0000 7f8fe61a1740 1 -- 172.21.15.43:0/2592849125 shutdown_connections 2024-04-21T08:43:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.411+0000 7f8fe61a1740 1 --2- 172.21.15.43:0/2592849125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cf4e79e10 0x556cf4e7c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.411+0000 7f8fe61a1740 1 --2- 172.21.15.43:0/2592849125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e71290 0x556cf4e79840 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.411+0000 7f8fe61a1740 1 --2- 172.21.15.43:0/2592849125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cf4e708e0 0x556cf4e70cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.411+0000 7f8fe61a1740 1 -- 172.21.15.43:0/2592849125 >> 172.21.15.43:0/2592849125 conn(0x556cf4e6afb0 msgr2=0x556cf4e6b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.411+0000 7f8fe61a1740 1 -- 172.21.15.43:0/2592849125 shutdown_connections 2024-04-21T08:43:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.411+0000 7f8fe61a1740 1 -- 172.21.15.43:0/2592849125 wait complete. 2024-04-21T08:43:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.411+0000 7f8fe61a1740 1 -- start start 2024-04-21T08:43:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.411+0000 7f8fe61a1740 1 Processor -- start 2024-04-21T08:43:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.412+0000 7f8fe61a1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cf4e708e0 0x556cf4e76da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.412+0000 7f8fe61a1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e71290 0x556cf4e772e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.412+0000 7f8fe61a1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cf4e79e10 0x556cf4e77820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.412+0000 7f8fe61a1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556cf4e7cd80 con 0x556cf4e71290 2024-04-21T08:43:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.412+0000 7f8fe61a1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556cf4d8e580 con 0x556cf4e708e0 2024-04-21T08:43:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.412+0000 7f8fe61a1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556cf4e7cf00 con 0x556cf4e79e10 2024-04-21T08:43:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.412+0000 7f8fe61a1740 10 client.0 _ll_register_callbacks cb 0x556cf4d94650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.412+0000 7f8fe0a6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e71290 0x556cf4e772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.412+0000 7f8fe0a6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e71290 0x556cf4e772e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58022/0 (socket says 172.21.15.43:58022) 2024-04-21T08:43:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.412+0000 7f8fe0a6d640 1 -- 172.21.15.43:0/498042296 learned_addr learned my addr 172.21.15.43:0/498042296 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.412+0000 7f8fe1a6f640 1 --2- 172.21.15.43:0/498042296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cf4e79e10 0x556cf4e77820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.412+0000 7f8fe126e640 1 --2- 172.21.15.43:0/498042296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cf4e708e0 0x556cf4e76da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.412+0000 7f8fe0a6d640 1 -- 172.21.15.43:0/498042296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cf4e79e10 msgr2=0x556cf4e77820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.412+0000 7f8fe0a6d640 1 --2- 172.21.15.43:0/498042296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cf4e79e10 0x556cf4e77820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.412+0000 7f8fe0a6d640 1 -- 172.21.15.43:0/498042296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cf4e708e0 msgr2=0x556cf4e76da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.412+0000 7f8fe0a6d640 1 --2- 172.21.15.43:0/498042296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cf4e708e0 0x556cf4e76da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.412+0000 7f8fe0a6d640 1 -- 172.21.15.43:0/498042296 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cf4c0ff70 con 0x556cf4e71290 2024-04-21T08:43:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.412+0000 7f8fe0a6d640 1 --2- 172.21.15.43:0/498042296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e71290 0x556cf4e772e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00e9c0 tx=0x7f8fcc0d4640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b611398571773ca server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.412+0000 7f8fc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.412+0000 7f8fc4ff9640 1 -- 172.21.15.43:0/498042296 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc09c430 con 0x556cf4e71290 2024-04-21T08:43:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.412+0000 7f8fc4ff9640 1 -- 172.21.15.43:0/498042296 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f8fcc099140 con 0x556cf4e71290 2024-04-21T08:43:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.412+0000 7f8fe61a1740 1 -- 172.21.15.43:0/498042296 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fd80047a0 con 0x556cf4e71290 2024-04-21T08:43:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.413+0000 7f8fc4ff9640 1 -- 172.21.15.43:0/498042296 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8fcc010720 con 0x556cf4e71290 2024-04-21T08:43:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.413+0000 7f8fe61a1740 10 client.17589 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.413+0000 7f8fe61a1740 1 -- 172.21.15.43:0/498042296 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556cf4aa7170 con 0x556cf4e71290 2024-04-21T08:43:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.413+0000 7f8fc4ff9640 1 -- 172.21.15.43:0/498042296 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fd80047a0 con 0x556cf4e71290 2024-04-21T08:43:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.413+0000 7f8fc4ff9640 1 -- 172.21.15.43:0/498042296 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8fcc0a52b0 con 0x556cf4e71290 2024-04-21T08:43:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.413+0000 7f8fe61a1740 10 client.17589 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.413+0000 7f8fe61a1740 10 client.17589 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.413+0000 7f8fe61a1740 20 client.17589 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.413+0000 7f8fe61a1740 1 --2- 172.21.15.43:0/498042296 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556cf4e8a270 0x556cf4e8c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.413+0000 7f8fe61a1740 4 client.17589 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.413+0000 7f8fe61a1740 1 -- 172.21.15.43:0/498042296 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cf4de2920 con 0x556cf4e8a270 2024-04-21T08:43:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.413+0000 7f8fe126e640 1 --2- 172.21.15.43:0/498042296 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556cf4e8a270 0x556cf4e8c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.414+0000 7f8fe126e640 1 --2- 172.21.15.43:0/498042296 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556cf4e8a270 0x556cf4e8c660 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9758f13143247399 server_cookie=8ec36917434855d in_seq=0 out_seq=0 2024-04-21T08:43:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.414+0000 7f8fc4ff9640 10 client.17589 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.414+0000 7f8fc4ff9640 1 -- 172.21.15.43:0/498042296 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556cf4de2920 con 0x556cf4e8a270 2024-04-21T08:43:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.414+0000 7f8fc4ff9640 10 client.17589 handle_command_reply: tid=0 2024-04-21T08:43:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.414+0000 7f8fe61a1740 1 client.17589 shutdown 2024-04-21T08:43:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.415+0000 7f8fe61a1740 1 -- 172.21.15.43:0/498042296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e71290 msgr2=0x556cf4e772e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.415+0000 7f8fe61a1740 1 --2- 172.21.15.43:0/498042296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e71290 0x556cf4e772e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00e9c0 tx=0x7f8fcc0d4640 comp rx=0 tx=0).stop 2024-04-21T08:43:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.415+0000 7f8fe61a1740 1 -- 172.21.15.43:0/498042296 shutdown_connections 2024-04-21T08:43:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.415+0000 7f8fe61a1740 1 --2- 172.21.15.43:0/498042296 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556cf4e8a270 0x556cf4e8c660 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.415+0000 7f8fe61a1740 1 --2- 172.21.15.43:0/498042296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cf4e79e10 0x556cf4e77820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.415+0000 7f8fc4ff9640 0 client.17589 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.415+0000 7f8fe61a1740 1 --2- 172.21.15.43:0/498042296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e71290 0x556cf4e772e0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.415+0000 7f8fe61a1740 1 --2- 172.21.15.43:0/498042296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cf4e708e0 0x556cf4e76da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.415+0000 7f8fe61a1740 1 -- 172.21.15.43:0/498042296 >> 172.21.15.43:0/498042296 conn(0x556cf4e6afb0 msgr2=0x556cf4e6cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.415+0000 7f8fe61a1740 1 -- 172.21.15.43:0/498042296 shutdown_connections 2024-04-21T08:43:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.415+0000 7f8fe61a1740 1 -- 172.21.15.43:0/498042296 wait complete. 2024-04-21T08:43:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.415+0000 7f8fe61a1740 20 client.17589 trim_cache size 0 max 16384 2024-04-21T08:43:21.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.433+0000 7f8fe61a1740 1 Processor -- start 2024-04-21T08:43:21.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.433+0000 7f8fe61a1740 1 -- start start 2024-04-21T08:43:21.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.433+0000 7f8fe61a1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cf4e708e0 0x556cf4e794d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.433+0000 7f8fe61a1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cf4e71290 0x556cf4e85c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.433+0000 7f8fe61a1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e79e10 0x556cf4e6cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.433+0000 7f8fe61a1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8fdc1b1740 con 0x556cf4e79e10 2024-04-21T08:43:21.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.433+0000 7f8fe61a1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8fdc1b3280 con 0x556cf4e71290 2024-04-21T08:43:21.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.433+0000 7f8fe61a1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8fdc1b1e10 con 0x556cf4e708e0 2024-04-21T08:43:21.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.433+0000 7f8fe1a6f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e79e10 0x556cf4e6cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.433+0000 7f8fe1a6f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e79e10 0x556cf4e6cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58024/0 (socket says 172.21.15.43:58024) 2024-04-21T08:43:21.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.433+0000 7f8fe1a6f640 1 -- 172.21.15.43:0/52878500 learned_addr learned my addr 172.21.15.43:0/52878500 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:21.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.433+0000 7f8fe0a6d640 1 --2- 172.21.15.43:0/52878500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cf4e71290 0x556cf4e85c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.433+0000 7f8fe1a6f640 1 -- 172.21.15.43:0/52878500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cf4e708e0 msgr2=0x556cf4e794d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:21.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.433+0000 7f8fe1a6f640 1 --2- 172.21.15.43:0/52878500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cf4e708e0 0x556cf4e794d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.433+0000 7f8fe1a6f640 1 -- 172.21.15.43:0/52878500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cf4e71290 msgr2=0x556cf4e85c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:21.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.433+0000 7f8fe1a6f640 1 --2- 172.21.15.43:0/52878500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cf4e71290 0x556cf4e85c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.433+0000 7f8fe1a6f640 1 -- 172.21.15.43:0/52878500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cf4e8c9d0 con 0x556cf4e79e10 2024-04-21T08:43:21.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.434+0000 7f8fe1a6f640 1 --2- 172.21.15.43:0/52878500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e79e10 0x556cf4e6cb10 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4009ca0 tx=0x7f8fd4040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24589578d61886e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:21.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.434+0000 7f8fc6ffd640 1 -- 172.21.15.43:0/52878500 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4039040 con 0x556cf4e79e10 2024-04-21T08:43:21.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.434+0000 7f8fc6ffd640 1 -- 172.21.15.43:0/52878500 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fd4052070 con 0x556cf4e79e10 2024-04-21T08:43:21.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.434+0000 7f8fc6ffd640 1 -- 172.21.15.43:0/52878500 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4004560 con 0x556cf4e79e10 2024-04-21T08:43:21.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.434+0000 7f8fe61a1740 1 -- 172.21.15.43:0/52878500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e79e10 msgr2=0x556cf4e6cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:21.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.434+0000 7f8fe61a1740 1 --2- 172.21.15.43:0/52878500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e79e10 0x556cf4e6cb10 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4009ca0 tx=0x7f8fd4040000 comp rx=0 tx=0).stop 2024-04-21T08:43:21.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.434+0000 7f8fe61a1740 1 -- 172.21.15.43:0/52878500 shutdown_connections 2024-04-21T08:43:21.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.434+0000 7f8fe61a1740 1 --2- 172.21.15.43:0/52878500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e79e10 0x556cf4e6cb10 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.434+0000 7f8fe61a1740 1 --2- 172.21.15.43:0/52878500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cf4e71290 0x556cf4e85c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.434+0000 7f8fe61a1740 1 --2- 172.21.15.43:0/52878500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cf4e708e0 0x556cf4e794d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.434+0000 7f8fe61a1740 1 -- 172.21.15.43:0/52878500 >> 172.21.15.43:0/52878500 conn(0x556cf4e6afb0 msgr2=0x556cf4e6c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:21.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.434+0000 7f8fe61a1740 1 -- 172.21.15.43:0/52878500 shutdown_connections 2024-04-21T08:43:21.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.434+0000 7f8fe61a1740 1 -- 172.21.15.43:0/52878500 wait complete. 2024-04-21T08:43:21.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.435+0000 7f8fe61a1740 1 -- start start 2024-04-21T08:43:21.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.435+0000 7f8fe61a1740 1 Processor -- start 2024-04-21T08:43:21.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.435+0000 7f8fe61a1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cf4e708e0 0x556cf4e7e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.435+0000 7f8fe61a1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e71290 0x556cf4e7e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.435+0000 7f8fe61a1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cf4e79e10 0x556cf4ee97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.435+0000 7f8fe61a1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8fdc1b2bb0 con 0x556cf4e71290 2024-04-21T08:43:21.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.435+0000 7f8fe61a1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8fdc1b24e0 con 0x556cf4e708e0 2024-04-21T08:43:21.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.435+0000 7f8fe61a1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8fdc1af720 con 0x556cf4e79e10 2024-04-21T08:43:21.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.435+0000 7f8fe61a1740 10 client.0 _ll_register_callbacks cb 0x7f8fcc0d4740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:21.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.435+0000 7f8fe0a6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e71290 0x556cf4e7e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.435+0000 7f8fe0a6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e71290 0x556cf4e7e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58040/0 (socket says 172.21.15.43:58040) 2024-04-21T08:43:21.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.435+0000 7f8fe0a6d640 1 -- 172.21.15.43:0/968978037 learned_addr learned my addr 172.21.15.43:0/968978037 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:21.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.435+0000 7f8fe126e640 1 --2- 172.21.15.43:0/968978037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cf4e708e0 0x556cf4e7e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.436+0000 7f8fe0a6d640 1 -- 172.21.15.43:0/968978037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cf4e79e10 msgr2=0x556cf4ee97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:21.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.436+0000 7f8fe1a6f640 1 --2- 172.21.15.43:0/968978037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cf4e79e10 0x556cf4ee97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.436+0000 7f8fe0a6d640 1 --2- 172.21.15.43:0/968978037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cf4e79e10 0x556cf4ee97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.436+0000 7f8fe0a6d640 1 -- 172.21.15.43:0/968978037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cf4e708e0 msgr2=0x556cf4e7e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:21.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.436+0000 7f8fe0a6d640 1 --2- 172.21.15.43:0/968978037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cf4e708e0 0x556cf4e7e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.436+0000 7f8fe0a6d640 1 -- 172.21.15.43:0/968978037 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cf4ee9d80 con 0x556cf4e71290 2024-04-21T08:43:21.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.436+0000 7f8fe126e640 1 --2- 172.21.15.43:0/968978037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cf4e708e0 0x556cf4e7e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:21.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.436+0000 7f8fe1a6f640 1 --2- 172.21.15.43:0/968978037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cf4e79e10 0x556cf4ee97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:21.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.436+0000 7f8fe0a6d640 1 --2- 172.21.15.43:0/968978037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e71290 0x556cf4e7e980 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0bd0b0 tx=0x7f8fcc009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db217139f49eda07 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:21.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.436+0000 7f8fc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:21.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.436+0000 7f8fc4ff9640 1 -- 172.21.15.43:0/968978037 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc098d70 con 0x556cf4e71290 2024-04-21T08:43:21.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.436+0000 7f8fc4ff9640 1 -- 172.21.15.43:0/968978037 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f8fcc09bf20 con 0x556cf4e71290 2024-04-21T08:43:21.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.436+0000 7f8fe61a1740 1 -- 172.21.15.43:0/968978037 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fd4052070 con 0x556cf4e71290 2024-04-21T08:43:21.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.436+0000 7f8fc4ff9640 1 -- 172.21.15.43:0/968978037 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8fcc09cad0 con 0x556cf4e71290 2024-04-21T08:43:21.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.436+0000 7f8fe61a1740 10 client.17595 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:21.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.436+0000 7f8fe61a1740 1 -- 172.21.15.43:0/968978037 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556cf4ef1170 con 0x556cf4e71290 2024-04-21T08:43:21.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.436+0000 7f8fc4ff9640 1 -- 172.21.15.43:0/968978037 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fd4052070 con 0x556cf4e71290 2024-04-21T08:43:21.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.436+0000 7f8fc4ff9640 1 -- 172.21.15.43:0/968978037 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8fcc09b990 con 0x556cf4e71290 2024-04-21T08:43:21.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.436+0000 7f8fe61a1740 10 client.17595 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:21.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.436+0000 7f8fe61a1740 10 client.17595 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:21.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.436+0000 7f8fe61a1740 20 client.17595 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:21.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.436+0000 7f8fe61a1740 1 --2- 172.21.15.43:0/968978037 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556cf4e8a270 0x556cf4ef38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.436+0000 7f8fe61a1740 4 client.17595 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:43:21.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.436+0000 7f8fe61a1740 1 -- 172.21.15.43:0/968978037 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556cf4ddc790 con 0x556cf4e8a270 2024-04-21T08:43:21.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.437+0000 7f8fe126e640 1 --2- 172.21.15.43:0/968978037 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556cf4e8a270 0x556cf4ef38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.437+0000 7f8fe126e640 1 --2- 172.21.15.43:0/968978037 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556cf4e8a270 0x556cf4ef38d0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbb2f457f4c70adf server_cookie=ce00506bb541a55e in_seq=0 out_seq=0 2024-04-21T08:43:21.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.437+0000 7f8fc4ff9640 10 client.17595 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:21.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.438+0000 7f8fc4ff9640 1 -- 172.21.15.43:0/968978037 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556cf4ddc790 con 0x556cf4e8a270 2024-04-21T08:43:21.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.438+0000 7f8fc4ff9640 10 client.17595 handle_command_reply: tid=0 2024-04-21T08:43:21.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.438+0000 7f8fe61a1740 1 client.17595 shutdown 2024-04-21T08:43:21.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.438+0000 7f8fe61a1740 1 -- 172.21.15.43:0/968978037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e71290 msgr2=0x556cf4e7e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:21.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.438+0000 7f8fe61a1740 1 --2- 172.21.15.43:0/968978037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e71290 0x556cf4e7e980 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0bd0b0 tx=0x7f8fcc009700 comp rx=0 tx=0).stop 2024-04-21T08:43:21.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.438+0000 7f8fe61a1740 1 -- 172.21.15.43:0/968978037 shutdown_connections 2024-04-21T08:43:21.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.438+0000 7f8fe61a1740 1 --2- 172.21.15.43:0/968978037 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556cf4e8a270 0x556cf4ef38d0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.438+0000 7f8fe61a1740 1 --2- 172.21.15.43:0/968978037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cf4e79e10 0x556cf4ee97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.438+0000 7f8fe61a1740 1 --2- 172.21.15.43:0/968978037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556cf4e71290 0x556cf4e7e980 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.438+0000 7f8fe61a1740 1 --2- 172.21.15.43:0/968978037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cf4e708e0 0x556cf4e7e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.438+0000 7f8fc4ff9640 0 client.17595 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:21.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.438+0000 7f8fe61a1740 1 -- 172.21.15.43:0/968978037 >> 172.21.15.43:0/968978037 conn(0x556cf4e6afb0 msgr2=0x556cf4e6a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:21.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.439+0000 7f8fe61a1740 1 -- 172.21.15.43:0/968978037 shutdown_connections 2024-04-21T08:43:21.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.439+0000 7f8fe61a1740 1 -- 172.21.15.43:0/968978037 wait complete. 2024-04-21T08:43:21.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.439+0000 7f8fe61a1740 20 client.17595 trim_cache size 0 max 16384 2024-04-21T08:43:21.443 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:43:21.444 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:43:21.444 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "e8f38c4f-39de-4534-946c-66500b376291", 2024-04-21T08:43:21.444 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:43:21.444 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:43:21.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.439+0000 7f8fe34d2640 1 -- 172.21.15.43:0/2180575225 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8fc80a3da0 msgr2=0x7f8fc80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:21.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.439+0000 7f8fe34d2640 1 --2- 172.21.15.43:0/2180575225 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8fc80a3da0 0x7f8fc80a6260 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4006010 tx=0x7f8fd402e040 comp rx=0 tx=0).stop 2024-04-21T08:43:21.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.439+0000 7f8fe34d2640 1 -- 172.21.15.43:0/2180575225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fdc06aef0 msgr2=0x7f8fdc1a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:21.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.439+0000 7f8fe34d2640 1 --2- 172.21.15.43:0/2180575225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fdc06aef0 0x7f8fdc1a0b50 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00e9f0 tx=0x7f8fcc00eeb0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.439+0000 7f8fe34d2640 1 -- 172.21.15.43:0/2180575225 shutdown_connections 2024-04-21T08:43:21.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.439+0000 7f8fe34d2640 1 --2- 172.21.15.43:0/2180575225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fdc10e960 0x7f8fdc19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.439+0000 7f8fe34d2640 1 --2- 172.21.15.43:0/2180575225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fdc06aef0 0x7f8fdc1a0b50 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.439+0000 7f8fe34d2640 1 --2- 172.21.15.43:0/2180575225 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8fc80a3da0 0x7f8fc80a6260 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.439+0000 7f8fe34d2640 1 --2- 172.21.15.43:0/2180575225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fdc06a5d0 0x7f8fdc1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.439+0000 7f8fe34d2640 1 -- 172.21.15.43:0/2180575225 >> 172.21.15.43:0/2180575225 conn(0x7f8fdc077eb0 msgr2=0x7f8fdc10fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:21.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.439+0000 7f8fe34d2640 1 -- 172.21.15.43:0/2180575225 shutdown_connections 2024-04-21T08:43:21.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.439+0000 7f8fe34d2640 1 -- 172.21.15.43:0/2180575225 wait complete. 2024-04-21T08:43:21.450 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:43:21.450 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:43:21.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.555+0000 7fc05e4f2640 1 -- 172.21.15.43:0/1626646454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc058101a40 msgr2=0x7fc05810dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:21.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.555+0000 7fc05e4f2640 1 --2- 172.21.15.43:0/1626646454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc058101a40 0x7fc05810dca0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fc050009ca0 tx=0x7fc05002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.555+0000 7fc05e4f2640 1 -- 172.21.15.43:0/1626646454 shutdown_connections 2024-04-21T08:43:21.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.555+0000 7fc05e4f2640 1 --2- 172.21.15.43:0/1626646454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc058101a40 0x7fc05810dca0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.555+0000 7fc05e4f2640 1 --2- 172.21.15.43:0/1626646454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0581010a0 0x7fc058101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.555+0000 7fc05e4f2640 1 --2- 172.21.15.43:0/1626646454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc058105dd0 0x7fc0581061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.555+0000 7fc05e4f2640 1 -- 172.21.15.43:0/1626646454 >> 172.21.15.43:0/1626646454 conn(0x7fc058077c40 msgr2=0x7fc058078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:21.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.555+0000 7fc05e4f2640 1 -- 172.21.15.43:0/1626646454 shutdown_connections 2024-04-21T08:43:21.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.555+0000 7fc05e4f2640 1 -- 172.21.15.43:0/1626646454 wait complete. 2024-04-21T08:43:21.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.556+0000 7fc05e4f2640 1 Processor -- start 2024-04-21T08:43:21.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.556+0000 7fc05e4f2640 1 -- start start 2024-04-21T08:43:21.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.556+0000 7fc05e4f2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0581010a0 0x7fc0581994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.556+0000 7fc05e4f2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc058101a40 0x7fc058199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.556+0000 7fc05e4f2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc058105dd0 0x7fc0581a4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.556+0000 7fc05e4f2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc058111160 con 0x7fc058105dd0 2024-04-21T08:43:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.556+0000 7fc05e4f2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc058110fe0 con 0x7fc0581010a0 2024-04-21T08:43:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.556+0000 7fc05e4f2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc0581112e0 con 0x7fc058101a40 2024-04-21T08:43:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.557+0000 7fc05ca8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc058105dd0 0x7fc0581a4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.557+0000 7fc05ca8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc058105dd0 0x7fc0581a4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58054/0 (socket says 172.21.15.43:58054) 2024-04-21T08:43:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.557+0000 7fc05ca8f640 1 -- 172.21.15.43:0/1514539328 learned_addr learned my addr 172.21.15.43:0/1514539328 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.557+0000 7fc057fff640 1 --2- 172.21.15.43:0/1514539328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0581010a0 0x7fc0581994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.557+0000 7fc0577fe640 1 --2- 172.21.15.43:0/1514539328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc058101a40 0x7fc058199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.557+0000 7fc05ca8f640 1 -- 172.21.15.43:0/1514539328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc058101a40 msgr2=0x7fc058199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.557+0000 7fc05ca8f640 1 --2- 172.21.15.43:0/1514539328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc058101a40 0x7fc058199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.557+0000 7fc05ca8f640 1 -- 172.21.15.43:0/1514539328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0581010a0 msgr2=0x7fc0581994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.557+0000 7fc05ca8f640 1 --2- 172.21.15.43:0/1514539328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0581010a0 0x7fc0581994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.557+0000 7fc05ca8f640 1 -- 172.21.15.43:0/1514539328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0581a4a80 con 0x7fc058105dd0 2024-04-21T08:43:21.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.557+0000 7fc05ca8f640 1 --2- 172.21.15.43:0/1514539328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc058105dd0 0x7fc0581a4540 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fc050005e30 tx=0x7fc050032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:21.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.557+0000 7fc0557fa640 1 -- 172.21.15.43:0/1514539328 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc050041020 con 0x7fc058105dd0 2024-04-21T08:43:21.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.557+0000 7fc0557fa640 1 -- 172.21.15.43:0/1514539328 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc050032970 con 0x7fc058105dd0 2024-04-21T08:43:21.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.557+0000 7fc05e4f2640 1 -- 172.21.15.43:0/1514539328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0581a4c50 con 0x7fc058105dd0 2024-04-21T08:43:21.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.557+0000 7fc0557fa640 1 -- 172.21.15.43:0/1514539328 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc050039040 con 0x7fc058105dd0 2024-04-21T08:43:21.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.557+0000 7fc05e4f2640 1 -- 172.21.15.43:0/1514539328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0581a50b0 con 0x7fc058105dd0 2024-04-21T08:43:21.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.559+0000 7fc0557fa640 1 -- 172.21.15.43:0/1514539328 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc050044020 con 0x7fc058105dd0 2024-04-21T08:43:21.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.560+0000 7fc0611be740 1 Processor -- start 2024-04-21T08:43:21.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.560+0000 7fc0611be740 1 -- start start 2024-04-21T08:43:21.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.560+0000 7fc0611be740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476230 0x55f0bf476610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.560+0000 7fc0557fa640 1 --2- 172.21.15.43:0/1514539328 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc0400a3d20 0x7fc0400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.560+0000 7fc0611be740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0bf476b50 0x55f0bf483950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.560+0000 7fc0611be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0bf483e90 0x55f0bf486280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.560+0000 7fc0611be740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f0bf3db1e0 con 0x55f0bf476230 2024-04-21T08:43:21.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.560+0000 7fc0611be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f0bf378390 con 0x55f0bf483e90 2024-04-21T08:43:21.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.560+0000 7fc0611be740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f0bf3d71a0 con 0x55f0bf476b50 2024-04-21T08:43:21.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.560+0000 7fc0577fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476230 0x55f0bf476610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.560+0000 7fc057fff640 1 --2- 172.21.15.43:0/1514539328 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc0400a3d20 0x7fc0400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.560+0000 7fc0577fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476230 0x55f0bf476610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58056/0 (socket says 172.21.15.43:58056) 2024-04-21T08:43:21.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.560+0000 7fc0577fe640 1 -- 172.21.15.43:0/3010553784 learned_addr learned my addr 172.21.15.43:0/3010553784 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:21.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.560+0000 7fc0557fa640 1 -- 172.21.15.43:0/1514539328 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc0500f2890 con 0x7fc058105dd0 2024-04-21T08:43:21.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.560+0000 7fc05ca8f640 1 --2- 172.21.15.43:0/3010553784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0bf483e90 0x55f0bf486280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.560+0000 7fc0577fe640 1 --2- 172.21.15.43:0/3010553784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0bf476b50 0x55f0bf483950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.560+0000 7fc0577fe640 1 -- 172.21.15.43:0/3010553784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0bf476b50 msgr2=0x55f0bf483950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:21.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.560+0000 7fc0577fe640 1 --2- 172.21.15.43:0/3010553784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0bf476b50 0x55f0bf483950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.560+0000 7fc0577fe640 1 -- 172.21.15.43:0/3010553784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0bf483e90 msgr2=0x55f0bf486280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:21.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.560+0000 7fc0577fe640 1 --2- 172.21.15.43:0/3010553784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0bf483e90 0x55f0bf486280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.560+0000 7fc0577fe640 1 -- 172.21.15.43:0/3010553784 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0bf1b77d0 con 0x55f0bf476230 2024-04-21T08:43:21.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.560+0000 7fc05ca8f640 1 --2- 172.21.15.43:0/3010553784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0bf483e90 0x55f0bf486280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:21.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.561+0000 7fc057fff640 1 --2- 172.21.15.43:0/1514539328 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc0400a3d20 0x7fc0400a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc04800ab60 tx=0x7fc04800a2d0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:21.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.561+0000 7fc0577fe640 1 --2- 172.21.15.43:0/3010553784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476230 0x55f0bf476610 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009dd0 tx=0x7fc04c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50517c75ea3fb518 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:21.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.561+0000 7fc03e7fc640 1 -- 172.21.15.43:0/3010553784 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c012070 con 0x55f0bf476230 2024-04-21T08:43:21.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.561+0000 7fc03e7fc640 1 -- 172.21.15.43:0/3010553784 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc04c0054d0 con 0x55f0bf476230 2024-04-21T08:43:21.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.561+0000 7fc03e7fc640 1 -- 172.21.15.43:0/3010553784 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c013400 con 0x55f0bf476230 2024-04-21T08:43:21.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.561+0000 7fc0611be740 1 -- 172.21.15.43:0/3010553784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476230 msgr2=0x55f0bf476610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:21.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.561+0000 7fc0611be740 1 --2- 172.21.15.43:0/3010553784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476230 0x55f0bf476610 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009dd0 tx=0x7fc04c0096c0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.561+0000 7fc0611be740 1 -- 172.21.15.43:0/3010553784 shutdown_connections 2024-04-21T08:43:21.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.562+0000 7fc0611be740 1 --2- 172.21.15.43:0/3010553784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0bf483e90 0x55f0bf486280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.562+0000 7fc0611be740 1 --2- 172.21.15.43:0/3010553784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0bf476b50 0x55f0bf483950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.562+0000 7fc0611be740 1 --2- 172.21.15.43:0/3010553784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476230 0x55f0bf476610 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.562+0000 7fc0611be740 1 -- 172.21.15.43:0/3010553784 >> 172.21.15.43:0/3010553784 conn(0x55f0bf470cb0 msgr2=0x55f0bf471090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:21.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.562+0000 7fc0611be740 1 -- 172.21.15.43:0/3010553784 shutdown_connections 2024-04-21T08:43:21.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.562+0000 7fc0611be740 1 -- 172.21.15.43:0/3010553784 wait complete. 2024-04-21T08:43:21.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.562+0000 7fc0611be740 1 -- start start 2024-04-21T08:43:21.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.562+0000 7fc0611be740 1 Processor -- start 2024-04-21T08:43:21.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.562+0000 7fc0611be740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476230 0x55f0bf480dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.562+0000 7fc0611be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0bf476b50 0x55f0bf481300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.562+0000 7fc0611be740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0bf483e90 0x55f0bf481840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.562+0000 7fc0611be740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f0bf486cd0 con 0x55f0bf476230 2024-04-21T08:43:21.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.562+0000 7fc0611be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f0bf3942f0 con 0x55f0bf476b50 2024-04-21T08:43:21.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.562+0000 7fc0611be740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f0bf486e50 con 0x55f0bf483e90 2024-04-21T08:43:21.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.562+0000 7fc0611be740 10 client.0 _ll_register_callbacks cb 0x55f0bf3a1580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:21.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.563+0000 7fc057fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476230 0x55f0bf480dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.563+0000 7fc057fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476230 0x55f0bf480dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58070/0 (socket says 172.21.15.43:58070) 2024-04-21T08:43:21.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.563+0000 7fc057fff640 1 -- 172.21.15.43:0/1932636277 learned_addr learned my addr 172.21.15.43:0/1932636277 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:21.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.563+0000 7fc0577fe640 1 --2- 172.21.15.43:0/1932636277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0bf476b50 0x55f0bf481300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.563+0000 7fc05ca8f640 1 --2- 172.21.15.43:0/1932636277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0bf483e90 0x55f0bf481840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.563+0000 7fc057fff640 1 -- 172.21.15.43:0/1932636277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0bf483e90 msgr2=0x55f0bf481840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:21.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.563+0000 7fc057fff640 1 --2- 172.21.15.43:0/1932636277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0bf483e90 0x55f0bf481840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.563+0000 7fc057fff640 1 -- 172.21.15.43:0/1932636277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0bf476b50 msgr2=0x55f0bf481300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.563+0000 7fc057fff640 1 --2- 172.21.15.43:0/1932636277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0bf476b50 0x55f0bf481300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.563+0000 7fc057fff640 1 -- 172.21.15.43:0/1932636277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0bf0ed3f0 con 0x55f0bf476230 2024-04-21T08:43:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.563+0000 7fc057fff640 1 --2- 172.21.15.43:0/1932636277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476230 0x55f0bf480dc0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fc048010a80 tx=0x7fc048004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d9d21e8f7e0d23 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.563+0000 7fc0377fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.563+0000 7fc0377fe640 1 -- 172.21.15.43:0/1932636277 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc048016070 con 0x55f0bf476230 2024-04-21T08:43:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.563+0000 7fc0611be740 1 -- 172.21.15.43:0/1932636277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc04c0054d0 con 0x55f0bf476230 2024-04-21T08:43:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.563+0000 7fc0377fe640 1 -- 172.21.15.43:0/1932636277 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc04801bcc0 con 0x55f0bf476230 2024-04-21T08:43:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.563+0000 7fc0377fe640 1 -- 172.21.15.43:0/1932636277 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc048030050 con 0x55f0bf476230 2024-04-21T08:43:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.563+0000 7fc0611be740 10 client.17607 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.563+0000 7fc0611be740 1 -- 172.21.15.43:0/1932636277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f0bf0ad0f0 con 0x55f0bf476230 2024-04-21T08:43:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.564+0000 7fc0377fe640 1 -- 172.21.15.43:0/1932636277 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc04c0054d0 con 0x55f0bf476230 2024-04-21T08:43:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.564+0000 7fc0377fe640 1 -- 172.21.15.43:0/1932636277 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc048019430 con 0x55f0bf476230 2024-04-21T08:43:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.564+0000 7fc0611be740 10 client.17607 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.564+0000 7fc0611be740 10 client.17607 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.564+0000 7fc0611be740 20 client.17607 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.564+0000 7fc0611be740 1 --2- 172.21.15.43:0/1932636277 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f0bf493f70 0x55f0bf4963e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.564+0000 7fc0611be740 4 client.17607 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.564+0000 7fc0611be740 1 -- 172.21.15.43:0/1932636277 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0bf3e8c00 con 0x55f0bf493f70 2024-04-21T08:43:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.564+0000 7fc0577fe640 1 --2- 172.21.15.43:0/1932636277 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f0bf493f70 0x55f0bf4963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.564+0000 7fc0577fe640 1 --2- 172.21.15.43:0/1932636277 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f0bf493f70 0x55f0bf4963e0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c00d8e0e739f8f88 server_cookie=a0631c4d9424889c in_seq=0 out_seq=0 2024-04-21T08:43:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.564+0000 7fc0377fe640 10 client.17607 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.565+0000 7fc0377fe640 1 -- 172.21.15.43:0/1932636277 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f0bf3e8c00 con 0x55f0bf493f70 2024-04-21T08:43:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.565+0000 7fc0377fe640 10 client.17607 handle_command_reply: tid=0 2024-04-21T08:43:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.565+0000 7fc0611be740 1 client.17607 shutdown 2024-04-21T08:43:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.565+0000 7fc0611be740 1 -- 172.21.15.43:0/1932636277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476230 msgr2=0x55f0bf480dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.565+0000 7fc0611be740 1 --2- 172.21.15.43:0/1932636277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476230 0x55f0bf480dc0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fc048010a80 tx=0x7fc048004730 comp rx=0 tx=0).stop 2024-04-21T08:43:21.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.565+0000 7fc0611be740 1 -- 172.21.15.43:0/1932636277 shutdown_connections 2024-04-21T08:43:21.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.565+0000 7fc0611be740 1 --2- 172.21.15.43:0/1932636277 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f0bf493f70 0x55f0bf4963e0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.565+0000 7fc0611be740 1 --2- 172.21.15.43:0/1932636277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0bf483e90 0x55f0bf481840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.565+0000 7fc0611be740 1 --2- 172.21.15.43:0/1932636277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0bf476b50 0x55f0bf481300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.565+0000 7fc0377fe640 0 client.17607 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:21.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.565+0000 7fc0611be740 1 --2- 172.21.15.43:0/1932636277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476230 0x55f0bf480dc0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.565+0000 7fc0611be740 1 -- 172.21.15.43:0/1932636277 >> 172.21.15.43:0/1932636277 conn(0x55f0bf470cb0 msgr2=0x55f0bf472a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:21.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.566+0000 7fc0611be740 1 -- 172.21.15.43:0/1932636277 shutdown_connections 2024-04-21T08:43:21.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.566+0000 7fc0611be740 1 -- 172.21.15.43:0/1932636277 wait complete. 2024-04-21T08:43:21.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.566+0000 7fc0611be740 20 client.17607 trim_cache size 0 max 16384 2024-04-21T08:43:21.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.583+0000 7fc0611be740 1 Processor -- start 2024-04-21T08:43:21.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.583+0000 7fc0611be740 1 -- start start 2024-04-21T08:43:21.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.583+0000 7fc0611be740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0bf476230 0x55f0bf4834f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.583+0000 7fc0611be740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476b50 0x55f0bf48fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.583+0000 7fc0611be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0bf483e90 0x55f0bf46fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.583+0000 7fc0611be740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc0581ad0b0 con 0x55f0bf476b50 2024-04-21T08:43:21.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.583+0000 7fc0611be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0581aebf0 con 0x55f0bf483e90 2024-04-21T08:43:21.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.583+0000 7fc0611be740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc0581ad780 con 0x55f0bf476230 2024-04-21T08:43:21.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.583+0000 7fc0577fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476b50 0x55f0bf48fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.583+0000 7fc0577fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476b50 0x55f0bf48fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58074/0 (socket says 172.21.15.43:58074) 2024-04-21T08:43:21.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.583+0000 7fc0577fe640 1 -- 172.21.15.43:0/2555333699 learned_addr learned my addr 172.21.15.43:0/2555333699 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:21.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.584+0000 7fc0577fe640 1 -- 172.21.15.43:0/2555333699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0bf476230 msgr2=0x55f0bf4834f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:21.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.584+0000 7fc0577fe640 1 --2- 172.21.15.43:0/2555333699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0bf476230 0x55f0bf4834f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.584+0000 7fc0577fe640 1 -- 172.21.15.43:0/2555333699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0bf483e90 msgr2=0x55f0bf46fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.584+0000 7fc0577fe640 1 --2- 172.21.15.43:0/2555333699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0bf483e90 0x55f0bf46fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.584+0000 7fc0577fe640 1 -- 172.21.15.43:0/2555333699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0bf470ae0 con 0x55f0bf476b50 2024-04-21T08:43:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.584+0000 7fc0577fe640 1 --2- 172.21.15.43:0/2555333699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476b50 0x55f0bf48fc30 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009da0 tx=0x7fc04c012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8126ccd030313d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.584+0000 7fc03dffb640 1 -- 172.21.15.43:0/2555333699 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c004660 con 0x55f0bf476b50 2024-04-21T08:43:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.584+0000 7fc03dffb640 1 -- 172.21.15.43:0/2555333699 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc04c004800 con 0x55f0bf476b50 2024-04-21T08:43:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.584+0000 7fc03dffb640 1 -- 172.21.15.43:0/2555333699 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c0035f0 con 0x55f0bf476b50 2024-04-21T08:43:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.584+0000 7fc0611be740 1 -- 172.21.15.43:0/2555333699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476b50 msgr2=0x55f0bf48fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.584+0000 7fc0611be740 1 --2- 172.21.15.43:0/2555333699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476b50 0x55f0bf48fc30 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009da0 tx=0x7fc04c012040 comp rx=0 tx=0).stop 2024-04-21T08:43:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.584+0000 7fc0611be740 1 -- 172.21.15.43:0/2555333699 shutdown_connections 2024-04-21T08:43:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.584+0000 7fc0611be740 1 --2- 172.21.15.43:0/2555333699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0bf483e90 0x55f0bf46fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.584+0000 7fc0611be740 1 --2- 172.21.15.43:0/2555333699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476b50 0x55f0bf48fc30 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.584+0000 7fc0611be740 1 --2- 172.21.15.43:0/2555333699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0bf476230 0x55f0bf4834f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.584+0000 7fc0611be740 1 -- 172.21.15.43:0/2555333699 >> 172.21.15.43:0/2555333699 conn(0x55f0bf470cb0 msgr2=0x55f0bf46f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.585+0000 7fc0611be740 1 -- 172.21.15.43:0/2555333699 shutdown_connections 2024-04-21T08:43:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.585+0000 7fc0611be740 1 -- 172.21.15.43:0/2555333699 wait complete. 2024-04-21T08:43:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.585+0000 7fc0611be740 1 -- start start 2024-04-21T08:43:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.585+0000 7fc0611be740 1 Processor -- start 2024-04-21T08:43:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.585+0000 7fc0611be740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476230 0x55f0bf487bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.585+0000 7fc0611be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0bf476b50 0x55f0bf488100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.585+0000 7fc0611be740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0bf483e90 0x55f0bf488640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.585+0000 7fc0611be740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc0581ae520 con 0x55f0bf476230 2024-04-21T08:43:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.585+0000 7fc0611be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0581ade50 con 0x55f0bf476b50 2024-04-21T08:43:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.585+0000 7fc0611be740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc0581ab090 con 0x55f0bf483e90 2024-04-21T08:43:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.585+0000 7fc0611be740 10 client.0 _ll_register_callbacks cb 0x7fc04800a850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.585+0000 7fc057fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476230 0x55f0bf487bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.585+0000 7fc057fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476230 0x55f0bf487bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58076/0 (socket says 172.21.15.43:58076) 2024-04-21T08:43:21.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.585+0000 7fc057fff640 1 -- 172.21.15.43:0/3467415626 learned_addr learned my addr 172.21.15.43:0/3467415626 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:21.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.585+0000 7fc0577fe640 1 --2- 172.21.15.43:0/3467415626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0bf476b50 0x55f0bf488100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.586+0000 7fc05ca8f640 1 --2- 172.21.15.43:0/3467415626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0bf483e90 0x55f0bf488640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.586+0000 7fc057fff640 1 -- 172.21.15.43:0/3467415626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0bf483e90 msgr2=0x55f0bf488640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:21.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.586+0000 7fc057fff640 1 --2- 172.21.15.43:0/3467415626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0bf483e90 0x55f0bf488640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.586+0000 7fc057fff640 1 -- 172.21.15.43:0/3467415626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0bf476b50 msgr2=0x55f0bf488100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:21.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.586+0000 7fc057fff640 1 --2- 172.21.15.43:0/3467415626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0bf476b50 0x55f0bf488100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.586+0000 7fc057fff640 1 -- 172.21.15.43:0/3467415626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0bf4f9570 con 0x55f0bf476230 2024-04-21T08:43:21.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.586+0000 7fc05ca8f640 1 --2- 172.21.15.43:0/3467415626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0bf483e90 0x55f0bf488640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:21.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.586+0000 7fc0577fe640 1 --2- 172.21.15.43:0/3467415626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0bf476b50 0x55f0bf488100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:21.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.586+0000 7fc057fff640 1 --2- 172.21.15.43:0/3467415626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476230 0x55f0bf487bc0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fc04802e040 tx=0x7fc048019d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0f94dad3b07d30 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:21.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.586+0000 7fc0377fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:21.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.586+0000 7fc0377fe640 1 -- 172.21.15.43:0/3467415626 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04800f8d0 con 0x55f0bf476230 2024-04-21T08:43:21.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.586+0000 7fc0377fe640 1 -- 172.21.15.43:0/3467415626 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc04801bac0 con 0x55f0bf476230 2024-04-21T08:43:21.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.586+0000 7fc0611be740 1 -- 172.21.15.43:0/3467415626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc04c004800 con 0x55f0bf476230 2024-04-21T08:43:21.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.586+0000 7fc0377fe640 1 -- 172.21.15.43:0/3467415626 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc048005ce0 con 0x55f0bf476230 2024-04-21T08:43:21.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.586+0000 7fc0611be740 10 client.17613 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:21.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.586+0000 7fc0611be740 1 -- 172.21.15.43:0/3467415626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f0bf4fa8e0 con 0x55f0bf476230 2024-04-21T08:43:21.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.586+0000 7fc0377fe640 1 -- 172.21.15.43:0/3467415626 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc04c004800 con 0x55f0bf476230 2024-04-21T08:43:21.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.586+0000 7fc0377fe640 1 -- 172.21.15.43:0/3467415626 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc048004780 con 0x55f0bf476230 2024-04-21T08:43:21.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.587+0000 7fc0611be740 10 client.17613 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:21.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.587+0000 7fc0611be740 10 client.17613 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:21.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.587+0000 7fc0611be740 20 client.17613 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:21.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.587+0000 7fc0611be740 1 --2- 172.21.15.43:0/3467415626 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f0bf493f70 0x55f0bf4fd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:21.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.587+0000 7fc0611be740 4 client.17613 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:43:21.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.587+0000 7fc0611be740 1 -- 172.21.15.43:0/3467415626 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f0bf3e2a70 con 0x55f0bf493f70 2024-04-21T08:43:21.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.587+0000 7fc0577fe640 1 --2- 172.21.15.43:0/3467415626 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f0bf493f70 0x55f0bf4fd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:21.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.587+0000 7fc0577fe640 1 --2- 172.21.15.43:0/3467415626 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f0bf493f70 0x55f0bf4fd0f0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=120a8138972d2f2 server_cookie=127cbe9aad987cc6 in_seq=0 out_seq=0 2024-04-21T08:43:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.587+0000 7fc0377fe640 10 client.17613 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.587+0000 7fc0377fe640 1 -- 172.21.15.43:0/3467415626 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f0bf3e2a70 con 0x55f0bf493f70 2024-04-21T08:43:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.587+0000 7fc0377fe640 10 client.17613 handle_command_reply: tid=0 2024-04-21T08:43:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.587+0000 7fc0611be740 1 client.17613 shutdown 2024-04-21T08:43:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.588+0000 7fc0611be740 1 -- 172.21.15.43:0/3467415626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476230 msgr2=0x55f0bf487bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.588+0000 7fc0611be740 1 --2- 172.21.15.43:0/3467415626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476230 0x55f0bf487bc0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fc04802e040 tx=0x7fc048019d20 comp rx=0 tx=0).stop 2024-04-21T08:43:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.588+0000 7fc0611be740 1 -- 172.21.15.43:0/3467415626 shutdown_connections 2024-04-21T08:43:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.588+0000 7fc0611be740 1 --2- 172.21.15.43:0/3467415626 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f0bf493f70 0x55f0bf4fd0f0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.588+0000 7fc0611be740 1 --2- 172.21.15.43:0/3467415626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f0bf483e90 0x55f0bf488640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.588+0000 7fc0611be740 1 --2- 172.21.15.43:0/3467415626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f0bf476b50 0x55f0bf488100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.588+0000 7fc0611be740 1 --2- 172.21.15.43:0/3467415626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0bf476230 0x55f0bf487bc0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.588+0000 7fc0377fe640 0 client.17613 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.588+0000 7fc0611be740 1 -- 172.21.15.43:0/3467415626 >> 172.21.15.43:0/3467415626 conn(0x55f0bf470cb0 msgr2=0x55f0bf46fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.588+0000 7fc0611be740 1 -- 172.21.15.43:0/3467415626 shutdown_connections 2024-04-21T08:43:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.588+0000 7fc0611be740 1 -- 172.21.15.43:0/3467415626 wait complete. 2024-04-21T08:43:21.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.588+0000 7fc0611be740 20 client.17613 trim_cache size 0 max 16384 2024-04-21T08:43:21.594 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:43:21.594 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:43:21.594 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-21T08:43:21.594 INFO:teuthology.orchestra.run.smithi043.stdout: "e8f38c4f-39de-4534-946c-66500b376291": { 2024-04-21T08:43:21.594 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-21T08:43:21.594 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "e8f38c4f-39de-4534-946c-66500b376291", 2024-04-21T08:43:21.594 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-21T08:43:21.594 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:43:21.595 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:43:21.595 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:43:21.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.588+0000 7fc05e4f2640 1 -- 172.21.15.43:0/1514539328 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc0400a3d20 msgr2=0x7fc0400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:21.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.588+0000 7fc05e4f2640 1 --2- 172.21.15.43:0/1514539328 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc0400a3d20 0x7fc0400a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc04800ab60 tx=0x7fc04800a2d0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.588+0000 7fc05e4f2640 1 -- 172.21.15.43:0/1514539328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc058105dd0 msgr2=0x7fc0581a4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:21.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.588+0000 7fc05e4f2640 1 --2- 172.21.15.43:0/1514539328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc058105dd0 0x7fc0581a4540 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fc050005e30 tx=0x7fc050032420 comp rx=0 tx=0).stop 2024-04-21T08:43:21.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.588+0000 7fc05e4f2640 1 -- 172.21.15.43:0/1514539328 shutdown_connections 2024-04-21T08:43:21.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.588+0000 7fc05e4f2640 1 --2- 172.21.15.43:0/1514539328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc058105dd0 0x7fc0581a4540 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.588+0000 7fc05e4f2640 1 --2- 172.21.15.43:0/1514539328 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc0400a3d20 0x7fc0400a61e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.588+0000 7fc05e4f2640 1 --2- 172.21.15.43:0/1514539328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc058101a40 0x7fc058199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.588+0000 7fc05e4f2640 1 --2- 172.21.15.43:0/1514539328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0581010a0 0x7fc0581994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:21.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.588+0000 7fc05e4f2640 1 -- 172.21.15.43:0/1514539328 >> 172.21.15.43:0/1514539328 conn(0x7fc058077c40 msgr2=0x7fc0580feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:21.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.588+0000 7fc05e4f2640 1 -- 172.21.15.43:0/1514539328 shutdown_connections 2024-04-21T08:43:21.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:21.588+0000 7fc05e4f2640 1 -- 172.21.15.43:0/1514539328 wait complete. 2024-04-21T08:43:21.598 INFO:tasks.cephfs.filesystem:scrub status for tag:e8f38c4f-39de-4534-946c-66500b376291 - {'path': '/', 'tag': 'e8f38c4f-39de-4534-946c-66500b376291', 'options': 'recursive,force'} 2024-04-21T08:43:22.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:22 smithi112 ceph-mon[31085]: pgmap v136: 129 pgs: 129 active+clean; 5.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 23 op/s 2024-04-21T08:43:22.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:22 smithi043 ceph-mon[27978]: pgmap v136: 129 pgs: 129 active+clean; 5.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 23 op/s 2024-04-21T08:43:22.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:22 smithi175 ceph-mon[31920]: pgmap v136: 129 pgs: 129 active+clean; 5.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 23 op/s 2024-04-21T08:43:23.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:23 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:43:23.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:23 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:43:23.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:23 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:43:23.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:23 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:43:23.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:23 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:43:23.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:23 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:43:23.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:23 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:43:23.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:23 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:43:23.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:23 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:43:23.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:23 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:43:23.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:23 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:43:23.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:23 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:43:24.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:24 smithi112 ceph-mon[31085]: pgmap v137: 129 pgs: 129 active+clean; 5.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 16 op/s 2024-04-21T08:43:24.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:24 smithi043 ceph-mon[27978]: pgmap v137: 129 pgs: 129 active+clean; 5.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 16 op/s 2024-04-21T08:43:24.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:24 smithi175 ceph-mon[31920]: pgmap v137: 129 pgs: 129 active+clean; 5.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 16 op/s 2024-04-21T08:43:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.984+0000 7f7640814640 1 -- 172.21.15.43:0/3392273196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f763c102b70 msgr2=0x7f763c106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.984+0000 7f7640814640 1 --2- 172.21.15.43:0/3392273196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f763c102b70 0x7f763c106bb0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f7624009900 tx=0x7f762402bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.984+0000 7f7640814640 1 -- 172.21.15.43:0/3392273196 shutdown_connections 2024-04-21T08:43:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.984+0000 7f7640814640 1 --2- 172.21.15.43:0/3392273196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f763c107290 0x7f763c10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.984+0000 7f7640814640 1 --2- 172.21.15.43:0/3392273196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f763c102b70 0x7f763c106bb0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.984+0000 7f7640814640 1 --2- 172.21.15.43:0/3392273196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f763c1021c0 0x7f763c1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.984+0000 7f7640814640 1 -- 172.21.15.43:0/3392273196 >> 172.21.15.43:0/3392273196 conn(0x7f763c0fda20 msgr2=0x7f763c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.984+0000 7f7640814640 1 -- 172.21.15.43:0/3392273196 shutdown_connections 2024-04-21T08:43:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.984+0000 7f7640814640 1 -- 172.21.15.43:0/3392273196 wait complete. 2024-04-21T08:43:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.985+0000 7f7640814640 1 Processor -- start 2024-04-21T08:43:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.985+0000 7f7640814640 1 -- start start 2024-04-21T08:43:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.985+0000 7f7640814640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f763c1021c0 0x7f763c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.986+0000 7f7640814640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f763c107290 0x7f763c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.986+0000 7f7640814640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f763c074cd0 0x7f763c1a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.986+0000 7f7640814640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f763c112220 con 0x7f763c1021c0 2024-04-21T08:43:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.986+0000 7f7640814640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f763c1120a0 con 0x7f763c074cd0 2024-04-21T08:43:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.986+0000 7f7640814640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f763c1123a0 con 0x7f763c107290 2024-04-21T08:43:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.986+0000 7f763a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f763c1021c0 0x7f763c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.986+0000 7f763a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f763c1021c0 0x7f763c074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58084/0 (socket says 172.21.15.43:58084) 2024-04-21T08:43:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.986+0000 7f763a59c640 1 -- 172.21.15.43:0/2608621012 learned_addr learned my addr 172.21.15.43:0/2608621012 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.986+0000 7f7639d9b640 1 --2- 172.21.15.43:0/2608621012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f763c107290 0x7f763c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.986+0000 7f763ad9d640 1 --2- 172.21.15.43:0/2608621012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f763c074cd0 0x7f763c1a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.986+0000 7f763a59c640 1 -- 172.21.15.43:0/2608621012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f763c107290 msgr2=0x7f763c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.986+0000 7f763a59c640 1 --2- 172.21.15.43:0/2608621012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f763c107290 0x7f763c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.986+0000 7f763a59c640 1 -- 172.21.15.43:0/2608621012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f763c074cd0 msgr2=0x7f763c1a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.986+0000 7f763a59c640 1 --2- 172.21.15.43:0/2608621012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f763c074cd0 0x7f763c1a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.986+0000 7f763a59c640 1 -- 172.21.15.43:0/2608621012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763c1a6fb0 con 0x7f763c1021c0 2024-04-21T08:43:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.986+0000 7f763ad9d640 1 --2- 172.21.15.43:0/2608621012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f763c074cd0 0x7f763c1a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.986+0000 7f763a59c640 1 --2- 172.21.15.43:0/2608621012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f763c1021c0 0x7f763c074250 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f763400e9f0 tx=0x7f763400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.986+0000 7f76337fe640 1 -- 172.21.15.43:0/2608621012 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763400cd80 con 0x7f763c1021c0 2024-04-21T08:43:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.986+0000 7f76337fe640 1 -- 172.21.15.43:0/2608621012 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76340108e0 con 0x7f763c1021c0 2024-04-21T08:43:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.986+0000 7f7640814640 1 -- 172.21.15.43:0/2608621012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f763c1a72a0 con 0x7f763c1021c0 2024-04-21T08:43:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.986+0000 7f76337fe640 1 -- 172.21.15.43:0/2608621012 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634018980 con 0x7f763c1021c0 2024-04-21T08:43:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.987+0000 7f7640814640 1 -- 172.21.15.43:0/2608621012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f763c1a77c0 con 0x7f763c1021c0 2024-04-21T08:43:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.989+0000 7f76337fe640 1 -- 172.21.15.43:0/2608621012 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7634010430 con 0x7f763c1021c0 2024-04-21T08:43:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.989+0000 7f7643409740 1 Processor -- start 2024-04-21T08:43:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.989+0000 7f7643409740 1 -- start start 2024-04-21T08:43:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.990+0000 7f7643409740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56068d5c88d0 0x56068d5c7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.990+0000 7f7643409740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068d5c7b60 0x56068d5c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.990+0000 7f7643409740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5d10e0 0x56068d5d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.990+0000 7f7643409740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56068d52c8d0 con 0x56068d5d10e0 2024-04-21T08:43:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.990+0000 7f7643409740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56068d4c9e80 con 0x56068d5c88d0 2024-04-21T08:43:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.990+0000 7f7643409740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56068d528890 con 0x56068d5c7b60 2024-04-21T08:43:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.990+0000 7f76337fe640 1 --2- 172.21.15.43:0/2608621012 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f76280a3df0 0x7f76280a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.990+0000 7f7639d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5d10e0 0x56068d5d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.990+0000 7f7639d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5d10e0 0x56068d5d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58098/0 (socket says 172.21.15.43:58098) 2024-04-21T08:43:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.990+0000 7f7639d9b640 1 -- 172.21.15.43:0/3114598450 learned_addr learned my addr 172.21.15.43:0/3114598450 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.990+0000 7f763ad9d640 1 --2- 172.21.15.43:0/2608621012 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f76280a3df0 0x7f76280a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.990+0000 7f76337fe640 1 -- 172.21.15.43:0/2608621012 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f76340d0fc0 con 0x7f763c1021c0 2024-04-21T08:43:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.990+0000 7f763a59c640 1 --2- 172.21.15.43:0/3114598450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068d5c7b60 0x56068d5c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.990+0000 7f763ad9d640 1 --2- 172.21.15.43:0/3114598450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56068d5c88d0 0x56068d5c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.990+0000 7f7639d9b640 1 -- 172.21.15.43:0/3114598450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068d5c7b60 msgr2=0x56068d5c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.990+0000 7f7639d9b640 1 --2- 172.21.15.43:0/3114598450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068d5c7b60 0x56068d5c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.990+0000 7f7639d9b640 1 -- 172.21.15.43:0/3114598450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56068d5c88d0 msgr2=0x56068d5c7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.990+0000 7f7639d9b640 1 --2- 172.21.15.43:0/3114598450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56068d5c88d0 0x56068d5c7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.990+0000 7f7639d9b640 1 -- 172.21.15.43:0/3114598450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56068d4eb320 con 0x56068d5d10e0 2024-04-21T08:43:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.991+0000 7f763ad9d640 1 --2- 172.21.15.43:0/2608621012 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f76280a3df0 0x7f76280a62b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f762c009fd0 tx=0x7f762c0091e0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.991+0000 7f7639d9b640 1 --2- 172.21.15.43:0/3114598450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5d10e0 0x56068d5d34d0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f7624005ab0 tx=0x7f7624009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8c609e40061dd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.991+0000 7f7630ff9640 1 -- 172.21.15.43:0/3114598450 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624045020 con 0x56068d5d10e0 2024-04-21T08:43:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.991+0000 7f7630ff9640 1 -- 172.21.15.43:0/3114598450 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f762402c7e0 con 0x56068d5d10e0 2024-04-21T08:43:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.991+0000 7f7630ff9640 1 -- 172.21.15.43:0/3114598450 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624005530 con 0x56068d5d10e0 2024-04-21T08:43:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.991+0000 7f7643409740 1 -- 172.21.15.43:0/3114598450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5d10e0 msgr2=0x56068d5d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.991+0000 7f7643409740 1 --2- 172.21.15.43:0/3114598450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5d10e0 0x56068d5d34d0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f7624005ab0 tx=0x7f7624009670 comp rx=0 tx=0).stop 2024-04-21T08:43:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.992+0000 7f7643409740 1 -- 172.21.15.43:0/3114598450 shutdown_connections 2024-04-21T08:43:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.992+0000 7f7643409740 1 --2- 172.21.15.43:0/3114598450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5d10e0 0x56068d5d34d0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.992+0000 7f7643409740 1 --2- 172.21.15.43:0/3114598450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068d5c7b60 0x56068d5c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.992+0000 7f7643409740 1 --2- 172.21.15.43:0/3114598450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56068d5c88d0 0x56068d5c7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.992+0000 7f7643409740 1 -- 172.21.15.43:0/3114598450 >> 172.21.15.43:0/3114598450 conn(0x56068d58ec90 msgr2=0x56068d5c2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.992+0000 7f7643409740 1 -- 172.21.15.43:0/3114598450 shutdown_connections 2024-04-21T08:43:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.992+0000 7f7643409740 1 -- 172.21.15.43:0/3114598450 wait complete. 2024-04-21T08:43:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.992+0000 7f7643409740 1 -- start start 2024-04-21T08:43:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.992+0000 7f7643409740 1 Processor -- start 2024-04-21T08:43:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.992+0000 7f7643409740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5c7b60 0x56068d5d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.992+0000 7f7643409740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068d5c88d0 0x56068d5cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.992+0000 7f7643409740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56068d5d10e0 0x56068d5ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.992+0000 7f7643409740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56068d5d3db0 con 0x56068d5c7b60 2024-04-21T08:43:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.992+0000 7f7643409740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56068d4b4eb0 con 0x56068d5d10e0 2024-04-21T08:43:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.992+0000 7f7643409740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56068d5d3f30 con 0x56068d5c88d0 2024-04-21T08:43:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.992+0000 7f7643409740 10 client.0 _ll_register_callbacks cb 0x56068d4e1b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.992+0000 7f763a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5c7b60 0x56068d5d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.993+0000 7f763a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5c7b60 0x56068d5d0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58110/0 (socket says 172.21.15.43:58110) 2024-04-21T08:43:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.993+0000 7f763a59c640 1 -- 172.21.15.43:0/550663245 learned_addr learned my addr 172.21.15.43:0/550663245 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.993+0000 7f7639d9b640 1 --2- 172.21.15.43:0/550663245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068d5c88d0 0x56068d5cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.993+0000 7f763ad9d640 1 --2- 172.21.15.43:0/550663245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56068d5d10e0 0x56068d5ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.993+0000 7f763a59c640 1 -- 172.21.15.43:0/550663245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068d5c88d0 msgr2=0x56068d5cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.993+0000 7f763a59c640 1 --2- 172.21.15.43:0/550663245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068d5c88d0 0x56068d5cddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.993+0000 7f763a59c640 1 -- 172.21.15.43:0/550663245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56068d5d10e0 msgr2=0x56068d5ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.993+0000 7f763a59c640 1 --2- 172.21.15.43:0/550663245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56068d5d10e0 0x56068d5ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.993+0000 7f763a59c640 1 -- 172.21.15.43:0/550663245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56068d23f210 con 0x56068d5c7b60 2024-04-21T08:43:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.993+0000 7f763a59c640 1 --2- 172.21.15.43:0/550663245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5c7b60 0x56068d5d0a30 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f763400e9c0 tx=0x7f763401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64be2cf07024504 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.993+0000 7f7621ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.993+0000 7f7621ffb640 1 -- 172.21.15.43:0/550663245 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763409de60 con 0x56068d5c7b60 2024-04-21T08:43:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.993+0000 7f7621ffb640 1 -- 172.21.15.43:0/550663245 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f76340991a0 con 0x56068d5c7b60 2024-04-21T08:43:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.993+0000 7f7643409740 1 -- 172.21.15.43:0/550663245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f762402c7e0 con 0x56068d5c7b60 2024-04-21T08:43:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.993+0000 7f7621ffb640 1 -- 172.21.15.43:0/550663245 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f76340a0ee0 con 0x56068d5c7b60 2024-04-21T08:43:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.993+0000 7f7643409740 10 client.17625 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.993+0000 7f7643409740 1 -- 172.21.15.43:0/550663245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56068d30b620 con 0x56068d5c7b60 2024-04-21T08:43:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.993+0000 7f7621ffb640 1 -- 172.21.15.43:0/550663245 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f762402c7e0 con 0x56068d5c7b60 2024-04-21T08:43:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.994+0000 7f7621ffb640 1 -- 172.21.15.43:0/550663245 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7634018b90 con 0x56068d5c7b60 2024-04-21T08:43:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.994+0000 7f7643409740 10 client.17625 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.994+0000 7f7643409740 10 client.17625 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.994+0000 7f7643409740 20 client.17625 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.994+0000 7f7643409740 1 --2- 172.21.15.43:0/550663245 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56068d5e1050 0x56068d5e3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.994+0000 7f7643409740 4 client.17625 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.994+0000 7f7643409740 1 -- 172.21.15.43:0/550663245 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56068d540880 con 0x56068d5e1050 2024-04-21T08:43:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.994+0000 7f7639d9b640 1 --2- 172.21.15.43:0/550663245 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56068d5e1050 0x56068d5e3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.994+0000 7f7639d9b640 1 --2- 172.21.15.43:0/550663245 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56068d5e1050 0x56068d5e3440 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4ae993e92e60e17 server_cookie=ea92cb9f6ca53d5b in_seq=0 out_seq=0 2024-04-21T08:43:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.994+0000 7f7621ffb640 10 client.17625 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.995+0000 7f7621ffb640 1 -- 172.21.15.43:0/550663245 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56068d540880 con 0x56068d5e1050 2024-04-21T08:43:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.995+0000 7f7621ffb640 10 client.17625 handle_command_reply: tid=0 2024-04-21T08:43:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.995+0000 7f7643409740 1 client.17625 shutdown 2024-04-21T08:43:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.995+0000 7f7643409740 1 -- 172.21.15.43:0/550663245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5c7b60 msgr2=0x56068d5d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.995+0000 7f7643409740 1 --2- 172.21.15.43:0/550663245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5c7b60 0x56068d5d0a30 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f763400e9c0 tx=0x7f763401f000 comp rx=0 tx=0).stop 2024-04-21T08:43:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.995+0000 7f7643409740 1 -- 172.21.15.43:0/550663245 shutdown_connections 2024-04-21T08:43:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.995+0000 7f7643409740 1 --2- 172.21.15.43:0/550663245 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56068d5e1050 0x56068d5e3440 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.995+0000 7f7643409740 1 --2- 172.21.15.43:0/550663245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56068d5d10e0 0x56068d5ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.995+0000 7f7643409740 1 --2- 172.21.15.43:0/550663245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068d5c88d0 0x56068d5cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.995+0000 7f7643409740 1 --2- 172.21.15.43:0/550663245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5c7b60 0x56068d5d0a30 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.995+0000 7f7621ffb640 0 client.17625 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.995+0000 7f7643409740 1 -- 172.21.15.43:0/550663245 >> 172.21.15.43:0/550663245 conn(0x56068d58ec90 msgr2=0x56068d5c45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.995+0000 7f7643409740 1 -- 172.21.15.43:0/550663245 shutdown_connections 2024-04-21T08:43:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.995+0000 7f7643409740 1 -- 172.21.15.43:0/550663245 wait complete. 2024-04-21T08:43:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:24.996+0000 7f7643409740 20 client.17625 trim_cache size 0 max 16384 2024-04-21T08:43:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.011+0000 7f7643409740 1 Processor -- start 2024-04-21T08:43:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.011+0000 7f7643409740 1 -- start start 2024-04-21T08:43:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.011+0000 7f7643409740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068d5c7b60 0x56068d5dccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.011+0000 7f7643409740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5c88d0 0x56068d5d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.011+0000 7f7643409740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56068d5d10e0 0x56068d5d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.011+0000 7f7643409740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f763c1b1590 con 0x56068d5c88d0 2024-04-21T08:43:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.011+0000 7f7643409740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f763c1b30d0 con 0x56068d5d10e0 2024-04-21T08:43:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.011+0000 7f7643409740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f763c1b1c60 con 0x56068d5c7b60 2024-04-21T08:43:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.011+0000 7f7639d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5c88d0 0x56068d5d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.011+0000 7f7639d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5c88d0 0x56068d5d04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58112/0 (socket says 172.21.15.43:58112) 2024-04-21T08:43:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.011+0000 7f7639d9b640 1 -- 172.21.15.43:0/1892278657 learned_addr learned my addr 172.21.15.43:0/1892278657 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.011+0000 7f763ad9d640 1 --2- 172.21.15.43:0/1892278657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56068d5d10e0 0x56068d5d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.011+0000 7f7639d9b640 1 -- 172.21.15.43:0/1892278657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068d5c7b60 msgr2=0x56068d5dccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.012+0000 7f763a59c640 1 --2- 172.21.15.43:0/1892278657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068d5c7b60 0x56068d5dccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.012+0000 7f7639d9b640 1 --2- 172.21.15.43:0/1892278657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068d5c7b60 0x56068d5dccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.012+0000 7f7639d9b640 1 -- 172.21.15.43:0/1892278657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56068d5d10e0 msgr2=0x56068d5d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.012+0000 7f7639d9b640 1 --2- 172.21.15.43:0/1892278657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56068d5d10e0 0x56068d5d0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.012+0000 7f7639d9b640 1 -- 172.21.15.43:0/1892278657 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56068d5c6e30 con 0x56068d5c88d0 2024-04-21T08:43:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.012+0000 7f7639d9b640 1 --2- 172.21.15.43:0/1892278657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5c88d0 0x56068d5d04f0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x56068d5c8600 tx=0x7f7624005150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc093b76d41c8797 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.012+0000 7f7623fff640 1 -- 172.21.15.43:0/1892278657 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624045040 con 0x56068d5c88d0 2024-04-21T08:43:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.012+0000 7f7623fff640 1 -- 172.21.15.43:0/1892278657 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f762402c7e0 con 0x56068d5c88d0 2024-04-21T08:43:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.012+0000 7f7623fff640 1 -- 172.21.15.43:0/1892278657 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762404f180 con 0x56068d5c88d0 2024-04-21T08:43:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.013+0000 7f7643409740 1 -- 172.21.15.43:0/1892278657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5c88d0 msgr2=0x56068d5d04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.013+0000 7f7643409740 1 --2- 172.21.15.43:0/1892278657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5c88d0 0x56068d5d04f0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x56068d5c8600 tx=0x7f7624005150 comp rx=0 tx=0).stop 2024-04-21T08:43:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.013+0000 7f7643409740 1 -- 172.21.15.43:0/1892278657 shutdown_connections 2024-04-21T08:43:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.013+0000 7f7643409740 1 --2- 172.21.15.43:0/1892278657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56068d5d10e0 0x56068d5d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.013+0000 7f7643409740 1 --2- 172.21.15.43:0/1892278657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5c88d0 0x56068d5d04f0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.013+0000 7f7643409740 1 --2- 172.21.15.43:0/1892278657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068d5c7b60 0x56068d5dccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.013+0000 7f7643409740 1 -- 172.21.15.43:0/1892278657 >> 172.21.15.43:0/1892278657 conn(0x56068d58ec90 msgr2=0x56068d5c2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.013+0000 7f7643409740 1 -- 172.21.15.43:0/1892278657 shutdown_connections 2024-04-21T08:43:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.013+0000 7f7643409740 1 -- 172.21.15.43:0/1892278657 wait complete. 2024-04-21T08:43:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.013+0000 7f7643409740 1 -- start start 2024-04-21T08:43:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.013+0000 7f7643409740 1 Processor -- start 2024-04-21T08:43:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.013+0000 7f7643409740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5c7b60 0x56068d5d91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.013+0000 7f7643409740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56068d5c88d0 0x56068d5d9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.013+0000 7f7643409740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068d5d10e0 0x56068d5d2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.013+0000 7f7643409740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f763c1b2a00 con 0x56068d5c7b60 2024-04-21T08:43:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.013+0000 7f7643409740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f763c1b2330 con 0x56068d5c88d0 2024-04-21T08:43:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.013+0000 7f7643409740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f763c1af570 con 0x56068d5d10e0 2024-04-21T08:43:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.013+0000 7f7643409740 10 client.0 _ll_register_callbacks cb 0x7f763409c670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.013+0000 7f763a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5c7b60 0x56068d5d91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.013+0000 7f763a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5c7b60 0x56068d5d91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58120/0 (socket says 172.21.15.43:58120) 2024-04-21T08:43:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.013+0000 7f763a59c640 1 -- 172.21.15.43:0/3247835303 learned_addr learned my addr 172.21.15.43:0/3247835303 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.014+0000 7f7639d9b640 1 --2- 172.21.15.43:0/3247835303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56068d5c88d0 0x56068d5d9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.014+0000 7f763a59c640 1 -- 172.21.15.43:0/3247835303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068d5d10e0 msgr2=0x56068d5d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.014+0000 7f763a59c640 1 --2- 172.21.15.43:0/3247835303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068d5d10e0 0x56068d5d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.014+0000 7f763a59c640 1 -- 172.21.15.43:0/3247835303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56068d5c88d0 msgr2=0x56068d5d9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.014+0000 7f763a59c640 1 --2- 172.21.15.43:0/3247835303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56068d5c88d0 0x56068d5d9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.014+0000 7f763a59c640 1 -- 172.21.15.43:0/3247835303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56068d5d2670 con 0x56068d5c7b60 2024-04-21T08:43:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.014+0000 7f763a59c640 1 --2- 172.21.15.43:0/3247835303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5c7b60 0x56068d5d91c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f7634098f90 tx=0x7f7634000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74d3f9443e0cbc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.014+0000 7f7621ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.014+0000 7f7621ffb640 1 -- 172.21.15.43:0/3247835303 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763409de60 con 0x56068d5c7b60 2024-04-21T08:43:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.014+0000 7f7621ffb640 1 -- 172.21.15.43:0/3247835303 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f7634018b20 con 0x56068d5c7b60 2024-04-21T08:43:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.014+0000 7f7643409740 1 -- 172.21.15.43:0/3247835303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f762402c7e0 con 0x56068d5c7b60 2024-04-21T08:43:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.014+0000 7f7621ffb640 1 -- 172.21.15.43:0/3247835303 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f763409c490 con 0x56068d5c7b60 2024-04-21T08:43:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.014+0000 7f7643409740 10 client.17631 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.014+0000 7f7643409740 1 -- 172.21.15.43:0/3247835303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56068d647cd0 con 0x56068d5c7b60 2024-04-21T08:43:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.014+0000 7f7621ffb640 1 -- 172.21.15.43:0/3247835303 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f762402c7e0 con 0x56068d5c7b60 2024-04-21T08:43:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.014+0000 7f7621ffb640 1 -- 172.21.15.43:0/3247835303 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f763409b9f0 con 0x56068d5c7b60 2024-04-21T08:43:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.014+0000 7f7643409740 10 client.17631 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.015+0000 7f7643409740 10 client.17631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.015+0000 7f7643409740 20 client.17631 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.015+0000 7f7643409740 1 --2- 172.21.15.43:0/3247835303 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56068d6484d0 0x56068d64a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.015+0000 7f7643409740 4 client.17631 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:43:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.015+0000 7f7643409740 1 -- 172.21.15.43:0/3247835303 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56068d4cc3d0 con 0x56068d6484d0 2024-04-21T08:43:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.015+0000 7f7639d9b640 1 --2- 172.21.15.43:0/3247835303 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56068d6484d0 0x56068d64a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.015+0000 7f7621ffb640 10 client.17631 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.015+0000 7f7639d9b640 1 --2- 172.21.15.43:0/3247835303 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56068d6484d0 0x56068d64a8f0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63b1670b99045a54 server_cookie=1b732e3b5ff42d82 in_seq=0 out_seq=0 2024-04-21T08:43:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.029+0000 7f7621ffb640 1 -- 172.21.15.43:0/3247835303 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56068d4cc3d0 con 0x56068d6484d0 2024-04-21T08:43:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.029+0000 7f7621ffb640 10 client.17631 handle_command_reply: tid=0 2024-04-21T08:43:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.029+0000 7f7643409740 1 client.17631 shutdown 2024-04-21T08:43:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.029+0000 7f7643409740 1 -- 172.21.15.43:0/3247835303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5c7b60 msgr2=0x56068d5d91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.029+0000 7f7643409740 1 --2- 172.21.15.43:0/3247835303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5c7b60 0x56068d5d91c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f7634098f90 tx=0x7f7634000bc0 comp rx=0 tx=0).stop 2024-04-21T08:43:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.029+0000 7f7643409740 1 -- 172.21.15.43:0/3247835303 shutdown_connections 2024-04-21T08:43:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.029+0000 7f7643409740 1 --2- 172.21.15.43:0/3247835303 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56068d6484d0 0x56068d64a8f0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.029+0000 7f7643409740 1 --2- 172.21.15.43:0/3247835303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56068d5d10e0 0x56068d5d2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.029+0000 7f7643409740 1 --2- 172.21.15.43:0/3247835303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56068d5c88d0 0x56068d5d9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.029+0000 7f7643409740 1 --2- 172.21.15.43:0/3247835303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56068d5c7b60 0x56068d5d91c0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.029+0000 7f7621ffb640 0 client.17631 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.029+0000 7f7643409740 1 -- 172.21.15.43:0/3247835303 >> 172.21.15.43:0/3247835303 conn(0x56068d58ec90 msgr2=0x56068d5c1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.029+0000 7f7643409740 1 -- 172.21.15.43:0/3247835303 shutdown_connections 2024-04-21T08:43:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.029+0000 7f7643409740 1 -- 172.21.15.43:0/3247835303 wait complete. 2024-04-21T08:43:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.029+0000 7f7643409740 20 client.17631 trim_cache size 0 max 16384 2024-04-21T08:43:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:43:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:43:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:43:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:43:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.030+0000 7f7640814640 1 -- 172.21.15.43:0/2608621012 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f76280a3df0 msgr2=0x7f76280a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.030+0000 7f7640814640 1 --2- 172.21.15.43:0/2608621012 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f76280a3df0 0x7f76280a62b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f762c009fd0 tx=0x7f762c0091e0 comp rx=0 tx=0).stop 2024-04-21T08:43:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.030+0000 7f7640814640 1 -- 172.21.15.43:0/2608621012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f763c1021c0 msgr2=0x7f763c074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.030+0000 7f7640814640 1 --2- 172.21.15.43:0/2608621012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f763c1021c0 0x7f763c074250 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f763400e9f0 tx=0x7f763400eeb0 comp rx=0 tx=0).stop 2024-04-21T08:43:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.030+0000 7f7640814640 1 -- 172.21.15.43:0/2608621012 shutdown_connections 2024-04-21T08:43:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.030+0000 7f7640814640 1 --2- 172.21.15.43:0/2608621012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f763c074cd0 0x7f763c1a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.030+0000 7f7640814640 1 --2- 172.21.15.43:0/2608621012 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f76280a3df0 0x7f76280a62b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.030+0000 7f7640814640 1 --2- 172.21.15.43:0/2608621012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f763c107290 0x7f763c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.030+0000 7f7640814640 1 --2- 172.21.15.43:0/2608621012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f763c1021c0 0x7f763c074250 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.030+0000 7f7640814640 1 -- 172.21.15.43:0/2608621012 >> 172.21.15.43:0/2608621012 conn(0x7f763c0fda20 msgr2=0x7f763c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.030+0000 7f7640814640 1 -- 172.21.15.43:0/2608621012 shutdown_connections 2024-04-21T08:43:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:25.030+0000 7f7640814640 1 -- 172.21.15.43:0/2608621012 wait complete. 2024-04-21T08:43:25.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:25 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:43:25.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:25 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:43:25.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:25 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:43:26.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:26 smithi112 ceph-mon[31085]: pgmap v138: 129 pgs: 129 active+clean; 5.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 16 op/s 2024-04-21T08:43:26.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:26 smithi043 ceph-mon[27978]: pgmap v138: 129 pgs: 129 active+clean; 5.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 16 op/s 2024-04-21T08:43:26.614 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:26 smithi175 ceph-mon[31920]: pgmap v138: 129 pgs: 129 active+clean; 5.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 16 op/s 2024-04-21T08:43:28.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:28 smithi112 ceph-mon[31085]: pgmap v139: 129 pgs: 129 active+clean; 1.7 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 27 op/s 2024-04-21T08:43:28.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:28 smithi043 ceph-mon[27978]: pgmap v139: 129 pgs: 129 active+clean; 1.7 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 27 op/s 2024-04-21T08:43:28.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:28 smithi175 ceph-mon[31920]: pgmap v139: 129 pgs: 129 active+clean; 1.7 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 27 op/s 2024-04-21T08:43:30.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:30 smithi112 ceph-mon[31085]: pgmap v140: 129 pgs: 129 active+clean; 1.7 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 18 op/s 2024-04-21T08:43:30.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:30 smithi043 ceph-mon[27978]: pgmap v140: 129 pgs: 129 active+clean; 1.7 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 18 op/s 2024-04-21T08:43:30.673 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.673 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.673 INFO:tasks.workunit.client.0.smithi043.stdout:127/600: open fd = 3 2024-04-21T08:43:30.673 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.673 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.673 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.673 INFO:tasks.workunit.client.0.smithi043.stdout:128/600: open fd = 3 2024-04-21T08:43:30.673 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.674 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.674 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.674 INFO:tasks.workunit.client.0.smithi043.stdout:129/600: open fd = 3 2024-04-21T08:43:30.674 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.674 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.674 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.674 INFO:tasks.workunit.client.0.smithi043.stdout:130/600: open fd = 3 2024-04-21T08:43:30.674 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.674 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.674 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.674 INFO:tasks.workunit.client.0.smithi043.stdout:131/600: open fd = 3 2024-04-21T08:43:30.674 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.674 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.674 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.675 INFO:tasks.workunit.client.0.smithi043.stdout:132/600: open fd = 3 2024-04-21T08:43:30.675 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.675 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.675 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.675 INFO:tasks.workunit.client.0.smithi043.stdout:133/600: open fd = 3 2024-04-21T08:43:30.675 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.675 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.675 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.675 INFO:tasks.workunit.client.0.smithi043.stdout:134/600: open fd = 3 2024-04-21T08:43:30.675 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.675 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.675 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.675 INFO:tasks.workunit.client.0.smithi043.stdout:135/600: open fd = 3 2024-04-21T08:43:30.675 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.675 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.675 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.676 INFO:tasks.workunit.client.0.smithi043.stdout:136/600: open fd = 3 2024-04-21T08:43:30.676 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.676 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.676 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.676 INFO:tasks.workunit.client.0.smithi043.stdout:137/600: open fd = 3 2024-04-21T08:43:30.676 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.676 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.676 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.676 INFO:tasks.workunit.client.0.smithi043.stdout:138/600: open fd = 3 2024-04-21T08:43:30.676 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.676 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.676 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.676 INFO:tasks.workunit.client.0.smithi043.stdout:139/600: open fd = 3 2024-04-21T08:43:30.676 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.676 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.677 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.677 INFO:tasks.workunit.client.0.smithi043.stdout:140/600: open fd = 3 2024-04-21T08:43:30.677 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.677 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.677 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.677 INFO:tasks.workunit.client.0.smithi043.stdout:141/600: open fd = 3 2024-04-21T08:43:30.677 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.677 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.677 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.677 INFO:tasks.workunit.client.0.smithi043.stdout:142/600: open fd = 3 2024-04-21T08:43:30.677 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.677 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.677 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.677 INFO:tasks.workunit.client.0.smithi043.stdout:143/600: open fd = 3 2024-04-21T08:43:30.677 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.678 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.678 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.678 INFO:tasks.workunit.client.0.smithi043.stdout:144/600: open fd = 3 2024-04-21T08:43:30.678 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.678 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.678 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.678 INFO:tasks.workunit.client.0.smithi043.stdout:145/600: open fd = 3 2024-04-21T08:43:30.678 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.678 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.678 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.678 INFO:tasks.workunit.client.0.smithi043.stdout:146/600: open fd = 3 2024-04-21T08:43:30.678 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.678 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.678 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.678 INFO:tasks.workunit.client.0.smithi043.stdout:147/600: open fd = 3 2024-04-21T08:43:30.679 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.679 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.679 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.679 INFO:tasks.workunit.client.0.smithi043.stdout:148/600: open fd = 3 2024-04-21T08:43:30.679 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.679 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.679 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.679 INFO:tasks.workunit.client.0.smithi043.stdout:149/600: open fd = 3 2024-04-21T08:43:30.679 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.679 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.679 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.679 INFO:tasks.workunit.client.0.smithi043.stdout:150/600: open fd = 3 2024-04-21T08:43:30.679 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.679 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.679 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.680 INFO:tasks.workunit.client.0.smithi043.stdout:151/600: open fd = 3 2024-04-21T08:43:30.680 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.680 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.680 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.680 INFO:tasks.workunit.client.0.smithi043.stdout:152/600: open fd = 3 2024-04-21T08:43:30.680 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.680 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.680 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.680 INFO:tasks.workunit.client.0.smithi043.stdout:153/600: open fd = 3 2024-04-21T08:43:30.680 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.680 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.680 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.680 INFO:tasks.workunit.client.0.smithi043.stdout:154/600: open fd = 3 2024-04-21T08:43:30.680 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.680 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.681 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.681 INFO:tasks.workunit.client.0.smithi043.stdout:155/600: open fd = 3 2024-04-21T08:43:30.681 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.681 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.681 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.681 INFO:tasks.workunit.client.0.smithi043.stdout:156/600: open fd = 3 2024-04-21T08:43:30.681 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.681 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.681 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.681 INFO:tasks.workunit.client.0.smithi043.stdout:157/600: open fd = 3 2024-04-21T08:43:30.681 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.681 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.681 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.681 INFO:tasks.workunit.client.0.smithi043.stdout:158/600: open fd = 3 2024-04-21T08:43:30.681 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.682 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.682 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.682 INFO:tasks.workunit.client.0.smithi043.stdout:159/600: open fd = 3 2024-04-21T08:43:30.682 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.682 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.682 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.682 INFO:tasks.workunit.client.0.smithi043.stdout:160/600: open fd = 3 2024-04-21T08:43:30.682 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.682 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.682 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.682 INFO:tasks.workunit.client.0.smithi043.stdout:161/600: open fd = 3 2024-04-21T08:43:30.682 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.682 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.682 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.683 INFO:tasks.workunit.client.0.smithi043.stdout:162/600: open fd = 3 2024-04-21T08:43:30.683 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.683 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.683 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.683 INFO:tasks.workunit.client.0.smithi043.stdout:163/600: open fd = 3 2024-04-21T08:43:30.683 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.683 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.683 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.683 INFO:tasks.workunit.client.0.smithi043.stdout:164/600: open fd = 3 2024-04-21T08:43:30.683 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.683 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.683 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.683 INFO:tasks.workunit.client.0.smithi043.stdout:165/600: open fd = 3 2024-04-21T08:43:30.683 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.683 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.684 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.684 INFO:tasks.workunit.client.0.smithi043.stdout:166/600: open fd = 3 2024-04-21T08:43:30.684 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.684 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.684 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.684 INFO:tasks.workunit.client.0.smithi043.stdout:167/600: open fd = 3 2024-04-21T08:43:30.684 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.684 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.684 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.684 INFO:tasks.workunit.client.0.smithi043.stdout:168/600: open fd = 3 2024-04-21T08:43:30.684 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.684 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.684 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.684 INFO:tasks.workunit.client.0.smithi043.stdout:169/600: open fd = 3 2024-04-21T08:43:30.684 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.685 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.685 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.685 INFO:tasks.workunit.client.0.smithi043.stdout:170/600: open fd = 3 2024-04-21T08:43:30.685 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.685 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.685 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.685 INFO:tasks.workunit.client.0.smithi043.stdout:171/600: open fd = 3 2024-04-21T08:43:30.685 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.685 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.685 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.685 INFO:tasks.workunit.client.0.smithi043.stdout:172/600: open fd = 3 2024-04-21T08:43:30.685 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.685 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.685 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.685 INFO:tasks.workunit.client.0.smithi043.stdout:173/600: open fd = 3 2024-04-21T08:43:30.686 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.686 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.686 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.686 INFO:tasks.workunit.client.0.smithi043.stdout:174/600: open fd = 3 2024-04-21T08:43:30.686 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.686 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.686 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.686 INFO:tasks.workunit.client.0.smithi043.stdout:175/600: open fd = 3 2024-04-21T08:43:30.686 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.686 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.686 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.686 INFO:tasks.workunit.client.0.smithi043.stdout:176/600: open fd = 3 2024-04-21T08:43:30.686 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.686 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.687 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.687 INFO:tasks.workunit.client.0.smithi043.stdout:177/600: open fd = 3 2024-04-21T08:43:30.687 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.687 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.687 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.687 INFO:tasks.workunit.client.0.smithi043.stdout:178/600: open fd = 3 2024-04-21T08:43:30.687 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.687 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.687 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.687 INFO:tasks.workunit.client.0.smithi043.stdout:179/600: open fd = 3 2024-04-21T08:43:30.687 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.687 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.687 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.687 INFO:tasks.workunit.client.0.smithi043.stdout:180/600: open fd = 3 2024-04-21T08:43:30.687 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.688 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.688 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.688 INFO:tasks.workunit.client.0.smithi043.stdout:181/600: open fd = 3 2024-04-21T08:43:30.688 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.688 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.688 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.688 INFO:tasks.workunit.client.0.smithi043.stdout:182/600: open fd = 3 2024-04-21T08:43:30.688 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.688 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.688 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.688 INFO:tasks.workunit.client.0.smithi043.stdout:183/600: open fd = 3 2024-04-21T08:43:30.688 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.688 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.688 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.689 INFO:tasks.workunit.client.0.smithi043.stdout:184/600: open fd = 3 2024-04-21T08:43:30.689 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.689 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.689 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.689 INFO:tasks.workunit.client.0.smithi043.stdout:185/600: open fd = 3 2024-04-21T08:43:30.689 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.689 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.689 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.689 INFO:tasks.workunit.client.0.smithi043.stdout:186/600: open fd = 3 2024-04-21T08:43:30.689 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.689 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.689 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.689 INFO:tasks.workunit.client.0.smithi043.stdout:187/600: open fd = 3 2024-04-21T08:43:30.689 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.689 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.689 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:43:30.690 INFO:tasks.workunit.client.0.smithi043.stdout:188/600: open fd = 3 2024-04-21T08:43:30.690 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.690 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:43:30.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:30 smithi175 ceph-mon[31920]: pgmap v140: 129 pgs: 129 active+clean; 1.7 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 18 op/s 2024-04-21T08:43:32.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:32 smithi112 ceph-mon[31085]: pgmap v141: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 47 KiB/s wr, 24 op/s 2024-04-21T08:43:32.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:32 smithi043 ceph-mon[27978]: pgmap v141: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 47 KiB/s wr, 24 op/s 2024-04-21T08:43:32.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:32 smithi175 ceph-mon[31920]: pgmap v141: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 47 KiB/s wr, 24 op/s 2024-04-21T08:43:34.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:34 smithi112 ceph-mon[31085]: pgmap v142: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 17 op/s 2024-04-21T08:43:34.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:34 smithi043 ceph-mon[27978]: pgmap v142: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 17 op/s 2024-04-21T08:43:34.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:34 smithi175 ceph-mon[31920]: pgmap v142: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 17 op/s 2024-04-21T08:43:36.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:36 smithi112 ceph-mon[31085]: pgmap v143: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 17 op/s 2024-04-21T08:43:36.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:36 smithi043 ceph-mon[27978]: pgmap v143: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 17 op/s 2024-04-21T08:43:36.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:36 smithi175 ceph-mon[31920]: pgmap v143: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 17 op/s 2024-04-21T08:43:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.128+0000 7f292297e640 1 -- 172.21.15.43:0/1884701154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f291c103180 msgr2=0x7f291c103560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.128+0000 7f292297e640 1 --2- 172.21.15.43:0/1884701154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f291c103180 0x7f291c103560 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f290c009900 tx=0x7f290c02bcd0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.128+0000 7f292297e640 1 -- 172.21.15.43:0/1884701154 shutdown_connections 2024-04-21T08:43:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.128+0000 7f292297e640 1 --2- 172.21.15.43:0/1884701154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f291c10c550 0x7f291c10e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.128+0000 7f292297e640 1 --2- 172.21.15.43:0/1884701154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f291c103aa0 0x7f291c10bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.128+0000 7f292297e640 1 --2- 172.21.15.43:0/1884701154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f291c103180 0x7f291c103560 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.128+0000 7f292297e640 1 -- 172.21.15.43:0/1884701154 >> 172.21.15.43:0/1884701154 conn(0x7f291c077fd0 msgr2=0x7f291c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.128+0000 7f292297e640 1 -- 172.21.15.43:0/1884701154 shutdown_connections 2024-04-21T08:43:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.128+0000 7f292297e640 1 -- 172.21.15.43:0/1884701154 wait complete. 2024-04-21T08:43:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.129+0000 7f292297e640 1 Processor -- start 2024-04-21T08:43:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.129+0000 7f292297e640 1 -- start start 2024-04-21T08:43:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.129+0000 7f292297e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f291c103180 0x7f291c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.129+0000 7f292297e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f291c103aa0 0x7f291c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.129+0000 7f292297e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f291c10c550 0x7f291c19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.129+0000 7f292297e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f291c1121b0 con 0x7f291c10c550 2024-04-21T08:43:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.129+0000 7f292297e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f291c112030 con 0x7f291c103180 2024-04-21T08:43:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.129+0000 7f292297e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f291c112330 con 0x7f291c103aa0 2024-04-21T08:43:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.130+0000 7f2920f1b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f291c10c550 0x7f291c19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.130+0000 7f2920f1b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f291c10c550 0x7f291c19cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34168/0 (socket says 172.21.15.43:34168) 2024-04-21T08:43:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.130+0000 7f2920f1b640 1 -- 172.21.15.43:0/3050035982 learned_addr learned my addr 172.21.15.43:0/3050035982 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.130+0000 7f291b7fe640 1 --2- 172.21.15.43:0/3050035982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f291c103aa0 0x7f291c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.130+0000 7f291bfff640 1 --2- 172.21.15.43:0/3050035982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f291c103180 0x7f291c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.130+0000 7f2920f1b640 1 -- 172.21.15.43:0/3050035982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f291c103aa0 msgr2=0x7f291c198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.130+0000 7f2920f1b640 1 --2- 172.21.15.43:0/3050035982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f291c103aa0 0x7f291c198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.130+0000 7f2920f1b640 1 -- 172.21.15.43:0/3050035982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f291c103180 msgr2=0x7f291c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.130+0000 7f2920f1b640 1 --2- 172.21.15.43:0/3050035982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f291c103180 0x7f291c1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.130+0000 7f2920f1b640 1 -- 172.21.15.43:0/3050035982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291c19d6e0 con 0x7f291c10c550 2024-04-21T08:43:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.130+0000 7f291b7fe640 1 --2- 172.21.15.43:0/3050035982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f291c103aa0 0x7f291c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.130+0000 7f2920f1b640 1 --2- 172.21.15.43:0/3050035982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f291c10c550 0x7f291c19cfb0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f2914009fd0 tx=0x7f291400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.130+0000 7f29197fa640 1 -- 172.21.15.43:0/3050035982 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914017070 con 0x7f291c10c550 2024-04-21T08:43:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.130+0000 7f29197fa640 1 -- 172.21.15.43:0/3050035982 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2914003e20 con 0x7f291c10c550 2024-04-21T08:43:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.130+0000 7f292297e640 1 -- 172.21.15.43:0/3050035982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291c19d9d0 con 0x7f291c10c550 2024-04-21T08:43:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.130+0000 7f29197fa640 1 -- 172.21.15.43:0/3050035982 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914010460 con 0x7f291c10c550 2024-04-21T08:43:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.130+0000 7f292297e640 1 -- 172.21.15.43:0/3050035982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f291c1a5260 con 0x7f291c10c550 2024-04-21T08:43:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.133+0000 7f29197fa640 1 -- 172.21.15.43:0/3050035982 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2914003970 con 0x7f291c10c550 2024-04-21T08:43:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.133+0000 7f2925569740 1 Processor -- start 2024-04-21T08:43:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.133+0000 7f2925569740 1 -- start start 2024-04-21T08:43:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.133+0000 7f2925569740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560207a3b8d0 0x560207a3a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.133+0000 7f2925569740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a3ab60 0x560207a3afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.133+0000 7f2925569740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560207a440e0 0x560207a464d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.133+0000 7f2925569740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56020799f8d0 con 0x560207a3ab60 2024-04-21T08:43:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.133+0000 7f2925569740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56020793ce80 con 0x560207a440e0 2024-04-21T08:43:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.133+0000 7f2925569740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56020799b890 con 0x560207a3b8d0 2024-04-21T08:43:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.134+0000 7f291bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a3ab60 0x560207a3afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.134+0000 7f29197fa640 1 --2- 172.21.15.43:0/3050035982 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f29040a3dc0 0x7f29040a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.134+0000 7f291bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a3ab60 0x560207a3afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34172/0 (socket says 172.21.15.43:34172) 2024-04-21T08:43:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.134+0000 7f291bfff640 1 -- 172.21.15.43:0/1499534441 learned_addr learned my addr 172.21.15.43:0/1499534441 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.134+0000 7f291b7fe640 1 --2- 172.21.15.43:0/1499534441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560207a3b8d0 0x560207a3a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.134+0000 7f291b7fe640 1 --2- 172.21.15.43:0/1499534441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560207a440e0 0x560207a464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.134+0000 7f2920f1b640 1 --2- 172.21.15.43:0/3050035982 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f29040a3dc0 0x7f29040a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.134+0000 7f291bfff640 1 -- 172.21.15.43:0/1499534441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560207a3b8d0 msgr2=0x560207a3a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.134+0000 7f29197fa640 1 -- 172.21.15.43:0/3050035982 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f29140d0a00 con 0x7f291c10c550 2024-04-21T08:43:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.134+0000 7f291bfff640 1 --2- 172.21.15.43:0/1499534441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560207a3b8d0 0x560207a3a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.134+0000 7f291bfff640 1 -- 172.21.15.43:0/1499534441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560207a440e0 msgr2=0x560207a464d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.134+0000 7f291bfff640 1 --2- 172.21.15.43:0/1499534441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560207a440e0 0x560207a464d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.134+0000 7f291bfff640 1 -- 172.21.15.43:0/1499534441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56020795e320 con 0x560207a3ab60 2024-04-21T08:43:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.134+0000 7f291b7fe640 1 --2- 172.21.15.43:0/1499534441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560207a440e0 0x560207a464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.134+0000 7f291bfff640 1 --2- 172.21.15.43:0/1499534441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a3ab60 0x560207a3afe0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f290c009900 tx=0x7f290c005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67aeaa6ad9b374e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.134+0000 7f2920f1b640 1 --2- 172.21.15.43:0/3050035982 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f29040a3dc0 0x7f29040a6280 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f29140989d0 tx=0x7f2914091a00 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.135+0000 7f29027fc640 1 -- 172.21.15.43:0/1499534441 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c004240 con 0x560207a3ab60 2024-04-21T08:43:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.135+0000 7f29027fc640 1 -- 172.21.15.43:0/1499534441 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f290c0043e0 con 0x560207a3ab60 2024-04-21T08:43:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.135+0000 7f29027fc640 1 -- 172.21.15.43:0/1499534441 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c0046c0 con 0x560207a3ab60 2024-04-21T08:43:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.135+0000 7f2925569740 1 -- 172.21.15.43:0/1499534441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a3ab60 msgr2=0x560207a3afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.135+0000 7f2925569740 1 --2- 172.21.15.43:0/1499534441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a3ab60 0x560207a3afe0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f290c009900 tx=0x7f290c005e70 comp rx=0 tx=0).stop 2024-04-21T08:43:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.135+0000 7f2925569740 1 -- 172.21.15.43:0/1499534441 shutdown_connections 2024-04-21T08:43:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.135+0000 7f2925569740 1 --2- 172.21.15.43:0/1499534441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560207a440e0 0x560207a464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.135+0000 7f2925569740 1 --2- 172.21.15.43:0/1499534441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a3ab60 0x560207a3afe0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.135+0000 7f2925569740 1 --2- 172.21.15.43:0/1499534441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560207a3b8d0 0x560207a3a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.135+0000 7f2925569740 1 -- 172.21.15.43:0/1499534441 >> 172.21.15.43:0/1499534441 conn(0x560207a01c90 msgr2=0x560207a35850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.135+0000 7f2925569740 1 -- 172.21.15.43:0/1499534441 shutdown_connections 2024-04-21T08:43:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.135+0000 7f2925569740 1 -- 172.21.15.43:0/1499534441 wait complete. 2024-04-21T08:43:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.136+0000 7f2925569740 1 -- start start 2024-04-21T08:43:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.136+0000 7f2925569740 1 Processor -- start 2024-04-21T08:43:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.136+0000 7f2925569740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560207a3ab60 0x560207a43a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.136+0000 7f2925569740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a3b8d0 0x560207a40de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.136+0000 7f2925569740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560207a440e0 0x560207a41340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.136+0000 7f2925569740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560207a46db0 con 0x560207a3b8d0 2024-04-21T08:43:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.136+0000 7f2925569740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560207927eb0 con 0x560207a440e0 2024-04-21T08:43:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.136+0000 7f2925569740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560207a46f30 con 0x560207a3ab60 2024-04-21T08:43:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.136+0000 7f2925569740 10 client.0 _ll_register_callbacks cb 0x560207954b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.136+0000 7f291bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a3b8d0 0x560207a40de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.136+0000 7f291bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a3b8d0 0x560207a40de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34184/0 (socket says 172.21.15.43:34184) 2024-04-21T08:43:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.136+0000 7f291bfff640 1 -- 172.21.15.43:0/1313153739 learned_addr learned my addr 172.21.15.43:0/1313153739 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.137+0000 7f291b7fe640 1 --2- 172.21.15.43:0/1313153739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560207a3ab60 0x560207a43a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.137+0000 7f291b7fe640 1 --2- 172.21.15.43:0/1313153739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560207a440e0 0x560207a41340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.137+0000 7f291bfff640 1 -- 172.21.15.43:0/1313153739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560207a3ab60 msgr2=0x560207a43a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.137+0000 7f291bfff640 1 --2- 172.21.15.43:0/1313153739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560207a3ab60 0x560207a43a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.137+0000 7f291bfff640 1 -- 172.21.15.43:0/1313153739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560207a440e0 msgr2=0x560207a41340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.137+0000 7f291bfff640 1 --2- 172.21.15.43:0/1313153739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560207a440e0 0x560207a41340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.137+0000 7f291bfff640 1 -- 172.21.15.43:0/1313153739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602076b2210 con 0x560207a3b8d0 2024-04-21T08:43:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.137+0000 7f291b7fe640 1 --2- 172.21.15.43:0/1313153739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560207a3ab60 0x560207a43a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.137+0000 7f291bfff640 1 --2- 172.21.15.43:0/1313153739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a3b8d0 0x560207a40de0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f290c02c210 tx=0x7f290c02bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed7b9d58e74fd9f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.137+0000 7f28fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.137+0000 7f28fb7fe640 1 -- 172.21.15.43:0/1313153739 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c04e070 con 0x560207a3b8d0 2024-04-21T08:43:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.137+0000 7f28fb7fe640 1 -- 172.21.15.43:0/1313153739 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f290c034b90 con 0x560207a3b8d0 2024-04-21T08:43:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.137+0000 7f2925569740 1 -- 172.21.15.43:0/1313153739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f290c0043e0 con 0x560207a3b8d0 2024-04-21T08:43:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.137+0000 7f28fb7fe640 1 -- 172.21.15.43:0/1313153739 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f290c05a050 con 0x560207a3b8d0 2024-04-21T08:43:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.137+0000 7f2925569740 10 client.17643 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.137+0000 7f2925569740 1 -- 172.21.15.43:0/1313153739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56020777e620 con 0x560207a3b8d0 2024-04-21T08:43:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.137+0000 7f28fb7fe640 1 -- 172.21.15.43:0/1313153739 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f290c0043e0 con 0x560207a3b8d0 2024-04-21T08:43:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.138+0000 7f28fb7fe640 1 -- 172.21.15.43:0/1313153739 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f290c0463f0 con 0x560207a3b8d0 2024-04-21T08:43:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.138+0000 7f2925569740 10 client.17643 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.138+0000 7f2925569740 10 client.17643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.138+0000 7f2925569740 20 client.17643 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.138+0000 7f2925569740 1 --2- 172.21.15.43:0/1313153739 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560207a53f10 0x560207a56300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.138+0000 7f2925569740 4 client.17643 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.138+0000 7f2925569740 1 -- 172.21.15.43:0/1313153739 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602079b3880 con 0x560207a53f10 2024-04-21T08:43:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.138+0000 7f291b7fe640 1 --2- 172.21.15.43:0/1313153739 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560207a53f10 0x560207a56300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.138+0000 7f28fb7fe640 10 client.17643 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.138+0000 7f291b7fe640 1 --2- 172.21.15.43:0/1313153739 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560207a53f10 0x560207a56300 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79684b9b76d938e server_cookie=8954474252d5b77c in_seq=0 out_seq=0 2024-04-21T08:43:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.139+0000 7f28fb7fe640 1 -- 172.21.15.43:0/1313153739 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5602079b3880 con 0x560207a53f10 2024-04-21T08:43:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.139+0000 7f28fb7fe640 10 client.17643 handle_command_reply: tid=0 2024-04-21T08:43:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.139+0000 7f2925569740 1 client.17643 shutdown 2024-04-21T08:43:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.139+0000 7f2925569740 1 -- 172.21.15.43:0/1313153739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a3b8d0 msgr2=0x560207a40de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.139+0000 7f2925569740 1 --2- 172.21.15.43:0/1313153739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a3b8d0 0x560207a40de0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f290c02c210 tx=0x7f290c02bcd0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.140+0000 7f2925569740 1 -- 172.21.15.43:0/1313153739 shutdown_connections 2024-04-21T08:43:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.140+0000 7f2925569740 1 --2- 172.21.15.43:0/1313153739 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560207a53f10 0x560207a56300 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.140+0000 7f2925569740 1 --2- 172.21.15.43:0/1313153739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560207a440e0 0x560207a41340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.140+0000 7f2925569740 1 --2- 172.21.15.43:0/1313153739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a3b8d0 0x560207a40de0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.140+0000 7f28fb7fe640 0 client.17643 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.140+0000 7f2925569740 1 --2- 172.21.15.43:0/1313153739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560207a3ab60 0x560207a43a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.140+0000 7f2925569740 1 -- 172.21.15.43:0/1313153739 >> 172.21.15.43:0/1313153739 conn(0x560207a01c90 msgr2=0x560207a375a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.140+0000 7f2925569740 1 -- 172.21.15.43:0/1313153739 shutdown_connections 2024-04-21T08:43:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.140+0000 7f2925569740 1 -- 172.21.15.43:0/1313153739 wait complete. 2024-04-21T08:43:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.140+0000 7f2925569740 20 client.17643 trim_cache size 0 max 16384 2024-04-21T08:43:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.155+0000 7f2925569740 1 Processor -- start 2024-04-21T08:43:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.155+0000 7f2925569740 1 -- start start 2024-04-21T08:43:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.155+0000 7f2925569740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560207a3ab60 0x560207a4fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.155+0000 7f2925569740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560207a3b8d0 0x560207a3c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.155+0000 7f2925569740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a440e0 0x560207a43510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.155+0000 7f2925569740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f291c1af3b0 con 0x560207a440e0 2024-04-21T08:43:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.155+0000 7f2925569740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f291c1b0ef0 con 0x560207a3b8d0 2024-04-21T08:43:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.155+0000 7f2925569740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f291c1afa80 con 0x560207a3ab60 2024-04-21T08:43:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.155+0000 7f291b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a440e0 0x560207a43510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.155+0000 7f291b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a440e0 0x560207a43510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34198/0 (socket says 172.21.15.43:34198) 2024-04-21T08:43:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.155+0000 7f291b7fe640 1 -- 172.21.15.43:0/484984235 learned_addr learned my addr 172.21.15.43:0/484984235 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.155+0000 7f291bfff640 1 --2- 172.21.15.43:0/484984235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560207a3b8d0 0x560207a3c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.155+0000 7f291b7fe640 1 --2- 172.21.15.43:0/484984235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560207a3ab60 0x560207a4fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.155+0000 7f291b7fe640 1 -- 172.21.15.43:0/484984235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560207a3ab60 msgr2=0x560207a4fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.155+0000 7f291b7fe640 1 --2- 172.21.15.43:0/484984235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560207a3ab60 0x560207a4fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.155+0000 7f291b7fe640 1 -- 172.21.15.43:0/484984235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560207a3b8d0 msgr2=0x560207a3c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.155+0000 7f291b7fe640 1 --2- 172.21.15.43:0/484984235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560207a3b8d0 0x560207a3c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.155+0000 7f291b7fe640 1 -- 172.21.15.43:0/484984235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560207a46f40 con 0x560207a440e0 2024-04-21T08:43:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.156+0000 7f291b7fe640 1 --2- 172.21.15.43:0/484984235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a440e0 0x560207a43510 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f291000c7e0 tx=0x7f2910018040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a36a5e6eab6d89 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.156+0000 7f2901ffb640 1 -- 172.21.15.43:0/484984235 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2910004020 con 0x560207a440e0 2024-04-21T08:43:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.156+0000 7f2901ffb640 1 -- 172.21.15.43:0/484984235 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f291001f6f0 con 0x560207a440e0 2024-04-21T08:43:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.156+0000 7f2901ffb640 1 -- 172.21.15.43:0/484984235 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2910003930 con 0x560207a440e0 2024-04-21T08:43:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.156+0000 7f2925569740 1 -- 172.21.15.43:0/484984235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a440e0 msgr2=0x560207a43510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.156+0000 7f2925569740 1 --2- 172.21.15.43:0/484984235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a440e0 0x560207a43510 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f291000c7e0 tx=0x7f2910018040 comp rx=0 tx=0).stop 2024-04-21T08:43:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.156+0000 7f2925569740 1 -- 172.21.15.43:0/484984235 shutdown_connections 2024-04-21T08:43:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.156+0000 7f2925569740 1 --2- 172.21.15.43:0/484984235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a440e0 0x560207a43510 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.156+0000 7f2925569740 1 --2- 172.21.15.43:0/484984235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560207a3b8d0 0x560207a3c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.156+0000 7f2925569740 1 --2- 172.21.15.43:0/484984235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560207a3ab60 0x560207a4fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.156+0000 7f2925569740 1 -- 172.21.15.43:0/484984235 >> 172.21.15.43:0/484984235 conn(0x560207a01c90 msgr2=0x560207a3bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.157+0000 7f2925569740 1 -- 172.21.15.43:0/484984235 shutdown_connections 2024-04-21T08:43:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.157+0000 7f2925569740 1 -- 172.21.15.43:0/484984235 wait complete. 2024-04-21T08:43:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.157+0000 7f2925569740 1 -- start start 2024-04-21T08:43:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.157+0000 7f2925569740 1 Processor -- start 2024-04-21T08:43:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.157+0000 7f2925569740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560207a3ab60 0x560207a4c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.157+0000 7f2925569740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a3b8d0 0x560207a4c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.157+0000 7f2925569740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560207a440e0 0x560207a491c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.157+0000 7f2925569740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f291c1b0820 con 0x560207a3b8d0 2024-04-21T08:43:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.157+0000 7f2925569740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f291c1b0150 con 0x560207a440e0 2024-04-21T08:43:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.157+0000 7f2925569740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f291c1ad390 con 0x560207a3ab60 2024-04-21T08:43:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.157+0000 7f2925569740 10 client.0 _ll_register_callbacks cb 0x7f291000cab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.157+0000 7f291bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a3b8d0 0x560207a4c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.158+0000 7f291bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a3b8d0 0x560207a4c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34208/0 (socket says 172.21.15.43:34208) 2024-04-21T08:43:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.158+0000 7f291bfff640 1 -- 172.21.15.43:0/791691536 learned_addr learned my addr 172.21.15.43:0/791691536 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.158+0000 7f291b7fe640 1 --2- 172.21.15.43:0/791691536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560207a3ab60 0x560207a4c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.158+0000 7f291b7fe640 1 --2- 172.21.15.43:0/791691536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560207a440e0 0x560207a491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.158+0000 7f291bfff640 1 -- 172.21.15.43:0/791691536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560207a3ab60 msgr2=0x560207a4c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.158+0000 7f291bfff640 1 --2- 172.21.15.43:0/791691536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560207a3ab60 0x560207a4c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.158+0000 7f291bfff640 1 -- 172.21.15.43:0/791691536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560207a440e0 msgr2=0x560207a491c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.158+0000 7f291bfff640 1 --2- 172.21.15.43:0/791691536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560207a440e0 0x560207a491c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.158+0000 7f291bfff640 1 -- 172.21.15.43:0/791691536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560207a497b0 con 0x560207a3b8d0 2024-04-21T08:43:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.158+0000 7f291bfff640 1 --2- 172.21.15.43:0/791691536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a3b8d0 0x560207a4c730 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f290c0059d0 tx=0x7f290c034ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595971de1920e66b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.158+0000 7f28fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.158+0000 7f28fb7fe640 1 -- 172.21.15.43:0/791691536 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c04e070 con 0x560207a3b8d0 2024-04-21T08:43:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.158+0000 7f28fb7fe640 1 -- 172.21.15.43:0/791691536 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f290c0469b0 con 0x560207a3b8d0 2024-04-21T08:43:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.158+0000 7f2925569740 1 -- 172.21.15.43:0/791691536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f291001f6f0 con 0x560207a3b8d0 2024-04-21T08:43:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.158+0000 7f28fb7fe640 1 -- 172.21.15.43:0/791691536 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f290c065050 con 0x560207a3b8d0 2024-04-21T08:43:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.158+0000 7f2925569740 10 client.17649 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.158+0000 7f2925569740 1 -- 172.21.15.43:0/791691536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560207abaab0 con 0x560207a3b8d0 2024-04-21T08:43:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.159+0000 7f28fb7fe640 1 -- 172.21.15.43:0/791691536 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f291001f6f0 con 0x560207a3b8d0 2024-04-21T08:43:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.159+0000 7f28fb7fe640 1 -- 172.21.15.43:0/791691536 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f290c0538f0 con 0x560207a3b8d0 2024-04-21T08:43:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.159+0000 7f2925569740 10 client.17649 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.159+0000 7f2925569740 10 client.17649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.159+0000 7f2925569740 20 client.17649 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.159+0000 7f2925569740 1 --2- 172.21.15.43:0/791691536 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560207abb140 0x560207abd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.159+0000 7f2925569740 4 client.17649 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:43:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.159+0000 7f2925569740 1 -- 172.21.15.43:0/791691536 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56020793f3d0 con 0x560207abb140 2024-04-21T08:43:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.159+0000 7f291b7fe640 1 --2- 172.21.15.43:0/791691536 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560207abb140 0x560207abd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.160+0000 7f291b7fe640 1 --2- 172.21.15.43:0/791691536 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560207abb140 0x560207abd560 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e1ea391c29119e1 server_cookie=b2c5e148ef7e3eab in_seq=0 out_seq=0 2024-04-21T08:43:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.160+0000 7f28fb7fe640 10 client.17649 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.174+0000 7f28fb7fe640 1 -- 172.21.15.43:0/791691536 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56020793f3d0 con 0x560207abb140 2024-04-21T08:43:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.174+0000 7f28fb7fe640 10 client.17649 handle_command_reply: tid=0 2024-04-21T08:43:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.174+0000 7f2925569740 1 client.17649 shutdown 2024-04-21T08:43:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.174+0000 7f2925569740 1 -- 172.21.15.43:0/791691536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a3b8d0 msgr2=0x560207a4c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.174+0000 7f2925569740 1 --2- 172.21.15.43:0/791691536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a3b8d0 0x560207a4c730 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f290c0059d0 tx=0x7f290c034ca0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.174+0000 7f2925569740 1 -- 172.21.15.43:0/791691536 shutdown_connections 2024-04-21T08:43:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.174+0000 7f2925569740 1 --2- 172.21.15.43:0/791691536 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560207abb140 0x560207abd560 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.175+0000 7f2925569740 1 --2- 172.21.15.43:0/791691536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560207a440e0 0x560207a491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.175+0000 7f2925569740 1 --2- 172.21.15.43:0/791691536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560207a3b8d0 0x560207a4c730 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.175+0000 7f28fb7fe640 0 client.17649 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.175+0000 7f2925569740 1 --2- 172.21.15.43:0/791691536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560207a3ab60 0x560207a4c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.175+0000 7f2925569740 1 -- 172.21.15.43:0/791691536 >> 172.21.15.43:0/791691536 conn(0x560207a01c90 msgr2=0x560207a34040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.175+0000 7f2925569740 1 -- 172.21.15.43:0/791691536 shutdown_connections 2024-04-21T08:43:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.175+0000 7f2925569740 1 -- 172.21.15.43:0/791691536 wait complete. 2024-04-21T08:43:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.175+0000 7f2925569740 20 client.17649 trim_cache size 0 max 16384 2024-04-21T08:43:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:43:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:43:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:43:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:43:38.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.175+0000 7f292297e640 1 -- 172.21.15.43:0/3050035982 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f29040a3dc0 msgr2=0x7f29040a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:38.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.175+0000 7f292297e640 1 --2- 172.21.15.43:0/3050035982 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f29040a3dc0 0x7f29040a6280 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f29140989d0 tx=0x7f2914091a00 comp rx=0 tx=0).stop 2024-04-21T08:43:38.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.175+0000 7f292297e640 1 -- 172.21.15.43:0/3050035982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f291c10c550 msgr2=0x7f291c19cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:38.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.175+0000 7f292297e640 1 --2- 172.21.15.43:0/3050035982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f291c10c550 0x7f291c19cfb0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f2914009fd0 tx=0x7f291400b260 comp rx=0 tx=0).stop 2024-04-21T08:43:38.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.175+0000 7f292297e640 1 -- 172.21.15.43:0/3050035982 shutdown_connections 2024-04-21T08:43:38.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.175+0000 7f292297e640 1 --2- 172.21.15.43:0/3050035982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f291c10c550 0x7f291c19cfb0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.175+0000 7f292297e640 1 --2- 172.21.15.43:0/3050035982 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f29040a3dc0 0x7f29040a6280 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.175+0000 7f292297e640 1 --2- 172.21.15.43:0/3050035982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f291c103aa0 0x7f291c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.175+0000 7f292297e640 1 --2- 172.21.15.43:0/3050035982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f291c103180 0x7f291c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.175+0000 7f292297e640 1 -- 172.21.15.43:0/3050035982 >> 172.21.15.43:0/3050035982 conn(0x7f291c077fd0 msgr2=0x7f291c0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.175+0000 7f292297e640 1 -- 172.21.15.43:0/3050035982 shutdown_connections 2024-04-21T08:43:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:38.175+0000 7f292297e640 1 -- 172.21.15.43:0/3050035982 wait complete. 2024-04-21T08:43:38.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:38 smithi112 ceph-mon[31085]: pgmap v144: 129 pgs: 129 active+clean; 2.0 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 47 KiB/s wr, 24 op/s 2024-04-21T08:43:38.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:38 smithi043 ceph-mon[27978]: pgmap v144: 129 pgs: 129 active+clean; 2.0 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 47 KiB/s wr, 24 op/s 2024-04-21T08:43:38.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:38 smithi175 ceph-mon[31920]: pgmap v144: 129 pgs: 129 active+clean; 2.0 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 47 KiB/s wr, 24 op/s 2024-04-21T08:43:40.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:40 smithi112 ceph-mon[31085]: pgmap v145: 129 pgs: 129 active+clean; 2.0 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-04-21T08:43:40.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:40 smithi043 ceph-mon[27978]: pgmap v145: 129 pgs: 129 active+clean; 2.0 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-04-21T08:43:40.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:40 smithi175 ceph-mon[31920]: pgmap v145: 129 pgs: 129 active+clean; 2.0 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-04-21T08:43:42.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:42 smithi112 ceph-mon[31085]: pgmap v146: 129 pgs: 129 active+clean; 2.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 22 op/s 2024-04-21T08:43:42.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:42 smithi043 ceph-mon[27978]: pgmap v146: 129 pgs: 129 active+clean; 2.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 22 op/s 2024-04-21T08:43:42.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:42 smithi175 ceph-mon[31920]: pgmap v146: 129 pgs: 129 active+clean; 2.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 22 op/s 2024-04-21T08:43:44.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:44 smithi112 ceph-mon[31085]: pgmap v147: 129 pgs: 129 active+clean; 2.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 15 op/s 2024-04-21T08:43:44.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:44 smithi043 ceph-mon[27978]: pgmap v147: 129 pgs: 129 active+clean; 2.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 15 op/s 2024-04-21T08:43:44.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:44 smithi175 ceph-mon[31920]: pgmap v147: 129 pgs: 129 active+clean; 2.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 15 op/s 2024-04-21T08:43:46.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:46 smithi112 ceph-mon[31085]: pgmap v148: 129 pgs: 129 active+clean; 2.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 15 op/s 2024-04-21T08:43:46.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:46 smithi043 ceph-mon[27978]: pgmap v148: 129 pgs: 129 active+clean; 2.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 15 op/s 2024-04-21T08:43:46.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:46 smithi175 ceph-mon[31920]: pgmap v148: 129 pgs: 129 active+clean; 2.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 15 op/s 2024-04-21T08:43:48.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:48 smithi112 ceph-mon[31085]: pgmap v149: 129 pgs: 129 active+clean; 2.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 48 KiB/s wr, 22 op/s 2024-04-21T08:43:48.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:48 smithi043 ceph-mon[27978]: pgmap v149: 129 pgs: 129 active+clean; 2.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 48 KiB/s wr, 22 op/s 2024-04-21T08:43:48.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:48 smithi175 ceph-mon[31920]: pgmap v149: 129 pgs: 129 active+clean; 2.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 48 KiB/s wr, 22 op/s 2024-04-21T08:43:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:50 smithi112 ceph-mon[31085]: pgmap v150: 129 pgs: 129 active+clean; 2.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-21T08:43:50.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:50 smithi043 ceph-mon[27978]: pgmap v150: 129 pgs: 129 active+clean; 2.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-21T08:43:50.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:50 smithi175 ceph-mon[31920]: pgmap v150: 129 pgs: 129 active+clean; 2.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-21T08:43:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.275+0000 7f81c8806640 1 -- 172.21.15.43:0/833811358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81c4101a90 msgr2=0x7f81c4105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.275+0000 7f81c8806640 1 --2- 172.21.15.43:0/833811358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81c4101a90 0x7f81c4105ad0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f81ac009900 tx=0x7f81ac02bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.275+0000 7f81c8806640 1 -- 172.21.15.43:0/833811358 shutdown_connections 2024-04-21T08:43:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.275+0000 7f81c8806640 1 --2- 172.21.15.43:0/833811358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81c41061b0 0x7f81c410dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.275+0000 7f81c8806640 1 --2- 172.21.15.43:0/833811358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81c4101a90 0x7f81c4105ad0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.275+0000 7f81c8806640 1 --2- 172.21.15.43:0/833811358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81c41010e0 0x7f81c41014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.275+0000 7f81c8806640 1 -- 172.21.15.43:0/833811358 >> 172.21.15.43:0/833811358 conn(0x7f81c40771d0 msgr2=0x7f81c40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.275+0000 7f81c8806640 1 -- 172.21.15.43:0/833811358 shutdown_connections 2024-04-21T08:43:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.275+0000 7f81c8806640 1 -- 172.21.15.43:0/833811358 wait complete. 2024-04-21T08:43:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.276+0000 7f81c8806640 1 Processor -- start 2024-04-21T08:43:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.276+0000 7f81c8806640 1 -- start start 2024-04-21T08:43:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.276+0000 7f81c8806640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81c41010e0 0x7f81c41994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.276+0000 7f81c8806640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81c41061b0 0x7f81c41999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.276+0000 7f81c8806640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81c4199f80 0x7f81c41a4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.276+0000 7f81c8806640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f81c41110f0 con 0x7f81c4199f80 2024-04-21T08:43:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.276+0000 7f81c8806640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f81c4110f70 con 0x7f81c41010e0 2024-04-21T08:43:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.276+0000 7f81c8806640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81c4111270 con 0x7f81c41061b0 2024-04-21T08:43:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.277+0000 7f81c2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81c4199f80 0x7f81c41a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.277+0000 7f81c2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81c4199f80 0x7f81c41a4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36916/0 (socket says 172.21.15.43:36916) 2024-04-21T08:43:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.277+0000 7f81c2d9d640 1 -- 172.21.15.43:0/2267448269 learned_addr learned my addr 172.21.15.43:0/2267448269 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.277+0000 7f81b9d9b640 1 --2- 172.21.15.43:0/2267448269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81c41061b0 0x7f81c41999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.277+0000 7f81c259c640 1 --2- 172.21.15.43:0/2267448269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81c41010e0 0x7f81c41994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.277+0000 7f81c2d9d640 1 -- 172.21.15.43:0/2267448269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81c41061b0 msgr2=0x7f81c41999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.277+0000 7f81c2d9d640 1 --2- 172.21.15.43:0/2267448269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81c41061b0 0x7f81c41999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.277+0000 7f81c2d9d640 1 -- 172.21.15.43:0/2267448269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81c41010e0 msgr2=0x7f81c41994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.277+0000 7f81c2d9d640 1 --2- 172.21.15.43:0/2267448269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81c41010e0 0x7f81c41994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.277+0000 7f81c2d9d640 1 -- 172.21.15.43:0/2267448269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c41a4d00 con 0x7f81c4199f80 2024-04-21T08:43:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.277+0000 7f81c2d9d640 1 --2- 172.21.15.43:0/2267448269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81c4199f80 0x7f81c41a4530 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00eaa0 tx=0x7f81bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.277+0000 7f81bbfff640 1 -- 172.21.15.43:0/2267448269 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc00cdb0 con 0x7f81c4199f80 2024-04-21T08:43:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.278+0000 7f81bbfff640 1 -- 172.21.15.43:0/2267448269 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81bc005020 con 0x7f81c4199f80 2024-04-21T08:43:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.278+0000 7f81c8806640 1 -- 172.21.15.43:0/2267448269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c41a4ff0 con 0x7f81c4199f80 2024-04-21T08:43:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.278+0000 7f81bbfff640 1 -- 172.21.15.43:0/2267448269 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc0116d0 con 0x7f81c4199f80 2024-04-21T08:43:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.278+0000 7f81c8806640 1 -- 172.21.15.43:0/2267448269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c41a54c0 con 0x7f81c4199f80 2024-04-21T08:43:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.280+0000 7f81bbfff640 1 -- 172.21.15.43:0/2267448269 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f81bc0051d0 con 0x7f81c4199f80 2024-04-21T08:43:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.280+0000 7f81cb3f9740 1 Processor -- start 2024-04-21T08:43:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.280+0000 7f81cb3f9740 1 -- start start 2024-04-21T08:43:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.281+0000 7f81cb3f9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561c3f03d8d0 0x561c3f03c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.281+0000 7f81cb3f9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c3f03cb60 0x561c3f03cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.281+0000 7f81cb3f9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f04a4f0 0x561c3f04c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.281+0000 7f81cb3f9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561c3efa18d0 con 0x561c3f04a4f0 2024-04-21T08:43:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.281+0000 7f81cb3f9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561c3ef3ee80 con 0x561c3f03cb60 2024-04-21T08:43:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.281+0000 7f81cb3f9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561c3ef9d890 con 0x561c3f03d8d0 2024-04-21T08:43:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.281+0000 7f81bbfff640 1 --2- 172.21.15.43:0/2267448269 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f81b00a3dc0 0x7f81b00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.281+0000 7f81b9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f04a4f0 0x561c3f04c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.281+0000 7f81b9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f04a4f0 0x561c3f04c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36924/0 (socket says 172.21.15.43:36924) 2024-04-21T08:43:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.281+0000 7f81b9d9b640 1 -- 172.21.15.43:0/3347037805 learned_addr learned my addr 172.21.15.43:0/3347037805 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.281+0000 7f81c2d9d640 1 --2- 172.21.15.43:0/2267448269 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f81b00a3dc0 0x7f81b00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.281+0000 7f81b9d9b640 1 --2- 172.21.15.43:0/3347037805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561c3f03d8d0 0x561c3f03c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.281+0000 7f81c259c640 1 --2- 172.21.15.43:0/3347037805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c3f03cb60 0x561c3f03cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.281+0000 7f81bbfff640 1 -- 172.21.15.43:0/2267448269 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f81bc0d0d80 con 0x7f81c4199f80 2024-04-21T08:43:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.281+0000 7f81b9d9b640 1 -- 172.21.15.43:0/3347037805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561c3f03d8d0 msgr2=0x561c3f03c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.281+0000 7f81b9d9b640 1 --2- 172.21.15.43:0/3347037805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561c3f03d8d0 0x561c3f03c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.281+0000 7f81b9d9b640 1 -- 172.21.15.43:0/3347037805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c3f03cb60 msgr2=0x561c3f03cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.281+0000 7f81b9d9b640 1 --2- 172.21.15.43:0/3347037805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c3f03cb60 0x561c3f03cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.281+0000 7f81b9d9b640 1 -- 172.21.15.43:0/3347037805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c3ef60320 con 0x561c3f04a4f0 2024-04-21T08:43:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.281+0000 7f81c2d9d640 1 --2- 172.21.15.43:0/2267448269 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f81b00a3dc0 0x7f81b00a6280 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f81bc09c5f0 tx=0x7f81bc081040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.281+0000 7f81b9d9b640 1 --2- 172.21.15.43:0/3347037805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f04a4f0 0x561c3f04c8e0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f81c406a570 tx=0x7f81ac005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9efb0cd46c5c45a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.282+0000 7f81b8d99640 1 -- 172.21.15.43:0/3347037805 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac04a070 con 0x561c3f04a4f0 2024-04-21T08:43:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.282+0000 7f81b8d99640 1 -- 172.21.15.43:0/3347037805 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81ac02cc90 con 0x561c3f04a4f0 2024-04-21T08:43:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.282+0000 7f81b8d99640 1 -- 172.21.15.43:0/3347037805 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac0048c0 con 0x561c3f04a4f0 2024-04-21T08:43:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.282+0000 7f81cb3f9740 1 -- 172.21.15.43:0/3347037805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f04a4f0 msgr2=0x561c3f04c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.282+0000 7f81cb3f9740 1 --2- 172.21.15.43:0/3347037805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f04a4f0 0x561c3f04c8e0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f81c406a570 tx=0x7f81ac005e80 comp rx=0 tx=0).stop 2024-04-21T08:43:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.282+0000 7f81cb3f9740 1 -- 172.21.15.43:0/3347037805 shutdown_connections 2024-04-21T08:43:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.282+0000 7f81cb3f9740 1 --2- 172.21.15.43:0/3347037805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f04a4f0 0x561c3f04c8e0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.282+0000 7f81cb3f9740 1 --2- 172.21.15.43:0/3347037805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c3f03cb60 0x561c3f03cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.282+0000 7f81cb3f9740 1 --2- 172.21.15.43:0/3347037805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561c3f03d8d0 0x561c3f03c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.282+0000 7f81cb3f9740 1 -- 172.21.15.43:0/3347037805 >> 172.21.15.43:0/3347037805 conn(0x561c3f003c90 msgr2=0x561c3f037850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.283+0000 7f81cb3f9740 1 -- 172.21.15.43:0/3347037805 shutdown_connections 2024-04-21T08:43:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.283+0000 7f81cb3f9740 1 -- 172.21.15.43:0/3347037805 wait complete. 2024-04-21T08:43:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.283+0000 7f81cb3f9740 1 -- start start 2024-04-21T08:43:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.283+0000 7f81cb3f9740 1 Processor -- start 2024-04-21T08:43:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.283+0000 7f81cb3f9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f03cb60 0x561c3f049f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.283+0000 7f81cb3f9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c3f03d8d0 0x561c3f0472c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.283+0000 7f81cb3f9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561c3f04a4f0 0x561c3f047a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.283+0000 7f81cb3f9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561c3f04d1c0 con 0x561c3f03cb60 2024-04-21T08:43:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.283+0000 7f81cb3f9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561c3ef29eb0 con 0x561c3f03d8d0 2024-04-21T08:43:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.283+0000 7f81cb3f9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561c3f04d340 con 0x561c3f04a4f0 2024-04-21T08:43:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.283+0000 7f81cb3f9740 10 client.0 _ll_register_callbacks cb 0x561c3ef56b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.283+0000 7f81b9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f03cb60 0x561c3f049f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.283+0000 7f81b9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f03cb60 0x561c3f049f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36934/0 (socket says 172.21.15.43:36934) 2024-04-21T08:43:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.283+0000 7f81b9d9b640 1 -- 172.21.15.43:0/2149930103 learned_addr learned my addr 172.21.15.43:0/2149930103 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.284+0000 7f81c259c640 1 --2- 172.21.15.43:0/2149930103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c3f03d8d0 0x561c3f0472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.284+0000 7f81b9d9b640 1 -- 172.21.15.43:0/2149930103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561c3f04a4f0 msgr2=0x561c3f047a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.284+0000 7f81b9d9b640 1 --2- 172.21.15.43:0/2149930103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561c3f04a4f0 0x561c3f047a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.284+0000 7f81b9d9b640 1 -- 172.21.15.43:0/2149930103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c3f03d8d0 msgr2=0x561c3f0472c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.284+0000 7f81b9d9b640 1 --2- 172.21.15.43:0/2149930103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c3f03d8d0 0x561c3f0472c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.284+0000 7f81b9d9b640 1 -- 172.21.15.43:0/2149930103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c3ecb4210 con 0x561c3f03cb60 2024-04-21T08:43:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.284+0000 7f81b9d9b640 1 --2- 172.21.15.43:0/2149930103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f03cb60 0x561c3f049f30 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f81ac02bc70 tx=0x7f81ac003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdfd5b3290ea45d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.284+0000 7f81a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.284+0000 7f81a9ffb640 1 -- 172.21.15.43:0/2149930103 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac055060 con 0x561c3f03cb60 2024-04-21T08:43:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.284+0000 7f81a9ffb640 1 -- 172.21.15.43:0/2149930103 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f81ac003b90 con 0x561c3f03cb60 2024-04-21T08:43:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.284+0000 7f81cb3f9740 1 -- 172.21.15.43:0/2149930103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81ac02cc90 con 0x561c3f03cb60 2024-04-21T08:43:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.284+0000 7f81a9ffb640 1 -- 172.21.15.43:0/2149930103 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f81ac05c050 con 0x561c3f03cb60 2024-04-21T08:43:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.284+0000 7f81cb3f9740 10 client.17661 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.284+0000 7f81cb3f9740 1 -- 172.21.15.43:0/2149930103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561c3ed80620 con 0x561c3f03cb60 2024-04-21T08:43:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.285+0000 7f81a9ffb640 1 -- 172.21.15.43:0/2149930103 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81ac02cc90 con 0x561c3f03cb60 2024-04-21T08:43:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.285+0000 7f81a9ffb640 1 -- 172.21.15.43:0/2149930103 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f81ac043900 con 0x561c3f03cb60 2024-04-21T08:43:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.285+0000 7f81cb3f9740 10 client.17661 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.285+0000 7f81cb3f9740 10 client.17661 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.285+0000 7f81cb3f9740 20 client.17661 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.285+0000 7f81cb3f9740 1 --2- 172.21.15.43:0/2149930103 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561c3f05a3b0 0x561c3f05c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.285+0000 7f81cb3f9740 4 client.17661 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.285+0000 7f81cb3f9740 1 -- 172.21.15.43:0/2149930103 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c3efb5880 con 0x561c3f05a3b0 2024-04-21T08:43:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.285+0000 7f81c259c640 1 --2- 172.21.15.43:0/2149930103 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561c3f05a3b0 0x561c3f05c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.286+0000 7f81c259c640 1 --2- 172.21.15.43:0/2149930103 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561c3f05a3b0 0x561c3f05c7a0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f96c900ea7357ff server_cookie=b23ba78b8bfd94d6 in_seq=0 out_seq=0 2024-04-21T08:43:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.286+0000 7f81a9ffb640 10 client.17661 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.287+0000 7f81a9ffb640 1 -- 172.21.15.43:0/2149930103 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561c3efb5880 con 0x561c3f05a3b0 2024-04-21T08:43:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.287+0000 7f81a9ffb640 10 client.17661 handle_command_reply: tid=0 2024-04-21T08:43:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.287+0000 7f81cb3f9740 1 client.17661 shutdown 2024-04-21T08:43:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.287+0000 7f81cb3f9740 1 -- 172.21.15.43:0/2149930103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f03cb60 msgr2=0x561c3f049f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.287+0000 7f81cb3f9740 1 --2- 172.21.15.43:0/2149930103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f03cb60 0x561c3f049f30 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f81ac02bc70 tx=0x7f81ac003820 comp rx=0 tx=0).stop 2024-04-21T08:43:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.287+0000 7f81cb3f9740 1 -- 172.21.15.43:0/2149930103 shutdown_connections 2024-04-21T08:43:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.287+0000 7f81cb3f9740 1 --2- 172.21.15.43:0/2149930103 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561c3f05a3b0 0x561c3f05c7a0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.287+0000 7f81cb3f9740 1 --2- 172.21.15.43:0/2149930103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561c3f04a4f0 0x561c3f047a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.287+0000 7f81cb3f9740 1 --2- 172.21.15.43:0/2149930103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c3f03d8d0 0x561c3f0472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.287+0000 7f81a9ffb640 0 client.17661 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.287+0000 7f81cb3f9740 1 --2- 172.21.15.43:0/2149930103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f03cb60 0x561c3f049f30 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.287+0000 7f81cb3f9740 1 -- 172.21.15.43:0/2149930103 >> 172.21.15.43:0/2149930103 conn(0x561c3f003c90 msgr2=0x561c3f0395a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.287+0000 7f81cb3f9740 1 -- 172.21.15.43:0/2149930103 shutdown_connections 2024-04-21T08:43:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.287+0000 7f81cb3f9740 1 -- 172.21.15.43:0/2149930103 wait complete. 2024-04-21T08:43:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.287+0000 7f81cb3f9740 20 client.17661 trim_cache size 0 max 16384 2024-04-21T08:43:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.302+0000 7f81cb3f9740 1 Processor -- start 2024-04-21T08:43:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.302+0000 7f81cb3f9740 1 -- start start 2024-04-21T08:43:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.302+0000 7f81cb3f9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561c3f03cb60 0x561c3f0499f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.302+0000 7f81cb3f9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f03d8d0 0x561c3f049f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.302+0000 7f81cb3f9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c3f04a4f0 0x561c3f04b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.302+0000 7f81cb3f9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f81c41ad1b0 con 0x561c3f03d8d0 2024-04-21T08:43:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.302+0000 7f81cb3f9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f81c41aecf0 con 0x561c3f04a4f0 2024-04-21T08:43:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.302+0000 7f81cb3f9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81c41ad880 con 0x561c3f03cb60 2024-04-21T08:43:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.303+0000 7f81c259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f03d8d0 0x561c3f049f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.303+0000 7f81c259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f03d8d0 0x561c3f049f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36936/0 (socket says 172.21.15.43:36936) 2024-04-21T08:43:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.303+0000 7f81c259c640 1 -- 172.21.15.43:0/1375429015 learned_addr learned my addr 172.21.15.43:0/1375429015 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.303+0000 7f81b9d9b640 1 --2- 172.21.15.43:0/1375429015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561c3f03cb60 0x561c3f0499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.303+0000 7f81b9d9b640 1 --2- 172.21.15.43:0/1375429015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c3f04a4f0 0x561c3f04b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.303+0000 7f81c259c640 1 -- 172.21.15.43:0/1375429015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561c3f03cb60 msgr2=0x561c3f0499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.303+0000 7f81c259c640 1 --2- 172.21.15.43:0/1375429015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561c3f03cb60 0x561c3f0499f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.303+0000 7f81c259c640 1 -- 172.21.15.43:0/1375429015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c3f04a4f0 msgr2=0x561c3f04b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.303+0000 7f81c259c640 1 --2- 172.21.15.43:0/1375429015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c3f04a4f0 0x561c3f04b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.303+0000 7f81c259c640 1 -- 172.21.15.43:0/1375429015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c3f03be30 con 0x561c3f03d8d0 2024-04-21T08:43:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.303+0000 7f81b9d9b640 1 --2- 172.21.15.43:0/1375429015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c3f04a4f0 0x561c3f04b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.303+0000 7f81c259c640 1 --2- 172.21.15.43:0/1375429015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f03d8d0 0x561c3f049f30 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f81b4000c00 tx=0x7f81b4004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457dfb7f0e1dfd97 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.303+0000 7f81abfff640 1 -- 172.21.15.43:0/1375429015 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4003d60 con 0x561c3f03d8d0 2024-04-21T08:43:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.303+0000 7f81abfff640 1 -- 172.21.15.43:0/1375429015 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81b4022070 con 0x561c3f03d8d0 2024-04-21T08:43:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.303+0000 7f81abfff640 1 -- 172.21.15.43:0/1375429015 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4011c30 con 0x561c3f03d8d0 2024-04-21T08:43:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.304+0000 7f81cb3f9740 1 -- 172.21.15.43:0/1375429015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f03d8d0 msgr2=0x561c3f049f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.304+0000 7f81cb3f9740 1 --2- 172.21.15.43:0/1375429015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f03d8d0 0x561c3f049f30 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f81b4000c00 tx=0x7f81b4004410 comp rx=0 tx=0).stop 2024-04-21T08:43:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.304+0000 7f81cb3f9740 1 -- 172.21.15.43:0/1375429015 shutdown_connections 2024-04-21T08:43:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.304+0000 7f81cb3f9740 1 --2- 172.21.15.43:0/1375429015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c3f04a4f0 0x561c3f04b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.304+0000 7f81cb3f9740 1 --2- 172.21.15.43:0/1375429015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f03d8d0 0x561c3f049f30 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.304+0000 7f81cb3f9740 1 --2- 172.21.15.43:0/1375429015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561c3f03cb60 0x561c3f0499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.304+0000 7f81cb3f9740 1 -- 172.21.15.43:0/1375429015 >> 172.21.15.43:0/1375429015 conn(0x561c3f003c90 msgr2=0x561c3f042490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.304+0000 7f81cb3f9740 1 -- 172.21.15.43:0/1375429015 shutdown_connections 2024-04-21T08:43:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.304+0000 7f81cb3f9740 1 -- 172.21.15.43:0/1375429015 wait complete. 2024-04-21T08:43:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.304+0000 7f81cb3f9740 1 -- start start 2024-04-21T08:43:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.305+0000 7f81cb3f9740 1 Processor -- start 2024-04-21T08:43:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.305+0000 7f81cb3f9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561c3f03cb60 0x561c3f05f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.305+0000 7f81cb3f9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f03d8d0 0x561c3f05f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.305+0000 7f81cb3f9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c3f04a4f0 0x561c3f05a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.305+0000 7f81cb3f9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f81c41ae620 con 0x561c3f03d8d0 2024-04-21T08:43:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.305+0000 7f81cb3f9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f81c41adf50 con 0x561c3f04a4f0 2024-04-21T08:43:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.305+0000 7f81cb3f9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81c41ab190 con 0x561c3f03cb60 2024-04-21T08:43:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.305+0000 7f81cb3f9740 10 client.0 _ll_register_callbacks cb 0x7f81ac048070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.305+0000 7f81c259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f03d8d0 0x561c3f05f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.305+0000 7f81c259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f03d8d0 0x561c3f05f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36946/0 (socket says 172.21.15.43:36946) 2024-04-21T08:43:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.305+0000 7f81c259c640 1 -- 172.21.15.43:0/3653712274 learned_addr learned my addr 172.21.15.43:0/3653712274 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.305+0000 7f81b9d9b640 1 --2- 172.21.15.43:0/3653712274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c3f04a4f0 0x561c3f05a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.305+0000 7f81b9d9b640 1 --2- 172.21.15.43:0/3653712274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561c3f03cb60 0x561c3f05f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.305+0000 7f81c259c640 1 -- 172.21.15.43:0/3653712274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561c3f03cb60 msgr2=0x561c3f05f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.305+0000 7f81c259c640 1 --2- 172.21.15.43:0/3653712274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561c3f03cb60 0x561c3f05f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.305+0000 7f81c259c640 1 -- 172.21.15.43:0/3653712274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c3f04a4f0 msgr2=0x561c3f05a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.305+0000 7f81c259c640 1 --2- 172.21.15.43:0/3653712274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c3f04a4f0 0x561c3f05a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.305+0000 7f81c259c640 1 -- 172.21.15.43:0/3653712274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c3f05b030 con 0x561c3f03d8d0 2024-04-21T08:43:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.305+0000 7f81b9d9b640 1 --2- 172.21.15.43:0/3653712274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561c3f03cb60 0x561c3f05f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.306+0000 7f81c259c640 1 --2- 172.21.15.43:0/3653712274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f03d8d0 0x561c3f05f970 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f81b400bc90 tx=0x7f81b4011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d59e7661d8e453 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.306+0000 7f81a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.306+0000 7f81a9ffb640 1 -- 172.21.15.43:0/3653712274 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4011630 con 0x561c3f03d8d0 2024-04-21T08:43:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.306+0000 7f81a9ffb640 1 -- 172.21.15.43:0/3653712274 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f81b401d420 con 0x561c3f03d8d0 2024-04-21T08:43:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.306+0000 7f81cb3f9740 1 -- 172.21.15.43:0/3653712274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81b4022070 con 0x561c3f03d8d0 2024-04-21T08:43:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.306+0000 7f81a9ffb640 1 -- 172.21.15.43:0/3653712274 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f81b401d620 con 0x561c3f03d8d0 2024-04-21T08:43:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.306+0000 7f81cb3f9740 10 client.17667 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.306+0000 7f81cb3f9740 1 -- 172.21.15.43:0/3653712274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561c3f0bff00 con 0x561c3f03d8d0 2024-04-21T08:43:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.306+0000 7f81a9ffb640 1 -- 172.21.15.43:0/3653712274 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81b4022070 con 0x561c3f03d8d0 2024-04-21T08:43:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.306+0000 7f81a9ffb640 1 -- 172.21.15.43:0/3653712274 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f81b4032770 con 0x561c3f03d8d0 2024-04-21T08:43:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.306+0000 7f81cb3f9740 10 client.17667 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.306+0000 7f81cb3f9740 10 client.17667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.306+0000 7f81cb3f9740 20 client.17667 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.306+0000 7f81cb3f9740 1 --2- 172.21.15.43:0/3653712274 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561c3f0c0620 0x561c3f0c2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.306+0000 7f81cb3f9740 4 client.17667 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:43:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.306+0000 7f81cb3f9740 1 -- 172.21.15.43:0/3653712274 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561c3ef413d0 con 0x561c3f0c0620 2024-04-21T08:43:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.307+0000 7f81b9d9b640 1 --2- 172.21.15.43:0/3653712274 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561c3f0c0620 0x561c3f0c2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.307+0000 7f81a9ffb640 10 client.17667 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.307+0000 7f81b9d9b640 1 --2- 172.21.15.43:0/3653712274 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561c3f0c0620 0x561c3f0c2a40 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e7fb18568c26b17 server_cookie=c76672419017e875 in_seq=0 out_seq=0 2024-04-21T08:43:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.322+0000 7f81a9ffb640 1 -- 172.21.15.43:0/3653712274 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561c3ef413d0 con 0x561c3f0c0620 2024-04-21T08:43:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.322+0000 7f81a9ffb640 10 client.17667 handle_command_reply: tid=0 2024-04-21T08:43:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.322+0000 7f81cb3f9740 1 client.17667 shutdown 2024-04-21T08:43:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.322+0000 7f81cb3f9740 1 -- 172.21.15.43:0/3653712274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f03d8d0 msgr2=0x561c3f05f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.322+0000 7f81cb3f9740 1 --2- 172.21.15.43:0/3653712274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f03d8d0 0x561c3f05f970 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f81b400bc90 tx=0x7f81b4011440 comp rx=0 tx=0).stop 2024-04-21T08:43:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.322+0000 7f81cb3f9740 1 -- 172.21.15.43:0/3653712274 shutdown_connections 2024-04-21T08:43:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.322+0000 7f81cb3f9740 1 --2- 172.21.15.43:0/3653712274 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561c3f0c0620 0x561c3f0c2a40 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.322+0000 7f81cb3f9740 1 --2- 172.21.15.43:0/3653712274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c3f04a4f0 0x561c3f05a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.322+0000 7f81a9ffb640 0 client.17667 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.322+0000 7f81cb3f9740 1 --2- 172.21.15.43:0/3653712274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c3f03d8d0 0x561c3f05f970 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.322+0000 7f81cb3f9740 1 --2- 172.21.15.43:0/3653712274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561c3f03cb60 0x561c3f05f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.322+0000 7f81cb3f9740 1 -- 172.21.15.43:0/3653712274 >> 172.21.15.43:0/3653712274 conn(0x561c3f003c90 msgr2=0x561c3f045b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.322+0000 7f81cb3f9740 1 -- 172.21.15.43:0/3653712274 shutdown_connections 2024-04-21T08:43:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.322+0000 7f81cb3f9740 1 -- 172.21.15.43:0/3653712274 wait complete. 2024-04-21T08:43:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.322+0000 7f81cb3f9740 20 client.17667 trim_cache size 0 max 16384 2024-04-21T08:43:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:43:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:43:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:43:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:43:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.323+0000 7f81c8806640 1 -- 172.21.15.43:0/2267448269 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f81b00a3dc0 msgr2=0x7f81b00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.323+0000 7f81c8806640 1 --2- 172.21.15.43:0/2267448269 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f81b00a3dc0 0x7f81b00a6280 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f81bc09c5f0 tx=0x7f81bc081040 comp rx=0 tx=0).stop 2024-04-21T08:43:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.323+0000 7f81c8806640 1 -- 172.21.15.43:0/2267448269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81c4199f80 msgr2=0x7f81c41a4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.323+0000 7f81c8806640 1 --2- 172.21.15.43:0/2267448269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81c4199f80 0x7f81c41a4530 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00eaa0 tx=0x7f81bc00ef60 comp rx=0 tx=0).stop 2024-04-21T08:43:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.323+0000 7f81c8806640 1 -- 172.21.15.43:0/2267448269 shutdown_connections 2024-04-21T08:43:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.323+0000 7f81c8806640 1 --2- 172.21.15.43:0/2267448269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81c4199f80 0x7f81c41a4530 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.323+0000 7f81c8806640 1 --2- 172.21.15.43:0/2267448269 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f81b00a3dc0 0x7f81b00a6280 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.323+0000 7f81c8806640 1 --2- 172.21.15.43:0/2267448269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81c41061b0 0x7f81c41999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.323+0000 7f81c8806640 1 --2- 172.21.15.43:0/2267448269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81c41010e0 0x7f81c41994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.323+0000 7f81c8806640 1 -- 172.21.15.43:0/2267448269 >> 172.21.15.43:0/2267448269 conn(0x7f81c40771d0 msgr2=0x7f81c40fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.323+0000 7f81c8806640 1 -- 172.21.15.43:0/2267448269 shutdown_connections 2024-04-21T08:43:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:43:51.323+0000 7f81c8806640 1 -- 172.21.15.43:0/2267448269 wait complete. 2024-04-21T08:43:51.600 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:43:51.600 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:43:51.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.709+0000 7fe0caaeb640 1 -- 172.21.15.43:0/2323188452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c41010a0 msgr2=0x7fe0c4101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:51.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.709+0000 7fe0caaeb640 1 --2- 172.21.15.43:0/2323188452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c41010a0 0x7fe0c4101500 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0009900 tx=0x7fe0b002bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:51.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.709+0000 7fe0caaeb640 1 -- 172.21.15.43:0/2323188452 shutdown_connections 2024-04-21T08:43:51.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.709+0000 7fe0caaeb640 1 --2- 172.21.15.43:0/2323188452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0c4101a40 0x7fe0c410dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.709+0000 7fe0caaeb640 1 --2- 172.21.15.43:0/2323188452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c41010a0 0x7fe0c4101500 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.709+0000 7fe0caaeb640 1 --2- 172.21.15.43:0/2323188452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0c4105dd0 0x7fe0c41061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.709+0000 7fe0caaeb640 1 -- 172.21.15.43:0/2323188452 >> 172.21.15.43:0/2323188452 conn(0x7fe0c4077c40 msgr2=0x7fe0c4078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.709+0000 7fe0caaeb640 1 -- 172.21.15.43:0/2323188452 shutdown_connections 2024-04-21T08:43:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.709+0000 7fe0caaeb640 1 -- 172.21.15.43:0/2323188452 wait complete. 2024-04-21T08:43:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.710+0000 7fe0caaeb640 1 Processor -- start 2024-04-21T08:43:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.710+0000 7fe0caaeb640 1 -- start start 2024-04-21T08:43:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.710+0000 7fe0caaeb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0c41010a0 0x7fe0c419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.710+0000 7fe0caaeb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c4101a40 0x7fe0c419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.710+0000 7fe0caaeb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0c4105dd0 0x7fe0c41985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.710+0000 7fe0caaeb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c4111170 con 0x7fe0c4101a40 2024-04-21T08:43:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.710+0000 7fe0caaeb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c4110ff0 con 0x7fe0c41010a0 2024-04-21T08:43:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.710+0000 7fe0caaeb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41112f0 con 0x7fe0c4105dd0 2024-04-21T08:43:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.710+0000 7fe0c3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c4101a40 0x7fe0c419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.710+0000 7fe0c3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c4101a40 0x7fe0c419e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36966/0 (socket says 172.21.15.43:36966) 2024-04-21T08:43:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.710+0000 7fe0c3fff640 1 -- 172.21.15.43:0/2646175578 learned_addr learned my addr 172.21.15.43:0/2646175578 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.710+0000 7fe0c8887640 1 --2- 172.21.15.43:0/2646175578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0c41010a0 0x7fe0c419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.710+0000 7fe0c9088640 1 --2- 172.21.15.43:0/2646175578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0c4105dd0 0x7fe0c41985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.711+0000 7fe0c3fff640 1 -- 172.21.15.43:0/2646175578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0c4105dd0 msgr2=0x7fe0c41985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.711+0000 7fe0c3fff640 1 --2- 172.21.15.43:0/2646175578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0c4105dd0 0x7fe0c41985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.711+0000 7fe0c3fff640 1 -- 172.21.15.43:0/2646175578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0c41010a0 msgr2=0x7fe0c419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.711+0000 7fe0c3fff640 1 --2- 172.21.15.43:0/2646175578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0c41010a0 0x7fe0c419e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.711+0000 7fe0c3fff640 1 -- 172.21.15.43:0/2646175578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c4198d20 con 0x7fe0c4101a40 2024-04-21T08:43:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.711+0000 7fe0c3fff640 1 --2- 172.21.15.43:0/2646175578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c4101a40 0x7fe0c419e930 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0005940 tx=0x7fe0b002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.711+0000 7fe0c1ffb640 1 -- 172.21.15.43:0/2646175578 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0043020 con 0x7fe0c4101a40 2024-04-21T08:43:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.711+0000 7fe0c1ffb640 1 -- 172.21.15.43:0/2646175578 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0b002cbe0 con 0x7fe0c4101a40 2024-04-21T08:43:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.711+0000 7fe0caaeb640 1 -- 172.21.15.43:0/2646175578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c4198fb0 con 0x7fe0c4101a40 2024-04-21T08:43:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.711+0000 7fe0c1ffb640 1 -- 172.21.15.43:0/2646175578 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0003bf0 con 0x7fe0c4101a40 2024-04-21T08:43:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.711+0000 7fe0caaeb640 1 -- 172.21.15.43:0/2646175578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c41a52b0 con 0x7fe0c4101a40 2024-04-21T08:43:51.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.713+0000 7fe0c1ffb640 1 -- 172.21.15.43:0/2646175578 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe0b0046020 con 0x7fe0c4101a40 2024-04-21T08:43:51.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.714+0000 7fe0cd7b8740 1 Processor -- start 2024-04-21T08:43:51.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.714+0000 7fe0cd7b8740 1 -- start start 2024-04-21T08:43:51.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.714+0000 7fe0cd7b8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c74cd7580 0x562c74cd7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.714+0000 7fe0cd7b8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c74cd7ea0 0x562c74ce0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.714+0000 7fe0cd7b8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74ce0a80 0x562c74ce2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.714+0000 7fe0cd7b8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562c74c3c1e0 con 0x562c74ce0a80 2024-04-21T08:43:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.714+0000 7fe0cd7b8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562c74bd9390 con 0x562c74cd7ea0 2024-04-21T08:43:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.714+0000 7fe0cd7b8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562c74c381a0 con 0x562c74cd7580 2024-04-21T08:43:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.714+0000 7fe0c1ffb640 1 --2- 172.21.15.43:0/2646175578 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe0b40a3da0 0x7fe0b40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.714+0000 7fe0c3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74ce0a80 0x562c74ce2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.714+0000 7fe0c8887640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c74cd7ea0 0x562c74ce0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.714+0000 7fe0c8887640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c74cd7ea0 0x562c74ce0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:59548/0 (socket says 172.21.15.43:59548) 2024-04-21T08:43:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.714+0000 7fe0c8887640 1 -- 172.21.15.43:0/3072310828 learned_addr learned my addr 172.21.15.43:0/3072310828 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.714+0000 7fe0c9088640 1 --2- 172.21.15.43:0/3072310828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c74cd7580 0x562c74cd7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.714+0000 7fe0c9088640 1 --2- 172.21.15.43:0/2646175578 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe0b40a3da0 0x7fe0b40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.714+0000 7fe0c8887640 1 -- 172.21.15.43:0/3072310828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c74cd7580 msgr2=0x562c74cd7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.714+0000 7fe0c8887640 1 --2- 172.21.15.43:0/3072310828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c74cd7580 0x562c74cd7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.714+0000 7fe0c8887640 1 -- 172.21.15.43:0/3072310828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74ce0a80 msgr2=0x562c74ce2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.714+0000 7fe0c8887640 1 --2- 172.21.15.43:0/3072310828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74ce0a80 0x562c74ce2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.714+0000 7fe0c1ffb640 1 -- 172.21.15.43:0/2646175578 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fe0b00f2d00 con 0x7fe0c4101a40 2024-04-21T08:43:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.714+0000 7fe0c8887640 1 -- 172.21.15.43:0/3072310828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c74a187d0 con 0x562c74cd7ea0 2024-04-21T08:43:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.715+0000 7fe0c8887640 1 --2- 172.21.15.43:0/3072310828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c74cd7ea0 0x562c74ce0540 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe0b800b760 tx=0x7fe0b800bc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d630b2eb55a5a6c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.715+0000 7fe0c9088640 1 --2- 172.21.15.43:0/2646175578 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe0b40a3da0 0x7fe0b40a6260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc005c40 tx=0x7fe0bc005bd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.715+0000 7fe0aeffd640 1 -- 172.21.15.43:0/3072310828 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8003dd0 con 0x562c74cd7ea0 2024-04-21T08:43:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.715+0000 7fe0aeffd640 1 -- 172.21.15.43:0/3072310828 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0b8015070 con 0x562c74cd7ea0 2024-04-21T08:43:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.715+0000 7fe0aeffd640 1 -- 172.21.15.43:0/3072310828 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8004d90 con 0x562c74cd7ea0 2024-04-21T08:43:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.715+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/3072310828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c74cd7ea0 msgr2=0x562c74ce0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.715+0000 7fe0cd7b8740 1 --2- 172.21.15.43:0/3072310828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c74cd7ea0 0x562c74ce0540 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe0b800b760 tx=0x7fe0b800bc20 comp rx=0 tx=0).stop 2024-04-21T08:43:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.716+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/3072310828 shutdown_connections 2024-04-21T08:43:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.716+0000 7fe0cd7b8740 1 --2- 172.21.15.43:0/3072310828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74ce0a80 0x562c74ce2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.716+0000 7fe0cd7b8740 1 --2- 172.21.15.43:0/3072310828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c74cd7ea0 0x562c74ce0540 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.716+0000 7fe0cd7b8740 1 --2- 172.21.15.43:0/3072310828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c74cd7580 0x562c74cd7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.716+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/3072310828 >> 172.21.15.43:0/3072310828 conn(0x562c74cd1cb0 msgr2=0x562c74cd2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:51.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.716+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/3072310828 shutdown_connections 2024-04-21T08:43:51.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.716+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/3072310828 wait complete. 2024-04-21T08:43:51.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.716+0000 7fe0cd7b8740 1 -- start start 2024-04-21T08:43:51.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.716+0000 7fe0cd7b8740 1 Processor -- start 2024-04-21T08:43:51.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.716+0000 7fe0cd7b8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74cd7580 0x562c74cddaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.716+0000 7fe0cd7b8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c74cd7ea0 0x562c74cddfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.716+0000 7fe0cd7b8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c74ce0a80 0x562c74cde520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.716+0000 7fe0cd7b8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562c74ce3870 con 0x562c74cd7580 2024-04-21T08:43:51.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.716+0000 7fe0cd7b8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562c74bf52f0 con 0x562c74cd7ea0 2024-04-21T08:43:51.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.716+0000 7fe0cd7b8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562c74ce39f0 con 0x562c74ce0a80 2024-04-21T08:43:51.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.716+0000 7fe0cd7b8740 10 client.0 _ll_register_callbacks cb 0x562c74c02580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:51.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.716+0000 7fe0c8887640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74cd7580 0x562c74cddaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.716+0000 7fe0c8887640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74cd7580 0x562c74cddaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36976/0 (socket says 172.21.15.43:36976) 2024-04-21T08:43:51.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.716+0000 7fe0c8887640 1 -- 172.21.15.43:0/2162441841 learned_addr learned my addr 172.21.15.43:0/2162441841 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:51.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.717+0000 7fe0c3fff640 1 --2- 172.21.15.43:0/2162441841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c74cd7ea0 0x562c74cddfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.717+0000 7fe0c9088640 1 --2- 172.21.15.43:0/2162441841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c74ce0a80 0x562c74cde520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.717+0000 7fe0c8887640 1 -- 172.21.15.43:0/2162441841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c74ce0a80 msgr2=0x562c74cde520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:51.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.717+0000 7fe0c8887640 1 --2- 172.21.15.43:0/2162441841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c74ce0a80 0x562c74cde520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.717+0000 7fe0c8887640 1 -- 172.21.15.43:0/2162441841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c74cd7ea0 msgr2=0x562c74cddfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:51.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.717+0000 7fe0c8887640 1 --2- 172.21.15.43:0/2162441841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c74cd7ea0 0x562c74cddfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.717+0000 7fe0c8887640 1 -- 172.21.15.43:0/2162441841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c7494e3f0 con 0x562c74cd7580 2024-04-21T08:43:51.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.717+0000 7fe0c9088640 1 --2- 172.21.15.43:0/2162441841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c74ce0a80 0x562c74cde520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:51.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.717+0000 7fe0c8887640 1 --2- 172.21.15.43:0/2162441841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74cd7580 0x562c74cddaa0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8006010 tx=0x7fe0b8003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf5b8635c660cdf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:51.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.717+0000 7fe0a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:51.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.717+0000 7fe0a3fff640 1 -- 172.21.15.43:0/2162441841 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8028060 con 0x562c74cd7580 2024-04-21T08:43:51.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.717+0000 7fe0a3fff640 1 -- 172.21.15.43:0/2162441841 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fe0b8011850 con 0x562c74cd7580 2024-04-21T08:43:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.717+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/2162441841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0b8015070 con 0x562c74cd7580 2024-04-21T08:43:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.717+0000 7fe0a3fff640 1 -- 172.21.15.43:0/2162441841 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe0b802f050 con 0x562c74cd7580 2024-04-21T08:43:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.717+0000 7fe0cd7b8740 10 client.17679 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.717+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/2162441841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562c7490e0f0 con 0x562c74cd7580 2024-04-21T08:43:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.718+0000 7fe0a3fff640 1 -- 172.21.15.43:0/2162441841 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0b8015070 con 0x562c74cd7580 2024-04-21T08:43:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.718+0000 7fe0a3fff640 1 -- 172.21.15.43:0/2162441841 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe0b8022070 con 0x562c74cd7580 2024-04-21T08:43:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.718+0000 7fe0cd7b8740 10 client.17679 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.718+0000 7fe0cd7b8740 10 client.17679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.718+0000 7fe0cd7b8740 20 client.17679 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.718+0000 7fe0cd7b8740 1 --2- 172.21.15.43:0/2162441841 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562c74cf09f0 0x562c74cf2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.718+0000 7fe0cd7b8740 4 client.17679 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.718+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/2162441841 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c74c49c00 con 0x562c74cf09f0 2024-04-21T08:43:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.718+0000 7fe0c3fff640 1 --2- 172.21.15.43:0/2162441841 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562c74cf09f0 0x562c74cf2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.719+0000 7fe0a3fff640 10 client.17679 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.719+0000 7fe0c3fff640 1 --2- 172.21.15.43:0/2162441841 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562c74cf09f0 0x562c74cf2de0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=573c77c028c37f3a server_cookie=8170cd15a38f9908 in_seq=0 out_seq=0 2024-04-21T08:43:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.719+0000 7fe0a3fff640 1 -- 172.21.15.43:0/2162441841 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562c74c49c00 con 0x562c74cf09f0 2024-04-21T08:43:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.719+0000 7fe0a3fff640 10 client.17679 handle_command_reply: tid=0 2024-04-21T08:43:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.719+0000 7fe0cd7b8740 1 client.17679 shutdown 2024-04-21T08:43:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.720+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/2162441841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74cd7580 msgr2=0x562c74cddaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.720+0000 7fe0cd7b8740 1 --2- 172.21.15.43:0/2162441841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74cd7580 0x562c74cddaa0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8006010 tx=0x7fe0b8003e00 comp rx=0 tx=0).stop 2024-04-21T08:43:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.720+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/2162441841 shutdown_connections 2024-04-21T08:43:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.720+0000 7fe0cd7b8740 1 --2- 172.21.15.43:0/2162441841 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562c74cf09f0 0x562c74cf2de0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.720+0000 7fe0cd7b8740 1 --2- 172.21.15.43:0/2162441841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c74ce0a80 0x562c74cde520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.720+0000 7fe0cd7b8740 1 --2- 172.21.15.43:0/2162441841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c74cd7ea0 0x562c74cddfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.720+0000 7fe0a3fff640 0 client.17679 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.720+0000 7fe0cd7b8740 1 --2- 172.21.15.43:0/2162441841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74cd7580 0x562c74cddaa0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.720+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/2162441841 >> 172.21.15.43:0/2162441841 conn(0x562c74cd1cb0 msgr2=0x562c74cd3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.720+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/2162441841 shutdown_connections 2024-04-21T08:43:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.720+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/2162441841 wait complete. 2024-04-21T08:43:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.720+0000 7fe0cd7b8740 20 client.17679 trim_cache size 0 max 16384 2024-04-21T08:43:51.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.737+0000 7fe0cd7b8740 1 Processor -- start 2024-04-21T08:43:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.737+0000 7fe0cd7b8740 1 -- start start 2024-04-21T08:43:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.738+0000 7fe0cd7b8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c74cd7580 0x562c74ce01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.738+0000 7fe0cd7b8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74cd7ea0 0x562c74cec730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.738+0000 7fe0cd7b8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c74ce0a80 0x562c74cd82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.738+0000 7fe0cd7b8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41af3c0 con 0x562c74cd7ea0 2024-04-21T08:43:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.738+0000 7fe0cd7b8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41b0f00 con 0x562c74cd7580 2024-04-21T08:43:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.738+0000 7fe0cd7b8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41afa90 con 0x562c74ce0a80 2024-04-21T08:43:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.738+0000 7fe0c3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74cd7ea0 0x562c74cec730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.738+0000 7fe0c3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74cd7ea0 0x562c74cec730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36992/0 (socket says 172.21.15.43:36992) 2024-04-21T08:43:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.738+0000 7fe0c3fff640 1 -- 172.21.15.43:0/505332885 learned_addr learned my addr 172.21.15.43:0/505332885 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.738+0000 7fe0c3fff640 1 -- 172.21.15.43:0/505332885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c74ce0a80 msgr2=0x562c74cd82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.738+0000 7fe0c8887640 1 --2- 172.21.15.43:0/505332885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c74cd7580 0x562c74ce01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.738+0000 7fe0c3fff640 1 --2- 172.21.15.43:0/505332885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c74ce0a80 0x562c74cd82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.738+0000 7fe0c3fff640 1 -- 172.21.15.43:0/505332885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c74cd7580 msgr2=0x562c74ce01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:51.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.738+0000 7fe0c3fff640 1 --2- 172.21.15.43:0/505332885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c74cd7580 0x562c74ce01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.738+0000 7fe0c3fff640 1 -- 172.21.15.43:0/505332885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c74cd7cd0 con 0x562c74cd7ea0 2024-04-21T08:43:51.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.738+0000 7fe0c3fff640 1 --2- 172.21.15.43:0/505332885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74cd7ea0 0x562c74cec730 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00be680 tx=0x7fe0b00a9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce21716eb2bc084 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:51.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.738+0000 7fe0ae7fc640 1 -- 172.21.15.43:0/505332885 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b00d7070 con 0x562c74cd7ea0 2024-04-21T08:43:51.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.738+0000 7fe0ae7fc640 1 -- 172.21.15.43:0/505332885 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0b00be980 con 0x562c74cd7ea0 2024-04-21T08:43:51.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.738+0000 7fe0ae7fc640 1 -- 172.21.15.43:0/505332885 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b00054d0 con 0x562c74cd7ea0 2024-04-21T08:43:51.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.739+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/505332885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74cd7ea0 msgr2=0x562c74cec730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:51.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.739+0000 7fe0cd7b8740 1 --2- 172.21.15.43:0/505332885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74cd7ea0 0x562c74cec730 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00be680 tx=0x7fe0b00a9040 comp rx=0 tx=0).stop 2024-04-21T08:43:51.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.739+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/505332885 shutdown_connections 2024-04-21T08:43:51.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.739+0000 7fe0cd7b8740 1 --2- 172.21.15.43:0/505332885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c74ce0a80 0x562c74cd82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.739+0000 7fe0cd7b8740 1 --2- 172.21.15.43:0/505332885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74cd7ea0 0x562c74cec730 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.739+0000 7fe0cd7b8740 1 --2- 172.21.15.43:0/505332885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c74cd7580 0x562c74ce01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.739+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/505332885 >> 172.21.15.43:0/505332885 conn(0x562c74cd1cb0 msgr2=0x562c74cd3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:51.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.739+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/505332885 shutdown_connections 2024-04-21T08:43:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.739+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/505332885 wait complete. 2024-04-21T08:43:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.739+0000 7fe0cd7b8740 1 -- start start 2024-04-21T08:43:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.739+0000 7fe0cd7b8740 1 Processor -- start 2024-04-21T08:43:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.740+0000 7fe0cd7b8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74cd7580 0x562c74cda920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.740+0000 7fe0cd7b8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c74cd7ea0 0x562c74cdae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.740+0000 7fe0cd7b8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c74ce0a80 0x562c74cdb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.740+0000 7fe0cd7b8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41b0830 con 0x562c74cd7580 2024-04-21T08:43:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.740+0000 7fe0cd7b8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41b0160 con 0x562c74cd7ea0 2024-04-21T08:43:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.740+0000 7fe0cd7b8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41ad3a0 con 0x562c74ce0a80 2024-04-21T08:43:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.740+0000 7fe0cd7b8740 10 client.0 _ll_register_callbacks cb 0x7fe0b00ba580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.740+0000 7fe0c8887640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74cd7580 0x562c74cda920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.740+0000 7fe0c8887640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74cd7580 0x562c74cda920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37008/0 (socket says 172.21.15.43:37008) 2024-04-21T08:43:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.740+0000 7fe0c8887640 1 -- 172.21.15.43:0/3501612442 learned_addr learned my addr 172.21.15.43:0/3501612442 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.740+0000 7fe0c3fff640 1 --2- 172.21.15.43:0/3501612442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c74cd7ea0 0x562c74cdae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.740+0000 7fe0c9088640 1 --2- 172.21.15.43:0/3501612442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c74ce0a80 0x562c74cdb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.740+0000 7fe0c8887640 1 -- 172.21.15.43:0/3501612442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c74ce0a80 msgr2=0x562c74cdb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.740+0000 7fe0c8887640 1 --2- 172.21.15.43:0/3501612442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c74ce0a80 0x562c74cdb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.740+0000 7fe0c8887640 1 -- 172.21.15.43:0/3501612442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c74cd7ea0 msgr2=0x562c74cdae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.740+0000 7fe0c8887640 1 --2- 172.21.15.43:0/3501612442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c74cd7ea0 0x562c74cdae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.740+0000 7fe0c8887640 1 -- 172.21.15.43:0/3501612442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c74d55e00 con 0x562c74cd7580 2024-04-21T08:43:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.740+0000 7fe0c9088640 1 --2- 172.21.15.43:0/3501612442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c74ce0a80 0x562c74cdb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.740+0000 7fe0c8887640 1 --2- 172.21.15.43:0/3501612442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74cd7580 0x562c74cda920 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8019ae0 tx=0x7fe0b8005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64dac83e9bb78d43 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.740+0000 7fe0a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.740+0000 7fe0a3fff640 1 -- 172.21.15.43:0/3501612442 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8038060 con 0x562c74cd7580 2024-04-21T08:43:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.740+0000 7fe0a3fff640 1 -- 172.21.15.43:0/3501612442 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fe0b8019420 con 0x562c74cd7580 2024-04-21T08:43:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.740+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/3501612442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0b00be980 con 0x562c74cd7580 2024-04-21T08:43:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.741+0000 7fe0a3fff640 1 -- 172.21.15.43:0/3501612442 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe0b803f050 con 0x562c74cd7580 2024-04-21T08:43:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.741+0000 7fe0cd7b8740 10 client.17685 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.741+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/3501612442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562c74d57150 con 0x562c74cd7580 2024-04-21T08:43:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.741+0000 7fe0a3fff640 1 -- 172.21.15.43:0/3501612442 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0b00be980 con 0x562c74cd7580 2024-04-21T08:43:51.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.741+0000 7fe0a3fff640 1 -- 172.21.15.43:0/3501612442 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe0b8022050 con 0x562c74cd7580 2024-04-21T08:43:51.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.741+0000 7fe0cd7b8740 10 client.17685 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:51.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.741+0000 7fe0cd7b8740 10 client.17685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:51.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.741+0000 7fe0cd7b8740 20 client.17685 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:51.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.741+0000 7fe0cd7b8740 1 --2- 172.21.15.43:0/3501612442 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562c74d57900 0x562c74d59d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.741+0000 7fe0cd7b8740 4 client.17685 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:43:51.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.741+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/3501612442 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562c74c43a70 con 0x562c74d57900 2024-04-21T08:43:51.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.741+0000 7fe0c3fff640 1 --2- 172.21.15.43:0/3501612442 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562c74d57900 0x562c74d59d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.742+0000 7fe0a3fff640 10 client.17685 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:51.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.742+0000 7fe0c3fff640 1 --2- 172.21.15.43:0/3501612442 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562c74d57900 0x562c74d59d20 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7ab3301b2998e2b server_cookie=c8a4396288d95c45 in_seq=0 out_seq=0 2024-04-21T08:43:51.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.742+0000 7fe0a3fff640 1 -- 172.21.15.43:0/3501612442 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562c74c43a70 con 0x562c74d57900 2024-04-21T08:43:51.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.742+0000 7fe0a3fff640 10 client.17685 handle_command_reply: tid=0 2024-04-21T08:43:51.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.742+0000 7fe0cd7b8740 1 client.17685 shutdown 2024-04-21T08:43:51.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.742+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/3501612442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74cd7580 msgr2=0x562c74cda920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:51.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.742+0000 7fe0cd7b8740 1 --2- 172.21.15.43:0/3501612442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74cd7580 0x562c74cda920 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8019ae0 tx=0x7fe0b8005b20 comp rx=0 tx=0).stop 2024-04-21T08:43:51.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.742+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/3501612442 shutdown_connections 2024-04-21T08:43:51.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.742+0000 7fe0cd7b8740 1 --2- 172.21.15.43:0/3501612442 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562c74d57900 0x562c74d59d20 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.742+0000 7fe0cd7b8740 1 --2- 172.21.15.43:0/3501612442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c74ce0a80 0x562c74cdb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.742+0000 7fe0a3fff640 0 client.17685 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:51.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.742+0000 7fe0cd7b8740 1 --2- 172.21.15.43:0/3501612442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c74cd7ea0 0x562c74cdae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.742+0000 7fe0cd7b8740 1 --2- 172.21.15.43:0/3501612442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c74cd7580 0x562c74cda920 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.742+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/3501612442 >> 172.21.15.43:0/3501612442 conn(0x562c74cd1cb0 msgr2=0x562c74cd3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:51.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.742+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/3501612442 shutdown_connections 2024-04-21T08:43:51.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.742+0000 7fe0cd7b8740 1 -- 172.21.15.43:0/3501612442 wait complete. 2024-04-21T08:43:51.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.742+0000 7fe0cd7b8740 20 client.17685 trim_cache size 0 max 16384 2024-04-21T08:43:51.748 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:43:51.749 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:43:51.749 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:43:51.749 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:43:51.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.743+0000 7fe0caaeb640 1 -- 172.21.15.43:0/2646175578 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe0b40a3da0 msgr2=0x7fe0b40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.743+0000 7fe0caaeb640 1 --2- 172.21.15.43:0/2646175578 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe0b40a3da0 0x7fe0b40a6260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc005c40 tx=0x7fe0bc005bd0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.743+0000 7fe0caaeb640 1 -- 172.21.15.43:0/2646175578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c4101a40 msgr2=0x7fe0c419e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.743+0000 7fe0caaeb640 1 --2- 172.21.15.43:0/2646175578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c4101a40 0x7fe0c419e930 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0005940 tx=0x7fe0b002c7e0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.743+0000 7fe0caaeb640 1 -- 172.21.15.43:0/2646175578 shutdown_connections 2024-04-21T08:43:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.743+0000 7fe0caaeb640 1 --2- 172.21.15.43:0/2646175578 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe0b40a3da0 0x7fe0b40a6260 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.743+0000 7fe0caaeb640 1 --2- 172.21.15.43:0/2646175578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0c4105dd0 0x7fe0c41985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.743+0000 7fe0caaeb640 1 --2- 172.21.15.43:0/2646175578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c4101a40 0x7fe0c419e930 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.743+0000 7fe0caaeb640 1 --2- 172.21.15.43:0/2646175578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0c41010a0 0x7fe0c419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.743+0000 7fe0caaeb640 1 -- 172.21.15.43:0/2646175578 >> 172.21.15.43:0/2646175578 conn(0x7fe0c4077c40 msgr2=0x7fe0c406a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.743+0000 7fe0caaeb640 1 -- 172.21.15.43:0/2646175578 shutdown_connections 2024-04-21T08:43:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.743+0000 7fe0caaeb640 1 -- 172.21.15.43:0/2646175578 wait complete. 2024-04-21T08:43:51.754 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:43:51.754 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:43:51.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.855+0000 7f4ab4585640 1 -- 172.21.15.43:0/2429614143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4aac1010b0 msgr2=0x7f4aac101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:51.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.855+0000 7f4ab4585640 1 --2- 172.21.15.43:0/2429614143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4aac1010b0 0x7f4aac101490 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8009900 tx=0x7f4aa802bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:51.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.855+0000 7f4ab4585640 1 -- 172.21.15.43:0/2429614143 shutdown_connections 2024-04-21T08:43:51.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.855+0000 7f4ab4585640 1 --2- 172.21.15.43:0/2429614143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4aac106180 0x7f4aac10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.855+0000 7f4ab4585640 1 --2- 172.21.15.43:0/2429614143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4aac101a60 0x7f4aac105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.855+0000 7f4ab4585640 1 --2- 172.21.15.43:0/2429614143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4aac1010b0 0x7f4aac101490 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.855+0000 7f4ab4585640 1 -- 172.21.15.43:0/2429614143 >> 172.21.15.43:0/2429614143 conn(0x7f4aac0771d0 msgr2=0x7f4aac0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:51.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.855+0000 7f4ab4585640 1 -- 172.21.15.43:0/2429614143 shutdown_connections 2024-04-21T08:43:51.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.855+0000 7f4ab4585640 1 -- 172.21.15.43:0/2429614143 wait complete. 2024-04-21T08:43:51.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.856+0000 7f4ab4585640 1 Processor -- start 2024-04-21T08:43:51.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.856+0000 7f4ab4585640 1 -- start start 2024-04-21T08:43:51.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.856+0000 7f4ab4585640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4aac1010b0 0x7f4aac198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.856+0000 7f4ab4585640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4aac106180 0x7f4aac198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.856+0000 7f4ab4585640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4aac19cff0 0x7f4aac19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.856+0000 7f4ab4585640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac1110c0 con 0x7f4aac106180 2024-04-21T08:43:51.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.856+0000 7f4ab4585640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac110f40 con 0x7f4aac1010b0 2024-04-21T08:43:51.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.856+0000 7f4ab4585640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac111240 con 0x7f4aac19cff0 2024-04-21T08:43:51.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.856+0000 7f4ab1b20640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4aac106180 0x7f4aac198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.856+0000 7f4ab1b20640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4aac106180 0x7f4aac198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37018/0 (socket says 172.21.15.43:37018) 2024-04-21T08:43:51.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.856+0000 7f4ab1b20640 1 -- 172.21.15.43:0/3908235331 learned_addr learned my addr 172.21.15.43:0/3908235331 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:51.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.856+0000 7f4ab2b22640 1 --2- 172.21.15.43:0/3908235331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4aac19cff0 0x7f4aac19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.856+0000 7f4ab2321640 1 --2- 172.21.15.43:0/3908235331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4aac1010b0 0x7f4aac198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.857+0000 7f4ab1b20640 1 -- 172.21.15.43:0/3908235331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4aac19cff0 msgr2=0x7f4aac19d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:51.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.857+0000 7f4ab1b20640 1 --2- 172.21.15.43:0/3908235331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4aac19cff0 0x7f4aac19d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.857+0000 7f4ab1b20640 1 -- 172.21.15.43:0/3908235331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4aac1010b0 msgr2=0x7f4aac198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:51.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.857+0000 7f4ab1b20640 1 --2- 172.21.15.43:0/3908235331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4aac1010b0 0x7f4aac198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:51.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.857+0000 7f4ab1b20640 1 -- 172.21.15.43:0/3908235331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aac19dae0 con 0x7f4aac106180 2024-04-21T08:43:51.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.857+0000 7f4ab2321640 1 --2- 172.21.15.43:0/3908235331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4aac1010b0 0x7f4aac198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:51.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.857+0000 7f4ab1b20640 1 --2- 172.21.15.43:0/3908235331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4aac106180 0x7f4aac198c70 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c005fd0 tx=0x7f4a9c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:51.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.857+0000 7f4aa37fe640 1 -- 172.21.15.43:0/3908235331 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c017070 con 0x7f4aac106180 2024-04-21T08:43:51.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.857+0000 7f4aa37fe640 1 -- 172.21.15.43:0/3908235331 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a9c009c30 con 0x7f4aac106180 2024-04-21T08:43:51.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.857+0000 7f4aa37fe640 1 -- 172.21.15.43:0/3908235331 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c003d30 con 0x7f4aac106180 2024-04-21T08:43:51.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.857+0000 7f4ab4585640 1 -- 172.21.15.43:0/3908235331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aac19ddd0 con 0x7f4aac106180 2024-04-21T08:43:51.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.857+0000 7f4ab4585640 1 -- 172.21.15.43:0/3908235331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aac1a55c0 con 0x7f4aac106180 2024-04-21T08:43:51.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.858+0000 7f4ab4585640 1 -- 172.21.15.43:0/3908235331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aac069230 con 0x7f4aac106180 2024-04-21T08:43:51.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.859+0000 7f4aa37fe640 1 -- 172.21.15.43:0/3908235331 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4a9c003ed0 con 0x7f4aac106180 2024-04-21T08:43:51.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.860+0000 7f4aa37fe640 1 --2- 172.21.15.43:0/3908235331 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4a980a3d20 0x7f4a980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:51.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.860+0000 7f4ab2321640 1 --2- 172.21.15.43:0/3908235331 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4a980a3d20 0x7f4a980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:51.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.860+0000 7f4aa37fe640 1 -- 172.21.15.43:0/3908235331 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f4a9c0d1110 con 0x7f4aac106180 2024-04-21T08:43:51.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.860+0000 7f4ab2321640 1 --2- 172.21.15.43:0/3908235331 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4a980a3d20 0x7f4a980a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8005ab0 tx=0x7f4aa8005a20 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:51.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:51.863+0000 7f4aa37fe640 1 -- 172.21.15.43:0/3908235331 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4a9c020030 con 0x7f4aac106180 2024-04-21T08:43:52.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.092+0000 7f4ab4585640 1 -- 172.21.15.43:0/3908235331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4aac101a60 con 0x7f4aac106180 2024-04-21T08:43:52.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.093+0000 7f4aa37fe640 1 -- 172.21.15.43:0/3908235331 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7f4a9c009de0 con 0x7f4aac106180 2024-04-21T08:43:52.093 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:43:52.094 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:43:52.094 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:43:52.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.095+0000 7f4ab4585640 1 -- 172.21.15.43:0/3908235331 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4a980a3d20 msgr2=0x7f4a980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.095+0000 7f4ab4585640 1 --2- 172.21.15.43:0/3908235331 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4a980a3d20 0x7f4a980a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8005ab0 tx=0x7f4aa8005a20 comp rx=0 tx=0).stop 2024-04-21T08:43:52.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.095+0000 7f4ab4585640 1 -- 172.21.15.43:0/3908235331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4aac106180 msgr2=0x7f4aac198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.095+0000 7f4ab4585640 1 --2- 172.21.15.43:0/3908235331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4aac106180 0x7f4aac198c70 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c005fd0 tx=0x7f4a9c00dde0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.095+0000 7f4ab4585640 1 -- 172.21.15.43:0/3908235331 shutdown_connections 2024-04-21T08:43:52.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.095+0000 7f4ab4585640 1 --2- 172.21.15.43:0/3908235331 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4a980a3d20 0x7f4a980a61e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.095+0000 7f4ab4585640 1 --2- 172.21.15.43:0/3908235331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4aac19cff0 0x7f4aac19d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.095+0000 7f4ab4585640 1 --2- 172.21.15.43:0/3908235331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4aac106180 0x7f4aac198c70 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.095+0000 7f4ab4585640 1 --2- 172.21.15.43:0/3908235331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4aac1010b0 0x7f4aac198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.095+0000 7f4ab4585640 1 -- 172.21.15.43:0/3908235331 >> 172.21.15.43:0/3908235331 conn(0x7f4aac0771d0 msgr2=0x7f4aac06a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.095+0000 7f4ab4585640 1 -- 172.21.15.43:0/3908235331 shutdown_connections 2024-04-21T08:43:52.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.095+0000 7f4ab4585640 1 -- 172.21.15.43:0/3908235331 wait complete. 2024-04-21T08:43:52.106 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:43:52.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.211+0000 7fe4887f4640 1 -- 172.21.15.43:0/2322992150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4801014a0 msgr2=0x7fe480105240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.211+0000 7fe4887f4640 1 --2- 172.21.15.43:0/2322992150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4801014a0 0x7fe480105240 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fe470009900 tx=0x7fe47002bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:52.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.211+0000 7fe4887f4640 1 -- 172.21.15.43:0/2322992150 shutdown_connections 2024-04-21T08:43:52.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.211+0000 7fe4887f4640 1 --2- 172.21.15.43:0/2322992150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480105970 0x7fe48010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.211+0000 7fe4887f4640 1 --2- 172.21.15.43:0/2322992150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4801014a0 0x7fe480105240 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.211+0000 7fe4887f4640 1 --2- 172.21.15.43:0/2322992150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe480100af0 0x7fe480100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.211+0000 7fe4887f4640 1 -- 172.21.15.43:0/2322992150 >> 172.21.15.43:0/2322992150 conn(0x7fe4800760e0 msgr2=0x7fe4800764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.211+0000 7fe4887f4640 1 -- 172.21.15.43:0/2322992150 shutdown_connections 2024-04-21T08:43:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.211+0000 7fe4887f4640 1 -- 172.21.15.43:0/2322992150 wait complete. 2024-04-21T08:43:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.211+0000 7fe4887f4640 1 Processor -- start 2024-04-21T08:43:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.212+0000 7fe4887f4640 1 -- start start 2024-04-21T08:43:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.212+0000 7fe4887f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe480100af0 0x7fe48019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.212+0000 7fe4887f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480105970 0x7fe48019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.212+0000 7fe4887f4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe480198590 0x7fe480198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.212+0000 7fe4887f4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe480112210 con 0x7fe480100af0 2024-04-21T08:43:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.212+0000 7fe4887f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe480112090 con 0x7fe480105970 2024-04-21T08:43:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.212+0000 7fe4887f4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe480112390 con 0x7fe480198590 2024-04-21T08:43:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.212+0000 7fe486590640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe480100af0 0x7fe48019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.212+0000 7fe486590640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe480100af0 0x7fe48019e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37026/0 (socket says 172.21.15.43:37026) 2024-04-21T08:43:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.212+0000 7fe486590640 1 -- 172.21.15.43:0/3582100880 learned_addr learned my addr 172.21.15.43:0/3582100880 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.212+0000 7fe485d8f640 1 --2- 172.21.15.43:0/3582100880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480105970 0x7fe48019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.212+0000 7fe486d91640 1 --2- 172.21.15.43:0/3582100880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe480198590 0x7fe480198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.212+0000 7fe486590640 1 -- 172.21.15.43:0/3582100880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe480198590 msgr2=0x7fe480198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.212+0000 7fe486590640 1 --2- 172.21.15.43:0/3582100880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe480198590 0x7fe480198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.212+0000 7fe486590640 1 -- 172.21.15.43:0/3582100880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480105970 msgr2=0x7fe48019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.212+0000 7fe486590640 1 --2- 172.21.15.43:0/3582100880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480105970 0x7fe48019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.212+0000 7fe486590640 1 -- 172.21.15.43:0/3582100880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4801990b0 con 0x7fe480100af0 2024-04-21T08:43:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.212+0000 7fe486590640 1 --2- 172.21.15.43:0/3582100880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe480100af0 0x7fe48019e3e0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00d950 tx=0x7fe47c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.212+0000 7fe4777fe640 1 -- 172.21.15.43:0/3582100880 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c017070 con 0x7fe480100af0 2024-04-21T08:43:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.213+0000 7fe4777fe640 1 -- 172.21.15.43:0/3582100880 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe47c009c30 con 0x7fe480100af0 2024-04-21T08:43:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.213+0000 7fe4887f4640 1 -- 172.21.15.43:0/3582100880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4801993a0 con 0x7fe480100af0 2024-04-21T08:43:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.213+0000 7fe4777fe640 1 -- 172.21.15.43:0/3582100880 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c00f620 con 0x7fe480100af0 2024-04-21T08:43:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.213+0000 7fe4887f4640 1 -- 172.21.15.43:0/3582100880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4801a55f0 con 0x7fe480100af0 2024-04-21T08:43:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.215+0000 7fe48b3e6740 1 Processor -- start 2024-04-21T08:43:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.216+0000 7fe48b3e6740 1 -- start start 2024-04-21T08:43:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.216+0000 7fe4777fe640 1 -- 172.21.15.43:0/3582100880 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe47c01a070 con 0x7fe480100af0 2024-04-21T08:43:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.216+0000 7fe48b3e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5590665324d0 0x5590665328b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.216+0000 7fe48b3e6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559066532e80 0x55906653b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.216+0000 7fe48b3e6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55906653ba00 0x55906653de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.216+0000 7fe48b3e6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5590664971f0 con 0x55906653ba00 2024-04-21T08:43:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.216+0000 7fe48b3e6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5590664344b0 con 0x559066532e80 2024-04-21T08:43:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.216+0000 7fe48b3e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5590664931b0 con 0x5590665324d0 2024-04-21T08:43:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.216+0000 7fe485d8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55906653ba00 0x55906653de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.216+0000 7fe485d8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55906653ba00 0x55906653de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37040/0 (socket says 172.21.15.43:37040) 2024-04-21T08:43:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.216+0000 7fe485d8f640 1 -- 172.21.15.43:0/1614929265 learned_addr learned my addr 172.21.15.43:0/1614929265 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.216+0000 7fe486590640 1 --2- 172.21.15.43:0/1614929265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559066532e80 0x55906653b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.216+0000 7fe486d91640 1 --2- 172.21.15.43:0/1614929265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5590665324d0 0x5590665328b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.216+0000 7fe485d8f640 1 -- 172.21.15.43:0/1614929265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5590665324d0 msgr2=0x5590665328b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.216+0000 7fe485d8f640 1 --2- 172.21.15.43:0/1614929265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5590665324d0 0x5590665328b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.216+0000 7fe485d8f640 1 -- 172.21.15.43:0/1614929265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559066532e80 msgr2=0x55906653b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.216+0000 7fe485d8f640 1 --2- 172.21.15.43:0/1614929265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559066532e80 0x55906653b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.216+0000 7fe485d8f640 1 -- 172.21.15.43:0/1614929265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590664261b0 con 0x55906653ba00 2024-04-21T08:43:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.216+0000 7fe486d91640 1 --2- 172.21.15.43:0/1614929265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5590665324d0 0x5590665328b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.216+0000 7fe486590640 1 --2- 172.21.15.43:0/1614929265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559066532e80 0x55906653b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.216+0000 7fe4777fe640 1 --2- 172.21.15.43:0/3582100880 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe46c0a3da0 0x7fe46c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.217+0000 7fe485d8f640 1 --2- 172.21.15.43:0/1614929265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55906653ba00 0x55906653de50 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fe470009230 tx=0x7fe47002c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba220d6f3adee55 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.217+0000 7fe486d91640 1 --2- 172.21.15.43:0/3582100880 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe46c0a3da0 0x7fe46c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.217+0000 7fe4777fe640 1 -- 172.21.15.43:0/3582100880 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fe47c0d05c0 con 0x7fe480100af0 2024-04-21T08:43:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.217+0000 7fe474ff9640 1 -- 172.21.15.43:0/1614929265 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe470045020 con 0x55906653ba00 2024-04-21T08:43:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.217+0000 7fe474ff9640 1 -- 172.21.15.43:0/1614929265 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe47002c7e0 con 0x55906653ba00 2024-04-21T08:43:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.217+0000 7fe474ff9640 1 -- 172.21.15.43:0/1614929265 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe470005540 con 0x55906653ba00 2024-04-21T08:43:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.218+0000 7fe48b3e6740 1 -- 172.21.15.43:0/1614929265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55906653ba00 msgr2=0x55906653de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.218+0000 7fe486d91640 1 --2- 172.21.15.43:0/3582100880 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe46c0a3da0 0x7fe46c0a6260 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe478005fd0 tx=0x7fe4780094e0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.218+0000 7fe48b3e6740 1 --2- 172.21.15.43:0/1614929265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55906653ba00 0x55906653de50 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fe470009230 tx=0x7fe47002c5f0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.218+0000 7fe48b3e6740 1 -- 172.21.15.43:0/1614929265 shutdown_connections 2024-04-21T08:43:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.218+0000 7fe48b3e6740 1 --2- 172.21.15.43:0/1614929265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55906653ba00 0x55906653de50 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.218+0000 7fe48b3e6740 1 --2- 172.21.15.43:0/1614929265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559066532e80 0x55906653b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.218+0000 7fe48b3e6740 1 --2- 172.21.15.43:0/1614929265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5590665324d0 0x5590665328b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.218+0000 7fe48b3e6740 1 -- 172.21.15.43:0/1614929265 >> 172.21.15.43:0/1614929265 conn(0x55906652cba0 msgr2=0x55906652cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.218+0000 7fe48b3e6740 1 -- 172.21.15.43:0/1614929265 shutdown_connections 2024-04-21T08:43:52.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.218+0000 7fe48b3e6740 1 -- 172.21.15.43:0/1614929265 wait complete. 2024-04-21T08:43:52.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.218+0000 7fe48b3e6740 1 -- start start 2024-04-21T08:43:52.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.218+0000 7fe48b3e6740 1 Processor -- start 2024-04-21T08:43:52.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.218+0000 7fe48b3e6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590665324d0 0x55906653b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.218+0000 7fe48b3e6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559066532e80 0x5590665386b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.218+0000 7fe48b3e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55906653ba00 0x559066538c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.218+0000 7fe48b3e6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55906653e910 con 0x559066532e80 2024-04-21T08:43:52.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.218+0000 7fe48b3e6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559066450500 con 0x5590665324d0 2024-04-21T08:43:52.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.218+0000 7fe48b3e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55906653ea90 con 0x55906653ba00 2024-04-21T08:43:52.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.218+0000 7fe48b3e6740 10 client.0 _ll_register_callbacks cb 0x55906641eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:52.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.218+0000 7fe485d8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559066532e80 0x5590665386b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.219+0000 7fe485d8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559066532e80 0x5590665386b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37042/0 (socket says 172.21.15.43:37042) 2024-04-21T08:43:52.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.219+0000 7fe485d8f640 1 -- 172.21.15.43:0/2723939908 learned_addr learned my addr 172.21.15.43:0/2723939908 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.219+0000 7fe486590640 1 --2- 172.21.15.43:0/2723939908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590665324d0 0x55906653b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.219+0000 7fe485d8f640 1 -- 172.21.15.43:0/2723939908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55906653ba00 msgr2=0x559066538c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.219+0000 7fe485d8f640 1 --2- 172.21.15.43:0/2723939908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55906653ba00 0x559066538c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.219+0000 7fe485d8f640 1 -- 172.21.15.43:0/2723939908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590665324d0 msgr2=0x55906653b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.219+0000 7fe485d8f640 1 --2- 172.21.15.43:0/2723939908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590665324d0 0x55906653b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.219+0000 7fe485d8f640 1 -- 172.21.15.43:0/2723939908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590661a93f0 con 0x559066532e80 2024-04-21T08:43:52.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.219+0000 7fe485d8f640 1 --2- 172.21.15.43:0/2723939908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559066532e80 0x5590665386b0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fe47002c780 tx=0x7fe4700054f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d6887d43dd6bd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.219+0000 7fe469ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:52.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.219+0000 7fe469ffb640 1 -- 172.21.15.43:0/2723939908 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe470045070 con 0x559066532e80 2024-04-21T08:43:52.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.219+0000 7fe469ffb640 1 -- 172.21.15.43:0/2723939908 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fe4700035f0 con 0x559066532e80 2024-04-21T08:43:52.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.219+0000 7fe48b3e6740 1 -- 172.21.15.43:0/2723939908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe47002c7e0 con 0x559066532e80 2024-04-21T08:43:52.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.219+0000 7fe469ffb640 1 -- 172.21.15.43:0/2723939908 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe470059050 con 0x559066532e80 2024-04-21T08:43:52.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.219+0000 7fe48b3e6740 10 client.17703 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:52.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.219+0000 7fe48b3e6740 1 -- 172.21.15.43:0/2723939908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5590662737d0 con 0x559066532e80 2024-04-21T08:43:52.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.219+0000 7fe469ffb640 1 -- 172.21.15.43:0/2723939908 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe47002c7e0 con 0x559066532e80 2024-04-21T08:43:52.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.220+0000 7fe469ffb640 1 -- 172.21.15.43:0/2723939908 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe470054030 con 0x559066532e80 2024-04-21T08:43:52.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.220+0000 7fe48b3e6740 10 client.17703 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:52.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.220+0000 7fe48b3e6740 10 client.17703 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:43:52.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.220+0000 7fe48b3e6740 20 client.17703 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:52.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.220+0000 7fe48b3e6740 1 --2- 172.21.15.43:0/2723939908 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55906654b9b0 0x55906654dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.220+0000 7fe48b3e6740 4 client.17703 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:52.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.220+0000 7fe48b3e6740 1 -- 172.21.15.43:0/2723939908 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559066436930 con 0x55906654b9b0 2024-04-21T08:43:52.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.220+0000 7fe486590640 1 --2- 172.21.15.43:0/2723939908 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55906654b9b0 0x55906654dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.220+0000 7fe486590640 1 --2- 172.21.15.43:0/2723939908 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55906654b9b0 0x55906654dda0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=783fb007597e7c42 server_cookie=511cb032a0761fe5 in_seq=0 out_seq=0 2024-04-21T08:43:52.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.220+0000 7fe469ffb640 10 client.17703 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:43:52.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.221+0000 7fe469ffb640 1 -- 172.21.15.43:0/2723939908 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559066436930 con 0x55906654b9b0 2024-04-21T08:43:52.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.221+0000 7fe469ffb640 10 client.17703 handle_command_reply: tid=0 2024-04-21T08:43:52.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.221+0000 7fe48b3e6740 1 client.17703 shutdown 2024-04-21T08:43:52.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.221+0000 7fe48b3e6740 1 -- 172.21.15.43:0/2723939908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559066532e80 msgr2=0x5590665386b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.221+0000 7fe48b3e6740 1 --2- 172.21.15.43:0/2723939908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559066532e80 0x5590665386b0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fe47002c780 tx=0x7fe4700054f0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.221+0000 7fe48b3e6740 1 -- 172.21.15.43:0/2723939908 shutdown_connections 2024-04-21T08:43:52.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.221+0000 7fe48b3e6740 1 --2- 172.21.15.43:0/2723939908 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55906654b9b0 0x55906654dda0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.221+0000 7fe48b3e6740 1 --2- 172.21.15.43:0/2723939908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55906653ba00 0x559066538c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.221+0000 7fe469ffb640 0 client.17703 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:43:52.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.221+0000 7fe48b3e6740 1 --2- 172.21.15.43:0/2723939908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559066532e80 0x5590665386b0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.221+0000 7fe48b3e6740 1 --2- 172.21.15.43:0/2723939908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590665324d0 0x55906653b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.221+0000 7fe48b3e6740 1 -- 172.21.15.43:0/2723939908 >> 172.21.15.43:0/2723939908 conn(0x55906652cba0 msgr2=0x55906652e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.221+0000 7fe48b3e6740 1 -- 172.21.15.43:0/2723939908 shutdown_connections 2024-04-21T08:43:52.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.221+0000 7fe48b3e6740 1 -- 172.21.15.43:0/2723939908 wait complete. 2024-04-21T08:43:52.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.222+0000 7fe48b3e6740 20 client.17703 trim_cache size 0 max 16384 2024-04-21T08:43:52.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.235+0000 7fe48b3e6740 1 Processor -- start 2024-04-21T08:43:52.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.235+0000 7fe48b3e6740 1 -- start start 2024-04-21T08:43:52.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.235+0000 7fe48b3e6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590665324d0 0x5590665477d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.235+0000 7fe48b3e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559066532e80 0x55906653ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.235+0000 7fe48b3e6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55906653ba00 0x55906653b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.235+0000 7fe48b3e6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe4801af240 con 0x55906653ba00 2024-04-21T08:43:52.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.235+0000 7fe48b3e6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4801b0d80 con 0x5590665324d0 2024-04-21T08:43:52.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.235+0000 7fe48b3e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe4801af910 con 0x559066532e80 2024-04-21T08:43:52.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.235+0000 7fe486d91640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55906653ba00 0x55906653b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.235+0000 7fe486d91640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55906653ba00 0x55906653b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37058/0 (socket says 172.21.15.43:37058) 2024-04-21T08:43:52.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.235+0000 7fe486d91640 1 -- 172.21.15.43:0/1197078696 learned_addr learned my addr 172.21.15.43:0/1197078696 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.235+0000 7fe486590640 1 --2- 172.21.15.43:0/1197078696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590665324d0 0x5590665477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.235+0000 7fe485d8f640 1 --2- 172.21.15.43:0/1197078696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559066532e80 0x55906653ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.235+0000 7fe486d91640 1 -- 172.21.15.43:0/1197078696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559066532e80 msgr2=0x55906653ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.235+0000 7fe486d91640 1 --2- 172.21.15.43:0/1197078696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559066532e80 0x55906653ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.235+0000 7fe486d91640 1 -- 172.21.15.43:0/1197078696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590665324d0 msgr2=0x5590665477d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.235+0000 7fe486d91640 1 --2- 172.21.15.43:0/1197078696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590665324d0 0x5590665477d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.235+0000 7fe486d91640 1 -- 172.21.15.43:0/1197078696 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55906653ddf0 con 0x55906653ba00 2024-04-21T08:43:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.235+0000 7fe485d8f640 1 --2- 172.21.15.43:0/1197078696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559066532e80 0x55906653ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.236+0000 7fe486d91640 1 --2- 172.21.15.43:0/1197078696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55906653ba00 0x55906653b320 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fe478012580 tx=0x7fe478014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08e8b319a71e6d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.236+0000 7fe46bfff640 1 -- 172.21.15.43:0/1197078696 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478004d20 con 0x55906653ba00 2024-04-21T08:43:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.236+0000 7fe46bfff640 1 -- 172.21.15.43:0/1197078696 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4780054d0 con 0x55906653ba00 2024-04-21T08:43:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.236+0000 7fe46bfff640 1 -- 172.21.15.43:0/1197078696 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47801f9b0 con 0x55906653ba00 2024-04-21T08:43:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.236+0000 7fe48b3e6740 1 -- 172.21.15.43:0/1197078696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55906653ba00 msgr2=0x55906653b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.236+0000 7fe48b3e6740 1 --2- 172.21.15.43:0/1197078696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55906653ba00 0x55906653b320 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fe478012580 tx=0x7fe478014070 comp rx=0 tx=0).stop 2024-04-21T08:43:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.236+0000 7fe48b3e6740 1 -- 172.21.15.43:0/1197078696 shutdown_connections 2024-04-21T08:43:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.236+0000 7fe48b3e6740 1 --2- 172.21.15.43:0/1197078696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55906653ba00 0x55906653b320 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.236+0000 7fe48b3e6740 1 --2- 172.21.15.43:0/1197078696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559066532e80 0x55906653ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.236+0000 7fe48b3e6740 1 --2- 172.21.15.43:0/1197078696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590665324d0 0x5590665477d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.236+0000 7fe48b3e6740 1 -- 172.21.15.43:0/1197078696 >> 172.21.15.43:0/1197078696 conn(0x55906652cba0 msgr2=0x559066536f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.236+0000 7fe48b3e6740 1 -- 172.21.15.43:0/1197078696 shutdown_connections 2024-04-21T08:43:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.236+0000 7fe48b3e6740 1 -- 172.21.15.43:0/1197078696 wait complete. 2024-04-21T08:43:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.237+0000 7fe48b3e6740 1 -- start start 2024-04-21T08:43:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.237+0000 7fe48b3e6740 1 Processor -- start 2024-04-21T08:43:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.237+0000 7fe48b3e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5590665324d0 0x55906653f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.237+0000 7fe48b3e6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559066532e80 0x55906653fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.237+0000 7fe48b3e6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55906653ba00 0x5590665401f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.237+0000 7fe48b3e6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe4801b06b0 con 0x55906653ba00 2024-04-21T08:43:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.237+0000 7fe48b3e6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4801affe0 con 0x559066532e80 2024-04-21T08:43:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.237+0000 7fe48b3e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe4801ad220 con 0x5590665324d0 2024-04-21T08:43:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.237+0000 7fe48b3e6740 10 client.0 _ll_register_callbacks cb 0x7fe47002cd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.237+0000 7fe486d91640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55906653ba00 0x5590665401f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.237+0000 7fe486d91640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55906653ba00 0x5590665401f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37074/0 (socket says 172.21.15.43:37074) 2024-04-21T08:43:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.237+0000 7fe486d91640 1 -- 172.21.15.43:0/441126911 learned_addr learned my addr 172.21.15.43:0/441126911 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.237+0000 7fe485d8f640 1 --2- 172.21.15.43:0/441126911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559066532e80 0x55906653fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.237+0000 7fe486590640 1 --2- 172.21.15.43:0/441126911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5590665324d0 0x55906653f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.238+0000 7fe486d91640 1 -- 172.21.15.43:0/441126911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5590665324d0 msgr2=0x55906653f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.238+0000 7fe486d91640 1 --2- 172.21.15.43:0/441126911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5590665324d0 0x55906653f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.238+0000 7fe486d91640 1 -- 172.21.15.43:0/441126911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559066532e80 msgr2=0x55906653fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.238+0000 7fe486d91640 1 --2- 172.21.15.43:0/441126911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559066532e80 0x55906653fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.238+0000 7fe486d91640 1 -- 172.21.15.43:0/441126911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590665b0c70 con 0x55906653ba00 2024-04-21T08:43:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.238+0000 7fe486d91640 1 --2- 172.21.15.43:0/441126911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55906653ba00 0x5590665401f0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fe478005960 tx=0x7fe478005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69cbde26679d9f65 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.238+0000 7fe469ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.238+0000 7fe469ffb640 1 -- 172.21.15.43:0/441126911 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478004e50 con 0x55906653ba00 2024-04-21T08:43:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.238+0000 7fe48b3e6740 1 -- 172.21.15.43:0/441126911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4780054d0 con 0x55906653ba00 2024-04-21T08:43:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.238+0000 7fe469ffb640 1 -- 172.21.15.43:0/441126911 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fe47801f730 con 0x55906653ba00 2024-04-21T08:43:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.238+0000 7fe469ffb640 1 -- 172.21.15.43:0/441126911 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe47801f930 con 0x55906653ba00 2024-04-21T08:43:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.238+0000 7fe48b3e6740 10 client.17709 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.238+0000 7fe48b3e6740 1 -- 172.21.15.43:0/441126911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5590665b1ec0 con 0x55906653ba00 2024-04-21T08:43:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.238+0000 7fe469ffb640 1 -- 172.21.15.43:0/441126911 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4780054d0 con 0x55906653ba00 2024-04-21T08:43:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.238+0000 7fe469ffb640 1 -- 172.21.15.43:0/441126911 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe47800f050 con 0x55906653ba00 2024-04-21T08:43:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.238+0000 7fe48b3e6740 10 client.17709 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.238+0000 7fe48b3e6740 10 client.17709 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:43:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.238+0000 7fe48b3e6740 20 client.17709 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.238+0000 7fe48b3e6740 1 --2- 172.21.15.43:0/441126911 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5590665b2580 0x5590665b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.238+0000 7fe48b3e6740 4 client.17709 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:43:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.238+0000 7fe48b3e6740 1 -- 172.21.15.43:0/441126911 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5590664a4c10 con 0x5590665b2580 2024-04-21T08:43:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.239+0000 7fe486590640 1 --2- 172.21.15.43:0/441126911 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5590665b2580 0x5590665b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.239+0000 7fe469ffb640 10 client.17709 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:43:52.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.239+0000 7fe486590640 1 --2- 172.21.15.43:0/441126911 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5590665b2580 0x5590665b49a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c6b56649c3ca8a0f server_cookie=9ca4109d3bef4f0 in_seq=0 out_seq=0 2024-04-21T08:43:52.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.239+0000 7fe469ffb640 1 -- 172.21.15.43:0/441126911 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5590664a4c10 con 0x5590665b2580 2024-04-21T08:43:52.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.239+0000 7fe469ffb640 10 client.17709 handle_command_reply: tid=0 2024-04-21T08:43:52.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.239+0000 7fe48b3e6740 1 client.17709 shutdown 2024-04-21T08:43:52.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.239+0000 7fe48b3e6740 1 -- 172.21.15.43:0/441126911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55906653ba00 msgr2=0x5590665401f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.239+0000 7fe48b3e6740 1 --2- 172.21.15.43:0/441126911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55906653ba00 0x5590665401f0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fe478005960 tx=0x7fe478005990 comp rx=0 tx=0).stop 2024-04-21T08:43:52.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.239+0000 7fe48b3e6740 1 -- 172.21.15.43:0/441126911 shutdown_connections 2024-04-21T08:43:52.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.239+0000 7fe48b3e6740 1 --2- 172.21.15.43:0/441126911 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5590665b2580 0x5590665b49a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.239+0000 7fe48b3e6740 1 --2- 172.21.15.43:0/441126911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55906653ba00 0x5590665401f0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.239+0000 7fe48b3e6740 1 --2- 172.21.15.43:0/441126911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559066532e80 0x55906653fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.239+0000 7fe469ffb640 0 client.17709 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:43:52.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.239+0000 7fe48b3e6740 1 --2- 172.21.15.43:0/441126911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5590665324d0 0x55906653f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.239+0000 7fe48b3e6740 1 -- 172.21.15.43:0/441126911 >> 172.21.15.43:0/441126911 conn(0x55906652cba0 msgr2=0x559066533260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.239+0000 7fe48b3e6740 1 -- 172.21.15.43:0/441126911 shutdown_connections 2024-04-21T08:43:52.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.240+0000 7fe48b3e6740 1 -- 172.21.15.43:0/441126911 wait complete. 2024-04-21T08:43:52.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.240+0000 7fe48b3e6740 20 client.17709 trim_cache size 0 max 16384 2024-04-21T08:43:52.247 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:43:52.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.240+0000 7fe4887f4640 1 -- 172.21.15.43:0/3582100880 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe46c0a3da0 msgr2=0x7fe46c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.240+0000 7fe4887f4640 1 --2- 172.21.15.43:0/3582100880 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe46c0a3da0 0x7fe46c0a6260 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe478005fd0 tx=0x7fe4780094e0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.240+0000 7fe4887f4640 1 -- 172.21.15.43:0/3582100880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe480100af0 msgr2=0x7fe48019e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.240+0000 7fe4887f4640 1 --2- 172.21.15.43:0/3582100880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe480100af0 0x7fe48019e3e0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00d950 tx=0x7fe47c00de10 comp rx=0 tx=0).stop 2024-04-21T08:43:52.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.240+0000 7fe4887f4640 1 -- 172.21.15.43:0/3582100880 shutdown_connections 2024-04-21T08:43:52.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.240+0000 7fe4887f4640 1 --2- 172.21.15.43:0/3582100880 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe46c0a3da0 0x7fe46c0a6260 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.240+0000 7fe4887f4640 1 --2- 172.21.15.43:0/3582100880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe480198590 0x7fe480198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.240+0000 7fe4887f4640 1 --2- 172.21.15.43:0/3582100880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480105970 0x7fe48019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.240+0000 7fe4887f4640 1 --2- 172.21.15.43:0/3582100880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe480100af0 0x7fe48019e3e0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.240+0000 7fe4887f4640 1 -- 172.21.15.43:0/3582100880 >> 172.21.15.43:0/3582100880 conn(0x7fe4800760e0 msgr2=0x7fe4800feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.240+0000 7fe4887f4640 1 -- 172.21.15.43:0/3582100880 shutdown_connections 2024-04-21T08:43:52.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.240+0000 7fe4887f4640 1 -- 172.21.15.43:0/3582100880 wait complete. 2024-04-21T08:43:52.251 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:43:52.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.356+0000 7f10dde11640 1 -- 172.21.15.43:0/3376383599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10d81072b0 msgr2=0x7f10d810ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.356+0000 7f10dde11640 1 --2- 172.21.15.43:0/3376383599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10d81072b0 0x7f10d810ee80 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f10d0009ca0 tx=0x7f10d002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.356+0000 7f10dde11640 1 -- 172.21.15.43:0/3376383599 shutdown_connections 2024-04-21T08:43:52.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.356+0000 7f10dde11640 1 --2- 172.21.15.43:0/3376383599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10d81072b0 0x7f10d810ee80 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.356+0000 7f10dde11640 1 --2- 172.21.15.43:0/3376383599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10d8102b90 0x7f10d8106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.356+0000 7f10dde11640 1 --2- 172.21.15.43:0/3376383599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10d81021e0 0x7f10d81025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.356+0000 7f10dde11640 1 -- 172.21.15.43:0/3376383599 >> 172.21.15.43:0/3376383599 conn(0x7f10d80fda60 msgr2=0x7f10d80ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.356+0000 7f10dde11640 1 -- 172.21.15.43:0/3376383599 shutdown_connections 2024-04-21T08:43:52.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.356+0000 7f10dde11640 1 -- 172.21.15.43:0/3376383599 wait complete. 2024-04-21T08:43:52.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.356+0000 7f10dde11640 1 Processor -- start 2024-04-21T08:43:52.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.357+0000 7f10dde11640 1 -- start start 2024-04-21T08:43:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.357+0000 7f10dde11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10d81021e0 0x7f10d819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.357+0000 7f10dde11640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10d81072b0 0x7f10d819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.357+0000 7f10dde11640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10d81985e0 0x7f10d8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.357+0000 7f10dde11640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f10d8112220 con 0x7f10d81021e0 2024-04-21T08:43:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.357+0000 7f10dde11640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f10d81120a0 con 0x7f10d81072b0 2024-04-21T08:43:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.357+0000 7f10dde11640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f10d81123a0 con 0x7f10d81985e0 2024-04-21T08:43:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.357+0000 7f10d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10d81021e0 0x7f10d819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.357+0000 7f10d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10d81021e0 0x7f10d819e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37094/0 (socket says 172.21.15.43:37094) 2024-04-21T08:43:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.357+0000 7f10d77fe640 1 -- 172.21.15.43:0/1428528083 learned_addr learned my addr 172.21.15.43:0/1428528083 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.357+0000 7f10d6ffd640 1 --2- 172.21.15.43:0/1428528083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10d81072b0 0x7f10d819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.357+0000 7f10d77fe640 1 -- 172.21.15.43:0/1428528083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10d81985e0 msgr2=0x7f10d8198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.357+0000 7f10d77fe640 1 --2- 172.21.15.43:0/1428528083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10d81985e0 0x7f10d8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.357+0000 7f10d77fe640 1 -- 172.21.15.43:0/1428528083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10d81072b0 msgr2=0x7f10d819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.357+0000 7f10d77fe640 1 --2- 172.21.15.43:0/1428528083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10d81072b0 0x7f10d819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.357+0000 7f10d77fe640 1 -- 172.21.15.43:0/1428528083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d8199100 con 0x7f10d81021e0 2024-04-21T08:43:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.357+0000 7f10d77fe640 1 --2- 172.21.15.43:0/1428528083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10d81021e0 0x7f10d819e3e0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f10c800d950 tx=0x7f10c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.358+0000 7f10d4ff9640 1 -- 172.21.15.43:0/1428528083 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8017070 con 0x7f10d81021e0 2024-04-21T08:43:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.358+0000 7f10dde11640 1 -- 172.21.15.43:0/1428528083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d81993f0 con 0x7f10d81021e0 2024-04-21T08:43:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.358+0000 7f10d4ff9640 1 -- 172.21.15.43:0/1428528083 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10c8009c30 con 0x7f10d81021e0 2024-04-21T08:43:52.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.358+0000 7f10d4ff9640 1 -- 172.21.15.43:0/1428528083 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c800f620 con 0x7f10d81021e0 2024-04-21T08:43:52.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.358+0000 7f10dde11640 1 -- 172.21.15.43:0/1428528083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d81a5640 con 0x7f10d81021e0 2024-04-21T08:43:52.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.360+0000 7f10d4ff9640 1 -- 172.21.15.43:0/1428528083 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f10c8010460 con 0x7f10d81021e0 2024-04-21T08:43:52.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.360+0000 7f10e0a00740 1 Processor -- start 2024-04-21T08:43:52.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.360+0000 7f10e0a00740 1 -- start start 2024-04-21T08:43:52.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.360+0000 7f10e0a00740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563d4ebc4d0 0x5563d4ebc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.360+0000 7f10e0a00740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ebce80 0x5563d4ec5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.360+0000 7f10e0a00740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d4ec5a00 0x5563d4ec7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.360+0000 7f10e0a00740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5563d4e211f0 con 0x5563d4ebce80 2024-04-21T08:43:52.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.360+0000 7f10e0a00740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5563d4dbe4b0 con 0x5563d4ebc4d0 2024-04-21T08:43:52.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.360+0000 7f10e0a00740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5563d4e1d1b0 con 0x5563d4ec5a00 2024-04-21T08:43:52.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.360+0000 7f10d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ebce80 0x5563d4ec5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.360+0000 7f10d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ebce80 0x5563d4ec5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37102/0 (socket says 172.21.15.43:37102) 2024-04-21T08:43:52.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.360+0000 7f10d77fe640 1 -- 172.21.15.43:0/1062715492 learned_addr learned my addr 172.21.15.43:0/1062715492 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.360+0000 7f10d7fff640 1 --2- 172.21.15.43:0/1062715492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563d4ebc4d0 0x5563d4ebc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.361+0000 7f10d4ff9640 1 --2- 172.21.15.43:0/1428528083 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f10c00a3d20 0x7f10c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.361+0000 7f10d77fe640 1 -- 172.21.15.43:0/1062715492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d4ec5a00 msgr2=0x5563d4ec7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.361+0000 7f10d77fe640 1 --2- 172.21.15.43:0/1062715492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d4ec5a00 0x5563d4ec7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.361+0000 7f10d77fe640 1 -- 172.21.15.43:0/1062715492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563d4ebc4d0 msgr2=0x5563d4ebc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.361+0000 7f10d77fe640 1 --2- 172.21.15.43:0/1062715492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563d4ebc4d0 0x5563d4ebc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.361+0000 7f10d77fe640 1 -- 172.21.15.43:0/1062715492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563d4db01b0 con 0x5563d4ebce80 2024-04-21T08:43:52.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.361+0000 7f10d7fff640 1 --2- 172.21.15.43:0/1428528083 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f10c00a3d20 0x7f10c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.361+0000 7f10d4ff9640 1 -- 172.21.15.43:0/1428528083 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f10c80d08c0 con 0x7f10d81021e0 2024-04-21T08:43:52.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.361+0000 7f10d77fe640 1 --2- 172.21.15.43:0/1062715492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ebce80 0x5563d4ec5430 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f10c80cdbb0 tx=0x7f10c8022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42de646121d283e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.361+0000 7f10bdffb640 1 -- 172.21.15.43:0/1062715492 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c809bfd0 con 0x5563d4ebce80 2024-04-21T08:43:52.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.361+0000 7f10d7fff640 1 --2- 172.21.15.43:0/1428528083 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f10c00a3d20 0x7f10c00a61e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f10d0004af0 tx=0x7f10d002e040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.361+0000 7f10bdffb640 1 -- 172.21.15.43:0/1062715492 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10c80859d0 con 0x5563d4ebce80 2024-04-21T08:43:52.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.361+0000 7f10bdffb640 1 -- 172.21.15.43:0/1062715492 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c809bdf0 con 0x5563d4ebce80 2024-04-21T08:43:52.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.362+0000 7f10e0a00740 1 -- 172.21.15.43:0/1062715492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ebce80 msgr2=0x5563d4ec5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.362+0000 7f10e0a00740 1 --2- 172.21.15.43:0/1062715492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ebce80 0x5563d4ec5430 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f10c80cdbb0 tx=0x7f10c8022000 comp rx=0 tx=0).stop 2024-04-21T08:43:52.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.362+0000 7f10e0a00740 1 -- 172.21.15.43:0/1062715492 shutdown_connections 2024-04-21T08:43:52.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.362+0000 7f10e0a00740 1 --2- 172.21.15.43:0/1062715492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d4ec5a00 0x5563d4ec7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.362+0000 7f10e0a00740 1 --2- 172.21.15.43:0/1062715492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ebce80 0x5563d4ec5430 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.362+0000 7f10e0a00740 1 --2- 172.21.15.43:0/1062715492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563d4ebc4d0 0x5563d4ebc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.362+0000 7f10e0a00740 1 -- 172.21.15.43:0/1062715492 >> 172.21.15.43:0/1062715492 conn(0x5563d4eb6ba0 msgr2=0x5563d4eb6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.362+0000 7f10e0a00740 1 -- 172.21.15.43:0/1062715492 shutdown_connections 2024-04-21T08:43:52.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.362+0000 7f10e0a00740 1 -- 172.21.15.43:0/1062715492 wait complete. 2024-04-21T08:43:52.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.362+0000 7f10e0a00740 1 -- start start 2024-04-21T08:43:52.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.363+0000 7f10e0a00740 1 Processor -- start 2024-04-21T08:43:52.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.363+0000 7f10e0a00740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d4ebc4d0 0x5563d4ec5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.363+0000 7f10e0a00740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563d4ebce80 0x5563d4ec2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.363+0000 7f10e0a00740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ec5a00 0x5563d4ec2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.363+0000 7f10e0a00740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5563d4ec88b0 con 0x5563d4ec5a00 2024-04-21T08:43:52.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.363+0000 7f10e0a00740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5563d4dda500 con 0x5563d4ebce80 2024-04-21T08:43:52.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.363+0000 7f10e0a00740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5563d4ec8a30 con 0x5563d4ebc4d0 2024-04-21T08:43:52.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.363+0000 7f10e0a00740 10 client.0 _ll_register_callbacks cb 0x5563d4da8b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:52.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.363+0000 7f10d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ec5a00 0x5563d4ec2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.363+0000 7f10d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ec5a00 0x5563d4ec2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37112/0 (socket says 172.21.15.43:37112) 2024-04-21T08:43:52.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.363+0000 7f10d7fff640 1 -- 172.21.15.43:0/3055127740 learned_addr learned my addr 172.21.15.43:0/3055127740 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.363+0000 7f10d6ffd640 1 --2- 172.21.15.43:0/3055127740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563d4ebce80 0x5563d4ec2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.363+0000 7f10d7fff640 1 -- 172.21.15.43:0/3055127740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d4ebc4d0 msgr2=0x5563d4ec5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.363+0000 7f10d7fff640 1 --2- 172.21.15.43:0/3055127740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d4ebc4d0 0x5563d4ec5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.363+0000 7f10d7fff640 1 -- 172.21.15.43:0/3055127740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563d4ebce80 msgr2=0x5563d4ec2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.363+0000 7f10d7fff640 1 --2- 172.21.15.43:0/3055127740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563d4ebce80 0x5563d4ec2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.363+0000 7f10d7fff640 1 -- 172.21.15.43:0/3055127740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563d4b333f0 con 0x5563d4ec5a00 2024-04-21T08:43:52.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.363+0000 7f10d6ffd640 1 --2- 172.21.15.43:0/3055127740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563d4ebce80 0x5563d4ec2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:52.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.364+0000 7f10d7fff640 1 --2- 172.21.15.43:0/3055127740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ec5a00 0x5563d4ec2c10 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f10d0009ca0 tx=0x7f10d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eadaffdef440d680 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.364+0000 7f10b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:52.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.364+0000 7f10b6ffd640 1 -- 172.21.15.43:0/3055127740 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0050060 con 0x5563d4ec5a00 2024-04-21T08:43:52.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.364+0000 7f10b6ffd640 1 -- 172.21.15.43:0/3055127740 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f10d0003b80 con 0x5563d4ec5a00 2024-04-21T08:43:52.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.364+0000 7f10e0a00740 1 -- 172.21.15.43:0/3055127740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10c80859d0 con 0x5563d4ec5a00 2024-04-21T08:43:52.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.364+0000 7f10b6ffd640 1 -- 172.21.15.43:0/3055127740 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f10d0057050 con 0x5563d4ec5a00 2024-04-21T08:43:52.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.364+0000 7f10e0a00740 10 client.17721 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:52.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.364+0000 7f10e0a00740 1 -- 172.21.15.43:0/3055127740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5563d4bfd7d0 con 0x5563d4ec5a00 2024-04-21T08:43:52.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.364+0000 7f10b6ffd640 1 -- 172.21.15.43:0/3055127740 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10c80859d0 con 0x5563d4ec5a00 2024-04-21T08:43:52.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.364+0000 7f10b6ffd640 1 -- 172.21.15.43:0/3055127740 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f10d00345f0 con 0x5563d4ec5a00 2024-04-21T08:43:52.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.364+0000 7f10e0a00740 10 client.17721 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:52.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.364+0000 7f10e0a00740 10 client.17721 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:43:52.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.364+0000 7f10e0a00740 20 client.17721 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:52.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.364+0000 7f10e0a00740 1 --2- 172.21.15.43:0/3055127740 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5563d4ed5b80 0x5563d4ed8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.364+0000 7f10e0a00740 4 client.17721 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:52.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.364+0000 7f10e0a00740 1 -- 172.21.15.43:0/3055127740 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563d4dc0930 con 0x5563d4ed5b80 2024-04-21T08:43:52.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.365+0000 7f10d77fe640 1 --2- 172.21.15.43:0/3055127740 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5563d4ed5b80 0x5563d4ed8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.365+0000 7f10d77fe640 1 --2- 172.21.15.43:0/3055127740 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5563d4ed5b80 0x5563d4ed8020 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ab1635c9d1f4656d server_cookie=5b05c28fc23b5197 in_seq=0 out_seq=0 2024-04-21T08:43:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.365+0000 7f10b6ffd640 10 client.17721 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:43:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.366+0000 7f10b6ffd640 1 -- 172.21.15.43:0/3055127740 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5563d4dc0930 con 0x5563d4ed5b80 2024-04-21T08:43:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.366+0000 7f10b6ffd640 10 client.17721 handle_command_reply: tid=0 2024-04-21T08:43:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.366+0000 7f10e0a00740 1 client.17721 shutdown 2024-04-21T08:43:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.366+0000 7f10e0a00740 1 -- 172.21.15.43:0/3055127740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ec5a00 msgr2=0x5563d4ec2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.366+0000 7f10e0a00740 1 --2- 172.21.15.43:0/3055127740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ec5a00 0x5563d4ec2c10 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f10d0009ca0 tx=0x7f10d0004a40 comp rx=0 tx=0).stop 2024-04-21T08:43:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.366+0000 7f10e0a00740 1 -- 172.21.15.43:0/3055127740 shutdown_connections 2024-04-21T08:43:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.366+0000 7f10e0a00740 1 --2- 172.21.15.43:0/3055127740 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5563d4ed5b80 0x5563d4ed8020 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.366+0000 7f10e0a00740 1 --2- 172.21.15.43:0/3055127740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ec5a00 0x5563d4ec2c10 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.366+0000 7f10b6ffd640 0 client.17721 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:43:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.366+0000 7f10e0a00740 1 --2- 172.21.15.43:0/3055127740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563d4ebce80 0x5563d4ec2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.366+0000 7f10e0a00740 1 --2- 172.21.15.43:0/3055127740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d4ebc4d0 0x5563d4ec5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.366+0000 7f10e0a00740 1 -- 172.21.15.43:0/3055127740 >> 172.21.15.43:0/3055127740 conn(0x5563d4eb6ba0 msgr2=0x5563d4eb88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.366+0000 7f10e0a00740 1 -- 172.21.15.43:0/3055127740 shutdown_connections 2024-04-21T08:43:52.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.366+0000 7f10e0a00740 1 -- 172.21.15.43:0/3055127740 wait complete. 2024-04-21T08:43:52.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.366+0000 7f10e0a00740 20 client.17721 trim_cache size 0 max 16384 2024-04-21T08:43:52.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10e0a00740 1 Processor -- start 2024-04-21T08:43:52.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10e0a00740 1 -- start start 2024-04-21T08:43:52.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10e0a00740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d4ebc4d0 0x5563d4ed1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10e0a00740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563d4ebce80 0x5563d4eb5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10e0a00740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ec5a00 0x5563d4ec4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10e0a00740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f10d81af440 con 0x5563d4ec5a00 2024-04-21T08:43:52.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10e0a00740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f10d81b0f80 con 0x5563d4ebce80 2024-04-21T08:43:52.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10e0a00740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f10d81afb10 con 0x5563d4ebc4d0 2024-04-21T08:43:52.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ec5a00 0x5563d4ec4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ec5a00 0x5563d4ec4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37122/0 (socket says 172.21.15.43:37122) 2024-04-21T08:43:52.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10d7fff640 1 -- 172.21.15.43:0/2186192387 learned_addr learned my addr 172.21.15.43:0/2186192387 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10d77fe640 1 --2- 172.21.15.43:0/2186192387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d4ebc4d0 0x5563d4ed1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10d6ffd640 1 --2- 172.21.15.43:0/2186192387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563d4ebce80 0x5563d4eb5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10d7fff640 1 -- 172.21.15.43:0/2186192387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d4ebc4d0 msgr2=0x5563d4ed1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10d7fff640 1 --2- 172.21.15.43:0/2186192387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d4ebc4d0 0x5563d4ed1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10d7fff640 1 -- 172.21.15.43:0/2186192387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563d4ebce80 msgr2=0x5563d4eb5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10d7fff640 1 --2- 172.21.15.43:0/2186192387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563d4ebce80 0x5563d4eb5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10d7fff640 1 -- 172.21.15.43:0/2186192387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563d4ebe270 con 0x5563d4ec5a00 2024-04-21T08:43:52.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10d7fff640 1 --2- 172.21.15.43:0/2186192387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ec5a00 0x5563d4ec4dc0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f10d002e010 tx=0x7f10d0032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ecf48e0fa4848 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10bd7fa640 1 -- 172.21.15.43:0/2186192387 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0050070 con 0x5563d4ec5a00 2024-04-21T08:43:52.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10bd7fa640 1 -- 172.21.15.43:0/2186192387 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10d0046050 con 0x5563d4ec5a00 2024-04-21T08:43:52.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10bd7fa640 1 -- 172.21.15.43:0/2186192387 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0039040 con 0x5563d4ec5a00 2024-04-21T08:43:52.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10e0a00740 1 -- 172.21.15.43:0/2186192387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ec5a00 msgr2=0x5563d4ec4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10e0a00740 1 --2- 172.21.15.43:0/2186192387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ec5a00 0x5563d4ec4dc0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f10d002e010 tx=0x7f10d0032c90 comp rx=0 tx=0).stop 2024-04-21T08:43:52.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10e0a00740 1 -- 172.21.15.43:0/2186192387 shutdown_connections 2024-04-21T08:43:52.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10e0a00740 1 --2- 172.21.15.43:0/2186192387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ec5a00 0x5563d4ec4dc0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10e0a00740 1 --2- 172.21.15.43:0/2186192387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563d4ebce80 0x5563d4eb5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10e0a00740 1 --2- 172.21.15.43:0/2186192387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d4ebc4d0 0x5563d4ed1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10e0a00740 1 -- 172.21.15.43:0/2186192387 >> 172.21.15.43:0/2186192387 conn(0x5563d4eb6ba0 msgr2=0x5563d4eb5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10e0a00740 1 -- 172.21.15.43:0/2186192387 shutdown_connections 2024-04-21T08:43:52.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.381+0000 7f10e0a00740 1 -- 172.21.15.43:0/2186192387 wait complete. 2024-04-21T08:43:52.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.382+0000 7f10e0a00740 1 -- start start 2024-04-21T08:43:52.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.382+0000 7f10e0a00740 1 Processor -- start 2024-04-21T08:43:52.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.382+0000 7f10e0a00740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563d4ebc4d0 0x5563d4ecdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.382+0000 7f10e0a00740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ebce80 0x5563d4ec92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.382+0000 7f10e0a00740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d4ec5a00 0x5563d4ec9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.382+0000 7f10e0a00740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f10d81b08b0 con 0x5563d4ebce80 2024-04-21T08:43:52.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.382+0000 7f10e0a00740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f10d81b01e0 con 0x5563d4ebc4d0 2024-04-21T08:43:52.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.382+0000 7f10e0a00740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f10d81ad420 con 0x5563d4ec5a00 2024-04-21T08:43:52.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.382+0000 7f10e0a00740 10 client.0 _ll_register_callbacks cb 0x7f10d0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:52.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.382+0000 7f10d6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ebce80 0x5563d4ec92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.382+0000 7f10d6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ebce80 0x5563d4ec92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37136/0 (socket says 172.21.15.43:37136) 2024-04-21T08:43:52.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.382+0000 7f10d6ffd640 1 -- 172.21.15.43:0/3586784139 learned_addr learned my addr 172.21.15.43:0/3586784139 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.382+0000 7f10d77fe640 1 --2- 172.21.15.43:0/3586784139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563d4ebc4d0 0x5563d4ecdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.382+0000 7f10d7fff640 1 --2- 172.21.15.43:0/3586784139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d4ec5a00 0x5563d4ec9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.383+0000 7f10d6ffd640 1 -- 172.21.15.43:0/3586784139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d4ec5a00 msgr2=0x5563d4ec9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.383+0000 7f10d6ffd640 1 --2- 172.21.15.43:0/3586784139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d4ec5a00 0x5563d4ec9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.383+0000 7f10d6ffd640 1 -- 172.21.15.43:0/3586784139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563d4ebc4d0 msgr2=0x5563d4ecdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.383+0000 7f10d6ffd640 1 --2- 172.21.15.43:0/3586784139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563d4ebc4d0 0x5563d4ecdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.383+0000 7f10d6ffd640 1 -- 172.21.15.43:0/3586784139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563d4ec9f40 con 0x5563d4ebce80 2024-04-21T08:43:52.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.383+0000 7f10d6ffd640 1 --2- 172.21.15.43:0/3586784139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ebce80 0x5563d4ec92c0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f10d80fdf00 tx=0x7f10cc012880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c25e4b0271503a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.383+0000 7f10b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:52.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.383+0000 7f10b6ffd640 1 -- 172.21.15.43:0/3586784139 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc01f060 con 0x5563d4ebce80 2024-04-21T08:43:52.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.383+0000 7f10b6ffd640 1 -- 172.21.15.43:0/3586784139 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f10cc010810 con 0x5563d4ebce80 2024-04-21T08:43:52.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.383+0000 7f10e0a00740 1 -- 172.21.15.43:0/3586784139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10d0046050 con 0x5563d4ebce80 2024-04-21T08:43:52.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.383+0000 7f10b6ffd640 1 -- 172.21.15.43:0/3586784139 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f10cc026050 con 0x5563d4ebce80 2024-04-21T08:43:52.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.383+0000 7f10e0a00740 10 client.17727 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:52.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.383+0000 7f10e0a00740 1 -- 172.21.15.43:0/3586784139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5563d4f3c0f0 con 0x5563d4ebce80 2024-04-21T08:43:52.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.383+0000 7f10b6ffd640 1 -- 172.21.15.43:0/3586784139 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10d0046050 con 0x5563d4ebce80 2024-04-21T08:43:52.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.383+0000 7f10b6ffd640 1 -- 172.21.15.43:0/3586784139 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f10cc019030 con 0x5563d4ebce80 2024-04-21T08:43:52.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.383+0000 7f10e0a00740 10 client.17727 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:52.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.383+0000 7f10e0a00740 10 client.17727 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:43:52.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.383+0000 7f10e0a00740 20 client.17727 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:52.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.383+0000 7f10e0a00740 1 --2- 172.21.15.43:0/3586784139 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5563d4f3c870 0x5563d4f3ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.383+0000 7f10e0a00740 4 client.17727 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:43:52.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.383+0000 7f10e0a00740 1 -- 172.21.15.43:0/3586784139 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5563d4e2ec10 con 0x5563d4f3c870 2024-04-21T08:43:52.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.384+0000 7f10d77fe640 1 --2- 172.21.15.43:0/3586784139 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5563d4f3c870 0x5563d4f3ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.384+0000 7f10b6ffd640 10 client.17727 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:43:52.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.384+0000 7f10d77fe640 1 --2- 172.21.15.43:0/3586784139 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5563d4f3c870 0x5563d4f3ec90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fbe7aa5c5d2868c4 server_cookie=61c0f4d2ff106a82 in_seq=0 out_seq=0 2024-04-21T08:43:52.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.384+0000 7f10b6ffd640 1 -- 172.21.15.43:0/3586784139 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5563d4e2ec10 con 0x5563d4f3c870 2024-04-21T08:43:52.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.384+0000 7f10b6ffd640 10 client.17727 handle_command_reply: tid=0 2024-04-21T08:43:52.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.384+0000 7f10e0a00740 1 client.17727 shutdown 2024-04-21T08:43:52.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.384+0000 7f10e0a00740 1 -- 172.21.15.43:0/3586784139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ebce80 msgr2=0x5563d4ec92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.384+0000 7f10e0a00740 1 --2- 172.21.15.43:0/3586784139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ebce80 0x5563d4ec92c0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f10d80fdf00 tx=0x7f10cc012880 comp rx=0 tx=0).stop 2024-04-21T08:43:52.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.384+0000 7f10e0a00740 1 -- 172.21.15.43:0/3586784139 shutdown_connections 2024-04-21T08:43:52.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.384+0000 7f10e0a00740 1 --2- 172.21.15.43:0/3586784139 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5563d4f3c870 0x5563d4f3ec90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.384+0000 7f10e0a00740 1 --2- 172.21.15.43:0/3586784139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5563d4ec5a00 0x5563d4ec9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.384+0000 7f10e0a00740 1 --2- 172.21.15.43:0/3586784139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5563d4ebce80 0x5563d4ec92c0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.384+0000 7f10b6ffd640 0 client.17727 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:43:52.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.384+0000 7f10e0a00740 1 --2- 172.21.15.43:0/3586784139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563d4ebc4d0 0x5563d4ecdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.384+0000 7f10e0a00740 1 -- 172.21.15.43:0/3586784139 >> 172.21.15.43:0/3586784139 conn(0x5563d4eb6ba0 msgr2=0x5563d4ebdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.385+0000 7f10e0a00740 1 -- 172.21.15.43:0/3586784139 shutdown_connections 2024-04-21T08:43:52.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.385+0000 7f10e0a00740 1 -- 172.21.15.43:0/3586784139 wait complete. 2024-04-21T08:43:52.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.385+0000 7f10e0a00740 20 client.17727 trim_cache size 0 max 16384 2024-04-21T08:43:52.392 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:43:52.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.385+0000 7f10dde11640 1 -- 172.21.15.43:0/1428528083 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f10c00a3d20 msgr2=0x7f10c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.385+0000 7f10dde11640 1 --2- 172.21.15.43:0/1428528083 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f10c00a3d20 0x7f10c00a61e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f10d0004af0 tx=0x7f10d002e040 comp rx=0 tx=0).stop 2024-04-21T08:43:52.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.385+0000 7f10dde11640 1 -- 172.21.15.43:0/1428528083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10d81021e0 msgr2=0x7f10d819e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.385+0000 7f10dde11640 1 --2- 172.21.15.43:0/1428528083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10d81021e0 0x7f10d819e3e0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f10c800d950 tx=0x7f10c800de10 comp rx=0 tx=0).stop 2024-04-21T08:43:52.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.385+0000 7f10dde11640 1 -- 172.21.15.43:0/1428528083 shutdown_connections 2024-04-21T08:43:52.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.385+0000 7f10dde11640 1 --2- 172.21.15.43:0/1428528083 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f10c00a3d20 0x7f10c00a61e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.385+0000 7f10dde11640 1 --2- 172.21.15.43:0/1428528083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10d81985e0 0x7f10d8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.385+0000 7f10dde11640 1 --2- 172.21.15.43:0/1428528083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10d81072b0 0x7f10d819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.385+0000 7f10dde11640 1 --2- 172.21.15.43:0/1428528083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10d81021e0 0x7f10d819e3e0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.385+0000 7f10dde11640 1 -- 172.21.15.43:0/1428528083 >> 172.21.15.43:0/1428528083 conn(0x7f10d80fda60 msgr2=0x7f10d80ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.385+0000 7f10dde11640 1 -- 172.21.15.43:0/1428528083 shutdown_connections 2024-04-21T08:43:52.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.385+0000 7f10dde11640 1 -- 172.21.15.43:0/1428528083 wait complete. 2024-04-21T08:43:52.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:52 smithi043 ceph-mon[27978]: pgmap v151: 129 pgs: 129 active+clean; 2.5 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-21T08:43:52.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:52 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3908235331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:43:52.396 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:43:52.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.500+0000 7fcb696ba640 1 -- 172.21.15.43:0/2019324977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb641072b0 msgr2=0x7fcb6410ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.500+0000 7fcb696ba640 1 --2- 172.21.15.43:0/2019324977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb641072b0 0x7fcb6410ee80 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009ca0 tx=0x7fcb5c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.500+0000 7fcb696ba640 1 -- 172.21.15.43:0/2019324977 shutdown_connections 2024-04-21T08:43:52.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.500+0000 7fcb696ba640 1 --2- 172.21.15.43:0/2019324977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb641072b0 0x7fcb6410ee80 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.500+0000 7fcb696ba640 1 --2- 172.21.15.43:0/2019324977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb64102b90 0x7fcb64106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.500+0000 7fcb696ba640 1 --2- 172.21.15.43:0/2019324977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb641021e0 0x7fcb641025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.500+0000 7fcb696ba640 1 -- 172.21.15.43:0/2019324977 >> 172.21.15.43:0/2019324977 conn(0x7fcb640fda60 msgr2=0x7fcb640ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.500+0000 7fcb696ba640 1 -- 172.21.15.43:0/2019324977 shutdown_connections 2024-04-21T08:43:52.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.500+0000 7fcb696ba640 1 -- 172.21.15.43:0/2019324977 wait complete. 2024-04-21T08:43:52.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.501+0000 7fcb696ba640 1 Processor -- start 2024-04-21T08:43:52.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.501+0000 7fcb696ba640 1 -- start start 2024-04-21T08:43:52.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.501+0000 7fcb696ba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb641021e0 0x7fcb6419e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.501+0000 7fcb696ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb641072b0 0x7fcb6419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.501+0000 7fcb696ba640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb641985c0 0x7fcb64198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.501+0000 7fcb696ba640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcb64112220 con 0x7fcb641021e0 2024-04-21T08:43:52.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.501+0000 7fcb696ba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb641120a0 con 0x7fcb641072b0 2024-04-21T08:43:52.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.501+0000 7fcb696ba640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcb641123a0 con 0x7fcb641985c0 2024-04-21T08:43:52.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.501+0000 7fcb62ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb641021e0 0x7fcb6419e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.501+0000 7fcb62ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb641021e0 0x7fcb6419e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37156/0 (socket says 172.21.15.43:37156) 2024-04-21T08:43:52.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.501+0000 7fcb62ffd640 1 -- 172.21.15.43:0/2805708525 learned_addr learned my addr 172.21.15.43:0/2805708525 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.502+0000 7fcb627fc640 1 --2- 172.21.15.43:0/2805708525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb641072b0 0x7fcb6419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.502+0000 7fcb637fe640 1 --2- 172.21.15.43:0/2805708525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb641985c0 0x7fcb64198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.502+0000 7fcb62ffd640 1 -- 172.21.15.43:0/2805708525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb641985c0 msgr2=0x7fcb64198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.502+0000 7fcb62ffd640 1 --2- 172.21.15.43:0/2805708525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb641985c0 0x7fcb64198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.502+0000 7fcb62ffd640 1 -- 172.21.15.43:0/2805708525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb641072b0 msgr2=0x7fcb6419e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.502+0000 7fcb62ffd640 1 --2- 172.21.15.43:0/2805708525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb641072b0 0x7fcb6419e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.502+0000 7fcb62ffd640 1 -- 172.21.15.43:0/2805708525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb641990e0 con 0x7fcb641021e0 2024-04-21T08:43:52.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.502+0000 7fcb627fc640 1 --2- 172.21.15.43:0/2805708525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb641072b0 0x7fcb6419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:52.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.502+0000 7fcb62ffd640 1 --2- 172.21.15.43:0/2805708525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb641021e0 0x7fcb6419e410 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400d950 tx=0x7fcb5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.502+0000 7fcb4bfff640 1 -- 172.21.15.43:0/2805708525 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54017070 con 0x7fcb641021e0 2024-04-21T08:43:52.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.502+0000 7fcb4bfff640 1 -- 172.21.15.43:0/2805708525 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb54009c30 con 0x7fcb641021e0 2024-04-21T08:43:52.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.502+0000 7fcb696ba640 1 -- 172.21.15.43:0/2805708525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb641993d0 con 0x7fcb641021e0 2024-04-21T08:43:52.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.502+0000 7fcb4bfff640 1 -- 172.21.15.43:0/2805708525 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5400f620 con 0x7fcb641021e0 2024-04-21T08:43:52.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.502+0000 7fcb696ba640 1 -- 172.21.15.43:0/2805708525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb641a5620 con 0x7fcb641021e0 2024-04-21T08:43:52.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.505+0000 7fcb4bfff640 1 -- 172.21.15.43:0/2805708525 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcb54010460 con 0x7fcb641021e0 2024-04-21T08:43:52.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.505+0000 7fcb6c384740 1 Processor -- start 2024-04-21T08:43:52.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.505+0000 7fcb6c384740 1 -- start start 2024-04-21T08:43:52.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.505+0000 7fcb6c384740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55631cc9f3b0 0x55631cc9f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.505+0000 7fcb6c384740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cc9fd60 0x55631cca8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.505+0000 7fcb6c384740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55631cca8e60 0x55631cca92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.505+0000 7fcb6c384740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55631cc041f0 con 0x55631cc9fd60 2024-04-21T08:43:52.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.505+0000 7fcb6c384740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55631cba14b0 con 0x55631cca8e60 2024-04-21T08:43:52.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.505+0000 7fcb6c384740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55631cc001b0 con 0x55631cc9f3b0 2024-04-21T08:43:52.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.505+0000 7fcb62ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cc9fd60 0x55631cca8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.505+0000 7fcb62ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cc9fd60 0x55631cca8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37162/0 (socket says 172.21.15.43:37162) 2024-04-21T08:43:52.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.505+0000 7fcb62ffd640 1 -- 172.21.15.43:0/3365985667 learned_addr learned my addr 172.21.15.43:0/3365985667 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.505+0000 7fcb627fc640 1 --2- 172.21.15.43:0/3365985667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55631cca8e60 0x55631cca92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.505+0000 7fcb4bfff640 1 --2- 172.21.15.43:0/2805708525 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcb4c0a3d20 0x7fcb4c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.505+0000 7fcb637fe640 1 --2- 172.21.15.43:0/3365985667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55631cc9f3b0 0x55631cc9f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.505+0000 7fcb637fe640 1 --2- 172.21.15.43:0/2805708525 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcb4c0a3d20 0x7fcb4c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.506+0000 7fcb4bfff640 1 -- 172.21.15.43:0/2805708525 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fcb540d08c0 con 0x7fcb641021e0 2024-04-21T08:43:52.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.506+0000 7fcb62ffd640 1 -- 172.21.15.43:0/3365985667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55631cc9f3b0 msgr2=0x55631cc9f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.506+0000 7fcb62ffd640 1 --2- 172.21.15.43:0/3365985667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55631cc9f3b0 0x55631cc9f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.506+0000 7fcb62ffd640 1 -- 172.21.15.43:0/3365985667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55631cca8e60 msgr2=0x55631cca92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.506+0000 7fcb62ffd640 1 --2- 172.21.15.43:0/3365985667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55631cca8e60 0x55631cca92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.506+0000 7fcb62ffd640 1 -- 172.21.15.43:0/3365985667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55631cb931b0 con 0x55631cc9fd60 2024-04-21T08:43:52.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.506+0000 7fcb627fc640 1 --2- 172.21.15.43:0/3365985667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55631cca8e60 0x55631cca92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:43:52.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.506+0000 7fcb637fe640 1 --2- 172.21.15.43:0/3365985667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55631cc9f3b0 0x55631cc9f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:52.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.506+0000 7fcb62ffd640 1 --2- 172.21.15.43:0/3365985667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cc9fd60 0x55631cca8890 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fcb540cdbe0 tx=0x7fcb54005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7868dc6c799355 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.506+0000 7fcb497fa640 1 -- 172.21.15.43:0/3365985667 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54020070 con 0x55631cc9fd60 2024-04-21T08:43:52.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.506+0000 7fcb637fe640 1 --2- 172.21.15.43:0/2805708525 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcb4c0a3d20 0x7fcb4c0a61e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c004af0 tx=0x7fcb5c0095c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.506+0000 7fcb497fa640 1 -- 172.21.15.43:0/3365985667 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb54009de0 con 0x55631cc9fd60 2024-04-21T08:43:52.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.506+0000 7fcb497fa640 1 -- 172.21.15.43:0/3365985667 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb540ce020 con 0x55631cc9fd60 2024-04-21T08:43:52.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.507+0000 7fcb6c384740 1 -- 172.21.15.43:0/3365985667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cc9fd60 msgr2=0x55631cca8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.507+0000 7fcb6c384740 1 --2- 172.21.15.43:0/3365985667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cc9fd60 0x55631cca8890 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fcb540cdbe0 tx=0x7fcb54005880 comp rx=0 tx=0).stop 2024-04-21T08:43:52.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.507+0000 7fcb6c384740 1 -- 172.21.15.43:0/3365985667 shutdown_connections 2024-04-21T08:43:52.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.507+0000 7fcb6c384740 1 --2- 172.21.15.43:0/3365985667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55631cca8e60 0x55631cca92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.507+0000 7fcb6c384740 1 --2- 172.21.15.43:0/3365985667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cc9fd60 0x55631cca8890 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.507+0000 7fcb6c384740 1 --2- 172.21.15.43:0/3365985667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55631cc9f3b0 0x55631cc9f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.507+0000 7fcb6c384740 1 -- 172.21.15.43:0/3365985667 >> 172.21.15.43:0/3365985667 conn(0x55631cc99ba0 msgr2=0x55631cc99f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.507+0000 7fcb6c384740 1 -- 172.21.15.43:0/3365985667 shutdown_connections 2024-04-21T08:43:52.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.507+0000 7fcb6c384740 1 -- 172.21.15.43:0/3365985667 wait complete. 2024-04-21T08:43:52.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.507+0000 7fcb6c384740 1 -- start start 2024-04-21T08:43:52.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.507+0000 7fcb6c384740 1 Processor -- start 2024-04-21T08:43:52.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.507+0000 7fcb6c384740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55631cc9f3b0 0x55631ccaf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.507+0000 7fcb6c384740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cc9fd60 0x55631ccac620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.507+0000 7fcb6c384740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55631cca8e60 0x55631ccacb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.507+0000 7fcb6c384740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55631ccafd40 con 0x55631cc9fd60 2024-04-21T08:43:52.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.507+0000 7fcb6c384740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55631cbbd500 con 0x55631cca8e60 2024-04-21T08:43:52.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.507+0000 7fcb6c384740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55631ccafec0 con 0x55631cc9f3b0 2024-04-21T08:43:52.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.507+0000 7fcb6c384740 10 client.0 _ll_register_callbacks cb 0x55631cb8bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:52.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.508+0000 7fcb627fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cc9fd60 0x55631ccac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.508+0000 7fcb627fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cc9fd60 0x55631ccac620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37166/0 (socket says 172.21.15.43:37166) 2024-04-21T08:43:52.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.508+0000 7fcb627fc640 1 -- 172.21.15.43:0/751909101 learned_addr learned my addr 172.21.15.43:0/751909101 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.508+0000 7fcb637fe640 1 --2- 172.21.15.43:0/751909101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55631cca8e60 0x55631ccacb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.508+0000 7fcb62ffd640 1 --2- 172.21.15.43:0/751909101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55631cc9f3b0 0x55631ccaf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.508+0000 7fcb627fc640 1 -- 172.21.15.43:0/751909101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55631cc9f3b0 msgr2=0x55631ccaf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.508+0000 7fcb627fc640 1 --2- 172.21.15.43:0/751909101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55631cc9f3b0 0x55631ccaf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.508+0000 7fcb627fc640 1 -- 172.21.15.43:0/751909101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55631cca8e60 msgr2=0x55631ccacb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.508+0000 7fcb627fc640 1 --2- 172.21.15.43:0/751909101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55631cca8e60 0x55631ccacb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.508+0000 7fcb627fc640 1 -- 172.21.15.43:0/751909101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55631c9163f0 con 0x55631cc9fd60 2024-04-21T08:43:52.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.508+0000 7fcb637fe640 1 --2- 172.21.15.43:0/751909101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55631cca8e60 0x55631ccacb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:52.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.508+0000 7fcb627fc640 1 --2- 172.21.15.43:0/751909101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cc9fd60 0x55631ccac620 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800bec0 tx=0x7fcb58012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191751e7afa7c074 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.508+0000 7fcb46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:52.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.508+0000 7fcb46ff5640 1 -- 172.21.15.43:0/751909101 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb58010960 con 0x55631cc9fd60 2024-04-21T08:43:52.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.508+0000 7fcb46ff5640 1 -- 172.21.15.43:0/751909101 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fcb58010b00 con 0x55631cc9fd60 2024-04-21T08:43:52.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.508+0000 7fcb6c384740 1 -- 172.21.15.43:0/751909101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb54009de0 con 0x55631cc9fd60 2024-04-21T08:43:52.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.508+0000 7fcb46ff5640 1 -- 172.21.15.43:0/751909101 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcb58010d00 con 0x55631cc9fd60 2024-04-21T08:43:52.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.508+0000 7fcb6c384740 10 client.17739 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:52.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.508+0000 7fcb6c384740 1 -- 172.21.15.43:0/751909101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55631c9e07d0 con 0x55631cc9fd60 2024-04-21T08:43:52.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.509+0000 7fcb46ff5640 1 -- 172.21.15.43:0/751909101 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb54009de0 con 0x55631cc9fd60 2024-04-21T08:43:52.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.509+0000 7fcb46ff5640 1 -- 172.21.15.43:0/751909101 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcb58019070 con 0x55631cc9fd60 2024-04-21T08:43:52.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.509+0000 7fcb6c384740 10 client.17739 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:52.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.509+0000 7fcb6c384740 10 client.17739 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:43:52.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.509+0000 7fcb6c384740 20 client.17739 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:52.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.509+0000 7fcb6c384740 1 --2- 172.21.15.43:0/751909101 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55631ccbd000 0x55631ccbf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.509+0000 7fcb6c384740 4 client.17739 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:52.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.509+0000 7fcb6c384740 1 -- 172.21.15.43:0/751909101 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55631cba3930 con 0x55631ccbd000 2024-04-21T08:43:52.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.509+0000 7fcb62ffd640 1 --2- 172.21.15.43:0/751909101 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55631ccbd000 0x55631ccbf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.510+0000 7fcb62ffd640 1 --2- 172.21.15.43:0/751909101 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55631ccbd000 0x55631ccbf440 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=736ae6bd90fb69e9 server_cookie=ebae87616fbdca92 in_seq=0 out_seq=0 2024-04-21T08:43:52.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.510+0000 7fcb46ff5640 10 client.17739 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:43:52.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.510+0000 7fcb46ff5640 1 -- 172.21.15.43:0/751909101 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55631cba3930 con 0x55631ccbd000 2024-04-21T08:43:52.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.510+0000 7fcb46ff5640 10 client.17739 handle_command_reply: tid=0 2024-04-21T08:43:52.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.510+0000 7fcb6c384740 1 client.17739 shutdown 2024-04-21T08:43:52.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.511+0000 7fcb6c384740 1 -- 172.21.15.43:0/751909101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cc9fd60 msgr2=0x55631ccac620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.511+0000 7fcb6c384740 1 --2- 172.21.15.43:0/751909101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cc9fd60 0x55631ccac620 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800bec0 tx=0x7fcb58012820 comp rx=0 tx=0).stop 2024-04-21T08:43:52.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.511+0000 7fcb6c384740 1 -- 172.21.15.43:0/751909101 shutdown_connections 2024-04-21T08:43:52.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.511+0000 7fcb6c384740 1 --2- 172.21.15.43:0/751909101 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55631ccbd000 0x55631ccbf440 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.511+0000 7fcb6c384740 1 --2- 172.21.15.43:0/751909101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55631cca8e60 0x55631ccacb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.511+0000 7fcb6c384740 1 --2- 172.21.15.43:0/751909101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cc9fd60 0x55631ccac620 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.511+0000 7fcb6c384740 1 --2- 172.21.15.43:0/751909101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55631cc9f3b0 0x55631ccaf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.511+0000 7fcb46ff5640 0 client.17739 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:43:52.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.511+0000 7fcb6c384740 1 -- 172.21.15.43:0/751909101 >> 172.21.15.43:0/751909101 conn(0x55631cc99ba0 msgr2=0x55631cc9b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.511+0000 7fcb6c384740 1 -- 172.21.15.43:0/751909101 shutdown_connections 2024-04-21T08:43:52.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.511+0000 7fcb6c384740 1 -- 172.21.15.43:0/751909101 wait complete. 2024-04-21T08:43:52.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.511+0000 7fcb6c384740 20 client.17739 trim_cache size 0 max 16384 2024-04-21T08:43:52.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.524+0000 7fcb6c384740 1 Processor -- start 2024-04-21T08:43:52.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.524+0000 7fcb6c384740 1 -- start start 2024-04-21T08:43:52.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.524+0000 7fcb6c384740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cc9f3b0 0x55631ccb8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.524+0000 7fcb6c384740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55631cc9fd60 0x55631cca8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.524+0000 7fcb6c384740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55631cca8e60 0x55631cc9b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.524+0000 7fcb6c384740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcb641ad320 con 0x55631cc9f3b0 2024-04-21T08:43:52.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.524+0000 7fcb6c384740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb641aee60 con 0x55631cca8e60 2024-04-21T08:43:52.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.524+0000 7fcb6c384740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcb641ad9f0 con 0x55631cc9fd60 2024-04-21T08:43:52.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.524+0000 7fcb62ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cc9f3b0 0x55631ccb8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.524+0000 7fcb62ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cc9f3b0 0x55631ccb8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37180/0 (socket says 172.21.15.43:37180) 2024-04-21T08:43:52.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.524+0000 7fcb62ffd640 1 -- 172.21.15.43:0/1218124684 learned_addr learned my addr 172.21.15.43:0/1218124684 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.524+0000 7fcb627fc640 1 --2- 172.21.15.43:0/1218124684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55631cc9fd60 0x55631cca8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.524+0000 7fcb637fe640 1 --2- 172.21.15.43:0/1218124684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55631cca8e60 0x55631cc9b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.525+0000 7fcb62ffd640 1 -- 172.21.15.43:0/1218124684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55631cc9fd60 msgr2=0x55631cca8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.525+0000 7fcb62ffd640 1 --2- 172.21.15.43:0/1218124684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55631cc9fd60 0x55631cca8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.525+0000 7fcb62ffd640 1 -- 172.21.15.43:0/1218124684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55631cca8e60 msgr2=0x55631cc9b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.525+0000 7fcb62ffd640 1 --2- 172.21.15.43:0/1218124684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55631cca8e60 0x55631cc9b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.525+0000 7fcb62ffd640 1 -- 172.21.15.43:0/1218124684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55631cc9e560 con 0x55631cc9f3b0 2024-04-21T08:43:52.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.525+0000 7fcb637fe640 1 --2- 172.21.15.43:0/1218124684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55631cca8e60 0x55631cc9b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:52.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.525+0000 7fcb627fc640 1 --2- 172.21.15.43:0/1218124684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55631cc9fd60 0x55631cca8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:52.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.525+0000 7fcb62ffd640 1 --2- 172.21.15.43:0/1218124684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cc9f3b0 0x55631ccb8c40 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fcb5409c3d0 tx=0x7fcb54010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c956bc13991498 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.525+0000 7fcb48ff9640 1 -- 172.21.15.43:0/1218124684 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54020040 con 0x55631cc9f3b0 2024-04-21T08:43:52.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.525+0000 7fcb48ff9640 1 -- 172.21.15.43:0/1218124684 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb54009de0 con 0x55631cc9f3b0 2024-04-21T08:43:52.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.525+0000 7fcb48ff9640 1 -- 172.21.15.43:0/1218124684 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb540981e0 con 0x55631cc9f3b0 2024-04-21T08:43:52.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.525+0000 7fcb6c384740 1 -- 172.21.15.43:0/1218124684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cc9f3b0 msgr2=0x55631ccb8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.525+0000 7fcb6c384740 1 --2- 172.21.15.43:0/1218124684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cc9f3b0 0x55631ccb8c40 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fcb5409c3d0 tx=0x7fcb54010cb0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.525+0000 7fcb6c384740 1 -- 172.21.15.43:0/1218124684 shutdown_connections 2024-04-21T08:43:52.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.525+0000 7fcb6c384740 1 --2- 172.21.15.43:0/1218124684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55631cca8e60 0x55631cc9b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.525+0000 7fcb6c384740 1 --2- 172.21.15.43:0/1218124684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55631cc9fd60 0x55631cca8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.525+0000 7fcb6c384740 1 --2- 172.21.15.43:0/1218124684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cc9f3b0 0x55631ccb8c40 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.525+0000 7fcb6c384740 1 -- 172.21.15.43:0/1218124684 >> 172.21.15.43:0/1218124684 conn(0x55631cc99ba0 msgr2=0x55631cc9afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.525+0000 7fcb6c384740 1 -- 172.21.15.43:0/1218124684 shutdown_connections 2024-04-21T08:43:52.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.526+0000 7fcb6c384740 1 -- 172.21.15.43:0/1218124684 wait complete. 2024-04-21T08:43:52.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.526+0000 7fcb6c384740 1 -- start start 2024-04-21T08:43:52.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.526+0000 7fcb6c384740 1 Processor -- start 2024-04-21T08:43:52.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.526+0000 7fcb6c384740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55631cc9f3b0 0x55631ccae320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.526+0000 7fcb6c384740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55631cc9fd60 0x55631ccae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.526+0000 7fcb6c384740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cca8e60 0x55631ccaeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.526+0000 7fcb6c384740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcb641ae790 con 0x55631cca8e60 2024-04-21T08:43:52.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.526+0000 7fcb6c384740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb641ae0c0 con 0x55631cc9fd60 2024-04-21T08:43:52.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.526+0000 7fcb6c384740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcb641ab300 con 0x55631cc9f3b0 2024-04-21T08:43:52.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.526+0000 7fcb6c384740 10 client.0 _ll_register_callbacks cb 0x7fcb5800bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:52.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.526+0000 7fcb637fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cca8e60 0x55631ccaeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.526+0000 7fcb637fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cca8e60 0x55631ccaeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37182/0 (socket says 172.21.15.43:37182) 2024-04-21T08:43:52.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.526+0000 7fcb637fe640 1 -- 172.21.15.43:0/3900544117 learned_addr learned my addr 172.21.15.43:0/3900544117 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.526+0000 7fcb62ffd640 1 --2- 172.21.15.43:0/3900544117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55631cc9f3b0 0x55631ccae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.526+0000 7fcb627fc640 1 --2- 172.21.15.43:0/3900544117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55631cc9fd60 0x55631ccae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.526+0000 7fcb637fe640 1 -- 172.21.15.43:0/3900544117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55631cc9f3b0 msgr2=0x55631ccae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.526+0000 7fcb637fe640 1 --2- 172.21.15.43:0/3900544117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55631cc9f3b0 0x55631ccae320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.526+0000 7fcb637fe640 1 -- 172.21.15.43:0/3900544117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55631cc9fd60 msgr2=0x55631ccae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.526+0000 7fcb637fe640 1 --2- 172.21.15.43:0/3900544117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55631cc9fd60 0x55631ccae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.526+0000 7fcb637fe640 1 -- 172.21.15.43:0/3900544117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55631ccb3500 con 0x55631cca8e60 2024-04-21T08:43:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.527+0000 7fcb637fe640 1 --2- 172.21.15.43:0/3900544117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cca8e60 0x55631ccaeda0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009ca0 tx=0x7fcb5c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f4f5d2a09c162b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.527+0000 7fcb46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.527+0000 7fcb46ff5640 1 -- 172.21.15.43:0/3900544117 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c039030 con 0x55631cca8e60 2024-04-21T08:43:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.527+0000 7fcb46ff5640 1 -- 172.21.15.43:0/3900544117 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fcb5c004240 con 0x55631cca8e60 2024-04-21T08:43:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.527+0000 7fcb6c384740 1 -- 172.21.15.43:0/3900544117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb54009de0 con 0x55631cca8e60 2024-04-21T08:43:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.527+0000 7fcb46ff5640 1 -- 172.21.15.43:0/3900544117 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcb5c004440 con 0x55631cca8e60 2024-04-21T08:43:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.527+0000 7fcb6c384740 10 client.17745 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.527+0000 7fcb6c384740 1 -- 172.21.15.43:0/3900544117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55631cd23900 con 0x55631cca8e60 2024-04-21T08:43:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.527+0000 7fcb46ff5640 1 -- 172.21.15.43:0/3900544117 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb54009de0 con 0x55631cca8e60 2024-04-21T08:43:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.527+0000 7fcb46ff5640 1 -- 172.21.15.43:0/3900544117 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcb5c03f070 con 0x55631cca8e60 2024-04-21T08:43:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.527+0000 7fcb6c384740 10 client.17745 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.527+0000 7fcb6c384740 10 client.17745 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:43:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.527+0000 7fcb6c384740 20 client.17745 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:52.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.527+0000 7fcb6c384740 1 --2- 172.21.15.43:0/3900544117 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55631cd24100 0x55631cd26520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.527+0000 7fcb6c384740 4 client.17745 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:43:52.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.527+0000 7fcb6c384740 1 -- 172.21.15.43:0/3900544117 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55631cc11c10 con 0x55631cd24100 2024-04-21T08:43:52.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.528+0000 7fcb62ffd640 1 --2- 172.21.15.43:0/3900544117 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55631cd24100 0x55631cd26520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.528+0000 7fcb62ffd640 1 --2- 172.21.15.43:0/3900544117 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55631cd24100 0x55631cd26520 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=74c37529c5761c28 server_cookie=e6a0119e134cb05f in_seq=0 out_seq=0 2024-04-21T08:43:52.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.528+0000 7fcb46ff5640 10 client.17745 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:43:52.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.528+0000 7fcb46ff5640 1 -- 172.21.15.43:0/3900544117 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55631cc11c10 con 0x55631cd24100 2024-04-21T08:43:52.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.528+0000 7fcb46ff5640 10 client.17745 handle_command_reply: tid=0 2024-04-21T08:43:52.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.528+0000 7fcb6c384740 1 client.17745 shutdown 2024-04-21T08:43:52.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.528+0000 7fcb6c384740 1 -- 172.21.15.43:0/3900544117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cca8e60 msgr2=0x55631ccaeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.528+0000 7fcb6c384740 1 --2- 172.21.15.43:0/3900544117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cca8e60 0x55631ccaeda0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009ca0 tx=0x7fcb5c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.528+0000 7fcb6c384740 1 -- 172.21.15.43:0/3900544117 shutdown_connections 2024-04-21T08:43:52.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.528+0000 7fcb6c384740 1 --2- 172.21.15.43:0/3900544117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55631cca8e60 0x55631ccaeda0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.528+0000 7fcb6c384740 1 --2- 172.21.15.43:0/3900544117 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55631cd24100 0x55631cd26520 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.528+0000 7fcb6c384740 1 --2- 172.21.15.43:0/3900544117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55631cc9fd60 0x55631ccae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.528+0000 7fcb46ff5640 0 client.17745 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:43:52.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.528+0000 7fcb6c384740 1 --2- 172.21.15.43:0/3900544117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55631cc9f3b0 0x55631ccae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.528+0000 7fcb6c384740 1 -- 172.21.15.43:0/3900544117 >> 172.21.15.43:0/3900544117 conn(0x55631cc99ba0 msgr2=0x55631cca84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.529+0000 7fcb6c384740 1 -- 172.21.15.43:0/3900544117 shutdown_connections 2024-04-21T08:43:52.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.529+0000 7fcb6c384740 1 -- 172.21.15.43:0/3900544117 wait complete. 2024-04-21T08:43:52.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.529+0000 7fcb6c384740 20 client.17745 trim_cache size 0 max 16384 2024-04-21T08:43:52.536 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:43:52.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.529+0000 7fcb696ba640 1 -- 172.21.15.43:0/2805708525 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcb4c0a3d20 msgr2=0x7fcb4c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.529+0000 7fcb696ba640 1 --2- 172.21.15.43:0/2805708525 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcb4c0a3d20 0x7fcb4c0a61e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c004af0 tx=0x7fcb5c0095c0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.529+0000 7fcb696ba640 1 -- 172.21.15.43:0/2805708525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb641021e0 msgr2=0x7fcb6419e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.529+0000 7fcb696ba640 1 --2- 172.21.15.43:0/2805708525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb641021e0 0x7fcb6419e410 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400d950 tx=0x7fcb5400de10 comp rx=0 tx=0).stop 2024-04-21T08:43:52.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.529+0000 7fcb696ba640 1 -- 172.21.15.43:0/2805708525 shutdown_connections 2024-04-21T08:43:52.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.529+0000 7fcb696ba640 1 --2- 172.21.15.43:0/2805708525 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcb4c0a3d20 0x7fcb4c0a61e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.529+0000 7fcb696ba640 1 --2- 172.21.15.43:0/2805708525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb641985c0 0x7fcb64198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.529+0000 7fcb696ba640 1 --2- 172.21.15.43:0/2805708525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb641072b0 0x7fcb6419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.529+0000 7fcb696ba640 1 --2- 172.21.15.43:0/2805708525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb641021e0 0x7fcb6419e410 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.529+0000 7fcb696ba640 1 -- 172.21.15.43:0/2805708525 >> 172.21.15.43:0/2805708525 conn(0x7fcb640fda60 msgr2=0x7fcb640ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.529+0000 7fcb696ba640 1 -- 172.21.15.43:0/2805708525 shutdown_connections 2024-04-21T08:43:52.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.529+0000 7fcb696ba640 1 -- 172.21.15.43:0/2805708525 wait complete. 2024-04-21T08:43:52.540 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:43:52.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:52 smithi112 ceph-mon[31085]: pgmap v151: 129 pgs: 129 active+clean; 2.5 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-21T08:43:52.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:52 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3908235331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:43:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.645+0000 7f81d147a640 1 -- 172.21.15.43:0/3507574043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81cc1010a0 msgr2=0x7f81cc101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.645+0000 7f81d147a640 1 --2- 172.21.15.43:0/3507574043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81cc1010a0 0x7f81cc101480 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f81bc009980 tx=0x7f81bc02de60 comp rx=0 tx=0).stop 2024-04-21T08:43:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.645+0000 7f81d147a640 1 -- 172.21.15.43:0/3507574043 shutdown_connections 2024-04-21T08:43:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.645+0000 7f81d147a640 1 --2- 172.21.15.43:0/3507574043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81cc106170 0x7f81cc10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.645+0000 7f81d147a640 1 --2- 172.21.15.43:0/3507574043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81cc101a50 0x7f81cc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.645+0000 7f81d147a640 1 --2- 172.21.15.43:0/3507574043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81cc1010a0 0x7f81cc101480 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.645+0000 7f81d147a640 1 -- 172.21.15.43:0/3507574043 >> 172.21.15.43:0/3507574043 conn(0x7f81cc077eb0 msgr2=0x7f81cc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.645+0000 7f81d147a640 1 -- 172.21.15.43:0/3507574043 shutdown_connections 2024-04-21T08:43:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.645+0000 7f81d147a640 1 -- 172.21.15.43:0/3507574043 wait complete. 2024-04-21T08:43:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.645+0000 7f81d147a640 1 Processor -- start 2024-04-21T08:43:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.645+0000 7f81d147a640 1 -- start start 2024-04-21T08:43:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.646+0000 7f81d147a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81cc1010a0 0x7f81cc1964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.646+0000 7f81d147a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81cc106170 0x7f81cc1969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.646+0000 7f81d147a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81cc19ad70 0x7f81cc19b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.646+0000 7f81d147a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc10ffd0 con 0x7f81cc1010a0 2024-04-21T08:43:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.646+0000 7f81d147a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc10fe50 con 0x7f81cc106170 2024-04-21T08:43:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.646+0000 7f81d147a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc110150 con 0x7f81cc19ad70 2024-04-21T08:43:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.646+0000 7f81caffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81cc1010a0 0x7f81cc1964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.646+0000 7f81caffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81cc1010a0 0x7f81cc1964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37208/0 (socket says 172.21.15.43:37208) 2024-04-21T08:43:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.646+0000 7f81caffd640 1 -- 172.21.15.43:0/3472769610 learned_addr learned my addr 172.21.15.43:0/3472769610 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.646+0000 7f81cb7fe640 1 --2- 172.21.15.43:0/3472769610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81cc19ad70 0x7f81cc19b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.646+0000 7f81ca7fc640 1 --2- 172.21.15.43:0/3472769610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81cc106170 0x7f81cc1969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.646+0000 7f81caffd640 1 -- 172.21.15.43:0/3472769610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81cc19ad70 msgr2=0x7f81cc19b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.646+0000 7f81caffd640 1 --2- 172.21.15.43:0/3472769610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81cc19ad70 0x7f81cc19b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.646+0000 7f81caffd640 1 -- 172.21.15.43:0/3472769610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81cc106170 msgr2=0x7f81cc1969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.646+0000 7f81caffd640 1 --2- 172.21.15.43:0/3472769610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81cc106170 0x7f81cc1969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.646+0000 7f81caffd640 1 -- 172.21.15.43:0/3472769610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81cc19b860 con 0x7f81cc1010a0 2024-04-21T08:43:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.646+0000 7f81caffd640 1 --2- 172.21.15.43:0/3472769610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81cc1010a0 0x7f81cc1964b0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f81bc006010 tx=0x7f81bc004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.647+0000 7f81b3fff640 1 -- 172.21.15.43:0/3472769610 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc044020 con 0x7f81cc1010a0 2024-04-21T08:43:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.647+0000 7f81d147a640 1 -- 172.21.15.43:0/3472769610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81cc19baf0 con 0x7f81cc1010a0 2024-04-21T08:43:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.647+0000 7f81b3fff640 1 -- 172.21.15.43:0/3472769610 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81bc036520 con 0x7f81cc1010a0 2024-04-21T08:43:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.647+0000 7f81b3fff640 1 -- 172.21.15.43:0/3472769610 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc0406c0 con 0x7f81cc1010a0 2024-04-21T08:43:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.647+0000 7f81d147a640 1 -- 172.21.15.43:0/3472769610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81cc1a3310 con 0x7f81cc1010a0 2024-04-21T08:43:52.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.649+0000 7f81b3fff640 1 -- 172.21.15.43:0/3472769610 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f81bc047020 con 0x7f81cc1010a0 2024-04-21T08:43:52.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.649+0000 7f81d406a740 1 Processor -- start 2024-04-21T08:43:52.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.649+0000 7f81d406a740 1 -- start start 2024-04-21T08:43:52.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.650+0000 7f81d406a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc84d0 0x561fc7bc88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.650+0000 7f81d406a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fc7bc8e80 0x561fc7bd1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.650+0000 7f81d406a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fc7bd1a00 0x561fc7bd3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.650+0000 7f81d406a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561fc7b2d1f0 con 0x561fc7bc84d0 2024-04-21T08:43:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.650+0000 7f81b3fff640 1 --2- 172.21.15.43:0/3472769610 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f81b40a3d20 0x7f81b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.650+0000 7f81d406a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561fc7aca4b0 con 0x561fc7bd1a00 2024-04-21T08:43:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.650+0000 7f81d406a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561fc7b291b0 con 0x561fc7bc8e80 2024-04-21T08:43:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.650+0000 7f81cb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc84d0 0x561fc7bc88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.650+0000 7f81cb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc84d0 0x561fc7bc88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37212/0 (socket says 172.21.15.43:37212) 2024-04-21T08:43:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.650+0000 7f81cb7fe640 1 -- 172.21.15.43:0/286161523 learned_addr learned my addr 172.21.15.43:0/286161523 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.650+0000 7f81cb7fe640 1 --2- 172.21.15.43:0/3472769610 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f81b40a3d20 0x7f81b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.650+0000 7f81b3fff640 1 -- 172.21.15.43:0/3472769610 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f81bc0f4da0 con 0x7f81cc1010a0 2024-04-21T08:43:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.650+0000 7f81ca7fc640 1 --2- 172.21.15.43:0/286161523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fc7bd1a00 0x561fc7bd3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.650+0000 7f81cb7fe640 1 -- 172.21.15.43:0/286161523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fc7bc8e80 msgr2=0x561fc7bd1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.650+0000 7f81cb7fe640 1 --2- 172.21.15.43:0/286161523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fc7bc8e80 0x561fc7bd1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.650+0000 7f81cb7fe640 1 -- 172.21.15.43:0/286161523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fc7bd1a00 msgr2=0x561fc7bd3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.650+0000 7f81cb7fe640 1 --2- 172.21.15.43:0/286161523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fc7bd1a00 0x561fc7bd3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.650+0000 7f81cb7fe640 1 -- 172.21.15.43:0/286161523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fc7abc1b0 con 0x561fc7bc84d0 2024-04-21T08:43:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.650+0000 7f81cb7fe640 1 --2- 172.21.15.43:0/286161523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc84d0 0x561fc7bc88b0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f81c4014e80 tx=0x7f81c40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62b46282d6c7dcb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.650+0000 7f81cb7fe640 1 --2- 172.21.15.43:0/3472769610 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f81b40a3d20 0x7f81b40a61e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f81c4011cb0 tx=0x7f81c4016000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.650+0000 7f81b17fa640 1 -- 172.21.15.43:0/286161523 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c4024070 con 0x561fc7bc84d0 2024-04-21T08:43:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.651+0000 7f81b17fa640 1 -- 172.21.15.43:0/286161523 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81c4007c60 con 0x561fc7bc84d0 2024-04-21T08:43:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.651+0000 7f81b17fa640 1 -- 172.21.15.43:0/286161523 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c4005770 con 0x561fc7bc84d0 2024-04-21T08:43:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.651+0000 7f81d406a740 1 -- 172.21.15.43:0/286161523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc84d0 msgr2=0x561fc7bc88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.651+0000 7f81d406a740 1 --2- 172.21.15.43:0/286161523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc84d0 0x561fc7bc88b0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f81c4014e80 tx=0x7f81c40117b0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.651+0000 7f81d406a740 1 -- 172.21.15.43:0/286161523 shutdown_connections 2024-04-21T08:43:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.651+0000 7f81d406a740 1 --2- 172.21.15.43:0/286161523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fc7bd1a00 0x561fc7bd3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.651+0000 7f81d406a740 1 --2- 172.21.15.43:0/286161523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fc7bc8e80 0x561fc7bd1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.651+0000 7f81d406a740 1 --2- 172.21.15.43:0/286161523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc84d0 0x561fc7bc88b0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.651+0000 7f81d406a740 1 -- 172.21.15.43:0/286161523 >> 172.21.15.43:0/286161523 conn(0x561fc7bc2ba0 msgr2=0x561fc7bc2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.651+0000 7f81d406a740 1 -- 172.21.15.43:0/286161523 shutdown_connections 2024-04-21T08:43:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.651+0000 7f81d406a740 1 -- 172.21.15.43:0/286161523 wait complete. 2024-04-21T08:43:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.652+0000 7f81d406a740 1 -- start start 2024-04-21T08:43:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.652+0000 7f81d406a740 1 Processor -- start 2024-04-21T08:43:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.652+0000 7f81d406a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fc7bc84d0 0x561fc7bd1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.652+0000 7f81d406a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc8e80 0x561fc7bce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.652+0000 7f81d406a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fc7bd1a00 0x561fc7bcedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.652+0000 7f81d406a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561fc7bd4910 con 0x561fc7bc8e80 2024-04-21T08:43:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.652+0000 7f81d406a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561fc7ae6500 con 0x561fc7bc84d0 2024-04-21T08:43:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.652+0000 7f81d406a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561fc7bd4a90 con 0x561fc7bd1a00 2024-04-21T08:43:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.652+0000 7f81d406a740 10 client.0 _ll_register_callbacks cb 0x561fc7ab4b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.652+0000 7f81ca7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc8e80 0x561fc7bce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.652+0000 7f81ca7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc8e80 0x561fc7bce750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37222/0 (socket says 172.21.15.43:37222) 2024-04-21T08:43:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.652+0000 7f81ca7fc640 1 -- 172.21.15.43:0/2641798292 learned_addr learned my addr 172.21.15.43:0/2641798292 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.652+0000 7f81caffd640 1 --2- 172.21.15.43:0/2641798292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fc7bc84d0 0x561fc7bd1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.652+0000 7f81ca7fc640 1 -- 172.21.15.43:0/2641798292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fc7bd1a00 msgr2=0x561fc7bcedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.652+0000 7f81ca7fc640 1 --2- 172.21.15.43:0/2641798292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fc7bd1a00 0x561fc7bcedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.652+0000 7f81ca7fc640 1 -- 172.21.15.43:0/2641798292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fc7bc84d0 msgr2=0x561fc7bd1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.652+0000 7f81ca7fc640 1 --2- 172.21.15.43:0/2641798292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fc7bc84d0 0x561fc7bd1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.652+0000 7f81ca7fc640 1 -- 172.21.15.43:0/2641798292 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fc783f3f0 con 0x561fc7bc8e80 2024-04-21T08:43:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.653+0000 7f81ca7fc640 1 --2- 172.21.15.43:0/2641798292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc8e80 0x561fc7bce750 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f81c000b990 tx=0x7f81c000bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39cc0f9a20f950bf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.653+0000 7f81aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.653+0000 7f81aeff5640 1 -- 172.21.15.43:0/2641798292 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0003e20 con 0x561fc7bc8e80 2024-04-21T08:43:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.653+0000 7f81aeff5640 1 -- 172.21.15.43:0/2641798292 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f81c0005020 con 0x561fc7bc8e80 2024-04-21T08:43:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.653+0000 7f81d406a740 1 -- 172.21.15.43:0/2641798292 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81c4007c60 con 0x561fc7bc8e80 2024-04-21T08:43:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.653+0000 7f81aeff5640 1 -- 172.21.15.43:0/2641798292 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f81c0005220 con 0x561fc7bc8e80 2024-04-21T08:43:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.653+0000 7f81d406a740 10 client.17757 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.653+0000 7f81d406a740 1 -- 172.21.15.43:0/2641798292 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561fc79097d0 con 0x561fc7bc8e80 2024-04-21T08:43:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.653+0000 7f81aeff5640 1 -- 172.21.15.43:0/2641798292 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81c4007c60 con 0x561fc7bc8e80 2024-04-21T08:43:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.653+0000 7f81aeff5640 1 -- 172.21.15.43:0/2641798292 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f81c001b470 con 0x561fc7bc8e80 2024-04-21T08:43:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.653+0000 7f81d406a740 10 client.17757 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.653+0000 7f81d406a740 10 client.17757 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:43:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.653+0000 7f81d406a740 20 client.17757 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.653+0000 7f81d406a740 1 --2- 172.21.15.43:0/2641798292 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x561fc7be1d60 0x561fc7be4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.653+0000 7f81d406a740 4 client.17757 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.653+0000 7f81d406a740 1 -- 172.21.15.43:0/2641798292 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fc7acc930 con 0x561fc7be1d60 2024-04-21T08:43:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.654+0000 7f81caffd640 1 --2- 172.21.15.43:0/2641798292 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x561fc7be1d60 0x561fc7be4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.654+0000 7f81caffd640 1 --2- 172.21.15.43:0/2641798292 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x561fc7be1d60 0x561fc7be4190 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d692076c40061773 server_cookie=be3f0b494102b015 in_seq=0 out_seq=0 2024-04-21T08:43:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.654+0000 7f81aeff5640 10 client.17757 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:43:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.655+0000 7f81aeff5640 1 -- 172.21.15.43:0/2641798292 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561fc7acc930 con 0x561fc7be1d60 2024-04-21T08:43:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.655+0000 7f81aeff5640 10 client.17757 handle_command_reply: tid=0 2024-04-21T08:43:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.655+0000 7f81d406a740 1 client.17757 shutdown 2024-04-21T08:43:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.655+0000 7f81d406a740 1 -- 172.21.15.43:0/2641798292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc8e80 msgr2=0x561fc7bce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.655+0000 7f81d406a740 1 --2- 172.21.15.43:0/2641798292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc8e80 0x561fc7bce750 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f81c000b990 tx=0x7f81c000bd90 comp rx=0 tx=0).stop 2024-04-21T08:43:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.655+0000 7f81d406a740 1 -- 172.21.15.43:0/2641798292 shutdown_connections 2024-04-21T08:43:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.655+0000 7f81d406a740 1 --2- 172.21.15.43:0/2641798292 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x561fc7be1d60 0x561fc7be4190 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.655+0000 7f81d406a740 1 --2- 172.21.15.43:0/2641798292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fc7bd1a00 0x561fc7bcedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.655+0000 7f81aeff5640 0 client.17757 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:43:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.655+0000 7f81d406a740 1 --2- 172.21.15.43:0/2641798292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc8e80 0x561fc7bce750 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.655+0000 7f81d406a740 1 --2- 172.21.15.43:0/2641798292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fc7bc84d0 0x561fc7bd1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.655+0000 7f81d406a740 1 -- 172.21.15.43:0/2641798292 >> 172.21.15.43:0/2641798292 conn(0x561fc7bc2ba0 msgr2=0x561fc7bc48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.655+0000 7f81d406a740 1 -- 172.21.15.43:0/2641798292 shutdown_connections 2024-04-21T08:43:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.655+0000 7f81d406a740 1 -- 172.21.15.43:0/2641798292 wait complete. 2024-04-21T08:43:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.655+0000 7f81d406a740 20 client.17757 trim_cache size 0 max 16384 2024-04-21T08:43:52.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.669+0000 7f81d406a740 1 Processor -- start 2024-04-21T08:43:52.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.669+0000 7f81d406a740 1 -- start start 2024-04-21T08:43:52.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.669+0000 7f81d406a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc84d0 0x561fc7bdd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.669+0000 7f81d406a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fc7bc8e80 0x561fc7bce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.669+0000 7f81d406a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fc7bd1a00 0x561fc7bda0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.669+0000 7f81d406a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc1ad0f0 con 0x561fc7bc84d0 2024-04-21T08:43:52.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.669+0000 7f81d406a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc1aec30 con 0x561fc7bd1a00 2024-04-21T08:43:52.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.669+0000 7f81d406a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc1ad7c0 con 0x561fc7bc8e80 2024-04-21T08:43:52.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.669+0000 7f81caffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc84d0 0x561fc7bdd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.669+0000 7f81caffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc84d0 0x561fc7bdd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37234/0 (socket says 172.21.15.43:37234) 2024-04-21T08:43:52.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.669+0000 7f81caffd640 1 -- 172.21.15.43:0/2243895470 learned_addr learned my addr 172.21.15.43:0/2243895470 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.669+0000 7f81cb7fe640 1 --2- 172.21.15.43:0/2243895470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fc7bd1a00 0x561fc7bda0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.669+0000 7f81ca7fc640 1 --2- 172.21.15.43:0/2243895470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fc7bc8e80 0x561fc7bce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.669+0000 7f81caffd640 1 -- 172.21.15.43:0/2243895470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fc7bc8e80 msgr2=0x561fc7bce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.669+0000 7f81caffd640 1 --2- 172.21.15.43:0/2243895470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fc7bc8e80 0x561fc7bce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.669+0000 7f81caffd640 1 -- 172.21.15.43:0/2243895470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fc7bd1a00 msgr2=0x561fc7bda0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.669+0000 7f81caffd640 1 --2- 172.21.15.43:0/2243895470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fc7bd1a00 0x561fc7bda0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.669+0000 7f81caffd640 1 -- 172.21.15.43:0/2243895470 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fc7bbe470 con 0x561fc7bc84d0 2024-04-21T08:43:52.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.669+0000 7f81ca7fc640 1 --2- 172.21.15.43:0/2243895470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fc7bc8e80 0x561fc7bce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:52.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.669+0000 7f81cb7fe640 1 --2- 172.21.15.43:0/2243895470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fc7bd1a00 0x561fc7bda0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:52.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.669+0000 7f81caffd640 1 --2- 172.21.15.43:0/2243895470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc84d0 0x561fc7bdd7d0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0049c0 tx=0x7f81bc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8210e6e13f0a30ff server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.669+0000 7f81b0ff9640 1 -- 172.21.15.43:0/2243895470 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc0c4dd0 con 0x561fc7bc84d0 2024-04-21T08:43:52.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.670+0000 7f81b0ff9640 1 -- 172.21.15.43:0/2243895470 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81bc0c9690 con 0x561fc7bc84d0 2024-04-21T08:43:52.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.670+0000 7f81b0ff9640 1 -- 172.21.15.43:0/2243895470 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc0369d0 con 0x561fc7bc84d0 2024-04-21T08:43:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.670+0000 7f81d406a740 1 -- 172.21.15.43:0/2243895470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc84d0 msgr2=0x561fc7bdd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.670+0000 7f81d406a740 1 --2- 172.21.15.43:0/2243895470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc84d0 0x561fc7bdd7d0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0049c0 tx=0x7f81bc000bc0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.670+0000 7f81d406a740 1 -- 172.21.15.43:0/2243895470 shutdown_connections 2024-04-21T08:43:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.670+0000 7f81d406a740 1 --2- 172.21.15.43:0/2243895470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fc7bd1a00 0x561fc7bda0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.670+0000 7f81d406a740 1 --2- 172.21.15.43:0/2243895470 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fc7bc8e80 0x561fc7bce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.670+0000 7f81d406a740 1 --2- 172.21.15.43:0/2243895470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc84d0 0x561fc7bdd7d0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.670+0000 7f81d406a740 1 -- 172.21.15.43:0/2243895470 >> 172.21.15.43:0/2243895470 conn(0x561fc7bc2ba0 msgr2=0x561fc7bc4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.670+0000 7f81d406a740 1 -- 172.21.15.43:0/2243895470 shutdown_connections 2024-04-21T08:43:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.670+0000 7f81d406a740 1 -- 172.21.15.43:0/2243895470 wait complete. 2024-04-21T08:43:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.670+0000 7f81d406a740 1 -- start start 2024-04-21T08:43:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.670+0000 7f81d406a740 1 Processor -- start 2024-04-21T08:43:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.671+0000 7f81d406a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc84d0 0x561fc7be9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.671+0000 7f81d406a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fc7bc8e80 0x561fc7be9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.671+0000 7f81d406a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fc7bd1a00 0x561fc7be9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.671+0000 7f81d406a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc1ae560 con 0x561fc7bc84d0 2024-04-21T08:43:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.671+0000 7f81d406a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc1ade90 con 0x561fc7bd1a00 2024-04-21T08:43:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.671+0000 7f81d406a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc1ab0d0 con 0x561fc7bc8e80 2024-04-21T08:43:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.671+0000 7f81d406a740 10 client.0 _ll_register_callbacks cb 0x7f81c0004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.671+0000 7f81caffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc84d0 0x561fc7be9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.671+0000 7f81caffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc84d0 0x561fc7be9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37242/0 (socket says 172.21.15.43:37242) 2024-04-21T08:43:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.671+0000 7f81caffd640 1 -- 172.21.15.43:0/579105412 learned_addr learned my addr 172.21.15.43:0/579105412 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.671+0000 7f81ca7fc640 1 --2- 172.21.15.43:0/579105412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fc7bc8e80 0x561fc7be9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.671+0000 7f81cb7fe640 1 --2- 172.21.15.43:0/579105412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fc7bd1a00 0x561fc7be9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.671+0000 7f81caffd640 1 -- 172.21.15.43:0/579105412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fc7bc8e80 msgr2=0x561fc7be9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.671+0000 7f81caffd640 1 --2- 172.21.15.43:0/579105412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fc7bc8e80 0x561fc7be9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.671+0000 7f81caffd640 1 -- 172.21.15.43:0/579105412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fc7bd1a00 msgr2=0x561fc7be9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.671+0000 7f81caffd640 1 --2- 172.21.15.43:0/579105412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fc7bd1a00 0x561fc7be9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.671+0000 7f81caffd640 1 -- 172.21.15.43:0/579105412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fc7c4b340 con 0x561fc7bc84d0 2024-04-21T08:43:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.671+0000 7f81ca7fc640 1 --2- 172.21.15.43:0/579105412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fc7bc8e80 0x561fc7be9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.671+0000 7f81cb7fe640 1 --2- 172.21.15.43:0/579105412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fc7bd1a00 0x561fc7be9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.671+0000 7f81caffd640 1 --2- 172.21.15.43:0/579105412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc84d0 0x561fc7be9430 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0c9160 tx=0x7f81bc0366d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8a1206d327c5a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.671+0000 7f81aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.671+0000 7f81aeff5640 1 -- 172.21.15.43:0/579105412 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc0369d0 con 0x561fc7bc84d0 2024-04-21T08:43:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.671+0000 7f81aeff5640 1 -- 172.21.15.43:0/579105412 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f81bc0df850 con 0x561fc7bc84d0 2024-04-21T08:43:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.671+0000 7f81d406a740 1 -- 172.21.15.43:0/579105412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81bc0c9690 con 0x561fc7bc84d0 2024-04-21T08:43:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.672+0000 7f81aeff5640 1 -- 172.21.15.43:0/579105412 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f81bc036b70 con 0x561fc7bc84d0 2024-04-21T08:43:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.672+0000 7f81d406a740 10 client.17763 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.672+0000 7f81d406a740 1 -- 172.21.15.43:0/579105412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561fc7c4c510 con 0x561fc7bc84d0 2024-04-21T08:43:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.672+0000 7f81aeff5640 1 -- 172.21.15.43:0/579105412 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81bc0c9690 con 0x561fc7bc84d0 2024-04-21T08:43:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.672+0000 7f81aeff5640 1 -- 172.21.15.43:0/579105412 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f81bc0c0d10 con 0x561fc7bc84d0 2024-04-21T08:43:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.672+0000 7f81d406a740 10 client.17763 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.672+0000 7f81d406a740 10 client.17763 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:43:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.672+0000 7f81d406a740 20 client.17763 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.672+0000 7f81d406a740 1 --2- 172.21.15.43:0/579105412 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x561fc7be1d60 0x561fc7c4e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.672+0000 7f81d406a740 4 client.17763 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:43:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.672+0000 7f81d406a740 1 -- 172.21.15.43:0/579105412 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561fc7b3ac10 con 0x561fc7be1d60 2024-04-21T08:43:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.672+0000 7f81ca7fc640 1 --2- 172.21.15.43:0/579105412 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x561fc7be1d60 0x561fc7c4e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.672+0000 7f81ca7fc640 1 --2- 172.21.15.43:0/579105412 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x561fc7be1d60 0x561fc7c4e9e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc4a3047642a6017 server_cookie=132f6e35511b914c in_seq=0 out_seq=0 2024-04-21T08:43:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.672+0000 7f81aeff5640 10 client.17763 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:43:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81aeff5640 1 -- 172.21.15.43:0/579105412 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561fc7b3ac10 con 0x561fc7be1d60 2024-04-21T08:43:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81aeff5640 10 client.17763 handle_command_reply: tid=0 2024-04-21T08:43:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81d406a740 1 client.17763 shutdown 2024-04-21T08:43:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81d406a740 1 -- 172.21.15.43:0/579105412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc84d0 msgr2=0x561fc7be9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81d406a740 1 --2- 172.21.15.43:0/579105412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc84d0 0x561fc7be9430 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0c9160 tx=0x7f81bc0366d0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81d406a740 1 -- 172.21.15.43:0/579105412 shutdown_connections 2024-04-21T08:43:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81d406a740 1 --2- 172.21.15.43:0/579105412 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x561fc7be1d60 0x561fc7c4e9e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81d406a740 1 --2- 172.21.15.43:0/579105412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561fc7bd1a00 0x561fc7be9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81aeff5640 0 client.17763 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:43:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81d406a740 1 --2- 172.21.15.43:0/579105412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561fc7bc8e80 0x561fc7be9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81d406a740 1 --2- 172.21.15.43:0/579105412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fc7bc84d0 0x561fc7be9430 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81d406a740 1 -- 172.21.15.43:0/579105412 >> 172.21.15.43:0/579105412 conn(0x561fc7bc2ba0 msgr2=0x561fc7bc3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81d406a740 1 -- 172.21.15.43:0/579105412 shutdown_connections 2024-04-21T08:43:52.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81d406a740 1 -- 172.21.15.43:0/579105412 wait complete. 2024-04-21T08:43:52.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81d406a740 20 client.17763 trim_cache size 0 max 16384 2024-04-21T08:43:52.681 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:43:52.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81d147a640 1 -- 172.21.15.43:0/3472769610 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f81b40a3d20 msgr2=0x7f81b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81d147a640 1 --2- 172.21.15.43:0/3472769610 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f81b40a3d20 0x7f81b40a61e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f81c4011cb0 tx=0x7f81c4016000 comp rx=0 tx=0).stop 2024-04-21T08:43:52.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81d147a640 1 -- 172.21.15.43:0/3472769610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81cc1010a0 msgr2=0x7f81cc1964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81d147a640 1 --2- 172.21.15.43:0/3472769610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81cc1010a0 0x7f81cc1964b0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f81bc006010 tx=0x7f81bc004430 comp rx=0 tx=0).stop 2024-04-21T08:43:52.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81d147a640 1 -- 172.21.15.43:0/3472769610 shutdown_connections 2024-04-21T08:43:52.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81d147a640 1 --2- 172.21.15.43:0/3472769610 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f81b40a3d20 0x7f81b40a61e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81d147a640 1 --2- 172.21.15.43:0/3472769610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81cc19ad70 0x7f81cc19b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81d147a640 1 --2- 172.21.15.43:0/3472769610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81cc106170 0x7f81cc1969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81d147a640 1 --2- 172.21.15.43:0/3472769610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81cc1010a0 0x7f81cc1964b0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.673+0000 7f81d147a640 1 -- 172.21.15.43:0/3472769610 >> 172.21.15.43:0/3472769610 conn(0x7f81cc077eb0 msgr2=0x7f81cc107340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.674+0000 7f81d147a640 1 -- 172.21.15.43:0/3472769610 shutdown_connections 2024-04-21T08:43:52.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.674+0000 7f81d147a640 1 -- 172.21.15.43:0/3472769610 wait complete. 2024-04-21T08:43:52.685 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:43:52.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:52 smithi175 ceph-mon[31920]: pgmap v151: 129 pgs: 129 active+clean; 2.5 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-21T08:43:52.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:52 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3908235331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:43:52.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.789+0000 7f2d81a11640 1 -- 172.21.15.43:0/782273078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7c100e10 msgr2=0x7f2d7c10ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.789+0000 7f2d81a11640 1 --2- 172.21.15.43:0/782273078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7c100e10 0x7f2d7c10ede0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009cd0 tx=0x7f2d7402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.789+0000 7f2d81a11640 1 -- 172.21.15.43:0/782273078 shutdown_connections 2024-04-21T08:43:52.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.789+0000 7f2d81a11640 1 --2- 172.21.15.43:0/782273078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7c100e10 0x7f2d7c10ede0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.789+0000 7f2d81a11640 1 --2- 172.21.15.43:0/782273078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7c100450 0x7f2d7c1008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.789+0000 7f2d81a11640 1 --2- 172.21.15.43:0/782273078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7c106f30 0x7f2d7c107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.789+0000 7f2d81a11640 1 -- 172.21.15.43:0/782273078 >> 172.21.15.43:0/782273078 conn(0x7f2d7c077410 msgr2=0x7f2d7c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.789+0000 7f2d81a11640 1 -- 172.21.15.43:0/782273078 shutdown_connections 2024-04-21T08:43:52.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.790+0000 7f2d81a11640 1 -- 172.21.15.43:0/782273078 wait complete. 2024-04-21T08:43:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.790+0000 7f2d81a11640 1 Processor -- start 2024-04-21T08:43:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.790+0000 7f2d81a11640 1 -- start start 2024-04-21T08:43:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.790+0000 7f2d81a11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7c100450 0x7f2d7c101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.790+0000 7f2d81a11640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7c100e10 0x7f2d7c101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.790+0000 7f2d81a11640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7c106f30 0x7f2d7c1020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.790+0000 7f2d81a11640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c112270 con 0x7f2d7c100450 2024-04-21T08:43:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.790+0000 7f2d81a11640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c1120f0 con 0x7f2d7c100e10 2024-04-21T08:43:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.790+0000 7f2d81a11640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c1123f0 con 0x7f2d7c106f30 2024-04-21T08:43:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.790+0000 7f2d7affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7c100450 0x7f2d7c101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.790+0000 7f2d7affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7c100450 0x7f2d7c101630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37252/0 (socket says 172.21.15.43:37252) 2024-04-21T08:43:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.790+0000 7f2d7affd640 1 -- 172.21.15.43:0/1268383607 learned_addr learned my addr 172.21.15.43:0/1268383607 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.790+0000 7f2d73fff640 1 --2- 172.21.15.43:0/1268383607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7c100e10 0x7f2d7c101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.791+0000 7f2d7b7fe640 1 --2- 172.21.15.43:0/1268383607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7c106f30 0x7f2d7c1020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.791+0000 7f2d7affd640 1 -- 172.21.15.43:0/1268383607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7c106f30 msgr2=0x7f2d7c1020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.791+0000 7f2d7affd640 1 --2- 172.21.15.43:0/1268383607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7c106f30 0x7f2d7c1020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.791+0000 7f2d7affd640 1 -- 172.21.15.43:0/1268383607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7c100e10 msgr2=0x7f2d7c101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.791+0000 7f2d7affd640 1 --2- 172.21.15.43:0/1268383607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7c100e10 0x7f2d7c101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.791+0000 7f2d7affd640 1 -- 172.21.15.43:0/1268383607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7c1a6bf0 con 0x7f2d7c100450 2024-04-21T08:43:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.791+0000 7f2d7b7fe640 1 --2- 172.21.15.43:0/1268383607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7c106f30 0x7f2d7c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.791+0000 7f2d7affd640 1 --2- 172.21.15.43:0/1268383607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7c100450 0x7f2d7c101630 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00c920 tx=0x7f2d6c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.791+0000 7f2d80a0f640 1 -- 172.21.15.43:0/1268383607 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c007c70 con 0x7f2d7c100450 2024-04-21T08:43:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.791+0000 7f2d80a0f640 1 -- 172.21.15.43:0/1268383607 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d6c007e10 con 0x7f2d7c100450 2024-04-21T08:43:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.791+0000 7f2d81a11640 1 -- 172.21.15.43:0/1268383607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7c1a6ee0 con 0x7f2d7c100450 2024-04-21T08:43:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.791+0000 7f2d80a0f640 1 -- 172.21.15.43:0/1268383607 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c018820 con 0x7f2d7c100450 2024-04-21T08:43:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.791+0000 7f2d81a11640 1 -- 172.21.15.43:0/1268383607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d7c1a73a0 con 0x7f2d7c100450 2024-04-21T08:43:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.793+0000 7f2d80a0f640 1 -- 172.21.15.43:0/1268383607 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2d6c00f450 con 0x7f2d7c100450 2024-04-21T08:43:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.794+0000 7f2d84602740 1 Processor -- start 2024-04-21T08:43:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.794+0000 7f2d84602740 1 -- start start 2024-04-21T08:43:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.794+0000 7f2d84602740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be0c4d0 0x56535be0c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.794+0000 7f2d84602740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56535be0ce80 0x56535be15430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.794+0000 7f2d80a0f640 1 --2- 172.21.15.43:0/1268383607 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2d680a3d20 0x7f2d680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.794+0000 7f2d84602740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56535be15a00 0x56535be17e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.794+0000 7f2d84602740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56535bd711f0 con 0x56535be0c4d0 2024-04-21T08:43:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.794+0000 7f2d84602740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56535bd0e4b0 con 0x56535be15a00 2024-04-21T08:43:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.794+0000 7f2d84602740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56535bd6d1b0 con 0x56535be0ce80 2024-04-21T08:43:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.794+0000 7f2d7b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be0c4d0 0x56535be0c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.794+0000 7f2d73fff640 1 --2- 172.21.15.43:0/1268383607 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2d680a3d20 0x7f2d680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.794+0000 7f2d7b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be0c4d0 0x56535be0c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37254/0 (socket says 172.21.15.43:37254) 2024-04-21T08:43:52.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.794+0000 7f2d7b7fe640 1 -- 172.21.15.43:0/3454374315 learned_addr learned my addr 172.21.15.43:0/3454374315 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.794+0000 7f2d80a0f640 1 -- 172.21.15.43:0/1268383607 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f2d6c0d0c60 con 0x7f2d7c100450 2024-04-21T08:43:52.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.794+0000 7f2d7affd640 1 --2- 172.21.15.43:0/3454374315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56535be0ce80 0x56535be15430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.794+0000 7f2d7b7fe640 1 --2- 172.21.15.43:0/3454374315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56535be15a00 0x56535be17e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.794+0000 7f2d7b7fe640 1 -- 172.21.15.43:0/3454374315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56535be0ce80 msgr2=0x56535be15430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.794+0000 7f2d7b7fe640 1 --2- 172.21.15.43:0/3454374315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56535be0ce80 0x56535be15430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.794+0000 7f2d7b7fe640 1 -- 172.21.15.43:0/3454374315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56535be15a00 msgr2=0x56535be17e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.794+0000 7f2d7b7fe640 1 --2- 172.21.15.43:0/3454374315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56535be15a00 0x56535be17e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.794+0000 7f2d7b7fe640 1 -- 172.21.15.43:0/3454374315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56535bd001b0 con 0x56535be0c4d0 2024-04-21T08:43:52.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.794+0000 7f2d7affd640 1 --2- 172.21.15.43:0/3454374315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56535be0ce80 0x56535be15430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:52.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.794+0000 7f2d73fff640 1 --2- 172.21.15.43:0/1268383607 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2d680a3d20 0x7f2d680a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c1030c0 tx=0x7f2d64005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.794+0000 7f2d7b7fe640 1 --2- 172.21.15.43:0/3454374315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be0c4d0 0x56535be0c8b0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f2d7402c700 tx=0x7f2d74005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df6e2cc7f6ea230 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.795+0000 7f2d71ffb640 1 -- 172.21.15.43:0/3454374315 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d74039040 con 0x56535be0c4d0 2024-04-21T08:43:52.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.795+0000 7f2d71ffb640 1 -- 172.21.15.43:0/3454374315 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d74032970 con 0x56535be0c4d0 2024-04-21T08:43:52.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.795+0000 7f2d71ffb640 1 -- 172.21.15.43:0/3454374315 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d74004290 con 0x56535be0c4d0 2024-04-21T08:43:52.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.795+0000 7f2d84602740 1 -- 172.21.15.43:0/3454374315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be0c4d0 msgr2=0x56535be0c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.795+0000 7f2d84602740 1 --2- 172.21.15.43:0/3454374315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be0c4d0 0x56535be0c8b0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f2d7402c700 tx=0x7f2d74005f30 comp rx=0 tx=0).stop 2024-04-21T08:43:52.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.795+0000 7f2d84602740 1 -- 172.21.15.43:0/3454374315 shutdown_connections 2024-04-21T08:43:52.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.795+0000 7f2d84602740 1 --2- 172.21.15.43:0/3454374315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56535be15a00 0x56535be17e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.795+0000 7f2d84602740 1 --2- 172.21.15.43:0/3454374315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56535be0ce80 0x56535be15430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.795+0000 7f2d84602740 1 --2- 172.21.15.43:0/3454374315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be0c4d0 0x56535be0c8b0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.795+0000 7f2d84602740 1 -- 172.21.15.43:0/3454374315 >> 172.21.15.43:0/3454374315 conn(0x56535be06ba0 msgr2=0x56535be06f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.795+0000 7f2d84602740 1 -- 172.21.15.43:0/3454374315 shutdown_connections 2024-04-21T08:43:52.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.795+0000 7f2d84602740 1 -- 172.21.15.43:0/3454374315 wait complete. 2024-04-21T08:43:52.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.796+0000 7f2d84602740 1 -- start start 2024-04-21T08:43:52.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.796+0000 7f2d84602740 1 Processor -- start 2024-04-21T08:43:52.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.796+0000 7f2d84602740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be0c4d0 0x56535be15370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.796+0000 7f2d84602740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56535be0ce80 0x56535be12700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.796+0000 7f2d84602740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56535be15a00 0x56535be12d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.796+0000 7f2d84602740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56535be18910 con 0x56535be0c4d0 2024-04-21T08:43:52.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.796+0000 7f2d84602740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56535bd2a500 con 0x56535be15a00 2024-04-21T08:43:52.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.796+0000 7f2d84602740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56535be18a90 con 0x56535be0ce80 2024-04-21T08:43:52.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.796+0000 7f2d84602740 10 client.0 _ll_register_callbacks cb 0x56535bcf8b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:52.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.796+0000 7f2d7affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be0c4d0 0x56535be15370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.796+0000 7f2d7affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be0c4d0 0x56535be15370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37270/0 (socket says 172.21.15.43:37270) 2024-04-21T08:43:52.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.796+0000 7f2d7affd640 1 -- 172.21.15.43:0/1624810954 learned_addr learned my addr 172.21.15.43:0/1624810954 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.796+0000 7f2d7b7fe640 1 --2- 172.21.15.43:0/1624810954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56535be15a00 0x56535be12d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.797+0000 7f2d73fff640 1 --2- 172.21.15.43:0/1624810954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56535be0ce80 0x56535be12700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.797+0000 7f2d7affd640 1 -- 172.21.15.43:0/1624810954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56535be0ce80 msgr2=0x56535be12700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.797+0000 7f2d7affd640 1 --2- 172.21.15.43:0/1624810954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56535be0ce80 0x56535be12700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.797+0000 7f2d7affd640 1 -- 172.21.15.43:0/1624810954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56535be15a00 msgr2=0x56535be12d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.797+0000 7f2d7affd640 1 --2- 172.21.15.43:0/1624810954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56535be15a00 0x56535be12d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.797+0000 7f2d7affd640 1 -- 172.21.15.43:0/1624810954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56535ba833f0 con 0x56535be0c4d0 2024-04-21T08:43:52.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.797+0000 7f2d7affd640 1 --2- 172.21.15.43:0/1624810954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be0c4d0 0x56535be15370 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0ce090 tx=0x7f2d6c0672d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3498c7d165500e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.797+0000 7f2d5affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:52.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.797+0000 7f2d5affd640 1 -- 172.21.15.43:0/1624810954 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c067340 con 0x56535be0c4d0 2024-04-21T08:43:52.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.797+0000 7f2d5affd640 1 -- 172.21.15.43:0/1624810954 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f2d6c00f740 con 0x56535be0c4d0 2024-04-21T08:43:52.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.797+0000 7f2d84602740 1 -- 172.21.15.43:0/1624810954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d74032970 con 0x56535be0c4d0 2024-04-21T08:43:52.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.797+0000 7f2d5affd640 1 -- 172.21.15.43:0/1624810954 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2d6c09c830 con 0x56535be0c4d0 2024-04-21T08:43:52.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.797+0000 7f2d84602740 10 client.17775 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:52.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.797+0000 7f2d84602740 1 -- 172.21.15.43:0/1624810954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56535bb4d7d0 con 0x56535be0c4d0 2024-04-21T08:43:52.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.797+0000 7f2d5affd640 1 -- 172.21.15.43:0/1624810954 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d74032970 con 0x56535be0c4d0 2024-04-21T08:43:52.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.797+0000 7f2d5affd640 1 -- 172.21.15.43:0/1624810954 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2d6c0b6ac0 con 0x56535be0c4d0 2024-04-21T08:43:52.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.797+0000 7f2d84602740 10 client.17775 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:52.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.797+0000 7f2d84602740 10 client.17775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:52.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.797+0000 7f2d84602740 20 client.17775 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:52.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.797+0000 7f2d84602740 1 --2- 172.21.15.43:0/1624810954 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56535be25bb0 0x56535be28000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.797+0000 7f2d84602740 4 client.17775 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:52.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.797+0000 7f2d84602740 1 -- 172.21.15.43:0/1624810954 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56535bd10930 con 0x56535be25bb0 2024-04-21T08:43:52.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.798+0000 7f2d73fff640 1 --2- 172.21.15.43:0/1624810954 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56535be25bb0 0x56535be28000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.798+0000 7f2d73fff640 1 --2- 172.21.15.43:0/1624810954 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56535be25bb0 0x56535be28000 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e975d754c958c17 server_cookie=da5dc77e1c34d68c in_seq=0 out_seq=0 2024-04-21T08:43:52.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.798+0000 7f2d5affd640 10 client.17775 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:52.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.799+0000 7f2d5affd640 1 -- 172.21.15.43:0/1624810954 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56535bd10930 con 0x56535be25bb0 2024-04-21T08:43:52.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.799+0000 7f2d5affd640 10 client.17775 handle_command_reply: tid=0 2024-04-21T08:43:52.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.799+0000 7f2d84602740 1 client.17775 shutdown 2024-04-21T08:43:52.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.799+0000 7f2d84602740 1 -- 172.21.15.43:0/1624810954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be0c4d0 msgr2=0x56535be15370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.799+0000 7f2d84602740 1 --2- 172.21.15.43:0/1624810954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be0c4d0 0x56535be15370 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0ce090 tx=0x7f2d6c0672d0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.799+0000 7f2d84602740 1 -- 172.21.15.43:0/1624810954 shutdown_connections 2024-04-21T08:43:52.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.799+0000 7f2d84602740 1 --2- 172.21.15.43:0/1624810954 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56535be25bb0 0x56535be28000 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.799+0000 7f2d84602740 1 --2- 172.21.15.43:0/1624810954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56535be15a00 0x56535be12d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.799+0000 7f2d84602740 1 --2- 172.21.15.43:0/1624810954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56535be0ce80 0x56535be12700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.799+0000 7f2d5affd640 0 client.17775 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:52.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.799+0000 7f2d84602740 1 --2- 172.21.15.43:0/1624810954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be0c4d0 0x56535be15370 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.799+0000 7f2d84602740 1 -- 172.21.15.43:0/1624810954 >> 172.21.15.43:0/1624810954 conn(0x56535be06ba0 msgr2=0x56535be088f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.799+0000 7f2d84602740 1 -- 172.21.15.43:0/1624810954 shutdown_connections 2024-04-21T08:43:52.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.799+0000 7f2d84602740 1 -- 172.21.15.43:0/1624810954 wait complete. 2024-04-21T08:43:52.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.799+0000 7f2d84602740 20 client.17775 trim_cache size 0 max 16384 2024-04-21T08:43:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.813+0000 7f2d84602740 1 Processor -- start 2024-04-21T08:43:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.813+0000 7f2d84602740 1 -- start start 2024-04-21T08:43:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.813+0000 7f2d84602740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56535be0c4d0 0x56535be217d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.813+0000 7f2d84602740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be0ce80 0x56535be14e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.813+0000 7f2d84602740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56535be15a00 0x56535be15370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.813+0000 7f2d84602740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c1b14e0 con 0x56535be0ce80 2024-04-21T08:43:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.813+0000 7f2d84602740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c1b3020 con 0x56535be15a00 2024-04-21T08:43:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.813+0000 7f2d84602740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c1b1bb0 con 0x56535be0c4d0 2024-04-21T08:43:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.813+0000 7f2d73fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be0ce80 0x56535be14e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.813+0000 7f2d73fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be0ce80 0x56535be14e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37278/0 (socket says 172.21.15.43:37278) 2024-04-21T08:43:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.813+0000 7f2d73fff640 1 -- 172.21.15.43:0/3966694562 learned_addr learned my addr 172.21.15.43:0/3966694562 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.813+0000 7f2d7affd640 1 --2- 172.21.15.43:0/3966694562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56535be0c4d0 0x56535be217d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.813+0000 7f2d73fff640 1 -- 172.21.15.43:0/3966694562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56535be0c4d0 msgr2=0x56535be217d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.813+0000 7f2d73fff640 1 --2- 172.21.15.43:0/3966694562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56535be0c4d0 0x56535be217d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.813+0000 7f2d73fff640 1 -- 172.21.15.43:0/3966694562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56535be15a00 msgr2=0x56535be15370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.813+0000 7f2d73fff640 1 --2- 172.21.15.43:0/3966694562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56535be15a00 0x56535be15370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.813+0000 7f2d73fff640 1 -- 172.21.15.43:0/3966694562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56535be0cc20 con 0x56535be0ce80 2024-04-21T08:43:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.813+0000 7f2d73fff640 1 --2- 172.21.15.43:0/3966694562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be0ce80 0x56535be14e30 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x56535be28870 tx=0x7f2d64014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9465061c9b0b91 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.814+0000 7f2d717fa640 1 -- 172.21.15.43:0/3966694562 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d640252d0 con 0x56535be0ce80 2024-04-21T08:43:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.814+0000 7f2d717fa640 1 -- 172.21.15.43:0/3966694562 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d6402b070 con 0x56535be0ce80 2024-04-21T08:43:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.814+0000 7f2d717fa640 1 -- 172.21.15.43:0/3966694562 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d640035f0 con 0x56535be0ce80 2024-04-21T08:43:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.814+0000 7f2d84602740 1 -- 172.21.15.43:0/3966694562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be0ce80 msgr2=0x56535be14e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.814+0000 7f2d84602740 1 --2- 172.21.15.43:0/3966694562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be0ce80 0x56535be14e30 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x56535be28870 tx=0x7f2d64014000 comp rx=0 tx=0).stop 2024-04-21T08:43:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.814+0000 7f2d84602740 1 -- 172.21.15.43:0/3966694562 shutdown_connections 2024-04-21T08:43:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.814+0000 7f2d84602740 1 --2- 172.21.15.43:0/3966694562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56535be15a00 0x56535be15370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.814+0000 7f2d84602740 1 --2- 172.21.15.43:0/3966694562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be0ce80 0x56535be14e30 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.814+0000 7f2d84602740 1 --2- 172.21.15.43:0/3966694562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56535be0c4d0 0x56535be217d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.814+0000 7f2d84602740 1 -- 172.21.15.43:0/3966694562 >> 172.21.15.43:0/3966694562 conn(0x56535be06ba0 msgr2=0x56535be08860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.814+0000 7f2d84602740 1 -- 172.21.15.43:0/3966694562 shutdown_connections 2024-04-21T08:43:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.814+0000 7f2d84602740 1 -- 172.21.15.43:0/3966694562 wait complete. 2024-04-21T08:43:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.814+0000 7f2d84602740 1 -- start start 2024-04-21T08:43:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.815+0000 7f2d84602740 1 Processor -- start 2024-04-21T08:43:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.815+0000 7f2d84602740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56535be0c4d0 0x56535be1dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.815+0000 7f2d84602740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56535be0ce80 0x56535be19360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.815+0000 7f2d84602740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be15a00 0x56535be198a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.815+0000 7f2d84602740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c1b2950 con 0x56535be15a00 2024-04-21T08:43:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.815+0000 7f2d84602740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c1b2280 con 0x56535be0c4d0 2024-04-21T08:43:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.815+0000 7f2d84602740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c1af4c0 con 0x56535be0ce80 2024-04-21T08:43:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.815+0000 7f2d84602740 10 client.0 _ll_register_callbacks cb 0x7f2d6c09c6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.815+0000 7f2d7b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be15a00 0x56535be198a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.815+0000 7f2d7b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be15a00 0x56535be198a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37292/0 (socket says 172.21.15.43:37292) 2024-04-21T08:43:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.815+0000 7f2d7b7fe640 1 -- 172.21.15.43:0/2652606963 learned_addr learned my addr 172.21.15.43:0/2652606963 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.815+0000 7f2d73fff640 1 --2- 172.21.15.43:0/2652606963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56535be0ce80 0x56535be19360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.815+0000 7f2d7affd640 1 --2- 172.21.15.43:0/2652606963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56535be0c4d0 0x56535be1dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.815+0000 7f2d7b7fe640 1 -- 172.21.15.43:0/2652606963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56535be0ce80 msgr2=0x56535be19360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.815+0000 7f2d7b7fe640 1 --2- 172.21.15.43:0/2652606963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56535be0ce80 0x56535be19360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.815+0000 7f2d7b7fe640 1 -- 172.21.15.43:0/2652606963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56535be0c4d0 msgr2=0x56535be1dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.815+0000 7f2d7b7fe640 1 --2- 172.21.15.43:0/2652606963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56535be0c4d0 0x56535be1dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.815+0000 7f2d7b7fe640 1 -- 172.21.15.43:0/2652606963 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56535be1a0a0 con 0x56535be15a00 2024-04-21T08:43:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.816+0000 7f2d7b7fe640 1 --2- 172.21.15.43:0/2652606963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be15a00 0x56535be198a0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f2d74003ce0 tx=0x7f2d7402cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ff89d85768aabb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.816+0000 7f2d5affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.816+0000 7f2d5affd640 1 -- 172.21.15.43:0/2652606963 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d74051060 con 0x56535be15a00 2024-04-21T08:43:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.816+0000 7f2d5affd640 1 -- 172.21.15.43:0/2652606963 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f2d740324c0 con 0x56535be15a00 2024-04-21T08:43:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.816+0000 7f2d84602740 1 -- 172.21.15.43:0/2652606963 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d6402b070 con 0x56535be15a00 2024-04-21T08:43:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.816+0000 7f2d5affd640 1 -- 172.21.15.43:0/2652606963 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2d74058050 con 0x56535be15a00 2024-04-21T08:43:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.816+0000 7f2d84602740 10 client.17781 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.816+0000 7f2d84602740 1 -- 172.21.15.43:0/2652606963 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56535be8c090 con 0x56535be15a00 2024-04-21T08:43:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.816+0000 7f2d5affd640 1 -- 172.21.15.43:0/2652606963 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d6402b070 con 0x56535be15a00 2024-04-21T08:43:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.816+0000 7f2d5affd640 1 -- 172.21.15.43:0/2652606963 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2d74034a10 con 0x56535be15a00 2024-04-21T08:43:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.816+0000 7f2d84602740 10 client.17781 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.816+0000 7f2d84602740 10 client.17781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.816+0000 7f2d84602740 20 client.17781 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.816+0000 7f2d84602740 1 --2- 172.21.15.43:0/2652606963 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56535be8c2b0 0x56535be8e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.816+0000 7f2d84602740 4 client.17781 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:43:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.816+0000 7f2d84602740 1 -- 172.21.15.43:0/2652606963 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56535bd7ec10 con 0x56535be8c2b0 2024-04-21T08:43:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.817+0000 7f2d7affd640 1 --2- 172.21.15.43:0/2652606963 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56535be8c2b0 0x56535be8e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.817+0000 7f2d7affd640 1 --2- 172.21.15.43:0/2652606963 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56535be8c2b0 0x56535be8e6a0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e74818d73f90f285 server_cookie=ef8aa707b190e683 in_seq=0 out_seq=0 2024-04-21T08:43:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.817+0000 7f2d5affd640 10 client.17781 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.817+0000 7f2d5affd640 1 -- 172.21.15.43:0/2652606963 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56535bd7ec10 con 0x56535be8c2b0 2024-04-21T08:43:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.817+0000 7f2d5affd640 10 client.17781 handle_command_reply: tid=0 2024-04-21T08:43:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.817+0000 7f2d84602740 1 client.17781 shutdown 2024-04-21T08:43:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.817+0000 7f2d84602740 1 -- 172.21.15.43:0/2652606963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be15a00 msgr2=0x56535be198a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.817+0000 7f2d84602740 1 --2- 172.21.15.43:0/2652606963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be15a00 0x56535be198a0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f2d74003ce0 tx=0x7f2d7402cf80 comp rx=0 tx=0).stop 2024-04-21T08:43:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.818+0000 7f2d84602740 1 -- 172.21.15.43:0/2652606963 shutdown_connections 2024-04-21T08:43:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.818+0000 7f2d84602740 1 --2- 172.21.15.43:0/2652606963 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56535be8c2b0 0x56535be8e6a0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.818+0000 7f2d84602740 1 --2- 172.21.15.43:0/2652606963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56535be15a00 0x56535be198a0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.818+0000 7f2d84602740 1 --2- 172.21.15.43:0/2652606963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56535be0ce80 0x56535be19360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.818+0000 7f2d5affd640 0 client.17781 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.818+0000 7f2d84602740 1 --2- 172.21.15.43:0/2652606963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56535be0c4d0 0x56535be1dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.818+0000 7f2d84602740 1 -- 172.21.15.43:0/2652606963 >> 172.21.15.43:0/2652606963 conn(0x56535be06ba0 msgr2=0x56535be05370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.818+0000 7f2d84602740 1 -- 172.21.15.43:0/2652606963 shutdown_connections 2024-04-21T08:43:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.818+0000 7f2d84602740 1 -- 172.21.15.43:0/2652606963 wait complete. 2024-04-21T08:43:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.818+0000 7f2d84602740 20 client.17781 trim_cache size 0 max 16384 2024-04-21T08:43:52.824 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:43:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.818+0000 7f2d81a11640 1 -- 172.21.15.43:0/1268383607 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2d680a3d20 msgr2=0x7f2d680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.818+0000 7f2d81a11640 1 --2- 172.21.15.43:0/1268383607 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2d680a3d20 0x7f2d680a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c1030c0 tx=0x7f2d64005f90 comp rx=0 tx=0).stop 2024-04-21T08:43:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.818+0000 7f2d81a11640 1 -- 172.21.15.43:0/1268383607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7c100450 msgr2=0x7f2d7c101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.818+0000 7f2d81a11640 1 --2- 172.21.15.43:0/1268383607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7c100450 0x7f2d7c101630 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00c920 tx=0x7f2d6c00cde0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.818+0000 7f2d81a11640 1 -- 172.21.15.43:0/1268383607 shutdown_connections 2024-04-21T08:43:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.818+0000 7f2d81a11640 1 --2- 172.21.15.43:0/1268383607 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2d680a3d20 0x7f2d680a61e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.818+0000 7f2d81a11640 1 --2- 172.21.15.43:0/1268383607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7c106f30 0x7f2d7c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.818+0000 7f2d81a11640 1 --2- 172.21.15.43:0/1268383607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7c100e10 0x7f2d7c101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.818+0000 7f2d81a11640 1 --2- 172.21.15.43:0/1268383607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7c100450 0x7f2d7c101630 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:52.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.818+0000 7f2d81a11640 1 -- 172.21.15.43:0/1268383607 >> 172.21.15.43:0/1268383607 conn(0x7f2d7c077410 msgr2=0x7f2d7c104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:52.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.818+0000 7f2d81a11640 1 -- 172.21.15.43:0/1268383607 shutdown_connections 2024-04-21T08:43:52.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:52.818+0000 7f2d81a11640 1 -- 172.21.15.43:0/1268383607 wait complete. 2024-04-21T08:43:53.830 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:43:53.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:43:53.830 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:43:53.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.951+0000 7f9eb385c640 1 -- 172.21.15.43:0/1365350535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eac1072e0 msgr2=0x7f9eac10eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:53.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.951+0000 7f9eb385c640 1 --2- 172.21.15.43:0/1365350535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eac1072e0 0x7f9eac10eeb0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009ca0 tx=0x7f9ea402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.951+0000 7f9eb385c640 1 -- 172.21.15.43:0/1365350535 shutdown_connections 2024-04-21T08:43:53.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.951+0000 7f9eb385c640 1 --2- 172.21.15.43:0/1365350535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eac1072e0 0x7f9eac10eeb0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.951+0000 7f9eb385c640 1 --2- 172.21.15.43:0/1365350535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9eac102bc0 0x7f9eac106c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.951+0000 7f9eb385c640 1 --2- 172.21.15.43:0/1365350535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eac102210 0x7f9eac1025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.951+0000 7f9eb385c640 1 -- 172.21.15.43:0/1365350535 >> 172.21.15.43:0/1365350535 conn(0x7f9eac0fdab0 msgr2=0x7f9eac0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:53.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.951+0000 7f9eb385c640 1 -- 172.21.15.43:0/1365350535 shutdown_connections 2024-04-21T08:43:53.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.951+0000 7f9eb385c640 1 -- 172.21.15.43:0/1365350535 wait complete. 2024-04-21T08:43:53.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.952+0000 7f9eb385c640 1 Processor -- start 2024-04-21T08:43:53.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.952+0000 7f9eb385c640 1 -- start start 2024-04-21T08:43:53.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.952+0000 7f9eb385c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9eac102210 0x7f9eac074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:53.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.952+0000 7f9eb385c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eac1072e0 0x7f9eac074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:53.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.952+0000 7f9eb385c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eac074cd0 0x7f9eac1a69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:53.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.952+0000 7f9eb385c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac112250 con 0x7f9eac074cd0 2024-04-21T08:43:53.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.952+0000 7f9eb385c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1120d0 con 0x7f9eac102210 2024-04-21T08:43:53.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.952+0000 7f9eb385c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1123d0 con 0x7f9eac1072e0 2024-04-21T08:43:53.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.952+0000 7f9eb1df9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eac074cd0 0x7f9eac1a69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:53.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.952+0000 7f9eb1df9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eac074cd0 0x7f9eac1a69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37312/0 (socket says 172.21.15.43:37312) 2024-04-21T08:43:53.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.952+0000 7f9eb1df9640 1 -- 172.21.15.43:0/2426247175 learned_addr learned my addr 172.21.15.43:0/2426247175 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:53.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.953+0000 7f9eb15f8640 1 --2- 172.21.15.43:0/2426247175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9eac102210 0x7f9eac074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:53.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.953+0000 7f9eb0df7640 1 --2- 172.21.15.43:0/2426247175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eac1072e0 0x7f9eac074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:53.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.953+0000 7f9eb1df9640 1 -- 172.21.15.43:0/2426247175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eac1072e0 msgr2=0x7f9eac074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:53.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.953+0000 7f9eb1df9640 1 --2- 172.21.15.43:0/2426247175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eac1072e0 0x7f9eac074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.953+0000 7f9eb1df9640 1 -- 172.21.15.43:0/2426247175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9eac102210 msgr2=0x7f9eac074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:53.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.953+0000 7f9eb1df9640 1 --2- 172.21.15.43:0/2426247175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9eac102210 0x7f9eac074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.953+0000 7f9eb1df9640 1 -- 172.21.15.43:0/2426247175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eac1a70a0 con 0x7f9eac074cd0 2024-04-21T08:43:53.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.953+0000 7f9eb0df7640 1 --2- 172.21.15.43:0/2426247175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eac1072e0 0x7f9eac074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:53.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.953+0000 7f9eb1df9640 1 --2- 172.21.15.43:0/2426247175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eac074cd0 0x7f9eac1a69a0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4005e30 tx=0x7f9ea4034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:53.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.953+0000 7f9ea27fc640 1 -- 172.21.15.43:0/2426247175 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4039040 con 0x7f9eac074cd0 2024-04-21T08:43:53.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.953+0000 7f9ea27fc640 1 -- 172.21.15.43:0/2426247175 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ea4044070 con 0x7f9eac074cd0 2024-04-21T08:43:53.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.953+0000 7f9eb385c640 1 -- 172.21.15.43:0/2426247175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eac1a7330 con 0x7f9eac074cd0 2024-04-21T08:43:53.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.953+0000 7f9ea27fc640 1 -- 172.21.15.43:0/2426247175 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4003e60 con 0x7f9eac074cd0 2024-04-21T08:43:53.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.953+0000 7f9eb385c640 1 -- 172.21.15.43:0/2426247175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eac1a7870 con 0x7f9eac074cd0 2024-04-21T08:43:53.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.955+0000 7f9ea27fc640 1 -- 172.21.15.43:0/2426247175 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9ea40324c0 con 0x7f9eac074cd0 2024-04-21T08:43:53.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.956+0000 7f9eb644d740 1 Processor -- start 2024-04-21T08:43:53.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.956+0000 7f9eb644d740 1 -- start start 2024-04-21T08:43:53.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.956+0000 7f9eb644d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce1c2f78e0 0x55ce1c2f7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:53.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.956+0000 7f9eb644d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c2f8290 0x55ce1c300840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:53.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.956+0000 7f9eb644d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce1c300e10 0x55ce1c303260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:53.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.956+0000 7f9eb644d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ce1c25bf00 con 0x55ce1c2f8290 2024-04-21T08:43:53.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.956+0000 7f9eb644d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ce1c1f9590 con 0x55ce1c2f78e0 2024-04-21T08:43:53.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.956+0000 7f9eb644d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ce1c257ec0 con 0x55ce1c300e10 2024-04-21T08:43:53.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.956+0000 7f9ea27fc640 1 --2- 172.21.15.43:0/2426247175 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9e980a3dd0 0x7f9e980a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:53.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.956+0000 7f9eb15f8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c2f8290 0x55ce1c300840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:53.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.957+0000 7f9eb15f8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c2f8290 0x55ce1c300840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37320/0 (socket says 172.21.15.43:37320) 2024-04-21T08:43:53.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.957+0000 7f9eb15f8640 1 -- 172.21.15.43:0/3067452649 learned_addr learned my addr 172.21.15.43:0/3067452649 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:53.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.957+0000 7f9eb1df9640 1 --2- 172.21.15.43:0/2426247175 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9e980a3dd0 0x7f9e980a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:53.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.957+0000 7f9eb0df7640 1 --2- 172.21.15.43:0/3067452649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce1c2f78e0 0x55ce1c2f7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:53.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.957+0000 7f9eb0df7640 1 --2- 172.21.15.43:0/3067452649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce1c300e10 0x55ce1c303260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:53.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.957+0000 7f9ea27fc640 1 -- 172.21.15.43:0/2426247175 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f9ea40f2d00 con 0x7f9eac074cd0 2024-04-21T08:43:53.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.957+0000 7f9eb0df7640 1 -- 172.21.15.43:0/3067452649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce1c300e10 msgr2=0x55ce1c303260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:53.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.957+0000 7f9eb0df7640 1 --2- 172.21.15.43:0/3067452649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce1c300e10 0x55ce1c303260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.957+0000 7f9eb0df7640 1 -- 172.21.15.43:0/3067452649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c2f8290 msgr2=0x55ce1c300840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:53.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.957+0000 7f9eb0df7640 1 --2- 172.21.15.43:0/3067452649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c2f8290 0x55ce1c300840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.957+0000 7f9eb0df7640 1 -- 172.21.15.43:0/3067452649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce1c0367e0 con 0x55ce1c2f78e0 2024-04-21T08:43:53.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.957+0000 7f9eb15f8640 1 --2- 172.21.15.43:0/3067452649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c2f8290 0x55ce1c300840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:43:53.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.957+0000 7f9eb1df9640 1 --2- 172.21.15.43:0/2426247175 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9e980a3dd0 0x7f9e980a6290 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40f0240 tx=0x7f9ea402c0f0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:53.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.957+0000 7f9eb0df7640 1 --2- 172.21.15.43:0/3067452649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce1c2f78e0 0x55ce1c2f7cc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c005fd0 tx=0x7f9e9c00e3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19b353d1b1bb0cb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:53.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.957+0000 7f9e977fe640 1 -- 172.21.15.43:0/3067452649 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c005220 con 0x55ce1c2f78e0 2024-04-21T08:43:53.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.958+0000 7f9e977fe640 1 -- 172.21.15.43:0/3067452649 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e9c013e30 con 0x55ce1c2f78e0 2024-04-21T08:43:53.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.958+0000 7f9e977fe640 1 -- 172.21.15.43:0/3067452649 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00b890 con 0x55ce1c2f78e0 2024-04-21T08:43:53.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.958+0000 7f9eb644d740 1 -- 172.21.15.43:0/3067452649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce1c2f78e0 msgr2=0x55ce1c2f7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:53.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.958+0000 7f9eb644d740 1 --2- 172.21.15.43:0/3067452649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce1c2f78e0 0x55ce1c2f7cc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c005fd0 tx=0x7f9e9c00e3b0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.958+0000 7f9eb644d740 1 -- 172.21.15.43:0/3067452649 shutdown_connections 2024-04-21T08:43:53.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.958+0000 7f9eb644d740 1 --2- 172.21.15.43:0/3067452649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce1c300e10 0x55ce1c303260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.958+0000 7f9eb644d740 1 --2- 172.21.15.43:0/3067452649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c2f8290 0x55ce1c300840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.958+0000 7f9eb644d740 1 --2- 172.21.15.43:0/3067452649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce1c2f78e0 0x55ce1c2f7cc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.958+0000 7f9eb644d740 1 -- 172.21.15.43:0/3067452649 >> 172.21.15.43:0/3067452649 conn(0x55ce1c2f1fb0 msgr2=0x55ce1c2f2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:53.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.958+0000 7f9eb644d740 1 -- 172.21.15.43:0/3067452649 shutdown_connections 2024-04-21T08:43:53.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.958+0000 7f9eb644d740 1 -- 172.21.15.43:0/3067452649 wait complete. 2024-04-21T08:43:53.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.959+0000 7f9eb644d740 1 -- start start 2024-04-21T08:43:53.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.959+0000 7f9eb644d740 1 Processor -- start 2024-04-21T08:43:53.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.959+0000 7f9eb644d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce1c2f78e0 0x55ce1c2fde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:53.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.959+0000 7f9eb644d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce1c2f8290 0x55ce1c2fe340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:53.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.959+0000 7f9eb644d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c300e10 0x55ce1c2fe880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:53.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.959+0000 7f9eb644d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ce1c303d80 con 0x55ce1c300e10 2024-04-21T08:43:53.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.959+0000 7f9eb644d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ce1c215580 con 0x55ce1c2f78e0 2024-04-21T08:43:53.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.959+0000 7f9eb644d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ce1c303f00 con 0x55ce1c2f8290 2024-04-21T08:43:53.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.959+0000 7f9eb644d740 10 client.0 _ll_register_callbacks cb 0x55ce1c21b650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:53.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.959+0000 7f9eb0df7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c300e10 0x55ce1c2fe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:53.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.959+0000 7f9eb0df7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c300e10 0x55ce1c2fe880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37332/0 (socket says 172.21.15.43:37332) 2024-04-21T08:43:53.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.960+0000 7f9eb0df7640 1 -- 172.21.15.43:0/1785790151 learned_addr learned my addr 172.21.15.43:0/1785790151 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:53.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.960+0000 7f9eb0df7640 1 --2- 172.21.15.43:0/1785790151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce1c2f78e0 0x55ce1c2fde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:53.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.960+0000 7f9eb15f8640 1 --2- 172.21.15.43:0/1785790151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce1c2f8290 0x55ce1c2fe340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:53.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.960+0000 7f9eb0df7640 1 -- 172.21.15.43:0/1785790151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce1c2f8290 msgr2=0x55ce1c2fe340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:53.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.960+0000 7f9eb0df7640 1 --2- 172.21.15.43:0/1785790151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce1c2f8290 0x55ce1c2fe340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.960+0000 7f9eb0df7640 1 -- 172.21.15.43:0/1785790151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce1c2f78e0 msgr2=0x55ce1c2fde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:53.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.960+0000 7f9eb0df7640 1 --2- 172.21.15.43:0/1785790151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce1c2f78e0 0x55ce1c2fde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.960+0000 7f9eb0df7640 1 -- 172.21.15.43:0/1785790151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce1c096f70 con 0x55ce1c300e10 2024-04-21T08:43:53.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.960+0000 7f9eb0df7640 1 --2- 172.21.15.43:0/1785790151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c300e10 0x55ce1c2fe880 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00ec70 tx=0x7f9e9c00ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9765d3dae0b667 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:53.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.960+0000 7f9e94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:53.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.960+0000 7f9e94ff9640 1 -- 172.21.15.43:0/1785790151 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c0052d0 con 0x55ce1c300e10 2024-04-21T08:43:53.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.960+0000 7f9e94ff9640 1 -- 172.21.15.43:0/1785790151 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f9e9c005470 con 0x55ce1c300e10 2024-04-21T08:43:53.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.960+0000 7f9eb644d740 1 -- 172.21.15.43:0/1785790151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e9c013e30 con 0x55ce1c300e10 2024-04-21T08:43:53.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.960+0000 7f9e94ff9640 1 -- 172.21.15.43:0/1785790151 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9e9c035050 con 0x55ce1c300e10 2024-04-21T08:43:53.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.961+0000 7f9eb644d740 10 client.17793 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:53.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.961+0000 7f9eb644d740 1 -- 172.21.15.43:0/1785790151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ce1bf2e170 con 0x55ce1c300e10 2024-04-21T08:43:53.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.961+0000 7f9e94ff9640 1 -- 172.21.15.43:0/1785790151 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e9c013e30 con 0x55ce1c300e10 2024-04-21T08:43:53.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.961+0000 7f9e94ff9640 1 -- 172.21.15.43:0/1785790151 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9e9c00f040 con 0x55ce1c300e10 2024-04-21T08:43:53.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.961+0000 7f9eb644d740 10 client.17793 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:53.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.961+0000 7f9eb644d740 10 client.17793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:53.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.961+0000 7f9eb644d740 20 client.17793 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:53.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.961+0000 7f9eb644d740 1 --2- 172.21.15.43:0/1785790151 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ce1c311220 0x55ce1c313610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:53.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.961+0000 7f9eb644d740 4 client.17793 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:53.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.961+0000 7f9eb644d740 1 -- 172.21.15.43:0/1785790151 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce1c269920 con 0x55ce1c311220 2024-04-21T08:43:53.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.961+0000 7f9eb15f8640 1 --2- 172.21.15.43:0/1785790151 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ce1c311220 0x55ce1c313610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:53.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.962+0000 7f9e94ff9640 10 client.17793 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:53.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.962+0000 7f9eb15f8640 1 --2- 172.21.15.43:0/1785790151 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ce1c311220 0x55ce1c313610 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdeeb4922d6a4bed server_cookie=2eeed2d7c6ee2de1 in_seq=0 out_seq=0 2024-04-21T08:43:53.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.962+0000 7f9e94ff9640 1 -- 172.21.15.43:0/1785790151 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ce1c269920 con 0x55ce1c311220 2024-04-21T08:43:53.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.962+0000 7f9e94ff9640 10 client.17793 handle_command_reply: tid=0 2024-04-21T08:43:53.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.962+0000 7f9eb644d740 1 client.17793 shutdown 2024-04-21T08:43:53.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.963+0000 7f9eb644d740 1 -- 172.21.15.43:0/1785790151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c300e10 msgr2=0x55ce1c2fe880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:53.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.963+0000 7f9eb644d740 1 --2- 172.21.15.43:0/1785790151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c300e10 0x55ce1c2fe880 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00ec70 tx=0x7f9e9c00ead0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.963+0000 7f9eb644d740 1 -- 172.21.15.43:0/1785790151 shutdown_connections 2024-04-21T08:43:53.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.963+0000 7f9eb644d740 1 --2- 172.21.15.43:0/1785790151 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ce1c311220 0x55ce1c313610 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.963+0000 7f9eb644d740 1 --2- 172.21.15.43:0/1785790151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c300e10 0x55ce1c2fe880 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.963+0000 7f9eb644d740 1 --2- 172.21.15.43:0/1785790151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce1c2f8290 0x55ce1c2fe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.963+0000 7f9eb644d740 1 --2- 172.21.15.43:0/1785790151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce1c2f78e0 0x55ce1c2fde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.963+0000 7f9e94ff9640 0 client.17793 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:53.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.963+0000 7f9eb644d740 1 -- 172.21.15.43:0/1785790151 >> 172.21.15.43:0/1785790151 conn(0x55ce1c2f1fb0 msgr2=0x55ce1c2f3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:53.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.963+0000 7f9eb644d740 1 -- 172.21.15.43:0/1785790151 shutdown_connections 2024-04-21T08:43:53.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.963+0000 7f9eb644d740 1 -- 172.21.15.43:0/1785790151 wait complete. 2024-04-21T08:43:53.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.963+0000 7f9eb644d740 20 client.17793 trim_cache size 0 max 16384 2024-04-21T08:43:53.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.980+0000 7f9eb644d740 1 Processor -- start 2024-04-21T08:43:53.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.981+0000 7f9eb644d740 1 -- start start 2024-04-21T08:43:53.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.981+0000 7f9eb644d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce1c2f78e0 0x55ce1c30cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:53.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.981+0000 7f9eb644d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce1c2f8290 0x55ce1c2fd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:53.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.981+0000 7f9eb644d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c300e10 0x55ce1c318d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:53.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.981+0000 7f9eb644d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1b1670 con 0x55ce1c300e10 2024-04-21T08:43:53.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.981+0000 7f9eb644d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1b31b0 con 0x55ce1c2f8290 2024-04-21T08:43:53.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.981+0000 7f9eb644d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1b1d40 con 0x55ce1c2f78e0 2024-04-21T08:43:53.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.981+0000 7f9eb0df7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c300e10 0x55ce1c318d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:53.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.981+0000 7f9eb0df7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c300e10 0x55ce1c318d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37336/0 (socket says 172.21.15.43:37336) 2024-04-21T08:43:53.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.981+0000 7f9eb0df7640 1 -- 172.21.15.43:0/1805169447 learned_addr learned my addr 172.21.15.43:0/1805169447 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:53.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.981+0000 7f9eb15f8640 1 --2- 172.21.15.43:0/1805169447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce1c2f8290 0x55ce1c2fd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:53.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.981+0000 7f9eb0df7640 1 -- 172.21.15.43:0/1805169447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce1c2f78e0 msgr2=0x55ce1c30cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:53.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.981+0000 7f9eb0df7640 1 --2- 172.21.15.43:0/1805169447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce1c2f78e0 0x55ce1c30cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.981+0000 7f9eb0df7640 1 -- 172.21.15.43:0/1805169447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce1c2f8290 msgr2=0x55ce1c2fd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:53.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.981+0000 7f9eb0df7640 1 --2- 172.21.15.43:0/1805169447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce1c2f8290 0x55ce1c2fd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.981+0000 7f9eb0df7640 1 -- 172.21.15.43:0/1805169447 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce1c303c60 con 0x55ce1c300e10 2024-04-21T08:43:53.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.981+0000 7f9eb0df7640 1 --2- 172.21.15.43:0/1805169447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c300e10 0x55ce1c318d80 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00b940 tx=0x7f9e9c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff72c0898fb65fc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:53.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.981+0000 7f9e96ffd640 1 -- 172.21.15.43:0/1805169447 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00b630 con 0x55ce1c300e10 2024-04-21T08:43:53.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.981+0000 7f9e96ffd640 1 -- 172.21.15.43:0/1805169447 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e9c013e30 con 0x55ce1c300e10 2024-04-21T08:43:53.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.981+0000 7f9e96ffd640 1 -- 172.21.15.43:0/1805169447 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c004680 con 0x55ce1c300e10 2024-04-21T08:43:53.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.982+0000 7f9eb644d740 1 -- 172.21.15.43:0/1805169447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c300e10 msgr2=0x55ce1c318d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:53.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.982+0000 7f9eb644d740 1 --2- 172.21.15.43:0/1805169447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c300e10 0x55ce1c318d80 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00b940 tx=0x7f9e9c005850 comp rx=0 tx=0).stop 2024-04-21T08:43:53.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.982+0000 7f9eb644d740 1 -- 172.21.15.43:0/1805169447 shutdown_connections 2024-04-21T08:43:53.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.982+0000 7f9eb644d740 1 --2- 172.21.15.43:0/1805169447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c300e10 0x55ce1c318d80 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.982+0000 7f9eb644d740 1 --2- 172.21.15.43:0/1805169447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce1c2f8290 0x55ce1c2fd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.982+0000 7f9eb644d740 1 --2- 172.21.15.43:0/1805169447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce1c2f78e0 0x55ce1c30cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.982+0000 7f9eb644d740 1 -- 172.21.15.43:0/1805169447 >> 172.21.15.43:0/1805169447 conn(0x55ce1c2f1fb0 msgr2=0x55ce1c2f1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:53.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.982+0000 7f9eb644d740 1 -- 172.21.15.43:0/1805169447 shutdown_connections 2024-04-21T08:43:53.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.982+0000 7f9eb644d740 1 -- 172.21.15.43:0/1805169447 wait complete. 2024-04-21T08:43:53.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.982+0000 7f9eb644d740 1 -- start start 2024-04-21T08:43:53.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.982+0000 7f9eb644d740 1 Processor -- start 2024-04-21T08:43:53.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.983+0000 7f9eb644d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce1c2f78e0 0x55ce1c309370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:53.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.983+0000 7f9eb644d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce1c2f8290 0x55ce1c306150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:53.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.983+0000 7f9eb644d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c300e10 0x55ce1c306690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:53.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.983+0000 7f9eb644d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1b2ae0 con 0x55ce1c300e10 2024-04-21T08:43:53.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.983+0000 7f9eb644d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1b2410 con 0x55ce1c2f8290 2024-04-21T08:43:53.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.983+0000 7f9eb644d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1af650 con 0x55ce1c2f78e0 2024-04-21T08:43:53.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.983+0000 7f9eb644d740 10 client.0 _ll_register_callbacks cb 0x7f9ea8004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:53.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.983+0000 7f9eb0df7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c300e10 0x55ce1c306690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:53.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.983+0000 7f9eb0df7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c300e10 0x55ce1c306690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37340/0 (socket says 172.21.15.43:37340) 2024-04-21T08:43:53.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.983+0000 7f9eb0df7640 1 -- 172.21.15.43:0/1492710110 learned_addr learned my addr 172.21.15.43:0/1492710110 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:53.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.983+0000 7f9eb15f8640 1 --2- 172.21.15.43:0/1492710110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce1c2f8290 0x55ce1c306150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:53.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.983+0000 7f9eb0df7640 1 --2- 172.21.15.43:0/1492710110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce1c2f78e0 0x55ce1c309370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:53.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.983+0000 7f9eb0df7640 1 -- 172.21.15.43:0/1492710110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce1c2f78e0 msgr2=0x55ce1c309370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:53.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.983+0000 7f9eb0df7640 1 --2- 172.21.15.43:0/1492710110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce1c2f78e0 0x55ce1c309370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.983+0000 7f9eb0df7640 1 -- 172.21.15.43:0/1492710110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce1c2f8290 msgr2=0x55ce1c306150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:53.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.983+0000 7f9eb0df7640 1 --2- 172.21.15.43:0/1492710110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce1c2f8290 0x55ce1c306150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.983+0000 7f9eb0df7640 1 -- 172.21.15.43:0/1492710110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce1c306bd0 con 0x55ce1c300e10 2024-04-21T08:43:53.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.983+0000 7f9eb15f8640 1 --2- 172.21.15.43:0/1492710110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce1c2f8290 0x55ce1c306150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:53.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.983+0000 7f9eb0df7640 1 --2- 172.21.15.43:0/1492710110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c300e10 0x55ce1c306690 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00bdc0 tx=0x7f9e9c015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc4efd845a92e426 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:53.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.983+0000 7f9e94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:53.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.984+0000 7f9e94ff9640 1 -- 172.21.15.43:0/1492710110 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c005bb0 con 0x55ce1c300e10 2024-04-21T08:43:53.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.984+0000 7f9e94ff9640 1 -- 172.21.15.43:0/1492710110 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f9e9c00be40 con 0x55ce1c300e10 2024-04-21T08:43:53.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.984+0000 7f9eb644d740 1 -- 172.21.15.43:0/1492710110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e9c013e30 con 0x55ce1c300e10 2024-04-21T08:43:53.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.984+0000 7f9e94ff9640 1 -- 172.21.15.43:0/1492710110 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9e9c04c050 con 0x55ce1c300e10 2024-04-21T08:43:53.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.984+0000 7f9eb644d740 10 client.17799 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:53.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.984+0000 7f9eb644d740 1 -- 172.21.15.43:0/1492710110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ce1c378360 con 0x55ce1c300e10 2024-04-21T08:43:53.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.984+0000 7f9e94ff9640 1 -- 172.21.15.43:0/1492710110 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e9c013e30 con 0x55ce1c300e10 2024-04-21T08:43:53.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.984+0000 7f9e94ff9640 1 -- 172.21.15.43:0/1492710110 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9e9c011470 con 0x55ce1c300e10 2024-04-21T08:43:53.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.984+0000 7f9eb644d740 10 client.17799 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:53.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.984+0000 7f9eb644d740 10 client.17799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:53.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.984+0000 7f9eb644d740 20 client.17799 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:53.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.984+0000 7f9eb644d740 1 --2- 172.21.15.43:0/1492710110 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ce1c378970 0x55ce1c37ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:53.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.984+0000 7f9eb644d740 4 client.17799 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:43:53.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.984+0000 7f9eb644d740 1 -- 172.21.15.43:0/1492710110 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ce1c263790 con 0x55ce1c378970 2024-04-21T08:43:53.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.985+0000 7f9eb15f8640 1 --2- 172.21.15.43:0/1492710110 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ce1c378970 0x55ce1c37ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:53.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.985+0000 7f9eb15f8640 1 --2- 172.21.15.43:0/1492710110 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ce1c378970 0x55ce1c37ad90 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=443742ffff43aa92 server_cookie=baaffce08548c714 in_seq=0 out_seq=0 2024-04-21T08:43:53.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.985+0000 7f9e94ff9640 10 client.17799 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:53.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.986+0000 7f9e94ff9640 1 -- 172.21.15.43:0/1492710110 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ce1c263790 con 0x55ce1c378970 2024-04-21T08:43:53.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.986+0000 7f9e94ff9640 10 client.17799 handle_command_reply: tid=0 2024-04-21T08:43:53.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.986+0000 7f9eb644d740 1 client.17799 shutdown 2024-04-21T08:43:53.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.986+0000 7f9eb644d740 1 -- 172.21.15.43:0/1492710110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c300e10 msgr2=0x55ce1c306690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:53.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.986+0000 7f9eb644d740 1 --2- 172.21.15.43:0/1492710110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c300e10 0x55ce1c306690 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00bdc0 tx=0x7f9e9c015040 comp rx=0 tx=0).stop 2024-04-21T08:43:53.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.986+0000 7f9eb644d740 1 -- 172.21.15.43:0/1492710110 shutdown_connections 2024-04-21T08:43:53.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.986+0000 7f9eb644d740 1 --2- 172.21.15.43:0/1492710110 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ce1c378970 0x55ce1c37ad90 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.986+0000 7f9eb644d740 1 --2- 172.21.15.43:0/1492710110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ce1c300e10 0x55ce1c306690 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.986+0000 7f9eb644d740 1 --2- 172.21.15.43:0/1492710110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce1c2f8290 0x55ce1c306150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.986+0000 7f9e94ff9640 0 client.17799 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:53.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.986+0000 7f9eb644d740 1 --2- 172.21.15.43:0/1492710110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce1c2f78e0 0x55ce1c309370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.986+0000 7f9eb644d740 1 -- 172.21.15.43:0/1492710110 >> 172.21.15.43:0/1492710110 conn(0x55ce1c2f1fb0 msgr2=0x55ce1c2f3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:53.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.986+0000 7f9eb644d740 1 -- 172.21.15.43:0/1492710110 shutdown_connections 2024-04-21T08:43:53.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.987+0000 7f9eb644d740 1 -- 172.21.15.43:0/1492710110 wait complete. 2024-04-21T08:43:53.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.987+0000 7f9eb644d740 20 client.17799 trim_cache size 0 max 16384 2024-04-21T08:43:53.991 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:43:53.991 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:43:53.991 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "1fd9e8ef-e437-4697-a4a3-3d32a551d199", 2024-04-21T08:43:53.991 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:43:53.992 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:43:53.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.987+0000 7f9eb385c640 1 -- 172.21.15.43:0/2426247175 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9e980a3dd0 msgr2=0x7f9e980a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:53.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.987+0000 7f9eb385c640 1 --2- 172.21.15.43:0/2426247175 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9e980a3dd0 0x7f9e980a6290 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40f0240 tx=0x7f9ea402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.987+0000 7f9eb385c640 1 -- 172.21.15.43:0/2426247175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eac074cd0 msgr2=0x7f9eac1a69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:53.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.987+0000 7f9eb385c640 1 --2- 172.21.15.43:0/2426247175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eac074cd0 0x7f9eac1a69a0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4005e30 tx=0x7f9ea4034760 comp rx=0 tx=0).stop 2024-04-21T08:43:53.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.987+0000 7f9eb385c640 1 -- 172.21.15.43:0/2426247175 shutdown_connections 2024-04-21T08:43:53.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.987+0000 7f9eb385c640 1 --2- 172.21.15.43:0/2426247175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eac074cd0 0x7f9eac1a69a0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.987+0000 7f9eb385c640 1 --2- 172.21.15.43:0/2426247175 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9e980a3dd0 0x7f9e980a6290 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.987+0000 7f9eb385c640 1 --2- 172.21.15.43:0/2426247175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eac1072e0 0x7f9eac074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.987+0000 7f9eb385c640 1 --2- 172.21.15.43:0/2426247175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9eac102210 0x7f9eac074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.987+0000 7f9eb385c640 1 -- 172.21.15.43:0/2426247175 >> 172.21.15.43:0/2426247175 conn(0x7f9eac0fdab0 msgr2=0x7f9eac0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:53.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.987+0000 7f9eb385c640 1 -- 172.21.15.43:0/2426247175 shutdown_connections 2024-04-21T08:43:53.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:53.987+0000 7f9eb385c640 1 -- 172.21.15.43:0/2426247175 wait complete. 2024-04-21T08:43:53.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:43:53.998 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:43:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.103+0000 7fba20e3e640 1 -- 172.21.15.43:0/1877775099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba1c106150 msgr2=0x7fba1c10dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.103+0000 7fba20e3e640 1 --2- 172.21.15.43:0/1877775099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba1c106150 0x7fba1c10dd20 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fba14009ca0 tx=0x7fba1402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.103+0000 7fba20e3e640 1 -- 172.21.15.43:0/1877775099 shutdown_connections 2024-04-21T08:43:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.103+0000 7fba20e3e640 1 --2- 172.21.15.43:0/1877775099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba1c106150 0x7fba1c10dd20 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.103+0000 7fba20e3e640 1 --2- 172.21.15.43:0/1877775099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba1c101a30 0x7fba1c105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.103+0000 7fba20e3e640 1 --2- 172.21.15.43:0/1877775099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba1c101080 0x7fba1c101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.103+0000 7fba20e3e640 1 -- 172.21.15.43:0/1877775099 >> 172.21.15.43:0/1877775099 conn(0x7fba1c078000 msgr2=0x7fba1c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.103+0000 7fba20e3e640 1 -- 172.21.15.43:0/1877775099 shutdown_connections 2024-04-21T08:43:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.103+0000 7fba20e3e640 1 -- 172.21.15.43:0/1877775099 wait complete. 2024-04-21T08:43:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.104+0000 7fba20e3e640 1 Processor -- start 2024-04-21T08:43:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.104+0000 7fba20e3e640 1 -- start start 2024-04-21T08:43:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.104+0000 7fba20e3e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba1c101080 0x7fba1c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.104+0000 7fba20e3e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba1c106150 0x7fba1c198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.104+0000 7fba20e3e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba1c19cfd0 0x7fba1c19d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.104+0000 7fba20e3e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c111110 con 0x7fba1c19cfd0 2024-04-21T08:43:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.104+0000 7fba20e3e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c110f90 con 0x7fba1c106150 2024-04-21T08:43:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.104+0000 7fba20e3e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c111290 con 0x7fba1c101080 2024-04-21T08:43:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.104+0000 7fba1ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba1c19cfd0 0x7fba1c19d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.104+0000 7fba1ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba1c19cfd0 0x7fba1c19d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37348/0 (socket says 172.21.15.43:37348) 2024-04-21T08:43:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.104+0000 7fba1ad9d640 1 -- 172.21.15.43:0/238866551 learned_addr learned my addr 172.21.15.43:0/238866551 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.104+0000 7fba13fff640 1 --2- 172.21.15.43:0/238866551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba1c106150 0x7fba1c198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.104+0000 7fba1ad9d640 1 -- 172.21.15.43:0/238866551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba1c101080 msgr2=0x7fba1c198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.104+0000 7fba1ad9d640 1 --2- 172.21.15.43:0/238866551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba1c101080 0x7fba1c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.104+0000 7fba1ad9d640 1 -- 172.21.15.43:0/238866551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba1c106150 msgr2=0x7fba1c198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.104+0000 7fba1ad9d640 1 --2- 172.21.15.43:0/238866551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba1c106150 0x7fba1c198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.104+0000 7fba1ad9d640 1 -- 172.21.15.43:0/238866551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1c19dac0 con 0x7fba1c19cfd0 2024-04-21T08:43:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.105+0000 7fba1ad9d640 1 --2- 172.21.15.43:0/238866551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba1c19cfd0 0x7fba1c19d480 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fba14005e30 tx=0x7fba14034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.105+0000 7fba137fe640 1 -- 172.21.15.43:0/238866551 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba14039040 con 0x7fba1c19cfd0 2024-04-21T08:43:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.105+0000 7fba137fe640 1 -- 172.21.15.43:0/238866551 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba14044070 con 0x7fba1c19cfd0 2024-04-21T08:43:54.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.105+0000 7fba20e3e640 1 -- 172.21.15.43:0/238866551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba1c19dd50 con 0x7fba1c19cfd0 2024-04-21T08:43:54.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.105+0000 7fba137fe640 1 -- 172.21.15.43:0/238866551 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba14003e60 con 0x7fba1c19cfd0 2024-04-21T08:43:54.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.105+0000 7fba20e3e640 1 -- 172.21.15.43:0/238866551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba1c1a5510 con 0x7fba1c19cfd0 2024-04-21T08:43:54.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.107+0000 7fba137fe640 1 -- 172.21.15.43:0/238866551 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fba140324c0 con 0x7fba1c19cfd0 2024-04-21T08:43:54.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.107+0000 7fba23a2a740 1 Processor -- start 2024-04-21T08:43:54.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.108+0000 7fba23a2a740 1 -- start start 2024-04-21T08:43:54.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.108+0000 7fba23a2a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c427f580 0x5641c427f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:54.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.108+0000 7fba23a2a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641c427fea0 0x5641c4288540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:54.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.108+0000 7fba137fe640 1 --2- 172.21.15.43:0/238866551 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fba080a3d20 0x7fba080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:54.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.108+0000 7fba23a2a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c4288a80 0x5641c428ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:54.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.108+0000 7fba23a2a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5641c41e41e0 con 0x5641c427f580 2024-04-21T08:43:54.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.108+0000 7fba23a2a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5641c4181390 con 0x5641c427fea0 2024-04-21T08:43:54.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.108+0000 7fba23a2a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5641c41e01a0 con 0x5641c4288a80 2024-04-21T08:43:54.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.108+0000 7fba13fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c427f580 0x5641c427f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:54.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.108+0000 7fba13fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c427f580 0x5641c427f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37362/0 (socket says 172.21.15.43:37362) 2024-04-21T08:43:54.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.108+0000 7fba13fff640 1 -- 172.21.15.43:0/555764438 learned_addr learned my addr 172.21.15.43:0/555764438 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:54.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.108+0000 7fba1ad9d640 1 --2- 172.21.15.43:0/555764438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c4288a80 0x5641c428ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:54.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.108+0000 7fba13fff640 1 --2- 172.21.15.43:0/238866551 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fba080a3d20 0x7fba080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:54.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.108+0000 7fba137fe640 1 -- 172.21.15.43:0/238866551 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fba140f2eb0 con 0x7fba1c19cfd0 2024-04-21T08:43:54.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.108+0000 7fba1a59c640 1 --2- 172.21.15.43:0/555764438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641c427fea0 0x5641c4288540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:54.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.108+0000 7fba13fff640 1 -- 172.21.15.43:0/555764438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c4288a80 msgr2=0x5641c428ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:54.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.108+0000 7fba13fff640 1 --2- 172.21.15.43:0/555764438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c4288a80 0x5641c428ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.108+0000 7fba13fff640 1 -- 172.21.15.43:0/555764438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641c427fea0 msgr2=0x5641c4288540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:54.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.108+0000 7fba13fff640 1 --2- 172.21.15.43:0/555764438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641c427fea0 0x5641c4288540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.108+0000 7fba13fff640 1 -- 172.21.15.43:0/555764438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641c3fc07d0 con 0x5641c427f580 2024-04-21T08:43:54.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.109+0000 7fba1a59c640 1 --2- 172.21.15.43:0/555764438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641c427fea0 0x5641c4288540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:54.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.109+0000 7fba13fff640 1 --2- 172.21.15.43:0/555764438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c427f580 0x5641c427f960 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fba04013b10 tx=0x7fba04013f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6d5a090cb0fd30 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:54.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.109+0000 7fba13fff640 1 --2- 172.21.15.43:0/238866551 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fba080a3d20 0x7fba080a61e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fba04010800 tx=0x7fba04015000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:54.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.109+0000 7fba10ff9640 1 -- 172.21.15.43:0/555764438 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04013520 con 0x5641c427f580 2024-04-21T08:43:54.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.109+0000 7fba10ff9640 1 -- 172.21.15.43:0/555764438 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba040047a0 con 0x5641c427f580 2024-04-21T08:43:54.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.109+0000 7fba10ff9640 1 -- 172.21.15.43:0/555764438 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0400b910 con 0x5641c427f580 2024-04-21T08:43:54.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.110+0000 7fba23a2a740 1 -- 172.21.15.43:0/555764438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c427f580 msgr2=0x5641c427f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:54.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.110+0000 7fba23a2a740 1 --2- 172.21.15.43:0/555764438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c427f580 0x5641c427f960 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fba04013b10 tx=0x7fba04013f10 comp rx=0 tx=0).stop 2024-04-21T08:43:54.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.110+0000 7fba23a2a740 1 -- 172.21.15.43:0/555764438 shutdown_connections 2024-04-21T08:43:54.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.110+0000 7fba23a2a740 1 --2- 172.21.15.43:0/555764438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c4288a80 0x5641c428ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.110+0000 7fba23a2a740 1 --2- 172.21.15.43:0/555764438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641c427fea0 0x5641c4288540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.110+0000 7fba23a2a740 1 --2- 172.21.15.43:0/555764438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c427f580 0x5641c427f960 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.110+0000 7fba23a2a740 1 -- 172.21.15.43:0/555764438 >> 172.21.15.43:0/555764438 conn(0x5641c4279cb0 msgr2=0x5641c427a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:54.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.110+0000 7fba23a2a740 1 -- 172.21.15.43:0/555764438 shutdown_connections 2024-04-21T08:43:54.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.110+0000 7fba23a2a740 1 -- 172.21.15.43:0/555764438 wait complete. 2024-04-21T08:43:54.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.110+0000 7fba23a2a740 1 -- start start 2024-04-21T08:43:54.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.110+0000 7fba23a2a740 1 Processor -- start 2024-04-21T08:43:54.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.110+0000 7fba23a2a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c427f580 0x5641c4285ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:54.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.110+0000 7fba23a2a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641c427fea0 0x5641c4286020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.110+0000 7fba23a2a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c4288a80 0x5641c4286560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.110+0000 7fba23a2a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5641c428b810 con 0x5641c4288a80 2024-04-21T08:43:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.110+0000 7fba23a2a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5641c419d2f0 con 0x5641c427fea0 2024-04-21T08:43:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.110+0000 7fba23a2a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5641c428b990 con 0x5641c427f580 2024-04-21T08:43:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.111+0000 7fba23a2a740 10 client.0 _ll_register_callbacks cb 0x5641c41aa580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.111+0000 7fba1ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c4288a80 0x5641c4286560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.111+0000 7fba1ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c4288a80 0x5641c4286560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37366/0 (socket says 172.21.15.43:37366) 2024-04-21T08:43:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.111+0000 7fba1ad9d640 1 -- 172.21.15.43:0/1564057287 learned_addr learned my addr 172.21.15.43:0/1564057287 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.111+0000 7fba1a59c640 1 --2- 172.21.15.43:0/1564057287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c427f580 0x5641c4285ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.111+0000 7fba13fff640 1 --2- 172.21.15.43:0/1564057287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641c427fea0 0x5641c4286020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.111+0000 7fba1ad9d640 1 -- 172.21.15.43:0/1564057287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c427f580 msgr2=0x5641c4285ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.111+0000 7fba1ad9d640 1 --2- 172.21.15.43:0/1564057287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c427f580 0x5641c4285ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.111+0000 7fba1ad9d640 1 -- 172.21.15.43:0/1564057287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641c427fea0 msgr2=0x5641c4286020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.111+0000 7fba1ad9d640 1 --2- 172.21.15.43:0/1564057287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641c427fea0 0x5641c4286020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.111+0000 7fba1ad9d640 1 -- 172.21.15.43:0/1564057287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641c3ef63f0 con 0x5641c4288a80 2024-04-21T08:43:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.111+0000 7fba1a59c640 1 --2- 172.21.15.43:0/1564057287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c427f580 0x5641c4285ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.111+0000 7fba1ad9d640 1 --2- 172.21.15.43:0/1564057287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c4288a80 0x5641c4286560 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fba140baf90 tx=0x7fba140be550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2908974f4210d1d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.111+0000 7fba01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.111+0000 7fba01ffb640 1 -- 172.21.15.43:0/1564057287 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba140bfe60 con 0x5641c4288a80 2024-04-21T08:43:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.111+0000 7fba23a2a740 1 -- 172.21.15.43:0/1564057287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba040047a0 con 0x5641c4288a80 2024-04-21T08:43:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.111+0000 7fba01ffb640 1 -- 172.21.15.43:0/1564057287 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fba140327b0 con 0x5641c4288a80 2024-04-21T08:43:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.111+0000 7fba01ffb640 1 -- 172.21.15.43:0/1564057287 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fba140bf230 con 0x5641c4288a80 2024-04-21T08:43:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.111+0000 7fba23a2a740 10 client.17811 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:54.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.111+0000 7fba23a2a740 1 -- 172.21.15.43:0/1564057287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5641c3eb60f0 con 0x5641c4288a80 2024-04-21T08:43:54.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.111+0000 7fba01ffb640 1 -- 172.21.15.43:0/1564057287 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba040047a0 con 0x5641c4288a80 2024-04-21T08:43:54.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.112+0000 7fba01ffb640 1 -- 172.21.15.43:0/1564057287 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fba1403c030 con 0x5641c4288a80 2024-04-21T08:43:54.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.112+0000 7fba23a2a740 10 client.17811 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:54.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.112+0000 7fba23a2a740 10 client.17811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:54.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.112+0000 7fba23a2a740 20 client.17811 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:54.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.112+0000 7fba23a2a740 1 --2- 172.21.15.43:0/1564057287 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5641c4298a10 0x5641c429ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:54.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.112+0000 7fba23a2a740 4 client.17811 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:54.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.112+0000 7fba23a2a740 1 -- 172.21.15.43:0/1564057287 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641c41f1c00 con 0x5641c4298a10 2024-04-21T08:43:54.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.112+0000 7fba1a59c640 1 --2- 172.21.15.43:0/1564057287 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5641c4298a10 0x5641c429ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:54.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.112+0000 7fba1a59c640 1 --2- 172.21.15.43:0/1564057287 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5641c4298a10 0x5641c429ae90 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e89439fb948835e7 server_cookie=c17e450b0bd7ed25 in_seq=0 out_seq=0 2024-04-21T08:43:54.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.112+0000 7fba01ffb640 10 client.17811 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:54.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.113+0000 7fba01ffb640 1 -- 172.21.15.43:0/1564057287 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5641c41f1c00 con 0x5641c4298a10 2024-04-21T08:43:54.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.113+0000 7fba01ffb640 10 client.17811 handle_command_reply: tid=0 2024-04-21T08:43:54.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.113+0000 7fba23a2a740 1 client.17811 shutdown 2024-04-21T08:43:54.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.113+0000 7fba23a2a740 1 -- 172.21.15.43:0/1564057287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c4288a80 msgr2=0x5641c4286560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:54.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.113+0000 7fba23a2a740 1 --2- 172.21.15.43:0/1564057287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c4288a80 0x5641c4286560 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fba140baf90 tx=0x7fba140be550 comp rx=0 tx=0).stop 2024-04-21T08:43:54.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.113+0000 7fba23a2a740 1 -- 172.21.15.43:0/1564057287 shutdown_connections 2024-04-21T08:43:54.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.113+0000 7fba23a2a740 1 --2- 172.21.15.43:0/1564057287 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5641c4298a10 0x5641c429ae90 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.113+0000 7fba23a2a740 1 --2- 172.21.15.43:0/1564057287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c4288a80 0x5641c4286560 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.113+0000 7fba23a2a740 1 --2- 172.21.15.43:0/1564057287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641c427fea0 0x5641c4286020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.113+0000 7fba01ffb640 0 client.17811 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:54.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.113+0000 7fba23a2a740 1 --2- 172.21.15.43:0/1564057287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c427f580 0x5641c4285ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.113+0000 7fba23a2a740 1 -- 172.21.15.43:0/1564057287 >> 172.21.15.43:0/1564057287 conn(0x5641c4279cb0 msgr2=0x5641c427ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:54.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.113+0000 7fba23a2a740 1 -- 172.21.15.43:0/1564057287 shutdown_connections 2024-04-21T08:43:54.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.113+0000 7fba23a2a740 1 -- 172.21.15.43:0/1564057287 wait complete. 2024-04-21T08:43:54.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.113+0000 7fba23a2a740 20 client.17811 trim_cache size 0 max 16384 2024-04-21T08:43:54.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.131+0000 7fba23a2a740 1 Processor -- start 2024-04-21T08:43:54.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.131+0000 7fba23a2a740 1 -- start start 2024-04-21T08:43:54.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.131+0000 7fba23a2a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c427f580 0x5641c4294700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:54.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.131+0000 7fba23a2a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c427fea0 0x5641c4278d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:54.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.131+0000 7fba23a2a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641c4288a80 0x5641c4290ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:54.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.131+0000 7fba23a2a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c1af3b0 con 0x5641c427fea0 2024-04-21T08:43:54.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.131+0000 7fba23a2a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c1b0ef0 con 0x5641c4288a80 2024-04-21T08:43:54.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.131+0000 7fba23a2a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c1afa80 con 0x5641c427f580 2024-04-21T08:43:54.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.131+0000 7fba13fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c427fea0 0x5641c4278d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:54.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.131+0000 7fba13fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c427fea0 0x5641c4278d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37378/0 (socket says 172.21.15.43:37378) 2024-04-21T08:43:54.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.131+0000 7fba13fff640 1 -- 172.21.15.43:0/1496044583 learned_addr learned my addr 172.21.15.43:0/1496044583 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:54.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.132+0000 7fba1a59c640 1 --2- 172.21.15.43:0/1496044583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c427f580 0x5641c4294700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:54.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.132+0000 7fba1ad9d640 1 --2- 172.21.15.43:0/1496044583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641c4288a80 0x5641c4290ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:54.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.132+0000 7fba13fff640 1 -- 172.21.15.43:0/1496044583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c427f580 msgr2=0x5641c4294700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:54.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.132+0000 7fba13fff640 1 --2- 172.21.15.43:0/1496044583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c427f580 0x5641c4294700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.132+0000 7fba13fff640 1 -- 172.21.15.43:0/1496044583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641c4288a80 msgr2=0x5641c4290ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:54.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.132+0000 7fba13fff640 1 --2- 172.21.15.43:0/1496044583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641c4288a80 0x5641c4290ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.132+0000 7fba13fff640 1 -- 172.21.15.43:0/1496044583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641c427fcd0 con 0x5641c427fea0 2024-04-21T08:43:54.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.132+0000 7fba1ad9d640 1 --2- 172.21.15.43:0/1496044583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641c4288a80 0x5641c4290ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:54.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.132+0000 7fba13fff640 1 --2- 172.21.15.43:0/1496044583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c427fea0 0x5641c4278d70 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fba04005e70 tx=0x7fba04013760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d276f99bb838714 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:54.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.132+0000 7fba03fff640 1 -- 172.21.15.43:0/1496044583 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04034070 con 0x5641c427fea0 2024-04-21T08:43:54.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.132+0000 7fba03fff640 1 -- 172.21.15.43:0/1496044583 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba0400b6a0 con 0x5641c427fea0 2024-04-21T08:43:54.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.132+0000 7fba03fff640 1 -- 172.21.15.43:0/1496044583 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04005350 con 0x5641c427fea0 2024-04-21T08:43:54.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.133+0000 7fba23a2a740 1 -- 172.21.15.43:0/1496044583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c427fea0 msgr2=0x5641c4278d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:54.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.133+0000 7fba23a2a740 1 --2- 172.21.15.43:0/1496044583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c427fea0 0x5641c4278d70 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fba04005e70 tx=0x7fba04013760 comp rx=0 tx=0).stop 2024-04-21T08:43:54.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.133+0000 7fba23a2a740 1 -- 172.21.15.43:0/1496044583 shutdown_connections 2024-04-21T08:43:54.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.133+0000 7fba23a2a740 1 --2- 172.21.15.43:0/1496044583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641c4288a80 0x5641c4290ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.133+0000 7fba23a2a740 1 --2- 172.21.15.43:0/1496044583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c427fea0 0x5641c4278d70 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.133+0000 7fba23a2a740 1 --2- 172.21.15.43:0/1496044583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c427f580 0x5641c4294700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.133+0000 7fba23a2a740 1 -- 172.21.15.43:0/1496044583 >> 172.21.15.43:0/1496044583 conn(0x5641c4279cb0 msgr2=0x5641c4285aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:54.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.133+0000 7fba23a2a740 1 -- 172.21.15.43:0/1496044583 shutdown_connections 2024-04-21T08:43:54.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.133+0000 7fba23a2a740 1 -- 172.21.15.43:0/1496044583 wait complete. 2024-04-21T08:43:54.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.133+0000 7fba23a2a740 1 -- start start 2024-04-21T08:43:54.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.133+0000 7fba23a2a740 1 Processor -- start 2024-04-21T08:43:54.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.133+0000 7fba23a2a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c427f580 0x5641c4282180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:54.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.133+0000 7fba23a2a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c427fea0 0x5641c42826c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:54.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.133+0000 7fba23a2a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641c4288a80 0x5641c4303050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:54.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.133+0000 7fba23a2a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c1b0820 con 0x5641c427fea0 2024-04-21T08:43:54.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.133+0000 7fba23a2a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c1b0150 con 0x5641c4288a80 2024-04-21T08:43:54.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.133+0000 7fba23a2a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c1ad390 con 0x5641c427f580 2024-04-21T08:43:54.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.133+0000 7fba23a2a740 10 client.0 _ll_register_callbacks cb 0x7fba140badd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:54.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.133+0000 7fba13fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c427fea0 0x5641c42826c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:54.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.133+0000 7fba13fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c427fea0 0x5641c42826c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37384/0 (socket says 172.21.15.43:37384) 2024-04-21T08:43:54.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.133+0000 7fba13fff640 1 -- 172.21.15.43:0/1970710577 learned_addr learned my addr 172.21.15.43:0/1970710577 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:43:54.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.134+0000 7fba1ad9d640 1 --2- 172.21.15.43:0/1970710577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641c4288a80 0x5641c4303050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:54.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.134+0000 7fba1a59c640 1 --2- 172.21.15.43:0/1970710577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c427f580 0x5641c4282180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:54.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.134+0000 7fba13fff640 1 -- 172.21.15.43:0/1970710577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c427f580 msgr2=0x5641c4282180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:54.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.134+0000 7fba13fff640 1 --2- 172.21.15.43:0/1970710577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c427f580 0x5641c4282180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.134+0000 7fba13fff640 1 -- 172.21.15.43:0/1970710577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641c4288a80 msgr2=0x5641c4303050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:54.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.134+0000 7fba13fff640 1 --2- 172.21.15.43:0/1970710577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641c4288a80 0x5641c4303050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.134+0000 7fba13fff640 1 -- 172.21.15.43:0/1970710577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641c4303590 con 0x5641c427fea0 2024-04-21T08:43:54.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.134+0000 7fba1a59c640 1 --2- 172.21.15.43:0/1970710577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c427f580 0x5641c4282180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:54.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.134+0000 7fba13fff640 1 --2- 172.21.15.43:0/1970710577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c427fea0 0x5641c42826c0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fba0400f010 tx=0x7fba0400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16882b0231889a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:54.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.134+0000 7fba01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:43:54.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.134+0000 7fba01ffb640 1 -- 172.21.15.43:0/1970710577 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04034070 con 0x5641c427fea0 2024-04-21T08:43:54.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.134+0000 7fba01ffb640 1 -- 172.21.15.43:0/1970710577 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fba04005250 con 0x5641c427fea0 2024-04-21T08:43:54.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.134+0000 7fba23a2a740 1 -- 172.21.15.43:0/1970710577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba0400b6a0 con 0x5641c427fea0 2024-04-21T08:43:54.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.134+0000 7fba01ffb640 1 -- 172.21.15.43:0/1970710577 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fba04013990 con 0x5641c427fea0 2024-04-21T08:43:54.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.134+0000 7fba23a2a740 10 client.17817 fetch_fsmap learned FSMap version 28 2024-04-21T08:43:54.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.134+0000 7fba23a2a740 1 -- 172.21.15.43:0/1970710577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5641c43047a0 con 0x5641c427fea0 2024-04-21T08:43:54.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.134+0000 7fba01ffb640 1 -- 172.21.15.43:0/1970710577 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba0400b6a0 con 0x5641c427fea0 2024-04-21T08:43:54.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.134+0000 7fba01ffb640 1 -- 172.21.15.43:0/1970710577 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fba0401c070 con 0x5641c427fea0 2024-04-21T08:43:54.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.135+0000 7fba23a2a740 10 client.17817 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:43:54.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.135+0000 7fba23a2a740 10 client.17817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:54.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.135+0000 7fba23a2a740 20 client.17817 populate_metadata read hostname 'smithi043' 2024-04-21T08:43:54.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.135+0000 7fba23a2a740 1 --2- 172.21.15.43:0/1970710577 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5641c4304c70 0x5641c4307060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:54.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.135+0000 7fba23a2a740 4 client.17817 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:43:54.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.135+0000 7fba23a2a740 1 -- 172.21.15.43:0/1970710577 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5641c41eba70 con 0x5641c4304c70 2024-04-21T08:43:54.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.135+0000 7fba1a59c640 1 --2- 172.21.15.43:0/1970710577 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5641c4304c70 0x5641c4307060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:54.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.135+0000 7fba1a59c640 1 --2- 172.21.15.43:0/1970710577 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5641c4304c70 0x5641c4307060 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfe45848f13245d6 server_cookie=4b13a62b41b75d09 in_seq=0 out_seq=0 2024-04-21T08:43:54.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.135+0000 7fba01ffb640 10 client.17817 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:54.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.135+0000 7fba01ffb640 1 -- 172.21.15.43:0/1970710577 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5641c41eba70 con 0x5641c4304c70 2024-04-21T08:43:54.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.135+0000 7fba01ffb640 10 client.17817 handle_command_reply: tid=0 2024-04-21T08:43:54.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.135+0000 7fba23a2a740 1 client.17817 shutdown 2024-04-21T08:43:54.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.135+0000 7fba23a2a740 1 -- 172.21.15.43:0/1970710577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c427fea0 msgr2=0x5641c42826c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:54.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.135+0000 7fba23a2a740 1 --2- 172.21.15.43:0/1970710577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c427fea0 0x5641c42826c0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fba0400f010 tx=0x7fba0400bfd0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.136+0000 7fba23a2a740 1 -- 172.21.15.43:0/1970710577 shutdown_connections 2024-04-21T08:43:54.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.136+0000 7fba23a2a740 1 --2- 172.21.15.43:0/1970710577 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5641c4304c70 0x5641c4307060 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.136+0000 7fba23a2a740 1 --2- 172.21.15.43:0/1970710577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641c4288a80 0x5641c4303050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.136+0000 7fba23a2a740 1 --2- 172.21.15.43:0/1970710577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641c427fea0 0x5641c42826c0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.136+0000 7fba01ffb640 0 client.17817 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:43:54.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.136+0000 7fba23a2a740 1 --2- 172.21.15.43:0/1970710577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641c427f580 0x5641c4282180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.136+0000 7fba23a2a740 1 -- 172.21.15.43:0/1970710577 >> 172.21.15.43:0/1970710577 conn(0x5641c4279cb0 msgr2=0x5641c4286330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:54.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.136+0000 7fba23a2a740 1 -- 172.21.15.43:0/1970710577 shutdown_connections 2024-04-21T08:43:54.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.136+0000 7fba23a2a740 1 -- 172.21.15.43:0/1970710577 wait complete. 2024-04-21T08:43:54.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.136+0000 7fba23a2a740 20 client.17817 trim_cache size 0 max 16384 2024-04-21T08:43:54.142 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:43:54.143 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:43:54.143 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-21T08:43:54.143 INFO:teuthology.orchestra.run.smithi043.stdout: "1fd9e8ef-e437-4697-a4a3-3d32a551d199": { 2024-04-21T08:43:54.143 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-21T08:43:54.143 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "1fd9e8ef-e437-4697-a4a3-3d32a551d199", 2024-04-21T08:43:54.143 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-21T08:43:54.143 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:43:54.143 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:43:54.143 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:43:54.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.136+0000 7fba20e3e640 1 -- 172.21.15.43:0/238866551 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fba080a3d20 msgr2=0x7fba080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:54.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.136+0000 7fba20e3e640 1 --2- 172.21.15.43:0/238866551 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fba080a3d20 0x7fba080a61e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fba04010800 tx=0x7fba04015000 comp rx=0 tx=0).stop 2024-04-21T08:43:54.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.136+0000 7fba20e3e640 1 -- 172.21.15.43:0/238866551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba1c19cfd0 msgr2=0x7fba1c19d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:54.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.136+0000 7fba20e3e640 1 --2- 172.21.15.43:0/238866551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba1c19cfd0 0x7fba1c19d480 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fba14005e30 tx=0x7fba14034760 comp rx=0 tx=0).stop 2024-04-21T08:43:54.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.136+0000 7fba20e3e640 1 -- 172.21.15.43:0/238866551 shutdown_connections 2024-04-21T08:43:54.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.136+0000 7fba20e3e640 1 --2- 172.21.15.43:0/238866551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba1c19cfd0 0x7fba1c19d480 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.136+0000 7fba20e3e640 1 --2- 172.21.15.43:0/238866551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba1c106150 0x7fba1c198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.136+0000 7fba20e3e640 1 --2- 172.21.15.43:0/238866551 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fba080a3d20 0x7fba080a61e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.136+0000 7fba20e3e640 1 --2- 172.21.15.43:0/238866551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba1c101080 0x7fba1c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.136+0000 7fba20e3e640 1 -- 172.21.15.43:0/238866551 >> 172.21.15.43:0/238866551 conn(0x7fba1c078000 msgr2=0x7fba1c104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:54.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.136+0000 7fba20e3e640 1 -- 172.21.15.43:0/238866551 shutdown_connections 2024-04-21T08:43:54.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:43:54.136+0000 7fba20e3e640 1 -- 172.21.15.43:0/238866551 wait complete. 2024-04-21T08:43:54.146 INFO:tasks.cephfs.filesystem:scrub status for tag:1fd9e8ef-e437-4697-a4a3-3d32a551d199 - {'path': '/', 'tag': '1fd9e8ef-e437-4697-a4a3-3d32a551d199', 'options': 'recursive,force'} 2024-04-21T08:43:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:54 smithi112 ceph-mon[31085]: pgmap v152: 129 pgs: 129 active+clean; 2.5 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-04-21T08:43:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:54 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:43:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:54 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:43:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:54 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:43:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:54 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:43:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:54 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:43:54.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:54 smithi043 ceph-mon[27978]: pgmap v152: 129 pgs: 129 active+clean; 2.5 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-04-21T08:43:54.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:54 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:43:54.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:54 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:43:54.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:54 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:43:54.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:54 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:43:54.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:54 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:43:54.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:54 smithi175 ceph-mon[31920]: pgmap v152: 129 pgs: 129 active+clean; 2.5 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-04-21T08:43:54.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:54 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:43:54.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:54 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:43:54.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:54 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:43:54.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:54 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:43:54.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:54 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:43:56.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:56 smithi112 ceph-mon[31085]: pgmap v153: 129 pgs: 129 active+clean; 2.5 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-04-21T08:43:56.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:56 smithi043 ceph-mon[27978]: pgmap v153: 129 pgs: 129 active+clean; 2.5 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-04-21T08:43:56.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:56 smithi175 ceph-mon[31920]: pgmap v153: 129 pgs: 129 active+clean; 2.5 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-04-21T08:43:58.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:43:58 smithi043 ceph-mon[27978]: pgmap v154: 129 pgs: 129 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 23 op/s 2024-04-21T08:43:58.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:43:58 smithi175 ceph-mon[31920]: pgmap v154: 129 pgs: 129 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 23 op/s 2024-04-21T08:43:58.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:43:58 smithi112 ceph-mon[31085]: pgmap v154: 129 pgs: 129 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 23 op/s 2024-04-21T08:44:00.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:00 smithi043 ceph-mon[27978]: pgmap v155: 129 pgs: 129 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-04-21T08:44:00.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:00 smithi175 ceph-mon[31920]: pgmap v155: 129 pgs: 129 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-04-21T08:44:00.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:00 smithi112 ceph-mon[31085]: pgmap v155: 129 pgs: 129 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-04-21T08:44:02.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:02 smithi043 ceph-mon[27978]: pgmap v156: 129 pgs: 129 active+clean; 2.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 70 KiB/s wr, 38 op/s 2024-04-21T08:44:02.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:02 smithi175 ceph-mon[31920]: pgmap v156: 129 pgs: 129 active+clean; 2.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 70 KiB/s wr, 38 op/s 2024-04-21T08:44:02.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:02 smithi112 ceph-mon[31085]: pgmap v156: 129 pgs: 129 active+clean; 2.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 70 KiB/s wr, 38 op/s 2024-04-21T08:44:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.413+0000 7f64c1269640 1 -- 172.21.15.43:0/1157640207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64bc103200 msgr2=0x7f64bc1035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.413+0000 7f64c1269640 1 --2- 172.21.15.43:0/1157640207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64bc103200 0x7f64bc1035e0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f64ac009900 tx=0x7f64ac02bcd0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.413+0000 7f64c1269640 1 -- 172.21.15.43:0/1157640207 shutdown_connections 2024-04-21T08:44:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.413+0000 7f64c1269640 1 --2- 172.21.15.43:0/1157640207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64bc10c5d0 0x7f64bc10ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.413+0000 7f64c1269640 1 --2- 172.21.15.43:0/1157640207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64bc103b20 0x7f64bc10c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.413+0000 7f64c1269640 1 --2- 172.21.15.43:0/1157640207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64bc103200 0x7f64bc1035e0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.413+0000 7f64c1269640 1 -- 172.21.15.43:0/1157640207 >> 172.21.15.43:0/1157640207 conn(0x7f64bc0771d0 msgr2=0x7f64bc0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.413+0000 7f64c1269640 1 -- 172.21.15.43:0/1157640207 shutdown_connections 2024-04-21T08:44:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.413+0000 7f64c1269640 1 -- 172.21.15.43:0/1157640207 wait complete. 2024-04-21T08:44:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.414+0000 7f64c1269640 1 Processor -- start 2024-04-21T08:44:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.414+0000 7f64c1269640 1 -- start start 2024-04-21T08:44:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.414+0000 7f64c1269640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64bc103200 0x7f64bc198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.414+0000 7f64c1269640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64bc103b20 0x7f64bc198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.414+0000 7f64c1269640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64bc10c5d0 0x7f64bc19cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.414+0000 7f64c1269640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc112270 con 0x7f64bc103b20 2024-04-21T08:44:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.414+0000 7f64c1269640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc1120f0 con 0x7f64bc10c5d0 2024-04-21T08:44:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.414+0000 7f64c1269640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc1123f0 con 0x7f64bc103200 2024-04-21T08:44:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.414+0000 7f64ba7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64bc103b20 0x7f64bc198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.414+0000 7f64ba7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64bc103b20 0x7f64bc198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33868/0 (socket says 172.21.15.43:33868) 2024-04-21T08:44:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.414+0000 7f64ba7fc640 1 -- 172.21.15.43:0/659111495 learned_addr learned my addr 172.21.15.43:0/659111495 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.414+0000 7f64baffd640 1 --2- 172.21.15.43:0/659111495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64bc103200 0x7f64bc198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.415+0000 7f64bb7fe640 1 --2- 172.21.15.43:0/659111495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64bc10c5d0 0x7f64bc19cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.415+0000 7f64ba7fc640 1 -- 172.21.15.43:0/659111495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64bc103200 msgr2=0x7f64bc198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.415+0000 7f64ba7fc640 1 --2- 172.21.15.43:0/659111495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64bc103200 0x7f64bc198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.415+0000 7f64ba7fc640 1 -- 172.21.15.43:0/659111495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64bc10c5d0 msgr2=0x7f64bc19cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.415+0000 7f64ba7fc640 1 --2- 172.21.15.43:0/659111495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64bc10c5d0 0x7f64bc19cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.415+0000 7f64ba7fc640 1 -- 172.21.15.43:0/659111495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64bc19d710 con 0x7f64bc103b20 2024-04-21T08:44:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.415+0000 7f64bb7fe640 1 --2- 172.21.15.43:0/659111495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64bc10c5d0 0x7f64bc19cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.415+0000 7f64ba7fc640 1 --2- 172.21.15.43:0/659111495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64bc103b20 0x7f64bc198cb0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f64b000d950 tx=0x7f64b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.415+0000 7f64a3fff640 1 -- 172.21.15.43:0/659111495 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0017070 con 0x7f64bc103b20 2024-04-21T08:44:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.415+0000 7f64a3fff640 1 -- 172.21.15.43:0/659111495 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64b0009c30 con 0x7f64bc103b20 2024-04-21T08:44:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.415+0000 7f64c1269640 1 -- 172.21.15.43:0/659111495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64bc19da00 con 0x7f64bc103b20 2024-04-21T08:44:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.415+0000 7f64a3fff640 1 -- 172.21.15.43:0/659111495 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b000f620 con 0x7f64bc103b20 2024-04-21T08:44:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.415+0000 7f64c1269640 1 -- 172.21.15.43:0/659111495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64bc1a5290 con 0x7f64bc103b20 2024-04-21T08:44:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.417+0000 7f64a3fff640 1 -- 172.21.15.43:0/659111495 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f64b0010460 con 0x7f64bc103b20 2024-04-21T08:44:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.418+0000 7f64c3e56740 1 Processor -- start 2024-04-21T08:44:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.418+0000 7f64c3e56740 1 -- start start 2024-04-21T08:44:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.418+0000 7f64c3e56740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc8fd08d0 0x564fc8fcf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.418+0000 7f64c3e56740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fc8fcfb60 0x564fc8fcffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.418+0000 7f64c3e56740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fd90e0 0x564fc8fdb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.418+0000 7f64c3e56740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564fc8f348d0 con 0x564fc8fd90e0 2024-04-21T08:44:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.418+0000 7f64c3e56740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564fc8ed1e80 con 0x564fc8fcfb60 2024-04-21T08:44:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.418+0000 7f64c3e56740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564fc8f30890 con 0x564fc8fd08d0 2024-04-21T08:44:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.418+0000 7f64a3fff640 1 --2- 172.21.15.43:0/659111495 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f64a40a3dc0 0x7f64a40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.418+0000 7f64ba7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fd90e0 0x564fc8fdb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.418+0000 7f64bb7fe640 1 --2- 172.21.15.43:0/659111495 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f64a40a3dc0 0x7f64a40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.418+0000 7f64ba7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fd90e0 0x564fc8fdb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33874/0 (socket says 172.21.15.43:33874) 2024-04-21T08:44:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.418+0000 7f64ba7fc640 1 -- 172.21.15.43:0/4218577014 learned_addr learned my addr 172.21.15.43:0/4218577014 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.418+0000 7f64baffd640 1 --2- 172.21.15.43:0/4218577014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fc8fcfb60 0x564fc8fcffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.418+0000 7f64bb7fe640 1 --2- 172.21.15.43:0/4218577014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc8fd08d0 0x564fc8fcf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.418+0000 7f64a3fff640 1 -- 172.21.15.43:0/659111495 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f64b00d0830 con 0x7f64bc103b20 2024-04-21T08:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.419+0000 7f64ba7fc640 1 -- 172.21.15.43:0/4218577014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc8fd08d0 msgr2=0x564fc8fcf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.419+0000 7f64ba7fc640 1 --2- 172.21.15.43:0/4218577014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc8fd08d0 0x564fc8fcf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.419+0000 7f64ba7fc640 1 -- 172.21.15.43:0/4218577014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fc8fcfb60 msgr2=0x564fc8fcffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.419+0000 7f64ba7fc640 1 --2- 172.21.15.43:0/4218577014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fc8fcfb60 0x564fc8fcffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.419+0000 7f64ba7fc640 1 -- 172.21.15.43:0/4218577014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fc8ef3320 con 0x564fc8fd90e0 2024-04-21T08:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.419+0000 7f64baffd640 1 --2- 172.21.15.43:0/4218577014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fc8fcfb60 0x564fc8fcffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.419+0000 7f64bb7fe640 1 --2- 172.21.15.43:0/659111495 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f64a40a3dc0 0x7f64a40a6280 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f64b4009fd0 tx=0x7f64b40091e0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.419+0000 7f64ba7fc640 1 --2- 172.21.15.43:0/4218577014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fd90e0 0x564fc8fdb4d0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f64b0098dd0 tx=0x7f64b006eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a302bda185d871e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.419+0000 7f64a17fa640 1 -- 172.21.15.43:0/4218577014 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0020070 con 0x564fc8fd90e0 2024-04-21T08:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.419+0000 7f64a17fa640 1 -- 172.21.15.43:0/4218577014 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64b006deb0 con 0x564fc8fd90e0 2024-04-21T08:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.419+0000 7f64a17fa640 1 -- 172.21.15.43:0/4218577014 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b009c400 con 0x564fc8fd90e0 2024-04-21T08:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.420+0000 7f64c3e56740 1 -- 172.21.15.43:0/4218577014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fd90e0 msgr2=0x564fc8fdb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.420+0000 7f64c3e56740 1 --2- 172.21.15.43:0/4218577014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fd90e0 0x564fc8fdb4d0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f64b0098dd0 tx=0x7f64b006eef0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.420+0000 7f64c3e56740 1 -- 172.21.15.43:0/4218577014 shutdown_connections 2024-04-21T08:44:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.420+0000 7f64c3e56740 1 --2- 172.21.15.43:0/4218577014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fd90e0 0x564fc8fdb4d0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.420+0000 7f64c3e56740 1 --2- 172.21.15.43:0/4218577014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fc8fcfb60 0x564fc8fcffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.420+0000 7f64c3e56740 1 --2- 172.21.15.43:0/4218577014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc8fd08d0 0x564fc8fcf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.420+0000 7f64c3e56740 1 -- 172.21.15.43:0/4218577014 >> 172.21.15.43:0/4218577014 conn(0x564fc8f96c90 msgr2=0x564fc8fca850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.420+0000 7f64c3e56740 1 -- 172.21.15.43:0/4218577014 shutdown_connections 2024-04-21T08:44:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.420+0000 7f64c3e56740 1 -- 172.21.15.43:0/4218577014 wait complete. 2024-04-21T08:44:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.420+0000 7f64c3e56740 1 -- start start 2024-04-21T08:44:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.420+0000 7f64c3e56740 1 Processor -- start 2024-04-21T08:44:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.421+0000 7f64c3e56740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fcfb60 0x564fc8fd8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.421+0000 7f64c3e56740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fc8fd08d0 0x564fc8fd5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.421+0000 7f64c3e56740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc8fd90e0 0x564fc8fd62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.421+0000 7f64c3e56740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564fc8fdbe10 con 0x564fc8fcfb60 2024-04-21T08:44:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.421+0000 7f64c3e56740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564fc8ebceb0 con 0x564fc8fd08d0 2024-04-21T08:44:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.421+0000 7f64c3e56740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564fc8fdbf90 con 0x564fc8fd90e0 2024-04-21T08:44:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.421+0000 7f64c3e56740 10 client.0 _ll_register_callbacks cb 0x564fc8ee9b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.421+0000 7f64baffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fcfb60 0x564fc8fd8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.421+0000 7f64baffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fcfb60 0x564fc8fd8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33884/0 (socket says 172.21.15.43:33884) 2024-04-21T08:44:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.421+0000 7f64baffd640 1 -- 172.21.15.43:0/3915267753 learned_addr learned my addr 172.21.15.43:0/3915267753 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.421+0000 7f64bb7fe640 1 --2- 172.21.15.43:0/3915267753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc8fd90e0 0x564fc8fd62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.421+0000 7f64ba7fc640 1 --2- 172.21.15.43:0/3915267753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fc8fd08d0 0x564fc8fd5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.421+0000 7f64baffd640 1 -- 172.21.15.43:0/3915267753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc8fd90e0 msgr2=0x564fc8fd62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.421+0000 7f64baffd640 1 --2- 172.21.15.43:0/3915267753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc8fd90e0 0x564fc8fd62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.421+0000 7f64baffd640 1 -- 172.21.15.43:0/3915267753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fc8fd08d0 msgr2=0x564fc8fd5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.421+0000 7f64baffd640 1 --2- 172.21.15.43:0/3915267753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fc8fd08d0 0x564fc8fd5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.421+0000 7f64baffd640 1 -- 172.21.15.43:0/3915267753 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fc8c47210 con 0x564fc8fcfb60 2024-04-21T08:44:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.421+0000 7f64bb7fe640 1 --2- 172.21.15.43:0/3915267753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc8fd90e0 0x564fc8fd62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.421+0000 7f64baffd640 1 --2- 172.21.15.43:0/3915267753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fcfb60 0x564fc8fd8a00 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f64ac000c00 tx=0x7f64ac02cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12180727869b9c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.421+0000 7f649eff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.421+0000 7f649eff5640 1 -- 172.21.15.43:0/3915267753 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac043070 con 0x564fc8fcfb60 2024-04-21T08:44:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.421+0000 7f64c3e56740 1 -- 172.21.15.43:0/3915267753 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564fc8f48880 con 0x564fc8fcfb60 2024-04-21T08:44:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.421+0000 7f649eff5640 1 -- 172.21.15.43:0/3915267753 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f64ac005250 con 0x564fc8fcfb60 2024-04-21T08:44:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.422+0000 7f649eff5640 1 -- 172.21.15.43:0/3915267753 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f64ac04f050 con 0x564fc8fcfb60 2024-04-21T08:44:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.422+0000 7f64c3e56740 10 client.17829 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.422+0000 7f64c3e56740 1 -- 172.21.15.43:0/3915267753 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564fc8d13620 con 0x564fc8fcfb60 2024-04-21T08:44:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.422+0000 7f649eff5640 1 -- 172.21.15.43:0/3915267753 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564fc8f48880 con 0x564fc8fcfb60 2024-04-21T08:44:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.422+0000 7f649eff5640 1 -- 172.21.15.43:0/3915267753 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f64ac0035f0 con 0x564fc8fcfb60 2024-04-21T08:44:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.422+0000 7f64c3e56740 10 client.17829 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.422+0000 7f64c3e56740 10 client.17829 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.422+0000 7f64c3e56740 20 client.17829 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.422+0000 7f64c3e56740 1 --2- 172.21.15.43:0/3915267753 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564fc8fe9090 0x564fc8feb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.422+0000 7f64c3e56740 4 client.17829 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.422+0000 7f64c3e56740 1 -- 172.21.15.43:0/3915267753 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fc8ed43d0 con 0x564fc8fe9090 2024-04-21T08:44:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.422+0000 7f64ba7fc640 1 --2- 172.21.15.43:0/3915267753 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564fc8fe9090 0x564fc8feb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.423+0000 7f64ba7fc640 1 --2- 172.21.15.43:0/3915267753 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564fc8fe9090 0x564fc8feb480 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a5468f46e1d297d server_cookie=ba33903ab5b8ae3e in_seq=0 out_seq=0 2024-04-21T08:44:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.423+0000 7f649eff5640 10 client.17829 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.423+0000 7f649eff5640 1 -- 172.21.15.43:0/3915267753 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564fc8ed43d0 con 0x564fc8fe9090 2024-04-21T08:44:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.423+0000 7f649eff5640 10 client.17829 handle_command_reply: tid=0 2024-04-21T08:44:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.423+0000 7f64c3e56740 1 client.17829 shutdown 2024-04-21T08:44:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.423+0000 7f64c3e56740 1 -- 172.21.15.43:0/3915267753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fcfb60 msgr2=0x564fc8fd8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.423+0000 7f64c3e56740 1 --2- 172.21.15.43:0/3915267753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fcfb60 0x564fc8fd8a00 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f64ac000c00 tx=0x7f64ac02cab0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.423+0000 7f64c3e56740 1 -- 172.21.15.43:0/3915267753 shutdown_connections 2024-04-21T08:44:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.423+0000 7f64c3e56740 1 --2- 172.21.15.43:0/3915267753 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564fc8fe9090 0x564fc8feb480 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.423+0000 7f64c3e56740 1 --2- 172.21.15.43:0/3915267753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc8fd90e0 0x564fc8fd62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.423+0000 7f64c3e56740 1 --2- 172.21.15.43:0/3915267753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fc8fd08d0 0x564fc8fd5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.423+0000 7f64c3e56740 1 --2- 172.21.15.43:0/3915267753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fcfb60 0x564fc8fd8a00 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.423+0000 7f649eff5640 0 client.17829 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.423+0000 7f64c3e56740 1 -- 172.21.15.43:0/3915267753 >> 172.21.15.43:0/3915267753 conn(0x564fc8f96c90 msgr2=0x564fc8fcc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.424+0000 7f64c3e56740 1 -- 172.21.15.43:0/3915267753 shutdown_connections 2024-04-21T08:44:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.424+0000 7f64c3e56740 1 -- 172.21.15.43:0/3915267753 wait complete. 2024-04-21T08:44:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.424+0000 7f64c3e56740 20 client.17829 trim_cache size 0 max 16384 2024-04-21T08:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.439+0000 7f64c3e56740 1 Processor -- start 2024-04-21T08:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.439+0000 7f64c3e56740 1 -- start start 2024-04-21T08:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.439+0000 7f64c3e56740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fc8f96c90 0x564fc8fe4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.439+0000 7f64c3e56740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fcfb60 0x564fc8fc9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.439+0000 7f64c3e56740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc8fd08d0 0x564fc8fd84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.439+0000 7f64c3e56740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc1af390 con 0x564fc8fcfb60 2024-04-21T08:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.439+0000 7f64c3e56740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc1b0ed0 con 0x564fc8f96c90 2024-04-21T08:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.439+0000 7f64c3e56740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc1afa60 con 0x564fc8fd08d0 2024-04-21T08:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.439+0000 7f64ba7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fcfb60 0x564fc8fc9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.439+0000 7f64ba7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fcfb60 0x564fc8fc9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33890/0 (socket says 172.21.15.43:33890) 2024-04-21T08:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.439+0000 7f64ba7fc640 1 -- 172.21.15.43:0/3791147578 learned_addr learned my addr 172.21.15.43:0/3791147578 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.439+0000 7f64baffd640 1 --2- 172.21.15.43:0/3791147578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fc8f96c90 0x564fc8fe4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.439+0000 7f64bb7fe640 1 --2- 172.21.15.43:0/3791147578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc8fd08d0 0x564fc8fd84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.439+0000 7f64ba7fc640 1 -- 172.21.15.43:0/3791147578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc8fd08d0 msgr2=0x564fc8fd84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.439+0000 7f64ba7fc640 1 --2- 172.21.15.43:0/3791147578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc8fd08d0 0x564fc8fd84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.439+0000 7f64ba7fc640 1 -- 172.21.15.43:0/3791147578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fc8f96c90 msgr2=0x564fc8fe4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.439+0000 7f64ba7fc640 1 --2- 172.21.15.43:0/3791147578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fc8f96c90 0x564fc8fe4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.439+0000 7f64ba7fc640 1 -- 172.21.15.43:0/3791147578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fc8fcee30 con 0x564fc8fcfb60 2024-04-21T08:44:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.439+0000 7f64ba7fc640 1 --2- 172.21.15.43:0/3791147578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fcfb60 0x564fc8fc9750 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f64b009c100 tx=0x7f64b00c7510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e92cde475cddfc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.439+0000 7f64a0ff9640 1 -- 172.21.15.43:0/3791147578 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0014030 con 0x564fc8fcfb60 2024-04-21T08:44:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.440+0000 7f64a0ff9640 1 -- 172.21.15.43:0/3791147578 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64b0085070 con 0x564fc8fcfb60 2024-04-21T08:44:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.440+0000 7f64a0ff9640 1 -- 172.21.15.43:0/3791147578 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0020040 con 0x564fc8fcfb60 2024-04-21T08:44:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.440+0000 7f64c3e56740 1 -- 172.21.15.43:0/3791147578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fcfb60 msgr2=0x564fc8fc9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.440+0000 7f64c3e56740 1 --2- 172.21.15.43:0/3791147578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fcfb60 0x564fc8fc9750 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f64b009c100 tx=0x7f64b00c7510 comp rx=0 tx=0).stop 2024-04-21T08:44:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.440+0000 7f64c3e56740 1 -- 172.21.15.43:0/3791147578 shutdown_connections 2024-04-21T08:44:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.440+0000 7f64c3e56740 1 --2- 172.21.15.43:0/3791147578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc8fd08d0 0x564fc8fd84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.440+0000 7f64c3e56740 1 --2- 172.21.15.43:0/3791147578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fcfb60 0x564fc8fc9750 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.440+0000 7f64c3e56740 1 --2- 172.21.15.43:0/3791147578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fc8f96c90 0x564fc8fe4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.440+0000 7f64c3e56740 1 -- 172.21.15.43:0/3791147578 >> 172.21.15.43:0/3791147578 conn(0x564fc8fd1760 msgr2=0x564fc8fc9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.440+0000 7f64c3e56740 1 -- 172.21.15.43:0/3791147578 shutdown_connections 2024-04-21T08:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.440+0000 7f64c3e56740 1 -- 172.21.15.43:0/3791147578 wait complete. 2024-04-21T08:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.440+0000 7f64c3e56740 1 -- start start 2024-04-21T08:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.441+0000 7f64c3e56740 1 Processor -- start 2024-04-21T08:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.441+0000 7f64c3e56740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc8f96c90 0x564fc8fe1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.441+0000 7f64c3e56740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fc8fcfb60 0x564fc8fdc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.441+0000 7f64c3e56740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fd08d0 0x564fc8fdcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.441+0000 7f64c3e56740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc1b0800 con 0x564fc8fd08d0 2024-04-21T08:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.441+0000 7f64c3e56740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc1b0130 con 0x564fc8fcfb60 2024-04-21T08:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.441+0000 7f64c3e56740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc1ad370 con 0x564fc8f96c90 2024-04-21T08:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.441+0000 7f64c3e56740 10 client.0 _ll_register_callbacks cb 0x7f64ac005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.441+0000 7f64bb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fd08d0 0x564fc8fdcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.441+0000 7f64bb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fd08d0 0x564fc8fdcd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33898/0 (socket says 172.21.15.43:33898) 2024-04-21T08:44:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.441+0000 7f64bb7fe640 1 -- 172.21.15.43:0/3977035183 learned_addr learned my addr 172.21.15.43:0/3977035183 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.441+0000 7f64baffd640 1 --2- 172.21.15.43:0/3977035183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc8f96c90 0x564fc8fe1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.441+0000 7f64ba7fc640 1 --2- 172.21.15.43:0/3977035183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fc8fcfb60 0x564fc8fdc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.441+0000 7f64bb7fe640 1 -- 172.21.15.43:0/3977035183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc8f96c90 msgr2=0x564fc8fe1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.441+0000 7f64bb7fe640 1 --2- 172.21.15.43:0/3977035183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc8f96c90 0x564fc8fe1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.441+0000 7f64bb7fe640 1 -- 172.21.15.43:0/3977035183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fc8fcfb60 msgr2=0x564fc8fdc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.441+0000 7f64bb7fe640 1 --2- 172.21.15.43:0/3977035183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fc8fcfb60 0x564fc8fdc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.441+0000 7f64bb7fe640 1 -- 172.21.15.43:0/3977035183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fc8fdd2b0 con 0x564fc8fd08d0 2024-04-21T08:44:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.441+0000 7f64baffd640 1 --2- 172.21.15.43:0/3977035183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc8f96c90 0x564fc8fe1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:44:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.441+0000 7f64bb7fe640 1 --2- 172.21.15.43:0/3977035183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fd08d0 0x564fc8fdcd70 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f64b4005fd0 tx=0x7f64b4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c63526d07e4ffdf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.441+0000 7f649eff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.442+0000 7f649eff5640 1 -- 172.21.15.43:0/3977035183 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b40045b0 con 0x564fc8fd08d0 2024-04-21T08:44:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.442+0000 7f649eff5640 1 -- 172.21.15.43:0/3977035183 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f64b4004750 con 0x564fc8fd08d0 2024-04-21T08:44:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.442+0000 7f64c3e56740 1 -- 172.21.15.43:0/3977035183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64b0085070 con 0x564fc8fd08d0 2024-04-21T08:44:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.442+0000 7f649eff5640 1 -- 172.21.15.43:0/3977035183 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f64b4012450 con 0x564fc8fd08d0 2024-04-21T08:44:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.442+0000 7f64c3e56740 10 client.17835 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.442+0000 7f64c3e56740 1 -- 172.21.15.43:0/3977035183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564fc904f360 con 0x564fc8fd08d0 2024-04-21T08:44:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.442+0000 7f649eff5640 1 -- 172.21.15.43:0/3977035183 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64b0085070 con 0x564fc8fd08d0 2024-04-21T08:44:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.442+0000 7f649eff5640 1 -- 172.21.15.43:0/3977035183 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f64b4024430 con 0x564fc8fd08d0 2024-04-21T08:44:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.442+0000 7f64c3e56740 10 client.17835 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.442+0000 7f64c3e56740 10 client.17835 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.442+0000 7f64c3e56740 20 client.17835 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.442+0000 7f64c3e56740 1 --2- 172.21.15.43:0/3977035183 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564fc8fd90e0 0x564fc9051b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.442+0000 7f64c3e56740 4 client.17835 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:44:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.442+0000 7f64c3e56740 1 -- 172.21.15.43:0/3977035183 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564fc8f422f0 con 0x564fc8fd90e0 2024-04-21T08:44:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.442+0000 7f64baffd640 1 --2- 172.21.15.43:0/3977035183 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564fc8fd90e0 0x564fc9051b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.443+0000 7f649eff5640 10 client.17835 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.443+0000 7f64baffd640 1 --2- 172.21.15.43:0/3977035183 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564fc8fd90e0 0x564fc9051b50 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=327c3e0cabab0c32 server_cookie=445a76be0cead9a2 in_seq=0 out_seq=0 2024-04-21T08:44:04.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.454+0000 7f649eff5640 1 -- 172.21.15.43:0/3977035183 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564fc8f422f0 con 0x564fc8fd90e0 2024-04-21T08:44:04.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.454+0000 7f649eff5640 10 client.17835 handle_command_reply: tid=0 2024-04-21T08:44:04.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.455+0000 7f64c3e56740 1 client.17835 shutdown 2024-04-21T08:44:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.455+0000 7f64c3e56740 1 -- 172.21.15.43:0/3977035183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fd08d0 msgr2=0x564fc8fdcd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.455+0000 7f64c3e56740 1 --2- 172.21.15.43:0/3977035183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fd08d0 0x564fc8fdcd70 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f64b4005fd0 tx=0x7f64b4004ab0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.455+0000 7f64c3e56740 1 -- 172.21.15.43:0/3977035183 shutdown_connections 2024-04-21T08:44:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.455+0000 7f64c3e56740 1 --2- 172.21.15.43:0/3977035183 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564fc8fd90e0 0x564fc9051b50 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.455+0000 7f64c3e56740 1 --2- 172.21.15.43:0/3977035183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc8fd08d0 0x564fc8fdcd70 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.455+0000 7f649eff5640 0 client.17835 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.455+0000 7f64c3e56740 1 --2- 172.21.15.43:0/3977035183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fc8fcfb60 0x564fc8fdc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.455+0000 7f64c3e56740 1 --2- 172.21.15.43:0/3977035183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fc8f96c90 0x564fc8fe1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.455+0000 7f64c3e56740 1 -- 172.21.15.43:0/3977035183 >> 172.21.15.43:0/3977035183 conn(0x564fc8fd1760 msgr2=0x564fc8fc98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.455+0000 7f64c3e56740 1 -- 172.21.15.43:0/3977035183 shutdown_connections 2024-04-21T08:44:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.455+0000 7f64c3e56740 1 -- 172.21.15.43:0/3977035183 wait complete. 2024-04-21T08:44:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.455+0000 7f64c3e56740 20 client.17835 trim_cache size 0 max 16384 2024-04-21T08:44:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:44:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:44:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:44:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:44:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.455+0000 7f64c1269640 1 -- 172.21.15.43:0/659111495 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f64a40a3dc0 msgr2=0x7f64a40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.455+0000 7f64c1269640 1 --2- 172.21.15.43:0/659111495 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f64a40a3dc0 0x7f64a40a6280 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f64b4009fd0 tx=0x7f64b40091e0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.455+0000 7f64c1269640 1 -- 172.21.15.43:0/659111495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64bc103b20 msgr2=0x7f64bc198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.455+0000 7f64c1269640 1 --2- 172.21.15.43:0/659111495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64bc103b20 0x7f64bc198cb0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f64b000d950 tx=0x7f64b000de10 comp rx=0 tx=0).stop 2024-04-21T08:44:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.456+0000 7f64c1269640 1 -- 172.21.15.43:0/659111495 shutdown_connections 2024-04-21T08:44:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.456+0000 7f64c1269640 1 --2- 172.21.15.43:0/659111495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64bc10c5d0 0x7f64bc19cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.456+0000 7f64c1269640 1 --2- 172.21.15.43:0/659111495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64bc103b20 0x7f64bc198cb0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.456+0000 7f64c1269640 1 --2- 172.21.15.43:0/659111495 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f64a40a3dc0 0x7f64a40a6280 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.456+0000 7f64c1269640 1 --2- 172.21.15.43:0/659111495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64bc103200 0x7f64bc198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.456+0000 7f64c1269640 1 -- 172.21.15.43:0/659111495 >> 172.21.15.43:0/659111495 conn(0x7f64bc0771d0 msgr2=0x7f64bc0fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.456+0000 7f64c1269640 1 -- 172.21.15.43:0/659111495 shutdown_connections 2024-04-21T08:44:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:04.456+0000 7f64c1269640 1 -- 172.21.15.43:0/659111495 wait complete. 2024-04-21T08:44:04.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:04 smithi043 ceph-mon[27978]: pgmap v157: 129 pgs: 129 active+clean; 2.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-21T08:44:04.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:04 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:44:04.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:04 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:04.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:04 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:44:04.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:04 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:44:04.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:04 smithi175 ceph-mon[31920]: pgmap v157: 129 pgs: 129 active+clean; 2.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-21T08:44:04.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:04 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:44:04.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:04 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:04.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:04 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:44:04.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:04 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:44:04.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:04 smithi112 ceph-mon[31085]: pgmap v157: 129 pgs: 129 active+clean; 2.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-21T08:44:04.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:04 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:44:04.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:04 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:04.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:04 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:44:04.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:04 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:44:06.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:06 smithi043 ceph-mon[27978]: pgmap v158: 129 pgs: 129 active+clean; 2.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-21T08:44:06.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:06 smithi175 ceph-mon[31920]: pgmap v158: 129 pgs: 129 active+clean; 2.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-21T08:44:06.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:06 smithi112 ceph-mon[31085]: pgmap v158: 129 pgs: 129 active+clean; 2.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-21T08:44:08.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:08 smithi043 ceph-mon[27978]: pgmap v159: 129 pgs: 129 active+clean; 3.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 75 KiB/s wr, 40 op/s 2024-04-21T08:44:08.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:08 smithi175 ceph-mon[31920]: pgmap v159: 129 pgs: 129 active+clean; 3.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 75 KiB/s wr, 40 op/s 2024-04-21T08:44:08.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:08 smithi112 ceph-mon[31085]: pgmap v159: 129 pgs: 129 active+clean; 3.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 75 KiB/s wr, 40 op/s 2024-04-21T08:44:10.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:10 smithi175 ceph-mon[31920]: pgmap v160: 129 pgs: 129 active+clean; 3.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 56 KiB/s wr, 30 op/s 2024-04-21T08:44:10.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:10 smithi112 ceph-mon[31085]: pgmap v160: 129 pgs: 129 active+clean; 3.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 56 KiB/s wr, 30 op/s 2024-04-21T08:44:10.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:10 smithi043 ceph-mon[27978]: pgmap v160: 129 pgs: 129 active+clean; 3.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 56 KiB/s wr, 30 op/s 2024-04-21T08:44:12.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:12 smithi175 ceph-mon[31920]: pgmap v161: 129 pgs: 129 active+clean; 3.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 71 KiB/s wr, 36 op/s 2024-04-21T08:44:12.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:12 smithi112 ceph-mon[31085]: pgmap v161: 129 pgs: 129 active+clean; 3.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 71 KiB/s wr, 36 op/s 2024-04-21T08:44:12.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:12 smithi043 ceph-mon[27978]: pgmap v161: 129 pgs: 129 active+clean; 3.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 71 KiB/s wr, 36 op/s 2024-04-21T08:44:14.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:14 smithi175 ceph-mon[31920]: pgmap v162: 129 pgs: 129 active+clean; 3.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-21T08:44:14.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:14 smithi112 ceph-mon[31085]: pgmap v162: 129 pgs: 129 active+clean; 3.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-21T08:44:14.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:14 smithi043 ceph-mon[27978]: pgmap v162: 129 pgs: 129 active+clean; 3.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-21T08:44:16.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:15 smithi112 ceph-mon[31085]: pgmap v163: 129 pgs: 129 active+clean; 3.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-21T08:44:16.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:15 smithi043 ceph-mon[27978]: pgmap v163: 129 pgs: 129 active+clean; 3.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-21T08:44:16.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:15 smithi175 ceph-mon[31920]: pgmap v163: 129 pgs: 129 active+clean; 3.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-21T08:44:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.555+0000 7fc9f06e4640 1 -- 172.21.15.43:0/585786498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9e81021d0 msgr2=0x7fc9e81025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.555+0000 7fc9f06e4640 1 --2- 172.21.15.43:0/585786498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9e81021d0 0x7fc9e81025b0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009900 tx=0x7fc9e402bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.555+0000 7fc9f06e4640 1 -- 172.21.15.43:0/585786498 shutdown_connections 2024-04-21T08:44:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.555+0000 7fc9f06e4640 1 --2- 172.21.15.43:0/585786498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9e81072a0 0x7fc9e810ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.555+0000 7fc9f06e4640 1 --2- 172.21.15.43:0/585786498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9e8102b80 0x7fc9e8106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.555+0000 7fc9f06e4640 1 --2- 172.21.15.43:0/585786498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9e81021d0 0x7fc9e81025b0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.555+0000 7fc9f06e4640 1 -- 172.21.15.43:0/585786498 >> 172.21.15.43:0/585786498 conn(0x7fc9e80793e0 msgr2=0x7fc9e80797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.555+0000 7fc9f06e4640 1 -- 172.21.15.43:0/585786498 shutdown_connections 2024-04-21T08:44:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.555+0000 7fc9f06e4640 1 -- 172.21.15.43:0/585786498 wait complete. 2024-04-21T08:44:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.556+0000 7fc9f06e4640 1 Processor -- start 2024-04-21T08:44:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.556+0000 7fc9f06e4640 1 -- start start 2024-04-21T08:44:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.556+0000 7fc9f06e4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9e81021d0 0x7fc9e8198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.556+0000 7fc9f06e4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9e81072a0 0x7fc9e8198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.556+0000 7fc9f06e4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9e819cf40 0x7fc9e819d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.556+0000 7fc9f06e4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e81121e0 con 0x7fc9e81072a0 2024-04-21T08:44:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.556+0000 7fc9f06e4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e8112060 con 0x7fc9e819cf40 2024-04-21T08:44:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.556+0000 7fc9f06e4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e8112360 con 0x7fc9e81021d0 2024-04-21T08:44:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.556+0000 7fc9edc7f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9e81072a0 0x7fc9e8198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.556+0000 7fc9edc7f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9e81072a0 0x7fc9e8198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34408/0 (socket says 172.21.15.43:34408) 2024-04-21T08:44:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.557+0000 7fc9edc7f640 1 -- 172.21.15.43:0/1580378320 learned_addr learned my addr 172.21.15.43:0/1580378320 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.557+0000 7fc9eec81640 1 --2- 172.21.15.43:0/1580378320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9e819cf40 0x7fc9e819d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.557+0000 7fc9ee480640 1 --2- 172.21.15.43:0/1580378320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9e81021d0 0x7fc9e8198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.557+0000 7fc9edc7f640 1 -- 172.21.15.43:0/1580378320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9e81021d0 msgr2=0x7fc9e8198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.557+0000 7fc9edc7f640 1 --2- 172.21.15.43:0/1580378320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9e81021d0 0x7fc9e8198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.557+0000 7fc9edc7f640 1 -- 172.21.15.43:0/1580378320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9e819cf40 msgr2=0x7fc9e819d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.557+0000 7fc9edc7f640 1 --2- 172.21.15.43:0/1580378320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9e819cf40 0x7fc9e819d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.557+0000 7fc9edc7f640 1 -- 172.21.15.43:0/1580378320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e819da30 con 0x7fc9e81072a0 2024-04-21T08:44:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.557+0000 7fc9ee480640 1 --2- 172.21.15.43:0/1580378320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9e81021d0 0x7fc9e8198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.557+0000 7fc9edc7f640 1 --2- 172.21.15.43:0/1580378320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9e81072a0 0x7fc9e8198bc0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fc9d800d770 tx=0x7fc9d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.557+0000 7fc9df7fe640 1 -- 172.21.15.43:0/1580378320 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8017070 con 0x7fc9e81072a0 2024-04-21T08:44:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.557+0000 7fc9df7fe640 1 -- 172.21.15.43:0/1580378320 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9d8009c30 con 0x7fc9e81072a0 2024-04-21T08:44:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.557+0000 7fc9f06e4640 1 -- 172.21.15.43:0/1580378320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e819dd20 con 0x7fc9e81072a0 2024-04-21T08:44:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.557+0000 7fc9df7fe640 1 -- 172.21.15.43:0/1580378320 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d800f620 con 0x7fc9e81072a0 2024-04-21T08:44:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.557+0000 7fc9f06e4640 1 -- 172.21.15.43:0/1580378320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e81a5560 con 0x7fc9e81072a0 2024-04-21T08:44:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.560+0000 7fc9df7fe640 1 -- 172.21.15.43:0/1580378320 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc9d801a070 con 0x7fc9e81072a0 2024-04-21T08:44:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.560+0000 7fc9f33b4740 1 Processor -- start 2024-04-21T08:44:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.561+0000 7fc9f33b4740 1 -- start start 2024-04-21T08:44:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.561+0000 7fc9f33b4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601336658d0 0x560133664620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.561+0000 7fc9f33b4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560133664b60 0x560133664fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.561+0000 7fc9f33b4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013366e0e0 0x5601336704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.561+0000 7fc9f33b4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5601335c98d0 con 0x560133664b60 2024-04-21T08:44:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.561+0000 7fc9f33b4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560133566e80 con 0x5601336658d0 2024-04-21T08:44:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.561+0000 7fc9f33b4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5601335c5890 con 0x56013366e0e0 2024-04-21T08:44:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.561+0000 7fc9ee480640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560133664b60 0x560133664fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.561+0000 7fc9eec81640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601336658d0 0x560133664620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.561+0000 7fc9ee480640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560133664b60 0x560133664fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34422/0 (socket says 172.21.15.43:34422) 2024-04-21T08:44:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.561+0000 7fc9ee480640 1 -- 172.21.15.43:0/124535126 learned_addr learned my addr 172.21.15.43:0/124535126 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.561+0000 7fc9edc7f640 1 --2- 172.21.15.43:0/124535126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013366e0e0 0x5601336704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.561+0000 7fc9df7fe640 1 --2- 172.21.15.43:0/1580378320 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc9d40a3e40 0x7fc9d40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.562+0000 7fc9ee480640 1 -- 172.21.15.43:0/124535126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013366e0e0 msgr2=0x5601336704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.562+0000 7fc9ee480640 1 --2- 172.21.15.43:0/124535126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013366e0e0 0x5601336704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.562+0000 7fc9ee480640 1 -- 172.21.15.43:0/124535126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601336658d0 msgr2=0x560133664620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.562+0000 7fc9ee480640 1 --2- 172.21.15.43:0/124535126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601336658d0 0x560133664620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.562+0000 7fc9ee480640 1 -- 172.21.15.43:0/124535126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560133588320 con 0x560133664b60 2024-04-21T08:44:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.562+0000 7fc9eec81640 1 --2- 172.21.15.43:0/124535126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601336658d0 0x560133664620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:44:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.562+0000 7fc9df7fe640 1 -- 172.21.15.43:0/1580378320 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc9d80d06c0 con 0x7fc9e81072a0 2024-04-21T08:44:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.562+0000 7fc9eec81640 1 --2- 172.21.15.43:0/1580378320 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc9d40a3e40 0x7fc9d40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.562+0000 7fc9ee480640 1 --2- 172.21.15.43:0/124535126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560133664b60 0x560133664fe0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4036040 tx=0x7fc9e402c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f164282d8786af65 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.562+0000 7fc9dcff9640 1 -- 172.21.15.43:0/124535126 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4043020 con 0x560133664b60 2024-04-21T08:44:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.562+0000 7fc9eec81640 1 --2- 172.21.15.43:0/1580378320 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc9d40a3e40 0x7fc9d40a6300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0005fd0 tx=0x7fc9e0004a40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.562+0000 7fc9dcff9640 1 -- 172.21.15.43:0/124535126 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9e402c7e0 con 0x560133664b60 2024-04-21T08:44:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.562+0000 7fc9dcff9640 1 -- 172.21.15.43:0/124535126 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e40055d0 con 0x560133664b60 2024-04-21T08:44:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.563+0000 7fc9f33b4740 1 -- 172.21.15.43:0/124535126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560133664b60 msgr2=0x560133664fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.563+0000 7fc9f33b4740 1 --2- 172.21.15.43:0/124535126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560133664b60 0x560133664fe0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4036040 tx=0x7fc9e402c680 comp rx=0 tx=0).stop 2024-04-21T08:44:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.563+0000 7fc9f33b4740 1 -- 172.21.15.43:0/124535126 shutdown_connections 2024-04-21T08:44:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.563+0000 7fc9f33b4740 1 --2- 172.21.15.43:0/124535126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013366e0e0 0x5601336704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.563+0000 7fc9f33b4740 1 --2- 172.21.15.43:0/124535126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560133664b60 0x560133664fe0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.563+0000 7fc9f33b4740 1 --2- 172.21.15.43:0/124535126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601336658d0 0x560133664620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.563+0000 7fc9f33b4740 1 -- 172.21.15.43:0/124535126 >> 172.21.15.43:0/124535126 conn(0x56013362bc90 msgr2=0x56013365f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.563+0000 7fc9f33b4740 1 -- 172.21.15.43:0/124535126 shutdown_connections 2024-04-21T08:44:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.563+0000 7fc9f33b4740 1 -- 172.21.15.43:0/124535126 wait complete. 2024-04-21T08:44:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.563+0000 7fc9f33b4740 1 -- start start 2024-04-21T08:44:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.564+0000 7fc9f33b4740 1 Processor -- start 2024-04-21T08:44:17.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.564+0000 7fc9f33b4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560133664b60 0x56013366da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.564+0000 7fc9f33b4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601336658d0 0x56013366ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.564+0000 7fc9f33b4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013366e0e0 0x56013366b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.564+0000 7fc9f33b4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560133670db0 con 0x560133664b60 2024-04-21T08:44:17.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.564+0000 7fc9f33b4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560133551eb0 con 0x5601336658d0 2024-04-21T08:44:17.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.564+0000 7fc9f33b4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560133670f30 con 0x56013366e0e0 2024-04-21T08:44:17.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.564+0000 7fc9f33b4740 10 client.0 _ll_register_callbacks cb 0x56013357eb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:17.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.564+0000 7fc9ee480640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560133664b60 0x56013366da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.564+0000 7fc9ee480640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560133664b60 0x56013366da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34428/0 (socket says 172.21.15.43:34428) 2024-04-21T08:44:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.564+0000 7fc9ee480640 1 -- 172.21.15.43:0/1014035616 learned_addr learned my addr 172.21.15.43:0/1014035616 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.564+0000 7fc9eec81640 1 --2- 172.21.15.43:0/1014035616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013366e0e0 0x56013366b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.564+0000 7fc9edc7f640 1 --2- 172.21.15.43:0/1014035616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601336658d0 0x56013366ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.564+0000 7fc9ee480640 1 -- 172.21.15.43:0/1014035616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013366e0e0 msgr2=0x56013366b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.564+0000 7fc9ee480640 1 --2- 172.21.15.43:0/1014035616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013366e0e0 0x56013366b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.564+0000 7fc9ee480640 1 -- 172.21.15.43:0/1014035616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601336658d0 msgr2=0x56013366ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.564+0000 7fc9ee480640 1 --2- 172.21.15.43:0/1014035616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601336658d0 0x56013366ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.564+0000 7fc9ee480640 1 -- 172.21.15.43:0/1014035616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601332dc210 con 0x560133664b60 2024-04-21T08:44:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.564+0000 7fc9eec81640 1 --2- 172.21.15.43:0/1014035616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013366e0e0 0x56013366b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.564+0000 7fc9edc7f640 1 --2- 172.21.15.43:0/1014035616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601336658d0 0x56013366ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.565+0000 7fc9ee480640 1 --2- 172.21.15.43:0/1014035616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560133664b60 0x56013366da90 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4000c00 tx=0x7fc9e4005150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da87abe6183214c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.565+0000 7fc9d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.565+0000 7fc9d1ffb640 1 -- 172.21.15.43:0/1014035616 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e404c070 con 0x560133664b60 2024-04-21T08:44:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.565+0000 7fc9d1ffb640 1 -- 172.21.15.43:0/1014035616 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc9e40035f0 con 0x560133664b60 2024-04-21T08:44:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.565+0000 7fc9f33b4740 1 -- 172.21.15.43:0/1014035616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9e402c7e0 con 0x560133664b60 2024-04-21T08:44:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.565+0000 7fc9d1ffb640 1 -- 172.21.15.43:0/1014035616 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc9e4057050 con 0x560133664b60 2024-04-21T08:44:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.565+0000 7fc9f33b4740 10 client.17847 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.565+0000 7fc9f33b4740 1 -- 172.21.15.43:0/1014035616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5601333a8620 con 0x560133664b60 2024-04-21T08:44:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.565+0000 7fc9d1ffb640 1 -- 172.21.15.43:0/1014035616 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9e402c7e0 con 0x560133664b60 2024-04-21T08:44:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.565+0000 7fc9d1ffb640 1 -- 172.21.15.43:0/1014035616 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc9e4004a00 con 0x560133664b60 2024-04-21T08:44:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.565+0000 7fc9f33b4740 10 client.17847 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.565+0000 7fc9f33b4740 10 client.17847 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.565+0000 7fc9f33b4740 20 client.17847 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.565+0000 7fc9f33b4740 1 --2- 172.21.15.43:0/1014035616 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56013367df20 0x560133680310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.565+0000 7fc9f33b4740 4 client.17847 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.565+0000 7fc9f33b4740 1 -- 172.21.15.43:0/1014035616 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601335dd880 con 0x56013367df20 2024-04-21T08:44:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.566+0000 7fc9edc7f640 1 --2- 172.21.15.43:0/1014035616 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56013367df20 0x560133680310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.566+0000 7fc9edc7f640 1 --2- 172.21.15.43:0/1014035616 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56013367df20 0x560133680310 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7756467da5049d14 server_cookie=8f5d976ed58df1b0 in_seq=0 out_seq=0 2024-04-21T08:44:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.566+0000 7fc9d1ffb640 10 client.17847 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.567+0000 7fc9d1ffb640 1 -- 172.21.15.43:0/1014035616 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5601335dd880 con 0x56013367df20 2024-04-21T08:44:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.567+0000 7fc9d1ffb640 10 client.17847 handle_command_reply: tid=0 2024-04-21T08:44:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.567+0000 7fc9f33b4740 1 client.17847 shutdown 2024-04-21T08:44:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.567+0000 7fc9f33b4740 1 -- 172.21.15.43:0/1014035616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560133664b60 msgr2=0x56013366da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.567+0000 7fc9f33b4740 1 --2- 172.21.15.43:0/1014035616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560133664b60 0x56013366da90 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4000c00 tx=0x7fc9e4005150 comp rx=0 tx=0).stop 2024-04-21T08:44:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.567+0000 7fc9f33b4740 1 -- 172.21.15.43:0/1014035616 shutdown_connections 2024-04-21T08:44:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.567+0000 7fc9f33b4740 1 --2- 172.21.15.43:0/1014035616 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56013367df20 0x560133680310 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.567+0000 7fc9f33b4740 1 --2- 172.21.15.43:0/1014035616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56013366e0e0 0x56013366b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.567+0000 7fc9f33b4740 1 --2- 172.21.15.43:0/1014035616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601336658d0 0x56013366ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.567+0000 7fc9d1ffb640 0 client.17847 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.567+0000 7fc9f33b4740 1 --2- 172.21.15.43:0/1014035616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560133664b60 0x56013366da90 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.567+0000 7fc9f33b4740 1 -- 172.21.15.43:0/1014035616 >> 172.21.15.43:0/1014035616 conn(0x56013362bc90 msgr2=0x5601336615a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:17.576 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.567+0000 7fc9f33b4740 1 -- 172.21.15.43:0/1014035616 shutdown_connections 2024-04-21T08:44:17.576 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.567+0000 7fc9f33b4740 1 -- 172.21.15.43:0/1014035616 wait complete. 2024-04-21T08:44:17.576 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.567+0000 7fc9f33b4740 20 client.17847 trim_cache size 0 max 16384 2024-04-21T08:44:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.582+0000 7fc9f33b4740 1 Processor -- start 2024-04-21T08:44:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.582+0000 7fc9f33b4740 1 -- start start 2024-04-21T08:44:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.583+0000 7fc9f33b4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560133664b60 0x560133679cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.583+0000 7fc9f33b4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601336658d0 0x56013366d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.583+0000 7fc9f33b4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56013366e0e0 0x56013366da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.583+0000 7fc9f33b4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e81af360 con 0x56013366e0e0 2024-04-21T08:44:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.583+0000 7fc9f33b4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e81b0ea0 con 0x5601336658d0 2024-04-21T08:44:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.583+0000 7fc9f33b4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e81afa30 con 0x560133664b60 2024-04-21T08:44:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.583+0000 7fc9eec81640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56013366e0e0 0x56013366da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.583+0000 7fc9eec81640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56013366e0e0 0x56013366da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34432/0 (socket says 172.21.15.43:34432) 2024-04-21T08:44:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.583+0000 7fc9eec81640 1 -- 172.21.15.43:0/1169242770 learned_addr learned my addr 172.21.15.43:0/1169242770 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.583+0000 7fc9ee480640 1 --2- 172.21.15.43:0/1169242770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560133664b60 0x560133679cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.583+0000 7fc9edc7f640 1 --2- 172.21.15.43:0/1169242770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601336658d0 0x56013366d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.583+0000 7fc9eec81640 1 -- 172.21.15.43:0/1169242770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560133664b60 msgr2=0x560133679cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.583+0000 7fc9eec81640 1 --2- 172.21.15.43:0/1169242770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560133664b60 0x560133679cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.583+0000 7fc9eec81640 1 -- 172.21.15.43:0/1169242770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601336658d0 msgr2=0x56013366d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.583+0000 7fc9eec81640 1 --2- 172.21.15.43:0/1169242770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601336658d0 0x56013366d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.583+0000 7fc9eec81640 1 -- 172.21.15.43:0/1169242770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560133663e30 con 0x56013366e0e0 2024-04-21T08:44:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.583+0000 7fc9eec81640 1 --2- 172.21.15.43:0/1169242770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56013366e0e0 0x56013366da90 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0003c00 tx=0x7fc9e0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda3f7f9994e2084 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.583+0000 7fc9d3fff640 1 -- 172.21.15.43:0/1169242770 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0004710 con 0x56013366e0e0 2024-04-21T08:44:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.584+0000 7fc9d3fff640 1 -- 172.21.15.43:0/1169242770 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9e0010b00 con 0x56013366e0e0 2024-04-21T08:44:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.584+0000 7fc9d3fff640 1 -- 172.21.15.43:0/1169242770 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e001d910 con 0x56013366e0e0 2024-04-21T08:44:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.584+0000 7fc9f33b4740 1 -- 172.21.15.43:0/1169242770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56013366e0e0 msgr2=0x56013366da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.584+0000 7fc9f33b4740 1 --2- 172.21.15.43:0/1169242770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56013366e0e0 0x56013366da90 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0003c00 tx=0x7fc9e0016000 comp rx=0 tx=0).stop 2024-04-21T08:44:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.584+0000 7fc9f33b4740 1 -- 172.21.15.43:0/1169242770 shutdown_connections 2024-04-21T08:44:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.584+0000 7fc9f33b4740 1 --2- 172.21.15.43:0/1169242770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56013366e0e0 0x56013366da90 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.584+0000 7fc9f33b4740 1 --2- 172.21.15.43:0/1169242770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601336658d0 0x56013366d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.584+0000 7fc9f33b4740 1 --2- 172.21.15.43:0/1169242770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560133664b60 0x560133679cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.584+0000 7fc9f33b4740 1 -- 172.21.15.43:0/1169242770 >> 172.21.15.43:0/1169242770 conn(0x56013362bc90 msgr2=0x56013365f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.584+0000 7fc9f33b4740 1 -- 172.21.15.43:0/1169242770 shutdown_connections 2024-04-21T08:44:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.584+0000 7fc9f33b4740 1 -- 172.21.15.43:0/1169242770 wait complete. 2024-04-21T08:44:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.584+0000 7fc9f33b4740 1 -- start start 2024-04-21T08:44:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.585+0000 7fc9f33b4740 1 Processor -- start 2024-04-21T08:44:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.585+0000 7fc9f33b4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560133664b60 0x56013366ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.585+0000 7fc9f33b4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601336658d0 0x56013366f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.585+0000 7fc9f33b4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56013366e0e0 0x56013366f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.585+0000 7fc9f33b4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e81b07d0 con 0x56013366e0e0 2024-04-21T08:44:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.585+0000 7fc9f33b4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e81b0100 con 0x5601336658d0 2024-04-21T08:44:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.585+0000 7fc9f33b4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e81ad340 con 0x560133664b60 2024-04-21T08:44:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.585+0000 7fc9f33b4740 10 client.0 _ll_register_callbacks cb 0x7fc9e00099f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.585+0000 7fc9eec81640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56013366e0e0 0x56013366f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.585+0000 7fc9eec81640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56013366e0e0 0x56013366f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34438/0 (socket says 172.21.15.43:34438) 2024-04-21T08:44:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.585+0000 7fc9eec81640 1 -- 172.21.15.43:0/516360184 learned_addr learned my addr 172.21.15.43:0/516360184 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.585+0000 7fc9edc7f640 1 --2- 172.21.15.43:0/516360184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601336658d0 0x56013366f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.585+0000 7fc9ee480640 1 --2- 172.21.15.43:0/516360184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560133664b60 0x56013366ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.585+0000 7fc9eec81640 1 -- 172.21.15.43:0/516360184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560133664b60 msgr2=0x56013366ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.585+0000 7fc9eec81640 1 --2- 172.21.15.43:0/516360184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560133664b60 0x56013366ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.585+0000 7fc9eec81640 1 -- 172.21.15.43:0/516360184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601336658d0 msgr2=0x56013366f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.585+0000 7fc9eec81640 1 --2- 172.21.15.43:0/516360184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601336658d0 0x56013366f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.585+0000 7fc9eec81640 1 -- 172.21.15.43:0/516360184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560133674300 con 0x56013366e0e0 2024-04-21T08:44:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.586+0000 7fc9eec81640 1 --2- 172.21.15.43:0/516360184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56013366e0e0 0x56013366f680 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0013ea0 tx=0x7fc9e001d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee844e5f265d0b41 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.586+0000 7fc9d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.586+0000 7fc9d1ffb640 1 -- 172.21.15.43:0/516360184 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0010650 con 0x56013366e0e0 2024-04-21T08:44:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.586+0000 7fc9f33b4740 1 -- 172.21.15.43:0/516360184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9e0010b00 con 0x56013366e0e0 2024-04-21T08:44:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.586+0000 7fc9d1ffb640 1 -- 172.21.15.43:0/516360184 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc9e002d040 con 0x56013366e0e0 2024-04-21T08:44:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.586+0000 7fc9d1ffb640 1 -- 172.21.15.43:0/516360184 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc9e00107f0 con 0x56013366e0e0 2024-04-21T08:44:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.586+0000 7fc9f33b4740 10 client.17853 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.586+0000 7fc9f33b4740 1 -- 172.21.15.43:0/516360184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5601336e43f0 con 0x56013366e0e0 2024-04-21T08:44:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.586+0000 7fc9d1ffb640 1 -- 172.21.15.43:0/516360184 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9e0010b00 con 0x56013366e0e0 2024-04-21T08:44:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.586+0000 7fc9d1ffb640 1 -- 172.21.15.43:0/516360184 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc9e000f040 con 0x56013366e0e0 2024-04-21T08:44:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.586+0000 7fc9f33b4740 10 client.17853 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.586+0000 7fc9f33b4740 10 client.17853 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.586+0000 7fc9f33b4740 20 client.17853 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.586+0000 7fc9f33b4740 1 --2- 172.21.15.43:0/516360184 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5601336e4b70 0x5601336e6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.586+0000 7fc9f33b4740 4 client.17853 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:44:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.586+0000 7fc9f33b4740 1 -- 172.21.15.43:0/516360184 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5601335693d0 con 0x5601336e4b70 2024-04-21T08:44:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.587+0000 7fc9ee480640 1 --2- 172.21.15.43:0/516360184 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5601336e4b70 0x5601336e6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.587+0000 7fc9d1ffb640 10 client.17853 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.587+0000 7fc9ee480640 1 --2- 172.21.15.43:0/516360184 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5601336e4b70 0x5601336e6f90 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c0f10e29200d21e server_cookie=e815a970abfb4c37 in_seq=0 out_seq=0 2024-04-21T08:44:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.601+0000 7fc9d1ffb640 1 -- 172.21.15.43:0/516360184 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5601335693d0 con 0x5601336e4b70 2024-04-21T08:44:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.601+0000 7fc9d1ffb640 10 client.17853 handle_command_reply: tid=0 2024-04-21T08:44:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.601+0000 7fc9f33b4740 1 client.17853 shutdown 2024-04-21T08:44:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.601+0000 7fc9f33b4740 1 -- 172.21.15.43:0/516360184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56013366e0e0 msgr2=0x56013366f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.601+0000 7fc9f33b4740 1 --2- 172.21.15.43:0/516360184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56013366e0e0 0x56013366f680 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0013ea0 tx=0x7fc9e001d440 comp rx=0 tx=0).stop 2024-04-21T08:44:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.601+0000 7fc9f33b4740 1 -- 172.21.15.43:0/516360184 shutdown_connections 2024-04-21T08:44:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.601+0000 7fc9f33b4740 1 --2- 172.21.15.43:0/516360184 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5601336e4b70 0x5601336e6f90 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.601+0000 7fc9f33b4740 1 --2- 172.21.15.43:0/516360184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56013366e0e0 0x56013366f680 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.601+0000 7fc9f33b4740 1 --2- 172.21.15.43:0/516360184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5601336658d0 0x56013366f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.601+0000 7fc9d1ffb640 0 client.17853 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.601+0000 7fc9f33b4740 1 --2- 172.21.15.43:0/516360184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560133664b60 0x56013366ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.601+0000 7fc9f33b4740 1 -- 172.21.15.43:0/516360184 >> 172.21.15.43:0/516360184 conn(0x56013362bc90 msgr2=0x56013365e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.602+0000 7fc9f33b4740 1 -- 172.21.15.43:0/516360184 shutdown_connections 2024-04-21T08:44:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.602+0000 7fc9f33b4740 1 -- 172.21.15.43:0/516360184 wait complete. 2024-04-21T08:44:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.602+0000 7fc9f33b4740 20 client.17853 trim_cache size 0 max 16384 2024-04-21T08:44:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:44:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:44:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:44:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:44:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.602+0000 7fc9f06e4640 1 -- 172.21.15.43:0/1580378320 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc9d40a3e40 msgr2=0x7fc9d40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.602+0000 7fc9f06e4640 1 --2- 172.21.15.43:0/1580378320 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc9d40a3e40 0x7fc9d40a6300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0005fd0 tx=0x7fc9e0004a40 comp rx=0 tx=0).stop 2024-04-21T08:44:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.602+0000 7fc9f06e4640 1 -- 172.21.15.43:0/1580378320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9e81072a0 msgr2=0x7fc9e8198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.602+0000 7fc9f06e4640 1 --2- 172.21.15.43:0/1580378320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9e81072a0 0x7fc9e8198bc0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fc9d800d770 tx=0x7fc9d800dc30 comp rx=0 tx=0).stop 2024-04-21T08:44:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.602+0000 7fc9f06e4640 1 -- 172.21.15.43:0/1580378320 shutdown_connections 2024-04-21T08:44:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.602+0000 7fc9f06e4640 1 --2- 172.21.15.43:0/1580378320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9e819cf40 0x7fc9e819d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.602+0000 7fc9f06e4640 1 --2- 172.21.15.43:0/1580378320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9e81072a0 0x7fc9e8198bc0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.602+0000 7fc9f06e4640 1 --2- 172.21.15.43:0/1580378320 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc9d40a3e40 0x7fc9d40a6300 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.602+0000 7fc9f06e4640 1 --2- 172.21.15.43:0/1580378320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9e81021d0 0x7fc9e8198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.602+0000 7fc9f06e4640 1 -- 172.21.15.43:0/1580378320 >> 172.21.15.43:0/1580378320 conn(0x7fc9e80793e0 msgr2=0x7fc9e80ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.602+0000 7fc9f06e4640 1 -- 172.21.15.43:0/1580378320 shutdown_connections 2024-04-21T08:44:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:17.602+0000 7fc9f06e4640 1 -- 172.21.15.43:0/1580378320 wait complete. 2024-04-21T08:44:18.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:18 smithi043 ceph-mon[27978]: pgmap v164: 129 pgs: 129 active+clean; 3.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 50 KiB/s wr, 22 op/s 2024-04-21T08:44:18.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:18 smithi175 ceph-mon[31920]: pgmap v164: 129 pgs: 129 active+clean; 3.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 50 KiB/s wr, 22 op/s 2024-04-21T08:44:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:18 smithi112 ceph-mon[31085]: pgmap v164: 129 pgs: 129 active+clean; 3.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 50 KiB/s wr, 22 op/s 2024-04-21T08:44:20.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:20 smithi043 ceph-mon[27978]: pgmap v165: 129 pgs: 129 active+clean; 3.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-04-21T08:44:20.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:20 smithi175 ceph-mon[31920]: pgmap v165: 129 pgs: 129 active+clean; 3.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-04-21T08:44:20.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:20 smithi112 ceph-mon[31085]: pgmap v165: 129 pgs: 129 active+clean; 3.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-04-21T08:44:22.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:22 smithi043 ceph-mon[27978]: pgmap v166: 129 pgs: 129 active+clean; 3.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 22 op/s 2024-04-21T08:44:22.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:22 smithi175 ceph-mon[31920]: pgmap v166: 129 pgs: 129 active+clean; 3.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 22 op/s 2024-04-21T08:44:22.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:22 smithi112 ceph-mon[31085]: pgmap v166: 129 pgs: 129 active+clean; 3.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 22 op/s 2024-04-21T08:44:24.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:44:24.148 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:44:24.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.265+0000 7faa63ca5640 1 -- 172.21.15.43:0/1903458611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa5c06af70 msgr2=0x7faa5c10ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.265+0000 7faa63ca5640 1 --2- 172.21.15.43:0/1903458611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa5c06af70 0x7faa5c10ede0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7faa54009cd0 tx=0x7faa5402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.265+0000 7faa63ca5640 1 -- 172.21.15.43:0/1903458611 shutdown_connections 2024-04-21T08:44:24.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.265+0000 7faa63ca5640 1 --2- 172.21.15.43:0/1903458611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa5c06af70 0x7faa5c10ede0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.265+0000 7faa63ca5640 1 --2- 172.21.15.43:0/1903458611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa5c06a5b0 0x7faa5c06aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.265+0000 7faa63ca5640 1 --2- 172.21.15.43:0/1903458611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa5c108060 0x7faa5c108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.265+0000 7faa63ca5640 1 -- 172.21.15.43:0/1903458611 >> 172.21.15.43:0/1903458611 conn(0x7faa5c077c40 msgr2=0x7faa5c078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:24.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.266+0000 7faa63ca5640 1 -- 172.21.15.43:0/1903458611 shutdown_connections 2024-04-21T08:44:24.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.266+0000 7faa63ca5640 1 -- 172.21.15.43:0/1903458611 wait complete. 2024-04-21T08:44:24.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.266+0000 7faa63ca5640 1 Processor -- start 2024-04-21T08:44:24.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.266+0000 7faa63ca5640 1 -- start start 2024-04-21T08:44:24.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.266+0000 7faa63ca5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa5c06a5b0 0x7faa5c101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.266+0000 7faa63ca5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa5c06af70 0x7faa5c1016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.266+0000 7faa63ca5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa5c108060 0x7faa5c101bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.266+0000 7faa63ca5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c1121a0 con 0x7faa5c06af70 2024-04-21T08:44:24.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.266+0000 7faa63ca5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c112020 con 0x7faa5c108060 2024-04-21T08:44:24.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.266+0000 7faa63ca5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c112320 con 0x7faa5c06a5b0 2024-04-21T08:44:24.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.266+0000 7faa61240640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa5c06af70 0x7faa5c1016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.267+0000 7faa61240640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa5c06af70 0x7faa5c1016b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34454/0 (socket says 172.21.15.43:34454) 2024-04-21T08:44:24.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.267+0000 7faa61240640 1 -- 172.21.15.43:0/1584642950 learned_addr learned my addr 172.21.15.43:0/1584642950 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:24.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.267+0000 7faa62242640 1 --2- 172.21.15.43:0/1584642950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa5c108060 0x7faa5c101bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.267+0000 7faa61a41640 1 --2- 172.21.15.43:0/1584642950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa5c06a5b0 0x7faa5c101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.267+0000 7faa61240640 1 -- 172.21.15.43:0/1584642950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa5c06a5b0 msgr2=0x7faa5c101170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.267+0000 7faa61240640 1 --2- 172.21.15.43:0/1584642950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa5c06a5b0 0x7faa5c101170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.267+0000 7faa61240640 1 -- 172.21.15.43:0/1584642950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa5c108060 msgr2=0x7faa5c101bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.267+0000 7faa61240640 1 --2- 172.21.15.43:0/1584642950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa5c108060 0x7faa5c101bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.267+0000 7faa61240640 1 -- 172.21.15.43:0/1584642950 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5c1a6ba0 con 0x7faa5c06af70 2024-04-21T08:44:24.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.267+0000 7faa62242640 1 --2- 172.21.15.43:0/1584642950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa5c108060 0x7faa5c101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:24.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.267+0000 7faa61240640 1 --2- 172.21.15.43:0/1584642950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa5c06af70 0x7faa5c1016b0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00e9f0 tx=0x7faa4c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:24.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.267+0000 7faa52ffd640 1 -- 172.21.15.43:0/1584642950 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c00cd80 con 0x7faa5c06af70 2024-04-21T08:44:24.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.267+0000 7faa52ffd640 1 -- 172.21.15.43:0/1584642950 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa4c0108e0 con 0x7faa5c06af70 2024-04-21T08:44:24.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.267+0000 7faa63ca5640 1 -- 172.21.15.43:0/1584642950 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa5c1a6e90 con 0x7faa5c06af70 2024-04-21T08:44:24.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.267+0000 7faa52ffd640 1 -- 172.21.15.43:0/1584642950 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c018980 con 0x7faa5c06af70 2024-04-21T08:44:24.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.267+0000 7faa63ca5640 1 -- 172.21.15.43:0/1584642950 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa5c1a7350 con 0x7faa5c06af70 2024-04-21T08:44:24.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.270+0000 7faa52ffd640 1 -- 172.21.15.43:0/1584642950 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faa4c010430 con 0x7faa5c06af70 2024-04-21T08:44:24.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.270+0000 7faa66973740 1 Processor -- start 2024-04-21T08:44:24.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.271+0000 7faa66973740 1 -- start start 2024-04-21T08:44:24.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.271+0000 7faa66973740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cbcdde4580 0x55cbcdde4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.271+0000 7faa66973740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4ea0 0x55cbcdded540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.271+0000 7faa66973740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cbcddeda80 0x55cbcddefe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.271+0000 7faa66973740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cbcdd491e0 con 0x55cbcdde4ea0 2024-04-21T08:44:24.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.271+0000 7faa66973740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cbcdce6390 con 0x55cbcdde4580 2024-04-21T08:44:24.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.271+0000 7faa66973740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55cbcdd451a0 con 0x55cbcddeda80 2024-04-21T08:44:24.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.271+0000 7faa52ffd640 1 --2- 172.21.15.43:0/1584642950 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faa480a3d20 0x7faa480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.271+0000 7faa61a41640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4ea0 0x55cbcdded540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.271+0000 7faa61a41640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4ea0 0x55cbcdded540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34464/0 (socket says 172.21.15.43:34464) 2024-04-21T08:44:24.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.271+0000 7faa61a41640 1 -- 172.21.15.43:0/1920592427 learned_addr learned my addr 172.21.15.43:0/1920592427 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:24.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.271+0000 7faa62242640 1 --2- 172.21.15.43:0/1920592427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cbcdde4580 0x55cbcdde4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.271+0000 7faa62242640 1 --2- 172.21.15.43:0/1584642950 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faa480a3d20 0x7faa480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.271+0000 7faa52ffd640 1 -- 172.21.15.43:0/1584642950 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7faa4c0d0ff0 con 0x7faa5c06af70 2024-04-21T08:44:24.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.271+0000 7faa61240640 1 --2- 172.21.15.43:0/1920592427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cbcddeda80 0x55cbcddefe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.271+0000 7faa61a41640 1 -- 172.21.15.43:0/1920592427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cbcddeda80 msgr2=0x55cbcddefe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:24.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.271+0000 7faa61a41640 1 --2- 172.21.15.43:0/1920592427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cbcddeda80 0x55cbcddefe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.271+0000 7faa61a41640 1 -- 172.21.15.43:0/1920592427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cbcdde4580 msgr2=0x55cbcdde4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:24.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.271+0000 7faa61a41640 1 --2- 172.21.15.43:0/1920592427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cbcdde4580 0x55cbcdde4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.271+0000 7faa61a41640 1 -- 172.21.15.43:0/1920592427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbcdb257d0 con 0x55cbcdde4ea0 2024-04-21T08:44:24.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.271+0000 7faa62242640 1 --2- 172.21.15.43:0/1920592427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cbcdde4580 0x55cbcdde4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:44:24.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.272+0000 7faa61a41640 1 --2- 172.21.15.43:0/1920592427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4ea0 0x55cbcdded540 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7faa5800bc20 tx=0x7faa5800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947a493cfed5c67a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:24.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.272+0000 7faa62242640 1 --2- 172.21.15.43:0/1584642950 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faa480a3d20 0x7faa480a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faa54006010 tx=0x7faa540095c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:24.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.272+0000 7faa47fff640 1 -- 172.21.15.43:0/1920592427 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58015070 con 0x55cbcdde4ea0 2024-04-21T08:44:24.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.272+0000 7faa47fff640 1 -- 172.21.15.43:0/1920592427 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa580054d0 con 0x55cbcdde4ea0 2024-04-21T08:44:24.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.272+0000 7faa47fff640 1 -- 172.21.15.43:0/1920592427 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58011440 con 0x55cbcdde4ea0 2024-04-21T08:44:24.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.273+0000 7faa66973740 1 -- 172.21.15.43:0/1920592427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4ea0 msgr2=0x55cbcdded540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.273+0000 7faa66973740 1 --2- 172.21.15.43:0/1920592427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4ea0 0x55cbcdded540 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7faa5800bc20 tx=0x7faa5800f730 comp rx=0 tx=0).stop 2024-04-21T08:44:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.273+0000 7faa66973740 1 -- 172.21.15.43:0/1920592427 shutdown_connections 2024-04-21T08:44:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.273+0000 7faa66973740 1 --2- 172.21.15.43:0/1920592427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cbcddeda80 0x55cbcddefe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.273+0000 7faa66973740 1 --2- 172.21.15.43:0/1920592427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4ea0 0x55cbcdded540 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.273+0000 7faa66973740 1 --2- 172.21.15.43:0/1920592427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cbcdde4580 0x55cbcdde4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.273+0000 7faa66973740 1 -- 172.21.15.43:0/1920592427 >> 172.21.15.43:0/1920592427 conn(0x55cbcdddecb0 msgr2=0x55cbcdddf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.273+0000 7faa66973740 1 -- 172.21.15.43:0/1920592427 shutdown_connections 2024-04-21T08:44:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.273+0000 7faa66973740 1 -- 172.21.15.43:0/1920592427 wait complete. 2024-04-21T08:44:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.273+0000 7faa66973740 1 -- start start 2024-04-21T08:44:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.273+0000 7faa66973740 1 Processor -- start 2024-04-21T08:44:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.274+0000 7faa66973740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4580 0x55cbcddeaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.274+0000 7faa66973740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cbcdde4ea0 0x55cbcddeaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.274+0000 7faa66973740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cbcddeda80 0x55cbcddeb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.274+0000 7faa66973740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cbcddf0870 con 0x55cbcdde4580 2024-04-21T08:44:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.274+0000 7faa66973740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cbcdd022f0 con 0x55cbcdde4ea0 2024-04-21T08:44:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.274+0000 7faa66973740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55cbcddf09f0 con 0x55cbcddeda80 2024-04-21T08:44:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.274+0000 7faa66973740 10 client.0 _ll_register_callbacks cb 0x55cbcdd0f580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.274+0000 7faa61a41640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4580 0x55cbcddeaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.274+0000 7faa61a41640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4580 0x55cbcddeaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34468/0 (socket says 172.21.15.43:34468) 2024-04-21T08:44:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.274+0000 7faa61a41640 1 -- 172.21.15.43:0/2260489850 learned_addr learned my addr 172.21.15.43:0/2260489850 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.274+0000 7faa62242640 1 --2- 172.21.15.43:0/2260489850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cbcddeda80 0x55cbcddeb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.274+0000 7faa61240640 1 --2- 172.21.15.43:0/2260489850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cbcdde4ea0 0x55cbcddeaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.274+0000 7faa61a41640 1 -- 172.21.15.43:0/2260489850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cbcddeda80 msgr2=0x55cbcddeb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.274+0000 7faa61a41640 1 --2- 172.21.15.43:0/2260489850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cbcddeda80 0x55cbcddeb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.274+0000 7faa61a41640 1 -- 172.21.15.43:0/2260489850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cbcdde4ea0 msgr2=0x55cbcddeaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.274+0000 7faa61a41640 1 --2- 172.21.15.43:0/2260489850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cbcdde4ea0 0x55cbcddeaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.274+0000 7faa61a41640 1 -- 172.21.15.43:0/2260489850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbcda5b3f0 con 0x55cbcdde4580 2024-04-21T08:44:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.274+0000 7faa61a41640 1 --2- 172.21.15.43:0/2260489850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4580 0x55cbcddeaa00 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7faa58005710 tx=0x7faa58005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7efdaed885d1af2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.274+0000 7faa457fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.274+0000 7faa457fa640 1 -- 172.21.15.43:0/2260489850 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58015030 con 0x55cbcdde4580 2024-04-21T08:44:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.275+0000 7faa457fa640 1 -- 172.21.15.43:0/2260489850 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7faa58004d00 con 0x55cbcdde4580 2024-04-21T08:44:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.275+0000 7faa66973740 1 -- 172.21.15.43:0/2260489850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa580054d0 con 0x55cbcdde4580 2024-04-21T08:44:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.275+0000 7faa457fa640 1 -- 172.21.15.43:0/2260489850 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faa5802d050 con 0x55cbcdde4580 2024-04-21T08:44:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.275+0000 7faa66973740 10 client.17865 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.275+0000 7faa66973740 1 -- 172.21.15.43:0/2260489850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cbcda1b0f0 con 0x55cbcdde4580 2024-04-21T08:44:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.275+0000 7faa457fa640 1 -- 172.21.15.43:0/2260489850 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa580054d0 con 0x55cbcdde4580 2024-04-21T08:44:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.275+0000 7faa457fa640 1 -- 172.21.15.43:0/2260489850 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faa58011af0 con 0x55cbcdde4580 2024-04-21T08:44:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.275+0000 7faa66973740 10 client.17865 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.275+0000 7faa66973740 10 client.17865 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.275+0000 7faa66973740 20 client.17865 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.275+0000 7faa66973740 1 --2- 172.21.15.43:0/2260489850 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cbcddfd940 0x55cbcddffd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.275+0000 7faa66973740 4 client.17865 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.275+0000 7faa66973740 1 -- 172.21.15.43:0/2260489850 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbcdd56c00 con 0x55cbcddfd940 2024-04-21T08:44:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.275+0000 7faa61240640 1 --2- 172.21.15.43:0/2260489850 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cbcddfd940 0x55cbcddffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.276+0000 7faa61240640 1 --2- 172.21.15.43:0/2260489850 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cbcddfd940 0x55cbcddffd30 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a680027ba248a22d server_cookie=1aef347db10203b4 in_seq=0 out_seq=0 2024-04-21T08:44:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.276+0000 7faa457fa640 10 client.17865 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.277+0000 7faa457fa640 1 -- 172.21.15.43:0/2260489850 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cbcdd56c00 con 0x55cbcddfd940 2024-04-21T08:44:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.277+0000 7faa457fa640 10 client.17865 handle_command_reply: tid=0 2024-04-21T08:44:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.277+0000 7faa66973740 1 client.17865 shutdown 2024-04-21T08:44:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.277+0000 7faa66973740 1 -- 172.21.15.43:0/2260489850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4580 msgr2=0x55cbcddeaa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.277+0000 7faa66973740 1 --2- 172.21.15.43:0/2260489850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4580 0x55cbcddeaa00 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7faa58005710 tx=0x7faa58005740 comp rx=0 tx=0).stop 2024-04-21T08:44:24.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.277+0000 7faa66973740 1 -- 172.21.15.43:0/2260489850 shutdown_connections 2024-04-21T08:44:24.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.277+0000 7faa66973740 1 --2- 172.21.15.43:0/2260489850 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cbcddfd940 0x55cbcddffd30 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.277+0000 7faa66973740 1 --2- 172.21.15.43:0/2260489850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cbcddeda80 0x55cbcddeb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.277+0000 7faa457fa640 0 client.17865 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:24.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.277+0000 7faa66973740 1 --2- 172.21.15.43:0/2260489850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cbcdde4ea0 0x55cbcddeaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.277+0000 7faa66973740 1 --2- 172.21.15.43:0/2260489850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4580 0x55cbcddeaa00 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.277+0000 7faa66973740 1 -- 172.21.15.43:0/2260489850 >> 172.21.15.43:0/2260489850 conn(0x55cbcdddecb0 msgr2=0x55cbcdde0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:24.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.277+0000 7faa66973740 1 -- 172.21.15.43:0/2260489850 shutdown_connections 2024-04-21T08:44:24.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.277+0000 7faa66973740 1 -- 172.21.15.43:0/2260489850 wait complete. 2024-04-21T08:44:24.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.277+0000 7faa66973740 20 client.17865 trim_cache size 0 max 16384 2024-04-21T08:44:24.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.295+0000 7faa66973740 1 Processor -- start 2024-04-21T08:44:24.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.295+0000 7faa66973740 1 -- start start 2024-04-21T08:44:24.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.295+0000 7faa66973740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4580 0x55cbcdded130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.295+0000 7faa66973740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cbcdde4ea0 0x55cbcddf9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.295+0000 7faa66973740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cbcddeda80 0x55cbcddddbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.295+0000 7faa66973740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c1b15a0 con 0x55cbcdde4580 2024-04-21T08:44:24.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.295+0000 7faa66973740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c1b30e0 con 0x55cbcdde4ea0 2024-04-21T08:44:24.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.295+0000 7faa66973740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c1b1c70 con 0x55cbcddeda80 2024-04-21T08:44:24.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.295+0000 7faa61a41640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4580 0x55cbcdded130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.295+0000 7faa61a41640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4580 0x55cbcdded130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34482/0 (socket says 172.21.15.43:34482) 2024-04-21T08:44:24.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.295+0000 7faa61a41640 1 -- 172.21.15.43:0/1920323682 learned_addr learned my addr 172.21.15.43:0/1920323682 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:24.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.295+0000 7faa61240640 1 --2- 172.21.15.43:0/1920323682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cbcdde4ea0 0x55cbcddf9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.295+0000 7faa62242640 1 --2- 172.21.15.43:0/1920323682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cbcddeda80 0x55cbcddddbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.295+0000 7faa61a41640 1 -- 172.21.15.43:0/1920323682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cbcddeda80 msgr2=0x55cbcddddbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:24.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.295+0000 7faa61a41640 1 --2- 172.21.15.43:0/1920323682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cbcddeda80 0x55cbcddddbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.295+0000 7faa61a41640 1 -- 172.21.15.43:0/1920323682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cbcdde4ea0 msgr2=0x55cbcddf9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:24.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.295+0000 7faa61a41640 1 --2- 172.21.15.43:0/1920323682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cbcdde4ea0 0x55cbcddf9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.295+0000 7faa61a41640 1 -- 172.21.15.43:0/1920323682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbcdddeae0 con 0x55cbcdde4580 2024-04-21T08:44:24.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.296+0000 7faa61a41640 1 --2- 172.21.15.43:0/1920323682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4580 0x55cbcdded130 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7faa5800f770 tx=0x7faa58004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef6f0f550b0af79 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:24.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.296+0000 7faa477fe640 1 -- 172.21.15.43:0/1920323682 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5801d070 con 0x55cbcdde4580 2024-04-21T08:44:24.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.296+0000 7faa477fe640 1 -- 172.21.15.43:0/1920323682 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa58024e40 con 0x55cbcdde4580 2024-04-21T08:44:24.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.296+0000 7faa477fe640 1 -- 172.21.15.43:0/1920323682 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58015070 con 0x55cbcdde4580 2024-04-21T08:44:24.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.296+0000 7faa66973740 1 -- 172.21.15.43:0/1920323682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4580 msgr2=0x55cbcdded130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.296+0000 7faa66973740 1 --2- 172.21.15.43:0/1920323682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4580 0x55cbcdded130 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7faa5800f770 tx=0x7faa58004a60 comp rx=0 tx=0).stop 2024-04-21T08:44:24.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.296+0000 7faa66973740 1 -- 172.21.15.43:0/1920323682 shutdown_connections 2024-04-21T08:44:24.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.296+0000 7faa66973740 1 --2- 172.21.15.43:0/1920323682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cbcddeda80 0x55cbcddddbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.296+0000 7faa66973740 1 --2- 172.21.15.43:0/1920323682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cbcdde4ea0 0x55cbcddf9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.296+0000 7faa66973740 1 --2- 172.21.15.43:0/1920323682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4580 0x55cbcdded130 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.296+0000 7faa66973740 1 -- 172.21.15.43:0/1920323682 >> 172.21.15.43:0/1920323682 conn(0x55cbcdddecb0 msgr2=0x55cbcdddd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:24.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.296+0000 7faa66973740 1 -- 172.21.15.43:0/1920323682 shutdown_connections 2024-04-21T08:44:24.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.296+0000 7faa66973740 1 -- 172.21.15.43:0/1920323682 wait complete. 2024-04-21T08:44:24.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.297+0000 7faa66973740 1 -- start start 2024-04-21T08:44:24.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.297+0000 7faa66973740 1 Processor -- start 2024-04-21T08:44:24.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.297+0000 7faa66973740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cbcdde4580 0x55cbcdde7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.297+0000 7faa66973740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4ea0 0x55cbcdde7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.297+0000 7faa66973740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cbcddeda80 0x55cbcdde82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.297+0000 7faa66973740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c1b2a10 con 0x55cbcdde4ea0 2024-04-21T08:44:24.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.297+0000 7faa66973740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c1b2340 con 0x55cbcdde4580 2024-04-21T08:44:24.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.297+0000 7faa66973740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c1af580 con 0x55cbcddeda80 2024-04-21T08:44:24.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.297+0000 7faa66973740 10 client.0 _ll_register_callbacks cb 0x7faa54037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:24.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.297+0000 7faa61240640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4ea0 0x55cbcdde7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.297+0000 7faa61240640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4ea0 0x55cbcdde7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34494/0 (socket says 172.21.15.43:34494) 2024-04-21T08:44:24.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.297+0000 7faa61240640 1 -- 172.21.15.43:0/2029698219 learned_addr learned my addr 172.21.15.43:0/2029698219 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:24.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.297+0000 7faa61a41640 1 --2- 172.21.15.43:0/2029698219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cbcdde4580 0x55cbcdde7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.298+0000 7faa62242640 1 --2- 172.21.15.43:0/2029698219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cbcddeda80 0x55cbcdde82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.298+0000 7faa61240640 1 -- 172.21.15.43:0/2029698219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cbcddeda80 msgr2=0x55cbcdde82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:24.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.298+0000 7faa61240640 1 --2- 172.21.15.43:0/2029698219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cbcddeda80 0x55cbcdde82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.298+0000 7faa61240640 1 -- 172.21.15.43:0/2029698219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cbcdde4580 msgr2=0x55cbcdde7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:24.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.298+0000 7faa61240640 1 --2- 172.21.15.43:0/2029698219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cbcdde4580 0x55cbcdde7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.298+0000 7faa61240640 1 -- 172.21.15.43:0/2029698219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbcddefe70 con 0x55cbcdde4ea0 2024-04-21T08:44:24.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.298+0000 7faa62242640 1 --2- 172.21.15.43:0/2029698219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cbcddeda80 0x55cbcdde82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:24.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.298+0000 7faa61240640 1 --2- 172.21.15.43:0/2029698219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4ea0 0x55cbcdde7d60 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7faa4c09c8c0 tx=0x7faa4c0d3500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bcc9523f9764fa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:24.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.298+0000 7faa457fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:24.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.298+0000 7faa457fa640 1 -- 172.21.15.43:0/2029698219 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c0a5830 con 0x55cbcdde4ea0 2024-04-21T08:44:24.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.298+0000 7faa457fa640 1 -- 172.21.15.43:0/2029698219 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7faa4c0ce5f0 con 0x55cbcdde4ea0 2024-04-21T08:44:24.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.298+0000 7faa66973740 1 -- 172.21.15.43:0/2029698219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cbcdd50a70 con 0x55cbcdde4ea0 2024-04-21T08:44:24.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.298+0000 7faa457fa640 1 -- 172.21.15.43:0/2029698219 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faa4c0bd0b0 con 0x55cbcdde4ea0 2024-04-21T08:44:24.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.298+0000 7faa66973740 10 client.17871 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:24.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.298+0000 7faa66973740 1 -- 172.21.15.43:0/2029698219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cbcde63e50 con 0x55cbcdde4ea0 2024-04-21T08:44:24.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.298+0000 7faa457fa640 1 -- 172.21.15.43:0/2029698219 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55cbcdd50a70 con 0x55cbcdde4ea0 2024-04-21T08:44:24.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.299+0000 7faa457fa640 1 -- 172.21.15.43:0/2029698219 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faa4c0a5430 con 0x55cbcdde4ea0 2024-04-21T08:44:24.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.299+0000 7faa66973740 10 client.17871 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:24.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.299+0000 7faa66973740 10 client.17871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:24.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.299+0000 7faa66973740 20 client.17871 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:24.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.299+0000 7faa66973740 1 --2- 172.21.15.43:0/2029698219 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cbcde64580 0x55cbcde669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.299+0000 7faa66973740 4 client.17871 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:44:24.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.299+0000 7faa66973740 1 -- 172.21.15.43:0/2029698219 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cbcdccc590 con 0x55cbcde64580 2024-04-21T08:44:24.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.299+0000 7faa61a41640 1 --2- 172.21.15.43:0/2029698219 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cbcde64580 0x55cbcde669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.299+0000 7faa61a41640 1 --2- 172.21.15.43:0/2029698219 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cbcde64580 0x55cbcde669a0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46b71ef425c71daa server_cookie=96d00af10dbee2e7 in_seq=0 out_seq=0 2024-04-21T08:44:24.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.299+0000 7faa457fa640 10 client.17871 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:24.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.299+0000 7faa457fa640 1 -- 172.21.15.43:0/2029698219 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cbcdccc590 con 0x55cbcde64580 2024-04-21T08:44:24.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.299+0000 7faa457fa640 10 client.17871 handle_command_reply: tid=0 2024-04-21T08:44:24.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.299+0000 7faa66973740 1 client.17871 shutdown 2024-04-21T08:44:24.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.300+0000 7faa66973740 1 -- 172.21.15.43:0/2029698219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4ea0 msgr2=0x55cbcdde7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.300+0000 7faa66973740 1 --2- 172.21.15.43:0/2029698219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4ea0 0x55cbcdde7d60 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7faa4c09c8c0 tx=0x7faa4c0d3500 comp rx=0 tx=0).stop 2024-04-21T08:44:24.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.300+0000 7faa66973740 1 -- 172.21.15.43:0/2029698219 shutdown_connections 2024-04-21T08:44:24.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.300+0000 7faa66973740 1 --2- 172.21.15.43:0/2029698219 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cbcde64580 0x55cbcde669a0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.300+0000 7faa66973740 1 --2- 172.21.15.43:0/2029698219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cbcddeda80 0x55cbcdde82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.300+0000 7faa66973740 1 --2- 172.21.15.43:0/2029698219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cbcdde4ea0 0x55cbcdde7d60 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.300+0000 7faa66973740 1 --2- 172.21.15.43:0/2029698219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cbcdde4580 0x55cbcdde7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.300+0000 7faa457fa640 0 client.17871 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:24.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.300+0000 7faa66973740 1 -- 172.21.15.43:0/2029698219 >> 172.21.15.43:0/2029698219 conn(0x55cbcdddecb0 msgr2=0x55cbcddddd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:24.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.300+0000 7faa66973740 1 -- 172.21.15.43:0/2029698219 shutdown_connections 2024-04-21T08:44:24.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.300+0000 7faa66973740 1 -- 172.21.15.43:0/2029698219 wait complete. 2024-04-21T08:44:24.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.300+0000 7faa66973740 20 client.17871 trim_cache size 0 max 16384 2024-04-21T08:44:24.306 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:44:24.306 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:44:24.306 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:44:24.306 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:44:24.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.300+0000 7faa63ca5640 1 -- 172.21.15.43:0/1584642950 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faa480a3d20 msgr2=0x7faa480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.300+0000 7faa63ca5640 1 --2- 172.21.15.43:0/1584642950 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faa480a3d20 0x7faa480a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faa54006010 tx=0x7faa540095c0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.300+0000 7faa63ca5640 1 -- 172.21.15.43:0/1584642950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa5c06af70 msgr2=0x7faa5c1016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.300+0000 7faa63ca5640 1 --2- 172.21.15.43:0/1584642950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa5c06af70 0x7faa5c1016b0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00e9f0 tx=0x7faa4c00eeb0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.300+0000 7faa63ca5640 1 -- 172.21.15.43:0/1584642950 shutdown_connections 2024-04-21T08:44:24.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.300+0000 7faa63ca5640 1 --2- 172.21.15.43:0/1584642950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa5c108060 0x7faa5c101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.300+0000 7faa63ca5640 1 --2- 172.21.15.43:0/1584642950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa5c06af70 0x7faa5c1016b0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.300+0000 7faa63ca5640 1 --2- 172.21.15.43:0/1584642950 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faa480a3d20 0x7faa480a61e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.300+0000 7faa63ca5640 1 --2- 172.21.15.43:0/1584642950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa5c06a5b0 0x7faa5c101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.300+0000 7faa63ca5640 1 -- 172.21.15.43:0/1584642950 >> 172.21.15.43:0/1584642950 conn(0x7faa5c077c40 msgr2=0x7faa5c0ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:24.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.300+0000 7faa63ca5640 1 -- 172.21.15.43:0/1584642950 shutdown_connections 2024-04-21T08:44:24.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.300+0000 7faa63ca5640 1 -- 172.21.15.43:0/1584642950 wait complete. 2024-04-21T08:44:24.309 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:24 smithi043 ceph-mon[27978]: pgmap v167: 129 pgs: 129 active+clean; 3.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-21T08:44:24.311 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:44:24.311 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:44:24.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.414+0000 7fb9cd34b640 1 -- 172.21.15.43:0/2255271456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9c8102b50 msgr2=0x7fb9c8106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.414+0000 7fb9cd34b640 1 --2- 172.21.15.43:0/2255271456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9c8102b50 0x7fb9c8106b90 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009900 tx=0x7fb9b002bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:24.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.414+0000 7fb9cd34b640 1 -- 172.21.15.43:0/2255271456 shutdown_connections 2024-04-21T08:44:24.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.414+0000 7fb9cd34b640 1 --2- 172.21.15.43:0/2255271456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c8107270 0x7fb9c810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.414+0000 7fb9cd34b640 1 --2- 172.21.15.43:0/2255271456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9c8102b50 0x7fb9c8106b90 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.414+0000 7fb9cd34b640 1 --2- 172.21.15.43:0/2255271456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9c81021a0 0x7fb9c8102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.414+0000 7fb9cd34b640 1 -- 172.21.15.43:0/2255271456 >> 172.21.15.43:0/2255271456 conn(0x7fb9c80793a0 msgr2=0x7fb9c80797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:24.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.414+0000 7fb9cd34b640 1 -- 172.21.15.43:0/2255271456 shutdown_connections 2024-04-21T08:44:24.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.414+0000 7fb9cd34b640 1 -- 172.21.15.43:0/2255271456 wait complete. 2024-04-21T08:44:24.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.415+0000 7fb9cd34b640 1 Processor -- start 2024-04-21T08:44:24.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.415+0000 7fb9cd34b640 1 -- start start 2024-04-21T08:44:24.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.415+0000 7fb9cd34b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9c81021a0 0x7fb9c8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.415+0000 7fb9cd34b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c8107270 0x7fb9c8198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.415+0000 7fb9cd34b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9c819cfd0 0x7fb9c819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.415+0000 7fb9cd34b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c81121b0 con 0x7fb9c819cfd0 2024-04-21T08:44:24.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.415+0000 7fb9cd34b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c8112030 con 0x7fb9c81021a0 2024-04-21T08:44:24.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.415+0000 7fb9cd34b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c8112330 con 0x7fb9c8107270 2024-04-21T08:44:24.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.415+0000 7fb9c77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9c819cfd0 0x7fb9c819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.415+0000 7fb9c77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9c819cfd0 0x7fb9c819d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34514/0 (socket says 172.21.15.43:34514) 2024-04-21T08:44:24.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.415+0000 7fb9c77fe640 1 -- 172.21.15.43:0/3895668211 learned_addr learned my addr 172.21.15.43:0/3895668211 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:24.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.415+0000 7fb9c6ffd640 1 --2- 172.21.15.43:0/3895668211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9c81021a0 0x7fb9c8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.415+0000 7fb9bffff640 1 --2- 172.21.15.43:0/3895668211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c8107270 0x7fb9c8198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.415+0000 7fb9c77fe640 1 -- 172.21.15.43:0/3895668211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c8107270 msgr2=0x7fb9c8198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.415+0000 7fb9c77fe640 1 --2- 172.21.15.43:0/3895668211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c8107270 0x7fb9c8198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.416+0000 7fb9c77fe640 1 -- 172.21.15.43:0/3895668211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9c81021a0 msgr2=0x7fb9c8198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.416+0000 7fb9c77fe640 1 --2- 172.21.15.43:0/3895668211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9c81021a0 0x7fb9c8198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.416+0000 7fb9c77fe640 1 -- 172.21.15.43:0/3895668211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c819dac0 con 0x7fb9c819cfd0 2024-04-21T08:44:24.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.416+0000 7fb9bffff640 1 --2- 172.21.15.43:0/3895668211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c8107270 0x7fb9c8198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:24.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.416+0000 7fb9c77fe640 1 --2- 172.21.15.43:0/3895668211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9c819cfd0 0x7fb9c819d480 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000eb10 tx=0x7fb9c000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:24.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.416+0000 7fb9c4ff9640 1 -- 172.21.15.43:0/3895668211 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0004b20 con 0x7fb9c819cfd0 2024-04-21T08:44:24.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.416+0000 7fb9cd34b640 1 -- 172.21.15.43:0/3895668211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c819ddb0 con 0x7fb9c819cfd0 2024-04-21T08:44:24.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.416+0000 7fb9c4ff9640 1 -- 172.21.15.43:0/3895668211 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9c0004cc0 con 0x7fb9c819cfd0 2024-04-21T08:44:24.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.416+0000 7fb9cd34b640 1 -- 172.21.15.43:0/3895668211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c81a55f0 con 0x7fb9c819cfd0 2024-04-21T08:44:24.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.416+0000 7fb9c4ff9640 1 -- 172.21.15.43:0/3895668211 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0018630 con 0x7fb9c819cfd0 2024-04-21T08:44:24.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.417+0000 7fb9cd34b640 1 -- 172.21.15.43:0/3895668211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9c8069230 con 0x7fb9c819cfd0 2024-04-21T08:44:24.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.419+0000 7fb9c4ff9640 1 -- 172.21.15.43:0/3895668211 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb9c0005000 con 0x7fb9c819cfd0 2024-04-21T08:44:24.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.419+0000 7fb9c4ff9640 1 --2- 172.21.15.43:0/3895668211 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb9b40a3d20 0x7fb9b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.420+0000 7fb9c4ff9640 1 -- 172.21.15.43:0/3895668211 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fb9c00d18d0 con 0x7fb9c819cfd0 2024-04-21T08:44:24.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.420+0000 7fb9c6ffd640 1 --2- 172.21.15.43:0/3895668211 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb9b40a3d20 0x7fb9b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.420+0000 7fb9c6ffd640 1 --2- 172.21.15.43:0/3895668211 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb9b40a3d20 0x7fb9b40a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8009bd0 tx=0x7fb9b8005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.423+0000 7fb9c4ff9640 1 -- 172.21.15.43:0/3895668211 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb9c009db40 con 0x7fb9c819cfd0 2024-04-21T08:44:24.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:24 smithi175 ceph-mon[31920]: pgmap v167: 129 pgs: 129 active+clean; 3.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-21T08:44:24.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:24 smithi112 ceph-mon[31085]: pgmap v167: 129 pgs: 129 active+clean; 3.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-21T08:44:24.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.654+0000 7fb9cd34b640 1 -- 172.21.15.43:0/3895668211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9c8199a60 con 0x7fb9c819cfd0 2024-04-21T08:44:24.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.654+0000 7fb9c4ff9640 1 -- 172.21.15.43:0/3895668211 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7fb9c8199a60 con 0x7fb9c819cfd0 2024-04-21T08:44:24.654 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:44:24.654 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:44:24.654 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:44:24.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.657+0000 7fb9cd34b640 1 -- 172.21.15.43:0/3895668211 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb9b40a3d20 msgr2=0x7fb9b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.657+0000 7fb9cd34b640 1 --2- 172.21.15.43:0/3895668211 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb9b40a3d20 0x7fb9b40a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8009bd0 tx=0x7fb9b8005f90 comp rx=0 tx=0).stop 2024-04-21T08:44:24.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.657+0000 7fb9cd34b640 1 -- 172.21.15.43:0/3895668211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9c819cfd0 msgr2=0x7fb9c819d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.657+0000 7fb9cd34b640 1 --2- 172.21.15.43:0/3895668211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9c819cfd0 0x7fb9c819d480 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000eb10 tx=0x7fb9c000c2d0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.657+0000 7fb9cd34b640 1 -- 172.21.15.43:0/3895668211 shutdown_connections 2024-04-21T08:44:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.657+0000 7fb9cd34b640 1 --2- 172.21.15.43:0/3895668211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9c819cfd0 0x7fb9c819d480 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.657+0000 7fb9cd34b640 1 --2- 172.21.15.43:0/3895668211 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb9b40a3d20 0x7fb9b40a61e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.657+0000 7fb9cd34b640 1 --2- 172.21.15.43:0/3895668211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c8107270 0x7fb9c8198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.657+0000 7fb9cd34b640 1 --2- 172.21.15.43:0/3895668211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9c81021a0 0x7fb9c8198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.657+0000 7fb9cd34b640 1 -- 172.21.15.43:0/3895668211 >> 172.21.15.43:0/3895668211 conn(0x7fb9c80793a0 msgr2=0x7fb9c80fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.657+0000 7fb9cd34b640 1 -- 172.21.15.43:0/3895668211 shutdown_connections 2024-04-21T08:44:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.657+0000 7fb9cd34b640 1 -- 172.21.15.43:0/3895668211 wait complete. 2024-04-21T08:44:24.668 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:44:24.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.772+0000 7f5b6defb640 1 -- 172.21.15.43:0/838479619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b68106160 msgr2=0x7f5b6810dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.773+0000 7f5b6defb640 1 --2- 172.21.15.43:0/838479619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b68106160 0x7f5b6810dd30 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009900 tx=0x7f5b6002bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.773+0000 7f5b6defb640 1 -- 172.21.15.43:0/838479619 shutdown_connections 2024-04-21T08:44:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.773+0000 7f5b6defb640 1 --2- 172.21.15.43:0/838479619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b68106160 0x7f5b6810dd30 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.773+0000 7f5b6defb640 1 --2- 172.21.15.43:0/838479619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b68101a40 0x7f5b68105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.773+0000 7f5b6defb640 1 --2- 172.21.15.43:0/838479619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b68101090 0x7f5b68101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.773+0000 7f5b6defb640 1 -- 172.21.15.43:0/838479619 >> 172.21.15.43:0/838479619 conn(0x7f5b68077f50 msgr2=0x7f5b68078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.773+0000 7f5b6defb640 1 -- 172.21.15.43:0/838479619 shutdown_connections 2024-04-21T08:44:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.773+0000 7f5b6defb640 1 -- 172.21.15.43:0/838479619 wait complete. 2024-04-21T08:44:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.773+0000 7f5b6defb640 1 Processor -- start 2024-04-21T08:44:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.773+0000 7f5b6defb640 1 -- start start 2024-04-21T08:44:24.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.774+0000 7f5b6defb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b68101090 0x7f5b68198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.774+0000 7f5b6defb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b68106160 0x7f5b68198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.774+0000 7f5b6defb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b6819cfd0 0x7f5b6819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.774+0000 7f5b6defb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681110d0 con 0x7f5b68106160 2024-04-21T08:44:24.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.774+0000 7f5b6defb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b68110f50 con 0x7f5b68101090 2024-04-21T08:44:24.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.774+0000 7f5b6defb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5b68111250 con 0x7f5b6819cfd0 2024-04-21T08:44:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.774+0000 7f5b66ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b68106160 0x7f5b68198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.774+0000 7f5b66ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b68106160 0x7f5b68198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34534/0 (socket says 172.21.15.43:34534) 2024-04-21T08:44:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.774+0000 7f5b66ffd640 1 -- 172.21.15.43:0/4059682758 learned_addr learned my addr 172.21.15.43:0/4059682758 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.774+0000 7f5b677fe640 1 --2- 172.21.15.43:0/4059682758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b68101090 0x7f5b68198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.774+0000 7f5b67fff640 1 --2- 172.21.15.43:0/4059682758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b6819cfd0 0x7f5b6819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.774+0000 7f5b66ffd640 1 -- 172.21.15.43:0/4059682758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b6819cfd0 msgr2=0x7f5b6819d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.774+0000 7f5b66ffd640 1 --2- 172.21.15.43:0/4059682758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b6819cfd0 0x7f5b6819d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.774+0000 7f5b66ffd640 1 -- 172.21.15.43:0/4059682758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b68101090 msgr2=0x7f5b68198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.774+0000 7f5b66ffd640 1 --2- 172.21.15.43:0/4059682758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b68101090 0x7f5b68198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.774+0000 7f5b66ffd640 1 -- 172.21.15.43:0/4059682758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6819dac0 con 0x7f5b68106160 2024-04-21T08:44:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.774+0000 7f5b66ffd640 1 --2- 172.21.15.43:0/4059682758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b68106160 0x7f5b68198c50 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f5b5000e9c0 tx=0x7f5b5000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.774+0000 7f5b64ff9640 1 -- 172.21.15.43:0/4059682758 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5000cd80 con 0x7f5b68106160 2024-04-21T08:44:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.774+0000 7f5b64ff9640 1 -- 172.21.15.43:0/4059682758 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b500054d0 con 0x7f5b68106160 2024-04-21T08:44:24.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.774+0000 7f5b6defb640 1 -- 172.21.15.43:0/4059682758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6819ddb0 con 0x7f5b68106160 2024-04-21T08:44:24.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.774+0000 7f5b64ff9640 1 -- 172.21.15.43:0/4059682758 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b50010430 con 0x7f5b68106160 2024-04-21T08:44:24.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.774+0000 7f5b6defb640 1 -- 172.21.15.43:0/4059682758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b681a55f0 con 0x7f5b68106160 2024-04-21T08:44:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.777+0000 7f5b64ff9640 1 -- 172.21.15.43:0/4059682758 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5b50004fd0 con 0x7f5b68106160 2024-04-21T08:44:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.777+0000 7f5b70bc5740 1 Processor -- start 2024-04-21T08:44:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.777+0000 7f5b70bc5740 1 -- start start 2024-04-21T08:44:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.777+0000 7f5b70bc5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ece4d0 0x55da16ece8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.777+0000 7f5b70bc5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da16ecee80 0x55da16ed7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.777+0000 7f5b70bc5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da16ed7a00 0x55da16ed9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.777+0000 7f5b70bc5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55da16e331f0 con 0x55da16ece4d0 2024-04-21T08:44:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.777+0000 7f5b70bc5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55da16dd04b0 con 0x55da16ecee80 2024-04-21T08:44:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.777+0000 7f5b70bc5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55da16e2f1b0 con 0x55da16ed7a00 2024-04-21T08:44:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.777+0000 7f5b67fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ece4d0 0x55da16ece8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.777+0000 7f5b67fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ece4d0 0x55da16ece8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34550/0 (socket says 172.21.15.43:34550) 2024-04-21T08:44:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.777+0000 7f5b67fff640 1 -- 172.21.15.43:0/541513168 learned_addr learned my addr 172.21.15.43:0/541513168 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.777+0000 7f5b677fe640 1 --2- 172.21.15.43:0/541513168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da16ecee80 0x55da16ed7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.778+0000 7f5b66ffd640 1 --2- 172.21.15.43:0/541513168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da16ed7a00 0x55da16ed9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.778+0000 7f5b67fff640 1 -- 172.21.15.43:0/541513168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da16ed7a00 msgr2=0x55da16ed9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.778+0000 7f5b67fff640 1 --2- 172.21.15.43:0/541513168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da16ed7a00 0x55da16ed9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.778+0000 7f5b67fff640 1 -- 172.21.15.43:0/541513168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da16ecee80 msgr2=0x55da16ed7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.778+0000 7f5b67fff640 1 --2- 172.21.15.43:0/541513168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da16ecee80 0x55da16ed7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.778+0000 7f5b67fff640 1 -- 172.21.15.43:0/541513168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da16dc21b0 con 0x55da16ece4d0 2024-04-21T08:44:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.778+0000 7f5b64ff9640 1 --2- 172.21.15.43:0/4059682758 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5b540a3da0 0x7f5b540a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.778+0000 7f5b677fe640 1 --2- 172.21.15.43:0/4059682758 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5b540a3da0 0x7f5b540a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.778+0000 7f5b64ff9640 1 -- 172.21.15.43:0/4059682758 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f5b500d0f40 con 0x7f5b68106160 2024-04-21T08:44:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.778+0000 7f5b67fff640 1 --2- 172.21.15.43:0/541513168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ece4d0 0x55da16ece8b0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009900 tx=0x7f5b60005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9816ab3f103124c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.778+0000 7f5b5dffb640 1 -- 172.21.15.43:0/541513168 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60044020 con 0x55da16ece4d0 2024-04-21T08:44:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.778+0000 7f5b5dffb640 1 -- 172.21.15.43:0/541513168 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b6002cc90 con 0x55da16ece4d0 2024-04-21T08:44:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.778+0000 7f5b5dffb640 1 -- 172.21.15.43:0/541513168 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b600046c0 con 0x55da16ece4d0 2024-04-21T08:44:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.779+0000 7f5b70bc5740 1 -- 172.21.15.43:0/541513168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ece4d0 msgr2=0x55da16ece8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.779+0000 7f5b677fe640 1 --2- 172.21.15.43:0/4059682758 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5b540a3da0 0x7f5b540a6260 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55da16ec9c10 tx=0x7f5b58009500 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.779+0000 7f5b70bc5740 1 --2- 172.21.15.43:0/541513168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ece4d0 0x55da16ece8b0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009900 tx=0x7f5b60005d50 comp rx=0 tx=0).stop 2024-04-21T08:44:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.779+0000 7f5b70bc5740 1 -- 172.21.15.43:0/541513168 shutdown_connections 2024-04-21T08:44:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.779+0000 7f5b70bc5740 1 --2- 172.21.15.43:0/541513168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da16ed7a00 0x55da16ed9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.779+0000 7f5b70bc5740 1 --2- 172.21.15.43:0/541513168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da16ecee80 0x55da16ed7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.779+0000 7f5b70bc5740 1 --2- 172.21.15.43:0/541513168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ece4d0 0x55da16ece8b0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.779+0000 7f5b70bc5740 1 -- 172.21.15.43:0/541513168 >> 172.21.15.43:0/541513168 conn(0x55da16ec8ba0 msgr2=0x55da16ec8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.779+0000 7f5b70bc5740 1 -- 172.21.15.43:0/541513168 shutdown_connections 2024-04-21T08:44:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.779+0000 7f5b70bc5740 1 -- 172.21.15.43:0/541513168 wait complete. 2024-04-21T08:44:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.779+0000 7f5b70bc5740 1 -- start start 2024-04-21T08:44:24.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.779+0000 7f5b70bc5740 1 Processor -- start 2024-04-21T08:44:24.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.780+0000 7f5b70bc5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da16ece4d0 0x55da16ed7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.780+0000 7f5b70bc5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da16ecee80 0x55da16ed46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.780+0000 7f5b70bc5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ed7a00 0x55da16ed4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.780+0000 7f5b70bc5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55da16eda910 con 0x55da16ed7a00 2024-04-21T08:44:24.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.780+0000 7f5b70bc5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55da16dec500 con 0x55da16ecee80 2024-04-21T08:44:24.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.780+0000 7f5b70bc5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55da16edaa90 con 0x55da16ece4d0 2024-04-21T08:44:24.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.780+0000 7f5b70bc5740 10 client.0 _ll_register_callbacks cb 0x55da16dbab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:24.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.780+0000 7f5b67fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ed7a00 0x55da16ed4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.780+0000 7f5b67fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ed7a00 0x55da16ed4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34566/0 (socket says 172.21.15.43:34566) 2024-04-21T08:44:24.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.780+0000 7f5b67fff640 1 -- 172.21.15.43:0/1993782809 learned_addr learned my addr 172.21.15.43:0/1993782809 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:24.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.780+0000 7f5b677fe640 1 --2- 172.21.15.43:0/1993782809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da16ece4d0 0x55da16ed7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.780+0000 7f5b66ffd640 1 --2- 172.21.15.43:0/1993782809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da16ecee80 0x55da16ed46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.780+0000 7f5b67fff640 1 -- 172.21.15.43:0/1993782809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da16ece4d0 msgr2=0x55da16ed7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:24.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.780+0000 7f5b67fff640 1 --2- 172.21.15.43:0/1993782809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da16ece4d0 0x55da16ed7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.780+0000 7f5b67fff640 1 -- 172.21.15.43:0/1993782809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da16ecee80 msgr2=0x55da16ed46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:24.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.780+0000 7f5b67fff640 1 --2- 172.21.15.43:0/1993782809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da16ecee80 0x55da16ed46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.780+0000 7f5b67fff640 1 -- 172.21.15.43:0/1993782809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da16b453f0 con 0x55da16ed7a00 2024-04-21T08:44:24.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.780+0000 7f5b67fff640 1 --2- 172.21.15.43:0/1993782809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ed7a00 0x55da16ed4cd0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f5b60004620 tx=0x7f5b6002ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633f21f6fea2e472 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:24.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.780+0000 7f5b46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:24.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.781+0000 7f5b46ffd640 1 -- 172.21.15.43:0/1993782809 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60044030 con 0x55da16ed7a00 2024-04-21T08:44:24.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.781+0000 7f5b46ffd640 1 -- 172.21.15.43:0/1993782809 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f5b60040880 con 0x55da16ed7a00 2024-04-21T08:44:24.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.781+0000 7f5b70bc5740 1 -- 172.21.15.43:0/1993782809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b6002cc90 con 0x55da16ed7a00 2024-04-21T08:44:24.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.781+0000 7f5b46ffd640 1 -- 172.21.15.43:0/1993782809 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5b6005a050 con 0x55da16ed7a00 2024-04-21T08:44:24.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.781+0000 7f5b70bc5740 10 client.17889 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:24.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.781+0000 7f5b70bc5740 1 -- 172.21.15.43:0/1993782809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55da16c0f7d0 con 0x55da16ed7a00 2024-04-21T08:44:24.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.781+0000 7f5b46ffd640 1 -- 172.21.15.43:0/1993782809 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b6002cc90 con 0x55da16ed7a00 2024-04-21T08:44:24.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.781+0000 7f5b46ffd640 1 -- 172.21.15.43:0/1993782809 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5b60004d70 con 0x55da16ed7a00 2024-04-21T08:44:24.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.781+0000 7f5b70bc5740 10 client.17889 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:24.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.781+0000 7f5b70bc5740 10 client.17889 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:44:24.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.781+0000 7f5b70bc5740 20 client.17889 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:24.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.781+0000 7f5b70bc5740 1 --2- 172.21.15.43:0/1993782809 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55da16ee7a60 0x55da16ee9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.781+0000 7f5b70bc5740 4 client.17889 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:24.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.781+0000 7f5b70bc5740 1 -- 172.21.15.43:0/1993782809 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da16dd2930 con 0x55da16ee7a60 2024-04-21T08:44:24.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.782+0000 7f5b677fe640 1 --2- 172.21.15.43:0/1993782809 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55da16ee7a60 0x55da16ee9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.782+0000 7f5b677fe640 1 --2- 172.21.15.43:0/1993782809 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55da16ee7a60 0x55da16ee9e50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=40aa0f2736ea9f33 server_cookie=35af96a15ed6e98d in_seq=0 out_seq=0 2024-04-21T08:44:24.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.782+0000 7f5b46ffd640 10 client.17889 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:44:24.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.783+0000 7f5b46ffd640 1 -- 172.21.15.43:0/1993782809 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da16dd2930 con 0x55da16ee7a60 2024-04-21T08:44:24.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.783+0000 7f5b46ffd640 10 client.17889 handle_command_reply: tid=0 2024-04-21T08:44:24.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.783+0000 7f5b70bc5740 1 client.17889 shutdown 2024-04-21T08:44:24.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.783+0000 7f5b70bc5740 1 -- 172.21.15.43:0/1993782809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ed7a00 msgr2=0x55da16ed4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.783+0000 7f5b70bc5740 1 --2- 172.21.15.43:0/1993782809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ed7a00 0x55da16ed4cd0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f5b60004620 tx=0x7f5b6002ca90 comp rx=0 tx=0).stop 2024-04-21T08:44:24.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.783+0000 7f5b70bc5740 1 -- 172.21.15.43:0/1993782809 shutdown_connections 2024-04-21T08:44:24.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.783+0000 7f5b70bc5740 1 --2- 172.21.15.43:0/1993782809 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55da16ee7a60 0x55da16ee9e50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.783+0000 7f5b70bc5740 1 --2- 172.21.15.43:0/1993782809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ed7a00 0x55da16ed4cd0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.783+0000 7f5b46ffd640 0 client.17889 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:44:24.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.783+0000 7f5b70bc5740 1 --2- 172.21.15.43:0/1993782809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da16ecee80 0x55da16ed46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.783+0000 7f5b70bc5740 1 --2- 172.21.15.43:0/1993782809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da16ece4d0 0x55da16ed7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.783+0000 7f5b70bc5740 1 -- 172.21.15.43:0/1993782809 >> 172.21.15.43:0/1993782809 conn(0x55da16ec8ba0 msgr2=0x55da16eca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:24.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.783+0000 7f5b70bc5740 1 -- 172.21.15.43:0/1993782809 shutdown_connections 2024-04-21T08:44:24.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.783+0000 7f5b70bc5740 1 -- 172.21.15.43:0/1993782809 wait complete. 2024-04-21T08:44:24.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.783+0000 7f5b70bc5740 20 client.17889 trim_cache size 0 max 16384 2024-04-21T08:44:24.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.796+0000 7f5b70bc5740 1 Processor -- start 2024-04-21T08:44:24.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.796+0000 7f5b70bc5740 1 -- start start 2024-04-21T08:44:24.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.797+0000 7f5b70bc5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da16ece4d0 0x55da16ed2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.797+0000 7f5b70bc5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da16ecee80 0x55da16ed6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.797+0000 7f5b70bc5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ed7a00 0x55da16ed7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.797+0000 7f5b70bc5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681af460 con 0x55da16ed7a00 2024-04-21T08:44:24.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.797+0000 7f5b70bc5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681b0fa0 con 0x55da16ecee80 2024-04-21T08:44:24.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.797+0000 7f5b70bc5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681afb30 con 0x55da16ece4d0 2024-04-21T08:44:24.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.797+0000 7f5b67fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ed7a00 0x55da16ed7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.797+0000 7f5b67fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ed7a00 0x55da16ed7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34570/0 (socket says 172.21.15.43:34570) 2024-04-21T08:44:24.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.797+0000 7f5b67fff640 1 -- 172.21.15.43:0/1104982190 learned_addr learned my addr 172.21.15.43:0/1104982190 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:24.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.797+0000 7f5b677fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da16ece4d0 0x55da16ed2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.797+0000 7f5b66ffd640 1 --2- 172.21.15.43:0/1104982190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da16ecee80 0x55da16ed6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.797+0000 7f5b67fff640 1 -- 172.21.15.43:0/1104982190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da16ece4d0 msgr2=0x55da16ed2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:24.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.797+0000 7f5b67fff640 1 --2- 172.21.15.43:0/1104982190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da16ece4d0 0x55da16ed2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.797+0000 7f5b67fff640 1 -- 172.21.15.43:0/1104982190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da16ecee80 msgr2=0x55da16ed6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:24.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.797+0000 7f5b67fff640 1 --2- 172.21.15.43:0/1104982190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da16ecee80 0x55da16ed6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.797+0000 7f5b67fff640 1 -- 172.21.15.43:0/1104982190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da16ee4340 con 0x55da16ed7a00 2024-04-21T08:44:24.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.797+0000 7f5b66ffd640 1 --2- 172.21.15.43:0/1104982190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da16ecee80 0x55da16ed6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:24.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.797+0000 7f5b67fff640 1 --2- 172.21.15.43:0/1104982190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ed7a00 0x55da16ed7330 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009850 tx=0x7f5b600056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d521f133fb20e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:24.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.797+0000 7f5b5d7fa640 1 -- 172.21.15.43:0/1104982190 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60044040 con 0x55da16ed7a00 2024-04-21T08:44:24.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.797+0000 7f5b5d7fa640 1 -- 172.21.15.43:0/1104982190 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b60034d60 con 0x55da16ed7a00 2024-04-21T08:44:24.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.797+0000 7f5b5d7fa640 1 -- 172.21.15.43:0/1104982190 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6004d070 con 0x55da16ed7a00 2024-04-21T08:44:24.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.798+0000 7f5b70bc5740 1 -- 172.21.15.43:0/1104982190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ed7a00 msgr2=0x55da16ed7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.798+0000 7f5b70bc5740 1 --2- 172.21.15.43:0/1104982190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ed7a00 0x55da16ed7330 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009850 tx=0x7f5b600056b0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.798+0000 7f5b70bc5740 1 -- 172.21.15.43:0/1104982190 shutdown_connections 2024-04-21T08:44:24.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.798+0000 7f5b70bc5740 1 --2- 172.21.15.43:0/1104982190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ed7a00 0x55da16ed7330 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.798+0000 7f5b70bc5740 1 --2- 172.21.15.43:0/1104982190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da16ecee80 0x55da16ed6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.798+0000 7f5b70bc5740 1 --2- 172.21.15.43:0/1104982190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da16ece4d0 0x55da16ed2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.798+0000 7f5b70bc5740 1 -- 172.21.15.43:0/1104982190 >> 172.21.15.43:0/1104982190 conn(0x55da16ec8ba0 msgr2=0x55da16ecfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.798+0000 7f5b70bc5740 1 -- 172.21.15.43:0/1104982190 shutdown_connections 2024-04-21T08:44:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.798+0000 7f5b70bc5740 1 -- 172.21.15.43:0/1104982190 wait complete. 2024-04-21T08:44:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.798+0000 7f5b70bc5740 1 -- start start 2024-04-21T08:44:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.799+0000 7f5b70bc5740 1 Processor -- start 2024-04-21T08:44:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.799+0000 7f5b70bc5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da16ece4d0 0x55da16eea9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.799+0000 7f5b70bc5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ecee80 0x55da16eeaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.799+0000 7f5b70bc5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da16ed7a00 0x55da16eeb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.799+0000 7f5b70bc5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681b08d0 con 0x55da16ecee80 2024-04-21T08:44:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.799+0000 7f5b70bc5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681b0200 con 0x55da16ed7a00 2024-04-21T08:44:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.799+0000 7f5b70bc5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681ad440 con 0x55da16ece4d0 2024-04-21T08:44:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.799+0000 7f5b70bc5740 10 client.0 _ll_register_callbacks cb 0x7f5b60009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.799+0000 7f5b66ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ecee80 0x55da16eeaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.799+0000 7f5b66ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ecee80 0x55da16eeaf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34572/0 (socket says 172.21.15.43:34572) 2024-04-21T08:44:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.799+0000 7f5b677fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da16ece4d0 0x55da16eea9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.799+0000 7f5b66ffd640 1 -- 172.21.15.43:0/1968591422 learned_addr learned my addr 172.21.15.43:0/1968591422 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.799+0000 7f5b67fff640 1 --2- 172.21.15.43:0/1968591422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da16ed7a00 0x55da16eeb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.799+0000 7f5b66ffd640 1 -- 172.21.15.43:0/1968591422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da16ece4d0 msgr2=0x55da16eea9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.799+0000 7f5b66ffd640 1 --2- 172.21.15.43:0/1968591422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da16ece4d0 0x55da16eea9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.799+0000 7f5b66ffd640 1 -- 172.21.15.43:0/1968591422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da16ed7a00 msgr2=0x55da16eeb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.799+0000 7f5b66ffd640 1 --2- 172.21.15.43:0/1968591422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da16ed7a00 0x55da16eeb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.799+0000 7f5b66ffd640 1 -- 172.21.15.43:0/1968591422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da16f4cd70 con 0x55da16ecee80 2024-04-21T08:44:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.799+0000 7f5b66ffd640 1 --2- 172.21.15.43:0/1968591422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ecee80 0x55da16eeaf30 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f5b500994e0 tx=0x7f5b50098ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e826370aa940306 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.799+0000 7f5b46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.800+0000 7f5b46ffd640 1 -- 172.21.15.43:0/1968591422 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b500046c0 con 0x55da16ecee80 2024-04-21T08:44:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.800+0000 7f5b46ffd640 1 -- 172.21.15.43:0/1968591422 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f5b50099180 con 0x55da16ecee80 2024-04-21T08:44:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.800+0000 7f5b70bc5740 1 -- 172.21.15.43:0/1968591422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b60034d60 con 0x55da16ecee80 2024-04-21T08:44:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.800+0000 7f5b46ffd640 1 -- 172.21.15.43:0/1968591422 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5b5009d210 con 0x55da16ecee80 2024-04-21T08:44:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.800+0000 7f5b70bc5740 10 client.17895 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.800+0000 7f5b70bc5740 1 -- 172.21.15.43:0/1968591422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55da16f4e0b0 con 0x55da16ecee80 2024-04-21T08:44:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.800+0000 7f5b46ffd640 1 -- 172.21.15.43:0/1968591422 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b60034d60 con 0x55da16ecee80 2024-04-21T08:44:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.800+0000 7f5b46ffd640 1 -- 172.21.15.43:0/1968591422 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5b500d30b0 con 0x55da16ecee80 2024-04-21T08:44:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.800+0000 7f5b70bc5740 10 client.17895 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.800+0000 7f5b70bc5740 10 client.17895 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:44:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.800+0000 7f5b70bc5740 20 client.17895 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.800+0000 7f5b70bc5740 1 --2- 172.21.15.43:0/1968591422 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55da16f4e830 0x55da16f50c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.800+0000 7f5b70bc5740 4 client.17895 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:44:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.800+0000 7f5b70bc5740 1 -- 172.21.15.43:0/1968591422 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da16e40c10 con 0x55da16f4e830 2024-04-21T08:44:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.800+0000 7f5b677fe640 1 --2- 172.21.15.43:0/1968591422 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55da16f4e830 0x55da16f50c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.801+0000 7f5b677fe640 1 --2- 172.21.15.43:0/1968591422 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55da16f4e830 0x55da16f50c50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bb229b9e4da5e476 server_cookie=c1c6183ae1ac47a9 in_seq=0 out_seq=0 2024-04-21T08:44:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.801+0000 7f5b46ffd640 10 client.17895 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:44:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.801+0000 7f5b46ffd640 1 -- 172.21.15.43:0/1968591422 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da16e40c10 con 0x55da16f4e830 2024-04-21T08:44:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.801+0000 7f5b46ffd640 10 client.17895 handle_command_reply: tid=0 2024-04-21T08:44:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.801+0000 7f5b70bc5740 1 client.17895 shutdown 2024-04-21T08:44:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.801+0000 7f5b70bc5740 1 -- 172.21.15.43:0/1968591422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ecee80 msgr2=0x55da16eeaf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.801+0000 7f5b70bc5740 1 --2- 172.21.15.43:0/1968591422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ecee80 0x55da16eeaf30 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f5b500994e0 tx=0x7f5b50098ee0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.801+0000 7f5b70bc5740 1 -- 172.21.15.43:0/1968591422 shutdown_connections 2024-04-21T08:44:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.801+0000 7f5b70bc5740 1 --2- 172.21.15.43:0/1968591422 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55da16f4e830 0x55da16f50c50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.801+0000 7f5b70bc5740 1 --2- 172.21.15.43:0/1968591422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da16ed7a00 0x55da16eeb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.801+0000 7f5b70bc5740 1 --2- 172.21.15.43:0/1968591422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da16ecee80 0x55da16eeaf30 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.801+0000 7f5b70bc5740 1 --2- 172.21.15.43:0/1968591422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da16ece4d0 0x55da16eea9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.801+0000 7f5b70bc5740 1 -- 172.21.15.43:0/1968591422 >> 172.21.15.43:0/1968591422 conn(0x55da16ec8ba0 msgr2=0x55da16ecf310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.801+0000 7f5b46ffd640 0 client.17895 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:44:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.801+0000 7f5b70bc5740 1 -- 172.21.15.43:0/1968591422 shutdown_connections 2024-04-21T08:44:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.801+0000 7f5b70bc5740 1 -- 172.21.15.43:0/1968591422 wait complete. 2024-04-21T08:44:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.802+0000 7f5b70bc5740 20 client.17895 trim_cache size 0 max 16384 2024-04-21T08:44:24.808 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:44:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.802+0000 7f5b6defb640 1 -- 172.21.15.43:0/4059682758 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5b540a3da0 msgr2=0x7f5b540a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.802+0000 7f5b6defb640 1 --2- 172.21.15.43:0/4059682758 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5b540a3da0 0x7f5b540a6260 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55da16ec9c10 tx=0x7f5b58009500 comp rx=0 tx=0).stop 2024-04-21T08:44:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.802+0000 7f5b6defb640 1 -- 172.21.15.43:0/4059682758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b68106160 msgr2=0x7f5b68198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.802+0000 7f5b6defb640 1 --2- 172.21.15.43:0/4059682758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b68106160 0x7f5b68198c50 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f5b5000e9c0 tx=0x7f5b5000ee80 comp rx=0 tx=0).stop 2024-04-21T08:44:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.802+0000 7f5b6defb640 1 -- 172.21.15.43:0/4059682758 shutdown_connections 2024-04-21T08:44:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.802+0000 7f5b6defb640 1 --2- 172.21.15.43:0/4059682758 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5b540a3da0 0x7f5b540a6260 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.802+0000 7f5b6defb640 1 --2- 172.21.15.43:0/4059682758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b6819cfd0 0x7f5b6819d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.802+0000 7f5b6defb640 1 --2- 172.21.15.43:0/4059682758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b68106160 0x7f5b68198c50 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.802+0000 7f5b6defb640 1 --2- 172.21.15.43:0/4059682758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b68101090 0x7f5b68198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.802+0000 7f5b6defb640 1 -- 172.21.15.43:0/4059682758 >> 172.21.15.43:0/4059682758 conn(0x7f5b68077f50 msgr2=0x7f5b680693c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.802+0000 7f5b6defb640 1 -- 172.21.15.43:0/4059682758 shutdown_connections 2024-04-21T08:44:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.802+0000 7f5b6defb640 1 -- 172.21.15.43:0/4059682758 wait complete. 2024-04-21T08:44:24.812 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:44:24.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.917+0000 7f6eaa37e640 1 -- 172.21.15.43:0/3772428452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ea4106170 msgr2=0x7f6ea410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.917+0000 7f6eaa37e640 1 --2- 172.21.15.43:0/3772428452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ea4106170 0x7f6ea410dd40 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009ca0 tx=0x7f6e9c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.917+0000 7f6eaa37e640 1 -- 172.21.15.43:0/3772428452 shutdown_connections 2024-04-21T08:44:24.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.917+0000 7f6eaa37e640 1 --2- 172.21.15.43:0/3772428452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ea4106170 0x7f6ea410dd40 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.917+0000 7f6eaa37e640 1 --2- 172.21.15.43:0/3772428452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ea4101a50 0x7f6ea4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.917+0000 7f6eaa37e640 1 --2- 172.21.15.43:0/3772428452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ea41010a0 0x7f6ea4101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.917+0000 7f6eaa37e640 1 -- 172.21.15.43:0/3772428452 >> 172.21.15.43:0/3772428452 conn(0x7f6ea4077eb0 msgr2=0x7f6ea40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:24.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.917+0000 7f6eaa37e640 1 -- 172.21.15.43:0/3772428452 shutdown_connections 2024-04-21T08:44:24.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.917+0000 7f6eaa37e640 1 -- 172.21.15.43:0/3772428452 wait complete. 2024-04-21T08:44:24.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.917+0000 7f6eaa37e640 1 Processor -- start 2024-04-21T08:44:24.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.918+0000 7f6eaa37e640 1 -- start start 2024-04-21T08:44:24.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.918+0000 7f6eaa37e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ea41010a0 0x7f6ea4198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.918+0000 7f6eaa37e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ea4106170 0x7f6ea4198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.918+0000 7f6eaa37e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ea419d000 0x7f6ea419d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.918+0000 7f6eaa37e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea41110e0 con 0x7f6ea419d000 2024-04-21T08:44:24.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.918+0000 7f6eaa37e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea4110f60 con 0x7f6ea41010a0 2024-04-21T08:44:24.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.918+0000 7f6eaa37e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea4111260 con 0x7f6ea4106170 2024-04-21T08:44:24.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.918+0000 7f6ea891b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ea419d000 0x7f6ea419d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.918+0000 7f6ea891b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ea419d000 0x7f6ea419d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34580/0 (socket says 172.21.15.43:34580) 2024-04-21T08:44:24.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.918+0000 7f6ea891b640 1 -- 172.21.15.43:0/2953041740 learned_addr learned my addr 172.21.15.43:0/2953041740 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:24.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.918+0000 7f6ea3fff640 1 --2- 172.21.15.43:0/2953041740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ea41010a0 0x7f6ea4198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.918+0000 7f6ea37fe640 1 --2- 172.21.15.43:0/2953041740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ea4106170 0x7f6ea4198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.918+0000 7f6ea891b640 1 -- 172.21.15.43:0/2953041740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ea4106170 msgr2=0x7f6ea4198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.918+0000 7f6ea891b640 1 --2- 172.21.15.43:0/2953041740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ea4106170 0x7f6ea4198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.918+0000 7f6ea891b640 1 -- 172.21.15.43:0/2953041740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ea41010a0 msgr2=0x7f6ea4198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.918+0000 7f6ea891b640 1 --2- 172.21.15.43:0/2953041740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ea41010a0 0x7f6ea4198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.918+0000 7f6ea891b640 1 -- 172.21.15.43:0/2953041740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea419daf0 con 0x7f6ea419d000 2024-04-21T08:44:24.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.919+0000 7f6ea891b640 1 --2- 172.21.15.43:0/2953041740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ea419d000 0x7f6ea419d4b0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c005e30 tx=0x7f6e9c034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:24.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.919+0000 7f6ea17fa640 1 -- 172.21.15.43:0/2953041740 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c039040 con 0x7f6ea419d000 2024-04-21T08:44:24.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.919+0000 7f6ea17fa640 1 -- 172.21.15.43:0/2953041740 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e9c044070 con 0x7f6ea419d000 2024-04-21T08:44:24.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.919+0000 7f6eaa37e640 1 -- 172.21.15.43:0/2953041740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea419dd80 con 0x7f6ea419d000 2024-04-21T08:44:24.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.919+0000 7f6ea17fa640 1 -- 172.21.15.43:0/2953041740 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c003e60 con 0x7f6ea419d000 2024-04-21T08:44:24.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.919+0000 7f6eaa37e640 1 -- 172.21.15.43:0/2953041740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea41a55a0 con 0x7f6ea419d000 2024-04-21T08:44:24.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.921+0000 7f6eacf6e740 1 Processor -- start 2024-04-21T08:44:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.921+0000 7f6ea17fa640 1 -- 172.21.15.43:0/2953041740 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6e9c0324c0 con 0x7f6ea419d000 2024-04-21T08:44:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.921+0000 7f6eacf6e740 1 -- start start 2024-04-21T08:44:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.921+0000 7f6eacf6e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559230bac330 0x559230bac710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.921+0000 7f6eacf6e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bacce0 0x559230bb5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.921+0000 7f6eacf6e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559230bb5e40 0x559230bb62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.921+0000 7f6eacf6e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559230b111f0 con 0x559230bacce0 2024-04-21T08:44:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.921+0000 7f6eacf6e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559230aae4b0 con 0x559230bb5e40 2024-04-21T08:44:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.921+0000 7f6eacf6e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559230b0d1b0 con 0x559230bac330 2024-04-21T08:44:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.922+0000 7f6ea3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bacce0 0x559230bb5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.922+0000 7f6ea3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bacce0 0x559230bb5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34596/0 (socket says 172.21.15.43:34596) 2024-04-21T08:44:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.922+0000 7f6ea3fff640 1 -- 172.21.15.43:0/1814752305 learned_addr learned my addr 172.21.15.43:0/1814752305 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.922+0000 7f6ea37fe640 1 --2- 172.21.15.43:0/1814752305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559230bb5e40 0x559230bb62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.922+0000 7f6ea37fe640 1 --2- 172.21.15.43:0/1814752305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559230bac330 0x559230bac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.922+0000 7f6ea3fff640 1 -- 172.21.15.43:0/1814752305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559230bac330 msgr2=0x559230bac710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.922+0000 7f6ea3fff640 1 --2- 172.21.15.43:0/1814752305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559230bac330 0x559230bac710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.922+0000 7f6ea3fff640 1 -- 172.21.15.43:0/1814752305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559230bb5e40 msgr2=0x559230bb62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.922+0000 7f6ea3fff640 1 --2- 172.21.15.43:0/1814752305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559230bb5e40 0x559230bb62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.922+0000 7f6ea3fff640 1 -- 172.21.15.43:0/1814752305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559230aa01b0 con 0x559230bacce0 2024-04-21T08:44:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.922+0000 7f6ea37fe640 1 --2- 172.21.15.43:0/1814752305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559230bac330 0x559230bac710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.922+0000 7f6ea17fa640 1 --2- 172.21.15.43:0/2953041740 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6e8c0a3da0 0x7f6e8c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.922+0000 7f6ea3fff640 1 --2- 172.21.15.43:0/1814752305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bacce0 0x559230bb5820 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400bcf0 tx=0x7f6e9400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75545bd81563d508 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.922+0000 7f6ea37fe640 1 --2- 172.21.15.43:0/2953041740 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6e8c0a3da0 0x7f6e8c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.922+0000 7f6ea17fa640 1 -- 172.21.15.43:0/2953041740 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f6e9c0f2dd0 con 0x7f6ea419d000 2024-04-21T08:44:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.922+0000 7f6e8a7fc640 1 -- 172.21.15.43:0/1814752305 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e94015070 con 0x559230bacce0 2024-04-21T08:44:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.922+0000 7f6e8a7fc640 1 -- 172.21.15.43:0/1814752305 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e940118f0 con 0x559230bacce0 2024-04-21T08:44:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.922+0000 7f6e8a7fc640 1 -- 172.21.15.43:0/1814752305 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e940107f0 con 0x559230bacce0 2024-04-21T08:44:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.923+0000 7f6eacf6e740 1 -- 172.21.15.43:0/1814752305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bacce0 msgr2=0x559230bb5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.923+0000 7f6eacf6e740 1 --2- 172.21.15.43:0/1814752305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bacce0 0x559230bb5820 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400bcf0 tx=0x7f6e9400f730 comp rx=0 tx=0).stop 2024-04-21T08:44:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.923+0000 7f6ea37fe640 1 --2- 172.21.15.43:0/2953041740 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6e8c0a3da0 0x7f6e8c0a6260 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x559230bac020 tx=0x7f6e98004730 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.923+0000 7f6eacf6e740 1 -- 172.21.15.43:0/1814752305 shutdown_connections 2024-04-21T08:44:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.923+0000 7f6eacf6e740 1 --2- 172.21.15.43:0/1814752305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559230bb5e40 0x559230bb62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.923+0000 7f6eacf6e740 1 --2- 172.21.15.43:0/1814752305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bacce0 0x559230bb5820 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.923+0000 7f6eacf6e740 1 --2- 172.21.15.43:0/1814752305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559230bac330 0x559230bac710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.923+0000 7f6eacf6e740 1 -- 172.21.15.43:0/1814752305 >> 172.21.15.43:0/1814752305 conn(0x559230ba6ba0 msgr2=0x559230ba6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.923+0000 7f6eacf6e740 1 -- 172.21.15.43:0/1814752305 shutdown_connections 2024-04-21T08:44:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.923+0000 7f6eacf6e740 1 -- 172.21.15.43:0/1814752305 wait complete. 2024-04-21T08:44:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.923+0000 7f6eacf6e740 1 -- start start 2024-04-21T08:44:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.924+0000 7f6eacf6e740 1 Processor -- start 2024-04-21T08:44:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.924+0000 7f6eacf6e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559230bac330 0x559230bbc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.924+0000 7f6eacf6e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bacce0 0x559230bb95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.924+0000 7f6eacf6e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559230bb5e40 0x559230bb9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.924+0000 7f6eacf6e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559230bbcdc0 con 0x559230bacce0 2024-04-21T08:44:24.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.924+0000 7f6eacf6e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559230aca500 con 0x559230bac330 2024-04-21T08:44:24.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.924+0000 7f6eacf6e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559230bbcf40 con 0x559230bb5e40 2024-04-21T08:44:24.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.924+0000 7f6eacf6e740 10 client.0 _ll_register_callbacks cb 0x559230a98b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:24.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.924+0000 7f6ea37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bacce0 0x559230bb95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.924+0000 7f6ea37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bacce0 0x559230bb95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34598/0 (socket says 172.21.15.43:34598) 2024-04-21T08:44:24.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.924+0000 7f6ea37fe640 1 -- 172.21.15.43:0/3414654220 learned_addr learned my addr 172.21.15.43:0/3414654220 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:24.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.924+0000 7f6ea3fff640 1 --2- 172.21.15.43:0/3414654220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559230bac330 0x559230bbc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.924+0000 7f6ea891b640 1 --2- 172.21.15.43:0/3414654220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559230bb5e40 0x559230bb9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.924+0000 7f6ea37fe640 1 -- 172.21.15.43:0/3414654220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559230bb5e40 msgr2=0x559230bb9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:24.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.924+0000 7f6ea37fe640 1 --2- 172.21.15.43:0/3414654220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559230bb5e40 0x559230bb9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.924+0000 7f6ea37fe640 1 -- 172.21.15.43:0/3414654220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559230bac330 msgr2=0x559230bbc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:24.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.924+0000 7f6ea37fe640 1 --2- 172.21.15.43:0/3414654220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559230bac330 0x559230bbc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.924+0000 7f6ea37fe640 1 -- 172.21.15.43:0/3414654220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592308233f0 con 0x559230bacce0 2024-04-21T08:44:24.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.924+0000 7f6ea3fff640 1 --2- 172.21.15.43:0/3414654220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559230bac330 0x559230bbc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:24.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.925+0000 7f6ea37fe640 1 --2- 172.21.15.43:0/3414654220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bacce0 0x559230bb95a0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f6e98011b90 tx=0x7f6e9800c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f8bc368036a685 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:24.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.925+0000 7f6e837fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:24.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.925+0000 7f6e837fe640 1 -- 172.21.15.43:0/3414654220 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98010d60 con 0x559230bacce0 2024-04-21T08:44:24.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.925+0000 7f6e837fe640 1 -- 172.21.15.43:0/3414654220 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f6e98027400 con 0x559230bacce0 2024-04-21T08:44:24.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.925+0000 7f6eacf6e740 1 -- 172.21.15.43:0/3414654220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e940118f0 con 0x559230bacce0 2024-04-21T08:44:24.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.925+0000 7f6e837fe640 1 -- 172.21.15.43:0/3414654220 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6e98027600 con 0x559230bacce0 2024-04-21T08:44:24.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.925+0000 7f6eacf6e740 10 client.17907 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:24.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.925+0000 7f6eacf6e740 1 -- 172.21.15.43:0/3414654220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5592308ed7d0 con 0x559230bacce0 2024-04-21T08:44:24.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.925+0000 7f6e837fe640 1 -- 172.21.15.43:0/3414654220 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e940118f0 con 0x559230bacce0 2024-04-21T08:44:24.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.925+0000 7f6e837fe640 1 -- 172.21.15.43:0/3414654220 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6e9801f730 con 0x559230bacce0 2024-04-21T08:44:24.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.925+0000 7f6eacf6e740 10 client.17907 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:24.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.925+0000 7f6eacf6e740 10 client.17907 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:44:24.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.925+0000 7f6eacf6e740 20 client.17907 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:24.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.925+0000 7f6eacf6e740 1 --2- 172.21.15.43:0/3414654220 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x559230bca090 0x559230bcc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.925+0000 7f6eacf6e740 4 client.17907 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:24.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.925+0000 7f6eacf6e740 1 -- 172.21.15.43:0/3414654220 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559230ab0930 con 0x559230bca090 2024-04-21T08:44:24.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.926+0000 7f6ea3fff640 1 --2- 172.21.15.43:0/3414654220 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x559230bca090 0x559230bcc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.926+0000 7f6ea3fff640 1 --2- 172.21.15.43:0/3414654220 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x559230bca090 0x559230bcc4e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5411cc23e89ccb3d server_cookie=5f8a787679acde5a in_seq=0 out_seq=0 2024-04-21T08:44:24.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.926+0000 7f6e837fe640 10 client.17907 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:44:24.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.927+0000 7f6e837fe640 1 -- 172.21.15.43:0/3414654220 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559230ab0930 con 0x559230bca090 2024-04-21T08:44:24.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.927+0000 7f6e837fe640 10 client.17907 handle_command_reply: tid=0 2024-04-21T08:44:24.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.927+0000 7f6eacf6e740 1 client.17907 shutdown 2024-04-21T08:44:24.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.927+0000 7f6eacf6e740 1 -- 172.21.15.43:0/3414654220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bacce0 msgr2=0x559230bb95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.927+0000 7f6eacf6e740 1 --2- 172.21.15.43:0/3414654220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bacce0 0x559230bb95a0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f6e98011b90 tx=0x7f6e9800c3b0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.927+0000 7f6eacf6e740 1 -- 172.21.15.43:0/3414654220 shutdown_connections 2024-04-21T08:44:24.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.927+0000 7f6eacf6e740 1 --2- 172.21.15.43:0/3414654220 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x559230bca090 0x559230bcc4e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.927+0000 7f6eacf6e740 1 --2- 172.21.15.43:0/3414654220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559230bb5e40 0x559230bb9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.927+0000 7f6e837fe640 0 client.17907 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:44:24.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.927+0000 7f6eacf6e740 1 --2- 172.21.15.43:0/3414654220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bacce0 0x559230bb95a0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.927+0000 7f6eacf6e740 1 --2- 172.21.15.43:0/3414654220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559230bac330 0x559230bbc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.927+0000 7f6eacf6e740 1 -- 172.21.15.43:0/3414654220 >> 172.21.15.43:0/3414654220 conn(0x559230ba6ba0 msgr2=0x559230ba88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:24.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.927+0000 7f6eacf6e740 1 -- 172.21.15.43:0/3414654220 shutdown_connections 2024-04-21T08:44:24.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.927+0000 7f6eacf6e740 1 -- 172.21.15.43:0/3414654220 wait complete. 2024-04-21T08:44:24.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.927+0000 7f6eacf6e740 20 client.17907 trim_cache size 0 max 16384 2024-04-21T08:44:24.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.941+0000 7f6eacf6e740 1 Processor -- start 2024-04-21T08:44:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.941+0000 7f6eacf6e740 1 -- start start 2024-04-21T08:44:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.941+0000 7f6eacf6e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559230bac330 0x559230bc5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.941+0000 7f6eacf6e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bacce0 0x559230bbbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.941+0000 7f6eacf6e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559230bb5e40 0x559230bbc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.941+0000 7f6eacf6e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea41ad110 con 0x559230bacce0 2024-04-21T08:44:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.941+0000 7f6eacf6e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea41aec50 con 0x559230bac330 2024-04-21T08:44:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.941+0000 7f6eacf6e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea41ad7e0 con 0x559230bb5e40 2024-04-21T08:44:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.941+0000 7f6ea37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bacce0 0x559230bbbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.941+0000 7f6ea37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bacce0 0x559230bbbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34600/0 (socket says 172.21.15.43:34600) 2024-04-21T08:44:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.941+0000 7f6ea37fe640 1 -- 172.21.15.43:0/1666600487 learned_addr learned my addr 172.21.15.43:0/1666600487 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.941+0000 7f6ea3fff640 1 --2- 172.21.15.43:0/1666600487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559230bac330 0x559230bc5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.941+0000 7f6ea891b640 1 --2- 172.21.15.43:0/1666600487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559230bb5e40 0x559230bbc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.941+0000 7f6ea37fe640 1 -- 172.21.15.43:0/1666600487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559230bb5e40 msgr2=0x559230bbc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:24.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.941+0000 7f6ea37fe640 1 --2- 172.21.15.43:0/1666600487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559230bb5e40 0x559230bbc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.941+0000 7f6ea37fe640 1 -- 172.21.15.43:0/1666600487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559230bac330 msgr2=0x559230bc5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:24.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.941+0000 7f6ea37fe640 1 --2- 172.21.15.43:0/1666600487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559230bac330 0x559230bc5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.941+0000 7f6ea37fe640 1 -- 172.21.15.43:0/1666600487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559230bb7af0 con 0x559230bacce0 2024-04-21T08:44:24.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.942+0000 7f6ea37fe640 1 --2- 172.21.15.43:0/1666600487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bacce0 0x559230bbbcd0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f6e9801fb50 tx=0x7f6e98011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55ba074ef9b9bc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:24.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.942+0000 7f6e89ffb640 1 -- 172.21.15.43:0/1666600487 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98038070 con 0x559230bacce0 2024-04-21T08:44:24.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.942+0000 7f6e89ffb640 1 -- 172.21.15.43:0/1666600487 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e980107a0 con 0x559230bacce0 2024-04-21T08:44:24.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.942+0000 7f6e89ffb640 1 -- 172.21.15.43:0/1666600487 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9801f880 con 0x559230bacce0 2024-04-21T08:44:24.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.942+0000 7f6eacf6e740 1 -- 172.21.15.43:0/1666600487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bacce0 msgr2=0x559230bbbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.942+0000 7f6eacf6e740 1 --2- 172.21.15.43:0/1666600487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bacce0 0x559230bbbcd0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f6e9801fb50 tx=0x7f6e98011750 comp rx=0 tx=0).stop 2024-04-21T08:44:24.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.943+0000 7f6eacf6e740 1 -- 172.21.15.43:0/1666600487 shutdown_connections 2024-04-21T08:44:24.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.943+0000 7f6eacf6e740 1 --2- 172.21.15.43:0/1666600487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559230bb5e40 0x559230bbc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.943+0000 7f6eacf6e740 1 --2- 172.21.15.43:0/1666600487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bacce0 0x559230bbbcd0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.943+0000 7f6eacf6e740 1 --2- 172.21.15.43:0/1666600487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559230bac330 0x559230bc5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.943+0000 7f6eacf6e740 1 -- 172.21.15.43:0/1666600487 >> 172.21.15.43:0/1666600487 conn(0x559230ba6ba0 msgr2=0x559230bb5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:24.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.943+0000 7f6eacf6e740 1 -- 172.21.15.43:0/1666600487 shutdown_connections 2024-04-21T08:44:24.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.943+0000 7f6eacf6e740 1 -- 172.21.15.43:0/1666600487 wait complete. 2024-04-21T08:44:24.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.943+0000 7f6eacf6e740 1 -- start start 2024-04-21T08:44:24.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.943+0000 7f6eacf6e740 1 Processor -- start 2024-04-21T08:44:24.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.943+0000 7f6eacf6e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559230bac330 0x559230bc2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.943+0000 7f6eacf6e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559230bacce0 0x559230bc2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.943+0000 7f6eacf6e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bb5e40 0x559230bbad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.943+0000 7f6eacf6e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea41ae580 con 0x559230bb5e40 2024-04-21T08:44:24.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.943+0000 7f6eacf6e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea41adeb0 con 0x559230bac330 2024-04-21T08:44:24.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.943+0000 7f6eacf6e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea41ab0f0 con 0x559230bacce0 2024-04-21T08:44:24.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.943+0000 7f6eacf6e740 10 client.0 _ll_register_callbacks cb 0x7f6e98011840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:24.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.944+0000 7f6ea891b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bb5e40 0x559230bbad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.944+0000 7f6ea891b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bb5e40 0x559230bbad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34612/0 (socket says 172.21.15.43:34612) 2024-04-21T08:44:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.944+0000 7f6ea891b640 1 -- 172.21.15.43:0/417533022 learned_addr learned my addr 172.21.15.43:0/417533022 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.944+0000 7f6ea3fff640 1 --2- 172.21.15.43:0/417533022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559230bac330 0x559230bc2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.944+0000 7f6ea37fe640 1 --2- 172.21.15.43:0/417533022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559230bacce0 0x559230bc2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.944+0000 7f6ea891b640 1 -- 172.21.15.43:0/417533022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559230bacce0 msgr2=0x559230bc2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.944+0000 7f6ea891b640 1 --2- 172.21.15.43:0/417533022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559230bacce0 0x559230bc2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.944+0000 7f6ea891b640 1 -- 172.21.15.43:0/417533022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559230bac330 msgr2=0x559230bc2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.944+0000 7f6ea891b640 1 --2- 172.21.15.43:0/417533022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559230bac330 0x559230bc2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.944+0000 7f6ea891b640 1 -- 172.21.15.43:0/417533022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559230bbb5c0 con 0x559230bb5e40 2024-04-21T08:44:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.944+0000 7f6ea3fff640 1 --2- 172.21.15.43:0/417533022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559230bac330 0x559230bc2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.944+0000 7f6ea891b640 1 --2- 172.21.15.43:0/417533022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bb5e40 0x559230bbad90 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0be7b0 tx=0x7f6e9c0be6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b97d64bcc8e4aee server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.944+0000 7f6e837fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:24.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.944+0000 7f6e837fe640 1 -- 172.21.15.43:0/417533022 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c0beab0 con 0x559230bb5e40 2024-04-21T08:44:24.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.944+0000 7f6e837fe640 1 -- 172.21.15.43:0/417533022 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f6e9c0f03d0 con 0x559230bb5e40 2024-04-21T08:44:24.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.944+0000 7f6eacf6e740 1 -- 172.21.15.43:0/417533022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e980107a0 con 0x559230bb5e40 2024-04-21T08:44:24.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.944+0000 7f6e837fe640 1 -- 172.21.15.43:0/417533022 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6e9c0c2e60 con 0x559230bb5e40 2024-04-21T08:44:24.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.944+0000 7f6eacf6e740 10 client.17913 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:24.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.944+0000 7f6eacf6e740 1 -- 172.21.15.43:0/417533022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559230c30300 con 0x559230bb5e40 2024-04-21T08:44:24.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.944+0000 7f6e837fe640 1 -- 172.21.15.43:0/417533022 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e980107a0 con 0x559230bb5e40 2024-04-21T08:44:24.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.945+0000 7f6e837fe640 1 -- 172.21.15.43:0/417533022 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6e9c03c030 con 0x559230bb5e40 2024-04-21T08:44:24.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.945+0000 7f6eacf6e740 10 client.17913 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:24.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.945+0000 7f6eacf6e740 10 client.17913 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:44:24.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.945+0000 7f6eacf6e740 20 client.17913 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:24.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.945+0000 7f6eacf6e740 1 --2- 172.21.15.43:0/417533022 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x559230c30aa0 0x559230c32ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:24.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.945+0000 7f6eacf6e740 4 client.17913 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:44:24.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.945+0000 7f6eacf6e740 1 -- 172.21.15.43:0/417533022 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559230b1ec10 con 0x559230c30aa0 2024-04-21T08:44:24.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.945+0000 7f6ea3fff640 1 --2- 172.21.15.43:0/417533022 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x559230c30aa0 0x559230c32ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:24.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.945+0000 7f6e837fe640 10 client.17913 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:44:24.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.945+0000 7f6ea3fff640 1 --2- 172.21.15.43:0/417533022 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x559230c30aa0 0x559230c32ec0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=117744da14b5c895 server_cookie=9ba58776b22d7899 in_seq=0 out_seq=0 2024-04-21T08:44:24.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.945+0000 7f6e837fe640 1 -- 172.21.15.43:0/417533022 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559230b1ec10 con 0x559230c30aa0 2024-04-21T08:44:24.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.945+0000 7f6e837fe640 10 client.17913 handle_command_reply: tid=0 2024-04-21T08:44:24.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.945+0000 7f6eacf6e740 1 client.17913 shutdown 2024-04-21T08:44:24.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.946+0000 7f6eacf6e740 1 -- 172.21.15.43:0/417533022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bb5e40 msgr2=0x559230bbad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.946+0000 7f6eacf6e740 1 --2- 172.21.15.43:0/417533022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bb5e40 0x559230bbad90 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0be7b0 tx=0x7f6e9c0be6b0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.946+0000 7f6eacf6e740 1 -- 172.21.15.43:0/417533022 shutdown_connections 2024-04-21T08:44:24.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.946+0000 7f6eacf6e740 1 --2- 172.21.15.43:0/417533022 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x559230c30aa0 0x559230c32ec0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.946+0000 7f6eacf6e740 1 --2- 172.21.15.43:0/417533022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559230bb5e40 0x559230bbad90 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.946+0000 7f6e837fe640 0 client.17913 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:44:24.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.946+0000 7f6eacf6e740 1 --2- 172.21.15.43:0/417533022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559230bacce0 0x559230bc2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.946+0000 7f6eacf6e740 1 --2- 172.21.15.43:0/417533022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559230bac330 0x559230bc2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.946+0000 7f6eacf6e740 1 -- 172.21.15.43:0/417533022 >> 172.21.15.43:0/417533022 conn(0x559230ba6ba0 msgr2=0x559230ba5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:24.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.946+0000 7f6eacf6e740 1 -- 172.21.15.43:0/417533022 shutdown_connections 2024-04-21T08:44:24.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.946+0000 7f6eacf6e740 1 -- 172.21.15.43:0/417533022 wait complete. 2024-04-21T08:44:24.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.946+0000 7f6eacf6e740 20 client.17913 trim_cache size 0 max 16384 2024-04-21T08:44:24.953 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:44:24.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.946+0000 7f6eaa37e640 1 -- 172.21.15.43:0/2953041740 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6e8c0a3da0 msgr2=0x7f6e8c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.946+0000 7f6eaa37e640 1 --2- 172.21.15.43:0/2953041740 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6e8c0a3da0 0x7f6e8c0a6260 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x559230bac020 tx=0x7f6e98004730 comp rx=0 tx=0).stop 2024-04-21T08:44:24.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.946+0000 7f6eaa37e640 1 -- 172.21.15.43:0/2953041740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ea419d000 msgr2=0x7f6ea419d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:24.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.946+0000 7f6eaa37e640 1 --2- 172.21.15.43:0/2953041740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ea419d000 0x7f6ea419d4b0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c005e30 tx=0x7f6e9c034760 comp rx=0 tx=0).stop 2024-04-21T08:44:24.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.946+0000 7f6eaa37e640 1 -- 172.21.15.43:0/2953041740 shutdown_connections 2024-04-21T08:44:24.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.946+0000 7f6eaa37e640 1 --2- 172.21.15.43:0/2953041740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ea419d000 0x7f6ea419d4b0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.946+0000 7f6eaa37e640 1 --2- 172.21.15.43:0/2953041740 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6e8c0a3da0 0x7f6e8c0a6260 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.946+0000 7f6eaa37e640 1 --2- 172.21.15.43:0/2953041740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ea4106170 0x7f6ea4198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.946+0000 7f6eaa37e640 1 --2- 172.21.15.43:0/2953041740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ea41010a0 0x7f6ea4198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:24.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.946+0000 7f6eaa37e640 1 -- 172.21.15.43:0/2953041740 >> 172.21.15.43:0/2953041740 conn(0x7f6ea4077eb0 msgr2=0x7f6ea4104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:24.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.946+0000 7f6eaa37e640 1 -- 172.21.15.43:0/2953041740 shutdown_connections 2024-04-21T08:44:24.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:24.947+0000 7f6eaa37e640 1 -- 172.21.15.43:0/2953041740 wait complete. 2024-04-21T08:44:24.957 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:44:25.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.063+0000 7fb52f5a4640 1 -- 172.21.15.43:0/488176284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb528103af0 msgr2=0x7fb52810bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.063+0000 7fb52f5a4640 1 --2- 172.21.15.43:0/488176284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb528103af0 0x7fb52810bfd0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fb518009900 tx=0x7fb51802bd60 comp rx=0 tx=0).stop 2024-04-21T08:44:25.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.063+0000 7fb52f5a4640 1 -- 172.21.15.43:0/488176284 shutdown_connections 2024-04-21T08:44:25.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.063+0000 7fb52f5a4640 1 --2- 172.21.15.43:0/488176284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb52810c510 0x7fb52810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.063+0000 7fb52f5a4640 1 --2- 172.21.15.43:0/488176284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb528103af0 0x7fb52810bfd0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.063+0000 7fb52f5a4640 1 --2- 172.21.15.43:0/488176284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5281031d0 0x7fb5281035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.063+0000 7fb52f5a4640 1 -- 172.21.15.43:0/488176284 >> 172.21.15.43:0/488176284 conn(0x7fb528077f50 msgr2=0x7fb528078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:25.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.063+0000 7fb52f5a4640 1 -- 172.21.15.43:0/488176284 shutdown_connections 2024-04-21T08:44:25.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.063+0000 7fb52f5a4640 1 -- 172.21.15.43:0/488176284 wait complete. 2024-04-21T08:44:25.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.064+0000 7fb52f5a4640 1 Processor -- start 2024-04-21T08:44:25.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.064+0000 7fb52f5a4640 1 -- start start 2024-04-21T08:44:25.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.064+0000 7fb52f5a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5281031d0 0x7fb528100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.064+0000 7fb52f5a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb528103af0 0x7fb528101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.064+0000 7fb52f5a4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb52810c510 0x7fb5281018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.064+0000 7fb52f5a4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb528112180 con 0x7fb528103af0 2024-04-21T08:44:25.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.064+0000 7fb52f5a4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb528112000 con 0x7fb5281031d0 2024-04-21T08:44:25.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.064+0000 7fb52f5a4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb528112300 con 0x7fb52810c510 2024-04-21T08:44:25.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.064+0000 7fb52cb3f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb528103af0 0x7fb528101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.064+0000 7fb52cb3f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb528103af0 0x7fb528101360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34632/0 (socket says 172.21.15.43:34632) 2024-04-21T08:44:25.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.064+0000 7fb52cb3f640 1 -- 172.21.15.43:0/3348670356 learned_addr learned my addr 172.21.15.43:0/3348670356 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:25.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.064+0000 7fb52d340640 1 --2- 172.21.15.43:0/3348670356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5281031d0 0x7fb528100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.064+0000 7fb52db41640 1 --2- 172.21.15.43:0/3348670356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb52810c510 0x7fb5281018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.064+0000 7fb52cb3f640 1 -- 172.21.15.43:0/3348670356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb52810c510 msgr2=0x7fb5281018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.064+0000 7fb52cb3f640 1 --2- 172.21.15.43:0/3348670356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb52810c510 0x7fb5281018a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.064+0000 7fb52cb3f640 1 -- 172.21.15.43:0/3348670356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5281031d0 msgr2=0x7fb528100e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.064+0000 7fb52cb3f640 1 --2- 172.21.15.43:0/3348670356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5281031d0 0x7fb528100e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.064+0000 7fb52cb3f640 1 -- 172.21.15.43:0/3348670356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5281a6c40 con 0x7fb528103af0 2024-04-21T08:44:25.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.064+0000 7fb52cb3f640 1 --2- 172.21.15.43:0/3348670356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb528103af0 0x7fb528101360 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fb518005940 tx=0x7fb51802c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:25.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.065+0000 7fb51e7fc640 1 -- 172.21.15.43:0/3348670356 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb518043020 con 0x7fb528103af0 2024-04-21T08:44:25.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.065+0000 7fb52f5a4640 1 -- 172.21.15.43:0/3348670356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5281a6ed0 con 0x7fb528103af0 2024-04-21T08:44:25.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.065+0000 7fb51e7fc640 1 -- 172.21.15.43:0/3348670356 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb51802dbb0 con 0x7fb528103af0 2024-04-21T08:44:25.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.065+0000 7fb51e7fc640 1 -- 172.21.15.43:0/3348670356 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb518003a10 con 0x7fb528103af0 2024-04-21T08:44:25.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.065+0000 7fb52f5a4640 1 -- 172.21.15.43:0/3348670356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5281a7390 con 0x7fb528103af0 2024-04-21T08:44:25.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.067+0000 7fb51e7fc640 1 -- 172.21.15.43:0/3348670356 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb518046020 con 0x7fb528103af0 2024-04-21T08:44:25.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.067+0000 7fb532196740 1 Processor -- start 2024-04-21T08:44:25.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.067+0000 7fb532196740 1 -- start start 2024-04-21T08:44:25.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.068+0000 7fb51e7fc640 1 --2- 172.21.15.43:0/3348670356 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb5140a3d20 0x7fb5140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.068+0000 7fb532196740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7e63a4d0 0x55dc7e63a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.068+0000 7fb532196740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e63ae80 0x55dc7e643430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.068+0000 7fb532196740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc7e643a00 0x55dc7e645e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.068+0000 7fb532196740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dc7e59f1f0 con 0x55dc7e63ae80 2024-04-21T08:44:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.068+0000 7fb532196740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dc7e53c4b0 con 0x55dc7e63a4d0 2024-04-21T08:44:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.068+0000 7fb532196740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dc7e59b1b0 con 0x55dc7e643a00 2024-04-21T08:44:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.068+0000 7fb52db41640 1 --2- 172.21.15.43:0/3348670356 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb5140a3d20 0x7fb5140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.068+0000 7fb52cb3f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e63ae80 0x55dc7e643430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.068+0000 7fb52cb3f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e63ae80 0x55dc7e643430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34634/0 (socket says 172.21.15.43:34634) 2024-04-21T08:44:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.068+0000 7fb52cb3f640 1 -- 172.21.15.43:0/3443388253 learned_addr learned my addr 172.21.15.43:0/3443388253 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.068+0000 7fb52d340640 1 --2- 172.21.15.43:0/3443388253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7e63a4d0 0x55dc7e63a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.068+0000 7fb51e7fc640 1 -- 172.21.15.43:0/3348670356 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fb5180f2b60 con 0x7fb528103af0 2024-04-21T08:44:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.068+0000 7fb52db41640 1 --2- 172.21.15.43:0/3443388253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc7e643a00 0x55dc7e645e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.068+0000 7fb52cb3f640 1 -- 172.21.15.43:0/3443388253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc7e643a00 msgr2=0x55dc7e645e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.068+0000 7fb52cb3f640 1 --2- 172.21.15.43:0/3443388253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc7e643a00 0x55dc7e645e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.068+0000 7fb52cb3f640 1 -- 172.21.15.43:0/3443388253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7e63a4d0 msgr2=0x55dc7e63a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.068+0000 7fb52cb3f640 1 --2- 172.21.15.43:0/3443388253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7e63a4d0 0x55dc7e63a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.068+0000 7fb52cb3f640 1 -- 172.21.15.43:0/3443388253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc7e52e1b0 con 0x55dc7e63ae80 2024-04-21T08:44:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.068+0000 7fb52db41640 1 --2- 172.21.15.43:0/3348670356 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb5140a3d20 0x7fb5140a61e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb520005c40 tx=0x7fb520012a90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.068+0000 7fb52cb3f640 1 --2- 172.21.15.43:0/3443388253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e63ae80 0x55dc7e643430 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fb5180c70a0 tx=0x7fb5180ba600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9668ea6b2a7ceb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.068+0000 7fb5137fe640 1 -- 172.21.15.43:0/3443388253 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5180be2c0 con 0x55dc7e63ae80 2024-04-21T08:44:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.069+0000 7fb5137fe640 1 -- 172.21.15.43:0/3443388253 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5180bf060 con 0x55dc7e63ae80 2024-04-21T08:44:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.069+0000 7fb5137fe640 1 -- 172.21.15.43:0/3443388253 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5180daa30 con 0x55dc7e63ae80 2024-04-21T08:44:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.069+0000 7fb532196740 1 -- 172.21.15.43:0/3443388253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e63ae80 msgr2=0x55dc7e643430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.069+0000 7fb532196740 1 --2- 172.21.15.43:0/3443388253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e63ae80 0x55dc7e643430 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fb5180c70a0 tx=0x7fb5180ba600 comp rx=0 tx=0).stop 2024-04-21T08:44:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.069+0000 7fb532196740 1 -- 172.21.15.43:0/3443388253 shutdown_connections 2024-04-21T08:44:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.069+0000 7fb532196740 1 --2- 172.21.15.43:0/3443388253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc7e643a00 0x55dc7e645e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.069+0000 7fb532196740 1 --2- 172.21.15.43:0/3443388253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e63ae80 0x55dc7e643430 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.069+0000 7fb532196740 1 --2- 172.21.15.43:0/3443388253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7e63a4d0 0x55dc7e63a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.069+0000 7fb532196740 1 -- 172.21.15.43:0/3443388253 >> 172.21.15.43:0/3443388253 conn(0x55dc7e634ba0 msgr2=0x55dc7e634f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.069+0000 7fb532196740 1 -- 172.21.15.43:0/3443388253 shutdown_connections 2024-04-21T08:44:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.069+0000 7fb532196740 1 -- 172.21.15.43:0/3443388253 wait complete. 2024-04-21T08:44:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.070+0000 7fb532196740 1 -- start start 2024-04-21T08:44:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.070+0000 7fb532196740 1 Processor -- start 2024-04-21T08:44:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.070+0000 7fb532196740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc7e63a4d0 0x55dc7e643270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.070+0000 7fb532196740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e63ae80 0x55dc7e640670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.070+0000 7fb532196740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7e643a00 0x55dc7e640bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.070+0000 7fb532196740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dc7e646730 con 0x55dc7e63ae80 2024-04-21T08:44:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.070+0000 7fb532196740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dc7e558500 con 0x55dc7e643a00 2024-04-21T08:44:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.070+0000 7fb532196740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dc7e6468b0 con 0x55dc7e63a4d0 2024-04-21T08:44:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.070+0000 7fb532196740 10 client.0 _ll_register_callbacks cb 0x55dc7e526b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.070+0000 7fb52cb3f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e63ae80 0x55dc7e640670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.070+0000 7fb52cb3f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e63ae80 0x55dc7e640670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34644/0 (socket says 172.21.15.43:34644) 2024-04-21T08:44:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.070+0000 7fb52cb3f640 1 -- 172.21.15.43:0/2334617520 learned_addr learned my addr 172.21.15.43:0/2334617520 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.070+0000 7fb52db41640 1 --2- 172.21.15.43:0/2334617520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7e643a00 0x55dc7e640bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.070+0000 7fb52d340640 1 --2- 172.21.15.43:0/2334617520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc7e63a4d0 0x55dc7e643270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.071+0000 7fb52cb3f640 1 -- 172.21.15.43:0/2334617520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc7e63a4d0 msgr2=0x55dc7e643270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.071+0000 7fb52cb3f640 1 --2- 172.21.15.43:0/2334617520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc7e63a4d0 0x55dc7e643270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.071+0000 7fb52cb3f640 1 -- 172.21.15.43:0/2334617520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7e643a00 msgr2=0x55dc7e640bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.071+0000 7fb52cb3f640 1 --2- 172.21.15.43:0/2334617520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7e643a00 0x55dc7e640bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.071+0000 7fb52cb3f640 1 -- 172.21.15.43:0/2334617520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc7e2b13f0 con 0x55dc7e63ae80 2024-04-21T08:44:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.071+0000 7fb52cb3f640 1 --2- 172.21.15.43:0/2334617520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e63ae80 0x55dc7e640670 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fb518006010 tx=0x7fb5180b9330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3bbb938840e3962 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.071+0000 7fb510ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.071+0000 7fb510ff9640 1 -- 172.21.15.43:0/2334617520 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5180d80e0 con 0x55dc7e63ae80 2024-04-21T08:44:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.071+0000 7fb510ff9640 1 -- 172.21.15.43:0/2334617520 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fb5180d6120 con 0x55dc7e63ae80 2024-04-21T08:44:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.071+0000 7fb532196740 1 -- 172.21.15.43:0/2334617520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5180bf060 con 0x55dc7e63ae80 2024-04-21T08:44:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.071+0000 7fb510ff9640 1 -- 172.21.15.43:0/2334617520 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb5180d82c0 con 0x55dc7e63ae80 2024-04-21T08:44:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.071+0000 7fb532196740 10 client.17925 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.071+0000 7fb532196740 1 -- 172.21.15.43:0/2334617520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc7e37b7d0 con 0x55dc7e63ae80 2024-04-21T08:44:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.071+0000 7fb510ff9640 1 -- 172.21.15.43:0/2334617520 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5180bf060 con 0x55dc7e63ae80 2024-04-21T08:44:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.071+0000 7fb510ff9640 1 -- 172.21.15.43:0/2334617520 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb5180c71d0 con 0x55dc7e63ae80 2024-04-21T08:44:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.072+0000 7fb532196740 10 client.17925 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.072+0000 7fb532196740 10 client.17925 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:44:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.072+0000 7fb532196740 20 client.17925 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.072+0000 7fb532196740 1 --2- 172.21.15.43:0/2334617520 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55dc7e653710 0x55dc7e655b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.072+0000 7fb532196740 4 client.17925 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.072+0000 7fb532196740 1 -- 172.21.15.43:0/2334617520 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc7e53e930 con 0x55dc7e653710 2024-04-21T08:44:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.072+0000 7fb52d340640 1 --2- 172.21.15.43:0/2334617520 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55dc7e653710 0x55dc7e655b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.072+0000 7fb510ff9640 10 client.17925 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:44:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.072+0000 7fb52d340640 1 --2- 172.21.15.43:0/2334617520 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55dc7e653710 0x55dc7e655b00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2825a7b35fb400de server_cookie=130ef2d02892f553 in_seq=0 out_seq=0 2024-04-21T08:44:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.073+0000 7fb510ff9640 1 -- 172.21.15.43:0/2334617520 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc7e53e930 con 0x55dc7e653710 2024-04-21T08:44:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.073+0000 7fb510ff9640 10 client.17925 handle_command_reply: tid=0 2024-04-21T08:44:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.073+0000 7fb532196740 1 client.17925 shutdown 2024-04-21T08:44:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.073+0000 7fb532196740 1 -- 172.21.15.43:0/2334617520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e63ae80 msgr2=0x55dc7e640670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.073+0000 7fb532196740 1 --2- 172.21.15.43:0/2334617520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e63ae80 0x55dc7e640670 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fb518006010 tx=0x7fb5180b9330 comp rx=0 tx=0).stop 2024-04-21T08:44:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.073+0000 7fb532196740 1 -- 172.21.15.43:0/2334617520 shutdown_connections 2024-04-21T08:44:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.073+0000 7fb532196740 1 --2- 172.21.15.43:0/2334617520 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55dc7e653710 0x55dc7e655b00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.073+0000 7fb532196740 1 --2- 172.21.15.43:0/2334617520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7e643a00 0x55dc7e640bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.073+0000 7fb510ff9640 0 client.17925 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:44:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.073+0000 7fb532196740 1 --2- 172.21.15.43:0/2334617520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e63ae80 0x55dc7e640670 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.073+0000 7fb532196740 1 --2- 172.21.15.43:0/2334617520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc7e63a4d0 0x55dc7e643270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.073+0000 7fb532196740 1 -- 172.21.15.43:0/2334617520 >> 172.21.15.43:0/2334617520 conn(0x55dc7e634ba0 msgr2=0x55dc7e6368f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:25.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.074+0000 7fb532196740 1 -- 172.21.15.43:0/2334617520 shutdown_connections 2024-04-21T08:44:25.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.074+0000 7fb532196740 1 -- 172.21.15.43:0/2334617520 wait complete. 2024-04-21T08:44:25.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.074+0000 7fb532196740 20 client.17925 trim_cache size 0 max 16384 2024-04-21T08:44:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.087+0000 7fb532196740 1 Processor -- start 2024-04-21T08:44:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.087+0000 7fb532196740 1 -- start start 2024-04-21T08:44:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.087+0000 7fb532196740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7e63a4d0 0x55dc7e64f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.087+0000 7fb532196740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc7e63ae80 0x55dc7e633a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.087+0000 7fb532196740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e643a00 0x55dc7e642da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.087+0000 7fb532196740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281b16b0 con 0x55dc7e643a00 2024-04-21T08:44:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.087+0000 7fb532196740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281b31f0 con 0x55dc7e63a4d0 2024-04-21T08:44:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.087+0000 7fb532196740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281b1d80 con 0x55dc7e63ae80 2024-04-21T08:44:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.087+0000 7fb52db41640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e643a00 0x55dc7e642da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.087+0000 7fb52db41640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e643a00 0x55dc7e642da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34658/0 (socket says 172.21.15.43:34658) 2024-04-21T08:44:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.087+0000 7fb52db41640 1 -- 172.21.15.43:0/3616384184 learned_addr learned my addr 172.21.15.43:0/3616384184 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.088+0000 7fb52d340640 1 --2- 172.21.15.43:0/3616384184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7e63a4d0 0x55dc7e64f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.088+0000 7fb52db41640 1 -- 172.21.15.43:0/3616384184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc7e63ae80 msgr2=0x55dc7e633a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.088+0000 7fb52db41640 1 --2- 172.21.15.43:0/3616384184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc7e63ae80 0x55dc7e633a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.088+0000 7fb52db41640 1 -- 172.21.15.43:0/3616384184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7e63a4d0 msgr2=0x55dc7e64f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.088+0000 7fb52db41640 1 --2- 172.21.15.43:0/3616384184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7e63a4d0 0x55dc7e64f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.088+0000 7fb52db41640 1 -- 172.21.15.43:0/3616384184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc7e640f40 con 0x55dc7e643a00 2024-04-21T08:44:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.088+0000 7fb52db41640 1 --2- 172.21.15.43:0/3616384184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e643a00 0x55dc7e642da0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fb520003d50 tx=0x7fb520009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6feef0442005fab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.088+0000 7fb512ffd640 1 -- 172.21.15.43:0/3616384184 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520004240 con 0x55dc7e643a00 2024-04-21T08:44:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.088+0000 7fb512ffd640 1 -- 172.21.15.43:0/3616384184 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb52001f400 con 0x55dc7e643a00 2024-04-21T08:44:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.088+0000 7fb512ffd640 1 -- 172.21.15.43:0/3616384184 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52000f620 con 0x55dc7e643a00 2024-04-21T08:44:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.088+0000 7fb532196740 1 -- 172.21.15.43:0/3616384184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e643a00 msgr2=0x55dc7e642da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.088+0000 7fb532196740 1 --2- 172.21.15.43:0/3616384184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e643a00 0x55dc7e642da0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fb520003d50 tx=0x7fb520009500 comp rx=0 tx=0).stop 2024-04-21T08:44:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.089+0000 7fb532196740 1 -- 172.21.15.43:0/3616384184 shutdown_connections 2024-04-21T08:44:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.089+0000 7fb532196740 1 --2- 172.21.15.43:0/3616384184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e643a00 0x55dc7e642da0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.089+0000 7fb532196740 1 --2- 172.21.15.43:0/3616384184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc7e63ae80 0x55dc7e633a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.089+0000 7fb532196740 1 --2- 172.21.15.43:0/3616384184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7e63a4d0 0x55dc7e64f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.089+0000 7fb532196740 1 -- 172.21.15.43:0/3616384184 >> 172.21.15.43:0/3616384184 conn(0x55dc7e634ba0 msgr2=0x55dc7e633370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.089+0000 7fb532196740 1 -- 172.21.15.43:0/3616384184 shutdown_connections 2024-04-21T08:44:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.089+0000 7fb532196740 1 -- 172.21.15.43:0/3616384184 wait complete. 2024-04-21T08:44:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.089+0000 7fb532196740 1 -- start start 2024-04-21T08:44:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.089+0000 7fb532196740 1 Processor -- start 2024-04-21T08:44:25.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.089+0000 7fb532196740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7e63a4d0 0x55dc7e64be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.089+0000 7fb532196740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e63ae80 0x55dc7e6471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.089+0000 7fb532196740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc7e643a00 0x55dc7e6476f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.089+0000 7fb532196740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281b2b20 con 0x55dc7e63ae80 2024-04-21T08:44:25.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.089+0000 7fb532196740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281b2450 con 0x55dc7e63a4d0 2024-04-21T08:44:25.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.089+0000 7fb532196740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281af690 con 0x55dc7e643a00 2024-04-21T08:44:25.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.089+0000 7fb532196740 10 client.0 _ll_register_callbacks cb 0x7fb518005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:25.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.089+0000 7fb52cb3f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e63ae80 0x55dc7e6471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.089+0000 7fb52cb3f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e63ae80 0x55dc7e6471b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34666/0 (socket says 172.21.15.43:34666) 2024-04-21T08:44:25.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.090+0000 7fb52cb3f640 1 -- 172.21.15.43:0/522064165 learned_addr learned my addr 172.21.15.43:0/522064165 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:25.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.090+0000 7fb52d340640 1 --2- 172.21.15.43:0/522064165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7e63a4d0 0x55dc7e64be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.090+0000 7fb52db41640 1 --2- 172.21.15.43:0/522064165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc7e643a00 0x55dc7e6476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.090+0000 7fb52cb3f640 1 -- 172.21.15.43:0/522064165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc7e643a00 msgr2=0x55dc7e6476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:25.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.090+0000 7fb52cb3f640 1 --2- 172.21.15.43:0/522064165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc7e643a00 0x55dc7e6476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.090+0000 7fb52cb3f640 1 -- 172.21.15.43:0/522064165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7e63a4d0 msgr2=0x55dc7e64be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:25.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.090+0000 7fb52cb3f640 1 --2- 172.21.15.43:0/522064165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7e63a4d0 0x55dc7e64be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.090+0000 7fb52cb3f640 1 -- 172.21.15.43:0/522064165 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc7e647f20 con 0x55dc7e63ae80 2024-04-21T08:44:25.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.090+0000 7fb52db41640 1 --2- 172.21.15.43:0/522064165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc7e643a00 0x55dc7e6476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:25.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.090+0000 7fb52d340640 1 --2- 172.21.15.43:0/522064165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7e63a4d0 0x55dc7e64be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:44:25.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.090+0000 7fb52cb3f640 1 --2- 172.21.15.43:0/522064165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e63ae80 0x55dc7e6471b0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fb5180be660 tx=0x7fb518003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de726da8db4c3a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:25.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.090+0000 7fb510ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:25.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.090+0000 7fb510ff9640 1 -- 172.21.15.43:0/522064165 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5180be730 con 0x55dc7e63ae80 2024-04-21T08:44:25.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.090+0000 7fb510ff9640 1 -- 172.21.15.43:0/522064165 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fb5180ba220 con 0x55dc7e63ae80 2024-04-21T08:44:25.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.090+0000 7fb532196740 1 -- 172.21.15.43:0/522064165 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb52001f400 con 0x55dc7e63ae80 2024-04-21T08:44:25.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.090+0000 7fb510ff9640 1 -- 172.21.15.43:0/522064165 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb5180be2c0 con 0x55dc7e63ae80 2024-04-21T08:44:25.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.090+0000 7fb532196740 10 client.17931 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:25.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.090+0000 7fb532196740 1 -- 172.21.15.43:0/522064165 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc7e647f20 con 0x55dc7e63ae80 2024-04-21T08:44:25.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.090+0000 7fb510ff9640 1 -- 172.21.15.43:0/522064165 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb52001f400 con 0x55dc7e63ae80 2024-04-21T08:44:25.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.091+0000 7fb510ff9640 1 -- 172.21.15.43:0/522064165 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb5180ba6b0 con 0x55dc7e63ae80 2024-04-21T08:44:25.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.091+0000 7fb532196740 10 client.17931 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:25.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.091+0000 7fb532196740 10 client.17931 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:44:25.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.091+0000 7fb532196740 20 client.17931 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:25.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.091+0000 7fb532196740 1 --2- 172.21.15.43:0/522064165 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55dc7e6b9eb0 0x55dc7e6bc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.091+0000 7fb532196740 4 client.17931 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:44:25.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.091+0000 7fb532196740 1 -- 172.21.15.43:0/522064165 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc7e5acc10 con 0x55dc7e6b9eb0 2024-04-21T08:44:25.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.091+0000 7fb52d340640 1 --2- 172.21.15.43:0/522064165 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55dc7e6b9eb0 0x55dc7e6bc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.092+0000 7fb52d340640 1 --2- 172.21.15.43:0/522064165 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55dc7e6b9eb0 0x55dc7e6bc2d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3401d0eff6ace93 server_cookie=4441f7f1fcf3ce91 in_seq=0 out_seq=0 2024-04-21T08:44:25.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.092+0000 7fb510ff9640 10 client.17931 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:44:25.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.092+0000 7fb510ff9640 1 -- 172.21.15.43:0/522064165 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc7e5acc10 con 0x55dc7e6b9eb0 2024-04-21T08:44:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.092+0000 7fb510ff9640 10 client.17931 handle_command_reply: tid=0 2024-04-21T08:44:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.092+0000 7fb532196740 1 client.17931 shutdown 2024-04-21T08:44:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.092+0000 7fb532196740 1 -- 172.21.15.43:0/522064165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e63ae80 msgr2=0x55dc7e6471b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.092+0000 7fb532196740 1 --2- 172.21.15.43:0/522064165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e63ae80 0x55dc7e6471b0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fb5180be660 tx=0x7fb518003f50 comp rx=0 tx=0).stop 2024-04-21T08:44:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.092+0000 7fb532196740 1 -- 172.21.15.43:0/522064165 shutdown_connections 2024-04-21T08:44:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.092+0000 7fb532196740 1 --2- 172.21.15.43:0/522064165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc7e643a00 0x55dc7e6476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.092+0000 7fb532196740 1 --2- 172.21.15.43:0/522064165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc7e63ae80 0x55dc7e6471b0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.092+0000 7fb532196740 1 --2- 172.21.15.43:0/522064165 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55dc7e6b9eb0 0x55dc7e6bc2d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.092+0000 7fb532196740 1 --2- 172.21.15.43:0/522064165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7e63a4d0 0x55dc7e64be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.092+0000 7fb510ff9640 0 client.17931 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:44:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.092+0000 7fb532196740 1 -- 172.21.15.43:0/522064165 >> 172.21.15.43:0/522064165 conn(0x55dc7e634ba0 msgr2=0x55dc7e63bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.092+0000 7fb532196740 1 -- 172.21.15.43:0/522064165 shutdown_connections 2024-04-21T08:44:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.092+0000 7fb532196740 1 -- 172.21.15.43:0/522064165 wait complete. 2024-04-21T08:44:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.092+0000 7fb532196740 20 client.17931 trim_cache size 0 max 16384 2024-04-21T08:44:25.099 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:44:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.093+0000 7fb52f5a4640 1 -- 172.21.15.43:0/3348670356 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb5140a3d20 msgr2=0x7fb5140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.093+0000 7fb52f5a4640 1 --2- 172.21.15.43:0/3348670356 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb5140a3d20 0x7fb5140a61e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb520005c40 tx=0x7fb520012a90 comp rx=0 tx=0).stop 2024-04-21T08:44:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.093+0000 7fb52f5a4640 1 -- 172.21.15.43:0/3348670356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb528103af0 msgr2=0x7fb528101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.093+0000 7fb52f5a4640 1 --2- 172.21.15.43:0/3348670356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb528103af0 0x7fb528101360 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fb518005940 tx=0x7fb51802c720 comp rx=0 tx=0).stop 2024-04-21T08:44:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.093+0000 7fb52f5a4640 1 -- 172.21.15.43:0/3348670356 shutdown_connections 2024-04-21T08:44:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.093+0000 7fb52f5a4640 1 --2- 172.21.15.43:0/3348670356 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb5140a3d20 0x7fb5140a61e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.093+0000 7fb52f5a4640 1 --2- 172.21.15.43:0/3348670356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb52810c510 0x7fb5281018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.093+0000 7fb52f5a4640 1 --2- 172.21.15.43:0/3348670356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb528103af0 0x7fb528101360 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.093+0000 7fb52f5a4640 1 --2- 172.21.15.43:0/3348670356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5281031d0 0x7fb528100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.093+0000 7fb52f5a4640 1 -- 172.21.15.43:0/3348670356 >> 172.21.15.43:0/3348670356 conn(0x7fb528077f50 msgr2=0x7fb5280fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.093+0000 7fb52f5a4640 1 -- 172.21.15.43:0/3348670356 shutdown_connections 2024-04-21T08:44:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.093+0000 7fb52f5a4640 1 -- 172.21.15.43:0/3348670356 wait complete. 2024-04-21T08:44:25.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:44:25.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.208+0000 7f11b0274640 1 -- 172.21.15.43:0/2720745168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11a8102190 msgr2=0x7f11a8102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.208+0000 7f11b0274640 1 --2- 172.21.15.43:0/2720745168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11a8102190 0x7f11a8102570 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009900 tx=0x7f11a402bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:25.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.208+0000 7f11b0274640 1 -- 172.21.15.43:0/2720745168 shutdown_connections 2024-04-21T08:44:25.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.208+0000 7f11b0274640 1 --2- 172.21.15.43:0/2720745168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11a8107260 0x7f11a810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.208+0000 7f11b0274640 1 --2- 172.21.15.43:0/2720745168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11a8102b40 0x7f11a8106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.208+0000 7f11b0274640 1 --2- 172.21.15.43:0/2720745168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11a8102190 0x7f11a8102570 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.208+0000 7f11b0274640 1 -- 172.21.15.43:0/2720745168 >> 172.21.15.43:0/2720745168 conn(0x7f11a80793a0 msgr2=0x7f11a80797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:25.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.208+0000 7f11b0274640 1 -- 172.21.15.43:0/2720745168 shutdown_connections 2024-04-21T08:44:25.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.208+0000 7f11b0274640 1 -- 172.21.15.43:0/2720745168 wait complete. 2024-04-21T08:44:25.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.209+0000 7f11b0274640 1 Processor -- start 2024-04-21T08:44:25.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.209+0000 7f11b0274640 1 -- start start 2024-04-21T08:44:25.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.209+0000 7f11b0274640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11a8102190 0x7f11a819e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.209+0000 7f11b0274640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11a8107260 0x7f11a819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.209+0000 7f11b0274640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11a81985f0 0x7f11a8198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.209+0000 7f11b0274640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11a8112210 con 0x7f11a81985f0 2024-04-21T08:44:25.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.209+0000 7f11b0274640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11a8112090 con 0x7f11a8102190 2024-04-21T08:44:25.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.209+0000 7f11b0274640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f11a8112390 con 0x7f11a8107260 2024-04-21T08:44:25.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.209+0000 7f11ae811640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11a81985f0 0x7f11a8198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.209+0000 7f11ae811640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11a81985f0 0x7f11a8198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34686/0 (socket says 172.21.15.43:34686) 2024-04-21T08:44:25.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.209+0000 7f11ae811640 1 -- 172.21.15.43:0/2169825725 learned_addr learned my addr 172.21.15.43:0/2169825725 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:25.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.209+0000 7f11ae010640 1 --2- 172.21.15.43:0/2169825725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11a8102190 0x7f11a819e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.209+0000 7f11ad80f640 1 --2- 172.21.15.43:0/2169825725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11a8107260 0x7f11a819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.209+0000 7f11ae811640 1 -- 172.21.15.43:0/2169825725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11a8107260 msgr2=0x7f11a819e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.209+0000 7f11ae811640 1 --2- 172.21.15.43:0/2169825725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11a8107260 0x7f11a819e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.209+0000 7f11ae811640 1 -- 172.21.15.43:0/2169825725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11a8102190 msgr2=0x7f11a819e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.209+0000 7f11ae811640 1 --2- 172.21.15.43:0/2169825725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11a8102190 0x7f11a819e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.209+0000 7f11ae811640 1 -- 172.21.15.43:0/2169825725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a8199110 con 0x7f11a81985f0 2024-04-21T08:44:25.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.210+0000 7f11ae811640 1 --2- 172.21.15.43:0/2169825725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11a81985f0 0x7f11a8198aa0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f11a000eaa0 tx=0x7f11a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:25.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.210+0000 7f119f7fe640 1 -- 172.21.15.43:0/2169825725 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a000cdb0 con 0x7f11a81985f0 2024-04-21T08:44:25.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.210+0000 7f119f7fe640 1 -- 172.21.15.43:0/2169825725 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11a0005020 con 0x7f11a81985f0 2024-04-21T08:44:25.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.210+0000 7f11b0274640 1 -- 172.21.15.43:0/2169825725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a8199400 con 0x7f11a81985f0 2024-04-21T08:44:25.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.210+0000 7f119f7fe640 1 -- 172.21.15.43:0/2169825725 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a00116d0 con 0x7f11a81985f0 2024-04-21T08:44:25.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.210+0000 7f11b0274640 1 -- 172.21.15.43:0/2169825725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a81a5600 con 0x7f11a81985f0 2024-04-21T08:44:25.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.212+0000 7f119f7fe640 1 -- 172.21.15.43:0/2169825725 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f11a00051d0 con 0x7f11a81985f0 2024-04-21T08:44:25.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.212+0000 7f11b2e6b740 1 Processor -- start 2024-04-21T08:44:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.213+0000 7f11b2e6b740 1 -- start start 2024-04-21T08:44:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.213+0000 7f11b2e6b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e1634d0 0x55ef0e1638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.213+0000 7f11b2e6b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef0e163e80 0x55ef0e16c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.213+0000 7f11b2e6b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef0e16ca00 0x55ef0e16ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.213+0000 7f11b2e6b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ef0e0c81f0 con 0x55ef0e1634d0 2024-04-21T08:44:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.213+0000 7f11b2e6b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ef0e0654b0 con 0x55ef0e163e80 2024-04-21T08:44:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.213+0000 7f11b2e6b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ef0e0c41b0 con 0x55ef0e16ca00 2024-04-21T08:44:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.213+0000 7f119f7fe640 1 --2- 172.21.15.43:0/2169825725 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f11940a3d20 0x7f11940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.213+0000 7f11ad80f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e1634d0 0x55ef0e1638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.213+0000 7f11ad80f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e1634d0 0x55ef0e1638b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34690/0 (socket says 172.21.15.43:34690) 2024-04-21T08:44:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.213+0000 7f11ad80f640 1 -- 172.21.15.43:0/2163909185 learned_addr learned my addr 172.21.15.43:0/2163909185 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.213+0000 7f11ae010640 1 --2- 172.21.15.43:0/2163909185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef0e163e80 0x55ef0e16c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.213+0000 7f11ae811640 1 --2- 172.21.15.43:0/2169825725 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f11940a3d20 0x7f11940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.213+0000 7f11ad80f640 1 --2- 172.21.15.43:0/2163909185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef0e16ca00 0x55ef0e16ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.213+0000 7f119f7fe640 1 -- 172.21.15.43:0/2169825725 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f11a00d0dd0 con 0x7f11a81985f0 2024-04-21T08:44:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.213+0000 7f11ad80f640 1 -- 172.21.15.43:0/2163909185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef0e16ca00 msgr2=0x55ef0e16ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.213+0000 7f11ad80f640 1 --2- 172.21.15.43:0/2163909185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef0e16ca00 0x55ef0e16ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.213+0000 7f11ad80f640 1 -- 172.21.15.43:0/2163909185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef0e163e80 msgr2=0x55ef0e16c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.213+0000 7f11ad80f640 1 --2- 172.21.15.43:0/2163909185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef0e163e80 0x55ef0e16c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.213+0000 7f11ad80f640 1 -- 172.21.15.43:0/2163909185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef0e0571b0 con 0x55ef0e1634d0 2024-04-21T08:44:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.213+0000 7f11ae010640 1 --2- 172.21.15.43:0/2163909185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef0e163e80 0x55ef0e16c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:44:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.214+0000 7f11ad80f640 1 --2- 172.21.15.43:0/2163909185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e1634d0 0x55ef0e1638b0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f11980134a0 tx=0x7f119800b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13df0adee1465e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.214+0000 7f11ae811640 1 --2- 172.21.15.43:0/2169825725 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f11940a3d20 0x7f11940a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f11a0098da0 tx=0x7f11a0087040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.214+0000 7f119cbfe640 1 -- 172.21.15.43:0/2163909185 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198019030 con 0x55ef0e1634d0 2024-04-21T08:44:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.214+0000 7f119cbfe640 1 -- 172.21.15.43:0/2163909185 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11980118f0 con 0x55ef0e1634d0 2024-04-21T08:44:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.214+0000 7f119cbfe640 1 -- 172.21.15.43:0/2163909185 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119800e890 con 0x55ef0e1634d0 2024-04-21T08:44:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.215+0000 7f11b2e6b740 1 -- 172.21.15.43:0/2163909185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e1634d0 msgr2=0x55ef0e1638b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.215+0000 7f11b2e6b740 1 --2- 172.21.15.43:0/2163909185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e1634d0 0x55ef0e1638b0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f11980134a0 tx=0x7f119800b640 comp rx=0 tx=0).stop 2024-04-21T08:44:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.215+0000 7f11b2e6b740 1 -- 172.21.15.43:0/2163909185 shutdown_connections 2024-04-21T08:44:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.215+0000 7f11b2e6b740 1 --2- 172.21.15.43:0/2163909185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef0e16ca00 0x55ef0e16ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.215+0000 7f11b2e6b740 1 --2- 172.21.15.43:0/2163909185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef0e163e80 0x55ef0e16c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.215+0000 7f11b2e6b740 1 --2- 172.21.15.43:0/2163909185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e1634d0 0x55ef0e1638b0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.215+0000 7f11b2e6b740 1 -- 172.21.15.43:0/2163909185 >> 172.21.15.43:0/2163909185 conn(0x55ef0e15dba0 msgr2=0x55ef0e15df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.215+0000 7f11b2e6b740 1 -- 172.21.15.43:0/2163909185 shutdown_connections 2024-04-21T08:44:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.215+0000 7f11b2e6b740 1 -- 172.21.15.43:0/2163909185 wait complete. 2024-04-21T08:44:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.215+0000 7f11b2e6b740 1 -- start start 2024-04-21T08:44:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.215+0000 7f11b2e6b740 1 Processor -- start 2024-04-21T08:44:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.215+0000 7f11b2e6b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef0e1634d0 0x55ef0e16c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.215+0000 7f11b2e6b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef0e163e80 0x55ef0e169700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.215+0000 7f11b2e6b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e16ca00 0x55ef0e169d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.215+0000 7f11b2e6b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ef0e16f970 con 0x55ef0e16ca00 2024-04-21T08:44:25.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.215+0000 7f11b2e6b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ef0e081500 con 0x55ef0e1634d0 2024-04-21T08:44:25.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.215+0000 7f11b2e6b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ef0e16faf0 con 0x55ef0e163e80 2024-04-21T08:44:25.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.215+0000 7f11b2e6b740 10 client.0 _ll_register_callbacks cb 0x55ef0e04fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:25.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.216+0000 7f11ad80f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e16ca00 0x55ef0e169d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.216+0000 7f11ad80f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e16ca00 0x55ef0e169d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34700/0 (socket says 172.21.15.43:34700) 2024-04-21T08:44:25.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.216+0000 7f11ad80f640 1 -- 172.21.15.43:0/3004511792 learned_addr learned my addr 172.21.15.43:0/3004511792 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:25.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.216+0000 7f11ae010640 1 --2- 172.21.15.43:0/3004511792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef0e163e80 0x55ef0e169700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.216+0000 7f11ad80f640 1 --2- 172.21.15.43:0/3004511792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef0e1634d0 0x55ef0e16c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.216+0000 7f11ad80f640 1 -- 172.21.15.43:0/3004511792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef0e163e80 msgr2=0x55ef0e169700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:25.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.216+0000 7f11ad80f640 1 --2- 172.21.15.43:0/3004511792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef0e163e80 0x55ef0e169700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.216+0000 7f11ad80f640 1 -- 172.21.15.43:0/3004511792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef0e1634d0 msgr2=0x55ef0e16c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:25.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.216+0000 7f11ad80f640 1 --2- 172.21.15.43:0/3004511792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef0e1634d0 0x55ef0e16c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.216+0000 7f11ad80f640 1 -- 172.21.15.43:0/3004511792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef0ddda3f0 con 0x55ef0e16ca00 2024-04-21T08:44:25.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.216+0000 7f11ad80f640 1 --2- 172.21.15.43:0/3004511792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e16ca00 0x55ef0e169d10 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f1198013470 tx=0x7f1198015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32645f1fedb1963d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:25.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.216+0000 7f1191ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:25.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.216+0000 7f1191ffb640 1 -- 172.21.15.43:0/3004511792 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198019070 con 0x55ef0e16ca00 2024-04-21T08:44:25.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.216+0000 7f1191ffb640 1 -- 172.21.15.43:0/3004511792 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f1198011440 con 0x55ef0e16ca00 2024-04-21T08:44:25.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.216+0000 7f11b2e6b740 1 -- 172.21.15.43:0/3004511792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11980118f0 con 0x55ef0e16ca00 2024-04-21T08:44:25.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.216+0000 7f1191ffb640 1 -- 172.21.15.43:0/3004511792 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1198034050 con 0x55ef0e16ca00 2024-04-21T08:44:25.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.216+0000 7f11b2e6b740 10 client.17943 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:25.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.216+0000 7f11b2e6b740 1 -- 172.21.15.43:0/3004511792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef0dea47d0 con 0x55ef0e16ca00 2024-04-21T08:44:25.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.216+0000 7f1191ffb640 1 -- 172.21.15.43:0/3004511792 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11980118f0 con 0x55ef0e16ca00 2024-04-21T08:44:25.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.217+0000 7f1191ffb640 1 -- 172.21.15.43:0/3004511792 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f119800ece0 con 0x55ef0e16ca00 2024-04-21T08:44:25.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.217+0000 7f11b2e6b740 10 client.17943 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:25.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.217+0000 7f11b2e6b740 10 client.17943 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:44:25.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.217+0000 7f11b2e6b740 20 client.17943 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:25.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.217+0000 7f11b2e6b740 1 --2- 172.21.15.43:0/3004511792 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55ef0e17cbf0 0x55ef0e17efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.217+0000 7f11b2e6b740 4 client.17943 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:25.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.217+0000 7f11b2e6b740 1 -- 172.21.15.43:0/3004511792 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef0e067930 con 0x55ef0e17cbf0 2024-04-21T08:44:25.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.217+0000 7f11ae010640 1 --2- 172.21.15.43:0/3004511792 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55ef0e17cbf0 0x55ef0e17efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.217+0000 7f11ae010640 1 --2- 172.21.15.43:0/3004511792 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55ef0e17cbf0 0x55ef0e17efe0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8ab4a6550957ed64 server_cookie=7859ce22be7342d0 in_seq=0 out_seq=0 2024-04-21T08:44:25.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.217+0000 7f1191ffb640 10 client.17943 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:44:25.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.218+0000 7f1191ffb640 1 -- 172.21.15.43:0/3004511792 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ef0e067930 con 0x55ef0e17cbf0 2024-04-21T08:44:25.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.218+0000 7f1191ffb640 10 client.17943 handle_command_reply: tid=0 2024-04-21T08:44:25.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.218+0000 7f11b2e6b740 1 client.17943 shutdown 2024-04-21T08:44:25.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.218+0000 7f11b2e6b740 1 -- 172.21.15.43:0/3004511792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e16ca00 msgr2=0x55ef0e169d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.218+0000 7f11b2e6b740 1 --2- 172.21.15.43:0/3004511792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e16ca00 0x55ef0e169d10 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f1198013470 tx=0x7f1198015040 comp rx=0 tx=0).stop 2024-04-21T08:44:25.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.218+0000 7f11b2e6b740 1 -- 172.21.15.43:0/3004511792 shutdown_connections 2024-04-21T08:44:25.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.218+0000 7f11b2e6b740 1 --2- 172.21.15.43:0/3004511792 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55ef0e17cbf0 0x55ef0e17efe0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.218+0000 7f11b2e6b740 1 --2- 172.21.15.43:0/3004511792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e16ca00 0x55ef0e169d10 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.218+0000 7f1191ffb640 0 client.17943 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:44:25.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.218+0000 7f11b2e6b740 1 --2- 172.21.15.43:0/3004511792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef0e163e80 0x55ef0e169700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.218+0000 7f11b2e6b740 1 --2- 172.21.15.43:0/3004511792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef0e1634d0 0x55ef0e16c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.218+0000 7f11b2e6b740 1 -- 172.21.15.43:0/3004511792 >> 172.21.15.43:0/3004511792 conn(0x55ef0e15dba0 msgr2=0x55ef0e15f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:25.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.218+0000 7f11b2e6b740 1 -- 172.21.15.43:0/3004511792 shutdown_connections 2024-04-21T08:44:25.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.218+0000 7f11b2e6b740 1 -- 172.21.15.43:0/3004511792 wait complete. 2024-04-21T08:44:25.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.218+0000 7f11b2e6b740 20 client.17943 trim_cache size 0 max 16384 2024-04-21T08:44:25.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.232+0000 7f11b2e6b740 1 Processor -- start 2024-04-21T08:44:25.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.232+0000 7f11b2e6b740 1 -- start start 2024-04-21T08:44:25.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.232+0000 7f11b2e6b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef0e1634d0 0x55ef0e168fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.232+0000 7f11b2e6b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef0e163e80 0x55ef0e1785d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.232+0000 7f11b2e6b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e16ca00 0x55ef0e16be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.232+0000 7f11b2e6b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81af490 con 0x55ef0e16ca00 2024-04-21T08:44:25.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.232+0000 7f11b2e6b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81b0fd0 con 0x55ef0e163e80 2024-04-21T08:44:25.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.232+0000 7f11b2e6b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81afb60 con 0x55ef0e1634d0 2024-04-21T08:44:25.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.232+0000 7f11ad80f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e16ca00 0x55ef0e16be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.232+0000 7f11ad80f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e16ca00 0x55ef0e16be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34716/0 (socket says 172.21.15.43:34716) 2024-04-21T08:44:25.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.232+0000 7f11ad80f640 1 -- 172.21.15.43:0/2427926799 learned_addr learned my addr 172.21.15.43:0/2427926799 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:25.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.232+0000 7f11ae010640 1 --2- 172.21.15.43:0/2427926799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef0e163e80 0x55ef0e1785d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.232+0000 7f11ad80f640 1 -- 172.21.15.43:0/2427926799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef0e1634d0 msgr2=0x55ef0e168fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:25.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.232+0000 7f11ad80f640 1 --2- 172.21.15.43:0/2427926799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef0e1634d0 0x55ef0e168fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.232+0000 7f11ad80f640 1 -- 172.21.15.43:0/2427926799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef0e163e80 msgr2=0x55ef0e1785d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:25.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.232+0000 7f11ad80f640 1 --2- 172.21.15.43:0/2427926799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef0e163e80 0x55ef0e1785d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.233+0000 7f11ad80f640 1 -- 172.21.15.43:0/2427926799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef0e184e20 con 0x55ef0e16ca00 2024-04-21T08:44:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.233+0000 7f11ad80f640 1 --2- 172.21.15.43:0/2427926799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e16ca00 0x55ef0e16be30 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f1198005c70 tx=0x7f1198005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93b1365c6049f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.233+0000 7f1193fff640 1 -- 172.21.15.43:0/2427926799 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198019040 con 0x55ef0e16ca00 2024-04-21T08:44:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.233+0000 7f1193fff640 1 -- 172.21.15.43:0/2427926799 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f119800b3a0 con 0x55ef0e16ca00 2024-04-21T08:44:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.233+0000 7f1193fff640 1 -- 172.21.15.43:0/2427926799 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198013020 con 0x55ef0e16ca00 2024-04-21T08:44:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.233+0000 7f11b2e6b740 1 -- 172.21.15.43:0/2427926799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e16ca00 msgr2=0x55ef0e16be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.233+0000 7f11b2e6b740 1 --2- 172.21.15.43:0/2427926799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e16ca00 0x55ef0e16be30 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f1198005c70 tx=0x7f1198005ca0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.233+0000 7f11b2e6b740 1 -- 172.21.15.43:0/2427926799 shutdown_connections 2024-04-21T08:44:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.233+0000 7f11b2e6b740 1 --2- 172.21.15.43:0/2427926799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e16ca00 0x55ef0e16be30 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.233+0000 7f11b2e6b740 1 --2- 172.21.15.43:0/2427926799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef0e163e80 0x55ef0e1785d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.233+0000 7f11b2e6b740 1 --2- 172.21.15.43:0/2427926799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef0e1634d0 0x55ef0e168fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.233+0000 7f11b2e6b740 1 -- 172.21.15.43:0/2427926799 >> 172.21.15.43:0/2427926799 conn(0x55ef0e15dba0 msgr2=0x55ef0e15cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.234+0000 7f11b2e6b740 1 -- 172.21.15.43:0/2427926799 shutdown_connections 2024-04-21T08:44:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.234+0000 7f11b2e6b740 1 -- 172.21.15.43:0/2427926799 wait complete. 2024-04-21T08:44:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.234+0000 7f11b2e6b740 1 -- start start 2024-04-21T08:44:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.234+0000 7f11b2e6b740 1 Processor -- start 2024-04-21T08:44:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.234+0000 7f11b2e6b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e1634d0 0x55ef0e182100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.234+0000 7f11b2e6b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef0e163e80 0x55ef0e17f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.234+0000 7f11b2e6b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef0e16ca00 0x55ef0e17f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.234+0000 7f11b2e6b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81b0900 con 0x55ef0e1634d0 2024-04-21T08:44:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.234+0000 7f11b2e6b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81b0230 con 0x55ef0e163e80 2024-04-21T08:44:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.234+0000 7f11b2e6b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81ad470 con 0x55ef0e16ca00 2024-04-21T08:44:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.234+0000 7f11b2e6b740 10 client.0 _ll_register_callbacks cb 0x7f119800bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.234+0000 7f11ad80f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e1634d0 0x55ef0e182100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.234+0000 7f11ad80f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e1634d0 0x55ef0e182100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34732/0 (socket says 172.21.15.43:34732) 2024-04-21T08:44:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.234+0000 7f11ad80f640 1 -- 172.21.15.43:0/3534492167 learned_addr learned my addr 172.21.15.43:0/3534492167 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.234+0000 7f11ae010640 1 --2- 172.21.15.43:0/3534492167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef0e163e80 0x55ef0e17f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.234+0000 7f11ad80f640 1 --2- 172.21.15.43:0/3534492167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef0e16ca00 0x55ef0e17f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.235+0000 7f11ad80f640 1 -- 172.21.15.43:0/3534492167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef0e16ca00 msgr2=0x55ef0e17f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.235+0000 7f11ad80f640 1 --2- 172.21.15.43:0/3534492167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef0e16ca00 0x55ef0e17f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.235+0000 7f11ad80f640 1 -- 172.21.15.43:0/3534492167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef0e163e80 msgr2=0x55ef0e17f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.235+0000 7f11ad80f640 1 --2- 172.21.15.43:0/3534492167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef0e163e80 0x55ef0e17f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.235+0000 7f11ad80f640 1 -- 172.21.15.43:0/3534492167 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef0e17ff10 con 0x55ef0e1634d0 2024-04-21T08:44:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.235+0000 7f11ad80f640 1 --2- 172.21.15.43:0/3534492167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e1634d0 0x55ef0e182100 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f119800be60 tx=0x7f1198015f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c7fbfb39c24628 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.235+0000 7f1191ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.235+0000 7f1191ffb640 1 -- 172.21.15.43:0/3534492167 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119803c070 con 0x55ef0e1634d0 2024-04-21T08:44:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.235+0000 7f1191ffb640 1 -- 172.21.15.43:0/3534492167 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f1198013240 con 0x55ef0e1634d0 2024-04-21T08:44:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.235+0000 7f11b2e6b740 1 -- 172.21.15.43:0/3534492167 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f119800b3a0 con 0x55ef0e1634d0 2024-04-21T08:44:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.235+0000 7f1191ffb640 1 -- 172.21.15.43:0/3534492167 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1198019040 con 0x55ef0e1634d0 2024-04-21T08:44:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.235+0000 7f11b2e6b740 10 client.17949 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.235+0000 7f11b2e6b740 1 -- 172.21.15.43:0/3534492167 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef0e1e2b00 con 0x55ef0e1634d0 2024-04-21T08:44:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.235+0000 7f1191ffb640 1 -- 172.21.15.43:0/3534492167 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f119800b3a0 con 0x55ef0e1634d0 2024-04-21T08:44:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.235+0000 7f1191ffb640 1 -- 172.21.15.43:0/3534492167 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1198011750 con 0x55ef0e1634d0 2024-04-21T08:44:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.235+0000 7f11b2e6b740 10 client.17949 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.235+0000 7f11b2e6b740 10 client.17949 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:44:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.235+0000 7f11b2e6b740 20 client.17949 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.235+0000 7f11b2e6b740 1 --2- 172.21.15.43:0/3534492167 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55ef0e1e3130 0x55ef0e1e5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.235+0000 7f11b2e6b740 4 client.17949 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:44:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.235+0000 7f11b2e6b740 1 -- 172.21.15.43:0/3534492167 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef0e0d5c10 con 0x55ef0e1e3130 2024-04-21T08:44:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.236+0000 7f11ae010640 1 --2- 172.21.15.43:0/3534492167 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55ef0e1e3130 0x55ef0e1e5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.236+0000 7f11ae010640 1 --2- 172.21.15.43:0/3534492167 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55ef0e1e3130 0x55ef0e1e5550 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f53f4a16473ac8f server_cookie=a25f7d4accba0ac7 in_seq=0 out_seq=0 2024-04-21T08:44:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.236+0000 7f1191ffb640 10 client.17949 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:44:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.236+0000 7f1191ffb640 1 -- 172.21.15.43:0/3534492167 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef0e0d5c10 con 0x55ef0e1e3130 2024-04-21T08:44:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.236+0000 7f1191ffb640 10 client.17949 handle_command_reply: tid=0 2024-04-21T08:44:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.236+0000 7f11b2e6b740 1 client.17949 shutdown 2024-04-21T08:44:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.236+0000 7f11b2e6b740 1 -- 172.21.15.43:0/3534492167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e1634d0 msgr2=0x55ef0e182100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.236+0000 7f11b2e6b740 1 --2- 172.21.15.43:0/3534492167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e1634d0 0x55ef0e182100 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f119800be60 tx=0x7f1198015f90 comp rx=0 tx=0).stop 2024-04-21T08:44:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.236+0000 7f11b2e6b740 1 -- 172.21.15.43:0/3534492167 shutdown_connections 2024-04-21T08:44:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.236+0000 7f11b2e6b740 1 --2- 172.21.15.43:0/3534492167 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55ef0e1e3130 0x55ef0e1e5550 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.236+0000 7f11b2e6b740 1 --2- 172.21.15.43:0/3534492167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef0e16ca00 0x55ef0e17f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.236+0000 7f11b2e6b740 1 --2- 172.21.15.43:0/3534492167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef0e163e80 0x55ef0e17f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.236+0000 7f11b2e6b740 1 --2- 172.21.15.43:0/3534492167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0e1634d0 0x55ef0e182100 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.236+0000 7f1191ffb640 0 client.17949 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:44:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.237+0000 7f11b2e6b740 1 -- 172.21.15.43:0/3534492167 >> 172.21.15.43:0/3534492167 conn(0x55ef0e15dba0 msgr2=0x55ef0e1785d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.237+0000 7f11b2e6b740 1 -- 172.21.15.43:0/3534492167 shutdown_connections 2024-04-21T08:44:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.237+0000 7f11b2e6b740 1 -- 172.21.15.43:0/3534492167 wait complete. 2024-04-21T08:44:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.237+0000 7f11b2e6b740 20 client.17949 trim_cache size 0 max 16384 2024-04-21T08:44:25.242 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:44:25.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.237+0000 7f11b0274640 1 -- 172.21.15.43:0/2169825725 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f11940a3d20 msgr2=0x7f11940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.237+0000 7f11b0274640 1 --2- 172.21.15.43:0/2169825725 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f11940a3d20 0x7f11940a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f11a0098da0 tx=0x7f11a0087040 comp rx=0 tx=0).stop 2024-04-21T08:44:25.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.237+0000 7f11b0274640 1 -- 172.21.15.43:0/2169825725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11a81985f0 msgr2=0x7f11a8198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.237+0000 7f11b0274640 1 --2- 172.21.15.43:0/2169825725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11a81985f0 0x7f11a8198aa0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f11a000eaa0 tx=0x7f11a000ef60 comp rx=0 tx=0).stop 2024-04-21T08:44:25.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.237+0000 7f11b0274640 1 -- 172.21.15.43:0/2169825725 shutdown_connections 2024-04-21T08:44:25.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.237+0000 7f11b0274640 1 --2- 172.21.15.43:0/2169825725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11a81985f0 0x7f11a8198aa0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.237+0000 7f11b0274640 1 --2- 172.21.15.43:0/2169825725 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f11940a3d20 0x7f11940a61e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.237+0000 7f11b0274640 1 --2- 172.21.15.43:0/2169825725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11a8107260 0x7f11a819e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.237+0000 7f11b0274640 1 --2- 172.21.15.43:0/2169825725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11a8102190 0x7f11a819e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.237+0000 7f11b0274640 1 -- 172.21.15.43:0/2169825725 >> 172.21.15.43:0/2169825725 conn(0x7f11a80793a0 msgr2=0x7f11a8100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:25.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.237+0000 7f11b0274640 1 -- 172.21.15.43:0/2169825725 shutdown_connections 2024-04-21T08:44:25.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.237+0000 7f11b0274640 1 -- 172.21.15.43:0/2169825725 wait complete. 2024-04-21T08:44:25.246 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:25 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3895668211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:25.247 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:44:25.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.352+0000 7f7598231640 1 -- 172.21.15.43:0/2011784582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7590107260 msgr2=0x7f759010ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.352+0000 7f7598231640 1 --2- 172.21.15.43:0/2011784582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7590107260 0x7f759010ee30 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f7588009ca0 tx=0x7f758802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.352+0000 7f7598231640 1 -- 172.21.15.43:0/2011784582 shutdown_connections 2024-04-21T08:44:25.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.352+0000 7f7598231640 1 --2- 172.21.15.43:0/2011784582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7590107260 0x7f759010ee30 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.352+0000 7f7598231640 1 --2- 172.21.15.43:0/2011784582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7590102b40 0x7f7590106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.352+0000 7f7598231640 1 --2- 172.21.15.43:0/2011784582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7590102190 0x7f7590102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.352+0000 7f7598231640 1 -- 172.21.15.43:0/2011784582 >> 172.21.15.43:0/2011784582 conn(0x7f75900793a0 msgr2=0x7f75900797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:25.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.352+0000 7f7598231640 1 -- 172.21.15.43:0/2011784582 shutdown_connections 2024-04-21T08:44:25.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.352+0000 7f7598231640 1 -- 172.21.15.43:0/2011784582 wait complete. 2024-04-21T08:44:25.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.353+0000 7f7598231640 1 Processor -- start 2024-04-21T08:44:25.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.353+0000 7f7598231640 1 -- start start 2024-04-21T08:44:25.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.353+0000 7f7598231640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7590102190 0x7f759019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.353+0000 7f7598231640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7590107260 0x7f759019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.353+0000 7f7598231640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7590198590 0x7f7590198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.353+0000 7f7598231640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75901121d0 con 0x7f7590102190 2024-04-21T08:44:25.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.353+0000 7f7598231640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7590112050 con 0x7f7590107260 2024-04-21T08:44:25.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.353+0000 7f7598231640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7590112350 con 0x7f7590198590 2024-04-21T08:44:25.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.353+0000 7f7595fcd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7590102190 0x7f759019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.353+0000 7f7595fcd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7590102190 0x7f759019e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34746/0 (socket says 172.21.15.43:34746) 2024-04-21T08:44:25.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.353+0000 7f7595fcd640 1 -- 172.21.15.43:0/4203595530 learned_addr learned my addr 172.21.15.43:0/4203595530 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:25.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.353+0000 7f75957cc640 1 --2- 172.21.15.43:0/4203595530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7590107260 0x7f759019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.353+0000 7f7595fcd640 1 -- 172.21.15.43:0/4203595530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7590198590 msgr2=0x7f7590198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.353+0000 7f7595fcd640 1 --2- 172.21.15.43:0/4203595530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7590198590 0x7f7590198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.353+0000 7f7595fcd640 1 -- 172.21.15.43:0/4203595530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7590107260 msgr2=0x7f759019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.353+0000 7f7595fcd640 1 --2- 172.21.15.43:0/4203595530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7590107260 0x7f759019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.353+0000 7f7595fcd640 1 -- 172.21.15.43:0/4203595530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75901990b0 con 0x7f7590102190 2024-04-21T08:44:25.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.354+0000 7f7595fcd640 1 --2- 172.21.15.43:0/4203595530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7590102190 0x7f759019e3e0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f758c00e9f0 tx=0x7f758c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:25.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.354+0000 7f7586ffd640 1 -- 172.21.15.43:0/4203595530 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c00cd80 con 0x7f7590102190 2024-04-21T08:44:25.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.354+0000 7f7586ffd640 1 -- 172.21.15.43:0/4203595530 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f758c0054d0 con 0x7f7590102190 2024-04-21T08:44:25.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.354+0000 7f7598231640 1 -- 172.21.15.43:0/4203595530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75901993a0 con 0x7f7590102190 2024-04-21T08:44:25.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.354+0000 7f7586ffd640 1 -- 172.21.15.43:0/4203595530 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c010690 con 0x7f7590102190 2024-04-21T08:44:25.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.354+0000 7f7598231640 1 -- 172.21.15.43:0/4203595530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75901a55a0 con 0x7f7590102190 2024-04-21T08:44:25.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.356+0000 7f7586ffd640 1 -- 172.21.15.43:0/4203595530 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f758c004e90 con 0x7f7590102190 2024-04-21T08:44:25.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.356+0000 7f759ae24740 1 Processor -- start 2024-04-21T08:44:25.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.357+0000 7f759ae24740 1 -- start start 2024-04-21T08:44:25.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.357+0000 7f759ae24740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c39e5e94d0 0x55c39e5e98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.357+0000 7f759ae24740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c39e5e9e80 0x55c39e5f2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.357+0000 7f759ae24740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5f2a00 0x55c39e5f4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.357+0000 7f759ae24740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c39e54e1f0 con 0x55c39e5f2a00 2024-04-21T08:44:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.357+0000 7f759ae24740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c39e4eb4b0 con 0x55c39e5e94d0 2024-04-21T08:44:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.357+0000 7f759ae24740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c39e54a1b0 con 0x55c39e5e9e80 2024-04-21T08:44:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.357+0000 7f7586ffd640 1 --2- 172.21.15.43:0/4203595530 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f757c0a3d20 0x7f757c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.357+0000 7f75957cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5f2a00 0x55c39e5f4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.357+0000 7f75957cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5f2a00 0x55c39e5f4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34760/0 (socket says 172.21.15.43:34760) 2024-04-21T08:44:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.357+0000 7f75957cc640 1 -- 172.21.15.43:0/2851892138 learned_addr learned my addr 172.21.15.43:0/2851892138 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.357+0000 7f75967ce640 1 --2- 172.21.15.43:0/4203595530 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f757c0a3d20 0x7f757c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.357+0000 7f7586ffd640 1 -- 172.21.15.43:0/4203595530 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f758c0d1130 con 0x7f7590102190 2024-04-21T08:44:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.357+0000 7f7595fcd640 1 --2- 172.21.15.43:0/2851892138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c39e5e9e80 0x55c39e5f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.357+0000 7f75967ce640 1 --2- 172.21.15.43:0/2851892138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c39e5e94d0 0x55c39e5e98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.357+0000 7f75957cc640 1 -- 172.21.15.43:0/2851892138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c39e5e9e80 msgr2=0x55c39e5f2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.357+0000 7f75957cc640 1 --2- 172.21.15.43:0/2851892138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c39e5e9e80 0x55c39e5f2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.357+0000 7f75957cc640 1 -- 172.21.15.43:0/2851892138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c39e5e94d0 msgr2=0x55c39e5e98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.357+0000 7f75957cc640 1 --2- 172.21.15.43:0/2851892138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c39e5e94d0 0x55c39e5e98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.357+0000 7f75957cc640 1 -- 172.21.15.43:0/2851892138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c39e4dd1b0 con 0x55c39e5f2a00 2024-04-21T08:44:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.357+0000 7f75957cc640 1 --2- 172.21.15.43:0/2851892138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5f2a00 0x55c39e5f4e50 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f7580009b40 tx=0x7f7580010370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62963870de4f8f63 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.357+0000 7f75967ce640 1 --2- 172.21.15.43:0/4203595530 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f757c0a3d20 0x7f757c0a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f758802c630 tx=0x7f7588009a20 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.358+0000 7f757bfff640 1 -- 172.21.15.43:0/2851892138 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580010da0 con 0x55c39e5f2a00 2024-04-21T08:44:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.358+0000 7f757bfff640 1 -- 172.21.15.43:0/2851892138 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75800035f0 con 0x55c39e5f2a00 2024-04-21T08:44:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.358+0000 7f757bfff640 1 -- 172.21.15.43:0/2851892138 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580012670 con 0x55c39e5f2a00 2024-04-21T08:44:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.358+0000 7f759ae24740 1 -- 172.21.15.43:0/2851892138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5f2a00 msgr2=0x55c39e5f4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.358+0000 7f759ae24740 1 --2- 172.21.15.43:0/2851892138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5f2a00 0x55c39e5f4e50 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f7580009b40 tx=0x7f7580010370 comp rx=0 tx=0).stop 2024-04-21T08:44:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.358+0000 7f759ae24740 1 -- 172.21.15.43:0/2851892138 shutdown_connections 2024-04-21T08:44:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.358+0000 7f759ae24740 1 --2- 172.21.15.43:0/2851892138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5f2a00 0x55c39e5f4e50 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.358+0000 7f759ae24740 1 --2- 172.21.15.43:0/2851892138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c39e5e9e80 0x55c39e5f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.358+0000 7f759ae24740 1 --2- 172.21.15.43:0/2851892138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c39e5e94d0 0x55c39e5e98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.358+0000 7f759ae24740 1 -- 172.21.15.43:0/2851892138 >> 172.21.15.43:0/2851892138 conn(0x55c39e5e3ba0 msgr2=0x55c39e5e3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:25.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.359+0000 7f759ae24740 1 -- 172.21.15.43:0/2851892138 shutdown_connections 2024-04-21T08:44:25.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.359+0000 7f759ae24740 1 -- 172.21.15.43:0/2851892138 wait complete. 2024-04-21T08:44:25.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.359+0000 7f759ae24740 1 -- start start 2024-04-21T08:44:25.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.359+0000 7f759ae24740 1 Processor -- start 2024-04-21T08:44:25.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.359+0000 7f759ae24740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c39e5e94d0 0x55c39e5f2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.359+0000 7f759ae24740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5e9e80 0x55c39e5ef6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.359+0000 7f759ae24740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c39e5f2a00 0x55c39e5efc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.359+0000 7f759ae24740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c39e5f5970 con 0x55c39e5e9e80 2024-04-21T08:44:25.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.359+0000 7f759ae24740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c39e507500 con 0x55c39e5f2a00 2024-04-21T08:44:25.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.359+0000 7f759ae24740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c39e5f5af0 con 0x55c39e5e94d0 2024-04-21T08:44:25.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.359+0000 7f759ae24740 10 client.0 _ll_register_callbacks cb 0x55c39e4d5b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.359+0000 7f75957cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5e9e80 0x55c39e5ef6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.359+0000 7f75957cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5e9e80 0x55c39e5ef6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34772/0 (socket says 172.21.15.43:34772) 2024-04-21T08:44:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.359+0000 7f75957cc640 1 -- 172.21.15.43:0/1523182257 learned_addr learned my addr 172.21.15.43:0/1523182257 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.360+0000 7f75967ce640 1 --2- 172.21.15.43:0/1523182257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c39e5f2a00 0x55c39e5efc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.360+0000 7f7595fcd640 1 --2- 172.21.15.43:0/1523182257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c39e5e94d0 0x55c39e5f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.360+0000 7f75957cc640 1 -- 172.21.15.43:0/1523182257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c39e5e94d0 msgr2=0x55c39e5f2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.360+0000 7f75957cc640 1 --2- 172.21.15.43:0/1523182257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c39e5e94d0 0x55c39e5f2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.360+0000 7f75957cc640 1 -- 172.21.15.43:0/1523182257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c39e5f2a00 msgr2=0x55c39e5efc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.360+0000 7f75957cc640 1 --2- 172.21.15.43:0/1523182257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c39e5f2a00 0x55c39e5efc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.360+0000 7f75957cc640 1 -- 172.21.15.43:0/1523182257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c39e2603f0 con 0x55c39e5e9e80 2024-04-21T08:44:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.360+0000 7f7595fcd640 1 --2- 172.21.15.43:0/1523182257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c39e5e94d0 0x55c39e5f2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.360+0000 7f75967ce640 1 --2- 172.21.15.43:0/1523182257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c39e5f2a00 0x55c39e5efc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.360+0000 7f75957cc640 1 --2- 172.21.15.43:0/1523182257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5e9e80 0x55c39e5ef6b0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f7580005e00 tx=0x7f758001c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db62bbc6f0acc2d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.360+0000 7f75797fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:25.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.360+0000 7f75797fa640 1 -- 172.21.15.43:0/1523182257 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758001c870 con 0x55c39e5e9e80 2024-04-21T08:44:25.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.360+0000 7f75797fa640 1 -- 172.21.15.43:0/1523182257 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f758001ca10 con 0x55c39e5e9e80 2024-04-21T08:44:25.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.360+0000 7f759ae24740 1 -- 172.21.15.43:0/1523182257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75800035f0 con 0x55c39e5e9e80 2024-04-21T08:44:25.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.360+0000 7f75797fa640 1 -- 172.21.15.43:0/1523182257 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f758001cc10 con 0x55c39e5e9e80 2024-04-21T08:44:25.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.360+0000 7f759ae24740 10 client.17961 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:25.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.360+0000 7f759ae24740 1 -- 172.21.15.43:0/1523182257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c39e32a7d0 con 0x55c39e5e9e80 2024-04-21T08:44:25.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.361+0000 7f75797fa640 1 -- 172.21.15.43:0/1523182257 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75800035f0 con 0x55c39e5e9e80 2024-04-21T08:44:25.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.361+0000 7f75797fa640 1 -- 172.21.15.43:0/1523182257 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f758000f040 con 0x55c39e5e9e80 2024-04-21T08:44:25.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.361+0000 7f759ae24740 10 client.17961 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:25.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.361+0000 7f759ae24740 10 client.17961 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:25.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.361+0000 7f759ae24740 20 client.17961 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:25.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.361+0000 7f759ae24740 1 --2- 172.21.15.43:0/1523182257 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c39e602aa0 0x55c39e604e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.361+0000 7f759ae24740 4 client.17961 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:25.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.361+0000 7f759ae24740 1 -- 172.21.15.43:0/1523182257 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c39e4ed930 con 0x55c39e602aa0 2024-04-21T08:44:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.361+0000 7f7595fcd640 1 --2- 172.21.15.43:0/1523182257 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c39e602aa0 0x55c39e604e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.361+0000 7f75797fa640 10 client.17961 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.361+0000 7f7595fcd640 1 --2- 172.21.15.43:0/1523182257 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c39e602aa0 0x55c39e604e90 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f14299c151f2400b server_cookie=9471808afd98acfd in_seq=0 out_seq=0 2024-04-21T08:44:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.362+0000 7f75797fa640 1 -- 172.21.15.43:0/1523182257 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c39e4ed930 con 0x55c39e602aa0 2024-04-21T08:44:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.362+0000 7f75797fa640 10 client.17961 handle_command_reply: tid=0 2024-04-21T08:44:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.362+0000 7f759ae24740 1 client.17961 shutdown 2024-04-21T08:44:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.362+0000 7f759ae24740 1 -- 172.21.15.43:0/1523182257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5e9e80 msgr2=0x55c39e5ef6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.362+0000 7f759ae24740 1 --2- 172.21.15.43:0/1523182257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5e9e80 0x55c39e5ef6b0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f7580005e00 tx=0x7f758001c680 comp rx=0 tx=0).stop 2024-04-21T08:44:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.362+0000 7f759ae24740 1 -- 172.21.15.43:0/1523182257 shutdown_connections 2024-04-21T08:44:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.362+0000 7f759ae24740 1 --2- 172.21.15.43:0/1523182257 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c39e602aa0 0x55c39e604e90 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.362+0000 7f759ae24740 1 --2- 172.21.15.43:0/1523182257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c39e5f2a00 0x55c39e5efc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.362+0000 7f759ae24740 1 --2- 172.21.15.43:0/1523182257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5e9e80 0x55c39e5ef6b0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.362+0000 7f75797fa640 0 client.17961 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.362+0000 7f759ae24740 1 --2- 172.21.15.43:0/1523182257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c39e5e94d0 0x55c39e5f2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.362+0000 7f759ae24740 1 -- 172.21.15.43:0/1523182257 >> 172.21.15.43:0/1523182257 conn(0x55c39e5e3ba0 msgr2=0x55c39e5e58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:25.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.362+0000 7f759ae24740 1 -- 172.21.15.43:0/1523182257 shutdown_connections 2024-04-21T08:44:25.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.362+0000 7f759ae24740 1 -- 172.21.15.43:0/1523182257 wait complete. 2024-04-21T08:44:25.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.363+0000 7f759ae24740 20 client.17961 trim_cache size 0 max 16384 2024-04-21T08:44:25.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.376+0000 7f759ae24740 1 Processor -- start 2024-04-21T08:44:25.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.376+0000 7f759ae24740 1 -- start start 2024-04-21T08:44:25.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.376+0000 7f759ae24740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c39e5e94d0 0x55c39e5fe830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.376+0000 7f759ae24740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5e9e80 0x55c39e5e2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.376+0000 7f759ae24740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c39e5f2a00 0x55c39e5f1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.376+0000 7f759ae24740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75901af4c0 con 0x55c39e5e9e80 2024-04-21T08:44:25.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.376+0000 7f759ae24740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75901b1000 con 0x55c39e5f2a00 2024-04-21T08:44:25.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.376+0000 7f759ae24740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75901afb90 con 0x55c39e5e94d0 2024-04-21T08:44:25.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.376+0000 7f75957cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5e9e80 0x55c39e5e2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.376+0000 7f75957cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5e9e80 0x55c39e5e2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34782/0 (socket says 172.21.15.43:34782) 2024-04-21T08:44:25.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.376+0000 7f75957cc640 1 -- 172.21.15.43:0/3282667068 learned_addr learned my addr 172.21.15.43:0/3282667068 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:25.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.376+0000 7f7595fcd640 1 --2- 172.21.15.43:0/3282667068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c39e5e94d0 0x55c39e5fe830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.376+0000 7f75957cc640 1 -- 172.21.15.43:0/3282667068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c39e5e94d0 msgr2=0x55c39e5fe830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:25.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.376+0000 7f75957cc640 1 --2- 172.21.15.43:0/3282667068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c39e5e94d0 0x55c39e5fe830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.376+0000 7f75957cc640 1 -- 172.21.15.43:0/3282667068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c39e5f2a00 msgr2=0x55c39e5f1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:25.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.376+0000 7f75957cc640 1 --2- 172.21.15.43:0/3282667068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c39e5f2a00 0x55c39e5f1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.376+0000 7f75957cc640 1 -- 172.21.15.43:0/3282667068 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c39e5ee660 con 0x55c39e5e9e80 2024-04-21T08:44:25.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.377+0000 7f75957cc640 1 --2- 172.21.15.43:0/3282667068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5e9e80 0x55c39e5e2a80 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f7580000c00 tx=0x7f758001cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2453c7915f8bdbd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:25.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.377+0000 7f757b7fe640 1 -- 172.21.15.43:0/3282667068 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580004660 con 0x55c39e5e9e80 2024-04-21T08:44:25.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.377+0000 7f757b7fe640 1 -- 172.21.15.43:0/3282667068 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7580009530 con 0x55c39e5e9e80 2024-04-21T08:44:25.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.377+0000 7f757b7fe640 1 -- 172.21.15.43:0/3282667068 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580012c40 con 0x55c39e5e9e80 2024-04-21T08:44:25.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.377+0000 7f759ae24740 1 -- 172.21.15.43:0/3282667068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5e9e80 msgr2=0x55c39e5e2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.377+0000 7f759ae24740 1 --2- 172.21.15.43:0/3282667068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5e9e80 0x55c39e5e2a80 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f7580000c00 tx=0x7f758001cf40 comp rx=0 tx=0).stop 2024-04-21T08:44:25.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.377+0000 7f759ae24740 1 -- 172.21.15.43:0/3282667068 shutdown_connections 2024-04-21T08:44:25.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.377+0000 7f759ae24740 1 --2- 172.21.15.43:0/3282667068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c39e5f2a00 0x55c39e5f1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.377+0000 7f759ae24740 1 --2- 172.21.15.43:0/3282667068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5e9e80 0x55c39e5e2a80 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.377+0000 7f759ae24740 1 --2- 172.21.15.43:0/3282667068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c39e5e94d0 0x55c39e5fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.377+0000 7f759ae24740 1 -- 172.21.15.43:0/3282667068 >> 172.21.15.43:0/3282667068 conn(0x55c39e5e3ba0 msgr2=0x55c39e5e2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:25.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.377+0000 7f759ae24740 1 -- 172.21.15.43:0/3282667068 shutdown_connections 2024-04-21T08:44:25.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.377+0000 7f759ae24740 1 -- 172.21.15.43:0/3282667068 wait complete. 2024-04-21T08:44:25.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.378+0000 7f759ae24740 1 -- start start 2024-04-21T08:44:25.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.378+0000 7f759ae24740 1 Processor -- start 2024-04-21T08:44:25.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.378+0000 7f759ae24740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c39e5e94d0 0x55c39e5faf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.378+0000 7f759ae24740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c39e5e9e80 0x55c39e5ebf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.378+0000 7f759ae24740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5f2a00 0x55c39e5ec440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.378+0000 7f759ae24740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75901b0930 con 0x55c39e5f2a00 2024-04-21T08:44:25.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.378+0000 7f759ae24740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75901b0260 con 0x55c39e5e94d0 2024-04-21T08:44:25.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.378+0000 7f759ae24740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75901ad4a0 con 0x55c39e5e9e80 2024-04-21T08:44:25.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.378+0000 7f759ae24740 10 client.0 _ll_register_callbacks cb 0x7f7580010f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:25.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.378+0000 7f75967ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5f2a00 0x55c39e5ec440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.378+0000 7f75967ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5f2a00 0x55c39e5ec440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34790/0 (socket says 172.21.15.43:34790) 2024-04-21T08:44:25.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.378+0000 7f75967ce640 1 -- 172.21.15.43:0/662174619 learned_addr learned my addr 172.21.15.43:0/662174619 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:25.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.378+0000 7f7595fcd640 1 --2- 172.21.15.43:0/662174619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c39e5e94d0 0x55c39e5faf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.378+0000 7f75957cc640 1 --2- 172.21.15.43:0/662174619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c39e5e9e80 0x55c39e5ebf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.378+0000 7f75967ce640 1 -- 172.21.15.43:0/662174619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c39e5e9e80 msgr2=0x55c39e5ebf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:25.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.378+0000 7f75967ce640 1 --2- 172.21.15.43:0/662174619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c39e5e9e80 0x55c39e5ebf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.379+0000 7f75967ce640 1 -- 172.21.15.43:0/662174619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c39e5e94d0 msgr2=0x55c39e5faf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:25.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.379+0000 7f75967ce640 1 --2- 172.21.15.43:0/662174619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c39e5e94d0 0x55c39e5faf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.379+0000 7f75967ce640 1 -- 172.21.15.43:0/662174619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c39e5e9c20 con 0x55c39e5f2a00 2024-04-21T08:44:25.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.379+0000 7f7595fcd640 1 --2- 172.21.15.43:0/662174619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c39e5e94d0 0x55c39e5faf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:25.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.379+0000 7f75967ce640 1 --2- 172.21.15.43:0/662174619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5f2a00 0x55c39e5ec440 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f7588009ca0 tx=0x7f75880095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f491ea475d377fb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:25.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.379+0000 7f75797fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:25.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.379+0000 7f75797fa640 1 -- 172.21.15.43:0/662174619 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588039040 con 0x55c39e5f2a00 2024-04-21T08:44:25.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.379+0000 7f75797fa640 1 -- 172.21.15.43:0/662174619 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f7588042480 con 0x55c39e5f2a00 2024-04-21T08:44:25.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.379+0000 7f759ae24740 1 -- 172.21.15.43:0/662174619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7580009530 con 0x55c39e5f2a00 2024-04-21T08:44:25.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.379+0000 7f75797fa640 1 -- 172.21.15.43:0/662174619 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f758805d050 con 0x55c39e5f2a00 2024-04-21T08:44:25.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.379+0000 7f759ae24740 10 client.17967 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:25.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.379+0000 7f759ae24740 1 -- 172.21.15.43:0/662174619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c39e668a30 con 0x55c39e5f2a00 2024-04-21T08:44:25.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.379+0000 7f75797fa640 1 -- 172.21.15.43:0/662174619 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7580009530 con 0x55c39e5f2a00 2024-04-21T08:44:25.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.379+0000 7f75797fa640 1 -- 172.21.15.43:0/662174619 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f758804e070 con 0x55c39e5f2a00 2024-04-21T08:44:25.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.379+0000 7f759ae24740 10 client.17967 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:25.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.379+0000 7f759ae24740 10 client.17967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:25.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.379+0000 7f759ae24740 20 client.17967 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:25.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.379+0000 7f759ae24740 1 --2- 172.21.15.43:0/662174619 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c39e669230 0x55c39e66b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:25.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.379+0000 7f759ae24740 4 client.17967 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:44:25.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.379+0000 7f759ae24740 1 -- 172.21.15.43:0/662174619 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c39e55bc10 con 0x55c39e669230 2024-04-21T08:44:25.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.380+0000 7f7595fcd640 1 --2- 172.21.15.43:0/662174619 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c39e669230 0x55c39e66b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:25.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.380+0000 7f7595fcd640 1 --2- 172.21.15.43:0/662174619 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c39e669230 0x55c39e66b650 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44533405d3ad26c5 server_cookie=dc9d59e94a81ecc8 in_seq=0 out_seq=0 2024-04-21T08:44:25.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.380+0000 7f75797fa640 10 client.17967 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:25.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.380+0000 7f75797fa640 1 -- 172.21.15.43:0/662174619 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c39e55bc10 con 0x55c39e669230 2024-04-21T08:44:25.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.380+0000 7f75797fa640 10 client.17967 handle_command_reply: tid=0 2024-04-21T08:44:25.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.380+0000 7f759ae24740 1 client.17967 shutdown 2024-04-21T08:44:25.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.380+0000 7f759ae24740 1 -- 172.21.15.43:0/662174619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5f2a00 msgr2=0x55c39e5ec440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.380+0000 7f759ae24740 1 --2- 172.21.15.43:0/662174619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5f2a00 0x55c39e5ec440 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f7588009ca0 tx=0x7f75880095c0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.381+0000 7f759ae24740 1 -- 172.21.15.43:0/662174619 shutdown_connections 2024-04-21T08:44:25.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.381+0000 7f759ae24740 1 --2- 172.21.15.43:0/662174619 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c39e669230 0x55c39e66b650 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.381+0000 7f759ae24740 1 --2- 172.21.15.43:0/662174619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c39e5f2a00 0x55c39e5ec440 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.381+0000 7f759ae24740 1 --2- 172.21.15.43:0/662174619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c39e5e9e80 0x55c39e5ebf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.381+0000 7f75797fa640 0 client.17967 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:25.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.381+0000 7f759ae24740 1 --2- 172.21.15.43:0/662174619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c39e5e94d0 0x55c39e5faf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.381+0000 7f759ae24740 1 -- 172.21.15.43:0/662174619 >> 172.21.15.43:0/662174619 conn(0x55c39e5e3ba0 msgr2=0x55c39e5e2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:25.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.381+0000 7f759ae24740 1 -- 172.21.15.43:0/662174619 shutdown_connections 2024-04-21T08:44:25.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.381+0000 7f759ae24740 1 -- 172.21.15.43:0/662174619 wait complete. 2024-04-21T08:44:25.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.381+0000 7f759ae24740 20 client.17967 trim_cache size 0 max 16384 2024-04-21T08:44:25.388 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:44:25.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.381+0000 7f7598231640 1 -- 172.21.15.43:0/4203595530 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f757c0a3d20 msgr2=0x7f757c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.381+0000 7f7598231640 1 --2- 172.21.15.43:0/4203595530 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f757c0a3d20 0x7f757c0a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f758802c630 tx=0x7f7588009a20 comp rx=0 tx=0).stop 2024-04-21T08:44:25.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.381+0000 7f7598231640 1 -- 172.21.15.43:0/4203595530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7590102190 msgr2=0x7f759019e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:25.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.381+0000 7f7598231640 1 --2- 172.21.15.43:0/4203595530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7590102190 0x7f759019e3e0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f758c00e9f0 tx=0x7f758c00edf0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.381+0000 7f7598231640 1 -- 172.21.15.43:0/4203595530 shutdown_connections 2024-04-21T08:44:25.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.381+0000 7f7598231640 1 --2- 172.21.15.43:0/4203595530 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f757c0a3d20 0x7f757c0a61e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.381+0000 7f7598231640 1 --2- 172.21.15.43:0/4203595530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7590198590 0x7f7590198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.381+0000 7f7598231640 1 --2- 172.21.15.43:0/4203595530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7590107260 0x7f759019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.381+0000 7f7598231640 1 --2- 172.21.15.43:0/4203595530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7590102190 0x7f759019e3e0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:25.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.381+0000 7f7598231640 1 -- 172.21.15.43:0/4203595530 >> 172.21.15.43:0/4203595530 conn(0x7f75900793a0 msgr2=0x7f75900ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:25.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.381+0000 7f7598231640 1 -- 172.21.15.43:0/4203595530 shutdown_connections 2024-04-21T08:44:25.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:25.381+0000 7f7598231640 1 -- 172.21.15.43:0/4203595530 wait complete. 2024-04-21T08:44:25.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:25 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3895668211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:25.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:25 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3895668211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:26.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:26 smithi043 ceph-mon[27978]: pgmap v168: 129 pgs: 129 active+clean; 3.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-21T08:44:26.364 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:26 smithi175 ceph-mon[31920]: pgmap v168: 129 pgs: 129 active+clean; 3.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-21T08:44:26.392 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:44:26.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:44:26.392 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:44:26.445 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:26 smithi112 ceph-mon[31085]: pgmap v168: 129 pgs: 129 active+clean; 3.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-21T08:44:26.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.512+0000 7f976b04a640 1 -- 172.21.15.43:0/745203384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9764107270 msgr2=0x7f976410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:26.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.512+0000 7f976b04a640 1 --2- 172.21.15.43:0/745203384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9764107270 0x7f976410ee40 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f975c009ca0 tx=0x7f975c030500 comp rx=0 tx=0).stop 2024-04-21T08:44:26.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.512+0000 7f976b04a640 1 -- 172.21.15.43:0/745203384 shutdown_connections 2024-04-21T08:44:26.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.512+0000 7f976b04a640 1 --2- 172.21.15.43:0/745203384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9764107270 0x7f976410ee40 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.512+0000 7f976b04a640 1 --2- 172.21.15.43:0/745203384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f976406b1f0 0x7f9764106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.512+0000 7f976b04a640 1 --2- 172.21.15.43:0/745203384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f976406a840 0x7f976406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.512+0000 7f976b04a640 1 -- 172.21.15.43:0/745203384 >> 172.21.15.43:0/745203384 conn(0x7f9764077eb0 msgr2=0x7f97640782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:26.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.512+0000 7f976b04a640 1 -- 172.21.15.43:0/745203384 shutdown_connections 2024-04-21T08:44:26.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.512+0000 7f976b04a640 1 -- 172.21.15.43:0/745203384 wait complete. 2024-04-21T08:44:26.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.512+0000 7f976b04a640 1 Processor -- start 2024-04-21T08:44:26.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.512+0000 7f976b04a640 1 -- start start 2024-04-21T08:44:26.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.513+0000 7f976b04a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f976406a840 0x7f9764196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.513+0000 7f976b04a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9764107270 0x7f9764196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.513+0000 7f976b04a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f976419ade0 0x7f976419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.513+0000 7f976b04a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f976410ffe0 con 0x7f976419ade0 2024-04-21T08:44:26.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.513+0000 7f976b04a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f976410fe60 con 0x7f9764107270 2024-04-21T08:44:26.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.513+0000 7f976b04a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9764110160 con 0x7f976406a840 2024-04-21T08:44:26.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.513+0000 7f97695e7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f976419ade0 0x7f976419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.513+0000 7f97695e7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f976419ade0 0x7f976419b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34806/0 (socket says 172.21.15.43:34806) 2024-04-21T08:44:26.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.513+0000 7f97695e7640 1 -- 172.21.15.43:0/2722367541 learned_addr learned my addr 172.21.15.43:0/2722367541 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:26.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.513+0000 7f9768de6640 1 --2- 172.21.15.43:0/2722367541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f976406a840 0x7f9764196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.513+0000 7f9763fff640 1 --2- 172.21.15.43:0/2722367541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9764107270 0x7f9764196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.513+0000 7f97695e7640 1 -- 172.21.15.43:0/2722367541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f976406a840 msgr2=0x7f9764196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:26.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.513+0000 7f97695e7640 1 --2- 172.21.15.43:0/2722367541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f976406a840 0x7f9764196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.513+0000 7f97695e7640 1 -- 172.21.15.43:0/2722367541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9764107270 msgr2=0x7f9764196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:26.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.513+0000 7f97695e7640 1 --2- 172.21.15.43:0/2722367541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9764107270 0x7f9764196a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.513+0000 7f97695e7640 1 -- 172.21.15.43:0/2722367541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976419b8d0 con 0x7f976419ade0 2024-04-21T08:44:26.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.513+0000 7f9763fff640 1 --2- 172.21.15.43:0/2722367541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9764107270 0x7f9764196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:26.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.513+0000 7f9768de6640 1 --2- 172.21.15.43:0/2722367541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f976406a840 0x7f9764196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:26.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.513+0000 7f97695e7640 1 --2- 172.21.15.43:0/2722367541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f976419ade0 0x7f976419b290 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f975c036040 tx=0x7f975c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:26.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.513+0000 7f9761ffb640 1 -- 172.21.15.43:0/2722367541 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c046070 con 0x7f976419ade0 2024-04-21T08:44:26.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.514+0000 7f976b04a640 1 -- 172.21.15.43:0/2722367541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f976419bb60 con 0x7f976419ade0 2024-04-21T08:44:26.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.514+0000 7f9761ffb640 1 -- 172.21.15.43:0/2722367541 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f975c037810 con 0x7f976419ade0 2024-04-21T08:44:26.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.514+0000 7f9761ffb640 1 -- 172.21.15.43:0/2722367541 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c004510 con 0x7f976419ade0 2024-04-21T08:44:26.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.514+0000 7f976b04a640 1 -- 172.21.15.43:0/2722367541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97641a3320 con 0x7f976419ade0 2024-04-21T08:44:26.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.516+0000 7f9761ffb640 1 -- 172.21.15.43:0/2722367541 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f975c03f040 con 0x7f976419ade0 2024-04-21T08:44:26.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.516+0000 7f976dc3e740 1 Processor -- start 2024-04-21T08:44:26.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.516+0000 7f976dc3e740 1 -- start start 2024-04-21T08:44:26.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.517+0000 7f976dc3e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b29a8e0 0x560b2b29acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.517+0000 7f976dc3e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b2b29b290 0x560b2b2a3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.517+0000 7f976dc3e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b2b2a3e10 0x560b2b2a6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.517+0000 7f976dc3e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560b2b1fef00 con 0x560b2b29a8e0 2024-04-21T08:44:26.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.517+0000 7f976dc3e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560b2b19c590 con 0x560b2b2a3e10 2024-04-21T08:44:26.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.517+0000 7f976dc3e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560b2b1faec0 con 0x560b2b29b290 2024-04-21T08:44:26.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.517+0000 7f9763fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b29a8e0 0x560b2b29acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.517+0000 7f9763fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b29a8e0 0x560b2b29acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34822/0 (socket says 172.21.15.43:34822) 2024-04-21T08:44:26.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.517+0000 7f9763fff640 1 -- 172.21.15.43:0/3657758768 learned_addr learned my addr 172.21.15.43:0/3657758768 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:26.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.517+0000 7f9761ffb640 1 --2- 172.21.15.43:0/2722367541 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f97540a3d20 0x7f97540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.517+0000 7f9768de6640 1 --2- 172.21.15.43:0/3657758768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b2b29b290 0x560b2b2a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.517+0000 7f9763fff640 1 --2- 172.21.15.43:0/3657758768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b2b2a3e10 0x560b2b2a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.517+0000 7f9763fff640 1 -- 172.21.15.43:0/3657758768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b2b29b290 msgr2=0x560b2b2a3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:26.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.517+0000 7f97695e7640 1 --2- 172.21.15.43:0/2722367541 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f97540a3d20 0x7f97540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.517+0000 7f9763fff640 1 --2- 172.21.15.43:0/3657758768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b2b29b290 0x560b2b2a3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.517+0000 7f9763fff640 1 -- 172.21.15.43:0/3657758768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b2b2a3e10 msgr2=0x560b2b2a6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.517+0000 7f9763fff640 1 --2- 172.21.15.43:0/3657758768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b2b2a3e10 0x560b2b2a6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.517+0000 7f9763fff640 1 -- 172.21.15.43:0/3657758768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b2afd97e0 con 0x560b2b29a8e0 2024-04-21T08:44:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.517+0000 7f9761ffb640 1 -- 172.21.15.43:0/2722367541 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f975c0f7f10 con 0x7f976419ade0 2024-04-21T08:44:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.517+0000 7f9763fff640 1 --2- 172.21.15.43:0/3657758768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b29a8e0 0x560b2b29acc0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f97500135d0 tx=0x7f975000b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ed2946bcb00c5a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.517+0000 7f974effd640 1 -- 172.21.15.43:0/3657758768 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750019030 con 0x560b2b29a8e0 2024-04-21T08:44:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.518+0000 7f97695e7640 1 --2- 172.21.15.43:0/2722367541 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f97540a3d20 0x7f97540a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f975c0f5890 tx=0x7f975c0b8f70 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.518+0000 7f974effd640 1 -- 172.21.15.43:0/3657758768 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97500035f0 con 0x560b2b29a8e0 2024-04-21T08:44:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.518+0000 7f974effd640 1 -- 172.21.15.43:0/3657758768 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750011720 con 0x560b2b29a8e0 2024-04-21T08:44:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.518+0000 7f976dc3e740 1 -- 172.21.15.43:0/3657758768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b29a8e0 msgr2=0x560b2b29acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.518+0000 7f976dc3e740 1 --2- 172.21.15.43:0/3657758768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b29a8e0 0x560b2b29acc0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f97500135d0 tx=0x7f975000b670 comp rx=0 tx=0).stop 2024-04-21T08:44:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.518+0000 7f976dc3e740 1 -- 172.21.15.43:0/3657758768 shutdown_connections 2024-04-21T08:44:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.518+0000 7f976dc3e740 1 --2- 172.21.15.43:0/3657758768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b2b2a3e10 0x560b2b2a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.518+0000 7f976dc3e740 1 --2- 172.21.15.43:0/3657758768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b2b29b290 0x560b2b2a3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.518+0000 7f976dc3e740 1 --2- 172.21.15.43:0/3657758768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b29a8e0 0x560b2b29acc0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.518+0000 7f976dc3e740 1 -- 172.21.15.43:0/3657758768 >> 172.21.15.43:0/3657758768 conn(0x560b2b294fb0 msgr2=0x560b2b295390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:26.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.518+0000 7f976dc3e740 1 -- 172.21.15.43:0/3657758768 shutdown_connections 2024-04-21T08:44:26.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.518+0000 7f976dc3e740 1 -- 172.21.15.43:0/3657758768 wait complete. 2024-04-21T08:44:26.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.519+0000 7f976dc3e740 1 -- start start 2024-04-21T08:44:26.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.519+0000 7f976dc3e740 1 Processor -- start 2024-04-21T08:44:26.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.519+0000 7f976dc3e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b2b29a8e0 0x560b2b2a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.519+0000 7f976dc3e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b2b29b290 0x560b2b2a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.519+0000 7f976dc3e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b2a3e10 0x560b2b2a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.519+0000 7f976dc3e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560b2b2a6d80 con 0x560b2b2a3e10 2024-04-21T08:44:26.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.519+0000 7f976dc3e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560b2b1b8580 con 0x560b2b29b290 2024-04-21T08:44:26.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.519+0000 7f976dc3e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560b2b2a6f00 con 0x560b2b29a8e0 2024-04-21T08:44:26.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.519+0000 7f976dc3e740 10 client.0 _ll_register_callbacks cb 0x560b2b1be650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:26.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.519+0000 7f9763fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b2a3e10 0x560b2b2a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.519+0000 7f9763fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b2a3e10 0x560b2b2a18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34824/0 (socket says 172.21.15.43:34824) 2024-04-21T08:44:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.519+0000 7f9763fff640 1 -- 172.21.15.43:0/299642320 learned_addr learned my addr 172.21.15.43:0/299642320 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.519+0000 7f9768de6640 1 --2- 172.21.15.43:0/299642320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b2b29b290 0x560b2b2a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.520+0000 7f9763fff640 1 --2- 172.21.15.43:0/299642320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b2b29a8e0 0x560b2b2a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.520+0000 7f9763fff640 1 -- 172.21.15.43:0/299642320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b2b29a8e0 msgr2=0x560b2b2a0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.520+0000 7f9763fff640 1 --2- 172.21.15.43:0/299642320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b2b29a8e0 0x560b2b2a0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.520+0000 7f9763fff640 1 -- 172.21.15.43:0/299642320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b2b29b290 msgr2=0x560b2b2a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.520+0000 7f9763fff640 1 --2- 172.21.15.43:0/299642320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b2b29b290 0x560b2b2a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.520+0000 7f9763fff640 1 -- 172.21.15.43:0/299642320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b2b039f70 con 0x560b2b2a3e10 2024-04-21T08:44:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.520+0000 7f9763fff640 1 --2- 172.21.15.43:0/299642320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b2a3e10 0x560b2b2a18e0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f975000b9a0 tx=0x7f9750005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360179f4ddd65a84 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.520+0000 7f9743fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.520+0000 7f9743fff640 1 -- 172.21.15.43:0/299642320 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750019030 con 0x560b2b2a3e10 2024-04-21T08:44:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.520+0000 7f9743fff640 1 -- 172.21.15.43:0/299642320 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f9750005320 con 0x560b2b2a3e10 2024-04-21T08:44:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.520+0000 7f976dc3e740 1 -- 172.21.15.43:0/299642320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97500035f0 con 0x560b2b2a3e10 2024-04-21T08:44:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.520+0000 7f9743fff640 1 -- 172.21.15.43:0/299642320 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9750033050 con 0x560b2b2a3e10 2024-04-21T08:44:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.520+0000 7f976dc3e740 10 client.17979 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.520+0000 7f976dc3e740 1 -- 172.21.15.43:0/299642320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560b2aed1170 con 0x560b2b2a3e10 2024-04-21T08:44:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.520+0000 7f9743fff640 1 -- 172.21.15.43:0/299642320 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97500035f0 con 0x560b2b2a3e10 2024-04-21T08:44:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.521+0000 7f9743fff640 1 -- 172.21.15.43:0/299642320 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f975002b020 con 0x560b2b2a3e10 2024-04-21T08:44:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.521+0000 7f976dc3e740 10 client.17979 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.521+0000 7f976dc3e740 10 client.17979 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.521+0000 7f976dc3e740 20 client.17979 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.521+0000 7f976dc3e740 1 --2- 172.21.15.43:0/299642320 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560b2b2b4340 0x560b2b2b6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.521+0000 7f976dc3e740 4 client.17979 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.521+0000 7f976dc3e740 1 -- 172.21.15.43:0/299642320 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b2b20c920 con 0x560b2b2b4340 2024-04-21T08:44:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.521+0000 7f9768de6640 1 --2- 172.21.15.43:0/299642320 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560b2b2b4340 0x560b2b2b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.521+0000 7f9743fff640 10 client.17979 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.521+0000 7f9768de6640 1 --2- 172.21.15.43:0/299642320 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560b2b2b4340 0x560b2b2b6730 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bd11e6a4d051189 server_cookie=19fd8f7dc3b950bc in_seq=0 out_seq=0 2024-04-21T08:44:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.522+0000 7f9743fff640 1 -- 172.21.15.43:0/299642320 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b2b20c920 con 0x560b2b2b4340 2024-04-21T08:44:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.522+0000 7f9743fff640 10 client.17979 handle_command_reply: tid=0 2024-04-21T08:44:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.522+0000 7f976dc3e740 1 client.17979 shutdown 2024-04-21T08:44:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.522+0000 7f976dc3e740 1 -- 172.21.15.43:0/299642320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b2a3e10 msgr2=0x560b2b2a18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.522+0000 7f976dc3e740 1 --2- 172.21.15.43:0/299642320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b2a3e10 0x560b2b2a18e0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f975000b9a0 tx=0x7f9750005d10 comp rx=0 tx=0).stop 2024-04-21T08:44:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.522+0000 7f976dc3e740 1 -- 172.21.15.43:0/299642320 shutdown_connections 2024-04-21T08:44:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.522+0000 7f976dc3e740 1 --2- 172.21.15.43:0/299642320 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560b2b2b4340 0x560b2b2b6730 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.522+0000 7f976dc3e740 1 --2- 172.21.15.43:0/299642320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b2a3e10 0x560b2b2a18e0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.522+0000 7f9743fff640 0 client.17979 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.522+0000 7f976dc3e740 1 --2- 172.21.15.43:0/299642320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b2b29b290 0x560b2b2a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.522+0000 7f976dc3e740 1 --2- 172.21.15.43:0/299642320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b2b29a8e0 0x560b2b2a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.522+0000 7f976dc3e740 1 -- 172.21.15.43:0/299642320 >> 172.21.15.43:0/299642320 conn(0x560b2b294fb0 msgr2=0x560b2b296d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.522+0000 7f976dc3e740 1 -- 172.21.15.43:0/299642320 shutdown_connections 2024-04-21T08:44:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.523+0000 7f976dc3e740 1 -- 172.21.15.43:0/299642320 wait complete. 2024-04-21T08:44:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.523+0000 7f976dc3e740 20 client.17979 trim_cache size 0 max 16384 2024-04-21T08:44:26.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.540+0000 7f976dc3e740 1 Processor -- start 2024-04-21T08:44:26.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.540+0000 7f976dc3e740 1 -- start start 2024-04-21T08:44:26.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.541+0000 7f976dc3e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b2b29a8e0 0x560b2b2a3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.541+0000 7f976dc3e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b29b290 0x560b2b2afc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.541+0000 7f976dc3e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b2b2a3e10 0x560b2b293e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.541+0000 7f976dc3e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f97641ad0b0 con 0x560b2b29b290 2024-04-21T08:44:26.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.541+0000 7f976dc3e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f97641aebf0 con 0x560b2b2a3e10 2024-04-21T08:44:26.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.541+0000 7f976dc3e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f97641ad780 con 0x560b2b29a8e0 2024-04-21T08:44:26.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.541+0000 7f9768de6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b29b290 0x560b2b2afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.541+0000 7f9768de6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b29b290 0x560b2b2afc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34836/0 (socket says 172.21.15.43:34836) 2024-04-21T08:44:26.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.541+0000 7f9768de6640 1 -- 172.21.15.43:0/2080569431 learned_addr learned my addr 172.21.15.43:0/2080569431 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.541+0000 7f9763fff640 1 --2- 172.21.15.43:0/2080569431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b2b29a8e0 0x560b2b2a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.541+0000 7f9763fff640 1 --2- 172.21.15.43:0/2080569431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b2b2a3e10 0x560b2b293e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.541+0000 7f9768de6640 1 -- 172.21.15.43:0/2080569431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b2b29a8e0 msgr2=0x560b2b2a3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.541+0000 7f9768de6640 1 --2- 172.21.15.43:0/2080569431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b2b29a8e0 0x560b2b2a3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.541+0000 7f9768de6640 1 -- 172.21.15.43:0/2080569431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b2b2a3e10 msgr2=0x560b2b293e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.541+0000 7f9768de6640 1 --2- 172.21.15.43:0/2080569431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b2b2a3e10 0x560b2b293e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.541+0000 7f9768de6640 1 -- 172.21.15.43:0/2080569431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b2b2958f0 con 0x560b2b29b290 2024-04-21T08:44:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.541+0000 7f9763fff640 1 --2- 172.21.15.43:0/2080569431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b2b29a8e0 0x560b2b2a3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.541+0000 7f9768de6640 1 --2- 172.21.15.43:0/2080569431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b29b290 0x560b2b2afc70 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x560b2b29b690 tx=0x7f97580149b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf2f73a92c0c288 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.541+0000 7f974e7fc640 1 -- 172.21.15.43:0/2080569431 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975801e430 con 0x560b2b29b290 2024-04-21T08:44:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.541+0000 7f974e7fc640 1 -- 172.21.15.43:0/2080569431 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9758005020 con 0x560b2b29b290 2024-04-21T08:44:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.541+0000 7f974e7fc640 1 -- 172.21.15.43:0/2080569431 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758004c70 con 0x560b2b29b290 2024-04-21T08:44:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.542+0000 7f976dc3e740 1 -- 172.21.15.43:0/2080569431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b29b290 msgr2=0x560b2b2afc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:26.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.542+0000 7f976dc3e740 1 --2- 172.21.15.43:0/2080569431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b29b290 0x560b2b2afc70 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x560b2b29b690 tx=0x7f97580149b0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.542+0000 7f976dc3e740 1 -- 172.21.15.43:0/2080569431 shutdown_connections 2024-04-21T08:44:26.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.542+0000 7f976dc3e740 1 --2- 172.21.15.43:0/2080569431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b2b2a3e10 0x560b2b293e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.542+0000 7f976dc3e740 1 --2- 172.21.15.43:0/2080569431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b29b290 0x560b2b2afc70 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.542+0000 7f976dc3e740 1 --2- 172.21.15.43:0/2080569431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b2b29a8e0 0x560b2b2a3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.542+0000 7f976dc3e740 1 -- 172.21.15.43:0/2080569431 >> 172.21.15.43:0/2080569431 conn(0x560b2b294fb0 msgr2=0x560b2b293780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:26.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.542+0000 7f976dc3e740 1 -- 172.21.15.43:0/2080569431 shutdown_connections 2024-04-21T08:44:26.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.542+0000 7f976dc3e740 1 -- 172.21.15.43:0/2080569431 wait complete. 2024-04-21T08:44:26.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.542+0000 7f976dc3e740 1 -- start start 2024-04-21T08:44:26.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.542+0000 7f976dc3e740 1 Processor -- start 2024-04-21T08:44:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.543+0000 7f976dc3e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b2b29a8e0 0x560b2b2a83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.543+0000 7f976dc3e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b2b29b290 0x560b2b2a8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.543+0000 7f976dc3e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b2a3e10 0x560b2b319600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.543+0000 7f976dc3e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f97641ae520 con 0x560b2b2a3e10 2024-04-21T08:44:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.543+0000 7f976dc3e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f97641ade50 con 0x560b2b29a8e0 2024-04-21T08:44:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.543+0000 7f976dc3e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f97641ab090 con 0x560b2b29b290 2024-04-21T08:44:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.543+0000 7f976dc3e740 10 client.0 _ll_register_callbacks cb 0x7f9750013400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.543+0000 7f9763fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b2a3e10 0x560b2b319600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.543+0000 7f9763fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b2a3e10 0x560b2b319600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34844/0 (socket says 172.21.15.43:34844) 2024-04-21T08:44:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.543+0000 7f9763fff640 1 -- 172.21.15.43:0/3182281596 learned_addr learned my addr 172.21.15.43:0/3182281596 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.543+0000 7f9768de6640 1 --2- 172.21.15.43:0/3182281596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b2b29b290 0x560b2b2a8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.543+0000 7f9763fff640 1 --2- 172.21.15.43:0/3182281596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b2b29a8e0 0x560b2b2a83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.543+0000 7f9763fff640 1 -- 172.21.15.43:0/3182281596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b2b29b290 msgr2=0x560b2b2a8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.543+0000 7f9763fff640 1 --2- 172.21.15.43:0/3182281596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b2b29b290 0x560b2b2a8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.543+0000 7f9763fff640 1 -- 172.21.15.43:0/3182281596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b2b29a8e0 msgr2=0x560b2b2a83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.543+0000 7f9763fff640 1 --2- 172.21.15.43:0/3182281596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b2b29a8e0 0x560b2b2a83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.543+0000 7f9763fff640 1 -- 172.21.15.43:0/3182281596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b2b319bf0 con 0x560b2b2a3e10 2024-04-21T08:44:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.543+0000 7f9763fff640 1 --2- 172.21.15.43:0/3182281596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b2a3e10 0x560b2b319600 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f9750004d70 tx=0x7f9750015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299fb3e0c139e3e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.543+0000 7f9743fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.544+0000 7f9743fff640 1 -- 172.21.15.43:0/3182281596 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975003e020 con 0x560b2b2a3e10 2024-04-21T08:44:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.544+0000 7f9743fff640 1 -- 172.21.15.43:0/3182281596 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f975000b630 con 0x560b2b2a3e10 2024-04-21T08:44:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.544+0000 7f976dc3e740 1 -- 172.21.15.43:0/3182281596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9758005020 con 0x560b2b2a3e10 2024-04-21T08:44:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.544+0000 7f9743fff640 1 -- 172.21.15.43:0/3182281596 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9750046050 con 0x560b2b2a3e10 2024-04-21T08:44:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.544+0000 7f976dc3e740 10 client.17985 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.544+0000 7f976dc3e740 1 -- 172.21.15.43:0/3182281596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560b2b31af10 con 0x560b2b2a3e10 2024-04-21T08:44:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.544+0000 7f9743fff640 1 -- 172.21.15.43:0/3182281596 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9758005020 con 0x560b2b2a3e10 2024-04-21T08:44:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.544+0000 7f9743fff640 1 -- 172.21.15.43:0/3182281596 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f975002b050 con 0x560b2b2a3e10 2024-04-21T08:44:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.544+0000 7f976dc3e740 10 client.17985 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.544+0000 7f976dc3e740 10 client.17985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.544+0000 7f976dc3e740 20 client.17985 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.544+0000 7f976dc3e740 1 --2- 172.21.15.43:0/3182281596 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560b2b31b5d0 0x560b2b31d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.544+0000 7f976dc3e740 4 client.17985 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:44:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.544+0000 7f976dc3e740 1 -- 172.21.15.43:0/3182281596 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560b2b206790 con 0x560b2b31b5d0 2024-04-21T08:44:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.545+0000 7f9768de6640 1 --2- 172.21.15.43:0/3182281596 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560b2b31b5d0 0x560b2b31d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.545+0000 7f9743fff640 10 client.17985 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.545+0000 7f9768de6640 1 --2- 172.21.15.43:0/3182281596 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560b2b31b5d0 0x560b2b31d9f0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fe3b6ab3e4effa0 server_cookie=3b689773b1bce3c5 in_seq=0 out_seq=0 2024-04-21T08:44:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.546+0000 7f9743fff640 1 -- 172.21.15.43:0/3182281596 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560b2b206790 con 0x560b2b31b5d0 2024-04-21T08:44:26.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.546+0000 7f9743fff640 10 client.17985 handle_command_reply: tid=0 2024-04-21T08:44:26.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.546+0000 7f976dc3e740 1 client.17985 shutdown 2024-04-21T08:44:26.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.546+0000 7f976dc3e740 1 -- 172.21.15.43:0/3182281596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b2a3e10 msgr2=0x560b2b319600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:26.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.546+0000 7f976dc3e740 1 --2- 172.21.15.43:0/3182281596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b2a3e10 0x560b2b319600 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f9750004d70 tx=0x7f9750015040 comp rx=0 tx=0).stop 2024-04-21T08:44:26.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.546+0000 7f976dc3e740 1 -- 172.21.15.43:0/3182281596 shutdown_connections 2024-04-21T08:44:26.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.546+0000 7f976dc3e740 1 --2- 172.21.15.43:0/3182281596 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560b2b31b5d0 0x560b2b31d9f0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.546+0000 7f976dc3e740 1 --2- 172.21.15.43:0/3182281596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b2b2a3e10 0x560b2b319600 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.546+0000 7f976dc3e740 1 --2- 172.21.15.43:0/3182281596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b2b29b290 0x560b2b2a8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.546+0000 7f976dc3e740 1 --2- 172.21.15.43:0/3182281596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b2b29a8e0 0x560b2b2a83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.546+0000 7f9743fff640 0 client.17985 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:26.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.546+0000 7f976dc3e740 1 -- 172.21.15.43:0/3182281596 >> 172.21.15.43:0/3182281596 conn(0x560b2b294fb0 msgr2=0x560b2b294010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:26.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.546+0000 7f976dc3e740 1 -- 172.21.15.43:0/3182281596 shutdown_connections 2024-04-21T08:44:26.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.546+0000 7f976dc3e740 1 -- 172.21.15.43:0/3182281596 wait complete. 2024-04-21T08:44:26.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.546+0000 7f976dc3e740 20 client.17985 trim_cache size 0 max 16384 2024-04-21T08:44:26.552 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:44:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:44:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "cb092025-3bea-4964-b540-ecb14a52ea97", 2024-04-21T08:44:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:44:26.552 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:44:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.547+0000 7f976b04a640 1 -- 172.21.15.43:0/2722367541 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f97540a3d20 msgr2=0x7f97540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.547+0000 7f976b04a640 1 --2- 172.21.15.43:0/2722367541 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f97540a3d20 0x7f97540a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f975c0f5890 tx=0x7f975c0b8f70 comp rx=0 tx=0).stop 2024-04-21T08:44:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.547+0000 7f976b04a640 1 -- 172.21.15.43:0/2722367541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f976419ade0 msgr2=0x7f976419b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.547+0000 7f976b04a640 1 --2- 172.21.15.43:0/2722367541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f976419ade0 0x7f976419b290 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f975c036040 tx=0x7f975c003ce0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.547+0000 7f976b04a640 1 -- 172.21.15.43:0/2722367541 shutdown_connections 2024-04-21T08:44:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.547+0000 7f976b04a640 1 --2- 172.21.15.43:0/2722367541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f976419ade0 0x7f976419b290 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.547+0000 7f976b04a640 1 --2- 172.21.15.43:0/2722367541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9764107270 0x7f9764196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.547+0000 7f976b04a640 1 --2- 172.21.15.43:0/2722367541 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f97540a3d20 0x7f97540a61e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.547+0000 7f976b04a640 1 --2- 172.21.15.43:0/2722367541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f976406a840 0x7f9764196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.547+0000 7f976b04a640 1 -- 172.21.15.43:0/2722367541 >> 172.21.15.43:0/2722367541 conn(0x7f9764077eb0 msgr2=0x7f97640fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.547+0000 7f976b04a640 1 -- 172.21.15.43:0/2722367541 shutdown_connections 2024-04-21T08:44:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.547+0000 7f976b04a640 1 -- 172.21.15.43:0/2722367541 wait complete. 2024-04-21T08:44:26.558 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:44:26.558 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:44:26.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.663+0000 7f4f89a43640 1 -- 172.21.15.43:0/1090796991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f84105dd0 msgr2=0x7f4f841061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:26.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.663+0000 7f4f89a43640 1 --2- 172.21.15.43:0/1090796991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f84105dd0 0x7f4f841061b0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f4f74009900 tx=0x7f4f7402bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:26.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.663+0000 7f4f89a43640 1 -- 172.21.15.43:0/1090796991 shutdown_connections 2024-04-21T08:44:26.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.663+0000 7f4f89a43640 1 --2- 172.21.15.43:0/1090796991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f84101a40 0x7f4f8410dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.663+0000 7f4f89a43640 1 --2- 172.21.15.43:0/1090796991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f841010a0 0x7f4f84101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.663+0000 7f4f89a43640 1 --2- 172.21.15.43:0/1090796991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f84105dd0 0x7f4f841061b0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.663+0000 7f4f89a43640 1 -- 172.21.15.43:0/1090796991 >> 172.21.15.43:0/1090796991 conn(0x7f4f84077c40 msgr2=0x7f4f84078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:26.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.663+0000 7f4f89a43640 1 -- 172.21.15.43:0/1090796991 shutdown_connections 2024-04-21T08:44:26.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.663+0000 7f4f89a43640 1 -- 172.21.15.43:0/1090796991 wait complete. 2024-04-21T08:44:26.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.664+0000 7f4f89a43640 1 Processor -- start 2024-04-21T08:44:26.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.664+0000 7f4f89a43640 1 -- start start 2024-04-21T08:44:26.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.664+0000 7f4f89a43640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f841010a0 0x7f4f8419e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.664+0000 7f4f89a43640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f84101a40 0x7f4f8419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.664+0000 7f4f89a43640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f84105dd0 0x7f4f84198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.664+0000 7f4f89a43640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4f841122d0 con 0x7f4f84101a40 2024-04-21T08:44:26.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.664+0000 7f4f89a43640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4f84112150 con 0x7f4f841010a0 2024-04-21T08:44:26.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.664+0000 7f4f89a43640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f84112450 con 0x7f4f84105dd0 2024-04-21T08:44:26.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.665+0000 7f4f827fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f84101a40 0x7f4f8419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.665+0000 7f4f827fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f84101a40 0x7f4f8419e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56992/0 (socket says 172.21.15.43:56992) 2024-04-21T08:44:26.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.665+0000 7f4f827fc640 1 -- 172.21.15.43:0/1121564601 learned_addr learned my addr 172.21.15.43:0/1121564601 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:26.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.665+0000 7f4f82ffd640 1 --2- 172.21.15.43:0/1121564601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f841010a0 0x7f4f8419e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.665+0000 7f4f837fe640 1 --2- 172.21.15.43:0/1121564601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f84105dd0 0x7f4f84198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.665+0000 7f4f827fc640 1 -- 172.21.15.43:0/1121564601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f84105dd0 msgr2=0x7f4f84198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.665+0000 7f4f827fc640 1 --2- 172.21.15.43:0/1121564601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f84105dd0 0x7f4f84198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.665+0000 7f4f827fc640 1 -- 172.21.15.43:0/1121564601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f841010a0 msgr2=0x7f4f8419e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.665+0000 7f4f827fc640 1 --2- 172.21.15.43:0/1121564601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f841010a0 0x7f4f8419e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.665+0000 7f4f827fc640 1 -- 172.21.15.43:0/1121564601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f84198d90 con 0x7f4f84101a40 2024-04-21T08:44:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.665+0000 7f4f827fc640 1 --2- 172.21.15.43:0/1121564601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f84101a40 0x7f4f8419e950 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800c970 tx=0x7f4f7800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.665+0000 7f4f88a41640 1 -- 172.21.15.43:0/1121564601 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78007bb0 con 0x7f4f84101a40 2024-04-21T08:44:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.665+0000 7f4f89a43640 1 -- 172.21.15.43:0/1121564601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f84199080 con 0x7f4f84101a40 2024-04-21T08:44:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.665+0000 7f4f88a41640 1 -- 172.21.15.43:0/1121564601 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f78007d50 con 0x7f4f84101a40 2024-04-21T08:44:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.665+0000 7f4f88a41640 1 -- 172.21.15.43:0/1121564601 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7800f450 con 0x7f4f84101a40 2024-04-21T08:44:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.665+0000 7f4f89a43640 1 -- 172.21.15.43:0/1121564601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f841a52d0 con 0x7f4f84101a40 2024-04-21T08:44:26.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.667+0000 7f4f88a41640 1 -- 172.21.15.43:0/1121564601 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4f78004f20 con 0x7f4f84101a40 2024-04-21T08:44:26.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.668+0000 7f4f8c638740 1 Processor -- start 2024-04-21T08:44:26.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.668+0000 7f4f8c638740 1 -- start start 2024-04-21T08:44:26.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.668+0000 7f4f8c638740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622474bef60 0x5622474bf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.668+0000 7f4f88a41640 1 --2- 172.21.15.43:0/1121564601 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4f6c0a3d20 0x7f4f6c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.668+0000 7f4f8c638740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622474bf910 0x5622474cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.668+0000 7f4f8c638740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474cce90 0x5622474cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.668+0000 7f4f8c638740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5622474241e0 con 0x5622474cce90 2024-04-21T08:44:26.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.668+0000 7f4f8c638740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5622473c1390 con 0x5622474bef60 2024-04-21T08:44:26.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.668+0000 7f4f8c638740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5622474201a0 con 0x5622474bf910 2024-04-21T08:44:26.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.668+0000 7f4f82ffd640 1 --2- 172.21.15.43:0/1121564601 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4f6c0a3d20 0x7f4f6c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.668+0000 7f4f837fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474cce90 0x5622474cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.668+0000 7f4f837fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474cce90 0x5622474cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56996/0 (socket says 172.21.15.43:56996) 2024-04-21T08:44:26.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.668+0000 7f4f837fe640 1 -- 172.21.15.43:0/2677126502 learned_addr learned my addr 172.21.15.43:0/2677126502 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:26.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.668+0000 7f4f88a41640 1 -- 172.21.15.43:0/1121564601 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f4f780d0990 con 0x7f4f84101a40 2024-04-21T08:44:26.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.669+0000 7f4f837fe640 1 --2- 172.21.15.43:0/2677126502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622474bef60 0x5622474bf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.669+0000 7f4f827fc640 1 --2- 172.21.15.43:0/2677126502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622474bf910 0x5622474cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.669+0000 7f4f837fe640 1 -- 172.21.15.43:0/2677126502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622474bf910 msgr2=0x5622474cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:26.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.669+0000 7f4f837fe640 1 --2- 172.21.15.43:0/2677126502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622474bf910 0x5622474cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.669+0000 7f4f837fe640 1 -- 172.21.15.43:0/2677126502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622474bef60 msgr2=0x5622474bf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:26.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.669+0000 7f4f837fe640 1 --2- 172.21.15.43:0/2677126502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622474bef60 0x5622474bf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.669+0000 7f4f837fe640 1 -- 172.21.15.43:0/2677126502 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622472007d0 con 0x5622474cce90 2024-04-21T08:44:26.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.669+0000 7f4f82ffd640 1 --2- 172.21.15.43:0/1121564601 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4f6c0a3d20 0x7f4f6c0a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4f74006010 tx=0x7f4f74005e40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:26.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.669+0000 7f4f837fe640 1 --2- 172.21.15.43:0/2677126502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474cce90 0x5622474cf280 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0138b0 tx=0x7f4f7c009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1596936a4b2a412 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:26.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.669+0000 7f4f69ffb640 1 -- 172.21.15.43:0/2677126502 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c019070 con 0x5622474cce90 2024-04-21T08:44:26.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.669+0000 7f4f69ffb640 1 -- 172.21.15.43:0/2677126502 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f7c011d90 con 0x5622474cce90 2024-04-21T08:44:26.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.669+0000 7f4f69ffb640 1 -- 172.21.15.43:0/2677126502 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c0043f0 con 0x5622474cce90 2024-04-21T08:44:26.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.670+0000 7f4f8c638740 1 -- 172.21.15.43:0/2677126502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474cce90 msgr2=0x5622474cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:26.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.670+0000 7f4f8c638740 1 --2- 172.21.15.43:0/2677126502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474cce90 0x5622474cf280 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0138b0 tx=0x7f4f7c009950 comp rx=0 tx=0).stop 2024-04-21T08:44:26.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.670+0000 7f4f8c638740 1 -- 172.21.15.43:0/2677126502 shutdown_connections 2024-04-21T08:44:26.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.670+0000 7f4f8c638740 1 --2- 172.21.15.43:0/2677126502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474cce90 0x5622474cf280 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.670+0000 7f4f8c638740 1 --2- 172.21.15.43:0/2677126502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622474bf910 0x5622474cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.670+0000 7f4f8c638740 1 --2- 172.21.15.43:0/2677126502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622474bef60 0x5622474bf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.670+0000 7f4f8c638740 1 -- 172.21.15.43:0/2677126502 >> 172.21.15.43:0/2677126502 conn(0x5622474b9cb0 msgr2=0x5622474ba090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:26.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.670+0000 7f4f8c638740 1 -- 172.21.15.43:0/2677126502 shutdown_connections 2024-04-21T08:44:26.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.670+0000 7f4f8c638740 1 -- 172.21.15.43:0/2677126502 wait complete. 2024-04-21T08:44:26.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.670+0000 7f4f8c638740 1 -- start start 2024-04-21T08:44:26.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.670+0000 7f4f8c638740 1 Processor -- start 2024-04-21T08:44:26.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.671+0000 7f4f8c638740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474bef60 0x5622474c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.671+0000 7f4f8c638740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622474bf910 0x5622474ca420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.671+0000 7f4f8c638740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622474cce90 0x5622474ca960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.671+0000 7f4f8c638740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5622474cfc50 con 0x5622474bef60 2024-04-21T08:44:26.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.671+0000 7f4f8c638740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5622473dd2f0 con 0x5622474cce90 2024-04-21T08:44:26.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.671+0000 7f4f8c638740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5622474cfdd0 con 0x5622474bf910 2024-04-21T08:44:26.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.671+0000 7f4f8c638740 10 client.0 _ll_register_callbacks cb 0x5622473ea580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:26.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.671+0000 7f4f82ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474bef60 0x5622474c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.671+0000 7f4f82ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474bef60 0x5622474c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56998/0 (socket says 172.21.15.43:56998) 2024-04-21T08:44:26.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.671+0000 7f4f82ffd640 1 -- 172.21.15.43:0/3841662973 learned_addr learned my addr 172.21.15.43:0/3841662973 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:26.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.671+0000 7f4f827fc640 1 --2- 172.21.15.43:0/3841662973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622474bf910 0x5622474ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.671+0000 7f4f837fe640 1 --2- 172.21.15.43:0/3841662973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622474cce90 0x5622474ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.671+0000 7f4f82ffd640 1 -- 172.21.15.43:0/3841662973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622474bf910 msgr2=0x5622474ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:26.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.671+0000 7f4f82ffd640 1 --2- 172.21.15.43:0/3841662973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622474bf910 0x5622474ca420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.671+0000 7f4f82ffd640 1 -- 172.21.15.43:0/3841662973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622474cce90 msgr2=0x5622474ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:26.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.671+0000 7f4f82ffd640 1 --2- 172.21.15.43:0/3841662973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622474cce90 0x5622474ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.671+0000 7f4f82ffd640 1 -- 172.21.15.43:0/3841662973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622471363f0 con 0x5622474bef60 2024-04-21T08:44:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.671+0000 7f4f827fc640 1 --2- 172.21.15.43:0/3841662973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622474bf910 0x5622474ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.671+0000 7f4f82ffd640 1 --2- 172.21.15.43:0/3841662973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474bef60 0x5622474c9ee0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f4f74009900 tx=0x7f4f74009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d5f9a5d89ad788 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.671+0000 7f4f62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.671+0000 7f4f62ffd640 1 -- 172.21.15.43:0/3841662973 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7404f020 con 0x5622474bef60 2024-04-21T08:44:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.671+0000 7f4f8c638740 1 -- 172.21.15.43:0/3841662973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f7c011d90 con 0x5622474bef60 2024-04-21T08:44:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.671+0000 7f4f62ffd640 1 -- 172.21.15.43:0/3841662973 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f4f740046c0 con 0x5622474bef60 2024-04-21T08:44:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.672+0000 7f4f62ffd640 1 -- 172.21.15.43:0/3841662973 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4f7405a050 con 0x5622474bef60 2024-04-21T08:44:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.672+0000 7f4f8c638740 10 client.17997 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.672+0000 7f4f8c638740 1 -- 172.21.15.43:0/3841662973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5622470f60f0 con 0x5622474bef60 2024-04-21T08:44:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.672+0000 7f4f62ffd640 1 -- 172.21.15.43:0/3841662973 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f7c011d90 con 0x5622474bef60 2024-04-21T08:44:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.672+0000 7f4f62ffd640 1 -- 172.21.15.43:0/3841662973 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4f74003b80 con 0x5622474bef60 2024-04-21T08:44:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.672+0000 7f4f8c638740 10 client.17997 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.672+0000 7f4f8c638740 10 client.17997 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.672+0000 7f4f8c638740 20 client.17997 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.672+0000 7f4f8c638740 1 --2- 172.21.15.43:0/3841662973 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5622474dcf80 0x5622474df410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.672+0000 7f4f8c638740 4 client.17997 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.672+0000 7f4f8c638740 1 -- 172.21.15.43:0/3841662973 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562247431c00 con 0x5622474dcf80 2024-04-21T08:44:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.672+0000 7f4f827fc640 1 --2- 172.21.15.43:0/3841662973 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5622474dcf80 0x5622474df410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.673+0000 7f4f62ffd640 10 client.17997 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.673+0000 7f4f827fc640 1 --2- 172.21.15.43:0/3841662973 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5622474dcf80 0x5622474df410 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91ca55c52e62fb08 server_cookie=72f1af8366aec4a9 in_seq=0 out_seq=0 2024-04-21T08:44:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.673+0000 7f4f62ffd640 1 -- 172.21.15.43:0/3841662973 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562247431c00 con 0x5622474dcf80 2024-04-21T08:44:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.673+0000 7f4f62ffd640 10 client.17997 handle_command_reply: tid=0 2024-04-21T08:44:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.673+0000 7f4f8c638740 1 client.17997 shutdown 2024-04-21T08:44:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.674+0000 7f4f8c638740 1 -- 172.21.15.43:0/3841662973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474bef60 msgr2=0x5622474c9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.674+0000 7f4f8c638740 1 --2- 172.21.15.43:0/3841662973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474bef60 0x5622474c9ee0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f4f74009900 tx=0x7f4f74009260 comp rx=0 tx=0).stop 2024-04-21T08:44:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.674+0000 7f4f8c638740 1 -- 172.21.15.43:0/3841662973 shutdown_connections 2024-04-21T08:44:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.674+0000 7f4f8c638740 1 --2- 172.21.15.43:0/3841662973 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5622474dcf80 0x5622474df410 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.674+0000 7f4f8c638740 1 --2- 172.21.15.43:0/3841662973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622474cce90 0x5622474ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.674+0000 7f4f8c638740 1 --2- 172.21.15.43:0/3841662973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622474bf910 0x5622474ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.674+0000 7f4f8c638740 1 --2- 172.21.15.43:0/3841662973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474bef60 0x5622474c9ee0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.674+0000 7f4f8c638740 1 -- 172.21.15.43:0/3841662973 >> 172.21.15.43:0/3841662973 conn(0x5622474b9cb0 msgr2=0x5622474bba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.674+0000 7f4f62ffd640 0 client.17997 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.674+0000 7f4f8c638740 1 -- 172.21.15.43:0/3841662973 shutdown_connections 2024-04-21T08:44:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.674+0000 7f4f8c638740 1 -- 172.21.15.43:0/3841662973 wait complete. 2024-04-21T08:44:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.674+0000 7f4f8c638740 20 client.17997 trim_cache size 0 max 16384 2024-04-21T08:44:26.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.691+0000 7f4f8c638740 1 Processor -- start 2024-04-21T08:44:26.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.691+0000 7f4f8c638740 1 -- start start 2024-04-21T08:44:26.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.692+0000 7f4f8c638740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622474bef60 0x5622474cc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.692+0000 7f4f8c638740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474bf910 0x5622474d8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.692+0000 7f4f8c638740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622474cce90 0x5622474e4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.692+0000 7f4f8c638740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4f841ad120 con 0x5622474bf910 2024-04-21T08:44:26.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.692+0000 7f4f8c638740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4f841aec60 con 0x5622474bef60 2024-04-21T08:44:26.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.692+0000 7f4f8c638740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f841ad7f0 con 0x5622474cce90 2024-04-21T08:44:26.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.692+0000 7f4f827fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474bf910 0x5622474d8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.692+0000 7f4f827fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474bf910 0x5622474d8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57000/0 (socket says 172.21.15.43:57000) 2024-04-21T08:44:26.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.692+0000 7f4f827fc640 1 -- 172.21.15.43:0/715952201 learned_addr learned my addr 172.21.15.43:0/715952201 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:26.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.692+0000 7f4f827fc640 1 -- 172.21.15.43:0/715952201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622474cce90 msgr2=0x5622474e4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:26.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.692+0000 7f4f827fc640 1 --2- 172.21.15.43:0/715952201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622474cce90 0x5622474e4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.692+0000 7f4f82ffd640 1 --2- 172.21.15.43:0/715952201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622474bef60 0x5622474cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.692+0000 7f4f827fc640 1 -- 172.21.15.43:0/715952201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622474bef60 msgr2=0x5622474cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.692+0000 7f4f827fc640 1 --2- 172.21.15.43:0/715952201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622474bef60 0x5622474cc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.692+0000 7f4f827fc640 1 -- 172.21.15.43:0/715952201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622474be670 con 0x5622474bf910 2024-04-21T08:44:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.692+0000 7f4f827fc640 1 --2- 172.21.15.43:0/715952201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474bf910 0x5622474d8b80 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f4f78098960 tx=0x7f4f780058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3bed30e3b2c8552 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.692+0000 7f4f697fa640 1 -- 172.21.15.43:0/715952201 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78085020 con 0x5622474bf910 2024-04-21T08:44:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.692+0000 7f4f697fa640 1 -- 172.21.15.43:0/715952201 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f7809cc00 con 0x5622474bf910 2024-04-21T08:44:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.692+0000 7f4f697fa640 1 -- 172.21.15.43:0/715952201 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7801d040 con 0x5622474bf910 2024-04-21T08:44:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.693+0000 7f4f8c638740 1 -- 172.21.15.43:0/715952201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474bf910 msgr2=0x5622474d8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.693+0000 7f4f8c638740 1 --2- 172.21.15.43:0/715952201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474bf910 0x5622474d8b80 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f4f78098960 tx=0x7f4f780058e0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.693+0000 7f4f8c638740 1 -- 172.21.15.43:0/715952201 shutdown_connections 2024-04-21T08:44:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.693+0000 7f4f8c638740 1 --2- 172.21.15.43:0/715952201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622474cce90 0x5622474e4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.693+0000 7f4f8c638740 1 --2- 172.21.15.43:0/715952201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474bf910 0x5622474d8b80 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.693+0000 7f4f8c638740 1 --2- 172.21.15.43:0/715952201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622474bef60 0x5622474cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.693+0000 7f4f8c638740 1 -- 172.21.15.43:0/715952201 >> 172.21.15.43:0/715952201 conn(0x5622474b9cb0 msgr2=0x5622474c87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.693+0000 7f4f8c638740 1 -- 172.21.15.43:0/715952201 shutdown_connections 2024-04-21T08:44:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.693+0000 7f4f8c638740 1 -- 172.21.15.43:0/715952201 wait complete. 2024-04-21T08:44:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.693+0000 7f4f8c638740 1 -- start start 2024-04-21T08:44:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.694+0000 7f4f8c638740 1 Processor -- start 2024-04-21T08:44:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.694+0000 7f4f8c638740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622474bef60 0x5622474d5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.694+0000 7f4f8c638740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474bf910 0x5622474c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.694+0000 7f4f8c638740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622474cce90 0x5622474c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.694+0000 7f4f8c638740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4f841ae590 con 0x5622474bf910 2024-04-21T08:44:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.694+0000 7f4f8c638740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4f841adec0 con 0x5622474cce90 2024-04-21T08:44:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.694+0000 7f4f8c638740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f841ab100 con 0x5622474bef60 2024-04-21T08:44:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.694+0000 7f4f8c638740 10 client.0 _ll_register_callbacks cb 0x7f4f7402c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.694+0000 7f4f827fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474bf910 0x5622474c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.694+0000 7f4f827fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474bf910 0x5622474c68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57008/0 (socket says 172.21.15.43:57008) 2024-04-21T08:44:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.694+0000 7f4f827fc640 1 -- 172.21.15.43:0/2151000253 learned_addr learned my addr 172.21.15.43:0/2151000253 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.694+0000 7f4f837fe640 1 --2- 172.21.15.43:0/2151000253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622474cce90 0x5622474c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.694+0000 7f4f82ffd640 1 --2- 172.21.15.43:0/2151000253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622474bef60 0x5622474d5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.694+0000 7f4f827fc640 1 -- 172.21.15.43:0/2151000253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622474bef60 msgr2=0x5622474d5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.694+0000 7f4f827fc640 1 --2- 172.21.15.43:0/2151000253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622474bef60 0x5622474d5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.694+0000 7f4f827fc640 1 -- 172.21.15.43:0/2151000253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622474cce90 msgr2=0x5622474c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.694+0000 7f4f827fc640 1 --2- 172.21.15.43:0/2151000253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622474cce90 0x5622474c6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.694+0000 7f4f827fc640 1 -- 172.21.15.43:0/2151000253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622474c5ae0 con 0x5622474bf910 2024-04-21T08:44:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.694+0000 7f4f827fc640 1 --2- 172.21.15.43:0/2151000253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474bf910 0x5622474c68c0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f4f7809b500 tx=0x7f4f780a5170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd4040531585c64 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.694+0000 7f4f62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.695+0000 7f4f62ffd640 1 -- 172.21.15.43:0/2151000253 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7801d030 con 0x5622474bf910 2024-04-21T08:44:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.695+0000 7f4f62ffd640 1 -- 172.21.15.43:0/2151000253 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f4f7809bb40 con 0x5622474bf910 2024-04-21T08:44:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.695+0000 7f4f8c638740 1 -- 172.21.15.43:0/2151000253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f7809cc00 con 0x5622474bf910 2024-04-21T08:44:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.695+0000 7f4f62ffd640 1 -- 172.21.15.43:0/2151000253 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4f78085070 con 0x5622474bf910 2024-04-21T08:44:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.695+0000 7f4f8c638740 10 client.18003 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.695+0000 7f4f8c638740 1 -- 172.21.15.43:0/2151000253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5622475433b0 con 0x5622474bf910 2024-04-21T08:44:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.695+0000 7f4f62ffd640 1 -- 172.21.15.43:0/2151000253 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f7809cc00 con 0x5622474bf910 2024-04-21T08:44:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.695+0000 7f4f62ffd640 1 -- 172.21.15.43:0/2151000253 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4f78005210 con 0x5622474bf910 2024-04-21T08:44:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.695+0000 7f4f8c638740 10 client.18003 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.695+0000 7f4f8c638740 10 client.18003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.695+0000 7f4f8c638740 20 client.18003 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.695+0000 7f4f8c638740 1 --2- 172.21.15.43:0/2151000253 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562247543b30 0x562247545f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.695+0000 7f4f8c638740 4 client.18003 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:44:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.695+0000 7f4f8c638740 1 -- 172.21.15.43:0/2151000253 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56224742ba70 con 0x562247543b30 2024-04-21T08:44:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.695+0000 7f4f82ffd640 1 --2- 172.21.15.43:0/2151000253 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562247543b30 0x562247545f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.696+0000 7f4f82ffd640 1 --2- 172.21.15.43:0/2151000253 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562247543b30 0x562247545f20 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77aefed2a4fd6501 server_cookie=a07004f6b2ee4483 in_seq=0 out_seq=0 2024-04-21T08:44:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.696+0000 7f4f62ffd640 10 client.18003 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.696+0000 7f4f62ffd640 1 -- 172.21.15.43:0/2151000253 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56224742ba70 con 0x562247543b30 2024-04-21T08:44:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.696+0000 7f4f62ffd640 10 client.18003 handle_command_reply: tid=0 2024-04-21T08:44:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.696+0000 7f4f8c638740 1 client.18003 shutdown 2024-04-21T08:44:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.696+0000 7f4f8c638740 1 -- 172.21.15.43:0/2151000253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474bf910 msgr2=0x5622474c68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.696+0000 7f4f8c638740 1 --2- 172.21.15.43:0/2151000253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474bf910 0x5622474c68c0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f4f7809b500 tx=0x7f4f780a5170 comp rx=0 tx=0).stop 2024-04-21T08:44:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.696+0000 7f4f8c638740 1 -- 172.21.15.43:0/2151000253 shutdown_connections 2024-04-21T08:44:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.696+0000 7f4f8c638740 1 --2- 172.21.15.43:0/2151000253 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562247543b30 0x562247545f20 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.696+0000 7f4f8c638740 1 --2- 172.21.15.43:0/2151000253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622474cce90 0x5622474c6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.696+0000 7f4f8c638740 1 --2- 172.21.15.43:0/2151000253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5622474bf910 0x5622474c68c0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.696+0000 7f4f62ffd640 0 client.18003 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.696+0000 7f4f8c638740 1 --2- 172.21.15.43:0/2151000253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5622474bef60 0x5622474d5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.696+0000 7f4f8c638740 1 -- 172.21.15.43:0/2151000253 >> 172.21.15.43:0/2151000253 conn(0x5622474b9cb0 msgr2=0x5622474bb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.696+0000 7f4f8c638740 1 -- 172.21.15.43:0/2151000253 shutdown_connections 2024-04-21T08:44:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.696+0000 7f4f8c638740 1 -- 172.21.15.43:0/2151000253 wait complete. 2024-04-21T08:44:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.696+0000 7f4f8c638740 20 client.18003 trim_cache size 0 max 16384 2024-04-21T08:44:26.703 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:44:26.703 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:44:26.703 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-21T08:44:26.703 INFO:teuthology.orchestra.run.smithi043.stdout: "cb092025-3bea-4964-b540-ecb14a52ea97": { 2024-04-21T08:44:26.703 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-21T08:44:26.703 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "cb092025-3bea-4964-b540-ecb14a52ea97", 2024-04-21T08:44:26.703 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-21T08:44:26.703 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:44:26.703 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:44:26.703 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:44:26.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.697+0000 7f4f89a43640 1 -- 172.21.15.43:0/1121564601 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4f6c0a3d20 msgr2=0x7f4f6c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:26.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.697+0000 7f4f89a43640 1 --2- 172.21.15.43:0/1121564601 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4f6c0a3d20 0x7f4f6c0a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4f74006010 tx=0x7f4f74005e40 comp rx=0 tx=0).stop 2024-04-21T08:44:26.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.697+0000 7f4f89a43640 1 -- 172.21.15.43:0/1121564601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f84101a40 msgr2=0x7f4f8419e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:26.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.697+0000 7f4f89a43640 1 --2- 172.21.15.43:0/1121564601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f84101a40 0x7f4f8419e950 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800c970 tx=0x7f4f7800ce30 comp rx=0 tx=0).stop 2024-04-21T08:44:26.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.697+0000 7f4f89a43640 1 -- 172.21.15.43:0/1121564601 shutdown_connections 2024-04-21T08:44:26.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.697+0000 7f4f89a43640 1 --2- 172.21.15.43:0/1121564601 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4f6c0a3d20 0x7f4f6c0a61e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.697+0000 7f4f89a43640 1 --2- 172.21.15.43:0/1121564601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f84105dd0 0x7f4f84198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.697+0000 7f4f89a43640 1 --2- 172.21.15.43:0/1121564601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f84101a40 0x7f4f8419e950 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.697+0000 7f4f89a43640 1 --2- 172.21.15.43:0/1121564601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f841010a0 0x7f4f8419e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.697+0000 7f4f89a43640 1 -- 172.21.15.43:0/1121564601 >> 172.21.15.43:0/1121564601 conn(0x7f4f84077c40 msgr2=0x7f4f840fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:26.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.697+0000 7f4f89a43640 1 -- 172.21.15.43:0/1121564601 shutdown_connections 2024-04-21T08:44:26.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:26.697+0000 7f4f89a43640 1 -- 172.21.15.43:0/1121564601 wait complete. 2024-04-21T08:44:26.707 INFO:tasks.cephfs.filesystem:scrub status for tag:cb092025-3bea-4964-b540-ecb14a52ea97 - {'path': '/', 'tag': 'cb092025-3bea-4964-b540-ecb14a52ea97', 'options': 'recursive,force'} 2024-04-21T08:44:28.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:28 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:44:28.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:28 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:44:28.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:28 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:44:28.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:28 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:44:28.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:28 smithi175 ceph-mon[31920]: pgmap v169: 129 pgs: 129 active+clean; 3.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-21T08:44:28.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:28 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:44:28.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:28 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:44:28.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:28 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:44:28.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:28 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:44:28.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:28 smithi112 ceph-mon[31085]: pgmap v169: 129 pgs: 129 active+clean; 3.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-21T08:44:28.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:28 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:44:28.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:28 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:44:28.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:28 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:44:28.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:28 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:44:28.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:28 smithi043 ceph-mon[27978]: pgmap v169: 129 pgs: 129 active+clean; 3.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-21T08:44:30.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:30 smithi175 ceph-mon[31920]: pgmap v170: 129 pgs: 129 active+clean; 3.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-21T08:44:30.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:30 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:44:30.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:30 smithi112 ceph-mon[31085]: pgmap v170: 129 pgs: 129 active+clean; 3.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-21T08:44:30.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:30 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:44:30.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:30 smithi043 ceph-mon[27978]: pgmap v170: 129 pgs: 129 active+clean; 3.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-21T08:44:30.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:30 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:44:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.702+0000 7f629e14b640 1 -- 172.21.15.43:0/1190884337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f629810e8a0 msgr2=0x7f6298110c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.702+0000 7f629e14b640 1 --2- 172.21.15.43:0/1190884337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f629810e8a0 0x7f6298110c90 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f6290009ca0 tx=0x7f629002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.702+0000 7f629e14b640 1 -- 172.21.15.43:0/1190884337 shutdown_connections 2024-04-21T08:44:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.702+0000 7f629e14b640 1 --2- 172.21.15.43:0/1190884337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f629810e8a0 0x7f6298110c90 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.702+0000 7f629e14b640 1 --2- 172.21.15.43:0/1190884337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f629806acc0 0x7f629806b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.702+0000 7f629e14b640 1 --2- 172.21.15.43:0/1190884337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6298102f00 0x7f629806a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.702+0000 7f629e14b640 1 -- 172.21.15.43:0/1190884337 >> 172.21.15.43:0/1190884337 conn(0x7f6298076c50 msgr2=0x7f6298077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.702+0000 7f629e14b640 1 -- 172.21.15.43:0/1190884337 shutdown_connections 2024-04-21T08:44:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.702+0000 7f629e14b640 1 -- 172.21.15.43:0/1190884337 wait complete. 2024-04-21T08:44:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.702+0000 7f629e14b640 1 Processor -- start 2024-04-21T08:44:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.703+0000 7f629e14b640 1 -- start start 2024-04-21T08:44:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.703+0000 7f629e14b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f629806acc0 0x7f62981a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.703+0000 7f629e14b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6298102f00 0x7f62981a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.703+0000 7f629e14b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f629810e8a0 0x7f629819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.703+0000 7f629e14b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6298114480 con 0x7f6298102f00 2024-04-21T08:44:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.703+0000 7f629e14b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6298114300 con 0x7f629806acc0 2024-04-21T08:44:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.703+0000 7f629e14b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6298114600 con 0x7f629810e8a0 2024-04-21T08:44:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.703+0000 7f6296ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6298102f00 0x7f62981a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.703+0000 7f6296ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6298102f00 0x7f62981a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57034/0 (socket says 172.21.15.43:57034) 2024-04-21T08:44:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.703+0000 7f6296ffd640 1 -- 172.21.15.43:0/3483801726 learned_addr learned my addr 172.21.15.43:0/3483801726 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.703+0000 7f62977fe640 1 --2- 172.21.15.43:0/3483801726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f629806acc0 0x7f62981a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.703+0000 7f6297fff640 1 --2- 172.21.15.43:0/3483801726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f629810e8a0 0x7f629819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.703+0000 7f6296ffd640 1 -- 172.21.15.43:0/3483801726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f629810e8a0 msgr2=0x7f629819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.703+0000 7f6296ffd640 1 --2- 172.21.15.43:0/3483801726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f629810e8a0 0x7f629819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.703+0000 7f6296ffd640 1 -- 172.21.15.43:0/3483801726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f629806acc0 msgr2=0x7f62981a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.703+0000 7f6296ffd640 1 --2- 172.21.15.43:0/3483801726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f629806acc0 0x7f62981a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.703+0000 7f6296ffd640 1 -- 172.21.15.43:0/3483801726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f629819afa0 con 0x7f6298102f00 2024-04-21T08:44:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.703+0000 7f6296ffd640 1 --2- 172.21.15.43:0/3483801726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6298102f00 0x7f62981a0b60 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f628c00d770 tx=0x7f628c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.704+0000 7f6294ff9640 1 -- 172.21.15.43:0/3483801726 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c017070 con 0x7f6298102f00 2024-04-21T08:44:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.704+0000 7f629e14b640 1 -- 172.21.15.43:0/3483801726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f629819b290 con 0x7f6298102f00 2024-04-21T08:44:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.704+0000 7f6294ff9640 1 -- 172.21.15.43:0/3483801726 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f628c0054d0 con 0x7f6298102f00 2024-04-21T08:44:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.704+0000 7f6294ff9640 1 -- 172.21.15.43:0/3483801726 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c004c30 con 0x7f6298102f00 2024-04-21T08:44:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.704+0000 7f629e14b640 1 -- 172.21.15.43:0/3483801726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62981a75a0 con 0x7f6298102f00 2024-04-21T08:44:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.707+0000 7f62a0e13740 1 Processor -- start 2024-04-21T08:44:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.707+0000 7f6294ff9640 1 -- 172.21.15.43:0/3483801726 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f628c004e10 con 0x7f6298102f00 2024-04-21T08:44:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.707+0000 7f62a0e13740 1 -- start start 2024-04-21T08:44:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.707+0000 7f62a0e13740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc40228d0 0x55fbc4021620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.707+0000 7f62a0e13740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbc4021b60 0x55fbc4021fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.707+0000 7f62a0e13740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbc402b0e0 0x55fbc402d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.707+0000 7f62a0e13740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fbc3f868d0 con 0x55fbc40228d0 2024-04-21T08:44:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.707+0000 7f62a0e13740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fbc3f23e80 con 0x55fbc4021b60 2024-04-21T08:44:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.707+0000 7f62a0e13740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fbc3f82890 con 0x55fbc402b0e0 2024-04-21T08:44:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.707+0000 7f6297fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc40228d0 0x55fbc4021620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.707+0000 7f6297fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc40228d0 0x55fbc4021620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57046/0 (socket says 172.21.15.43:57046) 2024-04-21T08:44:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.707+0000 7f6297fff640 1 -- 172.21.15.43:0/3395436284 learned_addr learned my addr 172.21.15.43:0/3395436284 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.707+0000 7f62977fe640 1 --2- 172.21.15.43:0/3395436284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbc4021b60 0x55fbc4021fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.708+0000 7f6296ffd640 1 --2- 172.21.15.43:0/3395436284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbc402b0e0 0x55fbc402d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.708+0000 7f6294ff9640 1 --2- 172.21.15.43:0/3483801726 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f62800a3dc0 0x7f62800a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.708+0000 7f6297fff640 1 -- 172.21.15.43:0/3395436284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbc402b0e0 msgr2=0x55fbc402d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.708+0000 7f6297fff640 1 --2- 172.21.15.43:0/3395436284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbc402b0e0 0x55fbc402d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.708+0000 7f6297fff640 1 -- 172.21.15.43:0/3395436284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbc4021b60 msgr2=0x55fbc4021fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.708+0000 7f6297fff640 1 --2- 172.21.15.43:0/3395436284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbc4021b60 0x55fbc4021fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.708+0000 7f6297fff640 1 -- 172.21.15.43:0/3395436284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbc3f45320 con 0x55fbc40228d0 2024-04-21T08:44:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.708+0000 7f6296ffd640 1 --2- 172.21.15.43:0/3395436284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbc402b0e0 0x55fbc402d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.708+0000 7f6294ff9640 1 -- 172.21.15.43:0/3483801726 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f628c0d07b0 con 0x7f6298102f00 2024-04-21T08:44:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.708+0000 7f6297fff640 1 --2- 172.21.15.43:0/3483801726 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f62800a3dc0 0x7f62800a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.708+0000 7f6297fff640 1 --2- 172.21.15.43:0/3395436284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc40228d0 0x55fbc4021620 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f6290009ca0 tx=0x7f6290009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f1468978d91635 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.708+0000 7f627dffb640 1 -- 172.21.15.43:0/3395436284 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6290045030 con 0x55fbc40228d0 2024-04-21T08:44:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.708+0000 7f627dffb640 1 -- 172.21.15.43:0/3395436284 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6290003e20 con 0x55fbc40228d0 2024-04-21T08:44:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.708+0000 7f627dffb640 1 -- 172.21.15.43:0/3395436284 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6290039040 con 0x55fbc40228d0 2024-04-21T08:44:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.709+0000 7f62a0e13740 1 -- 172.21.15.43:0/3395436284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc40228d0 msgr2=0x55fbc4021620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.709+0000 7f6297fff640 1 --2- 172.21.15.43:0/3483801726 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f62800a3dc0 0x7f62800a6280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6290049000 tx=0x7f629004a000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.709+0000 7f62a0e13740 1 --2- 172.21.15.43:0/3395436284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc40228d0 0x55fbc4021620 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f6290009ca0 tx=0x7f6290009930 comp rx=0 tx=0).stop 2024-04-21T08:44:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.709+0000 7f62a0e13740 1 -- 172.21.15.43:0/3395436284 shutdown_connections 2024-04-21T08:44:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.709+0000 7f62a0e13740 1 --2- 172.21.15.43:0/3395436284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbc402b0e0 0x55fbc402d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.709+0000 7f62a0e13740 1 --2- 172.21.15.43:0/3395436284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbc4021b60 0x55fbc4021fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.709+0000 7f62a0e13740 1 --2- 172.21.15.43:0/3395436284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc40228d0 0x55fbc4021620 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.709+0000 7f62a0e13740 1 -- 172.21.15.43:0/3395436284 >> 172.21.15.43:0/3395436284 conn(0x55fbc3fe8c90 msgr2=0x55fbc401c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.709+0000 7f62a0e13740 1 -- 172.21.15.43:0/3395436284 shutdown_connections 2024-04-21T08:44:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.709+0000 7f62a0e13740 1 -- 172.21.15.43:0/3395436284 wait complete. 2024-04-21T08:44:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.710+0000 7f62a0e13740 1 -- start start 2024-04-21T08:44:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.710+0000 7f62a0e13740 1 Processor -- start 2024-04-21T08:44:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.710+0000 7f62a0e13740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbc4021b60 0x55fbc402a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.710+0000 7f62a0e13740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbc40228d0 0x55fbc4027d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.710+0000 7f62a0e13740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc402b0e0 0x55fbc40282b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.710+0000 7f62a0e13740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fbc402de10 con 0x55fbc402b0e0 2024-04-21T08:44:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.710+0000 7f62a0e13740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fbc3f0eeb0 con 0x55fbc4021b60 2024-04-21T08:44:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.710+0000 7f62a0e13740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fbc402df90 con 0x55fbc40228d0 2024-04-21T08:44:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.710+0000 7f62a0e13740 10 client.0 _ll_register_callbacks cb 0x55fbc3f3bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.710+0000 7f6297fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc402b0e0 0x55fbc40282b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.710+0000 7f6297fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc402b0e0 0x55fbc40282b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57050/0 (socket says 172.21.15.43:57050) 2024-04-21T08:44:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.710+0000 7f6297fff640 1 -- 172.21.15.43:0/2261016398 learned_addr learned my addr 172.21.15.43:0/2261016398 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.710+0000 7f62977fe640 1 --2- 172.21.15.43:0/2261016398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbc4021b60 0x55fbc402a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.710+0000 7f6296ffd640 1 --2- 172.21.15.43:0/2261016398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbc40228d0 0x55fbc4027d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.711+0000 7f6297fff640 1 -- 172.21.15.43:0/2261016398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbc40228d0 msgr2=0x55fbc4027d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.711+0000 7f6297fff640 1 --2- 172.21.15.43:0/2261016398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbc40228d0 0x55fbc4027d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.711+0000 7f6297fff640 1 -- 172.21.15.43:0/2261016398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbc4021b60 msgr2=0x55fbc402a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.711+0000 7f6297fff640 1 --2- 172.21.15.43:0/2261016398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbc4021b60 0x55fbc402a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.711+0000 7f6297fff640 1 -- 172.21.15.43:0/2261016398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbc3c99210 con 0x55fbc402b0e0 2024-04-21T08:44:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.711+0000 7f6297fff640 1 --2- 172.21.15.43:0/2261016398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc402b0e0 0x55fbc40282b0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f62900042a0 tx=0x7f6290032a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49b8678cac7906d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.711+0000 7f6276ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.711+0000 7f6276ffd640 1 -- 172.21.15.43:0/2261016398 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6290058060 con 0x55fbc402b0e0 2024-04-21T08:44:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.711+0000 7f6276ffd640 1 -- 172.21.15.43:0/2261016398 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f62900043e0 con 0x55fbc402b0e0 2024-04-21T08:44:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.711+0000 7f62a0e13740 1 -- 172.21.15.43:0/2261016398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6290003e20 con 0x55fbc402b0e0 2024-04-21T08:44:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.711+0000 7f6276ffd640 1 -- 172.21.15.43:0/2261016398 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f629005f050 con 0x55fbc402b0e0 2024-04-21T08:44:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.711+0000 7f62a0e13740 10 client.18015 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.711+0000 7f62a0e13740 1 -- 172.21.15.43:0/2261016398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fbc3d65620 con 0x55fbc402b0e0 2024-04-21T08:44:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.711+0000 7f6276ffd640 1 -- 172.21.15.43:0/2261016398 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6290003e20 con 0x55fbc402b0e0 2024-04-21T08:44:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.712+0000 7f6276ffd640 1 -- 172.21.15.43:0/2261016398 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f629003fab0 con 0x55fbc402b0e0 2024-04-21T08:44:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.712+0000 7f62a0e13740 10 client.18015 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.712+0000 7f62a0e13740 10 client.18015 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.712+0000 7f62a0e13740 20 client.18015 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.712+0000 7f62a0e13740 1 --2- 172.21.15.43:0/2261016398 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fbc403ae80 0x55fbc403d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.712+0000 7f62a0e13740 4 client.18015 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.712+0000 7f62a0e13740 1 -- 172.21.15.43:0/2261016398 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbc3f9a880 con 0x55fbc403ae80 2024-04-21T08:44:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.712+0000 7f62977fe640 1 --2- 172.21.15.43:0/2261016398 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fbc403ae80 0x55fbc403d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.712+0000 7f6276ffd640 10 client.18015 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.712+0000 7f62977fe640 1 --2- 172.21.15.43:0/2261016398 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fbc403ae80 0x55fbc403d270 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd974d40ba486e0c server_cookie=7a9842dd5a3c7286 in_seq=0 out_seq=0 2024-04-21T08:44:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.713+0000 7f6276ffd640 1 -- 172.21.15.43:0/2261016398 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fbc3f9a880 con 0x55fbc403ae80 2024-04-21T08:44:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.713+0000 7f6276ffd640 10 client.18015 handle_command_reply: tid=0 2024-04-21T08:44:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.713+0000 7f62a0e13740 1 client.18015 shutdown 2024-04-21T08:44:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.713+0000 7f62a0e13740 1 -- 172.21.15.43:0/2261016398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc402b0e0 msgr2=0x55fbc40282b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.713+0000 7f62a0e13740 1 --2- 172.21.15.43:0/2261016398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc402b0e0 0x55fbc40282b0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f62900042a0 tx=0x7f6290032a50 comp rx=0 tx=0).stop 2024-04-21T08:44:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.713+0000 7f62a0e13740 1 -- 172.21.15.43:0/2261016398 shutdown_connections 2024-04-21T08:44:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.713+0000 7f62a0e13740 1 --2- 172.21.15.43:0/2261016398 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fbc403ae80 0x55fbc403d270 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.713+0000 7f62a0e13740 1 --2- 172.21.15.43:0/2261016398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc402b0e0 0x55fbc40282b0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.713+0000 7f62a0e13740 1 --2- 172.21.15.43:0/2261016398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbc40228d0 0x55fbc4027d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.713+0000 7f62a0e13740 1 --2- 172.21.15.43:0/2261016398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbc4021b60 0x55fbc402a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.713+0000 7f6276ffd640 0 client.18015 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.713+0000 7f62a0e13740 1 -- 172.21.15.43:0/2261016398 >> 172.21.15.43:0/2261016398 conn(0x55fbc3fe8c90 msgr2=0x55fbc401e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:30.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.713+0000 7f62a0e13740 1 -- 172.21.15.43:0/2261016398 shutdown_connections 2024-04-21T08:44:30.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.714+0000 7f62a0e13740 1 -- 172.21.15.43:0/2261016398 wait complete. 2024-04-21T08:44:30.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.714+0000 7f62a0e13740 20 client.18015 trim_cache size 0 max 16384 2024-04-21T08:44:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.728+0000 7f62a0e13740 1 Processor -- start 2024-04-21T08:44:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.729+0000 7f62a0e13740 1 -- start start 2024-04-21T08:44:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.729+0000 7f62a0e13740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbc3fe8c90 0x55fbc4021120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.729+0000 7f62a0e13740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbc4021b60 0x55fbc402a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.729+0000 7f62a0e13740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc40228d0 0x55fbc402a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.729+0000 7f62a0e13740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f62981b1530 con 0x55fbc40228d0 2024-04-21T08:44:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.729+0000 7f62a0e13740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f62981b3070 con 0x55fbc3fe8c90 2024-04-21T08:44:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.729+0000 7f62a0e13740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f62981b1c00 con 0x55fbc4021b60 2024-04-21T08:44:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.729+0000 7f6297fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc40228d0 0x55fbc402a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.729+0000 7f6297fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc40228d0 0x55fbc402a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57052/0 (socket says 172.21.15.43:57052) 2024-04-21T08:44:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.729+0000 7f6297fff640 1 -- 172.21.15.43:0/246026755 learned_addr learned my addr 172.21.15.43:0/246026755 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.729+0000 7f62977fe640 1 --2- 172.21.15.43:0/246026755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbc3fe8c90 0x55fbc4021120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.729+0000 7f6296ffd640 1 --2- 172.21.15.43:0/246026755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbc4021b60 0x55fbc402a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.729+0000 7f6297fff640 1 -- 172.21.15.43:0/246026755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbc4021b60 msgr2=0x55fbc402a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.729+0000 7f6297fff640 1 --2- 172.21.15.43:0/246026755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbc4021b60 0x55fbc402a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.729+0000 7f6297fff640 1 -- 172.21.15.43:0/246026755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbc3fe8c90 msgr2=0x55fbc4021120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.729+0000 7f6297fff640 1 --2- 172.21.15.43:0/246026755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbc3fe8c90 0x55fbc4021120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.729+0000 7f6297fff640 1 -- 172.21.15.43:0/246026755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbc4020e30 con 0x55fbc40228d0 2024-04-21T08:44:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.729+0000 7f6297fff640 1 --2- 172.21.15.43:0/246026755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc40228d0 0x55fbc402a9e0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f6290002230 tx=0x7f6290009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678470a231ed2741 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.729+0000 7f627d7fa640 1 -- 172.21.15.43:0/246026755 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6290058070 con 0x55fbc40228d0 2024-04-21T08:44:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.730+0000 7f627d7fa640 1 -- 172.21.15.43:0/246026755 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6290032de0 con 0x55fbc40228d0 2024-04-21T08:44:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.730+0000 7f627d7fa640 1 -- 172.21.15.43:0/246026755 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6290039050 con 0x55fbc40228d0 2024-04-21T08:44:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.730+0000 7f62a0e13740 1 -- 172.21.15.43:0/246026755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc40228d0 msgr2=0x55fbc402a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.730+0000 7f62a0e13740 1 --2- 172.21.15.43:0/246026755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc40228d0 0x55fbc402a9e0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f6290002230 tx=0x7f6290009360 comp rx=0 tx=0).stop 2024-04-21T08:44:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.730+0000 7f62a0e13740 1 -- 172.21.15.43:0/246026755 shutdown_connections 2024-04-21T08:44:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.730+0000 7f62a0e13740 1 --2- 172.21.15.43:0/246026755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc40228d0 0x55fbc402a9e0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.730+0000 7f62a0e13740 1 --2- 172.21.15.43:0/246026755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbc4021b60 0x55fbc402a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.730+0000 7f62a0e13740 1 --2- 172.21.15.43:0/246026755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbc3fe8c90 0x55fbc4021120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.730+0000 7f62a0e13740 1 -- 172.21.15.43:0/246026755 >> 172.21.15.43:0/246026755 conn(0x55fbc4028410 msgr2=0x55fbc4026640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.730+0000 7f62a0e13740 1 -- 172.21.15.43:0/246026755 shutdown_connections 2024-04-21T08:44:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.730+0000 7f62a0e13740 1 -- 172.21.15.43:0/246026755 wait complete. 2024-04-21T08:44:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.730+0000 7f62a0e13740 1 -- start start 2024-04-21T08:44:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.731+0000 7f62a0e13740 1 Processor -- start 2024-04-21T08:44:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.731+0000 7f62a0e13740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc3fe8c90 0x55fbc403dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.731+0000 7f62a0e13740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbc4021b60 0x55fbc403e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.731+0000 7f62a0e13740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbc40228d0 0x55fbc403e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.731+0000 7f62a0e13740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f62981b29a0 con 0x55fbc3fe8c90 2024-04-21T08:44:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.731+0000 7f62a0e13740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f62981b22d0 con 0x55fbc4021b60 2024-04-21T08:44:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.731+0000 7f62a0e13740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f62981af510 con 0x55fbc40228d0 2024-04-21T08:44:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.731+0000 7f62a0e13740 10 client.0 _ll_register_callbacks cb 0x7f628800c2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.731+0000 7f62977fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc3fe8c90 0x55fbc403dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.731+0000 7f62977fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc3fe8c90 0x55fbc403dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57064/0 (socket says 172.21.15.43:57064) 2024-04-21T08:44:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.731+0000 7f62977fe640 1 -- 172.21.15.43:0/3607612774 learned_addr learned my addr 172.21.15.43:0/3607612774 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.731+0000 7f6297fff640 1 --2- 172.21.15.43:0/3607612774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbc40228d0 0x55fbc403e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.731+0000 7f6296ffd640 1 --2- 172.21.15.43:0/3607612774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbc4021b60 0x55fbc403e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.731+0000 7f62977fe640 1 -- 172.21.15.43:0/3607612774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbc40228d0 msgr2=0x55fbc403e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.731+0000 7f62977fe640 1 --2- 172.21.15.43:0/3607612774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbc40228d0 0x55fbc403e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.731+0000 7f62977fe640 1 -- 172.21.15.43:0/3607612774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbc4021b60 msgr2=0x55fbc403e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.731+0000 7f62977fe640 1 --2- 172.21.15.43:0/3607612774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbc4021b60 0x55fbc403e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.731+0000 7f62977fe640 1 -- 172.21.15.43:0/3607612774 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbc401eb50 con 0x55fbc3fe8c90 2024-04-21T08:44:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.731+0000 7f6296ffd640 1 --2- 172.21.15.43:0/3607612774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbc4021b60 0x55fbc403e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.731+0000 7f6297fff640 1 --2- 172.21.15.43:0/3607612774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbc40228d0 0x55fbc403e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.732+0000 7f62977fe640 1 --2- 172.21.15.43:0/3607612774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc3fe8c90 0x55fbc403dcb0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f6288004700 tx=0x7f628800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90e12d691b7f01a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.732+0000 7f6276ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.732+0000 7f6276ffd640 1 -- 172.21.15.43:0/3607612774 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628801c3b0 con 0x55fbc3fe8c90 2024-04-21T08:44:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.732+0000 7f6276ffd640 1 -- 172.21.15.43:0/3607612774 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f628801c550 con 0x55fbc3fe8c90 2024-04-21T08:44:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.732+0000 7f62a0e13740 1 -- 172.21.15.43:0/3607612774 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6290032de0 con 0x55fbc3fe8c90 2024-04-21T08:44:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.732+0000 7f6276ffd640 1 -- 172.21.15.43:0/3607612774 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f628802c050 con 0x55fbc3fe8c90 2024-04-21T08:44:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.732+0000 7f62a0e13740 10 client.18021 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.732+0000 7f62a0e13740 1 -- 172.21.15.43:0/3607612774 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fbc40a11d0 con 0x55fbc3fe8c90 2024-04-21T08:44:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.732+0000 7f6276ffd640 1 -- 172.21.15.43:0/3607612774 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6290032de0 con 0x55fbc3fe8c90 2024-04-21T08:44:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.732+0000 7f6276ffd640 1 -- 172.21.15.43:0/3607612774 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f62880243c0 con 0x55fbc3fe8c90 2024-04-21T08:44:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.733+0000 7f62a0e13740 10 client.18021 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.733+0000 7f62a0e13740 10 client.18021 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.733+0000 7f62a0e13740 20 client.18021 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.733+0000 7f62a0e13740 1 --2- 172.21.15.43:0/3607612774 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fbc402b0e0 0x55fbc40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.733+0000 7f62a0e13740 4 client.18021 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:44:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.733+0000 7f62a0e13740 1 -- 172.21.15.43:0/3607612774 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fbc3f263d0 con 0x55fbc402b0e0 2024-04-21T08:44:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.733+0000 7f6296ffd640 1 --2- 172.21.15.43:0/3607612774 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fbc402b0e0 0x55fbc40a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.733+0000 7f6296ffd640 1 --2- 172.21.15.43:0/3607612774 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fbc402b0e0 0x55fbc40a3990 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb0ddc0b299e4ae3 server_cookie=a2c14d3f766a86fe in_seq=0 out_seq=0 2024-04-21T08:44:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.733+0000 7f6276ffd640 10 client.18021 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.746+0000 7f6276ffd640 1 -- 172.21.15.43:0/3607612774 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fbc3f263d0 con 0x55fbc402b0e0 2024-04-21T08:44:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.746+0000 7f6276ffd640 10 client.18021 handle_command_reply: tid=0 2024-04-21T08:44:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.747+0000 7f62a0e13740 1 client.18021 shutdown 2024-04-21T08:44:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.747+0000 7f62a0e13740 1 -- 172.21.15.43:0/3607612774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc3fe8c90 msgr2=0x55fbc403dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.747+0000 7f62a0e13740 1 --2- 172.21.15.43:0/3607612774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc3fe8c90 0x55fbc403dcb0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f6288004700 tx=0x7f628800cf50 comp rx=0 tx=0).stop 2024-04-21T08:44:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.747+0000 7f62a0e13740 1 -- 172.21.15.43:0/3607612774 shutdown_connections 2024-04-21T08:44:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.747+0000 7f62a0e13740 1 --2- 172.21.15.43:0/3607612774 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fbc402b0e0 0x55fbc40a3990 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.747+0000 7f62a0e13740 1 --2- 172.21.15.43:0/3607612774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbc40228d0 0x55fbc403e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.747+0000 7f62a0e13740 1 --2- 172.21.15.43:0/3607612774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbc4021b60 0x55fbc403e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.747+0000 7f62a0e13740 1 --2- 172.21.15.43:0/3607612774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbc3fe8c90 0x55fbc403dcb0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.747+0000 7f6276ffd640 0 client.18021 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.747+0000 7f62a0e13740 1 -- 172.21.15.43:0/3607612774 >> 172.21.15.43:0/3607612774 conn(0x55fbc4028410 msgr2=0x55fbc401e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.747+0000 7f62a0e13740 1 -- 172.21.15.43:0/3607612774 shutdown_connections 2024-04-21T08:44:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.747+0000 7f62a0e13740 1 -- 172.21.15.43:0/3607612774 wait complete. 2024-04-21T08:44:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.747+0000 7f62a0e13740 20 client.18021 trim_cache size 0 max 16384 2024-04-21T08:44:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:44:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:44:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:44:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:44:30.749 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.748+0000 7f629e14b640 1 -- 172.21.15.43:0/3483801726 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f62800a3dc0 msgr2=0x7f62800a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:30.749 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.748+0000 7f629e14b640 1 --2- 172.21.15.43:0/3483801726 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f62800a3dc0 0x7f62800a6280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6290049000 tx=0x7f629004a000 comp rx=0 tx=0).stop 2024-04-21T08:44:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.748+0000 7f629e14b640 1 -- 172.21.15.43:0/3483801726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6298102f00 msgr2=0x7f62981a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.748+0000 7f629e14b640 1 --2- 172.21.15.43:0/3483801726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6298102f00 0x7f62981a0b60 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f628c00d770 tx=0x7f628c00dc30 comp rx=0 tx=0).stop 2024-04-21T08:44:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.748+0000 7f629e14b640 1 -- 172.21.15.43:0/3483801726 shutdown_connections 2024-04-21T08:44:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.748+0000 7f629e14b640 1 --2- 172.21.15.43:0/3483801726 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f62800a3dc0 0x7f62800a6280 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.748+0000 7f629e14b640 1 --2- 172.21.15.43:0/3483801726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f629810e8a0 0x7f629819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.748+0000 7f629e14b640 1 --2- 172.21.15.43:0/3483801726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6298102f00 0x7f62981a0b60 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.748+0000 7f629e14b640 1 --2- 172.21.15.43:0/3483801726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f629806acc0 0x7f62981a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.748+0000 7f629e14b640 1 -- 172.21.15.43:0/3483801726 >> 172.21.15.43:0/3483801726 conn(0x7f6298076c50 msgr2=0x7f6298100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.748+0000 7f629e14b640 1 -- 172.21.15.43:0/3483801726 shutdown_connections 2024-04-21T08:44:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:30.748+0000 7f629e14b640 1 -- 172.21.15.43:0/3483801726 wait complete. 2024-04-21T08:44:32.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:32 smithi175 ceph-mon[31920]: pgmap v171: 129 pgs: 129 active+clean; 4.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 26 op/s 2024-04-21T08:44:32.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:32 smithi112 ceph-mon[31085]: pgmap v171: 129 pgs: 129 active+clean; 4.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 26 op/s 2024-04-21T08:44:32.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:32 smithi043 ceph-mon[27978]: pgmap v171: 129 pgs: 129 active+clean; 4.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 26 op/s 2024-04-21T08:44:33.179 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.179 INFO:tasks.workunit.client.0.smithi043.stdout:189/600: open fd = 3 2024-04-21T08:44:33.179 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.179 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.179 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.179 INFO:tasks.workunit.client.0.smithi043.stdout:190/600: open fd = 3 2024-04-21T08:44:33.179 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.179 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.179 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.179 INFO:tasks.workunit.client.0.smithi043.stdout:191/600: open fd = 3 2024-04-21T08:44:33.179 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.180 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.180 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.180 INFO:tasks.workunit.client.0.smithi043.stdout:192/600: open fd = 3 2024-04-21T08:44:33.180 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.180 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.180 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.180 INFO:tasks.workunit.client.0.smithi043.stdout:193/600: open fd = 3 2024-04-21T08:44:33.180 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.180 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.180 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.180 INFO:tasks.workunit.client.0.smithi043.stdout:194/600: open fd = 3 2024-04-21T08:44:33.180 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.180 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.180 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.180 INFO:tasks.workunit.client.0.smithi043.stdout:195/600: open fd = 3 2024-04-21T08:44:33.181 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.181 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.181 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.181 INFO:tasks.workunit.client.0.smithi043.stdout:196/600: open fd = 3 2024-04-21T08:44:33.181 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.181 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.181 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.181 INFO:tasks.workunit.client.0.smithi043.stdout:197/600: open fd = 3 2024-04-21T08:44:33.181 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.181 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.181 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.181 INFO:tasks.workunit.client.0.smithi043.stdout:198/600: open fd = 3 2024-04-21T08:44:33.181 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.182 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.182 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.182 INFO:tasks.workunit.client.0.smithi043.stdout:199/600: open fd = 3 2024-04-21T08:44:33.182 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.182 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.182 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.182 INFO:tasks.workunit.client.0.smithi043.stdout:200/600: open fd = 3 2024-04-21T08:44:33.182 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.182 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.182 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.182 INFO:tasks.workunit.client.0.smithi043.stdout:201/600: open fd = 3 2024-04-21T08:44:33.182 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.183 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.183 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.183 INFO:tasks.workunit.client.0.smithi043.stdout:202/600: open fd = 3 2024-04-21T08:44:33.183 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.183 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.183 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.183 INFO:tasks.workunit.client.0.smithi043.stdout:203/600: open fd = 3 2024-04-21T08:44:33.183 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.183 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.183 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.183 INFO:tasks.workunit.client.0.smithi043.stdout:204/600: open fd = 3 2024-04-21T08:44:33.183 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.183 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.183 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.183 INFO:tasks.workunit.client.0.smithi043.stdout:205/600: open fd = 3 2024-04-21T08:44:33.183 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.184 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.184 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.184 INFO:tasks.workunit.client.0.smithi043.stdout:206/600: open fd = 3 2024-04-21T08:44:33.184 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.184 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.184 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.184 INFO:tasks.workunit.client.0.smithi043.stdout:207/600: open fd = 3 2024-04-21T08:44:33.184 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.184 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.184 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.184 INFO:tasks.workunit.client.0.smithi043.stdout:208/600: open fd = 3 2024-04-21T08:44:33.184 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.184 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.184 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.184 INFO:tasks.workunit.client.0.smithi043.stdout:209/600: open fd = 3 2024-04-21T08:44:33.185 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.185 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.185 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.185 INFO:tasks.workunit.client.0.smithi043.stdout:210/600: open fd = 3 2024-04-21T08:44:33.185 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.185 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.185 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.185 INFO:tasks.workunit.client.0.smithi043.stdout:211/600: open fd = 3 2024-04-21T08:44:33.185 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.185 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.185 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.185 INFO:tasks.workunit.client.0.smithi043.stdout:212/600: open fd = 3 2024-04-21T08:44:33.185 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.185 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.185 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.186 INFO:tasks.workunit.client.0.smithi043.stdout:213/600: open fd = 3 2024-04-21T08:44:33.186 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.186 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.186 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.186 INFO:tasks.workunit.client.0.smithi043.stdout:214/600: open fd = 3 2024-04-21T08:44:33.186 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.186 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.186 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.186 INFO:tasks.workunit.client.0.smithi043.stdout:215/600: open fd = 3 2024-04-21T08:44:33.186 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.186 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.186 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.186 INFO:tasks.workunit.client.0.smithi043.stdout:216/600: open fd = 3 2024-04-21T08:44:33.186 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.187 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.187 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.187 INFO:tasks.workunit.client.0.smithi043.stdout:217/600: open fd = 3 2024-04-21T08:44:33.187 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.187 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.187 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.187 INFO:tasks.workunit.client.0.smithi043.stdout:218/600: open fd = 3 2024-04-21T08:44:33.187 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.187 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.187 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.187 INFO:tasks.workunit.client.0.smithi043.stdout:219/600: open fd = 3 2024-04-21T08:44:33.187 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.187 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.187 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.187 INFO:tasks.workunit.client.0.smithi043.stdout:220/600: open fd = 3 2024-04-21T08:44:33.187 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.188 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.188 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.188 INFO:tasks.workunit.client.0.smithi043.stdout:221/600: open fd = 3 2024-04-21T08:44:33.188 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.188 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.188 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.188 INFO:tasks.workunit.client.0.smithi043.stdout:222/600: open fd = 3 2024-04-21T08:44:33.188 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.188 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.188 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.188 INFO:tasks.workunit.client.0.smithi043.stdout:223/600: open fd = 3 2024-04-21T08:44:33.188 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.188 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.188 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.188 INFO:tasks.workunit.client.0.smithi043.stdout:224/600: open fd = 3 2024-04-21T08:44:33.189 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.189 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.189 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.189 INFO:tasks.workunit.client.0.smithi043.stdout:225/600: open fd = 3 2024-04-21T08:44:33.189 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.189 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.189 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.189 INFO:tasks.workunit.client.0.smithi043.stdout:226/600: open fd = 3 2024-04-21T08:44:33.189 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.189 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.189 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.189 INFO:tasks.workunit.client.0.smithi043.stdout:227/600: open fd = 3 2024-04-21T08:44:33.189 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.189 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.189 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.189 INFO:tasks.workunit.client.0.smithi043.stdout:228/600: open fd = 3 2024-04-21T08:44:33.190 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.190 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.190 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.190 INFO:tasks.workunit.client.0.smithi043.stdout:229/600: open fd = 3 2024-04-21T08:44:33.190 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.190 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.190 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.190 INFO:tasks.workunit.client.0.smithi043.stdout:230/600: open fd = 3 2024-04-21T08:44:33.190 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.190 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.190 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.190 INFO:tasks.workunit.client.0.smithi043.stdout:231/600: open fd = 3 2024-04-21T08:44:33.190 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.190 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.191 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.191 INFO:tasks.workunit.client.0.smithi043.stdout:232/600: open fd = 3 2024-04-21T08:44:33.191 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.191 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.191 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.191 INFO:tasks.workunit.client.0.smithi043.stdout:233/600: open fd = 3 2024-04-21T08:44:33.191 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.191 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.191 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.191 INFO:tasks.workunit.client.0.smithi043.stdout:234/600: open fd = 3 2024-04-21T08:44:33.191 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.191 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.192 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.192 INFO:tasks.workunit.client.0.smithi043.stdout:235/600: open fd = 3 2024-04-21T08:44:33.192 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.192 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.192 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.192 INFO:tasks.workunit.client.0.smithi043.stdout:236/600: open fd = 3 2024-04-21T08:44:33.192 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.192 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.192 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.192 INFO:tasks.workunit.client.0.smithi043.stdout:237/600: open fd = 3 2024-04-21T08:44:33.192 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.192 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.193 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.193 INFO:tasks.workunit.client.0.smithi043.stdout:238/600: open fd = 3 2024-04-21T08:44:33.193 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.193 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.193 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.193 INFO:tasks.workunit.client.0.smithi043.stdout:239/600: open fd = 3 2024-04-21T08:44:33.193 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.193 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.193 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.193 INFO:tasks.workunit.client.0.smithi043.stdout:240/600: open fd = 3 2024-04-21T08:44:33.193 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.193 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.193 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.194 INFO:tasks.workunit.client.0.smithi043.stdout:241/600: open fd = 3 2024-04-21T08:44:33.194 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.194 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.194 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.194 INFO:tasks.workunit.client.0.smithi043.stdout:242/600: open fd = 3 2024-04-21T08:44:33.194 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.194 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.194 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.194 INFO:tasks.workunit.client.0.smithi043.stdout:243/600: open fd = 3 2024-04-21T08:44:33.194 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.194 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.194 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.194 INFO:tasks.workunit.client.0.smithi043.stdout:244/600: open fd = 3 2024-04-21T08:44:33.195 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.195 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.195 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.195 INFO:tasks.workunit.client.0.smithi043.stdout:245/600: open fd = 3 2024-04-21T08:44:33.195 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.195 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.195 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.195 INFO:tasks.workunit.client.0.smithi043.stdout:246/600: open fd = 3 2024-04-21T08:44:33.195 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.195 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.195 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.195 INFO:tasks.workunit.client.0.smithi043.stdout:247/600: open fd = 3 2024-04-21T08:44:33.195 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.195 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.196 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.196 INFO:tasks.workunit.client.0.smithi043.stdout:248/600: open fd = 3 2024-04-21T08:44:33.196 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.196 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.196 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.196 INFO:tasks.workunit.client.0.smithi043.stdout:249/600: open fd = 3 2024-04-21T08:44:33.196 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.196 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.196 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:44:33.196 INFO:tasks.workunit.client.0.smithi043.stdout:250/600: open fd = 3 2024-04-21T08:44:33.196 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:33.196 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:44:34.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:34 smithi175 ceph-mon[31920]: pgmap v172: 129 pgs: 129 active+clean; 4.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 17 op/s 2024-04-21T08:44:34.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:34 smithi112 ceph-mon[31085]: pgmap v172: 129 pgs: 129 active+clean; 4.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 17 op/s 2024-04-21T08:44:34.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:34 smithi043 ceph-mon[27978]: pgmap v172: 129 pgs: 129 active+clean; 4.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 17 op/s 2024-04-21T08:44:36.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:36 smithi043 ceph-mon[27978]: pgmap v173: 129 pgs: 129 active+clean; 4.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 17 op/s 2024-04-21T08:44:36.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:36 smithi175 ceph-mon[31920]: pgmap v173: 129 pgs: 129 active+clean; 4.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 17 op/s 2024-04-21T08:44:36.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:36 smithi112 ceph-mon[31085]: pgmap v173: 129 pgs: 129 active+clean; 4.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 17 op/s 2024-04-21T08:44:38.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:38 smithi175 ceph-mon[31920]: pgmap v174: 129 pgs: 129 active+clean; 4.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 23 op/s 2024-04-21T08:44:38.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:38 smithi112 ceph-mon[31085]: pgmap v174: 129 pgs: 129 active+clean; 4.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 23 op/s 2024-04-21T08:44:38.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:38 smithi043 ceph-mon[27978]: pgmap v174: 129 pgs: 129 active+clean; 4.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 23 op/s 2024-04-21T08:44:40.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:40 smithi175 ceph-mon[31920]: pgmap v175: 129 pgs: 129 active+clean; 4.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 16 op/s 2024-04-21T08:44:40.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:40 smithi112 ceph-mon[31085]: pgmap v175: 129 pgs: 129 active+clean; 4.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 16 op/s 2024-04-21T08:44:40.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:40 smithi043 ceph-mon[27978]: pgmap v175: 129 pgs: 129 active+clean; 4.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 16 op/s 2024-04-21T08:44:42.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:42 smithi175 ceph-mon[31920]: pgmap v176: 129 pgs: 129 active+clean; 4.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 53 KiB/s wr, 23 op/s 2024-04-21T08:44:42.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:42 smithi112 ceph-mon[31085]: pgmap v176: 129 pgs: 129 active+clean; 4.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 53 KiB/s wr, 23 op/s 2024-04-21T08:44:42.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:42 smithi043 ceph-mon[27978]: pgmap v176: 129 pgs: 129 active+clean; 4.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 53 KiB/s wr, 23 op/s 2024-04-21T08:44:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.848+0000 7f0dd41a4640 1 -- 172.21.15.43:0/1682147657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dcc1021d0 msgr2=0x7f0dcc1025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.848+0000 7f0dd41a4640 1 --2- 172.21.15.43:0/1682147657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dcc1021d0 0x7f0dcc1025b0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009900 tx=0x7f0dc802bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.848+0000 7f0dd41a4640 1 -- 172.21.15.43:0/1682147657 shutdown_connections 2024-04-21T08:44:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.848+0000 7f0dd41a4640 1 --2- 172.21.15.43:0/1682147657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0dcc1072a0 0x7f0dcc10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.848+0000 7f0dd41a4640 1 --2- 172.21.15.43:0/1682147657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dcc102b80 0x7f0dcc106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.848+0000 7f0dd41a4640 1 --2- 172.21.15.43:0/1682147657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dcc1021d0 0x7f0dcc1025b0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.848+0000 7f0dd41a4640 1 -- 172.21.15.43:0/1682147657 >> 172.21.15.43:0/1682147657 conn(0x7f0dcc0793e0 msgr2=0x7f0dcc0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.848+0000 7f0dd41a4640 1 -- 172.21.15.43:0/1682147657 shutdown_connections 2024-04-21T08:44:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.848+0000 7f0dd41a4640 1 -- 172.21.15.43:0/1682147657 wait complete. 2024-04-21T08:44:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.848+0000 7f0dd41a4640 1 Processor -- start 2024-04-21T08:44:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.849+0000 7f0dd41a4640 1 -- start start 2024-04-21T08:44:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.849+0000 7f0dd41a4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0dcc1021d0 0x7f0dcc074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.849+0000 7f0dd41a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dcc1072a0 0x7f0dcc074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.849+0000 7f0dd41a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dcc074cd0 0x7f0dcc1a6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:43.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.849+0000 7f0dd41a4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc1121e0 con 0x7f0dcc1072a0 2024-04-21T08:44:43.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.849+0000 7f0dd41a4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc112060 con 0x7f0dcc074cd0 2024-04-21T08:44:43.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.849+0000 7f0dd41a4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc112360 con 0x7f0dcc1021d0 2024-04-21T08:44:43.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.849+0000 7f0dd173f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dcc1072a0 0x7f0dcc074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:43.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.849+0000 7f0dd173f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dcc1072a0 0x7f0dcc074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43204/0 (socket says 172.21.15.43:43204) 2024-04-21T08:44:43.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.849+0000 7f0dd173f640 1 -- 172.21.15.43:0/1650621929 learned_addr learned my addr 172.21.15.43:0/1650621929 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:43.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.849+0000 7f0dd1f40640 1 --2- 172.21.15.43:0/1650621929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0dcc1021d0 0x7f0dcc074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:43.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.849+0000 7f0dd2741640 1 --2- 172.21.15.43:0/1650621929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dcc074cd0 0x7f0dcc1a6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:43.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.849+0000 7f0dd173f640 1 -- 172.21.15.43:0/1650621929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0dcc1021d0 msgr2=0x7f0dcc074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:43.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.849+0000 7f0dd173f640 1 --2- 172.21.15.43:0/1650621929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0dcc1021d0 0x7f0dcc074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.849+0000 7f0dd173f640 1 -- 172.21.15.43:0/1650621929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dcc074cd0 msgr2=0x7f0dcc1a6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:43.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.849+0000 7f0dd173f640 1 --2- 172.21.15.43:0/1650621929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dcc074cd0 0x7f0dcc1a6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.849+0000 7f0dd173f640 1 -- 172.21.15.43:0/1650621929 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dcc1a6f70 con 0x7f0dcc1072a0 2024-04-21T08:44:43.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.849+0000 7f0dd2741640 1 --2- 172.21.15.43:0/1650621929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dcc074cd0 0x7f0dcc1a6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.850+0000 7f0dd173f640 1 --2- 172.21.15.43:0/1650621929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dcc1072a0 0x7f0dcc074790 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00d770 tx=0x7f0dbc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.850+0000 7f0dc2ffd640 1 -- 172.21.15.43:0/1650621929 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc017070 con 0x7f0dcc1072a0 2024-04-21T08:44:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.850+0000 7f0dc2ffd640 1 -- 172.21.15.43:0/1650621929 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dbc009c30 con 0x7f0dcc1072a0 2024-04-21T08:44:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.850+0000 7f0dd41a4640 1 -- 172.21.15.43:0/1650621929 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dcc1a7260 con 0x7f0dcc1072a0 2024-04-21T08:44:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.850+0000 7f0dc2ffd640 1 -- 172.21.15.43:0/1650621929 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc00f620 con 0x7f0dcc1072a0 2024-04-21T08:44:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.850+0000 7f0dd41a4640 1 -- 172.21.15.43:0/1650621929 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dcc1a77b0 con 0x7f0dcc1072a0 2024-04-21T08:44:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.852+0000 7f0dc2ffd640 1 -- 172.21.15.43:0/1650621929 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0dbc01a070 con 0x7f0dcc1072a0 2024-04-21T08:44:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.852+0000 7f0dd6d90740 1 Processor -- start 2024-04-21T08:44:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.853+0000 7f0dd6d90740 1 -- start start 2024-04-21T08:44:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.853+0000 7f0dd6d90740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592a39478d0 0x5592a3946620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.853+0000 7f0dd6d90740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592a3946b60 0x5592a3946fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.853+0000 7f0dd6d90740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a39500e0 0x5592a39524d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.853+0000 7f0dd6d90740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5592a38ab8d0 con 0x5592a39500e0 2024-04-21T08:44:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.853+0000 7f0dd6d90740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5592a3848e80 con 0x5592a3946b60 2024-04-21T08:44:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.853+0000 7f0dd6d90740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5592a38a7890 con 0x5592a39478d0 2024-04-21T08:44:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.853+0000 7f0dc2ffd640 1 --2- 172.21.15.43:0/1650621929 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0db80a3df0 0x7f0db80a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.853+0000 7f0dd173f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a39500e0 0x5592a39524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.853+0000 7f0dd1f40640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592a3946b60 0x5592a3946fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.853+0000 7f0dd1f40640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592a3946b60 0x5592a3946fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:47956/0 (socket says 172.21.15.43:47956) 2024-04-21T08:44:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.853+0000 7f0dd173f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a39500e0 0x5592a39524d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43214/0 (socket says 172.21.15.43:43214) 2024-04-21T08:44:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.853+0000 7f0dd1f40640 1 -- 172.21.15.43:0/2133223003 learned_addr learned my addr 172.21.15.43:0/2133223003 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.853+0000 7f0dd2741640 1 --2- 172.21.15.43:0/1650621929 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0db80a3df0 0x7f0db80a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.853+0000 7f0dd2741640 1 --2- 172.21.15.43:0/2133223003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592a39478d0 0x5592a3946620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.853+0000 7f0dc2ffd640 1 -- 172.21.15.43:0/1650621929 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f0dbc0d06d0 con 0x7f0dcc1072a0 2024-04-21T08:44:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.853+0000 7f0dd1f40640 1 -- 172.21.15.43:0/2133223003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592a39478d0 msgr2=0x5592a3946620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.853+0000 7f0dd1f40640 1 --2- 172.21.15.43:0/2133223003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592a39478d0 0x5592a3946620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.853+0000 7f0dd1f40640 1 -- 172.21.15.43:0/2133223003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a39500e0 msgr2=0x5592a39524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.853+0000 7f0dd1f40640 1 --2- 172.21.15.43:0/2133223003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a39500e0 0x5592a39524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.853+0000 7f0dd1f40640 1 -- 172.21.15.43:0/2133223003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592a386a320 con 0x5592a3946b60 2024-04-21T08:44:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.854+0000 7f0dd2741640 1 --2- 172.21.15.43:0/1650621929 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0db80a3df0 0x7f0db80a62b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009fd0 tx=0x7f0dc4009ec0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.854+0000 7f0dd1f40640 1 --2- 172.21.15.43:0/2133223003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592a3946b60 0x5592a3946fe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009900 tx=0x7f0dc802c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae14afe9680215c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.854+0000 7f0db7fff640 1 -- 172.21.15.43:0/2133223003 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8045020 con 0x5592a3946b60 2024-04-21T08:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.854+0000 7f0db7fff640 1 -- 172.21.15.43:0/2133223003 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dc802c7e0 con 0x5592a3946b60 2024-04-21T08:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.854+0000 7f0db7fff640 1 -- 172.21.15.43:0/2133223003 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc80046c0 con 0x5592a3946b60 2024-04-21T08:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.855+0000 7f0dd6d90740 1 -- 172.21.15.43:0/2133223003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592a3946b60 msgr2=0x5592a3946fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.855+0000 7f0dd6d90740 1 --2- 172.21.15.43:0/2133223003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592a3946b60 0x5592a3946fe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009900 tx=0x7f0dc802c6c0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.855+0000 7f0dd6d90740 1 -- 172.21.15.43:0/2133223003 shutdown_connections 2024-04-21T08:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.855+0000 7f0dd6d90740 1 --2- 172.21.15.43:0/2133223003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a39500e0 0x5592a39524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.855+0000 7f0dd6d90740 1 --2- 172.21.15.43:0/2133223003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592a3946b60 0x5592a3946fe0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.855+0000 7f0dd6d90740 1 --2- 172.21.15.43:0/2133223003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592a39478d0 0x5592a3946620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.855+0000 7f0dd6d90740 1 -- 172.21.15.43:0/2133223003 >> 172.21.15.43:0/2133223003 conn(0x5592a390dc90 msgr2=0x5592a3941850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.855+0000 7f0dd6d90740 1 -- 172.21.15.43:0/2133223003 shutdown_connections 2024-04-21T08:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.855+0000 7f0dd6d90740 1 -- 172.21.15.43:0/2133223003 wait complete. 2024-04-21T08:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.855+0000 7f0dd6d90740 1 -- start start 2024-04-21T08:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.855+0000 7f0dd6d90740 1 Processor -- start 2024-04-21T08:44:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.856+0000 7f0dd6d90740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592a3946b60 0x5592a394fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.856+0000 7f0dd6d90740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592a39478d0 0x5592a394cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.856+0000 7f0dd6d90740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a39500e0 0x5592a394d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.856+0000 7f0dd6d90740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5592a3952db0 con 0x5592a39500e0 2024-04-21T08:44:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.856+0000 7f0dd6d90740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5592a3833eb0 con 0x5592a3946b60 2024-04-21T08:44:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.856+0000 7f0dd6d90740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5592a3952f30 con 0x5592a39478d0 2024-04-21T08:44:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.856+0000 7f0dd6d90740 10 client.0 _ll_register_callbacks cb 0x5592a3860b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.856+0000 7f0dd2741640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a39500e0 0x5592a394d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.856+0000 7f0dd1f40640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592a3946b60 0x5592a394fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.856+0000 7f0dd2741640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a39500e0 0x5592a394d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43216/0 (socket says 172.21.15.43:43216) 2024-04-21T08:44:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.856+0000 7f0dd2741640 1 -- 172.21.15.43:0/4117148834 learned_addr learned my addr 172.21.15.43:0/4117148834 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.856+0000 7f0dd173f640 1 --2- 172.21.15.43:0/4117148834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592a39478d0 0x5592a394cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.856+0000 7f0dd2741640 1 -- 172.21.15.43:0/4117148834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592a39478d0 msgr2=0x5592a394cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.856+0000 7f0dd2741640 1 --2- 172.21.15.43:0/4117148834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592a39478d0 0x5592a394cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.856+0000 7f0dd2741640 1 -- 172.21.15.43:0/4117148834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592a3946b60 msgr2=0x5592a394fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.856+0000 7f0dd2741640 1 --2- 172.21.15.43:0/4117148834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592a3946b60 0x5592a394fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.856+0000 7f0dd2741640 1 -- 172.21.15.43:0/4117148834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592a35be210 con 0x5592a39500e0 2024-04-21T08:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.856+0000 7f0dd1f40640 1 --2- 172.21.15.43:0/4117148834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592a3946b60 0x5592a394fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.857+0000 7f0dd2741640 1 --2- 172.21.15.43:0/4117148834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a39500e0 0x5592a394d2f0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009d80 tx=0x7f0dc4009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e359b4219f1ef186 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.857+0000 7f0db57fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.857+0000 7f0db57fa640 1 -- 172.21.15.43:0/4117148834 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4012e60 con 0x5592a39500e0 2024-04-21T08:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.857+0000 7f0db57fa640 1 -- 172.21.15.43:0/4117148834 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f0dc4003b70 con 0x5592a39500e0 2024-04-21T08:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.857+0000 7f0dd6d90740 1 -- 172.21.15.43:0/4117148834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dc802c7e0 con 0x5592a39500e0 2024-04-21T08:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.857+0000 7f0db57fa640 1 -- 172.21.15.43:0/4117148834 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0dc4032050 con 0x5592a39500e0 2024-04-21T08:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.857+0000 7f0dd6d90740 10 client.18033 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.857+0000 7f0dd6d90740 1 -- 172.21.15.43:0/4117148834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5592a368a620 con 0x5592a39500e0 2024-04-21T08:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.857+0000 7f0db57fa640 1 -- 172.21.15.43:0/4117148834 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dc802c7e0 con 0x5592a39500e0 2024-04-21T08:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.857+0000 7f0db57fa640 1 -- 172.21.15.43:0/4117148834 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0dc4020630 con 0x5592a39500e0 2024-04-21T08:44:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.857+0000 7f0dd6d90740 10 client.18033 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.857+0000 7f0dd6d90740 10 client.18033 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.857+0000 7f0dd6d90740 20 client.18033 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.857+0000 7f0dd6d90740 1 --2- 172.21.15.43:0/4117148834 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5592a3960080 0x5592a3962490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.857+0000 7f0dd6d90740 4 client.18033 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.857+0000 7f0dd6d90740 1 -- 172.21.15.43:0/4117148834 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592a38bf880 con 0x5592a3960080 2024-04-21T08:44:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.858+0000 7f0dd1f40640 1 --2- 172.21.15.43:0/4117148834 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5592a3960080 0x5592a3962490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.858+0000 7f0dd1f40640 1 --2- 172.21.15.43:0/4117148834 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5592a3960080 0x5592a3962490 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5da88988d11a4345 server_cookie=dbfede48836ecc9b in_seq=0 out_seq=0 2024-04-21T08:44:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.858+0000 7f0db57fa640 10 client.18033 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.859+0000 7f0db57fa640 1 -- 172.21.15.43:0/4117148834 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5592a38bf880 con 0x5592a3960080 2024-04-21T08:44:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.859+0000 7f0db57fa640 10 client.18033 handle_command_reply: tid=0 2024-04-21T08:44:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.859+0000 7f0dd6d90740 1 client.18033 shutdown 2024-04-21T08:44:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.859+0000 7f0dd6d90740 1 -- 172.21.15.43:0/4117148834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a39500e0 msgr2=0x5592a394d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.859+0000 7f0dd6d90740 1 --2- 172.21.15.43:0/4117148834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a39500e0 0x5592a394d2f0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009d80 tx=0x7f0dc4009240 comp rx=0 tx=0).stop 2024-04-21T08:44:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.859+0000 7f0dd6d90740 1 -- 172.21.15.43:0/4117148834 shutdown_connections 2024-04-21T08:44:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.859+0000 7f0dd6d90740 1 --2- 172.21.15.43:0/4117148834 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5592a3960080 0x5592a3962490 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.859+0000 7f0dd6d90740 1 --2- 172.21.15.43:0/4117148834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a39500e0 0x5592a394d2f0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.859+0000 7f0dd6d90740 1 --2- 172.21.15.43:0/4117148834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592a39478d0 0x5592a394cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.859+0000 7f0dd6d90740 1 --2- 172.21.15.43:0/4117148834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592a3946b60 0x5592a394fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.859+0000 7f0db57fa640 0 client.18033 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.859+0000 7f0dd6d90740 1 -- 172.21.15.43:0/4117148834 >> 172.21.15.43:0/4117148834 conn(0x5592a390dc90 msgr2=0x5592a39435a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.860+0000 7f0dd6d90740 1 -- 172.21.15.43:0/4117148834 shutdown_connections 2024-04-21T08:44:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.860+0000 7f0dd6d90740 1 -- 172.21.15.43:0/4117148834 wait complete. 2024-04-21T08:44:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.860+0000 7f0dd6d90740 20 client.18033 trim_cache size 0 max 16384 2024-04-21T08:44:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.875+0000 7f0dd6d90740 1 Processor -- start 2024-04-21T08:44:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.875+0000 7f0dd6d90740 1 -- start start 2024-04-21T08:44:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.875+0000 7f0dd6d90740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592a3946b60 0x5592a395bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.875+0000 7f0dd6d90740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a39478d0 0x5592a394f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.875+0000 7f0dd6d90740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592a39500e0 0x5592a394fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.875+0000 7f0dd6d90740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc1b1400 con 0x5592a39478d0 2024-04-21T08:44:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.875+0000 7f0dd6d90740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc1b2f40 con 0x5592a3946b60 2024-04-21T08:44:43.877 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.875+0000 7f0dd6d90740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc1b1ad0 con 0x5592a39500e0 2024-04-21T08:44:43.877 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.875+0000 7f0dd173f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a39478d0 0x5592a394f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:43.877 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.875+0000 7f0dd173f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a39478d0 0x5592a394f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43228/0 (socket says 172.21.15.43:43228) 2024-04-21T08:44:43.877 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.875+0000 7f0dd173f640 1 -- 172.21.15.43:0/3776002432 learned_addr learned my addr 172.21.15.43:0/3776002432 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:43.877 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.875+0000 7f0dd2741640 1 --2- 172.21.15.43:0/3776002432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592a39500e0 0x5592a394fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:43.877 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.875+0000 7f0dd1f40640 1 --2- 172.21.15.43:0/3776002432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592a3946b60 0x5592a395bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:43.877 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.875+0000 7f0dd173f640 1 -- 172.21.15.43:0/3776002432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592a39500e0 msgr2=0x5592a394fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:43.877 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.875+0000 7f0dd173f640 1 --2- 172.21.15.43:0/3776002432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592a39500e0 0x5592a394fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.877 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.875+0000 7f0dd173f640 1 -- 172.21.15.43:0/3776002432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592a3946b60 msgr2=0x5592a395bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:43.877 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.875+0000 7f0dd173f640 1 --2- 172.21.15.43:0/3776002432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592a3946b60 0x5592a395bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.877 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.875+0000 7f0dd173f640 1 -- 172.21.15.43:0/3776002432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592a3945e30 con 0x5592a39478d0 2024-04-21T08:44:43.877 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.875+0000 7f0dd1f40640 1 --2- 172.21.15.43:0/3776002432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592a3946b60 0x5592a395bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:43.877 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.875+0000 7f0dd173f640 1 --2- 172.21.15.43:0/3776002432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a39478d0 0x5592a394f4e0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc0cdab0 tx=0x7f0dbc00df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4607dd6f44971d84 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:43.877 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.876+0000 7f0db77fe640 1 -- 172.21.15.43:0/3776002432 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc0b5070 con 0x5592a39478d0 2024-04-21T08:44:43.877 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.876+0000 7f0db77fe640 1 -- 172.21.15.43:0/3776002432 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dbc0a0650 con 0x5592a39478d0 2024-04-21T08:44:43.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.876+0000 7f0db77fe640 1 -- 172.21.15.43:0/3776002432 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc021040 con 0x5592a39478d0 2024-04-21T08:44:43.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.876+0000 7f0dd6d90740 1 -- 172.21.15.43:0/3776002432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a39478d0 msgr2=0x5592a394f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:43.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.876+0000 7f0dd6d90740 1 --2- 172.21.15.43:0/3776002432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a39478d0 0x5592a394f4e0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc0cdab0 tx=0x7f0dbc00df10 comp rx=0 tx=0).stop 2024-04-21T08:44:43.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.876+0000 7f0dd6d90740 1 -- 172.21.15.43:0/3776002432 shutdown_connections 2024-04-21T08:44:43.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.876+0000 7f0dd6d90740 1 --2- 172.21.15.43:0/3776002432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592a39500e0 0x5592a394fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.876+0000 7f0dd6d90740 1 --2- 172.21.15.43:0/3776002432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a39478d0 0x5592a394f4e0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.876+0000 7f0dd6d90740 1 --2- 172.21.15.43:0/3776002432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592a3946b60 0x5592a395bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.876+0000 7f0dd6d90740 1 -- 172.21.15.43:0/3776002432 >> 172.21.15.43:0/3776002432 conn(0x5592a390dc90 msgr2=0x5592a3941380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:43.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.876+0000 7f0dd6d90740 1 -- 172.21.15.43:0/3776002432 shutdown_connections 2024-04-21T08:44:43.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.876+0000 7f0dd6d90740 1 -- 172.21.15.43:0/3776002432 wait complete. 2024-04-21T08:44:43.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.877+0000 7f0dd6d90740 1 -- start start 2024-04-21T08:44:43.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.877+0000 7f0dd6d90740 1 Processor -- start 2024-04-21T08:44:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.877+0000 7f0dd6d90740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a3946b60 0x5592a3958540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.877+0000 7f0dd6d90740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592a39478d0 0x5592a394a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.877+0000 7f0dd6d90740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592a39500e0 0x5592a394a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.877+0000 7f0dd6d90740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc1b2870 con 0x5592a3946b60 2024-04-21T08:44:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.877+0000 7f0dd6d90740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc1b21a0 con 0x5592a39478d0 2024-04-21T08:44:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.877+0000 7f0dd6d90740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc1af3e0 con 0x5592a39500e0 2024-04-21T08:44:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.877+0000 7f0dd6d90740 10 client.0 _ll_register_callbacks cb 0x7f0dc4014590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.877+0000 7f0dd1f40640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a3946b60 0x5592a3958540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.877+0000 7f0dd1f40640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a3946b60 0x5592a3958540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43234/0 (socket says 172.21.15.43:43234) 2024-04-21T08:44:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.877+0000 7f0dd1f40640 1 -- 172.21.15.43:0/1946713714 learned_addr learned my addr 172.21.15.43:0/1946713714 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.878+0000 7f0dd173f640 1 --2- 172.21.15.43:0/1946713714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592a39478d0 0x5592a394a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.878+0000 7f0dd2741640 1 --2- 172.21.15.43:0/1946713714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592a39500e0 0x5592a394a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.878+0000 7f0dd1f40640 1 -- 172.21.15.43:0/1946713714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592a39500e0 msgr2=0x5592a394a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.878+0000 7f0dd1f40640 1 --2- 172.21.15.43:0/1946713714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592a39500e0 0x5592a394a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.878+0000 7f0dd1f40640 1 -- 172.21.15.43:0/1946713714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592a39478d0 msgr2=0x5592a394a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.878+0000 7f0dd1f40640 1 --2- 172.21.15.43:0/1946713714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592a39478d0 0x5592a394a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.878+0000 7f0dd1f40640 1 -- 172.21.15.43:0/1946713714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592a394ad90 con 0x5592a3946b60 2024-04-21T08:44:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.878+0000 7f0dd2741640 1 --2- 172.21.15.43:0/1946713714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592a39500e0 0x5592a394a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.878+0000 7f0dd1f40640 1 --2- 172.21.15.43:0/1946713714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a3946b60 0x5592a3958540 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f0dc80053d0 tx=0x7f0dc80051c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9254411569ab325b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.878+0000 7f0db57fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.878+0000 7f0db57fa640 1 -- 172.21.15.43:0/1946713714 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8050070 con 0x5592a3946b60 2024-04-21T08:44:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.878+0000 7f0db57fa640 1 -- 172.21.15.43:0/1946713714 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f0dc8004250 con 0x5592a3946b60 2024-04-21T08:44:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.878+0000 7f0dd6d90740 1 -- 172.21.15.43:0/1946713714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dbc0a0650 con 0x5592a3946b60 2024-04-21T08:44:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.878+0000 7f0db57fa640 1 -- 172.21.15.43:0/1946713714 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0dc8045040 con 0x5592a3946b60 2024-04-21T08:44:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.878+0000 7f0dd6d90740 10 client.18039 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.878+0000 7f0dd6d90740 1 -- 172.21.15.43:0/1946713714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5592a39c6610 con 0x5592a3946b60 2024-04-21T08:44:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.878+0000 7f0db57fa640 1 -- 172.21.15.43:0/1946713714 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dbc0a0650 con 0x5592a3946b60 2024-04-21T08:44:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.879+0000 7f0db57fa640 1 -- 172.21.15.43:0/1946713714 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0dc80035f0 con 0x5592a3946b60 2024-04-21T08:44:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.879+0000 7f0dd6d90740 10 client.18039 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.879+0000 7f0dd6d90740 10 client.18039 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.879+0000 7f0dd6d90740 20 client.18039 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.879+0000 7f0dd6d90740 1 --2- 172.21.15.43:0/1946713714 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5592a39c68f0 0x5592a39c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.879+0000 7f0dd6d90740 4 client.18039 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:44:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.879+0000 7f0dd6d90740 1 -- 172.21.15.43:0/1946713714 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592a384b3d0 con 0x5592a39c68f0 2024-04-21T08:44:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.879+0000 7f0dd173f640 1 --2- 172.21.15.43:0/1946713714 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5592a39c68f0 0x5592a39c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.879+0000 7f0dd173f640 1 --2- 172.21.15.43:0/1946713714 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5592a39c68f0 0x5592a39c8ce0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c62404a7b4ce6aa6 server_cookie=9807d7dd75bd2604 in_seq=0 out_seq=0 2024-04-21T08:44:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.879+0000 7f0db57fa640 10 client.18039 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.894+0000 7f0db57fa640 1 -- 172.21.15.43:0/1946713714 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5592a384b3d0 con 0x5592a39c68f0 2024-04-21T08:44:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.894+0000 7f0db57fa640 10 client.18039 handle_command_reply: tid=0 2024-04-21T08:44:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.894+0000 7f0dd6d90740 1 client.18039 shutdown 2024-04-21T08:44:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.895+0000 7f0dd6d90740 1 -- 172.21.15.43:0/1946713714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a3946b60 msgr2=0x5592a3958540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.895+0000 7f0dd6d90740 1 --2- 172.21.15.43:0/1946713714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a3946b60 0x5592a3958540 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f0dc80053d0 tx=0x7f0dc80051c0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.895+0000 7f0dd6d90740 1 -- 172.21.15.43:0/1946713714 shutdown_connections 2024-04-21T08:44:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.895+0000 7f0dd6d90740 1 --2- 172.21.15.43:0/1946713714 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5592a39c68f0 0x5592a39c8ce0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.895+0000 7f0dd6d90740 1 --2- 172.21.15.43:0/1946713714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592a39500e0 0x5592a394a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.895+0000 7f0dd6d90740 1 --2- 172.21.15.43:0/1946713714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592a39478d0 0x5592a394a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.895+0000 7f0dd6d90740 1 --2- 172.21.15.43:0/1946713714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592a3946b60 0x5592a3958540 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.895+0000 7f0db57fa640 0 client.18039 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.895+0000 7f0dd6d90740 1 -- 172.21.15.43:0/1946713714 >> 172.21.15.43:0/1946713714 conn(0x5592a390dc90 msgr2=0x5592a394b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.895+0000 7f0dd6d90740 1 -- 172.21.15.43:0/1946713714 shutdown_connections 2024-04-21T08:44:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.895+0000 7f0dd6d90740 1 -- 172.21.15.43:0/1946713714 wait complete. 2024-04-21T08:44:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.895+0000 7f0dd6d90740 20 client.18039 trim_cache size 0 max 16384 2024-04-21T08:44:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:44:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:44:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:44:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:44:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.895+0000 7f0dd41a4640 1 -- 172.21.15.43:0/1650621929 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0db80a3df0 msgr2=0x7f0db80a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.895+0000 7f0dd41a4640 1 --2- 172.21.15.43:0/1650621929 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0db80a3df0 0x7f0db80a62b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009fd0 tx=0x7f0dc4009ec0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.895+0000 7f0dd41a4640 1 -- 172.21.15.43:0/1650621929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dcc1072a0 msgr2=0x7f0dcc074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.895+0000 7f0dd41a4640 1 --2- 172.21.15.43:0/1650621929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dcc1072a0 0x7f0dcc074790 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00d770 tx=0x7f0dbc00dc30 comp rx=0 tx=0).stop 2024-04-21T08:44:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.895+0000 7f0dd41a4640 1 -- 172.21.15.43:0/1650621929 shutdown_connections 2024-04-21T08:44:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.896+0000 7f0dd41a4640 1 --2- 172.21.15.43:0/1650621929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0dcc074cd0 0x7f0dcc1a6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.896+0000 7f0dd41a4640 1 --2- 172.21.15.43:0/1650621929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dcc1072a0 0x7f0dcc074790 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.896+0000 7f0dd41a4640 1 --2- 172.21.15.43:0/1650621929 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0db80a3df0 0x7f0db80a62b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.896+0000 7f0dd41a4640 1 --2- 172.21.15.43:0/1650621929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0dcc1021d0 0x7f0dcc074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.896+0000 7f0dd41a4640 1 -- 172.21.15.43:0/1650621929 >> 172.21.15.43:0/1650621929 conn(0x7f0dcc0793e0 msgr2=0x7f0dcc0ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.896+0000 7f0dd41a4640 1 -- 172.21.15.43:0/1650621929 shutdown_connections 2024-04-21T08:44:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:43.896+0000 7f0dd41a4640 1 -- 172.21.15.43:0/1650621929 wait complete. 2024-04-21T08:44:44.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:44 smithi175 ceph-mon[31920]: pgmap v177: 129 pgs: 129 active+clean; 4.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-04-21T08:44:44.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:44 smithi112 ceph-mon[31085]: pgmap v177: 129 pgs: 129 active+clean; 4.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-04-21T08:44:44.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:44 smithi043 ceph-mon[27978]: pgmap v177: 129 pgs: 129 active+clean; 4.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-04-21T08:44:46.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:46 smithi175 ceph-mon[31920]: pgmap v178: 129 pgs: 129 active+clean; 4.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-04-21T08:44:46.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:46 smithi112 ceph-mon[31085]: pgmap v178: 129 pgs: 129 active+clean; 4.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-04-21T08:44:46.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:46 smithi043 ceph-mon[27978]: pgmap v178: 129 pgs: 129 active+clean; 4.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-04-21T08:44:48.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:48 smithi175 ceph-mon[31920]: pgmap v179: 129 pgs: 129 active+clean; 4.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-21T08:44:48.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:48 smithi112 ceph-mon[31085]: pgmap v179: 129 pgs: 129 active+clean; 4.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-21T08:44:48.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:48 smithi043 ceph-mon[27978]: pgmap v179: 129 pgs: 129 active+clean; 4.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-21T08:44:50.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:50 smithi175 ceph-mon[31920]: pgmap v180: 129 pgs: 129 active+clean; 4.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-21T08:44:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:50 smithi112 ceph-mon[31085]: pgmap v180: 129 pgs: 129 active+clean; 4.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-21T08:44:50.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:50 smithi043 ceph-mon[27978]: pgmap v180: 129 pgs: 129 active+clean; 4.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-21T08:44:52.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:52 smithi175 ceph-mon[31920]: pgmap v181: 129 pgs: 129 active+clean; 4.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-21T08:44:52.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:52 smithi112 ceph-mon[31085]: pgmap v181: 129 pgs: 129 active+clean; 4.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-21T08:44:52.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:52 smithi043 ceph-mon[27978]: pgmap v181: 129 pgs: 129 active+clean; 4.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-21T08:44:54.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:54 smithi175 ceph-mon[31920]: pgmap v182: 129 pgs: 129 active+clean; 4.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-21T08:44:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:54 smithi112 ceph-mon[31085]: pgmap v182: 129 pgs: 129 active+clean; 4.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-21T08:44:54.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:54 smithi043 ceph-mon[27978]: pgmap v182: 129 pgs: 129 active+clean; 4.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-21T08:44:56.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:56 smithi175 ceph-mon[31920]: pgmap v183: 129 pgs: 129 active+clean; 4.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-21T08:44:56.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:56 smithi112 ceph-mon[31085]: pgmap v183: 129 pgs: 129 active+clean; 4.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-21T08:44:56.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:56 smithi043 ceph-mon[27978]: pgmap v183: 129 pgs: 129 active+clean; 4.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-21T08:44:56.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:44:56.708 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:44:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.826+0000 7f35a465c640 1 -- 172.21.15.43:0/1374248667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f359c1021b0 msgr2=0x7f359c102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.826+0000 7f35a465c640 1 --2- 172.21.15.43:0/1374248667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f359c1021b0 0x7f359c102590 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f3598009900 tx=0x7f359802bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.826+0000 7f35a465c640 1 -- 172.21.15.43:0/1374248667 shutdown_connections 2024-04-21T08:44:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.826+0000 7f35a465c640 1 --2- 172.21.15.43:0/1374248667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f359c107280 0x7f359c10ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.826+0000 7f35a465c640 1 --2- 172.21.15.43:0/1374248667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f359c102b60 0x7f359c106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.826+0000 7f35a465c640 1 --2- 172.21.15.43:0/1374248667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f359c1021b0 0x7f359c102590 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.826+0000 7f35a465c640 1 -- 172.21.15.43:0/1374248667 >> 172.21.15.43:0/1374248667 conn(0x7f359c0fda10 msgr2=0x7f359c0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.826+0000 7f35a465c640 1 -- 172.21.15.43:0/1374248667 shutdown_connections 2024-04-21T08:44:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.826+0000 7f35a465c640 1 -- 172.21.15.43:0/1374248667 wait complete. 2024-04-21T08:44:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.827+0000 7f35a465c640 1 Processor -- start 2024-04-21T08:44:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.827+0000 7f35a465c640 1 -- start start 2024-04-21T08:44:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.827+0000 7f35a465c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f359c1021b0 0x7f359c19e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.827+0000 7f35a465c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f359c107280 0x7f359c19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.827+0000 7f35a465c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f359c198620 0x7f359c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.827+0000 7f35a465c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1121f0 con 0x7f359c107280 2024-04-21T08:44:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.827+0000 7f35a465c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f359c112070 con 0x7f359c198620 2024-04-21T08:44:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.827+0000 7f35a465c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f359c112370 con 0x7f359c1021b0 2024-04-21T08:44:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.827+0000 7f35a1bf7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f359c107280 0x7f359c19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.827+0000 7f35a1bf7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f359c107280 0x7f359c19e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33038/0 (socket says 172.21.15.43:33038) 2024-04-21T08:44:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.827+0000 7f35a23f8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f359c1021b0 0x7f359c19e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.827+0000 7f35a1bf7640 1 -- 172.21.15.43:0/1154946690 learned_addr learned my addr 172.21.15.43:0/1154946690 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.827+0000 7f35a2bf9640 1 --2- 172.21.15.43:0/1154946690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f359c198620 0x7f359c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.828+0000 7f35a1bf7640 1 -- 172.21.15.43:0/1154946690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f359c1021b0 msgr2=0x7f359c19e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.828+0000 7f35a1bf7640 1 --2- 172.21.15.43:0/1154946690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f359c1021b0 0x7f359c19e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.828+0000 7f35a1bf7640 1 -- 172.21.15.43:0/1154946690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f359c198620 msgr2=0x7f359c198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.828+0000 7f35a1bf7640 1 --2- 172.21.15.43:0/1154946690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f359c198620 0x7f359c198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.828+0000 7f35a1bf7640 1 -- 172.21.15.43:0/1154946690 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359c199140 con 0x7f359c107280 2024-04-21T08:44:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.828+0000 7f35a23f8640 1 --2- 172.21.15.43:0/1154946690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f359c1021b0 0x7f359c19e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:44:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.828+0000 7f35a1bf7640 1 --2- 172.21.15.43:0/1154946690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f359c107280 0x7f359c19e960 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f358c00e9c0 tx=0x7f358c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.828+0000 7f35937fe640 1 -- 172.21.15.43:0/1154946690 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c00cd80 con 0x7f359c107280 2024-04-21T08:44:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.828+0000 7f35937fe640 1 -- 172.21.15.43:0/1154946690 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f358c0054d0 con 0x7f359c107280 2024-04-21T08:44:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.828+0000 7f35a465c640 1 -- 172.21.15.43:0/1154946690 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359c199430 con 0x7f359c107280 2024-04-21T08:44:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.828+0000 7f35937fe640 1 -- 172.21.15.43:0/1154946690 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c010430 con 0x7f359c107280 2024-04-21T08:44:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.828+0000 7f35a465c640 1 -- 172.21.15.43:0/1154946690 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f359c1a5630 con 0x7f359c107280 2024-04-21T08:44:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.831+0000 7f35937fe640 1 -- 172.21.15.43:0/1154946690 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f358c004fd0 con 0x7f359c107280 2024-04-21T08:44:56.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.831+0000 7f35a724e740 1 Processor -- start 2024-04-21T08:44:56.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.832+0000 7f35a724e740 1 -- start start 2024-04-21T08:44:56.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.832+0000 7f35a724e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef29580 0x5632fef29960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.832+0000 7f35a724e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632fef29ea0 0x5632fef32540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.832+0000 7f35a724e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632fef32a80 0x5632fef34e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.832+0000 7f35a724e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5632fee8e1e0 con 0x5632fef29580 2024-04-21T08:44:56.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.832+0000 7f35a724e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5632fee2b390 con 0x5632fef29ea0 2024-04-21T08:44:56.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.832+0000 7f35a724e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5632fee8a1a0 con 0x5632fef32a80 2024-04-21T08:44:56.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.832+0000 7f35a2bf9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef29580 0x5632fef29960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.832+0000 7f35a2bf9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef29580 0x5632fef29960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33048/0 (socket says 172.21.15.43:33048) 2024-04-21T08:44:56.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.832+0000 7f35a2bf9640 1 -- 172.21.15.43:0/3573233105 learned_addr learned my addr 172.21.15.43:0/3573233105 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:56.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.832+0000 7f35a23f8640 1 --2- 172.21.15.43:0/3573233105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632fef29ea0 0x5632fef32540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.832+0000 7f35a1bf7640 1 --2- 172.21.15.43:0/3573233105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632fef32a80 0x5632fef34e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.832+0000 7f35937fe640 1 --2- 172.21.15.43:0/1154946690 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f35880a3d20 0x7f35880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.833+0000 7f35a2bf9640 1 -- 172.21.15.43:0/3573233105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632fef32a80 msgr2=0x5632fef34e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:56.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.833+0000 7f35a2bf9640 1 --2- 172.21.15.43:0/3573233105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632fef32a80 0x5632fef34e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.833+0000 7f35a2bf9640 1 -- 172.21.15.43:0/3573233105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632fef29ea0 msgr2=0x5632fef32540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:56.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.833+0000 7f35a2bf9640 1 --2- 172.21.15.43:0/3573233105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632fef29ea0 0x5632fef32540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.833+0000 7f35a2bf9640 1 -- 172.21.15.43:0/3573233105 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632fec6a7d0 con 0x5632fef29580 2024-04-21T08:44:56.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.833+0000 7f35a1bf7640 1 --2- 172.21.15.43:0/3573233105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632fef32a80 0x5632fef34e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:56.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.833+0000 7f35937fe640 1 -- 172.21.15.43:0/1154946690 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f358c0d0fd0 con 0x7f359c107280 2024-04-21T08:44:56.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.833+0000 7f35a2bf9640 1 --2- 172.21.15.43:0/1154946690 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f35880a3d20 0x7f35880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.833+0000 7f35a2bf9640 1 --2- 172.21.15.43:0/3573233105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef29580 0x5632fef29960 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f3594011590 tx=0x7f3594011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99793cf96741b597 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:56.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.833+0000 7f3590bfe640 1 -- 172.21.15.43:0/3573233105 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594017050 con 0x5632fef29580 2024-04-21T08:44:56.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.833+0000 7f3590bfe640 1 -- 172.21.15.43:0/3573233105 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f359401bb90 con 0x5632fef29580 2024-04-21T08:44:56.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.833+0000 7f3590bfe640 1 -- 172.21.15.43:0/3573233105 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594025e10 con 0x5632fef29580 2024-04-21T08:44:56.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.833+0000 7f35a2bf9640 1 --2- 172.21.15.43:0/1154946690 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f35880a3d20 0x7f35880a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3594022000 tx=0x7f3594023000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:56.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.834+0000 7f35a724e740 1 -- 172.21.15.43:0/3573233105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef29580 msgr2=0x5632fef29960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.834+0000 7f35a724e740 1 --2- 172.21.15.43:0/3573233105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef29580 0x5632fef29960 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f3594011590 tx=0x7f3594011a50 comp rx=0 tx=0).stop 2024-04-21T08:44:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.834+0000 7f35a724e740 1 -- 172.21.15.43:0/3573233105 shutdown_connections 2024-04-21T08:44:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.834+0000 7f35a724e740 1 --2- 172.21.15.43:0/3573233105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632fef32a80 0x5632fef34e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.834+0000 7f35a724e740 1 --2- 172.21.15.43:0/3573233105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632fef29ea0 0x5632fef32540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.834+0000 7f35a724e740 1 --2- 172.21.15.43:0/3573233105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef29580 0x5632fef29960 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.834+0000 7f35a724e740 1 -- 172.21.15.43:0/3573233105 >> 172.21.15.43:0/3573233105 conn(0x5632fef23cb0 msgr2=0x5632fef24090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.834+0000 7f35a724e740 1 -- 172.21.15.43:0/3573233105 shutdown_connections 2024-04-21T08:44:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.834+0000 7f35a724e740 1 -- 172.21.15.43:0/3573233105 wait complete. 2024-04-21T08:44:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.834+0000 7f35a724e740 1 -- start start 2024-04-21T08:44:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.834+0000 7f35a724e740 1 Processor -- start 2024-04-21T08:44:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.835+0000 7f35a724e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632fef29580 0x5632fef2fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.835+0000 7f35a724e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632fef29ea0 0x5632fef30050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.835+0000 7f35a724e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef32a80 0x5632fef30590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.835+0000 7f35a724e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5632fef35870 con 0x5632fef32a80 2024-04-21T08:44:56.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.835+0000 7f35a724e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5632fee472f0 con 0x5632fef29ea0 2024-04-21T08:44:56.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.835+0000 7f35a724e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5632fef359f0 con 0x5632fef29580 2024-04-21T08:44:56.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.835+0000 7f35a724e740 10 client.0 _ll_register_callbacks cb 0x5632fee54580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:56.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.835+0000 7f35a2bf9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef32a80 0x5632fef30590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.835+0000 7f35a2bf9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef32a80 0x5632fef30590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33064/0 (socket says 172.21.15.43:33064) 2024-04-21T08:44:56.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.835+0000 7f35a2bf9640 1 -- 172.21.15.43:0/627595094 learned_addr learned my addr 172.21.15.43:0/627595094 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:56.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.835+0000 7f35a1bf7640 1 --2- 172.21.15.43:0/627595094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632fef29ea0 0x5632fef30050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.835+0000 7f35a23f8640 1 --2- 172.21.15.43:0/627595094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632fef29580 0x5632fef2fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.835+0000 7f35a2bf9640 1 -- 172.21.15.43:0/627595094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632fef29580 msgr2=0x5632fef2fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:56.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.835+0000 7f35a2bf9640 1 --2- 172.21.15.43:0/627595094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632fef29580 0x5632fef2fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.835+0000 7f35a2bf9640 1 -- 172.21.15.43:0/627595094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632fef29ea0 msgr2=0x5632fef30050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:56.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.835+0000 7f35a2bf9640 1 --2- 172.21.15.43:0/627595094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632fef29ea0 0x5632fef30050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.835+0000 7f35a2bf9640 1 -- 172.21.15.43:0/627595094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632feba03f0 con 0x5632fef32a80 2024-04-21T08:44:56.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.835+0000 7f35a2bf9640 1 --2- 172.21.15.43:0/627595094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef32a80 0x5632fef30590 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f3594025f00 tx=0x7f3594025f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202fd4ffc7d2f4ab server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:56.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.835+0000 7f3585ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:56.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.836+0000 7f3585ffb640 1 -- 172.21.15.43:0/627595094 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359402a040 con 0x5632fef32a80 2024-04-21T08:44:56.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.836+0000 7f3585ffb640 1 -- 172.21.15.43:0/627595094 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f35940049c0 con 0x5632fef32a80 2024-04-21T08:44:56.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.836+0000 7f35a724e740 1 -- 172.21.15.43:0/627595094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f359401bb90 con 0x5632fef32a80 2024-04-21T08:44:56.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.836+0000 7f3585ffb640 1 -- 172.21.15.43:0/627595094 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3594017020 con 0x5632fef32a80 2024-04-21T08:44:56.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.836+0000 7f35a724e740 10 client.18051 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:56.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.836+0000 7f35a724e740 1 -- 172.21.15.43:0/627595094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5632feb600f0 con 0x5632fef32a80 2024-04-21T08:44:56.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.836+0000 7f3585ffb640 1 -- 172.21.15.43:0/627595094 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f359401bb90 con 0x5632fef32a80 2024-04-21T08:44:56.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.836+0000 7f3585ffb640 1 -- 172.21.15.43:0/627595094 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f359402d070 con 0x5632fef32a80 2024-04-21T08:44:56.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.836+0000 7f35a724e740 10 client.18051 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:56.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.836+0000 7f35a724e740 10 client.18051 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:56.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.836+0000 7f35a724e740 20 client.18051 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:56.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.836+0000 7f35a724e740 1 --2- 172.21.15.43:0/627595094 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5632fef42a70 0x5632fef44e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.836+0000 7f35a724e740 4 client.18051 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:56.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.836+0000 7f35a724e740 1 -- 172.21.15.43:0/627595094 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632fee9bc00 con 0x5632fef42a70 2024-04-21T08:44:56.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.836+0000 7f35a23f8640 1 --2- 172.21.15.43:0/627595094 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5632fef42a70 0x5632fef44e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.837+0000 7f35a23f8640 1 --2- 172.21.15.43:0/627595094 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5632fef42a70 0x5632fef44e60 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72ffe45a2fd599d server_cookie=868342540ce4bf8 in_seq=0 out_seq=0 2024-04-21T08:44:56.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.837+0000 7f3585ffb640 10 client.18051 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:56.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.838+0000 7f3585ffb640 1 -- 172.21.15.43:0/627595094 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5632fee9bc00 con 0x5632fef42a70 2024-04-21T08:44:56.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.838+0000 7f3585ffb640 10 client.18051 handle_command_reply: tid=0 2024-04-21T08:44:56.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.838+0000 7f35a724e740 1 client.18051 shutdown 2024-04-21T08:44:56.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.838+0000 7f35a724e740 1 -- 172.21.15.43:0/627595094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef32a80 msgr2=0x5632fef30590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.838+0000 7f35a724e740 1 --2- 172.21.15.43:0/627595094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef32a80 0x5632fef30590 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f3594025f00 tx=0x7f3594025f30 comp rx=0 tx=0).stop 2024-04-21T08:44:56.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.838+0000 7f35a724e740 1 -- 172.21.15.43:0/627595094 shutdown_connections 2024-04-21T08:44:56.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.838+0000 7f35a724e740 1 --2- 172.21.15.43:0/627595094 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5632fef42a70 0x5632fef44e60 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.838+0000 7f35a724e740 1 --2- 172.21.15.43:0/627595094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef32a80 0x5632fef30590 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.838+0000 7f35a724e740 1 --2- 172.21.15.43:0/627595094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632fef29ea0 0x5632fef30050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.838+0000 7f3585ffb640 0 client.18051 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:56.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.838+0000 7f35a724e740 1 --2- 172.21.15.43:0/627595094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632fef29580 0x5632fef2fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.838+0000 7f35a724e740 1 -- 172.21.15.43:0/627595094 >> 172.21.15.43:0/627595094 conn(0x5632fef23cb0 msgr2=0x5632fef25a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:56.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.838+0000 7f35a724e740 1 -- 172.21.15.43:0/627595094 shutdown_connections 2024-04-21T08:44:56.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.838+0000 7f35a724e740 1 -- 172.21.15.43:0/627595094 wait complete. 2024-04-21T08:44:56.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.838+0000 7f35a724e740 20 client.18051 trim_cache size 0 max 16384 2024-04-21T08:44:56.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.856+0000 7f35a724e740 1 Processor -- start 2024-04-21T08:44:56.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.856+0000 7f35a724e740 1 -- start start 2024-04-21T08:44:56.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.856+0000 7f35a724e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632fef29580 0x5632fef3e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.856+0000 7f35a724e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632fef29ea0 0x5632fef2ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.856+0000 7f35a724e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef32a80 0x5632fef303b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.856+0000 7f35a724e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1af4c0 con 0x5632fef32a80 2024-04-21T08:44:56.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.856+0000 7f35a724e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1b1000 con 0x5632fef29ea0 2024-04-21T08:44:56.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.856+0000 7f35a724e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1afb90 con 0x5632fef29580 2024-04-21T08:44:56.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.856+0000 7f35a2bf9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef32a80 0x5632fef303b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.856+0000 7f35a2bf9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef32a80 0x5632fef303b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33068/0 (socket says 172.21.15.43:33068) 2024-04-21T08:44:56.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.856+0000 7f35a2bf9640 1 -- 172.21.15.43:0/1500407141 learned_addr learned my addr 172.21.15.43:0/1500407141 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:56.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.856+0000 7f35a23f8640 1 --2- 172.21.15.43:0/1500407141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632fef29580 0x5632fef3e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.856+0000 7f35a1bf7640 1 --2- 172.21.15.43:0/1500407141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632fef29ea0 0x5632fef2ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.856+0000 7f35a2bf9640 1 -- 172.21.15.43:0/1500407141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632fef29580 msgr2=0x5632fef3e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:56.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.856+0000 7f35a2bf9640 1 --2- 172.21.15.43:0/1500407141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632fef29580 0x5632fef3e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.856+0000 7f35a2bf9640 1 -- 172.21.15.43:0/1500407141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632fef29ea0 msgr2=0x5632fef2ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:56.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.856+0000 7f35a2bf9640 1 --2- 172.21.15.43:0/1500407141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632fef29ea0 0x5632fef2ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.856+0000 7f35a2bf9640 1 -- 172.21.15.43:0/1500407141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632fef34e70 con 0x5632fef32a80 2024-04-21T08:44:56.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.856+0000 7f35a2bf9640 1 --2- 172.21.15.43:0/1500407141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef32a80 0x5632fef303b0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f3594011a90 tx=0x7f359401b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83e962ca8f2e821 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:56.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.856+0000 7f3587fff640 1 -- 172.21.15.43:0/1500407141 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359400f860 con 0x5632fef32a80 2024-04-21T08:44:56.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.857+0000 7f3587fff640 1 -- 172.21.15.43:0/1500407141 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3594033070 con 0x5632fef32a80 2024-04-21T08:44:56.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.857+0000 7f3587fff640 1 -- 172.21.15.43:0/1500407141 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359400e8e0 con 0x5632fef32a80 2024-04-21T08:44:56.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.857+0000 7f35a724e740 1 -- 172.21.15.43:0/1500407141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef32a80 msgr2=0x5632fef303b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.857+0000 7f35a724e740 1 --2- 172.21.15.43:0/1500407141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef32a80 0x5632fef303b0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f3594011a90 tx=0x7f359401b7d0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.857+0000 7f35a724e740 1 -- 172.21.15.43:0/1500407141 shutdown_connections 2024-04-21T08:44:56.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.857+0000 7f35a724e740 1 --2- 172.21.15.43:0/1500407141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef32a80 0x5632fef303b0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.857+0000 7f35a724e740 1 --2- 172.21.15.43:0/1500407141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632fef29ea0 0x5632fef2ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.857+0000 7f35a724e740 1 --2- 172.21.15.43:0/1500407141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632fef29580 0x5632fef3e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.857+0000 7f35a724e740 1 -- 172.21.15.43:0/1500407141 >> 172.21.15.43:0/1500407141 conn(0x5632fef23cb0 msgr2=0x5632fef224e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:56.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.857+0000 7f35a724e740 1 -- 172.21.15.43:0/1500407141 shutdown_connections 2024-04-21T08:44:56.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.857+0000 7f35a724e740 1 -- 172.21.15.43:0/1500407141 wait complete. 2024-04-21T08:44:56.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.857+0000 7f35a724e740 1 -- start start 2024-04-21T08:44:56.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.858+0000 7f35a724e740 1 Processor -- start 2024-04-21T08:44:56.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.858+0000 7f35a724e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632fef29580 0x5632fef3a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.858+0000 7f35a724e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632fef29ea0 0x5632fef3af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.858+0000 7f35a724e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef32a80 0x5632fef36010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.858+0000 7f35a724e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1b0930 con 0x5632fef32a80 2024-04-21T08:44:56.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.858+0000 7f35a724e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1b0260 con 0x5632fef29580 2024-04-21T08:44:56.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.858+0000 7f35a724e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1ad4a0 con 0x5632fef29ea0 2024-04-21T08:44:56.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.858+0000 7f35a724e740 10 client.0 _ll_register_callbacks cb 0x7f3594011f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:56.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.858+0000 7f35a2bf9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef32a80 0x5632fef36010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.858+0000 7f35a2bf9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef32a80 0x5632fef36010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33072/0 (socket says 172.21.15.43:33072) 2024-04-21T08:44:56.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.858+0000 7f35a2bf9640 1 -- 172.21.15.43:0/1477088661 learned_addr learned my addr 172.21.15.43:0/1477088661 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:56.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.858+0000 7f35a23f8640 1 --2- 172.21.15.43:0/1477088661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632fef29580 0x5632fef3a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.858+0000 7f35a1bf7640 1 --2- 172.21.15.43:0/1477088661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632fef29ea0 0x5632fef3af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.858+0000 7f35a2bf9640 1 -- 172.21.15.43:0/1477088661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632fef29ea0 msgr2=0x5632fef3af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:56.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.858+0000 7f35a2bf9640 1 --2- 172.21.15.43:0/1477088661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632fef29ea0 0x5632fef3af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.858+0000 7f35a2bf9640 1 -- 172.21.15.43:0/1477088661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632fef29580 msgr2=0x5632fef3a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:56.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.858+0000 7f35a2bf9640 1 --2- 172.21.15.43:0/1477088661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632fef29580 0x5632fef3a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.858+0000 7f35a2bf9640 1 -- 172.21.15.43:0/1477088661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632fef367f0 con 0x5632fef32a80 2024-04-21T08:44:56.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.858+0000 7f35a1bf7640 1 --2- 172.21.15.43:0/1477088661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632fef29ea0 0x5632fef3af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:56.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.858+0000 7f35a2bf9640 1 --2- 172.21.15.43:0/1477088661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef32a80 0x5632fef36010 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f359400e6f0 tx=0x7f3594025f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37fa52a1c6a5858c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:56.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.858+0000 7f3585ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:56.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.858+0000 7f3585ffb640 1 -- 172.21.15.43:0/1477088661 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359404d020 con 0x5632fef32a80 2024-04-21T08:44:56.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.859+0000 7f3585ffb640 1 -- 172.21.15.43:0/1477088661 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f3594011040 con 0x5632fef32a80 2024-04-21T08:44:56.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.859+0000 7f35a724e740 1 -- 172.21.15.43:0/1477088661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3594033070 con 0x5632fef32a80 2024-04-21T08:44:56.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.859+0000 7f3585ffb640 1 -- 172.21.15.43:0/1477088661 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3594058050 con 0x5632fef32a80 2024-04-21T08:44:56.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.859+0000 7f35a724e740 10 client.18057 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:56.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.859+0000 7f35a724e740 1 -- 172.21.15.43:0/1477088661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5632fefa8860 con 0x5632fef32a80 2024-04-21T08:44:56.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.859+0000 7f3585ffb640 1 -- 172.21.15.43:0/1477088661 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3594033070 con 0x5632fef32a80 2024-04-21T08:44:56.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.859+0000 7f3585ffb640 1 -- 172.21.15.43:0/1477088661 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f359402d070 con 0x5632fef32a80 2024-04-21T08:44:56.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.859+0000 7f35a724e740 10 client.18057 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:56.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.859+0000 7f35a724e740 10 client.18057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:56.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.859+0000 7f35a724e740 20 client.18057 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:56.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.859+0000 7f35a724e740 1 --2- 172.21.15.43:0/1477088661 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5632fefa8e60 0x5632fefab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.859+0000 7f35a724e740 4 client.18057 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:44:56.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.859+0000 7f35a724e740 1 -- 172.21.15.43:0/1477088661 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5632fee95a70 con 0x5632fefa8e60 2024-04-21T08:44:56.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.859+0000 7f35a23f8640 1 --2- 172.21.15.43:0/1477088661 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5632fefa8e60 0x5632fefab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.860+0000 7f35a23f8640 1 --2- 172.21.15.43:0/1477088661 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5632fefa8e60 0x5632fefab250 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0ddab6f0ed2efe server_cookie=b7d9692ae5ac3134 in_seq=0 out_seq=0 2024-04-21T08:44:56.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.860+0000 7f3585ffb640 10 client.18057 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:56.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.860+0000 7f3585ffb640 1 -- 172.21.15.43:0/1477088661 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5632fee95a70 con 0x5632fefa8e60 2024-04-21T08:44:56.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.860+0000 7f3585ffb640 10 client.18057 handle_command_reply: tid=0 2024-04-21T08:44:56.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.860+0000 7f35a724e740 1 client.18057 shutdown 2024-04-21T08:44:56.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.860+0000 7f35a724e740 1 -- 172.21.15.43:0/1477088661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef32a80 msgr2=0x5632fef36010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.860+0000 7f35a724e740 1 --2- 172.21.15.43:0/1477088661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef32a80 0x5632fef36010 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f359400e6f0 tx=0x7f3594025f80 comp rx=0 tx=0).stop 2024-04-21T08:44:56.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.860+0000 7f35a724e740 1 -- 172.21.15.43:0/1477088661 shutdown_connections 2024-04-21T08:44:56.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.860+0000 7f35a724e740 1 --2- 172.21.15.43:0/1477088661 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5632fefa8e60 0x5632fefab250 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.860+0000 7f35a724e740 1 --2- 172.21.15.43:0/1477088661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632fef32a80 0x5632fef36010 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.860+0000 7f35a724e740 1 --2- 172.21.15.43:0/1477088661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632fef29ea0 0x5632fef3af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.860+0000 7f3585ffb640 0 client.18057 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:56.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.860+0000 7f35a724e740 1 --2- 172.21.15.43:0/1477088661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632fef29580 0x5632fef3a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.860+0000 7f35a724e740 1 -- 172.21.15.43:0/1477088661 >> 172.21.15.43:0/1477088661 conn(0x5632fef23cb0 msgr2=0x5632fef224e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:56.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.860+0000 7f35a724e740 1 -- 172.21.15.43:0/1477088661 shutdown_connections 2024-04-21T08:44:56.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.860+0000 7f35a724e740 1 -- 172.21.15.43:0/1477088661 wait complete. 2024-04-21T08:44:56.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.860+0000 7f35a724e740 20 client.18057 trim_cache size 0 max 16384 2024-04-21T08:44:56.871 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:44:56.871 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:44:56.871 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:44:56.872 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:44:56.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.861+0000 7f35a465c640 1 -- 172.21.15.43:0/1154946690 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f35880a3d20 msgr2=0x7f35880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.861+0000 7f35a465c640 1 --2- 172.21.15.43:0/1154946690 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f35880a3d20 0x7f35880a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3594022000 tx=0x7f3594023000 comp rx=0 tx=0).stop 2024-04-21T08:44:56.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.861+0000 7f35a465c640 1 -- 172.21.15.43:0/1154946690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f359c107280 msgr2=0x7f359c19e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.861+0000 7f35a465c640 1 --2- 172.21.15.43:0/1154946690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f359c107280 0x7f359c19e960 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f358c00e9c0 tx=0x7f358c00ee80 comp rx=0 tx=0).stop 2024-04-21T08:44:56.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.861+0000 7f35a465c640 1 -- 172.21.15.43:0/1154946690 shutdown_connections 2024-04-21T08:44:56.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.861+0000 7f35a465c640 1 --2- 172.21.15.43:0/1154946690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f359c198620 0x7f359c198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.861+0000 7f35a465c640 1 --2- 172.21.15.43:0/1154946690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f359c107280 0x7f359c19e960 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.861+0000 7f35a465c640 1 --2- 172.21.15.43:0/1154946690 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f35880a3d20 0x7f35880a61e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.861+0000 7f35a465c640 1 --2- 172.21.15.43:0/1154946690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f359c1021b0 0x7f359c19e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.861+0000 7f35a465c640 1 -- 172.21.15.43:0/1154946690 >> 172.21.15.43:0/1154946690 conn(0x7f359c0fda10 msgr2=0x7f359c0fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:56.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.861+0000 7f35a465c640 1 -- 172.21.15.43:0/1154946690 shutdown_connections 2024-04-21T08:44:56.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.861+0000 7f35a465c640 1 -- 172.21.15.43:0/1154946690 wait complete. 2024-04-21T08:44:56.874 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:44:56.874 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:44:56.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.977+0000 7f563ad7e640 1 -- 172.21.15.43:0/2709658550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5634106f50 msgr2=0x7f5634107330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.977+0000 7f563ad7e640 1 --2- 172.21.15.43:0/2709658550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5634106f50 0x7f5634107330 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f5628009900 tx=0x7f562802bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:56.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.977+0000 7f563ad7e640 1 -- 172.21.15.43:0/2709658550 shutdown_connections 2024-04-21T08:44:56.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.977+0000 7f563ad7e640 1 --2- 172.21.15.43:0/2709658550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5634069aa0 0x7f563410ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.977+0000 7f563ad7e640 1 --2- 172.21.15.43:0/2709658550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56340690e0 0x7f5634069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.977+0000 7f563ad7e640 1 --2- 172.21.15.43:0/2709658550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5634106f50 0x7f5634107330 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.977+0000 7f563ad7e640 1 -- 172.21.15.43:0/2709658550 >> 172.21.15.43:0/2709658550 conn(0x7f5634077e30 msgr2=0x7f5634078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:56.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.977+0000 7f563ad7e640 1 -- 172.21.15.43:0/2709658550 shutdown_connections 2024-04-21T08:44:56.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.977+0000 7f563ad7e640 1 -- 172.21.15.43:0/2709658550 wait complete. 2024-04-21T08:44:56.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.978+0000 7f563ad7e640 1 Processor -- start 2024-04-21T08:44:56.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.978+0000 7f563ad7e640 1 -- start start 2024-04-21T08:44:56.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.978+0000 7f563ad7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56340690e0 0x7f5634101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.978+0000 7f563ad7e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5634069aa0 0x7f56341017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.978+0000 7f563ad7e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5634106f50 0x7f5634101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.978+0000 7f563ad7e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5634112170 con 0x7f5634069aa0 2024-04-21T08:44:56.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.978+0000 7f563ad7e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5634111ff0 con 0x7f56340690e0 2024-04-21T08:44:56.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.978+0000 7f563ad7e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f56341122f0 con 0x7f5634106f50 2024-04-21T08:44:56.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.978+0000 7f5633fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5634069aa0 0x7f56341017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.978+0000 7f5633fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5634069aa0 0x7f56341017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33094/0 (socket says 172.21.15.43:33094) 2024-04-21T08:44:56.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.978+0000 7f5633fff640 1 -- 172.21.15.43:0/2480823216 learned_addr learned my addr 172.21.15.43:0/2480823216 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:56.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.978+0000 7f5638b1a640 1 --2- 172.21.15.43:0/2480823216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56340690e0 0x7f5634101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.978+0000 7f563931b640 1 --2- 172.21.15.43:0/2480823216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5634106f50 0x7f5634101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.978+0000 7f5633fff640 1 -- 172.21.15.43:0/2480823216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5634106f50 msgr2=0x7f5634101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.978+0000 7f5633fff640 1 --2- 172.21.15.43:0/2480823216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5634106f50 0x7f5634101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.978+0000 7f5633fff640 1 -- 172.21.15.43:0/2480823216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56340690e0 msgr2=0x7f5634101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.978+0000 7f5633fff640 1 --2- 172.21.15.43:0/2480823216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56340690e0 0x7f5634101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.978+0000 7f5633fff640 1 -- 172.21.15.43:0/2480823216 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56341a6b90 con 0x7f5634069aa0 2024-04-21T08:44:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.978+0000 7f5638b1a640 1 --2- 172.21.15.43:0/2480823216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56340690e0 0x7f5634101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.978+0000 7f5633fff640 1 --2- 172.21.15.43:0/2480823216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5634069aa0 0x7f56341017b0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f562000d950 tx=0x7f562000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.979+0000 7f5631ffb640 1 -- 172.21.15.43:0/2480823216 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5620017070 con 0x7f5634069aa0 2024-04-21T08:44:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.979+0000 7f5631ffb640 1 -- 172.21.15.43:0/2480823216 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5620009c30 con 0x7f5634069aa0 2024-04-21T08:44:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.979+0000 7f563ad7e640 1 -- 172.21.15.43:0/2480823216 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56341a6e80 con 0x7f5634069aa0 2024-04-21T08:44:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.979+0000 7f5631ffb640 1 -- 172.21.15.43:0/2480823216 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562000f620 con 0x7f5634069aa0 2024-04-21T08:44:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.979+0000 7f563ad7e640 1 -- 172.21.15.43:0/2480823216 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56341a7340 con 0x7f5634069aa0 2024-04-21T08:44:56.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.980+0000 7f563ad7e640 1 -- 172.21.15.43:0/2480823216 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5634068de0 con 0x7f5634069aa0 2024-04-21T08:44:56.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.981+0000 7f5631ffb640 1 -- 172.21.15.43:0/2480823216 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5620010460 con 0x7f5634069aa0 2024-04-21T08:44:56.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.981+0000 7f5631ffb640 1 --2- 172.21.15.43:0/2480823216 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f56240a3d20 0x7f56240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.982+0000 7f5638b1a640 1 --2- 172.21.15.43:0/2480823216 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f56240a3d20 0x7f56240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.982+0000 7f5631ffb640 1 -- 172.21.15.43:0/2480823216 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f56200d08a0 con 0x7f5634069aa0 2024-04-21T08:44:56.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.982+0000 7f5638b1a640 1 --2- 172.21.15.43:0/2480823216 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f56240a3d20 0x7f56240a61e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5628000c00 tx=0x7f5628005dd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:56.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:56.986+0000 7f5631ffb640 1 -- 172.21.15.43:0/2480823216 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f562009cb70 con 0x7f5634069aa0 2024-04-21T08:44:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.987+0000 7f8f1f3ab640 1 -- 172.21.15.43:0/905331488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f18106170 msgr2=0x7f8f1810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.987+0000 7f8f1f3ab640 1 --2- 172.21.15.43:0/905331488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f18106170 0x7f8f1810dd40 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009ca0 tx=0x7f8f1002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.987+0000 7f8f1f3ab640 1 -- 172.21.15.43:0/905331488 shutdown_connections 2024-04-21T08:44:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.987+0000 7f8f1f3ab640 1 --2- 172.21.15.43:0/905331488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f18106170 0x7f8f1810dd40 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.987+0000 7f8f1f3ab640 1 --2- 172.21.15.43:0/905331488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f18101aa0 0x7f8f18105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.987+0000 7f8f1f3ab640 1 --2- 172.21.15.43:0/905331488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f181010f0 0x7f8f181014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.987+0000 7f8f1f3ab640 1 -- 172.21.15.43:0/905331488 >> 172.21.15.43:0/905331488 conn(0x7f8f18077fd0 msgr2=0x7f8f180783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.987+0000 7f8f1f3ab640 1 -- 172.21.15.43:0/905331488 shutdown_connections 2024-04-21T08:44:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.987+0000 7f8f1f3ab640 1 -- 172.21.15.43:0/905331488 wait complete. 2024-04-21T08:44:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.987+0000 7f8f1f3ab640 1 Processor -- start 2024-04-21T08:44:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.987+0000 7f8f1f3ab640 1 -- start start 2024-04-21T08:44:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.988+0000 7f8f1f3ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f181010f0 0x7f8f18102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.988+0000 7f8f1f3ab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f18106170 0x7f8f18103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.988+0000 7f8f1f3ab640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f181035b0 0x7f8f181a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.988+0000 7f8f1f3ab640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8f181110e0 con 0x7f8f18106170 2024-04-21T08:44:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.988+0000 7f8f1f3ab640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f18110f60 con 0x7f8f181010f0 2024-04-21T08:44:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.988+0000 7f8f1f3ab640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8f18111260 con 0x7f8f181035b0 2024-04-21T08:44:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.988+0000 7f8f1c946640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f18106170 0x7f8f18103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.988+0000 7f8f1c946640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f18106170 0x7f8f18103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33122/0 (socket says 172.21.15.43:33122) 2024-04-21T08:44:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.988+0000 7f8f1c946640 1 -- 172.21.15.43:0/846777732 learned_addr learned my addr 172.21.15.43:0/846777732 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.988+0000 7f8f1d147640 1 --2- 172.21.15.43:0/846777732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f181010f0 0x7f8f18102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.988+0000 7f8f1d948640 1 --2- 172.21.15.43:0/846777732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f181035b0 0x7f8f181a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.988+0000 7f8f1c946640 1 -- 172.21.15.43:0/846777732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f181035b0 msgr2=0x7f8f181a68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.988+0000 7f8f1c946640 1 --2- 172.21.15.43:0/846777732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f181035b0 0x7f8f181a68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.988+0000 7f8f1c946640 1 -- 172.21.15.43:0/846777732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f181010f0 msgr2=0x7f8f18102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.988+0000 7f8f1c946640 1 --2- 172.21.15.43:0/846777732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f181010f0 0x7f8f18102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.988+0000 7f8f1c946640 1 -- 172.21.15.43:0/846777732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f181a6fd0 con 0x7f8f18106170 2024-04-21T08:44:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.988+0000 7f8f1c946640 1 --2- 172.21.15.43:0/846777732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f18106170 0x7f8f18103070 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f8f08005fd0 tx=0x7f8f0800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.989+0000 7f8f0e7fc640 1 -- 172.21.15.43:0/846777732 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08017070 con 0x7f8f18106170 2024-04-21T08:44:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.989+0000 7f8f0e7fc640 1 -- 172.21.15.43:0/846777732 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f08009c30 con 0x7f8f18106170 2024-04-21T08:44:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.989+0000 7f8f1f3ab640 1 -- 172.21.15.43:0/846777732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f181a72c0 con 0x7f8f18106170 2024-04-21T08:44:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.989+0000 7f8f0e7fc640 1 -- 172.21.15.43:0/846777732 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08003d30 con 0x7f8f18106170 2024-04-21T08:44:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.989+0000 7f8f1f3ab640 1 -- 172.21.15.43:0/846777732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f181a7810 con 0x7f8f18106170 2024-04-21T08:44:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.991+0000 7f8f0e7fc640 1 -- 172.21.15.43:0/846777732 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8f08010460 con 0x7f8f18106170 2024-04-21T08:44:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.991+0000 7f8f21f9d740 1 Processor -- start 2024-04-21T08:44:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.992+0000 7f8f21f9d740 1 -- start start 2024-04-21T08:44:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.992+0000 7f8f21f9d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edf18d0 0x55941edf0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.992+0000 7f8f21f9d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55941edf0b60 0x55941edf0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.992+0000 7f8f21f9d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55941edfa0e0 0x55941edfc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.992+0000 7f8f21f9d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55941ed558d0 con 0x55941edf18d0 2024-04-21T08:44:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.992+0000 7f8f21f9d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55941ecf2e80 con 0x55941edfa0e0 2024-04-21T08:44:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.992+0000 7f8f21f9d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55941ed51890 con 0x55941edf0b60 2024-04-21T08:44:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.992+0000 7f8f1d948640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edf18d0 0x55941edf0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.992+0000 7f8f1d948640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edf18d0 0x55941edf0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33132/0 (socket says 172.21.15.43:33132) 2024-04-21T08:44:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.992+0000 7f8f1d147640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55941edf0b60 0x55941edf0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.992+0000 7f8f1d948640 1 -- 172.21.15.43:0/2793112566 learned_addr learned my addr 172.21.15.43:0/2793112566 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.992+0000 7f8f1c946640 1 --2- 172.21.15.43:0/2793112566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55941edfa0e0 0x55941edfc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.992+0000 7f8f0e7fc640 1 --2- 172.21.15.43:0/846777732 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8f040a3df0 0x7f8f040a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.992+0000 7f8f1d948640 1 -- 172.21.15.43:0/2793112566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55941edf0b60 msgr2=0x55941edf0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.992+0000 7f8f1d948640 1 --2- 172.21.15.43:0/2793112566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55941edf0b60 0x55941edf0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.992+0000 7f8f1d948640 1 -- 172.21.15.43:0/2793112566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55941edfa0e0 msgr2=0x55941edfc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.992+0000 7f8f1d948640 1 --2- 172.21.15.43:0/2793112566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55941edfa0e0 0x55941edfc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.992+0000 7f8f1d948640 1 -- 172.21.15.43:0/2793112566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55941ed14320 con 0x55941edf18d0 2024-04-21T08:44:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.992+0000 7f8f1d147640 1 --2- 172.21.15.43:0/2793112566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55941edf0b60 0x55941edf0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:44:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.992+0000 7f8f1d948640 1 --2- 172.21.15.43:0/846777732 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8f040a3df0 0x7f8f040a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.992+0000 7f8f0e7fc640 1 -- 172.21.15.43:0/846777732 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f8f080d08c0 con 0x7f8f18106170 2024-04-21T08:44:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.993+0000 7f8f1d948640 1 --2- 172.21.15.43:0/2793112566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edf18d0 0x55941edf0620 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f8f1002c700 tx=0x7f8f10009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f32be669c2ffd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.993+0000 7f8f037fe640 1 -- 172.21.15.43:0/2793112566 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10039040 con 0x55941edf18d0 2024-04-21T08:44:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.993+0000 7f8f037fe640 1 -- 172.21.15.43:0/2793112566 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f10003970 con 0x55941edf18d0 2024-04-21T08:44:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.993+0000 7f8f1d948640 1 --2- 172.21.15.43:0/846777732 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8f040a3df0 0x7f8f040a62b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8f10004af0 tx=0x7f8f10044000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.993+0000 7f8f037fe640 1 -- 172.21.15.43:0/2793112566 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10004390 con 0x55941edf18d0 2024-04-21T08:44:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.994+0000 7f8f21f9d740 1 -- 172.21.15.43:0/2793112566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edf18d0 msgr2=0x55941edf0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.994+0000 7f8f21f9d740 1 --2- 172.21.15.43:0/2793112566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edf18d0 0x55941edf0620 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f8f1002c700 tx=0x7f8f10009800 comp rx=0 tx=0).stop 2024-04-21T08:44:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.994+0000 7f8f21f9d740 1 -- 172.21.15.43:0/2793112566 shutdown_connections 2024-04-21T08:44:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.994+0000 7f8f21f9d740 1 --2- 172.21.15.43:0/2793112566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55941edfa0e0 0x55941edfc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.994+0000 7f8f21f9d740 1 --2- 172.21.15.43:0/2793112566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55941edf0b60 0x55941edf0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.994+0000 7f8f21f9d740 1 --2- 172.21.15.43:0/2793112566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edf18d0 0x55941edf0620 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.994+0000 7f8f21f9d740 1 -- 172.21.15.43:0/2793112566 >> 172.21.15.43:0/2793112566 conn(0x55941edb7c90 msgr2=0x55941edeb850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.994+0000 7f8f21f9d740 1 -- 172.21.15.43:0/2793112566 shutdown_connections 2024-04-21T08:44:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.994+0000 7f8f21f9d740 1 -- 172.21.15.43:0/2793112566 wait complete. 2024-04-21T08:44:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.994+0000 7f8f21f9d740 1 -- start start 2024-04-21T08:44:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.994+0000 7f8f21f9d740 1 Processor -- start 2024-04-21T08:44:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.995+0000 7f8f21f9d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55941edf0b60 0x55941edf99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.995+0000 7f8f21f9d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edf18d0 0x55941edf6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.995+0000 7f8f21f9d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55941edfa0e0 0x55941edf7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.995+0000 7f8f21f9d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55941edfceb0 con 0x55941edf18d0 2024-04-21T08:44:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.995+0000 7f8f21f9d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55941ecddeb0 con 0x55941edf0b60 2024-04-21T08:44:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.995+0000 7f8f1c946640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edf18d0 0x55941edf6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.995+0000 7f8f1c946640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edf18d0 0x55941edf6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33136/0 (socket says 172.21.15.43:33136) 2024-04-21T08:44:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.995+0000 7f8f1c946640 1 -- 172.21.15.43:0/54430510 learned_addr learned my addr 172.21.15.43:0/54430510 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.995+0000 7f8f21f9d740 1 -- 172.21.15.43:0/54430510 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55941edfd030 con 0x55941edfa0e0 2024-04-21T08:44:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.995+0000 7f8f1d147640 1 --2- 172.21.15.43:0/54430510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55941edf0b60 0x55941edf99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.995+0000 7f8f21f9d740 10 client.0 _ll_register_callbacks cb 0x55941ed0ab40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.995+0000 7f8f1d948640 1 --2- 172.21.15.43:0/54430510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55941edfa0e0 0x55941edf7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.995+0000 7f8f1c946640 1 -- 172.21.15.43:0/54430510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55941edfa0e0 msgr2=0x55941edf7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.995+0000 7f8f1c946640 1 --2- 172.21.15.43:0/54430510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55941edfa0e0 0x55941edf7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.995+0000 7f8f1c946640 1 -- 172.21.15.43:0/54430510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55941edf0b60 msgr2=0x55941edf99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.995+0000 7f8f1c946640 1 --2- 172.21.15.43:0/54430510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55941edf0b60 0x55941edf99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.995+0000 7f8f1c946640 1 -- 172.21.15.43:0/54430510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55941ea68210 con 0x55941edf18d0 2024-04-21T08:44:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.995+0000 7f8f1d948640 1 --2- 172.21.15.43:0/54430510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55941edfa0e0 0x55941edf7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.996+0000 7f8f1c946640 1 --2- 172.21.15.43:0/54430510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edf18d0 0x55941edf6df0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009610 tx=0x7f8f0809c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5992c4d202af7c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.996+0000 7f8f00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.996+0000 7f8f00ff9640 1 -- 172.21.15.43:0/54430510 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0809c490 con 0x55941edf18d0 2024-04-21T08:44:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.996+0000 7f8f00ff9640 1 -- 172.21.15.43:0/54430510 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f8f080cdec0 con 0x55941edf18d0 2024-04-21T08:44:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.996+0000 7f8f21f9d740 1 -- 172.21.15.43:0/54430510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f10003970 con 0x55941edf18d0 2024-04-21T08:44:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.996+0000 7f8f00ff9640 1 -- 172.21.15.43:0/54430510 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8f080a0840 con 0x55941edf18d0 2024-04-21T08:44:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.996+0000 7f8f21f9d740 10 client.18075 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.996+0000 7f8f21f9d740 1 -- 172.21.15.43:0/54430510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55941eb34620 con 0x55941edf18d0 2024-04-21T08:44:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.996+0000 7f8f00ff9640 1 -- 172.21.15.43:0/54430510 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f080a5100 con 0x55941edf18d0 2024-04-21T08:44:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.997+0000 7f8f00ff9640 1 -- 172.21.15.43:0/54430510 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8f0809b350 con 0x55941edf18d0 2024-04-21T08:44:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.997+0000 7f8f21f9d740 10 client.18075 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.997+0000 7f8f21f9d740 10 client.18075 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.997+0000 7f8f21f9d740 20 client.18075 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.997+0000 7f8f21f9d740 1 --2- 172.21.15.43:0/54430510 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55941ee0a190 0x55941ee0c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.997+0000 7f8f21f9d740 4 client.18075 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.997+0000 7f8f21f9d740 1 -- 172.21.15.43:0/54430510 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8f10003970 con 0x55941ee0a190 2024-04-21T08:44:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.997+0000 7f8f1d147640 1 --2- 172.21.15.43:0/54430510 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55941ee0a190 0x55941ee0c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.998+0000 7f8f00ff9640 10 client.18075 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.998+0000 7f8f1d147640 1 --2- 172.21.15.43:0/54430510 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55941ee0a190 0x55941ee0c580 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=689223a1027db1f server_cookie=5799cd350679542 in_seq=0 out_seq=0 2024-04-21T08:44:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.998+0000 7f8f00ff9640 1 -- 172.21.15.43:0/54430510 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f8f10003970 con 0x55941ee0a190 2024-04-21T08:44:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.998+0000 7f8f00ff9640 10 client.18075 handle_command_reply: tid=0 2024-04-21T08:44:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.998+0000 7f8f21f9d740 1 client.18075 shutdown 2024-04-21T08:44:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.998+0000 7f8f21f9d740 1 -- 172.21.15.43:0/54430510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edf18d0 msgr2=0x55941edf6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.998+0000 7f8f21f9d740 1 --2- 172.21.15.43:0/54430510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edf18d0 0x55941edf6df0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009610 tx=0x7f8f0809c2f0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.998+0000 7f8f21f9d740 1 -- 172.21.15.43:0/54430510 shutdown_connections 2024-04-21T08:44:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.998+0000 7f8f21f9d740 1 --2- 172.21.15.43:0/54430510 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55941ee0a190 0x55941ee0c580 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.998+0000 7f8f21f9d740 1 --2- 172.21.15.43:0/54430510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55941edfa0e0 0x55941edf7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.998+0000 7f8f21f9d740 1 --2- 172.21.15.43:0/54430510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edf18d0 0x55941edf6df0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.998+0000 7f8f21f9d740 1 --2- 172.21.15.43:0/54430510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55941edf0b60 0x55941edf99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.998+0000 7f8f00ff9640 0 client.18075 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.998+0000 7f8f21f9d740 1 -- 172.21.15.43:0/54430510 >> 172.21.15.43:0/54430510 conn(0x55941edb7c90 msgr2=0x55941eded5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.999+0000 7f8f21f9d740 1 -- 172.21.15.43:0/54430510 shutdown_connections 2024-04-21T08:44:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.999+0000 7f8f21f9d740 1 -- 172.21.15.43:0/54430510 wait complete. 2024-04-21T08:44:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:56.999+0000 7f8f21f9d740 20 client.18075 trim_cache size 0 max 16384 2024-04-21T08:44:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.013+0000 7f8f21f9d740 1 Processor -- start 2024-04-21T08:44:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.014+0000 7f8f21f9d740 1 -- start start 2024-04-21T08:44:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.014+0000 7f8f21f9d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55941edf0b60 0x55941ee05db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.014+0000 7f8f21f9d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edf18d0 0x55941edf0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.014+0000 7f8f21f9d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55941edfa0e0 0x55941ee11eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.014+0000 7f8f21f9d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8f181b1520 con 0x55941edf18d0 2024-04-21T08:44:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.014+0000 7f8f21f9d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f181b3060 con 0x55941edfa0e0 2024-04-21T08:44:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.014+0000 7f8f21f9d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8f181b1bf0 con 0x55941edf0b60 2024-04-21T08:44:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.014+0000 7f8f1c946640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edf18d0 0x55941edf0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.014+0000 7f8f1c946640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edf18d0 0x55941edf0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33140/0 (socket says 172.21.15.43:33140) 2024-04-21T08:44:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.014+0000 7f8f1c946640 1 -- 172.21.15.43:0/1051701688 learned_addr learned my addr 172.21.15.43:0/1051701688 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.014+0000 7f8f1d147640 1 --2- 172.21.15.43:0/1051701688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55941edf0b60 0x55941ee05db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.014+0000 7f8f1d948640 1 --2- 172.21.15.43:0/1051701688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55941edfa0e0 0x55941ee11eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.014+0000 7f8f1c946640 1 -- 172.21.15.43:0/1051701688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55941edf0b60 msgr2=0x55941ee05db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.014+0000 7f8f1c946640 1 --2- 172.21.15.43:0/1051701688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55941edf0b60 0x55941ee05db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.014+0000 7f8f1c946640 1 -- 172.21.15.43:0/1051701688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55941edfa0e0 msgr2=0x55941ee11eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.014+0000 7f8f1c946640 1 --2- 172.21.15.43:0/1051701688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55941edfa0e0 0x55941ee11eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.014+0000 7f8f1c946640 1 -- 172.21.15.43:0/1051701688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55941ee0cbd0 con 0x55941edf18d0 2024-04-21T08:44:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.014+0000 7f8f1c946640 1 --2- 172.21.15.43:0/1051701688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edf18d0 0x55941edf0080 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f8f080d0b10 tx=0x7f8f0809a9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2e247075fed9e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.015+0000 7f8f02ffd640 1 -- 172.21.15.43:0/1051701688 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08066070 con 0x55941edf18d0 2024-04-21T08:44:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.015+0000 7f8f02ffd640 1 -- 172.21.15.43:0/1051701688 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f080a5100 con 0x55941edf18d0 2024-04-21T08:44:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.015+0000 7f8f02ffd640 1 -- 172.21.15.43:0/1051701688 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0809b4b0 con 0x55941edf18d0 2024-04-21T08:44:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.015+0000 7f8f21f9d740 1 -- 172.21.15.43:0/1051701688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edf18d0 msgr2=0x55941edf0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.015+0000 7f8f21f9d740 1 --2- 172.21.15.43:0/1051701688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edf18d0 0x55941edf0080 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f8f080d0b10 tx=0x7f8f0809a9a0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.015+0000 7f8f21f9d740 1 -- 172.21.15.43:0/1051701688 shutdown_connections 2024-04-21T08:44:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.015+0000 7f8f21f9d740 1 --2- 172.21.15.43:0/1051701688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55941edfa0e0 0x55941ee11eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.015+0000 7f8f21f9d740 1 --2- 172.21.15.43:0/1051701688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edf18d0 0x55941edf0080 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.015+0000 7f8f21f9d740 1 --2- 172.21.15.43:0/1051701688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55941edf0b60 0x55941ee05db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.015+0000 7f8f21f9d740 1 -- 172.21.15.43:0/1051701688 >> 172.21.15.43:0/1051701688 conn(0x55941edb7c90 msgr2=0x55941edf2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.015+0000 7f8f21f9d740 1 -- 172.21.15.43:0/1051701688 shutdown_connections 2024-04-21T08:44:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.015+0000 7f8f21f9d740 1 -- 172.21.15.43:0/1051701688 wait complete. 2024-04-21T08:44:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.016+0000 7f8f21f9d740 1 -- start start 2024-04-21T08:44:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.016+0000 7f8f21f9d740 1 Processor -- start 2024-04-21T08:44:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.016+0000 7f8f21f9d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55941edf0b60 0x55941edf3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.016+0000 7f8f21f9d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55941edf18d0 0x55941edf4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.016+0000 7f8f21f9d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edfa0e0 0x55941edf4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.016+0000 7f8f21f9d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8f181b2990 con 0x55941edfa0e0 2024-04-21T08:44:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.016+0000 7f8f21f9d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f181b22c0 con 0x55941edf18d0 2024-04-21T08:44:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.016+0000 7f8f21f9d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8f181af500 con 0x55941edf0b60 2024-04-21T08:44:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.016+0000 7f8f21f9d740 10 client.0 _ll_register_callbacks cb 0x7f8f08005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.016+0000 7f8f1d948640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edfa0e0 0x55941edf4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.016+0000 7f8f1d948640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edfa0e0 0x55941edf4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33144/0 (socket says 172.21.15.43:33144) 2024-04-21T08:44:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.016+0000 7f8f1d948640 1 -- 172.21.15.43:0/2129082135 learned_addr learned my addr 172.21.15.43:0/2129082135 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.017+0000 7f8f1d147640 1 --2- 172.21.15.43:0/2129082135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55941edf0b60 0x55941edf3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.017+0000 7f8f1c946640 1 --2- 172.21.15.43:0/2129082135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55941edf18d0 0x55941edf4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.017+0000 7f8f1d948640 1 -- 172.21.15.43:0/2129082135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55941edf0b60 msgr2=0x55941edf3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.017+0000 7f8f1d948640 1 --2- 172.21.15.43:0/2129082135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55941edf0b60 0x55941edf3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.017+0000 7f8f1d948640 1 -- 172.21.15.43:0/2129082135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55941edf18d0 msgr2=0x55941edf4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.017+0000 7f8f1d948640 1 --2- 172.21.15.43:0/2129082135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55941edf18d0 0x55941edf4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.017+0000 7f8f1d948640 1 -- 172.21.15.43:0/2129082135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55941ee6f720 con 0x55941edfa0e0 2024-04-21T08:44:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.017+0000 7f8f1d147640 1 --2- 172.21.15.43:0/2129082135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55941edf0b60 0x55941edf3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.017+0000 7f8f1c946640 1 --2- 172.21.15.43:0/2129082135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55941edf18d0 0x55941edf4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.017+0000 7f8f1d948640 1 --2- 172.21.15.43:0/2129082135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edfa0e0 0x55941edf4550 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009430 tx=0x7f8f10032f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41b0c2e67111d78 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.017+0000 7f8f00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.017+0000 7f8f00ff9640 1 -- 172.21.15.43:0/2129082135 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10039070 con 0x55941edfa0e0 2024-04-21T08:44:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.017+0000 7f8f21f9d740 1 -- 172.21.15.43:0/2129082135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55941edf0850 con 0x55941edfa0e0 2024-04-21T08:44:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.017+0000 7f8f00ff9640 1 -- 172.21.15.43:0/2129082135 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f8f100045f0 con 0x55941edfa0e0 2024-04-21T08:44:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.017+0000 7f8f00ff9640 1 -- 172.21.15.43:0/2129082135 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8f10060050 con 0x55941edfa0e0 2024-04-21T08:44:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.017+0000 7f8f21f9d740 10 client.18081 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.017+0000 7f8f21f9d740 1 -- 172.21.15.43:0/2129082135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55941ee70a00 con 0x55941edfa0e0 2024-04-21T08:44:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.017+0000 7f8f00ff9640 1 -- 172.21.15.43:0/2129082135 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55941edf0850 con 0x55941edfa0e0 2024-04-21T08:44:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.018+0000 7f8f00ff9640 1 -- 172.21.15.43:0/2129082135 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8f10048020 con 0x55941edfa0e0 2024-04-21T08:44:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.018+0000 7f8f21f9d740 10 client.18081 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.018+0000 7f8f21f9d740 10 client.18081 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.018+0000 7f8f21f9d740 20 client.18081 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.018+0000 7f8f21f9d740 1 --2- 172.21.15.43:0/2129082135 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55941ee70c90 0x55941ee730b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.018+0000 7f8f21f9d740 4 client.18081 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:44:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.018+0000 7f8f21f9d740 1 -- 172.21.15.43:0/2129082135 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55941ed69880 con 0x55941ee70c90 2024-04-21T08:44:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.018+0000 7f8f1d147640 1 --2- 172.21.15.43:0/2129082135 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55941ee70c90 0x55941ee730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.018+0000 7f8f1d147640 1 --2- 172.21.15.43:0/2129082135 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55941ee70c90 0x55941ee730b0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f248a9a337e885a server_cookie=8c79a28ad9593dd in_seq=0 out_seq=0 2024-04-21T08:44:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.018+0000 7f8f00ff9640 10 client.18081 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.033+0000 7f8f00ff9640 1 -- 172.21.15.43:0/2129082135 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55941ed69880 con 0x55941ee70c90 2024-04-21T08:44:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.033+0000 7f8f00ff9640 10 client.18081 handle_command_reply: tid=0 2024-04-21T08:44:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.033+0000 7f8f21f9d740 1 client.18081 shutdown 2024-04-21T08:44:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.033+0000 7f8f21f9d740 1 -- 172.21.15.43:0/2129082135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edfa0e0 msgr2=0x55941edf4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.033+0000 7f8f21f9d740 1 --2- 172.21.15.43:0/2129082135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edfa0e0 0x55941edf4550 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009430 tx=0x7f8f10032f80 comp rx=0 tx=0).stop 2024-04-21T08:44:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.033+0000 7f8f21f9d740 1 -- 172.21.15.43:0/2129082135 shutdown_connections 2024-04-21T08:44:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.033+0000 7f8f21f9d740 1 --2- 172.21.15.43:0/2129082135 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55941ee70c90 0x55941ee730b0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.033+0000 7f8f21f9d740 1 --2- 172.21.15.43:0/2129082135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55941edfa0e0 0x55941edf4550 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.033+0000 7f8f21f9d740 1 --2- 172.21.15.43:0/2129082135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55941edf18d0 0x55941edf4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.033+0000 7f8f21f9d740 1 --2- 172.21.15.43:0/2129082135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55941edf0b60 0x55941edf3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.033+0000 7f8f21f9d740 1 -- 172.21.15.43:0/2129082135 >> 172.21.15.43:0/2129082135 conn(0x55941edb7c90 msgr2=0x55941edf53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.034+0000 7f8f00ff9640 0 client.18081 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.034+0000 7f8f21f9d740 1 -- 172.21.15.43:0/2129082135 shutdown_connections 2024-04-21T08:44:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.034+0000 7f8f21f9d740 1 -- 172.21.15.43:0/2129082135 wait complete. 2024-04-21T08:44:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.034+0000 7f8f21f9d740 20 client.18081 trim_cache size 0 max 16384 2024-04-21T08:44:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:44:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:44:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:44:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:44:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.034+0000 7f8f1f3ab640 1 -- 172.21.15.43:0/846777732 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8f040a3df0 msgr2=0x7f8f040a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.034+0000 7f8f1f3ab640 1 --2- 172.21.15.43:0/846777732 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8f040a3df0 0x7f8f040a62b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8f10004af0 tx=0x7f8f10044000 comp rx=0 tx=0).stop 2024-04-21T08:44:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.034+0000 7f8f1f3ab640 1 -- 172.21.15.43:0/846777732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f18106170 msgr2=0x7f8f18103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.034+0000 7f8f1f3ab640 1 --2- 172.21.15.43:0/846777732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f18106170 0x7f8f18103070 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f8f08005fd0 tx=0x7f8f0800dde0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.034+0000 7f8f1f3ab640 1 -- 172.21.15.43:0/846777732 shutdown_connections 2024-04-21T08:44:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.034+0000 7f8f1f3ab640 1 --2- 172.21.15.43:0/846777732 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8f040a3df0 0x7f8f040a62b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.034+0000 7f8f1f3ab640 1 --2- 172.21.15.43:0/846777732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f181035b0 0x7f8f181a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.034+0000 7f8f1f3ab640 1 --2- 172.21.15.43:0/846777732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f18106170 0x7f8f18103070 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.034+0000 7f8f1f3ab640 1 --2- 172.21.15.43:0/846777732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f181010f0 0x7f8f18102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.034+0000 7f8f1f3ab640 1 -- 172.21.15.43:0/846777732 >> 172.21.15.43:0/846777732 conn(0x7f8f18077fd0 msgr2=0x7f8f18069990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.034+0000 7f8f1f3ab640 1 -- 172.21.15.43:0/846777732 shutdown_connections 2024-04-21T08:44:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:44:57.035+0000 7f8f1f3ab640 1 -- 172.21.15.43:0/846777732 wait complete. 2024-04-21T08:44:57.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.222+0000 7f563ad7e640 1 -- 172.21.15.43:0/2480823216 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f563402d5c0 con 0x7f5634069aa0 2024-04-21T08:44:57.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.222+0000 7f5631ffb640 1 -- 172.21.15.43:0/2480823216 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7f5620009de0 con 0x7f5634069aa0 2024-04-21T08:44:57.223 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:44:57.223 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:44:57.223 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:44:57.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.225+0000 7f563ad7e640 1 -- 172.21.15.43:0/2480823216 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f56240a3d20 msgr2=0x7f56240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.225+0000 7f563ad7e640 1 --2- 172.21.15.43:0/2480823216 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f56240a3d20 0x7f56240a61e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5628000c00 tx=0x7f5628005dd0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.225+0000 7f563ad7e640 1 -- 172.21.15.43:0/2480823216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5634069aa0 msgr2=0x7f56341017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.225+0000 7f563ad7e640 1 --2- 172.21.15.43:0/2480823216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5634069aa0 0x7f56341017b0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f562000d950 tx=0x7f562000de10 comp rx=0 tx=0).stop 2024-04-21T08:44:57.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.225+0000 7f563ad7e640 1 -- 172.21.15.43:0/2480823216 shutdown_connections 2024-04-21T08:44:57.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.225+0000 7f563ad7e640 1 --2- 172.21.15.43:0/2480823216 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f56240a3d20 0x7f56240a61e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.225+0000 7f563ad7e640 1 --2- 172.21.15.43:0/2480823216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5634106f50 0x7f5634101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.225+0000 7f563ad7e640 1 --2- 172.21.15.43:0/2480823216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5634069aa0 0x7f56341017b0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.225+0000 7f563ad7e640 1 --2- 172.21.15.43:0/2480823216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56340690e0 0x7f5634101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.225+0000 7f563ad7e640 1 -- 172.21.15.43:0/2480823216 >> 172.21.15.43:0/2480823216 conn(0x7f5634077e30 msgr2=0x7f56340fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.225+0000 7f563ad7e640 1 -- 172.21.15.43:0/2480823216 shutdown_connections 2024-04-21T08:44:57.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.225+0000 7f563ad7e640 1 -- 172.21.15.43:0/2480823216 wait complete. 2024-04-21T08:44:57.236 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:44:57.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.340+0000 7fe968132640 1 -- 172.21.15.43:0/2813722597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe960106160 msgr2=0x7fe96010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.340+0000 7fe968132640 1 --2- 172.21.15.43:0/2813722597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe960106160 0x7fe96010dd30 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fe958009ca0 tx=0x7fe95802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.340+0000 7fe968132640 1 -- 172.21.15.43:0/2813722597 shutdown_connections 2024-04-21T08:44:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.340+0000 7fe968132640 1 --2- 172.21.15.43:0/2813722597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe960106160 0x7fe96010dd30 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.340+0000 7fe968132640 1 --2- 172.21.15.43:0/2813722597 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe960101a40 0x7fe960105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.340+0000 7fe968132640 1 --2- 172.21.15.43:0/2813722597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe960101090 0x7fe960101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.340+0000 7fe968132640 1 -- 172.21.15.43:0/2813722597 >> 172.21.15.43:0/2813722597 conn(0x7fe960077f50 msgr2=0x7fe960078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.340+0000 7fe968132640 1 -- 172.21.15.43:0/2813722597 shutdown_connections 2024-04-21T08:44:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.340+0000 7fe968132640 1 -- 172.21.15.43:0/2813722597 wait complete. 2024-04-21T08:44:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.340+0000 7fe968132640 1 Processor -- start 2024-04-21T08:44:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.340+0000 7fe968132640 1 -- start start 2024-04-21T08:44:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.341+0000 7fe968132640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe960101090 0x7fe960199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.341+0000 7fe968132640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe960106160 0x7fe960199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.341+0000 7fe968132640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe96019a040 0x7fe9601a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.341+0000 7fe968132640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe9601110d0 con 0x7fe960106160 2024-04-21T08:44:57.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.341+0000 7fe968132640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe960110f50 con 0x7fe96019a040 2024-04-21T08:44:57.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.341+0000 7fe968132640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe960111250 con 0x7fe960101090 2024-04-21T08:44:57.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.341+0000 7fe9656cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe960106160 0x7fe960199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.341+0000 7fe9656cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe960106160 0x7fe960199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33160/0 (socket says 172.21.15.43:33160) 2024-04-21T08:44:57.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.341+0000 7fe9656cd640 1 -- 172.21.15.43:0/2018631849 learned_addr learned my addr 172.21.15.43:0/2018631849 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.341+0000 7fe965ece640 1 --2- 172.21.15.43:0/2018631849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe960101090 0x7fe960199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.341+0000 7fe9666cf640 1 --2- 172.21.15.43:0/2018631849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe96019a040 0x7fe9601a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.341+0000 7fe9656cd640 1 -- 172.21.15.43:0/2018631849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe960101090 msgr2=0x7fe960199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.341+0000 7fe9656cd640 1 --2- 172.21.15.43:0/2018631849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe960101090 0x7fe960199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.341+0000 7fe9656cd640 1 -- 172.21.15.43:0/2018631849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe96019a040 msgr2=0x7fe9601a45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.341+0000 7fe9656cd640 1 --2- 172.21.15.43:0/2018631849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe96019a040 0x7fe9601a45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.341+0000 7fe9656cd640 1 -- 172.21.15.43:0/2018631849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9601a4df0 con 0x7fe960106160 2024-04-21T08:44:57.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.341+0000 7fe9666cf640 1 --2- 172.21.15.43:0/2018631849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe96019a040 0x7fe9601a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:57.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.341+0000 7fe9656cd640 1 --2- 172.21.15.43:0/2018631849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe960106160 0x7fe960199ab0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fe95000d740 tx=0x7fe95000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.341+0000 7fe956ffd640 1 -- 172.21.15.43:0/2018631849 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950017070 con 0x7fe960106160 2024-04-21T08:44:57.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.341+0000 7fe956ffd640 1 -- 172.21.15.43:0/2018631849 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe95000de50 con 0x7fe960106160 2024-04-21T08:44:57.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.341+0000 7fe956ffd640 1 -- 172.21.15.43:0/2018631849 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950004ce0 con 0x7fe960106160 2024-04-21T08:44:57.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.341+0000 7fe968132640 1 -- 172.21.15.43:0/2018631849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9601a50e0 con 0x7fe960106160 2024-04-21T08:44:57.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.341+0000 7fe968132640 1 -- 172.21.15.43:0/2018631849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9601a5630 con 0x7fe960106160 2024-04-21T08:44:57.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.343+0000 7fe956ffd640 1 -- 172.21.15.43:0/2018631849 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe950004e80 con 0x7fe960106160 2024-04-21T08:44:57.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.344+0000 7fe96ae02740 1 Processor -- start 2024-04-21T08:44:57.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.344+0000 7fe96ae02740 1 -- start start 2024-04-21T08:44:57.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.344+0000 7fe96ae02740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641172d5330 0x5641172d5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.344+0000 7fe96ae02740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641172d5ce0 0x5641172de820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.344+0000 7fe96ae02740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172dee40 0x5641172df2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.344+0000 7fe96ae02740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56411723a1f0 con 0x5641172dee40 2024-04-21T08:44:57.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.344+0000 7fe96ae02740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5641171d74b0 con 0x5641172d5330 2024-04-21T08:44:57.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.344+0000 7fe96ae02740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5641172361b0 con 0x5641172d5ce0 2024-04-21T08:44:57.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.344+0000 7fe9656cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172dee40 0x5641172df2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.344+0000 7fe956ffd640 1 --2- 172.21.15.43:0/2018631849 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe94c0a3da0 0x7fe94c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.344+0000 7fe9656cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172dee40 0x5641172df2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33170/0 (socket says 172.21.15.43:33170) 2024-04-21T08:44:57.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.344+0000 7fe9656cd640 1 -- 172.21.15.43:0/2176252266 learned_addr learned my addr 172.21.15.43:0/2176252266 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.344+0000 7fe965ece640 1 --2- 172.21.15.43:0/2176252266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641172d5ce0 0x5641172de820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.344+0000 7fe9666cf640 1 --2- 172.21.15.43:0/2176252266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641172d5330 0x5641172d5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.345+0000 7fe9666cf640 1 --2- 172.21.15.43:0/2018631849 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe94c0a3da0 0x7fe94c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.345+0000 7fe956ffd640 1 -- 172.21.15.43:0/2018631849 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fe9500d0470 con 0x7fe960106160 2024-04-21T08:44:57.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.345+0000 7fe965ece640 1 -- 172.21.15.43:0/2176252266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641172d5330 msgr2=0x5641172d5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.345+0000 7fe965ece640 1 --2- 172.21.15.43:0/2176252266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641172d5330 0x5641172d5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.345+0000 7fe965ece640 1 -- 172.21.15.43:0/2176252266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172dee40 msgr2=0x5641172df2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.345+0000 7fe965ece640 1 --2- 172.21.15.43:0/2176252266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172dee40 0x5641172df2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.345+0000 7fe965ece640 1 -- 172.21.15.43:0/2176252266 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641171c91b0 con 0x5641172d5ce0 2024-04-21T08:44:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.345+0000 7fe9656cd640 1 --2- 172.21.15.43:0/2176252266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172dee40 0x5641172df2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:44:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.345+0000 7fe9666cf640 1 --2- 172.21.15.43:0/2018631849 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe94c0a3da0 0x7fe94c0a6260 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe958033000 tx=0x7fe9580095c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.345+0000 7fe965ece640 1 --2- 172.21.15.43:0/2176252266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641172d5ce0 0x5641172de820 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe95c005fd0 tx=0x7fe95c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ee351bb608a4cb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.345+0000 7fe94bfff640 1 -- 172.21.15.43:0/2176252266 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c01c480 con 0x5641172d5ce0 2024-04-21T08:44:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.345+0000 7fe94bfff640 1 -- 172.21.15.43:0/2176252266 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe95c01cad0 con 0x5641172d5ce0 2024-04-21T08:44:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.345+0000 7fe94bfff640 1 -- 172.21.15.43:0/2176252266 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c011af0 con 0x5641172d5ce0 2024-04-21T08:44:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.346+0000 7fe96ae02740 1 -- 172.21.15.43:0/2176252266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641172d5ce0 msgr2=0x5641172de820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.346+0000 7fe96ae02740 1 --2- 172.21.15.43:0/2176252266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641172d5ce0 0x5641172de820 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe95c005fd0 tx=0x7fe95c0107a0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.346+0000 7fe96ae02740 1 -- 172.21.15.43:0/2176252266 shutdown_connections 2024-04-21T08:44:57.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.346+0000 7fe96ae02740 1 --2- 172.21.15.43:0/2176252266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172dee40 0x5641172df2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.346+0000 7fe96ae02740 1 --2- 172.21.15.43:0/2176252266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641172d5ce0 0x5641172de820 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.346+0000 7fe96ae02740 1 --2- 172.21.15.43:0/2176252266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641172d5330 0x5641172d5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.346+0000 7fe96ae02740 1 -- 172.21.15.43:0/2176252266 >> 172.21.15.43:0/2176252266 conn(0x5641172cfba0 msgr2=0x5641172cff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.346+0000 7fe96ae02740 1 -- 172.21.15.43:0/2176252266 shutdown_connections 2024-04-21T08:44:57.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.346+0000 7fe96ae02740 1 -- 172.21.15.43:0/2176252266 wait complete. 2024-04-21T08:44:57.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.346+0000 7fe96ae02740 1 -- start start 2024-04-21T08:44:57.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.346+0000 7fe96ae02740 1 Processor -- start 2024-04-21T08:44:57.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.347+0000 7fe96ae02740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641172d5330 0x5641172e5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.347+0000 7fe96ae02740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641172d5ce0 0x5641172e25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.347+0000 7fe96ae02740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172dee40 0x5641172e2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.347+0000 7fe96ae02740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5641172e5dc0 con 0x5641172dee40 2024-04-21T08:44:57.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.347+0000 7fe96ae02740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5641171f3500 con 0x5641172d5ce0 2024-04-21T08:44:57.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.347+0000 7fe96ae02740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5641172e5f40 con 0x5641172d5330 2024-04-21T08:44:57.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.347+0000 7fe96ae02740 10 client.0 _ll_register_callbacks cb 0x5641171c1b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:57.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.347+0000 7fe9666cf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172dee40 0x5641172e2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.347+0000 7fe9666cf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172dee40 0x5641172e2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33176/0 (socket says 172.21.15.43:33176) 2024-04-21T08:44:57.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.347+0000 7fe9666cf640 1 -- 172.21.15.43:0/2924411437 learned_addr learned my addr 172.21.15.43:0/2924411437 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.347+0000 7fe9656cd640 1 --2- 172.21.15.43:0/2924411437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641172d5ce0 0x5641172e25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.347+0000 7fe965ece640 1 --2- 172.21.15.43:0/2924411437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641172d5330 0x5641172e5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.347+0000 7fe9666cf640 1 -- 172.21.15.43:0/2924411437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641172d5330 msgr2=0x5641172e5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.347+0000 7fe9666cf640 1 --2- 172.21.15.43:0/2924411437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641172d5330 0x5641172e5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.347+0000 7fe9666cf640 1 -- 172.21.15.43:0/2924411437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641172d5ce0 msgr2=0x5641172e25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.347+0000 7fe9666cf640 1 --2- 172.21.15.43:0/2924411437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641172d5ce0 0x5641172e25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.347+0000 7fe9666cf640 1 -- 172.21.15.43:0/2924411437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564116f4c3f0 con 0x5641172dee40 2024-04-21T08:44:57.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.348+0000 7fe9666cf640 1 --2- 172.21.15.43:0/2924411437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172dee40 0x5641172e2b10 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fe958036000 tx=0x7fe9580093a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b5ce9bd74aaad7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.348+0000 7fe9497fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:57.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.348+0000 7fe9497fa640 1 -- 172.21.15.43:0/2924411437 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958053060 con 0x5641172dee40 2024-04-21T08:44:57.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.348+0000 7fe9497fa640 1 -- 172.21.15.43:0/2924411437 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fe958004450 con 0x5641172dee40 2024-04-21T08:44:57.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.348+0000 7fe96ae02740 1 -- 172.21.15.43:0/2924411437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe95c01cad0 con 0x5641172dee40 2024-04-21T08:44:57.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.348+0000 7fe9497fa640 1 -- 172.21.15.43:0/2924411437 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe95805a050 con 0x5641172dee40 2024-04-21T08:44:57.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.348+0000 7fe96ae02740 10 client.18093 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:57.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.348+0000 7fe96ae02740 1 -- 172.21.15.43:0/2924411437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5641170167d0 con 0x5641172dee40 2024-04-21T08:44:57.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.348+0000 7fe9497fa640 1 -- 172.21.15.43:0/2924411437 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe95c01cad0 con 0x5641172dee40 2024-04-21T08:44:57.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.348+0000 7fe9497fa640 1 -- 172.21.15.43:0/2924411437 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe95804d070 con 0x5641172dee40 2024-04-21T08:44:57.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.348+0000 7fe96ae02740 10 client.18093 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:57.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.348+0000 7fe96ae02740 10 client.18093 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:44:57.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.348+0000 7fe96ae02740 20 client.18093 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:57.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.348+0000 7fe96ae02740 1 --2- 172.21.15.43:0/2924411437 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5641172f30b0 0x5641172f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.348+0000 7fe96ae02740 4 client.18093 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:57.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.348+0000 7fe96ae02740 1 -- 172.21.15.43:0/2924411437 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641171d9930 con 0x5641172f30b0 2024-04-21T08:44:57.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.349+0000 7fe965ece640 1 --2- 172.21.15.43:0/2924411437 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5641172f30b0 0x5641172f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.349+0000 7fe965ece640 1 --2- 172.21.15.43:0/2924411437 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5641172f30b0 0x5641172f54e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ffef527a72665f92 server_cookie=910458e124bf3455 in_seq=0 out_seq=0 2024-04-21T08:44:57.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.349+0000 7fe9497fa640 10 client.18093 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:44:57.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.350+0000 7fe9497fa640 1 -- 172.21.15.43:0/2924411437 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5641171d9930 con 0x5641172f30b0 2024-04-21T08:44:57.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.350+0000 7fe9497fa640 10 client.18093 handle_command_reply: tid=0 2024-04-21T08:44:57.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.350+0000 7fe96ae02740 1 client.18093 shutdown 2024-04-21T08:44:57.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.350+0000 7fe96ae02740 1 -- 172.21.15.43:0/2924411437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172dee40 msgr2=0x5641172e2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.350+0000 7fe96ae02740 1 --2- 172.21.15.43:0/2924411437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172dee40 0x5641172e2b10 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fe958036000 tx=0x7fe9580093a0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.350+0000 7fe96ae02740 1 -- 172.21.15.43:0/2924411437 shutdown_connections 2024-04-21T08:44:57.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.350+0000 7fe96ae02740 1 --2- 172.21.15.43:0/2924411437 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5641172f30b0 0x5641172f54e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.350+0000 7fe96ae02740 1 --2- 172.21.15.43:0/2924411437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172dee40 0x5641172e2b10 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.350+0000 7fe9497fa640 0 client.18093 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:44:57.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.350+0000 7fe96ae02740 1 --2- 172.21.15.43:0/2924411437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641172d5ce0 0x5641172e25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.350+0000 7fe96ae02740 1 --2- 172.21.15.43:0/2924411437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641172d5330 0x5641172e5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.350+0000 7fe96ae02740 1 -- 172.21.15.43:0/2924411437 >> 172.21.15.43:0/2924411437 conn(0x5641172cfba0 msgr2=0x5641172d18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.350+0000 7fe96ae02740 1 -- 172.21.15.43:0/2924411437 shutdown_connections 2024-04-21T08:44:57.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.350+0000 7fe96ae02740 1 -- 172.21.15.43:0/2924411437 wait complete. 2024-04-21T08:44:57.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.350+0000 7fe96ae02740 20 client.18093 trim_cache size 0 max 16384 2024-04-21T08:44:57.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.364+0000 7fe96ae02740 1 Processor -- start 2024-04-21T08:44:57.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.364+0000 7fe96ae02740 1 -- start start 2024-04-21T08:44:57.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.364+0000 7fe96ae02740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172cfba0 0x5641172eecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.364+0000 7fe96ae02740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641172d5330 0x5641172cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.364+0000 7fe96ae02740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641172d5ce0 0x5641172e4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.364+0000 7fe96ae02740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe9601ad1f0 con 0x5641172cfba0 2024-04-21T08:44:57.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.364+0000 7fe96ae02740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe9601aed30 con 0x5641172d5ce0 2024-04-21T08:44:57.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.364+0000 7fe96ae02740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9601ad8c0 con 0x5641172d5330 2024-04-21T08:44:57.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.364+0000 7fe965ece640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172cfba0 0x5641172eecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.364+0000 7fe965ece640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172cfba0 0x5641172eecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33178/0 (socket says 172.21.15.43:33178) 2024-04-21T08:44:57.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.364+0000 7fe965ece640 1 -- 172.21.15.43:0/1613072210 learned_addr learned my addr 172.21.15.43:0/1613072210 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.364+0000 7fe9666cf640 1 --2- 172.21.15.43:0/1613072210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641172d5ce0 0x5641172e4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.364+0000 7fe9656cd640 1 --2- 172.21.15.43:0/1613072210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641172d5330 0x5641172cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.364+0000 7fe965ece640 1 -- 172.21.15.43:0/1613072210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641172d5330 msgr2=0x5641172cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.364+0000 7fe965ece640 1 --2- 172.21.15.43:0/1613072210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641172d5330 0x5641172cea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.364+0000 7fe965ece640 1 -- 172.21.15.43:0/1613072210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641172d5ce0 msgr2=0x5641172e4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.364+0000 7fe965ece640 1 --2- 172.21.15.43:0/1613072210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641172d5ce0 0x5641172e4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.364+0000 7fe965ece640 1 -- 172.21.15.43:0/1613072210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641172d5a80 con 0x5641172cfba0 2024-04-21T08:44:57.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.364+0000 7fe9666cf640 1 --2- 172.21.15.43:0/1613072210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641172d5ce0 0x5641172e4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:57.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.364+0000 7fe965ece640 1 --2- 172.21.15.43:0/1613072210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172cfba0 0x5641172eecc0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00bbb0 tx=0x7fe95c00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64382b29b67d6d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.365+0000 7fe94b7fe640 1 -- 172.21.15.43:0/1613072210 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c00bda0 con 0x5641172cfba0 2024-04-21T08:44:57.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.365+0000 7fe94b7fe640 1 -- 172.21.15.43:0/1613072210 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe95c005040 con 0x5641172cfba0 2024-04-21T08:44:57.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.365+0000 7fe94b7fe640 1 -- 172.21.15.43:0/1613072210 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c0119b0 con 0x5641172cfba0 2024-04-21T08:44:57.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.365+0000 7fe96ae02740 1 -- 172.21.15.43:0/1613072210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172cfba0 msgr2=0x5641172eecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.365+0000 7fe96ae02740 1 --2- 172.21.15.43:0/1613072210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172cfba0 0x5641172eecc0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00bbb0 tx=0x7fe95c00b410 comp rx=0 tx=0).stop 2024-04-21T08:44:57.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.365+0000 7fe96ae02740 1 -- 172.21.15.43:0/1613072210 shutdown_connections 2024-04-21T08:44:57.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.365+0000 7fe96ae02740 1 --2- 172.21.15.43:0/1613072210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641172d5ce0 0x5641172e4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.365+0000 7fe96ae02740 1 --2- 172.21.15.43:0/1613072210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641172d5330 0x5641172cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.365+0000 7fe96ae02740 1 --2- 172.21.15.43:0/1613072210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172cfba0 0x5641172eecc0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.365+0000 7fe96ae02740 1 -- 172.21.15.43:0/1613072210 >> 172.21.15.43:0/1613072210 conn(0x5641172de980 msgr2=0x5641172ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.365+0000 7fe96ae02740 1 -- 172.21.15.43:0/1613072210 shutdown_connections 2024-04-21T08:44:57.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.365+0000 7fe96ae02740 1 -- 172.21.15.43:0/1613072210 wait complete. 2024-04-21T08:44:57.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.365+0000 7fe96ae02740 1 -- start start 2024-04-21T08:44:57.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.366+0000 7fe96ae02740 1 Processor -- start 2024-04-21T08:44:57.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.366+0000 7fe96ae02740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641172cfba0 0x5641172eb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.366+0000 7fe96ae02740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172d5330 0x5641172e41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.366+0000 7fe96ae02740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641172d5ce0 0x5641172e4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.366+0000 7fe96ae02740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe9601ae660 con 0x5641172d5330 2024-04-21T08:44:57.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.366+0000 7fe96ae02740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe9601adf90 con 0x5641172cfba0 2024-04-21T08:44:57.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.366+0000 7fe96ae02740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9601ab1d0 con 0x5641172d5ce0 2024-04-21T08:44:57.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.366+0000 7fe96ae02740 10 client.0 _ll_register_callbacks cb 0x7fe958032a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:57.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.366+0000 7fe9656cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172d5330 0x5641172e41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.366+0000 7fe9656cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172d5330 0x5641172e41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33186/0 (socket says 172.21.15.43:33186) 2024-04-21T08:44:57.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.366+0000 7fe9656cd640 1 -- 172.21.15.43:0/3113193431 learned_addr learned my addr 172.21.15.43:0/3113193431 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.366+0000 7fe965ece640 1 --2- 172.21.15.43:0/3113193431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641172cfba0 0x5641172eb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.366+0000 7fe9666cf640 1 --2- 172.21.15.43:0/3113193431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641172d5ce0 0x5641172e4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.366+0000 7fe9656cd640 1 -- 172.21.15.43:0/3113193431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641172d5ce0 msgr2=0x5641172e4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.366+0000 7fe9656cd640 1 --2- 172.21.15.43:0/3113193431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641172d5ce0 0x5641172e4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.366+0000 7fe9656cd640 1 -- 172.21.15.43:0/3113193431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641172cfba0 msgr2=0x5641172eb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.366+0000 7fe9656cd640 1 --2- 172.21.15.43:0/3113193431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641172cfba0 0x5641172eb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.366+0000 7fe9656cd640 1 -- 172.21.15.43:0/3113193431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641172e5020 con 0x5641172d5330 2024-04-21T08:44:57.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.366+0000 7fe965ece640 1 --2- 172.21.15.43:0/3113193431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641172cfba0 0x5641172eb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:57.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.366+0000 7fe9656cd640 1 --2- 172.21.15.43:0/3113193431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172d5330 0x5641172e41f0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fe950009610 tx=0x7fe950005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ba4bf088ba33b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.366+0000 7fe9497fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:57.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.367+0000 7fe9497fa640 1 -- 172.21.15.43:0/3113193431 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95007ae60 con 0x5641172d5330 2024-04-21T08:44:57.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.367+0000 7fe9497fa640 1 -- 172.21.15.43:0/3113193431 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fe9500cda70 con 0x5641172d5330 2024-04-21T08:44:57.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.367+0000 7fe96ae02740 1 -- 172.21.15.43:0/3113193431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe95c005040 con 0x5641172d5330 2024-04-21T08:44:57.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.367+0000 7fe9497fa640 1 -- 172.21.15.43:0/3113193431 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe950020040 con 0x5641172d5330 2024-04-21T08:44:57.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.367+0000 7fe96ae02740 10 client.18099 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:57.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.367+0000 7fe96ae02740 1 -- 172.21.15.43:0/3113193431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5641173597b0 con 0x5641172d5330 2024-04-21T08:44:57.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.367+0000 7fe9497fa640 1 -- 172.21.15.43:0/3113193431 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe95c005040 con 0x5641172d5330 2024-04-21T08:44:57.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.367+0000 7fe9497fa640 1 -- 172.21.15.43:0/3113193431 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe9500986b0 con 0x5641172d5330 2024-04-21T08:44:57.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.367+0000 7fe96ae02740 10 client.18099 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:57.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.367+0000 7fe96ae02740 10 client.18099 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:44:57.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.367+0000 7fe96ae02740 20 client.18099 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:57.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.367+0000 7fe96ae02740 1 --2- 172.21.15.43:0/3113193431 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5641172dee40 0x56411735bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.367+0000 7fe96ae02740 4 client.18099 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:44:57.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.367+0000 7fe96ae02740 1 -- 172.21.15.43:0/3113193431 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564117247c10 con 0x5641172dee40 2024-04-21T08:44:57.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.367+0000 7fe965ece640 1 --2- 172.21.15.43:0/3113193431 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5641172dee40 0x56411735bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.368+0000 7fe965ece640 1 --2- 172.21.15.43:0/3113193431 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5641172dee40 0x56411735bf50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=692321c25e6ffdd0 server_cookie=c153ba613a96d9d5 in_seq=0 out_seq=0 2024-04-21T08:44:57.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.368+0000 7fe9497fa640 10 client.18099 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:44:57.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.368+0000 7fe9497fa640 1 -- 172.21.15.43:0/3113193431 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564117247c10 con 0x5641172dee40 2024-04-21T08:44:57.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.368+0000 7fe9497fa640 10 client.18099 handle_command_reply: tid=0 2024-04-21T08:44:57.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.368+0000 7fe96ae02740 1 client.18099 shutdown 2024-04-21T08:44:57.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.368+0000 7fe96ae02740 1 -- 172.21.15.43:0/3113193431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172d5330 msgr2=0x5641172e41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.368+0000 7fe96ae02740 1 --2- 172.21.15.43:0/3113193431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172d5330 0x5641172e41f0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fe950009610 tx=0x7fe950005bf0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.368+0000 7fe96ae02740 1 -- 172.21.15.43:0/3113193431 shutdown_connections 2024-04-21T08:44:57.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.368+0000 7fe96ae02740 1 --2- 172.21.15.43:0/3113193431 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5641172dee40 0x56411735bf50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.368+0000 7fe96ae02740 1 --2- 172.21.15.43:0/3113193431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641172d5ce0 0x5641172e4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.368+0000 7fe9497fa640 0 client.18099 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:44:57.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.368+0000 7fe96ae02740 1 --2- 172.21.15.43:0/3113193431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5641172d5330 0x5641172e41f0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.368+0000 7fe96ae02740 1 --2- 172.21.15.43:0/3113193431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641172cfba0 0x5641172eb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.368+0000 7fe96ae02740 1 -- 172.21.15.43:0/3113193431 >> 172.21.15.43:0/3113193431 conn(0x5641172de980 msgr2=0x5641172cec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.368+0000 7fe96ae02740 1 -- 172.21.15.43:0/3113193431 shutdown_connections 2024-04-21T08:44:57.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.368+0000 7fe96ae02740 1 -- 172.21.15.43:0/3113193431 wait complete. 2024-04-21T08:44:57.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.368+0000 7fe96ae02740 20 client.18099 trim_cache size 0 max 16384 2024-04-21T08:44:57.379 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:44:57.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.369+0000 7fe968132640 1 -- 172.21.15.43:0/2018631849 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe94c0a3da0 msgr2=0x7fe94c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.369+0000 7fe968132640 1 --2- 172.21.15.43:0/2018631849 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe94c0a3da0 0x7fe94c0a6260 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe958033000 tx=0x7fe9580095c0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.369+0000 7fe968132640 1 -- 172.21.15.43:0/2018631849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe960106160 msgr2=0x7fe960199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.369+0000 7fe968132640 1 --2- 172.21.15.43:0/2018631849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe960106160 0x7fe960199ab0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fe95000d740 tx=0x7fe95000dc00 comp rx=0 tx=0).stop 2024-04-21T08:44:57.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.369+0000 7fe968132640 1 -- 172.21.15.43:0/2018631849 shutdown_connections 2024-04-21T08:44:57.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.369+0000 7fe968132640 1 --2- 172.21.15.43:0/2018631849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe96019a040 0x7fe9601a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.369+0000 7fe968132640 1 --2- 172.21.15.43:0/2018631849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe960106160 0x7fe960199ab0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.369+0000 7fe968132640 1 --2- 172.21.15.43:0/2018631849 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe94c0a3da0 0x7fe94c0a6260 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.369+0000 7fe968132640 1 --2- 172.21.15.43:0/2018631849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe960101090 0x7fe960199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.369+0000 7fe968132640 1 -- 172.21.15.43:0/2018631849 >> 172.21.15.43:0/2018631849 conn(0x7fe960077f50 msgr2=0x7fe9600ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.369+0000 7fe968132640 1 -- 172.21.15.43:0/2018631849 shutdown_connections 2024-04-21T08:44:57.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.369+0000 7fe968132640 1 -- 172.21.15.43:0/2018631849 wait complete. 2024-04-21T08:44:57.382 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:44:57.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.486+0000 7fd4de42c640 1 -- 172.21.15.43:0/2086372894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4d8107260 msgr2=0x7fd4d810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.486+0000 7fd4de42c640 1 --2- 172.21.15.43:0/2086372894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4d8107260 0x7fd4d810ee30 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009ca0 tx=0x7fd4d002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.486+0000 7fd4de42c640 1 -- 172.21.15.43:0/2086372894 shutdown_connections 2024-04-21T08:44:57.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.486+0000 7fd4de42c640 1 --2- 172.21.15.43:0/2086372894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4d8107260 0x7fd4d810ee30 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.486+0000 7fd4de42c640 1 --2- 172.21.15.43:0/2086372894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4d8102b40 0x7fd4d8106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.486+0000 7fd4de42c640 1 --2- 172.21.15.43:0/2086372894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4d8102190 0x7fd4d8102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.486+0000 7fd4de42c640 1 -- 172.21.15.43:0/2086372894 >> 172.21.15.43:0/2086372894 conn(0x7fd4d80793a0 msgr2=0x7fd4d80797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.486+0000 7fd4de42c640 1 -- 172.21.15.43:0/2086372894 shutdown_connections 2024-04-21T08:44:57.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.486+0000 7fd4de42c640 1 -- 172.21.15.43:0/2086372894 wait complete. 2024-04-21T08:44:57.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.486+0000 7fd4de42c640 1 Processor -- start 2024-04-21T08:44:57.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.486+0000 7fd4de42c640 1 -- start start 2024-04-21T08:44:57.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.487+0000 7fd4de42c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4d8102190 0x7fd4d8198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.487+0000 7fd4de42c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4d8107260 0x7fd4d8198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.487+0000 7fd4de42c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4d819d010 0x7fd4d819d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.487+0000 7fd4de42c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d81121d0 con 0x7fd4d819d010 2024-04-21T08:44:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.487+0000 7fd4de42c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d8112050 con 0x7fd4d8107260 2024-04-21T08:44:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.487+0000 7fd4de42c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d8112350 con 0x7fd4d8102190 2024-04-21T08:44:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.487+0000 7fd4dc9c9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4d819d010 0x7fd4d819d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.487+0000 7fd4dc9c9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4d819d010 0x7fd4d819d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33204/0 (socket says 172.21.15.43:33204) 2024-04-21T08:44:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.487+0000 7fd4dc9c9640 1 -- 172.21.15.43:0/1123215432 learned_addr learned my addr 172.21.15.43:0/1123215432 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.487+0000 7fd4d7fff640 1 --2- 172.21.15.43:0/1123215432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4d8102190 0x7fd4d8198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.487+0000 7fd4d77fe640 1 --2- 172.21.15.43:0/1123215432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4d8107260 0x7fd4d8198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.487+0000 7fd4dc9c9640 1 -- 172.21.15.43:0/1123215432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4d8102190 msgr2=0x7fd4d8198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.487+0000 7fd4dc9c9640 1 --2- 172.21.15.43:0/1123215432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4d8102190 0x7fd4d8198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.487+0000 7fd4dc9c9640 1 -- 172.21.15.43:0/1123215432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4d8107260 msgr2=0x7fd4d8198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.487+0000 7fd4dc9c9640 1 --2- 172.21.15.43:0/1123215432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4d8107260 0x7fd4d8198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.487+0000 7fd4dc9c9640 1 -- 172.21.15.43:0/1123215432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d819db00 con 0x7fd4d819d010 2024-04-21T08:44:57.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.487+0000 7fd4d77fe640 1 --2- 172.21.15.43:0/1123215432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4d8107260 0x7fd4d8198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:57.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.487+0000 7fd4dc9c9640 1 --2- 172.21.15.43:0/1123215432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4d819d010 0x7fd4d819d4c0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0005e30 tx=0x7fd4d0032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.488+0000 7fd4d57fa640 1 -- 172.21.15.43:0/1123215432 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0041020 con 0x7fd4d819d010 2024-04-21T08:44:57.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.488+0000 7fd4de42c640 1 -- 172.21.15.43:0/1123215432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d819dd90 con 0x7fd4d819d010 2024-04-21T08:44:57.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.488+0000 7fd4d57fa640 1 -- 172.21.15.43:0/1123215432 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4d0032970 con 0x7fd4d819d010 2024-04-21T08:44:57.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.488+0000 7fd4d57fa640 1 -- 172.21.15.43:0/1123215432 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0039040 con 0x7fd4d819d010 2024-04-21T08:44:57.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.488+0000 7fd4de42c640 1 -- 172.21.15.43:0/1123215432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d81a5550 con 0x7fd4d819d010 2024-04-21T08:44:57.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.490+0000 7fd4d57fa640 1 -- 172.21.15.43:0/1123215432 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd4d0044020 con 0x7fd4d819d010 2024-04-21T08:44:57.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.490+0000 7fd4e1018740 1 Processor -- start 2024-04-21T08:44:57.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.490+0000 7fd4e1018740 1 -- start start 2024-04-21T08:44:57.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.490+0000 7fd4e1018740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b488fd4d0 0x555b488fd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.490+0000 7fd4e1018740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b488fde80 0x555b48906430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.490+0000 7fd4e1018740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b48906a00 0x555b48908e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.490+0000 7fd4e1018740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555b488621f0 con 0x555b488fd4d0 2024-04-21T08:44:57.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.490+0000 7fd4e1018740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555b487ff4b0 con 0x555b488fde80 2024-04-21T08:44:57.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.490+0000 7fd4e1018740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555b4885e1b0 con 0x555b48906a00 2024-04-21T08:44:57.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.490+0000 7fd4d57fa640 1 --2- 172.21.15.43:0/1123215432 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd4c40a3d20 0x7fd4c40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.490+0000 7fd4d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b488fd4d0 0x555b488fd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.490+0000 7fd4d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b488fd4d0 0x555b488fd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33214/0 (socket says 172.21.15.43:33214) 2024-04-21T08:44:57.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.490+0000 7fd4d77fe640 1 -- 172.21.15.43:0/1943599622 learned_addr learned my addr 172.21.15.43:0/1943599622 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.491+0000 7fd4dc9c9640 1 --2- 172.21.15.43:0/1123215432 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd4c40a3d20 0x7fd4c40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.491+0000 7fd4d7fff640 1 --2- 172.21.15.43:0/1943599622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b488fde80 0x555b48906430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.491+0000 7fd4d77fe640 1 --2- 172.21.15.43:0/1943599622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b48906a00 0x555b48908e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.491+0000 7fd4d57fa640 1 -- 172.21.15.43:0/1123215432 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd4d00f2890 con 0x7fd4d819d010 2024-04-21T08:44:57.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.491+0000 7fd4d77fe640 1 -- 172.21.15.43:0/1943599622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b48906a00 msgr2=0x555b48908e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.491+0000 7fd4d77fe640 1 --2- 172.21.15.43:0/1943599622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b48906a00 0x555b48908e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.491+0000 7fd4d77fe640 1 -- 172.21.15.43:0/1943599622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b488fde80 msgr2=0x555b48906430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.491+0000 7fd4d77fe640 1 --2- 172.21.15.43:0/1943599622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b488fde80 0x555b48906430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.491+0000 7fd4d77fe640 1 -- 172.21.15.43:0/1943599622 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b487f11b0 con 0x555b488fd4d0 2024-04-21T08:44:57.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.491+0000 7fd4d7fff640 1 --2- 172.21.15.43:0/1943599622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b488fde80 0x555b48906430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:57.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.491+0000 7fd4d77fe640 1 --2- 172.21.15.43:0/1943599622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b488fd4d0 0x555b488fd8b0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fd4c80134d0 tx=0x7fd4c8009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a3996720caaa5f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.491+0000 7fd4dc9c9640 1 --2- 172.21.15.43:0/1123215432 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd4c40a3d20 0x7fd4c40a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00be270 tx=0x7fd4d00b3a00 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.491+0000 7fd4ce7fc640 1 -- 172.21.15.43:0/1943599622 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8004030 con 0x555b488fd4d0 2024-04-21T08:44:57.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.491+0000 7fd4ce7fc640 1 -- 172.21.15.43:0/1943599622 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c80035f0 con 0x555b488fd4d0 2024-04-21T08:44:57.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.491+0000 7fd4ce7fc640 1 -- 172.21.15.43:0/1943599622 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c80106f0 con 0x555b488fd4d0 2024-04-21T08:44:57.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.492+0000 7fd4e1018740 1 -- 172.21.15.43:0/1943599622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b488fd4d0 msgr2=0x555b488fd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.492+0000 7fd4e1018740 1 --2- 172.21.15.43:0/1943599622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b488fd4d0 0x555b488fd8b0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fd4c80134d0 tx=0x7fd4c8009740 comp rx=0 tx=0).stop 2024-04-21T08:44:57.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.492+0000 7fd4e1018740 1 -- 172.21.15.43:0/1943599622 shutdown_connections 2024-04-21T08:44:57.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.492+0000 7fd4e1018740 1 --2- 172.21.15.43:0/1943599622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b48906a00 0x555b48908e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.492+0000 7fd4e1018740 1 --2- 172.21.15.43:0/1943599622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b488fde80 0x555b48906430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.492+0000 7fd4e1018740 1 --2- 172.21.15.43:0/1943599622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b488fd4d0 0x555b488fd8b0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.492+0000 7fd4e1018740 1 -- 172.21.15.43:0/1943599622 >> 172.21.15.43:0/1943599622 conn(0x555b488f7ba0 msgr2=0x555b488f7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.492+0000 7fd4e1018740 1 -- 172.21.15.43:0/1943599622 shutdown_connections 2024-04-21T08:44:57.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.492+0000 7fd4e1018740 1 -- 172.21.15.43:0/1943599622 wait complete. 2024-04-21T08:44:57.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.492+0000 7fd4e1018740 1 -- start start 2024-04-21T08:44:57.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.492+0000 7fd4e1018740 1 Processor -- start 2024-04-21T08:44:57.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.493+0000 7fd4e1018740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b488fd4d0 0x555b48906330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.493+0000 7fd4e1018740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b488fde80 0x555b489036c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.493+0000 7fd4e1018740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b48906a00 0x555b48903cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.493+0000 7fd4e1018740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555b48909970 con 0x555b488fd4d0 2024-04-21T08:44:57.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.493+0000 7fd4e1018740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555b4881b500 con 0x555b48906a00 2024-04-21T08:44:57.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.493+0000 7fd4e1018740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555b48909af0 con 0x555b488fde80 2024-04-21T08:44:57.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.493+0000 7fd4e1018740 10 client.0 _ll_register_callbacks cb 0x555b487e9b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:57.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.493+0000 7fd4d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b488fd4d0 0x555b48906330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.493+0000 7fd4d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b488fd4d0 0x555b48906330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33224/0 (socket says 172.21.15.43:33224) 2024-04-21T08:44:57.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.493+0000 7fd4d77fe640 1 -- 172.21.15.43:0/2765521394 learned_addr learned my addr 172.21.15.43:0/2765521394 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.493+0000 7fd4d7fff640 1 --2- 172.21.15.43:0/2765521394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b488fde80 0x555b489036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.493+0000 7fd4d77fe640 1 --2- 172.21.15.43:0/2765521394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b48906a00 0x555b48903cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.493+0000 7fd4d77fe640 1 -- 172.21.15.43:0/2765521394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b488fde80 msgr2=0x555b489036c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.493+0000 7fd4d77fe640 1 --2- 172.21.15.43:0/2765521394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b488fde80 0x555b489036c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.493+0000 7fd4d77fe640 1 -- 172.21.15.43:0/2765521394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b48906a00 msgr2=0x555b48903cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.493+0000 7fd4d77fe640 1 --2- 172.21.15.43:0/2765521394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b48906a00 0x555b48903cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.493+0000 7fd4d77fe640 1 -- 172.21.15.43:0/2765521394 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b485743f0 con 0x555b488fd4d0 2024-04-21T08:44:57.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.493+0000 7fd4d77fe640 1 --2- 172.21.15.43:0/2765521394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b488fd4d0 0x555b48906330 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009e50 tx=0x7fd4c80095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c26a8ef8ba11ae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.493+0000 7fd4b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:57.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.493+0000 7fd4b77fe640 1 -- 172.21.15.43:0/2765521394 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8005050 con 0x555b488fd4d0 2024-04-21T08:44:57.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.494+0000 7fd4b77fe640 1 -- 172.21.15.43:0/2765521394 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd4c80051f0 con 0x555b488fd4d0 2024-04-21T08:44:57.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.494+0000 7fd4e1018740 1 -- 172.21.15.43:0/2765521394 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4c80035f0 con 0x555b488fd4d0 2024-04-21T08:44:57.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.494+0000 7fd4b77fe640 1 -- 172.21.15.43:0/2765521394 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd4c80053f0 con 0x555b488fd4d0 2024-04-21T08:44:57.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.494+0000 7fd4e1018740 10 client.18111 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:57.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.494+0000 7fd4e1018740 1 -- 172.21.15.43:0/2765521394 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555b4863e7d0 con 0x555b488fd4d0 2024-04-21T08:44:57.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.494+0000 7fd4b77fe640 1 -- 172.21.15.43:0/2765521394 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c80035f0 con 0x555b488fd4d0 2024-04-21T08:44:57.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.494+0000 7fd4b77fe640 1 -- 172.21.15.43:0/2765521394 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd4c801e070 con 0x555b488fd4d0 2024-04-21T08:44:57.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.494+0000 7fd4e1018740 10 client.18111 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:57.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.494+0000 7fd4e1018740 10 client.18111 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:44:57.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.494+0000 7fd4e1018740 20 client.18111 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:57.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.494+0000 7fd4e1018740 1 --2- 172.21.15.43:0/2765521394 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x555b48916af0 0x555b48918ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.494+0000 7fd4e1018740 4 client.18111 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:57.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.494+0000 7fd4e1018740 1 -- 172.21.15.43:0/2765521394 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b48801930 con 0x555b48916af0 2024-04-21T08:44:57.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.494+0000 7fd4d7fff640 1 --2- 172.21.15.43:0/2765521394 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x555b48916af0 0x555b48918ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.495+0000 7fd4d7fff640 1 --2- 172.21.15.43:0/2765521394 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x555b48916af0 0x555b48918ee0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d18b451856a0ea03 server_cookie=2483234828f6b390 in_seq=0 out_seq=0 2024-04-21T08:44:57.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.495+0000 7fd4b77fe640 10 client.18111 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:44:57.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.495+0000 7fd4b77fe640 1 -- 172.21.15.43:0/2765521394 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555b48801930 con 0x555b48916af0 2024-04-21T08:44:57.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.495+0000 7fd4b77fe640 10 client.18111 handle_command_reply: tid=0 2024-04-21T08:44:57.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.496+0000 7fd4e1018740 1 client.18111 shutdown 2024-04-21T08:44:57.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.496+0000 7fd4e1018740 1 -- 172.21.15.43:0/2765521394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b488fd4d0 msgr2=0x555b48906330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.496+0000 7fd4e1018740 1 --2- 172.21.15.43:0/2765521394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b488fd4d0 0x555b48906330 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009e50 tx=0x7fd4c80095a0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.496+0000 7fd4e1018740 1 -- 172.21.15.43:0/2765521394 shutdown_connections 2024-04-21T08:44:57.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.496+0000 7fd4e1018740 1 --2- 172.21.15.43:0/2765521394 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x555b48916af0 0x555b48918ee0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.496+0000 7fd4e1018740 1 --2- 172.21.15.43:0/2765521394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b48906a00 0x555b48903cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.496+0000 7fd4e1018740 1 --2- 172.21.15.43:0/2765521394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b488fde80 0x555b489036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.496+0000 7fd4b77fe640 0 client.18111 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:44:57.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.496+0000 7fd4e1018740 1 --2- 172.21.15.43:0/2765521394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b488fd4d0 0x555b48906330 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.496+0000 7fd4e1018740 1 -- 172.21.15.43:0/2765521394 >> 172.21.15.43:0/2765521394 conn(0x555b488f7ba0 msgr2=0x555b488f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.496+0000 7fd4e1018740 1 -- 172.21.15.43:0/2765521394 shutdown_connections 2024-04-21T08:44:57.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.496+0000 7fd4e1018740 1 -- 172.21.15.43:0/2765521394 wait complete. 2024-04-21T08:44:57.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.496+0000 7fd4e1018740 20 client.18111 trim_cache size 0 max 16384 2024-04-21T08:44:57.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.509+0000 7fd4e1018740 1 Processor -- start 2024-04-21T08:44:57.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.509+0000 7fd4e1018740 1 -- start start 2024-04-21T08:44:57.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.509+0000 7fd4e1018740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b488fd4d0 0x555b48912860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.509+0000 7fd4e1018740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b488fde80 0x555b488f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.509+0000 7fd4e1018740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b48906a00 0x555b48905df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.509+0000 7fd4e1018740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d81af450 con 0x555b48906a00 2024-04-21T08:44:57.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.509+0000 7fd4e1018740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d81b0f90 con 0x555b488fd4d0 2024-04-21T08:44:57.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.509+0000 7fd4e1018740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d81afb20 con 0x555b488fde80 2024-04-21T08:44:57.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.509+0000 7fd4d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b48906a00 0x555b48905df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.510+0000 7fd4d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b48906a00 0x555b48905df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33230/0 (socket says 172.21.15.43:33230) 2024-04-21T08:44:57.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.510+0000 7fd4d77fe640 1 -- 172.21.15.43:0/2946675959 learned_addr learned my addr 172.21.15.43:0/2946675959 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.510+0000 7fd4d77fe640 1 --2- 172.21.15.43:0/2946675959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b488fd4d0 0x555b48912860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.510+0000 7fd4d7fff640 1 --2- 172.21.15.43:0/2946675959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b488fde80 0x555b488f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.510+0000 7fd4d77fe640 1 -- 172.21.15.43:0/2946675959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b488fde80 msgr2=0x555b488f6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.510+0000 7fd4d77fe640 1 --2- 172.21.15.43:0/2946675959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b488fde80 0x555b488f6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.510+0000 7fd4d77fe640 1 -- 172.21.15.43:0/2946675959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b488fd4d0 msgr2=0x555b48912860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.510+0000 7fd4d77fe640 1 --2- 172.21.15.43:0/2946675959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b488fd4d0 0x555b48912860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.510+0000 7fd4d77fe640 1 -- 172.21.15.43:0/2946675959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b48919470 con 0x555b48906a00 2024-04-21T08:44:57.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.510+0000 7fd4d77fe640 1 --2- 172.21.15.43:0/2946675959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b48906a00 0x555b48905df0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009530 tx=0x7fd4c8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e031a133b3d9ab7d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.510+0000 7fd4cdffb640 1 -- 172.21.15.43:0/2946675959 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c80035f0 con 0x555b48906a00 2024-04-21T08:44:57.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.510+0000 7fd4cdffb640 1 -- 172.21.15.43:0/2946675959 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c8003ca0 con 0x555b48906a00 2024-04-21T08:44:57.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.510+0000 7fd4cdffb640 1 -- 172.21.15.43:0/2946675959 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8005000 con 0x555b48906a00 2024-04-21T08:44:57.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.511+0000 7fd4e1018740 1 -- 172.21.15.43:0/2946675959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b48906a00 msgr2=0x555b48905df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.511+0000 7fd4e1018740 1 --2- 172.21.15.43:0/2946675959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b48906a00 0x555b48905df0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009530 tx=0x7fd4c8003ba0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.511+0000 7fd4e1018740 1 -- 172.21.15.43:0/2946675959 shutdown_connections 2024-04-21T08:44:57.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.511+0000 7fd4e1018740 1 --2- 172.21.15.43:0/2946675959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b48906a00 0x555b48905df0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.511+0000 7fd4e1018740 1 --2- 172.21.15.43:0/2946675959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b488fde80 0x555b488f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.511+0000 7fd4e1018740 1 --2- 172.21.15.43:0/2946675959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b488fd4d0 0x555b48912860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.511+0000 7fd4e1018740 1 -- 172.21.15.43:0/2946675959 >> 172.21.15.43:0/2946675959 conn(0x555b488f7ba0 msgr2=0x555b488f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.511+0000 7fd4e1018740 1 -- 172.21.15.43:0/2946675959 shutdown_connections 2024-04-21T08:44:57.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.511+0000 7fd4e1018740 1 -- 172.21.15.43:0/2946675959 wait complete. 2024-04-21T08:44:57.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.511+0000 7fd4e1018740 1 -- start start 2024-04-21T08:44:57.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.511+0000 7fd4e1018740 1 Processor -- start 2024-04-21T08:44:57.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.511+0000 7fd4e1018740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b488fd4d0 0x555b48909f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.511+0000 7fd4e1018740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b488fde80 0x555b4890a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.511+0000 7fd4e1018740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b48906a00 0x555b4897af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.511+0000 7fd4e1018740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d81b08c0 con 0x555b488fd4d0 2024-04-21T08:44:57.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.512+0000 7fd4e1018740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d81b01f0 con 0x555b488fde80 2024-04-21T08:44:57.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.512+0000 7fd4e1018740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d81ad430 con 0x555b48906a00 2024-04-21T08:44:57.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.512+0000 7fd4e1018740 10 client.0 _ll_register_callbacks cb 0x7fd4c8009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:57.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.512+0000 7fd4d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b488fd4d0 0x555b48909f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.512+0000 7fd4d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b488fd4d0 0x555b48909f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33242/0 (socket says 172.21.15.43:33242) 2024-04-21T08:44:57.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.512+0000 7fd4d77fe640 1 -- 172.21.15.43:0/295789153 learned_addr learned my addr 172.21.15.43:0/295789153 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.512+0000 7fd4d7fff640 1 --2- 172.21.15.43:0/295789153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b488fde80 0x555b4890a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.512+0000 7fd4d77fe640 1 -- 172.21.15.43:0/295789153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b48906a00 msgr2=0x555b4897af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.512+0000 7fd4d77fe640 1 --2- 172.21.15.43:0/295789153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b48906a00 0x555b4897af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.512+0000 7fd4d77fe640 1 -- 172.21.15.43:0/295789153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b488fde80 msgr2=0x555b4890a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.512+0000 7fd4d77fe640 1 --2- 172.21.15.43:0/295789153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b488fde80 0x555b4890a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.512+0000 7fd4d77fe640 1 -- 172.21.15.43:0/295789153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b4897b470 con 0x555b488fd4d0 2024-04-21T08:44:57.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.512+0000 7fd4d7fff640 1 --2- 172.21.15.43:0/295789153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b488fde80 0x555b4890a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:57.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.512+0000 7fd4d77fe640 1 --2- 172.21.15.43:0/295789153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b488fd4d0 0x555b48909f20 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fd4c80053c0 tx=0x7fd4c80035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7d5b5275f7fe46 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.512+0000 7fd4b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:57.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.512+0000 7fd4b77fe640 1 -- 172.21.15.43:0/295789153 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c80115e0 con 0x555b488fd4d0 2024-04-21T08:44:57.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.512+0000 7fd4b77fe640 1 -- 172.21.15.43:0/295789153 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd4c8011780 con 0x555b488fd4d0 2024-04-21T08:44:57.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.512+0000 7fd4e1018740 1 -- 172.21.15.43:0/295789153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4c8003ca0 con 0x555b488fd4d0 2024-04-21T08:44:57.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.512+0000 7fd4b77fe640 1 -- 172.21.15.43:0/295789153 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd4c8049050 con 0x555b488fd4d0 2024-04-21T08:44:57.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.512+0000 7fd4e1018740 10 client.18117 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:57.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.512+0000 7fd4e1018740 1 -- 172.21.15.43:0/295789153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555b4897b470 con 0x555b488fd4d0 2024-04-21T08:44:57.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.513+0000 7fd4b77fe640 1 -- 172.21.15.43:0/295789153 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c8003ca0 con 0x555b488fd4d0 2024-04-21T08:44:57.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.513+0000 7fd4b77fe640 1 -- 172.21.15.43:0/295789153 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd4c801e070 con 0x555b488fd4d0 2024-04-21T08:44:57.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.513+0000 7fd4e1018740 10 client.18117 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:57.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.513+0000 7fd4e1018740 10 client.18117 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:44:57.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.513+0000 7fd4e1018740 20 client.18117 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:57.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.513+0000 7fd4e1018740 1 --2- 172.21.15.43:0/295789153 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x555b4897caa0 0x555b4897ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.513+0000 7fd4e1018740 4 client.18117 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:44:57.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.513+0000 7fd4e1018740 1 -- 172.21.15.43:0/295789153 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b4886fc10 con 0x555b4897caa0 2024-04-21T08:44:57.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.513+0000 7fd4d7fff640 1 --2- 172.21.15.43:0/295789153 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x555b4897caa0 0x555b4897ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.513+0000 7fd4b77fe640 10 client.18117 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:44:57.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.513+0000 7fd4d7fff640 1 --2- 172.21.15.43:0/295789153 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x555b4897caa0 0x555b4897ee90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7e90beb8a773e57 server_cookie=c3a3b41c527fbd42 in_seq=0 out_seq=0 2024-04-21T08:44:57.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.514+0000 7fd4b77fe640 1 -- 172.21.15.43:0/295789153 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b4886fc10 con 0x555b4897caa0 2024-04-21T08:44:57.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.514+0000 7fd4b77fe640 10 client.18117 handle_command_reply: tid=0 2024-04-21T08:44:57.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.514+0000 7fd4e1018740 1 client.18117 shutdown 2024-04-21T08:44:57.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.514+0000 7fd4e1018740 1 -- 172.21.15.43:0/295789153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b488fd4d0 msgr2=0x555b48909f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.514+0000 7fd4e1018740 1 --2- 172.21.15.43:0/295789153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b488fd4d0 0x555b48909f20 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fd4c80053c0 tx=0x7fd4c80035f0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.514+0000 7fd4e1018740 1 -- 172.21.15.43:0/295789153 shutdown_connections 2024-04-21T08:44:57.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.514+0000 7fd4e1018740 1 --2- 172.21.15.43:0/295789153 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x555b4897caa0 0x555b4897ee90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.514+0000 7fd4e1018740 1 --2- 172.21.15.43:0/295789153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555b48906a00 0x555b4897af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.514+0000 7fd4b77fe640 0 client.18117 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:44:57.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.514+0000 7fd4e1018740 1 --2- 172.21.15.43:0/295789153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b488fde80 0x555b4890a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.514+0000 7fd4e1018740 1 --2- 172.21.15.43:0/295789153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b488fd4d0 0x555b48909f20 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.514+0000 7fd4e1018740 1 -- 172.21.15.43:0/295789153 >> 172.21.15.43:0/295789153 conn(0x555b488f7ba0 msgr2=0x555b488f96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.514+0000 7fd4e1018740 1 -- 172.21.15.43:0/295789153 shutdown_connections 2024-04-21T08:44:57.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.514+0000 7fd4e1018740 1 -- 172.21.15.43:0/295789153 wait complete. 2024-04-21T08:44:57.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.514+0000 7fd4e1018740 20 client.18117 trim_cache size 0 max 16384 2024-04-21T08:44:57.531 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:44:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.514+0000 7fd4de42c640 1 -- 172.21.15.43:0/1123215432 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd4c40a3d20 msgr2=0x7fd4c40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.514+0000 7fd4de42c640 1 --2- 172.21.15.43:0/1123215432 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd4c40a3d20 0x7fd4c40a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00be270 tx=0x7fd4d00b3a00 comp rx=0 tx=0).stop 2024-04-21T08:44:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.514+0000 7fd4de42c640 1 -- 172.21.15.43:0/1123215432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4d819d010 msgr2=0x7fd4d819d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.514+0000 7fd4de42c640 1 --2- 172.21.15.43:0/1123215432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4d819d010 0x7fd4d819d4c0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0005e30 tx=0x7fd4d0032420 comp rx=0 tx=0).stop 2024-04-21T08:44:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.515+0000 7fd4de42c640 1 -- 172.21.15.43:0/1123215432 shutdown_connections 2024-04-21T08:44:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.515+0000 7fd4de42c640 1 --2- 172.21.15.43:0/1123215432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4d819d010 0x7fd4d819d4c0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.515+0000 7fd4de42c640 1 --2- 172.21.15.43:0/1123215432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4d8107260 0x7fd4d8198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.515+0000 7fd4de42c640 1 --2- 172.21.15.43:0/1123215432 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd4c40a3d20 0x7fd4c40a61e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.515+0000 7fd4de42c640 1 --2- 172.21.15.43:0/1123215432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4d8102190 0x7fd4d8198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.515+0000 7fd4de42c640 1 -- 172.21.15.43:0/1123215432 >> 172.21.15.43:0/1123215432 conn(0x7fd4d80793a0 msgr2=0x7fd4d8100030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.515+0000 7fd4de42c640 1 -- 172.21.15.43:0/1123215432 shutdown_connections 2024-04-21T08:44:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.515+0000 7fd4de42c640 1 -- 172.21.15.43:0/1123215432 wait complete. 2024-04-21T08:44:57.536 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:44:57.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.640+0000 7f6dec985640 1 -- 172.21.15.43:0/1027282876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6de806b1f0 msgr2=0x7f6de8106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.640+0000 7f6dec985640 1 --2- 172.21.15.43:0/1027282876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6de806b1f0 0x7f6de8106b90 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc009900 tx=0x7f6ddc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:57.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.640+0000 7f6dec985640 1 -- 172.21.15.43:0/1027282876 shutdown_connections 2024-04-21T08:44:57.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.640+0000 7f6dec985640 1 --2- 172.21.15.43:0/1027282876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6de8107270 0x7f6de810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.640+0000 7f6dec985640 1 --2- 172.21.15.43:0/1027282876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6de806b1f0 0x7f6de8106b90 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.640+0000 7f6dec985640 1 --2- 172.21.15.43:0/1027282876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6de806a840 0x7f6de806ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.640+0000 7f6dec985640 1 -- 172.21.15.43:0/1027282876 >> 172.21.15.43:0/1027282876 conn(0x7f6de8077eb0 msgr2=0x7f6de80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.640+0000 7f6dec985640 1 -- 172.21.15.43:0/1027282876 shutdown_connections 2024-04-21T08:44:57.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.640+0000 7f6dec985640 1 -- 172.21.15.43:0/1027282876 wait complete. 2024-04-21T08:44:57.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.641+0000 7f6dec985640 1 Processor -- start 2024-04-21T08:44:57.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.641+0000 7f6dec985640 1 -- start start 2024-04-21T08:44:57.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.641+0000 7f6dec985640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6de806a840 0x7f6de8198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.641+0000 7f6dec985640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6de8107270 0x7f6de8198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.641+0000 7f6dec985640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6de819cff0 0x7f6de819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.641+0000 7f6dec985640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81121b0 con 0x7f6de806a840 2024-04-21T08:44:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.641+0000 7f6dec985640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6de8112030 con 0x7f6de819cff0 2024-04-21T08:44:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.641+0000 7f6dec985640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6de8112330 con 0x7f6de8107270 2024-04-21T08:44:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.641+0000 7f6de659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6de806a840 0x7f6de8198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.641+0000 7f6de659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6de806a840 0x7f6de8198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33260/0 (socket says 172.21.15.43:33260) 2024-04-21T08:44:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.641+0000 7f6de659c640 1 -- 172.21.15.43:0/4186234076 learned_addr learned my addr 172.21.15.43:0/4186234076 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.642+0000 7f6de6d9d640 1 --2- 172.21.15.43:0/4186234076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6de819cff0 0x7f6de819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.642+0000 7f6de5d9b640 1 --2- 172.21.15.43:0/4186234076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6de8107270 0x7f6de8198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.642+0000 7f6de659c640 1 -- 172.21.15.43:0/4186234076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6de8107270 msgr2=0x7f6de8198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.642+0000 7f6de659c640 1 --2- 172.21.15.43:0/4186234076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6de8107270 0x7f6de8198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.642+0000 7f6de659c640 1 -- 172.21.15.43:0/4186234076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6de819cff0 msgr2=0x7f6de819d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.642+0000 7f6de659c640 1 --2- 172.21.15.43:0/4186234076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6de819cff0 0x7f6de819d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.642+0000 7f6de659c640 1 -- 172.21.15.43:0/4186234076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de819dae0 con 0x7f6de806a840 2024-04-21T08:44:57.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.642+0000 7f6de6d9d640 1 --2- 172.21.15.43:0/4186234076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6de819cff0 0x7f6de819d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:57.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.642+0000 7f6de5d9b640 1 --2- 172.21.15.43:0/4186234076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6de8107270 0x7f6de8198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:57.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.642+0000 7f6de659c640 1 --2- 172.21.15.43:0/4186234076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6de806a840 0x7f6de8198730 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800c970 tx=0x7f6dd800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.642+0000 7f6dd77fe640 1 -- 172.21.15.43:0/4186234076 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8007bb0 con 0x7f6de806a840 2024-04-21T08:44:57.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.642+0000 7f6dd77fe640 1 -- 172.21.15.43:0/4186234076 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dd8007d50 con 0x7f6de806a840 2024-04-21T08:44:57.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.642+0000 7f6dec985640 1 -- 172.21.15.43:0/4186234076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de819ddd0 con 0x7f6de806a840 2024-04-21T08:44:57.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.642+0000 7f6dd77fe640 1 -- 172.21.15.43:0/4186234076 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800f450 con 0x7f6de806a840 2024-04-21T08:44:57.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.642+0000 7f6dec985640 1 -- 172.21.15.43:0/4186234076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de81a5570 con 0x7f6de806a840 2024-04-21T08:44:57.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.645+0000 7f6dd77fe640 1 -- 172.21.15.43:0/4186234076 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6dd8004f20 con 0x7f6de806a840 2024-04-21T08:44:57.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.645+0000 7f6def573740 1 Processor -- start 2024-04-21T08:44:57.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.645+0000 7f6def573740 1 -- start start 2024-04-21T08:44:57.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.645+0000 7f6def573740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca4d7d8330 0x55ca4d7d8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.645+0000 7f6def573740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca4d7d8ce0 0x55ca4d7e1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.645+0000 7f6def573740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7e1e40 0x55ca4d7e22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.645+0000 7f6def573740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ca4d73d1f0 con 0x55ca4d7e1e40 2024-04-21T08:44:57.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.645+0000 7f6def573740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ca4d6da4b0 con 0x55ca4d7d8ce0 2024-04-21T08:44:57.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.645+0000 7f6def573740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ca4d7391b0 con 0x55ca4d7d8330 2024-04-21T08:44:57.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.646+0000 7f6de5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7e1e40 0x55ca4d7e22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.646+0000 7f6de5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7e1e40 0x55ca4d7e22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33264/0 (socket says 172.21.15.43:33264) 2024-04-21T08:44:57.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.646+0000 7f6de5d9b640 1 -- 172.21.15.43:0/2046942379 learned_addr learned my addr 172.21.15.43:0/2046942379 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.646+0000 7f6de659c640 1 --2- 172.21.15.43:0/2046942379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca4d7d8ce0 0x55ca4d7e1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.646+0000 7f6de6d9d640 1 --2- 172.21.15.43:0/2046942379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca4d7d8330 0x55ca4d7d8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.646+0000 7f6dd77fe640 1 --2- 172.21.15.43:0/4186234076 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6dd00a3d20 0x7f6dd00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.646+0000 7f6de5d9b640 1 -- 172.21.15.43:0/2046942379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca4d7d8330 msgr2=0x55ca4d7d8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.646+0000 7f6de5d9b640 1 --2- 172.21.15.43:0/2046942379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca4d7d8330 0x55ca4d7d8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.646+0000 7f6de5d9b640 1 -- 172.21.15.43:0/2046942379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca4d7d8ce0 msgr2=0x55ca4d7e1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.646+0000 7f6de5d9b640 1 --2- 172.21.15.43:0/2046942379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca4d7d8ce0 0x55ca4d7e1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.646+0000 7f6de5d9b640 1 -- 172.21.15.43:0/2046942379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca4d6cc1b0 con 0x55ca4d7e1e40 2024-04-21T08:44:57.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.646+0000 7f6de659c640 1 --2- 172.21.15.43:0/2046942379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca4d7d8ce0 0x55ca4d7e1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:57.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.646+0000 7f6de6d9d640 1 --2- 172.21.15.43:0/4186234076 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6dd00a3d20 0x7f6dd00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.646+0000 7f6dd77fe640 1 -- 172.21.15.43:0/4186234076 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f6dd80d0a60 con 0x7f6de806a840 2024-04-21T08:44:57.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.646+0000 7f6de5d9b640 1 --2- 172.21.15.43:0/2046942379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7e1e40 0x55ca4d7e22c0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc006010 tx=0x7f6ddc02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8920804cc50e93d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.646+0000 7f6de6d9d640 1 --2- 172.21.15.43:0/4186234076 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6dd00a3d20 0x7f6dd00a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009370 tx=0x7f6de0004a40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.646+0000 7f6dd4ff9640 1 -- 172.21.15.43:0/2046942379 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc043020 con 0x55ca4d7e1e40 2024-04-21T08:44:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.646+0000 7f6dd4ff9640 1 -- 172.21.15.43:0/2046942379 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ddc02c7e0 con 0x55ca4d7e1e40 2024-04-21T08:44:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.646+0000 7f6dd4ff9640 1 -- 172.21.15.43:0/2046942379 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc005530 con 0x55ca4d7e1e40 2024-04-21T08:44:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.647+0000 7f6def573740 1 -- 172.21.15.43:0/2046942379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7e1e40 msgr2=0x55ca4d7e22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.647+0000 7f6def573740 1 --2- 172.21.15.43:0/2046942379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7e1e40 0x55ca4d7e22c0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc006010 tx=0x7f6ddc02c650 comp rx=0 tx=0).stop 2024-04-21T08:44:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.647+0000 7f6def573740 1 -- 172.21.15.43:0/2046942379 shutdown_connections 2024-04-21T08:44:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.647+0000 7f6def573740 1 --2- 172.21.15.43:0/2046942379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7e1e40 0x55ca4d7e22c0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.647+0000 7f6def573740 1 --2- 172.21.15.43:0/2046942379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca4d7d8ce0 0x55ca4d7e1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.647+0000 7f6def573740 1 --2- 172.21.15.43:0/2046942379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca4d7d8330 0x55ca4d7d8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.647+0000 7f6def573740 1 -- 172.21.15.43:0/2046942379 >> 172.21.15.43:0/2046942379 conn(0x55ca4d7d2ba0 msgr2=0x55ca4d7d2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.647+0000 7f6def573740 1 -- 172.21.15.43:0/2046942379 shutdown_connections 2024-04-21T08:44:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.647+0000 7f6def573740 1 -- 172.21.15.43:0/2046942379 wait complete. 2024-04-21T08:44:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.648+0000 7f6def573740 1 -- start start 2024-04-21T08:44:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.648+0000 7f6def573740 1 Processor -- start 2024-04-21T08:44:57.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.648+0000 7f6def573740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7d8330 0x55ca4d7e8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.648+0000 7f6def573740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca4d7d8ce0 0x55ca4d7e55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.648+0000 7f6def573740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca4d7e1e40 0x55ca4d7e5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.648+0000 7f6def573740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ca4d7e8d60 con 0x55ca4d7d8330 2024-04-21T08:44:57.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.648+0000 7f6def573740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ca4d6f6500 con 0x55ca4d7d8ce0 2024-04-21T08:44:57.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.648+0000 7f6def573740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ca4d7e8ee0 con 0x55ca4d7e1e40 2024-04-21T08:44:57.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.648+0000 7f6def573740 10 client.0 _ll_register_callbacks cb 0x55ca4d6c4b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:57.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.648+0000 7f6de659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7d8330 0x55ca4d7e8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.648+0000 7f6de659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7d8330 0x55ca4d7e8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33280/0 (socket says 172.21.15.43:33280) 2024-04-21T08:44:57.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.648+0000 7f6de659c640 1 -- 172.21.15.43:0/2620978692 learned_addr learned my addr 172.21.15.43:0/2620978692 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.648+0000 7f6de5d9b640 1 --2- 172.21.15.43:0/2620978692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca4d7d8ce0 0x55ca4d7e55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.648+0000 7f6de6d9d640 1 --2- 172.21.15.43:0/2620978692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca4d7e1e40 0x55ca4d7e5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.648+0000 7f6de659c640 1 -- 172.21.15.43:0/2620978692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca4d7e1e40 msgr2=0x55ca4d7e5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.648+0000 7f6de659c640 1 --2- 172.21.15.43:0/2620978692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca4d7e1e40 0x55ca4d7e5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.648+0000 7f6de659c640 1 -- 172.21.15.43:0/2620978692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca4d7d8ce0 msgr2=0x55ca4d7e55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.648+0000 7f6de659c640 1 --2- 172.21.15.43:0/2620978692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca4d7d8ce0 0x55ca4d7e55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.648+0000 7f6de659c640 1 -- 172.21.15.43:0/2620978692 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca4d44f3f0 con 0x55ca4d7d8330 2024-04-21T08:44:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.648+0000 7f6de6d9d640 1 --2- 172.21.15.43:0/2620978692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca4d7e1e40 0x55ca4d7e5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.649+0000 7f6de659c640 1 --2- 172.21.15.43:0/2620978692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7d8330 0x55ca4d7e8200 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80cdde0 tx=0x7f6dd809c020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d388cbf025ea2cd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.649+0000 7f6dcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.649+0000 7f6dcdffb640 1 -- 172.21.15.43:0/2620978692 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd801d040 con 0x55ca4d7d8330 2024-04-21T08:44:57.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.649+0000 7f6dcdffb640 1 -- 172.21.15.43:0/2620978692 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f6dd80a0590 con 0x55ca4d7d8330 2024-04-21T08:44:57.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.649+0000 7f6def573740 1 -- 172.21.15.43:0/2620978692 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ddc02c7e0 con 0x55ca4d7d8330 2024-04-21T08:44:57.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.649+0000 7f6dcdffb640 1 -- 172.21.15.43:0/2620978692 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6dd80b4020 con 0x55ca4d7d8330 2024-04-21T08:44:57.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.649+0000 7f6def573740 10 client.18129 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:57.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.649+0000 7f6def573740 1 -- 172.21.15.43:0/2620978692 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ca4d5197d0 con 0x55ca4d7d8330 2024-04-21T08:44:57.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.649+0000 7f6dcdffb640 1 -- 172.21.15.43:0/2620978692 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ddc02c7e0 con 0x55ca4d7d8330 2024-04-21T08:44:57.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.649+0000 7f6dcdffb640 1 -- 172.21.15.43:0/2620978692 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6dd8005210 con 0x55ca4d7d8330 2024-04-21T08:44:57.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.649+0000 7f6def573740 10 client.18129 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:57.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.649+0000 7f6def573740 10 client.18129 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:44:57.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.649+0000 7f6def573740 20 client.18129 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:57.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.649+0000 7f6def573740 1 --2- 172.21.15.43:0/2620978692 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55ca4d7f6040 0x55ca4d7f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.649+0000 7f6def573740 4 client.18129 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:57.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.649+0000 7f6def573740 1 -- 172.21.15.43:0/2620978692 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca4d6dc930 con 0x55ca4d7f6040 2024-04-21T08:44:57.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.650+0000 7f6de5d9b640 1 --2- 172.21.15.43:0/2620978692 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55ca4d7f6040 0x55ca4d7f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.650+0000 7f6de5d9b640 1 --2- 172.21.15.43:0/2620978692 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55ca4d7f6040 0x55ca4d7f84d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b769546aeaa7f141 server_cookie=21828a46fc7ef332 in_seq=0 out_seq=0 2024-04-21T08:44:57.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.650+0000 7f6dcdffb640 10 client.18129 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:44:57.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.651+0000 7f6dcdffb640 1 -- 172.21.15.43:0/2620978692 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ca4d6dc930 con 0x55ca4d7f6040 2024-04-21T08:44:57.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.651+0000 7f6dcdffb640 10 client.18129 handle_command_reply: tid=0 2024-04-21T08:44:57.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.651+0000 7f6def573740 1 client.18129 shutdown 2024-04-21T08:44:57.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.651+0000 7f6def573740 1 -- 172.21.15.43:0/2620978692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7d8330 msgr2=0x55ca4d7e8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.651+0000 7f6def573740 1 --2- 172.21.15.43:0/2620978692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7d8330 0x55ca4d7e8200 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80cdde0 tx=0x7f6dd809c020 comp rx=0 tx=0).stop 2024-04-21T08:44:57.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.651+0000 7f6def573740 1 -- 172.21.15.43:0/2620978692 shutdown_connections 2024-04-21T08:44:57.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.651+0000 7f6def573740 1 --2- 172.21.15.43:0/2620978692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca4d7e1e40 0x55ca4d7e5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.651+0000 7f6def573740 1 --2- 172.21.15.43:0/2620978692 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55ca4d7f6040 0x55ca4d7f84d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.651+0000 7f6def573740 1 --2- 172.21.15.43:0/2620978692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca4d7d8ce0 0x55ca4d7e55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.651+0000 7f6def573740 1 --2- 172.21.15.43:0/2620978692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7d8330 0x55ca4d7e8200 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.651+0000 7f6dcdffb640 0 client.18129 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:44:57.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.651+0000 7f6def573740 1 -- 172.21.15.43:0/2620978692 >> 172.21.15.43:0/2620978692 conn(0x55ca4d7d2ba0 msgr2=0x55ca4d7d48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.651+0000 7f6def573740 1 -- 172.21.15.43:0/2620978692 shutdown_connections 2024-04-21T08:44:57.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.651+0000 7f6def573740 1 -- 172.21.15.43:0/2620978692 wait complete. 2024-04-21T08:44:57.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.651+0000 7f6def573740 20 client.18129 trim_cache size 0 max 16384 2024-04-21T08:44:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.665+0000 7f6def573740 1 Processor -- start 2024-04-21T08:44:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.665+0000 7f6def573740 1 -- start start 2024-04-21T08:44:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.665+0000 7f6def573740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca4d7d8330 0x55ca4d7e3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.665+0000 7f6def573740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7d8ce0 0x55ca4d7e5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.665+0000 7f6def573740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca4d7e1e40 0x55ca4d7f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.665+0000 7f6def573740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81ad260 con 0x55ca4d7d8ce0 2024-04-21T08:44:57.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.665+0000 7f6def573740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81aeda0 con 0x55ca4d7e1e40 2024-04-21T08:44:57.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.665+0000 7f6def573740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81ad930 con 0x55ca4d7d8330 2024-04-21T08:44:57.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.665+0000 7f6de5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7d8ce0 0x55ca4d7e5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.665+0000 7f6de5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7d8ce0 0x55ca4d7e5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33292/0 (socket says 172.21.15.43:33292) 2024-04-21T08:44:57.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.665+0000 7f6de5d9b640 1 -- 172.21.15.43:0/132053477 learned_addr learned my addr 172.21.15.43:0/132053477 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.665+0000 7f6de659c640 1 --2- 172.21.15.43:0/132053477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca4d7d8330 0x55ca4d7e3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.665+0000 7f6de6d9d640 1 --2- 172.21.15.43:0/132053477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca4d7e1e40 0x55ca4d7f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.665+0000 7f6de5d9b640 1 -- 172.21.15.43:0/132053477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca4d7d8330 msgr2=0x55ca4d7e3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.665+0000 7f6de5d9b640 1 --2- 172.21.15.43:0/132053477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca4d7d8330 0x55ca4d7e3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.665+0000 7f6de5d9b640 1 -- 172.21.15.43:0/132053477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca4d7e1e40 msgr2=0x55ca4d7f1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.665+0000 7f6de5d9b640 1 --2- 172.21.15.43:0/132053477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca4d7e1e40 0x55ca4d7f1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.665+0000 7f6de5d9b640 1 -- 172.21.15.43:0/132053477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca4d7d7560 con 0x55ca4d7d8ce0 2024-04-21T08:44:57.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.665+0000 7f6de5d9b640 1 --2- 172.21.15.43:0/132053477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7d8ce0 0x55ca4d7e5b00 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc000c00 tx=0x7f6ddc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8b130040ed2df6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.666+0000 7f6dcffff640 1 -- 172.21.15.43:0/132053477 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc043040 con 0x55ca4d7d8ce0 2024-04-21T08:44:57.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.666+0000 7f6dcffff640 1 -- 172.21.15.43:0/132053477 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ddc02c7e0 con 0x55ca4d7d8ce0 2024-04-21T08:44:57.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.666+0000 7f6dcffff640 1 -- 172.21.15.43:0/132053477 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc053400 con 0x55ca4d7d8ce0 2024-04-21T08:44:57.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.666+0000 7f6def573740 1 -- 172.21.15.43:0/132053477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7d8ce0 msgr2=0x55ca4d7e5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.666+0000 7f6def573740 1 --2- 172.21.15.43:0/132053477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7d8ce0 0x55ca4d7e5b00 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc000c00 tx=0x7f6ddc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:57.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.666+0000 7f6def573740 1 -- 172.21.15.43:0/132053477 shutdown_connections 2024-04-21T08:44:57.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.666+0000 7f6def573740 1 --2- 172.21.15.43:0/132053477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca4d7e1e40 0x55ca4d7f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.666+0000 7f6def573740 1 --2- 172.21.15.43:0/132053477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7d8ce0 0x55ca4d7e5b00 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.666+0000 7f6def573740 1 --2- 172.21.15.43:0/132053477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca4d7d8330 0x55ca4d7e3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.666+0000 7f6def573740 1 -- 172.21.15.43:0/132053477 >> 172.21.15.43:0/132053477 conn(0x55ca4d7d2ba0 msgr2=0x55ca4d7d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.666+0000 7f6def573740 1 -- 172.21.15.43:0/132053477 shutdown_connections 2024-04-21T08:44:57.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.666+0000 7f6def573740 1 -- 172.21.15.43:0/132053477 wait complete. 2024-04-21T08:44:57.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.667+0000 7f6def573740 1 -- start start 2024-04-21T08:44:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.667+0000 7f6def573740 1 Processor -- start 2024-04-21T08:44:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.667+0000 7f6def573740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7d8330 0x55ca4d7e7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.667+0000 7f6def573740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca4d7d8ce0 0x55ca4d7e7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.667+0000 7f6def573740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca4d7e1e40 0x55ca4d85b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.667+0000 7f6def573740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81ae6d0 con 0x55ca4d7d8330 2024-04-21T08:44:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.667+0000 7f6def573740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81ae000 con 0x55ca4d7d8ce0 2024-04-21T08:44:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.667+0000 7f6def573740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81ab240 con 0x55ca4d7e1e40 2024-04-21T08:44:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.667+0000 7f6def573740 10 client.0 _ll_register_callbacks cb 0x7f6dd809c180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.667+0000 7f6de659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7d8330 0x55ca4d7e7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.667+0000 7f6de659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7d8330 0x55ca4d7e7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33294/0 (socket says 172.21.15.43:33294) 2024-04-21T08:44:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.667+0000 7f6de659c640 1 -- 172.21.15.43:0/1029731703 learned_addr learned my addr 172.21.15.43:0/1029731703 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.667+0000 7f6de5d9b640 1 --2- 172.21.15.43:0/1029731703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca4d7d8ce0 0x55ca4d7e7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.667+0000 7f6de6d9d640 1 --2- 172.21.15.43:0/1029731703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca4d7e1e40 0x55ca4d85b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.667+0000 7f6de659c640 1 -- 172.21.15.43:0/1029731703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca4d7e1e40 msgr2=0x55ca4d85b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.667+0000 7f6de659c640 1 --2- 172.21.15.43:0/1029731703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca4d7e1e40 0x55ca4d85b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.667+0000 7f6de659c640 1 -- 172.21.15.43:0/1029731703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca4d7d8ce0 msgr2=0x55ca4d7e7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.667+0000 7f6de659c640 1 --2- 172.21.15.43:0/1029731703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca4d7d8ce0 0x55ca4d7e7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.667+0000 7f6de659c640 1 -- 172.21.15.43:0/1029731703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca4d85bb00 con 0x55ca4d7d8330 2024-04-21T08:44:57.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.667+0000 7f6de659c640 1 --2- 172.21.15.43:0/1029731703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7d8330 0x55ca4d7e7510 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f6dd809c2d0 tx=0x7f6dd80ce210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d39feb6204366 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.667+0000 7f6dcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:57.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.668+0000 7f6dcdffb640 1 -- 172.21.15.43:0/1029731703 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd80b9070 con 0x55ca4d7d8330 2024-04-21T08:44:57.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.668+0000 7f6dcdffb640 1 -- 172.21.15.43:0/1029731703 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f6dd8085b10 con 0x55ca4d7d8330 2024-04-21T08:44:57.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.668+0000 7f6def573740 1 -- 172.21.15.43:0/1029731703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ddc02c7e0 con 0x55ca4d7d8330 2024-04-21T08:44:57.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.668+0000 7f6dcdffb640 1 -- 172.21.15.43:0/1029731703 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6dd801d040 con 0x55ca4d7d8330 2024-04-21T08:44:57.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.668+0000 7f6def573740 10 client.18135 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:57.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.668+0000 7f6def573740 1 -- 172.21.15.43:0/1029731703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ca4d85cc40 con 0x55ca4d7d8330 2024-04-21T08:44:57.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.668+0000 7f6dcdffb640 1 -- 172.21.15.43:0/1029731703 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ddc02c7e0 con 0x55ca4d7d8330 2024-04-21T08:44:57.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.668+0000 7f6dcdffb640 1 -- 172.21.15.43:0/1029731703 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6dd8098320 con 0x55ca4d7d8330 2024-04-21T08:44:57.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.668+0000 7f6def573740 10 client.18135 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:57.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.668+0000 7f6def573740 10 client.18135 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:44:57.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.668+0000 7f6def573740 20 client.18135 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:57.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.668+0000 7f6def573740 1 --2- 172.21.15.43:0/1029731703 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55ca4d85d3a0 0x55ca4d85f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.668+0000 7f6def573740 4 client.18135 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:44:57.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.668+0000 7f6def573740 1 -- 172.21.15.43:0/1029731703 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca4d74ac10 con 0x55ca4d85d3a0 2024-04-21T08:44:57.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.669+0000 7f6de5d9b640 1 --2- 172.21.15.43:0/1029731703 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55ca4d85d3a0 0x55ca4d85f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.669+0000 7f6de5d9b640 1 --2- 172.21.15.43:0/1029731703 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55ca4d85d3a0 0x55ca4d85f7c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b1c3ef8aa63aa61 server_cookie=3ec60e087e6f2ba5 in_seq=0 out_seq=0 2024-04-21T08:44:57.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.669+0000 7f6dcdffb640 10 client.18135 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:44:57.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.669+0000 7f6dcdffb640 1 -- 172.21.15.43:0/1029731703 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca4d74ac10 con 0x55ca4d85d3a0 2024-04-21T08:44:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.669+0000 7f6dcdffb640 10 client.18135 handle_command_reply: tid=0 2024-04-21T08:44:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.669+0000 7f6def573740 1 client.18135 shutdown 2024-04-21T08:44:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.669+0000 7f6def573740 1 -- 172.21.15.43:0/1029731703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7d8330 msgr2=0x55ca4d7e7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.669+0000 7f6def573740 1 --2- 172.21.15.43:0/1029731703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7d8330 0x55ca4d7e7510 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f6dd809c2d0 tx=0x7f6dd80ce210 comp rx=0 tx=0).stop 2024-04-21T08:44:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.669+0000 7f6def573740 1 -- 172.21.15.43:0/1029731703 shutdown_connections 2024-04-21T08:44:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.669+0000 7f6def573740 1 --2- 172.21.15.43:0/1029731703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ca4d7e1e40 0x55ca4d85b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.669+0000 7f6def573740 1 --2- 172.21.15.43:0/1029731703 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55ca4d85d3a0 0x55ca4d85f7c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.669+0000 7f6def573740 1 --2- 172.21.15.43:0/1029731703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca4d7d8ce0 0x55ca4d7e7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.669+0000 7f6def573740 1 --2- 172.21.15.43:0/1029731703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca4d7d8330 0x55ca4d7e7510 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.669+0000 7f6dcdffb640 0 client.18135 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:44:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.669+0000 7f6def573740 1 -- 172.21.15.43:0/1029731703 >> 172.21.15.43:0/1029731703 conn(0x55ca4d7d2ba0 msgr2=0x55ca4d7d3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.669+0000 7f6def573740 1 -- 172.21.15.43:0/1029731703 shutdown_connections 2024-04-21T08:44:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.670+0000 7f6def573740 1 -- 172.21.15.43:0/1029731703 wait complete. 2024-04-21T08:44:57.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.670+0000 7f6def573740 20 client.18135 trim_cache size 0 max 16384 2024-04-21T08:44:57.678 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:44:57.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.670+0000 7f6dec985640 1 -- 172.21.15.43:0/4186234076 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6dd00a3d20 msgr2=0x7f6dd00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.670+0000 7f6dec985640 1 --2- 172.21.15.43:0/4186234076 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6dd00a3d20 0x7f6dd00a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009370 tx=0x7f6de0004a40 comp rx=0 tx=0).stop 2024-04-21T08:44:57.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.670+0000 7f6dec985640 1 -- 172.21.15.43:0/4186234076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6de806a840 msgr2=0x7f6de8198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.670+0000 7f6dec985640 1 --2- 172.21.15.43:0/4186234076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6de806a840 0x7f6de8198730 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800c970 tx=0x7f6dd800ce30 comp rx=0 tx=0).stop 2024-04-21T08:44:57.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.670+0000 7f6dec985640 1 -- 172.21.15.43:0/4186234076 shutdown_connections 2024-04-21T08:44:57.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.670+0000 7f6dec985640 1 --2- 172.21.15.43:0/4186234076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6de819cff0 0x7f6de819d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.670+0000 7f6dec985640 1 --2- 172.21.15.43:0/4186234076 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6dd00a3d20 0x7f6dd00a61e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.670+0000 7f6dec985640 1 --2- 172.21.15.43:0/4186234076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6de8107270 0x7f6de8198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.670+0000 7f6dec985640 1 --2- 172.21.15.43:0/4186234076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6de806a840 0x7f6de8198730 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.670+0000 7f6dec985640 1 -- 172.21.15.43:0/4186234076 >> 172.21.15.43:0/4186234076 conn(0x7f6de8077eb0 msgr2=0x7f6de80ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.670+0000 7f6dec985640 1 -- 172.21.15.43:0/4186234076 shutdown_connections 2024-04-21T08:44:57.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.670+0000 7f6dec985640 1 -- 172.21.15.43:0/4186234076 wait complete. 2024-04-21T08:44:57.681 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:44:57.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.785+0000 7f05d81bd640 1 -- 172.21.15.43:0/3129453001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05d0107260 msgr2=0x7f05d010ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.785+0000 7f05d81bd640 1 --2- 172.21.15.43:0/3129453001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05d0107260 0x7f05d010ee30 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f05c8009ca0 tx=0x7f05c802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.785+0000 7f05d81bd640 1 -- 172.21.15.43:0/3129453001 shutdown_connections 2024-04-21T08:44:57.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.785+0000 7f05d81bd640 1 --2- 172.21.15.43:0/3129453001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05d0107260 0x7f05d010ee30 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.785+0000 7f05d81bd640 1 --2- 172.21.15.43:0/3129453001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05d0102b40 0x7f05d0106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.785+0000 7f05d81bd640 1 --2- 172.21.15.43:0/3129453001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05d0102190 0x7f05d0102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.785+0000 7f05d81bd640 1 -- 172.21.15.43:0/3129453001 >> 172.21.15.43:0/3129453001 conn(0x7f05d00793a0 msgr2=0x7f05d00797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.785+0000 7f05d81bd640 1 -- 172.21.15.43:0/3129453001 shutdown_connections 2024-04-21T08:44:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.785+0000 7f05d81bd640 1 -- 172.21.15.43:0/3129453001 wait complete. 2024-04-21T08:44:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.786+0000 7f05d81bd640 1 Processor -- start 2024-04-21T08:44:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.786+0000 7f05d81bd640 1 -- start start 2024-04-21T08:44:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.786+0000 7f05d81bd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05d0102190 0x7f05d019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.786+0000 7f05d81bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05d0107260 0x7f05d019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.786+0000 7f05d81bd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05d0198590 0x7f05d0198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.786+0000 7f05d81bd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f05d01121d0 con 0x7f05d0102190 2024-04-21T08:44:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.786+0000 7f05d81bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f05d0112050 con 0x7f05d0107260 2024-04-21T08:44:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.786+0000 7f05d81bd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05d0112350 con 0x7f05d0198590 2024-04-21T08:44:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.786+0000 7f05d5f59640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05d0102190 0x7f05d019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.786+0000 7f05d5f59640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05d0102190 0x7f05d019e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33322/0 (socket says 172.21.15.43:33322) 2024-04-21T08:44:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.786+0000 7f05d5f59640 1 -- 172.21.15.43:0/238476289 learned_addr learned my addr 172.21.15.43:0/238476289 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.787+0000 7f05d5758640 1 --2- 172.21.15.43:0/238476289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05d0107260 0x7f05d019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.787+0000 7f05d675a640 1 --2- 172.21.15.43:0/238476289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05d0198590 0x7f05d0198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.787+0000 7f05d5f59640 1 -- 172.21.15.43:0/238476289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05d0198590 msgr2=0x7f05d0198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.787+0000 7f05d5f59640 1 --2- 172.21.15.43:0/238476289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05d0198590 0x7f05d0198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.787+0000 7f05d5f59640 1 -- 172.21.15.43:0/238476289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05d0107260 msgr2=0x7f05d019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.787+0000 7f05d5f59640 1 --2- 172.21.15.43:0/238476289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05d0107260 0x7f05d019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.787+0000 7f05d5f59640 1 -- 172.21.15.43:0/238476289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d01990b0 con 0x7f05d0102190 2024-04-21T08:44:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.787+0000 7f05d5758640 1 --2- 172.21.15.43:0/238476289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05d0107260 0x7f05d019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.787+0000 7f05d5f59640 1 --2- 172.21.15.43:0/238476289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05d0102190 0x7f05d019e3e0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f05cc00e9f0 tx=0x7f05cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.787+0000 7f05c2ffd640 1 -- 172.21.15.43:0/238476289 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc00cd80 con 0x7f05d0102190 2024-04-21T08:44:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.787+0000 7f05d81bd640 1 -- 172.21.15.43:0/238476289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d01993a0 con 0x7f05d0102190 2024-04-21T08:44:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.787+0000 7f05c2ffd640 1 -- 172.21.15.43:0/238476289 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05cc0054d0 con 0x7f05d0102190 2024-04-21T08:44:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.787+0000 7f05c2ffd640 1 -- 172.21.15.43:0/238476289 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc010690 con 0x7f05d0102190 2024-04-21T08:44:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.787+0000 7f05d81bd640 1 -- 172.21.15.43:0/238476289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d01a55a0 con 0x7f05d0102190 2024-04-21T08:44:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.789+0000 7f05c2ffd640 1 -- 172.21.15.43:0/238476289 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f05cc004e90 con 0x7f05d0102190 2024-04-21T08:44:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.789+0000 7f05dadb4740 1 Processor -- start 2024-04-21T08:44:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.790+0000 7f05dadb4740 1 -- start start 2024-04-21T08:44:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.790+0000 7f05dadb4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a03b0 0x55e4ad8a0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.790+0000 7f05dadb4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4ad8a0d60 0x55e4ad8a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.790+0000 7f05dadb4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4ad8a9e60 0x55e4ad8aa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.790+0000 7f05dadb4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e4ad8051f0 con 0x55e4ad8a03b0 2024-04-21T08:44:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.790+0000 7f05dadb4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e4ad7a24b0 con 0x55e4ad8a9e60 2024-04-21T08:44:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.790+0000 7f05dadb4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e4ad8011b0 con 0x55e4ad8a0d60 2024-04-21T08:44:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.790+0000 7f05d675a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a03b0 0x55e4ad8a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.790+0000 7f05c2ffd640 1 --2- 172.21.15.43:0/238476289 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f05bc0a3d20 0x7f05bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.790+0000 7f05d675a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a03b0 0x55e4ad8a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33336/0 (socket says 172.21.15.43:33336) 2024-04-21T08:44:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.790+0000 7f05d675a640 1 -- 172.21.15.43:0/3857234622 learned_addr learned my addr 172.21.15.43:0/3857234622 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.790+0000 7f05d675a640 1 --2- 172.21.15.43:0/238476289 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f05bc0a3d20 0x7f05bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.790+0000 7f05d5758640 1 --2- 172.21.15.43:0/3857234622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4ad8a9e60 0x55e4ad8aa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.790+0000 7f05c2ffd640 1 -- 172.21.15.43:0/238476289 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f05cc0d1190 con 0x7f05d0102190 2024-04-21T08:44:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.790+0000 7f05d5f59640 1 --2- 172.21.15.43:0/3857234622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4ad8a0d60 0x55e4ad8a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.790+0000 7f05d675a640 1 -- 172.21.15.43:0/3857234622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4ad8a0d60 msgr2=0x55e4ad8a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.790+0000 7f05d675a640 1 --2- 172.21.15.43:0/3857234622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4ad8a0d60 0x55e4ad8a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.790+0000 7f05d675a640 1 -- 172.21.15.43:0/3857234622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4ad8a9e60 msgr2=0x55e4ad8aa2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.790+0000 7f05d675a640 1 --2- 172.21.15.43:0/3857234622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4ad8a9e60 0x55e4ad8aa2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.790+0000 7f05d675a640 1 -- 172.21.15.43:0/3857234622 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4ad7941b0 con 0x55e4ad8a03b0 2024-04-21T08:44:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.790+0000 7f05d5f59640 1 --2- 172.21.15.43:0/3857234622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4ad8a0d60 0x55e4ad8a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.791+0000 7f05d675a640 1 --2- 172.21.15.43:0/3857234622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a03b0 0x55e4ad8a0790 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f05c8006010 tx=0x7f05c8032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bfe56dafe21369 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.791+0000 7f05d675a640 1 --2- 172.21.15.43:0/238476289 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f05bc0a3d20 0x7f05bc0a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f05c802c700 tx=0x7f05c8040000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.791+0000 7f05bbfff640 1 -- 172.21.15.43:0/3857234622 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c8039040 con 0x55e4ad8a03b0 2024-04-21T08:44:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.791+0000 7f05bbfff640 1 -- 172.21.15.43:0/3857234622 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05c8050070 con 0x55e4ad8a03b0 2024-04-21T08:44:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.791+0000 7f05bbfff640 1 -- 172.21.15.43:0/3857234622 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c8034440 con 0x55e4ad8a03b0 2024-04-21T08:44:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.792+0000 7f05dadb4740 1 -- 172.21.15.43:0/3857234622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a03b0 msgr2=0x55e4ad8a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.792+0000 7f05dadb4740 1 --2- 172.21.15.43:0/3857234622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a03b0 0x55e4ad8a0790 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f05c8006010 tx=0x7f05c8032f90 comp rx=0 tx=0).stop 2024-04-21T08:44:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.792+0000 7f05dadb4740 1 -- 172.21.15.43:0/3857234622 shutdown_connections 2024-04-21T08:44:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.792+0000 7f05dadb4740 1 --2- 172.21.15.43:0/3857234622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4ad8a9e60 0x55e4ad8aa2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.792+0000 7f05dadb4740 1 --2- 172.21.15.43:0/3857234622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4ad8a0d60 0x55e4ad8a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.792+0000 7f05dadb4740 1 --2- 172.21.15.43:0/3857234622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a03b0 0x55e4ad8a0790 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.792+0000 7f05dadb4740 1 -- 172.21.15.43:0/3857234622 >> 172.21.15.43:0/3857234622 conn(0x55e4ad89aba0 msgr2=0x55e4ad89af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.792+0000 7f05dadb4740 1 -- 172.21.15.43:0/3857234622 shutdown_connections 2024-04-21T08:44:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.792+0000 7f05dadb4740 1 -- 172.21.15.43:0/3857234622 wait complete. 2024-04-21T08:44:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.792+0000 7f05dadb4740 1 -- start start 2024-04-21T08:44:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.792+0000 7f05dadb4740 1 Processor -- start 2024-04-21T08:44:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.793+0000 7f05dadb4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a03b0 0x55e4ad8b0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.793+0000 7f05dadb4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4ad8a0d60 0x55e4ad8ad700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.793+0000 7f05dadb4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4ad8a9e60 0x55e4ad8add10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.793+0000 7f05dadb4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e4ad8b0d40 con 0x55e4ad8a03b0 2024-04-21T08:44:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.793+0000 7f05dadb4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e4ad7be500 con 0x55e4ad8a0d60 2024-04-21T08:44:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.793+0000 7f05dadb4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e4ad8b0ec0 con 0x55e4ad8a9e60 2024-04-21T08:44:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.793+0000 7f05dadb4740 10 client.0 _ll_register_callbacks cb 0x55e4ad78cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.793+0000 7f05d5f59640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a03b0 0x55e4ad8b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.793+0000 7f05d5f59640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a03b0 0x55e4ad8b0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33340/0 (socket says 172.21.15.43:33340) 2024-04-21T08:44:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.793+0000 7f05d5f59640 1 -- 172.21.15.43:0/1680607472 learned_addr learned my addr 172.21.15.43:0/1680607472 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.793+0000 7f05d675a640 1 --2- 172.21.15.43:0/1680607472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4ad8a9e60 0x55e4ad8add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.793+0000 7f05d5758640 1 --2- 172.21.15.43:0/1680607472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4ad8a0d60 0x55e4ad8ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.793+0000 7f05d5f59640 1 -- 172.21.15.43:0/1680607472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4ad8a9e60 msgr2=0x55e4ad8add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.793+0000 7f05d5f59640 1 --2- 172.21.15.43:0/1680607472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4ad8a9e60 0x55e4ad8add10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.793+0000 7f05d5f59640 1 -- 172.21.15.43:0/1680607472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4ad8a0d60 msgr2=0x55e4ad8ad700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.793+0000 7f05d5f59640 1 --2- 172.21.15.43:0/1680607472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4ad8a0d60 0x55e4ad8ad700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.793+0000 7f05d5f59640 1 -- 172.21.15.43:0/1680607472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4ad5173f0 con 0x55e4ad8a03b0 2024-04-21T08:44:57.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.793+0000 7f05d675a640 1 --2- 172.21.15.43:0/1680607472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4ad8a9e60 0x55e4ad8add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:57.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.793+0000 7f05d5f59640 1 --2- 172.21.15.43:0/1680607472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a03b0 0x55e4ad8b0320 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f05cc00e9c0 tx=0x7f05cc09d4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74cc8b196e5001ec server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.793+0000 7f05b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:57.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.793+0000 7f05b97fa640 1 -- 172.21.15.43:0/1680607472 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc0b5070 con 0x55e4ad8a03b0 2024-04-21T08:44:57.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.793+0000 7f05b97fa640 1 -- 172.21.15.43:0/1680607472 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f05cc0ce730 con 0x55e4ad8a03b0 2024-04-21T08:44:57.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.793+0000 7f05dadb4740 1 -- 172.21.15.43:0/1680607472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05c8050070 con 0x55e4ad8a03b0 2024-04-21T08:44:57.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.794+0000 7f05b97fa640 1 -- 172.21.15.43:0/1680607472 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f05cc0b6270 con 0x55e4ad8a03b0 2024-04-21T08:44:57.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.794+0000 7f05dadb4740 10 client.18147 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:57.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.794+0000 7f05dadb4740 1 -- 172.21.15.43:0/1680607472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e4ad5e17d0 con 0x55e4ad8a03b0 2024-04-21T08:44:57.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.794+0000 7f05b97fa640 1 -- 172.21.15.43:0/1680607472 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05c8050070 con 0x55e4ad8a03b0 2024-04-21T08:44:57.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.794+0000 7f05b97fa640 1 -- 172.21.15.43:0/1680607472 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f05cc0bd0b0 con 0x55e4ad8a03b0 2024-04-21T08:44:57.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.794+0000 7f05dadb4740 10 client.18147 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:57.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.794+0000 7f05dadb4740 10 client.18147 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:44:57.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.794+0000 7f05dadb4740 20 client.18147 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:57.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.794+0000 7f05dadb4740 1 --2- 172.21.15.43:0/1680607472 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55e4ad8be110 0x55e4ad8c0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.794+0000 7f05dadb4740 4 client.18147 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:57.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.794+0000 7f05dadb4740 1 -- 172.21.15.43:0/1680607472 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4ad7a4930 con 0x55e4ad8be110 2024-04-21T08:44:57.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.794+0000 7f05d5758640 1 --2- 172.21.15.43:0/1680607472 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55e4ad8be110 0x55e4ad8c0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.795+0000 7f05d5758640 1 --2- 172.21.15.43:0/1680607472 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55e4ad8be110 0x55e4ad8c0520 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1ec300f4addc0707 server_cookie=23990b459b027a67 in_seq=0 out_seq=0 2024-04-21T08:44:57.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.795+0000 7f05b97fa640 10 client.18147 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:44:57.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.795+0000 7f05b97fa640 1 -- 172.21.15.43:0/1680607472 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e4ad7a4930 con 0x55e4ad8be110 2024-04-21T08:44:57.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.795+0000 7f05b97fa640 10 client.18147 handle_command_reply: tid=0 2024-04-21T08:44:57.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.795+0000 7f05dadb4740 1 client.18147 shutdown 2024-04-21T08:44:57.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.795+0000 7f05dadb4740 1 -- 172.21.15.43:0/1680607472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a03b0 msgr2=0x55e4ad8b0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.795+0000 7f05dadb4740 1 --2- 172.21.15.43:0/1680607472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a03b0 0x55e4ad8b0320 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f05cc00e9c0 tx=0x7f05cc09d4c0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.796+0000 7f05dadb4740 1 -- 172.21.15.43:0/1680607472 shutdown_connections 2024-04-21T08:44:57.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.796+0000 7f05dadb4740 1 --2- 172.21.15.43:0/1680607472 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55e4ad8be110 0x55e4ad8c0520 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.796+0000 7f05dadb4740 1 --2- 172.21.15.43:0/1680607472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4ad8a9e60 0x55e4ad8add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.796+0000 7f05b97fa640 0 client.18147 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:44:57.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.796+0000 7f05dadb4740 1 --2- 172.21.15.43:0/1680607472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4ad8a0d60 0x55e4ad8ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.796+0000 7f05dadb4740 1 --2- 172.21.15.43:0/1680607472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a03b0 0x55e4ad8b0320 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.796+0000 7f05dadb4740 1 -- 172.21.15.43:0/1680607472 >> 172.21.15.43:0/1680607472 conn(0x55e4ad89aba0 msgr2=0x55e4ad89c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.796+0000 7f05dadb4740 1 -- 172.21.15.43:0/1680607472 shutdown_connections 2024-04-21T08:44:57.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.796+0000 7f05dadb4740 1 -- 172.21.15.43:0/1680607472 wait complete. 2024-04-21T08:44:57.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.796+0000 7f05dadb4740 20 client.18147 trim_cache size 0 max 16384 2024-04-21T08:44:57.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.809+0000 7f05dadb4740 1 Processor -- start 2024-04-21T08:44:57.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.809+0000 7f05dadb4740 1 -- start start 2024-04-21T08:44:57.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.809+0000 7f05dadb4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a03b0 0x55e4ad8b9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.809+0000 7f05dadb4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4ad8a0d60 0x55e4ad8ab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.809+0000 7f05dadb4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4ad8a9e60 0x55e4ad8afde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.809+0000 7f05dadb4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f05d01ad250 con 0x55e4ad8a03b0 2024-04-21T08:44:57.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.809+0000 7f05dadb4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f05d01aed90 con 0x55e4ad8a0d60 2024-04-21T08:44:57.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.809+0000 7f05dadb4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05d01ad920 con 0x55e4ad8a9e60 2024-04-21T08:44:57.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.809+0000 7f05d5f59640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a03b0 0x55e4ad8b9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.809+0000 7f05d5f59640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a03b0 0x55e4ad8b9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33356/0 (socket says 172.21.15.43:33356) 2024-04-21T08:44:57.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.809+0000 7f05d5f59640 1 -- 172.21.15.43:0/160423544 learned_addr learned my addr 172.21.15.43:0/160423544 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.810+0000 7f05d5758640 1 --2- 172.21.15.43:0/160423544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4ad8a0d60 0x55e4ad8ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.810+0000 7f05d5f59640 1 -- 172.21.15.43:0/160423544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4ad8a9e60 msgr2=0x55e4ad8afde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.810+0000 7f05d675a640 1 --2- 172.21.15.43:0/160423544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4ad8a9e60 0x55e4ad8afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.810+0000 7f05d5f59640 1 --2- 172.21.15.43:0/160423544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4ad8a9e60 0x55e4ad8afde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.810+0000 7f05d5f59640 1 -- 172.21.15.43:0/160423544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4ad8a0d60 msgr2=0x55e4ad8ab250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.810+0000 7f05d5f59640 1 --2- 172.21.15.43:0/160423544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4ad8a0d60 0x55e4ad8ab250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.810+0000 7f05d5f59640 1 -- 172.21.15.43:0/160423544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4ad8abc50 con 0x55e4ad8a03b0 2024-04-21T08:44:57.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.810+0000 7f05d5758640 1 --2- 172.21.15.43:0/160423544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4ad8a0d60 0x55e4ad8ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:57.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.810+0000 7f05d675a640 1 --2- 172.21.15.43:0/160423544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4ad8a9e60 0x55e4ad8afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:57.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.810+0000 7f05d5f59640 1 --2- 172.21.15.43:0/160423544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a03b0 0x55e4ad8b9c40 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f05cc0bd010 tx=0x7f05cc09cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7d8584e39d0add server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.810+0000 7f05bb7fe640 1 -- 172.21.15.43:0/160423544 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc0ac070 con 0x55e4ad8a03b0 2024-04-21T08:44:57.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.810+0000 7f05bb7fe640 1 -- 172.21.15.43:0/160423544 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05cc0b6270 con 0x55e4ad8a03b0 2024-04-21T08:44:57.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.810+0000 7f05bb7fe640 1 -- 172.21.15.43:0/160423544 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc0ce730 con 0x55e4ad8a03b0 2024-04-21T08:44:57.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.810+0000 7f05dadb4740 1 -- 172.21.15.43:0/160423544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a03b0 msgr2=0x55e4ad8b9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.810+0000 7f05dadb4740 1 --2- 172.21.15.43:0/160423544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a03b0 0x55e4ad8b9c40 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f05cc0bd010 tx=0x7f05cc09cc00 comp rx=0 tx=0).stop 2024-04-21T08:44:57.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.811+0000 7f05dadb4740 1 -- 172.21.15.43:0/160423544 shutdown_connections 2024-04-21T08:44:57.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.811+0000 7f05dadb4740 1 --2- 172.21.15.43:0/160423544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4ad8a9e60 0x55e4ad8afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.811+0000 7f05dadb4740 1 --2- 172.21.15.43:0/160423544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4ad8a0d60 0x55e4ad8ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.811+0000 7f05dadb4740 1 --2- 172.21.15.43:0/160423544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a03b0 0x55e4ad8b9c40 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.811+0000 7f05dadb4740 1 -- 172.21.15.43:0/160423544 >> 172.21.15.43:0/160423544 conn(0x55e4ad89aba0 msgr2=0x55e4ad8ac730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.811+0000 7f05dadb4740 1 -- 172.21.15.43:0/160423544 shutdown_connections 2024-04-21T08:44:57.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.811+0000 7f05dadb4740 1 -- 172.21.15.43:0/160423544 wait complete. 2024-04-21T08:44:57.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.811+0000 7f05dadb4740 1 -- start start 2024-04-21T08:44:57.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.811+0000 7f05dadb4740 1 Processor -- start 2024-04-21T08:44:57.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.811+0000 7f05dadb4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4ad8a03b0 0x55e4ad8af360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.811+0000 7f05dadb4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a0d60 0x55e4ad8af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.811+0000 7f05dadb4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4ad8a9e60 0x55e4ad8afde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.811+0000 7f05dadb4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f05d01ae6c0 con 0x55e4ad8a0d60 2024-04-21T08:44:57.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.811+0000 7f05dadb4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f05d01adff0 con 0x55e4ad8a9e60 2024-04-21T08:44:57.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.811+0000 7f05dadb4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05d01ab230 con 0x55e4ad8a03b0 2024-04-21T08:44:57.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.812+0000 7f05dadb4740 10 client.0 _ll_register_callbacks cb 0x7f05cc005680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:57.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.812+0000 7f05d5758640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a0d60 0x55e4ad8af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.812+0000 7f05d5758640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a0d60 0x55e4ad8af8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33372/0 (socket says 172.21.15.43:33372) 2024-04-21T08:44:57.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.812+0000 7f05d5758640 1 -- 172.21.15.43:0/157441865 learned_addr learned my addr 172.21.15.43:0/157441865 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.812+0000 7f05d675a640 1 --2- 172.21.15.43:0/157441865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4ad8a9e60 0x55e4ad8afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.812+0000 7f05d5f59640 1 --2- 172.21.15.43:0/157441865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4ad8a03b0 0x55e4ad8af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.812+0000 7f05d5758640 1 -- 172.21.15.43:0/157441865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4ad8a03b0 msgr2=0x55e4ad8af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.812+0000 7f05d5758640 1 --2- 172.21.15.43:0/157441865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4ad8a03b0 0x55e4ad8af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.812+0000 7f05d5758640 1 -- 172.21.15.43:0/157441865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4ad8a9e60 msgr2=0x55e4ad8afde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.812+0000 7f05d5758640 1 --2- 172.21.15.43:0/157441865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4ad8a9e60 0x55e4ad8afde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.812+0000 7f05d5758640 1 -- 172.21.15.43:0/157441865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4ad8b43f0 con 0x55e4ad8a0d60 2024-04-21T08:44:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.812+0000 7f05d5758640 1 --2- 172.21.15.43:0/157441865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a0d60 0x55e4ad8af8a0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f05c40057a0 tx=0x7f05c401bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0124fbfa43fdf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.812+0000 7f05b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.812+0000 7f05b97fa640 1 -- 172.21.15.43:0/157441865 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c4005030 con 0x55e4ad8a0d60 2024-04-21T08:44:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.812+0000 7f05dadb4740 1 -- 172.21.15.43:0/157441865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05cc0b6270 con 0x55e4ad8a0d60 2024-04-21T08:44:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.812+0000 7f05b97fa640 1 -- 172.21.15.43:0/157441865 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f05c40051d0 con 0x55e4ad8a0d60 2024-04-21T08:44:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.812+0000 7f05b97fa640 1 -- 172.21.15.43:0/157441865 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f05c402b050 con 0x55e4ad8a0d60 2024-04-21T08:44:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.812+0000 7f05dadb4740 10 client.18153 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.812+0000 7f05dadb4740 1 -- 172.21.15.43:0/157441865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e4ad9249e0 con 0x55e4ad8a0d60 2024-04-21T08:44:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.813+0000 7f05b97fa640 1 -- 172.21.15.43:0/157441865 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05cc0b6270 con 0x55e4ad8a0d60 2024-04-21T08:44:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.813+0000 7f05b97fa640 1 -- 172.21.15.43:0/157441865 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f05c400f040 con 0x55e4ad8a0d60 2024-04-21T08:44:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.813+0000 7f05dadb4740 10 client.18153 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.813+0000 7f05dadb4740 10 client.18153 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:44:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.813+0000 7f05dadb4740 20 client.18153 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.813+0000 7f05dadb4740 1 --2- 172.21.15.43:0/157441865 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55e4ad8be110 0x55e4ad9271d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.813+0000 7f05dadb4740 4 client.18153 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:44:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.813+0000 7f05dadb4740 1 -- 172.21.15.43:0/157441865 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4ad812c10 con 0x55e4ad8be110 2024-04-21T08:44:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.813+0000 7f05d5f59640 1 --2- 172.21.15.43:0/157441865 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55e4ad8be110 0x55e4ad9271d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.813+0000 7f05d5f59640 1 --2- 172.21.15.43:0/157441865 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55e4ad8be110 0x55e4ad9271d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b6d7c48b440085c server_cookie=1a994a16a1398c96 in_seq=0 out_seq=0 2024-04-21T08:44:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.814+0000 7f05b97fa640 10 client.18153 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:44:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.814+0000 7f05b97fa640 1 -- 172.21.15.43:0/157441865 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4ad812c10 con 0x55e4ad8be110 2024-04-21T08:44:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.814+0000 7f05b97fa640 10 client.18153 handle_command_reply: tid=0 2024-04-21T08:44:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.814+0000 7f05dadb4740 1 client.18153 shutdown 2024-04-21T08:44:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.814+0000 7f05dadb4740 1 -- 172.21.15.43:0/157441865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a0d60 msgr2=0x55e4ad8af8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.814+0000 7f05dadb4740 1 --2- 172.21.15.43:0/157441865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a0d60 0x55e4ad8af8a0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f05c40057a0 tx=0x7f05c401bf90 comp rx=0 tx=0).stop 2024-04-21T08:44:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.814+0000 7f05dadb4740 1 -- 172.21.15.43:0/157441865 shutdown_connections 2024-04-21T08:44:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.814+0000 7f05dadb4740 1 --2- 172.21.15.43:0/157441865 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55e4ad8be110 0x55e4ad9271d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.814+0000 7f05dadb4740 1 --2- 172.21.15.43:0/157441865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4ad8a9e60 0x55e4ad8afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.814+0000 7f05b97fa640 0 client.18153 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:44:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.814+0000 7f05dadb4740 1 --2- 172.21.15.43:0/157441865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4ad8a0d60 0x55e4ad8af8a0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.814+0000 7f05dadb4740 1 --2- 172.21.15.43:0/157441865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4ad8a03b0 0x55e4ad8af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.814+0000 7f05dadb4740 1 -- 172.21.15.43:0/157441865 >> 172.21.15.43:0/157441865 conn(0x55e4ad89aba0 msgr2=0x55e4ad89beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.814+0000 7f05dadb4740 1 -- 172.21.15.43:0/157441865 shutdown_connections 2024-04-21T08:44:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.814+0000 7f05dadb4740 1 -- 172.21.15.43:0/157441865 wait complete. 2024-04-21T08:44:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.814+0000 7f05dadb4740 20 client.18153 trim_cache size 0 max 16384 2024-04-21T08:44:57.821 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:44:57.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.815+0000 7f05d81bd640 1 -- 172.21.15.43:0/238476289 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f05bc0a3d20 msgr2=0x7f05bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.815+0000 7f05d81bd640 1 --2- 172.21.15.43:0/238476289 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f05bc0a3d20 0x7f05bc0a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f05c802c700 tx=0x7f05c8040000 comp rx=0 tx=0).stop 2024-04-21T08:44:57.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.815+0000 7f05d81bd640 1 -- 172.21.15.43:0/238476289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05d0102190 msgr2=0x7f05d019e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.815+0000 7f05d81bd640 1 --2- 172.21.15.43:0/238476289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05d0102190 0x7f05d019e3e0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f05cc00e9f0 tx=0x7f05cc00edf0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.815+0000 7f05d81bd640 1 -- 172.21.15.43:0/238476289 shutdown_connections 2024-04-21T08:44:57.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.815+0000 7f05d81bd640 1 --2- 172.21.15.43:0/238476289 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f05bc0a3d20 0x7f05bc0a61e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.815+0000 7f05d81bd640 1 --2- 172.21.15.43:0/238476289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05d0198590 0x7f05d0198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.815+0000 7f05d81bd640 1 --2- 172.21.15.43:0/238476289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05d0107260 0x7f05d019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.815+0000 7f05d81bd640 1 --2- 172.21.15.43:0/238476289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05d0102190 0x7f05d019e3e0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.815+0000 7f05d81bd640 1 -- 172.21.15.43:0/238476289 >> 172.21.15.43:0/238476289 conn(0x7f05d00793a0 msgr2=0x7f05d00ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.815+0000 7f05d81bd640 1 -- 172.21.15.43:0/238476289 shutdown_connections 2024-04-21T08:44:57.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.815+0000 7f05d81bd640 1 -- 172.21.15.43:0/238476289 wait complete. 2024-04-21T08:44:57.826 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:44:57.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.930+0000 7fbf9984a640 1 -- 172.21.15.43:0/816407340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf94107260 msgr2=0x7fbf9410ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.930+0000 7fbf9984a640 1 --2- 172.21.15.43:0/816407340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf94107260 0x7fbf9410ee30 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c009ca0 tx=0x7fbf8c030500 comp rx=0 tx=0).stop 2024-04-21T08:44:57.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.930+0000 7fbf9984a640 1 -- 172.21.15.43:0/816407340 shutdown_connections 2024-04-21T08:44:57.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.930+0000 7fbf9984a640 1 --2- 172.21.15.43:0/816407340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf94107260 0x7fbf9410ee30 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.930+0000 7fbf9984a640 1 --2- 172.21.15.43:0/816407340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf94102b40 0x7fbf94106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.930+0000 7fbf9984a640 1 --2- 172.21.15.43:0/816407340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf94102190 0x7fbf94102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.930+0000 7fbf9984a640 1 -- 172.21.15.43:0/816407340 >> 172.21.15.43:0/816407340 conn(0x7fbf940fd990 msgr2=0x7fbf940ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.930+0000 7fbf9984a640 1 -- 172.21.15.43:0/816407340 shutdown_connections 2024-04-21T08:44:57.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.930+0000 7fbf9984a640 1 -- 172.21.15.43:0/816407340 wait complete. 2024-04-21T08:44:57.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.931+0000 7fbf9984a640 1 Processor -- start 2024-04-21T08:44:57.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.931+0000 7fbf9984a640 1 -- start start 2024-04-21T08:44:57.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.931+0000 7fbf9984a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf94102190 0x7fbf941965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.931+0000 7fbf9984a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf94107260 0x7fbf94196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.931+0000 7fbf9984a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf9419ae70 0x7fbf9419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.931+0000 7fbf9984a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9410ffd0 con 0x7fbf94107260 2024-04-21T08:44:57.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.931+0000 7fbf9984a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9410fe50 con 0x7fbf9419ae70 2024-04-21T08:44:57.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.931+0000 7fbf9984a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbf94110150 con 0x7fbf94102190 2024-04-21T08:44:57.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.931+0000 7fbf927fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf94107260 0x7fbf94196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.931+0000 7fbf927fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf94107260 0x7fbf94196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33386/0 (socket says 172.21.15.43:33386) 2024-04-21T08:44:57.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.931+0000 7fbf927fc640 1 -- 172.21.15.43:0/3542184314 learned_addr learned my addr 172.21.15.43:0/3542184314 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.931+0000 7fbf937fe640 1 --2- 172.21.15.43:0/3542184314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf9419ae70 0x7fbf9419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.931+0000 7fbf92ffd640 1 --2- 172.21.15.43:0/3542184314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf94102190 0x7fbf941965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.931+0000 7fbf927fc640 1 -- 172.21.15.43:0/3542184314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf94102190 msgr2=0x7fbf941965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.931+0000 7fbf927fc640 1 --2- 172.21.15.43:0/3542184314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf94102190 0x7fbf941965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.931+0000 7fbf927fc640 1 -- 172.21.15.43:0/3542184314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf9419ae70 msgr2=0x7fbf9419b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.931+0000 7fbf927fc640 1 --2- 172.21.15.43:0/3542184314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf9419ae70 0x7fbf9419b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.931+0000 7fbf927fc640 1 -- 172.21.15.43:0/3542184314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9419b960 con 0x7fbf94107260 2024-04-21T08:44:57.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.931+0000 7fbf937fe640 1 --2- 172.21.15.43:0/3542184314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf9419ae70 0x7fbf9419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:57.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.931+0000 7fbf92ffd640 1 --2- 172.21.15.43:0/3542184314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf94102190 0x7fbf941965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:57.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.931+0000 7fbf927fc640 1 --2- 172.21.15.43:0/3542184314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf94107260 0x7fbf94196af0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800e9f0 tx=0x7fbf8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.932+0000 7fbf98848640 1 -- 172.21.15.43:0/3542184314 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8800ccc0 con 0x7fbf94107260 2024-04-21T08:44:57.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.932+0000 7fbf98848640 1 -- 172.21.15.43:0/3542184314 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf880054d0 con 0x7fbf94107260 2024-04-21T08:44:57.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.932+0000 7fbf9984a640 1 -- 172.21.15.43:0/3542184314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf9419bc50 con 0x7fbf94107260 2024-04-21T08:44:57.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.932+0000 7fbf98848640 1 -- 172.21.15.43:0/3542184314 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf88010430 con 0x7fbf94107260 2024-04-21T08:44:57.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.932+0000 7fbf9984a640 1 -- 172.21.15.43:0/3542184314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf941a33f0 con 0x7fbf94107260 2024-04-21T08:44:57.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.934+0000 7fbf98848640 1 -- 172.21.15.43:0/3542184314 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbf880046c0 con 0x7fbf94107260 2024-04-21T08:44:57.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.934+0000 7fbf9c43e740 1 Processor -- start 2024-04-21T08:44:57.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.934+0000 7fbf9c43e740 1 -- start start 2024-04-21T08:44:57.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.934+0000 7fbf9c43e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e510c4d0 0x5591e510c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.934+0000 7fbf9c43e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e510ce80 0x5591e5115430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.934+0000 7fbf9c43e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e5115a00 0x5591e5117e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.934+0000 7fbf9c43e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5591e50711f0 con 0x5591e510ce80 2024-04-21T08:44:57.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.934+0000 7fbf9c43e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5591e500e4b0 con 0x5591e5115a00 2024-04-21T08:44:57.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.934+0000 7fbf9c43e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5591e506d1b0 con 0x5591e510c4d0 2024-04-21T08:44:57.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.935+0000 7fbf92ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e510ce80 0x5591e5115430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.935+0000 7fbf92ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e510ce80 0x5591e5115430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33398/0 (socket says 172.21.15.43:33398) 2024-04-21T08:44:57.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.935+0000 7fbf92ffd640 1 -- 172.21.15.43:0/968802192 learned_addr learned my addr 172.21.15.43:0/968802192 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.935+0000 7fbf937fe640 1 --2- 172.21.15.43:0/968802192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e510c4d0 0x5591e510c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.935+0000 7fbf927fc640 1 --2- 172.21.15.43:0/968802192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e5115a00 0x5591e5117e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.935+0000 7fbf98848640 1 --2- 172.21.15.43:0/3542184314 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbf7c0a3d20 0x7fbf7c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.935+0000 7fbf92ffd640 1 -- 172.21.15.43:0/968802192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e510c4d0 msgr2=0x5591e510c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.935+0000 7fbf92ffd640 1 --2- 172.21.15.43:0/968802192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e510c4d0 0x5591e510c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.935+0000 7fbf92ffd640 1 -- 172.21.15.43:0/968802192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e5115a00 msgr2=0x5591e5117e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.935+0000 7fbf92ffd640 1 --2- 172.21.15.43:0/968802192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e5115a00 0x5591e5117e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.935+0000 7fbf92ffd640 1 -- 172.21.15.43:0/968802192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591e50001b0 con 0x5591e510ce80 2024-04-21T08:44:57.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.935+0000 7fbf937fe640 1 --2- 172.21.15.43:0/3542184314 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbf7c0a3d20 0x7fbf7c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.935+0000 7fbf98848640 1 -- 172.21.15.43:0/3542184314 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fbf880d1050 con 0x7fbf94107260 2024-04-21T08:44:57.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.935+0000 7fbf92ffd640 1 --2- 172.21.15.43:0/968802192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e510ce80 0x5591e5115430 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fbf84005fd0 tx=0x7fbf8400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba1694d958cbf23 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.935+0000 7fbf79ffb640 1 -- 172.21.15.43:0/968802192 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf84016070 con 0x5591e510ce80 2024-04-21T08:44:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.935+0000 7fbf79ffb640 1 -- 172.21.15.43:0/968802192 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf840054d0 con 0x5591e510ce80 2024-04-21T08:44:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.935+0000 7fbf79ffb640 1 -- 172.21.15.43:0/968802192 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf84011470 con 0x5591e510ce80 2024-04-21T08:44:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.936+0000 7fbf9c43e740 1 -- 172.21.15.43:0/968802192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e510ce80 msgr2=0x5591e5115430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.936+0000 7fbf937fe640 1 --2- 172.21.15.43:0/3542184314 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbf7c0a3d20 0x7fbf7c0a61e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c036010 tx=0x7fbf8c0095c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.936+0000 7fbf9c43e740 1 --2- 172.21.15.43:0/968802192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e510ce80 0x5591e5115430 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fbf84005fd0 tx=0x7fbf8400e6c0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.936+0000 7fbf9c43e740 1 -- 172.21.15.43:0/968802192 shutdown_connections 2024-04-21T08:44:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.936+0000 7fbf9c43e740 1 --2- 172.21.15.43:0/968802192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e5115a00 0x5591e5117e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.936+0000 7fbf9c43e740 1 --2- 172.21.15.43:0/968802192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e510ce80 0x5591e5115430 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.936+0000 7fbf9c43e740 1 --2- 172.21.15.43:0/968802192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e510c4d0 0x5591e510c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.936+0000 7fbf9c43e740 1 -- 172.21.15.43:0/968802192 >> 172.21.15.43:0/968802192 conn(0x5591e5106ba0 msgr2=0x5591e5106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.936+0000 7fbf9c43e740 1 -- 172.21.15.43:0/968802192 shutdown_connections 2024-04-21T08:44:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.936+0000 7fbf9c43e740 1 -- 172.21.15.43:0/968802192 wait complete. 2024-04-21T08:44:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.936+0000 7fbf9c43e740 1 -- start start 2024-04-21T08:44:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.937+0000 7fbf9c43e740 1 Processor -- start 2024-04-21T08:44:57.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.937+0000 7fbf9c43e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e510c4d0 0x5591e51152f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.937+0000 7fbf9c43e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e510ce80 0x5591e5112680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.937+0000 7fbf9c43e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e5115a00 0x5591e5112c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.937+0000 7fbf9c43e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5591e5118970 con 0x5591e510c4d0 2024-04-21T08:44:57.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.937+0000 7fbf9c43e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5591e502a500 con 0x5591e5115a00 2024-04-21T08:44:57.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.937+0000 7fbf9c43e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5591e5118af0 con 0x5591e510ce80 2024-04-21T08:44:57.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.937+0000 7fbf9c43e740 10 client.0 _ll_register_callbacks cb 0x5591e4ff8b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:57.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.937+0000 7fbf92ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e510c4d0 0x5591e51152f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.937+0000 7fbf92ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e510c4d0 0x5591e51152f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33410/0 (socket says 172.21.15.43:33410) 2024-04-21T08:44:57.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.937+0000 7fbf92ffd640 1 -- 172.21.15.43:0/2496661216 learned_addr learned my addr 172.21.15.43:0/2496661216 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.937+0000 7fbf937fe640 1 --2- 172.21.15.43:0/2496661216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e5115a00 0x5591e5112c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.937+0000 7fbf927fc640 1 --2- 172.21.15.43:0/2496661216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e510ce80 0x5591e5112680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.937+0000 7fbf92ffd640 1 -- 172.21.15.43:0/2496661216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e510ce80 msgr2=0x5591e5112680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.937+0000 7fbf92ffd640 1 --2- 172.21.15.43:0/2496661216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e510ce80 0x5591e5112680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.937+0000 7fbf92ffd640 1 -- 172.21.15.43:0/2496661216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e5115a00 msgr2=0x5591e5112c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.937+0000 7fbf92ffd640 1 --2- 172.21.15.43:0/2496661216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e5115a00 0x5591e5112c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.937+0000 7fbf92ffd640 1 -- 172.21.15.43:0/2496661216 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591e4d833f0 con 0x5591e510c4d0 2024-04-21T08:44:57.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.937+0000 7fbf92ffd640 1 --2- 172.21.15.43:0/2496661216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e510c4d0 0x5591e51152f0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fbf84000c00 tx=0x7fbf84005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b89db7c013171f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.937+0000 7fbf72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:57.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.938+0000 7fbf72ffd640 1 -- 172.21.15.43:0/2496661216 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf84016040 con 0x5591e510c4d0 2024-04-21T08:44:57.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.938+0000 7fbf72ffd640 1 -- 172.21.15.43:0/2496661216 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fbf84009d60 con 0x5591e510c4d0 2024-04-21T08:44:57.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.938+0000 7fbf9c43e740 1 -- 172.21.15.43:0/2496661216 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf840054d0 con 0x5591e510c4d0 2024-04-21T08:44:57.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.938+0000 7fbf72ffd640 1 -- 172.21.15.43:0/2496661216 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbf84005320 con 0x5591e510c4d0 2024-04-21T08:44:57.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.938+0000 7fbf9c43e740 10 client.18165 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:57.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.938+0000 7fbf9c43e740 1 -- 172.21.15.43:0/2496661216 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5591e4e4d7d0 con 0x5591e510c4d0 2024-04-21T08:44:57.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.938+0000 7fbf72ffd640 1 -- 172.21.15.43:0/2496661216 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf840054d0 con 0x5591e510c4d0 2024-04-21T08:44:57.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.938+0000 7fbf72ffd640 1 -- 172.21.15.43:0/2496661216 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbf8400f040 con 0x5591e510c4d0 2024-04-21T08:44:57.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.938+0000 7fbf9c43e740 10 client.18165 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:57.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.938+0000 7fbf9c43e740 10 client.18165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:57.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.938+0000 7fbf9c43e740 20 client.18165 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:57.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.938+0000 7fbf9c43e740 1 --2- 172.21.15.43:0/2496661216 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5591e5125a80 0x5591e5127e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.938+0000 7fbf9c43e740 4 client.18165 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:57.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.938+0000 7fbf9c43e740 1 -- 172.21.15.43:0/2496661216 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591e5010930 con 0x5591e5125a80 2024-04-21T08:44:57.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.939+0000 7fbf927fc640 1 --2- 172.21.15.43:0/2496661216 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5591e5125a80 0x5591e5127e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.939+0000 7fbf72ffd640 10 client.18165 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:57.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.939+0000 7fbf927fc640 1 --2- 172.21.15.43:0/2496661216 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5591e5125a80 0x5591e5127e70 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bb55d34912095d0 server_cookie=e6b68592172010fe in_seq=0 out_seq=0 2024-04-21T08:44:57.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.940+0000 7fbf72ffd640 1 -- 172.21.15.43:0/2496661216 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5591e5010930 con 0x5591e5125a80 2024-04-21T08:44:57.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.940+0000 7fbf72ffd640 10 client.18165 handle_command_reply: tid=0 2024-04-21T08:44:57.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.940+0000 7fbf9c43e740 1 client.18165 shutdown 2024-04-21T08:44:57.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.940+0000 7fbf9c43e740 1 -- 172.21.15.43:0/2496661216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e510c4d0 msgr2=0x5591e51152f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.940+0000 7fbf9c43e740 1 --2- 172.21.15.43:0/2496661216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e510c4d0 0x5591e51152f0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fbf84000c00 tx=0x7fbf84005870 comp rx=0 tx=0).stop 2024-04-21T08:44:57.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.940+0000 7fbf9c43e740 1 -- 172.21.15.43:0/2496661216 shutdown_connections 2024-04-21T08:44:57.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.940+0000 7fbf9c43e740 1 --2- 172.21.15.43:0/2496661216 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5591e5125a80 0x5591e5127e70 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.940+0000 7fbf9c43e740 1 --2- 172.21.15.43:0/2496661216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e5115a00 0x5591e5112c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.940+0000 7fbf9c43e740 1 --2- 172.21.15.43:0/2496661216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e510ce80 0x5591e5112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.940+0000 7fbf72ffd640 0 client.18165 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:57.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.940+0000 7fbf9c43e740 1 --2- 172.21.15.43:0/2496661216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e510c4d0 0x5591e51152f0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.940+0000 7fbf9c43e740 1 -- 172.21.15.43:0/2496661216 >> 172.21.15.43:0/2496661216 conn(0x5591e5106ba0 msgr2=0x5591e51088f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.940+0000 7fbf9c43e740 1 -- 172.21.15.43:0/2496661216 shutdown_connections 2024-04-21T08:44:57.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.940+0000 7fbf9c43e740 1 -- 172.21.15.43:0/2496661216 wait complete. 2024-04-21T08:44:57.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.940+0000 7fbf9c43e740 20 client.18165 trim_cache size 0 max 16384 2024-04-21T08:44:57.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.953+0000 7fbf9c43e740 1 Processor -- start 2024-04-21T08:44:57.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.953+0000 7fbf9c43e740 1 -- start start 2024-04-21T08:44:57.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.954+0000 7fbf9c43e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e510c4d0 0x5591e5114db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.954+0000 7fbf9c43e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e510ce80 0x5591e51152f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.954+0000 7fbf9c43e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e5115a00 0x5591e512d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.954+0000 7fbf9c43e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbf941ad2c0 con 0x5591e5115a00 2024-04-21T08:44:57.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.954+0000 7fbf9c43e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbf941aee00 con 0x5591e510c4d0 2024-04-21T08:44:57.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.954+0000 7fbf9c43e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbf941ad990 con 0x5591e510ce80 2024-04-21T08:44:57.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.954+0000 7fbf937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e5115a00 0x5591e512d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.954+0000 7fbf937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e5115a00 0x5591e512d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33412/0 (socket says 172.21.15.43:33412) 2024-04-21T08:44:57.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.954+0000 7fbf937fe640 1 -- 172.21.15.43:0/2182515128 learned_addr learned my addr 172.21.15.43:0/2182515128 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.954+0000 7fbf937fe640 1 -- 172.21.15.43:0/2182515128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e510ce80 msgr2=0x5591e51152f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:44:57.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.954+0000 7fbf937fe640 1 --2- 172.21.15.43:0/2182515128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e510ce80 0x5591e51152f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.954+0000 7fbf937fe640 1 -- 172.21.15.43:0/2182515128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e510c4d0 msgr2=0x5591e5114db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.954+0000 7fbf937fe640 1 --2- 172.21.15.43:0/2182515128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e510c4d0 0x5591e5114db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.954+0000 7fbf937fe640 1 -- 172.21.15.43:0/2182515128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591e510cbe0 con 0x5591e5115a00 2024-04-21T08:44:57.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.954+0000 7fbf937fe640 1 --2- 172.21.15.43:0/2182515128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e5115a00 0x5591e512d440 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c009ca0 tx=0x7fbf8c047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4d6f17443be36c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.954+0000 7fbf797fa640 1 -- 172.21.15.43:0/2182515128 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c037600 con 0x5591e5115a00 2024-04-21T08:44:57.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.954+0000 7fbf797fa640 1 -- 172.21.15.43:0/2182515128 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf8c049070 con 0x5591e5115a00 2024-04-21T08:44:57.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.954+0000 7fbf797fa640 1 -- 172.21.15.43:0/2182515128 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c036900 con 0x5591e5115a00 2024-04-21T08:44:57.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.955+0000 7fbf9c43e740 1 -- 172.21.15.43:0/2182515128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e5115a00 msgr2=0x5591e512d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.955+0000 7fbf9c43e740 1 --2- 172.21.15.43:0/2182515128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e5115a00 0x5591e512d440 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c009ca0 tx=0x7fbf8c047000 comp rx=0 tx=0).stop 2024-04-21T08:44:57.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.955+0000 7fbf9c43e740 1 -- 172.21.15.43:0/2182515128 shutdown_connections 2024-04-21T08:44:57.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.955+0000 7fbf9c43e740 1 --2- 172.21.15.43:0/2182515128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e5115a00 0x5591e512d440 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.955+0000 7fbf9c43e740 1 --2- 172.21.15.43:0/2182515128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e510ce80 0x5591e51152f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.955+0000 7fbf9c43e740 1 --2- 172.21.15.43:0/2182515128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e510c4d0 0x5591e5114db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.955+0000 7fbf9c43e740 1 -- 172.21.15.43:0/2182515128 >> 172.21.15.43:0/2182515128 conn(0x5591e5106ba0 msgr2=0x5591e510db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.955+0000 7fbf9c43e740 1 -- 172.21.15.43:0/2182515128 shutdown_connections 2024-04-21T08:44:57.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.955+0000 7fbf9c43e740 1 -- 172.21.15.43:0/2182515128 wait complete. 2024-04-21T08:44:57.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.955+0000 7fbf9c43e740 1 -- start start 2024-04-21T08:44:57.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.955+0000 7fbf9c43e740 1 Processor -- start 2024-04-21T08:44:57.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.956+0000 7fbf9c43e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e510c4d0 0x5591e511df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.956+0000 7fbf9c43e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e510ce80 0x5591e510eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.956+0000 7fbf9c43e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e5115a00 0x5591e510f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.956+0000 7fbf9c43e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbf941ae730 con 0x5591e5115a00 2024-04-21T08:44:57.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.956+0000 7fbf9c43e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbf941ae060 con 0x5591e510c4d0 2024-04-21T08:44:57.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.956+0000 7fbf9c43e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbf941ab2a0 con 0x5591e510ce80 2024-04-21T08:44:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.956+0000 7fbf9c43e740 10 client.0 _ll_register_callbacks cb 0x7fbf840051e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.956+0000 7fbf937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e5115a00 0x5591e510f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.956+0000 7fbf937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e5115a00 0x5591e510f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33420/0 (socket says 172.21.15.43:33420) 2024-04-21T08:44:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.956+0000 7fbf937fe640 1 -- 172.21.15.43:0/4196918038 learned_addr learned my addr 172.21.15.43:0/4196918038 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.956+0000 7fbf92ffd640 1 --2- 172.21.15.43:0/4196918038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e510c4d0 0x5591e511df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.956+0000 7fbf937fe640 1 -- 172.21.15.43:0/4196918038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e510ce80 msgr2=0x5591e510eef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:44:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.956+0000 7fbf937fe640 1 --2- 172.21.15.43:0/4196918038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e510ce80 0x5591e510eef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.956+0000 7fbf937fe640 1 -- 172.21.15.43:0/4196918038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e510c4d0 msgr2=0x5591e511df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.956+0000 7fbf937fe640 1 --2- 172.21.15.43:0/4196918038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e510c4d0 0x5591e511df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.956+0000 7fbf937fe640 1 -- 172.21.15.43:0/4196918038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591e510f970 con 0x5591e5115a00 2024-04-21T08:44:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.956+0000 7fbf92ffd640 1 --2- 172.21.15.43:0/4196918038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e510c4d0 0x5591e511df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.956+0000 7fbf937fe640 1 --2- 172.21.15.43:0/4196918038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e5115a00 0x5591e510f430 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c036040 tx=0x7fbf8c037e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa8c56a91d0fa78 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.956+0000 7fbf72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.956+0000 7fbf72ffd640 1 -- 172.21.15.43:0/4196918038 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c004450 con 0x5591e5115a00 2024-04-21T08:44:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.956+0000 7fbf72ffd640 1 -- 172.21.15.43:0/4196918038 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fbf8c037400 con 0x5591e5115a00 2024-04-21T08:44:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.956+0000 7fbf9c43e740 1 -- 172.21.15.43:0/4196918038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf8c049070 con 0x5591e5115a00 2024-04-21T08:44:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.957+0000 7fbf72ffd640 1 -- 172.21.15.43:0/4196918038 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbf8c06d050 con 0x5591e5115a00 2024-04-21T08:44:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.957+0000 7fbf9c43e740 10 client.18171 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.957+0000 7fbf9c43e740 1 -- 172.21.15.43:0/4196918038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5591e518b9c0 con 0x5591e5115a00 2024-04-21T08:44:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.957+0000 7fbf72ffd640 1 -- 172.21.15.43:0/4196918038 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf8c049070 con 0x5591e5115a00 2024-04-21T08:44:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.957+0000 7fbf72ffd640 1 -- 172.21.15.43:0/4196918038 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbf8c035510 con 0x5591e5115a00 2024-04-21T08:44:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.957+0000 7fbf9c43e740 10 client.18171 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.957+0000 7fbf9c43e740 10 client.18171 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.957+0000 7fbf9c43e740 20 client.18171 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.957+0000 7fbf9c43e740 1 --2- 172.21.15.43:0/4196918038 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5591e518c0a0 0x5591e518e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.957+0000 7fbf9c43e740 4 client.18171 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:44:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.957+0000 7fbf9c43e740 1 -- 172.21.15.43:0/4196918038 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591e507ec10 con 0x5591e518c0a0 2024-04-21T08:44:57.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.957+0000 7fbf92ffd640 1 --2- 172.21.15.43:0/4196918038 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5591e518c0a0 0x5591e518e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.958+0000 7fbf92ffd640 1 --2- 172.21.15.43:0/4196918038 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5591e518c0a0 0x5591e518e4c0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2c2c8edd923a9f2 server_cookie=70197c1a71e12a64 in_seq=0 out_seq=0 2024-04-21T08:44:57.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.958+0000 7fbf72ffd640 10 client.18171 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:57.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.958+0000 7fbf72ffd640 1 -- 172.21.15.43:0/4196918038 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591e507ec10 con 0x5591e518c0a0 2024-04-21T08:44:57.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.958+0000 7fbf72ffd640 10 client.18171 handle_command_reply: tid=0 2024-04-21T08:44:57.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.958+0000 7fbf9c43e740 1 client.18171 shutdown 2024-04-21T08:44:57.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.958+0000 7fbf9c43e740 1 -- 172.21.15.43:0/4196918038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e5115a00 msgr2=0x5591e510f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.958+0000 7fbf9c43e740 1 --2- 172.21.15.43:0/4196918038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e5115a00 0x5591e510f430 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c036040 tx=0x7fbf8c037e00 comp rx=0 tx=0).stop 2024-04-21T08:44:57.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.958+0000 7fbf9c43e740 1 -- 172.21.15.43:0/4196918038 shutdown_connections 2024-04-21T08:44:57.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.958+0000 7fbf9c43e740 1 --2- 172.21.15.43:0/4196918038 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5591e518c0a0 0x5591e518e4c0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.958+0000 7fbf9c43e740 1 --2- 172.21.15.43:0/4196918038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e5115a00 0x5591e510f430 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.958+0000 7fbf9c43e740 1 --2- 172.21.15.43:0/4196918038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e510ce80 0x5591e510eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.958+0000 7fbf72ffd640 0 client.18171 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:57.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.958+0000 7fbf9c43e740 1 --2- 172.21.15.43:0/4196918038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e510c4d0 0x5591e511df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.958+0000 7fbf9c43e740 1 -- 172.21.15.43:0/4196918038 >> 172.21.15.43:0/4196918038 conn(0x5591e5106ba0 msgr2=0x5591e5107ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.958+0000 7fbf9c43e740 1 -- 172.21.15.43:0/4196918038 shutdown_connections 2024-04-21T08:44:57.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.958+0000 7fbf9c43e740 1 -- 172.21.15.43:0/4196918038 wait complete. 2024-04-21T08:44:57.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.958+0000 7fbf9c43e740 20 client.18171 trim_cache size 0 max 16384 2024-04-21T08:44:57.966 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:44:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.959+0000 7fbf9984a640 1 -- 172.21.15.43:0/3542184314 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbf7c0a3d20 msgr2=0x7fbf7c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.959+0000 7fbf9984a640 1 --2- 172.21.15.43:0/3542184314 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbf7c0a3d20 0x7fbf7c0a61e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c036010 tx=0x7fbf8c0095c0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.959+0000 7fbf9984a640 1 -- 172.21.15.43:0/3542184314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf94107260 msgr2=0x7fbf94196af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.959+0000 7fbf9984a640 1 --2- 172.21.15.43:0/3542184314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf94107260 0x7fbf94196af0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800e9f0 tx=0x7fbf8800eeb0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.959+0000 7fbf9984a640 1 -- 172.21.15.43:0/3542184314 shutdown_connections 2024-04-21T08:44:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.959+0000 7fbf9984a640 1 --2- 172.21.15.43:0/3542184314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf9419ae70 0x7fbf9419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.959+0000 7fbf9984a640 1 --2- 172.21.15.43:0/3542184314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf94107260 0x7fbf94196af0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.959+0000 7fbf9984a640 1 --2- 172.21.15.43:0/3542184314 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbf7c0a3d20 0x7fbf7c0a61e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.959+0000 7fbf9984a640 1 --2- 172.21.15.43:0/3542184314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf94102190 0x7fbf941965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.959+0000 7fbf9984a640 1 -- 172.21.15.43:0/3542184314 >> 172.21.15.43:0/3542184314 conn(0x7fbf940fd990 msgr2=0x7fbf940ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.959+0000 7fbf9984a640 1 -- 172.21.15.43:0/3542184314 shutdown_connections 2024-04-21T08:44:57.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:57.959+0000 7fbf9984a640 1 -- 172.21.15.43:0/3542184314 wait complete. 2024-04-21T08:44:58.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:58 smithi175 ceph-mon[31920]: pgmap v184: 129 pgs: 129 active+clean; 4.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 21 op/s 2024-04-21T08:44:58.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:44:58 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2480823216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:58.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:58 smithi112 ceph-mon[31085]: pgmap v184: 129 pgs: 129 active+clean; 4.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 21 op/s 2024-04-21T08:44:58.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:44:58 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2480823216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:58.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:58 smithi043 ceph-mon[27978]: pgmap v184: 129 pgs: 129 active+clean; 4.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 21 op/s 2024-04-21T08:44:58.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:44:58 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2480823216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:58.971 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:44:58.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:44:58.971 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:44:59.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.090+0000 7f5035df4640 1 -- 172.21.15.43:0/320165380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f503010c510 msgr2=0x7f503010e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.090+0000 7f5035df4640 1 --2- 172.21.15.43:0/320165380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f503010c510 0x7f503010e9b0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f5028009ca0 tx=0x7f502802c230 comp rx=0 tx=0).stop 2024-04-21T08:44:59.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.090+0000 7f5035df4640 1 -- 172.21.15.43:0/320165380 shutdown_connections 2024-04-21T08:44:59.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.090+0000 7f5035df4640 1 --2- 172.21.15.43:0/320165380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f503010c510 0x7f503010e9b0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.090+0000 7f5035df4640 1 --2- 172.21.15.43:0/320165380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5030103af0 0x7f503010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.090+0000 7f5035df4640 1 --2- 172.21.15.43:0/320165380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50301031d0 0x7f50301035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.090+0000 7f5035df4640 1 -- 172.21.15.43:0/320165380 >> 172.21.15.43:0/320165380 conn(0x7f5030077f50 msgr2=0x7f5030078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.090+0000 7f5035df4640 1 -- 172.21.15.43:0/320165380 shutdown_connections 2024-04-21T08:44:59.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.090+0000 7f5035df4640 1 -- 172.21.15.43:0/320165380 wait complete. 2024-04-21T08:44:59.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.091+0000 7f5035df4640 1 Processor -- start 2024-04-21T08:44:59.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.091+0000 7f5035df4640 1 -- start start 2024-04-21T08:44:59.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.091+0000 7f5035df4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50301031d0 0x7f50301986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.091+0000 7f5035df4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5030103af0 0x7f5030198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.091+0000 7f5035df4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f503010c510 0x7f503019cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.091+0000 7f5035df4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f50301121a0 con 0x7f503010c510 2024-04-21T08:44:59.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.091+0000 7f5035df4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5030112020 con 0x7f50301031d0 2024-04-21T08:44:59.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.091+0000 7f5035df4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5030112320 con 0x7f5030103af0 2024-04-21T08:44:59.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.092+0000 7f502ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f503010c510 0x7f503019cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.092+0000 7f502ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f503010c510 0x7f503019cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33448/0 (socket says 172.21.15.43:33448) 2024-04-21T08:44:59.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.092+0000 7f502ffff640 1 -- 172.21.15.43:0/1763114047 learned_addr learned my addr 172.21.15.43:0/1763114047 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:59.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.092+0000 7f502f7fe640 1 --2- 172.21.15.43:0/1763114047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50301031d0 0x7f50301986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.092+0000 7f502ffff640 1 -- 172.21.15.43:0/1763114047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5030103af0 msgr2=0x7f5030198c20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:44:59.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.092+0000 7f502ffff640 1 --2- 172.21.15.43:0/1763114047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5030103af0 0x7f5030198c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.092+0000 7f502ffff640 1 -- 172.21.15.43:0/1763114047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50301031d0 msgr2=0x7f50301986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.092+0000 7f502ffff640 1 --2- 172.21.15.43:0/1763114047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50301031d0 0x7f50301986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.092+0000 7f502ffff640 1 -- 172.21.15.43:0/1763114047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f503019d6d0 con 0x7f503010c510 2024-04-21T08:44:59.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.092+0000 7f502f7fe640 1 --2- 172.21.15.43:0/1763114047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50301031d0 0x7f50301986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:59.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.092+0000 7f502ffff640 1 --2- 172.21.15.43:0/1763114047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f503010c510 0x7f503019cfa0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f5028037040 tx=0x7f5028003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.092+0000 7f502cff9640 1 -- 172.21.15.43:0/1763114047 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028039040 con 0x7f503010c510 2024-04-21T08:44:59.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.092+0000 7f502cff9640 1 -- 172.21.15.43:0/1763114047 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5028044070 con 0x7f503010c510 2024-04-21T08:44:59.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.092+0000 7f5035df4640 1 -- 172.21.15.43:0/1763114047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f503019d960 con 0x7f503010c510 2024-04-21T08:44:59.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.092+0000 7f502cff9640 1 -- 172.21.15.43:0/1763114047 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50280328a0 con 0x7f503010c510 2024-04-21T08:44:59.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.092+0000 7f5035df4640 1 -- 172.21.15.43:0/1763114047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50301a5250 con 0x7f503010c510 2024-04-21T08:44:59.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.094+0000 7f502cff9640 1 -- 172.21.15.43:0/1763114047 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5028032a40 con 0x7f503010c510 2024-04-21T08:44:59.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.095+0000 7f50389e6740 1 Processor -- start 2024-04-21T08:44:59.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.095+0000 7f50389e6740 1 -- start start 2024-04-21T08:44:59.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.095+0000 7f502cff9640 1 --2- 172.21.15.43:0/1763114047 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f50180a3d20 0x7f50180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.095+0000 7f50389e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645be7388e0 0x5645be738cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.095+0000 7f50389e6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5645be739290 0x5645be741840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.095+0000 7f50389e6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be741e10 0x5645be744260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.095+0000 7f50389e6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5645be69cf00 con 0x5645be741e10 2024-04-21T08:44:59.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.095+0000 7f50389e6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5645be63a590 con 0x5645be739290 2024-04-21T08:44:59.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.095+0000 7f50389e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5645be698ec0 con 0x5645be7388e0 2024-04-21T08:44:59.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.095+0000 7f502effd640 1 --2- 172.21.15.43:0/1763114047 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f50180a3d20 0x7f50180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.095+0000 7f502ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be741e10 0x5645be744260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.095+0000 7f502ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be741e10 0x5645be744260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33450/0 (socket says 172.21.15.43:33450) 2024-04-21T08:44:59.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.095+0000 7f502ffff640 1 -- 172.21.15.43:0/753547339 learned_addr learned my addr 172.21.15.43:0/753547339 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:59.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.095+0000 7f502cff9640 1 -- 172.21.15.43:0/1763114047 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f50280f30c0 con 0x7f503010c510 2024-04-21T08:44:59.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.095+0000 7f502f7fe640 1 --2- 172.21.15.43:0/753547339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645be7388e0 0x5645be738cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.096+0000 7f502effd640 1 --2- 172.21.15.43:0/753547339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5645be739290 0x5645be741840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.096+0000 7f502ffff640 1 -- 172.21.15.43:0/753547339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645be7388e0 msgr2=0x5645be738cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.096+0000 7f502ffff640 1 --2- 172.21.15.43:0/753547339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645be7388e0 0x5645be738cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.096+0000 7f502ffff640 1 -- 172.21.15.43:0/753547339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5645be739290 msgr2=0x5645be741840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.096+0000 7f502ffff640 1 --2- 172.21.15.43:0/753547339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5645be739290 0x5645be741840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.096+0000 7f502ffff640 1 -- 172.21.15.43:0/753547339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645be4777e0 con 0x5645be741e10 2024-04-21T08:44:59.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.096+0000 7f502effd640 1 --2- 172.21.15.43:0/1763114047 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f50180a3d20 0x7f50180a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f50240058c0 tx=0x7f50240147e0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.096+0000 7f502ffff640 1 --2- 172.21.15.43:0/753547339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be741e10 0x5645be744260 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f50280beaa0 tx=0x7f502802c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b646c286e602182f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.096+0000 7f5015ffb640 1 -- 172.21.15.43:0/753547339 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50280be700 con 0x5645be741e10 2024-04-21T08:44:59.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.096+0000 7f5015ffb640 1 -- 172.21.15.43:0/753547339 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50280bf4a0 con 0x5645be741e10 2024-04-21T08:44:59.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.096+0000 7f5015ffb640 1 -- 172.21.15.43:0/753547339 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50280f06c0 con 0x5645be741e10 2024-04-21T08:44:59.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.097+0000 7f50389e6740 1 -- 172.21.15.43:0/753547339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be741e10 msgr2=0x5645be744260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.097+0000 7f50389e6740 1 --2- 172.21.15.43:0/753547339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be741e10 0x5645be744260 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f50280beaa0 tx=0x7f502802c230 comp rx=0 tx=0).stop 2024-04-21T08:44:59.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.097+0000 7f50389e6740 1 -- 172.21.15.43:0/753547339 shutdown_connections 2024-04-21T08:44:59.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.097+0000 7f50389e6740 1 --2- 172.21.15.43:0/753547339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be741e10 0x5645be744260 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.097+0000 7f50389e6740 1 --2- 172.21.15.43:0/753547339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5645be739290 0x5645be741840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.097+0000 7f50389e6740 1 --2- 172.21.15.43:0/753547339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645be7388e0 0x5645be738cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.097+0000 7f50389e6740 1 -- 172.21.15.43:0/753547339 >> 172.21.15.43:0/753547339 conn(0x5645be732fb0 msgr2=0x5645be733390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.097+0000 7f50389e6740 1 -- 172.21.15.43:0/753547339 shutdown_connections 2024-04-21T08:44:59.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.097+0000 7f50389e6740 1 -- 172.21.15.43:0/753547339 wait complete. 2024-04-21T08:44:59.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.097+0000 7f50389e6740 1 -- start start 2024-04-21T08:44:59.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.097+0000 7f50389e6740 1 Processor -- start 2024-04-21T08:44:59.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.098+0000 7f50389e6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5645be7388e0 0x5645be73ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.098+0000 7f50389e6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be739290 0x5645be73f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.098+0000 7f50389e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645be741e10 0x5645be73f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.098+0000 7f50389e6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5645be744b40 con 0x5645be739290 2024-04-21T08:44:59.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.098+0000 7f50389e6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5645be656580 con 0x5645be7388e0 2024-04-21T08:44:59.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.098+0000 7f50389e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5645be744cc0 con 0x5645be741e10 2024-04-21T08:44:59.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.098+0000 7f50389e6740 10 client.0 _ll_register_callbacks cb 0x5645be65c650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:59.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.098+0000 7f502effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be739290 0x5645be73f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.098+0000 7f502effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be739290 0x5645be73f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33464/0 (socket says 172.21.15.43:33464) 2024-04-21T08:44:59.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.098+0000 7f502effd640 1 -- 172.21.15.43:0/110457413 learned_addr learned my addr 172.21.15.43:0/110457413 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:59.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.098+0000 7f502f7fe640 1 --2- 172.21.15.43:0/110457413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5645be7388e0 0x5645be73ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.098+0000 7f502ffff640 1 --2- 172.21.15.43:0/110457413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645be741e10 0x5645be73f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.098+0000 7f502effd640 1 -- 172.21.15.43:0/110457413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645be741e10 msgr2=0x5645be73f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.098+0000 7f502effd640 1 --2- 172.21.15.43:0/110457413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645be741e10 0x5645be73f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.098+0000 7f502effd640 1 -- 172.21.15.43:0/110457413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5645be7388e0 msgr2=0x5645be73ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.098+0000 7f502effd640 1 --2- 172.21.15.43:0/110457413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5645be7388e0 0x5645be73ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.098+0000 7f502effd640 1 -- 172.21.15.43:0/110457413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645be4d7f70 con 0x5645be739290 2024-04-21T08:44:59.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.098+0000 7f502ffff640 1 --2- 172.21.15.43:0/110457413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645be741e10 0x5645be73f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:59.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.098+0000 7f502effd640 1 --2- 172.21.15.43:0/110457413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be739290 0x5645be73f380 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f5024005d80 tx=0x7f50240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fe95fcb059f0c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.098+0000 7f500effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:59.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.099+0000 7f500effd640 1 -- 172.21.15.43:0/110457413 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024024030 con 0x5645be739290 2024-04-21T08:44:59.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.099+0000 7f500effd640 1 -- 172.21.15.43:0/110457413 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f5024005020 con 0x5645be739290 2024-04-21T08:44:59.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.099+0000 7f50389e6740 1 -- 172.21.15.43:0/110457413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50280bf4a0 con 0x5645be739290 2024-04-21T08:44:59.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.099+0000 7f500effd640 1 -- 172.21.15.43:0/110457413 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f502402e050 con 0x5645be739290 2024-04-21T08:44:59.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.099+0000 7f50389e6740 10 client.18183 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:59.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.099+0000 7f50389e6740 1 -- 172.21.15.43:0/110457413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5645be36f170 con 0x5645be739290 2024-04-21T08:44:59.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.099+0000 7f500effd640 1 -- 172.21.15.43:0/110457413 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50280bf4a0 con 0x5645be739290 2024-04-21T08:44:59.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.099+0000 7f500effd640 1 -- 172.21.15.43:0/110457413 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f502401d070 con 0x5645be739290 2024-04-21T08:44:59.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.099+0000 7f50389e6740 10 client.18183 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:59.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.099+0000 7f50389e6740 10 client.18183 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:59.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.099+0000 7f50389e6740 20 client.18183 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:59.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.099+0000 7f50389e6740 1 --2- 172.21.15.43:0/110457413 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5645be752120 0x5645be7545a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.099+0000 7f50389e6740 4 client.18183 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:59.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.099+0000 7f50389e6740 1 -- 172.21.15.43:0/110457413 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645be6aa920 con 0x5645be752120 2024-04-21T08:44:59.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.100+0000 7f502f7fe640 1 --2- 172.21.15.43:0/110457413 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5645be752120 0x5645be7545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.100+0000 7f500effd640 10 client.18183 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:59.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.100+0000 7f502f7fe640 1 --2- 172.21.15.43:0/110457413 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5645be752120 0x5645be7545a0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e21909662b1f49 server_cookie=d26ca9746e230182 in_seq=0 out_seq=0 2024-04-21T08:44:59.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.101+0000 7f500effd640 1 -- 172.21.15.43:0/110457413 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5645be6aa920 con 0x5645be752120 2024-04-21T08:44:59.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.101+0000 7f500effd640 10 client.18183 handle_command_reply: tid=0 2024-04-21T08:44:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.101+0000 7f50389e6740 1 client.18183 shutdown 2024-04-21T08:44:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.101+0000 7f50389e6740 1 -- 172.21.15.43:0/110457413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be739290 msgr2=0x5645be73f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.101+0000 7f50389e6740 1 --2- 172.21.15.43:0/110457413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be739290 0x5645be73f380 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f5024005d80 tx=0x7f50240046c0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.101+0000 7f50389e6740 1 -- 172.21.15.43:0/110457413 shutdown_connections 2024-04-21T08:44:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.101+0000 7f50389e6740 1 --2- 172.21.15.43:0/110457413 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5645be752120 0x5645be7545a0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.101+0000 7f50389e6740 1 --2- 172.21.15.43:0/110457413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645be741e10 0x5645be73f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.101+0000 7f50389e6740 1 --2- 172.21.15.43:0/110457413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be739290 0x5645be73f380 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.101+0000 7f500effd640 0 client.18183 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.101+0000 7f50389e6740 1 --2- 172.21.15.43:0/110457413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5645be7388e0 0x5645be73ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.101+0000 7f50389e6740 1 -- 172.21.15.43:0/110457413 >> 172.21.15.43:0/110457413 conn(0x5645be732fb0 msgr2=0x5645be734d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.101+0000 7f50389e6740 1 -- 172.21.15.43:0/110457413 shutdown_connections 2024-04-21T08:44:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.101+0000 7f50389e6740 1 -- 172.21.15.43:0/110457413 wait complete. 2024-04-21T08:44:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.101+0000 7f50389e6740 20 client.18183 trim_cache size 0 max 16384 2024-04-21T08:44:59.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.119+0000 7f50389e6740 1 Processor -- start 2024-04-21T08:44:59.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.119+0000 7f50389e6740 1 -- start start 2024-04-21T08:44:59.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.119+0000 7f50389e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645be7388e0 0x5645be741570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.119+0000 7f50389e6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be739290 0x5645be74d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.119+0000 7f50389e6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5645be741e10 0x5645be759e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.119+0000 7f50389e6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f50301af4a0 con 0x5645be739290 2024-04-21T08:44:59.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.119+0000 7f50389e6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f50301b0fe0 con 0x5645be741e10 2024-04-21T08:44:59.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.119+0000 7f50389e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f50301afb70 con 0x5645be7388e0 2024-04-21T08:44:59.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.119+0000 7f502effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be739290 0x5645be74d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.119+0000 7f502effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be739290 0x5645be74d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33480/0 (socket says 172.21.15.43:33480) 2024-04-21T08:44:59.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.119+0000 7f502effd640 1 -- 172.21.15.43:0/774529163 learned_addr learned my addr 172.21.15.43:0/774529163 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:59.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.119+0000 7f502ffff640 1 --2- 172.21.15.43:0/774529163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5645be741e10 0x5645be759e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.119+0000 7f502effd640 1 -- 172.21.15.43:0/774529163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645be7388e0 msgr2=0x5645be741570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.119+0000 7f502effd640 1 --2- 172.21.15.43:0/774529163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645be7388e0 0x5645be741570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.119+0000 7f502effd640 1 -- 172.21.15.43:0/774529163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5645be741e10 msgr2=0x5645be759e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.119+0000 7f502effd640 1 --2- 172.21.15.43:0/774529163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5645be741e10 0x5645be759e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.119+0000 7f502effd640 1 -- 172.21.15.43:0/774529163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645be739020 con 0x5645be739290 2024-04-21T08:44:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.119+0000 7f502ffff640 1 --2- 172.21.15.43:0/774529163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5645be741e10 0x5645be759e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.120+0000 7f502effd640 1 --2- 172.21.15.43:0/774529163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be739290 0x5645be74d860 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f5024014d00 tx=0x7f5024014380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b660a48dac270350 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.120+0000 7f50157fa640 1 -- 172.21.15.43:0/774529163 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024024020 con 0x5645be739290 2024-04-21T08:44:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.120+0000 7f50157fa640 1 -- 172.21.15.43:0/774529163 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5024004be0 con 0x5645be739290 2024-04-21T08:44:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.120+0000 7f50157fa640 1 -- 172.21.15.43:0/774529163 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50240209d0 con 0x5645be739290 2024-04-21T08:44:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.121+0000 7f50389e6740 1 -- 172.21.15.43:0/774529163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be739290 msgr2=0x5645be74d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.121+0000 7f50389e6740 1 --2- 172.21.15.43:0/774529163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be739290 0x5645be74d860 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f5024014d00 tx=0x7f5024014380 comp rx=0 tx=0).stop 2024-04-21T08:44:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.121+0000 7f50389e6740 1 -- 172.21.15.43:0/774529163 shutdown_connections 2024-04-21T08:44:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.121+0000 7f50389e6740 1 --2- 172.21.15.43:0/774529163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5645be741e10 0x5645be759e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.121+0000 7f50389e6740 1 --2- 172.21.15.43:0/774529163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be739290 0x5645be74d860 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.121+0000 7f50389e6740 1 --2- 172.21.15.43:0/774529163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645be7388e0 0x5645be741570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.121+0000 7f50389e6740 1 -- 172.21.15.43:0/774529163 >> 172.21.15.43:0/774529163 conn(0x5645be732fb0 msgr2=0x5645be73f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.121+0000 7f50389e6740 1 -- 172.21.15.43:0/774529163 shutdown_connections 2024-04-21T08:44:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.121+0000 7f50389e6740 1 -- 172.21.15.43:0/774529163 wait complete. 2024-04-21T08:44:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.121+0000 7f50389e6740 1 -- start start 2024-04-21T08:44:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.121+0000 7f50389e6740 1 Processor -- start 2024-04-21T08:44:59.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.121+0000 7f50389e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645be7388e0 0x5645be749fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.121+0000 7f50389e6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be739290 0x5645be746d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.121+0000 7f50389e6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5645be741e10 0x5645be747280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.121+0000 7f50389e6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f50301b0910 con 0x5645be739290 2024-04-21T08:44:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.121+0000 7f50389e6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f50301b0240 con 0x5645be741e10 2024-04-21T08:44:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.121+0000 7f50389e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f50301ad480 con 0x5645be7388e0 2024-04-21T08:44:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.122+0000 7f50389e6740 10 client.0 _ll_register_callbacks cb 0x7f5024011410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.122+0000 7f502effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be739290 0x5645be746d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.122+0000 7f502effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be739290 0x5645be746d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33492/0 (socket says 172.21.15.43:33492) 2024-04-21T08:44:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.122+0000 7f502effd640 1 -- 172.21.15.43:0/17026087 learned_addr learned my addr 172.21.15.43:0/17026087 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.122+0000 7f502ffff640 1 --2- 172.21.15.43:0/17026087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5645be741e10 0x5645be747280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.122+0000 7f502f7fe640 1 --2- 172.21.15.43:0/17026087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645be7388e0 0x5645be749fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.122+0000 7f502effd640 1 -- 172.21.15.43:0/17026087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645be7388e0 msgr2=0x5645be749fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.122+0000 7f502effd640 1 --2- 172.21.15.43:0/17026087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645be7388e0 0x5645be749fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.122+0000 7f502effd640 1 -- 172.21.15.43:0/17026087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5645be741e10 msgr2=0x5645be747280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.122+0000 7f502effd640 1 --2- 172.21.15.43:0/17026087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5645be741e10 0x5645be747280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.122+0000 7f502effd640 1 -- 172.21.15.43:0/17026087 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645be7478f0 con 0x5645be739290 2024-04-21T08:44:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.122+0000 7f502ffff640 1 --2- 172.21.15.43:0/17026087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5645be741e10 0x5645be747280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.122+0000 7f502effd640 1 --2- 172.21.15.43:0/17026087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be739290 0x5645be746d40 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f5024004ea0 tx=0x7f5024005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ffce4d44855b2c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.122+0000 7f500effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:59.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.122+0000 7f500effd640 1 -- 172.21.15.43:0/17026087 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024024030 con 0x5645be739290 2024-04-21T08:44:59.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.122+0000 7f500effd640 1 -- 172.21.15.43:0/17026087 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f5024005200 con 0x5645be739290 2024-04-21T08:44:59.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.122+0000 7f50389e6740 1 -- 172.21.15.43:0/17026087 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5024004be0 con 0x5645be739290 2024-04-21T08:44:59.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.122+0000 7f500effd640 1 -- 172.21.15.43:0/17026087 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5024041050 con 0x5645be739290 2024-04-21T08:44:59.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.122+0000 7f50389e6740 10 client.18189 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:59.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.122+0000 7f50389e6740 1 -- 172.21.15.43:0/17026087 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5645be7b95c0 con 0x5645be739290 2024-04-21T08:44:59.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.123+0000 7f500effd640 1 -- 172.21.15.43:0/17026087 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5024004be0 con 0x5645be739290 2024-04-21T08:44:59.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.123+0000 7f500effd640 1 -- 172.21.15.43:0/17026087 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f502401d040 con 0x5645be739290 2024-04-21T08:44:59.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.123+0000 7f50389e6740 10 client.18189 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:59.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.123+0000 7f50389e6740 10 client.18189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:59.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.123+0000 7f50389e6740 20 client.18189 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:59.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.123+0000 7f50389e6740 1 --2- 172.21.15.43:0/17026087 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5645be7b9c70 0x5645be7bc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.123+0000 7f50389e6740 4 client.18189 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:44:59.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.123+0000 7f50389e6740 1 -- 172.21.15.43:0/17026087 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5645be6a4790 con 0x5645be7b9c70 2024-04-21T08:44:59.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.123+0000 7f502f7fe640 1 --2- 172.21.15.43:0/17026087 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5645be7b9c70 0x5645be7bc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.124+0000 7f500effd640 10 client.18189 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:59.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.124+0000 7f502f7fe640 1 --2- 172.21.15.43:0/17026087 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5645be7b9c70 0x5645be7bc090 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fb61f055c852a45 server_cookie=74d33c975d12ca5f in_seq=0 out_seq=0 2024-04-21T08:44:59.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.125+0000 7f500effd640 1 -- 172.21.15.43:0/17026087 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5645be6a4790 con 0x5645be7b9c70 2024-04-21T08:44:59.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.125+0000 7f500effd640 10 client.18189 handle_command_reply: tid=0 2024-04-21T08:44:59.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.125+0000 7f50389e6740 1 client.18189 shutdown 2024-04-21T08:44:59.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.125+0000 7f50389e6740 1 -- 172.21.15.43:0/17026087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be739290 msgr2=0x5645be746d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.125+0000 7f50389e6740 1 --2- 172.21.15.43:0/17026087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be739290 0x5645be746d40 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f5024004ea0 tx=0x7f5024005780 comp rx=0 tx=0).stop 2024-04-21T08:44:59.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.125+0000 7f50389e6740 1 -- 172.21.15.43:0/17026087 shutdown_connections 2024-04-21T08:44:59.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.125+0000 7f50389e6740 1 --2- 172.21.15.43:0/17026087 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5645be7b9c70 0x5645be7bc090 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.125+0000 7f50389e6740 1 --2- 172.21.15.43:0/17026087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5645be741e10 0x5645be747280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.125+0000 7f50389e6740 1 --2- 172.21.15.43:0/17026087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5645be739290 0x5645be746d40 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.125+0000 7f50389e6740 1 --2- 172.21.15.43:0/17026087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5645be7388e0 0x5645be749fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.125+0000 7f50389e6740 1 -- 172.21.15.43:0/17026087 >> 172.21.15.43:0/17026087 conn(0x5645be732fb0 msgr2=0x5645be731780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.125+0000 7f500effd640 0 client.18189 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:59.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.125+0000 7f50389e6740 1 -- 172.21.15.43:0/17026087 shutdown_connections 2024-04-21T08:44:59.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.125+0000 7f50389e6740 1 -- 172.21.15.43:0/17026087 wait complete. 2024-04-21T08:44:59.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.125+0000 7f50389e6740 20 client.18189 trim_cache size 0 max 16384 2024-04-21T08:44:59.131 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:44:59.131 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:44:59.131 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "e001a008-f4e9-48fc-bb49-ce05b5851deb", 2024-04-21T08:44:59.131 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:44:59.131 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:44:59.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.126+0000 7f5035df4640 1 -- 172.21.15.43:0/1763114047 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f50180a3d20 msgr2=0x7f50180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.126+0000 7f5035df4640 1 --2- 172.21.15.43:0/1763114047 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f50180a3d20 0x7f50180a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f50240058c0 tx=0x7f50240147e0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.126+0000 7f5035df4640 1 -- 172.21.15.43:0/1763114047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f503010c510 msgr2=0x7f503019cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.126+0000 7f5035df4640 1 --2- 172.21.15.43:0/1763114047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f503010c510 0x7f503019cfa0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f5028037040 tx=0x7f5028003ba0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.126+0000 7f5035df4640 1 -- 172.21.15.43:0/1763114047 shutdown_connections 2024-04-21T08:44:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.126+0000 7f5035df4640 1 --2- 172.21.15.43:0/1763114047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f503010c510 0x7f503019cfa0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.126+0000 7f5035df4640 1 --2- 172.21.15.43:0/1763114047 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f50180a3d20 0x7f50180a61e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.126+0000 7f5035df4640 1 --2- 172.21.15.43:0/1763114047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5030103af0 0x7f5030198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.126+0000 7f5035df4640 1 --2- 172.21.15.43:0/1763114047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50301031d0 0x7f50301986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.126+0000 7f5035df4640 1 -- 172.21.15.43:0/1763114047 >> 172.21.15.43:0/1763114047 conn(0x7f5030077f50 msgr2=0x7f50300fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.126+0000 7f5035df4640 1 -- 172.21.15.43:0/1763114047 shutdown_connections 2024-04-21T08:44:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.126+0000 7f5035df4640 1 -- 172.21.15.43:0/1763114047 wait complete. 2024-04-21T08:44:59.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:44:59.138 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:44:59.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.242+0000 7f38b0f96640 1 -- 172.21.15.43:0/2757132819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ac06a5b0 msgr2=0x7f38ac06aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.242+0000 7f38b0f96640 1 --2- 172.21.15.43:0/2757132819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ac06a5b0 0x7f38ac06aa30 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f3894009900 tx=0x7f389402bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:59.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.242+0000 7f38b0f96640 1 -- 172.21.15.43:0/2757132819 shutdown_connections 2024-04-21T08:44:59.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.242+0000 7f38b0f96640 1 --2- 172.21.15.43:0/2757132819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ac06af70 0x7f38ac10edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.242+0000 7f38b0f96640 1 --2- 172.21.15.43:0/2757132819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ac06a5b0 0x7f38ac06aa30 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.242+0000 7f38b0f96640 1 --2- 172.21.15.43:0/2757132819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38ac106f20 0x7f38ac107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.242+0000 7f38b0f96640 1 -- 172.21.15.43:0/2757132819 >> 172.21.15.43:0/2757132819 conn(0x7f38ac077c40 msgr2=0x7f38ac078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.242+0000 7f38b0f96640 1 -- 172.21.15.43:0/2757132819 shutdown_connections 2024-04-21T08:44:59.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.242+0000 7f38b0f96640 1 -- 172.21.15.43:0/2757132819 wait complete. 2024-04-21T08:44:59.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.243+0000 7f38b0f96640 1 Processor -- start 2024-04-21T08:44:59.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.243+0000 7f38b0f96640 1 -- start start 2024-04-21T08:44:59.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.243+0000 7f38b0f96640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38ac06a5b0 0x7f38ac198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.243+0000 7f38b0f96640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ac06af70 0x7f38ac198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.243+0000 7f38b0f96640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ac106f20 0x7f38ac19cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.243+0000 7f38b0f96640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac112190 con 0x7f38ac06af70 2024-04-21T08:44:59.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.243+0000 7f38b0f96640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac112010 con 0x7f38ac106f20 2024-04-21T08:44:59.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.243+0000 7f38b0f96640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac112310 con 0x7f38ac06a5b0 2024-04-21T08:44:59.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.243+0000 7f38a3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ac06af70 0x7f38ac198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.244+0000 7f38a3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ac06af70 0x7f38ac198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33514/0 (socket says 172.21.15.43:33514) 2024-04-21T08:44:59.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.244+0000 7f38a3fff640 1 -- 172.21.15.43:0/3748986716 learned_addr learned my addr 172.21.15.43:0/3748986716 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:59.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.244+0000 7f38aa59c640 1 --2- 172.21.15.43:0/3748986716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38ac06a5b0 0x7f38ac198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.244+0000 7f38aad9d640 1 --2- 172.21.15.43:0/3748986716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ac106f20 0x7f38ac19cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.244+0000 7f38a3fff640 1 -- 172.21.15.43:0/3748986716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38ac06a5b0 msgr2=0x7f38ac198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.244+0000 7f38a3fff640 1 --2- 172.21.15.43:0/3748986716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38ac06a5b0 0x7f38ac198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.244+0000 7f38a3fff640 1 -- 172.21.15.43:0/3748986716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ac106f20 msgr2=0x7f38ac19cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.244+0000 7f38a3fff640 1 --2- 172.21.15.43:0/3748986716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ac106f20 0x7f38ac19cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.244+0000 7f38a3fff640 1 -- 172.21.15.43:0/3748986716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ac19d620 con 0x7f38ac06af70 2024-04-21T08:44:59.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.244+0000 7f38a3fff640 1 --2- 172.21.15.43:0/3748986716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ac06af70 0x7f38ac198b70 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f3894006010 tx=0x7f3894004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.244+0000 7f38a37fe640 1 -- 172.21.15.43:0/3748986716 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3894043020 con 0x7f38ac06af70 2024-04-21T08:44:59.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.244+0000 7f38a37fe640 1 -- 172.21.15.43:0/3748986716 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3894004440 con 0x7f38ac06af70 2024-04-21T08:44:59.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.244+0000 7f38b0f96640 1 -- 172.21.15.43:0/3748986716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ac19d8b0 con 0x7f38ac06af70 2024-04-21T08:44:59.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.244+0000 7f38a37fe640 1 -- 172.21.15.43:0/3748986716 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3894003b50 con 0x7f38ac06af70 2024-04-21T08:44:59.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.244+0000 7f38b0f96640 1 -- 172.21.15.43:0/3748986716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ac1a51a0 con 0x7f38ac06af70 2024-04-21T08:44:59.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.246+0000 7f38a37fe640 1 -- 172.21.15.43:0/3748986716 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3894046020 con 0x7f38ac06af70 2024-04-21T08:44:59.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.247+0000 7f38b3b85740 1 Processor -- start 2024-04-21T08:44:59.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.247+0000 7f38b3b85740 1 -- start start 2024-04-21T08:44:59.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.247+0000 7f38b3b85740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb46f2b0 0x55dadb46f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.247+0000 7f38b3b85740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dadb46fbd0 0x55dadb47c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.247+0000 7f38b3b85740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dadb47ce90 0x55dadb47f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.247+0000 7f38b3b85740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dadb3d41e0 con 0x55dadb46f2b0 2024-04-21T08:44:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.247+0000 7f38b3b85740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dadb371390 con 0x55dadb46fbd0 2024-04-21T08:44:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.247+0000 7f38b3b85740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dadb3d01a0 con 0x55dadb47ce90 2024-04-21T08:44:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.247+0000 7f38a37fe640 1 --2- 172.21.15.43:0/3748986716 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f38980a3d20 0x7f38980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.247+0000 7f38aad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb46f2b0 0x55dadb46f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.248+0000 7f38aad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb46f2b0 0x55dadb46f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33526/0 (socket says 172.21.15.43:33526) 2024-04-21T08:44:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.248+0000 7f38aad9d640 1 -- 172.21.15.43:0/306357535 learned_addr learned my addr 172.21.15.43:0/306357535 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.248+0000 7f38aad9d640 1 --2- 172.21.15.43:0/3748986716 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f38980a3d20 0x7f38980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.248+0000 7f38aa59c640 1 --2- 172.21.15.43:0/306357535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dadb46fbd0 0x55dadb47c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.248+0000 7f38a37fe640 1 -- 172.21.15.43:0/3748986716 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f38940f2700 con 0x7f38ac06af70 2024-04-21T08:44:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.248+0000 7f38a3fff640 1 --2- 172.21.15.43:0/306357535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dadb47ce90 0x55dadb47f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.248+0000 7f38aad9d640 1 -- 172.21.15.43:0/306357535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dadb47ce90 msgr2=0x55dadb47f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.248+0000 7f38aad9d640 1 --2- 172.21.15.43:0/306357535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dadb47ce90 0x55dadb47f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.248+0000 7f38aad9d640 1 -- 172.21.15.43:0/306357535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dadb46fbd0 msgr2=0x55dadb47c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.248+0000 7f38aad9d640 1 --2- 172.21.15.43:0/306357535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dadb46fbd0 0x55dadb47c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.248+0000 7f38aad9d640 1 -- 172.21.15.43:0/306357535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dadb1b07d0 con 0x55dadb46f2b0 2024-04-21T08:44:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.248+0000 7f38aad9d640 1 --2- 172.21.15.43:0/306357535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb46f2b0 0x55dadb46f690 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f38a4013e60 tx=0x7f38a40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e14724dc88f250 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.248+0000 7f38aad9d640 1 --2- 172.21.15.43:0/3748986716 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f38980a3d20 0x7f38980a61e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f38a4010ca0 tx=0x7f38a4015000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.248+0000 7f38a0ff9640 1 -- 172.21.15.43:0/306357535 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a400ae10 con 0x55dadb46f2b0 2024-04-21T08:44:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.248+0000 7f38a0ff9640 1 -- 172.21.15.43:0/306357535 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a4007be0 con 0x55dadb46f2b0 2024-04-21T08:44:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.248+0000 7f38a0ff9640 1 -- 172.21.15.43:0/306357535 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4005930 con 0x55dadb46f2b0 2024-04-21T08:44:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.249+0000 7f38b3b85740 1 -- 172.21.15.43:0/306357535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb46f2b0 msgr2=0x55dadb46f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.249+0000 7f38b3b85740 1 --2- 172.21.15.43:0/306357535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb46f2b0 0x55dadb46f690 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f38a4013e60 tx=0x7f38a40107a0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.249+0000 7f38b3b85740 1 -- 172.21.15.43:0/306357535 shutdown_connections 2024-04-21T08:44:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.249+0000 7f38b3b85740 1 --2- 172.21.15.43:0/306357535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dadb47ce90 0x55dadb47f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.249+0000 7f38b3b85740 1 --2- 172.21.15.43:0/306357535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dadb46fbd0 0x55dadb47c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.249+0000 7f38b3b85740 1 --2- 172.21.15.43:0/306357535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb46f2b0 0x55dadb46f690 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.249+0000 7f38b3b85740 1 -- 172.21.15.43:0/306357535 >> 172.21.15.43:0/306357535 conn(0x55dadb469cb0 msgr2=0x55dadb46a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.249+0000 7f38b3b85740 1 -- 172.21.15.43:0/306357535 shutdown_connections 2024-04-21T08:44:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.249+0000 7f38b3b85740 1 -- 172.21.15.43:0/306357535 wait complete. 2024-04-21T08:44:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.249+0000 7f38b3b85740 1 -- start start 2024-04-21T08:44:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.250+0000 7f38b3b85740 1 Processor -- start 2024-04-21T08:44:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.250+0000 7f38b3b85740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dadb46f2b0 0x55dadb479e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.250+0000 7f38b3b85740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb46fbd0 0x55dadb47a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.250+0000 7f38b3b85740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dadb47ce90 0x55dadb47a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.250+0000 7f38b3b85740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dadb47fbf0 con 0x55dadb46fbd0 2024-04-21T08:44:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.250+0000 7f38b3b85740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dadb38d2f0 con 0x55dadb46f2b0 2024-04-21T08:44:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.250+0000 7f38b3b85740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dadb47fd70 con 0x55dadb47ce90 2024-04-21T08:44:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.250+0000 7f38b3b85740 10 client.0 _ll_register_callbacks cb 0x55dadb39a580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.250+0000 7f38a3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb46fbd0 0x55dadb47a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.250+0000 7f38a3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb46fbd0 0x55dadb47a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33540/0 (socket says 172.21.15.43:33540) 2024-04-21T08:44:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.250+0000 7f38aa59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dadb46f2b0 0x55dadb479e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.250+0000 7f38a3fff640 1 -- 172.21.15.43:0/397168186 learned_addr learned my addr 172.21.15.43:0/397168186 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.250+0000 7f38a3fff640 1 -- 172.21.15.43:0/397168186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dadb47ce90 msgr2=0x55dadb47a910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:44:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.250+0000 7f38a3fff640 1 --2- 172.21.15.43:0/397168186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dadb47ce90 0x55dadb47a910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.250+0000 7f38a3fff640 1 -- 172.21.15.43:0/397168186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dadb46f2b0 msgr2=0x55dadb479e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.250+0000 7f38a3fff640 1 --2- 172.21.15.43:0/397168186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dadb46f2b0 0x55dadb479e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.250+0000 7f38a3fff640 1 -- 172.21.15.43:0/397168186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dadb0e63f0 con 0x55dadb46fbd0 2024-04-21T08:44:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.250+0000 7f38a3fff640 1 --2- 172.21.15.43:0/397168186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb46fbd0 0x55dadb47a3d0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f3894000c00 tx=0x7f38940bdf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ba425f19d01e19 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.250+0000 7f3891ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.250+0000 7f3891ffb640 1 -- 172.21.15.43:0/397168186 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38940be3e0 con 0x55dadb46fbd0 2024-04-21T08:44:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.250+0000 7f3891ffb640 1 -- 172.21.15.43:0/397168186 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f38940efd00 con 0x55dadb46fbd0 2024-04-21T08:44:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.250+0000 7f38b3b85740 1 -- 172.21.15.43:0/397168186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38a4007be0 con 0x55dadb46fbd0 2024-04-21T08:44:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.251+0000 7f3891ffb640 1 -- 172.21.15.43:0/397168186 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f38940c2790 con 0x55dadb46fbd0 2024-04-21T08:44:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.251+0000 7f38b3b85740 10 client.18201 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.251+0000 7f38b3b85740 1 -- 172.21.15.43:0/397168186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dadb0a60f0 con 0x55dadb46fbd0 2024-04-21T08:44:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.251+0000 7f3891ffb640 1 -- 172.21.15.43:0/397168186 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a4007be0 con 0x55dadb46fbd0 2024-04-21T08:44:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.251+0000 7f3891ffb640 1 -- 172.21.15.43:0/397168186 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f38940dd1e0 con 0x55dadb46fbd0 2024-04-21T08:44:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.251+0000 7f38b3b85740 10 client.18201 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.251+0000 7f38b3b85740 10 client.18201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.251+0000 7f38b3b85740 20 client.18201 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.251+0000 7f38b3b85740 1 --2- 172.21.15.43:0/397168186 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55dadb48ccd0 0x55dadb48f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.251+0000 7f38b3b85740 4 client.18201 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.251+0000 7f38b3b85740 1 -- 172.21.15.43:0/397168186 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dadb3e1c00 con 0x55dadb48ccd0 2024-04-21T08:44:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.251+0000 7f38aa59c640 1 --2- 172.21.15.43:0/397168186 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55dadb48ccd0 0x55dadb48f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.252+0000 7f38aa59c640 1 --2- 172.21.15.43:0/397168186 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55dadb48ccd0 0x55dadb48f0c0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8183def06b3591c4 server_cookie=de21e7b98f649e1c in_seq=0 out_seq=0 2024-04-21T08:44:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.252+0000 7f3891ffb640 10 client.18201 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.252+0000 7f3891ffb640 1 -- 172.21.15.43:0/397168186 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dadb3e1c00 con 0x55dadb48ccd0 2024-04-21T08:44:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.252+0000 7f3891ffb640 10 client.18201 handle_command_reply: tid=0 2024-04-21T08:44:59.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.252+0000 7f38b3b85740 1 client.18201 shutdown 2024-04-21T08:44:59.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.252+0000 7f38b3b85740 1 -- 172.21.15.43:0/397168186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb46fbd0 msgr2=0x55dadb47a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.253+0000 7f38b3b85740 1 --2- 172.21.15.43:0/397168186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb46fbd0 0x55dadb47a3d0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f3894000c00 tx=0x7f38940bdf20 comp rx=0 tx=0).stop 2024-04-21T08:44:59.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.253+0000 7f38b3b85740 1 -- 172.21.15.43:0/397168186 shutdown_connections 2024-04-21T08:44:59.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.253+0000 7f38b3b85740 1 --2- 172.21.15.43:0/397168186 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55dadb48ccd0 0x55dadb48f0c0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.253+0000 7f38b3b85740 1 --2- 172.21.15.43:0/397168186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dadb47ce90 0x55dadb47a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.253+0000 7f38b3b85740 1 --2- 172.21.15.43:0/397168186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb46fbd0 0x55dadb47a3d0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.253+0000 7f38b3b85740 1 --2- 172.21.15.43:0/397168186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dadb46f2b0 0x55dadb479e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.253+0000 7f3891ffb640 0 client.18201 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:59.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.253+0000 7f38b3b85740 1 -- 172.21.15.43:0/397168186 >> 172.21.15.43:0/397168186 conn(0x55dadb469cb0 msgr2=0x55dadb46ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.253+0000 7f38b3b85740 1 -- 172.21.15.43:0/397168186 shutdown_connections 2024-04-21T08:44:59.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.253+0000 7f38b3b85740 1 -- 172.21.15.43:0/397168186 wait complete. 2024-04-21T08:44:59.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.253+0000 7f38b3b85740 20 client.18201 trim_cache size 0 max 16384 2024-04-21T08:44:59.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.270+0000 7f38b3b85740 1 Processor -- start 2024-04-21T08:44:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.270+0000 7f38b3b85740 1 -- start start 2024-04-21T08:44:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.270+0000 7f38b3b85740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dadb46f2b0 0x55dadb47c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.270+0000 7f38b3b85740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dadb46fbd0 0x55dadb478dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.270+0000 7f38b3b85740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb47ce90 0x55dadb488af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.270+0000 7f38b3b85740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac1ad120 con 0x55dadb47ce90 2024-04-21T08:44:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.270+0000 7f38b3b85740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac1aec60 con 0x55dadb46fbd0 2024-04-21T08:44:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.270+0000 7f38b3b85740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac1ad7f0 con 0x55dadb46f2b0 2024-04-21T08:44:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.270+0000 7f38aad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb47ce90 0x55dadb488af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.270+0000 7f38aad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb47ce90 0x55dadb488af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33554/0 (socket says 172.21.15.43:33554) 2024-04-21T08:44:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.270+0000 7f38aad9d640 1 -- 172.21.15.43:0/3391706473 learned_addr learned my addr 172.21.15.43:0/3391706473 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.271+0000 7f38aad9d640 1 -- 172.21.15.43:0/3391706473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dadb46f2b0 msgr2=0x55dadb47c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.271+0000 7f38aad9d640 1 --2- 172.21.15.43:0/3391706473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dadb46f2b0 0x55dadb47c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.271+0000 7f38aad9d640 1 -- 172.21.15.43:0/3391706473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dadb46fbd0 msgr2=0x55dadb478dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.271+0000 7f38aad9d640 1 --2- 172.21.15.43:0/3391706473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dadb46fbd0 0x55dadb478dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.271+0000 7f38aad9d640 1 -- 172.21.15.43:0/3391706473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dadb46e670 con 0x55dadb47ce90 2024-04-21T08:44:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.271+0000 7f38aad9d640 1 --2- 172.21.15.43:0/3391706473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb47ce90 0x55dadb488af0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f38a4013d10 tx=0x7f38a4013d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d2563dc4ef647b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.271+0000 7f3893fff640 1 -- 172.21.15.43:0/3391706473 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a402b070 con 0x55dadb47ce90 2024-04-21T08:44:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.271+0000 7f3893fff640 1 -- 172.21.15.43:0/3391706473 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a4007550 con 0x55dadb47ce90 2024-04-21T08:44:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.271+0000 7f3893fff640 1 -- 172.21.15.43:0/3391706473 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4005320 con 0x55dadb47ce90 2024-04-21T08:44:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.272+0000 7f38b3b85740 1 -- 172.21.15.43:0/3391706473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb47ce90 msgr2=0x55dadb488af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.272+0000 7f38b3b85740 1 --2- 172.21.15.43:0/3391706473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb47ce90 0x55dadb488af0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f38a4013d10 tx=0x7f38a4013d40 comp rx=0 tx=0).stop 2024-04-21T08:44:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.272+0000 7f38b3b85740 1 -- 172.21.15.43:0/3391706473 shutdown_connections 2024-04-21T08:44:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.272+0000 7f38b3b85740 1 --2- 172.21.15.43:0/3391706473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb47ce90 0x55dadb488af0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.272+0000 7f38b3b85740 1 --2- 172.21.15.43:0/3391706473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dadb46fbd0 0x55dadb478dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.272+0000 7f38b3b85740 1 --2- 172.21.15.43:0/3391706473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dadb46f2b0 0x55dadb47c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.272+0000 7f38b3b85740 1 -- 172.21.15.43:0/3391706473 >> 172.21.15.43:0/3391706473 conn(0x55dadb469cb0 msgr2=0x55dadb468d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.272+0000 7f38b3b85740 1 -- 172.21.15.43:0/3391706473 shutdown_connections 2024-04-21T08:44:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.272+0000 7f38b3b85740 1 -- 172.21.15.43:0/3391706473 wait complete. 2024-04-21T08:44:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.272+0000 7f38b3b85740 1 -- start start 2024-04-21T08:44:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.272+0000 7f38b3b85740 1 Processor -- start 2024-04-21T08:44:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.272+0000 7f38b3b85740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb46f2b0 0x55dadb480dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.272+0000 7f38b3b85740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dadb46fbd0 0x55dadb481310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.272+0000 7f38b3b85740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dadb47ce90 0x55dadb4f2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.272+0000 7f38b3b85740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac1ae590 con 0x55dadb46f2b0 2024-04-21T08:44:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.272+0000 7f38b3b85740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac1adec0 con 0x55dadb47ce90 2024-04-21T08:44:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.272+0000 7f38b3b85740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac1ab100 con 0x55dadb46fbd0 2024-04-21T08:44:59.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.272+0000 7f38b3b85740 10 client.0 _ll_register_callbacks cb 0x7f38940b8f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:59.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.272+0000 7f38aa59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb46f2b0 0x55dadb480dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.272+0000 7f38aa59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb46f2b0 0x55dadb480dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33564/0 (socket says 172.21.15.43:33564) 2024-04-21T08:44:59.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.272+0000 7f38aa59c640 1 -- 172.21.15.43:0/3744503361 learned_addr learned my addr 172.21.15.43:0/3744503361 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:59.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.272+0000 7f38aad9d640 1 --2- 172.21.15.43:0/3744503361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dadb47ce90 0x55dadb4f2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.272+0000 7f38a3fff640 1 --2- 172.21.15.43:0/3744503361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dadb46fbd0 0x55dadb481310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.273+0000 7f38aa59c640 1 -- 172.21.15.43:0/3744503361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dadb46fbd0 msgr2=0x55dadb481310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.273+0000 7f38aa59c640 1 --2- 172.21.15.43:0/3744503361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dadb46fbd0 0x55dadb481310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.273+0000 7f38aa59c640 1 -- 172.21.15.43:0/3744503361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dadb47ce90 msgr2=0x55dadb4f2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.273+0000 7f38aa59c640 1 --2- 172.21.15.43:0/3744503361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dadb47ce90 0x55dadb4f2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.273+0000 7f38aa59c640 1 -- 172.21.15.43:0/3744503361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dadb481be0 con 0x55dadb46f2b0 2024-04-21T08:44:59.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.273+0000 7f38a3fff640 1 --2- 172.21.15.43:0/3744503361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dadb46fbd0 0x55dadb481310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:59.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.273+0000 7f38aa59c640 1 --2- 172.21.15.43:0/3744503361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb46f2b0 0x55dadb480dd0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x55dadb48f900 tx=0x7f389c00bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b26bd04ac0eab4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.273+0000 7f3891ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:59.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.273+0000 7f3891ffb640 1 -- 172.21.15.43:0/3744503361 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c004440 con 0x55dadb46f2b0 2024-04-21T08:44:59.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.273+0000 7f3891ffb640 1 -- 172.21.15.43:0/3744503361 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f389c01c3b0 con 0x55dadb46f2b0 2024-04-21T08:44:59.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.273+0000 7f38b3b85740 1 -- 172.21.15.43:0/3744503361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38a4007550 con 0x55dadb46f2b0 2024-04-21T08:44:59.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.273+0000 7f3891ffb640 1 -- 172.21.15.43:0/3744503361 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f389c02a050 con 0x55dadb46f2b0 2024-04-21T08:44:59.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.273+0000 7f38b3b85740 10 client.18207 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:59.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.273+0000 7f38b3b85740 1 -- 172.21.15.43:0/3744503361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dadb4f38b0 con 0x55dadb46f2b0 2024-04-21T08:44:59.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.273+0000 7f3891ffb640 1 -- 172.21.15.43:0/3744503361 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a4007550 con 0x55dadb46f2b0 2024-04-21T08:44:59.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.274+0000 7f3891ffb640 1 -- 172.21.15.43:0/3744503361 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f389c0103d0 con 0x55dadb46f2b0 2024-04-21T08:44:59.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.274+0000 7f38b3b85740 10 client.18207 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:59.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.274+0000 7f38b3b85740 10 client.18207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:59.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.274+0000 7f38b3b85740 20 client.18207 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:59.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.274+0000 7f38b3b85740 1 --2- 172.21.15.43:0/3744503361 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55dadb4f4030 0x55dadb4f6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.274+0000 7f38b3b85740 4 client.18207 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:44:59.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.274+0000 7f38b3b85740 1 -- 172.21.15.43:0/3744503361 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dadb3dba70 con 0x55dadb4f4030 2024-04-21T08:44:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.274+0000 7f38a3fff640 1 --2- 172.21.15.43:0/3744503361 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55dadb4f4030 0x55dadb4f6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.274+0000 7f3891ffb640 10 client.18207 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.274+0000 7f38a3fff640 1 --2- 172.21.15.43:0/3744503361 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55dadb4f4030 0x55dadb4f6450 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec8a597b2bcb9352 server_cookie=43d2f3248d674238 in_seq=0 out_seq=0 2024-04-21T08:44:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.274+0000 7f3891ffb640 1 -- 172.21.15.43:0/3744503361 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dadb3dba70 con 0x55dadb4f4030 2024-04-21T08:44:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.274+0000 7f3891ffb640 10 client.18207 handle_command_reply: tid=0 2024-04-21T08:44:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.274+0000 7f38b3b85740 1 client.18207 shutdown 2024-04-21T08:44:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.275+0000 7f38b3b85740 1 -- 172.21.15.43:0/3744503361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb46f2b0 msgr2=0x55dadb480dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.275+0000 7f38b3b85740 1 --2- 172.21.15.43:0/3744503361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb46f2b0 0x55dadb480dd0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x55dadb48f900 tx=0x7f389c00bbd0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.275+0000 7f38b3b85740 1 -- 172.21.15.43:0/3744503361 shutdown_connections 2024-04-21T08:44:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.275+0000 7f38b3b85740 1 --2- 172.21.15.43:0/3744503361 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55dadb4f4030 0x55dadb4f6450 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.275+0000 7f38b3b85740 1 --2- 172.21.15.43:0/3744503361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dadb47ce90 0x55dadb4f2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.275+0000 7f38b3b85740 1 --2- 172.21.15.43:0/3744503361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dadb46fbd0 0x55dadb481310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.275+0000 7f38b3b85740 1 --2- 172.21.15.43:0/3744503361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dadb46f2b0 0x55dadb480dd0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.275+0000 7f3891ffb640 0 client.18207 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:44:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.275+0000 7f38b3b85740 1 -- 172.21.15.43:0/3744503361 >> 172.21.15.43:0/3744503361 conn(0x55dadb469cb0 msgr2=0x55dadb479e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.275+0000 7f38b3b85740 1 -- 172.21.15.43:0/3744503361 shutdown_connections 2024-04-21T08:44:59.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.275+0000 7f38b3b85740 1 -- 172.21.15.43:0/3744503361 wait complete. 2024-04-21T08:44:59.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.275+0000 7f38b3b85740 20 client.18207 trim_cache size 0 max 16384 2024-04-21T08:44:59.282 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:44:59.282 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:44:59.282 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:44:59.282 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:44:59.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.275+0000 7f38b0f96640 1 -- 172.21.15.43:0/3748986716 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f38980a3d20 msgr2=0x7f38980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.275+0000 7f38b0f96640 1 --2- 172.21.15.43:0/3748986716 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f38980a3d20 0x7f38980a61e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f38a4010ca0 tx=0x7f38a4015000 comp rx=0 tx=0).stop 2024-04-21T08:44:59.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.275+0000 7f38b0f96640 1 -- 172.21.15.43:0/3748986716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ac06af70 msgr2=0x7f38ac198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.275+0000 7f38b0f96640 1 --2- 172.21.15.43:0/3748986716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ac06af70 0x7f38ac198b70 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f3894006010 tx=0x7f3894004040 comp rx=0 tx=0).stop 2024-04-21T08:44:59.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.275+0000 7f38b0f96640 1 -- 172.21.15.43:0/3748986716 shutdown_connections 2024-04-21T08:44:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.275+0000 7f38b0f96640 1 --2- 172.21.15.43:0/3748986716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38ac106f20 0x7f38ac19cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.275+0000 7f38b0f96640 1 --2- 172.21.15.43:0/3748986716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ac06af70 0x7f38ac198b70 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.275+0000 7f38b0f96640 1 --2- 172.21.15.43:0/3748986716 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f38980a3d20 0x7f38980a61e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.275+0000 7f38b0f96640 1 --2- 172.21.15.43:0/3748986716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38ac06a5b0 0x7f38ac198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.275+0000 7f38b0f96640 1 -- 172.21.15.43:0/3748986716 >> 172.21.15.43:0/3748986716 conn(0x7f38ac077c40 msgr2=0x7f38ac0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.275+0000 7f38b0f96640 1 -- 172.21.15.43:0/3748986716 shutdown_connections 2024-04-21T08:44:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.275+0000 7f38b0f96640 1 -- 172.21.15.43:0/3748986716 wait complete. 2024-04-21T08:44:59.287 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:44:59.287 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:44:59.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.389+0000 7ff47abb6640 1 -- 172.21.15.43:0/1510404931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff474101a60 msgr2=0x7ff474105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.389+0000 7ff47abb6640 1 --2- 172.21.15.43:0/1510404931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff474101a60 0x7ff474105aa0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ff460009900 tx=0x7ff46002bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:59.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.389+0000 7ff47abb6640 1 -- 172.21.15.43:0/1510404931 shutdown_connections 2024-04-21T08:44:59.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.389+0000 7ff47abb6640 1 --2- 172.21.15.43:0/1510404931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff474106180 0x7ff47410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.389+0000 7ff47abb6640 1 --2- 172.21.15.43:0/1510404931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff474101a60 0x7ff474105aa0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.389+0000 7ff47abb6640 1 --2- 172.21.15.43:0/1510404931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4741010b0 0x7ff474101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.389+0000 7ff47abb6640 1 -- 172.21.15.43:0/1510404931 >> 172.21.15.43:0/1510404931 conn(0x7ff4740771d0 msgr2=0x7ff4740775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.389+0000 7ff47abb6640 1 -- 172.21.15.43:0/1510404931 shutdown_connections 2024-04-21T08:44:59.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.389+0000 7ff47abb6640 1 -- 172.21.15.43:0/1510404931 wait complete. 2024-04-21T08:44:59.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.390+0000 7ff47abb6640 1 Processor -- start 2024-04-21T08:44:59.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.390+0000 7ff47abb6640 1 -- start start 2024-04-21T08:44:59.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.390+0000 7ff47abb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4741010b0 0x7ff474199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.390+0000 7ff47abb6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff474106180 0x7ff474199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.390+0000 7ff47abb6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff47419a050 0x7ff4741a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.390+0000 7ff47abb6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff4741110c0 con 0x7ff47419a050 2024-04-21T08:44:59.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.390+0000 7ff47abb6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff474110f40 con 0x7ff4741010b0 2024-04-21T08:44:59.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.390+0000 7ff47abb6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff474111240 con 0x7ff474106180 2024-04-21T08:44:59.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.390+0000 7ff479153640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff47419a050 0x7ff4741a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.390+0000 7ff478952640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4741010b0 0x7ff474199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.390+0000 7ff478952640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4741010b0 0x7ff474199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:51360/0 (socket says 172.21.15.43:51360) 2024-04-21T08:44:59.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.390+0000 7ff478952640 1 -- 172.21.15.43:0/4294086830 learned_addr learned my addr 172.21.15.43:0/4294086830 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:59.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.390+0000 7ff479153640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff47419a050 0x7ff4741a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33582/0 (socket says 172.21.15.43:33582) 2024-04-21T08:44:59.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.390+0000 7ff473fff640 1 --2- 172.21.15.43:0/4294086830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff474106180 0x7ff474199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.390+0000 7ff479153640 1 -- 172.21.15.43:0/4294086830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff474106180 msgr2=0x7ff474199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.390+0000 7ff479153640 1 --2- 172.21.15.43:0/4294086830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff474106180 0x7ff474199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.390+0000 7ff479153640 1 -- 172.21.15.43:0/4294086830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4741010b0 msgr2=0x7ff474199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.390+0000 7ff479153640 1 --2- 172.21.15.43:0/4294086830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4741010b0 0x7ff474199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.390+0000 7ff479153640 1 -- 172.21.15.43:0/4294086830 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4741a4dd0 con 0x7ff47419a050 2024-04-21T08:44:59.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.390+0000 7ff478952640 1 --2- 172.21.15.43:0/4294086830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4741010b0 0x7ff474199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:44:59.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.391+0000 7ff479153640 1 --2- 172.21.15.43:0/4294086830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff47419a050 0x7ff4741a4600 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00e940 tx=0x7ff46c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.391+0000 7ff471ffb640 1 -- 172.21.15.43:0/4294086830 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c004d20 con 0x7ff47419a050 2024-04-21T08:44:59.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.391+0000 7ff471ffb640 1 -- 172.21.15.43:0/4294086830 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff46c005020 con 0x7ff47419a050 2024-04-21T08:44:59.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.391+0000 7ff47abb6640 1 -- 172.21.15.43:0/4294086830 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4741a50c0 con 0x7ff47419a050 2024-04-21T08:44:59.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.391+0000 7ff471ffb640 1 -- 172.21.15.43:0/4294086830 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c0116d0 con 0x7ff47419a050 2024-04-21T08:44:59.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.391+0000 7ff47abb6640 1 -- 172.21.15.43:0/4294086830 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4741a55e0 con 0x7ff47419a050 2024-04-21T08:44:59.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.392+0000 7ff47abb6640 1 -- 172.21.15.43:0/4294086830 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff474069230 con 0x7ff47419a050 2024-04-21T08:44:59.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.393+0000 7ff471ffb640 1 -- 172.21.15.43:0/4294086830 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff46c005460 con 0x7ff47419a050 2024-04-21T08:44:59.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.394+0000 7ff471ffb640 1 --2- 172.21.15.43:0/4294086830 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff4640a3da0 0x7ff4640a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.394+0000 7ff478952640 1 --2- 172.21.15.43:0/4294086830 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff4640a3da0 0x7ff4640a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.394+0000 7ff471ffb640 1 -- 172.21.15.43:0/4294086830 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7ff46c0d0ba0 con 0x7ff47419a050 2024-04-21T08:44:59.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.394+0000 7ff478952640 1 --2- 172.21.15.43:0/4294086830 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff4640a3da0 0x7ff4640a6260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff4680104e0 tx=0x7ff46800b410 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.398+0000 7ff471ffb640 1 -- 172.21.15.43:0/4294086830 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff46c016030 con 0x7ff47419a050 2024-04-21T08:44:59.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.624+0000 7ff47abb6640 1 -- 172.21.15.43:0/4294086830 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4741093b0 con 0x7ff47419a050 2024-04-21T08:44:59.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.625+0000 7ff471ffb640 1 -- 172.21.15.43:0/4294086830 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7ff46c0b4ec0 con 0x7ff47419a050 2024-04-21T08:44:59.625 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:44:59.625 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:44:59.625 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:44:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.627+0000 7ff47abb6640 1 -- 172.21.15.43:0/4294086830 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff4640a3da0 msgr2=0x7ff4640a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.627+0000 7ff47abb6640 1 --2- 172.21.15.43:0/4294086830 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff4640a3da0 0x7ff4640a6260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff4680104e0 tx=0x7ff46800b410 comp rx=0 tx=0).stop 2024-04-21T08:44:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.627+0000 7ff47abb6640 1 -- 172.21.15.43:0/4294086830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff47419a050 msgr2=0x7ff4741a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.627+0000 7ff47abb6640 1 --2- 172.21.15.43:0/4294086830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff47419a050 0x7ff4741a4600 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00e940 tx=0x7ff46c00ee00 comp rx=0 tx=0).stop 2024-04-21T08:44:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.627+0000 7ff47abb6640 1 -- 172.21.15.43:0/4294086830 shutdown_connections 2024-04-21T08:44:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.627+0000 7ff47abb6640 1 --2- 172.21.15.43:0/4294086830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff47419a050 0x7ff4741a4600 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.627+0000 7ff47abb6640 1 --2- 172.21.15.43:0/4294086830 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff4640a3da0 0x7ff4640a6260 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.627+0000 7ff47abb6640 1 --2- 172.21.15.43:0/4294086830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff474106180 0x7ff474199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.627+0000 7ff47abb6640 1 --2- 172.21.15.43:0/4294086830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4741010b0 0x7ff474199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.627+0000 7ff47abb6640 1 -- 172.21.15.43:0/4294086830 >> 172.21.15.43:0/4294086830 conn(0x7ff4740771d0 msgr2=0x7ff4740fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.627+0000 7ff47abb6640 1 -- 172.21.15.43:0/4294086830 shutdown_connections 2024-04-21T08:44:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.627+0000 7ff47abb6640 1 -- 172.21.15.43:0/4294086830 wait complete. 2024-04-21T08:44:59.639 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:44:59.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.743+0000 7fc0a3a93640 1 -- 172.21.15.43:0/2407881883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc09c107270 msgr2=0x7fc09c10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.743+0000 7fc0a3a93640 1 --2- 172.21.15.43:0/2407881883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc09c107270 0x7fc09c10ee40 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fc094009ca0 tx=0x7fc09402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.743+0000 7fc0a3a93640 1 -- 172.21.15.43:0/2407881883 shutdown_connections 2024-04-21T08:44:59.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.743+0000 7fc0a3a93640 1 --2- 172.21.15.43:0/2407881883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc09c107270 0x7fc09c10ee40 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.743+0000 7fc0a3a93640 1 --2- 172.21.15.43:0/2407881883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc09c06b1f0 0x7fc09c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.743+0000 7fc0a3a93640 1 --2- 172.21.15.43:0/2407881883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc09c06a840 0x7fc09c06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.743+0000 7fc0a3a93640 1 -- 172.21.15.43:0/2407881883 >> 172.21.15.43:0/2407881883 conn(0x7fc09c077eb0 msgr2=0x7fc09c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.743+0000 7fc0a3a93640 1 -- 172.21.15.43:0/2407881883 shutdown_connections 2024-04-21T08:44:59.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.743+0000 7fc0a3a93640 1 -- 172.21.15.43:0/2407881883 wait complete. 2024-04-21T08:44:59.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.743+0000 7fc0a3a93640 1 Processor -- start 2024-04-21T08:44:59.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.743+0000 7fc0a3a93640 1 -- start start 2024-04-21T08:44:59.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.743+0000 7fc0a3a93640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc09c06a840 0x7fc09c100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.743+0000 7fc0a3a93640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc09c107270 0x7fc09c1013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.743+0000 7fc0a3a93640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc09c101900 0x7fc09c1a6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.743+0000 7fc0a3a93640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1121b0 con 0x7fc09c101900 2024-04-21T08:44:59.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.743+0000 7fc0a3a93640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c112030 con 0x7fc09c06a840 2024-04-21T08:44:59.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.743+0000 7fc0a3a93640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c112330 con 0x7fc09c107270 2024-04-21T08:44:59.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.744+0000 7fc0a2030640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc09c101900 0x7fc09c1a6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.744+0000 7fc0a2030640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc09c101900 0x7fc09c1a6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33602/0 (socket says 172.21.15.43:33602) 2024-04-21T08:44:59.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.744+0000 7fc0a2030640 1 -- 172.21.15.43:0/1026854468 learned_addr learned my addr 172.21.15.43:0/1026854468 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:59.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.744+0000 7fc0a102e640 1 --2- 172.21.15.43:0/1026854468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc09c107270 0x7fc09c1013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.744+0000 7fc0a182f640 1 --2- 172.21.15.43:0/1026854468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc09c06a840 0x7fc09c100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.744+0000 7fc0a2030640 1 -- 172.21.15.43:0/1026854468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc09c107270 msgr2=0x7fc09c1013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.744+0000 7fc0a2030640 1 --2- 172.21.15.43:0/1026854468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc09c107270 0x7fc09c1013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.744+0000 7fc0a2030640 1 -- 172.21.15.43:0/1026854468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc09c06a840 msgr2=0x7fc09c100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.744+0000 7fc0a2030640 1 --2- 172.21.15.43:0/1026854468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc09c06a840 0x7fc09c100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.744+0000 7fc0a2030640 1 -- 172.21.15.43:0/1026854468 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc09c1a6ee0 con 0x7fc09c101900 2024-04-21T08:44:59.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.744+0000 7fc0a2030640 1 --2- 172.21.15.43:0/1026854468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc09c101900 0x7fc09c1a6840 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fc094005e30 tx=0x7fc094034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.744+0000 7fc092ffd640 1 -- 172.21.15.43:0/1026854468 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc094041020 con 0x7fc09c101900 2024-04-21T08:44:59.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.744+0000 7fc092ffd640 1 -- 172.21.15.43:0/1026854468 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc094032970 con 0x7fc09c101900 2024-04-21T08:44:59.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.744+0000 7fc0a3a93640 1 -- 172.21.15.43:0/1026854468 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc09c1a7170 con 0x7fc09c101900 2024-04-21T08:44:59.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.744+0000 7fc092ffd640 1 -- 172.21.15.43:0/1026854468 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc094039040 con 0x7fc09c101900 2024-04-21T08:44:59.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.744+0000 7fc0a3a93640 1 -- 172.21.15.43:0/1026854468 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc09c1a76b0 con 0x7fc09c101900 2024-04-21T08:44:59.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.746+0000 7fc092ffd640 1 -- 172.21.15.43:0/1026854468 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc094044020 con 0x7fc09c101900 2024-04-21T08:44:59.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.747+0000 7fc0a6765740 1 Processor -- start 2024-04-21T08:44:59.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.747+0000 7fc0a6765740 1 -- start start 2024-04-21T08:44:59.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.747+0000 7fc0a6765740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2e1fa04d0 0x55f2e1fa08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.747+0000 7fc0a6765740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa0e80 0x55f2e1fa9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.747+0000 7fc0a6765740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2e1fa9a00 0x55f2e1fabe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.747+0000 7fc092ffd640 1 --2- 172.21.15.43:0/1026854468 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc0880a3da0 0x7fc0880a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.747+0000 7fc0a6765740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f2e1f051f0 con 0x55f2e1fa0e80 2024-04-21T08:44:59.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.747+0000 7fc0a6765740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f2e1ea24b0 con 0x55f2e1fa9a00 2024-04-21T08:44:59.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.747+0000 7fc0a6765740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f2e1f011b0 con 0x55f2e1fa04d0 2024-04-21T08:44:59.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.748+0000 7fc0a182f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa0e80 0x55f2e1fa9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.748+0000 7fc0a182f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa0e80 0x55f2e1fa9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33612/0 (socket says 172.21.15.43:33612) 2024-04-21T08:44:59.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.748+0000 7fc0a182f640 1 -- 172.21.15.43:0/589877523 learned_addr learned my addr 172.21.15.43:0/589877523 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:59.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.748+0000 7fc092ffd640 1 -- 172.21.15.43:0/1026854468 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc0940f28f0 con 0x7fc09c101900 2024-04-21T08:44:59.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.748+0000 7fc0a2030640 1 --2- 172.21.15.43:0/1026854468 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc0880a3da0 0x7fc0880a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.748+0000 7fc0a102e640 1 --2- 172.21.15.43:0/589877523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2e1fa9a00 0x55f2e1fabe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.748+0000 7fc0a102e640 1 --2- 172.21.15.43:0/589877523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2e1fa04d0 0x55f2e1fa08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.748+0000 7fc0a182f640 1 -- 172.21.15.43:0/589877523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2e1fa04d0 msgr2=0x55f2e1fa08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.748+0000 7fc0a182f640 1 --2- 172.21.15.43:0/589877523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2e1fa04d0 0x55f2e1fa08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.748+0000 7fc0a182f640 1 -- 172.21.15.43:0/589877523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2e1fa9a00 msgr2=0x55f2e1fabe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.748+0000 7fc0a182f640 1 --2- 172.21.15.43:0/589877523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2e1fa9a00 0x55f2e1fabe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.748+0000 7fc0a182f640 1 -- 172.21.15.43:0/589877523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2e1e941b0 con 0x55f2e1fa0e80 2024-04-21T08:44:59.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.748+0000 7fc0a2030640 1 --2- 172.21.15.43:0/1026854468 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc0880a3da0 0x7fc0880a6260 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc0940efde0 tx=0x7fc0940a9040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.748+0000 7fc0a182f640 1 --2- 172.21.15.43:0/589877523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa0e80 0x55f2e1fa9430 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fc09800bcb0 tx=0x7fc09800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3f033c94e2b7d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.748+0000 7fc087fff640 1 -- 172.21.15.43:0/589877523 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098015070 con 0x55f2e1fa0e80 2024-04-21T08:44:59.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.748+0000 7fc087fff640 1 -- 172.21.15.43:0/589877523 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0980118f0 con 0x55f2e1fa0e80 2024-04-21T08:44:59.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.748+0000 7fc087fff640 1 -- 172.21.15.43:0/589877523 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0980108b0 con 0x55f2e1fa0e80 2024-04-21T08:44:59.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.749+0000 7fc0a6765740 1 -- 172.21.15.43:0/589877523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa0e80 msgr2=0x55f2e1fa9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.749+0000 7fc0a6765740 1 --2- 172.21.15.43:0/589877523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa0e80 0x55f2e1fa9430 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fc09800bcb0 tx=0x7fc09800f730 comp rx=0 tx=0).stop 2024-04-21T08:44:59.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.749+0000 7fc0a6765740 1 -- 172.21.15.43:0/589877523 shutdown_connections 2024-04-21T08:44:59.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.749+0000 7fc0a6765740 1 --2- 172.21.15.43:0/589877523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2e1fa9a00 0x55f2e1fabe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.749+0000 7fc0a6765740 1 --2- 172.21.15.43:0/589877523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa0e80 0x55f2e1fa9430 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.749+0000 7fc0a6765740 1 --2- 172.21.15.43:0/589877523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2e1fa04d0 0x55f2e1fa08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.749+0000 7fc0a6765740 1 -- 172.21.15.43:0/589877523 >> 172.21.15.43:0/589877523 conn(0x55f2e1f9aba0 msgr2=0x55f2e1f9af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.749+0000 7fc0a6765740 1 -- 172.21.15.43:0/589877523 shutdown_connections 2024-04-21T08:44:59.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.749+0000 7fc0a6765740 1 -- 172.21.15.43:0/589877523 wait complete. 2024-04-21T08:44:59.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.749+0000 7fc0a6765740 1 -- start start 2024-04-21T08:44:59.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.750+0000 7fc0a6765740 1 Processor -- start 2024-04-21T08:44:59.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.750+0000 7fc0a6765740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2e1fa04d0 0x55f2e1fa9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.750+0000 7fc0a6765740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa0e80 0x55f2e1fa6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.750+0000 7fc0a6765740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2e1fa9a00 0x55f2e1fa6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.750+0000 7fc0a6765740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f2e1fac970 con 0x55f2e1fa0e80 2024-04-21T08:44:59.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.750+0000 7fc0a6765740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f2e1ebe500 con 0x55f2e1fa9a00 2024-04-21T08:44:59.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.750+0000 7fc0a6765740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f2e1facaf0 con 0x55f2e1fa04d0 2024-04-21T08:44:59.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.750+0000 7fc0a6765740 10 client.0 _ll_register_callbacks cb 0x55f2e1e8cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:59.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.750+0000 7fc0a182f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa0e80 0x55f2e1fa6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.750+0000 7fc0a182f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa0e80 0x55f2e1fa6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33624/0 (socket says 172.21.15.43:33624) 2024-04-21T08:44:59.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.750+0000 7fc0a182f640 1 -- 172.21.15.43:0/94481796 learned_addr learned my addr 172.21.15.43:0/94481796 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:59.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.750+0000 7fc0a102e640 1 --2- 172.21.15.43:0/94481796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2e1fa9a00 0x55f2e1fa6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.750+0000 7fc0a102e640 1 --2- 172.21.15.43:0/94481796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2e1fa04d0 0x55f2e1fa9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.750+0000 7fc0a182f640 1 -- 172.21.15.43:0/94481796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2e1fa04d0 msgr2=0x55f2e1fa9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.750+0000 7fc0a182f640 1 --2- 172.21.15.43:0/94481796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2e1fa04d0 0x55f2e1fa9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.750+0000 7fc0a182f640 1 -- 172.21.15.43:0/94481796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2e1fa9a00 msgr2=0x55f2e1fa6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.750+0000 7fc0a182f640 1 --2- 172.21.15.43:0/94481796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2e1fa9a00 0x55f2e1fa6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.750+0000 7fc0a182f640 1 -- 172.21.15.43:0/94481796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2e1c173f0 con 0x55f2e1fa0e80 2024-04-21T08:44:59.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.751+0000 7fc0a182f640 1 --2- 172.21.15.43:0/94481796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa0e80 0x55f2e1fa6720 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fc098000c00 tx=0x7fc09800b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c57eef3d0b29f51 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.751+0000 7fc0857fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:59.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.751+0000 7fc0857fa640 1 -- 172.21.15.43:0/94481796 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098029060 con 0x55f2e1fa0e80 2024-04-21T08:44:59.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.751+0000 7fc0857fa640 1 -- 172.21.15.43:0/94481796 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc098011aa0 con 0x55f2e1fa0e80 2024-04-21T08:44:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.751+0000 7fc0a6765740 1 -- 172.21.15.43:0/94481796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0980118f0 con 0x55f2e1fa0e80 2024-04-21T08:44:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.751+0000 7fc0857fa640 1 -- 172.21.15.43:0/94481796 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc098030050 con 0x55f2e1fa0e80 2024-04-21T08:44:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.751+0000 7fc0a6765740 10 client.18225 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.751+0000 7fc0a6765740 1 -- 172.21.15.43:0/94481796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f2e1ce17d0 con 0x55f2e1fa0e80 2024-04-21T08:44:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.751+0000 7fc0857fa640 1 -- 172.21.15.43:0/94481796 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0980118f0 con 0x55f2e1fa0e80 2024-04-21T08:44:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.751+0000 7fc0857fa640 1 -- 172.21.15.43:0/94481796 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc098003850 con 0x55f2e1fa0e80 2024-04-21T08:44:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.751+0000 7fc0a6765740 10 client.18225 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.751+0000 7fc0a6765740 10 client.18225 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:44:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.751+0000 7fc0a6765740 20 client.18225 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.751+0000 7fc0a6765740 1 --2- 172.21.15.43:0/94481796 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55f2e1fb9bd0 0x55f2e1fbbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.751+0000 7fc0a6765740 4 client.18225 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.751+0000 7fc0a6765740 1 -- 172.21.15.43:0/94481796 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2e1ea4930 con 0x55f2e1fb9bd0 2024-04-21T08:44:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.752+0000 7fc0a102e640 1 --2- 172.21.15.43:0/94481796 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55f2e1fb9bd0 0x55f2e1fbbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.752+0000 7fc0857fa640 10 client.18225 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:44:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.752+0000 7fc0a102e640 1 --2- 172.21.15.43:0/94481796 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55f2e1fb9bd0 0x55f2e1fbbfc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eefe5f28c03d027d server_cookie=3233bf06fdb683d1 in_seq=0 out_seq=0 2024-04-21T08:44:59.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.753+0000 7fc0857fa640 1 -- 172.21.15.43:0/94481796 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2e1ea4930 con 0x55f2e1fb9bd0 2024-04-21T08:44:59.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.753+0000 7fc0857fa640 10 client.18225 handle_command_reply: tid=0 2024-04-21T08:44:59.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.753+0000 7fc0a6765740 1 client.18225 shutdown 2024-04-21T08:44:59.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.753+0000 7fc0a6765740 1 -- 172.21.15.43:0/94481796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa0e80 msgr2=0x55f2e1fa6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.753+0000 7fc0a6765740 1 --2- 172.21.15.43:0/94481796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa0e80 0x55f2e1fa6720 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fc098000c00 tx=0x7fc09800b9f0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.753+0000 7fc0a6765740 1 -- 172.21.15.43:0/94481796 shutdown_connections 2024-04-21T08:44:59.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.753+0000 7fc0a6765740 1 --2- 172.21.15.43:0/94481796 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55f2e1fb9bd0 0x55f2e1fbbfc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.753+0000 7fc0a6765740 1 --2- 172.21.15.43:0/94481796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2e1fa9a00 0x55f2e1fa6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.753+0000 7fc0a6765740 1 --2- 172.21.15.43:0/94481796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa0e80 0x55f2e1fa6720 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.753+0000 7fc0a6765740 1 --2- 172.21.15.43:0/94481796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2e1fa04d0 0x55f2e1fa9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.753+0000 7fc0857fa640 0 client.18225 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:44:59.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.753+0000 7fc0a6765740 1 -- 172.21.15.43:0/94481796 >> 172.21.15.43:0/94481796 conn(0x55f2e1f9aba0 msgr2=0x55f2e1f9c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.753+0000 7fc0a6765740 1 -- 172.21.15.43:0/94481796 shutdown_connections 2024-04-21T08:44:59.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.753+0000 7fc0a6765740 1 -- 172.21.15.43:0/94481796 wait complete. 2024-04-21T08:44:59.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.753+0000 7fc0a6765740 20 client.18225 trim_cache size 0 max 16384 2024-04-21T08:44:59.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.766+0000 7fc0a6765740 1 Processor -- start 2024-04-21T08:44:59.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.766+0000 7fc0a6765740 1 -- start start 2024-04-21T08:44:59.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.767+0000 7fc0a6765740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2e1fa04d0 0x55f2e1fb5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.767+0000 7fc0a6765740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2e1fa0e80 0x55f2e1fa8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.767+0000 7fc0a6765740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa9a00 0x55f2e1fa9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.767+0000 7fc0a6765740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1b15d0 con 0x55f2e1fa9a00 2024-04-21T08:44:59.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.767+0000 7fc0a6765740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1b3110 con 0x55f2e1fa0e80 2024-04-21T08:44:59.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.767+0000 7fc0a6765740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1b1ca0 con 0x55f2e1fa04d0 2024-04-21T08:44:59.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.767+0000 7fc0a102e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa9a00 0x55f2e1fa9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.767+0000 7fc0a102e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa9a00 0x55f2e1fa9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33636/0 (socket says 172.21.15.43:33636) 2024-04-21T08:44:59.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.767+0000 7fc0a102e640 1 -- 172.21.15.43:0/804261559 learned_addr learned my addr 172.21.15.43:0/804261559 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:59.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.767+0000 7fc0a102e640 1 --2- 172.21.15.43:0/804261559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2e1fa04d0 0x55f2e1fb5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.767+0000 7fc0a182f640 1 --2- 172.21.15.43:0/804261559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2e1fa0e80 0x55f2e1fa8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.767+0000 7fc0a102e640 1 -- 172.21.15.43:0/804261559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2e1fa04d0 msgr2=0x55f2e1fb5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.767+0000 7fc0a102e640 1 --2- 172.21.15.43:0/804261559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2e1fa04d0 0x55f2e1fb5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.767+0000 7fc0a102e640 1 -- 172.21.15.43:0/804261559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2e1fa0e80 msgr2=0x55f2e1fa8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.767+0000 7fc0a102e640 1 --2- 172.21.15.43:0/804261559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2e1fa0e80 0x55f2e1fa8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.767+0000 7fc0a102e640 1 -- 172.21.15.43:0/804261559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2e1fa70a0 con 0x55f2e1fa9a00 2024-04-21T08:44:59.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.767+0000 7fc0a102e640 1 --2- 172.21.15.43:0/804261559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa9a00 0x55f2e1fa9390 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0042e0 tx=0x7fc08c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2ff92948605b2a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.767+0000 7fc0877fe640 1 -- 172.21.15.43:0/804261559 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c01f350 con 0x55f2e1fa9a00 2024-04-21T08:44:59.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.768+0000 7fc0877fe640 1 -- 172.21.15.43:0/804261559 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc08c005020 con 0x55f2e1fa9a00 2024-04-21T08:44:59.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.768+0000 7fc0877fe640 1 -- 172.21.15.43:0/804261559 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c0047a0 con 0x55f2e1fa9a00 2024-04-21T08:44:59.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.768+0000 7fc0a6765740 1 -- 172.21.15.43:0/804261559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa9a00 msgr2=0x55f2e1fa9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.768+0000 7fc0a6765740 1 --2- 172.21.15.43:0/804261559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa9a00 0x55f2e1fa9390 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0042e0 tx=0x7fc08c00dc10 comp rx=0 tx=0).stop 2024-04-21T08:44:59.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.768+0000 7fc0a6765740 1 -- 172.21.15.43:0/804261559 shutdown_connections 2024-04-21T08:44:59.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.768+0000 7fc0a6765740 1 --2- 172.21.15.43:0/804261559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa9a00 0x55f2e1fa9390 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.768+0000 7fc0a6765740 1 --2- 172.21.15.43:0/804261559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2e1fa0e80 0x55f2e1fa8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.768+0000 7fc0a6765740 1 --2- 172.21.15.43:0/804261559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2e1fa04d0 0x55f2e1fb5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.768+0000 7fc0a6765740 1 -- 172.21.15.43:0/804261559 >> 172.21.15.43:0/804261559 conn(0x55f2e1f9aba0 msgr2=0x55f2e1fa1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.768+0000 7fc0a6765740 1 -- 172.21.15.43:0/804261559 shutdown_connections 2024-04-21T08:44:59.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.768+0000 7fc0a6765740 1 -- 172.21.15.43:0/804261559 wait complete. 2024-04-21T08:44:59.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.768+0000 7fc0a6765740 1 -- start start 2024-04-21T08:44:59.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.768+0000 7fc0a6765740 1 Processor -- start 2024-04-21T08:44:59.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.769+0000 7fc0a6765740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2e1fa04d0 0x55f2e1fa35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.769+0000 7fc0a6765740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2e1fa0e80 0x55f2e1fa3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.769+0000 7fc0a6765740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa9a00 0x55f2e1fa4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.769+0000 7fc0a6765740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1b2a40 con 0x55f2e1fa9a00 2024-04-21T08:44:59.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.769+0000 7fc0a6765740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1b2370 con 0x55f2e1fa04d0 2024-04-21T08:44:59.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.769+0000 7fc0a6765740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1af5b0 con 0x55f2e1fa0e80 2024-04-21T08:44:59.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.769+0000 7fc0a6765740 10 client.0 _ll_register_callbacks cb 0x7fc09800be30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:59.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.769+0000 7fc0a102e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa9a00 0x55f2e1fa4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.769+0000 7fc0a102e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa9a00 0x55f2e1fa4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33638/0 (socket says 172.21.15.43:33638) 2024-04-21T08:44:59.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.769+0000 7fc0a102e640 1 -- 172.21.15.43:0/3996942757 learned_addr learned my addr 172.21.15.43:0/3996942757 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:59.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.769+0000 7fc0a102e640 1 --2- 172.21.15.43:0/3996942757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2e1fa04d0 0x55f2e1fa35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.769+0000 7fc0a182f640 1 --2- 172.21.15.43:0/3996942757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2e1fa0e80 0x55f2e1fa3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.769+0000 7fc0a102e640 1 -- 172.21.15.43:0/3996942757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2e1fa0e80 msgr2=0x55f2e1fa3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.769+0000 7fc0a102e640 1 --2- 172.21.15.43:0/3996942757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2e1fa0e80 0x55f2e1fa3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.769+0000 7fc0a102e640 1 -- 172.21.15.43:0/3996942757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2e1fa04d0 msgr2=0x55f2e1fa35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.769+0000 7fc0a102e640 1 --2- 172.21.15.43:0/3996942757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2e1fa04d0 0x55f2e1fa35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.769+0000 7fc0a102e640 1 -- 172.21.15.43:0/3996942757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2e201ef60 con 0x55f2e1fa9a00 2024-04-21T08:44:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.770+0000 7fc0a102e640 1 --2- 172.21.15.43:0/3996942757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa9a00 0x55f2e1fa4040 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0056c0 tx=0x7fc08c00df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4224df63fd2c4c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.770+0000 7fc0857fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.770+0000 7fc0857fa640 1 -- 172.21.15.43:0/3996942757 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c004700 con 0x55f2e1fa9a00 2024-04-21T08:44:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.770+0000 7fc0857fa640 1 -- 172.21.15.43:0/3996942757 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc08c01a040 con 0x55f2e1fa9a00 2024-04-21T08:44:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.770+0000 7fc0a6765740 1 -- 172.21.15.43:0/3996942757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc08c005020 con 0x55f2e1fa9a00 2024-04-21T08:44:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.770+0000 7fc0857fa640 1 -- 172.21.15.43:0/3996942757 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc08c0048a0 con 0x55f2e1fa9a00 2024-04-21T08:44:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.770+0000 7fc0a6765740 10 client.18231 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.770+0000 7fc0a6765740 1 -- 172.21.15.43:0/3996942757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f2e2020240 con 0x55f2e1fa9a00 2024-04-21T08:44:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.770+0000 7fc0857fa640 1 -- 172.21.15.43:0/3996942757 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc08c005020 con 0x55f2e1fa9a00 2024-04-21T08:44:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.770+0000 7fc0857fa640 1 -- 172.21.15.43:0/3996942757 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc08c01fce0 con 0x55f2e1fa9a00 2024-04-21T08:44:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.770+0000 7fc0a6765740 10 client.18231 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.770+0000 7fc0a6765740 10 client.18231 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:44:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.770+0000 7fc0a6765740 20 client.18231 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.770+0000 7fc0a6765740 1 --2- 172.21.15.43:0/3996942757 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55f2e2020960 0x55f2e2022d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.770+0000 7fc0a6765740 4 client.18231 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:44:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.770+0000 7fc0a6765740 1 -- 172.21.15.43:0/3996942757 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f2e1f12c10 con 0x55f2e2020960 2024-04-21T08:44:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.771+0000 7fc0a182f640 1 --2- 172.21.15.43:0/3996942757 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55f2e2020960 0x55f2e2022d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.771+0000 7fc0a182f640 1 --2- 172.21.15.43:0/3996942757 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55f2e2020960 0x55f2e2022d80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3dbabe7c6775881b server_cookie=871cb750401d4342 in_seq=0 out_seq=0 2024-04-21T08:44:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.771+0000 7fc0857fa640 10 client.18231 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:44:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.771+0000 7fc0857fa640 1 -- 172.21.15.43:0/3996942757 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f2e1f12c10 con 0x55f2e2020960 2024-04-21T08:44:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.771+0000 7fc0857fa640 10 client.18231 handle_command_reply: tid=0 2024-04-21T08:44:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.771+0000 7fc0a6765740 1 client.18231 shutdown 2024-04-21T08:44:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.771+0000 7fc0a6765740 1 -- 172.21.15.43:0/3996942757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa9a00 msgr2=0x55f2e1fa4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.771+0000 7fc0a6765740 1 --2- 172.21.15.43:0/3996942757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa9a00 0x55f2e1fa4040 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0056c0 tx=0x7fc08c00df40 comp rx=0 tx=0).stop 2024-04-21T08:44:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.772+0000 7fc0a6765740 1 -- 172.21.15.43:0/3996942757 shutdown_connections 2024-04-21T08:44:59.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.772+0000 7fc0a6765740 1 --2- 172.21.15.43:0/3996942757 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55f2e2020960 0x55f2e2022d80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.772+0000 7fc0a6765740 1 --2- 172.21.15.43:0/3996942757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f2e1fa9a00 0x55f2e1fa4040 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.772+0000 7fc0a6765740 1 --2- 172.21.15.43:0/3996942757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f2e1fa0e80 0x55f2e1fa3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.772+0000 7fc0a6765740 1 --2- 172.21.15.43:0/3996942757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2e1fa04d0 0x55f2e1fa35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.772+0000 7fc0857fa640 0 client.18231 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:44:59.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.772+0000 7fc0a6765740 1 -- 172.21.15.43:0/3996942757 >> 172.21.15.43:0/3996942757 conn(0x55f2e1f9aba0 msgr2=0x55f2e1f99370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.772+0000 7fc0a6765740 1 -- 172.21.15.43:0/3996942757 shutdown_connections 2024-04-21T08:44:59.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.772+0000 7fc0a6765740 1 -- 172.21.15.43:0/3996942757 wait complete. 2024-04-21T08:44:59.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.772+0000 7fc0a6765740 20 client.18231 trim_cache size 0 max 16384 2024-04-21T08:44:59.778 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:44:59.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.772+0000 7fc0a3a93640 1 -- 172.21.15.43:0/1026854468 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc0880a3da0 msgr2=0x7fc0880a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.772+0000 7fc0a3a93640 1 --2- 172.21.15.43:0/1026854468 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc0880a3da0 0x7fc0880a6260 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc0940efde0 tx=0x7fc0940a9040 comp rx=0 tx=0).stop 2024-04-21T08:44:59.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.772+0000 7fc0a3a93640 1 -- 172.21.15.43:0/1026854468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc09c101900 msgr2=0x7fc09c1a6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.772+0000 7fc0a3a93640 1 --2- 172.21.15.43:0/1026854468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc09c101900 0x7fc09c1a6840 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fc094005e30 tx=0x7fc094034670 comp rx=0 tx=0).stop 2024-04-21T08:44:59.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.772+0000 7fc0a3a93640 1 -- 172.21.15.43:0/1026854468 shutdown_connections 2024-04-21T08:44:59.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.772+0000 7fc0a3a93640 1 --2- 172.21.15.43:0/1026854468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc09c101900 0x7fc09c1a6840 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.772+0000 7fc0a3a93640 1 --2- 172.21.15.43:0/1026854468 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc0880a3da0 0x7fc0880a6260 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.772+0000 7fc0a3a93640 1 --2- 172.21.15.43:0/1026854468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc09c107270 0x7fc09c1013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.772+0000 7fc0a3a93640 1 --2- 172.21.15.43:0/1026854468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc09c06a840 0x7fc09c100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.772+0000 7fc0a3a93640 1 -- 172.21.15.43:0/1026854468 >> 172.21.15.43:0/1026854468 conn(0x7fc09c077eb0 msgr2=0x7fc09c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.772+0000 7fc0a3a93640 1 -- 172.21.15.43:0/1026854468 shutdown_connections 2024-04-21T08:44:59.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.772+0000 7fc0a3a93640 1 -- 172.21.15.43:0/1026854468 wait complete. 2024-04-21T08:44:59.783 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:44:59.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.887+0000 7faafab04640 1 -- 172.21.15.43:0/3568571973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaf4106160 msgr2=0x7faaf410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.888+0000 7faafab04640 1 --2- 172.21.15.43:0/3568571973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaf4106160 0x7faaf410dd30 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7faaec009ca0 tx=0x7faaec02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.888+0000 7faafab04640 1 -- 172.21.15.43:0/3568571973 shutdown_connections 2024-04-21T08:44:59.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.888+0000 7faafab04640 1 --2- 172.21.15.43:0/3568571973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaf4106160 0x7faaf410dd30 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.888+0000 7faafab04640 1 --2- 172.21.15.43:0/3568571973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaf4101a40 0x7faaf4105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.888+0000 7faafab04640 1 --2- 172.21.15.43:0/3568571973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf4101090 0x7faaf4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.888+0000 7faafab04640 1 -- 172.21.15.43:0/3568571973 >> 172.21.15.43:0/3568571973 conn(0x7faaf4077f50 msgr2=0x7faaf4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.888+0000 7faafab04640 1 -- 172.21.15.43:0/3568571973 shutdown_connections 2024-04-21T08:44:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.888+0000 7faafab04640 1 -- 172.21.15.43:0/3568571973 wait complete. 2024-04-21T08:44:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.888+0000 7faafab04640 1 Processor -- start 2024-04-21T08:44:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.888+0000 7faafab04640 1 -- start start 2024-04-21T08:44:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.888+0000 7faafab04640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf4101090 0x7faaf41986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.888+0000 7faafab04640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaf4106160 0x7faaf4198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.888+0000 7faafab04640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaf419cf80 0x7faaf419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.888+0000 7faafab04640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41121e0 con 0x7faaf419cf80 2024-04-21T08:44:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.888+0000 7faafab04640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faaf4112060 con 0x7faaf4101090 2024-04-21T08:44:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.888+0000 7faafab04640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faaf4112360 con 0x7faaf4106160 2024-04-21T08:44:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.889+0000 7faaf90a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaf419cf80 0x7faaf419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.889+0000 7faaf90a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaf419cf80 0x7faaf419d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33664/0 (socket says 172.21.15.43:33664) 2024-04-21T08:44:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.889+0000 7faaf90a1640 1 -- 172.21.15.43:0/2368594595 learned_addr learned my addr 172.21.15.43:0/2368594595 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.889+0000 7faaf3fff640 1 --2- 172.21.15.43:0/2368594595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaf4106160 0x7faaf4198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.889+0000 7faaf88a0640 1 --2- 172.21.15.43:0/2368594595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf4101090 0x7faaf41986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.889+0000 7faaf90a1640 1 -- 172.21.15.43:0/2368594595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaf4106160 msgr2=0x7faaf4198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.889+0000 7faaf90a1640 1 --2- 172.21.15.43:0/2368594595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaf4106160 0x7faaf4198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.889+0000 7faaf90a1640 1 -- 172.21.15.43:0/2368594595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf4101090 msgr2=0x7faaf41986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.889+0000 7faaf90a1640 1 --2- 172.21.15.43:0/2368594595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf4101090 0x7faaf41986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.889+0000 7faaf90a1640 1 -- 172.21.15.43:0/2368594595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf419da70 con 0x7faaf419cf80 2024-04-21T08:44:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.889+0000 7faaf90a1640 1 --2- 172.21.15.43:0/2368594595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaf419cf80 0x7faaf419d430 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7faaec005e30 tx=0x7faaec0346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.889+0000 7faaf1ffb640 1 -- 172.21.15.43:0/2368594595 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec041020 con 0x7faaf419cf80 2024-04-21T08:44:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.889+0000 7faaf1ffb640 1 -- 172.21.15.43:0/2368594595 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaec032970 con 0x7faaf419cf80 2024-04-21T08:44:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.889+0000 7faafab04640 1 -- 172.21.15.43:0/2368594595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf419dd00 con 0x7faaf419cf80 2024-04-21T08:44:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.889+0000 7faaf1ffb640 1 -- 172.21.15.43:0/2368594595 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec039040 con 0x7faaf419cf80 2024-04-21T08:44:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.889+0000 7faafab04640 1 -- 172.21.15.43:0/2368594595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf41a5520 con 0x7faaf419cf80 2024-04-21T08:44:59.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.892+0000 7faaf1ffb640 1 -- 172.21.15.43:0/2368594595 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faaec044020 con 0x7faaf419cf80 2024-04-21T08:44:59.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.892+0000 7faafd7d1740 1 Processor -- start 2024-04-21T08:44:59.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.892+0000 7faafd7d1740 1 -- start start 2024-04-21T08:44:59.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.892+0000 7faafd7d1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565462b7d3b0 0x565462b7d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.892+0000 7faafd7d1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b7dd60 0x565462b86890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.892+0000 7faafd7d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565462b86e60 0x565462b872c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.892+0000 7faafd7d1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x565462ae21f0 con 0x565462b7dd60 2024-04-21T08:44:59.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.892+0000 7faafd7d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x565462a7f4b0 con 0x565462b86e60 2024-04-21T08:44:59.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.892+0000 7faafd7d1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x565462ade1b0 con 0x565462b7d3b0 2024-04-21T08:44:59.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.892+0000 7faaf88a0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b7dd60 0x565462b86890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.892+0000 7faaf1ffb640 1 --2- 172.21.15.43:0/2368594595 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faadc0a3da0 0x7faadc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.892+0000 7faaf88a0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b7dd60 0x565462b86890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33676/0 (socket says 172.21.15.43:33676) 2024-04-21T08:44:59.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.892+0000 7faaf88a0640 1 -- 172.21.15.43:0/803938340 learned_addr learned my addr 172.21.15.43:0/803938340 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:59.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.892+0000 7faaf3fff640 1 --2- 172.21.15.43:0/803938340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565462b86e60 0x565462b872c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.892+0000 7faaf90a1640 1 --2- 172.21.15.43:0/2368594595 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faadc0a3da0 0x7faadc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.892+0000 7faaf3fff640 1 --2- 172.21.15.43:0/803938340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565462b7d3b0 0x565462b7d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.892+0000 7faaf1ffb640 1 -- 172.21.15.43:0/2368594595 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7faaec0f26b0 con 0x7faaf419cf80 2024-04-21T08:44:59.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.893+0000 7faaf88a0640 1 -- 172.21.15.43:0/803938340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565462b7d3b0 msgr2=0x565462b7d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.893+0000 7faaf88a0640 1 --2- 172.21.15.43:0/803938340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565462b7d3b0 0x565462b7d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.893+0000 7faaf88a0640 1 -- 172.21.15.43:0/803938340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565462b86e60 msgr2=0x565462b872c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.893+0000 7faaf88a0640 1 --2- 172.21.15.43:0/803938340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565462b86e60 0x565462b872c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.893+0000 7faaf88a0640 1 -- 172.21.15.43:0/803938340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565462a711b0 con 0x565462b7dd60 2024-04-21T08:44:59.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.893+0000 7faaf3fff640 1 --2- 172.21.15.43:0/803938340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565462b7d3b0 0x565462b7d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:59.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.893+0000 7faaf88a0640 1 --2- 172.21.15.43:0/803938340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b7dd60 0x565462b86890 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7faae4009c80 tx=0x7faae40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a859943c80cd8fc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.893+0000 7faaf90a1640 1 --2- 172.21.15.43:0/2368594595 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faadc0a3da0 0x7faadc0a6260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7faaec0f0030 tx=0x7faaec02c0f0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.893+0000 7faadaffd640 1 -- 172.21.15.43:0/803938340 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4012450 con 0x565462b7dd60 2024-04-21T08:44:59.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.893+0000 7faadaffd640 1 -- 172.21.15.43:0/803938340 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faae4012aa0 con 0x565462b7dd60 2024-04-21T08:44:59.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.893+0000 7faadaffd640 1 -- 172.21.15.43:0/803938340 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae400fae0 con 0x565462b7dd60 2024-04-21T08:44:59.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.894+0000 7faafd7d1740 1 -- 172.21.15.43:0/803938340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b7dd60 msgr2=0x565462b86890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.894+0000 7faafd7d1740 1 --2- 172.21.15.43:0/803938340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b7dd60 0x565462b86890 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7faae4009c80 tx=0x7faae40117b0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.894+0000 7faafd7d1740 1 -- 172.21.15.43:0/803938340 shutdown_connections 2024-04-21T08:44:59.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.894+0000 7faafd7d1740 1 --2- 172.21.15.43:0/803938340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565462b86e60 0x565462b872c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.894+0000 7faafd7d1740 1 --2- 172.21.15.43:0/803938340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b7dd60 0x565462b86890 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.894+0000 7faafd7d1740 1 --2- 172.21.15.43:0/803938340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565462b7d3b0 0x565462b7d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.894+0000 7faafd7d1740 1 -- 172.21.15.43:0/803938340 >> 172.21.15.43:0/803938340 conn(0x565462b77ba0 msgr2=0x565462b77f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.894+0000 7faafd7d1740 1 -- 172.21.15.43:0/803938340 shutdown_connections 2024-04-21T08:44:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.894+0000 7faafd7d1740 1 -- 172.21.15.43:0/803938340 wait complete. 2024-04-21T08:44:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.894+0000 7faafd7d1740 1 -- start start 2024-04-21T08:44:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.894+0000 7faafd7d1740 1 Processor -- start 2024-04-21T08:44:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.895+0000 7faafd7d1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b7d3b0 0x565462b8d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.895+0000 7faafd7d1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565462b7dd60 0x565462b8a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.895+0000 7faafd7d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565462b86e60 0x565462b8aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.895+0000 7faafd7d1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x565462b8dda0 con 0x565462b7d3b0 2024-04-21T08:44:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.895+0000 7faafd7d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x565462a9b500 con 0x565462b86e60 2024-04-21T08:44:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.895+0000 7faafd7d1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x565462b8df20 con 0x565462b7dd60 2024-04-21T08:44:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.895+0000 7faafd7d1740 10 client.0 _ll_register_callbacks cb 0x565462a69b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.895+0000 7faaf3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b7d3b0 0x565462b8d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.895+0000 7faaf3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b7d3b0 0x565462b8d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33682/0 (socket says 172.21.15.43:33682) 2024-04-21T08:44:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.895+0000 7faaf3fff640 1 -- 172.21.15.43:0/1856686336 learned_addr learned my addr 172.21.15.43:0/1856686336 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.895+0000 7faaf88a0640 1 --2- 172.21.15.43:0/1856686336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565462b7dd60 0x565462b8a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.895+0000 7faaf3fff640 1 -- 172.21.15.43:0/1856686336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565462b7dd60 msgr2=0x565462b8a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.895+0000 7faaf3fff640 1 --2- 172.21.15.43:0/1856686336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565462b7dd60 0x565462b8a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.895+0000 7faaf3fff640 1 -- 172.21.15.43:0/1856686336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565462b86e60 msgr2=0x565462b8aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.895+0000 7faaf3fff640 1 --2- 172.21.15.43:0/1856686336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565462b86e60 0x565462b8aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.895+0000 7faaf3fff640 1 -- 172.21.15.43:0/1856686336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654627f43f0 con 0x565462b7d3b0 2024-04-21T08:44:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.895+0000 7faaf3fff640 1 --2- 172.21.15.43:0/1856686336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b7d3b0 0x565462b8d1d0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7faae8009fd0 tx=0x7faae8011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1be2979105d262 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.895+0000 7faad3fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.895+0000 7faad3fff640 1 -- 172.21.15.43:0/1856686336 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae8022060 con 0x565462b7d3b0 2024-04-21T08:44:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.895+0000 7faad3fff640 1 -- 172.21.15.43:0/1856686336 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7faae8009040 con 0x565462b7d3b0 2024-04-21T08:44:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.895+0000 7faafd7d1740 1 -- 172.21.15.43:0/1856686336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faae4012aa0 con 0x565462b7d3b0 2024-04-21T08:44:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.896+0000 7faad3fff640 1 -- 172.21.15.43:0/1856686336 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faae8029050 con 0x565462b7d3b0 2024-04-21T08:44:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.896+0000 7faafd7d1740 10 client.18243 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.896+0000 7faafd7d1740 1 -- 172.21.15.43:0/1856686336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654628be7d0 con 0x565462b7d3b0 2024-04-21T08:44:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.896+0000 7faad3fff640 1 -- 172.21.15.43:0/1856686336 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faae4012aa0 con 0x565462b7d3b0 2024-04-21T08:44:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.896+0000 7faad3fff640 1 -- 172.21.15.43:0/1856686336 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faae800fbb0 con 0x565462b7d3b0 2024-04-21T08:44:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.896+0000 7faafd7d1740 10 client.18243 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.896+0000 7faafd7d1740 10 client.18243 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:44:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.896+0000 7faafd7d1740 20 client.18243 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.896+0000 7faafd7d1740 1 --2- 172.21.15.43:0/1856686336 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x565462b9afe0 0x565462b9d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.896+0000 7faafd7d1740 4 client.18243 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.896+0000 7faafd7d1740 1 -- 172.21.15.43:0/1856686336 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565462a81930 con 0x565462b9afe0 2024-04-21T08:44:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.896+0000 7faaf88a0640 1 --2- 172.21.15.43:0/1856686336 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x565462b9afe0 0x565462b9d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.897+0000 7faad3fff640 10 client.18243 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:44:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.897+0000 7faaf88a0640 1 --2- 172.21.15.43:0/1856686336 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x565462b9afe0 0x565462b9d3d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7ac47475846390a3 server_cookie=a345f4dbfe88105 in_seq=0 out_seq=0 2024-04-21T08:44:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.897+0000 7faad3fff640 1 -- 172.21.15.43:0/1856686336 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565462a81930 con 0x565462b9afe0 2024-04-21T08:44:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.897+0000 7faad3fff640 10 client.18243 handle_command_reply: tid=0 2024-04-21T08:44:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.897+0000 7faafd7d1740 1 client.18243 shutdown 2024-04-21T08:44:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.897+0000 7faafd7d1740 1 -- 172.21.15.43:0/1856686336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b7d3b0 msgr2=0x565462b8d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.897+0000 7faafd7d1740 1 --2- 172.21.15.43:0/1856686336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b7d3b0 0x565462b8d1d0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7faae8009fd0 tx=0x7faae8011df0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.898+0000 7faafd7d1740 1 -- 172.21.15.43:0/1856686336 shutdown_connections 2024-04-21T08:44:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.898+0000 7faafd7d1740 1 --2- 172.21.15.43:0/1856686336 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x565462b9afe0 0x565462b9d3d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.898+0000 7faafd7d1740 1 --2- 172.21.15.43:0/1856686336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565462b86e60 0x565462b8aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.898+0000 7faafd7d1740 1 --2- 172.21.15.43:0/1856686336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565462b7dd60 0x565462b8a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.898+0000 7faafd7d1740 1 --2- 172.21.15.43:0/1856686336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b7d3b0 0x565462b8d1d0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.898+0000 7faad3fff640 0 client.18243 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:44:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.898+0000 7faafd7d1740 1 -- 172.21.15.43:0/1856686336 >> 172.21.15.43:0/1856686336 conn(0x565462b77ba0 msgr2=0x565462b798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.898+0000 7faafd7d1740 1 -- 172.21.15.43:0/1856686336 shutdown_connections 2024-04-21T08:44:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.898+0000 7faafd7d1740 1 -- 172.21.15.43:0/1856686336 wait complete. 2024-04-21T08:44:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.898+0000 7faafd7d1740 20 client.18243 trim_cache size 0 max 16384 2024-04-21T08:44:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.911+0000 7faafd7d1740 1 Processor -- start 2024-04-21T08:44:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.911+0000 7faafd7d1740 1 -- start start 2024-04-21T08:44:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.911+0000 7faafd7d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565462b7d3b0 0x565462b96cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.911+0000 7faafd7d1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b7dd60 0x565462b85800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.911+0000 7faafd7d1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565462b86e60 0x565462b8cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.911+0000 7faafd7d1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41ad280 con 0x565462b7dd60 2024-04-21T08:44:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.911+0000 7faafd7d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41aedc0 con 0x565462b7d3b0 2024-04-21T08:44:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.911+0000 7faafd7d1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41ad950 con 0x565462b86e60 2024-04-21T08:44:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.911+0000 7faaf88a0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b7dd60 0x565462b85800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.911+0000 7faaf88a0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b7dd60 0x565462b85800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33686/0 (socket says 172.21.15.43:33686) 2024-04-21T08:44:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.911+0000 7faaf88a0640 1 -- 172.21.15.43:0/1277245520 learned_addr learned my addr 172.21.15.43:0/1277245520 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.911+0000 7faaf88a0640 1 -- 172.21.15.43:0/1277245520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565462b86e60 msgr2=0x565462b8cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.912+0000 7faaf88a0640 1 --2- 172.21.15.43:0/1277245520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565462b86e60 0x565462b8cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.912+0000 7faaf88a0640 1 -- 172.21.15.43:0/1277245520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565462b7d3b0 msgr2=0x565462b96cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.912+0000 7faaf88a0640 1 --2- 172.21.15.43:0/1277245520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565462b7d3b0 0x565462b96cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.912+0000 7faaf88a0640 1 -- 172.21.15.43:0/1277245520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565462b7cfe0 con 0x565462b7dd60 2024-04-21T08:44:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.912+0000 7faaf88a0640 1 --2- 172.21.15.43:0/1277245520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b7dd60 0x565462b85800 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7faae4011cb0 tx=0x7faae4009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b37258a165b677 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.912+0000 7faada7fc640 1 -- 172.21.15.43:0/1277245520 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae400f440 con 0x565462b7dd60 2024-04-21T08:44:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.912+0000 7faada7fc640 1 -- 172.21.15.43:0/1277245520 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faae4029070 con 0x565462b7dd60 2024-04-21T08:44:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.912+0000 7faada7fc640 1 -- 172.21.15.43:0/1277245520 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4012980 con 0x565462b7dd60 2024-04-21T08:44:59.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.912+0000 7faafd7d1740 1 -- 172.21.15.43:0/1277245520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b7dd60 msgr2=0x565462b85800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.912+0000 7faafd7d1740 1 --2- 172.21.15.43:0/1277245520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b7dd60 0x565462b85800 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7faae4011cb0 tx=0x7faae4009530 comp rx=0 tx=0).stop 2024-04-21T08:44:59.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.912+0000 7faafd7d1740 1 -- 172.21.15.43:0/1277245520 shutdown_connections 2024-04-21T08:44:59.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.912+0000 7faafd7d1740 1 --2- 172.21.15.43:0/1277245520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565462b86e60 0x565462b8cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.912+0000 7faafd7d1740 1 --2- 172.21.15.43:0/1277245520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b7dd60 0x565462b85800 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.912+0000 7faafd7d1740 1 --2- 172.21.15.43:0/1277245520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565462b7d3b0 0x565462b96cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.912+0000 7faafd7d1740 1 -- 172.21.15.43:0/1277245520 >> 172.21.15.43:0/1277245520 conn(0x565462b77ba0 msgr2=0x565462b79830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.913+0000 7faafd7d1740 1 -- 172.21.15.43:0/1277245520 shutdown_connections 2024-04-21T08:44:59.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.913+0000 7faafd7d1740 1 -- 172.21.15.43:0/1277245520 wait complete. 2024-04-21T08:44:59.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.913+0000 7faafd7d1740 1 -- start start 2024-04-21T08:44:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.913+0000 7faafd7d1740 1 Processor -- start 2024-04-21T08:44:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.913+0000 7faafd7d1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565462b7d3b0 0x565462b8c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.913+0000 7faafd7d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565462b7dd60 0x565462b8cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.913+0000 7faafd7d1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b86e60 0x565462c00510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.913+0000 7faafd7d1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41ae6f0 con 0x565462b86e60 2024-04-21T08:44:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.913+0000 7faafd7d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41ae020 con 0x565462b7dd60 2024-04-21T08:44:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.913+0000 7faafd7d1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41ab260 con 0x565462b7d3b0 2024-04-21T08:44:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.913+0000 7faafd7d1740 10 client.0 _ll_register_callbacks cb 0x7faae80115a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.913+0000 7faaf3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b86e60 0x565462c00510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.913+0000 7faaf3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b86e60 0x565462c00510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33698/0 (socket says 172.21.15.43:33698) 2024-04-21T08:44:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.913+0000 7faaf3fff640 1 -- 172.21.15.43:0/2326343802 learned_addr learned my addr 172.21.15.43:0/2326343802 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:44:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.914+0000 7faaf3fff640 1 --2- 172.21.15.43:0/2326343802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565462b7d3b0 0x565462b8c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.914+0000 7faaf88a0640 1 --2- 172.21.15.43:0/2326343802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565462b7dd60 0x565462b8cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.914+0000 7faaf3fff640 1 -- 172.21.15.43:0/2326343802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565462b7d3b0 msgr2=0x565462b8c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.914+0000 7faaf3fff640 1 --2- 172.21.15.43:0/2326343802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565462b7d3b0 0x565462b8c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.914+0000 7faaf3fff640 1 -- 172.21.15.43:0/2326343802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565462b7dd60 msgr2=0x565462b8cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.914+0000 7faaf3fff640 1 --2- 172.21.15.43:0/2326343802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565462b7dd60 0x565462b8cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.914+0000 7faaf3fff640 1 -- 172.21.15.43:0/2326343802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565462c00a50 con 0x565462b86e60 2024-04-21T08:44:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.914+0000 7faaf88a0640 1 --2- 172.21.15.43:0/2326343802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565462b7dd60 0x565462b8cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.914+0000 7faaf3fff640 1 --2- 172.21.15.43:0/2326343802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b86e60 0x565462c00510 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7faae8005fd0 tx=0x7faae8011650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f268576376e995c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.914+0000 7faad3fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:44:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.914+0000 7faad3fff640 1 -- 172.21.15.43:0/2326343802 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae8032060 con 0x565462b86e60 2024-04-21T08:44:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.914+0000 7faad3fff640 1 -- 172.21.15.43:0/2326343802 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7faae800eb00 con 0x565462b86e60 2024-04-21T08:44:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.914+0000 7faafd7d1740 1 -- 172.21.15.43:0/2326343802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faae4029070 con 0x565462b86e60 2024-04-21T08:44:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.914+0000 7faad3fff640 1 -- 172.21.15.43:0/2326343802 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faae8039050 con 0x565462b86e60 2024-04-21T08:44:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.914+0000 7faafd7d1740 10 client.18249 fetch_fsmap learned FSMap version 28 2024-04-21T08:44:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.914+0000 7faafd7d1740 1 -- 172.21.15.43:0/2326343802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565462c01c80 con 0x565462b86e60 2024-04-21T08:44:59.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.914+0000 7faad3fff640 1 -- 172.21.15.43:0/2326343802 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faae4029070 con 0x565462b86e60 2024-04-21T08:44:59.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.915+0000 7faad3fff640 1 -- 172.21.15.43:0/2326343802 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faae800dbb0 con 0x565462b86e60 2024-04-21T08:44:59.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.915+0000 7faafd7d1740 10 client.18249 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:44:59.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.915+0000 7faafd7d1740 10 client.18249 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:44:59.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.915+0000 7faafd7d1740 20 client.18249 populate_metadata read hostname 'smithi043' 2024-04-21T08:44:59.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.915+0000 7faafd7d1740 1 --2- 172.21.15.43:0/2326343802 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x565462c02400 0x565462c04820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.915+0000 7faafd7d1740 4 client.18249 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:44:59.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.915+0000 7faafd7d1740 1 -- 172.21.15.43:0/2326343802 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565462aefc10 con 0x565462c02400 2024-04-21T08:44:59.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.915+0000 7faaf88a0640 1 --2- 172.21.15.43:0/2326343802 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x565462c02400 0x565462c04820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.915+0000 7faad3fff640 10 client.18249 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:44:59.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.915+0000 7faaf88a0640 1 --2- 172.21.15.43:0/2326343802 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x565462c02400 0x565462c04820 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=733147f20411605b server_cookie=3ee51d3eec327506 in_seq=0 out_seq=0 2024-04-21T08:44:59.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.915+0000 7faad3fff640 1 -- 172.21.15.43:0/2326343802 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565462aefc10 con 0x565462c02400 2024-04-21T08:44:59.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.915+0000 7faad3fff640 10 client.18249 handle_command_reply: tid=0 2024-04-21T08:44:59.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.915+0000 7faafd7d1740 1 client.18249 shutdown 2024-04-21T08:44:59.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.916+0000 7faafd7d1740 1 -- 172.21.15.43:0/2326343802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b86e60 msgr2=0x565462c00510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.916+0000 7faafd7d1740 1 --2- 172.21.15.43:0/2326343802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b86e60 0x565462c00510 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7faae8005fd0 tx=0x7faae8011650 comp rx=0 tx=0).stop 2024-04-21T08:44:59.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.916+0000 7faafd7d1740 1 -- 172.21.15.43:0/2326343802 shutdown_connections 2024-04-21T08:44:59.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.916+0000 7faafd7d1740 1 --2- 172.21.15.43:0/2326343802 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x565462c02400 0x565462c04820 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.916+0000 7faafd7d1740 1 --2- 172.21.15.43:0/2326343802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565462b86e60 0x565462c00510 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.916+0000 7faad3fff640 0 client.18249 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:44:59.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.916+0000 7faafd7d1740 1 --2- 172.21.15.43:0/2326343802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565462b7dd60 0x565462b8cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.916+0000 7faafd7d1740 1 --2- 172.21.15.43:0/2326343802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565462b7d3b0 0x565462b8c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.916+0000 7faafd7d1740 1 -- 172.21.15.43:0/2326343802 >> 172.21.15.43:0/2326343802 conn(0x565462b77ba0 msgr2=0x565462b76370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.916+0000 7faafd7d1740 1 -- 172.21.15.43:0/2326343802 shutdown_connections 2024-04-21T08:44:59.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.916+0000 7faafd7d1740 1 -- 172.21.15.43:0/2326343802 wait complete. 2024-04-21T08:44:59.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.916+0000 7faafd7d1740 20 client.18249 trim_cache size 0 max 16384 2024-04-21T08:44:59.925 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:44:59.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.916+0000 7faafab04640 1 -- 172.21.15.43:0/2368594595 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faadc0a3da0 msgr2=0x7faadc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.916+0000 7faafab04640 1 --2- 172.21.15.43:0/2368594595 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faadc0a3da0 0x7faadc0a6260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7faaec0f0030 tx=0x7faaec02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.916+0000 7faafab04640 1 -- 172.21.15.43:0/2368594595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaf419cf80 msgr2=0x7faaf419d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.916+0000 7faafab04640 1 --2- 172.21.15.43:0/2368594595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaf419cf80 0x7faaf419d430 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7faaec005e30 tx=0x7faaec0346d0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.916+0000 7faafab04640 1 -- 172.21.15.43:0/2368594595 shutdown_connections 2024-04-21T08:44:59.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.916+0000 7faafab04640 1 --2- 172.21.15.43:0/2368594595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaf419cf80 0x7faaf419d430 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.916+0000 7faafab04640 1 --2- 172.21.15.43:0/2368594595 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faadc0a3da0 0x7faadc0a6260 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.916+0000 7faafab04640 1 --2- 172.21.15.43:0/2368594595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaf4106160 0x7faaf4198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.916+0000 7faafab04640 1 --2- 172.21.15.43:0/2368594595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf4101090 0x7faaf41986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.916+0000 7faafab04640 1 -- 172.21.15.43:0/2368594595 >> 172.21.15.43:0/2368594595 conn(0x7faaf4077f50 msgr2=0x7faaf40ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.916+0000 7faafab04640 1 -- 172.21.15.43:0/2368594595 shutdown_connections 2024-04-21T08:44:59.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:44:59.917+0000 7faafab04640 1 -- 172.21.15.43:0/2368594595 wait complete. 2024-04-21T08:44:59.927 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:45:00.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.033+0000 7fdc54f15640 1 -- 172.21.15.43:0/251255643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc5006af10 msgr2=0x7fdc5010edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.033+0000 7fdc54f15640 1 --2- 172.21.15.43:0/251255643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc5006af10 0x7fdc5010edf0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fdc48009cd0 tx=0x7fdc4802c180 comp rx=0 tx=0).stop 2024-04-21T08:45:00.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.033+0000 7fdc54f15640 1 -- 172.21.15.43:0/251255643 shutdown_connections 2024-04-21T08:45:00.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.033+0000 7fdc54f15640 1 --2- 172.21.15.43:0/251255643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc5006af10 0x7fdc5010edf0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.033+0000 7fdc54f15640 1 --2- 172.21.15.43:0/251255643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc5006a550 0x7fdc5006a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.033+0000 7fdc54f15640 1 --2- 172.21.15.43:0/251255643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc50108150 0x7fdc50108530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.033+0000 7fdc54f15640 1 -- 172.21.15.43:0/251255643 >> 172.21.15.43:0/251255643 conn(0x7fdc50076c50 msgr2=0x7fdc50077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:00.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.033+0000 7fdc54f15640 1 -- 172.21.15.43:0/251255643 shutdown_connections 2024-04-21T08:45:00.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.033+0000 7fdc54f15640 1 -- 172.21.15.43:0/251255643 wait complete. 2024-04-21T08:45:00.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.034+0000 7fdc54f15640 1 Processor -- start 2024-04-21T08:45:00.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.034+0000 7fdc54f15640 1 -- start start 2024-04-21T08:45:00.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.034+0000 7fdc54f15640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc5006a550 0x7fdc50198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.034+0000 7fdc54f15640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc5006af10 0x7fdc50198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.034+0000 7fdc54f15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc50108150 0x7fdc5019d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.034+0000 7fdc54f15640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdc501121a0 con 0x7fdc5006af10 2024-04-21T08:45:00.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.034+0000 7fdc54f15640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdc50112020 con 0x7fdc50108150 2024-04-21T08:45:00.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.034+0000 7fdc54f15640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdc50112320 con 0x7fdc5006a550 2024-04-21T08:45:00.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.034+0000 7fdc47fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc5006af10 0x7fdc50198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.034+0000 7fdc47fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc5006af10 0x7fdc50198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33720/0 (socket says 172.21.15.43:33720) 2024-04-21T08:45:00.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.034+0000 7fdc47fff640 1 -- 172.21.15.43:0/605348469 learned_addr learned my addr 172.21.15.43:0/605348469 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:00.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.034+0000 7fdc47fff640 1 -- 172.21.15.43:0/605348469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc5006a550 msgr2=0x7fdc50198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.034+0000 7fdc47fff640 1 --2- 172.21.15.43:0/605348469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc5006a550 0x7fdc50198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.034+0000 7fdc47fff640 1 -- 172.21.15.43:0/605348469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc50108150 msgr2=0x7fdc5019d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.034+0000 7fdc47fff640 1 --2- 172.21.15.43:0/605348469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc50108150 0x7fdc5019d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.034+0000 7fdc47fff640 1 -- 172.21.15.43:0/605348469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5019d730 con 0x7fdc5006af10 2024-04-21T08:45:00.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.035+0000 7fdc47fff640 1 --2- 172.21.15.43:0/605348469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc5006af10 0x7fdc50198c80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fdc3800d950 tx=0x7fdc3800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:00.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.035+0000 7fdc477fe640 1 -- 172.21.15.43:0/605348469 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38017070 con 0x7fdc5006af10 2024-04-21T08:45:00.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.035+0000 7fdc54f15640 1 -- 172.21.15.43:0/605348469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc5019da20 con 0x7fdc5006af10 2024-04-21T08:45:00.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.035+0000 7fdc477fe640 1 -- 172.21.15.43:0/605348469 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc38009c30 con 0x7fdc5006af10 2024-04-21T08:45:00.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.035+0000 7fdc54f15640 1 -- 172.21.15.43:0/605348469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc501a52b0 con 0x7fdc5006af10 2024-04-21T08:45:00.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.035+0000 7fdc477fe640 1 -- 172.21.15.43:0/605348469 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3800f620 con 0x7fdc5006af10 2024-04-21T08:45:00.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.038+0000 7fdc477fe640 1 -- 172.21.15.43:0/605348469 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdc38010460 con 0x7fdc5006af10 2024-04-21T08:45:00.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.038+0000 7fdc57be2740 1 Processor -- start 2024-04-21T08:45:00.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.038+0000 7fdc57be2740 1 -- start start 2024-04-21T08:45:00.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.038+0000 7fdc57be2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd14214d0 0x561dd14218b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.038+0000 7fdc57be2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd1421e80 0x561dd142a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.038+0000 7fdc57be2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561dd142aa00 0x561dd142ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.038+0000 7fdc57be2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561dd13861f0 con 0x561dd1421e80 2024-04-21T08:45:00.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.038+0000 7fdc57be2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561dd13234b0 con 0x561dd14214d0 2024-04-21T08:45:00.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.038+0000 7fdc57be2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561dd13821b0 con 0x561dd142aa00 2024-04-21T08:45:00.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.038+0000 7fdc4e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd1421e80 0x561dd142a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.038+0000 7fdc4e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd1421e80 0x561dd142a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33726/0 (socket says 172.21.15.43:33726) 2024-04-21T08:45:00.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.038+0000 7fdc4e59c640 1 -- 172.21.15.43:0/1286225800 learned_addr learned my addr 172.21.15.43:0/1286225800 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:00.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.038+0000 7fdc4ed9d640 1 --2- 172.21.15.43:0/1286225800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd14214d0 0x561dd14218b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.038+0000 7fdc47fff640 1 --2- 172.21.15.43:0/1286225800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561dd142aa00 0x561dd142ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.038+0000 7fdc477fe640 1 --2- 172.21.15.43:0/605348469 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdc3c0a8130 0x7fdc3c0aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.038+0000 7fdc4e59c640 1 -- 172.21.15.43:0/1286225800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561dd142aa00 msgr2=0x561dd142ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:00.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.039+0000 7fdc4e59c640 1 --2- 172.21.15.43:0/1286225800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561dd142aa00 0x561dd142ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.039+0000 7fdc4e59c640 1 -- 172.21.15.43:0/1286225800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd14214d0 msgr2=0x561dd14218b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.039+0000 7fdc4e59c640 1 --2- 172.21.15.43:0/1286225800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd14214d0 0x561dd14218b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.039+0000 7fdc4e59c640 1 -- 172.21.15.43:0/1286225800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dd13151b0 con 0x561dd1421e80 2024-04-21T08:45:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.039+0000 7fdc4ed9d640 1 --2- 172.21.15.43:0/1286225800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd14214d0 0x561dd14218b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.039+0000 7fdc47fff640 1 --2- 172.21.15.43:0/1286225800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561dd142aa00 0x561dd142ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.039+0000 7fdc4ed9d640 1 --2- 172.21.15.43:0/605348469 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdc3c0a8130 0x7fdc3c0aa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.039+0000 7fdc477fe640 1 -- 172.21.15.43:0/605348469 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fdc380d08e0 con 0x7fdc5006af10 2024-04-21T08:45:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.039+0000 7fdc4e59c640 1 --2- 172.21.15.43:0/1286225800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd1421e80 0x561dd142a430 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000ba90 tx=0x7fdc4000bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26bcc4bccb3dff05 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.039+0000 7fdc44ff9640 1 -- 172.21.15.43:0/1286225800 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40016070 con 0x561dd1421e80 2024-04-21T08:45:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.039+0000 7fdc44ff9640 1 -- 172.21.15.43:0/1286225800 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc4000dca0 con 0x561dd1421e80 2024-04-21T08:45:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.039+0000 7fdc44ff9640 1 -- 172.21.15.43:0/1286225800 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4000f6b0 con 0x561dd1421e80 2024-04-21T08:45:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.039+0000 7fdc4ed9d640 1 --2- 172.21.15.43:0/605348469 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdc3c0a8130 0x7fdc3c0aa5f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdc48005e30 tx=0x7fdc48004a60 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.040+0000 7fdc57be2740 1 -- 172.21.15.43:0/1286225800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd1421e80 msgr2=0x561dd142a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.040+0000 7fdc57be2740 1 --2- 172.21.15.43:0/1286225800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd1421e80 0x561dd142a430 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000ba90 tx=0x7fdc4000bf50 comp rx=0 tx=0).stop 2024-04-21T08:45:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.040+0000 7fdc57be2740 1 -- 172.21.15.43:0/1286225800 shutdown_connections 2024-04-21T08:45:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.040+0000 7fdc57be2740 1 --2- 172.21.15.43:0/1286225800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561dd142aa00 0x561dd142ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.040+0000 7fdc57be2740 1 --2- 172.21.15.43:0/1286225800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd1421e80 0x561dd142a430 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.040+0000 7fdc57be2740 1 --2- 172.21.15.43:0/1286225800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd14214d0 0x561dd14218b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.040+0000 7fdc57be2740 1 -- 172.21.15.43:0/1286225800 >> 172.21.15.43:0/1286225800 conn(0x561dd141bba0 msgr2=0x561dd141bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.040+0000 7fdc57be2740 1 -- 172.21.15.43:0/1286225800 shutdown_connections 2024-04-21T08:45:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.040+0000 7fdc57be2740 1 -- 172.21.15.43:0/1286225800 wait complete. 2024-04-21T08:45:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.040+0000 7fdc57be2740 1 -- start start 2024-04-21T08:45:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.040+0000 7fdc57be2740 1 Processor -- start 2024-04-21T08:45:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.040+0000 7fdc57be2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd14214d0 0x561dd142a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.040+0000 7fdc57be2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561dd1421e80 0x561dd14275f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.041+0000 7fdc57be2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd142aa00 0x561dd1427b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.041+0000 7fdc57be2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561dd142d970 con 0x561dd142aa00 2024-04-21T08:45:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.041+0000 7fdc57be2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561dd133f500 con 0x561dd14214d0 2024-04-21T08:45:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.041+0000 7fdc57be2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561dd142daf0 con 0x561dd1421e80 2024-04-21T08:45:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.041+0000 7fdc57be2740 10 client.0 _ll_register_callbacks cb 0x561dd130db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.041+0000 7fdc4ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd142aa00 0x561dd1427b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.041+0000 7fdc4ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd142aa00 0x561dd1427b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33742/0 (socket says 172.21.15.43:33742) 2024-04-21T08:45:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.041+0000 7fdc4ed9d640 1 -- 172.21.15.43:0/481286454 learned_addr learned my addr 172.21.15.43:0/481286454 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.041+0000 7fdc4e59c640 1 --2- 172.21.15.43:0/481286454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd14214d0 0x561dd142a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.041+0000 7fdc47fff640 1 --2- 172.21.15.43:0/481286454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561dd1421e80 0x561dd14275f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.041+0000 7fdc4ed9d640 1 -- 172.21.15.43:0/481286454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561dd1421e80 msgr2=0x561dd14275f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:00.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.041+0000 7fdc4ed9d640 1 --2- 172.21.15.43:0/481286454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561dd1421e80 0x561dd14275f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.041+0000 7fdc4ed9d640 1 -- 172.21.15.43:0/481286454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd14214d0 msgr2=0x561dd142a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:00.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.041+0000 7fdc4ed9d640 1 --2- 172.21.15.43:0/481286454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd14214d0 0x561dd142a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.041+0000 7fdc4ed9d640 1 -- 172.21.15.43:0/481286454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dd10983f0 con 0x561dd142aa00 2024-04-21T08:45:00.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.041+0000 7fdc4ed9d640 1 --2- 172.21.15.43:0/481286454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd142aa00 0x561dd1427b30 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fdc48036000 tx=0x7fdc480095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb912c7561d4e66 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:00.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.041+0000 7fdc35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:00.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.041+0000 7fdc35ffb640 1 -- 172.21.15.43:0/481286454 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4804f060 con 0x561dd142aa00 2024-04-21T08:45:00.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.041+0000 7fdc35ffb640 1 -- 172.21.15.43:0/481286454 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fdc48032970 con 0x561dd142aa00 2024-04-21T08:45:00.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.041+0000 7fdc57be2740 1 -- 172.21.15.43:0/481286454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc4000dca0 con 0x561dd142aa00 2024-04-21T08:45:00.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.041+0000 7fdc35ffb640 1 -- 172.21.15.43:0/481286454 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdc48056050 con 0x561dd142aa00 2024-04-21T08:45:00.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.041+0000 7fdc57be2740 10 client.18261 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:00.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.041+0000 7fdc57be2740 1 -- 172.21.15.43:0/481286454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561dd11627d0 con 0x561dd142aa00 2024-04-21T08:45:00.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.042+0000 7fdc35ffb640 1 -- 172.21.15.43:0/481286454 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc4000dca0 con 0x561dd142aa00 2024-04-21T08:45:00.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.042+0000 7fdc35ffb640 1 -- 172.21.15.43:0/481286454 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdc4803fac0 con 0x561dd142aa00 2024-04-21T08:45:00.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.042+0000 7fdc57be2740 10 client.18261 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:00.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.042+0000 7fdc57be2740 10 client.18261 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:45:00.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.042+0000 7fdc57be2740 20 client.18261 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:00.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.042+0000 7fdc57be2740 1 --2- 172.21.15.43:0/481286454 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x561dd143aba0 0x561dd143cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.042+0000 7fdc57be2740 4 client.18261 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:00.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.042+0000 7fdc57be2740 1 -- 172.21.15.43:0/481286454 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dd1325930 con 0x561dd143aba0 2024-04-21T08:45:00.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.042+0000 7fdc4e59c640 1 --2- 172.21.15.43:0/481286454 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x561dd143aba0 0x561dd143cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.043+0000 7fdc4e59c640 1 --2- 172.21.15.43:0/481286454 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x561dd143aba0 0x561dd143cfd0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7aedc05cb22af839 server_cookie=4f89667ecd2bf5e5 in_seq=0 out_seq=0 2024-04-21T08:45:00.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.043+0000 7fdc35ffb640 10 client.18261 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:45:00.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.043+0000 7fdc35ffb640 1 -- 172.21.15.43:0/481286454 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561dd1325930 con 0x561dd143aba0 2024-04-21T08:45:00.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.043+0000 7fdc35ffb640 10 client.18261 handle_command_reply: tid=0 2024-04-21T08:45:00.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.043+0000 7fdc57be2740 1 client.18261 shutdown 2024-04-21T08:45:00.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.043+0000 7fdc57be2740 1 -- 172.21.15.43:0/481286454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd142aa00 msgr2=0x561dd1427b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.043+0000 7fdc57be2740 1 --2- 172.21.15.43:0/481286454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd142aa00 0x561dd1427b30 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fdc48036000 tx=0x7fdc480095c0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.043+0000 7fdc57be2740 1 -- 172.21.15.43:0/481286454 shutdown_connections 2024-04-21T08:45:00.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.043+0000 7fdc57be2740 1 --2- 172.21.15.43:0/481286454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd142aa00 0x561dd1427b30 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.043+0000 7fdc57be2740 1 --2- 172.21.15.43:0/481286454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561dd1421e80 0x561dd14275f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.043+0000 7fdc57be2740 1 --2- 172.21.15.43:0/481286454 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x561dd143aba0 0x561dd143cfd0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.043+0000 7fdc57be2740 1 --2- 172.21.15.43:0/481286454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd14214d0 0x561dd142a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.043+0000 7fdc35ffb640 0 client.18261 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:45:00.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.043+0000 7fdc57be2740 1 -- 172.21.15.43:0/481286454 >> 172.21.15.43:0/481286454 conn(0x561dd141bba0 msgr2=0x561dd141d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:00.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.044+0000 7fdc57be2740 1 -- 172.21.15.43:0/481286454 shutdown_connections 2024-04-21T08:45:00.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.044+0000 7fdc57be2740 1 -- 172.21.15.43:0/481286454 wait complete. 2024-04-21T08:45:00.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.044+0000 7fdc57be2740 20 client.18261 trim_cache size 0 max 16384 2024-04-21T08:45:00.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.057+0000 7fdc57be2740 1 Processor -- start 2024-04-21T08:45:00.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.057+0000 7fdc57be2740 1 -- start start 2024-04-21T08:45:00.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.057+0000 7fdc57be2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd14214d0 0x561dd1436860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.057+0000 7fdc57be2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd1421e80 0x561dd1429d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.057+0000 7fdc57be2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561dd142aa00 0x561dd142a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.057+0000 7fdc57be2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdc501ad2e0 con 0x561dd1421e80 2024-04-21T08:45:00.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.057+0000 7fdc57be2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdc501aee20 con 0x561dd14214d0 2024-04-21T08:45:00.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.057+0000 7fdc57be2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdc501ad9b0 con 0x561dd142aa00 2024-04-21T08:45:00.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.057+0000 7fdc47fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd1421e80 0x561dd1429d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.057+0000 7fdc47fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd1421e80 0x561dd1429d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33748/0 (socket says 172.21.15.43:33748) 2024-04-21T08:45:00.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.057+0000 7fdc47fff640 1 -- 172.21.15.43:0/936183871 learned_addr learned my addr 172.21.15.43:0/936183871 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:00.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.057+0000 7fdc47fff640 1 -- 172.21.15.43:0/936183871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561dd142aa00 msgr2=0x561dd142a260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:45:00.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.057+0000 7fdc47fff640 1 --2- 172.21.15.43:0/936183871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561dd142aa00 0x561dd142a260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.057+0000 7fdc47fff640 1 -- 172.21.15.43:0/936183871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd14214d0 msgr2=0x561dd1436860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:00.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.057+0000 7fdc47fff640 1 --2- 172.21.15.43:0/936183871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd14214d0 0x561dd1436860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.057+0000 7fdc47fff640 1 -- 172.21.15.43:0/936183871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dd142d210 con 0x561dd1421e80 2024-04-21T08:45:00.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.057+0000 7fdc47fff640 1 --2- 172.21.15.43:0/936183871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd1421e80 0x561dd1429d20 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fdc38098e80 tx=0x7fdc38000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7def10663c80fa8d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:00.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.058+0000 7fdc37fff640 1 -- 172.21.15.43:0/936183871 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38020040 con 0x561dd1421e80 2024-04-21T08:45:00.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.058+0000 7fdc37fff640 1 -- 172.21.15.43:0/936183871 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc3809cbb0 con 0x561dd1421e80 2024-04-21T08:45:00.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.058+0000 7fdc37fff640 1 -- 172.21.15.43:0/936183871 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38098b20 con 0x561dd1421e80 2024-04-21T08:45:00.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.058+0000 7fdc57be2740 1 -- 172.21.15.43:0/936183871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd1421e80 msgr2=0x561dd1429d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.058+0000 7fdc57be2740 1 --2- 172.21.15.43:0/936183871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd1421e80 0x561dd1429d20 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fdc38098e80 tx=0x7fdc38000bc0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.058+0000 7fdc57be2740 1 -- 172.21.15.43:0/936183871 shutdown_connections 2024-04-21T08:45:00.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.058+0000 7fdc57be2740 1 --2- 172.21.15.43:0/936183871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561dd142aa00 0x561dd142a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.058+0000 7fdc57be2740 1 --2- 172.21.15.43:0/936183871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd1421e80 0x561dd1429d20 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.058+0000 7fdc57be2740 1 --2- 172.21.15.43:0/936183871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd14214d0 0x561dd1436860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.058+0000 7fdc57be2740 1 -- 172.21.15.43:0/936183871 >> 172.21.15.43:0/936183871 conn(0x561dd141bba0 msgr2=0x561dd141ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:00.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.058+0000 7fdc57be2740 1 -- 172.21.15.43:0/936183871 shutdown_connections 2024-04-21T08:45:00.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.058+0000 7fdc57be2740 1 -- 172.21.15.43:0/936183871 wait complete. 2024-04-21T08:45:00.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.059+0000 7fdc57be2740 1 -- start start 2024-04-21T08:45:00.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.059+0000 7fdc57be2740 1 Processor -- start 2024-04-21T08:45:00.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.059+0000 7fdc57be2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd14214d0 0x561dd1424330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.059+0000 7fdc57be2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd1421e80 0x561dd1424870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.059+0000 7fdc57be2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561dd142aa00 0x561dd1424db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.059+0000 7fdc57be2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdc501ae750 con 0x561dd14214d0 2024-04-21T08:45:00.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.059+0000 7fdc57be2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdc501ae080 con 0x561dd1421e80 2024-04-21T08:45:00.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.059+0000 7fdc57be2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdc501ab2c0 con 0x561dd142aa00 2024-04-21T08:45:00.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.059+0000 7fdc57be2740 10 client.0 _ll_register_callbacks cb 0x7fdc48003db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:00.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.059+0000 7fdc4e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd14214d0 0x561dd1424330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.059+0000 7fdc4e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd14214d0 0x561dd1424330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33760/0 (socket says 172.21.15.43:33760) 2024-04-21T08:45:00.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.059+0000 7fdc4e59c640 1 -- 172.21.15.43:0/1281975006 learned_addr learned my addr 172.21.15.43:0/1281975006 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:00.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.059+0000 7fdc47fff640 1 --2- 172.21.15.43:0/1281975006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd1421e80 0x561dd1424870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.059+0000 7fdc4ed9d640 1 --2- 172.21.15.43:0/1281975006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561dd142aa00 0x561dd1424db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.059+0000 7fdc4e59c640 1 -- 172.21.15.43:0/1281975006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561dd142aa00 msgr2=0x561dd1424db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:00.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.059+0000 7fdc4e59c640 1 --2- 172.21.15.43:0/1281975006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561dd142aa00 0x561dd1424db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.059+0000 7fdc4e59c640 1 -- 172.21.15.43:0/1281975006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd1421e80 msgr2=0x561dd1424870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:00.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.059+0000 7fdc4e59c640 1 --2- 172.21.15.43:0/1281975006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd1421e80 0x561dd1424870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.059+0000 7fdc4e59c640 1 -- 172.21.15.43:0/1281975006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dd14a01e0 con 0x561dd14214d0 2024-04-21T08:45:00.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.059+0000 7fdc4e59c640 1 --2- 172.21.15.43:0/1281975006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd14214d0 0x561dd1424330 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fdc40018000 tx=0x7fdc40005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb734f0810db65c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:00.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.059+0000 7fdc35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:00.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.060+0000 7fdc35ffb640 1 -- 172.21.15.43:0/1281975006 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40016040 con 0x561dd14214d0 2024-04-21T08:45:00.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.060+0000 7fdc57be2740 1 -- 172.21.15.43:0/1281975006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc3809cbb0 con 0x561dd14214d0 2024-04-21T08:45:00.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.060+0000 7fdc35ffb640 1 -- 172.21.15.43:0/1281975006 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fdc400253b0 con 0x561dd14214d0 2024-04-21T08:45:00.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.060+0000 7fdc35ffb640 1 -- 172.21.15.43:0/1281975006 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdc40033050 con 0x561dd14214d0 2024-04-21T08:45:00.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.060+0000 7fdc57be2740 10 client.18267 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:00.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.060+0000 7fdc57be2740 1 -- 172.21.15.43:0/1281975006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561dd14a1540 con 0x561dd14214d0 2024-04-21T08:45:00.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.060+0000 7fdc35ffb640 1 -- 172.21.15.43:0/1281975006 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc3809cbb0 con 0x561dd14214d0 2024-04-21T08:45:00.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.060+0000 7fdc35ffb640 1 -- 172.21.15.43:0/1281975006 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdc40010c20 con 0x561dd14214d0 2024-04-21T08:45:00.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.060+0000 7fdc57be2740 10 client.18267 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:00.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.060+0000 7fdc57be2740 10 client.18267 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:45:00.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.060+0000 7fdc57be2740 20 client.18267 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.060+0000 7fdc57be2740 1 --2- 172.21.15.43:0/1281975006 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x561dd14a19e0 0x561dd14a3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.060+0000 7fdc57be2740 4 client.18267 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:45:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.060+0000 7fdc57be2740 1 -- 172.21.15.43:0/1281975006 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561dd1427ec0 con 0x561dd14a19e0 2024-04-21T08:45:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.061+0000 7fdc47fff640 1 --2- 172.21.15.43:0/1281975006 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x561dd14a19e0 0x561dd14a3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.061+0000 7fdc47fff640 1 --2- 172.21.15.43:0/1281975006 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x561dd14a19e0 0x561dd14a3e00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b0c7460e92766e36 server_cookie=1ea6be112f7646a7 in_seq=0 out_seq=0 2024-04-21T08:45:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.061+0000 7fdc35ffb640 10 client.18267 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:45:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc35ffb640 1 -- 172.21.15.43:0/1281975006 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561dd1427ec0 con 0x561dd14a19e0 2024-04-21T08:45:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc35ffb640 10 client.18267 handle_command_reply: tid=0 2024-04-21T08:45:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc57be2740 1 client.18267 shutdown 2024-04-21T08:45:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc57be2740 1 -- 172.21.15.43:0/1281975006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd14214d0 msgr2=0x561dd1424330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc57be2740 1 --2- 172.21.15.43:0/1281975006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd14214d0 0x561dd1424330 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fdc40018000 tx=0x7fdc40005ab0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc57be2740 1 -- 172.21.15.43:0/1281975006 shutdown_connections 2024-04-21T08:45:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc57be2740 1 --2- 172.21.15.43:0/1281975006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561dd142aa00 0x561dd1424db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc57be2740 1 --2- 172.21.15.43:0/1281975006 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x561dd14a19e0 0x561dd14a3e00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc57be2740 1 --2- 172.21.15.43:0/1281975006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd1421e80 0x561dd1424870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc57be2740 1 --2- 172.21.15.43:0/1281975006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd14214d0 0x561dd1424330 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc35ffb640 0 client.18267 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:45:00.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc57be2740 1 -- 172.21.15.43:0/1281975006 >> 172.21.15.43:0/1281975006 conn(0x561dd141bba0 msgr2=0x561dd14222b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:00.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc57be2740 1 -- 172.21.15.43:0/1281975006 shutdown_connections 2024-04-21T08:45:00.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc57be2740 1 -- 172.21.15.43:0/1281975006 wait complete. 2024-04-21T08:45:00.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc57be2740 20 client.18267 trim_cache size 0 max 16384 2024-04-21T08:45:00.076 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:45:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc54f15640 1 -- 172.21.15.43:0/605348469 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdc3c0a8130 msgr2=0x7fdc3c0aa5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc54f15640 1 --2- 172.21.15.43:0/605348469 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdc3c0a8130 0x7fdc3c0aa5f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdc48005e30 tx=0x7fdc48004a60 comp rx=0 tx=0).stop 2024-04-21T08:45:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc54f15640 1 -- 172.21.15.43:0/605348469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc5006af10 msgr2=0x7fdc50198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc54f15640 1 --2- 172.21.15.43:0/605348469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc5006af10 0x7fdc50198c80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fdc3800d950 tx=0x7fdc3800de10 comp rx=0 tx=0).stop 2024-04-21T08:45:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc54f15640 1 -- 172.21.15.43:0/605348469 shutdown_connections 2024-04-21T08:45:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc54f15640 1 --2- 172.21.15.43:0/605348469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc50108150 0x7fdc5019d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc54f15640 1 --2- 172.21.15.43:0/605348469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc5006af10 0x7fdc50198c80 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc54f15640 1 --2- 172.21.15.43:0/605348469 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdc3c0a8130 0x7fdc3c0aa5f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc54f15640 1 --2- 172.21.15.43:0/605348469 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc5006a550 0x7fdc50198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc54f15640 1 -- 172.21.15.43:0/605348469 >> 172.21.15.43:0/605348469 conn(0x7fdc50076c50 msgr2=0x7fdc500ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc54f15640 1 -- 172.21.15.43:0/605348469 shutdown_connections 2024-04-21T08:45:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.062+0000 7fdc54f15640 1 -- 172.21.15.43:0/605348469 wait complete. 2024-04-21T08:45:00.078 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:45:00.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.183+0000 7f85aeb52640 1 -- 172.21.15.43:0/1129456056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a8106170 msgr2=0x7f85a810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.183+0000 7f85aeb52640 1 --2- 172.21.15.43:0/1129456056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a8106170 0x7f85a810dd40 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009ca0 tx=0x7f85a002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.183+0000 7f85aeb52640 1 -- 172.21.15.43:0/1129456056 shutdown_connections 2024-04-21T08:45:00.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.183+0000 7f85aeb52640 1 --2- 172.21.15.43:0/1129456056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a8106170 0x7f85a810dd40 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.183+0000 7f85aeb52640 1 --2- 172.21.15.43:0/1129456056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85a8101a50 0x7f85a8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.183+0000 7f85aeb52640 1 --2- 172.21.15.43:0/1129456056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85a81010a0 0x7f85a8101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.183+0000 7f85aeb52640 1 -- 172.21.15.43:0/1129456056 >> 172.21.15.43:0/1129456056 conn(0x7f85a8077eb0 msgr2=0x7f85a80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:00.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.183+0000 7f85aeb52640 1 -- 172.21.15.43:0/1129456056 shutdown_connections 2024-04-21T08:45:00.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.183+0000 7f85aeb52640 1 -- 172.21.15.43:0/1129456056 wait complete. 2024-04-21T08:45:00.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.184+0000 7f85aeb52640 1 Processor -- start 2024-04-21T08:45:00.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.184+0000 7f85aeb52640 1 -- start start 2024-04-21T08:45:00.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.184+0000 7f85aeb52640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85a81010a0 0x7f85a81986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.184+0000 7f85aeb52640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a8106170 0x7f85a8198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.184+0000 7f85aeb52640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85a819cf70 0x7f85a819d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.184+0000 7f85aeb52640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f85a81121f0 con 0x7f85a8106170 2024-04-21T08:45:00.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.184+0000 7f85aeb52640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85a8112070 con 0x7f85a81010a0 2024-04-21T08:45:00.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.184+0000 7f85aeb52640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f85a8112370 con 0x7f85a819cf70 2024-04-21T08:45:00.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.184+0000 7f85a7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a8106170 0x7f85a8198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.184+0000 7f85a7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a8106170 0x7f85a8198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33768/0 (socket says 172.21.15.43:33768) 2024-04-21T08:45:00.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.184+0000 7f85a7fff640 1 -- 172.21.15.43:0/3361627017 learned_addr learned my addr 172.21.15.43:0/3361627017 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:00.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.184+0000 7f85ad0ef640 1 --2- 172.21.15.43:0/3361627017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85a819cf70 0x7f85a819d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.184+0000 7f85ac8ee640 1 --2- 172.21.15.43:0/3361627017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85a81010a0 0x7f85a81986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.184+0000 7f85a7fff640 1 -- 172.21.15.43:0/3361627017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85a819cf70 msgr2=0x7f85a819d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.184+0000 7f85a7fff640 1 --2- 172.21.15.43:0/3361627017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85a819cf70 0x7f85a819d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.184+0000 7f85a7fff640 1 -- 172.21.15.43:0/3361627017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85a81010a0 msgr2=0x7f85a81986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.184+0000 7f85a7fff640 1 --2- 172.21.15.43:0/3361627017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85a81010a0 0x7f85a81986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.184+0000 7f85a7fff640 1 -- 172.21.15.43:0/3361627017 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a819da60 con 0x7f85a8106170 2024-04-21T08:45:00.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.184+0000 7f85ac8ee640 1 --2- 172.21.15.43:0/3361627017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85a81010a0 0x7f85a81986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:00.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.185+0000 7f85a7fff640 1 --2- 172.21.15.43:0/3361627017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a8106170 0x7f85a8198bf0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f859c00d740 tx=0x7f859c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:00.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.185+0000 7f85a5ffb640 1 -- 172.21.15.43:0/3361627017 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c017070 con 0x7f85a8106170 2024-04-21T08:45:00.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.185+0000 7f85aeb52640 1 -- 172.21.15.43:0/3361627017 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a819dd50 con 0x7f85a8106170 2024-04-21T08:45:00.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.185+0000 7f85a5ffb640 1 -- 172.21.15.43:0/3361627017 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f859c00de50 con 0x7f85a8106170 2024-04-21T08:45:00.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.185+0000 7f85a5ffb640 1 -- 172.21.15.43:0/3361627017 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c004ce0 con 0x7f85a8106170 2024-04-21T08:45:00.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.185+0000 7f85aeb52640 1 -- 172.21.15.43:0/3361627017 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a81a54f0 con 0x7f85a8106170 2024-04-21T08:45:00.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.187+0000 7f85a5ffb640 1 -- 172.21.15.43:0/3361627017 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f859c004e80 con 0x7f85a8106170 2024-04-21T08:45:00.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.187+0000 7f85b1824740 1 Processor -- start 2024-04-21T08:45:00.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.187+0000 7f85b1824740 1 -- start start 2024-04-21T08:45:00.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.188+0000 7f85b1824740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e815e4d0 0x5591e815e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.188+0000 7f85b1824740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e815ee80 0x5591e8167430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.188+0000 7f85b1824740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e8167a00 0x5591e8169e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.188+0000 7f85b1824740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5591e80c31f0 con 0x5591e815e4d0 2024-04-21T08:45:00.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.188+0000 7f85b1824740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5591e80604b0 con 0x5591e815ee80 2024-04-21T08:45:00.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.188+0000 7f85b1824740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5591e80bf1b0 con 0x5591e8167a00 2024-04-21T08:45:00.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.188+0000 7f85a5ffb640 1 --2- 172.21.15.43:0/3361627017 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f85900a3d20 0x7f85900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.188+0000 7f85ad0ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e815e4d0 0x5591e815e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.188+0000 7f85a7fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e8167a00 0x5591e8169e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.188+0000 7f85ad0ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e815e4d0 0x5591e815e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33780/0 (socket says 172.21.15.43:33780) 2024-04-21T08:45:00.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.188+0000 7f85ad0ef640 1 -- 172.21.15.43:0/3027566948 learned_addr learned my addr 172.21.15.43:0/3027566948 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:00.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.188+0000 7f85ad0ef640 1 --2- 172.21.15.43:0/3361627017 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f85900a3d20 0x7f85900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.188+0000 7f85ac8ee640 1 --2- 172.21.15.43:0/3027566948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e815ee80 0x5591e8167430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.188+0000 7f85a5ffb640 1 -- 172.21.15.43:0/3361627017 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f859c0d0550 con 0x7f85a8106170 2024-04-21T08:45:00.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.188+0000 7f85ad0ef640 1 -- 172.21.15.43:0/3027566948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e8167a00 msgr2=0x5591e8169e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:00.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.188+0000 7f85ad0ef640 1 --2- 172.21.15.43:0/3027566948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e8167a00 0x5591e8169e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.188+0000 7f85ad0ef640 1 -- 172.21.15.43:0/3027566948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e815ee80 msgr2=0x5591e8167430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:00.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.188+0000 7f85ad0ef640 1 --2- 172.21.15.43:0/3027566948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e815ee80 0x5591e8167430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.188+0000 7f85ad0ef640 1 -- 172.21.15.43:0/3027566948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591e80521b0 con 0x5591e815e4d0 2024-04-21T08:45:00.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.188+0000 7f85ad0ef640 1 --2- 172.21.15.43:0/3027566948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e815e4d0 0x5591e815e8b0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f85a0037040 tx=0x7f85a0032f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e0f489e82958065 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:00.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.188+0000 7f85ad0ef640 1 --2- 172.21.15.43:0/3361627017 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f85900a3d20 0x7f85900a61e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009ca0 tx=0x7f85a0042000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:00.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.189+0000 7f858effd640 1 -- 172.21.15.43:0/3027566948 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0051070 con 0x5591e815e4d0 2024-04-21T08:45:00.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.189+0000 7f858effd640 1 -- 172.21.15.43:0/3027566948 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a00327c0 con 0x5591e815e4d0 2024-04-21T08:45:00.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.189+0000 7f858effd640 1 -- 172.21.15.43:0/3027566948 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0039040 con 0x5591e815e4d0 2024-04-21T08:45:00.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.189+0000 7f85b1824740 1 -- 172.21.15.43:0/3027566948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e815e4d0 msgr2=0x5591e815e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.189+0000 7f85b1824740 1 --2- 172.21.15.43:0/3027566948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e815e4d0 0x5591e815e8b0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f85a0037040 tx=0x7f85a0032f70 comp rx=0 tx=0).stop 2024-04-21T08:45:00.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.189+0000 7f85b1824740 1 -- 172.21.15.43:0/3027566948 shutdown_connections 2024-04-21T08:45:00.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.189+0000 7f85b1824740 1 --2- 172.21.15.43:0/3027566948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e8167a00 0x5591e8169e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.189+0000 7f85b1824740 1 --2- 172.21.15.43:0/3027566948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e815ee80 0x5591e8167430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.189+0000 7f85b1824740 1 --2- 172.21.15.43:0/3027566948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e815e4d0 0x5591e815e8b0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.189+0000 7f85b1824740 1 -- 172.21.15.43:0/3027566948 >> 172.21.15.43:0/3027566948 conn(0x5591e8158ba0 msgr2=0x5591e8158f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:00.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.189+0000 7f85b1824740 1 -- 172.21.15.43:0/3027566948 shutdown_connections 2024-04-21T08:45:00.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.189+0000 7f85b1824740 1 -- 172.21.15.43:0/3027566948 wait complete. 2024-04-21T08:45:00.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.190+0000 7f85b1824740 1 -- start start 2024-04-21T08:45:00.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.190+0000 7f85b1824740 1 Processor -- start 2024-04-21T08:45:00.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.190+0000 7f85b1824740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e815e4d0 0x5591e8167370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.190+0000 7f85b1824740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e815ee80 0x5591e8164700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.190+0000 7f85b1824740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e8167a00 0x5591e8164d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.190+0000 7f85b1824740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5591e816a910 con 0x5591e8167a00 2024-04-21T08:45:00.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.190+0000 7f85b1824740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5591e807c500 con 0x5591e815ee80 2024-04-21T08:45:00.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.190+0000 7f85b1824740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5591e816aa90 con 0x5591e815e4d0 2024-04-21T08:45:00.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.190+0000 7f85b1824740 10 client.0 _ll_register_callbacks cb 0x5591e804ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:00.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.190+0000 7f85ad0ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e8167a00 0x5591e8164d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.190+0000 7f85ad0ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e8167a00 0x5591e8164d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33784/0 (socket says 172.21.15.43:33784) 2024-04-21T08:45:00.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.190+0000 7f85ad0ef640 1 -- 172.21.15.43:0/4029739274 learned_addr learned my addr 172.21.15.43:0/4029739274 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:00.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.190+0000 7f85a7fff640 1 --2- 172.21.15.43:0/4029739274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e815ee80 0x5591e8164700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.191+0000 7f85ac8ee640 1 --2- 172.21.15.43:0/4029739274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e815e4d0 0x5591e8167370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.191+0000 7f85ad0ef640 1 -- 172.21.15.43:0/4029739274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e815e4d0 msgr2=0x5591e8167370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:00.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.191+0000 7f85ad0ef640 1 --2- 172.21.15.43:0/4029739274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e815e4d0 0x5591e8167370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.191+0000 7f85ad0ef640 1 -- 172.21.15.43:0/4029739274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e815ee80 msgr2=0x5591e8164700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:00.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.191+0000 7f85ad0ef640 1 --2- 172.21.15.43:0/4029739274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e815ee80 0x5591e8164700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.191+0000 7f85ad0ef640 1 -- 172.21.15.43:0/4029739274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591e7dd53f0 con 0x5591e8167a00 2024-04-21T08:45:00.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.191+0000 7f85a7fff640 1 --2- 172.21.15.43:0/4029739274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e815ee80 0x5591e8164700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:00.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.191+0000 7f85ac8ee640 1 --2- 172.21.15.43:0/4029739274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e815e4d0 0x5591e8167370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:00.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.191+0000 7f85ad0ef640 1 --2- 172.21.15.43:0/4029739274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e8167a00 0x5591e8164d10 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009600 tx=0x7f85a0004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d624cbbcbae807c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:00.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.191+0000 7f8587fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:00.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.191+0000 7f8587fff640 1 -- 172.21.15.43:0/4029739274 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a005a060 con 0x5591e8167a00 2024-04-21T08:45:00.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.191+0000 7f8587fff640 1 -- 172.21.15.43:0/4029739274 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f85a0004380 con 0x5591e8167a00 2024-04-21T08:45:00.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.191+0000 7f85b1824740 1 -- 172.21.15.43:0/4029739274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a00327c0 con 0x5591e8167a00 2024-04-21T08:45:00.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.191+0000 7f8587fff640 1 -- 172.21.15.43:0/4029739274 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f85a0061050 con 0x5591e8167a00 2024-04-21T08:45:00.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.191+0000 7f85b1824740 10 client.18279 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:00.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.191+0000 7f85b1824740 1 -- 172.21.15.43:0/4029739274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5591e7e9f7d0 con 0x5591e8167a00 2024-04-21T08:45:00.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.191+0000 7f8587fff640 1 -- 172.21.15.43:0/4029739274 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a00327c0 con 0x5591e8167a00 2024-04-21T08:45:00.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.191+0000 7f8587fff640 1 -- 172.21.15.43:0/4029739274 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f85a004b070 con 0x5591e8167a00 2024-04-21T08:45:00.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.192+0000 7f85b1824740 10 client.18279 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:00.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.192+0000 7f85b1824740 10 client.18279 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:45:00.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.192+0000 7f85b1824740 20 client.18279 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:00.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.192+0000 7f85b1824740 1 --2- 172.21.15.43:0/4029739274 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5591e8177a80 0x5591e8179e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.192+0000 7f85b1824740 4 client.18279 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:00.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.192+0000 7f85b1824740 1 -- 172.21.15.43:0/4029739274 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591e8062930 con 0x5591e8177a80 2024-04-21T08:45:00.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.192+0000 7f85ac8ee640 1 --2- 172.21.15.43:0/4029739274 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5591e8177a80 0x5591e8179e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.192+0000 7f8587fff640 10 client.18279 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:45:00.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.192+0000 7f85ac8ee640 1 --2- 172.21.15.43:0/4029739274 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5591e8177a80 0x5591e8179e70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=86a0b3262622909f server_cookie=e49d3a890a6ac33d in_seq=0 out_seq=0 2024-04-21T08:45:00.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.193+0000 7f8587fff640 1 -- 172.21.15.43:0/4029739274 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5591e8062930 con 0x5591e8177a80 2024-04-21T08:45:00.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.193+0000 7f8587fff640 10 client.18279 handle_command_reply: tid=0 2024-04-21T08:45:00.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.193+0000 7f85b1824740 1 client.18279 shutdown 2024-04-21T08:45:00.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.193+0000 7f85b1824740 1 -- 172.21.15.43:0/4029739274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e8167a00 msgr2=0x5591e8164d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.193+0000 7f85b1824740 1 --2- 172.21.15.43:0/4029739274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e8167a00 0x5591e8164d10 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009600 tx=0x7f85a0004f90 comp rx=0 tx=0).stop 2024-04-21T08:45:00.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.193+0000 7f85b1824740 1 -- 172.21.15.43:0/4029739274 shutdown_connections 2024-04-21T08:45:00.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.193+0000 7f85b1824740 1 --2- 172.21.15.43:0/4029739274 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5591e8177a80 0x5591e8179e70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.193+0000 7f85b1824740 1 --2- 172.21.15.43:0/4029739274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e8167a00 0x5591e8164d10 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.193+0000 7f85b1824740 1 --2- 172.21.15.43:0/4029739274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e815ee80 0x5591e8164700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.193+0000 7f85b1824740 1 --2- 172.21.15.43:0/4029739274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e815e4d0 0x5591e8167370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.193+0000 7f85b1824740 1 -- 172.21.15.43:0/4029739274 >> 172.21.15.43:0/4029739274 conn(0x5591e8158ba0 msgr2=0x5591e815a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:00.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.193+0000 7f8587fff640 0 client.18279 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:45:00.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.194+0000 7f85b1824740 1 -- 172.21.15.43:0/4029739274 shutdown_connections 2024-04-21T08:45:00.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.194+0000 7f85b1824740 1 -- 172.21.15.43:0/4029739274 wait complete. 2024-04-21T08:45:00.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.194+0000 7f85b1824740 20 client.18279 trim_cache size 0 max 16384 2024-04-21T08:45:00.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.207+0000 7f85b1824740 1 Processor -- start 2024-04-21T08:45:00.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.207+0000 7f85b1824740 1 -- start start 2024-04-21T08:45:00.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.207+0000 7f85b1824740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e815e4d0 0x5591e81737d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.207+0000 7f85b1824740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e815ee80 0x5591e8162ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.207+0000 7f85b1824740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e8167a00 0x5591e8166e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.207+0000 7f85b1824740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f85a81af390 con 0x5591e815ee80 2024-04-21T08:45:00.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.207+0000 7f85b1824740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85a81b0ed0 con 0x5591e8167a00 2024-04-21T08:45:00.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.207+0000 7f85b1824740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f85a81afa60 con 0x5591e815e4d0 2024-04-21T08:45:00.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.207+0000 7f85a7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e815ee80 0x5591e8162ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.207+0000 7f85a7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e815ee80 0x5591e8162ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33792/0 (socket says 172.21.15.43:33792) 2024-04-21T08:45:00.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.207+0000 7f85a7fff640 1 -- 172.21.15.43:0/437038763 learned_addr learned my addr 172.21.15.43:0/437038763 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:00.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.207+0000 7f85a7fff640 1 -- 172.21.15.43:0/437038763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e815e4d0 msgr2=0x5591e81737d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:00.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.207+0000 7f85a7fff640 1 --2- 172.21.15.43:0/437038763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e815e4d0 0x5591e81737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.207+0000 7f85a7fff640 1 -- 172.21.15.43:0/437038763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e8167a00 msgr2=0x5591e8166e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:00.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.207+0000 7f85a7fff640 1 --2- 172.21.15.43:0/437038763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e8167a00 0x5591e8166e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.207+0000 7f85a7fff640 1 -- 172.21.15.43:0/437038763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591e815e0b0 con 0x5591e815ee80 2024-04-21T08:45:00.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.208+0000 7f85a7fff640 1 --2- 172.21.15.43:0/437038763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e815ee80 0x5591e8162ea0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f859c009610 tx=0x7f859c0c71d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df432f8bead9e5c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:00.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.208+0000 7f858e7fc640 1 -- 172.21.15.43:0/437038763 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c020040 con 0x5591e815ee80 2024-04-21T08:45:00.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.208+0000 7f858e7fc640 1 -- 172.21.15.43:0/437038763 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f859c0a4d30 con 0x5591e815ee80 2024-04-21T08:45:00.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.208+0000 7f858e7fc640 1 -- 172.21.15.43:0/437038763 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c0cdaf0 con 0x5591e815ee80 2024-04-21T08:45:00.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.208+0000 7f85b1824740 1 -- 172.21.15.43:0/437038763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e815ee80 msgr2=0x5591e8162ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.208+0000 7f85b1824740 1 --2- 172.21.15.43:0/437038763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e815ee80 0x5591e8162ea0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f859c009610 tx=0x7f859c0c71d0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.208+0000 7f85b1824740 1 -- 172.21.15.43:0/437038763 shutdown_connections 2024-04-21T08:45:00.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.208+0000 7f85b1824740 1 --2- 172.21.15.43:0/437038763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e8167a00 0x5591e8166e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.208+0000 7f85b1824740 1 --2- 172.21.15.43:0/437038763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e815ee80 0x5591e8162ea0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.208+0000 7f85b1824740 1 --2- 172.21.15.43:0/437038763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e815e4d0 0x5591e81737d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.208+0000 7f85b1824740 1 -- 172.21.15.43:0/437038763 >> 172.21.15.43:0/437038763 conn(0x5591e8158ba0 msgr2=0x5591e815a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:00.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.208+0000 7f85b1824740 1 -- 172.21.15.43:0/437038763 shutdown_connections 2024-04-21T08:45:00.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.208+0000 7f85b1824740 1 -- 172.21.15.43:0/437038763 wait complete. 2024-04-21T08:45:00.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.209+0000 7f85b1824740 1 -- start start 2024-04-21T08:45:00.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.209+0000 7f85b1824740 1 Processor -- start 2024-04-21T08:45:00.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.209+0000 7f85b1824740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e815e4d0 0x5591e816ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.209+0000 7f85b1824740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e815ee80 0x5591e816bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.209+0000 7f85b1824740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e8167a00 0x5591e81dcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.209+0000 7f85b1824740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f85a81b0800 con 0x5591e815e4d0 2024-04-21T08:45:00.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.209+0000 7f85b1824740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85a81b0130 con 0x5591e8167a00 2024-04-21T08:45:00.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.209+0000 7f85b1824740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f85a81ad370 con 0x5591e815ee80 2024-04-21T08:45:00.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.209+0000 7f85b1824740 10 client.0 _ll_register_callbacks cb 0x7f859c005ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:00.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.209+0000 7f85ac8ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e815e4d0 0x5591e816ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.209+0000 7f85ac8ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e815e4d0 0x5591e816ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33808/0 (socket says 172.21.15.43:33808) 2024-04-21T08:45:00.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.209+0000 7f85ac8ee640 1 -- 172.21.15.43:0/2183293428 learned_addr learned my addr 172.21.15.43:0/2183293428 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:00.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.209+0000 7f85ad0ef640 1 --2- 172.21.15.43:0/2183293428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e8167a00 0x5591e81dcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.209+0000 7f85a7fff640 1 --2- 172.21.15.43:0/2183293428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e815ee80 0x5591e816bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.209+0000 7f85ac8ee640 1 -- 172.21.15.43:0/2183293428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e815ee80 msgr2=0x5591e816bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:00.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.209+0000 7f85ac8ee640 1 --2- 172.21.15.43:0/2183293428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e815ee80 0x5591e816bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.209+0000 7f85ac8ee640 1 -- 172.21.15.43:0/2183293428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e8167a00 msgr2=0x5591e81dcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:00.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.209+0000 7f85ac8ee640 1 --2- 172.21.15.43:0/2183293428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e8167a00 0x5591e81dcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.209+0000 7f85ac8ee640 1 -- 172.21.15.43:0/2183293428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591e816c810 con 0x5591e815e4d0 2024-04-21T08:45:00.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.210+0000 7f85ac8ee640 1 --2- 172.21.15.43:0/2183293428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e815e4d0 0x5591e816ba30 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x5591e815f280 tx=0x7f8598009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6090ac3ef1f770dc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:00.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.210+0000 7f8587fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:00.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.210+0000 7f8587fff640 1 -- 172.21.15.43:0/2183293428 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859801e050 con 0x5591e815e4d0 2024-04-21T08:45:00.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.210+0000 7f8587fff640 1 -- 172.21.15.43:0/2183293428 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f859801e1f0 con 0x5591e815e4d0 2024-04-21T08:45:00.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.210+0000 7f85b1824740 1 -- 172.21.15.43:0/2183293428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f859c0a4d30 con 0x5591e815e4d0 2024-04-21T08:45:00.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.210+0000 7f8587fff640 1 -- 172.21.15.43:0/2183293428 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f859802d050 con 0x5591e815e4d0 2024-04-21T08:45:00.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.210+0000 7f85b1824740 10 client.18285 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:00.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.210+0000 7f85b1824740 1 -- 172.21.15.43:0/2183293428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5591e81de380 con 0x5591e815e4d0 2024-04-21T08:45:00.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.210+0000 7f8587fff640 1 -- 172.21.15.43:0/2183293428 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f859c0a4d30 con 0x5591e815e4d0 2024-04-21T08:45:00.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.210+0000 7f8587fff640 1 -- 172.21.15.43:0/2183293428 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8598012890 con 0x5591e815e4d0 2024-04-21T08:45:00.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.210+0000 7f85b1824740 10 client.18285 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:00.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.210+0000 7f85b1824740 10 client.18285 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:45:00.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.210+0000 7f85b1824740 20 client.18285 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:00.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.210+0000 7f85b1824740 1 --2- 172.21.15.43:0/2183293428 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5591e81deb30 0x5591e81e0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.210+0000 7f85b1824740 4 client.18285 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:45:00.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.210+0000 7f85b1824740 1 -- 172.21.15.43:0/2183293428 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591e80d0c10 con 0x5591e81deb30 2024-04-21T08:45:00.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.211+0000 7f85a7fff640 1 --2- 172.21.15.43:0/2183293428 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5591e81deb30 0x5591e81e0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.211+0000 7f85a7fff640 1 --2- 172.21.15.43:0/2183293428 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5591e81deb30 0x5591e81e0f50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eac1c89405ab79b8 server_cookie=6539da345d635f1b in_seq=0 out_seq=0 2024-04-21T08:45:00.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.211+0000 7f8587fff640 10 client.18285 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:45:00.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.211+0000 7f8587fff640 1 -- 172.21.15.43:0/2183293428 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591e80d0c10 con 0x5591e81deb30 2024-04-21T08:45:00.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.211+0000 7f8587fff640 10 client.18285 handle_command_reply: tid=0 2024-04-21T08:45:00.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.211+0000 7f85b1824740 1 client.18285 shutdown 2024-04-21T08:45:00.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.211+0000 7f85b1824740 1 -- 172.21.15.43:0/2183293428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e815e4d0 msgr2=0x5591e816ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.211+0000 7f85b1824740 1 --2- 172.21.15.43:0/2183293428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e815e4d0 0x5591e816ba30 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x5591e815f280 tx=0x7f8598009cd0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.211+0000 7f85b1824740 1 -- 172.21.15.43:0/2183293428 shutdown_connections 2024-04-21T08:45:00.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.212+0000 7f85b1824740 1 --2- 172.21.15.43:0/2183293428 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5591e81deb30 0x5591e81e0f50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.212+0000 7f85b1824740 1 --2- 172.21.15.43:0/2183293428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591e8167a00 0x5591e81dcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.212+0000 7f85b1824740 1 --2- 172.21.15.43:0/2183293428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591e815ee80 0x5591e816bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.212+0000 7f8587fff640 0 client.18285 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:45:00.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.212+0000 7f85b1824740 1 --2- 172.21.15.43:0/2183293428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591e815e4d0 0x5591e816ba30 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.212+0000 7f85b1824740 1 -- 172.21.15.43:0/2183293428 >> 172.21.15.43:0/2183293428 conn(0x5591e8158ba0 msgr2=0x5591e815f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:00.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.212+0000 7f85b1824740 1 -- 172.21.15.43:0/2183293428 shutdown_connections 2024-04-21T08:45:00.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.212+0000 7f85b1824740 1 -- 172.21.15.43:0/2183293428 wait complete. 2024-04-21T08:45:00.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.212+0000 7f85b1824740 20 client.18285 trim_cache size 0 max 16384 2024-04-21T08:45:00.219 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:45:00.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.212+0000 7f85aeb52640 1 -- 172.21.15.43:0/3361627017 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f85900a3d20 msgr2=0x7f85900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.212+0000 7f85aeb52640 1 --2- 172.21.15.43:0/3361627017 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f85900a3d20 0x7f85900a61e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009ca0 tx=0x7f85a0042000 comp rx=0 tx=0).stop 2024-04-21T08:45:00.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.212+0000 7f85aeb52640 1 -- 172.21.15.43:0/3361627017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a8106170 msgr2=0x7f85a8198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.212+0000 7f85aeb52640 1 --2- 172.21.15.43:0/3361627017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a8106170 0x7f85a8198bf0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f859c00d740 tx=0x7f859c00dc00 comp rx=0 tx=0).stop 2024-04-21T08:45:00.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.212+0000 7f85aeb52640 1 -- 172.21.15.43:0/3361627017 shutdown_connections 2024-04-21T08:45:00.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.212+0000 7f85aeb52640 1 --2- 172.21.15.43:0/3361627017 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f85900a3d20 0x7f85900a61e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.212+0000 7f85aeb52640 1 --2- 172.21.15.43:0/3361627017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85a819cf70 0x7f85a819d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.212+0000 7f85aeb52640 1 --2- 172.21.15.43:0/3361627017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a8106170 0x7f85a8198bf0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.212+0000 7f85aeb52640 1 --2- 172.21.15.43:0/3361627017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85a81010a0 0x7f85a81986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.212+0000 7f85aeb52640 1 -- 172.21.15.43:0/3361627017 >> 172.21.15.43:0/3361627017 conn(0x7f85a8077eb0 msgr2=0x7f85a8100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:00.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.212+0000 7f85aeb52640 1 -- 172.21.15.43:0/3361627017 shutdown_connections 2024-04-21T08:45:00.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.212+0000 7f85aeb52640 1 -- 172.21.15.43:0/3361627017 wait complete. 2024-04-21T08:45:00.222 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:00 smithi043 ceph-mon[27978]: pgmap v185: 129 pgs: 129 active+clean; 4.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-21T08:45:00.222 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:00 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:45:00.222 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:00 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:00.222 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:00 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:45:00.223 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:00 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:00.223 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:00 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:45:00.223 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:00 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/4294086830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:00.224 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:45:00.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.327+0000 7f7a6a7a4640 1 -- 172.21.15.43:0/4198141221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a64101090 msgr2=0x7f7a64101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.327+0000 7f7a6a7a4640 1 --2- 172.21.15.43:0/4198141221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a64101090 0x7f7a64101470 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009900 tx=0x7f7a5402bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:00.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.327+0000 7f7a6a7a4640 1 -- 172.21.15.43:0/4198141221 shutdown_connections 2024-04-21T08:45:00.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.327+0000 7f7a6a7a4640 1 --2- 172.21.15.43:0/4198141221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a64106160 0x7f7a6410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.327+0000 7f7a6a7a4640 1 --2- 172.21.15.43:0/4198141221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a64101a40 0x7f7a64105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.327+0000 7f7a6a7a4640 1 --2- 172.21.15.43:0/4198141221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a64101090 0x7f7a64101470 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.327+0000 7f7a6a7a4640 1 -- 172.21.15.43:0/4198141221 >> 172.21.15.43:0/4198141221 conn(0x7f7a64077f50 msgr2=0x7f7a64078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:00.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.327+0000 7f7a6a7a4640 1 -- 172.21.15.43:0/4198141221 shutdown_connections 2024-04-21T08:45:00.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.327+0000 7f7a6a7a4640 1 -- 172.21.15.43:0/4198141221 wait complete. 2024-04-21T08:45:00.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.327+0000 7f7a6a7a4640 1 Processor -- start 2024-04-21T08:45:00.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.327+0000 7f7a6a7a4640 1 -- start start 2024-04-21T08:45:00.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.328+0000 7f7a6a7a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a64101090 0x7f7a64199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.328+0000 7f7a6a7a4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a64106160 0x7f7a64199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.328+0000 7f7a6a7a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a6419a0e0 0x7f7a641a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.328+0000 7f7a6a7a4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7a641110c0 con 0x7f7a64101090 2024-04-21T08:45:00.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.328+0000 7f7a6a7a4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7a64110f40 con 0x7f7a6419a0e0 2024-04-21T08:45:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.328+0000 7f7a6a7a4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7a64111240 con 0x7f7a64106160 2024-04-21T08:45:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.328+0000 7f7a63fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a64101090 0x7f7a64199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.328+0000 7f7a63fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a64101090 0x7f7a64199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33820/0 (socket says 172.21.15.43:33820) 2024-04-21T08:45:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.328+0000 7f7a63fff640 1 -- 172.21.15.43:0/1914554493 learned_addr learned my addr 172.21.15.43:0/1914554493 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.328+0000 7f7a637fe640 1 --2- 172.21.15.43:0/1914554493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a64106160 0x7f7a64199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.328+0000 7f7a68d41640 1 --2- 172.21.15.43:0/1914554493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a6419a0e0 0x7f7a641a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.328+0000 7f7a63fff640 1 -- 172.21.15.43:0/1914554493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a64106160 msgr2=0x7f7a64199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.328+0000 7f7a63fff640 1 --2- 172.21.15.43:0/1914554493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a64106160 0x7f7a64199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.328+0000 7f7a63fff640 1 -- 172.21.15.43:0/1914554493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a6419a0e0 msgr2=0x7f7a641a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.328+0000 7f7a63fff640 1 --2- 172.21.15.43:0/1914554493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a6419a0e0 0x7f7a641a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.328+0000 7f7a63fff640 1 -- 172.21.15.43:0/1914554493 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a641a4e60 con 0x7f7a64101090 2024-04-21T08:45:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.328+0000 7f7a68d41640 1 --2- 172.21.15.43:0/1914554493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a6419a0e0 0x7f7a641a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.328+0000 7f7a637fe640 1 --2- 172.21.15.43:0/1914554493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a64106160 0x7f7a64199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.328+0000 7f7a63fff640 1 --2- 172.21.15.43:0/1914554493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a64101090 0x7f7a64199660 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f7a54005940 tx=0x7f7a5402c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.328+0000 7f7a617fa640 1 -- 172.21.15.43:0/1914554493 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a54043020 con 0x7f7a64101090 2024-04-21T08:45:00.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.328+0000 7f7a617fa640 1 -- 172.21.15.43:0/1914554493 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a5402cc60 con 0x7f7a64101090 2024-04-21T08:45:00.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.328+0000 7f7a6a7a4640 1 -- 172.21.15.43:0/1914554493 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a641a50f0 con 0x7f7a64101090 2024-04-21T08:45:00.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.328+0000 7f7a617fa640 1 -- 172.21.15.43:0/1914554493 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a54003c30 con 0x7f7a64101090 2024-04-21T08:45:00.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.329+0000 7f7a6a7a4640 1 -- 172.21.15.43:0/1914554493 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a641a5640 con 0x7f7a64101090 2024-04-21T08:45:00.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.331+0000 7f7a617fa640 1 -- 172.21.15.43:0/1914554493 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7a54046020 con 0x7f7a64101090 2024-04-21T08:45:00.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.331+0000 7f7a6d391740 1 Processor -- start 2024-04-21T08:45:00.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.331+0000 7f7a6d391740 1 -- start start 2024-04-21T08:45:00.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.331+0000 7f7a6d391740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a5d0194d0 0x561a5d0198b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.331+0000 7f7a6d391740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d019e80 0x561a5d022430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.331+0000 7f7a6d391740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a5d022a00 0x561a5d024e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.331+0000 7f7a6d391740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561a5cf7e1f0 con 0x561a5d019e80 2024-04-21T08:45:00.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.331+0000 7f7a6d391740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561a5cf1b4b0 con 0x561a5d022a00 2024-04-21T08:45:00.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.331+0000 7f7a6d391740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561a5cf7a1b0 con 0x561a5d0194d0 2024-04-21T08:45:00.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.331+0000 7f7a617fa640 1 --2- 172.21.15.43:0/1914554493 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7a4c0a3d20 0x7f7a4c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.331+0000 7f7a63fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d019e80 0x561a5d022430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.331+0000 7f7a68d41640 1 --2- 172.21.15.43:0/1914554493 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7a4c0a3d20 0x7f7a4c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.331+0000 7f7a63fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d019e80 0x561a5d022430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33830/0 (socket says 172.21.15.43:33830) 2024-04-21T08:45:00.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.331+0000 7f7a63fff640 1 -- 172.21.15.43:0/158988393 learned_addr learned my addr 172.21.15.43:0/158988393 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:00.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.331+0000 7f7a637fe640 1 --2- 172.21.15.43:0/158988393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a5d022a00 0x561a5d024e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.332+0000 7f7a68d41640 1 --2- 172.21.15.43:0/158988393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a5d0194d0 0x561a5d0198b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.332+0000 7f7a617fa640 1 -- 172.21.15.43:0/1914554493 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f7a540f2fc0 con 0x7f7a64101090 2024-04-21T08:45:00.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.332+0000 7f7a63fff640 1 -- 172.21.15.43:0/158988393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a5d0194d0 msgr2=0x561a5d0198b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:00.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.332+0000 7f7a63fff640 1 --2- 172.21.15.43:0/158988393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a5d0194d0 0x561a5d0198b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.332+0000 7f7a63fff640 1 -- 172.21.15.43:0/158988393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a5d022a00 msgr2=0x561a5d024e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:00.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.332+0000 7f7a63fff640 1 --2- 172.21.15.43:0/158988393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a5d022a00 0x561a5d024e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.332+0000 7f7a63fff640 1 -- 172.21.15.43:0/158988393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a5cf0d1b0 con 0x561a5d019e80 2024-04-21T08:45:00.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.332+0000 7f7a637fe640 1 --2- 172.21.15.43:0/158988393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a5d022a00 0x561a5d024e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:45:00.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.332+0000 7f7a68d41640 1 --2- 172.21.15.43:0/1914554493 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7a4c0a3d20 0x7f7a4c0a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c014660 tx=0x7f7a5c014a60 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:00.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.332+0000 7f7a63fff640 1 --2- 172.21.15.43:0/158988393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d019e80 0x561a5d022430 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f7a540ba980 tx=0x7f7a54003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def577578f09fca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:00.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.332+0000 7f7a4a7fc640 1 -- 172.21.15.43:0/158988393 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a540dde60 con 0x561a5d019e80 2024-04-21T08:45:00.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.332+0000 7f7a4a7fc640 1 -- 172.21.15.43:0/158988393 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a54005560 con 0x561a5d019e80 2024-04-21T08:45:00.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.332+0000 7f7a4a7fc640 1 -- 172.21.15.43:0/158988393 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a540bfab0 con 0x561a5d019e80 2024-04-21T08:45:00.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.333+0000 7f7a6d391740 1 -- 172.21.15.43:0/158988393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d019e80 msgr2=0x561a5d022430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.333+0000 7f7a6d391740 1 --2- 172.21.15.43:0/158988393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d019e80 0x561a5d022430 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f7a540ba980 tx=0x7f7a54003760 comp rx=0 tx=0).stop 2024-04-21T08:45:00.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.333+0000 7f7a6d391740 1 -- 172.21.15.43:0/158988393 shutdown_connections 2024-04-21T08:45:00.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.333+0000 7f7a6d391740 1 --2- 172.21.15.43:0/158988393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a5d022a00 0x561a5d024e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.333+0000 7f7a6d391740 1 --2- 172.21.15.43:0/158988393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d019e80 0x561a5d022430 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.333+0000 7f7a6d391740 1 --2- 172.21.15.43:0/158988393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a5d0194d0 0x561a5d0198b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.333+0000 7f7a6d391740 1 -- 172.21.15.43:0/158988393 >> 172.21.15.43:0/158988393 conn(0x561a5d013ba0 msgr2=0x561a5d013f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:00.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.333+0000 7f7a6d391740 1 -- 172.21.15.43:0/158988393 shutdown_connections 2024-04-21T08:45:00.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.333+0000 7f7a6d391740 1 -- 172.21.15.43:0/158988393 wait complete. 2024-04-21T08:45:00.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.333+0000 7f7a6d391740 1 -- start start 2024-04-21T08:45:00.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.333+0000 7f7a6d391740 1 Processor -- start 2024-04-21T08:45:00.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.334+0000 7f7a6d391740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a5d0194d0 0x561a5d022390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.334+0000 7f7a6d391740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d019e80 0x561a5d01f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.334+0000 7f7a6d391740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a5d022a00 0x561a5d01fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.334+0000 7f7a6d391740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561a5d025730 con 0x561a5d019e80 2024-04-21T08:45:00.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.334+0000 7f7a6d391740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561a5cf37500 con 0x561a5d022a00 2024-04-21T08:45:00.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.334+0000 7f7a6d391740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561a5d0258b0 con 0x561a5d0194d0 2024-04-21T08:45:00.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.334+0000 7f7a6d391740 10 client.0 _ll_register_callbacks cb 0x561a5cf05b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:00.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.334+0000 7f7a637fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d019e80 0x561a5d01f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.334+0000 7f7a637fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d019e80 0x561a5d01f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33846/0 (socket says 172.21.15.43:33846) 2024-04-21T08:45:00.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.334+0000 7f7a637fe640 1 -- 172.21.15.43:0/2351851887 learned_addr learned my addr 172.21.15.43:0/2351851887 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:00.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.334+0000 7f7a63fff640 1 --2- 172.21.15.43:0/2351851887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a5d0194d0 0x561a5d022390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.334+0000 7f7a68d41640 1 --2- 172.21.15.43:0/2351851887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a5d022a00 0x561a5d01fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.334+0000 7f7a637fe640 1 -- 172.21.15.43:0/2351851887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a5d0194d0 msgr2=0x561a5d022390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:00.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.334+0000 7f7a637fe640 1 --2- 172.21.15.43:0/2351851887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a5d0194d0 0x561a5d022390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.334+0000 7f7a637fe640 1 -- 172.21.15.43:0/2351851887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a5d022a00 msgr2=0x561a5d01fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:00.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.334+0000 7f7a637fe640 1 --2- 172.21.15.43:0/2351851887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a5d022a00 0x561a5d01fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.334+0000 7f7a637fe640 1 -- 172.21.15.43:0/2351851887 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a5cc903f0 con 0x561a5d019e80 2024-04-21T08:45:00.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.334+0000 7f7a63fff640 1 --2- 172.21.15.43:0/2351851887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a5d0194d0 0x561a5d022390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:00.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.334+0000 7f7a637fe640 1 --2- 172.21.15.43:0/2351851887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d019e80 0x561a5d01f720 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f7a580107b0 tx=0x7f7a58010c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98686ccdfddf456c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:00.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.334+0000 7f7a437fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:00.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.334+0000 7f7a437fe640 1 -- 172.21.15.43:0/2351851887 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5801d060 con 0x561a5d019e80 2024-04-21T08:45:00.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.335+0000 7f7a437fe640 1 -- 172.21.15.43:0/2351851887 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f7a5800eda0 con 0x561a5d019e80 2024-04-21T08:45:00.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.335+0000 7f7a6d391740 1 -- 172.21.15.43:0/2351851887 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561a5cf1d930 con 0x561a5d019e80 2024-04-21T08:45:00.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.335+0000 7f7a437fe640 1 -- 172.21.15.43:0/2351851887 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a58024050 con 0x561a5d019e80 2024-04-21T08:45:00.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.335+0000 7f7a6d391740 10 client.18297 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:00.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.335+0000 7f7a6d391740 1 -- 172.21.15.43:0/2351851887 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561a5cd5a7d0 con 0x561a5d019e80 2024-04-21T08:45:00.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.335+0000 7f7a437fe640 1 -- 172.21.15.43:0/2351851887 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561a5cf1d930 con 0x561a5d019e80 2024-04-21T08:45:00.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.335+0000 7f7a437fe640 1 -- 172.21.15.43:0/2351851887 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7a580164d0 con 0x561a5d019e80 2024-04-21T08:45:00.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.335+0000 7f7a6d391740 10 client.18297 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:00.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.335+0000 7f7a6d391740 10 client.18297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:00.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.335+0000 7f7a6d391740 20 client.18297 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:00.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.335+0000 7f7a6d391740 1 --2- 172.21.15.43:0/2351851887 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561a5d0329a0 0x561a5d034db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.335+0000 7f7a6d391740 4 client.18297 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:00.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.335+0000 7f7a6d391740 1 -- 172.21.15.43:0/2351851887 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a5cf8bc10 con 0x561a5d0329a0 2024-04-21T08:45:00.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.335+0000 7f7a63fff640 1 --2- 172.21.15.43:0/2351851887 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561a5d0329a0 0x561a5d034db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.336+0000 7f7a437fe640 10 client.18297 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:00.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.336+0000 7f7a63fff640 1 --2- 172.21.15.43:0/2351851887 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561a5d0329a0 0x561a5d034db0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4901939ee93c5c3 server_cookie=6cf3032a58df61c9 in_seq=0 out_seq=0 2024-04-21T08:45:00.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.336+0000 7f7a437fe640 1 -- 172.21.15.43:0/2351851887 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a5cf8bc10 con 0x561a5d0329a0 2024-04-21T08:45:00.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.336+0000 7f7a437fe640 10 client.18297 handle_command_reply: tid=0 2024-04-21T08:45:00.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.336+0000 7f7a6d391740 1 client.18297 shutdown 2024-04-21T08:45:00.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.337+0000 7f7a6d391740 1 -- 172.21.15.43:0/2351851887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d019e80 msgr2=0x561a5d01f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.337+0000 7f7a6d391740 1 --2- 172.21.15.43:0/2351851887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d019e80 0x561a5d01f720 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f7a580107b0 tx=0x7f7a58010c70 comp rx=0 tx=0).stop 2024-04-21T08:45:00.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.337+0000 7f7a6d391740 1 -- 172.21.15.43:0/2351851887 shutdown_connections 2024-04-21T08:45:00.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.337+0000 7f7a6d391740 1 --2- 172.21.15.43:0/2351851887 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561a5d0329a0 0x561a5d034db0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.337+0000 7f7a6d391740 1 --2- 172.21.15.43:0/2351851887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a5d022a00 0x561a5d01fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.337+0000 7f7a6d391740 1 --2- 172.21.15.43:0/2351851887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d019e80 0x561a5d01f720 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.337+0000 7f7a437fe640 0 client.18297 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:00.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.337+0000 7f7a6d391740 1 --2- 172.21.15.43:0/2351851887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a5d0194d0 0x561a5d022390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.337+0000 7f7a6d391740 1 -- 172.21.15.43:0/2351851887 >> 172.21.15.43:0/2351851887 conn(0x561a5d013ba0 msgr2=0x561a5d0158f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:00.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.337+0000 7f7a6d391740 1 -- 172.21.15.43:0/2351851887 shutdown_connections 2024-04-21T08:45:00.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.337+0000 7f7a6d391740 1 -- 172.21.15.43:0/2351851887 wait complete. 2024-04-21T08:45:00.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.337+0000 7f7a6d391740 20 client.18297 trim_cache size 0 max 16384 2024-04-21T08:45:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.350+0000 7f7a6d391740 1 Processor -- start 2024-04-21T08:45:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.350+0000 7f7a6d391740 1 -- start start 2024-04-21T08:45:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.350+0000 7f7a6d391740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d0194d0 0x561a5d02e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.350+0000 7f7a6d391740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a5d019e80 0x561a5d021e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.350+0000 7f7a6d391740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a5d022a00 0x561a5d022390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.350+0000 7f7a6d391740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7a641af430 con 0x561a5d0194d0 2024-04-21T08:45:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.350+0000 7f7a6d391740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7a641b0f70 con 0x561a5d019e80 2024-04-21T08:45:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.350+0000 7f7a6d391740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7a641afb00 con 0x561a5d022a00 2024-04-21T08:45:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.350+0000 7f7a63fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d0194d0 0x561a5d02e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.350+0000 7f7a63fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d0194d0 0x561a5d02e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33856/0 (socket says 172.21.15.43:33856) 2024-04-21T08:45:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.350+0000 7f7a63fff640 1 -- 172.21.15.43:0/2835583020 learned_addr learned my addr 172.21.15.43:0/2835583020 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.351+0000 7f7a637fe640 1 --2- 172.21.15.43:0/2835583020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a5d019e80 0x561a5d021e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.351+0000 7f7a63fff640 1 -- 172.21.15.43:0/2835583020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a5d022a00 msgr2=0x561a5d022390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.351+0000 7f7a63fff640 1 --2- 172.21.15.43:0/2835583020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a5d022a00 0x561a5d022390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.351+0000 7f7a63fff640 1 -- 172.21.15.43:0/2835583020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a5d019e80 msgr2=0x561a5d021e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.351+0000 7f7a63fff640 1 --2- 172.21.15.43:0/2835583020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a5d019e80 0x561a5d021e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.351+0000 7f7a63fff640 1 -- 172.21.15.43:0/2835583020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a5d0200a0 con 0x561a5d0194d0 2024-04-21T08:45:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.351+0000 7f7a637fe640 1 --2- 172.21.15.43:0/2835583020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a5d019e80 0x561a5d021e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.351+0000 7f7a63fff640 1 --2- 172.21.15.43:0/2835583020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d0194d0 0x561a5d02e510 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f7a540096c0 tx=0x7f7a540df040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51d6ae088352113 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.351+0000 7f7a49ffb640 1 -- 172.21.15.43:0/2835583020 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a540dc070 con 0x561a5d0194d0 2024-04-21T08:45:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.351+0000 7f7a49ffb640 1 -- 172.21.15.43:0/2835583020 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a540bf3a0 con 0x561a5d0194d0 2024-04-21T08:45:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.351+0000 7f7a49ffb640 1 -- 172.21.15.43:0/2835583020 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a540dde60 con 0x561a5d0194d0 2024-04-21T08:45:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.352+0000 7f7a6d391740 1 -- 172.21.15.43:0/2835583020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d0194d0 msgr2=0x561a5d02e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.352+0000 7f7a6d391740 1 --2- 172.21.15.43:0/2835583020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d0194d0 0x561a5d02e510 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f7a540096c0 tx=0x7f7a540df040 comp rx=0 tx=0).stop 2024-04-21T08:45:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.352+0000 7f7a6d391740 1 -- 172.21.15.43:0/2835583020 shutdown_connections 2024-04-21T08:45:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.352+0000 7f7a6d391740 1 --2- 172.21.15.43:0/2835583020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a5d022a00 0x561a5d022390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.352+0000 7f7a6d391740 1 --2- 172.21.15.43:0/2835583020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a5d019e80 0x561a5d021e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.352+0000 7f7a6d391740 1 --2- 172.21.15.43:0/2835583020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d0194d0 0x561a5d02e510 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.352+0000 7f7a6d391740 1 -- 172.21.15.43:0/2835583020 >> 172.21.15.43:0/2835583020 conn(0x561a5d013ba0 msgr2=0x561a5d012c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.352+0000 7f7a6d391740 1 -- 172.21.15.43:0/2835583020 shutdown_connections 2024-04-21T08:45:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.352+0000 7f7a6d391740 1 -- 172.21.15.43:0/2835583020 wait complete. 2024-04-21T08:45:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.352+0000 7f7a6d391740 1 -- start start 2024-04-21T08:45:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.352+0000 7f7a6d391740 1 Processor -- start 2024-04-21T08:45:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.352+0000 7f7a6d391740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a5d0194d0 0x561a5d01cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.352+0000 7f7a6d391740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d019e80 0x561a5d01cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.352+0000 7f7a6d391740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a5d022a00 0x561a5d097ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.352+0000 7f7a6d391740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7a641b08a0 con 0x561a5d019e80 2024-04-21T08:45:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.352+0000 7f7a6d391740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7a641b01d0 con 0x561a5d022a00 2024-04-21T08:45:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.352+0000 7f7a6d391740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7a641ad410 con 0x561a5d0194d0 2024-04-21T08:45:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.353+0000 7f7a6d391740 10 client.0 _ll_register_callbacks cb 0x7f7a58010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.353+0000 7f7a637fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d019e80 0x561a5d01cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.353+0000 7f7a637fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d019e80 0x561a5d01cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33870/0 (socket says 172.21.15.43:33870) 2024-04-21T08:45:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.353+0000 7f7a637fe640 1 -- 172.21.15.43:0/222877708 learned_addr learned my addr 172.21.15.43:0/222877708 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.353+0000 7f7a63fff640 1 --2- 172.21.15.43:0/222877708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a5d0194d0 0x561a5d01cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.353+0000 7f7a68d41640 1 --2- 172.21.15.43:0/222877708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a5d022a00 0x561a5d097ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.353+0000 7f7a637fe640 1 -- 172.21.15.43:0/222877708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a5d0194d0 msgr2=0x561a5d01cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:00.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.353+0000 7f7a637fe640 1 --2- 172.21.15.43:0/222877708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a5d0194d0 0x561a5d01cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.353+0000 7f7a637fe640 1 -- 172.21.15.43:0/222877708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a5d022a00 msgr2=0x561a5d097ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:00.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.353+0000 7f7a637fe640 1 --2- 172.21.15.43:0/222877708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a5d022a00 0x561a5d097ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.353+0000 7f7a637fe640 1 -- 172.21.15.43:0/222877708 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a5d098430 con 0x561a5d019e80 2024-04-21T08:45:00.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.353+0000 7f7a637fe640 1 --2- 172.21.15.43:0/222877708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d019e80 0x561a5d01cff0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f7a58016a90 tx=0x7f7a58016dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56876bdc4f586298 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:00.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.353+0000 7f7a437fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:00.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.353+0000 7f7a437fe640 1 -- 172.21.15.43:0/222877708 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5800e780 con 0x561a5d019e80 2024-04-21T08:45:00.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.353+0000 7f7a6d391740 1 -- 172.21.15.43:0/222877708 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a540bf3a0 con 0x561a5d019e80 2024-04-21T08:45:00.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.353+0000 7f7a437fe640 1 -- 172.21.15.43:0/222877708 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f7a58016330 con 0x561a5d019e80 2024-04-21T08:45:00.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.353+0000 7f7a437fe640 1 -- 172.21.15.43:0/222877708 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a5801d020 con 0x561a5d019e80 2024-04-21T08:45:00.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.353+0000 7f7a6d391740 10 client.18303 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:00.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.354+0000 7f7a6d391740 1 -- 172.21.15.43:0/222877708 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561a5d099610 con 0x561a5d019e80 2024-04-21T08:45:00.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.354+0000 7f7a437fe640 1 -- 172.21.15.43:0/222877708 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a540bf3a0 con 0x561a5d019e80 2024-04-21T08:45:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.354+0000 7f7a437fe640 1 -- 172.21.15.43:0/222877708 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7a58004850 con 0x561a5d019e80 2024-04-21T08:45:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.354+0000 7f7a6d391740 10 client.18303 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.354+0000 7f7a6d391740 10 client.18303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.354+0000 7f7a6d391740 20 client.18303 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.354+0000 7f7a6d391740 1 --2- 172.21.15.43:0/222877708 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561a5d099cd0 0x561a5d09c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.354+0000 7f7a6d391740 4 client.18303 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:45:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.354+0000 7f7a6d391740 1 -- 172.21.15.43:0/222877708 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a5cf85a80 con 0x561a5d099cd0 2024-04-21T08:45:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.354+0000 7f7a63fff640 1 --2- 172.21.15.43:0/222877708 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561a5d099cd0 0x561a5d09c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.354+0000 7f7a437fe640 10 client.18303 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.354+0000 7f7a63fff640 1 --2- 172.21.15.43:0/222877708 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561a5d099cd0 0x561a5d09c0c0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a0e1b07bfeb51e1 server_cookie=979d86106f259955 in_seq=0 out_seq=0 2024-04-21T08:45:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.355+0000 7f7a437fe640 1 -- 172.21.15.43:0/222877708 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a5cf85a80 con 0x561a5d099cd0 2024-04-21T08:45:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.355+0000 7f7a437fe640 10 client.18303 handle_command_reply: tid=0 2024-04-21T08:45:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.355+0000 7f7a6d391740 1 client.18303 shutdown 2024-04-21T08:45:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.355+0000 7f7a6d391740 1 -- 172.21.15.43:0/222877708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d019e80 msgr2=0x561a5d01cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.355+0000 7f7a6d391740 1 --2- 172.21.15.43:0/222877708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d019e80 0x561a5d01cff0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f7a58016a90 tx=0x7f7a58016dc0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.355+0000 7f7a6d391740 1 -- 172.21.15.43:0/222877708 shutdown_connections 2024-04-21T08:45:00.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.355+0000 7f7a6d391740 1 --2- 172.21.15.43:0/222877708 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561a5d099cd0 0x561a5d09c0c0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.355+0000 7f7a6d391740 1 --2- 172.21.15.43:0/222877708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a5d022a00 0x561a5d097ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.355+0000 7f7a6d391740 1 --2- 172.21.15.43:0/222877708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a5d019e80 0x561a5d01cff0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.355+0000 7f7a437fe640 0 client.18303 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:00.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.355+0000 7f7a6d391740 1 --2- 172.21.15.43:0/222877708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a5d0194d0 0x561a5d01cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.355+0000 7f7a6d391740 1 -- 172.21.15.43:0/222877708 >> 172.21.15.43:0/222877708 conn(0x561a5d013ba0 msgr2=0x561a5d01a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:00.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.355+0000 7f7a6d391740 1 -- 172.21.15.43:0/222877708 shutdown_connections 2024-04-21T08:45:00.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.355+0000 7f7a6d391740 1 -- 172.21.15.43:0/222877708 wait complete. 2024-04-21T08:45:00.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.355+0000 7f7a6d391740 20 client.18303 trim_cache size 0 max 16384 2024-04-21T08:45:00.363 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:45:00.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.356+0000 7f7a6a7a4640 1 -- 172.21.15.43:0/1914554493 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7a4c0a3d20 msgr2=0x7f7a4c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.356+0000 7f7a6a7a4640 1 --2- 172.21.15.43:0/1914554493 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7a4c0a3d20 0x7f7a4c0a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c014660 tx=0x7f7a5c014a60 comp rx=0 tx=0).stop 2024-04-21T08:45:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.356+0000 7f7a6a7a4640 1 -- 172.21.15.43:0/1914554493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a64101090 msgr2=0x7f7a64199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.356+0000 7f7a6a7a4640 1 --2- 172.21.15.43:0/1914554493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a64101090 0x7f7a64199660 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f7a54005940 tx=0x7f7a5402c860 comp rx=0 tx=0).stop 2024-04-21T08:45:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.356+0000 7f7a6a7a4640 1 -- 172.21.15.43:0/1914554493 shutdown_connections 2024-04-21T08:45:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.356+0000 7f7a6a7a4640 1 --2- 172.21.15.43:0/1914554493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a6419a0e0 0x7f7a641a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.356+0000 7f7a6a7a4640 1 --2- 172.21.15.43:0/1914554493 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7a4c0a3d20 0x7f7a4c0a61e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.356+0000 7f7a6a7a4640 1 --2- 172.21.15.43:0/1914554493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a64106160 0x7f7a64199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.356+0000 7f7a6a7a4640 1 --2- 172.21.15.43:0/1914554493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a64101090 0x7f7a64199660 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.356+0000 7f7a6a7a4640 1 -- 172.21.15.43:0/1914554493 >> 172.21.15.43:0/1914554493 conn(0x7f7a64077f50 msgr2=0x7f7a640ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.356+0000 7f7a6a7a4640 1 -- 172.21.15.43:0/1914554493 shutdown_connections 2024-04-21T08:45:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:00.356+0000 7f7a6a7a4640 1 -- 172.21.15.43:0/1914554493 wait complete. 2024-04-21T08:45:00.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:00 smithi175 ceph-mon[31920]: pgmap v185: 129 pgs: 129 active+clean; 4.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-21T08:45:00.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:00 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:45:00.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:00 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:00.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:00 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:45:00.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:00 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:00.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:00 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:45:00.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:00 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/4294086830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:00.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:00 smithi112 ceph-mon[31085]: pgmap v185: 129 pgs: 129 active+clean; 4.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-21T08:45:00.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:00 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:45:00.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:00 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:00.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:00 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:45:00.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:00 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:00.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:00 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:45:00.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:00 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/4294086830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:01.367 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:45:01.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:45:01.368 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:45:01.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.488+0000 7f300a56d640 1 -- 172.21.15.43:0/2705588192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30041031d0 msgr2=0x7f30041035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.488+0000 7f300a56d640 1 --2- 172.21.15.43:0/2705588192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30041031d0 0x7f30041035b0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009900 tx=0x7f2ff402bd60 comp rx=0 tx=0).stop 2024-04-21T08:45:01.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.488+0000 7f300a56d640 1 -- 172.21.15.43:0/2705588192 shutdown_connections 2024-04-21T08:45:01.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.488+0000 7f300a56d640 1 --2- 172.21.15.43:0/2705588192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f300410c510 0x7f300410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.488+0000 7f300a56d640 1 --2- 172.21.15.43:0/2705588192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3004103af0 0x7f300410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.488+0000 7f300a56d640 1 --2- 172.21.15.43:0/2705588192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30041031d0 0x7f30041035b0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.488+0000 7f300a56d640 1 -- 172.21.15.43:0/2705588192 >> 172.21.15.43:0/2705588192 conn(0x7f3004077f50 msgr2=0x7f3004078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:01.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.488+0000 7f300a56d640 1 -- 172.21.15.43:0/2705588192 shutdown_connections 2024-04-21T08:45:01.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.488+0000 7f300a56d640 1 -- 172.21.15.43:0/2705588192 wait complete. 2024-04-21T08:45:01.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.488+0000 7f300a56d640 1 Processor -- start 2024-04-21T08:45:01.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.488+0000 7f300a56d640 1 -- start start 2024-04-21T08:45:01.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.489+0000 7f300a56d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30041031d0 0x7f300419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.489+0000 7f300a56d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3004103af0 0x7f300419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.489+0000 7f300a56d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300410c510 0x7f30041985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.489+0000 7f300a56d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3004112180 con 0x7f300410c510 2024-04-21T08:45:01.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.489+0000 7f300a56d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3004112000 con 0x7f30041031d0 2024-04-21T08:45:01.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.489+0000 7f300a56d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3004112300 con 0x7f3004103af0 2024-04-21T08:45:01.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.489+0000 7f3008b0a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300410c510 0x7f30041985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.489+0000 7f3008b0a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300410c510 0x7f30041985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33892/0 (socket says 172.21.15.43:33892) 2024-04-21T08:45:01.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.489+0000 7f3008b0a640 1 -- 172.21.15.43:0/68776984 learned_addr learned my addr 172.21.15.43:0/68776984 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:01.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.489+0000 7f3003fff640 1 --2- 172.21.15.43:0/68776984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30041031d0 0x7f300419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.489+0000 7f30037fe640 1 --2- 172.21.15.43:0/68776984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3004103af0 0x7f300419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.489+0000 7f3008b0a640 1 -- 172.21.15.43:0/68776984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3004103af0 msgr2=0x7f300419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.489+0000 7f3008b0a640 1 --2- 172.21.15.43:0/68776984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3004103af0 0x7f300419e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.489+0000 7f3008b0a640 1 -- 172.21.15.43:0/68776984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30041031d0 msgr2=0x7f300419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.489+0000 7f3008b0a640 1 --2- 172.21.15.43:0/68776984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30041031d0 0x7f300419e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.489+0000 7f3008b0a640 1 -- 172.21.15.43:0/68776984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3004198d20 con 0x7f300410c510 2024-04-21T08:45:01.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.490+0000 7f3008b0a640 1 --2- 172.21.15.43:0/68776984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300410c510 0x7f30041985a0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009fd0 tx=0x7f2ffc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:01.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.490+0000 7f30017fa640 1 -- 172.21.15.43:0/68776984 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc017070 con 0x7f300410c510 2024-04-21T08:45:01.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.490+0000 7f30017fa640 1 -- 172.21.15.43:0/68776984 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ffc003e20 con 0x7f300410c510 2024-04-21T08:45:01.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.490+0000 7f300a56d640 1 -- 172.21.15.43:0/68776984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3004199010 con 0x7f300410c510 2024-04-21T08:45:01.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.490+0000 7f30017fa640 1 -- 172.21.15.43:0/68776984 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc010460 con 0x7f300410c510 2024-04-21T08:45:01.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.490+0000 7f300a56d640 1 -- 172.21.15.43:0/68776984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30041a52b0 con 0x7f300410c510 2024-04-21T08:45:01.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.492+0000 7f30017fa640 1 -- 172.21.15.43:0/68776984 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2ffc003970 con 0x7f300410c510 2024-04-21T08:45:01.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.492+0000 7f300d235740 1 Processor -- start 2024-04-21T08:45:01.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.493+0000 7f300d235740 1 -- start start 2024-04-21T08:45:01.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.493+0000 7f300d235740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbc3c78e0 0x560dbc3c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.493+0000 7f300d235740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560dbc3c8290 0x560dbc3d0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.493+0000 7f300d235740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3d0e10 0x560dbc3d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.493+0000 7f300d235740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560dbc32bf00 con 0x560dbc3d0e10 2024-04-21T08:45:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.493+0000 7f300d235740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560dbc2c9590 con 0x560dbc3c8290 2024-04-21T08:45:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.493+0000 7f300d235740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560dbc327ec0 con 0x560dbc3c78e0 2024-04-21T08:45:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.493+0000 7f30017fa640 1 --2- 172.21.15.43:0/68776984 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2fec0a3d20 0x7f2fec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.493+0000 7f30037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3d0e10 0x560dbc3d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.493+0000 7f30037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3d0e10 0x560dbc3d3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33906/0 (socket says 172.21.15.43:33906) 2024-04-21T08:45:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.493+0000 7f30037fe640 1 -- 172.21.15.43:0/4038860854 learned_addr learned my addr 172.21.15.43:0/4038860854 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.493+0000 7f30037fe640 1 --2- 172.21.15.43:0/4038860854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbc3c78e0 0x560dbc3c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.493+0000 7f3008b0a640 1 --2- 172.21.15.43:0/68776984 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2fec0a3d20 0x7f2fec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.493+0000 7f3003fff640 1 --2- 172.21.15.43:0/4038860854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560dbc3c8290 0x560dbc3d0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.493+0000 7f30017fa640 1 -- 172.21.15.43:0/68776984 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f2ffc0d08a0 con 0x7f300410c510 2024-04-21T08:45:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.493+0000 7f30037fe640 1 -- 172.21.15.43:0/4038860854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbc3c78e0 msgr2=0x560dbc3c7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:01.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.493+0000 7f30037fe640 1 --2- 172.21.15.43:0/4038860854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbc3c78e0 0x560dbc3c7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.493+0000 7f30037fe640 1 -- 172.21.15.43:0/4038860854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560dbc3c8290 msgr2=0x560dbc3d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:01.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.493+0000 7f30037fe640 1 --2- 172.21.15.43:0/4038860854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560dbc3c8290 0x560dbc3d0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.493+0000 7f30037fe640 1 -- 172.21.15.43:0/4038860854 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dbc1067e0 con 0x560dbc3d0e10 2024-04-21T08:45:01.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.494+0000 7f3008b0a640 1 --2- 172.21.15.43:0/68776984 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2fec0a3d20 0x7f2fec0a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00dfd0 tx=0x7f2ffc004020 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:01.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.494+0000 7f30037fe640 1 --2- 172.21.15.43:0/4038860854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3d0e10 0x560dbc3d3260 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8012770 tx=0x7f2ff8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49fec1ef40e2ae95 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:01.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.494+0000 7f2fea7fc640 1 -- 172.21.15.43:0/4038860854 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8012cf0 con 0x560dbc3d0e10 2024-04-21T08:45:01.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.494+0000 7f2fea7fc640 1 -- 172.21.15.43:0/4038860854 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff8010770 con 0x560dbc3d0e10 2024-04-21T08:45:01.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.494+0000 7f2fea7fc640 1 -- 172.21.15.43:0/4038860854 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff80044e0 con 0x560dbc3d0e10 2024-04-21T08:45:01.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.494+0000 7f300d235740 1 -- 172.21.15.43:0/4038860854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3d0e10 msgr2=0x560dbc3d3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.494+0000 7f300d235740 1 --2- 172.21.15.43:0/4038860854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3d0e10 0x560dbc3d3260 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8012770 tx=0x7f2ff8004730 comp rx=0 tx=0).stop 2024-04-21T08:45:01.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.495+0000 7f300d235740 1 -- 172.21.15.43:0/4038860854 shutdown_connections 2024-04-21T08:45:01.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.495+0000 7f300d235740 1 --2- 172.21.15.43:0/4038860854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3d0e10 0x560dbc3d3260 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.495+0000 7f300d235740 1 --2- 172.21.15.43:0/4038860854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560dbc3c8290 0x560dbc3d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.495+0000 7f300d235740 1 --2- 172.21.15.43:0/4038860854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbc3c78e0 0x560dbc3c7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.495+0000 7f300d235740 1 -- 172.21.15.43:0/4038860854 >> 172.21.15.43:0/4038860854 conn(0x560dbc3c1fb0 msgr2=0x560dbc3c2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.495+0000 7f300d235740 1 -- 172.21.15.43:0/4038860854 shutdown_connections 2024-04-21T08:45:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.495+0000 7f300d235740 1 -- 172.21.15.43:0/4038860854 wait complete. 2024-04-21T08:45:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.495+0000 7f300d235740 1 -- start start 2024-04-21T08:45:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.495+0000 7f300d235740 1 Processor -- start 2024-04-21T08:45:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.495+0000 7f300d235740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbc3c78e0 0x560dbc3cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.495+0000 7f300d235740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3c8290 0x560dbc3ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.495+0000 7f300d235740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560dbc3d0e10 0x560dbc3ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.495+0000 7f300d235740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560dbc3d3d80 con 0x560dbc3c8290 2024-04-21T08:45:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.495+0000 7f300d235740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560dbc2e5580 con 0x560dbc3d0e10 2024-04-21T08:45:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.495+0000 7f300d235740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560dbc3d3f00 con 0x560dbc3c78e0 2024-04-21T08:45:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.496+0000 7f300d235740 10 client.0 _ll_register_callbacks cb 0x560dbc2eb650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.496+0000 7f3003fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3c8290 0x560dbc3ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.496+0000 7f3003fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3c8290 0x560dbc3ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33912/0 (socket says 172.21.15.43:33912) 2024-04-21T08:45:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.496+0000 7f3003fff640 1 -- 172.21.15.43:0/2797421519 learned_addr learned my addr 172.21.15.43:0/2797421519 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.496+0000 7f30037fe640 1 --2- 172.21.15.43:0/2797421519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560dbc3d0e10 0x560dbc3ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.496+0000 7f30037fe640 1 --2- 172.21.15.43:0/2797421519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbc3c78e0 0x560dbc3cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.496+0000 7f3003fff640 1 -- 172.21.15.43:0/2797421519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbc3c78e0 msgr2=0x560dbc3cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.496+0000 7f3003fff640 1 --2- 172.21.15.43:0/2797421519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbc3c78e0 0x560dbc3cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.496+0000 7f3003fff640 1 -- 172.21.15.43:0/2797421519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560dbc3d0e10 msgr2=0x560dbc3ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.496+0000 7f3003fff640 1 --2- 172.21.15.43:0/2797421519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560dbc3d0e10 0x560dbc3ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.496+0000 7f3003fff640 1 -- 172.21.15.43:0/2797421519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dbc166f70 con 0x560dbc3c8290 2024-04-21T08:45:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.496+0000 7f30037fe640 1 --2- 172.21.15.43:0/2797421519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbc3c78e0 0x560dbc3cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.496+0000 7f3003fff640 1 --2- 172.21.15.43:0/2797421519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3c8290 0x560dbc3ce320 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009900 tx=0x7f2ff402db30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9dcbe73af6d296 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.496+0000 7f2fe37fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.496+0000 7f2fe37fe640 1 -- 172.21.15.43:0/2797421519 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4044030 con 0x560dbc3c8290 2024-04-21T08:45:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.496+0000 7f2fe37fe640 1 -- 172.21.15.43:0/2797421519 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f2ff403a040 con 0x560dbc3c8290 2024-04-21T08:45:01.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.496+0000 7f300d235740 1 -- 172.21.15.43:0/2797421519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ff8010770 con 0x560dbc3c8290 2024-04-21T08:45:01.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.496+0000 7f2fe37fe640 1 -- 172.21.15.43:0/2797421519 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2ff4051050 con 0x560dbc3c8290 2024-04-21T08:45:01.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.496+0000 7f300d235740 10 client.18315 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:01.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.497+0000 7f300d235740 1 -- 172.21.15.43:0/2797421519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560dbbffe170 con 0x560dbc3c8290 2024-04-21T08:45:01.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.497+0000 7f2fe37fe640 1 -- 172.21.15.43:0/2797421519 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff8010770 con 0x560dbc3c8290 2024-04-21T08:45:01.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.497+0000 7f2fe37fe640 1 -- 172.21.15.43:0/2797421519 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2ff40035f0 con 0x560dbc3c8290 2024-04-21T08:45:01.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.497+0000 7f300d235740 10 client.18315 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:01.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.497+0000 7f300d235740 10 client.18315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:01.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.497+0000 7f300d235740 20 client.18315 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:01.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.497+0000 7f300d235740 1 --2- 172.21.15.43:0/2797421519 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560dbc3e13e0 0x560dbc3e3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.497+0000 7f300d235740 4 client.18315 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:01.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.497+0000 7f300d235740 1 -- 172.21.15.43:0/2797421519 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dbc339920 con 0x560dbc3e13e0 2024-04-21T08:45:01.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.497+0000 7f30037fe640 1 --2- 172.21.15.43:0/2797421519 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560dbc3e13e0 0x560dbc3e3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.498+0000 7f2fe37fe640 10 client.18315 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:01.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.498+0000 7f30037fe640 1 --2- 172.21.15.43:0/2797421519 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560dbc3e13e0 0x560dbc3e3820 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e7090de66ae29e1 server_cookie=6577cb7321be5891 in_seq=0 out_seq=0 2024-04-21T08:45:01.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.498+0000 7f2fe37fe640 1 -- 172.21.15.43:0/2797421519 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560dbc339920 con 0x560dbc3e13e0 2024-04-21T08:45:01.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.498+0000 7f2fe37fe640 10 client.18315 handle_command_reply: tid=0 2024-04-21T08:45:01.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.498+0000 7f300d235740 1 client.18315 shutdown 2024-04-21T08:45:01.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.499+0000 7f300d235740 1 -- 172.21.15.43:0/2797421519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3c8290 msgr2=0x560dbc3ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.499+0000 7f300d235740 1 --2- 172.21.15.43:0/2797421519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3c8290 0x560dbc3ce320 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009900 tx=0x7f2ff402db30 comp rx=0 tx=0).stop 2024-04-21T08:45:01.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.499+0000 7f300d235740 1 -- 172.21.15.43:0/2797421519 shutdown_connections 2024-04-21T08:45:01.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.499+0000 7f300d235740 1 --2- 172.21.15.43:0/2797421519 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560dbc3e13e0 0x560dbc3e3820 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.499+0000 7f300d235740 1 --2- 172.21.15.43:0/2797421519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560dbc3d0e10 0x560dbc3ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.499+0000 7f300d235740 1 --2- 172.21.15.43:0/2797421519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3c8290 0x560dbc3ce320 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.499+0000 7f2fe37fe640 0 client.18315 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:01.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.499+0000 7f300d235740 1 --2- 172.21.15.43:0/2797421519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbc3c78e0 0x560dbc3cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.499+0000 7f300d235740 1 -- 172.21.15.43:0/2797421519 >> 172.21.15.43:0/2797421519 conn(0x560dbc3c1fb0 msgr2=0x560dbc3c3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:01.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.499+0000 7f300d235740 1 -- 172.21.15.43:0/2797421519 shutdown_connections 2024-04-21T08:45:01.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.499+0000 7f300d235740 1 -- 172.21.15.43:0/2797421519 wait complete. 2024-04-21T08:45:01.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.499+0000 7f300d235740 20 client.18315 trim_cache size 0 max 16384 2024-04-21T08:45:01.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.517+0000 7f300d235740 1 Processor -- start 2024-04-21T08:45:01.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.517+0000 7f300d235740 1 -- start start 2024-04-21T08:45:01.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.517+0000 7f300d235740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560dbc3c78e0 0x560dbc3d0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.517+0000 7f300d235740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3c8290 0x560dbc3ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.517+0000 7f300d235740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbc3d0e10 0x560dbc3dcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.517+0000 7f300d235740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f30041af490 con 0x560dbc3c8290 2024-04-21T08:45:01.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.517+0000 7f300d235740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f30041b0fd0 con 0x560dbc3c78e0 2024-04-21T08:45:01.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.517+0000 7f300d235740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f30041afb60 con 0x560dbc3d0e10 2024-04-21T08:45:01.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.517+0000 7f3003fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3c8290 0x560dbc3ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.517+0000 7f3003fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3c8290 0x560dbc3ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33928/0 (socket says 172.21.15.43:33928) 2024-04-21T08:45:01.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.517+0000 7f3003fff640 1 -- 172.21.15.43:0/3534075905 learned_addr learned my addr 172.21.15.43:0/3534075905 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.517+0000 7f30037fe640 1 --2- 172.21.15.43:0/3534075905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560dbc3c78e0 0x560dbc3d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.517+0000 7f30037fe640 1 --2- 172.21.15.43:0/3534075905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbc3d0e10 0x560dbc3dcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.517+0000 7f3003fff640 1 -- 172.21.15.43:0/3534075905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbc3d0e10 msgr2=0x560dbc3dcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.517+0000 7f3003fff640 1 --2- 172.21.15.43:0/3534075905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbc3d0e10 0x560dbc3dcc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.517+0000 7f3003fff640 1 -- 172.21.15.43:0/3534075905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560dbc3c78e0 msgr2=0x560dbc3d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.517+0000 7f3003fff640 1 --2- 172.21.15.43:0/3534075905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560dbc3c78e0 0x560dbc3d0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.517+0000 7f3003fff640 1 -- 172.21.15.43:0/3534075905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dbc3cc330 con 0x560dbc3c8290 2024-04-21T08:45:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.517+0000 7f3003fff640 1 --2- 172.21.15.43:0/3534075905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3c8290 0x560dbc3ccd20 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f2ff402c880 tx=0x7f2ff402bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77970f9ddeaa2096 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.518+0000 7f2fe9ffb640 1 -- 172.21.15.43:0/3534075905 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff404e070 con 0x560dbc3c8290 2024-04-21T08:45:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.518+0000 7f2fe9ffb640 1 -- 172.21.15.43:0/3534075905 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff4005700 con 0x560dbc3c8290 2024-04-21T08:45:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.518+0000 7f2fe9ffb640 1 -- 172.21.15.43:0/3534075905 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4044070 con 0x560dbc3c8290 2024-04-21T08:45:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.518+0000 7f300d235740 1 -- 172.21.15.43:0/3534075905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3c8290 msgr2=0x560dbc3ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.518+0000 7f300d235740 1 --2- 172.21.15.43:0/3534075905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3c8290 0x560dbc3ccd20 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f2ff402c880 tx=0x7f2ff402bd60 comp rx=0 tx=0).stop 2024-04-21T08:45:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.518+0000 7f300d235740 1 -- 172.21.15.43:0/3534075905 shutdown_connections 2024-04-21T08:45:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.518+0000 7f300d235740 1 --2- 172.21.15.43:0/3534075905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbc3d0e10 0x560dbc3dcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.518+0000 7f300d235740 1 --2- 172.21.15.43:0/3534075905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3c8290 0x560dbc3ccd20 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.518+0000 7f300d235740 1 --2- 172.21.15.43:0/3534075905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560dbc3c78e0 0x560dbc3d0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.518+0000 7f300d235740 1 -- 172.21.15.43:0/3534075905 >> 172.21.15.43:0/3534075905 conn(0x560dbc3c1fb0 msgr2=0x560dbc3ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.518+0000 7f300d235740 1 -- 172.21.15.43:0/3534075905 shutdown_connections 2024-04-21T08:45:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.518+0000 7f300d235740 1 -- 172.21.15.43:0/3534075905 wait complete. 2024-04-21T08:45:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.518+0000 7f300d235740 1 -- start start 2024-04-21T08:45:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.519+0000 7f300d235740 1 Processor -- start 2024-04-21T08:45:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.519+0000 7f300d235740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbc3c78e0 0x560dbc3d5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.519+0000 7f300d235740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3c8290 0x560dbc3d5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.519+0000 7f300d235740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560dbc3d0e10 0x560dbc4409a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.519+0000 7f300d235740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f30041b0900 con 0x560dbc3c8290 2024-04-21T08:45:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.519+0000 7f300d235740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f30041b0230 con 0x560dbc3d0e10 2024-04-21T08:45:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.519+0000 7f300d235740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f30041ad470 con 0x560dbc3c78e0 2024-04-21T08:45:01.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.519+0000 7f300d235740 10 client.0 _ll_register_callbacks cb 0x7f2ff4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:01.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.519+0000 7f3003fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3c8290 0x560dbc3d5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.519+0000 7f3003fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3c8290 0x560dbc3d5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33936/0 (socket says 172.21.15.43:33936) 2024-04-21T08:45:01.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.519+0000 7f3003fff640 1 -- 172.21.15.43:0/824825767 learned_addr learned my addr 172.21.15.43:0/824825767 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:01.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.519+0000 7f30037fe640 1 --2- 172.21.15.43:0/824825767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbc3c78e0 0x560dbc3d5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.519+0000 7f30037fe640 1 --2- 172.21.15.43:0/824825767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560dbc3d0e10 0x560dbc4409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.519+0000 7f3003fff640 1 -- 172.21.15.43:0/824825767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbc3c78e0 msgr2=0x560dbc3d5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:01.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.519+0000 7f3003fff640 1 --2- 172.21.15.43:0/824825767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbc3c78e0 0x560dbc3d5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.519+0000 7f3003fff640 1 -- 172.21.15.43:0/824825767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560dbc3d0e10 msgr2=0x560dbc4409a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:01.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.519+0000 7f3003fff640 1 --2- 172.21.15.43:0/824825767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560dbc3d0e10 0x560dbc4409a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.520+0000 7f3003fff640 1 -- 172.21.15.43:0/824825767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dbc440fe0 con 0x560dbc3c8290 2024-04-21T08:45:01.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.520+0000 7f3003fff640 1 --2- 172.21.15.43:0/824825767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3c8290 0x560dbc3d5950 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009900 tx=0x7f2ff402db30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3021d5fb3afc89 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:01.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.520+0000 7f2fe37fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:01.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.520+0000 7f2fe37fe640 1 -- 172.21.15.43:0/824825767 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4044070 con 0x560dbc3c8290 2024-04-21T08:45:01.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.520+0000 7f300d235740 1 -- 172.21.15.43:0/824825767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ff4005700 con 0x560dbc3c8290 2024-04-21T08:45:01.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.520+0000 7f2fe37fe640 1 -- 172.21.15.43:0/824825767 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f2ff4005450 con 0x560dbc3c8290 2024-04-21T08:45:01.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.520+0000 7f2fe37fe640 1 -- 172.21.15.43:0/824825767 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2ff4066050 con 0x560dbc3c8290 2024-04-21T08:45:01.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.520+0000 7f300d235740 10 client.18321 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:01.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.520+0000 7f300d235740 1 -- 172.21.15.43:0/824825767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560dbc4483d0 con 0x560dbc3c8290 2024-04-21T08:45:01.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.520+0000 7f2fe37fe640 1 -- 172.21.15.43:0/824825767 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff4005700 con 0x560dbc3c8290 2024-04-21T08:45:01.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.520+0000 7f2fe37fe640 1 -- 172.21.15.43:0/824825767 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2ff4040720 con 0x560dbc3c8290 2024-04-21T08:45:01.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.520+0000 7f300d235740 10 client.18321 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:01.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.520+0000 7f300d235740 10 client.18321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:01.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.520+0000 7f300d235740 20 client.18321 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:01.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.520+0000 7f300d235740 1 --2- 172.21.15.43:0/824825767 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560dbc448a30 0x560dbc44ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.520+0000 7f300d235740 4 client.18321 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:45:01.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.520+0000 7f300d235740 1 -- 172.21.15.43:0/824825767 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560dbc333790 con 0x560dbc448a30 2024-04-21T08:45:01.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.521+0000 7f30037fe640 1 --2- 172.21.15.43:0/824825767 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560dbc448a30 0x560dbc44ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.521+0000 7f30037fe640 1 --2- 172.21.15.43:0/824825767 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560dbc448a30 0x560dbc44ae50 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b0b45dc9604b377 server_cookie=8ce85b77a1faf31f in_seq=0 out_seq=0 2024-04-21T08:45:01.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.521+0000 7f2fe37fe640 10 client.18321 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:01.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.522+0000 7f2fe37fe640 1 -- 172.21.15.43:0/824825767 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560dbc333790 con 0x560dbc448a30 2024-04-21T08:45:01.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.522+0000 7f2fe37fe640 10 client.18321 handle_command_reply: tid=0 2024-04-21T08:45:01.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.522+0000 7f300d235740 1 client.18321 shutdown 2024-04-21T08:45:01.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.523+0000 7f300d235740 1 -- 172.21.15.43:0/824825767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3c8290 msgr2=0x560dbc3d5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.523+0000 7f300d235740 1 --2- 172.21.15.43:0/824825767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3c8290 0x560dbc3d5950 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009900 tx=0x7f2ff402db30 comp rx=0 tx=0).stop 2024-04-21T08:45:01.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.523+0000 7f300d235740 1 -- 172.21.15.43:0/824825767 shutdown_connections 2024-04-21T08:45:01.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.523+0000 7f300d235740 1 --2- 172.21.15.43:0/824825767 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560dbc448a30 0x560dbc44ae50 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.523+0000 7f300d235740 1 --2- 172.21.15.43:0/824825767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560dbc3d0e10 0x560dbc4409a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.523+0000 7f300d235740 1 --2- 172.21.15.43:0/824825767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560dbc3c8290 0x560dbc3d5950 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.523+0000 7f300d235740 1 --2- 172.21.15.43:0/824825767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560dbc3c78e0 0x560dbc3d5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.523+0000 7f300d235740 1 -- 172.21.15.43:0/824825767 >> 172.21.15.43:0/824825767 conn(0x560dbc3c1fb0 msgr2=0x560dbc3c0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:01.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.523+0000 7f2fe37fe640 0 client.18321 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:01.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.523+0000 7f300d235740 1 -- 172.21.15.43:0/824825767 shutdown_connections 2024-04-21T08:45:01.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.523+0000 7f300d235740 1 -- 172.21.15.43:0/824825767 wait complete. 2024-04-21T08:45:01.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.523+0000 7f300d235740 20 client.18321 trim_cache size 0 max 16384 2024-04-21T08:45:01.529 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:45:01.529 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:45:01.529 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "d5f4eb51-04a1-45d9-ae78-62b90d1bdc79", 2024-04-21T08:45:01.529 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:45:01.529 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:45:01.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.523+0000 7f300a56d640 1 -- 172.21.15.43:0/68776984 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2fec0a3d20 msgr2=0x7f2fec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.523+0000 7f300a56d640 1 --2- 172.21.15.43:0/68776984 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2fec0a3d20 0x7f2fec0a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00dfd0 tx=0x7f2ffc004020 comp rx=0 tx=0).stop 2024-04-21T08:45:01.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.523+0000 7f300a56d640 1 -- 172.21.15.43:0/68776984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300410c510 msgr2=0x7f30041985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.523+0000 7f300a56d640 1 --2- 172.21.15.43:0/68776984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300410c510 0x7f30041985a0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009fd0 tx=0x7f2ffc00b260 comp rx=0 tx=0).stop 2024-04-21T08:45:01.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.523+0000 7f300a56d640 1 -- 172.21.15.43:0/68776984 shutdown_connections 2024-04-21T08:45:01.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.523+0000 7f300a56d640 1 --2- 172.21.15.43:0/68776984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300410c510 0x7f30041985a0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.523+0000 7f300a56d640 1 --2- 172.21.15.43:0/68776984 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2fec0a3d20 0x7f2fec0a61e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.523+0000 7f300a56d640 1 --2- 172.21.15.43:0/68776984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3004103af0 0x7f300419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.523+0000 7f300a56d640 1 --2- 172.21.15.43:0/68776984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30041031d0 0x7f300419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.523+0000 7f300a56d640 1 -- 172.21.15.43:0/68776984 >> 172.21.15.43:0/68776984 conn(0x7f3004077f50 msgr2=0x7f30040fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:01.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.523+0000 7f300a56d640 1 -- 172.21.15.43:0/68776984 shutdown_connections 2024-04-21T08:45:01.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.523+0000 7f300a56d640 1 -- 172.21.15.43:0/68776984 wait complete. 2024-04-21T08:45:01.535 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:45:01.535 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:45:01.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.639+0000 7fae68c6d640 1 -- 172.21.15.43:0/1405245944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae64101a40 msgr2=0x7fae6410dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.639+0000 7fae68c6d640 1 --2- 172.21.15.43:0/1405245944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae64101a40 0x7fae6410dca0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009ca0 tx=0x7fae5c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.639+0000 7fae68c6d640 1 -- 172.21.15.43:0/1405245944 shutdown_connections 2024-04-21T08:45:01.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.639+0000 7fae68c6d640 1 --2- 172.21.15.43:0/1405245944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae64101a40 0x7fae6410dca0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.639+0000 7fae68c6d640 1 --2- 172.21.15.43:0/1405245944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae641010a0 0x7fae64101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.639+0000 7fae68c6d640 1 --2- 172.21.15.43:0/1405245944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae64105dd0 0x7fae641061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.639+0000 7fae68c6d640 1 -- 172.21.15.43:0/1405245944 >> 172.21.15.43:0/1405245944 conn(0x7fae64077c40 msgr2=0x7fae64078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:01.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.639+0000 7fae68c6d640 1 -- 172.21.15.43:0/1405245944 shutdown_connections 2024-04-21T08:45:01.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.639+0000 7fae68c6d640 1 -- 172.21.15.43:0/1405245944 wait complete. 2024-04-21T08:45:01.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.640+0000 7fae68c6d640 1 Processor -- start 2024-04-21T08:45:01.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.640+0000 7fae68c6d640 1 -- start start 2024-04-21T08:45:01.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.640+0000 7fae68c6d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae641010a0 0x7fae6419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.640+0000 7fae68c6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae64101a40 0x7fae6419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.640+0000 7fae68c6d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae64105dd0 0x7fae64198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.640+0000 7fae68c6d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fae641111d0 con 0x7fae64101a40 2024-04-21T08:45:01.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.640+0000 7fae68c6d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fae64111050 con 0x7fae641010a0 2024-04-21T08:45:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.640+0000 7fae68c6d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fae64111350 con 0x7fae64105dd0 2024-04-21T08:45:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.640+0000 7fae5bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae64101a40 0x7fae6419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.640+0000 7fae5bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae64101a40 0x7fae6419e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33954/0 (socket says 172.21.15.43:33954) 2024-04-21T08:45:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.640+0000 7fae5bfff640 1 -- 172.21.15.43:0/3652674499 learned_addr learned my addr 172.21.15.43:0/3652674499 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.640+0000 7fae62d9d640 1 --2- 172.21.15.43:0/3652674499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae64105dd0 0x7fae64198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.640+0000 7fae6259c640 1 --2- 172.21.15.43:0/3652674499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae641010a0 0x7fae6419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.641+0000 7fae5bfff640 1 -- 172.21.15.43:0/3652674499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae64105dd0 msgr2=0x7fae64198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.641+0000 7fae5bfff640 1 --2- 172.21.15.43:0/3652674499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae64105dd0 0x7fae64198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.641+0000 7fae5bfff640 1 -- 172.21.15.43:0/3652674499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae641010a0 msgr2=0x7fae6419e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.641+0000 7fae5bfff640 1 --2- 172.21.15.43:0/3652674499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae641010a0 0x7fae6419e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.641+0000 7fae5bfff640 1 -- 172.21.15.43:0/3652674499 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae64198d80 con 0x7fae64101a40 2024-04-21T08:45:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.641+0000 7fae5bfff640 1 --2- 172.21.15.43:0/3652674499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae64101a40 0x7fae6419e990 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00c890 tx=0x7fae4c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.641+0000 7fae5b7fe640 1 -- 172.21.15.43:0/3652674499 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c004220 con 0x7fae64101a40 2024-04-21T08:45:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.641+0000 7fae68c6d640 1 -- 172.21.15.43:0/3652674499 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae64199070 con 0x7fae64101a40 2024-04-21T08:45:01.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.641+0000 7fae5b7fe640 1 -- 172.21.15.43:0/3652674499 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae4c0043c0 con 0x7fae64101a40 2024-04-21T08:45:01.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.641+0000 7fae5b7fe640 1 -- 172.21.15.43:0/3652674499 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c004a00 con 0x7fae64101a40 2024-04-21T08:45:01.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.641+0000 7fae68c6d640 1 -- 172.21.15.43:0/3652674499 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae641a5310 con 0x7fae64101a40 2024-04-21T08:45:01.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.643+0000 7fae5b7fe640 1 -- 172.21.15.43:0/3652674499 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fae4c0079b0 con 0x7fae64101a40 2024-04-21T08:45:01.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.643+0000 7fae6b865740 1 Processor -- start 2024-04-21T08:45:01.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.644+0000 7fae6b865740 1 -- start start 2024-04-21T08:45:01.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.644+0000 7fae6b865740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6580 0x55e9a6ae6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.644+0000 7fae6b865740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9a6ae6ea0 0x55e9a6aef540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.644+0000 7fae6b865740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a6aefa80 0x55e9a6af1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.644+0000 7fae5b7fe640 1 --2- 172.21.15.43:0/3652674499 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fae500a3d20 0x7fae500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.644+0000 7fae6b865740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e9a6a4b1e0 con 0x55e9a6ae6580 2024-04-21T08:45:01.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.644+0000 7fae6b865740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e9a69e8390 con 0x55e9a6aefa80 2024-04-21T08:45:01.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.644+0000 7fae6b865740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e9a6a471a0 con 0x55e9a6ae6ea0 2024-04-21T08:45:01.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.644+0000 7fae62d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6580 0x55e9a6ae6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.644+0000 7fae62d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6580 0x55e9a6ae6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33962/0 (socket says 172.21.15.43:33962) 2024-04-21T08:45:01.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.644+0000 7fae62d9d640 1 -- 172.21.15.43:0/1302502106 learned_addr learned my addr 172.21.15.43:0/1302502106 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:01.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.644+0000 7fae62d9d640 1 --2- 172.21.15.43:0/3652674499 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fae500a3d20 0x7fae500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.644+0000 7fae5b7fe640 1 -- 172.21.15.43:0/3652674499 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fae4c0d0810 con 0x7fae64101a40 2024-04-21T08:45:01.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.644+0000 7fae5bfff640 1 --2- 172.21.15.43:0/1302502106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a6aefa80 0x55e9a6af1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.644+0000 7fae6259c640 1 --2- 172.21.15.43:0/1302502106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9a6ae6ea0 0x55e9a6aef540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.644+0000 7fae62d9d640 1 -- 172.21.15.43:0/1302502106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9a6ae6ea0 msgr2=0x55e9a6aef540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:01.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.644+0000 7fae62d9d640 1 --2- 172.21.15.43:0/1302502106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9a6ae6ea0 0x55e9a6aef540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.644+0000 7fae62d9d640 1 -- 172.21.15.43:0/1302502106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a6aefa80 msgr2=0x55e9a6af1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:01.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.644+0000 7fae62d9d640 1 --2- 172.21.15.43:0/1302502106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a6aefa80 0x55e9a6af1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.644+0000 7fae62d9d640 1 -- 172.21.15.43:0/1302502106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9a68277d0 con 0x55e9a6ae6580 2024-04-21T08:45:01.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.645+0000 7fae62d9d640 1 --2- 172.21.15.43:0/1302502106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6580 0x55e9a6ae6960 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fae5c037040 tx=0x7fae5c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec3a67fc2c6c0c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:01.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.645+0000 7fae62d9d640 1 --2- 172.21.15.43:0/3652674499 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fae500a3d20 0x7fae500a61e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009a60 tx=0x7fae5c042000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:01.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.645+0000 7fae58ff9640 1 -- 172.21.15.43:0/1302502106 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c051070 con 0x55e9a6ae6580 2024-04-21T08:45:01.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.645+0000 7fae58ff9640 1 -- 172.21.15.43:0/1302502106 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae5c004040 con 0x55e9a6ae6580 2024-04-21T08:45:01.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.645+0000 7fae58ff9640 1 -- 172.21.15.43:0/1302502106 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c039040 con 0x55e9a6ae6580 2024-04-21T08:45:01.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.646+0000 7fae6b865740 1 -- 172.21.15.43:0/1302502106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6580 msgr2=0x55e9a6ae6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.646+0000 7fae6b865740 1 --2- 172.21.15.43:0/1302502106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6580 0x55e9a6ae6960 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fae5c037040 tx=0x7fae5c032670 comp rx=0 tx=0).stop 2024-04-21T08:45:01.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.646+0000 7fae6b865740 1 -- 172.21.15.43:0/1302502106 shutdown_connections 2024-04-21T08:45:01.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.646+0000 7fae6b865740 1 --2- 172.21.15.43:0/1302502106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a6aefa80 0x55e9a6af1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.646+0000 7fae6b865740 1 --2- 172.21.15.43:0/1302502106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9a6ae6ea0 0x55e9a6aef540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.646+0000 7fae6b865740 1 --2- 172.21.15.43:0/1302502106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6580 0x55e9a6ae6960 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.646+0000 7fae6b865740 1 -- 172.21.15.43:0/1302502106 >> 172.21.15.43:0/1302502106 conn(0x55e9a6ae0cb0 msgr2=0x55e9a6ae1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:01.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.646+0000 7fae6b865740 1 -- 172.21.15.43:0/1302502106 shutdown_connections 2024-04-21T08:45:01.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.646+0000 7fae6b865740 1 -- 172.21.15.43:0/1302502106 wait complete. 2024-04-21T08:45:01.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.646+0000 7fae6b865740 1 -- start start 2024-04-21T08:45:01.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.646+0000 7fae6b865740 1 Processor -- start 2024-04-21T08:45:01.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.646+0000 7fae6b865740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6580 0x55e9a6aecad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.646+0000 7fae6b865740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9a6ae6ea0 0x55e9a6aed010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.646+0000 7fae6b865740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a6aefa80 0x55e9a6aed550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.646+0000 7fae6b865740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e9a6af2810 con 0x55e9a6ae6580 2024-04-21T08:45:01.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.646+0000 7fae6b865740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e9a6a042f0 con 0x55e9a6aefa80 2024-04-21T08:45:01.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.646+0000 7fae6b865740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e9a6af2990 con 0x55e9a6ae6ea0 2024-04-21T08:45:01.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.646+0000 7fae6b865740 10 client.0 _ll_register_callbacks cb 0x55e9a6a11580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:01.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.647+0000 7fae6259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6580 0x55e9a6aecad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.647+0000 7fae6259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6580 0x55e9a6aecad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33976/0 (socket says 172.21.15.43:33976) 2024-04-21T08:45:01.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.647+0000 7fae6259c640 1 -- 172.21.15.43:0/3620255235 learned_addr learned my addr 172.21.15.43:0/3620255235 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:01.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.647+0000 7fae5bfff640 1 --2- 172.21.15.43:0/3620255235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9a6ae6ea0 0x55e9a6aed010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.647+0000 7fae62d9d640 1 --2- 172.21.15.43:0/3620255235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a6aefa80 0x55e9a6aed550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.647+0000 7fae6259c640 1 -- 172.21.15.43:0/3620255235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9a6ae6ea0 msgr2=0x55e9a6aed010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:01.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.647+0000 7fae6259c640 1 --2- 172.21.15.43:0/3620255235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9a6ae6ea0 0x55e9a6aed010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.647+0000 7fae6259c640 1 -- 172.21.15.43:0/3620255235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a6aefa80 msgr2=0x55e9a6aed550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:01.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.647+0000 7fae6259c640 1 --2- 172.21.15.43:0/3620255235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a6aefa80 0x55e9a6aed550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.647+0000 7fae6259c640 1 -- 172.21.15.43:0/3620255235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9a675d3f0 con 0x55e9a6ae6580 2024-04-21T08:45:01.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.647+0000 7fae6259c640 1 --2- 172.21.15.43:0/3620255235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6580 0x55e9a6aecad0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fae54011570 tx=0x7fae54011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b13d680fa826ee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:01.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.647+0000 7fae49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:01.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.647+0000 7fae49ffb640 1 -- 172.21.15.43:0/3620255235 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5400f6d0 con 0x55e9a6ae6580 2024-04-21T08:45:01.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.647+0000 7fae6b865740 1 -- 172.21.15.43:0/3620255235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae5c004040 con 0x55e9a6ae6580 2024-04-21T08:45:01.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.647+0000 7fae49ffb640 1 -- 172.21.15.43:0/3620255235 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fae5400f870 con 0x55e9a6ae6580 2024-04-21T08:45:01.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.647+0000 7fae49ffb640 1 -- 172.21.15.43:0/3620255235 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fae54026050 con 0x55e9a6ae6580 2024-04-21T08:45:01.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.647+0000 7fae6b865740 10 client.18333 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:01.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.647+0000 7fae6b865740 1 -- 172.21.15.43:0/3620255235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e9a671d0f0 con 0x55e9a6ae6580 2024-04-21T08:45:01.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.648+0000 7fae49ffb640 1 -- 172.21.15.43:0/3620255235 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae5c004040 con 0x55e9a6ae6580 2024-04-21T08:45:01.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.648+0000 7fae49ffb640 1 -- 172.21.15.43:0/3620255235 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fae54014be0 con 0x55e9a6ae6580 2024-04-21T08:45:01.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.648+0000 7fae6b865740 10 client.18333 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:01.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.648+0000 7fae6b865740 10 client.18333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:01.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.648+0000 7fae6b865740 20 client.18333 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:01.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.648+0000 7fae6b865740 1 --2- 172.21.15.43:0/3620255235 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e9a6affa90 0x55e9a6b01ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.648+0000 7fae6b865740 4 client.18333 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:01.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.648+0000 7fae6b865740 1 -- 172.21.15.43:0/3620255235 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9a6a58c00 con 0x55e9a6affa90 2024-04-21T08:45:01.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.648+0000 7fae5bfff640 1 --2- 172.21.15.43:0/3620255235 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e9a6affa90 0x55e9a6b01ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.648+0000 7fae5bfff640 1 --2- 172.21.15.43:0/3620255235 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e9a6affa90 0x55e9a6b01ee0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1788f6e3e3ee63b3 server_cookie=3dd1a65826ba21cb in_seq=0 out_seq=0 2024-04-21T08:45:01.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.648+0000 7fae49ffb640 10 client.18333 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:01.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.649+0000 7fae49ffb640 1 -- 172.21.15.43:0/3620255235 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e9a6a58c00 con 0x55e9a6affa90 2024-04-21T08:45:01.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.649+0000 7fae49ffb640 10 client.18333 handle_command_reply: tid=0 2024-04-21T08:45:01.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.649+0000 7fae6b865740 1 client.18333 shutdown 2024-04-21T08:45:01.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.649+0000 7fae6b865740 1 -- 172.21.15.43:0/3620255235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6580 msgr2=0x55e9a6aecad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.649+0000 7fae6b865740 1 --2- 172.21.15.43:0/3620255235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6580 0x55e9a6aecad0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fae54011570 tx=0x7fae54011a30 comp rx=0 tx=0).stop 2024-04-21T08:45:01.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.649+0000 7fae6b865740 1 -- 172.21.15.43:0/3620255235 shutdown_connections 2024-04-21T08:45:01.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.649+0000 7fae6b865740 1 --2- 172.21.15.43:0/3620255235 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e9a6affa90 0x55e9a6b01ee0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.649+0000 7fae6b865740 1 --2- 172.21.15.43:0/3620255235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a6aefa80 0x55e9a6aed550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.649+0000 7fae6b865740 1 --2- 172.21.15.43:0/3620255235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9a6ae6ea0 0x55e9a6aed010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.649+0000 7fae6b865740 1 --2- 172.21.15.43:0/3620255235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6580 0x55e9a6aecad0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.649+0000 7fae49ffb640 0 client.18333 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:01.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.649+0000 7fae6b865740 1 -- 172.21.15.43:0/3620255235 >> 172.21.15.43:0/3620255235 conn(0x55e9a6ae0cb0 msgr2=0x55e9a6ae2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:01.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.649+0000 7fae6b865740 1 -- 172.21.15.43:0/3620255235 shutdown_connections 2024-04-21T08:45:01.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.649+0000 7fae6b865740 1 -- 172.21.15.43:0/3620255235 wait complete. 2024-04-21T08:45:01.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.650+0000 7fae6b865740 20 client.18333 trim_cache size 0 max 16384 2024-04-21T08:45:01.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.667+0000 7fae6b865740 1 Processor -- start 2024-04-21T08:45:01.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.667+0000 7fae6b865740 1 -- start start 2024-04-21T08:45:01.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.667+0000 7fae6b865740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6580 0x55e9a6aef190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.667+0000 7fae6b865740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a6ae6ea0 0x55e9a6afb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.667+0000 7fae6b865740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9a6aefa80 0x55e9a6aebc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.667+0000 7fae6b865740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fae641af480 con 0x55e9a6ae6580 2024-04-21T08:45:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.667+0000 7fae6b865740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fae641b0fc0 con 0x55e9a6ae6ea0 2024-04-21T08:45:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.667+0000 7fae6b865740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fae641afb50 con 0x55e9a6aefa80 2024-04-21T08:45:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.667+0000 7fae6259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6580 0x55e9a6aef190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.667+0000 7fae6259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6580 0x55e9a6aef190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33986/0 (socket says 172.21.15.43:33986) 2024-04-21T08:45:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.667+0000 7fae6259c640 1 -- 172.21.15.43:0/484446878 learned_addr learned my addr 172.21.15.43:0/484446878 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.668+0000 7fae5bfff640 1 --2- 172.21.15.43:0/484446878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a6ae6ea0 0x55e9a6afb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.668+0000 7fae62d9d640 1 --2- 172.21.15.43:0/484446878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9a6aefa80 0x55e9a6aebc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.668+0000 7fae6259c640 1 -- 172.21.15.43:0/484446878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9a6aefa80 msgr2=0x55e9a6aebc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.668+0000 7fae6259c640 1 --2- 172.21.15.43:0/484446878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9a6aefa80 0x55e9a6aebc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.668+0000 7fae6259c640 1 -- 172.21.15.43:0/484446878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a6ae6ea0 msgr2=0x55e9a6afb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.668+0000 7fae6259c640 1 --2- 172.21.15.43:0/484446878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a6ae6ea0 0x55e9a6afb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.668+0000 7fae6259c640 1 -- 172.21.15.43:0/484446878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9a6af1e70 con 0x55e9a6ae6580 2024-04-21T08:45:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.668+0000 7fae62d9d640 1 --2- 172.21.15.43:0/484446878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9a6aefa80 0x55e9a6aebc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:01.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.668+0000 7fae6259c640 1 --2- 172.21.15.43:0/484446878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6580 0x55e9a6aef190 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fae54000c00 tx=0x7fae5400f270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333c116e0c490ba6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:01.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.668+0000 7fae4bfff640 1 -- 172.21.15.43:0/484446878 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5402f070 con 0x55e9a6ae6580 2024-04-21T08:45:01.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.668+0000 7fae4bfff640 1 -- 172.21.15.43:0/484446878 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae5400f6d0 con 0x55e9a6ae6580 2024-04-21T08:45:01.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.668+0000 7fae4bfff640 1 -- 172.21.15.43:0/484446878 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae54014680 con 0x55e9a6ae6580 2024-04-21T08:45:01.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.668+0000 7fae6b865740 1 -- 172.21.15.43:0/484446878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6580 msgr2=0x55e9a6aef190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.668+0000 7fae6b865740 1 --2- 172.21.15.43:0/484446878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6580 0x55e9a6aef190 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fae54000c00 tx=0x7fae5400f270 comp rx=0 tx=0).stop 2024-04-21T08:45:01.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.668+0000 7fae6b865740 1 -- 172.21.15.43:0/484446878 shutdown_connections 2024-04-21T08:45:01.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.668+0000 7fae6b865740 1 --2- 172.21.15.43:0/484446878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9a6aefa80 0x55e9a6aebc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.668+0000 7fae6b865740 1 --2- 172.21.15.43:0/484446878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a6ae6ea0 0x55e9a6afb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.668+0000 7fae6b865740 1 --2- 172.21.15.43:0/484446878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6580 0x55e9a6aef190 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.669+0000 7fae6b865740 1 -- 172.21.15.43:0/484446878 >> 172.21.15.43:0/484446878 conn(0x55e9a6ae0cb0 msgr2=0x55e9a6adf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:01.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.669+0000 7fae6b865740 1 -- 172.21.15.43:0/484446878 shutdown_connections 2024-04-21T08:45:01.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.669+0000 7fae6b865740 1 -- 172.21.15.43:0/484446878 wait complete. 2024-04-21T08:45:01.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.669+0000 7fae6b865740 1 -- start start 2024-04-21T08:45:01.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.669+0000 7fae6b865740 1 Processor -- start 2024-04-21T08:45:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.669+0000 7fae6b865740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a6ae6580 0x55e9a6ae9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.669+0000 7fae6b865740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6ea0 0x55e9a6ae9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.669+0000 7fae6b865740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9a6aefa80 0x55e9a6b65020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.669+0000 7fae6b865740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fae641b08f0 con 0x55e9a6ae6ea0 2024-04-21T08:45:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.669+0000 7fae6b865740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fae641b0220 con 0x55e9a6ae6580 2024-04-21T08:45:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.669+0000 7fae6b865740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fae641ad460 con 0x55e9a6aefa80 2024-04-21T08:45:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.669+0000 7fae6b865740 10 client.0 _ll_register_callbacks cb 0x7fae54009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.669+0000 7fae5bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6ea0 0x55e9a6ae9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.669+0000 7fae5bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6ea0 0x55e9a6ae9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33996/0 (socket says 172.21.15.43:33996) 2024-04-21T08:45:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.669+0000 7fae5bfff640 1 -- 172.21.15.43:0/3873923107 learned_addr learned my addr 172.21.15.43:0/3873923107 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.669+0000 7fae6259c640 1 --2- 172.21.15.43:0/3873923107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a6ae6580 0x55e9a6ae9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.669+0000 7fae62d9d640 1 --2- 172.21.15.43:0/3873923107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9a6aefa80 0x55e9a6b65020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.670+0000 7fae5bfff640 1 -- 172.21.15.43:0/3873923107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9a6aefa80 msgr2=0x55e9a6b65020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.670+0000 7fae5bfff640 1 --2- 172.21.15.43:0/3873923107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9a6aefa80 0x55e9a6b65020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.670+0000 7fae5bfff640 1 -- 172.21.15.43:0/3873923107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a6ae6580 msgr2=0x55e9a6ae9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.670+0000 7fae5bfff640 1 --2- 172.21.15.43:0/3873923107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a6ae6580 0x55e9a6ae9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.670+0000 7fae5bfff640 1 -- 172.21.15.43:0/3873923107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9a6aea870 con 0x55e9a6ae6ea0 2024-04-21T08:45:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.670+0000 7fae5bfff640 1 --2- 172.21.15.43:0/3873923107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6ea0 0x55e9a6ae9fe0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0cdb90 tx=0x7fae4c09bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c1b026bf234154 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.670+0000 7fae49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.670+0000 7fae49ffb640 1 -- 172.21.15.43:0/3873923107 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c01d040 con 0x55e9a6ae6ea0 2024-04-21T08:45:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.670+0000 7fae49ffb640 1 -- 172.21.15.43:0/3873923107 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fae4c09a680 con 0x55e9a6ae6ea0 2024-04-21T08:45:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.670+0000 7fae6b865740 1 -- 172.21.15.43:0/3873923107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae5400f6d0 con 0x55e9a6ae6ea0 2024-04-21T08:45:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.670+0000 7fae49ffb640 1 -- 172.21.15.43:0/3873923107 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fae4c09c380 con 0x55e9a6ae6ea0 2024-04-21T08:45:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.670+0000 7fae6b865740 10 client.18339 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.670+0000 7fae6b865740 1 -- 172.21.15.43:0/3873923107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e9a6b665b0 con 0x55e9a6ae6ea0 2024-04-21T08:45:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.670+0000 7fae49ffb640 1 -- 172.21.15.43:0/3873923107 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae5400f6d0 con 0x55e9a6ae6ea0 2024-04-21T08:45:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.670+0000 7fae49ffb640 1 -- 172.21.15.43:0/3873923107 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fae4c0d0040 con 0x55e9a6ae6ea0 2024-04-21T08:45:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.670+0000 7fae6b865740 10 client.18339 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.670+0000 7fae6b865740 10 client.18339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.670+0000 7fae6b865740 20 client.18339 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.670+0000 7fae6b865740 1 --2- 172.21.15.43:0/3873923107 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e9a6b66cb0 0x55e9a6b690d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.670+0000 7fae6b865740 4 client.18339 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:45:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.670+0000 7fae6b865740 1 -- 172.21.15.43:0/3873923107 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e9a6a52a70 con 0x55e9a6b66cb0 2024-04-21T08:45:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.671+0000 7fae6259c640 1 --2- 172.21.15.43:0/3873923107 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e9a6b66cb0 0x55e9a6b690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.671+0000 7fae49ffb640 10 client.18339 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.671+0000 7fae6259c640 1 --2- 172.21.15.43:0/3873923107 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e9a6b66cb0 0x55e9a6b690d0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=859806e7697c8859 server_cookie=ab2a95d244fc8177 in_seq=0 out_seq=0 2024-04-21T08:45:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.671+0000 7fae49ffb640 1 -- 172.21.15.43:0/3873923107 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e9a6a52a70 con 0x55e9a6b66cb0 2024-04-21T08:45:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.671+0000 7fae49ffb640 10 client.18339 handle_command_reply: tid=0 2024-04-21T08:45:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.671+0000 7fae6b865740 1 client.18339 shutdown 2024-04-21T08:45:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.671+0000 7fae6b865740 1 -- 172.21.15.43:0/3873923107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6ea0 msgr2=0x55e9a6ae9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.671+0000 7fae6b865740 1 --2- 172.21.15.43:0/3873923107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6ea0 0x55e9a6ae9fe0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0cdb90 tx=0x7fae4c09bf80 comp rx=0 tx=0).stop 2024-04-21T08:45:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.671+0000 7fae6b865740 1 -- 172.21.15.43:0/3873923107 shutdown_connections 2024-04-21T08:45:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.671+0000 7fae6b865740 1 --2- 172.21.15.43:0/3873923107 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e9a6b66cb0 0x55e9a6b690d0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.671+0000 7fae6b865740 1 --2- 172.21.15.43:0/3873923107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9a6aefa80 0x55e9a6b65020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.671+0000 7fae49ffb640 0 client.18339 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:01.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.671+0000 7fae6b865740 1 --2- 172.21.15.43:0/3873923107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9a6ae6ea0 0x55e9a6ae9fe0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.671+0000 7fae6b865740 1 --2- 172.21.15.43:0/3873923107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a6ae6580 0x55e9a6ae9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.671+0000 7fae6b865740 1 -- 172.21.15.43:0/3873923107 >> 172.21.15.43:0/3873923107 conn(0x55e9a6ae0cb0 msgr2=0x55e9a6aeca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:01.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.672+0000 7fae6b865740 1 -- 172.21.15.43:0/3873923107 shutdown_connections 2024-04-21T08:45:01.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.672+0000 7fae6b865740 1 -- 172.21.15.43:0/3873923107 wait complete. 2024-04-21T08:45:01.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.672+0000 7fae6b865740 20 client.18339 trim_cache size 0 max 16384 2024-04-21T08:45:01.679 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:45:01.679 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:45:01.680 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-21T08:45:01.680 INFO:teuthology.orchestra.run.smithi043.stdout: "d5f4eb51-04a1-45d9-ae78-62b90d1bdc79": { 2024-04-21T08:45:01.680 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-21T08:45:01.680 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "d5f4eb51-04a1-45d9-ae78-62b90d1bdc79", 2024-04-21T08:45:01.680 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-21T08:45:01.680 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:45:01.680 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:45:01.680 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:45:01.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.672+0000 7fae68c6d640 1 -- 172.21.15.43:0/3652674499 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fae500a3d20 msgr2=0x7fae500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.672+0000 7fae68c6d640 1 --2- 172.21.15.43:0/3652674499 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fae500a3d20 0x7fae500a61e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009a60 tx=0x7fae5c042000 comp rx=0 tx=0).stop 2024-04-21T08:45:01.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.672+0000 7fae68c6d640 1 -- 172.21.15.43:0/3652674499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae64101a40 msgr2=0x7fae6419e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.672+0000 7fae68c6d640 1 --2- 172.21.15.43:0/3652674499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae64101a40 0x7fae6419e990 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00c890 tx=0x7fae4c00cc90 comp rx=0 tx=0).stop 2024-04-21T08:45:01.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.672+0000 7fae68c6d640 1 -- 172.21.15.43:0/3652674499 shutdown_connections 2024-04-21T08:45:01.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.672+0000 7fae68c6d640 1 --2- 172.21.15.43:0/3652674499 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fae500a3d20 0x7fae500a61e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.672+0000 7fae68c6d640 1 --2- 172.21.15.43:0/3652674499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae64105dd0 0x7fae64198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.672+0000 7fae68c6d640 1 --2- 172.21.15.43:0/3652674499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae64101a40 0x7fae6419e990 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.672+0000 7fae68c6d640 1 --2- 172.21.15.43:0/3652674499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae641010a0 0x7fae6419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.672+0000 7fae68c6d640 1 -- 172.21.15.43:0/3652674499 >> 172.21.15.43:0/3652674499 conn(0x7fae64077c40 msgr2=0x7fae6406a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:01.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.672+0000 7fae68c6d640 1 -- 172.21.15.43:0/3652674499 shutdown_connections 2024-04-21T08:45:01.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:01.672+0000 7fae68c6d640 1 -- 172.21.15.43:0/3652674499 wait complete. 2024-04-21T08:45:01.683 INFO:tasks.cephfs.filesystem:scrub status for tag:d5f4eb51-04a1-45d9-ae78-62b90d1bdc79 - {'path': '/', 'tag': 'd5f4eb51-04a1-45d9-ae78-62b90d1bdc79', 'options': 'recursive,force'} 2024-04-21T08:45:02.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:02 smithi175 ceph-mon[31920]: pgmap v186: 129 pgs: 129 active+clean; 5.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 73 KiB/s wr, 38 op/s 2024-04-21T08:45:02.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:02 smithi112 ceph-mon[31085]: pgmap v186: 129 pgs: 129 active+clean; 5.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 73 KiB/s wr, 38 op/s 2024-04-21T08:45:02.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:02 smithi043 ceph-mon[27978]: pgmap v186: 129 pgs: 129 active+clean; 5.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 73 KiB/s wr, 38 op/s 2024-04-21T08:45:03.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:03 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:45:03.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:03 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:03.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:03 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:45:03.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:03 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:03.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:03 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:45:03.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:03 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:03.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:03 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:45:03.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:03 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:03.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:03 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:45:03.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:03 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:03.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:03 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:45:03.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:03 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:04.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:04 smithi175 ceph-mon[31920]: pgmap v187: 129 pgs: 129 active+clean; 5.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 58 KiB/s wr, 31 op/s 2024-04-21T08:45:04.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:04 smithi112 ceph-mon[31085]: pgmap v187: 129 pgs: 129 active+clean; 5.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 58 KiB/s wr, 31 op/s 2024-04-21T08:45:04.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:04 smithi043 ceph-mon[27978]: pgmap v187: 129 pgs: 129 active+clean; 5.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 58 KiB/s wr, 31 op/s 2024-04-21T08:45:05.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:05 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:45:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:05 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:45:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:05 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:45:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:05 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:45:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:05 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:45:05.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:05 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:45:05.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:05 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:45:05.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:05 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:45:05.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:05 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:45:05.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:05 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:45:05.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:05 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:45:05.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:05 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:45:05.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:05 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:45:05.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:05 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:45:05.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:05 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:45:06.463 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:06 smithi043 ceph-mon[27978]: pgmap v188: 129 pgs: 129 active+clean; 5.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 58 KiB/s wr, 31 op/s 2024-04-21T08:45:06.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:06 smithi175 ceph-mon[31920]: pgmap v188: 129 pgs: 129 active+clean; 5.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 58 KiB/s wr, 31 op/s 2024-04-21T08:45:06.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:06 smithi112 ceph-mon[31085]: pgmap v188: 129 pgs: 129 active+clean; 5.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 58 KiB/s wr, 31 op/s 2024-04-21T08:45:08.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:08 smithi175 ceph-mon[31920]: pgmap v189: 129 pgs: 129 active+clean; 5.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-04-21T08:45:08.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:08 smithi112 ceph-mon[31085]: pgmap v189: 129 pgs: 129 active+clean; 5.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-04-21T08:45:08.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:08 smithi043 ceph-mon[27978]: pgmap v189: 129 pgs: 129 active+clean; 5.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-04-21T08:45:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.137+0000 7f9a3cec0640 1 -- 172.21.15.43:0/2690549673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a381021d0 msgr2=0x7f9a381025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.137+0000 7f9a3cec0640 1 --2- 172.21.15.43:0/2690549673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a381021d0 0x7f9a381025b0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009900 tx=0x7f9a2802bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.137+0000 7f9a3cec0640 1 -- 172.21.15.43:0/2690549673 shutdown_connections 2024-04-21T08:45:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.137+0000 7f9a3cec0640 1 --2- 172.21.15.43:0/2690549673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a381072a0 0x7f9a3810ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.137+0000 7f9a3cec0640 1 --2- 172.21.15.43:0/2690549673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a38102b80 0x7f9a38106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.137+0000 7f9a3cec0640 1 --2- 172.21.15.43:0/2690549673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a381021d0 0x7f9a381025b0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.137+0000 7f9a3cec0640 1 -- 172.21.15.43:0/2690549673 >> 172.21.15.43:0/2690549673 conn(0x7f9a380793e0 msgr2=0x7f9a380797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.137+0000 7f9a3cec0640 1 -- 172.21.15.43:0/2690549673 shutdown_connections 2024-04-21T08:45:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.137+0000 7f9a3cec0640 1 -- 172.21.15.43:0/2690549673 wait complete. 2024-04-21T08:45:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.138+0000 7f9a3cec0640 1 Processor -- start 2024-04-21T08:45:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.138+0000 7f9a3cec0640 1 -- start start 2024-04-21T08:45:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.138+0000 7f9a3cec0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a381021d0 0x7f9a38198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.138+0000 7f9a3cec0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a381072a0 0x7f9a38198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.138+0000 7f9a3cec0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a3819cf10 0x7f9a3819d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.138+0000 7f9a3cec0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9a38112230 con 0x7f9a3819cf10 2024-04-21T08:45:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.138+0000 7f9a3cec0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9a381120b0 con 0x7f9a381021d0 2024-04-21T08:45:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.138+0000 7f9a3cec0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a381123b0 con 0x7f9a381072a0 2024-04-21T08:45:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.138+0000 7f9a36d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a3819cf10 0x7f9a3819d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.138+0000 7f9a36d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a3819cf10 0x7f9a3819d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48954/0 (socket says 172.21.15.43:48954) 2024-04-21T08:45:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.138+0000 7f9a36d9d640 1 -- 172.21.15.43:0/2656969311 learned_addr learned my addr 172.21.15.43:0/2656969311 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.138+0000 7f9a3659c640 1 --2- 172.21.15.43:0/2656969311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a381021d0 0x7f9a38198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.138+0000 7f9a35d9b640 1 --2- 172.21.15.43:0/2656969311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a381072a0 0x7f9a38198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.138+0000 7f9a36d9d640 1 -- 172.21.15.43:0/2656969311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a381072a0 msgr2=0x7f9a38198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.138+0000 7f9a36d9d640 1 --2- 172.21.15.43:0/2656969311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a381072a0 0x7f9a38198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.138+0000 7f9a36d9d640 1 -- 172.21.15.43:0/2656969311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a381021d0 msgr2=0x7f9a38198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.138+0000 7f9a36d9d640 1 --2- 172.21.15.43:0/2656969311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a381021d0 0x7f9a38198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.139+0000 7f9a36d9d640 1 -- 172.21.15.43:0/2656969311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a3819da00 con 0x7f9a3819cf10 2024-04-21T08:45:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.139+0000 7f9a36d9d640 1 --2- 172.21.15.43:0/2656969311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a3819cf10 0x7f9a3819d3c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000e960 tx=0x7f9a3000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.139+0000 7f9a277fe640 1 -- 172.21.15.43:0/2656969311 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a30004d20 con 0x7f9a3819cf10 2024-04-21T08:45:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.139+0000 7f9a277fe640 1 -- 172.21.15.43:0/2656969311 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a30005020 con 0x7f9a3819cf10 2024-04-21T08:45:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.139+0000 7f9a3cec0640 1 -- 172.21.15.43:0/2656969311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a3819dcf0 con 0x7f9a3819cf10 2024-04-21T08:45:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.139+0000 7f9a277fe640 1 -- 172.21.15.43:0/2656969311 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a300116d0 con 0x7f9a3819cf10 2024-04-21T08:45:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.139+0000 7f9a3cec0640 1 -- 172.21.15.43:0/2656969311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a381a54e0 con 0x7f9a3819cf10 2024-04-21T08:45:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.141+0000 7f9a277fe640 1 -- 172.21.15.43:0/2656969311 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9a300051d0 con 0x7f9a3819cf10 2024-04-21T08:45:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.142+0000 7f9a3fb81740 1 Processor -- start 2024-04-21T08:45:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.142+0000 7f9a3fb81740 1 -- start start 2024-04-21T08:45:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.143+0000 7f9a3fb81740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc918d0 0x55ddadc90620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.143+0000 7f9a3fb81740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddadc90b60 0x55ddadc90fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.143+0000 7f9a3fb81740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddadc9a0e0 0x55ddadc9c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.143+0000 7f9a3fb81740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ddadbf58d0 con 0x55ddadc918d0 2024-04-21T08:45:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.143+0000 7f9a3fb81740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ddadb92e80 con 0x55ddadc9a0e0 2024-04-21T08:45:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.143+0000 7f9a3fb81740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ddadbf1890 con 0x55ddadc90b60 2024-04-21T08:45:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.143+0000 7f9a277fe640 1 --2- 172.21.15.43:0/2656969311 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9a200a3e40 0x7f9a200a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.143+0000 7f9a35d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc918d0 0x55ddadc90620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.143+0000 7f9a35d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc918d0 0x55ddadc90620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48966/0 (socket says 172.21.15.43:48966) 2024-04-21T08:45:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.143+0000 7f9a35d9b640 1 -- 172.21.15.43:0/260178148 learned_addr learned my addr 172.21.15.43:0/260178148 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.143+0000 7f9a3659c640 1 --2- 172.21.15.43:0/260178148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddadc90b60 0x55ddadc90fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.143+0000 7f9a36d9d640 1 --2- 172.21.15.43:0/2656969311 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9a200a3e40 0x7f9a200a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.143+0000 7f9a35d9b640 1 --2- 172.21.15.43:0/260178148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddadc9a0e0 0x55ddadc9c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.143+0000 7f9a277fe640 1 -- 172.21.15.43:0/2656969311 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f9a300d0be0 con 0x7f9a3819cf10 2024-04-21T08:45:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.143+0000 7f9a35d9b640 1 -- 172.21.15.43:0/260178148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddadc90b60 msgr2=0x55ddadc90fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.143+0000 7f9a35d9b640 1 --2- 172.21.15.43:0/260178148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddadc90b60 0x55ddadc90fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.143+0000 7f9a35d9b640 1 -- 172.21.15.43:0/260178148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddadc9a0e0 msgr2=0x55ddadc9c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.143+0000 7f9a35d9b640 1 --2- 172.21.15.43:0/260178148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddadc9a0e0 0x55ddadc9c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.143+0000 7f9a35d9b640 1 -- 172.21.15.43:0/260178148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddadbb4320 con 0x55ddadc918d0 2024-04-21T08:45:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.143+0000 7f9a3659c640 1 --2- 172.21.15.43:0/260178148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddadc90b60 0x55ddadc90fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:45:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.143+0000 7f9a35d9b640 1 --2- 172.21.15.43:0/260178148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc918d0 0x55ddadc90620 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c0058a0 tx=0x7f9a2c00b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5327151309162c62 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.144+0000 7f9a36d9d640 1 --2- 172.21.15.43:0/2656969311 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9a200a3e40 0x7f9a200a6300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9a300991d0 tx=0x7f9a30087040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.144+0000 7f9a24ff9640 1 -- 172.21.15.43:0/260178148 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c004030 con 0x55ddadc918d0 2024-04-21T08:45:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.144+0000 7f9a24ff9640 1 -- 172.21.15.43:0/260178148 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a2c0054d0 con 0x55ddadc918d0 2024-04-21T08:45:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.144+0000 7f9a24ff9640 1 -- 172.21.15.43:0/260178148 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c0049e0 con 0x55ddadc918d0 2024-04-21T08:45:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.144+0000 7f9a3fb81740 1 -- 172.21.15.43:0/260178148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc918d0 msgr2=0x55ddadc90620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.144+0000 7f9a3fb81740 1 --2- 172.21.15.43:0/260178148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc918d0 0x55ddadc90620 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c0058a0 tx=0x7f9a2c00b820 comp rx=0 tx=0).stop 2024-04-21T08:45:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.144+0000 7f9a3fb81740 1 -- 172.21.15.43:0/260178148 shutdown_connections 2024-04-21T08:45:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.144+0000 7f9a3fb81740 1 --2- 172.21.15.43:0/260178148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddadc9a0e0 0x55ddadc9c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.144+0000 7f9a3fb81740 1 --2- 172.21.15.43:0/260178148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddadc90b60 0x55ddadc90fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.144+0000 7f9a3fb81740 1 --2- 172.21.15.43:0/260178148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc918d0 0x55ddadc90620 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.144+0000 7f9a3fb81740 1 -- 172.21.15.43:0/260178148 >> 172.21.15.43:0/260178148 conn(0x55ddadc57c90 msgr2=0x55ddadc8b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.145+0000 7f9a3fb81740 1 -- 172.21.15.43:0/260178148 shutdown_connections 2024-04-21T08:45:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.145+0000 7f9a3fb81740 1 -- 172.21.15.43:0/260178148 wait complete. 2024-04-21T08:45:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.145+0000 7f9a3fb81740 1 -- start start 2024-04-21T08:45:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.145+0000 7f9a3fb81740 1 Processor -- start 2024-04-21T08:45:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.145+0000 7f9a3fb81740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddadc90b60 0x55ddadc999e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.145+0000 7f9a3fb81740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc918d0 0x55ddadc96de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.145+0000 7f9a3fb81740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddadc9a0e0 0x55ddadc97340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.145+0000 7f9a3fb81740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ddadc9ce10 con 0x55ddadc918d0 2024-04-21T08:45:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.145+0000 7f9a3fb81740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ddadb7deb0 con 0x55ddadc9a0e0 2024-04-21T08:45:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.145+0000 7f9a3fb81740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ddadc9cf90 con 0x55ddadc90b60 2024-04-21T08:45:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.145+0000 7f9a3fb81740 10 client.0 _ll_register_callbacks cb 0x55ddadbaab40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.145+0000 7f9a3659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc918d0 0x55ddadc96de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.145+0000 7f9a3659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc918d0 0x55ddadc96de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48978/0 (socket says 172.21.15.43:48978) 2024-04-21T08:45:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.145+0000 7f9a3659c640 1 -- 172.21.15.43:0/1126951864 learned_addr learned my addr 172.21.15.43:0/1126951864 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.146+0000 7f9a35d9b640 1 --2- 172.21.15.43:0/1126951864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddadc90b60 0x55ddadc999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.146+0000 7f9a35d9b640 1 --2- 172.21.15.43:0/1126951864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddadc9a0e0 0x55ddadc97340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.146+0000 7f9a3659c640 1 -- 172.21.15.43:0/1126951864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddadc90b60 msgr2=0x55ddadc999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.146+0000 7f9a3659c640 1 --2- 172.21.15.43:0/1126951864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddadc90b60 0x55ddadc999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.146+0000 7f9a3659c640 1 -- 172.21.15.43:0/1126951864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddadc9a0e0 msgr2=0x55ddadc97340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.146+0000 7f9a3659c640 1 --2- 172.21.15.43:0/1126951864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddadc9a0e0 0x55ddadc97340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.146+0000 7f9a3659c640 1 -- 172.21.15.43:0/1126951864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddad908210 con 0x55ddadc918d0 2024-04-21T08:45:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.146+0000 7f9a35d9b640 1 --2- 172.21.15.43:0/1126951864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddadc9a0e0 0x55ddadc97340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.146+0000 7f9a3659c640 1 --2- 172.21.15.43:0/1126951864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc918d0 0x55ddadc96de0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f9a28006010 tx=0x7f9a2802ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed82ddf69f6f1fc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.146+0000 7f9a1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.146+0000 7f9a1dffb640 1 -- 172.21.15.43:0/1126951864 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a28005030 con 0x55ddadc918d0 2024-04-21T08:45:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.146+0000 7f9a1dffb640 1 -- 172.21.15.43:0/1126951864 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f9a280051d0 con 0x55ddadc918d0 2024-04-21T08:45:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.146+0000 7f9a3fb81740 1 -- 172.21.15.43:0/1126951864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a2c0054d0 con 0x55ddadc918d0 2024-04-21T08:45:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.146+0000 7f9a1dffb640 1 -- 172.21.15.43:0/1126951864 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9a28051050 con 0x55ddadc918d0 2024-04-21T08:45:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.147+0000 7f9a3fb81740 10 client.18351 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.147+0000 7f9a3fb81740 1 -- 172.21.15.43:0/1126951864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ddad9d4620 con 0x55ddadc918d0 2024-04-21T08:45:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.147+0000 7f9a1dffb640 1 -- 172.21.15.43:0/1126951864 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a2c0054d0 con 0x55ddadc918d0 2024-04-21T08:45:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.147+0000 7f9a1dffb640 1 -- 172.21.15.43:0/1126951864 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9a280035f0 con 0x55ddadc918d0 2024-04-21T08:45:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.147+0000 7f9a3fb81740 10 client.18351 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.147+0000 7f9a3fb81740 10 client.18351 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.147+0000 7f9a3fb81740 20 client.18351 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.147+0000 7f9a3fb81740 1 --2- 172.21.15.43:0/1126951864 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ddadcaa100 0x55ddadcac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.147+0000 7f9a3fb81740 4 client.18351 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.147+0000 7f9a3fb81740 1 -- 172.21.15.43:0/1126951864 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddadc09880 con 0x55ddadcaa100 2024-04-21T08:45:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.147+0000 7f9a35d9b640 1 --2- 172.21.15.43:0/1126951864 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ddadcaa100 0x55ddadcac4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.148+0000 7f9a35d9b640 1 --2- 172.21.15.43:0/1126951864 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ddadcaa100 0x55ddadcac4f0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c0b469e8066778 server_cookie=111458cc5952a3c9 in_seq=0 out_seq=0 2024-04-21T08:45:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.148+0000 7f9a1dffb640 10 client.18351 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.148+0000 7f9a1dffb640 1 -- 172.21.15.43:0/1126951864 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ddadc09880 con 0x55ddadcaa100 2024-04-21T08:45:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.148+0000 7f9a1dffb640 10 client.18351 handle_command_reply: tid=0 2024-04-21T08:45:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.149+0000 7f9a3fb81740 1 client.18351 shutdown 2024-04-21T08:45:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.149+0000 7f9a3fb81740 1 -- 172.21.15.43:0/1126951864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc918d0 msgr2=0x55ddadc96de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.149+0000 7f9a3fb81740 1 --2- 172.21.15.43:0/1126951864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc918d0 0x55ddadc96de0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f9a28006010 tx=0x7f9a2802ca10 comp rx=0 tx=0).stop 2024-04-21T08:45:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.149+0000 7f9a3fb81740 1 -- 172.21.15.43:0/1126951864 shutdown_connections 2024-04-21T08:45:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.149+0000 7f9a3fb81740 1 --2- 172.21.15.43:0/1126951864 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ddadcaa100 0x55ddadcac4f0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.149+0000 7f9a3fb81740 1 --2- 172.21.15.43:0/1126951864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddadc9a0e0 0x55ddadc97340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.149+0000 7f9a3fb81740 1 --2- 172.21.15.43:0/1126951864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc918d0 0x55ddadc96de0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.149+0000 7f9a1dffb640 0 client.18351 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.149+0000 7f9a3fb81740 1 --2- 172.21.15.43:0/1126951864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddadc90b60 0x55ddadc999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.149+0000 7f9a3fb81740 1 -- 172.21.15.43:0/1126951864 >> 172.21.15.43:0/1126951864 conn(0x55ddadc57c90 msgr2=0x55ddadc8d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.149+0000 7f9a3fb81740 1 -- 172.21.15.43:0/1126951864 shutdown_connections 2024-04-21T08:45:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.149+0000 7f9a3fb81740 1 -- 172.21.15.43:0/1126951864 wait complete. 2024-04-21T08:45:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.149+0000 7f9a3fb81740 20 client.18351 trim_cache size 0 max 16384 2024-04-21T08:45:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.164+0000 7f9a3fb81740 1 Processor -- start 2024-04-21T08:45:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.164+0000 7f9a3fb81740 1 -- start start 2024-04-21T08:45:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.164+0000 7f9a3fb81740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddadc90b60 0x55ddadca5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.164+0000 7f9a3fb81740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc918d0 0x55ddadc99680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.164+0000 7f9a3fb81740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddadc9a0e0 0x55ddadcb1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.164+0000 7f9a3fb81740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9a381af2f0 con 0x55ddadc918d0 2024-04-21T08:45:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.164+0000 7f9a3fb81740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9a381b0e30 con 0x55ddadc9a0e0 2024-04-21T08:45:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.164+0000 7f9a3fb81740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a381af9c0 con 0x55ddadc90b60 2024-04-21T08:45:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.164+0000 7f9a3659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc918d0 0x55ddadc99680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.164+0000 7f9a3659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc918d0 0x55ddadc99680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48984/0 (socket says 172.21.15.43:48984) 2024-04-21T08:45:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.164+0000 7f9a3659c640 1 -- 172.21.15.43:0/995645591 learned_addr learned my addr 172.21.15.43:0/995645591 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.165+0000 7f9a35d9b640 1 --2- 172.21.15.43:0/995645591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddadc90b60 0x55ddadca5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.165+0000 7f9a35d9b640 1 --2- 172.21.15.43:0/995645591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddadc9a0e0 0x55ddadcb1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.165+0000 7f9a3659c640 1 -- 172.21.15.43:0/995645591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddadc90b60 msgr2=0x55ddadca5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.165+0000 7f9a3659c640 1 --2- 172.21.15.43:0/995645591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddadc90b60 0x55ddadca5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.165+0000 7f9a3659c640 1 -- 172.21.15.43:0/995645591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddadc9a0e0 msgr2=0x55ddadcb1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.165+0000 7f9a3659c640 1 --2- 172.21.15.43:0/995645591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddadc9a0e0 0x55ddadcb1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.165+0000 7f9a3659c640 1 -- 172.21.15.43:0/995645591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddadc8fe30 con 0x55ddadc918d0 2024-04-21T08:45:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.165+0000 7f9a35d9b640 1 --2- 172.21.15.43:0/995645591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddadc9a0e0 0x55ddadcb1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.165+0000 7f9a3659c640 1 --2- 172.21.15.43:0/995645591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc918d0 0x55ddadc99680 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f9a28003a40 tx=0x7f9a2802cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8674ac861204c632 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.165+0000 7f9a1ffff640 1 -- 172.21.15.43:0/995645591 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a280035f0 con 0x55ddadc918d0 2024-04-21T08:45:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.165+0000 7f9a1ffff640 1 -- 172.21.15.43:0/995645591 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a28003790 con 0x55ddadc918d0 2024-04-21T08:45:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.165+0000 7f9a1ffff640 1 -- 172.21.15.43:0/995645591 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a28004920 con 0x55ddadc918d0 2024-04-21T08:45:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.166+0000 7f9a3fb81740 1 -- 172.21.15.43:0/995645591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc918d0 msgr2=0x55ddadc99680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.166+0000 7f9a3fb81740 1 --2- 172.21.15.43:0/995645591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc918d0 0x55ddadc99680 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f9a28003a40 tx=0x7f9a2802cd90 comp rx=0 tx=0).stop 2024-04-21T08:45:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.166+0000 7f9a3fb81740 1 -- 172.21.15.43:0/995645591 shutdown_connections 2024-04-21T08:45:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.166+0000 7f9a3fb81740 1 --2- 172.21.15.43:0/995645591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddadc9a0e0 0x55ddadcb1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.166+0000 7f9a3fb81740 1 --2- 172.21.15.43:0/995645591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc918d0 0x55ddadc99680 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.166+0000 7f9a3fb81740 1 --2- 172.21.15.43:0/995645591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddadc90b60 0x55ddadca5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.166+0000 7f9a3fb81740 1 -- 172.21.15.43:0/995645591 >> 172.21.15.43:0/995645591 conn(0x55ddadc57c90 msgr2=0x55ddadc956b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.166+0000 7f9a3fb81740 1 -- 172.21.15.43:0/995645591 shutdown_connections 2024-04-21T08:45:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.166+0000 7f9a3fb81740 1 -- 172.21.15.43:0/995645591 wait complete. 2024-04-21T08:45:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.166+0000 7f9a3fb81740 1 -- start start 2024-04-21T08:45:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.166+0000 7f9a3fb81740 1 Processor -- start 2024-04-21T08:45:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.166+0000 7f9a3fb81740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddadc90b60 0x55ddadca2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.166+0000 7f9a3fb81740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddadc918d0 0x55ddadc9d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.166+0000 7f9a3fb81740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc9a0e0 0x55ddadc9df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.166+0000 7f9a3fb81740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9a381b0760 con 0x55ddadc9a0e0 2024-04-21T08:45:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.166+0000 7f9a3fb81740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9a381b0090 con 0x55ddadc90b60 2024-04-21T08:45:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.166+0000 7f9a3fb81740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a381ad2d0 con 0x55ddadc918d0 2024-04-21T08:45:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.166+0000 7f9a3fb81740 10 client.0 _ll_register_callbacks cb 0x7f9a28009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.167+0000 7f9a35d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc9a0e0 0x55ddadc9df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.167+0000 7f9a35d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc9a0e0 0x55ddadc9df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48990/0 (socket says 172.21.15.43:48990) 2024-04-21T08:45:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.167+0000 7f9a35d9b640 1 -- 172.21.15.43:0/2458368 learned_addr learned my addr 172.21.15.43:0/2458368 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.167+0000 7f9a35d9b640 1 --2- 172.21.15.43:0/2458368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddadc90b60 0x55ddadca2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.167+0000 7f9a35d9b640 1 -- 172.21.15.43:0/2458368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddadc918d0 msgr2=0x55ddadc9d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.167+0000 7f9a35d9b640 1 --2- 172.21.15.43:0/2458368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddadc918d0 0x55ddadc9d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.167+0000 7f9a35d9b640 1 -- 172.21.15.43:0/2458368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddadc90b60 msgr2=0x55ddadca2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.167+0000 7f9a35d9b640 1 --2- 172.21.15.43:0/2458368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddadc90b60 0x55ddadca2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.167+0000 7f9a35d9b640 1 -- 172.21.15.43:0/2458368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddadc8db50 con 0x55ddadc9a0e0 2024-04-21T08:45:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.167+0000 7f9a35d9b640 1 --2- 172.21.15.43:0/2458368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc9a0e0 0x55ddadc9df30 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c005870 tx=0x7f9a280044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53458401a4a648f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.167+0000 7f9a1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.167+0000 7f9a1dffb640 1 -- 172.21.15.43:0/2458368 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c034060 con 0x55ddadc9a0e0 2024-04-21T08:45:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.167+0000 7f9a1dffb640 1 -- 172.21.15.43:0/2458368 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f9a2c02f5e0 con 0x55ddadc9a0e0 2024-04-21T08:45:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.167+0000 7f9a3fb81740 1 -- 172.21.15.43:0/2458368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a28003790 con 0x55ddadc9a0e0 2024-04-21T08:45:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.168+0000 7f9a1dffb640 1 -- 172.21.15.43:0/2458368 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9a2c027040 con 0x55ddadc9a0e0 2024-04-21T08:45:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.168+0000 7f9a3fb81740 10 client.18357 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.168+0000 7f9a3fb81740 1 -- 172.21.15.43:0/2458368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ddadd104b0 con 0x55ddadc9a0e0 2024-04-21T08:45:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.168+0000 7f9a1dffb640 1 -- 172.21.15.43:0/2458368 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a28003790 con 0x55ddadc9a0e0 2024-04-21T08:45:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.168+0000 7f9a1dffb640 1 -- 172.21.15.43:0/2458368 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9a2c02d040 con 0x55ddadc9a0e0 2024-04-21T08:45:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.168+0000 7f9a3fb81740 10 client.18357 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.168+0000 7f9a3fb81740 10 client.18357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.168+0000 7f9a3fb81740 20 client.18357 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.168+0000 7f9a3fb81740 1 --2- 172.21.15.43:0/2458368 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ddadd10ad0 0x55ddadd12f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.168+0000 7f9a3fb81740 4 client.18357 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:45:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.168+0000 7f9a3fb81740 1 -- 172.21.15.43:0/2458368 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ddadb953d0 con 0x55ddadd10ad0 2024-04-21T08:45:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.168+0000 7f9a3659c640 1 --2- 172.21.15.43:0/2458368 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ddadd10ad0 0x55ddadd12f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.169+0000 7f9a1dffb640 10 client.18357 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.169+0000 7f9a3659c640 1 --2- 172.21.15.43:0/2458368 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ddadd10ad0 0x55ddadd12f10 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8b5944daf6447bd server_cookie=3426148c0f864d42 in_seq=0 out_seq=0 2024-04-21T08:45:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.183+0000 7f9a1dffb640 1 -- 172.21.15.43:0/2458368 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ddadb953d0 con 0x55ddadd10ad0 2024-04-21T08:45:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.183+0000 7f9a1dffb640 10 client.18357 handle_command_reply: tid=0 2024-04-21T08:45:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.183+0000 7f9a3fb81740 1 client.18357 shutdown 2024-04-21T08:45:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.183+0000 7f9a3fb81740 1 -- 172.21.15.43:0/2458368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc9a0e0 msgr2=0x55ddadc9df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.183+0000 7f9a3fb81740 1 --2- 172.21.15.43:0/2458368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc9a0e0 0x55ddadc9df30 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c005870 tx=0x7f9a280044b0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.183+0000 7f9a3fb81740 1 -- 172.21.15.43:0/2458368 shutdown_connections 2024-04-21T08:45:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.183+0000 7f9a3fb81740 1 --2- 172.21.15.43:0/2458368 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ddadd10ad0 0x55ddadd12f10 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.183+0000 7f9a3fb81740 1 --2- 172.21.15.43:0/2458368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddadc9a0e0 0x55ddadc9df30 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.183+0000 7f9a3fb81740 1 --2- 172.21.15.43:0/2458368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddadc918d0 0x55ddadc9d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.183+0000 7f9a1dffb640 0 client.18357 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.183+0000 7f9a3fb81740 1 --2- 172.21.15.43:0/2458368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddadc90b60 0x55ddadca2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.183+0000 7f9a3fb81740 1 -- 172.21.15.43:0/2458368 >> 172.21.15.43:0/2458368 conn(0x55ddadc57c90 msgr2=0x55ddadc8ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.183+0000 7f9a3fb81740 1 -- 172.21.15.43:0/2458368 shutdown_connections 2024-04-21T08:45:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.183+0000 7f9a3fb81740 1 -- 172.21.15.43:0/2458368 wait complete. 2024-04-21T08:45:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.183+0000 7f9a3fb81740 20 client.18357 trim_cache size 0 max 16384 2024-04-21T08:45:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:45:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:45:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:45:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:45:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.184+0000 7f9a3cec0640 1 -- 172.21.15.43:0/2656969311 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9a200a3e40 msgr2=0x7f9a200a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.184+0000 7f9a3cec0640 1 --2- 172.21.15.43:0/2656969311 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9a200a3e40 0x7f9a200a6300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9a300991d0 tx=0x7f9a30087040 comp rx=0 tx=0).stop 2024-04-21T08:45:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.184+0000 7f9a3cec0640 1 -- 172.21.15.43:0/2656969311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a3819cf10 msgr2=0x7f9a3819d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.184+0000 7f9a3cec0640 1 --2- 172.21.15.43:0/2656969311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a3819cf10 0x7f9a3819d3c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000e960 tx=0x7f9a3000ee20 comp rx=0 tx=0).stop 2024-04-21T08:45:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.184+0000 7f9a3cec0640 1 -- 172.21.15.43:0/2656969311 shutdown_connections 2024-04-21T08:45:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.184+0000 7f9a3cec0640 1 --2- 172.21.15.43:0/2656969311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a3819cf10 0x7f9a3819d3c0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.184+0000 7f9a3cec0640 1 --2- 172.21.15.43:0/2656969311 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9a200a3e40 0x7f9a200a6300 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.184+0000 7f9a3cec0640 1 --2- 172.21.15.43:0/2656969311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a381072a0 0x7f9a38198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.184+0000 7f9a3cec0640 1 --2- 172.21.15.43:0/2656969311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a381021d0 0x7f9a38198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.184+0000 7f9a3cec0640 1 -- 172.21.15.43:0/2656969311 >> 172.21.15.43:0/2656969311 conn(0x7f9a380793e0 msgr2=0x7f9a380ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.184+0000 7f9a3cec0640 1 -- 172.21.15.43:0/2656969311 shutdown_connections 2024-04-21T08:45:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:10.184+0000 7f9a3cec0640 1 -- 172.21.15.43:0/2656969311 wait complete. 2024-04-21T08:45:10.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:10 smithi175 ceph-mon[31920]: pgmap v190: 129 pgs: 129 active+clean; 5.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 63 KiB/s wr, 33 op/s 2024-04-21T08:45:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:10 smithi112 ceph-mon[31085]: pgmap v190: 129 pgs: 129 active+clean; 5.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 63 KiB/s wr, 33 op/s 2024-04-21T08:45:10.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:10 smithi043 ceph-mon[27978]: pgmap v190: 129 pgs: 129 active+clean; 5.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 63 KiB/s wr, 33 op/s 2024-04-21T08:45:12.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:12 smithi175 ceph-mon[31920]: pgmap v191: 129 pgs: 129 active+clean; 1.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 87 KiB/s wr, 43 op/s 2024-04-21T08:45:12.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:12 smithi112 ceph-mon[31085]: pgmap v191: 129 pgs: 129 active+clean; 1.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 87 KiB/s wr, 43 op/s 2024-04-21T08:45:12.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:12 smithi043 ceph-mon[27978]: pgmap v191: 129 pgs: 129 active+clean; 1.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 87 KiB/s wr, 43 op/s 2024-04-21T08:45:14.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:14 smithi175 ceph-mon[31920]: pgmap v192: 129 pgs: 129 active+clean; 1.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-04-21T08:45:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:14 smithi112 ceph-mon[31085]: pgmap v192: 129 pgs: 129 active+clean; 1.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-04-21T08:45:14.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:14 smithi043 ceph-mon[27978]: pgmap v192: 129 pgs: 129 active+clean; 1.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-04-21T08:45:16.235 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-21T08:45:16.235 INFO:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-04-21T08:45:16.236 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:45:16.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.351+0000 7f8e72e88640 1 -- 172.21.15.43:0/4216006033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e6c106180 msgr2=0x7f8e6c10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:16.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.351+0000 7f8e72e88640 1 --2- 172.21.15.43:0/4216006033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e6c106180 0x7f8e6c10dd50 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009ca0 tx=0x7f8e6402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.351+0000 7f8e72e88640 1 -- 172.21.15.43:0/4216006033 shutdown_connections 2024-04-21T08:45:16.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.351+0000 7f8e72e88640 1 --2- 172.21.15.43:0/4216006033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e6c106180 0x7f8e6c10dd50 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.351+0000 7f8e72e88640 1 --2- 172.21.15.43:0/4216006033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e6c101a60 0x7f8e6c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.351+0000 7f8e72e88640 1 --2- 172.21.15.43:0/4216006033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e6c1010b0 0x7f8e6c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.351+0000 7f8e72e88640 1 -- 172.21.15.43:0/4216006033 >> 172.21.15.43:0/4216006033 conn(0x7f8e6c0771d0 msgr2=0x7f8e6c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:16.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.351+0000 7f8e72e88640 1 -- 172.21.15.43:0/4216006033 shutdown_connections 2024-04-21T08:45:16.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.351+0000 7f8e72e88640 1 -- 172.21.15.43:0/4216006033 wait complete. 2024-04-21T08:45:16.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.352+0000 7f8e72e88640 1 Processor -- start 2024-04-21T08:45:16.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.352+0000 7f8e72e88640 1 -- start start 2024-04-21T08:45:16.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.352+0000 7f8e72e88640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e6c1010b0 0x7f8e6c19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:16.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.352+0000 7f8e72e88640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e6c106180 0x7f8e6c19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:16.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.352+0000 7f8e72e88640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e6c1985a0 0x7f8e6c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:16.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.352+0000 7f8e72e88640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c111140 con 0x7f8e6c106180 2024-04-21T08:45:16.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.352+0000 7f8e72e88640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c110fc0 con 0x7f8e6c1985a0 2024-04-21T08:45:16.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.352+0000 7f8e72e88640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c1112c0 con 0x7f8e6c1010b0 2024-04-21T08:45:16.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.352+0000 7f8e6bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e6c106180 0x7f8e6c19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:16.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.352+0000 7f8e6bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e6c106180 0x7f8e6c19e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49014/0 (socket says 172.21.15.43:49014) 2024-04-21T08:45:16.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.352+0000 7f8e6bfff640 1 -- 172.21.15.43:0/1924846097 learned_addr learned my addr 172.21.15.43:0/1924846097 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:16.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.352+0000 7f8e71425640 1 --2- 172.21.15.43:0/1924846097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e6c1985a0 0x7f8e6c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:16.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.352+0000 7f8e70c24640 1 --2- 172.21.15.43:0/1924846097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e6c1010b0 0x7f8e6c19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:16.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.352+0000 7f8e6bfff640 1 -- 172.21.15.43:0/1924846097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e6c1010b0 msgr2=0x7f8e6c19e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:16.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.352+0000 7f8e6bfff640 1 --2- 172.21.15.43:0/1924846097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e6c1010b0 0x7f8e6c19e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.352+0000 7f8e6bfff640 1 -- 172.21.15.43:0/1924846097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e6c1985a0 msgr2=0x7f8e6c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:16.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.352+0000 7f8e6bfff640 1 --2- 172.21.15.43:0/1924846097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e6c1985a0 0x7f8e6c198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.352+0000 7f8e6bfff640 1 -- 172.21.15.43:0/1924846097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e6c1990c0 con 0x7f8e6c106180 2024-04-21T08:45:16.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.352+0000 7f8e70c24640 1 --2- 172.21.15.43:0/1924846097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e6c1010b0 0x7f8e6c19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:16.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.353+0000 7f8e6bfff640 1 --2- 172.21.15.43:0/1924846097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e6c106180 0x7f8e6c19e930 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400d770 tx=0x7f8e5400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:16.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.353+0000 7f8e69ffb640 1 -- 172.21.15.43:0/1924846097 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e54017070 con 0x7f8e6c106180 2024-04-21T08:45:16.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.353+0000 7f8e69ffb640 1 -- 172.21.15.43:0/1924846097 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e54009c30 con 0x7f8e6c106180 2024-04-21T08:45:16.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.353+0000 7f8e69ffb640 1 -- 172.21.15.43:0/1924846097 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5400f620 con 0x7f8e6c106180 2024-04-21T08:45:16.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.353+0000 7f8e72e88640 1 -- 172.21.15.43:0/1924846097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e6c1993b0 con 0x7f8e6c106180 2024-04-21T08:45:16.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.353+0000 7f8e72e88640 1 -- 172.21.15.43:0/1924846097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e6c1a5600 con 0x7f8e6c106180 2024-04-21T08:45:16.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.354+0000 7f8e72e88640 1 -- 172.21.15.43:0/1924846097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e6c069230 con 0x7f8e6c106180 2024-04-21T08:45:16.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.355+0000 7f8e69ffb640 1 -- 172.21.15.43:0/1924846097 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8e5401a070 con 0x7f8e6c106180 2024-04-21T08:45:16.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.357+0000 7f8e69ffb640 1 --2- 172.21.15.43:0/1924846097 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8e580a3d20 0x7f8e580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:16.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.357+0000 7f8e69ffb640 1 -- 172.21.15.43:0/1924846097 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f8e540d0850 con 0x7f8e6c106180 2024-04-21T08:45:16.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.357+0000 7f8e70c24640 1 --2- 172.21.15.43:0/1924846097 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8e580a3d20 0x7f8e580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:16.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.357+0000 7f8e70c24640 1 --2- 172.21.15.43:0/1924846097 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8e580a3d20 0x7f8e580a61e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c006fd0 tx=0x7f8e5c008040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:16.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.361+0000 7f8e69ffb640 1 -- 172.21.15.43:0/1924846097 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e54009de0 con 0x7f8e6c106180 2024-04-21T08:45:16.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:16 smithi175 ceph-mon[31920]: pgmap v193: 129 pgs: 129 active+clean; 1.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-04-21T08:45:16.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:16 smithi112 ceph-mon[31085]: pgmap v193: 129 pgs: 129 active+clean; 1.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-04-21T08:45:16.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:16 smithi043 ceph-mon[27978]: pgmap v193: 129 pgs: 129 active+clean; 1.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-04-21T08:45:16.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.593+0000 7f8e72e88640 1 -- 172.21.15.43:0/1924846097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e6c101af0 con 0x7f8e6c106180 2024-04-21T08:45:16.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.594+0000 7f8e69ffb640 1 -- 172.21.15.43:0/1924846097 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7f8e5409cac0 con 0x7f8e6c106180 2024-04-21T08:45:16.593 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:45:16.593 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:45:16.594 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:45:16.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.596+0000 7f8e72e88640 1 -- 172.21.15.43:0/1924846097 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8e580a3d20 msgr2=0x7f8e580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:16.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.596+0000 7f8e72e88640 1 --2- 172.21.15.43:0/1924846097 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8e580a3d20 0x7f8e580a61e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c006fd0 tx=0x7f8e5c008040 comp rx=0 tx=0).stop 2024-04-21T08:45:16.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.596+0000 7f8e72e88640 1 -- 172.21.15.43:0/1924846097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e6c106180 msgr2=0x7f8e6c19e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:16.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.596+0000 7f8e72e88640 1 --2- 172.21.15.43:0/1924846097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e6c106180 0x7f8e6c19e930 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400d770 tx=0x7f8e5400dc30 comp rx=0 tx=0).stop 2024-04-21T08:45:16.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.596+0000 7f8e72e88640 1 -- 172.21.15.43:0/1924846097 shutdown_connections 2024-04-21T08:45:16.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.596+0000 7f8e72e88640 1 --2- 172.21.15.43:0/1924846097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e6c1985a0 0x7f8e6c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.596+0000 7f8e72e88640 1 --2- 172.21.15.43:0/1924846097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e6c106180 0x7f8e6c19e930 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.596+0000 7f8e72e88640 1 --2- 172.21.15.43:0/1924846097 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8e580a3d20 0x7f8e580a61e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.596+0000 7f8e72e88640 1 --2- 172.21.15.43:0/1924846097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e6c1010b0 0x7f8e6c19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.596+0000 7f8e72e88640 1 -- 172.21.15.43:0/1924846097 >> 172.21.15.43:0/1924846097 conn(0x7f8e6c0771d0 msgr2=0x7f8e6c104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:16.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.596+0000 7f8e72e88640 1 -- 172.21.15.43:0/1924846097 shutdown_connections 2024-04-21T08:45:16.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.596+0000 7f8e72e88640 1 -- 172.21.15.43:0/1924846097 wait complete. 2024-04-21T08:45:16.606 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db / --timeout 30 --expiration 90.0 --await' 2024-04-21T08:45:16.607 DEBUG:teuthology.orchestra.run.smithi043:> timeout 45 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24527 quiesce db / --timeout 30 --expiration 90.0 --await 2024-04-21T08:45:16.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.718+0000 7fe90a4c8640 1 -- 172.21.15.43:0/2347868673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe904101a50 msgr2=0x7fe904105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:16.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.718+0000 7fe90a4c8640 1 --2- 172.21.15.43:0/2347868673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe904101a50 0x7fe904105a90 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009900 tx=0x7fe8ec02bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:16.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.718+0000 7fe90a4c8640 1 -- 172.21.15.43:0/2347868673 shutdown_connections 2024-04-21T08:45:16.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.718+0000 7fe90a4c8640 1 --2- 172.21.15.43:0/2347868673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe904106170 0x7fe90410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.718+0000 7fe90a4c8640 1 --2- 172.21.15.43:0/2347868673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe904101a50 0x7fe904105a90 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.718+0000 7fe90a4c8640 1 --2- 172.21.15.43:0/2347868673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9041010a0 0x7fe904101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.718+0000 7fe90a4c8640 1 -- 172.21.15.43:0/2347868673 >> 172.21.15.43:0/2347868673 conn(0x7fe904077f50 msgr2=0x7fe904078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:16.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.718+0000 7fe90a4c8640 1 -- 172.21.15.43:0/2347868673 shutdown_connections 2024-04-21T08:45:16.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.719+0000 7fe90a4c8640 1 -- 172.21.15.43:0/2347868673 wait complete. 2024-04-21T08:45:16.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.719+0000 7fe90a4c8640 1 Processor -- start 2024-04-21T08:45:16.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.719+0000 7fe90a4c8640 1 -- start start 2024-04-21T08:45:16.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.719+0000 7fe90a4c8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9041010a0 0x7fe90410e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:16.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.719+0000 7fe90a4c8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe904106170 0x7fe90410ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:16.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.719+0000 7fe90a4c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe90410f150 0x7fe9041a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:16.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.719+0000 7fe90a4c8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe904112210 con 0x7fe904106170 2024-04-21T08:45:16.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.719+0000 7fe90a4c8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe904112090 con 0x7fe90410f150 2024-04-21T08:45:16.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.719+0000 7fe90a4c8640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe904112390 con 0x7fe9041010a0 2024-04-21T08:45:16.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.720+0000 7fe9037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe904106170 0x7fe90410ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:16.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.720+0000 7fe9037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe904106170 0x7fe90410ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57590/0 (socket says 172.21.15.43:57590) 2024-04-21T08:45:16.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.720+0000 7fe9037fe640 1 -- 172.21.15.43:0/1407141298 learned_addr learned my addr 172.21.15.43:0/1407141298 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:16.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.720+0000 7fe903fff640 1 --2- 172.21.15.43:0/1407141298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9041010a0 0x7fe90410e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:16.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.720+0000 7fe908a65640 1 --2- 172.21.15.43:0/1407141298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe90410f150 0x7fe9041a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:16.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.720+0000 7fe9037fe640 1 -- 172.21.15.43:0/1407141298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9041010a0 msgr2=0x7fe90410e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:16.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.720+0000 7fe9037fe640 1 --2- 172.21.15.43:0/1407141298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9041010a0 0x7fe90410e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.720+0000 7fe9037fe640 1 -- 172.21.15.43:0/1407141298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe90410f150 msgr2=0x7fe9041a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:16.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.720+0000 7fe9037fe640 1 --2- 172.21.15.43:0/1407141298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe90410f150 0x7fe9041a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.720+0000 7fe9037fe640 1 -- 172.21.15.43:0/1407141298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9041a7040 con 0x7fe904106170 2024-04-21T08:45:16.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.720+0000 7fe9037fe640 1 --2- 172.21.15.43:0/1407141298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe904106170 0x7fe90410ec10 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec005ec0 tx=0x7fe8ec02ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:16.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.720+0000 7fe9017fa640 1 -- 172.21.15.43:0/1407141298 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec043020 con 0x7fe904106170 2024-04-21T08:45:16.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.720+0000 7fe9017fa640 1 -- 172.21.15.43:0/1407141298 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8ec0052b0 con 0x7fe904106170 2024-04-21T08:45:16.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.720+0000 7fe90a4c8640 1 -- 172.21.15.43:0/1407141298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9041a72d0 con 0x7fe904106170 2024-04-21T08:45:16.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.720+0000 7fe9017fa640 1 -- 172.21.15.43:0/1407141298 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec003d40 con 0x7fe904106170 2024-04-21T08:45:16.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.720+0000 7fe90a4c8640 1 -- 172.21.15.43:0/1407141298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9041a7820 con 0x7fe904106170 2024-04-21T08:45:16.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.722+0000 7fe9017fa640 1 -- 172.21.15.43:0/1407141298 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe8ec046020 con 0x7fe904106170 2024-04-21T08:45:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.723+0000 7fe90d198740 1 Processor -- start 2024-04-21T08:45:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.723+0000 7fe90d198740 1 -- start start 2024-04-21T08:45:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.723+0000 7fe9017fa640 1 --2- 172.21.15.43:0/1407141298 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe8f00a3d50 0x7fe8f00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.723+0000 7fe90d198740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694c19c0 0x55f5694c1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.723+0000 7fe90d198740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5694c22e0 0x55f5694ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.723+0000 7fe90d198740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5694caec0 0x55f5694cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.723+0000 7fe90d198740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f5694220b0 con 0x55f5694c19c0 2024-04-21T08:45:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.723+0000 7fe90d198740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f5693d3680 con 0x55f5694c22e0 2024-04-21T08:45:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.723+0000 7fe90d198740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f5693dfc60 con 0x55f5694caec0 2024-04-21T08:45:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.723+0000 7fe9017fa640 1 -- 172.21.15.43:0/1407141298 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fe8ec0f2f30 con 0x7fe904106170 2024-04-21T08:45:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.723+0000 7fe908a65640 1 --2- 172.21.15.43:0/1407141298 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe8f00a3d50 0x7fe8f00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.723+0000 7fe903fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694c19c0 0x55f5694c1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.723+0000 7fe903fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694c19c0 0x55f5694c1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57598/0 (socket says 172.21.15.43:57598) 2024-04-21T08:45:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.723+0000 7fe903fff640 1 -- 172.21.15.43:0/1902770816 learned_addr learned my addr 172.21.15.43:0/1902770816 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.724+0000 7fe903fff640 1 -- 172.21.15.43:0/1902770816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5694caec0 msgr2=0x55f5694cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.724+0000 7fe903fff640 1 --2- 172.21.15.43:0/1902770816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5694caec0 0x55f5694cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.724+0000 7fe9037fe640 1 --2- 172.21.15.43:0/1902770816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5694c22e0 0x55f5694ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.724+0000 7fe903fff640 1 -- 172.21.15.43:0/1902770816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5694c22e0 msgr2=0x55f5694ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.724+0000 7fe903fff640 1 --2- 172.21.15.43:0/1902770816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5694c22e0 0x55f5694ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.724+0000 7fe903fff640 1 -- 172.21.15.43:0/1902770816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f569204c90 con 0x55f5694c19c0 2024-04-21T08:45:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.724+0000 7fe9037fe640 1 --2- 172.21.15.43:0/1902770816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5694c22e0 0x55f5694ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.724+0000 7fe908a65640 1 --2- 172.21.15.43:0/1407141298 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe8f00a3d50 0x7fe8f00a6210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc007e70 tx=0x7fe8fc0077f0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.724+0000 7fe903fff640 1 --2- 172.21.15.43:0/1902770816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694c19c0 0x55f5694c1da0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400b820 tx=0x7fe8f400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9836b54ffe9d0fb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.724+0000 7fe8fa7fc640 1 -- 172.21.15.43:0/1902770816 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f400d950 con 0x55f5694c19c0 2024-04-21T08:45:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.724+0000 7fe8fa7fc640 1 -- 172.21.15.43:0/1902770816 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8f4019070 con 0x55f5694c19c0 2024-04-21T08:45:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.724+0000 7fe8fa7fc640 1 -- 172.21.15.43:0/1902770816 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f40123e0 con 0x55f5694c19c0 2024-04-21T08:45:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.725+0000 7fe90d198740 1 -- 172.21.15.43:0/1902770816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694c19c0 msgr2=0x55f5694c1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:16.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.725+0000 7fe90d198740 1 --2- 172.21.15.43:0/1902770816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694c19c0 0x55f5694c1da0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400b820 tx=0x7fe8f400bce0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.725+0000 7fe90d198740 1 -- 172.21.15.43:0/1902770816 shutdown_connections 2024-04-21T08:45:16.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.725+0000 7fe90d198740 1 --2- 172.21.15.43:0/1902770816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5694caec0 0x55f5694cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.725+0000 7fe90d198740 1 --2- 172.21.15.43:0/1902770816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5694c22e0 0x55f5694ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.725+0000 7fe90d198740 1 --2- 172.21.15.43:0/1902770816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694c19c0 0x55f5694c1da0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.725+0000 7fe90d198740 1 -- 172.21.15.43:0/1902770816 >> 172.21.15.43:0/1902770816 conn(0x55f5694bc0f0 msgr2=0x55f5694bc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:16.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.725+0000 7fe90d198740 1 -- 172.21.15.43:0/1902770816 shutdown_connections 2024-04-21T08:45:16.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.725+0000 7fe90d198740 1 -- 172.21.15.43:0/1902770816 wait complete. 2024-04-21T08:45:16.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.725+0000 7fe90d198740 1 -- start start 2024-04-21T08:45:16.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.725+0000 7fe90d198740 1 Processor -- start 2024-04-21T08:45:16.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.725+0000 7fe90d198740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694c19c0 0x55f5694c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:16.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.726+0000 7fe90d198740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5694c22e0 0x55f5694c8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:16.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.726+0000 7fe90d198740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5694caec0 0x55f5694c8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:16.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.726+0000 7fe90d198740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f569425400 con 0x55f5694c19c0 2024-04-21T08:45:16.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.726+0000 7fe90d198740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f5693e1840 con 0x55f5694c22e0 2024-04-21T08:45:16.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.726+0000 7fe90d198740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f56942a720 con 0x55f5694caec0 2024-04-21T08:45:16.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.726+0000 7fe90d198740 10 client.0 _ll_register_callbacks cb 0x55f569432ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:16.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.726+0000 7fe903fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694c19c0 0x55f5694c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:16.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.726+0000 7fe903fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694c19c0 0x55f5694c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57600/0 (socket says 172.21.15.43:57600) 2024-04-21T08:45:16.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.726+0000 7fe903fff640 1 -- 172.21.15.43:0/2047312883 learned_addr learned my addr 172.21.15.43:0/2047312883 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:16.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.726+0000 7fe9037fe640 1 --2- 172.21.15.43:0/2047312883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5694c22e0 0x55f5694c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:16.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.726+0000 7fe903fff640 1 -- 172.21.15.43:0/2047312883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5694caec0 msgr2=0x55f5694c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:16.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.726+0000 7fe903fff640 1 --2- 172.21.15.43:0/2047312883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5694caec0 0x55f5694c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.726+0000 7fe903fff640 1 -- 172.21.15.43:0/2047312883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5694c22e0 msgr2=0x55f5694c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:16.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.726+0000 7fe903fff640 1 --2- 172.21.15.43:0/2047312883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5694c22e0 0x55f5694c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.726+0000 7fe903fff640 1 -- 172.21.15.43:0/2047312883 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f569265420 con 0x55f5694c19c0 2024-04-21T08:45:16.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.726+0000 7fe9037fe640 1 --2- 172.21.15.43:0/2047312883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5694c22e0 0x55f5694c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:16.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.726+0000 7fe903fff640 1 --2- 172.21.15.43:0/2047312883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694c19c0 0x55f5694c7f10 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4006e10 tx=0x7fe8f4014660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7828ebaeac707116 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:16.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.726+0000 7fe8e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:16.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.726+0000 7fe8e37fe640 1 -- 172.21.15.43:0/2047312883 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4023070 con 0x55f5694c19c0 2024-04-21T08:45:16.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.726+0000 7fe8e37fe640 1 -- 172.21.15.43:0/2047312883 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fe8f400db50 con 0x55f5694c19c0 2024-04-21T08:45:16.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.726+0000 7fe90d198740 1 -- 172.21.15.43:0/2047312883 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8f4019070 con 0x55f5694c19c0 2024-04-21T08:45:16.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.726+0000 7fe8e37fe640 1 -- 172.21.15.43:0/2047312883 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe8f402f050 con 0x55f5694c19c0 2024-04-21T08:45:16.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.726+0000 7fe90d198740 10 client.18375 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:16.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.726+0000 7fe90d198740 1 -- 172.21.15.43:0/2047312883 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f5690f8580 con 0x55f5694c19c0 2024-04-21T08:45:16.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.727+0000 7fe8e37fe640 1 -- 172.21.15.43:0/2047312883 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8f4019070 con 0x55f5694c19c0 2024-04-21T08:45:16.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.727+0000 7fe8e37fe640 1 -- 172.21.15.43:0/2047312883 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe8f4013070 con 0x55f5694c19c0 2024-04-21T08:45:16.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.727+0000 7fe90d198740 10 client.18375 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:16.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.727+0000 7fe90d198740 10 client.18375 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-21T08:45:16.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.727+0000 7fe90d198740 20 client.18375 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:16.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.727+0000 7fe90d198740 1 --2- 172.21.15.43:0/2047312883 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f5694daa60 0x55f5694dce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:16.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.727+0000 7fe90d198740 4 client.18375 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:16.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.727+0000 7fe90d198740 1 -- 172.21.15.43:0/2047312883 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f569433b10 con 0x55f5694daa60 2024-04-21T08:45:16.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.727+0000 7fe9037fe640 1 --2- 172.21.15.43:0/2047312883 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f5694daa60 0x55f5694dce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:16.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.728+0000 7fe9037fe640 1 --2- 172.21.15.43:0/2047312883 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f5694daa60 0x55f5694dce50 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7482777cf0dd40e3 server_cookie=f2ccb6d4f0bfa3c2 in_seq=0 out_seq=0 2024-04-21T08:45:16.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.728+0000 7fe8e37fe640 10 client.18375 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:16.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.728+0000 7fe8e37fe640 1 -- 172.21.15.43:0/2047312883 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f569433b10 con 0x55f5694daa60 2024-04-21T08:45:16.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.728+0000 7fe8e37fe640 10 client.18375 handle_command_reply: tid=0 2024-04-21T08:45:16.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.728+0000 7fe90d198740 1 client.18375 shutdown 2024-04-21T08:45:16.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.728+0000 7fe90d198740 1 -- 172.21.15.43:0/2047312883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694c19c0 msgr2=0x55f5694c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:16.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.728+0000 7fe90d198740 1 --2- 172.21.15.43:0/2047312883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694c19c0 0x55f5694c7f10 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4006e10 tx=0x7fe8f4014660 comp rx=0 tx=0).stop 2024-04-21T08:45:16.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.729+0000 7fe90d198740 1 -- 172.21.15.43:0/2047312883 shutdown_connections 2024-04-21T08:45:16.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.729+0000 7fe90d198740 1 --2- 172.21.15.43:0/2047312883 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f5694daa60 0x55f5694dce50 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.729+0000 7fe90d198740 1 --2- 172.21.15.43:0/2047312883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5694caec0 0x55f5694c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.729+0000 7fe90d198740 1 --2- 172.21.15.43:0/2047312883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5694c22e0 0x55f5694c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.729+0000 7fe90d198740 1 --2- 172.21.15.43:0/2047312883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694c19c0 0x55f5694c7f10 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.729+0000 7fe8e37fe640 0 client.18375 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:16.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.729+0000 7fe90d198740 1 -- 172.21.15.43:0/2047312883 >> 172.21.15.43:0/2047312883 conn(0x55f5694bc0f0 msgr2=0x55f5694bde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:16.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.729+0000 7fe90d198740 1 -- 172.21.15.43:0/2047312883 shutdown_connections 2024-04-21T08:45:16.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.729+0000 7fe90d198740 1 -- 172.21.15.43:0/2047312883 wait complete. 2024-04-21T08:45:16.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.729+0000 7fe90d198740 20 client.18375 trim_cache size 0 max 16384 2024-04-21T08:45:16.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.746+0000 7fe90d198740 1 Processor -- start 2024-04-21T08:45:16.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.746+0000 7fe90d198740 1 -- start start 2024-04-21T08:45:16.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.746+0000 7fe90d198740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5694c19c0 0x55f5694c3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:16.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.746+0000 7fe90d198740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5694c22e0 0x55f5694e2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:16.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.746+0000 7fe90d198740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694caec0 0x55f5694e2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:16.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.746+0000 7fe90d198740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe9041b30c0 con 0x55f5694caec0 2024-04-21T08:45:16.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.746+0000 7fe90d198740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe9041b1c50 con 0x55f5694c22e0 2024-04-21T08:45:16.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.746+0000 7fe90d198740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9041b2320 con 0x55f5694c19c0 2024-04-21T08:45:16.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.746+0000 7fe908a65640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694caec0 0x55f5694e2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:16.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.746+0000 7fe908a65640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694caec0 0x55f5694e2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57616/0 (socket says 172.21.15.43:57616) 2024-04-21T08:45:16.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.746+0000 7fe908a65640 1 -- 172.21.15.43:0/2568287189 learned_addr learned my addr 172.21.15.43:0/2568287189 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:16.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.746+0000 7fe9037fe640 1 --2- 172.21.15.43:0/2568287189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5694c22e0 0x55f5694e2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:16.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.746+0000 7fe903fff640 1 --2- 172.21.15.43:0/2568287189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5694c19c0 0x55f5694c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:16.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.746+0000 7fe908a65640 1 -- 172.21.15.43:0/2568287189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5694c19c0 msgr2=0x55f5694c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:16.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.746+0000 7fe908a65640 1 --2- 172.21.15.43:0/2568287189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5694c19c0 0x55f5694c3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.746+0000 7fe908a65640 1 -- 172.21.15.43:0/2568287189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5694c22e0 msgr2=0x55f5694e2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:16.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.746+0000 7fe908a65640 1 --2- 172.21.15.43:0/2568287189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5694c22e0 0x55f5694e2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.746+0000 7fe908a65640 1 -- 172.21.15.43:0/2568287189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5694c2110 con 0x55f5694caec0 2024-04-21T08:45:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.747+0000 7fe908a65640 1 --2- 172.21.15.43:0/2568287189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694caec0 0x55f5694e2ab0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc007510 tx=0x7fe8fc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9937b569cd5336 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.747+0000 7fe8f9ffb640 1 -- 172.21.15.43:0/2568287189 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc023020 con 0x55f5694caec0 2024-04-21T08:45:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.747+0000 7fe8f9ffb640 1 -- 172.21.15.43:0/2568287189 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8fc01fd10 con 0x55f5694caec0 2024-04-21T08:45:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.747+0000 7fe8f9ffb640 1 -- 172.21.15.43:0/2568287189 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc011e30 con 0x55f5694caec0 2024-04-21T08:45:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.747+0000 7fe90d198740 1 -- 172.21.15.43:0/2568287189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694caec0 msgr2=0x55f5694e2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.747+0000 7fe90d198740 1 --2- 172.21.15.43:0/2568287189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694caec0 0x55f5694e2ab0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc007510 tx=0x7fe8fc004ab0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.747+0000 7fe90d198740 1 -- 172.21.15.43:0/2568287189 shutdown_connections 2024-04-21T08:45:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.747+0000 7fe90d198740 1 --2- 172.21.15.43:0/2568287189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694caec0 0x55f5694e2ab0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.747+0000 7fe90d198740 1 --2- 172.21.15.43:0/2568287189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5694c22e0 0x55f5694e2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.747+0000 7fe90d198740 1 --2- 172.21.15.43:0/2568287189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5694c19c0 0x55f5694c3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.747+0000 7fe90d198740 1 -- 172.21.15.43:0/2568287189 >> 172.21.15.43:0/2568287189 conn(0x55f5694bc0f0 msgr2=0x55f5694bd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:16.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.747+0000 7fe90d198740 1 -- 172.21.15.43:0/2568287189 shutdown_connections 2024-04-21T08:45:16.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.747+0000 7fe90d198740 1 -- 172.21.15.43:0/2568287189 wait complete. 2024-04-21T08:45:16.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.748+0000 7fe90d198740 1 -- start start 2024-04-21T08:45:16.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.748+0000 7fe90d198740 1 Processor -- start 2024-04-21T08:45:16.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.748+0000 7fe90d198740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5694c19c0 0x55f5694d06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:16.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.748+0000 7fe90d198740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694c22e0 0x55f5694d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:16.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.748+0000 7fe90d198740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5694caec0 0x55f5694d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:16.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.748+0000 7fe90d198740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe9041af560 con 0x55f5694c22e0 2024-04-21T08:45:16.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.748+0000 7fe90d198740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe9041b29f0 con 0x55f5694c19c0 2024-04-21T08:45:16.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.748+0000 7fe90d198740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9041afbd0 con 0x55f5694caec0 2024-04-21T08:45:16.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.748+0000 7fe90d198740 10 client.0 _ll_register_callbacks cb 0x7fe8f400b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:16.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.748+0000 7fe9037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694c22e0 0x55f5694d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:16.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.748+0000 7fe9037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694c22e0 0x55f5694d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57620/0 (socket says 172.21.15.43:57620) 2024-04-21T08:45:16.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.748+0000 7fe9037fe640 1 -- 172.21.15.43:0/1335869389 learned_addr learned my addr 172.21.15.43:0/1335869389 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:16.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.748+0000 7fe903fff640 1 --2- 172.21.15.43:0/1335869389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5694c19c0 0x55f5694d06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:16.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.748+0000 7fe908a65640 1 --2- 172.21.15.43:0/1335869389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5694caec0 0x55f5694d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:16.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.749+0000 7fe9037fe640 1 -- 172.21.15.43:0/1335869389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5694caec0 msgr2=0x55f5694d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:16.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.749+0000 7fe9037fe640 1 --2- 172.21.15.43:0/1335869389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5694caec0 0x55f5694d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.749+0000 7fe9037fe640 1 -- 172.21.15.43:0/1335869389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5694c19c0 msgr2=0x55f5694d06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:16.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.749+0000 7fe9037fe640 1 --2- 172.21.15.43:0/1335869389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5694c19c0 0x55f5694d06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.749+0000 7fe9037fe640 1 -- 172.21.15.43:0/1335869389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f56953fdf0 con 0x55f5694c22e0 2024-04-21T08:45:16.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.749+0000 7fe9037fe640 1 --2- 172.21.15.43:0/1335869389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694c22e0 0x55f5694d0c20 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec0f03c0 tx=0x7fe8ec005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82fb5fa27ed9aa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:16.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.749+0000 7fe8e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:16.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.749+0000 7fe8e37fe640 1 -- 172.21.15.43:0/1335869389 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec0d7070 con 0x55f5694c22e0 2024-04-21T08:45:16.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.749+0000 7fe8e37fe640 1 -- 172.21.15.43:0/1335869389 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fe8ec0bceb0 con 0x55f5694c22e0 2024-04-21T08:45:16.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.749+0000 7fe90d198740 1 -- 172.21.15.43:0/1335869389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8fc01fd10 con 0x55f5694c22e0 2024-04-21T08:45:16.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.749+0000 7fe8e37fe640 1 -- 172.21.15.43:0/1335869389 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe8ec0c2f60 con 0x55f5694c22e0 2024-04-21T08:45:16.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.749+0000 7fe90d198740 10 client.18381 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:16.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.749+0000 7fe90d198740 1 -- 172.21.15.43:0/1335869389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f569541010 con 0x55f5694c22e0 2024-04-21T08:45:16.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.749+0000 7fe8e37fe640 1 -- 172.21.15.43:0/1335869389 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8fc01fd10 con 0x55f5694c22e0 2024-04-21T08:45:16.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.749+0000 7fe8e37fe640 1 -- 172.21.15.43:0/1335869389 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe8ec0bb220 con 0x55f5694c22e0 2024-04-21T08:45:16.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.750+0000 7fe90d198740 10 client.18381 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:16.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.750+0000 7fe90d198740 10 client.18381 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-21T08:45:16.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.750+0000 7fe90d198740 20 client.18381 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:16.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.750+0000 7fe90d198740 1 --2- 172.21.15.43:0/1335869389 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f569541880 0x55f569543c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:16.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.750+0000 7fe90d198740 4 client.18381 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 30.0, "expiration": 90.0, "await": true}] 2024-04-21T08:45:16.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.750+0000 7fe90d198740 1 -- 172.21.15.43:0/1335869389 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 30.0, "expiration": 90.0, "await": true}) -- 0x55f56942d980 con 0x55f569541880 2024-04-21T08:45:16.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.750+0000 7fe903fff640 1 --2- 172.21.15.43:0/1335869389 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f569541880 0x55f569543c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:16.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.750+0000 7fe903fff640 1 --2- 172.21.15.43:0/1335869389 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f569541880 0x55f569543c70 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=325ac87302d38e7 server_cookie=78093ee46ddcdeec in_seq=0 out_seq=0 2024-04-21T08:45:16.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.750+0000 7fe8e37fe640 10 client.18381 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:16.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.763+0000 7fe8e37fe640 1 -- 172.21.15.43:0/1335869389 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55f56942d980 con 0x55f569541880 2024-04-21T08:45:16.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.763+0000 7fe8e37fe640 10 client.18381 handle_command_reply: tid=0 2024-04-21T08:45:16.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.763+0000 7fe90d198740 1 client.18381 shutdown 2024-04-21T08:45:16.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.763+0000 7fe90d198740 1 -- 172.21.15.43:0/1335869389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694c22e0 msgr2=0x55f5694d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:16.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.763+0000 7fe90d198740 1 --2- 172.21.15.43:0/1335869389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694c22e0 0x55f5694d0c20 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec0f03c0 tx=0x7fe8ec005fd0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.763+0000 7fe90d198740 1 -- 172.21.15.43:0/1335869389 shutdown_connections 2024-04-21T08:45:16.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.763+0000 7fe90d198740 1 --2- 172.21.15.43:0/1335869389 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f569541880 0x55f569543c70 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.764+0000 7fe90d198740 1 --2- 172.21.15.43:0/1335869389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5694caec0 0x55f5694d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.764+0000 7fe90d198740 1 --2- 172.21.15.43:0/1335869389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5694c22e0 0x55f5694d0c20 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.764+0000 7fe90d198740 1 --2- 172.21.15.43:0/1335869389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5694c19c0 0x55f5694d06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.764+0000 7fe8e37fe640 0 client.18381 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:16.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.764+0000 7fe90d198740 1 -- 172.21.15.43:0/1335869389 >> 172.21.15.43:0/1335869389 conn(0x55f5694bc0f0 msgr2=0x55f5694bd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:16.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.764+0000 7fe90d198740 1 -- 172.21.15.43:0/1335869389 shutdown_connections 2024-04-21T08:45:16.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.764+0000 7fe90d198740 1 -- 172.21.15.43:0/1335869389 wait complete. 2024-04-21T08:45:16.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.764+0000 7fe90d198740 20 client.18381 trim_cache size 0 max 16384 2024-04-21T08:45:16.765 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:45:16.765 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 28, 2024-04-21T08:45:16.765 INFO:teuthology.orchestra.run.smithi043.stdout: "leader": 24527, 2024-04-21T08:45:16.765 INFO:teuthology.orchestra.run.smithi043.stdout: "set_version": 6, 2024-04-21T08:45:16.765 INFO:teuthology.orchestra.run.smithi043.stdout: "sets": { 2024-04-21T08:45:16.765 INFO:teuthology.orchestra.run.smithi043.stdout: "683318c5": { 2024-04-21T08:45:16.765 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 6, 2024-04-21T08:45:16.765 INFO:teuthology.orchestra.run.smithi043.stdout: "age_ref": 0.0, 2024-04-21T08:45:16.765 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-04-21T08:45:16.765 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "QUIESCED", 2024-04-21T08:45:16.765 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-04-21T08:45:16.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:45:16.765 INFO:teuthology.orchestra.run.smithi043.stdout: "timeout": 30.0, 2024-04-21T08:45:16.766 INFO:teuthology.orchestra.run.smithi043.stdout: "expiration": 90.0, 2024-04-21T08:45:16.766 INFO:teuthology.orchestra.run.smithi043.stdout: "members": { 2024-04-21T08:45:16.766 INFO:teuthology.orchestra.run.smithi043.stdout: "file:/": { 2024-04-21T08:45:16.766 INFO:teuthology.orchestra.run.smithi043.stdout: "excluded": false, 2024-04-21T08:45:16.766 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-04-21T08:45:16.766 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "QUIESCED", 2024-04-21T08:45:16.766 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-04-21T08:45:16.766 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:45:16.766 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:45:16.766 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:45:16.766 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:45:16.766 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:45:16.766 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:45:16.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.764+0000 7fe90a4c8640 1 -- 172.21.15.43:0/1407141298 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe8f00a3d50 msgr2=0x7fe8f00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:16.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.764+0000 7fe90a4c8640 1 --2- 172.21.15.43:0/1407141298 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe8f00a3d50 0x7fe8f00a6210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc007e70 tx=0x7fe8fc0077f0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.764+0000 7fe90a4c8640 1 -- 172.21.15.43:0/1407141298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe904106170 msgr2=0x7fe90410ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:16.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.764+0000 7fe90a4c8640 1 --2- 172.21.15.43:0/1407141298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe904106170 0x7fe90410ec10 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec005ec0 tx=0x7fe8ec02ca10 comp rx=0 tx=0).stop 2024-04-21T08:45:16.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.764+0000 7fe90a4c8640 1 -- 172.21.15.43:0/1407141298 shutdown_connections 2024-04-21T08:45:16.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.764+0000 7fe90a4c8640 1 --2- 172.21.15.43:0/1407141298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe90410f150 0x7fe9041a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.764+0000 7fe90a4c8640 1 --2- 172.21.15.43:0/1407141298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe904106170 0x7fe90410ec10 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.764+0000 7fe90a4c8640 1 --2- 172.21.15.43:0/1407141298 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe8f00a3d50 0x7fe8f00a6210 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.764+0000 7fe90a4c8640 1 --2- 172.21.15.43:0/1407141298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9041010a0 0x7fe90410e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:16.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.764+0000 7fe90a4c8640 1 -- 172.21.15.43:0/1407141298 >> 172.21.15.43:0/1407141298 conn(0x7fe904077f50 msgr2=0x7fe9040ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:16.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.764+0000 7fe90a4c8640 1 -- 172.21.15.43:0/1407141298 shutdown_connections 2024-04-21T08:45:16.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:16.764+0000 7fe90a4c8640 1 -- 172.21.15.43:0/1407141298 wait complete. 2024-04-21T08:45:16.775 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced, set_id: 683318c5, quiesce duration: 30.0 2024-04-21T08:45:17.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:17 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1924846097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:17.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:17 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1924846097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:17.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:17 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1924846097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:18.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:18 smithi175 ceph-mon[31920]: pgmap v194: 129 pgs: 129 active+clean; 1.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 61 KiB/s wr, 24 op/s 2024-04-21T08:45:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:18 smithi112 ceph-mon[31085]: pgmap v194: 129 pgs: 129 active+clean; 1.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 61 KiB/s wr, 24 op/s 2024-04-21T08:45:18.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:18 smithi043 ceph-mon[27978]: pgmap v194: 129 pgs: 129 active+clean; 1.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 61 KiB/s wr, 24 op/s 2024-04-21T08:45:20.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:20 smithi175 ceph-mon[31920]: pgmap v195: 129 pgs: 129 active+clean; 1.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:45:20.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:20 smithi112 ceph-mon[31085]: pgmap v195: 129 pgs: 129 active+clean; 1.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:45:20.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:20 smithi043 ceph-mon[27978]: pgmap v195: 129 pgs: 129 active+clean; 1.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:45:22.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:22 smithi175 ceph-mon[31920]: pgmap v196: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 49 KiB/s wr, 21 op/s 2024-04-21T08:45:22.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:22 smithi112 ceph-mon[31085]: pgmap v196: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 49 KiB/s wr, 21 op/s 2024-04-21T08:45:22.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:22 smithi043 ceph-mon[27978]: pgmap v196: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 49 KiB/s wr, 21 op/s 2024-04-21T08:45:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.284+0000 7f7dbd3db640 1 -- 172.21.15.43:0/3722881995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7db81006c0 msgr2=0x7f7db8100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.284+0000 7f7dbd3db640 1 --2- 172.21.15.43:0/3722881995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7db81006c0 0x7f7db8100aa0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009ca0 tx=0x7f7db002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.284+0000 7f7dbd3db640 1 -- 172.21.15.43:0/3722881995 shutdown_connections 2024-04-21T08:45:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.284+0000 7f7dbd3db640 1 --2- 172.21.15.43:0/3722881995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db8107330 0x7f7db810eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.284+0000 7f7dbd3db640 1 --2- 172.21.15.43:0/3722881995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7db8101070 0x7f7db8106c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.284+0000 7f7dbd3db640 1 --2- 172.21.15.43:0/3722881995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7db81006c0 0x7f7db8100aa0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.284+0000 7f7dbd3db640 1 -- 172.21.15.43:0/3722881995 >> 172.21.15.43:0/3722881995 conn(0x7f7db8077390 msgr2=0x7f7db80fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.284+0000 7f7dbd3db640 1 -- 172.21.15.43:0/3722881995 shutdown_connections 2024-04-21T08:45:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.284+0000 7f7dbd3db640 1 -- 172.21.15.43:0/3722881995 wait complete. 2024-04-21T08:45:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.285+0000 7f7dbd3db640 1 Processor -- start 2024-04-21T08:45:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.285+0000 7f7dbd3db640 1 -- start start 2024-04-21T08:45:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.285+0000 7f7dbd3db640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db81006c0 0x7f7db8198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.286+0000 7f7dbd3db640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7db8107330 0x7f7db8198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.286+0000 7f7dbd3db640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7db819cf40 0x7f7db819d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.286+0000 7f7dbd3db640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7db8112220 con 0x7f7db819cf40 2024-04-21T08:45:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.286+0000 7f7dbd3db640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81120a0 con 0x7f7db81006c0 2024-04-21T08:45:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.286+0000 7f7dbd3db640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81123a0 con 0x7f7db8107330 2024-04-21T08:45:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.286+0000 7f7db77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7db819cf40 0x7f7db819d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.286+0000 7f7db77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7db819cf40 0x7f7db819d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57642/0 (socket says 172.21.15.43:57642) 2024-04-21T08:45:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.286+0000 7f7db77fe640 1 -- 172.21.15.43:0/2274715865 learned_addr learned my addr 172.21.15.43:0/2274715865 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.286+0000 7f7db67fc640 1 --2- 172.21.15.43:0/2274715865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7db8107330 0x7f7db8198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.286+0000 7f7db6ffd640 1 --2- 172.21.15.43:0/2274715865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db81006c0 0x7f7db8198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.286+0000 7f7db77fe640 1 -- 172.21.15.43:0/2274715865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7db8107330 msgr2=0x7f7db8198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.286+0000 7f7db77fe640 1 --2- 172.21.15.43:0/2274715865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7db8107330 0x7f7db8198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.286+0000 7f7db77fe640 1 -- 172.21.15.43:0/2274715865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db81006c0 msgr2=0x7f7db8198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.286+0000 7f7db77fe640 1 --2- 172.21.15.43:0/2274715865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db81006c0 0x7f7db8198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.286+0000 7f7db77fe640 1 -- 172.21.15.43:0/2274715865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db819da30 con 0x7f7db819cf40 2024-04-21T08:45:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.286+0000 7f7db77fe640 1 --2- 172.21.15.43:0/2274715865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7db819cf40 0x7f7db819d3f0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f7da800d950 tx=0x7f7da800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.286+0000 7f7daffff640 1 -- 172.21.15.43:0/2274715865 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8017030 con 0x7f7db819cf40 2024-04-21T08:45:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.287+0000 7f7daffff640 1 -- 172.21.15.43:0/2274715865 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7da8004020 con 0x7f7db819cf40 2024-04-21T08:45:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.287+0000 7f7dbd3db640 1 -- 172.21.15.43:0/2274715865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db819dd20 con 0x7f7db819cf40 2024-04-21T08:45:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.287+0000 7f7daffff640 1 -- 172.21.15.43:0/2274715865 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8003af0 con 0x7f7db819cf40 2024-04-21T08:45:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.287+0000 7f7dbd3db640 1 -- 172.21.15.43:0/2274715865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db81a54c0 con 0x7f7db819cf40 2024-04-21T08:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.289+0000 7f7daffff640 1 -- 172.21.15.43:0/2274715865 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7da801a070 con 0x7f7db819cf40 2024-04-21T08:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.289+0000 7f7dbffcf740 1 Processor -- start 2024-04-21T08:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.289+0000 7f7dbffcf740 1 -- start start 2024-04-21T08:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.290+0000 7f7dbffcf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed047fd8d0 0x55ed047fc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.290+0000 7f7dbffcf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed047fcb60 0x55ed047fcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.290+0000 7f7daffff640 1 --2- 172.21.15.43:0/2274715865 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7da40a3dc0 0x7f7da40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.290+0000 7f7dbffcf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed0480a4f0 0x55ed0480c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.290+0000 7f7dbffcf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ed047618d0 con 0x55ed047fcb60 2024-04-21T08:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.290+0000 7f7dbffcf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ed046fee80 con 0x55ed0480a4f0 2024-04-21T08:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.290+0000 7f7dbffcf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ed0475d890 con 0x55ed047fd8d0 2024-04-21T08:45:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.290+0000 7f7db6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed047fcb60 0x55ed047fcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.290+0000 7f7db6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed047fcb60 0x55ed047fcfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57654/0 (socket says 172.21.15.43:57654) 2024-04-21T08:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.290+0000 7f7db67fc640 1 --2- 172.21.15.43:0/2274715865 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7da40a3dc0 0x7f7da40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.290+0000 7f7db6ffd640 1 -- 172.21.15.43:0/4187086658 learned_addr learned my addr 172.21.15.43:0/4187086658 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.290+0000 7f7daffff640 1 -- 172.21.15.43:0/2274715865 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f7da80d05c0 con 0x7f7db819cf40 2024-04-21T08:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.290+0000 7f7db77fe640 1 --2- 172.21.15.43:0/4187086658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed0480a4f0 0x55ed0480c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.290+0000 7f7db67fc640 1 --2- 172.21.15.43:0/4187086658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed047fd8d0 0x55ed047fc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.290+0000 7f7db6ffd640 1 -- 172.21.15.43:0/4187086658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed047fd8d0 msgr2=0x55ed047fc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.290+0000 7f7db6ffd640 1 --2- 172.21.15.43:0/4187086658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed047fd8d0 0x55ed047fc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.290+0000 7f7db6ffd640 1 -- 172.21.15.43:0/4187086658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed0480a4f0 msgr2=0x55ed0480c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.290+0000 7f7db6ffd640 1 --2- 172.21.15.43:0/4187086658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed0480a4f0 0x55ed0480c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.290+0000 7f7db6ffd640 1 -- 172.21.15.43:0/4187086658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed04720320 con 0x55ed047fcb60 2024-04-21T08:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.290+0000 7f7db67fc640 1 --2- 172.21.15.43:0/4187086658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed047fd8d0 0x55ed047fc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.291+0000 7f7db6ffd640 1 --2- 172.21.15.43:0/4187086658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed047fcb60 0x55ed047fcfe0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f7db002c700 tx=0x7f7db00098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75783f727d3baf56 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.291+0000 7f7db67fc640 1 --2- 172.21.15.43:0/2274715865 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7da40a3dc0 0x7f7da40a6280 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7da000a5c0 tx=0x7f7da000a550 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.291+0000 7f7dad7fa640 1 -- 172.21.15.43:0/4187086658 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db0043070 con 0x55ed047fcb60 2024-04-21T08:45:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.291+0000 7f7dad7fa640 1 -- 172.21.15.43:0/4187086658 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7db0032970 con 0x55ed047fcb60 2024-04-21T08:45:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.291+0000 7f7dad7fa640 1 -- 172.21.15.43:0/4187086658 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db0039040 con 0x55ed047fcb60 2024-04-21T08:45:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.292+0000 7f7dbffcf740 1 -- 172.21.15.43:0/4187086658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed047fcb60 msgr2=0x55ed047fcfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.292+0000 7f7dbffcf740 1 --2- 172.21.15.43:0/4187086658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed047fcb60 0x55ed047fcfe0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f7db002c700 tx=0x7f7db00098c0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.292+0000 7f7dbffcf740 1 -- 172.21.15.43:0/4187086658 shutdown_connections 2024-04-21T08:45:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.292+0000 7f7dbffcf740 1 --2- 172.21.15.43:0/4187086658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed0480a4f0 0x55ed0480c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.292+0000 7f7dbffcf740 1 --2- 172.21.15.43:0/4187086658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed047fcb60 0x55ed047fcfe0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.292+0000 7f7dbffcf740 1 --2- 172.21.15.43:0/4187086658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed047fd8d0 0x55ed047fc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.292+0000 7f7dbffcf740 1 -- 172.21.15.43:0/4187086658 >> 172.21.15.43:0/4187086658 conn(0x55ed047c3c90 msgr2=0x55ed047f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.292+0000 7f7dbffcf740 1 -- 172.21.15.43:0/4187086658 shutdown_connections 2024-04-21T08:45:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.292+0000 7f7dbffcf740 1 -- 172.21.15.43:0/4187086658 wait complete. 2024-04-21T08:45:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.292+0000 7f7dbffcf740 1 -- start start 2024-04-21T08:45:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.292+0000 7f7dbffcf740 1 Processor -- start 2024-04-21T08:45:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.292+0000 7f7dbffcf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed047fcb60 0x55ed04809e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.292+0000 7f7dbffcf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed047fd8d0 0x55ed048071e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.292+0000 7f7dbffcf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed0480a4f0 0x55ed04807740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.292+0000 7f7dbffcf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ed0480d1c0 con 0x55ed047fcb60 2024-04-21T08:45:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.292+0000 7f7dbffcf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ed046e9eb0 con 0x55ed0480a4f0 2024-04-21T08:45:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.292+0000 7f7dbffcf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ed0480d340 con 0x55ed047fd8d0 2024-04-21T08:45:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.293+0000 7f7dbffcf740 10 client.0 _ll_register_callbacks cb 0x55ed04716b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.293+0000 7f7db6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed047fcb60 0x55ed04809e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.293+0000 7f7db6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed047fcb60 0x55ed04809e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57660/0 (socket says 172.21.15.43:57660) 2024-04-21T08:45:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.293+0000 7f7db6ffd640 1 -- 172.21.15.43:0/545876636 learned_addr learned my addr 172.21.15.43:0/545876636 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.293+0000 7f7db77fe640 1 --2- 172.21.15.43:0/545876636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed0480a4f0 0x55ed04807740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.293+0000 7f7db67fc640 1 --2- 172.21.15.43:0/545876636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed047fd8d0 0x55ed048071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.293+0000 7f7db6ffd640 1 -- 172.21.15.43:0/545876636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed047fd8d0 msgr2=0x55ed048071e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.293+0000 7f7db6ffd640 1 --2- 172.21.15.43:0/545876636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed047fd8d0 0x55ed048071e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.293+0000 7f7db6ffd640 1 -- 172.21.15.43:0/545876636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed0480a4f0 msgr2=0x55ed04807740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.293+0000 7f7db6ffd640 1 --2- 172.21.15.43:0/545876636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed0480a4f0 0x55ed04807740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.293+0000 7f7db6ffd640 1 -- 172.21.15.43:0/545876636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed04474210 con 0x55ed047fcb60 2024-04-21T08:45:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.293+0000 7f7db67fc640 1 --2- 172.21.15.43:0/545876636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed047fd8d0 0x55ed048071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.293+0000 7f7db6ffd640 1 --2- 172.21.15.43:0/545876636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed047fcb60 0x55ed04809e50 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f7db0037040 tx=0x7f7db002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede93b2d1eb6e82e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.293+0000 7f7d9e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.293+0000 7f7d9e7fc640 1 -- 172.21.15.43:0/545876636 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db004c030 con 0x55ed047fcb60 2024-04-21T08:45:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.293+0000 7f7d9e7fc640 1 -- 172.21.15.43:0/545876636 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f7db00326d0 con 0x55ed047fcb60 2024-04-21T08:45:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.293+0000 7f7dbffcf740 1 -- 172.21.15.43:0/545876636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7db0032970 con 0x55ed047fcb60 2024-04-21T08:45:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.293+0000 7f7d9e7fc640 1 -- 172.21.15.43:0/545876636 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7db0059050 con 0x55ed047fcb60 2024-04-21T08:45:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.293+0000 7f7dbffcf740 10 client.18393 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.294+0000 7f7dbffcf740 1 -- 172.21.15.43:0/545876636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ed04540620 con 0x55ed047fcb60 2024-04-21T08:45:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.294+0000 7f7d9e7fc640 1 -- 172.21.15.43:0/545876636 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7db0032970 con 0x55ed047fcb60 2024-04-21T08:45:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.294+0000 7f7d9e7fc640 1 -- 172.21.15.43:0/545876636 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7db003f580 con 0x55ed047fcb60 2024-04-21T08:45:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.294+0000 7f7dbffcf740 10 client.18393 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.294+0000 7f7dbffcf740 10 client.18393 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.294+0000 7f7dbffcf740 20 client.18393 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.294+0000 7f7dbffcf740 1 --2- 172.21.15.43:0/545876636 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ed0481a2f0 0x55ed0481c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.294+0000 7f7dbffcf740 4 client.18393 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.294+0000 7f7dbffcf740 1 -- 172.21.15.43:0/545876636 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed04775880 con 0x55ed0481a2f0 2024-04-21T08:45:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.294+0000 7f7db67fc640 1 --2- 172.21.15.43:0/545876636 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ed0481a2f0 0x55ed0481c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.295+0000 7f7d9e7fc640 10 client.18393 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.295+0000 7f7db67fc640 1 --2- 172.21.15.43:0/545876636 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ed0481a2f0 0x55ed0481c6e0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57b6a8a053184077 server_cookie=e7edb445adf96ca0 in_seq=0 out_seq=0 2024-04-21T08:45:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.295+0000 7f7d9e7fc640 1 -- 172.21.15.43:0/545876636 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ed04775880 con 0x55ed0481a2f0 2024-04-21T08:45:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.295+0000 7f7d9e7fc640 10 client.18393 handle_command_reply: tid=0 2024-04-21T08:45:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.295+0000 7f7dbffcf740 1 client.18393 shutdown 2024-04-21T08:45:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.296+0000 7f7dbffcf740 1 -- 172.21.15.43:0/545876636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed047fcb60 msgr2=0x55ed04809e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.296+0000 7f7dbffcf740 1 --2- 172.21.15.43:0/545876636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed047fcb60 0x55ed04809e50 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f7db0037040 tx=0x7f7db002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.296+0000 7f7dbffcf740 1 -- 172.21.15.43:0/545876636 shutdown_connections 2024-04-21T08:45:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.296+0000 7f7dbffcf740 1 --2- 172.21.15.43:0/545876636 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ed0481a2f0 0x55ed0481c6e0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.296+0000 7f7dbffcf740 1 --2- 172.21.15.43:0/545876636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed0480a4f0 0x55ed04807740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.296+0000 7f7dbffcf740 1 --2- 172.21.15.43:0/545876636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed047fd8d0 0x55ed048071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.296+0000 7f7d9e7fc640 0 client.18393 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.296+0000 7f7dbffcf740 1 --2- 172.21.15.43:0/545876636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed047fcb60 0x55ed04809e50 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.296+0000 7f7dbffcf740 1 -- 172.21.15.43:0/545876636 >> 172.21.15.43:0/545876636 conn(0x55ed047c3c90 msgr2=0x55ed047f95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.296+0000 7f7dbffcf740 1 -- 172.21.15.43:0/545876636 shutdown_connections 2024-04-21T08:45:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.296+0000 7f7dbffcf740 1 -- 172.21.15.43:0/545876636 wait complete. 2024-04-21T08:45:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.296+0000 7f7dbffcf740 20 client.18393 trim_cache size 0 max 16384 2024-04-21T08:45:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.311+0000 7f7dbffcf740 1 Processor -- start 2024-04-21T08:45:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.311+0000 7f7dbffcf740 1 -- start start 2024-04-21T08:45:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.311+0000 7f7dbffcf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed047fcb60 0x55ed048160c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.311+0000 7f7dbffcf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed047fd8d0 0x55ed04809910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.311+0000 7f7dbffcf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed0480a4f0 0x55ed04809e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.311+0000 7f7dbffcf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81ad180 con 0x55ed0480a4f0 2024-04-21T08:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.311+0000 7f7dbffcf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81aecc0 con 0x55ed047fd8d0 2024-04-21T08:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.311+0000 7f7dbffcf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81ad850 con 0x55ed047fcb60 2024-04-21T08:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.311+0000 7f7db77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed0480a4f0 0x55ed04809e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.311+0000 7f7db77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed0480a4f0 0x55ed04809e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57668/0 (socket says 172.21.15.43:57668) 2024-04-21T08:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.312+0000 7f7db77fe640 1 -- 172.21.15.43:0/3962206909 learned_addr learned my addr 172.21.15.43:0/3962206909 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.312+0000 7f7db67fc640 1 --2- 172.21.15.43:0/3962206909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed047fd8d0 0x55ed04809910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.312+0000 7f7db6ffd640 1 --2- 172.21.15.43:0/3962206909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed047fcb60 0x55ed048160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.312+0000 7f7db77fe640 1 -- 172.21.15.43:0/3962206909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed047fcb60 msgr2=0x55ed048160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.312+0000 7f7db77fe640 1 --2- 172.21.15.43:0/3962206909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed047fcb60 0x55ed048160c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.312+0000 7f7db77fe640 1 -- 172.21.15.43:0/3962206909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed047fd8d0 msgr2=0x55ed04809910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.312+0000 7f7db77fe640 1 --2- 172.21.15.43:0/3962206909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed047fd8d0 0x55ed04809910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.312+0000 7f7db77fe640 1 -- 172.21.15.43:0/3962206909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed047fbe30 con 0x55ed0480a4f0 2024-04-21T08:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.312+0000 7f7db77fe640 1 --2- 172.21.15.43:0/3962206909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed0480a4f0 0x55ed04809e50 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f7da80cd9a0 tx=0x7f7da809c090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2536e348582c3448 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.312+0000 7f7dacff9640 1 -- 172.21.15.43:0/3962206909 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da80af250 con 0x55ed0480a4f0 2024-04-21T08:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.312+0000 7f7dacff9640 1 -- 172.21.15.43:0/3962206909 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7da809c890 con 0x55ed0480a4f0 2024-04-21T08:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.312+0000 7f7dacff9640 1 -- 172.21.15.43:0/3962206909 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da800be60 con 0x55ed0480a4f0 2024-04-21T08:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.313+0000 7f7dbffcf740 1 -- 172.21.15.43:0/3962206909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed0480a4f0 msgr2=0x55ed04809e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.313+0000 7f7dbffcf740 1 --2- 172.21.15.43:0/3962206909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed0480a4f0 0x55ed04809e50 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f7da80cd9a0 tx=0x7f7da809c090 comp rx=0 tx=0).stop 2024-04-21T08:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.313+0000 7f7dbffcf740 1 -- 172.21.15.43:0/3962206909 shutdown_connections 2024-04-21T08:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.313+0000 7f7dbffcf740 1 --2- 172.21.15.43:0/3962206909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed0480a4f0 0x55ed04809e50 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.313+0000 7f7dbffcf740 1 --2- 172.21.15.43:0/3962206909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed047fd8d0 0x55ed04809910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.313+0000 7f7dbffcf740 1 --2- 172.21.15.43:0/3962206909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed047fcb60 0x55ed048160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.313+0000 7f7dbffcf740 1 -- 172.21.15.43:0/3962206909 >> 172.21.15.43:0/3962206909 conn(0x55ed047c3c90 msgr2=0x55ed04805af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.313+0000 7f7dbffcf740 1 -- 172.21.15.43:0/3962206909 shutdown_connections 2024-04-21T08:45:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.313+0000 7f7dbffcf740 1 -- 172.21.15.43:0/3962206909 wait complete. 2024-04-21T08:45:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.313+0000 7f7dbffcf740 1 -- start start 2024-04-21T08:45:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.313+0000 7f7dbffcf740 1 Processor -- start 2024-04-21T08:45:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.314+0000 7f7dbffcf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed047fcb60 0x55ed04804260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.314+0000 7f7dbffcf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed047fd8d0 0x55ed048047a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.314+0000 7f7dbffcf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed0480a4f0 0x55ed0487f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.314+0000 7f7dbffcf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81ae5f0 con 0x55ed0480a4f0 2024-04-21T08:45:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.314+0000 7f7dbffcf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81adf20 con 0x55ed047fd8d0 2024-04-21T08:45:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.314+0000 7f7dbffcf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81ab160 con 0x55ed047fcb60 2024-04-21T08:45:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.314+0000 7f7dbffcf740 10 client.0 _ll_register_callbacks cb 0x7f7da80044f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.314+0000 7f7db77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed0480a4f0 0x55ed0487f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.314+0000 7f7db77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed0480a4f0 0x55ed0487f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57670/0 (socket says 172.21.15.43:57670) 2024-04-21T08:45:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.314+0000 7f7db77fe640 1 -- 172.21.15.43:0/4270585946 learned_addr learned my addr 172.21.15.43:0/4270585946 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.314+0000 7f7db67fc640 1 --2- 172.21.15.43:0/4270585946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed047fd8d0 0x55ed048047a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.314+0000 7f7db6ffd640 1 --2- 172.21.15.43:0/4270585946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed047fcb60 0x55ed04804260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.314+0000 7f7db77fe640 1 -- 172.21.15.43:0/4270585946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed047fcb60 msgr2=0x55ed04804260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.314+0000 7f7db77fe640 1 --2- 172.21.15.43:0/4270585946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed047fcb60 0x55ed04804260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.314+0000 7f7db77fe640 1 -- 172.21.15.43:0/4270585946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed047fd8d0 msgr2=0x55ed048047a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.314+0000 7f7db77fe640 1 --2- 172.21.15.43:0/4270585946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed047fd8d0 0x55ed048047a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.314+0000 7f7db77fe640 1 -- 172.21.15.43:0/4270585946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed0487fb20 con 0x55ed0480a4f0 2024-04-21T08:45:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.314+0000 7f7db77fe640 1 --2- 172.21.15.43:0/4270585946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed0480a4f0 0x55ed0487f5e0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f7da809be90 tx=0x7f7da8005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a849bae8dff7fc2a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.314+0000 7f7d9e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.314+0000 7f7d9e7fc640 1 -- 172.21.15.43:0/4270585946 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da80a0540 con 0x55ed0480a4f0 2024-04-21T08:45:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.314+0000 7f7d9e7fc640 1 -- 172.21.15.43:0/4270585946 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f7da809a490 con 0x55ed0480a4f0 2024-04-21T08:45:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.314+0000 7f7dbffcf740 1 -- 172.21.15.43:0/4270585946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7da809c890 con 0x55ed0480a4f0 2024-04-21T08:45:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.315+0000 7f7d9e7fc640 1 -- 172.21.15.43:0/4270585946 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7da80a4e00 con 0x55ed0480a4f0 2024-04-21T08:45:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.315+0000 7f7dbffcf740 10 client.18399 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.315+0000 7f7dbffcf740 1 -- 172.21.15.43:0/4270585946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ed04880c70 con 0x55ed0480a4f0 2024-04-21T08:45:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.315+0000 7f7d9e7fc640 1 -- 172.21.15.43:0/4270585946 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7da809c890 con 0x55ed0480a4f0 2024-04-21T08:45:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.315+0000 7f7d9e7fc640 1 -- 172.21.15.43:0/4270585946 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7da8024060 con 0x55ed0480a4f0 2024-04-21T08:45:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.315+0000 7f7dbffcf740 10 client.18399 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.315+0000 7f7dbffcf740 10 client.18399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.315+0000 7f7dbffcf740 20 client.18399 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.315+0000 7f7dbffcf740 1 --2- 172.21.15.43:0/4270585946 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ed048812d0 0x55ed048836c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.315+0000 7f7dbffcf740 4 client.18399 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:45:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.315+0000 7f7dbffcf740 1 -- 172.21.15.43:0/4270585946 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed047013d0 con 0x55ed048812d0 2024-04-21T08:45:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.315+0000 7f7db6ffd640 1 --2- 172.21.15.43:0/4270585946 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ed048812d0 0x55ed048836c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.316+0000 7f7db6ffd640 1 --2- 172.21.15.43:0/4270585946 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ed048812d0 0x55ed048836c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5356ecea925dd75e server_cookie=5c82885c898425ba in_seq=0 out_seq=0 2024-04-21T08:45:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.316+0000 7f7d9e7fc640 10 client.18399 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.330+0000 7f7d9e7fc640 1 -- 172.21.15.43:0/4270585946 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed047013d0 con 0x55ed048812d0 2024-04-21T08:45:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.330+0000 7f7d9e7fc640 10 client.18399 handle_command_reply: tid=0 2024-04-21T08:45:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.330+0000 7f7dbffcf740 1 client.18399 shutdown 2024-04-21T08:45:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.330+0000 7f7dbffcf740 1 -- 172.21.15.43:0/4270585946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed0480a4f0 msgr2=0x55ed0487f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.330+0000 7f7dbffcf740 1 --2- 172.21.15.43:0/4270585946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed0480a4f0 0x55ed0487f5e0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f7da809be90 tx=0x7f7da8005cd0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.330+0000 7f7dbffcf740 1 -- 172.21.15.43:0/4270585946 shutdown_connections 2024-04-21T08:45:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.330+0000 7f7dbffcf740 1 --2- 172.21.15.43:0/4270585946 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ed048812d0 0x55ed048836c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.330+0000 7f7dbffcf740 1 --2- 172.21.15.43:0/4270585946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed0480a4f0 0x55ed0487f5e0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.330+0000 7f7dbffcf740 1 --2- 172.21.15.43:0/4270585946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed047fd8d0 0x55ed048047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.330+0000 7f7d9e7fc640 0 client.18399 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.330+0000 7f7dbffcf740 1 --2- 172.21.15.43:0/4270585946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ed047fcb60 0x55ed04804260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.330+0000 7f7dbffcf740 1 -- 172.21.15.43:0/4270585946 >> 172.21.15.43:0/4270585946 conn(0x55ed047c3c90 msgr2=0x55ed047f8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.330+0000 7f7dbffcf740 1 -- 172.21.15.43:0/4270585946 shutdown_connections 2024-04-21T08:45:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.330+0000 7f7dbffcf740 1 -- 172.21.15.43:0/4270585946 wait complete. 2024-04-21T08:45:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.330+0000 7f7dbffcf740 20 client.18399 trim_cache size 0 max 16384 2024-04-21T08:45:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:45:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:45:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:45:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:45:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.331+0000 7f7dbd3db640 1 -- 172.21.15.43:0/2274715865 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7da40a3dc0 msgr2=0x7f7da40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:23.332 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.331+0000 7f7dbd3db640 1 --2- 172.21.15.43:0/2274715865 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7da40a3dc0 0x7f7da40a6280 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7da000a5c0 tx=0x7f7da000a550 comp rx=0 tx=0).stop 2024-04-21T08:45:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.331+0000 7f7dbd3db640 1 -- 172.21.15.43:0/2274715865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7db819cf40 msgr2=0x7f7db819d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.331+0000 7f7dbd3db640 1 --2- 172.21.15.43:0/2274715865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7db819cf40 0x7f7db819d3f0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f7da800d950 tx=0x7f7da800dd50 comp rx=0 tx=0).stop 2024-04-21T08:45:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.331+0000 7f7dbd3db640 1 -- 172.21.15.43:0/2274715865 shutdown_connections 2024-04-21T08:45:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.331+0000 7f7dbd3db640 1 --2- 172.21.15.43:0/2274715865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7db819cf40 0x7f7db819d3f0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.331+0000 7f7dbd3db640 1 --2- 172.21.15.43:0/2274715865 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7da40a3dc0 0x7f7da40a6280 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.331+0000 7f7dbd3db640 1 --2- 172.21.15.43:0/2274715865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7db8107330 0x7f7db8198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.331+0000 7f7dbd3db640 1 --2- 172.21.15.43:0/2274715865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db81006c0 0x7f7db8198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.331+0000 7f7dbd3db640 1 -- 172.21.15.43:0/2274715865 >> 172.21.15.43:0/2274715865 conn(0x7f7db8077390 msgr2=0x7f7db8105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.331+0000 7f7dbd3db640 1 -- 172.21.15.43:0/2274715865 shutdown_connections 2024-04-21T08:45:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:23.331+0000 7f7dbd3db640 1 -- 172.21.15.43:0/2274715865 wait complete. 2024-04-21T08:45:24.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:24 smithi175 ceph-mon[31920]: pgmap v197: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-04-21T08:45:24.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:24 smithi112 ceph-mon[31085]: pgmap v197: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-04-21T08:45:24.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:24 smithi043 ceph-mon[27978]: pgmap v197: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-04-21T08:45:26.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:26 smithi175 ceph-mon[31920]: pgmap v198: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-04-21T08:45:26.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:26 smithi112 ceph-mon[31085]: pgmap v198: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-04-21T08:45:26.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:26 smithi043 ceph-mon[27978]: pgmap v198: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-04-21T08:45:28.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:28 smithi175 ceph-mon[31920]: pgmap v199: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-04-21T08:45:28.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:28 smithi112 ceph-mon[31085]: pgmap v199: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-04-21T08:45:28.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:28 smithi043 ceph-mon[27978]: pgmap v199: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-04-21T08:45:30.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:30 smithi175 ceph-mon[31920]: pgmap v200: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-21T08:45:30.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:30 smithi112 ceph-mon[31085]: pgmap v200: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-21T08:45:30.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:30 smithi043 ceph-mon[27978]: pgmap v200: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-21T08:45:31.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:45:31.685 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:45:31.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.803+0000 7f209f97e640 1 -- 172.21.15.43:0/1007951521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2098107260 msgr2=0x7f209810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:31.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.803+0000 7f209f97e640 1 --2- 172.21.15.43:0/1007951521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2098107260 0x7f209810ee30 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f2090009900 tx=0x7f209002bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:31.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.803+0000 7f209f97e640 1 -- 172.21.15.43:0/1007951521 shutdown_connections 2024-04-21T08:45:31.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.803+0000 7f209f97e640 1 --2- 172.21.15.43:0/1007951521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2098107260 0x7f209810ee30 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.803+0000 7f209f97e640 1 --2- 172.21.15.43:0/1007951521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2098102b40 0x7f2098106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.803+0000 7f209f97e640 1 --2- 172.21.15.43:0/1007951521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2098102190 0x7f2098102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.803+0000 7f209f97e640 1 -- 172.21.15.43:0/1007951521 >> 172.21.15.43:0/1007951521 conn(0x7f20980fda30 msgr2=0x7f20980ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.803+0000 7f209f97e640 1 -- 172.21.15.43:0/1007951521 shutdown_connections 2024-04-21T08:45:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.803+0000 7f209f97e640 1 -- 172.21.15.43:0/1007951521 wait complete. 2024-04-21T08:45:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.804+0000 7f209f97e640 1 Processor -- start 2024-04-21T08:45:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.804+0000 7f209f97e640 1 -- start start 2024-04-21T08:45:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.804+0000 7f209f97e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2098102190 0x7f209819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.804+0000 7f209f97e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2098107260 0x7f209819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.804+0000 7f209f97e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2098198590 0x7f2098198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.804+0000 7f209f97e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2098112270 con 0x7f2098107260 2024-04-21T08:45:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.804+0000 7f209f97e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f20981120f0 con 0x7f2098102190 2024-04-21T08:45:31.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.804+0000 7f209f97e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20981123f0 con 0x7f2098198590 2024-04-21T08:45:31.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.804+0000 7f209cf19640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2098107260 0x7f209819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:31.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.804+0000 7f209cf19640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2098107260 0x7f209819e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38760/0 (socket says 172.21.15.43:38760) 2024-04-21T08:45:31.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.804+0000 7f209cf19640 1 -- 172.21.15.43:0/533750338 learned_addr learned my addr 172.21.15.43:0/533750338 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:31.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.804+0000 7f209d71a640 1 --2- 172.21.15.43:0/533750338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2098102190 0x7f209819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:31.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.804+0000 7f209df1b640 1 --2- 172.21.15.43:0/533750338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2098198590 0x7f2098198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:31.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.804+0000 7f209cf19640 1 -- 172.21.15.43:0/533750338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2098198590 msgr2=0x7f2098198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:31.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.804+0000 7f209cf19640 1 --2- 172.21.15.43:0/533750338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2098198590 0x7f2098198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.804+0000 7f209cf19640 1 -- 172.21.15.43:0/533750338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2098102190 msgr2=0x7f209819e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:31.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.804+0000 7f209cf19640 1 --2- 172.21.15.43:0/533750338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2098102190 0x7f209819e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.804+0000 7f209cf19640 1 -- 172.21.15.43:0/533750338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20981990b0 con 0x7f2098107260 2024-04-21T08:45:31.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.804+0000 7f209df1b640 1 --2- 172.21.15.43:0/533750338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2098198590 0x7f2098198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:31.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.805+0000 7f209cf19640 1 --2- 172.21.15.43:0/533750338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2098107260 0x7f209819e920 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f208800d770 tx=0x7f208800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:31.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.805+0000 7f208e7fc640 1 -- 172.21.15.43:0/533750338 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088017070 con 0x7f2098107260 2024-04-21T08:45:31.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.805+0000 7f208e7fc640 1 -- 172.21.15.43:0/533750338 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2088009c30 con 0x7f2098107260 2024-04-21T08:45:31.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.805+0000 7f209f97e640 1 -- 172.21.15.43:0/533750338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20981993a0 con 0x7f2098107260 2024-04-21T08:45:31.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.805+0000 7f208e7fc640 1 -- 172.21.15.43:0/533750338 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208800f620 con 0x7f2098107260 2024-04-21T08:45:31.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.805+0000 7f209f97e640 1 -- 172.21.15.43:0/533750338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20981a5640 con 0x7f2098107260 2024-04-21T08:45:31.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.807+0000 7f208e7fc640 1 -- 172.21.15.43:0/533750338 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f208801a070 con 0x7f2098107260 2024-04-21T08:45:31.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.807+0000 7f20a256d740 1 Processor -- start 2024-04-21T08:45:31.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.807+0000 7f20a256d740 1 -- start start 2024-04-21T08:45:31.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.808+0000 7f20a256d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dc14d1230 0x559dc14d1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:31.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.808+0000 7f20a256d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc14d1b50 0x559dc14de950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:31.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.808+0000 7f20a256d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 0x559dc14e1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:31.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.808+0000 7f20a256d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559dc14361e0 con 0x559dc14dee90 2024-04-21T08:45:31.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.808+0000 7f20a256d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559dc13d3390 con 0x559dc14d1230 2024-04-21T08:45:31.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.808+0000 7f20a256d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559dc14321a0 con 0x559dc14d1b50 2024-04-21T08:45:31.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.808+0000 7f208e7fc640 1 --2- 172.21.15.43:0/533750338 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f20840a3da0 0x7f20840a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:31.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.808+0000 7f209cf19640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 0x559dc14e1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:31.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.808+0000 7f209d71a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc14d1b50 0x559dc14de950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:31.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.808+0000 7f209cf19640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 0x559dc14e1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38766/0 (socket says 172.21.15.43:38766) 2024-04-21T08:45:31.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.808+0000 7f209cf19640 1 -- 172.21.15.43:0/4164141828 learned_addr learned my addr 172.21.15.43:0/4164141828 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:31.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.808+0000 7f209df1b640 1 --2- 172.21.15.43:0/4164141828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dc14d1230 0x559dc14d1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:31.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.808+0000 7f209df1b640 1 --2- 172.21.15.43:0/533750338 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f20840a3da0 0x7f20840a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:31.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.808+0000 7f208e7fc640 1 -- 172.21.15.43:0/533750338 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f20880d0720 con 0x7f2098107260 2024-04-21T08:45:31.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.808+0000 7f209cf19640 1 -- 172.21.15.43:0/4164141828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc14d1b50 msgr2=0x559dc14de950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:31.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.808+0000 7f209cf19640 1 --2- 172.21.15.43:0/4164141828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc14d1b50 0x559dc14de950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.808+0000 7f209cf19640 1 -- 172.21.15.43:0/4164141828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dc14d1230 msgr2=0x559dc14d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:31.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.808+0000 7f209cf19640 1 --2- 172.21.15.43:0/4164141828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dc14d1230 0x559dc14d1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.808+0000 7f209cf19640 1 -- 172.21.15.43:0/4164141828 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dc12127d0 con 0x559dc14dee90 2024-04-21T08:45:31.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.808+0000 7f209df1b640 1 --2- 172.21.15.43:0/4164141828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dc14d1230 0x559dc14d1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:31.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.808+0000 7f209d71a640 1 --2- 172.21.15.43:0/4164141828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc14d1b50 0x559dc14de950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:45:31.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.808+0000 7f209cf19640 1 --2- 172.21.15.43:0/4164141828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 0x559dc14e1280 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f20880cdaa0 tx=0x7f208800df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3efe0524323d51c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:31.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.809+0000 7f209df1b640 1 --2- 172.21.15.43:0/533750338 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f20840a3da0 0x7f20840a6260 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2090006010 tx=0x7f2090005a20 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:31.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.809+0000 7f20837fe640 1 -- 172.21.15.43:0/4164141828 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088021070 con 0x559dc14dee90 2024-04-21T08:45:31.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.809+0000 7f20837fe640 1 -- 172.21.15.43:0/4164141828 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20880a0640 con 0x559dc14dee90 2024-04-21T08:45:31.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.809+0000 7f20837fe640 1 -- 172.21.15.43:0/4164141828 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20880cdcc0 con 0x559dc14dee90 2024-04-21T08:45:31.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.809+0000 7f20a256d740 1 -- 172.21.15.43:0/4164141828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 msgr2=0x559dc14e1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:31.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.809+0000 7f20a256d740 1 --2- 172.21.15.43:0/4164141828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 0x559dc14e1280 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f20880cdaa0 tx=0x7f208800df10 comp rx=0 tx=0).stop 2024-04-21T08:45:31.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.809+0000 7f20a256d740 1 -- 172.21.15.43:0/4164141828 shutdown_connections 2024-04-21T08:45:31.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.809+0000 7f20a256d740 1 --2- 172.21.15.43:0/4164141828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 0x559dc14e1280 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.809+0000 7f20a256d740 1 --2- 172.21.15.43:0/4164141828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc14d1b50 0x559dc14de950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.809+0000 7f20a256d740 1 --2- 172.21.15.43:0/4164141828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dc14d1230 0x559dc14d1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.809+0000 7f20a256d740 1 -- 172.21.15.43:0/4164141828 >> 172.21.15.43:0/4164141828 conn(0x559dc14cbcb0 msgr2=0x559dc14cc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:31.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.810+0000 7f20a256d740 1 -- 172.21.15.43:0/4164141828 shutdown_connections 2024-04-21T08:45:31.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.810+0000 7f20a256d740 1 -- 172.21.15.43:0/4164141828 wait complete. 2024-04-21T08:45:31.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.810+0000 7f20a256d740 1 -- start start 2024-04-21T08:45:31.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.810+0000 7f20a256d740 1 Processor -- start 2024-04-21T08:45:31.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.810+0000 7f20a256d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc14d1230 0x559dc14dbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:31.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.810+0000 7f20a256d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dc14d1b50 0x559dc14dc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:31.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.810+0000 7f20a256d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 0x559dc14dc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:31.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.810+0000 7f20a256d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559dc14e1c70 con 0x559dc14dee90 2024-04-21T08:45:31.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.810+0000 7f20a256d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559dc13ef2f0 con 0x559dc14d1b50 2024-04-21T08:45:31.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.810+0000 7f20a256d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559dc14e1df0 con 0x559dc14d1230 2024-04-21T08:45:31.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.810+0000 7f20a256d740 10 client.0 _ll_register_callbacks cb 0x559dc13fc580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.811+0000 7f209df1b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 0x559dc14dc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.811+0000 7f209df1b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 0x559dc14dc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38778/0 (socket says 172.21.15.43:38778) 2024-04-21T08:45:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.811+0000 7f209df1b640 1 -- 172.21.15.43:0/3428271475 learned_addr learned my addr 172.21.15.43:0/3428271475 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.811+0000 7f209cf19640 1 --2- 172.21.15.43:0/3428271475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dc14d1b50 0x559dc14dc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.811+0000 7f209d71a640 1 --2- 172.21.15.43:0/3428271475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc14d1230 0x559dc14dbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.811+0000 7f209df1b640 1 -- 172.21.15.43:0/3428271475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc14d1230 msgr2=0x559dc14dbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.811+0000 7f209df1b640 1 --2- 172.21.15.43:0/3428271475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc14d1230 0x559dc14dbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.811+0000 7f209df1b640 1 -- 172.21.15.43:0/3428271475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dc14d1b50 msgr2=0x559dc14dc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.811+0000 7f209df1b640 1 --2- 172.21.15.43:0/3428271475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dc14d1b50 0x559dc14dc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.811+0000 7f209df1b640 1 -- 172.21.15.43:0/3428271475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dc11483f0 con 0x559dc14dee90 2024-04-21T08:45:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.811+0000 7f209cf19640 1 --2- 172.21.15.43:0/3428271475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dc14d1b50 0x559dc14dc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.811+0000 7f209d71a640 1 --2- 172.21.15.43:0/3428271475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc14d1230 0x559dc14dbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.811+0000 7f209df1b640 1 --2- 172.21.15.43:0/3428271475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 0x559dc14dc930 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f2090009900 tx=0x7f2090009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557fa681091b8296 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.811+0000 7f2080ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:31.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.811+0000 7f2080ff9640 1 -- 172.21.15.43:0/3428271475 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090052020 con 0x559dc14dee90 2024-04-21T08:45:31.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.811+0000 7f2080ff9640 1 -- 172.21.15.43:0/3428271475 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f20900048d0 con 0x559dc14dee90 2024-04-21T08:45:31.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.811+0000 7f20a256d740 1 -- 172.21.15.43:0/3428271475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20880a0640 con 0x559dc14dee90 2024-04-21T08:45:31.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.811+0000 7f2080ff9640 1 -- 172.21.15.43:0/3428271475 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f209005d050 con 0x559dc14dee90 2024-04-21T08:45:31.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.811+0000 7f20a256d740 10 client.18411 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:31.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.811+0000 7f20a256d740 1 -- 172.21.15.43:0/3428271475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559dc11080f0 con 0x559dc14dee90 2024-04-21T08:45:31.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.812+0000 7f2080ff9640 1 -- 172.21.15.43:0/3428271475 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20880a0640 con 0x559dc14dee90 2024-04-21T08:45:31.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.812+0000 7f2080ff9640 1 -- 172.21.15.43:0/3428271475 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2090040040 con 0x559dc14dee90 2024-04-21T08:45:31.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.812+0000 7f20a256d740 10 client.18411 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:31.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.812+0000 7f20a256d740 10 client.18411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:31.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.812+0000 7f20a256d740 20 client.18411 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:31.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.812+0000 7f20a256d740 1 --2- 172.21.15.43:0/3428271475 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559dc14eefc0 0x559dc14f1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:31.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.812+0000 7f20a256d740 4 client.18411 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:31.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.812+0000 7f20a256d740 1 -- 172.21.15.43:0/3428271475 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dc1443c00 con 0x559dc14eefc0 2024-04-21T08:45:31.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.812+0000 7f209d71a640 1 --2- 172.21.15.43:0/3428271475 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559dc14eefc0 0x559dc14f1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:31.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.813+0000 7f209d71a640 1 --2- 172.21.15.43:0/3428271475 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559dc14eefc0 0x559dc14f1410 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e97b447691ae2f62 server_cookie=8fb93be85535a9ad in_seq=0 out_seq=0 2024-04-21T08:45:31.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.813+0000 7f2080ff9640 10 client.18411 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:31.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.813+0000 7f2080ff9640 1 -- 172.21.15.43:0/3428271475 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559dc1443c00 con 0x559dc14eefc0 2024-04-21T08:45:31.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.813+0000 7f2080ff9640 10 client.18411 handle_command_reply: tid=0 2024-04-21T08:45:31.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.813+0000 7f20a256d740 1 client.18411 shutdown 2024-04-21T08:45:31.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.814+0000 7f20a256d740 1 -- 172.21.15.43:0/3428271475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 msgr2=0x559dc14dc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:31.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.814+0000 7f20a256d740 1 --2- 172.21.15.43:0/3428271475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 0x559dc14dc930 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f2090009900 tx=0x7f2090009260 comp rx=0 tx=0).stop 2024-04-21T08:45:31.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.814+0000 7f20a256d740 1 -- 172.21.15.43:0/3428271475 shutdown_connections 2024-04-21T08:45:31.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.814+0000 7f20a256d740 1 --2- 172.21.15.43:0/3428271475 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559dc14eefc0 0x559dc14f1410 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.814+0000 7f20a256d740 1 --2- 172.21.15.43:0/3428271475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 0x559dc14dc930 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.814+0000 7f20a256d740 1 --2- 172.21.15.43:0/3428271475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dc14d1b50 0x559dc14dc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.814+0000 7f2080ff9640 0 client.18411 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:31.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.814+0000 7f20a256d740 1 --2- 172.21.15.43:0/3428271475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc14d1230 0x559dc14dbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.814+0000 7f20a256d740 1 -- 172.21.15.43:0/3428271475 >> 172.21.15.43:0/3428271475 conn(0x559dc14cbcb0 msgr2=0x559dc14cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:31.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.814+0000 7f20a256d740 1 -- 172.21.15.43:0/3428271475 shutdown_connections 2024-04-21T08:45:31.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.814+0000 7f20a256d740 1 -- 172.21.15.43:0/3428271475 wait complete. 2024-04-21T08:45:31.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.814+0000 7f20a256d740 20 client.18411 trim_cache size 0 max 16384 2024-04-21T08:45:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.831+0000 7f20a256d740 1 Processor -- start 2024-04-21T08:45:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.831+0000 7f20a256d740 1 -- start start 2024-04-21T08:45:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.832+0000 7f20a256d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dc14d1230 0x559dc14eaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.832+0000 7f20a256d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc14d1b50 0x559dc14dadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.832+0000 7f20a256d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 0x559dc14f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.832+0000 7f20a256d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f20981ad300 con 0x559dc14dee90 2024-04-21T08:45:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.832+0000 7f20a256d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f20981aee40 con 0x559dc14d1230 2024-04-21T08:45:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.832+0000 7f20a256d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20981ad9d0 con 0x559dc14d1b50 2024-04-21T08:45:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.832+0000 7f209df1b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 0x559dc14f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.832+0000 7f209df1b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 0x559dc14f6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38780/0 (socket says 172.21.15.43:38780) 2024-04-21T08:45:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.832+0000 7f209df1b640 1 -- 172.21.15.43:0/2393963285 learned_addr learned my addr 172.21.15.43:0/2393963285 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.832+0000 7f209d71a640 1 --2- 172.21.15.43:0/2393963285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dc14d1230 0x559dc14eaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.832+0000 7f209df1b640 1 -- 172.21.15.43:0/2393963285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc14d1b50 msgr2=0x559dc14dadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.832+0000 7f209df1b640 1 --2- 172.21.15.43:0/2393963285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc14d1b50 0x559dc14dadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.832+0000 7f209df1b640 1 -- 172.21.15.43:0/2393963285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dc14d1230 msgr2=0x559dc14eaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.832+0000 7f209df1b640 1 --2- 172.21.15.43:0/2393963285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dc14d1230 0x559dc14eaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.832+0000 7f209df1b640 1 -- 172.21.15.43:0/2393963285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dc14d2010 con 0x559dc14dee90 2024-04-21T08:45:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.832+0000 7f209df1b640 1 --2- 172.21.15.43:0/2393963285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 0x559dc14f6ce0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f2090009850 tx=0x7f209002c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5024e2dc0b21ed server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.832+0000 7f2082ffd640 1 -- 172.21.15.43:0/2393963285 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090066070 con 0x559dc14dee90 2024-04-21T08:45:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.832+0000 7f2082ffd640 1 -- 172.21.15.43:0/2393963285 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2090047040 con 0x559dc14dee90 2024-04-21T08:45:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.832+0000 7f2082ffd640 1 -- 172.21.15.43:0/2393963285 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090052040 con 0x559dc14dee90 2024-04-21T08:45:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.833+0000 7f20a256d740 1 -- 172.21.15.43:0/2393963285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 msgr2=0x559dc14f6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.833+0000 7f20a256d740 1 --2- 172.21.15.43:0/2393963285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 0x559dc14f6ce0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f2090009850 tx=0x7f209002c2b0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.833+0000 7f20a256d740 1 -- 172.21.15.43:0/2393963285 shutdown_connections 2024-04-21T08:45:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.833+0000 7f20a256d740 1 --2- 172.21.15.43:0/2393963285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 0x559dc14f6ce0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.833+0000 7f20a256d740 1 --2- 172.21.15.43:0/2393963285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc14d1b50 0x559dc14dadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.833+0000 7f20a256d740 1 --2- 172.21.15.43:0/2393963285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dc14d1230 0x559dc14eaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.833+0000 7f20a256d740 1 -- 172.21.15.43:0/2393963285 >> 172.21.15.43:0/2393963285 conn(0x559dc14cbcb0 msgr2=0x559dc14cad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.833+0000 7f20a256d740 1 -- 172.21.15.43:0/2393963285 shutdown_connections 2024-04-21T08:45:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.833+0000 7f20a256d740 1 -- 172.21.15.43:0/2393963285 wait complete. 2024-04-21T08:45:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.833+0000 7f20a256d740 1 -- start start 2024-04-21T08:45:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.834+0000 7f20a256d740 1 Processor -- start 2024-04-21T08:45:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.834+0000 7f20a256d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc14d1230 0x559dc14d91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.834+0000 7f20a256d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dc14d1b50 0x559dc14d9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.834+0000 7f20a256d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 0x559dc1554550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.834+0000 7f20a256d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f20981ae770 con 0x559dc14dee90 2024-04-21T08:45:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.834+0000 7f20a256d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f20981ae0a0 con 0x559dc14d1b50 2024-04-21T08:45:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.834+0000 7f20a256d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20981ab2e0 con 0x559dc14d1230 2024-04-21T08:45:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.834+0000 7f20a256d740 10 client.0 _ll_register_callbacks cb 0x7f2090009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.834+0000 7f209df1b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 0x559dc1554550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.834+0000 7f209df1b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 0x559dc1554550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38790/0 (socket says 172.21.15.43:38790) 2024-04-21T08:45:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.834+0000 7f209df1b640 1 -- 172.21.15.43:0/1789415709 learned_addr learned my addr 172.21.15.43:0/1789415709 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.834+0000 7f209d71a640 1 --2- 172.21.15.43:0/1789415709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc14d1230 0x559dc14d91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.834+0000 7f209cf19640 1 --2- 172.21.15.43:0/1789415709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dc14d1b50 0x559dc14d9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.834+0000 7f209df1b640 1 -- 172.21.15.43:0/1789415709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc14d1230 msgr2=0x559dc14d91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.834+0000 7f209df1b640 1 --2- 172.21.15.43:0/1789415709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc14d1230 0x559dc14d91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.834+0000 7f209df1b640 1 -- 172.21.15.43:0/1789415709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dc14d1b50 msgr2=0x559dc14d9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.834+0000 7f209df1b640 1 --2- 172.21.15.43:0/1789415709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dc14d1b50 0x559dc14d9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.834+0000 7f209df1b640 1 -- 172.21.15.43:0/1789415709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dc1554a90 con 0x559dc14dee90 2024-04-21T08:45:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.835+0000 7f209df1b640 1 --2- 172.21.15.43:0/1789415709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 0x559dc1554550 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f2090005ab0 tx=0x7f2090005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6845cb4ddb37201a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.835+0000 7f2080ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.835+0000 7f2080ff9640 1 -- 172.21.15.43:0/1789415709 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090066070 con 0x559dc14dee90 2024-04-21T08:45:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.835+0000 7f2080ff9640 1 -- 172.21.15.43:0/1789415709 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f20900035f0 con 0x559dc14dee90 2024-04-21T08:45:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.835+0000 7f20a256d740 1 -- 172.21.15.43:0/1789415709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2090047040 con 0x559dc14dee90 2024-04-21T08:45:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.835+0000 7f2080ff9640 1 -- 172.21.15.43:0/1789415709 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2090072050 con 0x559dc14dee90 2024-04-21T08:45:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.835+0000 7f20a256d740 10 client.18417 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.835+0000 7f20a256d740 1 -- 172.21.15.43:0/1789415709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559dc1555d00 con 0x559dc14dee90 2024-04-21T08:45:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.835+0000 7f2080ff9640 1 -- 172.21.15.43:0/1789415709 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2090047040 con 0x559dc14dee90 2024-04-21T08:45:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.835+0000 7f2080ff9640 1 -- 172.21.15.43:0/1789415709 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2090034440 con 0x559dc14dee90 2024-04-21T08:45:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.835+0000 7f20a256d740 10 client.18417 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.835+0000 7f20a256d740 10 client.18417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.835+0000 7f20a256d740 20 client.18417 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.835+0000 7f20a256d740 1 --2- 172.21.15.43:0/1789415709 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559dc14eefc0 0x559dc1558340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.835+0000 7f20a256d740 4 client.18417 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:45:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.835+0000 7f20a256d740 1 -- 172.21.15.43:0/1789415709 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559dc143da70 con 0x559dc14eefc0 2024-04-21T08:45:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.836+0000 7f209d71a640 1 --2- 172.21.15.43:0/1789415709 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559dc14eefc0 0x559dc1558340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.836+0000 7f209d71a640 1 --2- 172.21.15.43:0/1789415709 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559dc14eefc0 0x559dc1558340 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c722e9c86dbc8a36 server_cookie=ca9384df8ba62763 in_seq=0 out_seq=0 2024-04-21T08:45:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.836+0000 7f2080ff9640 10 client.18417 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.836+0000 7f2080ff9640 1 -- 172.21.15.43:0/1789415709 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559dc143da70 con 0x559dc14eefc0 2024-04-21T08:45:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.836+0000 7f2080ff9640 10 client.18417 handle_command_reply: tid=0 2024-04-21T08:45:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.836+0000 7f20a256d740 1 client.18417 shutdown 2024-04-21T08:45:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.836+0000 7f20a256d740 1 -- 172.21.15.43:0/1789415709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 msgr2=0x559dc1554550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.836+0000 7f20a256d740 1 --2- 172.21.15.43:0/1789415709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 0x559dc1554550 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f2090005ab0 tx=0x7f2090005780 comp rx=0 tx=0).stop 2024-04-21T08:45:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.836+0000 7f20a256d740 1 -- 172.21.15.43:0/1789415709 shutdown_connections 2024-04-21T08:45:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.836+0000 7f20a256d740 1 --2- 172.21.15.43:0/1789415709 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559dc14eefc0 0x559dc1558340 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.836+0000 7f20a256d740 1 --2- 172.21.15.43:0/1789415709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559dc14dee90 0x559dc1554550 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.836+0000 7f20a256d740 1 --2- 172.21.15.43:0/1789415709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dc14d1b50 0x559dc14d9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.836+0000 7f2080ff9640 0 client.18417 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.836+0000 7f20a256d740 1 --2- 172.21.15.43:0/1789415709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559dc14d1230 0x559dc14d91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.836+0000 7f20a256d740 1 -- 172.21.15.43:0/1789415709 >> 172.21.15.43:0/1789415709 conn(0x559dc14cbcb0 msgr2=0x559dc14dbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.837+0000 7f20a256d740 1 -- 172.21.15.43:0/1789415709 shutdown_connections 2024-04-21T08:45:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.837+0000 7f20a256d740 1 -- 172.21.15.43:0/1789415709 wait complete. 2024-04-21T08:45:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.837+0000 7f20a256d740 20 client.18417 trim_cache size 0 max 16384 2024-04-21T08:45:31.843 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:45:31.843 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:45:31.843 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:45:31.843 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:45:31.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.837+0000 7f209f97e640 1 -- 172.21.15.43:0/533750338 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f20840a3da0 msgr2=0x7f20840a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:31.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.837+0000 7f209f97e640 1 --2- 172.21.15.43:0/533750338 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f20840a3da0 0x7f20840a6260 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2090006010 tx=0x7f2090005a20 comp rx=0 tx=0).stop 2024-04-21T08:45:31.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.837+0000 7f209f97e640 1 -- 172.21.15.43:0/533750338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2098107260 msgr2=0x7f209819e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.837+0000 7f209f97e640 1 --2- 172.21.15.43:0/533750338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2098107260 0x7f209819e920 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f208800d770 tx=0x7f208800dc30 comp rx=0 tx=0).stop 2024-04-21T08:45:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.837+0000 7f209f97e640 1 -- 172.21.15.43:0/533750338 shutdown_connections 2024-04-21T08:45:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.837+0000 7f209f97e640 1 --2- 172.21.15.43:0/533750338 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f20840a3da0 0x7f20840a6260 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.837+0000 7f209f97e640 1 --2- 172.21.15.43:0/533750338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2098198590 0x7f2098198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.837+0000 7f209f97e640 1 --2- 172.21.15.43:0/533750338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2098107260 0x7f209819e920 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.837+0000 7f209f97e640 1 --2- 172.21.15.43:0/533750338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2098102190 0x7f209819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.837+0000 7f209f97e640 1 -- 172.21.15.43:0/533750338 >> 172.21.15.43:0/533750338 conn(0x7f20980fda30 msgr2=0x7f20980ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.837+0000 7f209f97e640 1 -- 172.21.15.43:0/533750338 shutdown_connections 2024-04-21T08:45:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.837+0000 7f209f97e640 1 -- 172.21.15.43:0/533750338 wait complete. 2024-04-21T08:45:31.848 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:45:31.849 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:45:31.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.953+0000 7f1543855640 1 -- 172.21.15.43:0/918007941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f153c106180 msgr2=0x7f153c10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:31.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.953+0000 7f1543855640 1 --2- 172.21.15.43:0/918007941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f153c106180 0x7f153c10dd50 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f1534009ca0 tx=0x7f153402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.953+0000 7f1543855640 1 -- 172.21.15.43:0/918007941 shutdown_connections 2024-04-21T08:45:31.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.953+0000 7f1543855640 1 --2- 172.21.15.43:0/918007941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f153c106180 0x7f153c10dd50 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.953+0000 7f1543855640 1 --2- 172.21.15.43:0/918007941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f153c101a60 0x7f153c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.953+0000 7f1543855640 1 --2- 172.21.15.43:0/918007941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f153c1010b0 0x7f153c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.953+0000 7f1543855640 1 -- 172.21.15.43:0/918007941 >> 172.21.15.43:0/918007941 conn(0x7f153c0771d0 msgr2=0x7f153c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:31.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.953+0000 7f1543855640 1 -- 172.21.15.43:0/918007941 shutdown_connections 2024-04-21T08:45:31.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.953+0000 7f1543855640 1 -- 172.21.15.43:0/918007941 wait complete. 2024-04-21T08:45:31.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.953+0000 7f1543855640 1 Processor -- start 2024-04-21T08:45:31.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.953+0000 7f1543855640 1 -- start start 2024-04-21T08:45:31.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.954+0000 7f1543855640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f153c1010b0 0x7f153c102b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:31.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.954+0000 7f1543855640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f153c106180 0x7f153c103080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:31.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.954+0000 7f1543855640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f153c1035c0 0x7f153c1a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:31.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.954+0000 7f1543855640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f153c1110f0 con 0x7f153c1035c0 2024-04-21T08:45:31.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.954+0000 7f1543855640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f153c110f70 con 0x7f153c106180 2024-04-21T08:45:31.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.954+0000 7f1543855640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f153c111270 con 0x7f153c1010b0 2024-04-21T08:45:31.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.954+0000 7f1541df2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f153c1035c0 0x7f153c1a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:31.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.954+0000 7f1541df2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f153c1035c0 0x7f153c1a68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38800/0 (socket says 172.21.15.43:38800) 2024-04-21T08:45:31.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.954+0000 7f1541df2640 1 -- 172.21.15.43:0/108881562 learned_addr learned my addr 172.21.15.43:0/108881562 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:31.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.954+0000 7f1540df0640 1 --2- 172.21.15.43:0/108881562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f153c106180 0x7f153c103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:31.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.954+0000 7f15415f1640 1 --2- 172.21.15.43:0/108881562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f153c1010b0 0x7f153c102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:31.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.954+0000 7f1541df2640 1 -- 172.21.15.43:0/108881562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f153c1010b0 msgr2=0x7f153c102b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:31.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.954+0000 7f1541df2640 1 --2- 172.21.15.43:0/108881562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f153c1010b0 0x7f153c102b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.954+0000 7f1541df2640 1 -- 172.21.15.43:0/108881562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f153c106180 msgr2=0x7f153c103080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:31.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.954+0000 7f1541df2640 1 --2- 172.21.15.43:0/108881562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f153c106180 0x7f153c103080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:31.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.954+0000 7f1541df2640 1 -- 172.21.15.43:0/108881562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153c1a6fd0 con 0x7f153c1035c0 2024-04-21T08:45:31.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.954+0000 7f1540df0640 1 --2- 172.21.15.43:0/108881562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f153c106180 0x7f153c103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:31.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.954+0000 7f1541df2640 1 --2- 172.21.15.43:0/108881562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f153c1035c0 0x7f153c1a68d0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f1534005e30 tx=0x7f1534032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:31.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.954+0000 7f15327fc640 1 -- 172.21.15.43:0/108881562 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1534041020 con 0x7f153c1035c0 2024-04-21T08:45:31.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.954+0000 7f15327fc640 1 -- 172.21.15.43:0/108881562 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1534032970 con 0x7f153c1035c0 2024-04-21T08:45:31.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.954+0000 7f1543855640 1 -- 172.21.15.43:0/108881562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f153c1a7260 con 0x7f153c1035c0 2024-04-21T08:45:31.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.954+0000 7f15327fc640 1 -- 172.21.15.43:0/108881562 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1534039040 con 0x7f153c1035c0 2024-04-21T08:45:31.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.955+0000 7f1543855640 1 -- 172.21.15.43:0/108881562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f153c1a7750 con 0x7f153c1035c0 2024-04-21T08:45:31.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.956+0000 7f1543855640 1 -- 172.21.15.43:0/108881562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f153c069230 con 0x7f153c1035c0 2024-04-21T08:45:31.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.957+0000 7f15327fc640 1 -- 172.21.15.43:0/108881562 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1534044020 con 0x7f153c1035c0 2024-04-21T08:45:31.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.957+0000 7f15327fc640 1 --2- 172.21.15.43:0/108881562 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f15280a3d50 0x7f15280a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:31.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.958+0000 7f15415f1640 1 --2- 172.21.15.43:0/108881562 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f15280a3d50 0x7f15280a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:31.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.958+0000 7f15327fc640 1 -- 172.21.15.43:0/108881562 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f15340f2910 con 0x7f153c1035c0 2024-04-21T08:45:31.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.958+0000 7f15415f1640 1 --2- 172.21.15.43:0/108881562 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f15280a3d50 0x7f15280a6210 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1538009b60 tx=0x7f1538005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:31.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:31.962+0000 7f15327fc640 1 -- 172.21.15.43:0/108881562 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f153404b020 con 0x7f153c1035c0 2024-04-21T08:45:32.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.195+0000 7f1543855640 1 -- 172.21.15.43:0/108881562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f153c10a420 con 0x7f153c1035c0 2024-04-21T08:45:32.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.196+0000 7f15327fc640 1 -- 172.21.15.43:0/108881562 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7f15340becf0 con 0x7f153c1035c0 2024-04-21T08:45:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:45:32.196 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:45:32.196 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:45:32.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.198+0000 7f1543855640 1 -- 172.21.15.43:0/108881562 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f15280a3d50 msgr2=0x7f15280a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.198+0000 7f1543855640 1 --2- 172.21.15.43:0/108881562 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f15280a3d50 0x7f15280a6210 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1538009b60 tx=0x7f1538005f90 comp rx=0 tx=0).stop 2024-04-21T08:45:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.198+0000 7f1543855640 1 -- 172.21.15.43:0/108881562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f153c1035c0 msgr2=0x7f153c1a68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.198+0000 7f1543855640 1 --2- 172.21.15.43:0/108881562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f153c1035c0 0x7f153c1a68d0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f1534005e30 tx=0x7f1534032420 comp rx=0 tx=0).stop 2024-04-21T08:45:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.198+0000 7f1543855640 1 -- 172.21.15.43:0/108881562 shutdown_connections 2024-04-21T08:45:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.198+0000 7f1543855640 1 --2- 172.21.15.43:0/108881562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f153c1035c0 0x7f153c1a68d0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.198+0000 7f1543855640 1 --2- 172.21.15.43:0/108881562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f153c106180 0x7f153c103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.198+0000 7f1543855640 1 --2- 172.21.15.43:0/108881562 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f15280a3d50 0x7f15280a6210 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.198+0000 7f1543855640 1 --2- 172.21.15.43:0/108881562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f153c1010b0 0x7f153c102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.198+0000 7f1543855640 1 -- 172.21.15.43:0/108881562 >> 172.21.15.43:0/108881562 conn(0x7f153c0771d0 msgr2=0x7f153c06a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.198+0000 7f1543855640 1 -- 172.21.15.43:0/108881562 shutdown_connections 2024-04-21T08:45:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.198+0000 7f1543855640 1 -- 172.21.15.43:0/108881562 wait complete. 2024-04-21T08:45:32.208 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:45:32.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.314+0000 7faec77ce640 1 -- 172.21.15.43:0/1036087815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec0102190 msgr2=0x7faec0102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.314+0000 7faec77ce640 1 --2- 172.21.15.43:0/1036087815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec0102190 0x7faec0102570 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7faebc009900 tx=0x7faebc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.314+0000 7faec77ce640 1 -- 172.21.15.43:0/1036087815 shutdown_connections 2024-04-21T08:45:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.314+0000 7faec77ce640 1 --2- 172.21.15.43:0/1036087815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec0107260 0x7faec010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.314+0000 7faec77ce640 1 --2- 172.21.15.43:0/1036087815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faec0102b40 0x7faec0106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.314+0000 7faec77ce640 1 --2- 172.21.15.43:0/1036087815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec0102190 0x7faec0102570 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.314+0000 7faec77ce640 1 -- 172.21.15.43:0/1036087815 >> 172.21.15.43:0/1036087815 conn(0x7faec00fd990 msgr2=0x7faec00ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.314+0000 7faec77ce640 1 -- 172.21.15.43:0/1036087815 shutdown_connections 2024-04-21T08:45:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.314+0000 7faec77ce640 1 -- 172.21.15.43:0/1036087815 wait complete. 2024-04-21T08:45:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.314+0000 7faec77ce640 1 Processor -- start 2024-04-21T08:45:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.314+0000 7faec77ce640 1 -- start start 2024-04-21T08:45:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.315+0000 7faec77ce640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec0102190 0x7faec0198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.315+0000 7faec77ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec0107260 0x7faec0198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.315+0000 7faec77ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faec019d020 0x7faec019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.315+0000 7faec77ce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faec0112220 con 0x7faec0107260 2024-04-21T08:45:32.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.315+0000 7faec77ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faec01120a0 con 0x7faec019d020 2024-04-21T08:45:32.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.315+0000 7faec77ce640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faec01123a0 con 0x7faec0102190 2024-04-21T08:45:32.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.315+0000 7faec4d69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec0107260 0x7faec0198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.315+0000 7faec4d69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec0107260 0x7faec0198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38810/0 (socket says 172.21.15.43:38810) 2024-04-21T08:45:32.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.315+0000 7faec4d69640 1 -- 172.21.15.43:0/2684621878 learned_addr learned my addr 172.21.15.43:0/2684621878 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.315+0000 7faec556a640 1 --2- 172.21.15.43:0/2684621878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec0102190 0x7faec0198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.315+0000 7faec4d69640 1 -- 172.21.15.43:0/2684621878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec0102190 msgr2=0x7faec0198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.315+0000 7faec4d69640 1 --2- 172.21.15.43:0/2684621878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec0102190 0x7faec0198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.315+0000 7faec4d69640 1 -- 172.21.15.43:0/2684621878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faec019d020 msgr2=0x7faec019d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.315+0000 7faec4d69640 1 --2- 172.21.15.43:0/2684621878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faec019d020 0x7faec019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.315+0000 7faec4d69640 1 -- 172.21.15.43:0/2684621878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec019db10 con 0x7faec0107260 2024-04-21T08:45:32.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.315+0000 7faec556a640 1 --2- 172.21.15.43:0/2684621878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec0102190 0x7faec0198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:32.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.315+0000 7faec4d69640 1 --2- 172.21.15.43:0/2684621878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec0107260 0x7faec0198ca0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7faeb000c970 tx=0x7faeb000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.315+0000 7faeb67fc640 1 -- 172.21.15.43:0/2684621878 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb0014070 con 0x7faec0107260 2024-04-21T08:45:32.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.315+0000 7faeb67fc640 1 -- 172.21.15.43:0/2684621878 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faeb0004440 con 0x7faec0107260 2024-04-21T08:45:32.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.315+0000 7faec77ce640 1 -- 172.21.15.43:0/2684621878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec019de00 con 0x7faec0107260 2024-04-21T08:45:32.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.315+0000 7faeb67fc640 1 -- 172.21.15.43:0/2684621878 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb0003e50 con 0x7faec0107260 2024-04-21T08:45:32.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.315+0000 7faec77ce640 1 -- 172.21.15.43:0/2684621878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec01a5640 con 0x7faec0107260 2024-04-21T08:45:32.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.317+0000 7faeb67fc640 1 -- 172.21.15.43:0/2684621878 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faeb0017070 con 0x7faec0107260 2024-04-21T08:45:32.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.318+0000 7faeca3c0740 1 Processor -- start 2024-04-21T08:45:32.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.318+0000 7faeca3c0740 1 -- start start 2024-04-21T08:45:32.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.318+0000 7faeca3c0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616b30ea4d0 0x5616b30ea8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.318+0000 7faeca3c0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616b30eae80 0x5616b30f3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.318+0000 7faeca3c0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30f3a00 0x5616b30f5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.318+0000 7faeca3c0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5616b304f1f0 con 0x5616b30f3a00 2024-04-21T08:45:32.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.318+0000 7faeca3c0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5616b2fec4b0 con 0x5616b30eae80 2024-04-21T08:45:32.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.318+0000 7faeca3c0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5616b304b1b0 con 0x5616b30ea4d0 2024-04-21T08:45:32.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.318+0000 7faeb67fc640 1 --2- 172.21.15.43:0/2684621878 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faeac0a3f40 0x7faeac0a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.318+0000 7faec4d69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30f3a00 0x5616b30f5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.318+0000 7faec4d69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30f3a00 0x5616b30f5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38816/0 (socket says 172.21.15.43:38816) 2024-04-21T08:45:32.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.318+0000 7faec4d69640 1 -- 172.21.15.43:0/3021493727 learned_addr learned my addr 172.21.15.43:0/3021493727 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.318+0000 7faec5d6b640 1 --2- 172.21.15.43:0/2684621878 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faeac0a3f40 0x7faeac0a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.318+0000 7faec556a640 1 --2- 172.21.15.43:0/3021493727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616b30eae80 0x5616b30f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.319+0000 7faec5d6b640 1 --2- 172.21.15.43:0/3021493727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616b30ea4d0 0x5616b30ea8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.319+0000 7faeb67fc640 1 -- 172.21.15.43:0/2684621878 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7faeb00d0a80 con 0x7faec0107260 2024-04-21T08:45:32.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.319+0000 7faec4d69640 1 -- 172.21.15.43:0/3021493727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616b30ea4d0 msgr2=0x5616b30ea8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.319+0000 7faec4d69640 1 --2- 172.21.15.43:0/3021493727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616b30ea4d0 0x5616b30ea8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.319+0000 7faec4d69640 1 -- 172.21.15.43:0/3021493727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616b30eae80 msgr2=0x5616b30f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.319+0000 7faec4d69640 1 --2- 172.21.15.43:0/3021493727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616b30eae80 0x5616b30f3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.319+0000 7faec4d69640 1 -- 172.21.15.43:0/3021493727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b2fde1b0 con 0x5616b30f3a00 2024-04-21T08:45:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.319+0000 7faec5d6b640 1 --2- 172.21.15.43:0/2684621878 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faeac0a3f40 0x7faeac0a6400 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faeb80133e0 tx=0x7faeb80137e0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.319+0000 7faec4d69640 1 --2- 172.21.15.43:0/3021493727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30f3a00 0x5616b30f5e50 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7faeb0013760 tx=0x7faeb0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d9c9e523469e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.319+0000 7faeab7fe640 1 -- 172.21.15.43:0/3021493727 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb00a0a00 con 0x5616b30f3a00 2024-04-21T08:45:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.319+0000 7faeab7fe640 1 -- 172.21.15.43:0/3021493727 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faeb00a52c0 con 0x5616b30f3a00 2024-04-21T08:45:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.319+0000 7faeab7fe640 1 -- 172.21.15.43:0/3021493727 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb00ce210 con 0x5616b30f3a00 2024-04-21T08:45:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.320+0000 7faeca3c0740 1 -- 172.21.15.43:0/3021493727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30f3a00 msgr2=0x5616b30f5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.320+0000 7faeca3c0740 1 --2- 172.21.15.43:0/3021493727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30f3a00 0x5616b30f5e50 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7faeb0013760 tx=0x7faeb0000bc0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.320+0000 7faeca3c0740 1 -- 172.21.15.43:0/3021493727 shutdown_connections 2024-04-21T08:45:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.320+0000 7faeca3c0740 1 --2- 172.21.15.43:0/3021493727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30f3a00 0x5616b30f5e50 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.320+0000 7faeca3c0740 1 --2- 172.21.15.43:0/3021493727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616b30eae80 0x5616b30f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.320+0000 7faeca3c0740 1 --2- 172.21.15.43:0/3021493727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616b30ea4d0 0x5616b30ea8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.320+0000 7faeca3c0740 1 -- 172.21.15.43:0/3021493727 >> 172.21.15.43:0/3021493727 conn(0x5616b30e4ba0 msgr2=0x5616b30e4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.320+0000 7faeca3c0740 1 -- 172.21.15.43:0/3021493727 shutdown_connections 2024-04-21T08:45:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.320+0000 7faeca3c0740 1 -- 172.21.15.43:0/3021493727 wait complete. 2024-04-21T08:45:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.320+0000 7faeca3c0740 1 -- start start 2024-04-21T08:45:32.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.320+0000 7faeca3c0740 1 Processor -- start 2024-04-21T08:45:32.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.321+0000 7faeca3c0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616b30ea4d0 0x5616b30f3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.321+0000 7faeca3c0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30eae80 0x5616b30f06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.321+0000 7faeca3c0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616b30f3a00 0x5616b30f0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.321+0000 7faeca3c0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5616b30f6970 con 0x5616b30eae80 2024-04-21T08:45:32.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.321+0000 7faeca3c0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5616b3008500 con 0x5616b30ea4d0 2024-04-21T08:45:32.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.321+0000 7faeca3c0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5616b30f6af0 con 0x5616b30f3a00 2024-04-21T08:45:32.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.321+0000 7faeca3c0740 10 client.0 _ll_register_callbacks cb 0x5616b2fd6b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:32.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.321+0000 7faec4d69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30eae80 0x5616b30f06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.321+0000 7faec4d69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30eae80 0x5616b30f06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38832/0 (socket says 172.21.15.43:38832) 2024-04-21T08:45:32.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.321+0000 7faec4d69640 1 -- 172.21.15.43:0/1216495450 learned_addr learned my addr 172.21.15.43:0/1216495450 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.321+0000 7faec556a640 1 --2- 172.21.15.43:0/1216495450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616b30ea4d0 0x5616b30f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.321+0000 7faec4d69640 1 -- 172.21.15.43:0/1216495450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616b30f3a00 msgr2=0x5616b30f0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.321+0000 7faec4d69640 1 --2- 172.21.15.43:0/1216495450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616b30f3a00 0x5616b30f0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.321+0000 7faec4d69640 1 -- 172.21.15.43:0/1216495450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616b30ea4d0 msgr2=0x5616b30f3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.321+0000 7faec4d69640 1 --2- 172.21.15.43:0/1216495450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616b30ea4d0 0x5616b30f3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.321+0000 7faec4d69640 1 -- 172.21.15.43:0/1216495450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b2d613f0 con 0x5616b30eae80 2024-04-21T08:45:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.321+0000 7faec556a640 1 --2- 172.21.15.43:0/1216495450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616b30ea4d0 0x5616b30f3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.321+0000 7faec4d69640 1 --2- 172.21.15.43:0/1216495450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30eae80 0x5616b30f06b0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7faeb000c940 tx=0x7faeb000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d41c5f47d9b0b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.321+0000 7faea8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.321+0000 7faea8ff9640 1 -- 172.21.15.43:0/1216495450 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb00ce1f0 con 0x5616b30eae80 2024-04-21T08:45:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.321+0000 7faea8ff9640 1 -- 172.21.15.43:0/1216495450 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7faeb000ee40 con 0x5616b30eae80 2024-04-21T08:45:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.321+0000 7faeca3c0740 1 -- 172.21.15.43:0/1216495450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faeb00a52c0 con 0x5616b30eae80 2024-04-21T08:45:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.321+0000 7faea8ff9640 1 -- 172.21.15.43:0/1216495450 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faeb00a0a00 con 0x5616b30eae80 2024-04-21T08:45:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.321+0000 7faeca3c0740 10 client.18435 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.322+0000 7faeca3c0740 1 -- 172.21.15.43:0/1216495450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5616b2e2b7d0 con 0x5616b30eae80 2024-04-21T08:45:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.322+0000 7faea8ff9640 1 -- 172.21.15.43:0/1216495450 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faeb00a52c0 con 0x5616b30eae80 2024-04-21T08:45:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.322+0000 7faea8ff9640 1 -- 172.21.15.43:0/1216495450 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faeb0061ce0 con 0x5616b30eae80 2024-04-21T08:45:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.322+0000 7faeca3c0740 10 client.18435 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:32.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.322+0000 7faeca3c0740 10 client.18435 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:45:32.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.322+0000 7faeca3c0740 20 client.18435 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:32.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.322+0000 7faeca3c0740 1 --2- 172.21.15.43:0/1216495450 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5616b3103a10 0x5616b3105e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.322+0000 7faeca3c0740 4 client.18435 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:32.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.322+0000 7faeca3c0740 1 -- 172.21.15.43:0/1216495450 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616b2fee930 con 0x5616b3103a10 2024-04-21T08:45:32.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.322+0000 7faec556a640 1 --2- 172.21.15.43:0/1216495450 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5616b3103a10 0x5616b3105e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.323+0000 7faec556a640 1 --2- 172.21.15.43:0/1216495450 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5616b3103a10 0x5616b3105e00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ff0ec5efc992df23 server_cookie=309904fcd9c99d48 in_seq=0 out_seq=0 2024-04-21T08:45:32.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.323+0000 7faea8ff9640 10 client.18435 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:45:32.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.324+0000 7faea8ff9640 1 -- 172.21.15.43:0/1216495450 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5616b2fee930 con 0x5616b3103a10 2024-04-21T08:45:32.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.324+0000 7faea8ff9640 10 client.18435 handle_command_reply: tid=0 2024-04-21T08:45:32.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.324+0000 7faeca3c0740 1 client.18435 shutdown 2024-04-21T08:45:32.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.324+0000 7faeca3c0740 1 -- 172.21.15.43:0/1216495450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30eae80 msgr2=0x5616b30f06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.324+0000 7faeca3c0740 1 --2- 172.21.15.43:0/1216495450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30eae80 0x5616b30f06b0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7faeb000c940 tx=0x7faeb000cf60 comp rx=0 tx=0).stop 2024-04-21T08:45:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.324+0000 7faeca3c0740 1 -- 172.21.15.43:0/1216495450 shutdown_connections 2024-04-21T08:45:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.324+0000 7faeca3c0740 1 --2- 172.21.15.43:0/1216495450 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5616b3103a10 0x5616b3105e00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.324+0000 7faeca3c0740 1 --2- 172.21.15.43:0/1216495450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616b30f3a00 0x5616b30f0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.324+0000 7faeca3c0740 1 --2- 172.21.15.43:0/1216495450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30eae80 0x5616b30f06b0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.324+0000 7faea8ff9640 0 client.18435 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:45:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.324+0000 7faeca3c0740 1 --2- 172.21.15.43:0/1216495450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616b30ea4d0 0x5616b30f3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.324+0000 7faeca3c0740 1 -- 172.21.15.43:0/1216495450 >> 172.21.15.43:0/1216495450 conn(0x5616b30e4ba0 msgr2=0x5616b30e68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.324+0000 7faeca3c0740 1 -- 172.21.15.43:0/1216495450 shutdown_connections 2024-04-21T08:45:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.324+0000 7faeca3c0740 1 -- 172.21.15.43:0/1216495450 wait complete. 2024-04-21T08:45:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.324+0000 7faeca3c0740 20 client.18435 trim_cache size 0 max 16384 2024-04-21T08:45:32.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.337+0000 7faeca3c0740 1 Processor -- start 2024-04-21T08:45:32.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.338+0000 7faeca3c0740 1 -- start start 2024-04-21T08:45:32.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.338+0000 7faeca3c0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616b30ea4d0 0x5616b30e6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.338+0000 7faeca3c0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30eae80 0x5616b30f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.338+0000 7faeca3c0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616b30f3a00 0x5616b30f3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.338+0000 7faeca3c0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faec01af4a0 con 0x5616b30eae80 2024-04-21T08:45:32.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.338+0000 7faeca3c0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faec01b0fe0 con 0x5616b30f3a00 2024-04-21T08:45:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.338+0000 7faeca3c0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faec01afb70 con 0x5616b30ea4d0 2024-04-21T08:45:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.338+0000 7faec4d69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30eae80 0x5616b30f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.338+0000 7faec4d69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30eae80 0x5616b30f2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38844/0 (socket says 172.21.15.43:38844) 2024-04-21T08:45:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.338+0000 7faec4d69640 1 -- 172.21.15.43:0/3533886284 learned_addr learned my addr 172.21.15.43:0/3533886284 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.338+0000 7faec556a640 1 --2- 172.21.15.43:0/3533886284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616b30ea4d0 0x5616b30e6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.338+0000 7faec5d6b640 1 --2- 172.21.15.43:0/3533886284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616b30f3a00 0x5616b30f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.338+0000 7faec4d69640 1 -- 172.21.15.43:0/3533886284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616b30ea4d0 msgr2=0x5616b30e6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.338+0000 7faec4d69640 1 --2- 172.21.15.43:0/3533886284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616b30ea4d0 0x5616b30e6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.338+0000 7faec4d69640 1 -- 172.21.15.43:0/3533886284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616b30f3a00 msgr2=0x5616b30f3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.338+0000 7faec4d69640 1 --2- 172.21.15.43:0/3533886284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616b30f3a00 0x5616b30f3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.338+0000 7faec4d69640 1 -- 172.21.15.43:0/3533886284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b30eac20 con 0x5616b30eae80 2024-04-21T08:45:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.338+0000 7faec4d69640 1 --2- 172.21.15.43:0/3533886284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30eae80 0x5616b30f2de0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7faeb0098fd0 tx=0x7faeb00672d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997766f57635cc6d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.338+0000 7faeaaffd640 1 -- 172.21.15.43:0/3533886284 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb00ba080 con 0x5616b30eae80 2024-04-21T08:45:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.339+0000 7faeaaffd640 1 -- 172.21.15.43:0/3533886284 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faeb000ee40 con 0x5616b30eae80 2024-04-21T08:45:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.339+0000 7faeaaffd640 1 -- 172.21.15.43:0/3533886284 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb00ce080 con 0x5616b30eae80 2024-04-21T08:45:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.339+0000 7faeca3c0740 1 -- 172.21.15.43:0/3533886284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30eae80 msgr2=0x5616b30f2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.339+0000 7faeca3c0740 1 --2- 172.21.15.43:0/3533886284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30eae80 0x5616b30f2de0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7faeb0098fd0 tx=0x7faeb00672d0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.339+0000 7faeca3c0740 1 -- 172.21.15.43:0/3533886284 shutdown_connections 2024-04-21T08:45:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.339+0000 7faeca3c0740 1 --2- 172.21.15.43:0/3533886284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616b30f3a00 0x5616b30f3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.339+0000 7faeca3c0740 1 --2- 172.21.15.43:0/3533886284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30eae80 0x5616b30f2de0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.339+0000 7faeca3c0740 1 --2- 172.21.15.43:0/3533886284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616b30ea4d0 0x5616b30e6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.339+0000 7faeca3c0740 1 -- 172.21.15.43:0/3533886284 >> 172.21.15.43:0/3533886284 conn(0x5616b30e4ba0 msgr2=0x5616b30e5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.339+0000 7faeca3c0740 1 -- 172.21.15.43:0/3533886284 shutdown_connections 2024-04-21T08:45:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.339+0000 7faeca3c0740 1 -- 172.21.15.43:0/3533886284 wait complete. 2024-04-21T08:45:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.339+0000 7faeca3c0740 1 -- start start 2024-04-21T08:45:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.339+0000 7faeca3c0740 1 Processor -- start 2024-04-21T08:45:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.340+0000 7faeca3c0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616b30ea4d0 0x5616b3109360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.340+0000 7faeca3c0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616b30eae80 0x5616b31066f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.340+0000 7faeca3c0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30f3a00 0x5616b3106e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.340+0000 7faeca3c0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faec01b0910 con 0x5616b30f3a00 2024-04-21T08:45:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.340+0000 7faeca3c0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faec01b0240 con 0x5616b30eae80 2024-04-21T08:45:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.340+0000 7faeca3c0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faec01ad480 con 0x5616b30ea4d0 2024-04-21T08:45:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.340+0000 7faeca3c0740 10 client.0 _ll_register_callbacks cb 0x7faeb00989c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.340+0000 7faec5d6b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30f3a00 0x5616b3106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.340+0000 7faec5d6b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30f3a00 0x5616b3106e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38858/0 (socket says 172.21.15.43:38858) 2024-04-21T08:45:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.340+0000 7faec5d6b640 1 -- 172.21.15.43:0/2418207904 learned_addr learned my addr 172.21.15.43:0/2418207904 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.340+0000 7faec556a640 1 --2- 172.21.15.43:0/2418207904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616b30ea4d0 0x5616b3109360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.340+0000 7faec4d69640 1 --2- 172.21.15.43:0/2418207904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616b30eae80 0x5616b31066f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.340+0000 7faec5d6b640 1 -- 172.21.15.43:0/2418207904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616b30ea4d0 msgr2=0x5616b3109360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.340+0000 7faec5d6b640 1 --2- 172.21.15.43:0/2418207904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616b30ea4d0 0x5616b3109360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.340+0000 7faec5d6b640 1 -- 172.21.15.43:0/2418207904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616b30eae80 msgr2=0x5616b31066f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.340+0000 7faec5d6b640 1 --2- 172.21.15.43:0/2418207904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616b30eae80 0x5616b31066f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.340+0000 7faec5d6b640 1 -- 172.21.15.43:0/2418207904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b3168bf0 con 0x5616b30f3a00 2024-04-21T08:45:32.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.341+0000 7faec5d6b640 1 --2- 172.21.15.43:0/2418207904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30f3a00 0x5616b3106e10 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7faeb8002230 tx=0x7faeb80013c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588a7393712b448e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.341+0000 7faea8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:32.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.341+0000 7faea8ff9640 1 -- 172.21.15.43:0/2418207904 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb8001cc0 con 0x5616b30f3a00 2024-04-21T08:45:32.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.341+0000 7faea8ff9640 1 -- 172.21.15.43:0/2418207904 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7faeb8018ab0 con 0x5616b30f3a00 2024-04-21T08:45:32.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.341+0000 7faeca3c0740 1 -- 172.21.15.43:0/2418207904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faeb000ee40 con 0x5616b30f3a00 2024-04-21T08:45:32.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.341+0000 7faea8ff9640 1 -- 172.21.15.43:0/2418207904 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faeb8031050 con 0x5616b30f3a00 2024-04-21T08:45:32.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.341+0000 7faeca3c0740 10 client.18441 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:32.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.341+0000 7faeca3c0740 1 -- 172.21.15.43:0/2418207904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5616b3169d70 con 0x5616b30f3a00 2024-04-21T08:45:32.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.341+0000 7faea8ff9640 1 -- 172.21.15.43:0/2418207904 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faeb000ee40 con 0x5616b30f3a00 2024-04-21T08:45:32.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.341+0000 7faea8ff9640 1 -- 172.21.15.43:0/2418207904 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faeb80233f0 con 0x5616b30f3a00 2024-04-21T08:45:32.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.341+0000 7faeca3c0740 10 client.18441 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:32.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.341+0000 7faeca3c0740 10 client.18441 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:45:32.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.341+0000 7faeca3c0740 20 client.18441 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:32.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.341+0000 7faeca3c0740 1 --2- 172.21.15.43:0/2418207904 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5616b316a570 0x5616b316c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.341+0000 7faeca3c0740 4 client.18441 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:45:32.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.341+0000 7faeca3c0740 1 -- 172.21.15.43:0/2418207904 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616b305cc10 con 0x5616b316a570 2024-04-21T08:45:32.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.342+0000 7faec556a640 1 --2- 172.21.15.43:0/2418207904 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5616b316a570 0x5616b316c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.342+0000 7faec556a640 1 --2- 172.21.15.43:0/2418207904 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5616b316a570 0x5616b316c990 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=58b8fc244e80f1a8 server_cookie=56c845e091a77402 in_seq=0 out_seq=0 2024-04-21T08:45:32.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.342+0000 7faea8ff9640 10 client.18441 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:45:32.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.342+0000 7faea8ff9640 1 -- 172.21.15.43:0/2418207904 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616b305cc10 con 0x5616b316a570 2024-04-21T08:45:32.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.342+0000 7faea8ff9640 10 client.18441 handle_command_reply: tid=0 2024-04-21T08:45:32.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.342+0000 7faeca3c0740 1 client.18441 shutdown 2024-04-21T08:45:32.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.342+0000 7faeca3c0740 1 -- 172.21.15.43:0/2418207904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30f3a00 msgr2=0x5616b3106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.342+0000 7faeca3c0740 1 --2- 172.21.15.43:0/2418207904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30f3a00 0x5616b3106e10 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7faeb8002230 tx=0x7faeb80013c0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.342+0000 7faeca3c0740 1 -- 172.21.15.43:0/2418207904 shutdown_connections 2024-04-21T08:45:32.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.342+0000 7faeca3c0740 1 --2- 172.21.15.43:0/2418207904 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5616b316a570 0x5616b316c990 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.342+0000 7faeca3c0740 1 --2- 172.21.15.43:0/2418207904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616b30f3a00 0x5616b3106e10 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.342+0000 7faeca3c0740 1 --2- 172.21.15.43:0/2418207904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616b30eae80 0x5616b31066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.342+0000 7faea8ff9640 0 client.18441 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:45:32.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.342+0000 7faeca3c0740 1 --2- 172.21.15.43:0/2418207904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616b30ea4d0 0x5616b3109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.342+0000 7faeca3c0740 1 -- 172.21.15.43:0/2418207904 >> 172.21.15.43:0/2418207904 conn(0x5616b30e4ba0 msgr2=0x5616b30ebaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.342+0000 7faeca3c0740 1 -- 172.21.15.43:0/2418207904 shutdown_connections 2024-04-21T08:45:32.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.342+0000 7faeca3c0740 1 -- 172.21.15.43:0/2418207904 wait complete. 2024-04-21T08:45:32.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.343+0000 7faeca3c0740 20 client.18441 trim_cache size 0 max 16384 2024-04-21T08:45:32.350 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:45:32.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.343+0000 7faec77ce640 1 -- 172.21.15.43:0/2684621878 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faeac0a3f40 msgr2=0x7faeac0a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.343+0000 7faec77ce640 1 --2- 172.21.15.43:0/2684621878 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faeac0a3f40 0x7faeac0a6400 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faeb80133e0 tx=0x7faeb80137e0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.343+0000 7faec77ce640 1 -- 172.21.15.43:0/2684621878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec0107260 msgr2=0x7faec0198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.343+0000 7faec77ce640 1 --2- 172.21.15.43:0/2684621878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec0107260 0x7faec0198ca0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7faeb000c970 tx=0x7faeb000ce30 comp rx=0 tx=0).stop 2024-04-21T08:45:32.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.343+0000 7faec77ce640 1 -- 172.21.15.43:0/2684621878 shutdown_connections 2024-04-21T08:45:32.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.343+0000 7faec77ce640 1 --2- 172.21.15.43:0/2684621878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faec019d020 0x7faec019d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.343+0000 7faec77ce640 1 --2- 172.21.15.43:0/2684621878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec0107260 0x7faec0198ca0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.343+0000 7faec77ce640 1 --2- 172.21.15.43:0/2684621878 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faeac0a3f40 0x7faeac0a6400 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.343+0000 7faec77ce640 1 --2- 172.21.15.43:0/2684621878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec0102190 0x7faec0198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.343+0000 7faec77ce640 1 -- 172.21.15.43:0/2684621878 >> 172.21.15.43:0/2684621878 conn(0x7faec00fd990 msgr2=0x7faec00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.343+0000 7faec77ce640 1 -- 172.21.15.43:0/2684621878 shutdown_connections 2024-04-21T08:45:32.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.343+0000 7faec77ce640 1 -- 172.21.15.43:0/2684621878 wait complete. 2024-04-21T08:45:32.353 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:45:32.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.458+0000 7fddaa025640 1 -- 172.21.15.43:0/872634805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdda41014a0 msgr2=0x7fdda4106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.458+0000 7fddaa025640 1 --2- 172.21.15.43:0/872634805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdda41014a0 0x7fdda4106bd0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fdd98009900 tx=0x7fdd9802bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:32.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.458+0000 7fddaa025640 1 -- 172.21.15.43:0/872634805 shutdown_connections 2024-04-21T08:45:32.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.458+0000 7fddaa025640 1 --2- 172.21.15.43:0/872634805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdda41072b0 0x7fdda410eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.458+0000 7fddaa025640 1 --2- 172.21.15.43:0/872634805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdda41014a0 0x7fdda4106bd0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.458+0000 7fddaa025640 1 --2- 172.21.15.43:0/872634805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdda4100af0 0x7fdda4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.458+0000 7fddaa025640 1 -- 172.21.15.43:0/872634805 >> 172.21.15.43:0/872634805 conn(0x7fdda40760e0 msgr2=0x7fdda40764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.458+0000 7fddaa025640 1 -- 172.21.15.43:0/872634805 shutdown_connections 2024-04-21T08:45:32.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.458+0000 7fddaa025640 1 -- 172.21.15.43:0/872634805 wait complete. 2024-04-21T08:45:32.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.459+0000 7fddaa025640 1 Processor -- start 2024-04-21T08:45:32.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.459+0000 7fddaa025640 1 -- start start 2024-04-21T08:45:32.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.459+0000 7fddaa025640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdda4100af0 0x7fdda419e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.459+0000 7fddaa025640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdda41072b0 0x7fdda419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.459+0000 7fddaa025640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdda4198610 0x7fdda4198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.459+0000 7fddaa025640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdda4112230 con 0x7fdda4100af0 2024-04-21T08:45:32.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.459+0000 7fddaa025640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41120b0 con 0x7fdda4198610 2024-04-21T08:45:32.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.459+0000 7fddaa025640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41123b0 con 0x7fdda41072b0 2024-04-21T08:45:32.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.459+0000 7fdda37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdda4100af0 0x7fdda419e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.459+0000 7fdda37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdda4100af0 0x7fdda419e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38864/0 (socket says 172.21.15.43:38864) 2024-04-21T08:45:32.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.459+0000 7fdda37fe640 1 -- 172.21.15.43:0/1711092971 learned_addr learned my addr 172.21.15.43:0/1711092971 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.460+0000 7fdda3fff640 1 --2- 172.21.15.43:0/1711092971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdda4198610 0x7fdda4198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.460+0000 7fdda2ffd640 1 --2- 172.21.15.43:0/1711092971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdda41072b0 0x7fdda419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.460+0000 7fdda37fe640 1 -- 172.21.15.43:0/1711092971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdda41072b0 msgr2=0x7fdda419e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.460+0000 7fdda37fe640 1 --2- 172.21.15.43:0/1711092971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdda41072b0 0x7fdda419e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.460+0000 7fdda37fe640 1 -- 172.21.15.43:0/1711092971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdda4198610 msgr2=0x7fdda4198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.460+0000 7fdda37fe640 1 --2- 172.21.15.43:0/1711092971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdda4198610 0x7fdda4198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.460+0000 7fdda37fe640 1 -- 172.21.15.43:0/1711092971 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda4199130 con 0x7fdda4100af0 2024-04-21T08:45:32.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.460+0000 7fdda3fff640 1 --2- 172.21.15.43:0/1711092971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdda4198610 0x7fdda4198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:32.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.460+0000 7fdda37fe640 1 --2- 172.21.15.43:0/1711092971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdda4100af0 0x7fdda419e410 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400d770 tx=0x7fdd9400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.460+0000 7fdda0ff9640 1 -- 172.21.15.43:0/1711092971 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd94017070 con 0x7fdda4100af0 2024-04-21T08:45:32.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.460+0000 7fdda0ff9640 1 -- 172.21.15.43:0/1711092971 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd940054d0 con 0x7fdda4100af0 2024-04-21T08:45:32.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.460+0000 7fddaa025640 1 -- 172.21.15.43:0/1711092971 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda4199420 con 0x7fdda4100af0 2024-04-21T08:45:32.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.460+0000 7fdda0ff9640 1 -- 172.21.15.43:0/1711092971 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd94004c30 con 0x7fdda4100af0 2024-04-21T08:45:32.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.460+0000 7fddaa025640 1 -- 172.21.15.43:0/1711092971 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda41a55d0 con 0x7fdda4100af0 2024-04-21T08:45:32.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:32 smithi175 ceph-mon[31920]: pgmap v201: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-21T08:45:32.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.462+0000 7fdda0ff9640 1 -- 172.21.15.43:0/1711092971 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdd94004e10 con 0x7fdda4100af0 2024-04-21T08:45:32.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.463+0000 7fddacc13740 1 Processor -- start 2024-04-21T08:45:32.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.463+0000 7fddacc13740 1 -- start start 2024-04-21T08:45:32.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.463+0000 7fdda0ff9640 1 --2- 172.21.15.43:0/1711092971 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdd8c0a3d20 0x7fdd8c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.463+0000 7fddacc13740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564ed934330 0x5564ed934710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.463+0000 7fddacc13740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed934ce0 0x5564ed93d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.463+0000 7fddacc13740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed93de40 0x5564ed93e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.463+0000 7fddacc13740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5564ed8991f0 con 0x5564ed934ce0 2024-04-21T08:45:32.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.463+0000 7fddacc13740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5564ed8364b0 con 0x5564ed934330 2024-04-21T08:45:32.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.463+0000 7fddacc13740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5564ed8951b0 con 0x5564ed93de40 2024-04-21T08:45:32.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.463+0000 7fdda3fff640 1 --2- 172.21.15.43:0/1711092971 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdd8c0a3d20 0x7fdd8c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.463+0000 7fdda0ff9640 1 -- 172.21.15.43:0/1711092971 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fdd940d07b0 con 0x7fdda4100af0 2024-04-21T08:45:32.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.463+0000 7fdda2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed934ce0 0x5564ed93d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.463+0000 7fdda2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed934ce0 0x5564ed93d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38870/0 (socket says 172.21.15.43:38870) 2024-04-21T08:45:32.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.463+0000 7fdda2ffd640 1 -- 172.21.15.43:0/3975979549 learned_addr learned my addr 172.21.15.43:0/3975979549 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.463+0000 7fdda37fe640 1 --2- 172.21.15.43:0/3975979549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564ed934330 0x5564ed934710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.463+0000 7fdda3fff640 1 --2- 172.21.15.43:0/3975979549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed93de40 0x5564ed93e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.463+0000 7fdda2ffd640 1 -- 172.21.15.43:0/3975979549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed93de40 msgr2=0x5564ed93e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.463+0000 7fdda2ffd640 1 --2- 172.21.15.43:0/3975979549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed93de40 0x5564ed93e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.463+0000 7fdda2ffd640 1 -- 172.21.15.43:0/3975979549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564ed934330 msgr2=0x5564ed934710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.463+0000 7fdda2ffd640 1 --2- 172.21.15.43:0/3975979549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564ed934330 0x5564ed934710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.463+0000 7fdda2ffd640 1 -- 172.21.15.43:0/3975979549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564ed8281b0 con 0x5564ed934ce0 2024-04-21T08:45:32.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.464+0000 7fdda3fff640 1 --2- 172.21.15.43:0/1711092971 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdd8c0a3d20 0x7fdd8c0a61e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c005fd0 tx=0x7fdd9c014af0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.464+0000 7fdda2ffd640 1 --2- 172.21.15.43:0/3975979549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed934ce0 0x5564ed93d820 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fdd98006010 tx=0x7fdd98005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35e0e029a5ead1c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.464+0000 7fdd89ffb640 1 -- 172.21.15.43:0/3975979549 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98043020 con 0x5564ed934ce0 2024-04-21T08:45:32.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.464+0000 7fdd89ffb640 1 -- 172.21.15.43:0/3975979549 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd9802c7e0 con 0x5564ed934ce0 2024-04-21T08:45:32.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.464+0000 7fdd89ffb640 1 -- 172.21.15.43:0/3975979549 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98005510 con 0x5564ed934ce0 2024-04-21T08:45:32.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.464+0000 7fddacc13740 1 -- 172.21.15.43:0/3975979549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed934ce0 msgr2=0x5564ed93d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.464+0000 7fddacc13740 1 --2- 172.21.15.43:0/3975979549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed934ce0 0x5564ed93d820 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fdd98006010 tx=0x7fdd98005e80 comp rx=0 tx=0).stop 2024-04-21T08:45:32.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.465+0000 7fddacc13740 1 -- 172.21.15.43:0/3975979549 shutdown_connections 2024-04-21T08:45:32.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.465+0000 7fddacc13740 1 --2- 172.21.15.43:0/3975979549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed93de40 0x5564ed93e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.465+0000 7fddacc13740 1 --2- 172.21.15.43:0/3975979549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed934ce0 0x5564ed93d820 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.465+0000 7fddacc13740 1 --2- 172.21.15.43:0/3975979549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564ed934330 0x5564ed934710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.465+0000 7fddacc13740 1 -- 172.21.15.43:0/3975979549 >> 172.21.15.43:0/3975979549 conn(0x5564ed92eba0 msgr2=0x5564ed92ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.465+0000 7fddacc13740 1 -- 172.21.15.43:0/3975979549 shutdown_connections 2024-04-21T08:45:32.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.465+0000 7fddacc13740 1 -- 172.21.15.43:0/3975979549 wait complete. 2024-04-21T08:45:32.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.465+0000 7fddacc13740 1 -- start start 2024-04-21T08:45:32.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.465+0000 7fddacc13740 1 Processor -- start 2024-04-21T08:45:32.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.465+0000 7fddacc13740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed934330 0x5564ed9441f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.465+0000 7fddacc13740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564ed934ce0 0x5564ed9415d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.465+0000 7fddacc13740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed93de40 0x5564ed941b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.465+0000 7fddacc13740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5564ed944d60 con 0x5564ed93de40 2024-04-21T08:45:32.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.465+0000 7fddacc13740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5564ed852500 con 0x5564ed934ce0 2024-04-21T08:45:32.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.465+0000 7fddacc13740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5564ed944ee0 con 0x5564ed934330 2024-04-21T08:45:32.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.466+0000 7fddacc13740 10 client.0 _ll_register_callbacks cb 0x5564ed820b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:32.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.466+0000 7fdda3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed93de40 0x5564ed941b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.466+0000 7fdda3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed93de40 0x5564ed941b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38872/0 (socket says 172.21.15.43:38872) 2024-04-21T08:45:32.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.466+0000 7fdda3fff640 1 -- 172.21.15.43:0/4072041180 learned_addr learned my addr 172.21.15.43:0/4072041180 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.466+0000 7fdda2ffd640 1 --2- 172.21.15.43:0/4072041180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564ed934ce0 0x5564ed9415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.466+0000 7fdda37fe640 1 --2- 172.21.15.43:0/4072041180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed934330 0x5564ed9441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.466+0000 7fdda3fff640 1 -- 172.21.15.43:0/4072041180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed934330 msgr2=0x5564ed9441f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.466+0000 7fdda3fff640 1 --2- 172.21.15.43:0/4072041180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed934330 0x5564ed9441f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.466+0000 7fdda3fff640 1 -- 172.21.15.43:0/4072041180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564ed934ce0 msgr2=0x5564ed9415d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.466+0000 7fdda3fff640 1 --2- 172.21.15.43:0/4072041180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564ed934ce0 0x5564ed9415d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.466+0000 7fdda3fff640 1 -- 172.21.15.43:0/4072041180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564ed5ab3f0 con 0x5564ed93de40 2024-04-21T08:45:32.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.466+0000 7fdda2ffd640 1 --2- 172.21.15.43:0/4072041180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564ed934ce0 0x5564ed9415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:32.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.466+0000 7fdda3fff640 1 --2- 172.21.15.43:0/4072041180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed93de40 0x5564ed941b10 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c0142a0 tx=0x7fdd9c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338d8de4d687eb60 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.466+0000 7fdd82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:32.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.466+0000 7fdd82ffd640 1 -- 172.21.15.43:0/4072041180 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c026020 con 0x5564ed93de40 2024-04-21T08:45:32.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.466+0000 7fdd82ffd640 1 -- 172.21.15.43:0/4072041180 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fdd9c021950 con 0x5564ed93de40 2024-04-21T08:45:32.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.466+0000 7fddacc13740 1 -- 172.21.15.43:0/4072041180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd9802c7e0 con 0x5564ed93de40 2024-04-21T08:45:32.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.466+0000 7fdd82ffd640 1 -- 172.21.15.43:0/4072041180 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdd9c031050 con 0x5564ed93de40 2024-04-21T08:45:32.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.466+0000 7fddacc13740 10 client.18453 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:32.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.466+0000 7fddacc13740 1 -- 172.21.15.43:0/4072041180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5564ed6757d0 con 0x5564ed93de40 2024-04-21T08:45:32.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.467+0000 7fdd82ffd640 1 -- 172.21.15.43:0/4072041180 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd9802c7e0 con 0x5564ed93de40 2024-04-21T08:45:32.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.467+0000 7fdd82ffd640 1 -- 172.21.15.43:0/4072041180 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdd9c00f040 con 0x5564ed93de40 2024-04-21T08:45:32.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.467+0000 7fddacc13740 10 client.18453 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:32.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.467+0000 7fddacc13740 10 client.18453 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:45:32.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.467+0000 7fddacc13740 20 client.18453 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:32.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.467+0000 7fddacc13740 1 --2- 172.21.15.43:0/4072041180 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5564ed952050 0x5564ed9544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.467+0000 7fddacc13740 4 client.18453 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:32.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.467+0000 7fddacc13740 1 -- 172.21.15.43:0/4072041180 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564ed838930 con 0x5564ed952050 2024-04-21T08:45:32.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.467+0000 7fdda37fe640 1 --2- 172.21.15.43:0/4072041180 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5564ed952050 0x5564ed9544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.468+0000 7fdda37fe640 1 --2- 172.21.15.43:0/4072041180 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5564ed952050 0x5564ed9544f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=de37a606360ec5cf server_cookie=9822e1a9d391a0bd in_seq=0 out_seq=0 2024-04-21T08:45:32.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.468+0000 7fdd82ffd640 10 client.18453 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:45:32.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.468+0000 7fdd82ffd640 1 -- 172.21.15.43:0/4072041180 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5564ed838930 con 0x5564ed952050 2024-04-21T08:45:32.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.468+0000 7fdd82ffd640 10 client.18453 handle_command_reply: tid=0 2024-04-21T08:45:32.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.468+0000 7fddacc13740 1 client.18453 shutdown 2024-04-21T08:45:32.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.469+0000 7fddacc13740 1 -- 172.21.15.43:0/4072041180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed93de40 msgr2=0x5564ed941b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.469+0000 7fddacc13740 1 --2- 172.21.15.43:0/4072041180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed93de40 0x5564ed941b10 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c0142a0 tx=0x7fdd9c004ab0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.469+0000 7fddacc13740 1 -- 172.21.15.43:0/4072041180 shutdown_connections 2024-04-21T08:45:32.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.469+0000 7fddacc13740 1 --2- 172.21.15.43:0/4072041180 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5564ed952050 0x5564ed9544f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.469+0000 7fddacc13740 1 --2- 172.21.15.43:0/4072041180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed93de40 0x5564ed941b10 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.469+0000 7fddacc13740 1 --2- 172.21.15.43:0/4072041180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564ed934ce0 0x5564ed9415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.469+0000 7fdd82ffd640 0 client.18453 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:45:32.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.469+0000 7fddacc13740 1 --2- 172.21.15.43:0/4072041180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed934330 0x5564ed9441f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.469+0000 7fddacc13740 1 -- 172.21.15.43:0/4072041180 >> 172.21.15.43:0/4072041180 conn(0x5564ed92eba0 msgr2=0x5564ed9308f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.469+0000 7fddacc13740 1 -- 172.21.15.43:0/4072041180 shutdown_connections 2024-04-21T08:45:32.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.469+0000 7fddacc13740 1 -- 172.21.15.43:0/4072041180 wait complete. 2024-04-21T08:45:32.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.469+0000 7fddacc13740 20 client.18453 trim_cache size 0 max 16384 2024-04-21T08:45:32.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.482+0000 7fddacc13740 1 Processor -- start 2024-04-21T08:45:32.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.482+0000 7fddacc13740 1 -- start start 2024-04-21T08:45:32.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.482+0000 7fddacc13740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed934330 0x5564ed94dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.482+0000 7fddacc13740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed934ce0 0x5564ed92da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.482+0000 7fddacc13740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564ed93de40 0x5564ed943cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.482+0000 7fddacc13740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41ad1d0 con 0x5564ed934ce0 2024-04-21T08:45:32.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.482+0000 7fddacc13740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41aed10 con 0x5564ed93de40 2024-04-21T08:45:32.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.482+0000 7fddacc13740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41ad8a0 con 0x5564ed934330 2024-04-21T08:45:32.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.482+0000 7fdda2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed934ce0 0x5564ed92da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.482+0000 7fdda2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed934ce0 0x5564ed92da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38874/0 (socket says 172.21.15.43:38874) 2024-04-21T08:45:32.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.482+0000 7fdda2ffd640 1 -- 172.21.15.43:0/535346355 learned_addr learned my addr 172.21.15.43:0/535346355 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.483+0000 7fdda3fff640 1 --2- 172.21.15.43:0/535346355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564ed93de40 0x5564ed943cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.483+0000 7fdda2ffd640 1 -- 172.21.15.43:0/535346355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed934330 msgr2=0x5564ed94dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.483+0000 7fdda2ffd640 1 --2- 172.21.15.43:0/535346355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed934330 0x5564ed94dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.483+0000 7fdda2ffd640 1 -- 172.21.15.43:0/535346355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564ed93de40 msgr2=0x5564ed943cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.483+0000 7fdda2ffd640 1 --2- 172.21.15.43:0/535346355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564ed93de40 0x5564ed943cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.483+0000 7fdda2ffd640 1 -- 172.21.15.43:0/535346355 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564ed93b7f0 con 0x5564ed934ce0 2024-04-21T08:45:32.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.483+0000 7fdda2ffd640 1 --2- 172.21.15.43:0/535346355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed934ce0 0x5564ed92da80 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fdd98006010 tx=0x7fdd98005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a616243d56f3e4f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.483+0000 7fdd897fa640 1 -- 172.21.15.43:0/535346355 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9804c030 con 0x5564ed934ce0 2024-04-21T08:45:32.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.483+0000 7fdd897fa640 1 -- 172.21.15.43:0/535346355 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd98004040 con 0x5564ed934ce0 2024-04-21T08:45:32.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.483+0000 7fdd897fa640 1 -- 172.21.15.43:0/535346355 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98043040 con 0x5564ed934ce0 2024-04-21T08:45:32.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.484+0000 7fddacc13740 1 -- 172.21.15.43:0/535346355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed934ce0 msgr2=0x5564ed92da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.484+0000 7fddacc13740 1 --2- 172.21.15.43:0/535346355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed934ce0 0x5564ed92da80 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fdd98006010 tx=0x7fdd98005a20 comp rx=0 tx=0).stop 2024-04-21T08:45:32.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.484+0000 7fddacc13740 1 -- 172.21.15.43:0/535346355 shutdown_connections 2024-04-21T08:45:32.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.484+0000 7fddacc13740 1 --2- 172.21.15.43:0/535346355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564ed93de40 0x5564ed943cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.484+0000 7fddacc13740 1 --2- 172.21.15.43:0/535346355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed934ce0 0x5564ed92da80 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.484+0000 7fddacc13740 1 --2- 172.21.15.43:0/535346355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed934330 0x5564ed94dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.484+0000 7fddacc13740 1 -- 172.21.15.43:0/535346355 >> 172.21.15.43:0/535346355 conn(0x5564ed92eba0 msgr2=0x5564ed92d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.484+0000 7fddacc13740 1 -- 172.21.15.43:0/535346355 shutdown_connections 2024-04-21T08:45:32.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.484+0000 7fddacc13740 1 -- 172.21.15.43:0/535346355 wait complete. 2024-04-21T08:45:32.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.484+0000 7fddacc13740 1 -- start start 2024-04-21T08:45:32.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.484+0000 7fddacc13740 1 Processor -- start 2024-04-21T08:45:32.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.484+0000 7fddacc13740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed934330 0x5564ed94a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.484+0000 7fddacc13740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564ed934ce0 0x5564ed942f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.484+0000 7fddacc13740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed93de40 0x5564ed9434a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.484+0000 7fddacc13740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41ae640 con 0x5564ed934330 2024-04-21T08:45:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.484+0000 7fddacc13740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41adf70 con 0x5564ed934ce0 2024-04-21T08:45:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.484+0000 7fddacc13740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41ab1b0 con 0x5564ed93de40 2024-04-21T08:45:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.484+0000 7fddacc13740 10 client.0 _ll_register_callbacks cb 0x7fdd9c0141b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.485+0000 7fdda37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed934330 0x5564ed94a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.485+0000 7fdda37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed934330 0x5564ed94a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38890/0 (socket says 172.21.15.43:38890) 2024-04-21T08:45:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.485+0000 7fdda37fe640 1 -- 172.21.15.43:0/596324058 learned_addr learned my addr 172.21.15.43:0/596324058 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.485+0000 7fdda2ffd640 1 --2- 172.21.15.43:0/596324058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564ed934ce0 0x5564ed942f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.485+0000 7fdda37fe640 1 -- 172.21.15.43:0/596324058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed93de40 msgr2=0x5564ed9434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.485+0000 7fdda37fe640 1 --2- 172.21.15.43:0/596324058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed93de40 0x5564ed9434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.485+0000 7fdda37fe640 1 -- 172.21.15.43:0/596324058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564ed934ce0 msgr2=0x5564ed942f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.485+0000 7fdda37fe640 1 --2- 172.21.15.43:0/596324058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564ed934ce0 0x5564ed942f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.485+0000 7fdda37fe640 1 -- 172.21.15.43:0/596324058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564ed943cd0 con 0x5564ed934330 2024-04-21T08:45:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.485+0000 7fdda37fe640 1 --2- 172.21.15.43:0/596324058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed934330 0x5564ed94a4f0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fdd9409c020 tx=0x7fdd9409c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5cc96b5fe461e67 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.485+0000 7fdd82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.485+0000 7fdd82ffd640 1 -- 172.21.15.43:0/596324058 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd94020040 con 0x5564ed934330 2024-04-21T08:45:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.485+0000 7fdd82ffd640 1 -- 172.21.15.43:0/596324058 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fdd9409a620 con 0x5564ed934330 2024-04-21T08:45:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.485+0000 7fddacc13740 1 -- 172.21.15.43:0/596324058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd98004040 con 0x5564ed934330 2024-04-21T08:45:32.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.485+0000 7fdd82ffd640 1 -- 172.21.15.43:0/596324058 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdd94098990 con 0x5564ed934330 2024-04-21T08:45:32.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.485+0000 7fddacc13740 10 client.18459 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:32.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.485+0000 7fddacc13740 1 -- 172.21.15.43:0/596324058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5564ed9b8630 con 0x5564ed934330 2024-04-21T08:45:32.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.485+0000 7fdd82ffd640 1 -- 172.21.15.43:0/596324058 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd98004040 con 0x5564ed934330 2024-04-21T08:45:32.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.486+0000 7fdd82ffd640 1 -- 172.21.15.43:0/596324058 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdd940d0300 con 0x5564ed934330 2024-04-21T08:45:32.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.486+0000 7fddacc13740 10 client.18459 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:32.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.486+0000 7fddacc13740 10 client.18459 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:45:32.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.486+0000 7fddacc13740 20 client.18459 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:32.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.486+0000 7fddacc13740 1 --2- 172.21.15.43:0/596324058 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5564ed9b8dd0 0x5564ed9bb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.486+0000 7fddacc13740 4 client.18459 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:45:32.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.486+0000 7fddacc13740 1 -- 172.21.15.43:0/596324058 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5564ed941ea0 con 0x5564ed9b8dd0 2024-04-21T08:45:32.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.486+0000 7fdda2ffd640 1 --2- 172.21.15.43:0/596324058 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5564ed9b8dd0 0x5564ed9bb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.486+0000 7fdd82ffd640 10 client.18459 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:45:32.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.486+0000 7fdda2ffd640 1 --2- 172.21.15.43:0/596324058 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5564ed9b8dd0 0x5564ed9bb1f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3ef3e485807122f6 server_cookie=fb2cd69c8ccbf284 in_seq=0 out_seq=0 2024-04-21T08:45:32.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.486+0000 7fdd82ffd640 1 -- 172.21.15.43:0/596324058 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5564ed941ea0 con 0x5564ed9b8dd0 2024-04-21T08:45:32.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.486+0000 7fdd82ffd640 10 client.18459 handle_command_reply: tid=0 2024-04-21T08:45:32.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.486+0000 7fddacc13740 1 client.18459 shutdown 2024-04-21T08:45:32.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.487+0000 7fddacc13740 1 -- 172.21.15.43:0/596324058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed934330 msgr2=0x5564ed94a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.487+0000 7fddacc13740 1 --2- 172.21.15.43:0/596324058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed934330 0x5564ed94a4f0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fdd9409c020 tx=0x7fdd9409c3e0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.487+0000 7fddacc13740 1 -- 172.21.15.43:0/596324058 shutdown_connections 2024-04-21T08:45:32.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.487+0000 7fddacc13740 1 --2- 172.21.15.43:0/596324058 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5564ed9b8dd0 0x5564ed9bb1f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.487+0000 7fddacc13740 1 --2- 172.21.15.43:0/596324058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564ed93de40 0x5564ed9434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.487+0000 7fdd82ffd640 0 client.18459 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:45:32.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.487+0000 7fddacc13740 1 --2- 172.21.15.43:0/596324058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564ed934ce0 0x5564ed942f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.487+0000 7fddacc13740 1 --2- 172.21.15.43:0/596324058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ed934330 0x5564ed94a4f0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.487+0000 7fddacc13740 1 -- 172.21.15.43:0/596324058 >> 172.21.15.43:0/596324058 conn(0x5564ed92eba0 msgr2=0x5564ed92dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.487+0000 7fddacc13740 1 -- 172.21.15.43:0/596324058 shutdown_connections 2024-04-21T08:45:32.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.487+0000 7fddacc13740 1 -- 172.21.15.43:0/596324058 wait complete. 2024-04-21T08:45:32.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.487+0000 7fddacc13740 20 client.18459 trim_cache size 0 max 16384 2024-04-21T08:45:32.507 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:45:32.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.487+0000 7fddaa025640 1 -- 172.21.15.43:0/1711092971 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdd8c0a3d20 msgr2=0x7fdd8c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.487+0000 7fddaa025640 1 --2- 172.21.15.43:0/1711092971 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdd8c0a3d20 0x7fdd8c0a61e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c005fd0 tx=0x7fdd9c014af0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.487+0000 7fddaa025640 1 -- 172.21.15.43:0/1711092971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdda4100af0 msgr2=0x7fdda419e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.487+0000 7fddaa025640 1 --2- 172.21.15.43:0/1711092971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdda4100af0 0x7fdda419e410 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400d770 tx=0x7fdd9400dc30 comp rx=0 tx=0).stop 2024-04-21T08:45:32.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.487+0000 7fddaa025640 1 -- 172.21.15.43:0/1711092971 shutdown_connections 2024-04-21T08:45:32.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.487+0000 7fddaa025640 1 --2- 172.21.15.43:0/1711092971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdda4198610 0x7fdda4198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.487+0000 7fddaa025640 1 --2- 172.21.15.43:0/1711092971 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fdd8c0a3d20 0x7fdd8c0a61e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.487+0000 7fddaa025640 1 --2- 172.21.15.43:0/1711092971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdda41072b0 0x7fdda419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.487+0000 7fddaa025640 1 --2- 172.21.15.43:0/1711092971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdda4100af0 0x7fdda419e410 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.487+0000 7fddaa025640 1 -- 172.21.15.43:0/1711092971 >> 172.21.15.43:0/1711092971 conn(0x7fdda40760e0 msgr2=0x7fdda40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.488+0000 7fddaa025640 1 -- 172.21.15.43:0/1711092971 shutdown_connections 2024-04-21T08:45:32.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.488+0000 7fddaa025640 1 -- 172.21.15.43:0/1711092971 wait complete. 2024-04-21T08:45:32.511 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:32 smithi043 ceph-mon[27978]: pgmap v201: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-21T08:45:32.512 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:45:32.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:32 smithi112 ceph-mon[31085]: pgmap v201: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-21T08:45:32.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.618+0000 7f609dc31640 1 -- 172.21.15.43:0/2056963149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6098101a40 msgr2=0x7f6098105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.618+0000 7f609dc31640 1 --2- 172.21.15.43:0/2056963149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6098101a40 0x7f6098105a80 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f608c009900 tx=0x7f608c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:32.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.618+0000 7f609dc31640 1 -- 172.21.15.43:0/2056963149 shutdown_connections 2024-04-21T08:45:32.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.618+0000 7f609dc31640 1 --2- 172.21.15.43:0/2056963149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6098106160 0x7f609810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.618+0000 7f609dc31640 1 --2- 172.21.15.43:0/2056963149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6098101a40 0x7f6098105a80 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.618+0000 7f609dc31640 1 --2- 172.21.15.43:0/2056963149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6098101090 0x7f6098101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.618+0000 7f609dc31640 1 -- 172.21.15.43:0/2056963149 >> 172.21.15.43:0/2056963149 conn(0x7f6098077f50 msgr2=0x7f6098078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.618+0000 7f609dc31640 1 -- 172.21.15.43:0/2056963149 shutdown_connections 2024-04-21T08:45:32.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.618+0000 7f609dc31640 1 -- 172.21.15.43:0/2056963149 wait complete. 2024-04-21T08:45:32.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.618+0000 7f609dc31640 1 Processor -- start 2024-04-21T08:45:32.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.619+0000 7f609dc31640 1 -- start start 2024-04-21T08:45:32.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.619+0000 7f609dc31640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6098101090 0x7f6098199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.619+0000 7f609dc31640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6098106160 0x7f6098199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.619+0000 7f609dc31640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f609819a070 0x7f60981a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.619+0000 7f609dc31640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f60981110f0 con 0x7f609819a070 2024-04-21T08:45:32.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.619+0000 7f609dc31640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6098110f70 con 0x7f6098101090 2024-04-21T08:45:32.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.619+0000 7f609dc31640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6098111270 con 0x7f6098106160 2024-04-21T08:45:32.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.619+0000 7f6097fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f609819a070 0x7f60981a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.619+0000 7f6097fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f609819a070 0x7f60981a45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38906/0 (socket says 172.21.15.43:38906) 2024-04-21T08:45:32.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.619+0000 7f60977fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6098101090 0x7f6098199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.619+0000 7f6097fff640 1 -- 172.21.15.43:0/3552405869 learned_addr learned my addr 172.21.15.43:0/3552405869 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.619+0000 7f6096ffd640 1 --2- 172.21.15.43:0/3552405869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6098106160 0x7f6098199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.619+0000 7f6097fff640 1 -- 172.21.15.43:0/3552405869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6098106160 msgr2=0x7f6098199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.619+0000 7f6097fff640 1 --2- 172.21.15.43:0/3552405869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6098106160 0x7f6098199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.619+0000 7f6097fff640 1 -- 172.21.15.43:0/3552405869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6098101090 msgr2=0x7f6098199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.619+0000 7f6097fff640 1 --2- 172.21.15.43:0/3552405869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6098101090 0x7f6098199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.619+0000 7f6097fff640 1 -- 172.21.15.43:0/3552405869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60981a4df0 con 0x7f609819a070 2024-04-21T08:45:32.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.619+0000 7f6097fff640 1 --2- 172.21.15.43:0/3552405869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f609819a070 0x7f60981a45d0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f609000eb10 tx=0x7f609000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.620+0000 7f6094ff9640 1 -- 172.21.15.43:0/3552405869 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6090004b20 con 0x7f609819a070 2024-04-21T08:45:32.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.620+0000 7f6094ff9640 1 -- 172.21.15.43:0/3552405869 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6090004cc0 con 0x7f609819a070 2024-04-21T08:45:32.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.620+0000 7f609dc31640 1 -- 172.21.15.43:0/3552405869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60981a50e0 con 0x7f609819a070 2024-04-21T08:45:32.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.620+0000 7f6094ff9640 1 -- 172.21.15.43:0/3552405869 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6090010670 con 0x7f609819a070 2024-04-21T08:45:32.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.620+0000 7f609dc31640 1 -- 172.21.15.43:0/3552405869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60981a5600 con 0x7f609819a070 2024-04-21T08:45:32.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.622+0000 7f6094ff9640 1 -- 172.21.15.43:0/3552405869 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6090005020 con 0x7f609819a070 2024-04-21T08:45:32.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.622+0000 7f60a081d740 1 Processor -- start 2024-04-21T08:45:32.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.622+0000 7f60a081d740 1 -- start start 2024-04-21T08:45:32.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.622+0000 7f60a081d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d0d2a43b0 0x561d0d2a4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.622+0000 7f60a081d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d0d2a4d60 0x561d0d2ad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.622+0000 7f60a081d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2ade60 0x561d0d2ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.622+0000 7f60a081d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561d0d2091f0 con 0x561d0d2ade60 2024-04-21T08:45:32.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.622+0000 7f60a081d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561d0d1a64b0 con 0x561d0d2a43b0 2024-04-21T08:45:32.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.622+0000 7f60a081d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561d0d2051b0 con 0x561d0d2a4d60 2024-04-21T08:45:32.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.622+0000 7f6096ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2ade60 0x561d0d2ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.622+0000 7f6096ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2ade60 0x561d0d2ae2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38908/0 (socket says 172.21.15.43:38908) 2024-04-21T08:45:32.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.622+0000 7f6096ffd640 1 -- 172.21.15.43:0/3848927795 learned_addr learned my addr 172.21.15.43:0/3848927795 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.622+0000 7f6096ffd640 1 --2- 172.21.15.43:0/3848927795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d0d2a43b0 0x561d0d2a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.623+0000 7f60977fe640 1 --2- 172.21.15.43:0/3848927795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d0d2a4d60 0x561d0d2ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.623+0000 7f6094ff9640 1 --2- 172.21.15.43:0/3552405869 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f60800a3d20 0x7f60800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.623+0000 7f6096ffd640 1 -- 172.21.15.43:0/3848927795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d0d2a4d60 msgr2=0x561d0d2ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.623+0000 7f6096ffd640 1 --2- 172.21.15.43:0/3848927795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d0d2a4d60 0x561d0d2ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.623+0000 7f6096ffd640 1 -- 172.21.15.43:0/3848927795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d0d2a43b0 msgr2=0x561d0d2a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.623+0000 7f6096ffd640 1 --2- 172.21.15.43:0/3848927795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d0d2a43b0 0x561d0d2a4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.623+0000 7f6096ffd640 1 -- 172.21.15.43:0/3848927795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d0d1981b0 con 0x561d0d2ade60 2024-04-21T08:45:32.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.623+0000 7f6097fff640 1 --2- 172.21.15.43:0/3552405869 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f60800a3d20 0x7f60800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.623+0000 7f6096ffd640 1 --2- 172.21.15.43:0/3848927795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2ade60 0x561d0d2ae2c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f608c009900 tx=0x7f608c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1f4dfd05f86400 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.623+0000 7f6094ff9640 1 -- 172.21.15.43:0/3552405869 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f60900d1160 con 0x7f609819a070 2024-04-21T08:45:32.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.623+0000 7f607dffb640 1 -- 172.21.15.43:0/3848927795 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c049070 con 0x561d0d2ade60 2024-04-21T08:45:32.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.623+0000 7f607dffb640 1 -- 172.21.15.43:0/3848927795 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f608c02cc90 con 0x561d0d2ade60 2024-04-21T08:45:32.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.623+0000 7f607dffb640 1 -- 172.21.15.43:0/3848927795 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c043450 con 0x561d0d2ade60 2024-04-21T08:45:32.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.624+0000 7f60a081d740 1 -- 172.21.15.43:0/3848927795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2ade60 msgr2=0x561d0d2ae2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.624+0000 7f60a081d740 1 --2- 172.21.15.43:0/3848927795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2ade60 0x561d0d2ae2c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f608c009900 tx=0x7f608c005fd0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.624+0000 7f6097fff640 1 --2- 172.21.15.43:0/3552405869 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f60800a3d20 0x7f60800a61e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6090086270 tx=0x7f609009cb00 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.624+0000 7f60a081d740 1 -- 172.21.15.43:0/3848927795 shutdown_connections 2024-04-21T08:45:32.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.624+0000 7f60a081d740 1 --2- 172.21.15.43:0/3848927795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2ade60 0x561d0d2ae2c0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.624+0000 7f60a081d740 1 --2- 172.21.15.43:0/3848927795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d0d2a4d60 0x561d0d2ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.624+0000 7f60a081d740 1 --2- 172.21.15.43:0/3848927795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d0d2a43b0 0x561d0d2a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.624+0000 7f60a081d740 1 -- 172.21.15.43:0/3848927795 >> 172.21.15.43:0/3848927795 conn(0x561d0d29eba0 msgr2=0x561d0d29ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.624+0000 7f60a081d740 1 -- 172.21.15.43:0/3848927795 shutdown_connections 2024-04-21T08:45:32.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.624+0000 7f60a081d740 1 -- 172.21.15.43:0/3848927795 wait complete. 2024-04-21T08:45:32.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.624+0000 7f60a081d740 1 -- start start 2024-04-21T08:45:32.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.625+0000 7f60a081d740 1 Processor -- start 2024-04-21T08:45:32.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.625+0000 7f60a081d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2a43b0 0x561d0d2b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.625+0000 7f60a081d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d0d2a4d60 0x561d0d2b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.625+0000 7f60a081d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d0d2ade60 0x561d0d2b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.625+0000 7f60a081d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561d0d2b4d40 con 0x561d0d2a43b0 2024-04-21T08:45:32.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.625+0000 7f60a081d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561d0d1c2500 con 0x561d0d2a4d60 2024-04-21T08:45:32.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.625+0000 7f60a081d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561d0d2b4ec0 con 0x561d0d2ade60 2024-04-21T08:45:32.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.625+0000 7f60a081d740 10 client.0 _ll_register_callbacks cb 0x561d0d190b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:32.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.625+0000 7f6096ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2a43b0 0x561d0d2b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.625+0000 7f6096ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2a43b0 0x561d0d2b42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38924/0 (socket says 172.21.15.43:38924) 2024-04-21T08:45:32.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.625+0000 7f6096ffd640 1 -- 172.21.15.43:0/3487454425 learned_addr learned my addr 172.21.15.43:0/3487454425 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.625+0000 7f60977fe640 1 --2- 172.21.15.43:0/3487454425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d0d2a4d60 0x561d0d2b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.625+0000 7f6096ffd640 1 --2- 172.21.15.43:0/3487454425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d0d2ade60 0x561d0d2b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.625+0000 7f6096ffd640 1 -- 172.21.15.43:0/3487454425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d0d2ade60 msgr2=0x561d0d2b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.625+0000 7f6096ffd640 1 --2- 172.21.15.43:0/3487454425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d0d2ade60 0x561d0d2b1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.625+0000 7f6096ffd640 1 -- 172.21.15.43:0/3487454425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d0d2a4d60 msgr2=0x561d0d2b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.625+0000 7f6096ffd640 1 --2- 172.21.15.43:0/3487454425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d0d2a4d60 0x561d0d2b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.625+0000 7f6096ffd640 1 -- 172.21.15.43:0/3487454425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d0cf1b3f0 con 0x561d0d2a43b0 2024-04-21T08:45:32.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.625+0000 7f6096ffd640 1 --2- 172.21.15.43:0/3487454425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2a43b0 0x561d0d2b42b0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f608c009430 tx=0x7f608c003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a06cf60024d0d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.625+0000 7f6076ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:32.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.626+0000 7f6076ffd640 1 -- 172.21.15.43:0/3487454425 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c049070 con 0x561d0d2a43b0 2024-04-21T08:45:32.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.626+0000 7f6076ffd640 1 -- 172.21.15.43:0/3487454425 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f608c02c9e0 con 0x561d0d2a43b0 2024-04-21T08:45:32.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.626+0000 7f60a081d740 1 -- 172.21.15.43:0/3487454425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f608c02cc90 con 0x561d0d2a43b0 2024-04-21T08:45:32.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.626+0000 7f6076ffd640 1 -- 172.21.15.43:0/3487454425 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f608c005020 con 0x561d0d2a43b0 2024-04-21T08:45:32.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.626+0000 7f60a081d740 10 client.18471 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:32.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.626+0000 7f60a081d740 1 -- 172.21.15.43:0/3487454425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561d0cfe57d0 con 0x561d0d2a43b0 2024-04-21T08:45:32.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.626+0000 7f6076ffd640 1 -- 172.21.15.43:0/3487454425 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f608c02cc90 con 0x561d0d2a43b0 2024-04-21T08:45:32.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.626+0000 7f6076ffd640 1 -- 172.21.15.43:0/3487454425 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f608c040040 con 0x561d0d2a43b0 2024-04-21T08:45:32.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.626+0000 7f60a081d740 10 client.18471 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:32.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.626+0000 7f60a081d740 10 client.18471 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:45:32.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.626+0000 7f60a081d740 20 client.18471 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:32.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.626+0000 7f60a081d740 1 --2- 172.21.15.43:0/3487454425 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x561d0d2c1e70 0x561d0d2c4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.626+0000 7f60a081d740 4 client.18471 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:32.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.626+0000 7f60a081d740 1 -- 172.21.15.43:0/3487454425 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d0d1a8930 con 0x561d0d2c1e70 2024-04-21T08:45:32.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.627+0000 7f60977fe640 1 --2- 172.21.15.43:0/3487454425 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x561d0d2c1e70 0x561d0d2c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.627+0000 7f6076ffd640 10 client.18471 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:45:32.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.627+0000 7f60977fe640 1 --2- 172.21.15.43:0/3487454425 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x561d0d2c1e70 0x561d0d2c4260 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6b317d2ecf5eb123 server_cookie=f9123f8fd5d0e13f in_seq=0 out_seq=0 2024-04-21T08:45:32.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.628+0000 7f6076ffd640 1 -- 172.21.15.43:0/3487454425 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561d0d1a8930 con 0x561d0d2c1e70 2024-04-21T08:45:32.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.628+0000 7f6076ffd640 10 client.18471 handle_command_reply: tid=0 2024-04-21T08:45:32.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.628+0000 7f60a081d740 1 client.18471 shutdown 2024-04-21T08:45:32.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.628+0000 7f60a081d740 1 -- 172.21.15.43:0/3487454425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2a43b0 msgr2=0x561d0d2b42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.628+0000 7f60a081d740 1 --2- 172.21.15.43:0/3487454425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2a43b0 0x561d0d2b42b0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f608c009430 tx=0x7f608c003b20 comp rx=0 tx=0).stop 2024-04-21T08:45:32.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.628+0000 7f60a081d740 1 -- 172.21.15.43:0/3487454425 shutdown_connections 2024-04-21T08:45:32.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.628+0000 7f60a081d740 1 --2- 172.21.15.43:0/3487454425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d0d2ade60 0x561d0d2b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.628+0000 7f60a081d740 1 --2- 172.21.15.43:0/3487454425 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x561d0d2c1e70 0x561d0d2c4260 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.628+0000 7f6076ffd640 0 client.18471 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:45:32.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.628+0000 7f60a081d740 1 --2- 172.21.15.43:0/3487454425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d0d2a4d60 0x561d0d2b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.628+0000 7f60a081d740 1 --2- 172.21.15.43:0/3487454425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2a43b0 0x561d0d2b42b0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.628+0000 7f60a081d740 1 -- 172.21.15.43:0/3487454425 >> 172.21.15.43:0/3487454425 conn(0x561d0d29eba0 msgr2=0x561d0d2a08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.628+0000 7f60a081d740 1 -- 172.21.15.43:0/3487454425 shutdown_connections 2024-04-21T08:45:32.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.628+0000 7f60a081d740 1 -- 172.21.15.43:0/3487454425 wait complete. 2024-04-21T08:45:32.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.628+0000 7f60a081d740 20 client.18471 trim_cache size 0 max 16384 2024-04-21T08:45:32.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.641+0000 7f60a081d740 1 Processor -- start 2024-04-21T08:45:32.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.641+0000 7f60a081d740 1 -- start start 2024-04-21T08:45:32.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.641+0000 7f60a081d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d0d2a43b0 0x561d0d2af270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.641+0000 7f60a081d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d0d2a4d60 0x561d0d2b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.641+0000 7f60a081d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2ade60 0x561d0d2b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.642+0000 7f60a081d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f60981ad1c0 con 0x561d0d2ade60 2024-04-21T08:45:32.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.642+0000 7f60a081d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f60981aed00 con 0x561d0d2a43b0 2024-04-21T08:45:32.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.642+0000 7f60a081d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f60981ad890 con 0x561d0d2a4d60 2024-04-21T08:45:32.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.642+0000 7f6096ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2ade60 0x561d0d2b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.642+0000 7f6096ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2ade60 0x561d0d2b42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38932/0 (socket says 172.21.15.43:38932) 2024-04-21T08:45:32.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.642+0000 7f6096ffd640 1 -- 172.21.15.43:0/757920395 learned_addr learned my addr 172.21.15.43:0/757920395 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.642+0000 7f6096ffd640 1 --2- 172.21.15.43:0/757920395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d0d2a43b0 0x561d0d2af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.642+0000 7f60977fe640 1 --2- 172.21.15.43:0/757920395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d0d2a4d60 0x561d0d2b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.642+0000 7f6096ffd640 1 -- 172.21.15.43:0/757920395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d0d2a4d60 msgr2=0x561d0d2b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.642+0000 7f6096ffd640 1 --2- 172.21.15.43:0/757920395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d0d2a4d60 0x561d0d2b3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.642+0000 7f6096ffd640 1 -- 172.21.15.43:0/757920395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d0d2a43b0 msgr2=0x561d0d2af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.642+0000 7f6096ffd640 1 --2- 172.21.15.43:0/757920395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d0d2a43b0 0x561d0d2af270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.642+0000 7f6096ffd640 1 -- 172.21.15.43:0/757920395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d0d2b1f10 con 0x561d0d2ade60 2024-04-21T08:45:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.642+0000 7f6096ffd640 1 --2- 172.21.15.43:0/757920395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2ade60 0x561d0d2b42b0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f608c004d90 tx=0x7f608c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3aba1670337c662 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.642+0000 7f607d7fa640 1 -- 172.21.15.43:0/757920395 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c049070 con 0x561d0d2ade60 2024-04-21T08:45:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.642+0000 7f607d7fa640 1 -- 172.21.15.43:0/757920395 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f608c02cc90 con 0x561d0d2ade60 2024-04-21T08:45:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.642+0000 7f607d7fa640 1 -- 172.21.15.43:0/757920395 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c02c7e0 con 0x561d0d2ade60 2024-04-21T08:45:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.643+0000 7f60a081d740 1 -- 172.21.15.43:0/757920395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2ade60 msgr2=0x561d0d2b42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.643+0000 7f60a081d740 1 --2- 172.21.15.43:0/757920395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2ade60 0x561d0d2b42b0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f608c004d90 tx=0x7f608c005fd0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.643+0000 7f60a081d740 1 -- 172.21.15.43:0/757920395 shutdown_connections 2024-04-21T08:45:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.643+0000 7f60a081d740 1 --2- 172.21.15.43:0/757920395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2ade60 0x561d0d2b42b0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.643+0000 7f60a081d740 1 --2- 172.21.15.43:0/757920395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d0d2a4d60 0x561d0d2b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.643+0000 7f60a081d740 1 --2- 172.21.15.43:0/757920395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d0d2a43b0 0x561d0d2af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.643+0000 7f60a081d740 1 -- 172.21.15.43:0/757920395 >> 172.21.15.43:0/757920395 conn(0x561d0d29eba0 msgr2=0x561d0d2bd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.643+0000 7f60a081d740 1 -- 172.21.15.43:0/757920395 shutdown_connections 2024-04-21T08:45:32.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.643+0000 7f60a081d740 1 -- 172.21.15.43:0/757920395 wait complete. 2024-04-21T08:45:32.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.643+0000 7f60a081d740 1 -- start start 2024-04-21T08:45:32.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.643+0000 7f60a081d740 1 Processor -- start 2024-04-21T08:45:32.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.644+0000 7f60a081d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d0d2a43b0 0x561d0d2c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.644+0000 7f60a081d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d0d2a4d60 0x561d0d2c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.644+0000 7f60a081d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2ade60 0x561d0d2c5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.644+0000 7f60a081d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f60981ae630 con 0x561d0d2ade60 2024-04-21T08:45:32.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.644+0000 7f60a081d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f60981adf60 con 0x561d0d2a4d60 2024-04-21T08:45:32.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.644+0000 7f60a081d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f60981ab1a0 con 0x561d0d2a43b0 2024-04-21T08:45:32.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.644+0000 7f60a081d740 10 client.0 _ll_register_callbacks cb 0x7f608c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:32.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.644+0000 7f6096ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2ade60 0x561d0d2c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.644+0000 7f6096ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2ade60 0x561d0d2c5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38934/0 (socket says 172.21.15.43:38934) 2024-04-21T08:45:32.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.644+0000 7f6096ffd640 1 -- 172.21.15.43:0/3760139089 learned_addr learned my addr 172.21.15.43:0/3760139089 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.644+0000 7f6096ffd640 1 --2- 172.21.15.43:0/3760139089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d0d2a43b0 0x561d0d2c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.644+0000 7f60977fe640 1 --2- 172.21.15.43:0/3760139089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d0d2a4d60 0x561d0d2c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.644+0000 7f6096ffd640 1 -- 172.21.15.43:0/3760139089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d0d2a43b0 msgr2=0x561d0d2c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.644+0000 7f6096ffd640 1 --2- 172.21.15.43:0/3760139089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d0d2a43b0 0x561d0d2c6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.644+0000 7f6096ffd640 1 -- 172.21.15.43:0/3760139089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d0d2a4d60 msgr2=0x561d0d2c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.644+0000 7f6096ffd640 1 --2- 172.21.15.43:0/3760139089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d0d2a4d60 0x561d0d2c74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.644+0000 7f6096ffd640 1 -- 172.21.15.43:0/3760139089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d0d2b0610 con 0x561d0d2ade60 2024-04-21T08:45:32.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.644+0000 7f60977fe640 1 --2- 172.21.15.43:0/3760139089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d0d2a4d60 0x561d0d2c74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:32.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.644+0000 7f6096ffd640 1 --2- 172.21.15.43:0/3760139089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2ade60 0x561d0d2c5400 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f608c005a80 tx=0x7f608c02c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b44419f8f54712 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.644+0000 7f6076ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:32.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.644+0000 7f6076ffd640 1 -- 172.21.15.43:0/3760139089 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c049030 con 0x561d0d2ade60 2024-04-21T08:45:32.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.645+0000 7f6076ffd640 1 -- 172.21.15.43:0/3760139089 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f608c0053c0 con 0x561d0d2ade60 2024-04-21T08:45:32.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.645+0000 7f60a081d740 1 -- 172.21.15.43:0/3760139089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f608c02cc90 con 0x561d0d2ade60 2024-04-21T08:45:32.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.645+0000 7f6076ffd640 1 -- 172.21.15.43:0/3760139089 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f608c005d50 con 0x561d0d2ade60 2024-04-21T08:45:32.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.645+0000 7f60a081d740 10 client.18477 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:32.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.645+0000 7f60a081d740 1 -- 172.21.15.43:0/3760139089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561d0d3288a0 con 0x561d0d2ade60 2024-04-21T08:45:32.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.645+0000 7f6076ffd640 1 -- 172.21.15.43:0/3760139089 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f608c02cc90 con 0x561d0d2ade60 2024-04-21T08:45:32.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.645+0000 7f6076ffd640 1 -- 172.21.15.43:0/3760139089 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f608c034cc0 con 0x561d0d2ade60 2024-04-21T08:45:32.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.645+0000 7f60a081d740 10 client.18477 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:32.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.645+0000 7f60a081d740 10 client.18477 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:45:32.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.645+0000 7f60a081d740 20 client.18477 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:32.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.645+0000 7f60a081d740 1 --2- 172.21.15.43:0/3760139089 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x561d0d2c1e70 0x561d0d32af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.645+0000 7f60a081d740 4 client.18477 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:45:32.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.645+0000 7f60a081d740 1 -- 172.21.15.43:0/3760139089 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561d0d216c10 con 0x561d0d2c1e70 2024-04-21T08:45:32.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.645+0000 7f60977fe640 1 --2- 172.21.15.43:0/3760139089 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x561d0d2c1e70 0x561d0d32af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.646+0000 7f60977fe640 1 --2- 172.21.15.43:0/3760139089 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x561d0d2c1e70 0x561d0d32af00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c15f9eb3d3fda2a server_cookie=d14dbec3491a9632 in_seq=0 out_seq=0 2024-04-21T08:45:32.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.646+0000 7f6076ffd640 10 client.18477 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:45:32.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.646+0000 7f6076ffd640 1 -- 172.21.15.43:0/3760139089 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561d0d216c10 con 0x561d0d2c1e70 2024-04-21T08:45:32.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.646+0000 7f6076ffd640 10 client.18477 handle_command_reply: tid=0 2024-04-21T08:45:32.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.646+0000 7f60a081d740 1 client.18477 shutdown 2024-04-21T08:45:32.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.646+0000 7f60a081d740 1 -- 172.21.15.43:0/3760139089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2ade60 msgr2=0x561d0d2c5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.646+0000 7f60a081d740 1 --2- 172.21.15.43:0/3760139089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2ade60 0x561d0d2c5400 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f608c005a80 tx=0x7f608c02c330 comp rx=0 tx=0).stop 2024-04-21T08:45:32.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.646+0000 7f60a081d740 1 -- 172.21.15.43:0/3760139089 shutdown_connections 2024-04-21T08:45:32.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.646+0000 7f60a081d740 1 --2- 172.21.15.43:0/3760139089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561d0d2ade60 0x561d0d2c5400 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.646+0000 7f60a081d740 1 --2- 172.21.15.43:0/3760139089 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x561d0d2c1e70 0x561d0d32af00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.646+0000 7f60a081d740 1 --2- 172.21.15.43:0/3760139089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561d0d2a4d60 0x561d0d2c74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.646+0000 7f60a081d740 1 --2- 172.21.15.43:0/3760139089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d0d2a43b0 0x561d0d2c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.646+0000 7f6076ffd640 0 client.18477 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:45:32.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.646+0000 7f60a081d740 1 -- 172.21.15.43:0/3760139089 >> 172.21.15.43:0/3760139089 conn(0x561d0d29eba0 msgr2=0x561d0d29ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.646+0000 7f60a081d740 1 -- 172.21.15.43:0/3760139089 shutdown_connections 2024-04-21T08:45:32.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.646+0000 7f60a081d740 1 -- 172.21.15.43:0/3760139089 wait complete. 2024-04-21T08:45:32.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.646+0000 7f60a081d740 20 client.18477 trim_cache size 0 max 16384 2024-04-21T08:45:32.654 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:45:32.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.647+0000 7f609dc31640 1 -- 172.21.15.43:0/3552405869 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f60800a3d20 msgr2=0x7f60800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.647+0000 7f609dc31640 1 --2- 172.21.15.43:0/3552405869 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f60800a3d20 0x7f60800a61e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6090086270 tx=0x7f609009cb00 comp rx=0 tx=0).stop 2024-04-21T08:45:32.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.647+0000 7f609dc31640 1 -- 172.21.15.43:0/3552405869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f609819a070 msgr2=0x7f60981a45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.647+0000 7f609dc31640 1 --2- 172.21.15.43:0/3552405869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f609819a070 0x7f60981a45d0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f609000eb10 tx=0x7f609000c2d0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.647+0000 7f609dc31640 1 -- 172.21.15.43:0/3552405869 shutdown_connections 2024-04-21T08:45:32.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.647+0000 7f609dc31640 1 --2- 172.21.15.43:0/3552405869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f609819a070 0x7f60981a45d0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.647+0000 7f609dc31640 1 --2- 172.21.15.43:0/3552405869 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f60800a3d20 0x7f60800a61e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.647+0000 7f609dc31640 1 --2- 172.21.15.43:0/3552405869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6098106160 0x7f6098199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.647+0000 7f609dc31640 1 --2- 172.21.15.43:0/3552405869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6098101090 0x7f6098199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.647+0000 7f609dc31640 1 -- 172.21.15.43:0/3552405869 >> 172.21.15.43:0/3552405869 conn(0x7f6098077f50 msgr2=0x7f60980ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.647+0000 7f609dc31640 1 -- 172.21.15.43:0/3552405869 shutdown_connections 2024-04-21T08:45:32.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.647+0000 7f609dc31640 1 -- 172.21.15.43:0/3552405869 wait complete. 2024-04-21T08:45:32.657 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:45:32.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.762+0000 7fd942984640 1 -- 172.21.15.43:0/2340764083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd93c107270 msgr2=0x7fd93c10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.762+0000 7fd942984640 1 --2- 172.21.15.43:0/2340764083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd93c107270 0x7fd93c10ee40 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fd934009ca0 tx=0x7fd93402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.762+0000 7fd942984640 1 -- 172.21.15.43:0/2340764083 shutdown_connections 2024-04-21T08:45:32.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.762+0000 7fd942984640 1 --2- 172.21.15.43:0/2340764083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd93c107270 0x7fd93c10ee40 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.762+0000 7fd942984640 1 --2- 172.21.15.43:0/2340764083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd93c06b1f0 0x7fd93c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.762+0000 7fd942984640 1 --2- 172.21.15.43:0/2340764083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd93c06a840 0x7fd93c06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.762+0000 7fd942984640 1 -- 172.21.15.43:0/2340764083 >> 172.21.15.43:0/2340764083 conn(0x7fd93c077eb0 msgr2=0x7fd93c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.762+0000 7fd942984640 1 -- 172.21.15.43:0/2340764083 shutdown_connections 2024-04-21T08:45:32.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.762+0000 7fd942984640 1 -- 172.21.15.43:0/2340764083 wait complete. 2024-04-21T08:45:32.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.763+0000 7fd942984640 1 Processor -- start 2024-04-21T08:45:32.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.763+0000 7fd942984640 1 -- start start 2024-04-21T08:45:32.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.763+0000 7fd942984640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd93c06a840 0x7fd93c100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.763+0000 7fd942984640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd93c107270 0x7fd93c1013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.763+0000 7fd942984640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd93c101900 0x7fd93c1a6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.763+0000 7fd942984640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c1121d0 con 0x7fd93c107270 2024-04-21T08:45:32.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.763+0000 7fd942984640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c112050 con 0x7fd93c101900 2024-04-21T08:45:32.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.763+0000 7fd942984640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c112350 con 0x7fd93c06a840 2024-04-21T08:45:32.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.763+0000 7fd93b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd93c107270 0x7fd93c1013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.763+0000 7fd93b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd93c107270 0x7fd93c1013c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38958/0 (socket says 172.21.15.43:38958) 2024-04-21T08:45:32.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.763+0000 7fd93b7fe640 1 -- 172.21.15.43:0/1158142211 learned_addr learned my addr 172.21.15.43:0/1158142211 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.763+0000 7fd93bfff640 1 --2- 172.21.15.43:0/1158142211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd93c06a840 0x7fd93c100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.764+0000 7fd940f21640 1 --2- 172.21.15.43:0/1158142211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd93c101900 0x7fd93c1a6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.764+0000 7fd93b7fe640 1 -- 172.21.15.43:0/1158142211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd93c06a840 msgr2=0x7fd93c100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.764+0000 7fd93b7fe640 1 --2- 172.21.15.43:0/1158142211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd93c06a840 0x7fd93c100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.764+0000 7fd93b7fe640 1 -- 172.21.15.43:0/1158142211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd93c101900 msgr2=0x7fd93c1a6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.764+0000 7fd93b7fe640 1 --2- 172.21.15.43:0/1158142211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd93c101900 0x7fd93c1a6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.764+0000 7fd93b7fe640 1 -- 172.21.15.43:0/1158142211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93c1a6f00 con 0x7fd93c107270 2024-04-21T08:45:32.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.764+0000 7fd93b7fe640 1 --2- 172.21.15.43:0/1158142211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd93c107270 0x7fd93c1013c0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fd92c00d740 tx=0x7fd92c00db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.764+0000 7fd9397fa640 1 -- 172.21.15.43:0/1158142211 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92c017070 con 0x7fd93c107270 2024-04-21T08:45:32.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.764+0000 7fd9397fa640 1 -- 172.21.15.43:0/1158142211 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd92c009820 con 0x7fd93c107270 2024-04-21T08:45:32.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.764+0000 7fd942984640 1 -- 172.21.15.43:0/1158142211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd93c1a71f0 con 0x7fd93c107270 2024-04-21T08:45:32.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.764+0000 7fd9397fa640 1 -- 172.21.15.43:0/1158142211 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92c004ce0 con 0x7fd93c107270 2024-04-21T08:45:32.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.764+0000 7fd942984640 1 -- 172.21.15.43:0/1158142211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd93c1a76f0 con 0x7fd93c107270 2024-04-21T08:45:32.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.766+0000 7fd9397fa640 1 -- 172.21.15.43:0/1158142211 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd92c005220 con 0x7fd93c107270 2024-04-21T08:45:32.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.766+0000 7fd94557a740 1 Processor -- start 2024-04-21T08:45:32.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.766+0000 7fd94557a740 1 -- start start 2024-04-21T08:45:32.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.767+0000 7fd94557a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c638d4d0 0x5557c638d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.767+0000 7fd94557a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557c638de80 0x5557c6396430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.767+0000 7fd94557a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c6396a00 0x5557c6398e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.767+0000 7fd94557a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5557c62f21f0 con 0x5557c638d4d0 2024-04-21T08:45:32.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.767+0000 7fd94557a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5557c628f4b0 con 0x5557c6396a00 2024-04-21T08:45:32.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.767+0000 7fd94557a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5557c62ee1b0 con 0x5557c638de80 2024-04-21T08:45:32.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.767+0000 7fd940f21640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c638d4d0 0x5557c638d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.767+0000 7fd940f21640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c638d4d0 0x5557c638d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38972/0 (socket says 172.21.15.43:38972) 2024-04-21T08:45:32.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.767+0000 7fd940f21640 1 -- 172.21.15.43:0/3785568603 learned_addr learned my addr 172.21.15.43:0/3785568603 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.767+0000 7fd9397fa640 1 --2- 172.21.15.43:0/1158142211 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd9280a3d50 0x7fd9280a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.767+0000 7fd93bfff640 1 --2- 172.21.15.43:0/3785568603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557c638de80 0x5557c6396430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.767+0000 7fd940f21640 1 --2- 172.21.15.43:0/1158142211 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd9280a3d50 0x7fd9280a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.767+0000 7fd93b7fe640 1 --2- 172.21.15.43:0/3785568603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c6396a00 0x5557c6398e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.767+0000 7fd940f21640 1 -- 172.21.15.43:0/3785568603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557c638de80 msgr2=0x5557c6396430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.767+0000 7fd940f21640 1 --2- 172.21.15.43:0/3785568603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557c638de80 0x5557c6396430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.767+0000 7fd940f21640 1 -- 172.21.15.43:0/3785568603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c6396a00 msgr2=0x5557c6398e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.767+0000 7fd940f21640 1 --2- 172.21.15.43:0/3785568603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c6396a00 0x5557c6398e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.767+0000 7fd940f21640 1 -- 172.21.15.43:0/3785568603 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557c62811b0 con 0x5557c638d4d0 2024-04-21T08:45:32.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.767+0000 7fd9397fa640 1 -- 172.21.15.43:0/1158142211 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd92c0d0510 con 0x7fd93c107270 2024-04-21T08:45:32.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.767+0000 7fd940f21640 1 --2- 172.21.15.43:0/3785568603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c638d4d0 0x5557c638d8b0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fd934009430 tx=0x7fd934032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5221b485fd201b94 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.767+0000 7fd9327fc640 1 -- 172.21.15.43:0/3785568603 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9340327a0 con 0x5557c638d4d0 2024-04-21T08:45:32.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.768+0000 7fd9327fc640 1 -- 172.21.15.43:0/3785568603 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd934032940 con 0x5557c638d4d0 2024-04-21T08:45:32.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.768+0000 7fd9327fc640 1 -- 172.21.15.43:0/3785568603 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934039070 con 0x5557c638d4d0 2024-04-21T08:45:32.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.768+0000 7fd940f21640 1 --2- 172.21.15.43:0/1158142211 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd9280a3d50 0x7fd9280a6210 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd934049000 tx=0x7fd934048000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.768+0000 7fd94557a740 1 -- 172.21.15.43:0/3785568603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c638d4d0 msgr2=0x5557c638d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.768+0000 7fd94557a740 1 --2- 172.21.15.43:0/3785568603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c638d4d0 0x5557c638d8b0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fd934009430 tx=0x7fd934032eb0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.768+0000 7fd94557a740 1 -- 172.21.15.43:0/3785568603 shutdown_connections 2024-04-21T08:45:32.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.768+0000 7fd94557a740 1 --2- 172.21.15.43:0/3785568603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c6396a00 0x5557c6398e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.768+0000 7fd94557a740 1 --2- 172.21.15.43:0/3785568603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557c638de80 0x5557c6396430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.768+0000 7fd94557a740 1 --2- 172.21.15.43:0/3785568603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c638d4d0 0x5557c638d8b0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.768+0000 7fd94557a740 1 -- 172.21.15.43:0/3785568603 >> 172.21.15.43:0/3785568603 conn(0x5557c6387ba0 msgr2=0x5557c6387f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.768+0000 7fd94557a740 1 -- 172.21.15.43:0/3785568603 shutdown_connections 2024-04-21T08:45:32.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.768+0000 7fd94557a740 1 -- 172.21.15.43:0/3785568603 wait complete. 2024-04-21T08:45:32.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.769+0000 7fd94557a740 1 -- start start 2024-04-21T08:45:32.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.769+0000 7fd94557a740 1 Processor -- start 2024-04-21T08:45:32.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.769+0000 7fd94557a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c638d4d0 0x5557c6396330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.769+0000 7fd94557a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c638de80 0x5557c63936c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.769+0000 7fd94557a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557c6396a00 0x5557c6393cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.769+0000 7fd94557a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5557c6399970 con 0x5557c638d4d0 2024-04-21T08:45:32.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.769+0000 7fd94557a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5557c62ab500 con 0x5557c638de80 2024-04-21T08:45:32.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.769+0000 7fd94557a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5557c6399af0 con 0x5557c6396a00 2024-04-21T08:45:32.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.769+0000 7fd94557a740 10 client.0 _ll_register_callbacks cb 0x5557c6279b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:32.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.769+0000 7fd93bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c638d4d0 0x5557c6396330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.769+0000 7fd93bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c638d4d0 0x5557c6396330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38976/0 (socket says 172.21.15.43:38976) 2024-04-21T08:45:32.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.769+0000 7fd93bfff640 1 -- 172.21.15.43:0/3542399444 learned_addr learned my addr 172.21.15.43:0/3542399444 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.769+0000 7fd93b7fe640 1 --2- 172.21.15.43:0/3542399444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c638de80 0x5557c63936c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.769+0000 7fd940f21640 1 --2- 172.21.15.43:0/3542399444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557c6396a00 0x5557c6393cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.770+0000 7fd93bfff640 1 -- 172.21.15.43:0/3542399444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557c6396a00 msgr2=0x5557c6393cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.770+0000 7fd93bfff640 1 --2- 172.21.15.43:0/3542399444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557c6396a00 0x5557c6393cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.770+0000 7fd93bfff640 1 -- 172.21.15.43:0/3542399444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c638de80 msgr2=0x5557c63936c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.770+0000 7fd93bfff640 1 --2- 172.21.15.43:0/3542399444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c638de80 0x5557c63936c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.770+0000 7fd93bfff640 1 -- 172.21.15.43:0/3542399444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557c60043f0 con 0x5557c638d4d0 2024-04-21T08:45:32.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.770+0000 7fd93bfff640 1 --2- 172.21.15.43:0/3542399444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c638d4d0 0x5557c6396330 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fd924010a50 tx=0x7fd924010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345e423f5f265ddf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.770+0000 7fd91b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:32.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.770+0000 7fd91b7fe640 1 -- 172.21.15.43:0/3542399444 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92400fa90 con 0x5557c638d4d0 2024-04-21T08:45:32.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.770+0000 7fd91b7fe640 1 -- 172.21.15.43:0/3542399444 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd92400fc30 con 0x5557c638d4d0 2024-04-21T08:45:32.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.770+0000 7fd94557a740 1 -- 172.21.15.43:0/3542399444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd934032940 con 0x5557c638d4d0 2024-04-21T08:45:32.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.770+0000 7fd91b7fe640 1 -- 172.21.15.43:0/3542399444 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd924024050 con 0x5557c638d4d0 2024-04-21T08:45:32.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.770+0000 7fd94557a740 10 client.18489 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:32.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.770+0000 7fd94557a740 1 -- 172.21.15.43:0/3542399444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5557c60ce7d0 con 0x5557c638d4d0 2024-04-21T08:45:32.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.770+0000 7fd91b7fe640 1 -- 172.21.15.43:0/3542399444 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd934032940 con 0x5557c638d4d0 2024-04-21T08:45:32.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.771+0000 7fd91b7fe640 1 -- 172.21.15.43:0/3542399444 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd92400e3b0 con 0x5557c638d4d0 2024-04-21T08:45:32.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.771+0000 7fd94557a740 10 client.18489 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:32.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.771+0000 7fd94557a740 10 client.18489 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:45:32.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.771+0000 7fd94557a740 20 client.18489 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:32.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.771+0000 7fd94557a740 1 --2- 172.21.15.43:0/3542399444 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5557c63a6d60 0x5557c63a9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.771+0000 7fd94557a740 4 client.18489 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:32.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.771+0000 7fd94557a740 1 -- 172.21.15.43:0/3542399444 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557c6291930 con 0x5557c63a6d60 2024-04-21T08:45:32.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.771+0000 7fd93b7fe640 1 --2- 172.21.15.43:0/3542399444 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5557c63a6d60 0x5557c63a9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.771+0000 7fd91b7fe640 10 client.18489 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:45:32.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.771+0000 7fd93b7fe640 1 --2- 172.21.15.43:0/3542399444 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5557c63a6d60 0x5557c63a9190 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7a2107a93479b904 server_cookie=2618bb422af99174 in_seq=0 out_seq=0 2024-04-21T08:45:32.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.772+0000 7fd91b7fe640 1 -- 172.21.15.43:0/3542399444 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557c6291930 con 0x5557c63a6d60 2024-04-21T08:45:32.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.772+0000 7fd91b7fe640 10 client.18489 handle_command_reply: tid=0 2024-04-21T08:45:32.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.772+0000 7fd94557a740 1 client.18489 shutdown 2024-04-21T08:45:32.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.772+0000 7fd94557a740 1 -- 172.21.15.43:0/3542399444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c638d4d0 msgr2=0x5557c6396330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.772+0000 7fd94557a740 1 --2- 172.21.15.43:0/3542399444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c638d4d0 0x5557c6396330 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fd924010a50 tx=0x7fd924010f10 comp rx=0 tx=0).stop 2024-04-21T08:45:32.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.772+0000 7fd94557a740 1 -- 172.21.15.43:0/3542399444 shutdown_connections 2024-04-21T08:45:32.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.772+0000 7fd94557a740 1 --2- 172.21.15.43:0/3542399444 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5557c63a6d60 0x5557c63a9190 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.772+0000 7fd94557a740 1 --2- 172.21.15.43:0/3542399444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557c6396a00 0x5557c6393cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.772+0000 7fd94557a740 1 --2- 172.21.15.43:0/3542399444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c638de80 0x5557c63936c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.772+0000 7fd94557a740 1 --2- 172.21.15.43:0/3542399444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c638d4d0 0x5557c6396330 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.772+0000 7fd91b7fe640 0 client.18489 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:45:32.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.772+0000 7fd94557a740 1 -- 172.21.15.43:0/3542399444 >> 172.21.15.43:0/3542399444 conn(0x5557c6387ba0 msgr2=0x5557c63898f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.772+0000 7fd94557a740 1 -- 172.21.15.43:0/3542399444 shutdown_connections 2024-04-21T08:45:32.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.772+0000 7fd94557a740 1 -- 172.21.15.43:0/3542399444 wait complete. 2024-04-21T08:45:32.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.773+0000 7fd94557a740 20 client.18489 trim_cache size 0 max 16384 2024-04-21T08:45:32.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.786+0000 7fd94557a740 1 Processor -- start 2024-04-21T08:45:32.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.786+0000 7fd94557a740 1 -- start start 2024-04-21T08:45:32.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.786+0000 7fd94557a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c638d4d0 0x5557c63a2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.786+0000 7fd94557a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557c638de80 0x5557c6395df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.786+0000 7fd94557a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c6396a00 0x5557c6396330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.786+0000 7fd94557a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c1b14b0 con 0x5557c6396a00 2024-04-21T08:45:32.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.786+0000 7fd94557a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c1b2ff0 con 0x5557c638d4d0 2024-04-21T08:45:32.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.786+0000 7fd94557a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c1b1b80 con 0x5557c638de80 2024-04-21T08:45:32.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.786+0000 7fd940f21640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c6396a00 0x5557c6396330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.786+0000 7fd940f21640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c6396a00 0x5557c6396330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38980/0 (socket says 172.21.15.43:38980) 2024-04-21T08:45:32.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.786+0000 7fd940f21640 1 -- 172.21.15.43:0/84012443 learned_addr learned my addr 172.21.15.43:0/84012443 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.786+0000 7fd93b7fe640 1 --2- 172.21.15.43:0/84012443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557c638de80 0x5557c6395df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.786+0000 7fd93bfff640 1 --2- 172.21.15.43:0/84012443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c638d4d0 0x5557c63a2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.786+0000 7fd940f21640 1 -- 172.21.15.43:0/84012443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557c638de80 msgr2=0x5557c6395df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.786+0000 7fd940f21640 1 --2- 172.21.15.43:0/84012443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557c638de80 0x5557c6395df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.786+0000 7fd940f21640 1 -- 172.21.15.43:0/84012443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c638d4d0 msgr2=0x5557c63a2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.786+0000 7fd940f21640 1 --2- 172.21.15.43:0/84012443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c638d4d0 0x5557c63a2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.786+0000 7fd940f21640 1 -- 172.21.15.43:0/84012443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557c63a58b0 con 0x5557c6396a00 2024-04-21T08:45:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.786+0000 7fd93bfff640 1 --2- 172.21.15.43:0/84012443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c638d4d0 0x5557c63a2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.786+0000 7fd93b7fe640 1 --2- 172.21.15.43:0/84012443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557c638de80 0x5557c6395df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.787+0000 7fd940f21640 1 --2- 172.21.15.43:0/84012443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c6396a00 0x5557c6396330 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fd93402c630 tx=0x7fd93402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d9e7307e4c6499 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.787+0000 7fd931ffb640 1 -- 172.21.15.43:0/84012443 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934039040 con 0x5557c6396a00 2024-04-21T08:45:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.787+0000 7fd931ffb640 1 -- 172.21.15.43:0/84012443 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd934034df0 con 0x5557c6396a00 2024-04-21T08:45:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.787+0000 7fd931ffb640 1 -- 172.21.15.43:0/84012443 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9340042a0 con 0x5557c6396a00 2024-04-21T08:45:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.787+0000 7fd94557a740 1 -- 172.21.15.43:0/84012443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c6396a00 msgr2=0x5557c6396330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.787+0000 7fd94557a740 1 --2- 172.21.15.43:0/84012443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c6396a00 0x5557c6396330 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fd93402c630 tx=0x7fd93402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.787+0000 7fd94557a740 1 -- 172.21.15.43:0/84012443 shutdown_connections 2024-04-21T08:45:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.787+0000 7fd94557a740 1 --2- 172.21.15.43:0/84012443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c6396a00 0x5557c6396330 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.787+0000 7fd94557a740 1 --2- 172.21.15.43:0/84012443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557c638de80 0x5557c6395df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.787+0000 7fd94557a740 1 --2- 172.21.15.43:0/84012443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c638d4d0 0x5557c63a2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.787+0000 7fd94557a740 1 -- 172.21.15.43:0/84012443 >> 172.21.15.43:0/84012443 conn(0x5557c6387ba0 msgr2=0x5557c63926f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.787+0000 7fd94557a740 1 -- 172.21.15.43:0/84012443 shutdown_connections 2024-04-21T08:45:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.787+0000 7fd94557a740 1 -- 172.21.15.43:0/84012443 wait complete. 2024-04-21T08:45:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.788+0000 7fd94557a740 1 -- start start 2024-04-21T08:45:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.788+0000 7fd94557a740 1 Processor -- start 2024-04-21T08:45:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.788+0000 7fd94557a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c638d4d0 0x5557c6399f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.788+0000 7fd94557a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c638de80 0x5557c639a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.788+0000 7fd94557a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557c6396a00 0x5557c6413ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.788+0000 7fd94557a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c1b2920 con 0x5557c638de80 2024-04-21T08:45:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.788+0000 7fd94557a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c1b2250 con 0x5557c638d4d0 2024-04-21T08:45:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.788+0000 7fd94557a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c1af490 con 0x5557c6396a00 2024-04-21T08:45:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.788+0000 7fd94557a740 10 client.0 _ll_register_callbacks cb 0x7fd92400ad00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.788+0000 7fd93b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c638de80 0x5557c639a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.788+0000 7fd93b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c638de80 0x5557c639a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38994/0 (socket says 172.21.15.43:38994) 2024-04-21T08:45:32.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.788+0000 7fd93b7fe640 1 -- 172.21.15.43:0/608554465 learned_addr learned my addr 172.21.15.43:0/608554465 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.789+0000 7fd93bfff640 1 --2- 172.21.15.43:0/608554465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c638d4d0 0x5557c6399f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.789+0000 7fd93b7fe640 1 -- 172.21.15.43:0/608554465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557c6396a00 msgr2=0x5557c6413ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.789+0000 7fd93b7fe640 1 --2- 172.21.15.43:0/608554465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557c6396a00 0x5557c6413ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.789+0000 7fd93b7fe640 1 -- 172.21.15.43:0/608554465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c638d4d0 msgr2=0x5557c6399f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.789+0000 7fd93b7fe640 1 --2- 172.21.15.43:0/608554465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c638d4d0 0x5557c6399f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.789+0000 7fd93b7fe640 1 -- 172.21.15.43:0/608554465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557c6414010 con 0x5557c638de80 2024-04-21T08:45:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.789+0000 7fd93b7fe640 1 --2- 172.21.15.43:0/608554465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c638de80 0x5557c639a460 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fd92c09c070 tx=0x7fd92c06aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231c3a5818048cc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.789+0000 7fd91b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.789+0000 7fd91b7fe640 1 -- 172.21.15.43:0/608554465 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92c020040 con 0x5557c638de80 2024-04-21T08:45:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.789+0000 7fd91b7fe640 1 -- 172.21.15.43:0/608554465 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fd92c005510 con 0x5557c638de80 2024-04-21T08:45:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.789+0000 7fd94557a740 1 -- 172.21.15.43:0/608554465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd934034df0 con 0x5557c638de80 2024-04-21T08:45:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.789+0000 7fd91b7fe640 1 -- 172.21.15.43:0/608554465 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd92c09c0e0 con 0x5557c638de80 2024-04-21T08:45:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.789+0000 7fd94557a740 10 client.18495 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.789+0000 7fd94557a740 1 -- 172.21.15.43:0/608554465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5557c6415260 con 0x5557c638de80 2024-04-21T08:45:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.789+0000 7fd91b7fe640 1 -- 172.21.15.43:0/608554465 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd934034df0 con 0x5557c638de80 2024-04-21T08:45:32.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.789+0000 7fd91b7fe640 1 -- 172.21.15.43:0/608554465 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd92c0cdb10 con 0x5557c638de80 2024-04-21T08:45:32.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.790+0000 7fd94557a740 10 client.18495 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:32.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.790+0000 7fd94557a740 10 client.18495 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:45:32.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.790+0000 7fd94557a740 20 client.18495 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:32.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.790+0000 7fd94557a740 1 --2- 172.21.15.43:0/608554465 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5557c6415a30 0x5557c6417e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.790+0000 7fd94557a740 4 client.18495 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:45:32.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.790+0000 7fd94557a740 1 -- 172.21.15.43:0/608554465 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557c62ffc10 con 0x5557c6415a30 2024-04-21T08:45:32.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.790+0000 7fd93bfff640 1 --2- 172.21.15.43:0/608554465 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5557c6415a30 0x5557c6417e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.790+0000 7fd93bfff640 1 --2- 172.21.15.43:0/608554465 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5557c6415a30 0x5557c6417e50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c16d5d3ee070266 server_cookie=7e8aec93ab39e8d4 in_seq=0 out_seq=0 2024-04-21T08:45:32.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.790+0000 7fd91b7fe640 10 client.18495 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:45:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.790+0000 7fd91b7fe640 1 -- 172.21.15.43:0/608554465 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557c62ffc10 con 0x5557c6415a30 2024-04-21T08:45:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.790+0000 7fd91b7fe640 10 client.18495 handle_command_reply: tid=0 2024-04-21T08:45:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.791+0000 7fd94557a740 1 client.18495 shutdown 2024-04-21T08:45:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.791+0000 7fd94557a740 1 -- 172.21.15.43:0/608554465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c638de80 msgr2=0x5557c639a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.791+0000 7fd94557a740 1 --2- 172.21.15.43:0/608554465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c638de80 0x5557c639a460 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fd92c09c070 tx=0x7fd92c06aa90 comp rx=0 tx=0).stop 2024-04-21T08:45:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.791+0000 7fd94557a740 1 -- 172.21.15.43:0/608554465 shutdown_connections 2024-04-21T08:45:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.791+0000 7fd94557a740 1 --2- 172.21.15.43:0/608554465 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5557c6415a30 0x5557c6417e50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.791+0000 7fd94557a740 1 --2- 172.21.15.43:0/608554465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557c6396a00 0x5557c6413ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.791+0000 7fd94557a740 1 --2- 172.21.15.43:0/608554465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557c638de80 0x5557c639a460 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.791+0000 7fd91b7fe640 0 client.18495 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:45:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.791+0000 7fd94557a740 1 --2- 172.21.15.43:0/608554465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557c638d4d0 0x5557c6399f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.791+0000 7fd94557a740 1 -- 172.21.15.43:0/608554465 >> 172.21.15.43:0/608554465 conn(0x5557c6387ba0 msgr2=0x5557c6388ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.791+0000 7fd94557a740 1 -- 172.21.15.43:0/608554465 shutdown_connections 2024-04-21T08:45:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.791+0000 7fd94557a740 1 -- 172.21.15.43:0/608554465 wait complete. 2024-04-21T08:45:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.791+0000 7fd94557a740 20 client.18495 trim_cache size 0 max 16384 2024-04-21T08:45:32.797 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:45:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.791+0000 7fd942984640 1 -- 172.21.15.43:0/1158142211 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd9280a3d50 msgr2=0x7fd9280a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.791+0000 7fd942984640 1 --2- 172.21.15.43:0/1158142211 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd9280a3d50 0x7fd9280a6210 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd934049000 tx=0x7fd934048000 comp rx=0 tx=0).stop 2024-04-21T08:45:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.791+0000 7fd942984640 1 -- 172.21.15.43:0/1158142211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd93c107270 msgr2=0x7fd93c1013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.791+0000 7fd942984640 1 --2- 172.21.15.43:0/1158142211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd93c107270 0x7fd93c1013c0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fd92c00d740 tx=0x7fd92c00db40 comp rx=0 tx=0).stop 2024-04-21T08:45:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.792+0000 7fd942984640 1 -- 172.21.15.43:0/1158142211 shutdown_connections 2024-04-21T08:45:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.792+0000 7fd942984640 1 --2- 172.21.15.43:0/1158142211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd93c101900 0x7fd93c1a6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.792+0000 7fd942984640 1 --2- 172.21.15.43:0/1158142211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd93c107270 0x7fd93c1013c0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.792+0000 7fd942984640 1 --2- 172.21.15.43:0/1158142211 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd9280a3d50 0x7fd9280a6210 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.792+0000 7fd942984640 1 --2- 172.21.15.43:0/1158142211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd93c06a840 0x7fd93c100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.792+0000 7fd942984640 1 -- 172.21.15.43:0/1158142211 >> 172.21.15.43:0/1158142211 conn(0x7fd93c077eb0 msgr2=0x7fd93c0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.792+0000 7fd942984640 1 -- 172.21.15.43:0/1158142211 shutdown_connections 2024-04-21T08:45:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.792+0000 7fd942984640 1 -- 172.21.15.43:0/1158142211 wait complete. 2024-04-21T08:45:32.802 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:45:32.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.907+0000 7f9aa3631640 1 -- 172.21.15.43:0/3943575221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a9c101090 msgr2=0x7f9a9c101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.907+0000 7f9aa3631640 1 --2- 172.21.15.43:0/3943575221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a9c101090 0x7f9a9c101470 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009900 tx=0x7f9a9802bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:32.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.907+0000 7f9aa3631640 1 -- 172.21.15.43:0/3943575221 shutdown_connections 2024-04-21T08:45:32.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.907+0000 7f9aa3631640 1 --2- 172.21.15.43:0/3943575221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a9c106160 0x7f9a9c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.907+0000 7f9aa3631640 1 --2- 172.21.15.43:0/3943575221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a9c101a40 0x7f9a9c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.907+0000 7f9aa3631640 1 --2- 172.21.15.43:0/3943575221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a9c101090 0x7f9a9c101470 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.907+0000 7f9aa3631640 1 -- 172.21.15.43:0/3943575221 >> 172.21.15.43:0/3943575221 conn(0x7f9a9c077f50 msgr2=0x7f9a9c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.907+0000 7f9aa3631640 1 -- 172.21.15.43:0/3943575221 shutdown_connections 2024-04-21T08:45:32.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.907+0000 7f9aa3631640 1 -- 172.21.15.43:0/3943575221 wait complete. 2024-04-21T08:45:32.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.907+0000 7f9aa3631640 1 Processor -- start 2024-04-21T08:45:32.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.907+0000 7f9aa3631640 1 -- start start 2024-04-21T08:45:32.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.908+0000 7f9aa3631640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a9c101090 0x7f9a9c197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.908+0000 7f9aa3631640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a9c106160 0x7f9a9c197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.908+0000 7f9aa3631640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a9c19bf20 0x7f9a9c19c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.908+0000 7f9aa3631640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c1110a0 con 0x7f9a9c19bf20 2024-04-21T08:45:32.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.908+0000 7f9aa3631640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c110f20 con 0x7f9a9c106160 2024-04-21T08:45:32.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.908+0000 7f9aa3631640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c111220 con 0x7f9a9c101090 2024-04-21T08:45:32.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.908+0000 7f9aa1bce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a9c19bf20 0x7f9a9c19c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.908+0000 7f9aa1bce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a9c19bf20 0x7f9a9c19c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39012/0 (socket says 172.21.15.43:39012) 2024-04-21T08:45:32.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.908+0000 7f9aa1bce640 1 -- 172.21.15.43:0/2426601833 learned_addr learned my addr 172.21.15.43:0/2426601833 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.908+0000 7f9aa13cd640 1 --2- 172.21.15.43:0/2426601833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a9c101090 0x7f9a9c197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.908+0000 7f9aa0bcc640 1 --2- 172.21.15.43:0/2426601833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a9c106160 0x7f9a9c197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.908+0000 7f9aa1bce640 1 -- 172.21.15.43:0/2426601833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a9c101090 msgr2=0x7f9a9c197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.908+0000 7f9aa1bce640 1 --2- 172.21.15.43:0/2426601833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a9c101090 0x7f9a9c197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.908+0000 7f9aa1bce640 1 -- 172.21.15.43:0/2426601833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a9c106160 msgr2=0x7f9a9c197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.908+0000 7f9aa1bce640 1 --2- 172.21.15.43:0/2426601833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a9c106160 0x7f9a9c197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.908+0000 7f9aa1bce640 1 -- 172.21.15.43:0/2426601833 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a9c19ca10 con 0x7f9a9c19bf20 2024-04-21T08:45:32.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.908+0000 7f9aa1bce640 1 --2- 172.21.15.43:0/2426601833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a9c19bf20 0x7f9a9c19c3d0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f9a9400ebb0 tx=0x7f9a94007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.908+0000 7f9a927fc640 1 -- 172.21.15.43:0/2426601833 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a94007ca0 con 0x7f9a9c19bf20 2024-04-21T08:45:32.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.908+0000 7f9a927fc640 1 -- 172.21.15.43:0/2426601833 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a94007e40 con 0x7f9a9c19bf20 2024-04-21T08:45:32.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.908+0000 7f9a927fc640 1 -- 172.21.15.43:0/2426601833 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a940057d0 con 0x7f9a9c19bf20 2024-04-21T08:45:32.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.908+0000 7f9aa3631640 1 -- 172.21.15.43:0/2426601833 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a9c19cca0 con 0x7f9a9c19bf20 2024-04-21T08:45:32.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.909+0000 7f9aa3631640 1 -- 172.21.15.43:0/2426601833 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a9c069380 con 0x7f9a9c19bf20 2024-04-21T08:45:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.911+0000 7f9a927fc640 1 -- 172.21.15.43:0/2426601833 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9a94004a40 con 0x7f9a9c19bf20 2024-04-21T08:45:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.912+0000 7f9aa6226740 1 Processor -- start 2024-04-21T08:45:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.912+0000 7f9aa6226740 1 -- start start 2024-04-21T08:45:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.912+0000 7f9aa6226740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b196913b0 0x563b19691790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.912+0000 7f9aa6226740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b19691d60 0x563b1969a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.912+0000 7f9aa6226740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1969ae60 0x563b1969b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.912+0000 7f9aa6226740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563b195f61f0 con 0x563b1969ae60 2024-04-21T08:45:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.912+0000 7f9aa6226740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563b195934b0 con 0x563b19691d60 2024-04-21T08:45:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.912+0000 7f9aa6226740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563b195f21b0 con 0x563b196913b0 2024-04-21T08:45:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.912+0000 7f9aa0bcc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1969ae60 0x563b1969b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.912+0000 7f9aa0bcc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1969ae60 0x563b1969b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39018/0 (socket says 172.21.15.43:39018) 2024-04-21T08:45:32.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.912+0000 7f9aa0bcc640 1 -- 172.21.15.43:0/4214640595 learned_addr learned my addr 172.21.15.43:0/4214640595 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.912+0000 7f9aa13cd640 1 --2- 172.21.15.43:0/4214640595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b19691d60 0x563b1969a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.912+0000 7f9aa0bcc640 1 --2- 172.21.15.43:0/4214640595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b196913b0 0x563b19691790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.912+0000 7f9a927fc640 1 --2- 172.21.15.43:0/2426601833 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9a880a3d20 0x7f9a880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.912+0000 7f9aa0bcc640 1 -- 172.21.15.43:0/4214640595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b196913b0 msgr2=0x563b19691790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.912+0000 7f9aa0bcc640 1 --2- 172.21.15.43:0/4214640595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b196913b0 0x563b19691790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.912+0000 7f9aa0bcc640 1 -- 172.21.15.43:0/4214640595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b19691d60 msgr2=0x563b1969a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.912+0000 7f9aa0bcc640 1 --2- 172.21.15.43:0/4214640595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b19691d60 0x563b1969a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.912+0000 7f9aa0bcc640 1 -- 172.21.15.43:0/4214640595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b195851b0 con 0x563b1969ae60 2024-04-21T08:45:32.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.912+0000 7f9aa1bce640 1 --2- 172.21.15.43:0/2426601833 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9a880a3d20 0x7f9a880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.913+0000 7f9a927fc640 1 -- 172.21.15.43:0/2426601833 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f9a940d2090 con 0x7f9a9c19bf20 2024-04-21T08:45:32.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.913+0000 7f9aa0bcc640 1 --2- 172.21.15.43:0/4214640595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1969ae60 0x563b1969b2c0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009560 tx=0x7f9a8c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d9bae3c99618a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.913+0000 7f9a877fe640 1 -- 172.21.15.43:0/4214640595 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c004050 con 0x563b1969ae60 2024-04-21T08:45:32.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.913+0000 7f9a877fe640 1 -- 172.21.15.43:0/4214640595 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a8c0054d0 con 0x563b1969ae60 2024-04-21T08:45:32.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.913+0000 7f9a877fe640 1 -- 172.21.15.43:0/4214640595 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c004b40 con 0x563b1969ae60 2024-04-21T08:45:32.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.913+0000 7f9aa6226740 1 -- 172.21.15.43:0/4214640595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1969ae60 msgr2=0x563b1969b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.913+0000 7f9aa1bce640 1 --2- 172.21.15.43:0/2426601833 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9a880a3d20 0x7f9a880a61e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9a9409a170 tx=0x7f9a9406c6c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.913+0000 7f9aa6226740 1 --2- 172.21.15.43:0/4214640595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1969ae60 0x563b1969b2c0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009560 tx=0x7f9a8c005850 comp rx=0 tx=0).stop 2024-04-21T08:45:32.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.914+0000 7f9aa6226740 1 -- 172.21.15.43:0/4214640595 shutdown_connections 2024-04-21T08:45:32.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.914+0000 7f9aa6226740 1 --2- 172.21.15.43:0/4214640595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1969ae60 0x563b1969b2c0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.914+0000 7f9aa6226740 1 --2- 172.21.15.43:0/4214640595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b19691d60 0x563b1969a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.914+0000 7f9aa6226740 1 --2- 172.21.15.43:0/4214640595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b196913b0 0x563b19691790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.914+0000 7f9aa6226740 1 -- 172.21.15.43:0/4214640595 >> 172.21.15.43:0/4214640595 conn(0x563b1968bba0 msgr2=0x563b1968bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.914+0000 7f9aa6226740 1 -- 172.21.15.43:0/4214640595 shutdown_connections 2024-04-21T08:45:32.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.914+0000 7f9aa6226740 1 -- 172.21.15.43:0/4214640595 wait complete. 2024-04-21T08:45:32.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.914+0000 7f9aa6226740 1 -- start start 2024-04-21T08:45:32.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.914+0000 7f9aa6226740 1 Processor -- start 2024-04-21T08:45:32.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.914+0000 7f9aa6226740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b196913b0 0x563b196a1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.914+0000 7f9aa6226740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b19691d60 0x563b1969e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.914+0000 7f9aa6226740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1969ae60 0x563b1969eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.914+0000 7f9aa6226740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563b196a1da0 con 0x563b196913b0 2024-04-21T08:45:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.914+0000 7f9aa6226740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563b195af500 con 0x563b1969ae60 2024-04-21T08:45:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.914+0000 7f9aa6226740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563b196a1f20 con 0x563b19691d60 2024-04-21T08:45:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.915+0000 7f9aa6226740 10 client.0 _ll_register_callbacks cb 0x563b1957db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.915+0000 7f9aa0bcc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b196913b0 0x563b196a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.915+0000 7f9aa0bcc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b196913b0 0x563b196a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39032/0 (socket says 172.21.15.43:39032) 2024-04-21T08:45:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.915+0000 7f9aa0bcc640 1 -- 172.21.15.43:0/831423459 learned_addr learned my addr 172.21.15.43:0/831423459 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.915+0000 7f9aa0bcc640 1 --2- 172.21.15.43:0/831423459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1969ae60 0x563b1969eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.915+0000 7f9aa13cd640 1 --2- 172.21.15.43:0/831423459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b19691d60 0x563b1969e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.915+0000 7f9aa0bcc640 1 -- 172.21.15.43:0/831423459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b19691d60 msgr2=0x563b1969e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.915+0000 7f9aa0bcc640 1 --2- 172.21.15.43:0/831423459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b19691d60 0x563b1969e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.915+0000 7f9aa0bcc640 1 -- 172.21.15.43:0/831423459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1969ae60 msgr2=0x563b1969eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.915+0000 7f9aa0bcc640 1 --2- 172.21.15.43:0/831423459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1969ae60 0x563b1969eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.915+0000 7f9aa0bcc640 1 -- 172.21.15.43:0/831423459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b193083f0 con 0x563b196913b0 2024-04-21T08:45:32.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.915+0000 7f9aa0bcc640 1 --2- 172.21.15.43:0/831423459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b196913b0 0x563b196a1230 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009980 tx=0x7f9a8c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7540164db1717e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.915+0000 7f9a84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:32.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.915+0000 7f9a84ff9640 1 -- 172.21.15.43:0/831423459 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c01e460 con 0x563b196913b0 2024-04-21T08:45:32.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.915+0000 7f9a84ff9640 1 -- 172.21.15.43:0/831423459 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f9a8c01e600 con 0x563b196913b0 2024-04-21T08:45:32.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.915+0000 7f9aa6226740 1 -- 172.21.15.43:0/831423459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a8c0054d0 con 0x563b196913b0 2024-04-21T08:45:32.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.915+0000 7f9a84ff9640 1 -- 172.21.15.43:0/831423459 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9a8c034050 con 0x563b196913b0 2024-04-21T08:45:32.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.915+0000 7f9aa6226740 10 client.18507 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:32.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.915+0000 7f9aa6226740 1 -- 172.21.15.43:0/831423459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563b193d27d0 con 0x563b196913b0 2024-04-21T08:45:32.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.916+0000 7f9a84ff9640 1 -- 172.21.15.43:0/831423459 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a8c0054d0 con 0x563b196913b0 2024-04-21T08:45:32.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.916+0000 7f9a84ff9640 1 -- 172.21.15.43:0/831423459 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9a8c02b670 con 0x563b196913b0 2024-04-21T08:45:32.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.916+0000 7f9aa6226740 10 client.18507 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:32.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.916+0000 7f9aa6226740 10 client.18507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:32.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.916+0000 7f9aa6226740 20 client.18507 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:32.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.916+0000 7f9aa6226740 1 --2- 172.21.15.43:0/831423459 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563b196aeed0 0x563b196b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.916+0000 7f9aa6226740 4 client.18507 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:32.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.916+0000 7f9aa6226740 1 -- 172.21.15.43:0/831423459 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b19595930 con 0x563b196aeed0 2024-04-21T08:45:32.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.916+0000 7f9aa13cd640 1 --2- 172.21.15.43:0/831423459 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563b196aeed0 0x563b196b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.916+0000 7f9aa13cd640 1 --2- 172.21.15.43:0/831423459 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563b196aeed0 0x563b196b12c0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=febdae27b2f6a700 server_cookie=9cbdffce08d4c7de in_seq=0 out_seq=0 2024-04-21T08:45:32.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.916+0000 7f9a84ff9640 10 client.18507 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:32.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.917+0000 7f9a84ff9640 1 -- 172.21.15.43:0/831423459 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563b19595930 con 0x563b196aeed0 2024-04-21T08:45:32.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.917+0000 7f9a84ff9640 10 client.18507 handle_command_reply: tid=0 2024-04-21T08:45:32.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.917+0000 7f9aa6226740 1 client.18507 shutdown 2024-04-21T08:45:32.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.917+0000 7f9aa6226740 1 -- 172.21.15.43:0/831423459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b196913b0 msgr2=0x563b196a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.917+0000 7f9aa6226740 1 --2- 172.21.15.43:0/831423459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b196913b0 0x563b196a1230 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009980 tx=0x7f9a8c0096a0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.917+0000 7f9aa6226740 1 -- 172.21.15.43:0/831423459 shutdown_connections 2024-04-21T08:45:32.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.917+0000 7f9aa6226740 1 --2- 172.21.15.43:0/831423459 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563b196aeed0 0x563b196b12c0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.917+0000 7f9aa6226740 1 --2- 172.21.15.43:0/831423459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1969ae60 0x563b1969eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.917+0000 7f9a84ff9640 0 client.18507 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:32.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.917+0000 7f9aa6226740 1 --2- 172.21.15.43:0/831423459 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b19691d60 0x563b1969e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.917+0000 7f9aa6226740 1 --2- 172.21.15.43:0/831423459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b196913b0 0x563b196a1230 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.917+0000 7f9aa6226740 1 -- 172.21.15.43:0/831423459 >> 172.21.15.43:0/831423459 conn(0x563b1968bba0 msgr2=0x563b1968d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.917+0000 7f9aa6226740 1 -- 172.21.15.43:0/831423459 shutdown_connections 2024-04-21T08:45:32.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.917+0000 7f9aa6226740 1 -- 172.21.15.43:0/831423459 wait complete. 2024-04-21T08:45:32.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.918+0000 7f9aa6226740 20 client.18507 trim_cache size 0 max 16384 2024-04-21T08:45:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.931+0000 7f9aa6226740 1 Processor -- start 2024-04-21T08:45:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.931+0000 7f9aa6226740 1 -- start start 2024-04-21T08:45:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.931+0000 7f9aa6226740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b196913b0 0x563b196aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.931+0000 7f9aa6226740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b19691d60 0x563b196a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.931+0000 7f9aa6226740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1969ae60 0x563b196a1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.931+0000 7f9aa6226740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c1ad230 con 0x563b1969ae60 2024-04-21T08:45:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.931+0000 7f9aa6226740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c1aed70 con 0x563b196913b0 2024-04-21T08:45:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.931+0000 7f9aa6226740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c1ad900 con 0x563b19691d60 2024-04-21T08:45:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.931+0000 7f9aa0bcc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1969ae60 0x563b196a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.931+0000 7f9aa0bcc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1969ae60 0x563b196a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39046/0 (socket says 172.21.15.43:39046) 2024-04-21T08:45:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.931+0000 7f9aa0bcc640 1 -- 172.21.15.43:0/133467785 learned_addr learned my addr 172.21.15.43:0/133467785 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.931+0000 7f9aa13cd640 1 --2- 172.21.15.43:0/133467785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b19691d60 0x563b196a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.931+0000 7f9aa0bcc640 1 -- 172.21.15.43:0/133467785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b19691d60 msgr2=0x563b196a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.931+0000 7f9aa0bcc640 1 --2- 172.21.15.43:0/133467785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b19691d60 0x563b196a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.931+0000 7f9aa0bcc640 1 -- 172.21.15.43:0/133467785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b196913b0 msgr2=0x563b196aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.931+0000 7f9aa0bcc640 1 --2- 172.21.15.43:0/133467785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b196913b0 0x563b196aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.931+0000 7f9aa0bcc640 1 -- 172.21.15.43:0/133467785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b19690fe0 con 0x563b1969ae60 2024-04-21T08:45:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.932+0000 7f9aa0bcc640 1 --2- 172.21.15.43:0/133467785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1969ae60 0x563b196a1230 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0057c0 tx=0x7f9a8c0097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2770a51a8b0cf4ea server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.932+0000 7f9a86ffd640 1 -- 172.21.15.43:0/133467785 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c0054d0 con 0x563b1969ae60 2024-04-21T08:45:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.932+0000 7f9a86ffd640 1 -- 172.21.15.43:0/133467785 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a8c011d60 con 0x563b1969ae60 2024-04-21T08:45:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.932+0000 7f9a86ffd640 1 -- 172.21.15.43:0/133467785 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c02e310 con 0x563b1969ae60 2024-04-21T08:45:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.932+0000 7f9aa6226740 1 -- 172.21.15.43:0/133467785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1969ae60 msgr2=0x563b196a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.932+0000 7f9aa6226740 1 --2- 172.21.15.43:0/133467785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1969ae60 0x563b196a1230 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0057c0 tx=0x7f9a8c0097a0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.932+0000 7f9aa6226740 1 -- 172.21.15.43:0/133467785 shutdown_connections 2024-04-21T08:45:32.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.932+0000 7f9aa6226740 1 --2- 172.21.15.43:0/133467785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1969ae60 0x563b196a1230 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.932+0000 7f9aa6226740 1 --2- 172.21.15.43:0/133467785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b19691d60 0x563b196a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.932+0000 7f9aa6226740 1 --2- 172.21.15.43:0/133467785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b196913b0 0x563b196aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.932+0000 7f9aa6226740 1 -- 172.21.15.43:0/133467785 >> 172.21.15.43:0/133467785 conn(0x563b1968bba0 msgr2=0x563b1968d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.932+0000 7f9aa6226740 1 -- 172.21.15.43:0/133467785 shutdown_connections 2024-04-21T08:45:32.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.932+0000 7f9aa6226740 1 -- 172.21.15.43:0/133467785 wait complete. 2024-04-21T08:45:32.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.933+0000 7f9aa6226740 1 -- start start 2024-04-21T08:45:32.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.933+0000 7f9aa6226740 1 Processor -- start 2024-04-21T08:45:32.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.933+0000 7f9aa6226740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b196913b0 0x563b196a71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.933+0000 7f9aa6226740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b19691d60 0x563b196a76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.933+0000 7f9aa6226740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1969ae60 0x563b1969fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.933+0000 7f9aa6226740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c1ae6a0 con 0x563b196913b0 2024-04-21T08:45:32.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.933+0000 7f9aa6226740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c1adfd0 con 0x563b1969ae60 2024-04-21T08:45:32.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.933+0000 7f9aa6226740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c1ab210 con 0x563b19691d60 2024-04-21T08:45:32.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.933+0000 7f9aa6226740 10 client.0 _ll_register_callbacks cb 0x7f9a8c005680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:32.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.933+0000 7f9aa0bcc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b196913b0 0x563b196a71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.933+0000 7f9aa0bcc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b196913b0 0x563b196a71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39062/0 (socket says 172.21.15.43:39062) 2024-04-21T08:45:32.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.933+0000 7f9aa0bcc640 1 -- 172.21.15.43:0/1905106883 learned_addr learned my addr 172.21.15.43:0/1905106883 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:32.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.933+0000 7f9aa13cd640 1 --2- 172.21.15.43:0/1905106883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b19691d60 0x563b196a76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.933+0000 7f9aa0bcc640 1 --2- 172.21.15.43:0/1905106883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1969ae60 0x563b1969fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.933+0000 7f9aa0bcc640 1 -- 172.21.15.43:0/1905106883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b19691d60 msgr2=0x563b196a76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.933+0000 7f9aa0bcc640 1 --2- 172.21.15.43:0/1905106883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b19691d60 0x563b196a76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.933+0000 7f9aa0bcc640 1 -- 172.21.15.43:0/1905106883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1969ae60 msgr2=0x563b1969fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:32.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.933+0000 7f9aa0bcc640 1 --2- 172.21.15.43:0/1905106883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1969ae60 0x563b1969fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.934+0000 7f9aa0bcc640 1 -- 172.21.15.43:0/1905106883 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b196a0330 con 0x563b196913b0 2024-04-21T08:45:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.934+0000 7f9aa0bcc640 1 --2- 172.21.15.43:0/1905106883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b196913b0 0x563b196a71b0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009700 tx=0x7f9a8c02bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e2294a2e81778c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.934+0000 7f9a84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.934+0000 7f9a84ff9640 1 -- 172.21.15.43:0/1905106883 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c040020 con 0x563b196913b0 2024-04-21T08:45:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.934+0000 7f9aa6226740 1 -- 172.21.15.43:0/1905106883 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a8c011d60 con 0x563b196913b0 2024-04-21T08:45:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.934+0000 7f9a84ff9640 1 -- 172.21.15.43:0/1905106883 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f9a8c02e2b0 con 0x563b196913b0 2024-04-21T08:45:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.934+0000 7f9a84ff9640 1 -- 172.21.15.43:0/1905106883 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9a8c0048e0 con 0x563b196913b0 2024-04-21T08:45:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.934+0000 7f9aa6226740 10 client.18513 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.934+0000 7f9aa6226740 1 -- 172.21.15.43:0/1905106883 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563b19714cd0 con 0x563b196913b0 2024-04-21T08:45:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.934+0000 7f9a84ff9640 1 -- 172.21.15.43:0/1905106883 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a8c011d60 con 0x563b196913b0 2024-04-21T08:45:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.934+0000 7f9a84ff9640 1 -- 172.21.15.43:0/1905106883 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9a8c0035f0 con 0x563b196913b0 2024-04-21T08:45:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.934+0000 7f9aa6226740 10 client.18513 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.934+0000 7f9aa6226740 10 client.18513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.934+0000 7f9aa6226740 20 client.18513 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.934+0000 7f9aa6226740 1 --2- 172.21.15.43:0/1905106883 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563b197151e0 0x563b197175d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.934+0000 7f9aa6226740 4 client.18513 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:45:32.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.934+0000 7f9aa6226740 1 -- 172.21.15.43:0/1905106883 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b19603c10 con 0x563b197151e0 2024-04-21T08:45:32.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.935+0000 7f9aa13cd640 1 --2- 172.21.15.43:0/1905106883 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563b197151e0 0x563b197175d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.935+0000 7f9aa13cd640 1 --2- 172.21.15.43:0/1905106883 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563b197151e0 0x563b197175d0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d15af4ed0fa37abe server_cookie=aa2085bb51d814c0 in_seq=0 out_seq=0 2024-04-21T08:45:32.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.935+0000 7f9a84ff9640 10 client.18513 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:32.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.935+0000 7f9a84ff9640 1 -- 172.21.15.43:0/1905106883 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b19603c10 con 0x563b197151e0 2024-04-21T08:45:32.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.935+0000 7f9a84ff9640 10 client.18513 handle_command_reply: tid=0 2024-04-21T08:45:32.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.935+0000 7f9aa6226740 1 client.18513 shutdown 2024-04-21T08:45:32.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.936+0000 7f9aa6226740 1 -- 172.21.15.43:0/1905106883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b196913b0 msgr2=0x563b196a71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.936+0000 7f9aa6226740 1 --2- 172.21.15.43:0/1905106883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b196913b0 0x563b196a71b0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009700 tx=0x7f9a8c02bf90 comp rx=0 tx=0).stop 2024-04-21T08:45:32.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.936+0000 7f9aa6226740 1 -- 172.21.15.43:0/1905106883 shutdown_connections 2024-04-21T08:45:32.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.936+0000 7f9aa6226740 1 --2- 172.21.15.43:0/1905106883 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563b197151e0 0x563b197175d0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.936+0000 7f9aa6226740 1 --2- 172.21.15.43:0/1905106883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b1969ae60 0x563b1969fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.936+0000 7f9aa6226740 1 --2- 172.21.15.43:0/1905106883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b19691d60 0x563b196a76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.936+0000 7f9aa6226740 1 --2- 172.21.15.43:0/1905106883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b196913b0 0x563b196a71b0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.936+0000 7f9a84ff9640 0 client.18513 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:32.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.936+0000 7f9aa6226740 1 -- 172.21.15.43:0/1905106883 >> 172.21.15.43:0/1905106883 conn(0x563b1968bba0 msgr2=0x563b1968a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.936+0000 7f9aa6226740 1 -- 172.21.15.43:0/1905106883 shutdown_connections 2024-04-21T08:45:32.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.936+0000 7f9aa6226740 1 -- 172.21.15.43:0/1905106883 wait complete. 2024-04-21T08:45:32.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.936+0000 7f9aa6226740 20 client.18513 trim_cache size 0 max 16384 2024-04-21T08:45:32.942 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:45:32.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.936+0000 7f9aa3631640 1 -- 172.21.15.43:0/2426601833 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9a880a3d20 msgr2=0x7f9a880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.936+0000 7f9aa3631640 1 --2- 172.21.15.43:0/2426601833 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9a880a3d20 0x7f9a880a61e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9a9409a170 tx=0x7f9a9406c6c0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.936+0000 7f9aa3631640 1 -- 172.21.15.43:0/2426601833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a9c19bf20 msgr2=0x7f9a9c19c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.936+0000 7f9aa3631640 1 --2- 172.21.15.43:0/2426601833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a9c19bf20 0x7f9a9c19c3d0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f9a9400ebb0 tx=0x7f9a94007590 comp rx=0 tx=0).stop 2024-04-21T08:45:32.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.936+0000 7f9aa3631640 1 -- 172.21.15.43:0/2426601833 shutdown_connections 2024-04-21T08:45:32.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.936+0000 7f9aa3631640 1 --2- 172.21.15.43:0/2426601833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a9c19bf20 0x7f9a9c19c3d0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.936+0000 7f9aa3631640 1 --2- 172.21.15.43:0/2426601833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a9c106160 0x7f9a9c197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.936+0000 7f9aa3631640 1 --2- 172.21.15.43:0/2426601833 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9a880a3d20 0x7f9a880a61e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.936+0000 7f9aa3631640 1 --2- 172.21.15.43:0/2426601833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a9c101090 0x7f9a9c197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.936+0000 7f9aa3631640 1 -- 172.21.15.43:0/2426601833 >> 172.21.15.43:0/2426601833 conn(0x7f9a9c077f50 msgr2=0x7f9a9c0ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.936+0000 7f9aa3631640 1 -- 172.21.15.43:0/2426601833 shutdown_connections 2024-04-21T08:45:32.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:32.936+0000 7f9aa3631640 1 -- 172.21.15.43:0/2426601833 wait complete. 2024-04-21T08:45:33.463 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:33 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/108881562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:33.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:33 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/108881562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:33 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/108881562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:33.948 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:45:33.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:45:33.949 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:45:34.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.069+0000 7fa05f7f4640 1 -- 172.21.15.43:0/1136174740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0581010a0 msgr2=0x7fa058101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.069+0000 7fa05f7f4640 1 --2- 172.21.15.43:0/1136174740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0581010a0 0x7fa058101480 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fa054009900 tx=0x7fa05402bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:34.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.069+0000 7fa05f7f4640 1 -- 172.21.15.43:0/1136174740 shutdown_connections 2024-04-21T08:45:34.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.069+0000 7fa05f7f4640 1 --2- 172.21.15.43:0/1136174740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa058106170 0x7fa05810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.069+0000 7fa05f7f4640 1 --2- 172.21.15.43:0/1136174740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa058101a50 0x7fa058105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.069+0000 7fa05f7f4640 1 --2- 172.21.15.43:0/1136174740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0581010a0 0x7fa058101480 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.069+0000 7fa05f7f4640 1 -- 172.21.15.43:0/1136174740 >> 172.21.15.43:0/1136174740 conn(0x7fa058077eb0 msgr2=0x7fa0580782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.069+0000 7fa05f7f4640 1 -- 172.21.15.43:0/1136174740 shutdown_connections 2024-04-21T08:45:34.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.069+0000 7fa05f7f4640 1 -- 172.21.15.43:0/1136174740 wait complete. 2024-04-21T08:45:34.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.069+0000 7fa05f7f4640 1 Processor -- start 2024-04-21T08:45:34.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.070+0000 7fa05f7f4640 1 -- start start 2024-04-21T08:45:34.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.070+0000 7fa05f7f4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0581010a0 0x7fa058198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.070+0000 7fa05f7f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa058106170 0x7fa058198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.070+0000 7fa05f7f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa05819cff0 0x7fa05819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.070+0000 7fa05f7f4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa0581110d0 con 0x7fa05819cff0 2024-04-21T08:45:34.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.070+0000 7fa05f7f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa058110f50 con 0x7fa058106170 2024-04-21T08:45:34.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.070+0000 7fa05f7f4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa058111250 con 0x7fa0581010a0 2024-04-21T08:45:34.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.070+0000 7fa05dd91640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa05819cff0 0x7fa05819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.070+0000 7fa05dd91640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa05819cff0 0x7fa05819d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39080/0 (socket says 172.21.15.43:39080) 2024-04-21T08:45:34.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.070+0000 7fa05dd91640 1 -- 172.21.15.43:0/2621523793 learned_addr learned my addr 172.21.15.43:0/2621523793 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:34.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.070+0000 7fa05cd8f640 1 --2- 172.21.15.43:0/2621523793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa058106170 0x7fa058198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.070+0000 7fa05d590640 1 --2- 172.21.15.43:0/2621523793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0581010a0 0x7fa058198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.070+0000 7fa05dd91640 1 -- 172.21.15.43:0/2621523793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0581010a0 msgr2=0x7fa058198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.070+0000 7fa05dd91640 1 --2- 172.21.15.43:0/2621523793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0581010a0 0x7fa058198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.070+0000 7fa05dd91640 1 -- 172.21.15.43:0/2621523793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa058106170 msgr2=0x7fa058198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.070+0000 7fa05dd91640 1 --2- 172.21.15.43:0/2621523793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa058106170 0x7fa058198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.070+0000 7fa05dd91640 1 -- 172.21.15.43:0/2621523793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05819dae0 con 0x7fa05819cff0 2024-04-21T08:45:34.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.070+0000 7fa05cd8f640 1 --2- 172.21.15.43:0/2621523793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa058106170 0x7fa058198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:45:34.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.070+0000 7fa05dd91640 1 --2- 172.21.15.43:0/2621523793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa05819cff0 0x7fa05819d4a0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fa05000eb10 tx=0x7fa05000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.071+0000 7fa04e7fc640 1 -- 172.21.15.43:0/2621523793 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050004b20 con 0x7fa05819cff0 2024-04-21T08:45:34.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.071+0000 7fa05f7f4640 1 -- 172.21.15.43:0/2621523793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05819ddd0 con 0x7fa05819cff0 2024-04-21T08:45:34.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.071+0000 7fa04e7fc640 1 -- 172.21.15.43:0/2621523793 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa050004cc0 con 0x7fa05819cff0 2024-04-21T08:45:34.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.071+0000 7fa04e7fc640 1 -- 172.21.15.43:0/2621523793 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050010670 con 0x7fa05819cff0 2024-04-21T08:45:34.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.071+0000 7fa05f7f4640 1 -- 172.21.15.43:0/2621523793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0581a5570 con 0x7fa05819cff0 2024-04-21T08:45:34.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.073+0000 7fa04e7fc640 1 -- 172.21.15.43:0/2621523793 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa050005020 con 0x7fa05819cff0 2024-04-21T08:45:34.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.073+0000 7fa0623e7740 1 Processor -- start 2024-04-21T08:45:34.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.073+0000 7fa0623e7740 1 -- start start 2024-04-21T08:45:34.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.074+0000 7fa0623e7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591f15fb8e0 0x5591f15fbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.074+0000 7fa0623e7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fc290 0x5591f1604840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.074+0000 7fa0623e7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591f1604e10 0x5591f1607260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.074+0000 7fa0623e7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5591f155ff00 con 0x5591f15fc290 2024-04-21T08:45:34.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.074+0000 7fa0623e7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5591f14fd590 con 0x5591f15fb8e0 2024-04-21T08:45:34.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.074+0000 7fa0623e7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5591f155bec0 con 0x5591f1604e10 2024-04-21T08:45:34.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.074+0000 7fa04e7fc640 1 --2- 172.21.15.43:0/2621523793 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa0440a3d20 0x7fa0440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.074+0000 7fa05d590640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fc290 0x5591f1604840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.074+0000 7fa05d590640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fc290 0x5591f1604840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39090/0 (socket says 172.21.15.43:39090) 2024-04-21T08:45:34.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.074+0000 7fa05d590640 1 -- 172.21.15.43:0/1229717581 learned_addr learned my addr 172.21.15.43:0/1229717581 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:34.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.074+0000 7fa05dd91640 1 --2- 172.21.15.43:0/2621523793 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa0440a3d20 0x7fa0440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.074+0000 7fa05cd8f640 1 --2- 172.21.15.43:0/1229717581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591f1604e10 0x5591f1607260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.074+0000 7fa05cd8f640 1 --2- 172.21.15.43:0/1229717581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591f15fb8e0 0x5591f15fbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.074+0000 7fa04e7fc640 1 -- 172.21.15.43:0/2621523793 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fa0500d1180 con 0x7fa05819cff0 2024-04-21T08:45:34.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.074+0000 7fa05d590640 1 -- 172.21.15.43:0/1229717581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591f1604e10 msgr2=0x5591f1607260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.074+0000 7fa05d590640 1 --2- 172.21.15.43:0/1229717581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591f1604e10 0x5591f1607260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.074+0000 7fa05d590640 1 -- 172.21.15.43:0/1229717581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591f15fb8e0 msgr2=0x5591f15fbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.074+0000 7fa05d590640 1 --2- 172.21.15.43:0/1229717581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591f15fb8e0 0x5591f15fbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.074+0000 7fa05d590640 1 -- 172.21.15.43:0/1229717581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591f133a7e0 con 0x5591f15fc290 2024-04-21T08:45:34.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.074+0000 7fa05cd8f640 1 --2- 172.21.15.43:0/1229717581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591f15fb8e0 0x5591f15fbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:34.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.075+0000 7fa05dd91640 1 --2- 172.21.15.43:0/2621523793 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa0440a3d20 0x7fa0440a61e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa0500991b0 tx=0x7fa05006b700 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.075+0000 7fa05d590640 1 --2- 172.21.15.43:0/1229717581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fc290 0x5591f1604840 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fa054009900 tx=0x7fa054005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c15cca0460de85 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.075+0000 7fa0437fe640 1 -- 172.21.15.43:0/1229717581 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054044020 con 0x5591f15fc290 2024-04-21T08:45:34.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.075+0000 7fa0437fe640 1 -- 172.21.15.43:0/1229717581 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa05402cc90 con 0x5591f15fc290 2024-04-21T08:45:34.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.075+0000 7fa0437fe640 1 -- 172.21.15.43:0/1229717581 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0540046c0 con 0x5591f15fc290 2024-04-21T08:45:34.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.075+0000 7fa0623e7740 1 -- 172.21.15.43:0/1229717581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fc290 msgr2=0x5591f1604840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.075+0000 7fa0623e7740 1 --2- 172.21.15.43:0/1229717581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fc290 0x5591f1604840 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fa054009900 tx=0x7fa054005dc0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.076+0000 7fa0623e7740 1 -- 172.21.15.43:0/1229717581 shutdown_connections 2024-04-21T08:45:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.076+0000 7fa0623e7740 1 --2- 172.21.15.43:0/1229717581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591f1604e10 0x5591f1607260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.076+0000 7fa0623e7740 1 --2- 172.21.15.43:0/1229717581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fc290 0x5591f1604840 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.076+0000 7fa0623e7740 1 --2- 172.21.15.43:0/1229717581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591f15fb8e0 0x5591f15fbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.076+0000 7fa0623e7740 1 -- 172.21.15.43:0/1229717581 >> 172.21.15.43:0/1229717581 conn(0x5591f15f5fb0 msgr2=0x5591f15f6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.076+0000 7fa0623e7740 1 -- 172.21.15.43:0/1229717581 shutdown_connections 2024-04-21T08:45:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.076+0000 7fa0623e7740 1 -- 172.21.15.43:0/1229717581 wait complete. 2024-04-21T08:45:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.076+0000 7fa0623e7740 1 -- start start 2024-04-21T08:45:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.076+0000 7fa0623e7740 1 Processor -- start 2024-04-21T08:45:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.076+0000 7fa0623e7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fb8e0 0x5591f1601e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.077+0000 7fa0623e7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591f15fc290 0x5591f1602340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.077+0000 7fa0623e7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591f1604e10 0x5591f1602880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.077+0000 7fa0623e7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5591f1607d20 con 0x5591f15fb8e0 2024-04-21T08:45:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.077+0000 7fa0623e7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5591f1519580 con 0x5591f15fc290 2024-04-21T08:45:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.077+0000 7fa0623e7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5591f1607ea0 con 0x5591f1604e10 2024-04-21T08:45:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.077+0000 7fa05cd8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fb8e0 0x5591f1601e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.077+0000 7fa0623e7740 10 client.0 _ll_register_callbacks cb 0x5591f151f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.077+0000 7fa05cd8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fb8e0 0x5591f1601e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39100/0 (socket says 172.21.15.43:39100) 2024-04-21T08:45:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.077+0000 7fa05cd8f640 1 -- 172.21.15.43:0/3722847387 learned_addr learned my addr 172.21.15.43:0/3722847387 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.077+0000 7fa05d590640 1 --2- 172.21.15.43:0/3722847387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591f15fc290 0x5591f1602340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.077+0000 7fa05cd8f640 1 --2- 172.21.15.43:0/3722847387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591f1604e10 0x5591f1602880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.077+0000 7fa05cd8f640 1 -- 172.21.15.43:0/3722847387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591f1604e10 msgr2=0x5591f1602880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.077+0000 7fa05cd8f640 1 --2- 172.21.15.43:0/3722847387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591f1604e10 0x5591f1602880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.077+0000 7fa05cd8f640 1 -- 172.21.15.43:0/3722847387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591f15fc290 msgr2=0x5591f1602340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.077+0000 7fa05cd8f640 1 --2- 172.21.15.43:0/3722847387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591f15fc290 0x5591f1602340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.077+0000 7fa05cd8f640 1 -- 172.21.15.43:0/3722847387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591f139af70 con 0x5591f15fb8e0 2024-04-21T08:45:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.077+0000 7fa05cd8f640 1 --2- 172.21.15.43:0/3722847387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fb8e0 0x5591f1601e00 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fa0480036e0 tx=0x7fa048005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a515fb17f7e3ced server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.077+0000 7fa040ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.077+0000 7fa040ff9640 1 -- 172.21.15.43:0/3722847387 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048011ab0 con 0x5591f15fb8e0 2024-04-21T08:45:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.077+0000 7fa040ff9640 1 -- 172.21.15.43:0/3722847387 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fa048011c50 con 0x5591f15fb8e0 2024-04-21T08:45:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.077+0000 7fa0623e7740 1 -- 172.21.15.43:0/3722847387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa05402cc90 con 0x5591f15fb8e0 2024-04-21T08:45:34.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.078+0000 7fa040ff9640 1 -- 172.21.15.43:0/3722847387 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa048029050 con 0x5591f15fb8e0 2024-04-21T08:45:34.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.078+0000 7fa0623e7740 10 client.18525 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:34.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.078+0000 7fa0623e7740 1 -- 172.21.15.43:0/3722847387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5591f1232170 con 0x5591f15fb8e0 2024-04-21T08:45:34.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.078+0000 7fa040ff9640 1 -- 172.21.15.43:0/3722847387 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa05402cc90 con 0x5591f15fb8e0 2024-04-21T08:45:34.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.078+0000 7fa040ff9640 1 -- 172.21.15.43:0/3722847387 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa04801d030 con 0x5591f15fb8e0 2024-04-21T08:45:34.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.078+0000 7fa0623e7740 10 client.18525 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:34.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.078+0000 7fa0623e7740 10 client.18525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:34.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.078+0000 7fa0623e7740 20 client.18525 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:34.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.078+0000 7fa0623e7740 1 --2- 172.21.15.43:0/3722847387 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5591f1615360 0x5591f16177b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.078+0000 7fa0623e7740 4 client.18525 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:34.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.078+0000 7fa0623e7740 1 -- 172.21.15.43:0/3722847387 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591f156d920 con 0x5591f1615360 2024-04-21T08:45:34.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.078+0000 7fa05d590640 1 --2- 172.21.15.43:0/3722847387 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5591f1615360 0x5591f16177b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.079+0000 7fa05d590640 1 --2- 172.21.15.43:0/3722847387 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5591f1615360 0x5591f16177b0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1fbe54bd929f652 server_cookie=8b83376df94daa72 in_seq=0 out_seq=0 2024-04-21T08:45:34.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.079+0000 7fa040ff9640 10 client.18525 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:34.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.079+0000 7fa040ff9640 1 -- 172.21.15.43:0/3722847387 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5591f156d920 con 0x5591f1615360 2024-04-21T08:45:34.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.079+0000 7fa040ff9640 10 client.18525 handle_command_reply: tid=0 2024-04-21T08:45:34.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.079+0000 7fa0623e7740 1 client.18525 shutdown 2024-04-21T08:45:34.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.080+0000 7fa0623e7740 1 -- 172.21.15.43:0/3722847387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fb8e0 msgr2=0x5591f1601e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.080+0000 7fa0623e7740 1 --2- 172.21.15.43:0/3722847387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fb8e0 0x5591f1601e00 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fa0480036e0 tx=0x7fa048005f90 comp rx=0 tx=0).stop 2024-04-21T08:45:34.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.080+0000 7fa0623e7740 1 -- 172.21.15.43:0/3722847387 shutdown_connections 2024-04-21T08:45:34.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.080+0000 7fa0623e7740 1 --2- 172.21.15.43:0/3722847387 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5591f1615360 0x5591f16177b0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.080+0000 7fa0623e7740 1 --2- 172.21.15.43:0/3722847387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591f1604e10 0x5591f1602880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.080+0000 7fa0623e7740 1 --2- 172.21.15.43:0/3722847387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591f15fc290 0x5591f1602340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.080+0000 7fa040ff9640 0 client.18525 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:34.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.080+0000 7fa0623e7740 1 --2- 172.21.15.43:0/3722847387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fb8e0 0x5591f1601e00 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.080+0000 7fa0623e7740 1 -- 172.21.15.43:0/3722847387 >> 172.21.15.43:0/3722847387 conn(0x5591f15f5fb0 msgr2=0x5591f15f7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.080+0000 7fa0623e7740 1 -- 172.21.15.43:0/3722847387 shutdown_connections 2024-04-21T08:45:34.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.080+0000 7fa0623e7740 1 -- 172.21.15.43:0/3722847387 wait complete. 2024-04-21T08:45:34.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.080+0000 7fa0623e7740 20 client.18525 trim_cache size 0 max 16384 2024-04-21T08:45:34.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.098+0000 7fa0623e7740 1 Processor -- start 2024-04-21T08:45:34.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.098+0000 7fa0623e7740 1 -- start start 2024-04-21T08:45:34.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.098+0000 7fa0623e7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591f15fb8e0 0x5591f1604530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.098+0000 7fa0623e7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fc290 0x5591f1610be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.098+0000 7fa0623e7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591f1604e10 0x5591f15f7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.098+0000 7fa0623e7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa0581af3f0 con 0x5591f15fc290 2024-04-21T08:45:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.098+0000 7fa0623e7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa0581b0f30 con 0x5591f1604e10 2024-04-21T08:45:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.098+0000 7fa0623e7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa0581afac0 con 0x5591f15fb8e0 2024-04-21T08:45:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.098+0000 7fa05d590640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fc290 0x5591f1610be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.098+0000 7fa05d590640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fc290 0x5591f1610be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39114/0 (socket says 172.21.15.43:39114) 2024-04-21T08:45:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.098+0000 7fa05d590640 1 -- 172.21.15.43:0/627559107 learned_addr learned my addr 172.21.15.43:0/627559107 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.098+0000 7fa05cd8f640 1 --2- 172.21.15.43:0/627559107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591f1604e10 0x5591f15f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.098+0000 7fa05d590640 1 -- 172.21.15.43:0/627559107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591f15fb8e0 msgr2=0x5591f1604530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.099+0000 7fa05d590640 1 --2- 172.21.15.43:0/627559107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591f15fb8e0 0x5591f1604530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.099+0000 7fa05d590640 1 -- 172.21.15.43:0/627559107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591f1604e10 msgr2=0x5591f15f7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.099+0000 7fa05d590640 1 --2- 172.21.15.43:0/627559107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591f1604e10 0x5591f15f7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.099+0000 7fa05d590640 1 -- 172.21.15.43:0/627559107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591f15faba0 con 0x5591f15fc290 2024-04-21T08:45:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.099+0000 7fa05d590640 1 --2- 172.21.15.43:0/627559107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fc290 0x5591f1610be0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x5591f15fb3b0 tx=0x7fa054000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2b21fa22dc6500 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.099+0000 7fa042ffd640 1 -- 172.21.15.43:0/627559107 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054044040 con 0x5591f15fc290 2024-04-21T08:45:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.099+0000 7fa042ffd640 1 -- 172.21.15.43:0/627559107 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa05402c5d0 con 0x5591f15fc290 2024-04-21T08:45:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.099+0000 7fa042ffd640 1 -- 172.21.15.43:0/627559107 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05402cad0 con 0x5591f15fc290 2024-04-21T08:45:34.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.099+0000 7fa0623e7740 1 -- 172.21.15.43:0/627559107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fc290 msgr2=0x5591f1610be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.100+0000 7fa0623e7740 1 --2- 172.21.15.43:0/627559107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fc290 0x5591f1610be0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x5591f15fb3b0 tx=0x7fa054000bc0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.100+0000 7fa0623e7740 1 -- 172.21.15.43:0/627559107 shutdown_connections 2024-04-21T08:45:34.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.100+0000 7fa0623e7740 1 --2- 172.21.15.43:0/627559107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591f1604e10 0x5591f15f7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.100+0000 7fa0623e7740 1 --2- 172.21.15.43:0/627559107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fc290 0x5591f1610be0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.100+0000 7fa0623e7740 1 --2- 172.21.15.43:0/627559107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591f15fb8e0 0x5591f1604530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.100+0000 7fa0623e7740 1 -- 172.21.15.43:0/627559107 >> 172.21.15.43:0/627559107 conn(0x5591f15f5fb0 msgr2=0x5591f15f7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.100+0000 7fa0623e7740 1 -- 172.21.15.43:0/627559107 shutdown_connections 2024-04-21T08:45:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.100+0000 7fa0623e7740 1 -- 172.21.15.43:0/627559107 wait complete. 2024-04-21T08:45:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.100+0000 7fa0623e7740 1 -- start start 2024-04-21T08:45:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.100+0000 7fa0623e7740 1 Processor -- start 2024-04-21T08:45:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.100+0000 7fa0623e7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fb8e0 0x5591f15feb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.100+0000 7fa0623e7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591f15fc290 0x5591f15ff0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.100+0000 7fa0623e7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591f1604e10 0x5591f15ff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.100+0000 7fa0623e7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa0581b0860 con 0x5591f15fb8e0 2024-04-21T08:45:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.100+0000 7fa0623e7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa0581b0190 con 0x5591f15fc290 2024-04-21T08:45:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.100+0000 7fa0623e7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa0581ad3d0 con 0x5591f1604e10 2024-04-21T08:45:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.100+0000 7fa0623e7740 10 client.0 _ll_register_callbacks cb 0x7fa048004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.100+0000 7fa05cd8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fb8e0 0x5591f15feb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.101+0000 7fa05cd8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fb8e0 0x5591f15feb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39116/0 (socket says 172.21.15.43:39116) 2024-04-21T08:45:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.101+0000 7fa05cd8f640 1 -- 172.21.15.43:0/4143577834 learned_addr learned my addr 172.21.15.43:0/4143577834 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.101+0000 7fa05cd8f640 1 --2- 172.21.15.43:0/4143577834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591f1604e10 0x5591f15ff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.101+0000 7fa05d590640 1 --2- 172.21.15.43:0/4143577834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591f15fc290 0x5591f15ff0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.101+0000 7fa05cd8f640 1 -- 172.21.15.43:0/4143577834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591f1604e10 msgr2=0x5591f15ff5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.101+0000 7fa05cd8f640 1 --2- 172.21.15.43:0/4143577834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591f1604e10 0x5591f15ff5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.101+0000 7fa05cd8f640 1 -- 172.21.15.43:0/4143577834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591f15fc290 msgr2=0x5591f15ff0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.101+0000 7fa05cd8f640 1 --2- 172.21.15.43:0/4143577834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591f15fc290 0x5591f15ff0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.101+0000 7fa05cd8f640 1 -- 172.21.15.43:0/4143577834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591f167a9a0 con 0x5591f15fb8e0 2024-04-21T08:45:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.101+0000 7fa05d590640 1 --2- 172.21.15.43:0/4143577834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591f15fc290 0x5591f15ff0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.101+0000 7fa05cd8f640 1 --2- 172.21.15.43:0/4143577834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fb8e0 0x5591f15feb70 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fa048001eb0 tx=0x7fa048011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac04b0dc1fde713b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.101+0000 7fa040ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.101+0000 7fa040ff9640 1 -- 172.21.15.43:0/4143577834 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048035060 con 0x5591f15fb8e0 2024-04-21T08:45:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.101+0000 7fa0623e7740 1 -- 172.21.15.43:0/4143577834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa05402c5d0 con 0x5591f15fb8e0 2024-04-21T08:45:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.101+0000 7fa040ff9640 1 -- 172.21.15.43:0/4143577834 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fa04800fce0 con 0x5591f15fb8e0 2024-04-21T08:45:34.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.101+0000 7fa040ff9640 1 -- 172.21.15.43:0/4143577834 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa04803c050 con 0x5591f15fb8e0 2024-04-21T08:45:34.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.101+0000 7fa0623e7740 10 client.18531 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:34.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.101+0000 7fa0623e7740 1 -- 172.21.15.43:0/4143577834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5591f167bcc0 con 0x5591f15fb8e0 2024-04-21T08:45:34.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.102+0000 7fa040ff9640 1 -- 172.21.15.43:0/4143577834 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa05402c5d0 con 0x5591f15fb8e0 2024-04-21T08:45:34.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.102+0000 7fa040ff9640 1 -- 172.21.15.43:0/4143577834 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa04801d030 con 0x5591f15fb8e0 2024-04-21T08:45:34.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.102+0000 7fa0623e7740 10 client.18531 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:34.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.102+0000 7fa0623e7740 10 client.18531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:34.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.102+0000 7fa0623e7740 20 client.18531 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:34.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.102+0000 7fa0623e7740 1 --2- 172.21.15.43:0/4143577834 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5591f167c470 0x5591f167e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.102+0000 7fa0623e7740 4 client.18531 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:45:34.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.102+0000 7fa0623e7740 1 -- 172.21.15.43:0/4143577834 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5591f1567790 con 0x5591f167c470 2024-04-21T08:45:34.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.102+0000 7fa05d590640 1 --2- 172.21.15.43:0/4143577834 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5591f167c470 0x5591f167e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.102+0000 7fa040ff9640 10 client.18531 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:34.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.102+0000 7fa05d590640 1 --2- 172.21.15.43:0/4143577834 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5591f167c470 0x5591f167e8b0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a906b1eec8ff3502 server_cookie=47075cc74c9bf5fa in_seq=0 out_seq=0 2024-04-21T08:45:34.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.104+0000 7fa040ff9640 1 -- 172.21.15.43:0/4143577834 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5591f1567790 con 0x5591f167c470 2024-04-21T08:45:34.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.104+0000 7fa040ff9640 10 client.18531 handle_command_reply: tid=0 2024-04-21T08:45:34.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.104+0000 7fa0623e7740 1 client.18531 shutdown 2024-04-21T08:45:34.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.104+0000 7fa0623e7740 1 -- 172.21.15.43:0/4143577834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fb8e0 msgr2=0x5591f15feb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.104+0000 7fa0623e7740 1 --2- 172.21.15.43:0/4143577834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fb8e0 0x5591f15feb70 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fa048001eb0 tx=0x7fa048011440 comp rx=0 tx=0).stop 2024-04-21T08:45:34.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.104+0000 7fa0623e7740 1 -- 172.21.15.43:0/4143577834 shutdown_connections 2024-04-21T08:45:34.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.104+0000 7fa0623e7740 1 --2- 172.21.15.43:0/4143577834 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5591f167c470 0x5591f167e8b0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.104+0000 7fa0623e7740 1 --2- 172.21.15.43:0/4143577834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591f1604e10 0x5591f15ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.104+0000 7fa0623e7740 1 --2- 172.21.15.43:0/4143577834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591f15fc290 0x5591f15ff0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.104+0000 7fa0623e7740 1 --2- 172.21.15.43:0/4143577834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591f15fb8e0 0x5591f15feb70 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.104+0000 7fa0623e7740 1 -- 172.21.15.43:0/4143577834 >> 172.21.15.43:0/4143577834 conn(0x5591f15f5fb0 msgr2=0x5591f15f7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.104+0000 7fa040ff9640 0 client.18531 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:34.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.104+0000 7fa0623e7740 1 -- 172.21.15.43:0/4143577834 shutdown_connections 2024-04-21T08:45:34.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.104+0000 7fa0623e7740 1 -- 172.21.15.43:0/4143577834 wait complete. 2024-04-21T08:45:34.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.104+0000 7fa0623e7740 20 client.18531 trim_cache size 0 max 16384 2024-04-21T08:45:34.109 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:45:34.109 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:45:34.109 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "23c1ff26-ff3a-4939-87a3-2266010f5c23", 2024-04-21T08:45:34.109 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:45:34.109 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:45:34.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.105+0000 7fa05f7f4640 1 -- 172.21.15.43:0/2621523793 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa0440a3d20 msgr2=0x7fa0440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.105+0000 7fa05f7f4640 1 --2- 172.21.15.43:0/2621523793 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa0440a3d20 0x7fa0440a61e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa0500991b0 tx=0x7fa05006b700 comp rx=0 tx=0).stop 2024-04-21T08:45:34.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.105+0000 7fa05f7f4640 1 -- 172.21.15.43:0/2621523793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa05819cff0 msgr2=0x7fa05819d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.105+0000 7fa05f7f4640 1 --2- 172.21.15.43:0/2621523793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa05819cff0 0x7fa05819d4a0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fa05000eb10 tx=0x7fa05000c2d0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.105+0000 7fa05f7f4640 1 -- 172.21.15.43:0/2621523793 shutdown_connections 2024-04-21T08:45:34.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.105+0000 7fa05f7f4640 1 --2- 172.21.15.43:0/2621523793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa05819cff0 0x7fa05819d4a0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.105+0000 7fa05f7f4640 1 --2- 172.21.15.43:0/2621523793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa058106170 0x7fa058198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.105+0000 7fa05f7f4640 1 --2- 172.21.15.43:0/2621523793 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa0440a3d20 0x7fa0440a61e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.105+0000 7fa05f7f4640 1 --2- 172.21.15.43:0/2621523793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0581010a0 0x7fa058198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.105+0000 7fa05f7f4640 1 -- 172.21.15.43:0/2621523793 >> 172.21.15.43:0/2621523793 conn(0x7fa058077eb0 msgr2=0x7fa05806a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.105+0000 7fa05f7f4640 1 -- 172.21.15.43:0/2621523793 shutdown_connections 2024-04-21T08:45:34.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.105+0000 7fa05f7f4640 1 -- 172.21.15.43:0/2621523793 wait complete. 2024-04-21T08:45:34.115 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:45:34.115 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:45:34.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.221+0000 7fe950077640 1 -- 172.21.15.43:0/3048965790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe94806af70 msgr2=0x7fe94810ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.221+0000 7fe950077640 1 --2- 172.21.15.43:0/3048965790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe94806af70 0x7fe94810ede0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fe940009cd0 tx=0x7fe94002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.221+0000 7fe950077640 1 -- 172.21.15.43:0/3048965790 shutdown_connections 2024-04-21T08:45:34.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.221+0000 7fe950077640 1 --2- 172.21.15.43:0/3048965790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe94806af70 0x7fe94810ede0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.221+0000 7fe950077640 1 --2- 172.21.15.43:0/3048965790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe94806a5b0 0x7fe94806aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.221+0000 7fe950077640 1 --2- 172.21.15.43:0/3048965790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe948108060 0x7fe948108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.221+0000 7fe950077640 1 -- 172.21.15.43:0/3048965790 >> 172.21.15.43:0/3048965790 conn(0x7fe948077c40 msgr2=0x7fe948078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.221+0000 7fe950077640 1 -- 172.21.15.43:0/3048965790 shutdown_connections 2024-04-21T08:45:34.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.221+0000 7fe950077640 1 -- 172.21.15.43:0/3048965790 wait complete. 2024-04-21T08:45:34.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.221+0000 7fe950077640 1 Processor -- start 2024-04-21T08:45:34.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.221+0000 7fe950077640 1 -- start start 2024-04-21T08:45:34.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.221+0000 7fe950077640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe94806a5b0 0x7fe9481984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.221+0000 7fe950077640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe94806af70 0x7fe948198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.221+0000 7fe950077640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe948108060 0x7fe94819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.221+0000 7fe950077640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe948112180 con 0x7fe94806a5b0 2024-04-21T08:45:34.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.222+0000 7fe950077640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe948112000 con 0x7fe94806af70 2024-04-21T08:45:34.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.222+0000 7fe950077640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe948112300 con 0x7fe948108060 2024-04-21T08:45:34.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.222+0000 7fe94de13640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe94806a5b0 0x7fe9481984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.222+0000 7fe94de13640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe94806a5b0 0x7fe9481984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39134/0 (socket says 172.21.15.43:39134) 2024-04-21T08:45:34.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.222+0000 7fe94de13640 1 -- 172.21.15.43:0/4233122646 learned_addr learned my addr 172.21.15.43:0/4233122646 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:34.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.222+0000 7fe94e614640 1 --2- 172.21.15.43:0/4233122646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe948108060 0x7fe94819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.222+0000 7fe94d612640 1 --2- 172.21.15.43:0/4233122646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe94806af70 0x7fe948198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.222+0000 7fe94de13640 1 -- 172.21.15.43:0/4233122646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe948108060 msgr2=0x7fe94819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.222+0000 7fe94de13640 1 --2- 172.21.15.43:0/4233122646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe948108060 0x7fe94819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.222+0000 7fe94de13640 1 -- 172.21.15.43:0/4233122646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe94806af70 msgr2=0x7fe948198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.222+0000 7fe94de13640 1 --2- 172.21.15.43:0/4233122646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe94806af70 0x7fe948198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.222+0000 7fe94de13640 1 -- 172.21.15.43:0/4233122646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe94819d4d0 con 0x7fe94806a5b0 2024-04-21T08:45:34.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.222+0000 7fe94de13640 1 --2- 172.21.15.43:0/4233122646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe94806a5b0 0x7fe9481984e0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fe94400c920 tx=0x7fe94400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.222+0000 7fe93effd640 1 -- 172.21.15.43:0/4233122646 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944007c70 con 0x7fe94806a5b0 2024-04-21T08:45:34.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.222+0000 7fe93effd640 1 -- 172.21.15.43:0/4233122646 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe944007e10 con 0x7fe94806a5b0 2024-04-21T08:45:34.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.222+0000 7fe950077640 1 -- 172.21.15.43:0/4233122646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe94819d7c0 con 0x7fe94806a5b0 2024-04-21T08:45:34.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.222+0000 7fe93effd640 1 -- 172.21.15.43:0/4233122646 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944018820 con 0x7fe94806a5b0 2024-04-21T08:45:34.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.222+0000 7fe950077640 1 -- 172.21.15.43:0/4233122646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9481a5050 con 0x7fe94806a5b0 2024-04-21T08:45:34.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.225+0000 7fe93effd640 1 -- 172.21.15.43:0/4233122646 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe94400f450 con 0x7fe94806a5b0 2024-04-21T08:45:34.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.225+0000 7fe952c61740 1 Processor -- start 2024-04-21T08:45:34.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.225+0000 7fe952c61740 1 -- start start 2024-04-21T08:45:34.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.225+0000 7fe952c61740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72b2b0 0x55ddee72b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.225+0000 7fe952c61740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddee72bbd0 0x55ddee738950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.225+0000 7fe952c61740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddee738e90 0x55ddee73b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.225+0000 7fe952c61740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ddee6901e0 con 0x55ddee72b2b0 2024-04-21T08:45:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.225+0000 7fe952c61740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ddee62d390 con 0x55ddee72bbd0 2024-04-21T08:45:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.225+0000 7fe952c61740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ddee68c1a0 con 0x55ddee738e90 2024-04-21T08:45:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.225+0000 7fe94e614640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72b2b0 0x55ddee72b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.225+0000 7fe94e614640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72b2b0 0x55ddee72b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39136/0 (socket says 172.21.15.43:39136) 2024-04-21T08:45:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.225+0000 7fe94e614640 1 -- 172.21.15.43:0/3601871321 learned_addr learned my addr 172.21.15.43:0/3601871321 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.225+0000 7fe93effd640 1 --2- 172.21.15.43:0/4233122646 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe9340a3da0 0x7fe9340a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.226+0000 7fe94de13640 1 --2- 172.21.15.43:0/3601871321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddee72bbd0 0x55ddee738950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.226+0000 7fe94d612640 1 --2- 172.21.15.43:0/3601871321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddee738e90 0x55ddee73b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.226+0000 7fe94e614640 1 --2- 172.21.15.43:0/4233122646 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe9340a3da0 0x7fe9340a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.226+0000 7fe93effd640 1 -- 172.21.15.43:0/4233122646 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fe9440d0d50 con 0x7fe94806a5b0 2024-04-21T08:45:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.226+0000 7fe94e614640 1 -- 172.21.15.43:0/3601871321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddee738e90 msgr2=0x55ddee73b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.226+0000 7fe94e614640 1 --2- 172.21.15.43:0/3601871321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddee738e90 0x55ddee73b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.226+0000 7fe94e614640 1 -- 172.21.15.43:0/3601871321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddee72bbd0 msgr2=0x55ddee738950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.226+0000 7fe94e614640 1 --2- 172.21.15.43:0/3601871321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddee72bbd0 0x55ddee738950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.226+0000 7fe94e614640 1 -- 172.21.15.43:0/3601871321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddee46c7d0 con 0x55ddee72b2b0 2024-04-21T08:45:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.226+0000 7fe94de13640 1 --2- 172.21.15.43:0/3601871321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddee72bbd0 0x55ddee738950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.226+0000 7fe94e614640 1 --2- 172.21.15.43:0/3601871321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72b2b0 0x55ddee72b690 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fe940009430 tx=0x7fe940032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28fc4b2cf367760 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.226+0000 7fe933fff640 1 -- 172.21.15.43:0/3601871321 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe940039040 con 0x55ddee72b2b0 2024-04-21T08:45:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.226+0000 7fe933fff640 1 -- 172.21.15.43:0/3601871321 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe940003b70 con 0x55ddee72b2b0 2024-04-21T08:45:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.226+0000 7fe933fff640 1 -- 172.21.15.43:0/3601871321 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe940034440 con 0x55ddee72b2b0 2024-04-21T08:45:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.227+0000 7fe952c61740 1 -- 172.21.15.43:0/3601871321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72b2b0 msgr2=0x55ddee72b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.227+0000 7fe94e614640 1 --2- 172.21.15.43:0/4233122646 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe9340a3da0 0x7fe9340a6260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe940049000 tx=0x7fe940048000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.227+0000 7fe952c61740 1 --2- 172.21.15.43:0/3601871321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72b2b0 0x55ddee72b690 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fe940009430 tx=0x7fe940032cd0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.227+0000 7fe952c61740 1 -- 172.21.15.43:0/3601871321 shutdown_connections 2024-04-21T08:45:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.227+0000 7fe952c61740 1 --2- 172.21.15.43:0/3601871321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddee738e90 0x55ddee73b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.227+0000 7fe952c61740 1 --2- 172.21.15.43:0/3601871321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddee72bbd0 0x55ddee738950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.227+0000 7fe952c61740 1 --2- 172.21.15.43:0/3601871321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72b2b0 0x55ddee72b690 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.227+0000 7fe952c61740 1 -- 172.21.15.43:0/3601871321 >> 172.21.15.43:0/3601871321 conn(0x55ddee725cb0 msgr2=0x55ddee726090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.227+0000 7fe952c61740 1 -- 172.21.15.43:0/3601871321 shutdown_connections 2024-04-21T08:45:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.227+0000 7fe952c61740 1 -- 172.21.15.43:0/3601871321 wait complete. 2024-04-21T08:45:34.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.227+0000 7fe952c61740 1 -- start start 2024-04-21T08:45:34.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.228+0000 7fe952c61740 1 Processor -- start 2024-04-21T08:45:34.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.228+0000 7fe952c61740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddee72b2b0 0x55ddee735ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.228+0000 7fe952c61740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72bbd0 0x55ddee736430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.228+0000 7fe952c61740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddee738e90 0x55ddee736970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.228+0000 7fe952c61740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ddee73bc50 con 0x55ddee72bbd0 2024-04-21T08:45:34.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.228+0000 7fe952c61740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ddee6492f0 con 0x55ddee72b2b0 2024-04-21T08:45:34.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.228+0000 7fe952c61740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ddee73bdd0 con 0x55ddee738e90 2024-04-21T08:45:34.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.228+0000 7fe952c61740 10 client.0 _ll_register_callbacks cb 0x55ddee656580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:34.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.228+0000 7fe94d612640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72bbd0 0x55ddee736430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.228+0000 7fe94d612640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72bbd0 0x55ddee736430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39144/0 (socket says 172.21.15.43:39144) 2024-04-21T08:45:34.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.228+0000 7fe94d612640 1 -- 172.21.15.43:0/2883750481 learned_addr learned my addr 172.21.15.43:0/2883750481 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:34.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.228+0000 7fe94de13640 1 --2- 172.21.15.43:0/2883750481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddee72b2b0 0x55ddee735ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.228+0000 7fe94e614640 1 --2- 172.21.15.43:0/2883750481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddee738e90 0x55ddee736970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.228+0000 7fe94d612640 1 -- 172.21.15.43:0/2883750481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddee738e90 msgr2=0x55ddee736970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.228+0000 7fe94d612640 1 --2- 172.21.15.43:0/2883750481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddee738e90 0x55ddee736970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.228+0000 7fe94d612640 1 -- 172.21.15.43:0/2883750481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddee72b2b0 msgr2=0x55ddee735ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.228+0000 7fe94d612640 1 --2- 172.21.15.43:0/2883750481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddee72b2b0 0x55ddee735ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.228+0000 7fe94d612640 1 -- 172.21.15.43:0/2883750481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddee3a23f0 con 0x55ddee72bbd0 2024-04-21T08:45:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.229+0000 7fe94d612640 1 --2- 172.21.15.43:0/2883750481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72bbd0 0x55ddee736430 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fe938004650 tx=0x7fe9380039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea77203ebfeaa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.229+0000 7fe9317fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.229+0000 7fe9317fa640 1 -- 172.21.15.43:0/2883750481 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938011910 con 0x55ddee72bbd0 2024-04-21T08:45:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.229+0000 7fe9317fa640 1 -- 172.21.15.43:0/2883750481 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fe938011ab0 con 0x55ddee72bbd0 2024-04-21T08:45:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.229+0000 7fe952c61740 1 -- 172.21.15.43:0/2883750481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe940003b70 con 0x55ddee72bbd0 2024-04-21T08:45:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.229+0000 7fe9317fa640 1 -- 172.21.15.43:0/2883750481 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe938011cb0 con 0x55ddee72bbd0 2024-04-21T08:45:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.229+0000 7fe952c61740 10 client.18543 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.229+0000 7fe952c61740 1 -- 172.21.15.43:0/2883750481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ddee3620f0 con 0x55ddee72bbd0 2024-04-21T08:45:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.229+0000 7fe9317fa640 1 -- 172.21.15.43:0/2883750481 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe940003b70 con 0x55ddee72bbd0 2024-04-21T08:45:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.229+0000 7fe9317fa640 1 -- 172.21.15.43:0/2883750481 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe938018070 con 0x55ddee72bbd0 2024-04-21T08:45:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.229+0000 7fe952c61740 10 client.18543 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.229+0000 7fe952c61740 10 client.18543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.229+0000 7fe952c61740 20 client.18543 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.229+0000 7fe952c61740 1 --2- 172.21.15.43:0/2883750481 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ddee748e90 0x55ddee74b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.229+0000 7fe952c61740 4 client.18543 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.229+0000 7fe952c61740 1 -- 172.21.15.43:0/2883750481 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddee69dc00 con 0x55ddee748e90 2024-04-21T08:45:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.230+0000 7fe94de13640 1 --2- 172.21.15.43:0/2883750481 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ddee748e90 0x55ddee74b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.230+0000 7fe9317fa640 10 client.18543 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.230+0000 7fe94de13640 1 --2- 172.21.15.43:0/2883750481 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ddee748e90 0x55ddee74b350 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c178df0fc718bfec server_cookie=d0ef246244c252c7 in_seq=0 out_seq=0 2024-04-21T08:45:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.231+0000 7fe9317fa640 1 -- 172.21.15.43:0/2883750481 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ddee69dc00 con 0x55ddee748e90 2024-04-21T08:45:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.231+0000 7fe9317fa640 10 client.18543 handle_command_reply: tid=0 2024-04-21T08:45:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.231+0000 7fe952c61740 1 client.18543 shutdown 2024-04-21T08:45:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.231+0000 7fe952c61740 1 -- 172.21.15.43:0/2883750481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72bbd0 msgr2=0x55ddee736430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.231+0000 7fe952c61740 1 --2- 172.21.15.43:0/2883750481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72bbd0 0x55ddee736430 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fe938004650 tx=0x7fe9380039c0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.231+0000 7fe952c61740 1 -- 172.21.15.43:0/2883750481 shutdown_connections 2024-04-21T08:45:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.231+0000 7fe952c61740 1 --2- 172.21.15.43:0/2883750481 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ddee748e90 0x55ddee74b350 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.231+0000 7fe952c61740 1 --2- 172.21.15.43:0/2883750481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddee738e90 0x55ddee736970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.231+0000 7fe952c61740 1 --2- 172.21.15.43:0/2883750481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72bbd0 0x55ddee736430 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.231+0000 7fe9317fa640 0 client.18543 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.231+0000 7fe952c61740 1 --2- 172.21.15.43:0/2883750481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddee72b2b0 0x55ddee735ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.231+0000 7fe952c61740 1 -- 172.21.15.43:0/2883750481 >> 172.21.15.43:0/2883750481 conn(0x55ddee725cb0 msgr2=0x55ddee727a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.231+0000 7fe952c61740 1 -- 172.21.15.43:0/2883750481 shutdown_connections 2024-04-21T08:45:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.231+0000 7fe952c61740 1 -- 172.21.15.43:0/2883750481 wait complete. 2024-04-21T08:45:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.231+0000 7fe952c61740 20 client.18543 trim_cache size 0 max 16384 2024-04-21T08:45:34.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.248+0000 7fe952c61740 1 Processor -- start 2024-04-21T08:45:34.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.248+0000 7fe952c61740 1 -- start start 2024-04-21T08:45:34.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.249+0000 7fe952c61740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddee72b2b0 0x55ddee738620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.249+0000 7fe952c61740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72bbd0 0x55ddee734e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.249+0000 7fe952c61740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddee738e90 0x55ddee744b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.249+0000 7fe952c61740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe9481ad0d0 con 0x55ddee72bbd0 2024-04-21T08:45:34.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.249+0000 7fe952c61740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe9481aec10 con 0x55ddee738e90 2024-04-21T08:45:34.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.249+0000 7fe952c61740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9481ad7a0 con 0x55ddee72b2b0 2024-04-21T08:45:34.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.249+0000 7fe94d612640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72bbd0 0x55ddee734e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.249+0000 7fe94d612640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72bbd0 0x55ddee734e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39152/0 (socket says 172.21.15.43:39152) 2024-04-21T08:45:34.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.249+0000 7fe94d612640 1 -- 172.21.15.43:0/3328776765 learned_addr learned my addr 172.21.15.43:0/3328776765 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:34.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.249+0000 7fe94d612640 1 -- 172.21.15.43:0/3328776765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddee72b2b0 msgr2=0x55ddee738620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.249+0000 7fe94d612640 1 --2- 172.21.15.43:0/3328776765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddee72b2b0 0x55ddee738620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.249+0000 7fe94d612640 1 -- 172.21.15.43:0/3328776765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddee738e90 msgr2=0x55ddee744b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.249+0000 7fe94d612640 1 --2- 172.21.15.43:0/3328776765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddee738e90 0x55ddee744b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.249+0000 7fe94d612640 1 -- 172.21.15.43:0/3328776765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddee72ba00 con 0x55ddee72bbd0 2024-04-21T08:45:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.249+0000 7fe94d612640 1 --2- 172.21.15.43:0/3328776765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72bbd0 0x55ddee734e30 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fe9380058c0 tx=0x7fe9380040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e39ccfef0c2657 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.249+0000 7fe9337fe640 1 -- 172.21.15.43:0/3328776765 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938011910 con 0x55ddee72bbd0 2024-04-21T08:45:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.249+0000 7fe9337fe640 1 -- 172.21.15.43:0/3328776765 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe93802d070 con 0x55ddee72bbd0 2024-04-21T08:45:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.249+0000 7fe9337fe640 1 -- 172.21.15.43:0/3328776765 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938019b80 con 0x55ddee72bbd0 2024-04-21T08:45:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.250+0000 7fe952c61740 1 -- 172.21.15.43:0/3328776765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72bbd0 msgr2=0x55ddee734e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.250+0000 7fe952c61740 1 --2- 172.21.15.43:0/3328776765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72bbd0 0x55ddee734e30 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fe9380058c0 tx=0x7fe9380040b0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.250+0000 7fe952c61740 1 -- 172.21.15.43:0/3328776765 shutdown_connections 2024-04-21T08:45:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.250+0000 7fe952c61740 1 --2- 172.21.15.43:0/3328776765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddee738e90 0x55ddee744b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.250+0000 7fe952c61740 1 --2- 172.21.15.43:0/3328776765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72bbd0 0x55ddee734e30 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.250+0000 7fe952c61740 1 --2- 172.21.15.43:0/3328776765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddee72b2b0 0x55ddee738620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.250+0000 7fe952c61740 1 -- 172.21.15.43:0/3328776765 >> 172.21.15.43:0/3328776765 conn(0x55ddee725cb0 msgr2=0x55ddee727970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.250+0000 7fe952c61740 1 -- 172.21.15.43:0/3328776765 shutdown_connections 2024-04-21T08:45:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.250+0000 7fe952c61740 1 -- 172.21.15.43:0/3328776765 wait complete. 2024-04-21T08:45:34.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.250+0000 7fe952c61740 1 -- start start 2024-04-21T08:45:34.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.250+0000 7fe952c61740 1 Processor -- start 2024-04-21T08:45:34.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.251+0000 7fe952c61740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72b2b0 0x55ddee73ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.251+0000 7fe952c61740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddee72bbd0 0x55ddee73d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.251+0000 7fe952c61740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddee738e90 0x55ddee7ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.251+0000 7fe952c61740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe9481ae540 con 0x55ddee72b2b0 2024-04-21T08:45:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.251+0000 7fe952c61740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe9481ade70 con 0x55ddee738e90 2024-04-21T08:45:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.251+0000 7fe952c61740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9481ab0b0 con 0x55ddee72bbd0 2024-04-21T08:45:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.251+0000 7fe952c61740 10 client.0 _ll_register_callbacks cb 0x7fe938004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.251+0000 7fe94de13640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72b2b0 0x55ddee73ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.251+0000 7fe94de13640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72b2b0 0x55ddee73ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39166/0 (socket says 172.21.15.43:39166) 2024-04-21T08:45:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.251+0000 7fe94de13640 1 -- 172.21.15.43:0/1645722898 learned_addr learned my addr 172.21.15.43:0/1645722898 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.251+0000 7fe94d612640 1 --2- 172.21.15.43:0/1645722898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddee72bbd0 0x55ddee73d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.251+0000 7fe94e614640 1 --2- 172.21.15.43:0/1645722898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddee738e90 0x55ddee7ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.251+0000 7fe94de13640 1 -- 172.21.15.43:0/1645722898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddee72bbd0 msgr2=0x55ddee73d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.251+0000 7fe94de13640 1 --2- 172.21.15.43:0/1645722898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddee72bbd0 0x55ddee73d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.251+0000 7fe94de13640 1 -- 172.21.15.43:0/1645722898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddee738e90 msgr2=0x55ddee7ae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.251+0000 7fe94de13640 1 --2- 172.21.15.43:0/1645722898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddee738e90 0x55ddee7ae4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.251+0000 7fe94de13640 1 -- 172.21.15.43:0/1645722898 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddee7aea10 con 0x55ddee72b2b0 2024-04-21T08:45:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.251+0000 7fe94e614640 1 --2- 172.21.15.43:0/1645722898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddee738e90 0x55ddee7ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.251+0000 7fe94de13640 1 --2- 172.21.15.43:0/1645722898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72b2b0 0x55ddee73ce30 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fe944007590 tx=0x7fe944015aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73d8a23f57e2a26 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.251+0000 7fe9317fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.251+0000 7fe9317fa640 1 -- 172.21.15.43:0/1645722898 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944085e60 con 0x55ddee72b2b0 2024-04-21T08:45:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.251+0000 7fe9317fa640 1 -- 172.21.15.43:0/1645722898 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fe94400fa90 con 0x55ddee72b2b0 2024-04-21T08:45:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.251+0000 7fe952c61740 1 -- 172.21.15.43:0/1645722898 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe93802d070 con 0x55ddee72b2b0 2024-04-21T08:45:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.252+0000 7fe9317fa640 1 -- 172.21.15.43:0/1645722898 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe94409c920 con 0x55ddee72b2b0 2024-04-21T08:45:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.252+0000 7fe952c61740 10 client.18549 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.252+0000 7fe952c61740 1 -- 172.21.15.43:0/1645722898 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ddee7afbe0 con 0x55ddee72b2b0 2024-04-21T08:45:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.252+0000 7fe9317fa640 1 -- 172.21.15.43:0/1645722898 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe93802d070 con 0x55ddee72b2b0 2024-04-21T08:45:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.252+0000 7fe9317fa640 1 -- 172.21.15.43:0/1645722898 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe94400f740 con 0x55ddee72b2b0 2024-04-21T08:45:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.252+0000 7fe952c61740 10 client.18549 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.252+0000 7fe952c61740 10 client.18549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.252+0000 7fe952c61740 20 client.18549 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.252+0000 7fe952c61740 1 --2- 172.21.15.43:0/1645722898 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ddee7b02b0 0x55ddee7b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.252+0000 7fe952c61740 4 client.18549 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:45:34.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.252+0000 7fe952c61740 1 -- 172.21.15.43:0/1645722898 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ddee697a70 con 0x55ddee7b02b0 2024-04-21T08:45:34.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.252+0000 7fe94d612640 1 --2- 172.21.15.43:0/1645722898 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ddee7b02b0 0x55ddee7b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.253+0000 7fe94d612640 1 --2- 172.21.15.43:0/1645722898 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ddee7b02b0 0x55ddee7b26d0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e334dfadbbc4bc6 server_cookie=bb07110e2dc826ba in_seq=0 out_seq=0 2024-04-21T08:45:34.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.253+0000 7fe9317fa640 10 client.18549 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:34.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.253+0000 7fe9317fa640 1 -- 172.21.15.43:0/1645722898 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ddee697a70 con 0x55ddee7b02b0 2024-04-21T08:45:34.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.253+0000 7fe9317fa640 10 client.18549 handle_command_reply: tid=0 2024-04-21T08:45:34.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.253+0000 7fe952c61740 1 client.18549 shutdown 2024-04-21T08:45:34.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.253+0000 7fe952c61740 1 -- 172.21.15.43:0/1645722898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72b2b0 msgr2=0x55ddee73ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.253+0000 7fe952c61740 1 --2- 172.21.15.43:0/1645722898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72b2b0 0x55ddee73ce30 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fe944007590 tx=0x7fe944015aa0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.253+0000 7fe952c61740 1 -- 172.21.15.43:0/1645722898 shutdown_connections 2024-04-21T08:45:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.253+0000 7fe952c61740 1 --2- 172.21.15.43:0/1645722898 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ddee7b02b0 0x55ddee7b26d0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.253+0000 7fe952c61740 1 --2- 172.21.15.43:0/1645722898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddee738e90 0x55ddee7ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.253+0000 7fe952c61740 1 --2- 172.21.15.43:0/1645722898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ddee72bbd0 0x55ddee73d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.253+0000 7fe952c61740 1 --2- 172.21.15.43:0/1645722898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddee72b2b0 0x55ddee73ce30 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.253+0000 7fe9317fa640 0 client.18549 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.253+0000 7fe952c61740 1 -- 172.21.15.43:0/1645722898 >> 172.21.15.43:0/1645722898 conn(0x55ddee725cb0 msgr2=0x55ddee7270e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.253+0000 7fe952c61740 1 -- 172.21.15.43:0/1645722898 shutdown_connections 2024-04-21T08:45:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.253+0000 7fe952c61740 1 -- 172.21.15.43:0/1645722898 wait complete. 2024-04-21T08:45:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.253+0000 7fe952c61740 20 client.18549 trim_cache size 0 max 16384 2024-04-21T08:45:34.260 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:45:34.260 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:45:34.260 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:45:34.260 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:45:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.254+0000 7fe950077640 1 -- 172.21.15.43:0/4233122646 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe9340a3da0 msgr2=0x7fe9340a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.254+0000 7fe950077640 1 --2- 172.21.15.43:0/4233122646 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe9340a3da0 0x7fe9340a6260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe940049000 tx=0x7fe940048000 comp rx=0 tx=0).stop 2024-04-21T08:45:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.254+0000 7fe950077640 1 -- 172.21.15.43:0/4233122646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe94806a5b0 msgr2=0x7fe9481984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.254+0000 7fe950077640 1 --2- 172.21.15.43:0/4233122646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe94806a5b0 0x7fe9481984e0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fe94400c920 tx=0x7fe94400cde0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.254+0000 7fe950077640 1 -- 172.21.15.43:0/4233122646 shutdown_connections 2024-04-21T08:45:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.254+0000 7fe950077640 1 --2- 172.21.15.43:0/4233122646 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe9340a3da0 0x7fe9340a6260 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.254+0000 7fe950077640 1 --2- 172.21.15.43:0/4233122646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe948108060 0x7fe94819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.254+0000 7fe950077640 1 --2- 172.21.15.43:0/4233122646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe94806af70 0x7fe948198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.254+0000 7fe950077640 1 --2- 172.21.15.43:0/4233122646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe94806a5b0 0x7fe9481984e0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.254+0000 7fe950077640 1 -- 172.21.15.43:0/4233122646 >> 172.21.15.43:0/4233122646 conn(0x7fe948077c40 msgr2=0x7fe9480ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.254+0000 7fe950077640 1 -- 172.21.15.43:0/4233122646 shutdown_connections 2024-04-21T08:45:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.254+0000 7fe950077640 1 -- 172.21.15.43:0/4233122646 wait complete. 2024-04-21T08:45:34.263 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:34 smithi043 ceph-mon[27978]: pgmap v202: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.9 KiB/s wr, 11 op/s 2024-04-21T08:45:34.264 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:45:34.264 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:45:34.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.366+0000 7fc13a1d5640 1 -- 172.21.15.43:0/3351185319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1341031d0 msgr2=0x7fc1341035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.366+0000 7fc13a1d5640 1 --2- 172.21.15.43:0/3351185319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1341031d0 0x7fc1341035b0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fc124009900 tx=0x7fc12402bcd0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.366+0000 7fc13a1d5640 1 -- 172.21.15.43:0/3351185319 shutdown_connections 2024-04-21T08:45:34.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.366+0000 7fc13a1d5640 1 --2- 172.21.15.43:0/3351185319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc13410c5a0 0x7fc13410e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.366+0000 7fc13a1d5640 1 --2- 172.21.15.43:0/3351185319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc134103af0 0x7fc13410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.366+0000 7fc13a1d5640 1 --2- 172.21.15.43:0/3351185319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1341031d0 0x7fc1341035b0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.366+0000 7fc13a1d5640 1 -- 172.21.15.43:0/3351185319 >> 172.21.15.43:0/3351185319 conn(0x7fc1340771d0 msgr2=0x7fc1340775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.367+0000 7fc13a1d5640 1 -- 172.21.15.43:0/3351185319 shutdown_connections 2024-04-21T08:45:34.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.367+0000 7fc13a1d5640 1 -- 172.21.15.43:0/3351185319 wait complete. 2024-04-21T08:45:34.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.367+0000 7fc13a1d5640 1 Processor -- start 2024-04-21T08:45:34.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.367+0000 7fc13a1d5640 1 -- start start 2024-04-21T08:45:34.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.368+0000 7fc13a1d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1341031d0 0x7fc13419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.368+0000 7fc13a1d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc134103af0 0x7fc13419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.368+0000 7fc13a1d5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc13410c5a0 0x7fc1341985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.368+0000 7fc13a1d5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc1341121f0 con 0x7fc134103af0 2024-04-21T08:45:34.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.368+0000 7fc13a1d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc134112070 con 0x7fc1341031d0 2024-04-21T08:45:34.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.368+0000 7fc13a1d5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc134112370 con 0x7fc13410c5a0 2024-04-21T08:45:34.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.368+0000 7fc132ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc134103af0 0x7fc13419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.368+0000 7fc132ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc134103af0 0x7fc13419e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39180/0 (socket says 172.21.15.43:39180) 2024-04-21T08:45:34.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.368+0000 7fc132ffd640 1 -- 172.21.15.43:0/2572065107 learned_addr learned my addr 172.21.15.43:0/2572065107 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:34.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.368+0000 7fc1337fe640 1 --2- 172.21.15.43:0/2572065107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1341031d0 0x7fc13419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.368+0000 7fc132ffd640 1 -- 172.21.15.43:0/2572065107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc13410c5a0 msgr2=0x7fc1341985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.368+0000 7fc132ffd640 1 --2- 172.21.15.43:0/2572065107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc13410c5a0 0x7fc1341985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.368+0000 7fc132ffd640 1 -- 172.21.15.43:0/2572065107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1341031d0 msgr2=0x7fc13419e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.368+0000 7fc132ffd640 1 --2- 172.21.15.43:0/2572065107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1341031d0 0x7fc13419e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.368+0000 7fc132ffd640 1 -- 172.21.15.43:0/2572065107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc134198d30 con 0x7fc134103af0 2024-04-21T08:45:34.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.368+0000 7fc132ffd640 1 --2- 172.21.15.43:0/2572065107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc134103af0 0x7fc13419e940 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fc12800d950 tx=0x7fc12800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.368+0000 7fc130ff9640 1 -- 172.21.15.43:0/2572065107 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc128017070 con 0x7fc134103af0 2024-04-21T08:45:34.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.368+0000 7fc13a1d5640 1 -- 172.21.15.43:0/2572065107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc134199020 con 0x7fc134103af0 2024-04-21T08:45:34.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.368+0000 7fc130ff9640 1 -- 172.21.15.43:0/2572065107 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc128009c30 con 0x7fc134103af0 2024-04-21T08:45:34.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.368+0000 7fc130ff9640 1 -- 172.21.15.43:0/2572065107 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12800f620 con 0x7fc134103af0 2024-04-21T08:45:34.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.368+0000 7fc13a1d5640 1 -- 172.21.15.43:0/2572065107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1341a52c0 con 0x7fc134103af0 2024-04-21T08:45:34.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.369+0000 7fc13a1d5640 1 -- 172.21.15.43:0/2572065107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc134069230 con 0x7fc134103af0 2024-04-21T08:45:34.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.371+0000 7fc130ff9640 1 -- 172.21.15.43:0/2572065107 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc128010460 con 0x7fc134103af0 2024-04-21T08:45:34.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.371+0000 7fc130ff9640 1 --2- 172.21.15.43:0/2572065107 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc11c0a3d20 0x7fc11c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.371+0000 7fc1337fe640 1 --2- 172.21.15.43:0/2572065107 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc11c0a3d20 0x7fc11c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.371+0000 7fc130ff9640 1 -- 172.21.15.43:0/2572065107 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc1280d0830 con 0x7fc134103af0 2024-04-21T08:45:34.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.372+0000 7fc1337fe640 1 --2- 172.21.15.43:0/2572065107 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc11c0a3d20 0x7fc11c0a61e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc124006010 tx=0x7fc124005e40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.376+0000 7fc130ff9640 1 -- 172.21.15.43:0/2572065107 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc12809cb00 con 0x7fc134103af0 2024-04-21T08:45:34.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:34 smithi175 ceph-mon[31920]: pgmap v202: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.9 KiB/s wr, 11 op/s 2024-04-21T08:45:34.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:34 smithi112 ceph-mon[31085]: pgmap v202: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.9 KiB/s wr, 11 op/s 2024-04-21T08:45:34.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.602+0000 7fc13a1d5640 1 -- 172.21.15.43:0/2572065107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc134107100 con 0x7fc134103af0 2024-04-21T08:45:34.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.603+0000 7fc130ff9640 1 -- 172.21.15.43:0/2572065107 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7fc128009de0 con 0x7fc134103af0 2024-04-21T08:45:34.603 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:45:34.603 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:45:34.603 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:45:34.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.605+0000 7fc13a1d5640 1 -- 172.21.15.43:0/2572065107 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc11c0a3d20 msgr2=0x7fc11c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.605+0000 7fc13a1d5640 1 --2- 172.21.15.43:0/2572065107 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc11c0a3d20 0x7fc11c0a61e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc124006010 tx=0x7fc124005e40 comp rx=0 tx=0).stop 2024-04-21T08:45:34.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.605+0000 7fc13a1d5640 1 -- 172.21.15.43:0/2572065107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc134103af0 msgr2=0x7fc13419e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.605+0000 7fc13a1d5640 1 --2- 172.21.15.43:0/2572065107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc134103af0 0x7fc13419e940 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fc12800d950 tx=0x7fc12800de10 comp rx=0 tx=0).stop 2024-04-21T08:45:34.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.605+0000 7fc13a1d5640 1 -- 172.21.15.43:0/2572065107 shutdown_connections 2024-04-21T08:45:34.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.605+0000 7fc13a1d5640 1 --2- 172.21.15.43:0/2572065107 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc11c0a3d20 0x7fc11c0a61e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.605+0000 7fc13a1d5640 1 --2- 172.21.15.43:0/2572065107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc13410c5a0 0x7fc1341985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.605+0000 7fc13a1d5640 1 --2- 172.21.15.43:0/2572065107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc134103af0 0x7fc13419e940 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.605+0000 7fc13a1d5640 1 --2- 172.21.15.43:0/2572065107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1341031d0 0x7fc13419e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.605+0000 7fc13a1d5640 1 -- 172.21.15.43:0/2572065107 >> 172.21.15.43:0/2572065107 conn(0x7fc1340771d0 msgr2=0x7fc1340ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.605+0000 7fc13a1d5640 1 -- 172.21.15.43:0/2572065107 shutdown_connections 2024-04-21T08:45:34.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.605+0000 7fc13a1d5640 1 -- 172.21.15.43:0/2572065107 wait complete. 2024-04-21T08:45:34.616 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:45:34.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.722+0000 7f5d6feba640 1 -- 172.21.15.43:0/1007723397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d68102b40 msgr2=0x7f5d68106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.722+0000 7f5d6feba640 1 --2- 172.21.15.43:0/1007723397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d68102b40 0x7f5d68106b80 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009900 tx=0x7f5d5802bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:34.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.722+0000 7f5d6feba640 1 -- 172.21.15.43:0/1007723397 shutdown_connections 2024-04-21T08:45:34.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.722+0000 7f5d6feba640 1 --2- 172.21.15.43:0/1007723397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d68107260 0x7f5d6810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.722+0000 7f5d6feba640 1 --2- 172.21.15.43:0/1007723397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d68102b40 0x7f5d68106b80 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.722+0000 7f5d6feba640 1 --2- 172.21.15.43:0/1007723397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d68102190 0x7f5d68102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.722+0000 7f5d6feba640 1 -- 172.21.15.43:0/1007723397 >> 172.21.15.43:0/1007723397 conn(0x7f5d680fd990 msgr2=0x7f5d680ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.722+0000 7f5d6feba640 1 -- 172.21.15.43:0/1007723397 shutdown_connections 2024-04-21T08:45:34.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.722+0000 7f5d6feba640 1 -- 172.21.15.43:0/1007723397 wait complete. 2024-04-21T08:45:34.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.722+0000 7f5d6feba640 1 Processor -- start 2024-04-21T08:45:34.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.723+0000 7f5d6feba640 1 -- start start 2024-04-21T08:45:34.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.723+0000 7f5d6feba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d68102190 0x7f5d6819e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.723+0000 7f5d6feba640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d68107260 0x7f5d6819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.723+0000 7f5d6feba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d681986b0 0x7f5d68198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.723+0000 7f5d6feba640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5d681121a0 con 0x7f5d681986b0 2024-04-21T08:45:34.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.723+0000 7f5d6feba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5d68112020 con 0x7f5d68102190 2024-04-21T08:45:34.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.723+0000 7f5d6feba640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5d68112320 con 0x7f5d68107260 2024-04-21T08:45:34.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.723+0000 7f5d6e457640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d681986b0 0x7f5d68198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.723+0000 7f5d6e457640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d681986b0 0x7f5d68198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39202/0 (socket says 172.21.15.43:39202) 2024-04-21T08:45:34.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.723+0000 7f5d6e457640 1 -- 172.21.15.43:0/1063399268 learned_addr learned my addr 172.21.15.43:0/1063399268 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:34.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.723+0000 7f5d6dc56640 1 --2- 172.21.15.43:0/1063399268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d68102190 0x7f5d6819e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.723+0000 7f5d6d455640 1 --2- 172.21.15.43:0/1063399268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d68107260 0x7f5d6819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.723+0000 7f5d6e457640 1 -- 172.21.15.43:0/1063399268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d68107260 msgr2=0x7f5d6819eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.723+0000 7f5d6e457640 1 --2- 172.21.15.43:0/1063399268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d68107260 0x7f5d6819eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.723+0000 7f5d6e457640 1 -- 172.21.15.43:0/1063399268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d68102190 msgr2=0x7f5d6819e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.723+0000 7f5d6e457640 1 --2- 172.21.15.43:0/1063399268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d68102190 0x7f5d6819e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.723+0000 7f5d6e457640 1 -- 172.21.15.43:0/1063399268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d68199290 con 0x7f5d681986b0 2024-04-21T08:45:34.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.723+0000 7f5d6e457640 1 --2- 172.21.15.43:0/1063399268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d681986b0 0x7f5d68198b60 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000eaa0 tx=0x7f5d6000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.724+0000 7f5d5effd640 1 -- 172.21.15.43:0/1063399268 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6000cdb0 con 0x7f5d681986b0 2024-04-21T08:45:34.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.724+0000 7f5d6feba640 1 -- 172.21.15.43:0/1063399268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d681a5370 con 0x7f5d681986b0 2024-04-21T08:45:34.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.724+0000 7f5d5effd640 1 -- 172.21.15.43:0/1063399268 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d60005020 con 0x7f5d681986b0 2024-04-21T08:45:34.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.724+0000 7f5d5effd640 1 -- 172.21.15.43:0/1063399268 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d600116d0 con 0x7f5d681986b0 2024-04-21T08:45:34.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.724+0000 7f5d6feba640 1 -- 172.21.15.43:0/1063399268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d681a5720 con 0x7f5d681986b0 2024-04-21T08:45:34.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.726+0000 7f5d5effd640 1 -- 172.21.15.43:0/1063399268 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5d600051d0 con 0x7f5d681986b0 2024-04-21T08:45:34.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.726+0000 7f5d72b87740 1 Processor -- start 2024-04-21T08:45:34.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.726+0000 7f5d72b87740 1 -- start start 2024-04-21T08:45:34.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.727+0000 7f5d72b87740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c46654d0 0x5594c46658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.727+0000 7f5d72b87740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5594c4665e80 0x5594c466e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.727+0000 7f5d72b87740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594c466ea00 0x5594c4670e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.727+0000 7f5d72b87740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5594c45ca1f0 con 0x5594c46654d0 2024-04-21T08:45:34.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.727+0000 7f5d72b87740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5594c45674b0 con 0x5594c466ea00 2024-04-21T08:45:34.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.727+0000 7f5d72b87740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5594c45c61b0 con 0x5594c4665e80 2024-04-21T08:45:34.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.727+0000 7f5d5effd640 1 --2- 172.21.15.43:0/1063399268 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5d540a3d20 0x7f5d540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.727+0000 7f5d6d455640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c46654d0 0x5594c46658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.727+0000 7f5d6d455640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c46654d0 0x5594c46658b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39218/0 (socket says 172.21.15.43:39218) 2024-04-21T08:45:34.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.727+0000 7f5d6d455640 1 -- 172.21.15.43:0/4015074383 learned_addr learned my addr 172.21.15.43:0/4015074383 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:34.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.727+0000 7f5d6dc56640 1 --2- 172.21.15.43:0/4015074383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5594c4665e80 0x5594c466e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.727+0000 7f5d6d455640 1 --2- 172.21.15.43:0/4015074383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594c466ea00 0x5594c4670e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.727+0000 7f5d6e457640 1 --2- 172.21.15.43:0/1063399268 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5d540a3d20 0x7f5d540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.727+0000 7f5d5effd640 1 -- 172.21.15.43:0/1063399268 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f5d600d0cf0 con 0x7f5d681986b0 2024-04-21T08:45:34.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.727+0000 7f5d6d455640 1 -- 172.21.15.43:0/4015074383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5594c4665e80 msgr2=0x5594c466e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.727+0000 7f5d6d455640 1 --2- 172.21.15.43:0/4015074383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5594c4665e80 0x5594c466e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.727+0000 7f5d6d455640 1 -- 172.21.15.43:0/4015074383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594c466ea00 msgr2=0x5594c4670e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.727+0000 7f5d6d455640 1 --2- 172.21.15.43:0/4015074383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594c466ea00 0x5594c4670e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.727+0000 7f5d6d455640 1 -- 172.21.15.43:0/4015074383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594c45591b0 con 0x5594c46654d0 2024-04-21T08:45:34.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.727+0000 7f5d6d455640 1 --2- 172.21.15.43:0/4015074383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c46654d0 0x5594c46658b0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f5d58005c40 tx=0x7f5d58005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42dab1053c05ed0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.728+0000 7f5d6e457640 1 --2- 172.21.15.43:0/1063399268 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5d540a3d20 0x7f5d540a61e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5d6009c560 tx=0x7f5d60087040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.728+0000 7f5d53fff640 1 -- 172.21.15.43:0/4015074383 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d58004480 con 0x5594c46654d0 2024-04-21T08:45:34.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.728+0000 7f5d53fff640 1 -- 172.21.15.43:0/4015074383 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d58048070 con 0x5594c46654d0 2024-04-21T08:45:34.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.728+0000 7f5d53fff640 1 -- 172.21.15.43:0/4015074383 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d58004920 con 0x5594c46654d0 2024-04-21T08:45:34.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.728+0000 7f5d72b87740 1 -- 172.21.15.43:0/4015074383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c46654d0 msgr2=0x5594c46658b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.728+0000 7f5d72b87740 1 --2- 172.21.15.43:0/4015074383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c46654d0 0x5594c46658b0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f5d58005c40 tx=0x7f5d58005b50 comp rx=0 tx=0).stop 2024-04-21T08:45:34.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.728+0000 7f5d72b87740 1 -- 172.21.15.43:0/4015074383 shutdown_connections 2024-04-21T08:45:34.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.728+0000 7f5d72b87740 1 --2- 172.21.15.43:0/4015074383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594c466ea00 0x5594c4670e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.728+0000 7f5d72b87740 1 --2- 172.21.15.43:0/4015074383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5594c4665e80 0x5594c466e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.728+0000 7f5d72b87740 1 --2- 172.21.15.43:0/4015074383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c46654d0 0x5594c46658b0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.728+0000 7f5d72b87740 1 -- 172.21.15.43:0/4015074383 >> 172.21.15.43:0/4015074383 conn(0x5594c465fba0 msgr2=0x5594c465ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.729+0000 7f5d72b87740 1 -- 172.21.15.43:0/4015074383 shutdown_connections 2024-04-21T08:45:34.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.729+0000 7f5d72b87740 1 -- 172.21.15.43:0/4015074383 wait complete. 2024-04-21T08:45:34.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.729+0000 7f5d72b87740 1 -- start start 2024-04-21T08:45:34.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.729+0000 7f5d72b87740 1 Processor -- start 2024-04-21T08:45:34.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.729+0000 7f5d72b87740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5594c46654d0 0x5594c466e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.729+0000 7f5d72b87740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594c4665e80 0x5594c466b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.729+0000 7f5d72b87740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c466ea00 0x5594c466be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.729+0000 7f5d72b87740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5594c4671910 con 0x5594c466ea00 2024-04-21T08:45:34.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.729+0000 7f5d72b87740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5594c4583500 con 0x5594c4665e80 2024-04-21T08:45:34.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.729+0000 7f5d72b87740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5594c4671a90 con 0x5594c46654d0 2024-04-21T08:45:34.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.729+0000 7f5d72b87740 10 client.0 _ll_register_callbacks cb 0x5594c4551b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:34.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.729+0000 7f5d6d455640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c466ea00 0x5594c466be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.730+0000 7f5d6d455640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c466ea00 0x5594c466be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39224/0 (socket says 172.21.15.43:39224) 2024-04-21T08:45:34.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.730+0000 7f5d6d455640 1 -- 172.21.15.43:0/1056735250 learned_addr learned my addr 172.21.15.43:0/1056735250 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:34.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.730+0000 7f5d6d455640 1 --2- 172.21.15.43:0/1056735250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5594c46654d0 0x5594c466e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.730+0000 7f5d6dc56640 1 --2- 172.21.15.43:0/1056735250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594c4665e80 0x5594c466b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.730+0000 7f5d6d455640 1 -- 172.21.15.43:0/1056735250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5594c46654d0 msgr2=0x5594c466e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.730+0000 7f5d6d455640 1 --2- 172.21.15.43:0/1056735250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5594c46654d0 0x5594c466e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.730+0000 7f5d6d455640 1 -- 172.21.15.43:0/1056735250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594c4665e80 msgr2=0x5594c466b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.730+0000 7f5d6d455640 1 --2- 172.21.15.43:0/1056735250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594c4665e80 0x5594c466b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.730+0000 7f5d6d455640 1 -- 172.21.15.43:0/1056735250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594c42dc3f0 con 0x5594c466ea00 2024-04-21T08:45:34.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.730+0000 7f5d6d455640 1 --2- 172.21.15.43:0/1056735250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c466ea00 0x5594c466be70 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f5d58006010 tx=0x7f5d580040d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c56b023cd59958 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.730+0000 7f5d517fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:34.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.730+0000 7f5d517fa640 1 -- 172.21.15.43:0/1056735250 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d58051070 con 0x5594c466ea00 2024-04-21T08:45:34.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.730+0000 7f5d517fa640 1 -- 172.21.15.43:0/1056735250 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f5d5802cbf0 con 0x5594c466ea00 2024-04-21T08:45:34.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.730+0000 7f5d72b87740 1 -- 172.21.15.43:0/1056735250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d58048070 con 0x5594c466ea00 2024-04-21T08:45:34.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.730+0000 7f5d517fa640 1 -- 172.21.15.43:0/1056735250 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d5805f050 con 0x5594c466ea00 2024-04-21T08:45:34.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.730+0000 7f5d72b87740 10 client.18567 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:34.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.730+0000 7f5d72b87740 1 -- 172.21.15.43:0/1056735250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5594c43a67d0 con 0x5594c466ea00 2024-04-21T08:45:34.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.731+0000 7f5d517fa640 1 -- 172.21.15.43:0/1056735250 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d58048070 con 0x5594c466ea00 2024-04-21T08:45:34.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.731+0000 7f5d517fa640 1 -- 172.21.15.43:0/1056735250 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5d580036d0 con 0x5594c466ea00 2024-04-21T08:45:34.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.731+0000 7f5d72b87740 10 client.18567 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:34.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.731+0000 7f5d72b87740 10 client.18567 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:45:34.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.731+0000 7f5d72b87740 20 client.18567 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:34.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.731+0000 7f5d72b87740 1 --2- 172.21.15.43:0/1056735250 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5594c467eab0 0x5594c4680ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.731+0000 7f5d72b87740 4 client.18567 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:34.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.731+0000 7f5d72b87740 1 -- 172.21.15.43:0/1056735250 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594c4569930 con 0x5594c467eab0 2024-04-21T08:45:34.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.731+0000 7f5d6dc56640 1 --2- 172.21.15.43:0/1056735250 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5594c467eab0 0x5594c4680ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.732+0000 7f5d6dc56640 1 --2- 172.21.15.43:0/1056735250 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5594c467eab0 0x5594c4680ea0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2f800f5c42f32d server_cookie=c533a6e66f598ce6 in_seq=0 out_seq=0 2024-04-21T08:45:34.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.732+0000 7f5d517fa640 10 client.18567 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:45:34.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.732+0000 7f5d517fa640 1 -- 172.21.15.43:0/1056735250 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5594c4569930 con 0x5594c467eab0 2024-04-21T08:45:34.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.732+0000 7f5d517fa640 10 client.18567 handle_command_reply: tid=0 2024-04-21T08:45:34.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.732+0000 7f5d72b87740 1 client.18567 shutdown 2024-04-21T08:45:34.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.732+0000 7f5d72b87740 1 -- 172.21.15.43:0/1056735250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c466ea00 msgr2=0x5594c466be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.732+0000 7f5d72b87740 1 --2- 172.21.15.43:0/1056735250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c466ea00 0x5594c466be70 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f5d58006010 tx=0x7f5d580040d0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.733+0000 7f5d72b87740 1 -- 172.21.15.43:0/1056735250 shutdown_connections 2024-04-21T08:45:34.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.733+0000 7f5d72b87740 1 --2- 172.21.15.43:0/1056735250 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5594c467eab0 0x5594c4680ea0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.733+0000 7f5d72b87740 1 --2- 172.21.15.43:0/1056735250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c466ea00 0x5594c466be70 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.733+0000 7f5d72b87740 1 --2- 172.21.15.43:0/1056735250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594c4665e80 0x5594c466b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.733+0000 7f5d72b87740 1 --2- 172.21.15.43:0/1056735250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5594c46654d0 0x5594c466e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.733+0000 7f5d517fa640 0 client.18567 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:45:34.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.733+0000 7f5d72b87740 1 -- 172.21.15.43:0/1056735250 >> 172.21.15.43:0/1056735250 conn(0x5594c465fba0 msgr2=0x5594c46618f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.733+0000 7f5d72b87740 1 -- 172.21.15.43:0/1056735250 shutdown_connections 2024-04-21T08:45:34.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.733+0000 7f5d72b87740 1 -- 172.21.15.43:0/1056735250 wait complete. 2024-04-21T08:45:34.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.733+0000 7f5d72b87740 20 client.18567 trim_cache size 0 max 16384 2024-04-21T08:45:34.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.746+0000 7f5d72b87740 1 Processor -- start 2024-04-21T08:45:34.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.746+0000 7f5d72b87740 1 -- start start 2024-04-21T08:45:34.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.746+0000 7f5d72b87740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c46654d0 0x5594c46842b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.746+0000 7f5d72b87740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594c4665e80 0x5594c466de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.746+0000 7f5d72b87740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5594c466ea00 0x5594c466e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.746+0000 7f5d72b87740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5d681af3c0 con 0x5594c46654d0 2024-04-21T08:45:34.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.746+0000 7f5d72b87740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5d681b0f00 con 0x5594c4665e80 2024-04-21T08:45:34.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.746+0000 7f5d72b87740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5d681afa90 con 0x5594c466ea00 2024-04-21T08:45:34.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.746+0000 7f5d6d455640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c46654d0 0x5594c46842b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.746+0000 7f5d6d455640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c46654d0 0x5594c46842b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39226/0 (socket says 172.21.15.43:39226) 2024-04-21T08:45:34.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.747+0000 7f5d6d455640 1 -- 172.21.15.43:0/4052862169 learned_addr learned my addr 172.21.15.43:0/4052862169 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:34.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.747+0000 7f5d6dc56640 1 --2- 172.21.15.43:0/4052862169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594c4665e80 0x5594c466de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.747+0000 7f5d6d455640 1 -- 172.21.15.43:0/4052862169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5594c466ea00 msgr2=0x5594c466e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.747+0000 7f5d6d455640 1 --2- 172.21.15.43:0/4052862169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5594c466ea00 0x5594c466e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.747+0000 7f5d6d455640 1 -- 172.21.15.43:0/4052862169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594c4665e80 msgr2=0x5594c466de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.747+0000 7f5d6d455640 1 --2- 172.21.15.43:0/4052862169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594c4665e80 0x5594c466de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.747+0000 7f5d6d455640 1 -- 172.21.15.43:0/4052862169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594c465b470 con 0x5594c46654d0 2024-04-21T08:45:34.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.747+0000 7f5d6d455640 1 --2- 172.21.15.43:0/4052862169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c46654d0 0x5594c46842b0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f5d58005930 tx=0x7f5d5802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ddd1ecfc8b51b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.747+0000 7f5d537fe640 1 -- 172.21.15.43:0/4052862169 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d58051040 con 0x5594c46654d0 2024-04-21T08:45:34.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.747+0000 7f5d537fe640 1 -- 172.21.15.43:0/4052862169 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d58048040 con 0x5594c46654d0 2024-04-21T08:45:34.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.747+0000 7f5d537fe640 1 -- 172.21.15.43:0/4052862169 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5802c9f0 con 0x5594c46654d0 2024-04-21T08:45:34.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.748+0000 7f5d72b87740 1 -- 172.21.15.43:0/4052862169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c46654d0 msgr2=0x5594c46842b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.748+0000 7f5d72b87740 1 --2- 172.21.15.43:0/4052862169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c46654d0 0x5594c46842b0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f5d58005930 tx=0x7f5d5802bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:34.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.748+0000 7f5d72b87740 1 -- 172.21.15.43:0/4052862169 shutdown_connections 2024-04-21T08:45:34.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.748+0000 7f5d72b87740 1 --2- 172.21.15.43:0/4052862169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5594c466ea00 0x5594c466e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.748+0000 7f5d72b87740 1 --2- 172.21.15.43:0/4052862169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594c4665e80 0x5594c466de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.748+0000 7f5d72b87740 1 --2- 172.21.15.43:0/4052862169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c46654d0 0x5594c46842b0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.748+0000 7f5d72b87740 1 -- 172.21.15.43:0/4052862169 >> 172.21.15.43:0/4052862169 conn(0x5594c465fba0 msgr2=0x5594c466a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.748+0000 7f5d72b87740 1 -- 172.21.15.43:0/4052862169 shutdown_connections 2024-04-21T08:45:34.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.748+0000 7f5d72b87740 1 -- 172.21.15.43:0/4052862169 wait complete. 2024-04-21T08:45:34.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.748+0000 7f5d72b87740 1 -- start start 2024-04-21T08:45:34.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.748+0000 7f5d72b87740 1 Processor -- start 2024-04-21T08:45:34.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.748+0000 7f5d72b87740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c46654d0 0x5594c46e1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.748+0000 7f5d72b87740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594c4665e80 0x5594c46e21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.748+0000 7f5d72b87740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5594c466ea00 0x5594c46dc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.748+0000 7f5d72b87740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5d681b0830 con 0x5594c46654d0 2024-04-21T08:45:34.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.749+0000 7f5d72b87740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5d681b0160 con 0x5594c4665e80 2024-04-21T08:45:34.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.749+0000 7f5d72b87740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5d681ad3a0 con 0x5594c466ea00 2024-04-21T08:45:34.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.749+0000 7f5d72b87740 10 client.0 _ll_register_callbacks cb 0x7f5d58005040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:34.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.749+0000 7f5d6d455640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c46654d0 0x5594c46e1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.749+0000 7f5d6d455640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c46654d0 0x5594c46e1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39230/0 (socket says 172.21.15.43:39230) 2024-04-21T08:45:34.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.749+0000 7f5d6d455640 1 -- 172.21.15.43:0/1615960485 learned_addr learned my addr 172.21.15.43:0/1615960485 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:34.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.749+0000 7f5d6dc56640 1 --2- 172.21.15.43:0/1615960485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594c4665e80 0x5594c46e21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.749+0000 7f5d6d455640 1 -- 172.21.15.43:0/1615960485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5594c466ea00 msgr2=0x5594c46dc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.749+0000 7f5d6d455640 1 --2- 172.21.15.43:0/1615960485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5594c466ea00 0x5594c46dc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.749+0000 7f5d6d455640 1 -- 172.21.15.43:0/1615960485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594c4665e80 msgr2=0x5594c46e21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.749+0000 7f5d6d455640 1 --2- 172.21.15.43:0/1615960485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594c4665e80 0x5594c46e21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.749+0000 7f5d6d455640 1 -- 172.21.15.43:0/1615960485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594c46dc7a0 con 0x5594c46654d0 2024-04-21T08:45:34.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.749+0000 7f5d6d455640 1 --2- 172.21.15.43:0/1615960485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c46654d0 0x5594c46e1ca0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f5d58000c00 tx=0x7f5d58005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7a750e514c9e81 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.749+0000 7f5d517fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:34.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.749+0000 7f5d517fa640 1 -- 172.21.15.43:0/1615960485 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d58051040 con 0x5594c46654d0 2024-04-21T08:45:34.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.749+0000 7f5d517fa640 1 -- 172.21.15.43:0/1615960485 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f5d5802cbf0 con 0x5594c46654d0 2024-04-21T08:45:34.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.749+0000 7f5d72b87740 1 -- 172.21.15.43:0/1615960485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d58048040 con 0x5594c46654d0 2024-04-21T08:45:34.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.749+0000 7f5d517fa640 1 -- 172.21.15.43:0/1615960485 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d58009520 con 0x5594c46654d0 2024-04-21T08:45:34.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.749+0000 7f5d72b87740 10 client.18573 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:34.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.749+0000 7f5d72b87740 1 -- 172.21.15.43:0/1615960485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5594c46e94a0 con 0x5594c46654d0 2024-04-21T08:45:34.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.750+0000 7f5d517fa640 1 -- 172.21.15.43:0/1615960485 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d58048040 con 0x5594c46654d0 2024-04-21T08:45:34.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.750+0000 7f5d517fa640 1 -- 172.21.15.43:0/1615960485 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5d58004b80 con 0x5594c46654d0 2024-04-21T08:45:34.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.750+0000 7f5d72b87740 10 client.18573 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:34.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.750+0000 7f5d72b87740 10 client.18573 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:45:34.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.750+0000 7f5d72b87740 20 client.18573 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:34.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.750+0000 7f5d72b87740 1 --2- 172.21.15.43:0/1615960485 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5594c46e9920 0x5594c46ebd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.750+0000 7f5d72b87740 4 client.18573 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:45:34.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.750+0000 7f5d72b87740 1 -- 172.21.15.43:0/1615960485 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5594c45d7c10 con 0x5594c46e9920 2024-04-21T08:45:34.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.750+0000 7f5d6dc56640 1 --2- 172.21.15.43:0/1615960485 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5594c46e9920 0x5594c46ebd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.750+0000 7f5d6dc56640 1 --2- 172.21.15.43:0/1615960485 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5594c46e9920 0x5594c46ebd10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9d9f59e51b12da11 server_cookie=3ab0a5415ba108ef in_seq=0 out_seq=0 2024-04-21T08:45:34.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.750+0000 7f5d517fa640 10 client.18573 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:45:34.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.751+0000 7f5d517fa640 1 -- 172.21.15.43:0/1615960485 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5594c45d7c10 con 0x5594c46e9920 2024-04-21T08:45:34.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.751+0000 7f5d517fa640 10 client.18573 handle_command_reply: tid=0 2024-04-21T08:45:34.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.751+0000 7f5d72b87740 1 client.18573 shutdown 2024-04-21T08:45:34.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.751+0000 7f5d72b87740 1 -- 172.21.15.43:0/1615960485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c46654d0 msgr2=0x5594c46e1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.751+0000 7f5d72b87740 1 --2- 172.21.15.43:0/1615960485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c46654d0 0x5594c46e1ca0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f5d58000c00 tx=0x7f5d58005420 comp rx=0 tx=0).stop 2024-04-21T08:45:34.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.751+0000 7f5d72b87740 1 -- 172.21.15.43:0/1615960485 shutdown_connections 2024-04-21T08:45:34.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.751+0000 7f5d72b87740 1 --2- 172.21.15.43:0/1615960485 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5594c46e9920 0x5594c46ebd10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.751+0000 7f5d72b87740 1 --2- 172.21.15.43:0/1615960485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5594c466ea00 0x5594c46dc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.751+0000 7f5d72b87740 1 --2- 172.21.15.43:0/1615960485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594c4665e80 0x5594c46e21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.751+0000 7f5d517fa640 0 client.18573 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:45:34.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.751+0000 7f5d72b87740 1 --2- 172.21.15.43:0/1615960485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594c46654d0 0x5594c46e1ca0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.751+0000 7f5d72b87740 1 -- 172.21.15.43:0/1615960485 >> 172.21.15.43:0/1615960485 conn(0x5594c465fba0 msgr2=0x5594c4660e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.751+0000 7f5d72b87740 1 -- 172.21.15.43:0/1615960485 shutdown_connections 2024-04-21T08:45:34.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.751+0000 7f5d72b87740 1 -- 172.21.15.43:0/1615960485 wait complete. 2024-04-21T08:45:34.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.751+0000 7f5d72b87740 20 client.18573 trim_cache size 0 max 16384 2024-04-21T08:45:34.759 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:45:34.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.751+0000 7f5d6feba640 1 -- 172.21.15.43:0/1063399268 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5d540a3d20 msgr2=0x7f5d540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.751+0000 7f5d6feba640 1 --2- 172.21.15.43:0/1063399268 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5d540a3d20 0x7f5d540a61e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5d6009c560 tx=0x7f5d60087040 comp rx=0 tx=0).stop 2024-04-21T08:45:34.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.751+0000 7f5d6feba640 1 -- 172.21.15.43:0/1063399268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d681986b0 msgr2=0x7f5d68198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.751+0000 7f5d6feba640 1 --2- 172.21.15.43:0/1063399268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d681986b0 0x7f5d68198b60 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000eaa0 tx=0x7f5d6000ef60 comp rx=0 tx=0).stop 2024-04-21T08:45:34.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.752+0000 7f5d6feba640 1 -- 172.21.15.43:0/1063399268 shutdown_connections 2024-04-21T08:45:34.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.752+0000 7f5d6feba640 1 --2- 172.21.15.43:0/1063399268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d681986b0 0x7f5d68198b60 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.752+0000 7f5d6feba640 1 --2- 172.21.15.43:0/1063399268 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5d540a3d20 0x7f5d540a61e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.752+0000 7f5d6feba640 1 --2- 172.21.15.43:0/1063399268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d68107260 0x7f5d6819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.752+0000 7f5d6feba640 1 --2- 172.21.15.43:0/1063399268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d68102190 0x7f5d6819e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.752+0000 7f5d6feba640 1 -- 172.21.15.43:0/1063399268 >> 172.21.15.43:0/1063399268 conn(0x7f5d680fd990 msgr2=0x7f5d680fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.752+0000 7f5d6feba640 1 -- 172.21.15.43:0/1063399268 shutdown_connections 2024-04-21T08:45:34.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.752+0000 7f5d6feba640 1 -- 172.21.15.43:0/1063399268 wait complete. 2024-04-21T08:45:34.762 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:45:34.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.866+0000 7f3f8b0ba640 1 -- 172.21.15.43:0/2667809462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f84107270 msgr2=0x7f3f8410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.866+0000 7f3f8b0ba640 1 --2- 172.21.15.43:0/2667809462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f84107270 0x7f3f8410ee40 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009ca0 tx=0x7f3f7c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.866+0000 7f3f8b0ba640 1 -- 172.21.15.43:0/2667809462 shutdown_connections 2024-04-21T08:45:34.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.866+0000 7f3f8b0ba640 1 --2- 172.21.15.43:0/2667809462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f84107270 0x7f3f8410ee40 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.866+0000 7f3f8b0ba640 1 --2- 172.21.15.43:0/2667809462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f8406b1f0 0x7f3f84106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.866+0000 7f3f8b0ba640 1 --2- 172.21.15.43:0/2667809462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f8406a840 0x7f3f8406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.866+0000 7f3f8b0ba640 1 -- 172.21.15.43:0/2667809462 >> 172.21.15.43:0/2667809462 conn(0x7f3f84077eb0 msgr2=0x7f3f840782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.866+0000 7f3f8b0ba640 1 -- 172.21.15.43:0/2667809462 shutdown_connections 2024-04-21T08:45:34.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.866+0000 7f3f8b0ba640 1 -- 172.21.15.43:0/2667809462 wait complete. 2024-04-21T08:45:34.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.867+0000 7f3f8b0ba640 1 Processor -- start 2024-04-21T08:45:34.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.867+0000 7f3f8b0ba640 1 -- start start 2024-04-21T08:45:34.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.867+0000 7f3f8b0ba640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f8406a840 0x7f3f84198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.867+0000 7f3f8b0ba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f84107270 0x7f3f84198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.867+0000 7f3f8b0ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f8419cea0 0x7f3f8419d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.867+0000 7f3f8b0ba640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841121b0 con 0x7f3f84107270 2024-04-21T08:45:34.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.867+0000 7f3f8b0ba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f84112030 con 0x7f3f8419cea0 2024-04-21T08:45:34.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.867+0000 7f3f8b0ba640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3f84112330 con 0x7f3f8406a840 2024-04-21T08:45:34.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.867+0000 7f3f83fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f84107270 0x7f3f84198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.867+0000 7f3f83fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f84107270 0x7f3f84198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39258/0 (socket says 172.21.15.43:39258) 2024-04-21T08:45:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.867+0000 7f3f83fff640 1 -- 172.21.15.43:0/1622184290 learned_addr learned my addr 172.21.15.43:0/1622184290 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.867+0000 7f3f88e56640 1 --2- 172.21.15.43:0/1622184290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f8406a840 0x7f3f84198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.867+0000 7f3f89657640 1 --2- 172.21.15.43:0/1622184290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f8419cea0 0x7f3f8419d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.867+0000 7f3f83fff640 1 -- 172.21.15.43:0/1622184290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f8406a840 msgr2=0x7f3f84198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.867+0000 7f3f83fff640 1 --2- 172.21.15.43:0/1622184290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f8406a840 0x7f3f84198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.867+0000 7f3f83fff640 1 -- 172.21.15.43:0/1622184290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f8419cea0 msgr2=0x7f3f8419d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.867+0000 7f3f83fff640 1 --2- 172.21.15.43:0/1622184290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f8419cea0 0x7f3f8419d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.867+0000 7f3f83fff640 1 -- 172.21.15.43:0/1622184290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8419d990 con 0x7f3f84107270 2024-04-21T08:45:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.867+0000 7f3f89657640 1 --2- 172.21.15.43:0/1622184290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f8419cea0 0x7f3f8419d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.867+0000 7f3f88e56640 1 --2- 172.21.15.43:0/1622184290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f8406a840 0x7f3f84198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.868+0000 7f3f83fff640 1 --2- 172.21.15.43:0/1622184290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f84107270 0x7f3f84198b70 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000e9c0 tx=0x7f3f7000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.868+0000 7f3f81ffb640 1 -- 172.21.15.43:0/1622184290 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7000cd80 con 0x7f3f84107270 2024-04-21T08:45:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.868+0000 7f3f8b0ba640 1 -- 172.21.15.43:0/1622184290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8419dc80 con 0x7f3f84107270 2024-04-21T08:45:34.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.868+0000 7f3f81ffb640 1 -- 172.21.15.43:0/1622184290 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f700054d0 con 0x7f3f84107270 2024-04-21T08:45:34.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.868+0000 7f3f81ffb640 1 -- 172.21.15.43:0/1622184290 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70018430 con 0x7f3f84107270 2024-04-21T08:45:34.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.868+0000 7f3f8b0ba640 1 -- 172.21.15.43:0/1622184290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f841a54c0 con 0x7f3f84107270 2024-04-21T08:45:34.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.870+0000 7f3f81ffb640 1 -- 172.21.15.43:0/1622184290 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3f70004f70 con 0x7f3f84107270 2024-04-21T08:45:34.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.870+0000 7f3f8dd87740 1 Processor -- start 2024-04-21T08:45:34.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.871+0000 7f3f8dd87740 1 -- start start 2024-04-21T08:45:34.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.871+0000 7f3f81ffb640 1 --2- 172.21.15.43:0/1622184290 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3f740a3d20 0x7f3f740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.871+0000 7f3f8dd87740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419c94d0 0x5632419c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.871+0000 7f3f8dd87740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632419c9e80 0x5632419d2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.871+0000 7f3f8dd87740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632419d2a00 0x5632419d4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.871+0000 7f3f8dd87740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56324192e1f0 con 0x5632419c94d0 2024-04-21T08:45:34.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.871+0000 7f3f8dd87740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5632418cb4b0 con 0x5632419c9e80 2024-04-21T08:45:34.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.871+0000 7f3f8dd87740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56324192a1b0 con 0x5632419d2a00 2024-04-21T08:45:34.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.871+0000 7f3f81ffb640 1 -- 172.21.15.43:0/1622184290 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f3f700d1200 con 0x7f3f84107270 2024-04-21T08:45:34.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.871+0000 7f3f88e56640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419c94d0 0x5632419c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.871+0000 7f3f89657640 1 --2- 172.21.15.43:0/1622184290 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3f740a3d20 0x7f3f740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.871+0000 7f3f88e56640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419c94d0 0x5632419c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39272/0 (socket says 172.21.15.43:39272) 2024-04-21T08:45:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.871+0000 7f3f88e56640 1 -- 172.21.15.43:0/2392195927 learned_addr learned my addr 172.21.15.43:0/2392195927 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.871+0000 7f3f83fff640 1 --2- 172.21.15.43:0/2392195927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632419c9e80 0x5632419d2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.871+0000 7f3f89657640 1 --2- 172.21.15.43:0/2392195927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632419d2a00 0x5632419d4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.871+0000 7f3f88e56640 1 -- 172.21.15.43:0/2392195927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632419d2a00 msgr2=0x5632419d4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.871+0000 7f3f88e56640 1 --2- 172.21.15.43:0/2392195927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632419d2a00 0x5632419d4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.871+0000 7f3f88e56640 1 -- 172.21.15.43:0/2392195927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632419c9e80 msgr2=0x5632419d2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.871+0000 7f3f88e56640 1 --2- 172.21.15.43:0/2392195927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632419c9e80 0x5632419d2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.871+0000 7f3f88e56640 1 -- 172.21.15.43:0/2392195927 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632418bd1b0 con 0x5632419c94d0 2024-04-21T08:45:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.871+0000 7f3f89657640 1 --2- 172.21.15.43:0/2392195927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632419d2a00 0x5632419d4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.871+0000 7f3f89657640 1 --2- 172.21.15.43:0/1622184290 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3f740a3d20 0x7f3f740a61e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c004af0 tx=0x7f3f7c004a40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.871+0000 7f3f88e56640 1 --2- 172.21.15.43:0/2392195927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419c94d0 0x5632419c98b0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f3f78008c60 tx=0x7f3f7800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae768551147f468 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.872+0000 7f3f6effd640 1 -- 172.21.15.43:0/2392195927 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78014070 con 0x5632419c94d0 2024-04-21T08:45:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.872+0000 7f3f6effd640 1 -- 172.21.15.43:0/2392195927 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f780047a0 con 0x5632419c94d0 2024-04-21T08:45:34.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.872+0000 7f3f6effd640 1 -- 172.21.15.43:0/2392195927 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78010670 con 0x5632419c94d0 2024-04-21T08:45:34.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.872+0000 7f3f8dd87740 1 -- 172.21.15.43:0/2392195927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419c94d0 msgr2=0x5632419c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.872+0000 7f3f8dd87740 1 --2- 172.21.15.43:0/2392195927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419c94d0 0x5632419c98b0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f3f78008c60 tx=0x7f3f7800e720 comp rx=0 tx=0).stop 2024-04-21T08:45:34.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.872+0000 7f3f8dd87740 1 -- 172.21.15.43:0/2392195927 shutdown_connections 2024-04-21T08:45:34.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.872+0000 7f3f8dd87740 1 --2- 172.21.15.43:0/2392195927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632419d2a00 0x5632419d4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.872+0000 7f3f8dd87740 1 --2- 172.21.15.43:0/2392195927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632419c9e80 0x5632419d2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.872+0000 7f3f8dd87740 1 --2- 172.21.15.43:0/2392195927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419c94d0 0x5632419c98b0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.872+0000 7f3f8dd87740 1 -- 172.21.15.43:0/2392195927 >> 172.21.15.43:0/2392195927 conn(0x5632419c3ba0 msgr2=0x5632419c3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.872+0000 7f3f8dd87740 1 -- 172.21.15.43:0/2392195927 shutdown_connections 2024-04-21T08:45:34.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.872+0000 7f3f8dd87740 1 -- 172.21.15.43:0/2392195927 wait complete. 2024-04-21T08:45:34.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.873+0000 7f3f8dd87740 1 -- start start 2024-04-21T08:45:34.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.873+0000 7f3f8dd87740 1 Processor -- start 2024-04-21T08:45:34.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.873+0000 7f3f8dd87740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632419c94d0 0x5632419d2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.873+0000 7f3f8dd87740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632419c9e80 0x5632419cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.873+0000 7f3f8dd87740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419d2a00 0x5632419cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.873+0000 7f3f8dd87740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5632419d5970 con 0x5632419d2a00 2024-04-21T08:45:34.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.873+0000 7f3f8dd87740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5632418e7500 con 0x5632419c9e80 2024-04-21T08:45:34.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.873+0000 7f3f8dd87740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5632419d5af0 con 0x5632419c94d0 2024-04-21T08:45:34.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.873+0000 7f3f8dd87740 10 client.0 _ll_register_callbacks cb 0x5632418b5b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:34.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.873+0000 7f3f89657640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419d2a00 0x5632419cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.873+0000 7f3f89657640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419d2a00 0x5632419cfcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39282/0 (socket says 172.21.15.43:39282) 2024-04-21T08:45:34.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.873+0000 7f3f89657640 1 -- 172.21.15.43:0/3571846300 learned_addr learned my addr 172.21.15.43:0/3571846300 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:34.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.873+0000 7f3f83fff640 1 --2- 172.21.15.43:0/3571846300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632419c9e80 0x5632419cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.874+0000 7f3f88e56640 1 --2- 172.21.15.43:0/3571846300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632419c94d0 0x5632419d2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.874+0000 7f3f89657640 1 -- 172.21.15.43:0/3571846300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632419c94d0 msgr2=0x5632419d2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.874+0000 7f3f89657640 1 --2- 172.21.15.43:0/3571846300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632419c94d0 0x5632419d2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.874+0000 7f3f89657640 1 -- 172.21.15.43:0/3571846300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632419c9e80 msgr2=0x5632419cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.874+0000 7f3f89657640 1 --2- 172.21.15.43:0/3571846300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632419c9e80 0x5632419cf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.874+0000 7f3f89657640 1 -- 172.21.15.43:0/3571846300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632416403f0 con 0x5632419d2a00 2024-04-21T08:45:34.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.874+0000 7f3f89657640 1 --2- 172.21.15.43:0/3571846300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419d2a00 0x5632419cfcf0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c036000 tx=0x7f3f7c005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64df1e1654e5a412 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.874+0000 7f3f63fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:34.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.874+0000 7f3f63fff640 1 -- 172.21.15.43:0/3571846300 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c053060 con 0x5632419d2a00 2024-04-21T08:45:34.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.874+0000 7f3f63fff640 1 -- 172.21.15.43:0/3571846300 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f3f7c003c40 con 0x5632419d2a00 2024-04-21T08:45:34.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.874+0000 7f3f8dd87740 1 -- 172.21.15.43:0/3571846300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f780047a0 con 0x5632419d2a00 2024-04-21T08:45:34.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.874+0000 7f3f63fff640 1 -- 172.21.15.43:0/3571846300 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3f7c05a050 con 0x5632419d2a00 2024-04-21T08:45:34.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.874+0000 7f3f8dd87740 10 client.18585 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:34.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.874+0000 7f3f8dd87740 1 -- 172.21.15.43:0/3571846300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56324170a7d0 con 0x5632419d2a00 2024-04-21T08:45:34.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.874+0000 7f3f63fff640 1 -- 172.21.15.43:0/3571846300 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f780047a0 con 0x5632419d2a00 2024-04-21T08:45:34.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.875+0000 7f3f63fff640 1 -- 172.21.15.43:0/3571846300 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3f7c04e6e0 con 0x5632419d2a00 2024-04-21T08:45:34.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.875+0000 7f3f8dd87740 10 client.18585 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:34.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.875+0000 7f3f8dd87740 10 client.18585 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:45:34.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.875+0000 7f3f8dd87740 20 client.18585 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:34.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.875+0000 7f3f8dd87740 1 --2- 172.21.15.43:0/3571846300 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5632419e2c50 0x5632419e5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.875+0000 7f3f8dd87740 4 client.18585 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:34.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.875+0000 7f3f8dd87740 1 -- 172.21.15.43:0/3571846300 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632418cd930 con 0x5632419e2c50 2024-04-21T08:45:34.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.875+0000 7f3f88e56640 1 --2- 172.21.15.43:0/3571846300 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5632419e2c50 0x5632419e5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.875+0000 7f3f63fff640 10 client.18585 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:45:34.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.875+0000 7f3f88e56640 1 --2- 172.21.15.43:0/3571846300 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5632419e2c50 0x5632419e5080 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=259c9649002b75fb server_cookie=34ced5949b1573e4 in_seq=0 out_seq=0 2024-04-21T08:45:34.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.876+0000 7f3f63fff640 1 -- 172.21.15.43:0/3571846300 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5632418cd930 con 0x5632419e2c50 2024-04-21T08:45:34.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.876+0000 7f3f63fff640 10 client.18585 handle_command_reply: tid=0 2024-04-21T08:45:34.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.876+0000 7f3f8dd87740 1 client.18585 shutdown 2024-04-21T08:45:34.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.876+0000 7f3f8dd87740 1 -- 172.21.15.43:0/3571846300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419d2a00 msgr2=0x5632419cfcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.876+0000 7f3f8dd87740 1 --2- 172.21.15.43:0/3571846300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419d2a00 0x5632419cfcf0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c036000 tx=0x7f3f7c005c30 comp rx=0 tx=0).stop 2024-04-21T08:45:34.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.876+0000 7f3f8dd87740 1 -- 172.21.15.43:0/3571846300 shutdown_connections 2024-04-21T08:45:34.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.876+0000 7f3f8dd87740 1 --2- 172.21.15.43:0/3571846300 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5632419e2c50 0x5632419e5080 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.876+0000 7f3f8dd87740 1 --2- 172.21.15.43:0/3571846300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419d2a00 0x5632419cfcf0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.876+0000 7f3f8dd87740 1 --2- 172.21.15.43:0/3571846300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632419c9e80 0x5632419cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.876+0000 7f3f63fff640 0 client.18585 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:45:34.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.876+0000 7f3f8dd87740 1 --2- 172.21.15.43:0/3571846300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632419c94d0 0x5632419d2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.876+0000 7f3f8dd87740 1 -- 172.21.15.43:0/3571846300 >> 172.21.15.43:0/3571846300 conn(0x5632419c3ba0 msgr2=0x5632419c58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.876+0000 7f3f8dd87740 1 -- 172.21.15.43:0/3571846300 shutdown_connections 2024-04-21T08:45:34.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.876+0000 7f3f8dd87740 1 -- 172.21.15.43:0/3571846300 wait complete. 2024-04-21T08:45:34.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.876+0000 7f3f8dd87740 20 client.18585 trim_cache size 0 max 16384 2024-04-21T08:45:34.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.890+0000 7f3f8dd87740 1 Processor -- start 2024-04-21T08:45:34.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.890+0000 7f3f8dd87740 1 -- start start 2024-04-21T08:45:34.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.890+0000 7f3f8dd87740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419c94d0 0x5632419de830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.890+0000 7f3f8dd87740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632419c9e80 0x5632419c2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.890+0000 7f3f8dd87740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632419d2a00 0x5632419d1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.890+0000 7f3f8dd87740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841af410 con 0x5632419c94d0 2024-04-21T08:45:34.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.890+0000 7f3f8dd87740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841b0f50 con 0x5632419d2a00 2024-04-21T08:45:34.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.890+0000 7f3f8dd87740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841afae0 con 0x5632419c9e80 2024-04-21T08:45:34.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.890+0000 7f3f88e56640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419c94d0 0x5632419de830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.890+0000 7f3f88e56640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419c94d0 0x5632419de830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39286/0 (socket says 172.21.15.43:39286) 2024-04-21T08:45:34.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.890+0000 7f3f88e56640 1 -- 172.21.15.43:0/2738843624 learned_addr learned my addr 172.21.15.43:0/2738843624 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:34.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.890+0000 7f3f83fff640 1 --2- 172.21.15.43:0/2738843624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632419c9e80 0x5632419c2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.890+0000 7f3f88e56640 1 -- 172.21.15.43:0/2738843624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632419c9e80 msgr2=0x5632419c2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.890+0000 7f3f88e56640 1 --2- 172.21.15.43:0/2738843624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632419c9e80 0x5632419c2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.890+0000 7f3f88e56640 1 -- 172.21.15.43:0/2738843624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632419d2a00 msgr2=0x5632419d1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.890+0000 7f3f88e56640 1 --2- 172.21.15.43:0/2738843624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632419d2a00 0x5632419d1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.890+0000 7f3f88e56640 1 -- 172.21.15.43:0/2738843624 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632419d59c0 con 0x5632419c94d0 2024-04-21T08:45:34.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.890+0000 7f3f88e56640 1 --2- 172.21.15.43:0/2738843624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419c94d0 0x5632419de830 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f3f78008c60 tx=0x7f3f78008fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2048b24f8842dde5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.891+0000 7f3f6e7fc640 1 -- 172.21.15.43:0/2738843624 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78014040 con 0x5632419c94d0 2024-04-21T08:45:34.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.891+0000 7f3f6e7fc640 1 -- 172.21.15.43:0/2738843624 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f780046f0 con 0x5632419c94d0 2024-04-21T08:45:34.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.891+0000 7f3f6e7fc640 1 -- 172.21.15.43:0/2738843624 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f780035f0 con 0x5632419c94d0 2024-04-21T08:45:34.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.891+0000 7f3f8dd87740 1 -- 172.21.15.43:0/2738843624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419c94d0 msgr2=0x5632419de830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.891+0000 7f3f8dd87740 1 --2- 172.21.15.43:0/2738843624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419c94d0 0x5632419de830 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f3f78008c60 tx=0x7f3f78008fa0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.891+0000 7f3f8dd87740 1 -- 172.21.15.43:0/2738843624 shutdown_connections 2024-04-21T08:45:34.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.891+0000 7f3f8dd87740 1 --2- 172.21.15.43:0/2738843624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632419d2a00 0x5632419d1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.891+0000 7f3f8dd87740 1 --2- 172.21.15.43:0/2738843624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632419c9e80 0x5632419c2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.891+0000 7f3f8dd87740 1 --2- 172.21.15.43:0/2738843624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419c94d0 0x5632419de830 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.891+0000 7f3f8dd87740 1 -- 172.21.15.43:0/2738843624 >> 172.21.15.43:0/2738843624 conn(0x5632419c3ba0 msgr2=0x5632419c2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.891+0000 7f3f8dd87740 1 -- 172.21.15.43:0/2738843624 shutdown_connections 2024-04-21T08:45:34.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.891+0000 7f3f8dd87740 1 -- 172.21.15.43:0/2738843624 wait complete. 2024-04-21T08:45:34.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.891+0000 7f3f8dd87740 1 -- start start 2024-04-21T08:45:34.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.892+0000 7f3f8dd87740 1 Processor -- start 2024-04-21T08:45:34.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.892+0000 7f3f8dd87740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419c94d0 0x5632419db030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.892+0000 7f3f8dd87740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632419c9e80 0x5632419cc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.892+0000 7f3f8dd87740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632419d2a00 0x5632419cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.892+0000 7f3f8dd87740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841b0880 con 0x5632419c94d0 2024-04-21T08:45:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.892+0000 7f3f8dd87740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841b01b0 con 0x5632419c9e80 2024-04-21T08:45:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.892+0000 7f3f8dd87740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841ad3f0 con 0x5632419d2a00 2024-04-21T08:45:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.892+0000 7f3f8dd87740 10 client.0 _ll_register_callbacks cb 0x7f3f7c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.892+0000 7f3f88e56640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419c94d0 0x5632419db030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.892+0000 7f3f88e56640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419c94d0 0x5632419db030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39288/0 (socket says 172.21.15.43:39288) 2024-04-21T08:45:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.892+0000 7f3f88e56640 1 -- 172.21.15.43:0/1953114614 learned_addr learned my addr 172.21.15.43:0/1953114614 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.892+0000 7f3f89657640 1 --2- 172.21.15.43:0/1953114614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632419d2a00 0x5632419cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.892+0000 7f3f83fff640 1 --2- 172.21.15.43:0/1953114614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632419c9e80 0x5632419cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.892+0000 7f3f88e56640 1 -- 172.21.15.43:0/1953114614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632419d2a00 msgr2=0x5632419cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.892+0000 7f3f88e56640 1 --2- 172.21.15.43:0/1953114614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632419d2a00 0x5632419cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.892+0000 7f3f88e56640 1 -- 172.21.15.43:0/1953114614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632419c9e80 msgr2=0x5632419cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.892+0000 7f3f88e56640 1 --2- 172.21.15.43:0/1953114614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632419c9e80 0x5632419cc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.892+0000 7f3f88e56640 1 -- 172.21.15.43:0/1953114614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632419cccb0 con 0x5632419c94d0 2024-04-21T08:45:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.893+0000 7f3f88e56640 1 --2- 172.21.15.43:0/1953114614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419c94d0 0x5632419db030 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f3f78008c30 tx=0x7f3f78025950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f952dcb9f8eacda5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:34.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.893+0000 7f3f63fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:34.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.893+0000 7f3f63fff640 1 -- 172.21.15.43:0/1953114614 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78014040 con 0x5632419c94d0 2024-04-21T08:45:34.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.893+0000 7f3f63fff640 1 -- 172.21.15.43:0/1953114614 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f3f78004040 con 0x5632419c94d0 2024-04-21T08:45:34.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.893+0000 7f3f8dd87740 1 -- 172.21.15.43:0/1953114614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f780046f0 con 0x5632419c94d0 2024-04-21T08:45:34.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.893+0000 7f3f63fff640 1 -- 172.21.15.43:0/1953114614 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3f7803c050 con 0x5632419c94d0 2024-04-21T08:45:34.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.893+0000 7f3f8dd87740 10 client.18591 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:34.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.893+0000 7f3f8dd87740 1 -- 172.21.15.43:0/1953114614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563241a49130 con 0x5632419c94d0 2024-04-21T08:45:34.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.893+0000 7f3f63fff640 1 -- 172.21.15.43:0/1953114614 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f780046f0 con 0x5632419c94d0 2024-04-21T08:45:34.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.893+0000 7f3f63fff640 1 -- 172.21.15.43:0/1953114614 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3f7802e460 con 0x5632419c94d0 2024-04-21T08:45:34.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.893+0000 7f3f8dd87740 10 client.18591 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.893+0000 7f3f8dd87740 10 client.18591 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:45:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.893+0000 7f3f8dd87740 20 client.18591 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.893+0000 7f3f8dd87740 1 --2- 172.21.15.43:0/1953114614 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x563241a49740 0x563241a4bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.893+0000 7f3f8dd87740 4 client.18591 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:45:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.893+0000 7f3f8dd87740 1 -- 172.21.15.43:0/1953114614 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56324193bc10 con 0x563241a49740 2024-04-21T08:45:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.894+0000 7f3f83fff640 1 --2- 172.21.15.43:0/1953114614 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x563241a49740 0x563241a4bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.894+0000 7f3f83fff640 1 --2- 172.21.15.43:0/1953114614 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x563241a49740 0x563241a4bb30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=72911f21e0e32397 server_cookie=5f7638c2973999e3 in_seq=0 out_seq=0 2024-04-21T08:45:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.894+0000 7f3f63fff640 10 client.18591 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:45:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.894+0000 7f3f63fff640 1 -- 172.21.15.43:0/1953114614 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56324193bc10 con 0x563241a49740 2024-04-21T08:45:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.894+0000 7f3f63fff640 10 client.18591 handle_command_reply: tid=0 2024-04-21T08:45:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.894+0000 7f3f8dd87740 1 client.18591 shutdown 2024-04-21T08:45:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.894+0000 7f3f8dd87740 1 -- 172.21.15.43:0/1953114614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419c94d0 msgr2=0x5632419db030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.894+0000 7f3f8dd87740 1 --2- 172.21.15.43:0/1953114614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419c94d0 0x5632419db030 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f3f78008c30 tx=0x7f3f78025950 comp rx=0 tx=0).stop 2024-04-21T08:45:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.894+0000 7f3f8dd87740 1 -- 172.21.15.43:0/1953114614 shutdown_connections 2024-04-21T08:45:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.894+0000 7f3f8dd87740 1 --2- 172.21.15.43:0/1953114614 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x563241a49740 0x563241a4bb30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.894+0000 7f3f8dd87740 1 --2- 172.21.15.43:0/1953114614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5632419d2a00 0x5632419cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.894+0000 7f3f8dd87740 1 --2- 172.21.15.43:0/1953114614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5632419c9e80 0x5632419cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.894+0000 7f3f63fff640 0 client.18591 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:45:34.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.894+0000 7f3f8dd87740 1 --2- 172.21.15.43:0/1953114614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632419c94d0 0x5632419db030 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.894+0000 7f3f8dd87740 1 -- 172.21.15.43:0/1953114614 >> 172.21.15.43:0/1953114614 conn(0x5632419c3ba0 msgr2=0x5632419c2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.894+0000 7f3f8dd87740 1 -- 172.21.15.43:0/1953114614 shutdown_connections 2024-04-21T08:45:34.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.894+0000 7f3f8dd87740 1 -- 172.21.15.43:0/1953114614 wait complete. 2024-04-21T08:45:34.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.894+0000 7f3f8dd87740 20 client.18591 trim_cache size 0 max 16384 2024-04-21T08:45:34.901 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:45:34.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.895+0000 7f3f8b0ba640 1 -- 172.21.15.43:0/1622184290 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3f740a3d20 msgr2=0x7f3f740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.895+0000 7f3f8b0ba640 1 --2- 172.21.15.43:0/1622184290 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3f740a3d20 0x7f3f740a61e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c004af0 tx=0x7f3f7c004a40 comp rx=0 tx=0).stop 2024-04-21T08:45:34.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.895+0000 7f3f8b0ba640 1 -- 172.21.15.43:0/1622184290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f84107270 msgr2=0x7f3f84198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:34.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.895+0000 7f3f8b0ba640 1 --2- 172.21.15.43:0/1622184290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f84107270 0x7f3f84198b70 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000e9c0 tx=0x7f3f7000ee80 comp rx=0 tx=0).stop 2024-04-21T08:45:34.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.895+0000 7f3f8b0ba640 1 -- 172.21.15.43:0/1622184290 shutdown_connections 2024-04-21T08:45:34.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.895+0000 7f3f8b0ba640 1 --2- 172.21.15.43:0/1622184290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f8419cea0 0x7f3f8419d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.895+0000 7f3f8b0ba640 1 --2- 172.21.15.43:0/1622184290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f84107270 0x7f3f84198b70 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.895+0000 7f3f8b0ba640 1 --2- 172.21.15.43:0/1622184290 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3f740a3d20 0x7f3f740a61e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.895+0000 7f3f8b0ba640 1 --2- 172.21.15.43:0/1622184290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f8406a840 0x7f3f84198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:34.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.895+0000 7f3f8b0ba640 1 -- 172.21.15.43:0/1622184290 >> 172.21.15.43:0/1622184290 conn(0x7f3f84077eb0 msgr2=0x7f3f840fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:34.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.895+0000 7f3f8b0ba640 1 -- 172.21.15.43:0/1622184290 shutdown_connections 2024-04-21T08:45:34.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:34.895+0000 7f3f8b0ba640 1 -- 172.21.15.43:0/1622184290 wait complete. 2024-04-21T08:45:34.905 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:45:35.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.011+0000 7f682d65c640 1 -- 172.21.15.43:0/1386302549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6828107260 msgr2=0x7f682810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.011+0000 7f682d65c640 1 --2- 172.21.15.43:0/1386302549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6828107260 0x7f682810ee30 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f6820009ca0 tx=0x7f682002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.011+0000 7f682d65c640 1 -- 172.21.15.43:0/1386302549 shutdown_connections 2024-04-21T08:45:35.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.011+0000 7f682d65c640 1 --2- 172.21.15.43:0/1386302549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6828107260 0x7f682810ee30 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.011+0000 7f682d65c640 1 --2- 172.21.15.43:0/1386302549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6828102b40 0x7f6828106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.011+0000 7f682d65c640 1 --2- 172.21.15.43:0/1386302549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6828102190 0x7f6828102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.011+0000 7f682d65c640 1 -- 172.21.15.43:0/1386302549 >> 172.21.15.43:0/1386302549 conn(0x7f68280793a0 msgr2=0x7f68280797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:35.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.011+0000 7f682d65c640 1 -- 172.21.15.43:0/1386302549 shutdown_connections 2024-04-21T08:45:35.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.011+0000 7f682d65c640 1 -- 172.21.15.43:0/1386302549 wait complete. 2024-04-21T08:45:35.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.011+0000 7f682d65c640 1 Processor -- start 2024-04-21T08:45:35.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.011+0000 7f682d65c640 1 -- start start 2024-04-21T08:45:35.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.012+0000 7f682d65c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6828102190 0x7f6828198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.012+0000 7f682d65c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6828107260 0x7f6828198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.012+0000 7f682d65c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f682819cfd0 0x7f682819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.012+0000 7f682d65c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f68281121b0 con 0x7f6828107260 2024-04-21T08:45:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.012+0000 7f682d65c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6828112030 con 0x7f682819cfd0 2024-04-21T08:45:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.012+0000 7f682d65c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6828112330 con 0x7f6828102190 2024-04-21T08:45:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.012+0000 7f681ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6828107260 0x7f6828198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.012+0000 7f681ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6828107260 0x7f6828198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39308/0 (socket says 172.21.15.43:39308) 2024-04-21T08:45:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.012+0000 7f681ffff640 1 -- 172.21.15.43:0/4144911602 learned_addr learned my addr 172.21.15.43:0/4144911602 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.012+0000 7f68277fe640 1 --2- 172.21.15.43:0/4144911602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f682819cfd0 0x7f682819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.012+0000 7f6826ffd640 1 --2- 172.21.15.43:0/4144911602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6828102190 0x7f6828198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.012+0000 7f681ffff640 1 -- 172.21.15.43:0/4144911602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6828102190 msgr2=0x7f6828198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.012+0000 7f681ffff640 1 --2- 172.21.15.43:0/4144911602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6828102190 0x7f6828198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.012+0000 7f681ffff640 1 -- 172.21.15.43:0/4144911602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f682819cfd0 msgr2=0x7f682819d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.012+0000 7f681ffff640 1 --2- 172.21.15.43:0/4144911602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f682819cfd0 0x7f682819d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.012+0000 7f681ffff640 1 -- 172.21.15.43:0/4144911602 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682819dac0 con 0x7f6828107260 2024-04-21T08:45:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.012+0000 7f681ffff640 1 --2- 172.21.15.43:0/4144911602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6828107260 0x7f6828198c50 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f6810005fd0 tx=0x7f681000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.013+0000 7f6824ff9640 1 -- 172.21.15.43:0/4144911602 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6810017070 con 0x7f6828107260 2024-04-21T08:45:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.013+0000 7f6824ff9640 1 -- 172.21.15.43:0/4144911602 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6810009c30 con 0x7f6828107260 2024-04-21T08:45:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.013+0000 7f682d65c640 1 -- 172.21.15.43:0/4144911602 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f682819ddb0 con 0x7f6828107260 2024-04-21T08:45:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.013+0000 7f6824ff9640 1 -- 172.21.15.43:0/4144911602 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6810003d30 con 0x7f6828107260 2024-04-21T08:45:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.013+0000 7f682d65c640 1 -- 172.21.15.43:0/4144911602 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68281a55a0 con 0x7f6828107260 2024-04-21T08:45:35.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.015+0000 7f6824ff9640 1 -- 172.21.15.43:0/4144911602 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6810003ed0 con 0x7f6828107260 2024-04-21T08:45:35.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.015+0000 7f683024d740 1 Processor -- start 2024-04-21T08:45:35.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.015+0000 7f683024d740 1 -- start start 2024-04-21T08:45:35.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.016+0000 7f683024d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592843c54d0 0x5592843c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.016+0000 7f683024d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592843c5e80 0x5592843ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.016+0000 7f683024d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843cea00 0x5592843d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.016+0000 7f683024d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55928432a1f0 con 0x5592843cea00 2024-04-21T08:45:35.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.016+0000 7f683024d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5592842c74b0 con 0x5592843c5e80 2024-04-21T08:45:35.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.016+0000 7f683024d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5592843261b0 con 0x5592843c54d0 2024-04-21T08:45:35.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.016+0000 7f681ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843cea00 0x5592843d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.016+0000 7f6824ff9640 1 --2- 172.21.15.43:0/4144911602 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f68140a3d20 0x7f68140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.016+0000 7f681ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843cea00 0x5592843d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39322/0 (socket says 172.21.15.43:39322) 2024-04-21T08:45:35.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.016+0000 7f681ffff640 1 -- 172.21.15.43:0/2649416245 learned_addr learned my addr 172.21.15.43:0/2649416245 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:35.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.016+0000 7f68277fe640 1 --2- 172.21.15.43:0/2649416245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592843c54d0 0x5592843c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.016+0000 7f6826ffd640 1 --2- 172.21.15.43:0/2649416245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592843c5e80 0x5592843ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.016+0000 7f68277fe640 1 --2- 172.21.15.43:0/4144911602 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f68140a3d20 0x7f68140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.016+0000 7f6824ff9640 1 -- 172.21.15.43:0/4144911602 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f68100d0810 con 0x7f6828107260 2024-04-21T08:45:35.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.016+0000 7f681ffff640 1 -- 172.21.15.43:0/2649416245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592843c54d0 msgr2=0x5592843c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:35.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.016+0000 7f681ffff640 1 --2- 172.21.15.43:0/2649416245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592843c54d0 0x5592843c58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.016+0000 7f681ffff640 1 -- 172.21.15.43:0/2649416245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592843c5e80 msgr2=0x5592843ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:35.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.016+0000 7f681ffff640 1 --2- 172.21.15.43:0/2649416245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592843c5e80 0x5592843ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.016+0000 7f681ffff640 1 -- 172.21.15.43:0/2649416245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592842b91b0 con 0x5592843cea00 2024-04-21T08:45:35.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.017+0000 7f681ffff640 1 --2- 172.21.15.43:0/2649416245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843cea00 0x5592843d0e50 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f681001c000 tx=0x7f681000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc0ba1b18f3f9c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:35.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.017+0000 7f68277fe640 1 --2- 172.21.15.43:0/4144911602 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f68140a3d20 0x7f68140a61e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6820005e30 tx=0x7f6820005d80 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:35.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.017+0000 7f681d7fa640 1 -- 172.21.15.43:0/2649416245 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68100a0730 con 0x5592843cea00 2024-04-21T08:45:35.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.017+0000 7f681d7fa640 1 -- 172.21.15.43:0/2649416245 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68100a4ff0 con 0x5592843cea00 2024-04-21T08:45:35.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.017+0000 7f681d7fa640 1 -- 172.21.15.43:0/2649416245 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68100d0380 con 0x5592843cea00 2024-04-21T08:45:35.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.017+0000 7f683024d740 1 -- 172.21.15.43:0/2649416245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843cea00 msgr2=0x5592843d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.017+0000 7f683024d740 1 --2- 172.21.15.43:0/2649416245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843cea00 0x5592843d0e50 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f681001c000 tx=0x7f681000bf60 comp rx=0 tx=0).stop 2024-04-21T08:45:35.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.018+0000 7f683024d740 1 -- 172.21.15.43:0/2649416245 shutdown_connections 2024-04-21T08:45:35.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.018+0000 7f683024d740 1 --2- 172.21.15.43:0/2649416245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843cea00 0x5592843d0e50 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.018+0000 7f683024d740 1 --2- 172.21.15.43:0/2649416245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592843c5e80 0x5592843ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.018+0000 7f683024d740 1 --2- 172.21.15.43:0/2649416245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592843c54d0 0x5592843c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.018+0000 7f683024d740 1 -- 172.21.15.43:0/2649416245 >> 172.21.15.43:0/2649416245 conn(0x5592843bfba0 msgr2=0x5592843bff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:35.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.018+0000 7f683024d740 1 -- 172.21.15.43:0/2649416245 shutdown_connections 2024-04-21T08:45:35.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.018+0000 7f683024d740 1 -- 172.21.15.43:0/2649416245 wait complete. 2024-04-21T08:45:35.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.018+0000 7f683024d740 1 -- start start 2024-04-21T08:45:35.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.018+0000 7f683024d740 1 Processor -- start 2024-04-21T08:45:35.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.018+0000 7f683024d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592843c54d0 0x5592843ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.018+0000 7f683024d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592843c5e80 0x5592843cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.018+0000 7f683024d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843cea00 0x5592843cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.018+0000 7f683024d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5592843d1910 con 0x5592843cea00 2024-04-21T08:45:35.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.018+0000 7f683024d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5592842e3500 con 0x5592843c54d0 2024-04-21T08:45:35.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.018+0000 7f683024d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5592843d1a90 con 0x5592843c5e80 2024-04-21T08:45:35.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.018+0000 7f683024d740 10 client.0 _ll_register_callbacks cb 0x5592842b1b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:35.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.018+0000 7f68277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843cea00 0x5592843cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.018+0000 7f68277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843cea00 0x5592843cbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39332/0 (socket says 172.21.15.43:39332) 2024-04-21T08:45:35.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.018+0000 7f68277fe640 1 -- 172.21.15.43:0/1977337338 learned_addr learned my addr 172.21.15.43:0/1977337338 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:35.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.018+0000 7f6826ffd640 1 --2- 172.21.15.43:0/1977337338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592843c54d0 0x5592843ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.019+0000 7f681ffff640 1 --2- 172.21.15.43:0/1977337338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592843c5e80 0x5592843cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.019+0000 7f68277fe640 1 -- 172.21.15.43:0/1977337338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592843c5e80 msgr2=0x5592843cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:35.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.019+0000 7f68277fe640 1 --2- 172.21.15.43:0/1977337338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592843c5e80 0x5592843cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.019+0000 7f68277fe640 1 -- 172.21.15.43:0/1977337338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592843c54d0 msgr2=0x5592843ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:35.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.019+0000 7f68277fe640 1 --2- 172.21.15.43:0/1977337338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592843c54d0 0x5592843ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.019+0000 7f68277fe640 1 -- 172.21.15.43:0/1977337338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55928403c3f0 con 0x5592843cea00 2024-04-21T08:45:35.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.019+0000 7f68277fe640 1 --2- 172.21.15.43:0/1977337338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843cea00 0x5592843cbe70 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f6820036000 tx=0x7f68200095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ce89771c0c5ef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:35.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.019+0000 7f680e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:35.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.019+0000 7f680e7fc640 1 -- 172.21.15.43:0/1977337338 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820054060 con 0x5592843cea00 2024-04-21T08:45:35.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.019+0000 7f680e7fc640 1 -- 172.21.15.43:0/1977337338 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f6820004040 con 0x5592843cea00 2024-04-21T08:45:35.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.019+0000 7f683024d740 1 -- 172.21.15.43:0/1977337338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68100a4ff0 con 0x5592843cea00 2024-04-21T08:45:35.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.019+0000 7f680e7fc640 1 -- 172.21.15.43:0/1977337338 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f682005b050 con 0x5592843cea00 2024-04-21T08:45:35.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.019+0000 7f683024d740 10 client.18603 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:35.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.019+0000 7f683024d740 1 -- 172.21.15.43:0/1977337338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5592841067d0 con 0x5592843cea00 2024-04-21T08:45:35.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.019+0000 7f680e7fc640 1 -- 172.21.15.43:0/1977337338 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68100a4ff0 con 0x5592843cea00 2024-04-21T08:45:35.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.019+0000 7f680e7fc640 1 -- 172.21.15.43:0/1977337338 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f682004f880 con 0x5592843cea00 2024-04-21T08:45:35.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.020+0000 7f683024d740 10 client.18603 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:35.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.020+0000 7f683024d740 10 client.18603 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:45:35.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.020+0000 7f683024d740 20 client.18603 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:35.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.020+0000 7f683024d740 1 --2- 172.21.15.43:0/1977337338 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5592843decc0 0x5592843e1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.020+0000 7f683024d740 4 client.18603 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:35.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.020+0000 7f683024d740 1 -- 172.21.15.43:0/1977337338 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592842c9930 con 0x5592843decc0 2024-04-21T08:45:35.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.020+0000 7f6826ffd640 1 --2- 172.21.15.43:0/1977337338 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5592843decc0 0x5592843e1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.020+0000 7f6826ffd640 1 --2- 172.21.15.43:0/1977337338 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5592843decc0 0x5592843e1130 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c9cdba0c9bba103 server_cookie=168f94d57eaa9f02 in_seq=0 out_seq=0 2024-04-21T08:45:35.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.020+0000 7f680e7fc640 10 client.18603 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:45:35.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.021+0000 7f680e7fc640 1 -- 172.21.15.43:0/1977337338 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5592842c9930 con 0x5592843decc0 2024-04-21T08:45:35.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.021+0000 7f680e7fc640 10 client.18603 handle_command_reply: tid=0 2024-04-21T08:45:35.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.021+0000 7f683024d740 1 client.18603 shutdown 2024-04-21T08:45:35.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.021+0000 7f683024d740 1 -- 172.21.15.43:0/1977337338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843cea00 msgr2=0x5592843cbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.021+0000 7f683024d740 1 --2- 172.21.15.43:0/1977337338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843cea00 0x5592843cbe70 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f6820036000 tx=0x7f68200095c0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.021+0000 7f683024d740 1 -- 172.21.15.43:0/1977337338 shutdown_connections 2024-04-21T08:45:35.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.021+0000 7f683024d740 1 --2- 172.21.15.43:0/1977337338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843cea00 0x5592843cbe70 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.021+0000 7f683024d740 1 --2- 172.21.15.43:0/1977337338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592843c5e80 0x5592843cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.021+0000 7f683024d740 1 --2- 172.21.15.43:0/1977337338 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5592843decc0 0x5592843e1130 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.021+0000 7f683024d740 1 --2- 172.21.15.43:0/1977337338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592843c54d0 0x5592843ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.021+0000 7f680e7fc640 0 client.18603 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:45:35.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.021+0000 7f683024d740 1 -- 172.21.15.43:0/1977337338 >> 172.21.15.43:0/1977337338 conn(0x5592843bfba0 msgr2=0x5592843c18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:35.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.021+0000 7f683024d740 1 -- 172.21.15.43:0/1977337338 shutdown_connections 2024-04-21T08:45:35.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.021+0000 7f683024d740 1 -- 172.21.15.43:0/1977337338 wait complete. 2024-04-21T08:45:35.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.021+0000 7f683024d740 20 client.18603 trim_cache size 0 max 16384 2024-04-21T08:45:35.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.034+0000 7f683024d740 1 Processor -- start 2024-04-21T08:45:35.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.035+0000 7f683024d740 1 -- start start 2024-04-21T08:45:35.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.035+0000 7f683024d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843c54d0 0x5592843da820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.035+0000 7f683024d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592843c5e80 0x5592843bea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.035+0000 7f683024d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592843cea00 0x5592843e6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.035+0000 7f683024d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f68281af460 con 0x5592843c54d0 2024-04-21T08:45:35.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.035+0000 7f683024d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f68281b0fa0 con 0x5592843cea00 2024-04-21T08:45:35.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.035+0000 7f683024d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68281afb30 con 0x5592843c5e80 2024-04-21T08:45:35.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.035+0000 7f6826ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843c54d0 0x5592843da820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.035+0000 7f6826ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843c54d0 0x5592843da820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39342/0 (socket says 172.21.15.43:39342) 2024-04-21T08:45:35.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.035+0000 7f6826ffd640 1 -- 172.21.15.43:0/1743125573 learned_addr learned my addr 172.21.15.43:0/1743125573 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:35.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.035+0000 7f68277fe640 1 --2- 172.21.15.43:0/1743125573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592843cea00 0x5592843e6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.035+0000 7f681ffff640 1 --2- 172.21.15.43:0/1743125573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592843c5e80 0x5592843bea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.035+0000 7f6826ffd640 1 -- 172.21.15.43:0/1743125573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592843c5e80 msgr2=0x5592843bea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:35.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.035+0000 7f6826ffd640 1 --2- 172.21.15.43:0/1743125573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592843c5e80 0x5592843bea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.035+0000 7f6826ffd640 1 -- 172.21.15.43:0/1743125573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592843cea00 msgr2=0x5592843e6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:35.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.035+0000 7f6826ffd640 1 --2- 172.21.15.43:0/1743125573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592843cea00 0x5592843e6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.035+0000 7f6826ffd640 1 -- 172.21.15.43:0/1743125573 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592843c1e90 con 0x5592843c54d0 2024-04-21T08:45:35.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.035+0000 7f6826ffd640 1 --2- 172.21.15.43:0/1743125573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843c54d0 0x5592843da820 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x5592843e19a0 tx=0x7f6818005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a5198b1bcdeb4c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:35.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.036+0000 7f681cff9640 1 -- 172.21.15.43:0/1743125573 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818021360 con 0x5592843c54d0 2024-04-21T08:45:35.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.036+0000 7f681cff9640 1 -- 172.21.15.43:0/1743125573 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6818011440 con 0x5592843c54d0 2024-04-21T08:45:35.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.036+0000 7f681cff9640 1 -- 172.21.15.43:0/1743125573 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681801e650 con 0x5592843c54d0 2024-04-21T08:45:35.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.036+0000 7f683024d740 1 -- 172.21.15.43:0/1743125573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843c54d0 msgr2=0x5592843da820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.036+0000 7f683024d740 1 --2- 172.21.15.43:0/1743125573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843c54d0 0x5592843da820 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x5592843e19a0 tx=0x7f6818005f90 comp rx=0 tx=0).stop 2024-04-21T08:45:35.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.036+0000 7f683024d740 1 -- 172.21.15.43:0/1743125573 shutdown_connections 2024-04-21T08:45:35.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.036+0000 7f683024d740 1 --2- 172.21.15.43:0/1743125573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592843cea00 0x5592843e6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.036+0000 7f683024d740 1 --2- 172.21.15.43:0/1743125573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592843c5e80 0x5592843bea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.036+0000 7f683024d740 1 --2- 172.21.15.43:0/1743125573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843c54d0 0x5592843da820 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.036+0000 7f683024d740 1 -- 172.21.15.43:0/1743125573 >> 172.21.15.43:0/1743125573 conn(0x5592843bfba0 msgr2=0x5592843be370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:35.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.036+0000 7f683024d740 1 -- 172.21.15.43:0/1743125573 shutdown_connections 2024-04-21T08:45:35.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.036+0000 7f683024d740 1 -- 172.21.15.43:0/1743125573 wait complete. 2024-04-21T08:45:35.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.037+0000 7f683024d740 1 -- start start 2024-04-21T08:45:35.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.037+0000 7f683024d740 1 Processor -- start 2024-04-21T08:45:35.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.037+0000 7f683024d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592843c54d0 0x5592843d2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.037+0000 7f683024d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843c5e80 0x5592843d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.037+0000 7f683024d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592843cea00 0x5592843d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.037+0000 7f683024d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f68281b08d0 con 0x5592843c5e80 2024-04-21T08:45:35.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.037+0000 7f683024d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f68281b0200 con 0x5592843c54d0 2024-04-21T08:45:35.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.037+0000 7f683024d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68281ad440 con 0x5592843cea00 2024-04-21T08:45:35.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.037+0000 7f683024d740 10 client.0 _ll_register_callbacks cb 0x7f68200328a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:35.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.037+0000 7f681ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843c5e80 0x5592843d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.037+0000 7f681ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843c5e80 0x5592843d2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39348/0 (socket says 172.21.15.43:39348) 2024-04-21T08:45:35.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.037+0000 7f681ffff640 1 -- 172.21.15.43:0/3081725581 learned_addr learned my addr 172.21.15.43:0/3081725581 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:35.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.037+0000 7f68277fe640 1 --2- 172.21.15.43:0/3081725581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592843cea00 0x5592843d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.037+0000 7f6826ffd640 1 --2- 172.21.15.43:0/3081725581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592843c54d0 0x5592843d2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.037+0000 7f681ffff640 1 -- 172.21.15.43:0/3081725581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592843cea00 msgr2=0x5592843d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:35.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.037+0000 7f681ffff640 1 --2- 172.21.15.43:0/3081725581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592843cea00 0x5592843d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.037+0000 7f681ffff640 1 -- 172.21.15.43:0/3081725581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592843c54d0 msgr2=0x5592843d2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:35.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.037+0000 7f681ffff640 1 --2- 172.21.15.43:0/3081725581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592843c54d0 0x5592843d2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.037+0000 7f681ffff640 1 -- 172.21.15.43:0/3081725581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559284444510 con 0x5592843c5e80 2024-04-21T08:45:35.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.037+0000 7f6826ffd640 1 --2- 172.21.15.43:0/3081725581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592843c54d0 0x5592843d2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:35.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.038+0000 7f681ffff640 1 --2- 172.21.15.43:0/3081725581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843c5e80 0x5592843d2f70 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f681009c310 tx=0x7f68100d0b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed020dc08fac6e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:35.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.038+0000 7f680e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:35.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.038+0000 7f680e7fc640 1 -- 172.21.15.43:0/3081725581 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68100b4070 con 0x5592843c5e80 2024-04-21T08:45:35.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.038+0000 7f680e7fc640 1 -- 172.21.15.43:0/3081725581 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f68100989f0 con 0x5592843c5e80 2024-04-21T08:45:35.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.038+0000 7f683024d740 1 -- 172.21.15.43:0/3081725581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6818011440 con 0x5592843c5e80 2024-04-21T08:45:35.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.038+0000 7f680e7fc640 1 -- 172.21.15.43:0/3081725581 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6810009de0 con 0x5592843c5e80 2024-04-21T08:45:35.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.038+0000 7f683024d740 10 client.18609 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:35.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.038+0000 7f683024d740 1 -- 172.21.15.43:0/3081725581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559284445810 con 0x5592843c5e80 2024-04-21T08:45:35.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.038+0000 7f680e7fc640 1 -- 172.21.15.43:0/3081725581 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6818011440 con 0x5592843c5e80 2024-04-21T08:45:35.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.038+0000 7f680e7fc640 1 -- 172.21.15.43:0/3081725581 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f68100bb100 con 0x5592843c5e80 2024-04-21T08:45:35.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.038+0000 7f683024d740 10 client.18609 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:35.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.038+0000 7f683024d740 10 client.18609 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:45:35.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.038+0000 7f683024d740 20 client.18609 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:35.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.038+0000 7f683024d740 1 --2- 172.21.15.43:0/3081725581 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x559284445aa0 0x559284447e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.038+0000 7f683024d740 4 client.18609 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:45:35.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.038+0000 7f683024d740 1 -- 172.21.15.43:0/3081725581 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559284337c10 con 0x559284445aa0 2024-04-21T08:45:35.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.038+0000 7f6826ffd640 1 --2- 172.21.15.43:0/3081725581 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x559284445aa0 0x559284447e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.039+0000 7f6826ffd640 1 --2- 172.21.15.43:0/3081725581 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x559284445aa0 0x559284447e90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d8b55775702fd73e server_cookie=4dbb00c22972d468 in_seq=0 out_seq=0 2024-04-21T08:45:35.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.039+0000 7f680e7fc640 10 client.18609 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:45:35.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.039+0000 7f680e7fc640 1 -- 172.21.15.43:0/3081725581 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559284337c10 con 0x559284445aa0 2024-04-21T08:45:35.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.039+0000 7f680e7fc640 10 client.18609 handle_command_reply: tid=0 2024-04-21T08:45:35.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.039+0000 7f683024d740 1 client.18609 shutdown 2024-04-21T08:45:35.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.039+0000 7f683024d740 1 -- 172.21.15.43:0/3081725581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843c5e80 msgr2=0x5592843d2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.039+0000 7f683024d740 1 --2- 172.21.15.43:0/3081725581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843c5e80 0x5592843d2f70 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f681009c310 tx=0x7f68100d0b10 comp rx=0 tx=0).stop 2024-04-21T08:45:35.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.039+0000 7f683024d740 1 -- 172.21.15.43:0/3081725581 shutdown_connections 2024-04-21T08:45:35.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.039+0000 7f683024d740 1 --2- 172.21.15.43:0/3081725581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5592843cea00 0x5592843d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.039+0000 7f683024d740 1 --2- 172.21.15.43:0/3081725581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592843c5e80 0x5592843d2f70 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.039+0000 7f683024d740 1 --2- 172.21.15.43:0/3081725581 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x559284445aa0 0x559284447e90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.039+0000 7f683024d740 1 --2- 172.21.15.43:0/3081725581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592843c54d0 0x5592843d2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.039+0000 7f680e7fc640 0 client.18609 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:45:35.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.039+0000 7f683024d740 1 -- 172.21.15.43:0/3081725581 >> 172.21.15.43:0/3081725581 conn(0x5592843bfba0 msgr2=0x5592843bec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:35.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.039+0000 7f683024d740 1 -- 172.21.15.43:0/3081725581 shutdown_connections 2024-04-21T08:45:35.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.039+0000 7f683024d740 1 -- 172.21.15.43:0/3081725581 wait complete. 2024-04-21T08:45:35.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.039+0000 7f683024d740 20 client.18609 trim_cache size 0 max 16384 2024-04-21T08:45:35.046 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:45:35.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.040+0000 7f682d65c640 1 -- 172.21.15.43:0/4144911602 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f68140a3d20 msgr2=0x7f68140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.040+0000 7f682d65c640 1 --2- 172.21.15.43:0/4144911602 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f68140a3d20 0x7f68140a61e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6820005e30 tx=0x7f6820005d80 comp rx=0 tx=0).stop 2024-04-21T08:45:35.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.040+0000 7f682d65c640 1 -- 172.21.15.43:0/4144911602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6828107260 msgr2=0x7f6828198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.040+0000 7f682d65c640 1 --2- 172.21.15.43:0/4144911602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6828107260 0x7f6828198c50 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f6810005fd0 tx=0x7f681000dde0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.040+0000 7f682d65c640 1 -- 172.21.15.43:0/4144911602 shutdown_connections 2024-04-21T08:45:35.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.040+0000 7f682d65c640 1 --2- 172.21.15.43:0/4144911602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f682819cfd0 0x7f682819d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.040+0000 7f682d65c640 1 --2- 172.21.15.43:0/4144911602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6828107260 0x7f6828198c50 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.040+0000 7f682d65c640 1 --2- 172.21.15.43:0/4144911602 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f68140a3d20 0x7f68140a61e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.040+0000 7f682d65c640 1 --2- 172.21.15.43:0/4144911602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6828102190 0x7f6828198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.040+0000 7f682d65c640 1 -- 172.21.15.43:0/4144911602 >> 172.21.15.43:0/4144911602 conn(0x7f68280793a0 msgr2=0x7f68280ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:35.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.040+0000 7f682d65c640 1 -- 172.21.15.43:0/4144911602 shutdown_connections 2024-04-21T08:45:35.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.040+0000 7f682d65c640 1 -- 172.21.15.43:0/4144911602 wait complete. 2024-04-21T08:45:35.050 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:45:35.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.155+0000 7fe26707a640 1 -- 172.21.15.43:0/2663147422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2601010a0 msgr2=0x7fe260101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.155+0000 7fe26707a640 1 --2- 172.21.15.43:0/2663147422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2601010a0 0x7fe260101480 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fe248009900 tx=0x7fe24802bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.155+0000 7fe26707a640 1 -- 172.21.15.43:0/2663147422 shutdown_connections 2024-04-21T08:45:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.155+0000 7fe26707a640 1 --2- 172.21.15.43:0/2663147422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe260106170 0x7fe26010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.155+0000 7fe26707a640 1 --2- 172.21.15.43:0/2663147422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe260101a50 0x7fe260105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.155+0000 7fe26707a640 1 --2- 172.21.15.43:0/2663147422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2601010a0 0x7fe260101480 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.155+0000 7fe26707a640 1 -- 172.21.15.43:0/2663147422 >> 172.21.15.43:0/2663147422 conn(0x7fe260077eb0 msgr2=0x7fe2600782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.155+0000 7fe26707a640 1 -- 172.21.15.43:0/2663147422 shutdown_connections 2024-04-21T08:45:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.155+0000 7fe26707a640 1 -- 172.21.15.43:0/2663147422 wait complete. 2024-04-21T08:45:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.155+0000 7fe26707a640 1 Processor -- start 2024-04-21T08:45:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.155+0000 7fe26707a640 1 -- start start 2024-04-21T08:45:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.155+0000 7fe26707a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2601010a0 0x7fe2601a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.155+0000 7fe26707a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe260106170 0x7fe2601a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.156+0000 7fe26707a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe26019a790 0x7fe26019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.156+0000 7fe26707a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe260111130 con 0x7fe2601010a0 2024-04-21T08:45:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.156+0000 7fe26707a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe260110fb0 con 0x7fe26019a790 2024-04-21T08:45:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.156+0000 7fe26707a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe2601112b0 con 0x7fe260106170 2024-04-21T08:45:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.156+0000 7fe264e16640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2601010a0 0x7fe2601a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.156+0000 7fe264e16640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2601010a0 0x7fe2601a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39368/0 (socket says 172.21.15.43:39368) 2024-04-21T08:45:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.156+0000 7fe264e16640 1 -- 172.21.15.43:0/636726533 learned_addr learned my addr 172.21.15.43:0/636726533 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.156+0000 7fe265617640 1 --2- 172.21.15.43:0/636726533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe26019a790 0x7fe26019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.156+0000 7fe264e16640 1 -- 172.21.15.43:0/636726533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe260106170 msgr2=0x7fe2601a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.156+0000 7fe25ffff640 1 --2- 172.21.15.43:0/636726533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe260106170 0x7fe2601a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.156+0000 7fe264e16640 1 --2- 172.21.15.43:0/636726533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe260106170 0x7fe2601a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.156+0000 7fe264e16640 1 -- 172.21.15.43:0/636726533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe26019a790 msgr2=0x7fe26019ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.156+0000 7fe264e16640 1 --2- 172.21.15.43:0/636726533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe26019a790 0x7fe26019ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.156+0000 7fe264e16640 1 -- 172.21.15.43:0/636726533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26019b320 con 0x7fe2601010a0 2024-04-21T08:45:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.156+0000 7fe265617640 1 --2- 172.21.15.43:0/636726533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe26019a790 0x7fe26019ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.156+0000 7fe264e16640 1 --2- 172.21.15.43:0/636726533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2601010a0 0x7fe2601a0620 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fe248005940 tx=0x7fe24802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.156+0000 7fe25dffb640 1 -- 172.21.15.43:0/636726533 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe248043020 con 0x7fe2601010a0 2024-04-21T08:45:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.156+0000 7fe25dffb640 1 -- 172.21.15.43:0/636726533 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe24802cbe0 con 0x7fe2601010a0 2024-04-21T08:45:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.156+0000 7fe26707a640 1 -- 172.21.15.43:0/636726533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26019b5b0 con 0x7fe2601010a0 2024-04-21T08:45:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.156+0000 7fe25dffb640 1 -- 172.21.15.43:0/636726533 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe248003bf0 con 0x7fe2601010a0 2024-04-21T08:45:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.157+0000 7fe26707a640 1 -- 172.21.15.43:0/636726533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2601a7790 con 0x7fe2601010a0 2024-04-21T08:45:35.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.159+0000 7fe25dffb640 1 -- 172.21.15.43:0/636726533 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe248046020 con 0x7fe2601010a0 2024-04-21T08:45:35.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.159+0000 7fe269c6a740 1 Processor -- start 2024-04-21T08:45:35.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.159+0000 7fe269c6a740 1 -- start start 2024-04-21T08:45:35.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.159+0000 7fe25dffb640 1 --2- 172.21.15.43:0/636726533 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe24c0a3d20 0x7fe24c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.159+0000 7fe269c6a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638dd8274d0 0x5638dd8278b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.159+0000 7fe269c6a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638dd827e80 0x5638dd830430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.159+0000 7fe269c6a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd830a00 0x5638dd832e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.159+0000 7fe269c6a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5638dd78c1f0 con 0x5638dd830a00 2024-04-21T08:45:35.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.159+0000 7fe269c6a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5638dd7294b0 con 0x5638dd8274d0 2024-04-21T08:45:35.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.159+0000 7fe269c6a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5638dd7881b0 con 0x5638dd827e80 2024-04-21T08:45:35.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.159+0000 7fe25dffb640 1 -- 172.21.15.43:0/636726533 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fe2480f2d50 con 0x7fe2601010a0 2024-04-21T08:45:35.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.159+0000 7fe265617640 1 --2- 172.21.15.43:0/636726533 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe24c0a3d20 0x7fe24c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.159+0000 7fe265617640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd830a00 0x5638dd832e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.159+0000 7fe265617640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd830a00 0x5638dd832e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39384/0 (socket says 172.21.15.43:39384) 2024-04-21T08:45:35.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.160+0000 7fe265617640 1 -- 172.21.15.43:0/1600981154 learned_addr learned my addr 172.21.15.43:0/1600981154 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:35.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.160+0000 7fe264e16640 1 --2- 172.21.15.43:0/1600981154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638dd8274d0 0x5638dd8278b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.160+0000 7fe25ffff640 1 --2- 172.21.15.43:0/1600981154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638dd827e80 0x5638dd830430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.160+0000 7fe265617640 1 -- 172.21.15.43:0/1600981154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638dd827e80 msgr2=0x5638dd830430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:35.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.160+0000 7fe265617640 1 --2- 172.21.15.43:0/1600981154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638dd827e80 0x5638dd830430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.160+0000 7fe265617640 1 -- 172.21.15.43:0/1600981154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638dd8274d0 msgr2=0x5638dd8278b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:35.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.160+0000 7fe265617640 1 --2- 172.21.15.43:0/1600981154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638dd8274d0 0x5638dd8278b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.160+0000 7fe265617640 1 -- 172.21.15.43:0/1600981154 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638dd71b1b0 con 0x5638dd830a00 2024-04-21T08:45:35.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.160+0000 7fe265617640 1 --2- 172.21.15.43:0/636726533 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe24c0a3d20 0x7fe24c0a61e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe258014a40 tx=0x7fe25800cc10 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:35.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.160+0000 7fe265617640 1 --2- 172.21.15.43:0/1600981154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd830a00 0x5638dd832e50 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fe258011ce0 tx=0x7fe25801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a581de474e28ca server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:35.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.160+0000 7fe256ffd640 1 -- 172.21.15.43:0/1600981154 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe258022070 con 0x5638dd830a00 2024-04-21T08:45:35.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.160+0000 7fe256ffd640 1 -- 172.21.15.43:0/1600981154 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe25800fc60 con 0x5638dd830a00 2024-04-21T08:45:35.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.160+0000 7fe256ffd640 1 -- 172.21.15.43:0/1600981154 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe258005350 con 0x5638dd830a00 2024-04-21T08:45:35.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.161+0000 7fe269c6a740 1 -- 172.21.15.43:0/1600981154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd830a00 msgr2=0x5638dd832e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.161+0000 7fe269c6a740 1 --2- 172.21.15.43:0/1600981154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd830a00 0x5638dd832e50 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fe258011ce0 tx=0x7fe25801b000 comp rx=0 tx=0).stop 2024-04-21T08:45:35.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.161+0000 7fe269c6a740 1 -- 172.21.15.43:0/1600981154 shutdown_connections 2024-04-21T08:45:35.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.161+0000 7fe269c6a740 1 --2- 172.21.15.43:0/1600981154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd830a00 0x5638dd832e50 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.161+0000 7fe269c6a740 1 --2- 172.21.15.43:0/1600981154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638dd827e80 0x5638dd830430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.161+0000 7fe269c6a740 1 --2- 172.21.15.43:0/1600981154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638dd8274d0 0x5638dd8278b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.161+0000 7fe269c6a740 1 -- 172.21.15.43:0/1600981154 >> 172.21.15.43:0/1600981154 conn(0x5638dd821ba0 msgr2=0x5638dd821f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:35.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.161+0000 7fe269c6a740 1 -- 172.21.15.43:0/1600981154 shutdown_connections 2024-04-21T08:45:35.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.161+0000 7fe269c6a740 1 -- 172.21.15.43:0/1600981154 wait complete. 2024-04-21T08:45:35.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.161+0000 7fe269c6a740 1 -- start start 2024-04-21T08:45:35.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.161+0000 7fe269c6a740 1 Processor -- start 2024-04-21T08:45:35.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.162+0000 7fe269c6a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638dd8274d0 0x5638dd830320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.162+0000 7fe269c6a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638dd827e80 0x5638dd82d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.162+0000 7fe269c6a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd830a00 0x5638dd82dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.162+0000 7fe269c6a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5638dd833910 con 0x5638dd830a00 2024-04-21T08:45:35.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.162+0000 7fe269c6a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5638dd745500 con 0x5638dd8274d0 2024-04-21T08:45:35.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.162+0000 7fe269c6a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5638dd833a90 con 0x5638dd827e80 2024-04-21T08:45:35.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.162+0000 7fe269c6a740 10 client.0 _ll_register_callbacks cb 0x5638dd713b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:35.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.162+0000 7fe264e16640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638dd8274d0 0x5638dd830320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.162+0000 7fe264e16640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638dd8274d0 0x5638dd830320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:38426/0 (socket says 172.21.15.43:38426) 2024-04-21T08:45:35.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.162+0000 7fe265617640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd830a00 0x5638dd82dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.162+0000 7fe264e16640 1 -- 172.21.15.43:0/1654897113 learned_addr learned my addr 172.21.15.43:0/1654897113 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:35.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.162+0000 7fe25ffff640 1 --2- 172.21.15.43:0/1654897113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638dd827e80 0x5638dd82d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.162+0000 7fe264e16640 1 -- 172.21.15.43:0/1654897113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638dd827e80 msgr2=0x5638dd82d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:35.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.162+0000 7fe264e16640 1 --2- 172.21.15.43:0/1654897113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638dd827e80 0x5638dd82d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.162+0000 7fe264e16640 1 -- 172.21.15.43:0/1654897113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd830a00 msgr2=0x5638dd82dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:35.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.162+0000 7fe264e16640 1 --2- 172.21.15.43:0/1654897113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd830a00 0x5638dd82dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.162+0000 7fe264e16640 1 -- 172.21.15.43:0/1654897113 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638dd49e3f0 con 0x5638dd8274d0 2024-04-21T08:45:35.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.162+0000 7fe265617640 1 --2- 172.21.15.43:0/1654897113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd830a00 0x5638dd82dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:45:35.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.162+0000 7fe264e16640 1 --2- 172.21.15.43:0/1654897113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638dd8274d0 0x5638dd830320 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe2480bae90 tx=0x7fe2480be450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de3983772ae99b7b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:35.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.162+0000 7fe23ffff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-21T08:45:35.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.163+0000 7fe23ffff640 1 -- 172.21.15.43:0/1654897113 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2480bf130 con 0x5638dd8274d0 2024-04-21T08:45:35.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.163+0000 7fe269c6a740 1 -- 172.21.15.43:0/1654897113 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe25800fc60 con 0x5638dd8274d0 2024-04-21T08:45:35.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.163+0000 7fe23ffff640 1 -- 172.21.15.43:0/1654897113 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fe2480054d0 con 0x5638dd8274d0 2024-04-21T08:45:35.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.163+0000 7fe23ffff640 1 -- 172.21.15.43:0/1654897113 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe2480bea30 con 0x5638dd8274d0 2024-04-21T08:45:35.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.163+0000 7fe269c6a740 10 client.27620 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:35.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.163+0000 7fe269c6a740 1 -- 172.21.15.43:0/1654897113 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5638dd5687d0 con 0x5638dd8274d0 2024-04-21T08:45:35.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.163+0000 7fe23ffff640 1 -- 172.21.15.43:0/1654897113 <== mon.2 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe25800fc60 con 0x5638dd8274d0 2024-04-21T08:45:35.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.163+0000 7fe23ffff640 1 -- 172.21.15.43:0/1654897113 <== mon.2 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe2480a28f0 con 0x5638dd8274d0 2024-04-21T08:45:35.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.164+0000 7fe269c6a740 10 client.27620 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:35.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.164+0000 7fe269c6a740 10 client.27620 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:45:35.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.164+0000 7fe269c6a740 20 client.27620 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:35.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.164+0000 7fe269c6a740 1 --2- 172.21.15.43:0/1654897113 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5638dd840ba0 0x5638dd842fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.164+0000 7fe269c6a740 4 client.27620 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:35.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.164+0000 7fe269c6a740 1 -- 172.21.15.43:0/1654897113 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638dd72b930 con 0x5638dd840ba0 2024-04-21T08:45:35.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.164+0000 7fe25ffff640 1 --2- 172.21.15.43:0/1654897113 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5638dd840ba0 0x5638dd842fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.164+0000 7fe23ffff640 10 client.27620 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:45:35.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.164+0000 7fe25ffff640 1 --2- 172.21.15.43:0/1654897113 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5638dd840ba0 0x5638dd842fb0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9568120ff155cd6c server_cookie=49eaf08116fcdab5 in_seq=0 out_seq=0 2024-04-21T08:45:35.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.165+0000 7fe23ffff640 1 -- 172.21.15.43:0/1654897113 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5638dd72b930 con 0x5638dd840ba0 2024-04-21T08:45:35.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.165+0000 7fe23ffff640 10 client.27620 handle_command_reply: tid=0 2024-04-21T08:45:35.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.165+0000 7fe269c6a740 1 client.27620 shutdown 2024-04-21T08:45:35.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.165+0000 7fe269c6a740 1 -- 172.21.15.43:0/1654897113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638dd8274d0 msgr2=0x5638dd830320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.165+0000 7fe269c6a740 1 --2- 172.21.15.43:0/1654897113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638dd8274d0 0x5638dd830320 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe2480bae90 tx=0x7fe2480be450 comp rx=0 tx=0).stop 2024-04-21T08:45:35.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.165+0000 7fe269c6a740 1 -- 172.21.15.43:0/1654897113 shutdown_connections 2024-04-21T08:45:35.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.165+0000 7fe269c6a740 1 --2- 172.21.15.43:0/1654897113 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5638dd840ba0 0x5638dd842fb0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.165+0000 7fe269c6a740 1 --2- 172.21.15.43:0/1654897113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd830a00 0x5638dd82dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.165+0000 7fe23ffff640 0 client.27620 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:45:35.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.165+0000 7fe269c6a740 1 --2- 172.21.15.43:0/1654897113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638dd827e80 0x5638dd82d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.165+0000 7fe269c6a740 1 --2- 172.21.15.43:0/1654897113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638dd8274d0 0x5638dd830320 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.165+0000 7fe269c6a740 1 -- 172.21.15.43:0/1654897113 >> 172.21.15.43:0/1654897113 conn(0x5638dd821ba0 msgr2=0x5638dd8238f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:35.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.165+0000 7fe269c6a740 1 -- 172.21.15.43:0/1654897113 shutdown_connections 2024-04-21T08:45:35.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.165+0000 7fe269c6a740 1 -- 172.21.15.43:0/1654897113 wait complete. 2024-04-21T08:45:35.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.165+0000 7fe269c6a740 20 client.27620 trim_cache size 0 max 16384 2024-04-21T08:45:35.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.179+0000 7fe269c6a740 1 Processor -- start 2024-04-21T08:45:35.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.179+0000 7fe269c6a740 1 -- start start 2024-04-21T08:45:35.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.179+0000 7fe269c6a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd8274d0 0x5638dd83c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.179+0000 7fe269c6a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638dd827e80 0x5638dd82fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.179+0000 7fe269c6a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638dd830a00 0x5638dd830320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.179+0000 7fe269c6a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe2601b15e0 con 0x5638dd8274d0 2024-04-21T08:45:35.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.179+0000 7fe269c6a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe2601b3120 con 0x5638dd827e80 2024-04-21T08:45:35.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.179+0000 7fe269c6a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe2601b1cb0 con 0x5638dd830a00 2024-04-21T08:45:35.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.179+0000 7fe264e16640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd8274d0 0x5638dd83c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.179+0000 7fe264e16640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd8274d0 0x5638dd83c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39406/0 (socket says 172.21.15.43:39406) 2024-04-21T08:45:35.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.179+0000 7fe264e16640 1 -- 172.21.15.43:0/1528884003 learned_addr learned my addr 172.21.15.43:0/1528884003 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:35.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.179+0000 7fe25ffff640 1 --2- 172.21.15.43:0/1528884003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638dd827e80 0x5638dd82fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.179+0000 7fe265617640 1 --2- 172.21.15.43:0/1528884003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638dd830a00 0x5638dd830320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.179+0000 7fe264e16640 1 -- 172.21.15.43:0/1528884003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638dd830a00 msgr2=0x5638dd830320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:35.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.179+0000 7fe264e16640 1 --2- 172.21.15.43:0/1528884003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638dd830a00 0x5638dd830320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.179+0000 7fe264e16640 1 -- 172.21.15.43:0/1528884003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638dd827e80 msgr2=0x5638dd82fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:35.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.179+0000 7fe264e16640 1 --2- 172.21.15.43:0/1528884003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638dd827e80 0x5638dd82fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.179+0000 7fe264e16640 1 -- 172.21.15.43:0/1528884003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638dd827c20 con 0x5638dd8274d0 2024-04-21T08:45:35.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.179+0000 7fe25ffff640 1 --2- 172.21.15.43:0/1528884003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638dd827e80 0x5638dd82fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:35.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.179+0000 7fe264e16640 1 --2- 172.21.15.43:0/1528884003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd8274d0 0x5638dd83c820 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fe248005ec0 tx=0x7fe24802cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7a398772c8cabb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:35.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.181+0000 7fe2567fc640 1 -- 172.21.15.43:0/1528884003 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2480df0b0 con 0x5638dd8274d0 2024-04-21T08:45:35.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.181+0000 7fe2567fc640 1 -- 172.21.15.43:0/1528884003 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2480bf130 con 0x5638dd8274d0 2024-04-21T08:45:35.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.181+0000 7fe2567fc640 1 -- 172.21.15.43:0/1528884003 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2480bacd0 con 0x5638dd8274d0 2024-04-21T08:45:35.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.182+0000 7fe269c6a740 1 -- 172.21.15.43:0/1528884003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd8274d0 msgr2=0x5638dd83c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.182+0000 7fe269c6a740 1 --2- 172.21.15.43:0/1528884003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd8274d0 0x5638dd83c820 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fe248005ec0 tx=0x7fe24802cd90 comp rx=0 tx=0).stop 2024-04-21T08:45:35.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.182+0000 7fe269c6a740 1 -- 172.21.15.43:0/1528884003 shutdown_connections 2024-04-21T08:45:35.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.182+0000 7fe269c6a740 1 --2- 172.21.15.43:0/1528884003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638dd830a00 0x5638dd830320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.182+0000 7fe269c6a740 1 --2- 172.21.15.43:0/1528884003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638dd827e80 0x5638dd82fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.182+0000 7fe269c6a740 1 --2- 172.21.15.43:0/1528884003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd8274d0 0x5638dd83c820 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.182+0000 7fe269c6a740 1 -- 172.21.15.43:0/1528884003 >> 172.21.15.43:0/1528884003 conn(0x5638dd821ba0 msgr2=0x5638dd82d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:35.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.182+0000 7fe269c6a740 1 -- 172.21.15.43:0/1528884003 shutdown_connections 2024-04-21T08:45:35.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.182+0000 7fe269c6a740 1 -- 172.21.15.43:0/1528884003 wait complete. 2024-04-21T08:45:35.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.182+0000 7fe269c6a740 1 -- start start 2024-04-21T08:45:35.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.182+0000 7fe269c6a740 1 Processor -- start 2024-04-21T08:45:35.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.183+0000 7fe269c6a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638dd8274d0 0x5638dd838d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.183+0000 7fe269c6a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd827e80 0x5638dd839270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.183+0000 7fe269c6a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638dd830a00 0x5638dd829a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.183+0000 7fe269c6a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe2601b2a50 con 0x5638dd827e80 2024-04-21T08:45:35.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.183+0000 7fe269c6a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe2601b2380 con 0x5638dd8274d0 2024-04-21T08:45:35.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.183+0000 7fe269c6a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe2601af5c0 con 0x5638dd830a00 2024-04-21T08:45:35.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.183+0000 7fe25ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd827e80 0x5638dd839270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.183+0000 7fe269c6a740 10 client.0 _ll_register_callbacks cb 0x7fe2480be620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:35.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.183+0000 7fe25ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd827e80 0x5638dd839270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39418/0 (socket says 172.21.15.43:39418) 2024-04-21T08:45:35.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.183+0000 7fe25ffff640 1 -- 172.21.15.43:0/278300848 learned_addr learned my addr 172.21.15.43:0/278300848 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:35.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.183+0000 7fe264e16640 1 --2- 172.21.15.43:0/278300848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638dd8274d0 0x5638dd838d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.183+0000 7fe265617640 1 --2- 172.21.15.43:0/278300848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638dd830a00 0x5638dd829a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.183+0000 7fe25ffff640 1 -- 172.21.15.43:0/278300848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638dd830a00 msgr2=0x5638dd829a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:35.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.183+0000 7fe25ffff640 1 --2- 172.21.15.43:0/278300848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638dd830a00 0x5638dd829a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.183+0000 7fe25ffff640 1 -- 172.21.15.43:0/278300848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638dd8274d0 msgr2=0x5638dd838d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:35.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.183+0000 7fe25ffff640 1 --2- 172.21.15.43:0/278300848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638dd8274d0 0x5638dd838d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.183+0000 7fe25ffff640 1 -- 172.21.15.43:0/278300848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638dd82a100 con 0x5638dd827e80 2024-04-21T08:45:35.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.184+0000 7fe264e16640 1 --2- 172.21.15.43:0/278300848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638dd8274d0 0x5638dd838d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:45:35.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.184+0000 7fe25ffff640 1 --2- 172.21.15.43:0/278300848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd827e80 0x5638dd839270 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fe250004b90 tx=0x7fe2500128a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c362d4920fa1a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:35.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.184+0000 7fe23ffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:35.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.186+0000 7fe23ffff640 1 -- 172.21.15.43:0/278300848 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25001b3b0 con 0x5638dd827e80 2024-04-21T08:45:35.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.186+0000 7fe23ffff640 1 -- 172.21.15.43:0/278300848 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fe25001b550 con 0x5638dd827e80 2024-04-21T08:45:35.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.186+0000 7fe269c6a740 1 -- 172.21.15.43:0/278300848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2480bf130 con 0x5638dd827e80 2024-04-21T08:45:35.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.186+0000 7fe23ffff640 1 -- 172.21.15.43:0/278300848 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe25002d050 con 0x5638dd827e80 2024-04-21T08:45:35.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.186+0000 7fe269c6a740 10 client.18627 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:35.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.186+0000 7fe269c6a740 1 -- 172.21.15.43:0/278300848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5638dd8a6ae0 con 0x5638dd827e80 2024-04-21T08:45:35.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.186+0000 7fe23ffff640 1 -- 172.21.15.43:0/278300848 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2480bf130 con 0x5638dd827e80 2024-04-21T08:45:35.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.186+0000 7fe23ffff640 1 -- 172.21.15.43:0/278300848 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe250011040 con 0x5638dd827e80 2024-04-21T08:45:35.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.186+0000 7fe269c6a740 10 client.18627 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:35.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.186+0000 7fe269c6a740 10 client.18627 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:45:35.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.186+0000 7fe269c6a740 20 client.18627 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:35.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.186+0000 7fe269c6a740 1 --2- 172.21.15.43:0/278300848 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5638dd8a7240 0x5638dd8a9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.186+0000 7fe269c6a740 4 client.18627 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:45:35.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.186+0000 7fe269c6a740 1 -- 172.21.15.43:0/278300848 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5638dd799c10 con 0x5638dd8a7240 2024-04-21T08:45:35.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.187+0000 7fe264e16640 1 --2- 172.21.15.43:0/278300848 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5638dd8a7240 0x5638dd8a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.187+0000 7fe264e16640 1 --2- 172.21.15.43:0/278300848 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5638dd8a7240 0x5638dd8a9660 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=40a52e973698058e server_cookie=473d291fe3ba92ec in_seq=0 out_seq=0 2024-04-21T08:45:35.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.187+0000 7fe23ffff640 10 client.18627 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:45:35.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.187+0000 7fe23ffff640 1 -- 172.21.15.43:0/278300848 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5638dd799c10 con 0x5638dd8a7240 2024-04-21T08:45:35.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.187+0000 7fe23ffff640 10 client.18627 handle_command_reply: tid=0 2024-04-21T08:45:35.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.187+0000 7fe269c6a740 1 client.18627 shutdown 2024-04-21T08:45:35.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.187+0000 7fe269c6a740 1 -- 172.21.15.43:0/278300848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd827e80 msgr2=0x5638dd839270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.187+0000 7fe269c6a740 1 --2- 172.21.15.43:0/278300848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd827e80 0x5638dd839270 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fe250004b90 tx=0x7fe2500128a0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.187+0000 7fe269c6a740 1 -- 172.21.15.43:0/278300848 shutdown_connections 2024-04-21T08:45:35.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.187+0000 7fe269c6a740 1 --2- 172.21.15.43:0/278300848 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5638dd8a7240 0x5638dd8a9660 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.187+0000 7fe269c6a740 1 --2- 172.21.15.43:0/278300848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638dd830a00 0x5638dd829a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.187+0000 7fe269c6a740 1 --2- 172.21.15.43:0/278300848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638dd827e80 0x5638dd839270 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.187+0000 7fe23ffff640 0 client.18627 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:45:35.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.187+0000 7fe269c6a740 1 --2- 172.21.15.43:0/278300848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638dd8274d0 0x5638dd838d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.187+0000 7fe269c6a740 1 -- 172.21.15.43:0/278300848 >> 172.21.15.43:0/278300848 conn(0x5638dd821ba0 msgr2=0x5638dd8282b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:35.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.188+0000 7fe269c6a740 1 -- 172.21.15.43:0/278300848 shutdown_connections 2024-04-21T08:45:35.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.188+0000 7fe269c6a740 1 -- 172.21.15.43:0/278300848 wait complete. 2024-04-21T08:45:35.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.188+0000 7fe269c6a740 20 client.18627 trim_cache size 0 max 16384 2024-04-21T08:45:35.191 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:45:35.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.188+0000 7fe26707a640 1 -- 172.21.15.43:0/636726533 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe24c0a3d20 msgr2=0x7fe24c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.188+0000 7fe26707a640 1 --2- 172.21.15.43:0/636726533 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe24c0a3d20 0x7fe24c0a61e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe258014a40 tx=0x7fe25800cc10 comp rx=0 tx=0).stop 2024-04-21T08:45:35.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.188+0000 7fe26707a640 1 -- 172.21.15.43:0/636726533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2601010a0 msgr2=0x7fe2601a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.188+0000 7fe26707a640 1 --2- 172.21.15.43:0/636726533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2601010a0 0x7fe2601a0620 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fe248005940 tx=0x7fe24802c7e0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.188+0000 7fe26707a640 1 -- 172.21.15.43:0/636726533 shutdown_connections 2024-04-21T08:45:35.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.188+0000 7fe26707a640 1 --2- 172.21.15.43:0/636726533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe26019a790 0x7fe26019ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.188+0000 7fe26707a640 1 --2- 172.21.15.43:0/636726533 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe24c0a3d20 0x7fe24c0a61e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.188+0000 7fe26707a640 1 --2- 172.21.15.43:0/636726533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe260106170 0x7fe2601a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.188+0000 7fe26707a640 1 --2- 172.21.15.43:0/636726533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2601010a0 0x7fe2601a0620 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.188+0000 7fe26707a640 1 -- 172.21.15.43:0/636726533 >> 172.21.15.43:0/636726533 conn(0x7fe260077eb0 msgr2=0x7fe260100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:35.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.188+0000 7fe26707a640 1 -- 172.21.15.43:0/636726533 shutdown_connections 2024-04-21T08:45:35.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.188+0000 7fe26707a640 1 -- 172.21.15.43:0/636726533 wait complete. 2024-04-21T08:45:35.198 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:45:35.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.303+0000 7f76403ce640 1 -- 172.21.15.43:0/627836947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7638102190 msgr2=0x7f7638102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.303+0000 7f76403ce640 1 --2- 172.21.15.43:0/627836947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7638102190 0x7f7638102570 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f7630009900 tx=0x7f763002bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:35.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.303+0000 7f76403ce640 1 -- 172.21.15.43:0/627836947 shutdown_connections 2024-04-21T08:45:35.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.303+0000 7f76403ce640 1 --2- 172.21.15.43:0/627836947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7638107260 0x7f763810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.303+0000 7f76403ce640 1 --2- 172.21.15.43:0/627836947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7638102b40 0x7f7638106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.303+0000 7f76403ce640 1 --2- 172.21.15.43:0/627836947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7638102190 0x7f7638102570 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.303+0000 7f76403ce640 1 -- 172.21.15.43:0/627836947 >> 172.21.15.43:0/627836947 conn(0x7f76380793a0 msgr2=0x7f76380797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:35.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.303+0000 7f76403ce640 1 -- 172.21.15.43:0/627836947 shutdown_connections 2024-04-21T08:45:35.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.303+0000 7f76403ce640 1 -- 172.21.15.43:0/627836947 wait complete. 2024-04-21T08:45:35.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.304+0000 7f76403ce640 1 Processor -- start 2024-04-21T08:45:35.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.304+0000 7f76403ce640 1 -- start start 2024-04-21T08:45:35.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.304+0000 7f76403ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7638102190 0x7f7638074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.304+0000 7f76403ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7638107260 0x7f7638074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.304+0000 7f76403ce640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7638074cd0 0x7f76381a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.304+0000 7f76403ce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f76381121d0 con 0x7f7638102190 2024-04-21T08:45:35.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.304+0000 7f76403ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7638112050 con 0x7f7638107260 2024-04-21T08:45:35.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.304+0000 7f76403ce640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7638112350 con 0x7f7638074cd0 2024-04-21T08:45:35.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.304+0000 7f763e16a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7638102190 0x7f7638074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.304+0000 7f763e16a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7638102190 0x7f7638074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39432/0 (socket says 172.21.15.43:39432) 2024-04-21T08:45:35.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.304+0000 7f763e16a640 1 -- 172.21.15.43:0/2293142698 learned_addr learned my addr 172.21.15.43:0/2293142698 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:35.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.304+0000 7f763e96b640 1 --2- 172.21.15.43:0/2293142698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7638074cd0 0x7f76381a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.304+0000 7f763d969640 1 --2- 172.21.15.43:0/2293142698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7638107260 0x7f7638074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.304+0000 7f763e16a640 1 -- 172.21.15.43:0/2293142698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7638074cd0 msgr2=0x7f76381a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.304+0000 7f763e16a640 1 --2- 172.21.15.43:0/2293142698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7638074cd0 0x7f76381a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.304+0000 7f763e16a640 1 -- 172.21.15.43:0/2293142698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7638107260 msgr2=0x7f7638074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.304+0000 7f763e16a640 1 --2- 172.21.15.43:0/2293142698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7638107260 0x7f7638074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.304+0000 7f763e16a640 1 -- 172.21.15.43:0/2293142698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76381a7040 con 0x7f7638102190 2024-04-21T08:45:35.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.304+0000 7f763e96b640 1 --2- 172.21.15.43:0/2293142698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7638074cd0 0x7f76381a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:35.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.305+0000 7f763e16a640 1 --2- 172.21.15.43:0/2293142698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7638102190 0x7f7638074250 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f7630005940 tx=0x7f763002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:35.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.305+0000 7f76377fe640 1 -- 172.21.15.43:0/2293142698 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7630043020 con 0x7f7638102190 2024-04-21T08:45:35.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.305+0000 7f76403ce640 1 -- 172.21.15.43:0/2293142698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76381a72d0 con 0x7f7638102190 2024-04-21T08:45:35.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.305+0000 7f76377fe640 1 -- 172.21.15.43:0/2293142698 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f763002cbe0 con 0x7f7638102190 2024-04-21T08:45:35.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.305+0000 7f76377fe640 1 -- 172.21.15.43:0/2293142698 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7630003bf0 con 0x7f7638102190 2024-04-21T08:45:35.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.305+0000 7f76403ce640 1 -- 172.21.15.43:0/2293142698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76381a7820 con 0x7f7638102190 2024-04-21T08:45:35.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.307+0000 7f76377fe640 1 -- 172.21.15.43:0/2293142698 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7630046020 con 0x7f7638102190 2024-04-21T08:45:35.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.308+0000 7f7642fbf740 1 Processor -- start 2024-04-21T08:45:35.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.308+0000 7f7642fbf740 1 -- start start 2024-04-21T08:45:35.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.308+0000 7f7642fbf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafd14d0 0x5646eafd18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.308+0000 7f7642fbf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646eafd1e80 0x5646eafda430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.308+0000 7f7642fbf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5646eafdaa00 0x5646eafdce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.308+0000 7f7642fbf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5646eaf361f0 con 0x5646eafd14d0 2024-04-21T08:45:35.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.308+0000 7f7642fbf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5646eaed34b0 con 0x5646eafd1e80 2024-04-21T08:45:35.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.308+0000 7f7642fbf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5646eaf321b0 con 0x5646eafdaa00 2024-04-21T08:45:35.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.308+0000 7f76377fe640 1 --2- 172.21.15.43:0/2293142698 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f76200a3d50 0x7f76200a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.308+0000 7f763e96b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafd14d0 0x5646eafd18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.308+0000 7f763e96b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafd14d0 0x5646eafd18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39440/0 (socket says 172.21.15.43:39440) 2024-04-21T08:45:35.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.308+0000 7f763e16a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646eafd1e80 0x5646eafda430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.308+0000 7f763e96b640 1 -- 172.21.15.43:0/3701565994 learned_addr learned my addr 172.21.15.43:0/3701565994 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:35.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.308+0000 7f763e96b640 1 --2- 172.21.15.43:0/2293142698 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f76200a3d50 0x7f76200a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.308+0000 7f763d969640 1 --2- 172.21.15.43:0/3701565994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5646eafdaa00 0x5646eafdce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.309+0000 7f76377fe640 1 -- 172.21.15.43:0/2293142698 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f76300f2dc0 con 0x7f7638102190 2024-04-21T08:45:35.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.309+0000 7f763e16a640 1 -- 172.21.15.43:0/3701565994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5646eafdaa00 msgr2=0x5646eafdce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:35.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.309+0000 7f763e16a640 1 --2- 172.21.15.43:0/3701565994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5646eafdaa00 0x5646eafdce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.309+0000 7f763e16a640 1 -- 172.21.15.43:0/3701565994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafd14d0 msgr2=0x5646eafd18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:35.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.309+0000 7f763e16a640 1 --2- 172.21.15.43:0/3701565994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafd14d0 0x5646eafd18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.309+0000 7f763e16a640 1 -- 172.21.15.43:0/3701565994 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646eaec51b0 con 0x5646eafd1e80 2024-04-21T08:45:35.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.309+0000 7f763e96b640 1 --2- 172.21.15.43:0/3701565994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafd14d0 0x5646eafd18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:45:35.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.309+0000 7f763e96b640 1 --2- 172.21.15.43:0/2293142698 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f76200a3d50 0x7f76200a6210 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7628009fd0 tx=0x7f7628009200 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:35.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.309+0000 7f763e16a640 1 --2- 172.21.15.43:0/3701565994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646eafd1e80 0x5646eafda430 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f76300f0230 tx=0x7f7630091570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ba66c3da7806888 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:35.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.309+0000 7f7634bfe640 1 -- 172.21.15.43:0/3701565994 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7630089590 con 0x5646eafd1e80 2024-04-21T08:45:35.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.309+0000 7f7634bfe640 1 -- 172.21.15.43:0/3701565994 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76300bf180 con 0x5646eafd1e80 2024-04-21T08:45:35.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.309+0000 7f7634bfe640 1 -- 172.21.15.43:0/3701565994 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76300df0b0 con 0x5646eafd1e80 2024-04-21T08:45:35.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.310+0000 7f7642fbf740 1 -- 172.21.15.43:0/3701565994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646eafd1e80 msgr2=0x5646eafda430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.310+0000 7f7642fbf740 1 --2- 172.21.15.43:0/3701565994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646eafd1e80 0x5646eafda430 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f76300f0230 tx=0x7f7630091570 comp rx=0 tx=0).stop 2024-04-21T08:45:35.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.310+0000 7f7642fbf740 1 -- 172.21.15.43:0/3701565994 shutdown_connections 2024-04-21T08:45:35.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.310+0000 7f7642fbf740 1 --2- 172.21.15.43:0/3701565994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5646eafdaa00 0x5646eafdce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.310+0000 7f7642fbf740 1 --2- 172.21.15.43:0/3701565994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646eafd1e80 0x5646eafda430 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.310+0000 7f7642fbf740 1 --2- 172.21.15.43:0/3701565994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafd14d0 0x5646eafd18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.310+0000 7f7642fbf740 1 -- 172.21.15.43:0/3701565994 >> 172.21.15.43:0/3701565994 conn(0x5646eafcbba0 msgr2=0x5646eafcbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:35.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.310+0000 7f7642fbf740 1 -- 172.21.15.43:0/3701565994 shutdown_connections 2024-04-21T08:45:35.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.310+0000 7f7642fbf740 1 -- 172.21.15.43:0/3701565994 wait complete. 2024-04-21T08:45:35.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.310+0000 7f7642fbf740 1 -- start start 2024-04-21T08:45:35.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.311+0000 7f7642fbf740 1 Processor -- start 2024-04-21T08:45:35.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.311+0000 7f7642fbf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646eafd14d0 0x5646eafda2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.311+0000 7f7642fbf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5646eafd1e80 0x5646eafd7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.311+0000 7f7642fbf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafdaa00 0x5646eafd7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.311+0000 7f7642fbf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5646eafdd730 con 0x5646eafdaa00 2024-04-21T08:45:35.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.311+0000 7f7642fbf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5646eaeef500 con 0x5646eafd14d0 2024-04-21T08:45:35.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.311+0000 7f7642fbf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5646eafdd8b0 con 0x5646eafd1e80 2024-04-21T08:45:35.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.311+0000 7f7642fbf740 10 client.0 _ll_register_callbacks cb 0x5646eaebdb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:35.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.311+0000 7f763e96b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafdaa00 0x5646eafd7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.311+0000 7f763e16a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646eafd14d0 0x5646eafda2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.311+0000 7f763e16a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646eafd14d0 0x5646eafda2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:38470/0 (socket says 172.21.15.43:38470) 2024-04-21T08:45:35.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.311+0000 7f763e96b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafdaa00 0x5646eafd7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39456/0 (socket says 172.21.15.43:39456) 2024-04-21T08:45:35.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.311+0000 7f763e16a640 1 -- 172.21.15.43:0/3077050466 learned_addr learned my addr 172.21.15.43:0/3077050466 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:35.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.311+0000 7f763d969640 1 --2- 172.21.15.43:0/3077050466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5646eafd1e80 0x5646eafd7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.311+0000 7f763e96b640 1 -- 172.21.15.43:0/3077050466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5646eafd1e80 msgr2=0x5646eafd7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:35.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.311+0000 7f763e96b640 1 --2- 172.21.15.43:0/3077050466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5646eafd1e80 0x5646eafd7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.311+0000 7f763e96b640 1 -- 172.21.15.43:0/3077050466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646eafd14d0 msgr2=0x5646eafda2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:35.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.311+0000 7f763e96b640 1 --2- 172.21.15.43:0/3077050466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646eafd14d0 0x5646eafda2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.311+0000 7f763e96b640 1 -- 172.21.15.43:0/3077050466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646eac483f0 con 0x5646eafdaa00 2024-04-21T08:45:35.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.311+0000 7f763e16a640 1 --2- 172.21.15.43:0/3077050466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646eafd14d0 0x5646eafda2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:45:35.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.311+0000 7f763e96b640 1 --2- 172.21.15.43:0/3077050466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafdaa00 0x5646eafd7bd0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f76280092f0 tx=0x7f7628004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fb63b952b58a08 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:35.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.311+0000 7f761dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:35.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.312+0000 7f761dffb640 1 -- 172.21.15.43:0/3077050466 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7628026070 con 0x5646eafdaa00 2024-04-21T08:45:35.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.312+0000 7f761dffb640 1 -- 172.21.15.43:0/3077050466 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f7628004b20 con 0x5646eafdaa00 2024-04-21T08:45:35.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.312+0000 7f7642fbf740 1 -- 172.21.15.43:0/3077050466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76300bf180 con 0x5646eafdaa00 2024-04-21T08:45:35.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.312+0000 7f761dffb640 1 -- 172.21.15.43:0/3077050466 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7628031050 con 0x5646eafdaa00 2024-04-21T08:45:35.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.312+0000 7f7642fbf740 10 client.18639 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:35.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.312+0000 7f7642fbf740 1 -- 172.21.15.43:0/3077050466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5646ead127d0 con 0x5646eafdaa00 2024-04-21T08:45:35.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.312+0000 7f761dffb640 1 -- 172.21.15.43:0/3077050466 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76300bf180 con 0x5646eafdaa00 2024-04-21T08:45:35.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.312+0000 7f761dffb640 1 -- 172.21.15.43:0/3077050466 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f76280057a0 con 0x5646eafdaa00 2024-04-21T08:45:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.312+0000 7f7642fbf740 10 client.18639 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.312+0000 7f7642fbf740 10 client.18639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.312+0000 7f7642fbf740 20 client.18639 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.312+0000 7f7642fbf740 1 --2- 172.21.15.43:0/3077050466 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5646eafea8a0 0x5646eafecd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.312+0000 7f7642fbf740 4 client.18639 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.312+0000 7f7642fbf740 1 -- 172.21.15.43:0/3077050466 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646eaed5930 con 0x5646eafea8a0 2024-04-21T08:45:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.313+0000 7f763e16a640 1 --2- 172.21.15.43:0/3077050466 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5646eafea8a0 0x5646eafecd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.313+0000 7f761dffb640 10 client.18639 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.313+0000 7f763e16a640 1 --2- 172.21.15.43:0/3077050466 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5646eafea8a0 0x5646eafecd20 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efc4621a4cfbb33d server_cookie=fa01141ed19f808a in_seq=0 out_seq=0 2024-04-21T08:45:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.314+0000 7f761dffb640 1 -- 172.21.15.43:0/3077050466 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5646eaed5930 con 0x5646eafea8a0 2024-04-21T08:45:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.314+0000 7f761dffb640 10 client.18639 handle_command_reply: tid=0 2024-04-21T08:45:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.314+0000 7f7642fbf740 1 client.18639 shutdown 2024-04-21T08:45:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.314+0000 7f7642fbf740 1 -- 172.21.15.43:0/3077050466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafdaa00 msgr2=0x5646eafd7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.314+0000 7f7642fbf740 1 --2- 172.21.15.43:0/3077050466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafdaa00 0x5646eafd7bd0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f76280092f0 tx=0x7f7628004a40 comp rx=0 tx=0).stop 2024-04-21T08:45:35.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.314+0000 7f7642fbf740 1 -- 172.21.15.43:0/3077050466 shutdown_connections 2024-04-21T08:45:35.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.314+0000 7f7642fbf740 1 --2- 172.21.15.43:0/3077050466 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5646eafea8a0 0x5646eafecd20 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.314+0000 7f7642fbf740 1 --2- 172.21.15.43:0/3077050466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafdaa00 0x5646eafd7bd0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.314+0000 7f761dffb640 0 client.18639 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:35.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.314+0000 7f7642fbf740 1 --2- 172.21.15.43:0/3077050466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5646eafd1e80 0x5646eafd7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.314+0000 7f7642fbf740 1 --2- 172.21.15.43:0/3077050466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646eafd14d0 0x5646eafda2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.314+0000 7f7642fbf740 1 -- 172.21.15.43:0/3077050466 >> 172.21.15.43:0/3077050466 conn(0x5646eafcbba0 msgr2=0x5646eafcd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:35.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.314+0000 7f7642fbf740 1 -- 172.21.15.43:0/3077050466 shutdown_connections 2024-04-21T08:45:35.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.314+0000 7f7642fbf740 1 -- 172.21.15.43:0/3077050466 wait complete. 2024-04-21T08:45:35.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.314+0000 7f7642fbf740 20 client.18639 trim_cache size 0 max 16384 2024-04-21T08:45:35.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.327+0000 7f7642fbf740 1 Processor -- start 2024-04-21T08:45:35.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.327+0000 7f7642fbf740 1 -- start start 2024-04-21T08:45:35.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.327+0000 7f7642fbf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5646eafd14d0 0x5646eafe64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.328+0000 7f7642fbf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafd1e80 0x5646eafd6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.328+0000 7f7642fbf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646eafdaa00 0x5646eafcaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.328+0000 7f7642fbf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f76381b1680 con 0x5646eafd1e80 2024-04-21T08:45:35.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.328+0000 7f7642fbf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f76381b31c0 con 0x5646eafdaa00 2024-04-21T08:45:35.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.328+0000 7f7642fbf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f76381b1d50 con 0x5646eafd14d0 2024-04-21T08:45:35.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.328+0000 7f763d969640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafd1e80 0x5646eafd6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.328+0000 7f763d969640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafd1e80 0x5646eafd6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39466/0 (socket says 172.21.15.43:39466) 2024-04-21T08:45:35.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.328+0000 7f763d969640 1 -- 172.21.15.43:0/3488312612 learned_addr learned my addr 172.21.15.43:0/3488312612 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:35.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.328+0000 7f763e16a640 1 --2- 172.21.15.43:0/3488312612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5646eafd14d0 0x5646eafe64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.328+0000 7f763e96b640 1 --2- 172.21.15.43:0/3488312612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646eafdaa00 0x5646eafcaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.328+0000 7f763d969640 1 -- 172.21.15.43:0/3488312612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5646eafd14d0 msgr2=0x5646eafe64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:35.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.328+0000 7f763d969640 1 --2- 172.21.15.43:0/3488312612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5646eafd14d0 0x5646eafe64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.328+0000 7f763d969640 1 -- 172.21.15.43:0/3488312612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646eafdaa00 msgr2=0x5646eafcaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:35.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.328+0000 7f763d969640 1 --2- 172.21.15.43:0/3488312612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646eafdaa00 0x5646eafcaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.328+0000 7f763d969640 1 -- 172.21.15.43:0/3488312612 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646eafd1080 con 0x5646eafd1e80 2024-04-21T08:45:35.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.328+0000 7f763d969640 1 --2- 172.21.15.43:0/3488312612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafd1e80 0x5646eafd6810 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f762c00bde0 tx=0x7f762c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00efe84d097075a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:35.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.328+0000 7f761ffff640 1 -- 172.21.15.43:0/3488312612 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c0040f0 con 0x5646eafd1e80 2024-04-21T08:45:35.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.328+0000 7f761ffff640 1 -- 172.21.15.43:0/3488312612 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f762c004290 con 0x5646eafd1e80 2024-04-21T08:45:35.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.328+0000 7f761ffff640 1 -- 172.21.15.43:0/3488312612 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c010880 con 0x5646eafd1e80 2024-04-21T08:45:35.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.329+0000 7f7642fbf740 1 -- 172.21.15.43:0/3488312612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafd1e80 msgr2=0x5646eafd6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.329+0000 7f7642fbf740 1 --2- 172.21.15.43:0/3488312612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafd1e80 0x5646eafd6810 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f762c00bde0 tx=0x7f762c012820 comp rx=0 tx=0).stop 2024-04-21T08:45:35.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.329+0000 7f7642fbf740 1 -- 172.21.15.43:0/3488312612 shutdown_connections 2024-04-21T08:45:35.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.329+0000 7f7642fbf740 1 --2- 172.21.15.43:0/3488312612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646eafdaa00 0x5646eafcaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.329+0000 7f7642fbf740 1 --2- 172.21.15.43:0/3488312612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafd1e80 0x5646eafd6810 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.329+0000 7f7642fbf740 1 --2- 172.21.15.43:0/3488312612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5646eafd14d0 0x5646eafe64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.329+0000 7f7642fbf740 1 -- 172.21.15.43:0/3488312612 >> 172.21.15.43:0/3488312612 conn(0x5646eafcbba0 msgr2=0x5646eafca370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:35.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.329+0000 7f7642fbf740 1 -- 172.21.15.43:0/3488312612 shutdown_connections 2024-04-21T08:45:35.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.329+0000 7f7642fbf740 1 -- 172.21.15.43:0/3488312612 wait complete. 2024-04-21T08:45:35.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.329+0000 7f7642fbf740 1 -- start start 2024-04-21T08:45:35.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.329+0000 7f7642fbf740 1 Processor -- start 2024-04-21T08:45:35.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.329+0000 7f7642fbf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafd14d0 0x5646eafe2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.329+0000 7f7642fbf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5646eafd1e80 0x5646eafe2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.329+0000 7f7642fbf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646eafdaa00 0x5646eafd3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.329+0000 7f7642fbf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f76381b2af0 con 0x5646eafd14d0 2024-04-21T08:45:35.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.329+0000 7f7642fbf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f76381b2420 con 0x5646eafdaa00 2024-04-21T08:45:35.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.329+0000 7f7642fbf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f76381af660 con 0x5646eafd1e80 2024-04-21T08:45:35.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.329+0000 7f7642fbf740 10 client.0 _ll_register_callbacks cb 0x7f76280144a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:35.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.329+0000 7f763e16a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafd14d0 0x5646eafe2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.330+0000 7f763e16a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafd14d0 0x5646eafe2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39468/0 (socket says 172.21.15.43:39468) 2024-04-21T08:45:35.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.330+0000 7f763e16a640 1 -- 172.21.15.43:0/3283037399 learned_addr learned my addr 172.21.15.43:0/3283037399 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:35.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.330+0000 7f763d969640 1 --2- 172.21.15.43:0/3283037399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5646eafd1e80 0x5646eafe2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.330+0000 7f763e96b640 1 --2- 172.21.15.43:0/3283037399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646eafdaa00 0x5646eafd3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.330+0000 7f763e16a640 1 -- 172.21.15.43:0/3283037399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5646eafd1e80 msgr2=0x5646eafe2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:35.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.330+0000 7f763e16a640 1 --2- 172.21.15.43:0/3283037399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5646eafd1e80 0x5646eafe2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.330+0000 7f763e16a640 1 -- 172.21.15.43:0/3283037399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646eafdaa00 msgr2=0x5646eafd3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:35.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.330+0000 7f763e16a640 1 --2- 172.21.15.43:0/3283037399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646eafdaa00 0x5646eafd3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.330+0000 7f763e16a640 1 -- 172.21.15.43:0/3283037399 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646eafd4560 con 0x5646eafd14d0 2024-04-21T08:45:35.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.330+0000 7f763e96b640 1 --2- 172.21.15.43:0/3283037399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646eafdaa00 0x5646eafd3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:35.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.330+0000 7f763e16a640 1 --2- 172.21.15.43:0/3283037399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafd14d0 0x5646eafe2a00 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f76300bb400 tx=0x7f76300e9b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2190c426c8390c5b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:35.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.330+0000 7f761dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:35.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.330+0000 7f761dffb640 1 -- 172.21.15.43:0/3283037399 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76300bf180 con 0x5646eafd14d0 2024-04-21T08:45:35.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.330+0000 7f761dffb640 1 -- 172.21.15.43:0/3283037399 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f76300054d0 con 0x5646eafd14d0 2024-04-21T08:45:35.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.330+0000 7f7642fbf740 1 -- 172.21.15.43:0/3283037399 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f762c004290 con 0x5646eafd14d0 2024-04-21T08:45:35.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.330+0000 7f761dffb640 1 -- 172.21.15.43:0/3283037399 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f76300c2e30 con 0x5646eafd14d0 2024-04-21T08:45:35.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.330+0000 7f7642fbf740 10 client.18645 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:35.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.330+0000 7f7642fbf740 1 -- 172.21.15.43:0/3283037399 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5646eb0509c0 con 0x5646eafd14d0 2024-04-21T08:45:35.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.330+0000 7f761dffb640 1 -- 172.21.15.43:0/3283037399 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f762c004290 con 0x5646eafd14d0 2024-04-21T08:45:35.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.331+0000 7f761dffb640 1 -- 172.21.15.43:0/3283037399 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f76300db050 con 0x5646eafd14d0 2024-04-21T08:45:35.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.331+0000 7f7642fbf740 10 client.18645 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:35.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.331+0000 7f7642fbf740 10 client.18645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:35.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.331+0000 7f7642fbf740 20 client.18645 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:35.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.331+0000 7f7642fbf740 1 --2- 172.21.15.43:0/3283037399 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5646eb050f90 0x5646eb053380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.331+0000 7f7642fbf740 4 client.18645 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:45:35.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.331+0000 7f7642fbf740 1 -- 172.21.15.43:0/3283037399 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5646eaf43c10 con 0x5646eb050f90 2024-04-21T08:45:35.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.331+0000 7f763d969640 1 --2- 172.21.15.43:0/3283037399 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5646eb050f90 0x5646eb053380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.331+0000 7f763d969640 1 --2- 172.21.15.43:0/3283037399 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5646eb050f90 0x5646eb053380 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd42cbca839aeb12 server_cookie=7771a72daf68a010 in_seq=0 out_seq=0 2024-04-21T08:45:35.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.331+0000 7f761dffb640 10 client.18645 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:35.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f761dffb640 1 -- 172.21.15.43:0/3283037399 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5646eaf43c10 con 0x5646eb050f90 2024-04-21T08:45:35.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f761dffb640 10 client.18645 handle_command_reply: tid=0 2024-04-21T08:45:35.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f7642fbf740 1 client.18645 shutdown 2024-04-21T08:45:35.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f7642fbf740 1 -- 172.21.15.43:0/3283037399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafd14d0 msgr2=0x5646eafe2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f7642fbf740 1 --2- 172.21.15.43:0/3283037399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafd14d0 0x5646eafe2a00 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f76300bb400 tx=0x7f76300e9b90 comp rx=0 tx=0).stop 2024-04-21T08:45:35.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f7642fbf740 1 -- 172.21.15.43:0/3283037399 shutdown_connections 2024-04-21T08:45:35.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f7642fbf740 1 --2- 172.21.15.43:0/3283037399 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5646eb050f90 0x5646eb053380 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f7642fbf740 1 --2- 172.21.15.43:0/3283037399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646eafdaa00 0x5646eafd3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f7642fbf740 1 --2- 172.21.15.43:0/3283037399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5646eafd1e80 0x5646eafe2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f761dffb640 0 client.18645 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:35.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f7642fbf740 1 --2- 172.21.15.43:0/3283037399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646eafd14d0 0x5646eafe2a00 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f7642fbf740 1 -- 172.21.15.43:0/3283037399 >> 172.21.15.43:0/3283037399 conn(0x5646eafcbba0 msgr2=0x5646eafcac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:35.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f7642fbf740 1 -- 172.21.15.43:0/3283037399 shutdown_connections 2024-04-21T08:45:35.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f7642fbf740 1 -- 172.21.15.43:0/3283037399 wait complete. 2024-04-21T08:45:35.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f7642fbf740 20 client.18645 trim_cache size 0 max 16384 2024-04-21T08:45:35.340 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:45:35.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f76403ce640 1 -- 172.21.15.43:0/2293142698 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f76200a3d50 msgr2=0x7f76200a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f76403ce640 1 --2- 172.21.15.43:0/2293142698 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f76200a3d50 0x7f76200a6210 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7628009fd0 tx=0x7f7628009200 comp rx=0 tx=0).stop 2024-04-21T08:45:35.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f76403ce640 1 -- 172.21.15.43:0/2293142698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7638102190 msgr2=0x7f7638074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f76403ce640 1 --2- 172.21.15.43:0/2293142698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7638102190 0x7f7638074250 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f7630005940 tx=0x7f763002c7e0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f76403ce640 1 -- 172.21.15.43:0/2293142698 shutdown_connections 2024-04-21T08:45:35.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f76403ce640 1 --2- 172.21.15.43:0/2293142698 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f76200a3d50 0x7f76200a6210 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f76403ce640 1 --2- 172.21.15.43:0/2293142698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7638074cd0 0x7f76381a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f76403ce640 1 --2- 172.21.15.43:0/2293142698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7638107260 0x7f7638074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f76403ce640 1 --2- 172.21.15.43:0/2293142698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7638102190 0x7f7638074250 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f76403ce640 1 -- 172.21.15.43:0/2293142698 >> 172.21.15.43:0/2293142698 conn(0x7f76380793a0 msgr2=0x7f76380fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:35.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f76403ce640 1 -- 172.21.15.43:0/2293142698 shutdown_connections 2024-04-21T08:45:35.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:35.332+0000 7f76403ce640 1 -- 172.21.15.43:0/2293142698 wait complete. 2024-04-21T08:45:35.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:35 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:45:35.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:35 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:35.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:35 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:45:35.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:35 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:35.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:35 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:45:35.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:35 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2572065107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:35.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:35 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:45:35.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:35 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:35.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:35 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:45:35.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:35 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:35.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:35 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:45:35.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:35 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2572065107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:35.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:35 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:45:35.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:35 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:35.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:35 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:45:35.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:35 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:35.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:35 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:45:35.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:35 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2572065107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:36.344 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:45:36.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:45:36.344 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:45:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.433+0000 7f7d2c643640 1 -- 172.21.15.43:0/1124353337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d24100450 msgr2=0x7f7d241008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.433+0000 7f7d2c643640 1 --2- 172.21.15.43:0/1124353337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d24100450 0x7f7d241008d0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009930 tx=0x7f7d2002bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.433+0000 7f7d2c643640 1 -- 172.21.15.43:0/1124353337 shutdown_connections 2024-04-21T08:45:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.433+0000 7f7d2c643640 1 --2- 172.21.15.43:0/1124353337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24100e10 0x7f7d2410ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.433+0000 7f7d2c643640 1 --2- 172.21.15.43:0/1124353337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d24100450 0x7f7d241008d0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.433+0000 7f7d2c643640 1 --2- 172.21.15.43:0/1124353337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d24106f30 0x7f7d24107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.433+0000 7f7d2c643640 1 -- 172.21.15.43:0/1124353337 >> 172.21.15.43:0/1124353337 conn(0x7f7d240773e0 msgr2=0x7f7d240fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.433+0000 7f7d2c643640 1 -- 172.21.15.43:0/1124353337 shutdown_connections 2024-04-21T08:45:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.433+0000 7f7d2c643640 1 -- 172.21.15.43:0/1124353337 wait complete. 2024-04-21T08:45:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.434+0000 7f7d2c643640 1 Processor -- start 2024-04-21T08:45:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.434+0000 7f7d2c643640 1 -- start start 2024-04-21T08:45:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.434+0000 7f7d2c643640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24100450 0x7f7d241a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.434+0000 7f7d2c643640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d24100e10 0x7f7d241a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.434+0000 7f7d2c643640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d24106f30 0x7f7d2419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.434+0000 7f7d2c643640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7d24112270 con 0x7f7d24100e10 2024-04-21T08:45:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.434+0000 7f7d2c643640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d241120f0 con 0x7f7d24100450 2024-04-21T08:45:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.434+0000 7f7d2c643640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7d241123f0 con 0x7f7d24106f30 2024-04-21T08:45:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.434+0000 7f7d29bde640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d24100e10 0x7f7d241a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.434+0000 7f7d29bde640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d24100e10 0x7f7d241a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39490/0 (socket says 172.21.15.43:39490) 2024-04-21T08:45:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.434+0000 7f7d29bde640 1 -- 172.21.15.43:0/3622631327 learned_addr learned my addr 172.21.15.43:0/3622631327 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.435+0000 7f7d2abe0640 1 --2- 172.21.15.43:0/3622631327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d24106f30 0x7f7d2419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.435+0000 7f7d2a3df640 1 --2- 172.21.15.43:0/3622631327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24100450 0x7f7d241a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.435+0000 7f7d29bde640 1 -- 172.21.15.43:0/3622631327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d24106f30 msgr2=0x7f7d2419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.435+0000 7f7d29bde640 1 --2- 172.21.15.43:0/3622631327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d24106f30 0x7f7d2419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.435+0000 7f7d29bde640 1 -- 172.21.15.43:0/3622631327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24100450 msgr2=0x7f7d241a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.435+0000 7f7d29bde640 1 --2- 172.21.15.43:0/3622631327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24100450 0x7f7d241a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.435+0000 7f7d29bde640 1 -- 172.21.15.43:0/3622631327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2419afe0 con 0x7f7d24100e10 2024-04-21T08:45:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.435+0000 7f7d29bde640 1 --2- 172.21.15.43:0/3622631327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d24100e10 0x7f7d241a0ba0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009230 tx=0x7f7d200045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.435+0000 7f7d177fe640 1 -- 172.21.15.43:0/3622631327 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20043020 con 0x7f7d24100e10 2024-04-21T08:45:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.435+0000 7f7d2c643640 1 -- 172.21.15.43:0/3622631327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2419b270 con 0x7f7d24100e10 2024-04-21T08:45:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.435+0000 7f7d177fe640 1 -- 172.21.15.43:0/3622631327 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d2002cb70 con 0x7f7d24100e10 2024-04-21T08:45:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.435+0000 7f7d177fe640 1 -- 172.21.15.43:0/3622631327 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20003b50 con 0x7f7d24100e10 2024-04-21T08:45:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.435+0000 7f7d2c643640 1 -- 172.21.15.43:0/3622631327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d241a75e0 con 0x7f7d24100e10 2024-04-21T08:45:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.438+0000 7f7d177fe640 1 -- 172.21.15.43:0/3622631327 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7d20046020 con 0x7f7d24100e10 2024-04-21T08:45:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.438+0000 7f7d2f235740 1 Processor -- start 2024-04-21T08:45:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.439+0000 7f7d2f235740 1 -- start start 2024-04-21T08:45:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.439+0000 7f7d2f235740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56465c51f8d0 0x56465c51e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.439+0000 7f7d2f235740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56465c51eb60 0x56465c51efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.439+0000 7f7d2f235740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c5280e0 0x56465c52a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.439+0000 7f7d2f235740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56465c4838d0 con 0x56465c5280e0 2024-04-21T08:45:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.439+0000 7f7d2f235740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56465c420e80 con 0x56465c51eb60 2024-04-21T08:45:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.439+0000 7f7d2f235740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56465c47f890 con 0x56465c51f8d0 2024-04-21T08:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.439+0000 7f7d177fe640 1 --2- 172.21.15.43:0/3622631327 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7d100a3dc0 0x7f7d100a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.439+0000 7f7d2a3df640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56465c51eb60 0x56465c51efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.439+0000 7f7d29bde640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c5280e0 0x56465c52a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.439+0000 7f7d2a3df640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56465c51eb60 0x56465c51efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:38526/0 (socket says 172.21.15.43:38526) 2024-04-21T08:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.439+0000 7f7d2a3df640 1 -- 172.21.15.43:0/3486937169 learned_addr learned my addr 172.21.15.43:0/3486937169 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.439+0000 7f7d2abe0640 1 --2- 172.21.15.43:0/3486937169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56465c51f8d0 0x56465c51e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.439+0000 7f7d2abe0640 1 --2- 172.21.15.43:0/3622631327 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7d100a3dc0 0x7f7d100a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.439+0000 7f7d177fe640 1 -- 172.21.15.43:0/3622631327 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f7d200f2bb0 con 0x7f7d24100e10 2024-04-21T08:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.439+0000 7f7d2a3df640 1 -- 172.21.15.43:0/3486937169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56465c51f8d0 msgr2=0x56465c51e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.439+0000 7f7d2a3df640 1 --2- 172.21.15.43:0/3486937169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56465c51f8d0 0x56465c51e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.439+0000 7f7d2a3df640 1 -- 172.21.15.43:0/3486937169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c5280e0 msgr2=0x56465c52a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.439+0000 7f7d2a3df640 1 --2- 172.21.15.43:0/3486937169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c5280e0 0x56465c52a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.439+0000 7f7d2a3df640 1 -- 172.21.15.43:0/3486937169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56465c442320 con 0x56465c51eb60 2024-04-21T08:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.440+0000 7f7d2abe0640 1 --2- 172.21.15.43:0/3622631327 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7d100a3dc0 0x7f7d100a6280 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009fd0 tx=0x7f7d1c009600 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.440+0000 7f7d2a3df640 1 --2- 172.21.15.43:0/3486937169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56465c51eb60 0x56465c51efe0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800ac20 tx=0x7f7d1800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cd82cbeb5647484 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.440+0000 7f7d14bfe640 1 -- 172.21.15.43:0/3486937169 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1801d070 con 0x56465c51eb60 2024-04-21T08:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.440+0000 7f7d14bfe640 1 -- 172.21.15.43:0/3486937169 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d18018af0 con 0x56465c51eb60 2024-04-21T08:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.440+0000 7f7d14bfe640 1 -- 172.21.15.43:0/3486937169 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d18010aa0 con 0x56465c51eb60 2024-04-21T08:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.441+0000 7f7d2f235740 1 -- 172.21.15.43:0/3486937169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56465c51eb60 msgr2=0x56465c51efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.441+0000 7f7d2f235740 1 --2- 172.21.15.43:0/3486937169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56465c51eb60 0x56465c51efe0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800ac20 tx=0x7f7d1800f790 comp rx=0 tx=0).stop 2024-04-21T08:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.441+0000 7f7d2f235740 1 -- 172.21.15.43:0/3486937169 shutdown_connections 2024-04-21T08:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.441+0000 7f7d2f235740 1 --2- 172.21.15.43:0/3486937169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c5280e0 0x56465c52a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.441+0000 7f7d2f235740 1 --2- 172.21.15.43:0/3486937169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56465c51eb60 0x56465c51efe0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.441+0000 7f7d2f235740 1 --2- 172.21.15.43:0/3486937169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56465c51f8d0 0x56465c51e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.441+0000 7f7d2f235740 1 -- 172.21.15.43:0/3486937169 >> 172.21.15.43:0/3486937169 conn(0x56465c4e5c90 msgr2=0x56465c519850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.441+0000 7f7d2f235740 1 -- 172.21.15.43:0/3486937169 shutdown_connections 2024-04-21T08:45:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.441+0000 7f7d2f235740 1 -- 172.21.15.43:0/3486937169 wait complete. 2024-04-21T08:45:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.441+0000 7f7d2f235740 1 -- start start 2024-04-21T08:45:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.441+0000 7f7d2f235740 1 Processor -- start 2024-04-21T08:45:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.441+0000 7f7d2f235740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c51eb60 0x56465c527a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.442+0000 7f7d2f235740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56465c51f8d0 0x56465c524e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.442+0000 7f7d2f235740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56465c5280e0 0x56465c5253b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.442+0000 7f7d2f235740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56465c52ae10 con 0x56465c51eb60 2024-04-21T08:45:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.442+0000 7f7d2f235740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56465c40beb0 con 0x56465c5280e0 2024-04-21T08:45:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.442+0000 7f7d2f235740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56465c52af90 con 0x56465c51f8d0 2024-04-21T08:45:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.442+0000 7f7d2f235740 10 client.0 _ll_register_callbacks cb 0x56465c438b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.442+0000 7f7d2a3df640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c51eb60 0x56465c527a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.442+0000 7f7d2abe0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56465c5280e0 0x56465c5253b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.442+0000 7f7d2a3df640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c51eb60 0x56465c527a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39522/0 (socket says 172.21.15.43:39522) 2024-04-21T08:45:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.442+0000 7f7d2a3df640 1 -- 172.21.15.43:0/2075321482 learned_addr learned my addr 172.21.15.43:0/2075321482 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.442+0000 7f7d29bde640 1 --2- 172.21.15.43:0/2075321482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56465c51f8d0 0x56465c524e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.442+0000 7f7d2abe0640 1 -- 172.21.15.43:0/2075321482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56465c51f8d0 msgr2=0x56465c524e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.442+0000 7f7d2abe0640 1 --2- 172.21.15.43:0/2075321482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56465c51f8d0 0x56465c524e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.442+0000 7f7d2abe0640 1 -- 172.21.15.43:0/2075321482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c51eb60 msgr2=0x56465c527a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.442+0000 7f7d2abe0640 1 --2- 172.21.15.43:0/2075321482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c51eb60 0x56465c527a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.442+0000 7f7d2abe0640 1 -- 172.21.15.43:0/2075321482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56465c196210 con 0x56465c5280e0 2024-04-21T08:45:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.442+0000 7f7d29bde640 1 --2- 172.21.15.43:0/2075321482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56465c51f8d0 0x56465c524e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.442+0000 7f7d2a3df640 1 --2- 172.21.15.43:0/2075321482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c51eb60 0x56465c527a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:45:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.442+0000 7f7d2abe0640 1 --2- 172.21.15.43:0/2075321482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56465c5280e0 0x56465c5253b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c005fd0 tx=0x7f7d1c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=154c7378473061d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.442+0000 7f7d0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-21T08:45:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.443+0000 7f7d0dffb640 1 -- 172.21.15.43:0/2075321482 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c011cf0 con 0x56465c5280e0 2024-04-21T08:45:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.443+0000 7f7d2f235740 1 -- 172.21.15.43:0/2075321482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d18018af0 con 0x56465c5280e0 2024-04-21T08:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.443+0000 7f7d0dffb640 1 -- 172.21.15.43:0/2075321482 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f7d1c029400 con 0x56465c5280e0 2024-04-21T08:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.443+0000 7f7d0dffb640 1 -- 172.21.15.43:0/2075321482 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7d1c032050 con 0x56465c5280e0 2024-04-21T08:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.443+0000 7f7d2f235740 10 client.27653 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.443+0000 7f7d2f235740 1 -- 172.21.15.43:0/2075321482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56465c262620 con 0x56465c5280e0 2024-04-21T08:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.443+0000 7f7d0dffb640 1 -- 172.21.15.43:0/2075321482 <== mon.2 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d18018af0 con 0x56465c5280e0 2024-04-21T08:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.443+0000 7f7d0dffb640 1 -- 172.21.15.43:0/2075321482 <== mon.2 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7d1c00f040 con 0x56465c5280e0 2024-04-21T08:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.443+0000 7f7d2f235740 10 client.27653 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.443+0000 7f7d2f235740 10 client.27653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.443+0000 7f7d2f235740 20 client.27653 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.443+0000 7f7d2f235740 1 --2- 172.21.15.43:0/2075321482 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56465c538150 0x56465c53a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.443+0000 7f7d2f235740 4 client.27653 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.443+0000 7f7d2f235740 1 -- 172.21.15.43:0/2075321482 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56465c497880 con 0x56465c538150 2024-04-21T08:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.444+0000 7f7d2a3df640 1 --2- 172.21.15.43:0/2075321482 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56465c538150 0x56465c53a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.444+0000 7f7d0dffb640 10 client.27653 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.444+0000 7f7d2a3df640 1 --2- 172.21.15.43:0/2075321482 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56465c538150 0x56465c53a560 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0deb77498456e62 server_cookie=2997714884bf866b in_seq=0 out_seq=0 2024-04-21T08:45:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.445+0000 7f7d0dffb640 1 -- 172.21.15.43:0/2075321482 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56465c497880 con 0x56465c538150 2024-04-21T08:45:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.445+0000 7f7d0dffb640 10 client.27653 handle_command_reply: tid=0 2024-04-21T08:45:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.445+0000 7f7d2f235740 1 client.27653 shutdown 2024-04-21T08:45:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.445+0000 7f7d2f235740 1 -- 172.21.15.43:0/2075321482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56465c5280e0 msgr2=0x56465c5253b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.445+0000 7f7d2f235740 1 --2- 172.21.15.43:0/2075321482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56465c5280e0 0x56465c5253b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c005fd0 tx=0x7f7d1c004a40 comp rx=0 tx=0).stop 2024-04-21T08:45:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.445+0000 7f7d2f235740 1 -- 172.21.15.43:0/2075321482 shutdown_connections 2024-04-21T08:45:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.445+0000 7f7d2f235740 1 --2- 172.21.15.43:0/2075321482 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56465c538150 0x56465c53a560 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.445+0000 7f7d2f235740 1 --2- 172.21.15.43:0/2075321482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56465c5280e0 0x56465c5253b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.445+0000 7f7d2f235740 1 --2- 172.21.15.43:0/2075321482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56465c51f8d0 0x56465c524e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.445+0000 7f7d2f235740 1 --2- 172.21.15.43:0/2075321482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c51eb60 0x56465c527a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.445+0000 7f7d0dffb640 0 client.27653 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.445+0000 7f7d2f235740 1 -- 172.21.15.43:0/2075321482 >> 172.21.15.43:0/2075321482 conn(0x56465c4e5c90 msgr2=0x56465c51b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.445+0000 7f7d2f235740 1 -- 172.21.15.43:0/2075321482 shutdown_connections 2024-04-21T08:45:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.445+0000 7f7d2f235740 1 -- 172.21.15.43:0/2075321482 wait complete. 2024-04-21T08:45:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.445+0000 7f7d2f235740 20 client.27653 trim_cache size 0 max 16384 2024-04-21T08:45:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.460+0000 7f7d2f235740 1 Processor -- start 2024-04-21T08:45:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.460+0000 7f7d2f235740 1 -- start start 2024-04-21T08:45:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.460+0000 7f7d2f235740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c51eb60 0x56465c533d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.460+0000 7f7d2f235740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56465c51f8d0 0x56465c5195b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.460+0000 7f7d2f235740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56465c5280e0 0x56465c527540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.460+0000 7f7d2f235740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7d241b1590 con 0x56465c51eb60 2024-04-21T08:45:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.460+0000 7f7d2f235740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d241b30d0 con 0x56465c5280e0 2024-04-21T08:45:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.460+0000 7f7d2f235740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7d241b1c60 con 0x56465c51f8d0 2024-04-21T08:45:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.461+0000 7f7d2a3df640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c51eb60 0x56465c533d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.461+0000 7f7d2a3df640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c51eb60 0x56465c533d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39528/0 (socket says 172.21.15.43:39528) 2024-04-21T08:45:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.461+0000 7f7d2a3df640 1 -- 172.21.15.43:0/234005199 learned_addr learned my addr 172.21.15.43:0/234005199 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.461+0000 7f7d2abe0640 1 --2- 172.21.15.43:0/234005199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56465c5280e0 0x56465c527540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.461+0000 7f7d29bde640 1 --2- 172.21.15.43:0/234005199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56465c51f8d0 0x56465c5195b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.461+0000 7f7d2a3df640 1 -- 172.21.15.43:0/234005199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56465c51f8d0 msgr2=0x56465c5195b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.461+0000 7f7d2a3df640 1 --2- 172.21.15.43:0/234005199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56465c51f8d0 0x56465c5195b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.461+0000 7f7d2a3df640 1 -- 172.21.15.43:0/234005199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56465c5280e0 msgr2=0x56465c527540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.461+0000 7f7d2a3df640 1 --2- 172.21.15.43:0/234005199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56465c5280e0 0x56465c527540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.461+0000 7f7d2a3df640 1 -- 172.21.15.43:0/234005199 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56465c53abe0 con 0x56465c51eb60 2024-04-21T08:45:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.461+0000 7f7d2abe0640 1 --2- 172.21.15.43:0/234005199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56465c5280e0 0x56465c527540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.461+0000 7f7d2a3df640 1 --2- 172.21.15.43:0/234005199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c51eb60 0x56465c533d10 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800ab70 tx=0x7f7d180106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd6d993c5735bbf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.461+0000 7f7d0ffff640 1 -- 172.21.15.43:0/234005199 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1801d040 con 0x56465c51eb60 2024-04-21T08:45:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.461+0000 7f7d0ffff640 1 -- 172.21.15.43:0/234005199 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d18028030 con 0x56465c51eb60 2024-04-21T08:45:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.461+0000 7f7d0ffff640 1 -- 172.21.15.43:0/234005199 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d180109b0 con 0x56465c51eb60 2024-04-21T08:45:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.462+0000 7f7d2f235740 1 -- 172.21.15.43:0/234005199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c51eb60 msgr2=0x56465c533d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.462+0000 7f7d2f235740 1 --2- 172.21.15.43:0/234005199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c51eb60 0x56465c533d10 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800ab70 tx=0x7f7d180106e0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.462+0000 7f7d2f235740 1 -- 172.21.15.43:0/234005199 shutdown_connections 2024-04-21T08:45:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.462+0000 7f7d2f235740 1 --2- 172.21.15.43:0/234005199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56465c5280e0 0x56465c527540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.462+0000 7f7d2f235740 1 --2- 172.21.15.43:0/234005199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56465c51f8d0 0x56465c5195b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.462+0000 7f7d2f235740 1 --2- 172.21.15.43:0/234005199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c51eb60 0x56465c533d10 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.462+0000 7f7d2f235740 1 -- 172.21.15.43:0/234005199 >> 172.21.15.43:0/234005199 conn(0x56465c4e5c90 msgr2=0x56465c51fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.462+0000 7f7d2f235740 1 -- 172.21.15.43:0/234005199 shutdown_connections 2024-04-21T08:45:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.462+0000 7f7d2f235740 1 -- 172.21.15.43:0/234005199 wait complete. 2024-04-21T08:45:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.463+0000 7f7d2f235740 1 -- start start 2024-04-21T08:45:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.463+0000 7f7d2f235740 1 Processor -- start 2024-04-21T08:45:36.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:36 smithi043 ceph-mon[27978]: pgmap v203: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.9 KiB/s wr, 11 op/s 2024-04-21T08:45:36.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:36 smithi175 ceph-mon[31920]: pgmap v203: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.9 KiB/s wr, 11 op/s 2024-04-21T08:45:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.463+0000 7f7d2f235740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56465c51eb60 0x56465c530220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.463+0000 7f7d2f235740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56465c51f8d0 0x56465c530760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.463+0000 7f7d2f235740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c5280e0 0x56465c522090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.463+0000 7f7d2f235740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7d241b2a00 con 0x56465c5280e0 2024-04-21T08:45:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.463+0000 7f7d2f235740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d241b2330 con 0x56465c51eb60 2024-04-21T08:45:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.463+0000 7f7d2f235740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7d241af570 con 0x56465c51f8d0 2024-04-21T08:45:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.463+0000 7f7d2f235740 10 client.0 _ll_register_callbacks cb 0x7f7d1c0127e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.463+0000 7f7d2abe0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c5280e0 0x56465c522090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.463+0000 7f7d2abe0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c5280e0 0x56465c522090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39534/0 (socket says 172.21.15.43:39534) 2024-04-21T08:45:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.463+0000 7f7d2abe0640 1 -- 172.21.15.43:0/2608573051 learned_addr learned my addr 172.21.15.43:0/2608573051 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.463+0000 7f7d2a3df640 1 --2- 172.21.15.43:0/2608573051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56465c51eb60 0x56465c530220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.463+0000 7f7d29bde640 1 --2- 172.21.15.43:0/2608573051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56465c51f8d0 0x56465c530760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.464+0000 7f7d2abe0640 1 -- 172.21.15.43:0/2608573051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56465c51f8d0 msgr2=0x56465c530760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.464+0000 7f7d2abe0640 1 --2- 172.21.15.43:0/2608573051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56465c51f8d0 0x56465c530760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.464+0000 7f7d2abe0640 1 -- 172.21.15.43:0/2608573051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56465c51eb60 msgr2=0x56465c530220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.464+0000 7f7d2abe0640 1 --2- 172.21.15.43:0/2608573051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56465c51eb60 0x56465c530220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.464+0000 7f7d2abe0640 1 -- 172.21.15.43:0/2608573051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56465c536d40 con 0x56465c5280e0 2024-04-21T08:45:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.464+0000 7f7d2abe0640 1 --2- 172.21.15.43:0/2608573051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c5280e0 0x56465c522090 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c005d10 tx=0x7f7d1c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e638fd63b626e78c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.464+0000 7f7d0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.464+0000 7f7d0dffb640 1 -- 172.21.15.43:0/2608573051 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c009d40 con 0x56465c5280e0 2024-04-21T08:45:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.464+0000 7f7d0dffb640 1 -- 172.21.15.43:0/2608573051 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f7d1c01b040 con 0x56465c5280e0 2024-04-21T08:45:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.464+0000 7f7d2f235740 1 -- 172.21.15.43:0/2608573051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d18028030 con 0x56465c5280e0 2024-04-21T08:45:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.464+0000 7f7d0dffb640 1 -- 172.21.15.43:0/2608573051 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7d1c048050 con 0x56465c5280e0 2024-04-21T08:45:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.464+0000 7f7d2f235740 10 client.18663 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.464+0000 7f7d2f235740 1 -- 172.21.15.43:0/2608573051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56465c528cf0 con 0x56465c5280e0 2024-04-21T08:45:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.464+0000 7f7d0dffb640 1 -- 172.21.15.43:0/2608573051 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d18028030 con 0x56465c5280e0 2024-04-21T08:45:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.465+0000 7f7d0dffb640 1 -- 172.21.15.43:0/2608573051 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7d1c021660 con 0x56465c5280e0 2024-04-21T08:45:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.465+0000 7f7d2f235740 10 client.18663 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.465+0000 7f7d2f235740 10 client.18663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.465+0000 7f7d2f235740 20 client.18663 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.465+0000 7f7d2f235740 1 --2- 172.21.15.43:0/2608573051 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56465c59e6e0 0x56465c5a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.465+0000 7f7d2f235740 4 client.18663 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:45:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.465+0000 7f7d2f235740 1 -- 172.21.15.43:0/2608573051 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56465c4233d0 con 0x56465c59e6e0 2024-04-21T08:45:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.465+0000 7f7d2a3df640 1 --2- 172.21.15.43:0/2608573051 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56465c59e6e0 0x56465c5a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.465+0000 7f7d2a3df640 1 --2- 172.21.15.43:0/2608573051 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56465c59e6e0 0x56465c5a0ad0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=435f271eaa873605 server_cookie=2c742cf1cc954334 in_seq=0 out_seq=0 2024-04-21T08:45:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.465+0000 7f7d0dffb640 10 client.18663 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:36.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.468+0000 7f4fe6f7e640 1 -- 172.21.15.43:0/1329043613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fe0106160 msgr2=0x7f4fe010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.468+0000 7f4fe6f7e640 1 --2- 172.21.15.43:0/1329043613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fe0106160 0x7f4fe010dd30 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8009ca0 tx=0x7f4fd802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.468+0000 7f4fe6f7e640 1 -- 172.21.15.43:0/1329043613 shutdown_connections 2024-04-21T08:45:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.468+0000 7f4fe6f7e640 1 --2- 172.21.15.43:0/1329043613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fe0106160 0x7f4fe010dd30 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.468+0000 7f4fe6f7e640 1 --2- 172.21.15.43:0/1329043613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe0101a40 0x7f4fe0105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.468+0000 7f4fe6f7e640 1 --2- 172.21.15.43:0/1329043613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fe0101090 0x7f4fe0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.468+0000 7f4fe6f7e640 1 -- 172.21.15.43:0/1329043613 >> 172.21.15.43:0/1329043613 conn(0x7f4fe0077f50 msgr2=0x7f4fe0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.468+0000 7f4fe6f7e640 1 -- 172.21.15.43:0/1329043613 shutdown_connections 2024-04-21T08:45:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.468+0000 7f4fe6f7e640 1 -- 172.21.15.43:0/1329043613 wait complete. 2024-04-21T08:45:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.469+0000 7f4fe6f7e640 1 Processor -- start 2024-04-21T08:45:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.469+0000 7f4fe6f7e640 1 -- start start 2024-04-21T08:45:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.469+0000 7f4fe6f7e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fe0101090 0x7f4fe0102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.469+0000 7f4fe6f7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe0106160 0x7f4fe0103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.469+0000 7f4fe6f7e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fe01035a0 0x7f4fe01a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.470+0000 7f4fe4d1a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fe0101090 0x7f4fe0102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.470+0000 7f4fe4d1a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fe0101090 0x7f4fe0102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.43:34994/0 (socket says 172.21.15.43:34994) 2024-04-21T08:45:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.470+0000 7f4fe551b640 1 --2- 172.21.15.43:0/1886530283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fe01035a0 0x7f4fe01a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.470+0000 7f4fe4d1a640 1 -- 172.21.15.43:0/1886530283 learned_addr learned my addr 172.21.15.43:0/1886530283 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.470+0000 7f4fe6f7e640 1 -- 172.21.15.43:0/1886530283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe01110d0 con 0x7f4fe01035a0 2024-04-21T08:45:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.470+0000 7f4fe6f7e640 1 -- 172.21.15.43:0/1886530283 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe0110f50 con 0x7f4fe0106160 2024-04-21T08:45:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.470+0000 7f4fe6f7e640 1 -- 172.21.15.43:0/1886530283 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe0111250 con 0x7f4fe0101090 2024-04-21T08:45:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.470+0000 7f4fdffff640 1 --2- 172.21.15.43:0/1886530283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe0106160 0x7f4fe0103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.470+0000 7f4fe551b640 1 -- 172.21.15.43:0/1886530283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fe0101090 msgr2=0x7f4fe0102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.470+0000 7f4fe551b640 1 --2- 172.21.15.43:0/1886530283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fe0101090 0x7f4fe0102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.470+0000 7f4fe551b640 1 -- 172.21.15.43:0/1886530283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe0106160 msgr2=0x7f4fe0103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.470+0000 7f4fe551b640 1 --2- 172.21.15.43:0/1886530283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe0106160 0x7f4fe0103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.470+0000 7f4fe551b640 1 -- 172.21.15.43:0/1886530283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe01a6fe0 con 0x7f4fe01035a0 2024-04-21T08:45:36.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.470+0000 7f4fdffff640 1 --2- 172.21.15.43:0/1886530283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe0106160 0x7f4fe0103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:36.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.470+0000 7f4fe551b640 1 --2- 172.21.15.43:0/1886530283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fe01035a0 0x7f4fe01a68e0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f4fd802c700 tx=0x7f4fd80346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:36.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.471+0000 7f4fddffb640 1 -- 172.21.15.43:0/1886530283 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8041020 con 0x7f4fe01035a0 2024-04-21T08:45:36.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.471+0000 7f4fddffb640 1 -- 172.21.15.43:0/1886530283 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd8032970 con 0x7f4fe01035a0 2024-04-21T08:45:36.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.471+0000 7f4fe6f7e640 1 -- 172.21.15.43:0/1886530283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe01a7270 con 0x7f4fe01035a0 2024-04-21T08:45:36.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.471+0000 7f4fddffb640 1 -- 172.21.15.43:0/1886530283 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8039040 con 0x7f4fe01035a0 2024-04-21T08:45:36.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.471+0000 7f4fe6f7e640 1 -- 172.21.15.43:0/1886530283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe01a7730 con 0x7f4fe01035a0 2024-04-21T08:45:36.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.473+0000 7f4fddffb640 1 -- 172.21.15.43:0/1886530283 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4fd8044020 con 0x7f4fe01035a0 2024-04-21T08:45:36.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.473+0000 7f4fe9b6c740 1 Processor -- start 2024-04-21T08:45:36.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.473+0000 7f4fe9b6c740 1 -- start start 2024-04-21T08:45:36.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.474+0000 7f4fe9b6c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56473ffd08e0 0x56473ffd0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.474+0000 7f4fe9b6c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd1290 0x56473ffd9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.474+0000 7f4fe9b6c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56473ffd9e10 0x56473ffdc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.474+0000 7f4fe9b6c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56473ff34f00 con 0x56473ffd1290 2024-04-21T08:45:36.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.474+0000 7f4fe9b6c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56473fed2590 con 0x56473ffd9e10 2024-04-21T08:45:36.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.474+0000 7f4fe9b6c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56473ff30ec0 con 0x56473ffd08e0 2024-04-21T08:45:36.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.474+0000 7f4fe4d1a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd1290 0x56473ffd9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.474+0000 7f4fe4d1a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd1290 0x56473ffd9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39560/0 (socket says 172.21.15.43:39560) 2024-04-21T08:45:36.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.474+0000 7f4fe4d1a640 1 -- 172.21.15.43:0/4148596520 learned_addr learned my addr 172.21.15.43:0/4148596520 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:36.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.474+0000 7f4fddffb640 1 --2- 172.21.15.43:0/1886530283 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4fd00a3d50 0x7f4fd00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.474+0000 7f4fdffff640 1 --2- 172.21.15.43:0/4148596520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56473ffd08e0 0x56473ffd0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.474+0000 7f4fdffff640 1 --2- 172.21.15.43:0/4148596520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56473ffd9e10 0x56473ffdc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.474+0000 7f4fe551b640 1 --2- 172.21.15.43:0/1886530283 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4fd00a3d50 0x7f4fd00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.474+0000 7f4fddffb640 1 -- 172.21.15.43:0/1886530283 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f4fd80f26d0 con 0x7f4fe01035a0 2024-04-21T08:45:36.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.474+0000 7f4fe4d1a640 1 -- 172.21.15.43:0/4148596520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56473ffd08e0 msgr2=0x56473ffd0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:36.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.474+0000 7f4fe4d1a640 1 --2- 172.21.15.43:0/4148596520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56473ffd08e0 0x56473ffd0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.474+0000 7f4fe4d1a640 1 -- 172.21.15.43:0/4148596520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56473ffd9e10 msgr2=0x56473ffdc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:36.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.474+0000 7f4fe4d1a640 1 --2- 172.21.15.43:0/4148596520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56473ffd9e10 0x56473ffdc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.474+0000 7f4fe4d1a640 1 -- 172.21.15.43:0/4148596520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56473fd0f7e0 con 0x56473ffd1290 2024-04-21T08:45:36.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.474+0000 7f4fdffff640 1 --2- 172.21.15.43:0/4148596520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56473ffd08e0 0x56473ffd0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:45:36.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.475+0000 7f4fe4d1a640 1 --2- 172.21.15.43:0/4148596520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd1290 0x56473ffd9840 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009e10 tx=0x7f4fd4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58899f388443513f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:36.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.475+0000 7f4fe551b640 1 --2- 172.21.15.43:0/1886530283 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4fd00a3d50 0x7f4fd00a6210 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4fd80efc10 tx=0x7f4fd80095c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.475+0000 7f7d0dffb640 1 -- 172.21.15.43:0/2608573051 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56465c4233d0 con 0x56465c59e6e0 2024-04-21T08:45:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.475+0000 7f7d0dffb640 10 client.18663 handle_command_reply: tid=0 2024-04-21T08:45:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.475+0000 7f7d2f235740 1 client.18663 shutdown 2024-04-21T08:45:36.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.475+0000 7f4fcaffd640 1 -- 172.21.15.43:0/4148596520 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd401e070 con 0x56473ffd1290 2024-04-21T08:45:36.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.475+0000 7f4fcaffd640 1 -- 172.21.15.43:0/4148596520 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd4019ae0 con 0x56473ffd1290 2024-04-21T08:45:36.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.475+0000 7f4fcaffd640 1 -- 172.21.15.43:0/4148596520 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd400faa0 con 0x56473ffd1290 2024-04-21T08:45:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.475+0000 7f7d2f235740 1 -- 172.21.15.43:0/2608573051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c5280e0 msgr2=0x56465c522090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.475+0000 7f7d2f235740 1 --2- 172.21.15.43:0/2608573051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c5280e0 0x56465c522090 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c005d10 tx=0x7f7d1c0117b0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.475+0000 7f7d2f235740 1 -- 172.21.15.43:0/2608573051 shutdown_connections 2024-04-21T08:45:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.475+0000 7f7d2f235740 1 --2- 172.21.15.43:0/2608573051 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56465c59e6e0 0x56465c5a0ad0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.475+0000 7f7d2f235740 1 --2- 172.21.15.43:0/2608573051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56465c5280e0 0x56465c522090 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.475+0000 7f7d2f235740 1 --2- 172.21.15.43:0/2608573051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56465c51f8d0 0x56465c530760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.475+0000 7f7d2f235740 1 --2- 172.21.15.43:0/2608573051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56465c51eb60 0x56465c530220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.475+0000 7f7d2f235740 1 -- 172.21.15.43:0/2608573051 >> 172.21.15.43:0/2608573051 conn(0x56465c4e5c90 msgr2=0x56465c523630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.475+0000 7f7d0dffb640 0 client.18663 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.475+0000 7f7d2f235740 1 -- 172.21.15.43:0/2608573051 shutdown_connections 2024-04-21T08:45:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.475+0000 7f7d2f235740 1 -- 172.21.15.43:0/2608573051 wait complete. 2024-04-21T08:45:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.475+0000 7f7d2f235740 20 client.18663 trim_cache size 0 max 16384 2024-04-21T08:45:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:45:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:45:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:45:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:45:36.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.476+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/4148596520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd1290 msgr2=0x56473ffd9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.476+0000 7f4fe9b6c740 1 --2- 172.21.15.43:0/4148596520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd1290 0x56473ffd9840 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009e10 tx=0x7f4fd4011810 comp rx=0 tx=0).stop 2024-04-21T08:45:36.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.476+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/4148596520 shutdown_connections 2024-04-21T08:45:36.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.476+0000 7f4fe9b6c740 1 --2- 172.21.15.43:0/4148596520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56473ffd9e10 0x56473ffdc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.476+0000 7f4fe9b6c740 1 --2- 172.21.15.43:0/4148596520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd1290 0x56473ffd9840 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.476+0000 7f4fe9b6c740 1 --2- 172.21.15.43:0/4148596520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56473ffd08e0 0x56473ffd0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.476+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/4148596520 >> 172.21.15.43:0/4148596520 conn(0x56473ffcafb0 msgr2=0x56473ffcb390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.476+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/4148596520 shutdown_connections 2024-04-21T08:45:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.476+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/4148596520 wait complete. 2024-04-21T08:45:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.476+0000 7f7d2c643640 1 -- 172.21.15.43:0/3622631327 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7d100a3dc0 msgr2=0x7f7d100a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.476+0000 7f7d2c643640 1 --2- 172.21.15.43:0/3622631327 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7d100a3dc0 0x7f7d100a6280 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009fd0 tx=0x7f7d1c009600 comp rx=0 tx=0).stop 2024-04-21T08:45:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.476+0000 7f7d2c643640 1 -- 172.21.15.43:0/3622631327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d24100e10 msgr2=0x7f7d241a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.476+0000 7f7d2c643640 1 --2- 172.21.15.43:0/3622631327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d24100e10 0x7f7d241a0ba0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009230 tx=0x7f7d200045e0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.476+0000 7f7d2c643640 1 -- 172.21.15.43:0/3622631327 shutdown_connections 2024-04-21T08:45:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.476+0000 7f7d2c643640 1 --2- 172.21.15.43:0/3622631327 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7d100a3dc0 0x7f7d100a6280 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.476+0000 7f7d2c643640 1 --2- 172.21.15.43:0/3622631327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d24106f30 0x7f7d2419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.476+0000 7f7d2c643640 1 --2- 172.21.15.43:0/3622631327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d24100e10 0x7f7d241a0ba0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.476+0000 7f7d2c643640 1 --2- 172.21.15.43:0/3622631327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24100450 0x7f7d241a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.476+0000 7f7d2c643640 1 -- 172.21.15.43:0/3622631327 >> 172.21.15.43:0/3622631327 conn(0x7f7d240773e0 msgr2=0x7f7d24108460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.476+0000 7f7d2c643640 1 -- 172.21.15.43:0/3622631327 shutdown_connections 2024-04-21T08:45:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:36.476+0000 7f7d2c643640 1 -- 172.21.15.43:0/3622631327 wait complete. 2024-04-21T08:45:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.476+0000 7f4fe9b6c740 1 -- start start 2024-04-21T08:45:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.476+0000 7f4fe9b6c740 1 Processor -- start 2024-04-21T08:45:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.477+0000 7f4fe9b6c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd08e0 0x56473ffd6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.477+0000 7f4fe9b6c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56473ffd1290 0x56473ffd7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.477+0000 7f4fe9b6c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56473ffd9e10 0x56473ffd78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.477+0000 7f4fe9b6c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56473ffdcd80 con 0x56473ffd08e0 2024-04-21T08:45:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.477+0000 7f4fe9b6c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56473feee580 con 0x56473ffd1290 2024-04-21T08:45:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.477+0000 7f4fe9b6c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56473ffdcf00 con 0x56473ffd9e10 2024-04-21T08:45:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.477+0000 7f4fe9b6c740 10 client.0 _ll_register_callbacks cb 0x56473fef4650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.477+0000 7f4fdffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd08e0 0x56473ffd6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.477+0000 7f4fdffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd08e0 0x56473ffd6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39576/0 (socket says 172.21.15.43:39576) 2024-04-21T08:45:36.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.477+0000 7f4fdffff640 1 -- 172.21.15.43:0/2377159245 learned_addr learned my addr 172.21.15.43:0/2377159245 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:36.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.477+0000 7f4fe4d1a640 1 --2- 172.21.15.43:0/2377159245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56473ffd1290 0x56473ffd7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.477+0000 7f4fdffff640 1 -- 172.21.15.43:0/2377159245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56473ffd9e10 msgr2=0x56473ffd78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:36.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.477+0000 7f4fdffff640 1 --2- 172.21.15.43:0/2377159245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56473ffd9e10 0x56473ffd78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.477+0000 7f4fdffff640 1 -- 172.21.15.43:0/2377159245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56473ffd1290 msgr2=0x56473ffd7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:36.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.477+0000 7f4fdffff640 1 --2- 172.21.15.43:0/2377159245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56473ffd1290 0x56473ffd7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.477+0000 7f4fdffff640 1 -- 172.21.15.43:0/2377159245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56473fd6ff70 con 0x56473ffd08e0 2024-04-21T08:45:36.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.477+0000 7f4fe4d1a640 1 --2- 172.21.15.43:0/2377159245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56473ffd1290 0x56473ffd7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:36.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.478+0000 7f4fdffff640 1 --2- 172.21.15.43:0/2377159245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd08e0 0x56473ffd6e50 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009cb0 tx=0x7f4fcc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2cc843276c76667 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:36.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.478+0000 7f4fbffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:36.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.478+0000 7f4fbffff640 1 -- 172.21.15.43:0/2377159245 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc004050 con 0x56473ffd08e0 2024-04-21T08:45:36.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.478+0000 7f4fbffff640 1 -- 172.21.15.43:0/2377159245 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f4fcc0041f0 con 0x56473ffd08e0 2024-04-21T08:45:36.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.478+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/2377159245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fd4019ae0 con 0x56473ffd08e0 2024-04-21T08:45:36.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.478+0000 7f4fbffff640 1 -- 172.21.15.43:0/2377159245 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4fcc027050 con 0x56473ffd08e0 2024-04-21T08:45:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.478+0000 7f4fe9b6c740 10 client.18675 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.478+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/2377159245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56473fc07170 con 0x56473ffd08e0 2024-04-21T08:45:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.478+0000 7f4fbffff640 1 -- 172.21.15.43:0/2377159245 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd4019ae0 con 0x56473ffd08e0 2024-04-21T08:45:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.478+0000 7f4fbffff640 1 -- 172.21.15.43:0/2377159245 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4fcc0214e0 con 0x56473ffd08e0 2024-04-21T08:45:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.478+0000 7f4fe9b6c740 10 client.18675 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.478+0000 7f4fe9b6c740 10 client.18675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.478+0000 7f4fe9b6c740 20 client.18675 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.478+0000 7f4fe9b6c740 1 --2- 172.21.15.43:0/2377159245 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56473ffea340 0x56473ffec770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.478+0000 7f4fe9b6c740 4 client.18675 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.478+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/2377159245 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56473ff42920 con 0x56473ffea340 2024-04-21T08:45:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.479+0000 7f4fe4d1a640 1 --2- 172.21.15.43:0/2377159245 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56473ffea340 0x56473ffec770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.479+0000 7f4fbffff640 10 client.18675 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.479+0000 7f4fe4d1a640 1 --2- 172.21.15.43:0/2377159245 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56473ffea340 0x56473ffec770 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=495b0a514a99c041 server_cookie=ef9d735cd58fb7ce in_seq=0 out_seq=0 2024-04-21T08:45:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.480+0000 7f4fbffff640 1 -- 172.21.15.43:0/2377159245 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56473ff42920 con 0x56473ffea340 2024-04-21T08:45:36.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.480+0000 7f4fbffff640 10 client.18675 handle_command_reply: tid=0 2024-04-21T08:45:36.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.480+0000 7f4fe9b6c740 1 client.18675 shutdown 2024-04-21T08:45:36.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.480+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/2377159245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd08e0 msgr2=0x56473ffd6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.480+0000 7f4fe9b6c740 1 --2- 172.21.15.43:0/2377159245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd08e0 0x56473ffd6e50 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009cb0 tx=0x7f4fcc011810 comp rx=0 tx=0).stop 2024-04-21T08:45:36.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.480+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/2377159245 shutdown_connections 2024-04-21T08:45:36.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.480+0000 7f4fe9b6c740 1 --2- 172.21.15.43:0/2377159245 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56473ffea340 0x56473ffec770 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.480+0000 7f4fe9b6c740 1 --2- 172.21.15.43:0/2377159245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56473ffd9e10 0x56473ffd78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.480+0000 7f4fe9b6c740 1 --2- 172.21.15.43:0/2377159245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56473ffd1290 0x56473ffd7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.480+0000 7f4fe9b6c740 1 --2- 172.21.15.43:0/2377159245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd08e0 0x56473ffd6e50 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.480+0000 7f4fbffff640 0 client.18675 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:36.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.480+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/2377159245 >> 172.21.15.43:0/2377159245 conn(0x56473ffcafb0 msgr2=0x56473ffccd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:36.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.480+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/2377159245 shutdown_connections 2024-04-21T08:45:36.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.480+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/2377159245 wait complete. 2024-04-21T08:45:36.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.480+0000 7f4fe9b6c740 20 client.18675 trim_cache size 0 max 16384 2024-04-21T08:45:36.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.498+0000 7f4fe9b6c740 1 Processor -- start 2024-04-21T08:45:36.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.498+0000 7f4fe9b6c740 1 -- start start 2024-04-21T08:45:36.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.498+0000 7f4fe9b6c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd08e0 0x56473ffd9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.498+0000 7f4fe9b6c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56473ffd1290 0x56473ffe5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.499+0000 7f4fe9b6c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56473ffd9e10 0x56473fff2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.499+0000 7f4fe9b6c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe01b15f0 con 0x56473ffd08e0 2024-04-21T08:45:36.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.499+0000 7f4fe9b6c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe01b3130 con 0x56473ffd1290 2024-04-21T08:45:36.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.499+0000 7f4fe9b6c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe01b1cc0 con 0x56473ffd9e10 2024-04-21T08:45:36.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.499+0000 7f4fdffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd08e0 0x56473ffd9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.499+0000 7f4fdffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd08e0 0x56473ffd9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39590/0 (socket says 172.21.15.43:39590) 2024-04-21T08:45:36.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.499+0000 7f4fdffff640 1 -- 172.21.15.43:0/3263693185 learned_addr learned my addr 172.21.15.43:0/3263693185 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:36.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.499+0000 7f4fe4d1a640 1 --2- 172.21.15.43:0/3263693185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56473ffd1290 0x56473ffe5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.499+0000 7f4fdffff640 1 -- 172.21.15.43:0/3263693185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56473ffd9e10 msgr2=0x56473fff2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:36.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.499+0000 7f4fdffff640 1 --2- 172.21.15.43:0/3263693185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56473ffd9e10 0x56473fff2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.499+0000 7f4fdffff640 1 -- 172.21.15.43:0/3263693185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56473ffd1290 msgr2=0x56473ffe5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:36.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.499+0000 7f4fdffff640 1 --2- 172.21.15.43:0/3263693185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56473ffd1290 0x56473ffe5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.499+0000 7f4fdffff640 1 -- 172.21.15.43:0/3263693185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56473ffd7db0 con 0x56473ffd08e0 2024-04-21T08:45:36.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.499+0000 7f4fdffff640 1 --2- 172.21.15.43:0/3263693185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd08e0 0x56473ffd9580 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009820 tx=0x7f4fcc005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e52b9d03900df88 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:36.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.499+0000 7f4fca7fc640 1 -- 172.21.15.43:0/3263693185 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc021b20 con 0x56473ffd08e0 2024-04-21T08:45:36.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.499+0000 7f4fca7fc640 1 -- 172.21.15.43:0/3263693185 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fcc017030 con 0x56473ffd08e0 2024-04-21T08:45:36.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.499+0000 7f4fca7fc640 1 -- 172.21.15.43:0/3263693185 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc0214d0 con 0x56473ffd08e0 2024-04-21T08:45:36.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.500+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/3263693185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd08e0 msgr2=0x56473ffd9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.500+0000 7f4fe9b6c740 1 --2- 172.21.15.43:0/3263693185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd08e0 0x56473ffd9580 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009820 tx=0x7f4fcc005d30 comp rx=0 tx=0).stop 2024-04-21T08:45:36.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.500+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/3263693185 shutdown_connections 2024-04-21T08:45:36.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.500+0000 7f4fe9b6c740 1 --2- 172.21.15.43:0/3263693185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56473ffd9e10 0x56473fff2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.500+0000 7f4fe9b6c740 1 --2- 172.21.15.43:0/3263693185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56473ffd1290 0x56473ffe5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.500+0000 7f4fe9b6c740 1 --2- 172.21.15.43:0/3263693185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd08e0 0x56473ffd9580 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.500+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/3263693185 >> 172.21.15.43:0/3263693185 conn(0x56473ffcafb0 msgr2=0x56473ffd76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:36.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.500+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/3263693185 shutdown_connections 2024-04-21T08:45:36.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.500+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/3263693185 wait complete. 2024-04-21T08:45:36.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.501+0000 7f4fe9b6c740 1 -- start start 2024-04-21T08:45:36.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.501+0000 7f4fe9b6c740 1 Processor -- start 2024-04-21T08:45:36.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.501+0000 7f4fe9b6c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd08e0 0x56473ffe2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.501+0000 7f4fe9b6c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56473ffd1290 0x56473ffd3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.501+0000 7f4fe9b6c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56473ffd9e10 0x56473ffd4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.501+0000 7f4fe9b6c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe01b2a60 con 0x56473ffd08e0 2024-04-21T08:45:36.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.501+0000 7f4fe9b6c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe01b2390 con 0x56473ffd1290 2024-04-21T08:45:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.501+0000 7f4fe9b6c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe01af5d0 con 0x56473ffd9e10 2024-04-21T08:45:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.501+0000 7f4fe9b6c740 10 client.0 _ll_register_callbacks cb 0x7f4fcc013840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.501+0000 7f4fdffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd08e0 0x56473ffe2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.501+0000 7f4fdffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd08e0 0x56473ffe2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39604/0 (socket says 172.21.15.43:39604) 2024-04-21T08:45:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.501+0000 7f4fdffff640 1 -- 172.21.15.43:0/2188682426 learned_addr learned my addr 172.21.15.43:0/2188682426 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.501+0000 7f4fdffff640 1 -- 172.21.15.43:0/2188682426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56473ffd9e10 msgr2=0x56473ffd4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.501+0000 7f4fdffff640 1 --2- 172.21.15.43:0/2188682426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56473ffd9e10 0x56473ffd4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.501+0000 7f4fdffff640 1 -- 172.21.15.43:0/2188682426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56473ffd1290 msgr2=0x56473ffd3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.501+0000 7f4fe4d1a640 1 --2- 172.21.15.43:0/2188682426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56473ffd1290 0x56473ffd3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.501+0000 7f4fdffff640 1 --2- 172.21.15.43:0/2188682426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56473ffd1290 0x56473ffd3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.501+0000 7f4fdffff640 1 -- 172.21.15.43:0/2188682426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56474004f8b0 con 0x56473ffd08e0 2024-04-21T08:45:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.501+0000 7f4fe4d1a640 1 --2- 172.21.15.43:0/2188682426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56473ffd1290 0x56473ffd3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.502+0000 7f4fdffff640 1 --2- 172.21.15.43:0/2188682426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd08e0 0x56473ffe2690 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc013bd0 tx=0x7f4fcc0095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf56e872c83b530 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.502+0000 7f4fbffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.502+0000 7f4fbffff640 1 -- 172.21.15.43:0/2188682426 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc039060 con 0x56473ffd08e0 2024-04-21T08:45:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.502+0000 7f4fbffff640 1 -- 172.21.15.43:0/2188682426 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f4fcc021210 con 0x56473ffd08e0 2024-04-21T08:45:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.502+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/2188682426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fcc017030 con 0x56473ffd08e0 2024-04-21T08:45:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.502+0000 7f4fbffff640 1 -- 172.21.15.43:0/2188682426 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4fcc040050 con 0x56473ffd08e0 2024-04-21T08:45:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.502+0000 7f4fe9b6c740 10 client.18681 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.502+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/2188682426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564740050a50 con 0x56473ffd08e0 2024-04-21T08:45:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.502+0000 7f4fbffff640 1 -- 172.21.15.43:0/2188682426 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fcc017030 con 0x56473ffd08e0 2024-04-21T08:45:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.502+0000 7f4fbffff640 1 -- 172.21.15.43:0/2188682426 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4fcc003680 con 0x56473ffd08e0 2024-04-21T08:45:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.502+0000 7f4fe9b6c740 10 client.18681 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.502+0000 7f4fe9b6c740 10 client.18681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.502+0000 7f4fe9b6c740 20 client.18681 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.502+0000 7f4fe9b6c740 1 --2- 172.21.15.43:0/2188682426 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5647400510c0 0x5647400534e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.502+0000 7f4fe9b6c740 4 client.18681 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:45:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.502+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/2188682426 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56473ff3c790 con 0x5647400510c0 2024-04-21T08:45:36.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.503+0000 7f4fe4d1a640 1 --2- 172.21.15.43:0/2188682426 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5647400510c0 0x5647400534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.503+0000 7f4fbffff640 10 client.18681 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:36.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.503+0000 7f4fe4d1a640 1 --2- 172.21.15.43:0/2188682426 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5647400510c0 0x5647400534e0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11de0f504c06f2c0 server_cookie=3af3389d8047b5cb in_seq=0 out_seq=0 2024-04-21T08:45:36.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.504+0000 7f4fbffff640 1 -- 172.21.15.43:0/2188682426 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56473ff3c790 con 0x5647400510c0 2024-04-21T08:45:36.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.505+0000 7f4fbffff640 10 client.18681 handle_command_reply: tid=0 2024-04-21T08:45:36.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.505+0000 7f4fe9b6c740 1 client.18681 shutdown 2024-04-21T08:45:36.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.505+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/2188682426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd08e0 msgr2=0x56473ffe2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.505+0000 7f4fe9b6c740 1 --2- 172.21.15.43:0/2188682426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd08e0 0x56473ffe2690 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc013bd0 tx=0x7f4fcc0095b0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.505+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/2188682426 shutdown_connections 2024-04-21T08:45:36.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.505+0000 7f4fe9b6c740 1 --2- 172.21.15.43:0/2188682426 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5647400510c0 0x5647400534e0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.505+0000 7f4fe9b6c740 1 --2- 172.21.15.43:0/2188682426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56473ffd9e10 0x56473ffd4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.505+0000 7f4fe9b6c740 1 --2- 172.21.15.43:0/2188682426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56473ffd1290 0x56473ffd3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.505+0000 7f4fbffff640 0 client.18681 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:36.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.505+0000 7f4fe9b6c740 1 --2- 172.21.15.43:0/2188682426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56473ffd08e0 0x56473ffe2690 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.505+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/2188682426 >> 172.21.15.43:0/2188682426 conn(0x56473ffcafb0 msgr2=0x56473ffc9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:36.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.505+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/2188682426 shutdown_connections 2024-04-21T08:45:36.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.505+0000 7f4fe9b6c740 1 -- 172.21.15.43:0/2188682426 wait complete. 2024-04-21T08:45:36.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.505+0000 7f4fe9b6c740 20 client.18681 trim_cache size 0 max 16384 2024-04-21T08:45:36.509 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:45:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:45:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "79159daa-f79c-43f3-b831-f578646f88dd", 2024-04-21T08:45:36.509 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:45:36.509 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:45:36.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.506+0000 7f4fe6f7e640 1 -- 172.21.15.43:0/1886530283 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4fd00a3d50 msgr2=0x7f4fd00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.506+0000 7f4fe6f7e640 1 --2- 172.21.15.43:0/1886530283 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4fd00a3d50 0x7f4fd00a6210 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4fd80efc10 tx=0x7f4fd80095c0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.506+0000 7f4fe6f7e640 1 -- 172.21.15.43:0/1886530283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fe01035a0 msgr2=0x7f4fe01a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.506+0000 7f4fe6f7e640 1 --2- 172.21.15.43:0/1886530283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fe01035a0 0x7f4fe01a68e0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f4fd802c700 tx=0x7f4fd80346f0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.506+0000 7f4fe6f7e640 1 -- 172.21.15.43:0/1886530283 shutdown_connections 2024-04-21T08:45:36.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.506+0000 7f4fe6f7e640 1 --2- 172.21.15.43:0/1886530283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fe01035a0 0x7f4fe01a68e0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.506+0000 7f4fe6f7e640 1 --2- 172.21.15.43:0/1886530283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fe0106160 0x7f4fe0103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.506+0000 7f4fe6f7e640 1 --2- 172.21.15.43:0/1886530283 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4fd00a3d50 0x7f4fd00a6210 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.506+0000 7f4fe6f7e640 1 --2- 172.21.15.43:0/1886530283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fe0101090 0x7f4fe0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.506+0000 7f4fe6f7e640 1 -- 172.21.15.43:0/1886530283 >> 172.21.15.43:0/1886530283 conn(0x7f4fe0077f50 msgr2=0x7f4fe0069450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:36.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.506+0000 7f4fe6f7e640 1 -- 172.21.15.43:0/1886530283 shutdown_connections 2024-04-21T08:45:36.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.506+0000 7f4fe6f7e640 1 -- 172.21.15.43:0/1886530283 wait complete. 2024-04-21T08:45:36.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:45:36.516 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:45:36.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:36 smithi112 ceph-mon[31085]: pgmap v203: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.9 KiB/s wr, 11 op/s 2024-04-21T08:45:36.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.622+0000 7f0414e26640 1 -- 172.21.15.43:0/2163198943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041010c500 msgr2=0x7f041010e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.622+0000 7f0414e26640 1 --2- 172.21.15.43:0/2163198943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041010c500 0x7f041010e9a0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f0408009ca0 tx=0x7f040802c230 comp rx=0 tx=0).stop 2024-04-21T08:45:36.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.622+0000 7f0414e26640 1 -- 172.21.15.43:0/2163198943 shutdown_connections 2024-04-21T08:45:36.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.622+0000 7f0414e26640 1 --2- 172.21.15.43:0/2163198943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041010c500 0x7f041010e9a0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.622+0000 7f0414e26640 1 --2- 172.21.15.43:0/2163198943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0410103ae0 0x7f041010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.622+0000 7f0414e26640 1 --2- 172.21.15.43:0/2163198943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04101031c0 0x7f04101035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.622+0000 7f0414e26640 1 -- 172.21.15.43:0/2163198943 >> 172.21.15.43:0/2163198943 conn(0x7f0410078000 msgr2=0x7f0410078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:36.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.622+0000 7f0414e26640 1 -- 172.21.15.43:0/2163198943 shutdown_connections 2024-04-21T08:45:36.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.622+0000 7f0414e26640 1 -- 172.21.15.43:0/2163198943 wait complete. 2024-04-21T08:45:36.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.622+0000 7f0414e26640 1 Processor -- start 2024-04-21T08:45:36.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.622+0000 7f0414e26640 1 -- start start 2024-04-21T08:45:36.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.623+0000 7f0414e26640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04101031c0 0x7f0410198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.623+0000 7f0414e26640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0410103ae0 0x7f0410198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.623+0000 7f0414e26640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041010c500 0x7f041019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.623+0000 7f0414e26640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0410112170 con 0x7f041010c500 2024-04-21T08:45:36.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.623+0000 7f0414e26640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0410111ff0 con 0x7f04101031c0 2024-04-21T08:45:36.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.623+0000 7f0414e26640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04101122f0 con 0x7f0410103ae0 2024-04-21T08:45:36.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.623+0000 7f040ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041010c500 0x7f041019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.623+0000 7f040ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041010c500 0x7f041019cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39624/0 (socket says 172.21.15.43:39624) 2024-04-21T08:45:36.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.623+0000 7f040ed9d640 1 -- 172.21.15.43:0/1688479803 learned_addr learned my addr 172.21.15.43:0/1688479803 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:36.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.623+0000 7f040e59c640 1 --2- 172.21.15.43:0/1688479803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04101031c0 0x7f0410198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.623+0000 7f040dd9b640 1 --2- 172.21.15.43:0/1688479803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0410103ae0 0x7f0410198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.623+0000 7f040ed9d640 1 -- 172.21.15.43:0/1688479803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0410103ae0 msgr2=0x7f0410198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.623+0000 7f040ed9d640 1 --2- 172.21.15.43:0/1688479803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0410103ae0 0x7f0410198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.623+0000 7f040ed9d640 1 -- 172.21.15.43:0/1688479803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04101031c0 msgr2=0x7f0410198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.623+0000 7f040ed9d640 1 --2- 172.21.15.43:0/1688479803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04101031c0 0x7f0410198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.623+0000 7f040ed9d640 1 -- 172.21.15.43:0/1688479803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041019d680 con 0x7f041010c500 2024-04-21T08:45:36.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.623+0000 7f040e59c640 1 --2- 172.21.15.43:0/1688479803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04101031c0 0x7f0410198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:36.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.623+0000 7f040dd9b640 1 --2- 172.21.15.43:0/1688479803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0410103ae0 0x7f0410198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:36.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.623+0000 7f040ed9d640 1 --2- 172.21.15.43:0/1688479803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041010c500 0x7f041019cf50 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f040802c840 tx=0x7f0408003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:36.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.623+0000 7f04077fe640 1 -- 172.21.15.43:0/1688479803 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0408039040 con 0x7f041010c500 2024-04-21T08:45:36.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.623+0000 7f04077fe640 1 -- 172.21.15.43:0/1688479803 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0408044070 con 0x7f041010c500 2024-04-21T08:45:36.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.624+0000 7f0414e26640 1 -- 172.21.15.43:0/1688479803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041019d910 con 0x7f041010c500 2024-04-21T08:45:36.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.624+0000 7f04077fe640 1 -- 172.21.15.43:0/1688479803 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04080329c0 con 0x7f041010c500 2024-04-21T08:45:36.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.624+0000 7f0414e26640 1 -- 172.21.15.43:0/1688479803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04101a5200 con 0x7f041010c500 2024-04-21T08:45:36.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.626+0000 7f04077fe640 1 -- 172.21.15.43:0/1688479803 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0408032b60 con 0x7f041010c500 2024-04-21T08:45:36.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.626+0000 7f0417a10740 1 Processor -- start 2024-04-21T08:45:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.626+0000 7f0417a10740 1 -- start start 2024-04-21T08:45:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.626+0000 7f0417a10740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a282045580 0x55a282045960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.626+0000 7f0417a10740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a282045ea0 0x55a28204e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.626+0000 7f0417a10740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a28204ea80 0x55a282050e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.626+0000 7f0417a10740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a281faa1e0 con 0x55a282045580 2024-04-21T08:45:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.626+0000 7f0417a10740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a281f47390 con 0x55a282045ea0 2024-04-21T08:45:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.626+0000 7f0417a10740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a281fa61a0 con 0x55a28204ea80 2024-04-21T08:45:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.627+0000 7f040dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a282045580 0x55a282045960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.627+0000 7f040dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a282045580 0x55a282045960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39628/0 (socket says 172.21.15.43:39628) 2024-04-21T08:45:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.627+0000 7f040dd9b640 1 -- 172.21.15.43:0/2166718360 learned_addr learned my addr 172.21.15.43:0/2166718360 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.627+0000 7f04077fe640 1 --2- 172.21.15.43:0/1688479803 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f03fc0a3d20 0x7f03fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.627+0000 7f040e59c640 1 --2- 172.21.15.43:0/2166718360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a282045ea0 0x55a28204e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.627+0000 7f040ed9d640 1 --2- 172.21.15.43:0/1688479803 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f03fc0a3d20 0x7f03fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.627+0000 7f040dd9b640 1 -- 172.21.15.43:0/2166718360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a28204ea80 msgr2=0x55a282050e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.627+0000 7f040dd9b640 1 --2- 172.21.15.43:0/2166718360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a28204ea80 0x55a282050e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.627+0000 7f040dd9b640 1 -- 172.21.15.43:0/2166718360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a282045ea0 msgr2=0x55a28204e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.627+0000 7f040dd9b640 1 --2- 172.21.15.43:0/2166718360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a282045ea0 0x55a28204e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.627+0000 7f040dd9b640 1 -- 172.21.15.43:0/2166718360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a281d867d0 con 0x55a282045580 2024-04-21T08:45:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.627+0000 7f040e59c640 1 --2- 172.21.15.43:0/2166718360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a282045ea0 0x55a28204e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.627+0000 7f04077fe640 1 -- 172.21.15.43:0/1688479803 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f04080f3090 con 0x7f041010c500 2024-04-21T08:45:36.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.627+0000 7f040dd9b640 1 --2- 172.21.15.43:0/2166718360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a282045580 0x55a282045960 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f03f800bd30 tx=0x7f03f8011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85af44af54d1bc41 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:36.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.627+0000 7f0404ff9640 1 -- 172.21.15.43:0/2166718360 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8004030 con 0x55a282045580 2024-04-21T08:45:36.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.627+0000 7f040ed9d640 1 --2- 172.21.15.43:0/1688479803 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f03fc0a3d20 0x7f03fc0a61e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f04080bed00 tx=0x7f04080b4200 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:36.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.627+0000 7f0404ff9640 1 -- 172.21.15.43:0/2166718360 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03f80041d0 con 0x55a282045580 2024-04-21T08:45:36.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.627+0000 7f0404ff9640 1 -- 172.21.15.43:0/2166718360 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8012650 con 0x55a282045580 2024-04-21T08:45:36.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.628+0000 7f0417a10740 1 -- 172.21.15.43:0/2166718360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a282045580 msgr2=0x55a282045960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.628+0000 7f0417a10740 1 --2- 172.21.15.43:0/2166718360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a282045580 0x55a282045960 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f03f800bd30 tx=0x7f03f8011750 comp rx=0 tx=0).stop 2024-04-21T08:45:36.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.628+0000 7f0417a10740 1 -- 172.21.15.43:0/2166718360 shutdown_connections 2024-04-21T08:45:36.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.628+0000 7f0417a10740 1 --2- 172.21.15.43:0/2166718360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a28204ea80 0x55a282050e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.628+0000 7f0417a10740 1 --2- 172.21.15.43:0/2166718360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a282045ea0 0x55a28204e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.628+0000 7f0417a10740 1 --2- 172.21.15.43:0/2166718360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a282045580 0x55a282045960 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.628+0000 7f0417a10740 1 -- 172.21.15.43:0/2166718360 >> 172.21.15.43:0/2166718360 conn(0x55a28203fcb0 msgr2=0x55a282040090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.628+0000 7f0417a10740 1 -- 172.21.15.43:0/2166718360 shutdown_connections 2024-04-21T08:45:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.628+0000 7f0417a10740 1 -- 172.21.15.43:0/2166718360 wait complete. 2024-04-21T08:45:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.629+0000 7f0417a10740 1 -- start start 2024-04-21T08:45:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.629+0000 7f0417a10740 1 Processor -- start 2024-04-21T08:45:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.629+0000 7f0417a10740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a282045580 0x55a28204baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.629+0000 7f0417a10740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a282045ea0 0x55a28204bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.629+0000 7f0417a10740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a28204ea80 0x55a28204c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.629+0000 7f0417a10740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a282051870 con 0x55a28204ea80 2024-04-21T08:45:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.629+0000 7f0417a10740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a281f632f0 con 0x55a282045580 2024-04-21T08:45:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.629+0000 7f0417a10740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a2820519f0 con 0x55a282045ea0 2024-04-21T08:45:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.629+0000 7f0417a10740 10 client.0 _ll_register_callbacks cb 0x55a281f70580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.629+0000 7f040dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a28204ea80 0x55a28204c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.629+0000 7f040dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a28204ea80 0x55a28204c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39644/0 (socket says 172.21.15.43:39644) 2024-04-21T08:45:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.629+0000 7f040dd9b640 1 -- 172.21.15.43:0/5332955 learned_addr learned my addr 172.21.15.43:0/5332955 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.629+0000 7f040dd9b640 1 --2- 172.21.15.43:0/5332955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a282045580 0x55a28204baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.629+0000 7f040e59c640 1 --2- 172.21.15.43:0/5332955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a282045ea0 0x55a28204bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.629+0000 7f040dd9b640 1 -- 172.21.15.43:0/5332955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a282045ea0 msgr2=0x55a28204bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.629+0000 7f040dd9b640 1 --2- 172.21.15.43:0/5332955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a282045ea0 0x55a28204bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.629+0000 7f040dd9b640 1 -- 172.21.15.43:0/5332955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a282045580 msgr2=0x55a28204baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.629+0000 7f040dd9b640 1 --2- 172.21.15.43:0/5332955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a282045580 0x55a28204baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.629+0000 7f040dd9b640 1 -- 172.21.15.43:0/5332955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a281cbc3f0 con 0x55a28204ea80 2024-04-21T08:45:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.630+0000 7f040dd9b640 1 --2- 172.21.15.43:0/5332955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a28204ea80 0x55a28204c520 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f03f800bb70 tx=0x7f03f8003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f677bfb889abef server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.630+0000 7f03f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.630+0000 7f03f5ffb640 1 -- 172.21.15.43:0/5332955 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8025040 con 0x55a28204ea80 2024-04-21T08:45:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.630+0000 7f03f5ffb640 1 -- 172.21.15.43:0/5332955 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f03f8005460 con 0x55a28204ea80 2024-04-21T08:45:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.630+0000 7f0417a10740 1 -- 172.21.15.43:0/5332955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03f80041d0 con 0x55a28204ea80 2024-04-21T08:45:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.630+0000 7f03f5ffb640 1 -- 172.21.15.43:0/5332955 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f03f80035f0 con 0x55a28204ea80 2024-04-21T08:45:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.630+0000 7f0417a10740 10 client.18693 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.630+0000 7f0417a10740 1 -- 172.21.15.43:0/5332955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a281c7c0f0 con 0x55a28204ea80 2024-04-21T08:45:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.630+0000 7f03f5ffb640 1 -- 172.21.15.43:0/5332955 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03f80041d0 con 0x55a28204ea80 2024-04-21T08:45:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.630+0000 7f03f5ffb640 1 -- 172.21.15.43:0/5332955 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f03f8012b60 con 0x55a28204ea80 2024-04-21T08:45:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.630+0000 7f0417a10740 10 client.18693 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.630+0000 7f0417a10740 10 client.18693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.630+0000 7f0417a10740 20 client.18693 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.630+0000 7f0417a10740 1 --2- 172.21.15.43:0/5332955 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a28205ea20 0x55a282060e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.630+0000 7f0417a10740 4 client.18693 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.630+0000 7f0417a10740 1 -- 172.21.15.43:0/5332955 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a281fb7c00 con 0x55a28205ea20 2024-04-21T08:45:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.631+0000 7f040e59c640 1 --2- 172.21.15.43:0/5332955 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a28205ea20 0x55a282060e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.631+0000 7f03f5ffb640 10 client.18693 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.631+0000 7f040e59c640 1 --2- 172.21.15.43:0/5332955 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a28205ea20 0x55a282060e10 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=324900c5e480421d server_cookie=b80dfe21dd19a5d0 in_seq=0 out_seq=0 2024-04-21T08:45:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.632+0000 7f03f5ffb640 1 -- 172.21.15.43:0/5332955 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a281fb7c00 con 0x55a28205ea20 2024-04-21T08:45:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.632+0000 7f03f5ffb640 10 client.18693 handle_command_reply: tid=0 2024-04-21T08:45:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.632+0000 7f0417a10740 1 client.18693 shutdown 2024-04-21T08:45:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.632+0000 7f0417a10740 1 -- 172.21.15.43:0/5332955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a28204ea80 msgr2=0x55a28204c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.632+0000 7f0417a10740 1 --2- 172.21.15.43:0/5332955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a28204ea80 0x55a28204c520 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f03f800bb70 tx=0x7f03f8003e50 comp rx=0 tx=0).stop 2024-04-21T08:45:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.632+0000 7f0417a10740 1 -- 172.21.15.43:0/5332955 shutdown_connections 2024-04-21T08:45:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.632+0000 7f0417a10740 1 --2- 172.21.15.43:0/5332955 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a28205ea20 0x55a282060e10 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.632+0000 7f0417a10740 1 --2- 172.21.15.43:0/5332955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a28204ea80 0x55a28204c520 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.632+0000 7f03f5ffb640 0 client.18693 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.632+0000 7f0417a10740 1 --2- 172.21.15.43:0/5332955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a282045ea0 0x55a28204bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.632+0000 7f0417a10740 1 --2- 172.21.15.43:0/5332955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a282045580 0x55a28204baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.632+0000 7f0417a10740 1 -- 172.21.15.43:0/5332955 >> 172.21.15.43:0/5332955 conn(0x55a28203fcb0 msgr2=0x55a282041a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.632+0000 7f0417a10740 1 -- 172.21.15.43:0/5332955 shutdown_connections 2024-04-21T08:45:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.632+0000 7f0417a10740 1 -- 172.21.15.43:0/5332955 wait complete. 2024-04-21T08:45:36.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.632+0000 7f0417a10740 20 client.18693 trim_cache size 0 max 16384 2024-04-21T08:45:36.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.649+0000 7f0417a10740 1 Processor -- start 2024-04-21T08:45:36.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.649+0000 7f0417a10740 1 -- start start 2024-04-21T08:45:36.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.649+0000 7f0417a10740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a282045580 0x55a28205a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.649+0000 7f0417a10740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a282045ea0 0x55a28203ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.649+0000 7f0417a10740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a28204ea80 0x55a28204c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.649+0000 7f0417a10740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f04101af430 con 0x55a282045580 2024-04-21T08:45:36.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.649+0000 7f0417a10740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f04101b0f70 con 0x55a28204ea80 2024-04-21T08:45:36.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.649+0000 7f0417a10740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04101afb00 con 0x55a282045ea0 2024-04-21T08:45:36.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.649+0000 7f040dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a282045580 0x55a28205a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.649+0000 7f040dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a282045580 0x55a28205a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39658/0 (socket says 172.21.15.43:39658) 2024-04-21T08:45:36.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.649+0000 7f040dd9b640 1 -- 172.21.15.43:0/1438418846 learned_addr learned my addr 172.21.15.43:0/1438418846 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:36.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.650+0000 7f040dd9b640 1 -- 172.21.15.43:0/1438418846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a282045ea0 msgr2=0x55a28203ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:36.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.650+0000 7f040dd9b640 1 --2- 172.21.15.43:0/1438418846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a282045ea0 0x55a28203ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.650+0000 7f040dd9b640 1 -- 172.21.15.43:0/1438418846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a28204ea80 msgr2=0x55a28204c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:36.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.650+0000 7f040dd9b640 1 --2- 172.21.15.43:0/1438418846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a28204ea80 0x55a28204c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.650+0000 7f040dd9b640 1 -- 172.21.15.43:0/1438418846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a282051750 con 0x55a282045580 2024-04-21T08:45:36.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.650+0000 7f040dd9b640 1 --2- 172.21.15.43:0/1438418846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a282045580 0x55a28205a760 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f03f80057f0 tx=0x7f03f8005820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1865fe13be800876 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:36.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.650+0000 7f03f7fff640 1 -- 172.21.15.43:0/1438418846 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8004960 con 0x55a282045580 2024-04-21T08:45:36.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.650+0000 7f03f7fff640 1 -- 172.21.15.43:0/1438418846 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03f8004c80 con 0x55a282045580 2024-04-21T08:45:36.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.650+0000 7f03f7fff640 1 -- 172.21.15.43:0/1438418846 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8003790 con 0x55a282045580 2024-04-21T08:45:36.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.651+0000 7f0417a10740 1 -- 172.21.15.43:0/1438418846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a282045580 msgr2=0x55a28205a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.651+0000 7f0417a10740 1 --2- 172.21.15.43:0/1438418846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a282045580 0x55a28205a760 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f03f80057f0 tx=0x7f03f8005820 comp rx=0 tx=0).stop 2024-04-21T08:45:36.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.651+0000 7f0417a10740 1 -- 172.21.15.43:0/1438418846 shutdown_connections 2024-04-21T08:45:36.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.651+0000 7f0417a10740 1 --2- 172.21.15.43:0/1438418846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a28204ea80 0x55a28204c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.651+0000 7f0417a10740 1 --2- 172.21.15.43:0/1438418846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a282045ea0 0x55a28203ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.651+0000 7f0417a10740 1 --2- 172.21.15.43:0/1438418846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a282045580 0x55a28205a760 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.651+0000 7f0417a10740 1 -- 172.21.15.43:0/1438418846 >> 172.21.15.43:0/1438418846 conn(0x55a28203fcb0 msgr2=0x55a28204ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:36.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.651+0000 7f0417a10740 1 -- 172.21.15.43:0/1438418846 shutdown_connections 2024-04-21T08:45:36.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.651+0000 7f0417a10740 1 -- 172.21.15.43:0/1438418846 wait complete. 2024-04-21T08:45:36.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.651+0000 7f0417a10740 1 -- start start 2024-04-21T08:45:36.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.651+0000 7f0417a10740 1 Processor -- start 2024-04-21T08:45:36.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.651+0000 7f0417a10740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a282045580 0x55a2820489d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.651+0000 7f0417a10740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a282045ea0 0x55a282048f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.651+0000 7f0417a10740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a28204ea80 0x55a282049450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.651+0000 7f0417a10740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f04101b08a0 con 0x55a28204ea80 2024-04-21T08:45:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.651+0000 7f0417a10740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f04101b01d0 con 0x55a282045ea0 2024-04-21T08:45:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.651+0000 7f0417a10740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04101ad410 con 0x55a282045580 2024-04-21T08:45:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.651+0000 7f0417a10740 10 client.0 _ll_register_callbacks cb 0x7f03f800bc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.651+0000 7f040dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a28204ea80 0x55a282049450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.651+0000 7f040dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a28204ea80 0x55a282049450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39670/0 (socket says 172.21.15.43:39670) 2024-04-21T08:45:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.651+0000 7f040dd9b640 1 -- 172.21.15.43:0/2254011624 learned_addr learned my addr 172.21.15.43:0/2254011624 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.652+0000 7f040e59c640 1 --2- 172.21.15.43:0/2254011624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a282045ea0 0x55a282048f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.652+0000 7f040dd9b640 1 -- 172.21.15.43:0/2254011624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a282045580 msgr2=0x55a2820489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.652+0000 7f040dd9b640 1 --2- 172.21.15.43:0/2254011624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a282045580 0x55a2820489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.652+0000 7f040dd9b640 1 -- 172.21.15.43:0/2254011624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a282045ea0 msgr2=0x55a282048f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.652+0000 7f040dd9b640 1 --2- 172.21.15.43:0/2254011624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a282045ea0 0x55a282048f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.652+0000 7f040dd9b640 1 -- 172.21.15.43:0/2254011624 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2820c3f90 con 0x55a28204ea80 2024-04-21T08:45:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.652+0000 7f040dd9b640 1 --2- 172.21.15.43:0/2254011624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a28204ea80 0x55a282049450 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f03f8012010 tx=0x7f03f8004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1de930d2f99e294 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.652+0000 7f03f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.652+0000 7f03f5ffb640 1 -- 172.21.15.43:0/2254011624 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8046060 con 0x55a28204ea80 2024-04-21T08:45:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.652+0000 7f03f5ffb640 1 -- 172.21.15.43:0/2254011624 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f03f8005260 con 0x55a28204ea80 2024-04-21T08:45:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.652+0000 7f0417a10740 1 -- 172.21.15.43:0/2254011624 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03f8004c80 con 0x55a28204ea80 2024-04-21T08:45:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.652+0000 7f03f5ffb640 1 -- 172.21.15.43:0/2254011624 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f03f8005460 con 0x55a28204ea80 2024-04-21T08:45:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.652+0000 7f0417a10740 10 client.18699 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.652+0000 7f0417a10740 1 -- 172.21.15.43:0/2254011624 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a2820c5260 con 0x55a28204ea80 2024-04-21T08:45:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.652+0000 7f03f5ffb640 1 -- 172.21.15.43:0/2254011624 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03f8004c80 con 0x55a28204ea80 2024-04-21T08:45:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.652+0000 7f03f5ffb640 1 -- 172.21.15.43:0/2254011624 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f03f8012b10 con 0x55a28204ea80 2024-04-21T08:45:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.652+0000 7f0417a10740 10 client.18699 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.653+0000 7f0417a10740 10 client.18699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.653+0000 7f0417a10740 20 client.18699 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.653+0000 7f0417a10740 1 --2- 172.21.15.43:0/2254011624 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a2820c5820 0x55a2820c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.653+0000 7f0417a10740 4 client.18699 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:45:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.653+0000 7f0417a10740 1 -- 172.21.15.43:0/2254011624 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a281fb1a70 con 0x55a2820c5820 2024-04-21T08:45:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.653+0000 7f040e59c640 1 --2- 172.21.15.43:0/2254011624 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a2820c5820 0x55a2820c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.653+0000 7f040e59c640 1 --2- 172.21.15.43:0/2254011624 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a2820c5820 0x55a2820c7c10 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9581f67cb8b59c09 server_cookie=a2da915aeb9207e5 in_seq=0 out_seq=0 2024-04-21T08:45:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.653+0000 7f03f5ffb640 10 client.18699 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.653+0000 7f03f5ffb640 1 -- 172.21.15.43:0/2254011624 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a281fb1a70 con 0x55a2820c5820 2024-04-21T08:45:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.653+0000 7f03f5ffb640 10 client.18699 handle_command_reply: tid=0 2024-04-21T08:45:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.653+0000 7f0417a10740 1 client.18699 shutdown 2024-04-21T08:45:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.653+0000 7f0417a10740 1 -- 172.21.15.43:0/2254011624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a28204ea80 msgr2=0x55a282049450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.653+0000 7f0417a10740 1 --2- 172.21.15.43:0/2254011624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a28204ea80 0x55a282049450 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f03f8012010 tx=0x7f03f8004f60 comp rx=0 tx=0).stop 2024-04-21T08:45:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.653+0000 7f0417a10740 1 -- 172.21.15.43:0/2254011624 shutdown_connections 2024-04-21T08:45:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.653+0000 7f0417a10740 1 --2- 172.21.15.43:0/2254011624 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a2820c5820 0x55a2820c7c10 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.653+0000 7f0417a10740 1 --2- 172.21.15.43:0/2254011624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a28204ea80 0x55a282049450 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.653+0000 7f03f5ffb640 0 client.18699 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:36.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.653+0000 7f0417a10740 1 --2- 172.21.15.43:0/2254011624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a282045ea0 0x55a282048f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.653+0000 7f0417a10740 1 --2- 172.21.15.43:0/2254011624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a282045580 0x55a2820489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.654+0000 7f0417a10740 1 -- 172.21.15.43:0/2254011624 >> 172.21.15.43:0/2254011624 conn(0x55a28203fcb0 msgr2=0x55a28203ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:36.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.654+0000 7f0417a10740 1 -- 172.21.15.43:0/2254011624 shutdown_connections 2024-04-21T08:45:36.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.654+0000 7f0417a10740 1 -- 172.21.15.43:0/2254011624 wait complete. 2024-04-21T08:45:36.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.654+0000 7f0417a10740 20 client.18699 trim_cache size 0 max 16384 2024-04-21T08:45:36.661 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:45:36.661 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:45:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-21T08:45:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "79159daa-f79c-43f3-b831-f578646f88dd": { 2024-04-21T08:45:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-21T08:45:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "79159daa-f79c-43f3-b831-f578646f88dd", 2024-04-21T08:45:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-21T08:45:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:45:36.662 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:45:36.662 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:45:36.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.654+0000 7f0414e26640 1 -- 172.21.15.43:0/1688479803 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f03fc0a3d20 msgr2=0x7f03fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.654+0000 7f0414e26640 1 --2- 172.21.15.43:0/1688479803 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f03fc0a3d20 0x7f03fc0a61e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f04080bed00 tx=0x7f04080b4200 comp rx=0 tx=0).stop 2024-04-21T08:45:36.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.654+0000 7f0414e26640 1 -- 172.21.15.43:0/1688479803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041010c500 msgr2=0x7f041019cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.654+0000 7f0414e26640 1 --2- 172.21.15.43:0/1688479803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041010c500 0x7f041019cf50 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f040802c840 tx=0x7f0408003c20 comp rx=0 tx=0).stop 2024-04-21T08:45:36.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.654+0000 7f0414e26640 1 -- 172.21.15.43:0/1688479803 shutdown_connections 2024-04-21T08:45:36.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.654+0000 7f0414e26640 1 --2- 172.21.15.43:0/1688479803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041010c500 0x7f041019cf50 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.654+0000 7f0414e26640 1 --2- 172.21.15.43:0/1688479803 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f03fc0a3d20 0x7f03fc0a61e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.654+0000 7f0414e26640 1 --2- 172.21.15.43:0/1688479803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0410103ae0 0x7f0410198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.654+0000 7f0414e26640 1 --2- 172.21.15.43:0/1688479803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04101031c0 0x7f0410198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.654+0000 7f0414e26640 1 -- 172.21.15.43:0/1688479803 >> 172.21.15.43:0/1688479803 conn(0x7f0410078000 msgr2=0x7f04100fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:36.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.654+0000 7f0414e26640 1 -- 172.21.15.43:0/1688479803 shutdown_connections 2024-04-21T08:45:36.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:36.654+0000 7f0414e26640 1 -- 172.21.15.43:0/1688479803 wait complete. 2024-04-21T08:45:36.665 INFO:tasks.cephfs.filesystem:scrub status for tag:79159daa-f79c-43f3-b831-f578646f88dd - {'path': '/', 'tag': '79159daa-f79c-43f3-b831-f578646f88dd', 'options': 'recursive,force'} 2024-04-21T08:45:38.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:38 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:45:38.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:38 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:38.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:38 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:45:38.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:38 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:38.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:38 smithi175 ceph-mon[31920]: pgmap v204: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.3 KiB/s wr, 16 op/s 2024-04-21T08:45:38.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:38 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:45:38.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:38 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:38.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:38 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:45:38.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:38 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:38.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:38 smithi112 ceph-mon[31085]: pgmap v204: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.3 KiB/s wr, 16 op/s 2024-04-21T08:45:38.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:38 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:45:38.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:38 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:38.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:38 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:45:38.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:38 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:38.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:38 smithi043 ceph-mon[27978]: pgmap v204: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.3 KiB/s wr, 16 op/s 2024-04-21T08:45:40.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:40 smithi175 ceph-mon[31920]: pgmap v205: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-04-21T08:45:40.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:40 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:45:40.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:40 smithi112 ceph-mon[31085]: pgmap v205: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-04-21T08:45:40.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:40 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:45:40.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:40 smithi043 ceph-mon[27978]: pgmap v205: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-04-21T08:45:40.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:40 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:45:42.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:42 smithi175 ceph-mon[31920]: pgmap v206: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.7 KiB/s wr, 17 op/s 2024-04-21T08:45:42.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:42 smithi112 ceph-mon[31085]: pgmap v206: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.7 KiB/s wr, 17 op/s 2024-04-21T08:45:42.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:42 smithi043 ceph-mon[27978]: pgmap v206: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.7 KiB/s wr, 17 op/s 2024-04-21T08:45:44.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:44 smithi175 ceph-mon[31920]: pgmap v207: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-04-21T08:45:44.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:44 smithi112 ceph-mon[31085]: pgmap v207: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-04-21T08:45:44.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:44 smithi043 ceph-mon[27978]: pgmap v207: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-04-21T08:45:46.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:46 smithi175 ceph-mon[31920]: pgmap v208: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-04-21T08:45:46.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:46 smithi112 ceph-mon[31085]: pgmap v208: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-04-21T08:45:46.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:46 smithi043 ceph-mon[27978]: pgmap v208: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-04-21T08:45:46.776 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:45:46.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.892+0000 7fc910206640 1 -- 172.21.15.43:0/813552047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9081011a0 msgr2=0x7fc908101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:46.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.892+0000 7fc910206640 1 --2- 172.21.15.43:0/813552047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9081011a0 0x7fc908101620 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8009930 tx=0x7fc8f802bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:46.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.892+0000 7fc910206640 1 -- 172.21.15.43:0/813552047 shutdown_connections 2024-04-21T08:45:46.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.892+0000 7fc910206640 1 --2- 172.21.15.43:0/813552047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc908101b60 0x7fc90810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:46.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.892+0000 7fc910206640 1 --2- 172.21.15.43:0/813552047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9081011a0 0x7fc908101620 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.892+0000 7fc910206640 1 --2- 172.21.15.43:0/813552047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc908105e30 0x7fc908106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.892+0000 7fc910206640 1 -- 172.21.15.43:0/813552047 >> 172.21.15.43:0/813552047 conn(0x7fc908077e30 msgr2=0x7fc908078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.892+0000 7fc910206640 1 -- 172.21.15.43:0/813552047 shutdown_connections 2024-04-21T08:45:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.892+0000 7fc910206640 1 -- 172.21.15.43:0/813552047 wait complete. 2024-04-21T08:45:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.892+0000 7fc910206640 1 Processor -- start 2024-04-21T08:45:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.893+0000 7fc910206640 1 -- start start 2024-04-21T08:45:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.893+0000 7fc910206640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9081011a0 0x7fc908198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.893+0000 7fc910206640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc908101b60 0x7fc908198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.893+0000 7fc910206640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc908105e30 0x7fc90819d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.893+0000 7fc910206640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc908112250 con 0x7fc9081011a0 2024-04-21T08:45:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.893+0000 7fc910206640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc9081120d0 con 0x7fc908105e30 2024-04-21T08:45:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.893+0000 7fc910206640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc9081123d0 con 0x7fc908101b60 2024-04-21T08:45:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.893+0000 7fc90dfa2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9081011a0 0x7fc908198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.893+0000 7fc90dfa2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9081011a0 0x7fc908198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43706/0 (socket says 172.21.15.43:43706) 2024-04-21T08:45:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.893+0000 7fc90dfa2640 1 -- 172.21.15.43:0/2854108529 learned_addr learned my addr 172.21.15.43:0/2854108529 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.893+0000 7fc90d7a1640 1 --2- 172.21.15.43:0/2854108529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc908101b60 0x7fc908198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.893+0000 7fc90e7a3640 1 --2- 172.21.15.43:0/2854108529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc908105e30 0x7fc90819d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.893+0000 7fc90dfa2640 1 -- 172.21.15.43:0/2854108529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc908101b60 msgr2=0x7fc908198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.893+0000 7fc90dfa2640 1 --2- 172.21.15.43:0/2854108529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc908101b60 0x7fc908198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.893+0000 7fc90dfa2640 1 -- 172.21.15.43:0/2854108529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc908105e30 msgr2=0x7fc90819d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.893+0000 7fc90dfa2640 1 --2- 172.21.15.43:0/2854108529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc908105e30 0x7fc90819d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.893+0000 7fc90dfa2640 1 -- 172.21.15.43:0/2854108529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90819d730 con 0x7fc9081011a0 2024-04-21T08:45:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.893+0000 7fc90e7a3640 1 --2- 172.21.15.43:0/2854108529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc908105e30 0x7fc90819d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.893+0000 7fc90dfa2640 1 --2- 172.21.15.43:0/2854108529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9081011a0 0x7fc908198790 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fc90400cce0 tx=0x7fc90400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.894+0000 7fc8feffd640 1 -- 172.21.15.43:0/2854108529 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc904004b20 con 0x7fc9081011a0 2024-04-21T08:45:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.894+0000 7fc8feffd640 1 -- 172.21.15.43:0/2854108529 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc904004cc0 con 0x7fc9081011a0 2024-04-21T08:45:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.894+0000 7fc910206640 1 -- 172.21.15.43:0/2854108529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90819da20 con 0x7fc9081011a0 2024-04-21T08:45:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.894+0000 7fc8feffd640 1 -- 172.21.15.43:0/2854108529 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90400f6b0 con 0x7fc9081011a0 2024-04-21T08:45:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.894+0000 7fc910206640 1 -- 172.21.15.43:0/2854108529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9081a52b0 con 0x7fc9081011a0 2024-04-21T08:45:46.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.895+0000 7fc910206640 1 -- 172.21.15.43:0/2854108529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc908068de0 con 0x7fc9081011a0 2024-04-21T08:45:46.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.896+0000 7fc8feffd640 1 -- 172.21.15.43:0/2854108529 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc90400f850 con 0x7fc9081011a0 2024-04-21T08:45:46.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.896+0000 7fc8feffd640 1 --2- 172.21.15.43:0/2854108529 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc8f40a3d20 0x7fc8f40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:46.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.897+0000 7fc90d7a1640 1 --2- 172.21.15.43:0/2854108529 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc8f40a3d20 0x7fc8f40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:46.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.897+0000 7fc8feffd640 1 -- 172.21.15.43:0/2854108529 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7fc9040d10b0 con 0x7fc9081011a0 2024-04-21T08:45:46.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.897+0000 7fc90d7a1640 1 --2- 172.21.15.43:0/2854108529 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc8f40a3d20 0x7fc8f40a61e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8005ab0 tx=0x7fc8f8005fd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:46.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:46.900+0000 7fc8feffd640 1 -- 172.21.15.43:0/2854108529 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc904014030 con 0x7fc9081011a0 2024-04-21T08:45:47.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.128+0000 7fc910206640 1 -- 172.21.15.43:0/2854108529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc908199a90 con 0x7fc9081011a0 2024-04-21T08:45:47.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.128+0000 7fc8feffd640 1 -- 172.21.15.43:0/2854108529 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7fc90409d320 con 0x7fc9081011a0 2024-04-21T08:45:47.128 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:45:47.128 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:45:47.128 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:45:47.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.130+0000 7fc910206640 1 -- 172.21.15.43:0/2854108529 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc8f40a3d20 msgr2=0x7fc8f40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.130+0000 7fc910206640 1 --2- 172.21.15.43:0/2854108529 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc8f40a3d20 0x7fc8f40a61e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8005ab0 tx=0x7fc8f8005fd0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.130+0000 7fc910206640 1 -- 172.21.15.43:0/2854108529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9081011a0 msgr2=0x7fc908198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.130+0000 7fc910206640 1 --2- 172.21.15.43:0/2854108529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9081011a0 0x7fc908198790 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fc90400cce0 tx=0x7fc90400a620 comp rx=0 tx=0).stop 2024-04-21T08:45:47.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.130+0000 7fc910206640 1 -- 172.21.15.43:0/2854108529 shutdown_connections 2024-04-21T08:45:47.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.130+0000 7fc910206640 1 --2- 172.21.15.43:0/2854108529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc908105e30 0x7fc90819d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.130+0000 7fc910206640 1 --2- 172.21.15.43:0/2854108529 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc8f40a3d20 0x7fc8f40a61e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.130+0000 7fc910206640 1 --2- 172.21.15.43:0/2854108529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc908101b60 0x7fc908198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.130+0000 7fc910206640 1 --2- 172.21.15.43:0/2854108529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9081011a0 0x7fc908198790 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.130+0000 7fc910206640 1 -- 172.21.15.43:0/2854108529 >> 172.21.15.43:0/2854108529 conn(0x7fc908077e30 msgr2=0x7fc9080fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:47.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.130+0000 7fc910206640 1 -- 172.21.15.43:0/2854108529 shutdown_connections 2024-04-21T08:45:47.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.130+0000 7fc910206640 1 -- 172.21.15.43:0/2854108529 wait complete. 2024-04-21T08:45:47.141 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db --set-id 683318c5 --release --await' 2024-04-21T08:45:47.141 DEBUG:teuthology.orchestra.run.smithi043:> timeout 45 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24527 quiesce db --set-id 683318c5 --release --await 2024-04-21T08:45:47.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.252+0000 7f235fd32640 1 -- 172.21.15.43:0/58043561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2358101a50 msgr2=0x7f2358105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.252+0000 7f235fd32640 1 --2- 172.21.15.43:0/58043561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2358101a50 0x7f2358105a90 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f2348009900 tx=0x7f234802bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:47.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.252+0000 7f235fd32640 1 -- 172.21.15.43:0/58043561 shutdown_connections 2024-04-21T08:45:47.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.252+0000 7f235fd32640 1 --2- 172.21.15.43:0/58043561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2358106170 0x7f235810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.252+0000 7f235fd32640 1 --2- 172.21.15.43:0/58043561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2358101a50 0x7f2358105a90 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.252+0000 7f235fd32640 1 --2- 172.21.15.43:0/58043561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23581010a0 0x7f2358101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.252+0000 7f235fd32640 1 -- 172.21.15.43:0/58043561 >> 172.21.15.43:0/58043561 conn(0x7f2358077eb0 msgr2=0x7f23580782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:47.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.252+0000 7f235fd32640 1 -- 172.21.15.43:0/58043561 shutdown_connections 2024-04-21T08:45:47.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.252+0000 7f235fd32640 1 -- 172.21.15.43:0/58043561 wait complete. 2024-04-21T08:45:47.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.253+0000 7f235fd32640 1 Processor -- start 2024-04-21T08:45:47.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.253+0000 7f235fd32640 1 -- start start 2024-04-21T08:45:47.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.253+0000 7f235fd32640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23581010a0 0x7f235819e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.253+0000 7f235fd32640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2358106170 0x7f235819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.253+0000 7f235fd32640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23581985d0 0x7f2358198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.253+0000 7f235fd32640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2358111160 con 0x7f2358106170 2024-04-21T08:45:47.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.253+0000 7f235fd32640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2358110fe0 con 0x7f23581985d0 2024-04-21T08:45:47.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.253+0000 7f235fd32640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23581112e0 con 0x7f23581010a0 2024-04-21T08:45:47.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.253+0000 7f235d2cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2358106170 0x7f235819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.253+0000 7f235d2cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2358106170 0x7f235819e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43714/0 (socket says 172.21.15.43:43714) 2024-04-21T08:45:47.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.253+0000 7f235d2cd640 1 -- 172.21.15.43:0/1788264306 learned_addr learned my addr 172.21.15.43:0/1788264306 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:47.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.253+0000 7f235e2cf640 1 --2- 172.21.15.43:0/1788264306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23581985d0 0x7f2358198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.253+0000 7f235dace640 1 --2- 172.21.15.43:0/1788264306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23581010a0 0x7f235819e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.253+0000 7f235d2cd640 1 -- 172.21.15.43:0/1788264306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23581010a0 msgr2=0x7f235819e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.253+0000 7f235d2cd640 1 --2- 172.21.15.43:0/1788264306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23581010a0 0x7f235819e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.253+0000 7f235d2cd640 1 -- 172.21.15.43:0/1788264306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23581985d0 msgr2=0x7f2358198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.253+0000 7f235d2cd640 1 --2- 172.21.15.43:0/1788264306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23581985d0 0x7f2358198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.253+0000 7f235d2cd640 1 -- 172.21.15.43:0/1788264306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23581990f0 con 0x7f2358106170 2024-04-21T08:45:47.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.254+0000 7f235d2cd640 1 --2- 172.21.15.43:0/1788264306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2358106170 0x7f235819e960 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f2348000c00 tx=0x7f234802c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:47.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.254+0000 7f234effd640 1 -- 172.21.15.43:0/1788264306 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348043020 con 0x7f2358106170 2024-04-21T08:45:47.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.254+0000 7f234effd640 1 -- 172.21.15.43:0/1788264306 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f234802cc60 con 0x7f2358106170 2024-04-21T08:45:47.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.254+0000 7f235fd32640 1 -- 172.21.15.43:0/1788264306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2358199380 con 0x7f2358106170 2024-04-21T08:45:47.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.254+0000 7f234effd640 1 -- 172.21.15.43:0/1788264306 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348003c30 con 0x7f2358106170 2024-04-21T08:45:47.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.254+0000 7f235fd32640 1 -- 172.21.15.43:0/1788264306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23581a55d0 con 0x7f2358106170 2024-04-21T08:45:47.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.257+0000 7f2362a02740 1 Processor -- start 2024-04-21T08:45:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.257+0000 7f234effd640 1 -- 172.21.15.43:0/1788264306 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2348046020 con 0x7f2358106170 2024-04-21T08:45:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.257+0000 7f2362a02740 1 -- start start 2024-04-21T08:45:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.257+0000 7f2362a02740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f26cb75310 0x55f26cb756f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.257+0000 7f2362a02740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb75c30 0x55f26cb7e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.257+0000 7f2362a02740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f26cb7e810 0x55f26cb80c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.257+0000 7f2362a02740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f26cada500 con 0x55f26cb75c30 2024-04-21T08:45:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.257+0000 7f2362a02740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f26ca806a0 con 0x55f26cb75310 2024-04-21T08:45:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.257+0000 7f2362a02740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f26cad64c0 con 0x55f26cb7e810 2024-04-21T08:45:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.257+0000 7f235dace640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb75c30 0x55f26cb7e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.257+0000 7f235dace640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb75c30 0x55f26cb7e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43724/0 (socket says 172.21.15.43:43724) 2024-04-21T08:45:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.258+0000 7f235dace640 1 -- 172.21.15.43:0/875663218 learned_addr learned my addr 172.21.15.43:0/875663218 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.258+0000 7f235e2cf640 1 --2- 172.21.15.43:0/875663218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f26cb75310 0x55f26cb756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.258+0000 7f235d2cd640 1 --2- 172.21.15.43:0/875663218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f26cb7e810 0x55f26cb80c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.258+0000 7f235dace640 1 -- 172.21.15.43:0/875663218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f26cb7e810 msgr2=0x55f26cb80c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:47.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.258+0000 7f235dace640 1 --2- 172.21.15.43:0/875663218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f26cb7e810 0x55f26cb80c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.258+0000 7f235dace640 1 -- 172.21.15.43:0/875663218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f26cb75310 msgr2=0x55f26cb756f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:47.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.258+0000 7f235dace640 1 --2- 172.21.15.43:0/875663218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f26cb75310 0x55f26cb756f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.258+0000 7f235dace640 1 -- 172.21.15.43:0/875663218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f26ca9f5e0 con 0x55f26cb75c30 2024-04-21T08:45:47.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.258+0000 7f234effd640 1 --2- 172.21.15.43:0/1788264306 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f23440a3d20 0x7f23440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.258+0000 7f235e2cf640 1 --2- 172.21.15.43:0/1788264306 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f23440a3d20 0x7f23440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.258+0000 7f235dace640 1 --2- 172.21.15.43:0/875663218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb75c30 0x55f26cb7e2d0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f235400b820 tx=0x7f235400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e06f9193fa8069 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:47.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.258+0000 7f234effd640 1 -- 172.21.15.43:0/1788264306 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f23480f2cf0 con 0x7f2358106170 2024-04-21T08:45:47.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.258+0000 7f2343fff640 1 -- 172.21.15.43:0/875663218 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235400d950 con 0x55f26cb75c30 2024-04-21T08:45:47.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.258+0000 7f2343fff640 1 -- 172.21.15.43:0/875663218 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2354019070 con 0x55f26cb75c30 2024-04-21T08:45:47.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.258+0000 7f2343fff640 1 -- 172.21.15.43:0/875663218 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23540123e0 con 0x55f26cb75c30 2024-04-21T08:45:47.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.259+0000 7f2362a02740 1 -- 172.21.15.43:0/875663218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb75c30 msgr2=0x55f26cb7e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.259+0000 7f235e2cf640 1 --2- 172.21.15.43:0/1788264306 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f23440a3d20 0x7f23440a61e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2350005fd0 tx=0x7f2350009600 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:47.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.259+0000 7f2362a02740 1 --2- 172.21.15.43:0/875663218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb75c30 0x55f26cb7e2d0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f235400b820 tx=0x7f235400bce0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.259+0000 7f2362a02740 1 -- 172.21.15.43:0/875663218 shutdown_connections 2024-04-21T08:45:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.259+0000 7f2362a02740 1 --2- 172.21.15.43:0/875663218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f26cb7e810 0x55f26cb80c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.259+0000 7f2362a02740 1 --2- 172.21.15.43:0/875663218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb75c30 0x55f26cb7e2d0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.259+0000 7f2362a02740 1 --2- 172.21.15.43:0/875663218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f26cb75310 0x55f26cb756f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.259+0000 7f2362a02740 1 -- 172.21.15.43:0/875663218 >> 172.21.15.43:0/875663218 conn(0x55f26cb6fa40 msgr2=0x55f26cb6fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.259+0000 7f2362a02740 1 -- 172.21.15.43:0/875663218 shutdown_connections 2024-04-21T08:45:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.259+0000 7f2362a02740 1 -- 172.21.15.43:0/875663218 wait complete. 2024-04-21T08:45:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.259+0000 7f2362a02740 1 -- start start 2024-04-21T08:45:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.260+0000 7f2362a02740 1 Processor -- start 2024-04-21T08:45:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.260+0000 7f2362a02740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb75310 0x55f26cb7e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.260+0000 7f2362a02740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f26cb75c30 0x55f26cb7b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.260+0000 7f2362a02740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f26cb7e810 0x55f26cb7bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.260+0000 7f2362a02740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f26cb81600 con 0x55f26cb75310 2024-04-21T08:45:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.260+0000 7f2362a02740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f26ca93a60 con 0x55f26cb7e810 2024-04-21T08:45:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.260+0000 7f2362a02740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f26cb81780 con 0x55f26cb75c30 2024-04-21T08:45:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.260+0000 7f2362a02740 10 client.0 _ll_register_callbacks cb 0x55f26c7ec110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.260+0000 7f235dace640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb75310 0x55f26cb7e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.260+0000 7f235dace640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb75310 0x55f26cb7e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43728/0 (socket says 172.21.15.43:43728) 2024-04-21T08:45:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.260+0000 7f235dace640 1 -- 172.21.15.43:0/1215049538 learned_addr learned my addr 172.21.15.43:0/1215049538 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.260+0000 7f235d2cd640 1 --2- 172.21.15.43:0/1215049538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f26cb75c30 0x55f26cb7b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.260+0000 7f235e2cf640 1 --2- 172.21.15.43:0/1215049538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f26cb7e810 0x55f26cb7bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.260+0000 7f235dace640 1 -- 172.21.15.43:0/1215049538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f26cb75c30 msgr2=0x55f26cb7b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.260+0000 7f235dace640 1 --2- 172.21.15.43:0/1215049538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f26cb75c30 0x55f26cb7b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.260+0000 7f235dace640 1 -- 172.21.15.43:0/1215049538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f26cb7e810 msgr2=0x55f26cb7bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.260+0000 7f235dace640 1 --2- 172.21.15.43:0/1215049538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f26cb7e810 0x55f26cb7bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.260+0000 7f235dace640 1 -- 172.21.15.43:0/1215049538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f26c7ec920 con 0x55f26cb75310 2024-04-21T08:45:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.261+0000 7f235dace640 1 --2- 172.21.15.43:0/1215049538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb75310 0x55f26cb7e2b0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f2354004700 tx=0x7f2354014e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3690886d3b98336c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.261+0000 7f23417fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.261+0000 7f23417fa640 1 -- 172.21.15.43:0/1215049538 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2354023070 con 0x55f26cb75310 2024-04-21T08:45:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.261+0000 7f23417fa640 1 -- 172.21.15.43:0/1215049538 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f235400d950 con 0x55f26cb75310 2024-04-21T08:45:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.261+0000 7f2362a02740 1 -- 172.21.15.43:0/1215049538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2354019070 con 0x55f26cb75310 2024-04-21T08:45:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.261+0000 7f23417fa640 1 -- 172.21.15.43:0/1215049538 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f235402f050 con 0x55f26cb75310 2024-04-21T08:45:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.261+0000 7f2362a02740 10 client.18717 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.261+0000 7f2362a02740 1 -- 172.21.15.43:0/1215049538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f26c8b8d30 con 0x55f26cb75310 2024-04-21T08:45:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.261+0000 7f23417fa640 1 -- 172.21.15.43:0/1215049538 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2354019070 con 0x55f26cb75310 2024-04-21T08:45:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.261+0000 7f23417fa640 1 -- 172.21.15.43:0/1215049538 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2354013070 con 0x55f26cb75310 2024-04-21T08:45:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.261+0000 7f2362a02740 10 client.18717 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.261+0000 7f2362a02740 10 client.18717 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-21T08:45:47.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.261+0000 7f2362a02740 20 client.18717 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:47.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.261+0000 7f2362a02740 1 --2- 172.21.15.43:0/1215049538 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f26cb8e870 0x55f26cb90c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.261+0000 7f2362a02740 4 client.18717 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:47.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.261+0000 7f2362a02740 1 -- 172.21.15.43:0/1215049538 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f26caada10 con 0x55f26cb8e870 2024-04-21T08:45:47.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.262+0000 7f235d2cd640 1 --2- 172.21.15.43:0/1215049538 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f26cb8e870 0x55f26cb90c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.262+0000 7f235d2cd640 1 --2- 172.21.15.43:0/1215049538 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f26cb8e870 0x55f26cb90c60 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54dfe6a39bbfb36b server_cookie=7a8bd4847a2d01bf in_seq=0 out_seq=0 2024-04-21T08:45:47.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.262+0000 7f23417fa640 10 client.18717 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:47.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.263+0000 7f23417fa640 1 -- 172.21.15.43:0/1215049538 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f26caada10 con 0x55f26cb8e870 2024-04-21T08:45:47.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.263+0000 7f23417fa640 10 client.18717 handle_command_reply: tid=0 2024-04-21T08:45:47.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.263+0000 7f2362a02740 1 client.18717 shutdown 2024-04-21T08:45:47.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.263+0000 7f2362a02740 1 -- 172.21.15.43:0/1215049538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb75310 msgr2=0x55f26cb7e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.263+0000 7f2362a02740 1 --2- 172.21.15.43:0/1215049538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb75310 0x55f26cb7e2b0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f2354004700 tx=0x7f2354014e70 comp rx=0 tx=0).stop 2024-04-21T08:45:47.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.263+0000 7f2362a02740 1 -- 172.21.15.43:0/1215049538 shutdown_connections 2024-04-21T08:45:47.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.263+0000 7f2362a02740 1 --2- 172.21.15.43:0/1215049538 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f26cb8e870 0x55f26cb90c60 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.263+0000 7f2362a02740 1 --2- 172.21.15.43:0/1215049538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f26cb7e810 0x55f26cb7bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.263+0000 7f2362a02740 1 --2- 172.21.15.43:0/1215049538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f26cb75c30 0x55f26cb7b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.263+0000 7f2362a02740 1 --2- 172.21.15.43:0/1215049538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb75310 0x55f26cb7e2b0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.263+0000 7f23417fa640 0 client.18717 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:47.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.263+0000 7f2362a02740 1 -- 172.21.15.43:0/1215049538 >> 172.21.15.43:0/1215049538 conn(0x55f26cb6fa40 msgr2=0x55f26cb71790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:47.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.263+0000 7f2362a02740 1 -- 172.21.15.43:0/1215049538 shutdown_connections 2024-04-21T08:45:47.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.263+0000 7f2362a02740 1 -- 172.21.15.43:0/1215049538 wait complete. 2024-04-21T08:45:47.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.263+0000 7f2362a02740 20 client.18717 trim_cache size 0 max 16384 2024-04-21T08:45:47.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.281+0000 7f2362a02740 1 Processor -- start 2024-04-21T08:45:47.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.281+0000 7f2362a02740 1 -- start start 2024-04-21T08:45:47.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.281+0000 7f2362a02740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f26cb75310 0x55f26cb6e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.281+0000 7f2362a02740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f26cb75c30 0x55f26cb7dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.281+0000 7f2362a02740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb7e810 0x55f26cb7e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.281+0000 7f2362a02740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f23581af4d0 con 0x55f26cb7e810 2024-04-21T08:45:47.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.281+0000 7f2362a02740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f23581b1010 con 0x55f26cb75310 2024-04-21T08:45:47.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.281+0000 7f2362a02740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23581afba0 con 0x55f26cb75c30 2024-04-21T08:45:47.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.281+0000 7f235e2cf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb7e810 0x55f26cb7e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.281+0000 7f235e2cf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb7e810 0x55f26cb7e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43732/0 (socket says 172.21.15.43:43732) 2024-04-21T08:45:47.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.281+0000 7f235e2cf640 1 -- 172.21.15.43:0/2956226640 learned_addr learned my addr 172.21.15.43:0/2956226640 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:47.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.281+0000 7f235dace640 1 --2- 172.21.15.43:0/2956226640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f26cb75310 0x55f26cb6e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.281+0000 7f235e2cf640 1 -- 172.21.15.43:0/2956226640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f26cb75c30 msgr2=0x55f26cb7dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:47.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.281+0000 7f235e2cf640 1 --2- 172.21.15.43:0/2956226640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f26cb75c30 0x55f26cb7dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.281+0000 7f235e2cf640 1 -- 172.21.15.43:0/2956226640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f26cb75310 msgr2=0x55f26cb6e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:47.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.281+0000 7f235e2cf640 1 --2- 172.21.15.43:0/2956226640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f26cb75310 0x55f26cb6e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.281+0000 7f235e2cf640 1 -- 172.21.15.43:0/2956226640 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f26cb7c0a0 con 0x55f26cb7e810 2024-04-21T08:45:47.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.281+0000 7f235e2cf640 1 --2- 172.21.15.43:0/2956226640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb7e810 0x55f26cb7e2b0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f2350003bf0 tx=0x7f2350015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7d6265b99966e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:47.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.281+0000 7f23437fe640 1 -- 172.21.15.43:0/2956226640 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235002b070 con 0x55f26cb7e810 2024-04-21T08:45:47.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.282+0000 7f23437fe640 1 -- 172.21.15.43:0/2956226640 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23500046c0 con 0x55f26cb7e810 2024-04-21T08:45:47.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.282+0000 7f23437fe640 1 -- 172.21.15.43:0/2956226640 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235001c770 con 0x55f26cb7e810 2024-04-21T08:45:47.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.282+0000 7f2362a02740 1 -- 172.21.15.43:0/2956226640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb7e810 msgr2=0x55f26cb7e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.282+0000 7f2362a02740 1 --2- 172.21.15.43:0/2956226640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb7e810 0x55f26cb7e2b0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f2350003bf0 tx=0x7f2350015000 comp rx=0 tx=0).stop 2024-04-21T08:45:47.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.282+0000 7f2362a02740 1 -- 172.21.15.43:0/2956226640 shutdown_connections 2024-04-21T08:45:47.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.282+0000 7f2362a02740 1 --2- 172.21.15.43:0/2956226640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb7e810 0x55f26cb7e2b0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.282+0000 7f2362a02740 1 --2- 172.21.15.43:0/2956226640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f26cb75c30 0x55f26cb7dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.282+0000 7f2362a02740 1 --2- 172.21.15.43:0/2956226640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f26cb75310 0x55f26cb6e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.282+0000 7f2362a02740 1 -- 172.21.15.43:0/2956226640 >> 172.21.15.43:0/2956226640 conn(0x55f26cb6fa40 msgr2=0x55f26cb6e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:47.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.282+0000 7f2362a02740 1 -- 172.21.15.43:0/2956226640 shutdown_connections 2024-04-21T08:45:47.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.282+0000 7f2362a02740 1 -- 172.21.15.43:0/2956226640 wait complete. 2024-04-21T08:45:47.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.282+0000 7f2362a02740 1 -- start start 2024-04-21T08:45:47.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.282+0000 7f2362a02740 1 Processor -- start 2024-04-21T08:45:47.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.283+0000 7f2362a02740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f26cb75310 0x55f26cb86bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.283+0000 7f2362a02740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f26cb75c30 0x55f26cb81990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.283+0000 7f2362a02740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb7e810 0x55f26cb81ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.283+0000 7f2362a02740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f23581b0940 con 0x55f26cb7e810 2024-04-21T08:45:47.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.283+0000 7f2362a02740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f23581b0270 con 0x55f26cb75310 2024-04-21T08:45:47.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.283+0000 7f2362a02740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23581ad490 con 0x55f26cb75c30 2024-04-21T08:45:47.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.283+0000 7f2362a02740 10 client.0 _ll_register_callbacks cb 0x7f235400b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:47.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.283+0000 7f235e2cf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb7e810 0x55f26cb81ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.283+0000 7f235e2cf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb7e810 0x55f26cb81ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43744/0 (socket says 172.21.15.43:43744) 2024-04-21T08:45:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.283+0000 7f235e2cf640 1 -- 172.21.15.43:0/1199922818 learned_addr learned my addr 172.21.15.43:0/1199922818 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.283+0000 7f235dace640 1 --2- 172.21.15.43:0/1199922818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f26cb75310 0x55f26cb86bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.283+0000 7f235d2cd640 1 --2- 172.21.15.43:0/1199922818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f26cb75c30 0x55f26cb81990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.283+0000 7f235e2cf640 1 -- 172.21.15.43:0/1199922818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f26cb75c30 msgr2=0x55f26cb81990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.283+0000 7f235e2cf640 1 --2- 172.21.15.43:0/1199922818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f26cb75c30 0x55f26cb81990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.283+0000 7f235e2cf640 1 -- 172.21.15.43:0/1199922818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f26cb75310 msgr2=0x55f26cb86bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.283+0000 7f235e2cf640 1 --2- 172.21.15.43:0/1199922818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f26cb75310 0x55f26cb86bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.283+0000 7f235e2cf640 1 -- 172.21.15.43:0/1199922818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f26cb82410 con 0x55f26cb7e810 2024-04-21T08:45:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.283+0000 7f235d2cd640 1 --2- 172.21.15.43:0/1199922818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f26cb75c30 0x55f26cb81990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.283+0000 7f235dace640 1 --2- 172.21.15.43:0/1199922818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f26cb75310 0x55f26cb86bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.283+0000 7f235e2cf640 1 --2- 172.21.15.43:0/1199922818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb7e810 0x55f26cb81ed0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f2350029040 tx=0x7f2350026ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36c507cca82005e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.283+0000 7f23417fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:47.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.284+0000 7f23417fa640 1 -- 172.21.15.43:0/1199922818 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235002b040 con 0x55f26cb7e810 2024-04-21T08:45:47.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.284+0000 7f2362a02740 1 -- 172.21.15.43:0/1199922818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23500046c0 con 0x55f26cb7e810 2024-04-21T08:45:47.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.284+0000 7f23417fa640 1 -- 172.21.15.43:0/1199922818 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f23500265d0 con 0x55f26cb7e810 2024-04-21T08:45:47.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.284+0000 7f23417fa640 1 -- 172.21.15.43:0/1199922818 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f23500267d0 con 0x55f26cb7e810 2024-04-21T08:45:47.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.284+0000 7f2362a02740 10 client.18723 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:47.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.284+0000 7f2362a02740 1 -- 172.21.15.43:0/1199922818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f26cbf9c80 con 0x55f26cb7e810 2024-04-21T08:45:47.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.284+0000 7f23417fa640 1 -- 172.21.15.43:0/1199922818 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23500046c0 con 0x55f26cb7e810 2024-04-21T08:45:47.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.284+0000 7f23417fa640 1 -- 172.21.15.43:0/1199922818 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f235000f040 con 0x55f26cb7e810 2024-04-21T08:45:47.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.284+0000 7f2362a02740 10 client.18723 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:47.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.284+0000 7f2362a02740 10 client.18723 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-21T08:45:47.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.284+0000 7f2362a02740 20 client.18723 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:47.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.284+0000 7f2362a02740 1 --2- 172.21.15.43:0/1199922818 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f26cbfa380 0x55f26cbfc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.284+0000 7f2362a02740 4 client.18723 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "683318c5", "release": true, "await": true}] 2024-04-21T08:45:47.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.284+0000 7f2362a02740 1 -- 172.21.15.43:0/1199922818 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "quiesce db", "set_id": "683318c5", "release": true, "await": true}) -- 0x55f26cae7f20 con 0x55f26cbfa380 2024-04-21T08:45:47.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.285+0000 7f235dace640 1 --2- 172.21.15.43:0/1199922818 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f26cbfa380 0x55f26cbfc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.285+0000 7f235dace640 1 --2- 172.21.15.43:0/1199922818 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f26cbfa380 0x55f26cbfc7a0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c66164bb2ed1dd1 server_cookie=cc45140fe53f5ea5 in_seq=0 out_seq=0 2024-04-21T08:45:47.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.285+0000 7f23417fa640 10 client.18723 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:47.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.288+0000 7f23417fa640 1 -- 172.21.15.43:0/1199922818 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55f26cae7f20 con 0x55f26cbfa380 2024-04-21T08:45:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.288+0000 7f23417fa640 10 client.18723 handle_command_reply: tid=0 2024-04-21T08:45:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.288+0000 7f2362a02740 1 client.18723 shutdown 2024-04-21T08:45:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.288+0000 7f2362a02740 1 -- 172.21.15.43:0/1199922818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb7e810 msgr2=0x55f26cb81ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.288+0000 7f2362a02740 1 --2- 172.21.15.43:0/1199922818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb7e810 0x55f26cb81ed0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f2350029040 tx=0x7f2350026ec0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.288+0000 7f2362a02740 1 -- 172.21.15.43:0/1199922818 shutdown_connections 2024-04-21T08:45:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.288+0000 7f2362a02740 1 --2- 172.21.15.43:0/1199922818 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f26cbfa380 0x55f26cbfc7a0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.288+0000 7f2362a02740 1 --2- 172.21.15.43:0/1199922818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f26cb7e810 0x55f26cb81ed0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.288+0000 7f2362a02740 1 --2- 172.21.15.43:0/1199922818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f26cb75c30 0x55f26cb81990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.288+0000 7f2362a02740 1 --2- 172.21.15.43:0/1199922818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f26cb75310 0x55f26cb86bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.288+0000 7f23417fa640 0 client.18723 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:47.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.288+0000 7f2362a02740 1 -- 172.21.15.43:0/1199922818 >> 172.21.15.43:0/1199922818 conn(0x55f26cb6fa40 msgr2=0x55f26cb6eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:47.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.288+0000 7f2362a02740 1 -- 172.21.15.43:0/1199922818 shutdown_connections 2024-04-21T08:45:47.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.288+0000 7f2362a02740 1 -- 172.21.15.43:0/1199922818 wait complete. 2024-04-21T08:45:47.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.288+0000 7f2362a02740 20 client.18723 trim_cache size 0 max 16384 2024-04-21T08:45:47.305 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:45:47.305 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 28, 2024-04-21T08:45:47.306 INFO:teuthology.orchestra.run.smithi043.stdout: "leader": 24527, 2024-04-21T08:45:47.306 INFO:teuthology.orchestra.run.smithi043.stdout: "set_version": 8, 2024-04-21T08:45:47.306 INFO:teuthology.orchestra.run.smithi043.stdout: "sets": { 2024-04-21T08:45:47.306 INFO:teuthology.orchestra.run.smithi043.stdout: "683318c5": { 2024-04-21T08:45:47.307 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 8, 2024-04-21T08:45:47.307 INFO:teuthology.orchestra.run.smithi043.stdout: "age_ref": 0.0, 2024-04-21T08:45:47.307 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-04-21T08:45:47.307 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RELEASED", 2024-04-21T08:45:47.307 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-04-21T08:45:47.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:45:47.308 INFO:teuthology.orchestra.run.smithi043.stdout: "timeout": 30.0, 2024-04-21T08:45:47.308 INFO:teuthology.orchestra.run.smithi043.stdout: "expiration": 90.0, 2024-04-21T08:45:47.308 INFO:teuthology.orchestra.run.smithi043.stdout: "members": { 2024-04-21T08:45:47.308 INFO:teuthology.orchestra.run.smithi043.stdout: "file:/": { 2024-04-21T08:45:47.308 INFO:teuthology.orchestra.run.smithi043.stdout: "excluded": false, 2024-04-21T08:45:47.308 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-04-21T08:45:47.309 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RELEASED", 2024-04-21T08:45:47.309 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-04-21T08:45:47.309 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:45:47.309 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:45:47.309 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:45:47.309 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:45:47.309 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:45:47.310 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:45:47.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.289+0000 7f235fd32640 1 -- 172.21.15.43:0/1788264306 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f23440a3d20 msgr2=0x7f23440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.289+0000 7f235fd32640 1 --2- 172.21.15.43:0/1788264306 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f23440a3d20 0x7f23440a61e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2350005fd0 tx=0x7f2350009600 comp rx=0 tx=0).stop 2024-04-21T08:45:47.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.289+0000 7f235fd32640 1 -- 172.21.15.43:0/1788264306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2358106170 msgr2=0x7f235819e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.289+0000 7f235fd32640 1 --2- 172.21.15.43:0/1788264306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2358106170 0x7f235819e960 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f2348000c00 tx=0x7f234802c860 comp rx=0 tx=0).stop 2024-04-21T08:45:47.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.289+0000 7f235fd32640 1 -- 172.21.15.43:0/1788264306 shutdown_connections 2024-04-21T08:45:47.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.289+0000 7f235fd32640 1 --2- 172.21.15.43:0/1788264306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23581985d0 0x7f2358198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.289+0000 7f235fd32640 1 --2- 172.21.15.43:0/1788264306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2358106170 0x7f235819e960 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.289+0000 7f235fd32640 1 --2- 172.21.15.43:0/1788264306 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f23440a3d20 0x7f23440a61e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.289+0000 7f235fd32640 1 --2- 172.21.15.43:0/1788264306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23581010a0 0x7f235819e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.289+0000 7f235fd32640 1 -- 172.21.15.43:0/1788264306 >> 172.21.15.43:0/1788264306 conn(0x7f2358077eb0 msgr2=0x7f235806a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:47.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.289+0000 7f235fd32640 1 -- 172.21.15.43:0/1788264306 shutdown_connections 2024-04-21T08:45:47.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.289+0000 7f235fd32640 1 -- 172.21.15.43:0/1788264306 wait complete. 2024-04-21T08:45:47.314 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set_id: 683318c5, seconds elapsed: 31.1 2024-04-21T08:45:47.315 INFO:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-04-21T08:45:47.315 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:45:47.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.419+0000 7ff5bef15640 1 -- 172.21.15.43:0/4286557312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5b8101110 msgr2=0x7ff5b8101570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.419+0000 7ff5bef15640 1 --2- 172.21.15.43:0/4286557312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5b8101110 0x7ff5b8101570 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac009930 tx=0x7ff5ac02bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:47.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.419+0000 7ff5bef15640 1 -- 172.21.15.43:0/4286557312 shutdown_connections 2024-04-21T08:45:47.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.419+0000 7ff5bef15640 1 --2- 172.21.15.43:0/4286557312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5b8101ab0 0x7ff5b810dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.419+0000 7ff5bef15640 1 --2- 172.21.15.43:0/4286557312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5b8101110 0x7ff5b8101570 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.419+0000 7ff5bef15640 1 --2- 172.21.15.43:0/4286557312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5b8107010 0x7ff5b81073f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.419+0000 7ff5bef15640 1 -- 172.21.15.43:0/4286557312 >> 172.21.15.43:0/4286557312 conn(0x7ff5b8077e30 msgr2=0x7ff5b8078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:47.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.419+0000 7ff5bef15640 1 -- 172.21.15.43:0/4286557312 shutdown_connections 2024-04-21T08:45:47.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.419+0000 7ff5bef15640 1 -- 172.21.15.43:0/4286557312 wait complete. 2024-04-21T08:45:47.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.419+0000 7ff5bef15640 1 Processor -- start 2024-04-21T08:45:47.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.419+0000 7ff5bef15640 1 -- start start 2024-04-21T08:45:47.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.420+0000 7ff5bef15640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5b8101110 0x7ff5b8198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.420+0000 7ff5bef15640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5b8101ab0 0x7ff5b8198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.420+0000 7ff5bef15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5b8107010 0x7ff5b819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.420+0000 7ff5bef15640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b81110f0 con 0x7ff5b8101110 2024-04-21T08:45:47.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.420+0000 7ff5bef15640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b8110f70 con 0x7ff5b8107010 2024-04-21T08:45:47.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.420+0000 7ff5bef15640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b8111270 con 0x7ff5b8101ab0 2024-04-21T08:45:47.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.420+0000 7ff5bccb1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5b8101110 0x7ff5b8198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.420+0000 7ff5bccb1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5b8101110 0x7ff5b8198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43764/0 (socket says 172.21.15.43:43764) 2024-04-21T08:45:47.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.420+0000 7ff5bccb1640 1 -- 172.21.15.43:0/1758390293 learned_addr learned my addr 172.21.15.43:0/1758390293 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:47.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.420+0000 7ff5b7fff640 1 --2- 172.21.15.43:0/1758390293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5b8101ab0 0x7ff5b8198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.420+0000 7ff5bd4b2640 1 --2- 172.21.15.43:0/1758390293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5b8107010 0x7ff5b819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.420+0000 7ff5bccb1640 1 -- 172.21.15.43:0/1758390293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5b8101ab0 msgr2=0x7ff5b8198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.420+0000 7ff5bccb1640 1 --2- 172.21.15.43:0/1758390293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5b8101ab0 0x7ff5b8198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.420+0000 7ff5bccb1640 1 -- 172.21.15.43:0/1758390293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5b8107010 msgr2=0x7ff5b819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.420+0000 7ff5bccb1640 1 --2- 172.21.15.43:0/1758390293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5b8107010 0x7ff5b819d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.420+0000 7ff5bccb1640 1 -- 172.21.15.43:0/1758390293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b819d750 con 0x7ff5b8101110 2024-04-21T08:45:47.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.420+0000 7ff5b7fff640 1 --2- 172.21.15.43:0/1758390293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5b8101ab0 0x7ff5b8198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:47.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.420+0000 7ff5bccb1640 1 --2- 172.21.15.43:0/1758390293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5b8101110 0x7ff5b8198760 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800c890 tx=0x7ff5a800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:47.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.420+0000 7ff5b5ffb640 1 -- 172.21.15.43:0/1758390293 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8004220 con 0x7ff5b8101110 2024-04-21T08:45:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.420+0000 7ff5b5ffb640 1 -- 172.21.15.43:0/1758390293 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5a80043c0 con 0x7ff5b8101110 2024-04-21T08:45:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.420+0000 7ff5bef15640 1 -- 172.21.15.43:0/1758390293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b819da40 con 0x7ff5b8101110 2024-04-21T08:45:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.420+0000 7ff5b5ffb640 1 -- 172.21.15.43:0/1758390293 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8004a00 con 0x7ff5b8101110 2024-04-21T08:45:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.421+0000 7ff5bef15640 1 -- 172.21.15.43:0/1758390293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b81a52d0 con 0x7ff5b8101110 2024-04-21T08:45:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.422+0000 7ff5bef15640 1 -- 172.21.15.43:0/1758390293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5b8069230 con 0x7ff5b8101110 2024-04-21T08:45:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.423+0000 7ff5b5ffb640 1 -- 172.21.15.43:0/1758390293 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff5a80079b0 con 0x7ff5b8101110 2024-04-21T08:45:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.423+0000 7ff5b5ffb640 1 --2- 172.21.15.43:0/1758390293 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff5a00a3d20 0x7ff5a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.423+0000 7ff5b7fff640 1 --2- 172.21.15.43:0/1758390293 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff5a00a3d20 0x7ff5a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.423+0000 7ff5b5ffb640 1 -- 172.21.15.43:0/1758390293 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7ff5a80d09d0 con 0x7ff5b8101110 2024-04-21T08:45:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.424+0000 7ff5b7fff640 1 --2- 172.21.15.43:0/1758390293 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff5a00a3d20 0x7ff5a00a61e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac006010 tx=0x7ff5ac009610 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:47.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.428+0000 7ff5b5ffb640 1 -- 172.21.15.43:0/1758390293 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff5a8014030 con 0x7ff5b8101110 2024-04-21T08:45:47.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:47 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2854108529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:47.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:47 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2854108529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:47.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:47 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2854108529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:47.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.653+0000 7ff5bef15640 1 -- 172.21.15.43:0/1758390293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5b8199ab0 con 0x7ff5b8101110 2024-04-21T08:45:47.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.654+0000 7ff5b5ffb640 1 -- 172.21.15.43:0/1758390293 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7ff5a809cca0 con 0x7ff5b8101110 2024-04-21T08:45:47.653 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:45:47.653 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:45:47.654 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:45:47.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.656+0000 7ff5bef15640 1 -- 172.21.15.43:0/1758390293 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff5a00a3d20 msgr2=0x7ff5a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.656+0000 7ff5bef15640 1 --2- 172.21.15.43:0/1758390293 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff5a00a3d20 0x7ff5a00a61e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac006010 tx=0x7ff5ac009610 comp rx=0 tx=0).stop 2024-04-21T08:45:47.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.656+0000 7ff5bef15640 1 -- 172.21.15.43:0/1758390293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5b8101110 msgr2=0x7ff5b8198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.656+0000 7ff5bef15640 1 --2- 172.21.15.43:0/1758390293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5b8101110 0x7ff5b8198760 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800c890 tx=0x7ff5a800cd50 comp rx=0 tx=0).stop 2024-04-21T08:45:47.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.656+0000 7ff5bef15640 1 -- 172.21.15.43:0/1758390293 shutdown_connections 2024-04-21T08:45:47.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.656+0000 7ff5bef15640 1 --2- 172.21.15.43:0/1758390293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5b8107010 0x7ff5b819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.656+0000 7ff5bef15640 1 --2- 172.21.15.43:0/1758390293 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff5a00a3d20 0x7ff5a00a61e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.656+0000 7ff5bef15640 1 --2- 172.21.15.43:0/1758390293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5b8101ab0 0x7ff5b8198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.656+0000 7ff5bef15640 1 --2- 172.21.15.43:0/1758390293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5b8101110 0x7ff5b8198760 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.656+0000 7ff5bef15640 1 -- 172.21.15.43:0/1758390293 >> 172.21.15.43:0/1758390293 conn(0x7ff5b8077e30 msgr2=0x7ff5b806a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:47.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.656+0000 7ff5bef15640 1 -- 172.21.15.43:0/1758390293 shutdown_connections 2024-04-21T08:45:47.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.656+0000 7ff5bef15640 1 -- 172.21.15.43:0/1758390293 wait complete. 2024-04-21T08:45:47.666 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db / --timeout 30 --expiration 90.0 --await' 2024-04-21T08:45:47.666 DEBUG:teuthology.orchestra.run.smithi043:> timeout 45 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24527 quiesce db / --timeout 30 --expiration 90.0 --await 2024-04-21T08:45:47.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.778+0000 7f2bbd1bb640 1 -- 172.21.15.43:0/2235574062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bb8102bf0 msgr2=0x7f2bb8106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.778+0000 7f2bbd1bb640 1 --2- 172.21.15.43:0/2235574062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bb8102bf0 0x7f2bb8106ba0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0009900 tx=0x7f2ba002bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:47.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.778+0000 7f2bbd1bb640 1 -- 172.21.15.43:0/2235574062 shutdown_connections 2024-04-21T08:45:47.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.778+0000 7f2bbd1bb640 1 --2- 172.21.15.43:0/2235574062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bb8107280 0x7f2bb810ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.778+0000 7f2bbd1bb640 1 --2- 172.21.15.43:0/2235574062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bb8102bf0 0x7f2bb8106ba0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.778+0000 7f2bbd1bb640 1 --2- 172.21.15.43:0/2235574062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bb8102240 0x7f2bb8102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.778+0000 7f2bbd1bb640 1 -- 172.21.15.43:0/2235574062 >> 172.21.15.43:0/2235574062 conn(0x7f2bb80790e0 msgr2=0x7f2bb80794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:47.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.778+0000 7f2bbd1bb640 1 -- 172.21.15.43:0/2235574062 shutdown_connections 2024-04-21T08:45:47.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.778+0000 7f2bbd1bb640 1 -- 172.21.15.43:0/2235574062 wait complete. 2024-04-21T08:45:47.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.779+0000 7f2bbd1bb640 1 Processor -- start 2024-04-21T08:45:47.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.779+0000 7f2bbd1bb640 1 -- start start 2024-04-21T08:45:47.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.779+0000 7f2bbd1bb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bb8102240 0x7f2bb81986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.779+0000 7f2bbd1bb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bb8107280 0x7f2bb8198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.779+0000 7f2bbd1bb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bb819cf90 0x7f2bb819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.779+0000 7f2bbd1bb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb8112210 con 0x7f2bb8107280 2024-04-21T08:45:47.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.779+0000 7f2bbd1bb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb8112090 con 0x7f2bb8102240 2024-04-21T08:45:47.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.779+0000 7f2bbd1bb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb8112390 con 0x7f2bb819cf90 2024-04-21T08:45:47.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.780+0000 7f2baffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bb8107280 0x7f2bb8198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.780+0000 7f2baffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bb8107280 0x7f2bb8198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43776/0 (socket says 172.21.15.43:43776) 2024-04-21T08:45:47.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.780+0000 7f2baffff640 1 -- 172.21.15.43:0/3963048128 learned_addr learned my addr 172.21.15.43:0/3963048128 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:47.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.780+0000 7f2bb759e640 1 --2- 172.21.15.43:0/3963048128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bb819cf90 0x7f2bb819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.780+0000 7f2bb6d9d640 1 --2- 172.21.15.43:0/3963048128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bb8102240 0x7f2bb81986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.780+0000 7f2baffff640 1 -- 172.21.15.43:0/3963048128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bb819cf90 msgr2=0x7f2bb819d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.780+0000 7f2baffff640 1 --2- 172.21.15.43:0/3963048128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bb819cf90 0x7f2bb819d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.780+0000 7f2baffff640 1 -- 172.21.15.43:0/3963048128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bb8102240 msgr2=0x7f2bb81986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.780+0000 7f2baffff640 1 --2- 172.21.15.43:0/3963048128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bb8102240 0x7f2bb81986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.780+0000 7f2baffff640 1 -- 172.21.15.43:0/3963048128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb819da80 con 0x7f2bb8107280 2024-04-21T08:45:47.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.780+0000 7f2bb759e640 1 --2- 172.21.15.43:0/3963048128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bb819cf90 0x7f2bb819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:47.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.780+0000 7f2baffff640 1 --2- 172.21.15.43:0/3963048128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bb8107280 0x7f2bb8198c10 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0005ab0 tx=0x7f2ba002ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:47.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.780+0000 7f2bb4d99640 1 -- 172.21.15.43:0/3963048128 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0043020 con 0x7f2bb8107280 2024-04-21T08:45:47.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.780+0000 7f2bbd1bb640 1 -- 172.21.15.43:0/3963048128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb819dd10 con 0x7f2bb8107280 2024-04-21T08:45:47.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.780+0000 7f2bb4d99640 1 -- 172.21.15.43:0/3963048128 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ba002cc80 con 0x7f2bb8107280 2024-04-21T08:45:47.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.780+0000 7f2bb4d99640 1 -- 172.21.15.43:0/3963048128 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0003cd0 con 0x7f2bb8107280 2024-04-21T08:45:47.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.780+0000 7f2bbd1bb640 1 -- 172.21.15.43:0/3963048128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb81a54d0 con 0x7f2bb8107280 2024-04-21T08:45:47.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.782+0000 7f2bb4d99640 1 -- 172.21.15.43:0/3963048128 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2ba0046020 con 0x7f2bb8107280 2024-04-21T08:45:47.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.783+0000 7f2bbfda8740 1 Processor -- start 2024-04-21T08:45:47.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.783+0000 7f2bbfda8740 1 -- start start 2024-04-21T08:45:47.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.783+0000 7f2bb4d99640 1 --2- 172.21.15.43:0/3963048128 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2ba40a3d20 0x7f2ba40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.783+0000 7f2bb759e640 1 --2- 172.21.15.43:0/3963048128 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2ba40a3d20 0x7f2ba40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.783+0000 7f2bb4d99640 1 -- 172.21.15.43:0/3963048128 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f2ba00f2e20 con 0x7f2bb8107280 2024-04-21T08:45:47.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.783+0000 7f2bbfda8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5eb3af9c0 0x55c5eb3afda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.783+0000 7f2bbfda8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5eb3b02e0 0x55c5eb3b8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.783+0000 7f2bbfda8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b8ec0 0x55c5eb3bb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.783+0000 7f2bbfda8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c5eb3100b0 con 0x55c5eb3b8ec0 2024-04-21T08:45:47.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.783+0000 7f2bbfda8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c5eb2c1680 con 0x55c5eb3b02e0 2024-04-21T08:45:47.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.783+0000 7f2bbfda8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c5eb2cdc60 con 0x55c5eb3af9c0 2024-04-21T08:45:47.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.783+0000 7f2bb6d9d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5eb3af9c0 0x55c5eb3afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.784+0000 7f2bb6d9d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5eb3af9c0 0x55c5eb3afda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.43:39052/0 (socket says 172.21.15.43:39052) 2024-04-21T08:45:47.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.784+0000 7f2bb6d9d640 1 -- 172.21.15.43:0/2444670498 learned_addr learned my addr 172.21.15.43:0/2444670498 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:47.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.784+0000 7f2baffff640 1 --2- 172.21.15.43:0/2444670498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5eb3b02e0 0x55c5eb3b8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.784+0000 7f2bb759e640 1 --2- 172.21.15.43:0/2444670498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b8ec0 0x55c5eb3bb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.784+0000 7f2bb759e640 1 --2- 172.21.15.43:0/3963048128 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2ba40a3d20 0x7f2ba40a61e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0008ca0 tx=0x7f2bb000d710 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:47.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.784+0000 7f2bb759e640 1 -- 172.21.15.43:0/2444670498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5eb3af9c0 msgr2=0x55c5eb3afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:47.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.784+0000 7f2bb759e640 1 --2- 172.21.15.43:0/2444670498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5eb3af9c0 0x55c5eb3afda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.784+0000 7f2bb759e640 1 -- 172.21.15.43:0/2444670498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5eb3b02e0 msgr2=0x55c5eb3b8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:47.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.784+0000 7f2bb759e640 1 --2- 172.21.15.43:0/2444670498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5eb3b02e0 0x55c5eb3b8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.784+0000 7f2bb759e640 1 -- 172.21.15.43:0/2444670498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5eb0f2c90 con 0x55c5eb3b8ec0 2024-04-21T08:45:47.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.784+0000 7f2bb6d9d640 1 --2- 172.21.15.43:0/2444670498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5eb3af9c0 0x55c5eb3afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:45:47.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.784+0000 7f2bb759e640 1 --2- 172.21.15.43:0/2444670498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b8ec0 0x55c5eb3bb2b0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0008fa0 tx=0x7f2bb0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94301003ee1910b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:47.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.784+0000 7f2bad7fa640 1 -- 172.21.15.43:0/2444670498 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb001c860 con 0x55c5eb3b8ec0 2024-04-21T08:45:47.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.784+0000 7f2bad7fa640 1 -- 172.21.15.43:0/2444670498 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb000f410 con 0x55c5eb3b8ec0 2024-04-21T08:45:47.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.784+0000 7f2bad7fa640 1 -- 172.21.15.43:0/2444670498 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb000e610 con 0x55c5eb3b8ec0 2024-04-21T08:45:47.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.785+0000 7f2bbfda8740 1 -- 172.21.15.43:0/2444670498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b8ec0 msgr2=0x55c5eb3bb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.785+0000 7f2bbfda8740 1 --2- 172.21.15.43:0/2444670498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b8ec0 0x55c5eb3bb2b0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0008fa0 tx=0x7f2bb0018000 comp rx=0 tx=0).stop 2024-04-21T08:45:47.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.785+0000 7f2bbfda8740 1 -- 172.21.15.43:0/2444670498 shutdown_connections 2024-04-21T08:45:47.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.785+0000 7f2bbfda8740 1 --2- 172.21.15.43:0/2444670498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b8ec0 0x55c5eb3bb2b0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.785+0000 7f2bbfda8740 1 --2- 172.21.15.43:0/2444670498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5eb3b02e0 0x55c5eb3b8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.785+0000 7f2bbfda8740 1 --2- 172.21.15.43:0/2444670498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5eb3af9c0 0x55c5eb3afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.785+0000 7f2bbfda8740 1 -- 172.21.15.43:0/2444670498 >> 172.21.15.43:0/2444670498 conn(0x55c5eb3aa0f0 msgr2=0x55c5eb3aa4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:47.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.785+0000 7f2bbfda8740 1 -- 172.21.15.43:0/2444670498 shutdown_connections 2024-04-21T08:45:47.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.785+0000 7f2bbfda8740 1 -- 172.21.15.43:0/2444670498 wait complete. 2024-04-21T08:45:47.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.785+0000 7f2bbfda8740 1 -- start start 2024-04-21T08:45:47.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.785+0000 7f2bbfda8740 1 Processor -- start 2024-04-21T08:45:47.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.786+0000 7f2bbfda8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5eb3af9c0 0x55c5eb3b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.786+0000 7f2bbfda8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b02e0 0x55c5eb3b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.786+0000 7f2bbfda8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5eb3b8ec0 0x55c5eb3b6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.786+0000 7f2bbfda8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c5eb313400 con 0x55c5eb3b02e0 2024-04-21T08:45:47.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.786+0000 7f2bbfda8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c5eb2cf840 con 0x55c5eb3b8ec0 2024-04-21T08:45:47.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.786+0000 7f2bbfda8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c5eb318720 con 0x55c5eb3af9c0 2024-04-21T08:45:47.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.786+0000 7f2bbfda8740 10 client.0 _ll_register_callbacks cb 0x55c5eb320ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:47.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.786+0000 7f2baffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b02e0 0x55c5eb3b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.786+0000 7f2baffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b02e0 0x55c5eb3b64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43804/0 (socket says 172.21.15.43:43804) 2024-04-21T08:45:47.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.786+0000 7f2baffff640 1 -- 172.21.15.43:0/1312076867 learned_addr learned my addr 172.21.15.43:0/1312076867 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:47.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.786+0000 7f2bb6d9d640 1 --2- 172.21.15.43:0/1312076867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5eb3af9c0 0x55c5eb3b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.786+0000 7f2bb759e640 1 --2- 172.21.15.43:0/1312076867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5eb3b8ec0 0x55c5eb3b6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.786+0000 7f2baffff640 1 -- 172.21.15.43:0/1312076867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5eb3af9c0 msgr2=0x55c5eb3b5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:47.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.786+0000 7f2baffff640 1 --2- 172.21.15.43:0/1312076867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5eb3af9c0 0x55c5eb3b5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.786+0000 7f2baffff640 1 -- 172.21.15.43:0/1312076867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5eb3b8ec0 msgr2=0x55c5eb3b6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:47.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.786+0000 7f2baffff640 1 --2- 172.21.15.43:0/1312076867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5eb3b8ec0 0x55c5eb3b6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.786+0000 7f2baffff640 1 -- 172.21.15.43:0/1312076867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5eb153420 con 0x55c5eb3b02e0 2024-04-21T08:45:47.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.786+0000 7f2bb759e640 1 --2- 172.21.15.43:0/1312076867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5eb3b8ec0 0x55c5eb3b6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:47.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.786+0000 7f2baffff640 1 --2- 172.21.15.43:0/1312076867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b02e0 0x55c5eb3b64f0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0006010 tx=0x7f2ba00be770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1279c4b1a87766 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:47.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.786+0000 7f2b9e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:47.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.786+0000 7f2b9e7fc640 1 -- 172.21.15.43:0/1312076867 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba00d8020 con 0x55c5eb3b02e0 2024-04-21T08:45:47.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.786+0000 7f2b9e7fc640 1 -- 172.21.15.43:0/1312076867 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f2ba00bce00 con 0x55c5eb3b02e0 2024-04-21T08:45:47.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.786+0000 7f2bbfda8740 1 -- 172.21.15.43:0/1312076867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bb000f410 con 0x55c5eb3b02e0 2024-04-21T08:45:47.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.787+0000 7f2b9e7fc640 1 -- 172.21.15.43:0/1312076867 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2ba00c2eb0 con 0x55c5eb3b02e0 2024-04-21T08:45:47.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.787+0000 7f2bbfda8740 10 client.18741 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:47.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.787+0000 7f2bbfda8740 1 -- 172.21.15.43:0/1312076867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c5eafe6580 con 0x55c5eb3b02e0 2024-04-21T08:45:47.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.787+0000 7f2b9e7fc640 1 -- 172.21.15.43:0/1312076867 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb000f410 con 0x55c5eb3b02e0 2024-04-21T08:45:47.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.787+0000 7f2b9e7fc640 1 -- 172.21.15.43:0/1312076867 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2ba00c72e0 con 0x55c5eb3b02e0 2024-04-21T08:45:47.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.787+0000 7f2bbfda8740 10 client.18741 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:47.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.787+0000 7f2bbfda8740 10 client.18741 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-21T08:45:47.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.787+0000 7f2bbfda8740 20 client.18741 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:47.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.787+0000 7f2bbfda8740 1 --2- 172.21.15.43:0/1312076867 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c5eb3c8cb0 0x55c5eb3cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.787+0000 7f2bbfda8740 4 client.18741 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:47.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.787+0000 7f2bbfda8740 1 -- 172.21.15.43:0/1312076867 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5eb321b10 con 0x55c5eb3c8cb0 2024-04-21T08:45:47.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.787+0000 7f2bb6d9d640 1 --2- 172.21.15.43:0/1312076867 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c5eb3c8cb0 0x55c5eb3cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.788+0000 7f2bb6d9d640 1 --2- 172.21.15.43:0/1312076867 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c5eb3c8cb0 0x55c5eb3cb0a0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=519263e0fd6e5c4f server_cookie=e5680f708ffec578 in_seq=0 out_seq=0 2024-04-21T08:45:47.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.788+0000 7f2b9e7fc640 10 client.18741 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.788+0000 7f2b9e7fc640 1 -- 172.21.15.43:0/1312076867 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c5eb321b10 con 0x55c5eb3c8cb0 2024-04-21T08:45:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.788+0000 7f2b9e7fc640 10 client.18741 handle_command_reply: tid=0 2024-04-21T08:45:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.788+0000 7f2bbfda8740 1 client.18741 shutdown 2024-04-21T08:45:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.789+0000 7f2bbfda8740 1 -- 172.21.15.43:0/1312076867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b02e0 msgr2=0x55c5eb3b64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.789+0000 7f2bbfda8740 1 --2- 172.21.15.43:0/1312076867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b02e0 0x55c5eb3b64f0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0006010 tx=0x7f2ba00be770 comp rx=0 tx=0).stop 2024-04-21T08:45:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.789+0000 7f2bbfda8740 1 -- 172.21.15.43:0/1312076867 shutdown_connections 2024-04-21T08:45:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.789+0000 7f2bbfda8740 1 --2- 172.21.15.43:0/1312076867 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c5eb3c8cb0 0x55c5eb3cb0a0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.789+0000 7f2bbfda8740 1 --2- 172.21.15.43:0/1312076867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5eb3b8ec0 0x55c5eb3b6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.789+0000 7f2bbfda8740 1 --2- 172.21.15.43:0/1312076867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b02e0 0x55c5eb3b64f0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.789+0000 7f2bbfda8740 1 --2- 172.21.15.43:0/1312076867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5eb3af9c0 0x55c5eb3b5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.789+0000 7f2b9e7fc640 0 client.18741 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:47.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.789+0000 7f2bbfda8740 1 -- 172.21.15.43:0/1312076867 >> 172.21.15.43:0/1312076867 conn(0x55c5eb3aa0f0 msgr2=0x55c5eb3abe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:47.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.789+0000 7f2bbfda8740 1 -- 172.21.15.43:0/1312076867 shutdown_connections 2024-04-21T08:45:47.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.789+0000 7f2bbfda8740 1 -- 172.21.15.43:0/1312076867 wait complete. 2024-04-21T08:45:47.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.789+0000 7f2bbfda8740 20 client.18741 trim_cache size 0 max 16384 2024-04-21T08:45:47.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.806+0000 7f2bbfda8740 1 Processor -- start 2024-04-21T08:45:47.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.806+0000 7f2bbfda8740 1 -- start start 2024-04-21T08:45:47.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.806+0000 7f2bbfda8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5eb3af9c0 0x55c5eb3c4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.806+0000 7f2bbfda8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5eb3b02e0 0x55c5eb3b6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.806+0000 7f2bbfda8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b8ec0 0x55c5eb3d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.806+0000 7f2bbfda8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb81b0e50 con 0x55c5eb3b8ec0 2024-04-21T08:45:47.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.806+0000 7f2bbfda8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb81af9e0 con 0x55c5eb3af9c0 2024-04-21T08:45:47.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.806+0000 7f2bbfda8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb81b00b0 con 0x55c5eb3b02e0 2024-04-21T08:45:47.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.806+0000 7f2bb759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b8ec0 0x55c5eb3d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.806+0000 7f2bb759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b8ec0 0x55c5eb3d0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43814/0 (socket says 172.21.15.43:43814) 2024-04-21T08:45:47.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.806+0000 7f2bb759e640 1 -- 172.21.15.43:0/389712878 learned_addr learned my addr 172.21.15.43:0/389712878 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:47.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.806+0000 7f2bb6d9d640 1 --2- 172.21.15.43:0/389712878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5eb3af9c0 0x55c5eb3c4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.807+0000 7f2baffff640 1 --2- 172.21.15.43:0/389712878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5eb3b02e0 0x55c5eb3b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.807+0000 7f2bb759e640 1 -- 172.21.15.43:0/389712878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5eb3b02e0 msgr2=0x55c5eb3b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:47.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.807+0000 7f2bb759e640 1 --2- 172.21.15.43:0/389712878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5eb3b02e0 0x55c5eb3b6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.807+0000 7f2bb759e640 1 -- 172.21.15.43:0/389712878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5eb3af9c0 msgr2=0x55c5eb3c4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:47.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.807+0000 7f2bb759e640 1 --2- 172.21.15.43:0/389712878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5eb3af9c0 0x55c5eb3c4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.807+0000 7f2bb759e640 1 -- 172.21.15.43:0/389712878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5eb3b4fc0 con 0x55c5eb3b8ec0 2024-04-21T08:45:47.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.807+0000 7f2bb6d9d640 1 --2- 172.21.15.43:0/389712878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5eb3af9c0 0x55c5eb3c4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:47.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.807+0000 7f2bb759e640 1 --2- 172.21.15.43:0/389712878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b8ec0 0x55c5eb3d0920 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0008fa0 tx=0x7f2bb000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7751b126cb45ece9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:47.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.807+0000 7f2bacff9640 1 -- 172.21.15.43:0/389712878 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0032070 con 0x55c5eb3b8ec0 2024-04-21T08:45:47.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.807+0000 7f2bacff9640 1 -- 172.21.15.43:0/389712878 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb001cd10 con 0x55c5eb3b8ec0 2024-04-21T08:45:47.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.807+0000 7f2bacff9640 1 -- 172.21.15.43:0/389712878 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb000fd60 con 0x55c5eb3b8ec0 2024-04-21T08:45:47.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.808+0000 7f2bbfda8740 1 -- 172.21.15.43:0/389712878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b8ec0 msgr2=0x55c5eb3d0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.808+0000 7f2bbfda8740 1 --2- 172.21.15.43:0/389712878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b8ec0 0x55c5eb3d0920 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0008fa0 tx=0x7f2bb000df80 comp rx=0 tx=0).stop 2024-04-21T08:45:47.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.808+0000 7f2bbfda8740 1 -- 172.21.15.43:0/389712878 shutdown_connections 2024-04-21T08:45:47.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.808+0000 7f2bbfda8740 1 --2- 172.21.15.43:0/389712878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b8ec0 0x55c5eb3d0920 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.808+0000 7f2bbfda8740 1 --2- 172.21.15.43:0/389712878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5eb3b02e0 0x55c5eb3b6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.808+0000 7f2bbfda8740 1 --2- 172.21.15.43:0/389712878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5eb3af9c0 0x55c5eb3c4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.808+0000 7f2bbfda8740 1 -- 172.21.15.43:0/389712878 >> 172.21.15.43:0/389712878 conn(0x55c5eb3aa0f0 msgr2=0x55c5eb3b5f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.808+0000 7f2bbfda8740 1 -- 172.21.15.43:0/389712878 shutdown_connections 2024-04-21T08:45:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.808+0000 7f2bbfda8740 1 -- 172.21.15.43:0/389712878 wait complete. 2024-04-21T08:45:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.808+0000 7f2bbfda8740 1 -- start start 2024-04-21T08:45:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.808+0000 7f2bbfda8740 1 Processor -- start 2024-04-21T08:45:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.808+0000 7f2bbfda8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5eb3af9c0 0x55c5eb3c11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.808+0000 7f2bbfda8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5eb3b02e0 0x55c5eb3bdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.808+0000 7f2bbfda8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b8ec0 0x55c5eb3be290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.808+0000 7f2bbfda8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb81ad2f0 con 0x55c5eb3b8ec0 2024-04-21T08:45:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.808+0000 7f2bbfda8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb81b0780 con 0x55c5eb3af9c0 2024-04-21T08:45:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.808+0000 7f2bbfda8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb81ad960 con 0x55c5eb3b02e0 2024-04-21T08:45:47.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.808+0000 7f2bbfda8740 10 client.0 _ll_register_callbacks cb 0x7f2ba00be640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:47.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.808+0000 7f2bb759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b8ec0 0x55c5eb3be290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.808+0000 7f2bb759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b8ec0 0x55c5eb3be290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43822/0 (socket says 172.21.15.43:43822) 2024-04-21T08:45:47.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.808+0000 7f2bb6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5eb3af9c0 0x55c5eb3c11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.808+0000 7f2bb759e640 1 -- 172.21.15.43:0/2504543976 learned_addr learned my addr 172.21.15.43:0/2504543976 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:47.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.808+0000 7f2baffff640 1 --2- 172.21.15.43:0/2504543976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5eb3b02e0 0x55c5eb3bdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.809+0000 7f2bb759e640 1 -- 172.21.15.43:0/2504543976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5eb3b02e0 msgr2=0x55c5eb3bdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:47.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.809+0000 7f2bb759e640 1 --2- 172.21.15.43:0/2504543976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5eb3b02e0 0x55c5eb3bdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.809+0000 7f2bb759e640 1 -- 172.21.15.43:0/2504543976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5eb3af9c0 msgr2=0x55c5eb3c11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:47.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.809+0000 7f2bb759e640 1 --2- 172.21.15.43:0/2504543976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5eb3af9c0 0x55c5eb3c11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.809+0000 7f2bb759e640 1 -- 172.21.15.43:0/2504543976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5eb3bea90 con 0x55c5eb3b8ec0 2024-04-21T08:45:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.809+0000 7f2bb759e640 1 --2- 172.21.15.43:0/2504543976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b8ec0 0x55c5eb3be290 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0006010 tx=0x7f2bb000fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c901ec9419fd97 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.809+0000 7f2b9e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.809+0000 7f2b9e7fc640 1 -- 172.21.15.43:0/2504543976 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0032040 con 0x55c5eb3b8ec0 2024-04-21T08:45:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.809+0000 7f2b9e7fc640 1 -- 172.21.15.43:0/2504543976 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f2bb001c860 con 0x55c5eb3b8ec0 2024-04-21T08:45:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.809+0000 7f2bbfda8740 1 -- 172.21.15.43:0/2504543976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bb001cd10 con 0x55c5eb3b8ec0 2024-04-21T08:45:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.809+0000 7f2b9e7fc640 1 -- 172.21.15.43:0/2504543976 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2bb001cae0 con 0x55c5eb3b8ec0 2024-04-21T08:45:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.809+0000 7f2bbfda8740 10 client.18747 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.809+0000 7f2bbfda8740 1 -- 172.21.15.43:0/2504543976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c5eb4300c0 con 0x55c5eb3b8ec0 2024-04-21T08:45:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.809+0000 7f2b9e7fc640 1 -- 172.21.15.43:0/2504543976 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb001cd10 con 0x55c5eb3b8ec0 2024-04-21T08:45:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.810+0000 7f2b9e7fc640 1 -- 172.21.15.43:0/2504543976 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2bb0038450 con 0x55c5eb3b8ec0 2024-04-21T08:45:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.810+0000 7f2bbfda8740 10 client.18747 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.810+0000 7f2bbfda8740 10 client.18747 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-21T08:45:47.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.810+0000 7f2bbfda8740 20 client.18747 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:47.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.810+0000 7f2bbfda8740 1 --2- 172.21.15.43:0/2504543976 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c5eb430650 0x55c5eb432a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.810+0000 7f2bbfda8740 4 client.18747 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 30.0, "expiration": 90.0, "await": true}] 2024-04-21T08:45:47.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.810+0000 7f2bbfda8740 1 -- 172.21.15.43:0/2504543976 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 30.0, "expiration": 90.0, "await": true}) -- 0x55c5eb31b980 con 0x55c5eb430650 2024-04-21T08:45:47.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.810+0000 7f2bb6d9d640 1 --2- 172.21.15.43:0/2504543976 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c5eb430650 0x55c5eb432a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.810+0000 7f2bb6d9d640 1 --2- 172.21.15.43:0/2504543976 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c5eb430650 0x55c5eb432a40 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e09bbb741df4440f server_cookie=69b396d9f41cddaf in_seq=0 out_seq=0 2024-04-21T08:45:47.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.810+0000 7f2b9e7fc640 10 client.18747 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:47.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.823+0000 7f2b9e7fc640 1 -- 172.21.15.43:0/2504543976 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55c5eb31b980 con 0x55c5eb430650 2024-04-21T08:45:47.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.823+0000 7f2b9e7fc640 10 client.18747 handle_command_reply: tid=0 2024-04-21T08:45:47.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.823+0000 7f2bbfda8740 1 client.18747 shutdown 2024-04-21T08:45:47.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.823+0000 7f2bbfda8740 1 -- 172.21.15.43:0/2504543976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b8ec0 msgr2=0x55c5eb3be290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.823+0000 7f2bbfda8740 1 --2- 172.21.15.43:0/2504543976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b8ec0 0x55c5eb3be290 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0006010 tx=0x7f2bb000fc90 comp rx=0 tx=0).stop 2024-04-21T08:45:47.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.823+0000 7f2bbfda8740 1 -- 172.21.15.43:0/2504543976 shutdown_connections 2024-04-21T08:45:47.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.823+0000 7f2bbfda8740 1 --2- 172.21.15.43:0/2504543976 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c5eb430650 0x55c5eb432a40 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.823+0000 7f2bbfda8740 1 --2- 172.21.15.43:0/2504543976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5eb3b8ec0 0x55c5eb3be290 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.823+0000 7f2bbfda8740 1 --2- 172.21.15.43:0/2504543976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5eb3b02e0 0x55c5eb3bdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.823+0000 7f2b9e7fc640 0 client.18747 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:47.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.823+0000 7f2bbfda8740 1 --2- 172.21.15.43:0/2504543976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5eb3af9c0 0x55c5eb3c11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.823+0000 7f2bbfda8740 1 -- 172.21.15.43:0/2504543976 >> 172.21.15.43:0/2504543976 conn(0x55c5eb3aa0f0 msgr2=0x55c5eb3b6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:47.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.823+0000 7f2bbfda8740 1 -- 172.21.15.43:0/2504543976 shutdown_connections 2024-04-21T08:45:47.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.823+0000 7f2bbfda8740 1 -- 172.21.15.43:0/2504543976 wait complete. 2024-04-21T08:45:47.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.823+0000 7f2bbfda8740 20 client.18747 trim_cache size 0 max 16384 2024-04-21T08:45:47.824 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:45:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 28, 2024-04-21T08:45:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: "leader": 24527, 2024-04-21T08:45:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: "set_version": 10, 2024-04-21T08:45:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: "sets": { 2024-04-21T08:45:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: "c9e85759": { 2024-04-21T08:45:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 10, 2024-04-21T08:45:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: "age_ref": 0.0, 2024-04-21T08:45:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-04-21T08:45:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "QUIESCED", 2024-04-21T08:45:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-04-21T08:45:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:45:47.825 INFO:teuthology.orchestra.run.smithi043.stdout: "timeout": 30.0, 2024-04-21T08:45:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: "expiration": 90.0, 2024-04-21T08:45:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: "members": { 2024-04-21T08:45:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: "file:/": { 2024-04-21T08:45:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: "excluded": false, 2024-04-21T08:45:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-04-21T08:45:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "QUIESCED", 2024-04-21T08:45:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-04-21T08:45:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:45:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:45:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:45:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:45:47.826 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:45:47.827 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:45:47.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.823+0000 7f2bbd1bb640 1 -- 172.21.15.43:0/3963048128 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2ba40a3d20 msgr2=0x7f2ba40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.824+0000 7f2bbd1bb640 1 --2- 172.21.15.43:0/3963048128 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2ba40a3d20 0x7f2ba40a61e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0008ca0 tx=0x7f2bb000d710 comp rx=0 tx=0).stop 2024-04-21T08:45:47.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.824+0000 7f2bbd1bb640 1 -- 172.21.15.43:0/3963048128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bb8107280 msgr2=0x7f2bb8198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.824+0000 7f2bbd1bb640 1 --2- 172.21.15.43:0/3963048128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bb8107280 0x7f2bb8198c10 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0005ab0 tx=0x7f2ba002ca90 comp rx=0 tx=0).stop 2024-04-21T08:45:47.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.824+0000 7f2bbd1bb640 1 -- 172.21.15.43:0/3963048128 shutdown_connections 2024-04-21T08:45:47.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.824+0000 7f2bbd1bb640 1 --2- 172.21.15.43:0/3963048128 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2ba40a3d20 0x7f2ba40a61e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.824+0000 7f2bbd1bb640 1 --2- 172.21.15.43:0/3963048128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bb819cf90 0x7f2bb819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.824+0000 7f2bbd1bb640 1 --2- 172.21.15.43:0/3963048128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bb8107280 0x7f2bb8198c10 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.824+0000 7f2bbd1bb640 1 --2- 172.21.15.43:0/3963048128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bb8102240 0x7f2bb81986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.824+0000 7f2bbd1bb640 1 -- 172.21.15.43:0/3963048128 >> 172.21.15.43:0/3963048128 conn(0x7f2bb80790e0 msgr2=0x7f2bb8100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:47.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.824+0000 7f2bbd1bb640 1 -- 172.21.15.43:0/3963048128 shutdown_connections 2024-04-21T08:45:47.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:45:47.824+0000 7f2bbd1bb640 1 -- 172.21.15.43:0/3963048128 wait complete. 2024-04-21T08:45:47.834 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced, set_id: c9e85759, quiesce duration: 30.0 2024-04-21T08:45:48.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:48 smithi175 ceph-mon[31920]: pgmap v209: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.0 KiB/s wr, 16 op/s 2024-04-21T08:45:48.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:48 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1758390293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:48.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:48 smithi112 ceph-mon[31085]: pgmap v209: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.0 KiB/s wr, 16 op/s 2024-04-21T08:45:48.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:48 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1758390293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:48.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:48 smithi043 ceph-mon[27978]: pgmap v209: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.0 KiB/s wr, 16 op/s 2024-04-21T08:45:48.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:48 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1758390293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.576+0000 7f3823f8f640 1 -- 172.21.15.43:0/3751538808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f381c06ad50 msgr2=0x7f381c06b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.576+0000 7f3823f8f640 1 --2- 172.21.15.43:0/3751538808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f381c06ad50 0x7f381c06b1d0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f3810009900 tx=0x7f381002bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.576+0000 7f3823f8f640 1 -- 172.21.15.43:0/3751538808 shutdown_connections 2024-04-21T08:45:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.576+0000 7f3823f8f640 1 --2- 172.21.15.43:0/3751538808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f381c10e940 0x7f381c110d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.577+0000 7f3823f8f640 1 --2- 172.21.15.43:0/3751538808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f381c06ad50 0x7f381c06b1d0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.577+0000 7f3823f8f640 1 --2- 172.21.15.43:0/3751538808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f381c102fa0 0x7f381c06a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.577+0000 7f3823f8f640 1 -- 172.21.15.43:0/3751538808 >> 172.21.15.43:0/3751538808 conn(0x7f381c076c50 msgr2=0x7f381c077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.577+0000 7f3823f8f640 1 -- 172.21.15.43:0/3751538808 shutdown_connections 2024-04-21T08:45:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.577+0000 7f3823f8f640 1 -- 172.21.15.43:0/3751538808 wait complete. 2024-04-21T08:45:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.577+0000 7f3823f8f640 1 Processor -- start 2024-04-21T08:45:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.577+0000 7f3823f8f640 1 -- start start 2024-04-21T08:45:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.577+0000 7f3823f8f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f381c06ad50 0x7f381c1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.577+0000 7f3823f8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f381c102fa0 0x7f381c1a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.577+0000 7f3823f8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f381c10e940 0x7f381c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.577+0000 7f3823f8f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f381c1143e0 con 0x7f381c102fa0 2024-04-21T08:45:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.577+0000 7f3823f8f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f381c114260 con 0x7f381c10e940 2024-04-21T08:45:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.578+0000 7f3823f8f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f381c114560 con 0x7f381c06ad50 2024-04-21T08:45:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.578+0000 7f382152a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f381c102fa0 0x7f381c1a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.578+0000 7f382152a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f381c102fa0 0x7f381c1a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43828/0 (socket says 172.21.15.43:43828) 2024-04-21T08:45:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.578+0000 7f382152a640 1 -- 172.21.15.43:0/3080248232 learned_addr learned my addr 172.21.15.43:0/3080248232 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.578+0000 7f3821d2b640 1 --2- 172.21.15.43:0/3080248232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f381c06ad50 0x7f381c1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.578+0000 7f382252c640 1 --2- 172.21.15.43:0/3080248232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f381c10e940 0x7f381c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.578+0000 7f382152a640 1 -- 172.21.15.43:0/3080248232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f381c06ad50 msgr2=0x7f381c1a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.578+0000 7f382152a640 1 --2- 172.21.15.43:0/3080248232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f381c06ad50 0x7f381c1a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.578+0000 7f382152a640 1 -- 172.21.15.43:0/3080248232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f381c10e940 msgr2=0x7f381c19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.578+0000 7f382152a640 1 --2- 172.21.15.43:0/3080248232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f381c10e940 0x7f381c19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.578+0000 7f382152a640 1 -- 172.21.15.43:0/3080248232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f381c19af60 con 0x7f381c102fa0 2024-04-21T08:45:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.578+0000 7f382152a640 1 --2- 172.21.15.43:0/3080248232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f381c102fa0 0x7f381c1a0b20 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f3810036040 tx=0x7f381002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.578+0000 7f381affd640 1 -- 172.21.15.43:0/3080248232 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810043020 con 0x7f381c102fa0 2024-04-21T08:45:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.578+0000 7f381affd640 1 -- 172.21.15.43:0/3080248232 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f381002cbe0 con 0x7f381c102fa0 2024-04-21T08:45:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.578+0000 7f3823f8f640 1 -- 172.21.15.43:0/3080248232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f381c19b1f0 con 0x7f381c102fa0 2024-04-21T08:45:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.578+0000 7f381affd640 1 -- 172.21.15.43:0/3080248232 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810003bf0 con 0x7f381c102fa0 2024-04-21T08:45:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.579+0000 7f3823f8f640 1 -- 172.21.15.43:0/3080248232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f381c1a7560 con 0x7f381c102fa0 2024-04-21T08:45:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.581+0000 7f381affd640 1 -- 172.21.15.43:0/3080248232 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3810046020 con 0x7f381c102fa0 2024-04-21T08:45:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.582+0000 7f3826b80740 1 Processor -- start 2024-04-21T08:45:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.582+0000 7f3826b80740 1 -- start start 2024-04-21T08:45:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.582+0000 7f3826b80740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5b1a298d0 0x55a5b1a28620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.582+0000 7f3826b80740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5b1a28b60 0x55a5b1a28fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.582+0000 7f3826b80740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a320e0 0x55a5b1a344d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.582+0000 7f3826b80740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a5b198d8d0 con 0x55a5b1a320e0 2024-04-21T08:45:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.582+0000 7f3826b80740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a5b192ae80 con 0x55a5b1a28b60 2024-04-21T08:45:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.582+0000 7f3826b80740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a5b1989890 con 0x55a5b1a298d0 2024-04-21T08:45:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.582+0000 7f382152a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a320e0 0x55a5b1a344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.582+0000 7f382152a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a320e0 0x55a5b1a344d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43836/0 (socket says 172.21.15.43:43836) 2024-04-21T08:45:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.582+0000 7f382152a640 1 -- 172.21.15.43:0/3761937986 learned_addr learned my addr 172.21.15.43:0/3761937986 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.582+0000 7f3821d2b640 1 --2- 172.21.15.43:0/3761937986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5b1a28b60 0x55a5b1a28fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.582+0000 7f381affd640 1 --2- 172.21.15.43:0/3080248232 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f38040a3e40 0x7f38040a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.582+0000 7f382252c640 1 --2- 172.21.15.43:0/3761937986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5b1a298d0 0x55a5b1a28620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.583+0000 7f382252c640 1 --2- 172.21.15.43:0/3080248232 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f38040a3e40 0x7f38040a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.583+0000 7f3821d2b640 1 -- 172.21.15.43:0/3761937986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5b1a298d0 msgr2=0x55a5b1a28620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.583+0000 7f3821d2b640 1 --2- 172.21.15.43:0/3761937986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5b1a298d0 0x55a5b1a28620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.583+0000 7f3821d2b640 1 -- 172.21.15.43:0/3761937986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a320e0 msgr2=0x55a5b1a344d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.583+0000 7f3821d2b640 1 --2- 172.21.15.43:0/3761937986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a320e0 0x55a5b1a344d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.583+0000 7f381affd640 1 -- 172.21.15.43:0/3080248232 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f38100f2c50 con 0x7f381c102fa0 2024-04-21T08:45:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.583+0000 7f3821d2b640 1 -- 172.21.15.43:0/3761937986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5b194c320 con 0x55a5b1a28b60 2024-04-21T08:45:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.583+0000 7f3821d2b640 1 --2- 172.21.15.43:0/3761937986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5b1a28b60 0x55a5b1a28fe0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3814004630 tx=0x7f381400bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef09610f6564c44f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.583+0000 7f382252c640 1 --2- 172.21.15.43:0/3080248232 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f38040a3e40 0x7f38040a6300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f381c06a590 tx=0x7f380c004ab0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.583+0000 7f3803fff640 1 -- 172.21.15.43:0/3761937986 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814003bd0 con 0x55a5b1a28b60 2024-04-21T08:45:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.583+0000 7f3803fff640 1 -- 172.21.15.43:0/3761937986 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3814015070 con 0x55a5b1a28b60 2024-04-21T08:45:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.583+0000 7f3803fff640 1 -- 172.21.15.43:0/3761937986 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814004c80 con 0x55a5b1a28b60 2024-04-21T08:45:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.584+0000 7f3826b80740 1 -- 172.21.15.43:0/3761937986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5b1a28b60 msgr2=0x55a5b1a28fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.584+0000 7f3826b80740 1 --2- 172.21.15.43:0/3761937986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5b1a28b60 0x55a5b1a28fe0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3814004630 tx=0x7f381400bb20 comp rx=0 tx=0).stop 2024-04-21T08:45:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.584+0000 7f3826b80740 1 -- 172.21.15.43:0/3761937986 shutdown_connections 2024-04-21T08:45:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.584+0000 7f3826b80740 1 --2- 172.21.15.43:0/3761937986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a320e0 0x55a5b1a344d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.584+0000 7f3826b80740 1 --2- 172.21.15.43:0/3761937986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5b1a28b60 0x55a5b1a28fe0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.584+0000 7f3826b80740 1 --2- 172.21.15.43:0/3761937986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5b1a298d0 0x55a5b1a28620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.584+0000 7f3826b80740 1 -- 172.21.15.43:0/3761937986 >> 172.21.15.43:0/3761937986 conn(0x55a5b19efc90 msgr2=0x55a5b1a23850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.584+0000 7f3826b80740 1 -- 172.21.15.43:0/3761937986 shutdown_connections 2024-04-21T08:45:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.584+0000 7f3826b80740 1 -- 172.21.15.43:0/3761937986 wait complete. 2024-04-21T08:45:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.584+0000 7f3826b80740 1 -- start start 2024-04-21T08:45:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.585+0000 7f3826b80740 1 Processor -- start 2024-04-21T08:45:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.585+0000 7f3826b80740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5b1a28b60 0x55a5b1a31a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.585+0000 7f3826b80740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5b1a298d0 0x55a5b1a2ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.585+0000 7f3826b80740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a320e0 0x55a5b1a2f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.585+0000 7f3826b80740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a5b1a34e10 con 0x55a5b1a320e0 2024-04-21T08:45:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.585+0000 7f3826b80740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a5b1915eb0 con 0x55a5b1a298d0 2024-04-21T08:45:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.585+0000 7f3826b80740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a5b1a34f90 con 0x55a5b1a28b60 2024-04-21T08:45:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.585+0000 7f3826b80740 10 client.0 _ll_register_callbacks cb 0x55a5b1942b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.585+0000 7f382252c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a320e0 0x55a5b1a2f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.585+0000 7f382252c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a320e0 0x55a5b1a2f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43838/0 (socket says 172.21.15.43:43838) 2024-04-21T08:45:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.585+0000 7f382252c640 1 -- 172.21.15.43:0/4079315372 learned_addr learned my addr 172.21.15.43:0/4079315372 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.585+0000 7f382152a640 1 --2- 172.21.15.43:0/4079315372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5b1a298d0 0x55a5b1a2ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.585+0000 7f3821d2b640 1 --2- 172.21.15.43:0/4079315372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5b1a28b60 0x55a5b1a31a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.586+0000 7f382252c640 1 -- 172.21.15.43:0/4079315372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5b1a28b60 msgr2=0x55a5b1a31a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.586+0000 7f382252c640 1 --2- 172.21.15.43:0/4079315372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5b1a28b60 0x55a5b1a31a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.586+0000 7f382252c640 1 -- 172.21.15.43:0/4079315372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5b1a298d0 msgr2=0x55a5b1a2ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.586+0000 7f382252c640 1 --2- 172.21.15.43:0/4079315372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5b1a298d0 0x55a5b1a2ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.586+0000 7f382252c640 1 -- 172.21.15.43:0/4079315372 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5b16a0210 con 0x55a5b1a320e0 2024-04-21T08:45:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.586+0000 7f382152a640 1 --2- 172.21.15.43:0/4079315372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5b1a298d0 0x55a5b1a2ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.586+0000 7f382252c640 1 --2- 172.21.15.43:0/4079315372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a320e0 0x55a5b1a2f3b0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f380c010a60 tx=0x7f380c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd0e3b6b8edf82 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.586+0000 7f38017fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.586+0000 7f38017fa640 1 -- 172.21.15.43:0/4079315372 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c018070 con 0x55a5b1a320e0 2024-04-21T08:45:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.586+0000 7f3826b80740 1 -- 172.21.15.43:0/4079315372 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3814015070 con 0x55a5b1a320e0 2024-04-21T08:45:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.586+0000 7f38017fa640 1 -- 172.21.15.43:0/4079315372 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f380c01de00 con 0x55a5b1a320e0 2024-04-21T08:45:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.586+0000 7f38017fa640 1 -- 172.21.15.43:0/4079315372 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f380c004b20 con 0x55a5b1a320e0 2024-04-21T08:45:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.586+0000 7f3826b80740 10 client.18759 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.586+0000 7f3826b80740 1 -- 172.21.15.43:0/4079315372 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a5b176c620 con 0x55a5b1a320e0 2024-04-21T08:45:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.586+0000 7f38017fa640 1 -- 172.21.15.43:0/4079315372 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3814015070 con 0x55a5b1a320e0 2024-04-21T08:45:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.587+0000 7f38017fa640 1 -- 172.21.15.43:0/4079315372 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f380c00f040 con 0x55a5b1a320e0 2024-04-21T08:45:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.587+0000 7f3826b80740 10 client.18759 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.587+0000 7f3826b80740 10 client.18759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.587+0000 7f3826b80740 20 client.18759 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.587+0000 7f3826b80740 1 --2- 172.21.15.43:0/4079315372 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a5b1a42150 0x55a5b1a44540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.587+0000 7f3826b80740 4 client.18759 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.587+0000 7f3826b80740 1 -- 172.21.15.43:0/4079315372 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5b19a1880 con 0x55a5b1a42150 2024-04-21T08:45:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.587+0000 7f3821d2b640 1 --2- 172.21.15.43:0/4079315372 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a5b1a42150 0x55a5b1a44540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.588+0000 7f38017fa640 10 client.18759 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.588+0000 7f3821d2b640 1 --2- 172.21.15.43:0/4079315372 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a5b1a42150 0x55a5b1a44540 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=126b6163c5e0a774 server_cookie=9a14e587bb815ae8 in_seq=0 out_seq=0 2024-04-21T08:45:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.588+0000 7f38017fa640 1 -- 172.21.15.43:0/4079315372 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a5b19a1880 con 0x55a5b1a42150 2024-04-21T08:45:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.588+0000 7f38017fa640 10 client.18759 handle_command_reply: tid=0 2024-04-21T08:45:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.588+0000 7f3826b80740 1 client.18759 shutdown 2024-04-21T08:45:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.588+0000 7f3826b80740 1 -- 172.21.15.43:0/4079315372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a320e0 msgr2=0x55a5b1a2f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.589+0000 7f3826b80740 1 --2- 172.21.15.43:0/4079315372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a320e0 0x55a5b1a2f3b0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f380c010a60 tx=0x7f380c004a40 comp rx=0 tx=0).stop 2024-04-21T08:45:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.589+0000 7f3826b80740 1 -- 172.21.15.43:0/4079315372 shutdown_connections 2024-04-21T08:45:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.589+0000 7f3826b80740 1 --2- 172.21.15.43:0/4079315372 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a5b1a42150 0x55a5b1a44540 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.589+0000 7f3826b80740 1 --2- 172.21.15.43:0/4079315372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a320e0 0x55a5b1a2f3b0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.589+0000 7f3826b80740 1 --2- 172.21.15.43:0/4079315372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5b1a298d0 0x55a5b1a2ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.589+0000 7f38017fa640 0 client.18759 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.589+0000 7f3826b80740 1 --2- 172.21.15.43:0/4079315372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5b1a28b60 0x55a5b1a31a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.589+0000 7f3826b80740 1 -- 172.21.15.43:0/4079315372 >> 172.21.15.43:0/4079315372 conn(0x55a5b19efc90 msgr2=0x55a5b1a255a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.589+0000 7f3826b80740 1 -- 172.21.15.43:0/4079315372 shutdown_connections 2024-04-21T08:45:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.589+0000 7f3826b80740 1 -- 172.21.15.43:0/4079315372 wait complete. 2024-04-21T08:45:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.589+0000 7f3826b80740 20 client.18759 trim_cache size 0 max 16384 2024-04-21T08:45:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.604+0000 7f3826b80740 1 Processor -- start 2024-04-21T08:45:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.604+0000 7f3826b80740 1 -- start start 2024-04-21T08:45:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.604+0000 7f3826b80740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a28b60 0x55a5b1a3dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.604+0000 7f3826b80740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5b1a298d0 0x55a5b1a31540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.604+0000 7f3826b80740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5b1a320e0 0x55a5b1a31a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.604+0000 7f3826b80740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f381c1b1560 con 0x55a5b1a28b60 2024-04-21T08:45:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.604+0000 7f3826b80740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f381c1b30a0 con 0x55a5b1a320e0 2024-04-21T08:45:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.604+0000 7f3826b80740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f381c1b1c30 con 0x55a5b1a298d0 2024-04-21T08:45:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.604+0000 7f3821d2b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a28b60 0x55a5b1a3dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.604+0000 7f3821d2b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a28b60 0x55a5b1a3dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43850/0 (socket says 172.21.15.43:43850) 2024-04-21T08:45:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.604+0000 7f3821d2b640 1 -- 172.21.15.43:0/1992753071 learned_addr learned my addr 172.21.15.43:0/1992753071 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.605+0000 7f3821d2b640 1 -- 172.21.15.43:0/1992753071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5b1a298d0 msgr2=0x55a5b1a31540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.605+0000 7f3821d2b640 1 --2- 172.21.15.43:0/1992753071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5b1a298d0 0x55a5b1a31540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.605+0000 7f3821d2b640 1 -- 172.21.15.43:0/1992753071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5b1a320e0 msgr2=0x55a5b1a31a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.605+0000 7f3821d2b640 1 --2- 172.21.15.43:0/1992753071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5b1a320e0 0x55a5b1a31a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.605+0000 7f3821d2b640 1 -- 172.21.15.43:0/1992753071 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5b1a27e30 con 0x55a5b1a28b60 2024-04-21T08:45:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.605+0000 7f3821d2b640 1 --2- 172.21.15.43:0/1992753071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a28b60 0x55a5b1a3dd10 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f3814004630 tx=0x7f3814013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5ec52d284e150a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.605+0000 7f38037fe640 1 -- 172.21.15.43:0/1992753071 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381402d070 con 0x55a5b1a28b60 2024-04-21T08:45:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.605+0000 7f38037fe640 1 -- 172.21.15.43:0/1992753071 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3814015070 con 0x55a5b1a28b60 2024-04-21T08:45:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.605+0000 7f38037fe640 1 -- 172.21.15.43:0/1992753071 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814028840 con 0x55a5b1a28b60 2024-04-21T08:45:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.605+0000 7f3826b80740 1 -- 172.21.15.43:0/1992753071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a28b60 msgr2=0x55a5b1a3dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.605+0000 7f3826b80740 1 --2- 172.21.15.43:0/1992753071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a28b60 0x55a5b1a3dd10 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f3814004630 tx=0x7f3814013040 comp rx=0 tx=0).stop 2024-04-21T08:45:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.605+0000 7f3826b80740 1 -- 172.21.15.43:0/1992753071 shutdown_connections 2024-04-21T08:45:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.605+0000 7f3826b80740 1 --2- 172.21.15.43:0/1992753071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5b1a320e0 0x55a5b1a31a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.605+0000 7f3826b80740 1 --2- 172.21.15.43:0/1992753071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5b1a298d0 0x55a5b1a31540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.606+0000 7f3826b80740 1 --2- 172.21.15.43:0/1992753071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a28b60 0x55a5b1a3dd10 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.606+0000 7f3826b80740 1 -- 172.21.15.43:0/1992753071 >> 172.21.15.43:0/1992753071 conn(0x55a5b19efc90 msgr2=0x55a5b1a2a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.606+0000 7f3826b80740 1 -- 172.21.15.43:0/1992753071 shutdown_connections 2024-04-21T08:45:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.606+0000 7f3826b80740 1 -- 172.21.15.43:0/1992753071 wait complete. 2024-04-21T08:45:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.606+0000 7f3826b80740 1 -- start start 2024-04-21T08:45:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.606+0000 7f3826b80740 1 Processor -- start 2024-04-21T08:45:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.606+0000 7f3826b80740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a28b60 0x55a5b1a3a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.606+0000 7f3826b80740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5b1a298d0 0x55a5b1a2c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.606+0000 7f3826b80740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5b1a320e0 0x55a5b1a2c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.606+0000 7f3826b80740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f381c1b29d0 con 0x55a5b1a28b60 2024-04-21T08:45:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.606+0000 7f3826b80740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f381c1b2300 con 0x55a5b1a298d0 2024-04-21T08:45:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.606+0000 7f3826b80740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f381c1af540 con 0x55a5b1a320e0 2024-04-21T08:45:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.606+0000 7f3826b80740 10 client.0 _ll_register_callbacks cb 0x7f380c0106e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.606+0000 7f3821d2b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a28b60 0x55a5b1a3a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.607+0000 7f3821d2b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a28b60 0x55a5b1a3a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43862/0 (socket says 172.21.15.43:43862) 2024-04-21T08:45:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.607+0000 7f3821d2b640 1 -- 172.21.15.43:0/2150234494 learned_addr learned my addr 172.21.15.43:0/2150234494 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:45:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.607+0000 7f382152a640 1 --2- 172.21.15.43:0/2150234494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5b1a298d0 0x55a5b1a2c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.607+0000 7f3821d2b640 1 -- 172.21.15.43:0/2150234494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5b1a320e0 msgr2=0x55a5b1a2c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.607+0000 7f3821d2b640 1 --2- 172.21.15.43:0/2150234494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5b1a320e0 0x55a5b1a2c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.607+0000 7f3821d2b640 1 -- 172.21.15.43:0/2150234494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5b1a298d0 msgr2=0x55a5b1a2c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.607+0000 7f3821d2b640 1 --2- 172.21.15.43:0/2150234494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5b1a298d0 0x55a5b1a2c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.607+0000 7f3821d2b640 1 -- 172.21.15.43:0/2150234494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5b1a2cc90 con 0x55a5b1a28b60 2024-04-21T08:45:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.607+0000 7f3821d2b640 1 --2- 172.21.15.43:0/2150234494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a28b60 0x55a5b1a3a4e0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f3814003e30 tx=0x7f3814003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c825f97c49a7efa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.607+0000 7f38017fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:45:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.607+0000 7f38017fa640 1 -- 172.21.15.43:0/2150234494 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381402d070 con 0x55a5b1a28b60 2024-04-21T08:45:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.607+0000 7f38017fa640 1 -- 172.21.15.43:0/2150234494 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f3814005040 con 0x55a5b1a28b60 2024-04-21T08:45:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.607+0000 7f3826b80740 1 -- 172.21.15.43:0/2150234494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3814015070 con 0x55a5b1a28b60 2024-04-21T08:45:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.607+0000 7f38017fa640 1 -- 172.21.15.43:0/2150234494 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f381403b050 con 0x55a5b1a28b60 2024-04-21T08:45:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.607+0000 7f3826b80740 10 client.18765 fetch_fsmap learned FSMap version 28 2024-04-21T08:45:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.607+0000 7f3826b80740 1 -- 172.21.15.43:0/2150234494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a5b1aa86a0 con 0x55a5b1a28b60 2024-04-21T08:45:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.608+0000 7f38017fa640 1 -- 172.21.15.43:0/2150234494 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3814015070 con 0x55a5b1a28b60 2024-04-21T08:45:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.608+0000 7f38017fa640 1 -- 172.21.15.43:0/2150234494 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3814004c00 con 0x55a5b1a28b60 2024-04-21T08:45:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.608+0000 7f3826b80740 10 client.18765 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:45:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.608+0000 7f3826b80740 10 client.18765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.608+0000 7f3826b80740 20 client.18765 populate_metadata read hostname 'smithi043' 2024-04-21T08:45:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.608+0000 7f3826b80740 1 --2- 172.21.15.43:0/2150234494 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a5b1aa8b20 0x55a5b1aaaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.608+0000 7f3826b80740 4 client.18765 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:45:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.608+0000 7f3826b80740 1 -- 172.21.15.43:0/2150234494 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a5b192d3d0 con 0x55a5b1aa8b20 2024-04-21T08:45:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.608+0000 7f382152a640 1 --2- 172.21.15.43:0/2150234494 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a5b1aa8b20 0x55a5b1aaaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.608+0000 7f382152a640 1 --2- 172.21.15.43:0/2150234494 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a5b1aa8b20 0x55a5b1aaaf40 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f53fc138846822a server_cookie=fe7dbc2867a4f463 in_seq=0 out_seq=0 2024-04-21T08:45:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.608+0000 7f38017fa640 10 client.18765 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.619+0000 7f38017fa640 1 -- 172.21.15.43:0/2150234494 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a5b192d3d0 con 0x55a5b1aa8b20 2024-04-21T08:45:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.619+0000 7f38017fa640 10 client.18765 handle_command_reply: tid=0 2024-04-21T08:45:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.619+0000 7f3826b80740 1 client.18765 shutdown 2024-04-21T08:45:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.619+0000 7f3826b80740 1 -- 172.21.15.43:0/2150234494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a28b60 msgr2=0x55a5b1a3a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.619+0000 7f3826b80740 1 --2- 172.21.15.43:0/2150234494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a28b60 0x55a5b1a3a4e0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f3814003e30 tx=0x7f3814003e60 comp rx=0 tx=0).stop 2024-04-21T08:45:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.619+0000 7f3826b80740 1 -- 172.21.15.43:0/2150234494 shutdown_connections 2024-04-21T08:45:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.619+0000 7f3826b80740 1 --2- 172.21.15.43:0/2150234494 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a5b1aa8b20 0x55a5b1aaaf40 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.619+0000 7f3826b80740 1 --2- 172.21.15.43:0/2150234494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5b1a320e0 0x55a5b1a2c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.619+0000 7f3826b80740 1 --2- 172.21.15.43:0/2150234494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5b1a298d0 0x55a5b1a2c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.619+0000 7f3826b80740 1 --2- 172.21.15.43:0/2150234494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5b1a28b60 0x55a5b1a3a4e0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.619+0000 7f38017fa640 0 client.18765 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:45:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.619+0000 7f3826b80740 1 -- 172.21.15.43:0/2150234494 >> 172.21.15.43:0/2150234494 conn(0x55a5b19efc90 msgr2=0x55a5b1a2d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.620+0000 7f3826b80740 1 -- 172.21.15.43:0/2150234494 shutdown_connections 2024-04-21T08:45:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.620+0000 7f3826b80740 1 -- 172.21.15.43:0/2150234494 wait complete. 2024-04-21T08:45:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.620+0000 7f3826b80740 20 client.18765 trim_cache size 0 max 16384 2024-04-21T08:45:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:45:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:45:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:45:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:45:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.620+0000 7f3823f8f640 1 -- 172.21.15.43:0/3080248232 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f38040a3e40 msgr2=0x7f38040a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.620+0000 7f3823f8f640 1 --2- 172.21.15.43:0/3080248232 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f38040a3e40 0x7f38040a6300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f381c06a590 tx=0x7f380c004ab0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.620+0000 7f3823f8f640 1 -- 172.21.15.43:0/3080248232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f381c102fa0 msgr2=0x7f381c1a0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.620+0000 7f3823f8f640 1 --2- 172.21.15.43:0/3080248232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f381c102fa0 0x7f381c1a0b20 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f3810036040 tx=0x7f381002c7e0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.620+0000 7f3823f8f640 1 -- 172.21.15.43:0/3080248232 shutdown_connections 2024-04-21T08:45:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.620+0000 7f3823f8f640 1 --2- 172.21.15.43:0/3080248232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f381c10e940 0x7f381c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.620+0000 7f3823f8f640 1 --2- 172.21.15.43:0/3080248232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f381c102fa0 0x7f381c1a0b20 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.620+0000 7f3823f8f640 1 --2- 172.21.15.43:0/3080248232 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f38040a3e40 0x7f38040a6300 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.620+0000 7f3823f8f640 1 --2- 172.21.15.43:0/3080248232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f381c06ad50 0x7f381c1a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.620+0000 7f3823f8f640 1 -- 172.21.15.43:0/3080248232 >> 172.21.15.43:0/3080248232 conn(0x7f381c076c50 msgr2=0x7f381c10f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.620+0000 7f3823f8f640 1 -- 172.21.15.43:0/3080248232 shutdown_connections 2024-04-21T08:45:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:45:49.620+0000 7f3823f8f640 1 -- 172.21.15.43:0/3080248232 wait complete. 2024-04-21T08:45:50.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:50 smithi175 ceph-mon[31920]: pgmap v210: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-04-21T08:45:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:50 smithi112 ceph-mon[31085]: pgmap v210: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-04-21T08:45:50.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:50 smithi043 ceph-mon[27978]: pgmap v210: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-04-21T08:45:52.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:52 smithi175 ceph-mon[31920]: pgmap v211: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-04-21T08:45:52.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:52 smithi112 ceph-mon[31085]: pgmap v211: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-04-21T08:45:52.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:52 smithi043 ceph-mon[27978]: pgmap v211: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-04-21T08:45:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:54 smithi112 ceph-mon[31085]: pgmap v212: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.5 KiB/s wr, 11 op/s 2024-04-21T08:45:54.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:54 smithi043 ceph-mon[27978]: pgmap v212: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.5 KiB/s wr, 11 op/s 2024-04-21T08:45:54.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:54 smithi175 ceph-mon[31920]: pgmap v212: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.5 KiB/s wr, 11 op/s 2024-04-21T08:45:56.516 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:56 smithi175 ceph-mon[31920]: pgmap v213: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.5 KiB/s wr, 11 op/s 2024-04-21T08:45:56.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:56 smithi112 ceph-mon[31085]: pgmap v213: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.5 KiB/s wr, 11 op/s 2024-04-21T08:45:56.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:56 smithi043 ceph-mon[27978]: pgmap v213: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.5 KiB/s wr, 11 op/s 2024-04-21T08:45:58.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:45:58 smithi112 ceph-mon[31085]: pgmap v214: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 KiB/s wr, 15 op/s 2024-04-21T08:45:58.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:45:58 smithi043 ceph-mon[27978]: pgmap v214: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 KiB/s wr, 15 op/s 2024-04-21T08:45:58.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:45:58 smithi175 ceph-mon[31920]: pgmap v214: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 KiB/s wr, 15 op/s 2024-04-21T08:46:00.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:00 smithi112 ceph-mon[31085]: pgmap v215: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.6 KiB/s wr, 11 op/s 2024-04-21T08:46:00.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:00 smithi043 ceph-mon[27978]: pgmap v215: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.6 KiB/s wr, 11 op/s 2024-04-21T08:46:00.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:00 smithi175 ceph-mon[31920]: pgmap v215: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.6 KiB/s wr, 11 op/s 2024-04-21T08:46:02.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:02 smithi112 ceph-mon[31085]: pgmap v216: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 53 KiB/s wr, 32 op/s 2024-04-21T08:46:02.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:02 smithi043 ceph-mon[27978]: pgmap v216: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 53 KiB/s wr, 32 op/s 2024-04-21T08:46:02.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:02 smithi175 ceph-mon[31920]: pgmap v216: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 53 KiB/s wr, 32 op/s 2024-04-21T08:46:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.721+0000 7f7774411640 1 -- 172.21.15.43:0/2674771966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f776c102b70 msgr2=0x7f776c106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.721+0000 7f7774411640 1 --2- 172.21.15.43:0/2674771966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f776c102b70 0x7f776c106bb0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f775c009900 tx=0x7f775c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.721+0000 7f7774411640 1 -- 172.21.15.43:0/2674771966 shutdown_connections 2024-04-21T08:46:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.721+0000 7f7774411640 1 --2- 172.21.15.43:0/2674771966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f776c107290 0x7f776c10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.721+0000 7f7774411640 1 --2- 172.21.15.43:0/2674771966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f776c102b70 0x7f776c106bb0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.721+0000 7f7774411640 1 --2- 172.21.15.43:0/2674771966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f776c1021c0 0x7f776c1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.721+0000 7f7774411640 1 -- 172.21.15.43:0/2674771966 >> 172.21.15.43:0/2674771966 conn(0x7f776c0fda20 msgr2=0x7f776c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.721+0000 7f7774411640 1 -- 172.21.15.43:0/2674771966 shutdown_connections 2024-04-21T08:46:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.721+0000 7f7774411640 1 -- 172.21.15.43:0/2674771966 wait complete. 2024-04-21T08:46:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.722+0000 7f7774411640 1 Processor -- start 2024-04-21T08:46:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.722+0000 7f7774411640 1 -- start start 2024-04-21T08:46:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.722+0000 7f7774411640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f776c1021c0 0x7f776c1a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.722+0000 7f7774411640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f776c107290 0x7f776c1a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.722+0000 7f7774411640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f776c19a6b0 0x7f776c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.722+0000 7f7774411640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1121d0 con 0x7f776c107290 2024-04-21T08:46:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.722+0000 7f7774411640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f776c112050 con 0x7f776c1021c0 2024-04-21T08:46:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.722+0000 7f7774411640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f776c112350 con 0x7f776c19a6b0 2024-04-21T08:46:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.722+0000 7f77719ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f776c107290 0x7f776c1a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.722+0000 7f77719ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f776c107290 0x7f776c1a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51142/0 (socket says 172.21.15.43:51142) 2024-04-21T08:46:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.722+0000 7f77719ac640 1 -- 172.21.15.43:0/1838275893 learned_addr learned my addr 172.21.15.43:0/1838275893 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.722+0000 7f77721ad640 1 --2- 172.21.15.43:0/1838275893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f776c1021c0 0x7f776c1a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.722+0000 7f77729ae640 1 --2- 172.21.15.43:0/1838275893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f776c19a6b0 0x7f776c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.722+0000 7f77719ac640 1 -- 172.21.15.43:0/1838275893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f776c19a6b0 msgr2=0x7f776c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.722+0000 7f77719ac640 1 --2- 172.21.15.43:0/1838275893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f776c19a6b0 0x7f776c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.722+0000 7f77719ac640 1 -- 172.21.15.43:0/1838275893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f776c1021c0 msgr2=0x7f776c1a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.722+0000 7f77719ac640 1 --2- 172.21.15.43:0/1838275893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f776c1021c0 0x7f776c1a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.722+0000 7f77719ac640 1 -- 172.21.15.43:0/1838275893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776c19b2b0 con 0x7f776c107290 2024-04-21T08:46:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.722+0000 7f77729ae640 1 --2- 172.21.15.43:0/1838275893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f776c19a6b0 0x7f776c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.723+0000 7f77719ac640 1 --2- 172.21.15.43:0/1838275893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f776c107290 0x7f776c1a0b30 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f775c036040 tx=0x7f775c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.723+0000 7f77637fe640 1 -- 172.21.15.43:0/1838275893 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c043020 con 0x7f776c107290 2024-04-21T08:46:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.723+0000 7f77637fe640 1 -- 172.21.15.43:0/1838275893 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f775c02cb70 con 0x7f776c107290 2024-04-21T08:46:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.723+0000 7f7774411640 1 -- 172.21.15.43:0/1838275893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776c19b540 con 0x7f776c107290 2024-04-21T08:46:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.723+0000 7f77637fe640 1 -- 172.21.15.43:0/1838275893 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c003bf0 con 0x7f776c107290 2024-04-21T08:46:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.723+0000 7f7774411640 1 -- 172.21.15.43:0/1838275893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f776c1a76e0 con 0x7f776c107290 2024-04-21T08:46:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.725+0000 7f77637fe640 1 -- 172.21.15.43:0/1838275893 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f775c046020 con 0x7f776c107290 2024-04-21T08:46:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.726+0000 7f7777001740 1 Processor -- start 2024-04-21T08:46:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.726+0000 7f7777001740 1 -- start start 2024-04-21T08:46:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.726+0000 7f7777001740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0648508d0 0x55a06484f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.726+0000 7f7777001740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a06484fb60 0x55a06484ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.726+0000 7f77637fe640 1 --2- 172.21.15.43:0/1838275893 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f77580a3dc0 0x7f77580a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.726+0000 7f7777001740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0648590e0 0x55a06485b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.726+0000 7f7777001740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a0647b48d0 con 0x55a06484fb60 2024-04-21T08:46:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.726+0000 7f7777001740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a064751e80 con 0x55a0648590e0 2024-04-21T08:46:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.726+0000 7f7777001740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a0647b0890 con 0x55a0648508d0 2024-04-21T08:46:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.727+0000 7f77721ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a06484fb60 0x55a06484ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.727+0000 7f77729ae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0648590e0 0x55a06485b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.727+0000 7f77719ac640 1 --2- 172.21.15.43:0/1838275893 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f77580a3dc0 0x7f77580a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.727+0000 7f77721ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a06484fb60 0x55a06484ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51158/0 (socket says 172.21.15.43:51158) 2024-04-21T08:46:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.727+0000 7f77721ad640 1 -- 172.21.15.43:0/673220048 learned_addr learned my addr 172.21.15.43:0/673220048 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.727+0000 7f77637fe640 1 -- 172.21.15.43:0/1838275893 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f775c0f2cb0 con 0x7f776c107290 2024-04-21T08:46:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.727+0000 7f77729ae640 1 --2- 172.21.15.43:0/673220048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0648508d0 0x55a06484f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.727+0000 7f77721ad640 1 -- 172.21.15.43:0/673220048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0648508d0 msgr2=0x55a06484f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.727+0000 7f77721ad640 1 --2- 172.21.15.43:0/673220048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0648508d0 0x55a06484f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.727+0000 7f77721ad640 1 -- 172.21.15.43:0/673220048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0648590e0 msgr2=0x55a06485b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.727+0000 7f77721ad640 1 --2- 172.21.15.43:0/673220048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0648590e0 0x55a06485b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.727+0000 7f77721ad640 1 -- 172.21.15.43:0/673220048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a064773320 con 0x55a06484fb60 2024-04-21T08:46:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.727+0000 7f77729ae640 1 --2- 172.21.15.43:0/673220048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0648590e0 0x55a06485b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.727+0000 7f77719ac640 1 --2- 172.21.15.43:0/1838275893 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f77580a3dc0 0x7f77580a6280 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f775c000c00 tx=0x7f775c0df040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.727+0000 7f77721ad640 1 --2- 172.21.15.43:0/673220048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a06484fb60 0x55a06484ffe0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f7768009cf0 tx=0x7f7768011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b5c89ce43717dc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.727+0000 7f7760bfe640 1 -- 172.21.15.43:0/673220048 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7768004230 con 0x55a06484fb60 2024-04-21T08:46:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.727+0000 7f7760bfe640 1 -- 172.21.15.43:0/673220048 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77680043d0 con 0x55a06484fb60 2024-04-21T08:46:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.727+0000 7f7760bfe640 1 -- 172.21.15.43:0/673220048 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77680197c0 con 0x55a06484fb60 2024-04-21T08:46:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.728+0000 7f7777001740 1 -- 172.21.15.43:0/673220048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a06484fb60 msgr2=0x55a06484ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.728+0000 7f7777001740 1 --2- 172.21.15.43:0/673220048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a06484fb60 0x55a06484ffe0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f7768009cf0 tx=0x7f7768011810 comp rx=0 tx=0).stop 2024-04-21T08:46:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.728+0000 7f7777001740 1 -- 172.21.15.43:0/673220048 shutdown_connections 2024-04-21T08:46:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.728+0000 7f7777001740 1 --2- 172.21.15.43:0/673220048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0648590e0 0x55a06485b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.728+0000 7f7777001740 1 --2- 172.21.15.43:0/673220048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a06484fb60 0x55a06484ffe0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.728+0000 7f7777001740 1 --2- 172.21.15.43:0/673220048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0648508d0 0x55a06484f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.728+0000 7f7777001740 1 -- 172.21.15.43:0/673220048 >> 172.21.15.43:0/673220048 conn(0x55a064816c90 msgr2=0x55a06484a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.728+0000 7f7777001740 1 -- 172.21.15.43:0/673220048 shutdown_connections 2024-04-21T08:46:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.728+0000 7f7777001740 1 -- 172.21.15.43:0/673220048 wait complete. 2024-04-21T08:46:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.729+0000 7f7777001740 1 -- start start 2024-04-21T08:46:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.729+0000 7f7777001740 1 Processor -- start 2024-04-21T08:46:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.729+0000 7f7777001740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a06484fb60 0x55a064858a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.729+0000 7f7777001740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0648508d0 0x55a064855de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.729+0000 7f7777001740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0648590e0 0x55a064856340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.729+0000 7f7777001740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a06485be10 con 0x55a0648590e0 2024-04-21T08:46:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.729+0000 7f7777001740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a06473ceb0 con 0x55a06484fb60 2024-04-21T08:46:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.729+0000 7f7777001740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a06485bf90 con 0x55a0648508d0 2024-04-21T08:46:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.729+0000 7f7777001740 10 client.0 _ll_register_callbacks cb 0x55a064769b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.729+0000 7f77729ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0648590e0 0x55a064856340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.729+0000 7f77729ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0648590e0 0x55a064856340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51168/0 (socket says 172.21.15.43:51168) 2024-04-21T08:46:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.729+0000 7f77729ae640 1 -- 172.21.15.43:0/3617879964 learned_addr learned my addr 172.21.15.43:0/3617879964 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.729+0000 7f77721ad640 1 --2- 172.21.15.43:0/3617879964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0648508d0 0x55a064855de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.729+0000 7f77729ae640 1 --2- 172.21.15.43:0/3617879964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a06484fb60 0x55a064858a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.729+0000 7f77729ae640 1 -- 172.21.15.43:0/3617879964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0648508d0 msgr2=0x55a064855de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.729+0000 7f77729ae640 1 --2- 172.21.15.43:0/3617879964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0648508d0 0x55a064855de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.729+0000 7f77729ae640 1 -- 172.21.15.43:0/3617879964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a06484fb60 msgr2=0x55a064858a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.729+0000 7f77729ae640 1 --2- 172.21.15.43:0/3617879964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a06484fb60 0x55a064858a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.729+0000 7f77729ae640 1 -- 172.21.15.43:0/3617879964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0644c7210 con 0x55a0648590e0 2024-04-21T08:46:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.729+0000 7f77721ad640 1 --2- 172.21.15.43:0/3617879964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0648508d0 0x55a064855de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.730+0000 7f77729ae640 1 --2- 172.21.15.43:0/3617879964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0648590e0 0x55a064856340 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f7764014c30 tx=0x7f7764016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1acbe63b95c19e7c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.730+0000 7f7755ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.730+0000 7f7755ffb640 1 -- 172.21.15.43:0/3617879964 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764014110 con 0x55a0648590e0 2024-04-21T08:46:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.730+0000 7f7755ffb640 1 -- 172.21.15.43:0/3617879964 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f776400fcf0 con 0x55a0648590e0 2024-04-21T08:46:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.730+0000 7f7777001740 1 -- 172.21.15.43:0/3617879964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77680043d0 con 0x55a0648590e0 2024-04-21T08:46:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.730+0000 7f7755ffb640 1 -- 172.21.15.43:0/3617879964 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7764029050 con 0x55a0648590e0 2024-04-21T08:46:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.730+0000 7f7777001740 10 client.18777 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.730+0000 7f7777001740 1 -- 172.21.15.43:0/3617879964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a064593620 con 0x55a0648590e0 2024-04-21T08:46:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.730+0000 7f7755ffb640 1 -- 172.21.15.43:0/3617879964 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77680043d0 con 0x55a0648590e0 2024-04-21T08:46:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.730+0000 7f7755ffb640 1 -- 172.21.15.43:0/3617879964 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f77640123f0 con 0x55a0648590e0 2024-04-21T08:46:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.730+0000 7f7777001740 10 client.18777 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.730+0000 7f7777001740 10 client.18777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.730+0000 7f7777001740 20 client.18777 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.730+0000 7f7777001740 1 --2- 172.21.15.43:0/3617879964 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a064869190 0x55a06486b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.730+0000 7f7777001740 4 client.18777 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.730+0000 7f7777001740 1 -- 172.21.15.43:0/3617879964 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0647c8880 con 0x55a064869190 2024-04-21T08:46:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.731+0000 7f77721ad640 1 --2- 172.21.15.43:0/3617879964 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a064869190 0x55a06486b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.731+0000 7f77721ad640 1 --2- 172.21.15.43:0/3617879964 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a064869190 0x55a06486b580 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=574194ce481c0f4b server_cookie=b830e787c4a04f70 in_seq=0 out_seq=0 2024-04-21T08:46:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.731+0000 7f7755ffb640 10 client.18777 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.732+0000 7f7755ffb640 1 -- 172.21.15.43:0/3617879964 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a0647c8880 con 0x55a064869190 2024-04-21T08:46:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.732+0000 7f7755ffb640 10 client.18777 handle_command_reply: tid=0 2024-04-21T08:46:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.732+0000 7f7777001740 1 client.18777 shutdown 2024-04-21T08:46:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.732+0000 7f7777001740 1 -- 172.21.15.43:0/3617879964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0648590e0 msgr2=0x55a064856340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.732+0000 7f7777001740 1 --2- 172.21.15.43:0/3617879964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0648590e0 0x55a064856340 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f7764014c30 tx=0x7f7764016040 comp rx=0 tx=0).stop 2024-04-21T08:46:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.732+0000 7f7777001740 1 -- 172.21.15.43:0/3617879964 shutdown_connections 2024-04-21T08:46:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.732+0000 7f7777001740 1 --2- 172.21.15.43:0/3617879964 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a064869190 0x55a06486b580 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.732+0000 7f7777001740 1 --2- 172.21.15.43:0/3617879964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0648590e0 0x55a064856340 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.732+0000 7f7777001740 1 --2- 172.21.15.43:0/3617879964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0648508d0 0x55a064855de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.732+0000 7f7777001740 1 --2- 172.21.15.43:0/3617879964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a06484fb60 0x55a064858a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.732+0000 7f7755ffb640 0 client.18777 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.732+0000 7f7777001740 1 -- 172.21.15.43:0/3617879964 >> 172.21.15.43:0/3617879964 conn(0x55a064816c90 msgr2=0x55a06484c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.732+0000 7f7777001740 1 -- 172.21.15.43:0/3617879964 shutdown_connections 2024-04-21T08:46:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.732+0000 7f7777001740 1 -- 172.21.15.43:0/3617879964 wait complete. 2024-04-21T08:46:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.732+0000 7f7777001740 20 client.18777 trim_cache size 0 max 16384 2024-04-21T08:46:02.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.747+0000 7f7777001740 1 Processor -- start 2024-04-21T08:46:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.748+0000 7f7777001740 1 -- start start 2024-04-21T08:46:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.748+0000 7f7777001740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a06484fb60 0x55a064864d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.748+0000 7f7777001740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0648508d0 0x55a064858510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.748+0000 7f7777001740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0648590e0 0x55a064858a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.748+0000 7f7777001740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1b1520 con 0x55a0648590e0 2024-04-21T08:46:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.748+0000 7f7777001740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1b3060 con 0x55a06484fb60 2024-04-21T08:46:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.748+0000 7f7777001740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1b1bf0 con 0x55a0648508d0 2024-04-21T08:46:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.748+0000 7f77729ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0648590e0 0x55a064858a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.748+0000 7f77729ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0648590e0 0x55a064858a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51172/0 (socket says 172.21.15.43:51172) 2024-04-21T08:46:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.748+0000 7f77729ae640 1 -- 172.21.15.43:0/3804245945 learned_addr learned my addr 172.21.15.43:0/3804245945 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.748+0000 7f77721ad640 1 --2- 172.21.15.43:0/3804245945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0648508d0 0x55a064858510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.748+0000 7f77729ae640 1 --2- 172.21.15.43:0/3804245945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a06484fb60 0x55a064864d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.748+0000 7f77729ae640 1 -- 172.21.15.43:0/3804245945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0648508d0 msgr2=0x55a064858510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.748+0000 7f77729ae640 1 --2- 172.21.15.43:0/3804245945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0648508d0 0x55a064858510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.748+0000 7f77729ae640 1 -- 172.21.15.43:0/3804245945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a06484fb60 msgr2=0x55a064864d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.748+0000 7f77729ae640 1 --2- 172.21.15.43:0/3804245945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a06484fb60 0x55a064864d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.748+0000 7f77729ae640 1 -- 172.21.15.43:0/3804245945 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a064855ad0 con 0x55a0648590e0 2024-04-21T08:46:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.748+0000 7f77729ae640 1 --2- 172.21.15.43:0/3804245945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0648590e0 0x55a064858a50 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f7764012960 tx=0x7f7764004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa46dfa57d05d7c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.748+0000 7f7757fff640 1 -- 172.21.15.43:0/3804245945 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764031020 con 0x55a0648590e0 2024-04-21T08:46:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.748+0000 7f7757fff640 1 -- 172.21.15.43:0/3804245945 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7764014110 con 0x55a0648590e0 2024-04-21T08:46:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.749+0000 7f7757fff640 1 -- 172.21.15.43:0/3804245945 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764012e60 con 0x55a0648590e0 2024-04-21T08:46:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.749+0000 7f7777001740 1 -- 172.21.15.43:0/3804245945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0648590e0 msgr2=0x55a064858a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.749+0000 7f7777001740 1 --2- 172.21.15.43:0/3804245945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0648590e0 0x55a064858a50 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f7764012960 tx=0x7f7764004a40 comp rx=0 tx=0).stop 2024-04-21T08:46:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.749+0000 7f7777001740 1 -- 172.21.15.43:0/3804245945 shutdown_connections 2024-04-21T08:46:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.749+0000 7f7777001740 1 --2- 172.21.15.43:0/3804245945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0648590e0 0x55a064858a50 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.749+0000 7f7777001740 1 --2- 172.21.15.43:0/3804245945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0648508d0 0x55a064858510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.749+0000 7f7777001740 1 --2- 172.21.15.43:0/3804245945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a06484fb60 0x55a064864d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.749+0000 7f7777001740 1 -- 172.21.15.43:0/3804245945 >> 172.21.15.43:0/3804245945 conn(0x55a064816c90 msgr2=0x55a06484a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.749+0000 7f7777001740 1 -- 172.21.15.43:0/3804245945 shutdown_connections 2024-04-21T08:46:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.749+0000 7f7777001740 1 -- 172.21.15.43:0/3804245945 wait complete. 2024-04-21T08:46:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.749+0000 7f7777001740 1 -- start start 2024-04-21T08:46:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.749+0000 7f7777001740 1 Processor -- start 2024-04-21T08:46:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.750+0000 7f7777001740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a06484fb60 0x55a064861250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.750+0000 7f7777001740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0648508d0 0x55a064861790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.750+0000 7f7777001740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0648590e0 0x55a064852480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.750+0000 7f7777001740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1b2990 con 0x55a0648508d0 2024-04-21T08:46:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.750+0000 7f7777001740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1b22c0 con 0x55a06484fb60 2024-04-21T08:46:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.750+0000 7f7777001740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1af500 con 0x55a0648590e0 2024-04-21T08:46:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.750+0000 7f7777001740 10 client.0 _ll_register_callbacks cb 0x7f7764014840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.750+0000 7f77721ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0648508d0 0x55a064861790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.750+0000 7f77721ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0648508d0 0x55a064861790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51186/0 (socket says 172.21.15.43:51186) 2024-04-21T08:46:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.750+0000 7f77721ad640 1 -- 172.21.15.43:0/1388979160 learned_addr learned my addr 172.21.15.43:0/1388979160 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.750+0000 7f77729ae640 1 --2- 172.21.15.43:0/1388979160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0648590e0 0x55a064852480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.750+0000 7f77729ae640 1 --2- 172.21.15.43:0/1388979160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a06484fb60 0x55a064861250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.750+0000 7f77721ad640 1 -- 172.21.15.43:0/1388979160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0648590e0 msgr2=0x55a064852480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.750+0000 7f77721ad640 1 --2- 172.21.15.43:0/1388979160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0648590e0 0x55a064852480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.750+0000 7f77721ad640 1 -- 172.21.15.43:0/1388979160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a06484fb60 msgr2=0x55a064861250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.750+0000 7f77721ad640 1 --2- 172.21.15.43:0/1388979160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a06484fb60 0x55a064861250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.750+0000 7f77721ad640 1 -- 172.21.15.43:0/1388979160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a064852c90 con 0x55a0648508d0 2024-04-21T08:46:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.750+0000 7f77729ae640 1 --2- 172.21.15.43:0/1388979160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0648590e0 0x55a064852480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.750+0000 7f77721ad640 1 --2- 172.21.15.43:0/1388979160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0648508d0 0x55a064861790 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f7768009cf0 tx=0x7f7768003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532daa0c6938e908 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.750+0000 7f7755ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.750+0000 7f7755ffb640 1 -- 172.21.15.43:0/1388979160 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776802cb50 con 0x55a0648508d0 2024-04-21T08:46:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.751+0000 7f7755ffb640 1 -- 172.21.15.43:0/1388979160 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9495+0+0 (secure 0 0 0) 0x7f776802ccf0 con 0x55a0648508d0 2024-04-21T08:46:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.751+0000 7f7777001740 1 -- 172.21.15.43:0/1388979160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7764014110 con 0x55a0648508d0 2024-04-21T08:46:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.751+0000 7f7755ffb640 1 -- 172.21.15.43:0/1388979160 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7768039050 con 0x55a0648508d0 2024-04-21T08:46:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.751+0000 7f7777001740 10 client.18783 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.751+0000 7f7777001740 1 -- 172.21.15.43:0/1388979160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a0648cf150 con 0x55a0648508d0 2024-04-21T08:46:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.751+0000 7f7755ffb640 1 -- 172.21.15.43:0/1388979160 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7764014110 con 0x55a0648508d0 2024-04-21T08:46:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.751+0000 7f7755ffb640 1 -- 172.21.15.43:0/1388979160 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f77680035f0 con 0x55a0648508d0 2024-04-21T08:46:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.751+0000 7f7777001740 10 client.18783 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.751+0000 7f7777001740 10 client.18783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.751+0000 7f7777001740 20 client.18783 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.751+0000 7f7777001740 1 --2- 172.21.15.43:0/1388979160 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a0648cf680 0x55a0648d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.751+0000 7f7777001740 4 client.18783 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:46:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.751+0000 7f7777001740 1 -- 172.21.15.43:0/1388979160 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0647543d0 con 0x55a0648cf680 2024-04-21T08:46:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.751+0000 7f77729ae640 1 --2- 172.21.15.43:0/1388979160 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a0648cf680 0x55a0648d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.752+0000 7f7755ffb640 10 client.18783 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:02.752+0000 7f77729ae640 1 --2- 172.21.15.43:0/1388979160 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a0648cf680 0x55a0648d1aa0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a343c4fd5f671cca server_cookie=8d4751f655913137 in_seq=0 out_seq=0 2024-04-21T08:46:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.199+0000 7f7755ffb640 1 -- 172.21.15.43:0/1388979160 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a0647543d0 con 0x55a0648cf680 2024-04-21T08:46:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.199+0000 7f7755ffb640 10 client.18783 handle_command_reply: tid=0 2024-04-21T08:46:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.199+0000 7f7777001740 1 client.18783 shutdown 2024-04-21T08:46:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.199+0000 7f7777001740 1 -- 172.21.15.43:0/1388979160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0648508d0 msgr2=0x55a064861790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.199+0000 7f7777001740 1 --2- 172.21.15.43:0/1388979160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0648508d0 0x55a064861790 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f7768009cf0 tx=0x7f7768003e70 comp rx=0 tx=0).stop 2024-04-21T08:46:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.199+0000 7f7777001740 1 -- 172.21.15.43:0/1388979160 shutdown_connections 2024-04-21T08:46:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.199+0000 7f7777001740 1 --2- 172.21.15.43:0/1388979160 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a0648cf680 0x55a0648d1aa0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.199+0000 7f7777001740 1 --2- 172.21.15.43:0/1388979160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a0648590e0 0x55a064852480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.199+0000 7f7777001740 1 --2- 172.21.15.43:0/1388979160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0648508d0 0x55a064861790 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.199+0000 7f7777001740 1 --2- 172.21.15.43:0/1388979160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a06484fb60 0x55a064861250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.199+0000 7f7755ffb640 0 client.18783 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.199+0000 7f7777001740 1 -- 172.21.15.43:0/1388979160 >> 172.21.15.43:0/1388979160 conn(0x55a064816c90 msgr2=0x55a064854660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.199+0000 7f7777001740 1 -- 172.21.15.43:0/1388979160 shutdown_connections 2024-04-21T08:46:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.199+0000 7f7777001740 1 -- 172.21.15.43:0/1388979160 wait complete. 2024-04-21T08:46:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.199+0000 7f7777001740 20 client.18783 trim_cache size 0 max 16384 2024-04-21T08:46:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:46:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:46:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:46:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:46:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.200+0000 7f7774411640 1 -- 172.21.15.43:0/1838275893 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f77580a3dc0 msgr2=0x7f77580a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.200+0000 7f7774411640 1 --2- 172.21.15.43:0/1838275893 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f77580a3dc0 0x7f77580a6280 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f775c000c00 tx=0x7f775c0df040 comp rx=0 tx=0).stop 2024-04-21T08:46:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.200+0000 7f7774411640 1 -- 172.21.15.43:0/1838275893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f776c107290 msgr2=0x7f776c1a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.200+0000 7f7774411640 1 --2- 172.21.15.43:0/1838275893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f776c107290 0x7f776c1a0b30 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f775c036040 tx=0x7f775c0045e0 comp rx=0 tx=0).stop 2024-04-21T08:46:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.200+0000 7f7774411640 1 -- 172.21.15.43:0/1838275893 shutdown_connections 2024-04-21T08:46:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.200+0000 7f7774411640 1 --2- 172.21.15.43:0/1838275893 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f77580a3dc0 0x7f77580a6280 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.200+0000 7f7774411640 1 --2- 172.21.15.43:0/1838275893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f776c19a6b0 0x7f776c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.200+0000 7f7774411640 1 --2- 172.21.15.43:0/1838275893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f776c107290 0x7f776c1a0b30 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.200+0000 7f7774411640 1 --2- 172.21.15.43:0/1838275893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f776c1021c0 0x7f776c1a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.200+0000 7f7774411640 1 -- 172.21.15.43:0/1838275893 >> 172.21.15.43:0/1838275893 conn(0x7f776c0fda20 msgr2=0x7f776c0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.200+0000 7f7774411640 1 -- 172.21.15.43:0/1838275893 shutdown_connections 2024-04-21T08:46:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:04.200+0000 7f7774411640 1 -- 172.21.15.43:0/1838275893 wait complete. 2024-04-21T08:46:04.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:04 smithi112 ceph-mon[31085]: pgmap v217: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-21T08:46:04.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:04 smithi043 ceph-mon[27978]: pgmap v217: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-21T08:46:04.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:04 smithi175 ceph-mon[31920]: pgmap v217: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-21T08:46:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-mon[31920]: osdmap e96: 12 total, 12 up, 12 in 2024-04-21T08:46:05.464 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: *** Caught signal (Segmentation fault) ** 2024-04-21T08:46:05.464 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: in thread 7f31a7ff3700 thread_name:md_log_replay 2024-04-21T08:46:05.464 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev) 2024-04-21T08:46:05.464 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f31b93b4d20] 2024-04-21T08:46:05.464 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 2: /usr/bin/ceph-mds(+0x54934f) [0x55fb4ee8634f] 2024-04-21T08:46:05.464 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55fb4ee89ae3] 2024-04-21T08:46:05.465 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 4: (MDLog::_replay_thread()+0x617) [0x55fb4eee4607] 2024-04-21T08:46:05.465 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55fb4eeedcf1] 2024-04-21T08:46:05.465 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f31b93aa1ca] 2024-04-21T08:46:05.465 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 7: clone() 2024-04-21T08:46:05.465 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 2024-04-21T08:46:05.131+0000 7f31a7ff3700 -1 *** Caught signal (Segmentation fault) ** 2024-04-21T08:46:05.465 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: in thread 7f31a7ff3700 thread_name:md_log_replay 2024-04-21T08:46:05.465 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 2024-04-21T08:46:05.465 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev) 2024-04-21T08:46:05.465 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f31b93b4d20] 2024-04-21T08:46:05.465 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 2: /usr/bin/ceph-mds(+0x54934f) [0x55fb4ee8634f] 2024-04-21T08:46:05.465 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55fb4ee89ae3] 2024-04-21T08:46:05.465 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 4: (MDLog::_replay_thread()+0x617) [0x55fb4eee4607] 2024-04-21T08:46:05.465 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55fb4eeedcf1] 2024-04-21T08:46:05.465 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f31b93aa1ca] 2024-04-21T08:46:05.466 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 7: clone() 2024-04-21T08:46:05.466 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-21T08:46:05.466 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 2024-04-21T08:46:05.466 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 0> 2024-04-21T08:46:05.131+0000 7f31a7ff3700 -1 *** Caught signal (Segmentation fault) ** 2024-04-21T08:46:05.466 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: in thread 7f31a7ff3700 thread_name:md_log_replay 2024-04-21T08:46:05.466 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 2024-04-21T08:46:05.466 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev) 2024-04-21T08:46:05.466 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f31b93b4d20] 2024-04-21T08:46:05.466 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 2: /usr/bin/ceph-mds(+0x54934f) [0x55fb4ee8634f] 2024-04-21T08:46:05.467 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55fb4ee89ae3] 2024-04-21T08:46:05.467 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 4: (MDLog::_replay_thread()+0x617) [0x55fb4eee4607] 2024-04-21T08:46:05.467 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55fb4eeedcf1] 2024-04-21T08:46:05.467 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f31b93aa1ca] 2024-04-21T08:46:05.467 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 7: clone() 2024-04-21T08:46:05.467 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-21T08:46:05.467 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 2024-04-21T08:46:05.467 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 0> 2024-04-21T08:46:05.131+0000 7f31a7ff3700 -1 *** Caught signal (Segmentation fault) ** 2024-04-21T08:46:05.467 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: in thread 7f31a7ff3700 thread_name:md_log_replay 2024-04-21T08:46:05.467 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 2024-04-21T08:46:05.467 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev) 2024-04-21T08:46:05.468 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f31b93b4d20] 2024-04-21T08:46:05.468 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 2: /usr/bin/ceph-mds(+0x54934f) [0x55fb4ee8634f] 2024-04-21T08:46:05.468 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55fb4ee89ae3] 2024-04-21T08:46:05.468 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 4: (MDLog::_replay_thread()+0x617) [0x55fb4eee4607] 2024-04-21T08:46:05.468 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55fb4eeedcf1] 2024-04-21T08:46:05.468 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f31b93aa1ca] 2024-04-21T08:46:05.468 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 7: clone() 2024-04-21T08:46:05.468 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-21T08:46:05.468 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[62164]: 2024-04-21T08:46:05.468 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:05 smithi175 podman[66561]: 2024-04-21 08:46:05.195207226 +0000 UTC m=+0.022205667 container died bc8e2184d8abade86548f8ef1359450b62f0faec5c6341cd01d1f41e816b8001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS) 2024-04-21T08:46:05.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:05 smithi112 ceph-mon[31085]: osdmap e96: 12 total, 12 up, 12 in 2024-04-21T08:46:05.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:05 smithi043 ceph-mon[27978]: osdmap e96: 12 total, 12 up, 12 in 2024-04-21T08:46:06.464 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:06 smithi175 podman[66561]: 2024-04-21 08:46:06.002717306 +0000 UTC m=+0.829715789 container remove bc8e2184d8abade86548f8ef1359450b62f0faec5c6341cd01d1f41e816b8001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-21T08:46:06.464 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:06 smithi175 systemd[1]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.c.service: Main process exited, code=exited, status=139/n/a 2024-04-21T08:46:06.464 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:06 smithi175 systemd[1]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-04-21T08:46:06.464 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:06 smithi175 systemd[1]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.c.service: Consumed 1.978s CPU time. 2024-04-21T08:46:06.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:06 smithi175 ceph-mon[31920]: pgmap v219: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-21T08:46:06.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:06 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:46:06.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:06 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:46:06.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:06 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:46:06.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:06 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:46:06.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:06 smithi112 ceph-mon[31085]: pgmap v219: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-21T08:46:06.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:06 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:46:06.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:06 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:46:06.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:06 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:46:06.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:06 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:46:06.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:06 smithi043 ceph-mon[27978]: pgmap v219: 129 pgs: 129 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-21T08:46:06.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:06 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:46:06.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:06 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:46:06.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:06 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:46:06.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:06 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:46:06.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:46:06.666 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:46:06.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.776+0000 7fdde9b2c640 1 -- 172.21.15.43:0/639575500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdde4101a60 msgr2=0x7fdde4105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:06.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.776+0000 7fdde9b2c640 1 --2- 172.21.15.43:0/639575500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdde4101a60 0x7fdde4105aa0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009900 tx=0x7fddd802bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:06.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.777+0000 7fdde9b2c640 1 -- 172.21.15.43:0/639575500 shutdown_connections 2024-04-21T08:46:06.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.777+0000 7fdde9b2c640 1 --2- 172.21.15.43:0/639575500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdde4106180 0x7fdde410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.777+0000 7fdde9b2c640 1 --2- 172.21.15.43:0/639575500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdde4101a60 0x7fdde4105aa0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.777+0000 7fdde9b2c640 1 --2- 172.21.15.43:0/639575500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdde41010b0 0x7fdde4101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.777+0000 7fdde9b2c640 1 -- 172.21.15.43:0/639575500 >> 172.21.15.43:0/639575500 conn(0x7fdde40771d0 msgr2=0x7fdde40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:06.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.777+0000 7fdde9b2c640 1 -- 172.21.15.43:0/639575500 shutdown_connections 2024-04-21T08:46:06.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.777+0000 7fdde9b2c640 1 -- 172.21.15.43:0/639575500 wait complete. 2024-04-21T08:46:06.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.777+0000 7fdde9b2c640 1 Processor -- start 2024-04-21T08:46:06.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.777+0000 7fdde9b2c640 1 -- start start 2024-04-21T08:46:06.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.778+0000 7fdde9b2c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdde41010b0 0x7fdde41986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:06.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.778+0000 7fdde9b2c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdde4106180 0x7fdde4198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:06.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.778+0000 7fdde9b2c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdde419cf90 0x7fdde419d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:06.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.778+0000 7fdde9b2c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41110c0 con 0x7fdde419cf90 2024-04-21T08:46:06.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.778+0000 7fdde9b2c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdde4110f40 con 0x7fdde41010b0 2024-04-21T08:46:06.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.778+0000 7fdde9b2c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdde4111240 con 0x7fdde4106180 2024-04-21T08:46:06.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.778+0000 7fdde3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdde419cf90 0x7fdde419d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:06.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.778+0000 7fdde3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdde419cf90 0x7fdde419d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36908/0 (socket says 172.21.15.43:36908) 2024-04-21T08:46:06.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.778+0000 7fdde3fff640 1 -- 172.21.15.43:0/406940116 learned_addr learned my addr 172.21.15.43:0/406940116 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:06.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.778+0000 7fdde2ffd640 1 --2- 172.21.15.43:0/406940116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdde4106180 0x7fdde4198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:06.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.778+0000 7fdde37fe640 1 --2- 172.21.15.43:0/406940116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdde41010b0 0x7fdde41986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:06.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.778+0000 7fdde3fff640 1 -- 172.21.15.43:0/406940116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdde4106180 msgr2=0x7fdde4198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:06.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.778+0000 7fdde3fff640 1 --2- 172.21.15.43:0/406940116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdde4106180 0x7fdde4198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.778+0000 7fdde3fff640 1 -- 172.21.15.43:0/406940116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdde41010b0 msgr2=0x7fdde41986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:06.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.778+0000 7fdde3fff640 1 --2- 172.21.15.43:0/406940116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdde41010b0 0x7fdde41986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.778+0000 7fdde3fff640 1 -- 172.21.15.43:0/406940116 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde419da80 con 0x7fdde419cf90 2024-04-21T08:46:06.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.778+0000 7fdde2ffd640 1 --2- 172.21.15.43:0/406940116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdde4106180 0x7fdde4198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:06.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.778+0000 7fdde3fff640 1 --2- 172.21.15.43:0/406940116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdde419cf90 0x7fdde419d440 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00eb10 tx=0x7fdddc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:06.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.779+0000 7fdde0ff9640 1 -- 172.21.15.43:0/406940116 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc004b20 con 0x7fdde419cf90 2024-04-21T08:46:06.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.779+0000 7fdde0ff9640 1 -- 172.21.15.43:0/406940116 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdddc004cc0 con 0x7fdde419cf90 2024-04-21T08:46:06.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.779+0000 7fdde9b2c640 1 -- 172.21.15.43:0/406940116 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde419dd70 con 0x7fdde419cf90 2024-04-21T08:46:06.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.779+0000 7fdde0ff9640 1 -- 172.21.15.43:0/406940116 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc010670 con 0x7fdde419cf90 2024-04-21T08:46:06.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.779+0000 7fdde9b2c640 1 -- 172.21.15.43:0/406940116 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde41a5530 con 0x7fdde419cf90 2024-04-21T08:46:06.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.781+0000 7fdde0ff9640 1 -- 172.21.15.43:0/406940116 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdddc005020 con 0x7fdde419cf90 2024-04-21T08:46:06.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.781+0000 7fddec7f0740 1 Processor -- start 2024-04-21T08:46:06.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.781+0000 7fddec7f0740 1 -- start start 2024-04-21T08:46:06.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.782+0000 7fddec7f0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7b580 0x55a93aa7b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:06.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.782+0000 7fddec7f0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a93aa7bea0 0x55a93aa84540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:06.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.782+0000 7fddec7f0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a93aa84a80 0x55a93aa86e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:06.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.782+0000 7fddec7f0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a93a9e01e0 con 0x55a93aa7b580 2024-04-21T08:46:06.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.782+0000 7fddec7f0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a93a97d390 con 0x55a93aa7bea0 2024-04-21T08:46:06.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.782+0000 7fddec7f0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a93a9dc1a0 con 0x55a93aa84a80 2024-04-21T08:46:06.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.782+0000 7fdde0ff9640 1 --2- 172.21.15.43:0/406940116 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fddcc0a3d20 0x7fddcc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:06.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.782+0000 7fdde2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7b580 0x55a93aa7b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:06.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.782+0000 7fdde2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7b580 0x55a93aa7b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36914/0 (socket says 172.21.15.43:36914) 2024-04-21T08:46:06.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.782+0000 7fdde2ffd640 1 -- 172.21.15.43:0/3794474901 learned_addr learned my addr 172.21.15.43:0/3794474901 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:06.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.782+0000 7fdde3fff640 1 --2- 172.21.15.43:0/406940116 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fddcc0a3d20 0x7fddcc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:06.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.782+0000 7fdde37fe640 1 --2- 172.21.15.43:0/3794474901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a93aa7bea0 0x55a93aa84540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:06.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.782+0000 7fdde2ffd640 1 --2- 172.21.15.43:0/3794474901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a93aa84a80 0x55a93aa86e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:06.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.782+0000 7fdde0ff9640 1 -- 172.21.15.43:0/406940116 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7fdddc0d1270 con 0x7fdde419cf90 2024-04-21T08:46:06.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.782+0000 7fdde2ffd640 1 -- 172.21.15.43:0/3794474901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a93aa84a80 msgr2=0x55a93aa86e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:06.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.782+0000 7fdde2ffd640 1 --2- 172.21.15.43:0/3794474901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a93aa84a80 0x55a93aa86e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.782+0000 7fdde2ffd640 1 -- 172.21.15.43:0/3794474901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a93aa7bea0 msgr2=0x55a93aa84540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:06.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.782+0000 7fdde2ffd640 1 --2- 172.21.15.43:0/3794474901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a93aa7bea0 0x55a93aa84540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.782+0000 7fdde2ffd640 1 -- 172.21.15.43:0/3794474901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a93a7bc7d0 con 0x55a93aa7b580 2024-04-21T08:46:06.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.782+0000 7fdde2ffd640 1 --2- 172.21.15.43:0/3794474901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7b580 0x55a93aa7b960 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fddd8005c40 tx=0x7fddd8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de80826793e45ec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:06.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.783+0000 7fdde3fff640 1 --2- 172.21.15.43:0/406940116 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fddcc0a3d20 0x7fddcc0a61e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdddc098a10 tx=0x7fdddc06e770 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:06.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.783+0000 7fddc9ffb640 1 -- 172.21.15.43:0/3794474901 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd802c7e0 con 0x55a93aa7b580 2024-04-21T08:46:06.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.783+0000 7fddc9ffb640 1 -- 172.21.15.43:0/3794474901 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddd80035f0 con 0x55a93aa7b580 2024-04-21T08:46:06.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.783+0000 7fddc9ffb640 1 -- 172.21.15.43:0/3794474901 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd80426b0 con 0x55a93aa7b580 2024-04-21T08:46:06.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.783+0000 7fddec7f0740 1 -- 172.21.15.43:0/3794474901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7b580 msgr2=0x55a93aa7b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:06.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.783+0000 7fddec7f0740 1 --2- 172.21.15.43:0/3794474901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7b580 0x55a93aa7b960 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fddd8005c40 tx=0x7fddd8005b50 comp rx=0 tx=0).stop 2024-04-21T08:46:06.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.783+0000 7fddec7f0740 1 -- 172.21.15.43:0/3794474901 shutdown_connections 2024-04-21T08:46:06.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.783+0000 7fddec7f0740 1 --2- 172.21.15.43:0/3794474901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a93aa84a80 0x55a93aa86e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.783+0000 7fddec7f0740 1 --2- 172.21.15.43:0/3794474901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a93aa7bea0 0x55a93aa84540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.783+0000 7fddec7f0740 1 --2- 172.21.15.43:0/3794474901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7b580 0x55a93aa7b960 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.783+0000 7fddec7f0740 1 -- 172.21.15.43:0/3794474901 >> 172.21.15.43:0/3794474901 conn(0x55a93aa75cb0 msgr2=0x55a93aa76090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:06.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.783+0000 7fddec7f0740 1 -- 172.21.15.43:0/3794474901 shutdown_connections 2024-04-21T08:46:06.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.783+0000 7fddec7f0740 1 -- 172.21.15.43:0/3794474901 wait complete. 2024-04-21T08:46:06.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.784+0000 7fddec7f0740 1 -- start start 2024-04-21T08:46:06.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.784+0000 7fddec7f0740 1 Processor -- start 2024-04-21T08:46:06.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.784+0000 7fddec7f0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7b580 0x55a93aa81ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:06.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.784+0000 7fddec7f0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a93aa7bea0 0x55a93aa82000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:06.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.784+0000 7fddec7f0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a93aa84a80 0x55a93aa82540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:06.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.784+0000 7fddec7f0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a93aa87810 con 0x55a93aa7b580 2024-04-21T08:46:06.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.784+0000 7fddec7f0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a93a9992f0 con 0x55a93aa84a80 2024-04-21T08:46:06.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.784+0000 7fddec7f0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a93aa87990 con 0x55a93aa7bea0 2024-04-21T08:46:06.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.784+0000 7fddec7f0740 10 client.0 _ll_register_callbacks cb 0x55a93a9a6580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:06.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.784+0000 7fdde2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7b580 0x55a93aa81ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:06.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.784+0000 7fdde2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7b580 0x55a93aa81ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36920/0 (socket says 172.21.15.43:36920) 2024-04-21T08:46:06.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.784+0000 7fdde2ffd640 1 -- 172.21.15.43:0/4104503754 learned_addr learned my addr 172.21.15.43:0/4104503754 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:06.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.784+0000 7fdde2ffd640 1 --2- 172.21.15.43:0/4104503754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a93aa84a80 0x55a93aa82540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:06.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.785+0000 7fdde37fe640 1 --2- 172.21.15.43:0/4104503754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a93aa7bea0 0x55a93aa82000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:06.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.785+0000 7fdde2ffd640 1 -- 172.21.15.43:0/4104503754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a93aa7bea0 msgr2=0x55a93aa82000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:06.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.785+0000 7fdde2ffd640 1 --2- 172.21.15.43:0/4104503754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a93aa7bea0 0x55a93aa82000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.785+0000 7fdde2ffd640 1 -- 172.21.15.43:0/4104503754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a93aa84a80 msgr2=0x55a93aa82540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:06.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.785+0000 7fdde2ffd640 1 --2- 172.21.15.43:0/4104503754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a93aa84a80 0x55a93aa82540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.785+0000 7fdde2ffd640 1 -- 172.21.15.43:0/4104503754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a93a6f23f0 con 0x55a93aa7b580 2024-04-21T08:46:06.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.785+0000 7fdde2ffd640 1 --2- 172.21.15.43:0/4104503754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7b580 0x55a93aa81ac0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fddd802cdc0 tx=0x7fddd8003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34da9e67266832e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:06.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.785+0000 7fddc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:06.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.785+0000 7fddc2ffd640 1 -- 172.21.15.43:0/4104503754 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8003be0 con 0x55a93aa7b580 2024-04-21T08:46:06.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.785+0000 7fddc2ffd640 1 -- 172.21.15.43:0/4104503754 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7fddd8042470 con 0x55a93aa7b580 2024-04-21T08:46:06.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.785+0000 7fddec7f0740 1 -- 172.21.15.43:0/4104503754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddd80035f0 con 0x55a93aa7b580 2024-04-21T08:46:06.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.785+0000 7fddc2ffd640 1 -- 172.21.15.43:0/4104503754 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fddd8003d80 con 0x55a93aa7b580 2024-04-21T08:46:06.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.785+0000 7fddec7f0740 10 client.18795 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:06.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.785+0000 7fddec7f0740 1 -- 172.21.15.43:0/4104503754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a93a6b20f0 con 0x55a93aa7b580 2024-04-21T08:46:06.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.785+0000 7fddc2ffd640 1 -- 172.21.15.43:0/4104503754 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddd80035f0 con 0x55a93aa7b580 2024-04-21T08:46:06.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.786+0000 7fddc2ffd640 1 -- 172.21.15.43:0/4104503754 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fddd80344d0 con 0x55a93aa7b580 2024-04-21T08:46:06.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.786+0000 7fddec7f0740 10 client.18795 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:06.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.786+0000 7fddec7f0740 10 client.18795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:06.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.786+0000 7fddec7f0740 20 client.18795 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:06.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.786+0000 7fddec7f0740 1 --2- 172.21.15.43:0/4104503754 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a93aa948a0 0x55a93aa96c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:06.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.786+0000 7fddec7f0740 4 client.18795 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:06.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.786+0000 7fddec7f0740 1 -- 172.21.15.43:0/4104503754 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a93a9edc00 con 0x55a93aa948a0 2024-04-21T08:46:06.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.786+0000 7fdde37fe640 1 --2- 172.21.15.43:0/4104503754 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a93aa948a0 0x55a93aa96c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:06.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.786+0000 7fddc2ffd640 10 client.18795 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:06.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.786+0000 7fdde37fe640 1 --2- 172.21.15.43:0/4104503754 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a93aa948a0 0x55a93aa96c90 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56c05508ff1d8f2f server_cookie=b0fead1f7c37ec59 in_seq=0 out_seq=0 2024-04-21T08:46:06.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.787+0000 7fddc2ffd640 1 -- 172.21.15.43:0/4104503754 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a93a9edc00 con 0x55a93aa948a0 2024-04-21T08:46:06.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.787+0000 7fddc2ffd640 10 client.18795 handle_command_reply: tid=0 2024-04-21T08:46:06.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.787+0000 7fddec7f0740 1 client.18795 shutdown 2024-04-21T08:46:06.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.787+0000 7fddec7f0740 1 -- 172.21.15.43:0/4104503754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7b580 msgr2=0x55a93aa81ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:06.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.787+0000 7fddec7f0740 1 --2- 172.21.15.43:0/4104503754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7b580 0x55a93aa81ac0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fddd802cdc0 tx=0x7fddd8003f50 comp rx=0 tx=0).stop 2024-04-21T08:46:06.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.787+0000 7fddec7f0740 1 -- 172.21.15.43:0/4104503754 shutdown_connections 2024-04-21T08:46:06.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.787+0000 7fddec7f0740 1 --2- 172.21.15.43:0/4104503754 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a93aa948a0 0x55a93aa96c90 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.787+0000 7fddec7f0740 1 --2- 172.21.15.43:0/4104503754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a93aa84a80 0x55a93aa82540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.787+0000 7fddc2ffd640 0 client.18795 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:06.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.787+0000 7fddec7f0740 1 --2- 172.21.15.43:0/4104503754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a93aa7bea0 0x55a93aa82000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.787+0000 7fddec7f0740 1 --2- 172.21.15.43:0/4104503754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7b580 0x55a93aa81ac0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.787+0000 7fddec7f0740 1 -- 172.21.15.43:0/4104503754 >> 172.21.15.43:0/4104503754 conn(0x55a93aa75cb0 msgr2=0x55a93aa77a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:06.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.787+0000 7fddec7f0740 1 -- 172.21.15.43:0/4104503754 shutdown_connections 2024-04-21T08:46:06.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.787+0000 7fddec7f0740 1 -- 172.21.15.43:0/4104503754 wait complete. 2024-04-21T08:46:06.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.787+0000 7fddec7f0740 20 client.18795 trim_cache size 0 max 16384 2024-04-21T08:46:06.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.805+0000 7fddec7f0740 1 Processor -- start 2024-04-21T08:46:06.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.805+0000 7fddec7f0740 1 -- start start 2024-04-21T08:46:06.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.805+0000 7fddec7f0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a93aa7b580 0x55a93aa906d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:06.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.805+0000 7fddec7f0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7bea0 0x55a93aa841f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:06.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.805+0000 7fddec7f0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a93aa84a80 0x55a93aa76090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:06.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.805+0000 7fddec7f0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41af6e0 con 0x55a93aa7bea0 2024-04-21T08:46:06.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.805+0000 7fddec7f0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41b1220 con 0x55a93aa84a80 2024-04-21T08:46:06.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.805+0000 7fddec7f0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41afdb0 con 0x55a93aa7b580 2024-04-21T08:46:06.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.805+0000 7fdde37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7bea0 0x55a93aa841f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:06.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.805+0000 7fdde37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7bea0 0x55a93aa841f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36926/0 (socket says 172.21.15.43:36926) 2024-04-21T08:46:06.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.805+0000 7fdde37fe640 1 -- 172.21.15.43:0/3155667335 learned_addr learned my addr 172.21.15.43:0/3155667335 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:06.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.805+0000 7fdde2ffd640 1 --2- 172.21.15.43:0/3155667335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a93aa7b580 0x55a93aa906d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:06.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.805+0000 7fdde37fe640 1 -- 172.21.15.43:0/3155667335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a93aa7b580 msgr2=0x55a93aa906d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:06.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.805+0000 7fdde37fe640 1 --2- 172.21.15.43:0/3155667335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a93aa7b580 0x55a93aa906d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.805+0000 7fdde37fe640 1 -- 172.21.15.43:0/3155667335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a93aa84a80 msgr2=0x55a93aa76090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:06.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.805+0000 7fdde37fe640 1 --2- 172.21.15.43:0/3155667335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a93aa84a80 0x55a93aa76090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.805+0000 7fdde37fe640 1 -- 172.21.15.43:0/3155667335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a93aa97000 con 0x55a93aa7bea0 2024-04-21T08:46:06.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.806+0000 7fdde37fe640 1 --2- 172.21.15.43:0/3155667335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7bea0 0x55a93aa841f0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fddd4000c00 tx=0x7fddd400dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ee8676d1c52d47 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:06.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.806+0000 7fddc97fa640 1 -- 172.21.15.43:0/3155667335 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4003d20 con 0x55a93aa7bea0 2024-04-21T08:46:06.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.806+0000 7fddc97fa640 1 -- 172.21.15.43:0/3155667335 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddd401d920 con 0x55a93aa7bea0 2024-04-21T08:46:06.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.806+0000 7fddc97fa640 1 -- 172.21.15.43:0/3155667335 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4011d60 con 0x55a93aa7bea0 2024-04-21T08:46:06.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.806+0000 7fddec7f0740 1 -- 172.21.15.43:0/3155667335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7bea0 msgr2=0x55a93aa841f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:06.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.806+0000 7fddec7f0740 1 --2- 172.21.15.43:0/3155667335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7bea0 0x55a93aa841f0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fddd4000c00 tx=0x7fddd400dcb0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.806+0000 7fddec7f0740 1 -- 172.21.15.43:0/3155667335 shutdown_connections 2024-04-21T08:46:06.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.806+0000 7fddec7f0740 1 --2- 172.21.15.43:0/3155667335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a93aa84a80 0x55a93aa76090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.806+0000 7fddec7f0740 1 --2- 172.21.15.43:0/3155667335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7bea0 0x55a93aa841f0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.806+0000 7fddec7f0740 1 --2- 172.21.15.43:0/3155667335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a93aa7b580 0x55a93aa906d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.806+0000 7fddec7f0740 1 -- 172.21.15.43:0/3155667335 >> 172.21.15.43:0/3155667335 conn(0x55a93aa75cb0 msgr2=0x55a93aa744e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:06.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.806+0000 7fddec7f0740 1 -- 172.21.15.43:0/3155667335 shutdown_connections 2024-04-21T08:46:06.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.806+0000 7fddec7f0740 1 -- 172.21.15.43:0/3155667335 wait complete. 2024-04-21T08:46:06.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.807+0000 7fddec7f0740 1 -- start start 2024-04-21T08:46:06.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.807+0000 7fddec7f0740 1 Processor -- start 2024-04-21T08:46:06.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.807+0000 7fddec7f0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a93aa7b580 0x55a93aa8cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:06.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.807+0000 7fddec7f0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7bea0 0x55a93aa882a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:06.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.807+0000 7fddec7f0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a93aa84a80 0x55a93aa887e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:06.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.807+0000 7fddec7f0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41b0b50 con 0x55a93aa7bea0 2024-04-21T08:46:06.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.807+0000 7fddec7f0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41b0480 con 0x55a93aa7b580 2024-04-21T08:46:06.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.807+0000 7fddec7f0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41ad6e0 con 0x55a93aa84a80 2024-04-21T08:46:06.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.807+0000 7fddec7f0740 10 client.0 _ll_register_callbacks cb 0x7fddd400d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:06.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.807+0000 7fdde37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7bea0 0x55a93aa882a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:06.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.807+0000 7fdde37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7bea0 0x55a93aa882a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36938/0 (socket says 172.21.15.43:36938) 2024-04-21T08:46:06.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.807+0000 7fdde37fe640 1 -- 172.21.15.43:0/817322756 learned_addr learned my addr 172.21.15.43:0/817322756 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:06.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.807+0000 7fdde2ffd640 1 --2- 172.21.15.43:0/817322756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a93aa84a80 0x55a93aa887e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:06.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.807+0000 7fdde2ffd640 1 --2- 172.21.15.43:0/817322756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a93aa7b580 0x55a93aa8cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:06.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.808+0000 7fdde37fe640 1 -- 172.21.15.43:0/817322756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a93aa84a80 msgr2=0x55a93aa887e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:06.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.808+0000 7fdde37fe640 1 --2- 172.21.15.43:0/817322756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a93aa84a80 0x55a93aa887e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.808+0000 7fdde37fe640 1 -- 172.21.15.43:0/817322756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a93aa7b580 msgr2=0x55a93aa8cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:06.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.808+0000 7fdde37fe640 1 --2- 172.21.15.43:0/817322756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a93aa7b580 0x55a93aa8cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.808+0000 7fdde37fe640 1 -- 172.21.15.43:0/817322756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a93aa89010 con 0x55a93aa7bea0 2024-04-21T08:46:06.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.808+0000 7fdde37fe640 1 --2- 172.21.15.43:0/817322756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7bea0 0x55a93aa882a0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fddd400d7d0 tx=0x7fddd401d5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2f46a50ad4a126 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:06.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.808+0000 7fddc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.808+0000 7fddc2ffd640 1 -- 172.21.15.43:0/817322756 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4003b20 con 0x55a93aa7bea0 2024-04-21T08:46:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.808+0000 7fddc2ffd640 1 -- 172.21.15.43:0/817322756 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7fddd4003cc0 con 0x55a93aa7bea0 2024-04-21T08:46:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.808+0000 7fddec7f0740 1 -- 172.21.15.43:0/817322756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a93a9e7a70 con 0x55a93aa7bea0 2024-04-21T08:46:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.808+0000 7fddc2ffd640 1 -- 172.21.15.43:0/817322756 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fddd4038050 con 0x55a93aa7bea0 2024-04-21T08:46:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.808+0000 7fddec7f0740 10 client.18801 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.808+0000 7fddec7f0740 1 -- 172.21.15.43:0/817322756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a93aafaae0 con 0x55a93aa7bea0 2024-04-21T08:46:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.808+0000 7fddc2ffd640 1 -- 172.21.15.43:0/817322756 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a93a9e7a70 con 0x55a93aa7bea0 2024-04-21T08:46:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.809+0000 7fddc2ffd640 1 -- 172.21.15.43:0/817322756 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fddd402f8d0 con 0x55a93aa7bea0 2024-04-21T08:46:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.809+0000 7fddec7f0740 10 client.18801 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.809+0000 7fddec7f0740 10 client.18801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.809+0000 7fddec7f0740 20 client.18801 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.809+0000 7fddec7f0740 1 --2- 172.21.15.43:0/817322756 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a93aafb1d0 0x55a93aafd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.809+0000 7fddec7f0740 4 client.18801 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:46:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.809+0000 7fddec7f0740 1 -- 172.21.15.43:0/817322756 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a93a963590 con 0x55a93aafb1d0 2024-04-21T08:46:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.809+0000 7fdde2ffd640 1 --2- 172.21.15.43:0/817322756 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a93aafb1d0 0x55a93aafd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:06.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.809+0000 7fddc2ffd640 10 client.18801 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:06.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.809+0000 7fdde2ffd640 1 --2- 172.21.15.43:0/817322756 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a93aafb1d0 0x55a93aafd5f0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb363400fa4aec39 server_cookie=be521a0b07123a3d in_seq=0 out_seq=0 2024-04-21T08:46:06.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.809+0000 7fddc2ffd640 1 -- 172.21.15.43:0/817322756 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a93a963590 con 0x55a93aafb1d0 2024-04-21T08:46:06.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.809+0000 7fddc2ffd640 10 client.18801 handle_command_reply: tid=0 2024-04-21T08:46:06.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.809+0000 7fddec7f0740 1 client.18801 shutdown 2024-04-21T08:46:06.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.810+0000 7fddec7f0740 1 -- 172.21.15.43:0/817322756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7bea0 msgr2=0x55a93aa882a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:06.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.810+0000 7fddec7f0740 1 --2- 172.21.15.43:0/817322756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7bea0 0x55a93aa882a0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fddd400d7d0 tx=0x7fddd401d5d0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.810+0000 7fddec7f0740 1 -- 172.21.15.43:0/817322756 shutdown_connections 2024-04-21T08:46:06.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.810+0000 7fddec7f0740 1 --2- 172.21.15.43:0/817322756 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a93aafb1d0 0x55a93aafd5f0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.810+0000 7fddec7f0740 1 --2- 172.21.15.43:0/817322756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a93aa84a80 0x55a93aa887e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.810+0000 7fddec7f0740 1 --2- 172.21.15.43:0/817322756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a93aa7bea0 0x55a93aa882a0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.810+0000 7fddc2ffd640 0 client.18801 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:06.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.810+0000 7fddec7f0740 1 --2- 172.21.15.43:0/817322756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a93aa7b580 0x55a93aa8cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.810+0000 7fddec7f0740 1 -- 172.21.15.43:0/817322756 >> 172.21.15.43:0/817322756 conn(0x55a93aa75cb0 msgr2=0x55a93aa82310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:06.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.810+0000 7fddec7f0740 1 -- 172.21.15.43:0/817322756 shutdown_connections 2024-04-21T08:46:06.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.810+0000 7fddec7f0740 1 -- 172.21.15.43:0/817322756 wait complete. 2024-04-21T08:46:06.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.810+0000 7fddec7f0740 20 client.18801 trim_cache size 0 max 16384 2024-04-21T08:46:06.816 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:46:06.816 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:46:06.816 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:46:06.816 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:46:06.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.810+0000 7fdde9b2c640 1 -- 172.21.15.43:0/406940116 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fddcc0a3d20 msgr2=0x7fddcc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:06.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.810+0000 7fdde9b2c640 1 --2- 172.21.15.43:0/406940116 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fddcc0a3d20 0x7fddcc0a61e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdddc098a10 tx=0x7fdddc06e770 comp rx=0 tx=0).stop 2024-04-21T08:46:06.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.810+0000 7fdde9b2c640 1 -- 172.21.15.43:0/406940116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdde419cf90 msgr2=0x7fdde419d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:06.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.810+0000 7fdde9b2c640 1 --2- 172.21.15.43:0/406940116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdde419cf90 0x7fdde419d440 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00eb10 tx=0x7fdddc00c2d0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.810+0000 7fdde9b2c640 1 -- 172.21.15.43:0/406940116 shutdown_connections 2024-04-21T08:46:06.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.810+0000 7fdde9b2c640 1 --2- 172.21.15.43:0/406940116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdde419cf90 0x7fdde419d440 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.810+0000 7fdde9b2c640 1 --2- 172.21.15.43:0/406940116 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fddcc0a3d20 0x7fddcc0a61e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.810+0000 7fdde9b2c640 1 --2- 172.21.15.43:0/406940116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdde4106180 0x7fdde4198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.810+0000 7fdde9b2c640 1 --2- 172.21.15.43:0/406940116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdde41010b0 0x7fdde41986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.810+0000 7fdde9b2c640 1 -- 172.21.15.43:0/406940116 >> 172.21.15.43:0/406940116 conn(0x7fdde40771d0 msgr2=0x7fdde406a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:06.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.810+0000 7fdde9b2c640 1 -- 172.21.15.43:0/406940116 shutdown_connections 2024-04-21T08:46:06.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.810+0000 7fdde9b2c640 1 -- 172.21.15.43:0/406940116 wait complete. 2024-04-21T08:46:06.820 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:46:06.821 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:46:06.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.923+0000 7fbdd085c640 1 -- 172.21.15.43:0/2769535390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdcc102b50 msgr2=0x7fbdcc106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:06.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.923+0000 7fbdd085c640 1 --2- 172.21.15.43:0/2769535390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdcc102b50 0x7fbdcc106b90 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4009900 tx=0x7fbdb402bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:06.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.923+0000 7fbdd085c640 1 -- 172.21.15.43:0/2769535390 shutdown_connections 2024-04-21T08:46:06.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.923+0000 7fbdd085c640 1 --2- 172.21.15.43:0/2769535390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdcc107270 0x7fbdcc10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.923+0000 7fbdd085c640 1 --2- 172.21.15.43:0/2769535390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdcc102b50 0x7fbdcc106b90 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.923+0000 7fbdd085c640 1 --2- 172.21.15.43:0/2769535390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdcc1021a0 0x7fbdcc102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.923+0000 7fbdd085c640 1 -- 172.21.15.43:0/2769535390 >> 172.21.15.43:0/2769535390 conn(0x7fbdcc0793a0 msgr2=0x7fbdcc0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:06.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.923+0000 7fbdd085c640 1 -- 172.21.15.43:0/2769535390 shutdown_connections 2024-04-21T08:46:06.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.923+0000 7fbdd085c640 1 -- 172.21.15.43:0/2769535390 wait complete. 2024-04-21T08:46:06.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.924+0000 7fbdd085c640 1 Processor -- start 2024-04-21T08:46:06.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.924+0000 7fbdd085c640 1 -- start start 2024-04-21T08:46:06.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.924+0000 7fbdd085c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdcc1021a0 0x7fbdcc198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:06.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.924+0000 7fbdd085c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdcc107270 0x7fbdcc198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:06.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.924+0000 7fbdd085c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdcc19cfd0 0x7fbdcc19d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:06.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.924+0000 7fbdd085c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc1121b0 con 0x7fbdcc1021a0 2024-04-21T08:46:06.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.924+0000 7fbdd085c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc112030 con 0x7fbdcc19cfd0 2024-04-21T08:46:06.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.924+0000 7fbdd085c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc112330 con 0x7fbdcc107270 2024-04-21T08:46:06.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.924+0000 7fbdca59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdcc1021a0 0x7fbdcc198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:06.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.924+0000 7fbdca59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdcc1021a0 0x7fbdcc198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36966/0 (socket says 172.21.15.43:36966) 2024-04-21T08:46:06.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.924+0000 7fbdca59c640 1 -- 172.21.15.43:0/2534645665 learned_addr learned my addr 172.21.15.43:0/2534645665 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:06.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.924+0000 7fbdc3fff640 1 --2- 172.21.15.43:0/2534645665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdcc107270 0x7fbdcc198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:06.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.925+0000 7fbdcad9d640 1 --2- 172.21.15.43:0/2534645665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdcc19cfd0 0x7fbdcc19d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:06.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.925+0000 7fbdca59c640 1 -- 172.21.15.43:0/2534645665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdcc107270 msgr2=0x7fbdcc198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:06.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.925+0000 7fbdca59c640 1 --2- 172.21.15.43:0/2534645665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdcc107270 0x7fbdcc198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.925+0000 7fbdca59c640 1 -- 172.21.15.43:0/2534645665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdcc19cfd0 msgr2=0x7fbdcc19d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:06.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.925+0000 7fbdca59c640 1 --2- 172.21.15.43:0/2534645665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdcc19cfd0 0x7fbdcc19d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:06.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.925+0000 7fbdca59c640 1 -- 172.21.15.43:0/2534645665 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdcc19dac0 con 0x7fbdcc1021a0 2024-04-21T08:46:06.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.925+0000 7fbdc3fff640 1 --2- 172.21.15.43:0/2534645665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdcc107270 0x7fbdcc198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:06.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.925+0000 7fbdcad9d640 1 --2- 172.21.15.43:0/2534645665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdcc19cfd0 0x7fbdcc19d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:06.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.925+0000 7fbdca59c640 1 --2- 172.21.15.43:0/2534645665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdcc1021a0 0x7fbdcc198710 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc00d770 tx=0x7fbdbc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:06.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.925+0000 7fbdc37fe640 1 -- 172.21.15.43:0/2534645665 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc017070 con 0x7fbdcc1021a0 2024-04-21T08:46:06.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.925+0000 7fbdc37fe640 1 -- 172.21.15.43:0/2534645665 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdbc0054d0 con 0x7fbdcc1021a0 2024-04-21T08:46:06.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.925+0000 7fbdd085c640 1 -- 172.21.15.43:0/2534645665 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdcc19ddb0 con 0x7fbdcc1021a0 2024-04-21T08:46:06.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.925+0000 7fbdc37fe640 1 -- 172.21.15.43:0/2534645665 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc004c30 con 0x7fbdcc1021a0 2024-04-21T08:46:06.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.925+0000 7fbdd085c640 1 -- 172.21.15.43:0/2534645665 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdcc1a5520 con 0x7fbdcc1021a0 2024-04-21T08:46:06.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.926+0000 7fbdd085c640 1 -- 172.21.15.43:0/2534645665 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdcc069230 con 0x7fbdcc1021a0 2024-04-21T08:46:06.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.927+0000 7fbdc37fe640 1 -- 172.21.15.43:0/2534645665 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbdbc004dd0 con 0x7fbdcc1021a0 2024-04-21T08:46:06.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.928+0000 7fbdc37fe640 1 --2- 172.21.15.43:0/2534645665 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbdb80a3d20 0x7fbdb80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:06.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.928+0000 7fbdc3fff640 1 --2- 172.21.15.43:0/2534645665 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbdb80a3d20 0x7fbdb80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:06.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.928+0000 7fbdc37fe640 1 -- 172.21.15.43:0/2534645665 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7fbdbc0d0c40 con 0x7fbdcc1021a0 2024-04-21T08:46:06.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.928+0000 7fbdc3fff640 1 --2- 172.21.15.43:0/2534645665 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbdb80a3d20 0x7fbdb80a61e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4006010 tx=0x7fbdb4005a20 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:06.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:06.931+0000 7fbdc37fe640 1 -- 172.21.15.43:0/2534645665 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbdbc09ce60 con 0x7fbdcc1021a0 2024-04-21T08:46:07.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.157+0000 7fbdd085c640 1 -- 172.21.15.43:0/2534645665 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbdcc199a60 con 0x7fbdcc1021a0 2024-04-21T08:46:07.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.157+0000 7fbdc37fe640 1 -- 172.21.15.43:0/2534645665 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7fbdcc199a60 con 0x7fbdcc1021a0 2024-04-21T08:46:07.157 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:46:07.158 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:46:07.158 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:46:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.159+0000 7fbdd085c640 1 -- 172.21.15.43:0/2534645665 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbdb80a3d20 msgr2=0x7fbdb80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.159+0000 7fbdd085c640 1 --2- 172.21.15.43:0/2534645665 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbdb80a3d20 0x7fbdb80a61e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4006010 tx=0x7fbdb4005a20 comp rx=0 tx=0).stop 2024-04-21T08:46:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.159+0000 7fbdd085c640 1 -- 172.21.15.43:0/2534645665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdcc1021a0 msgr2=0x7fbdcc198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.159+0000 7fbdd085c640 1 --2- 172.21.15.43:0/2534645665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdcc1021a0 0x7fbdcc198710 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc00d770 tx=0x7fbdbc00dc30 comp rx=0 tx=0).stop 2024-04-21T08:46:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.159+0000 7fbdd085c640 1 -- 172.21.15.43:0/2534645665 shutdown_connections 2024-04-21T08:46:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.159+0000 7fbdd085c640 1 --2- 172.21.15.43:0/2534645665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdcc19cfd0 0x7fbdcc19d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.159+0000 7fbdd085c640 1 --2- 172.21.15.43:0/2534645665 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbdb80a3d20 0x7fbdb80a61e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.159+0000 7fbdd085c640 1 --2- 172.21.15.43:0/2534645665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdcc107270 0x7fbdcc198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.159+0000 7fbdd085c640 1 --2- 172.21.15.43:0/2534645665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdcc1021a0 0x7fbdcc198710 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.159+0000 7fbdd085c640 1 -- 172.21.15.43:0/2534645665 >> 172.21.15.43:0/2534645665 conn(0x7fbdcc0793a0 msgr2=0x7fbdcc0fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.159+0000 7fbdd085c640 1 -- 172.21.15.43:0/2534645665 shutdown_connections 2024-04-21T08:46:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.159+0000 7fbdd085c640 1 -- 172.21.15.43:0/2534645665 wait complete. 2024-04-21T08:46:07.170 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:46:07.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.276+0000 7f1832bce640 1 -- 172.21.15.43:0/2532647860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182c107270 msgr2=0x7f182c10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.276+0000 7f1832bce640 1 --2- 172.21.15.43:0/2532647860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182c107270 0x7f182c10ee40 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f1814009900 tx=0x7f181402bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.276+0000 7f1832bce640 1 -- 172.21.15.43:0/2532647860 shutdown_connections 2024-04-21T08:46:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.276+0000 7f1832bce640 1 --2- 172.21.15.43:0/2532647860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182c107270 0x7f182c10ee40 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.276+0000 7f1832bce640 1 --2- 172.21.15.43:0/2532647860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f182c06b1f0 0x7f182c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.276+0000 7f1832bce640 1 --2- 172.21.15.43:0/2532647860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f182c06a840 0x7f182c06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.276+0000 7f1832bce640 1 -- 172.21.15.43:0/2532647860 >> 172.21.15.43:0/2532647860 conn(0x7f182c077eb0 msgr2=0x7f182c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.276+0000 7f1832bce640 1 -- 172.21.15.43:0/2532647860 shutdown_connections 2024-04-21T08:46:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.276+0000 7f1832bce640 1 -- 172.21.15.43:0/2532647860 wait complete. 2024-04-21T08:46:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.276+0000 7f1832bce640 1 Processor -- start 2024-04-21T08:46:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.277+0000 7f1832bce640 1 -- start start 2024-04-21T08:46:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.277+0000 7f1832bce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f182c06a840 0x7f182c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.277+0000 7f1832bce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182c107270 0x7f182c198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.277+0000 7f1832bce640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f182c19cf70 0x7f182c19d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.277+0000 7f1832bce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f182c1121d0 con 0x7f182c107270 2024-04-21T08:46:07.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.277+0000 7f1832bce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f182c112050 con 0x7f182c06a840 2024-04-21T08:46:07.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.277+0000 7f1832bce640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f182c112350 con 0x7f182c19cf70 2024-04-21T08:46:07.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.277+0000 7f182bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182c107270 0x7f182c198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.277+0000 7f182bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182c107270 0x7f182c198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36990/0 (socket says 172.21.15.43:36990) 2024-04-21T08:46:07.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.277+0000 7f182bfff640 1 -- 172.21.15.43:0/4284327466 learned_addr learned my addr 172.21.15.43:0/4284327466 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.277+0000 7f183096a640 1 --2- 172.21.15.43:0/4284327466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f182c06a840 0x7f182c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.277+0000 7f183116b640 1 --2- 172.21.15.43:0/4284327466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f182c19cf70 0x7f182c19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.277+0000 7f182bfff640 1 -- 172.21.15.43:0/4284327466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f182c19cf70 msgr2=0x7f182c19d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.277+0000 7f182bfff640 1 --2- 172.21.15.43:0/4284327466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f182c19cf70 0x7f182c19d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.277+0000 7f182bfff640 1 -- 172.21.15.43:0/4284327466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f182c06a840 msgr2=0x7f182c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.277+0000 7f182bfff640 1 --2- 172.21.15.43:0/4284327466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f182c06a840 0x7f182c1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.277+0000 7f182bfff640 1 -- 172.21.15.43:0/4284327466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f182c19da60 con 0x7f182c107270 2024-04-21T08:46:07.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.277+0000 7f183116b640 1 --2- 172.21.15.43:0/4284327466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f182c19cf70 0x7f182c19d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.277+0000 7f182bfff640 1 --2- 172.21.15.43:0/4284327466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182c107270 0x7f182c198bf0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f181c00ea80 tx=0x7f181c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.277+0000 7f1829ffb640 1 -- 172.21.15.43:0/4284327466 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181c00ce40 con 0x7f182c107270 2024-04-21T08:46:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.278+0000 7f1829ffb640 1 -- 172.21.15.43:0/4284327466 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f181c011920 con 0x7f182c107270 2024-04-21T08:46:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.278+0000 7f1832bce640 1 -- 172.21.15.43:0/4284327466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f182c19dd50 con 0x7f182c107270 2024-04-21T08:46:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.278+0000 7f1829ffb640 1 -- 172.21.15.43:0/4284327466 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181c01aa20 con 0x7f182c107270 2024-04-21T08:46:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.278+0000 7f1832bce640 1 -- 172.21.15.43:0/4284327466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f182c1a5590 con 0x7f182c107270 2024-04-21T08:46:07.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.280+0000 7f1829ffb640 1 -- 172.21.15.43:0/4284327466 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f181c01abc0 con 0x7f182c107270 2024-04-21T08:46:07.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.280+0000 7f18357c1740 1 Processor -- start 2024-04-21T08:46:07.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.280+0000 7f18357c1740 1 -- start start 2024-04-21T08:46:07.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.280+0000 7f18357c1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562679adb8e0 0x562679adbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.280+0000 7f18357c1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562679adc290 0x562679ae4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.280+0000 7f18357c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679ae4e10 0x562679ae7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.280+0000 7f18357c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562679a3c1f0 con 0x562679ae4e10 2024-04-21T08:46:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.280+0000 7f18357c1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5626799d94b0 con 0x562679adb8e0 2024-04-21T08:46:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.280+0000 7f18357c1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562679a381b0 con 0x562679adc290 2024-04-21T08:46:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.280+0000 7f1829ffb640 1 --2- 172.21.15.43:0/4284327466 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f18180a3da0 0x7f18180a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.281+0000 7f182bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679ae4e10 0x562679ae7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.281+0000 7f183116b640 1 --2- 172.21.15.43:0/4284327466 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f18180a3da0 0x7f18180a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.281+0000 7f182bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679ae4e10 0x562679ae7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37004/0 (socket says 172.21.15.43:37004) 2024-04-21T08:46:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.281+0000 7f182bfff640 1 -- 172.21.15.43:0/3268018557 learned_addr learned my addr 172.21.15.43:0/3268018557 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.281+0000 7f183096a640 1 --2- 172.21.15.43:0/3268018557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562679adc290 0x562679ae4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.281+0000 7f183116b640 1 --2- 172.21.15.43:0/3268018557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562679adb8e0 0x562679adbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.281+0000 7f1829ffb640 1 -- 172.21.15.43:0/4284327466 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7f181c0d11a0 con 0x7f182c107270 2024-04-21T08:46:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.281+0000 7f182bfff640 1 -- 172.21.15.43:0/3268018557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562679adc290 msgr2=0x562679ae4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.281+0000 7f182bfff640 1 --2- 172.21.15.43:0/3268018557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562679adc290 0x562679ae4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.281+0000 7f182bfff640 1 -- 172.21.15.43:0/3268018557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562679adb8e0 msgr2=0x562679adbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.281+0000 7f182bfff640 1 --2- 172.21.15.43:0/3268018557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562679adb8e0 0x562679adbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.281+0000 7f182bfff640 1 -- 172.21.15.43:0/3268018557 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626799cb1b0 con 0x562679ae4e10 2024-04-21T08:46:07.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.281+0000 7f182bfff640 1 --2- 172.21.15.43:0/3268018557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679ae4e10 0x562679ae7260 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f181c0ce380 tx=0x7f181c01a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914703a75568257b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.281+0000 7f183116b640 1 --2- 172.21.15.43:0/4284327466 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f18180a3da0 0x7f18180a6260 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1814005940 tx=0x7f1814005e80 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.281+0000 7f1822ffd640 1 -- 172.21.15.43:0/3268018557 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181c09cc60 con 0x562679ae4e10 2024-04-21T08:46:07.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.281+0000 7f1822ffd640 1 -- 172.21.15.43:0/3268018557 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f181c0a1010 con 0x562679ae4e10 2024-04-21T08:46:07.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.281+0000 7f1822ffd640 1 -- 172.21.15.43:0/3268018557 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181c0ce690 con 0x562679ae4e10 2024-04-21T08:46:07.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.282+0000 7f18357c1740 1 -- 172.21.15.43:0/3268018557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679ae4e10 msgr2=0x562679ae7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.282+0000 7f18357c1740 1 --2- 172.21.15.43:0/3268018557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679ae4e10 0x562679ae7260 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f181c0ce380 tx=0x7f181c01a4c0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.282+0000 7f18357c1740 1 -- 172.21.15.43:0/3268018557 shutdown_connections 2024-04-21T08:46:07.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.282+0000 7f18357c1740 1 --2- 172.21.15.43:0/3268018557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679ae4e10 0x562679ae7260 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.282+0000 7f18357c1740 1 --2- 172.21.15.43:0/3268018557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562679adc290 0x562679ae4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.282+0000 7f18357c1740 1 --2- 172.21.15.43:0/3268018557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562679adb8e0 0x562679adbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.282+0000 7f18357c1740 1 -- 172.21.15.43:0/3268018557 >> 172.21.15.43:0/3268018557 conn(0x562679ad5fb0 msgr2=0x562679ad6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.282+0000 7f18357c1740 1 -- 172.21.15.43:0/3268018557 shutdown_connections 2024-04-21T08:46:07.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.282+0000 7f18357c1740 1 -- 172.21.15.43:0/3268018557 wait complete. 2024-04-21T08:46:07.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.282+0000 7f18357c1740 1 -- start start 2024-04-21T08:46:07.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.283+0000 7f18357c1740 1 Processor -- start 2024-04-21T08:46:07.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.283+0000 7f18357c1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562679adb8e0 0x562679ae4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.283+0000 7f18357c1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562679adc290 0x562679ae1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.283+0000 7f18357c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679ae4e10 0x562679ae2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.283+0000 7f18357c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562679ae7cc0 con 0x562679ae4e10 2024-04-21T08:46:07.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.283+0000 7f18357c1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5626799f5500 con 0x562679adc290 2024-04-21T08:46:07.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.283+0000 7f18357c1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562679ae7e40 con 0x562679adb8e0 2024-04-21T08:46:07.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.283+0000 7f18357c1740 10 client.0 _ll_register_callbacks cb 0x5626799c3b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:07.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.283+0000 7f183116b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679ae4e10 0x562679ae2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.283+0000 7f183116b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679ae4e10 0x562679ae2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37020/0 (socket says 172.21.15.43:37020) 2024-04-21T08:46:07.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.283+0000 7f182bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562679adc290 0x562679ae1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.283+0000 7f183116b640 1 -- 172.21.15.43:0/1313502809 learned_addr learned my addr 172.21.15.43:0/1313502809 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.283+0000 7f183096a640 1 --2- 172.21.15.43:0/1313502809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562679adb8e0 0x562679ae4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.284+0000 7f183116b640 1 -- 172.21.15.43:0/1313502809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562679adb8e0 msgr2=0x562679ae4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.284+0000 7f183116b640 1 --2- 172.21.15.43:0/1313502809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562679adb8e0 0x562679ae4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.284+0000 7f183116b640 1 -- 172.21.15.43:0/1313502809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562679adc290 msgr2=0x562679ae1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.284+0000 7f183116b640 1 --2- 172.21.15.43:0/1313502809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562679adc290 0x562679ae1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.284+0000 7f183116b640 1 -- 172.21.15.43:0/1313502809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56267974e3f0 con 0x562679ae4e10 2024-04-21T08:46:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.284+0000 7f182bfff640 1 --2- 172.21.15.43:0/1313502809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562679adc290 0x562679ae1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.284+0000 7f183116b640 1 --2- 172.21.15.43:0/1313502809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679ae4e10 0x562679ae2050 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f1814004290 tx=0x7f1814045040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39600541b873093b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.284+0000 7f180bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.284+0000 7f180bfff640 1 -- 172.21.15.43:0/1313502809 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18140046c0 con 0x562679ae4e10 2024-04-21T08:46:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.284+0000 7f180bfff640 1 -- 172.21.15.43:0/1313502809 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7f1814004860 con 0x562679ae4e10 2024-04-21T08:46:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.284+0000 7f18357c1740 1 -- 172.21.15.43:0/1313502809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f181c0a1010 con 0x562679ae4e10 2024-04-21T08:46:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.284+0000 7f180bfff640 1 -- 172.21.15.43:0/1313502809 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f181405c050 con 0x562679ae4e10 2024-04-21T08:46:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.284+0000 7f18357c1740 10 client.18819 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.284+0000 7f18357c1740 1 -- 172.21.15.43:0/1313502809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5626798187d0 con 0x562679ae4e10 2024-04-21T08:46:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.284+0000 7f180bfff640 1 -- 172.21.15.43:0/1313502809 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f181c0a1010 con 0x562679ae4e10 2024-04-21T08:46:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.284+0000 7f180bfff640 1 -- 172.21.15.43:0/1313502809 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1814040040 con 0x562679ae4e10 2024-04-21T08:46:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.285+0000 7f18357c1740 10 client.18819 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.285+0000 7f18357c1740 10 client.18819 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:46:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.285+0000 7f18357c1740 20 client.18819 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.285+0000 7f18357c1740 1 --2- 172.21.15.43:0/1313502809 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x562679af4f90 0x562679af73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.285+0000 7f18357c1740 4 client.18819 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.285+0000 7f18357c1740 1 -- 172.21.15.43:0/1313502809 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626799db930 con 0x562679af4f90 2024-04-21T08:46:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.285+0000 7f183096a640 1 --2- 172.21.15.43:0/1313502809 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x562679af4f90 0x562679af73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.285+0000 7f180bfff640 10 client.18819 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:46:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.285+0000 7f183096a640 1 --2- 172.21.15.43:0/1313502809 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x562679af4f90 0x562679af73d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8f5266f518838060 server_cookie=ada6f718a2b7b5e5 in_seq=0 out_seq=0 2024-04-21T08:46:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.286+0000 7f180bfff640 1 -- 172.21.15.43:0/1313502809 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5626799db930 con 0x562679af4f90 2024-04-21T08:46:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.286+0000 7f180bfff640 10 client.18819 handle_command_reply: tid=0 2024-04-21T08:46:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.286+0000 7f18357c1740 1 client.18819 shutdown 2024-04-21T08:46:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.286+0000 7f18357c1740 1 -- 172.21.15.43:0/1313502809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679ae4e10 msgr2=0x562679ae2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.286+0000 7f18357c1740 1 --2- 172.21.15.43:0/1313502809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679ae4e10 0x562679ae2050 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f1814004290 tx=0x7f1814045040 comp rx=0 tx=0).stop 2024-04-21T08:46:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.286+0000 7f18357c1740 1 -- 172.21.15.43:0/1313502809 shutdown_connections 2024-04-21T08:46:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.286+0000 7f18357c1740 1 --2- 172.21.15.43:0/1313502809 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x562679af4f90 0x562679af73d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.286+0000 7f18357c1740 1 --2- 172.21.15.43:0/1313502809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679ae4e10 0x562679ae2050 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.286+0000 7f18357c1740 1 --2- 172.21.15.43:0/1313502809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562679adc290 0x562679ae1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.286+0000 7f18357c1740 1 --2- 172.21.15.43:0/1313502809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562679adb8e0 0x562679ae4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.286+0000 7f180bfff640 0 client.18819 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:46:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.286+0000 7f18357c1740 1 -- 172.21.15.43:0/1313502809 >> 172.21.15.43:0/1313502809 conn(0x562679ad5fb0 msgr2=0x562679ad7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.286+0000 7f18357c1740 1 -- 172.21.15.43:0/1313502809 shutdown_connections 2024-04-21T08:46:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.286+0000 7f18357c1740 1 -- 172.21.15.43:0/1313502809 wait complete. 2024-04-21T08:46:07.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.286+0000 7f18357c1740 20 client.18819 trim_cache size 0 max 16384 2024-04-21T08:46:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.300+0000 7f18357c1740 1 Processor -- start 2024-04-21T08:46:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.300+0000 7f18357c1740 1 -- start start 2024-04-21T08:46:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.300+0000 7f18357c1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562679adb8e0 0x562679af0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.300+0000 7f18357c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679adc290 0x562679adcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.300+0000 7f18357c1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562679ae4e10 0x562679ae41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.300+0000 7f18357c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f182c1ad420 con 0x562679adc290 2024-04-21T08:46:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.300+0000 7f18357c1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f182c1aef60 con 0x562679adb8e0 2024-04-21T08:46:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.300+0000 7f18357c1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f182c1adaf0 con 0x562679ae4e10 2024-04-21T08:46:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.300+0000 7f182bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679adc290 0x562679adcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.300+0000 7f182bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679adc290 0x562679adcef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37034/0 (socket says 172.21.15.43:37034) 2024-04-21T08:46:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.300+0000 7f182bfff640 1 -- 172.21.15.43:0/3527183851 learned_addr learned my addr 172.21.15.43:0/3527183851 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.300+0000 7f183096a640 1 --2- 172.21.15.43:0/3527183851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562679adb8e0 0x562679af0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.300+0000 7f183116b640 1 --2- 172.21.15.43:0/3527183851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562679ae4e10 0x562679ae41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.300+0000 7f182bfff640 1 -- 172.21.15.43:0/3527183851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562679ae4e10 msgr2=0x562679ae41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.300+0000 7f182bfff640 1 --2- 172.21.15.43:0/3527183851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562679ae4e10 0x562679ae41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.300+0000 7f182bfff640 1 -- 172.21.15.43:0/3527183851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562679adb8e0 msgr2=0x562679af0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.300+0000 7f182bfff640 1 --2- 172.21.15.43:0/3527183851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562679adb8e0 0x562679af0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.300+0000 7f182bfff640 1 -- 172.21.15.43:0/3527183851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562679ad5ae0 con 0x562679adc290 2024-04-21T08:46:07.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.300+0000 7f183116b640 1 --2- 172.21.15.43:0/3527183851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562679ae4e10 0x562679ae41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:07.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.300+0000 7f182bfff640 1 --2- 172.21.15.43:0/3527183851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679adc290 0x562679adcef0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f181c0995e0 tx=0x7f181c0097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc08a4dd471ac9c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.301+0000 7f18227fc640 1 -- 172.21.15.43:0/3527183851 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181c0a1010 con 0x562679adc290 2024-04-21T08:46:07.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.301+0000 7f18227fc640 1 -- 172.21.15.43:0/3527183851 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f181c09ad00 con 0x562679adc290 2024-04-21T08:46:07.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.301+0000 7f18227fc640 1 -- 172.21.15.43:0/3527183851 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181c0bd0b0 con 0x562679adc290 2024-04-21T08:46:07.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.301+0000 7f18357c1740 1 -- 172.21.15.43:0/3527183851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679adc290 msgr2=0x562679adcef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.301+0000 7f18357c1740 1 --2- 172.21.15.43:0/3527183851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679adc290 0x562679adcef0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f181c0995e0 tx=0x7f181c0097b0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.301+0000 7f18357c1740 1 -- 172.21.15.43:0/3527183851 shutdown_connections 2024-04-21T08:46:07.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.301+0000 7f18357c1740 1 --2- 172.21.15.43:0/3527183851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562679ae4e10 0x562679ae41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.301+0000 7f18357c1740 1 --2- 172.21.15.43:0/3527183851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679adc290 0x562679adcef0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.301+0000 7f18357c1740 1 --2- 172.21.15.43:0/3527183851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562679adb8e0 0x562679af0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.301+0000 7f18357c1740 1 -- 172.21.15.43:0/3527183851 >> 172.21.15.43:0/3527183851 conn(0x562679ad5fb0 msgr2=0x562679adc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.301+0000 7f18357c1740 1 -- 172.21.15.43:0/3527183851 shutdown_connections 2024-04-21T08:46:07.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.301+0000 7f18357c1740 1 -- 172.21.15.43:0/3527183851 wait complete. 2024-04-21T08:46:07.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.301+0000 7f18357c1740 1 -- start start 2024-04-21T08:46:07.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.302+0000 7f18357c1740 1 Processor -- start 2024-04-21T08:46:07.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.302+0000 7f18357c1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562679adb8e0 0x562679aed5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.302+0000 7f18357c1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562679adc290 0x562679ae8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.302+0000 7f18357c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679ae4e10 0x562679ae9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.302+0000 7f18357c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f182c1ae890 con 0x562679ae4e10 2024-04-21T08:46:07.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.302+0000 7f18357c1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f182c1ae1c0 con 0x562679adb8e0 2024-04-21T08:46:07.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.302+0000 7f18357c1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f182c1ab420 con 0x562679adc290 2024-04-21T08:46:07.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.302+0000 7f18357c1740 10 client.0 _ll_register_callbacks cb 0x7f1814009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:07.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.302+0000 7f183116b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679ae4e10 0x562679ae9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.302+0000 7f183116b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679ae4e10 0x562679ae9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37040/0 (socket says 172.21.15.43:37040) 2024-04-21T08:46:07.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.302+0000 7f183116b640 1 -- 172.21.15.43:0/233451253 learned_addr learned my addr 172.21.15.43:0/233451253 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.302+0000 7f183096a640 1 --2- 172.21.15.43:0/233451253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562679adb8e0 0x562679aed5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.302+0000 7f182bfff640 1 --2- 172.21.15.43:0/233451253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562679adc290 0x562679ae8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.302+0000 7f183116b640 1 -- 172.21.15.43:0/233451253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562679adc290 msgr2=0x562679ae8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.302+0000 7f183116b640 1 --2- 172.21.15.43:0/233451253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562679adc290 0x562679ae8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.302+0000 7f183116b640 1 -- 172.21.15.43:0/233451253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562679adb8e0 msgr2=0x562679aed5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.302+0000 7f183116b640 1 --2- 172.21.15.43:0/233451253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562679adb8e0 0x562679aed5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.302+0000 7f183116b640 1 -- 172.21.15.43:0/233451253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562679b5a560 con 0x562679ae4e10 2024-04-21T08:46:07.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.302+0000 7f182bfff640 1 --2- 172.21.15.43:0/233451253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562679adc290 0x562679ae8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:07.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.303+0000 7f183116b640 1 --2- 172.21.15.43:0/233451253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679ae4e10 0x562679ae9090 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f1814003e40 tx=0x7f1814043780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a13e09575e38eb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.303+0000 7f180bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:07.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.303+0000 7f180bfff640 1 -- 172.21.15.43:0/233451253 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1814065060 con 0x562679ae4e10 2024-04-21T08:46:07.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.303+0000 7f180bfff640 1 -- 172.21.15.43:0/233451253 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7f1814043b10 con 0x562679ae4e10 2024-04-21T08:46:07.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.303+0000 7f18357c1740 1 -- 172.21.15.43:0/233451253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f181c09ad00 con 0x562679ae4e10 2024-04-21T08:46:07.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.303+0000 7f180bfff640 1 -- 172.21.15.43:0/233451253 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1814043d10 con 0x562679ae4e10 2024-04-21T08:46:07.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.303+0000 7f18357c1740 10 client.18825 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:07.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.303+0000 7f18357c1740 1 -- 172.21.15.43:0/233451253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562679b5b630 con 0x562679ae4e10 2024-04-21T08:46:07.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.303+0000 7f180bfff640 1 -- 172.21.15.43:0/233451253 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f181c09ad00 con 0x562679ae4e10 2024-04-21T08:46:07.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.303+0000 7f180bfff640 1 -- 172.21.15.43:0/233451253 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f18140046c0 con 0x562679ae4e10 2024-04-21T08:46:07.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.303+0000 7f18357c1740 10 client.18825 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:07.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.303+0000 7f18357c1740 10 client.18825 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:46:07.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.303+0000 7f18357c1740 20 client.18825 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:07.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.303+0000 7f18357c1740 1 --2- 172.21.15.43:0/233451253 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x562679b5bd80 0x562679b5e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.303+0000 7f18357c1740 4 client.18825 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:07.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.303+0000 7f18357c1740 1 -- 172.21.15.43:0/233451253 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562679a49c10 con 0x562679b5bd80 2024-04-21T08:46:07.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.304+0000 7f183096a640 1 --2- 172.21.15.43:0/233451253 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x562679b5bd80 0x562679b5e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.304+0000 7f180bfff640 10 client.18825 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:46:07.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.304+0000 7f183096a640 1 --2- 172.21.15.43:0/233451253 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x562679b5bd80 0x562679b5e1a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8940d122d7c373cd server_cookie=46782203d82436b in_seq=0 out_seq=0 2024-04-21T08:46:07.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.304+0000 7f180bfff640 1 -- 172.21.15.43:0/233451253 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562679a49c10 con 0x562679b5bd80 2024-04-21T08:46:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.304+0000 7f180bfff640 10 client.18825 handle_command_reply: tid=0 2024-04-21T08:46:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.304+0000 7f18357c1740 1 client.18825 shutdown 2024-04-21T08:46:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.304+0000 7f18357c1740 1 -- 172.21.15.43:0/233451253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679ae4e10 msgr2=0x562679ae9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.304+0000 7f18357c1740 1 --2- 172.21.15.43:0/233451253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679ae4e10 0x562679ae9090 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f1814003e40 tx=0x7f1814043780 comp rx=0 tx=0).stop 2024-04-21T08:46:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.304+0000 7f18357c1740 1 -- 172.21.15.43:0/233451253 shutdown_connections 2024-04-21T08:46:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.304+0000 7f18357c1740 1 --2- 172.21.15.43:0/233451253 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x562679b5bd80 0x562679b5e1a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.304+0000 7f18357c1740 1 --2- 172.21.15.43:0/233451253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562679ae4e10 0x562679ae9090 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.304+0000 7f18357c1740 1 --2- 172.21.15.43:0/233451253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562679adc290 0x562679ae8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.304+0000 7f180bfff640 0 client.18825 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:46:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.304+0000 7f18357c1740 1 --2- 172.21.15.43:0/233451253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562679adb8e0 0x562679aed5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.304+0000 7f18357c1740 1 -- 172.21.15.43:0/233451253 >> 172.21.15.43:0/233451253 conn(0x562679ad5fb0 msgr2=0x562679ad0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.305+0000 7f18357c1740 1 -- 172.21.15.43:0/233451253 shutdown_connections 2024-04-21T08:46:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.305+0000 7f18357c1740 1 -- 172.21.15.43:0/233451253 wait complete. 2024-04-21T08:46:07.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.305+0000 7f18357c1740 20 client.18825 trim_cache size 0 max 16384 2024-04-21T08:46:07.311 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:46:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.305+0000 7f1832bce640 1 -- 172.21.15.43:0/4284327466 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f18180a3da0 msgr2=0x7f18180a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.305+0000 7f1832bce640 1 --2- 172.21.15.43:0/4284327466 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f18180a3da0 0x7f18180a6260 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1814005940 tx=0x7f1814005e80 comp rx=0 tx=0).stop 2024-04-21T08:46:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.305+0000 7f1832bce640 1 -- 172.21.15.43:0/4284327466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182c107270 msgr2=0x7f182c198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.305+0000 7f1832bce640 1 --2- 172.21.15.43:0/4284327466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182c107270 0x7f182c198bf0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f181c00ea80 tx=0x7f181c00ef40 comp rx=0 tx=0).stop 2024-04-21T08:46:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.305+0000 7f1832bce640 1 -- 172.21.15.43:0/4284327466 shutdown_connections 2024-04-21T08:46:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.305+0000 7f1832bce640 1 --2- 172.21.15.43:0/4284327466 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f18180a3da0 0x7f18180a6260 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.305+0000 7f1832bce640 1 --2- 172.21.15.43:0/4284327466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f182c19cf70 0x7f182c19d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.305+0000 7f1832bce640 1 --2- 172.21.15.43:0/4284327466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182c107270 0x7f182c198bf0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.305+0000 7f1832bce640 1 --2- 172.21.15.43:0/4284327466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f182c06a840 0x7f182c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.305+0000 7f1832bce640 1 -- 172.21.15.43:0/4284327466 >> 172.21.15.43:0/4284327466 conn(0x7f182c077eb0 msgr2=0x7f182c0fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.305+0000 7f1832bce640 1 -- 172.21.15.43:0/4284327466 shutdown_connections 2024-04-21T08:46:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.305+0000 7f1832bce640 1 -- 172.21.15.43:0/4284327466 wait complete. 2024-04-21T08:46:07.315 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:46:07.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.420+0000 7f6b6e78f640 1 -- 172.21.15.43:0/1425972968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b68107260 msgr2=0x7f6b6810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.420+0000 7f6b6e78f640 1 --2- 172.21.15.43:0/1425972968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b68107260 0x7f6b6810ee30 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009ca0 tx=0x7f6b6002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.420+0000 7f6b6e78f640 1 -- 172.21.15.43:0/1425972968 shutdown_connections 2024-04-21T08:46:07.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.420+0000 7f6b6e78f640 1 --2- 172.21.15.43:0/1425972968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b68107260 0x7f6b6810ee30 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.420+0000 7f6b6e78f640 1 --2- 172.21.15.43:0/1425972968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b68102b40 0x7f6b68106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.420+0000 7f6b6e78f640 1 --2- 172.21.15.43:0/1425972968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b68102190 0x7f6b68102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.420+0000 7f6b6e78f640 1 -- 172.21.15.43:0/1425972968 >> 172.21.15.43:0/1425972968 conn(0x7f6b680fd990 msgr2=0x7f6b680ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.420+0000 7f6b6e78f640 1 -- 172.21.15.43:0/1425972968 shutdown_connections 2024-04-21T08:46:07.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.420+0000 7f6b6e78f640 1 -- 172.21.15.43:0/1425972968 wait complete. 2024-04-21T08:46:07.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.420+0000 7f6b6e78f640 1 Processor -- start 2024-04-21T08:46:07.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.420+0000 7f6b6e78f640 1 -- start start 2024-04-21T08:46:07.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.421+0000 7f6b6e78f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b68102190 0x7f6b68198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.421+0000 7f6b6e78f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b68107260 0x7f6b68198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.421+0000 7f6b6e78f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b6819cf50 0x7f6b6819d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.421+0000 7f6b6e78f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6b681121d0 con 0x7f6b68107260 2024-04-21T08:46:07.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.421+0000 7f6b6e78f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6b68112050 con 0x7f6b68102190 2024-04-21T08:46:07.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.421+0000 7f6b6e78f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6b68112350 con 0x7f6b6819cf50 2024-04-21T08:46:07.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.421+0000 7f6b677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b68107260 0x7f6b68198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.421+0000 7f6b677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b68107260 0x7f6b68198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37052/0 (socket says 172.21.15.43:37052) 2024-04-21T08:46:07.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.421+0000 7f6b677fe640 1 -- 172.21.15.43:0/3420872675 learned_addr learned my addr 172.21.15.43:0/3420872675 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.421+0000 7f6b67fff640 1 --2- 172.21.15.43:0/3420872675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b68102190 0x7f6b68198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.421+0000 7f6b6cd2c640 1 --2- 172.21.15.43:0/3420872675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b6819cf50 0x7f6b6819d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.421+0000 7f6b677fe640 1 -- 172.21.15.43:0/3420872675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b6819cf50 msgr2=0x7f6b6819d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.421+0000 7f6b677fe640 1 --2- 172.21.15.43:0/3420872675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b6819cf50 0x7f6b6819d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.421+0000 7f6b677fe640 1 -- 172.21.15.43:0/3420872675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b68102190 msgr2=0x7f6b68198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.421+0000 7f6b677fe640 1 --2- 172.21.15.43:0/3420872675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b68102190 0x7f6b68198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.421+0000 7f6b677fe640 1 -- 172.21.15.43:0/3420872675 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6819da40 con 0x7f6b68107260 2024-04-21T08:46:07.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.421+0000 7f6b677fe640 1 --2- 172.21.15.43:0/3420872675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b68107260 0x7f6b68198bd0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00c970 tx=0x7f6b5c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.422+0000 7f6b657fa640 1 -- 172.21.15.43:0/3420872675 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c014070 con 0x7f6b68107260 2024-04-21T08:46:07.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.422+0000 7f6b657fa640 1 -- 172.21.15.43:0/3420872675 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b5c004440 con 0x7f6b68107260 2024-04-21T08:46:07.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.422+0000 7f6b6e78f640 1 -- 172.21.15.43:0/3420872675 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b6819dd30 con 0x7f6b68107260 2024-04-21T08:46:07.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.422+0000 7f6b657fa640 1 -- 172.21.15.43:0/3420872675 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c003e50 con 0x7f6b68107260 2024-04-21T08:46:07.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.422+0000 7f6b6e78f640 1 -- 172.21.15.43:0/3420872675 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b681a5520 con 0x7f6b68107260 2024-04-21T08:46:07.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.424+0000 7f6b657fa640 1 -- 172.21.15.43:0/3420872675 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6b5c017070 con 0x7f6b68107260 2024-04-21T08:46:07.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.424+0000 7f6b71380740 1 Processor -- start 2024-04-21T08:46:07.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.424+0000 7f6b71380740 1 -- start start 2024-04-21T08:46:07.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b71380740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55895e1024d0 0x55895e1028b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b71380740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e102e80 0x55895e10b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b71380740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55895e10ba00 0x55895e10de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b71380740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55895e0671f0 con 0x55895e102e80 2024-04-21T08:46:07.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b71380740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55895e0044b0 con 0x55895e1024d0 2024-04-21T08:46:07.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b71380740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55895e0631b0 con 0x55895e10ba00 2024-04-21T08:46:07.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b67fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e102e80 0x55895e10b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b67fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e102e80 0x55895e10b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37056/0 (socket says 172.21.15.43:37056) 2024-04-21T08:46:07.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b67fff640 1 -- 172.21.15.43:0/1412418379 learned_addr learned my addr 172.21.15.43:0/1412418379 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b657fa640 1 --2- 172.21.15.43:0/3420872675 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6b500a3ec0 0x7f6b500a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b677fe640 1 --2- 172.21.15.43:0/1412418379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55895e10ba00 0x55895e10de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b6cd2c640 1 --2- 172.21.15.43:0/1412418379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55895e1024d0 0x55895e1028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b67fff640 1 -- 172.21.15.43:0/1412418379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55895e10ba00 msgr2=0x55895e10de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b67fff640 1 --2- 172.21.15.43:0/1412418379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55895e10ba00 0x55895e10de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b67fff640 1 -- 172.21.15.43:0/1412418379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55895e1024d0 msgr2=0x55895e1028b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b67fff640 1 --2- 172.21.15.43:0/1412418379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55895e1024d0 0x55895e1028b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b67fff640 1 -- 172.21.15.43:0/1412418379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55895dff61b0 con 0x55895e102e80 2024-04-21T08:46:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b677fe640 1 --2- 172.21.15.43:0/1412418379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55895e10ba00 0x55895e10de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b6cd2c640 1 --2- 172.21.15.43:0/1412418379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55895e1024d0 0x55895e1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b657fa640 1 -- 172.21.15.43:0/3420872675 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7f6b5c0d0b90 con 0x7f6b68107260 2024-04-21T08:46:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b6cd2c640 1 --2- 172.21.15.43:0/3420872675 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6b500a3ec0 0x7f6b500a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b67fff640 1 --2- 172.21.15.43:0/1412418379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e102e80 0x55895e10b430 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f6b5800bcf0 tx=0x7f6b5800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c31be8a1ae62ae server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b4e7fc640 1 -- 172.21.15.43:0/1412418379 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b58015070 con 0x55895e102e80 2024-04-21T08:46:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b4e7fc640 1 -- 172.21.15.43:0/1412418379 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b5801a910 con 0x55895e102e80 2024-04-21T08:46:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.425+0000 7f6b4e7fc640 1 -- 172.21.15.43:0/1412418379 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b58011930 con 0x55895e102e80 2024-04-21T08:46:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.426+0000 7f6b71380740 1 -- 172.21.15.43:0/1412418379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e102e80 msgr2=0x55895e10b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.426+0000 7f6b6cd2c640 1 --2- 172.21.15.43:0/3420872675 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6b500a3ec0 0x7f6b500a6380 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009430 tx=0x7f6b6002e040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.426+0000 7f6b71380740 1 --2- 172.21.15.43:0/1412418379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e102e80 0x55895e10b430 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f6b5800bcf0 tx=0x7f6b5800f730 comp rx=0 tx=0).stop 2024-04-21T08:46:07.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.426+0000 7f6b71380740 1 -- 172.21.15.43:0/1412418379 shutdown_connections 2024-04-21T08:46:07.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.426+0000 7f6b71380740 1 --2- 172.21.15.43:0/1412418379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55895e10ba00 0x55895e10de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.426+0000 7f6b71380740 1 --2- 172.21.15.43:0/1412418379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e102e80 0x55895e10b430 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.426+0000 7f6b71380740 1 --2- 172.21.15.43:0/1412418379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55895e1024d0 0x55895e1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.426+0000 7f6b71380740 1 -- 172.21.15.43:0/1412418379 >> 172.21.15.43:0/1412418379 conn(0x55895e0fcba0 msgr2=0x55895e0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.426+0000 7f6b71380740 1 -- 172.21.15.43:0/1412418379 shutdown_connections 2024-04-21T08:46:07.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.426+0000 7f6b71380740 1 -- 172.21.15.43:0/1412418379 wait complete. 2024-04-21T08:46:07.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.426+0000 7f6b71380740 1 -- start start 2024-04-21T08:46:07.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.427+0000 7f6b71380740 1 Processor -- start 2024-04-21T08:46:07.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.427+0000 7f6b71380740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55895e1024d0 0x55895e115b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.427+0000 7f6b71380740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e102e80 0x55895e116070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.427+0000 7f6b71380740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55895e10ba00 0x55895e10ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.427+0000 7f6b71380740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55895e10e970 con 0x55895e102e80 2024-04-21T08:46:07.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.427+0000 7f6b71380740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55895e020500 con 0x55895e10ba00 2024-04-21T08:46:07.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.427+0000 7f6b71380740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55895e10eaf0 con 0x55895e1024d0 2024-04-21T08:46:07.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.427+0000 7f6b71380740 10 client.0 _ll_register_callbacks cb 0x55895dfeeb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:07.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.427+0000 7f6b677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e102e80 0x55895e116070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.427+0000 7f6b677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e102e80 0x55895e116070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37068/0 (socket says 172.21.15.43:37068) 2024-04-21T08:46:07.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.427+0000 7f6b677fe640 1 -- 172.21.15.43:0/105751328 learned_addr learned my addr 172.21.15.43:0/105751328 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.427+0000 7f6b67fff640 1 --2- 172.21.15.43:0/105751328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55895e1024d0 0x55895e115b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.427+0000 7f6b6cd2c640 1 --2- 172.21.15.43:0/105751328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55895e10ba00 0x55895e10ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.427+0000 7f6b677fe640 1 -- 172.21.15.43:0/105751328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55895e1024d0 msgr2=0x55895e115b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.427+0000 7f6b677fe640 1 --2- 172.21.15.43:0/105751328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55895e1024d0 0x55895e115b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.427+0000 7f6b677fe640 1 -- 172.21.15.43:0/105751328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55895e10ba00 msgr2=0x55895e10ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.427+0000 7f6b677fe640 1 --2- 172.21.15.43:0/105751328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55895e10ba00 0x55895e10ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.427+0000 7f6b677fe640 1 -- 172.21.15.43:0/105751328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55895dd793f0 con 0x55895e102e80 2024-04-21T08:46:07.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.427+0000 7f6b6cd2c640 1 --2- 172.21.15.43:0/105751328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55895e10ba00 0x55895e10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.428+0000 7f6b677fe640 1 --2- 172.21.15.43:0/105751328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e102e80 0x55895e116070 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c004630 tx=0x7f6b5c09c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eff633ce59baa8d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.428+0000 7f6b477fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.428+0000 7f6b477fe640 1 -- 172.21.15.43:0/105751328 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c0ce280 con 0x55895e102e80 2024-04-21T08:46:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.428+0000 7f6b477fe640 1 -- 172.21.15.43:0/105751328 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7f6b5c0a4e30 con 0x55895e102e80 2024-04-21T08:46:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.428+0000 7f6b71380740 1 -- 172.21.15.43:0/105751328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b5801a910 con 0x55895e102e80 2024-04-21T08:46:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.428+0000 7f6b477fe640 1 -- 172.21.15.43:0/105751328 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6b5c0a0a00 con 0x55895e102e80 2024-04-21T08:46:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.428+0000 7f6b71380740 10 client.18837 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.428+0000 7f6b71380740 1 -- 172.21.15.43:0/105751328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55895de437d0 con 0x55895e102e80 2024-04-21T08:46:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.428+0000 7f6b477fe640 1 -- 172.21.15.43:0/105751328 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b5801a910 con 0x55895e102e80 2024-04-21T08:46:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.428+0000 7f6b477fe640 1 -- 172.21.15.43:0/105751328 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6b5c085b90 con 0x55895e102e80 2024-04-21T08:46:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.428+0000 7f6b71380740 10 client.18837 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.428+0000 7f6b71380740 10 client.18837 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:46:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.428+0000 7f6b71380740 20 client.18837 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.428+0000 7f6b71380740 1 --2- 172.21.15.43:0/105751328 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55895e120100 0x55895e1224f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.428+0000 7f6b71380740 4 client.18837 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.428+0000 7f6b71380740 1 -- 172.21.15.43:0/105751328 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55895e006930 con 0x55895e120100 2024-04-21T08:46:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.429+0000 7f6b67fff640 1 --2- 172.21.15.43:0/105751328 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55895e120100 0x55895e1224f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.429+0000 7f6b477fe640 10 client.18837 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:46:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.429+0000 7f6b67fff640 1 --2- 172.21.15.43:0/105751328 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55895e120100 0x55895e1224f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6d23c3c1598452a1 server_cookie=a12e894be7b804d8 in_seq=0 out_seq=0 2024-04-21T08:46:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.430+0000 7f6b477fe640 1 -- 172.21.15.43:0/105751328 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55895e006930 con 0x55895e120100 2024-04-21T08:46:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.430+0000 7f6b477fe640 10 client.18837 handle_command_reply: tid=0 2024-04-21T08:46:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.430+0000 7f6b71380740 1 client.18837 shutdown 2024-04-21T08:46:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.430+0000 7f6b71380740 1 -- 172.21.15.43:0/105751328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e102e80 msgr2=0x55895e116070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.430+0000 7f6b71380740 1 --2- 172.21.15.43:0/105751328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e102e80 0x55895e116070 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c004630 tx=0x7f6b5c09c250 comp rx=0 tx=0).stop 2024-04-21T08:46:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.430+0000 7f6b71380740 1 -- 172.21.15.43:0/105751328 shutdown_connections 2024-04-21T08:46:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.430+0000 7f6b71380740 1 --2- 172.21.15.43:0/105751328 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55895e120100 0x55895e1224f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.430+0000 7f6b71380740 1 --2- 172.21.15.43:0/105751328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55895e10ba00 0x55895e10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.430+0000 7f6b71380740 1 --2- 172.21.15.43:0/105751328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e102e80 0x55895e116070 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.430+0000 7f6b477fe640 0 client.18837 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:46:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.430+0000 7f6b71380740 1 --2- 172.21.15.43:0/105751328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55895e1024d0 0x55895e115b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.430+0000 7f6b71380740 1 -- 172.21.15.43:0/105751328 >> 172.21.15.43:0/105751328 conn(0x55895e0fcba0 msgr2=0x55895e0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.430+0000 7f6b71380740 1 -- 172.21.15.43:0/105751328 shutdown_connections 2024-04-21T08:46:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.430+0000 7f6b71380740 1 -- 172.21.15.43:0/105751328 wait complete. 2024-04-21T08:46:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.430+0000 7f6b71380740 20 client.18837 trim_cache size 0 max 16384 2024-04-21T08:46:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.444+0000 7f6b71380740 1 Processor -- start 2024-04-21T08:46:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.444+0000 7f6b71380740 1 -- start start 2024-04-21T08:46:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.444+0000 7f6b71380740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e1024d0 0x55895e127dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.444+0000 7f6b71380740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55895e102e80 0x55895e128300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.444+0000 7f6b71380740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55895e10ba00 0x55895e109db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.444+0000 7f6b71380740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6b681ad3f0 con 0x55895e1024d0 2024-04-21T08:46:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.444+0000 7f6b71380740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6b681aef30 con 0x55895e102e80 2024-04-21T08:46:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.444+0000 7f6b71380740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6b681adac0 con 0x55895e10ba00 2024-04-21T08:46:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.444+0000 7f6b67fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e1024d0 0x55895e127dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.444+0000 7f6b67fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e1024d0 0x55895e127dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37082/0 (socket says 172.21.15.43:37082) 2024-04-21T08:46:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.444+0000 7f6b67fff640 1 -- 172.21.15.43:0/445370912 learned_addr learned my addr 172.21.15.43:0/445370912 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.444+0000 7f6b677fe640 1 --2- 172.21.15.43:0/445370912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55895e102e80 0x55895e128300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.444+0000 7f6b67fff640 1 -- 172.21.15.43:0/445370912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55895e10ba00 msgr2=0x55895e109db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.444+0000 7f6b67fff640 1 --2- 172.21.15.43:0/445370912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55895e10ba00 0x55895e109db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.444+0000 7f6b67fff640 1 -- 172.21.15.43:0/445370912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55895e102e80 msgr2=0x55895e128300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.444+0000 7f6b67fff640 1 --2- 172.21.15.43:0/445370912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55895e102e80 0x55895e128300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.444+0000 7f6b67fff640 1 -- 172.21.15.43:0/445370912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55895e10f260 con 0x55895e1024d0 2024-04-21T08:46:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.444+0000 7f6b677fe640 1 --2- 172.21.15.43:0/445370912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55895e102e80 0x55895e128300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.445+0000 7f6b67fff640 1 --2- 172.21.15.43:0/445370912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e1024d0 0x55895e127dc0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f6b5800bcf0 tx=0x7f6b58005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc9727cfdd36a10 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.445+0000 7f6b4dffb640 1 -- 172.21.15.43:0/445370912 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b58015040 con 0x55895e1024d0 2024-04-21T08:46:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.445+0000 7f6b4dffb640 1 -- 172.21.15.43:0/445370912 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b580054f0 con 0x55895e1024d0 2024-04-21T08:46:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.445+0000 7f6b4dffb640 1 -- 172.21.15.43:0/445370912 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b58026080 con 0x55895e1024d0 2024-04-21T08:46:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.445+0000 7f6b71380740 1 -- 172.21.15.43:0/445370912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e1024d0 msgr2=0x55895e127dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.445+0000 7f6b71380740 1 --2- 172.21.15.43:0/445370912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e1024d0 0x55895e127dc0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f6b5800bcf0 tx=0x7f6b58005c50 comp rx=0 tx=0).stop 2024-04-21T08:46:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.445+0000 7f6b71380740 1 -- 172.21.15.43:0/445370912 shutdown_connections 2024-04-21T08:46:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.445+0000 7f6b71380740 1 --2- 172.21.15.43:0/445370912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55895e10ba00 0x55895e109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.445+0000 7f6b71380740 1 --2- 172.21.15.43:0/445370912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55895e102e80 0x55895e128300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.445+0000 7f6b71380740 1 --2- 172.21.15.43:0/445370912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e1024d0 0x55895e127dc0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.445+0000 7f6b71380740 1 -- 172.21.15.43:0/445370912 >> 172.21.15.43:0/445370912 conn(0x55895e0fcba0 msgr2=0x55895e0fbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.445+0000 7f6b71380740 1 -- 172.21.15.43:0/445370912 shutdown_connections 2024-04-21T08:46:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.445+0000 7f6b71380740 1 -- 172.21.15.43:0/445370912 wait complete. 2024-04-21T08:46:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.446+0000 7f6b71380740 1 -- start start 2024-04-21T08:46:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.446+0000 7f6b71380740 1 Processor -- start 2024-04-21T08:46:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.446+0000 7f6b71380740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55895e1024d0 0x55895e119e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.446+0000 7f6b71380740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e102e80 0x55895e11a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.446+0000 7f6b71380740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55895e10ba00 0x55895e185640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.446+0000 7f6b71380740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6b681ae860 con 0x55895e102e80 2024-04-21T08:46:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.446+0000 7f6b71380740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6b681ae190 con 0x55895e10ba00 2024-04-21T08:46:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.446+0000 7f6b71380740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6b681ab3f0 con 0x55895e1024d0 2024-04-21T08:46:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.446+0000 7f6b71380740 10 client.0 _ll_register_callbacks cb 0x7f6b5c00a580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.446+0000 7f6b677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e102e80 0x55895e11a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.446+0000 7f6b677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e102e80 0x55895e11a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37086/0 (socket says 172.21.15.43:37086) 2024-04-21T08:46:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.446+0000 7f6b677fe640 1 -- 172.21.15.43:0/4180818125 learned_addr learned my addr 172.21.15.43:0/4180818125 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.446+0000 7f6b6cd2c640 1 --2- 172.21.15.43:0/4180818125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55895e10ba00 0x55895e185640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.447+0000 7f6b677fe640 1 -- 172.21.15.43:0/4180818125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55895e1024d0 msgr2=0x55895e119e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.447+0000 7f6b677fe640 1 --2- 172.21.15.43:0/4180818125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55895e1024d0 0x55895e119e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.447+0000 7f6b677fe640 1 -- 172.21.15.43:0/4180818125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55895e10ba00 msgr2=0x55895e185640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.447+0000 7f6b677fe640 1 --2- 172.21.15.43:0/4180818125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55895e10ba00 0x55895e185640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.447+0000 7f6b677fe640 1 -- 172.21.15.43:0/4180818125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55895e11ac30 con 0x55895e102e80 2024-04-21T08:46:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.447+0000 7f6b677fe640 1 --2- 172.21.15.43:0/4180818125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e102e80 0x55895e11a360 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c099020 tx=0x7f6b5c09ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4791c7024b6358 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.447+0000 7f6b477fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.447+0000 7f6b477fe640 1 -- 172.21.15.43:0/4180818125 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c0ce290 con 0x55895e102e80 2024-04-21T08:46:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.447+0000 7f6b477fe640 1 -- 172.21.15.43:0/4180818125 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7f6b5c0a5030 con 0x55895e102e80 2024-04-21T08:46:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.447+0000 7f6b71380740 1 -- 172.21.15.43:0/4180818125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b580054f0 con 0x55895e102e80 2024-04-21T08:46:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.447+0000 7f6b477fe640 1 -- 172.21.15.43:0/4180818125 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6b5c0a52c0 con 0x55895e102e80 2024-04-21T08:46:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.447+0000 7f6b71380740 10 client.18843 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.447+0000 7f6b71380740 1 -- 172.21.15.43:0/4180818125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55895e186bc0 con 0x55895e102e80 2024-04-21T08:46:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.447+0000 7f6b477fe640 1 -- 172.21.15.43:0/4180818125 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b580054f0 con 0x55895e102e80 2024-04-21T08:46:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.447+0000 7f6b477fe640 1 -- 172.21.15.43:0/4180818125 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6b5c0a1b90 con 0x55895e102e80 2024-04-21T08:46:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.447+0000 7f6b71380740 10 client.18843 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.447+0000 7f6b71380740 10 client.18843 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:46:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.447+0000 7f6b71380740 20 client.18843 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.447+0000 7f6b71380740 1 --2- 172.21.15.43:0/4180818125 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55895e187150 0x55895e189540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.447+0000 7f6b71380740 4 client.18843 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.447+0000 7f6b71380740 1 -- 172.21.15.43:0/4180818125 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55895e074c10 con 0x55895e187150 2024-04-21T08:46:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.448+0000 7f6b67fff640 1 --2- 172.21.15.43:0/4180818125 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55895e187150 0x55895e189540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.448+0000 7f6b67fff640 1 --2- 172.21.15.43:0/4180818125 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55895e187150 0x55895e189540 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4a2b9190bc1657f2 server_cookie=96527f8b2b1fa690 in_seq=0 out_seq=0 2024-04-21T08:46:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.448+0000 7f6b477fe640 10 client.18843 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:46:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.448+0000 7f6b477fe640 1 -- 172.21.15.43:0/4180818125 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55895e074c10 con 0x55895e187150 2024-04-21T08:46:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.448+0000 7f6b477fe640 10 client.18843 handle_command_reply: tid=0 2024-04-21T08:46:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.448+0000 7f6b71380740 1 client.18843 shutdown 2024-04-21T08:46:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.448+0000 7f6b71380740 1 -- 172.21.15.43:0/4180818125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e102e80 msgr2=0x55895e11a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.448+0000 7f6b71380740 1 --2- 172.21.15.43:0/4180818125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e102e80 0x55895e11a360 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c099020 tx=0x7f6b5c09ce10 comp rx=0 tx=0).stop 2024-04-21T08:46:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.448+0000 7f6b71380740 1 -- 172.21.15.43:0/4180818125 shutdown_connections 2024-04-21T08:46:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.448+0000 7f6b71380740 1 --2- 172.21.15.43:0/4180818125 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55895e187150 0x55895e189540 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.448+0000 7f6b71380740 1 --2- 172.21.15.43:0/4180818125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55895e10ba00 0x55895e185640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.448+0000 7f6b71380740 1 --2- 172.21.15.43:0/4180818125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55895e102e80 0x55895e11a360 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.448+0000 7f6b477fe640 0 client.18843 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:46:07.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.448+0000 7f6b71380740 1 --2- 172.21.15.43:0/4180818125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55895e1024d0 0x55895e119e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.448+0000 7f6b71380740 1 -- 172.21.15.43:0/4180818125 >> 172.21.15.43:0/4180818125 conn(0x55895e0fcba0 msgr2=0x55895e1077e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.448+0000 7f6b71380740 1 -- 172.21.15.43:0/4180818125 shutdown_connections 2024-04-21T08:46:07.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.448+0000 7f6b71380740 1 -- 172.21.15.43:0/4180818125 wait complete. 2024-04-21T08:46:07.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.449+0000 7f6b71380740 20 client.18843 trim_cache size 0 max 16384 2024-04-21T08:46:07.454 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:46:07.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.449+0000 7f6b6e78f640 1 -- 172.21.15.43:0/3420872675 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6b500a3ec0 msgr2=0x7f6b500a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.449+0000 7f6b6e78f640 1 --2- 172.21.15.43:0/3420872675 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6b500a3ec0 0x7f6b500a6380 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009430 tx=0x7f6b6002e040 comp rx=0 tx=0).stop 2024-04-21T08:46:07.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.449+0000 7f6b6e78f640 1 -- 172.21.15.43:0/3420872675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b68107260 msgr2=0x7f6b68198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.449+0000 7f6b6e78f640 1 --2- 172.21.15.43:0/3420872675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b68107260 0x7f6b68198bd0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00c970 tx=0x7f6b5c00ce30 comp rx=0 tx=0).stop 2024-04-21T08:46:07.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.449+0000 7f6b6e78f640 1 -- 172.21.15.43:0/3420872675 shutdown_connections 2024-04-21T08:46:07.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.449+0000 7f6b6e78f640 1 --2- 172.21.15.43:0/3420872675 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6b500a3ec0 0x7f6b500a6380 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.449+0000 7f6b6e78f640 1 --2- 172.21.15.43:0/3420872675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b6819cf50 0x7f6b6819d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.449+0000 7f6b6e78f640 1 --2- 172.21.15.43:0/3420872675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b68107260 0x7f6b68198bd0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.449+0000 7f6b6e78f640 1 --2- 172.21.15.43:0/3420872675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b68102190 0x7f6b68198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.449+0000 7f6b6e78f640 1 -- 172.21.15.43:0/3420872675 >> 172.21.15.43:0/3420872675 conn(0x7f6b680fd990 msgr2=0x7f6b680ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.449+0000 7f6b6e78f640 1 -- 172.21.15.43:0/3420872675 shutdown_connections 2024-04-21T08:46:07.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.449+0000 7f6b6e78f640 1 -- 172.21.15.43:0/3420872675 wait complete. 2024-04-21T08:46:07.458 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:07 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2534645665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:07.459 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:46:07.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:07 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2534645665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:07.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.565+0000 7f733d1a4640 1 -- 172.21.15.43:0/3479680089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f733806a840 msgr2=0x7f733806ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.565+0000 7f733d1a4640 1 --2- 172.21.15.43:0/3479680089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f733806a840 0x7f733806ac20 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f7328009900 tx=0x7f732802bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:07.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.565+0000 7f733d1a4640 1 -- 172.21.15.43:0/3479680089 shutdown_connections 2024-04-21T08:46:07.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.565+0000 7f733d1a4640 1 --2- 172.21.15.43:0/3479680089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7338107270 0x7f733810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.565+0000 7f733d1a4640 1 --2- 172.21.15.43:0/3479680089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f733806b1f0 0x7f7338106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.565+0000 7f733d1a4640 1 --2- 172.21.15.43:0/3479680089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f733806a840 0x7f733806ac20 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.565+0000 7f733d1a4640 1 -- 172.21.15.43:0/3479680089 >> 172.21.15.43:0/3479680089 conn(0x7f7338077eb0 msgr2=0x7f73380782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.565+0000 7f733d1a4640 1 -- 172.21.15.43:0/3479680089 shutdown_connections 2024-04-21T08:46:07.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.565+0000 7f733d1a4640 1 -- 172.21.15.43:0/3479680089 wait complete. 2024-04-21T08:46:07.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.566+0000 7f733d1a4640 1 Processor -- start 2024-04-21T08:46:07.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.566+0000 7f733d1a4640 1 -- start start 2024-04-21T08:46:07.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.566+0000 7f733d1a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f733806a840 0x7f7338100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.566+0000 7f733d1a4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7338107270 0x7f73381013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.566+0000 7f733d1a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7338102b20 0x7f7338101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.566+0000 7f733d1a4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f73381121b0 con 0x7f733806a840 2024-04-21T08:46:07.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.566+0000 7f733d1a4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7338112030 con 0x7f7338102b20 2024-04-21T08:46:07.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.566+0000 7f733d1a4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7338112330 con 0x7f7338107270 2024-04-21T08:46:07.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.566+0000 7f7336d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f733806a840 0x7f7338100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.567+0000 7f7336d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f733806a840 0x7f7338100e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37110/0 (socket says 172.21.15.43:37110) 2024-04-21T08:46:07.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.567+0000 7f7336d9d640 1 -- 172.21.15.43:0/2734339934 learned_addr learned my addr 172.21.15.43:0/2734339934 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.567+0000 7f733759e640 1 --2- 172.21.15.43:0/2734339934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7338102b20 0x7f7338101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.567+0000 7f732ffff640 1 --2- 172.21.15.43:0/2734339934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7338107270 0x7f73381013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.567+0000 7f7336d9d640 1 -- 172.21.15.43:0/2734339934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7338107270 msgr2=0x7f73381013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.567+0000 7f7336d9d640 1 --2- 172.21.15.43:0/2734339934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7338107270 0x7f73381013c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.567+0000 7f7336d9d640 1 -- 172.21.15.43:0/2734339934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7338102b20 msgr2=0x7f7338101900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.567+0000 7f7336d9d640 1 --2- 172.21.15.43:0/2734339934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7338102b20 0x7f7338101900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.567+0000 7f7336d9d640 1 -- 172.21.15.43:0/2734339934 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73381ab3e0 con 0x7f733806a840 2024-04-21T08:46:07.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.567+0000 7f733759e640 1 --2- 172.21.15.43:0/2734339934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7338102b20 0x7f7338101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:07.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.567+0000 7f7336d9d640 1 --2- 172.21.15.43:0/2734339934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f733806a840 0x7f7338100e80 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f7328005940 tx=0x7f732802c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.567+0000 7f7334d99640 1 -- 172.21.15.43:0/2734339934 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7328043020 con 0x7f733806a840 2024-04-21T08:46:07.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.567+0000 7f733d1a4640 1 -- 172.21.15.43:0/2734339934 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73381ab670 con 0x7f733806a840 2024-04-21T08:46:07.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.567+0000 7f7334d99640 1 -- 172.21.15.43:0/2734339934 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f732802ca70 con 0x7f733806a840 2024-04-21T08:46:07.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.567+0000 7f7334d99640 1 -- 172.21.15.43:0/2734339934 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73280039f0 con 0x7f733806a840 2024-04-21T08:46:07.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.567+0000 7f733d1a4640 1 -- 172.21.15.43:0/2734339934 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73381abbc0 con 0x7f733806a840 2024-04-21T08:46:07.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.569+0000 7f7334d99640 1 -- 172.21.15.43:0/2734339934 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7328046020 con 0x7f733806a840 2024-04-21T08:46:07.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.570+0000 7f733fd92740 1 Processor -- start 2024-04-21T08:46:07.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.570+0000 7f733fd92740 1 -- start start 2024-04-21T08:46:07.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.570+0000 7f733fd92740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570398ac4d0 0x5570398ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.570+0000 7f7334d99640 1 --2- 172.21.15.43:0/2734339934 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f73240a3d20 0x7f73240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.570+0000 7f733fd92740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570398ace80 0x5570398b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.570+0000 7f733fd92740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398b5a00 0x5570398b7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.570+0000 7f733fd92740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5570398111f0 con 0x5570398b5a00 2024-04-21T08:46:07.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.570+0000 7f733fd92740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5570397ae4b0 con 0x5570398ac4d0 2024-04-21T08:46:07.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.570+0000 7f733fd92740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55703980d1b0 con 0x5570398ace80 2024-04-21T08:46:07.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.570+0000 7f7336d9d640 1 --2- 172.21.15.43:0/2734339934 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f73240a3d20 0x7f73240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.570+0000 7f733759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398b5a00 0x5570398b7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.571+0000 7f733759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398b5a00 0x5570398b7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37116/0 (socket says 172.21.15.43:37116) 2024-04-21T08:46:07.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.571+0000 7f733759e640 1 -- 172.21.15.43:0/1146140308 learned_addr learned my addr 172.21.15.43:0/1146140308 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.571+0000 7f733759e640 1 --2- 172.21.15.43:0/1146140308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570398ac4d0 0x5570398ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.571+0000 7f7334d99640 1 -- 172.21.15.43:0/2734339934 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7f73280f2d00 con 0x7f733806a840 2024-04-21T08:46:07.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.571+0000 7f732ffff640 1 --2- 172.21.15.43:0/1146140308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570398ace80 0x5570398b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.571+0000 7f733759e640 1 -- 172.21.15.43:0/1146140308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570398ace80 msgr2=0x5570398b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.571+0000 7f733759e640 1 --2- 172.21.15.43:0/1146140308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570398ace80 0x5570398b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.571+0000 7f733759e640 1 -- 172.21.15.43:0/1146140308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570398ac4d0 msgr2=0x5570398ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.571+0000 7f733759e640 1 --2- 172.21.15.43:0/1146140308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570398ac4d0 0x5570398ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.571+0000 7f733759e640 1 -- 172.21.15.43:0/1146140308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570397a01b0 con 0x5570398b5a00 2024-04-21T08:46:07.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.571+0000 7f732ffff640 1 --2- 172.21.15.43:0/1146140308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570398ace80 0x5570398b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:07.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.571+0000 7f7336d9d640 1 --2- 172.21.15.43:0/2734339934 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f73240a3d20 0x7f73240a61e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f73280f05d0 tx=0x7f732808e330 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.571+0000 7f733759e640 1 --2- 172.21.15.43:0/1146140308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398b5a00 0x5570398b7e50 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f7330012ab0 tx=0x7f7330009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61822b99cbf1d322 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.571+0000 7f732d7fa640 1 -- 172.21.15.43:0/1146140308 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330010890 con 0x5570398b5a00 2024-04-21T08:46:07.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.571+0000 7f732d7fa640 1 -- 172.21.15.43:0/1146140308 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f733001e3c0 con 0x5570398b5a00 2024-04-21T08:46:07.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.571+0000 7f732d7fa640 1 -- 172.21.15.43:0/1146140308 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733000e5b0 con 0x5570398b5a00 2024-04-21T08:46:07.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.572+0000 7f733fd92740 1 -- 172.21.15.43:0/1146140308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398b5a00 msgr2=0x5570398b7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.572+0000 7f733fd92740 1 --2- 172.21.15.43:0/1146140308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398b5a00 0x5570398b7e50 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f7330012ab0 tx=0x7f7330009a20 comp rx=0 tx=0).stop 2024-04-21T08:46:07.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.572+0000 7f733fd92740 1 -- 172.21.15.43:0/1146140308 shutdown_connections 2024-04-21T08:46:07.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.572+0000 7f733fd92740 1 --2- 172.21.15.43:0/1146140308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398b5a00 0x5570398b7e50 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.572+0000 7f733fd92740 1 --2- 172.21.15.43:0/1146140308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570398ace80 0x5570398b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.572+0000 7f733fd92740 1 --2- 172.21.15.43:0/1146140308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570398ac4d0 0x5570398ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.572+0000 7f733fd92740 1 -- 172.21.15.43:0/1146140308 >> 172.21.15.43:0/1146140308 conn(0x5570398a6ba0 msgr2=0x5570398a6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.572+0000 7f733fd92740 1 -- 172.21.15.43:0/1146140308 shutdown_connections 2024-04-21T08:46:07.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.572+0000 7f733fd92740 1 -- 172.21.15.43:0/1146140308 wait complete. 2024-04-21T08:46:07.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.572+0000 7f733fd92740 1 -- start start 2024-04-21T08:46:07.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.572+0000 7f733fd92740 1 Processor -- start 2024-04-21T08:46:07.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.573+0000 7f733fd92740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398ac4d0 0x5570398b5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.573+0000 7f733fd92740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570398ace80 0x5570398b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.573+0000 7f733fd92740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570398b5a00 0x5570398b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.573+0000 7f733fd92740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5570398b8970 con 0x5570398ac4d0 2024-04-21T08:46:07.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.573+0000 7f733fd92740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5570397ca500 con 0x5570398b5a00 2024-04-21T08:46:07.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.573+0000 7f733fd92740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5570398b8af0 con 0x5570398ace80 2024-04-21T08:46:07.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.573+0000 7f733759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398ac4d0 0x5570398b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.573+0000 7f733fd92740 10 client.0 _ll_register_callbacks cb 0x557039798b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:07.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.573+0000 7f733759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398ac4d0 0x5570398b5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37120/0 (socket says 172.21.15.43:37120) 2024-04-21T08:46:07.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.573+0000 7f733759e640 1 -- 172.21.15.43:0/4046257411 learned_addr learned my addr 172.21.15.43:0/4046257411 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.573+0000 7f733759e640 1 --2- 172.21.15.43:0/4046257411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570398b5a00 0x5570398b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.573+0000 7f732ffff640 1 --2- 172.21.15.43:0/4046257411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570398ace80 0x5570398b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.573+0000 7f733759e640 1 -- 172.21.15.43:0/4046257411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570398ace80 msgr2=0x5570398b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.573+0000 7f733759e640 1 --2- 172.21.15.43:0/4046257411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570398ace80 0x5570398b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.573+0000 7f733759e640 1 -- 172.21.15.43:0/4046257411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570398b5a00 msgr2=0x5570398b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.573+0000 7f733759e640 1 --2- 172.21.15.43:0/4046257411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570398b5a00 0x5570398b2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.573+0000 7f733759e640 1 -- 172.21.15.43:0/4046257411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570395233f0 con 0x5570398ac4d0 2024-04-21T08:46:07.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.573+0000 7f733759e640 1 --2- 172.21.15.43:0/4046257411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398ac4d0 0x5570398b5320 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f733001ebf0 tx=0x7f7330009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d69c2ab9f4e272 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.573+0000 7f731e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:07.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.573+0000 7f731e7fc640 1 -- 172.21.15.43:0/4046257411 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330004a40 con 0x5570398ac4d0 2024-04-21T08:46:07.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.573+0000 7f731e7fc640 1 -- 172.21.15.43:0/4046257411 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7f7330004be0 con 0x5570398ac4d0 2024-04-21T08:46:07.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.573+0000 7f733fd92740 1 -- 172.21.15.43:0/4046257411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f733001e3c0 con 0x5570398ac4d0 2024-04-21T08:46:07.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.574+0000 7f731e7fc640 1 -- 172.21.15.43:0/4046257411 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7330004de0 con 0x5570398ac4d0 2024-04-21T08:46:07.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.574+0000 7f733fd92740 10 client.18855 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:07.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.574+0000 7f733fd92740 1 -- 172.21.15.43:0/4046257411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5570395ed7d0 con 0x5570398ac4d0 2024-04-21T08:46:07.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.574+0000 7f731e7fc640 1 -- 172.21.15.43:0/4046257411 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f733001e3c0 con 0x5570398ac4d0 2024-04-21T08:46:07.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.574+0000 7f731e7fc640 1 -- 172.21.15.43:0/4046257411 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7330005470 con 0x5570398ac4d0 2024-04-21T08:46:07.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.574+0000 7f733fd92740 10 client.18855 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:07.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.574+0000 7f733fd92740 10 client.18855 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:46:07.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.574+0000 7f733fd92740 20 client.18855 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:07.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.574+0000 7f733fd92740 1 --2- 172.21.15.43:0/4046257411 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5570398c5b50 0x5570398c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.574+0000 7f733fd92740 4 client.18855 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:07.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.574+0000 7f733fd92740 1 -- 172.21.15.43:0/4046257411 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570397b0930 con 0x5570398c5b50 2024-04-21T08:46:07.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.575+0000 7f732ffff640 1 --2- 172.21.15.43:0/4046257411 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5570398c5b50 0x5570398c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.575+0000 7f732ffff640 1 --2- 172.21.15.43:0/4046257411 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5570398c5b50 0x5570398c7f40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a714f02702f418d9 server_cookie=857008f2e6f90203 in_seq=0 out_seq=0 2024-04-21T08:46:07.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.575+0000 7f731e7fc640 10 client.18855 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:46:07.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.576+0000 7f731e7fc640 1 -- 172.21.15.43:0/4046257411 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5570397b0930 con 0x5570398c5b50 2024-04-21T08:46:07.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.576+0000 7f731e7fc640 10 client.18855 handle_command_reply: tid=0 2024-04-21T08:46:07.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.576+0000 7f733fd92740 1 client.18855 shutdown 2024-04-21T08:46:07.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.576+0000 7f733fd92740 1 -- 172.21.15.43:0/4046257411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398ac4d0 msgr2=0x5570398b5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.576+0000 7f733fd92740 1 --2- 172.21.15.43:0/4046257411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398ac4d0 0x5570398b5320 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f733001ebf0 tx=0x7f7330009fb0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.576+0000 7f733fd92740 1 -- 172.21.15.43:0/4046257411 shutdown_connections 2024-04-21T08:46:07.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.576+0000 7f733fd92740 1 --2- 172.21.15.43:0/4046257411 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5570398c5b50 0x5570398c7f40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.576+0000 7f733fd92740 1 --2- 172.21.15.43:0/4046257411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570398b5a00 0x5570398b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.576+0000 7f733fd92740 1 --2- 172.21.15.43:0/4046257411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570398ace80 0x5570398b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.576+0000 7f731e7fc640 0 client.18855 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:46:07.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.576+0000 7f733fd92740 1 --2- 172.21.15.43:0/4046257411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398ac4d0 0x5570398b5320 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.576+0000 7f733fd92740 1 -- 172.21.15.43:0/4046257411 >> 172.21.15.43:0/4046257411 conn(0x5570398a6ba0 msgr2=0x5570398a88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.576+0000 7f733fd92740 1 -- 172.21.15.43:0/4046257411 shutdown_connections 2024-04-21T08:46:07.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.576+0000 7f733fd92740 1 -- 172.21.15.43:0/4046257411 wait complete. 2024-04-21T08:46:07.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.576+0000 7f733fd92740 20 client.18855 trim_cache size 0 max 16384 2024-04-21T08:46:07.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.589+0000 7f733fd92740 1 Processor -- start 2024-04-21T08:46:07.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.589+0000 7f733fd92740 1 -- start start 2024-04-21T08:46:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.590+0000 7f733fd92740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570398ac4d0 0x5570398c1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.590+0000 7f733fd92740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570398ace80 0x5570398add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.590+0000 7f733fd92740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398b5a00 0x5570398cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.590+0000 7f733fd92740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f73381b5d40 con 0x5570398b5a00 2024-04-21T08:46:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.590+0000 7f733fd92740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f73381b7880 con 0x5570398ac4d0 2024-04-21T08:46:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.590+0000 7f733fd92740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f73381b6410 con 0x5570398ace80 2024-04-21T08:46:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.590+0000 7f733759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398b5a00 0x5570398cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.590+0000 7f733759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398b5a00 0x5570398cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37126/0 (socket says 172.21.15.43:37126) 2024-04-21T08:46:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.590+0000 7f733759e640 1 -- 172.21.15.43:0/3030179836 learned_addr learned my addr 172.21.15.43:0/3030179836 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.590+0000 7f732ffff640 1 --2- 172.21.15.43:0/3030179836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570398ace80 0x5570398add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.590+0000 7f733759e640 1 --2- 172.21.15.43:0/3030179836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570398ac4d0 0x5570398c1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.590+0000 7f733759e640 1 -- 172.21.15.43:0/3030179836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570398ace80 msgr2=0x5570398add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.590+0000 7f733759e640 1 --2- 172.21.15.43:0/3030179836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570398ace80 0x5570398add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.590+0000 7f733759e640 1 -- 172.21.15.43:0/3030179836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570398ac4d0 msgr2=0x5570398c1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.590+0000 7f733759e640 1 --2- 172.21.15.43:0/3030179836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570398ac4d0 0x5570398c1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.590+0000 7f733759e640 1 -- 172.21.15.43:0/3030179836 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570398b8850 con 0x5570398b5a00 2024-04-21T08:46:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.590+0000 7f733759e640 1 --2- 172.21.15.43:0/3030179836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398b5a00 0x5570398cd630 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f7330005cc0 tx=0x7f7330009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360b13ac4a45b019 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.590+0000 7f732cff9640 1 -- 172.21.15.43:0/3030179836 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73300099c0 con 0x5570398b5a00 2024-04-21T08:46:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.590+0000 7f732cff9640 1 -- 172.21.15.43:0/3030179836 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f733001edc0 con 0x5570398b5a00 2024-04-21T08:46:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.590+0000 7f732cff9640 1 -- 172.21.15.43:0/3030179836 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330004c00 con 0x5570398b5a00 2024-04-21T08:46:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.591+0000 7f733fd92740 1 -- 172.21.15.43:0/3030179836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398b5a00 msgr2=0x5570398cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.591+0000 7f733fd92740 1 --2- 172.21.15.43:0/3030179836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398b5a00 0x5570398cd630 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f7330005cc0 tx=0x7f7330009d30 comp rx=0 tx=0).stop 2024-04-21T08:46:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.591+0000 7f733fd92740 1 -- 172.21.15.43:0/3030179836 shutdown_connections 2024-04-21T08:46:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.591+0000 7f733fd92740 1 --2- 172.21.15.43:0/3030179836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398b5a00 0x5570398cd630 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.591+0000 7f733fd92740 1 --2- 172.21.15.43:0/3030179836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570398ace80 0x5570398add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.591+0000 7f733fd92740 1 --2- 172.21.15.43:0/3030179836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570398ac4d0 0x5570398c1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.591+0000 7f733fd92740 1 -- 172.21.15.43:0/3030179836 >> 172.21.15.43:0/3030179836 conn(0x5570398a6ba0 msgr2=0x5570398a8660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.591+0000 7f733fd92740 1 -- 172.21.15.43:0/3030179836 shutdown_connections 2024-04-21T08:46:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.591+0000 7f733fd92740 1 -- 172.21.15.43:0/3030179836 wait complete. 2024-04-21T08:46:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.591+0000 7f733fd92740 1 -- start start 2024-04-21T08:46:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.592+0000 7f733fd92740 1 Processor -- start 2024-04-21T08:46:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.592+0000 7f733fd92740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398ac4d0 0x5570398bdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.592+0000 7f733fd92740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570398ace80 0x5570398aef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.592+0000 7f733fd92740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570398b5a00 0x5570398af460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.592+0000 7f733fd92740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f73381b71b0 con 0x5570398ac4d0 2024-04-21T08:46:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.592+0000 7f733fd92740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f73381b6ae0 con 0x5570398ace80 2024-04-21T08:46:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.592+0000 7f733fd92740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f73381b3d40 con 0x5570398b5a00 2024-04-21T08:46:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.592+0000 7f733fd92740 10 client.0 _ll_register_callbacks cb 0x7f733001e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.592+0000 7f733759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398ac4d0 0x5570398bdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.592+0000 7f733759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398ac4d0 0x5570398bdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37142/0 (socket says 172.21.15.43:37142) 2024-04-21T08:46:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.592+0000 7f733759e640 1 -- 172.21.15.43:0/4247753581 learned_addr learned my addr 172.21.15.43:0/4247753581 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.592+0000 7f732ffff640 1 --2- 172.21.15.43:0/4247753581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570398ace80 0x5570398aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.592+0000 7f733759e640 1 -- 172.21.15.43:0/4247753581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570398b5a00 msgr2=0x5570398af460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.592+0000 7f733759e640 1 --2- 172.21.15.43:0/4247753581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570398b5a00 0x5570398af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.592+0000 7f733759e640 1 -- 172.21.15.43:0/4247753581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570398ace80 msgr2=0x5570398aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.592+0000 7f733759e640 1 --2- 172.21.15.43:0/4247753581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570398ace80 0x5570398aef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.592+0000 7f733759e640 1 -- 172.21.15.43:0/4247753581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570398af9a0 con 0x5570398ac4d0 2024-04-21T08:46:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.592+0000 7f733759e640 1 --2- 172.21.15.43:0/4247753581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398ac4d0 0x5570398bdf30 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f733000a010 tx=0x7f7330005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cd2d7c43963fd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.592+0000 7f731e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.593+0000 7f731e7fc640 1 -- 172.21.15.43:0/4247753581 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73300099c0 con 0x5570398ac4d0 2024-04-21T08:46:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.593+0000 7f731e7fc640 1 -- 172.21.15.43:0/4247753581 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7f7330004bd0 con 0x5570398ac4d0 2024-04-21T08:46:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.593+0000 7f733fd92740 1 -- 172.21.15.43:0/4247753581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f733001edc0 con 0x5570398ac4d0 2024-04-21T08:46:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.593+0000 7f731e7fc640 1 -- 172.21.15.43:0/4247753581 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7330012020 con 0x5570398ac4d0 2024-04-21T08:46:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.593+0000 7f733fd92740 10 client.18861 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.593+0000 7f733fd92740 1 -- 172.21.15.43:0/4247753581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55703992bbc0 con 0x5570398ac4d0 2024-04-21T08:46:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.593+0000 7f731e7fc640 1 -- 172.21.15.43:0/4247753581 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f733001edc0 con 0x5570398ac4d0 2024-04-21T08:46:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.593+0000 7f731e7fc640 1 -- 172.21.15.43:0/4247753581 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f733002b9f0 con 0x5570398ac4d0 2024-04-21T08:46:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.593+0000 7f733fd92740 10 client.18861 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.593+0000 7f733fd92740 10 client.18861 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:46:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.593+0000 7f733fd92740 20 client.18861 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.593+0000 7f733fd92740 1 --2- 172.21.15.43:0/4247753581 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55703992c170 0x55703992e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.593+0000 7f733fd92740 4 client.18861 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.593+0000 7f733fd92740 1 -- 172.21.15.43:0/4247753581 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55703981ec10 con 0x55703992c170 2024-04-21T08:46:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.593+0000 7f732ffff640 1 --2- 172.21.15.43:0/4247753581 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55703992c170 0x55703992e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.594+0000 7f732ffff640 1 --2- 172.21.15.43:0/4247753581 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55703992c170 0x55703992e560 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5677c6aec94c5ae1 server_cookie=eeaa6195ad337ca9 in_seq=0 out_seq=0 2024-04-21T08:46:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.594+0000 7f731e7fc640 10 client.18861 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:46:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.594+0000 7f731e7fc640 1 -- 172.21.15.43:0/4247753581 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55703981ec10 con 0x55703992c170 2024-04-21T08:46:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.594+0000 7f731e7fc640 10 client.18861 handle_command_reply: tid=0 2024-04-21T08:46:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.594+0000 7f733fd92740 1 client.18861 shutdown 2024-04-21T08:46:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.594+0000 7f733fd92740 1 -- 172.21.15.43:0/4247753581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398ac4d0 msgr2=0x5570398bdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.594+0000 7f733fd92740 1 --2- 172.21.15.43:0/4247753581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398ac4d0 0x5570398bdf30 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f733000a010 tx=0x7f7330005b20 comp rx=0 tx=0).stop 2024-04-21T08:46:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.594+0000 7f733fd92740 1 -- 172.21.15.43:0/4247753581 shutdown_connections 2024-04-21T08:46:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.594+0000 7f733fd92740 1 --2- 172.21.15.43:0/4247753581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570398b5a00 0x5570398af460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.594+0000 7f733fd92740 1 --2- 172.21.15.43:0/4247753581 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55703992c170 0x55703992e560 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.594+0000 7f733fd92740 1 --2- 172.21.15.43:0/4247753581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570398ace80 0x5570398aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.594+0000 7f733fd92740 1 --2- 172.21.15.43:0/4247753581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570398ac4d0 0x5570398bdf30 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.594+0000 7f731e7fc640 0 client.18861 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:46:07.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.594+0000 7f733fd92740 1 -- 172.21.15.43:0/4247753581 >> 172.21.15.43:0/4247753581 conn(0x5570398a6ba0 msgr2=0x5570398c0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.594+0000 7f733fd92740 1 -- 172.21.15.43:0/4247753581 shutdown_connections 2024-04-21T08:46:07.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.594+0000 7f733fd92740 1 -- 172.21.15.43:0/4247753581 wait complete. 2024-04-21T08:46:07.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.595+0000 7f733fd92740 20 client.18861 trim_cache size 0 max 16384 2024-04-21T08:46:07.600 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:46:07.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.595+0000 7f733d1a4640 1 -- 172.21.15.43:0/2734339934 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f73240a3d20 msgr2=0x7f73240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.595+0000 7f733d1a4640 1 --2- 172.21.15.43:0/2734339934 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f73240a3d20 0x7f73240a61e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f73280f05d0 tx=0x7f732808e330 comp rx=0 tx=0).stop 2024-04-21T08:46:07.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.595+0000 7f733d1a4640 1 -- 172.21.15.43:0/2734339934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f733806a840 msgr2=0x7f7338100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.595+0000 7f733d1a4640 1 --2- 172.21.15.43:0/2734339934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f733806a840 0x7f7338100e80 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f7328005940 tx=0x7f732802c5f0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.595+0000 7f733d1a4640 1 -- 172.21.15.43:0/2734339934 shutdown_connections 2024-04-21T08:46:07.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.595+0000 7f733d1a4640 1 --2- 172.21.15.43:0/2734339934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7338102b20 0x7f7338101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.595+0000 7f733d1a4640 1 --2- 172.21.15.43:0/2734339934 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f73240a3d20 0x7f73240a61e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.595+0000 7f733d1a4640 1 --2- 172.21.15.43:0/2734339934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7338107270 0x7f73381013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.595+0000 7f733d1a4640 1 --2- 172.21.15.43:0/2734339934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f733806a840 0x7f7338100e80 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.595+0000 7f733d1a4640 1 -- 172.21.15.43:0/2734339934 >> 172.21.15.43:0/2734339934 conn(0x7f7338077eb0 msgr2=0x7f7338108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.595+0000 7f733d1a4640 1 -- 172.21.15.43:0/2734339934 shutdown_connections 2024-04-21T08:46:07.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.595+0000 7f733d1a4640 1 -- 172.21.15.43:0/2734339934 wait complete. 2024-04-21T08:46:07.605 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:46:07.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.710+0000 7fbcf552e640 1 -- 172.21.15.43:0/505595555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcf0106170 msgr2=0x7fbcf010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.710+0000 7fbcf552e640 1 --2- 172.21.15.43:0/505595555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcf0106170 0x7fbcf010dd40 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009ca0 tx=0x7fbce802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.710+0000 7fbcf552e640 1 -- 172.21.15.43:0/505595555 shutdown_connections 2024-04-21T08:46:07.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.710+0000 7fbcf552e640 1 --2- 172.21.15.43:0/505595555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcf0106170 0x7fbcf010dd40 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.710+0000 7fbcf552e640 1 --2- 172.21.15.43:0/505595555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcf0101a50 0x7fbcf0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.710+0000 7fbcf552e640 1 --2- 172.21.15.43:0/505595555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcf01010a0 0x7fbcf0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.710+0000 7fbcf552e640 1 -- 172.21.15.43:0/505595555 >> 172.21.15.43:0/505595555 conn(0x7fbcf0077eb0 msgr2=0x7fbcf00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.710+0000 7fbcf552e640 1 -- 172.21.15.43:0/505595555 shutdown_connections 2024-04-21T08:46:07.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.710+0000 7fbcf552e640 1 -- 172.21.15.43:0/505595555 wait complete. 2024-04-21T08:46:07.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.711+0000 7fbcf552e640 1 Processor -- start 2024-04-21T08:46:07.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.711+0000 7fbcf552e640 1 -- start start 2024-04-21T08:46:07.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.711+0000 7fbcf552e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcf01010a0 0x7fbcf01986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.711+0000 7fbcf552e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcf0106170 0x7fbcf0198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.711+0000 7fbcf552e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcf019cf60 0x7fbcf019d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.711+0000 7fbcf552e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf01110e0 con 0x7fbcf0106170 2024-04-21T08:46:07.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.711+0000 7fbcf552e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf0110f60 con 0x7fbcf019cf60 2024-04-21T08:46:07.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.711+0000 7fbcf552e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf0111260 con 0x7fbcf01010a0 2024-04-21T08:46:07.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.711+0000 7fbce7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcf0106170 0x7fbcf0198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.711+0000 7fbce7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcf0106170 0x7fbcf0198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37162/0 (socket says 172.21.15.43:37162) 2024-04-21T08:46:07.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.711+0000 7fbce7fff640 1 -- 172.21.15.43:0/887517322 learned_addr learned my addr 172.21.15.43:0/887517322 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.711+0000 7fbceeffd640 1 --2- 172.21.15.43:0/887517322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcf01010a0 0x7fbcf01986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.712+0000 7fbcef7fe640 1 --2- 172.21.15.43:0/887517322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcf019cf60 0x7fbcf019d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.712+0000 7fbce7fff640 1 -- 172.21.15.43:0/887517322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcf01010a0 msgr2=0x7fbcf01986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.712+0000 7fbce7fff640 1 --2- 172.21.15.43:0/887517322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcf01010a0 0x7fbcf01986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.712+0000 7fbce7fff640 1 -- 172.21.15.43:0/887517322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcf019cf60 msgr2=0x7fbcf019d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.712+0000 7fbce7fff640 1 --2- 172.21.15.43:0/887517322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcf019cf60 0x7fbcf019d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.712+0000 7fbce7fff640 1 -- 172.21.15.43:0/887517322 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf019da50 con 0x7fbcf0106170 2024-04-21T08:46:07.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.712+0000 7fbceeffd640 1 --2- 172.21.15.43:0/887517322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcf01010a0 0x7fbcf01986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:07.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.712+0000 7fbcef7fe640 1 --2- 172.21.15.43:0/887517322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcf019cf60 0x7fbcf019d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:07.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.712+0000 7fbce7fff640 1 --2- 172.21.15.43:0/887517322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcf0106170 0x7fbcf0198be0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800e9c0 tx=0x7fbcd800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.712+0000 7fbcecff9640 1 -- 172.21.15.43:0/887517322 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd800cd80 con 0x7fbcf0106170 2024-04-21T08:46:07.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.712+0000 7fbcecff9640 1 -- 172.21.15.43:0/887517322 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcd80054d0 con 0x7fbcf0106170 2024-04-21T08:46:07.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.712+0000 7fbcf552e640 1 -- 172.21.15.43:0/887517322 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf019dd40 con 0x7fbcf0106170 2024-04-21T08:46:07.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.712+0000 7fbcecff9640 1 -- 172.21.15.43:0/887517322 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8010430 con 0x7fbcf0106170 2024-04-21T08:46:07.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.712+0000 7fbcf552e640 1 -- 172.21.15.43:0/887517322 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf01a5530 con 0x7fbcf0106170 2024-04-21T08:46:07.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:07 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2534645665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:07.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.714+0000 7fbcecff9640 1 -- 172.21.15.43:0/887517322 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbcd8004fd0 con 0x7fbcf0106170 2024-04-21T08:46:07.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.715+0000 7fbcf81f3740 1 Processor -- start 2024-04-21T08:46:07.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.715+0000 7fbcf81f3740 1 -- start start 2024-04-21T08:46:07.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.715+0000 7fbcf81f3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec7dc234d0 0x55ec7dc238b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.715+0000 7fbcf81f3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc23e80 0x55ec7dc2c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.715+0000 7fbcf81f3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec7dc2ca00 0x55ec7dc2ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.715+0000 7fbcf81f3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ec7db881f0 con 0x55ec7dc23e80 2024-04-21T08:46:07.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.715+0000 7fbcf81f3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ec7db254b0 con 0x55ec7dc2ca00 2024-04-21T08:46:07.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.715+0000 7fbcf81f3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ec7db841b0 con 0x55ec7dc234d0 2024-04-21T08:46:07.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.715+0000 7fbcecff9640 1 --2- 172.21.15.43:0/887517322 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbcdc0a3d20 0x7fbcdc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.715+0000 7fbceeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc23e80 0x55ec7dc2c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.716+0000 7fbceeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc23e80 0x55ec7dc2c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37176/0 (socket says 172.21.15.43:37176) 2024-04-21T08:46:07.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.716+0000 7fbceeffd640 1 -- 172.21.15.43:0/3312188710 learned_addr learned my addr 172.21.15.43:0/3312188710 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.716+0000 7fbcef7fe640 1 --2- 172.21.15.43:0/3312188710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec7dc234d0 0x55ec7dc238b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.716+0000 7fbce7fff640 1 --2- 172.21.15.43:0/3312188710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec7dc2ca00 0x55ec7dc2ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.716+0000 7fbcef7fe640 1 --2- 172.21.15.43:0/887517322 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbcdc0a3d20 0x7fbcdc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.716+0000 7fbcecff9640 1 -- 172.21.15.43:0/887517322 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7fbcd80d1130 con 0x7fbcf0106170 2024-04-21T08:46:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.716+0000 7fbceeffd640 1 -- 172.21.15.43:0/3312188710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec7dc234d0 msgr2=0x55ec7dc238b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.716+0000 7fbceeffd640 1 --2- 172.21.15.43:0/3312188710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec7dc234d0 0x55ec7dc238b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.716+0000 7fbceeffd640 1 -- 172.21.15.43:0/3312188710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec7dc2ca00 msgr2=0x55ec7dc2ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.716+0000 7fbceeffd640 1 --2- 172.21.15.43:0/3312188710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec7dc2ca00 0x55ec7dc2ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.716+0000 7fbceeffd640 1 -- 172.21.15.43:0/3312188710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec7db171b0 con 0x55ec7dc23e80 2024-04-21T08:46:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.716+0000 7fbce7fff640 1 --2- 172.21.15.43:0/3312188710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec7dc2ca00 0x55ec7dc2ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.716+0000 7fbceeffd640 1 --2- 172.21.15.43:0/3312188710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc23e80 0x55ec7dc2c430 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fbce000bd40 tx=0x7fbce000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ebe197cc9ff0861 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.716+0000 7fbcef7fe640 1 --2- 172.21.15.43:0/887517322 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbcdc0a3d20 0x7fbcdc0a61e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbce8004af0 tx=0x7fbce80095c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.716+0000 7fbce57fa640 1 -- 172.21.15.43:0/3312188710 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0015070 con 0x55ec7dc23e80 2024-04-21T08:46:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.716+0000 7fbce57fa640 1 -- 172.21.15.43:0/3312188710 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbce000be30 con 0x55ec7dc23e80 2024-04-21T08:46:07.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.716+0000 7fbce57fa640 1 -- 172.21.15.43:0/3312188710 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce00106e0 con 0x55ec7dc23e80 2024-04-21T08:46:07.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.717+0000 7fbcf81f3740 1 -- 172.21.15.43:0/3312188710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc23e80 msgr2=0x55ec7dc2c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.717+0000 7fbcf81f3740 1 --2- 172.21.15.43:0/3312188710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc23e80 0x55ec7dc2c430 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fbce000bd40 tx=0x7fbce000f730 comp rx=0 tx=0).stop 2024-04-21T08:46:07.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.717+0000 7fbcf81f3740 1 -- 172.21.15.43:0/3312188710 shutdown_connections 2024-04-21T08:46:07.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.717+0000 7fbcf81f3740 1 --2- 172.21.15.43:0/3312188710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec7dc2ca00 0x55ec7dc2ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.717+0000 7fbcf81f3740 1 --2- 172.21.15.43:0/3312188710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc23e80 0x55ec7dc2c430 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.717+0000 7fbcf81f3740 1 --2- 172.21.15.43:0/3312188710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec7dc234d0 0x55ec7dc238b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.717+0000 7fbcf81f3740 1 -- 172.21.15.43:0/3312188710 >> 172.21.15.43:0/3312188710 conn(0x55ec7dc1dba0 msgr2=0x55ec7dc1df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.717+0000 7fbcf81f3740 1 -- 172.21.15.43:0/3312188710 shutdown_connections 2024-04-21T08:46:07.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.717+0000 7fbcf81f3740 1 -- 172.21.15.43:0/3312188710 wait complete. 2024-04-21T08:46:07.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.717+0000 7fbcf81f3740 1 -- start start 2024-04-21T08:46:07.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.717+0000 7fbcf81f3740 1 Processor -- start 2024-04-21T08:46:07.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.718+0000 7fbcf81f3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec7dc234d0 0x55ec7dc2c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.718+0000 7fbcf81f3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc23e80 0x55ec7dc29660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.718+0000 7fbcf81f3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec7dc2ca00 0x55ec7dc29bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.718+0000 7fbcf81f3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ec7dc2f970 con 0x55ec7dc23e80 2024-04-21T08:46:07.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.718+0000 7fbcf81f3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ec7db41500 con 0x55ec7dc234d0 2024-04-21T08:46:07.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.718+0000 7fbcf81f3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ec7dc2faf0 con 0x55ec7dc2ca00 2024-04-21T08:46:07.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.718+0000 7fbcf81f3740 10 client.0 _ll_register_callbacks cb 0x55ec7db0fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:07.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.718+0000 7fbce7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc23e80 0x55ec7dc29660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.718+0000 7fbce7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc23e80 0x55ec7dc29660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37184/0 (socket says 172.21.15.43:37184) 2024-04-21T08:46:07.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.718+0000 7fbce7fff640 1 -- 172.21.15.43:0/168884406 learned_addr learned my addr 172.21.15.43:0/168884406 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.718+0000 7fbceeffd640 1 --2- 172.21.15.43:0/168884406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec7dc234d0 0x55ec7dc2c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.718+0000 7fbcef7fe640 1 --2- 172.21.15.43:0/168884406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec7dc2ca00 0x55ec7dc29bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.718+0000 7fbce7fff640 1 -- 172.21.15.43:0/168884406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec7dc2ca00 msgr2=0x55ec7dc29bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.718+0000 7fbce7fff640 1 --2- 172.21.15.43:0/168884406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec7dc2ca00 0x55ec7dc29bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.718+0000 7fbce7fff640 1 -- 172.21.15.43:0/168884406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec7dc234d0 msgr2=0x55ec7dc2c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.718+0000 7fbce7fff640 1 --2- 172.21.15.43:0/168884406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec7dc234d0 0x55ec7dc2c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.718+0000 7fbce7fff640 1 -- 172.21.15.43:0/168884406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec7d89a3f0 con 0x55ec7dc23e80 2024-04-21T08:46:07.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.718+0000 7fbcef7fe640 1 --2- 172.21.15.43:0/168884406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec7dc2ca00 0x55ec7dc29bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:07.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.718+0000 7fbce7fff640 1 --2- 172.21.15.43:0/168884406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc23e80 0x55ec7dc29660 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80ce460 tx=0x7fbcd800a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b2d9d5192cb26a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.718+0000 7fbcd67fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:07.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.718+0000 7fbcd67fc640 1 -- 172.21.15.43:0/168884406 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd809d350 con 0x55ec7dc23e80 2024-04-21T08:46:07.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.718+0000 7fbcd67fc640 1 -- 172.21.15.43:0/168884406 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7fbcd809af50 con 0x55ec7dc23e80 2024-04-21T08:46:07.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.718+0000 7fbcf81f3740 1 -- 172.21.15.43:0/168884406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbce000be30 con 0x55ec7dc23e80 2024-04-21T08:46:07.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.719+0000 7fbcd67fc640 1 -- 172.21.15.43:0/168884406 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbcd809cc50 con 0x55ec7dc23e80 2024-04-21T08:46:07.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.719+0000 7fbcf81f3740 10 client.18873 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:07.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.719+0000 7fbcf81f3740 1 -- 172.21.15.43:0/168884406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ec7d9647d0 con 0x55ec7dc23e80 2024-04-21T08:46:07.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.719+0000 7fbcd67fc640 1 -- 172.21.15.43:0/168884406 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbce000be30 con 0x55ec7dc23e80 2024-04-21T08:46:07.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.719+0000 7fbcd67fc640 1 -- 172.21.15.43:0/168884406 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbcd80bd0b0 con 0x55ec7dc23e80 2024-04-21T08:46:07.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.719+0000 7fbcf81f3740 10 client.18873 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:07.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.719+0000 7fbcf81f3740 10 client.18873 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:46:07.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.719+0000 7fbcf81f3740 20 client.18873 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:07.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.719+0000 7fbcf81f3740 1 --2- 172.21.15.43:0/168884406 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55ec7dc3cc30 0x55ec7dc3f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.719+0000 7fbcf81f3740 4 client.18873 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:07.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.719+0000 7fbcf81f3740 1 -- 172.21.15.43:0/168884406 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec7db27930 con 0x55ec7dc3cc30 2024-04-21T08:46:07.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.719+0000 7fbceeffd640 1 --2- 172.21.15.43:0/168884406 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55ec7dc3cc30 0x55ec7dc3f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.720+0000 7fbcd67fc640 10 client.18873 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:46:07.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.720+0000 7fbceeffd640 1 --2- 172.21.15.43:0/168884406 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55ec7dc3cc30 0x55ec7dc3f060 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=34a7e9d2a53107a8 server_cookie=2fa995f18ec372b in_seq=0 out_seq=0 2024-04-21T08:46:07.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.720+0000 7fbcd67fc640 1 -- 172.21.15.43:0/168884406 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ec7db27930 con 0x55ec7dc3cc30 2024-04-21T08:46:07.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.720+0000 7fbcd67fc640 10 client.18873 handle_command_reply: tid=0 2024-04-21T08:46:07.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.721+0000 7fbcf81f3740 1 client.18873 shutdown 2024-04-21T08:46:07.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.721+0000 7fbcf81f3740 1 -- 172.21.15.43:0/168884406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc23e80 msgr2=0x55ec7dc29660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.721+0000 7fbcf81f3740 1 --2- 172.21.15.43:0/168884406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc23e80 0x55ec7dc29660 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80ce460 tx=0x7fbcd800a440 comp rx=0 tx=0).stop 2024-04-21T08:46:07.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.721+0000 7fbcf81f3740 1 -- 172.21.15.43:0/168884406 shutdown_connections 2024-04-21T08:46:07.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.721+0000 7fbcf81f3740 1 --2- 172.21.15.43:0/168884406 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55ec7dc3cc30 0x55ec7dc3f060 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.721+0000 7fbcf81f3740 1 --2- 172.21.15.43:0/168884406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec7dc2ca00 0x55ec7dc29bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.721+0000 7fbcf81f3740 1 --2- 172.21.15.43:0/168884406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc23e80 0x55ec7dc29660 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.721+0000 7fbcf81f3740 1 --2- 172.21.15.43:0/168884406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec7dc234d0 0x55ec7dc2c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.721+0000 7fbcd67fc640 0 client.18873 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:46:07.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.721+0000 7fbcf81f3740 1 -- 172.21.15.43:0/168884406 >> 172.21.15.43:0/168884406 conn(0x55ec7dc1dba0 msgr2=0x55ec7dc1f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.721+0000 7fbcf81f3740 1 -- 172.21.15.43:0/168884406 shutdown_connections 2024-04-21T08:46:07.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.721+0000 7fbcf81f3740 1 -- 172.21.15.43:0/168884406 wait complete. 2024-04-21T08:46:07.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.721+0000 7fbcf81f3740 20 client.18873 trim_cache size 0 max 16384 2024-04-21T08:46:07.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.734+0000 7fbcf81f3740 1 Processor -- start 2024-04-21T08:46:07.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.734+0000 7fbcf81f3740 1 -- start start 2024-04-21T08:46:07.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.734+0000 7fbcf81f3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc234d0 0x55ec7dc38830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.734+0000 7fbcf81f3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec7dc23e80 0x55ec7dc2bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.734+0000 7fbcf81f3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec7dc2ca00 0x55ec7dc2c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.735+0000 7fbcf81f3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf01af6a0 con 0x55ec7dc234d0 2024-04-21T08:46:07.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.735+0000 7fbcf81f3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf01b11e0 con 0x55ec7dc2ca00 2024-04-21T08:46:07.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.735+0000 7fbcf81f3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf01afd70 con 0x55ec7dc23e80 2024-04-21T08:46:07.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.735+0000 7fbceeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc234d0 0x55ec7dc38830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.735+0000 7fbceeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc234d0 0x55ec7dc38830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37194/0 (socket says 172.21.15.43:37194) 2024-04-21T08:46:07.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.735+0000 7fbceeffd640 1 -- 172.21.15.43:0/3217222908 learned_addr learned my addr 172.21.15.43:0/3217222908 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.735+0000 7fbceeffd640 1 -- 172.21.15.43:0/3217222908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec7dc23e80 msgr2=0x55ec7dc2bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.735+0000 7fbce7fff640 1 --2- 172.21.15.43:0/3217222908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec7dc23e80 0x55ec7dc2bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.735+0000 7fbceeffd640 1 --2- 172.21.15.43:0/3217222908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec7dc23e80 0x55ec7dc2bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.735+0000 7fbcef7fe640 1 --2- 172.21.15.43:0/3217222908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec7dc2ca00 0x55ec7dc2c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.735+0000 7fbceeffd640 1 -- 172.21.15.43:0/3217222908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec7dc2ca00 msgr2=0x55ec7dc2c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.735+0000 7fbceeffd640 1 --2- 172.21.15.43:0/3217222908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec7dc2ca00 0x55ec7dc2c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.735+0000 7fbceeffd640 1 -- 172.21.15.43:0/3217222908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec7dc28610 con 0x55ec7dc234d0 2024-04-21T08:46:07.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.735+0000 7fbce7fff640 1 --2- 172.21.15.43:0/3217222908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec7dc23e80 0x55ec7dc2bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:07.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.735+0000 7fbcef7fe640 1 --2- 172.21.15.43:0/3217222908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec7dc2ca00 0x55ec7dc2c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:07.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.735+0000 7fbceeffd640 1 --2- 172.21.15.43:0/3217222908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc234d0 0x55ec7dc38830 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fbce000bd40 tx=0x7fbce000b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c9553f10d33278 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.735+0000 7fbce4ff9640 1 -- 172.21.15.43:0/3217222908 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0015040 con 0x55ec7dc234d0 2024-04-21T08:46:07.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.735+0000 7fbce4ff9640 1 -- 172.21.15.43:0/3217222908 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbce002d070 con 0x55ec7dc234d0 2024-04-21T08:46:07.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.735+0000 7fbce4ff9640 1 -- 172.21.15.43:0/3217222908 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0011a00 con 0x55ec7dc234d0 2024-04-21T08:46:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.736+0000 7fbcf81f3740 1 -- 172.21.15.43:0/3217222908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc234d0 msgr2=0x55ec7dc38830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.736+0000 7fbcf81f3740 1 --2- 172.21.15.43:0/3217222908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc234d0 0x55ec7dc38830 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fbce000bd40 tx=0x7fbce000b530 comp rx=0 tx=0).stop 2024-04-21T08:46:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.736+0000 7fbcf81f3740 1 -- 172.21.15.43:0/3217222908 shutdown_connections 2024-04-21T08:46:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.736+0000 7fbcf81f3740 1 --2- 172.21.15.43:0/3217222908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec7dc2ca00 0x55ec7dc2c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.736+0000 7fbcf81f3740 1 --2- 172.21.15.43:0/3217222908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec7dc23e80 0x55ec7dc2bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.736+0000 7fbcf81f3740 1 --2- 172.21.15.43:0/3217222908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc234d0 0x55ec7dc38830 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.736+0000 7fbcf81f3740 1 -- 172.21.15.43:0/3217222908 >> 172.21.15.43:0/3217222908 conn(0x55ec7dc1dba0 msgr2=0x55ec7dc1f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.736+0000 7fbcf81f3740 1 -- 172.21.15.43:0/3217222908 shutdown_connections 2024-04-21T08:46:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.736+0000 7fbcf81f3740 1 -- 172.21.15.43:0/3217222908 wait complete. 2024-04-21T08:46:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.736+0000 7fbcf81f3740 1 -- start start 2024-04-21T08:46:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.736+0000 7fbcf81f3740 1 Processor -- start 2024-04-21T08:46:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.737+0000 7fbcf81f3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec7dc234d0 0x55ec7dc30ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.737+0000 7fbcf81f3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec7dc23e80 0x55ec7dc30ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.737+0000 7fbcf81f3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc2ca00 0x55ec7dca21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.737+0000 7fbcf81f3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf01b0b10 con 0x55ec7dc2ca00 2024-04-21T08:46:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.737+0000 7fbcf81f3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf01b0440 con 0x55ec7dc23e80 2024-04-21T08:46:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.737+0000 7fbcf81f3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf01ad6a0 con 0x55ec7dc234d0 2024-04-21T08:46:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.737+0000 7fbcf81f3740 10 client.0 _ll_register_callbacks cb 0x7fbcd8098f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.737+0000 7fbcef7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc2ca00 0x55ec7dca21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.737+0000 7fbcef7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc2ca00 0x55ec7dca21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37198/0 (socket says 172.21.15.43:37198) 2024-04-21T08:46:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.737+0000 7fbcef7fe640 1 -- 172.21.15.43:0/1467471125 learned_addr learned my addr 172.21.15.43:0/1467471125 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.737+0000 7fbceeffd640 1 --2- 172.21.15.43:0/1467471125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec7dc234d0 0x55ec7dc30ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.737+0000 7fbce7fff640 1 --2- 172.21.15.43:0/1467471125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec7dc23e80 0x55ec7dc30ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.737+0000 7fbcef7fe640 1 -- 172.21.15.43:0/1467471125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec7dc234d0 msgr2=0x55ec7dc30ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.737+0000 7fbcef7fe640 1 --2- 172.21.15.43:0/1467471125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec7dc234d0 0x55ec7dc30ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.737+0000 7fbcef7fe640 1 -- 172.21.15.43:0/1467471125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec7dc23e80 msgr2=0x55ec7dc30ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.737+0000 7fbcef7fe640 1 --2- 172.21.15.43:0/1467471125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec7dc23e80 0x55ec7dc30ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.737+0000 7fbcef7fe640 1 -- 172.21.15.43:0/1467471125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec7dc318c0 con 0x55ec7dc2ca00 2024-04-21T08:46:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.737+0000 7fbceeffd640 1 --2- 172.21.15.43:0/1467471125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec7dc234d0 0x55ec7dc30ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.737+0000 7fbcef7fe640 1 --2- 172.21.15.43:0/1467471125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc2ca00 0x55ec7dca21c0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fbce802c630 tx=0x7fbce8005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649efb2d707e931c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.737+0000 7fbcd67fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.737+0000 7fbcd67fc640 1 -- 172.21.15.43:0/1467471125 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce8039040 con 0x55ec7dc2ca00 2024-04-21T08:46:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.737+0000 7fbcd67fc640 1 -- 172.21.15.43:0/1467471125 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7fbce8004450 con 0x55ec7dc2ca00 2024-04-21T08:46:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.737+0000 7fbcf81f3740 1 -- 172.21.15.43:0/1467471125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbce002d070 con 0x55ec7dc2ca00 2024-04-21T08:46:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.738+0000 7fbcd67fc640 1 -- 172.21.15.43:0/1467471125 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbce805b050 con 0x55ec7dc2ca00 2024-04-21T08:46:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.738+0000 7fbcf81f3740 10 client.18879 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.738+0000 7fbcf81f3740 1 -- 172.21.15.43:0/1467471125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ec7dca3770 con 0x55ec7dc2ca00 2024-04-21T08:46:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.738+0000 7fbcd67fc640 1 -- 172.21.15.43:0/1467471125 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbce002d070 con 0x55ec7dc2ca00 2024-04-21T08:46:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.738+0000 7fbcd67fc640 1 -- 172.21.15.43:0/1467471125 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbce804dc40 con 0x55ec7dc2ca00 2024-04-21T08:46:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.738+0000 7fbcf81f3740 10 client.18879 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.738+0000 7fbcf81f3740 10 client.18879 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:46:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.738+0000 7fbcf81f3740 20 client.18879 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.738+0000 7fbcf81f3740 1 --2- 172.21.15.43:0/1467471125 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55ec7dca3f10 0x55ec7dca6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.738+0000 7fbcf81f3740 4 client.18879 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.738+0000 7fbcf81f3740 1 -- 172.21.15.43:0/1467471125 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec7db95c10 con 0x55ec7dca3f10 2024-04-21T08:46:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.738+0000 7fbceeffd640 1 --2- 172.21.15.43:0/1467471125 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55ec7dca3f10 0x55ec7dca6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.739+0000 7fbcd67fc640 10 client.18879 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:46:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.739+0000 7fbceeffd640 1 --2- 172.21.15.43:0/1467471125 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55ec7dca3f10 0x55ec7dca6330 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=67622b851ee524 server_cookie=1afce547366e3a81 in_seq=0 out_seq=0 2024-04-21T08:46:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.739+0000 7fbcd67fc640 1 -- 172.21.15.43:0/1467471125 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec7db95c10 con 0x55ec7dca3f10 2024-04-21T08:46:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.739+0000 7fbcd67fc640 10 client.18879 handle_command_reply: tid=0 2024-04-21T08:46:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.739+0000 7fbcf81f3740 1 client.18879 shutdown 2024-04-21T08:46:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.739+0000 7fbcf81f3740 1 -- 172.21.15.43:0/1467471125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc2ca00 msgr2=0x55ec7dca21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.739+0000 7fbcf81f3740 1 --2- 172.21.15.43:0/1467471125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc2ca00 0x55ec7dca21c0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fbce802c630 tx=0x7fbce8005c50 comp rx=0 tx=0).stop 2024-04-21T08:46:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.739+0000 7fbcf81f3740 1 -- 172.21.15.43:0/1467471125 shutdown_connections 2024-04-21T08:46:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.739+0000 7fbcf81f3740 1 --2- 172.21.15.43:0/1467471125 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55ec7dca3f10 0x55ec7dca6330 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.739+0000 7fbcf81f3740 1 --2- 172.21.15.43:0/1467471125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec7dc2ca00 0x55ec7dca21c0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.739+0000 7fbcf81f3740 1 --2- 172.21.15.43:0/1467471125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec7dc23e80 0x55ec7dc30ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.739+0000 7fbcd67fc640 0 client.18879 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:46:07.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.739+0000 7fbcf81f3740 1 --2- 172.21.15.43:0/1467471125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ec7dc234d0 0x55ec7dc30ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.739+0000 7fbcf81f3740 1 -- 172.21.15.43:0/1467471125 >> 172.21.15.43:0/1467471125 conn(0x55ec7dc1dba0 msgr2=0x55ec7dc1eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.739+0000 7fbcf81f3740 1 -- 172.21.15.43:0/1467471125 shutdown_connections 2024-04-21T08:46:07.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.739+0000 7fbcf81f3740 1 -- 172.21.15.43:0/1467471125 wait complete. 2024-04-21T08:46:07.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.739+0000 7fbcf81f3740 20 client.18879 trim_cache size 0 max 16384 2024-04-21T08:46:07.745 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:46:07.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.740+0000 7fbcf552e640 1 -- 172.21.15.43:0/887517322 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbcdc0a3d20 msgr2=0x7fbcdc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.740+0000 7fbcf552e640 1 --2- 172.21.15.43:0/887517322 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbcdc0a3d20 0x7fbcdc0a61e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbce8004af0 tx=0x7fbce80095c0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.740+0000 7fbcf552e640 1 -- 172.21.15.43:0/887517322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcf0106170 msgr2=0x7fbcf0198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.740+0000 7fbcf552e640 1 --2- 172.21.15.43:0/887517322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcf0106170 0x7fbcf0198be0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800e9c0 tx=0x7fbcd800ee80 comp rx=0 tx=0).stop 2024-04-21T08:46:07.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.740+0000 7fbcf552e640 1 -- 172.21.15.43:0/887517322 shutdown_connections 2024-04-21T08:46:07.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.740+0000 7fbcf552e640 1 --2- 172.21.15.43:0/887517322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcf019cf60 0x7fbcf019d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.740+0000 7fbcf552e640 1 --2- 172.21.15.43:0/887517322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcf0106170 0x7fbcf0198be0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.740+0000 7fbcf552e640 1 --2- 172.21.15.43:0/887517322 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbcdc0a3d20 0x7fbcdc0a61e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.740+0000 7fbcf552e640 1 --2- 172.21.15.43:0/887517322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcf01010a0 0x7fbcf01986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.740+0000 7fbcf552e640 1 -- 172.21.15.43:0/887517322 >> 172.21.15.43:0/887517322 conn(0x7fbcf0077eb0 msgr2=0x7fbcf006a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.740+0000 7fbcf552e640 1 -- 172.21.15.43:0/887517322 shutdown_connections 2024-04-21T08:46:07.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.740+0000 7fbcf552e640 1 -- 172.21.15.43:0/887517322 wait complete. 2024-04-21T08:46:07.750 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:46:07.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.855+0000 7f95a7281640 1 -- 172.21.15.43:0/575277435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95a006b1f0 msgr2=0x7f95a0106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.855+0000 7f95a7281640 1 --2- 172.21.15.43:0/575277435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95a006b1f0 0x7f95a0106b90 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f9590009900 tx=0x7f959002bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:07.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.855+0000 7f95a7281640 1 -- 172.21.15.43:0/575277435 shutdown_connections 2024-04-21T08:46:07.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.855+0000 7f95a7281640 1 --2- 172.21.15.43:0/575277435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95a0107270 0x7f95a010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.855+0000 7f95a7281640 1 --2- 172.21.15.43:0/575277435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95a006b1f0 0x7f95a0106b90 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.855+0000 7f95a7281640 1 --2- 172.21.15.43:0/575277435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95a006a840 0x7f95a006ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.855+0000 7f95a7281640 1 -- 172.21.15.43:0/575277435 >> 172.21.15.43:0/575277435 conn(0x7f95a0077eb0 msgr2=0x7f95a00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.855+0000 7f95a7281640 1 -- 172.21.15.43:0/575277435 shutdown_connections 2024-04-21T08:46:07.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.856+0000 7f95a7281640 1 -- 172.21.15.43:0/575277435 wait complete. 2024-04-21T08:46:07.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.856+0000 7f95a7281640 1 Processor -- start 2024-04-21T08:46:07.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.856+0000 7f95a7281640 1 -- start start 2024-04-21T08:46:07.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.856+0000 7f95a7281640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95a006a840 0x7f95a01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.856+0000 7f95a7281640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95a0107270 0x7f95a0198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.856+0000 7f95a7281640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95a019cf90 0x7f95a019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.856+0000 7f95a7281640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f95a0112190 con 0x7f95a019cf90 2024-04-21T08:46:07.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.856+0000 7f95a7281640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f95a0112010 con 0x7f95a006a840 2024-04-21T08:46:07.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.856+0000 7f95a7281640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f95a0112310 con 0x7f95a0107270 2024-04-21T08:46:07.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.856+0000 7f95a581e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95a019cf90 0x7f95a019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.856+0000 7f95a581e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95a019cf90 0x7f95a019d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37218/0 (socket says 172.21.15.43:37218) 2024-04-21T08:46:07.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.856+0000 7f95a581e640 1 -- 172.21.15.43:0/993933556 learned_addr learned my addr 172.21.15.43:0/993933556 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.857+0000 7f95a481c640 1 --2- 172.21.15.43:0/993933556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95a0107270 0x7f95a0198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.857+0000 7f95a501d640 1 --2- 172.21.15.43:0/993933556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95a006a840 0x7f95a01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.857+0000 7f95a581e640 1 -- 172.21.15.43:0/993933556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95a0107270 msgr2=0x7f95a0198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.857+0000 7f95a581e640 1 --2- 172.21.15.43:0/993933556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95a0107270 0x7f95a0198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.857+0000 7f95a581e640 1 -- 172.21.15.43:0/993933556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95a006a840 msgr2=0x7f95a01986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.857+0000 7f95a581e640 1 --2- 172.21.15.43:0/993933556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95a006a840 0x7f95a01986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.857+0000 7f95a581e640 1 -- 172.21.15.43:0/993933556 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a019da80 con 0x7f95a019cf90 2024-04-21T08:46:07.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.857+0000 7f95a581e640 1 --2- 172.21.15.43:0/993933556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95a019cf90 0x7f95a019d440 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f959800eb10 tx=0x7f959800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.857+0000 7f95967fc640 1 -- 172.21.15.43:0/993933556 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598004b20 con 0x7f95a019cf90 2024-04-21T08:46:07.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.857+0000 7f95967fc640 1 -- 172.21.15.43:0/993933556 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9598004cc0 con 0x7f95a019cf90 2024-04-21T08:46:07.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.857+0000 7f95a7281640 1 -- 172.21.15.43:0/993933556 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a019dd70 con 0x7f95a019cf90 2024-04-21T08:46:07.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.857+0000 7f95967fc640 1 -- 172.21.15.43:0/993933556 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598010670 con 0x7f95a019cf90 2024-04-21T08:46:07.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.857+0000 7f95a7281640 1 -- 172.21.15.43:0/993933556 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a01a5530 con 0x7f95a019cf90 2024-04-21T08:46:07.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.860+0000 7f95967fc640 1 -- 172.21.15.43:0/993933556 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9598005020 con 0x7f95a019cf90 2024-04-21T08:46:07.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.860+0000 7f95a9e70740 1 Processor -- start 2024-04-21T08:46:07.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.860+0000 7f95a9e70740 1 -- start start 2024-04-21T08:46:07.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.860+0000 7f95a9e70740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562efe5fe330 0x562efe5fe710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.860+0000 7f95a9e70740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562efe5fece0 0x562efe607820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.860+0000 7f95a9e70740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe607e40 0x562efe6082c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.860+0000 7f95a9e70740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562efe5631f0 con 0x562efe607e40 2024-04-21T08:46:07.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.860+0000 7f95a9e70740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562efe5004b0 con 0x562efe5fece0 2024-04-21T08:46:07.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.860+0000 7f95a9e70740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562efe55f1b0 con 0x562efe5fe330 2024-04-21T08:46:07.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.860+0000 7f95a481c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe607e40 0x562efe6082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.860+0000 7f95a481c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe607e40 0x562efe6082c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37220/0 (socket says 172.21.15.43:37220) 2024-04-21T08:46:07.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.860+0000 7f95a481c640 1 -- 172.21.15.43:0/3224966125 learned_addr learned my addr 172.21.15.43:0/3224966125 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.860+0000 7f95a501d640 1 --2- 172.21.15.43:0/3224966125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562efe5fece0 0x562efe607820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.860+0000 7f95a481c640 1 --2- 172.21.15.43:0/3224966125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562efe5fe330 0x562efe5fe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.860+0000 7f95967fc640 1 --2- 172.21.15.43:0/993933556 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f958c0a3da0 0x7f958c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.860+0000 7f95a581e640 1 --2- 172.21.15.43:0/993933556 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f958c0a3da0 0x7f958c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.860+0000 7f95a481c640 1 -- 172.21.15.43:0/3224966125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562efe5fe330 msgr2=0x562efe5fe710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.860+0000 7f95a481c640 1 --2- 172.21.15.43:0/3224966125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562efe5fe330 0x562efe5fe710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.860+0000 7f95a481c640 1 -- 172.21.15.43:0/3224966125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562efe5fece0 msgr2=0x562efe607820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.860+0000 7f95a481c640 1 --2- 172.21.15.43:0/3224966125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562efe5fece0 0x562efe607820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.861+0000 7f95a481c640 1 -- 172.21.15.43:0/3224966125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562efe4f21b0 con 0x562efe607e40 2024-04-21T08:46:07.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.861+0000 7f95a501d640 1 --2- 172.21.15.43:0/3224966125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562efe5fece0 0x562efe607820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:07.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.861+0000 7f95967fc640 1 -- 172.21.15.43:0/993933556 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7f95980d1120 con 0x7f95a019cf90 2024-04-21T08:46:07.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.861+0000 7f95a481c640 1 --2- 172.21.15.43:0/3224966125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe607e40 0x562efe6082c0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f95900096c0 tx=0x7f9590005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76ba0e41873f73d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.861+0000 7f95a581e640 1 --2- 172.21.15.43:0/993933556 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f958c0a3da0 0x7f958c0a6260 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9598062650 tx=0x7f95980bd040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.861+0000 7f958b7fe640 1 -- 172.21.15.43:0/3224966125 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590047020 con 0x562efe607e40 2024-04-21T08:46:07.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.861+0000 7f958b7fe640 1 -- 172.21.15.43:0/3224966125 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f959002cc90 con 0x562efe607e40 2024-04-21T08:46:07.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.861+0000 7f958b7fe640 1 -- 172.21.15.43:0/3224966125 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95900046c0 con 0x562efe607e40 2024-04-21T08:46:07.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.862+0000 7f95a9e70740 1 -- 172.21.15.43:0/3224966125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe607e40 msgr2=0x562efe6082c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.862+0000 7f95a9e70740 1 --2- 172.21.15.43:0/3224966125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe607e40 0x562efe6082c0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f95900096c0 tx=0x7f9590005e80 comp rx=0 tx=0).stop 2024-04-21T08:46:07.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.862+0000 7f95a9e70740 1 -- 172.21.15.43:0/3224966125 shutdown_connections 2024-04-21T08:46:07.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.862+0000 7f95a9e70740 1 --2- 172.21.15.43:0/3224966125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe607e40 0x562efe6082c0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.862+0000 7f95a9e70740 1 --2- 172.21.15.43:0/3224966125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562efe5fece0 0x562efe607820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.862+0000 7f95a9e70740 1 --2- 172.21.15.43:0/3224966125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562efe5fe330 0x562efe5fe710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.862+0000 7f95a9e70740 1 -- 172.21.15.43:0/3224966125 >> 172.21.15.43:0/3224966125 conn(0x562efe5f8ba0 msgr2=0x562efe5f8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.862+0000 7f95a9e70740 1 -- 172.21.15.43:0/3224966125 shutdown_connections 2024-04-21T08:46:07.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.862+0000 7f95a9e70740 1 -- 172.21.15.43:0/3224966125 wait complete. 2024-04-21T08:46:07.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.862+0000 7f95a9e70740 1 -- start start 2024-04-21T08:46:07.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.862+0000 7f95a9e70740 1 Processor -- start 2024-04-21T08:46:07.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.863+0000 7f95a9e70740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe5fe330 0x562efe60e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.863+0000 7f95a9e70740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562efe5fece0 0x562efe60b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.863+0000 7f95a9e70740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562efe607e40 0x562efe60bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.863+0000 7f95a9e70740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562efe60ed60 con 0x562efe5fe330 2024-04-21T08:46:07.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.863+0000 7f95a9e70740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562efe51c500 con 0x562efe607e40 2024-04-21T08:46:07.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.863+0000 7f95a9e70740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562efe60eee0 con 0x562efe5fece0 2024-04-21T08:46:07.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.863+0000 7f95a9e70740 10 client.0 _ll_register_callbacks cb 0x562efe4eab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:07.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.863+0000 7f95a481c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe5fe330 0x562efe60e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.863+0000 7f95a481c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe5fe330 0x562efe60e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37224/0 (socket says 172.21.15.43:37224) 2024-04-21T08:46:07.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.863+0000 7f95a481c640 1 -- 172.21.15.43:0/1368020652 learned_addr learned my addr 172.21.15.43:0/1368020652 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.863+0000 7f95a501d640 1 --2- 172.21.15.43:0/1368020652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562efe5fece0 0x562efe60b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.863+0000 7f95a481c640 1 -- 172.21.15.43:0/1368020652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562efe5fece0 msgr2=0x562efe60b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.863+0000 7f95a481c640 1 --2- 172.21.15.43:0/1368020652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562efe5fece0 0x562efe60b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.863+0000 7f95a481c640 1 -- 172.21.15.43:0/1368020652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562efe607e40 msgr2=0x562efe60bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.863+0000 7f95a481c640 1 --2- 172.21.15.43:0/1368020652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562efe607e40 0x562efe60bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.863+0000 7f95a481c640 1 -- 172.21.15.43:0/1368020652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562efe2753f0 con 0x562efe5fe330 2024-04-21T08:46:07.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.863+0000 7f95a481c640 1 --2- 172.21.15.43:0/1368020652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe5fe330 0x562efe60e2b0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f9590036040 tx=0x7f9590009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b4a2fc23a06068 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.863+0000 7f9588ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:07.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.863+0000 7f9588ff9640 1 -- 172.21.15.43:0/1368020652 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590052020 con 0x562efe5fe330 2024-04-21T08:46:07.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.863+0000 7f9588ff9640 1 -- 172.21.15.43:0/1368020652 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7f9590003ac0 con 0x562efe5fe330 2024-04-21T08:46:07.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.863+0000 7f95a9e70740 1 -- 172.21.15.43:0/1368020652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f959002cc90 con 0x562efe5fe330 2024-04-21T08:46:07.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.863+0000 7f9588ff9640 1 -- 172.21.15.43:0/1368020652 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f959005d050 con 0x562efe5fe330 2024-04-21T08:46:07.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.863+0000 7f95a9e70740 10 client.18891 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:07.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.863+0000 7f95a9e70740 1 -- 172.21.15.43:0/1368020652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562efe33f7d0 con 0x562efe5fe330 2024-04-21T08:46:07.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.864+0000 7f9588ff9640 1 -- 172.21.15.43:0/1368020652 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f959002cc90 con 0x562efe5fe330 2024-04-21T08:46:07.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.864+0000 7f9588ff9640 1 -- 172.21.15.43:0/1368020652 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f959004c450 con 0x562efe5fe330 2024-04-21T08:46:07.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.864+0000 7f95a9e70740 10 client.18891 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:07.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.864+0000 7f95a9e70740 10 client.18891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.864+0000 7f95a9e70740 20 client.18891 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.864+0000 7f95a9e70740 1 --2- 172.21.15.43:0/1368020652 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562efe61bf10 0x562efe61e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.864+0000 7f95a9e70740 4 client.18891 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.864+0000 7f95a9e70740 1 -- 172.21.15.43:0/1368020652 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562efe502930 con 0x562efe61bf10 2024-04-21T08:46:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.864+0000 7f95a501d640 1 --2- 172.21.15.43:0/1368020652 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562efe61bf10 0x562efe61e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.864+0000 7f9588ff9640 10 client.18891 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.864+0000 7f95a501d640 1 --2- 172.21.15.43:0/1368020652 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562efe61bf10 0x562efe61e300 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7e103abcbefe41e server_cookie=bdfdaf0cbbeb10d5 in_seq=0 out_seq=0 2024-04-21T08:46:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.865+0000 7f9588ff9640 1 -- 172.21.15.43:0/1368020652 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562efe502930 con 0x562efe61bf10 2024-04-21T08:46:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.865+0000 7f9588ff9640 10 client.18891 handle_command_reply: tid=0 2024-04-21T08:46:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.865+0000 7f95a9e70740 1 client.18891 shutdown 2024-04-21T08:46:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.865+0000 7f95a9e70740 1 -- 172.21.15.43:0/1368020652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe5fe330 msgr2=0x562efe60e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.865+0000 7f95a9e70740 1 --2- 172.21.15.43:0/1368020652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe5fe330 0x562efe60e2b0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f9590036040 tx=0x7f9590009230 comp rx=0 tx=0).stop 2024-04-21T08:46:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.865+0000 7f95a9e70740 1 -- 172.21.15.43:0/1368020652 shutdown_connections 2024-04-21T08:46:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.865+0000 7f95a9e70740 1 --2- 172.21.15.43:0/1368020652 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562efe61bf10 0x562efe61e300 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.865+0000 7f95a9e70740 1 --2- 172.21.15.43:0/1368020652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562efe607e40 0x562efe60bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.865+0000 7f95a9e70740 1 --2- 172.21.15.43:0/1368020652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562efe5fece0 0x562efe60b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.865+0000 7f95a9e70740 1 --2- 172.21.15.43:0/1368020652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe5fe330 0x562efe60e2b0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.865+0000 7f9588ff9640 0 client.18891 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:07.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.865+0000 7f95a9e70740 1 -- 172.21.15.43:0/1368020652 >> 172.21.15.43:0/1368020652 conn(0x562efe5f8ba0 msgr2=0x562efe5fa8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.865+0000 7f95a9e70740 1 -- 172.21.15.43:0/1368020652 shutdown_connections 2024-04-21T08:46:07.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.865+0000 7f95a9e70740 1 -- 172.21.15.43:0/1368020652 wait complete. 2024-04-21T08:46:07.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.866+0000 7f95a9e70740 20 client.18891 trim_cache size 0 max 16384 2024-04-21T08:46:07.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.879+0000 7f95a9e70740 1 Processor -- start 2024-04-21T08:46:07.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.879+0000 7f95a9e70740 1 -- start start 2024-04-21T08:46:07.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.879+0000 7f95a9e70740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562efe5fe330 0x562efe60dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.879+0000 7f95a9e70740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562efe5fece0 0x562efe60e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.879+0000 7f95a9e70740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe607e40 0x562efe5f9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.879+0000 7f95a9e70740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f95a01ad340 con 0x562efe607e40 2024-04-21T08:46:07.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.879+0000 7f95a9e70740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f95a01aee80 con 0x562efe5fece0 2024-04-21T08:46:07.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.879+0000 7f95a9e70740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f95a01ada10 con 0x562efe5fe330 2024-04-21T08:46:07.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.879+0000 7f95a481c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe607e40 0x562efe5f9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.879+0000 7f95a481c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe607e40 0x562efe5f9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37234/0 (socket says 172.21.15.43:37234) 2024-04-21T08:46:07.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.879+0000 7f95a481c640 1 -- 172.21.15.43:0/552125490 learned_addr learned my addr 172.21.15.43:0/552125490 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.879+0000 7f95a481c640 1 --2- 172.21.15.43:0/552125490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562efe5fe330 0x562efe60dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.879+0000 7f95a501d640 1 --2- 172.21.15.43:0/552125490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562efe5fece0 0x562efe60e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.879+0000 7f95a481c640 1 -- 172.21.15.43:0/552125490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562efe5fe330 msgr2=0x562efe60dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.879+0000 7f95a481c640 1 --2- 172.21.15.43:0/552125490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562efe5fe330 0x562efe60dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.879+0000 7f95a481c640 1 -- 172.21.15.43:0/552125490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562efe5fece0 msgr2=0x562efe60e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.879+0000 7f95a481c640 1 --2- 172.21.15.43:0/552125490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562efe5fece0 0x562efe60e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.880+0000 7f95a481c640 1 -- 172.21.15.43:0/552125490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562efe5f9930 con 0x562efe607e40 2024-04-21T08:46:07.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.880+0000 7f95a481c640 1 --2- 172.21.15.43:0/552125490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe607e40 0x562efe5f9ee0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f9590005ec0 tx=0x7f9590045040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401852b6795c52f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.880+0000 7f958affd640 1 -- 172.21.15.43:0/552125490 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590052070 con 0x562efe607e40 2024-04-21T08:46:07.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.880+0000 7f958affd640 1 -- 172.21.15.43:0/552125490 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9590004150 con 0x562efe607e40 2024-04-21T08:46:07.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.880+0000 7f958affd640 1 -- 172.21.15.43:0/552125490 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590047030 con 0x562efe607e40 2024-04-21T08:46:07.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.880+0000 7f95a9e70740 1 -- 172.21.15.43:0/552125490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe607e40 msgr2=0x562efe5f9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.880+0000 7f95a9e70740 1 --2- 172.21.15.43:0/552125490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe607e40 0x562efe5f9ee0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f9590005ec0 tx=0x7f9590045040 comp rx=0 tx=0).stop 2024-04-21T08:46:07.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.880+0000 7f95a9e70740 1 -- 172.21.15.43:0/552125490 shutdown_connections 2024-04-21T08:46:07.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.880+0000 7f95a9e70740 1 --2- 172.21.15.43:0/552125490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe607e40 0x562efe5f9ee0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.880+0000 7f95a9e70740 1 --2- 172.21.15.43:0/552125490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562efe5fece0 0x562efe60e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.880+0000 7f95a9e70740 1 --2- 172.21.15.43:0/552125490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562efe5fe330 0x562efe60dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.880+0000 7f95a9e70740 1 -- 172.21.15.43:0/552125490 >> 172.21.15.43:0/552125490 conn(0x562efe5f8ba0 msgr2=0x562efe617a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.880+0000 7f95a9e70740 1 -- 172.21.15.43:0/552125490 shutdown_connections 2024-04-21T08:46:07.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.880+0000 7f95a9e70740 1 -- 172.21.15.43:0/552125490 wait complete. 2024-04-21T08:46:07.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.881+0000 7f95a9e70740 1 -- start start 2024-04-21T08:46:07.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.881+0000 7f95a9e70740 1 Processor -- start 2024-04-21T08:46:07.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.881+0000 7f95a9e70740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562efe5fe330 0x562efe6216f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.881+0000 7f95a9e70740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562efe5fece0 0x562efe61ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.881+0000 7f95a9e70740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe607e40 0x562efe61efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.881+0000 7f95a9e70740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f95a01ae7b0 con 0x562efe607e40 2024-04-21T08:46:07.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.881+0000 7f95a9e70740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f95a01ae0e0 con 0x562efe5fe330 2024-04-21T08:46:07.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.881+0000 7f95a9e70740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f95a01ab340 con 0x562efe5fece0 2024-04-21T08:46:07.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.881+0000 7f95a9e70740 10 client.0 _ll_register_callbacks cb 0x7f9590005550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:07.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.881+0000 7f95a481c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe607e40 0x562efe61efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.881+0000 7f95a481c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe607e40 0x562efe61efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37242/0 (socket says 172.21.15.43:37242) 2024-04-21T08:46:07.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.881+0000 7f95a481c640 1 -- 172.21.15.43:0/2544910041 learned_addr learned my addr 172.21.15.43:0/2544910041 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:07.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.881+0000 7f95a501d640 1 --2- 172.21.15.43:0/2544910041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562efe5fece0 0x562efe61ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.881+0000 7f95a481c640 1 --2- 172.21.15.43:0/2544910041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562efe5fe330 0x562efe6216f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.882+0000 7f95a481c640 1 -- 172.21.15.43:0/2544910041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562efe5fece0 msgr2=0x562efe61ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.882+0000 7f95a481c640 1 --2- 172.21.15.43:0/2544910041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562efe5fece0 0x562efe61ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.882+0000 7f95a481c640 1 -- 172.21.15.43:0/2544910041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562efe5fe330 msgr2=0x562efe6216f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.882+0000 7f95a481c640 1 --2- 172.21.15.43:0/2544910041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562efe5fe330 0x562efe6216f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.882+0000 7f95a481c640 1 -- 172.21.15.43:0/2544910041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562efe61f7a0 con 0x562efe607e40 2024-04-21T08:46:07.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.882+0000 7f95a481c640 1 --2- 172.21.15.43:0/2544910041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe607e40 0x562efe61efc0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f95900096c0 tx=0x7f9590005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ca108866df4d12 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.882+0000 7f9588ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:07.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.882+0000 7f9588ff9640 1 -- 172.21.15.43:0/2544910041 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590047040 con 0x562efe607e40 2024-04-21T08:46:07.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.882+0000 7f95a9e70740 1 -- 172.21.15.43:0/2544910041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9590004150 con 0x562efe607e40 2024-04-21T08:46:07.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.882+0000 7f9588ff9640 1 -- 172.21.15.43:0/2544910041 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7f95900038c0 con 0x562efe607e40 2024-04-21T08:46:07.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.882+0000 7f9588ff9640 1 -- 172.21.15.43:0/2544910041 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9590052040 con 0x562efe607e40 2024-04-21T08:46:07.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.882+0000 7f95a9e70740 10 client.18897 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:07.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.882+0000 7f95a9e70740 1 -- 172.21.15.43:0/2544910041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562efe682100 con 0x562efe607e40 2024-04-21T08:46:07.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.882+0000 7f9588ff9640 1 -- 172.21.15.43:0/2544910041 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9590004150 con 0x562efe607e40 2024-04-21T08:46:07.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.882+0000 7f9588ff9640 1 -- 172.21.15.43:0/2544910041 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9590041aa0 con 0x562efe607e40 2024-04-21T08:46:07.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.882+0000 7f95a9e70740 10 client.18897 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:07.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.882+0000 7f95a9e70740 10 client.18897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:07.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.882+0000 7f95a9e70740 20 client.18897 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:07.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.883+0000 7f95a9e70740 1 --2- 172.21.15.43:0/2544910041 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562efe682810 0x562efe684c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.883+0000 7f95a9e70740 4 client.18897 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:07.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.883+0000 7f95a9e70740 1 -- 172.21.15.43:0/2544910041 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562efe570c10 con 0x562efe682810 2024-04-21T08:46:07.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.883+0000 7f95a501d640 1 --2- 172.21.15.43:0/2544910041 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562efe682810 0x562efe684c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.883+0000 7f95a501d640 1 --2- 172.21.15.43:0/2544910041 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562efe682810 0x562efe684c00 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b5f88b28c92a1e4 server_cookie=525eb5f3467d8dfa in_seq=0 out_seq=0 2024-04-21T08:46:07.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.883+0000 7f9588ff9640 10 client.18897 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:07.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.883+0000 7f9588ff9640 1 -- 172.21.15.43:0/2544910041 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562efe570c10 con 0x562efe682810 2024-04-21T08:46:07.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.883+0000 7f9588ff9640 10 client.18897 handle_command_reply: tid=0 2024-04-21T08:46:07.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.883+0000 7f95a9e70740 1 client.18897 shutdown 2024-04-21T08:46:07.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.884+0000 7f95a9e70740 1 -- 172.21.15.43:0/2544910041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe607e40 msgr2=0x562efe61efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.884+0000 7f95a9e70740 1 --2- 172.21.15.43:0/2544910041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe607e40 0x562efe61efc0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f95900096c0 tx=0x7f9590005fd0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.884+0000 7f95a9e70740 1 -- 172.21.15.43:0/2544910041 shutdown_connections 2024-04-21T08:46:07.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.884+0000 7f95a9e70740 1 --2- 172.21.15.43:0/2544910041 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562efe682810 0x562efe684c00 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.884+0000 7f95a9e70740 1 --2- 172.21.15.43:0/2544910041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562efe607e40 0x562efe61efc0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.884+0000 7f9588ff9640 0 client.18897 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:07.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.884+0000 7f95a9e70740 1 --2- 172.21.15.43:0/2544910041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562efe5fece0 0x562efe61ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.884+0000 7f95a9e70740 1 --2- 172.21.15.43:0/2544910041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562efe5fe330 0x562efe6216f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.884+0000 7f95a9e70740 1 -- 172.21.15.43:0/2544910041 >> 172.21.15.43:0/2544910041 conn(0x562efe5f8ba0 msgr2=0x562efe5f7370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.884+0000 7f95a9e70740 1 -- 172.21.15.43:0/2544910041 shutdown_connections 2024-04-21T08:46:07.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.884+0000 7f95a9e70740 1 -- 172.21.15.43:0/2544910041 wait complete. 2024-04-21T08:46:07.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.884+0000 7f95a9e70740 20 client.18897 trim_cache size 0 max 16384 2024-04-21T08:46:07.890 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:46:07.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.884+0000 7f95a7281640 1 -- 172.21.15.43:0/993933556 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f958c0a3da0 msgr2=0x7f958c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.884+0000 7f95a7281640 1 --2- 172.21.15.43:0/993933556 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f958c0a3da0 0x7f958c0a6260 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9598062650 tx=0x7f95980bd040 comp rx=0 tx=0).stop 2024-04-21T08:46:07.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.884+0000 7f95a7281640 1 -- 172.21.15.43:0/993933556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95a019cf90 msgr2=0x7f95a019d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.884+0000 7f95a7281640 1 --2- 172.21.15.43:0/993933556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95a019cf90 0x7f95a019d440 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f959800eb10 tx=0x7f959800c2d0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.884+0000 7f95a7281640 1 -- 172.21.15.43:0/993933556 shutdown_connections 2024-04-21T08:46:07.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.884+0000 7f95a7281640 1 --2- 172.21.15.43:0/993933556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95a019cf90 0x7f95a019d440 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.884+0000 7f95a7281640 1 --2- 172.21.15.43:0/993933556 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f958c0a3da0 0x7f958c0a6260 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.884+0000 7f95a7281640 1 --2- 172.21.15.43:0/993933556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95a0107270 0x7f95a0198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.884+0000 7f95a7281640 1 --2- 172.21.15.43:0/993933556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95a006a840 0x7f95a01986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.884+0000 7f95a7281640 1 -- 172.21.15.43:0/993933556 >> 172.21.15.43:0/993933556 conn(0x7f95a0077eb0 msgr2=0x7f95a0108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.884+0000 7f95a7281640 1 -- 172.21.15.43:0/993933556 shutdown_connections 2024-04-21T08:46:07.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:07.884+0000 7f95a7281640 1 -- 172.21.15.43:0/993933556 wait complete. 2024-04-21T08:46:08.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:08 smithi112 ceph-mon[31085]: pgmap v220: 129 pgs: 129 active+clean; 1.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 56 KiB/s wr, 33 op/s 2024-04-21T08:46:08.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:08 smithi043 ceph-mon[27978]: pgmap v220: 129 pgs: 129 active+clean; 1.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 56 KiB/s wr, 33 op/s 2024-04-21T08:46:08.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:08 smithi175 ceph-mon[31920]: pgmap v220: 129 pgs: 129 active+clean; 1.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 56 KiB/s wr, 33 op/s 2024-04-21T08:46:08.895 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:46:08.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:46:08.896 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:46:09.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.015+0000 7fe1918fd640 1 -- 172.21.15.43:0/3611719774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18c1010a0 msgr2=0x7fe18c101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.015+0000 7fe1918fd640 1 --2- 172.21.15.43:0/3611719774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18c1010a0 0x7fe18c101480 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009900 tx=0x7fe17c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:09.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.015+0000 7fe1918fd640 1 -- 172.21.15.43:0/3611719774 shutdown_connections 2024-04-21T08:46:09.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.015+0000 7fe1918fd640 1 --2- 172.21.15.43:0/3611719774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe18c106170 0x7fe18c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.015+0000 7fe1918fd640 1 --2- 172.21.15.43:0/3611719774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe18c101a50 0x7fe18c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.015+0000 7fe1918fd640 1 --2- 172.21.15.43:0/3611719774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18c1010a0 0x7fe18c101480 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.015+0000 7fe1918fd640 1 -- 172.21.15.43:0/3611719774 >> 172.21.15.43:0/3611719774 conn(0x7fe18c077eb0 msgr2=0x7fe18c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.015+0000 7fe1918fd640 1 -- 172.21.15.43:0/3611719774 shutdown_connections 2024-04-21T08:46:09.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.015+0000 7fe1918fd640 1 -- 172.21.15.43:0/3611719774 wait complete. 2024-04-21T08:46:09.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.016+0000 7fe1918fd640 1 Processor -- start 2024-04-21T08:46:09.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.016+0000 7fe1918fd640 1 -- start start 2024-04-21T08:46:09.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.016+0000 7fe1918fd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe18c1010a0 0x7fe18c10e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.016+0000 7fe1918fd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe18c106170 0x7fe18c10ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.016+0000 7fe1918fd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18c10f150 0x7fe18c1a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.016+0000 7fe1918fd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c1121f0 con 0x7fe18c10f150 2024-04-21T08:46:09.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.016+0000 7fe1918fd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c112070 con 0x7fe18c106170 2024-04-21T08:46:09.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.016+0000 7fe1918fd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c112370 con 0x7fe18c1010a0 2024-04-21T08:46:09.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.017+0000 7fe18b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18c10f150 0x7fe18c1a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.017+0000 7fe18b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18c10f150 0x7fe18c1a68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37268/0 (socket says 172.21.15.43:37268) 2024-04-21T08:46:09.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.017+0000 7fe18b7fe640 1 -- 172.21.15.43:0/2672815330 learned_addr learned my addr 172.21.15.43:0/2672815330 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.017+0000 7fe18affd640 1 --2- 172.21.15.43:0/2672815330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe18c1010a0 0x7fe18c10e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.017+0000 7fe18a7fc640 1 --2- 172.21.15.43:0/2672815330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe18c106170 0x7fe18c10ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.017+0000 7fe18b7fe640 1 -- 172.21.15.43:0/2672815330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe18c1010a0 msgr2=0x7fe18c10e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.017+0000 7fe18b7fe640 1 --2- 172.21.15.43:0/2672815330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe18c1010a0 0x7fe18c10e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.017+0000 7fe18b7fe640 1 -- 172.21.15.43:0/2672815330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe18c106170 msgr2=0x7fe18c10ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.017+0000 7fe18b7fe640 1 --2- 172.21.15.43:0/2672815330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe18c106170 0x7fe18c10ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.017+0000 7fe18b7fe640 1 -- 172.21.15.43:0/2672815330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18c1a6fe0 con 0x7fe18c10f150 2024-04-21T08:46:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.017+0000 7fe18b7fe640 1 --2- 172.21.15.43:0/2672815330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18c10f150 0x7fe18c1a68e0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fe18400eb10 tx=0x7fe18400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.017+0000 7fe1908fb640 1 -- 172.21.15.43:0/2672815330 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184004b20 con 0x7fe18c10f150 2024-04-21T08:46:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.017+0000 7fe1908fb640 1 -- 172.21.15.43:0/2672815330 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe184004cc0 con 0x7fe18c10f150 2024-04-21T08:46:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.017+0000 7fe1918fd640 1 -- 172.21.15.43:0/2672815330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe18c1a72d0 con 0x7fe18c10f150 2024-04-21T08:46:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.017+0000 7fe1908fb640 1 -- 172.21.15.43:0/2672815330 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184010670 con 0x7fe18c10f150 2024-04-21T08:46:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.017+0000 7fe1918fd640 1 -- 172.21.15.43:0/2672815330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe18c1a77d0 con 0x7fe18c10f150 2024-04-21T08:46:09.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.020+0000 7fe1908fb640 1 -- 172.21.15.43:0/2672815330 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe184005000 con 0x7fe18c10f150 2024-04-21T08:46:09.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.020+0000 7fe1945c9740 1 Processor -- start 2024-04-21T08:46:09.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.020+0000 7fe1945c9740 1 -- start start 2024-04-21T08:46:09.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.020+0000 7fe1908fb640 1 --2- 172.21.15.43:0/2672815330 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe1740a3d50 0x7fe1740a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.020+0000 7fe1945c9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76c8e0 0x56064d76ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.020+0000 7fe1945c9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56064d76d290 0x56064d775840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.020+0000 7fe1945c9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56064d775e10 0x56064d778260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.020+0000 7fe1945c9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56064d6d0f00 con 0x56064d76c8e0 2024-04-21T08:46:09.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.020+0000 7fe1945c9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56064d66e590 con 0x56064d775e10 2024-04-21T08:46:09.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.020+0000 7fe1945c9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56064d6ccec0 con 0x56064d76d290 2024-04-21T08:46:09.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.020+0000 7fe18a7fc640 1 --2- 172.21.15.43:0/2672815330 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe1740a3d50 0x7fe1740a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.020+0000 7fe18affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76c8e0 0x56064d76ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.021+0000 7fe18affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76c8e0 0x56064d76ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37282/0 (socket says 172.21.15.43:37282) 2024-04-21T08:46:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.021+0000 7fe18affd640 1 -- 172.21.15.43:0/2195594071 learned_addr learned my addr 172.21.15.43:0/2195594071 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.021+0000 7fe1908fb640 1 -- 172.21.15.43:0/2672815330 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7fe1840d1200 con 0x7fe18c10f150 2024-04-21T08:46:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.021+0000 7fe18b7fe640 1 --2- 172.21.15.43:0/2195594071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56064d775e10 0x56064d778260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.021+0000 7fe18a7fc640 1 --2- 172.21.15.43:0/2195594071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56064d76d290 0x56064d775840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.021+0000 7fe18affd640 1 -- 172.21.15.43:0/2195594071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56064d76d290 msgr2=0x56064d775840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.021+0000 7fe18affd640 1 --2- 172.21.15.43:0/2195594071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56064d76d290 0x56064d775840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.021+0000 7fe18affd640 1 -- 172.21.15.43:0/2195594071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56064d775e10 msgr2=0x56064d778260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.021+0000 7fe18affd640 1 --2- 172.21.15.43:0/2195594071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56064d775e10 0x56064d778260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.021+0000 7fe18affd640 1 -- 172.21.15.43:0/2195594071 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56064d4ab7e0 con 0x56064d76c8e0 2024-04-21T08:46:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.021+0000 7fe18a7fc640 1 --2- 172.21.15.43:0/2195594071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56064d76d290 0x56064d775840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.021+0000 7fe18a7fc640 1 --2- 172.21.15.43:0/2672815330 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe1740a3d50 0x7fe1740a6210 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe18c110160 tx=0x7fe18000bf90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.021+0000 7fe18affd640 1 --2- 172.21.15.43:0/2195594071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76c8e0 0x56064d76ccc0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009900 tx=0x7fe17c02c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ae760cd4d71fad server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.021+0000 7fe171ffb640 1 -- 172.21.15.43:0/2195594071 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c045020 con 0x56064d76c8e0 2024-04-21T08:46:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.021+0000 7fe171ffb640 1 -- 172.21.15.43:0/2195594071 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe17c02c7e0 con 0x56064d76c8e0 2024-04-21T08:46:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.021+0000 7fe171ffb640 1 -- 172.21.15.43:0/2195594071 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c0055a0 con 0x56064d76c8e0 2024-04-21T08:46:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.022+0000 7fe1945c9740 1 -- 172.21.15.43:0/2195594071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76c8e0 msgr2=0x56064d76ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.022+0000 7fe1945c9740 1 --2- 172.21.15.43:0/2195594071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76c8e0 0x56064d76ccc0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009900 tx=0x7fe17c02c610 comp rx=0 tx=0).stop 2024-04-21T08:46:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.022+0000 7fe1945c9740 1 -- 172.21.15.43:0/2195594071 shutdown_connections 2024-04-21T08:46:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.022+0000 7fe1945c9740 1 --2- 172.21.15.43:0/2195594071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56064d775e10 0x56064d778260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.022+0000 7fe1945c9740 1 --2- 172.21.15.43:0/2195594071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56064d76d290 0x56064d775840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.022+0000 7fe1945c9740 1 --2- 172.21.15.43:0/2195594071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76c8e0 0x56064d76ccc0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.022+0000 7fe1945c9740 1 -- 172.21.15.43:0/2195594071 >> 172.21.15.43:0/2195594071 conn(0x56064d766fb0 msgr2=0x56064d767390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.022+0000 7fe1945c9740 1 -- 172.21.15.43:0/2195594071 shutdown_connections 2024-04-21T08:46:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.022+0000 7fe1945c9740 1 -- 172.21.15.43:0/2195594071 wait complete. 2024-04-21T08:46:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.023+0000 7fe1945c9740 1 -- start start 2024-04-21T08:46:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.023+0000 7fe1945c9740 1 Processor -- start 2024-04-21T08:46:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.023+0000 7fe1945c9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56064d76c8e0 0x56064d772de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.023+0000 7fe1945c9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76d290 0x56064d773320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.023+0000 7fe1945c9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56064d775e10 0x56064d773860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.023+0000 7fe1945c9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56064d778d20 con 0x56064d76d290 2024-04-21T08:46:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.023+0000 7fe1945c9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56064d68a580 con 0x56064d76c8e0 2024-04-21T08:46:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.023+0000 7fe1945c9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56064d778ea0 con 0x56064d775e10 2024-04-21T08:46:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.023+0000 7fe1945c9740 10 client.0 _ll_register_callbacks cb 0x56064d690650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.023+0000 7fe18a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76d290 0x56064d773320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.023+0000 7fe18a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76d290 0x56064d773320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37292/0 (socket says 172.21.15.43:37292) 2024-04-21T08:46:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.023+0000 7fe18a7fc640 1 -- 172.21.15.43:0/296830353 learned_addr learned my addr 172.21.15.43:0/296830353 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.023+0000 7fe18affd640 1 --2- 172.21.15.43:0/296830353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56064d76c8e0 0x56064d772de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.023+0000 7fe18b7fe640 1 --2- 172.21.15.43:0/296830353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56064d775e10 0x56064d773860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.024+0000 7fe18a7fc640 1 -- 172.21.15.43:0/296830353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56064d775e10 msgr2=0x56064d773860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.024+0000 7fe18a7fc640 1 --2- 172.21.15.43:0/296830353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56064d775e10 0x56064d773860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.024+0000 7fe18a7fc640 1 -- 172.21.15.43:0/296830353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56064d76c8e0 msgr2=0x56064d772de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.024+0000 7fe18a7fc640 1 --2- 172.21.15.43:0/296830353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56064d76c8e0 0x56064d772de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.024+0000 7fe18a7fc640 1 -- 172.21.15.43:0/296830353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56064d50bf70 con 0x56064d76d290 2024-04-21T08:46:09.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.024+0000 7fe18a7fc640 1 --2- 172.21.15.43:0/296830353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76d290 0x56064d773320 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fe180011720 tx=0x7fe180004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d9ef7db88d6be2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.024+0000 7fe16affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:09.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.024+0000 7fe16affd640 1 -- 172.21.15.43:0/296830353 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180010920 con 0x56064d76d290 2024-04-21T08:46:09.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.024+0000 7fe16affd640 1 -- 172.21.15.43:0/296830353 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7fe180010ac0 con 0x56064d76d290 2024-04-21T08:46:09.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.024+0000 7fe1945c9740 1 -- 172.21.15.43:0/296830353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe17c02c7e0 con 0x56064d76d290 2024-04-21T08:46:09.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.024+0000 7fe16affd640 1 -- 172.21.15.43:0/296830353 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe18002d050 con 0x56064d76d290 2024-04-21T08:46:09.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.024+0000 7fe1945c9740 10 client.18909 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:09.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.024+0000 7fe1945c9740 1 -- 172.21.15.43:0/296830353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56064d3a3170 con 0x56064d76d290 2024-04-21T08:46:09.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.024+0000 7fe16affd640 1 -- 172.21.15.43:0/296830353 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe17c02c7e0 con 0x56064d76d290 2024-04-21T08:46:09.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.025+0000 7fe16affd640 1 -- 172.21.15.43:0/296830353 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe1800283f0 con 0x56064d76d290 2024-04-21T08:46:09.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.025+0000 7fe1945c9740 10 client.18909 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:09.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.025+0000 7fe1945c9740 10 client.18909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:09.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.025+0000 7fe1945c9740 20 client.18909 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:09.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.025+0000 7fe1945c9740 1 --2- 172.21.15.43:0/296830353 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56064d7862b0 0x56064d788770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.025+0000 7fe1945c9740 4 client.18909 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:09.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.025+0000 7fe1945c9740 1 -- 172.21.15.43:0/296830353 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56064d6de920 con 0x56064d7862b0 2024-04-21T08:46:09.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.025+0000 7fe18affd640 1 --2- 172.21.15.43:0/296830353 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56064d7862b0 0x56064d788770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.025+0000 7fe16affd640 10 client.18909 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:09.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.025+0000 7fe18affd640 1 --2- 172.21.15.43:0/296830353 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56064d7862b0 0x56064d788770 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5fd080216412509 server_cookie=be0ab428e94bdefa in_seq=0 out_seq=0 2024-04-21T08:46:09.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.026+0000 7fe16affd640 1 -- 172.21.15.43:0/296830353 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56064d6de920 con 0x56064d7862b0 2024-04-21T08:46:09.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.026+0000 7fe16affd640 10 client.18909 handle_command_reply: tid=0 2024-04-21T08:46:09.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.026+0000 7fe1945c9740 1 client.18909 shutdown 2024-04-21T08:46:09.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.027+0000 7fe1945c9740 1 -- 172.21.15.43:0/296830353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76d290 msgr2=0x56064d773320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.027+0000 7fe1945c9740 1 --2- 172.21.15.43:0/296830353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76d290 0x56064d773320 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fe180011720 tx=0x7fe180004730 comp rx=0 tx=0).stop 2024-04-21T08:46:09.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.027+0000 7fe1945c9740 1 -- 172.21.15.43:0/296830353 shutdown_connections 2024-04-21T08:46:09.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.027+0000 7fe1945c9740 1 --2- 172.21.15.43:0/296830353 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56064d7862b0 0x56064d788770 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.027+0000 7fe1945c9740 1 --2- 172.21.15.43:0/296830353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56064d775e10 0x56064d773860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.027+0000 7fe1945c9740 1 --2- 172.21.15.43:0/296830353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76d290 0x56064d773320 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.027+0000 7fe16affd640 0 client.18909 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:09.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.027+0000 7fe1945c9740 1 --2- 172.21.15.43:0/296830353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56064d76c8e0 0x56064d772de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.027+0000 7fe1945c9740 1 -- 172.21.15.43:0/296830353 >> 172.21.15.43:0/296830353 conn(0x56064d766fb0 msgr2=0x56064d768d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.027+0000 7fe1945c9740 1 -- 172.21.15.43:0/296830353 shutdown_connections 2024-04-21T08:46:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.027+0000 7fe1945c9740 1 -- 172.21.15.43:0/296830353 wait complete. 2024-04-21T08:46:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.027+0000 7fe1945c9740 20 client.18909 trim_cache size 0 max 16384 2024-04-21T08:46:09.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.045+0000 7fe1945c9740 1 Processor -- start 2024-04-21T08:46:09.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.045+0000 7fe1945c9740 1 -- start start 2024-04-21T08:46:09.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.045+0000 7fe1945c9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56064d76c8e0 0x56064d775510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.045+0000 7fe1945c9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76d290 0x56064d781be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.045+0000 7fe1945c9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56064d775e10 0x56064d78dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.045+0000 7fe1945c9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c1b1820 con 0x56064d76d290 2024-04-21T08:46:09.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.045+0000 7fe1945c9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c1b3360 con 0x56064d76c8e0 2024-04-21T08:46:09.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.045+0000 7fe1945c9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c1b1ef0 con 0x56064d775e10 2024-04-21T08:46:09.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.045+0000 7fe18a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76d290 0x56064d781be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.045+0000 7fe18a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76d290 0x56064d781be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37298/0 (socket says 172.21.15.43:37298) 2024-04-21T08:46:09.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.045+0000 7fe18a7fc640 1 -- 172.21.15.43:0/1632571031 learned_addr learned my addr 172.21.15.43:0/1632571031 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.045+0000 7fe18affd640 1 --2- 172.21.15.43:0/1632571031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56064d76c8e0 0x56064d775510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.045+0000 7fe18b7fe640 1 --2- 172.21.15.43:0/1632571031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56064d775e10 0x56064d78dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.045+0000 7fe18a7fc640 1 -- 172.21.15.43:0/1632571031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56064d775e10 msgr2=0x56064d78dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.045+0000 7fe18a7fc640 1 --2- 172.21.15.43:0/1632571031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56064d775e10 0x56064d78dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.045+0000 7fe18a7fc640 1 -- 172.21.15.43:0/1632571031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56064d76c8e0 msgr2=0x56064d775510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.045+0000 7fe18a7fc640 1 --2- 172.21.15.43:0/1632571031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56064d76c8e0 0x56064d775510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.045+0000 7fe18a7fc640 1 -- 172.21.15.43:0/1632571031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56064d771330 con 0x56064d76d290 2024-04-21T08:46:09.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.045+0000 7fe18b7fe640 1 --2- 172.21.15.43:0/1632571031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56064d775e10 0x56064d78dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:09.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.045+0000 7fe18a7fc640 1 --2- 172.21.15.43:0/1632571031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76d290 0x56064d781be0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fe180028860 tx=0x7fe18000b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c51a6bcf7ed4855 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.046+0000 7fe1717fa640 1 -- 172.21.15.43:0/1632571031 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180035020 con 0x56064d76d290 2024-04-21T08:46:09.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.046+0000 7fe1717fa640 1 -- 172.21.15.43:0/1632571031 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1800283f0 con 0x56064d76d290 2024-04-21T08:46:09.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.046+0000 7fe1717fa640 1 -- 172.21.15.43:0/1632571031 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18001f6d0 con 0x56064d76d290 2024-04-21T08:46:09.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.046+0000 7fe1945c9740 1 -- 172.21.15.43:0/1632571031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76d290 msgr2=0x56064d781be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.046+0000 7fe1945c9740 1 --2- 172.21.15.43:0/1632571031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76d290 0x56064d781be0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fe180028860 tx=0x7fe18000b950 comp rx=0 tx=0).stop 2024-04-21T08:46:09.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.046+0000 7fe1945c9740 1 -- 172.21.15.43:0/1632571031 shutdown_connections 2024-04-21T08:46:09.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.046+0000 7fe1945c9740 1 --2- 172.21.15.43:0/1632571031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56064d775e10 0x56064d78dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.046+0000 7fe1945c9740 1 --2- 172.21.15.43:0/1632571031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76d290 0x56064d781be0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.046+0000 7fe1945c9740 1 --2- 172.21.15.43:0/1632571031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56064d76c8e0 0x56064d775510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.046+0000 7fe1945c9740 1 -- 172.21.15.43:0/1632571031 >> 172.21.15.43:0/1632571031 conn(0x56064d766fb0 msgr2=0x56064d773630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.046+0000 7fe1945c9740 1 -- 172.21.15.43:0/1632571031 shutdown_connections 2024-04-21T08:46:09.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.046+0000 7fe1945c9740 1 -- 172.21.15.43:0/1632571031 wait complete. 2024-04-21T08:46:09.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.046+0000 7fe1945c9740 1 -- start start 2024-04-21T08:46:09.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.047+0000 7fe1945c9740 1 Processor -- start 2024-04-21T08:46:09.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.047+0000 7fe1945c9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76c8e0 0x56064d77a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.047+0000 7fe1945c9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56064d76d290 0x56064d77abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.047+0000 7fe1945c9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56064d775e10 0x56064d7e4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.047+0000 7fe1945c9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c1b2c90 con 0x56064d76c8e0 2024-04-21T08:46:09.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.047+0000 7fe1945c9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c1b25c0 con 0x56064d775e10 2024-04-21T08:46:09.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.047+0000 7fe1945c9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c1af820 con 0x56064d76d290 2024-04-21T08:46:09.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.047+0000 7fe1945c9740 10 client.0 _ll_register_callbacks cb 0x7fe18000fad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:09.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.047+0000 7fe18affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76c8e0 0x56064d77a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.047+0000 7fe18affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76c8e0 0x56064d77a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37312/0 (socket says 172.21.15.43:37312) 2024-04-21T08:46:09.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.047+0000 7fe18affd640 1 -- 172.21.15.43:0/2902110801 learned_addr learned my addr 172.21.15.43:0/2902110801 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.047+0000 7fe18a7fc640 1 --2- 172.21.15.43:0/2902110801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56064d76d290 0x56064d77abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.047+0000 7fe18b7fe640 1 --2- 172.21.15.43:0/2902110801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56064d775e10 0x56064d7e4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.047+0000 7fe18affd640 1 -- 172.21.15.43:0/2902110801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56064d76d290 msgr2=0x56064d77abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.047+0000 7fe18affd640 1 --2- 172.21.15.43:0/2902110801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56064d76d290 0x56064d77abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.047+0000 7fe18affd640 1 -- 172.21.15.43:0/2902110801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56064d775e10 msgr2=0x56064d7e4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.047+0000 7fe18affd640 1 --2- 172.21.15.43:0/2902110801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56064d775e10 0x56064d7e4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.047+0000 7fe18affd640 1 -- 172.21.15.43:0/2902110801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56064d7e5020 con 0x56064d76c8e0 2024-04-21T08:46:09.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.047+0000 7fe18b7fe640 1 --2- 172.21.15.43:0/2902110801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56064d775e10 0x56064d7e4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:09.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.048+0000 7fe18affd640 1 --2- 172.21.15.43:0/2902110801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76c8e0 0x56064d77a6b0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fe17c005150 tx=0x7fe17c0050e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1109e63afdad112f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.048+0000 7fe16affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:09.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.048+0000 7fe16affd640 1 -- 172.21.15.43:0/2902110801 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c053070 con 0x56064d76c8e0 2024-04-21T08:46:09.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.048+0000 7fe16affd640 1 -- 172.21.15.43:0/2902110801 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7fe17c02c7e0 con 0x56064d76c8e0 2024-04-21T08:46:09.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.048+0000 7fe1945c9740 1 -- 172.21.15.43:0/2902110801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1800283f0 con 0x56064d76c8e0 2024-04-21T08:46:09.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.048+0000 7fe16affd640 1 -- 172.21.15.43:0/2902110801 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe17c05e050 con 0x56064d76c8e0 2024-04-21T08:46:09.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.048+0000 7fe1945c9740 10 client.18915 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:09.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.048+0000 7fe1945c9740 1 -- 172.21.15.43:0/2902110801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56064d7ed390 con 0x56064d76c8e0 2024-04-21T08:46:09.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.048+0000 7fe16affd640 1 -- 172.21.15.43:0/2902110801 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1800283f0 con 0x56064d76c8e0 2024-04-21T08:46:09.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.048+0000 7fe16affd640 1 -- 172.21.15.43:0/2902110801 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe17c0046c0 con 0x56064d76c8e0 2024-04-21T08:46:09.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.048+0000 7fe1945c9740 10 client.18915 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:09.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.048+0000 7fe1945c9740 10 client.18915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:09.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.048+0000 7fe1945c9740 20 client.18915 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:09.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.049+0000 7fe1945c9740 1 --2- 172.21.15.43:0/2902110801 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56064d7ed680 0x56064d7efa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.049+0000 7fe1945c9740 4 client.18915 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:46:09.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.049+0000 7fe1945c9740 1 -- 172.21.15.43:0/2902110801 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56064d6d8790 con 0x56064d7ed680 2024-04-21T08:46:09.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.049+0000 7fe18a7fc640 1 --2- 172.21.15.43:0/2902110801 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56064d7ed680 0x56064d7efa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.049+0000 7fe16affd640 10 client.18915 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:09.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.049+0000 7fe18a7fc640 1 --2- 172.21.15.43:0/2902110801 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56064d7ed680 0x56064d7efa70 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=506c03be0524a803 server_cookie=83e4b131a6c9dfa4 in_seq=0 out_seq=0 2024-04-21T08:46:09.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.050+0000 7fe16affd640 1 -- 172.21.15.43:0/2902110801 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56064d6d8790 con 0x56064d7ed680 2024-04-21T08:46:09.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.050+0000 7fe16affd640 10 client.18915 handle_command_reply: tid=0 2024-04-21T08:46:09.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.050+0000 7fe1945c9740 1 client.18915 shutdown 2024-04-21T08:46:09.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.050+0000 7fe1945c9740 1 -- 172.21.15.43:0/2902110801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76c8e0 msgr2=0x56064d77a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.050+0000 7fe1945c9740 1 --2- 172.21.15.43:0/2902110801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76c8e0 0x56064d77a6b0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fe17c005150 tx=0x7fe17c0050e0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.051+0000 7fe1945c9740 1 -- 172.21.15.43:0/2902110801 shutdown_connections 2024-04-21T08:46:09.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.051+0000 7fe1945c9740 1 --2- 172.21.15.43:0/2902110801 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56064d7ed680 0x56064d7efa70 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.051+0000 7fe1945c9740 1 --2- 172.21.15.43:0/2902110801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56064d775e10 0x56064d7e4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.051+0000 7fe1945c9740 1 --2- 172.21.15.43:0/2902110801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56064d76d290 0x56064d77abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.051+0000 7fe16affd640 0 client.18915 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:09.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.051+0000 7fe1945c9740 1 --2- 172.21.15.43:0/2902110801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56064d76c8e0 0x56064d77a6b0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.051+0000 7fe1945c9740 1 -- 172.21.15.43:0/2902110801 >> 172.21.15.43:0/2902110801 conn(0x56064d766fb0 msgr2=0x56064d768330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.051+0000 7fe1945c9740 1 -- 172.21.15.43:0/2902110801 shutdown_connections 2024-04-21T08:46:09.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.051+0000 7fe1945c9740 1 -- 172.21.15.43:0/2902110801 wait complete. 2024-04-21T08:46:09.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.051+0000 7fe1945c9740 20 client.18915 trim_cache size 0 max 16384 2024-04-21T08:46:09.056 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:46:09.056 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:46:09.056 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "58e85a6d-7481-4a4b-9960-0d4c519d15a4", 2024-04-21T08:46:09.056 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:46:09.056 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:46:09.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.051+0000 7fe1918fd640 1 -- 172.21.15.43:0/2672815330 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe1740a3d50 msgr2=0x7fe1740a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.051+0000 7fe1918fd640 1 --2- 172.21.15.43:0/2672815330 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe1740a3d50 0x7fe1740a6210 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe18c110160 tx=0x7fe18000bf90 comp rx=0 tx=0).stop 2024-04-21T08:46:09.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.051+0000 7fe1918fd640 1 -- 172.21.15.43:0/2672815330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18c10f150 msgr2=0x7fe18c1a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.051+0000 7fe1918fd640 1 --2- 172.21.15.43:0/2672815330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18c10f150 0x7fe18c1a68e0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fe18400eb10 tx=0x7fe18400c2d0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.051+0000 7fe1918fd640 1 -- 172.21.15.43:0/2672815330 shutdown_connections 2024-04-21T08:46:09.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.051+0000 7fe1918fd640 1 --2- 172.21.15.43:0/2672815330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18c10f150 0x7fe18c1a68e0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.051+0000 7fe1918fd640 1 --2- 172.21.15.43:0/2672815330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe18c106170 0x7fe18c10ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.051+0000 7fe1918fd640 1 --2- 172.21.15.43:0/2672815330 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe1740a3d50 0x7fe1740a6210 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.051+0000 7fe1918fd640 1 --2- 172.21.15.43:0/2672815330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe18c1010a0 0x7fe18c10e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.051+0000 7fe1918fd640 1 -- 172.21.15.43:0/2672815330 >> 172.21.15.43:0/2672815330 conn(0x7fe18c077eb0 msgr2=0x7fe18c0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.051+0000 7fe1918fd640 1 -- 172.21.15.43:0/2672815330 shutdown_connections 2024-04-21T08:46:09.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.051+0000 7fe1918fd640 1 -- 172.21.15.43:0/2672815330 wait complete. 2024-04-21T08:46:09.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:46:09.062 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:46:09.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.166+0000 7effecd3d640 1 -- 172.21.15.43:0/1982193909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effe8108060 msgr2=0x7effe8108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.166+0000 7effecd3d640 1 --2- 172.21.15.43:0/1982193909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effe8108060 0x7effe8108440 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7effd8009930 tx=0x7effd802bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:09.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.166+0000 7effecd3d640 1 -- 172.21.15.43:0/1982193909 shutdown_connections 2024-04-21T08:46:09.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.166+0000 7effecd3d640 1 --2- 172.21.15.43:0/1982193909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effe806af70 0x7effe810ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.166+0000 7effecd3d640 1 --2- 172.21.15.43:0/1982193909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effe806a5b0 0x7effe806aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.166+0000 7effecd3d640 1 --2- 172.21.15.43:0/1982193909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effe8108060 0x7effe8108440 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.166+0000 7effecd3d640 1 -- 172.21.15.43:0/1982193909 >> 172.21.15.43:0/1982193909 conn(0x7effe8077c40 msgr2=0x7effe8078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.166+0000 7effecd3d640 1 -- 172.21.15.43:0/1982193909 shutdown_connections 2024-04-21T08:46:09.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.166+0000 7effecd3d640 1 -- 172.21.15.43:0/1982193909 wait complete. 2024-04-21T08:46:09.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.167+0000 7effecd3d640 1 Processor -- start 2024-04-21T08:46:09.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.167+0000 7effecd3d640 1 -- start start 2024-04-21T08:46:09.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.167+0000 7effecd3d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effe806a5b0 0x7effe8198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.167+0000 7effecd3d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effe806af70 0x7effe8198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.167+0000 7effecd3d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effe8108060 0x7effe819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.167+0000 7effecd3d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7effe8112180 con 0x7effe806af70 2024-04-21T08:46:09.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.167+0000 7effecd3d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7effe8112000 con 0x7effe806a5b0 2024-04-21T08:46:09.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.167+0000 7effecd3d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7effe8112300 con 0x7effe8108060 2024-04-21T08:46:09.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.167+0000 7effdffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effe806af70 0x7effe8198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.167+0000 7effdffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effe806af70 0x7effe8198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37320/0 (socket says 172.21.15.43:37320) 2024-04-21T08:46:09.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.167+0000 7effdffff640 1 -- 172.21.15.43:0/1566257410 learned_addr learned my addr 172.21.15.43:0/1566257410 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.167+0000 7effe659c640 1 --2- 172.21.15.43:0/1566257410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effe806a5b0 0x7effe8198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.168+0000 7effe6d9d640 1 --2- 172.21.15.43:0/1566257410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effe8108060 0x7effe819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.168+0000 7effdffff640 1 -- 172.21.15.43:0/1566257410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effe8108060 msgr2=0x7effe819cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.168+0000 7effdffff640 1 --2- 172.21.15.43:0/1566257410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effe8108060 0x7effe819cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.168+0000 7effdffff640 1 -- 172.21.15.43:0/1566257410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effe806a5b0 msgr2=0x7effe8198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.168+0000 7effdffff640 1 --2- 172.21.15.43:0/1566257410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effe806a5b0 0x7effe8198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.168+0000 7effdffff640 1 -- 172.21.15.43:0/1566257410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe819d6a0 con 0x7effe806af70 2024-04-21T08:46:09.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.168+0000 7effdffff640 1 --2- 172.21.15.43:0/1566257410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effe806af70 0x7effe8198c40 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7effd000d950 tx=0x7effd000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.168+0000 7effdf7fe640 1 -- 172.21.15.43:0/1566257410 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0017070 con 0x7effe806af70 2024-04-21T08:46:09.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.168+0000 7effdf7fe640 1 -- 172.21.15.43:0/1566257410 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effd0009c30 con 0x7effe806af70 2024-04-21T08:46:09.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.168+0000 7effecd3d640 1 -- 172.21.15.43:0/1566257410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe819d990 con 0x7effe806af70 2024-04-21T08:46:09.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.168+0000 7effdf7fe640 1 -- 172.21.15.43:0/1566257410 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd000f620 con 0x7effe806af70 2024-04-21T08:46:09.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.168+0000 7effecd3d640 1 -- 172.21.15.43:0/1566257410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe81a5220 con 0x7effe806af70 2024-04-21T08:46:09.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.171+0000 7effdf7fe640 1 -- 172.21.15.43:0/1566257410 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7effd0010460 con 0x7effe806af70 2024-04-21T08:46:09.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.171+0000 7effefa08740 1 Processor -- start 2024-04-21T08:46:09.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.171+0000 7effefa08740 1 -- start start 2024-04-21T08:46:09.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.171+0000 7effefa08740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f385580 0x55fe7f385960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.171+0000 7effefa08740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe7f385ea0 0x55fe7f38e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.171+0000 7effefa08740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe7f38ea80 0x55fe7f390e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.171+0000 7effefa08740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fe7f2ea1e0 con 0x55fe7f385580 2024-04-21T08:46:09.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.171+0000 7effefa08740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fe7f287390 con 0x55fe7f385ea0 2024-04-21T08:46:09.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.171+0000 7effefa08740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fe7f2e61a0 con 0x55fe7f38ea80 2024-04-21T08:46:09.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.171+0000 7effe6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f385580 0x55fe7f385960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.171+0000 7effe6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f385580 0x55fe7f385960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37334/0 (socket says 172.21.15.43:37334) 2024-04-21T08:46:09.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.171+0000 7effe6d9d640 1 -- 172.21.15.43:0/3986662598 learned_addr learned my addr 172.21.15.43:0/3986662598 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.171+0000 7effdf7fe640 1 --2- 172.21.15.43:0/1566257410 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7effd40a3da0 0x7effd40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.172+0000 7effe659c640 1 --2- 172.21.15.43:0/3986662598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe7f385ea0 0x55fe7f38e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.172+0000 7effdffff640 1 --2- 172.21.15.43:0/3986662598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe7f38ea80 0x55fe7f390e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.172+0000 7effe6d9d640 1 -- 172.21.15.43:0/3986662598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe7f38ea80 msgr2=0x55fe7f390e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.172+0000 7effe6d9d640 1 --2- 172.21.15.43:0/3986662598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe7f38ea80 0x55fe7f390e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.172+0000 7effe6d9d640 1 -- 172.21.15.43:0/3986662598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe7f385ea0 msgr2=0x55fe7f38e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.172+0000 7effe6d9d640 1 --2- 172.21.15.43:0/3986662598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe7f385ea0 0x55fe7f38e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.172+0000 7effe6d9d640 1 -- 172.21.15.43:0/3986662598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe7f0c67d0 con 0x55fe7f385580 2024-04-21T08:46:09.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.172+0000 7effdf7fe640 1 -- 172.21.15.43:0/1566257410 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7effd00d0860 con 0x7effe806af70 2024-04-21T08:46:09.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.172+0000 7effe6d9d640 1 --2- 172.21.15.43:0/1566257410 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7effd40a3da0 0x7effd40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.172+0000 7effe6d9d640 1 --2- 172.21.15.43:0/3986662598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f385580 0x55fe7f385960 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7effe000e2f0 tx=0x7effe000e7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db207ba776ade62d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.172+0000 7effdcff9640 1 -- 172.21.15.43:0/3986662598 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe0004b60 con 0x55fe7f385580 2024-04-21T08:46:09.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.172+0000 7effdcff9640 1 -- 172.21.15.43:0/3986662598 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effe0004d00 con 0x55fe7f385580 2024-04-21T08:46:09.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.172+0000 7effdcff9640 1 -- 172.21.15.43:0/3986662598 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe0011730 con 0x55fe7f385580 2024-04-21T08:46:09.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.173+0000 7effefa08740 1 -- 172.21.15.43:0/3986662598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f385580 msgr2=0x55fe7f385960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.173+0000 7effefa08740 1 --2- 172.21.15.43:0/3986662598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f385580 0x55fe7f385960 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7effe000e2f0 tx=0x7effe000e7b0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.173+0000 7effe6d9d640 1 --2- 172.21.15.43:0/1566257410 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7effd40a3da0 0x7effd40a6260 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7effe001f000 tx=0x7effe0020000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.173+0000 7effefa08740 1 -- 172.21.15.43:0/3986662598 shutdown_connections 2024-04-21T08:46:09.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.173+0000 7effefa08740 1 --2- 172.21.15.43:0/3986662598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe7f38ea80 0x55fe7f390e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.173+0000 7effefa08740 1 --2- 172.21.15.43:0/3986662598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe7f385ea0 0x55fe7f38e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.173+0000 7effefa08740 1 --2- 172.21.15.43:0/3986662598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f385580 0x55fe7f385960 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.173+0000 7effefa08740 1 -- 172.21.15.43:0/3986662598 >> 172.21.15.43:0/3986662598 conn(0x55fe7f37fcb0 msgr2=0x55fe7f380090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.173+0000 7effefa08740 1 -- 172.21.15.43:0/3986662598 shutdown_connections 2024-04-21T08:46:09.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.173+0000 7effefa08740 1 -- 172.21.15.43:0/3986662598 wait complete. 2024-04-21T08:46:09.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.173+0000 7effefa08740 1 -- start start 2024-04-21T08:46:09.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.174+0000 7effefa08740 1 Processor -- start 2024-04-21T08:46:09.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.174+0000 7effefa08740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f385580 0x55fe7f38bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.174+0000 7effefa08740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe7f385ea0 0x55fe7f38c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.174+0000 7effefa08740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe7f38ea80 0x55fe7f38c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.174+0000 7effefa08740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fe7f391870 con 0x55fe7f385580 2024-04-21T08:46:09.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.174+0000 7effefa08740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fe7f2a32f0 con 0x55fe7f385ea0 2024-04-21T08:46:09.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.174+0000 7effefa08740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fe7f3919f0 con 0x55fe7f38ea80 2024-04-21T08:46:09.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.174+0000 7effefa08740 10 client.0 _ll_register_callbacks cb 0x55fe7f2b0580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:09.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.174+0000 7effe659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f385580 0x55fe7f38bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.174+0000 7effe659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f385580 0x55fe7f38bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37350/0 (socket says 172.21.15.43:37350) 2024-04-21T08:46:09.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.174+0000 7effe659c640 1 -- 172.21.15.43:0/3388433414 learned_addr learned my addr 172.21.15.43:0/3388433414 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.174+0000 7effe6d9d640 1 --2- 172.21.15.43:0/3388433414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe7f38ea80 0x55fe7f38c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.174+0000 7effdffff640 1 --2- 172.21.15.43:0/3388433414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe7f385ea0 0x55fe7f38c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.174+0000 7effe659c640 1 -- 172.21.15.43:0/3388433414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe7f38ea80 msgr2=0x55fe7f38c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.174+0000 7effe659c640 1 --2- 172.21.15.43:0/3388433414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe7f38ea80 0x55fe7f38c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.174+0000 7effe659c640 1 -- 172.21.15.43:0/3388433414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe7f385ea0 msgr2=0x55fe7f38c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.174+0000 7effe659c640 1 --2- 172.21.15.43:0/3388433414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe7f385ea0 0x55fe7f38c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.174+0000 7effe659c640 1 -- 172.21.15.43:0/3388433414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe7effc3f0 con 0x55fe7f385580 2024-04-21T08:46:09.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.174+0000 7effe6d9d640 1 --2- 172.21.15.43:0/3388433414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe7f38ea80 0x55fe7f38c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:09.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.174+0000 7effe659c640 1 --2- 172.21.15.43:0/3388433414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f385580 0x55fe7f38bad0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7effd8009930 tx=0x7effd802ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d382b61507a625 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.174+0000 7effcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:09.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.175+0000 7effcdffb640 1 -- 172.21.15.43:0/3388433414 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd802c5d0 con 0x55fe7f385580 2024-04-21T08:46:09.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.175+0000 7effcdffb640 1 -- 172.21.15.43:0/3388433414 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7effd8005030 con 0x55fe7f385580 2024-04-21T08:46:09.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.175+0000 7effefa08740 1 -- 172.21.15.43:0/3388433414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effe0004d00 con 0x55fe7f385580 2024-04-21T08:46:09.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.175+0000 7effcdffb640 1 -- 172.21.15.43:0/3388433414 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7effd8052050 con 0x55fe7f385580 2024-04-21T08:46:09.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.175+0000 7effefa08740 10 client.18927 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:09.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.175+0000 7effefa08740 1 -- 172.21.15.43:0/3388433414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fe7efbc0f0 con 0x55fe7f385580 2024-04-21T08:46:09.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.175+0000 7effcdffb640 1 -- 172.21.15.43:0/3388433414 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effe0004d00 con 0x55fe7f385580 2024-04-21T08:46:09.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.175+0000 7effcdffb640 1 -- 172.21.15.43:0/3388433414 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7effd80035f0 con 0x55fe7f385580 2024-04-21T08:46:09.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.175+0000 7effefa08740 10 client.18927 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:09.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.175+0000 7effefa08740 10 client.18927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:09.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.175+0000 7effefa08740 20 client.18927 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:09.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.175+0000 7effefa08740 1 --2- 172.21.15.43:0/3388433414 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fe7f39eb30 0x55fe7f3a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.175+0000 7effefa08740 4 client.18927 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:09.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.175+0000 7effefa08740 1 -- 172.21.15.43:0/3388433414 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe7f2f7c00 con 0x55fe7f39eb30 2024-04-21T08:46:09.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.175+0000 7effdffff640 1 --2- 172.21.15.43:0/3388433414 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fe7f39eb30 0x55fe7f3a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.176+0000 7effcdffb640 10 client.18927 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:09.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.176+0000 7effdffff640 1 --2- 172.21.15.43:0/3388433414 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fe7f39eb30 0x55fe7f3a0f80 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=974ef4c9a4188724 server_cookie=ba3898abca7ef1ed in_seq=0 out_seq=0 2024-04-21T08:46:09.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.176+0000 7effcdffb640 1 -- 172.21.15.43:0/3388433414 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fe7f2f7c00 con 0x55fe7f39eb30 2024-04-21T08:46:09.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.176+0000 7effcdffb640 10 client.18927 handle_command_reply: tid=0 2024-04-21T08:46:09.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.177+0000 7effefa08740 1 client.18927 shutdown 2024-04-21T08:46:09.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.177+0000 7effefa08740 1 -- 172.21.15.43:0/3388433414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f385580 msgr2=0x55fe7f38bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.177+0000 7effefa08740 1 --2- 172.21.15.43:0/3388433414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f385580 0x55fe7f38bad0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7effd8009930 tx=0x7effd802ca10 comp rx=0 tx=0).stop 2024-04-21T08:46:09.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.177+0000 7effefa08740 1 -- 172.21.15.43:0/3388433414 shutdown_connections 2024-04-21T08:46:09.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.177+0000 7effefa08740 1 --2- 172.21.15.43:0/3388433414 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fe7f39eb30 0x55fe7f3a0f80 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.177+0000 7effefa08740 1 --2- 172.21.15.43:0/3388433414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe7f38ea80 0x55fe7f38c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.177+0000 7effcdffb640 0 client.18927 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:09.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.177+0000 7effefa08740 1 --2- 172.21.15.43:0/3388433414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe7f385ea0 0x55fe7f38c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.177+0000 7effefa08740 1 --2- 172.21.15.43:0/3388433414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f385580 0x55fe7f38bad0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.177+0000 7effefa08740 1 -- 172.21.15.43:0/3388433414 >> 172.21.15.43:0/3388433414 conn(0x55fe7f37fcb0 msgr2=0x55fe7f381a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.177+0000 7effefa08740 1 -- 172.21.15.43:0/3388433414 shutdown_connections 2024-04-21T08:46:09.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.177+0000 7effefa08740 1 -- 172.21.15.43:0/3388433414 wait complete. 2024-04-21T08:46:09.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.177+0000 7effefa08740 20 client.18927 trim_cache size 0 max 16384 2024-04-21T08:46:09.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.194+0000 7effefa08740 1 Processor -- start 2024-04-21T08:46:09.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.194+0000 7effefa08740 1 -- start start 2024-04-21T08:46:09.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.195+0000 7effefa08740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe7f385580 0x55fe7f38e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.195+0000 7effefa08740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe7f385ea0 0x55fe7f39a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.195+0000 7effefa08740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f38ea80 0x55fe7f397020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.195+0000 7effefa08740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7effe81af5c0 con 0x55fe7f38ea80 2024-04-21T08:46:09.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.195+0000 7effefa08740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7effe81b1100 con 0x55fe7f385580 2024-04-21T08:46:09.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.195+0000 7effefa08740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7effe81afc90 con 0x55fe7f385ea0 2024-04-21T08:46:09.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.195+0000 7effe6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f38ea80 0x55fe7f397020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.195+0000 7effe6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f38ea80 0x55fe7f397020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37356/0 (socket says 172.21.15.43:37356) 2024-04-21T08:46:09.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.195+0000 7effe6d9d640 1 -- 172.21.15.43:0/2074110988 learned_addr learned my addr 172.21.15.43:0/2074110988 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.195+0000 7effe659c640 1 --2- 172.21.15.43:0/2074110988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe7f385580 0x55fe7f38e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.195+0000 7effdffff640 1 --2- 172.21.15.43:0/2074110988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe7f385ea0 0x55fe7f39a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.195+0000 7effe6d9d640 1 -- 172.21.15.43:0/2074110988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe7f385ea0 msgr2=0x55fe7f39a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.195+0000 7effe6d9d640 1 --2- 172.21.15.43:0/2074110988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe7f385ea0 0x55fe7f39a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.195+0000 7effe6d9d640 1 -- 172.21.15.43:0/2074110988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe7f385580 msgr2=0x55fe7f38e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.195+0000 7effe6d9d640 1 --2- 172.21.15.43:0/2074110988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe7f385580 0x55fe7f38e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.195+0000 7effe6d9d640 1 -- 172.21.15.43:0/2074110988 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe7f38a020 con 0x55fe7f38ea80 2024-04-21T08:46:09.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.195+0000 7effe6d9d640 1 --2- 172.21.15.43:0/2074110988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f38ea80 0x55fe7f397020 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7effe0009790 tx=0x7effe00049a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb75adb1481b8d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.195+0000 7effcffff640 1 -- 172.21.15.43:0/2074110988 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe0011460 con 0x55fe7f38ea80 2024-04-21T08:46:09.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.195+0000 7effcffff640 1 -- 172.21.15.43:0/2074110988 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effe0011600 con 0x55fe7f38ea80 2024-04-21T08:46:09.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.195+0000 7effcffff640 1 -- 172.21.15.43:0/2074110988 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe001b830 con 0x55fe7f38ea80 2024-04-21T08:46:09.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.196+0000 7effefa08740 1 -- 172.21.15.43:0/2074110988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f38ea80 msgr2=0x55fe7f397020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.196+0000 7effefa08740 1 --2- 172.21.15.43:0/2074110988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f38ea80 0x55fe7f397020 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7effe0009790 tx=0x7effe00049a0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.196+0000 7effefa08740 1 -- 172.21.15.43:0/2074110988 shutdown_connections 2024-04-21T08:46:09.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.196+0000 7effefa08740 1 --2- 172.21.15.43:0/2074110988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f38ea80 0x55fe7f397020 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.196+0000 7effefa08740 1 --2- 172.21.15.43:0/2074110988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe7f385ea0 0x55fe7f39a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.196+0000 7effefa08740 1 --2- 172.21.15.43:0/2074110988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe7f385580 0x55fe7f38e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.196+0000 7effefa08740 1 -- 172.21.15.43:0/2074110988 >> 172.21.15.43:0/2074110988 conn(0x55fe7f37fcb0 msgr2=0x55fe7f37e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.196+0000 7effefa08740 1 -- 172.21.15.43:0/2074110988 shutdown_connections 2024-04-21T08:46:09.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.196+0000 7effefa08740 1 -- 172.21.15.43:0/2074110988 wait complete. 2024-04-21T08:46:09.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.196+0000 7effefa08740 1 -- start start 2024-04-21T08:46:09.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.196+0000 7effefa08740 1 Processor -- start 2024-04-21T08:46:09.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.197+0000 7effefa08740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f385580 0x55fe7f389120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.197+0000 7effefa08740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe7f385ea0 0x55fe7f389660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.197+0000 7effefa08740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe7f38ea80 0x55fe7f387d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.197+0000 7effefa08740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7effe81b0a30 con 0x55fe7f385580 2024-04-21T08:46:09.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.197+0000 7effefa08740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7effe81b0360 con 0x55fe7f38ea80 2024-04-21T08:46:09.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.197+0000 7effefa08740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7effe81ad5c0 con 0x55fe7f385ea0 2024-04-21T08:46:09.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.197+0000 7effefa08740 10 client.0 _ll_register_callbacks cb 0x7effd8009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:09.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.197+0000 7effe659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f385580 0x55fe7f389120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.197+0000 7effe659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f385580 0x55fe7f389120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37366/0 (socket says 172.21.15.43:37366) 2024-04-21T08:46:09.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.197+0000 7effe659c640 1 -- 172.21.15.43:0/2290852750 learned_addr learned my addr 172.21.15.43:0/2290852750 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.197+0000 7effdffff640 1 --2- 172.21.15.43:0/2290852750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe7f385ea0 0x55fe7f389660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.197+0000 7effe6d9d640 1 --2- 172.21.15.43:0/2290852750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe7f38ea80 0x55fe7f387d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.197+0000 7effe659c640 1 -- 172.21.15.43:0/2290852750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe7f385ea0 msgr2=0x55fe7f389660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.197+0000 7effe659c640 1 --2- 172.21.15.43:0/2290852750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe7f385ea0 0x55fe7f389660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.197+0000 7effe659c640 1 -- 172.21.15.43:0/2290852750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe7f38ea80 msgr2=0x55fe7f387d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.197+0000 7effe659c640 1 --2- 172.21.15.43:0/2290852750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe7f38ea80 0x55fe7f387d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.197+0000 7effe659c640 1 -- 172.21.15.43:0/2290852750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe7f388520 con 0x55fe7f385580 2024-04-21T08:46:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.197+0000 7effe659c640 1 --2- 172.21.15.43:0/2290852750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f385580 0x55fe7f389120 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7effd8009850 tx=0x7effd8004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d9292f92d98b37 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.197+0000 7effcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.197+0000 7effcdffb640 1 -- 172.21.15.43:0/2290852750 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd8034440 con 0x55fe7f385580 2024-04-21T08:46:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.197+0000 7effcdffb640 1 -- 172.21.15.43:0/2290852750 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7effd80345e0 con 0x55fe7f385580 2024-04-21T08:46:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.197+0000 7effefa08740 1 -- 172.21.15.43:0/2290852750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effe0011600 con 0x55fe7f385580 2024-04-21T08:46:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.198+0000 7effcdffb640 1 -- 172.21.15.43:0/2290852750 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7effd8003e00 con 0x55fe7f385580 2024-04-21T08:46:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.198+0000 7effefa08740 10 client.18933 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.198+0000 7effefa08740 1 -- 172.21.15.43:0/2290852750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fe7f409b90 con 0x55fe7f385580 2024-04-21T08:46:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.198+0000 7effcdffb640 1 -- 172.21.15.43:0/2290852750 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effe0011600 con 0x55fe7f385580 2024-04-21T08:46:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.198+0000 7effcdffb640 1 -- 172.21.15.43:0/2290852750 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7effd8040490 con 0x55fe7f385580 2024-04-21T08:46:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.198+0000 7effefa08740 10 client.18933 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.198+0000 7effefa08740 10 client.18933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.198+0000 7effefa08740 20 client.18933 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.198+0000 7effefa08740 1 --2- 172.21.15.43:0/2290852750 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fe7f40a300 0x55fe7f40c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.198+0000 7effefa08740 4 client.18933 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:46:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.198+0000 7effefa08740 1 -- 172.21.15.43:0/2290852750 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fe7f2f1a70 con 0x55fe7f40a300 2024-04-21T08:46:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.198+0000 7effdffff640 1 --2- 172.21.15.43:0/2290852750 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fe7f40a300 0x55fe7f40c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.199+0000 7effdffff640 1 --2- 172.21.15.43:0/2290852750 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fe7f40a300 0x55fe7f40c720 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a189f6b04b221f0 server_cookie=93155812703f4f43 in_seq=0 out_seq=0 2024-04-21T08:46:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.199+0000 7effcdffb640 10 client.18933 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.199+0000 7effcdffb640 1 -- 172.21.15.43:0/2290852750 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fe7f2f1a70 con 0x55fe7f40a300 2024-04-21T08:46:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.199+0000 7effcdffb640 10 client.18933 handle_command_reply: tid=0 2024-04-21T08:46:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.199+0000 7effefa08740 1 client.18933 shutdown 2024-04-21T08:46:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.199+0000 7effefa08740 1 -- 172.21.15.43:0/2290852750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f385580 msgr2=0x55fe7f389120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.199+0000 7effefa08740 1 --2- 172.21.15.43:0/2290852750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f385580 0x55fe7f389120 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7effd8009850 tx=0x7effd8004610 comp rx=0 tx=0).stop 2024-04-21T08:46:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.199+0000 7effefa08740 1 -- 172.21.15.43:0/2290852750 shutdown_connections 2024-04-21T08:46:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.199+0000 7effefa08740 1 --2- 172.21.15.43:0/2290852750 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fe7f40a300 0x55fe7f40c720 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.199+0000 7effefa08740 1 --2- 172.21.15.43:0/2290852750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe7f38ea80 0x55fe7f387d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.199+0000 7effcdffb640 0 client.18933 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.199+0000 7effefa08740 1 --2- 172.21.15.43:0/2290852750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fe7f385ea0 0x55fe7f389660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.199+0000 7effefa08740 1 --2- 172.21.15.43:0/2290852750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fe7f385580 0x55fe7f389120 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.199+0000 7effefa08740 1 -- 172.21.15.43:0/2290852750 >> 172.21.15.43:0/2290852750 conn(0x55fe7f37fcb0 msgr2=0x55fe7f38c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.199+0000 7effefa08740 1 -- 172.21.15.43:0/2290852750 shutdown_connections 2024-04-21T08:46:09.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.199+0000 7effefa08740 1 -- 172.21.15.43:0/2290852750 wait complete. 2024-04-21T08:46:09.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.199+0000 7effefa08740 20 client.18933 trim_cache size 0 max 16384 2024-04-21T08:46:09.205 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:46:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:46:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:46:09.205 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:46:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.199+0000 7effecd3d640 1 -- 172.21.15.43:0/1566257410 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7effd40a3da0 msgr2=0x7effd40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.199+0000 7effecd3d640 1 --2- 172.21.15.43:0/1566257410 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7effd40a3da0 0x7effd40a6260 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7effe001f000 tx=0x7effe0020000 comp rx=0 tx=0).stop 2024-04-21T08:46:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.199+0000 7effecd3d640 1 -- 172.21.15.43:0/1566257410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effe806af70 msgr2=0x7effe8198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.199+0000 7effecd3d640 1 --2- 172.21.15.43:0/1566257410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effe806af70 0x7effe8198c40 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7effd000d950 tx=0x7effd000de10 comp rx=0 tx=0).stop 2024-04-21T08:46:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.200+0000 7effecd3d640 1 -- 172.21.15.43:0/1566257410 shutdown_connections 2024-04-21T08:46:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.200+0000 7effecd3d640 1 --2- 172.21.15.43:0/1566257410 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7effd40a3da0 0x7effd40a6260 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.200+0000 7effecd3d640 1 --2- 172.21.15.43:0/1566257410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effe8108060 0x7effe819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.200+0000 7effecd3d640 1 --2- 172.21.15.43:0/1566257410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effe806af70 0x7effe8198c40 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.200+0000 7effecd3d640 1 --2- 172.21.15.43:0/1566257410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effe806a5b0 0x7effe8198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.200+0000 7effecd3d640 1 -- 172.21.15.43:0/1566257410 >> 172.21.15.43:0/1566257410 conn(0x7effe8077c40 msgr2=0x7effe80ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.200+0000 7effecd3d640 1 -- 172.21.15.43:0/1566257410 shutdown_connections 2024-04-21T08:46:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.200+0000 7effecd3d640 1 -- 172.21.15.43:0/1566257410 wait complete. 2024-04-21T08:46:09.210 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:46:09.210 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:46:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.313+0000 7fd76817e640 1 -- 172.21.15.43:0/1282653492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd760102b50 msgr2=0x7fd760106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.313+0000 7fd76817e640 1 --2- 172.21.15.43:0/1282653492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd760102b50 0x7fd760106b90 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fd750009900 tx=0x7fd75002bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.313+0000 7fd76817e640 1 -- 172.21.15.43:0/1282653492 shutdown_connections 2024-04-21T08:46:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.313+0000 7fd76817e640 1 --2- 172.21.15.43:0/1282653492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd760107270 0x7fd76010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.313+0000 7fd76817e640 1 --2- 172.21.15.43:0/1282653492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd760102b50 0x7fd760106b90 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.313+0000 7fd76817e640 1 --2- 172.21.15.43:0/1282653492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7601021a0 0x7fd760102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.313+0000 7fd76817e640 1 -- 172.21.15.43:0/1282653492 >> 172.21.15.43:0/1282653492 conn(0x7fd7600793a0 msgr2=0x7fd7600797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.313+0000 7fd76817e640 1 -- 172.21.15.43:0/1282653492 shutdown_connections 2024-04-21T08:46:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.313+0000 7fd76817e640 1 -- 172.21.15.43:0/1282653492 wait complete. 2024-04-21T08:46:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.314+0000 7fd76817e640 1 Processor -- start 2024-04-21T08:46:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.314+0000 7fd76817e640 1 -- start start 2024-04-21T08:46:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.314+0000 7fd76817e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7601021a0 0x7fd760198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.314+0000 7fd76817e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd760107270 0x7fd760198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.314+0000 7fd76817e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd76019cfd0 0x7fd76019d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.314+0000 7fd76817e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd7601121b0 con 0x7fd7601021a0 2024-04-21T08:46:09.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.314+0000 7fd76817e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd760112030 con 0x7fd76019cfd0 2024-04-21T08:46:09.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.314+0000 7fd76817e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd760112330 con 0x7fd760107270 2024-04-21T08:46:09.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.314+0000 7fd765f1a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7601021a0 0x7fd760198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.314+0000 7fd765f1a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7601021a0 0x7fd760198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37376/0 (socket says 172.21.15.43:37376) 2024-04-21T08:46:09.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.314+0000 7fd765f1a640 1 -- 172.21.15.43:0/2811523957 learned_addr learned my addr 172.21.15.43:0/2811523957 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.314+0000 7fd76671b640 1 --2- 172.21.15.43:0/2811523957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd76019cfd0 0x7fd76019d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.314+0000 7fd765719640 1 --2- 172.21.15.43:0/2811523957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd760107270 0x7fd760198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.314+0000 7fd765f1a640 1 -- 172.21.15.43:0/2811523957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd760107270 msgr2=0x7fd760198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.314+0000 7fd765f1a640 1 --2- 172.21.15.43:0/2811523957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd760107270 0x7fd760198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.314+0000 7fd765f1a640 1 -- 172.21.15.43:0/2811523957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd76019cfd0 msgr2=0x7fd76019d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.314+0000 7fd765f1a640 1 --2- 172.21.15.43:0/2811523957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd76019cfd0 0x7fd76019d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.315+0000 7fd765f1a640 1 -- 172.21.15.43:0/2811523957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76019dac0 con 0x7fd7601021a0 2024-04-21T08:46:09.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.315+0000 7fd765f1a640 1 --2- 172.21.15.43:0/2811523957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7601021a0 0x7fd760198710 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fd75800eab0 tx=0x7fd75800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.315+0000 7fd756ffd640 1 -- 172.21.15.43:0/2811523957 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75800ce40 con 0x7fd7601021a0 2024-04-21T08:46:09.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.315+0000 7fd76817e640 1 -- 172.21.15.43:0/2811523957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd76019ddb0 con 0x7fd7601021a0 2024-04-21T08:46:09.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.315+0000 7fd756ffd640 1 -- 172.21.15.43:0/2811523957 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd758011920 con 0x7fd7601021a0 2024-04-21T08:46:09.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.315+0000 7fd756ffd640 1 -- 172.21.15.43:0/2811523957 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd758010a30 con 0x7fd7601021a0 2024-04-21T08:46:09.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.315+0000 7fd76817e640 1 -- 172.21.15.43:0/2811523957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7601a5570 con 0x7fd7601021a0 2024-04-21T08:46:09.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.316+0000 7fd76817e640 1 -- 172.21.15.43:0/2811523957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd760069230 con 0x7fd7601021a0 2024-04-21T08:46:09.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.317+0000 7fd756ffd640 1 -- 172.21.15.43:0/2811523957 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd758010bd0 con 0x7fd7601021a0 2024-04-21T08:46:09.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.318+0000 7fd756ffd640 1 --2- 172.21.15.43:0/2811523957 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd74c0a3d20 0x7fd74c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.318+0000 7fd765719640 1 --2- 172.21.15.43:0/2811523957 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd74c0a3d20 0x7fd74c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.318+0000 7fd756ffd640 1 -- 172.21.15.43:0/2811523957 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7fd7580d1240 con 0x7fd7601021a0 2024-04-21T08:46:09.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.318+0000 7fd765719640 1 --2- 172.21.15.43:0/2811523957 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd74c0a3d20 0x7fd74c0a61e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd750009230 tx=0x7fd750005fd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.321+0000 7fd756ffd640 1 -- 172.21.15.43:0/2811523957 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd758016030 con 0x7fd7601021a0 2024-04-21T08:46:09.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.546+0000 7fd76817e640 1 -- 172.21.15.43:0/2811523957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd76010a500 con 0x7fd7601021a0 2024-04-21T08:46:09.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.547+0000 7fd756ffd640 1 -- 172.21.15.43:0/2811523957 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11938 (secure 0 0 0) 0x7fd75809d400 con 0x7fd7601021a0 2024-04-21T08:46:09.547 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:46:09.547 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:38:53.219677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/3198693848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3198693848},{"type":"v1","addr":"172.21.15.175:6837","nonce":3198693848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24412,24425,24487,24491,24527]},"id":1}]} 2024-04-21T08:46:09.547 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-04-21T08:46:09.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.549+0000 7fd76817e640 1 -- 172.21.15.43:0/2811523957 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd74c0a3d20 msgr2=0x7fd74c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.549+0000 7fd76817e640 1 --2- 172.21.15.43:0/2811523957 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd74c0a3d20 0x7fd74c0a61e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd750009230 tx=0x7fd750005fd0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.549+0000 7fd76817e640 1 -- 172.21.15.43:0/2811523957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7601021a0 msgr2=0x7fd760198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.549+0000 7fd76817e640 1 --2- 172.21.15.43:0/2811523957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7601021a0 0x7fd760198710 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fd75800eab0 tx=0x7fd75800ef70 comp rx=0 tx=0).stop 2024-04-21T08:46:09.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.549+0000 7fd76817e640 1 -- 172.21.15.43:0/2811523957 shutdown_connections 2024-04-21T08:46:09.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.549+0000 7fd76817e640 1 --2- 172.21.15.43:0/2811523957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd76019cfd0 0x7fd76019d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.549+0000 7fd76817e640 1 --2- 172.21.15.43:0/2811523957 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd74c0a3d20 0x7fd74c0a61e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.549+0000 7fd76817e640 1 --2- 172.21.15.43:0/2811523957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd760107270 0x7fd760198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.549+0000 7fd76817e640 1 --2- 172.21.15.43:0/2811523957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7601021a0 0x7fd760198710 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.549+0000 7fd76817e640 1 -- 172.21.15.43:0/2811523957 >> 172.21.15.43:0/2811523957 conn(0x7fd7600793a0 msgr2=0x7fd7600fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.549+0000 7fd76817e640 1 -- 172.21.15.43:0/2811523957 shutdown_connections 2024-04-21T08:46:09.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.549+0000 7fd76817e640 1 -- 172.21.15.43:0/2811523957 wait complete. 2024-04-21T08:46:09.560 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:46:09.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.665+0000 7f9252d59640 1 -- 172.21.15.43:0/3662704573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924c101090 msgr2=0x7f924c101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.665+0000 7f9252d59640 1 --2- 172.21.15.43:0/3662704573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924c101090 0x7f924c101470 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f9248009900 tx=0x7f924802bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:09.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.665+0000 7f9252d59640 1 -- 172.21.15.43:0/3662704573 shutdown_connections 2024-04-21T08:46:09.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.665+0000 7f9252d59640 1 --2- 172.21.15.43:0/3662704573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f924c106160 0x7f924c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.665+0000 7f9252d59640 1 --2- 172.21.15.43:0/3662704573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f924c101a40 0x7f924c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.665+0000 7f9252d59640 1 --2- 172.21.15.43:0/3662704573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924c101090 0x7f924c101470 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.665+0000 7f9252d59640 1 -- 172.21.15.43:0/3662704573 >> 172.21.15.43:0/3662704573 conn(0x7f924c077f50 msgr2=0x7f924c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.665+0000 7f9252d59640 1 -- 172.21.15.43:0/3662704573 shutdown_connections 2024-04-21T08:46:09.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.665+0000 7f9252d59640 1 -- 172.21.15.43:0/3662704573 wait complete. 2024-04-21T08:46:09.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.665+0000 7f9252d59640 1 Processor -- start 2024-04-21T08:46:09.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.665+0000 7f9252d59640 1 -- start start 2024-04-21T08:46:09.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.666+0000 7f9252d59640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f924c101090 0x7f924c197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.666+0000 7f9252d59640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924c106160 0x7f924c197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.666+0000 7f9252d59640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f924c19bf20 0x7f924c19c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.666+0000 7f9252d59640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1110a0 con 0x7f924c106160 2024-04-21T08:46:09.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.666+0000 7f9252d59640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f924c110f20 con 0x7f924c101090 2024-04-21T08:46:09.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.666+0000 7f9252d59640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f924c111220 con 0x7f924c19bf20 2024-04-21T08:46:09.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.666+0000 7f9243fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924c106160 0x7f924c197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.666+0000 7f9243fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924c106160 0x7f924c197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37396/0 (socket says 172.21.15.43:37396) 2024-04-21T08:46:09.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.666+0000 7f9243fff640 1 -- 172.21.15.43:0/3549375510 learned_addr learned my addr 172.21.15.43:0/3549375510 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.666+0000 7f9250af5640 1 --2- 172.21.15.43:0/3549375510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f924c101090 0x7f924c197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.666+0000 7f92512f6640 1 --2- 172.21.15.43:0/3549375510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f924c19bf20 0x7f924c19c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.666+0000 7f9243fff640 1 -- 172.21.15.43:0/3549375510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f924c19bf20 msgr2=0x7f924c19c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.666+0000 7f9243fff640 1 --2- 172.21.15.43:0/3549375510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f924c19bf20 0x7f924c19c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.666+0000 7f9243fff640 1 -- 172.21.15.43:0/3549375510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f924c101090 msgr2=0x7f924c197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.666+0000 7f9243fff640 1 --2- 172.21.15.43:0/3549375510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f924c101090 0x7f924c197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.666+0000 7f9243fff640 1 -- 172.21.15.43:0/3549375510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924c19ca10 con 0x7f924c106160 2024-04-21T08:46:09.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.666+0000 7f9243fff640 1 --2- 172.21.15.43:0/3549375510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924c106160 0x7f924c197ba0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f923c007920 tx=0x7f923c007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.666+0000 7f9241ffb640 1 -- 172.21.15.43:0/3549375510 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c019020 con 0x7f924c106160 2024-04-21T08:46:09.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.666+0000 7f9241ffb640 1 -- 172.21.15.43:0/3549375510 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f923c0054d0 con 0x7f924c106160 2024-04-21T08:46:09.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.666+0000 7f9252d59640 1 -- 172.21.15.43:0/3549375510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924c19cd00 con 0x7f924c106160 2024-04-21T08:46:09.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.666+0000 7f9241ffb640 1 -- 172.21.15.43:0/3549375510 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c00e6f0 con 0x7f924c106160 2024-04-21T08:46:09.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.667+0000 7f9252d59640 1 -- 172.21.15.43:0/3549375510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f924c069450 con 0x7f924c106160 2024-04-21T08:46:09.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.669+0000 7f9241ffb640 1 -- 172.21.15.43:0/3549375510 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f923c0046c0 con 0x7f924c106160 2024-04-21T08:46:09.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.669+0000 7f9255a1f740 1 Processor -- start 2024-04-21T08:46:09.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.669+0000 7f9255a1f740 1 -- start start 2024-04-21T08:46:09.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.669+0000 7f9241ffb640 1 --2- 172.21.15.43:0/3549375510 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f92340a3da0 0x7f92340a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.669+0000 7f9255a1f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d5244d0 0x555c8d5248b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.669+0000 7f9255a1f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8d524e80 0x555c8d52d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.669+0000 7f9255a1f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d52da00 0x555c8d52fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.669+0000 7f9255a1f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555c8d4891f0 con 0x555c8d5244d0 2024-04-21T08:46:09.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.669+0000 7f9255a1f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555c8d4264b0 con 0x555c8d52da00 2024-04-21T08:46:09.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.669+0000 7f9255a1f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555c8d4851b0 con 0x555c8d524e80 2024-04-21T08:46:09.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.669+0000 7f92512f6640 1 --2- 172.21.15.43:0/3549375510 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f92340a3da0 0x7f92340a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.669+0000 7f9241ffb640 1 -- 172.21.15.43:0/3549375510 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7f923c0d20a0 con 0x7f924c106160 2024-04-21T08:46:09.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.669+0000 7f9250af5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d5244d0 0x555c8d5248b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.669+0000 7f9250af5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d5244d0 0x555c8d5248b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37404/0 (socket says 172.21.15.43:37404) 2024-04-21T08:46:09.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.669+0000 7f9250af5640 1 -- 172.21.15.43:0/604491933 learned_addr learned my addr 172.21.15.43:0/604491933 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.670+0000 7f9243fff640 1 --2- 172.21.15.43:0/604491933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8d524e80 0x555c8d52d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.670+0000 7f9250af5640 1 -- 172.21.15.43:0/604491933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8d524e80 msgr2=0x555c8d52d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.670+0000 7f9250af5640 1 --2- 172.21.15.43:0/604491933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8d524e80 0x555c8d52d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.670+0000 7f9250af5640 1 -- 172.21.15.43:0/604491933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d52da00 msgr2=0x555c8d52fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.670+0000 7f9250af5640 1 --2- 172.21.15.43:0/604491933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d52da00 0x555c8d52fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.670+0000 7f9250af5640 1 -- 172.21.15.43:0/604491933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c8d4181b0 con 0x555c8d5244d0 2024-04-21T08:46:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.670+0000 7f92512f6640 1 --2- 172.21.15.43:0/3549375510 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f92340a3da0 0x7f92340a6260 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9244005fd0 tx=0x7f9244009a60 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.670+0000 7f9250af5640 1 --2- 172.21.15.43:0/604491933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d5244d0 0x555c8d5248b0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f9248009900 tx=0x7f9248004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43976b3705a36ed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.670+0000 7f9232ffd640 1 -- 172.21.15.43:0/604491933 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9248045020 con 0x555c8d5244d0 2024-04-21T08:46:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.670+0000 7f9232ffd640 1 -- 172.21.15.43:0/604491933 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f924802c7e0 con 0x555c8d5244d0 2024-04-21T08:46:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.670+0000 7f9232ffd640 1 -- 172.21.15.43:0/604491933 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92480046c0 con 0x555c8d5244d0 2024-04-21T08:46:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.671+0000 7f9255a1f740 1 -- 172.21.15.43:0/604491933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d5244d0 msgr2=0x555c8d5248b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.671+0000 7f9255a1f740 1 --2- 172.21.15.43:0/604491933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d5244d0 0x555c8d5248b0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f9248009900 tx=0x7f9248004040 comp rx=0 tx=0).stop 2024-04-21T08:46:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.671+0000 7f9255a1f740 1 -- 172.21.15.43:0/604491933 shutdown_connections 2024-04-21T08:46:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.671+0000 7f9255a1f740 1 --2- 172.21.15.43:0/604491933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d52da00 0x555c8d52fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.671+0000 7f9255a1f740 1 --2- 172.21.15.43:0/604491933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8d524e80 0x555c8d52d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.671+0000 7f9255a1f740 1 --2- 172.21.15.43:0/604491933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d5244d0 0x555c8d5248b0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.671+0000 7f9255a1f740 1 -- 172.21.15.43:0/604491933 >> 172.21.15.43:0/604491933 conn(0x555c8d51eba0 msgr2=0x555c8d51ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.671+0000 7f9255a1f740 1 -- 172.21.15.43:0/604491933 shutdown_connections 2024-04-21T08:46:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.671+0000 7f9255a1f740 1 -- 172.21.15.43:0/604491933 wait complete. 2024-04-21T08:46:09.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.671+0000 7f9255a1f740 1 -- start start 2024-04-21T08:46:09.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.671+0000 7f9255a1f740 1 Processor -- start 2024-04-21T08:46:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.672+0000 7f9255a1f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8d5244d0 0x555c8d52d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.672+0000 7f9255a1f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d524e80 0x555c8d52a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.672+0000 7f9255a1f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d52da00 0x555c8d52abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.672+0000 7f9255a1f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555c8d530910 con 0x555c8d524e80 2024-04-21T08:46:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.672+0000 7f9255a1f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555c8d442500 con 0x555c8d52da00 2024-04-21T08:46:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.672+0000 7f9255a1f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555c8d530a90 con 0x555c8d5244d0 2024-04-21T08:46:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.672+0000 7f9255a1f740 10 client.0 _ll_register_callbacks cb 0x555c8d410b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.672+0000 7f9243fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d524e80 0x555c8d52a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.672+0000 7f9250af5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8d5244d0 0x555c8d52d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.672+0000 7f9243fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d524e80 0x555c8d52a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37410/0 (socket says 172.21.15.43:37410) 2024-04-21T08:46:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.672+0000 7f9243fff640 1 -- 172.21.15.43:0/2192015652 learned_addr learned my addr 172.21.15.43:0/2192015652 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.672+0000 7f92512f6640 1 --2- 172.21.15.43:0/2192015652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d52da00 0x555c8d52abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.672+0000 7f9250af5640 1 -- 172.21.15.43:0/2192015652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d52da00 msgr2=0x555c8d52abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.672+0000 7f9250af5640 1 --2- 172.21.15.43:0/2192015652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d52da00 0x555c8d52abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.672+0000 7f9250af5640 1 -- 172.21.15.43:0/2192015652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d524e80 msgr2=0x555c8d52a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.672+0000 7f9250af5640 1 --2- 172.21.15.43:0/2192015652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d524e80 0x555c8d52a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.672+0000 7f9250af5640 1 -- 172.21.15.43:0/2192015652 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c8d19b3f0 con 0x555c8d5244d0 2024-04-21T08:46:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.672+0000 7f9243fff640 1 --2- 172.21.15.43:0/2192015652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d524e80 0x555c8d52a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.673+0000 7f9250af5640 1 --2- 172.21.15.43:0/2192015652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8d5244d0 0x555c8d52d2d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9248009900 tx=0x7f9248000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dbd0c98b61a0a83 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.673+0000 7f922bfff640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-04-21T08:46:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.673+0000 7f922bfff640 1 -- 172.21.15.43:0/2192015652 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92480037f0 con 0x555c8d5244d0 2024-04-21T08:46:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.673+0000 7f9255a1f740 1 -- 172.21.15.43:0/2192015652 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f924802c7e0 con 0x555c8d5244d0 2024-04-21T08:46:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.673+0000 7f922bfff640 1 -- 172.21.15.43:0/2192015652 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7f9248003990 con 0x555c8d5244d0 2024-04-21T08:46:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.673+0000 7f922bfff640 1 -- 172.21.15.43:0/2192015652 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9248058050 con 0x555c8d5244d0 2024-04-21T08:46:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.673+0000 7f9255a1f740 10 client.27481 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:09.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.673+0000 7f9255a1f740 1 -- 172.21.15.43:0/2192015652 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555c8d2657d0 con 0x555c8d5244d0 2024-04-21T08:46:09.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.673+0000 7f922bfff640 1 -- 172.21.15.43:0/2192015652 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f924802c7e0 con 0x555c8d5244d0 2024-04-21T08:46:09.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.674+0000 7f922bfff640 1 -- 172.21.15.43:0/2192015652 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9248041c90 con 0x555c8d5244d0 2024-04-21T08:46:09.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.674+0000 7f9255a1f740 10 client.27481 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:09.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.674+0000 7f9255a1f740 10 client.27481 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:46:09.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.674+0000 7f9255a1f740 20 client.27481 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:09.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.674+0000 7f9255a1f740 1 --2- 172.21.15.43:0/2192015652 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x555c8d53da00 0x555c8d53fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.674+0000 7f9255a1f740 4 client.27481 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:09.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.674+0000 7f9255a1f740 1 -- 172.21.15.43:0/2192015652 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c8d428930 con 0x555c8d53da00 2024-04-21T08:46:09.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.674+0000 7f9243fff640 1 --2- 172.21.15.43:0/2192015652 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x555c8d53da00 0x555c8d53fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.675+0000 7f922bfff640 10 client.27481 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:46:09.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.675+0000 7f9243fff640 1 --2- 172.21.15.43:0/2192015652 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x555c8d53da00 0x555c8d53fdf0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d2f11e22dc2207fb server_cookie=ffba970be73ea5eb in_seq=0 out_seq=0 2024-04-21T08:46:09.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.675+0000 7f922bfff640 1 -- 172.21.15.43:0/2192015652 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555c8d428930 con 0x555c8d53da00 2024-04-21T08:46:09.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.675+0000 7f922bfff640 10 client.27481 handle_command_reply: tid=0 2024-04-21T08:46:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.675+0000 7f9255a1f740 1 client.27481 shutdown 2024-04-21T08:46:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.676+0000 7f9255a1f740 1 -- 172.21.15.43:0/2192015652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8d5244d0 msgr2=0x555c8d52d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.676+0000 7f9255a1f740 1 --2- 172.21.15.43:0/2192015652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8d5244d0 0x555c8d52d2d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9248009900 tx=0x7f9248000bc0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.676+0000 7f9255a1f740 1 -- 172.21.15.43:0/2192015652 shutdown_connections 2024-04-21T08:46:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.676+0000 7f9255a1f740 1 --2- 172.21.15.43:0/2192015652 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x555c8d53da00 0x555c8d53fdf0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.676+0000 7f9255a1f740 1 --2- 172.21.15.43:0/2192015652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d52da00 0x555c8d52abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.676+0000 7f922bfff640 0 client.27481 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:46:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.676+0000 7f9255a1f740 1 --2- 172.21.15.43:0/2192015652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d524e80 0x555c8d52a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.676+0000 7f9255a1f740 1 --2- 172.21.15.43:0/2192015652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8d5244d0 0x555c8d52d2d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.676+0000 7f9255a1f740 1 -- 172.21.15.43:0/2192015652 >> 172.21.15.43:0/2192015652 conn(0x555c8d51eba0 msgr2=0x555c8d5208f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.676+0000 7f9255a1f740 1 -- 172.21.15.43:0/2192015652 shutdown_connections 2024-04-21T08:46:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.676+0000 7f9255a1f740 1 -- 172.21.15.43:0/2192015652 wait complete. 2024-04-21T08:46:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.676+0000 7f9255a1f740 20 client.27481 trim_cache size 0 max 16384 2024-04-21T08:46:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.689+0000 7f9255a1f740 1 Processor -- start 2024-04-21T08:46:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.689+0000 7f9255a1f740 1 -- start start 2024-04-21T08:46:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.689+0000 7f9255a1f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d5244d0 0x555c8d5397d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.689+0000 7f9255a1f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8d524e80 0x555c8d5298d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.689+0000 7f9255a1f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d52da00 0x555c8d52cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.689+0000 7f9255a1f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1ad3a0 con 0x555c8d52da00 2024-04-21T08:46:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.689+0000 7f9255a1f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1aeee0 con 0x555c8d5244d0 2024-04-21T08:46:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.689+0000 7f9255a1f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1ada70 con 0x555c8d524e80 2024-04-21T08:46:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.689+0000 7f92512f6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d52da00 0x555c8d52cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.689+0000 7f92512f6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d52da00 0x555c8d52cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37422/0 (socket says 172.21.15.43:37422) 2024-04-21T08:46:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.689+0000 7f92512f6640 1 -- 172.21.15.43:0/1496320601 learned_addr learned my addr 172.21.15.43:0/1496320601 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.689+0000 7f9250af5640 1 --2- 172.21.15.43:0/1496320601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d5244d0 0x555c8d5397d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.689+0000 7f9243fff640 1 --2- 172.21.15.43:0/1496320601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8d524e80 0x555c8d5298d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.690+0000 7f92512f6640 1 -- 172.21.15.43:0/1496320601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8d524e80 msgr2=0x555c8d5298d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.690+0000 7f92512f6640 1 --2- 172.21.15.43:0/1496320601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8d524e80 0x555c8d5298d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.690+0000 7f92512f6640 1 -- 172.21.15.43:0/1496320601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d5244d0 msgr2=0x555c8d5397d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.690+0000 7f92512f6640 1 --2- 172.21.15.43:0/1496320601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d5244d0 0x555c8d5397d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.690+0000 7f92512f6640 1 -- 172.21.15.43:0/1496320601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c8d524c20 con 0x555c8d52da00 2024-04-21T08:46:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.690+0000 7f9250af5640 1 --2- 172.21.15.43:0/1496320601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d5244d0 0x555c8d5397d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.690+0000 7f92512f6640 1 --2- 172.21.15.43:0/1496320601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d52da00 0x555c8d52cd90 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f92440137e0 tx=0x7f9244015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25bf08dbaaf4941 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.690+0000 7f92327fc640 1 -- 172.21.15.43:0/1496320601 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924400f8e0 con 0x555c8d52da00 2024-04-21T08:46:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.690+0000 7f92327fc640 1 -- 172.21.15.43:0/1496320601 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9244017070 con 0x555c8d52da00 2024-04-21T08:46:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.690+0000 7f92327fc640 1 -- 172.21.15.43:0/1496320601 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9244021400 con 0x555c8d52da00 2024-04-21T08:46:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.691+0000 7f9255a1f740 1 -- 172.21.15.43:0/1496320601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d52da00 msgr2=0x555c8d52cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.691+0000 7f9255a1f740 1 --2- 172.21.15.43:0/1496320601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d52da00 0x555c8d52cd90 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f92440137e0 tx=0x7f9244015000 comp rx=0 tx=0).stop 2024-04-21T08:46:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.691+0000 7f9255a1f740 1 -- 172.21.15.43:0/1496320601 shutdown_connections 2024-04-21T08:46:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.691+0000 7f9255a1f740 1 --2- 172.21.15.43:0/1496320601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d52da00 0x555c8d52cd90 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.691+0000 7f9255a1f740 1 --2- 172.21.15.43:0/1496320601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8d524e80 0x555c8d5298d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.691+0000 7f9255a1f740 1 --2- 172.21.15.43:0/1496320601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d5244d0 0x555c8d5397d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.691+0000 7f9255a1f740 1 -- 172.21.15.43:0/1496320601 >> 172.21.15.43:0/1496320601 conn(0x555c8d51eba0 msgr2=0x555c8d51dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.691+0000 7f9255a1f740 1 -- 172.21.15.43:0/1496320601 shutdown_connections 2024-04-21T08:46:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.691+0000 7f9255a1f740 1 -- 172.21.15.43:0/1496320601 wait complete. 2024-04-21T08:46:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.691+0000 7f9255a1f740 1 -- start start 2024-04-21T08:46:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.691+0000 7f9255a1f740 1 Processor -- start 2024-04-21T08:46:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.691+0000 7f9255a1f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d5244d0 0x555c8d535ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.691+0000 7f9255a1f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d524e80 0x555c8d5361e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.691+0000 7f9255a1f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8d52da00 0x555c8d5312d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.691+0000 7f9255a1f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1ae810 con 0x555c8d524e80 2024-04-21T08:46:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.691+0000 7f9255a1f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1ae140 con 0x555c8d5244d0 2024-04-21T08:46:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.691+0000 7f9255a1f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1ab3a0 con 0x555c8d52da00 2024-04-21T08:46:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.692+0000 7f9255a1f740 10 client.0 _ll_register_callbacks cb 0x7f924802cd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.692+0000 7f9243fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d524e80 0x555c8d5361e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.692+0000 7f9243fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d524e80 0x555c8d5361e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37434/0 (socket says 172.21.15.43:37434) 2024-04-21T08:46:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.692+0000 7f9243fff640 1 -- 172.21.15.43:0/2249798679 learned_addr learned my addr 172.21.15.43:0/2249798679 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.692+0000 7f9250af5640 1 --2- 172.21.15.43:0/2249798679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d5244d0 0x555c8d535ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.692+0000 7f9243fff640 1 -- 172.21.15.43:0/2249798679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8d52da00 msgr2=0x555c8d5312d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.692+0000 7f9243fff640 1 --2- 172.21.15.43:0/2249798679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8d52da00 0x555c8d5312d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.692+0000 7f9243fff640 1 -- 172.21.15.43:0/2249798679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d5244d0 msgr2=0x555c8d535ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.692+0000 7f9243fff640 1 --2- 172.21.15.43:0/2249798679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d5244d0 0x555c8d535ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.692+0000 7f9243fff640 1 -- 172.21.15.43:0/2249798679 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c8d531b00 con 0x555c8d524e80 2024-04-21T08:46:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.692+0000 7f9243fff640 1 --2- 172.21.15.43:0/2249798679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d524e80 0x555c8d5361e0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f923c0cf970 tx=0x7f923c00c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632da9af89098935 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.692+0000 7f922bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.692+0000 7f922bfff640 1 -- 172.21.15.43:0/2249798679 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c0be0d0 con 0x555c8d524e80 2024-04-21T08:46:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.692+0000 7f9255a1f740 1 -- 172.21.15.43:0/2249798679 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9244017070 con 0x555c8d524e80 2024-04-21T08:46:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.692+0000 7f922bfff640 1 -- 172.21.15.43:0/2249798679 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7f923c09a2c0 con 0x555c8d524e80 2024-04-21T08:46:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.692+0000 7f922bfff640 1 -- 172.21.15.43:0/2249798679 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f923c09d0a0 con 0x555c8d524e80 2024-04-21T08:46:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.692+0000 7f9255a1f740 10 client.18957 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.692+0000 7f9255a1f740 1 -- 172.21.15.43:0/2249798679 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555c8d5a39b0 con 0x555c8d524e80 2024-04-21T08:46:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.692+0000 7f922bfff640 1 -- 172.21.15.43:0/2249798679 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9244017070 con 0x555c8d524e80 2024-04-21T08:46:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.693+0000 7f922bfff640 1 -- 172.21.15.43:0/2249798679 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f923c014070 con 0x555c8d524e80 2024-04-21T08:46:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.693+0000 7f9255a1f740 10 client.18957 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.693+0000 7f9255a1f740 10 client.18957 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:46:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.693+0000 7f9255a1f740 20 client.18957 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.693+0000 7f9255a1f740 1 --2- 172.21.15.43:0/2249798679 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x555c8d5a4130 0x555c8d5a6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.693+0000 7f9255a1f740 4 client.18957 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.693+0000 7f9255a1f740 1 -- 172.21.15.43:0/2249798679 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c8d496c10 con 0x555c8d5a4130 2024-04-21T08:46:09.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.694+0000 7f9250af5640 1 --2- 172.21.15.43:0/2249798679 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x555c8d5a4130 0x555c8d5a6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.696+0000 7f9250af5640 1 --2- 172.21.15.43:0/2249798679 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x555c8d5a4130 0x555c8d5a6550 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9855ad5b607c56c0 server_cookie=3167373b1919de07 in_seq=0 out_seq=0 2024-04-21T08:46:09.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.696+0000 7f922bfff640 10 client.18957 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:46:09.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.697+0000 7f922bfff640 1 -- 172.21.15.43:0/2249798679 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c8d496c10 con 0x555c8d5a4130 2024-04-21T08:46:09.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.697+0000 7f922bfff640 10 client.18957 handle_command_reply: tid=0 2024-04-21T08:46:09.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.697+0000 7f9255a1f740 1 client.18957 shutdown 2024-04-21T08:46:09.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.697+0000 7f9255a1f740 1 -- 172.21.15.43:0/2249798679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d524e80 msgr2=0x555c8d5361e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.697+0000 7f9255a1f740 1 --2- 172.21.15.43:0/2249798679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d524e80 0x555c8d5361e0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f923c0cf970 tx=0x7f923c00c580 comp rx=0 tx=0).stop 2024-04-21T08:46:09.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.697+0000 7f9255a1f740 1 -- 172.21.15.43:0/2249798679 shutdown_connections 2024-04-21T08:46:09.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.697+0000 7f9255a1f740 1 --2- 172.21.15.43:0/2249798679 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x555c8d5a4130 0x555c8d5a6550 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.697+0000 7f9255a1f740 1 --2- 172.21.15.43:0/2249798679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c8d52da00 0x555c8d5312d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.697+0000 7f9255a1f740 1 --2- 172.21.15.43:0/2249798679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c8d524e80 0x555c8d5361e0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.697+0000 7f922bfff640 0 client.18957 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:46:09.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.697+0000 7f9255a1f740 1 --2- 172.21.15.43:0/2249798679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c8d5244d0 0x555c8d535ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.697+0000 7f9255a1f740 1 -- 172.21.15.43:0/2249798679 >> 172.21.15.43:0/2249798679 conn(0x555c8d51eba0 msgr2=0x555c8d525260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.697+0000 7f9255a1f740 1 -- 172.21.15.43:0/2249798679 shutdown_connections 2024-04-21T08:46:09.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.697+0000 7f9255a1f740 1 -- 172.21.15.43:0/2249798679 wait complete. 2024-04-21T08:46:09.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.697+0000 7f9255a1f740 20 client.18957 trim_cache size 0 max 16384 2024-04-21T08:46:09.701 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:46:09.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.698+0000 7f9252d59640 1 -- 172.21.15.43:0/3549375510 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f92340a3da0 msgr2=0x7f92340a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.698+0000 7f9252d59640 1 --2- 172.21.15.43:0/3549375510 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f92340a3da0 0x7f92340a6260 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9244005fd0 tx=0x7f9244009a60 comp rx=0 tx=0).stop 2024-04-21T08:46:09.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.698+0000 7f9252d59640 1 -- 172.21.15.43:0/3549375510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924c106160 msgr2=0x7f924c197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.698+0000 7f9252d59640 1 --2- 172.21.15.43:0/3549375510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924c106160 0x7f924c197ba0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f923c007920 tx=0x7f923c007de0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.698+0000 7f9252d59640 1 -- 172.21.15.43:0/3549375510 shutdown_connections 2024-04-21T08:46:09.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.698+0000 7f9252d59640 1 --2- 172.21.15.43:0/3549375510 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f92340a3da0 0x7f92340a6260 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.698+0000 7f9252d59640 1 --2- 172.21.15.43:0/3549375510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f924c19bf20 0x7f924c19c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.698+0000 7f9252d59640 1 --2- 172.21.15.43:0/3549375510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924c106160 0x7f924c197ba0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.698+0000 7f9252d59640 1 --2- 172.21.15.43:0/3549375510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f924c101090 0x7f924c197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.698+0000 7f9252d59640 1 -- 172.21.15.43:0/3549375510 >> 172.21.15.43:0/3549375510 conn(0x7f924c077f50 msgr2=0x7f924c0ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.698+0000 7f9252d59640 1 -- 172.21.15.43:0/3549375510 shutdown_connections 2024-04-21T08:46:09.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.698+0000 7f9252d59640 1 -- 172.21.15.43:0/3549375510 wait complete. 2024-04-21T08:46:09.708 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:46:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.813+0000 7f749565c640 1 -- 172.21.15.43:0/1919952503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7490107260 msgr2=0x7f749010ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.813+0000 7f749565c640 1 --2- 172.21.15.43:0/1919952503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7490107260 0x7f749010ee30 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f7488009ca0 tx=0x7f748802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.813+0000 7f749565c640 1 -- 172.21.15.43:0/1919952503 shutdown_connections 2024-04-21T08:46:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.813+0000 7f749565c640 1 --2- 172.21.15.43:0/1919952503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7490107260 0x7f749010ee30 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.813+0000 7f749565c640 1 --2- 172.21.15.43:0/1919952503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7490102b40 0x7f7490106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.813+0000 7f749565c640 1 --2- 172.21.15.43:0/1919952503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7490102190 0x7f7490102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.813+0000 7f749565c640 1 -- 172.21.15.43:0/1919952503 >> 172.21.15.43:0/1919952503 conn(0x7f74900793a0 msgr2=0x7f74900797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.813+0000 7f749565c640 1 -- 172.21.15.43:0/1919952503 shutdown_connections 2024-04-21T08:46:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.813+0000 7f749565c640 1 -- 172.21.15.43:0/1919952503 wait complete. 2024-04-21T08:46:09.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.814+0000 7f749565c640 1 Processor -- start 2024-04-21T08:46:09.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.814+0000 7f749565c640 1 -- start start 2024-04-21T08:46:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.814+0000 7f749565c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7490102190 0x7f749019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.814+0000 7f749565c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7490107260 0x7f749019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.814+0000 7f749565c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7490198590 0x7f7490198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.814+0000 7f749565c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f74901121d0 con 0x7f7490107260 2024-04-21T08:46:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.814+0000 7f749565c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7490112050 con 0x7f7490102190 2024-04-21T08:46:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.814+0000 7f749565c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7490112350 con 0x7f7490198590 2024-04-21T08:46:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.814+0000 7f748e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7490107260 0x7f749019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.814+0000 7f748e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7490107260 0x7f749019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37442/0 (socket says 172.21.15.43:37442) 2024-04-21T08:46:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.814+0000 7f748e7fc640 1 -- 172.21.15.43:0/1874474437 learned_addr learned my addr 172.21.15.43:0/1874474437 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.814+0000 7f748effd640 1 --2- 172.21.15.43:0/1874474437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7490102190 0x7f749019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.814+0000 7f748f7fe640 1 --2- 172.21.15.43:0/1874474437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7490198590 0x7f7490198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.814+0000 7f748e7fc640 1 -- 172.21.15.43:0/1874474437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7490198590 msgr2=0x7f7490198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.814+0000 7f748e7fc640 1 --2- 172.21.15.43:0/1874474437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7490198590 0x7f7490198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.814+0000 7f748e7fc640 1 -- 172.21.15.43:0/1874474437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7490102190 msgr2=0x7f749019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.814+0000 7f748e7fc640 1 --2- 172.21.15.43:0/1874474437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7490102190 0x7f749019e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.814+0000 7f748e7fc640 1 -- 172.21.15.43:0/1874474437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74901990b0 con 0x7f7490107260 2024-04-21T08:46:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.814+0000 7f748f7fe640 1 --2- 172.21.15.43:0/1874474437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7490198590 0x7f7490198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.814+0000 7f748effd640 1 --2- 172.21.15.43:0/1874474437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7490102190 0x7f749019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.815+0000 7f748e7fc640 1 --2- 172.21.15.43:0/1874474437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7490107260 0x7f749019e920 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f748400d770 tx=0x7f748400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.815+0000 7f7477fff640 1 -- 172.21.15.43:0/1874474437 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7484017070 con 0x7f7490107260 2024-04-21T08:46:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.815+0000 7f749565c640 1 -- 172.21.15.43:0/1874474437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74901993a0 con 0x7f7490107260 2024-04-21T08:46:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.815+0000 7f7477fff640 1 -- 172.21.15.43:0/1874474437 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7484009c30 con 0x7f7490107260 2024-04-21T08:46:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.815+0000 7f7477fff640 1 -- 172.21.15.43:0/1874474437 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748400f620 con 0x7f7490107260 2024-04-21T08:46:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.815+0000 7f749565c640 1 -- 172.21.15.43:0/1874474437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74901a5640 con 0x7f7490107260 2024-04-21T08:46:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.817+0000 7f7477fff640 1 -- 172.21.15.43:0/1874474437 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f748401a070 con 0x7f7490107260 2024-04-21T08:46:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.817+0000 7f7498252740 1 Processor -- start 2024-04-21T08:46:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.818+0000 7f7498252740 1 -- start start 2024-04-21T08:46:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.818+0000 7f7498252740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be48cc63b0 0x55be48cc6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.818+0000 7f7498252740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be48cc6d60 0x55be48ccf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.818+0000 7f7498252740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48ccfe60 0x55be48cd02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.818+0000 7f7498252740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55be48c2b1f0 con 0x55be48ccfe60 2024-04-21T08:46:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.818+0000 7f7498252740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55be48bc84b0 con 0x55be48cc6d60 2024-04-21T08:46:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.818+0000 7f7498252740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55be48c271b0 con 0x55be48cc63b0 2024-04-21T08:46:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.818+0000 7f7477fff640 1 --2- 172.21.15.43:0/1874474437 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f74780a3da0 0x7f74780a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.818+0000 7f748e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48ccfe60 0x55be48cd02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.818+0000 7f748e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48ccfe60 0x55be48cd02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37450/0 (socket says 172.21.15.43:37450) 2024-04-21T08:46:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.818+0000 7f748e7fc640 1 -- 172.21.15.43:0/220952065 learned_addr learned my addr 172.21.15.43:0/220952065 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.818+0000 7f748f7fe640 1 --2- 172.21.15.43:0/220952065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be48cc63b0 0x55be48cc6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.818+0000 7f748effd640 1 --2- 172.21.15.43:0/220952065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be48cc6d60 0x55be48ccf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.818+0000 7f748f7fe640 1 --2- 172.21.15.43:0/1874474437 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f74780a3da0 0x7f74780a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.818+0000 7f7477fff640 1 -- 172.21.15.43:0/1874474437 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7f74840d0780 con 0x7f7490107260 2024-04-21T08:46:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.818+0000 7f748effd640 1 -- 172.21.15.43:0/220952065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be48cc63b0 msgr2=0x55be48cc6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.818+0000 7f748effd640 1 --2- 172.21.15.43:0/220952065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be48cc63b0 0x55be48cc6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.818+0000 7f748effd640 1 -- 172.21.15.43:0/220952065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48ccfe60 msgr2=0x55be48cd02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.818+0000 7f748effd640 1 --2- 172.21.15.43:0/220952065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48ccfe60 0x55be48cd02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.818+0000 7f748effd640 1 -- 172.21.15.43:0/220952065 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be48bba1b0 con 0x55be48cc6d60 2024-04-21T08:46:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.818+0000 7f748e7fc640 1 --2- 172.21.15.43:0/220952065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48ccfe60 0x55be48cd02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.818+0000 7f748f7fe640 1 --2- 172.21.15.43:0/1874474437 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f74780a3da0 0x7f74780a6260 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7488004af0 tx=0x7f748802c0f0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.819+0000 7f748effd640 1 --2- 172.21.15.43:0/220952065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be48cc6d60 0x55be48ccf890 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f748000bc80 tx=0x7f748000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1e9c43d0d01bae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.819+0000 7f74757fa640 1 -- 172.21.15.43:0/220952065 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480015070 con 0x55be48cc6d60 2024-04-21T08:46:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.819+0000 7f74757fa640 1 -- 172.21.15.43:0/220952065 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74800118f0 con 0x55be48cc6d60 2024-04-21T08:46:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.819+0000 7f74757fa640 1 -- 172.21.15.43:0/220952065 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480010870 con 0x55be48cc6d60 2024-04-21T08:46:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.819+0000 7f7498252740 1 -- 172.21.15.43:0/220952065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be48cc6d60 msgr2=0x55be48ccf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.819+0000 7f7498252740 1 --2- 172.21.15.43:0/220952065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be48cc6d60 0x55be48ccf890 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f748000bc80 tx=0x7f748000f730 comp rx=0 tx=0).stop 2024-04-21T08:46:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.819+0000 7f7498252740 1 -- 172.21.15.43:0/220952065 shutdown_connections 2024-04-21T08:46:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.819+0000 7f7498252740 1 --2- 172.21.15.43:0/220952065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48ccfe60 0x55be48cd02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.819+0000 7f7498252740 1 --2- 172.21.15.43:0/220952065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be48cc6d60 0x55be48ccf890 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.819+0000 7f7498252740 1 --2- 172.21.15.43:0/220952065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be48cc63b0 0x55be48cc6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.819+0000 7f7498252740 1 -- 172.21.15.43:0/220952065 >> 172.21.15.43:0/220952065 conn(0x55be48cc0ba0 msgr2=0x55be48cc0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.819+0000 7f7498252740 1 -- 172.21.15.43:0/220952065 shutdown_connections 2024-04-21T08:46:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.820+0000 7f7498252740 1 -- 172.21.15.43:0/220952065 wait complete. 2024-04-21T08:46:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.820+0000 7f7498252740 1 -- start start 2024-04-21T08:46:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.820+0000 7f7498252740 1 Processor -- start 2024-04-21T08:46:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.820+0000 7f7498252740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48cc63b0 0x55be48cd6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.820+0000 7f7498252740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be48cc6d60 0x55be48cd3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.820+0000 7f7498252740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be48ccfe60 0x55be48cd3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.820+0000 7f7498252740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55be48cd6da0 con 0x55be48cc63b0 2024-04-21T08:46:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.820+0000 7f7498252740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55be48be4500 con 0x55be48ccfe60 2024-04-21T08:46:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.820+0000 7f7498252740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55be48cd6f20 con 0x55be48cc6d60 2024-04-21T08:46:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.820+0000 7f7498252740 10 client.0 _ll_register_callbacks cb 0x55be48bb2b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.820+0000 7f748effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48cc63b0 0x55be48cd6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.820+0000 7f748effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48cc63b0 0x55be48cd6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37452/0 (socket says 172.21.15.43:37452) 2024-04-21T08:46:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.820+0000 7f748effd640 1 -- 172.21.15.43:0/3715900472 learned_addr learned my addr 172.21.15.43:0/3715900472 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.820+0000 7f748f7fe640 1 --2- 172.21.15.43:0/3715900472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be48ccfe60 0x55be48cd3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.820+0000 7f748effd640 1 -- 172.21.15.43:0/3715900472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be48cc6d60 msgr2=0x55be48cd3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.820+0000 7f748e7fc640 1 --2- 172.21.15.43:0/3715900472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be48cc6d60 0x55be48cd3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.820+0000 7f748effd640 1 --2- 172.21.15.43:0/3715900472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be48cc6d60 0x55be48cd3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.821+0000 7f748effd640 1 -- 172.21.15.43:0/3715900472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be48ccfe60 msgr2=0x55be48cd3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.821+0000 7f748effd640 1 --2- 172.21.15.43:0/3715900472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be48ccfe60 0x55be48cd3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.821+0000 7f748effd640 1 -- 172.21.15.43:0/3715900472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be4893d3f0 con 0x55be48cc63b0 2024-04-21T08:46:09.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.821+0000 7f748f7fe640 1 --2- 172.21.15.43:0/3715900472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be48ccfe60 0x55be48cd3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:09.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.821+0000 7f748effd640 1 --2- 172.21.15.43:0/3715900472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48cc63b0 0x55be48cd6260 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f7480010410 tx=0x7f748000bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4490c4d5e6cb18e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.821+0000 7f7472ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:09.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.821+0000 7f7472ff5640 1 -- 172.21.15.43:0/3715900472 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480028060 con 0x55be48cc63b0 2024-04-21T08:46:09.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.821+0000 7f7472ff5640 1 -- 172.21.15.43:0/3715900472 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7f7480011aa0 con 0x55be48cc63b0 2024-04-21T08:46:09.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.821+0000 7f7498252740 1 -- 172.21.15.43:0/3715900472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74800118f0 con 0x55be48cc63b0 2024-04-21T08:46:09.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.821+0000 7f7472ff5640 1 -- 172.21.15.43:0/3715900472 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f748002f050 con 0x55be48cc63b0 2024-04-21T08:46:09.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.821+0000 7f7498252740 10 client.18969 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:09.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.821+0000 7f7498252740 1 -- 172.21.15.43:0/3715900472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55be48a077d0 con 0x55be48cc63b0 2024-04-21T08:46:09.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.821+0000 7f7472ff5640 1 -- 172.21.15.43:0/3715900472 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74800118f0 con 0x55be48cc63b0 2024-04-21T08:46:09.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.821+0000 7f7472ff5640 1 -- 172.21.15.43:0/3715900472 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f74800047c0 con 0x55be48cc63b0 2024-04-21T08:46:09.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.821+0000 7f7498252740 10 client.18969 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:09.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.821+0000 7f7498252740 10 client.18969 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:46:09.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.821+0000 7f7498252740 20 client.18969 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:09.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.821+0000 7f7498252740 1 --2- 172.21.15.43:0/3715900472 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55be48ce3f30 0x55be48ce6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.822+0000 7f7498252740 4 client.18969 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:09.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.822+0000 7f7498252740 1 -- 172.21.15.43:0/3715900472 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be48bca930 con 0x55be48ce3f30 2024-04-21T08:46:09.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.822+0000 7f748e7fc640 1 --2- 172.21.15.43:0/3715900472 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55be48ce3f30 0x55be48ce6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.822+0000 7f748e7fc640 1 --2- 172.21.15.43:0/3715900472 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55be48ce3f30 0x55be48ce6320 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e4da92fa806868cb server_cookie=9a70c8ce5b5e2127 in_seq=0 out_seq=0 2024-04-21T08:46:09.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.822+0000 7f7472ff5640 10 client.18969 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:46:09.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.823+0000 7f7472ff5640 1 -- 172.21.15.43:0/3715900472 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55be48bca930 con 0x55be48ce3f30 2024-04-21T08:46:09.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.823+0000 7f7472ff5640 10 client.18969 handle_command_reply: tid=0 2024-04-21T08:46:09.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.823+0000 7f7498252740 1 client.18969 shutdown 2024-04-21T08:46:09.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.823+0000 7f7498252740 1 -- 172.21.15.43:0/3715900472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48cc63b0 msgr2=0x55be48cd6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.823+0000 7f7498252740 1 --2- 172.21.15.43:0/3715900472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48cc63b0 0x55be48cd6260 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f7480010410 tx=0x7f748000bfa0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.823+0000 7f7498252740 1 -- 172.21.15.43:0/3715900472 shutdown_connections 2024-04-21T08:46:09.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.823+0000 7f7498252740 1 --2- 172.21.15.43:0/3715900472 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55be48ce3f30 0x55be48ce6320 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.823+0000 7f7498252740 1 --2- 172.21.15.43:0/3715900472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be48ccfe60 0x55be48cd3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.823+0000 7f7498252740 1 --2- 172.21.15.43:0/3715900472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be48cc6d60 0x55be48cd3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.823+0000 7f7472ff5640 0 client.18969 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:46:09.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.823+0000 7f7498252740 1 --2- 172.21.15.43:0/3715900472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48cc63b0 0x55be48cd6260 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.823+0000 7f7498252740 1 -- 172.21.15.43:0/3715900472 >> 172.21.15.43:0/3715900472 conn(0x55be48cc0ba0 msgr2=0x55be48cc28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.823+0000 7f7498252740 1 -- 172.21.15.43:0/3715900472 shutdown_connections 2024-04-21T08:46:09.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.823+0000 7f7498252740 1 -- 172.21.15.43:0/3715900472 wait complete. 2024-04-21T08:46:09.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.823+0000 7f7498252740 20 client.18969 trim_cache size 0 max 16384 2024-04-21T08:46:09.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.837+0000 7f7498252740 1 Processor -- start 2024-04-21T08:46:09.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.837+0000 7f7498252740 1 -- start start 2024-04-21T08:46:09.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.837+0000 7f7498252740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be48cc63b0 0x55be48cdfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.837+0000 7f7498252740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be48cc6d60 0x55be48ccf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.837+0000 7f7498252740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48ccfe60 0x55be48cd1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.837+0000 7f7498252740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f74901ad510 con 0x55be48ccfe60 2024-04-21T08:46:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.837+0000 7f7498252740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f74901af050 con 0x55be48cc6d60 2024-04-21T08:46:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.837+0000 7f7498252740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74901adbe0 con 0x55be48cc63b0 2024-04-21T08:46:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.837+0000 7f748f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48ccfe60 0x55be48cd1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.837+0000 7f748f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48ccfe60 0x55be48cd1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37464/0 (socket says 172.21.15.43:37464) 2024-04-21T08:46:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.837+0000 7f748f7fe640 1 -- 172.21.15.43:0/120594560 learned_addr learned my addr 172.21.15.43:0/120594560 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.837+0000 7f748effd640 1 --2- 172.21.15.43:0/120594560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be48cc63b0 0x55be48cdfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.837+0000 7f748e7fc640 1 --2- 172.21.15.43:0/120594560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be48cc6d60 0x55be48ccf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.837+0000 7f748f7fe640 1 -- 172.21.15.43:0/120594560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be48cc63b0 msgr2=0x55be48cdfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.837+0000 7f748f7fe640 1 --2- 172.21.15.43:0/120594560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be48cc63b0 0x55be48cdfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.837+0000 7f748f7fe640 1 -- 172.21.15.43:0/120594560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be48cc6d60 msgr2=0x55be48ccf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.837+0000 7f748f7fe640 1 --2- 172.21.15.43:0/120594560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be48cc6d60 0x55be48ccf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.837+0000 7f748f7fe640 1 -- 172.21.15.43:0/120594560 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be48cbc470 con 0x55be48ccfe60 2024-04-21T08:46:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.837+0000 7f748f7fe640 1 --2- 172.21.15.43:0/120594560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48ccfe60 0x55be48cd1250 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f7488009ca0 tx=0x7f7488043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836056a1c630017c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.838+0000 7f7474ff9640 1 -- 172.21.15.43:0/120594560 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7488039030 con 0x55be48ccfe60 2024-04-21T08:46:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.838+0000 7f7474ff9640 1 -- 172.21.15.43:0/120594560 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7488054070 con 0x55be48ccfe60 2024-04-21T08:46:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.838+0000 7f7474ff9640 1 -- 172.21.15.43:0/120594560 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74880043a0 con 0x55be48ccfe60 2024-04-21T08:46:09.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.838+0000 7f7498252740 1 -- 172.21.15.43:0/120594560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48ccfe60 msgr2=0x55be48cd1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.838+0000 7f7498252740 1 --2- 172.21.15.43:0/120594560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48ccfe60 0x55be48cd1250 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f7488009ca0 tx=0x7f7488043000 comp rx=0 tx=0).stop 2024-04-21T08:46:09.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.838+0000 7f7498252740 1 -- 172.21.15.43:0/120594560 shutdown_connections 2024-04-21T08:46:09.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.838+0000 7f7498252740 1 --2- 172.21.15.43:0/120594560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48ccfe60 0x55be48cd1250 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.838+0000 7f7498252740 1 --2- 172.21.15.43:0/120594560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be48cc6d60 0x55be48ccf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.838+0000 7f7498252740 1 --2- 172.21.15.43:0/120594560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be48cc63b0 0x55be48cdfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.838+0000 7f7498252740 1 -- 172.21.15.43:0/120594560 >> 172.21.15.43:0/120594560 conn(0x55be48cc0ba0 msgr2=0x55be48ccedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.838+0000 7f7498252740 1 -- 172.21.15.43:0/120594560 shutdown_connections 2024-04-21T08:46:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.838+0000 7f7498252740 1 -- 172.21.15.43:0/120594560 wait complete. 2024-04-21T08:46:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.839+0000 7f7498252740 1 -- start start 2024-04-21T08:46:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.839+0000 7f7498252740 1 Processor -- start 2024-04-21T08:46:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.839+0000 7f7498252740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be48cc63b0 0x55be48cdc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.839+0000 7f7498252740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be48cc6d60 0x55be48cdc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.839+0000 7f7498252740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48ccfe60 0x55be48cd0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.839+0000 7f7498252740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f74901ae980 con 0x55be48ccfe60 2024-04-21T08:46:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.839+0000 7f7498252740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f74901ae2b0 con 0x55be48cc63b0 2024-04-21T08:46:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.839+0000 7f7498252740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74901ab510 con 0x55be48cc6d60 2024-04-21T08:46:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.839+0000 7f7498252740 10 client.0 _ll_register_callbacks cb 0x7f748000bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.839+0000 7f748f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48ccfe60 0x55be48cd0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.839+0000 7f748f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48ccfe60 0x55be48cd0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37478/0 (socket says 172.21.15.43:37478) 2024-04-21T08:46:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.839+0000 7f748f7fe640 1 -- 172.21.15.43:0/3309055578 learned_addr learned my addr 172.21.15.43:0/3309055578 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.839+0000 7f748effd640 1 --2- 172.21.15.43:0/3309055578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be48cc63b0 0x55be48cdc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.839+0000 7f748e7fc640 1 --2- 172.21.15.43:0/3309055578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be48cc6d60 0x55be48cdc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.839+0000 7f748f7fe640 1 -- 172.21.15.43:0/3309055578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be48cc6d60 msgr2=0x55be48cdc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.839+0000 7f748f7fe640 1 --2- 172.21.15.43:0/3309055578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be48cc6d60 0x55be48cdc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.839+0000 7f748f7fe640 1 -- 172.21.15.43:0/3309055578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be48cc63b0 msgr2=0x55be48cdc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.839+0000 7f748f7fe640 1 --2- 172.21.15.43:0/3309055578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be48cc63b0 0x55be48cdc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.839+0000 7f748f7fe640 1 -- 172.21.15.43:0/3309055578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be48cd0a50 con 0x55be48ccfe60 2024-04-21T08:46:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.839+0000 7f748e7fc640 1 --2- 172.21.15.43:0/3309055578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be48cc6d60 0x55be48cdc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.839+0000 7f748effd640 1 --2- 172.21.15.43:0/3309055578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be48cc63b0 0x55be48cdc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.840+0000 7f748f7fe640 1 --2- 172.21.15.43:0/3309055578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48ccfe60 0x55be48cd0240 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f7488003ba0 tx=0x7f7488003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dae4eecc6c6779b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.840+0000 7f7472ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.840+0000 7f7472ff5640 1 -- 172.21.15.43:0/3309055578 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7488004320 con 0x55be48ccfe60 2024-04-21T08:46:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.840+0000 7f7498252740 1 -- 172.21.15.43:0/3309055578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7488054070 con 0x55be48ccfe60 2024-04-21T08:46:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.840+0000 7f7472ff5640 1 -- 172.21.15.43:0/3309055578 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7f74880044c0 con 0x55be48ccfe60 2024-04-21T08:46:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.840+0000 7f7472ff5640 1 -- 172.21.15.43:0/3309055578 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f74880046c0 con 0x55be48ccfe60 2024-04-21T08:46:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.840+0000 7f7498252740 10 client.18975 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.840+0000 7f7498252740 1 -- 172.21.15.43:0/3309055578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55be48d49bd0 con 0x55be48ccfe60 2024-04-21T08:46:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.840+0000 7f7472ff5640 1 -- 172.21.15.43:0/3309055578 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7488054070 con 0x55be48ccfe60 2024-04-21T08:46:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.840+0000 7f7472ff5640 1 -- 172.21.15.43:0/3309055578 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7488005600 con 0x55be48ccfe60 2024-04-21T08:46:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.840+0000 7f7498252740 10 client.18975 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.840+0000 7f7498252740 10 client.18975 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:46:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.840+0000 7f7498252740 20 client.18975 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.840+0000 7f7498252740 1 --2- 172.21.15.43:0/3309055578 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55be48d4a2c0 0x55be48d4c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.840+0000 7f7498252740 4 client.18975 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.840+0000 7f7498252740 1 -- 172.21.15.43:0/3309055578 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be48cd3f10 con 0x55be48d4a2c0 2024-04-21T08:46:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.841+0000 7f748effd640 1 --2- 172.21.15.43:0/3309055578 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55be48d4a2c0 0x55be48d4c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.841+0000 7f748effd640 1 --2- 172.21.15.43:0/3309055578 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55be48d4a2c0 0x55be48d4c6e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8f7b15bf75b7a693 server_cookie=3357866ba1169589 in_seq=0 out_seq=0 2024-04-21T08:46:09.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.841+0000 7f7472ff5640 10 client.18975 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:46:09.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.841+0000 7f7472ff5640 1 -- 172.21.15.43:0/3309055578 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be48cd3f10 con 0x55be48d4a2c0 2024-04-21T08:46:09.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.841+0000 7f7472ff5640 10 client.18975 handle_command_reply: tid=0 2024-04-21T08:46:09.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.841+0000 7f7498252740 1 client.18975 shutdown 2024-04-21T08:46:09.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.841+0000 7f7498252740 1 -- 172.21.15.43:0/3309055578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48ccfe60 msgr2=0x55be48cd0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.841+0000 7f7498252740 1 --2- 172.21.15.43:0/3309055578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48ccfe60 0x55be48cd0240 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f7488003ba0 tx=0x7f7488003bd0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.841+0000 7f7498252740 1 -- 172.21.15.43:0/3309055578 shutdown_connections 2024-04-21T08:46:09.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.841+0000 7f7498252740 1 --2- 172.21.15.43:0/3309055578 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55be48d4a2c0 0x55be48d4c6e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.841+0000 7f7498252740 1 --2- 172.21.15.43:0/3309055578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be48ccfe60 0x55be48cd0240 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.841+0000 7f7472ff5640 0 client.18975 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:46:09.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.841+0000 7f7498252740 1 --2- 172.21.15.43:0/3309055578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be48cc6d60 0x55be48cdc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.841+0000 7f7498252740 1 --2- 172.21.15.43:0/3309055578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be48cc63b0 0x55be48cdc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.841+0000 7f7498252740 1 -- 172.21.15.43:0/3309055578 >> 172.21.15.43:0/3309055578 conn(0x55be48cc0ba0 msgr2=0x55be48ccf660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.841+0000 7f7498252740 1 -- 172.21.15.43:0/3309055578 shutdown_connections 2024-04-21T08:46:09.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.841+0000 7f7498252740 1 -- 172.21.15.43:0/3309055578 wait complete. 2024-04-21T08:46:09.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.841+0000 7f7498252740 20 client.18975 trim_cache size 0 max 16384 2024-04-21T08:46:09.848 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:46:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.842+0000 7f749565c640 1 -- 172.21.15.43:0/1874474437 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f74780a3da0 msgr2=0x7f74780a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.842+0000 7f749565c640 1 --2- 172.21.15.43:0/1874474437 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f74780a3da0 0x7f74780a6260 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7488004af0 tx=0x7f748802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.842+0000 7f749565c640 1 -- 172.21.15.43:0/1874474437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7490107260 msgr2=0x7f749019e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.842+0000 7f749565c640 1 --2- 172.21.15.43:0/1874474437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7490107260 0x7f749019e920 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f748400d770 tx=0x7f748400dc30 comp rx=0 tx=0).stop 2024-04-21T08:46:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.842+0000 7f749565c640 1 -- 172.21.15.43:0/1874474437 shutdown_connections 2024-04-21T08:46:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.842+0000 7f749565c640 1 --2- 172.21.15.43:0/1874474437 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f74780a3da0 0x7f74780a6260 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.842+0000 7f749565c640 1 --2- 172.21.15.43:0/1874474437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7490198590 0x7f7490198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.842+0000 7f749565c640 1 --2- 172.21.15.43:0/1874474437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7490107260 0x7f749019e920 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.842+0000 7f749565c640 1 --2- 172.21.15.43:0/1874474437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7490102190 0x7f749019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.842+0000 7f749565c640 1 -- 172.21.15.43:0/1874474437 >> 172.21.15.43:0/1874474437 conn(0x7f74900793a0 msgr2=0x7f74900ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.842+0000 7f749565c640 1 -- 172.21.15.43:0/1874474437 shutdown_connections 2024-04-21T08:46:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.842+0000 7f749565c640 1 -- 172.21.15.43:0/1874474437 wait complete. 2024-04-21T08:46:09.852 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:46:09.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.957+0000 7fc26e38f640 1 -- 172.21.15.43:0/3208766108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc268107270 msgr2=0x7fc26810ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.957+0000 7fc26e38f640 1 --2- 172.21.15.43:0/3208766108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc268107270 0x7fc26810ee40 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fc260009ca0 tx=0x7fc26002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.957+0000 7fc26e38f640 1 -- 172.21.15.43:0/3208766108 shutdown_connections 2024-04-21T08:46:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.957+0000 7fc26e38f640 1 --2- 172.21.15.43:0/3208766108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc268107270 0x7fc26810ee40 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.957+0000 7fc26e38f640 1 --2- 172.21.15.43:0/3208766108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc26806b1f0 0x7fc268106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.957+0000 7fc26e38f640 1 --2- 172.21.15.43:0/3208766108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc26806a840 0x7fc26806ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.957+0000 7fc26e38f640 1 -- 172.21.15.43:0/3208766108 >> 172.21.15.43:0/3208766108 conn(0x7fc268077eb0 msgr2=0x7fc2680782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.957+0000 7fc26e38f640 1 -- 172.21.15.43:0/3208766108 shutdown_connections 2024-04-21T08:46:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.958+0000 7fc26e38f640 1 -- 172.21.15.43:0/3208766108 wait complete. 2024-04-21T08:46:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.958+0000 7fc26e38f640 1 Processor -- start 2024-04-21T08:46:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.958+0000 7fc26e38f640 1 -- start start 2024-04-21T08:46:09.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.958+0000 7fc26e38f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc26806a840 0x7fc268100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.958+0000 7fc26e38f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc268107270 0x7fc2681013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.958+0000 7fc26e38f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc268101900 0x7fc2681a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.958+0000 7fc26e38f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc2681121d0 con 0x7fc268101900 2024-04-21T08:46:09.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.958+0000 7fc26e38f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc268112050 con 0x7fc268107270 2024-04-21T08:46:09.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.958+0000 7fc26e38f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc268112350 con 0x7fc26806a840 2024-04-21T08:46:09.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.959+0000 7fc26c92c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc268101900 0x7fc2681a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.959+0000 7fc26c92c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc268101900 0x7fc2681a68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37502/0 (socket says 172.21.15.43:37502) 2024-04-21T08:46:09.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.959+0000 7fc26c92c640 1 -- 172.21.15.43:0/800221435 learned_addr learned my addr 172.21.15.43:0/800221435 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.959+0000 7fc267fff640 1 --2- 172.21.15.43:0/800221435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc26806a840 0x7fc268100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.959+0000 7fc2677fe640 1 --2- 172.21.15.43:0/800221435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc268107270 0x7fc2681013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.959+0000 7fc26c92c640 1 -- 172.21.15.43:0/800221435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc26806a840 msgr2=0x7fc268100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.959+0000 7fc26c92c640 1 --2- 172.21.15.43:0/800221435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc26806a840 0x7fc268100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.959+0000 7fc26c92c640 1 -- 172.21.15.43:0/800221435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc268107270 msgr2=0x7fc2681013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.959+0000 7fc26c92c640 1 --2- 172.21.15.43:0/800221435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc268107270 0x7fc2681013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.959+0000 7fc26c92c640 1 -- 172.21.15.43:0/800221435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2681a6fa0 con 0x7fc268101900 2024-04-21T08:46:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.959+0000 7fc2677fe640 1 --2- 172.21.15.43:0/800221435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc268107270 0x7fc2681013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.959+0000 7fc26c92c640 1 --2- 172.21.15.43:0/800221435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc268101900 0x7fc2681a68a0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fc260005e30 tx=0x7fc260032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.959+0000 7fc2657fa640 1 -- 172.21.15.43:0/800221435 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260041020 con 0x7fc268101900 2024-04-21T08:46:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.959+0000 7fc2657fa640 1 -- 172.21.15.43:0/800221435 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc260032970 con 0x7fc268101900 2024-04-21T08:46:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.959+0000 7fc26e38f640 1 -- 172.21.15.43:0/800221435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2681a7230 con 0x7fc268101900 2024-04-21T08:46:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.959+0000 7fc2657fa640 1 -- 172.21.15.43:0/800221435 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260039040 con 0x7fc268101900 2024-04-21T08:46:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.959+0000 7fc26e38f640 1 -- 172.21.15.43:0/800221435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2681a7720 con 0x7fc268101900 2024-04-21T08:46:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.962+0000 7fc2657fa640 1 -- 172.21.15.43:0/800221435 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc260044020 con 0x7fc268101900 2024-04-21T08:46:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.962+0000 7fc270f7d740 1 Processor -- start 2024-04-21T08:46:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.962+0000 7fc270f7d740 1 -- start start 2024-04-21T08:46:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.962+0000 7fc270f7d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56453e6954d0 0x56453e6958b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.962+0000 7fc270f7d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56453e695e80 0x56453e69e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.962+0000 7fc270f7d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e69ea00 0x56453e6a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.962+0000 7fc270f7d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56453e5fa1f0 con 0x56453e69ea00 2024-04-21T08:46:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.962+0000 7fc270f7d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56453e5974b0 con 0x56453e6954d0 2024-04-21T08:46:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.962+0000 7fc270f7d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56453e5f61b0 con 0x56453e695e80 2024-04-21T08:46:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.962+0000 7fc2657fa640 1 --2- 172.21.15.43:0/800221435 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc2540a3d50 0x7fc2540a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.962+0000 7fc2677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e69ea00 0x56453e6a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.962+0000 7fc2677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e69ea00 0x56453e6a0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37512/0 (socket says 172.21.15.43:37512) 2024-04-21T08:46:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.962+0000 7fc2677fe640 1 -- 172.21.15.43:0/1704420451 learned_addr learned my addr 172.21.15.43:0/1704420451 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.962+0000 7fc26c92c640 1 --2- 172.21.15.43:0/800221435 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc2540a3d50 0x7fc2540a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.962+0000 7fc2677fe640 1 --2- 172.21.15.43:0/1704420451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56453e6954d0 0x56453e6958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.962+0000 7fc267fff640 1 --2- 172.21.15.43:0/1704420451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56453e695e80 0x56453e69e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.962+0000 7fc2657fa640 1 -- 172.21.15.43:0/800221435 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7fc2600f28d0 con 0x7fc268101900 2024-04-21T08:46:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.963+0000 7fc2677fe640 1 -- 172.21.15.43:0/1704420451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56453e695e80 msgr2=0x56453e69e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.963+0000 7fc2677fe640 1 --2- 172.21.15.43:0/1704420451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56453e695e80 0x56453e69e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.963+0000 7fc2677fe640 1 -- 172.21.15.43:0/1704420451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56453e6954d0 msgr2=0x56453e6958b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.963+0000 7fc2677fe640 1 --2- 172.21.15.43:0/1704420451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56453e6954d0 0x56453e6958b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.963+0000 7fc2677fe640 1 -- 172.21.15.43:0/1704420451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56453e5891b0 con 0x56453e69ea00 2024-04-21T08:46:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.963+0000 7fc2677fe640 1 --2- 172.21.15.43:0/1704420451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e69ea00 0x56453e6a0e50 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fc258014880 tx=0x7fc2580095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba97da7da4bfaa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.963+0000 7fc26c92c640 1 --2- 172.21.15.43:0/800221435 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc2540a3d50 0x7fc2540a6210 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc2600efd60 tx=0x7fc2600b3990 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.963+0000 7fc25e7fc640 1 -- 172.21.15.43:0/1704420451 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc258014110 con 0x56453e69ea00 2024-04-21T08:46:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.963+0000 7fc25e7fc640 1 -- 172.21.15.43:0/1704420451 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2580142b0 con 0x56453e69ea00 2024-04-21T08:46:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.963+0000 7fc25e7fc640 1 -- 172.21.15.43:0/1704420451 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2580035f0 con 0x56453e69ea00 2024-04-21T08:46:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.964+0000 7fc270f7d740 1 -- 172.21.15.43:0/1704420451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e69ea00 msgr2=0x56453e6a0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.964+0000 7fc270f7d740 1 --2- 172.21.15.43:0/1704420451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e69ea00 0x56453e6a0e50 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fc258014880 tx=0x7fc2580095b0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.964+0000 7fc270f7d740 1 -- 172.21.15.43:0/1704420451 shutdown_connections 2024-04-21T08:46:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.964+0000 7fc270f7d740 1 --2- 172.21.15.43:0/1704420451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e69ea00 0x56453e6a0e50 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.964+0000 7fc270f7d740 1 --2- 172.21.15.43:0/1704420451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56453e695e80 0x56453e69e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.964+0000 7fc270f7d740 1 --2- 172.21.15.43:0/1704420451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56453e6954d0 0x56453e6958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.964+0000 7fc270f7d740 1 -- 172.21.15.43:0/1704420451 >> 172.21.15.43:0/1704420451 conn(0x56453e68fba0 msgr2=0x56453e68ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.964+0000 7fc270f7d740 1 -- 172.21.15.43:0/1704420451 shutdown_connections 2024-04-21T08:46:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.964+0000 7fc270f7d740 1 -- 172.21.15.43:0/1704420451 wait complete. 2024-04-21T08:46:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.964+0000 7fc270f7d740 1 -- start start 2024-04-21T08:46:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.964+0000 7fc270f7d740 1 Processor -- start 2024-04-21T08:46:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc270f7d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56453e6954d0 0x56453e69e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc270f7d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e695e80 0x56453e69b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc270f7d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56453e69ea00 0x56453e69bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc270f7d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56453e6a1970 con 0x56453e695e80 2024-04-21T08:46:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc270f7d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56453e5b3500 con 0x56453e6954d0 2024-04-21T08:46:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc270f7d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56453e6a1af0 con 0x56453e69ea00 2024-04-21T08:46:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc270f7d740 10 client.0 _ll_register_callbacks cb 0x56453e581b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc267fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e695e80 0x56453e69b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc267fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e695e80 0x56453e69b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37518/0 (socket says 172.21.15.43:37518) 2024-04-21T08:46:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc267fff640 1 -- 172.21.15.43:0/1550483849 learned_addr learned my addr 172.21.15.43:0/1550483849 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc2677fe640 1 --2- 172.21.15.43:0/1550483849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56453e69ea00 0x56453e69bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc2677fe640 1 --2- 172.21.15.43:0/1550483849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56453e6954d0 0x56453e69e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc267fff640 1 -- 172.21.15.43:0/1550483849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56453e69ea00 msgr2=0x56453e69bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc267fff640 1 --2- 172.21.15.43:0/1550483849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56453e69ea00 0x56453e69bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc267fff640 1 -- 172.21.15.43:0/1550483849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56453e6954d0 msgr2=0x56453e69e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc267fff640 1 --2- 172.21.15.43:0/1550483849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56453e6954d0 0x56453e69e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc267fff640 1 -- 172.21.15.43:0/1550483849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56453e30c3f0 con 0x56453e695e80 2024-04-21T08:46:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc2677fe640 1 --2- 172.21.15.43:0/1550483849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56453e6954d0 0x56453e69e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc267fff640 1 --2- 172.21.15.43:0/1550483849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e695e80 0x56453e69b610 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fc25000bad0 tx=0x7fc25000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d620aa529cb2aba server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc2477fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc2477fe640 1 -- 172.21.15.43:0/1550483849 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc250003e00 con 0x56453e695e80 2024-04-21T08:46:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc2477fe640 1 -- 172.21.15.43:0/1550483849 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7fc250011440 con 0x56453e695e80 2024-04-21T08:46:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc270f7d740 1 -- 172.21.15.43:0/1550483849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2580142b0 con 0x56453e695e80 2024-04-21T08:46:09.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc2477fe640 1 -- 172.21.15.43:0/1550483849 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc250011680 con 0x56453e695e80 2024-04-21T08:46:09.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc270f7d740 10 client.18987 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:09.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.965+0000 7fc270f7d740 1 -- 172.21.15.43:0/1550483849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56453e3d67d0 con 0x56453e695e80 2024-04-21T08:46:09.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.966+0000 7fc2477fe640 1 -- 172.21.15.43:0/1550483849 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2580142b0 con 0x56453e695e80 2024-04-21T08:46:09.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.966+0000 7fc2477fe640 1 -- 172.21.15.43:0/1550483849 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc25001ba70 con 0x56453e695e80 2024-04-21T08:46:09.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.966+0000 7fc270f7d740 10 client.18987 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:09.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.966+0000 7fc270f7d740 10 client.18987 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:46:09.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.966+0000 7fc270f7d740 20 client.18987 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:09.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.966+0000 7fc270f7d740 1 --2- 172.21.15.43:0/1550483849 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x56453e6aecb0 0x56453e6b1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.966+0000 7fc270f7d740 4 client.18987 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:09.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.966+0000 7fc270f7d740 1 -- 172.21.15.43:0/1550483849 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56453e599930 con 0x56453e6aecb0 2024-04-21T08:46:09.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.966+0000 7fc2677fe640 1 --2- 172.21.15.43:0/1550483849 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x56453e6aecb0 0x56453e6b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.967+0000 7fc2477fe640 10 client.18987 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:46:09.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.967+0000 7fc2677fe640 1 --2- 172.21.15.43:0/1550483849 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x56453e6aecb0 0x56453e6b1110 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7e5d5c1c4d4d9e7 server_cookie=e2242372dc9990b2 in_seq=0 out_seq=0 2024-04-21T08:46:09.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.967+0000 7fc2477fe640 1 -- 172.21.15.43:0/1550483849 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56453e599930 con 0x56453e6aecb0 2024-04-21T08:46:09.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.967+0000 7fc2477fe640 10 client.18987 handle_command_reply: tid=0 2024-04-21T08:46:09.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.967+0000 7fc270f7d740 1 client.18987 shutdown 2024-04-21T08:46:09.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.968+0000 7fc270f7d740 1 -- 172.21.15.43:0/1550483849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e695e80 msgr2=0x56453e69b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.968+0000 7fc270f7d740 1 --2- 172.21.15.43:0/1550483849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e695e80 0x56453e69b610 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fc25000bad0 tx=0x7fc25000bf90 comp rx=0 tx=0).stop 2024-04-21T08:46:09.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.968+0000 7fc270f7d740 1 -- 172.21.15.43:0/1550483849 shutdown_connections 2024-04-21T08:46:09.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.968+0000 7fc270f7d740 1 --2- 172.21.15.43:0/1550483849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56453e69ea00 0x56453e69bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.968+0000 7fc270f7d740 1 --2- 172.21.15.43:0/1550483849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e695e80 0x56453e69b610 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.968+0000 7fc270f7d740 1 --2- 172.21.15.43:0/1550483849 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x56453e6aecb0 0x56453e6b1110 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.968+0000 7fc270f7d740 1 --2- 172.21.15.43:0/1550483849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56453e6954d0 0x56453e69e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.968+0000 7fc2477fe640 0 client.18987 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:46:09.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.968+0000 7fc270f7d740 1 -- 172.21.15.43:0/1550483849 >> 172.21.15.43:0/1550483849 conn(0x56453e68fba0 msgr2=0x56453e6918f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.968+0000 7fc270f7d740 1 -- 172.21.15.43:0/1550483849 shutdown_connections 2024-04-21T08:46:09.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.968+0000 7fc270f7d740 1 -- 172.21.15.43:0/1550483849 wait complete. 2024-04-21T08:46:09.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.968+0000 7fc270f7d740 20 client.18987 trim_cache size 0 max 16384 2024-04-21T08:46:09.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.981+0000 7fc270f7d740 1 Processor -- start 2024-04-21T08:46:09.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.981+0000 7fc270f7d740 1 -- start start 2024-04-21T08:46:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.981+0000 7fc270f7d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e6954d0 0x56453e6aa860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.981+0000 7fc270f7d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56453e695e80 0x56453e69a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.981+0000 7fc270f7d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56453e69ea00 0x56453e691700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.981+0000 7fc270f7d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc2681b1790 con 0x56453e6954d0 2024-04-21T08:46:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.981+0000 7fc270f7d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2681b32d0 con 0x56453e695e80 2024-04-21T08:46:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.981+0000 7fc270f7d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc2681b1e60 con 0x56453e69ea00 2024-04-21T08:46:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.982+0000 7fc2677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e6954d0 0x56453e6aa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.982+0000 7fc2677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e6954d0 0x56453e6aa860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37530/0 (socket says 172.21.15.43:37530) 2024-04-21T08:46:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.982+0000 7fc2677fe640 1 -- 172.21.15.43:0/3657950079 learned_addr learned my addr 172.21.15.43:0/3657950079 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.982+0000 7fc2677fe640 1 -- 172.21.15.43:0/3657950079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56453e69ea00 msgr2=0x56453e691700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.982+0000 7fc2677fe640 1 --2- 172.21.15.43:0/3657950079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56453e69ea00 0x56453e691700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.982+0000 7fc2677fe640 1 -- 172.21.15.43:0/3657950079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56453e695e80 msgr2=0x56453e69a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.982+0000 7fc267fff640 1 --2- 172.21.15.43:0/3657950079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56453e695e80 0x56453e69a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.982+0000 7fc2677fe640 1 --2- 172.21.15.43:0/3657950079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56453e695e80 0x56453e69a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.982+0000 7fc2677fe640 1 -- 172.21.15.43:0/3657950079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56453e695090 con 0x56453e6954d0 2024-04-21T08:46:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.982+0000 7fc267fff640 1 --2- 172.21.15.43:0/3657950079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56453e695e80 0x56453e69a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.982+0000 7fc2677fe640 1 --2- 172.21.15.43:0/3657950079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e6954d0 0x56453e6aa860 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fc258001eb0 tx=0x7fc258005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7681be88a602f068 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.982+0000 7fc25dffb640 1 -- 172.21.15.43:0/3657950079 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25800f730 con 0x56453e6954d0 2024-04-21T08:46:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.982+0000 7fc25dffb640 1 -- 172.21.15.43:0/3657950079 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc258014230 con 0x56453e6954d0 2024-04-21T08:46:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.982+0000 7fc25dffb640 1 -- 172.21.15.43:0/3657950079 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25802b430 con 0x56453e6954d0 2024-04-21T08:46:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.983+0000 7fc270f7d740 1 -- 172.21.15.43:0/3657950079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e6954d0 msgr2=0x56453e6aa860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.983+0000 7fc270f7d740 1 --2- 172.21.15.43:0/3657950079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e6954d0 0x56453e6aa860 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fc258001eb0 tx=0x7fc258005f90 comp rx=0 tx=0).stop 2024-04-21T08:46:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.983+0000 7fc270f7d740 1 -- 172.21.15.43:0/3657950079 shutdown_connections 2024-04-21T08:46:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.983+0000 7fc270f7d740 1 --2- 172.21.15.43:0/3657950079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56453e69ea00 0x56453e691700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.983+0000 7fc270f7d740 1 --2- 172.21.15.43:0/3657950079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56453e695e80 0x56453e69a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.983+0000 7fc270f7d740 1 --2- 172.21.15.43:0/3657950079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e6954d0 0x56453e6aa860 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.983+0000 7fc270f7d740 1 -- 172.21.15.43:0/3657950079 >> 172.21.15.43:0/3657950079 conn(0x56453e68fba0 msgr2=0x56453e690ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.983+0000 7fc270f7d740 1 -- 172.21.15.43:0/3657950079 shutdown_connections 2024-04-21T08:46:09.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.983+0000 7fc270f7d740 1 -- 172.21.15.43:0/3657950079 wait complete. 2024-04-21T08:46:09.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.983+0000 7fc270f7d740 1 -- start start 2024-04-21T08:46:09.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.983+0000 7fc270f7d740 1 Processor -- start 2024-04-21T08:46:09.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc270f7d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e6954d0 0x56453e6a6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc270f7d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56453e695e80 0x56453e6a72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc270f7d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56453e69ea00 0x56453e697960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc270f7d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc2681b2c00 con 0x56453e6954d0 2024-04-21T08:46:09.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc270f7d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2681b2530 con 0x56453e695e80 2024-04-21T08:46:09.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc270f7d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc2681af790 con 0x56453e69ea00 2024-04-21T08:46:09.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc270f7d740 10 client.0 _ll_register_callbacks cb 0x7fc250004400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:09.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc2677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e6954d0 0x56453e6a6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc2677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e6954d0 0x56453e6a6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37536/0 (socket says 172.21.15.43:37536) 2024-04-21T08:46:09.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc2677fe640 1 -- 172.21.15.43:0/4137035104 learned_addr learned my addr 172.21.15.43:0/4137035104 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:09.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc267fff640 1 --2- 172.21.15.43:0/4137035104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56453e695e80 0x56453e6a72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc2677fe640 1 --2- 172.21.15.43:0/4137035104 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56453e69ea00 0x56453e697960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc2677fe640 1 -- 172.21.15.43:0/4137035104 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56453e69ea00 msgr2=0x56453e697960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc2677fe640 1 --2- 172.21.15.43:0/4137035104 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56453e69ea00 0x56453e697960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc2677fe640 1 -- 172.21.15.43:0/4137035104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56453e695e80 msgr2=0x56453e6a72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:09.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc2677fe640 1 --2- 172.21.15.43:0/4137035104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56453e695e80 0x56453e6a72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc2677fe640 1 -- 172.21.15.43:0/4137035104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56453e697fd0 con 0x56453e6954d0 2024-04-21T08:46:09.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc2677fe640 1 --2- 172.21.15.43:0/4137035104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e6954d0 0x56453e6a6d70 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fc258009460 tx=0x7fc2580052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d76246dd9038bf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc2477fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:09.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc2477fe640 1 -- 172.21.15.43:0/4137035104 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc258009aa0 con 0x56453e6954d0 2024-04-21T08:46:09.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc2477fe640 1 -- 172.21.15.43:0/4137035104 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9671+0+0 (secure 0 0 0) 0x7fc258005390 con 0x56453e6954d0 2024-04-21T08:46:09.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc270f7d740 1 -- 172.21.15.43:0/4137035104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc258014230 con 0x56453e6954d0 2024-04-21T08:46:09.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc2477fe640 1 -- 172.21.15.43:0/4137035104 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc258005590 con 0x56453e6954d0 2024-04-21T08:46:09.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc270f7d740 10 client.18993 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:09.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.984+0000 7fc270f7d740 1 -- 172.21.15.43:0/4137035104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56453e714b20 con 0x56453e6954d0 2024-04-21T08:46:09.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.985+0000 7fc2477fe640 1 -- 172.21.15.43:0/4137035104 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc258014230 con 0x56453e6954d0 2024-04-21T08:46:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.985+0000 7fc2477fe640 1 -- 172.21.15.43:0/4137035104 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc258034070 con 0x56453e6954d0 2024-04-21T08:46:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.985+0000 7fc270f7d740 10 client.18993 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.985+0000 7fc270f7d740 10 client.18993 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:46:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.985+0000 7fc270f7d740 20 client.18993 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.985+0000 7fc270f7d740 1 --2- 172.21.15.43:0/4137035104 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x56453e714ea0 0x56453e7172c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.985+0000 7fc270f7d740 4 client.18993 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.985+0000 7fc270f7d740 1 -- 172.21.15.43:0/4137035104 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56453e607c10 con 0x56453e714ea0 2024-04-21T08:46:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.985+0000 7fc267fff640 1 --2- 172.21.15.43:0/4137035104 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x56453e714ea0 0x56453e7172c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.986+0000 7fc2477fe640 10 client.18993 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:46:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.986+0000 7fc267fff640 1 --2- 172.21.15.43:0/4137035104 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x56453e714ea0 0x56453e7172c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65e0ed880c92f20f server_cookie=7e1e4c5a75d62b45 in_seq=0 out_seq=0 2024-04-21T08:46:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.986+0000 7fc2477fe640 1 -- 172.21.15.43:0/4137035104 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56453e607c10 con 0x56453e714ea0 2024-04-21T08:46:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.986+0000 7fc2477fe640 10 client.18993 handle_command_reply: tid=0 2024-04-21T08:46:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.986+0000 7fc270f7d740 1 client.18993 shutdown 2024-04-21T08:46:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.986+0000 7fc270f7d740 1 -- 172.21.15.43:0/4137035104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e6954d0 msgr2=0x56453e6a6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.986+0000 7fc270f7d740 1 --2- 172.21.15.43:0/4137035104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e6954d0 0x56453e6a6d70 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fc258009460 tx=0x7fc2580052b0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.986+0000 7fc270f7d740 1 -- 172.21.15.43:0/4137035104 shutdown_connections 2024-04-21T08:46:09.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.986+0000 7fc270f7d740 1 --2- 172.21.15.43:0/4137035104 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56453e69ea00 0x56453e697960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.986+0000 7fc270f7d740 1 --2- 172.21.15.43:0/4137035104 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x56453e714ea0 0x56453e7172c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.986+0000 7fc270f7d740 1 --2- 172.21.15.43:0/4137035104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56453e695e80 0x56453e6a72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.986+0000 7fc2477fe640 0 client.18993 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:46:09.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.986+0000 7fc270f7d740 1 --2- 172.21.15.43:0/4137035104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56453e6954d0 0x56453e6a6d70 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.986+0000 7fc270f7d740 1 -- 172.21.15.43:0/4137035104 >> 172.21.15.43:0/4137035104 conn(0x56453e68fba0 msgr2=0x56453e691860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.986+0000 7fc270f7d740 1 -- 172.21.15.43:0/4137035104 shutdown_connections 2024-04-21T08:46:09.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.986+0000 7fc270f7d740 1 -- 172.21.15.43:0/4137035104 wait complete. 2024-04-21T08:46:09.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.986+0000 7fc270f7d740 20 client.18993 trim_cache size 0 max 16384 2024-04-21T08:46:09.992 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:46:09.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.987+0000 7fc26e38f640 1 -- 172.21.15.43:0/800221435 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc2540a3d50 msgr2=0x7fc2540a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.987+0000 7fc26e38f640 1 --2- 172.21.15.43:0/800221435 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc2540a3d50 0x7fc2540a6210 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc2600efd60 tx=0x7fc2600b3990 comp rx=0 tx=0).stop 2024-04-21T08:46:09.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.987+0000 7fc26e38f640 1 -- 172.21.15.43:0/800221435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc268101900 msgr2=0x7fc2681a68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.987+0000 7fc26e38f640 1 --2- 172.21.15.43:0/800221435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc268101900 0x7fc2681a68a0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fc260005e30 tx=0x7fc260032420 comp rx=0 tx=0).stop 2024-04-21T08:46:09.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.987+0000 7fc26e38f640 1 -- 172.21.15.43:0/800221435 shutdown_connections 2024-04-21T08:46:09.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.987+0000 7fc26e38f640 1 --2- 172.21.15.43:0/800221435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc268101900 0x7fc2681a68a0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.987+0000 7fc26e38f640 1 --2- 172.21.15.43:0/800221435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc268107270 0x7fc2681013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.987+0000 7fc26e38f640 1 --2- 172.21.15.43:0/800221435 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc2540a3d50 0x7fc2540a6210 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.987+0000 7fc26e38f640 1 --2- 172.21.15.43:0/800221435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc26806a840 0x7fc268100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.987+0000 7fc26e38f640 1 -- 172.21.15.43:0/800221435 >> 172.21.15.43:0/800221435 conn(0x7fc268077eb0 msgr2=0x7fc268100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.987+0000 7fc26e38f640 1 -- 172.21.15.43:0/800221435 shutdown_connections 2024-04-21T08:46:09.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:09.987+0000 7fc26e38f640 1 -- 172.21.15.43:0/800221435 wait complete. 2024-04-21T08:46:09.998 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:46:10.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.104+0000 7f6cdca48640 1 -- 172.21.15.43:0/1549578001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cd806b1f0 msgr2=0x7f6cd8106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.104+0000 7f6cdca48640 1 --2- 172.21.15.43:0/1549578001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cd806b1f0 0x7f6cd8106b90 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009900 tx=0x7f6ccc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:10.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.104+0000 7f6cdca48640 1 -- 172.21.15.43:0/1549578001 shutdown_connections 2024-04-21T08:46:10.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.104+0000 7f6cdca48640 1 --2- 172.21.15.43:0/1549578001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cd8107270 0x7f6cd810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.104+0000 7f6cdca48640 1 --2- 172.21.15.43:0/1549578001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cd806b1f0 0x7f6cd8106b90 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.104+0000 7f6cdca48640 1 --2- 172.21.15.43:0/1549578001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd806a840 0x7f6cd806ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.104+0000 7f6cdca48640 1 -- 172.21.15.43:0/1549578001 >> 172.21.15.43:0/1549578001 conn(0x7f6cd8077eb0 msgr2=0x7f6cd80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:10.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.104+0000 7f6cdca48640 1 -- 172.21.15.43:0/1549578001 shutdown_connections 2024-04-21T08:46:10.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.104+0000 7f6cdca48640 1 -- 172.21.15.43:0/1549578001 wait complete. 2024-04-21T08:46:10.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.105+0000 7f6cdca48640 1 Processor -- start 2024-04-21T08:46:10.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.105+0000 7f6cdca48640 1 -- start start 2024-04-21T08:46:10.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.105+0000 7f6cdca48640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd806a840 0x7f6cd8198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.105+0000 7f6cdca48640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cd8107270 0x7f6cd8198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.105+0000 7f6cdca48640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cd819cf50 0x7f6cd819d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.105+0000 7f6cdca48640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd81121b0 con 0x7f6cd8107270 2024-04-21T08:46:10.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.105+0000 7f6cdca48640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd8112030 con 0x7f6cd806a840 2024-04-21T08:46:10.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.105+0000 7f6cdca48640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd8112330 con 0x7f6cd819cf50 2024-04-21T08:46:10.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.105+0000 7f6cd5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cd8107270 0x7f6cd8198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.105+0000 7f6cd5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cd8107270 0x7f6cd8198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37542/0 (socket says 172.21.15.43:37542) 2024-04-21T08:46:10.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.105+0000 7f6cd5d9b640 1 -- 172.21.15.43:0/334765430 learned_addr learned my addr 172.21.15.43:0/334765430 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:10.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.105+0000 7f6cd659c640 1 --2- 172.21.15.43:0/334765430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd806a840 0x7f6cd8198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.105+0000 7f6cd6d9d640 1 --2- 172.21.15.43:0/334765430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cd819cf50 0x7f6cd819d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.105+0000 7f6cd5d9b640 1 -- 172.21.15.43:0/334765430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cd819cf50 msgr2=0x7f6cd819d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.105+0000 7f6cd5d9b640 1 --2- 172.21.15.43:0/334765430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cd819cf50 0x7f6cd819d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.105+0000 7f6cd5d9b640 1 -- 172.21.15.43:0/334765430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd806a840 msgr2=0x7f6cd8198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.105+0000 7f6cd5d9b640 1 --2- 172.21.15.43:0/334765430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd806a840 0x7f6cd8198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.105+0000 7f6cd5d9b640 1 -- 172.21.15.43:0/334765430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd819da40 con 0x7f6cd8107270 2024-04-21T08:46:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.106+0000 7f6cd5d9b640 1 --2- 172.21.15.43:0/334765430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cd8107270 0x7f6cd8198bd0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc005ec0 tx=0x7f6ccc02ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.106+0000 7f6cc77fe640 1 -- 172.21.15.43:0/334765430 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc043020 con 0x7f6cd8107270 2024-04-21T08:46:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.106+0000 7f6cc77fe640 1 -- 172.21.15.43:0/334765430 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ccc0052b0 con 0x7f6cd8107270 2024-04-21T08:46:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.106+0000 7f6cdca48640 1 -- 172.21.15.43:0/334765430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd819dcd0 con 0x7f6cd8107270 2024-04-21T08:46:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.106+0000 7f6cc77fe640 1 -- 172.21.15.43:0/334765430 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc003c60 con 0x7f6cd8107270 2024-04-21T08:46:10.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.106+0000 7f6cdca48640 1 -- 172.21.15.43:0/334765430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd81a54f0 con 0x7f6cd8107270 2024-04-21T08:46:10.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.108+0000 7f6cc77fe640 1 -- 172.21.15.43:0/334765430 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6ccc046020 con 0x7f6cd8107270 2024-04-21T08:46:10.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.108+0000 7f6cdf632740 1 Processor -- start 2024-04-21T08:46:10.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.109+0000 7f6cdf632740 1 -- start start 2024-04-21T08:46:10.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.109+0000 7f6cdf632740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67af3b0 0x55feb67af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.109+0000 7f6cdf632740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55feb67afd60 0x55feb67b8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.109+0000 7f6cdf632740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55feb67b8e60 0x55feb67b92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.109+0000 7f6cdf632740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55feb67141f0 con 0x55feb67af3b0 2024-04-21T08:46:10.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.109+0000 7f6cdf632740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55feb66b14b0 con 0x55feb67afd60 2024-04-21T08:46:10.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.109+0000 7f6cdf632740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55feb67101b0 con 0x55feb67b8e60 2024-04-21T08:46:10.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.109+0000 7f6cc77fe640 1 --2- 172.21.15.43:0/334765430 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6cc00a3d20 0x7f6cc00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.109+0000 7f6cd6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67af3b0 0x55feb67af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.109+0000 7f6cd5d9b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55feb67b8e60 0x55feb67b92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.109+0000 7f6cd659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55feb67afd60 0x55feb67b8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.109+0000 7f6cd659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55feb67afd60 0x55feb67b8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:59652/0 (socket says 172.21.15.43:59652) 2024-04-21T08:46:10.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.109+0000 7f6cd6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67af3b0 0x55feb67af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37554/0 (socket says 172.21.15.43:37554) 2024-04-21T08:46:10.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.109+0000 7f6cd659c640 1 -- 172.21.15.43:0/283030317 learned_addr learned my addr 172.21.15.43:0/283030317 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:10.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.109+0000 7f6cd5d9b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55feb67b8e60 0x55feb67b92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.43:42846/0 (socket says 172.21.15.43:42846) 2024-04-21T08:46:10.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.109+0000 7f6cc77fe640 1 -- 172.21.15.43:0/334765430 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9583+0+0 (secure 0 0 0) 0x7f6ccc0f2e60 con 0x7f6cd8107270 2024-04-21T08:46:10.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.109+0000 7f6cd6d9d640 1 --2- 172.21.15.43:0/334765430 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6cc00a3d20 0x7f6cc00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.109+0000 7f6cd659c640 1 -- 172.21.15.43:0/283030317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55feb67b8e60 msgr2=0x55feb67b92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:10.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.109+0000 7f6cd659c640 1 --2- 172.21.15.43:0/283030317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55feb67b8e60 0x55feb67b92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.109+0000 7f6cd659c640 1 -- 172.21.15.43:0/283030317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67af3b0 msgr2=0x55feb67af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:10.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.109+0000 7f6cd659c640 1 --2- 172.21.15.43:0/283030317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67af3b0 0x55feb67af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.109+0000 7f6cd659c640 1 -- 172.21.15.43:0/283030317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feb66a31b0 con 0x55feb67afd60 2024-04-21T08:46:10.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.110+0000 7f6cd659c640 1 --2- 172.21.15.43:0/283030317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55feb67afd60 0x55feb67b8890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800ad00 tx=0x7f6cc800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a03868c72df7882 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:10.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.110+0000 7f6cd6d9d640 1 --2- 172.21.15.43:0/334765430 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6cc00a3d20 0x7f6cc00a61e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0012730 tx=0x7f6cd0004ab0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:10.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.110+0000 7f6cc4ff9640 1 -- 172.21.15.43:0/283030317 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc801d070 con 0x55feb67afd60 2024-04-21T08:46:10.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.110+0000 7f6cc4ff9640 1 -- 172.21.15.43:0/283030317 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cc8018af0 con 0x55feb67afd60 2024-04-21T08:46:10.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.110+0000 7f6cc4ff9640 1 -- 172.21.15.43:0/283030317 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8010a70 con 0x55feb67afd60 2024-04-21T08:46:10.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.110+0000 7f6cdf632740 1 -- 172.21.15.43:0/283030317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55feb67afd60 msgr2=0x55feb67b8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.110+0000 7f6cdf632740 1 --2- 172.21.15.43:0/283030317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55feb67afd60 0x55feb67b8890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800ad00 tx=0x7f6cc800f790 comp rx=0 tx=0).stop 2024-04-21T08:46:10.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.111+0000 7f6cdf632740 1 -- 172.21.15.43:0/283030317 shutdown_connections 2024-04-21T08:46:10.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.111+0000 7f6cdf632740 1 --2- 172.21.15.43:0/283030317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55feb67b8e60 0x55feb67b92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.111+0000 7f6cdf632740 1 --2- 172.21.15.43:0/283030317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55feb67afd60 0x55feb67b8890 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.111+0000 7f6cdf632740 1 --2- 172.21.15.43:0/283030317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67af3b0 0x55feb67af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.111+0000 7f6cdf632740 1 -- 172.21.15.43:0/283030317 >> 172.21.15.43:0/283030317 conn(0x55feb67a9ba0 msgr2=0x55feb67a9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:10.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.111+0000 7f6cdf632740 1 -- 172.21.15.43:0/283030317 shutdown_connections 2024-04-21T08:46:10.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.111+0000 7f6cdf632740 1 -- 172.21.15.43:0/283030317 wait complete. 2024-04-21T08:46:10.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.111+0000 7f6cdf632740 1 -- start start 2024-04-21T08:46:10.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.111+0000 7f6cdf632740 1 Processor -- start 2024-04-21T08:46:10.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.111+0000 7f6cdf632740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55feb67af3b0 0x55feb67bf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.111+0000 7f6cdf632740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55feb67afd60 0x55feb67bc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.111+0000 7f6cdf632740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67b8e60 0x55feb67bcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.111+0000 7f6cdf632740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55feb67bfda0 con 0x55feb67b8e60 2024-04-21T08:46:10.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.111+0000 7f6cdf632740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55feb66cd500 con 0x55feb67afd60 2024-04-21T08:46:10.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.111+0000 7f6cdf632740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55feb67bff20 con 0x55feb67af3b0 2024-04-21T08:46:10.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.111+0000 7f6cdf632740 10 client.0 _ll_register_callbacks cb 0x55feb669bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:10.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.111+0000 7f6cd6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67b8e60 0x55feb67bcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.112+0000 7f6cd5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55feb67afd60 0x55feb67bc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.112+0000 7f6cd659c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55feb67af3b0 0x55feb67bf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.112+0000 7f6cd5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55feb67afd60 0x55feb67bc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:59660/0 (socket says 172.21.15.43:59660) 2024-04-21T08:46:10.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.112+0000 7f6cd5d9b640 1 -- 172.21.15.43:0/669358139 learned_addr learned my addr 172.21.15.43:0/669358139 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:10.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.112+0000 7f6cd6d9d640 1 -- 172.21.15.43:0/669358139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55feb67af3b0 msgr2=0x55feb67bf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:10.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.112+0000 7f6cd6d9d640 1 --2- 172.21.15.43:0/669358139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55feb67af3b0 0x55feb67bf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.112+0000 7f6cd6d9d640 1 -- 172.21.15.43:0/669358139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55feb67afd60 msgr2=0x55feb67bc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:10.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.112+0000 7f6cd6d9d640 1 --2- 172.21.15.43:0/669358139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55feb67afd60 0x55feb67bc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.112+0000 7f6cd6d9d640 1 -- 172.21.15.43:0/669358139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feb64263f0 con 0x55feb67b8e60 2024-04-21T08:46:10.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.112+0000 7f6cd6d9d640 1 --2- 172.21.15.43:0/669358139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67b8e60 0x55feb67bcac0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0010e50 tx=0x7f6cd0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9767ea11f3d9fa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:10.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.112+0000 7f6cbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:10.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.112+0000 7f6cbdffb640 1 -- 172.21.15.43:0/669358139 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0018070 con 0x55feb67b8e60 2024-04-21T08:46:10.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.112+0000 7f6cbdffb640 1 -- 172.21.15.43:0/669358139 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9583+0+0 (secure 0 0 0) 0x7f6cd000fe10 con 0x55feb67b8e60 2024-04-21T08:46:10.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.112+0000 7f6cdf632740 1 -- 172.21.15.43:0/669358139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cc8018af0 con 0x55feb67b8e60 2024-04-21T08:46:10.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.112+0000 7f6cbdffb640 1 -- 172.21.15.43:0/669358139 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6cd002f050 con 0x55feb67b8e60 2024-04-21T08:46:10.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.112+0000 7f6cdf632740 10 client.19005 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:10.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.112+0000 7f6cdf632740 1 -- 172.21.15.43:0/669358139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55feb64f07d0 con 0x55feb67b8e60 2024-04-21T08:46:10.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.113+0000 7f6cbdffb640 1 -- 172.21.15.43:0/669358139 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cc8018af0 con 0x55feb67b8e60 2024-04-21T08:46:10.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.113+0000 7f6cbdffb640 1 -- 172.21.15.43:0/669358139 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6cd0004640 con 0x55feb67b8e60 2024-04-21T08:46:10.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.113+0000 7f6cdf632740 10 client.19005 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:10.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.113+0000 7f6cdf632740 10 client.19005 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:46:10.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.113+0000 7f6cdf632740 20 client.19005 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:10.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.113+0000 7f6cdf632740 1 --2- 172.21.15.43:0/669358139 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55feb67ccfb0 0x55feb67cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.113+0000 7f6cdf632740 4 client.19005 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:10.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.113+0000 7f6cdf632740 1 -- 172.21.15.43:0/669358139 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55feb66b3930 con 0x55feb67ccfb0 2024-04-21T08:46:10.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.113+0000 7f6cd659c640 1 --2- 172.21.15.43:0/669358139 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55feb67ccfb0 0x55feb67cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.113+0000 7f6cd659c640 1 --2- 172.21.15.43:0/669358139 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55feb67ccfb0 0x55feb67cf410 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3dd9a2f33f4c77a1 server_cookie=85471ffaebef03cf in_seq=0 out_seq=0 2024-04-21T08:46:10.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.113+0000 7f6cbdffb640 10 client.19005 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:46:10.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.114+0000 7f6cbdffb640 1 -- 172.21.15.43:0/669358139 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55feb66b3930 con 0x55feb67ccfb0 2024-04-21T08:46:10.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.114+0000 7f6cbdffb640 10 client.19005 handle_command_reply: tid=0 2024-04-21T08:46:10.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.114+0000 7f6cdf632740 1 client.19005 shutdown 2024-04-21T08:46:10.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.114+0000 7f6cdf632740 1 -- 172.21.15.43:0/669358139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67b8e60 msgr2=0x55feb67bcac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.114+0000 7f6cdf632740 1 --2- 172.21.15.43:0/669358139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67b8e60 0x55feb67bcac0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0010e50 tx=0x7f6cd0004a40 comp rx=0 tx=0).stop 2024-04-21T08:46:10.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.114+0000 7f6cdf632740 1 -- 172.21.15.43:0/669358139 shutdown_connections 2024-04-21T08:46:10.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.114+0000 7f6cdf632740 1 --2- 172.21.15.43:0/669358139 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55feb67ccfb0 0x55feb67cf410 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.114+0000 7f6cdf632740 1 --2- 172.21.15.43:0/669358139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67b8e60 0x55feb67bcac0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.114+0000 7f6cdf632740 1 --2- 172.21.15.43:0/669358139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55feb67afd60 0x55feb67bc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.114+0000 7f6cbdffb640 0 client.19005 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:46:10.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.114+0000 7f6cdf632740 1 --2- 172.21.15.43:0/669358139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55feb67af3b0 0x55feb67bf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.114+0000 7f6cdf632740 1 -- 172.21.15.43:0/669358139 >> 172.21.15.43:0/669358139 conn(0x55feb67a9ba0 msgr2=0x55feb67ab8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:10.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.114+0000 7f6cdf632740 1 -- 172.21.15.43:0/669358139 shutdown_connections 2024-04-21T08:46:10.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.114+0000 7f6cdf632740 1 -- 172.21.15.43:0/669358139 wait complete. 2024-04-21T08:46:10.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.115+0000 7f6cdf632740 20 client.19005 trim_cache size 0 max 16384 2024-04-21T08:46:10.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.128+0000 7f6cdf632740 1 Processor -- start 2024-04-21T08:46:10.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.128+0000 7f6cdf632740 1 -- start start 2024-04-21T08:46:10.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.128+0000 7f6cdf632740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67a9ba0 0x55feb67c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.128+0000 7f6cdf632740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55feb67af3b0 0x55feb67a8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.128+0000 7f6cdf632740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55feb67afd60 0x55feb67becb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.128+0000 7f6cdf632740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd81ad220 con 0x55feb67a9ba0 2024-04-21T08:46:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.128+0000 7f6cdf632740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd81aed60 con 0x55feb67afd60 2024-04-21T08:46:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.128+0000 7f6cdf632740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd81ad8f0 con 0x55feb67af3b0 2024-04-21T08:46:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.128+0000 7f6cd659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67a9ba0 0x55feb67c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.128+0000 7f6cd659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67a9ba0 0x55feb67c8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37564/0 (socket says 172.21.15.43:37564) 2024-04-21T08:46:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.128+0000 7f6cd659c640 1 -- 172.21.15.43:0/486852569 learned_addr learned my addr 172.21.15.43:0/486852569 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.128+0000 7f6cd6d9d640 1 --2- 172.21.15.43:0/486852569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55feb67afd60 0x55feb67becb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.128+0000 7f6cd659c640 1 -- 172.21.15.43:0/486852569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55feb67af3b0 msgr2=0x55feb67a8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.128+0000 7f6cd659c640 1 --2- 172.21.15.43:0/486852569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55feb67af3b0 0x55feb67a8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.128+0000 7f6cd659c640 1 -- 172.21.15.43:0/486852569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55feb67afd60 msgr2=0x55feb67becb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.128+0000 7f6cd659c640 1 --2- 172.21.15.43:0/486852569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55feb67afd60 0x55feb67becb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.128+0000 7f6cd659c640 1 -- 172.21.15.43:0/486852569 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feb67ae560 con 0x55feb67a9ba0 2024-04-21T08:46:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.128+0000 7f6cd6d9d640 1 --2- 172.21.15.43:0/486852569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55feb67afd60 0x55feb67becb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.128+0000 7f6cd659c640 1 --2- 172.21.15.43:0/486852569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67a9ba0 0x55feb67c8cd0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800fc90 tx=0x7f6cc800a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565dacc2667cd6e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.129+0000 7f6cbffff640 1 -- 172.21.15.43:0/486852569 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc801d040 con 0x55feb67a9ba0 2024-04-21T08:46:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.129+0000 7f6cbffff640 1 -- 172.21.15.43:0/486852569 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cc801e040 con 0x55feb67a9ba0 2024-04-21T08:46:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.129+0000 7f6cbffff640 1 -- 172.21.15.43:0/486852569 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8018770 con 0x55feb67a9ba0 2024-04-21T08:46:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.129+0000 7f6cdf632740 1 -- 172.21.15.43:0/486852569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67a9ba0 msgr2=0x55feb67c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.129+0000 7f6cdf632740 1 --2- 172.21.15.43:0/486852569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67a9ba0 0x55feb67c8cd0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800fc90 tx=0x7f6cc800a530 comp rx=0 tx=0).stop 2024-04-21T08:46:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.129+0000 7f6cdf632740 1 -- 172.21.15.43:0/486852569 shutdown_connections 2024-04-21T08:46:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.129+0000 7f6cdf632740 1 --2- 172.21.15.43:0/486852569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55feb67afd60 0x55feb67becb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.129+0000 7f6cdf632740 1 --2- 172.21.15.43:0/486852569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55feb67af3b0 0x55feb67a8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.129+0000 7f6cdf632740 1 --2- 172.21.15.43:0/486852569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67a9ba0 0x55feb67c8cd0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.129+0000 7f6cdf632740 1 -- 172.21.15.43:0/486852569 >> 172.21.15.43:0/486852569 conn(0x55feb67bcc20 msgr2=0x55feb67a8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.129+0000 7f6cdf632740 1 -- 172.21.15.43:0/486852569 shutdown_connections 2024-04-21T08:46:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.129+0000 7f6cdf632740 1 -- 172.21.15.43:0/486852569 wait complete. 2024-04-21T08:46:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.130+0000 7f6cdf632740 1 -- start start 2024-04-21T08:46:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.130+0000 7f6cdf632740 1 Processor -- start 2024-04-21T08:46:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.130+0000 7f6cdf632740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55feb67a9ba0 0x55feb67be480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.130+0000 7f6cdf632740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55feb67af3b0 0x55feb67be9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.130+0000 7f6cdf632740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67afd60 0x55feb67bef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.130+0000 7f6cdf632740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd81ae690 con 0x55feb67afd60 2024-04-21T08:46:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.130+0000 7f6cdf632740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd81adfc0 con 0x55feb67a9ba0 2024-04-21T08:46:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.130+0000 7f6cdf632740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd81ab220 con 0x55feb67af3b0 2024-04-21T08:46:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.130+0000 7f6cdf632740 10 client.0 _ll_register_callbacks cb 0x7f6cd0010b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.130+0000 7f6cd6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67afd60 0x55feb67bef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.130+0000 7f6cd6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67afd60 0x55feb67bef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37576/0 (socket says 172.21.15.43:37576) 2024-04-21T08:46:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.130+0000 7f6cd6d9d640 1 -- 172.21.15.43:0/1561419712 learned_addr learned my addr 172.21.15.43:0/1561419712 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.130+0000 7f6cd659c640 1 --2- 172.21.15.43:0/1561419712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55feb67a9ba0 0x55feb67be480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.130+0000 7f6cd6d9d640 1 -- 172.21.15.43:0/1561419712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55feb67af3b0 msgr2=0x55feb67be9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.130+0000 7f6cd6d9d640 1 --2- 172.21.15.43:0/1561419712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55feb67af3b0 0x55feb67be9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.130+0000 7f6cd6d9d640 1 -- 172.21.15.43:0/1561419712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55feb67a9ba0 msgr2=0x55feb67be480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.130+0000 7f6cd6d9d640 1 --2- 172.21.15.43:0/1561419712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55feb67a9ba0 0x55feb67be480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.130+0000 7f6cd6d9d640 1 -- 172.21.15.43:0/1561419712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feb6832840 con 0x55feb67afd60 2024-04-21T08:46:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.130+0000 7f6cd6d9d640 1 --2- 172.21.15.43:0/1561419712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67afd60 0x55feb67bef00 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f6cd00278a0 tx=0x7f6cd00278d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41c9ee6d5993bc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.130+0000 7f6cbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.131+0000 7f6cbdffb640 1 -- 172.21.15.43:0/1561419712 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0018040 con 0x55feb67afd60 2024-04-21T08:46:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.131+0000 7f6cbdffb640 1 -- 172.21.15.43:0/1561419712 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9583+0+0 (secure 0 0 0) 0x7f6cd000fe10 con 0x55feb67afd60 2024-04-21T08:46:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.131+0000 7f6cdf632740 1 -- 172.21.15.43:0/1561419712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cc801e040 con 0x55feb67afd60 2024-04-21T08:46:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.131+0000 7f6cbdffb640 1 -- 172.21.15.43:0/1561419712 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6cd0043050 con 0x55feb67afd60 2024-04-21T08:46:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.131+0000 7f6cdf632740 10 client.19011 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.131+0000 7f6cdf632740 1 -- 172.21.15.43:0/1561419712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55feb6833a90 con 0x55feb67afd60 2024-04-21T08:46:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.131+0000 7f6cbdffb640 1 -- 172.21.15.43:0/1561419712 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cc801e040 con 0x55feb67afd60 2024-04-21T08:46:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.131+0000 7f6cbdffb640 1 -- 172.21.15.43:0/1561419712 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6cd0003a70 con 0x55feb67afd60 2024-04-21T08:46:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.131+0000 7f6cdf632740 10 client.19011 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.131+0000 7f6cdf632740 10 client.19011 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:46:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.131+0000 7f6cdf632740 20 client.19011 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.131+0000 7f6cdf632740 1 --2- 172.21.15.43:0/1561419712 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55feb67b8e60 0x55feb6836220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.131+0000 7f6cdf632740 4 client.19011 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.131+0000 7f6cdf632740 1 -- 172.21.15.43:0/1561419712 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55feb6721c10 con 0x55feb67b8e60 2024-04-21T08:46:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.131+0000 7f6cd659c640 1 --2- 172.21.15.43:0/1561419712 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55feb67b8e60 0x55feb6836220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.132+0000 7f6cd659c640 1 --2- 172.21.15.43:0/1561419712 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55feb67b8e60 0x55feb6836220 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1bd26c50b841209b server_cookie=766b346016cc0576 in_seq=0 out_seq=0 2024-04-21T08:46:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.132+0000 7f6cbdffb640 10 client.19011 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:46:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.132+0000 7f6cbdffb640 1 -- 172.21.15.43:0/1561419712 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55feb6721c10 con 0x55feb67b8e60 2024-04-21T08:46:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.132+0000 7f6cbdffb640 10 client.19011 handle_command_reply: tid=0 2024-04-21T08:46:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.132+0000 7f6cdf632740 1 client.19011 shutdown 2024-04-21T08:46:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.132+0000 7f6cdf632740 1 -- 172.21.15.43:0/1561419712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67afd60 msgr2=0x55feb67bef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.132+0000 7f6cdf632740 1 --2- 172.21.15.43:0/1561419712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67afd60 0x55feb67bef00 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f6cd00278a0 tx=0x7f6cd00278d0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.132+0000 7f6cdf632740 1 -- 172.21.15.43:0/1561419712 shutdown_connections 2024-04-21T08:46:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.132+0000 7f6cdf632740 1 --2- 172.21.15.43:0/1561419712 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55feb67b8e60 0x55feb6836220 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.132+0000 7f6cdf632740 1 --2- 172.21.15.43:0/1561419712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55feb67afd60 0x55feb67bef00 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.132+0000 7f6cdf632740 1 --2- 172.21.15.43:0/1561419712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55feb67af3b0 0x55feb67be9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.132+0000 7f6cbdffb640 0 client.19011 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:46:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.132+0000 7f6cdf632740 1 --2- 172.21.15.43:0/1561419712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55feb67a9ba0 0x55feb67be480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.132+0000 7f6cdf632740 1 -- 172.21.15.43:0/1561419712 >> 172.21.15.43:0/1561419712 conn(0x55feb67bcc20 msgr2=0x55feb67a8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.132+0000 7f6cdf632740 1 -- 172.21.15.43:0/1561419712 shutdown_connections 2024-04-21T08:46:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.132+0000 7f6cdf632740 1 -- 172.21.15.43:0/1561419712 wait complete. 2024-04-21T08:46:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.132+0000 7f6cdf632740 20 client.19011 trim_cache size 0 max 16384 2024-04-21T08:46:10.139 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:46:10.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.133+0000 7f6cdca48640 1 -- 172.21.15.43:0/334765430 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6cc00a3d20 msgr2=0x7f6cc00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.133+0000 7f6cdca48640 1 --2- 172.21.15.43:0/334765430 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6cc00a3d20 0x7f6cc00a61e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0012730 tx=0x7f6cd0004ab0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.133+0000 7f6cdca48640 1 -- 172.21.15.43:0/334765430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cd8107270 msgr2=0x7f6cd8198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.133+0000 7f6cdca48640 1 --2- 172.21.15.43:0/334765430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cd8107270 0x7f6cd8198bd0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc005ec0 tx=0x7f6ccc02ca10 comp rx=0 tx=0).stop 2024-04-21T08:46:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.133+0000 7f6cdca48640 1 -- 172.21.15.43:0/334765430 shutdown_connections 2024-04-21T08:46:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.133+0000 7f6cdca48640 1 --2- 172.21.15.43:0/334765430 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6cc00a3d20 0x7f6cc00a61e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.133+0000 7f6cdca48640 1 --2- 172.21.15.43:0/334765430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cd819cf50 0x7f6cd819d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.133+0000 7f6cdca48640 1 --2- 172.21.15.43:0/334765430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cd8107270 0x7f6cd8198bd0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.133+0000 7f6cdca48640 1 --2- 172.21.15.43:0/334765430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd806a840 0x7f6cd8198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.133+0000 7f6cdca48640 1 -- 172.21.15.43:0/334765430 >> 172.21.15.43:0/334765430 conn(0x7f6cd8077eb0 msgr2=0x7f6cd8100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.133+0000 7f6cdca48640 1 -- 172.21.15.43:0/334765430 shutdown_connections 2024-04-21T08:46:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.133+0000 7f6cdca48640 1 -- 172.21.15.43:0/334765430 wait complete. 2024-04-21T08:46:10.143 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:46:10.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.248+0000 7fc1a2826640 1 -- 172.21.15.43:0/3554493121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19c103050 msgr2=0x7fc19c100450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.248+0000 7fc1a2826640 1 --2- 172.21.15.43:0/3554493121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19c103050 0x7fc19c100450 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009900 tx=0x7fc18c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:10.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.248+0000 7fc1a2826640 1 -- 172.21.15.43:0/3554493121 shutdown_connections 2024-04-21T08:46:10.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.248+0000 7fc1a2826640 1 --2- 172.21.15.43:0/3554493121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc19c10ebd0 0x7fc19c110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.248+0000 7fc1a2826640 1 --2- 172.21.15.43:0/3554493121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc19c100990 0x7fc19c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.248+0000 7fc1a2826640 1 --2- 172.21.15.43:0/3554493121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19c103050 0x7fc19c100450 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.248+0000 7fc1a2826640 1 -- 172.21.15.43:0/3554493121 >> 172.21.15.43:0/3554493121 conn(0x7fc19c077410 msgr2=0x7fc19c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:10.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.248+0000 7fc1a2826640 1 -- 172.21.15.43:0/3554493121 shutdown_connections 2024-04-21T08:46:10.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.248+0000 7fc1a2826640 1 -- 172.21.15.43:0/3554493121 wait complete. 2024-04-21T08:46:10.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.249+0000 7fc1a2826640 1 Processor -- start 2024-04-21T08:46:10.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.249+0000 7fc1a2826640 1 -- start start 2024-04-21T08:46:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.249+0000 7fc1a2826640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19c100990 0x7fc19c1a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.249+0000 7fc1a2826640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc19c103050 0x7fc19c1a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.249+0000 7fc1a2826640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc19c10ebd0 0x7fc19c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.249+0000 7fc1a2826640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c1145d0 con 0x7fc19c100990 2024-04-21T08:46:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.249+0000 7fc1a2826640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c114450 con 0x7fc19c10ebd0 2024-04-21T08:46:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.249+0000 7fc1a2826640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c114750 con 0x7fc19c103050 2024-04-21T08:46:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.249+0000 7fc19bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19c100990 0x7fc19c1a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.249+0000 7fc19bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19c100990 0x7fc19c1a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37592/0 (socket says 172.21.15.43:37592) 2024-04-21T08:46:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.249+0000 7fc19bfff640 1 -- 172.21.15.43:0/1231242891 learned_addr learned my addr 172.21.15.43:0/1231242891 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.249+0000 7fc1a0dc3640 1 --2- 172.21.15.43:0/1231242891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc19c10ebd0 0x7fc19c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.249+0000 7fc19b7fe640 1 --2- 172.21.15.43:0/1231242891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc19c103050 0x7fc19c1a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.249+0000 7fc19bfff640 1 -- 172.21.15.43:0/1231242891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc19c103050 msgr2=0x7fc19c1a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.249+0000 7fc19bfff640 1 --2- 172.21.15.43:0/1231242891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc19c103050 0x7fc19c1a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.249+0000 7fc19bfff640 1 -- 172.21.15.43:0/1231242891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc19c10ebd0 msgr2=0x7fc19c19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.249+0000 7fc19bfff640 1 --2- 172.21.15.43:0/1231242891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc19c10ebd0 0x7fc19c19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.249+0000 7fc19bfff640 1 -- 172.21.15.43:0/1231242891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19c19b1e0 con 0x7fc19c100990 2024-04-21T08:46:10.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.249+0000 7fc1a0dc3640 1 --2- 172.21.15.43:0/1231242891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc19c10ebd0 0x7fc19c19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:10.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.250+0000 7fc19bfff640 1 --2- 172.21.15.43:0/1231242891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19c100990 0x7fc19c1a08b0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fc18c006010 tx=0x7fc18c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:10.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.250+0000 7fc1997fa640 1 -- 172.21.15.43:0/1231242891 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c043020 con 0x7fc19c100990 2024-04-21T08:46:10.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.250+0000 7fc1997fa640 1 -- 172.21.15.43:0/1231242891 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc18c004440 con 0x7fc19c100990 2024-04-21T08:46:10.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.250+0000 7fc1997fa640 1 -- 172.21.15.43:0/1231242891 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c003b50 con 0x7fc19c100990 2024-04-21T08:46:10.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.250+0000 7fc1a2826640 1 -- 172.21.15.43:0/1231242891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19c19b470 con 0x7fc19c100990 2024-04-21T08:46:10.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.250+0000 7fc1a2826640 1 -- 172.21.15.43:0/1231242891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc19c1a7770 con 0x7fc19c100990 2024-04-21T08:46:10.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.252+0000 7fc1997fa640 1 -- 172.21.15.43:0/1231242891 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc18c046020 con 0x7fc19c100990 2024-04-21T08:46:10.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.252+0000 7fc1a540f740 1 Processor -- start 2024-04-21T08:46:10.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.252+0000 7fc1a540f740 1 -- start start 2024-04-21T08:46:10.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.253+0000 7fc1a540f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b700b084d0 0x55b700b088b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.253+0000 7fc1a540f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b700b08e80 0x55b700b11430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.253+0000 7fc1a540f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b11a00 0x55b700b13e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.253+0000 7fc1a540f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b700a6d1f0 con 0x55b700b11a00 2024-04-21T08:46:10.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.253+0000 7fc1a540f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b700a0a4b0 con 0x55b700b08e80 2024-04-21T08:46:10.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.253+0000 7fc1a540f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b700a691b0 con 0x55b700b084d0 2024-04-21T08:46:10.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.253+0000 7fc1997fa640 1 --2- 172.21.15.43:0/1231242891 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc1840a3d20 0x7fc1840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.253+0000 7fc19b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b11a00 0x55b700b13e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.253+0000 7fc19b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b11a00 0x55b700b13e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37602/0 (socket says 172.21.15.43:37602) 2024-04-21T08:46:10.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.253+0000 7fc19b7fe640 1 -- 172.21.15.43:0/446078048 learned_addr learned my addr 172.21.15.43:0/446078048 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:10.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.253+0000 7fc1a0dc3640 1 --2- 172.21.15.43:0/446078048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b700b084d0 0x55b700b088b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.253+0000 7fc1997fa640 1 -- 172.21.15.43:0/1231242891 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9583+0+0 (secure 0 0 0) 0x7fc18c0f2750 con 0x7fc19c100990 2024-04-21T08:46:10.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.253+0000 7fc1a0dc3640 1 --2- 172.21.15.43:0/1231242891 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc1840a3d20 0x7fc1840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.253+0000 7fc19bfff640 1 --2- 172.21.15.43:0/446078048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b700b08e80 0x55b700b11430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.253+0000 7fc19b7fe640 1 -- 172.21.15.43:0/446078048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b700b084d0 msgr2=0x55b700b088b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:10.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.253+0000 7fc19b7fe640 1 --2- 172.21.15.43:0/446078048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b700b084d0 0x55b700b088b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.253+0000 7fc19b7fe640 1 -- 172.21.15.43:0/446078048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b700b08e80 msgr2=0x55b700b11430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:10.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.253+0000 7fc19b7fe640 1 --2- 172.21.15.43:0/446078048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b700b08e80 0x55b700b11430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.253+0000 7fc19b7fe640 1 -- 172.21.15.43:0/446078048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7009fc1b0 con 0x55b700b11a00 2024-04-21T08:46:10.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.253+0000 7fc19bfff640 1 --2- 172.21.15.43:0/446078048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b700b08e80 0x55b700b11430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:10.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.254+0000 7fc19b7fe640 1 --2- 172.21.15.43:0/446078048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b11a00 0x55b700b13e50 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fc19000bc50 tx=0x7fc19000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81fff2a40d28604c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:10.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.254+0000 7fc1a0dc3640 1 --2- 172.21.15.43:0/1231242891 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc1840a3d20 0x7fc1840a61e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc194005c40 tx=0x7fc194009280 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:10.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.256+0000 7fc1827fc640 1 -- 172.21.15.43:0/446078048 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190015070 con 0x55b700b11a00 2024-04-21T08:46:10.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.256+0000 7fc1827fc640 1 -- 172.21.15.43:0/446078048 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc19001a910 con 0x55b700b11a00 2024-04-21T08:46:10.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.256+0000 7fc1827fc640 1 -- 172.21.15.43:0/446078048 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190011980 con 0x55b700b11a00 2024-04-21T08:46:10.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.257+0000 7fc1a540f740 1 -- 172.21.15.43:0/446078048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b11a00 msgr2=0x55b700b13e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.257+0000 7fc1a540f740 1 --2- 172.21.15.43:0/446078048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b11a00 0x55b700b13e50 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fc19000bc50 tx=0x7fc19000f730 comp rx=0 tx=0).stop 2024-04-21T08:46:10.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.257+0000 7fc1a540f740 1 -- 172.21.15.43:0/446078048 shutdown_connections 2024-04-21T08:46:10.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.257+0000 7fc1a540f740 1 --2- 172.21.15.43:0/446078048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b11a00 0x55b700b13e50 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.257+0000 7fc1a540f740 1 --2- 172.21.15.43:0/446078048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b700b08e80 0x55b700b11430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.257+0000 7fc1a540f740 1 --2- 172.21.15.43:0/446078048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b700b084d0 0x55b700b088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.257+0000 7fc1a540f740 1 -- 172.21.15.43:0/446078048 >> 172.21.15.43:0/446078048 conn(0x55b700b02ba0 msgr2=0x55b700b02f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:10.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.257+0000 7fc1a540f740 1 -- 172.21.15.43:0/446078048 shutdown_connections 2024-04-21T08:46:10.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.257+0000 7fc1a540f740 1 -- 172.21.15.43:0/446078048 wait complete. 2024-04-21T08:46:10.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.258+0000 7fc1a540f740 1 -- start start 2024-04-21T08:46:10.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.258+0000 7fc1a540f740 1 Processor -- start 2024-04-21T08:46:10.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.258+0000 7fc1a540f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b700b084d0 0x55b700b11260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.258+0000 7fc1a540f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b700b08e80 0x55b700b0e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.258+0000 7fc1a540f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b11a00 0x55b700b0eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.258+0000 7fc1a540f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b700b14970 con 0x55b700b11a00 2024-04-21T08:46:10.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.258+0000 7fc1a540f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b700a26500 con 0x55b700b08e80 2024-04-21T08:46:10.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.258+0000 7fc1a540f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b700b14af0 con 0x55b700b084d0 2024-04-21T08:46:10.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.258+0000 7fc1a540f740 10 client.0 _ll_register_callbacks cb 0x55b7009f4b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:10.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.258+0000 7fc1a0dc3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b11a00 0x55b700b0eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.258+0000 7fc1a0dc3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b11a00 0x55b700b0eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37616/0 (socket says 172.21.15.43:37616) 2024-04-21T08:46:10.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.258+0000 7fc1a0dc3640 1 -- 172.21.15.43:0/4115342172 learned_addr learned my addr 172.21.15.43:0/4115342172 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:10.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.258+0000 7fc19b7fe640 1 --2- 172.21.15.43:0/4115342172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b700b08e80 0x55b700b0e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.258+0000 7fc19bfff640 1 --2- 172.21.15.43:0/4115342172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b700b084d0 0x55b700b11260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.259+0000 7fc1a0dc3640 1 -- 172.21.15.43:0/4115342172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b700b084d0 msgr2=0x55b700b11260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:10.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.259+0000 7fc1a0dc3640 1 --2- 172.21.15.43:0/4115342172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b700b084d0 0x55b700b11260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.259+0000 7fc1a0dc3640 1 -- 172.21.15.43:0/4115342172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b700b08e80 msgr2=0x55b700b0e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:10.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.259+0000 7fc1a0dc3640 1 --2- 172.21.15.43:0/4115342172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b700b08e80 0x55b700b0e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.259+0000 7fc1a0dc3640 1 -- 172.21.15.43:0/4115342172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b70077f3f0 con 0x55b700b11a00 2024-04-21T08:46:10.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.259+0000 7fc1a0dc3640 1 --2- 172.21.15.43:0/4115342172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b11a00 0x55b700b0eb30 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fc194012fb0 tx=0x7fc194004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52857c77a9487c49 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:10.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.259+0000 7fc17b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:10.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.259+0000 7fc17b7fe640 1 -- 172.21.15.43:0/4115342172 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194025070 con 0x55b700b11a00 2024-04-21T08:46:10.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.259+0000 7fc17b7fe640 1 -- 172.21.15.43:0/4115342172 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9583+0+0 (secure 0 0 0) 0x7fc194010d60 con 0x55b700b11a00 2024-04-21T08:46:10.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.259+0000 7fc1a540f740 1 -- 172.21.15.43:0/4115342172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc19001a910 con 0x55b700b11a00 2024-04-21T08:46:10.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.259+0000 7fc17b7fe640 1 -- 172.21.15.43:0/4115342172 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc19402a460 con 0x55b700b11a00 2024-04-21T08:46:10.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.259+0000 7fc1a540f740 10 client.19023 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:10.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.259+0000 7fc1a540f740 1 -- 172.21.15.43:0/4115342172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b7008497d0 con 0x55b700b11a00 2024-04-21T08:46:10.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.259+0000 7fc17b7fe640 1 -- 172.21.15.43:0/4115342172 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc19001a910 con 0x55b700b11a00 2024-04-21T08:46:10.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.259+0000 7fc17b7fe640 1 -- 172.21.15.43:0/4115342172 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc19401e040 con 0x55b700b11a00 2024-04-21T08:46:10.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.259+0000 7fc1a540f740 10 client.19023 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:10.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.259+0000 7fc1a540f740 10 client.19023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:10.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.259+0000 7fc1a540f740 20 client.19023 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:10.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.259+0000 7fc1a540f740 1 --2- 172.21.15.43:0/4115342172 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55b700b21b80 0x55b700b23f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.260+0000 7fc1a540f740 4 client.19023 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:10.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.260+0000 7fc1a540f740 1 -- 172.21.15.43:0/4115342172 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b700a0c930 con 0x55b700b21b80 2024-04-21T08:46:10.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.260+0000 7fc19bfff640 1 --2- 172.21.15.43:0/4115342172 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55b700b21b80 0x55b700b23f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.260+0000 7fc17b7fe640 10 client.19023 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:10.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.260+0000 7fc19bfff640 1 --2- 172.21.15.43:0/4115342172 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55b700b21b80 0x55b700b23f90 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de9ef6ebb2519910 server_cookie=ab2383d424bc5404 in_seq=0 out_seq=0 2024-04-21T08:46:10.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.261+0000 7fc17b7fe640 1 -- 172.21.15.43:0/4115342172 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b700a0c930 con 0x55b700b21b80 2024-04-21T08:46:10.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.261+0000 7fc17b7fe640 10 client.19023 handle_command_reply: tid=0 2024-04-21T08:46:10.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.261+0000 7fc1a540f740 1 client.19023 shutdown 2024-04-21T08:46:10.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.261+0000 7fc1a540f740 1 -- 172.21.15.43:0/4115342172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b11a00 msgr2=0x55b700b0eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.261+0000 7fc1a540f740 1 --2- 172.21.15.43:0/4115342172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b11a00 0x55b700b0eb30 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fc194012fb0 tx=0x7fc194004a40 comp rx=0 tx=0).stop 2024-04-21T08:46:10.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.261+0000 7fc1a540f740 1 -- 172.21.15.43:0/4115342172 shutdown_connections 2024-04-21T08:46:10.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.261+0000 7fc1a540f740 1 --2- 172.21.15.43:0/4115342172 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55b700b21b80 0x55b700b23f90 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.261+0000 7fc1a540f740 1 --2- 172.21.15.43:0/4115342172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b11a00 0x55b700b0eb30 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.261+0000 7fc1a540f740 1 --2- 172.21.15.43:0/4115342172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b700b08e80 0x55b700b0e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.261+0000 7fc17b7fe640 0 client.19023 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:10.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.261+0000 7fc1a540f740 1 --2- 172.21.15.43:0/4115342172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b700b084d0 0x55b700b11260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.261+0000 7fc1a540f740 1 -- 172.21.15.43:0/4115342172 >> 172.21.15.43:0/4115342172 conn(0x55b700b02ba0 msgr2=0x55b700b048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:10.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.261+0000 7fc1a540f740 1 -- 172.21.15.43:0/4115342172 shutdown_connections 2024-04-21T08:46:10.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.261+0000 7fc1a540f740 1 -- 172.21.15.43:0/4115342172 wait complete. 2024-04-21T08:46:10.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.261+0000 7fc1a540f740 20 client.19023 trim_cache size 0 max 16384 2024-04-21T08:46:10.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.275+0000 7fc1a540f740 1 Processor -- start 2024-04-21T08:46:10.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.275+0000 7fc1a540f740 1 -- start start 2024-04-21T08:46:10.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.275+0000 7fc1a540f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b084d0 0x55b700b1d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.275+0000 7fc1a540f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b700b08e80 0x55b700b04700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.275+0000 7fc1a540f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b700b11a00 0x55b700b10d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.275+0000 7fc1a540f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c1b1690 con 0x55b700b084d0 2024-04-21T08:46:10.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.275+0000 7fc1a540f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c1b31d0 con 0x55b700b08e80 2024-04-21T08:46:10.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.275+0000 7fc1a540f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c1b1d60 con 0x55b700b11a00 2024-04-21T08:46:10.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.275+0000 7fc19bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b084d0 0x55b700b1d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.275+0000 7fc19bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b084d0 0x55b700b1d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37632/0 (socket says 172.21.15.43:37632) 2024-04-21T08:46:10.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.275+0000 7fc19bfff640 1 -- 172.21.15.43:0/3805395294 learned_addr learned my addr 172.21.15.43:0/3805395294 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:10.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.275+0000 7fc1a0dc3640 1 --2- 172.21.15.43:0/3805395294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b700b11a00 0x55b700b10d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.275+0000 7fc19b7fe640 1 --2- 172.21.15.43:0/3805395294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b700b08e80 0x55b700b04700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.275+0000 7fc19bfff640 1 -- 172.21.15.43:0/3805395294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b700b11a00 msgr2=0x55b700b10d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:10.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.275+0000 7fc19bfff640 1 --2- 172.21.15.43:0/3805395294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b700b11a00 0x55b700b10d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.275+0000 7fc19bfff640 1 -- 172.21.15.43:0/3805395294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b700b08e80 msgr2=0x55b700b04700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:10.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.275+0000 7fc19bfff640 1 --2- 172.21.15.43:0/3805395294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b700b08e80 0x55b700b04700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.275+0000 7fc19bfff640 1 -- 172.21.15.43:0/3805395294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b700b142b0 con 0x55b700b084d0 2024-04-21T08:46:10.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.275+0000 7fc19bfff640 1 --2- 172.21.15.43:0/3805395294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b084d0 0x55b700b1d830 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fc18c0efc40 tx=0x7fc18c0ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ded83c0914ee24 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:10.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.275+0000 7fc181ffb640 1 -- 172.21.15.43:0/3805395294 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c0c2790 con 0x55b700b084d0 2024-04-21T08:46:10.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.276+0000 7fc181ffb640 1 -- 172.21.15.43:0/3805395294 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc18c0c7050 con 0x55b700b084d0 2024-04-21T08:46:10.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.276+0000 7fc181ffb640 1 -- 172.21.15.43:0/3805395294 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c0baa50 con 0x55b700b084d0 2024-04-21T08:46:10.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.276+0000 7fc1a540f740 1 -- 172.21.15.43:0/3805395294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b084d0 msgr2=0x55b700b1d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.276+0000 7fc1a540f740 1 --2- 172.21.15.43:0/3805395294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b084d0 0x55b700b1d830 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fc18c0efc40 tx=0x7fc18c0ba040 comp rx=0 tx=0).stop 2024-04-21T08:46:10.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.276+0000 7fc1a540f740 1 -- 172.21.15.43:0/3805395294 shutdown_connections 2024-04-21T08:46:10.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.276+0000 7fc1a540f740 1 --2- 172.21.15.43:0/3805395294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b700b11a00 0x55b700b10d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.276+0000 7fc1a540f740 1 --2- 172.21.15.43:0/3805395294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b700b08e80 0x55b700b04700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.276+0000 7fc1a540f740 1 --2- 172.21.15.43:0/3805395294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b084d0 0x55b700b1d830 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.276+0000 7fc1a540f740 1 -- 172.21.15.43:0/3805395294 >> 172.21.15.43:0/3805395294 conn(0x55b700b02ba0 msgr2=0x55b700b03ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:10.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.276+0000 7fc1a540f740 1 -- 172.21.15.43:0/3805395294 shutdown_connections 2024-04-21T08:46:10.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.276+0000 7fc1a540f740 1 -- 172.21.15.43:0/3805395294 wait complete. 2024-04-21T08:46:10.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.276+0000 7fc1a540f740 1 -- start start 2024-04-21T08:46:10.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.277+0000 7fc1a540f740 1 Processor -- start 2024-04-21T08:46:10.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.277+0000 7fc1a540f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b084d0 0x55b700b19fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.277+0000 7fc1a540f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b700b08e80 0x55b700b0aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.277+0000 7fc1a540f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b700b11a00 0x55b700b0b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.277+0000 7fc1a540f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c1b2b00 con 0x55b700b084d0 2024-04-21T08:46:10.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.277+0000 7fc1a540f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c1b2430 con 0x55b700b11a00 2024-04-21T08:46:10.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.277+0000 7fc1a540f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c1af690 con 0x55b700b08e80 2024-04-21T08:46:10.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.277+0000 7fc1a540f740 10 client.0 _ll_register_callbacks cb 0x7fc1940128c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:10.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.277+0000 7fc19bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b084d0 0x55b700b19fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.277+0000 7fc19bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b084d0 0x55b700b19fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37644/0 (socket says 172.21.15.43:37644) 2024-04-21T08:46:10.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.277+0000 7fc19bfff640 1 -- 172.21.15.43:0/3631658020 learned_addr learned my addr 172.21.15.43:0/3631658020 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:10.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.277+0000 7fc19b7fe640 1 --2- 172.21.15.43:0/3631658020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b700b08e80 0x55b700b0aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.277+0000 7fc1a0dc3640 1 --2- 172.21.15.43:0/3631658020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b700b11a00 0x55b700b0b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.277+0000 7fc19bfff640 1 -- 172.21.15.43:0/3631658020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b700b08e80 msgr2=0x55b700b0aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:10.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.277+0000 7fc19bfff640 1 --2- 172.21.15.43:0/3631658020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b700b08e80 0x55b700b0aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.277+0000 7fc19bfff640 1 -- 172.21.15.43:0/3631658020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b700b11a00 msgr2=0x55b700b0b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:10.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.277+0000 7fc19bfff640 1 --2- 172.21.15.43:0/3631658020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b700b11a00 0x55b700b0b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.277+0000 7fc19bfff640 1 -- 172.21.15.43:0/3631658020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b700b0baa0 con 0x55b700b084d0 2024-04-21T08:46:10.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.277+0000 7fc1a0dc3640 1 --2- 172.21.15.43:0/3631658020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b700b11a00 0x55b700b0b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:10.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.278+0000 7fc19bfff640 1 --2- 172.21.15.43:0/3631658020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b084d0 0x55b700b19fa0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fc18c0be370 tx=0x7fc18c0be050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f068446cd92f300 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:10.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.278+0000 7fc17b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:10.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.278+0000 7fc17b7fe640 1 -- 172.21.15.43:0/3631658020 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c0054b0 con 0x55b700b084d0 2024-04-21T08:46:10.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.278+0000 7fc17b7fe640 1 -- 172.21.15.43:0/3631658020 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9583+0+0 (secure 0 0 0) 0x7fc18c0baa50 con 0x55b700b084d0 2024-04-21T08:46:10.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.278+0000 7fc1a540f740 1 -- 172.21.15.43:0/3631658020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc18c0c7050 con 0x55b700b084d0 2024-04-21T08:46:10.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.278+0000 7fc17b7fe640 1 -- 172.21.15.43:0/3631658020 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc18c0efd00 con 0x55b700b084d0 2024-04-21T08:46:10.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.278+0000 7fc1a540f740 10 client.19029 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:10.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.278+0000 7fc1a540f740 1 -- 172.21.15.43:0/3631658020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b700b87fa0 con 0x55b700b084d0 2024-04-21T08:46:10.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.278+0000 7fc17b7fe640 1 -- 172.21.15.43:0/3631658020 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc18c0c7050 con 0x55b700b084d0 2024-04-21T08:46:10.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.278+0000 7fc17b7fe640 1 -- 172.21.15.43:0/3631658020 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc18c0f2bd0 con 0x55b700b084d0 2024-04-21T08:46:10.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.278+0000 7fc1a540f740 10 client.19029 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:10.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.278+0000 7fc1a540f740 10 client.19029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:10.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.278+0000 7fc1a540f740 20 client.19029 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:10.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.278+0000 7fc1a540f740 1 --2- 172.21.15.43:0/3631658020 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55b700b88600 0x55b700b8aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.278+0000 7fc1a540f740 4 client.19029 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:10.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.278+0000 7fc1a540f740 1 -- 172.21.15.43:0/3631658020 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b700b0eec0 con 0x55b700b88600 2024-04-21T08:46:10.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.279+0000 7fc19b7fe640 1 --2- 172.21.15.43:0/3631658020 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55b700b88600 0x55b700b8aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.279+0000 7fc19b7fe640 1 --2- 172.21.15.43:0/3631658020 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55b700b88600 0x55b700b8aa20 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5d6b53eeed36f84 server_cookie=445e26caf1126468 in_seq=0 out_seq=0 2024-04-21T08:46:10.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.279+0000 7fc17b7fe640 10 client.19029 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:10.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.279+0000 7fc17b7fe640 1 -- 172.21.15.43:0/3631658020 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b700b0eec0 con 0x55b700b88600 2024-04-21T08:46:10.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.279+0000 7fc17b7fe640 10 client.19029 handle_command_reply: tid=0 2024-04-21T08:46:10.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.279+0000 7fc1a540f740 1 client.19029 shutdown 2024-04-21T08:46:10.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.280+0000 7fc1a540f740 1 -- 172.21.15.43:0/3631658020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b084d0 msgr2=0x55b700b19fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.280+0000 7fc1a540f740 1 --2- 172.21.15.43:0/3631658020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b084d0 0x55b700b19fa0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fc18c0be370 tx=0x7fc18c0be050 comp rx=0 tx=0).stop 2024-04-21T08:46:10.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.280+0000 7fc1a540f740 1 -- 172.21.15.43:0/3631658020 shutdown_connections 2024-04-21T08:46:10.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.280+0000 7fc1a540f740 1 --2- 172.21.15.43:0/3631658020 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55b700b88600 0x55b700b8aa20 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.280+0000 7fc1a540f740 1 --2- 172.21.15.43:0/3631658020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b700b11a00 0x55b700b0b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.280+0000 7fc1a540f740 1 --2- 172.21.15.43:0/3631658020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b700b08e80 0x55b700b0aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.280+0000 7fc17b7fe640 0 client.19029 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:10.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.280+0000 7fc1a540f740 1 --2- 172.21.15.43:0/3631658020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b700b084d0 0x55b700b19fa0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.280+0000 7fc1a540f740 1 -- 172.21.15.43:0/3631658020 >> 172.21.15.43:0/3631658020 conn(0x55b700b02ba0 msgr2=0x55b700b04880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:10.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.280+0000 7fc1a540f740 1 -- 172.21.15.43:0/3631658020 shutdown_connections 2024-04-21T08:46:10.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.280+0000 7fc1a540f740 1 -- 172.21.15.43:0/3631658020 wait complete. 2024-04-21T08:46:10.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.280+0000 7fc1a540f740 20 client.19029 trim_cache size 0 max 16384 2024-04-21T08:46:10.286 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:46:10.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.280+0000 7fc1a2826640 1 -- 172.21.15.43:0/1231242891 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc1840a3d20 msgr2=0x7fc1840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.280+0000 7fc1a2826640 1 --2- 172.21.15.43:0/1231242891 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc1840a3d20 0x7fc1840a61e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc194005c40 tx=0x7fc194009280 comp rx=0 tx=0).stop 2024-04-21T08:46:10.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.280+0000 7fc1a2826640 1 -- 172.21.15.43:0/1231242891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19c100990 msgr2=0x7fc19c1a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.280+0000 7fc1a2826640 1 --2- 172.21.15.43:0/1231242891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19c100990 0x7fc19c1a08b0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fc18c006010 tx=0x7fc18c004040 comp rx=0 tx=0).stop 2024-04-21T08:46:10.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.280+0000 7fc1a2826640 1 -- 172.21.15.43:0/1231242891 shutdown_connections 2024-04-21T08:46:10.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.280+0000 7fc1a2826640 1 --2- 172.21.15.43:0/1231242891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc19c10ebd0 0x7fc19c19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.281+0000 7fc1a2826640 1 --2- 172.21.15.43:0/1231242891 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc1840a3d20 0x7fc1840a61e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.281+0000 7fc1a2826640 1 --2- 172.21.15.43:0/1231242891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc19c103050 0x7fc19c1a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.281+0000 7fc1a2826640 1 --2- 172.21.15.43:0/1231242891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19c100990 0x7fc19c1a08b0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.281+0000 7fc1a2826640 1 -- 172.21.15.43:0/1231242891 >> 172.21.15.43:0/1231242891 conn(0x7fc19c077410 msgr2=0x7fc19c110150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:10.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.281+0000 7fc1a2826640 1 -- 172.21.15.43:0/1231242891 shutdown_connections 2024-04-21T08:46:10.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:10.281+0000 7fc1a2826640 1 -- 172.21.15.43:0/1231242891 wait complete. 2024-04-21T08:46:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:10 smithi112 ceph-mon[31085]: pgmap v221: 129 pgs: 129 active+clean; 1.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 56 KiB/s wr, 33 op/s 2024-04-21T08:46:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:10 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:46:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:10 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:46:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:10 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:46:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:10 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:46:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:10 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:46:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:10 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2811523957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:10 smithi112 ceph-mon[31085]: osdmap e97: 12 total, 12 up, 12 in 2024-04-21T08:46:10.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:10 smithi043 ceph-mon[27978]: pgmap v221: 129 pgs: 129 active+clean; 1.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 56 KiB/s wr, 33 op/s 2024-04-21T08:46:10.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:10 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:46:10.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:10 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:46:10.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:10 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:46:10.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:10 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:46:10.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:10 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:46:10.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:10 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2811523957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:10.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:10 smithi043 ceph-mon[27978]: osdmap e97: 12 total, 12 up, 12 in 2024-04-21T08:46:10.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:10 smithi175 ceph-mon[31920]: pgmap v221: 129 pgs: 129 active+clean; 1.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 56 KiB/s wr, 33 op/s 2024-04-21T08:46:10.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:10 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:46:10.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:10 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:46:10.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:10 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:46:10.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:10 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:46:10.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:10 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:46:10.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:10 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2811523957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:10.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:10 smithi175 ceph-mon[31920]: osdmap e97: 12 total, 12 up, 12 in 2024-04-21T08:46:11.292 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:46:11.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:46:11.292 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:46:11.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.412+0000 7f40f08d9640 1 -- 172.21.15.43:0/645463860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40ec107260 msgr2=0x7f40ec10ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:11.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.412+0000 7f40f08d9640 1 --2- 172.21.15.43:0/645463860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40ec107260 0x7f40ec10ee30 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009900 tx=0x7f40e402bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:11.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.413+0000 7f40f08d9640 1 -- 172.21.15.43:0/645463860 shutdown_connections 2024-04-21T08:46:11.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.413+0000 7f40f08d9640 1 --2- 172.21.15.43:0/645463860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40ec107260 0x7f40ec10ee30 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.413+0000 7f40f08d9640 1 --2- 172.21.15.43:0/645463860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40ec102b40 0x7f40ec106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.413+0000 7f40f08d9640 1 --2- 172.21.15.43:0/645463860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec102190 0x7f40ec102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.413+0000 7f40f08d9640 1 -- 172.21.15.43:0/645463860 >> 172.21.15.43:0/645463860 conn(0x7f40ec0fd990 msgr2=0x7f40ec0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:11.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.413+0000 7f40f08d9640 1 -- 172.21.15.43:0/645463860 shutdown_connections 2024-04-21T08:46:11.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.413+0000 7f40f08d9640 1 -- 172.21.15.43:0/645463860 wait complete. 2024-04-21T08:46:11.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.413+0000 7f40f08d9640 1 Processor -- start 2024-04-21T08:46:11.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.413+0000 7f40f08d9640 1 -- start start 2024-04-21T08:46:11.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.414+0000 7f40f08d9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40ec102190 0x7f40ec074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.414+0000 7f40f08d9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec107260 0x7f40ec074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.414+0000 7f40f08d9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40ec074cd0 0x7f40ec1a6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.414+0000 7f40f08d9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1121b0 con 0x7f40ec074cd0 2024-04-21T08:46:11.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.414+0000 7f40f08d9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec112030 con 0x7f40ec107260 2024-04-21T08:46:11.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.414+0000 7f40f08d9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec112330 con 0x7f40ec102190 2024-04-21T08:46:11.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.414+0000 7f40ead9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40ec074cd0 0x7f40ec1a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.414+0000 7f40ead9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40ec074cd0 0x7f40ec1a6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37666/0 (socket says 172.21.15.43:37666) 2024-04-21T08:46:11.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.414+0000 7f40ead9d640 1 -- 172.21.15.43:0/739800824 learned_addr learned my addr 172.21.15.43:0/739800824 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:11.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.414+0000 7f40ea59c640 1 --2- 172.21.15.43:0/739800824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40ec102190 0x7f40ec074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.414+0000 7f40e9d9b640 1 --2- 172.21.15.43:0/739800824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec107260 0x7f40ec074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.414+0000 7f40ead9d640 1 -- 172.21.15.43:0/739800824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40ec102190 msgr2=0x7f40ec074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:11.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.414+0000 7f40ead9d640 1 --2- 172.21.15.43:0/739800824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40ec102190 0x7f40ec074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.414+0000 7f40ead9d640 1 -- 172.21.15.43:0/739800824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec107260 msgr2=0x7f40ec074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:11.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.414+0000 7f40ead9d640 1 --2- 172.21.15.43:0/739800824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec107260 0x7f40ec074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.414+0000 7f40ead9d640 1 -- 172.21.15.43:0/739800824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40ec1a6f80 con 0x7f40ec074cd0 2024-04-21T08:46:11.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.414+0000 7f40e9d9b640 1 --2- 172.21.15.43:0/739800824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec107260 0x7f40ec074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:11.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.414+0000 7f40ea59c640 1 --2- 172.21.15.43:0/739800824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40ec102190 0x7f40ec074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:11.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.414+0000 7f40ead9d640 1 --2- 172.21.15.43:0/739800824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40ec074cd0 0x7f40ec1a6880 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009230 tx=0x7f40e402c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:11.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.414+0000 7f40db7fe640 1 -- 172.21.15.43:0/739800824 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4043020 con 0x7f40ec074cd0 2024-04-21T08:46:11.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.415+0000 7f40db7fe640 1 -- 172.21.15.43:0/739800824 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40e402caf0 con 0x7f40ec074cd0 2024-04-21T08:46:11.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.415+0000 7f40db7fe640 1 -- 172.21.15.43:0/739800824 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4003990 con 0x7f40ec074cd0 2024-04-21T08:46:11.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.415+0000 7f40f08d9640 1 -- 172.21.15.43:0/739800824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40ec1a7210 con 0x7f40ec074cd0 2024-04-21T08:46:11.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.415+0000 7f40f08d9640 1 -- 172.21.15.43:0/739800824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40ec1a7710 con 0x7f40ec074cd0 2024-04-21T08:46:11.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.417+0000 7f40db7fe640 1 -- 172.21.15.43:0/739800824 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f40e4046020 con 0x7f40ec074cd0 2024-04-21T08:46:11.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.417+0000 7f40f35aa740 1 Processor -- start 2024-04-21T08:46:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.418+0000 7f40f35aa740 1 -- start start 2024-04-21T08:46:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.418+0000 7f40f35aa740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c981108e0 0x556c98110cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.418+0000 7f40f35aa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c98111290 0x556c98119840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.418+0000 7f40f35aa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c98119e10 0x556c9811c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.418+0000 7f40f35aa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556c98074f00 con 0x556c98119e10 2024-04-21T08:46:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.418+0000 7f40f35aa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556c98012590 con 0x556c98111290 2024-04-21T08:46:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.418+0000 7f40f35aa740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556c98070ec0 con 0x556c981108e0 2024-04-21T08:46:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.418+0000 7f40e9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c98119e10 0x556c9811c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.418+0000 7f40ea59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c98111290 0x556c98119840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.418+0000 7f40db7fe640 1 --2- 172.21.15.43:0/739800824 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f40d40a3d50 0x7f40d40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.418+0000 7f40e9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c98119e10 0x556c9811c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37682/0 (socket says 172.21.15.43:37682) 2024-04-21T08:46:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.418+0000 7f40ea59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c98111290 0x556c98119840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:59764/0 (socket says 172.21.15.43:59764) 2024-04-21T08:46:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.418+0000 7f40e9d9b640 1 -- 172.21.15.43:0/368409468 learned_addr learned my addr 172.21.15.43:0/368409468 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:11.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.418+0000 7f40e9d9b640 1 --2- 172.21.15.43:0/368409468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c981108e0 0x556c98110cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.418+0000 7f40ead9d640 1 --2- 172.21.15.43:0/739800824 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f40d40a3d50 0x7f40d40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.418+0000 7f40ea59c640 1 -- 172.21.15.43:0/368409468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c981108e0 msgr2=0x556c98110cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:11.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.418+0000 7f40ea59c640 1 --2- 172.21.15.43:0/368409468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c981108e0 0x556c98110cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.418+0000 7f40ea59c640 1 -- 172.21.15.43:0/368409468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c98119e10 msgr2=0x556c9811c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:11.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.418+0000 7f40ea59c640 1 --2- 172.21.15.43:0/368409468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c98119e10 0x556c9811c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.418+0000 7f40ea59c640 1 -- 172.21.15.43:0/368409468 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c97e4f7e0 con 0x556c98111290 2024-04-21T08:46:11.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.418+0000 7f40db7fe640 1 -- 172.21.15.43:0/739800824 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9583+0+0 (secure 0 0 0) 0x7f40e40f2dc0 con 0x7f40ec074cd0 2024-04-21T08:46:11.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.419+0000 7f40ea59c640 1 --2- 172.21.15.43:0/368409468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c98111290 0x556c98119840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f40dc010450 tx=0x7f40dc009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e02e2b5f6d8dde8c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:11.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.419+0000 7f40d8ff9640 1 -- 172.21.15.43:0/368409468 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc003b70 con 0x556c98111290 2024-04-21T08:46:11.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.419+0000 7f40ead9d640 1 --2- 172.21.15.43:0/739800824 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f40d40a3d50 0x7f40d40a6210 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f40e40baeb0 tx=0x7f40e40bacf0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:11.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.419+0000 7f40d8ff9640 1 -- 172.21.15.43:0/368409468 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40dc003d10 con 0x556c98111290 2024-04-21T08:46:11.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.419+0000 7f40d8ff9640 1 -- 172.21.15.43:0/368409468 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc011990 con 0x556c98111290 2024-04-21T08:46:11.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.420+0000 7f40f35aa740 1 -- 172.21.15.43:0/368409468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c98111290 msgr2=0x556c98119840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:11.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.420+0000 7f40f35aa740 1 --2- 172.21.15.43:0/368409468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c98111290 0x556c98119840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f40dc010450 tx=0x7f40dc009f90 comp rx=0 tx=0).stop 2024-04-21T08:46:11.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.420+0000 7f40f35aa740 1 -- 172.21.15.43:0/368409468 shutdown_connections 2024-04-21T08:46:11.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.420+0000 7f40f35aa740 1 --2- 172.21.15.43:0/368409468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c98119e10 0x556c9811c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.420+0000 7f40f35aa740 1 --2- 172.21.15.43:0/368409468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c98111290 0x556c98119840 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.420+0000 7f40f35aa740 1 --2- 172.21.15.43:0/368409468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c981108e0 0x556c98110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.420+0000 7f40f35aa740 1 -- 172.21.15.43:0/368409468 >> 172.21.15.43:0/368409468 conn(0x556c9810afb0 msgr2=0x556c9810b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:11.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.420+0000 7f40f35aa740 1 -- 172.21.15.43:0/368409468 shutdown_connections 2024-04-21T08:46:11.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.420+0000 7f40f35aa740 1 -- 172.21.15.43:0/368409468 wait complete. 2024-04-21T08:46:11.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.420+0000 7f40f35aa740 1 -- start start 2024-04-21T08:46:11.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.420+0000 7f40f35aa740 1 Processor -- start 2024-04-21T08:46:11.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.420+0000 7f40f35aa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c981108e0 0x556c98116da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.420+0000 7f40f35aa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c98111290 0x556c981172e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.420+0000 7f40f35aa740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c98119e10 0x556c98117820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.420+0000 7f40f35aa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556c9811cd80 con 0x556c981108e0 2024-04-21T08:46:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.420+0000 7f40f35aa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556c9802e580 con 0x556c98111290 2024-04-21T08:46:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.420+0000 7f40f35aa740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556c9811cf00 con 0x556c98119e10 2024-04-21T08:46:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.421+0000 7f40f35aa740 10 client.0 _ll_register_callbacks cb 0x556c98034650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.421+0000 7f40e9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c981108e0 0x556c98116da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.421+0000 7f40e9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c981108e0 0x556c98116da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37694/0 (socket says 172.21.15.43:37694) 2024-04-21T08:46:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.421+0000 7f40e9d9b640 1 -- 172.21.15.43:0/2554144635 learned_addr learned my addr 172.21.15.43:0/2554144635 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.421+0000 7f40ea59c640 1 --2- 172.21.15.43:0/2554144635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c98111290 0x556c981172e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.421+0000 7f40e9d9b640 1 -- 172.21.15.43:0/2554144635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c98119e10 msgr2=0x556c98117820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.421+0000 7f40e9d9b640 1 --2- 172.21.15.43:0/2554144635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c98119e10 0x556c98117820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.421+0000 7f40e9d9b640 1 -- 172.21.15.43:0/2554144635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c98111290 msgr2=0x556c981172e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.421+0000 7f40e9d9b640 1 --2- 172.21.15.43:0/2554144635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c98111290 0x556c981172e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.421+0000 7f40e9d9b640 1 -- 172.21.15.43:0/2554144635 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c97eaff70 con 0x556c981108e0 2024-04-21T08:46:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.421+0000 7f40e9d9b640 1 --2- 172.21.15.43:0/2554144635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c981108e0 0x556c98116da0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f40e000dd70 tx=0x7f40e0012f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928da7abd9c1dd97 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.421+0000 7f40d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.421+0000 7f40d1ffb640 1 -- 172.21.15.43:0/2554144635 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0005020 con 0x556c981108e0 2024-04-21T08:46:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.421+0000 7f40d1ffb640 1 -- 172.21.15.43:0/2554144635 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9583+0+0 (secure 0 0 0) 0x7f40e00051c0 con 0x556c981108e0 2024-04-21T08:46:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.421+0000 7f40f35aa740 1 -- 172.21.15.43:0/2554144635 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40dc003d10 con 0x556c981108e0 2024-04-21T08:46:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.421+0000 7f40d1ffb640 1 -- 172.21.15.43:0/2554144635 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f40e0027050 con 0x556c981108e0 2024-04-21T08:46:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.421+0000 7f40f35aa740 10 client.19041 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.422+0000 7f40f35aa740 1 -- 172.21.15.43:0/2554144635 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556c97d47170 con 0x556c981108e0 2024-04-21T08:46:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.422+0000 7f40d1ffb640 1 -- 172.21.15.43:0/2554144635 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40dc003d10 con 0x556c981108e0 2024-04-21T08:46:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.422+0000 7f40d1ffb640 1 -- 172.21.15.43:0/2554144635 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f40e001c420 con 0x556c981108e0 2024-04-21T08:46:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.422+0000 7f40f35aa740 10 client.19041 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.422+0000 7f40f35aa740 10 client.19041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.422+0000 7f40f35aa740 20 client.19041 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.422+0000 7f40f35aa740 1 --2- 172.21.15.43:0/2554144635 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556c9812a370 0x556c9812c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.422+0000 7f40f35aa740 4 client.19041 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.422+0000 7f40f35aa740 1 -- 172.21.15.43:0/2554144635 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c98082920 con 0x556c9812a370 2024-04-21T08:46:11.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.422+0000 7f40ea59c640 1 --2- 172.21.15.43:0/2554144635 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556c9812a370 0x556c9812c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.423+0000 7f40ea59c640 1 --2- 172.21.15.43:0/2554144635 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556c9812a370 0x556c9812c7c0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68769655e1372a42 server_cookie=8b35d1c1c7e700c5 in_seq=0 out_seq=0 2024-04-21T08:46:11.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.423+0000 7f40d1ffb640 10 client.19041 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:11.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.423+0000 7f40d1ffb640 1 -- 172.21.15.43:0/2554144635 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556c98082920 con 0x556c9812a370 2024-04-21T08:46:11.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.423+0000 7f40d1ffb640 10 client.19041 handle_command_reply: tid=0 2024-04-21T08:46:11.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.423+0000 7f40f35aa740 1 client.19041 shutdown 2024-04-21T08:46:11.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.423+0000 7f40f35aa740 1 -- 172.21.15.43:0/2554144635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c981108e0 msgr2=0x556c98116da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:11.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.423+0000 7f40f35aa740 1 --2- 172.21.15.43:0/2554144635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c981108e0 0x556c98116da0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f40e000dd70 tx=0x7f40e0012f20 comp rx=0 tx=0).stop 2024-04-21T08:46:11.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.424+0000 7f40f35aa740 1 -- 172.21.15.43:0/2554144635 shutdown_connections 2024-04-21T08:46:11.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.424+0000 7f40f35aa740 1 --2- 172.21.15.43:0/2554144635 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556c9812a370 0x556c9812c7c0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.424+0000 7f40f35aa740 1 --2- 172.21.15.43:0/2554144635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c98119e10 0x556c98117820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.424+0000 7f40f35aa740 1 --2- 172.21.15.43:0/2554144635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c98111290 0x556c981172e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.424+0000 7f40f35aa740 1 --2- 172.21.15.43:0/2554144635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c981108e0 0x556c98116da0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.424+0000 7f40f35aa740 1 -- 172.21.15.43:0/2554144635 >> 172.21.15.43:0/2554144635 conn(0x556c9810afb0 msgr2=0x556c9810cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:11.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.424+0000 7f40d1ffb640 0 client.19041 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:11.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.424+0000 7f40f35aa740 1 -- 172.21.15.43:0/2554144635 shutdown_connections 2024-04-21T08:46:11.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.424+0000 7f40f35aa740 1 -- 172.21.15.43:0/2554144635 wait complete. 2024-04-21T08:46:11.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.424+0000 7f40f35aa740 20 client.19041 trim_cache size 0 max 16384 2024-04-21T08:46:11.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.441+0000 7f40f35aa740 1 Processor -- start 2024-04-21T08:46:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.441+0000 7f40f35aa740 1 -- start start 2024-04-21T08:46:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.442+0000 7f40f35aa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c981108e0 0x556c981194d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.442+0000 7f40f35aa740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c98111290 0x556c98125c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.442+0000 7f40f35aa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c98119e10 0x556c98117470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.442+0000 7f40f35aa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1b1660 con 0x556c981108e0 2024-04-21T08:46:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.442+0000 7f40f35aa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1b31a0 con 0x556c98119e10 2024-04-21T08:46:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.442+0000 7f40f35aa740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1b1d30 con 0x556c98111290 2024-04-21T08:46:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.442+0000 7f40e9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c981108e0 0x556c981194d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.442+0000 7f40e9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c981108e0 0x556c981194d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37704/0 (socket says 172.21.15.43:37704) 2024-04-21T08:46:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.442+0000 7f40e9d9b640 1 -- 172.21.15.43:0/2756114947 learned_addr learned my addr 172.21.15.43:0/2756114947 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.442+0000 7f40ea59c640 1 --2- 172.21.15.43:0/2756114947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c98111290 0x556c98125c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.442+0000 7f40e9d9b640 1 -- 172.21.15.43:0/2756114947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c98111290 msgr2=0x556c98125c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.442+0000 7f40e9d9b640 1 --2- 172.21.15.43:0/2756114947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c98111290 0x556c98125c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.442+0000 7f40e9d9b640 1 -- 172.21.15.43:0/2756114947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c98119e10 msgr2=0x556c98117470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.442+0000 7f40e9d9b640 1 --2- 172.21.15.43:0/2756114947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c98119e10 0x556c98117470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.442+0000 7f40e9d9b640 1 -- 172.21.15.43:0/2756114947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c981152f0 con 0x556c981108e0 2024-04-21T08:46:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.442+0000 7f40ea59c640 1 --2- 172.21.15.43:0/2756114947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c98111290 0x556c98125c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.442+0000 7f40e9d9b640 1 --2- 172.21.15.43:0/2756114947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c981108e0 0x556c981194d0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f40e0005940 tx=0x7f40e0004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbec7524d24e99a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.442+0000 7f40d3fff640 1 -- 172.21.15.43:0/2756114947 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e002f020 con 0x556c981108e0 2024-04-21T08:46:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.442+0000 7f40d3fff640 1 -- 172.21.15.43:0/2756114947 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40e0005be0 con 0x556c981108e0 2024-04-21T08:46:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.442+0000 7f40d3fff640 1 -- 172.21.15.43:0/2756114947 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e002ba00 con 0x556c981108e0 2024-04-21T08:46:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.443+0000 7f40f35aa740 1 -- 172.21.15.43:0/2756114947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c981108e0 msgr2=0x556c981194d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.443+0000 7f40f35aa740 1 --2- 172.21.15.43:0/2756114947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c981108e0 0x556c981194d0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f40e0005940 tx=0x7f40e0004660 comp rx=0 tx=0).stop 2024-04-21T08:46:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.443+0000 7f40f35aa740 1 -- 172.21.15.43:0/2756114947 shutdown_connections 2024-04-21T08:46:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.443+0000 7f40f35aa740 1 --2- 172.21.15.43:0/2756114947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c98119e10 0x556c98117470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.443+0000 7f40f35aa740 1 --2- 172.21.15.43:0/2756114947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c98111290 0x556c98125c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.443+0000 7f40f35aa740 1 --2- 172.21.15.43:0/2756114947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c981108e0 0x556c981194d0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.443+0000 7f40f35aa740 1 -- 172.21.15.43:0/2756114947 >> 172.21.15.43:0/2756114947 conn(0x556c9810afb0 msgr2=0x556c98116d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:11.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.443+0000 7f40f35aa740 1 -- 172.21.15.43:0/2756114947 shutdown_connections 2024-04-21T08:46:11.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.443+0000 7f40f35aa740 1 -- 172.21.15.43:0/2756114947 wait complete. 2024-04-21T08:46:11.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.443+0000 7f40f35aa740 1 -- start start 2024-04-21T08:46:11.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.443+0000 7f40f35aa740 1 Processor -- start 2024-04-21T08:46:11.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.444+0000 7f40f35aa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c981108e0 0x556c98113c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.444+0000 7f40f35aa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c98111290 0x556c981141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.444+0000 7f40f35aa740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c98119e10 0x556c98114710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.444+0000 7f40f35aa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1b2ad0 con 0x556c98111290 2024-04-21T08:46:11.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.444+0000 7f40f35aa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1b2400 con 0x556c981108e0 2024-04-21T08:46:11.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.444+0000 7f40f35aa740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1af660 con 0x556c98119e10 2024-04-21T08:46:11.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.444+0000 7f40f35aa740 10 client.0 _ll_register_callbacks cb 0x7f40e00126d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:11.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.444+0000 7f40ea59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c98111290 0x556c981141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.444+0000 7f40ea59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c98111290 0x556c981141d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37716/0 (socket says 172.21.15.43:37716) 2024-04-21T08:46:11.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.444+0000 7f40ea59c640 1 -- 172.21.15.43:0/975338994 learned_addr learned my addr 172.21.15.43:0/975338994 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:11.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.444+0000 7f40e9d9b640 1 --2- 172.21.15.43:0/975338994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c98119e10 0x556c98114710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.444+0000 7f40ea59c640 1 -- 172.21.15.43:0/975338994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c98119e10 msgr2=0x556c98114710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:11.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.444+0000 7f40ea59c640 1 --2- 172.21.15.43:0/975338994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c98119e10 0x556c98114710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.444+0000 7f40ea59c640 1 -- 172.21.15.43:0/975338994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c981108e0 msgr2=0x556c98113c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:11.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.444+0000 7f40e9d9b640 1 --2- 172.21.15.43:0/975338994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c981108e0 0x556c98113c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.444+0000 7f40ea59c640 1 --2- 172.21.15.43:0/975338994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c981108e0 0x556c98113c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.444+0000 7f40ea59c640 1 -- 172.21.15.43:0/975338994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c9818fab0 con 0x556c98111290 2024-04-21T08:46:11.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.444+0000 7f40e9d9b640 1 --2- 172.21.15.43:0/975338994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c981108e0 0x556c98113c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:11.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.444+0000 7f40ea59c640 1 --2- 172.21.15.43:0/975338994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c98111290 0x556c981141d0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f40dc009f60 tx=0x7f40dc0059e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bdab5d5f84676ad server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:11.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.444+0000 7f40d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:11.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.444+0000 7f40d1ffb640 1 -- 172.21.15.43:0/975338994 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc0099c0 con 0x556c98111290 2024-04-21T08:46:11.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.444+0000 7f40d1ffb640 1 -- 172.21.15.43:0/975338994 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9583+0+0 (secure 0 0 0) 0x7f40dc01f040 con 0x556c98111290 2024-04-21T08:46:11.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.444+0000 7f40f35aa740 1 -- 172.21.15.43:0/975338994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e0005be0 con 0x556c98111290 2024-04-21T08:46:11.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.445+0000 7f40d1ffb640 1 -- 172.21.15.43:0/975338994 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f40dc037050 con 0x556c98111290 2024-04-21T08:46:11.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.445+0000 7f40f35aa740 10 client.19047 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:11.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.445+0000 7f40f35aa740 1 -- 172.21.15.43:0/975338994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556c98190d70 con 0x556c98111290 2024-04-21T08:46:11.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.445+0000 7f40d1ffb640 1 -- 172.21.15.43:0/975338994 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40e0005be0 con 0x556c98111290 2024-04-21T08:46:11.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.445+0000 7f40d1ffb640 1 -- 172.21.15.43:0/975338994 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f40dc0118a0 con 0x556c98111290 2024-04-21T08:46:11.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.445+0000 7f40f35aa740 10 client.19047 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:11.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.445+0000 7f40f35aa740 10 client.19047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:11.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.445+0000 7f40f35aa740 20 client.19047 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:11.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.445+0000 7f40f35aa740 1 --2- 172.21.15.43:0/975338994 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556c98191430 0x556c98193850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.445+0000 7f40f35aa740 4 client.19047 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:46:11.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.445+0000 7f40f35aa740 1 -- 172.21.15.43:0/975338994 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556c9807c790 con 0x556c98191430 2024-04-21T08:46:11.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.445+0000 7f40e9d9b640 1 --2- 172.21.15.43:0/975338994 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556c98191430 0x556c98193850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.446+0000 7f40e9d9b640 1 --2- 172.21.15.43:0/975338994 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556c98191430 0x556c98193850 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5874dd71df1aba49 server_cookie=6f2802a69aedb1e4 in_seq=0 out_seq=0 2024-04-21T08:46:11.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.446+0000 7f40d1ffb640 10 client.19047 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:11.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.447+0000 7f40d1ffb640 1 -- 172.21.15.43:0/975338994 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556c9807c790 con 0x556c98191430 2024-04-21T08:46:11.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.447+0000 7f40d1ffb640 10 client.19047 handle_command_reply: tid=0 2024-04-21T08:46:11.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.447+0000 7f40f35aa740 1 client.19047 shutdown 2024-04-21T08:46:11.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.447+0000 7f40f35aa740 1 -- 172.21.15.43:0/975338994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c98111290 msgr2=0x556c981141d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:11.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.447+0000 7f40f35aa740 1 --2- 172.21.15.43:0/975338994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c98111290 0x556c981141d0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f40dc009f60 tx=0x7f40dc0059e0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.447+0000 7f40f35aa740 1 -- 172.21.15.43:0/975338994 shutdown_connections 2024-04-21T08:46:11.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.447+0000 7f40f35aa740 1 --2- 172.21.15.43:0/975338994 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556c98191430 0x556c98193850 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.447+0000 7f40f35aa740 1 --2- 172.21.15.43:0/975338994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556c98119e10 0x556c98114710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.447+0000 7f40f35aa740 1 --2- 172.21.15.43:0/975338994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c98111290 0x556c981141d0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.447+0000 7f40f35aa740 1 --2- 172.21.15.43:0/975338994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556c981108e0 0x556c98113c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.447+0000 7f40d1ffb640 0 client.19047 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:11.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.447+0000 7f40f35aa740 1 -- 172.21.15.43:0/975338994 >> 172.21.15.43:0/975338994 conn(0x556c9810afb0 msgr2=0x556c9810cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:11.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.447+0000 7f40f35aa740 1 -- 172.21.15.43:0/975338994 shutdown_connections 2024-04-21T08:46:11.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.447+0000 7f40f35aa740 1 -- 172.21.15.43:0/975338994 wait complete. 2024-04-21T08:46:11.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.447+0000 7f40f35aa740 20 client.19047 trim_cache size 0 max 16384 2024-04-21T08:46:11.453 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:46:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:46:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "83e09990-d6f6-4cb0-b537-e596fe6d9bc5", 2024-04-21T08:46:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:46:11.453 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:46:11.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.448+0000 7f40f08d9640 1 -- 172.21.15.43:0/739800824 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f40d40a3d50 msgr2=0x7f40d40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:11.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.448+0000 7f40f08d9640 1 --2- 172.21.15.43:0/739800824 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f40d40a3d50 0x7f40d40a6210 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f40e40baeb0 tx=0x7f40e40bacf0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.448+0000 7f40f08d9640 1 -- 172.21.15.43:0/739800824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40ec074cd0 msgr2=0x7f40ec1a6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:11.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.448+0000 7f40f08d9640 1 --2- 172.21.15.43:0/739800824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40ec074cd0 0x7f40ec1a6880 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009230 tx=0x7f40e402c710 comp rx=0 tx=0).stop 2024-04-21T08:46:11.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.448+0000 7f40f08d9640 1 -- 172.21.15.43:0/739800824 shutdown_connections 2024-04-21T08:46:11.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.448+0000 7f40f08d9640 1 --2- 172.21.15.43:0/739800824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40ec074cd0 0x7f40ec1a6880 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.448+0000 7f40f08d9640 1 --2- 172.21.15.43:0/739800824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec107260 0x7f40ec074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.448+0000 7f40f08d9640 1 --2- 172.21.15.43:0/739800824 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f40d40a3d50 0x7f40d40a6210 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.448+0000 7f40f08d9640 1 --2- 172.21.15.43:0/739800824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40ec102190 0x7f40ec074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.448+0000 7f40f08d9640 1 -- 172.21.15.43:0/739800824 >> 172.21.15.43:0/739800824 conn(0x7f40ec0fd990 msgr2=0x7f40ec0fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:11.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.448+0000 7f40f08d9640 1 -- 172.21.15.43:0/739800824 shutdown_connections 2024-04-21T08:46:11.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.448+0000 7f40f08d9640 1 -- 172.21.15.43:0/739800824 wait complete. 2024-04-21T08:46:11.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:46:11.458 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:46:11.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.564+0000 7f9220e9a640 1 -- 172.21.15.43:0/1084616067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f921c108060 msgr2=0x7f921c108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:11.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.564+0000 7f9220e9a640 1 --2- 172.21.15.43:0/1084616067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f921c108060 0x7f921c108440 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f920c009930 tx=0x7f920c02bc20 comp rx=0 tx=0).stop 2024-04-21T08:46:11.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.564+0000 7f9220e9a640 1 -- 172.21.15.43:0/1084616067 shutdown_connections 2024-04-21T08:46:11.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.564+0000 7f9220e9a640 1 --2- 172.21.15.43:0/1084616067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f921c06af70 0x7f921c10ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.564+0000 7f9220e9a640 1 --2- 172.21.15.43:0/1084616067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f921c06a5b0 0x7f921c06aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.564+0000 7f9220e9a640 1 --2- 172.21.15.43:0/1084616067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f921c108060 0x7f921c108440 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.564+0000 7f9220e9a640 1 -- 172.21.15.43:0/1084616067 >> 172.21.15.43:0/1084616067 conn(0x7f921c077c40 msgr2=0x7f921c078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:11.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.564+0000 7f9220e9a640 1 -- 172.21.15.43:0/1084616067 shutdown_connections 2024-04-21T08:46:11.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.564+0000 7f9220e9a640 1 -- 172.21.15.43:0/1084616067 wait complete. 2024-04-21T08:46:11.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.564+0000 7f9220e9a640 1 Processor -- start 2024-04-21T08:46:11.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.565+0000 7f9220e9a640 1 -- start start 2024-04-21T08:46:11.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.565+0000 7f9220e9a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f921c06a5b0 0x7f921c101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.565+0000 7f9220e9a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f921c06af70 0x7f921c1016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.565+0000 7f9220e9a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f921c108060 0x7f921c101bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.565+0000 7f9220e9a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f921c112180 con 0x7f921c06af70 2024-04-21T08:46:11.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.565+0000 7f9220e9a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f921c112000 con 0x7f921c06a5b0 2024-04-21T08:46:11.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.565+0000 7f9220e9a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f921c112300 con 0x7f921c108060 2024-04-21T08:46:11.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.565+0000 7f9213fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f921c06af70 0x7f921c1016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.565+0000 7f9213fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f921c06af70 0x7f921c1016b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37722/0 (socket says 172.21.15.43:37722) 2024-04-21T08:46:11.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.565+0000 7f9213fff640 1 -- 172.21.15.43:0/2544704218 learned_addr learned my addr 172.21.15.43:0/2544704218 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:11.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.565+0000 7f921ad9d640 1 --2- 172.21.15.43:0/2544704218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f921c108060 0x7f921c101bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.565+0000 7f921a59c640 1 --2- 172.21.15.43:0/2544704218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f921c06a5b0 0x7f921c101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.565+0000 7f9213fff640 1 -- 172.21.15.43:0/2544704218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f921c108060 msgr2=0x7f921c101bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:11.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.565+0000 7f9213fff640 1 --2- 172.21.15.43:0/2544704218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f921c108060 0x7f921c101bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.565+0000 7f9213fff640 1 -- 172.21.15.43:0/2544704218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f921c06a5b0 msgr2=0x7f921c101170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:11.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.565+0000 7f9213fff640 1 --2- 172.21.15.43:0/2544704218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f921c06a5b0 0x7f921c101170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.565+0000 7f9213fff640 1 -- 172.21.15.43:0/2544704218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f921c1a6b20 con 0x7f921c06af70 2024-04-21T08:46:11.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.565+0000 7f921a59c640 1 --2- 172.21.15.43:0/2544704218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f921c06a5b0 0x7f921c101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:11.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.565+0000 7f9213fff640 1 --2- 172.21.15.43:0/2544704218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f921c06af70 0x7f921c1016b0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f920400c920 tx=0x7f920400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:11.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.565+0000 7f92137fe640 1 -- 172.21.15.43:0/2544704218 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9204007c70 con 0x7f921c06af70 2024-04-21T08:46:11.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.566+0000 7f9220e9a640 1 -- 172.21.15.43:0/2544704218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f921c1a6e10 con 0x7f921c06af70 2024-04-21T08:46:11.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.566+0000 7f92137fe640 1 -- 172.21.15.43:0/2544704218 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9204007e10 con 0x7f921c06af70 2024-04-21T08:46:11.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.566+0000 7f92137fe640 1 -- 172.21.15.43:0/2544704218 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9204018820 con 0x7f921c06af70 2024-04-21T08:46:11.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.566+0000 7f9220e9a640 1 -- 172.21.15.43:0/2544704218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f921c1a72d0 con 0x7f921c06af70 2024-04-21T08:46:11.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.568+0000 7f92137fe640 1 -- 172.21.15.43:0/2544704218 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f920400f450 con 0x7f921c06af70 2024-04-21T08:46:11.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.568+0000 7f9223b6b740 1 Processor -- start 2024-04-21T08:46:11.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.568+0000 7f9223b6b740 1 -- start start 2024-04-21T08:46:11.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.568+0000 7f9223b6b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757681d580 0x55757681d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.568+0000 7f9223b6b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55757681dea0 0x557576826540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.568+0000 7f9223b6b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557576826a80 0x557576828e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.568+0000 7f9223b6b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5575767821e0 con 0x55757681dea0 2024-04-21T08:46:11.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.568+0000 7f9223b6b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55757671f390 con 0x55757681d580 2024-04-21T08:46:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.568+0000 7f9223b6b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55757677e1a0 con 0x557576826a80 2024-04-21T08:46:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.569+0000 7f921a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55757681dea0 0x557576826540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.569+0000 7f921a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55757681dea0 0x557576826540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37730/0 (socket says 172.21.15.43:37730) 2024-04-21T08:46:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.569+0000 7f921a59c640 1 -- 172.21.15.43:0/2059390543 learned_addr learned my addr 172.21.15.43:0/2059390543 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.569+0000 7f92137fe640 1 --2- 172.21.15.43:0/2544704218 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f92080a3d20 0x7f92080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.569+0000 7f921ad9d640 1 --2- 172.21.15.43:0/2059390543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757681d580 0x55757681d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.569+0000 7f9213fff640 1 --2- 172.21.15.43:0/2059390543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557576826a80 0x557576828e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.569+0000 7f921a59c640 1 -- 172.21.15.43:0/2059390543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557576826a80 msgr2=0x557576828e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.569+0000 7f921ad9d640 1 --2- 172.21.15.43:0/2544704218 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f92080a3d20 0x7f92080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.569+0000 7f921a59c640 1 --2- 172.21.15.43:0/2059390543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557576826a80 0x557576828e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.569+0000 7f92137fe640 1 -- 172.21.15.43:0/2544704218 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9583+0+0 (secure 0 0 0) 0x7f92040d0d40 con 0x7f921c06af70 2024-04-21T08:46:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.569+0000 7f921a59c640 1 -- 172.21.15.43:0/2059390543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757681d580 msgr2=0x55757681d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.569+0000 7f921a59c640 1 --2- 172.21.15.43:0/2059390543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757681d580 0x55757681d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.569+0000 7f921a59c640 1 -- 172.21.15.43:0/2059390543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55757655e7d0 con 0x55757681dea0 2024-04-21T08:46:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.569+0000 7f9213fff640 1 --2- 172.21.15.43:0/2059390543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557576826a80 0x557576828e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:11.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.569+0000 7f921a59c640 1 --2- 172.21.15.43:0/2059390543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55757681dea0 0x557576826540 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f920c036040 tx=0x7f920c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0e209774c5c375 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:11.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.569+0000 7f921ad9d640 1 --2- 172.21.15.43:0/2544704218 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f92080a3d20 0x7f92080a61e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f92140077f0 tx=0x7f9214007520 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:11.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.569+0000 7f9210ff9640 1 -- 172.21.15.43:0/2059390543 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c044020 con 0x55757681dea0 2024-04-21T08:46:11.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.569+0000 7f9210ff9640 1 -- 172.21.15.43:0/2059390543 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f920c02cc80 con 0x55757681dea0 2024-04-21T08:46:11.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.569+0000 7f9210ff9640 1 -- 172.21.15.43:0/2059390543 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c004920 con 0x55757681dea0 2024-04-21T08:46:11.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.570+0000 7f9223b6b740 1 -- 172.21.15.43:0/2059390543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55757681dea0 msgr2=0x557576826540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:11.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.570+0000 7f9223b6b740 1 --2- 172.21.15.43:0/2059390543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55757681dea0 0x557576826540 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f920c036040 tx=0x7f920c004040 comp rx=0 tx=0).stop 2024-04-21T08:46:11.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.570+0000 7f9223b6b740 1 -- 172.21.15.43:0/2059390543 shutdown_connections 2024-04-21T08:46:11.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.570+0000 7f9223b6b740 1 --2- 172.21.15.43:0/2059390543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557576826a80 0x557576828e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.570+0000 7f9223b6b740 1 --2- 172.21.15.43:0/2059390543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55757681dea0 0x557576826540 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.570+0000 7f9223b6b740 1 --2- 172.21.15.43:0/2059390543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757681d580 0x55757681d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.570+0000 7f9223b6b740 1 -- 172.21.15.43:0/2059390543 >> 172.21.15.43:0/2059390543 conn(0x557576817cb0 msgr2=0x557576818090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:11.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.570+0000 7f9223b6b740 1 -- 172.21.15.43:0/2059390543 shutdown_connections 2024-04-21T08:46:11.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.570+0000 7f9223b6b740 1 -- 172.21.15.43:0/2059390543 wait complete. 2024-04-21T08:46:11.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.571+0000 7f9223b6b740 1 -- start start 2024-04-21T08:46:11.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.571+0000 7f9223b6b740 1 Processor -- start 2024-04-21T08:46:11.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.571+0000 7f9223b6b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55757681d580 0x557576823ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.571+0000 7f9223b6b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55757681dea0 0x557576824020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.571+0000 7f9223b6b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557576826a80 0x557576824560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.571+0000 7f9223b6b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557576829810 con 0x55757681d580 2024-04-21T08:46:11.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.571+0000 7f9223b6b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55757673b2f0 con 0x557576826a80 2024-04-21T08:46:11.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.571+0000 7f9223b6b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557576829990 con 0x55757681dea0 2024-04-21T08:46:11.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.571+0000 7f921a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55757681d580 0x557576823ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.571+0000 7f9223b6b740 10 client.0 _ll_register_callbacks cb 0x557576748580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:11.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.571+0000 7f921a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55757681d580 0x557576823ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37740/0 (socket says 172.21.15.43:37740) 2024-04-21T08:46:11.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.571+0000 7f921a59c640 1 -- 172.21.15.43:0/2604610358 learned_addr learned my addr 172.21.15.43:0/2604610358 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:11.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.571+0000 7f921ad9d640 1 --2- 172.21.15.43:0/2604610358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557576826a80 0x557576824560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.571+0000 7f921a59c640 1 -- 172.21.15.43:0/2604610358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55757681dea0 msgr2=0x557576824020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:11.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.571+0000 7f9213fff640 1 --2- 172.21.15.43:0/2604610358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55757681dea0 0x557576824020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.571+0000 7f921a59c640 1 --2- 172.21.15.43:0/2604610358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55757681dea0 0x557576824020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.571+0000 7f921a59c640 1 -- 172.21.15.43:0/2604610358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557576826a80 msgr2=0x557576824560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:11.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.571+0000 7f921a59c640 1 --2- 172.21.15.43:0/2604610358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557576826a80 0x557576824560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.571+0000 7f921a59c640 1 -- 172.21.15.43:0/2604610358 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575764943f0 con 0x55757681d580 2024-04-21T08:46:11.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.571+0000 7f921ad9d640 1 --2- 172.21.15.43:0/2604610358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557576826a80 0x557576824560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:11.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.572+0000 7f921a59c640 1 --2- 172.21.15.43:0/2604610358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55757681d580 0x557576823ae0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f920c000c00 tx=0x7f920c02ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4836d2ccef97ab server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:11.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.572+0000 7f9201ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:11.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.572+0000 7f9201ffb640 1 -- 172.21.15.43:0/2604610358 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c04d070 con 0x55757681d580 2024-04-21T08:46:11.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.572+0000 7f9201ffb640 1 -- 172.21.15.43:0/2604610358 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9583+0+0 (secure 0 0 0) 0x7f920c003a00 con 0x55757681d580 2024-04-21T08:46:11.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.572+0000 7f9223b6b740 1 -- 172.21.15.43:0/2604610358 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f920c02cc80 con 0x55757681d580 2024-04-21T08:46:11.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.572+0000 7f9201ffb640 1 -- 172.21.15.43:0/2604610358 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f920c058050 con 0x55757681d580 2024-04-21T08:46:11.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.572+0000 7f9223b6b740 10 client.19059 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:11.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.572+0000 7f9223b6b740 1 -- 172.21.15.43:0/2604610358 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5575764540f0 con 0x55757681d580 2024-04-21T08:46:11.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.572+0000 7f9201ffb640 1 -- 172.21.15.43:0/2604610358 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f920c02cc80 con 0x55757681d580 2024-04-21T08:46:11.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.572+0000 7f9201ffb640 1 -- 172.21.15.43:0/2604610358 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f920c049450 con 0x55757681d580 2024-04-21T08:46:11.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.572+0000 7f9223b6b740 10 client.19059 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:11.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.572+0000 7f9223b6b740 10 client.19059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:11.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.572+0000 7f9223b6b740 20 client.19059 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:11.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.572+0000 7f9223b6b740 1 --2- 172.21.15.43:0/2604610358 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5575768368a0 0x557576838c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.572+0000 7f9223b6b740 4 client.19059 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:11.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.572+0000 7f9223b6b740 1 -- 172.21.15.43:0/2604610358 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55757678fc00 con 0x5575768368a0 2024-04-21T08:46:11.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.573+0000 7f9213fff640 1 --2- 172.21.15.43:0/2604610358 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5575768368a0 0x557576838c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.573+0000 7f9201ffb640 10 client.19059 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:11.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.573+0000 7f9213fff640 1 --2- 172.21.15.43:0/2604610358 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5575768368a0 0x557576838c90 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce011ef29f2c6d65 server_cookie=7580508b205029f3 in_seq=0 out_seq=0 2024-04-21T08:46:11.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.574+0000 7f9201ffb640 1 -- 172.21.15.43:0/2604610358 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55757678fc00 con 0x5575768368a0 2024-04-21T08:46:11.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.574+0000 7f9201ffb640 10 client.19059 handle_command_reply: tid=0 2024-04-21T08:46:11.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.574+0000 7f9223b6b740 1 client.19059 shutdown 2024-04-21T08:46:11.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.574+0000 7f9223b6b740 1 -- 172.21.15.43:0/2604610358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55757681d580 msgr2=0x557576823ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:11.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.574+0000 7f9223b6b740 1 --2- 172.21.15.43:0/2604610358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55757681d580 0x557576823ae0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f920c000c00 tx=0x7f920c02ca00 comp rx=0 tx=0).stop 2024-04-21T08:46:11.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.574+0000 7f9223b6b740 1 -- 172.21.15.43:0/2604610358 shutdown_connections 2024-04-21T08:46:11.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.574+0000 7f9223b6b740 1 --2- 172.21.15.43:0/2604610358 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5575768368a0 0x557576838c90 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.574+0000 7f9223b6b740 1 --2- 172.21.15.43:0/2604610358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557576826a80 0x557576824560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.574+0000 7f9201ffb640 0 client.19059 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:11.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.574+0000 7f9223b6b740 1 --2- 172.21.15.43:0/2604610358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55757681dea0 0x557576824020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.574+0000 7f9223b6b740 1 --2- 172.21.15.43:0/2604610358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55757681d580 0x557576823ae0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.574+0000 7f9223b6b740 1 -- 172.21.15.43:0/2604610358 >> 172.21.15.43:0/2604610358 conn(0x557576817cb0 msgr2=0x557576819a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:11.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.574+0000 7f9223b6b740 1 -- 172.21.15.43:0/2604610358 shutdown_connections 2024-04-21T08:46:11.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.574+0000 7f9223b6b740 1 -- 172.21.15.43:0/2604610358 wait complete. 2024-04-21T08:46:11.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.574+0000 7f9223b6b740 20 client.19059 trim_cache size 0 max 16384 2024-04-21T08:46:11.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.592+0000 7f9223b6b740 1 Processor -- start 2024-04-21T08:46:11.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.592+0000 7f9223b6b740 1 -- start start 2024-04-21T08:46:11.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.592+0000 7f9223b6b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55757681d580 0x557576826210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.592+0000 7f9223b6b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757681dea0 0x557576832730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.592+0000 7f9223b6b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557576826a80 0x557576822c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.592+0000 7f9223b6b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f921c1b15e0 con 0x557576826a80 2024-04-21T08:46:11.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.592+0000 7f9223b6b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f921c1b3120 con 0x55757681dea0 2024-04-21T08:46:11.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.592+0000 7f9223b6b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f921c1b1cb0 con 0x55757681d580 2024-04-21T08:46:11.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.592+0000 7f921ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557576826a80 0x557576822c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.592+0000 7f921ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557576826a80 0x557576822c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37754/0 (socket says 172.21.15.43:37754) 2024-04-21T08:46:11.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.592+0000 7f921ad9d640 1 -- 172.21.15.43:0/939421662 learned_addr learned my addr 172.21.15.43:0/939421662 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:11.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.592+0000 7f9213fff640 1 --2- 172.21.15.43:0/939421662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757681dea0 0x557576832730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.592+0000 7f921a59c640 1 --2- 172.21.15.43:0/939421662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55757681d580 0x557576826210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.592+0000 7f921ad9d640 1 -- 172.21.15.43:0/939421662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55757681d580 msgr2=0x557576826210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:11.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.592+0000 7f921ad9d640 1 --2- 172.21.15.43:0/939421662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55757681d580 0x557576826210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.592+0000 7f921ad9d640 1 -- 172.21.15.43:0/939421662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757681dea0 msgr2=0x557576832730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:11.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.592+0000 7f921ad9d640 1 --2- 172.21.15.43:0/939421662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757681dea0 0x557576832730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.592+0000 7f921ad9d640 1 -- 172.21.15.43:0/939421662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557576828e70 con 0x557576826a80 2024-04-21T08:46:11.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.592+0000 7f9213fff640 1 --2- 172.21.15.43:0/939421662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757681dea0 0x557576832730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:11.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.592+0000 7f921ad9d640 1 --2- 172.21.15.43:0/939421662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557576826a80 0x557576822c40 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f9214007390 tx=0x7f92140078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824e1ff124efdfde server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:11.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.593+0000 7f9203fff640 1 -- 172.21.15.43:0/939421662 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9214003cb0 con 0x557576826a80 2024-04-21T08:46:11.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.593+0000 7f9203fff640 1 -- 172.21.15.43:0/939421662 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f921401a030 con 0x557576826a80 2024-04-21T08:46:11.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.593+0000 7f9203fff640 1 -- 172.21.15.43:0/939421662 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9214005020 con 0x557576826a80 2024-04-21T08:46:11.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.593+0000 7f9223b6b740 1 -- 172.21.15.43:0/939421662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557576826a80 msgr2=0x557576822c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:11.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.593+0000 7f9223b6b740 1 --2- 172.21.15.43:0/939421662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557576826a80 0x557576822c40 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f9214007390 tx=0x7f92140078c0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.593+0000 7f9223b6b740 1 -- 172.21.15.43:0/939421662 shutdown_connections 2024-04-21T08:46:11.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.593+0000 7f9223b6b740 1 --2- 172.21.15.43:0/939421662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557576826a80 0x557576822c40 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.593+0000 7f9223b6b740 1 --2- 172.21.15.43:0/939421662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757681dea0 0x557576832730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.593+0000 7f9223b6b740 1 --2- 172.21.15.43:0/939421662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55757681d580 0x557576826210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.593+0000 7f9223b6b740 1 -- 172.21.15.43:0/939421662 >> 172.21.15.43:0/939421662 conn(0x557576817cb0 msgr2=0x5575768164e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:11.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.593+0000 7f9223b6b740 1 -- 172.21.15.43:0/939421662 shutdown_connections 2024-04-21T08:46:11.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.593+0000 7f9223b6b740 1 -- 172.21.15.43:0/939421662 wait complete. 2024-04-21T08:46:11.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.594+0000 7f9223b6b740 1 -- start start 2024-04-21T08:46:11.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.594+0000 7f9223b6b740 1 Processor -- start 2024-04-21T08:46:11.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.594+0000 7f9223b6b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55757681d580 0x55757682f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.594+0000 7f9223b6b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757681dea0 0x55757682a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.594+0000 7f9223b6b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557576826a80 0x55757682aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.594+0000 7f9223b6b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f921c1b2a50 con 0x55757681d580 2024-04-21T08:46:11.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.594+0000 7f9223b6b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f921c1b2380 con 0x55757681dea0 2024-04-21T08:46:11.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.594+0000 7f9223b6b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f921c1af5e0 con 0x557576826a80 2024-04-21T08:46:11.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.594+0000 7f921a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55757681d580 0x55757682f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.594+0000 7f9223b6b740 10 client.0 _ll_register_callbacks cb 0x7f920c02c650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:11.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.594+0000 7f921a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55757681d580 0x55757682f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37766/0 (socket says 172.21.15.43:37766) 2024-04-21T08:46:11.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.594+0000 7f921a59c640 1 -- 172.21.15.43:0/603110184 learned_addr learned my addr 172.21.15.43:0/603110184 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:11.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.594+0000 7f9213fff640 1 --2- 172.21.15.43:0/603110184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757681dea0 0x55757682a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.594+0000 7f921a59c640 1 -- 172.21.15.43:0/603110184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557576826a80 msgr2=0x55757682aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:11.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.594+0000 7f921a59c640 1 --2- 172.21.15.43:0/603110184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557576826a80 0x55757682aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.594+0000 7f921a59c640 1 -- 172.21.15.43:0/603110184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757681dea0 msgr2=0x55757682a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:11.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.594+0000 7f921a59c640 1 --2- 172.21.15.43:0/603110184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757681dea0 0x55757682a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.594+0000 7f921a59c640 1 -- 172.21.15.43:0/603110184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55757682b290 con 0x55757681d580 2024-04-21T08:46:11.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.594+0000 7f9213fff640 1 --2- 172.21.15.43:0/603110184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757681dea0 0x55757682a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:11.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.595+0000 7f921a59c640 1 --2- 172.21.15.43:0/603110184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55757681d580 0x55757682f110 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f920c02c310 tx=0x7f920c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7692bce75d16d34a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:11.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.595+0000 7f9201ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:11.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.595+0000 7f9201ffb640 1 -- 172.21.15.43:0/603110184 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c055070 con 0x55757681d580 2024-04-21T08:46:11.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.595+0000 7f9201ffb640 1 -- 172.21.15.43:0/603110184 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9583+0+0 (secure 0 0 0) 0x7f920c049c90 con 0x55757681d580 2024-04-21T08:46:11.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.595+0000 7f9223b6b740 1 -- 172.21.15.43:0/603110184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f921401a030 con 0x55757681d580 2024-04-21T08:46:11.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.595+0000 7f9201ffb640 1 -- 172.21.15.43:0/603110184 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f920c04d040 con 0x55757681d580 2024-04-21T08:46:11.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.595+0000 7f9223b6b740 10 client.19065 fetch_fsmap learned FSMap version 28 2024-04-21T08:46:11.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.595+0000 7f9223b6b740 1 -- 172.21.15.43:0/603110184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55757689cc40 con 0x55757681d580 2024-04-21T08:46:11.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.595+0000 7f9201ffb640 1 -- 172.21.15.43:0/603110184 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f921401a030 con 0x55757681d580 2024-04-21T08:46:11.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.595+0000 7f9201ffb640 1 -- 172.21.15.43:0/603110184 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f920c040590 con 0x55757681d580 2024-04-21T08:46:11.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.595+0000 7f9223b6b740 10 client.19065 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:46:11.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.595+0000 7f9223b6b740 10 client.19065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:11.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.595+0000 7f9223b6b740 20 client.19065 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:11.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.595+0000 7f9223b6b740 1 --2- 172.21.15.43:0/603110184 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5575768368a0 0x55757689f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:11.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.595+0000 7f9223b6b740 4 client.19065 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:46:11.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.595+0000 7f9223b6b740 1 -- 172.21.15.43:0/603110184 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557576789a70 con 0x5575768368a0 2024-04-21T08:46:11.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.596+0000 7f9213fff640 1 --2- 172.21.15.43:0/603110184 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5575768368a0 0x55757689f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:11.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.596+0000 7f9201ffb640 10 client.19065 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:11.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.596+0000 7f9213fff640 1 --2- 172.21.15.43:0/603110184 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5575768368a0 0x55757689f2c0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae567cca26de3068 server_cookie=cdd2b274503c96ca in_seq=0 out_seq=0 2024-04-21T08:46:11.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.596+0000 7f9201ffb640 1 -- 172.21.15.43:0/603110184 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557576789a70 con 0x5575768368a0 2024-04-21T08:46:11.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.596+0000 7f9201ffb640 10 client.19065 handle_command_reply: tid=0 2024-04-21T08:46:11.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.596+0000 7f9223b6b740 1 client.19065 shutdown 2024-04-21T08:46:11.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.596+0000 7f9223b6b740 1 -- 172.21.15.43:0/603110184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55757681d580 msgr2=0x55757682f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:11.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.596+0000 7f9223b6b740 1 --2- 172.21.15.43:0/603110184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55757681d580 0x55757682f110 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f920c02c310 tx=0x7f920c004610 comp rx=0 tx=0).stop 2024-04-21T08:46:11.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.596+0000 7f9223b6b740 1 -- 172.21.15.43:0/603110184 shutdown_connections 2024-04-21T08:46:11.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.596+0000 7f9223b6b740 1 --2- 172.21.15.43:0/603110184 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5575768368a0 0x55757689f2c0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.596+0000 7f9223b6b740 1 --2- 172.21.15.43:0/603110184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557576826a80 0x55757682aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.596+0000 7f9223b6b740 1 --2- 172.21.15.43:0/603110184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757681dea0 0x55757682a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.596+0000 7f9201ffb640 0 client.19065 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:11.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.596+0000 7f9223b6b740 1 --2- 172.21.15.43:0/603110184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55757681d580 0x55757682f110 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.596+0000 7f9223b6b740 1 -- 172.21.15.43:0/603110184 >> 172.21.15.43:0/603110184 conn(0x557576817cb0 msgr2=0x5575768164e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:11.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.596+0000 7f9223b6b740 1 -- 172.21.15.43:0/603110184 shutdown_connections 2024-04-21T08:46:11.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.596+0000 7f9223b6b740 1 -- 172.21.15.43:0/603110184 wait complete. 2024-04-21T08:46:11.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.597+0000 7f9223b6b740 20 client.19065 trim_cache size 0 max 16384 2024-04-21T08:46:11.603 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:46:11.603 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:46:11.603 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-21T08:46:11.603 INFO:teuthology.orchestra.run.smithi043.stdout: "83e09990-d6f6-4cb0-b537-e596fe6d9bc5": { 2024-04-21T08:46:11.603 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-21T08:46:11.603 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "83e09990-d6f6-4cb0-b537-e596fe6d9bc5", 2024-04-21T08:46:11.603 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-21T08:46:11.603 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:46:11.603 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:46:11.603 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:46:11.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.597+0000 7f9220e9a640 1 -- 172.21.15.43:0/2544704218 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f92080a3d20 msgr2=0x7f92080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:11.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.597+0000 7f9220e9a640 1 --2- 172.21.15.43:0/2544704218 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f92080a3d20 0x7f92080a61e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f92140077f0 tx=0x7f9214007520 comp rx=0 tx=0).stop 2024-04-21T08:46:11.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.597+0000 7f9220e9a640 1 -- 172.21.15.43:0/2544704218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f921c06af70 msgr2=0x7f921c1016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:11.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.597+0000 7f9220e9a640 1 --2- 172.21.15.43:0/2544704218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f921c06af70 0x7f921c1016b0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f920400c920 tx=0x7f920400cde0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.597+0000 7f9220e9a640 1 -- 172.21.15.43:0/2544704218 shutdown_connections 2024-04-21T08:46:11.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.597+0000 7f9220e9a640 1 --2- 172.21.15.43:0/2544704218 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f92080a3d20 0x7f92080a61e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.597+0000 7f9220e9a640 1 --2- 172.21.15.43:0/2544704218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f921c108060 0x7f921c101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.597+0000 7f9220e9a640 1 --2- 172.21.15.43:0/2544704218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f921c06af70 0x7f921c1016b0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.597+0000 7f9220e9a640 1 --2- 172.21.15.43:0/2544704218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f921c06a5b0 0x7f921c101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:11.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.597+0000 7f9220e9a640 1 -- 172.21.15.43:0/2544704218 >> 172.21.15.43:0/2544704218 conn(0x7f921c077c40 msgr2=0x7f921c0ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:11.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.597+0000 7f9220e9a640 1 -- 172.21.15.43:0/2544704218 shutdown_connections 2024-04-21T08:46:11.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:11.597+0000 7f9220e9a640 1 -- 172.21.15.43:0/2544704218 wait complete. 2024-04-21T08:46:11.607 INFO:tasks.cephfs.filesystem:scrub status for tag:83e09990-d6f6-4cb0-b537-e596fe6d9bc5 - {'path': '/', 'tag': '83e09990-d6f6-4cb0-b537-e596fe6d9bc5', 'options': 'recursive,force'} 2024-04-21T08:46:12.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:12 smithi112 ceph-mon[31085]: pgmap v223: 129 pgs: 129 active+clean; 1.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.4 KiB/s wr, 16 op/s 2024-04-21T08:46:12.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:12 smithi043 ceph-mon[27978]: pgmap v223: 129 pgs: 129 active+clean; 1.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.4 KiB/s wr, 16 op/s 2024-04-21T08:46:12.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:12 smithi175 ceph-mon[31920]: pgmap v223: 129 pgs: 129 active+clean; 1.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.4 KiB/s wr, 16 op/s 2024-04-21T08:46:13.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:13 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:46:13.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:13 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:46:13.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:13 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:46:13.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:13 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:46:13.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:13 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:46:13.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:13 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:46:13.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:13 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:46:13.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:13 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:46:13.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:13 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:46:13.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:13 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:46:13.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:13 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:46:13.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:13 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:46:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:14 smithi112 ceph-mon[31085]: pgmap v224: 129 pgs: 129 active+clean; 1.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.1 KiB/s wr, 15 op/s 2024-04-21T08:46:14.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:14 smithi043 ceph-mon[27978]: pgmap v224: 129 pgs: 129 active+clean; 1.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.1 KiB/s wr, 15 op/s 2024-04-21T08:46:14.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:14 smithi175 ceph-mon[31920]: pgmap v224: 129 pgs: 129 active+clean; 1.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.1 KiB/s wr, 15 op/s 2024-04-21T08:46:15.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:15 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:46:15.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:15 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:46:15.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:15 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:46:16.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:16 smithi112 ceph-mon[31085]: pgmap v225: 129 pgs: 129 active+clean; 1.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-04-21T08:46:16.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:16 smithi043 ceph-mon[27978]: pgmap v225: 129 pgs: 129 active+clean; 1.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-04-21T08:46:16.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:16 smithi175 ceph-mon[31920]: pgmap v225: 129 pgs: 129 active+clean; 1.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-04-21T08:46:16.701 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:16 smithi175 systemd[1]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 1. 2024-04-21T08:46:16.701 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:16 smithi175 systemd[1]: Stopped Ceph mds.c for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:46:16.701 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:16 smithi175 systemd[1]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.c.service: Consumed 1.978s CPU time. 2024-04-21T08:46:16.701 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:16 smithi175 systemd[1]: Starting Ceph mds.c for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:46:16.964 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:16 smithi175 podman[66785]: 2024-04-21 08:46:16.605197432 +0000 UTC m=+0.018440475 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:46:16.964 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:16 smithi175 podman[66785]: 2024-04-21 08:46:16.753695118 +0000 UTC m=+0.166938151 container create 5397489a561c84c2806161ea4c962df51bf8893a6d3c47bf00f7d7503abd8b29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-21T08:46:17.273 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:16 smithi175 podman[66785]: 2024-04-21 08:46:16.998129904 +0000 UTC m=+0.411372939 container init 5397489a561c84c2806161ea4c962df51bf8893a6d3c47bf00f7d7503abd8b29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-21T08:46:17.273 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:17 smithi175 podman[66785]: 2024-04-21 08:46:17.000888001 +0000 UTC m=+0.414131034 container start 5397489a561c84c2806161ea4c962df51bf8893a6d3c47bf00f7d7503abd8b29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux ) 2024-04-21T08:46:17.273 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:17 smithi175 ceph-mds[66804]: -- 172.21.15.175:0/3564434109 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e7f84ba40 con 0x563e7f81e800 2024-04-21T08:46:17.274 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:17 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[66800]: starting mds.c at 2024-04-21T08:46:17.274 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:17 smithi175 bash[66785]: 5397489a561c84c2806161ea4c962df51bf8893a6d3c47bf00f7d7503abd8b29 2024-04-21T08:46:17.274 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:46:17 smithi175 systemd[1]: Started Ceph mds.c for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:46:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.308+0000 7f2d7cc25640 1 -- 172.21.15.43:0/844559545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d781021c0 msgr2=0x7f2d781025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.308+0000 7f2d7cc25640 1 --2- 172.21.15.43:0/844559545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d781021c0 0x7f2d781025a0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009900 tx=0x7f2d6802bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.308+0000 7f2d7cc25640 1 -- 172.21.15.43:0/844559545 shutdown_connections 2024-04-21T08:46:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.308+0000 7f2d7cc25640 1 --2- 172.21.15.43:0/844559545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d78107290 0x7f2d7810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.308+0000 7f2d7cc25640 1 --2- 172.21.15.43:0/844559545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d78102b70 0x7f2d78106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.308+0000 7f2d7cc25640 1 --2- 172.21.15.43:0/844559545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d781021c0 0x7f2d781025a0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.308+0000 7f2d7cc25640 1 -- 172.21.15.43:0/844559545 >> 172.21.15.43:0/844559545 conn(0x7f2d780fda20 msgr2=0x7f2d780ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.308+0000 7f2d7cc25640 1 -- 172.21.15.43:0/844559545 shutdown_connections 2024-04-21T08:46:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.308+0000 7f2d7cc25640 1 -- 172.21.15.43:0/844559545 wait complete. 2024-04-21T08:46:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.309+0000 7f2d7cc25640 1 Processor -- start 2024-04-21T08:46:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.309+0000 7f2d7cc25640 1 -- start start 2024-04-21T08:46:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.309+0000 7f2d7cc25640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d781021c0 0x7f2d78198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.309+0000 7f2d7cc25640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d78107290 0x7f2d78198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.309+0000 7f2d7cc25640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7819cfc0 0x7f2d7819d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.309+0000 7f2d7cc25640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2d78112220 con 0x7f2d781021c0 2024-04-21T08:46:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.309+0000 7f2d7cc25640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2d781120a0 con 0x7f2d7819cfc0 2024-04-21T08:46:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.309+0000 7f2d7cc25640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2d781123a0 con 0x7f2d78107290 2024-04-21T08:46:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.310+0000 7f2d7659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d781021c0 0x7f2d78198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.310+0000 7f2d7659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d781021c0 0x7f2d78198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33450/0 (socket says 172.21.15.43:33450) 2024-04-21T08:46:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.310+0000 7f2d7659c640 1 -- 172.21.15.43:0/766673885 learned_addr learned my addr 172.21.15.43:0/766673885 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.310+0000 7f2d75d9b640 1 --2- 172.21.15.43:0/766673885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d78107290 0x7f2d78198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.310+0000 7f2d76d9d640 1 --2- 172.21.15.43:0/766673885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7819cfc0 0x7f2d7819d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.310+0000 7f2d7659c640 1 -- 172.21.15.43:0/766673885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d78107290 msgr2=0x7f2d78198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.310+0000 7f2d7659c640 1 --2- 172.21.15.43:0/766673885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d78107290 0x7f2d78198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.310+0000 7f2d7659c640 1 -- 172.21.15.43:0/766673885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7819cfc0 msgr2=0x7f2d7819d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.310+0000 7f2d7659c640 1 --2- 172.21.15.43:0/766673885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7819cfc0 0x7f2d7819d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.310+0000 7f2d7659c640 1 -- 172.21.15.43:0/766673885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7819dab0 con 0x7f2d781021c0 2024-04-21T08:46:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.310+0000 7f2d7659c640 1 --2- 172.21.15.43:0/766673885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d781021c0 0x7f2d78198700 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f2d68000c00 tx=0x7f2d6802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.310+0000 7f2d677fe640 1 -- 172.21.15.43:0/766673885 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d68043020 con 0x7f2d781021c0 2024-04-21T08:46:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.310+0000 7f2d677fe640 1 -- 172.21.15.43:0/766673885 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d680052b0 con 0x7f2d781021c0 2024-04-21T08:46:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.310+0000 7f2d7cc25640 1 -- 172.21.15.43:0/766673885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7819dd40 con 0x7f2d781021c0 2024-04-21T08:46:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.310+0000 7f2d677fe640 1 -- 172.21.15.43:0/766673885 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d68003c60 con 0x7f2d781021c0 2024-04-21T08:46:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.311+0000 7f2d7cc25640 1 -- 172.21.15.43:0/766673885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d781a5560 con 0x7f2d781021c0 2024-04-21T08:46:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.313+0000 7f2d677fe640 1 -- 172.21.15.43:0/766673885 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2d68046020 con 0x7f2d781021c0 2024-04-21T08:46:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.313+0000 7f2d7f813740 1 Processor -- start 2024-04-21T08:46:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.314+0000 7f2d7f813740 1 -- start start 2024-04-21T08:46:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.314+0000 7f2d7f813740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55805f4ab8d0 0x55805f4aa620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.314+0000 7f2d7f813740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55805f4aab60 0x55805f4aafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.314+0000 7f2d7f813740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4b40e0 0x55805f4b64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.314+0000 7f2d7f813740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55805f40f8d0 con 0x55805f4b40e0 2024-04-21T08:46:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.314+0000 7f2d7f813740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55805f3ace80 con 0x55805f4aab60 2024-04-21T08:46:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.314+0000 7f2d7f813740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55805f40b890 con 0x55805f4ab8d0 2024-04-21T08:46:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.314+0000 7f2d677fe640 1 --2- 172.21.15.43:0/766673885 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2d600a3dc0 0x7f2d600a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.314+0000 7f2d75d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4b40e0 0x55805f4b64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.314+0000 7f2d7659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55805f4aab60 0x55805f4aafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.314+0000 7f2d75d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4b40e0 0x55805f4b64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33464/0 (socket says 172.21.15.43:33464) 2024-04-21T08:46:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.314+0000 7f2d76d9d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55805f4ab8d0 0x55805f4aa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.314+0000 7f2d75d9b640 1 -- 172.21.15.43:0/2564551200 learned_addr learned my addr 172.21.15.43:0/2564551200 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.314+0000 7f2d76d9d640 1 --2- 172.21.15.43:0/766673885 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2d600a3dc0 0x7f2d600a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.314+0000 7f2d677fe640 1 -- 172.21.15.43:0/766673885 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f2d680f2cc0 con 0x7f2d781021c0 2024-04-21T08:46:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.314+0000 7f2d7659c640 1 -- 172.21.15.43:0/2564551200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55805f4ab8d0 msgr2=0x55805f4aa620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.314+0000 7f2d7659c640 1 --2- 172.21.15.43:0/2564551200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55805f4ab8d0 0x55805f4aa620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.314+0000 7f2d7659c640 1 -- 172.21.15.43:0/2564551200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4b40e0 msgr2=0x55805f4b64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.314+0000 7f2d7659c640 1 --2- 172.21.15.43:0/2564551200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4b40e0 0x55805f4b64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.314+0000 7f2d7659c640 1 -- 172.21.15.43:0/2564551200 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55805f3ce320 con 0x55805f4aab60 2024-04-21T08:46:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.314+0000 7f2d75d9b640 1 --2- 172.21.15.43:0/2564551200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4b40e0 0x55805f4b64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.314+0000 7f2d76d9d640 1 --2- 172.21.15.43:0/2564551200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55805f4ab8d0 0x55805f4aa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.315+0000 7f2d7659c640 1 --2- 172.21.15.43:0/2564551200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55805f4aab60 0x55805f4aafe0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2d680f00f0 tx=0x7f2d680be840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=880507c3646fa014 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.315+0000 7f2d76d9d640 1 --2- 172.21.15.43:0/766673885 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2d600a3dc0 0x7f2d600a6280 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009990 tx=0x7f2d700096a0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.315+0000 7f2d64ff9640 1 -- 172.21.15.43:0/2564551200 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d680be480 con 0x55805f4aab60 2024-04-21T08:46:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.315+0000 7f2d64ff9640 1 -- 172.21.15.43:0/2564551200 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d680bf040 con 0x55805f4aab60 2024-04-21T08:46:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.315+0000 7f2d64ff9640 1 -- 172.21.15.43:0/2564551200 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6802cca0 con 0x55805f4aab60 2024-04-21T08:46:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.316+0000 7f2d7f813740 1 -- 172.21.15.43:0/2564551200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55805f4aab60 msgr2=0x55805f4aafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.316+0000 7f2d7f813740 1 --2- 172.21.15.43:0/2564551200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55805f4aab60 0x55805f4aafe0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2d680f00f0 tx=0x7f2d680be840 comp rx=0 tx=0).stop 2024-04-21T08:46:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.316+0000 7f2d7f813740 1 -- 172.21.15.43:0/2564551200 shutdown_connections 2024-04-21T08:46:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.316+0000 7f2d7f813740 1 --2- 172.21.15.43:0/2564551200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4b40e0 0x55805f4b64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.316+0000 7f2d7f813740 1 --2- 172.21.15.43:0/2564551200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55805f4aab60 0x55805f4aafe0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.316+0000 7f2d7f813740 1 --2- 172.21.15.43:0/2564551200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55805f4ab8d0 0x55805f4aa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.316+0000 7f2d7f813740 1 -- 172.21.15.43:0/2564551200 >> 172.21.15.43:0/2564551200 conn(0x55805f471c90 msgr2=0x55805f4a5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.316+0000 7f2d7f813740 1 -- 172.21.15.43:0/2564551200 shutdown_connections 2024-04-21T08:46:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.316+0000 7f2d7f813740 1 -- 172.21.15.43:0/2564551200 wait complete. 2024-04-21T08:46:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.316+0000 7f2d7f813740 1 -- start start 2024-04-21T08:46:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.316+0000 7f2d7f813740 1 Processor -- start 2024-04-21T08:46:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.316+0000 7f2d7f813740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55805f4aab60 0x55805f4b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.316+0000 7f2d7f813740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4ab8d0 0x55805f4b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.316+0000 7f2d7f813740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55805f4b40e0 0x55805f4b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.316+0000 7f2d7f813740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55805f4b6bd0 con 0x55805f4ab8d0 2024-04-21T08:46:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.316+0000 7f2d7f813740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55805f397eb0 con 0x55805f4b40e0 2024-04-21T08:46:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.316+0000 7f2d7f813740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55805f4b6d50 con 0x55805f4aab60 2024-04-21T08:46:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.317+0000 7f2d7f813740 10 client.0 _ll_register_callbacks cb 0x55805f3c4b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.317+0000 7f2d75d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4ab8d0 0x55805f4b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.317+0000 7f2d76d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55805f4b40e0 0x55805f4b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.317+0000 7f2d75d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4ab8d0 0x55805f4b0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33478/0 (socket says 172.21.15.43:33478) 2024-04-21T08:46:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.317+0000 7f2d7659c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55805f4aab60 0x55805f4b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.317+0000 7f2d75d9b640 1 -- 172.21.15.43:0/3065703127 learned_addr learned my addr 172.21.15.43:0/3065703127 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.317+0000 7f2d76d9d640 1 -- 172.21.15.43:0/3065703127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55805f4aab60 msgr2=0x55805f4b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.317+0000 7f2d76d9d640 1 --2- 172.21.15.43:0/3065703127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55805f4aab60 0x55805f4b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.317+0000 7f2d76d9d640 1 -- 172.21.15.43:0/3065703127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4ab8d0 msgr2=0x55805f4b0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.317+0000 7f2d76d9d640 1 --2- 172.21.15.43:0/3065703127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4ab8d0 0x55805f4b0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.317+0000 7f2d76d9d640 1 -- 172.21.15.43:0/3065703127 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55805f122210 con 0x55805f4b40e0 2024-04-21T08:46:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.317+0000 7f2d75d9b640 1 --2- 172.21.15.43:0/3065703127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4ab8d0 0x55805f4b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.317+0000 7f2d76d9d640 1 --2- 172.21.15.43:0/3065703127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55805f4b40e0 0x55805f4b15f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2d70003cc0 tx=0x7f2d70004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b6a413acc1869d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.317+0000 7f2d5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-21T08:46:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.317+0000 7f2d5dffb640 1 -- 172.21.15.43:0/3065703127 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7000bc80 con 0x55805f4b40e0 2024-04-21T08:46:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.317+0000 7f2d7f813740 1 -- 172.21.15.43:0/3065703127 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d680bf040 con 0x55805f4b40e0 2024-04-21T08:46:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.317+0000 7f2d5dffb640 1 -- 172.21.15.43:0/3065703127 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f2d700045c0 con 0x55805f4b40e0 2024-04-21T08:46:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.318+0000 7f2d5dffb640 1 -- 172.21.15.43:0/3065703127 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2d7000be20 con 0x55805f4b40e0 2024-04-21T08:46:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.318+0000 7f2d7f813740 10 client.27962 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.318+0000 7f2d7f813740 1 -- 172.21.15.43:0/3065703127 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55805f1ee620 con 0x55805f4b40e0 2024-04-21T08:46:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.318+0000 7f2d5dffb640 1 -- 172.21.15.43:0/3065703127 <== mon.2 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d680bf040 con 0x55805f4b40e0 2024-04-21T08:46:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.318+0000 7f2d5dffb640 1 -- 172.21.15.43:0/3065703127 <== mon.2 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2d70010730 con 0x55805f4b40e0 2024-04-21T08:46:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.318+0000 7f2d7f813740 10 client.27962 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.318+0000 7f2d7f813740 10 client.27962 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.318+0000 7f2d7f813740 20 client.27962 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.318+0000 7f2d7f813740 1 --2- 172.21.15.43:0/3065703127 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55805f4c3f20 0x55805f4c6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.318+0000 7f2d7f813740 4 client.27962 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.318+0000 7f2d7f813740 1 -- 172.21.15.43:0/3065703127 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55805f423880 con 0x55805f4c3f20 2024-04-21T08:46:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.318+0000 7f2d7659c640 1 --2- 172.21.15.43:0/3065703127 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55805f4c3f20 0x55805f4c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.319+0000 7f2d7659c640 1 --2- 172.21.15.43:0/3065703127 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55805f4c3f20 0x55805f4c6310 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cffebe457b07cf3 server_cookie=87e2ca00d4eefc92 in_seq=0 out_seq=0 2024-04-21T08:46:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.319+0000 7f2d5dffb640 10 client.27962 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.319+0000 7f2d5dffb640 1 -- 172.21.15.43:0/3065703127 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55805f423880 con 0x55805f4c3f20 2024-04-21T08:46:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.319+0000 7f2d5dffb640 10 client.27962 handle_command_reply: tid=0 2024-04-21T08:46:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.320+0000 7f2d7f813740 1 client.27962 shutdown 2024-04-21T08:46:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.320+0000 7f2d7f813740 1 -- 172.21.15.43:0/3065703127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55805f4b40e0 msgr2=0x55805f4b15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.320+0000 7f2d7f813740 1 --2- 172.21.15.43:0/3065703127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55805f4b40e0 0x55805f4b15f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2d70003cc0 tx=0x7f2d70004a40 comp rx=0 tx=0).stop 2024-04-21T08:46:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.320+0000 7f2d7f813740 1 -- 172.21.15.43:0/3065703127 shutdown_connections 2024-04-21T08:46:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.320+0000 7f2d7f813740 1 --2- 172.21.15.43:0/3065703127 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55805f4c3f20 0x55805f4c6310 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.320+0000 7f2d7f813740 1 --2- 172.21.15.43:0/3065703127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55805f4b40e0 0x55805f4b15f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.320+0000 7f2d7f813740 1 --2- 172.21.15.43:0/3065703127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4ab8d0 0x55805f4b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.320+0000 7f2d7f813740 1 --2- 172.21.15.43:0/3065703127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55805f4aab60 0x55805f4b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.320+0000 7f2d5dffb640 0 client.27962 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.320+0000 7f2d7f813740 1 -- 172.21.15.43:0/3065703127 >> 172.21.15.43:0/3065703127 conn(0x55805f471c90 msgr2=0x55805f4a75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.320+0000 7f2d7f813740 1 -- 172.21.15.43:0/3065703127 shutdown_connections 2024-04-21T08:46:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.320+0000 7f2d7f813740 1 -- 172.21.15.43:0/3065703127 wait complete. 2024-04-21T08:46:17.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.320+0000 7f2d7f813740 20 client.27962 trim_cache size 0 max 16384 2024-04-21T08:46:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.335+0000 7f2d7f813740 1 Processor -- start 2024-04-21T08:46:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.335+0000 7f2d7f813740 1 -- start start 2024-04-21T08:46:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.336+0000 7f2d7f813740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55805f4aab60 0x55805f4cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.336+0000 7f2d7f813740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55805f4ab8d0 0x55805f4cc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.336+0000 7f2d7f813740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4b40e0 0x55805f4b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.336+0000 7f2d7f813740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2d781af420 con 0x55805f4b40e0 2024-04-21T08:46:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.336+0000 7f2d7f813740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2d781b0f60 con 0x55805f4ab8d0 2024-04-21T08:46:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.336+0000 7f2d7f813740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2d781afaf0 con 0x55805f4aab60 2024-04-21T08:46:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.336+0000 7f2d76d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4b40e0 0x55805f4b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.336+0000 7f2d76d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4b40e0 0x55805f4b35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33484/0 (socket says 172.21.15.43:33484) 2024-04-21T08:46:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.336+0000 7f2d76d9d640 1 -- 172.21.15.43:0/3788891098 learned_addr learned my addr 172.21.15.43:0/3788891098 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.336+0000 7f2d75d9b640 1 --2- 172.21.15.43:0/3788891098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55805f4ab8d0 0x55805f4cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.336+0000 7f2d7659c640 1 --2- 172.21.15.43:0/3788891098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55805f4aab60 0x55805f4cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.336+0000 7f2d76d9d640 1 -- 172.21.15.43:0/3788891098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55805f4aab60 msgr2=0x55805f4cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.336+0000 7f2d76d9d640 1 --2- 172.21.15.43:0/3788891098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55805f4aab60 0x55805f4cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.336+0000 7f2d76d9d640 1 -- 172.21.15.43:0/3788891098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55805f4ab8d0 msgr2=0x55805f4cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.336+0000 7f2d76d9d640 1 --2- 172.21.15.43:0/3788891098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55805f4ab8d0 0x55805f4cc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.336+0000 7f2d76d9d640 1 -- 172.21.15.43:0/3788891098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55805f4a9e30 con 0x55805f4b40e0 2024-04-21T08:46:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.336+0000 7f2d75d9b640 1 --2- 172.21.15.43:0/3788891098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55805f4ab8d0 0x55805f4cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.336+0000 7f2d76d9d640 1 --2- 172.21.15.43:0/3788891098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4b40e0 0x55805f4b35e0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000b7c0 tx=0x7f2d70009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4387692fd034c23 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.336+0000 7f2d5ffff640 1 -- 172.21.15.43:0/3788891098 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d70003e70 con 0x55805f4b40e0 2024-04-21T08:46:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.336+0000 7f2d5ffff640 1 -- 172.21.15.43:0/3788891098 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d700120b0 con 0x55805f4b40e0 2024-04-21T08:46:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.336+0000 7f2d5ffff640 1 -- 172.21.15.43:0/3788891098 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d70010740 con 0x55805f4b40e0 2024-04-21T08:46:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.337+0000 7f2d7f813740 1 -- 172.21.15.43:0/3788891098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4b40e0 msgr2=0x55805f4b35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.337+0000 7f2d7f813740 1 --2- 172.21.15.43:0/3788891098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4b40e0 0x55805f4b35e0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000b7c0 tx=0x7f2d70009d50 comp rx=0 tx=0).stop 2024-04-21T08:46:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.337+0000 7f2d7f813740 1 -- 172.21.15.43:0/3788891098 shutdown_connections 2024-04-21T08:46:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.337+0000 7f2d7f813740 1 --2- 172.21.15.43:0/3788891098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4b40e0 0x55805f4b35e0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.337+0000 7f2d7f813740 1 --2- 172.21.15.43:0/3788891098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55805f4ab8d0 0x55805f4cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.337+0000 7f2d7f813740 1 --2- 172.21.15.43:0/3788891098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55805f4aab60 0x55805f4cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.337+0000 7f2d7f813740 1 -- 172.21.15.43:0/3788891098 >> 172.21.15.43:0/3788891098 conn(0x55805f471c90 msgr2=0x55805f4ac080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.337+0000 7f2d7f813740 1 -- 172.21.15.43:0/3788891098 shutdown_connections 2024-04-21T08:46:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.337+0000 7f2d7f813740 1 -- 172.21.15.43:0/3788891098 wait complete. 2024-04-21T08:46:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.337+0000 7f2d7f813740 1 -- start start 2024-04-21T08:46:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.337+0000 7f2d7f813740 1 Processor -- start 2024-04-21T08:46:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.338+0000 7f2d7f813740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55805f4aab60 0x55805f4bbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.338+0000 7f2d7f813740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4ab8d0 0x55805f4bc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.338+0000 7f2d7f813740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55805f4b40e0 0x55805f4ae150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.338+0000 7f2d7f813740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2d781b0890 con 0x55805f4ab8d0 2024-04-21T08:46:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.338+0000 7f2d7f813740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2d781b01c0 con 0x55805f4aab60 2024-04-21T08:46:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.338+0000 7f2d7f813740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2d781ad400 con 0x55805f4b40e0 2024-04-21T08:46:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.338+0000 7f2d7f813740 10 client.0 _ll_register_callbacks cb 0x7f2d70003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.338+0000 7f2d75d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4ab8d0 0x55805f4bc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.338+0000 7f2d75d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4ab8d0 0x55805f4bc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33498/0 (socket says 172.21.15.43:33498) 2024-04-21T08:46:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.338+0000 7f2d75d9b640 1 -- 172.21.15.43:0/3338316323 learned_addr learned my addr 172.21.15.43:0/3338316323 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.338+0000 7f2d7659c640 1 --2- 172.21.15.43:0/3338316323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55805f4aab60 0x55805f4bbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.338+0000 7f2d76d9d640 1 --2- 172.21.15.43:0/3338316323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55805f4b40e0 0x55805f4ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.338+0000 7f2d75d9b640 1 -- 172.21.15.43:0/3338316323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55805f4b40e0 msgr2=0x55805f4ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.338+0000 7f2d75d9b640 1 --2- 172.21.15.43:0/3338316323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55805f4b40e0 0x55805f4ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.338+0000 7f2d75d9b640 1 -- 172.21.15.43:0/3338316323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55805f4aab60 msgr2=0x55805f4bbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.338+0000 7f2d75d9b640 1 --2- 172.21.15.43:0/3338316323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55805f4aab60 0x55805f4bbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.338+0000 7f2d75d9b640 1 -- 172.21.15.43:0/3338316323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55805f4ae980 con 0x55805f4ab8d0 2024-04-21T08:46:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.338+0000 7f2d7659c640 1 --2- 172.21.15.43:0/3338316323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55805f4aab60 0x55805f4bbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.338+0000 7f2d75d9b640 1 --2- 172.21.15.43:0/3338316323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4ab8d0 0x55805f4bc4b0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c005e00 tx=0x7f2d6c011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f096524bf79476e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.338+0000 7f2d5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.338+0000 7f2d5dffb640 1 -- 172.21.15.43:0/3338316323 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c020030 con 0x55805f4ab8d0 2024-04-21T08:46:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.339+0000 7f2d5dffb640 1 -- 172.21.15.43:0/3338316323 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f2d6c010b70 con 0x55805f4ab8d0 2024-04-21T08:46:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.339+0000 7f2d7f813740 1 -- 172.21.15.43:0/3338316323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d700120b0 con 0x55805f4ab8d0 2024-04-21T08:46:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.339+0000 7f2d5dffb640 1 -- 172.21.15.43:0/3338316323 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2d6c02a050 con 0x55805f4ab8d0 2024-04-21T08:46:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.339+0000 7f2d7f813740 10 client.19086 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.339+0000 7f2d7f813740 1 -- 172.21.15.43:0/3338316323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55805f4b4ff0 con 0x55805f4ab8d0 2024-04-21T08:46:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.339+0000 7f2d5dffb640 1 -- 172.21.15.43:0/3338316323 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d700120b0 con 0x55805f4ab8d0 2024-04-21T08:46:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.339+0000 7f2d5dffb640 1 -- 172.21.15.43:0/3338316323 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2d6c025400 con 0x55805f4ab8d0 2024-04-21T08:46:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.339+0000 7f2d7f813740 10 client.19086 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.339+0000 7f2d7f813740 10 client.19086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.339+0000 7f2d7f813740 20 client.19086 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.339+0000 7f2d7f813740 1 --2- 172.21.15.43:0/3338316323 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55805f52a780 0x55805f52cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.339+0000 7f2d7f813740 4 client.19086 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:46:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.339+0000 7f2d7f813740 1 -- 172.21.15.43:0/3338316323 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55805f3af3d0 con 0x55805f52a780 2024-04-21T08:46:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.339+0000 7f2d7659c640 1 --2- 172.21.15.43:0/3338316323 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55805f52a780 0x55805f52cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.340+0000 7f2d7659c640 1 --2- 172.21.15.43:0/3338316323 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55805f52a780 0x55805f52cba0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cabff0cd53b7e2b7 server_cookie=e2bcb4f51b296888 in_seq=0 out_seq=0 2024-04-21T08:46:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.340+0000 7f2d5dffb640 10 client.19086 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.346+0000 7f2d5dffb640 1 -- 172.21.15.43:0/3338316323 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55805f3af3d0 con 0x55805f52a780 2024-04-21T08:46:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.346+0000 7f2d5dffb640 10 client.19086 handle_command_reply: tid=0 2024-04-21T08:46:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.346+0000 7f2d7f813740 1 client.19086 shutdown 2024-04-21T08:46:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.346+0000 7f2d7f813740 1 -- 172.21.15.43:0/3338316323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4ab8d0 msgr2=0x55805f4bc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.346+0000 7f2d7f813740 1 --2- 172.21.15.43:0/3338316323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4ab8d0 0x55805f4bc4b0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c005e00 tx=0x7f2d6c011cb0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.346+0000 7f2d7f813740 1 -- 172.21.15.43:0/3338316323 shutdown_connections 2024-04-21T08:46:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.346+0000 7f2d7f813740 1 --2- 172.21.15.43:0/3338316323 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55805f52a780 0x55805f52cba0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.346+0000 7f2d7f813740 1 --2- 172.21.15.43:0/3338316323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55805f4b40e0 0x55805f4ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.346+0000 7f2d7f813740 1 --2- 172.21.15.43:0/3338316323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55805f4ab8d0 0x55805f4bc4b0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.346+0000 7f2d7f813740 1 --2- 172.21.15.43:0/3338316323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55805f4aab60 0x55805f4bbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.346+0000 7f2d5dffb640 0 client.19086 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.346+0000 7f2d7f813740 1 -- 172.21.15.43:0/3338316323 >> 172.21.15.43:0/3338316323 conn(0x55805f471c90 msgr2=0x55805f4af460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.346+0000 7f2d7f813740 1 -- 172.21.15.43:0/3338316323 shutdown_connections 2024-04-21T08:46:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.346+0000 7f2d7f813740 1 -- 172.21.15.43:0/3338316323 wait complete. 2024-04-21T08:46:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.346+0000 7f2d7f813740 20 client.19086 trim_cache size 0 max 16384 2024-04-21T08:46:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:46:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:46:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:46:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:46:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.347+0000 7f2d7cc25640 1 -- 172.21.15.43:0/766673885 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2d600a3dc0 msgr2=0x7f2d600a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.347+0000 7f2d7cc25640 1 --2- 172.21.15.43:0/766673885 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2d600a3dc0 0x7f2d600a6280 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009990 tx=0x7f2d700096a0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.347+0000 7f2d7cc25640 1 -- 172.21.15.43:0/766673885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d781021c0 msgr2=0x7f2d78198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.347+0000 7f2d7cc25640 1 --2- 172.21.15.43:0/766673885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d781021c0 0x7f2d78198700 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f2d68000c00 tx=0x7f2d6802c7e0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.347+0000 7f2d7cc25640 1 -- 172.21.15.43:0/766673885 shutdown_connections 2024-04-21T08:46:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.347+0000 7f2d7cc25640 1 --2- 172.21.15.43:0/766673885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7819cfc0 0x7f2d7819d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.347+0000 7f2d7cc25640 1 --2- 172.21.15.43:0/766673885 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2d600a3dc0 0x7f2d600a6280 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.347+0000 7f2d7cc25640 1 --2- 172.21.15.43:0/766673885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d78107290 0x7f2d78198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.347+0000 7f2d7cc25640 1 --2- 172.21.15.43:0/766673885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d781021c0 0x7f2d78198700 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.347+0000 7f2d7cc25640 1 -- 172.21.15.43:0/766673885 >> 172.21.15.43:0/766673885 conn(0x7f2d780fda20 msgr2=0x7f2d780ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.347+0000 7f2d7cc25640 1 -- 172.21.15.43:0/766673885 shutdown_connections 2024-04-21T08:46:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:17.347+0000 7f2d7cc25640 1 -- 172.21.15.43:0/766673885 wait complete. 2024-04-21T08:46:17.529 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:17 smithi175 ceph-mon[31920]: daemon mds.c restarted 2024-04-21T08:46:17.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:17 smithi112 ceph-mon[31085]: daemon mds.c restarted 2024-04-21T08:46:17.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:17 smithi043 ceph-mon[27978]: daemon mds.c restarted 2024-04-21T08:46:17.835 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:46:17.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.950+0000 7fce37a8c640 1 -- 172.21.15.43:0/860007834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce301011a0 msgr2=0x7fce30101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:17.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.950+0000 7fce37a8c640 1 --2- 172.21.15.43:0/860007834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce301011a0 0x7fce30101620 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fce20009930 tx=0x7fce2002bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:17.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.950+0000 7fce37a8c640 1 -- 172.21.15.43:0/860007834 shutdown_connections 2024-04-21T08:46:17.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.950+0000 7fce37a8c640 1 --2- 172.21.15.43:0/860007834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce30101b60 0x7fce3010dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.950+0000 7fce37a8c640 1 --2- 172.21.15.43:0/860007834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce301011a0 0x7fce30101620 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.950+0000 7fce37a8c640 1 --2- 172.21.15.43:0/860007834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce30105e30 0x7fce30106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.950+0000 7fce37a8c640 1 -- 172.21.15.43:0/860007834 >> 172.21.15.43:0/860007834 conn(0x7fce30077e30 msgr2=0x7fce30078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:17.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.950+0000 7fce37a8c640 1 -- 172.21.15.43:0/860007834 shutdown_connections 2024-04-21T08:46:17.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.950+0000 7fce37a8c640 1 -- 172.21.15.43:0/860007834 wait complete. 2024-04-21T08:46:17.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.951+0000 7fce37a8c640 1 Processor -- start 2024-04-21T08:46:17.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.951+0000 7fce37a8c640 1 -- start start 2024-04-21T08:46:17.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.951+0000 7fce37a8c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce301011a0 0x7fce30198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.951+0000 7fce37a8c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce30101b60 0x7fce30198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.951+0000 7fce37a8c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce30105e30 0x7fce3019cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.951+0000 7fce37a8c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fce301110e0 con 0x7fce30101b60 2024-04-21T08:46:17.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.951+0000 7fce37a8c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fce30110f60 con 0x7fce301011a0 2024-04-21T08:46:17.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.951+0000 7fce37a8c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fce30111260 con 0x7fce30105e30 2024-04-21T08:46:17.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.951+0000 7fce35027640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce30101b60 0x7fce30198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.951+0000 7fce35828640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce301011a0 0x7fce30198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.951+0000 7fce35027640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce30101b60 0x7fce30198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33524/0 (socket says 172.21.15.43:33524) 2024-04-21T08:46:17.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.951+0000 7fce35828640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce301011a0 0x7fce30198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:33330/0 (socket says 172.21.15.43:33330) 2024-04-21T08:46:17.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.951+0000 7fce35027640 1 -- 172.21.15.43:0/4270511773 learned_addr learned my addr 172.21.15.43:0/4270511773 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:17.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.951+0000 7fce36029640 1 --2- 172.21.15.43:0/4270511773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce30105e30 0x7fce3019cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.951+0000 7fce35828640 1 -- 172.21.15.43:0/4270511773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce30105e30 msgr2=0x7fce3019cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:17.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.951+0000 7fce35828640 1 --2- 172.21.15.43:0/4270511773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce30105e30 0x7fce3019cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.951+0000 7fce35828640 1 -- 172.21.15.43:0/4270511773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce30101b60 msgr2=0x7fce30198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:17.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.951+0000 7fce35828640 1 --2- 172.21.15.43:0/4270511773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce30101b60 0x7fce30198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.951+0000 7fce35828640 1 -- 172.21.15.43:0/4270511773 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3019d710 con 0x7fce301011a0 2024-04-21T08:46:17.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.951+0000 7fce35027640 1 --2- 172.21.15.43:0/4270511773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce30101b60 0x7fce30198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:17.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.951+0000 7fce36029640 1 --2- 172.21.15.43:0/4270511773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce30105e30 0x7fce3019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:17.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.952+0000 7fce35828640 1 --2- 172.21.15.43:0/4270511773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce301011a0 0x7fce30198720 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00e9f0 tx=0x7fce2c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:17.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.952+0000 7fce26ffd640 1 -- 172.21.15.43:0/4270511773 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c00ccc0 con 0x7fce301011a0 2024-04-21T08:46:17.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.952+0000 7fce26ffd640 1 -- 172.21.15.43:0/4270511773 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce2c0108e0 con 0x7fce301011a0 2024-04-21T08:46:17.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.952+0000 7fce37a8c640 1 -- 172.21.15.43:0/4270511773 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce3019da00 con 0x7fce301011a0 2024-04-21T08:46:17.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.952+0000 7fce26ffd640 1 -- 172.21.15.43:0/4270511773 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c018880 con 0x7fce301011a0 2024-04-21T08:46:17.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.952+0000 7fce37a8c640 1 -- 172.21.15.43:0/4270511773 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce301a5290 con 0x7fce301011a0 2024-04-21T08:46:17.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.953+0000 7fce37a8c640 1 -- 172.21.15.43:0/4270511773 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce30068de0 con 0x7fce301011a0 2024-04-21T08:46:17.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.954+0000 7fce26ffd640 1 -- 172.21.15.43:0/4270511773 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fce2c010430 con 0x7fce301011a0 2024-04-21T08:46:17.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.955+0000 7fce26ffd640 1 --2- 172.21.15.43:0/4270511773 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fce1c0a3d20 0x7fce1c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.955+0000 7fce35027640 1 --2- 172.21.15.43:0/4270511773 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fce1c0a3d20 0x7fce1c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.955+0000 7fce26ffd640 1 -- 172.21.15.43:0/4270511773 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7fce2c0d13a0 con 0x7fce301011a0 2024-04-21T08:46:17.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.955+0000 7fce35027640 1 --2- 172.21.15.43:0/4270511773 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fce1c0a3d20 0x7fce1c0a61e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fce20000c00 tx=0x7fce200058d0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:17.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:17.958+0000 7fce26ffd640 1 -- 172.21.15.43:0/4270511773 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce2c09d020 con 0x7fce301011a0 2024-04-21T08:46:18.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.183+0000 7fce37a8c640 1 -- 172.21.15.43:0/4270511773 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce301a54c0 con 0x7fce301011a0 2024-04-21T08:46:18.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.185+0000 7fce26ffd640 1 -- 172.21.15.43:0/4270511773 <== mon.2 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11935 (secure 0 0 0) 0x7fce2c0a1320 con 0x7fce301011a0 2024-04-21T08:46:18.184 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:46:18.185 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:46:17.283806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27556":{"gid":27556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/229985237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":229985237},{"type":"v1","addr":"172.21.15.175:6837","nonce":229985237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24425,24412,24487,24527,24491]},"id":1}]} 2024-04-21T08:46:18.185 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-04-21T08:46:18.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.186+0000 7fce37a8c640 1 -- 172.21.15.43:0/4270511773 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fce1c0a3d20 msgr2=0x7fce1c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:18.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.186+0000 7fce37a8c640 1 --2- 172.21.15.43:0/4270511773 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fce1c0a3d20 0x7fce1c0a61e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fce20000c00 tx=0x7fce200058d0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.186+0000 7fce37a8c640 1 -- 172.21.15.43:0/4270511773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce301011a0 msgr2=0x7fce30198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:18.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.186+0000 7fce37a8c640 1 --2- 172.21.15.43:0/4270511773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce301011a0 0x7fce30198720 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00e9f0 tx=0x7fce2c00eeb0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.186+0000 7fce37a8c640 1 -- 172.21.15.43:0/4270511773 shutdown_connections 2024-04-21T08:46:18.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.186+0000 7fce37a8c640 1 --2- 172.21.15.43:0/4270511773 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fce1c0a3d20 0x7fce1c0a61e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.186+0000 7fce37a8c640 1 --2- 172.21.15.43:0/4270511773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce30105e30 0x7fce3019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.186+0000 7fce37a8c640 1 --2- 172.21.15.43:0/4270511773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce30101b60 0x7fce30198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.186+0000 7fce37a8c640 1 --2- 172.21.15.43:0/4270511773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce301011a0 0x7fce30198720 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.186+0000 7fce37a8c640 1 -- 172.21.15.43:0/4270511773 >> 172.21.15.43:0/4270511773 conn(0x7fce30077e30 msgr2=0x7fce30104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:18.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.186+0000 7fce37a8c640 1 -- 172.21.15.43:0/4270511773 shutdown_connections 2024-04-21T08:46:18.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.186+0000 7fce37a8c640 1 -- 172.21.15.43:0/4270511773 wait complete. 2024-04-21T08:46:18.197 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db --set-id c9e85759 --release --await' 2024-04-21T08:46:18.198 DEBUG:teuthology.orchestra.run.smithi043:> timeout 45 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24527 quiesce db --set-id c9e85759 --release --await 2024-04-21T08:46:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.308+0000 7ff953df4640 1 -- 172.21.15.43:0/975473273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff94c107270 msgr2=0x7ff94c10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.308+0000 7ff953df4640 1 --2- 172.21.15.43:0/975473273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff94c107270 0x7ff94c10ee40 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7ff944009ca0 tx=0x7ff94402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.308+0000 7ff953df4640 1 -- 172.21.15.43:0/975473273 shutdown_connections 2024-04-21T08:46:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.308+0000 7ff953df4640 1 --2- 172.21.15.43:0/975473273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff94c107270 0x7ff94c10ee40 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.308+0000 7ff953df4640 1 --2- 172.21.15.43:0/975473273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff94c06b1d0 0x7ff94c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.308+0000 7ff953df4640 1 --2- 172.21.15.43:0/975473273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff94c06a820 0x7ff94c06ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.308+0000 7ff953df4640 1 -- 172.21.15.43:0/975473273 >> 172.21.15.43:0/975473273 conn(0x7ff94c077eb0 msgr2=0x7ff94c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.308+0000 7ff953df4640 1 -- 172.21.15.43:0/975473273 shutdown_connections 2024-04-21T08:46:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.308+0000 7ff953df4640 1 -- 172.21.15.43:0/975473273 wait complete. 2024-04-21T08:46:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.309+0000 7ff953df4640 1 Processor -- start 2024-04-21T08:46:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.309+0000 7ff953df4640 1 -- start start 2024-04-21T08:46:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.309+0000 7ff953df4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff94c06a820 0x7ff94c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.309+0000 7ff953df4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff94c107270 0x7ff94c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.309+0000 7ff953df4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff94c19cfb0 0x7ff94c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.309+0000 7ff953df4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1121d0 con 0x7ff94c06a820 2024-04-21T08:46:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.309+0000 7ff953df4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c112050 con 0x7ff94c19cfb0 2024-04-21T08:46:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.309+0000 7ff953df4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c112350 con 0x7ff94c107270 2024-04-21T08:46:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.309+0000 7ff951b90640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff94c06a820 0x7ff94c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.309+0000 7ff951b90640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff94c06a820 0x7ff94c1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33540/0 (socket says 172.21.15.43:33540) 2024-04-21T08:46:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.309+0000 7ff951b90640 1 -- 172.21.15.43:0/3062633947 learned_addr learned my addr 172.21.15.43:0/3062633947 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.309+0000 7ff95138f640 1 --2- 172.21.15.43:0/3062633947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff94c107270 0x7ff94c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.309+0000 7ff952391640 1 --2- 172.21.15.43:0/3062633947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff94c19cfb0 0x7ff94c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.309+0000 7ff951b90640 1 -- 172.21.15.43:0/3062633947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff94c107270 msgr2=0x7ff94c198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.309+0000 7ff951b90640 1 --2- 172.21.15.43:0/3062633947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff94c107270 0x7ff94c198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.309+0000 7ff951b90640 1 -- 172.21.15.43:0/3062633947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff94c19cfb0 msgr2=0x7ff94c19d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.309+0000 7ff951b90640 1 --2- 172.21.15.43:0/3062633947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff94c19cfb0 0x7ff94c19d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.309+0000 7ff951b90640 1 -- 172.21.15.43:0/3062633947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94c19daa0 con 0x7ff94c06a820 2024-04-21T08:46:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.309+0000 7ff952391640 1 --2- 172.21.15.43:0/3062633947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff94c19cfb0 0x7ff94c19d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.310+0000 7ff951b90640 1 --2- 172.21.15.43:0/3062633947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff94c06a820 0x7ff94c1986f0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7ff94800e9f0 tx=0x7ff94800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.310+0000 7ff942ffd640 1 -- 172.21.15.43:0/3062633947 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94800cd80 con 0x7ff94c06a820 2024-04-21T08:46:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.310+0000 7ff942ffd640 1 -- 172.21.15.43:0/3062633947 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9480054d0 con 0x7ff94c06a820 2024-04-21T08:46:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.310+0000 7ff953df4640 1 -- 172.21.15.43:0/3062633947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff94c19dd90 con 0x7ff94c06a820 2024-04-21T08:46:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.310+0000 7ff942ffd640 1 -- 172.21.15.43:0/3062633947 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948010690 con 0x7ff94c06a820 2024-04-21T08:46:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.310+0000 7ff953df4640 1 -- 172.21.15.43:0/3062633947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff94c1a5530 con 0x7ff94c06a820 2024-04-21T08:46:18.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.312+0000 7ff942ffd640 1 -- 172.21.15.43:0/3062633947 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff948004e90 con 0x7ff94c06a820 2024-04-21T08:46:18.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.312+0000 7ff9569ed740 1 Processor -- start 2024-04-21T08:46:18.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.313+0000 7ff9569ed740 1 -- start start 2024-04-21T08:46:18.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.313+0000 7ff9569ed740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc0310 0x55cb3bbc06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:18.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.313+0000 7ff9569ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3bbc0c30 0x55cb3bbc92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:18.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.313+0000 7ff9569ed740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb3bbc9810 0x55cb3bbcbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:18.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.313+0000 7ff9569ed740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cb3bb25500 con 0x55cb3bbc0310 2024-04-21T08:46:18.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.313+0000 7ff9569ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cb3bacb6a0 con 0x55cb3bbc0c30 2024-04-21T08:46:18.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.313+0000 7ff9569ed740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55cb3bb214c0 con 0x55cb3bbc9810 2024-04-21T08:46:18.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.313+0000 7ff942ffd640 1 --2- 172.21.15.43:0/3062633947 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff9380a3d20 0x7ff9380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:18.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.313+0000 7ff951b90640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3bbc0c30 0x55cb3bbc92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:18.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.313+0000 7ff952391640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc0310 0x55cb3bbc06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:18.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.313+0000 7ff952391640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc0310 0x55cb3bbc06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33542/0 (socket says 172.21.15.43:33542) 2024-04-21T08:46:18.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.313+0000 7ff952391640 1 -- 172.21.15.43:0/2769038677 learned_addr learned my addr 172.21.15.43:0/2769038677 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:18.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.313+0000 7ff952391640 1 --2- 172.21.15.43:0/3062633947 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff9380a3d20 0x7ff9380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:18.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.313+0000 7ff95138f640 1 --2- 172.21.15.43:0/2769038677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb3bbc9810 0x55cb3bbcbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:18.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.313+0000 7ff942ffd640 1 -- 172.21.15.43:0/3062633947 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7ff9480d1080 con 0x7ff94c06a820 2024-04-21T08:46:18.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.313+0000 7ff951b90640 1 -- 172.21.15.43:0/2769038677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb3bbc9810 msgr2=0x55cb3bbcbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:18.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.313+0000 7ff951b90640 1 --2- 172.21.15.43:0/2769038677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb3bbc9810 0x55cb3bbcbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.313+0000 7ff951b90640 1 -- 172.21.15.43:0/2769038677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc0310 msgr2=0x55cb3bbc06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:18.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.313+0000 7ff951b90640 1 --2- 172.21.15.43:0/2769038677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc0310 0x55cb3bbc06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.313+0000 7ff951b90640 1 -- 172.21.15.43:0/2769038677 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb3baea5e0 con 0x55cb3bbc0c30 2024-04-21T08:46:18.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.314+0000 7ff951b90640 1 --2- 172.21.15.43:0/2769038677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3bbc0c30 0x55cb3bbc92d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff9480ce400 tx=0x7ff94806f800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1e7847145b5d585 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:18.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.314+0000 7ff952391640 1 --2- 172.21.15.43:0/3062633947 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff9380a3d20 0x7ff9380a61e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff944033000 tx=0x7ff9440095c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:18.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.314+0000 7ff937fff640 1 -- 172.21.15.43:0/2769038677 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948005180 con 0x55cb3bbc0c30 2024-04-21T08:46:18.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.314+0000 7ff937fff640 1 -- 172.21.15.43:0/2769038677 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff94809d350 con 0x55cb3bbc0c30 2024-04-21T08:46:18.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.314+0000 7ff937fff640 1 -- 172.21.15.43:0/2769038677 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9480a58c0 con 0x55cb3bbc0c30 2024-04-21T08:46:18.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.314+0000 7ff9569ed740 1 -- 172.21.15.43:0/2769038677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3bbc0c30 msgr2=0x55cb3bbc92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:18.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.314+0000 7ff9569ed740 1 --2- 172.21.15.43:0/2769038677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3bbc0c30 0x55cb3bbc92d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff9480ce400 tx=0x7ff94806f800 comp rx=0 tx=0).stop 2024-04-21T08:46:18.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.315+0000 7ff9569ed740 1 -- 172.21.15.43:0/2769038677 shutdown_connections 2024-04-21T08:46:18.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.315+0000 7ff9569ed740 1 --2- 172.21.15.43:0/2769038677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb3bbc9810 0x55cb3bbcbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.315+0000 7ff9569ed740 1 --2- 172.21.15.43:0/2769038677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3bbc0c30 0x55cb3bbc92d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.315+0000 7ff9569ed740 1 --2- 172.21.15.43:0/2769038677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc0310 0x55cb3bbc06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.315+0000 7ff9569ed740 1 -- 172.21.15.43:0/2769038677 >> 172.21.15.43:0/2769038677 conn(0x55cb3bbbaa40 msgr2=0x55cb3bbbae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:18.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.315+0000 7ff9569ed740 1 -- 172.21.15.43:0/2769038677 shutdown_connections 2024-04-21T08:46:18.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.315+0000 7ff9569ed740 1 -- 172.21.15.43:0/2769038677 wait complete. 2024-04-21T08:46:18.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.315+0000 7ff9569ed740 1 -- start start 2024-04-21T08:46:18.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.315+0000 7ff9569ed740 1 Processor -- start 2024-04-21T08:46:18.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.315+0000 7ff9569ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3bbc0310 0x55cb3bbc6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:18.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.315+0000 7ff9569ed740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb3bbc0c30 0x55cb3bbc6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:18.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.315+0000 7ff9569ed740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc9810 0x55cb3bbc71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:18.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.315+0000 7ff9569ed740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cb3bbcc540 con 0x55cb3bbc9810 2024-04-21T08:46:18.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.315+0000 7ff9569ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cb3badea60 con 0x55cb3bbc0310 2024-04-21T08:46:18.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.315+0000 7ff9569ed740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55cb3bbcc6c0 con 0x55cb3bbc0c30 2024-04-21T08:46:18.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.315+0000 7ff9569ed740 10 client.0 _ll_register_callbacks cb 0x55cb3b837110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:18.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.316+0000 7ff952391640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc9810 0x55cb3bbc71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:18.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.316+0000 7ff951b90640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3bbc0310 0x55cb3bbc6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:18.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.316+0000 7ff952391640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc9810 0x55cb3bbc71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33558/0 (socket says 172.21.15.43:33558) 2024-04-21T08:46:18.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.316+0000 7ff952391640 1 -- 172.21.15.43:0/1466027513 learned_addr learned my addr 172.21.15.43:0/1466027513 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:18.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.316+0000 7ff95138f640 1 --2- 172.21.15.43:0/1466027513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb3bbc0c30 0x55cb3bbc6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:18.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.316+0000 7ff951b90640 1 -- 172.21.15.43:0/1466027513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb3bbc0c30 msgr2=0x55cb3bbc6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:18.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.316+0000 7ff951b90640 1 --2- 172.21.15.43:0/1466027513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb3bbc0c30 0x55cb3bbc6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.316+0000 7ff951b90640 1 -- 172.21.15.43:0/1466027513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc9810 msgr2=0x55cb3bbc71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:18.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.316+0000 7ff951b90640 1 --2- 172.21.15.43:0/1466027513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc9810 0x55cb3bbc71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.316+0000 7ff951b90640 1 -- 172.21.15.43:0/1466027513 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb3b837920 con 0x55cb3bbc0310 2024-04-21T08:46:18.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.316+0000 7ff952391640 1 --2- 172.21.15.43:0/1466027513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc9810 0x55cb3bbc71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:18.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.316+0000 7ff951b90640 1 --2- 172.21.15.43:0/1466027513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3bbc0310 0x55cb3bbc6730 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff9480990b0 tx=0x7ff9480048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25ea2386c5dcc291 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:18.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.316+0000 7ff9357fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-21T08:46:18.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.316+0000 7ff9357fa640 1 -- 172.21.15.43:0/1466027513 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948005180 con 0x55cb3bbc0310 2024-04-21T08:46:18.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.316+0000 7ff9357fa640 1 -- 172.21.15.43:0/1466027513 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7ff9480a5430 con 0x55cb3bbc0310 2024-04-21T08:46:18.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.316+0000 7ff9569ed740 1 -- 172.21.15.43:0/1466027513 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff94809d350 con 0x55cb3bbc0310 2024-04-21T08:46:18.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.316+0000 7ff9357fa640 1 -- 172.21.15.43:0/1466027513 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff9480bd2d0 con 0x55cb3bbc0310 2024-04-21T08:46:18.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.316+0000 7ff9569ed740 10 client.27983 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:18.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.316+0000 7ff9569ed740 1 -- 172.21.15.43:0/1466027513 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cb3b903d30 con 0x55cb3bbc0310 2024-04-21T08:46:18.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.316+0000 7ff9357fa640 1 -- 172.21.15.43:0/1466027513 <== mon.2 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff94809d350 con 0x55cb3bbc0310 2024-04-21T08:46:18.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.317+0000 7ff9357fa640 1 -- 172.21.15.43:0/1466027513 <== mon.2 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff94809c230 con 0x55cb3bbc0310 2024-04-21T08:46:18.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.317+0000 7ff9569ed740 10 client.27983 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:18.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.317+0000 7ff9569ed740 10 client.27983 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-21T08:46:18.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.317+0000 7ff9569ed740 20 client.27983 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:18.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.317+0000 7ff9569ed740 1 --2- 172.21.15.43:0/1466027513 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cb3bbd9850 0x55cb3bbdbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:18.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.317+0000 7ff9569ed740 4 client.27983 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:18.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.317+0000 7ff9569ed740 1 -- 172.21.15.43:0/1466027513 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb3baf8a10 con 0x55cb3bbd9850 2024-04-21T08:46:18.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.317+0000 7ff95138f640 1 --2- 172.21.15.43:0/1466027513 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cb3bbd9850 0x55cb3bbdbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:18.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.317+0000 7ff95138f640 1 --2- 172.21.15.43:0/1466027513 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cb3bbd9850 0x55cb3bbdbc40 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2856fcf62c97adc9 server_cookie=35f21a0afb4fe2d0 in_seq=0 out_seq=0 2024-04-21T08:46:18.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.317+0000 7ff9357fa640 10 client.27983 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:18.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.318+0000 7ff9357fa640 1 -- 172.21.15.43:0/1466027513 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb3baf8a10 con 0x55cb3bbd9850 2024-04-21T08:46:18.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.318+0000 7ff9357fa640 10 client.27983 handle_command_reply: tid=0 2024-04-21T08:46:18.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.318+0000 7ff9569ed740 1 client.27983 shutdown 2024-04-21T08:46:18.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.318+0000 7ff9569ed740 1 -- 172.21.15.43:0/1466027513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3bbc0310 msgr2=0x55cb3bbc6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:18.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.318+0000 7ff9569ed740 1 --2- 172.21.15.43:0/1466027513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3bbc0310 0x55cb3bbc6730 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff9480990b0 tx=0x7ff9480048f0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.318+0000 7ff9569ed740 1 -- 172.21.15.43:0/1466027513 shutdown_connections 2024-04-21T08:46:18.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.318+0000 7ff9569ed740 1 --2- 172.21.15.43:0/1466027513 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cb3bbd9850 0x55cb3bbdbc40 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.318+0000 7ff9569ed740 1 --2- 172.21.15.43:0/1466027513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc9810 0x55cb3bbc71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.318+0000 7ff9569ed740 1 --2- 172.21.15.43:0/1466027513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb3bbc0c30 0x55cb3bbc6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.318+0000 7ff9569ed740 1 --2- 172.21.15.43:0/1466027513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3bbc0310 0x55cb3bbc6730 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.318+0000 7ff9357fa640 0 client.27983 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:18.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.318+0000 7ff9569ed740 1 -- 172.21.15.43:0/1466027513 >> 172.21.15.43:0/1466027513 conn(0x55cb3bbbaa40 msgr2=0x55cb3bbbc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:18.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.318+0000 7ff9569ed740 1 -- 172.21.15.43:0/1466027513 shutdown_connections 2024-04-21T08:46:18.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.318+0000 7ff9569ed740 1 -- 172.21.15.43:0/1466027513 wait complete. 2024-04-21T08:46:18.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.318+0000 7ff9569ed740 20 client.27983 trim_cache size 0 max 16384 2024-04-21T08:46:18.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.336+0000 7ff9569ed740 1 Processor -- start 2024-04-21T08:46:18.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.336+0000 7ff9569ed740 1 -- start start 2024-04-21T08:46:18.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.336+0000 7ff9569ed740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb3bbc0310 0x55cb3bbc8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:18.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.336+0000 7ff9569ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3bbc0c30 0x55cb3bbc6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:18.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.336+0000 7ff9569ed740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc9810 0x55cb3bbdec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:18.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.336+0000 7ff9569ed740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1af440 con 0x55cb3bbc9810 2024-04-21T08:46:18.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.336+0000 7ff9569ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1b0f80 con 0x55cb3bbc0c30 2024-04-21T08:46:18.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.336+0000 7ff9569ed740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1afb10 con 0x55cb3bbc0310 2024-04-21T08:46:18.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.336+0000 7ff952391640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc9810 0x55cb3bbdec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:18.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.336+0000 7ff952391640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc9810 0x55cb3bbdec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33564/0 (socket says 172.21.15.43:33564) 2024-04-21T08:46:18.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.336+0000 7ff952391640 1 -- 172.21.15.43:0/1596805762 learned_addr learned my addr 172.21.15.43:0/1596805762 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:18.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.336+0000 7ff95138f640 1 --2- 172.21.15.43:0/1596805762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3bbc0c30 0x55cb3bbc6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:18.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.336+0000 7ff951b90640 1 --2- 172.21.15.43:0/1596805762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb3bbc0310 0x55cb3bbc8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:18.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.336+0000 7ff952391640 1 -- 172.21.15.43:0/1596805762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb3bbc0310 msgr2=0x55cb3bbc8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:18.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.336+0000 7ff952391640 1 --2- 172.21.15.43:0/1596805762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb3bbc0310 0x55cb3bbc8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.336+0000 7ff952391640 1 -- 172.21.15.43:0/1596805762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3bbc0c30 msgr2=0x55cb3bbc6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:18.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.336+0000 7ff952391640 1 --2- 172.21.15.43:0/1596805762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3bbc0c30 0x55cb3bbc6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.336+0000 7ff952391640 1 -- 172.21.15.43:0/1596805762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb3bbc0a60 con 0x55cb3bbc9810 2024-04-21T08:46:18.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.336+0000 7ff95138f640 1 --2- 172.21.15.43:0/1596805762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3bbc0c30 0x55cb3bbc6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:18.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.336+0000 7ff952391640 1 --2- 172.21.15.43:0/1596805762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc9810 0x55cb3bbdec60 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7ff944004a90 tx=0x7ff944040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54d2118203e4438 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:18.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.337+0000 7ff9377fe640 1 -- 172.21.15.43:0/1596805762 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944039070 con 0x55cb3bbc9810 2024-04-21T08:46:18.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.337+0000 7ff9377fe640 1 -- 172.21.15.43:0/1596805762 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff94403a040 con 0x55cb3bbc9810 2024-04-21T08:46:18.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.337+0000 7ff9377fe640 1 -- 172.21.15.43:0/1596805762 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9440046f0 con 0x55cb3bbc9810 2024-04-21T08:46:18.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.337+0000 7ff9569ed740 1 -- 172.21.15.43:0/1596805762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc9810 msgr2=0x55cb3bbdec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:18.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.337+0000 7ff9569ed740 1 --2- 172.21.15.43:0/1596805762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc9810 0x55cb3bbdec60 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7ff944004a90 tx=0x7ff944040000 comp rx=0 tx=0).stop 2024-04-21T08:46:18.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.337+0000 7ff9569ed740 1 -- 172.21.15.43:0/1596805762 shutdown_connections 2024-04-21T08:46:18.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.337+0000 7ff9569ed740 1 --2- 172.21.15.43:0/1596805762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc9810 0x55cb3bbdec60 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.337+0000 7ff9569ed740 1 --2- 172.21.15.43:0/1596805762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3bbc0c30 0x55cb3bbc6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.337+0000 7ff9569ed740 1 --2- 172.21.15.43:0/1596805762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb3bbc0310 0x55cb3bbc8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.337+0000 7ff9569ed740 1 -- 172.21.15.43:0/1596805762 >> 172.21.15.43:0/1596805762 conn(0x55cb3bbbaa40 msgr2=0x55cb3bbc66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:18.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.337+0000 7ff9569ed740 1 -- 172.21.15.43:0/1596805762 shutdown_connections 2024-04-21T08:46:18.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.337+0000 7ff9569ed740 1 -- 172.21.15.43:0/1596805762 wait complete. 2024-04-21T08:46:18.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.337+0000 7ff9569ed740 1 -- start start 2024-04-21T08:46:18.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.338+0000 7ff9569ed740 1 Processor -- start 2024-04-21T08:46:18.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.338+0000 7ff9569ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3bbc0310 0x55cb3bbdc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:18.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.338+0000 7ff9569ed740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb3bbc0c30 0x55cb3bbdcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:18.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.338+0000 7ff9569ed740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc9810 0x55cb3bbdd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:18.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.338+0000 7ff9569ed740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1b08b0 con 0x55cb3bbc9810 2024-04-21T08:46:18.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.338+0000 7ff9569ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1b01e0 con 0x55cb3bbc0310 2024-04-21T08:46:18.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.338+0000 7ff9569ed740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1ad420 con 0x55cb3bbc0c30 2024-04-21T08:46:18.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.338+0000 7ff9569ed740 10 client.0 _ll_register_callbacks cb 0x7ff944037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:18.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.338+0000 7ff952391640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc9810 0x55cb3bbdd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:18.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.338+0000 7ff952391640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc9810 0x55cb3bbdd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33572/0 (socket says 172.21.15.43:33572) 2024-04-21T08:46:18.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.338+0000 7ff952391640 1 -- 172.21.15.43:0/1600441429 learned_addr learned my addr 172.21.15.43:0/1600441429 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:18.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.338+0000 7ff95138f640 1 --2- 172.21.15.43:0/1600441429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb3bbc0c30 0x55cb3bbdcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:18.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.338+0000 7ff951b90640 1 --2- 172.21.15.43:0/1600441429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3bbc0310 0x55cb3bbdc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:18.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.338+0000 7ff952391640 1 -- 172.21.15.43:0/1600441429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb3bbc0c30 msgr2=0x55cb3bbdcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:18.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.338+0000 7ff952391640 1 --2- 172.21.15.43:0/1600441429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb3bbc0c30 0x55cb3bbdcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.338+0000 7ff952391640 1 -- 172.21.15.43:0/1600441429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3bbc0310 msgr2=0x55cb3bbdc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:18.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.338+0000 7ff952391640 1 --2- 172.21.15.43:0/1600441429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3bbc0310 0x55cb3bbdc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.338+0000 7ff952391640 1 -- 172.21.15.43:0/1600441429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb3bc3ec30 con 0x55cb3bbc9810 2024-04-21T08:46:18.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.339+0000 7ff952391640 1 --2- 172.21.15.43:0/1600441429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc9810 0x55cb3bbdd320 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7ff94402c770 tx=0x7ff94402c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d6dc116ebc5dd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:18.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.339+0000 7ff9357fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:18.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.339+0000 7ff9357fa640 1 -- 172.21.15.43:0/1600441429 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944039040 con 0x55cb3bbc9810 2024-04-21T08:46:18.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.339+0000 7ff9357fa640 1 -- 172.21.15.43:0/1600441429 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7ff9440324c0 con 0x55cb3bbc9810 2024-04-21T08:46:18.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.339+0000 7ff9569ed740 1 -- 172.21.15.43:0/1600441429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff94403a040 con 0x55cb3bbc9810 2024-04-21T08:46:18.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.339+0000 7ff9357fa640 1 -- 172.21.15.43:0/1600441429 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff944063050 con 0x55cb3bbc9810 2024-04-21T08:46:18.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.339+0000 7ff9569ed740 10 client.19110 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:18.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.339+0000 7ff9569ed740 1 -- 172.21.15.43:0/1600441429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cb3bc3ff50 con 0x55cb3bbc9810 2024-04-21T08:46:18.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.339+0000 7ff9357fa640 1 -- 172.21.15.43:0/1600441429 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff94403a040 con 0x55cb3bbc9810 2024-04-21T08:46:18.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.339+0000 7ff9357fa640 1 -- 172.21.15.43:0/1600441429 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff944034810 con 0x55cb3bbc9810 2024-04-21T08:46:18.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.339+0000 7ff9569ed740 10 client.19110 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:18.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.339+0000 7ff9569ed740 10 client.19110 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-21T08:46:18.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.339+0000 7ff9569ed740 20 client.19110 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:18.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.339+0000 7ff9569ed740 1 --2- 172.21.15.43:0/1600441429 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cb3bc40630 0x55cb3bc42a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:18.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.339+0000 7ff9569ed740 4 client.19110 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c9e85759", "release": true, "await": true}] 2024-04-21T08:46:18.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.339+0000 7ff9569ed740 1 -- 172.21.15.43:0/1600441429 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c9e85759", "release": true, "await": true}) -- 0x55cb3bb32f20 con 0x55cb3bc40630 2024-04-21T08:46:18.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.340+0000 7ff951b90640 1 --2- 172.21.15.43:0/1600441429 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cb3bc40630 0x55cb3bc42a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:18.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.340+0000 7ff951b90640 1 --2- 172.21.15.43:0/1600441429 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cb3bc40630 0x55cb3bc42a50 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d545ec1bee0bff59 server_cookie=eb0f20a830192bcf in_seq=0 out_seq=0 2024-04-21T08:46:18.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.340+0000 7ff9357fa640 10 client.19110 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:18.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.342+0000 7ff9357fa640 1 -- 172.21.15.43:0/1600441429 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55cb3bb32f20 con 0x55cb3bc40630 2024-04-21T08:46:18.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.342+0000 7ff9357fa640 10 client.19110 handle_command_reply: tid=0 2024-04-21T08:46:18.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.342+0000 7ff9569ed740 1 client.19110 shutdown 2024-04-21T08:46:18.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.342+0000 7ff9569ed740 1 -- 172.21.15.43:0/1600441429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc9810 msgr2=0x55cb3bbdd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:18.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.342+0000 7ff9569ed740 1 --2- 172.21.15.43:0/1600441429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc9810 0x55cb3bbdd320 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7ff94402c770 tx=0x7ff94402c7a0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.343+0000 7ff9569ed740 1 -- 172.21.15.43:0/1600441429 shutdown_connections 2024-04-21T08:46:18.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.343+0000 7ff9569ed740 1 --2- 172.21.15.43:0/1600441429 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cb3bc40630 0x55cb3bc42a50 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.343+0000 7ff9569ed740 1 --2- 172.21.15.43:0/1600441429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3bbc9810 0x55cb3bbdd320 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.343+0000 7ff9569ed740 1 --2- 172.21.15.43:0/1600441429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb3bbc0c30 0x55cb3bbdcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.343+0000 7ff9357fa640 0 client.19110 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:18.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.343+0000 7ff9569ed740 1 --2- 172.21.15.43:0/1600441429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3bbc0310 0x55cb3bbdc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.343+0000 7ff9569ed740 1 -- 172.21.15.43:0/1600441429 >> 172.21.15.43:0/1600441429 conn(0x55cb3bbbaa40 msgr2=0x55cb3bbc6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:18.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.343+0000 7ff9569ed740 1 -- 172.21.15.43:0/1600441429 shutdown_connections 2024-04-21T08:46:18.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.343+0000 7ff9569ed740 1 -- 172.21.15.43:0/1600441429 wait complete. 2024-04-21T08:46:18.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.343+0000 7ff9569ed740 20 client.19110 trim_cache size 0 max 16384 2024-04-21T08:46:18.348 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:46:18.349 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 30, 2024-04-21T08:46:18.349 INFO:teuthology.orchestra.run.smithi043.stdout: "leader": 24527, 2024-04-21T08:46:18.349 INFO:teuthology.orchestra.run.smithi043.stdout: "set_version": 12, 2024-04-21T08:46:18.349 INFO:teuthology.orchestra.run.smithi043.stdout: "sets": { 2024-04-21T08:46:18.349 INFO:teuthology.orchestra.run.smithi043.stdout: "c9e85759": { 2024-04-21T08:46:18.349 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 12, 2024-04-21T08:46:18.349 INFO:teuthology.orchestra.run.smithi043.stdout: "age_ref": 0.0, 2024-04-21T08:46:18.349 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-04-21T08:46:18.349 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RELEASED", 2024-04-21T08:46:18.349 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-04-21T08:46:18.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:46:18.349 INFO:teuthology.orchestra.run.smithi043.stdout: "timeout": 30.0, 2024-04-21T08:46:18.349 INFO:teuthology.orchestra.run.smithi043.stdout: "expiration": 90.0, 2024-04-21T08:46:18.349 INFO:teuthology.orchestra.run.smithi043.stdout: "members": { 2024-04-21T08:46:18.350 INFO:teuthology.orchestra.run.smithi043.stdout: "file:/": { 2024-04-21T08:46:18.350 INFO:teuthology.orchestra.run.smithi043.stdout: "excluded": false, 2024-04-21T08:46:18.350 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-04-21T08:46:18.350 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RELEASED", 2024-04-21T08:46:18.350 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-04-21T08:46:18.350 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:46:18.350 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:46:18.350 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:46:18.350 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:46:18.350 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:46:18.350 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:46:18.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.343+0000 7ff953df4640 1 -- 172.21.15.43:0/3062633947 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff9380a3d20 msgr2=0x7ff9380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:18.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.343+0000 7ff953df4640 1 --2- 172.21.15.43:0/3062633947 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff9380a3d20 0x7ff9380a61e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff944033000 tx=0x7ff9440095c0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.343+0000 7ff953df4640 1 -- 172.21.15.43:0/3062633947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff94c06a820 msgr2=0x7ff94c1986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:18.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.343+0000 7ff953df4640 1 --2- 172.21.15.43:0/3062633947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff94c06a820 0x7ff94c1986f0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7ff94800e9f0 tx=0x7ff94800edf0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.343+0000 7ff953df4640 1 -- 172.21.15.43:0/3062633947 shutdown_connections 2024-04-21T08:46:18.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.343+0000 7ff953df4640 1 --2- 172.21.15.43:0/3062633947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff94c19cfb0 0x7ff94c19d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.343+0000 7ff953df4640 1 --2- 172.21.15.43:0/3062633947 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff9380a3d20 0x7ff9380a61e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.343+0000 7ff953df4640 1 --2- 172.21.15.43:0/3062633947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff94c107270 0x7ff94c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.343+0000 7ff953df4640 1 --2- 172.21.15.43:0/3062633947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff94c06a820 0x7ff94c1986f0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:18.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.343+0000 7ff953df4640 1 -- 172.21.15.43:0/3062633947 >> 172.21.15.43:0/3062633947 conn(0x7ff94c077eb0 msgr2=0x7ff94c0fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:18.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.343+0000 7ff953df4640 1 -- 172.21.15.43:0/3062633947 shutdown_connections 2024-04-21T08:46:18.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:18.343+0000 7ff953df4640 1 -- 172.21.15.43:0/3062633947 wait complete. 2024-04-21T08:46:18.354 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set_id: c9e85759, seconds elapsed: 31.0 2024-04-21T08:46:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:18 smithi112 ceph-mon[31085]: pgmap v226: 129 pgs: 129 active+clean; 1.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-21T08:46:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:18 smithi112 ceph-mon[31085]: osdmap e98: 12 total, 12 up, 12 in 2024-04-21T08:46:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:18 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.175:6836/229985237,v1:172.21.15.175:6837/229985237] up:boot 2024-04-21T08:46:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:18 smithi112 ceph-mon[31085]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-21T08:46:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:18 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:46:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:18 smithi112 ceph-mon[31085]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:46:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:18 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/4270511773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:18.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:18 smithi043 ceph-mon[27978]: pgmap v226: 129 pgs: 129 active+clean; 1.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-21T08:46:18.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:18 smithi043 ceph-mon[27978]: osdmap e98: 12 total, 12 up, 12 in 2024-04-21T08:46:18.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:18 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.175:6836/229985237,v1:172.21.15.175:6837/229985237] up:boot 2024-04-21T08:46:18.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:18 smithi043 ceph-mon[27978]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-21T08:46:18.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:18 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:46:18.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:18 smithi043 ceph-mon[27978]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:46:18.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:18 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/4270511773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:18.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:18 smithi175 ceph-mon[31920]: pgmap v226: 129 pgs: 129 active+clean; 1.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-21T08:46:18.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:18 smithi175 ceph-mon[31920]: osdmap e98: 12 total, 12 up, 12 in 2024-04-21T08:46:18.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:18 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.175:6836/229985237,v1:172.21.15.175:6837/229985237] up:boot 2024-04-21T08:46:18.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:18 smithi175 ceph-mon[31920]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-21T08:46:18.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:18 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:46:18.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:18 smithi175 ceph-mon[31920]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:46:18.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:18 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/4270511773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:20.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:20 smithi112 ceph-mon[31085]: pgmap v228: 129 pgs: 129 active+clean; 1.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 113 B/s wr, 12 op/s 2024-04-21T08:46:20.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:20 smithi043 ceph-mon[27978]: pgmap v228: 129 pgs: 129 active+clean; 1.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 113 B/s wr, 12 op/s 2024-04-21T08:46:20.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:20 smithi175 ceph-mon[31920]: pgmap v228: 129 pgs: 129 active+clean; 1.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 113 B/s wr, 12 op/s 2024-04-21T08:46:22.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:22 smithi112 ceph-mon[31085]: pgmap v229: 129 pgs: 129 active+clean; 2.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-21T08:46:22.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:22 smithi043 ceph-mon[27978]: pgmap v229: 129 pgs: 129 active+clean; 2.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-21T08:46:22.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:22 smithi175 ceph-mon[31920]: pgmap v229: 129 pgs: 129 active+clean; 2.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-21T08:46:24.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:24 smithi112 ceph-mon[31085]: pgmap v230: 129 pgs: 129 active+clean; 2.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-21T08:46:24.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:24 smithi043 ceph-mon[27978]: pgmap v230: 129 pgs: 129 active+clean; 2.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-21T08:46:24.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:24 smithi175 ceph-mon[31920]: pgmap v230: 129 pgs: 129 active+clean; 2.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-21T08:46:26.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:26 smithi112 ceph-mon[31085]: pgmap v231: 129 pgs: 129 active+clean; 2.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-21T08:46:26.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:26 smithi043 ceph-mon[27978]: pgmap v231: 129 pgs: 129 active+clean; 2.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-21T08:46:26.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:26 smithi175 ceph-mon[31920]: pgmap v231: 129 pgs: 129 active+clean; 2.0 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-21T08:46:28.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:28 smithi112 ceph-mon[31085]: pgmap v232: 129 pgs: 129 active+clean; 2.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-21T08:46:28.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:28 smithi043 ceph-mon[27978]: pgmap v232: 129 pgs: 129 active+clean; 2.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-21T08:46:28.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:28 smithi175 ceph-mon[31920]: pgmap v232: 129 pgs: 129 active+clean; 2.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-21T08:46:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.446+0000 7fcf04132640 1 -- 172.21.15.43:0/3694113504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcefc101aa0 msgr2=0x7fcefc105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.446+0000 7fcf04132640 1 --2- 172.21.15.43:0/3694113504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcefc101aa0 0x7fcefc105a90 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fceec009900 tx=0x7fceec02bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.446+0000 7fcf04132640 1 -- 172.21.15.43:0/3694113504 shutdown_connections 2024-04-21T08:46:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.446+0000 7fcf04132640 1 --2- 172.21.15.43:0/3694113504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcefc106170 0x7fcefc10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.446+0000 7fcf04132640 1 --2- 172.21.15.43:0/3694113504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcefc101aa0 0x7fcefc105a90 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.446+0000 7fcf04132640 1 --2- 172.21.15.43:0/3694113504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcefc1010f0 0x7fcefc1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.446+0000 7fcf04132640 1 -- 172.21.15.43:0/3694113504 >> 172.21.15.43:0/3694113504 conn(0x7fcefc077fd0 msgr2=0x7fcefc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.447+0000 7fcf04132640 1 -- 172.21.15.43:0/3694113504 shutdown_connections 2024-04-21T08:46:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.447+0000 7fcf04132640 1 -- 172.21.15.43:0/3694113504 wait complete. 2024-04-21T08:46:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.447+0000 7fcf04132640 1 Processor -- start 2024-04-21T08:46:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.447+0000 7fcf04132640 1 -- start start 2024-04-21T08:46:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.447+0000 7fcf04132640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcefc1010f0 0x7fcefc198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.447+0000 7fcf04132640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcefc106170 0x7fcefc198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.447+0000 7fcf04132640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcefc19cfe0 0x7fcefc19d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.447+0000 7fcf04132640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1110b0 con 0x7fcefc19cfe0 2024-04-21T08:46:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.447+0000 7fcf04132640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc110f30 con 0x7fcefc1010f0 2024-04-21T08:46:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.447+0000 7fcf04132640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc111230 con 0x7fcefc106170 2024-04-21T08:46:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.448+0000 7fcf026cf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcefc19cfe0 0x7fcefc19d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.448+0000 7fcf026cf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcefc19cfe0 0x7fcefc19d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41440/0 (socket says 172.21.15.43:41440) 2024-04-21T08:46:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.448+0000 7fcf026cf640 1 -- 172.21.15.43:0/318100745 learned_addr learned my addr 172.21.15.43:0/318100745 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.448+0000 7fcf016cd640 1 --2- 172.21.15.43:0/318100745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcefc106170 0x7fcefc198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.448+0000 7fcf01ece640 1 --2- 172.21.15.43:0/318100745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcefc1010f0 0x7fcefc198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.448+0000 7fcf026cf640 1 -- 172.21.15.43:0/318100745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcefc106170 msgr2=0x7fcefc198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.448+0000 7fcf026cf640 1 --2- 172.21.15.43:0/318100745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcefc106170 0x7fcefc198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.448+0000 7fcf026cf640 1 -- 172.21.15.43:0/318100745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcefc1010f0 msgr2=0x7fcefc198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.448+0000 7fcf026cf640 1 --2- 172.21.15.43:0/318100745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcefc1010f0 0x7fcefc198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.448+0000 7fcf026cf640 1 -- 172.21.15.43:0/318100745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcefc19dad0 con 0x7fcefc19cfe0 2024-04-21T08:46:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.448+0000 7fcf01ece640 1 --2- 172.21.15.43:0/318100745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcefc1010f0 0x7fcefc198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.448+0000 7fcf026cf640 1 --2- 172.21.15.43:0/318100745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcefc19cfe0 0x7fcefc19d490 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fcef400e970 tx=0x7fcef400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.448+0000 7fcef2ffd640 1 -- 172.21.15.43:0/318100745 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef400ccc0 con 0x7fcefc19cfe0 2024-04-21T08:46:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.448+0000 7fcef2ffd640 1 -- 172.21.15.43:0/318100745 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcef400ce60 con 0x7fcefc19cfe0 2024-04-21T08:46:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.448+0000 7fcf04132640 1 -- 172.21.15.43:0/318100745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcefc19ddc0 con 0x7fcefc19cfe0 2024-04-21T08:46:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.448+0000 7fcef2ffd640 1 -- 172.21.15.43:0/318100745 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4010430 con 0x7fcefc19cfe0 2024-04-21T08:46:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.448+0000 7fcf04132640 1 -- 172.21.15.43:0/318100745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcefc1a55d0 con 0x7fcefc19cfe0 2024-04-21T08:46:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.451+0000 7fcef2ffd640 1 -- 172.21.15.43:0/318100745 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcef4004f90 con 0x7fcefc19cfe0 2024-04-21T08:46:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.452+0000 7fcf06e03740 1 Processor -- start 2024-04-21T08:46:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.452+0000 7fcf06e03740 1 -- start start 2024-04-21T08:46:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.452+0000 7fcf06e03740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c278d0 0x55ba56c26620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.452+0000 7fcf06e03740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba56c26b60 0x55ba56c26fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.452+0000 7fcf06e03740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba56c300e0 0x55ba56c324d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.452+0000 7fcf06e03740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ba56b8b8d0 con 0x55ba56c278d0 2024-04-21T08:46:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.452+0000 7fcf06e03740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ba56b28e80 con 0x55ba56c300e0 2024-04-21T08:46:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.452+0000 7fcf06e03740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ba56b87890 con 0x55ba56c26b60 2024-04-21T08:46:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.452+0000 7fcf016cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c278d0 0x55ba56c26620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.452+0000 7fcf016cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c278d0 0x55ba56c26620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41446/0 (socket says 172.21.15.43:41446) 2024-04-21T08:46:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.452+0000 7fcf016cd640 1 -- 172.21.15.43:0/535507496 learned_addr learned my addr 172.21.15.43:0/535507496 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.452+0000 7fcef2ffd640 1 --2- 172.21.15.43:0/318100745 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcee80a3e40 0x7fcee80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.452+0000 7fcf016cd640 1 --2- 172.21.15.43:0/535507496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba56c300e0 0x55ba56c324d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.452+0000 7fcf01ece640 1 --2- 172.21.15.43:0/535507496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba56c26b60 0x55ba56c26fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.452+0000 7fcf026cf640 1 --2- 172.21.15.43:0/318100745 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcee80a3e40 0x7fcee80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.453+0000 7fcf016cd640 1 -- 172.21.15.43:0/535507496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba56c26b60 msgr2=0x55ba56c26fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.453+0000 7fcf016cd640 1 --2- 172.21.15.43:0/535507496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba56c26b60 0x55ba56c26fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.453+0000 7fcf016cd640 1 -- 172.21.15.43:0/535507496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba56c300e0 msgr2=0x55ba56c324d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.453+0000 7fcf016cd640 1 --2- 172.21.15.43:0/535507496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba56c300e0 0x55ba56c324d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.453+0000 7fcf016cd640 1 -- 172.21.15.43:0/535507496 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba56b4a320 con 0x55ba56c278d0 2024-04-21T08:46:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.453+0000 7fcef2ffd640 1 -- 172.21.15.43:0/318100745 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7fcef40d0f30 con 0x7fcefc19cfe0 2024-04-21T08:46:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.453+0000 7fcf016cd640 1 --2- 172.21.15.43:0/535507496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c278d0 0x55ba56c26620 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fcefc06b730 tx=0x7fceec005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92841d65f5989a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.453+0000 7fcf026cf640 1 --2- 172.21.15.43:0/318100745 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcee80a3e40 0x7fcee80a6300 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcef4098f60 tx=0x7fcef4087040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.453+0000 7fcee7fff640 1 -- 172.21.15.43:0/535507496 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec04a070 con 0x55ba56c278d0 2024-04-21T08:46:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.453+0000 7fcee7fff640 1 -- 172.21.15.43:0/535507496 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceec02cc90 con 0x55ba56c278d0 2024-04-21T08:46:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.453+0000 7fcee7fff640 1 -- 172.21.15.43:0/535507496 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec0046c0 con 0x55ba56c278d0 2024-04-21T08:46:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.454+0000 7fcf06e03740 1 -- 172.21.15.43:0/535507496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c278d0 msgr2=0x55ba56c26620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.454+0000 7fcf06e03740 1 --2- 172.21.15.43:0/535507496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c278d0 0x55ba56c26620 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fcefc06b730 tx=0x7fceec005e40 comp rx=0 tx=0).stop 2024-04-21T08:46:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.454+0000 7fcf06e03740 1 -- 172.21.15.43:0/535507496 shutdown_connections 2024-04-21T08:46:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.454+0000 7fcf06e03740 1 --2- 172.21.15.43:0/535507496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba56c300e0 0x55ba56c324d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.454+0000 7fcf06e03740 1 --2- 172.21.15.43:0/535507496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba56c26b60 0x55ba56c26fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.454+0000 7fcf06e03740 1 --2- 172.21.15.43:0/535507496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c278d0 0x55ba56c26620 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.454+0000 7fcf06e03740 1 -- 172.21.15.43:0/535507496 >> 172.21.15.43:0/535507496 conn(0x55ba56bedc90 msgr2=0x55ba56c21850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.454+0000 7fcf06e03740 1 -- 172.21.15.43:0/535507496 shutdown_connections 2024-04-21T08:46:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.454+0000 7fcf06e03740 1 -- 172.21.15.43:0/535507496 wait complete. 2024-04-21T08:46:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.454+0000 7fcf06e03740 1 -- start start 2024-04-21T08:46:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.454+0000 7fcf06e03740 1 Processor -- start 2024-04-21T08:46:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.455+0000 7fcf06e03740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba56c26b60 0x55ba56c2faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.455+0000 7fcf06e03740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba56c278d0 0x55ba56c2ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.455+0000 7fcf06e03740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c300e0 0x55ba56c2d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.455+0000 7fcf06e03740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ba56c32db0 con 0x55ba56c300e0 2024-04-21T08:46:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.455+0000 7fcf06e03740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ba56b13eb0 con 0x55ba56c278d0 2024-04-21T08:46:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.455+0000 7fcf06e03740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ba56c32f30 con 0x55ba56c26b60 2024-04-21T08:46:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.455+0000 7fcf06e03740 10 client.0 _ll_register_callbacks cb 0x55ba56b40b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.455+0000 7fcf016cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c300e0 0x55ba56c2d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.455+0000 7fcf016cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c300e0 0x55ba56c2d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41454/0 (socket says 172.21.15.43:41454) 2024-04-21T08:46:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.455+0000 7fcf016cd640 1 -- 172.21.15.43:0/2859276065 learned_addr learned my addr 172.21.15.43:0/2859276065 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.455+0000 7fcf01ece640 1 --2- 172.21.15.43:0/2859276065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba56c278d0 0x55ba56c2ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.455+0000 7fcf016cd640 1 --2- 172.21.15.43:0/2859276065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba56c26b60 0x55ba56c2faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.455+0000 7fcf016cd640 1 -- 172.21.15.43:0/2859276065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba56c26b60 msgr2=0x55ba56c2faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.455+0000 7fcf016cd640 1 --2- 172.21.15.43:0/2859276065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba56c26b60 0x55ba56c2faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.455+0000 7fcf016cd640 1 -- 172.21.15.43:0/2859276065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba56c278d0 msgr2=0x55ba56c2ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.455+0000 7fcf016cd640 1 --2- 172.21.15.43:0/2859276065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba56c278d0 0x55ba56c2ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.455+0000 7fcf016cd640 1 -- 172.21.15.43:0/2859276065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba5689e210 con 0x55ba56c300e0 2024-04-21T08:46:30.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.456+0000 7fcf016cd640 1 --2- 172.21.15.43:0/2859276065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c300e0 0x55ba56c2d490 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fceec048040 tx=0x7fceec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a58a60f9acfb30 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:30.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.456+0000 7fcee57fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:30.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.456+0000 7fcee57fa640 1 -- 172.21.15.43:0/2859276065 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec056060 con 0x55ba56c300e0 2024-04-21T08:46:30.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.456+0000 7fcee57fa640 1 -- 172.21.15.43:0/2859276065 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7fceec02c9e0 con 0x55ba56c300e0 2024-04-21T08:46:30.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.456+0000 7fcf06e03740 1 -- 172.21.15.43:0/2859276065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceec02cc90 con 0x55ba56c300e0 2024-04-21T08:46:30.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.456+0000 7fcee57fa640 1 -- 172.21.15.43:0/2859276065 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fceec05d050 con 0x55ba56c300e0 2024-04-21T08:46:30.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.456+0000 7fcf06e03740 10 client.19122 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:30.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.456+0000 7fcf06e03740 1 -- 172.21.15.43:0/2859276065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ba5696a620 con 0x55ba56c300e0 2024-04-21T08:46:30.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.456+0000 7fcee57fa640 1 -- 172.21.15.43:0/2859276065 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceec02cc90 con 0x55ba56c300e0 2024-04-21T08:46:30.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.456+0000 7fcee57fa640 1 -- 172.21.15.43:0/2859276065 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fceec050040 con 0x55ba56c300e0 2024-04-21T08:46:30.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.456+0000 7fcf06e03740 10 client.19122 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:30.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.456+0000 7fcf06e03740 10 client.19122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:30.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.456+0000 7fcf06e03740 20 client.19122 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:30.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.456+0000 7fcf06e03740 1 --2- 172.21.15.43:0/2859276065 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ba56c3ffa0 0x55ba56c42390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:30.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.456+0000 7fcf06e03740 4 client.19122 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:30.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.456+0000 7fcf06e03740 1 -- 172.21.15.43:0/2859276065 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba56b9f880 con 0x55ba56c3ffa0 2024-04-21T08:46:30.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.457+0000 7fcf01ece640 1 --2- 172.21.15.43:0/2859276065 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ba56c3ffa0 0x55ba56c42390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:30.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.457+0000 7fcee57fa640 10 client.19122 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:30.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.457+0000 7fcf01ece640 1 --2- 172.21.15.43:0/2859276065 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ba56c3ffa0 0x55ba56c42390 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8ddd6c6150ac5c3 server_cookie=827b74bce46e855f in_seq=0 out_seq=0 2024-04-21T08:46:30.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.458+0000 7fcee57fa640 1 -- 172.21.15.43:0/2859276065 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ba56b9f880 con 0x55ba56c3ffa0 2024-04-21T08:46:30.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.458+0000 7fcee57fa640 10 client.19122 handle_command_reply: tid=0 2024-04-21T08:46:30.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.458+0000 7fcf06e03740 1 client.19122 shutdown 2024-04-21T08:46:30.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.458+0000 7fcf06e03740 1 -- 172.21.15.43:0/2859276065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c300e0 msgr2=0x55ba56c2d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:30.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.458+0000 7fcf06e03740 1 --2- 172.21.15.43:0/2859276065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c300e0 0x55ba56c2d490 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fceec048040 tx=0x7fceec0042f0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.458+0000 7fcf06e03740 1 -- 172.21.15.43:0/2859276065 shutdown_connections 2024-04-21T08:46:30.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.458+0000 7fcf06e03740 1 --2- 172.21.15.43:0/2859276065 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ba56c3ffa0 0x55ba56c42390 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.458+0000 7fcf06e03740 1 --2- 172.21.15.43:0/2859276065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c300e0 0x55ba56c2d490 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.458+0000 7fcf06e03740 1 --2- 172.21.15.43:0/2859276065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba56c278d0 0x55ba56c2ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.458+0000 7fcf06e03740 1 --2- 172.21.15.43:0/2859276065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba56c26b60 0x55ba56c2faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.458+0000 7fcee57fa640 0 client.19122 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:30.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.458+0000 7fcf06e03740 1 -- 172.21.15.43:0/2859276065 >> 172.21.15.43:0/2859276065 conn(0x55ba56bedc90 msgr2=0x55ba56c235a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:30.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.458+0000 7fcf06e03740 1 -- 172.21.15.43:0/2859276065 shutdown_connections 2024-04-21T08:46:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.458+0000 7fcf06e03740 1 -- 172.21.15.43:0/2859276065 wait complete. 2024-04-21T08:46:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.458+0000 7fcf06e03740 20 client.19122 trim_cache size 0 max 16384 2024-04-21T08:46:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.473+0000 7fcf06e03740 1 Processor -- start 2024-04-21T08:46:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.473+0000 7fcf06e03740 1 -- start start 2024-04-21T08:46:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.473+0000 7fcf06e03740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c26b60 0x55ba56c2be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.473+0000 7fcf06e03740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba56c278d0 0x55ba56c2f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.473+0000 7fcf06e03740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba56c300e0 0x55ba56c2faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.473+0000 7fcf06e03740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1af3b0 con 0x55ba56c26b60 2024-04-21T08:46:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.473+0000 7fcf06e03740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1b0ef0 con 0x55ba56c278d0 2024-04-21T08:46:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.473+0000 7fcf06e03740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1afa80 con 0x55ba56c300e0 2024-04-21T08:46:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.473+0000 7fcf016cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c26b60 0x55ba56c2be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.474+0000 7fcf016cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c26b60 0x55ba56c2be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41468/0 (socket says 172.21.15.43:41468) 2024-04-21T08:46:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.474+0000 7fcf016cd640 1 -- 172.21.15.43:0/3924872388 learned_addr learned my addr 172.21.15.43:0/3924872388 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.474+0000 7fcf01ece640 1 --2- 172.21.15.43:0/3924872388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba56c278d0 0x55ba56c2f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.474+0000 7fcf016cd640 1 -- 172.21.15.43:0/3924872388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba56c300e0 msgr2=0x55ba56c2faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.474+0000 7fcf016cd640 1 --2- 172.21.15.43:0/3924872388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba56c300e0 0x55ba56c2faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.474+0000 7fcf016cd640 1 -- 172.21.15.43:0/3924872388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba56c278d0 msgr2=0x55ba56c2f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.474+0000 7fcf016cd640 1 --2- 172.21.15.43:0/3924872388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba56c278d0 0x55ba56c2f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.474+0000 7fcf016cd640 1 -- 172.21.15.43:0/3924872388 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba56c25e30 con 0x55ba56c26b60 2024-04-21T08:46:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.474+0000 7fcf016cd640 1 --2- 172.21.15.43:0/3924872388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c26b60 0x55ba56c2be80 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fceec004a90 tx=0x7fceec0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da319e647668a6da server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.474+0000 7fcee77fe640 1 -- 172.21.15.43:0/3924872388 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec056070 con 0x55ba56c26b60 2024-04-21T08:46:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.474+0000 7fcee77fe640 1 -- 172.21.15.43:0/3924872388 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceec047030 con 0x55ba56c26b60 2024-04-21T08:46:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.474+0000 7fcee77fe640 1 -- 172.21.15.43:0/3924872388 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec04a040 con 0x55ba56c26b60 2024-04-21T08:46:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.475+0000 7fcf06e03740 1 -- 172.21.15.43:0/3924872388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c26b60 msgr2=0x55ba56c2be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.475+0000 7fcf06e03740 1 --2- 172.21.15.43:0/3924872388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c26b60 0x55ba56c2be80 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fceec004a90 tx=0x7fceec0095c0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.475+0000 7fcf06e03740 1 -- 172.21.15.43:0/3924872388 shutdown_connections 2024-04-21T08:46:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.475+0000 7fcf06e03740 1 --2- 172.21.15.43:0/3924872388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba56c300e0 0x55ba56c2faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.475+0000 7fcf06e03740 1 --2- 172.21.15.43:0/3924872388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba56c278d0 0x55ba56c2f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.475+0000 7fcf06e03740 1 --2- 172.21.15.43:0/3924872388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c26b60 0x55ba56c2be80 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.475+0000 7fcf06e03740 1 -- 172.21.15.43:0/3924872388 >> 172.21.15.43:0/3924872388 conn(0x55ba56bedc90 msgr2=0x55ba56c2b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.475+0000 7fcf06e03740 1 -- 172.21.15.43:0/3924872388 shutdown_connections 2024-04-21T08:46:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.475+0000 7fcf06e03740 1 -- 172.21.15.43:0/3924872388 wait complete. 2024-04-21T08:46:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.475+0000 7fcf06e03740 1 -- start start 2024-04-21T08:46:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.475+0000 7fcf06e03740 1 Processor -- start 2024-04-21T08:46:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.476+0000 7fcf06e03740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c26b60 0x55ba56c42f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.476+0000 7fcf06e03740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba56c278d0 0x55ba56c43450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.476+0000 7fcf06e03740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba56c300e0 0x55ba56c43990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.476+0000 7fcf06e03740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1b0820 con 0x55ba56c26b60 2024-04-21T08:46:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.476+0000 7fcf06e03740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1b0150 con 0x55ba56c278d0 2024-04-21T08:46:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.476+0000 7fcf06e03740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1ad390 con 0x55ba56c300e0 2024-04-21T08:46:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.476+0000 7fcf06e03740 10 client.0 _ll_register_callbacks cb 0x7fceec048070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.476+0000 7fcf016cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c26b60 0x55ba56c42f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.476+0000 7fcf016cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c26b60 0x55ba56c42f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41484/0 (socket says 172.21.15.43:41484) 2024-04-21T08:46:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.476+0000 7fcf016cd640 1 -- 172.21.15.43:0/2666886073 learned_addr learned my addr 172.21.15.43:0/2666886073 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.476+0000 7fcf01ece640 1 --2- 172.21.15.43:0/2666886073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba56c278d0 0x55ba56c43450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.476+0000 7fcf016cd640 1 -- 172.21.15.43:0/2666886073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba56c300e0 msgr2=0x55ba56c43990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.476+0000 7fcf016cd640 1 --2- 172.21.15.43:0/2666886073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba56c300e0 0x55ba56c43990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.476+0000 7fcf016cd640 1 -- 172.21.15.43:0/2666886073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba56c278d0 msgr2=0x55ba56c43450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.476+0000 7fcf016cd640 1 --2- 172.21.15.43:0/2666886073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba56c278d0 0x55ba56c43450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.476+0000 7fcf016cd640 1 -- 172.21.15.43:0/2666886073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba56ca9690 con 0x55ba56c26b60 2024-04-21T08:46:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.476+0000 7fcf01ece640 1 --2- 172.21.15.43:0/2666886073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba56c278d0 0x55ba56c43450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.476+0000 7fcf016cd640 1 --2- 172.21.15.43:0/2666886073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c26b60 0x55ba56c42f10 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fceec009900 tx=0x7fceec005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e92355567a6866 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.476+0000 7fcee57fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.476+0000 7fcee57fa640 1 -- 172.21.15.43:0/2666886073 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec066070 con 0x55ba56c26b60 2024-04-21T08:46:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.476+0000 7fcee57fa640 1 -- 172.21.15.43:0/2666886073 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7fceec005190 con 0x55ba56c26b60 2024-04-21T08:46:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.476+0000 7fcf06e03740 1 -- 172.21.15.43:0/2666886073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceec047030 con 0x55ba56c26b60 2024-04-21T08:46:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.477+0000 7fcee57fa640 1 -- 172.21.15.43:0/2666886073 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fceec072050 con 0x55ba56c26b60 2024-04-21T08:46:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.477+0000 7fcf06e03740 10 client.19128 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.477+0000 7fcf06e03740 1 -- 172.21.15.43:0/2666886073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ba56caa9a0 con 0x55ba56c26b60 2024-04-21T08:46:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.477+0000 7fcee57fa640 1 -- 172.21.15.43:0/2666886073 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceec047030 con 0x55ba56c26b60 2024-04-21T08:46:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.477+0000 7fcee57fa640 1 -- 172.21.15.43:0/2666886073 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fceec003940 con 0x55ba56c26b60 2024-04-21T08:46:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.477+0000 7fcf06e03740 10 client.19128 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.477+0000 7fcf06e03740 10 client.19128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.477+0000 7fcf06e03740 20 client.19128 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.477+0000 7fcf06e03740 1 --2- 172.21.15.43:0/2666886073 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ba56caafa0 0x55ba56cad3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.477+0000 7fcf06e03740 4 client.19128 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:46:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.477+0000 7fcf06e03740 1 -- 172.21.15.43:0/2666886073 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba56b2b3d0 con 0x55ba56caafa0 2024-04-21T08:46:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.477+0000 7fcf01ece640 1 --2- 172.21.15.43:0/2666886073 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ba56caafa0 0x55ba56cad3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.478+0000 7fcf01ece640 1 --2- 172.21.15.43:0/2666886073 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ba56caafa0 0x55ba56cad3c0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49110f15d9a7ae42 server_cookie=e0d075e1a898306c in_seq=0 out_seq=0 2024-04-21T08:46:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.478+0000 7fcee57fa640 10 client.19128 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.488+0000 7fcee57fa640 1 -- 172.21.15.43:0/2666886073 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ba56b2b3d0 con 0x55ba56caafa0 2024-04-21T08:46:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.488+0000 7fcee57fa640 10 client.19128 handle_command_reply: tid=0 2024-04-21T08:46:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.488+0000 7fcf06e03740 1 client.19128 shutdown 2024-04-21T08:46:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.489+0000 7fcf06e03740 1 -- 172.21.15.43:0/2666886073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c26b60 msgr2=0x55ba56c42f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.489+0000 7fcf06e03740 1 --2- 172.21.15.43:0/2666886073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c26b60 0x55ba56c42f10 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fceec009900 tx=0x7fceec005900 comp rx=0 tx=0).stop 2024-04-21T08:46:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.489+0000 7fcf06e03740 1 -- 172.21.15.43:0/2666886073 shutdown_connections 2024-04-21T08:46:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.489+0000 7fcf06e03740 1 --2- 172.21.15.43:0/2666886073 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ba56caafa0 0x55ba56cad3c0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.489+0000 7fcf06e03740 1 --2- 172.21.15.43:0/2666886073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ba56c300e0 0x55ba56c43990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.489+0000 7fcf06e03740 1 --2- 172.21.15.43:0/2666886073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ba56c278d0 0x55ba56c43450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.489+0000 7fcf06e03740 1 --2- 172.21.15.43:0/2666886073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba56c26b60 0x55ba56c42f10 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.489+0000 7fcf06e03740 1 -- 172.21.15.43:0/2666886073 >> 172.21.15.43:0/2666886073 conn(0x55ba56bedc90 msgr2=0x55ba56c22bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.489+0000 7fcee57fa640 0 client.19128 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.489+0000 7fcf06e03740 1 -- 172.21.15.43:0/2666886073 shutdown_connections 2024-04-21T08:46:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.489+0000 7fcf06e03740 1 -- 172.21.15.43:0/2666886073 wait complete. 2024-04-21T08:46:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.489+0000 7fcf06e03740 20 client.19128 trim_cache size 0 max 16384 2024-04-21T08:46:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:46:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:46:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:46:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:46:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.489+0000 7fcf04132640 1 -- 172.21.15.43:0/318100745 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcee80a3e40 msgr2=0x7fcee80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.489+0000 7fcf04132640 1 --2- 172.21.15.43:0/318100745 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcee80a3e40 0x7fcee80a6300 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcef4098f60 tx=0x7fcef4087040 comp rx=0 tx=0).stop 2024-04-21T08:46:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.489+0000 7fcf04132640 1 -- 172.21.15.43:0/318100745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcefc19cfe0 msgr2=0x7fcefc19d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.489+0000 7fcf04132640 1 --2- 172.21.15.43:0/318100745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcefc19cfe0 0x7fcefc19d490 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fcef400e970 tx=0x7fcef400ed70 comp rx=0 tx=0).stop 2024-04-21T08:46:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.489+0000 7fcf04132640 1 -- 172.21.15.43:0/318100745 shutdown_connections 2024-04-21T08:46:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.489+0000 7fcf04132640 1 --2- 172.21.15.43:0/318100745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcefc19cfe0 0x7fcefc19d490 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.489+0000 7fcf04132640 1 --2- 172.21.15.43:0/318100745 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcee80a3e40 0x7fcee80a6300 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.489+0000 7fcf04132640 1 --2- 172.21.15.43:0/318100745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcefc106170 0x7fcefc198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.490+0000 7fcf04132640 1 --2- 172.21.15.43:0/318100745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcefc1010f0 0x7fcefc198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.490+0000 7fcf04132640 1 -- 172.21.15.43:0/318100745 >> 172.21.15.43:0/318100745 conn(0x7fcefc077fd0 msgr2=0x7fcefc0699a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.490+0000 7fcf04132640 1 -- 172.21.15.43:0/318100745 shutdown_connections 2024-04-21T08:46:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:30.490+0000 7fcf04132640 1 -- 172.21.15.43:0/318100745 wait complete. 2024-04-21T08:46:30.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:30 smithi112 ceph-mon[31085]: pgmap v233: 129 pgs: 129 active+clean; 2.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-04-21T08:46:30.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:30 smithi043 ceph-mon[27978]: pgmap v233: 129 pgs: 129 active+clean; 2.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-04-21T08:46:30.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:30 smithi175 ceph-mon[31920]: pgmap v233: 129 pgs: 129 active+clean; 2.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-04-21T08:46:32.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:32 smithi043 ceph-mon[27978]: pgmap v234: 129 pgs: 129 active+clean; 2.5 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 52 KiB/s wr, 21 op/s 2024-04-21T08:46:32.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:32 smithi175 ceph-mon[31920]: pgmap v234: 129 pgs: 129 active+clean; 2.5 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 52 KiB/s wr, 21 op/s 2024-04-21T08:46:32.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:32 smithi112 ceph-mon[31085]: pgmap v234: 129 pgs: 129 active+clean; 2.5 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 52 KiB/s wr, 21 op/s 2024-04-21T08:46:34.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:34 smithi043 ceph-mon[27978]: pgmap v235: 129 pgs: 129 active+clean; 2.5 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:46:34.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:34 smithi175 ceph-mon[31920]: pgmap v235: 129 pgs: 129 active+clean; 2.5 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:46:34.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:34 smithi112 ceph-mon[31085]: pgmap v235: 129 pgs: 129 active+clean; 2.5 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:46:35.478 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.479 INFO:tasks.workunit.client.0.smithi043.stdout:251/600: open fd = 3 2024-04-21T08:46:35.479 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.479 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.479 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.479 INFO:tasks.workunit.client.0.smithi043.stdout:252/600: open fd = 3 2024-04-21T08:46:35.479 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.479 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.479 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.479 INFO:tasks.workunit.client.0.smithi043.stdout:253/600: open fd = 3 2024-04-21T08:46:35.479 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.479 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.479 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.479 INFO:tasks.workunit.client.0.smithi043.stdout:254/600: open fd = 3 2024-04-21T08:46:35.479 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.479 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.480 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.480 INFO:tasks.workunit.client.0.smithi043.stdout:255/600: open fd = 3 2024-04-21T08:46:35.480 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.480 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.480 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.480 INFO:tasks.workunit.client.0.smithi043.stdout:256/600: open fd = 3 2024-04-21T08:46:35.480 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.480 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.480 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.480 INFO:tasks.workunit.client.0.smithi043.stdout:257/600: open fd = 3 2024-04-21T08:46:35.480 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.480 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.480 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.480 INFO:tasks.workunit.client.0.smithi043.stdout:258/600: open fd = 3 2024-04-21T08:46:35.481 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.481 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.481 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.481 INFO:tasks.workunit.client.0.smithi043.stdout:259/600: open fd = 3 2024-04-21T08:46:35.481 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.481 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.481 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.481 INFO:tasks.workunit.client.0.smithi043.stdout:260/600: open fd = 3 2024-04-21T08:46:35.481 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.481 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.481 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.481 INFO:tasks.workunit.client.0.smithi043.stdout:261/600: open fd = 3 2024-04-21T08:46:35.481 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.481 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.481 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.482 INFO:tasks.workunit.client.0.smithi043.stdout:262/600: open fd = 3 2024-04-21T08:46:35.482 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.482 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.482 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.482 INFO:tasks.workunit.client.0.smithi043.stdout:263/600: open fd = 3 2024-04-21T08:46:35.482 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.482 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.482 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.482 INFO:tasks.workunit.client.0.smithi043.stdout:264/600: open fd = 3 2024-04-21T08:46:35.482 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.482 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.482 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.482 INFO:tasks.workunit.client.0.smithi043.stdout:265/600: open fd = 3 2024-04-21T08:46:35.482 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.483 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.483 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.483 INFO:tasks.workunit.client.0.smithi043.stdout:266/600: open fd = 3 2024-04-21T08:46:35.483 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.483 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.483 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.483 INFO:tasks.workunit.client.0.smithi043.stdout:267/600: open fd = 3 2024-04-21T08:46:35.483 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.483 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.483 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.483 INFO:tasks.workunit.client.0.smithi043.stdout:268/600: open fd = 3 2024-04-21T08:46:35.483 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.483 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.484 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.484 INFO:tasks.workunit.client.0.smithi043.stdout:269/600: open fd = 3 2024-04-21T08:46:35.484 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.484 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.484 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.484 INFO:tasks.workunit.client.0.smithi043.stdout:270/600: open fd = 3 2024-04-21T08:46:35.484 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.484 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.484 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.484 INFO:tasks.workunit.client.0.smithi043.stdout:271/600: open fd = 3 2024-04-21T08:46:35.484 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.484 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.484 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.484 INFO:tasks.workunit.client.0.smithi043.stdout:272/600: open fd = 3 2024-04-21T08:46:35.485 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.485 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.485 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.485 INFO:tasks.workunit.client.0.smithi043.stdout:273/600: open fd = 3 2024-04-21T08:46:35.485 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.485 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.485 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.485 INFO:tasks.workunit.client.0.smithi043.stdout:274/600: open fd = 3 2024-04-21T08:46:35.485 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.485 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.485 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.485 INFO:tasks.workunit.client.0.smithi043.stdout:275/600: open fd = 3 2024-04-21T08:46:35.485 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.485 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.485 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.486 INFO:tasks.workunit.client.0.smithi043.stdout:276/600: open fd = 3 2024-04-21T08:46:35.486 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.486 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.486 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.486 INFO:tasks.workunit.client.0.smithi043.stdout:277/600: open fd = 3 2024-04-21T08:46:35.486 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.486 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.486 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.486 INFO:tasks.workunit.client.0.smithi043.stdout:278/600: open fd = 3 2024-04-21T08:46:35.486 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.486 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.486 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.486 INFO:tasks.workunit.client.0.smithi043.stdout:279/600: open fd = 3 2024-04-21T08:46:35.486 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.486 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.487 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.487 INFO:tasks.workunit.client.0.smithi043.stdout:280/600: open fd = 3 2024-04-21T08:46:35.487 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.487 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.487 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.487 INFO:tasks.workunit.client.0.smithi043.stdout:281/600: open fd = 3 2024-04-21T08:46:35.487 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.487 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.487 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.487 INFO:tasks.workunit.client.0.smithi043.stdout:282/600: open fd = 3 2024-04-21T08:46:35.487 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.487 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.487 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.487 INFO:tasks.workunit.client.0.smithi043.stdout:283/600: open fd = 3 2024-04-21T08:46:35.487 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.488 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.488 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.488 INFO:tasks.workunit.client.0.smithi043.stdout:284/600: open fd = 3 2024-04-21T08:46:35.488 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.488 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.488 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.488 INFO:tasks.workunit.client.0.smithi043.stdout:285/600: open fd = 3 2024-04-21T08:46:35.488 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.488 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.488 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.488 INFO:tasks.workunit.client.0.smithi043.stdout:286/600: open fd = 3 2024-04-21T08:46:35.488 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.488 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.488 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.489 INFO:tasks.workunit.client.0.smithi043.stdout:287/600: open fd = 3 2024-04-21T08:46:35.489 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.489 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.489 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.489 INFO:tasks.workunit.client.0.smithi043.stdout:288/600: open fd = 3 2024-04-21T08:46:35.489 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.489 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.489 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.489 INFO:tasks.workunit.client.0.smithi043.stdout:289/600: open fd = 3 2024-04-21T08:46:35.489 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.489 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.489 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.489 INFO:tasks.workunit.client.0.smithi043.stdout:290/600: open fd = 3 2024-04-21T08:46:35.489 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.489 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.489 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.490 INFO:tasks.workunit.client.0.smithi043.stdout:291/600: open fd = 3 2024-04-21T08:46:35.490 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.490 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.490 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.490 INFO:tasks.workunit.client.0.smithi043.stdout:292/600: open fd = 3 2024-04-21T08:46:35.490 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.490 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.490 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.490 INFO:tasks.workunit.client.0.smithi043.stdout:293/600: open fd = 3 2024-04-21T08:46:35.490 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.490 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.490 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.490 INFO:tasks.workunit.client.0.smithi043.stdout:294/600: open fd = 3 2024-04-21T08:46:35.490 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.491 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.491 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.491 INFO:tasks.workunit.client.0.smithi043.stdout:295/600: open fd = 3 2024-04-21T08:46:35.491 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.491 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.491 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.491 INFO:tasks.workunit.client.0.smithi043.stdout:296/600: open fd = 3 2024-04-21T08:46:35.491 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.491 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.491 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.491 INFO:tasks.workunit.client.0.smithi043.stdout:297/600: open fd = 3 2024-04-21T08:46:35.491 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.491 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.491 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.491 INFO:tasks.workunit.client.0.smithi043.stdout:298/600: open fd = 3 2024-04-21T08:46:35.492 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.492 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.492 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.492 INFO:tasks.workunit.client.0.smithi043.stdout:299/600: open fd = 3 2024-04-21T08:46:35.492 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.492 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.492 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.492 INFO:tasks.workunit.client.0.smithi043.stdout:300/600: open fd = 3 2024-04-21T08:46:35.492 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.492 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.492 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.492 INFO:tasks.workunit.client.0.smithi043.stdout:301/600: open fd = 3 2024-04-21T08:46:35.492 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.492 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.492 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.492 INFO:tasks.workunit.client.0.smithi043.stdout:302/600: open fd = 3 2024-04-21T08:46:35.493 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.493 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.493 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.493 INFO:tasks.workunit.client.0.smithi043.stdout:303/600: open fd = 3 2024-04-21T08:46:35.493 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.493 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.493 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.493 INFO:tasks.workunit.client.0.smithi043.stdout:304/600: open fd = 3 2024-04-21T08:46:35.493 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.493 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.493 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.493 INFO:tasks.workunit.client.0.smithi043.stdout:305/600: open fd = 3 2024-04-21T08:46:35.493 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.493 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.493 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.494 INFO:tasks.workunit.client.0.smithi043.stdout:306/600: open fd = 3 2024-04-21T08:46:35.494 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.494 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.494 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.494 INFO:tasks.workunit.client.0.smithi043.stdout:307/600: open fd = 3 2024-04-21T08:46:35.494 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.494 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.494 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.494 INFO:tasks.workunit.client.0.smithi043.stdout:308/600: open fd = 3 2024-04-21T08:46:35.494 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.494 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.494 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.494 INFO:tasks.workunit.client.0.smithi043.stdout:309/600: open fd = 3 2024-04-21T08:46:35.494 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.495 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.495 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.495 INFO:tasks.workunit.client.0.smithi043.stdout:310/600: open fd = 3 2024-04-21T08:46:35.495 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.495 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.495 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.495 INFO:tasks.workunit.client.0.smithi043.stdout:311/600: open fd = 3 2024-04-21T08:46:35.495 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.495 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.495 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:46:35.495 INFO:tasks.workunit.client.0.smithi043.stdout:312/600: open fd = 3 2024-04-21T08:46:35.495 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:35.495 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:46:36.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:36 smithi043 ceph-mon[27978]: pgmap v236: 129 pgs: 129 active+clean; 2.5 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:46:36.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:36 smithi175 ceph-mon[31920]: pgmap v236: 129 pgs: 129 active+clean; 2.5 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:46:36.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:36 smithi112 ceph-mon[31085]: pgmap v236: 129 pgs: 129 active+clean; 2.5 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:46:38.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:38 smithi043 ceph-mon[27978]: pgmap v237: 129 pgs: 129 active+clean; 2.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-21T08:46:38.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:38 smithi175 ceph-mon[31920]: pgmap v237: 129 pgs: 129 active+clean; 2.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-21T08:46:38.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:38 smithi112 ceph-mon[31085]: pgmap v237: 129 pgs: 129 active+clean; 2.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-21T08:46:40.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:40 smithi043 ceph-mon[27978]: pgmap v238: 129 pgs: 129 active+clean; 2.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:46:40.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:40 smithi175 ceph-mon[31920]: pgmap v238: 129 pgs: 129 active+clean; 2.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:46:40.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:40 smithi112 ceph-mon[31085]: pgmap v238: 129 pgs: 129 active+clean; 2.6 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:46:41.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:46:41.608 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:46:41.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.727+0000 7fc9417fd640 1 -- 172.21.15.43:0/3084235849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc93c06a5b0 msgr2=0x7fc93c06aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:41.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.727+0000 7fc9417fd640 1 --2- 172.21.15.43:0/3084235849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc93c06a5b0 0x7fc93c06aa30 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fc924009900 tx=0x7fc92402bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:41.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.727+0000 7fc9417fd640 1 -- 172.21.15.43:0/3084235849 shutdown_connections 2024-04-21T08:46:41.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.727+0000 7fc9417fd640 1 --2- 172.21.15.43:0/3084235849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc93c06af70 0x7fc93c10edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.727+0000 7fc9417fd640 1 --2- 172.21.15.43:0/3084235849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc93c06a5b0 0x7fc93c06aa30 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.727+0000 7fc9417fd640 1 --2- 172.21.15.43:0/3084235849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc93c106f20 0x7fc93c107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.727+0000 7fc9417fd640 1 -- 172.21.15.43:0/3084235849 >> 172.21.15.43:0/3084235849 conn(0x7fc93c077c40 msgr2=0x7fc93c078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:41.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.727+0000 7fc9417fd640 1 -- 172.21.15.43:0/3084235849 shutdown_connections 2024-04-21T08:46:41.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.727+0000 7fc9417fd640 1 -- 172.21.15.43:0/3084235849 wait complete. 2024-04-21T08:46:41.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.727+0000 7fc9417fd640 1 Processor -- start 2024-04-21T08:46:41.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.727+0000 7fc9417fd640 1 -- start start 2024-04-21T08:46:41.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.728+0000 7fc9417fd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc93c06a5b0 0x7fc93c101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:41.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.728+0000 7fc9417fd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc93c06af70 0x7fc93c1016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:41.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.728+0000 7fc9417fd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc93c106f20 0x7fc93c101bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:41.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.728+0000 7fc9417fd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c112190 con 0x7fc93c06af70 2024-04-21T08:46:41.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.728+0000 7fc9417fd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c112010 con 0x7fc93c06a5b0 2024-04-21T08:46:41.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.728+0000 7fc9417fd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c112310 con 0x7fc93c106f20 2024-04-21T08:46:41.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.728+0000 7fc933fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc93c06af70 0x7fc93c1016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:41.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.728+0000 7fc933fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc93c06af70 0x7fc93c1016b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54096/0 (socket says 172.21.15.43:54096) 2024-04-21T08:46:41.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.728+0000 7fc933fff640 1 -- 172.21.15.43:0/305588819 learned_addr learned my addr 172.21.15.43:0/305588819 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:41.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.728+0000 7fc93affd640 1 --2- 172.21.15.43:0/305588819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc93c06a5b0 0x7fc93c101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:41.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.728+0000 7fc93b7fe640 1 --2- 172.21.15.43:0/305588819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc93c106f20 0x7fc93c101bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:41.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.728+0000 7fc933fff640 1 -- 172.21.15.43:0/305588819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc93c106f20 msgr2=0x7fc93c101bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:41.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.728+0000 7fc933fff640 1 --2- 172.21.15.43:0/305588819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc93c106f20 0x7fc93c101bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.728+0000 7fc933fff640 1 -- 172.21.15.43:0/305588819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc93c06a5b0 msgr2=0x7fc93c101170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:41.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.728+0000 7fc933fff640 1 --2- 172.21.15.43:0/305588819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc93c06a5b0 0x7fc93c101170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.728+0000 7fc933fff640 1 -- 172.21.15.43:0/305588819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93c1a6b70 con 0x7fc93c06af70 2024-04-21T08:46:41.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.728+0000 7fc93affd640 1 --2- 172.21.15.43:0/305588819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc93c06a5b0 0x7fc93c101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:41.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.728+0000 7fc93b7fe640 1 --2- 172.21.15.43:0/305588819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc93c106f20 0x7fc93c101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:41.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.728+0000 7fc933fff640 1 --2- 172.21.15.43:0/305588819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc93c06af70 0x7fc93c1016b0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fc924006010 tx=0x7fc924004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:41.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.728+0000 7fc938ff9640 1 -- 172.21.15.43:0/305588819 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924043020 con 0x7fc93c06af70 2024-04-21T08:46:41.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.728+0000 7fc938ff9640 1 -- 172.21.15.43:0/305588819 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc924004440 con 0x7fc93c06af70 2024-04-21T08:46:41.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.729+0000 7fc9417fd640 1 -- 172.21.15.43:0/305588819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc93c1a6e00 con 0x7fc93c06af70 2024-04-21T08:46:41.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.729+0000 7fc938ff9640 1 -- 172.21.15.43:0/305588819 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924003b50 con 0x7fc93c06af70 2024-04-21T08:46:41.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.729+0000 7fc9417fd640 1 -- 172.21.15.43:0/305588819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc93c1a72c0 con 0x7fc93c06af70 2024-04-21T08:46:41.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.731+0000 7fc938ff9640 1 -- 172.21.15.43:0/305588819 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc924046020 con 0x7fc93c06af70 2024-04-21T08:46:41.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.732+0000 7fc9443df740 1 Processor -- start 2024-04-21T08:46:41.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.732+0000 7fc9443df740 1 -- start start 2024-04-21T08:46:41.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.732+0000 7fc9443df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f39addd580 0x55f39addd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:41.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.732+0000 7fc9443df740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39adddea0 0x55f39ade6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:41.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.732+0000 7fc9443df740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f39ade6a80 0x55f39ade8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:41.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.732+0000 7fc9443df740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f39ad421e0 con 0x55f39adddea0 2024-04-21T08:46:41.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.732+0000 7fc9443df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f39acdf390 con 0x55f39addd580 2024-04-21T08:46:41.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.732+0000 7fc9443df740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f39ad3e1a0 con 0x55f39ade6a80 2024-04-21T08:46:41.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.732+0000 7fc93affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39adddea0 0x55f39ade6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:41.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.732+0000 7fc93affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39adddea0 0x55f39ade6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54110/0 (socket says 172.21.15.43:54110) 2024-04-21T08:46:41.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.732+0000 7fc93b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f39addd580 0x55f39addd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:41.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.732+0000 7fc93affd640 1 -- 172.21.15.43:0/2272916053 learned_addr learned my addr 172.21.15.43:0/2272916053 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:41.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.732+0000 7fc938ff9640 1 --2- 172.21.15.43:0/305588819 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc9280a3d50 0x7fc9280a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:41.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.732+0000 7fc933fff640 1 --2- 172.21.15.43:0/2272916053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f39ade6a80 0x55f39ade8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:41.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.733+0000 7fc93affd640 1 -- 172.21.15.43:0/2272916053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f39ade6a80 msgr2=0x55f39ade8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:41.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.733+0000 7fc93affd640 1 --2- 172.21.15.43:0/2272916053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f39ade6a80 0x55f39ade8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.733+0000 7fc93affd640 1 -- 172.21.15.43:0/2272916053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f39addd580 msgr2=0x55f39addd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:41.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.733+0000 7fc93affd640 1 --2- 172.21.15.43:0/2272916053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f39addd580 0x55f39addd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.733+0000 7fc93affd640 1 -- 172.21.15.43:0/2272916053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f39ab1e7d0 con 0x55f39adddea0 2024-04-21T08:46:41.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.733+0000 7fc93b7fe640 1 --2- 172.21.15.43:0/2272916053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f39addd580 0x55f39addd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:41.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.733+0000 7fc938ff9640 1 -- 172.21.15.43:0/305588819 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7fc9240f2680 con 0x7fc93c06af70 2024-04-21T08:46:41.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.733+0000 7fc93b7fe640 1 --2- 172.21.15.43:0/305588819 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc9280a3d50 0x7fc9280a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:41.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.733+0000 7fc93affd640 1 --2- 172.21.15.43:0/2272916053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39adddea0 0x55f39ade6540 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00dc40 tx=0x7fc92c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3d9369383c05fa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:41.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.733+0000 7fc9317fa640 1 -- 172.21.15.43:0/2272916053 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c005020 con 0x55f39adddea0 2024-04-21T08:46:41.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.733+0000 7fc9317fa640 1 -- 172.21.15.43:0/2272916053 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc92c0047a0 con 0x55f39adddea0 2024-04-21T08:46:41.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.733+0000 7fc9317fa640 1 -- 172.21.15.43:0/2272916053 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c0116a0 con 0x55f39adddea0 2024-04-21T08:46:41.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.734+0000 7fc9443df740 1 -- 172.21.15.43:0/2272916053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39adddea0 msgr2=0x55f39ade6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:41.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.734+0000 7fc93b7fe640 1 --2- 172.21.15.43:0/305588819 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc9280a3d50 0x7fc9280a6210 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc934005fd0 tx=0x7fc934004a40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:41.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.734+0000 7fc9443df740 1 --2- 172.21.15.43:0/2272916053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39adddea0 0x55f39ade6540 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00dc40 tx=0x7fc92c00f730 comp rx=0 tx=0).stop 2024-04-21T08:46:41.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.734+0000 7fc9443df740 1 -- 172.21.15.43:0/2272916053 shutdown_connections 2024-04-21T08:46:41.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.734+0000 7fc9443df740 1 --2- 172.21.15.43:0/2272916053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f39ade6a80 0x55f39ade8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.734+0000 7fc9443df740 1 --2- 172.21.15.43:0/2272916053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39adddea0 0x55f39ade6540 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.734+0000 7fc9443df740 1 --2- 172.21.15.43:0/2272916053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f39addd580 0x55f39addd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.734+0000 7fc9443df740 1 -- 172.21.15.43:0/2272916053 >> 172.21.15.43:0/2272916053 conn(0x55f39add7cb0 msgr2=0x55f39add8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:41.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.734+0000 7fc9443df740 1 -- 172.21.15.43:0/2272916053 shutdown_connections 2024-04-21T08:46:41.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.734+0000 7fc9443df740 1 -- 172.21.15.43:0/2272916053 wait complete. 2024-04-21T08:46:41.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.734+0000 7fc9443df740 1 -- start start 2024-04-21T08:46:41.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.734+0000 7fc9443df740 1 Processor -- start 2024-04-21T08:46:41.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.735+0000 7fc9443df740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39addd580 0x55f39ade3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:41.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.735+0000 7fc9443df740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f39adddea0 0x55f39ade3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:41.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.735+0000 7fc9443df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f39ade6a80 0x55f39ade4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:41.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.735+0000 7fc9443df740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f39ade9870 con 0x55f39addd580 2024-04-21T08:46:41.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.735+0000 7fc9443df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f39acfb2f0 con 0x55f39ade6a80 2024-04-21T08:46:41.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.735+0000 7fc9443df740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f39ade99f0 con 0x55f39adddea0 2024-04-21T08:46:41.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.735+0000 7fc9443df740 10 client.0 _ll_register_callbacks cb 0x55f39ad08580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:41.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.735+0000 7fc93affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39addd580 0x55f39ade3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:41.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.735+0000 7fc93affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39addd580 0x55f39ade3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54122/0 (socket says 172.21.15.43:54122) 2024-04-21T08:46:41.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.735+0000 7fc93affd640 1 -- 172.21.15.43:0/4280525471 learned_addr learned my addr 172.21.15.43:0/4280525471 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:41.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.735+0000 7fc933fff640 1 --2- 172.21.15.43:0/4280525471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f39adddea0 0x55f39ade3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:41.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.735+0000 7fc93b7fe640 1 --2- 172.21.15.43:0/4280525471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f39ade6a80 0x55f39ade4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.735+0000 7fc93affd640 1 -- 172.21.15.43:0/4280525471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f39adddea0 msgr2=0x55f39ade3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.735+0000 7fc93affd640 1 --2- 172.21.15.43:0/4280525471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f39adddea0 0x55f39ade3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.735+0000 7fc93affd640 1 -- 172.21.15.43:0/4280525471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f39ade6a80 msgr2=0x55f39ade4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.735+0000 7fc93affd640 1 --2- 172.21.15.43:0/4280525471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f39ade6a80 0x55f39ade4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.735+0000 7fc93affd640 1 -- 172.21.15.43:0/4280525471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f39aa543f0 con 0x55f39addd580 2024-04-21T08:46:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.735+0000 7fc93affd640 1 --2- 172.21.15.43:0/4280525471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39addd580 0x55f39ade3ab0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00df60 tx=0x7fc92c00df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55f2a1ec0132342 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.735+0000 7fc9227fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.736+0000 7fc9227fc640 1 -- 172.21.15.43:0/4280525471 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c02b060 con 0x55f39addd580 2024-04-21T08:46:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.736+0000 7fc9227fc640 1 -- 172.21.15.43:0/4280525471 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7fc92c004b60 con 0x55f39addd580 2024-04-21T08:46:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.736+0000 7fc9443df740 1 -- 172.21.15.43:0/4280525471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc92c0047a0 con 0x55f39addd580 2024-04-21T08:46:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.736+0000 7fc9227fc640 1 -- 172.21.15.43:0/4280525471 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc92c032050 con 0x55f39addd580 2024-04-21T08:46:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.736+0000 7fc9443df740 10 client.19140 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.736+0000 7fc9443df740 1 -- 172.21.15.43:0/4280525471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f39aa140f0 con 0x55f39addd580 2024-04-21T08:46:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.736+0000 7fc9227fc640 1 -- 172.21.15.43:0/4280525471 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc92c0047a0 con 0x55f39addd580 2024-04-21T08:46:41.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.736+0000 7fc9227fc640 1 -- 172.21.15.43:0/4280525471 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc92c026960 con 0x55f39addd580 2024-04-21T08:46:41.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.736+0000 7fc9443df740 10 client.19140 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:41.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.736+0000 7fc9443df740 10 client.19140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:41.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.736+0000 7fc9443df740 20 client.19140 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:41.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.736+0000 7fc9443df740 1 --2- 172.21.15.43:0/4280525471 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f39adf6a10 0x55f39adf8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:41.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.736+0000 7fc9443df740 4 client.19140 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:41.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.736+0000 7fc9443df740 1 -- 172.21.15.43:0/4280525471 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f39ad4fc00 con 0x55f39adf6a10 2024-04-21T08:46:41.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.737+0000 7fc933fff640 1 --2- 172.21.15.43:0/4280525471 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f39adf6a10 0x55f39adf8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:41.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.737+0000 7fc933fff640 1 --2- 172.21.15.43:0/4280525471 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f39adf6a10 0x55f39adf8e00 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da1ed9395616a759 server_cookie=1321110c7348a863 in_seq=0 out_seq=0 2024-04-21T08:46:41.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.737+0000 7fc9227fc640 10 client.19140 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:41.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.738+0000 7fc9227fc640 1 -- 172.21.15.43:0/4280525471 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fc9240a3ae0 con 0x55f39adf6a10 2024-04-21T08:46:41.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.738+0000 7fc9227fc640 10 client.19140 handle_command_reply: tid=0 2024-04-21T08:46:41.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.738+0000 7fc9443df740 1 client.19140 shutdown 2024-04-21T08:46:41.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.738+0000 7fc9443df740 1 -- 172.21.15.43:0/4280525471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39addd580 msgr2=0x55f39ade3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.738+0000 7fc9443df740 1 --2- 172.21.15.43:0/4280525471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39addd580 0x55f39ade3ab0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00df60 tx=0x7fc92c00df90 comp rx=0 tx=0).stop 2024-04-21T08:46:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.738+0000 7fc9443df740 1 -- 172.21.15.43:0/4280525471 shutdown_connections 2024-04-21T08:46:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.738+0000 7fc9443df740 1 --2- 172.21.15.43:0/4280525471 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f39adf6a10 0x55f39adf8e00 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.738+0000 7fc9443df740 1 --2- 172.21.15.43:0/4280525471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f39ade6a80 0x55f39ade4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.738+0000 7fc9443df740 1 --2- 172.21.15.43:0/4280525471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f39adddea0 0x55f39ade3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.738+0000 7fc9443df740 1 --2- 172.21.15.43:0/4280525471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39addd580 0x55f39ade3ab0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.738+0000 7fc9227fc640 0 client.19140 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.738+0000 7fc9443df740 1 -- 172.21.15.43:0/4280525471 >> 172.21.15.43:0/4280525471 conn(0x55f39add7cb0 msgr2=0x55f39add9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.738+0000 7fc9443df740 1 -- 172.21.15.43:0/4280525471 shutdown_connections 2024-04-21T08:46:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.738+0000 7fc9443df740 1 -- 172.21.15.43:0/4280525471 wait complete. 2024-04-21T08:46:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.738+0000 7fc9443df740 20 client.19140 trim_cache size 0 max 16384 2024-04-21T08:46:41.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.756+0000 7fc9443df740 1 Processor -- start 2024-04-21T08:46:41.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.756+0000 7fc9443df740 1 -- start start 2024-04-21T08:46:41.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.756+0000 7fc9443df740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f39addd580 0x55f39ade61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.756+0000 7fc9443df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f39adddea0 0x55f39adf2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.756+0000 7fc9443df740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39ade6a80 0x55f39ade2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.756+0000 7fc9443df740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c1b1650 con 0x55f39ade6a80 2024-04-21T08:46:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.756+0000 7fc9443df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c1b3190 con 0x55f39adddea0 2024-04-21T08:46:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.756+0000 7fc9443df740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c1b1d20 con 0x55f39addd580 2024-04-21T08:46:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.756+0000 7fc93b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39ade6a80 0x55f39ade2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.756+0000 7fc93b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39ade6a80 0x55f39ade2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54128/0 (socket says 172.21.15.43:54128) 2024-04-21T08:46:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.756+0000 7fc93b7fe640 1 -- 172.21.15.43:0/2936346337 learned_addr learned my addr 172.21.15.43:0/2936346337 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.756+0000 7fc933fff640 1 --2- 172.21.15.43:0/2936346337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f39adddea0 0x55f39adf2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.756+0000 7fc93affd640 1 --2- 172.21.15.43:0/2936346337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f39addd580 0x55f39ade61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.757+0000 7fc93b7fe640 1 -- 172.21.15.43:0/2936346337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f39addd580 msgr2=0x55f39ade61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.757+0000 7fc93b7fe640 1 --2- 172.21.15.43:0/2936346337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f39addd580 0x55f39ade61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.757+0000 7fc93b7fe640 1 -- 172.21.15.43:0/2936346337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f39adddea0 msgr2=0x55f39adf2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:41.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.757+0000 7fc93b7fe640 1 --2- 172.21.15.43:0/2936346337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f39adddea0 0x55f39adf2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.757+0000 7fc93b7fe640 1 -- 172.21.15.43:0/2936346337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f39add8630 con 0x55f39ade6a80 2024-04-21T08:46:41.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.757+0000 7fc933fff640 1 --2- 172.21.15.43:0/2936346337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f39adddea0 0x55f39adf2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:41.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.757+0000 7fc93b7fe640 1 --2- 172.21.15.43:0/2936346337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39ade6a80 0x55f39ade2960 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fc934010780 tx=0x7fc934016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc87c0a276d4aba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:41.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.757+0000 7fc930ff9640 1 -- 172.21.15.43:0/2936346337 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934010d00 con 0x55f39ade6a80 2024-04-21T08:46:41.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.757+0000 7fc930ff9640 1 -- 172.21.15.43:0/2936346337 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc93402c070 con 0x55f39ade6a80 2024-04-21T08:46:41.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.757+0000 7fc930ff9640 1 -- 172.21.15.43:0/2936346337 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934027cb0 con 0x55f39ade6a80 2024-04-21T08:46:41.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.758+0000 7fc9443df740 1 -- 172.21.15.43:0/2936346337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39ade6a80 msgr2=0x55f39ade2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:41.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.758+0000 7fc9443df740 1 --2- 172.21.15.43:0/2936346337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39ade6a80 0x55f39ade2960 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fc934010780 tx=0x7fc934016000 comp rx=0 tx=0).stop 2024-04-21T08:46:41.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.758+0000 7fc9443df740 1 -- 172.21.15.43:0/2936346337 shutdown_connections 2024-04-21T08:46:41.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.758+0000 7fc9443df740 1 --2- 172.21.15.43:0/2936346337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39ade6a80 0x55f39ade2960 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.758+0000 7fc9443df740 1 --2- 172.21.15.43:0/2936346337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f39adddea0 0x55f39adf2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.758+0000 7fc9443df740 1 --2- 172.21.15.43:0/2936346337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f39addd580 0x55f39ade61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.758+0000 7fc9443df740 1 -- 172.21.15.43:0/2936346337 >> 172.21.15.43:0/2936346337 conn(0x55f39add7cb0 msgr2=0x55f39ade2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:41.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.758+0000 7fc9443df740 1 -- 172.21.15.43:0/2936346337 shutdown_connections 2024-04-21T08:46:41.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.758+0000 7fc9443df740 1 -- 172.21.15.43:0/2936346337 wait complete. 2024-04-21T08:46:41.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.758+0000 7fc9443df740 1 -- start start 2024-04-21T08:46:41.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.758+0000 7fc9443df740 1 Processor -- start 2024-04-21T08:46:41.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.758+0000 7fc9443df740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39addd580 0x55f39adea950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:41.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.758+0000 7fc9443df740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f39adddea0 0x55f39adeae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:41.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.758+0000 7fc9443df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f39ade6a80 0x55f39adeb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:41.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.758+0000 7fc9443df740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c1b2ac0 con 0x55f39addd580 2024-04-21T08:46:41.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.759+0000 7fc9443df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c1b23f0 con 0x55f39ade6a80 2024-04-21T08:46:41.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.759+0000 7fc9443df740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c1af630 con 0x55f39adddea0 2024-04-21T08:46:41.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.759+0000 7fc9443df740 10 client.0 _ll_register_callbacks cb 0x7fc92c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:41.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.759+0000 7fc93affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39addd580 0x55f39adea950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:41.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.759+0000 7fc93affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39addd580 0x55f39adea950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54142/0 (socket says 172.21.15.43:54142) 2024-04-21T08:46:41.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.759+0000 7fc93affd640 1 -- 172.21.15.43:0/3818815738 learned_addr learned my addr 172.21.15.43:0/3818815738 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:41.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.759+0000 7fc93b7fe640 1 --2- 172.21.15.43:0/3818815738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f39ade6a80 0x55f39adeb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:41.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.759+0000 7fc93affd640 1 -- 172.21.15.43:0/3818815738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f39adddea0 msgr2=0x55f39adeae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:41.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.759+0000 7fc933fff640 1 --2- 172.21.15.43:0/3818815738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f39adddea0 0x55f39adeae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:41.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.759+0000 7fc93affd640 1 --2- 172.21.15.43:0/3818815738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f39adddea0 0x55f39adeae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.759+0000 7fc93affd640 1 -- 172.21.15.43:0/3818815738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f39ade6a80 msgr2=0x55f39adeb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:41.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.759+0000 7fc93affd640 1 --2- 172.21.15.43:0/3818815738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f39ade6a80 0x55f39adeb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.759+0000 7fc93affd640 1 -- 172.21.15.43:0/3818815738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f39ae5bf60 con 0x55f39addd580 2024-04-21T08:46:41.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.759+0000 7fc93b7fe640 1 --2- 172.21.15.43:0/3818815738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f39ade6a80 0x55f39adeb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:41.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.759+0000 7fc93affd640 1 --2- 172.21.15.43:0/3818815738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39addd580 0x55f39adea950 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00d7f0 tx=0x7fc92c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5369f7a2d65b5a47 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:41.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.759+0000 7fc9227fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:41.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.759+0000 7fc9227fc640 1 -- 172.21.15.43:0/3818815738 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c03b060 con 0x55f39addd580 2024-04-21T08:46:41.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.759+0000 7fc9227fc640 1 -- 172.21.15.43:0/3818815738 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7fc92c026470 con 0x55f39addd580 2024-04-21T08:46:41.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.759+0000 7fc9443df740 1 -- 172.21.15.43:0/3818815738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc93402c070 con 0x55f39addd580 2024-04-21T08:46:41.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.760+0000 7fc9227fc640 1 -- 172.21.15.43:0/3818815738 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc92c042050 con 0x55f39addd580 2024-04-21T08:46:41.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.760+0000 7fc9443df740 10 client.19146 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:41.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.760+0000 7fc9443df740 1 -- 172.21.15.43:0/3818815738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f39ae5d270 con 0x55f39addd580 2024-04-21T08:46:41.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.760+0000 7fc9227fc640 1 -- 172.21.15.43:0/3818815738 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc93402c070 con 0x55f39addd580 2024-04-21T08:46:41.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.760+0000 7fc9227fc640 1 -- 172.21.15.43:0/3818815738 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc92c011980 con 0x55f39addd580 2024-04-21T08:46:41.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.760+0000 7fc9443df740 10 client.19146 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:41.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.760+0000 7fc9443df740 10 client.19146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:41.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.760+0000 7fc9443df740 20 client.19146 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:41.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.760+0000 7fc9443df740 1 --2- 172.21.15.43:0/3818815738 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f39ae5d9f0 0x55f39ae5fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:41.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.760+0000 7fc9443df740 4 client.19146 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:46:41.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.760+0000 7fc9443df740 1 -- 172.21.15.43:0/3818815738 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f39ad49a70 con 0x55f39ae5d9f0 2024-04-21T08:46:41.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.760+0000 7fc933fff640 1 --2- 172.21.15.43:0/3818815738 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f39ae5d9f0 0x55f39ae5fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:41.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.761+0000 7fc9227fc640 10 client.19146 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:41.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.761+0000 7fc933fff640 1 --2- 172.21.15.43:0/3818815738 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f39ae5d9f0 0x55f39ae5fe10 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a83a44ac71fed91d server_cookie=4d3b6fe7bd73253c in_seq=0 out_seq=0 2024-04-21T08:46:41.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.761+0000 7fc9227fc640 1 -- 172.21.15.43:0/3818815738 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f39ad49a70 con 0x55f39ae5d9f0 2024-04-21T08:46:41.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.761+0000 7fc9227fc640 10 client.19146 handle_command_reply: tid=0 2024-04-21T08:46:41.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.761+0000 7fc9443df740 1 client.19146 shutdown 2024-04-21T08:46:41.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.761+0000 7fc9443df740 1 -- 172.21.15.43:0/3818815738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39addd580 msgr2=0x55f39adea950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:41.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.761+0000 7fc9443df740 1 --2- 172.21.15.43:0/3818815738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39addd580 0x55f39adea950 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00d7f0 tx=0x7fc92c000bc0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.761+0000 7fc9443df740 1 -- 172.21.15.43:0/3818815738 shutdown_connections 2024-04-21T08:46:41.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.761+0000 7fc9443df740 1 --2- 172.21.15.43:0/3818815738 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f39ae5d9f0 0x55f39ae5fe10 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.761+0000 7fc9443df740 1 --2- 172.21.15.43:0/3818815738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f39ade6a80 0x55f39adeb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.761+0000 7fc9443df740 1 --2- 172.21.15.43:0/3818815738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f39adddea0 0x55f39adeae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.761+0000 7fc9443df740 1 --2- 172.21.15.43:0/3818815738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f39addd580 0x55f39adea950 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.761+0000 7fc9227fc640 0 client.19146 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:41.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.761+0000 7fc9443df740 1 -- 172.21.15.43:0/3818815738 >> 172.21.15.43:0/3818815738 conn(0x55f39add7cb0 msgr2=0x55f39ade2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:41.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.761+0000 7fc9443df740 1 -- 172.21.15.43:0/3818815738 shutdown_connections 2024-04-21T08:46:41.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.761+0000 7fc9443df740 1 -- 172.21.15.43:0/3818815738 wait complete. 2024-04-21T08:46:41.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.761+0000 7fc9443df740 20 client.19146 trim_cache size 0 max 16384 2024-04-21T08:46:41.767 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:46:41.768 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:46:41.768 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:46:41.768 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:46:41.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.762+0000 7fc9417fd640 1 -- 172.21.15.43:0/305588819 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc9280a3d50 msgr2=0x7fc9280a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:41.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.762+0000 7fc9417fd640 1 --2- 172.21.15.43:0/305588819 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc9280a3d50 0x7fc9280a6210 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc934005fd0 tx=0x7fc934004a40 comp rx=0 tx=0).stop 2024-04-21T08:46:41.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.762+0000 7fc9417fd640 1 -- 172.21.15.43:0/305588819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc93c06af70 msgr2=0x7fc93c1016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:41.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.762+0000 7fc9417fd640 1 --2- 172.21.15.43:0/305588819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc93c06af70 0x7fc93c1016b0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fc924006010 tx=0x7fc924004040 comp rx=0 tx=0).stop 2024-04-21T08:46:41.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.762+0000 7fc9417fd640 1 -- 172.21.15.43:0/305588819 shutdown_connections 2024-04-21T08:46:41.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.762+0000 7fc9417fd640 1 --2- 172.21.15.43:0/305588819 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc9280a3d50 0x7fc9280a6210 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.762+0000 7fc9417fd640 1 --2- 172.21.15.43:0/305588819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc93c106f20 0x7fc93c101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.762+0000 7fc9417fd640 1 --2- 172.21.15.43:0/305588819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc93c06af70 0x7fc93c1016b0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.762+0000 7fc9417fd640 1 --2- 172.21.15.43:0/305588819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc93c06a5b0 0x7fc93c101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.762+0000 7fc9417fd640 1 -- 172.21.15.43:0/305588819 >> 172.21.15.43:0/305588819 conn(0x7fc93c077c40 msgr2=0x7fc93c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:41.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.762+0000 7fc9417fd640 1 -- 172.21.15.43:0/305588819 shutdown_connections 2024-04-21T08:46:41.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.762+0000 7fc9417fd640 1 -- 172.21.15.43:0/305588819 wait complete. 2024-04-21T08:46:41.772 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:46:41.772 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:46:41.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.876+0000 7fa0447f4640 1 -- 172.21.15.43:0/2140823059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa03c100e50 msgr2=0x7fa03c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:41.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.876+0000 7fa0447f4640 1 --2- 172.21.15.43:0/2140823059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa03c100e50 0x7fa03c1012b0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fa02c009900 tx=0x7fa02c02bcd0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.876+0000 7fa0447f4640 1 -- 172.21.15.43:0/2140823059 shutdown_connections 2024-04-21T08:46:41.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.876+0000 7fa0447f4640 1 --2- 172.21.15.43:0/2140823059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa03c1017f0 0x7fa03c10dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.876+0000 7fa0447f4640 1 --2- 172.21.15.43:0/2140823059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa03c100e50 0x7fa03c1012b0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.876+0000 7fa0447f4640 1 --2- 172.21.15.43:0/2140823059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa03c106f10 0x7fa03c1072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.876+0000 7fa0447f4640 1 -- 172.21.15.43:0/2140823059 >> 172.21.15.43:0/2140823059 conn(0x7fa03c077f80 msgr2=0x7fa03c078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:41.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.876+0000 7fa0447f4640 1 -- 172.21.15.43:0/2140823059 shutdown_connections 2024-04-21T08:46:41.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.876+0000 7fa0447f4640 1 -- 172.21.15.43:0/2140823059 wait complete. 2024-04-21T08:46:41.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.876+0000 7fa0447f4640 1 Processor -- start 2024-04-21T08:46:41.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.876+0000 7fa0447f4640 1 -- start start 2024-04-21T08:46:41.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.876+0000 7fa0447f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa03c100e50 0x7fa03c102b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:41.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.876+0000 7fa0447f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa03c1017f0 0x7fa03c103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:41.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.877+0000 7fa0447f4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa03c106f10 0x7fa03c103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:41.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.877+0000 7fa0447f4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa03c1110d0 con 0x7fa03c100e50 2024-04-21T08:46:41.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.877+0000 7fa0447f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa03c110f50 con 0x7fa03c1017f0 2024-04-21T08:46:41.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.877+0000 7fa0447f4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa03c111250 con 0x7fa03c106f10 2024-04-21T08:46:41.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.877+0000 7fa042590640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa03c100e50 0x7fa03c102b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:41.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.877+0000 7fa042590640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa03c100e50 0x7fa03c102b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54160/0 (socket says 172.21.15.43:54160) 2024-04-21T08:46:41.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.877+0000 7fa042590640 1 -- 172.21.15.43:0/4166528013 learned_addr learned my addr 172.21.15.43:0/4166528013 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:41.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.877+0000 7fa042d91640 1 --2- 172.21.15.43:0/4166528013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa03c106f10 0x7fa03c103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:41.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.877+0000 7fa042590640 1 -- 172.21.15.43:0/4166528013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa03c106f10 msgr2=0x7fa03c103580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:41.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.877+0000 7fa042590640 1 --2- 172.21.15.43:0/4166528013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa03c106f10 0x7fa03c103580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.877+0000 7fa042590640 1 -- 172.21.15.43:0/4166528013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa03c1017f0 msgr2=0x7fa03c103040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:41.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.877+0000 7fa041d8f640 1 --2- 172.21.15.43:0/4166528013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa03c1017f0 0x7fa03c103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:41.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.877+0000 7fa042590640 1 --2- 172.21.15.43:0/4166528013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa03c1017f0 0x7fa03c103040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.877+0000 7fa042590640 1 -- 172.21.15.43:0/4166528013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa03c1a6b00 con 0x7fa03c100e50 2024-04-21T08:46:41.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.877+0000 7fa042d91640 1 --2- 172.21.15.43:0/4166528013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa03c106f10 0x7fa03c103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:41.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.877+0000 7fa042590640 1 --2- 172.21.15.43:0/4166528013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa03c100e50 0x7fa03c102b00 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fa03800c920 tx=0x7fa03800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:41.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.877+0000 7fa0337fe640 1 -- 172.21.15.43:0/4166528013 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa038007c70 con 0x7fa03c100e50 2024-04-21T08:46:41.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.877+0000 7fa0337fe640 1 -- 172.21.15.43:0/4166528013 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa038007e10 con 0x7fa03c100e50 2024-04-21T08:46:41.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.877+0000 7fa0447f4640 1 -- 172.21.15.43:0/4166528013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa03c1a6df0 con 0x7fa03c100e50 2024-04-21T08:46:41.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.877+0000 7fa0337fe640 1 -- 172.21.15.43:0/4166528013 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa038018820 con 0x7fa03c100e50 2024-04-21T08:46:41.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.877+0000 7fa0447f4640 1 -- 172.21.15.43:0/4166528013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa03c1a7300 con 0x7fa03c100e50 2024-04-21T08:46:41.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.878+0000 7fa0447f4640 1 -- 172.21.15.43:0/4166528013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa03c068550 con 0x7fa03c100e50 2024-04-21T08:46:41.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.880+0000 7fa0337fe640 1 -- 172.21.15.43:0/4166528013 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa03800f450 con 0x7fa03c100e50 2024-04-21T08:46:41.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.880+0000 7fa0337fe640 1 --2- 172.21.15.43:0/4166528013 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa0280a3da0 0x7fa0280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:41.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.880+0000 7fa0337fe640 1 -- 172.21.15.43:0/4166528013 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7fa0380d0f60 con 0x7fa03c100e50 2024-04-21T08:46:41.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.880+0000 7fa041d8f640 1 --2- 172.21.15.43:0/4166528013 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa0280a3da0 0x7fa0280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:41.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.881+0000 7fa041d8f640 1 --2- 172.21.15.43:0/4166528013 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa0280a3da0 0x7fa0280a6260 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa02c036040 tx=0x7fa02c005fd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:41.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:41.884+0000 7fa0337fe640 1 -- 172.21.15.43:0/4166528013 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa038014030 con 0x7fa03c100e50 2024-04-21T08:46:42.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.112+0000 7fa0447f4640 1 -- 172.21.15.43:0/4166528013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa03c104030 con 0x7fa03c100e50 2024-04-21T08:46:42.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.113+0000 7fa0337fe640 1 -- 172.21.15.43:0/4166528013 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11935 (secure 0 0 0) 0x7fa03809ccf0 con 0x7fa03c100e50 2024-04-21T08:46:42.113 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:46:42.113 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:46:17.283806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27556":{"gid":27556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/229985237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":229985237},{"type":"v1","addr":"172.21.15.175:6837","nonce":229985237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24425,24412,24487,24527,24491]},"id":1}]} 2024-04-21T08:46:42.113 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-04-21T08:46:42.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.115+0000 7fa0447f4640 1 -- 172.21.15.43:0/4166528013 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa0280a3da0 msgr2=0x7fa0280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.115+0000 7fa0447f4640 1 --2- 172.21.15.43:0/4166528013 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa0280a3da0 0x7fa0280a6260 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa02c036040 tx=0x7fa02c005fd0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.115+0000 7fa0447f4640 1 -- 172.21.15.43:0/4166528013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa03c100e50 msgr2=0x7fa03c102b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.115+0000 7fa0447f4640 1 --2- 172.21.15.43:0/4166528013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa03c100e50 0x7fa03c102b00 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fa03800c920 tx=0x7fa03800cde0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.115+0000 7fa0447f4640 1 -- 172.21.15.43:0/4166528013 shutdown_connections 2024-04-21T08:46:42.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.115+0000 7fa0447f4640 1 --2- 172.21.15.43:0/4166528013 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa0280a3da0 0x7fa0280a6260 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.115+0000 7fa0447f4640 1 --2- 172.21.15.43:0/4166528013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa03c106f10 0x7fa03c103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.115+0000 7fa0447f4640 1 --2- 172.21.15.43:0/4166528013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa03c1017f0 0x7fa03c103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.115+0000 7fa0447f4640 1 --2- 172.21.15.43:0/4166528013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa03c100e50 0x7fa03c102b00 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.115+0000 7fa0447f4640 1 -- 172.21.15.43:0/4166528013 >> 172.21.15.43:0/4166528013 conn(0x7fa03c077f80 msgr2=0x7fa03c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.115+0000 7fa0447f4640 1 -- 172.21.15.43:0/4166528013 shutdown_connections 2024-04-21T08:46:42.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.115+0000 7fa0447f4640 1 -- 172.21.15.43:0/4166528013 wait complete. 2024-04-21T08:46:42.126 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:46:42.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.231+0000 7f06bf469640 1 -- 172.21.15.43:0/753666554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06b8101090 msgr2=0x7f06b8101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.231+0000 7f06bf469640 1 --2- 172.21.15.43:0/753666554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06b8101090 0x7f06b8101470 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009900 tx=0x7f06b002bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:42.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.231+0000 7f06bf469640 1 -- 172.21.15.43:0/753666554 shutdown_connections 2024-04-21T08:46:42.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.231+0000 7f06bf469640 1 --2- 172.21.15.43:0/753666554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06b8106160 0x7f06b810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.231+0000 7f06bf469640 1 --2- 172.21.15.43:0/753666554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06b8101a40 0x7f06b8105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.231+0000 7f06bf469640 1 --2- 172.21.15.43:0/753666554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06b8101090 0x7f06b8101470 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.231+0000 7f06bf469640 1 -- 172.21.15.43:0/753666554 >> 172.21.15.43:0/753666554 conn(0x7f06b8077f50 msgr2=0x7f06b8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.231+0000 7f06bf469640 1 -- 172.21.15.43:0/753666554 shutdown_connections 2024-04-21T08:46:42.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.231+0000 7f06bf469640 1 -- 172.21.15.43:0/753666554 wait complete. 2024-04-21T08:46:42.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.231+0000 7f06bf469640 1 Processor -- start 2024-04-21T08:46:42.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.231+0000 7f06bf469640 1 -- start start 2024-04-21T08:46:42.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.232+0000 7f06bf469640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06b8101090 0x7f06b81986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.232+0000 7f06bf469640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06b8106160 0x7f06b8198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.232+0000 7f06bf469640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06b819cf70 0x7f06b819d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.232+0000 7f06bf469640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f06b81110d0 con 0x7f06b8101090 2024-04-21T08:46:42.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.232+0000 7f06bf469640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06b8110f50 con 0x7f06b8106160 2024-04-21T08:46:42.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.232+0000 7f06bf469640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f06b8111250 con 0x7f06b819cf70 2024-04-21T08:46:42.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.232+0000 7f06bd205640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06b8101090 0x7f06b81986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.232+0000 7f06bd205640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06b8101090 0x7f06b81986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54172/0 (socket says 172.21.15.43:54172) 2024-04-21T08:46:42.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.232+0000 7f06bd205640 1 -- 172.21.15.43:0/1828753624 learned_addr learned my addr 172.21.15.43:0/1828753624 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.232+0000 7f06bca04640 1 --2- 172.21.15.43:0/1828753624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06b8106160 0x7f06b8198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.232+0000 7f06bda06640 1 --2- 172.21.15.43:0/1828753624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06b819cf70 0x7f06b819d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.232+0000 7f06bd205640 1 -- 172.21.15.43:0/1828753624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06b819cf70 msgr2=0x7f06b819d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.232+0000 7f06bd205640 1 --2- 172.21.15.43:0/1828753624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06b819cf70 0x7f06b819d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.232+0000 7f06bd205640 1 -- 172.21.15.43:0/1828753624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06b8106160 msgr2=0x7f06b8198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.232+0000 7f06bd205640 1 --2- 172.21.15.43:0/1828753624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06b8106160 0x7f06b8198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.232+0000 7f06bd205640 1 -- 172.21.15.43:0/1828753624 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b819da60 con 0x7f06b8101090 2024-04-21T08:46:42.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.232+0000 7f06bda06640 1 --2- 172.21.15.43:0/1828753624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06b819cf70 0x7f06b819d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:42.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.232+0000 7f06bd205640 1 --2- 172.21.15.43:0/1828753624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06b8101090 0x7f06b81986b0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f06b0005940 tx=0x7f06b002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.232+0000 7f06ae7fc640 1 -- 172.21.15.43:0/1828753624 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b0043020 con 0x7f06b8101090 2024-04-21T08:46:42.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.232+0000 7f06ae7fc640 1 -- 172.21.15.43:0/1828753624 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b002cbe0 con 0x7f06b8101090 2024-04-21T08:46:42.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.232+0000 7f06bf469640 1 -- 172.21.15.43:0/1828753624 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06b819dcf0 con 0x7f06b8101090 2024-04-21T08:46:42.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.232+0000 7f06ae7fc640 1 -- 172.21.15.43:0/1828753624 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b0003bf0 con 0x7f06b8101090 2024-04-21T08:46:42.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.232+0000 7f06bf469640 1 -- 172.21.15.43:0/1828753624 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06b81a5510 con 0x7f06b8101090 2024-04-21T08:46:42.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.235+0000 7f06c2056740 1 Processor -- start 2024-04-21T08:46:42.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.236+0000 7f06c2056740 1 -- start start 2024-04-21T08:46:42.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.236+0000 7f06ae7fc640 1 -- 172.21.15.43:0/1828753624 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f06b0046020 con 0x7f06b8101090 2024-04-21T08:46:42.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.236+0000 7f06c2056740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a82cb44d0 0x559a82cb48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.236+0000 7f06c2056740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a82cb4e80 0x559a82cbd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.236+0000 7f06c2056740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cbda00 0x559a82cbfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.236+0000 7f06c2056740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559a82c191f0 con 0x559a82cbda00 2024-04-21T08:46:42.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.236+0000 7f06c2056740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559a82bb64b0 con 0x559a82cb44d0 2024-04-21T08:46:42.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.236+0000 7f06c2056740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559a82c151b0 con 0x559a82cb4e80 2024-04-21T08:46:42.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.236+0000 7f06bca04640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cbda00 0x559a82cbfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.236+0000 7f06bca04640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cbda00 0x559a82cbfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54184/0 (socket says 172.21.15.43:54184) 2024-04-21T08:46:42.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.236+0000 7f06bca04640 1 -- 172.21.15.43:0/3898303018 learned_addr learned my addr 172.21.15.43:0/3898303018 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.236+0000 7f06bd205640 1 --2- 172.21.15.43:0/3898303018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a82cb4e80 0x559a82cbd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.236+0000 7f06bda06640 1 --2- 172.21.15.43:0/3898303018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a82cb44d0 0x559a82cb48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.236+0000 7f06bca04640 1 -- 172.21.15.43:0/3898303018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a82cb4e80 msgr2=0x559a82cbd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.236+0000 7f06bca04640 1 --2- 172.21.15.43:0/3898303018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a82cb4e80 0x559a82cbd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.236+0000 7f06bca04640 1 -- 172.21.15.43:0/3898303018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a82cb44d0 msgr2=0x559a82cb48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.236+0000 7f06bca04640 1 --2- 172.21.15.43:0/3898303018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a82cb44d0 0x559a82cb48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.236+0000 7f06bca04640 1 -- 172.21.15.43:0/3898303018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a82ba81b0 con 0x559a82cbda00 2024-04-21T08:46:42.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.236+0000 7f06ae7fc640 1 --2- 172.21.15.43:0/1828753624 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f06a40a3d20 0x7f06a40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.237+0000 7f06bca04640 1 --2- 172.21.15.43:0/3898303018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cbda00 0x559a82cbfe50 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f06a800cc00 tx=0x7f06a800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb2c3bab9d55c570 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.237+0000 7f06bda06640 1 --2- 172.21.15.43:0/1828753624 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f06a40a3d20 0x7f06a40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.237+0000 7f06ae7fc640 1 -- 172.21.15.43:0/1828753624 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f06b00f2db0 con 0x7f06b8101090 2024-04-21T08:46:42.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.237+0000 7f06a37fe640 1 -- 172.21.15.43:0/3898303018 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a8005220 con 0x559a82cbda00 2024-04-21T08:46:42.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.237+0000 7f06a37fe640 1 -- 172.21.15.43:0/3898303018 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06a8004c50 con 0x559a82cbda00 2024-04-21T08:46:42.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.237+0000 7f06a37fe640 1 -- 172.21.15.43:0/3898303018 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a8010790 con 0x559a82cbda00 2024-04-21T08:46:42.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.237+0000 7f06bda06640 1 --2- 172.21.15.43:0/1828753624 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f06a40a3d20 0x7f06a40a61e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f06b4005fd0 tx=0x7f06b4009680 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.237+0000 7f06c2056740 1 -- 172.21.15.43:0/3898303018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cbda00 msgr2=0x559a82cbfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.237+0000 7f06c2056740 1 --2- 172.21.15.43:0/3898303018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cbda00 0x559a82cbfe50 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f06a800cc00 tx=0x7f06a800e720 comp rx=0 tx=0).stop 2024-04-21T08:46:42.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.238+0000 7f06c2056740 1 -- 172.21.15.43:0/3898303018 shutdown_connections 2024-04-21T08:46:42.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.238+0000 7f06c2056740 1 --2- 172.21.15.43:0/3898303018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cbda00 0x559a82cbfe50 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.238+0000 7f06c2056740 1 --2- 172.21.15.43:0/3898303018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a82cb4e80 0x559a82cbd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.238+0000 7f06c2056740 1 --2- 172.21.15.43:0/3898303018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a82cb44d0 0x559a82cb48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.238+0000 7f06c2056740 1 -- 172.21.15.43:0/3898303018 >> 172.21.15.43:0/3898303018 conn(0x559a82caeba0 msgr2=0x559a82caef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.238+0000 7f06c2056740 1 -- 172.21.15.43:0/3898303018 shutdown_connections 2024-04-21T08:46:42.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.238+0000 7f06c2056740 1 -- 172.21.15.43:0/3898303018 wait complete. 2024-04-21T08:46:42.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.238+0000 7f06c2056740 1 -- start start 2024-04-21T08:46:42.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.238+0000 7f06c2056740 1 Processor -- start 2024-04-21T08:46:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.238+0000 7f06c2056740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a82cb44d0 0x559a82cbd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.238+0000 7f06c2056740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cb4e80 0x559a82cba640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.238+0000 7f06c2056740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a82cbda00 0x559a82cbab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.238+0000 7f06c2056740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559a82cc0970 con 0x559a82cb4e80 2024-04-21T08:46:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.238+0000 7f06c2056740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559a82bd2500 con 0x559a82cb44d0 2024-04-21T08:46:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.238+0000 7f06c2056740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559a82cc0af0 con 0x559a82cbda00 2024-04-21T08:46:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.239+0000 7f06c2056740 10 client.0 _ll_register_callbacks cb 0x559a82ba0b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.239+0000 7f06bca04640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cb4e80 0x559a82cba640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.239+0000 7f06bca04640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cb4e80 0x559a82cba640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54196/0 (socket says 172.21.15.43:54196) 2024-04-21T08:46:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.239+0000 7f06bca04640 1 -- 172.21.15.43:0/1852197243 learned_addr learned my addr 172.21.15.43:0/1852197243 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.239+0000 7f06bd205640 1 --2- 172.21.15.43:0/1852197243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a82cb44d0 0x559a82cbd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.239+0000 7f06bda06640 1 --2- 172.21.15.43:0/1852197243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a82cbda00 0x559a82cbab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.239+0000 7f06bca04640 1 -- 172.21.15.43:0/1852197243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a82cbda00 msgr2=0x559a82cbab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.239+0000 7f06bca04640 1 --2- 172.21.15.43:0/1852197243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a82cbda00 0x559a82cbab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.239+0000 7f06bca04640 1 -- 172.21.15.43:0/1852197243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a82cb44d0 msgr2=0x559a82cbd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.239+0000 7f06bca04640 1 --2- 172.21.15.43:0/1852197243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a82cb44d0 0x559a82cbd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.239+0000 7f06bca04640 1 -- 172.21.15.43:0/1852197243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a8292b3f0 con 0x559a82cb4e80 2024-04-21T08:46:42.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.239+0000 7f06bda06640 1 --2- 172.21.15.43:0/1852197243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a82cbda00 0x559a82cbab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:42.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.239+0000 7f06bca04640 1 --2- 172.21.15.43:0/1852197243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cb4e80 0x559a82cba640 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f06a8005720 tx=0x7f06a800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef7fa35d8b95d9f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.239+0000 7f06a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:42.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.239+0000 7f06a0ff9640 1 -- 172.21.15.43:0/1852197243 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a802a060 con 0x559a82cb4e80 2024-04-21T08:46:42.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.239+0000 7f06a0ff9640 1 -- 172.21.15.43:0/1852197243 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f06a8005010 con 0x559a82cb4e80 2024-04-21T08:46:42.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.239+0000 7f06c2056740 1 -- 172.21.15.43:0/1852197243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06a8004c50 con 0x559a82cb4e80 2024-04-21T08:46:42.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.239+0000 7f06a0ff9640 1 -- 172.21.15.43:0/1852197243 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f06a8031050 con 0x559a82cb4e80 2024-04-21T08:46:42.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.239+0000 7f06c2056740 10 client.19164 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:42.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.239+0000 7f06c2056740 1 -- 172.21.15.43:0/1852197243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559a829f57d0 con 0x559a82cb4e80 2024-04-21T08:46:42.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.240+0000 7f06a0ff9640 1 -- 172.21.15.43:0/1852197243 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06a8004c50 con 0x559a82cb4e80 2024-04-21T08:46:42.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.240+0000 7f06a0ff9640 1 -- 172.21.15.43:0/1852197243 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f06a801a460 con 0x559a82cb4e80 2024-04-21T08:46:42.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.240+0000 7f06c2056740 10 client.19164 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:42.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.240+0000 7f06c2056740 10 client.19164 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:46:42.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.240+0000 7f06c2056740 20 client.19164 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:42.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.240+0000 7f06c2056740 1 --2- 172.21.15.43:0/1852197243 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x559a82ccdb50 0x559a82ccff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.240+0000 7f06c2056740 4 client.19164 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:42.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.240+0000 7f06c2056740 1 -- 172.21.15.43:0/1852197243 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a82bb8930 con 0x559a82ccdb50 2024-04-21T08:46:42.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.240+0000 7f06bd205640 1 --2- 172.21.15.43:0/1852197243 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x559a82ccdb50 0x559a82ccff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.241+0000 7f06bd205640 1 --2- 172.21.15.43:0/1852197243 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x559a82ccdb50 0x559a82ccff40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=17aa6c9c8cd7d4aa server_cookie=5ad26f8defeb4cce in_seq=0 out_seq=0 2024-04-21T08:46:42.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.241+0000 7f06a0ff9640 10 client.19164 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:46:42.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.242+0000 7f06a0ff9640 1 -- 172.21.15.43:0/1852197243 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a82bb8930 con 0x559a82ccdb50 2024-04-21T08:46:42.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.242+0000 7f06a0ff9640 10 client.19164 handle_command_reply: tid=0 2024-04-21T08:46:42.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.242+0000 7f06c2056740 1 client.19164 shutdown 2024-04-21T08:46:42.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.242+0000 7f06c2056740 1 -- 172.21.15.43:0/1852197243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cb4e80 msgr2=0x559a82cba640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.242+0000 7f06c2056740 1 --2- 172.21.15.43:0/1852197243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cb4e80 0x559a82cba640 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f06a8005720 tx=0x7f06a800c940 comp rx=0 tx=0).stop 2024-04-21T08:46:42.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.242+0000 7f06c2056740 1 -- 172.21.15.43:0/1852197243 shutdown_connections 2024-04-21T08:46:42.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.242+0000 7f06c2056740 1 --2- 172.21.15.43:0/1852197243 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x559a82ccdb50 0x559a82ccff40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.242+0000 7f06c2056740 1 --2- 172.21.15.43:0/1852197243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a82cbda00 0x559a82cbab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.242+0000 7f06c2056740 1 --2- 172.21.15.43:0/1852197243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cb4e80 0x559a82cba640 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.242+0000 7f06a0ff9640 0 client.19164 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:46:42.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.242+0000 7f06c2056740 1 --2- 172.21.15.43:0/1852197243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a82cb44d0 0x559a82cbd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.242+0000 7f06c2056740 1 -- 172.21.15.43:0/1852197243 >> 172.21.15.43:0/1852197243 conn(0x559a82caeba0 msgr2=0x559a82cb08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.242+0000 7f06c2056740 1 -- 172.21.15.43:0/1852197243 shutdown_connections 2024-04-21T08:46:42.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.242+0000 7f06c2056740 1 -- 172.21.15.43:0/1852197243 wait complete. 2024-04-21T08:46:42.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.242+0000 7f06c2056740 20 client.19164 trim_cache size 0 max 16384 2024-04-21T08:46:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.256+0000 7f06c2056740 1 Processor -- start 2024-04-21T08:46:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.256+0000 7f06c2056740 1 -- start start 2024-04-21T08:46:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.256+0000 7f06c2056740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a82cb44d0 0x559a82cc9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.256+0000 7f06c2056740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cb4e80 0x559a82cada80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.256+0000 7f06c2056740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a82cbda00 0x559a82cbcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.256+0000 7f06c2056740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f06b81af2b0 con 0x559a82cb4e80 2024-04-21T08:46:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.256+0000 7f06c2056740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06b81b0df0 con 0x559a82cbda00 2024-04-21T08:46:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.256+0000 7f06c2056740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f06b81af980 con 0x559a82cb44d0 2024-04-21T08:46:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.256+0000 7f06bca04640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cb4e80 0x559a82cada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.256+0000 7f06bca04640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cb4e80 0x559a82cada80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54210/0 (socket says 172.21.15.43:54210) 2024-04-21T08:46:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.256+0000 7f06bca04640 1 -- 172.21.15.43:0/545694295 learned_addr learned my addr 172.21.15.43:0/545694295 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.256+0000 7f06bda06640 1 --2- 172.21.15.43:0/545694295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a82cbda00 0x559a82cbcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.256+0000 7f06bca04640 1 -- 172.21.15.43:0/545694295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a82cb44d0 msgr2=0x559a82cc9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.256+0000 7f06bca04640 1 --2- 172.21.15.43:0/545694295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a82cb44d0 0x559a82cc9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.256+0000 7f06bca04640 1 -- 172.21.15.43:0/545694295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a82cbda00 msgr2=0x559a82cbcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.256+0000 7f06bca04640 1 --2- 172.21.15.43:0/545694295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a82cbda00 0x559a82cbcd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.256+0000 7f06bca04640 1 -- 172.21.15.43:0/545694295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a82caa5e0 con 0x559a82cb4e80 2024-04-21T08:46:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.256+0000 7f06bda06640 1 --2- 172.21.15.43:0/545694295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a82cbda00 0x559a82cbcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.256+0000 7f06bca04640 1 --2- 172.21.15.43:0/545694295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cb4e80 0x559a82cada80 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f06a800cb20 tx=0x7f06a8005b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9666c6b91168824b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.257+0000 7f06a2ffd640 1 -- 172.21.15.43:0/545694295 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a802a070 con 0x559a82cb4e80 2024-04-21T08:46:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.257+0000 7f06a2ffd640 1 -- 172.21.15.43:0/545694295 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06a8004ad0 con 0x559a82cb4e80 2024-04-21T08:46:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.257+0000 7f06a2ffd640 1 -- 172.21.15.43:0/545694295 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a8010430 con 0x559a82cb4e80 2024-04-21T08:46:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.257+0000 7f06c2056740 1 -- 172.21.15.43:0/545694295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cb4e80 msgr2=0x559a82cada80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.257+0000 7f06c2056740 1 --2- 172.21.15.43:0/545694295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cb4e80 0x559a82cada80 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f06a800cb20 tx=0x7f06a8005b30 comp rx=0 tx=0).stop 2024-04-21T08:46:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.257+0000 7f06c2056740 1 -- 172.21.15.43:0/545694295 shutdown_connections 2024-04-21T08:46:42.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.257+0000 7f06c2056740 1 --2- 172.21.15.43:0/545694295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a82cbda00 0x559a82cbcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.257+0000 7f06c2056740 1 --2- 172.21.15.43:0/545694295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cb4e80 0x559a82cada80 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.257+0000 7f06c2056740 1 --2- 172.21.15.43:0/545694295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a82cb44d0 0x559a82cc9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.257+0000 7f06c2056740 1 -- 172.21.15.43:0/545694295 >> 172.21.15.43:0/545694295 conn(0x559a82caeba0 msgr2=0x559a82cad370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.257+0000 7f06c2056740 1 -- 172.21.15.43:0/545694295 shutdown_connections 2024-04-21T08:46:42.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.257+0000 7f06c2056740 1 -- 172.21.15.43:0/545694295 wait complete. 2024-04-21T08:46:42.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.257+0000 7f06c2056740 1 -- start start 2024-04-21T08:46:42.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.258+0000 7f06c2056740 1 Processor -- start 2024-04-21T08:46:42.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.258+0000 7f06c2056740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a82cb44d0 0x559a82cc5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.258+0000 7f06c2056740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a82cb4e80 0x559a82cc62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.258+0000 7f06c2056740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cbda00 0x559a82cc12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.258+0000 7f06c2056740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f06b81b0720 con 0x559a82cbda00 2024-04-21T08:46:42.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.258+0000 7f06c2056740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06b81b0050 con 0x559a82cb44d0 2024-04-21T08:46:42.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.258+0000 7f06c2056740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f06b81ad290 con 0x559a82cb4e80 2024-04-21T08:46:42.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.258+0000 7f06c2056740 10 client.0 _ll_register_callbacks cb 0x7f06b00f67d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:42.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.258+0000 7f06bda06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cbda00 0x559a82cc12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.258+0000 7f06bd205640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a82cb44d0 0x559a82cc5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.258+0000 7f06bda06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cbda00 0x559a82cc12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54224/0 (socket says 172.21.15.43:54224) 2024-04-21T08:46:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.258+0000 7f06bda06640 1 -- 172.21.15.43:0/3946776856 learned_addr learned my addr 172.21.15.43:0/3946776856 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.258+0000 7f06bca04640 1 --2- 172.21.15.43:0/3946776856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a82cb4e80 0x559a82cc62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.258+0000 7f06bda06640 1 -- 172.21.15.43:0/3946776856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a82cb4e80 msgr2=0x559a82cc62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.258+0000 7f06bda06640 1 --2- 172.21.15.43:0/3946776856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a82cb4e80 0x559a82cc62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.258+0000 7f06bda06640 1 -- 172.21.15.43:0/3946776856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a82cb44d0 msgr2=0x559a82cc5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.258+0000 7f06bda06640 1 --2- 172.21.15.43:0/3946776856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a82cb44d0 0x559a82cc5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.258+0000 7f06bda06640 1 -- 172.21.15.43:0/3946776856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a82cc1af0 con 0x559a82cbda00 2024-04-21T08:46:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.258+0000 7f06bd205640 1 --2- 172.21.15.43:0/3946776856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a82cb44d0 0x559a82cc5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.259+0000 7f06bda06640 1 --2- 172.21.15.43:0/3946776856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cbda00 0x559a82cc12f0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f06b4012860 tx=0x7f06b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca2f6e6eb745545 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.259+0000 7f06a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.259+0000 7f06a0ff9640 1 -- 172.21.15.43:0/3946776856 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4004d20 con 0x559a82cbda00 2024-04-21T08:46:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.259+0000 7f06a0ff9640 1 -- 172.21.15.43:0/3946776856 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f06b401f6f0 con 0x559a82cbda00 2024-04-21T08:46:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.259+0000 7f06c2056740 1 -- 172.21.15.43:0/3946776856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06a8004ad0 con 0x559a82cbda00 2024-04-21T08:46:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.259+0000 7f06a0ff9640 1 -- 172.21.15.43:0/3946776856 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f06b401f8f0 con 0x559a82cbda00 2024-04-21T08:46:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.259+0000 7f06c2056740 10 client.19170 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:42.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.259+0000 7f06c2056740 1 -- 172.21.15.43:0/3946776856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559a82d37ed0 con 0x559a82cbda00 2024-04-21T08:46:42.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.259+0000 7f06a0ff9640 1 -- 172.21.15.43:0/3946776856 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06a8004ad0 con 0x559a82cbda00 2024-04-21T08:46:42.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.259+0000 7f06a0ff9640 1 -- 172.21.15.43:0/3946776856 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f06b4026070 con 0x559a82cbda00 2024-04-21T08:46:42.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.259+0000 7f06c2056740 10 client.19170 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:42.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.259+0000 7f06c2056740 10 client.19170 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:46:42.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.259+0000 7f06c2056740 20 client.19170 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:42.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.259+0000 7f06c2056740 1 --2- 172.21.15.43:0/3946776856 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x559a82d386d0 0x559a82d3aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.259+0000 7f06c2056740 4 client.19170 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:42.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.259+0000 7f06c2056740 1 -- 172.21.15.43:0/3946776856 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a82cbaf10 con 0x559a82d386d0 2024-04-21T08:46:42.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.260+0000 7f06bd205640 1 --2- 172.21.15.43:0/3946776856 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x559a82d386d0 0x559a82d3aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.260+0000 7f06a0ff9640 10 client.19170 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:46:42.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.260+0000 7f06bd205640 1 --2- 172.21.15.43:0/3946776856 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x559a82d386d0 0x559a82d3aaf0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9a7f18d5b27519e1 server_cookie=d78fa921d316b76e in_seq=0 out_seq=0 2024-04-21T08:46:42.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.260+0000 7f06a0ff9640 1 -- 172.21.15.43:0/3946776856 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a82cbaf10 con 0x559a82d386d0 2024-04-21T08:46:42.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.260+0000 7f06a0ff9640 10 client.19170 handle_command_reply: tid=0 2024-04-21T08:46:42.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.260+0000 7f06c2056740 1 client.19170 shutdown 2024-04-21T08:46:42.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.261+0000 7f06c2056740 1 -- 172.21.15.43:0/3946776856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cbda00 msgr2=0x559a82cc12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.261+0000 7f06c2056740 1 --2- 172.21.15.43:0/3946776856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cbda00 0x559a82cc12f0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f06b4012860 tx=0x7f06b4004a40 comp rx=0 tx=0).stop 2024-04-21T08:46:42.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.261+0000 7f06c2056740 1 -- 172.21.15.43:0/3946776856 shutdown_connections 2024-04-21T08:46:42.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.261+0000 7f06c2056740 1 --2- 172.21.15.43:0/3946776856 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x559a82d386d0 0x559a82d3aaf0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.261+0000 7f06c2056740 1 --2- 172.21.15.43:0/3946776856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a82cbda00 0x559a82cc12f0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.261+0000 7f06c2056740 1 --2- 172.21.15.43:0/3946776856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a82cb4e80 0x559a82cc62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.261+0000 7f06a0ff9640 0 client.19170 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:46:42.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.261+0000 7f06c2056740 1 --2- 172.21.15.43:0/3946776856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559a82cb44d0 0x559a82cc5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.261+0000 7f06c2056740 1 -- 172.21.15.43:0/3946776856 >> 172.21.15.43:0/3946776856 conn(0x559a82caeba0 msgr2=0x559a82cadc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.261+0000 7f06c2056740 1 -- 172.21.15.43:0/3946776856 shutdown_connections 2024-04-21T08:46:42.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.261+0000 7f06c2056740 1 -- 172.21.15.43:0/3946776856 wait complete. 2024-04-21T08:46:42.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.261+0000 7f06c2056740 20 client.19170 trim_cache size 0 max 16384 2024-04-21T08:46:42.267 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:46:42.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.261+0000 7f06bf469640 1 -- 172.21.15.43:0/1828753624 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f06a40a3d20 msgr2=0x7f06a40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.261+0000 7f06bf469640 1 --2- 172.21.15.43:0/1828753624 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f06a40a3d20 0x7f06a40a61e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f06b4005fd0 tx=0x7f06b4009680 comp rx=0 tx=0).stop 2024-04-21T08:46:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.261+0000 7f06bf469640 1 -- 172.21.15.43:0/1828753624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06b8101090 msgr2=0x7f06b81986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.261+0000 7f06bf469640 1 --2- 172.21.15.43:0/1828753624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06b8101090 0x7f06b81986b0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f06b0005940 tx=0x7f06b002c7e0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.261+0000 7f06bf469640 1 -- 172.21.15.43:0/1828753624 shutdown_connections 2024-04-21T08:46:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.261+0000 7f06bf469640 1 --2- 172.21.15.43:0/1828753624 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f06a40a3d20 0x7f06a40a61e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.261+0000 7f06bf469640 1 --2- 172.21.15.43:0/1828753624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06b819cf70 0x7f06b819d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.261+0000 7f06bf469640 1 --2- 172.21.15.43:0/1828753624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06b8106160 0x7f06b8198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.261+0000 7f06bf469640 1 --2- 172.21.15.43:0/1828753624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06b8101090 0x7f06b81986b0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.261+0000 7f06bf469640 1 -- 172.21.15.43:0/1828753624 >> 172.21.15.43:0/1828753624 conn(0x7f06b8077f50 msgr2=0x7f06b81000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.262+0000 7f06bf469640 1 -- 172.21.15.43:0/1828753624 shutdown_connections 2024-04-21T08:46:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.262+0000 7f06bf469640 1 -- 172.21.15.43:0/1828753624 wait complete. 2024-04-21T08:46:42.271 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:46:42.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.377+0000 7f9de49f4640 1 -- 172.21.15.43:0/2590443392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9de0101a40 msgr2=0x7f9de0105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.377+0000 7f9de49f4640 1 --2- 172.21.15.43:0/2590443392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9de0101a40 0x7f9de0105a80 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009900 tx=0x7f9dc802bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:42.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.377+0000 7f9de49f4640 1 -- 172.21.15.43:0/2590443392 shutdown_connections 2024-04-21T08:46:42.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.377+0000 7f9de49f4640 1 --2- 172.21.15.43:0/2590443392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de0106160 0x7f9de010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.377+0000 7f9de49f4640 1 --2- 172.21.15.43:0/2590443392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9de0101a40 0x7f9de0105a80 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.377+0000 7f9de49f4640 1 --2- 172.21.15.43:0/2590443392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9de0101090 0x7f9de0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.377+0000 7f9de49f4640 1 -- 172.21.15.43:0/2590443392 >> 172.21.15.43:0/2590443392 conn(0x7f9de00771d0 msgr2=0x7f9de00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.377+0000 7f9de49f4640 1 -- 172.21.15.43:0/2590443392 shutdown_connections 2024-04-21T08:46:42.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.377+0000 7f9de49f4640 1 -- 172.21.15.43:0/2590443392 wait complete. 2024-04-21T08:46:42.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.378+0000 7f9de49f4640 1 Processor -- start 2024-04-21T08:46:42.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.378+0000 7f9de49f4640 1 -- start start 2024-04-21T08:46:42.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.378+0000 7f9de49f4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de0101090 0x7f9de01995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.378+0000 7f9de49f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9de0106160 0x7f9de0199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.378+0000 7f9de49f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9de019a090 0x7f9de01a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.378+0000 7f9de49f4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9de01110a0 con 0x7f9de019a090 2024-04-21T08:46:42.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.378+0000 7f9de49f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9de0110f20 con 0x7f9de0106160 2024-04-21T08:46:42.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.378+0000 7f9de49f4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9de0111220 con 0x7f9de0101090 2024-04-21T08:46:42.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.378+0000 7f9dded9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9de019a090 0x7f9de01a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.378+0000 7f9dded9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9de019a090 0x7f9de01a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54236/0 (socket says 172.21.15.43:54236) 2024-04-21T08:46:42.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.378+0000 7f9dded9d640 1 -- 172.21.15.43:0/561954059 learned_addr learned my addr 172.21.15.43:0/561954059 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.379+0000 7f9dd7fff640 1 --2- 172.21.15.43:0/561954059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9de0106160 0x7f9de0199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.379+0000 7f9dde59c640 1 --2- 172.21.15.43:0/561954059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de0101090 0x7f9de01995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.379+0000 7f9dded9d640 1 -- 172.21.15.43:0/561954059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de0101090 msgr2=0x7f9de01995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.379+0000 7f9dded9d640 1 --2- 172.21.15.43:0/561954059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de0101090 0x7f9de01995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.379+0000 7f9dded9d640 1 -- 172.21.15.43:0/561954059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9de0106160 msgr2=0x7f9de0199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.379+0000 7f9dded9d640 1 --2- 172.21.15.43:0/561954059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9de0106160 0x7f9de0199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.379+0000 7f9dded9d640 1 -- 172.21.15.43:0/561954059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de01a4e40 con 0x7f9de019a090 2024-04-21T08:46:42.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.379+0000 7f9dd7fff640 1 --2- 172.21.15.43:0/561954059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9de0106160 0x7f9de0199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:42.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.379+0000 7f9dded9d640 1 --2- 172.21.15.43:0/561954059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9de019a090 0x7f9de01a4640 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800cad0 tx=0x7f9dd800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.379+0000 7f9dd77fe640 1 -- 172.21.15.43:0/561954059 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd800ae20 con 0x7f9de019a090 2024-04-21T08:46:42.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.379+0000 7f9dd77fe640 1 -- 172.21.15.43:0/561954059 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dd8005020 con 0x7f9de019a090 2024-04-21T08:46:42.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.379+0000 7f9de49f4640 1 -- 172.21.15.43:0/561954059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de01a5130 con 0x7f9de019a090 2024-04-21T08:46:42.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.379+0000 7f9dd77fe640 1 -- 172.21.15.43:0/561954059 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd800f6b0 con 0x7f9de019a090 2024-04-21T08:46:42.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.379+0000 7f9de49f4640 1 -- 172.21.15.43:0/561954059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de01a5650 con 0x7f9de019a090 2024-04-21T08:46:42.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.381+0000 7f9dd77fe640 1 -- 172.21.15.43:0/561954059 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9dd80051d0 con 0x7f9de019a090 2024-04-21T08:46:42.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.381+0000 7f9de75e4740 1 Processor -- start 2024-04-21T08:46:42.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.382+0000 7f9de75e4740 1 -- start start 2024-04-21T08:46:42.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.382+0000 7f9de75e4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649dc8094d0 0x5649dc8098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.382+0000 7f9de75e4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc809e80 0x5649dc812430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.382+0000 7f9de75e4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649dc812a00 0x5649dc814e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.382+0000 7f9de75e4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5649dc76e1f0 con 0x5649dc809e80 2024-04-21T08:46:42.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.382+0000 7f9de75e4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5649dc70b4b0 con 0x5649dc8094d0 2024-04-21T08:46:42.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.382+0000 7f9de75e4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5649dc76a1b0 con 0x5649dc812a00 2024-04-21T08:46:42.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.382+0000 7f9dde59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc809e80 0x5649dc812430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.382+0000 7f9dde59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc809e80 0x5649dc812430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54238/0 (socket says 172.21.15.43:54238) 2024-04-21T08:46:42.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.382+0000 7f9dde59c640 1 -- 172.21.15.43:0/1790171732 learned_addr learned my addr 172.21.15.43:0/1790171732 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.382+0000 7f9dd7fff640 1 --2- 172.21.15.43:0/1790171732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649dc8094d0 0x5649dc8098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.382+0000 7f9dd77fe640 1 --2- 172.21.15.43:0/561954059 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9dcc0a3d20 0x7f9dcc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.382+0000 7f9dd7fff640 1 --2- 172.21.15.43:0/1790171732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649dc812a00 0x5649dc814e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.382+0000 7f9dded9d640 1 --2- 172.21.15.43:0/561954059 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9dcc0a3d20 0x7f9dcc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.382+0000 7f9dde59c640 1 -- 172.21.15.43:0/1790171732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649dc812a00 msgr2=0x5649dc814e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.382+0000 7f9dde59c640 1 --2- 172.21.15.43:0/1790171732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649dc812a00 0x5649dc814e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.382+0000 7f9dde59c640 1 -- 172.21.15.43:0/1790171732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649dc8094d0 msgr2=0x5649dc8098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.382+0000 7f9dde59c640 1 --2- 172.21.15.43:0/1790171732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649dc8094d0 0x5649dc8098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.382+0000 7f9dde59c640 1 -- 172.21.15.43:0/1790171732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649dc6fd1b0 con 0x5649dc809e80 2024-04-21T08:46:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.382+0000 7f9dd77fe640 1 -- 172.21.15.43:0/561954059 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f9dd80d0c60 con 0x7f9de019a090 2024-04-21T08:46:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.383+0000 7f9dde59c640 1 --2- 172.21.15.43:0/1790171732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc809e80 0x5649dc812430 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009bc0 tx=0x7f9dd000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce047ec1da53c53 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.383+0000 7f9dded9d640 1 --2- 172.21.15.43:0/561954059 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9dcc0a3d20 0x7f9dcc0a61e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8099200 tx=0x7f9dd8000f40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.383+0000 7f9dd4ff9640 1 -- 172.21.15.43:0/1790171732 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0016070 con 0x5649dc809e80 2024-04-21T08:46:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.383+0000 7f9dd4ff9640 1 -- 172.21.15.43:0/1790171732 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dd000ee70 con 0x5649dc809e80 2024-04-21T08:46:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.383+0000 7f9dd4ff9640 1 -- 172.21.15.43:0/1790171732 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0011470 con 0x5649dc809e80 2024-04-21T08:46:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.384+0000 7f9de75e4740 1 -- 172.21.15.43:0/1790171732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc809e80 msgr2=0x5649dc812430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.384+0000 7f9de75e4740 1 --2- 172.21.15.43:0/1790171732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc809e80 0x5649dc812430 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009bc0 tx=0x7f9dd000e6c0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.384+0000 7f9de75e4740 1 -- 172.21.15.43:0/1790171732 shutdown_connections 2024-04-21T08:46:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.384+0000 7f9de75e4740 1 --2- 172.21.15.43:0/1790171732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649dc812a00 0x5649dc814e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.384+0000 7f9de75e4740 1 --2- 172.21.15.43:0/1790171732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc809e80 0x5649dc812430 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.384+0000 7f9de75e4740 1 --2- 172.21.15.43:0/1790171732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649dc8094d0 0x5649dc8098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.384+0000 7f9de75e4740 1 -- 172.21.15.43:0/1790171732 >> 172.21.15.43:0/1790171732 conn(0x5649dc803ba0 msgr2=0x5649dc803f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.384+0000 7f9de75e4740 1 -- 172.21.15.43:0/1790171732 shutdown_connections 2024-04-21T08:46:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.384+0000 7f9de75e4740 1 -- 172.21.15.43:0/1790171732 wait complete. 2024-04-21T08:46:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.384+0000 7f9de75e4740 1 -- start start 2024-04-21T08:46:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.384+0000 7f9de75e4740 1 Processor -- start 2024-04-21T08:46:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.384+0000 7f9de75e4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc8094d0 0x5649dc812320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.384+0000 7f9de75e4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649dc809e80 0x5649dc80f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.384+0000 7f9de75e4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649dc812a00 0x5649dc80fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.384+0000 7f9de75e4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5649dc815970 con 0x5649dc8094d0 2024-04-21T08:46:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.384+0000 7f9de75e4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5649dc727500 con 0x5649dc812a00 2024-04-21T08:46:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.384+0000 7f9de75e4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5649dc815af0 con 0x5649dc809e80 2024-04-21T08:46:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.384+0000 7f9de75e4740 10 client.0 _ll_register_callbacks cb 0x5649dc6f5b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.384+0000 7f9dd7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc8094d0 0x5649dc812320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.384+0000 7f9dd7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc8094d0 0x5649dc812320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54250/0 (socket says 172.21.15.43:54250) 2024-04-21T08:46:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.384+0000 7f9dd7fff640 1 -- 172.21.15.43:0/1186906621 learned_addr learned my addr 172.21.15.43:0/1186906621 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.385+0000 7f9dde59c640 1 --2- 172.21.15.43:0/1186906621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649dc809e80 0x5649dc80f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.385+0000 7f9dd7fff640 1 --2- 172.21.15.43:0/1186906621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649dc812a00 0x5649dc80fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.385+0000 7f9dd7fff640 1 -- 172.21.15.43:0/1186906621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649dc809e80 msgr2=0x5649dc80f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.385+0000 7f9dd7fff640 1 --2- 172.21.15.43:0/1186906621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649dc809e80 0x5649dc80f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.385+0000 7f9dd7fff640 1 -- 172.21.15.43:0/1186906621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649dc812a00 msgr2=0x5649dc80fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.385+0000 7f9dd7fff640 1 --2- 172.21.15.43:0/1186906621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649dc812a00 0x5649dc80fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.385+0000 7f9dd7fff640 1 -- 172.21.15.43:0/1186906621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649dc4803f0 con 0x5649dc8094d0 2024-04-21T08:46:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.385+0000 7f9dd7fff640 1 --2- 172.21.15.43:0/1186906621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc8094d0 0x5649dc812320 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f9de006a380 tx=0x7f9dc8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e154e4ad93acc41 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.385+0000 7f9dc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.385+0000 7f9dc5ffb640 1 -- 172.21.15.43:0/1186906621 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8049070 con 0x5649dc8094d0 2024-04-21T08:46:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.385+0000 7f9de75e4740 1 -- 172.21.15.43:0/1186906621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dd000ee70 con 0x5649dc8094d0 2024-04-21T08:46:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.385+0000 7f9dc5ffb640 1 -- 172.21.15.43:0/1186906621 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f9dc8005250 con 0x5649dc8094d0 2024-04-21T08:46:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.385+0000 7f9dc5ffb640 1 -- 172.21.15.43:0/1186906621 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9dc8055050 con 0x5649dc8094d0 2024-04-21T08:46:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.385+0000 7f9de75e4740 10 client.19182 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.385+0000 7f9de75e4740 1 -- 172.21.15.43:0/1186906621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5649dc54a7d0 con 0x5649dc8094d0 2024-04-21T08:46:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.385+0000 7f9dc5ffb640 1 -- 172.21.15.43:0/1186906621 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dd000ee70 con 0x5649dc8094d0 2024-04-21T08:46:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.386+0000 7f9dc5ffb640 1 -- 172.21.15.43:0/1186906621 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9dc8040040 con 0x5649dc8094d0 2024-04-21T08:46:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.386+0000 7f9de75e4740 10 client.19182 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.386+0000 7f9de75e4740 10 client.19182 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:46:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.386+0000 7f9de75e4740 20 client.19182 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.386+0000 7f9de75e4740 1 --2- 172.21.15.43:0/1186906621 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5649dc822bb0 0x5649dc824fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.386+0000 7f9de75e4740 4 client.19182 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.386+0000 7f9de75e4740 1 -- 172.21.15.43:0/1186906621 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649dc70d930 con 0x5649dc822bb0 2024-04-21T08:46:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.386+0000 7f9dde59c640 1 --2- 172.21.15.43:0/1186906621 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5649dc822bb0 0x5649dc824fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.386+0000 7f9dc5ffb640 10 client.19182 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:46:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.386+0000 7f9dde59c640 1 --2- 172.21.15.43:0/1186906621 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5649dc822bb0 0x5649dc824fa0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6d3e35f2699867e6 server_cookie=41dca93ef46531b8 in_seq=0 out_seq=0 2024-04-21T08:46:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.387+0000 7f9dc5ffb640 1 -- 172.21.15.43:0/1186906621 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5649dc70d930 con 0x5649dc822bb0 2024-04-21T08:46:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.387+0000 7f9dc5ffb640 10 client.19182 handle_command_reply: tid=0 2024-04-21T08:46:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.387+0000 7f9de75e4740 1 client.19182 shutdown 2024-04-21T08:46:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.387+0000 7f9de75e4740 1 -- 172.21.15.43:0/1186906621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc8094d0 msgr2=0x5649dc812320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.387+0000 7f9de75e4740 1 --2- 172.21.15.43:0/1186906621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc8094d0 0x5649dc812320 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f9de006a380 tx=0x7f9dc8004040 comp rx=0 tx=0).stop 2024-04-21T08:46:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.387+0000 7f9de75e4740 1 -- 172.21.15.43:0/1186906621 shutdown_connections 2024-04-21T08:46:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.387+0000 7f9de75e4740 1 --2- 172.21.15.43:0/1186906621 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5649dc822bb0 0x5649dc824fa0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.387+0000 7f9de75e4740 1 --2- 172.21.15.43:0/1186906621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649dc812a00 0x5649dc80fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.387+0000 7f9de75e4740 1 --2- 172.21.15.43:0/1186906621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649dc809e80 0x5649dc80f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.387+0000 7f9de75e4740 1 --2- 172.21.15.43:0/1186906621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc8094d0 0x5649dc812320 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.387+0000 7f9dc5ffb640 0 client.19182 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:46:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.387+0000 7f9de75e4740 1 -- 172.21.15.43:0/1186906621 >> 172.21.15.43:0/1186906621 conn(0x5649dc803ba0 msgr2=0x5649dc8058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.387+0000 7f9de75e4740 1 -- 172.21.15.43:0/1186906621 shutdown_connections 2024-04-21T08:46:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.387+0000 7f9de75e4740 1 -- 172.21.15.43:0/1186906621 wait complete. 2024-04-21T08:46:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.388+0000 7f9de75e4740 20 client.19182 trim_cache size 0 max 16384 2024-04-21T08:46:42.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.401+0000 7f9de75e4740 1 Processor -- start 2024-04-21T08:46:42.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.401+0000 7f9de75e4740 1 -- start start 2024-04-21T08:46:42.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.401+0000 7f9de75e4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649dc8094d0 0x5649dc81e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.401+0000 7f9de75e4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649dc809e80 0x5649dc811de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.401+0000 7f9de75e4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc812a00 0x5649dc812320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.401+0000 7f9de75e4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9de01af450 con 0x5649dc812a00 2024-04-21T08:46:42.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.401+0000 7f9de75e4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9de01b0f90 con 0x5649dc809e80 2024-04-21T08:46:42.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.401+0000 7f9de75e4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9de01afb20 con 0x5649dc8094d0 2024-04-21T08:46:42.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.401+0000 7f9dd7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc812a00 0x5649dc812320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.401+0000 7f9dd7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc812a00 0x5649dc812320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54266/0 (socket says 172.21.15.43:54266) 2024-04-21T08:46:42.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.401+0000 7f9dd7fff640 1 -- 172.21.15.43:0/4194581040 learned_addr learned my addr 172.21.15.43:0/4194581040 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.402+0000 7f9dd7fff640 1 -- 172.21.15.43:0/4194581040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649dc8094d0 msgr2=0x5649dc81e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.402+0000 7f9dd7fff640 1 --2- 172.21.15.43:0/4194581040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649dc8094d0 0x5649dc81e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.402+0000 7f9dd7fff640 1 -- 172.21.15.43:0/4194581040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649dc809e80 msgr2=0x5649dc811de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.402+0000 7f9dd7fff640 1 --2- 172.21.15.43:0/4194581040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649dc809e80 0x5649dc811de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.402+0000 7f9dd7fff640 1 -- 172.21.15.43:0/4194581040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649dc80ffc0 con 0x5649dc812a00 2024-04-21T08:46:42.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.402+0000 7f9dd7fff640 1 --2- 172.21.15.43:0/4194581040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc812a00 0x5649dc812320 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8001eb0 tx=0x7f9dc8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bf952b60b7ba80 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.402+0000 7f9dc7fff640 1 -- 172.21.15.43:0/4194581040 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8052070 con 0x5649dc812a00 2024-04-21T08:46:42.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.402+0000 7f9dc7fff640 1 -- 172.21.15.43:0/4194581040 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dc80040b0 con 0x5649dc812a00 2024-04-21T08:46:42.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.402+0000 7f9dc7fff640 1 -- 172.21.15.43:0/4194581040 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8049070 con 0x5649dc812a00 2024-04-21T08:46:42.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.402+0000 7f9de75e4740 1 -- 172.21.15.43:0/4194581040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc812a00 msgr2=0x5649dc812320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.402+0000 7f9de75e4740 1 --2- 172.21.15.43:0/4194581040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc812a00 0x5649dc812320 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8001eb0 tx=0x7f9dc8000bc0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.402+0000 7f9de75e4740 1 -- 172.21.15.43:0/4194581040 shutdown_connections 2024-04-21T08:46:42.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.402+0000 7f9de75e4740 1 --2- 172.21.15.43:0/4194581040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc812a00 0x5649dc812320 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.402+0000 7f9de75e4740 1 --2- 172.21.15.43:0/4194581040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649dc809e80 0x5649dc811de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.402+0000 7f9de75e4740 1 --2- 172.21.15.43:0/4194581040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649dc8094d0 0x5649dc81e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.402+0000 7f9de75e4740 1 -- 172.21.15.43:0/4194581040 >> 172.21.15.43:0/4194581040 conn(0x5649dc803ba0 msgr2=0x5649dc805880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.402+0000 7f9de75e4740 1 -- 172.21.15.43:0/4194581040 shutdown_connections 2024-04-21T08:46:42.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.402+0000 7f9de75e4740 1 -- 172.21.15.43:0/4194581040 wait complete. 2024-04-21T08:46:42.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.403+0000 7f9de75e4740 1 -- start start 2024-04-21T08:46:42.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.403+0000 7f9de75e4740 1 Processor -- start 2024-04-21T08:46:42.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.403+0000 7f9de75e4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649dc8094d0 0x5649dc80c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.403+0000 7f9de75e4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc809e80 0x5649dc80cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.403+0000 7f9de75e4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649dc812a00 0x5649dc80d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.403+0000 7f9de75e4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9de01b08c0 con 0x5649dc809e80 2024-04-21T08:46:42.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.403+0000 7f9de75e4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9de01b01f0 con 0x5649dc8094d0 2024-04-21T08:46:42.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.403+0000 7f9de75e4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9de01ad430 con 0x5649dc812a00 2024-04-21T08:46:42.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.403+0000 7f9de75e4740 10 client.0 _ll_register_callbacks cb 0x7f9dc802c8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:42.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.403+0000 7f9dde59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc809e80 0x5649dc80cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.403+0000 7f9dde59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc809e80 0x5649dc80cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54270/0 (socket says 172.21.15.43:54270) 2024-04-21T08:46:42.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.403+0000 7f9dde59c640 1 -- 172.21.15.43:0/2437638099 learned_addr learned my addr 172.21.15.43:0/2437638099 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.403+0000 7f9dd7fff640 1 --2- 172.21.15.43:0/2437638099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649dc8094d0 0x5649dc80c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.403+0000 7f9dd7fff640 1 --2- 172.21.15.43:0/2437638099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649dc812a00 0x5649dc80d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.403+0000 7f9dde59c640 1 -- 172.21.15.43:0/2437638099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649dc812a00 msgr2=0x5649dc80d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.403+0000 7f9dde59c640 1 --2- 172.21.15.43:0/2437638099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649dc812a00 0x5649dc80d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.403+0000 7f9dde59c640 1 -- 172.21.15.43:0/2437638099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649dc8094d0 msgr2=0x5649dc80c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.403+0000 7f9dde59c640 1 --2- 172.21.15.43:0/2437638099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649dc8094d0 0x5649dc80c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.403+0000 7f9dde59c640 1 -- 172.21.15.43:0/2437638099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649dc88c750 con 0x5649dc809e80 2024-04-21T08:46:42.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.403+0000 7f9dd7fff640 1 --2- 172.21.15.43:0/2437638099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649dc8094d0 0x5649dc80c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:42.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.404+0000 7f9dde59c640 1 --2- 172.21.15.43:0/2437638099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc809e80 0x5649dc80cb50 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009bc0 tx=0x7f9dd000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a363ed595d72fd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.404+0000 7f9dc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:42.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.404+0000 7f9dc5ffb640 1 -- 172.21.15.43:0/2437638099 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0030060 con 0x5649dc809e80 2024-04-21T08:46:42.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.404+0000 7f9dc5ffb640 1 -- 172.21.15.43:0/2437638099 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f9dd0009570 con 0x5649dc809e80 2024-04-21T08:46:42.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.404+0000 7f9de75e4740 1 -- 172.21.15.43:0/2437638099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dc80040b0 con 0x5649dc809e80 2024-04-21T08:46:42.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.404+0000 7f9dc5ffb640 1 -- 172.21.15.43:0/2437638099 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9dd0037050 con 0x5649dc809e80 2024-04-21T08:46:42.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.404+0000 7f9de75e4740 10 client.19188 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:42.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.404+0000 7f9de75e4740 1 -- 172.21.15.43:0/2437638099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5649dc88da30 con 0x5649dc809e80 2024-04-21T08:46:42.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.404+0000 7f9dc5ffb640 1 -- 172.21.15.43:0/2437638099 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dc80040b0 con 0x5649dc809e80 2024-04-21T08:46:42.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.404+0000 7f9dc5ffb640 1 -- 172.21.15.43:0/2437638099 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9dd000f040 con 0x5649dc809e80 2024-04-21T08:46:42.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.404+0000 7f9de75e4740 10 client.19188 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:42.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.404+0000 7f9de75e4740 10 client.19188 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:46:42.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.404+0000 7f9de75e4740 20 client.19188 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:42.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.404+0000 7f9de75e4740 1 --2- 172.21.15.43:0/2437638099 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5649dc88de80 0x5649dc8902a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.404+0000 7f9de75e4740 4 client.19188 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:42.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.404+0000 7f9de75e4740 1 -- 172.21.15.43:0/2437638099 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5649dc77bc10 con 0x5649dc88de80 2024-04-21T08:46:42.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.405+0000 7f9dd7fff640 1 --2- 172.21.15.43:0/2437638099 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5649dc88de80 0x5649dc8902a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.405+0000 7f9dd7fff640 1 --2- 172.21.15.43:0/2437638099 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5649dc88de80 0x5649dc8902a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1b2a1464e299f1b6 server_cookie=34a24f05c55ba306 in_seq=0 out_seq=0 2024-04-21T08:46:42.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.405+0000 7f9dc5ffb640 10 client.19188 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:46:42.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.405+0000 7f9dc5ffb640 1 -- 172.21.15.43:0/2437638099 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5649dc77bc10 con 0x5649dc88de80 2024-04-21T08:46:42.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.405+0000 7f9dc5ffb640 10 client.19188 handle_command_reply: tid=0 2024-04-21T08:46:42.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.405+0000 7f9de75e4740 1 client.19188 shutdown 2024-04-21T08:46:42.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.405+0000 7f9de75e4740 1 -- 172.21.15.43:0/2437638099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc809e80 msgr2=0x5649dc80cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.405+0000 7f9de75e4740 1 --2- 172.21.15.43:0/2437638099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc809e80 0x5649dc80cb50 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009bc0 tx=0x7f9dd000ee70 comp rx=0 tx=0).stop 2024-04-21T08:46:42.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.405+0000 7f9de75e4740 1 -- 172.21.15.43:0/2437638099 shutdown_connections 2024-04-21T08:46:42.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.405+0000 7f9de75e4740 1 --2- 172.21.15.43:0/2437638099 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5649dc88de80 0x5649dc8902a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.405+0000 7f9de75e4740 1 --2- 172.21.15.43:0/2437638099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5649dc812a00 0x5649dc80d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.405+0000 7f9de75e4740 1 --2- 172.21.15.43:0/2437638099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649dc809e80 0x5649dc80cb50 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.405+0000 7f9de75e4740 1 --2- 172.21.15.43:0/2437638099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5649dc8094d0 0x5649dc80c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.405+0000 7f9de75e4740 1 -- 172.21.15.43:0/2437638099 >> 172.21.15.43:0/2437638099 conn(0x5649dc803ba0 msgr2=0x5649dc802370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.405+0000 7f9dc5ffb640 0 client.19188 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:46:42.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.405+0000 7f9de75e4740 1 -- 172.21.15.43:0/2437638099 shutdown_connections 2024-04-21T08:46:42.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.405+0000 7f9de75e4740 1 -- 172.21.15.43:0/2437638099 wait complete. 2024-04-21T08:46:42.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.405+0000 7f9de75e4740 20 client.19188 trim_cache size 0 max 16384 2024-04-21T08:46:42.413 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:46:42.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.406+0000 7f9de49f4640 1 -- 172.21.15.43:0/561954059 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9dcc0a3d20 msgr2=0x7f9dcc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.406+0000 7f9de49f4640 1 --2- 172.21.15.43:0/561954059 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9dcc0a3d20 0x7f9dcc0a61e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8099200 tx=0x7f9dd8000f40 comp rx=0 tx=0).stop 2024-04-21T08:46:42.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.406+0000 7f9de49f4640 1 -- 172.21.15.43:0/561954059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9de019a090 msgr2=0x7f9de01a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.406+0000 7f9de49f4640 1 --2- 172.21.15.43:0/561954059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9de019a090 0x7f9de01a4640 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800cad0 tx=0x7f9dd800cf90 comp rx=0 tx=0).stop 2024-04-21T08:46:42.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.406+0000 7f9de49f4640 1 -- 172.21.15.43:0/561954059 shutdown_connections 2024-04-21T08:46:42.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.406+0000 7f9de49f4640 1 --2- 172.21.15.43:0/561954059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9de019a090 0x7f9de01a4640 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.406+0000 7f9de49f4640 1 --2- 172.21.15.43:0/561954059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9de0106160 0x7f9de0199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.406+0000 7f9de49f4640 1 --2- 172.21.15.43:0/561954059 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9dcc0a3d20 0x7f9dcc0a61e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.406+0000 7f9de49f4640 1 --2- 172.21.15.43:0/561954059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de0101090 0x7f9de01995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.406+0000 7f9de49f4640 1 -- 172.21.15.43:0/561954059 >> 172.21.15.43:0/561954059 conn(0x7f9de00771d0 msgr2=0x7f9de00feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.406+0000 7f9de49f4640 1 -- 172.21.15.43:0/561954059 shutdown_connections 2024-04-21T08:46:42.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.406+0000 7f9de49f4640 1 -- 172.21.15.43:0/561954059 wait complete. 2024-04-21T08:46:42.415 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:42 smithi043 ceph-mon[27978]: pgmap v239: 129 pgs: 129 active+clean; 2.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-21T08:46:42.415 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:42 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/4166528013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:42.416 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:46:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.521+0000 7f274cba4640 1 -- 172.21.15.43:0/3263871637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2748107260 msgr2=0x7f274810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.521+0000 7f274cba4640 1 --2- 172.21.15.43:0/3263871637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2748107260 0x7f274810ee30 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f2740009ca0 tx=0x7f274002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.521+0000 7f274cba4640 1 -- 172.21.15.43:0/3263871637 shutdown_connections 2024-04-21T08:46:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.521+0000 7f274cba4640 1 --2- 172.21.15.43:0/3263871637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2748107260 0x7f274810ee30 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.521+0000 7f274cba4640 1 --2- 172.21.15.43:0/3263871637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2748102b40 0x7f2748106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.521+0000 7f274cba4640 1 --2- 172.21.15.43:0/3263871637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2748102190 0x7f2748102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.521+0000 7f274cba4640 1 -- 172.21.15.43:0/3263871637 >> 172.21.15.43:0/3263871637 conn(0x7f27480fd9f0 msgr2=0x7f27480ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.521+0000 7f274cba4640 1 -- 172.21.15.43:0/3263871637 shutdown_connections 2024-04-21T08:46:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.521+0000 7f274cba4640 1 -- 172.21.15.43:0/3263871637 wait complete. 2024-04-21T08:46:42.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.522+0000 7f274cba4640 1 Processor -- start 2024-04-21T08:46:42.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.522+0000 7f274cba4640 1 -- start start 2024-04-21T08:46:42.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.522+0000 7f274cba4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2748102190 0x7f2748198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.522+0000 7f274cba4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2748107260 0x7f2748198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.522+0000 7f274cba4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f274819d000 0x7f274819d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.522+0000 7f274cba4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f27481121b0 con 0x7f2748107260 2024-04-21T08:46:42.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.522+0000 7f274cba4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2748112030 con 0x7f274819d000 2024-04-21T08:46:42.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.522+0000 7f274cba4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2748112330 con 0x7f2748102190 2024-04-21T08:46:42.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.522+0000 7f2745d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2748107260 0x7f2748198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.522+0000 7f2745d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2748107260 0x7f2748198cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54288/0 (socket says 172.21.15.43:54288) 2024-04-21T08:46:42.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.522+0000 7f2745d9b640 1 -- 172.21.15.43:0/1533566799 learned_addr learned my addr 172.21.15.43:0/1533566799 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.522+0000 7f2746d9d640 1 --2- 172.21.15.43:0/1533566799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f274819d000 0x7f274819d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.522+0000 7f274659c640 1 --2- 172.21.15.43:0/1533566799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2748102190 0x7f2748198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.522+0000 7f2745d9b640 1 -- 172.21.15.43:0/1533566799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2748102190 msgr2=0x7f2748198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.522+0000 7f2745d9b640 1 --2- 172.21.15.43:0/1533566799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2748102190 0x7f2748198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.522+0000 7f2745d9b640 1 -- 172.21.15.43:0/1533566799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f274819d000 msgr2=0x7f274819d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.522+0000 7f2745d9b640 1 --2- 172.21.15.43:0/1533566799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f274819d000 0x7f274819d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.522+0000 7f2745d9b640 1 -- 172.21.15.43:0/1533566799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274819daf0 con 0x7f2748107260 2024-04-21T08:46:42.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.523+0000 7f2745d9b640 1 --2- 172.21.15.43:0/1533566799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2748107260 0x7f2748198cd0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f273c00d770 tx=0x7f273c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.523+0000 7f27377fe640 1 -- 172.21.15.43:0/1533566799 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c017070 con 0x7f2748107260 2024-04-21T08:46:42.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.523+0000 7f27377fe640 1 -- 172.21.15.43:0/1533566799 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f273c009c30 con 0x7f2748107260 2024-04-21T08:46:42.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.523+0000 7f274cba4640 1 -- 172.21.15.43:0/1533566799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274819dde0 con 0x7f2748107260 2024-04-21T08:46:42.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.523+0000 7f27377fe640 1 -- 172.21.15.43:0/1533566799 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c00f620 con 0x7f2748107260 2024-04-21T08:46:42.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.523+0000 7f274cba4640 1 -- 172.21.15.43:0/1533566799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27481a55d0 con 0x7f2748107260 2024-04-21T08:46:42.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.525+0000 7f27377fe640 1 -- 172.21.15.43:0/1533566799 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f273c01a070 con 0x7f2748107260 2024-04-21T08:46:42.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.525+0000 7f274f798740 1 Processor -- start 2024-04-21T08:46:42.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.525+0000 7f274f798740 1 -- start start 2024-04-21T08:46:42.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.525+0000 7f274f798740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55881c91c4d0 0x55881c91c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.525+0000 7f274f798740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91ce80 0x55881c925430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.526+0000 7f274f798740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55881c925a00 0x55881c927e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.526+0000 7f274f798740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55881c8811f0 con 0x55881c91ce80 2024-04-21T08:46:42.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.526+0000 7f274f798740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55881c81e4b0 con 0x55881c925a00 2024-04-21T08:46:42.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.526+0000 7f274f798740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55881c87d1b0 con 0x55881c91c4d0 2024-04-21T08:46:42.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.526+0000 7f274659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91ce80 0x55881c925430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.526+0000 7f274659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91ce80 0x55881c925430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54294/0 (socket says 172.21.15.43:54294) 2024-04-21T08:46:42.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.526+0000 7f274659c640 1 -- 172.21.15.43:0/1060796617 learned_addr learned my addr 172.21.15.43:0/1060796617 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.526+0000 7f27377fe640 1 --2- 172.21.15.43:0/1533566799 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f27300a3d20 0x7f27300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.526+0000 7f2745d9b640 1 --2- 172.21.15.43:0/1060796617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55881c925a00 0x55881c927e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.526+0000 7f2746d9d640 1 --2- 172.21.15.43:0/1060796617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55881c91c4d0 0x55881c91c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.526+0000 7f2746d9d640 1 --2- 172.21.15.43:0/1533566799 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f27300a3d20 0x7f27300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.526+0000 7f274659c640 1 -- 172.21.15.43:0/1060796617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55881c91c4d0 msgr2=0x55881c91c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.526+0000 7f274659c640 1 --2- 172.21.15.43:0/1060796617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55881c91c4d0 0x55881c91c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.526+0000 7f274659c640 1 -- 172.21.15.43:0/1060796617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55881c925a00 msgr2=0x55881c927e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.526+0000 7f274659c640 1 --2- 172.21.15.43:0/1060796617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55881c925a00 0x55881c927e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.526+0000 7f274659c640 1 -- 172.21.15.43:0/1060796617 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55881c8101b0 con 0x55881c91ce80 2024-04-21T08:46:42.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.526+0000 7f27377fe640 1 -- 172.21.15.43:0/1533566799 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f273c0d0660 con 0x7f2748107260 2024-04-21T08:46:42.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.526+0000 7f274659c640 1 --2- 172.21.15.43:0/1060796617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91ce80 0x55881c925430 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f2738009cf0 tx=0x7f2738011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d54f93a164d794 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.526+0000 7f2746d9d640 1 --2- 172.21.15.43:0/1533566799 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f27300a3d20 0x7f27300a61e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2740005e30 tx=0x7f2740005d80 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.526+0000 7f2734ff9640 1 -- 172.21.15.43:0/1060796617 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2738004230 con 0x55881c91ce80 2024-04-21T08:46:42.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.526+0000 7f2734ff9640 1 -- 172.21.15.43:0/1060796617 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27380043d0 con 0x55881c91ce80 2024-04-21T08:46:42.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.526+0000 7f2734ff9640 1 -- 172.21.15.43:0/1060796617 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2738019820 con 0x55881c91ce80 2024-04-21T08:46:42.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.527+0000 7f274f798740 1 -- 172.21.15.43:0/1060796617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91ce80 msgr2=0x55881c925430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.527+0000 7f274f798740 1 --2- 172.21.15.43:0/1060796617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91ce80 0x55881c925430 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f2738009cf0 tx=0x7f2738011810 comp rx=0 tx=0).stop 2024-04-21T08:46:42.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.527+0000 7f274f798740 1 -- 172.21.15.43:0/1060796617 shutdown_connections 2024-04-21T08:46:42.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.527+0000 7f274f798740 1 --2- 172.21.15.43:0/1060796617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55881c925a00 0x55881c927e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.527+0000 7f274f798740 1 --2- 172.21.15.43:0/1060796617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91ce80 0x55881c925430 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.527+0000 7f274f798740 1 --2- 172.21.15.43:0/1060796617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55881c91c4d0 0x55881c91c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.527+0000 7f274f798740 1 -- 172.21.15.43:0/1060796617 >> 172.21.15.43:0/1060796617 conn(0x55881c916ba0 msgr2=0x55881c916f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.527+0000 7f274f798740 1 -- 172.21.15.43:0/1060796617 shutdown_connections 2024-04-21T08:46:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.527+0000 7f274f798740 1 -- 172.21.15.43:0/1060796617 wait complete. 2024-04-21T08:46:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.528+0000 7f274f798740 1 -- start start 2024-04-21T08:46:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.528+0000 7f274f798740 1 Processor -- start 2024-04-21T08:46:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.528+0000 7f274f798740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91c4d0 0x55881c9252a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.528+0000 7f274f798740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55881c91ce80 0x55881c922630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.528+0000 7f274f798740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55881c925a00 0x55881c922b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.528+0000 7f274f798740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55881c928970 con 0x55881c91c4d0 2024-04-21T08:46:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.528+0000 7f274f798740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55881c83a500 con 0x55881c925a00 2024-04-21T08:46:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.528+0000 7f274f798740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55881c928af0 con 0x55881c91ce80 2024-04-21T08:46:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.528+0000 7f274f798740 10 client.0 _ll_register_callbacks cb 0x55881c808b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.528+0000 7f274659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91c4d0 0x55881c9252a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.528+0000 7f274659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91c4d0 0x55881c9252a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54298/0 (socket says 172.21.15.43:54298) 2024-04-21T08:46:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.528+0000 7f274659c640 1 -- 172.21.15.43:0/4280389652 learned_addr learned my addr 172.21.15.43:0/4280389652 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.528+0000 7f2746d9d640 1 --2- 172.21.15.43:0/4280389652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55881c925a00 0x55881c922b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.528+0000 7f274659c640 1 -- 172.21.15.43:0/4280389652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55881c91ce80 msgr2=0x55881c922630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.528+0000 7f2745d9b640 1 --2- 172.21.15.43:0/4280389652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55881c91ce80 0x55881c922630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.528+0000 7f274659c640 1 --2- 172.21.15.43:0/4280389652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55881c91ce80 0x55881c922630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.528+0000 7f274659c640 1 -- 172.21.15.43:0/4280389652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55881c925a00 msgr2=0x55881c922b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.528+0000 7f274659c640 1 --2- 172.21.15.43:0/4280389652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55881c925a00 0x55881c922b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.528+0000 7f274659c640 1 -- 172.21.15.43:0/4280389652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55881c5933f0 con 0x55881c91c4d0 2024-04-21T08:46:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.528+0000 7f2746d9d640 1 --2- 172.21.15.43:0/4280389652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55881c925a00 0x55881c922b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.529+0000 7f274659c640 1 --2- 172.21.15.43:0/4280389652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91c4d0 0x55881c9252a0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f2738003f50 tx=0x7f27380038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38a5b61b2c455af server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.529+0000 7f272dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.529+0000 7f272dffb640 1 -- 172.21.15.43:0/4280389652 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2738027070 con 0x55881c91c4d0 2024-04-21T08:46:42.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.529+0000 7f274f798740 1 -- 172.21.15.43:0/4280389652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27380043d0 con 0x55881c91c4d0 2024-04-21T08:46:42.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.529+0000 7f272dffb640 1 -- 172.21.15.43:0/4280389652 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f2738005240 con 0x55881c91c4d0 2024-04-21T08:46:42.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.529+0000 7f272dffb640 1 -- 172.21.15.43:0/4280389652 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2738033050 con 0x55881c91c4d0 2024-04-21T08:46:42.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.529+0000 7f274f798740 10 client.19200 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:42.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.529+0000 7f274f798740 1 -- 172.21.15.43:0/4280389652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55881c65d7d0 con 0x55881c91c4d0 2024-04-21T08:46:42.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.529+0000 7f272dffb640 1 -- 172.21.15.43:0/4280389652 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27380043d0 con 0x55881c91c4d0 2024-04-21T08:46:42.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.529+0000 7f272dffb640 1 -- 172.21.15.43:0/4280389652 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2738004c00 con 0x55881c91c4d0 2024-04-21T08:46:42.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.529+0000 7f274f798740 10 client.19200 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:42.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.529+0000 7f274f798740 10 client.19200 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:46:42.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.529+0000 7f274f798740 20 client.19200 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:42.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.529+0000 7f274f798740 1 --2- 172.21.15.43:0/4280389652 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55881c935b00 0x55881c937ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.529+0000 7f274f798740 4 client.19200 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:42.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.529+0000 7f274f798740 1 -- 172.21.15.43:0/4280389652 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55881c820930 con 0x55881c935b00 2024-04-21T08:46:42.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.530+0000 7f2745d9b640 1 --2- 172.21.15.43:0/4280389652 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55881c935b00 0x55881c937ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.530+0000 7f2745d9b640 1 --2- 172.21.15.43:0/4280389652 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55881c935b00 0x55881c937ef0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=958ce548d411070c server_cookie=58c3e9b8693c5a7b in_seq=0 out_seq=0 2024-04-21T08:46:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.530+0000 7f272dffb640 10 client.19200 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:46:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.531+0000 7f272dffb640 1 -- 172.21.15.43:0/4280389652 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55881c820930 con 0x55881c935b00 2024-04-21T08:46:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.531+0000 7f272dffb640 10 client.19200 handle_command_reply: tid=0 2024-04-21T08:46:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.531+0000 7f274f798740 1 client.19200 shutdown 2024-04-21T08:46:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.531+0000 7f274f798740 1 -- 172.21.15.43:0/4280389652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91c4d0 msgr2=0x55881c9252a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.531+0000 7f274f798740 1 --2- 172.21.15.43:0/4280389652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91c4d0 0x55881c9252a0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f2738003f50 tx=0x7f27380038a0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.531+0000 7f274f798740 1 -- 172.21.15.43:0/4280389652 shutdown_connections 2024-04-21T08:46:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.531+0000 7f274f798740 1 --2- 172.21.15.43:0/4280389652 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55881c935b00 0x55881c937ef0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.531+0000 7f274f798740 1 --2- 172.21.15.43:0/4280389652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55881c925a00 0x55881c922b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.531+0000 7f274f798740 1 --2- 172.21.15.43:0/4280389652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55881c91ce80 0x55881c922630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.531+0000 7f274f798740 1 --2- 172.21.15.43:0/4280389652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91c4d0 0x55881c9252a0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.531+0000 7f272dffb640 0 client.19200 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:46:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.531+0000 7f274f798740 1 -- 172.21.15.43:0/4280389652 >> 172.21.15.43:0/4280389652 conn(0x55881c916ba0 msgr2=0x55881c9188f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.531+0000 7f274f798740 1 -- 172.21.15.43:0/4280389652 shutdown_connections 2024-04-21T08:46:42.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.531+0000 7f274f798740 1 -- 172.21.15.43:0/4280389652 wait complete. 2024-04-21T08:46:42.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.532+0000 7f274f798740 20 client.19200 trim_cache size 0 max 16384 2024-04-21T08:46:42.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.545+0000 7f274f798740 1 Processor -- start 2024-04-21T08:46:42.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.545+0000 7f274f798740 1 -- start start 2024-04-21T08:46:42.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.545+0000 7f274f798740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55881c91c4d0 0x55881c931830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.545+0000 7f274f798740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91ce80 0x55881c91dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.545+0000 7f274f798740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55881c925a00 0x55881c924d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.545+0000 7f274f798740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f27481af490 con 0x55881c91ce80 2024-04-21T08:46:42.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.545+0000 7f274f798740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27481b0fd0 con 0x55881c91c4d0 2024-04-21T08:46:42.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.545+0000 7f274f798740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f27481afb60 con 0x55881c925a00 2024-04-21T08:46:42.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.545+0000 7f2745d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91ce80 0x55881c91dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.545+0000 7f2745d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91ce80 0x55881c91dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54300/0 (socket says 172.21.15.43:54300) 2024-04-21T08:46:42.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.545+0000 7f2745d9b640 1 -- 172.21.15.43:0/112704859 learned_addr learned my addr 172.21.15.43:0/112704859 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.545+0000 7f274659c640 1 --2- 172.21.15.43:0/112704859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55881c91c4d0 0x55881c931830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.545+0000 7f2746d9d640 1 --2- 172.21.15.43:0/112704859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55881c925a00 0x55881c924d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.545+0000 7f2745d9b640 1 -- 172.21.15.43:0/112704859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55881c925a00 msgr2=0x55881c924d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.545+0000 7f2745d9b640 1 --2- 172.21.15.43:0/112704859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55881c925a00 0x55881c924d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.545+0000 7f2745d9b640 1 -- 172.21.15.43:0/112704859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55881c91c4d0 msgr2=0x55881c931830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.545+0000 7f2745d9b640 1 --2- 172.21.15.43:0/112704859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55881c91c4d0 0x55881c931830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.545+0000 7f2745d9b640 1 -- 172.21.15.43:0/112704859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55881c922ed0 con 0x55881c91ce80 2024-04-21T08:46:42.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.545+0000 7f2746d9d640 1 --2- 172.21.15.43:0/112704859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55881c925a00 0x55881c924d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:42.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.545+0000 7f2745d9b640 1 --2- 172.21.15.43:0/112704859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91ce80 0x55881c91dba0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f273c0986e0 tx=0x7f273c06dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bcff631551ad9c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.546+0000 7f272ffff640 1 -- 172.21.15.43:0/112704859 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c021040 con 0x55881c91ce80 2024-04-21T08:46:42.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.546+0000 7f272ffff640 1 -- 172.21.15.43:0/112704859 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f273c09c280 con 0x55881c91ce80 2024-04-21T08:46:42.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.546+0000 7f272ffff640 1 -- 172.21.15.43:0/112704859 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c0cdc00 con 0x55881c91ce80 2024-04-21T08:46:42.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.546+0000 7f274f798740 1 -- 172.21.15.43:0/112704859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91ce80 msgr2=0x55881c91dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.546+0000 7f274f798740 1 --2- 172.21.15.43:0/112704859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91ce80 0x55881c91dba0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f273c0986e0 tx=0x7f273c06dcc0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.546+0000 7f274f798740 1 -- 172.21.15.43:0/112704859 shutdown_connections 2024-04-21T08:46:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.546+0000 7f274f798740 1 --2- 172.21.15.43:0/112704859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55881c925a00 0x55881c924d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.546+0000 7f274f798740 1 --2- 172.21.15.43:0/112704859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91ce80 0x55881c91dba0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.546+0000 7f274f798740 1 --2- 172.21.15.43:0/112704859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55881c91c4d0 0x55881c931830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.546+0000 7f274f798740 1 -- 172.21.15.43:0/112704859 >> 172.21.15.43:0/112704859 conn(0x55881c916ba0 msgr2=0x55881c915c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.546+0000 7f274f798740 1 -- 172.21.15.43:0/112704859 shutdown_connections 2024-04-21T08:46:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.546+0000 7f274f798740 1 -- 172.21.15.43:0/112704859 wait complete. 2024-04-21T08:46:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.546+0000 7f274f798740 1 -- start start 2024-04-21T08:46:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.547+0000 7f274f798740 1 Processor -- start 2024-04-21T08:46:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.547+0000 7f274f798740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55881c91c4d0 0x55881c92de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.547+0000 7f274f798740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91ce80 0x55881c929300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.547+0000 7f274f798740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55881c925a00 0x55881c929840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.547+0000 7f274f798740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f27481b0900 con 0x55881c91ce80 2024-04-21T08:46:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.547+0000 7f274f798740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27481b0230 con 0x55881c925a00 2024-04-21T08:46:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.547+0000 7f274f798740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f27481ad470 con 0x55881c91c4d0 2024-04-21T08:46:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.547+0000 7f274f798740 10 client.0 _ll_register_callbacks cb 0x7f2738009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:42.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.547+0000 7f2745d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91ce80 0x55881c929300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.547+0000 7f2745d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91ce80 0x55881c929300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54310/0 (socket says 172.21.15.43:54310) 2024-04-21T08:46:42.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.547+0000 7f2745d9b640 1 -- 172.21.15.43:0/1592547182 learned_addr learned my addr 172.21.15.43:0/1592547182 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.547+0000 7f274659c640 1 --2- 172.21.15.43:0/1592547182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55881c91c4d0 0x55881c92de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.547+0000 7f2746d9d640 1 --2- 172.21.15.43:0/1592547182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55881c925a00 0x55881c929840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.547+0000 7f2745d9b640 1 -- 172.21.15.43:0/1592547182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55881c91c4d0 msgr2=0x55881c92de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.547+0000 7f2745d9b640 1 --2- 172.21.15.43:0/1592547182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55881c91c4d0 0x55881c92de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.547+0000 7f2745d9b640 1 -- 172.21.15.43:0/1592547182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55881c925a00 msgr2=0x55881c929840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.547+0000 7f2745d9b640 1 --2- 172.21.15.43:0/1592547182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55881c925a00 0x55881c929840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.547+0000 7f2745d9b640 1 -- 172.21.15.43:0/1592547182 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55881c929d80 con 0x55881c91ce80 2024-04-21T08:46:42.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.547+0000 7f2746d9d640 1 --2- 172.21.15.43:0/1592547182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55881c925a00 0x55881c929840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:42.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.547+0000 7f2745d9b640 1 --2- 172.21.15.43:0/1592547182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91ce80 0x55881c929300 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f273c098c50 tx=0x7f273c0a0670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fc75f5c35a10ef server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.547+0000 7f272dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:42.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.548+0000 7f272dffb640 1 -- 172.21.15.43:0/1592547182 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c014020 con 0x55881c91ce80 2024-04-21T08:46:42.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.548+0000 7f272dffb640 1 -- 172.21.15.43:0/1592547182 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f273c0988f0 con 0x55881c91ce80 2024-04-21T08:46:42.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.548+0000 7f274f798740 1 -- 172.21.15.43:0/1592547182 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f273c09c280 con 0x55881c91ce80 2024-04-21T08:46:42.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.548+0000 7f272dffb640 1 -- 172.21.15.43:0/1592547182 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f273c021030 con 0x55881c91ce80 2024-04-21T08:46:42.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.548+0000 7f274f798740 10 client.19206 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:42.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.548+0000 7f274f798740 1 -- 172.21.15.43:0/1592547182 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55881c99bbf0 con 0x55881c91ce80 2024-04-21T08:46:42.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.548+0000 7f272dffb640 1 -- 172.21.15.43:0/1592547182 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f273c09c280 con 0x55881c91ce80 2024-04-21T08:46:42.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.548+0000 7f272dffb640 1 -- 172.21.15.43:0/1592547182 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f273c0a49b0 con 0x55881c91ce80 2024-04-21T08:46:42.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.548+0000 7f274f798740 10 client.19206 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:42.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.548+0000 7f274f798740 10 client.19206 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:46:42.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.548+0000 7f274f798740 20 client.19206 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:42.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.548+0000 7f274f798740 1 --2- 172.21.15.43:0/1592547182 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55881c99c3c0 0x55881c99e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.548+0000 7f274f798740 4 client.19206 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:42.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.548+0000 7f274f798740 1 -- 172.21.15.43:0/1592547182 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55881c88ec10 con 0x55881c99c3c0 2024-04-21T08:46:42.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.548+0000 7f274659c640 1 --2- 172.21.15.43:0/1592547182 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55881c99c3c0 0x55881c99e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.549+0000 7f274659c640 1 --2- 172.21.15.43:0/1592547182 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55881c99c3c0 0x55881c99e7e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db19957d6ceab23 server_cookie=267e3c57b95ee293 in_seq=0 out_seq=0 2024-04-21T08:46:42.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.549+0000 7f272dffb640 10 client.19206 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:46:42.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.549+0000 7f272dffb640 1 -- 172.21.15.43:0/1592547182 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55881c88ec10 con 0x55881c99c3c0 2024-04-21T08:46:42.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.549+0000 7f272dffb640 10 client.19206 handle_command_reply: tid=0 2024-04-21T08:46:42.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.549+0000 7f274f798740 1 client.19206 shutdown 2024-04-21T08:46:42.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.549+0000 7f274f798740 1 -- 172.21.15.43:0/1592547182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91ce80 msgr2=0x55881c929300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.549+0000 7f274f798740 1 --2- 172.21.15.43:0/1592547182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91ce80 0x55881c929300 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f273c098c50 tx=0x7f273c0a0670 comp rx=0 tx=0).stop 2024-04-21T08:46:42.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.549+0000 7f274f798740 1 -- 172.21.15.43:0/1592547182 shutdown_connections 2024-04-21T08:46:42.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.549+0000 7f274f798740 1 --2- 172.21.15.43:0/1592547182 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55881c99c3c0 0x55881c99e7e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.549+0000 7f274f798740 1 --2- 172.21.15.43:0/1592547182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55881c925a00 0x55881c929840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.549+0000 7f274f798740 1 --2- 172.21.15.43:0/1592547182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55881c91ce80 0x55881c929300 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.549+0000 7f272dffb640 0 client.19206 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:46:42.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.549+0000 7f274f798740 1 --2- 172.21.15.43:0/1592547182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55881c91c4d0 0x55881c92de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.549+0000 7f274f798740 1 -- 172.21.15.43:0/1592547182 >> 172.21.15.43:0/1592547182 conn(0x55881c916ba0 msgr2=0x55881c915c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.549+0000 7f274f798740 1 -- 172.21.15.43:0/1592547182 shutdown_connections 2024-04-21T08:46:42.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.549+0000 7f274f798740 1 -- 172.21.15.43:0/1592547182 wait complete. 2024-04-21T08:46:42.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.549+0000 7f274f798740 20 client.19206 trim_cache size 0 max 16384 2024-04-21T08:46:42.557 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:46:42.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.550+0000 7f274cba4640 1 -- 172.21.15.43:0/1533566799 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f27300a3d20 msgr2=0x7f27300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.550+0000 7f274cba4640 1 --2- 172.21.15.43:0/1533566799 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f27300a3d20 0x7f27300a61e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2740005e30 tx=0x7f2740005d80 comp rx=0 tx=0).stop 2024-04-21T08:46:42.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.550+0000 7f274cba4640 1 -- 172.21.15.43:0/1533566799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2748107260 msgr2=0x7f2748198cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.550+0000 7f274cba4640 1 --2- 172.21.15.43:0/1533566799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2748107260 0x7f2748198cd0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f273c00d770 tx=0x7f273c00dc30 comp rx=0 tx=0).stop 2024-04-21T08:46:42.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.550+0000 7f274cba4640 1 -- 172.21.15.43:0/1533566799 shutdown_connections 2024-04-21T08:46:42.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.550+0000 7f274cba4640 1 --2- 172.21.15.43:0/1533566799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f274819d000 0x7f274819d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.550+0000 7f274cba4640 1 --2- 172.21.15.43:0/1533566799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2748107260 0x7f2748198cd0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.550+0000 7f274cba4640 1 --2- 172.21.15.43:0/1533566799 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f27300a3d20 0x7f27300a61e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.550+0000 7f274cba4640 1 --2- 172.21.15.43:0/1533566799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2748102190 0x7f2748198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.550+0000 7f274cba4640 1 -- 172.21.15.43:0/1533566799 >> 172.21.15.43:0/1533566799 conn(0x7f27480fd9f0 msgr2=0x7f27480feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.550+0000 7f274cba4640 1 -- 172.21.15.43:0/1533566799 shutdown_connections 2024-04-21T08:46:42.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.550+0000 7f274cba4640 1 -- 172.21.15.43:0/1533566799 wait complete. 2024-04-21T08:46:42.560 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:46:42.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.666+0000 7f5f33881640 1 -- 172.21.15.43:0/1946608655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f2c104400 msgr2=0x7f5f2c1047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.666+0000 7f5f33881640 1 --2- 172.21.15.43:0/1946608655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f2c104400 0x7f5f2c1047e0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009900 tx=0x7f5f2402bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:42.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.666+0000 7f5f33881640 1 -- 172.21.15.43:0/1946608655 shutdown_connections 2024-04-21T08:46:42.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.666+0000 7f5f33881640 1 --2- 172.21.15.43:0/1946608655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f2c1094d0 0x7f5f2c1110a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.666+0000 7f5f33881640 1 --2- 172.21.15.43:0/1946608655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f2c104db0 0x7f5f2c108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.666+0000 7f5f33881640 1 --2- 172.21.15.43:0/1946608655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f2c104400 0x7f5f2c1047e0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.666+0000 7f5f33881640 1 -- 172.21.15.43:0/1946608655 >> 172.21.15.43:0/1946608655 conn(0x7f5f2c0ffce0 msgr2=0x7f5f2c102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.666+0000 7f5f33881640 1 -- 172.21.15.43:0/1946608655 shutdown_connections 2024-04-21T08:46:42.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.666+0000 7f5f33881640 1 -- 172.21.15.43:0/1946608655 wait complete. 2024-04-21T08:46:42.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.666+0000 7f5f33881640 1 Processor -- start 2024-04-21T08:46:42.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.667+0000 7f5f33881640 1 -- start start 2024-04-21T08:46:42.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.667+0000 7f5f33881640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f2c104400 0x7f5f2c1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.667+0000 7f5f33881640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f2c1094d0 0x7f5f2c1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.667+0000 7f5f33881640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f2c19a880 0x7f5f2c19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.667+0000 7f5f33881640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c114430 con 0x7f5f2c104400 2024-04-21T08:46:42.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.667+0000 7f5f33881640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c1142b0 con 0x7f5f2c19a880 2024-04-21T08:46:42.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.667+0000 7f5f33881640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c1145b0 con 0x7f5f2c1094d0 2024-04-21T08:46:42.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.667+0000 7f5f3161d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f2c104400 0x7f5f2c1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.667+0000 7f5f3161d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f2c104400 0x7f5f2c1a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54324/0 (socket says 172.21.15.43:54324) 2024-04-21T08:46:42.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.667+0000 7f5f3161d640 1 -- 172.21.15.43:0/939963546 learned_addr learned my addr 172.21.15.43:0/939963546 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.667+0000 7f5f31e1e640 1 --2- 172.21.15.43:0/939963546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f2c19a880 0x7f5f2c19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.667+0000 7f5f30e1c640 1 --2- 172.21.15.43:0/939963546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f2c1094d0 0x7f5f2c1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.667+0000 7f5f3161d640 1 -- 172.21.15.43:0/939963546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f2c1094d0 msgr2=0x7f5f2c1a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.667+0000 7f5f3161d640 1 --2- 172.21.15.43:0/939963546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f2c1094d0 0x7f5f2c1a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.667+0000 7f5f3161d640 1 -- 172.21.15.43:0/939963546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f2c19a880 msgr2=0x7f5f2c19ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.667+0000 7f5f3161d640 1 --2- 172.21.15.43:0/939963546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f2c19a880 0x7f5f2c19ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.667+0000 7f5f3161d640 1 -- 172.21.15.43:0/939963546 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2c19b410 con 0x7f5f2c104400 2024-04-21T08:46:42.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.667+0000 7f5f3161d640 1 --2- 172.21.15.43:0/939963546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f2c104400 0x7f5f2c1a06c0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f5f24005940 tx=0x7f5f2402c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.668+0000 7f5f227fc640 1 -- 172.21.15.43:0/939963546 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24043020 con 0x7f5f2c104400 2024-04-21T08:46:42.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.668+0000 7f5f227fc640 1 -- 172.21.15.43:0/939963546 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f2402ca70 con 0x7f5f2c104400 2024-04-21T08:46:42.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.668+0000 7f5f33881640 1 -- 172.21.15.43:0/939963546 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f2c19b6a0 con 0x7f5f2c104400 2024-04-21T08:46:42.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.668+0000 7f5f227fc640 1 -- 172.21.15.43:0/939963546 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f240039f0 con 0x7f5f2c104400 2024-04-21T08:46:42.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.668+0000 7f5f33881640 1 -- 172.21.15.43:0/939963546 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f2c1a7830 con 0x7f5f2c104400 2024-04-21T08:46:42.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.670+0000 7f5f227fc640 1 -- 172.21.15.43:0/939963546 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5f24046020 con 0x7f5f2c104400 2024-04-21T08:46:42.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.670+0000 7f5f36479740 1 Processor -- start 2024-04-21T08:46:42.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.670+0000 7f5f36479740 1 -- start start 2024-04-21T08:46:42.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.670+0000 7f5f36479740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb950194d0 0x55bb950198b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.671+0000 7f5f36479740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb95019e80 0x55bb95022430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.671+0000 7f5f36479740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb95022a00 0x55bb95024e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.671+0000 7f5f36479740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bb94f7e1f0 con 0x55bb95022a00 2024-04-21T08:46:42.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.671+0000 7f5f36479740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bb94f1b4b0 con 0x55bb95019e80 2024-04-21T08:46:42.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.671+0000 7f5f36479740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bb94f7a1b0 con 0x55bb950194d0 2024-04-21T08:46:42.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.671+0000 7f5f30e1c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb95022a00 0x55bb95024e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.671+0000 7f5f30e1c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb95022a00 0x55bb95024e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54326/0 (socket says 172.21.15.43:54326) 2024-04-21T08:46:42.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.671+0000 7f5f30e1c640 1 -- 172.21.15.43:0/2941355102 learned_addr learned my addr 172.21.15.43:0/2941355102 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.671+0000 7f5f227fc640 1 --2- 172.21.15.43:0/939963546 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5f180a3d20 0x7f5f180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.671+0000 7f5f31e1e640 1 --2- 172.21.15.43:0/2941355102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb950194d0 0x55bb950198b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.671+0000 7f5f3161d640 1 --2- 172.21.15.43:0/2941355102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb95019e80 0x55bb95022430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.671+0000 7f5f30e1c640 1 -- 172.21.15.43:0/2941355102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb950194d0 msgr2=0x55bb950198b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.671+0000 7f5f30e1c640 1 --2- 172.21.15.43:0/2941355102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb950194d0 0x55bb950198b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.671+0000 7f5f30e1c640 1 -- 172.21.15.43:0/2941355102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb95019e80 msgr2=0x55bb95022430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.671+0000 7f5f30e1c640 1 --2- 172.21.15.43:0/2941355102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb95019e80 0x55bb95022430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.671+0000 7f5f30e1c640 1 -- 172.21.15.43:0/2941355102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb94f0d1b0 con 0x55bb95022a00 2024-04-21T08:46:42.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.671+0000 7f5f227fc640 1 -- 172.21.15.43:0/939963546 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f5f240f2c60 con 0x7f5f2c104400 2024-04-21T08:46:42.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.671+0000 7f5f3161d640 1 --2- 172.21.15.43:0/2941355102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb95019e80 0x55bb95022430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:42.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.671+0000 7f5f31e1e640 1 --2- 172.21.15.43:0/939963546 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5f180a3d20 0x7f5f180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.671+0000 7f5f30e1c640 1 --2- 172.21.15.43:0/2941355102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb95022a00 0x55bb95024e50 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009bb0 tx=0x7f5f1c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777b5a7b28f71d01 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.671+0000 7f5f177fe640 1 -- 172.21.15.43:0/2941355102 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c005020 con 0x55bb95022a00 2024-04-21T08:46:42.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.672+0000 7f5f31e1e640 1 --2- 172.21.15.43:0/939963546 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5f180a3d20 0x7f5f180a61e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5f280094d0 tx=0x7f5f28004a40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.672+0000 7f5f177fe640 1 -- 172.21.15.43:0/2941355102 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f1c0047a0 con 0x55bb95022a00 2024-04-21T08:46:42.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.672+0000 7f5f177fe640 1 -- 172.21.15.43:0/2941355102 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c0126b0 con 0x55bb95022a00 2024-04-21T08:46:42.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.672+0000 7f5f36479740 1 -- 172.21.15.43:0/2941355102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb95022a00 msgr2=0x55bb95024e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.672+0000 7f5f36479740 1 --2- 172.21.15.43:0/2941355102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb95022a00 0x55bb95024e50 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009bb0 tx=0x7f5f1c0117b0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.672+0000 7f5f36479740 1 -- 172.21.15.43:0/2941355102 shutdown_connections 2024-04-21T08:46:42.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.672+0000 7f5f36479740 1 --2- 172.21.15.43:0/2941355102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb95022a00 0x55bb95024e50 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.672+0000 7f5f36479740 1 --2- 172.21.15.43:0/2941355102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb95019e80 0x55bb95022430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.672+0000 7f5f36479740 1 --2- 172.21.15.43:0/2941355102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb950194d0 0x55bb950198b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.672+0000 7f5f36479740 1 -- 172.21.15.43:0/2941355102 >> 172.21.15.43:0/2941355102 conn(0x55bb95013ba0 msgr2=0x55bb95013f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.672+0000 7f5f36479740 1 -- 172.21.15.43:0/2941355102 shutdown_connections 2024-04-21T08:46:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.672+0000 7f5f36479740 1 -- 172.21.15.43:0/2941355102 wait complete. 2024-04-21T08:46:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.673+0000 7f5f36479740 1 -- start start 2024-04-21T08:46:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.673+0000 7f5f36479740 1 Processor -- start 2024-04-21T08:46:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.673+0000 7f5f36479740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb950194d0 0x55bb950222d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.673+0000 7f5f36479740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb95019e80 0x55bb9501f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.673+0000 7f5f36479740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb95022a00 0x55bb9501fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.673+0000 7f5f36479740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bb95025970 con 0x55bb95022a00 2024-04-21T08:46:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.673+0000 7f5f36479740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bb94f37500 con 0x55bb95019e80 2024-04-21T08:46:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.673+0000 7f5f36479740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bb95025af0 con 0x55bb950194d0 2024-04-21T08:46:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.673+0000 7f5f36479740 10 client.0 _ll_register_callbacks cb 0x55bb94f05b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.673+0000 7f5f31e1e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb95022a00 0x55bb9501fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.673+0000 7f5f31e1e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb95022a00 0x55bb9501fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54340/0 (socket says 172.21.15.43:54340) 2024-04-21T08:46:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.673+0000 7f5f31e1e640 1 -- 172.21.15.43:0/2319904270 learned_addr learned my addr 172.21.15.43:0/2319904270 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.674+0000 7f5f3161d640 1 --2- 172.21.15.43:0/2319904270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb950194d0 0x55bb950222d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.674+0000 7f5f30e1c640 1 --2- 172.21.15.43:0/2319904270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb95019e80 0x55bb9501f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.674+0000 7f5f31e1e640 1 -- 172.21.15.43:0/2319904270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb950194d0 msgr2=0x55bb950222d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.674+0000 7f5f31e1e640 1 --2- 172.21.15.43:0/2319904270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb950194d0 0x55bb950222d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.674+0000 7f5f31e1e640 1 -- 172.21.15.43:0/2319904270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb95019e80 msgr2=0x55bb9501f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.674+0000 7f5f31e1e640 1 --2- 172.21.15.43:0/2319904270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb95019e80 0x55bb9501f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.674+0000 7f5f31e1e640 1 -- 172.21.15.43:0/2319904270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb94c903f0 con 0x55bb95022a00 2024-04-21T08:46:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.674+0000 7f5f31e1e640 1 --2- 172.21.15.43:0/2319904270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb95022a00 0x55bb9501fbc0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f5f28011c80 tx=0x7f5f28004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12cbde1a41961818 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.674+0000 7f5f14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.674+0000 7f5f14ff9640 1 -- 172.21.15.43:0/2319904270 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f28027640 con 0x55bb95022a00 2024-04-21T08:46:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.674+0000 7f5f14ff9640 1 -- 172.21.15.43:0/2319904270 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f5f2801b410 con 0x55bb95022a00 2024-04-21T08:46:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.674+0000 7f5f36479740 1 -- 172.21.15.43:0/2319904270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f1c0047a0 con 0x55bb95022a00 2024-04-21T08:46:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.674+0000 7f5f14ff9640 1 -- 172.21.15.43:0/2319904270 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5f280277e0 con 0x55bb95022a00 2024-04-21T08:46:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.674+0000 7f5f36479740 10 client.19218 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:42.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.674+0000 7f5f36479740 1 -- 172.21.15.43:0/2319904270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bb94d5a7d0 con 0x55bb95022a00 2024-04-21T08:46:42.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.674+0000 7f5f14ff9640 1 -- 172.21.15.43:0/2319904270 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f1c0047a0 con 0x55bb95022a00 2024-04-21T08:46:42.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.675+0000 7f5f14ff9640 1 -- 172.21.15.43:0/2319904270 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5f28025070 con 0x55bb95022a00 2024-04-21T08:46:42.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.675+0000 7f5f36479740 10 client.19218 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:42.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.675+0000 7f5f36479740 10 client.19218 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:46:42.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.675+0000 7f5f36479740 20 client.19218 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:42.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.675+0000 7f5f36479740 1 --2- 172.21.15.43:0/2319904270 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55bb95032c80 0x55bb950350d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.675+0000 7f5f36479740 4 client.19218 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:42.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.675+0000 7f5f36479740 1 -- 172.21.15.43:0/2319904270 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb94f1d930 con 0x55bb95032c80 2024-04-21T08:46:42.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.675+0000 7f5f3161d640 1 --2- 172.21.15.43:0/2319904270 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55bb95032c80 0x55bb950350d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.675+0000 7f5f3161d640 1 --2- 172.21.15.43:0/2319904270 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55bb95032c80 0x55bb950350d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bab12d93218c73d4 server_cookie=23f569f57a29e9a6 in_seq=0 out_seq=0 2024-04-21T08:46:42.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.675+0000 7f5f14ff9640 10 client.19218 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:46:42.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.676+0000 7f5f14ff9640 1 -- 172.21.15.43:0/2319904270 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bb94f1d930 con 0x55bb95032c80 2024-04-21T08:46:42.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.676+0000 7f5f14ff9640 10 client.19218 handle_command_reply: tid=0 2024-04-21T08:46:42.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.676+0000 7f5f36479740 1 client.19218 shutdown 2024-04-21T08:46:42.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.676+0000 7f5f36479740 1 -- 172.21.15.43:0/2319904270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb95022a00 msgr2=0x55bb9501fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.676+0000 7f5f36479740 1 --2- 172.21.15.43:0/2319904270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb95022a00 0x55bb9501fbc0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f5f28011c80 tx=0x7f5f28004ab0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.676+0000 7f5f36479740 1 -- 172.21.15.43:0/2319904270 shutdown_connections 2024-04-21T08:46:42.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.676+0000 7f5f36479740 1 --2- 172.21.15.43:0/2319904270 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55bb95032c80 0x55bb950350d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.676+0000 7f5f36479740 1 --2- 172.21.15.43:0/2319904270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb95022a00 0x55bb9501fbc0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.676+0000 7f5f36479740 1 --2- 172.21.15.43:0/2319904270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb95019e80 0x55bb9501f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.676+0000 7f5f14ff9640 0 client.19218 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:46:42.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.676+0000 7f5f36479740 1 --2- 172.21.15.43:0/2319904270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb950194d0 0x55bb950222d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.676+0000 7f5f36479740 1 -- 172.21.15.43:0/2319904270 >> 172.21.15.43:0/2319904270 conn(0x55bb95013ba0 msgr2=0x55bb950158f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.676+0000 7f5f36479740 1 -- 172.21.15.43:0/2319904270 shutdown_connections 2024-04-21T08:46:42.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.676+0000 7f5f36479740 1 -- 172.21.15.43:0/2319904270 wait complete. 2024-04-21T08:46:42.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.677+0000 7f5f36479740 20 client.19218 trim_cache size 0 max 16384 2024-04-21T08:46:42.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.690+0000 7f5f36479740 1 Processor -- start 2024-04-21T08:46:42.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.690+0000 7f5f36479740 1 -- start start 2024-04-21T08:46:42.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.690+0000 7f5f36479740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb950194d0 0x55bb9502e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.690+0000 7f5f36479740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb95019e80 0x55bb95021d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.690+0000 7f5f36479740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb95022a00 0x55bb950222d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.690+0000 7f5f36479740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c1b1560 con 0x55bb950194d0 2024-04-21T08:46:42.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.690+0000 7f5f36479740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c1b30a0 con 0x55bb95022a00 2024-04-21T08:46:42.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.690+0000 7f5f36479740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c1b1c30 con 0x55bb95019e80 2024-04-21T08:46:42.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.690+0000 7f5f3161d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb950194d0 0x55bb9502e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.690+0000 7f5f3161d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb950194d0 0x55bb9502e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54342/0 (socket says 172.21.15.43:54342) 2024-04-21T08:46:42.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.690+0000 7f5f3161d640 1 -- 172.21.15.43:0/130789331 learned_addr learned my addr 172.21.15.43:0/130789331 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.690+0000 7f5f30e1c640 1 --2- 172.21.15.43:0/130789331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb95019e80 0x55bb95021d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.690+0000 7f5f3161d640 1 -- 172.21.15.43:0/130789331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb95019e80 msgr2=0x55bb95021d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.690+0000 7f5f3161d640 1 --2- 172.21.15.43:0/130789331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb95019e80 0x55bb95021d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.690+0000 7f5f3161d640 1 -- 172.21.15.43:0/130789331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb95022a00 msgr2=0x55bb950222d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.690+0000 7f5f31e1e640 1 --2- 172.21.15.43:0/130789331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb95022a00 0x55bb950222d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.690+0000 7f5f3161d640 1 --2- 172.21.15.43:0/130789331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb95022a00 0x55bb950222d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.691+0000 7f5f3161d640 1 -- 172.21.15.43:0/130789331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb9501fd30 con 0x55bb950194d0 2024-04-21T08:46:42.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.691+0000 7f5f30e1c640 1 --2- 172.21.15.43:0/130789331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb95019e80 0x55bb95021d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:42.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.691+0000 7f5f3161d640 1 --2- 172.21.15.43:0/130789331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb950194d0 0x55bb9502e830 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f5f240bb2b0 tx=0x7f5f2404a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ac62ba6e5b4987 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.691+0000 7f5f16ffd640 1 -- 172.21.15.43:0/130789331 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f240be8e0 con 0x55bb950194d0 2024-04-21T08:46:42.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.691+0000 7f5f16ffd640 1 -- 172.21.15.43:0/130789331 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f240c2c90 con 0x55bb950194d0 2024-04-21T08:46:42.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.691+0000 7f5f16ffd640 1 -- 172.21.15.43:0/130789331 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f240bb0e0 con 0x55bb950194d0 2024-04-21T08:46:42.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.691+0000 7f5f36479740 1 -- 172.21.15.43:0/130789331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb950194d0 msgr2=0x55bb9502e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.691+0000 7f5f36479740 1 --2- 172.21.15.43:0/130789331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb950194d0 0x55bb9502e830 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f5f240bb2b0 tx=0x7f5f2404a040 comp rx=0 tx=0).stop 2024-04-21T08:46:42.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.691+0000 7f5f36479740 1 -- 172.21.15.43:0/130789331 shutdown_connections 2024-04-21T08:46:42.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.691+0000 7f5f36479740 1 --2- 172.21.15.43:0/130789331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb95022a00 0x55bb950222d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.691+0000 7f5f36479740 1 --2- 172.21.15.43:0/130789331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb95019e80 0x55bb95021d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.691+0000 7f5f36479740 1 --2- 172.21.15.43:0/130789331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb950194d0 0x55bb9502e830 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.691+0000 7f5f36479740 1 -- 172.21.15.43:0/130789331 >> 172.21.15.43:0/130789331 conn(0x55bb95013ba0 msgr2=0x55bb9501f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.691+0000 7f5f36479740 1 -- 172.21.15.43:0/130789331 shutdown_connections 2024-04-21T08:46:42.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.692+0000 7f5f36479740 1 -- 172.21.15.43:0/130789331 wait complete. 2024-04-21T08:46:42.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.692+0000 7f5f36479740 1 -- start start 2024-04-21T08:46:42.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.692+0000 7f5f36479740 1 Processor -- start 2024-04-21T08:46:42.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.692+0000 7f5f36479740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb950194d0 0x55bb9502b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.692+0000 7f5f36479740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb95019e80 0x55bb95026430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.692+0000 7f5f36479740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb95022a00 0x55bb95026970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.692+0000 7f5f36479740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c1b29d0 con 0x55bb95019e80 2024-04-21T08:46:42.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.692+0000 7f5f36479740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c1b2300 con 0x55bb950194d0 2024-04-21T08:46:42.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.692+0000 7f5f36479740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c1af540 con 0x55bb95022a00 2024-04-21T08:46:42.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.692+0000 7f5f36479740 10 client.0 _ll_register_callbacks cb 0x7f5f28011900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:42.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.692+0000 7f5f30e1c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb95019e80 0x55bb95026430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.692+0000 7f5f30e1c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb95019e80 0x55bb95026430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54358/0 (socket says 172.21.15.43:54358) 2024-04-21T08:46:42.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.692+0000 7f5f30e1c640 1 -- 172.21.15.43:0/72745218 learned_addr learned my addr 172.21.15.43:0/72745218 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.693+0000 7f5f3161d640 1 --2- 172.21.15.43:0/72745218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb950194d0 0x55bb9502b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.693+0000 7f5f31e1e640 1 --2- 172.21.15.43:0/72745218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb95022a00 0x55bb95026970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.693+0000 7f5f30e1c640 1 -- 172.21.15.43:0/72745218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb95022a00 msgr2=0x55bb95026970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.693+0000 7f5f30e1c640 1 --2- 172.21.15.43:0/72745218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb95022a00 0x55bb95026970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.693+0000 7f5f30e1c640 1 -- 172.21.15.43:0/72745218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb950194d0 msgr2=0x55bb9502b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.693+0000 7f5f30e1c640 1 --2- 172.21.15.43:0/72745218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb950194d0 0x55bb9502b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.693+0000 7f5f30e1c640 1 -- 172.21.15.43:0/72745218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb95027170 con 0x55bb95019e80 2024-04-21T08:46:42.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.693+0000 7f5f31e1e640 1 --2- 172.21.15.43:0/72745218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb95022a00 0x55bb95026970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:42.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.693+0000 7f5f3161d640 1 --2- 172.21.15.43:0/72745218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb950194d0 0x55bb9502b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:42.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.693+0000 7f5f30e1c640 1 --2- 172.21.15.43:0/72745218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb95019e80 0x55bb95026430 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009bb0 tx=0x7f5f1c005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48b6037e288ec83 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.693+0000 7f5f14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:42.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.693+0000 7f5f14ff9640 1 -- 172.21.15.43:0/72745218 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c01f040 con 0x55bb95019e80 2024-04-21T08:46:42.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.693+0000 7f5f14ff9640 1 -- 172.21.15.43:0/72745218 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f5f1c005570 con 0x55bb95019e80 2024-04-21T08:46:42.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.693+0000 7f5f36479740 1 -- 172.21.15.43:0/72745218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f240c2c90 con 0x55bb95019e80 2024-04-21T08:46:42.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.693+0000 7f5f14ff9640 1 -- 172.21.15.43:0/72745218 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5f1c030050 con 0x55bb95019e80 2024-04-21T08:46:42.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.693+0000 7f5f36479740 10 client.19224 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:42.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.693+0000 7f5f36479740 1 -- 172.21.15.43:0/72745218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bb95099180 con 0x55bb95019e80 2024-04-21T08:46:42.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.693+0000 7f5f14ff9640 1 -- 172.21.15.43:0/72745218 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f240c2c90 con 0x55bb95019e80 2024-04-21T08:46:42.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.693+0000 7f5f14ff9640 1 -- 172.21.15.43:0/72745218 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5f1c02b030 con 0x55bb95019e80 2024-04-21T08:46:42.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.693+0000 7f5f36479740 10 client.19224 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:42.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.693+0000 7f5f36479740 10 client.19224 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:46:42.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.694+0000 7f5f36479740 20 client.19224 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:42.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.694+0000 7f5f36479740 1 --2- 172.21.15.43:0/72745218 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55bb95032c80 0x55bb9509b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.694+0000 7f5f36479740 4 client.19224 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:42.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.694+0000 7f5f36479740 1 -- 172.21.15.43:0/72745218 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb94f8bc10 con 0x55bb95032c80 2024-04-21T08:46:42.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.694+0000 7f5f3161d640 1 --2- 172.21.15.43:0/72745218 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55bb95032c80 0x55bb9509b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.694+0000 7f5f14ff9640 10 client.19224 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:46:42.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.694+0000 7f5f3161d640 1 --2- 172.21.15.43:0/72745218 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55bb95032c80 0x55bb9509b790 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e5dfb103ea62033 server_cookie=5259b3a834de41a1 in_seq=0 out_seq=0 2024-04-21T08:46:42.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.694+0000 7f5f14ff9640 1 -- 172.21.15.43:0/72745218 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb94f8bc10 con 0x55bb95032c80 2024-04-21T08:46:42.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.694+0000 7f5f14ff9640 10 client.19224 handle_command_reply: tid=0 2024-04-21T08:46:42.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.694+0000 7f5f36479740 1 client.19224 shutdown 2024-04-21T08:46:42.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.694+0000 7f5f36479740 1 -- 172.21.15.43:0/72745218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb95019e80 msgr2=0x55bb95026430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.694+0000 7f5f36479740 1 --2- 172.21.15.43:0/72745218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb95019e80 0x55bb95026430 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009bb0 tx=0x7f5f1c005280 comp rx=0 tx=0).stop 2024-04-21T08:46:42.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.694+0000 7f5f36479740 1 -- 172.21.15.43:0/72745218 shutdown_connections 2024-04-21T08:46:42.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.694+0000 7f5f36479740 1 --2- 172.21.15.43:0/72745218 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55bb95032c80 0x55bb9509b790 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.694+0000 7f5f36479740 1 --2- 172.21.15.43:0/72745218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb95022a00 0x55bb95026970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.694+0000 7f5f36479740 1 --2- 172.21.15.43:0/72745218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb95019e80 0x55bb95026430 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.694+0000 7f5f14ff9640 0 client.19224 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:46:42.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.694+0000 7f5f36479740 1 --2- 172.21.15.43:0/72745218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb950194d0 0x55bb9502b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.694+0000 7f5f36479740 1 -- 172.21.15.43:0/72745218 >> 172.21.15.43:0/72745218 conn(0x55bb95013ba0 msgr2=0x55bb95012370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.695+0000 7f5f36479740 1 -- 172.21.15.43:0/72745218 shutdown_connections 2024-04-21T08:46:42.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.695+0000 7f5f36479740 1 -- 172.21.15.43:0/72745218 wait complete. 2024-04-21T08:46:42.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.695+0000 7f5f36479740 20 client.19224 trim_cache size 0 max 16384 2024-04-21T08:46:42.701 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:46:42.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.695+0000 7f5f33881640 1 -- 172.21.15.43:0/939963546 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5f180a3d20 msgr2=0x7f5f180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.695+0000 7f5f33881640 1 --2- 172.21.15.43:0/939963546 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5f180a3d20 0x7f5f180a61e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5f280094d0 tx=0x7f5f28004a40 comp rx=0 tx=0).stop 2024-04-21T08:46:42.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.695+0000 7f5f33881640 1 -- 172.21.15.43:0/939963546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f2c104400 msgr2=0x7f5f2c1a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.695+0000 7f5f33881640 1 --2- 172.21.15.43:0/939963546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f2c104400 0x7f5f2c1a06c0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f5f24005940 tx=0x7f5f2402c5f0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.695+0000 7f5f33881640 1 -- 172.21.15.43:0/939963546 shutdown_connections 2024-04-21T08:46:42.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.695+0000 7f5f33881640 1 --2- 172.21.15.43:0/939963546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f2c19a880 0x7f5f2c19ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.695+0000 7f5f33881640 1 --2- 172.21.15.43:0/939963546 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5f180a3d20 0x7f5f180a61e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.695+0000 7f5f33881640 1 --2- 172.21.15.43:0/939963546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f2c1094d0 0x7f5f2c1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.695+0000 7f5f33881640 1 --2- 172.21.15.43:0/939963546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f2c104400 0x7f5f2c1a06c0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.695+0000 7f5f33881640 1 -- 172.21.15.43:0/939963546 >> 172.21.15.43:0/939963546 conn(0x7f5f2c0ffce0 msgr2=0x7f5f2c101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.695+0000 7f5f33881640 1 -- 172.21.15.43:0/939963546 shutdown_connections 2024-04-21T08:46:42.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.695+0000 7f5f33881640 1 -- 172.21.15.43:0/939963546 wait complete. 2024-04-21T08:46:42.705 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:46:42.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:42 smithi175 ceph-mon[31920]: pgmap v239: 129 pgs: 129 active+clean; 2.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-21T08:46:42.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:42 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/4166528013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:42.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:42 smithi112 ceph-mon[31085]: pgmap v239: 129 pgs: 129 active+clean; 2.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-21T08:46:42.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:42 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/4166528013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:42.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.815+0000 7fb534ff4640 1 -- 172.21.15.43:0/1003252182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb53006b1f0 msgr2=0x7fb530106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.815+0000 7fb534ff4640 1 --2- 172.21.15.43:0/1003252182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb53006b1f0 0x7fb530106b90 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fb524009900 tx=0x7fb52402bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:42.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.815+0000 7fb534ff4640 1 -- 172.21.15.43:0/1003252182 shutdown_connections 2024-04-21T08:46:42.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.815+0000 7fb534ff4640 1 --2- 172.21.15.43:0/1003252182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb530107270 0x7fb53010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.815+0000 7fb534ff4640 1 --2- 172.21.15.43:0/1003252182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb53006b1f0 0x7fb530106b90 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.815+0000 7fb534ff4640 1 --2- 172.21.15.43:0/1003252182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb53006a840 0x7fb53006ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.815+0000 7fb534ff4640 1 -- 172.21.15.43:0/1003252182 >> 172.21.15.43:0/1003252182 conn(0x7fb530077eb0 msgr2=0x7fb5300782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.815+0000 7fb534ff4640 1 -- 172.21.15.43:0/1003252182 shutdown_connections 2024-04-21T08:46:42.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.815+0000 7fb534ff4640 1 -- 172.21.15.43:0/1003252182 wait complete. 2024-04-21T08:46:42.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.816+0000 7fb534ff4640 1 Processor -- start 2024-04-21T08:46:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.816+0000 7fb534ff4640 1 -- start start 2024-04-21T08:46:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.816+0000 7fb534ff4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb53006a840 0x7fb530198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.816+0000 7fb534ff4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb530107270 0x7fb530198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.816+0000 7fb534ff4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb53019cff0 0x7fb53019d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.816+0000 7fb534ff4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb5301121b0 con 0x7fb53006a840 2024-04-21T08:46:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.816+0000 7fb534ff4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb530112030 con 0x7fb530107270 2024-04-21T08:46:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.816+0000 7fb534ff4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb530112330 con 0x7fb53019cff0 2024-04-21T08:46:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.816+0000 7fb52e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb53006a840 0x7fb530198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.816+0000 7fb52e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb53006a840 0x7fb530198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54384/0 (socket says 172.21.15.43:54384) 2024-04-21T08:46:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.816+0000 7fb52e59c640 1 -- 172.21.15.43:0/2005197528 learned_addr learned my addr 172.21.15.43:0/2005197528 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.816+0000 7fb52dd9b640 1 --2- 172.21.15.43:0/2005197528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb530107270 0x7fb530198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.817+0000 7fb52ed9d640 1 --2- 172.21.15.43:0/2005197528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb53019cff0 0x7fb53019d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.817+0000 7fb52e59c640 1 -- 172.21.15.43:0/2005197528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb53019cff0 msgr2=0x7fb53019d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.817+0000 7fb52e59c640 1 --2- 172.21.15.43:0/2005197528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb53019cff0 0x7fb53019d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.817+0000 7fb52e59c640 1 -- 172.21.15.43:0/2005197528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb530107270 msgr2=0x7fb530198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.817+0000 7fb52e59c640 1 --2- 172.21.15.43:0/2005197528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb530107270 0x7fb530198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.817+0000 7fb52e59c640 1 -- 172.21.15.43:0/2005197528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb53019dae0 con 0x7fb53006a840 2024-04-21T08:46:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.817+0000 7fb52e59c640 1 --2- 172.21.15.43:0/2005197528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb53006a840 0x7fb530198730 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fb52000e9f0 tx=0x7fb52000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.817+0000 7fb51f7fe640 1 -- 172.21.15.43:0/2005197528 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52000cd80 con 0x7fb53006a840 2024-04-21T08:46:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.817+0000 7fb534ff4640 1 -- 172.21.15.43:0/2005197528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb53019ddd0 con 0x7fb53006a840 2024-04-21T08:46:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.817+0000 7fb51f7fe640 1 -- 172.21.15.43:0/2005197528 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5200054d0 con 0x7fb53006a840 2024-04-21T08:46:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.817+0000 7fb51f7fe640 1 -- 172.21.15.43:0/2005197528 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520010690 con 0x7fb53006a840 2024-04-21T08:46:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.817+0000 7fb534ff4640 1 -- 172.21.15.43:0/2005197528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5301a5540 con 0x7fb53006a840 2024-04-21T08:46:42.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.819+0000 7fb51f7fe640 1 -- 172.21.15.43:0/2005197528 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb520004e90 con 0x7fb53006a840 2024-04-21T08:46:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.820+0000 7fb537be5740 1 Processor -- start 2024-04-21T08:46:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.820+0000 7fb537be5740 1 -- start start 2024-04-21T08:46:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.820+0000 7fb51f7fe640 1 --2- 172.21.15.43:0/2005197528 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb5180a3d20 0x7fb5180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.820+0000 7fb537be5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23c4d0 0x560bdf23c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.820+0000 7fb537be5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bdf23ce80 0x560bdf245430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.820+0000 7fb537be5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bdf245a00 0x560bdf247e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.820+0000 7fb537be5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560bdf1a11f0 con 0x560bdf23c4d0 2024-04-21T08:46:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.820+0000 7fb537be5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560bdf13e4b0 con 0x560bdf245a00 2024-04-21T08:46:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.820+0000 7fb537be5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560bdf19d1b0 con 0x560bdf23ce80 2024-04-21T08:46:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.820+0000 7fb52ed9d640 1 --2- 172.21.15.43:0/2005197528 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb5180a3d20 0x7fb5180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.820+0000 7fb52e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23c4d0 0x560bdf23c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.821+0000 7fb51f7fe640 1 -- 172.21.15.43:0/2005197528 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7fb5200d1080 con 0x7fb53006a840 2024-04-21T08:46:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.821+0000 7fb52e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23c4d0 0x560bdf23c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54400/0 (socket says 172.21.15.43:54400) 2024-04-21T08:46:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.821+0000 7fb52e59c640 1 -- 172.21.15.43:0/2552806089 learned_addr learned my addr 172.21.15.43:0/2552806089 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.821+0000 7fb52dd9b640 1 --2- 172.21.15.43:0/2552806089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bdf23ce80 0x560bdf245430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.821+0000 7fb52ed9d640 1 --2- 172.21.15.43:0/2552806089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bdf245a00 0x560bdf247e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.821+0000 7fb52e59c640 1 -- 172.21.15.43:0/2552806089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bdf23ce80 msgr2=0x560bdf245430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.821+0000 7fb52e59c640 1 --2- 172.21.15.43:0/2552806089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bdf23ce80 0x560bdf245430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.821+0000 7fb52e59c640 1 -- 172.21.15.43:0/2552806089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bdf245a00 msgr2=0x560bdf247e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.821+0000 7fb52e59c640 1 --2- 172.21.15.43:0/2552806089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bdf245a00 0x560bdf247e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.821+0000 7fb52e59c640 1 -- 172.21.15.43:0/2552806089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bdf1301b0 con 0x560bdf23c4d0 2024-04-21T08:46:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.821+0000 7fb52ed9d640 1 --2- 172.21.15.43:0/2005197528 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb5180a3d20 0x7fb5180a61e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb528005c40 tx=0x7fb528009d70 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.821+0000 7fb52e59c640 1 --2- 172.21.15.43:0/2552806089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23c4d0 0x560bdf23c8b0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fb52009c950 tx=0x7fb52009ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e18a88e917e5b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.821+0000 7fb51cff9640 1 -- 172.21.15.43:0/2552806089 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52004be60 con 0x560bdf23c4d0 2024-04-21T08:46:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.821+0000 7fb51cff9640 1 -- 172.21.15.43:0/2552806089 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb52009d350 con 0x560bdf23c4d0 2024-04-21T08:46:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.821+0000 7fb51cff9640 1 -- 172.21.15.43:0/2552806089 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5200a58c0 con 0x560bdf23c4d0 2024-04-21T08:46:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.822+0000 7fb537be5740 1 -- 172.21.15.43:0/2552806089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23c4d0 msgr2=0x560bdf23c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.822+0000 7fb537be5740 1 --2- 172.21.15.43:0/2552806089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23c4d0 0x560bdf23c8b0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fb52009c950 tx=0x7fb52009ca20 comp rx=0 tx=0).stop 2024-04-21T08:46:42.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.822+0000 7fb537be5740 1 -- 172.21.15.43:0/2552806089 shutdown_connections 2024-04-21T08:46:42.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.822+0000 7fb537be5740 1 --2- 172.21.15.43:0/2552806089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bdf245a00 0x560bdf247e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.822+0000 7fb537be5740 1 --2- 172.21.15.43:0/2552806089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bdf23ce80 0x560bdf245430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.822+0000 7fb537be5740 1 --2- 172.21.15.43:0/2552806089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23c4d0 0x560bdf23c8b0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.822+0000 7fb537be5740 1 -- 172.21.15.43:0/2552806089 >> 172.21.15.43:0/2552806089 conn(0x560bdf236ba0 msgr2=0x560bdf236f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.822+0000 7fb537be5740 1 -- 172.21.15.43:0/2552806089 shutdown_connections 2024-04-21T08:46:42.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.822+0000 7fb537be5740 1 -- 172.21.15.43:0/2552806089 wait complete. 2024-04-21T08:46:42.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.822+0000 7fb537be5740 1 -- start start 2024-04-21T08:46:42.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.822+0000 7fb537be5740 1 Processor -- start 2024-04-21T08:46:42.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.823+0000 7fb537be5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23c4d0 0x560bdf245340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.823+0000 7fb537be5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bdf23ce80 0x560bdf2426d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.823+0000 7fb537be5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bdf245a00 0x560bdf242ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.823+0000 7fb537be5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560bdf248910 con 0x560bdf23c4d0 2024-04-21T08:46:42.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.823+0000 7fb537be5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560bdf15a500 con 0x560bdf245a00 2024-04-21T08:46:42.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.823+0000 7fb537be5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560bdf248a90 con 0x560bdf23ce80 2024-04-21T08:46:42.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.823+0000 7fb537be5740 10 client.0 _ll_register_callbacks cb 0x560bdf128b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:42.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.823+0000 7fb52e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23c4d0 0x560bdf245340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.823+0000 7fb52e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23c4d0 0x560bdf245340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54416/0 (socket says 172.21.15.43:54416) 2024-04-21T08:46:42.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.823+0000 7fb52e59c640 1 -- 172.21.15.43:0/99230579 learned_addr learned my addr 172.21.15.43:0/99230579 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.823+0000 7fb52dd9b640 1 --2- 172.21.15.43:0/99230579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bdf23ce80 0x560bdf2426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.823+0000 7fb52ed9d640 1 --2- 172.21.15.43:0/99230579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bdf245a00 0x560bdf242ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.823+0000 7fb52e59c640 1 -- 172.21.15.43:0/99230579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bdf23ce80 msgr2=0x560bdf2426d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.823+0000 7fb52e59c640 1 --2- 172.21.15.43:0/99230579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bdf23ce80 0x560bdf2426d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.823+0000 7fb52e59c640 1 -- 172.21.15.43:0/99230579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bdf245a00 msgr2=0x560bdf242ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.823+0000 7fb52e59c640 1 --2- 172.21.15.43:0/99230579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bdf245a00 0x560bdf242ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.823+0000 7fb52e59c640 1 -- 172.21.15.43:0/99230579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bdeeb33f0 con 0x560bdf23c4d0 2024-04-21T08:46:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.823+0000 7fb52ed9d640 1 --2- 172.21.15.43:0/99230579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bdf245a00 0x560bdf242ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.823+0000 7fb52e59c640 1 --2- 172.21.15.43:0/99230579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23c4d0 0x560bdf245340 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fb5200990b0 tx=0x7fb5200048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d392e8f55876f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.823+0000 7fb515ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.823+0000 7fb515ffb640 1 -- 172.21.15.43:0/99230579 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5200b6060 con 0x560bdf23c4d0 2024-04-21T08:46:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.823+0000 7fb515ffb640 1 -- 172.21.15.43:0/99230579 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7fb5200a5430 con 0x560bdf23c4d0 2024-04-21T08:46:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.823+0000 7fb537be5740 1 -- 172.21.15.43:0/99230579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb52009d350 con 0x560bdf23c4d0 2024-04-21T08:46:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.824+0000 7fb515ffb640 1 -- 172.21.15.43:0/99230579 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb520005180 con 0x560bdf23c4d0 2024-04-21T08:46:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.824+0000 7fb537be5740 10 client.19236 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.824+0000 7fb537be5740 1 -- 172.21.15.43:0/99230579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560bdef7d7d0 con 0x560bdf23c4d0 2024-04-21T08:46:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.824+0000 7fb515ffb640 1 -- 172.21.15.43:0/99230579 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb52009d350 con 0x560bdf23c4d0 2024-04-21T08:46:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.824+0000 7fb515ffb640 1 -- 172.21.15.43:0/99230579 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb52009c0a0 con 0x560bdf23c4d0 2024-04-21T08:46:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.824+0000 7fb537be5740 10 client.19236 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.824+0000 7fb537be5740 10 client.19236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.824+0000 7fb537be5740 20 client.19236 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.824+0000 7fb537be5740 1 --2- 172.21.15.43:0/99230579 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560bdf255a60 0x560bdf257e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.824+0000 7fb537be5740 4 client.19236 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.824+0000 7fb537be5740 1 -- 172.21.15.43:0/99230579 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bdf140930 con 0x560bdf255a60 2024-04-21T08:46:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.824+0000 7fb52dd9b640 1 --2- 172.21.15.43:0/99230579 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560bdf255a60 0x560bdf257e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.825+0000 7fb515ffb640 10 client.19236 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.825+0000 7fb52dd9b640 1 --2- 172.21.15.43:0/99230579 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560bdf255a60 0x560bdf257e50 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b39b04043e00cda server_cookie=b0d8a61c7ba35c54 in_seq=0 out_seq=0 2024-04-21T08:46:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.825+0000 7fb515ffb640 1 -- 172.21.15.43:0/99230579 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560bdf140930 con 0x560bdf255a60 2024-04-21T08:46:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.825+0000 7fb515ffb640 10 client.19236 handle_command_reply: tid=0 2024-04-21T08:46:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.825+0000 7fb537be5740 1 client.19236 shutdown 2024-04-21T08:46:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.825+0000 7fb537be5740 1 -- 172.21.15.43:0/99230579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23c4d0 msgr2=0x560bdf245340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.826+0000 7fb537be5740 1 --2- 172.21.15.43:0/99230579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23c4d0 0x560bdf245340 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fb5200990b0 tx=0x7fb5200048f0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.826+0000 7fb537be5740 1 -- 172.21.15.43:0/99230579 shutdown_connections 2024-04-21T08:46:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.826+0000 7fb537be5740 1 --2- 172.21.15.43:0/99230579 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560bdf255a60 0x560bdf257e50 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.826+0000 7fb537be5740 1 --2- 172.21.15.43:0/99230579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bdf245a00 0x560bdf242ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.826+0000 7fb537be5740 1 --2- 172.21.15.43:0/99230579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bdf23ce80 0x560bdf2426d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.826+0000 7fb515ffb640 0 client.19236 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.826+0000 7fb537be5740 1 --2- 172.21.15.43:0/99230579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23c4d0 0x560bdf245340 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.826+0000 7fb537be5740 1 -- 172.21.15.43:0/99230579 >> 172.21.15.43:0/99230579 conn(0x560bdf236ba0 msgr2=0x560bdf2388f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.826+0000 7fb537be5740 1 -- 172.21.15.43:0/99230579 shutdown_connections 2024-04-21T08:46:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.826+0000 7fb537be5740 1 -- 172.21.15.43:0/99230579 wait complete. 2024-04-21T08:46:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.826+0000 7fb537be5740 20 client.19236 trim_cache size 0 max 16384 2024-04-21T08:46:42.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.839+0000 7fb537be5740 1 Processor -- start 2024-04-21T08:46:42.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.839+0000 7fb537be5740 1 -- start start 2024-04-21T08:46:42.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.839+0000 7fb537be5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bdf23c4d0 0x560bdf2517b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.839+0000 7fb537be5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23ce80 0x560bdf25b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.839+0000 7fb537be5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bdf245a00 0x560bdf244e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.839+0000 7fb537be5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb5301af400 con 0x560bdf23ce80 2024-04-21T08:46:42.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.839+0000 7fb537be5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb5301b0f40 con 0x560bdf23c4d0 2024-04-21T08:46:42.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.839+0000 7fb537be5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb5301afad0 con 0x560bdf245a00 2024-04-21T08:46:42.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.839+0000 7fb52dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23ce80 0x560bdf25b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.839+0000 7fb52dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23ce80 0x560bdf25b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54430/0 (socket says 172.21.15.43:54430) 2024-04-21T08:46:42.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.839+0000 7fb52dd9b640 1 -- 172.21.15.43:0/1753306151 learned_addr learned my addr 172.21.15.43:0/1753306151 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.839+0000 7fb52e59c640 1 --2- 172.21.15.43:0/1753306151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bdf23c4d0 0x560bdf2517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.839+0000 7fb52dd9b640 1 -- 172.21.15.43:0/1753306151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bdf245a00 msgr2=0x560bdf244e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.839+0000 7fb52ed9d640 1 --2- 172.21.15.43:0/1753306151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bdf245a00 0x560bdf244e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.839+0000 7fb52dd9b640 1 --2- 172.21.15.43:0/1753306151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bdf245a00 0x560bdf244e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.840+0000 7fb52dd9b640 1 -- 172.21.15.43:0/1753306151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bdf23c4d0 msgr2=0x560bdf2517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.840+0000 7fb52dd9b640 1 --2- 172.21.15.43:0/1753306151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bdf23c4d0 0x560bdf2517b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.840+0000 7fb52dd9b640 1 -- 172.21.15.43:0/1753306151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bdf23cc20 con 0x560bdf23ce80 2024-04-21T08:46:42.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.840+0000 7fb52e59c640 1 --2- 172.21.15.43:0/1753306151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bdf23c4d0 0x560bdf2517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:42.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.840+0000 7fb52dd9b640 1 --2- 172.21.15.43:0/1753306151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23ce80 0x560bdf25b260 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fb53006a420 tx=0x7fb524005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873c42f26dafc7dc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.840+0000 7fb517fff640 1 -- 172.21.15.43:0/1753306151 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52404f020 con 0x560bdf23ce80 2024-04-21T08:46:42.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.840+0000 7fb517fff640 1 -- 172.21.15.43:0/1753306151 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb52402cc90 con 0x560bdf23ce80 2024-04-21T08:46:42.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.840+0000 7fb517fff640 1 -- 172.21.15.43:0/1753306151 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb524049070 con 0x560bdf23ce80 2024-04-21T08:46:42.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.840+0000 7fb537be5740 1 -- 172.21.15.43:0/1753306151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23ce80 msgr2=0x560bdf25b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.840+0000 7fb537be5740 1 --2- 172.21.15.43:0/1753306151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23ce80 0x560bdf25b260 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fb53006a420 tx=0x7fb524005fd0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.840+0000 7fb537be5740 1 -- 172.21.15.43:0/1753306151 shutdown_connections 2024-04-21T08:46:42.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.840+0000 7fb537be5740 1 --2- 172.21.15.43:0/1753306151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bdf245a00 0x560bdf244e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.840+0000 7fb537be5740 1 --2- 172.21.15.43:0/1753306151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23ce80 0x560bdf25b260 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.840+0000 7fb537be5740 1 --2- 172.21.15.43:0/1753306151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bdf23c4d0 0x560bdf2517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.840+0000 7fb537be5740 1 -- 172.21.15.43:0/1753306151 >> 172.21.15.43:0/1753306151 conn(0x560bdf236ba0 msgr2=0x560bdf241700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.840+0000 7fb537be5740 1 -- 172.21.15.43:0/1753306151 shutdown_connections 2024-04-21T08:46:42.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.840+0000 7fb537be5740 1 -- 172.21.15.43:0/1753306151 wait complete. 2024-04-21T08:46:42.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.841+0000 7fb537be5740 1 -- start start 2024-04-21T08:46:42.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.841+0000 7fb537be5740 1 Processor -- start 2024-04-21T08:46:42.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.841+0000 7fb537be5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23c4d0 0x560bdf2b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.841+0000 7fb537be5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bdf23ce80 0x560bdf2b8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.841+0000 7fb537be5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bdf245a00 0x560bdf2b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.841+0000 7fb537be5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb5301b0870 con 0x560bdf23c4d0 2024-04-21T08:46:42.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.841+0000 7fb537be5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb5301b01a0 con 0x560bdf23ce80 2024-04-21T08:46:42.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.841+0000 7fb537be5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb5301ad3e0 con 0x560bdf245a00 2024-04-21T08:46:42.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.841+0000 7fb537be5740 10 client.0 _ll_register_callbacks cb 0x7fb520098860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:42.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.841+0000 7fb52e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23c4d0 0x560bdf2b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.841+0000 7fb52e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23c4d0 0x560bdf2b8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54436/0 (socket says 172.21.15.43:54436) 2024-04-21T08:46:42.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.841+0000 7fb52e59c640 1 -- 172.21.15.43:0/4007360426 learned_addr learned my addr 172.21.15.43:0/4007360426 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:42.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.841+0000 7fb52dd9b640 1 --2- 172.21.15.43:0/4007360426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bdf23ce80 0x560bdf2b8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.841+0000 7fb52e59c640 1 -- 172.21.15.43:0/4007360426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bdf245a00 msgr2=0x560bdf2b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.841+0000 7fb52e59c640 1 --2- 172.21.15.43:0/4007360426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bdf245a00 0x560bdf2b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.841+0000 7fb52e59c640 1 -- 172.21.15.43:0/4007360426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bdf23ce80 msgr2=0x560bdf2b8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:42.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.841+0000 7fb52e59c640 1 --2- 172.21.15.43:0/4007360426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bdf23ce80 0x560bdf2b8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.841+0000 7fb52e59c640 1 -- 172.21.15.43:0/4007360426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bdf2b3570 con 0x560bdf23c4d0 2024-04-21T08:46:42.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.842+0000 7fb52e59c640 1 --2- 172.21.15.43:0/4007360426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23c4d0 0x560bdf2b8a70 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fb5200cea00 tx=0x7fb5200ce4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251e76ddd1a25de6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:42.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.842+0000 7fb515ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:42.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.842+0000 7fb515ffb640 1 -- 172.21.15.43:0/4007360426 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5200b6040 con 0x560bdf23c4d0 2024-04-21T08:46:42.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.842+0000 7fb515ffb640 1 -- 172.21.15.43:0/4007360426 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7fb5200ceb50 con 0x560bdf23c4d0 2024-04-21T08:46:42.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.842+0000 7fb537be5740 1 -- 172.21.15.43:0/4007360426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb52402cc90 con 0x560bdf23c4d0 2024-04-21T08:46:42.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.842+0000 7fb515ffb640 1 -- 172.21.15.43:0/4007360426 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb52009c790 con 0x560bdf23c4d0 2024-04-21T08:46:42.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.842+0000 7fb537be5740 10 client.19242 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:42.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.842+0000 7fb537be5740 1 -- 172.21.15.43:0/4007360426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560bdf2c0270 con 0x560bdf23c4d0 2024-04-21T08:46:42.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.842+0000 7fb515ffb640 1 -- 172.21.15.43:0/4007360426 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb52402cc90 con 0x560bdf23c4d0 2024-04-21T08:46:42.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.842+0000 7fb515ffb640 1 -- 172.21.15.43:0/4007360426 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb5200a5b10 con 0x560bdf23c4d0 2024-04-21T08:46:42.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.842+0000 7fb537be5740 10 client.19242 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:42.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.842+0000 7fb537be5740 10 client.19242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:42.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.842+0000 7fb537be5740 20 client.19242 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:42.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.842+0000 7fb537be5740 1 --2- 172.21.15.43:0/4007360426 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560bdf2c0770 0x560bdf2c2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:42.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.842+0000 7fb537be5740 4 client.19242 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:42.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.842+0000 7fb537be5740 1 -- 172.21.15.43:0/4007360426 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560bdf1aec10 con 0x560bdf2c0770 2024-04-21T08:46:42.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.843+0000 7fb52dd9b640 1 --2- 172.21.15.43:0/4007360426 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560bdf2c0770 0x560bdf2c2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:42.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.843+0000 7fb515ffb640 10 client.19242 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:42.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.843+0000 7fb52dd9b640 1 --2- 172.21.15.43:0/4007360426 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560bdf2c0770 0x560bdf2c2bd0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf7a38c8846668c4 server_cookie=6e1e68aee45d88c9 in_seq=0 out_seq=0 2024-04-21T08:46:42.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.843+0000 7fb515ffb640 1 -- 172.21.15.43:0/4007360426 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560bdf1aec10 con 0x560bdf2c0770 2024-04-21T08:46:42.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.843+0000 7fb515ffb640 10 client.19242 handle_command_reply: tid=0 2024-04-21T08:46:42.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.843+0000 7fb537be5740 1 client.19242 shutdown 2024-04-21T08:46:42.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.843+0000 7fb537be5740 1 -- 172.21.15.43:0/4007360426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23c4d0 msgr2=0x560bdf2b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.843+0000 7fb537be5740 1 --2- 172.21.15.43:0/4007360426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23c4d0 0x560bdf2b8a70 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fb5200cea00 tx=0x7fb5200ce4b0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.843+0000 7fb537be5740 1 -- 172.21.15.43:0/4007360426 shutdown_connections 2024-04-21T08:46:42.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.843+0000 7fb537be5740 1 --2- 172.21.15.43:0/4007360426 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560bdf2c0770 0x560bdf2c2bd0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.843+0000 7fb537be5740 1 --2- 172.21.15.43:0/4007360426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bdf245a00 0x560bdf2b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.843+0000 7fb515ffb640 0 client.19242 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:42.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.843+0000 7fb537be5740 1 --2- 172.21.15.43:0/4007360426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bdf23ce80 0x560bdf2b8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.843+0000 7fb537be5740 1 --2- 172.21.15.43:0/4007360426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bdf23c4d0 0x560bdf2b8a70 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.843+0000 7fb537be5740 1 -- 172.21.15.43:0/4007360426 >> 172.21.15.43:0/4007360426 conn(0x560bdf236ba0 msgr2=0x560bdf237f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.843+0000 7fb537be5740 1 -- 172.21.15.43:0/4007360426 shutdown_connections 2024-04-21T08:46:42.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.843+0000 7fb537be5740 1 -- 172.21.15.43:0/4007360426 wait complete. 2024-04-21T08:46:42.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.843+0000 7fb537be5740 20 client.19242 trim_cache size 0 max 16384 2024-04-21T08:46:42.850 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:46:42.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.844+0000 7fb534ff4640 1 -- 172.21.15.43:0/2005197528 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb5180a3d20 msgr2=0x7fb5180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.844+0000 7fb534ff4640 1 --2- 172.21.15.43:0/2005197528 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb5180a3d20 0x7fb5180a61e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb528005c40 tx=0x7fb528009d70 comp rx=0 tx=0).stop 2024-04-21T08:46:42.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.844+0000 7fb534ff4640 1 -- 172.21.15.43:0/2005197528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb53006a840 msgr2=0x7fb530198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:42.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.844+0000 7fb534ff4640 1 --2- 172.21.15.43:0/2005197528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb53006a840 0x7fb530198730 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fb52000e9f0 tx=0x7fb52000edf0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.844+0000 7fb534ff4640 1 -- 172.21.15.43:0/2005197528 shutdown_connections 2024-04-21T08:46:42.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.844+0000 7fb534ff4640 1 --2- 172.21.15.43:0/2005197528 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb5180a3d20 0x7fb5180a61e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.844+0000 7fb534ff4640 1 --2- 172.21.15.43:0/2005197528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb53019cff0 0x7fb53019d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.844+0000 7fb534ff4640 1 --2- 172.21.15.43:0/2005197528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb530107270 0x7fb530198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.844+0000 7fb534ff4640 1 --2- 172.21.15.43:0/2005197528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb53006a840 0x7fb530198730 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:42.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.844+0000 7fb534ff4640 1 -- 172.21.15.43:0/2005197528 >> 172.21.15.43:0/2005197528 conn(0x7fb530077eb0 msgr2=0x7fb5300ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:42.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.844+0000 7fb534ff4640 1 -- 172.21.15.43:0/2005197528 shutdown_connections 2024-04-21T08:46:42.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:42.844+0000 7fb534ff4640 1 -- 172.21.15.43:0/2005197528 wait complete. 2024-04-21T08:46:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.589+0000 7f24d6a21640 1 -- 172.21.15.43:0/3444253334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24d01031e0 msgr2=0x7f24d01035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.589+0000 7f24d6a21640 1 --2- 172.21.15.43:0/3444253334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24d01031e0 0x7f24d01035c0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009900 tx=0x7f24c002bcd0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.589+0000 7f24d6a21640 1 -- 172.21.15.43:0/3444253334 shutdown_connections 2024-04-21T08:46:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.589+0000 7f24d6a21640 1 --2- 172.21.15.43:0/3444253334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24d010c5b0 0x7f24d010e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.589+0000 7f24d6a21640 1 --2- 172.21.15.43:0/3444253334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24d0103b00 0x7f24d010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.589+0000 7f24d6a21640 1 --2- 172.21.15.43:0/3444253334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24d01031e0 0x7f24d01035c0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.589+0000 7f24d6a21640 1 -- 172.21.15.43:0/3444253334 >> 172.21.15.43:0/3444253334 conn(0x7f24d0077f60 msgr2=0x7f24d0078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.589+0000 7f24d6a21640 1 -- 172.21.15.43:0/3444253334 shutdown_connections 2024-04-21T08:46:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.589+0000 7f24d6a21640 1 -- 172.21.15.43:0/3444253334 wait complete. 2024-04-21T08:46:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.589+0000 7f24d6a21640 1 Processor -- start 2024-04-21T08:46:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.589+0000 7f24d6a21640 1 -- start start 2024-04-21T08:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.590+0000 7f24d6a21640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24d01031e0 0x7f24d0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.590+0000 7f24d6a21640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24d0103b00 0x7f24d0198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.590+0000 7f24d6a21640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24d010c5b0 0x7f24d019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.590+0000 7f24d6a21640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24d0112200 con 0x7f24d010c5b0 2024-04-21T08:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.590+0000 7f24d6a21640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24d0112080 con 0x7f24d01031e0 2024-04-21T08:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.590+0000 7f24d6a21640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f24d0112380 con 0x7f24d0103b00 2024-04-21T08:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.590+0000 7f24d4fbe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24d010c5b0 0x7f24d019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.590+0000 7f24d4fbe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24d010c5b0 0x7f24d019cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54466/0 (socket says 172.21.15.43:54466) 2024-04-21T08:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.590+0000 7f24d4fbe640 1 -- 172.21.15.43:0/1866485706 learned_addr learned my addr 172.21.15.43:0/1866485706 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.590+0000 7f24cffff640 1 --2- 172.21.15.43:0/1866485706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24d01031e0 0x7f24d0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.590+0000 7f24cf7fe640 1 --2- 172.21.15.43:0/1866485706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24d0103b00 0x7f24d0198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.590+0000 7f24d4fbe640 1 -- 172.21.15.43:0/1866485706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24d0103b00 msgr2=0x7f24d0198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.590+0000 7f24d4fbe640 1 --2- 172.21.15.43:0/1866485706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24d0103b00 0x7f24d0198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.590+0000 7f24d4fbe640 1 -- 172.21.15.43:0/1866485706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24d01031e0 msgr2=0x7f24d0198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.590+0000 7f24d4fbe640 1 --2- 172.21.15.43:0/1866485706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24d01031e0 0x7f24d0198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.590+0000 7f24d4fbe640 1 -- 172.21.15.43:0/1866485706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d019d6c0 con 0x7f24d010c5b0 2024-04-21T08:46:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.591+0000 7f24d4fbe640 1 --2- 172.21.15.43:0/1866485706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24d010c5b0 0x7f24d019cf90 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f24c800e9d0 tx=0x7f24c800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.591+0000 7f24cd7fa640 1 -- 172.21.15.43:0/1866485706 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c800cd80 con 0x7f24d010c5b0 2024-04-21T08:46:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.591+0000 7f24cd7fa640 1 -- 172.21.15.43:0/1866485706 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c8005020 con 0x7f24d010c5b0 2024-04-21T08:46:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.591+0000 7f24d6a21640 1 -- 172.21.15.43:0/1866485706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d019d9b0 con 0x7f24d010c5b0 2024-04-21T08:46:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.591+0000 7f24cd7fa640 1 -- 172.21.15.43:0/1866485706 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c80116d0 con 0x7f24d010c5b0 2024-04-21T08:46:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.591+0000 7f24d6a21640 1 -- 172.21.15.43:0/1866485706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d01a5240 con 0x7f24d010c5b0 2024-04-21T08:46:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.593+0000 7f24cd7fa640 1 -- 172.21.15.43:0/1866485706 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f24c80051d0 con 0x7f24d010c5b0 2024-04-21T08:46:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.594+0000 7f24d960b740 1 Processor -- start 2024-04-21T08:46:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.594+0000 7f24d960b740 1 -- start start 2024-04-21T08:46:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.594+0000 7f24cd7fa640 1 --2- 172.21.15.43:0/1866485706 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f24b80a3dc0 0x7f24b80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.594+0000 7f24d960b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7c8d0 0x55fa56c7b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.594+0000 7f24d960b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa56c7bb60 0x55fa56c7bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.594+0000 7f24d960b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa56c894f0 0x55fa56c8b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.594+0000 7f24d960b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fa56be08d0 con 0x55fa56c7c8d0 2024-04-21T08:46:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.594+0000 7f24d960b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fa56b7de80 con 0x55fa56c894f0 2024-04-21T08:46:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.594+0000 7f24d960b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fa56bdc890 con 0x55fa56c7bb60 2024-04-21T08:46:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.594+0000 7f24cd7fa640 1 -- 172.21.15.43:0/1866485706 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f24c80d0fd0 con 0x7f24d010c5b0 2024-04-21T08:46:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.594+0000 7f24cffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7c8d0 0x55fa56c7b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.594+0000 7f24cf7fe640 1 --2- 172.21.15.43:0/1866485706 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f24b80a3dc0 0x7f24b80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.594+0000 7f24cffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7c8d0 0x55fa56c7b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54476/0 (socket says 172.21.15.43:54476) 2024-04-21T08:46:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.594+0000 7f24cffff640 1 -- 172.21.15.43:0/1165209477 learned_addr learned my addr 172.21.15.43:0/1165209477 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.594+0000 7f24d4fbe640 1 --2- 172.21.15.43:0/1165209477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa56c894f0 0x55fa56c8b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.594+0000 7f24cf7fe640 1 --2- 172.21.15.43:0/1165209477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa56c7bb60 0x55fa56c7bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.594+0000 7f24cffff640 1 -- 172.21.15.43:0/1165209477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa56c7bb60 msgr2=0x55fa56c7bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.594+0000 7f24cffff640 1 --2- 172.21.15.43:0/1165209477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa56c7bb60 0x55fa56c7bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.594+0000 7f24cffff640 1 -- 172.21.15.43:0/1165209477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa56c894f0 msgr2=0x55fa56c8b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.594+0000 7f24cffff640 1 --2- 172.21.15.43:0/1165209477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa56c894f0 0x55fa56c8b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.594+0000 7f24cffff640 1 -- 172.21.15.43:0/1165209477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa56b9f320 con 0x55fa56c7c8d0 2024-04-21T08:46:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.594+0000 7f24d4fbe640 1 --2- 172.21.15.43:0/1165209477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa56c894f0 0x55fa56c8b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.594+0000 7f24cf7fe640 1 --2- 172.21.15.43:0/1165209477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa56c7bb60 0x55fa56c7bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.595+0000 7f24cffff640 1 --2- 172.21.15.43:0/1165209477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7c8d0 0x55fa56c7b620 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009900 tx=0x7f24c002c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81d830c529b17c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.595+0000 7f24cf7fe640 1 --2- 172.21.15.43:0/1866485706 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f24b80a3dc0 0x7f24b80a6280 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55fa56c7d030 tx=0x7f24c400b770 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.595+0000 7f24b67fc640 1 -- 172.21.15.43:0/1165209477 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0004180 con 0x55fa56c7c8d0 2024-04-21T08:46:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.595+0000 7f24b67fc640 1 -- 172.21.15.43:0/1165209477 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c00054f0 con 0x55fa56c7c8d0 2024-04-21T08:46:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.595+0000 7f24b67fc640 1 -- 172.21.15.43:0/1165209477 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c00046c0 con 0x55fa56c7c8d0 2024-04-21T08:46:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.596+0000 7f24d960b740 1 -- 172.21.15.43:0/1165209477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7c8d0 msgr2=0x55fa56c7b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.596+0000 7f24d960b740 1 --2- 172.21.15.43:0/1165209477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7c8d0 0x55fa56c7b620 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009900 tx=0x7f24c002c760 comp rx=0 tx=0).stop 2024-04-21T08:46:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.596+0000 7f24d960b740 1 -- 172.21.15.43:0/1165209477 shutdown_connections 2024-04-21T08:46:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.596+0000 7f24d960b740 1 --2- 172.21.15.43:0/1165209477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa56c894f0 0x55fa56c8b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.596+0000 7f24d960b740 1 --2- 172.21.15.43:0/1165209477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa56c7bb60 0x55fa56c7bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.596+0000 7f24d960b740 1 --2- 172.21.15.43:0/1165209477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7c8d0 0x55fa56c7b620 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.596+0000 7f24d960b740 1 -- 172.21.15.43:0/1165209477 >> 172.21.15.43:0/1165209477 conn(0x55fa56c42c90 msgr2=0x55fa56c76850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.596+0000 7f24d960b740 1 -- 172.21.15.43:0/1165209477 shutdown_connections 2024-04-21T08:46:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.596+0000 7f24d960b740 1 -- 172.21.15.43:0/1165209477 wait complete. 2024-04-21T08:46:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.596+0000 7f24d960b740 1 -- start start 2024-04-21T08:46:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.596+0000 7f24d960b740 1 Processor -- start 2024-04-21T08:46:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.597+0000 7f24d960b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa56c7bb60 0x55fa56c88df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.597+0000 7f24d960b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7c8d0 0x55fa56c86180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.597+0000 7f24d960b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa56c894f0 0x55fa56c866c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.597+0000 7f24d960b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fa56c8c1c0 con 0x55fa56c7c8d0 2024-04-21T08:46:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.597+0000 7f24d960b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fa56b68eb0 con 0x55fa56c7bb60 2024-04-21T08:46:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.597+0000 7f24d960b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fa56c8c340 con 0x55fa56c894f0 2024-04-21T08:46:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.597+0000 7f24d960b740 10 client.0 _ll_register_callbacks cb 0x55fa56b95b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.597+0000 7f24cf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7c8d0 0x55fa56c86180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.597+0000 7f24cf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7c8d0 0x55fa56c86180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54488/0 (socket says 172.21.15.43:54488) 2024-04-21T08:46:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.597+0000 7f24cf7fe640 1 -- 172.21.15.43:0/3778839427 learned_addr learned my addr 172.21.15.43:0/3778839427 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.597+0000 7f24d4fbe640 1 --2- 172.21.15.43:0/3778839427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa56c894f0 0x55fa56c866c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.597+0000 7f24cffff640 1 --2- 172.21.15.43:0/3778839427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa56c7bb60 0x55fa56c88df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.597+0000 7f24cf7fe640 1 -- 172.21.15.43:0/3778839427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa56c894f0 msgr2=0x55fa56c866c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.597+0000 7f24cf7fe640 1 --2- 172.21.15.43:0/3778839427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa56c894f0 0x55fa56c866c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.597+0000 7f24cf7fe640 1 -- 172.21.15.43:0/3778839427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa56c7bb60 msgr2=0x55fa56c88df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.597+0000 7f24cf7fe640 1 --2- 172.21.15.43:0/3778839427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa56c7bb60 0x55fa56c88df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.597+0000 7f24cf7fe640 1 -- 172.21.15.43:0/3778839427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa568f3210 con 0x55fa56c7c8d0 2024-04-21T08:46:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.597+0000 7f24cf7fe640 1 --2- 172.21.15.43:0/3778839427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7c8d0 0x55fa56c86180 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f24c400b5c0 tx=0x7f24c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c7e83af9a11009 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.597+0000 7f24af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.597+0000 7f24af7fe640 1 -- 172.21.15.43:0/3778839427 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c401d940 con 0x55fa56c7c8d0 2024-04-21T08:46:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.598+0000 7f24af7fe640 1 -- 172.21.15.43:0/3778839427 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f24c401dae0 con 0x55fa56c7c8d0 2024-04-21T08:46:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.598+0000 7f24d960b740 1 -- 172.21.15.43:0/3778839427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c00054f0 con 0x55fa56c7c8d0 2024-04-21T08:46:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.598+0000 7f24af7fe640 1 -- 172.21.15.43:0/3778839427 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24c4030050 con 0x55fa56c7c8d0 2024-04-21T08:46:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.598+0000 7f24d960b740 10 client.19254 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.598+0000 7f24d960b740 1 -- 172.21.15.43:0/3778839427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fa569bf620 con 0x55fa56c7c8d0 2024-04-21T08:46:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.598+0000 7f24af7fe640 1 -- 172.21.15.43:0/3778839427 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c00054f0 con 0x55fa56c7c8d0 2024-04-21T08:46:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.598+0000 7f24af7fe640 1 -- 172.21.15.43:0/3778839427 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f24c401c040 con 0x55fa56c7c8d0 2024-04-21T08:46:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.598+0000 7f24d960b740 10 client.19254 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.598+0000 7f24d960b740 10 client.19254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.598+0000 7f24d960b740 20 client.19254 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.598+0000 7f24d960b740 1 --2- 172.21.15.43:0/3778839427 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fa56c993d0 0x55fa56c9b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.598+0000 7f24d960b740 4 client.19254 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.598+0000 7f24d960b740 1 -- 172.21.15.43:0/3778839427 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa56bf4880 con 0x55fa56c993d0 2024-04-21T08:46:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.599+0000 7f24cffff640 1 --2- 172.21.15.43:0/3778839427 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fa56c993d0 0x55fa56c9b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.599+0000 7f24cffff640 1 --2- 172.21.15.43:0/3778839427 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fa56c993d0 0x55fa56c9b840 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee1cc6128db07f3a server_cookie=d2ec79be7aab1569 in_seq=0 out_seq=0 2024-04-21T08:46:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.599+0000 7f24af7fe640 10 client.19254 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.600+0000 7f24af7fe640 1 -- 172.21.15.43:0/3778839427 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa56bf4880 con 0x55fa56c993d0 2024-04-21T08:46:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.600+0000 7f24af7fe640 10 client.19254 handle_command_reply: tid=0 2024-04-21T08:46:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.600+0000 7f24d960b740 1 client.19254 shutdown 2024-04-21T08:46:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.600+0000 7f24d960b740 1 -- 172.21.15.43:0/3778839427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7c8d0 msgr2=0x55fa56c86180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.600+0000 7f24d960b740 1 --2- 172.21.15.43:0/3778839427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7c8d0 0x55fa56c86180 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f24c400b5c0 tx=0x7f24c40046c0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.600+0000 7f24d960b740 1 -- 172.21.15.43:0/3778839427 shutdown_connections 2024-04-21T08:46:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.600+0000 7f24d960b740 1 --2- 172.21.15.43:0/3778839427 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fa56c993d0 0x55fa56c9b840 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.600+0000 7f24d960b740 1 --2- 172.21.15.43:0/3778839427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa56c894f0 0x55fa56c866c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.600+0000 7f24d960b740 1 --2- 172.21.15.43:0/3778839427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7c8d0 0x55fa56c86180 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.600+0000 7f24af7fe640 0 client.19254 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.600+0000 7f24d960b740 1 --2- 172.21.15.43:0/3778839427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa56c7bb60 0x55fa56c88df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.600+0000 7f24d960b740 1 -- 172.21.15.43:0/3778839427 >> 172.21.15.43:0/3778839427 conn(0x55fa56c42c90 msgr2=0x55fa56c785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.600+0000 7f24d960b740 1 -- 172.21.15.43:0/3778839427 shutdown_connections 2024-04-21T08:46:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.600+0000 7f24d960b740 1 -- 172.21.15.43:0/3778839427 wait complete. 2024-04-21T08:46:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.600+0000 7f24d960b740 20 client.19254 trim_cache size 0 max 16384 2024-04-21T08:46:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.615+0000 7f24d960b740 1 Processor -- start 2024-04-21T08:46:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.615+0000 7f24d960b740 1 -- start start 2024-04-21T08:46:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.616+0000 7f24d960b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7bb60 0x55fa56c950c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.616+0000 7f24d960b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa56c7c8d0 0x55fa56c888b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.616+0000 7f24d960b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa56c894f0 0x55fa56c88df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.616+0000 7f24d960b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01ad1f0 con 0x55fa56c7bb60 2024-04-21T08:46:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.616+0000 7f24d960b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01aed30 con 0x55fa56c7c8d0 2024-04-21T08:46:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.616+0000 7f24d960b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01ad8c0 con 0x55fa56c894f0 2024-04-21T08:46:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.616+0000 7f24cffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7bb60 0x55fa56c950c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.616+0000 7f24cffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7bb60 0x55fa56c950c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54500/0 (socket says 172.21.15.43:54500) 2024-04-21T08:46:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.616+0000 7f24cffff640 1 -- 172.21.15.43:0/1442990024 learned_addr learned my addr 172.21.15.43:0/1442990024 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.616+0000 7f24cffff640 1 -- 172.21.15.43:0/1442990024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa56c894f0 msgr2=0x55fa56c88df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.616+0000 7f24cffff640 1 --2- 172.21.15.43:0/1442990024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa56c894f0 0x55fa56c88df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.616+0000 7f24cffff640 1 -- 172.21.15.43:0/1442990024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa56c7c8d0 msgr2=0x55fa56c888b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.616+0000 7f24cf7fe640 1 --2- 172.21.15.43:0/1442990024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa56c7c8d0 0x55fa56c888b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.616+0000 7f24cffff640 1 --2- 172.21.15.43:0/1442990024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa56c7c8d0 0x55fa56c888b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.616+0000 7f24cffff640 1 -- 172.21.15.43:0/1442990024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa56c7ae30 con 0x55fa56c7bb60 2024-04-21T08:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.616+0000 7f24cffff640 1 --2- 172.21.15.43:0/1442990024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7bb60 0x55fa56c950c0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f24c002cb50 tx=0x7f24c002cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fde3e6a302f4f19 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.616+0000 7f24b5ffb640 1 -- 172.21.15.43:0/1442990024 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0004100 con 0x55fa56c7bb60 2024-04-21T08:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.616+0000 7f24b5ffb640 1 -- 172.21.15.43:0/1442990024 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c0005500 con 0x55fa56c7bb60 2024-04-21T08:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.616+0000 7f24b5ffb640 1 -- 172.21.15.43:0/1442990024 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0004cd0 con 0x55fa56c7bb60 2024-04-21T08:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.617+0000 7f24d960b740 1 -- 172.21.15.43:0/1442990024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7bb60 msgr2=0x55fa56c950c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.617+0000 7f24d960b740 1 --2- 172.21.15.43:0/1442990024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7bb60 0x55fa56c950c0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f24c002cb50 tx=0x7f24c002cb80 comp rx=0 tx=0).stop 2024-04-21T08:46:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.617+0000 7f24d960b740 1 -- 172.21.15.43:0/1442990024 shutdown_connections 2024-04-21T08:46:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.617+0000 7f24d960b740 1 --2- 172.21.15.43:0/1442990024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa56c894f0 0x55fa56c88df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.617+0000 7f24d960b740 1 --2- 172.21.15.43:0/1442990024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa56c7c8d0 0x55fa56c888b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.617+0000 7f24d960b740 1 --2- 172.21.15.43:0/1442990024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7bb60 0x55fa56c950c0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.617+0000 7f24d960b740 1 -- 172.21.15.43:0/1442990024 >> 172.21.15.43:0/1442990024 conn(0x55fa56c42c90 msgr2=0x55fa56c81490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.617+0000 7f24d960b740 1 -- 172.21.15.43:0/1442990024 shutdown_connections 2024-04-21T08:46:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.617+0000 7f24d960b740 1 -- 172.21.15.43:0/1442990024 wait complete. 2024-04-21T08:46:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.617+0000 7f24d960b740 1 -- start start 2024-04-21T08:46:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.618+0000 7f24d960b740 1 Processor -- start 2024-04-21T08:46:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.618+0000 7f24d960b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7bb60 0x55fa56c917e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.618+0000 7f24d960b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa56c7c8d0 0x55fa56c8cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.618+0000 7f24d960b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa56c894f0 0x55fa56c8d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.618+0000 7f24d960b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01ae660 con 0x55fa56c7bb60 2024-04-21T08:46:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.618+0000 7f24d960b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01adf90 con 0x55fa56c7c8d0 2024-04-21T08:46:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.618+0000 7f24d960b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01ab1d0 con 0x55fa56c894f0 2024-04-21T08:46:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.618+0000 7f24d960b740 10 client.0 _ll_register_callbacks cb 0x7f24c400b940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.618+0000 7f24cffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7bb60 0x55fa56c917e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.618+0000 7f24cffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7bb60 0x55fa56c917e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54508/0 (socket says 172.21.15.43:54508) 2024-04-21T08:46:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.618+0000 7f24cffff640 1 -- 172.21.15.43:0/1496968308 learned_addr learned my addr 172.21.15.43:0/1496968308 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.618+0000 7f24cf7fe640 1 --2- 172.21.15.43:0/1496968308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa56c7c8d0 0x55fa56c8cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.618+0000 7f24cffff640 1 -- 172.21.15.43:0/1496968308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa56c894f0 msgr2=0x55fa56c8d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.618+0000 7f24cffff640 1 --2- 172.21.15.43:0/1496968308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa56c894f0 0x55fa56c8d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.618+0000 7f24cffff640 1 -- 172.21.15.43:0/1496968308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa56c7c8d0 msgr2=0x55fa56c8cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.618+0000 7f24cffff640 1 --2- 172.21.15.43:0/1496968308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa56c7c8d0 0x55fa56c8cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.618+0000 7f24cffff640 1 -- 172.21.15.43:0/1496968308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa56c8d6d0 con 0x55fa56c7bb60 2024-04-21T08:46:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.618+0000 7f24cffff640 1 --2- 172.21.15.43:0/1496968308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7bb60 0x55fa56c917e0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f24c002ca90 tx=0x7f24c0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4958319f589857 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.618+0000 7f24af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.619+0000 7f24af7fe640 1 -- 172.21.15.43:0/1496968308 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0005260 con 0x55fa56c7bb60 2024-04-21T08:46:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.619+0000 7f24af7fe640 1 -- 172.21.15.43:0/1496968308 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f24c0004100 con 0x55fa56c7bb60 2024-04-21T08:46:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.619+0000 7f24d960b740 1 -- 172.21.15.43:0/1496968308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c0005500 con 0x55fa56c7bb60 2024-04-21T08:46:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.619+0000 7f24af7fe640 1 -- 172.21.15.43:0/1496968308 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24c005d050 con 0x55fa56c7bb60 2024-04-21T08:46:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.619+0000 7f24d960b740 10 client.19260 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.619+0000 7f24d960b740 1 -- 172.21.15.43:0/1496968308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fa56cff820 con 0x55fa56c7bb60 2024-04-21T08:46:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.619+0000 7f24af7fe640 1 -- 172.21.15.43:0/1496968308 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c0005500 con 0x55fa56c7bb60 2024-04-21T08:46:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.619+0000 7f24af7fe640 1 -- 172.21.15.43:0/1496968308 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f24c00037f0 con 0x55fa56c7bb60 2024-04-21T08:46:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.619+0000 7f24d960b740 10 client.19260 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.619+0000 7f24d960b740 10 client.19260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.619+0000 7f24d960b740 20 client.19260 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.619+0000 7f24d960b740 1 --2- 172.21.15.43:0/1496968308 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fa56cffa40 0x55fa56d01e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.619+0000 7f24d960b740 4 client.19260 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:46:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.619+0000 7f24d960b740 1 -- 172.21.15.43:0/1496968308 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa56b803d0 con 0x55fa56cffa40 2024-04-21T08:46:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.619+0000 7f24cf7fe640 1 --2- 172.21.15.43:0/1496968308 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fa56cffa40 0x55fa56d01e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.620+0000 7f24af7fe640 10 client.19260 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.620+0000 7f24cf7fe640 1 --2- 172.21.15.43:0/1496968308 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fa56cffa40 0x55fa56d01e60 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eef0be5f43056bb0 server_cookie=266316f3cd63e78f in_seq=0 out_seq=0 2024-04-21T08:46:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.634+0000 7f24af7fe640 1 -- 172.21.15.43:0/1496968308 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fa56b803d0 con 0x55fa56cffa40 2024-04-21T08:46:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.634+0000 7f24af7fe640 10 client.19260 handle_command_reply: tid=0 2024-04-21T08:46:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.634+0000 7f24d960b740 1 client.19260 shutdown 2024-04-21T08:46:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.634+0000 7f24d960b740 1 -- 172.21.15.43:0/1496968308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7bb60 msgr2=0x55fa56c917e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.634+0000 7f24d960b740 1 --2- 172.21.15.43:0/1496968308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7bb60 0x55fa56c917e0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f24c002ca90 tx=0x7f24c0005fd0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.634+0000 7f24d960b740 1 -- 172.21.15.43:0/1496968308 shutdown_connections 2024-04-21T08:46:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.634+0000 7f24d960b740 1 --2- 172.21.15.43:0/1496968308 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55fa56cffa40 0x55fa56d01e60 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.634+0000 7f24d960b740 1 --2- 172.21.15.43:0/1496968308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fa56c894f0 0x55fa56c8d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.634+0000 7f24d960b740 1 --2- 172.21.15.43:0/1496968308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa56c7c8d0 0x55fa56c8cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.634+0000 7f24d960b740 1 --2- 172.21.15.43:0/1496968308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa56c7bb60 0x55fa56c917e0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.634+0000 7f24af7fe640 0 client.19260 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.634+0000 7f24d960b740 1 -- 172.21.15.43:0/1496968308 >> 172.21.15.43:0/1496968308 conn(0x55fa56c42c90 msgr2=0x55fa56c77be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.634+0000 7f24d960b740 1 -- 172.21.15.43:0/1496968308 shutdown_connections 2024-04-21T08:46:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.634+0000 7f24d960b740 1 -- 172.21.15.43:0/1496968308 wait complete. 2024-04-21T08:46:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.634+0000 7f24d960b740 20 client.19260 trim_cache size 0 max 16384 2024-04-21T08:46:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:46:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:46:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:46:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:46:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.635+0000 7f24d6a21640 1 -- 172.21.15.43:0/1866485706 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f24b80a3dc0 msgr2=0x7f24b80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.635+0000 7f24d6a21640 1 --2- 172.21.15.43:0/1866485706 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f24b80a3dc0 0x7f24b80a6280 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55fa56c7d030 tx=0x7f24c400b770 comp rx=0 tx=0).stop 2024-04-21T08:46:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.635+0000 7f24d6a21640 1 -- 172.21.15.43:0/1866485706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24d010c5b0 msgr2=0x7f24d019cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.635+0000 7f24d6a21640 1 --2- 172.21.15.43:0/1866485706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24d010c5b0 0x7f24d019cf90 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f24c800e9d0 tx=0x7f24c800ee90 comp rx=0 tx=0).stop 2024-04-21T08:46:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.635+0000 7f24d6a21640 1 -- 172.21.15.43:0/1866485706 shutdown_connections 2024-04-21T08:46:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.635+0000 7f24d6a21640 1 --2- 172.21.15.43:0/1866485706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24d010c5b0 0x7f24d019cf90 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.635+0000 7f24d6a21640 1 --2- 172.21.15.43:0/1866485706 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f24b80a3dc0 0x7f24b80a6280 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.635+0000 7f24d6a21640 1 --2- 172.21.15.43:0/1866485706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24d0103b00 0x7f24d0198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.635+0000 7f24d6a21640 1 --2- 172.21.15.43:0/1866485706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24d01031e0 0x7f24d0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.635+0000 7f24d6a21640 1 -- 172.21.15.43:0/1866485706 >> 172.21.15.43:0/1866485706 conn(0x7f24d0077f60 msgr2=0x7f24d00fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.635+0000 7f24d6a21640 1 -- 172.21.15.43:0/1866485706 shutdown_connections 2024-04-21T08:46:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:43.635+0000 7f24d6a21640 1 -- 172.21.15.43:0/1866485706 wait complete. 2024-04-21T08:46:43.855 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:46:43.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:46:43.855 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:46:43.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.966+0000 7f1e531e6640 1 -- 172.21.15.43:0/1487895387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e4c1072b0 msgr2=0x7f1e4c10ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:43.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.966+0000 7f1e531e6640 1 --2- 172.21.15.43:0/1487895387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e4c1072b0 0x7f1e4c10ee80 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009ca0 tx=0x7f1e4402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.966+0000 7f1e531e6640 1 -- 172.21.15.43:0/1487895387 shutdown_connections 2024-04-21T08:46:43.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.966+0000 7f1e531e6640 1 --2- 172.21.15.43:0/1487895387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e4c1072b0 0x7f1e4c10ee80 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.966+0000 7f1e531e6640 1 --2- 172.21.15.43:0/1487895387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e4c102b90 0x7f1e4c106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.966+0000 7f1e531e6640 1 --2- 172.21.15.43:0/1487895387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e4c1021e0 0x7f1e4c1025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.966+0000 7f1e531e6640 1 -- 172.21.15.43:0/1487895387 >> 172.21.15.43:0/1487895387 conn(0x7f1e4c0fda60 msgr2=0x7f1e4c0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:43.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.966+0000 7f1e531e6640 1 -- 172.21.15.43:0/1487895387 shutdown_connections 2024-04-21T08:46:43.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.967+0000 7f1e531e6640 1 -- 172.21.15.43:0/1487895387 wait complete. 2024-04-21T08:46:43.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.967+0000 7f1e531e6640 1 Processor -- start 2024-04-21T08:46:43.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.967+0000 7f1e531e6640 1 -- start start 2024-04-21T08:46:43.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.967+0000 7f1e531e6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e4c1021e0 0x7f1e4c19e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.967+0000 7f1e531e6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e4c1072b0 0x7f1e4c19ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.967+0000 7f1e531e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e4c1986e0 0x7f1e4c198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.967+0000 7f1e531e6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c112220 con 0x7f1e4c1986e0 2024-04-21T08:46:43.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.967+0000 7f1e531e6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1120a0 con 0x7f1e4c1072b0 2024-04-21T08:46:43.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.967+0000 7f1e531e6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1123a0 con 0x7f1e4c1021e0 2024-04-21T08:46:43.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.967+0000 7f1e51783640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e4c1986e0 0x7f1e4c198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.967+0000 7f1e51783640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e4c1986e0 0x7f1e4c198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54536/0 (socket says 172.21.15.43:54536) 2024-04-21T08:46:43.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.967+0000 7f1e51783640 1 -- 172.21.15.43:0/64125559 learned_addr learned my addr 172.21.15.43:0/64125559 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:43.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.968+0000 7f1e4bfff640 1 --2- 172.21.15.43:0/64125559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e4c1072b0 0x7f1e4c19ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.968+0000 7f1e51783640 1 -- 172.21.15.43:0/64125559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e4c1021e0 msgr2=0x7f1e4c19e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:43.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.968+0000 7f1e51783640 1 --2- 172.21.15.43:0/64125559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e4c1021e0 0x7f1e4c19e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.968+0000 7f1e51783640 1 -- 172.21.15.43:0/64125559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e4c1072b0 msgr2=0x7f1e4c19ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:43.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.968+0000 7f1e51783640 1 --2- 172.21.15.43:0/64125559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e4c1072b0 0x7f1e4c19ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.968+0000 7f1e51783640 1 -- 172.21.15.43:0/64125559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4c1992c0 con 0x7f1e4c1986e0 2024-04-21T08:46:43.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.968+0000 7f1e4bfff640 1 --2- 172.21.15.43:0/64125559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e4c1072b0 0x7f1e4c19ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:43.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.968+0000 7f1e51783640 1 --2- 172.21.15.43:0/64125559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e4c1986e0 0x7f1e4c198b90 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f1e44005e30 tx=0x7f1e44034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:43.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.968+0000 7f1e49ffb640 1 -- 172.21.15.43:0/64125559 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44039040 con 0x7f1e4c1986e0 2024-04-21T08:46:43.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.968+0000 7f1e49ffb640 1 -- 172.21.15.43:0/64125559 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e44044070 con 0x7f1e4c1986e0 2024-04-21T08:46:43.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.968+0000 7f1e49ffb640 1 -- 172.21.15.43:0/64125559 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44003e60 con 0x7f1e4c1986e0 2024-04-21T08:46:43.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.968+0000 7f1e531e6640 1 -- 172.21.15.43:0/64125559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4c1a53a0 con 0x7f1e4c1986e0 2024-04-21T08:46:43.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.968+0000 7f1e531e6640 1 -- 172.21.15.43:0/64125559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e4c1a5750 con 0x7f1e4c1986e0 2024-04-21T08:46:43.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.970+0000 7f1e49ffb640 1 -- 172.21.15.43:0/64125559 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1e440324c0 con 0x7f1e4c1986e0 2024-04-21T08:46:43.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.971+0000 7f1e55dd1740 1 Processor -- start 2024-04-21T08:46:43.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.971+0000 7f1e55dd1740 1 -- start start 2024-04-21T08:46:43.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.971+0000 7f1e55dd1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b4812e7c0 0x564b4812eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.971+0000 7f1e55dd1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b4812f170 0x564b48137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.971+0000 7f1e55dd1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b48138270 0x564b481386d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.971+0000 7f1e55dd1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564b48092f00 con 0x564b4812e7c0 2024-04-21T08:46:43.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.971+0000 7f1e55dd1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564b48030590 con 0x564b48138270 2024-04-21T08:46:43.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.971+0000 7f1e55dd1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564b4808eec0 con 0x564b4812f170 2024-04-21T08:46:43.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.971+0000 7f1e49ffb640 1 --2- 172.21.15.43:0/64125559 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1e380a3da0 0x7f1e380a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.971+0000 7f1e4bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b4812e7c0 0x564b4812eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.971+0000 7f1e4bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b4812e7c0 0x564b4812eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54540/0 (socket says 172.21.15.43:54540) 2024-04-21T08:46:43.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.971+0000 7f1e4bfff640 1 -- 172.21.15.43:0/2566185656 learned_addr learned my addr 172.21.15.43:0/2566185656 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:43.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.971+0000 7f1e51783640 1 --2- 172.21.15.43:0/64125559 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1e380a3da0 0x7f1e380a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.971+0000 7f1e4bfff640 1 --2- 172.21.15.43:0/2566185656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b48138270 0x564b481386d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.971+0000 7f1e50f82640 1 --2- 172.21.15.43:0/2566185656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b4812f170 0x564b48137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.971+0000 7f1e49ffb640 1 -- 172.21.15.43:0/64125559 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f1e440f2dd0 con 0x7f1e4c1986e0 2024-04-21T08:46:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.972+0000 7f1e4bfff640 1 -- 172.21.15.43:0/2566185656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b4812f170 msgr2=0x564b48137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.972+0000 7f1e4bfff640 1 --2- 172.21.15.43:0/2566185656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b4812f170 0x564b48137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.972+0000 7f1e4bfff640 1 -- 172.21.15.43:0/2566185656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b48138270 msgr2=0x564b481386d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.972+0000 7f1e4bfff640 1 --2- 172.21.15.43:0/2566185656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b48138270 0x564b481386d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.972+0000 7f1e4bfff640 1 -- 172.21.15.43:0/2566185656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b47e6d7e0 con 0x564b4812e7c0 2024-04-21T08:46:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.972+0000 7f1e4bfff640 1 --2- 172.21.15.43:0/2566185656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b4812e7c0 0x564b4812eba0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0058c0 tx=0x7f1e3c009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217e203db8f56fee server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.972+0000 7f1e51783640 1 --2- 172.21.15.43:0/64125559 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1e380a3da0 0x7f1e380a6260 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1e440f0310 tx=0x7f1e440095c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.972+0000 7f1e42ffd640 1 -- 172.21.15.43:0/2566185656 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c0150d0 con 0x564b4812e7c0 2024-04-21T08:46:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.972+0000 7f1e42ffd640 1 -- 172.21.15.43:0/2566185656 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e3c015270 con 0x564b4812e7c0 2024-04-21T08:46:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.972+0000 7f1e42ffd640 1 -- 172.21.15.43:0/2566185656 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c004e10 con 0x564b4812e7c0 2024-04-21T08:46:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.972+0000 7f1e55dd1740 1 -- 172.21.15.43:0/2566185656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b4812e7c0 msgr2=0x564b4812eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.972+0000 7f1e55dd1740 1 --2- 172.21.15.43:0/2566185656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b4812e7c0 0x564b4812eba0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0058c0 tx=0x7f1e3c009550 comp rx=0 tx=0).stop 2024-04-21T08:46:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.973+0000 7f1e55dd1740 1 -- 172.21.15.43:0/2566185656 shutdown_connections 2024-04-21T08:46:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.973+0000 7f1e55dd1740 1 --2- 172.21.15.43:0/2566185656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b48138270 0x564b481386d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.973+0000 7f1e55dd1740 1 --2- 172.21.15.43:0/2566185656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b4812f170 0x564b48137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.973+0000 7f1e55dd1740 1 --2- 172.21.15.43:0/2566185656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b4812e7c0 0x564b4812eba0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.973+0000 7f1e55dd1740 1 -- 172.21.15.43:0/2566185656 >> 172.21.15.43:0/2566185656 conn(0x564b48128fb0 msgr2=0x564b48129390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.973+0000 7f1e55dd1740 1 -- 172.21.15.43:0/2566185656 shutdown_connections 2024-04-21T08:46:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.973+0000 7f1e55dd1740 1 -- 172.21.15.43:0/2566185656 wait complete. 2024-04-21T08:46:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.973+0000 7f1e55dd1740 1 -- start start 2024-04-21T08:46:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.973+0000 7f1e55dd1740 1 Processor -- start 2024-04-21T08:46:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.973+0000 7f1e55dd1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b4812e7c0 0x564b4813bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.973+0000 7f1e55dd1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b4812f170 0x564b4813c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.973+0000 7f1e55dd1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b48138270 0x564b4813c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.973+0000 7f1e55dd1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564b4813f150 con 0x564b48138270 2024-04-21T08:46:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.973+0000 7f1e55dd1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564b4804c580 con 0x564b4812e7c0 2024-04-21T08:46:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.973+0000 7f1e55dd1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564b4813f2d0 con 0x564b4812f170 2024-04-21T08:46:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.973+0000 7f1e55dd1740 10 client.0 _ll_register_callbacks cb 0x564b48052650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.974+0000 7f1e4bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b48138270 0x564b4813c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.974+0000 7f1e4bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b48138270 0x564b4813c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54550/0 (socket says 172.21.15.43:54550) 2024-04-21T08:46:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.974+0000 7f1e4bfff640 1 -- 172.21.15.43:0/3367943151 learned_addr learned my addr 172.21.15.43:0/3367943151 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.974+0000 7f1e4bfff640 1 --2- 172.21.15.43:0/3367943151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b4812e7c0 0x564b4813bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.974+0000 7f1e50f82640 1 --2- 172.21.15.43:0/3367943151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b4812f170 0x564b4813c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.974+0000 7f1e4bfff640 1 -- 172.21.15.43:0/3367943151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b4812f170 msgr2=0x564b4813c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.974+0000 7f1e4bfff640 1 --2- 172.21.15.43:0/3367943151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b4812f170 0x564b4813c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.974+0000 7f1e4bfff640 1 -- 172.21.15.43:0/3367943151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b4812e7c0 msgr2=0x564b4813bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.974+0000 7f1e4bfff640 1 --2- 172.21.15.43:0/3367943151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b4812e7c0 0x564b4813bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.974+0000 7f1e4bfff640 1 -- 172.21.15.43:0/3367943151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b47ecdf70 con 0x564b48138270 2024-04-21T08:46:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.974+0000 7f1e4bfff640 1 --2- 172.21.15.43:0/3367943151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b48138270 0x564b4813c780 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0096e0 tx=0x7f1e3c017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebf730abb37fe80 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:43.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.974+0000 7f1e2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:43.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.974+0000 7f1e2bfff640 1 -- 172.21.15.43:0/3367943151 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c010920 con 0x564b48138270 2024-04-21T08:46:43.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.974+0000 7f1e2bfff640 1 -- 172.21.15.43:0/3367943151 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f1e3c010ac0 con 0x564b48138270 2024-04-21T08:46:43.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.974+0000 7f1e55dd1740 1 -- 172.21.15.43:0/3367943151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e3c015270 con 0x564b48138270 2024-04-21T08:46:43.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.974+0000 7f1e2bfff640 1 -- 172.21.15.43:0/3367943151 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1e3c010cc0 con 0x564b48138270 2024-04-21T08:46:43.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.974+0000 7f1e55dd1740 10 client.19272 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:43.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.974+0000 7f1e55dd1740 1 -- 172.21.15.43:0/3367943151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564b47d65170 con 0x564b48138270 2024-04-21T08:46:43.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.975+0000 7f1e2bfff640 1 -- 172.21.15.43:0/3367943151 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e3c015270 con 0x564b48138270 2024-04-21T08:46:43.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.975+0000 7f1e2bfff640 1 -- 172.21.15.43:0/3367943151 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1e3c022400 con 0x564b48138270 2024-04-21T08:46:43.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.975+0000 7f1e55dd1740 10 client.19272 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:43.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.975+0000 7f1e55dd1740 10 client.19272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:43.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.975+0000 7f1e55dd1740 20 client.19272 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:43.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.975+0000 7f1e55dd1740 1 --2- 172.21.15.43:0/3367943151 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564b4814c610 0x564b4814ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.975+0000 7f1e55dd1740 4 client.19272 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:43.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.975+0000 7f1e55dd1740 1 -- 172.21.15.43:0/3367943151 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b480a0920 con 0x564b4814c610 2024-04-21T08:46:43.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.975+0000 7f1e50f82640 1 --2- 172.21.15.43:0/3367943151 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564b4814c610 0x564b4814ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.976+0000 7f1e50f82640 1 --2- 172.21.15.43:0/3367943151 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564b4814c610 0x564b4814ea00 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b51b44d14e1be005 server_cookie=5333a1f678cf50b2 in_seq=0 out_seq=0 2024-04-21T08:46:43.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.976+0000 7f1e2bfff640 10 client.19272 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:43.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.976+0000 7f1e2bfff640 1 -- 172.21.15.43:0/3367943151 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564b480a0920 con 0x564b4814c610 2024-04-21T08:46:43.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.976+0000 7f1e2bfff640 10 client.19272 handle_command_reply: tid=0 2024-04-21T08:46:43.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.976+0000 7f1e55dd1740 1 client.19272 shutdown 2024-04-21T08:46:43.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.976+0000 7f1e55dd1740 1 -- 172.21.15.43:0/3367943151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b48138270 msgr2=0x564b4813c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:43.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.976+0000 7f1e55dd1740 1 --2- 172.21.15.43:0/3367943151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b48138270 0x564b4813c780 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0096e0 tx=0x7f1e3c017040 comp rx=0 tx=0).stop 2024-04-21T08:46:43.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.976+0000 7f1e55dd1740 1 -- 172.21.15.43:0/3367943151 shutdown_connections 2024-04-21T08:46:43.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.976+0000 7f1e55dd1740 1 --2- 172.21.15.43:0/3367943151 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564b4814c610 0x564b4814ea00 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.976+0000 7f1e55dd1740 1 --2- 172.21.15.43:0/3367943151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b48138270 0x564b4813c780 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.976+0000 7f1e2bfff640 0 client.19272 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:43.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.976+0000 7f1e55dd1740 1 --2- 172.21.15.43:0/3367943151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b4812f170 0x564b4813c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.976+0000 7f1e55dd1740 1 --2- 172.21.15.43:0/3367943151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b4812e7c0 0x564b4813bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.976+0000 7f1e55dd1740 1 -- 172.21.15.43:0/3367943151 >> 172.21.15.43:0/3367943151 conn(0x564b48128fb0 msgr2=0x564b4812ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:43.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.977+0000 7f1e55dd1740 1 -- 172.21.15.43:0/3367943151 shutdown_connections 2024-04-21T08:46:43.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.977+0000 7f1e55dd1740 1 -- 172.21.15.43:0/3367943151 wait complete. 2024-04-21T08:46:43.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.977+0000 7f1e55dd1740 20 client.19272 trim_cache size 0 max 16384 2024-04-21T08:46:43.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.994+0000 7f1e55dd1740 1 Processor -- start 2024-04-21T08:46:43.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.994+0000 7f1e55dd1740 1 -- start start 2024-04-21T08:46:43.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.994+0000 7f1e55dd1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b4812e7c0 0x564b48148080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.994+0000 7f1e55dd1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b4812f170 0x564b4813b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.994+0000 7f1e55dd1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b48138270 0x564b4813db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.994+0000 7f1e55dd1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1ad290 con 0x564b4812e7c0 2024-04-21T08:46:43.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.994+0000 7f1e55dd1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1aedd0 con 0x564b4812f170 2024-04-21T08:46:43.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.994+0000 7f1e55dd1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1ad960 con 0x564b48138270 2024-04-21T08:46:43.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.994+0000 7f1e4bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b4812e7c0 0x564b48148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.994+0000 7f1e4bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b4812e7c0 0x564b48148080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54560/0 (socket says 172.21.15.43:54560) 2024-04-21T08:46:43.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.994+0000 7f1e4bfff640 1 -- 172.21.15.43:0/3080032507 learned_addr learned my addr 172.21.15.43:0/3080032507 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:43.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.995+0000 7f1e4bfff640 1 -- 172.21.15.43:0/3080032507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b48138270 msgr2=0x564b4813db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:43.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.995+0000 7f1e4bfff640 1 --2- 172.21.15.43:0/3080032507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b48138270 0x564b4813db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.995+0000 7f1e4bfff640 1 -- 172.21.15.43:0/3080032507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b4812f170 msgr2=0x564b4813b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:43.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.995+0000 7f1e4bfff640 1 --2- 172.21.15.43:0/3080032507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b4812f170 0x564b4813b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.995+0000 7f1e4bfff640 1 -- 172.21.15.43:0/3080032507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b4813f090 con 0x564b4812e7c0 2024-04-21T08:46:43.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.995+0000 7f1e4bfff640 1 --2- 172.21.15.43:0/3080032507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b4812e7c0 0x564b48148080 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c000c00 tx=0x7f1e3c005fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a721550cde2ab3bc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:43.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.995+0000 7f1e427fc640 1 -- 172.21.15.43:0/3080032507 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c010e60 con 0x564b4812e7c0 2024-04-21T08:46:43.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.995+0000 7f1e427fc640 1 -- 172.21.15.43:0/3080032507 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e3c022e50 con 0x564b4812e7c0 2024-04-21T08:46:43.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.995+0000 7f1e427fc640 1 -- 172.21.15.43:0/3080032507 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c0150d0 con 0x564b4812e7c0 2024-04-21T08:46:43.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.995+0000 7f1e55dd1740 1 -- 172.21.15.43:0/3080032507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b4812e7c0 msgr2=0x564b48148080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:43.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.995+0000 7f1e55dd1740 1 --2- 172.21.15.43:0/3080032507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b4812e7c0 0x564b48148080 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c000c00 tx=0x7f1e3c005fc0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.995+0000 7f1e55dd1740 1 -- 172.21.15.43:0/3080032507 shutdown_connections 2024-04-21T08:46:43.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.995+0000 7f1e55dd1740 1 --2- 172.21.15.43:0/3080032507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b48138270 0x564b4813db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.995+0000 7f1e55dd1740 1 --2- 172.21.15.43:0/3080032507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b4812f170 0x564b4813b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.995+0000 7f1e55dd1740 1 --2- 172.21.15.43:0/3080032507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b4812e7c0 0x564b48148080 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.995+0000 7f1e55dd1740 1 -- 172.21.15.43:0/3080032507 >> 172.21.15.43:0/3080032507 conn(0x564b48128fb0 msgr2=0x564b4813d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:43.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.996+0000 7f1e55dd1740 1 -- 172.21.15.43:0/3080032507 shutdown_connections 2024-04-21T08:46:44.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.996+0000 7f1e55dd1740 1 -- 172.21.15.43:0/3080032507 wait complete. 2024-04-21T08:46:44.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.996+0000 7f1e55dd1740 1 -- start start 2024-04-21T08:46:44.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.996+0000 7f1e55dd1740 1 Processor -- start 2024-04-21T08:46:44.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.996+0000 7f1e55dd1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b4812e7c0 0x564b481425f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:44.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.996+0000 7f1e55dd1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b4812f170 0x564b48142b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:44.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.996+0000 7f1e55dd1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b48138270 0x564b481b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:44.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.996+0000 7f1e55dd1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1ae700 con 0x564b4812e7c0 2024-04-21T08:46:44.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.996+0000 7f1e55dd1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1ae030 con 0x564b48138270 2024-04-21T08:46:44.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.996+0000 7f1e55dd1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1ab270 con 0x564b4812f170 2024-04-21T08:46:44.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.996+0000 7f1e55dd1740 10 client.0 _ll_register_callbacks cb 0x7f1e3c009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:44.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.996+0000 7f1e4bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b4812e7c0 0x564b481425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:44.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.996+0000 7f1e4bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b4812e7c0 0x564b481425f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54568/0 (socket says 172.21.15.43:54568) 2024-04-21T08:46:44.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.996+0000 7f1e4bfff640 1 -- 172.21.15.43:0/353684327 learned_addr learned my addr 172.21.15.43:0/353684327 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:44.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.997+0000 7f1e50f82640 1 --2- 172.21.15.43:0/353684327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b4812f170 0x564b48142b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:44.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.997+0000 7f1e4bfff640 1 --2- 172.21.15.43:0/353684327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b48138270 0x564b481b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:44.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.997+0000 7f1e4bfff640 1 -- 172.21.15.43:0/353684327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b4812f170 msgr2=0x564b48142b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:44.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.997+0000 7f1e4bfff640 1 --2- 172.21.15.43:0/353684327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b4812f170 0x564b48142b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.997+0000 7f1e4bfff640 1 -- 172.21.15.43:0/353684327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b48138270 msgr2=0x564b481b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:44.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.997+0000 7f1e4bfff640 1 --2- 172.21.15.43:0/353684327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b48138270 0x564b481b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.997+0000 7f1e4bfff640 1 -- 172.21.15.43:0/353684327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b481b1f50 con 0x564b4812e7c0 2024-04-21T08:46:44.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.997+0000 7f1e4bfff640 1 --2- 172.21.15.43:0/353684327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b4812e7c0 0x564b481425f0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c022950 tx=0x7f1e3c009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5347eeb261f1b31f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:44.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.997+0000 7f1e2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:44.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.997+0000 7f1e2bfff640 1 -- 172.21.15.43:0/353684327 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c0152e0 con 0x564b4812e7c0 2024-04-21T08:46:44.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.997+0000 7f1e55dd1740 1 -- 172.21.15.43:0/353684327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e3c022e50 con 0x564b4812e7c0 2024-04-21T08:46:44.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.997+0000 7f1e2bfff640 1 -- 172.21.15.43:0/353684327 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f1e3c022400 con 0x564b4812e7c0 2024-04-21T08:46:44.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.997+0000 7f1e2bfff640 1 -- 172.21.15.43:0/353684327 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1e3c009550 con 0x564b4812e7c0 2024-04-21T08:46:44.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.997+0000 7f1e55dd1740 10 client.19278 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:44.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.997+0000 7f1e55dd1740 1 -- 172.21.15.43:0/353684327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564b481b3190 con 0x564b4812e7c0 2024-04-21T08:46:44.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.997+0000 7f1e2bfff640 1 -- 172.21.15.43:0/353684327 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e3c022e50 con 0x564b4812e7c0 2024-04-21T08:46:44.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.998+0000 7f1e2bfff640 1 -- 172.21.15.43:0/353684327 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1e3c01f900 con 0x564b4812e7c0 2024-04-21T08:46:44.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.998+0000 7f1e55dd1740 10 client.19278 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:44.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.998+0000 7f1e55dd1740 10 client.19278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:44.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.998+0000 7f1e55dd1740 20 client.19278 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:44.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.998+0000 7f1e55dd1740 1 --2- 172.21.15.43:0/353684327 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564b481b37a0 0x564b481b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:44.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.998+0000 7f1e55dd1740 4 client.19278 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:46:44.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.998+0000 7f1e55dd1740 1 -- 172.21.15.43:0/353684327 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564b4809a790 con 0x564b481b37a0 2024-04-21T08:46:44.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.998+0000 7f1e50f82640 1 --2- 172.21.15.43:0/353684327 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564b481b37a0 0x564b481b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:44.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.998+0000 7f1e2bfff640 10 client.19278 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:44.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:43.998+0000 7f1e50f82640 1 --2- 172.21.15.43:0/353684327 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564b481b37a0 0x564b481b5b90 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d67b07eaf4b7888 server_cookie=17e522c8a9094344 in_seq=0 out_seq=0 2024-04-21T08:46:44.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.000+0000 7f1e2bfff640 1 -- 172.21.15.43:0/353684327 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564b4809a790 con 0x564b481b37a0 2024-04-21T08:46:44.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.000+0000 7f1e2bfff640 10 client.19278 handle_command_reply: tid=0 2024-04-21T08:46:44.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.000+0000 7f1e55dd1740 1 client.19278 shutdown 2024-04-21T08:46:44.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.000+0000 7f1e55dd1740 1 -- 172.21.15.43:0/353684327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b4812e7c0 msgr2=0x564b481425f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:44.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.000+0000 7f1e55dd1740 1 --2- 172.21.15.43:0/353684327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b4812e7c0 0x564b481425f0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c022950 tx=0x7f1e3c009f70 comp rx=0 tx=0).stop 2024-04-21T08:46:44.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.000+0000 7f1e55dd1740 1 -- 172.21.15.43:0/353684327 shutdown_connections 2024-04-21T08:46:44.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.000+0000 7f1e55dd1740 1 --2- 172.21.15.43:0/353684327 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x564b481b37a0 0x564b481b5b90 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.000+0000 7f1e55dd1740 1 --2- 172.21.15.43:0/353684327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b48138270 0x564b481b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.000+0000 7f1e55dd1740 1 --2- 172.21.15.43:0/353684327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564b4812f170 0x564b48142b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.000+0000 7f1e55dd1740 1 --2- 172.21.15.43:0/353684327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b4812e7c0 0x564b481425f0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.000+0000 7f1e2bfff640 0 client.19278 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:44.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.000+0000 7f1e55dd1740 1 -- 172.21.15.43:0/353684327 >> 172.21.15.43:0/353684327 conn(0x564b48128fb0 msgr2=0x564b4813c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:44.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.000+0000 7f1e55dd1740 1 -- 172.21.15.43:0/353684327 shutdown_connections 2024-04-21T08:46:44.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.000+0000 7f1e55dd1740 1 -- 172.21.15.43:0/353684327 wait complete. 2024-04-21T08:46:44.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.000+0000 7f1e55dd1740 20 client.19278 trim_cache size 0 max 16384 2024-04-21T08:46:44.005 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:46:44.005 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:46:44.005 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "47b37bbf-792b-43aa-bdf2-a7eacad516c7", 2024-04-21T08:46:44.005 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:46:44.005 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:46:44.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.001+0000 7f1e531e6640 1 -- 172.21.15.43:0/64125559 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1e380a3da0 msgr2=0x7f1e380a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:44.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.001+0000 7f1e531e6640 1 --2- 172.21.15.43:0/64125559 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1e380a3da0 0x7f1e380a6260 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1e440f0310 tx=0x7f1e440095c0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.001+0000 7f1e531e6640 1 -- 172.21.15.43:0/64125559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e4c1986e0 msgr2=0x7f1e4c198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:44.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.001+0000 7f1e531e6640 1 --2- 172.21.15.43:0/64125559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e4c1986e0 0x7f1e4c198b90 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f1e44005e30 tx=0x7f1e44034760 comp rx=0 tx=0).stop 2024-04-21T08:46:44.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.001+0000 7f1e531e6640 1 -- 172.21.15.43:0/64125559 shutdown_connections 2024-04-21T08:46:44.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.001+0000 7f1e531e6640 1 --2- 172.21.15.43:0/64125559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e4c1986e0 0x7f1e4c198b90 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.001+0000 7f1e531e6640 1 --2- 172.21.15.43:0/64125559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e4c1072b0 0x7f1e4c19ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.001+0000 7f1e531e6640 1 --2- 172.21.15.43:0/64125559 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1e380a3da0 0x7f1e380a6260 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.001+0000 7f1e531e6640 1 --2- 172.21.15.43:0/64125559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e4c1021e0 0x7f1e4c19e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.001+0000 7f1e531e6640 1 -- 172.21.15.43:0/64125559 >> 172.21.15.43:0/64125559 conn(0x7f1e4c0fda60 msgr2=0x7f1e4c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:44.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.001+0000 7f1e531e6640 1 -- 172.21.15.43:0/64125559 shutdown_connections 2024-04-21T08:46:44.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.001+0000 7f1e531e6640 1 -- 172.21.15.43:0/64125559 wait complete. 2024-04-21T08:46:44.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:46:44.011 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:46:44.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.118+0000 7f4a6199f640 1 -- 172.21.15.43:0/2260686683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a5c100e70 msgr2=0x7f4a5c1069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:44.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.118+0000 7f4a6199f640 1 --2- 172.21.15.43:0/2260686683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a5c100e70 0x7f4a5c1069a0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f4a48009900 tx=0x7f4a4802bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:44.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.118+0000 7f4a6199f640 1 -- 172.21.15.43:0/2260686683 shutdown_connections 2024-04-21T08:46:44.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.118+0000 7f4a6199f640 1 --2- 172.21.15.43:0/2260686683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a5c107080 0x7f4a5c10ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.118+0000 7f4a6199f640 1 --2- 172.21.15.43:0/2260686683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a5c100e70 0x7f4a5c1069a0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.118+0000 7f4a6199f640 1 --2- 172.21.15.43:0/2260686683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a5c1004c0 0x7f4a5c1008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.118+0000 7f4a6199f640 1 -- 172.21.15.43:0/2260686683 >> 172.21.15.43:0/2260686683 conn(0x7f4a5c077170 msgr2=0x7f4a5c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:44.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.118+0000 7f4a6199f640 1 -- 172.21.15.43:0/2260686683 shutdown_connections 2024-04-21T08:46:44.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.118+0000 7f4a6199f640 1 -- 172.21.15.43:0/2260686683 wait complete. 2024-04-21T08:46:44.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.118+0000 7f4a6199f640 1 Processor -- start 2024-04-21T08:46:44.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.118+0000 7f4a6199f640 1 -- start start 2024-04-21T08:46:44.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.119+0000 7f4a6199f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a5c1004c0 0x7f4a5c101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:44.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.119+0000 7f4a6199f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a5c107080 0x7f4a5c101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:44.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.119+0000 7f4a6199f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a5c101eb0 0x7f4a5c1a6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:44.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.119+0000 7f4a6199f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c111fc0 con 0x7f4a5c101eb0 2024-04-21T08:46:44.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.119+0000 7f4a6199f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c111e40 con 0x7f4a5c1004c0 2024-04-21T08:46:44.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.119+0000 7f4a6199f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c112140 con 0x7f4a5c107080 2024-04-21T08:46:44.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.119+0000 7f4a6119e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a5c101eb0 0x7f4a5c1a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:44.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.119+0000 7f4a6119e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a5c101eb0 0x7f4a5c1a6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54592/0 (socket says 172.21.15.43:54592) 2024-04-21T08:46:44.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.119+0000 7f4a6119e640 1 -- 172.21.15.43:0/4201775135 learned_addr learned my addr 172.21.15.43:0/4201775135 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:44.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.119+0000 7f4a6099d640 1 --2- 172.21.15.43:0/4201775135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a5c1004c0 0x7f4a5c101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:44.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.119+0000 7f4a5bfff640 1 --2- 172.21.15.43:0/4201775135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a5c107080 0x7f4a5c101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:44.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.119+0000 7f4a6119e640 1 -- 172.21.15.43:0/4201775135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a5c107080 msgr2=0x7f4a5c101970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:44.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.119+0000 7f4a6119e640 1 --2- 172.21.15.43:0/4201775135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a5c107080 0x7f4a5c101970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.119+0000 7f4a6119e640 1 -- 172.21.15.43:0/4201775135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a5c1004c0 msgr2=0x7f4a5c101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:44.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.119+0000 7f4a6119e640 1 --2- 172.21.15.43:0/4201775135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a5c1004c0 0x7f4a5c101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.119+0000 7f4a6119e640 1 -- 172.21.15.43:0/4201775135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5c1a6e30 con 0x7f4a5c101eb0 2024-04-21T08:46:44.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.119+0000 7f4a5bfff640 1 --2- 172.21.15.43:0/4201775135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a5c107080 0x7f4a5c101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:44.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.119+0000 7f4a6119e640 1 --2- 172.21.15.43:0/4201775135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a5c101eb0 0x7f4a5c1a6730 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400eaa0 tx=0x7f4a5400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:44.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.120+0000 7f4a59ffb640 1 -- 172.21.15.43:0/4201775135 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5400cdb0 con 0x7f4a5c101eb0 2024-04-21T08:46:44.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.120+0000 7f4a59ffb640 1 -- 172.21.15.43:0/4201775135 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a54005020 con 0x7f4a5c101eb0 2024-04-21T08:46:44.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.120+0000 7f4a6199f640 1 -- 172.21.15.43:0/4201775135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a5c1a7120 con 0x7f4a5c101eb0 2024-04-21T08:46:44.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.120+0000 7f4a59ffb640 1 -- 172.21.15.43:0/4201775135 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a540116d0 con 0x7f4a5c101eb0 2024-04-21T08:46:44.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.120+0000 7f4a6199f640 1 -- 172.21.15.43:0/4201775135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a5c1a7690 con 0x7f4a5c101eb0 2024-04-21T08:46:44.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.122+0000 7f4a59ffb640 1 -- 172.21.15.43:0/4201775135 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4a540051d0 con 0x7f4a5c101eb0 2024-04-21T08:46:44.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.122+0000 7f4a657f1740 1 Processor -- start 2024-04-21T08:46:44.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.122+0000 7f4a657f1740 1 -- start start 2024-04-21T08:46:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.122+0000 7f4a59ffb640 1 --2- 172.21.15.43:0/4201775135 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4a4c0a3dd0 0x7f4a4c0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.122+0000 7f4a657f1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560463aa6710 0x560463aa6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.122+0000 7f4a657f1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560463aa7030 0x560463aaf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.122+0000 7f4a657f1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aafc10 0x560463ab2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.122+0000 7f4a657f1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560463a0b1e0 con 0x560463aafc10 2024-04-21T08:46:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.122+0000 7f4a657f1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5604639a8390 con 0x560463aa6710 2024-04-21T08:46:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.122+0000 7f4a657f1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560463a071a0 con 0x560463aa7030 2024-04-21T08:46:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.123+0000 7f4a5bfff640 1 --2- 172.21.15.43:0/4201775135 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4a4c0a3dd0 0x7f4a4c0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.123+0000 7f4a6119e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aafc10 0x560463ab2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.123+0000 7f4a6119e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aafc10 0x560463ab2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54600/0 (socket says 172.21.15.43:54600) 2024-04-21T08:46:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.123+0000 7f4a6119e640 1 -- 172.21.15.43:0/3173441196 learned_addr learned my addr 172.21.15.43:0/3173441196 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.123+0000 7f4a59ffb640 1 -- 172.21.15.43:0/4201775135 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f4a540d0c40 con 0x7f4a5c101eb0 2024-04-21T08:46:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.123+0000 7f4a6099d640 1 --2- 172.21.15.43:0/3173441196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560463aa6710 0x560463aa6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.123+0000 7f4a5bfff640 1 --2- 172.21.15.43:0/3173441196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560463aa7030 0x560463aaf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:44.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.123+0000 7f4a6119e640 1 -- 172.21.15.43:0/3173441196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560463aa7030 msgr2=0x560463aaf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:44.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.123+0000 7f4a6119e640 1 --2- 172.21.15.43:0/3173441196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560463aa7030 0x560463aaf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.123+0000 7f4a6119e640 1 -- 172.21.15.43:0/3173441196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560463aa6710 msgr2=0x560463aa6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:44.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.123+0000 7f4a6119e640 1 --2- 172.21.15.43:0/3173441196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560463aa6710 0x560463aa6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.123+0000 7f4a6119e640 1 -- 172.21.15.43:0/3173441196 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604637e77d0 con 0x560463aafc10 2024-04-21T08:46:44.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.123+0000 7f4a6119e640 1 --2- 172.21.15.43:0/3173441196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aafc10 0x560463ab2000 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f4a5409c7a0 tx=0x7f4a5401eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d310025e374eea server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:44.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.123+0000 7f4a5bfff640 1 --2- 172.21.15.43:0/4201775135 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4a4c0a3dd0 0x7f4a4c0a6290 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4a48006010 tx=0x7f4a48004390 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:44.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.123+0000 7f4a46ffd640 1 -- 172.21.15.43:0/3173441196 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5409c810 con 0x560463aafc10 2024-04-21T08:46:44.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.123+0000 7f4a46ffd640 1 -- 172.21.15.43:0/3173441196 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a540a0bc0 con 0x560463aafc10 2024-04-21T08:46:44.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.123+0000 7f4a46ffd640 1 -- 172.21.15.43:0/3173441196 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a540054f0 con 0x560463aafc10 2024-04-21T08:46:44.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.124+0000 7f4a657f1740 1 -- 172.21.15.43:0/3173441196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aafc10 msgr2=0x560463ab2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:44.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.124+0000 7f4a657f1740 1 --2- 172.21.15.43:0/3173441196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aafc10 0x560463ab2000 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f4a5409c7a0 tx=0x7f4a5401eb40 comp rx=0 tx=0).stop 2024-04-21T08:46:44.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.124+0000 7f4a657f1740 1 -- 172.21.15.43:0/3173441196 shutdown_connections 2024-04-21T08:46:44.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.124+0000 7f4a657f1740 1 --2- 172.21.15.43:0/3173441196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aafc10 0x560463ab2000 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.124+0000 7f4a657f1740 1 --2- 172.21.15.43:0/3173441196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560463aa7030 0x560463aaf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.124+0000 7f4a657f1740 1 --2- 172.21.15.43:0/3173441196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560463aa6710 0x560463aa6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.124+0000 7f4a657f1740 1 -- 172.21.15.43:0/3173441196 >> 172.21.15.43:0/3173441196 conn(0x560463aa0e40 msgr2=0x560463aa1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:44.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.124+0000 7f4a657f1740 1 -- 172.21.15.43:0/3173441196 shutdown_connections 2024-04-21T08:46:44.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.124+0000 7f4a657f1740 1 -- 172.21.15.43:0/3173441196 wait complete. 2024-04-21T08:46:44.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.124+0000 7f4a657f1740 1 -- start start 2024-04-21T08:46:44.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.124+0000 7f4a657f1740 1 Processor -- start 2024-04-21T08:46:44.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.125+0000 7f4a657f1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560463aa6710 0x560463aacc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:44.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.125+0000 7f4a657f1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aa7030 0x560463aad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:44.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.125+0000 7f4a657f1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560463aafc10 0x560463aad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:44.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.125+0000 7f4a657f1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560463ab29a0 con 0x560463aa7030 2024-04-21T08:46:44.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.125+0000 7f4a657f1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5604639c42f0 con 0x560463aafc10 2024-04-21T08:46:44.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.125+0000 7f4a657f1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560463ab2b20 con 0x560463aa6710 2024-04-21T08:46:44.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.125+0000 7f4a657f1740 10 client.0 _ll_register_callbacks cb 0x5604639d1580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:44.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.125+0000 7f4a5bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aa7030 0x560463aad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:44.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.125+0000 7f4a5bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aa7030 0x560463aad1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54604/0 (socket says 172.21.15.43:54604) 2024-04-21T08:46:44.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.125+0000 7f4a5bfff640 1 -- 172.21.15.43:0/3188129590 learned_addr learned my addr 172.21.15.43:0/3188129590 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:44.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.125+0000 7f4a6099d640 1 --2- 172.21.15.43:0/3188129590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560463aa6710 0x560463aacc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:44.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.125+0000 7f4a6119e640 1 --2- 172.21.15.43:0/3188129590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560463aafc10 0x560463aad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:44.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.125+0000 7f4a5bfff640 1 -- 172.21.15.43:0/3188129590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560463aa6710 msgr2=0x560463aacc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:44.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.125+0000 7f4a5bfff640 1 --2- 172.21.15.43:0/3188129590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560463aa6710 0x560463aacc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.125+0000 7f4a5bfff640 1 -- 172.21.15.43:0/3188129590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560463aafc10 msgr2=0x560463aad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:44.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.125+0000 7f4a5bfff640 1 --2- 172.21.15.43:0/3188129590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560463aafc10 0x560463aad6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.125+0000 7f4a5bfff640 1 -- 172.21.15.43:0/3188129590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56046371d3f0 con 0x560463aa7030 2024-04-21T08:46:44.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.125+0000 7f4a6099d640 1 --2- 172.21.15.43:0/3188129590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560463aa6710 0x560463aacc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:44.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.125+0000 7f4a5bfff640 1 --2- 172.21.15.43:0/3188129590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aa7030 0x560463aad1a0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f4a48035000 tx=0x7f4a4802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32b3982e8eff13 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:44.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.125+0000 7f4a3bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:44.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.126+0000 7f4a3bfff640 1 -- 172.21.15.43:0/3188129590 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a48052020 con 0x560463aa7030 2024-04-21T08:46:44.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.126+0000 7f4a3bfff640 1 -- 172.21.15.43:0/3188129590 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f4a480048c0 con 0x560463aa7030 2024-04-21T08:46:44.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.126+0000 7f4a657f1740 1 -- 172.21.15.43:0/3188129590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a540a0bc0 con 0x560463aa7030 2024-04-21T08:46:44.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.126+0000 7f4a3bfff640 1 -- 172.21.15.43:0/3188129590 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4a4805d050 con 0x560463aa7030 2024-04-21T08:46:44.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.126+0000 7f4a657f1740 10 client.19290 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:44.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.126+0000 7f4a657f1740 1 -- 172.21.15.43:0/3188129590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5604636dd0f0 con 0x560463aa7030 2024-04-21T08:46:44.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.126+0000 7f4a3bfff640 1 -- 172.21.15.43:0/3188129590 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a540a0bc0 con 0x560463aa7030 2024-04-21T08:46:44.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.126+0000 7f4a3bfff640 1 -- 172.21.15.43:0/3188129590 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4a48003b20 con 0x560463aa7030 2024-04-21T08:46:44.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.126+0000 7f4a657f1740 10 client.19290 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:44.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.126+0000 7f4a657f1740 10 client.19290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:44.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.126+0000 7f4a657f1740 20 client.19290 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:44.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.126+0000 7f4a657f1740 1 --2- 172.21.15.43:0/3188129590 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560463abfbc0 0x560463ac2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:44.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.126+0000 7f4a657f1740 4 client.19290 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:44.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.126+0000 7f4a657f1740 1 -- 172.21.15.43:0/3188129590 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560463a18c00 con 0x560463abfbc0 2024-04-21T08:46:44.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.127+0000 7f4a6099d640 1 --2- 172.21.15.43:0/3188129590 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560463abfbc0 0x560463ac2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:44.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.127+0000 7f4a3bfff640 10 client.19290 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:44.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.127+0000 7f4a6099d640 1 --2- 172.21.15.43:0/3188129590 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560463abfbc0 0x560463ac2060 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=397e5dc9c88638aa server_cookie=74b282bd94828339 in_seq=0 out_seq=0 2024-04-21T08:46:44.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.128+0000 7f4a3bfff640 1 -- 172.21.15.43:0/3188129590 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560463a18c00 con 0x560463abfbc0 2024-04-21T08:46:44.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.128+0000 7f4a3bfff640 10 client.19290 handle_command_reply: tid=0 2024-04-21T08:46:44.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.128+0000 7f4a657f1740 1 client.19290 shutdown 2024-04-21T08:46:44.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.128+0000 7f4a657f1740 1 -- 172.21.15.43:0/3188129590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aa7030 msgr2=0x560463aad1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:44.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.128+0000 7f4a657f1740 1 --2- 172.21.15.43:0/3188129590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aa7030 0x560463aad1a0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f4a48035000 tx=0x7f4a4802bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:44.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.128+0000 7f4a657f1740 1 -- 172.21.15.43:0/3188129590 shutdown_connections 2024-04-21T08:46:44.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.128+0000 7f4a657f1740 1 --2- 172.21.15.43:0/3188129590 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560463abfbc0 0x560463ac2060 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.128+0000 7f4a657f1740 1 --2- 172.21.15.43:0/3188129590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560463aafc10 0x560463aad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.128+0000 7f4a657f1740 1 --2- 172.21.15.43:0/3188129590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aa7030 0x560463aad1a0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.128+0000 7f4a3bfff640 0 client.19290 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:44.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.128+0000 7f4a657f1740 1 --2- 172.21.15.43:0/3188129590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560463aa6710 0x560463aacc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.128+0000 7f4a657f1740 1 -- 172.21.15.43:0/3188129590 >> 172.21.15.43:0/3188129590 conn(0x560463aa0e40 msgr2=0x560463aa0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:44.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.128+0000 7f4a657f1740 1 -- 172.21.15.43:0/3188129590 shutdown_connections 2024-04-21T08:46:44.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.128+0000 7f4a657f1740 1 -- 172.21.15.43:0/3188129590 wait complete. 2024-04-21T08:46:44.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.128+0000 7f4a657f1740 20 client.19290 trim_cache size 0 max 16384 2024-04-21T08:46:44.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.146+0000 7f4a657f1740 1 Processor -- start 2024-04-21T08:46:44.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.146+0000 7f4a657f1740 1 -- start start 2024-04-21T08:46:44.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.146+0000 7f4a657f1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aa0e40 0x560463abb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:44.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.146+0000 7f4a657f1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560463aa6710 0x560463aa2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:44.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.146+0000 7f4a657f1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560463aa7030 0x560463ac7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:44.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.146+0000 7f4a657f1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c1b1530 con 0x560463aa0e40 2024-04-21T08:46:44.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.146+0000 7f4a657f1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c1b3070 con 0x560463aa6710 2024-04-21T08:46:44.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.146+0000 7f4a657f1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c1b1c00 con 0x560463aa7030 2024-04-21T08:46:44.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.146+0000 7f4a6099d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aa0e40 0x560463abb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:44.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.146+0000 7f4a6099d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aa0e40 0x560463abb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54614/0 (socket says 172.21.15.43:54614) 2024-04-21T08:46:44.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.146+0000 7f4a6099d640 1 -- 172.21.15.43:0/2113564836 learned_addr learned my addr 172.21.15.43:0/2113564836 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:44.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.146+0000 7f4a5bfff640 1 --2- 172.21.15.43:0/2113564836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560463aa6710 0x560463aa2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:44.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.146+0000 7f4a6099d640 1 -- 172.21.15.43:0/2113564836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560463aa7030 msgr2=0x560463ac7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:44.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.146+0000 7f4a6119e640 1 --2- 172.21.15.43:0/2113564836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560463aa7030 0x560463ac7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:44.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.146+0000 7f4a6099d640 1 --2- 172.21.15.43:0/2113564836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560463aa7030 0x560463ac7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.146+0000 7f4a6099d640 1 -- 172.21.15.43:0/2113564836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560463aa6710 msgr2=0x560463aa2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:44.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.146+0000 7f4a6099d640 1 --2- 172.21.15.43:0/2113564836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560463aa6710 0x560463aa2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.146+0000 7f4a6099d640 1 -- 172.21.15.43:0/2113564836 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560463aa6e60 con 0x560463aa0e40 2024-04-21T08:46:44.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.146+0000 7f4a6119e640 1 --2- 172.21.15.43:0/2113564836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560463aa7030 0x560463ac7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:44.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.146+0000 7f4a5bfff640 1 --2- 172.21.15.43:0/2113564836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560463aa6710 0x560463aa2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:44.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.147+0000 7f4a6099d640 1 --2- 172.21.15.43:0/2113564836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aa0e40 0x560463abb860 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x560463ac28d0 tx=0x7f4a5000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f0dd4c80dedc58 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:44.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.147+0000 7f4a467fc640 1 -- 172.21.15.43:0/2113564836 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50004220 con 0x560463aa0e40 2024-04-21T08:46:44.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.147+0000 7f4a467fc640 1 -- 172.21.15.43:0/2113564836 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a50023070 con 0x560463aa0e40 2024-04-21T08:46:44.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.147+0000 7f4a467fc640 1 -- 172.21.15.43:0/2113564836 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a500047a0 con 0x560463aa0e40 2024-04-21T08:46:44.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.147+0000 7f4a657f1740 1 -- 172.21.15.43:0/2113564836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aa0e40 msgr2=0x560463abb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:44.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.147+0000 7f4a657f1740 1 --2- 172.21.15.43:0/2113564836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aa0e40 0x560463abb860 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x560463ac28d0 tx=0x7f4a5000b410 comp rx=0 tx=0).stop 2024-04-21T08:46:44.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.147+0000 7f4a657f1740 1 -- 172.21.15.43:0/2113564836 shutdown_connections 2024-04-21T08:46:44.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.147+0000 7f4a657f1740 1 --2- 172.21.15.43:0/2113564836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560463aa7030 0x560463ac7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.147+0000 7f4a657f1740 1 --2- 172.21.15.43:0/2113564836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560463aa6710 0x560463aa2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.147+0000 7f4a657f1740 1 --2- 172.21.15.43:0/2113564836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aa0e40 0x560463abb860 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.147+0000 7f4a657f1740 1 -- 172.21.15.43:0/2113564836 >> 172.21.15.43:0/2113564836 conn(0x560463aab8a0 msgr2=0x560463aa2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:44.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.147+0000 7f4a657f1740 1 -- 172.21.15.43:0/2113564836 shutdown_connections 2024-04-21T08:46:44.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.147+0000 7f4a657f1740 1 -- 172.21.15.43:0/2113564836 wait complete. 2024-04-21T08:46:44.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.148+0000 7f4a657f1740 1 -- start start 2024-04-21T08:46:44.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.148+0000 7f4a657f1740 1 Processor -- start 2024-04-21T08:46:44.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.148+0000 7f4a657f1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560463aa0e40 0x560463ab7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:44.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.148+0000 7f4a657f1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aa6710 0x560463ab4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:44.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.148+0000 7f4a657f1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560463aa7030 0x560463ab5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:44.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.148+0000 7f4a657f1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c1b29a0 con 0x560463aa6710 2024-04-21T08:46:44.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.148+0000 7f4a657f1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c1b22d0 con 0x560463aa7030 2024-04-21T08:46:44.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.148+0000 7f4a657f1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c1af510 con 0x560463aa0e40 2024-04-21T08:46:44.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.148+0000 7f4a657f1740 10 client.0 _ll_register_callbacks cb 0x7f4a48046040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:44.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.148+0000 7f4a5bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aa6710 0x560463ab4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:44.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.148+0000 7f4a5bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aa6710 0x560463ab4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54630/0 (socket says 172.21.15.43:54630) 2024-04-21T08:46:44.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.148+0000 7f4a5bfff640 1 -- 172.21.15.43:0/660132303 learned_addr learned my addr 172.21.15.43:0/660132303 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:44.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.148+0000 7f4a6099d640 1 --2- 172.21.15.43:0/660132303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560463aa0e40 0x560463ab7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:44.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.148+0000 7f4a6119e640 1 --2- 172.21.15.43:0/660132303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560463aa7030 0x560463ab5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:44.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.148+0000 7f4a5bfff640 1 -- 172.21.15.43:0/660132303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560463aa0e40 msgr2=0x560463ab7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:44.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.148+0000 7f4a5bfff640 1 --2- 172.21.15.43:0/660132303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560463aa0e40 0x560463ab7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.148+0000 7f4a5bfff640 1 -- 172.21.15.43:0/660132303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560463aa7030 msgr2=0x560463ab5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:44.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.148+0000 7f4a5bfff640 1 --2- 172.21.15.43:0/660132303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560463aa7030 0x560463ab5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.148+0000 7f4a5bfff640 1 -- 172.21.15.43:0/660132303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560463ab57c0 con 0x560463aa6710 2024-04-21T08:46:44.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.149+0000 7f4a6099d640 1 --2- 172.21.15.43:0/660132303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560463aa0e40 0x560463ab7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:44.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.149+0000 7f4a6119e640 1 --2- 172.21.15.43:0/660132303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560463aa7030 0x560463ab5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:44.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.149+0000 7f4a5bfff640 1 --2- 172.21.15.43:0/660132303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aa6710 0x560463ab4d40 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x560463aae230 tx=0x7f4a48043700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5292dcef05838562 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:44.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.149+0000 7f4a3bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:44.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.149+0000 7f4a3bfff640 1 -- 172.21.15.43:0/660132303 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a48052040 con 0x560463aa6710 2024-04-21T08:46:44.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.149+0000 7f4a3bfff640 1 -- 172.21.15.43:0/660132303 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f4a48003a00 con 0x560463aa6710 2024-04-21T08:46:44.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.149+0000 7f4a657f1740 1 -- 172.21.15.43:0/660132303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a50023070 con 0x560463aa6710 2024-04-21T08:46:44.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.149+0000 7f4a3bfff640 1 -- 172.21.15.43:0/660132303 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4a4806f050 con 0x560463aa6710 2024-04-21T08:46:44.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.149+0000 7f4a657f1740 10 client.19296 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:44.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.149+0000 7f4a657f1740 1 -- 172.21.15.43:0/660132303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560463b26e40 con 0x560463aa6710 2024-04-21T08:46:44.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.149+0000 7f4a3bfff640 1 -- 172.21.15.43:0/660132303 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a50023070 con 0x560463aa6710 2024-04-21T08:46:44.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.149+0000 7f4a3bfff640 1 -- 172.21.15.43:0/660132303 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4a480046c0 con 0x560463aa6710 2024-04-21T08:46:44.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.149+0000 7f4a657f1740 10 client.19296 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:44.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.149+0000 7f4a657f1740 10 client.19296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:44.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.149+0000 7f4a657f1740 20 client.19296 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:44.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.149+0000 7f4a657f1740 1 --2- 172.21.15.43:0/660132303 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560463aafc10 0x560463b295c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:44.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.149+0000 7f4a657f1740 4 client.19296 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:46:44.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.149+0000 7f4a657f1740 1 -- 172.21.15.43:0/660132303 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560463a12a70 con 0x560463aafc10 2024-04-21T08:46:44.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.150+0000 7f4a6099d640 1 --2- 172.21.15.43:0/660132303 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560463aafc10 0x560463b295c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:44.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.150+0000 7f4a6099d640 1 --2- 172.21.15.43:0/660132303 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560463aafc10 0x560463b295c0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb36beab016861d6 server_cookie=ea972b962b041e20 in_seq=0 out_seq=0 2024-04-21T08:46:44.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.150+0000 7f4a3bfff640 10 client.19296 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:44.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.150+0000 7f4a3bfff640 1 -- 172.21.15.43:0/660132303 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560463a12a70 con 0x560463aafc10 2024-04-21T08:46:44.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.150+0000 7f4a3bfff640 10 client.19296 handle_command_reply: tid=0 2024-04-21T08:46:44.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.150+0000 7f4a657f1740 1 client.19296 shutdown 2024-04-21T08:46:44.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.150+0000 7f4a657f1740 1 -- 172.21.15.43:0/660132303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aa6710 msgr2=0x560463ab4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:44.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.150+0000 7f4a657f1740 1 --2- 172.21.15.43:0/660132303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aa6710 0x560463ab4d40 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x560463aae230 tx=0x7f4a48043700 comp rx=0 tx=0).stop 2024-04-21T08:46:44.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.150+0000 7f4a657f1740 1 -- 172.21.15.43:0/660132303 shutdown_connections 2024-04-21T08:46:44.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.150+0000 7f4a657f1740 1 --2- 172.21.15.43:0/660132303 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560463aafc10 0x560463b295c0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.151+0000 7f4a657f1740 1 --2- 172.21.15.43:0/660132303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560463aa7030 0x560463ab5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.151+0000 7f4a657f1740 1 --2- 172.21.15.43:0/660132303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560463aa6710 0x560463ab4d40 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.151+0000 7f4a3bfff640 0 client.19296 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:44.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.151+0000 7f4a657f1740 1 --2- 172.21.15.43:0/660132303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560463aa0e40 0x560463ab7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.151+0000 7f4a657f1740 1 -- 172.21.15.43:0/660132303 >> 172.21.15.43:0/660132303 conn(0x560463aab8a0 msgr2=0x560463aa2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:44.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.151+0000 7f4a657f1740 1 -- 172.21.15.43:0/660132303 shutdown_connections 2024-04-21T08:46:44.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.151+0000 7f4a657f1740 1 -- 172.21.15.43:0/660132303 wait complete. 2024-04-21T08:46:44.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.151+0000 7f4a657f1740 20 client.19296 trim_cache size 0 max 16384 2024-04-21T08:46:44.158 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:46:44.158 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:46:44.158 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-21T08:46:44.158 INFO:teuthology.orchestra.run.smithi043.stdout: "47b37bbf-792b-43aa-bdf2-a7eacad516c7": { 2024-04-21T08:46:44.158 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-21T08:46:44.158 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "47b37bbf-792b-43aa-bdf2-a7eacad516c7", 2024-04-21T08:46:44.158 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-21T08:46:44.159 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:46:44.159 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:46:44.159 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:46:44.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.151+0000 7f4a6199f640 1 -- 172.21.15.43:0/4201775135 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4a4c0a3dd0 msgr2=0x7f4a4c0a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:44.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.151+0000 7f4a6199f640 1 --2- 172.21.15.43:0/4201775135 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4a4c0a3dd0 0x7f4a4c0a6290 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4a48006010 tx=0x7f4a48004390 comp rx=0 tx=0).stop 2024-04-21T08:46:44.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.151+0000 7f4a6199f640 1 -- 172.21.15.43:0/4201775135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a5c101eb0 msgr2=0x7f4a5c1a6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:44.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.151+0000 7f4a6199f640 1 --2- 172.21.15.43:0/4201775135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a5c101eb0 0x7f4a5c1a6730 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400eaa0 tx=0x7f4a5400ef60 comp rx=0 tx=0).stop 2024-04-21T08:46:44.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.151+0000 7f4a6199f640 1 -- 172.21.15.43:0/4201775135 shutdown_connections 2024-04-21T08:46:44.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.151+0000 7f4a6199f640 1 --2- 172.21.15.43:0/4201775135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a5c101eb0 0x7f4a5c1a6730 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.151+0000 7f4a6199f640 1 --2- 172.21.15.43:0/4201775135 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f4a4c0a3dd0 0x7f4a4c0a6290 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.151+0000 7f4a6199f640 1 --2- 172.21.15.43:0/4201775135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a5c107080 0x7f4a5c101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.151+0000 7f4a6199f640 1 --2- 172.21.15.43:0/4201775135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a5c1004c0 0x7f4a5c101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.151+0000 7f4a6199f640 1 -- 172.21.15.43:0/4201775135 >> 172.21.15.43:0/4201775135 conn(0x7f4a5c077170 msgr2=0x7f4a5c105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:44.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.151+0000 7f4a6199f640 1 -- 172.21.15.43:0/4201775135 shutdown_connections 2024-04-21T08:46:44.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:46:44.151+0000 7f4a6199f640 1 -- 172.21.15.43:0/4201775135 wait complete. 2024-04-21T08:46:44.162 INFO:tasks.cephfs.filesystem:scrub status for tag:47b37bbf-792b-43aa-bdf2-a7eacad516c7 - {'path': '/', 'tag': '47b37bbf-792b-43aa-bdf2-a7eacad516c7', 'options': 'recursive,force'} 2024-04-21T08:46:44.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:44 smithi043 ceph-mon[27978]: pgmap v240: 129 pgs: 129 active+clean; 2.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-21T08:46:44.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:44 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:46:44.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:44 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:46:44.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:44 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:46:44.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:44 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:46:44.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:44 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:46:44.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:44 smithi175 ceph-mon[31920]: pgmap v240: 129 pgs: 129 active+clean; 2.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-21T08:46:44.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:44 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:46:44.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:44 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:46:44.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:44 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:46:44.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:44 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:46:44.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:44 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:46:44.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:44 smithi112 ceph-mon[31085]: pgmap v240: 129 pgs: 129 active+clean; 2.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-21T08:46:44.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:44 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:46:44.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:44 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:46:44.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:44 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:46:44.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:44 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:46:44.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:44 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:46:46.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:46 smithi112 ceph-mon[31085]: pgmap v241: 129 pgs: 129 active+clean; 2.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-21T08:46:46.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:46 smithi043 ceph-mon[27978]: pgmap v241: 129 pgs: 129 active+clean; 2.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-21T08:46:46.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:46 smithi175 ceph-mon[31920]: pgmap v241: 129 pgs: 129 active+clean; 2.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-21T08:46:48.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:48 smithi175 ceph-mon[31920]: pgmap v242: 129 pgs: 129 active+clean; 3.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:46:48.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:48 smithi112 ceph-mon[31085]: pgmap v242: 129 pgs: 129 active+clean; 3.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:46:48.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:48 smithi043 ceph-mon[27978]: pgmap v242: 129 pgs: 129 active+clean; 3.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:46:50.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:50 smithi175 ceph-mon[31920]: pgmap v243: 129 pgs: 129 active+clean; 3.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:46:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:50 smithi112 ceph-mon[31085]: pgmap v243: 129 pgs: 129 active+clean; 3.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:46:50.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:50 smithi043 ceph-mon[27978]: pgmap v243: 129 pgs: 129 active+clean; 3.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:46:52.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:52 smithi175 ceph-mon[31920]: pgmap v244: 129 pgs: 129 active+clean; 3.3 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:46:52.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:52 smithi112 ceph-mon[31085]: pgmap v244: 129 pgs: 129 active+clean; 3.3 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:46:52.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:52 smithi043 ceph-mon[27978]: pgmap v244: 129 pgs: 129 active+clean; 3.3 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:46:54.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:54 smithi175 ceph-mon[31920]: pgmap v245: 129 pgs: 129 active+clean; 3.3 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-04-21T08:46:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:54 smithi112 ceph-mon[31085]: pgmap v245: 129 pgs: 129 active+clean; 3.3 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-04-21T08:46:54.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:54 smithi043 ceph-mon[27978]: pgmap v245: 129 pgs: 129 active+clean; 3.3 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-04-21T08:46:56.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:56 smithi175 ceph-mon[31920]: pgmap v246: 129 pgs: 129 active+clean; 3.3 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-04-21T08:46:56.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:56 smithi112 ceph-mon[31085]: pgmap v246: 129 pgs: 129 active+clean; 3.3 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-04-21T08:46:56.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:56 smithi043 ceph-mon[27978]: pgmap v246: 129 pgs: 129 active+clean; 3.3 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-04-21T08:46:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.735+0000 7f11f4081640 1 -- 172.21.15.43:0/3654576871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11ec1010e0 msgr2=0x7f11ec1014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.735+0000 7f11f4081640 1 --2- 172.21.15.43:0/3654576871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11ec1010e0 0x7f11ec1014c0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009900 tx=0x7f11e802bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.735+0000 7f11f4081640 1 -- 172.21.15.43:0/3654576871 shutdown_connections 2024-04-21T08:46:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.735+0000 7f11f4081640 1 --2- 172.21.15.43:0/3654576871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11ec1061b0 0x7f11ec10dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.735+0000 7f11f4081640 1 --2- 172.21.15.43:0/3654576871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11ec101a90 0x7f11ec105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.735+0000 7f11f4081640 1 --2- 172.21.15.43:0/3654576871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11ec1010e0 0x7f11ec1014c0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.735+0000 7f11f4081640 1 -- 172.21.15.43:0/3654576871 >> 172.21.15.43:0/3654576871 conn(0x7f11ec0771d0 msgr2=0x7f11ec0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.735+0000 7f11f4081640 1 -- 172.21.15.43:0/3654576871 shutdown_connections 2024-04-21T08:46:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.735+0000 7f11f4081640 1 -- 172.21.15.43:0/3654576871 wait complete. 2024-04-21T08:46:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.735+0000 7f11f4081640 1 Processor -- start 2024-04-21T08:46:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.735+0000 7f11f4081640 1 -- start start 2024-04-21T08:46:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.736+0000 7f11f4081640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11ec1010e0 0x7f11ec1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.736+0000 7f11f4081640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11ec1061b0 0x7f11ec198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.736+0000 7f11f4081640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11ec19cf50 0x7f11ec19d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.736+0000 7f11f4081640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec112220 con 0x7f11ec1061b0 2024-04-21T08:46:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.736+0000 7f11f4081640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec1120a0 con 0x7f11ec19cf50 2024-04-21T08:46:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.736+0000 7f11f4081640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec1123a0 con 0x7f11ec1010e0 2024-04-21T08:46:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.736+0000 7f11f161c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11ec1061b0 0x7f11ec198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.736+0000 7f11f161c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11ec1061b0 0x7f11ec198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51442/0 (socket says 172.21.15.43:51442) 2024-04-21T08:46:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.736+0000 7f11f161c640 1 -- 172.21.15.43:0/1597102641 learned_addr learned my addr 172.21.15.43:0/1597102641 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.736+0000 7f11f261e640 1 --2- 172.21.15.43:0/1597102641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11ec19cf50 0x7f11ec19d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.736+0000 7f11f1e1d640 1 --2- 172.21.15.43:0/1597102641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11ec1010e0 0x7f11ec1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.736+0000 7f11f161c640 1 -- 172.21.15.43:0/1597102641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11ec1010e0 msgr2=0x7f11ec1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.736+0000 7f11f161c640 1 --2- 172.21.15.43:0/1597102641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11ec1010e0 0x7f11ec1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.736+0000 7f11f161c640 1 -- 172.21.15.43:0/1597102641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11ec19cf50 msgr2=0x7f11ec19d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.736+0000 7f11f161c640 1 --2- 172.21.15.43:0/1597102641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11ec19cf50 0x7f11ec19d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.736+0000 7f11f161c640 1 -- 172.21.15.43:0/1597102641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ec19da40 con 0x7f11ec1061b0 2024-04-21T08:46:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.736+0000 7f11f161c640 1 --2- 172.21.15.43:0/1597102641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11ec1061b0 0x7f11ec198c20 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00d770 tx=0x7f11dc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.737+0000 7f11e2ffd640 1 -- 172.21.15.43:0/1597102641 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc017070 con 0x7f11ec1061b0 2024-04-21T08:46:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.737+0000 7f11e2ffd640 1 -- 172.21.15.43:0/1597102641 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11dc009c30 con 0x7f11ec1061b0 2024-04-21T08:46:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.737+0000 7f11f4081640 1 -- 172.21.15.43:0/1597102641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11ec19dd30 con 0x7f11ec1061b0 2024-04-21T08:46:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.737+0000 7f11e2ffd640 1 -- 172.21.15.43:0/1597102641 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc00f620 con 0x7f11ec1061b0 2024-04-21T08:46:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.737+0000 7f11f4081640 1 -- 172.21.15.43:0/1597102641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11ec1a5570 con 0x7f11ec1061b0 2024-04-21T08:46:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.739+0000 7f11e2ffd640 1 -- 172.21.15.43:0/1597102641 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f11dc01a070 con 0x7f11ec1061b0 2024-04-21T08:46:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.739+0000 7f11f6c71740 1 Processor -- start 2024-04-21T08:46:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.740+0000 7f11f6c71740 1 -- start start 2024-04-21T08:46:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.740+0000 7f11f6c71740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bca78d0 0x560a6bca6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.740+0000 7f11f6c71740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a6bca6b60 0x560a6bca6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.740+0000 7f11f6c71740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6bcb00e0 0x560a6bcb24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.740+0000 7f11f6c71740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560a6bc0b8d0 con 0x560a6bca78d0 2024-04-21T08:46:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.740+0000 7f11f6c71740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560a6bba8e80 con 0x560a6bca6b60 2024-04-21T08:46:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.740+0000 7f11f6c71740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560a6bc07890 con 0x560a6bcb00e0 2024-04-21T08:46:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.740+0000 7f11f261e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bca78d0 0x560a6bca6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.740+0000 7f11e2ffd640 1 --2- 172.21.15.43:0/1597102641 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f11d80a3e40 0x7f11d80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.740+0000 7f11f261e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bca78d0 0x560a6bca6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51448/0 (socket says 172.21.15.43:51448) 2024-04-21T08:46:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.740+0000 7f11f261e640 1 -- 172.21.15.43:0/3463906257 learned_addr learned my addr 172.21.15.43:0/3463906257 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.740+0000 7f11f1e1d640 1 --2- 172.21.15.43:0/3463906257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a6bca6b60 0x560a6bca6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.740+0000 7f11e2ffd640 1 -- 172.21.15.43:0/1597102641 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f11dc0d0610 con 0x7f11ec1061b0 2024-04-21T08:46:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.740+0000 7f11f261e640 1 --2- 172.21.15.43:0/1597102641 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f11d80a3e40 0x7f11d80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.740+0000 7f11f161c640 1 --2- 172.21.15.43:0/3463906257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6bcb00e0 0x560a6bcb24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.740+0000 7f11f1e1d640 1 -- 172.21.15.43:0/3463906257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6bcb00e0 msgr2=0x560a6bcb24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.740+0000 7f11f1e1d640 1 --2- 172.21.15.43:0/3463906257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6bcb00e0 0x560a6bcb24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.740+0000 7f11f1e1d640 1 -- 172.21.15.43:0/3463906257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bca78d0 msgr2=0x560a6bca6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.740+0000 7f11f1e1d640 1 --2- 172.21.15.43:0/3463906257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bca78d0 0x560a6bca6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.740+0000 7f11f1e1d640 1 -- 172.21.15.43:0/3463906257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a6bbca320 con 0x560a6bca6b60 2024-04-21T08:46:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.741+0000 7f11f1e1d640 1 --2- 172.21.15.43:0/3463906257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a6bca6b60 0x560a6bca6fe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f11e8000c00 tx=0x7f11e802c610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=736ea4d3fc55dfee server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.741+0000 7f11f261e640 1 --2- 172.21.15.43:0/1597102641 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f11d80a3e40 0x7f11d80a6300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f11e40049d0 tx=0x7f11e4004ab0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.741+0000 7f11d7fff640 1 -- 172.21.15.43:0/3463906257 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8045020 con 0x560a6bca6b60 2024-04-21T08:46:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.741+0000 7f11d7fff640 1 -- 172.21.15.43:0/3463906257 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11e802c7e0 con 0x560a6bca6b60 2024-04-21T08:46:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.741+0000 7f11d7fff640 1 -- 172.21.15.43:0/3463906257 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e80046c0 con 0x560a6bca6b60 2024-04-21T08:46:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.742+0000 7f11f6c71740 1 -- 172.21.15.43:0/3463906257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a6bca6b60 msgr2=0x560a6bca6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.742+0000 7f11f6c71740 1 --2- 172.21.15.43:0/3463906257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a6bca6b60 0x560a6bca6fe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f11e8000c00 tx=0x7f11e802c610 comp rx=0 tx=0).stop 2024-04-21T08:46:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.742+0000 7f11f6c71740 1 -- 172.21.15.43:0/3463906257 shutdown_connections 2024-04-21T08:46:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.742+0000 7f11f6c71740 1 --2- 172.21.15.43:0/3463906257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6bcb00e0 0x560a6bcb24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.742+0000 7f11f6c71740 1 --2- 172.21.15.43:0/3463906257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a6bca6b60 0x560a6bca6fe0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.742+0000 7f11f6c71740 1 --2- 172.21.15.43:0/3463906257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bca78d0 0x560a6bca6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.742+0000 7f11f6c71740 1 -- 172.21.15.43:0/3463906257 >> 172.21.15.43:0/3463906257 conn(0x560a6bc6dc90 msgr2=0x560a6bca1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.742+0000 7f11f6c71740 1 -- 172.21.15.43:0/3463906257 shutdown_connections 2024-04-21T08:46:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.742+0000 7f11f6c71740 1 -- 172.21.15.43:0/3463906257 wait complete. 2024-04-21T08:46:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.742+0000 7f11f6c71740 1 -- start start 2024-04-21T08:46:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.742+0000 7f11f6c71740 1 Processor -- start 2024-04-21T08:46:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.743+0000 7f11f6c71740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bca6b60 0x560a6bcafaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.743+0000 7f11f6c71740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6bca78d0 0x560a6bcace30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.743+0000 7f11f6c71740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a6bcb00e0 0x560a6bcad440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.743+0000 7f11f6c71740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560a6bcb2db0 con 0x560a6bca6b60 2024-04-21T08:46:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.743+0000 7f11f6c71740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560a6bb93eb0 con 0x560a6bcb00e0 2024-04-21T08:46:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.743+0000 7f11f6c71740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560a6bcb2f30 con 0x560a6bca78d0 2024-04-21T08:46:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.743+0000 7f11f6c71740 10 client.0 _ll_register_callbacks cb 0x560a6bbc0b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.743+0000 7f11f1e1d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bca6b60 0x560a6bcafaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.743+0000 7f11f1e1d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bca6b60 0x560a6bcafaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51456/0 (socket says 172.21.15.43:51456) 2024-04-21T08:46:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.743+0000 7f11f1e1d640 1 -- 172.21.15.43:0/2006779558 learned_addr learned my addr 172.21.15.43:0/2006779558 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.743+0000 7f11f261e640 1 --2- 172.21.15.43:0/2006779558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a6bcb00e0 0x560a6bcad440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.743+0000 7f11f161c640 1 --2- 172.21.15.43:0/2006779558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6bca78d0 0x560a6bcace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.743+0000 7f11f1e1d640 1 -- 172.21.15.43:0/2006779558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6bca78d0 msgr2=0x560a6bcace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.743+0000 7f11f1e1d640 1 --2- 172.21.15.43:0/2006779558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6bca78d0 0x560a6bcace30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.743+0000 7f11f1e1d640 1 -- 172.21.15.43:0/2006779558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a6bcb00e0 msgr2=0x560a6bcad440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.743+0000 7f11f1e1d640 1 --2- 172.21.15.43:0/2006779558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a6bcb00e0 0x560a6bcad440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.743+0000 7f11f1e1d640 1 -- 172.21.15.43:0/2006779558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a6b91e210 con 0x560a6bca6b60 2024-04-21T08:46:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.743+0000 7f11f1e1d640 1 --2- 172.21.15.43:0/2006779558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bca6b60 0x560a6bcafaa0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f11e8036040 tx=0x7f11e802cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada9b5638ae31896 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.743+0000 7f11d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.744+0000 7f11d57fa640 1 -- 172.21.15.43:0/2006779558 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8045070 con 0x560a6bca6b60 2024-04-21T08:46:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.744+0000 7f11d57fa640 1 -- 172.21.15.43:0/2006779558 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f11e80035f0 con 0x560a6bca6b60 2024-04-21T08:46:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.744+0000 7f11f6c71740 1 -- 172.21.15.43:0/2006779558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11e802c7e0 con 0x560a6bca6b60 2024-04-21T08:46:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.744+0000 7f11d57fa640 1 -- 172.21.15.43:0/2006779558 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f11e8059050 con 0x560a6bca6b60 2024-04-21T08:46:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.744+0000 7f11f6c71740 10 client.19308 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.744+0000 7f11f6c71740 1 -- 172.21.15.43:0/2006779558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560a6b9ea620 con 0x560a6bca6b60 2024-04-21T08:46:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.744+0000 7f11d57fa640 1 -- 172.21.15.43:0/2006779558 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11e802c7e0 con 0x560a6bca6b60 2024-04-21T08:46:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.744+0000 7f11d57fa640 1 -- 172.21.15.43:0/2006779558 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f11e8041920 con 0x560a6bca6b60 2024-04-21T08:46:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.744+0000 7f11f6c71740 10 client.19308 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.744+0000 7f11f6c71740 10 client.19308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.744+0000 7f11f6c71740 20 client.19308 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.744+0000 7f11f6c71740 1 --2- 172.21.15.43:0/2006779558 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560a6bcbfeb0 0x560a6bcc22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.744+0000 7f11f6c71740 4 client.19308 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.744+0000 7f11f6c71740 1 -- 172.21.15.43:0/2006779558 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a6bc1f880 con 0x560a6bcbfeb0 2024-04-21T08:46:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.745+0000 7f11f161c640 1 --2- 172.21.15.43:0/2006779558 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560a6bcbfeb0 0x560a6bcc22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.745+0000 7f11f161c640 1 --2- 172.21.15.43:0/2006779558 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560a6bcbfeb0 0x560a6bcc22a0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf4690e7868fbe85 server_cookie=98daf269b958b8fa in_seq=0 out_seq=0 2024-04-21T08:46:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.745+0000 7f11d57fa640 10 client.19308 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.746+0000 7f11d57fa640 1 -- 172.21.15.43:0/2006779558 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a6bc1f880 con 0x560a6bcbfeb0 2024-04-21T08:46:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.746+0000 7f11d57fa640 10 client.19308 handle_command_reply: tid=0 2024-04-21T08:46:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.746+0000 7f11f6c71740 1 client.19308 shutdown 2024-04-21T08:46:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.746+0000 7f11f6c71740 1 -- 172.21.15.43:0/2006779558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bca6b60 msgr2=0x560a6bcafaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.746+0000 7f11f6c71740 1 --2- 172.21.15.43:0/2006779558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bca6b60 0x560a6bcafaa0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f11e8036040 tx=0x7f11e802cc40 comp rx=0 tx=0).stop 2024-04-21T08:46:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.746+0000 7f11f6c71740 1 -- 172.21.15.43:0/2006779558 shutdown_connections 2024-04-21T08:46:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.746+0000 7f11f6c71740 1 --2- 172.21.15.43:0/2006779558 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560a6bcbfeb0 0x560a6bcc22a0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.746+0000 7f11f6c71740 1 --2- 172.21.15.43:0/2006779558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a6bcb00e0 0x560a6bcad440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.746+0000 7f11d57fa640 0 client.19308 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.746+0000 7f11f6c71740 1 --2- 172.21.15.43:0/2006779558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6bca78d0 0x560a6bcace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.746+0000 7f11f6c71740 1 --2- 172.21.15.43:0/2006779558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bca6b60 0x560a6bcafaa0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.746+0000 7f11f6c71740 1 -- 172.21.15.43:0/2006779558 >> 172.21.15.43:0/2006779558 conn(0x560a6bc6dc90 msgr2=0x560a6bca35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.746+0000 7f11f6c71740 1 -- 172.21.15.43:0/2006779558 shutdown_connections 2024-04-21T08:46:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.746+0000 7f11f6c71740 1 -- 172.21.15.43:0/2006779558 wait complete. 2024-04-21T08:46:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.746+0000 7f11f6c71740 20 client.19308 trim_cache size 0 max 16384 2024-04-21T08:46:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.761+0000 7f11f6c71740 1 Processor -- start 2024-04-21T08:46:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.761+0000 7f11f6c71740 1 -- start start 2024-04-21T08:46:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.761+0000 7f11f6c71740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a6bca6b60 0x560a6bcac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.761+0000 7f11f6c71740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6bca78d0 0x560a6bca3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.761+0000 7f11f6c71740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bcb00e0 0x560a6bcaf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.761+0000 7f11f6c71740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec1af270 con 0x560a6bcb00e0 2024-04-21T08:46:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.761+0000 7f11f6c71740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec1b0db0 con 0x560a6bca6b60 2024-04-21T08:46:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.761+0000 7f11f6c71740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec1af940 con 0x560a6bca78d0 2024-04-21T08:46:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.762+0000 7f11f261e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bcb00e0 0x560a6bcaf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.762+0000 7f11f261e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bcb00e0 0x560a6bcaf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51460/0 (socket says 172.21.15.43:51460) 2024-04-21T08:46:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.762+0000 7f11f261e640 1 -- 172.21.15.43:0/4264442382 learned_addr learned my addr 172.21.15.43:0/4264442382 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.762+0000 7f11f161c640 1 --2- 172.21.15.43:0/4264442382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6bca78d0 0x560a6bca3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.762+0000 7f11f1e1d640 1 --2- 172.21.15.43:0/4264442382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a6bca6b60 0x560a6bcac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.762+0000 7f11f261e640 1 -- 172.21.15.43:0/4264442382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6bca78d0 msgr2=0x560a6bca3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.762+0000 7f11f261e640 1 --2- 172.21.15.43:0/4264442382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6bca78d0 0x560a6bca3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.762+0000 7f11f261e640 1 -- 172.21.15.43:0/4264442382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a6bca6b60 msgr2=0x560a6bcac5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.762+0000 7f11f261e640 1 --2- 172.21.15.43:0/4264442382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a6bca6b60 0x560a6bcac5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.762+0000 7f11f261e640 1 -- 172.21.15.43:0/4264442382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a6bca1a60 con 0x560a6bcb00e0 2024-04-21T08:46:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.762+0000 7f11f261e640 1 --2- 172.21.15.43:0/4264442382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bcb00e0 0x560a6bcaf560 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f11e40134b0 tx=0x7f11e4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9331384b3c0b4ada server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.762+0000 7f11d77fe640 1 -- 172.21.15.43:0/4264442382 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e4004d20 con 0x560a6bcb00e0 2024-04-21T08:46:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.762+0000 7f11d77fe640 1 -- 172.21.15.43:0/4264442382 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11e400fe40 con 0x560a6bcb00e0 2024-04-21T08:46:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.762+0000 7f11d77fe640 1 -- 172.21.15.43:0/4264442382 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e4005720 con 0x560a6bcb00e0 2024-04-21T08:46:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.763+0000 7f11f6c71740 1 -- 172.21.15.43:0/4264442382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bcb00e0 msgr2=0x560a6bcaf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.763+0000 7f11f6c71740 1 --2- 172.21.15.43:0/4264442382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bcb00e0 0x560a6bcaf560 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f11e40134b0 tx=0x7f11e4005bd0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.763+0000 7f11f6c71740 1 -- 172.21.15.43:0/4264442382 shutdown_connections 2024-04-21T08:46:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.763+0000 7f11f6c71740 1 --2- 172.21.15.43:0/4264442382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bcb00e0 0x560a6bcaf560 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.763+0000 7f11f6c71740 1 --2- 172.21.15.43:0/4264442382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6bca78d0 0x560a6bca3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.763+0000 7f11f6c71740 1 --2- 172.21.15.43:0/4264442382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a6bca6b60 0x560a6bcac5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.763+0000 7f11f6c71740 1 -- 172.21.15.43:0/4264442382 >> 172.21.15.43:0/4264442382 conn(0x560a6bc6dc90 msgr2=0x560a6bca2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.763+0000 7f11f6c71740 1 -- 172.21.15.43:0/4264442382 shutdown_connections 2024-04-21T08:46:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.763+0000 7f11f6c71740 1 -- 172.21.15.43:0/4264442382 wait complete. 2024-04-21T08:46:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.763+0000 7f11f6c71740 1 -- start start 2024-04-21T08:46:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.763+0000 7f11f6c71740 1 Processor -- start 2024-04-21T08:46:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.764+0000 7f11f6c71740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a6bca6b60 0x560a6bcb8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.764+0000 7f11f6c71740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6bca78d0 0x560a6bcb0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.764+0000 7f11f6c71740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bcb00e0 0x560a6bcb1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.764+0000 7f11f6c71740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec1b06e0 con 0x560a6bcb00e0 2024-04-21T08:46:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.764+0000 7f11f6c71740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec1b0010 con 0x560a6bca6b60 2024-04-21T08:46:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.764+0000 7f11f6c71740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec1ad250 con 0x560a6bca78d0 2024-04-21T08:46:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.764+0000 7f11f6c71740 10 client.0 _ll_register_callbacks cb 0x7f11e4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.764+0000 7f11f261e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bcb00e0 0x560a6bcb1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.764+0000 7f11f261e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bcb00e0 0x560a6bcb1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51468/0 (socket says 172.21.15.43:51468) 2024-04-21T08:46:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.764+0000 7f11f261e640 1 -- 172.21.15.43:0/3697121644 learned_addr learned my addr 172.21.15.43:0/3697121644 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:46:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.764+0000 7f11f161c640 1 --2- 172.21.15.43:0/3697121644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6bca78d0 0x560a6bcb0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.764+0000 7f11f1e1d640 1 --2- 172.21.15.43:0/3697121644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a6bca6b60 0x560a6bcb8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.764+0000 7f11f261e640 1 -- 172.21.15.43:0/3697121644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6bca78d0 msgr2=0x560a6bcb0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.764+0000 7f11f261e640 1 --2- 172.21.15.43:0/3697121644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6bca78d0 0x560a6bcb0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.764+0000 7f11f261e640 1 -- 172.21.15.43:0/3697121644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a6bca6b60 msgr2=0x560a6bcb8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.764+0000 7f11f261e640 1 --2- 172.21.15.43:0/3697121644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a6bca6b60 0x560a6bcb8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.764+0000 7f11f261e640 1 -- 172.21.15.43:0/3697121644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a6bcb16f0 con 0x560a6bcb00e0 2024-04-21T08:46:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.765+0000 7f11f261e640 1 --2- 172.21.15.43:0/3697121644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bcb00e0 0x560a6bcb1000 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f11e4006010 tx=0x7f11e4004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfdf8a189a4c2c5b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.765+0000 7f11d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:46:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.765+0000 7f11d57fa640 1 -- 172.21.15.43:0/3697121644 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e4038070 con 0x560a6bcb00e0 2024-04-21T08:46:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.765+0000 7f11d57fa640 1 -- 172.21.15.43:0/3697121644 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9495+0+0 (secure 0 0 0) 0x7f11e4005420 con 0x560a6bcb00e0 2024-04-21T08:46:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.765+0000 7f11f6c71740 1 -- 172.21.15.43:0/3697121644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560a6bbab3d0 con 0x560a6bcb00e0 2024-04-21T08:46:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.765+0000 7f11d57fa640 1 -- 172.21.15.43:0/3697121644 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f11e4009480 con 0x560a6bcb00e0 2024-04-21T08:46:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.765+0000 7f11f6c71740 10 client.19314 fetch_fsmap learned FSMap version 30 2024-04-21T08:46:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.765+0000 7f11f6c71740 1 -- 172.21.15.43:0/3697121644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560a6bd2a550 con 0x560a6bcb00e0 2024-04-21T08:46:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.765+0000 7f11d57fa640 1 -- 172.21.15.43:0/3697121644 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560a6bbab3d0 con 0x560a6bcb00e0 2024-04-21T08:46:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.765+0000 7f11d57fa640 1 -- 172.21.15.43:0/3697121644 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f11e402a070 con 0x560a6bcb00e0 2024-04-21T08:46:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.765+0000 7f11f6c71740 10 client.19314 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:46:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.765+0000 7f11f6c71740 10 client.19314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.765+0000 7f11f6c71740 20 client.19314 populate_metadata read hostname 'smithi043' 2024-04-21T08:46:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.765+0000 7f11f6c71740 1 --2- 172.21.15.43:0/3697121644 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560a6bd2aa60 0x560a6bd2ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.765+0000 7f11f6c71740 4 client.19314 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:46:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.765+0000 7f11f6c71740 1 -- 172.21.15.43:0/3697121644 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a6bc192f0 con 0x560a6bd2aa60 2024-04-21T08:46:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.766+0000 7f11f1e1d640 1 --2- 172.21.15.43:0/3697121644 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560a6bd2aa60 0x560a6bd2ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.766+0000 7f11f1e1d640 1 --2- 172.21.15.43:0/3697121644 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560a6bd2aa60 0x560a6bd2ce80 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69834850adf5cd3e server_cookie=7c3f1e10b6a32b9b in_seq=0 out_seq=0 2024-04-21T08:46:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.766+0000 7f11d57fa640 10 client.19314 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.781+0000 7f11d57fa640 1 -- 172.21.15.43:0/3697121644 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560a6bc192f0 con 0x560a6bd2aa60 2024-04-21T08:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.781+0000 7f11d57fa640 10 client.19314 handle_command_reply: tid=0 2024-04-21T08:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.781+0000 7f11f6c71740 1 client.19314 shutdown 2024-04-21T08:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.781+0000 7f11f6c71740 1 -- 172.21.15.43:0/3697121644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bcb00e0 msgr2=0x560a6bcb1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.781+0000 7f11f6c71740 1 --2- 172.21.15.43:0/3697121644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bcb00e0 0x560a6bcb1000 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f11e4006010 tx=0x7f11e4004dd0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.781+0000 7f11f6c71740 1 -- 172.21.15.43:0/3697121644 shutdown_connections 2024-04-21T08:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.781+0000 7f11f6c71740 1 --2- 172.21.15.43:0/3697121644 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560a6bd2aa60 0x560a6bd2ce80 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.781+0000 7f11f6c71740 1 --2- 172.21.15.43:0/3697121644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a6bcb00e0 0x560a6bcb1000 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.781+0000 7f11f6c71740 1 --2- 172.21.15.43:0/3697121644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6bca78d0 0x560a6bcb0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.781+0000 7f11f6c71740 1 --2- 172.21.15.43:0/3697121644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a6bca6b60 0x560a6bcb8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.781+0000 7f11d57fa640 0 client.19314 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.781+0000 7f11f6c71740 1 -- 172.21.15.43:0/3697121644 >> 172.21.15.43:0/3697121644 conn(0x560a6bc6dc90 msgr2=0x560a6bca08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.781+0000 7f11f6c71740 1 -- 172.21.15.43:0/3697121644 shutdown_connections 2024-04-21T08:46:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.781+0000 7f11f6c71740 1 -- 172.21.15.43:0/3697121644 wait complete. 2024-04-21T08:46:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.781+0000 7f11f6c71740 20 client.19314 trim_cache size 0 max 16384 2024-04-21T08:46:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:46:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:46:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:46:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:46:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.782+0000 7f11f4081640 1 -- 172.21.15.43:0/1597102641 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f11d80a3e40 msgr2=0x7f11d80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.782+0000 7f11f4081640 1 --2- 172.21.15.43:0/1597102641 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f11d80a3e40 0x7f11d80a6300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f11e40049d0 tx=0x7f11e4004ab0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.782+0000 7f11f4081640 1 -- 172.21.15.43:0/1597102641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11ec1061b0 msgr2=0x7f11ec198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.782+0000 7f11f4081640 1 --2- 172.21.15.43:0/1597102641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11ec1061b0 0x7f11ec198c20 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00d770 tx=0x7f11dc00dc30 comp rx=0 tx=0).stop 2024-04-21T08:46:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.782+0000 7f11f4081640 1 -- 172.21.15.43:0/1597102641 shutdown_connections 2024-04-21T08:46:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.782+0000 7f11f4081640 1 --2- 172.21.15.43:0/1597102641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11ec19cf50 0x7f11ec19d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.782+0000 7f11f4081640 1 --2- 172.21.15.43:0/1597102641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11ec1061b0 0x7f11ec198c20 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.782+0000 7f11f4081640 1 --2- 172.21.15.43:0/1597102641 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f11d80a3e40 0x7f11d80a6300 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.782+0000 7f11f4081640 1 --2- 172.21.15.43:0/1597102641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11ec1010e0 0x7f11ec1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.782+0000 7f11f4081640 1 -- 172.21.15.43:0/1597102641 >> 172.21.15.43:0/1597102641 conn(0x7f11ec0771d0 msgr2=0x7f11ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.782+0000 7f11f4081640 1 -- 172.21.15.43:0/1597102641 shutdown_connections 2024-04-21T08:46:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:46:56.782+0000 7f11f4081640 1 -- 172.21.15.43:0/1597102641 wait complete. 2024-04-21T08:46:58.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:46:58 smithi175 ceph-mon[31920]: pgmap v247: 129 pgs: 129 active+clean; 3.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-04-21T08:46:58.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:46:58 smithi112 ceph-mon[31085]: pgmap v247: 129 pgs: 129 active+clean; 3.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-04-21T08:46:58.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:46:58 smithi043 ceph-mon[27978]: pgmap v247: 129 pgs: 129 active+clean; 3.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-04-21T08:47:00.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:00 smithi175 ceph-mon[31920]: pgmap v248: 129 pgs: 129 active+clean; 3.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-04-21T08:47:00.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:00 smithi112 ceph-mon[31085]: pgmap v248: 129 pgs: 129 active+clean; 3.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-04-21T08:47:00.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:00 smithi043 ceph-mon[27978]: pgmap v248: 129 pgs: 129 active+clean; 3.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-04-21T08:47:02.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:02 smithi175 ceph-mon[31920]: pgmap v249: 129 pgs: 129 active+clean; 3.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 81 KiB/s wr, 36 op/s 2024-04-21T08:47:02.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:02 smithi112 ceph-mon[31085]: pgmap v249: 129 pgs: 129 active+clean; 3.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 81 KiB/s wr, 36 op/s 2024-04-21T08:47:02.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:02 smithi043 ceph-mon[27978]: pgmap v249: 129 pgs: 129 active+clean; 3.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 81 KiB/s wr, 36 op/s 2024-04-21T08:47:04.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:04 smithi175 ceph-mon[31920]: pgmap v250: 129 pgs: 129 active+clean; 3.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 64 KiB/s wr, 30 op/s 2024-04-21T08:47:04.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:04 smithi112 ceph-mon[31085]: pgmap v250: 129 pgs: 129 active+clean; 3.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 64 KiB/s wr, 30 op/s 2024-04-21T08:47:04.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:04 smithi043 ceph-mon[27978]: pgmap v250: 129 pgs: 129 active+clean; 3.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 64 KiB/s wr, 30 op/s 2024-04-21T08:47:06.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:06 smithi175 ceph-mon[31920]: pgmap v251: 129 pgs: 129 active+clean; 3.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 64 KiB/s wr, 30 op/s 2024-04-21T08:47:06.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:06 smithi175 ceph-mon[31920]: osdmap e99: 12 total, 12 up, 12 in 2024-04-21T08:47:06.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:06 smithi112 ceph-mon[31085]: pgmap v251: 129 pgs: 129 active+clean; 3.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 64 KiB/s wr, 30 op/s 2024-04-21T08:47:06.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:06 smithi112 ceph-mon[31085]: osdmap e99: 12 total, 12 up, 12 in 2024-04-21T08:47:06.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:06 smithi043 ceph-mon[27978]: pgmap v251: 129 pgs: 129 active+clean; 3.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 64 KiB/s wr, 30 op/s 2024-04-21T08:47:06.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:06 smithi043 ceph-mon[27978]: osdmap e99: 12 total, 12 up, 12 in 2024-04-21T08:47:07.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:07 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:47:07.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:07 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:47:07.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:07 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:47:07.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:07 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:47:07.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:07 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:47:07.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:07 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:47:07.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:07 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:47:07.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:07 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:47:07.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:07 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:47:07.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:07 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:47:07.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:07 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:47:07.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:07 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:47:08.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:08 smithi175 ceph-mon[31920]: pgmap v253: 129 pgs: 129 active+clean; 3.9 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-21T08:47:08.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:08 smithi112 ceph-mon[31085]: pgmap v253: 129 pgs: 129 active+clean; 3.9 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-21T08:47:08.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:08 smithi043 ceph-mon[27978]: pgmap v253: 129 pgs: 129 active+clean; 3.9 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-21T08:47:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.881+0000 7f7e235db640 1 -- 172.21.15.43:0/2563089462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e1c102b80 msgr2=0x7f7e1c106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.881+0000 7f7e235db640 1 --2- 172.21.15.43:0/2563089462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e1c102b80 0x7f7e1c106bc0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009900 tx=0x7f7e0c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.881+0000 7f7e235db640 1 -- 172.21.15.43:0/2563089462 shutdown_connections 2024-04-21T08:47:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.881+0000 7f7e235db640 1 --2- 172.21.15.43:0/2563089462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e1c1072a0 0x7f7e1c10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.881+0000 7f7e235db640 1 --2- 172.21.15.43:0/2563089462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e1c102b80 0x7f7e1c106bc0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.881+0000 7f7e235db640 1 --2- 172.21.15.43:0/2563089462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e1c1021d0 0x7f7e1c1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.881+0000 7f7e235db640 1 -- 172.21.15.43:0/2563089462 >> 172.21.15.43:0/2563089462 conn(0x7f7e1c0793e0 msgr2=0x7f7e1c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.882+0000 7f7e235db640 1 -- 172.21.15.43:0/2563089462 shutdown_connections 2024-04-21T08:47:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.882+0000 7f7e235db640 1 -- 172.21.15.43:0/2563089462 wait complete. 2024-04-21T08:47:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.882+0000 7f7e235db640 1 Processor -- start 2024-04-21T08:47:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.882+0000 7f7e235db640 1 -- start start 2024-04-21T08:47:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.882+0000 7f7e235db640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e1c1021d0 0x7f7e1c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.882+0000 7f7e235db640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e1c1072a0 0x7f7e1c198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.883+0000 7f7e235db640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e1c19cfa0 0x7f7e1c19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.883+0000 7f7e235db640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c1121e0 con 0x7f7e1c1072a0 2024-04-21T08:47:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.883+0000 7f7e235db640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c112060 con 0x7f7e1c1021d0 2024-04-21T08:47:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.883+0000 7f7e235db640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c112360 con 0x7f7e1c19cfa0 2024-04-21T08:47:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.883+0000 7f7e20b76640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e1c1072a0 0x7f7e1c198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.883+0000 7f7e20b76640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e1c1072a0 0x7f7e1c198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43606/0 (socket says 172.21.15.43:43606) 2024-04-21T08:47:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.883+0000 7f7e20b76640 1 -- 172.21.15.43:0/666728961 learned_addr learned my addr 172.21.15.43:0/666728961 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.883+0000 7f7e21377640 1 --2- 172.21.15.43:0/666728961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e1c1021d0 0x7f7e1c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.883+0000 7f7e21b78640 1 --2- 172.21.15.43:0/666728961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e1c19cfa0 0x7f7e1c19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.883+0000 7f7e20b76640 1 -- 172.21.15.43:0/666728961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e1c19cfa0 msgr2=0x7f7e1c19d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.883+0000 7f7e20b76640 1 --2- 172.21.15.43:0/666728961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e1c19cfa0 0x7f7e1c19d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.883+0000 7f7e20b76640 1 -- 172.21.15.43:0/666728961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e1c1021d0 msgr2=0x7f7e1c1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.883+0000 7f7e20b76640 1 --2- 172.21.15.43:0/666728961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e1c1021d0 0x7f7e1c1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.883+0000 7f7e20b76640 1 -- 172.21.15.43:0/666728961 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e1c19da90 con 0x7f7e1c1072a0 2024-04-21T08:47:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.883+0000 7f7e21b78640 1 --2- 172.21.15.43:0/666728961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e1c19cfa0 0x7f7e1c19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.883+0000 7f7e20b76640 1 --2- 172.21.15.43:0/666728961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e1c1072a0 0x7f7e1c198c20 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c036040 tx=0x7f7e0c02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.883+0000 7f7e127fc640 1 -- 172.21.15.43:0/666728961 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c043020 con 0x7f7e1c1072a0 2024-04-21T08:47:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.883+0000 7f7e127fc640 1 -- 172.21.15.43:0/666728961 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e0c02cbe0 con 0x7f7e1c1072a0 2024-04-21T08:47:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.883+0000 7f7e235db640 1 -- 172.21.15.43:0/666728961 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e1c19dd20 con 0x7f7e1c1072a0 2024-04-21T08:47:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.883+0000 7f7e127fc640 1 -- 172.21.15.43:0/666728961 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c003bf0 con 0x7f7e1c1072a0 2024-04-21T08:47:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.883+0000 7f7e235db640 1 -- 172.21.15.43:0/666728961 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e1c1a5570 con 0x7f7e1c1072a0 2024-04-21T08:47:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.886+0000 7f7e127fc640 1 -- 172.21.15.43:0/666728961 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7e0c046020 con 0x7f7e1c1072a0 2024-04-21T08:47:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.886+0000 7f7e261c5740 1 Processor -- start 2024-04-21T08:47:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.886+0000 7f7e261c5740 1 -- start start 2024-04-21T08:47:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.887+0000 7f7e261c5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf368d0 0x5626fcf35620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.887+0000 7f7e261c5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626fcf35b60 0x5626fcf35fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.887+0000 7f7e261c5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626fcf3f0e0 0x5626fcf414d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.887+0000 7f7e261c5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5626fce9a8d0 con 0x5626fcf368d0 2024-04-21T08:47:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.887+0000 7f7e261c5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5626fce37e80 con 0x5626fcf3f0e0 2024-04-21T08:47:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.887+0000 7f7e261c5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5626fce96890 con 0x5626fcf35b60 2024-04-21T08:47:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.887+0000 7f7e127fc640 1 --2- 172.21.15.43:0/666728961 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7e080a3e40 0x7f7e080a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.887+0000 7f7e21b78640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf368d0 0x5626fcf35620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.887+0000 7f7e21b78640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf368d0 0x5626fcf35620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43608/0 (socket says 172.21.15.43:43608) 2024-04-21T08:47:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.887+0000 7f7e21b78640 1 -- 172.21.15.43:0/4214726022 learned_addr learned my addr 172.21.15.43:0/4214726022 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.887+0000 7f7e21b78640 1 --2- 172.21.15.43:0/666728961 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7e080a3e40 0x7f7e080a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.887+0000 7f7e20b76640 1 --2- 172.21.15.43:0/4214726022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626fcf3f0e0 0x5626fcf414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.887+0000 7f7e21377640 1 --2- 172.21.15.43:0/4214726022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626fcf35b60 0x5626fcf35fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.887+0000 7f7e127fc640 1 -- 172.21.15.43:0/666728961 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9671+0+0 (secure 0 0 0) 0x7f7e0c0f2d90 con 0x7f7e1c1072a0 2024-04-21T08:47:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.887+0000 7f7e21b78640 1 -- 172.21.15.43:0/4214726022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626fcf35b60 msgr2=0x5626fcf35fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.887+0000 7f7e21b78640 1 --2- 172.21.15.43:0/4214726022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626fcf35b60 0x5626fcf35fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.887+0000 7f7e21b78640 1 -- 172.21.15.43:0/4214726022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626fcf3f0e0 msgr2=0x5626fcf414d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.887+0000 7f7e21b78640 1 --2- 172.21.15.43:0/4214726022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626fcf3f0e0 0x5626fcf414d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.887+0000 7f7e21b78640 1 -- 172.21.15.43:0/4214726022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626fce59320 con 0x5626fcf368d0 2024-04-21T08:47:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.888+0000 7f7e21b78640 1 --2- 172.21.15.43:0/4214726022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf368d0 0x5626fcf35620 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f7e14005c40 tx=0x7f7e14010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95f93b19e37e4db server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.888+0000 7f7e21b78640 1 --2- 172.21.15.43:0/666728961 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7e080a3e40 0x7f7e080a6300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7e14010c40 tx=0x7f7e14017000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.888+0000 7f7e077fe640 1 -- 172.21.15.43:0/4214726022 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14022070 con 0x5626fcf368d0 2024-04-21T08:47:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.888+0000 7f7e077fe640 1 -- 172.21.15.43:0/4214726022 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e14012e90 con 0x5626fcf368d0 2024-04-21T08:47:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.888+0000 7f7e077fe640 1 -- 172.21.15.43:0/4214726022 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14004d60 con 0x5626fcf368d0 2024-04-21T08:47:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.888+0000 7f7e261c5740 1 -- 172.21.15.43:0/4214726022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf368d0 msgr2=0x5626fcf35620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.888+0000 7f7e261c5740 1 --2- 172.21.15.43:0/4214726022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf368d0 0x5626fcf35620 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f7e14005c40 tx=0x7f7e14010740 comp rx=0 tx=0).stop 2024-04-21T08:47:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.889+0000 7f7e261c5740 1 -- 172.21.15.43:0/4214726022 shutdown_connections 2024-04-21T08:47:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.889+0000 7f7e261c5740 1 --2- 172.21.15.43:0/4214726022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626fcf3f0e0 0x5626fcf414d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.889+0000 7f7e261c5740 1 --2- 172.21.15.43:0/4214726022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626fcf35b60 0x5626fcf35fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.889+0000 7f7e261c5740 1 --2- 172.21.15.43:0/4214726022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf368d0 0x5626fcf35620 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.889+0000 7f7e261c5740 1 -- 172.21.15.43:0/4214726022 >> 172.21.15.43:0/4214726022 conn(0x5626fcefcc90 msgr2=0x5626fcf30850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.889+0000 7f7e261c5740 1 -- 172.21.15.43:0/4214726022 shutdown_connections 2024-04-21T08:47:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.889+0000 7f7e261c5740 1 -- 172.21.15.43:0/4214726022 wait complete. 2024-04-21T08:47:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.889+0000 7f7e261c5740 1 -- start start 2024-04-21T08:47:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.889+0000 7f7e261c5740 1 Processor -- start 2024-04-21T08:47:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.890+0000 7f7e261c5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626fcf35b60 0x5626fcf3eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.890+0000 7f7e261c5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf368d0 0x5626fcf3be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.890+0000 7f7e261c5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626fcf3f0e0 0x5626fcf3c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.890+0000 7f7e261c5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5626fcf41db0 con 0x5626fcf368d0 2024-04-21T08:47:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.890+0000 7f7e261c5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5626fce22eb0 con 0x5626fcf35b60 2024-04-21T08:47:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.890+0000 7f7e261c5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5626fcf41f30 con 0x5626fcf3f0e0 2024-04-21T08:47:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.890+0000 7f7e261c5740 10 client.0 _ll_register_callbacks cb 0x5626fce4fb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.890+0000 7f7e20b76640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf368d0 0x5626fcf3be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.890+0000 7f7e20b76640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf368d0 0x5626fcf3be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43622/0 (socket says 172.21.15.43:43622) 2024-04-21T08:47:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.890+0000 7f7e20b76640 1 -- 172.21.15.43:0/3232912417 learned_addr learned my addr 172.21.15.43:0/3232912417 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.890+0000 7f7e21b78640 1 --2- 172.21.15.43:0/3232912417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626fcf3f0e0 0x5626fcf3c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.890+0000 7f7e21377640 1 --2- 172.21.15.43:0/3232912417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626fcf35b60 0x5626fcf3eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.890+0000 7f7e20b76640 1 -- 172.21.15.43:0/3232912417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626fcf3f0e0 msgr2=0x5626fcf3c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.890+0000 7f7e20b76640 1 --2- 172.21.15.43:0/3232912417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626fcf3f0e0 0x5626fcf3c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.890+0000 7f7e20b76640 1 -- 172.21.15.43:0/3232912417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626fcf35b60 msgr2=0x5626fcf3eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.890+0000 7f7e20b76640 1 --2- 172.21.15.43:0/3232912417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626fcf35b60 0x5626fcf3eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.890+0000 7f7e20b76640 1 -- 172.21.15.43:0/3232912417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626fcbad210 con 0x5626fcf368d0 2024-04-21T08:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.890+0000 7f7e21377640 1 --2- 172.21.15.43:0/3232912417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626fcf35b60 0x5626fcf3eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.890+0000 7f7e20b76640 1 --2- 172.21.15.43:0/3232912417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf368d0 0x5626fcf3be80 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0ba6a0 tx=0x7f7e0c0095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9d9c52428dd5bd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.890+0000 7f7e04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.890+0000 7f7e04ff9640 1 -- 172.21.15.43:0/3232912417 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c0d7060 con 0x5626fcf368d0 2024-04-21T08:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.890+0000 7f7e04ff9640 1 -- 172.21.15.43:0/3232912417 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9671+0+0 (secure 0 0 0) 0x7f7e0c0c4190 con 0x5626fcf368d0 2024-04-21T08:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.890+0000 7f7e261c5740 1 -- 172.21.15.43:0/3232912417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e14012e90 con 0x5626fcf368d0 2024-04-21T08:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.891+0000 7f7e04ff9640 1 -- 172.21.15.43:0/3232912417 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7e0c0be320 con 0x5626fcf368d0 2024-04-21T08:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.891+0000 7f7e261c5740 10 client.19326 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.891+0000 7f7e261c5740 1 -- 172.21.15.43:0/3232912417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5626fcc79620 con 0x5626fcf368d0 2024-04-21T08:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.891+0000 7f7e04ff9640 1 -- 172.21.15.43:0/3232912417 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e14012e90 con 0x5626fcf368d0 2024-04-21T08:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.891+0000 7f7e04ff9640 1 -- 172.21.15.43:0/3232912417 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7e0c0bb030 con 0x5626fcf368d0 2024-04-21T08:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.891+0000 7f7e261c5740 10 client.19326 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.891+0000 7f7e261c5740 10 client.19326 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.891+0000 7f7e261c5740 20 client.19326 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.891+0000 7f7e261c5740 1 --2- 172.21.15.43:0/3232912417 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5626fcf4f060 0x5626fcf51490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.891+0000 7f7e261c5740 4 client.19326 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.891+0000 7f7e261c5740 1 -- 172.21.15.43:0/3232912417 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626fceae880 con 0x5626fcf4f060 2024-04-21T08:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.891+0000 7f7e21377640 1 --2- 172.21.15.43:0/3232912417 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5626fcf4f060 0x5626fcf51490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.892+0000 7f7e21377640 1 --2- 172.21.15.43:0/3232912417 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5626fcf4f060 0x5626fcf51490 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac0bdd8e4dd072b6 server_cookie=e787b91460f47b68 in_seq=0 out_seq=0 2024-04-21T08:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.892+0000 7f7e04ff9640 10 client.19326 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.893+0000 7f7e04ff9640 1 -- 172.21.15.43:0/3232912417 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5626fceae880 con 0x5626fcf4f060 2024-04-21T08:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.893+0000 7f7e04ff9640 10 client.19326 handle_command_reply: tid=0 2024-04-21T08:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.893+0000 7f7e261c5740 1 client.19326 shutdown 2024-04-21T08:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.893+0000 7f7e261c5740 1 -- 172.21.15.43:0/3232912417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf368d0 msgr2=0x5626fcf3be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.893+0000 7f7e261c5740 1 --2- 172.21.15.43:0/3232912417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf368d0 0x5626fcf3be80 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0ba6a0 tx=0x7f7e0c0095d0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.893+0000 7f7e261c5740 1 -- 172.21.15.43:0/3232912417 shutdown_connections 2024-04-21T08:47:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.893+0000 7f7e261c5740 1 --2- 172.21.15.43:0/3232912417 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5626fcf4f060 0x5626fcf51490 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.893+0000 7f7e261c5740 1 --2- 172.21.15.43:0/3232912417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626fcf3f0e0 0x5626fcf3c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.893+0000 7f7e261c5740 1 --2- 172.21.15.43:0/3232912417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf368d0 0x5626fcf3be80 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.893+0000 7f7e261c5740 1 --2- 172.21.15.43:0/3232912417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626fcf35b60 0x5626fcf3eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.893+0000 7f7e04ff9640 0 client.19326 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.893+0000 7f7e261c5740 1 -- 172.21.15.43:0/3232912417 >> 172.21.15.43:0/3232912417 conn(0x5626fcefcc90 msgr2=0x5626fcf325a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.893+0000 7f7e261c5740 1 -- 172.21.15.43:0/3232912417 shutdown_connections 2024-04-21T08:47:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.893+0000 7f7e261c5740 1 -- 172.21.15.43:0/3232912417 wait complete. 2024-04-21T08:47:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.893+0000 7f7e261c5740 20 client.19326 trim_cache size 0 max 16384 2024-04-21T08:47:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.909+0000 7f7e261c5740 1 Processor -- start 2024-04-21T08:47:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.909+0000 7f7e261c5740 1 -- start start 2024-04-21T08:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.909+0000 7f7e261c5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626fcefcc90 0x5626fcf4acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.909+0000 7f7e261c5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf35b60 0x5626fcf3e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.909+0000 7f7e261c5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626fcf368d0 0x5626fcf3eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.909+0000 7f7e261c5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c1af560 con 0x5626fcf35b60 2024-04-21T08:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.909+0000 7f7e261c5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c1b10a0 con 0x5626fcf368d0 2024-04-21T08:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.909+0000 7f7e261c5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c1afc30 con 0x5626fcefcc90 2024-04-21T08:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.909+0000 7f7e20b76640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf35b60 0x5626fcf3e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.909+0000 7f7e20b76640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf35b60 0x5626fcf3e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43624/0 (socket says 172.21.15.43:43624) 2024-04-21T08:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.909+0000 7f7e20b76640 1 -- 172.21.15.43:0/2071028883 learned_addr learned my addr 172.21.15.43:0/2071028883 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.909+0000 7f7e21b78640 1 --2- 172.21.15.43:0/2071028883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626fcf368d0 0x5626fcf3eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.909+0000 7f7e21377640 1 --2- 172.21.15.43:0/2071028883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626fcefcc90 0x5626fcf4acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.910+0000 7f7e20b76640 1 -- 172.21.15.43:0/2071028883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626fcefcc90 msgr2=0x5626fcf4acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.910+0000 7f7e20b76640 1 --2- 172.21.15.43:0/2071028883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626fcefcc90 0x5626fcf4acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.910+0000 7f7e20b76640 1 -- 172.21.15.43:0/2071028883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626fcf368d0 msgr2=0x5626fcf3eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.910+0000 7f7e20b76640 1 --2- 172.21.15.43:0/2071028883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626fcf368d0 0x5626fcf3eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.910+0000 7f7e20b76640 1 -- 172.21.15.43:0/2071028883 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626fcf2bf30 con 0x5626fcf35b60 2024-04-21T08:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.910+0000 7f7e21377640 1 --2- 172.21.15.43:0/2071028883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626fcefcc90 0x5626fcf4acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.910+0000 7f7e20b76640 1 --2- 172.21.15.43:0/2071028883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf35b60 0x5626fcf3e5b0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c005940 tx=0x7f7e0c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91bf9ef5afbdf482 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.910+0000 7f7e06ffd640 1 -- 172.21.15.43:0/2071028883 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c0c3f00 con 0x5626fcf35b60 2024-04-21T08:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.910+0000 7f7e06ffd640 1 -- 172.21.15.43:0/2071028883 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e0c0bf0c0 con 0x5626fcf35b60 2024-04-21T08:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.910+0000 7f7e06ffd640 1 -- 172.21.15.43:0/2071028883 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c0bec20 con 0x5626fcf35b60 2024-04-21T08:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.911+0000 7f7e261c5740 1 -- 172.21.15.43:0/2071028883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf35b60 msgr2=0x5626fcf3e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.911+0000 7f7e261c5740 1 --2- 172.21.15.43:0/2071028883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf35b60 0x5626fcf3e5b0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c005940 tx=0x7f7e0c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.911+0000 7f7e261c5740 1 -- 172.21.15.43:0/2071028883 shutdown_connections 2024-04-21T08:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.911+0000 7f7e261c5740 1 --2- 172.21.15.43:0/2071028883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626fcf368d0 0x5626fcf3eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.911+0000 7f7e261c5740 1 --2- 172.21.15.43:0/2071028883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf35b60 0x5626fcf3e5b0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.911+0000 7f7e261c5740 1 --2- 172.21.15.43:0/2071028883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626fcefcc90 0x5626fcf4acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.911+0000 7f7e261c5740 1 -- 172.21.15.43:0/2071028883 >> 172.21.15.43:0/2071028883 conn(0x5626fcf356c0 msgr2=0x5626fcf37060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.911+0000 7f7e261c5740 1 -- 172.21.15.43:0/2071028883 shutdown_connections 2024-04-21T08:47:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.911+0000 7f7e261c5740 1 -- 172.21.15.43:0/2071028883 wait complete. 2024-04-21T08:47:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.911+0000 7f7e261c5740 1 -- start start 2024-04-21T08:47:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.911+0000 7f7e261c5740 1 Processor -- start 2024-04-21T08:47:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.911+0000 7f7e261c5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626fcefcc90 0x5626fcf38af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.911+0000 7f7e261c5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626fcf35b60 0x5626fcf39030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.912+0000 7f7e261c5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf368d0 0x5626fcf39570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.912+0000 7f7e261c5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c1b09d0 con 0x5626fcf368d0 2024-04-21T08:47:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.912+0000 7f7e261c5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c1b0300 con 0x5626fcefcc90 2024-04-21T08:47:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.912+0000 7f7e261c5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c1ad560 con 0x5626fcf35b60 2024-04-21T08:47:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.912+0000 7f7e261c5740 10 client.0 _ll_register_callbacks cb 0x7f7e0c0057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.912+0000 7f7e21b78640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf368d0 0x5626fcf39570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.912+0000 7f7e21b78640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf368d0 0x5626fcf39570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43636/0 (socket says 172.21.15.43:43636) 2024-04-21T08:47:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.912+0000 7f7e21b78640 1 -- 172.21.15.43:0/3990844001 learned_addr learned my addr 172.21.15.43:0/3990844001 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.912+0000 7f7e21377640 1 --2- 172.21.15.43:0/3990844001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626fcefcc90 0x5626fcf38af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.912+0000 7f7e20b76640 1 --2- 172.21.15.43:0/3990844001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626fcf35b60 0x5626fcf39030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.912+0000 7f7e21b78640 1 -- 172.21.15.43:0/3990844001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626fcf35b60 msgr2=0x5626fcf39030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.912+0000 7f7e21b78640 1 --2- 172.21.15.43:0/3990844001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626fcf35b60 0x5626fcf39030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.912+0000 7f7e21b78640 1 -- 172.21.15.43:0/3990844001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626fcefcc90 msgr2=0x5626fcf38af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.912+0000 7f7e21b78640 1 --2- 172.21.15.43:0/3990844001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626fcefcc90 0x5626fcf38af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.912+0000 7f7e21b78640 1 -- 172.21.15.43:0/3990844001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626fcfb4640 con 0x5626fcf368d0 2024-04-21T08:47:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.912+0000 7f7e21377640 1 --2- 172.21.15.43:0/3990844001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626fcefcc90 0x5626fcf38af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.912+0000 7f7e21b78640 1 --2- 172.21.15.43:0/3990844001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf368d0 0x5626fcf39570 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009fd0 tx=0x7f7e14012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598d8e7652d33436 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.912+0000 7f7e04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.912+0000 7f7e04ff9640 1 -- 172.21.15.43:0/3990844001 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14022040 con 0x5626fcf368d0 2024-04-21T08:47:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.912+0000 7f7e04ff9640 1 -- 172.21.15.43:0/3990844001 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9671+0+0 (secure 0 0 0) 0x7f7e1400cb40 con 0x5626fcf368d0 2024-04-21T08:47:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.912+0000 7f7e261c5740 1 -- 172.21.15.43:0/3990844001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e0c0bf0c0 con 0x5626fcf368d0 2024-04-21T08:47:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.913+0000 7f7e04ff9640 1 -- 172.21.15.43:0/3990844001 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7e1403b050 con 0x5626fcf368d0 2024-04-21T08:47:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.913+0000 7f7e261c5740 10 client.19332 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.913+0000 7f7e261c5740 1 -- 172.21.15.43:0/3990844001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5626fcfb5930 con 0x5626fcf368d0 2024-04-21T08:47:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.913+0000 7f7e04ff9640 1 -- 172.21.15.43:0/3990844001 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e0c0bf0c0 con 0x5626fcf368d0 2024-04-21T08:47:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.913+0000 7f7e04ff9640 1 -- 172.21.15.43:0/3990844001 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7e14005250 con 0x5626fcf368d0 2024-04-21T08:47:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.913+0000 7f7e261c5740 10 client.19332 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.913+0000 7f7e261c5740 10 client.19332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.913+0000 7f7e261c5740 20 client.19332 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.913+0000 7f7e261c5740 1 --2- 172.21.15.43:0/3990844001 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5626fcf3f0e0 0x5626fcfb7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.913+0000 7f7e261c5740 4 client.19332 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:47:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.913+0000 7f7e261c5740 1 -- 172.21.15.43:0/3990844001 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5626fce3a3d0 con 0x5626fcf3f0e0 2024-04-21T08:47:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.913+0000 7f7e21377640 1 --2- 172.21.15.43:0/3990844001 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5626fcf3f0e0 0x5626fcfb7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.914+0000 7f7e21377640 1 --2- 172.21.15.43:0/3990844001 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5626fcf3f0e0 0x5626fcfb7e90 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=930b3d22c18af77e server_cookie=c295901d47fdd777 in_seq=0 out_seq=0 2024-04-21T08:47:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.914+0000 7f7e04ff9640 10 client.19332 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.929+0000 7f7e04ff9640 1 -- 172.21.15.43:0/3990844001 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5626fce3a3d0 con 0x5626fcf3f0e0 2024-04-21T08:47:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.929+0000 7f7e04ff9640 10 client.19332 handle_command_reply: tid=0 2024-04-21T08:47:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.929+0000 7f7e261c5740 1 client.19332 shutdown 2024-04-21T08:47:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.929+0000 7f7e261c5740 1 -- 172.21.15.43:0/3990844001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf368d0 msgr2=0x5626fcf39570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.929+0000 7f7e261c5740 1 --2- 172.21.15.43:0/3990844001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf368d0 0x5626fcf39570 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009fd0 tx=0x7f7e14012bc0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.929+0000 7f7e261c5740 1 -- 172.21.15.43:0/3990844001 shutdown_connections 2024-04-21T08:47:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.929+0000 7f7e261c5740 1 --2- 172.21.15.43:0/3990844001 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5626fcf3f0e0 0x5626fcfb7e90 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.929+0000 7f7e261c5740 1 --2- 172.21.15.43:0/3990844001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626fcf368d0 0x5626fcf39570 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.929+0000 7f7e261c5740 1 --2- 172.21.15.43:0/3990844001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626fcf35b60 0x5626fcf39030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.929+0000 7f7e04ff9640 0 client.19332 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.929+0000 7f7e261c5740 1 --2- 172.21.15.43:0/3990844001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5626fcefcc90 0x5626fcf38af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.929+0000 7f7e261c5740 1 -- 172.21.15.43:0/3990844001 >> 172.21.15.43:0/3990844001 conn(0x5626fcf356c0 msgr2=0x5626fcf2f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.930+0000 7f7e261c5740 1 -- 172.21.15.43:0/3990844001 shutdown_connections 2024-04-21T08:47:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.930+0000 7f7e261c5740 1 -- 172.21.15.43:0/3990844001 wait complete. 2024-04-21T08:47:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.930+0000 7f7e261c5740 20 client.19332 trim_cache size 0 max 16384 2024-04-21T08:47:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:47:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:47:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:47:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:47:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.930+0000 7f7e235db640 1 -- 172.21.15.43:0/666728961 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7e080a3e40 msgr2=0x7f7e080a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.930+0000 7f7e235db640 1 --2- 172.21.15.43:0/666728961 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7e080a3e40 0x7f7e080a6300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7e14010c40 tx=0x7f7e14017000 comp rx=0 tx=0).stop 2024-04-21T08:47:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.930+0000 7f7e235db640 1 -- 172.21.15.43:0/666728961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e1c1072a0 msgr2=0x7f7e1c198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.930+0000 7f7e235db640 1 --2- 172.21.15.43:0/666728961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e1c1072a0 0x7f7e1c198c20 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c036040 tx=0x7f7e0c02c7e0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.930+0000 7f7e235db640 1 -- 172.21.15.43:0/666728961 shutdown_connections 2024-04-21T08:47:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.930+0000 7f7e235db640 1 --2- 172.21.15.43:0/666728961 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7e080a3e40 0x7f7e080a6300 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.930+0000 7f7e235db640 1 --2- 172.21.15.43:0/666728961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e1c19cfa0 0x7f7e1c19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.930+0000 7f7e235db640 1 --2- 172.21.15.43:0/666728961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e1c1072a0 0x7f7e1c198c20 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.930+0000 7f7e235db640 1 --2- 172.21.15.43:0/666728961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e1c1021d0 0x7f7e1c1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.930+0000 7f7e235db640 1 -- 172.21.15.43:0/666728961 >> 172.21.15.43:0/666728961 conn(0x7f7e1c0793e0 msgr2=0x7f7e1c0ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.930+0000 7f7e235db640 1 -- 172.21.15.43:0/666728961 shutdown_connections 2024-04-21T08:47:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:09.930+0000 7f7e235db640 1 -- 172.21.15.43:0/666728961 wait complete. 2024-04-21T08:47:10.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:10 smithi175 ceph-mon[31920]: pgmap v254: 129 pgs: 129 active+clean; 3.9 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-21T08:47:10.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:10 smithi175 ceph-mon[31920]: osdmap e100: 12 total, 12 up, 12 in 2024-04-21T08:47:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:10 smithi112 ceph-mon[31085]: pgmap v254: 129 pgs: 129 active+clean; 3.9 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-21T08:47:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:10 smithi112 ceph-mon[31085]: osdmap e100: 12 total, 12 up, 12 in 2024-04-21T08:47:10.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:10 smithi043 ceph-mon[27978]: pgmap v254: 129 pgs: 129 active+clean; 3.9 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-21T08:47:10.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:10 smithi043 ceph-mon[27978]: osdmap e100: 12 total, 12 up, 12 in 2024-04-21T08:47:12.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:12 smithi175 ceph-mon[31920]: pgmap v256: 129 pgs: 129 active+clean; 4.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 103 KiB/s wr, 27 op/s 2024-04-21T08:47:12.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:12 smithi112 ceph-mon[31085]: pgmap v256: 129 pgs: 129 active+clean; 4.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 103 KiB/s wr, 27 op/s 2024-04-21T08:47:12.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:12 smithi043 ceph-mon[27978]: pgmap v256: 129 pgs: 129 active+clean; 4.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 103 KiB/s wr, 27 op/s 2024-04-21T08:47:14.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:47:14.163 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:47:14.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.281+0000 7f4a148ba640 1 -- 172.21.15.43:0/3009091305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a10105dd0 msgr2=0x7f4a101061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.281+0000 7f4a148ba640 1 --2- 172.21.15.43:0/3009091305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a10105dd0 0x7f4a101061b0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009900 tx=0x7f4a0002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:14.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.281+0000 7f4a148ba640 1 -- 172.21.15.43:0/3009091305 shutdown_connections 2024-04-21T08:47:14.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.281+0000 7f4a148ba640 1 --2- 172.21.15.43:0/3009091305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a10101a40 0x7f4a1010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.281+0000 7f4a148ba640 1 --2- 172.21.15.43:0/3009091305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a101010a0 0x7f4a10101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.281+0000 7f4a148ba640 1 --2- 172.21.15.43:0/3009091305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a10105dd0 0x7f4a101061b0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.281+0000 7f4a148ba640 1 -- 172.21.15.43:0/3009091305 >> 172.21.15.43:0/3009091305 conn(0x7f4a10077c40 msgr2=0x7f4a10078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.281+0000 7f4a148ba640 1 -- 172.21.15.43:0/3009091305 shutdown_connections 2024-04-21T08:47:14.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.281+0000 7f4a148ba640 1 -- 172.21.15.43:0/3009091305 wait complete. 2024-04-21T08:47:14.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.282+0000 7f4a148ba640 1 Processor -- start 2024-04-21T08:47:14.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.282+0000 7f4a148ba640 1 -- start start 2024-04-21T08:47:14.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.282+0000 7f4a148ba640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a101010a0 0x7f4a1019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.282+0000 7f4a148ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a10101a40 0x7f4a1019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.282+0000 7f4a148ba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a10105dd0 0x7f4a10198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.283+0000 7f4a148ba640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4a10111150 con 0x7f4a10105dd0 2024-04-21T08:47:14.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.283+0000 7f4a148ba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a10110fd0 con 0x7f4a10101a40 2024-04-21T08:47:14.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.283+0000 7f4a148ba640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4a101112d0 con 0x7f4a101010a0 2024-04-21T08:47:14.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.283+0000 7f4a0ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a10105dd0 0x7f4a10198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.283+0000 7f4a0ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a10105dd0 0x7f4a10198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43650/0 (socket says 172.21.15.43:43650) 2024-04-21T08:47:14.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.283+0000 7f4a0ed9d640 1 -- 172.21.15.43:0/3783827607 learned_addr learned my addr 172.21.15.43:0/3783827607 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:14.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.283+0000 7f4a0e59c640 1 --2- 172.21.15.43:0/3783827607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a101010a0 0x7f4a1019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.283+0000 7f4a0dd9b640 1 --2- 172.21.15.43:0/3783827607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a10101a40 0x7f4a1019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.283+0000 7f4a0e59c640 1 -- 172.21.15.43:0/3783827607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a10101a40 msgr2=0x7f4a1019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.283+0000 7f4a0e59c640 1 --2- 172.21.15.43:0/3783827607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a10101a40 0x7f4a1019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.283+0000 7f4a0e59c640 1 -- 172.21.15.43:0/3783827607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a10105dd0 msgr2=0x7f4a10198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.283+0000 7f4a0e59c640 1 --2- 172.21.15.43:0/3783827607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a10105dd0 0x7f4a10198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.283+0000 7f4a0e59c640 1 -- 172.21.15.43:0/3783827607 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a10198d10 con 0x7f4a101010a0 2024-04-21T08:47:14.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.283+0000 7f4a0ed9d640 1 --2- 172.21.15.43:0/3783827607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a10105dd0 0x7f4a10198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:14.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.283+0000 7f4a0e59c640 1 --2- 172.21.15.43:0/3783827607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a101010a0 0x7f4a1019e3e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4a00036040 tx=0x7f4a000045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.283+0000 7f49ff7fe640 1 -- 172.21.15.43:0/3783827607 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00043020 con 0x7f4a101010a0 2024-04-21T08:47:14.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.283+0000 7f49ff7fe640 1 -- 172.21.15.43:0/3783827607 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a0002ca70 con 0x7f4a101010a0 2024-04-21T08:47:14.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.283+0000 7f4a148ba640 1 -- 172.21.15.43:0/3783827607 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a10198fa0 con 0x7f4a101010a0 2024-04-21T08:47:14.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.284+0000 7f49ff7fe640 1 -- 172.21.15.43:0/3783827607 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00003a50 con 0x7f4a101010a0 2024-04-21T08:47:14.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.284+0000 7f4a148ba640 1 -- 172.21.15.43:0/3783827607 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a101a52a0 con 0x7f4a101010a0 2024-04-21T08:47:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.286+0000 7f49ff7fe640 1 -- 172.21.15.43:0/3783827607 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4a00046020 con 0x7f4a101010a0 2024-04-21T08:47:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.286+0000 7f4a17587740 1 Processor -- start 2024-04-21T08:47:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.286+0000 7f4a17587740 1 -- start start 2024-04-21T08:47:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.286+0000 7f4a17587740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 0x55db36c8fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.286+0000 7f4a17587740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db36c902b0 0x55db36c98950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.286+0000 7f4a17587740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db36c98e90 0x55db36c9b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.286+0000 7f4a17587740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55db36bf01e0 con 0x55db36c8f990 2024-04-21T08:47:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.286+0000 7f4a17587740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55db36b8d390 con 0x55db36c98e90 2024-04-21T08:47:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.286+0000 7f4a17587740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55db36bec1a0 con 0x55db36c902b0 2024-04-21T08:47:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.287+0000 7f4a0ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 0x55db36c8fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.287+0000 7f4a0ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 0x55db36c8fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43666/0 (socket says 172.21.15.43:43666) 2024-04-21T08:47:14.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.287+0000 7f4a0ed9d640 1 -- 172.21.15.43:0/348682736 learned_addr learned my addr 172.21.15.43:0/348682736 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:14.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.287+0000 7f4a0e59c640 1 --2- 172.21.15.43:0/348682736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db36c902b0 0x55db36c98950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.287+0000 7f49ff7fe640 1 --2- 172.21.15.43:0/3783827607 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f49f80a3d20 0x7f49f80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.287+0000 7f4a0dd9b640 1 --2- 172.21.15.43:0/348682736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db36c98e90 0x55db36c9b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.287+0000 7f4a0ed9d640 1 -- 172.21.15.43:0/348682736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db36c902b0 msgr2=0x55db36c98950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:14.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.287+0000 7f4a0ed9d640 1 --2- 172.21.15.43:0/348682736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db36c902b0 0x55db36c98950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.287+0000 7f4a0ed9d640 1 -- 172.21.15.43:0/348682736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db36c98e90 msgr2=0x55db36c9b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:14.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.287+0000 7f4a0ed9d640 1 --2- 172.21.15.43:0/348682736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db36c98e90 0x55db36c9b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.287+0000 7f4a0ed9d640 1 -- 172.21.15.43:0/348682736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db369cc7d0 con 0x55db36c8f990 2024-04-21T08:47:14.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.287+0000 7f49ff7fe640 1 -- 172.21.15.43:0/3783827607 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f4a000f2a40 con 0x7f4a101010a0 2024-04-21T08:47:14.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.287+0000 7f4a0ed9d640 1 --2- 172.21.15.43:0/3783827607 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f49f80a3d20 0x7f49f80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.287+0000 7f4a0ed9d640 1 --2- 172.21.15.43:0/348682736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 0x55db36c8fd70 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f4a08011a30 tx=0x7f4a08011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ef5919a7ca843e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.287+0000 7f4a0ed9d640 1 --2- 172.21.15.43:0/3783827607 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f49f80a3d20 0x7f49f80a61e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4a08023000 tx=0x7f4a08024000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.287+0000 7f49fcff9640 1 -- 172.21.15.43:0/348682736 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a08017070 con 0x55db36c8f990 2024-04-21T08:47:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.288+0000 7f49fcff9640 1 -- 172.21.15.43:0/348682736 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a0801ccc0 con 0x55db36c8f990 2024-04-21T08:47:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.288+0000 7f49fcff9640 1 -- 172.21.15.43:0/348682736 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0800e3b0 con 0x55db36c8f990 2024-04-21T08:47:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.288+0000 7f4a17587740 1 -- 172.21.15.43:0/348682736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 msgr2=0x55db36c8fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.288+0000 7f4a17587740 1 --2- 172.21.15.43:0/348682736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 0x55db36c8fd70 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f4a08011a30 tx=0x7f4a08011ef0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.288+0000 7f4a17587740 1 -- 172.21.15.43:0/348682736 shutdown_connections 2024-04-21T08:47:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.288+0000 7f4a17587740 1 --2- 172.21.15.43:0/348682736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db36c98e90 0x55db36c9b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.288+0000 7f4a17587740 1 --2- 172.21.15.43:0/348682736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db36c902b0 0x55db36c98950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.288+0000 7f4a17587740 1 --2- 172.21.15.43:0/348682736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 0x55db36c8fd70 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.288+0000 7f4a17587740 1 -- 172.21.15.43:0/348682736 >> 172.21.15.43:0/348682736 conn(0x55db36c8a0c0 msgr2=0x55db36c8a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.288+0000 7f4a17587740 1 -- 172.21.15.43:0/348682736 shutdown_connections 2024-04-21T08:47:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.288+0000 7f4a17587740 1 -- 172.21.15.43:0/348682736 wait complete. 2024-04-21T08:47:14.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.289+0000 7f4a17587740 1 -- start start 2024-04-21T08:47:14.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.289+0000 7f4a17587740 1 Processor -- start 2024-04-21T08:47:14.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.289+0000 7f4a17587740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 0x55db36c98870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.289+0000 7f4a17587740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db36c902b0 0x55db36c95e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.289+0000 7f4a17587740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db36c98e90 0x55db36c963a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.289+0000 7f4a17587740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55db36c9bc80 con 0x55db36c8f990 2024-04-21T08:47:14.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.289+0000 7f4a17587740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55db36ba92f0 con 0x55db36c98e90 2024-04-21T08:47:14.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.289+0000 7f4a17587740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55db36c9be00 con 0x55db36c902b0 2024-04-21T08:47:14.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.289+0000 7f4a17587740 10 client.0 _ll_register_callbacks cb 0x55db36bb6580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:14.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.289+0000 7f4a0e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 0x55db36c98870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.289+0000 7f4a0e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 0x55db36c98870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43672/0 (socket says 172.21.15.43:43672) 2024-04-21T08:47:14.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.289+0000 7f4a0e59c640 1 -- 172.21.15.43:0/1171438975 learned_addr learned my addr 172.21.15.43:0/1171438975 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:14.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.289+0000 7f4a0ed9d640 1 --2- 172.21.15.43:0/1171438975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db36c98e90 0x55db36c963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.289+0000 7f4a0dd9b640 1 --2- 172.21.15.43:0/1171438975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db36c902b0 0x55db36c95e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.290+0000 7f4a0e59c640 1 -- 172.21.15.43:0/1171438975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db36c902b0 msgr2=0x55db36c95e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:14.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.290+0000 7f4a0e59c640 1 --2- 172.21.15.43:0/1171438975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db36c902b0 0x55db36c95e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.290+0000 7f4a0e59c640 1 -- 172.21.15.43:0/1171438975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db36c98e90 msgr2=0x55db36c963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:14.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.290+0000 7f4a0e59c640 1 --2- 172.21.15.43:0/1171438975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db36c98e90 0x55db36c963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.290+0000 7f4a0e59c640 1 -- 172.21.15.43:0/1171438975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db369023f0 con 0x55db36c8f990 2024-04-21T08:47:14.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.290+0000 7f4a0e59c640 1 --2- 172.21.15.43:0/1171438975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 0x55db36c98870 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f4a000ba3b0 tx=0x7f4a00005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5c584591de1a87 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.290+0000 7f49f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:14.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.290+0000 7f49f5ffb640 1 -- 172.21.15.43:0/1171438975 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a000f9070 con 0x55db36c8f990 2024-04-21T08:47:14.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.290+0000 7f49f5ffb640 1 -- 172.21.15.43:0/1171438975 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f4a000be930 con 0x55db36c8f990 2024-04-21T08:47:14.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.290+0000 7f4a17587740 1 -- 172.21.15.43:0/1171438975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a0801ccc0 con 0x55db36c8f990 2024-04-21T08:47:14.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.290+0000 7f49f5ffb640 1 -- 172.21.15.43:0/1171438975 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4a000be030 con 0x55db36c8f990 2024-04-21T08:47:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.290+0000 7f4a17587740 10 client.19344 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.290+0000 7f4a17587740 1 -- 172.21.15.43:0/1171438975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55db368c20f0 con 0x55db36c8f990 2024-04-21T08:47:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.290+0000 7f49f5ffb640 1 -- 172.21.15.43:0/1171438975 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a000d7e50 con 0x55db36c8f990 2024-04-21T08:47:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.290+0000 7f49f5ffb640 1 -- 172.21.15.43:0/1171438975 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4a000c6eb0 con 0x55db36c8f990 2024-04-21T08:47:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.291+0000 7f4a17587740 10 client.19344 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.291+0000 7f4a17587740 10 client.19344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.291+0000 7f4a17587740 20 client.19344 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.291+0000 7f4a17587740 1 --2- 172.21.15.43:0/1171438975 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55db36cad260 0x55db36caf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.291+0000 7f4a17587740 4 client.19344 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.291+0000 7f4a17587740 1 -- 172.21.15.43:0/1171438975 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db36bfdc00 con 0x55db36cad260 2024-04-21T08:47:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.291+0000 7f4a0dd9b640 1 --2- 172.21.15.43:0/1171438975 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55db36cad260 0x55db36caf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.291+0000 7f4a0dd9b640 1 --2- 172.21.15.43:0/1171438975 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55db36cad260 0x55db36caf6b0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b861c51f544805d server_cookie=8f36983255ad3b3 in_seq=0 out_seq=0 2024-04-21T08:47:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.291+0000 7f49f5ffb640 10 client.19344 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:14.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.292+0000 7f49f5ffb640 1 -- 172.21.15.43:0/1171438975 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55db36bfdc00 con 0x55db36cad260 2024-04-21T08:47:14.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.292+0000 7f49f5ffb640 10 client.19344 handle_command_reply: tid=0 2024-04-21T08:47:14.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.292+0000 7f4a17587740 1 client.19344 shutdown 2024-04-21T08:47:14.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.292+0000 7f4a17587740 1 -- 172.21.15.43:0/1171438975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 msgr2=0x55db36c98870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.292+0000 7f4a17587740 1 --2- 172.21.15.43:0/1171438975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 0x55db36c98870 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f4a000ba3b0 tx=0x7f4a00005c90 comp rx=0 tx=0).stop 2024-04-21T08:47:14.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.292+0000 7f4a17587740 1 -- 172.21.15.43:0/1171438975 shutdown_connections 2024-04-21T08:47:14.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.292+0000 7f4a17587740 1 --2- 172.21.15.43:0/1171438975 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55db36cad260 0x55db36caf6b0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.292+0000 7f4a17587740 1 --2- 172.21.15.43:0/1171438975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db36c98e90 0x55db36c963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.292+0000 7f4a17587740 1 --2- 172.21.15.43:0/1171438975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db36c902b0 0x55db36c95e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.292+0000 7f4a17587740 1 --2- 172.21.15.43:0/1171438975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 0x55db36c98870 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.292+0000 7f49f5ffb640 0 client.19344 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:14.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.292+0000 7f4a17587740 1 -- 172.21.15.43:0/1171438975 >> 172.21.15.43:0/1171438975 conn(0x55db36c8a0c0 msgr2=0x55db36c8be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.292+0000 7f4a17587740 1 -- 172.21.15.43:0/1171438975 shutdown_connections 2024-04-21T08:47:14.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.292+0000 7f4a17587740 1 -- 172.21.15.43:0/1171438975 wait complete. 2024-04-21T08:47:14.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.292+0000 7f4a17587740 20 client.19344 trim_cache size 0 max 16384 2024-04-21T08:47:14.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.310+0000 7f4a17587740 1 Processor -- start 2024-04-21T08:47:14.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.310+0000 7f4a17587740 1 -- start start 2024-04-21T08:47:14.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.310+0000 7f4a17587740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 0x55db36c84d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.310+0000 7f4a17587740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db36c902b0 0x55db36c98520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.310+0000 7f4a17587740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db36c98e90 0x55db36c8bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.310+0000 7f4a17587740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4a101ad2c0 con 0x55db36c8f990 2024-04-21T08:47:14.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.310+0000 7f4a17587740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a101aee00 con 0x55db36c902b0 2024-04-21T08:47:14.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.310+0000 7f4a17587740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4a101ad990 con 0x55db36c98e90 2024-04-21T08:47:14.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.310+0000 7f4a0e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 0x55db36c84d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.310+0000 7f4a0e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 0x55db36c84d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43682/0 (socket says 172.21.15.43:43682) 2024-04-21T08:47:14.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.310+0000 7f4a0e59c640 1 -- 172.21.15.43:0/3967384179 learned_addr learned my addr 172.21.15.43:0/3967384179 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:14.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.310+0000 7f4a0ed9d640 1 --2- 172.21.15.43:0/3967384179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db36c98e90 0x55db36c8bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.311+0000 7f4a0dd9b640 1 --2- 172.21.15.43:0/3967384179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db36c902b0 0x55db36c98520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.311+0000 7f4a0e59c640 1 -- 172.21.15.43:0/3967384179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db36c98e90 msgr2=0x55db36c8bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:14.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.311+0000 7f4a0e59c640 1 --2- 172.21.15.43:0/3967384179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db36c98e90 0x55db36c8bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.311+0000 7f4a0e59c640 1 -- 172.21.15.43:0/3967384179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db36c902b0 msgr2=0x55db36c98520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:14.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.311+0000 7f4a0e59c640 1 --2- 172.21.15.43:0/3967384179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db36c902b0 0x55db36c98520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.311+0000 7f4a0e59c640 1 -- 172.21.15.43:0/3967384179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db36c8ed10 con 0x55db36c8f990 2024-04-21T08:47:14.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.311+0000 7f4a0dd9b640 1 --2- 172.21.15.43:0/3967384179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db36c902b0 0x55db36c98520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:14.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.311+0000 7f4a0e59c640 1 --2- 172.21.15.43:0/3967384179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 0x55db36c84d70 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f4a000be660 tx=0x7f4a0002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46db05658f13cf1e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.311+0000 7f49f7fff640 1 -- 172.21.15.43:0/3967384179 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a000f9070 con 0x55db36c8f990 2024-04-21T08:47:14.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.311+0000 7f49f7fff640 1 -- 172.21.15.43:0/3967384179 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a0801ccc0 con 0x55db36c8f990 2024-04-21T08:47:14.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.311+0000 7f49f7fff640 1 -- 172.21.15.43:0/3967384179 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a000d4030 con 0x55db36c8f990 2024-04-21T08:47:14.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.311+0000 7f4a17587740 1 -- 172.21.15.43:0/3967384179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 msgr2=0x55db36c84d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.311+0000 7f4a17587740 1 --2- 172.21.15.43:0/3967384179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 0x55db36c84d70 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f4a000be660 tx=0x7f4a0002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:14.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.311+0000 7f4a17587740 1 -- 172.21.15.43:0/3967384179 shutdown_connections 2024-04-21T08:47:14.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.312+0000 7f4a17587740 1 --2- 172.21.15.43:0/3967384179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db36c98e90 0x55db36c8bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.312+0000 7f4a17587740 1 --2- 172.21.15.43:0/3967384179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db36c902b0 0x55db36c98520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.312+0000 7f4a17587740 1 --2- 172.21.15.43:0/3967384179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 0x55db36c84d70 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.312+0000 7f4a17587740 1 -- 172.21.15.43:0/3967384179 >> 172.21.15.43:0/3967384179 conn(0x55db36c8a0c0 msgr2=0x55db36c8b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.312+0000 7f4a17587740 1 -- 172.21.15.43:0/3967384179 shutdown_connections 2024-04-21T08:47:14.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.312+0000 7f4a17587740 1 -- 172.21.15.43:0/3967384179 wait complete. 2024-04-21T08:47:14.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.312+0000 7f4a17587740 1 -- start start 2024-04-21T08:47:14.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.312+0000 7f4a17587740 1 Processor -- start 2024-04-21T08:47:14.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.312+0000 7f4a17587740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 0x55db36ca5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.312+0000 7f4a17587740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db36c902b0 0x55db36ca6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.312+0000 7f4a17587740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db36c98e90 0x55db36d127a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.312+0000 7f4a17587740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4a101ae730 con 0x55db36c8f990 2024-04-21T08:47:14.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.312+0000 7f4a17587740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a101ae060 con 0x55db36c902b0 2024-04-21T08:47:14.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.312+0000 7f4a17587740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4a101ab2c0 con 0x55db36c98e90 2024-04-21T08:47:14.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.312+0000 7f4a17587740 10 client.0 _ll_register_callbacks cb 0x7f4a00005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:14.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.312+0000 7f4a0e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 0x55db36ca5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.312+0000 7f4a0e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 0x55db36ca5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43684/0 (socket says 172.21.15.43:43684) 2024-04-21T08:47:14.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.312+0000 7f4a0e59c640 1 -- 172.21.15.43:0/2877275364 learned_addr learned my addr 172.21.15.43:0/2877275364 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:14.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.312+0000 7f4a0ed9d640 1 --2- 172.21.15.43:0/2877275364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db36c98e90 0x55db36d127a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.313+0000 7f4a0dd9b640 1 --2- 172.21.15.43:0/2877275364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db36c902b0 0x55db36ca6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.313+0000 7f4a0e59c640 1 -- 172.21.15.43:0/2877275364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db36c98e90 msgr2=0x55db36d127a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:14.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.313+0000 7f4a0e59c640 1 --2- 172.21.15.43:0/2877275364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db36c98e90 0x55db36d127a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.313+0000 7f4a0e59c640 1 -- 172.21.15.43:0/2877275364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db36c902b0 msgr2=0x55db36ca6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:14.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.313+0000 7f4a0e59c640 1 --2- 172.21.15.43:0/2877275364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db36c902b0 0x55db36ca6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.313+0000 7f4a0e59c640 1 -- 172.21.15.43:0/2877275364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db36c9b280 con 0x55db36c8f990 2024-04-21T08:47:14.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.313+0000 7f4a0e59c640 1 --2- 172.21.15.43:0/2877275364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 0x55db36ca5c30 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f4a000c6e10 tx=0x7f4a00005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2f713002a39b98 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.313+0000 7f49f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:14.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.313+0000 7f49f5ffb640 1 -- 172.21.15.43:0/2877275364 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a000db070 con 0x55db36c8f990 2024-04-21T08:47:14.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.313+0000 7f49f5ffb640 1 -- 172.21.15.43:0/2877275364 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f4a000bdd20 con 0x55db36c8f990 2024-04-21T08:47:14.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.313+0000 7f4a17587740 1 -- 172.21.15.43:0/2877275364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a0801ccc0 con 0x55db36c8f990 2024-04-21T08:47:14.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.313+0000 7f49f5ffb640 1 -- 172.21.15.43:0/2877275364 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4a000d4070 con 0x55db36c8f990 2024-04-21T08:47:14.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.313+0000 7f4a17587740 10 client.19350 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:14.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.313+0000 7f4a17587740 1 -- 172.21.15.43:0/2877275364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55db36c9b280 con 0x55db36c8f990 2024-04-21T08:47:14.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.313+0000 7f49f5ffb640 1 -- 172.21.15.43:0/2877275364 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a0801ccc0 con 0x55db36c8f990 2024-04-21T08:47:14.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.313+0000 7f49f5ffb640 1 -- 172.21.15.43:0/2877275364 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4a000ba420 con 0x55db36c8f990 2024-04-21T08:47:14.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.313+0000 7f4a17587740 10 client.19350 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:14.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.313+0000 7f4a17587740 10 client.19350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:14.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.313+0000 7f4a17587740 20 client.19350 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:14.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.313+0000 7f4a17587740 1 --2- 172.21.15.43:0/2877275364 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55db36d13e20 0x55db36d16210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.313+0000 7f4a17587740 4 client.19350 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:47:14.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.313+0000 7f4a17587740 1 -- 172.21.15.43:0/2877275364 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55db36bf7a70 con 0x55db36d13e20 2024-04-21T08:47:14.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.314+0000 7f4a0dd9b640 1 --2- 172.21.15.43:0/2877275364 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55db36d13e20 0x55db36d16210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.314+0000 7f4a0dd9b640 1 --2- 172.21.15.43:0/2877275364 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55db36d13e20 0x55db36d16210 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a42c1dcc0818411 server_cookie=24fcb1affff9161d in_seq=0 out_seq=0 2024-04-21T08:47:14.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.314+0000 7f49f5ffb640 10 client.19350 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:14.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.314+0000 7f49f5ffb640 1 -- 172.21.15.43:0/2877275364 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55db36bf7a70 con 0x55db36d13e20 2024-04-21T08:47:14.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.314+0000 7f49f5ffb640 10 client.19350 handle_command_reply: tid=0 2024-04-21T08:47:14.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.314+0000 7f4a17587740 1 client.19350 shutdown 2024-04-21T08:47:14.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.314+0000 7f4a17587740 1 -- 172.21.15.43:0/2877275364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 msgr2=0x55db36ca5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.314+0000 7f4a17587740 1 --2- 172.21.15.43:0/2877275364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 0x55db36ca5c30 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f4a000c6e10 tx=0x7f4a00005c90 comp rx=0 tx=0).stop 2024-04-21T08:47:14.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.314+0000 7f4a17587740 1 -- 172.21.15.43:0/2877275364 shutdown_connections 2024-04-21T08:47:14.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.314+0000 7f4a17587740 1 --2- 172.21.15.43:0/2877275364 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55db36d13e20 0x55db36d16210 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.314+0000 7f4a17587740 1 --2- 172.21.15.43:0/2877275364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55db36c98e90 0x55db36d127a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.314+0000 7f4a17587740 1 --2- 172.21.15.43:0/2877275364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db36c902b0 0x55db36ca6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.314+0000 7f49f5ffb640 0 client.19350 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:14.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.314+0000 7f4a17587740 1 --2- 172.21.15.43:0/2877275364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db36c8f990 0x55db36ca5c30 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.314+0000 7f4a17587740 1 -- 172.21.15.43:0/2877275364 >> 172.21.15.43:0/2877275364 conn(0x55db36c8a0c0 msgr2=0x55db36c84bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.315+0000 7f4a17587740 1 -- 172.21.15.43:0/2877275364 shutdown_connections 2024-04-21T08:47:14.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.315+0000 7f4a17587740 1 -- 172.21.15.43:0/2877275364 wait complete. 2024-04-21T08:47:14.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.315+0000 7f4a17587740 20 client.19350 trim_cache size 0 max 16384 2024-04-21T08:47:14.321 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:47:14.321 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:47:14.321 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:47:14.321 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:47:14.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.315+0000 7f4a148ba640 1 -- 172.21.15.43:0/3783827607 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f49f80a3d20 msgr2=0x7f49f80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.315+0000 7f4a148ba640 1 --2- 172.21.15.43:0/3783827607 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f49f80a3d20 0x7f49f80a61e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4a08023000 tx=0x7f4a08024000 comp rx=0 tx=0).stop 2024-04-21T08:47:14.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.315+0000 7f4a148ba640 1 -- 172.21.15.43:0/3783827607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a101010a0 msgr2=0x7f4a1019e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.315+0000 7f4a148ba640 1 --2- 172.21.15.43:0/3783827607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a101010a0 0x7f4a1019e3e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4a00036040 tx=0x7f4a000045e0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.315+0000 7f4a148ba640 1 -- 172.21.15.43:0/3783827607 shutdown_connections 2024-04-21T08:47:14.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.315+0000 7f4a148ba640 1 --2- 172.21.15.43:0/3783827607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a10105dd0 0x7f4a10198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.315+0000 7f4a148ba640 1 --2- 172.21.15.43:0/3783827607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a10101a40 0x7f4a1019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.315+0000 7f4a148ba640 1 --2- 172.21.15.43:0/3783827607 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f49f80a3d20 0x7f49f80a61e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.315+0000 7f4a148ba640 1 --2- 172.21.15.43:0/3783827607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a101010a0 0x7f4a1019e3e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.315+0000 7f4a148ba640 1 -- 172.21.15.43:0/3783827607 >> 172.21.15.43:0/3783827607 conn(0x7f4a10077c40 msgr2=0x7f4a1006a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.315+0000 7f4a148ba640 1 -- 172.21.15.43:0/3783827607 shutdown_connections 2024-04-21T08:47:14.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.315+0000 7f4a148ba640 1 -- 172.21.15.43:0/3783827607 wait complete. 2024-04-21T08:47:14.326 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:47:14.326 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:47:14.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.429+0000 7f112a975640 1 -- 172.21.15.43:0/3536553675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11241021a0 msgr2=0x7f1124102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.429+0000 7f112a975640 1 --2- 172.21.15.43:0/3536553675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11241021a0 0x7f1124102580 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f1114009900 tx=0x7f111402bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:14.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.429+0000 7f112a975640 1 -- 172.21.15.43:0/3536553675 shutdown_connections 2024-04-21T08:47:14.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.429+0000 7f112a975640 1 --2- 172.21.15.43:0/3536553675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1124107270 0x7f112410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.429+0000 7f112a975640 1 --2- 172.21.15.43:0/3536553675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1124102b50 0x7f1124106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.429+0000 7f112a975640 1 --2- 172.21.15.43:0/3536553675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11241021a0 0x7f1124102580 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.429+0000 7f112a975640 1 -- 172.21.15.43:0/3536553675 >> 172.21.15.43:0/3536553675 conn(0x7f11240793a0 msgr2=0x7f11240797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.429+0000 7f112a975640 1 -- 172.21.15.43:0/3536553675 shutdown_connections 2024-04-21T08:47:14.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.429+0000 7f112a975640 1 -- 172.21.15.43:0/3536553675 wait complete. 2024-04-21T08:47:14.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.429+0000 7f112a975640 1 Processor -- start 2024-04-21T08:47:14.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.429+0000 7f112a975640 1 -- start start 2024-04-21T08:47:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.430+0000 7f112a975640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11241021a0 0x7f11241a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.430+0000 7f112a975640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1124107270 0x7f11241a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.430+0000 7f112a975640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f112419a6e0 0x7f112419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.430+0000 7f112a975640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11241121e0 con 0x7f11241021a0 2024-04-21T08:47:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.430+0000 7f112a975640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1124112060 con 0x7f1124107270 2024-04-21T08:47:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.430+0000 7f112a975640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1124112360 con 0x7f112419a6e0 2024-04-21T08:47:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.430+0000 7f1123fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11241021a0 0x7f11241a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.430+0000 7f1123fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11241021a0 0x7f11241a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43706/0 (socket says 172.21.15.43:43706) 2024-04-21T08:47:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.430+0000 7f1123fff640 1 -- 172.21.15.43:0/4287231420 learned_addr learned my addr 172.21.15.43:0/4287231420 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.430+0000 7f1128f12640 1 --2- 172.21.15.43:0/4287231420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f112419a6e0 0x7f112419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.430+0000 7f11237fe640 1 --2- 172.21.15.43:0/4287231420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1124107270 0x7f11241a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.430+0000 7f1123fff640 1 -- 172.21.15.43:0/4287231420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f112419a6e0 msgr2=0x7f112419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.430+0000 7f1123fff640 1 --2- 172.21.15.43:0/4287231420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f112419a6e0 0x7f112419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.430+0000 7f1123fff640 1 -- 172.21.15.43:0/4287231420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1124107270 msgr2=0x7f11241a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.430+0000 7f1123fff640 1 --2- 172.21.15.43:0/4287231420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1124107270 0x7f11241a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.430+0000 7f1123fff640 1 -- 172.21.15.43:0/4287231420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f112419b2e0 con 0x7f11241021a0 2024-04-21T08:47:14.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.430+0000 7f1128f12640 1 --2- 172.21.15.43:0/4287231420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f112419a6e0 0x7f112419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:14.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.430+0000 7f11237fe640 1 --2- 172.21.15.43:0/4287231420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1124107270 0x7f11241a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:14.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.430+0000 7f1123fff640 1 --2- 172.21.15.43:0/4287231420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11241021a0 0x7f11241a0620 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f1114005940 tx=0x7f111402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.431+0000 7f11217fa640 1 -- 172.21.15.43:0/4287231420 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1114043020 con 0x7f11241021a0 2024-04-21T08:47:14.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.431+0000 7f11217fa640 1 -- 172.21.15.43:0/4287231420 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f111402cbe0 con 0x7f11241021a0 2024-04-21T08:47:14.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.431+0000 7f112a975640 1 -- 172.21.15.43:0/4287231420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f112419b570 con 0x7f11241021a0 2024-04-21T08:47:14.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.431+0000 7f11217fa640 1 -- 172.21.15.43:0/4287231420 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1114003bf0 con 0x7f11241021a0 2024-04-21T08:47:14.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.431+0000 7f112a975640 1 -- 172.21.15.43:0/4287231420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11241a7760 con 0x7f11241021a0 2024-04-21T08:47:14.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.432+0000 7f112a975640 1 -- 172.21.15.43:0/4287231420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1124103670 con 0x7f11241021a0 2024-04-21T08:47:14.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.433+0000 7f11217fa640 1 -- 172.21.15.43:0/4287231420 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1114046020 con 0x7f11241021a0 2024-04-21T08:47:14.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.434+0000 7f11217fa640 1 --2- 172.21.15.43:0/4287231420 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f110c0a3da0 0x7f110c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.434+0000 7f11217fa640 1 -- 172.21.15.43:0/4287231420 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f11140f33a0 con 0x7f11241021a0 2024-04-21T08:47:14.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.434+0000 7f11237fe640 1 --2- 172.21.15.43:0/4287231420 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f110c0a3da0 0x7f110c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.434+0000 7f11237fe640 1 --2- 172.21.15.43:0/4287231420 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f110c0a3da0 0x7f110c0a6260 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f112419bf30 tx=0x7f111800a5c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.438+0000 7f11217fa640 1 -- 172.21.15.43:0/4287231420 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f111404d020 con 0x7f11241021a0 2024-04-21T08:47:14.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:14 smithi175 ceph-mon[31920]: pgmap v257: 129 pgs: 129 active+clean; 4.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 103 KiB/s wr, 27 op/s 2024-04-21T08:47:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:14 smithi112 ceph-mon[31085]: pgmap v257: 129 pgs: 129 active+clean; 4.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 103 KiB/s wr, 27 op/s 2024-04-21T08:47:14.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:14 smithi043 ceph-mon[27978]: pgmap v257: 129 pgs: 129 active+clean; 4.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 103 KiB/s wr, 27 op/s 2024-04-21T08:47:14.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.664+0000 7f112a975640 1 -- 172.21.15.43:0/4287231420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1124102bf0 con 0x7f11241021a0 2024-04-21T08:47:14.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.664+0000 7f11217fa640 1 -- 172.21.15.43:0/4287231420 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11935 (secure 0 0 0) 0x7f11140f0e20 con 0x7f11241021a0 2024-04-21T08:47:14.664 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:47:14.664 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:46:17.283806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27556":{"gid":27556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/229985237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":229985237},{"type":"v1","addr":"172.21.15.175:6837","nonce":229985237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24425,24412,24487,24527,24491]},"id":1}]} 2024-04-21T08:47:14.664 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-04-21T08:47:14.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.667+0000 7f112a975640 1 -- 172.21.15.43:0/4287231420 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f110c0a3da0 msgr2=0x7f110c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.667+0000 7f112a975640 1 --2- 172.21.15.43:0/4287231420 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f110c0a3da0 0x7f110c0a6260 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f112419bf30 tx=0x7f111800a5c0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.667+0000 7f112a975640 1 -- 172.21.15.43:0/4287231420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11241021a0 msgr2=0x7f11241a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.667+0000 7f112a975640 1 --2- 172.21.15.43:0/4287231420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11241021a0 0x7f11241a0620 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f1114005940 tx=0x7f111402c7e0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.667+0000 7f112a975640 1 -- 172.21.15.43:0/4287231420 shutdown_connections 2024-04-21T08:47:14.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.667+0000 7f112a975640 1 --2- 172.21.15.43:0/4287231420 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f110c0a3da0 0x7f110c0a6260 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.667+0000 7f112a975640 1 --2- 172.21.15.43:0/4287231420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f112419a6e0 0x7f112419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.667+0000 7f112a975640 1 --2- 172.21.15.43:0/4287231420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1124107270 0x7f11241a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.667+0000 7f112a975640 1 --2- 172.21.15.43:0/4287231420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11241021a0 0x7f11241a0620 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.667+0000 7f112a975640 1 -- 172.21.15.43:0/4287231420 >> 172.21.15.43:0/4287231420 conn(0x7f11240793a0 msgr2=0x7f11241000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.667+0000 7f112a975640 1 -- 172.21.15.43:0/4287231420 shutdown_connections 2024-04-21T08:47:14.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.667+0000 7f112a975640 1 -- 172.21.15.43:0/4287231420 wait complete. 2024-04-21T08:47:14.677 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:47:14.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.783+0000 7f79c7043640 1 -- 172.21.15.43:0/3869931850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79c0107270 msgr2=0x7f79c010ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.783+0000 7f79c7043640 1 --2- 172.21.15.43:0/3869931850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79c0107270 0x7f79c010ee40 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009ca0 tx=0x7f79b802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.783+0000 7f79c7043640 1 -- 172.21.15.43:0/3869931850 shutdown_connections 2024-04-21T08:47:14.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.783+0000 7f79c7043640 1 --2- 172.21.15.43:0/3869931850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79c0107270 0x7f79c010ee40 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.783+0000 7f79c7043640 1 --2- 172.21.15.43:0/3869931850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79c006b1f0 0x7f79c0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.783+0000 7f79c7043640 1 --2- 172.21.15.43:0/3869931850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79c006a840 0x7f79c006ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.783+0000 7f79c7043640 1 -- 172.21.15.43:0/3869931850 >> 172.21.15.43:0/3869931850 conn(0x7f79c0077eb0 msgr2=0x7f79c00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.783+0000 7f79c7043640 1 -- 172.21.15.43:0/3869931850 shutdown_connections 2024-04-21T08:47:14.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.783+0000 7f79c7043640 1 -- 172.21.15.43:0/3869931850 wait complete. 2024-04-21T08:47:14.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.783+0000 7f79c7043640 1 Processor -- start 2024-04-21T08:47:14.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.783+0000 7f79c7043640 1 -- start start 2024-04-21T08:47:14.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.783+0000 7f79c7043640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79c006a840 0x7f79c01986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.783+0000 7f79c7043640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79c0107270 0x7f79c0198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.783+0000 7f79c7043640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79c019cfb0 0x7f79c019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.784+0000 7f79c7043640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f79c01121d0 con 0x7f79c019cfb0 2024-04-21T08:47:14.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.784+0000 7f79c7043640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f79c0112050 con 0x7f79c0107270 2024-04-21T08:47:14.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.784+0000 7f79c7043640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f79c0112350 con 0x7f79c006a840 2024-04-21T08:47:14.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.784+0000 7f79c55e0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79c019cfb0 0x7f79c019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.784+0000 7f79c55e0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79c019cfb0 0x7f79c019d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43714/0 (socket says 172.21.15.43:43714) 2024-04-21T08:47:14.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.784+0000 7f79c55e0640 1 -- 172.21.15.43:0/2801732584 learned_addr learned my addr 172.21.15.43:0/2801732584 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:14.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.784+0000 7f79bffff640 1 --2- 172.21.15.43:0/2801732584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79c0107270 0x7f79c0198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.784+0000 7f79c4ddf640 1 --2- 172.21.15.43:0/2801732584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79c006a840 0x7f79c01986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.784+0000 7f79c55e0640 1 -- 172.21.15.43:0/2801732584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79c006a840 msgr2=0x7f79c01986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.784+0000 7f79c55e0640 1 --2- 172.21.15.43:0/2801732584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79c006a840 0x7f79c01986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.784+0000 7f79c55e0640 1 -- 172.21.15.43:0/2801732584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79c0107270 msgr2=0x7f79c0198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.784+0000 7f79c55e0640 1 --2- 172.21.15.43:0/2801732584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79c0107270 0x7f79c0198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.784+0000 7f79c55e0640 1 -- 172.21.15.43:0/2801732584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c019daa0 con 0x7f79c019cfb0 2024-04-21T08:47:14.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.784+0000 7f79c55e0640 1 --2- 172.21.15.43:0/2801732584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79c019cfb0 0x7f79c019d460 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f79b8005e30 tx=0x7f79b8034650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.784+0000 7f79bdffb640 1 -- 172.21.15.43:0/2801732584 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8041020 con 0x7f79c019cfb0 2024-04-21T08:47:14.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.784+0000 7f79c7043640 1 -- 172.21.15.43:0/2801732584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c019dd30 con 0x7f79c019cfb0 2024-04-21T08:47:14.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.784+0000 7f79bdffb640 1 -- 172.21.15.43:0/2801732584 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79b8032970 con 0x7f79c019cfb0 2024-04-21T08:47:14.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.784+0000 7f79bdffb640 1 -- 172.21.15.43:0/2801732584 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8039040 con 0x7f79c019cfb0 2024-04-21T08:47:14.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.785+0000 7f79c7043640 1 -- 172.21.15.43:0/2801732584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c01a5500 con 0x7f79c019cfb0 2024-04-21T08:47:14.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.787+0000 7f79bdffb640 1 -- 172.21.15.43:0/2801732584 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f79b8044020 con 0x7f79c019cfb0 2024-04-21T08:47:14.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.787+0000 7f79c9c35740 1 Processor -- start 2024-04-21T08:47:14.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.787+0000 7f79c9c35740 1 -- start start 2024-04-21T08:47:14.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.788+0000 7f79c9c35740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0975384d0 0x55c0975388b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.788+0000 7f79c9c35740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c097538e80 0x55c097541430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.788+0000 7f79c9c35740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c097541a00 0x55c097543e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.788+0000 7f79c9c35740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c09749d1f0 con 0x55c097538e80 2024-04-21T08:47:14.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.788+0000 7f79c9c35740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c09743a4b0 con 0x55c097541a00 2024-04-21T08:47:14.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.788+0000 7f79c9c35740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c0974991b0 con 0x55c0975384d0 2024-04-21T08:47:14.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.788+0000 7f79c4ddf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c097538e80 0x55c097541430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.788+0000 7f79bdffb640 1 --2- 172.21.15.43:0/2801732584 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f79b00a3d20 0x7f79b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.788+0000 7f79c4ddf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c097538e80 0x55c097541430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43718/0 (socket says 172.21.15.43:43718) 2024-04-21T08:47:14.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.788+0000 7f79c4ddf640 1 -- 172.21.15.43:0/319605558 learned_addr learned my addr 172.21.15.43:0/319605558 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:14.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.788+0000 7f79bffff640 1 --2- 172.21.15.43:0/319605558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0975384d0 0x55c0975388b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.788+0000 7f79c55e0640 1 --2- 172.21.15.43:0/2801732584 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f79b00a3d20 0x7f79b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.788+0000 7f79bffff640 1 --2- 172.21.15.43:0/319605558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c097541a00 0x55c097543e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.788+0000 7f79bdffb640 1 -- 172.21.15.43:0/2801732584 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f79b80f2760 con 0x7f79c019cfb0 2024-04-21T08:47:14.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.788+0000 7f79c4ddf640 1 -- 172.21.15.43:0/319605558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0975384d0 msgr2=0x55c0975388b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:14.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.788+0000 7f79c4ddf640 1 --2- 172.21.15.43:0/319605558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0975384d0 0x55c0975388b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.788+0000 7f79c4ddf640 1 -- 172.21.15.43:0/319605558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c097541a00 msgr2=0x55c097543e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:14.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.788+0000 7f79c4ddf640 1 --2- 172.21.15.43:0/319605558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c097541a00 0x55c097543e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.788+0000 7f79c4ddf640 1 -- 172.21.15.43:0/319605558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c09742c1b0 con 0x55c097538e80 2024-04-21T08:47:14.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.788+0000 7f79c4ddf640 1 --2- 172.21.15.43:0/319605558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c097538e80 0x55c097541430 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009d20 tx=0x7f79b4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a963ab6c778acdce server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.788+0000 7f79c55e0640 1 --2- 172.21.15.43:0/2801732584 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f79b00a3d20 0x7f79b00a61e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f79b80be090 tx=0x7f79b808cd40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.788+0000 7f79aaffd640 1 -- 172.21.15.43:0/319605558 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b401e070 con 0x55c097538e80 2024-04-21T08:47:14.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.789+0000 7f79aaffd640 1 -- 172.21.15.43:0/319605558 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79b4019ae0 con 0x55c097538e80 2024-04-21T08:47:14.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.789+0000 7f79aaffd640 1 -- 172.21.15.43:0/319605558 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b400ea80 con 0x55c097538e80 2024-04-21T08:47:14.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.789+0000 7f79c9c35740 1 -- 172.21.15.43:0/319605558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c097538e80 msgr2=0x55c097541430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.789+0000 7f79c9c35740 1 --2- 172.21.15.43:0/319605558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c097538e80 0x55c097541430 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009d20 tx=0x7f79b4011810 comp rx=0 tx=0).stop 2024-04-21T08:47:14.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.789+0000 7f79c9c35740 1 -- 172.21.15.43:0/319605558 shutdown_connections 2024-04-21T08:47:14.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.789+0000 7f79c9c35740 1 --2- 172.21.15.43:0/319605558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c097541a00 0x55c097543e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.789+0000 7f79c9c35740 1 --2- 172.21.15.43:0/319605558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c097538e80 0x55c097541430 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.789+0000 7f79c9c35740 1 --2- 172.21.15.43:0/319605558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0975384d0 0x55c0975388b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.789+0000 7f79c9c35740 1 -- 172.21.15.43:0/319605558 >> 172.21.15.43:0/319605558 conn(0x55c097532ba0 msgr2=0x55c097532f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.789+0000 7f79c9c35740 1 -- 172.21.15.43:0/319605558 shutdown_connections 2024-04-21T08:47:14.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.789+0000 7f79c9c35740 1 -- 172.21.15.43:0/319605558 wait complete. 2024-04-21T08:47:14.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.790+0000 7f79c9c35740 1 -- start start 2024-04-21T08:47:14.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.790+0000 7f79c9c35740 1 Processor -- start 2024-04-21T08:47:14.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.790+0000 7f79c9c35740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0975384d0 0x55c097541320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.790+0000 7f79c9c35740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c097538e80 0x55c09753e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.790+0000 7f79c9c35740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c097541a00 0x55c09753ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.790+0000 7f79c9c35740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c097544970 con 0x55c097538e80 2024-04-21T08:47:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.790+0000 7f79c9c35740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c097456500 con 0x55c097541a00 2024-04-21T08:47:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.790+0000 7f79c9c35740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c097544af0 con 0x55c0975384d0 2024-04-21T08:47:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.790+0000 7f79c9c35740 10 client.0 _ll_register_callbacks cb 0x55c097424b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.790+0000 7f79c4ddf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c097538e80 0x55c09753e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.790+0000 7f79c4ddf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c097538e80 0x55c09753e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43734/0 (socket says 172.21.15.43:43734) 2024-04-21T08:47:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.790+0000 7f79c4ddf640 1 -- 172.21.15.43:0/727301443 learned_addr learned my addr 172.21.15.43:0/727301443 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.790+0000 7f79bffff640 1 --2- 172.21.15.43:0/727301443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0975384d0 0x55c097541320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.790+0000 7f79bffff640 1 --2- 172.21.15.43:0/727301443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c097541a00 0x55c09753ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.790+0000 7f79c4ddf640 1 -- 172.21.15.43:0/727301443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0975384d0 msgr2=0x55c097541320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.790+0000 7f79c4ddf640 1 --2- 172.21.15.43:0/727301443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0975384d0 0x55c097541320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.790+0000 7f79c4ddf640 1 -- 172.21.15.43:0/727301443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c097541a00 msgr2=0x55c09753ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.790+0000 7f79c4ddf640 1 --2- 172.21.15.43:0/727301443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c097541a00 0x55c09753ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.790+0000 7f79c4ddf640 1 -- 172.21.15.43:0/727301443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0971af3f0 con 0x55c097538e80 2024-04-21T08:47:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.790+0000 7f79bffff640 1 --2- 172.21.15.43:0/727301443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0975384d0 0x55c097541320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.791+0000 7f79c4ddf640 1 --2- 172.21.15.43:0/727301443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c097538e80 0x55c09753e6b0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f79b4005890 tx=0x7f79b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8e09990dfb248d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.791+0000 7f799ffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.791+0000 7f799ffff640 1 -- 172.21.15.43:0/727301443 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4027020 con 0x55c097538e80 2024-04-21T08:47:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.791+0000 7f799ffff640 1 -- 172.21.15.43:0/727301443 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f79b4019630 con 0x55c097538e80 2024-04-21T08:47:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.791+0000 7f79c9c35740 1 -- 172.21.15.43:0/727301443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79b4019ae0 con 0x55c097538e80 2024-04-21T08:47:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.791+0000 7f799ffff640 1 -- 172.21.15.43:0/727301443 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f79b4032050 con 0x55c097538e80 2024-04-21T08:47:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.791+0000 7f79c9c35740 10 client.19368 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.791+0000 7f79c9c35740 1 -- 172.21.15.43:0/727301443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c0972797d0 con 0x55c097538e80 2024-04-21T08:47:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.791+0000 7f799ffff640 1 -- 172.21.15.43:0/727301443 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79b4019ae0 con 0x55c097538e80 2024-04-21T08:47:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.791+0000 7f799ffff640 1 -- 172.21.15.43:0/727301443 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f79b400ea70 con 0x55c097538e80 2024-04-21T08:47:14.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.791+0000 7f79c9c35740 10 client.19368 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:14.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.791+0000 7f79c9c35740 10 client.19368 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:47:14.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.791+0000 7f79c9c35740 20 client.19368 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:14.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.791+0000 7f79c9c35740 1 --2- 172.21.15.43:0/727301443 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55c097551bb0 0x55c097553fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.791+0000 7f79c9c35740 4 client.19368 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:14.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.791+0000 7f79c9c35740 1 -- 172.21.15.43:0/727301443 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c09743c930 con 0x55c097551bb0 2024-04-21T08:47:14.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.792+0000 7f79bffff640 1 --2- 172.21.15.43:0/727301443 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55c097551bb0 0x55c097553fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.792+0000 7f79bffff640 1 --2- 172.21.15.43:0/727301443 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55c097551bb0 0x55c097553fa0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=30e4530ff881e813 server_cookie=ec2ea760a52687cb in_seq=0 out_seq=0 2024-04-21T08:47:14.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.792+0000 7f799ffff640 10 client.19368 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:47:14.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.793+0000 7f799ffff640 1 -- 172.21.15.43:0/727301443 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c09743c930 con 0x55c097551bb0 2024-04-21T08:47:14.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.793+0000 7f799ffff640 10 client.19368 handle_command_reply: tid=0 2024-04-21T08:47:14.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.793+0000 7f79c9c35740 1 client.19368 shutdown 2024-04-21T08:47:14.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.793+0000 7f79c9c35740 1 -- 172.21.15.43:0/727301443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c097538e80 msgr2=0x55c09753e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.793+0000 7f79c9c35740 1 --2- 172.21.15.43:0/727301443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c097538e80 0x55c09753e6b0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f79b4005890 tx=0x7f79b4004270 comp rx=0 tx=0).stop 2024-04-21T08:47:14.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.793+0000 7f79c9c35740 1 -- 172.21.15.43:0/727301443 shutdown_connections 2024-04-21T08:47:14.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.793+0000 7f79c9c35740 1 --2- 172.21.15.43:0/727301443 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55c097551bb0 0x55c097553fa0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.793+0000 7f79c9c35740 1 --2- 172.21.15.43:0/727301443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c097541a00 0x55c09753ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.793+0000 7f799ffff640 0 client.19368 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:47:14.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.793+0000 7f79c9c35740 1 --2- 172.21.15.43:0/727301443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c097538e80 0x55c09753e6b0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.793+0000 7f79c9c35740 1 --2- 172.21.15.43:0/727301443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c0975384d0 0x55c097541320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.793+0000 7f79c9c35740 1 -- 172.21.15.43:0/727301443 >> 172.21.15.43:0/727301443 conn(0x55c097532ba0 msgr2=0x55c0975348f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.793+0000 7f79c9c35740 1 -- 172.21.15.43:0/727301443 shutdown_connections 2024-04-21T08:47:14.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.793+0000 7f79c9c35740 1 -- 172.21.15.43:0/727301443 wait complete. 2024-04-21T08:47:14.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.793+0000 7f79c9c35740 20 client.19368 trim_cache size 0 max 16384 2024-04-21T08:47:14.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.807+0000 7f79c9c35740 1 Processor -- start 2024-04-21T08:47:14.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.807+0000 7f79c9c35740 1 -- start start 2024-04-21T08:47:14.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.807+0000 7f79c9c35740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0975384d0 0x55c09754d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.807+0000 7f79c9c35740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c097538e80 0x55c097540de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.807+0000 7f79c9c35740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c097541a00 0x55c097541320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.807+0000 7f79c9c35740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f79c01af530 con 0x55c097538e80 2024-04-21T08:47:14.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.807+0000 7f79c9c35740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f79c01b1070 con 0x55c0975384d0 2024-04-21T08:47:14.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.807+0000 7f79c9c35740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f79c01afc00 con 0x55c097541a00 2024-04-21T08:47:14.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.807+0000 7f79c4ddf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c097538e80 0x55c097540de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.807+0000 7f79c4ddf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c097538e80 0x55c097540de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43750/0 (socket says 172.21.15.43:43750) 2024-04-21T08:47:14.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.807+0000 7f79c4ddf640 1 -- 172.21.15.43:0/1909452432 learned_addr learned my addr 172.21.15.43:0/1909452432 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:14.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.807+0000 7f79bffff640 1 --2- 172.21.15.43:0/1909452432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0975384d0 0x55c09754d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.807+0000 7f79bffff640 1 --2- 172.21.15.43:0/1909452432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c097541a00 0x55c097541320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.807+0000 7f79c4ddf640 1 -- 172.21.15.43:0/1909452432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c097541a00 msgr2=0x55c097541320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:14.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.807+0000 7f79c4ddf640 1 --2- 172.21.15.43:0/1909452432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c097541a00 0x55c097541320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.807+0000 7f79c4ddf640 1 -- 172.21.15.43:0/1909452432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0975384d0 msgr2=0x55c09754d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:14.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.807+0000 7f79c4ddf640 1 --2- 172.21.15.43:0/1909452432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0975384d0 0x55c09754d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.807+0000 7f79c4ddf640 1 -- 172.21.15.43:0/1909452432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c09753efc0 con 0x55c097538e80 2024-04-21T08:47:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.808+0000 7f79c4ddf640 1 --2- 172.21.15.43:0/1909452432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c097538e80 0x55c097540de0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f79b401c040 tx=0x7f79b4009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5231c54a3a5981f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.808+0000 7f79aa7fc640 1 -- 172.21.15.43:0/1909452432 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4027070 con 0x55c097538e80 2024-04-21T08:47:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.808+0000 7f79aa7fc640 1 -- 172.21.15.43:0/1909452432 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79b4015070 con 0x55c097538e80 2024-04-21T08:47:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.808+0000 7f79aa7fc640 1 -- 172.21.15.43:0/1909452432 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b401e070 con 0x55c097538e80 2024-04-21T08:47:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.808+0000 7f79c9c35740 1 -- 172.21.15.43:0/1909452432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c097538e80 msgr2=0x55c097540de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.808+0000 7f79c9c35740 1 --2- 172.21.15.43:0/1909452432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c097538e80 0x55c097540de0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f79b401c040 tx=0x7f79b4009fa0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.808+0000 7f79c9c35740 1 -- 172.21.15.43:0/1909452432 shutdown_connections 2024-04-21T08:47:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.808+0000 7f79c9c35740 1 --2- 172.21.15.43:0/1909452432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c097541a00 0x55c097541320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.808+0000 7f79c9c35740 1 --2- 172.21.15.43:0/1909452432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c097538e80 0x55c097540de0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.808+0000 7f79c9c35740 1 --2- 172.21.15.43:0/1909452432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0975384d0 0x55c09754d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.808+0000 7f79c9c35740 1 -- 172.21.15.43:0/1909452432 >> 172.21.15.43:0/1909452432 conn(0x55c097532ba0 msgr2=0x55c097539c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.809+0000 7f79c9c35740 1 -- 172.21.15.43:0/1909452432 shutdown_connections 2024-04-21T08:47:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.809+0000 7f79c9c35740 1 -- 172.21.15.43:0/1909452432 wait complete. 2024-04-21T08:47:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.809+0000 7f79c9c35740 1 -- start start 2024-04-21T08:47:14.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.809+0000 7f79c9c35740 1 Processor -- start 2024-04-21T08:47:14.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.809+0000 7f79c9c35740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0975384d0 0x55c09754a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.809+0000 7f79c9c35740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c097538e80 0x55c09753b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.809+0000 7f79c9c35740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c097541a00 0x55c09753b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.809+0000 7f79c9c35740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f79c01b09a0 con 0x55c0975384d0 2024-04-21T08:47:14.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.809+0000 7f79c9c35740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f79c01b02d0 con 0x55c097541a00 2024-04-21T08:47:14.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.809+0000 7f79c9c35740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f79c01ad530 con 0x55c097538e80 2024-04-21T08:47:14.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.809+0000 7f79c9c35740 10 client.0 _ll_register_callbacks cb 0x7f79b4009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:14.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.809+0000 7f79bffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0975384d0 0x55c09754a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.809+0000 7f79bffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0975384d0 0x55c09754a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43762/0 (socket says 172.21.15.43:43762) 2024-04-21T08:47:14.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.809+0000 7f79bffff640 1 -- 172.21.15.43:0/1687473823 learned_addr learned my addr 172.21.15.43:0/1687473823 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:14.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.810+0000 7f79c4ddf640 1 --2- 172.21.15.43:0/1687473823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c097538e80 0x55c09753b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.810+0000 7f79bffff640 1 --2- 172.21.15.43:0/1687473823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c097541a00 0x55c09753b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.810+0000 7f79bffff640 1 -- 172.21.15.43:0/1687473823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c097538e80 msgr2=0x55c09753b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:14.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.810+0000 7f79bffff640 1 --2- 172.21.15.43:0/1687473823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c097538e80 0x55c09753b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.810+0000 7f79bffff640 1 -- 172.21.15.43:0/1687473823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c097541a00 msgr2=0x55c09753b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:14.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.810+0000 7f79bffff640 1 --2- 172.21.15.43:0/1687473823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c097541a00 0x55c09753b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.810+0000 7f79bffff640 1 -- 172.21.15.43:0/1687473823 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c09753d660 con 0x55c0975384d0 2024-04-21T08:47:14.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.810+0000 7f79c4ddf640 1 --2- 172.21.15.43:0/1687473823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c097538e80 0x55c09753b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:14.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.810+0000 7f79bffff640 1 --2- 172.21.15.43:0/1687473823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0975384d0 0x55c09754a090 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f79c01087a0 tx=0x7f79ac023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75b27be85b3b804 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.810+0000 7f799ffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:14.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.810+0000 7f799ffff640 1 -- 172.21.15.43:0/1687473823 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac005020 con 0x55c0975384d0 2024-04-21T08:47:14.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.810+0000 7f799ffff640 1 -- 172.21.15.43:0/1687473823 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f79ac01a040 con 0x55c0975384d0 2024-04-21T08:47:14.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.810+0000 7f79c9c35740 1 -- 172.21.15.43:0/1687473823 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79b4015070 con 0x55c0975384d0 2024-04-21T08:47:14.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.810+0000 7f799ffff640 1 -- 172.21.15.43:0/1687473823 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f79ac032050 con 0x55c0975384d0 2024-04-21T08:47:14.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.810+0000 7f79c9c35740 10 client.19374 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:14.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.810+0000 7f79c9c35740 1 -- 172.21.15.43:0/1687473823 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c0975b7da0 con 0x55c0975384d0 2024-04-21T08:47:14.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.810+0000 7f799ffff640 1 -- 172.21.15.43:0/1687473823 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79b4015070 con 0x55c0975384d0 2024-04-21T08:47:14.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.811+0000 7f799ffff640 1 -- 172.21.15.43:0/1687473823 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f79ac01d5b0 con 0x55c0975384d0 2024-04-21T08:47:14.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.811+0000 7f79c9c35740 10 client.19374 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:14.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.811+0000 7f79c9c35740 10 client.19374 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:47:14.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.811+0000 7f79c9c35740 20 client.19374 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:14.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.811+0000 7f79c9c35740 1 --2- 172.21.15.43:0/1687473823 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55c0975b8520 0x55c0975ba940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.811+0000 7f79c9c35740 4 client.19374 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:14.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.811+0000 7f79c9c35740 1 -- 172.21.15.43:0/1687473823 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0974aac10 con 0x55c0975b8520 2024-04-21T08:47:14.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.811+0000 7f79c4ddf640 1 --2- 172.21.15.43:0/1687473823 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55c0975b8520 0x55c0975ba940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.811+0000 7f79c4ddf640 1 --2- 172.21.15.43:0/1687473823 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55c0975b8520 0x55c0975ba940 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d48a69f1c43ce075 server_cookie=686454a304519371 in_seq=0 out_seq=0 2024-04-21T08:47:14.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.811+0000 7f799ffff640 10 client.19374 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:47:14.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.811+0000 7f799ffff640 1 -- 172.21.15.43:0/1687473823 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0974aac10 con 0x55c0975b8520 2024-04-21T08:47:14.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.811+0000 7f799ffff640 10 client.19374 handle_command_reply: tid=0 2024-04-21T08:47:14.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.811+0000 7f79c9c35740 1 client.19374 shutdown 2024-04-21T08:47:14.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.812+0000 7f79c9c35740 1 -- 172.21.15.43:0/1687473823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0975384d0 msgr2=0x55c09754a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.812+0000 7f79c9c35740 1 --2- 172.21.15.43:0/1687473823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0975384d0 0x55c09754a090 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f79c01087a0 tx=0x7f79ac023040 comp rx=0 tx=0).stop 2024-04-21T08:47:14.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.812+0000 7f79c9c35740 1 -- 172.21.15.43:0/1687473823 shutdown_connections 2024-04-21T08:47:14.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.812+0000 7f79c9c35740 1 --2- 172.21.15.43:0/1687473823 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55c0975b8520 0x55c0975ba940 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.812+0000 7f79c9c35740 1 --2- 172.21.15.43:0/1687473823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c097541a00 0x55c09753b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.812+0000 7f799ffff640 0 client.19374 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:47:14.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.812+0000 7f79c9c35740 1 --2- 172.21.15.43:0/1687473823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c097538e80 0x55c09753b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.812+0000 7f79c9c35740 1 --2- 172.21.15.43:0/1687473823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c0975384d0 0x55c09754a090 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.812+0000 7f79c9c35740 1 -- 172.21.15.43:0/1687473823 >> 172.21.15.43:0/1687473823 conn(0x55c097532ba0 msgr2=0x55c097531370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.812+0000 7f79c9c35740 1 -- 172.21.15.43:0/1687473823 shutdown_connections 2024-04-21T08:47:14.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.812+0000 7f79c9c35740 1 -- 172.21.15.43:0/1687473823 wait complete. 2024-04-21T08:47:14.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.812+0000 7f79c9c35740 20 client.19374 trim_cache size 0 max 16384 2024-04-21T08:47:14.819 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:47:14.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.812+0000 7f79c7043640 1 -- 172.21.15.43:0/2801732584 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f79b00a3d20 msgr2=0x7f79b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.812+0000 7f79c7043640 1 --2- 172.21.15.43:0/2801732584 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f79b00a3d20 0x7f79b00a61e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f79b80be090 tx=0x7f79b808cd40 comp rx=0 tx=0).stop 2024-04-21T08:47:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.812+0000 7f79c7043640 1 -- 172.21.15.43:0/2801732584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79c019cfb0 msgr2=0x7f79c019d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.812+0000 7f79c7043640 1 --2- 172.21.15.43:0/2801732584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79c019cfb0 0x7f79c019d460 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f79b8005e30 tx=0x7f79b8034650 comp rx=0 tx=0).stop 2024-04-21T08:47:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.812+0000 7f79c7043640 1 -- 172.21.15.43:0/2801732584 shutdown_connections 2024-04-21T08:47:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.812+0000 7f79c7043640 1 --2- 172.21.15.43:0/2801732584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79c019cfb0 0x7f79c019d460 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.812+0000 7f79c7043640 1 --2- 172.21.15.43:0/2801732584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79c0107270 0x7f79c0198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.812+0000 7f79c7043640 1 --2- 172.21.15.43:0/2801732584 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f79b00a3d20 0x7f79b00a61e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.812+0000 7f79c7043640 1 --2- 172.21.15.43:0/2801732584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79c006a840 0x7f79c01986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.812+0000 7f79c7043640 1 -- 172.21.15.43:0/2801732584 >> 172.21.15.43:0/2801732584 conn(0x7f79c0077eb0 msgr2=0x7f79c00fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.813+0000 7f79c7043640 1 -- 172.21.15.43:0/2801732584 shutdown_connections 2024-04-21T08:47:14.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.813+0000 7f79c7043640 1 -- 172.21.15.43:0/2801732584 wait complete. 2024-04-21T08:47:14.823 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:47:14.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.927+0000 7f5b4c575640 1 -- 172.21.15.43:0/2079581332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b44101a50 msgr2=0x7f5b44105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.927+0000 7f5b4c575640 1 --2- 172.21.15.43:0/2079581332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b44101a50 0x7f5b44105a90 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009900 tx=0x7f5b3402bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:14.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.927+0000 7f5b4c575640 1 -- 172.21.15.43:0/2079581332 shutdown_connections 2024-04-21T08:47:14.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.927+0000 7f5b4c575640 1 --2- 172.21.15.43:0/2079581332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b44106170 0x7f5b4410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.927+0000 7f5b4c575640 1 --2- 172.21.15.43:0/2079581332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b44101a50 0x7f5b44105a90 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.927+0000 7f5b4c575640 1 --2- 172.21.15.43:0/2079581332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b441010a0 0x7f5b44101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.927+0000 7f5b4c575640 1 -- 172.21.15.43:0/2079581332 >> 172.21.15.43:0/2079581332 conn(0x7f5b44077eb0 msgr2=0x7f5b440782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.927+0000 7f5b4c575640 1 -- 172.21.15.43:0/2079581332 shutdown_connections 2024-04-21T08:47:14.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.927+0000 7f5b4c575640 1 -- 172.21.15.43:0/2079581332 wait complete. 2024-04-21T08:47:14.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.928+0000 7f5b4c575640 1 Processor -- start 2024-04-21T08:47:14.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.928+0000 7f5b4c575640 1 -- start start 2024-04-21T08:47:14.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.928+0000 7f5b4c575640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b441010a0 0x7f5b4419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.928+0000 7f5b4c575640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b44106170 0x7f5b4419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.928+0000 7f5b4c575640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b441985d0 0x7f5b44198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.928+0000 7f5b4c575640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5b44111110 con 0x7f5b441010a0 2024-04-21T08:47:14.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.928+0000 7f5b4c575640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b44110f90 con 0x7f5b44106170 2024-04-21T08:47:14.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.928+0000 7f5b4c575640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5b44111290 con 0x7f5b441985d0 2024-04-21T08:47:14.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.928+0000 7f5b4a311640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b441010a0 0x7f5b4419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.928+0000 7f5b4a311640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b441010a0 0x7f5b4419e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43784/0 (socket says 172.21.15.43:43784) 2024-04-21T08:47:14.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.928+0000 7f5b4a311640 1 -- 172.21.15.43:0/230637827 learned_addr learned my addr 172.21.15.43:0/230637827 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:14.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.929+0000 7f5b49b10640 1 --2- 172.21.15.43:0/230637827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b44106170 0x7f5b4419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.929+0000 7f5b4ab12640 1 --2- 172.21.15.43:0/230637827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b441985d0 0x7f5b44198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.929+0000 7f5b4a311640 1 -- 172.21.15.43:0/230637827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b441985d0 msgr2=0x7f5b44198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.929+0000 7f5b4a311640 1 --2- 172.21.15.43:0/230637827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b441985d0 0x7f5b44198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.929+0000 7f5b4a311640 1 -- 172.21.15.43:0/230637827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b44106170 msgr2=0x7f5b4419e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.929+0000 7f5b4a311640 1 --2- 172.21.15.43:0/230637827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b44106170 0x7f5b4419e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.929+0000 7f5b4a311640 1 -- 172.21.15.43:0/230637827 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b441990f0 con 0x7f5b441010a0 2024-04-21T08:47:14.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.929+0000 7f5b4ab12640 1 --2- 172.21.15.43:0/230637827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b441985d0 0x7f5b44198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:14.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.929+0000 7f5b4a311640 1 --2- 172.21.15.43:0/230637827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b441010a0 0x7f5b4419e420 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000d950 tx=0x7f5b4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.929+0000 7f5b3b7fe640 1 -- 172.21.15.43:0/230637827 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b40017070 con 0x7f5b441010a0 2024-04-21T08:47:14.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.929+0000 7f5b3b7fe640 1 -- 172.21.15.43:0/230637827 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b40009c30 con 0x7f5b441010a0 2024-04-21T08:47:14.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.929+0000 7f5b4c575640 1 -- 172.21.15.43:0/230637827 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b441993e0 con 0x7f5b441010a0 2024-04-21T08:47:14.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.929+0000 7f5b3b7fe640 1 -- 172.21.15.43:0/230637827 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4000f620 con 0x7f5b441010a0 2024-04-21T08:47:14.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.929+0000 7f5b4c575640 1 -- 172.21.15.43:0/230637827 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b441a5680 con 0x7f5b441010a0 2024-04-21T08:47:14.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.932+0000 7f5b3b7fe640 1 -- 172.21.15.43:0/230637827 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5b40010460 con 0x7f5b441010a0 2024-04-21T08:47:14.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.932+0000 7f5b4f164740 1 Processor -- start 2024-04-21T08:47:14.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.932+0000 7f5b4f164740 1 -- start start 2024-04-21T08:47:14.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.932+0000 7f5b4f164740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e32aba08e0 0x55e32aba0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.932+0000 7f5b4f164740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba1290 0x55e32aba9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.932+0000 7f5b4f164740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e32aba9e10 0x55e32abac260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.932+0000 7f5b4f164740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e32ab011f0 con 0x55e32aba1290 2024-04-21T08:47:14.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.932+0000 7f5b4f164740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e32aa9e4b0 con 0x55e32aba08e0 2024-04-21T08:47:14.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.932+0000 7f5b4f164740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e32aafd1b0 con 0x55e32aba9e10 2024-04-21T08:47:14.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.932+0000 7f5b4a311640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba1290 0x55e32aba9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.932+0000 7f5b4a311640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba1290 0x55e32aba9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43790/0 (socket says 172.21.15.43:43790) 2024-04-21T08:47:14.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.932+0000 7f5b4a311640 1 -- 172.21.15.43:0/2050233484 learned_addr learned my addr 172.21.15.43:0/2050233484 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:14.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.933+0000 7f5b4ab12640 1 --2- 172.21.15.43:0/2050233484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e32aba08e0 0x55e32aba0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.933+0000 7f5b49b10640 1 --2- 172.21.15.43:0/2050233484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e32aba9e10 0x55e32abac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.933+0000 7f5b3b7fe640 1 --2- 172.21.15.43:0/230637827 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5b300a3da0 0x7f5b300a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.933+0000 7f5b4a311640 1 -- 172.21.15.43:0/2050233484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e32aba9e10 msgr2=0x55e32abac260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:14.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.933+0000 7f5b4a311640 1 --2- 172.21.15.43:0/2050233484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e32aba9e10 0x55e32abac260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.933+0000 7f5b4a311640 1 -- 172.21.15.43:0/2050233484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e32aba08e0 msgr2=0x55e32aba0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:14.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.933+0000 7f5b4a311640 1 --2- 172.21.15.43:0/2050233484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e32aba08e0 0x55e32aba0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.933+0000 7f5b4a311640 1 -- 172.21.15.43:0/2050233484 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e32aa901b0 con 0x55e32aba1290 2024-04-21T08:47:14.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.933+0000 7f5b4ab12640 1 --2- 172.21.15.43:0/2050233484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e32aba08e0 0x55e32aba0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.933+0000 7f5b4ab12640 1 --2- 172.21.15.43:0/230637827 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5b300a3da0 0x7f5b300a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.933+0000 7f5b3b7fe640 1 -- 172.21.15.43:0/230637827 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f5b400d07e0 con 0x7f5b441010a0 2024-04-21T08:47:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.933+0000 7f5b4a311640 1 --2- 172.21.15.43:0/2050233484 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba1290 0x55e32aba9840 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f5b400cda80 tx=0x7f5b40022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc04f2456195fee server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.933+0000 7f5b38bfe640 1 -- 172.21.15.43:0/2050233484 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4009bea0 con 0x55e32aba1290 2024-04-21T08:47:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.933+0000 7f5b38bfe640 1 -- 172.21.15.43:0/2050233484 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b4000ee20 con 0x55e32aba1290 2024-04-21T08:47:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.933+0000 7f5b4ab12640 1 --2- 172.21.15.43:0/230637827 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5b300a3da0 0x7f5b300a6260 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00a5a0 tx=0x7f5b3c00c040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.933+0000 7f5b38bfe640 1 -- 172.21.15.43:0/2050233484 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b40016170 con 0x55e32aba1290 2024-04-21T08:47:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.934+0000 7f5b4f164740 1 -- 172.21.15.43:0/2050233484 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba1290 msgr2=0x55e32aba9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.934+0000 7f5b4f164740 1 --2- 172.21.15.43:0/2050233484 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba1290 0x55e32aba9840 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f5b400cda80 tx=0x7f5b40022000 comp rx=0 tx=0).stop 2024-04-21T08:47:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.934+0000 7f5b4f164740 1 -- 172.21.15.43:0/2050233484 shutdown_connections 2024-04-21T08:47:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.934+0000 7f5b4f164740 1 --2- 172.21.15.43:0/2050233484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e32aba9e10 0x55e32abac260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.934+0000 7f5b4f164740 1 --2- 172.21.15.43:0/2050233484 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba1290 0x55e32aba9840 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.934+0000 7f5b4f164740 1 --2- 172.21.15.43:0/2050233484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e32aba08e0 0x55e32aba0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.934+0000 7f5b4f164740 1 -- 172.21.15.43:0/2050233484 >> 172.21.15.43:0/2050233484 conn(0x55e32ab9afb0 msgr2=0x55e32ab9b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.934+0000 7f5b4f164740 1 -- 172.21.15.43:0/2050233484 shutdown_connections 2024-04-21T08:47:14.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.934+0000 7f5b4f164740 1 -- 172.21.15.43:0/2050233484 wait complete. 2024-04-21T08:47:14.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.934+0000 7f5b4f164740 1 -- start start 2024-04-21T08:47:14.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.935+0000 7f5b4f164740 1 Processor -- start 2024-04-21T08:47:14.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.935+0000 7f5b4f164740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e32aba08e0 0x55e32aba96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.935+0000 7f5b4f164740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba1290 0x55e32aba6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.935+0000 7f5b4f164740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e32aba9e10 0x55e32aba6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.935+0000 7f5b4f164740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e32abaccc0 con 0x55e32aba1290 2024-04-21T08:47:14.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.935+0000 7f5b4f164740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e32aaba500 con 0x55e32aba9e10 2024-04-21T08:47:14.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.935+0000 7f5b4f164740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e32abace40 con 0x55e32aba08e0 2024-04-21T08:47:14.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.935+0000 7f5b4f164740 10 client.0 _ll_register_callbacks cb 0x55e32aa88b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:14.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.935+0000 7f5b49b10640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba1290 0x55e32aba6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.935+0000 7f5b49b10640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba1290 0x55e32aba6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43792/0 (socket says 172.21.15.43:43792) 2024-04-21T08:47:14.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.935+0000 7f5b49b10640 1 -- 172.21.15.43:0/2117525265 learned_addr learned my addr 172.21.15.43:0/2117525265 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:14.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.935+0000 7f5b4ab12640 1 --2- 172.21.15.43:0/2117525265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e32aba9e10 0x55e32aba6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.935+0000 7f5b4a311640 1 --2- 172.21.15.43:0/2117525265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e32aba08e0 0x55e32aba96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.935+0000 7f5b49b10640 1 -- 172.21.15.43:0/2117525265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e32aba08e0 msgr2=0x55e32aba96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:14.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.935+0000 7f5b49b10640 1 --2- 172.21.15.43:0/2117525265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e32aba08e0 0x55e32aba96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.935+0000 7f5b49b10640 1 -- 172.21.15.43:0/2117525265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e32aba9e10 msgr2=0x55e32aba6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:14.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.935+0000 7f5b49b10640 1 --2- 172.21.15.43:0/2117525265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e32aba9e10 0x55e32aba6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.935+0000 7f5b49b10640 1 -- 172.21.15.43:0/2117525265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e32a8133f0 con 0x55e32aba1290 2024-04-21T08:47:14.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.935+0000 7f5b49b10640 1 --2- 172.21.15.43:0/2117525265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba1290 0x55e32aba6a30 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f5b4406a420 tx=0x7f5b3402c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ab80e49e8bf994 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.935+0000 7f5b2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:14.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.936+0000 7f5b2dffb640 1 -- 172.21.15.43:0/2117525265 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3402c610 con 0x55e32aba1290 2024-04-21T08:47:14.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.936+0000 7f5b2dffb640 1 -- 172.21.15.43:0/2117525265 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f5b34005030 con 0x55e32aba1290 2024-04-21T08:47:14.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.936+0000 7f5b4f164740 1 -- 172.21.15.43:0/2117525265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b4000ee20 con 0x55e32aba1290 2024-04-21T08:47:14.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.936+0000 7f5b2dffb640 1 -- 172.21.15.43:0/2117525265 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5b34005230 con 0x55e32aba1290 2024-04-21T08:47:14.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.936+0000 7f5b4f164740 10 client.19386 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:14.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.936+0000 7f5b4f164740 1 -- 172.21.15.43:0/2117525265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e32a8dd7d0 con 0x55e32aba1290 2024-04-21T08:47:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.936+0000 7f5b2dffb640 1 -- 172.21.15.43:0/2117525265 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b4000ee20 con 0x55e32aba1290 2024-04-21T08:47:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.936+0000 7f5b2dffb640 1 -- 172.21.15.43:0/2117525265 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5b34003710 con 0x55e32aba1290 2024-04-21T08:47:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.936+0000 7f5b4f164740 10 client.19386 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.936+0000 7f5b4f164740 10 client.19386 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:47:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.936+0000 7f5b4f164740 20 client.19386 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.936+0000 7f5b4f164740 1 --2- 172.21.15.43:0/2117525265 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55e32abb9f20 0x55e32abbc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.936+0000 7f5b4f164740 4 client.19386 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.936+0000 7f5b4f164740 1 -- 172.21.15.43:0/2117525265 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e32aaa0930 con 0x55e32abb9f20 2024-04-21T08:47:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.937+0000 7f5b4a311640 1 --2- 172.21.15.43:0/2117525265 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55e32abb9f20 0x55e32abbc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.937+0000 7f5b4a311640 1 --2- 172.21.15.43:0/2117525265 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55e32abb9f20 0x55e32abbc360 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c495aab1e99f85f1 server_cookie=7073f3515708c5a7 in_seq=0 out_seq=0 2024-04-21T08:47:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.937+0000 7f5b2dffb640 10 client.19386 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:47:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.938+0000 7f5b2dffb640 1 -- 172.21.15.43:0/2117525265 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f5b40081ae0 con 0x55e32abb9f20 2024-04-21T08:47:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.938+0000 7f5b2dffb640 10 client.19386 handle_command_reply: tid=0 2024-04-21T08:47:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.938+0000 7f5b4f164740 1 client.19386 shutdown 2024-04-21T08:47:14.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.938+0000 7f5b4f164740 1 -- 172.21.15.43:0/2117525265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba1290 msgr2=0x55e32aba6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.938+0000 7f5b4f164740 1 --2- 172.21.15.43:0/2117525265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba1290 0x55e32aba6a30 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f5b4406a420 tx=0x7f5b3402c9f0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.938+0000 7f5b4f164740 1 -- 172.21.15.43:0/2117525265 shutdown_connections 2024-04-21T08:47:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.938+0000 7f5b4f164740 1 --2- 172.21.15.43:0/2117525265 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55e32abb9f20 0x55e32abbc360 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.938+0000 7f5b4f164740 1 --2- 172.21.15.43:0/2117525265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e32aba9e10 0x55e32aba6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.938+0000 7f5b4f164740 1 --2- 172.21.15.43:0/2117525265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba1290 0x55e32aba6a30 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.938+0000 7f5b2dffb640 0 client.19386 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:47:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.938+0000 7f5b4f164740 1 --2- 172.21.15.43:0/2117525265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e32aba08e0 0x55e32aba96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.938+0000 7f5b4f164740 1 -- 172.21.15.43:0/2117525265 >> 172.21.15.43:0/2117525265 conn(0x55e32ab9afb0 msgr2=0x55e32ab9cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.938+0000 7f5b4f164740 1 -- 172.21.15.43:0/2117525265 shutdown_connections 2024-04-21T08:47:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.938+0000 7f5b4f164740 1 -- 172.21.15.43:0/2117525265 wait complete. 2024-04-21T08:47:14.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.938+0000 7f5b4f164740 20 client.19386 trim_cache size 0 max 16384 2024-04-21T08:47:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.951+0000 7f5b4f164740 1 Processor -- start 2024-04-21T08:47:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.951+0000 7f5b4f164740 1 -- start start 2024-04-21T08:47:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.951+0000 7f5b4f164740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e32aba08e0 0x55e32abb5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.951+0000 7f5b4f164740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e32aba1290 0x55e32aba9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.951+0000 7f5b4f164740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba9e10 0x55e32aba96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.951+0000 7f5b4f164740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5b441ad3c0 con 0x55e32aba9e10 2024-04-21T08:47:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.951+0000 7f5b4f164740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b441aef00 con 0x55e32aba08e0 2024-04-21T08:47:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.951+0000 7f5b4f164740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5b441ada90 con 0x55e32aba1290 2024-04-21T08:47:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.951+0000 7f5b4ab12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba9e10 0x55e32aba96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.952+0000 7f5b4ab12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba9e10 0x55e32aba96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43806/0 (socket says 172.21.15.43:43806) 2024-04-21T08:47:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.952+0000 7f5b4ab12640 1 -- 172.21.15.43:0/3827606756 learned_addr learned my addr 172.21.15.43:0/3827606756 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.952+0000 7f5b4a311640 1 --2- 172.21.15.43:0/3827606756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e32aba08e0 0x55e32abb5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.952+0000 7f5b4ab12640 1 -- 172.21.15.43:0/3827606756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e32aba1290 msgr2=0x55e32aba9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.952+0000 7f5b4ab12640 1 --2- 172.21.15.43:0/3827606756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e32aba1290 0x55e32aba9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.952+0000 7f5b4ab12640 1 -- 172.21.15.43:0/3827606756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e32aba08e0 msgr2=0x55e32abb5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.952+0000 7f5b4ab12640 1 --2- 172.21.15.43:0/3827606756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e32aba08e0 0x55e32abb5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.952+0000 7f5b4ab12640 1 -- 172.21.15.43:0/3827606756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e32aba72d0 con 0x55e32aba9e10 2024-04-21T08:47:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.952+0000 7f5b4a311640 1 --2- 172.21.15.43:0/3827606756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e32aba08e0 0x55e32abb5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.952+0000 7f5b4ab12640 1 --2- 172.21.15.43:0/3827606756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba9e10 0x55e32aba96a0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c017820 tx=0x7f5b3c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c044f7ccdec4f86 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.952+0000 7f5b2ffff640 1 -- 172.21.15.43:0/3827606756 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c017e60 con 0x55e32aba9e10 2024-04-21T08:47:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.952+0000 7f5b2ffff640 1 -- 172.21.15.43:0/3827606756 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b3c028070 con 0x55e32aba9e10 2024-04-21T08:47:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.952+0000 7f5b2ffff640 1 -- 172.21.15.43:0/3827606756 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c022350 con 0x55e32aba9e10 2024-04-21T08:47:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.953+0000 7f5b4f164740 1 -- 172.21.15.43:0/3827606756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba9e10 msgr2=0x55e32aba96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.953+0000 7f5b4f164740 1 --2- 172.21.15.43:0/3827606756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba9e10 0x55e32aba96a0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c017820 tx=0x7f5b3c013000 comp rx=0 tx=0).stop 2024-04-21T08:47:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.953+0000 7f5b4f164740 1 -- 172.21.15.43:0/3827606756 shutdown_connections 2024-04-21T08:47:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.953+0000 7f5b4f164740 1 --2- 172.21.15.43:0/3827606756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba9e10 0x55e32aba96a0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.953+0000 7f5b4f164740 1 --2- 172.21.15.43:0/3827606756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e32aba1290 0x55e32aba9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.953+0000 7f5b4f164740 1 --2- 172.21.15.43:0/3827606756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e32aba08e0 0x55e32abb5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.953+0000 7f5b4f164740 1 -- 172.21.15.43:0/3827606756 >> 172.21.15.43:0/3827606756 conn(0x55e32ab9afb0 msgr2=0x55e32aba2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.953+0000 7f5b4f164740 1 -- 172.21.15.43:0/3827606756 shutdown_connections 2024-04-21T08:47:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.953+0000 7f5b4f164740 1 -- 172.21.15.43:0/3827606756 wait complete. 2024-04-21T08:47:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.953+0000 7f5b4f164740 1 -- start start 2024-04-21T08:47:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.953+0000 7f5b4f164740 1 Processor -- start 2024-04-21T08:47:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.953+0000 7f5b4f164740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba08e0 0x55e32abade90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.953+0000 7f5b4f164740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e32aba1290 0x55e32abae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.953+0000 7f5b4f164740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e32aba9e10 0x55e32ac1f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.953+0000 7f5b4f164740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5b441ae830 con 0x55e32aba08e0 2024-04-21T08:47:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.953+0000 7f5b4f164740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b441ae160 con 0x55e32aba1290 2024-04-21T08:47:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.953+0000 7f5b4f164740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5b441ab3c0 con 0x55e32aba9e10 2024-04-21T08:47:14.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.953+0000 7f5b4f164740 10 client.0 _ll_register_callbacks cb 0x7f5b34004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:14.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.953+0000 7f5b4a311640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba08e0 0x55e32abade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.953+0000 7f5b4a311640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba08e0 0x55e32abade90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43810/0 (socket says 172.21.15.43:43810) 2024-04-21T08:47:14.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.953+0000 7f5b4a311640 1 -- 172.21.15.43:0/1651814568 learned_addr learned my addr 172.21.15.43:0/1651814568 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:14.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.953+0000 7f5b49b10640 1 --2- 172.21.15.43:0/1651814568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e32aba1290 0x55e32abae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.954+0000 7f5b4a311640 1 -- 172.21.15.43:0/1651814568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e32aba9e10 msgr2=0x55e32ac1f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:14.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.954+0000 7f5b4a311640 1 --2- 172.21.15.43:0/1651814568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e32aba9e10 0x55e32ac1f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.954+0000 7f5b4a311640 1 -- 172.21.15.43:0/1651814568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e32aba1290 msgr2=0x55e32abae3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:14.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.954+0000 7f5b4a311640 1 --2- 172.21.15.43:0/1651814568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e32aba1290 0x55e32abae3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.954+0000 7f5b4a311640 1 -- 172.21.15.43:0/1651814568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e32ac1f9e0 con 0x55e32aba08e0 2024-04-21T08:47:14.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.954+0000 7f5b49b10640 1 --2- 172.21.15.43:0/1651814568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e32aba1290 0x55e32abae3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:14.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.954+0000 7f5b4a311640 1 --2- 172.21.15.43:0/1651814568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba08e0 0x55e32abade90 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f5b400cdb40 tx=0x7f5b4000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99f0beceb1674b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.954+0000 7f5b2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:14.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.954+0000 7f5b2dffb640 1 -- 172.21.15.43:0/1651814568 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b40016360 con 0x55e32aba08e0 2024-04-21T08:47:14.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.954+0000 7f5b2dffb640 1 -- 172.21.15.43:0/1651814568 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f5b400b0b00 con 0x55e32aba08e0 2024-04-21T08:47:14.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.954+0000 7f5b4f164740 1 -- 172.21.15.43:0/1651814568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b3c028070 con 0x55e32aba08e0 2024-04-21T08:47:14.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.954+0000 7f5b2dffb640 1 -- 172.21.15.43:0/1651814568 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5b4009bcc0 con 0x55e32aba08e0 2024-04-21T08:47:14.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.954+0000 7f5b4f164740 10 client.19392 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:14.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.954+0000 7f5b4f164740 1 -- 172.21.15.43:0/1651814568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e32ac20af0 con 0x55e32aba08e0 2024-04-21T08:47:14.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.954+0000 7f5b2dffb640 1 -- 172.21.15.43:0/1651814568 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b3c028070 con 0x55e32aba08e0 2024-04-21T08:47:14.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.954+0000 7f5b2dffb640 1 -- 172.21.15.43:0/1651814568 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5b400cdd90 con 0x55e32aba08e0 2024-04-21T08:47:14.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.954+0000 7f5b4f164740 10 client.19392 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:14.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.954+0000 7f5b4f164740 10 client.19392 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:47:14.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.954+0000 7f5b4f164740 20 client.19392 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:14.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.954+0000 7f5b4f164740 1 --2- 172.21.15.43:0/1651814568 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55e32ac20e30 0x55e32ac23220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.954+0000 7f5b4f164740 4 client.19392 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:14.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.954+0000 7f5b4f164740 1 -- 172.21.15.43:0/1651814568 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e32ab0ec10 con 0x55e32ac20e30 2024-04-21T08:47:14.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.955+0000 7f5b49b10640 1 --2- 172.21.15.43:0/1651814568 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55e32ac20e30 0x55e32ac23220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.955+0000 7f5b49b10640 1 --2- 172.21.15.43:0/1651814568 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55e32ac20e30 0x55e32ac23220 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b2dee88fc573ae4b server_cookie=7fca065051a2d018 in_seq=0 out_seq=0 2024-04-21T08:47:14.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.955+0000 7f5b2dffb640 10 client.19392 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:47:14.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.955+0000 7f5b2dffb640 1 -- 172.21.15.43:0/1651814568 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e32ab0ec10 con 0x55e32ac20e30 2024-04-21T08:47:14.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.955+0000 7f5b2dffb640 10 client.19392 handle_command_reply: tid=0 2024-04-21T08:47:14.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.955+0000 7f5b4f164740 1 client.19392 shutdown 2024-04-21T08:47:14.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.955+0000 7f5b4f164740 1 -- 172.21.15.43:0/1651814568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba08e0 msgr2=0x55e32abade90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.955+0000 7f5b4f164740 1 --2- 172.21.15.43:0/1651814568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba08e0 0x55e32abade90 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f5b400cdb40 tx=0x7f5b4000bf60 comp rx=0 tx=0).stop 2024-04-21T08:47:14.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.955+0000 7f5b4f164740 1 -- 172.21.15.43:0/1651814568 shutdown_connections 2024-04-21T08:47:14.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.955+0000 7f5b4f164740 1 --2- 172.21.15.43:0/1651814568 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55e32ac20e30 0x55e32ac23220 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.955+0000 7f5b4f164740 1 --2- 172.21.15.43:0/1651814568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e32aba9e10 0x55e32ac1f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.955+0000 7f5b2dffb640 0 client.19392 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:47:14.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.955+0000 7f5b4f164740 1 --2- 172.21.15.43:0/1651814568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e32aba1290 0x55e32abae3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.955+0000 7f5b4f164740 1 --2- 172.21.15.43:0/1651814568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e32aba08e0 0x55e32abade90 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.955+0000 7f5b4f164740 1 -- 172.21.15.43:0/1651814568 >> 172.21.15.43:0/1651814568 conn(0x55e32ab9afb0 msgr2=0x55e32aba17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.955+0000 7f5b4f164740 1 -- 172.21.15.43:0/1651814568 shutdown_connections 2024-04-21T08:47:14.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.955+0000 7f5b4f164740 1 -- 172.21.15.43:0/1651814568 wait complete. 2024-04-21T08:47:14.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.956+0000 7f5b4f164740 20 client.19392 trim_cache size 0 max 16384 2024-04-21T08:47:14.962 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:47:14.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.956+0000 7f5b4c575640 1 -- 172.21.15.43:0/230637827 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5b300a3da0 msgr2=0x7f5b300a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.956+0000 7f5b4c575640 1 --2- 172.21.15.43:0/230637827 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5b300a3da0 0x7f5b300a6260 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00a5a0 tx=0x7f5b3c00c040 comp rx=0 tx=0).stop 2024-04-21T08:47:14.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.956+0000 7f5b4c575640 1 -- 172.21.15.43:0/230637827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b441010a0 msgr2=0x7f5b4419e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.956+0000 7f5b4c575640 1 --2- 172.21.15.43:0/230637827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b441010a0 0x7f5b4419e420 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000d950 tx=0x7f5b4000de10 comp rx=0 tx=0).stop 2024-04-21T08:47:14.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.956+0000 7f5b4c575640 1 -- 172.21.15.43:0/230637827 shutdown_connections 2024-04-21T08:47:14.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.956+0000 7f5b4c575640 1 --2- 172.21.15.43:0/230637827 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5b300a3da0 0x7f5b300a6260 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.956+0000 7f5b4c575640 1 --2- 172.21.15.43:0/230637827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b441985d0 0x7f5b44198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.956+0000 7f5b4c575640 1 --2- 172.21.15.43:0/230637827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b44106170 0x7f5b4419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.956+0000 7f5b4c575640 1 --2- 172.21.15.43:0/230637827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b441010a0 0x7f5b4419e420 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.956+0000 7f5b4c575640 1 -- 172.21.15.43:0/230637827 >> 172.21.15.43:0/230637827 conn(0x7f5b44077eb0 msgr2=0x7f5b4406a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.956+0000 7f5b4c575640 1 -- 172.21.15.43:0/230637827 shutdown_connections 2024-04-21T08:47:14.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:14.956+0000 7f5b4c575640 1 -- 172.21.15.43:0/230637827 wait complete. 2024-04-21T08:47:14.966 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:47:15.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.072+0000 7f46a50e0640 1 -- 172.21.15.43:0/2786911144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46a0101090 msgr2=0x7f46a0101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.072+0000 7f46a50e0640 1 --2- 172.21.15.43:0/2786911144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46a0101090 0x7f46a0101470 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f4698009900 tx=0x7f469802bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:15.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.072+0000 7f46a50e0640 1 -- 172.21.15.43:0/2786911144 shutdown_connections 2024-04-21T08:47:15.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.072+0000 7f46a50e0640 1 --2- 172.21.15.43:0/2786911144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46a0106160 0x7f46a010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.072+0000 7f46a50e0640 1 --2- 172.21.15.43:0/2786911144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46a0101a40 0x7f46a0105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.072+0000 7f46a50e0640 1 --2- 172.21.15.43:0/2786911144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46a0101090 0x7f46a0101470 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.072+0000 7f46a50e0640 1 -- 172.21.15.43:0/2786911144 >> 172.21.15.43:0/2786911144 conn(0x7f46a0077f50 msgr2=0x7f46a0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:15.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.072+0000 7f46a50e0640 1 -- 172.21.15.43:0/2786911144 shutdown_connections 2024-04-21T08:47:15.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.072+0000 7f46a50e0640 1 -- 172.21.15.43:0/2786911144 wait complete. 2024-04-21T08:47:15.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.072+0000 7f46a50e0640 1 Processor -- start 2024-04-21T08:47:15.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.073+0000 7f46a50e0640 1 -- start start 2024-04-21T08:47:15.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.073+0000 7f46a50e0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46a0101090 0x7f46a0102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.073+0000 7f46a50e0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46a0106160 0x7f46a0103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.073+0000 7f46a50e0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46a01035a0 0x7f46a01a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.073+0000 7f46a50e0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f46a0111120 con 0x7f46a0101090 2024-04-21T08:47:15.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.073+0000 7f46a50e0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46a0110fa0 con 0x7f46a01035a0 2024-04-21T08:47:15.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.073+0000 7f46a50e0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01112a0 con 0x7f46a0106160 2024-04-21T08:47:15.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.073+0000 7f469ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46a0101090 0x7f46a0102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.073+0000 7f469ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46a0101090 0x7f46a0102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43830/0 (socket says 172.21.15.43:43830) 2024-04-21T08:47:15.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.073+0000 7f469ed9d640 1 -- 172.21.15.43:0/3758367564 learned_addr learned my addr 172.21.15.43:0/3758367564 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:15.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.073+0000 7f469f59e640 1 --2- 172.21.15.43:0/3758367564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46a01035a0 0x7f46a01a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.073+0000 7f469e59c640 1 --2- 172.21.15.43:0/3758367564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46a0106160 0x7f46a0103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.073+0000 7f469ed9d640 1 -- 172.21.15.43:0/3758367564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46a0106160 msgr2=0x7f46a0103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.073+0000 7f469ed9d640 1 --2- 172.21.15.43:0/3758367564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46a0106160 0x7f46a0103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.073+0000 7f469ed9d640 1 -- 172.21.15.43:0/3758367564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46a01035a0 msgr2=0x7f46a01a6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.073+0000 7f469ed9d640 1 --2- 172.21.15.43:0/3758367564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46a01035a0 0x7f46a01a6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.073+0000 7f469ed9d640 1 -- 172.21.15.43:0/3758367564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a01a7010 con 0x7f46a0101090 2024-04-21T08:47:15.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.073+0000 7f469f59e640 1 --2- 172.21.15.43:0/3758367564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46a01035a0 0x7f46a01a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:15.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.073+0000 7f469ed9d640 1 --2- 172.21.15.43:0/3758367564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46a0101090 0x7f46a0102b20 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f4698005940 tx=0x7f469802c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:15.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.074+0000 7f4687fff640 1 -- 172.21.15.43:0/3758367564 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4698043020 con 0x7f46a0101090 2024-04-21T08:47:15.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.074+0000 7f4687fff640 1 -- 172.21.15.43:0/3758367564 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f469802cc60 con 0x7f46a0101090 2024-04-21T08:47:15.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.074+0000 7f46a50e0640 1 -- 172.21.15.43:0/3758367564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a01a72a0 con 0x7f46a0101090 2024-04-21T08:47:15.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.074+0000 7f4687fff640 1 -- 172.21.15.43:0/3758367564 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4698003c30 con 0x7f46a0101090 2024-04-21T08:47:15.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.074+0000 7f46a50e0640 1 -- 172.21.15.43:0/3758367564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a01a77f0 con 0x7f46a0101090 2024-04-21T08:47:15.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.076+0000 7f4687fff640 1 -- 172.21.15.43:0/3758367564 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4698046020 con 0x7f46a0101090 2024-04-21T08:47:15.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.076+0000 7f46a7dab740 1 Processor -- start 2024-04-21T08:47:15.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.076+0000 7f46a7dab740 1 -- start start 2024-04-21T08:47:15.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.077+0000 7f46a7dab740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5603e85294d0 0x5603e85298b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.077+0000 7f46a7dab740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603e8529e80 0x5603e8532430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.077+0000 7f46a7dab740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e8532a00 0x5603e8534e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.077+0000 7f46a7dab740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5603e848e1f0 con 0x5603e8532a00 2024-04-21T08:47:15.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.077+0000 7f46a7dab740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5603e842b4b0 con 0x5603e85294d0 2024-04-21T08:47:15.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.077+0000 7f46a7dab740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5603e848a1b0 con 0x5603e8529e80 2024-04-21T08:47:15.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.077+0000 7f469e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e8532a00 0x5603e8534e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.077+0000 7f4687fff640 1 --2- 172.21.15.43:0/3758367564 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f46880a3d50 0x7f46880a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.077+0000 7f469e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e8532a00 0x5603e8534e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43844/0 (socket says 172.21.15.43:43844) 2024-04-21T08:47:15.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.077+0000 7f469e59c640 1 -- 172.21.15.43:0/2356300972 learned_addr learned my addr 172.21.15.43:0/2356300972 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:15.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.077+0000 7f469ed9d640 1 --2- 172.21.15.43:0/2356300972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603e8529e80 0x5603e8532430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.077+0000 7f469f59e640 1 --2- 172.21.15.43:0/2356300972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5603e85294d0 0x5603e85298b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.077+0000 7f469f59e640 1 --2- 172.21.15.43:0/3758367564 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f46880a3d50 0x7f46880a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.077+0000 7f4687fff640 1 -- 172.21.15.43:0/3758367564 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f46980f2fd0 con 0x7f46a0101090 2024-04-21T08:47:15.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.077+0000 7f469e59c640 1 -- 172.21.15.43:0/2356300972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603e8529e80 msgr2=0x5603e8532430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:15.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.077+0000 7f469e59c640 1 --2- 172.21.15.43:0/2356300972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603e8529e80 0x5603e8532430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.077+0000 7f469e59c640 1 -- 172.21.15.43:0/2356300972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5603e85294d0 msgr2=0x5603e85298b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:15.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.077+0000 7f469e59c640 1 --2- 172.21.15.43:0/2356300972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5603e85294d0 0x5603e85298b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.077+0000 7f469e59c640 1 -- 172.21.15.43:0/2356300972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603e841d1b0 con 0x5603e8532a00 2024-04-21T08:47:15.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.077+0000 7f469e59c640 1 --2- 172.21.15.43:0/2356300972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e8532a00 0x5603e8534e50 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f469400cc00 tx=0x7f469400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591278a5b31533f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:15.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.078+0000 7f469f59e640 1 --2- 172.21.15.43:0/3758367564 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f46880a3d50 0x7f46880a6210 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4690009440 tx=0x7f469000c040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:15.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.078+0000 7f46857fa640 1 -- 172.21.15.43:0/2356300972 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694005220 con 0x5603e8532a00 2024-04-21T08:47:15.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.078+0000 7f46857fa640 1 -- 172.21.15.43:0/2356300972 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4694004c50 con 0x5603e8532a00 2024-04-21T08:47:15.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.078+0000 7f46857fa640 1 -- 172.21.15.43:0/2356300972 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694010730 con 0x5603e8532a00 2024-04-21T08:47:15.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.078+0000 7f46a7dab740 1 -- 172.21.15.43:0/2356300972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e8532a00 msgr2=0x5603e8534e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.078+0000 7f46a7dab740 1 --2- 172.21.15.43:0/2356300972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e8532a00 0x5603e8534e50 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f469400cc00 tx=0x7f469400e720 comp rx=0 tx=0).stop 2024-04-21T08:47:15.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.078+0000 7f46a7dab740 1 -- 172.21.15.43:0/2356300972 shutdown_connections 2024-04-21T08:47:15.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.078+0000 7f46a7dab740 1 --2- 172.21.15.43:0/2356300972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e8532a00 0x5603e8534e50 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.078+0000 7f46a7dab740 1 --2- 172.21.15.43:0/2356300972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603e8529e80 0x5603e8532430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.078+0000 7f46a7dab740 1 --2- 172.21.15.43:0/2356300972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5603e85294d0 0x5603e85298b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.078+0000 7f46a7dab740 1 -- 172.21.15.43:0/2356300972 >> 172.21.15.43:0/2356300972 conn(0x5603e8523ba0 msgr2=0x5603e8523f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:15.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.078+0000 7f46a7dab740 1 -- 172.21.15.43:0/2356300972 shutdown_connections 2024-04-21T08:47:15.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.078+0000 7f46a7dab740 1 -- 172.21.15.43:0/2356300972 wait complete. 2024-04-21T08:47:15.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.079+0000 7f46a7dab740 1 -- start start 2024-04-21T08:47:15.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.079+0000 7f46a7dab740 1 Processor -- start 2024-04-21T08:47:15.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.079+0000 7f46a7dab740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603e85294d0 0x5603e85322a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.079+0000 7f46a7dab740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5603e8529e80 0x5603e852f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.079+0000 7f46a7dab740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e8532a00 0x5603e852fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.079+0000 7f46a7dab740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5603e8535970 con 0x5603e8532a00 2024-04-21T08:47:15.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.079+0000 7f46a7dab740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5603e8447500 con 0x5603e8529e80 2024-04-21T08:47:15.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.079+0000 7f46a7dab740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5603e8535af0 con 0x5603e85294d0 2024-04-21T08:47:15.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.079+0000 7f46a7dab740 10 client.0 _ll_register_callbacks cb 0x5603e8415b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:15.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.079+0000 7f469f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e8532a00 0x5603e852fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.079+0000 7f469f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e8532a00 0x5603e852fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43858/0 (socket says 172.21.15.43:43858) 2024-04-21T08:47:15.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.079+0000 7f469f59e640 1 -- 172.21.15.43:0/3394047978 learned_addr learned my addr 172.21.15.43:0/3394047978 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:15.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.079+0000 7f469e59c640 1 --2- 172.21.15.43:0/3394047978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5603e8529e80 0x5603e852f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.079+0000 7f469ed9d640 1 --2- 172.21.15.43:0/3394047978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603e85294d0 0x5603e85322a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.079+0000 7f469f59e640 1 -- 172.21.15.43:0/3394047978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603e85294d0 msgr2=0x5603e85322a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:15.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.079+0000 7f469f59e640 1 --2- 172.21.15.43:0/3394047978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603e85294d0 0x5603e85322a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.079+0000 7f469f59e640 1 -- 172.21.15.43:0/3394047978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5603e8529e80 msgr2=0x5603e852f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:15.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.079+0000 7f469f59e640 1 --2- 172.21.15.43:0/3394047978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5603e8529e80 0x5603e852f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.079+0000 7f469f59e640 1 -- 172.21.15.43:0/3394047978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603e81a03f0 con 0x5603e8532a00 2024-04-21T08:47:15.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.080+0000 7f469f59e640 1 --2- 172.21.15.43:0/3394047978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e8532a00 0x5603e852fb70 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f4690017c90 tx=0x7f4690004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8f46b8729c2ff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:15.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.080+0000 7f4682ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:15.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.080+0000 7f4682ff5640 1 -- 172.21.15.43:0/3394047978 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4690027060 con 0x5603e8532a00 2024-04-21T08:47:15.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.080+0000 7f4682ff5640 1 -- 172.21.15.43:0/3394047978 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f4690022600 con 0x5603e8532a00 2024-04-21T08:47:15.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.080+0000 7f46a7dab740 1 -- 172.21.15.43:0/3394047978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4694004c50 con 0x5603e8532a00 2024-04-21T08:47:15.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.080+0000 7f4682ff5640 1 -- 172.21.15.43:0/3394047978 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f469002e050 con 0x5603e8532a00 2024-04-21T08:47:15.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.080+0000 7f46a7dab740 10 client.19404 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:15.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.080+0000 7f46a7dab740 1 -- 172.21.15.43:0/3394047978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5603e826a7d0 con 0x5603e8532a00 2024-04-21T08:47:15.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.080+0000 7f4682ff5640 1 -- 172.21.15.43:0/3394047978 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4694004c50 con 0x5603e8532a00 2024-04-21T08:47:15.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.080+0000 7f4682ff5640 1 -- 172.21.15.43:0/3394047978 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4690012330 con 0x5603e8532a00 2024-04-21T08:47:15.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.080+0000 7f46a7dab740 10 client.19404 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:15.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.080+0000 7f46a7dab740 10 client.19404 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:47:15.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.080+0000 7f46a7dab740 20 client.19404 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:15.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.080+0000 7f46a7dab740 1 --2- 172.21.15.43:0/3394047978 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5603e8542c10 0x5603e8545060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.080+0000 7f46a7dab740 4 client.19404 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:15.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.080+0000 7f46a7dab740 1 -- 172.21.15.43:0/3394047978 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603e842d930 con 0x5603e8542c10 2024-04-21T08:47:15.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.081+0000 7f469ed9d640 1 --2- 172.21.15.43:0/3394047978 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5603e8542c10 0x5603e8545060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.081+0000 7f4682ff5640 10 client.19404 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:47:15.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.081+0000 7f469ed9d640 1 --2- 172.21.15.43:0/3394047978 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5603e8542c10 0x5603e8545060 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4729fdc58f00bb2f server_cookie=61e606b901f04924 in_seq=0 out_seq=0 2024-04-21T08:47:15.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.082+0000 7f4682ff5640 1 -- 172.21.15.43:0/3394047978 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5603e842d930 con 0x5603e8542c10 2024-04-21T08:47:15.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.082+0000 7f4682ff5640 10 client.19404 handle_command_reply: tid=0 2024-04-21T08:47:15.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.082+0000 7f46a7dab740 1 client.19404 shutdown 2024-04-21T08:47:15.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.082+0000 7f46a7dab740 1 -- 172.21.15.43:0/3394047978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e8532a00 msgr2=0x5603e852fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.082+0000 7f46a7dab740 1 --2- 172.21.15.43:0/3394047978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e8532a00 0x5603e852fb70 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f4690017c90 tx=0x7f4690004b50 comp rx=0 tx=0).stop 2024-04-21T08:47:15.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.082+0000 7f46a7dab740 1 -- 172.21.15.43:0/3394047978 shutdown_connections 2024-04-21T08:47:15.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.082+0000 7f46a7dab740 1 --2- 172.21.15.43:0/3394047978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e8532a00 0x5603e852fb70 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.082+0000 7f46a7dab740 1 --2- 172.21.15.43:0/3394047978 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5603e8542c10 0x5603e8545060 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.082+0000 7f46a7dab740 1 --2- 172.21.15.43:0/3394047978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5603e8529e80 0x5603e852f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.082+0000 7f46a7dab740 1 --2- 172.21.15.43:0/3394047978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603e85294d0 0x5603e85322a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.082+0000 7f4682ff5640 0 client.19404 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:47:15.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.082+0000 7f46a7dab740 1 -- 172.21.15.43:0/3394047978 >> 172.21.15.43:0/3394047978 conn(0x5603e8523ba0 msgr2=0x5603e85258f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:15.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.082+0000 7f46a7dab740 1 -- 172.21.15.43:0/3394047978 shutdown_connections 2024-04-21T08:47:15.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.082+0000 7f46a7dab740 1 -- 172.21.15.43:0/3394047978 wait complete. 2024-04-21T08:47:15.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.082+0000 7f46a7dab740 20 client.19404 trim_cache size 0 max 16384 2024-04-21T08:47:15.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.095+0000 7f46a7dab740 1 Processor -- start 2024-04-21T08:47:15.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.095+0000 7f46a7dab740 1 -- start start 2024-04-21T08:47:15.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.095+0000 7f46a7dab740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603e85294d0 0x5603e853e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.096+0000 7f46a7dab740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5603e8529e80 0x5603e8522a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.096+0000 7f46a7dab740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e8532a00 0x5603e8531d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.096+0000 7f46a7dab740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01b16a0 con 0x5603e8532a00 2024-04-21T08:47:15.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.096+0000 7f46a7dab740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01b31e0 con 0x5603e8529e80 2024-04-21T08:47:15.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.096+0000 7f46a7dab740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01b1d70 con 0x5603e85294d0 2024-04-21T08:47:15.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.096+0000 7f469f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e8532a00 0x5603e8531d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.096+0000 7f469f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e8532a00 0x5603e8531d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43862/0 (socket says 172.21.15.43:43862) 2024-04-21T08:47:15.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.096+0000 7f469f59e640 1 -- 172.21.15.43:0/823810243 learned_addr learned my addr 172.21.15.43:0/823810243 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:15.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.096+0000 7f469f59e640 1 -- 172.21.15.43:0/823810243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603e85294d0 msgr2=0x5603e853e830 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:47:15.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.096+0000 7f469f59e640 1 --2- 172.21.15.43:0/823810243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603e85294d0 0x5603e853e830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.096+0000 7f469f59e640 1 -- 172.21.15.43:0/823810243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5603e8529e80 msgr2=0x5603e8522a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:15.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.096+0000 7f469f59e640 1 --2- 172.21.15.43:0/823810243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5603e8529e80 0x5603e8522a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.096+0000 7f469f59e640 1 -- 172.21.15.43:0/823810243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603e8529c20 con 0x5603e8532a00 2024-04-21T08:47:15.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.096+0000 7f469f59e640 1 --2- 172.21.15.43:0/823810243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e8532a00 0x5603e8531d60 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f4690017510 tx=0x7f46900121f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314c83be6789466f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:15.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.096+0000 7f4684ff9640 1 -- 172.21.15.43:0/823810243 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4690012050 con 0x5603e8532a00 2024-04-21T08:47:15.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.096+0000 7f4684ff9640 1 -- 172.21.15.43:0/823810243 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4690037070 con 0x5603e8532a00 2024-04-21T08:47:15.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.096+0000 7f4684ff9640 1 -- 172.21.15.43:0/823810243 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4690013040 con 0x5603e8532a00 2024-04-21T08:47:15.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.097+0000 7f46a7dab740 1 -- 172.21.15.43:0/823810243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e8532a00 msgr2=0x5603e8531d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.097+0000 7f46a7dab740 1 --2- 172.21.15.43:0/823810243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e8532a00 0x5603e8531d60 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f4690017510 tx=0x7f46900121f0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.097+0000 7f46a7dab740 1 -- 172.21.15.43:0/823810243 shutdown_connections 2024-04-21T08:47:15.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.097+0000 7f46a7dab740 1 --2- 172.21.15.43:0/823810243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e8532a00 0x5603e8531d60 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.097+0000 7f46a7dab740 1 --2- 172.21.15.43:0/823810243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5603e8529e80 0x5603e8522a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.097+0000 7f46a7dab740 1 --2- 172.21.15.43:0/823810243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603e85294d0 0x5603e853e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.097+0000 7f46a7dab740 1 -- 172.21.15.43:0/823810243 >> 172.21.15.43:0/823810243 conn(0x5603e8523ba0 msgr2=0x5603e8522370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.097+0000 7f46a7dab740 1 -- 172.21.15.43:0/823810243 shutdown_connections 2024-04-21T08:47:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.097+0000 7f46a7dab740 1 -- 172.21.15.43:0/823810243 wait complete. 2024-04-21T08:47:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.097+0000 7f46a7dab740 1 -- start start 2024-04-21T08:47:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.097+0000 7f46a7dab740 1 Processor -- start 2024-04-21T08:47:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.097+0000 7f46a7dab740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e85294d0 0x5603e853b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.097+0000 7f46a7dab740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5603e8529e80 0x5603e85363e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.097+0000 7f46a7dab740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603e8532a00 0x5603e8536920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.097+0000 7f46a7dab740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01b2b10 con 0x5603e85294d0 2024-04-21T08:47:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.097+0000 7f46a7dab740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01b2440 con 0x5603e8529e80 2024-04-21T08:47:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.097+0000 7f46a7dab740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01af6a0 con 0x5603e8532a00 2024-04-21T08:47:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.097+0000 7f46a7dab740 10 client.0 _ll_register_callbacks cb 0x7f4690017940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.098+0000 7f469ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e85294d0 0x5603e853b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.098+0000 7f469ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e85294d0 0x5603e853b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43864/0 (socket says 172.21.15.43:43864) 2024-04-21T08:47:15.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.098+0000 7f469ed9d640 1 -- 172.21.15.43:0/1193897158 learned_addr learned my addr 172.21.15.43:0/1193897158 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:15.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.098+0000 7f469f59e640 1 --2- 172.21.15.43:0/1193897158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603e8532a00 0x5603e8536920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.098+0000 7f469e59c640 1 --2- 172.21.15.43:0/1193897158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5603e8529e80 0x5603e85363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.098+0000 7f469ed9d640 1 -- 172.21.15.43:0/1193897158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603e8532a00 msgr2=0x5603e8536920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:15.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.098+0000 7f469ed9d640 1 --2- 172.21.15.43:0/1193897158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603e8532a00 0x5603e8536920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.098+0000 7f469ed9d640 1 -- 172.21.15.43:0/1193897158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5603e8529e80 msgr2=0x5603e85363e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:15.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.098+0000 7f469ed9d640 1 --2- 172.21.15.43:0/1193897158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5603e8529e80 0x5603e85363e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.098+0000 7f469ed9d640 1 -- 172.21.15.43:0/1193897158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603e8537060 con 0x5603e85294d0 2024-04-21T08:47:15.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.098+0000 7f469e59c640 1 --2- 172.21.15.43:0/1193897158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5603e8529e80 0x5603e85363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:15.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.098+0000 7f469ed9d640 1 --2- 172.21.15.43:0/1193897158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e85294d0 0x5603e853b020 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f46980bb060 tx=0x7f4698003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2c25f53a3218b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:15.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.098+0000 7f4682ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:15.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.098+0000 7f4682ff5640 1 -- 172.21.15.43:0/1193897158 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46980bec00 con 0x5603e85294d0 2024-04-21T08:47:15.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.098+0000 7f4682ff5640 1 -- 172.21.15.43:0/1193897158 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f46980bb270 con 0x5603e85294d0 2024-04-21T08:47:15.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.098+0000 7f46a7dab740 1 -- 172.21.15.43:0/1193897158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4690037070 con 0x5603e85294d0 2024-04-21T08:47:15.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.098+0000 7f4682ff5640 1 -- 172.21.15.43:0/1193897158 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f46980c2fb0 con 0x5603e85294d0 2024-04-21T08:47:15.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.098+0000 7f46a7dab740 10 client.19410 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:15.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.098+0000 7f46a7dab740 1 -- 172.21.15.43:0/1193897158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5603e85a9130 con 0x5603e85294d0 2024-04-21T08:47:15.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.099+0000 7f4682ff5640 1 -- 172.21.15.43:0/1193897158 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4690037070 con 0x5603e85294d0 2024-04-21T08:47:15.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.099+0000 7f4682ff5640 1 -- 172.21.15.43:0/1193897158 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f46980f05b0 con 0x5603e85294d0 2024-04-21T08:47:15.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.099+0000 7f46a7dab740 10 client.19410 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:15.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.099+0000 7f46a7dab740 10 client.19410 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:47:15.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.099+0000 7f46a7dab740 20 client.19410 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:15.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.099+0000 7f46a7dab740 1 --2- 172.21.15.43:0/1193897158 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5603e85a9800 0x5603e85abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.099+0000 7f46a7dab740 4 client.19410 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:15.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.099+0000 7f46a7dab740 1 -- 172.21.15.43:0/1193897158 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5603e852ff00 con 0x5603e85a9800 2024-04-21T08:47:15.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.099+0000 7f469e59c640 1 --2- 172.21.15.43:0/1193897158 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5603e85a9800 0x5603e85abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.099+0000 7f4682ff5640 10 client.19410 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:47:15.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.099+0000 7f469e59c640 1 --2- 172.21.15.43:0/1193897158 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5603e85a9800 0x5603e85abbf0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=589e22be897ee810 server_cookie=f2832f05baff74a1 in_seq=0 out_seq=0 2024-04-21T08:47:15.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f4682ff5640 1 -- 172.21.15.43:0/1193897158 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5603e852ff00 con 0x5603e85a9800 2024-04-21T08:47:15.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f4682ff5640 10 client.19410 handle_command_reply: tid=0 2024-04-21T08:47:15.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f46a7dab740 1 client.19410 shutdown 2024-04-21T08:47:15.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f46a7dab740 1 -- 172.21.15.43:0/1193897158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e85294d0 msgr2=0x5603e853b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f46a7dab740 1 --2- 172.21.15.43:0/1193897158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e85294d0 0x5603e853b020 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f46980bb060 tx=0x7f4698003760 comp rx=0 tx=0).stop 2024-04-21T08:47:15.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f46a7dab740 1 -- 172.21.15.43:0/1193897158 shutdown_connections 2024-04-21T08:47:15.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f46a7dab740 1 --2- 172.21.15.43:0/1193897158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5603e8532a00 0x5603e8536920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f46a7dab740 1 --2- 172.21.15.43:0/1193897158 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5603e85a9800 0x5603e85abbf0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f46a7dab740 1 --2- 172.21.15.43:0/1193897158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5603e8529e80 0x5603e85363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f4682ff5640 0 client.19410 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:47:15.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f46a7dab740 1 --2- 172.21.15.43:0/1193897158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603e85294d0 0x5603e853b020 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f46a7dab740 1 -- 172.21.15.43:0/1193897158 >> 172.21.15.43:0/1193897158 conn(0x5603e8523ba0 msgr2=0x5603e8522c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:15.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f46a7dab740 1 -- 172.21.15.43:0/1193897158 shutdown_connections 2024-04-21T08:47:15.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f46a7dab740 1 -- 172.21.15.43:0/1193897158 wait complete. 2024-04-21T08:47:15.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f46a7dab740 20 client.19410 trim_cache size 0 max 16384 2024-04-21T08:47:15.106 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:47:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f46a50e0640 1 -- 172.21.15.43:0/3758367564 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f46880a3d50 msgr2=0x7f46880a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f46a50e0640 1 --2- 172.21.15.43:0/3758367564 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f46880a3d50 0x7f46880a6210 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4690009440 tx=0x7f469000c040 comp rx=0 tx=0).stop 2024-04-21T08:47:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f46a50e0640 1 -- 172.21.15.43:0/3758367564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46a0101090 msgr2=0x7f46a0102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f46a50e0640 1 --2- 172.21.15.43:0/3758367564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46a0101090 0x7f46a0102b20 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f4698005940 tx=0x7f469802c860 comp rx=0 tx=0).stop 2024-04-21T08:47:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f46a50e0640 1 -- 172.21.15.43:0/3758367564 shutdown_connections 2024-04-21T08:47:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f46a50e0640 1 --2- 172.21.15.43:0/3758367564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46a01035a0 0x7f46a01a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f46a50e0640 1 --2- 172.21.15.43:0/3758367564 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f46880a3d50 0x7f46880a6210 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f46a50e0640 1 --2- 172.21.15.43:0/3758367564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46a0106160 0x7f46a0103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f46a50e0640 1 --2- 172.21.15.43:0/3758367564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46a0101090 0x7f46a0102b20 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.100+0000 7f46a50e0640 1 -- 172.21.15.43:0/3758367564 >> 172.21.15.43:0/3758367564 conn(0x7f46a0077f50 msgr2=0x7f46a00693e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.101+0000 7f46a50e0640 1 -- 172.21.15.43:0/3758367564 shutdown_connections 2024-04-21T08:47:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.101+0000 7f46a50e0640 1 -- 172.21.15.43:0/3758367564 wait complete. 2024-04-21T08:47:15.111 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:47:15.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.216+0000 7ff215031640 1 -- 172.21.15.43:0/678995352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff21006b170 msgr2=0x7ff210106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.216+0000 7ff215031640 1 --2- 172.21.15.43:0/678995352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff21006b170 0x7ff210106ba0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009900 tx=0x7ff1f802bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:15.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.216+0000 7ff215031640 1 -- 172.21.15.43:0/678995352 shutdown_connections 2024-04-21T08:47:15.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.216+0000 7ff215031640 1 --2- 172.21.15.43:0/678995352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff210107280 0x7ff21010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.216+0000 7ff215031640 1 --2- 172.21.15.43:0/678995352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff21006b170 0x7ff210106ba0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.216+0000 7ff215031640 1 --2- 172.21.15.43:0/678995352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff21006a7c0 0x7ff21006aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.216+0000 7ff215031640 1 -- 172.21.15.43:0/678995352 >> 172.21.15.43:0/678995352 conn(0x7ff210076c50 msgr2=0x7ff210077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:15.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.216+0000 7ff215031640 1 -- 172.21.15.43:0/678995352 shutdown_connections 2024-04-21T08:47:15.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.216+0000 7ff215031640 1 -- 172.21.15.43:0/678995352 wait complete. 2024-04-21T08:47:15.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.217+0000 7ff215031640 1 Processor -- start 2024-04-21T08:47:15.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.217+0000 7ff215031640 1 -- start start 2024-04-21T08:47:15.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.217+0000 7ff215031640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff21006a7c0 0x7ff210198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.217+0000 7ff215031640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff210107280 0x7ff210198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.217+0000 7ff215031640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff21019cfc0 0x7ff21019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.217+0000 7ff215031640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff2101121a0 con 0x7ff21006a7c0 2024-04-21T08:47:15.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.217+0000 7ff215031640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff210112020 con 0x7ff21019cfc0 2024-04-21T08:47:15.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.217+0000 7ff215031640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff210112320 con 0x7ff210107280 2024-04-21T08:47:15.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.217+0000 7ff20ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff21006a7c0 0x7ff210198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.217+0000 7ff20ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff21006a7c0 0x7ff210198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43876/0 (socket says 172.21.15.43:43876) 2024-04-21T08:47:15.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.217+0000 7ff20ed9d640 1 -- 172.21.15.43:0/393543708 learned_addr learned my addr 172.21.15.43:0/393543708 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:15.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.217+0000 7ff20f59e640 1 --2- 172.21.15.43:0/393543708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff21019cfc0 0x7ff21019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.217+0000 7ff207fff640 1 --2- 172.21.15.43:0/393543708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff210107280 0x7ff210198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.217+0000 7ff20ed9d640 1 -- 172.21.15.43:0/393543708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff210107280 msgr2=0x7ff210198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.217+0000 7ff20ed9d640 1 --2- 172.21.15.43:0/393543708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff210107280 0x7ff210198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.217+0000 7ff20ed9d640 1 -- 172.21.15.43:0/393543708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff21019cfc0 msgr2=0x7ff21019d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.217+0000 7ff20ed9d640 1 --2- 172.21.15.43:0/393543708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff21019cfc0 0x7ff21019d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.217+0000 7ff20ed9d640 1 -- 172.21.15.43:0/393543708 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21019dab0 con 0x7ff21006a7c0 2024-04-21T08:47:15.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.218+0000 7ff20ed9d640 1 --2- 172.21.15.43:0/393543708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff21006a7c0 0x7ff210198700 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7ff20000c890 tx=0x7ff20000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:15.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.218+0000 7ff20cd99640 1 -- 172.21.15.43:0/393543708 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff200004220 con 0x7ff21006a7c0 2024-04-21T08:47:15.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.218+0000 7ff20cd99640 1 -- 172.21.15.43:0/393543708 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2000043c0 con 0x7ff21006a7c0 2024-04-21T08:47:15.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.218+0000 7ff215031640 1 -- 172.21.15.43:0/393543708 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff21019dda0 con 0x7ff21006a7c0 2024-04-21T08:47:15.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.218+0000 7ff20cd99640 1 -- 172.21.15.43:0/393543708 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff200004a00 con 0x7ff21006a7c0 2024-04-21T08:47:15.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.218+0000 7ff215031640 1 -- 172.21.15.43:0/393543708 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2101a5560 con 0x7ff21006a7c0 2024-04-21T08:47:15.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.221+0000 7ff20cd99640 1 -- 172.21.15.43:0/393543708 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff2000079b0 con 0x7ff21006a7c0 2024-04-21T08:47:15.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.221+0000 7ff217c26740 1 Processor -- start 2024-04-21T08:47:15.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.221+0000 7ff217c26740 1 -- start start 2024-04-21T08:47:15.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.221+0000 7ff217c26740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c77f898250 0x55c77f898630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.221+0000 7ff217c26740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898c00 0x55c77f8a5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.221+0000 7ff217c26740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77f8a5e10 0x55c77f8a8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.221+0000 7ff217c26740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c77f7fd1f0 con 0x55c77f898c00 2024-04-21T08:47:15.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.221+0000 7ff217c26740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c77f79a4b0 con 0x55c77f8a5e10 2024-04-21T08:47:15.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.221+0000 7ff217c26740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c77f7f91b0 con 0x55c77f898250 2024-04-21T08:47:15.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.221+0000 7ff20ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898c00 0x55c77f8a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.221+0000 7ff20ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898c00 0x55c77f8a5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43884/0 (socket says 172.21.15.43:43884) 2024-04-21T08:47:15.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.221+0000 7ff20ed9d640 1 -- 172.21.15.43:0/3096751751 learned_addr learned my addr 172.21.15.43:0/3096751751 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:15.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.222+0000 7ff20cd99640 1 --2- 172.21.15.43:0/393543708 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff1fc0a3d20 0x7ff1fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.222+0000 7ff207fff640 1 --2- 172.21.15.43:0/3096751751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77f8a5e10 0x55c77f8a8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.222+0000 7ff20f59e640 1 --2- 172.21.15.43:0/3096751751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c77f898250 0x55c77f898630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.222+0000 7ff20f59e640 1 --2- 172.21.15.43:0/393543708 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff1fc0a3d20 0x7ff1fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.222+0000 7ff20ed9d640 1 -- 172.21.15.43:0/3096751751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c77f898250 msgr2=0x55c77f898630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:15.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.222+0000 7ff20ed9d640 1 --2- 172.21.15.43:0/3096751751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c77f898250 0x55c77f898630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.222+0000 7ff20ed9d640 1 -- 172.21.15.43:0/3096751751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77f8a5e10 msgr2=0x55c77f8a8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:15.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.222+0000 7ff20ed9d640 1 --2- 172.21.15.43:0/3096751751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77f8a5e10 0x55c77f8a8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.222+0000 7ff20cd99640 1 -- 172.21.15.43:0/393543708 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7ff2000d0a20 con 0x7ff21006a7c0 2024-04-21T08:47:15.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.222+0000 7ff20ed9d640 1 -- 172.21.15.43:0/3096751751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c77f78c1b0 con 0x55c77f898c00 2024-04-21T08:47:15.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.222+0000 7ff207fff640 1 --2- 172.21.15.43:0/3096751751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77f8a5e10 0x55c77f8a8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:15.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.222+0000 7ff20ed9d640 1 --2- 172.21.15.43:0/3096751751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898c00 0x55c77f8a5840 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7ff20009c530 tx=0x7ff200000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9db5ba7d95231fc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:15.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.222+0000 7ff20f59e640 1 --2- 172.21.15.43:0/393543708 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff1fc0a3d20 0x7ff1fc0a61e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff208009fd0 tx=0x7ff208005bd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:15.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.222+0000 7ff2057fa640 1 -- 172.21.15.43:0/3096751751 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20001d070 con 0x55c77f898c00 2024-04-21T08:47:15.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.222+0000 7ff2057fa640 1 -- 172.21.15.43:0/3096751751 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff200005290 con 0x55c77f898c00 2024-04-21T08:47:15.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.222+0000 7ff2057fa640 1 -- 172.21.15.43:0/3096751751 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2000a0950 con 0x55c77f898c00 2024-04-21T08:47:15.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.223+0000 7ff217c26740 1 -- 172.21.15.43:0/3096751751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898c00 msgr2=0x55c77f8a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.223+0000 7ff217c26740 1 --2- 172.21.15.43:0/3096751751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898c00 0x55c77f8a5840 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7ff20009c530 tx=0x7ff200000bc0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.223+0000 7ff217c26740 1 -- 172.21.15.43:0/3096751751 shutdown_connections 2024-04-21T08:47:15.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.223+0000 7ff217c26740 1 --2- 172.21.15.43:0/3096751751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77f8a5e10 0x55c77f8a8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.223+0000 7ff217c26740 1 --2- 172.21.15.43:0/3096751751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898c00 0x55c77f8a5840 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.223+0000 7ff217c26740 1 --2- 172.21.15.43:0/3096751751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c77f898250 0x55c77f898630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.223+0000 7ff217c26740 1 -- 172.21.15.43:0/3096751751 >> 172.21.15.43:0/3096751751 conn(0x55c77f892ba0 msgr2=0x55c77f892f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:15.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.223+0000 7ff217c26740 1 -- 172.21.15.43:0/3096751751 shutdown_connections 2024-04-21T08:47:15.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.223+0000 7ff217c26740 1 -- 172.21.15.43:0/3096751751 wait complete. 2024-04-21T08:47:15.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.223+0000 7ff217c26740 1 -- start start 2024-04-21T08:47:15.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.224+0000 7ff217c26740 1 Processor -- start 2024-04-21T08:47:15.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.224+0000 7ff217c26740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898250 0x55c77f8a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.224+0000 7ff217c26740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c77f898c00 0x55c77f8a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.224+0000 7ff217c26740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77f8a5e10 0x55c77f8a2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.224+0000 7ff217c26740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c77f8a8d40 con 0x55c77f898250 2024-04-21T08:47:15.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.224+0000 7ff217c26740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c77f7b6500 con 0x55c77f8a5e10 2024-04-21T08:47:15.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.224+0000 7ff217c26740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c77f8a8ec0 con 0x55c77f898c00 2024-04-21T08:47:15.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.224+0000 7ff217c26740 10 client.0 _ll_register_callbacks cb 0x55c77f784b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:15.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.224+0000 7ff20ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898250 0x55c77f8a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.224+0000 7ff20ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898250 0x55c77f8a56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43898/0 (socket says 172.21.15.43:43898) 2024-04-21T08:47:15.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.224+0000 7ff20ed9d640 1 -- 172.21.15.43:0/2755545920 learned_addr learned my addr 172.21.15.43:0/2755545920 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:15.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.224+0000 7ff20f59e640 1 --2- 172.21.15.43:0/2755545920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77f8a5e10 0x55c77f8a2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.224+0000 7ff207fff640 1 --2- 172.21.15.43:0/2755545920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c77f898c00 0x55c77f8a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.224+0000 7ff20ed9d640 1 -- 172.21.15.43:0/2755545920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c77f898c00 msgr2=0x55c77f8a2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:15.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.224+0000 7ff20ed9d640 1 --2- 172.21.15.43:0/2755545920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c77f898c00 0x55c77f8a2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.224+0000 7ff20ed9d640 1 -- 172.21.15.43:0/2755545920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77f8a5e10 msgr2=0x55c77f8a2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:15.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.224+0000 7ff20ed9d640 1 --2- 172.21.15.43:0/2755545920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77f8a5e10 0x55c77f8a2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.224+0000 7ff20ed9d640 1 -- 172.21.15.43:0/2755545920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c77f50f3f0 con 0x55c77f898250 2024-04-21T08:47:15.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.224+0000 7ff20ed9d640 1 --2- 172.21.15.43:0/2755545920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898250 0x55c77f8a56d0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7ff200098280 tx=0x7ff20009c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa92b0445ec26fd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:15.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.224+0000 7ff1f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:15.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.225+0000 7ff1f67fc640 1 -- 172.21.15.43:0/2755545920 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20001d040 con 0x55c77f898250 2024-04-21T08:47:15.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.225+0000 7ff1f67fc640 1 -- 172.21.15.43:0/2755545920 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7ff2000a05e0 con 0x55c77f898250 2024-04-21T08:47:15.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.225+0000 7ff217c26740 1 -- 172.21.15.43:0/2755545920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c77f79c930 con 0x55c77f898250 2024-04-21T08:47:15.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.225+0000 7ff1f67fc640 1 -- 172.21.15.43:0/2755545920 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff2000a5210 con 0x55c77f898250 2024-04-21T08:47:15.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.225+0000 7ff217c26740 10 client.19422 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:15.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.225+0000 7ff217c26740 1 -- 172.21.15.43:0/2755545920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c77f5d97d0 con 0x55c77f898250 2024-04-21T08:47:15.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.225+0000 7ff1f67fc640 1 -- 172.21.15.43:0/2755545920 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c77f79c930 con 0x55c77f898250 2024-04-21T08:47:15.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.225+0000 7ff1f67fc640 1 -- 172.21.15.43:0/2755545920 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff20009c890 con 0x55c77f898250 2024-04-21T08:47:15.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.225+0000 7ff217c26740 10 client.19422 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:15.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.225+0000 7ff217c26740 10 client.19422 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:47:15.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.225+0000 7ff217c26740 20 client.19422 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:15.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.225+0000 7ff217c26740 1 --2- 172.21.15.43:0/2755545920 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55c77f8b5e70 0x55c77f8b8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.225+0000 7ff217c26740 4 client.19422 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:15.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.225+0000 7ff217c26740 1 -- 172.21.15.43:0/2755545920 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c77f80ac10 con 0x55c77f8b5e70 2024-04-21T08:47:15.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.225+0000 7ff207fff640 1 --2- 172.21.15.43:0/2755545920 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55c77f8b5e70 0x55c77f8b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.226+0000 7ff207fff640 1 --2- 172.21.15.43:0/2755545920 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55c77f8b5e70 0x55c77f8b8260 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=30c66c3722bc223a server_cookie=e86566261098b980 in_seq=0 out_seq=0 2024-04-21T08:47:15.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.226+0000 7ff1f67fc640 10 client.19422 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:47:15.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.227+0000 7ff1f67fc640 1 -- 172.21.15.43:0/2755545920 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c77f80ac10 con 0x55c77f8b5e70 2024-04-21T08:47:15.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.227+0000 7ff1f67fc640 10 client.19422 handle_command_reply: tid=0 2024-04-21T08:47:15.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.227+0000 7ff217c26740 1 client.19422 shutdown 2024-04-21T08:47:15.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.227+0000 7ff217c26740 1 -- 172.21.15.43:0/2755545920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898250 msgr2=0x55c77f8a56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.227+0000 7ff217c26740 1 --2- 172.21.15.43:0/2755545920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898250 0x55c77f8a56d0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7ff200098280 tx=0x7ff20009c700 comp rx=0 tx=0).stop 2024-04-21T08:47:15.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.227+0000 7ff217c26740 1 -- 172.21.15.43:0/2755545920 shutdown_connections 2024-04-21T08:47:15.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.227+0000 7ff217c26740 1 --2- 172.21.15.43:0/2755545920 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55c77f8b5e70 0x55c77f8b8260 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.227+0000 7ff217c26740 1 --2- 172.21.15.43:0/2755545920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77f8a5e10 0x55c77f8a2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.227+0000 7ff217c26740 1 --2- 172.21.15.43:0/2755545920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c77f898c00 0x55c77f8a2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.227+0000 7ff217c26740 1 --2- 172.21.15.43:0/2755545920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898250 0x55c77f8a56d0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.227+0000 7ff1f67fc640 0 client.19422 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:47:15.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.227+0000 7ff217c26740 1 -- 172.21.15.43:0/2755545920 >> 172.21.15.43:0/2755545920 conn(0x55c77f892ba0 msgr2=0x55c77f8948f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:15.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.227+0000 7ff217c26740 1 -- 172.21.15.43:0/2755545920 shutdown_connections 2024-04-21T08:47:15.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.227+0000 7ff217c26740 1 -- 172.21.15.43:0/2755545920 wait complete. 2024-04-21T08:47:15.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.227+0000 7ff217c26740 20 client.19422 trim_cache size 0 max 16384 2024-04-21T08:47:15.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.240+0000 7ff217c26740 1 Processor -- start 2024-04-21T08:47:15.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.240+0000 7ff217c26740 1 -- start start 2024-04-21T08:47:15.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.241+0000 7ff217c26740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898250 0x55c77f8b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.241+0000 7ff217c26740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77f898c00 0x55c77f8a5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.241+0000 7ff217c26740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c77f8a5e10 0x55c77f8a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.241+0000 7ff217c26740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff2101ad2c0 con 0x55c77f898250 2024-04-21T08:47:15.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.241+0000 7ff217c26740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2101aee00 con 0x55c77f898c00 2024-04-21T08:47:15.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.241+0000 7ff217c26740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff2101ad990 con 0x55c77f8a5e10 2024-04-21T08:47:15.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.241+0000 7ff20ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898250 0x55c77f8b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.241+0000 7ff20ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898250 0x55c77f8b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43904/0 (socket says 172.21.15.43:43904) 2024-04-21T08:47:15.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.241+0000 7ff20ed9d640 1 -- 172.21.15.43:0/1937100981 learned_addr learned my addr 172.21.15.43:0/1937100981 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:15.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.241+0000 7ff20f59e640 1 --2- 172.21.15.43:0/1937100981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c77f8a5e10 0x55c77f8a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.241+0000 7ff207fff640 1 --2- 172.21.15.43:0/1937100981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77f898c00 0x55c77f8a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.241+0000 7ff20ed9d640 1 -- 172.21.15.43:0/1937100981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c77f8a5e10 msgr2=0x55c77f8a56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:15.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.241+0000 7ff20ed9d640 1 --2- 172.21.15.43:0/1937100981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c77f8a5e10 0x55c77f8a56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.241+0000 7ff20ed9d640 1 -- 172.21.15.43:0/1937100981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77f898c00 msgr2=0x55c77f8a5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:15.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.241+0000 7ff20ed9d640 1 --2- 172.21.15.43:0/1937100981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77f898c00 0x55c77f8a5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.241+0000 7ff20ed9d640 1 -- 172.21.15.43:0/1937100981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c77f88d070 con 0x55c77f898250 2024-04-21T08:47:15.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.241+0000 7ff20f59e640 1 --2- 172.21.15.43:0/1937100981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c77f8a5e10 0x55c77f8a56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:15.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.241+0000 7ff20ed9d640 1 --2- 172.21.15.43:0/1937100981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898250 0x55c77f8b1860 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7ff200098f70 tx=0x7ff2000d4130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd82618e7f9eee45 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:15.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.242+0000 7ff204ff9640 1 -- 172.21.15.43:0/1937100981 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20001d070 con 0x55c77f898250 2024-04-21T08:47:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.242+0000 7ff204ff9640 1 -- 172.21.15.43:0/1937100981 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2000a0950 con 0x55c77f898250 2024-04-21T08:47:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.242+0000 7ff204ff9640 1 -- 172.21.15.43:0/1937100981 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff200085ae0 con 0x55c77f898250 2024-04-21T08:47:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.242+0000 7ff217c26740 1 -- 172.21.15.43:0/1937100981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898250 msgr2=0x55c77f8b1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.242+0000 7ff217c26740 1 --2- 172.21.15.43:0/1937100981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898250 0x55c77f8b1860 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7ff200098f70 tx=0x7ff2000d4130 comp rx=0 tx=0).stop 2024-04-21T08:47:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.242+0000 7ff217c26740 1 -- 172.21.15.43:0/1937100981 shutdown_connections 2024-04-21T08:47:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.242+0000 7ff217c26740 1 --2- 172.21.15.43:0/1937100981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c77f8a5e10 0x55c77f8a56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.242+0000 7ff217c26740 1 --2- 172.21.15.43:0/1937100981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77f898c00 0x55c77f8a5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.242+0000 7ff217c26740 1 --2- 172.21.15.43:0/1937100981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898250 0x55c77f8b1860 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.242+0000 7ff217c26740 1 -- 172.21.15.43:0/1937100981 >> 172.21.15.43:0/1937100981 conn(0x55c77f892ba0 msgr2=0x55c77f8b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.242+0000 7ff217c26740 1 -- 172.21.15.43:0/1937100981 shutdown_connections 2024-04-21T08:47:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.242+0000 7ff217c26740 1 -- 172.21.15.43:0/1937100981 wait complete. 2024-04-21T08:47:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.243+0000 7ff217c26740 1 -- start start 2024-04-21T08:47:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.243+0000 7ff217c26740 1 Processor -- start 2024-04-21T08:47:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.243+0000 7ff217c26740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c77f898250 0x55c77f8b8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.243+0000 7ff217c26740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898c00 0x55c77f8b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.243+0000 7ff217c26740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77f8a5e10 0x55c77f914080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.243+0000 7ff217c26740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff2101ae730 con 0x55c77f898c00 2024-04-21T08:47:15.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.243+0000 7ff217c26740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2101ae060 con 0x55c77f8a5e10 2024-04-21T08:47:15.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.243+0000 7ff217c26740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff2101ab2c0 con 0x55c77f898250 2024-04-21T08:47:15.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.243+0000 7ff217c26740 10 client.0 _ll_register_callbacks cb 0x7ff20004d0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:15.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.243+0000 7ff207fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898c00 0x55c77f8b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.243+0000 7ff207fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898c00 0x55c77f8b8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43908/0 (socket says 172.21.15.43:43908) 2024-04-21T08:47:15.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.243+0000 7ff207fff640 1 -- 172.21.15.43:0/632102001 learned_addr learned my addr 172.21.15.43:0/632102001 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:15.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.243+0000 7ff20f59e640 1 --2- 172.21.15.43:0/632102001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77f8a5e10 0x55c77f914080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.243+0000 7ff20ed9d640 1 --2- 172.21.15.43:0/632102001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c77f898250 0x55c77f8b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.243+0000 7ff207fff640 1 -- 172.21.15.43:0/632102001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c77f898250 msgr2=0x55c77f8b8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:15.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.243+0000 7ff207fff640 1 --2- 172.21.15.43:0/632102001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c77f898250 0x55c77f8b8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.243+0000 7ff207fff640 1 -- 172.21.15.43:0/632102001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77f8a5e10 msgr2=0x55c77f914080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:15.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.243+0000 7ff207fff640 1 --2- 172.21.15.43:0/632102001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77f8a5e10 0x55c77f914080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.243+0000 7ff207fff640 1 -- 172.21.15.43:0/632102001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c77f914640 con 0x55c77f898c00 2024-04-21T08:47:15.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.244+0000 7ff207fff640 1 --2- 172.21.15.43:0/632102001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898c00 0x55c77f8b8ac0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009230 tx=0x7ff1f8005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=472a4015310294aa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:15.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.244+0000 7ff1f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:15.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.244+0000 7ff1f67fc640 1 -- 172.21.15.43:0/632102001 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8048dd0 con 0x55c77f898c00 2024-04-21T08:47:15.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.244+0000 7ff1f67fc640 1 -- 172.21.15.43:0/632102001 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7ff1f802c7e0 con 0x55c77f898c00 2024-04-21T08:47:15.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.244+0000 7ff217c26740 1 -- 172.21.15.43:0/632102001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2000a0950 con 0x55c77f898c00 2024-04-21T08:47:15.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.244+0000 7ff1f67fc640 1 -- 172.21.15.43:0/632102001 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff1f8056050 con 0x55c77f898c00 2024-04-21T08:47:15.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.244+0000 7ff217c26740 10 client.19428 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:15.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.244+0000 7ff217c26740 1 -- 172.21.15.43:0/632102001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c77f91ba80 con 0x55c77f898c00 2024-04-21T08:47:15.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.244+0000 7ff1f67fc640 1 -- 172.21.15.43:0/632102001 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2000a0950 con 0x55c77f898c00 2024-04-21T08:47:15.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.244+0000 7ff1f67fc640 1 -- 172.21.15.43:0/632102001 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff1f8005340 con 0x55c77f898c00 2024-04-21T08:47:15.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.244+0000 7ff217c26740 10 client.19428 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:15.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.244+0000 7ff217c26740 10 client.19428 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:47:15.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.244+0000 7ff217c26740 20 client.19428 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:15.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.244+0000 7ff217c26740 1 --2- 172.21.15.43:0/632102001 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55c77f91c230 0x55c77f91e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.244+0000 7ff217c26740 4 client.19428 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:15.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.244+0000 7ff217c26740 1 -- 172.21.15.43:0/632102001 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c77f804a80 con 0x55c77f91c230 2024-04-21T08:47:15.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.245+0000 7ff20ed9d640 1 --2- 172.21.15.43:0/632102001 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55c77f91c230 0x55c77f91e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.245+0000 7ff20ed9d640 1 --2- 172.21.15.43:0/632102001 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55c77f91c230 0x55c77f91e650 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3bfe5cac00f38e8e server_cookie=97aa9cd825968733 in_seq=0 out_seq=0 2024-04-21T08:47:15.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.245+0000 7ff1f67fc640 10 client.19428 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:47:15.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.245+0000 7ff1f67fc640 1 -- 172.21.15.43:0/632102001 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c77f804a80 con 0x55c77f91c230 2024-04-21T08:47:15.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.245+0000 7ff1f67fc640 10 client.19428 handle_command_reply: tid=0 2024-04-21T08:47:15.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.245+0000 7ff217c26740 1 client.19428 shutdown 2024-04-21T08:47:15.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.245+0000 7ff217c26740 1 -- 172.21.15.43:0/632102001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898c00 msgr2=0x55c77f8b8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.245+0000 7ff217c26740 1 --2- 172.21.15.43:0/632102001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898c00 0x55c77f8b8ac0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009230 tx=0x7ff1f8005a40 comp rx=0 tx=0).stop 2024-04-21T08:47:15.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.245+0000 7ff217c26740 1 -- 172.21.15.43:0/632102001 shutdown_connections 2024-04-21T08:47:15.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.245+0000 7ff217c26740 1 --2- 172.21.15.43:0/632102001 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55c77f91c230 0x55c77f91e650 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.245+0000 7ff217c26740 1 --2- 172.21.15.43:0/632102001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c77f8a5e10 0x55c77f914080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.245+0000 7ff217c26740 1 --2- 172.21.15.43:0/632102001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c77f898c00 0x55c77f8b8ac0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.245+0000 7ff217c26740 1 --2- 172.21.15.43:0/632102001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c77f898250 0x55c77f8b8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.245+0000 7ff1f67fc640 0 client.19428 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:47:15.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.245+0000 7ff217c26740 1 -- 172.21.15.43:0/632102001 >> 172.21.15.43:0/632102001 conn(0x55c77f892ba0 msgr2=0x55c77f89e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:15.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.246+0000 7ff217c26740 1 -- 172.21.15.43:0/632102001 shutdown_connections 2024-04-21T08:47:15.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.246+0000 7ff217c26740 1 -- 172.21.15.43:0/632102001 wait complete. 2024-04-21T08:47:15.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.246+0000 7ff217c26740 20 client.19428 trim_cache size 0 max 16384 2024-04-21T08:47:15.251 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:47:15.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.246+0000 7ff215031640 1 -- 172.21.15.43:0/393543708 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff1fc0a3d20 msgr2=0x7ff1fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.246+0000 7ff215031640 1 --2- 172.21.15.43:0/393543708 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff1fc0a3d20 0x7ff1fc0a61e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff208009fd0 tx=0x7ff208005bd0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.246+0000 7ff215031640 1 -- 172.21.15.43:0/393543708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff21006a7c0 msgr2=0x7ff210198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.246+0000 7ff215031640 1 --2- 172.21.15.43:0/393543708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff21006a7c0 0x7ff210198700 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7ff20000c890 tx=0x7ff20000cd50 comp rx=0 tx=0).stop 2024-04-21T08:47:15.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.246+0000 7ff215031640 1 -- 172.21.15.43:0/393543708 shutdown_connections 2024-04-21T08:47:15.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.246+0000 7ff215031640 1 --2- 172.21.15.43:0/393543708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff21019cfc0 0x7ff21019d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.246+0000 7ff215031640 1 --2- 172.21.15.43:0/393543708 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff1fc0a3d20 0x7ff1fc0a61e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.246+0000 7ff215031640 1 --2- 172.21.15.43:0/393543708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff210107280 0x7ff210198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.246+0000 7ff215031640 1 --2- 172.21.15.43:0/393543708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff21006a7c0 0x7ff210198700 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.246+0000 7ff215031640 1 -- 172.21.15.43:0/393543708 >> 172.21.15.43:0/393543708 conn(0x7ff210076c50 msgr2=0x7ff2100ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:15.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.246+0000 7ff215031640 1 -- 172.21.15.43:0/393543708 shutdown_connections 2024-04-21T08:47:15.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.246+0000 7ff215031640 1 -- 172.21.15.43:0/393543708 wait complete. 2024-04-21T08:47:15.255 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:15 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/4287231420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:15.256 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:47:15.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.361+0000 7f28f2f2e640 1 -- 172.21.15.43:0/606926527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28ec102190 msgr2=0x7f28ec102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.361+0000 7f28f2f2e640 1 --2- 172.21.15.43:0/606926527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28ec102190 0x7f28ec102570 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009900 tx=0x7f28dc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:15.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.361+0000 7f28f2f2e640 1 -- 172.21.15.43:0/606926527 shutdown_connections 2024-04-21T08:47:15.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.361+0000 7f28f2f2e640 1 --2- 172.21.15.43:0/606926527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28ec107260 0x7f28ec10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.361+0000 7f28f2f2e640 1 --2- 172.21.15.43:0/606926527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ec102b40 0x7f28ec106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.361+0000 7f28f2f2e640 1 --2- 172.21.15.43:0/606926527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28ec102190 0x7f28ec102570 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.361+0000 7f28f2f2e640 1 -- 172.21.15.43:0/606926527 >> 172.21.15.43:0/606926527 conn(0x7f28ec0fd990 msgr2=0x7f28ec0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:15.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.361+0000 7f28f2f2e640 1 -- 172.21.15.43:0/606926527 shutdown_connections 2024-04-21T08:47:15.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.361+0000 7f28f2f2e640 1 -- 172.21.15.43:0/606926527 wait complete. 2024-04-21T08:47:15.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.361+0000 7f28f2f2e640 1 Processor -- start 2024-04-21T08:47:15.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.361+0000 7f28f2f2e640 1 -- start start 2024-04-21T08:47:15.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.361+0000 7f28f2f2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28ec102190 0x7f28ec19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.362+0000 7f28f2f2e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ec107260 0x7f28ec19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.362+0000 7f28f2f2e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28ec1985f0 0x7f28ec198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.362+0000 7f28f2f2e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1121d0 con 0x7f28ec102190 2024-04-21T08:47:15.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.362+0000 7f28f2f2e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec112050 con 0x7f28ec107260 2024-04-21T08:47:15.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.362+0000 7f28f2f2e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec112350 con 0x7f28ec1985f0 2024-04-21T08:47:15.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.362+0000 7f28f0cca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28ec102190 0x7f28ec19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.362+0000 7f28f0cca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28ec102190 0x7f28ec19e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43926/0 (socket says 172.21.15.43:43926) 2024-04-21T08:47:15.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.362+0000 7f28f0cca640 1 -- 172.21.15.43:0/1012188267 learned_addr learned my addr 172.21.15.43:0/1012188267 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:15.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.362+0000 7f28ebfff640 1 --2- 172.21.15.43:0/1012188267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ec107260 0x7f28ec19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.362+0000 7f28f0cca640 1 -- 172.21.15.43:0/1012188267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28ec1985f0 msgr2=0x7f28ec198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.362+0000 7f28f14cb640 1 --2- 172.21.15.43:0/1012188267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28ec1985f0 0x7f28ec198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.362+0000 7f28f0cca640 1 --2- 172.21.15.43:0/1012188267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28ec1985f0 0x7f28ec198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.362+0000 7f28f0cca640 1 -- 172.21.15.43:0/1012188267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ec107260 msgr2=0x7f28ec19e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.362+0000 7f28f0cca640 1 --2- 172.21.15.43:0/1012188267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ec107260 0x7f28ec19e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.362+0000 7f28f0cca640 1 -- 172.21.15.43:0/1012188267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ec199110 con 0x7f28ec102190 2024-04-21T08:47:15.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.362+0000 7f28ebfff640 1 --2- 172.21.15.43:0/1012188267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ec107260 0x7f28ec19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:15.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.362+0000 7f28f0cca640 1 --2- 172.21.15.43:0/1012188267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28ec102190 0x7f28ec19e3f0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f28dc005940 tx=0x7f28dc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:15.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.362+0000 7f28e9ffb640 1 -- 172.21.15.43:0/1012188267 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc043020 con 0x7f28ec102190 2024-04-21T08:47:15.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.362+0000 7f28e9ffb640 1 -- 172.21.15.43:0/1012188267 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28dc02caf0 con 0x7f28ec102190 2024-04-21T08:47:15.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.362+0000 7f28f2f2e640 1 -- 172.21.15.43:0/1012188267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28ec1993a0 con 0x7f28ec102190 2024-04-21T08:47:15.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.362+0000 7f28e9ffb640 1 -- 172.21.15.43:0/1012188267 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc003af0 con 0x7f28ec102190 2024-04-21T08:47:15.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.363+0000 7f28f2f2e640 1 -- 172.21.15.43:0/1012188267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28ec1a55a0 con 0x7f28ec102190 2024-04-21T08:47:15.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.365+0000 7f28e9ffb640 1 -- 172.21.15.43:0/1012188267 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f28dc046020 con 0x7f28ec102190 2024-04-21T08:47:15.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.365+0000 7f28f5bf3740 1 Processor -- start 2024-04-21T08:47:15.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.366+0000 7f28f5bf3740 1 -- start start 2024-04-21T08:47:15.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.366+0000 7f28f5bf3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e753b0 0x5589a1e75790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.366+0000 7f28f5bf3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589a1e75d60 0x5589a1e7e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.366+0000 7f28f5bf3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589a1e7ee60 0x5589a1e7f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.366+0000 7f28f5bf3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5589a1dda1f0 con 0x5589a1e753b0 2024-04-21T08:47:15.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.366+0000 7f28f5bf3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5589a1d774b0 con 0x5589a1e7ee60 2024-04-21T08:47:15.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.366+0000 7f28f5bf3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5589a1dd61b0 con 0x5589a1e75d60 2024-04-21T08:47:15.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.366+0000 7f28f14cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e753b0 0x5589a1e75790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.366+0000 7f28f14cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e753b0 0x5589a1e75790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43938/0 (socket says 172.21.15.43:43938) 2024-04-21T08:47:15.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.366+0000 7f28f14cb640 1 -- 172.21.15.43:0/2372868364 learned_addr learned my addr 172.21.15.43:0/2372868364 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:15.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.366+0000 7f28f0cca640 1 --2- 172.21.15.43:0/2372868364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589a1e75d60 0x5589a1e7e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.366+0000 7f28e9ffb640 1 --2- 172.21.15.43:0/1012188267 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f28d40a3d20 0x7f28d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.366+0000 7f28ebfff640 1 --2- 172.21.15.43:0/2372868364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589a1e7ee60 0x5589a1e7f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.366+0000 7f28f14cb640 1 -- 172.21.15.43:0/2372868364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589a1e75d60 msgr2=0x5589a1e7e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:15.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.366+0000 7f28f14cb640 1 --2- 172.21.15.43:0/2372868364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589a1e75d60 0x5589a1e7e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.366+0000 7f28f14cb640 1 -- 172.21.15.43:0/2372868364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589a1e7ee60 msgr2=0x5589a1e7f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:15.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.366+0000 7f28f14cb640 1 --2- 172.21.15.43:0/2372868364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589a1e7ee60 0x5589a1e7f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.366+0000 7f28f14cb640 1 -- 172.21.15.43:0/2372868364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589a1d691b0 con 0x5589a1e753b0 2024-04-21T08:47:15.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.366+0000 7f28ebfff640 1 --2- 172.21.15.43:0/2372868364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589a1e7ee60 0x5589a1e7f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:15.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.366+0000 7f28f0cca640 1 --2- 172.21.15.43:0/2372868364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589a1e75d60 0x5589a1e7e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:15.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.366+0000 7f28e9ffb640 1 -- 172.21.15.43:0/1012188267 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f28dc0f2db0 con 0x7f28ec102190 2024-04-21T08:47:15.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.366+0000 7f28f14cb640 1 --2- 172.21.15.43:0/1012188267 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f28d40a3d20 0x7f28d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.366+0000 7f28f14cb640 1 --2- 172.21.15.43:0/2372868364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e753b0 0x5589a1e75790 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009fb0 tx=0x7f28e400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e7aa441f1a5ba1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:15.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.367+0000 7f28d2ffd640 1 -- 172.21.15.43:0/2372868364 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4012e40 con 0x5589a1e753b0 2024-04-21T08:47:15.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.367+0000 7f28d2ffd640 1 -- 172.21.15.43:0/2372868364 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28e401c8b0 con 0x5589a1e753b0 2024-04-21T08:47:15.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.367+0000 7f28d2ffd640 1 -- 172.21.15.43:0/2372868364 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e401b760 con 0x5589a1e753b0 2024-04-21T08:47:15.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.367+0000 7f28f5bf3740 1 -- 172.21.15.43:0/2372868364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e753b0 msgr2=0x5589a1e75790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.367+0000 7f28f14cb640 1 --2- 172.21.15.43:0/1012188267 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f28d40a3d20 0x7f28d40a61e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f28e4020000 tx=0x7f28e401d000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:15.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.367+0000 7f28f5bf3740 1 --2- 172.21.15.43:0/2372868364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e753b0 0x5589a1e75790 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009fb0 tx=0x7f28e400f7f0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.367+0000 7f28f5bf3740 1 -- 172.21.15.43:0/2372868364 shutdown_connections 2024-04-21T08:47:15.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.367+0000 7f28f5bf3740 1 --2- 172.21.15.43:0/2372868364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589a1e7ee60 0x5589a1e7f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.367+0000 7f28f5bf3740 1 --2- 172.21.15.43:0/2372868364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589a1e75d60 0x5589a1e7e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.367+0000 7f28f5bf3740 1 --2- 172.21.15.43:0/2372868364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e753b0 0x5589a1e75790 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.367+0000 7f28f5bf3740 1 -- 172.21.15.43:0/2372868364 >> 172.21.15.43:0/2372868364 conn(0x5589a1e6fba0 msgr2=0x5589a1e6ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.367+0000 7f28f5bf3740 1 -- 172.21.15.43:0/2372868364 shutdown_connections 2024-04-21T08:47:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.367+0000 7f28f5bf3740 1 -- 172.21.15.43:0/2372868364 wait complete. 2024-04-21T08:47:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.368+0000 7f28f5bf3740 1 -- start start 2024-04-21T08:47:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.368+0000 7f28f5bf3740 1 Processor -- start 2024-04-21T08:47:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.368+0000 7f28f5bf3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e753b0 0x5589a1e85220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.368+0000 7f28f5bf3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589a1e75d60 0x5589a1e825b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.368+0000 7f28f5bf3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589a1e7ee60 0x5589a1e82af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.368+0000 7f28f5bf3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5589a1e85da0 con 0x5589a1e753b0 2024-04-21T08:47:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.368+0000 7f28f5bf3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5589a1d93500 con 0x5589a1e7ee60 2024-04-21T08:47:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.368+0000 7f28f5bf3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5589a1e85f20 con 0x5589a1e75d60 2024-04-21T08:47:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.368+0000 7f28f5bf3740 10 client.0 _ll_register_callbacks cb 0x5589a1d61b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.368+0000 7f28f0cca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e753b0 0x5589a1e85220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.368+0000 7f28f0cca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e753b0 0x5589a1e85220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43948/0 (socket says 172.21.15.43:43948) 2024-04-21T08:47:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.368+0000 7f28f0cca640 1 -- 172.21.15.43:0/84836379 learned_addr learned my addr 172.21.15.43:0/84836379 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.369+0000 7f28f14cb640 1 --2- 172.21.15.43:0/84836379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589a1e7ee60 0x5589a1e82af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.369+0000 7f28f0cca640 1 -- 172.21.15.43:0/84836379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589a1e75d60 msgr2=0x5589a1e825b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.369+0000 7f28f0cca640 1 --2- 172.21.15.43:0/84836379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589a1e75d60 0x5589a1e825b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.369+0000 7f28f0cca640 1 -- 172.21.15.43:0/84836379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589a1e7ee60 msgr2=0x5589a1e82af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.369+0000 7f28f0cca640 1 --2- 172.21.15.43:0/84836379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589a1e7ee60 0x5589a1e82af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.369+0000 7f28f0cca640 1 -- 172.21.15.43:0/84836379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589a1aec3f0 con 0x5589a1e753b0 2024-04-21T08:47:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.369+0000 7f28f14cb640 1 --2- 172.21.15.43:0/84836379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589a1e7ee60 0x5589a1e82af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.369+0000 7f28f0cca640 1 --2- 172.21.15.43:0/84836379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e753b0 0x5589a1e85220 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f28dc005ec0 tx=0x7f28dc0be870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6433ff0e128d9313 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.369+0000 7f28cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.369+0000 7f28cbfff640 1 -- 172.21.15.43:0/84836379 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc0c7650 con 0x5589a1e753b0 2024-04-21T08:47:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.369+0000 7f28cbfff640 1 -- 172.21.15.43:0/84836379 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f28dc0f0300 con 0x5589a1e753b0 2024-04-21T08:47:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.369+0000 7f28f5bf3740 1 -- 172.21.15.43:0/84836379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28e401c8b0 con 0x5589a1e753b0 2024-04-21T08:47:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.369+0000 7f28cbfff640 1 -- 172.21.15.43:0/84836379 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f28dc0f0500 con 0x5589a1e753b0 2024-04-21T08:47:15.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.369+0000 7f28f5bf3740 10 client.19440 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:15.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.369+0000 7f28f5bf3740 1 -- 172.21.15.43:0/84836379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5589a1bb67d0 con 0x5589a1e753b0 2024-04-21T08:47:15.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.369+0000 7f28cbfff640 1 -- 172.21.15.43:0/84836379 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28e401c8b0 con 0x5589a1e753b0 2024-04-21T08:47:15.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.369+0000 7f28cbfff640 1 -- 172.21.15.43:0/84836379 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f28dc0ddba0 con 0x5589a1e753b0 2024-04-21T08:47:15.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.369+0000 7f28f5bf3740 10 client.19440 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:15.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.369+0000 7f28f5bf3740 10 client.19440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:15.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.370+0000 7f28f5bf3740 20 client.19440 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:15.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.370+0000 7f28f5bf3740 1 --2- 172.21.15.43:0/84836379 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5589a1e92ec0 0x5589a1e95330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.370+0000 7f28f5bf3740 4 client.19440 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:15.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.370+0000 7f28f5bf3740 1 -- 172.21.15.43:0/84836379 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589a1d79930 con 0x5589a1e92ec0 2024-04-21T08:47:15.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.370+0000 7f28ebfff640 1 --2- 172.21.15.43:0/84836379 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5589a1e92ec0 0x5589a1e95330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.370+0000 7f28ebfff640 1 --2- 172.21.15.43:0/84836379 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5589a1e92ec0 0x5589a1e95330 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce6f618cab207c5a server_cookie=e30f237f618dfac0 in_seq=0 out_seq=0 2024-04-21T08:47:15.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.370+0000 7f28cbfff640 10 client.19440 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:15.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.371+0000 7f28cbfff640 1 -- 172.21.15.43:0/84836379 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5589a1d79930 con 0x5589a1e92ec0 2024-04-21T08:47:15.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.371+0000 7f28cbfff640 10 client.19440 handle_command_reply: tid=0 2024-04-21T08:47:15.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.371+0000 7f28f5bf3740 1 client.19440 shutdown 2024-04-21T08:47:15.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.371+0000 7f28f5bf3740 1 -- 172.21.15.43:0/84836379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e753b0 msgr2=0x5589a1e85220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.371+0000 7f28f5bf3740 1 --2- 172.21.15.43:0/84836379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e753b0 0x5589a1e85220 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f28dc005ec0 tx=0x7f28dc0be870 comp rx=0 tx=0).stop 2024-04-21T08:47:15.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.371+0000 7f28f5bf3740 1 -- 172.21.15.43:0/84836379 shutdown_connections 2024-04-21T08:47:15.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.371+0000 7f28f5bf3740 1 --2- 172.21.15.43:0/84836379 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5589a1e92ec0 0x5589a1e95330 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.371+0000 7f28f5bf3740 1 --2- 172.21.15.43:0/84836379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589a1e7ee60 0x5589a1e82af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.371+0000 7f28f5bf3740 1 --2- 172.21.15.43:0/84836379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589a1e75d60 0x5589a1e825b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.371+0000 7f28cbfff640 0 client.19440 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:15.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.371+0000 7f28f5bf3740 1 --2- 172.21.15.43:0/84836379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e753b0 0x5589a1e85220 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.371+0000 7f28f5bf3740 1 -- 172.21.15.43:0/84836379 >> 172.21.15.43:0/84836379 conn(0x5589a1e6fba0 msgr2=0x5589a1e718f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:15.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.371+0000 7f28f5bf3740 1 -- 172.21.15.43:0/84836379 shutdown_connections 2024-04-21T08:47:15.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.371+0000 7f28f5bf3740 1 -- 172.21.15.43:0/84836379 wait complete. 2024-04-21T08:47:15.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.371+0000 7f28f5bf3740 20 client.19440 trim_cache size 0 max 16384 2024-04-21T08:47:15.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.385+0000 7f28f5bf3740 1 Processor -- start 2024-04-21T08:47:15.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.385+0000 7f28f5bf3740 1 -- start start 2024-04-21T08:47:15.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.385+0000 7f28f5bf3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589a1e753b0 0x5589a1e8ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.385+0000 7f28f5bf3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589a1e75d60 0x5589a1e7d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.385+0000 7f28f5bf3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e7ee60 0x5589a1e71670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.385+0000 7f28f5bf3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1ad2e0 con 0x5589a1e7ee60 2024-04-21T08:47:15.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.385+0000 7f28f5bf3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1aee20 con 0x5589a1e75d60 2024-04-21T08:47:15.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.385+0000 7f28f5bf3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1ad9b0 con 0x5589a1e753b0 2024-04-21T08:47:15.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.385+0000 7f28f14cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e7ee60 0x5589a1e71670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.385+0000 7f28f14cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e7ee60 0x5589a1e71670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43958/0 (socket says 172.21.15.43:43958) 2024-04-21T08:47:15.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.385+0000 7f28f14cb640 1 -- 172.21.15.43:0/4101007523 learned_addr learned my addr 172.21.15.43:0/4101007523 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:15.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.385+0000 7f28ebfff640 1 --2- 172.21.15.43:0/4101007523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589a1e75d60 0x5589a1e7d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.385+0000 7f28f0cca640 1 --2- 172.21.15.43:0/4101007523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589a1e753b0 0x5589a1e8ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.385+0000 7f28f14cb640 1 -- 172.21.15.43:0/4101007523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589a1e753b0 msgr2=0x5589a1e8ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:15.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.385+0000 7f28f14cb640 1 --2- 172.21.15.43:0/4101007523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589a1e753b0 0x5589a1e8ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.385+0000 7f28f14cb640 1 -- 172.21.15.43:0/4101007523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589a1e75d60 msgr2=0x5589a1e7d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:15.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.385+0000 7f28f14cb640 1 --2- 172.21.15.43:0/4101007523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589a1e75d60 0x5589a1e7d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.385+0000 7f28f14cb640 1 -- 172.21.15.43:0/4101007523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589a1e959b0 con 0x5589a1e7ee60 2024-04-21T08:47:15.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.385+0000 7f28f0cca640 1 --2- 172.21.15.43:0/4101007523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589a1e753b0 0x5589a1e8ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:15.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.385+0000 7f28f14cb640 1 --2- 172.21.15.43:0/4101007523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e7ee60 0x5589a1e71670 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009540 tx=0x7f28e4027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ebd95ff068f8bf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:15.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.386+0000 7f28d27fc640 1 -- 172.21.15.43:0/4101007523 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4035070 con 0x5589a1e7ee60 2024-04-21T08:47:15.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.386+0000 7f28d27fc640 1 -- 172.21.15.43:0/4101007523 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28e40098e0 con 0x5589a1e7ee60 2024-04-21T08:47:15.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.386+0000 7f28d27fc640 1 -- 172.21.15.43:0/4101007523 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4003a60 con 0x5589a1e7ee60 2024-04-21T08:47:15.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.386+0000 7f28f5bf3740 1 -- 172.21.15.43:0/4101007523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e7ee60 msgr2=0x5589a1e71670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.386+0000 7f28f5bf3740 1 --2- 172.21.15.43:0/4101007523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e7ee60 0x5589a1e71670 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009540 tx=0x7f28e4027000 comp rx=0 tx=0).stop 2024-04-21T08:47:15.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.386+0000 7f28f5bf3740 1 -- 172.21.15.43:0/4101007523 shutdown_connections 2024-04-21T08:47:15.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.386+0000 7f28f5bf3740 1 --2- 172.21.15.43:0/4101007523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e7ee60 0x5589a1e71670 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.386+0000 7f28f5bf3740 1 --2- 172.21.15.43:0/4101007523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589a1e75d60 0x5589a1e7d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.386+0000 7f28f5bf3740 1 --2- 172.21.15.43:0/4101007523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589a1e753b0 0x5589a1e8ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.386+0000 7f28f5bf3740 1 -- 172.21.15.43:0/4101007523 >> 172.21.15.43:0/4101007523 conn(0x5589a1e6fba0 msgr2=0x5589a1e70f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:15.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.386+0000 7f28f5bf3740 1 -- 172.21.15.43:0/4101007523 shutdown_connections 2024-04-21T08:47:15.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.386+0000 7f28f5bf3740 1 -- 172.21.15.43:0/4101007523 wait complete. 2024-04-21T08:47:15.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.387+0000 7f28f5bf3740 1 -- start start 2024-04-21T08:47:15.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.387+0000 7f28f5bf3740 1 Processor -- start 2024-04-21T08:47:15.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.387+0000 7f28f5bf3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589a1e753b0 0x5589a1e842e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.387+0000 7f28f5bf3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589a1e75d60 0x5589a1e84820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.387+0000 7f28f5bf3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e7ee60 0x5589a1e84d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.387+0000 7f28f5bf3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1ae750 con 0x5589a1e7ee60 2024-04-21T08:47:15.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.387+0000 7f28f5bf3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1ae080 con 0x5589a1e75d60 2024-04-21T08:47:15.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.387+0000 7f28f5bf3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1ab2e0 con 0x5589a1e753b0 2024-04-21T08:47:15.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.387+0000 7f28f5bf3740 10 client.0 _ll_register_callbacks cb 0x7f28dc0be520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:15.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.387+0000 7f28f14cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e7ee60 0x5589a1e84d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.387+0000 7f28f14cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e7ee60 0x5589a1e84d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43972/0 (socket says 172.21.15.43:43972) 2024-04-21T08:47:15.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.387+0000 7f28f14cb640 1 -- 172.21.15.43:0/3116928382 learned_addr learned my addr 172.21.15.43:0/3116928382 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:15.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.387+0000 7f28ebfff640 1 --2- 172.21.15.43:0/3116928382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589a1e75d60 0x5589a1e84820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.387+0000 7f28f0cca640 1 --2- 172.21.15.43:0/3116928382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589a1e753b0 0x5589a1e842e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.387+0000 7f28f14cb640 1 -- 172.21.15.43:0/3116928382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589a1e753b0 msgr2=0x5589a1e842e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:15.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.387+0000 7f28f14cb640 1 --2- 172.21.15.43:0/3116928382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589a1e753b0 0x5589a1e842e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.387+0000 7f28f14cb640 1 -- 172.21.15.43:0/3116928382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589a1e75d60 msgr2=0x5589a1e84820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:15.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.387+0000 7f28f14cb640 1 --2- 172.21.15.43:0/3116928382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589a1e75d60 0x5589a1e84820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.387+0000 7f28f14cb640 1 -- 172.21.15.43:0/3116928382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589a1ef85e0 con 0x5589a1e7ee60 2024-04-21T08:47:15.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.388+0000 7f28f14cb640 1 --2- 172.21.15.43:0/3116928382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e7ee60 0x5589a1e84d60 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f28e4012670 tx=0x7f28e4009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4415299482664edc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:15.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.388+0000 7f28cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:15.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.388+0000 7f28cbfff640 1 -- 172.21.15.43:0/3116928382 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4035040 con 0x5589a1e7ee60 2024-04-21T08:47:15.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.388+0000 7f28cbfff640 1 -- 172.21.15.43:0/3116928382 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f28e402d040 con 0x5589a1e7ee60 2024-04-21T08:47:15.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.388+0000 7f28f5bf3740 1 -- 172.21.15.43:0/3116928382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28e40098e0 con 0x5589a1e7ee60 2024-04-21T08:47:15.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.388+0000 7f28cbfff640 1 -- 172.21.15.43:0/3116928382 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f28e401ba80 con 0x5589a1e7ee60 2024-04-21T08:47:15.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.388+0000 7f28f5bf3740 10 client.19446 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:15.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.388+0000 7f28f5bf3740 1 -- 172.21.15.43:0/3116928382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5589a1ef9830 con 0x5589a1e7ee60 2024-04-21T08:47:15.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.388+0000 7f28cbfff640 1 -- 172.21.15.43:0/3116928382 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28e40098e0 con 0x5589a1e7ee60 2024-04-21T08:47:15.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.388+0000 7f28cbfff640 1 -- 172.21.15.43:0/3116928382 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f28e4005080 con 0x5589a1e7ee60 2024-04-21T08:47:15.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.388+0000 7f28f5bf3740 10 client.19446 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:15.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.388+0000 7f28f5bf3740 10 client.19446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:15.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.388+0000 7f28f5bf3740 20 client.19446 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:15.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.388+0000 7f28f5bf3740 1 --2- 172.21.15.43:0/3116928382 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5589a1ef9cf0 0x5589a1efc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.388+0000 7f28f5bf3740 4 client.19446 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:15.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.388+0000 7f28f5bf3740 1 -- 172.21.15.43:0/3116928382 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5589a1e82e80 con 0x5589a1ef9cf0 2024-04-21T08:47:15.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.389+0000 7f28f0cca640 1 --2- 172.21.15.43:0/3116928382 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5589a1ef9cf0 0x5589a1efc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.389+0000 7f28f0cca640 1 --2- 172.21.15.43:0/3116928382 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5589a1ef9cf0 0x5589a1efc0e0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eff493e8846682fa server_cookie=22face367405e4c9 in_seq=0 out_seq=0 2024-04-21T08:47:15.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.389+0000 7f28cbfff640 10 client.19446 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:15.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.389+0000 7f28cbfff640 1 -- 172.21.15.43:0/3116928382 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5589a1e82e80 con 0x5589a1ef9cf0 2024-04-21T08:47:15.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.389+0000 7f28cbfff640 10 client.19446 handle_command_reply: tid=0 2024-04-21T08:47:15.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.389+0000 7f28f5bf3740 1 client.19446 shutdown 2024-04-21T08:47:15.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.389+0000 7f28f5bf3740 1 -- 172.21.15.43:0/3116928382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e7ee60 msgr2=0x5589a1e84d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.389+0000 7f28f5bf3740 1 --2- 172.21.15.43:0/3116928382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e7ee60 0x5589a1e84d60 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f28e4012670 tx=0x7f28e4009dc0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.389+0000 7f28f5bf3740 1 -- 172.21.15.43:0/3116928382 shutdown_connections 2024-04-21T08:47:15.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.389+0000 7f28f5bf3740 1 --2- 172.21.15.43:0/3116928382 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5589a1ef9cf0 0x5589a1efc0e0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.389+0000 7f28f5bf3740 1 --2- 172.21.15.43:0/3116928382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589a1e7ee60 0x5589a1e84d60 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.389+0000 7f28f5bf3740 1 --2- 172.21.15.43:0/3116928382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589a1e75d60 0x5589a1e84820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.389+0000 7f28cbfff640 0 client.19446 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:15.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.389+0000 7f28f5bf3740 1 --2- 172.21.15.43:0/3116928382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589a1e753b0 0x5589a1e842e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.389+0000 7f28f5bf3740 1 -- 172.21.15.43:0/3116928382 >> 172.21.15.43:0/3116928382 conn(0x5589a1e6fba0 msgr2=0x5589a1e6ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:15.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.389+0000 7f28f5bf3740 1 -- 172.21.15.43:0/3116928382 shutdown_connections 2024-04-21T08:47:15.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.390+0000 7f28f5bf3740 1 -- 172.21.15.43:0/3116928382 wait complete. 2024-04-21T08:47:15.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.390+0000 7f28f5bf3740 20 client.19446 trim_cache size 0 max 16384 2024-04-21T08:47:15.396 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:47:15.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.390+0000 7f28f2f2e640 1 -- 172.21.15.43:0/1012188267 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f28d40a3d20 msgr2=0x7f28d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.390+0000 7f28f2f2e640 1 --2- 172.21.15.43:0/1012188267 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f28d40a3d20 0x7f28d40a61e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f28e4020000 tx=0x7f28e401d000 comp rx=0 tx=0).stop 2024-04-21T08:47:15.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.390+0000 7f28f2f2e640 1 -- 172.21.15.43:0/1012188267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28ec102190 msgr2=0x7f28ec19e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.390+0000 7f28f2f2e640 1 --2- 172.21.15.43:0/1012188267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28ec102190 0x7f28ec19e3f0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f28dc005940 tx=0x7f28dc0045e0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.390+0000 7f28f2f2e640 1 -- 172.21.15.43:0/1012188267 shutdown_connections 2024-04-21T08:47:15.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.390+0000 7f28f2f2e640 1 --2- 172.21.15.43:0/1012188267 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f28d40a3d20 0x7f28d40a61e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.390+0000 7f28f2f2e640 1 --2- 172.21.15.43:0/1012188267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28ec1985f0 0x7f28ec198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.390+0000 7f28f2f2e640 1 --2- 172.21.15.43:0/1012188267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28ec107260 0x7f28ec19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.390+0000 7f28f2f2e640 1 --2- 172.21.15.43:0/1012188267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28ec102190 0x7f28ec19e3f0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.390+0000 7f28f2f2e640 1 -- 172.21.15.43:0/1012188267 >> 172.21.15.43:0/1012188267 conn(0x7f28ec0fd990 msgr2=0x7f28ec0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:15.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.390+0000 7f28f2f2e640 1 -- 172.21.15.43:0/1012188267 shutdown_connections 2024-04-21T08:47:15.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:15.390+0000 7f28f2f2e640 1 -- 172.21.15.43:0/1012188267 wait complete. 2024-04-21T08:47:15.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:15 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/4287231420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:15.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:15 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/4287231420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:16.401 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:47:16.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:47:16.402 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:47:16.429 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:16 smithi043 ceph-mon[27978]: pgmap v258: 129 pgs: 129 active+clean; 4.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 83 KiB/s wr, 22 op/s 2024-04-21T08:47:16.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:16 smithi175 ceph-mon[31920]: pgmap v258: 129 pgs: 129 active+clean; 4.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 83 KiB/s wr, 22 op/s 2024-04-21T08:47:16.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.521+0000 7fa03624e640 1 -- 172.21.15.43:0/1183967761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa030102190 msgr2=0x7fa030102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.521+0000 7fa03624e640 1 --2- 172.21.15.43:0/1183967761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa030102190 0x7fa030102570 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fa020009900 tx=0x7fa02002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:16.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.521+0000 7fa03624e640 1 -- 172.21.15.43:0/1183967761 shutdown_connections 2024-04-21T08:47:16.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.521+0000 7fa03624e640 1 --2- 172.21.15.43:0/1183967761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa030107260 0x7fa03010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.521+0000 7fa03624e640 1 --2- 172.21.15.43:0/1183967761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa030102b40 0x7fa030106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.521+0000 7fa03624e640 1 --2- 172.21.15.43:0/1183967761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa030102190 0x7fa030102570 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.521+0000 7fa03624e640 1 -- 172.21.15.43:0/1183967761 >> 172.21.15.43:0/1183967761 conn(0x7fa0300fd990 msgr2=0x7fa0300ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:16.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.521+0000 7fa03624e640 1 -- 172.21.15.43:0/1183967761 shutdown_connections 2024-04-21T08:47:16.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.521+0000 7fa03624e640 1 -- 172.21.15.43:0/1183967761 wait complete. 2024-04-21T08:47:16.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.522+0000 7fa03624e640 1 Processor -- start 2024-04-21T08:47:16.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.522+0000 7fa03624e640 1 -- start start 2024-04-21T08:47:16.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.522+0000 7fa03624e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa030102190 0x7fa030198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.522+0000 7fa03624e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa030107260 0x7fa030198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.522+0000 7fa03624e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa03019d020 0x7fa03019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.522+0000 7fa03624e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa0301121a0 con 0x7fa030102190 2024-04-21T08:47:16.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.522+0000 7fa03624e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa030112020 con 0x7fa030107260 2024-04-21T08:47:16.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.522+0000 7fa03624e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa030112320 con 0x7fa03019d020 2024-04-21T08:47:16.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.522+0000 7fa02f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa030102190 0x7fa030198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.523+0000 7fa02f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa030102190 0x7fa030198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44002/0 (socket says 172.21.15.43:44002) 2024-04-21T08:47:16.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.523+0000 7fa02f7fe640 1 -- 172.21.15.43:0/2115926734 learned_addr learned my addr 172.21.15.43:0/2115926734 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:16.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.523+0000 7fa02ffff640 1 --2- 172.21.15.43:0/2115926734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa03019d020 0x7fa03019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.523+0000 7fa02effd640 1 --2- 172.21.15.43:0/2115926734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa030107260 0x7fa030198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.523+0000 7fa02f7fe640 1 -- 172.21.15.43:0/2115926734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa03019d020 msgr2=0x7fa03019d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.523+0000 7fa02f7fe640 1 --2- 172.21.15.43:0/2115926734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa03019d020 0x7fa03019d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.523+0000 7fa02f7fe640 1 -- 172.21.15.43:0/2115926734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa030107260 msgr2=0x7fa030198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.523+0000 7fa02f7fe640 1 --2- 172.21.15.43:0/2115926734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa030107260 0x7fa030198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.523+0000 7fa02f7fe640 1 -- 172.21.15.43:0/2115926734 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa03019db10 con 0x7fa030102190 2024-04-21T08:47:16.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.523+0000 7fa02effd640 1 --2- 172.21.15.43:0/2115926734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa030107260 0x7fa030198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:16.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.523+0000 7fa02f7fe640 1 --2- 172.21.15.43:0/2115926734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa030102190 0x7fa030198760 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fa020005940 tx=0x7fa02002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:16.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.523+0000 7fa02cff9640 1 -- 172.21.15.43:0/2115926734 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa020043020 con 0x7fa030102190 2024-04-21T08:47:16.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.523+0000 7fa02cff9640 1 -- 172.21.15.43:0/2115926734 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa02002cbe0 con 0x7fa030102190 2024-04-21T08:47:16.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.523+0000 7fa02cff9640 1 -- 172.21.15.43:0/2115926734 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa020003bf0 con 0x7fa030102190 2024-04-21T08:47:16.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.523+0000 7fa03624e640 1 -- 172.21.15.43:0/2115926734 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa03019dda0 con 0x7fa030102190 2024-04-21T08:47:16.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.523+0000 7fa03624e640 1 -- 172.21.15.43:0/2115926734 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0301a55c0 con 0x7fa030102190 2024-04-21T08:47:16.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.526+0000 7fa02cff9640 1 -- 172.21.15.43:0/2115926734 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa020046020 con 0x7fa030102190 2024-04-21T08:47:16.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.526+0000 7fa038e43740 1 Processor -- start 2024-04-21T08:47:16.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.526+0000 7fa038e43740 1 -- start start 2024-04-21T08:47:16.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.526+0000 7fa038e43740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94e8e0 0x55e9cd94ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.526+0000 7fa038e43740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9cd94f290 0x55e9cd957840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.526+0000 7fa038e43740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9cd957e10 0x55e9cd95a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.526+0000 7fa038e43740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e9cd8b2f00 con 0x55e9cd94e8e0 2024-04-21T08:47:16.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.526+0000 7fa038e43740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e9cd850590 con 0x55e9cd957e10 2024-04-21T08:47:16.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.526+0000 7fa038e43740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e9cd8aeec0 con 0x55e9cd94f290 2024-04-21T08:47:16.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.526+0000 7fa02ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94e8e0 0x55e9cd94ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.526+0000 7fa02cff9640 1 --2- 172.21.15.43:0/2115926734 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa0180a3d20 0x7fa0180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.526+0000 7fa02ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94e8e0 0x55e9cd94ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44012/0 (socket says 172.21.15.43:44012) 2024-04-21T08:47:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.526+0000 7fa02ffff640 1 -- 172.21.15.43:0/1453635266 learned_addr learned my addr 172.21.15.43:0/1453635266 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.526+0000 7fa02f7fe640 1 --2- 172.21.15.43:0/1453635266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9cd94f290 0x55e9cd957840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.526+0000 7fa02effd640 1 --2- 172.21.15.43:0/1453635266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9cd957e10 0x55e9cd95a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.526+0000 7fa02ffff640 1 --2- 172.21.15.43:0/2115926734 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa0180a3d20 0x7fa0180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.527+0000 7fa02ffff640 1 -- 172.21.15.43:0/1453635266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9cd94f290 msgr2=0x55e9cd957840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.527+0000 7fa02ffff640 1 --2- 172.21.15.43:0/1453635266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9cd94f290 0x55e9cd957840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.527+0000 7fa02cff9640 1 -- 172.21.15.43:0/2115926734 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fa0200f2da0 con 0x7fa030102190 2024-04-21T08:47:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.527+0000 7fa02ffff640 1 -- 172.21.15.43:0/1453635266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9cd957e10 msgr2=0x55e9cd95a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.527+0000 7fa02ffff640 1 --2- 172.21.15.43:0/1453635266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9cd957e10 0x55e9cd95a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.527+0000 7fa02ffff640 1 -- 172.21.15.43:0/1453635266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9cd68d7e0 con 0x55e9cd94e8e0 2024-04-21T08:47:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.527+0000 7fa02f7fe640 1 --2- 172.21.15.43:0/1453635266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9cd94f290 0x55e9cd957840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:16.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.527+0000 7fa02ffff640 1 --2- 172.21.15.43:0/1453635266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94e8e0 0x55e9cd94ecc0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fa028014a60 tx=0x7fa028014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76590911c5959ef server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:16.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.527+0000 7fa015ffb640 1 -- 172.21.15.43:0/1453635266 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02801ec50 con 0x55e9cd94e8e0 2024-04-21T08:47:16.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.527+0000 7fa015ffb640 1 -- 172.21.15.43:0/1453635266 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa028023070 con 0x55e9cd94e8e0 2024-04-21T08:47:16.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.527+0000 7fa015ffb640 1 -- 172.21.15.43:0/1453635266 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02801dd60 con 0x55e9cd94e8e0 2024-04-21T08:47:16.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.528+0000 7fa02ffff640 1 --2- 172.21.15.43:0/2115926734 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa0180a3d20 0x7fa0180a61e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa028022000 tx=0x7fa02801e000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:16.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.528+0000 7fa038e43740 1 -- 172.21.15.43:0/1453635266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94e8e0 msgr2=0x55e9cd94ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.528+0000 7fa038e43740 1 --2- 172.21.15.43:0/1453635266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94e8e0 0x55e9cd94ecc0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fa028014a60 tx=0x7fa028014f20 comp rx=0 tx=0).stop 2024-04-21T08:47:16.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.528+0000 7fa038e43740 1 -- 172.21.15.43:0/1453635266 shutdown_connections 2024-04-21T08:47:16.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.528+0000 7fa038e43740 1 --2- 172.21.15.43:0/1453635266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9cd957e10 0x55e9cd95a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.528+0000 7fa038e43740 1 --2- 172.21.15.43:0/1453635266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9cd94f290 0x55e9cd957840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.528+0000 7fa038e43740 1 --2- 172.21.15.43:0/1453635266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94e8e0 0x55e9cd94ecc0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.528+0000 7fa038e43740 1 -- 172.21.15.43:0/1453635266 >> 172.21.15.43:0/1453635266 conn(0x55e9cd948fb0 msgr2=0x55e9cd949390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:16.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.528+0000 7fa038e43740 1 -- 172.21.15.43:0/1453635266 shutdown_connections 2024-04-21T08:47:16.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.528+0000 7fa038e43740 1 -- 172.21.15.43:0/1453635266 wait complete. 2024-04-21T08:47:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.528+0000 7fa038e43740 1 -- start start 2024-04-21T08:47:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.528+0000 7fa038e43740 1 Processor -- start 2024-04-21T08:47:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.529+0000 7fa038e43740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9cd94e8e0 0x55e9cd954d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.529+0000 7fa038e43740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94f290 0x55e9cd9552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.529+0000 7fa038e43740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9cd957e10 0x55e9cd955810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.529+0000 7fa038e43740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e9cd95ad80 con 0x55e9cd94f290 2024-04-21T08:47:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.529+0000 7fa038e43740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e9cd86c580 con 0x55e9cd957e10 2024-04-21T08:47:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.529+0000 7fa038e43740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e9cd95af00 con 0x55e9cd94e8e0 2024-04-21T08:47:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.529+0000 7fa038e43740 10 client.0 _ll_register_callbacks cb 0x55e9cd872650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.529+0000 7fa02effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94f290 0x55e9cd9552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.529+0000 7fa02effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94f290 0x55e9cd9552d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44022/0 (socket says 172.21.15.43:44022) 2024-04-21T08:47:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.529+0000 7fa02effd640 1 -- 172.21.15.43:0/2166343248 learned_addr learned my addr 172.21.15.43:0/2166343248 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.529+0000 7fa02ffff640 1 --2- 172.21.15.43:0/2166343248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9cd957e10 0x55e9cd955810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.529+0000 7fa02f7fe640 1 --2- 172.21.15.43:0/2166343248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9cd94e8e0 0x55e9cd954d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.529+0000 7fa02effd640 1 -- 172.21.15.43:0/2166343248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9cd94e8e0 msgr2=0x55e9cd954d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:16.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.529+0000 7fa02effd640 1 --2- 172.21.15.43:0/2166343248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9cd94e8e0 0x55e9cd954d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.529+0000 7fa02effd640 1 -- 172.21.15.43:0/2166343248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9cd957e10 msgr2=0x55e9cd955810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:16.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.529+0000 7fa02effd640 1 --2- 172.21.15.43:0/2166343248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9cd957e10 0x55e9cd955810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.529+0000 7fa02effd640 1 -- 172.21.15.43:0/2166343248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9cd6edf70 con 0x55e9cd94f290 2024-04-21T08:47:16.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.529+0000 7fa02effd640 1 --2- 172.21.15.43:0/2166343248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94f290 0x55e9cd9552d0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fa02400bf50 tx=0x7fa0240128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc797eaf26f53bc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:16.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.529+0000 7fa00effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:16.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.530+0000 7fa00effd640 1 -- 172.21.15.43:0/2166343248 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0240035f0 con 0x55e9cd94f290 2024-04-21T08:47:16.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.530+0000 7fa00effd640 1 -- 172.21.15.43:0/2166343248 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fa024003790 con 0x55e9cd94f290 2024-04-21T08:47:16.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.530+0000 7fa038e43740 1 -- 172.21.15.43:0/2166343248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa028023070 con 0x55e9cd94f290 2024-04-21T08:47:16.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.530+0000 7fa00effd640 1 -- 172.21.15.43:0/2166343248 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa024024050 con 0x55e9cd94f290 2024-04-21T08:47:16.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.530+0000 7fa038e43740 10 client.19458 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:16.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.530+0000 7fa038e43740 1 -- 172.21.15.43:0/2166343248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e9cd585170 con 0x55e9cd94f290 2024-04-21T08:47:16.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.530+0000 7fa00effd640 1 -- 172.21.15.43:0/2166343248 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa028023070 con 0x55e9cd94f290 2024-04-21T08:47:16.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.530+0000 7fa00effd640 1 -- 172.21.15.43:0/2166343248 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa024019020 con 0x55e9cd94f290 2024-04-21T08:47:16.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.530+0000 7fa038e43740 10 client.19458 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:16.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.530+0000 7fa038e43740 10 client.19458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:16.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.530+0000 7fa038e43740 20 client.19458 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:16.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.530+0000 7fa038e43740 1 --2- 172.21.15.43:0/2166343248 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e9cd9683f0 0x55e9cd96a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.530+0000 7fa038e43740 4 client.19458 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:16.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.530+0000 7fa038e43740 1 -- 172.21.15.43:0/2166343248 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9cd8c0920 con 0x55e9cd9683f0 2024-04-21T08:47:16.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.531+0000 7fa02f7fe640 1 --2- 172.21.15.43:0/2166343248 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e9cd9683f0 0x55e9cd96a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.531+0000 7fa00effd640 10 client.19458 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.531+0000 7fa02f7fe640 1 --2- 172.21.15.43:0/2166343248 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e9cd9683f0 0x55e9cd96a800 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2236e5e2fb824303 server_cookie=4ce50d14efe8dc8b in_seq=0 out_seq=0 2024-04-21T08:47:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.532+0000 7fa00effd640 1 -- 172.21.15.43:0/2166343248 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e9cd8c0920 con 0x55e9cd9683f0 2024-04-21T08:47:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.532+0000 7fa00effd640 10 client.19458 handle_command_reply: tid=0 2024-04-21T08:47:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.532+0000 7fa038e43740 1 client.19458 shutdown 2024-04-21T08:47:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.532+0000 7fa038e43740 1 -- 172.21.15.43:0/2166343248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94f290 msgr2=0x55e9cd9552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.532+0000 7fa038e43740 1 --2- 172.21.15.43:0/2166343248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94f290 0x55e9cd9552d0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fa02400bf50 tx=0x7fa0240128e0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.532+0000 7fa038e43740 1 -- 172.21.15.43:0/2166343248 shutdown_connections 2024-04-21T08:47:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.532+0000 7fa038e43740 1 --2- 172.21.15.43:0/2166343248 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e9cd9683f0 0x55e9cd96a800 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.532+0000 7fa038e43740 1 --2- 172.21.15.43:0/2166343248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9cd957e10 0x55e9cd955810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.532+0000 7fa038e43740 1 --2- 172.21.15.43:0/2166343248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94f290 0x55e9cd9552d0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.532+0000 7fa00effd640 0 client.19458 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.532+0000 7fa038e43740 1 --2- 172.21.15.43:0/2166343248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9cd94e8e0 0x55e9cd954d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.532+0000 7fa038e43740 1 -- 172.21.15.43:0/2166343248 >> 172.21.15.43:0/2166343248 conn(0x55e9cd948fb0 msgr2=0x55e9cd94ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.532+0000 7fa038e43740 1 -- 172.21.15.43:0/2166343248 shutdown_connections 2024-04-21T08:47:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.532+0000 7fa038e43740 1 -- 172.21.15.43:0/2166343248 wait complete. 2024-04-21T08:47:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.532+0000 7fa038e43740 20 client.19458 trim_cache size 0 max 16384 2024-04-21T08:47:16.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:16 smithi112 ceph-mon[31085]: pgmap v258: 129 pgs: 129 active+clean; 4.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 83 KiB/s wr, 22 op/s 2024-04-21T08:47:16.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.550+0000 7fa038e43740 1 Processor -- start 2024-04-21T08:47:16.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.550+0000 7fa038e43740 1 -- start start 2024-04-21T08:47:16.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.550+0000 7fa038e43740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9cd948fb0 0x55e9cd9574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.550+0000 7fa038e43740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9cd94e8e0 0x55e9cd963c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.550+0000 7fa038e43740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94f290 0x55e9cd970110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.550+0000 7fa038e43740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa0301af4c0 con 0x55e9cd94f290 2024-04-21T08:47:16.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.550+0000 7fa038e43740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa0301b1000 con 0x55e9cd948fb0 2024-04-21T08:47:16.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.550+0000 7fa038e43740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa0301afb90 con 0x55e9cd94e8e0 2024-04-21T08:47:16.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.550+0000 7fa02ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94f290 0x55e9cd970110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.550+0000 7fa02ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94f290 0x55e9cd970110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44038/0 (socket says 172.21.15.43:44038) 2024-04-21T08:47:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.550+0000 7fa02ffff640 1 -- 172.21.15.43:0/1338438103 learned_addr learned my addr 172.21.15.43:0/1338438103 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.550+0000 7fa02f7fe640 1 --2- 172.21.15.43:0/1338438103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9cd948fb0 0x55e9cd9574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.550+0000 7fa02effd640 1 --2- 172.21.15.43:0/1338438103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9cd94e8e0 0x55e9cd963c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.550+0000 7fa02ffff640 1 -- 172.21.15.43:0/1338438103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9cd94e8e0 msgr2=0x55e9cd963c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.550+0000 7fa02ffff640 1 --2- 172.21.15.43:0/1338438103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9cd94e8e0 0x55e9cd963c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.550+0000 7fa02ffff640 1 -- 172.21.15.43:0/1338438103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9cd948fb0 msgr2=0x55e9cd9574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.550+0000 7fa02ffff640 1 --2- 172.21.15.43:0/1338438103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9cd948fb0 0x55e9cd9574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.550+0000 7fa02ffff640 1 -- 172.21.15.43:0/1338438103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9cd94b2a0 con 0x55e9cd94f290 2024-04-21T08:47:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.550+0000 7fa02effd640 1 --2- 172.21.15.43:0/1338438103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9cd94e8e0 0x55e9cd963c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.551+0000 7fa02ffff640 1 --2- 172.21.15.43:0/1338438103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94f290 0x55e9cd970110 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fa02801d910 tx=0x7fa0280144f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef572d04cb7b96e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.551+0000 7fa0157fa640 1 -- 172.21.15.43:0/1338438103 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa028012c70 con 0x55e9cd94f290 2024-04-21T08:47:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.551+0000 7fa0157fa640 1 -- 172.21.15.43:0/1338438103 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa028036070 con 0x55e9cd94f290 2024-04-21T08:47:16.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.551+0000 7fa0157fa640 1 -- 172.21.15.43:0/1338438103 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02801d640 con 0x55e9cd94f290 2024-04-21T08:47:16.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.551+0000 7fa038e43740 1 -- 172.21.15.43:0/1338438103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94f290 msgr2=0x55e9cd970110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.551+0000 7fa038e43740 1 --2- 172.21.15.43:0/1338438103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94f290 0x55e9cd970110 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fa02801d910 tx=0x7fa0280144f0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.551+0000 7fa038e43740 1 -- 172.21.15.43:0/1338438103 shutdown_connections 2024-04-21T08:47:16.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.551+0000 7fa038e43740 1 --2- 172.21.15.43:0/1338438103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94f290 0x55e9cd970110 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.551+0000 7fa038e43740 1 --2- 172.21.15.43:0/1338438103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9cd94e8e0 0x55e9cd963c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.551+0000 7fa038e43740 1 --2- 172.21.15.43:0/1338438103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9cd948fb0 0x55e9cd9574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.551+0000 7fa038e43740 1 -- 172.21.15.43:0/1338438103 >> 172.21.15.43:0/1338438103 conn(0x55e9cd94f9e0 msgr2=0x55e9cd9555e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:16.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.551+0000 7fa038e43740 1 -- 172.21.15.43:0/1338438103 shutdown_connections 2024-04-21T08:47:16.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.551+0000 7fa038e43740 1 -- 172.21.15.43:0/1338438103 wait complete. 2024-04-21T08:47:16.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.552+0000 7fa038e43740 1 -- start start 2024-04-21T08:47:16.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.552+0000 7fa038e43740 1 Processor -- start 2024-04-21T08:47:16.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.552+0000 7fa038e43740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9cd948fb0 0x55e9cd95d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.552+0000 7fa038e43740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94e8e0 0x55e9cd95d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.552+0000 7fa038e43740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9cd94f290 0x55e9cd95dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.552+0000 7fa038e43740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa0301b0930 con 0x55e9cd94e8e0 2024-04-21T08:47:16.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.552+0000 7fa038e43740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa0301b0260 con 0x55e9cd94f290 2024-04-21T08:47:16.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.552+0000 7fa038e43740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa0301ad4c0 con 0x55e9cd948fb0 2024-04-21T08:47:16.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.552+0000 7fa038e43740 10 client.0 _ll_register_callbacks cb 0x7fa02400b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:16.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.552+0000 7fa02effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94e8e0 0x55e9cd95d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.552+0000 7fa02effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94e8e0 0x55e9cd95d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44046/0 (socket says 172.21.15.43:44046) 2024-04-21T08:47:16.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.552+0000 7fa02effd640 1 -- 172.21.15.43:0/3557350297 learned_addr learned my addr 172.21.15.43:0/3557350297 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:16.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.552+0000 7fa02ffff640 1 --2- 172.21.15.43:0/3557350297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9cd94f290 0x55e9cd95dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.552+0000 7fa02effd640 1 -- 172.21.15.43:0/3557350297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9cd948fb0 msgr2=0x55e9cd95d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:16.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.552+0000 7fa02effd640 1 --2- 172.21.15.43:0/3557350297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9cd948fb0 0x55e9cd95d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.552+0000 7fa02effd640 1 -- 172.21.15.43:0/3557350297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9cd94f290 msgr2=0x55e9cd95dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:16.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.552+0000 7fa02effd640 1 --2- 172.21.15.43:0/3557350297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9cd94f290 0x55e9cd95dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.552+0000 7fa02effd640 1 -- 172.21.15.43:0/3557350297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9cd9ceac0 con 0x55e9cd94e8e0 2024-04-21T08:47:16.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.552+0000 7fa02ffff640 1 --2- 172.21.15.43:0/3557350297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9cd94f290 0x55e9cd95dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:16.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.553+0000 7fa02effd640 1 --2- 172.21.15.43:0/3557350297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94e8e0 0x55e9cd95d660 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fa024010d00 tx=0x7fa024005ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84982c5632da7d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:16.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.553+0000 7fa00effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:16.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.553+0000 7fa00effd640 1 -- 172.21.15.43:0/3557350297 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa024021040 con 0x55e9cd94e8e0 2024-04-21T08:47:16.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.553+0000 7fa00effd640 1 -- 172.21.15.43:0/3557350297 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fa024010690 con 0x55e9cd94e8e0 2024-04-21T08:47:16.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.553+0000 7fa038e43740 1 -- 172.21.15.43:0/3557350297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa028036070 con 0x55e9cd94e8e0 2024-04-21T08:47:16.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.553+0000 7fa00effd640 1 -- 172.21.15.43:0/3557350297 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa0240108b0 con 0x55e9cd94e8e0 2024-04-21T08:47:16.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.553+0000 7fa038e43740 10 client.19464 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:16.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.553+0000 7fa038e43740 1 -- 172.21.15.43:0/3557350297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e9cd9cfd50 con 0x55e9cd94e8e0 2024-04-21T08:47:16.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.553+0000 7fa00effd640 1 -- 172.21.15.43:0/3557350297 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa028036070 con 0x55e9cd94e8e0 2024-04-21T08:47:16.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.553+0000 7fa00effd640 1 -- 172.21.15.43:0/3557350297 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa024019070 con 0x55e9cd94e8e0 2024-04-21T08:47:16.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.553+0000 7fa038e43740 10 client.19464 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:16.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.553+0000 7fa038e43740 10 client.19464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:16.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.553+0000 7fa038e43740 20 client.19464 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:16.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.553+0000 7fa038e43740 1 --2- 172.21.15.43:0/3557350297 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e9cd957e10 0x55e9cd9d2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.553+0000 7fa038e43740 4 client.19464 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:47:16.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.553+0000 7fa038e43740 1 -- 172.21.15.43:0/3557350297 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e9cd8ba790 con 0x55e9cd957e10 2024-04-21T08:47:16.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.554+0000 7fa02f7fe640 1 --2- 172.21.15.43:0/3557350297 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e9cd957e10 0x55e9cd9d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.554+0000 7fa02f7fe640 1 --2- 172.21.15.43:0/3557350297 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e9cd957e10 0x55e9cd9d2510 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=157d9976fbd69918 server_cookie=3d96e0c602a30db5 in_seq=0 out_seq=0 2024-04-21T08:47:16.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.554+0000 7fa00effd640 10 client.19464 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.555+0000 7fa00effd640 1 -- 172.21.15.43:0/3557350297 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e9cd8ba790 con 0x55e9cd957e10 2024-04-21T08:47:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.555+0000 7fa00effd640 10 client.19464 handle_command_reply: tid=0 2024-04-21T08:47:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.555+0000 7fa038e43740 1 client.19464 shutdown 2024-04-21T08:47:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.555+0000 7fa038e43740 1 -- 172.21.15.43:0/3557350297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94e8e0 msgr2=0x55e9cd95d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.555+0000 7fa038e43740 1 --2- 172.21.15.43:0/3557350297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94e8e0 0x55e9cd95d660 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fa024010d00 tx=0x7fa024005ec0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.555+0000 7fa038e43740 1 -- 172.21.15.43:0/3557350297 shutdown_connections 2024-04-21T08:47:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.555+0000 7fa038e43740 1 --2- 172.21.15.43:0/3557350297 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e9cd957e10 0x55e9cd9d2510 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.555+0000 7fa038e43740 1 --2- 172.21.15.43:0/3557350297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9cd94f290 0x55e9cd95dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.555+0000 7fa038e43740 1 --2- 172.21.15.43:0/3557350297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e9cd94e8e0 0x55e9cd95d660 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.555+0000 7fa038e43740 1 --2- 172.21.15.43:0/3557350297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e9cd948fb0 0x55e9cd95d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.555+0000 7fa038e43740 1 -- 172.21.15.43:0/3557350297 >> 172.21.15.43:0/3557350297 conn(0x55e9cd94f9e0 msgr2=0x55e9cd954d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:16.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.556+0000 7fa00effd640 0 client.19464 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:16.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.556+0000 7fa038e43740 1 -- 172.21.15.43:0/3557350297 shutdown_connections 2024-04-21T08:47:16.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.556+0000 7fa038e43740 1 -- 172.21.15.43:0/3557350297 wait complete. 2024-04-21T08:47:16.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.556+0000 7fa038e43740 20 client.19464 trim_cache size 0 max 16384 2024-04-21T08:47:16.562 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:47:16.562 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:47:16.562 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "868a1875-7a60-440c-8e95-2d5a477d5ecc", 2024-04-21T08:47:16.563 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:47:16.563 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:47:16.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.556+0000 7fa03624e640 1 -- 172.21.15.43:0/2115926734 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa0180a3d20 msgr2=0x7fa0180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.556+0000 7fa03624e640 1 --2- 172.21.15.43:0/2115926734 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa0180a3d20 0x7fa0180a61e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa028022000 tx=0x7fa02801e000 comp rx=0 tx=0).stop 2024-04-21T08:47:16.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.556+0000 7fa03624e640 1 -- 172.21.15.43:0/2115926734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa030102190 msgr2=0x7fa030198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.556+0000 7fa03624e640 1 --2- 172.21.15.43:0/2115926734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa030102190 0x7fa030198760 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fa020005940 tx=0x7fa02002c7e0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.556+0000 7fa03624e640 1 -- 172.21.15.43:0/2115926734 shutdown_connections 2024-04-21T08:47:16.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.556+0000 7fa03624e640 1 --2- 172.21.15.43:0/2115926734 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa0180a3d20 0x7fa0180a61e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.556+0000 7fa03624e640 1 --2- 172.21.15.43:0/2115926734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa03019d020 0x7fa03019d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.556+0000 7fa03624e640 1 --2- 172.21.15.43:0/2115926734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa030107260 0x7fa030198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.556+0000 7fa03624e640 1 --2- 172.21.15.43:0/2115926734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa030102190 0x7fa030198760 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.556+0000 7fa03624e640 1 -- 172.21.15.43:0/2115926734 >> 172.21.15.43:0/2115926734 conn(0x7fa0300fd990 msgr2=0x7fa0300fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:16.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.556+0000 7fa03624e640 1 -- 172.21.15.43:0/2115926734 shutdown_connections 2024-04-21T08:47:16.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.556+0000 7fa03624e640 1 -- 172.21.15.43:0/2115926734 wait complete. 2024-04-21T08:47:16.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:47:16.567 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:47:16.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.713+0000 7fd4d2c25640 1 -- 172.21.15.43:0/3275656120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4cc106150 msgr2=0x7fd4cc10dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.713+0000 7fd4d2c25640 1 --2- 172.21.15.43:0/3275656120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4cc106150 0x7fd4cc10dd20 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4009ca0 tx=0x7fd4c402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.713+0000 7fd4d2c25640 1 -- 172.21.15.43:0/3275656120 shutdown_connections 2024-04-21T08:47:16.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.713+0000 7fd4d2c25640 1 --2- 172.21.15.43:0/3275656120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4cc106150 0x7fd4cc10dd20 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.713+0000 7fd4d2c25640 1 --2- 172.21.15.43:0/3275656120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4cc101a30 0x7fd4cc105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.713+0000 7fd4d2c25640 1 --2- 172.21.15.43:0/3275656120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4cc101080 0x7fd4cc101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.713+0000 7fd4d2c25640 1 -- 172.21.15.43:0/3275656120 >> 172.21.15.43:0/3275656120 conn(0x7fd4cc078000 msgr2=0x7fd4cc078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:16.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.713+0000 7fd4d2c25640 1 -- 172.21.15.43:0/3275656120 shutdown_connections 2024-04-21T08:47:16.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.713+0000 7fd4d2c25640 1 -- 172.21.15.43:0/3275656120 wait complete. 2024-04-21T08:47:16.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.713+0000 7fd4d2c25640 1 Processor -- start 2024-04-21T08:47:16.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.713+0000 7fd4d2c25640 1 -- start start 2024-04-21T08:47:16.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.713+0000 7fd4d2c25640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4cc101080 0x7fd4cc1993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.713+0000 7fd4d2c25640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4cc106150 0x7fd4cc199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.713+0000 7fd4d2c25640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4cc199eb0 0x7fd4cc1a4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.713+0000 7fd4d2c25640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc1110a0 con 0x7fd4cc199eb0 2024-04-21T08:47:16.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.713+0000 7fd4d2c25640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc110f20 con 0x7fd4cc106150 2024-04-21T08:47:16.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.713+0000 7fd4d2c25640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc111220 con 0x7fd4cc101080 2024-04-21T08:47:16.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.714+0000 7fd4d11c2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4cc199eb0 0x7fd4cc1a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.714+0000 7fd4d11c2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4cc199eb0 0x7fd4cc1a4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42424/0 (socket says 172.21.15.43:42424) 2024-04-21T08:47:16.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.714+0000 7fd4d11c2640 1 -- 172.21.15.43:0/1376695676 learned_addr learned my addr 172.21.15.43:0/1376695676 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:16.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.714+0000 7fd4d11c2640 1 -- 172.21.15.43:0/1376695676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4cc101080 msgr2=0x7fd4cc1993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.714+0000 7fd4c3fff640 1 --2- 172.21.15.43:0/1376695676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4cc106150 0x7fd4cc199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.714+0000 7fd4d11c2640 1 --2- 172.21.15.43:0/1376695676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4cc101080 0x7fd4cc1993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.714+0000 7fd4d11c2640 1 -- 172.21.15.43:0/1376695676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4cc106150 msgr2=0x7fd4cc199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.714+0000 7fd4d11c2640 1 --2- 172.21.15.43:0/1376695676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4cc106150 0x7fd4cc199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.714+0000 7fd4d11c2640 1 -- 172.21.15.43:0/1376695676 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4cc1a4c60 con 0x7fd4cc199eb0 2024-04-21T08:47:16.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.714+0000 7fd4c3fff640 1 --2- 172.21.15.43:0/1376695676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4cc106150 0x7fd4cc199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:16.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.714+0000 7fd4d11c2640 1 --2- 172.21.15.43:0/1376695676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4cc199eb0 0x7fd4cc1a4460 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40098d0 tx=0x7fd4c4035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:16.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.714+0000 7fd4c1ffb640 1 -- 172.21.15.43:0/1376695676 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4039040 con 0x7fd4cc199eb0 2024-04-21T08:47:16.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.714+0000 7fd4c1ffb640 1 -- 172.21.15.43:0/1376695676 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c4044070 con 0x7fd4cc199eb0 2024-04-21T08:47:16.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.714+0000 7fd4d2c25640 1 -- 172.21.15.43:0/1376695676 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4cc1a4ef0 con 0x7fd4cc199eb0 2024-04-21T08:47:16.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.714+0000 7fd4c1ffb640 1 -- 172.21.15.43:0/1376695676 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c40042a0 con 0x7fd4cc199eb0 2024-04-21T08:47:16.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.714+0000 7fd4d2c25640 1 -- 172.21.15.43:0/1376695676 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4cc1a5400 con 0x7fd4cc199eb0 2024-04-21T08:47:16.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.717+0000 7fd4d5813740 1 Processor -- start 2024-04-21T08:47:16.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.717+0000 7fd4c1ffb640 1 -- 172.21.15.43:0/1376695676 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd4c4004440 con 0x7fd4cc199eb0 2024-04-21T08:47:16.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.717+0000 7fd4d5813740 1 -- start start 2024-04-21T08:47:16.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.718+0000 7fd4d5813740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2580 0x55b2711d2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.718+0000 7fd4d5813740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b2711d2ea0 0x55b2711db540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.718+0000 7fd4d5813740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2711dba80 0x55b2711dde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.718+0000 7fd4d5813740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b2711371e0 con 0x55b2711d2580 2024-04-21T08:47:16.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.718+0000 7fd4d5813740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b2710d4390 con 0x55b2711d2ea0 2024-04-21T08:47:16.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.718+0000 7fd4d5813740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b2711331a0 con 0x55b2711dba80 2024-04-21T08:47:16.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.718+0000 7fd4c3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2580 0x55b2711d2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.718+0000 7fd4c3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2580 0x55b2711d2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42432/0 (socket says 172.21.15.43:42432) 2024-04-21T08:47:16.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.718+0000 7fd4c3fff640 1 -- 172.21.15.43:0/3105923513 learned_addr learned my addr 172.21.15.43:0/3105923513 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:16.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.718+0000 7fd4d09c1640 1 --2- 172.21.15.43:0/3105923513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b2711d2ea0 0x55b2711db540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.718+0000 7fd4c3fff640 1 -- 172.21.15.43:0/3105923513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2711dba80 msgr2=0x55b2711dde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:16.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.718+0000 7fd4c3fff640 1 --2- 172.21.15.43:0/3105923513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2711dba80 0x55b2711dde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.718+0000 7fd4c3fff640 1 -- 172.21.15.43:0/3105923513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b2711d2ea0 msgr2=0x55b2711db540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.718+0000 7fd4c1ffb640 1 --2- 172.21.15.43:0/1376695676 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd4b40a3da0 0x7fd4b40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.718+0000 7fd4c3fff640 1 --2- 172.21.15.43:0/3105923513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b2711d2ea0 0x55b2711db540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.718+0000 7fd4c3fff640 1 -- 172.21.15.43:0/3105923513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b270f137d0 con 0x55b2711d2580 2024-04-21T08:47:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.718+0000 7fd4d11c2640 1 --2- 172.21.15.43:0/1376695676 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd4b40a3da0 0x7fd4b40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.718+0000 7fd4c3fff640 1 --2- 172.21.15.43:0/3105923513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2580 0x55b2711d2960 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00bbd0 tx=0x7fd4bc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107b0f883c8567de server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.718+0000 7fd4c1ffb640 1 -- 172.21.15.43:0/1376695676 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fd4c40f3110 con 0x7fd4cc199eb0 2024-04-21T08:47:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.719+0000 7fd4b2ffd640 1 -- 172.21.15.43:0/3105923513 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc005220 con 0x55b2711d2580 2024-04-21T08:47:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.719+0000 7fd4b2ffd640 1 -- 172.21.15.43:0/3105923513 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4bc004c50 con 0x55b2711d2580 2024-04-21T08:47:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.719+0000 7fd4b2ffd640 1 -- 172.21.15.43:0/3105923513 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc0117f0 con 0x55b2711d2580 2024-04-21T08:47:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.719+0000 7fd4d5813740 1 -- 172.21.15.43:0/3105923513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2580 msgr2=0x55b2711d2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.719+0000 7fd4d5813740 1 --2- 172.21.15.43:0/3105923513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2580 0x55b2711d2960 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00bbd0 tx=0x7fd4bc00f730 comp rx=0 tx=0).stop 2024-04-21T08:47:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.719+0000 7fd4d11c2640 1 --2- 172.21.15.43:0/1376695676 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd4b40a3da0 0x7fd4b40a6260 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40bb1a0 tx=0x7fd4c4091830 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.719+0000 7fd4d5813740 1 -- 172.21.15.43:0/3105923513 shutdown_connections 2024-04-21T08:47:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.719+0000 7fd4d5813740 1 --2- 172.21.15.43:0/3105923513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2711dba80 0x55b2711dde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.719+0000 7fd4d5813740 1 --2- 172.21.15.43:0/3105923513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b2711d2ea0 0x55b2711db540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.719+0000 7fd4d5813740 1 --2- 172.21.15.43:0/3105923513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2580 0x55b2711d2960 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.719+0000 7fd4d5813740 1 -- 172.21.15.43:0/3105923513 >> 172.21.15.43:0/3105923513 conn(0x55b2711cccb0 msgr2=0x55b2711cd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.719+0000 7fd4d5813740 1 -- 172.21.15.43:0/3105923513 shutdown_connections 2024-04-21T08:47:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.719+0000 7fd4d5813740 1 -- 172.21.15.43:0/3105923513 wait complete. 2024-04-21T08:47:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.720+0000 7fd4d5813740 1 -- start start 2024-04-21T08:47:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.720+0000 7fd4d5813740 1 Processor -- start 2024-04-21T08:47:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.720+0000 7fd4d5813740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2580 0x55b2711d8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.720+0000 7fd4d5813740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b2711d2ea0 0x55b2711d9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.720+0000 7fd4d5813740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2711dba80 0x55b2711d95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.720+0000 7fd4d5813740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b2711de870 con 0x55b2711d2580 2024-04-21T08:47:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.720+0000 7fd4d5813740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b2710f02f0 con 0x55b2711d2ea0 2024-04-21T08:47:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.720+0000 7fd4d5813740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b2711de9f0 con 0x55b2711dba80 2024-04-21T08:47:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.720+0000 7fd4d5813740 10 client.0 _ll_register_callbacks cb 0x55b2710fd580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.720+0000 7fd4c3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2580 0x55b2711d8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.720+0000 7fd4c3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2580 0x55b2711d8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42434/0 (socket says 172.21.15.43:42434) 2024-04-21T08:47:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.720+0000 7fd4c3fff640 1 -- 172.21.15.43:0/900686093 learned_addr learned my addr 172.21.15.43:0/900686093 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.720+0000 7fd4d09c1640 1 --2- 172.21.15.43:0/900686093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b2711d2ea0 0x55b2711d9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.721+0000 7fd4c3fff640 1 -- 172.21.15.43:0/900686093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2711dba80 msgr2=0x55b2711d95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.721+0000 7fd4c3fff640 1 --2- 172.21.15.43:0/900686093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2711dba80 0x55b2711d95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.721+0000 7fd4c3fff640 1 -- 172.21.15.43:0/900686093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b2711d2ea0 msgr2=0x55b2711d9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.721+0000 7fd4c3fff640 1 --2- 172.21.15.43:0/900686093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b2711d2ea0 0x55b2711d9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.721+0000 7fd4c3fff640 1 -- 172.21.15.43:0/900686093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b270e493f0 con 0x55b2711d2580 2024-04-21T08:47:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.721+0000 7fd4c3fff640 1 --2- 172.21.15.43:0/900686093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2580 0x55b2711d8b50 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc005740 tx=0x7fd4bc00b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2d2cf46f12e856 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:16.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.721+0000 7fd4abfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:16.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.721+0000 7fd4abfff640 1 -- 172.21.15.43:0/900686093 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc005430 con 0x55b2711d2580 2024-04-21T08:47:16.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.721+0000 7fd4abfff640 1 -- 172.21.15.43:0/900686093 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fd4bc011570 con 0x55b2711d2580 2024-04-21T08:47:16.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.721+0000 7fd4d5813740 1 -- 172.21.15.43:0/900686093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4bc004c50 con 0x55b2711d2580 2024-04-21T08:47:16.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.721+0000 7fd4abfff640 1 -- 172.21.15.43:0/900686093 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd4bc011770 con 0x55b2711d2580 2024-04-21T08:47:16.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.721+0000 7fd4d5813740 10 client.19476 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:16.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.721+0000 7fd4d5813740 1 -- 172.21.15.43:0/900686093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b270e090f0 con 0x55b2711d2580 2024-04-21T08:47:16.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.721+0000 7fd4abfff640 1 -- 172.21.15.43:0/900686093 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4bc004c50 con 0x55b2711d2580 2024-04-21T08:47:16.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.721+0000 7fd4abfff640 1 -- 172.21.15.43:0/900686093 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd4bc025830 con 0x55b2711d2580 2024-04-21T08:47:16.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.722+0000 7fd4d5813740 10 client.19476 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:16.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.722+0000 7fd4d5813740 10 client.19476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:16.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.722+0000 7fd4d5813740 20 client.19476 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:16.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.722+0000 7fd4d5813740 1 --2- 172.21.15.43:0/900686093 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55b2711ebad0 0x55b2711edec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.722+0000 7fd4d5813740 4 client.19476 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.722+0000 7fd4d5813740 1 -- 172.21.15.43:0/900686093 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b271144c00 con 0x55b2711ebad0 2024-04-21T08:47:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.722+0000 7fd4d09c1640 1 --2- 172.21.15.43:0/900686093 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55b2711ebad0 0x55b2711edec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.722+0000 7fd4d09c1640 1 --2- 172.21.15.43:0/900686093 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55b2711ebad0 0x55b2711edec0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97f45bfe77b83a7a server_cookie=10b4c62455a646d7 in_seq=0 out_seq=0 2024-04-21T08:47:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.722+0000 7fd4abfff640 10 client.19476 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.723+0000 7fd4abfff640 1 -- 172.21.15.43:0/900686093 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b271144c00 con 0x55b2711ebad0 2024-04-21T08:47:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.723+0000 7fd4abfff640 10 client.19476 handle_command_reply: tid=0 2024-04-21T08:47:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.723+0000 7fd4d5813740 1 client.19476 shutdown 2024-04-21T08:47:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.723+0000 7fd4d5813740 1 -- 172.21.15.43:0/900686093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2580 msgr2=0x55b2711d8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.723+0000 7fd4d5813740 1 --2- 172.21.15.43:0/900686093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2580 0x55b2711d8b50 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc005740 tx=0x7fd4bc00b9c0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.723+0000 7fd4d5813740 1 -- 172.21.15.43:0/900686093 shutdown_connections 2024-04-21T08:47:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.723+0000 7fd4d5813740 1 --2- 172.21.15.43:0/900686093 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55b2711ebad0 0x55b2711edec0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.723+0000 7fd4abfff640 0 client.19476 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.723+0000 7fd4d5813740 1 --2- 172.21.15.43:0/900686093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2711dba80 0x55b2711d95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.723+0000 7fd4d5813740 1 --2- 172.21.15.43:0/900686093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b2711d2ea0 0x55b2711d9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.723+0000 7fd4d5813740 1 --2- 172.21.15.43:0/900686093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2580 0x55b2711d8b50 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.723+0000 7fd4d5813740 1 -- 172.21.15.43:0/900686093 >> 172.21.15.43:0/900686093 conn(0x55b2711cccb0 msgr2=0x55b2711cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.723+0000 7fd4d5813740 1 -- 172.21.15.43:0/900686093 shutdown_connections 2024-04-21T08:47:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.724+0000 7fd4d5813740 1 -- 172.21.15.43:0/900686093 wait complete. 2024-04-21T08:47:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.724+0000 7fd4d5813740 20 client.19476 trim_cache size 0 max 16384 2024-04-21T08:47:16.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.741+0000 7fd4d5813740 1 Processor -- start 2024-04-21T08:47:16.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.741+0000 7fd4d5813740 1 -- start start 2024-04-21T08:47:16.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.741+0000 7fd4d5813740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2711d2580 0x55b2711e7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.742+0000 7fd4d5813740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2ea0 0x55b2711cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.742+0000 7fd4d5813740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b2711dba80 0x55b2711d9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.742+0000 7fd4d5813740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc1af450 con 0x55b2711d2ea0 2024-04-21T08:47:16.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.742+0000 7fd4d5813740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc1b0f90 con 0x55b2711dba80 2024-04-21T08:47:16.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.742+0000 7fd4d5813740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc1afb20 con 0x55b2711d2580 2024-04-21T08:47:16.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.742+0000 7fd4d09c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2ea0 0x55b2711cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.742+0000 7fd4d09c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2ea0 0x55b2711cbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42442/0 (socket says 172.21.15.43:42442) 2024-04-21T08:47:16.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.742+0000 7fd4d09c1640 1 -- 172.21.15.43:0/1974345252 learned_addr learned my addr 172.21.15.43:0/1974345252 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:16.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.742+0000 7fd4c3fff640 1 --2- 172.21.15.43:0/1974345252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b2711dba80 0x55b2711d9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.742+0000 7fd4c3fff640 1 --2- 172.21.15.43:0/1974345252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2711d2580 0x55b2711e7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.742+0000 7fd4d09c1640 1 -- 172.21.15.43:0/1974345252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2711d2580 msgr2=0x55b2711e7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:16.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.742+0000 7fd4d09c1640 1 --2- 172.21.15.43:0/1974345252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2711d2580 0x55b2711e7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.742+0000 7fd4d09c1640 1 -- 172.21.15.43:0/1974345252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b2711dba80 msgr2=0x55b2711d9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:16.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.742+0000 7fd4d09c1640 1 --2- 172.21.15.43:0/1974345252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b2711dba80 0x55b2711d9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.742+0000 7fd4d09c1640 1 -- 172.21.15.43:0/1974345252 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2711d70a0 con 0x55b2711d2ea0 2024-04-21T08:47:16.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.742+0000 7fd4d09c1640 1 --2- 172.21.15.43:0/1974345252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2ea0 0x55b2711cbd70 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x55b2711d2050 tx=0x7fd4c8012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e932cdc0680e2ca server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:16.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.742+0000 7fd4b27fc640 1 -- 172.21.15.43:0/1974345252 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8004020 con 0x55b2711d2ea0 2024-04-21T08:47:16.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.742+0000 7fd4b27fc640 1 -- 172.21.15.43:0/1974345252 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c801d8e0 con 0x55b2711d2ea0 2024-04-21T08:47:16.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.742+0000 7fd4b27fc640 1 -- 172.21.15.43:0/1974345252 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8003850 con 0x55b2711d2ea0 2024-04-21T08:47:16.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.743+0000 7fd4d5813740 1 -- 172.21.15.43:0/1974345252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2ea0 msgr2=0x55b2711cbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.743+0000 7fd4d5813740 1 --2- 172.21.15.43:0/1974345252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2ea0 0x55b2711cbd70 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x55b2711d2050 tx=0x7fd4c8012820 comp rx=0 tx=0).stop 2024-04-21T08:47:16.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.743+0000 7fd4d5813740 1 -- 172.21.15.43:0/1974345252 shutdown_connections 2024-04-21T08:47:16.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.743+0000 7fd4d5813740 1 --2- 172.21.15.43:0/1974345252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b2711dba80 0x55b2711d9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.743+0000 7fd4d5813740 1 --2- 172.21.15.43:0/1974345252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2ea0 0x55b2711cbd70 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.743+0000 7fd4d5813740 1 --2- 172.21.15.43:0/1974345252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2711d2580 0x55b2711e7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.743+0000 7fd4d5813740 1 -- 172.21.15.43:0/1974345252 >> 172.21.15.43:0/1974345252 conn(0x55b2711cccb0 msgr2=0x55b2711d8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:16.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.743+0000 7fd4d5813740 1 -- 172.21.15.43:0/1974345252 shutdown_connections 2024-04-21T08:47:16.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.743+0000 7fd4d5813740 1 -- 172.21.15.43:0/1974345252 wait complete. 2024-04-21T08:47:16.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.743+0000 7fd4d5813740 1 -- start start 2024-04-21T08:47:16.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.743+0000 7fd4d5813740 1 Processor -- start 2024-04-21T08:47:16.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.744+0000 7fd4d5813740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2580 0x55b2711dfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.744+0000 7fd4d5813740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2711d2ea0 0x55b2711dff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.744+0000 7fd4d5813740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b2711dba80 0x55b271250d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.744+0000 7fd4d5813740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc1b08c0 con 0x55b2711d2580 2024-04-21T08:47:16.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.744+0000 7fd4d5813740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc1b01f0 con 0x55b2711dba80 2024-04-21T08:47:16.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.744+0000 7fd4d5813740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc1ad450 con 0x55b2711d2ea0 2024-04-21T08:47:16.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.744+0000 7fd4d5813740 10 client.0 _ll_register_callbacks cb 0x7fd4c8005f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:16.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.744+0000 7fd4c3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2580 0x55b2711dfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.744+0000 7fd4c3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2580 0x55b2711dfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42450/0 (socket says 172.21.15.43:42450) 2024-04-21T08:47:16.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.744+0000 7fd4c3fff640 1 -- 172.21.15.43:0/4283968117 learned_addr learned my addr 172.21.15.43:0/4283968117 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:16.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.744+0000 7fd4c3fff640 1 --2- 172.21.15.43:0/4283968117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b2711dba80 0x55b271250d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.744+0000 7fd4c3fff640 1 -- 172.21.15.43:0/4283968117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2711d2ea0 msgr2=0x55b2711dff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:16.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.744+0000 7fd4c3fff640 1 --2- 172.21.15.43:0/4283968117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2711d2ea0 0x55b2711dff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.744+0000 7fd4c3fff640 1 -- 172.21.15.43:0/4283968117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b2711dba80 msgr2=0x55b271250d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:16.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.744+0000 7fd4c3fff640 1 --2- 172.21.15.43:0/4283968117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b2711dba80 0x55b271250d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.744+0000 7fd4c3fff640 1 -- 172.21.15.43:0/4283968117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2712512a0 con 0x55b2711d2580 2024-04-21T08:47:16.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.744+0000 7fd4c3fff640 1 --2- 172.21.15.43:0/4283968117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2580 0x55b2711dfa40 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc001eb0 tx=0x7fd4bc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8315589dd0868807 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:16.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.744+0000 7fd4abfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.745+0000 7fd4abfff640 1 -- 172.21.15.43:0/4283968117 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc00fc30 con 0x55b2711d2580 2024-04-21T08:47:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.745+0000 7fd4abfff640 1 -- 172.21.15.43:0/4283968117 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fd4bc011910 con 0x55b2711d2580 2024-04-21T08:47:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.745+0000 7fd4d5813740 1 -- 172.21.15.43:0/4283968117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4c801d8e0 con 0x55b2711d2580 2024-04-21T08:47:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.745+0000 7fd4abfff640 1 -- 172.21.15.43:0/4283968117 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd4bc011b10 con 0x55b2711d2580 2024-04-21T08:47:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.745+0000 7fd4d5813740 10 client.19482 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.745+0000 7fd4d5813740 1 -- 172.21.15.43:0/4283968117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b271252410 con 0x55b2711d2580 2024-04-21T08:47:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.745+0000 7fd4abfff640 1 -- 172.21.15.43:0/4283968117 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c801d8e0 con 0x55b2711d2580 2024-04-21T08:47:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.745+0000 7fd4abfff640 1 -- 172.21.15.43:0/4283968117 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd4bc0196d0 con 0x55b2711d2580 2024-04-21T08:47:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.745+0000 7fd4d5813740 10 client.19482 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.745+0000 7fd4d5813740 10 client.19482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.745+0000 7fd4d5813740 20 client.19482 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.745+0000 7fd4d5813740 1 --2- 172.21.15.43:0/4283968117 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55b2711ebad0 0x55b271254b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.745+0000 7fd4d5813740 4 client.19482 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:47:16.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.745+0000 7fd4d5813740 1 -- 172.21.15.43:0/4283968117 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b27113ea70 con 0x55b2711ebad0 2024-04-21T08:47:16.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.745+0000 7fd4d09c1640 1 --2- 172.21.15.43:0/4283968117 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55b2711ebad0 0x55b271254b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.746+0000 7fd4abfff640 10 client.19482 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:16.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.746+0000 7fd4d09c1640 1 --2- 172.21.15.43:0/4283968117 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55b2711ebad0 0x55b271254b70 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87194ed580aa6056 server_cookie=af6f5ef633cc1640 in_seq=0 out_seq=0 2024-04-21T08:47:16.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.746+0000 7fd4abfff640 1 -- 172.21.15.43:0/4283968117 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b27113ea70 con 0x55b2711ebad0 2024-04-21T08:47:16.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.746+0000 7fd4abfff640 10 client.19482 handle_command_reply: tid=0 2024-04-21T08:47:16.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.746+0000 7fd4d5813740 1 client.19482 shutdown 2024-04-21T08:47:16.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.746+0000 7fd4d5813740 1 -- 172.21.15.43:0/4283968117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2580 msgr2=0x55b2711dfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.746+0000 7fd4d5813740 1 --2- 172.21.15.43:0/4283968117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2580 0x55b2711dfa40 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc001eb0 tx=0x7fd4bc000bc0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.746+0000 7fd4d5813740 1 -- 172.21.15.43:0/4283968117 shutdown_connections 2024-04-21T08:47:16.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.746+0000 7fd4d5813740 1 --2- 172.21.15.43:0/4283968117 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55b2711ebad0 0x55b271254b70 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.746+0000 7fd4d5813740 1 --2- 172.21.15.43:0/4283968117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b2711dba80 0x55b271250d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.746+0000 7fd4abfff640 0 client.19482 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:16.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.746+0000 7fd4d5813740 1 --2- 172.21.15.43:0/4283968117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b2711d2ea0 0x55b2711dff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.746+0000 7fd4d5813740 1 --2- 172.21.15.43:0/4283968117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b2711d2580 0x55b2711dfa40 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.746+0000 7fd4d5813740 1 -- 172.21.15.43:0/4283968117 >> 172.21.15.43:0/4283968117 conn(0x55b2711cccb0 msgr2=0x55b2711cbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:16.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.746+0000 7fd4d5813740 1 -- 172.21.15.43:0/4283968117 shutdown_connections 2024-04-21T08:47:16.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.746+0000 7fd4d5813740 1 -- 172.21.15.43:0/4283968117 wait complete. 2024-04-21T08:47:16.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.746+0000 7fd4d5813740 20 client.19482 trim_cache size 0 max 16384 2024-04-21T08:47:16.754 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:47:16.754 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:47:16.755 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-21T08:47:16.755 INFO:teuthology.orchestra.run.smithi043.stdout: "868a1875-7a60-440c-8e95-2d5a477d5ecc": { 2024-04-21T08:47:16.755 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-21T08:47:16.755 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "868a1875-7a60-440c-8e95-2d5a477d5ecc", 2024-04-21T08:47:16.755 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-21T08:47:16.755 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:47:16.755 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:47:16.755 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:47:16.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.747+0000 7fd4d2c25640 1 -- 172.21.15.43:0/1376695676 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd4b40a3da0 msgr2=0x7fd4b40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.747+0000 7fd4d2c25640 1 --2- 172.21.15.43:0/1376695676 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd4b40a3da0 0x7fd4b40a6260 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40bb1a0 tx=0x7fd4c4091830 comp rx=0 tx=0).stop 2024-04-21T08:47:16.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.747+0000 7fd4d2c25640 1 -- 172.21.15.43:0/1376695676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4cc199eb0 msgr2=0x7fd4cc1a4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.747+0000 7fd4d2c25640 1 --2- 172.21.15.43:0/1376695676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4cc199eb0 0x7fd4cc1a4460 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40098d0 tx=0x7fd4c4035d60 comp rx=0 tx=0).stop 2024-04-21T08:47:16.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.747+0000 7fd4d2c25640 1 -- 172.21.15.43:0/1376695676 shutdown_connections 2024-04-21T08:47:16.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.747+0000 7fd4d2c25640 1 --2- 172.21.15.43:0/1376695676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4cc199eb0 0x7fd4cc1a4460 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.747+0000 7fd4d2c25640 1 --2- 172.21.15.43:0/1376695676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4cc106150 0x7fd4cc199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.747+0000 7fd4d2c25640 1 --2- 172.21.15.43:0/1376695676 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd4b40a3da0 0x7fd4b40a6260 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.747+0000 7fd4d2c25640 1 --2- 172.21.15.43:0/1376695676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4cc101080 0x7fd4cc1993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.747+0000 7fd4d2c25640 1 -- 172.21.15.43:0/1376695676 >> 172.21.15.43:0/1376695676 conn(0x7fd4cc078000 msgr2=0x7fd4cc0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:16.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.747+0000 7fd4d2c25640 1 -- 172.21.15.43:0/1376695676 shutdown_connections 2024-04-21T08:47:16.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:16.747+0000 7fd4d2c25640 1 -- 172.21.15.43:0/1376695676 wait complete. 2024-04-21T08:47:16.758 INFO:tasks.cephfs.filesystem:scrub status for tag:868a1875-7a60-440c-8e95-2d5a477d5ecc - {'path': '/', 'tag': '868a1875-7a60-440c-8e95-2d5a477d5ecc', 'options': 'recursive,force'} 2024-04-21T08:47:18.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:18 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:47:18.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:18 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:18.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:18 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:47:18.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:18 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:18.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:18 smithi175 ceph-mon[31920]: pgmap v259: 129 pgs: 129 active+clean; 4.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-21T08:47:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:18 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:47:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:18 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:18 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:47:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:18 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:18 smithi112 ceph-mon[31085]: pgmap v259: 129 pgs: 129 active+clean; 4.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-21T08:47:18.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:18 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:47:18.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:18 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:18.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:18 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:47:18.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:18 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:18.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:18 smithi043 ceph-mon[27978]: pgmap v259: 129 pgs: 129 active+clean; 4.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-21T08:47:20.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:20 smithi175 ceph-mon[31920]: pgmap v260: 129 pgs: 129 active+clean; 4.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-21T08:47:20.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:20 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:47:20.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:20 smithi112 ceph-mon[31085]: pgmap v260: 129 pgs: 129 active+clean; 4.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-21T08:47:20.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:20 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:47:20.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:20 smithi043 ceph-mon[27978]: pgmap v260: 129 pgs: 129 active+clean; 4.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-21T08:47:20.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:20 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:47:22.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:22 smithi175 ceph-mon[31920]: pgmap v261: 129 pgs: 129 active+clean; 4.6 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 25 op/s 2024-04-21T08:47:22.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:22 smithi112 ceph-mon[31085]: pgmap v261: 129 pgs: 129 active+clean; 4.6 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 25 op/s 2024-04-21T08:47:22.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:22 smithi043 ceph-mon[27978]: pgmap v261: 129 pgs: 129 active+clean; 4.6 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 25 op/s 2024-04-21T08:47:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.031+0000 7fb441f85640 1 -- 172.21.15.43:0/2818673201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb43c102b70 msgr2=0x7fb43c106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.031+0000 7fb441f85640 1 --2- 172.21.15.43:0/2818673201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb43c102b70 0x7fb43c106bb0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fb430009900 tx=0x7fb43002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.031+0000 7fb441f85640 1 -- 172.21.15.43:0/2818673201 shutdown_connections 2024-04-21T08:47:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.031+0000 7fb441f85640 1 --2- 172.21.15.43:0/2818673201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb43c107290 0x7fb43c10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.031+0000 7fb441f85640 1 --2- 172.21.15.43:0/2818673201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb43c102b70 0x7fb43c106bb0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.031+0000 7fb441f85640 1 --2- 172.21.15.43:0/2818673201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb43c1021c0 0x7fb43c1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.031+0000 7fb441f85640 1 -- 172.21.15.43:0/2818673201 >> 172.21.15.43:0/2818673201 conn(0x7fb43c0fda20 msgr2=0x7fb43c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.031+0000 7fb441f85640 1 -- 172.21.15.43:0/2818673201 shutdown_connections 2024-04-21T08:47:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.031+0000 7fb441f85640 1 -- 172.21.15.43:0/2818673201 wait complete. 2024-04-21T08:47:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.032+0000 7fb441f85640 1 Processor -- start 2024-04-21T08:47:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.032+0000 7fb441f85640 1 -- start start 2024-04-21T08:47:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.032+0000 7fb441f85640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb43c1021c0 0x7fb43c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.032+0000 7fb441f85640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb43c107290 0x7fb43c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.032+0000 7fb441f85640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb43c075ef0 0x7fb43c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.032+0000 7fb441f85640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1121b0 con 0x7fb43c075ef0 2024-04-21T08:47:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.032+0000 7fb441f85640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c112030 con 0x7fb43c1021c0 2024-04-21T08:47:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.032+0000 7fb441f85640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c112330 con 0x7fb43c107290 2024-04-21T08:47:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.033+0000 7fb43bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb43c075ef0 0x7fb43c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.033+0000 7fb43bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb43c075ef0 0x7fb43c074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42464/0 (socket says 172.21.15.43:42464) 2024-04-21T08:47:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.033+0000 7fb43bfff640 1 -- 172.21.15.43:0/12336542 learned_addr learned my addr 172.21.15.43:0/12336542 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.033+0000 7fb43b7fe640 1 --2- 172.21.15.43:0/12336542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb43c1021c0 0x7fb43c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.033+0000 7fb43affd640 1 --2- 172.21.15.43:0/12336542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb43c107290 0x7fb43c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.033+0000 7fb43bfff640 1 -- 172.21.15.43:0/12336542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb43c107290 msgr2=0x7fb43c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.033+0000 7fb43bfff640 1 --2- 172.21.15.43:0/12336542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb43c107290 0x7fb43c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.033+0000 7fb43bfff640 1 -- 172.21.15.43:0/12336542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb43c1021c0 msgr2=0x7fb43c074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.033+0000 7fb43bfff640 1 --2- 172.21.15.43:0/12336542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb43c1021c0 0x7fb43c074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.033+0000 7fb43bfff640 1 -- 172.21.15.43:0/12336542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43c1ab3a0 con 0x7fb43c075ef0 2024-04-21T08:47:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.033+0000 7fb43bfff640 1 --2- 172.21.15.43:0/12336542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb43c075ef0 0x7fb43c074cd0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fb434009fd0 tx=0x7fb43400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.033+0000 7fb438ff9640 1 -- 172.21.15.43:0/12336542 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb434017070 con 0x7fb43c075ef0 2024-04-21T08:47:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.033+0000 7fb438ff9640 1 -- 172.21.15.43:0/12336542 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb434003e20 con 0x7fb43c075ef0 2024-04-21T08:47:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.033+0000 7fb441f85640 1 -- 172.21.15.43:0/12336542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb43c1ab690 con 0x7fb43c075ef0 2024-04-21T08:47:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.033+0000 7fb438ff9640 1 -- 172.21.15.43:0/12336542 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb434010460 con 0x7fb43c075ef0 2024-04-21T08:47:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.033+0000 7fb441f85640 1 -- 172.21.15.43:0/12336542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb43c1abb60 con 0x7fb43c075ef0 2024-04-21T08:47:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.036+0000 7fb438ff9640 1 -- 172.21.15.43:0/12336542 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb434003970 con 0x7fb43c075ef0 2024-04-21T08:47:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.036+0000 7fb444b73740 1 Processor -- start 2024-04-21T08:47:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.036+0000 7fb444b73740 1 -- start start 2024-04-21T08:47:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.036+0000 7fb444b73740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e538d0 0x559af7e52620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.036+0000 7fb444b73740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559af7e52b60 0x559af7e52fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.036+0000 7fb444b73740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af7e5c0e0 0x559af7e5e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.036+0000 7fb444b73740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559af7db78d0 con 0x559af7e538d0 2024-04-21T08:47:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.036+0000 7fb444b73740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559af7d54e80 con 0x559af7e52b60 2024-04-21T08:47:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.036+0000 7fb444b73740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559af7db3890 con 0x559af7e5c0e0 2024-04-21T08:47:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.036+0000 7fb43affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e538d0 0x559af7e52620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.036+0000 7fb43affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e538d0 0x559af7e52620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42472/0 (socket says 172.21.15.43:42472) 2024-04-21T08:47:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.036+0000 7fb43affd640 1 -- 172.21.15.43:0/3779303286 learned_addr learned my addr 172.21.15.43:0/3779303286 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.036+0000 7fb438ff9640 1 --2- 172.21.15.43:0/12336542 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb4240a3dc0 0x7fb4240a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.037+0000 7fb43b7fe640 1 --2- 172.21.15.43:0/3779303286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559af7e52b60 0x559af7e52fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.037+0000 7fb43bfff640 1 --2- 172.21.15.43:0/12336542 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb4240a3dc0 0x7fb4240a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.037+0000 7fb43affd640 1 --2- 172.21.15.43:0/3779303286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af7e5c0e0 0x559af7e5e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.037+0000 7fb43affd640 1 -- 172.21.15.43:0/3779303286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af7e5c0e0 msgr2=0x559af7e5e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.037+0000 7fb43affd640 1 --2- 172.21.15.43:0/3779303286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af7e5c0e0 0x559af7e5e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.037+0000 7fb43affd640 1 -- 172.21.15.43:0/3779303286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559af7e52b60 msgr2=0x559af7e52fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.037+0000 7fb43affd640 1 --2- 172.21.15.43:0/3779303286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559af7e52b60 0x559af7e52fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.037+0000 7fb43affd640 1 -- 172.21.15.43:0/3779303286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559af7d76320 con 0x559af7e538d0 2024-04-21T08:47:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.037+0000 7fb43b7fe640 1 --2- 172.21.15.43:0/3779303286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559af7e52b60 0x559af7e52fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.037+0000 7fb438ff9640 1 -- 172.21.15.43:0/12336542 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fb4340d0a10 con 0x7fb43c075ef0 2024-04-21T08:47:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.037+0000 7fb43affd640 1 --2- 172.21.15.43:0/3779303286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e538d0 0x559af7e52620 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fb430000c00 tx=0x7fb430005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bb8bc2eebd9fc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.037+0000 7fb43bfff640 1 --2- 172.21.15.43:0/12336542 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb4240a3dc0 0x7fb4240a6280 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb43409c290 tx=0x7fb434087040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.037+0000 7fb421ffb640 1 -- 172.21.15.43:0/3779303286 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430004320 con 0x559af7e538d0 2024-04-21T08:47:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.037+0000 7fb421ffb640 1 -- 172.21.15.43:0/3779303286 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb43002cc90 con 0x559af7e538d0 2024-04-21T08:47:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.037+0000 7fb421ffb640 1 -- 172.21.15.43:0/3779303286 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430004920 con 0x559af7e538d0 2024-04-21T08:47:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.038+0000 7fb444b73740 1 -- 172.21.15.43:0/3779303286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e538d0 msgr2=0x559af7e52620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.038+0000 7fb444b73740 1 --2- 172.21.15.43:0/3779303286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e538d0 0x559af7e52620 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fb430000c00 tx=0x7fb430005b50 comp rx=0 tx=0).stop 2024-04-21T08:47:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.038+0000 7fb444b73740 1 -- 172.21.15.43:0/3779303286 shutdown_connections 2024-04-21T08:47:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.038+0000 7fb444b73740 1 --2- 172.21.15.43:0/3779303286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af7e5c0e0 0x559af7e5e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.038+0000 7fb444b73740 1 --2- 172.21.15.43:0/3779303286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559af7e52b60 0x559af7e52fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.038+0000 7fb444b73740 1 --2- 172.21.15.43:0/3779303286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e538d0 0x559af7e52620 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.038+0000 7fb444b73740 1 -- 172.21.15.43:0/3779303286 >> 172.21.15.43:0/3779303286 conn(0x559af7e19c90 msgr2=0x559af7e4d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.038+0000 7fb444b73740 1 -- 172.21.15.43:0/3779303286 shutdown_connections 2024-04-21T08:47:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.038+0000 7fb444b73740 1 -- 172.21.15.43:0/3779303286 wait complete. 2024-04-21T08:47:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.038+0000 7fb444b73740 1 -- start start 2024-04-21T08:47:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.039+0000 7fb444b73740 1 Processor -- start 2024-04-21T08:47:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.039+0000 7fb444b73740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e52b60 0x559af7e5b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.039+0000 7fb444b73740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af7e538d0 0x559af7e58dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.039+0000 7fb444b73740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559af7e5c0e0 0x559af7e59330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.039+0000 7fb444b73740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559af7e5edb0 con 0x559af7e52b60 2024-04-21T08:47:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.039+0000 7fb444b73740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559af7d3feb0 con 0x559af7e5c0e0 2024-04-21T08:47:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.039+0000 7fb444b73740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559af7e5ef30 con 0x559af7e538d0 2024-04-21T08:47:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.039+0000 7fb444b73740 10 client.0 _ll_register_callbacks cb 0x559af7d6cb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.039+0000 7fb43affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e52b60 0x559af7e5b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.039+0000 7fb43affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e52b60 0x559af7e5b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42478/0 (socket says 172.21.15.43:42478) 2024-04-21T08:47:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.039+0000 7fb43affd640 1 -- 172.21.15.43:0/1624996534 learned_addr learned my addr 172.21.15.43:0/1624996534 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.039+0000 7fb43b7fe640 1 --2- 172.21.15.43:0/1624996534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af7e538d0 0x559af7e58dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.039+0000 7fb43affd640 1 --2- 172.21.15.43:0/1624996534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559af7e5c0e0 0x559af7e59330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.039+0000 7fb43affd640 1 -- 172.21.15.43:0/1624996534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af7e538d0 msgr2=0x559af7e58dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.039+0000 7fb43affd640 1 --2- 172.21.15.43:0/1624996534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af7e538d0 0x559af7e58dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.039+0000 7fb43affd640 1 -- 172.21.15.43:0/1624996534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559af7e5c0e0 msgr2=0x559af7e59330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.039+0000 7fb43affd640 1 --2- 172.21.15.43:0/1624996534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559af7e5c0e0 0x559af7e59330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.039+0000 7fb43affd640 1 -- 172.21.15.43:0/1624996534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559af7aca210 con 0x559af7e52b60 2024-04-21T08:47:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.039+0000 7fb43b7fe640 1 --2- 172.21.15.43:0/1624996534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af7e538d0 0x559af7e58dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.040+0000 7fb43affd640 1 --2- 172.21.15.43:0/1624996534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e52b60 0x559af7e5b9d0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fb430036040 tx=0x7fb4300058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2764de2b80d926c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.040+0000 7fb41affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.040+0000 7fb41affd640 1 -- 172.21.15.43:0/1624996534 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430003800 con 0x559af7e52b60 2024-04-21T08:47:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.040+0000 7fb41affd640 1 -- 172.21.15.43:0/1624996534 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fb4300039a0 con 0x559af7e52b60 2024-04-21T08:47:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.040+0000 7fb444b73740 1 -- 172.21.15.43:0/1624996534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb43002cc90 con 0x559af7e52b60 2024-04-21T08:47:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.040+0000 7fb41affd640 1 -- 172.21.15.43:0/1624996534 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb43005f050 con 0x559af7e52b60 2024-04-21T08:47:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.040+0000 7fb444b73740 10 client.19494 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.040+0000 7fb444b73740 1 -- 172.21.15.43:0/1624996534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559af7b96620 con 0x559af7e52b60 2024-04-21T08:47:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.040+0000 7fb41affd640 1 -- 172.21.15.43:0/1624996534 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb43002cc90 con 0x559af7e52b60 2024-04-21T08:47:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.040+0000 7fb41affd640 1 -- 172.21.15.43:0/1624996534 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb430043a50 con 0x559af7e52b60 2024-04-21T08:47:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.040+0000 7fb444b73740 10 client.19494 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.040+0000 7fb444b73740 10 client.19494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.040+0000 7fb444b73740 20 client.19494 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.040+0000 7fb444b73740 1 --2- 172.21.15.43:0/1624996534 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559af7e6bee0 0x559af7e6e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.040+0000 7fb444b73740 4 client.19494 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.040+0000 7fb444b73740 1 -- 172.21.15.43:0/1624996534 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559af7dcb880 con 0x559af7e6bee0 2024-04-21T08:47:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.041+0000 7fb43b7fe640 1 --2- 172.21.15.43:0/1624996534 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559af7e6bee0 0x559af7e6e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.041+0000 7fb43b7fe640 1 --2- 172.21.15.43:0/1624996534 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559af7e6bee0 0x559af7e6e2d0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10db1944fd45383 server_cookie=a6c43f8a2140b424 in_seq=0 out_seq=0 2024-04-21T08:47:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.041+0000 7fb41affd640 10 client.19494 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.042+0000 7fb41affd640 1 -- 172.21.15.43:0/1624996534 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559af7dcb880 con 0x559af7e6bee0 2024-04-21T08:47:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.042+0000 7fb41affd640 10 client.19494 handle_command_reply: tid=0 2024-04-21T08:47:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.042+0000 7fb444b73740 1 client.19494 shutdown 2024-04-21T08:47:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.042+0000 7fb444b73740 1 -- 172.21.15.43:0/1624996534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e52b60 msgr2=0x559af7e5b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.042+0000 7fb444b73740 1 --2- 172.21.15.43:0/1624996534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e52b60 0x559af7e5b9d0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fb430036040 tx=0x7fb4300058b0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.042+0000 7fb444b73740 1 -- 172.21.15.43:0/1624996534 shutdown_connections 2024-04-21T08:47:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.042+0000 7fb444b73740 1 --2- 172.21.15.43:0/1624996534 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559af7e6bee0 0x559af7e6e2d0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.042+0000 7fb444b73740 1 --2- 172.21.15.43:0/1624996534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559af7e5c0e0 0x559af7e59330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.042+0000 7fb444b73740 1 --2- 172.21.15.43:0/1624996534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af7e538d0 0x559af7e58dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.042+0000 7fb41affd640 0 client.19494 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.042+0000 7fb444b73740 1 --2- 172.21.15.43:0/1624996534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e52b60 0x559af7e5b9d0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.042+0000 7fb444b73740 1 -- 172.21.15.43:0/1624996534 >> 172.21.15.43:0/1624996534 conn(0x559af7e19c90 msgr2=0x559af7e4f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.042+0000 7fb444b73740 1 -- 172.21.15.43:0/1624996534 shutdown_connections 2024-04-21T08:47:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.042+0000 7fb444b73740 1 -- 172.21.15.43:0/1624996534 wait complete. 2024-04-21T08:47:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.043+0000 7fb444b73740 20 client.19494 trim_cache size 0 max 16384 2024-04-21T08:47:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.058+0000 7fb444b73740 1 Processor -- start 2024-04-21T08:47:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.058+0000 7fb444b73740 1 -- start start 2024-04-21T08:47:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.058+0000 7fb444b73740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af7e52b60 0x559af7e67cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.058+0000 7fb444b73740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e538d0 0x559af7e52080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.058+0000 7fb444b73740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559af7e5c0e0 0x559af7e5b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.058+0000 7fb444b73740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1b59c0 con 0x559af7e538d0 2024-04-21T08:47:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.058+0000 7fb444b73740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1b7500 con 0x559af7e5c0e0 2024-04-21T08:47:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.058+0000 7fb444b73740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1b6090 con 0x559af7e52b60 2024-04-21T08:47:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.058+0000 7fb43b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e538d0 0x559af7e52080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.058+0000 7fb43b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e538d0 0x559af7e52080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42490/0 (socket says 172.21.15.43:42490) 2024-04-21T08:47:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.058+0000 7fb43b7fe640 1 -- 172.21.15.43:0/20685556 learned_addr learned my addr 172.21.15.43:0/20685556 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.059+0000 7fb43affd640 1 --2- 172.21.15.43:0/20685556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af7e52b60 0x559af7e67cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.059+0000 7fb43affd640 1 --2- 172.21.15.43:0/20685556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559af7e5c0e0 0x559af7e5b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.059+0000 7fb43b7fe640 1 -- 172.21.15.43:0/20685556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af7e52b60 msgr2=0x559af7e67cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.059+0000 7fb43b7fe640 1 --2- 172.21.15.43:0/20685556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af7e52b60 0x559af7e67cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.059+0000 7fb43b7fe640 1 -- 172.21.15.43:0/20685556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559af7e5c0e0 msgr2=0x559af7e5b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.059+0000 7fb43b7fe640 1 --2- 172.21.15.43:0/20685556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559af7e5c0e0 0x559af7e5b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.059+0000 7fb43b7fe640 1 -- 172.21.15.43:0/20685556 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559af7e51e30 con 0x559af7e538d0 2024-04-21T08:47:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.059+0000 7fb43affd640 1 --2- 172.21.15.43:0/20685556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af7e52b60 0x559af7e67cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.059+0000 7fb43b7fe640 1 --2- 172.21.15.43:0/20685556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e538d0 0x559af7e52080 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x559af7e54030 tx=0x7fb42c014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a13e4ccf7dcf75 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.059+0000 7fb4217fa640 1 -- 172.21.15.43:0/20685556 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c024070 con 0x559af7e538d0 2024-04-21T08:47:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.059+0000 7fb4217fa640 1 -- 172.21.15.43:0/20685556 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb42c01e430 con 0x559af7e538d0 2024-04-21T08:47:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.059+0000 7fb4217fa640 1 -- 172.21.15.43:0/20685556 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c003a80 con 0x559af7e538d0 2024-04-21T08:47:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.060+0000 7fb444b73740 1 -- 172.21.15.43:0/20685556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e538d0 msgr2=0x559af7e52080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.060+0000 7fb444b73740 1 --2- 172.21.15.43:0/20685556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e538d0 0x559af7e52080 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x559af7e54030 tx=0x7fb42c014840 comp rx=0 tx=0).stop 2024-04-21T08:47:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.060+0000 7fb444b73740 1 -- 172.21.15.43:0/20685556 shutdown_connections 2024-04-21T08:47:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.060+0000 7fb444b73740 1 --2- 172.21.15.43:0/20685556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559af7e5c0e0 0x559af7e5b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.060+0000 7fb444b73740 1 --2- 172.21.15.43:0/20685556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e538d0 0x559af7e52080 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.060+0000 7fb444b73740 1 --2- 172.21.15.43:0/20685556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af7e52b60 0x559af7e67cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.060+0000 7fb444b73740 1 -- 172.21.15.43:0/20685556 >> 172.21.15.43:0/20685556 conn(0x559af7e19c90 msgr2=0x559af7e576c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.060+0000 7fb444b73740 1 -- 172.21.15.43:0/20685556 shutdown_connections 2024-04-21T08:47:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.060+0000 7fb444b73740 1 -- 172.21.15.43:0/20685556 wait complete. 2024-04-21T08:47:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.060+0000 7fb444b73740 1 -- start start 2024-04-21T08:47:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.060+0000 7fb444b73740 1 Processor -- start 2024-04-21T08:47:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.061+0000 7fb444b73740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559af7e52b60 0x559af7e559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.061+0000 7fb444b73740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af7e538d0 0x559af7e55f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.061+0000 7fb444b73740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e5c0e0 0x559af7e56440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.061+0000 7fb444b73740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1b6e30 con 0x559af7e5c0e0 2024-04-21T08:47:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.061+0000 7fb444b73740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1b6760 con 0x559af7e52b60 2024-04-21T08:47:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.061+0000 7fb444b73740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1b39c0 con 0x559af7e538d0 2024-04-21T08:47:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.061+0000 7fb444b73740 10 client.0 _ll_register_callbacks cb 0x7fb430009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.061+0000 7fb43affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e5c0e0 0x559af7e56440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.061+0000 7fb43affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e5c0e0 0x559af7e56440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42506/0 (socket says 172.21.15.43:42506) 2024-04-21T08:47:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.061+0000 7fb43affd640 1 -- 172.21.15.43:0/3543584636 learned_addr learned my addr 172.21.15.43:0/3543584636 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.061+0000 7fb43b7fe640 1 --2- 172.21.15.43:0/3543584636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af7e538d0 0x559af7e55f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.061+0000 7fb43affd640 1 --2- 172.21.15.43:0/3543584636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559af7e52b60 0x559af7e559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.061+0000 7fb43affd640 1 -- 172.21.15.43:0/3543584636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af7e538d0 msgr2=0x559af7e55f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.061+0000 7fb43affd640 1 --2- 172.21.15.43:0/3543584636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af7e538d0 0x559af7e55f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.061+0000 7fb43affd640 1 -- 172.21.15.43:0/3543584636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559af7e52b60 msgr2=0x559af7e559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.061+0000 7fb43affd640 1 --2- 172.21.15.43:0/3543584636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559af7e52b60 0x559af7e559c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.061+0000 7fb43affd640 1 -- 172.21.15.43:0/3543584636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559af7ed1130 con 0x559af7e5c0e0 2024-04-21T08:47:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.061+0000 7fb43affd640 1 --2- 172.21.15.43:0/3543584636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e5c0e0 0x559af7e56440 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fb43002c170 tx=0x7fb4300058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d70c372569dcf1a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.061+0000 7fb41affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.061+0000 7fb41affd640 1 -- 172.21.15.43:0/3543584636 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430067030 con 0x559af7e5c0e0 2024-04-21T08:47:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.061+0000 7fb41affd640 1 -- 172.21.15.43:0/3543584636 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fb430048040 con 0x559af7e5c0e0 2024-04-21T08:47:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.061+0000 7fb444b73740 1 -- 172.21.15.43:0/3543584636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb42c01e430 con 0x559af7e5c0e0 2024-04-21T08:47:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.062+0000 7fb41affd640 1 -- 172.21.15.43:0/3543584636 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb430071050 con 0x559af7e5c0e0 2024-04-21T08:47:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.062+0000 7fb444b73740 10 client.19500 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.062+0000 7fb444b73740 1 -- 172.21.15.43:0/3543584636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559af7ed22b0 con 0x559af7e5c0e0 2024-04-21T08:47:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.062+0000 7fb41affd640 1 -- 172.21.15.43:0/3543584636 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb42c01e430 con 0x559af7e5c0e0 2024-04-21T08:47:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.062+0000 7fb41affd640 1 -- 172.21.15.43:0/3543584636 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb430003800 con 0x559af7e5c0e0 2024-04-21T08:47:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.062+0000 7fb444b73740 10 client.19500 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.062+0000 7fb444b73740 10 client.19500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.062+0000 7fb444b73740 20 client.19500 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.062+0000 7fb444b73740 1 --2- 172.21.15.43:0/3543584636 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559af7ed2910 0x559af7ed4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.062+0000 7fb444b73740 4 client.19500 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:47:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.062+0000 7fb444b73740 1 -- 172.21.15.43:0/3543584636 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559af7e52850 con 0x559af7ed2910 2024-04-21T08:47:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.062+0000 7fb43b7fe640 1 --2- 172.21.15.43:0/3543584636 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559af7ed2910 0x559af7ed4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.063+0000 7fb43b7fe640 1 --2- 172.21.15.43:0/3543584636 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559af7ed2910 0x559af7ed4d30 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ddc710e2f93dc72 server_cookie=2391c4cddfd8b3af in_seq=0 out_seq=0 2024-04-21T08:47:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.063+0000 7fb41affd640 10 client.19500 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.077+0000 7fb41affd640 1 -- 172.21.15.43:0/3543584636 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559af7e52850 con 0x559af7ed2910 2024-04-21T08:47:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.077+0000 7fb41affd640 10 client.19500 handle_command_reply: tid=0 2024-04-21T08:47:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.077+0000 7fb444b73740 1 client.19500 shutdown 2024-04-21T08:47:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.077+0000 7fb444b73740 1 -- 172.21.15.43:0/3543584636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e5c0e0 msgr2=0x559af7e56440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.077+0000 7fb444b73740 1 --2- 172.21.15.43:0/3543584636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e5c0e0 0x559af7e56440 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fb43002c170 tx=0x7fb4300058b0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.077+0000 7fb444b73740 1 -- 172.21.15.43:0/3543584636 shutdown_connections 2024-04-21T08:47:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.077+0000 7fb444b73740 1 --2- 172.21.15.43:0/3543584636 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559af7ed2910 0x559af7ed4d30 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.077+0000 7fb444b73740 1 --2- 172.21.15.43:0/3543584636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559af7e5c0e0 0x559af7e56440 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.077+0000 7fb444b73740 1 --2- 172.21.15.43:0/3543584636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559af7e538d0 0x559af7e55f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.077+0000 7fb444b73740 1 --2- 172.21.15.43:0/3543584636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559af7e52b60 0x559af7e559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.077+0000 7fb41affd640 0 client.19500 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.077+0000 7fb444b73740 1 -- 172.21.15.43:0/3543584636 >> 172.21.15.43:0/3543584636 conn(0x559af7e19c90 msgr2=0x559af7e57b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.077+0000 7fb444b73740 1 -- 172.21.15.43:0/3543584636 shutdown_connections 2024-04-21T08:47:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.077+0000 7fb444b73740 1 -- 172.21.15.43:0/3543584636 wait complete. 2024-04-21T08:47:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.077+0000 7fb444b73740 20 client.19500 trim_cache size 0 max 16384 2024-04-21T08:47:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:47:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:47:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:47:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:47:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.078+0000 7fb441f85640 1 -- 172.21.15.43:0/12336542 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb4240a3dc0 msgr2=0x7fb4240a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.078+0000 7fb441f85640 1 --2- 172.21.15.43:0/12336542 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb4240a3dc0 0x7fb4240a6280 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb43409c290 tx=0x7fb434087040 comp rx=0 tx=0).stop 2024-04-21T08:47:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.078+0000 7fb441f85640 1 -- 172.21.15.43:0/12336542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb43c075ef0 msgr2=0x7fb43c074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.078+0000 7fb441f85640 1 --2- 172.21.15.43:0/12336542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb43c075ef0 0x7fb43c074cd0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fb434009fd0 tx=0x7fb43400b260 comp rx=0 tx=0).stop 2024-04-21T08:47:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.078+0000 7fb441f85640 1 -- 172.21.15.43:0/12336542 shutdown_connections 2024-04-21T08:47:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.078+0000 7fb441f85640 1 --2- 172.21.15.43:0/12336542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb43c075ef0 0x7fb43c074cd0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.078+0000 7fb441f85640 1 --2- 172.21.15.43:0/12336542 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb4240a3dc0 0x7fb4240a6280 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.078+0000 7fb441f85640 1 --2- 172.21.15.43:0/12336542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb43c107290 0x7fb43c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.078+0000 7fb441f85640 1 --2- 172.21.15.43:0/12336542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb43c1021c0 0x7fb43c074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.078+0000 7fb441f85640 1 -- 172.21.15.43:0/12336542 >> 172.21.15.43:0/12336542 conn(0x7fb43c0fda20 msgr2=0x7fb43c0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.078+0000 7fb441f85640 1 -- 172.21.15.43:0/12336542 shutdown_connections 2024-04-21T08:47:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:23.078+0000 7fb441f85640 1 -- 172.21.15.43:0/12336542 wait complete. 2024-04-21T08:47:24.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:24 smithi175 ceph-mon[31920]: pgmap v262: 129 pgs: 129 active+clean; 4.6 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-04-21T08:47:24.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:24 smithi112 ceph-mon[31085]: pgmap v262: 129 pgs: 129 active+clean; 4.6 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-04-21T08:47:24.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:24 smithi043 ceph-mon[27978]: pgmap v262: 129 pgs: 129 active+clean; 4.6 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-04-21T08:47:26.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:26 smithi175 ceph-mon[31920]: pgmap v263: 129 pgs: 129 active+clean; 4.6 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-04-21T08:47:26.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:26 smithi112 ceph-mon[31085]: pgmap v263: 129 pgs: 129 active+clean; 4.6 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-04-21T08:47:26.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:26 smithi043 ceph-mon[27978]: pgmap v263: 129 pgs: 129 active+clean; 4.6 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-04-21T08:47:28.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:28 smithi175 ceph-mon[31920]: pgmap v264: 129 pgs: 129 active+clean; 4.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:47:28.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:28 smithi112 ceph-mon[31085]: pgmap v264: 129 pgs: 129 active+clean; 4.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:47:28.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:28 smithi043 ceph-mon[27978]: pgmap v264: 129 pgs: 129 active+clean; 4.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:47:30.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:30 smithi175 ceph-mon[31920]: pgmap v265: 129 pgs: 129 active+clean; 4.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-21T08:47:30.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:30 smithi112 ceph-mon[31085]: pgmap v265: 129 pgs: 129 active+clean; 4.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-21T08:47:30.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:30 smithi043 ceph-mon[27978]: pgmap v265: 129 pgs: 129 active+clean; 4.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-21T08:47:32.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:32 smithi175 ceph-mon[31920]: pgmap v266: 129 pgs: 129 active+clean; 5.0 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 61 KiB/s wr, 23 op/s 2024-04-21T08:47:32.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:32 smithi112 ceph-mon[31085]: pgmap v266: 129 pgs: 129 active+clean; 5.0 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 61 KiB/s wr, 23 op/s 2024-04-21T08:47:32.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:32 smithi043 ceph-mon[27978]: pgmap v266: 129 pgs: 129 active+clean; 5.0 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 61 KiB/s wr, 23 op/s 2024-04-21T08:47:34.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:34 smithi175 ceph-mon[31920]: pgmap v267: 129 pgs: 129 active+clean; 5.0 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:47:34.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:34 smithi112 ceph-mon[31085]: pgmap v267: 129 pgs: 129 active+clean; 5.0 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:47:34.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:34 smithi043 ceph-mon[27978]: pgmap v267: 129 pgs: 129 active+clean; 5.0 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:47:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.180+0000 7fe51eba4640 1 -- 172.21.15.43:0/2266232447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe518103b00 msgr2=0x7fe51810bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.180+0000 7fe51eba4640 1 --2- 172.21.15.43:0/2266232447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe518103b00 0x7fe51810bfe0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009900 tx=0x7fe50c02bd60 comp rx=0 tx=0).stop 2024-04-21T08:47:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.180+0000 7fe51eba4640 1 -- 172.21.15.43:0/2266232447 shutdown_connections 2024-04-21T08:47:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.180+0000 7fe51eba4640 1 --2- 172.21.15.43:0/2266232447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe51810c520 0x7fe51810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.180+0000 7fe51eba4640 1 --2- 172.21.15.43:0/2266232447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe518103b00 0x7fe51810bfe0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.180+0000 7fe51eba4640 1 --2- 172.21.15.43:0/2266232447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5181031e0 0x7fe5181035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.180+0000 7fe51eba4640 1 -- 172.21.15.43:0/2266232447 >> 172.21.15.43:0/2266232447 conn(0x7fe518077fd0 msgr2=0x7fe5180783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.180+0000 7fe51eba4640 1 -- 172.21.15.43:0/2266232447 shutdown_connections 2024-04-21T08:47:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.180+0000 7fe51eba4640 1 -- 172.21.15.43:0/2266232447 wait complete. 2024-04-21T08:47:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.181+0000 7fe51eba4640 1 Processor -- start 2024-04-21T08:47:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.181+0000 7fe51eba4640 1 -- start start 2024-04-21T08:47:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.181+0000 7fe51eba4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5181031e0 0x7fe518198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.181+0000 7fe51eba4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe518103b00 0x7fe518198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.181+0000 7fe51eba4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe51810c520 0x7fe51819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.181+0000 7fe51eba4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe5181121b0 con 0x7fe51810c520 2024-04-21T08:47:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.181+0000 7fe51eba4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe518112030 con 0x7fe5181031e0 2024-04-21T08:47:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.181+0000 7fe51eba4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe518112330 con 0x7fe518103b00 2024-04-21T08:47:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.181+0000 7fe51d141640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe51810c520 0x7fe51819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.181+0000 7fe51d141640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe51810c520 0x7fe51819cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37416/0 (socket says 172.21.15.43:37416) 2024-04-21T08:47:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.181+0000 7fe51d141640 1 -- 172.21.15.43:0/4134779577 learned_addr learned my addr 172.21.15.43:0/4134779577 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.181+0000 7fe51c940640 1 --2- 172.21.15.43:0/4134779577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5181031e0 0x7fe518198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.181+0000 7fe517fff640 1 --2- 172.21.15.43:0/4134779577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe518103b00 0x7fe518198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.181+0000 7fe51d141640 1 -- 172.21.15.43:0/4134779577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe518103b00 msgr2=0x7fe518198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.181+0000 7fe51d141640 1 --2- 172.21.15.43:0/4134779577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe518103b00 0x7fe518198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.181+0000 7fe51d141640 1 -- 172.21.15.43:0/4134779577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5181031e0 msgr2=0x7fe518198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.181+0000 7fe51d141640 1 --2- 172.21.15.43:0/4134779577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5181031e0 0x7fe518198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.181+0000 7fe51d141640 1 -- 172.21.15.43:0/4134779577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51819d680 con 0x7fe51810c520 2024-04-21T08:47:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.182+0000 7fe517fff640 1 --2- 172.21.15.43:0/4134779577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe518103b00 0x7fe518198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.182+0000 7fe51d141640 1 --2- 172.21.15.43:0/4134779577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe51810c520 0x7fe51819cf50 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fe51000cce0 tx=0x7fe51000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.182+0000 7fe515ffb640 1 -- 172.21.15.43:0/4134779577 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51000f450 con 0x7fe51810c520 2024-04-21T08:47:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.182+0000 7fe515ffb640 1 -- 172.21.15.43:0/4134779577 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe51000faa0 con 0x7fe51810c520 2024-04-21T08:47:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.182+0000 7fe51eba4640 1 -- 172.21.15.43:0/4134779577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe51819d970 con 0x7fe51810c520 2024-04-21T08:47:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.182+0000 7fe515ffb640 1 -- 172.21.15.43:0/4134779577 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510018b70 con 0x7fe51810c520 2024-04-21T08:47:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.182+0000 7fe51eba4640 1 -- 172.21.15.43:0/4134779577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5181a5200 con 0x7fe51810c520 2024-04-21T08:47:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.184+0000 7fe515ffb640 1 -- 172.21.15.43:0/4134779577 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe510018d10 con 0x7fe51810c520 2024-04-21T08:47:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.185+0000 7fe515ffb640 1 --2- 172.21.15.43:0/4134779577 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe5000a3dc0 0x7fe5000a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.185+0000 7fe521793740 1 Processor -- start 2024-04-21T08:47:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.185+0000 7fe521793740 1 -- start start 2024-04-21T08:47:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.185+0000 7fe515ffb640 1 -- 172.21.15.43:0/4134779577 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fe5100d1030 con 0x7fe51810c520 2024-04-21T08:47:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.185+0000 7fe51c940640 1 --2- 172.21.15.43:0/4134779577 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe5000a3dc0 0x7fe5000a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.185+0000 7fe521793740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecb3aaf8d0 0x55ecb3aae620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.185+0000 7fe521793740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaeb60 0x55ecb3aaefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.185+0000 7fe521793740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3ab80e0 0x55ecb3aba4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.185+0000 7fe521793740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ecb3a138d0 con 0x55ecb3aaeb60 2024-04-21T08:47:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.185+0000 7fe521793740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ecb39b0e80 con 0x55ecb3ab80e0 2024-04-21T08:47:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.185+0000 7fe521793740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ecb3a0f890 con 0x55ecb3aaf8d0 2024-04-21T08:47:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.185+0000 7fe517fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaeb60 0x55ecb3aaefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.185+0000 7fe517fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaeb60 0x55ecb3aaefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37422/0 (socket says 172.21.15.43:37422) 2024-04-21T08:47:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.185+0000 7fe517fff640 1 -- 172.21.15.43:0/4129761586 learned_addr learned my addr 172.21.15.43:0/4129761586 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.185+0000 7fe51d141640 1 --2- 172.21.15.43:0/4129761586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3ab80e0 0x55ecb3aba4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.186+0000 7fe51c940640 1 --2- 172.21.15.43:0/4129761586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecb3aaf8d0 0x55ecb3aae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.186+0000 7fe51c940640 1 --2- 172.21.15.43:0/4134779577 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe5000a3dc0 0x7fe5000a6280 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe508005fd0 tx=0x7fe50800ba60 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.186+0000 7fe517fff640 1 -- 172.21.15.43:0/4129761586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecb3aaf8d0 msgr2=0x55ecb3aae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.186+0000 7fe517fff640 1 --2- 172.21.15.43:0/4129761586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecb3aaf8d0 0x55ecb3aae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.186+0000 7fe517fff640 1 -- 172.21.15.43:0/4129761586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3ab80e0 msgr2=0x55ecb3aba4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.186+0000 7fe517fff640 1 --2- 172.21.15.43:0/4129761586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3ab80e0 0x55ecb3aba4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.186+0000 7fe517fff640 1 -- 172.21.15.43:0/4129761586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecb39d2320 con 0x55ecb3aaeb60 2024-04-21T08:47:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.186+0000 7fe51d141640 1 --2- 172.21.15.43:0/4129761586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3ab80e0 0x55ecb3aba4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.186+0000 7fe517fff640 1 --2- 172.21.15.43:0/4129761586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaeb60 0x55ecb3aaefe0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fe50c000c00 tx=0x7fe50c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b942cae0bb229343 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.186+0000 7fe4feffd640 1 -- 172.21.15.43:0/4129761586 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c004400 con 0x55ecb3aaeb60 2024-04-21T08:47:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.186+0000 7fe4feffd640 1 -- 172.21.15.43:0/4129761586 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe50c02cdc0 con 0x55ecb3aaeb60 2024-04-21T08:47:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.186+0000 7fe4feffd640 1 -- 172.21.15.43:0/4129761586 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c004920 con 0x55ecb3aaeb60 2024-04-21T08:47:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.187+0000 7fe521793740 1 -- 172.21.15.43:0/4129761586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaeb60 msgr2=0x55ecb3aaefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.187+0000 7fe521793740 1 --2- 172.21.15.43:0/4129761586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaeb60 0x55ecb3aaefe0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fe50c000c00 tx=0x7fe50c005e80 comp rx=0 tx=0).stop 2024-04-21T08:47:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.187+0000 7fe521793740 1 -- 172.21.15.43:0/4129761586 shutdown_connections 2024-04-21T08:47:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.187+0000 7fe521793740 1 --2- 172.21.15.43:0/4129761586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3ab80e0 0x55ecb3aba4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.187+0000 7fe521793740 1 --2- 172.21.15.43:0/4129761586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaeb60 0x55ecb3aaefe0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.187+0000 7fe521793740 1 --2- 172.21.15.43:0/4129761586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecb3aaf8d0 0x55ecb3aae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.187+0000 7fe521793740 1 -- 172.21.15.43:0/4129761586 >> 172.21.15.43:0/4129761586 conn(0x55ecb3a75c90 msgr2=0x55ecb3aa9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.187+0000 7fe521793740 1 -- 172.21.15.43:0/4129761586 shutdown_connections 2024-04-21T08:47:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.187+0000 7fe521793740 1 -- 172.21.15.43:0/4129761586 wait complete. 2024-04-21T08:47:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.187+0000 7fe521793740 1 -- start start 2024-04-21T08:47:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.187+0000 7fe521793740 1 Processor -- start 2024-04-21T08:47:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.188+0000 7fe521793740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3aaeb60 0x55ecb3ab7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.188+0000 7fe521793740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaf8d0 0x55ecb3ab4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.188+0000 7fe521793740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecb3ab80e0 0x55ecb3ab5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.188+0000 7fe521793740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ecb3abadb0 con 0x55ecb3aaf8d0 2024-04-21T08:47:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.188+0000 7fe521793740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ecb399beb0 con 0x55ecb3aaeb60 2024-04-21T08:47:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.188+0000 7fe521793740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ecb3abaf30 con 0x55ecb3ab80e0 2024-04-21T08:47:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.188+0000 7fe521793740 10 client.0 _ll_register_callbacks cb 0x55ecb39c8b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.188+0000 7fe51c940640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3aaeb60 0x55ecb3ab7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.188+0000 7fe517fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaf8d0 0x55ecb3ab4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.188+0000 7fe51c940640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3aaeb60 0x55ecb3ab7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:35818/0 (socket says 172.21.15.43:35818) 2024-04-21T08:47:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.188+0000 7fe51c940640 1 -- 172.21.15.43:0/3186176145 learned_addr learned my addr 172.21.15.43:0/3186176145 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.188+0000 7fe51d141640 1 --2- 172.21.15.43:0/3186176145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecb3ab80e0 0x55ecb3ab5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.188+0000 7fe51c940640 1 -- 172.21.15.43:0/3186176145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecb3ab80e0 msgr2=0x55ecb3ab5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.188+0000 7fe51c940640 1 --2- 172.21.15.43:0/3186176145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecb3ab80e0 0x55ecb3ab5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.188+0000 7fe51c940640 1 -- 172.21.15.43:0/3186176145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaf8d0 msgr2=0x55ecb3ab4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.188+0000 7fe51c940640 1 --2- 172.21.15.43:0/3186176145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaf8d0 0x55ecb3ab4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.188+0000 7fe51c940640 1 -- 172.21.15.43:0/3186176145 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecb3726210 con 0x55ecb3aaeb60 2024-04-21T08:47:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.188+0000 7fe51c940640 1 --2- 172.21.15.43:0/3186176145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3aaeb60 0x55ecb3ab7ad0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe508003f30 tx=0x7fe508004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=284641cd97fbda83 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.188+0000 7fe4f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-21T08:47:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.189+0000 7fe4f7fff640 1 -- 172.21.15.43:0/3186176145 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe508028470 con 0x55ecb3aaeb60 2024-04-21T08:47:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.189+0000 7fe4f7fff640 1 -- 172.21.15.43:0/3186176145 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fe508028610 con 0x55ecb3aaeb60 2024-04-21T08:47:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.189+0000 7fe521793740 1 -- 172.21.15.43:0/3186176145 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe50c02cdc0 con 0x55ecb3aaeb60 2024-04-21T08:47:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.189+0000 7fe4f7fff640 1 -- 172.21.15.43:0/3186176145 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe508028810 con 0x55ecb3aaeb60 2024-04-21T08:47:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.189+0000 7fe521793740 10 client.28307 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.189+0000 7fe521793740 1 -- 172.21.15.43:0/3186176145 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ecb37f2620 con 0x55ecb3aaeb60 2024-04-21T08:47:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.189+0000 7fe4f7fff640 1 -- 172.21.15.43:0/3186176145 <== mon.2 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe50c02cdc0 con 0x55ecb3aaeb60 2024-04-21T08:47:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.189+0000 7fe4f7fff640 1 -- 172.21.15.43:0/3186176145 <== mon.2 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe50801f9e0 con 0x55ecb3aaeb60 2024-04-21T08:47:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.189+0000 7fe521793740 10 client.28307 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.189+0000 7fe521793740 10 client.28307 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.189+0000 7fe521793740 20 client.28307 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.189+0000 7fe521793740 1 --2- 172.21.15.43:0/3186176145 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ecb3ac8130 0x55ecb3aca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.189+0000 7fe521793740 4 client.28307 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.189+0000 7fe521793740 1 -- 172.21.15.43:0/3186176145 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecb3a27880 con 0x55ecb3ac8130 2024-04-21T08:47:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.190+0000 7fe517fff640 1 --2- 172.21.15.43:0/3186176145 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ecb3ac8130 0x55ecb3aca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.190+0000 7fe517fff640 1 --2- 172.21.15.43:0/3186176145 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ecb3ac8130 0x55ecb3aca520 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92e40d0ea89c93e8 server_cookie=dc2f4fafdb778b38 in_seq=0 out_seq=0 2024-04-21T08:47:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.190+0000 7fe4f7fff640 10 client.28307 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.191+0000 7fe4f7fff640 1 -- 172.21.15.43:0/3186176145 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ecb3a27880 con 0x55ecb3ac8130 2024-04-21T08:47:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.191+0000 7fe4f7fff640 10 client.28307 handle_command_reply: tid=0 2024-04-21T08:47:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.191+0000 7fe521793740 1 client.28307 shutdown 2024-04-21T08:47:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.191+0000 7fe521793740 1 -- 172.21.15.43:0/3186176145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3aaeb60 msgr2=0x55ecb3ab7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.191+0000 7fe521793740 1 --2- 172.21.15.43:0/3186176145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3aaeb60 0x55ecb3ab7ad0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe508003f30 tx=0x7fe508004730 comp rx=0 tx=0).stop 2024-04-21T08:47:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.191+0000 7fe521793740 1 -- 172.21.15.43:0/3186176145 shutdown_connections 2024-04-21T08:47:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.191+0000 7fe521793740 1 --2- 172.21.15.43:0/3186176145 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ecb3ac8130 0x55ecb3aca520 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.191+0000 7fe521793740 1 --2- 172.21.15.43:0/3186176145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecb3ab80e0 0x55ecb3ab5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.191+0000 7fe521793740 1 --2- 172.21.15.43:0/3186176145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaf8d0 0x55ecb3ab4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.191+0000 7fe521793740 1 --2- 172.21.15.43:0/3186176145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3aaeb60 0x55ecb3ab7ad0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.191+0000 7fe4f7fff640 0 client.28307 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.191+0000 7fe521793740 1 -- 172.21.15.43:0/3186176145 >> 172.21.15.43:0/3186176145 conn(0x55ecb3a75c90 msgr2=0x55ecb3aab5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.191+0000 7fe521793740 1 -- 172.21.15.43:0/3186176145 shutdown_connections 2024-04-21T08:47:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.191+0000 7fe521793740 1 -- 172.21.15.43:0/3186176145 wait complete. 2024-04-21T08:47:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.191+0000 7fe521793740 20 client.28307 trim_cache size 0 max 16384 2024-04-21T08:47:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.206+0000 7fe521793740 1 Processor -- start 2024-04-21T08:47:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.206+0000 7fe521793740 1 -- start start 2024-04-21T08:47:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.206+0000 7fe521793740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaeb60 0x55ecb3ac3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.206+0000 7fe521793740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecb3aaf8d0 0x55ecb3ab7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.206+0000 7fe521793740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3ab80e0 0x55ecb3ab7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.206+0000 7fe521793740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe5181af4d0 con 0x55ecb3aaeb60 2024-04-21T08:47:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.206+0000 7fe521793740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe5181b1010 con 0x55ecb3ab80e0 2024-04-21T08:47:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.206+0000 7fe521793740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe5181afba0 con 0x55ecb3aaf8d0 2024-04-21T08:47:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.206+0000 7fe51c940640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaeb60 0x55ecb3ac3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.206+0000 7fe51c940640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaeb60 0x55ecb3ac3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37440/0 (socket says 172.21.15.43:37440) 2024-04-21T08:47:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.206+0000 7fe51c940640 1 -- 172.21.15.43:0/2890594471 learned_addr learned my addr 172.21.15.43:0/2890594471 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.207+0000 7fe51d141640 1 --2- 172.21.15.43:0/2890594471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3ab80e0 0x55ecb3ab7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.207+0000 7fe51c940640 1 -- 172.21.15.43:0/2890594471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecb3aaf8d0 msgr2=0x55ecb3ab7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.207+0000 7fe51c940640 1 --2- 172.21.15.43:0/2890594471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecb3aaf8d0 0x55ecb3ab7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.207+0000 7fe51c940640 1 -- 172.21.15.43:0/2890594471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3ab80e0 msgr2=0x55ecb3ab7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.207+0000 7fe51c940640 1 --2- 172.21.15.43:0/2890594471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3ab80e0 0x55ecb3ab7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.207+0000 7fe51c940640 1 -- 172.21.15.43:0/2890594471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecb3aca890 con 0x55ecb3aaeb60 2024-04-21T08:47:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.207+0000 7fe51d141640 1 --2- 172.21.15.43:0/2890594471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3ab80e0 0x55ecb3ab7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.207+0000 7fe51c940640 1 --2- 172.21.15.43:0/2890594471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaeb60 0x55ecb3ac3cb0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fe50801fe00 tx=0x7fe508000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4615526e2b596c48 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.207+0000 7fe4fe7fc640 1 -- 172.21.15.43:0/2890594471 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe508028470 con 0x55ecb3aaeb60 2024-04-21T08:47:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.207+0000 7fe4fe7fc640 1 -- 172.21.15.43:0/2890594471 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe508023070 con 0x55ecb3aaeb60 2024-04-21T08:47:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.207+0000 7fe4fe7fc640 1 -- 172.21.15.43:0/2890594471 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50801f9e0 con 0x55ecb3aaeb60 2024-04-21T08:47:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.208+0000 7fe521793740 1 -- 172.21.15.43:0/2890594471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaeb60 msgr2=0x55ecb3ac3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.208+0000 7fe521793740 1 --2- 172.21.15.43:0/2890594471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaeb60 0x55ecb3ac3cb0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fe50801fe00 tx=0x7fe508000bc0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.208+0000 7fe521793740 1 -- 172.21.15.43:0/2890594471 shutdown_connections 2024-04-21T08:47:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.208+0000 7fe521793740 1 --2- 172.21.15.43:0/2890594471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3ab80e0 0x55ecb3ab7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.208+0000 7fe521793740 1 --2- 172.21.15.43:0/2890594471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecb3aaf8d0 0x55ecb3ab7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.208+0000 7fe521793740 1 --2- 172.21.15.43:0/2890594471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaeb60 0x55ecb3ac3cb0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.208+0000 7fe521793740 1 -- 172.21.15.43:0/2890594471 >> 172.21.15.43:0/2890594471 conn(0x55ecb3a75c90 msgr2=0x55ecb3aba2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.208+0000 7fe521793740 1 -- 172.21.15.43:0/2890594471 shutdown_connections 2024-04-21T08:47:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.208+0000 7fe521793740 1 -- 172.21.15.43:0/2890594471 wait complete. 2024-04-21T08:47:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.208+0000 7fe521793740 1 -- start start 2024-04-21T08:47:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.208+0000 7fe521793740 1 Processor -- start 2024-04-21T08:47:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.209+0000 7fe521793740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaeb60 0x55ecb3ac05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.209+0000 7fe521793740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecb3aaf8d0 0x55ecb3ab9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.209+0000 7fe521793740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3ab80e0 0x55ecb3ab97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.209+0000 7fe521793740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe5181b0940 con 0x55ecb3aaeb60 2024-04-21T08:47:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.209+0000 7fe521793740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe5181b0270 con 0x55ecb3ab80e0 2024-04-21T08:47:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.209+0000 7fe521793740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe5181ad4d0 con 0x55ecb3aaf8d0 2024-04-21T08:47:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.209+0000 7fe521793740 10 client.0 _ll_register_callbacks cb 0x7fe508003940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.209+0000 7fe51c940640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaeb60 0x55ecb3ac05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.209+0000 7fe51c940640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaeb60 0x55ecb3ac05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37456/0 (socket says 172.21.15.43:37456) 2024-04-21T08:47:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.209+0000 7fe51c940640 1 -- 172.21.15.43:0/3329231129 learned_addr learned my addr 172.21.15.43:0/3329231129 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.209+0000 7fe517fff640 1 --2- 172.21.15.43:0/3329231129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecb3aaf8d0 0x55ecb3ab9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.209+0000 7fe51d141640 1 --2- 172.21.15.43:0/3329231129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3ab80e0 0x55ecb3ab97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.209+0000 7fe51c940640 1 -- 172.21.15.43:0/3329231129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecb3aaf8d0 msgr2=0x55ecb3ab9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.209+0000 7fe51c940640 1 --2- 172.21.15.43:0/3329231129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecb3aaf8d0 0x55ecb3ab9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.209+0000 7fe51c940640 1 -- 172.21.15.43:0/3329231129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3ab80e0 msgr2=0x55ecb3ab97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.209+0000 7fe51c940640 1 --2- 172.21.15.43:0/3329231129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3ab80e0 0x55ecb3ab97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.209+0000 7fe51c940640 1 -- 172.21.15.43:0/3329231129 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecb3aba040 con 0x55ecb3aaeb60 2024-04-21T08:47:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.209+0000 7fe517fff640 1 --2- 172.21.15.43:0/3329231129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecb3aaf8d0 0x55ecb3ab9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.209+0000 7fe51d141640 1 --2- 172.21.15.43:0/3329231129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3ab80e0 0x55ecb3ab97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.209+0000 7fe51c940640 1 --2- 172.21.15.43:0/3329231129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaeb60 0x55ecb3ac05e0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fe508028c60 tx=0x7fe508028c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d432514f16a98a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.209+0000 7fe4f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.210+0000 7fe4f7fff640 1 -- 172.21.15.43:0/3329231129 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50801f820 con 0x55ecb3aaeb60 2024-04-21T08:47:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.210+0000 7fe4f7fff640 1 -- 172.21.15.43:0/3329231129 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fe50801f9c0 con 0x55ecb3aaeb60 2024-04-21T08:47:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.210+0000 7fe521793740 1 -- 172.21.15.43:0/3329231129 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe508023070 con 0x55ecb3aaeb60 2024-04-21T08:47:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.210+0000 7fe4f7fff640 1 -- 172.21.15.43:0/3329231129 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe508035480 con 0x55ecb3aaeb60 2024-04-21T08:47:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.210+0000 7fe521793740 10 client.19518 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.210+0000 7fe521793740 1 -- 172.21.15.43:0/3329231129 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ecb3b2e760 con 0x55ecb3aaeb60 2024-04-21T08:47:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.210+0000 7fe4f7fff640 1 -- 172.21.15.43:0/3329231129 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe508023070 con 0x55ecb3aaeb60 2024-04-21T08:47:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.210+0000 7fe4f7fff640 1 -- 172.21.15.43:0/3329231129 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe508046470 con 0x55ecb3aaeb60 2024-04-21T08:47:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.210+0000 7fe521793740 10 client.19518 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.210+0000 7fe521793740 10 client.19518 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.210+0000 7fe521793740 20 client.19518 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.210+0000 7fe521793740 1 --2- 172.21.15.43:0/3329231129 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ecb3b2ed70 0x55ecb3b31160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.210+0000 7fe521793740 4 client.19518 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:47:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.210+0000 7fe521793740 1 -- 172.21.15.43:0/3329231129 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ecb39b33d0 con 0x55ecb3b2ed70 2024-04-21T08:47:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.210+0000 7fe517fff640 1 --2- 172.21.15.43:0/3329231129 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ecb3b2ed70 0x55ecb3b31160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.211+0000 7fe517fff640 1 --2- 172.21.15.43:0/3329231129 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ecb3b2ed70 0x55ecb3b31160 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42f2fe74327434cd server_cookie=6e6a9d2b79406a94 in_seq=0 out_seq=0 2024-04-21T08:47:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.211+0000 7fe4f7fff640 10 client.19518 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.227+0000 7fe4f7fff640 1 -- 172.21.15.43:0/3329231129 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ecb39b33d0 con 0x55ecb3b2ed70 2024-04-21T08:47:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.227+0000 7fe4f7fff640 10 client.19518 handle_command_reply: tid=0 2024-04-21T08:47:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.227+0000 7fe521793740 1 client.19518 shutdown 2024-04-21T08:47:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.227+0000 7fe521793740 1 -- 172.21.15.43:0/3329231129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaeb60 msgr2=0x55ecb3ac05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.227+0000 7fe521793740 1 --2- 172.21.15.43:0/3329231129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaeb60 0x55ecb3ac05e0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fe508028c60 tx=0x7fe508028c90 comp rx=0 tx=0).stop 2024-04-21T08:47:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.228+0000 7fe521793740 1 -- 172.21.15.43:0/3329231129 shutdown_connections 2024-04-21T08:47:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.228+0000 7fe521793740 1 --2- 172.21.15.43:0/3329231129 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ecb3b2ed70 0x55ecb3b31160 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.228+0000 7fe521793740 1 --2- 172.21.15.43:0/3329231129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb3ab80e0 0x55ecb3ab97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.228+0000 7fe521793740 1 --2- 172.21.15.43:0/3329231129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ecb3aaf8d0 0x55ecb3ab9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.228+0000 7fe4f7fff640 0 client.19518 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.228+0000 7fe521793740 1 --2- 172.21.15.43:0/3329231129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ecb3aaeb60 0x55ecb3ac05e0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.228+0000 7fe521793740 1 -- 172.21.15.43:0/3329231129 >> 172.21.15.43:0/3329231129 conn(0x55ecb3a75c90 msgr2=0x55ecb3aa8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.228+0000 7fe521793740 1 -- 172.21.15.43:0/3329231129 shutdown_connections 2024-04-21T08:47:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.228+0000 7fe521793740 1 -- 172.21.15.43:0/3329231129 wait complete. 2024-04-21T08:47:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.228+0000 7fe521793740 20 client.19518 trim_cache size 0 max 16384 2024-04-21T08:47:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:47:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:47:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:47:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:47:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.228+0000 7fe51eba4640 1 -- 172.21.15.43:0/4134779577 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe5000a3dc0 msgr2=0x7fe5000a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.228+0000 7fe51eba4640 1 --2- 172.21.15.43:0/4134779577 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe5000a3dc0 0x7fe5000a6280 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe508005fd0 tx=0x7fe50800ba60 comp rx=0 tx=0).stop 2024-04-21T08:47:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.228+0000 7fe51eba4640 1 -- 172.21.15.43:0/4134779577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe51810c520 msgr2=0x7fe51819cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.228+0000 7fe51eba4640 1 --2- 172.21.15.43:0/4134779577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe51810c520 0x7fe51819cf50 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fe51000cce0 tx=0x7fe51000a620 comp rx=0 tx=0).stop 2024-04-21T08:47:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.228+0000 7fe51eba4640 1 -- 172.21.15.43:0/4134779577 shutdown_connections 2024-04-21T08:47:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.229+0000 7fe51eba4640 1 --2- 172.21.15.43:0/4134779577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe51810c520 0x7fe51819cf50 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.229+0000 7fe51eba4640 1 --2- 172.21.15.43:0/4134779577 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe5000a3dc0 0x7fe5000a6280 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.229+0000 7fe51eba4640 1 --2- 172.21.15.43:0/4134779577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe518103b00 0x7fe518198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.229+0000 7fe51eba4640 1 --2- 172.21.15.43:0/4134779577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5181031e0 0x7fe518198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.229+0000 7fe51eba4640 1 -- 172.21.15.43:0/4134779577 >> 172.21.15.43:0/4134779577 conn(0x7fe518077fd0 msgr2=0x7fe5180fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.229+0000 7fe51eba4640 1 -- 172.21.15.43:0/4134779577 shutdown_connections 2024-04-21T08:47:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:36.229+0000 7fe51eba4640 1 -- 172.21.15.43:0/4134779577 wait complete. 2024-04-21T08:47:36.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:36 smithi175 ceph-mon[31920]: pgmap v268: 129 pgs: 129 active+clean; 5.0 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:47:36.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:36 smithi112 ceph-mon[31085]: pgmap v268: 129 pgs: 129 active+clean; 5.0 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:47:36.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:36 smithi043 ceph-mon[27978]: pgmap v268: 129 pgs: 129 active+clean; 5.0 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:47:38.014 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.014 INFO:tasks.workunit.client.0.smithi043.stdout:313/600: open fd = 3 2024-04-21T08:47:38.014 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.014 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.014 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.014 INFO:tasks.workunit.client.0.smithi043.stdout:314/600: open fd = 3 2024-04-21T08:47:38.014 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.014 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.014 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.014 INFO:tasks.workunit.client.0.smithi043.stdout:315/600: open fd = 3 2024-04-21T08:47:38.014 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.014 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.015 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.015 INFO:tasks.workunit.client.0.smithi043.stdout:316/600: open fd = 3 2024-04-21T08:47:38.015 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.015 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.015 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.015 INFO:tasks.workunit.client.0.smithi043.stdout:317/600: open fd = 3 2024-04-21T08:47:38.015 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.015 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.015 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.015 INFO:tasks.workunit.client.0.smithi043.stdout:318/600: open fd = 3 2024-04-21T08:47:38.015 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.015 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.016 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.016 INFO:tasks.workunit.client.0.smithi043.stdout:319/600: open fd = 3 2024-04-21T08:47:38.016 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.016 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.016 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.016 INFO:tasks.workunit.client.0.smithi043.stdout:320/600: open fd = 3 2024-04-21T08:47:38.016 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.016 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.016 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.016 INFO:tasks.workunit.client.0.smithi043.stdout:321/600: open fd = 3 2024-04-21T08:47:38.016 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.016 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.016 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.016 INFO:tasks.workunit.client.0.smithi043.stdout:322/600: open fd = 3 2024-04-21T08:47:38.017 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.017 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.017 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.017 INFO:tasks.workunit.client.0.smithi043.stdout:323/600: open fd = 3 2024-04-21T08:47:38.017 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.017 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.017 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.017 INFO:tasks.workunit.client.0.smithi043.stdout:324/600: open fd = 3 2024-04-21T08:47:38.017 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.017 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.017 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.017 INFO:tasks.workunit.client.0.smithi043.stdout:325/600: open fd = 3 2024-04-21T08:47:38.017 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.017 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.017 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.018 INFO:tasks.workunit.client.0.smithi043.stdout:326/600: open fd = 3 2024-04-21T08:47:38.018 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.018 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.018 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.018 INFO:tasks.workunit.client.0.smithi043.stdout:327/600: open fd = 3 2024-04-21T08:47:38.018 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.018 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.018 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.018 INFO:tasks.workunit.client.0.smithi043.stdout:328/600: open fd = 3 2024-04-21T08:47:38.018 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.018 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.018 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.018 INFO:tasks.workunit.client.0.smithi043.stdout:329/600: open fd = 3 2024-04-21T08:47:38.018 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.019 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.019 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.019 INFO:tasks.workunit.client.0.smithi043.stdout:330/600: open fd = 3 2024-04-21T08:47:38.019 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.019 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.019 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.019 INFO:tasks.workunit.client.0.smithi043.stdout:331/600: open fd = 3 2024-04-21T08:47:38.019 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.019 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.019 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.019 INFO:tasks.workunit.client.0.smithi043.stdout:332/600: open fd = 3 2024-04-21T08:47:38.019 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.019 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.020 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.020 INFO:tasks.workunit.client.0.smithi043.stdout:333/600: open fd = 3 2024-04-21T08:47:38.020 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.020 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.020 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.020 INFO:tasks.workunit.client.0.smithi043.stdout:334/600: open fd = 3 2024-04-21T08:47:38.020 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.020 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.020 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.020 INFO:tasks.workunit.client.0.smithi043.stdout:335/600: open fd = 3 2024-04-21T08:47:38.020 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.020 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.020 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.020 INFO:tasks.workunit.client.0.smithi043.stdout:336/600: open fd = 3 2024-04-21T08:47:38.021 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.021 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.021 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.021 INFO:tasks.workunit.client.0.smithi043.stdout:337/600: open fd = 3 2024-04-21T08:47:38.021 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.021 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.021 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.021 INFO:tasks.workunit.client.0.smithi043.stdout:338/600: open fd = 3 2024-04-21T08:47:38.021 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.021 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.021 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.021 INFO:tasks.workunit.client.0.smithi043.stdout:339/600: open fd = 3 2024-04-21T08:47:38.021 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.021 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.021 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.022 INFO:tasks.workunit.client.0.smithi043.stdout:340/600: open fd = 3 2024-04-21T08:47:38.022 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.022 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.022 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.022 INFO:tasks.workunit.client.0.smithi043.stdout:341/600: open fd = 3 2024-04-21T08:47:38.022 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.022 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.022 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.022 INFO:tasks.workunit.client.0.smithi043.stdout:342/600: open fd = 3 2024-04-21T08:47:38.022 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.022 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.022 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.022 INFO:tasks.workunit.client.0.smithi043.stdout:343/600: open fd = 3 2024-04-21T08:47:38.022 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.023 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.023 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.023 INFO:tasks.workunit.client.0.smithi043.stdout:344/600: open fd = 3 2024-04-21T08:47:38.023 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.023 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.023 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.023 INFO:tasks.workunit.client.0.smithi043.stdout:345/600: open fd = 3 2024-04-21T08:47:38.023 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.023 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.023 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.023 INFO:tasks.workunit.client.0.smithi043.stdout:346/600: open fd = 3 2024-04-21T08:47:38.023 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.023 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.023 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.024 INFO:tasks.workunit.client.0.smithi043.stdout:347/600: open fd = 3 2024-04-21T08:47:38.024 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.024 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.024 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.024 INFO:tasks.workunit.client.0.smithi043.stdout:348/600: open fd = 3 2024-04-21T08:47:38.024 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.024 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.024 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.024 INFO:tasks.workunit.client.0.smithi043.stdout:349/600: open fd = 3 2024-04-21T08:47:38.024 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.024 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.024 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.024 INFO:tasks.workunit.client.0.smithi043.stdout:350/600: open fd = 3 2024-04-21T08:47:38.024 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.025 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.025 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.025 INFO:tasks.workunit.client.0.smithi043.stdout:351/600: open fd = 3 2024-04-21T08:47:38.025 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.025 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.025 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.025 INFO:tasks.workunit.client.0.smithi043.stdout:352/600: open fd = 3 2024-04-21T08:47:38.025 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.025 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.025 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.025 INFO:tasks.workunit.client.0.smithi043.stdout:353/600: open fd = 3 2024-04-21T08:47:38.025 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.025 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.026 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.026 INFO:tasks.workunit.client.0.smithi043.stdout:354/600: open fd = 3 2024-04-21T08:47:38.026 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.026 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.026 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.026 INFO:tasks.workunit.client.0.smithi043.stdout:355/600: open fd = 3 2024-04-21T08:47:38.026 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.026 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.026 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.026 INFO:tasks.workunit.client.0.smithi043.stdout:356/600: open fd = 3 2024-04-21T08:47:38.026 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.027 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.027 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.027 INFO:tasks.workunit.client.0.smithi043.stdout:357/600: open fd = 3 2024-04-21T08:47:38.027 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.027 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.027 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.027 INFO:tasks.workunit.client.0.smithi043.stdout:358/600: open fd = 3 2024-04-21T08:47:38.027 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.027 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.027 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.027 INFO:tasks.workunit.client.0.smithi043.stdout:359/600: open fd = 3 2024-04-21T08:47:38.027 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.027 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.028 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.028 INFO:tasks.workunit.client.0.smithi043.stdout:360/600: open fd = 3 2024-04-21T08:47:38.028 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.028 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.028 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.028 INFO:tasks.workunit.client.0.smithi043.stdout:361/600: open fd = 3 2024-04-21T08:47:38.028 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.028 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.028 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.028 INFO:tasks.workunit.client.0.smithi043.stdout:362/600: open fd = 3 2024-04-21T08:47:38.028 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.028 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.029 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.029 INFO:tasks.workunit.client.0.smithi043.stdout:363/600: open fd = 3 2024-04-21T08:47:38.029 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.029 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.029 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.029 INFO:tasks.workunit.client.0.smithi043.stdout:364/600: open fd = 3 2024-04-21T08:47:38.029 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.029 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.029 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.029 INFO:tasks.workunit.client.0.smithi043.stdout:365/600: open fd = 3 2024-04-21T08:47:38.029 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.029 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.029 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.029 INFO:tasks.workunit.client.0.smithi043.stdout:366/600: open fd = 3 2024-04-21T08:47:38.030 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.030 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.030 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.030 INFO:tasks.workunit.client.0.smithi043.stdout:367/600: open fd = 3 2024-04-21T08:47:38.030 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.030 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.030 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.030 INFO:tasks.workunit.client.0.smithi043.stdout:368/600: open fd = 3 2024-04-21T08:47:38.030 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.030 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.030 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.030 INFO:tasks.workunit.client.0.smithi043.stdout:369/600: open fd = 3 2024-04-21T08:47:38.030 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.031 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.031 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.031 INFO:tasks.workunit.client.0.smithi043.stdout:370/600: open fd = 3 2024-04-21T08:47:38.031 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.031 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.031 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.031 INFO:tasks.workunit.client.0.smithi043.stdout:371/600: open fd = 3 2024-04-21T08:47:38.031 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.031 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.031 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.031 INFO:tasks.workunit.client.0.smithi043.stdout:372/600: open fd = 3 2024-04-21T08:47:38.031 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.031 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.031 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.032 INFO:tasks.workunit.client.0.smithi043.stdout:373/600: open fd = 3 2024-04-21T08:47:38.032 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.032 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.032 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.032 INFO:tasks.workunit.client.0.smithi043.stdout:374/600: open fd = 3 2024-04-21T08:47:38.032 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.032 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:47:38.032 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:47:38.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:38 smithi175 ceph-mon[31920]: pgmap v269: 129 pgs: 129 active+clean; 5.2 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-21T08:47:38.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:38 smithi112 ceph-mon[31085]: pgmap v269: 129 pgs: 129 active+clean; 5.2 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-21T08:47:38.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:38 smithi043 ceph-mon[27978]: pgmap v269: 129 pgs: 129 active+clean; 5.2 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-21T08:47:40.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:40 smithi175 ceph-mon[31920]: pgmap v270: 129 pgs: 129 active+clean; 5.2 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-21T08:47:40.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:40 smithi112 ceph-mon[31085]: pgmap v270: 129 pgs: 129 active+clean; 5.2 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-21T08:47:40.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:40 smithi043 ceph-mon[27978]: pgmap v270: 129 pgs: 129 active+clean; 5.2 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-21T08:47:42.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:42 smithi175 ceph-mon[31920]: pgmap v271: 129 pgs: 129 active+clean; 5.4 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-21T08:47:42.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:42 smithi112 ceph-mon[31085]: pgmap v271: 129 pgs: 129 active+clean; 5.4 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-21T08:47:42.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:42 smithi043 ceph-mon[27978]: pgmap v271: 129 pgs: 129 active+clean; 5.4 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-21T08:47:44.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:44 smithi175 ceph-mon[31920]: pgmap v272: 129 pgs: 129 active+clean; 5.4 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-21T08:47:44.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:44 smithi112 ceph-mon[31085]: pgmap v272: 129 pgs: 129 active+clean; 5.4 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-21T08:47:44.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:44 smithi043 ceph-mon[27978]: pgmap v272: 129 pgs: 129 active+clean; 5.4 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-21T08:47:46.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:46 smithi112 ceph-mon[31085]: pgmap v273: 129 pgs: 129 active+clean; 5.4 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-21T08:47:46.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:46 smithi043 ceph-mon[27978]: pgmap v273: 129 pgs: 129 active+clean; 5.4 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-21T08:47:46.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:46 smithi175 ceph-mon[31920]: pgmap v273: 129 pgs: 129 active+clean; 5.4 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-21T08:47:46.759 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:47:46.760 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:47:46.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.877+0000 7f241c19f640 1 -- 172.21.15.43:0/499997834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2414101080 msgr2=0x7f2414101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:46.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.877+0000 7f241c19f640 1 --2- 172.21.15.43:0/499997834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2414101080 0x7f2414101460 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f2410009ca0 tx=0x7f241002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.877+0000 7f241c19f640 1 -- 172.21.15.43:0/499997834 shutdown_connections 2024-04-21T08:47:46.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.877+0000 7f241c19f640 1 --2- 172.21.15.43:0/499997834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2414106150 0x7f241410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.877+0000 7f241c19f640 1 --2- 172.21.15.43:0/499997834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2414101a30 0x7f2414105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.877+0000 7f241c19f640 1 --2- 172.21.15.43:0/499997834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2414101080 0x7f2414101460 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.877+0000 7f241c19f640 1 -- 172.21.15.43:0/499997834 >> 172.21.15.43:0/499997834 conn(0x7f2414078000 msgr2=0x7f2414078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:46.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.877+0000 7f241c19f640 1 -- 172.21.15.43:0/499997834 shutdown_connections 2024-04-21T08:47:46.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.877+0000 7f241c19f640 1 -- 172.21.15.43:0/499997834 wait complete. 2024-04-21T08:47:46.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.877+0000 7f241c19f640 1 Processor -- start 2024-04-21T08:47:46.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.877+0000 7f241c19f640 1 -- start start 2024-04-21T08:47:46.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.878+0000 7f241c19f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2414101080 0x7f24141994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:46.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.878+0000 7f241c19f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2414106150 0x7f2414199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:46.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.878+0000 7f241c19f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2414199fa0 0x7f24141a4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:46.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.878+0000 7f241c19f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2414111090 con 0x7f2414101080 2024-04-21T08:47:46.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.878+0000 7f241c19f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2414110f10 con 0x7f2414199fa0 2024-04-21T08:47:46.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.878+0000 7f241c19f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2414111210 con 0x7f2414106150 2024-04-21T08:47:46.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.878+0000 7f2419f3b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2414101080 0x7f24141994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:46.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.878+0000 7f2419f3b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2414101080 0x7f24141994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57308/0 (socket says 172.21.15.43:57308) 2024-04-21T08:47:46.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.878+0000 7f2419f3b640 1 -- 172.21.15.43:0/3878944502 learned_addr learned my addr 172.21.15.43:0/3878944502 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:46.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.878+0000 7f241973a640 1 --2- 172.21.15.43:0/3878944502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2414106150 0x7f2414199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:46.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.878+0000 7f241a73c640 1 --2- 172.21.15.43:0/3878944502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2414199fa0 0x7f24141a4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:46.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.878+0000 7f2419f3b640 1 -- 172.21.15.43:0/3878944502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2414106150 msgr2=0x7f2414199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:46.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.878+0000 7f2419f3b640 1 --2- 172.21.15.43:0/3878944502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2414106150 0x7f2414199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.878+0000 7f2419f3b640 1 -- 172.21.15.43:0/3878944502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2414199fa0 msgr2=0x7f24141a4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:46.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.878+0000 7f2419f3b640 1 --2- 172.21.15.43:0/3878944502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2414199fa0 0x7f24141a4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.878+0000 7f2419f3b640 1 -- 172.21.15.43:0/3878944502 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24141a4d50 con 0x7f2414101080 2024-04-21T08:47:46.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.878+0000 7f241a73c640 1 --2- 172.21.15.43:0/3878944502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2414199fa0 0x7f24141a4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:46.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.878+0000 7f2419f3b640 1 --2- 172.21.15.43:0/3878944502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2414101080 0x7f24141994d0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f2410005e30 tx=0x7f2410034780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:46.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.879+0000 7f240affd640 1 -- 172.21.15.43:0/3878944502 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410039040 con 0x7f2414101080 2024-04-21T08:47:46.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.879+0000 7f241c19f640 1 -- 172.21.15.43:0/3878944502 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24141a4fe0 con 0x7f2414101080 2024-04-21T08:47:46.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.879+0000 7f240affd640 1 -- 172.21.15.43:0/3878944502 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2410044070 con 0x7f2414101080 2024-04-21T08:47:46.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.879+0000 7f240affd640 1 -- 172.21.15.43:0/3878944502 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410004040 con 0x7f2414101080 2024-04-21T08:47:46.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.879+0000 7f241c19f640 1 -- 172.21.15.43:0/3878944502 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24141a5450 con 0x7f2414101080 2024-04-21T08:47:46.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.881+0000 7f240affd640 1 -- 172.21.15.43:0/3878944502 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f24100324c0 con 0x7f2414101080 2024-04-21T08:47:46.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.881+0000 7f241ed8b740 1 Processor -- start 2024-04-21T08:47:46.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.881+0000 7f241ed8b740 1 -- start start 2024-04-21T08:47:46.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.882+0000 7f241ed8b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c18580 0x559091c18960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:46.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.882+0000 7f241ed8b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559091c18ea0 0x559091c21540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.882+0000 7f241ed8b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559091c21a80 0x559091c23e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.882+0000 7f241ed8b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559091b7d1e0 con 0x559091c18580 2024-04-21T08:47:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.882+0000 7f241ed8b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559091b1a390 con 0x559091c21a80 2024-04-21T08:47:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.882+0000 7f241ed8b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559091b791a0 con 0x559091c18ea0 2024-04-21T08:47:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.882+0000 7f240affd640 1 --2- 172.21.15.43:0/3878944502 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f24000a3d20 0x7f24000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.882+0000 7f241a73c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c18580 0x559091c18960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.882+0000 7f241a73c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c18580 0x559091c18960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57320/0 (socket says 172.21.15.43:57320) 2024-04-21T08:47:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.882+0000 7f241a73c640 1 -- 172.21.15.43:0/1738616211 learned_addr learned my addr 172.21.15.43:0/1738616211 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.882+0000 7f2419f3b640 1 --2- 172.21.15.43:0/1738616211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559091c18ea0 0x559091c21540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.882+0000 7f241a73c640 1 --2- 172.21.15.43:0/3878944502 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f24000a3d20 0x7f24000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.882+0000 7f241973a640 1 --2- 172.21.15.43:0/1738616211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559091c21a80 0x559091c23e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.882+0000 7f240affd640 1 -- 172.21.15.43:0/3878944502 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f24100f2e50 con 0x7f2414101080 2024-04-21T08:47:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.882+0000 7f241a73c640 1 -- 172.21.15.43:0/1738616211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559091c18ea0 msgr2=0x559091c21540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:46.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.882+0000 7f241a73c640 1 --2- 172.21.15.43:0/1738616211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559091c18ea0 0x559091c21540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.882+0000 7f241a73c640 1 -- 172.21.15.43:0/1738616211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559091c21a80 msgr2=0x559091c23e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:46.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.882+0000 7f241a73c640 1 --2- 172.21.15.43:0/1738616211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559091c21a80 0x559091c23e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.882+0000 7f241a73c640 1 -- 172.21.15.43:0/1738616211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590919597d0 con 0x559091c18580 2024-04-21T08:47:46.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.882+0000 7f2419f3b640 1 --2- 172.21.15.43:0/1738616211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559091c18ea0 0x559091c21540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:46.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.883+0000 7f241a73c640 1 --2- 172.21.15.43:0/1738616211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c18580 0x559091c18960 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f240c014c50 tx=0x7f240c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea9f2390c136fc11 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:46.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.883+0000 7f241a73c640 1 --2- 172.21.15.43:0/3878944502 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f24000a3d20 0x7f24000a61e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f240c011cb0 tx=0x7f240c016000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:46.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.883+0000 7f23fffff640 1 -- 172.21.15.43:0/1738616211 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c022070 con 0x559091c18580 2024-04-21T08:47:46.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.883+0000 7f23fffff640 1 -- 172.21.15.43:0/1738616211 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f240c014620 con 0x559091c18580 2024-04-21T08:47:46.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.883+0000 7f23fffff640 1 -- 172.21.15.43:0/1738616211 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c0039d0 con 0x559091c18580 2024-04-21T08:47:46.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.883+0000 7f241ed8b740 1 -- 172.21.15.43:0/1738616211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c18580 msgr2=0x559091c18960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:46.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.884+0000 7f241ed8b740 1 --2- 172.21.15.43:0/1738616211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c18580 0x559091c18960 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f240c014c50 tx=0x7f240c0117b0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.884+0000 7f241ed8b740 1 -- 172.21.15.43:0/1738616211 shutdown_connections 2024-04-21T08:47:46.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.884+0000 7f241ed8b740 1 --2- 172.21.15.43:0/1738616211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559091c21a80 0x559091c23e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.884+0000 7f241ed8b740 1 --2- 172.21.15.43:0/1738616211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559091c18ea0 0x559091c21540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.884+0000 7f241ed8b740 1 --2- 172.21.15.43:0/1738616211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c18580 0x559091c18960 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.884+0000 7f241ed8b740 1 -- 172.21.15.43:0/1738616211 >> 172.21.15.43:0/1738616211 conn(0x559091c12cb0 msgr2=0x559091c13090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:46.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.884+0000 7f241ed8b740 1 -- 172.21.15.43:0/1738616211 shutdown_connections 2024-04-21T08:47:46.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.884+0000 7f241ed8b740 1 -- 172.21.15.43:0/1738616211 wait complete. 2024-04-21T08:47:46.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.884+0000 7f241ed8b740 1 -- start start 2024-04-21T08:47:46.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.884+0000 7f241ed8b740 1 Processor -- start 2024-04-21T08:47:46.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.884+0000 7f241ed8b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559091c18580 0x559091c1eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:46.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.884+0000 7f241ed8b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c18ea0 0x559091c1f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:46.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.885+0000 7f241ed8b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559091c21a80 0x559091c1f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:46.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.885+0000 7f241ed8b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559091c24810 con 0x559091c18ea0 2024-04-21T08:47:46.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.885+0000 7f241ed8b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559091b362f0 con 0x559091c18580 2024-04-21T08:47:46.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.885+0000 7f241ed8b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559091c24990 con 0x559091c21a80 2024-04-21T08:47:46.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.885+0000 7f241ed8b740 10 client.0 _ll_register_callbacks cb 0x559091b43580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:46.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.885+0000 7f241973a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c18ea0 0x559091c1f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:46.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.885+0000 7f241973a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c18ea0 0x559091c1f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57334/0 (socket says 172.21.15.43:57334) 2024-04-21T08:47:46.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.885+0000 7f241973a640 1 -- 172.21.15.43:0/3993413268 learned_addr learned my addr 172.21.15.43:0/3993413268 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:46.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.885+0000 7f2419f3b640 1 --2- 172.21.15.43:0/3993413268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559091c18580 0x559091c1eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:46.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.885+0000 7f241a73c640 1 --2- 172.21.15.43:0/3993413268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559091c21a80 0x559091c1f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:46.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.885+0000 7f241973a640 1 -- 172.21.15.43:0/3993413268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559091c21a80 msgr2=0x559091c1f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:46.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.885+0000 7f241973a640 1 --2- 172.21.15.43:0/3993413268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559091c21a80 0x559091c1f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.885+0000 7f241973a640 1 -- 172.21.15.43:0/3993413268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559091c18580 msgr2=0x559091c1eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:46.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.885+0000 7f241973a640 1 --2- 172.21.15.43:0/3993413268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559091c18580 0x559091c1eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.885+0000 7f241973a640 1 -- 172.21.15.43:0/3993413268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55909188f3f0 con 0x559091c18ea0 2024-04-21T08:47:46.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.885+0000 7f241a73c640 1 --2- 172.21.15.43:0/3993413268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559091c21a80 0x559091c1f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:46.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.885+0000 7f2419f3b640 1 --2- 172.21.15.43:0/3993413268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559091c18580 0x559091c1eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:46.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.885+0000 7f241973a640 1 --2- 172.21.15.43:0/3993413268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c18ea0 0x559091c1f060 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f2404010820 tx=0x7f2404010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e324a79aea603d49 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:46.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.885+0000 7f23fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:46.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.885+0000 7f23fd7fa640 1 -- 172.21.15.43:0/3993413268 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240401e060 con 0x559091c18ea0 2024-04-21T08:47:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.885+0000 7f23fd7fa640 1 -- 172.21.15.43:0/3993413268 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f240400fb90 con 0x559091c18ea0 2024-04-21T08:47:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.885+0000 7f241ed8b740 1 -- 172.21.15.43:0/3993413268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f240c014620 con 0x559091c18ea0 2024-04-21T08:47:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.886+0000 7f23fd7fa640 1 -- 172.21.15.43:0/3993413268 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2404025050 con 0x559091c18ea0 2024-04-21T08:47:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.886+0000 7f241ed8b740 10 client.19530 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.886+0000 7f241ed8b740 1 -- 172.21.15.43:0/3993413268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55909184f0f0 con 0x559091c18ea0 2024-04-21T08:47:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.886+0000 7f23fd7fa640 1 -- 172.21.15.43:0/3993413268 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f240c014620 con 0x559091c18ea0 2024-04-21T08:47:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.886+0000 7f23fd7fa640 1 -- 172.21.15.43:0/3993413268 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2404018370 con 0x559091c18ea0 2024-04-21T08:47:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.886+0000 7f241ed8b740 10 client.19530 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.886+0000 7f241ed8b740 10 client.19530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.886+0000 7f241ed8b740 20 client.19530 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.886+0000 7f241ed8b740 1 --2- 172.21.15.43:0/3993413268 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559091c31ad0 0x559091c33f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.886+0000 7f241ed8b740 4 client.19530 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.886+0000 7f241ed8b740 1 -- 172.21.15.43:0/3993413268 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559091b8ac00 con 0x559091c31ad0 2024-04-21T08:47:46.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.886+0000 7f2419f3b640 1 --2- 172.21.15.43:0/3993413268 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559091c31ad0 0x559091c33f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.887+0000 7f23fd7fa640 10 client.19530 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.887+0000 7f2419f3b640 1 --2- 172.21.15.43:0/3993413268 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559091c31ad0 0x559091c33f40 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90e6ff081749307d server_cookie=e94a8b6114e2ae79 in_seq=0 out_seq=0 2024-04-21T08:47:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.887+0000 7f23fd7fa640 1 -- 172.21.15.43:0/3993413268 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559091b8ac00 con 0x559091c31ad0 2024-04-21T08:47:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.887+0000 7f23fd7fa640 10 client.19530 handle_command_reply: tid=0 2024-04-21T08:47:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.887+0000 7f241ed8b740 1 client.19530 shutdown 2024-04-21T08:47:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.888+0000 7f241ed8b740 1 -- 172.21.15.43:0/3993413268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c18ea0 msgr2=0x559091c1f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.888+0000 7f241ed8b740 1 --2- 172.21.15.43:0/3993413268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c18ea0 0x559091c1f060 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f2404010820 tx=0x7f2404010ce0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.888+0000 7f241ed8b740 1 -- 172.21.15.43:0/3993413268 shutdown_connections 2024-04-21T08:47:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.888+0000 7f241ed8b740 1 --2- 172.21.15.43:0/3993413268 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559091c31ad0 0x559091c33f40 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.888+0000 7f241ed8b740 1 --2- 172.21.15.43:0/3993413268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559091c21a80 0x559091c1f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.888+0000 7f23fd7fa640 0 client.19530 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.888+0000 7f241ed8b740 1 --2- 172.21.15.43:0/3993413268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c18ea0 0x559091c1f060 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.888+0000 7f241ed8b740 1 --2- 172.21.15.43:0/3993413268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559091c18580 0x559091c1eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.888+0000 7f241ed8b740 1 -- 172.21.15.43:0/3993413268 >> 172.21.15.43:0/3993413268 conn(0x559091c12cb0 msgr2=0x559091c14a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:46.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.888+0000 7f241ed8b740 1 -- 172.21.15.43:0/3993413268 shutdown_connections 2024-04-21T08:47:46.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.888+0000 7f241ed8b740 1 -- 172.21.15.43:0/3993413268 wait complete. 2024-04-21T08:47:46.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.888+0000 7f241ed8b740 20 client.19530 trim_cache size 0 max 16384 2024-04-21T08:47:46.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.906+0000 7f241ed8b740 1 Processor -- start 2024-04-21T08:47:46.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.906+0000 7f241ed8b740 1 -- start start 2024-04-21T08:47:46.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.906+0000 7f241ed8b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559091c18580 0x559091c21250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:46.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.906+0000 7f241ed8b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559091c18ea0 0x559091c1da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:46.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.906+0000 7f241ed8b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c21a80 0x559091c2d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:46.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.906+0000 7f241ed8b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24141af480 con 0x559091c21a80 2024-04-21T08:47:46.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.906+0000 7f241ed8b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24141b0fc0 con 0x559091c18ea0 2024-04-21T08:47:46.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.906+0000 7f241ed8b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f24141afb50 con 0x559091c18580 2024-04-21T08:47:46.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.906+0000 7f241a73c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c21a80 0x559091c2d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:46.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.906+0000 7f241a73c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c21a80 0x559091c2d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57338/0 (socket says 172.21.15.43:57338) 2024-04-21T08:47:46.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.906+0000 7f241a73c640 1 -- 172.21.15.43:0/131868491 learned_addr learned my addr 172.21.15.43:0/131868491 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:46.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.906+0000 7f2419f3b640 1 --2- 172.21.15.43:0/131868491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559091c18580 0x559091c21250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:46.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.906+0000 7f241973a640 1 --2- 172.21.15.43:0/131868491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559091c18ea0 0x559091c1da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:46.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.906+0000 7f241a73c640 1 -- 172.21.15.43:0/131868491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559091c18580 msgr2=0x559091c21250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:46.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.906+0000 7f241a73c640 1 --2- 172.21.15.43:0/131868491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559091c18580 0x559091c21250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.906+0000 7f241a73c640 1 -- 172.21.15.43:0/131868491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559091c18ea0 msgr2=0x559091c1da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:46.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.906+0000 7f241a73c640 1 --2- 172.21.15.43:0/131868491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559091c18ea0 0x559091c1da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.906+0000 7f241a73c640 1 -- 172.21.15.43:0/131868491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559091c18cd0 con 0x559091c21a80 2024-04-21T08:47:46.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.906+0000 7f2419f3b640 1 --2- 172.21.15.43:0/131868491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559091c18580 0x559091c21250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:46.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.906+0000 7f241a73c640 1 --2- 172.21.15.43:0/131868491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c21a80 0x559091c2d6d0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f240c006010 tx=0x7f240c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184e97473e21abd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:46.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.907+0000 7f23ff7fe640 1 -- 172.21.15.43:0/131868491 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c022040 con 0x559091c21a80 2024-04-21T08:47:46.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.907+0000 7f23ff7fe640 1 -- 172.21.15.43:0/131868491 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f240c0035f0 con 0x559091c21a80 2024-04-21T08:47:46.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.907+0000 7f23ff7fe640 1 -- 172.21.15.43:0/131868491 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c0053a0 con 0x559091c21a80 2024-04-21T08:47:46.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.907+0000 7f241ed8b740 1 -- 172.21.15.43:0/131868491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c21a80 msgr2=0x559091c2d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:46.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.907+0000 7f241ed8b740 1 --2- 172.21.15.43:0/131868491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c21a80 0x559091c2d6d0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f240c006010 tx=0x7f240c00cbd0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.907+0000 7f241ed8b740 1 -- 172.21.15.43:0/131868491 shutdown_connections 2024-04-21T08:47:46.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.907+0000 7f241ed8b740 1 --2- 172.21.15.43:0/131868491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c21a80 0x559091c2d6d0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.907+0000 7f241ed8b740 1 --2- 172.21.15.43:0/131868491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559091c18ea0 0x559091c1da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.907+0000 7f241ed8b740 1 --2- 172.21.15.43:0/131868491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559091c18580 0x559091c21250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.907+0000 7f241ed8b740 1 -- 172.21.15.43:0/131868491 >> 172.21.15.43:0/131868491 conn(0x559091c12cb0 msgr2=0x559091c14950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:46.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.907+0000 7f241ed8b740 1 -- 172.21.15.43:0/131868491 shutdown_connections 2024-04-21T08:47:46.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.907+0000 7f241ed8b740 1 -- 172.21.15.43:0/131868491 wait complete. 2024-04-21T08:47:46.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.908+0000 7f241ed8b740 1 -- start start 2024-04-21T08:47:46.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.908+0000 7f241ed8b740 1 Processor -- start 2024-04-21T08:47:46.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.908+0000 7f241ed8b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559091c18580 0x559091c29be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:46.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.908+0000 7f241ed8b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559091c18ea0 0x559091c2a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:46.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.908+0000 7f241ed8b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c21a80 0x559091c26bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:46.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.908+0000 7f241ed8b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24141b08f0 con 0x559091c21a80 2024-04-21T08:47:46.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.908+0000 7f241ed8b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24141b0220 con 0x559091c18580 2024-04-21T08:47:46.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.908+0000 7f241ed8b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f24141ad480 con 0x559091c18ea0 2024-04-21T08:47:46.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.908+0000 7f241ed8b740 10 client.0 _ll_register_callbacks cb 0x7f240400a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:46.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.908+0000 7f241a73c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c21a80 0x559091c26bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:46.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.908+0000 7f241a73c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c21a80 0x559091c26bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57352/0 (socket says 172.21.15.43:57352) 2024-04-21T08:47:46.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.908+0000 7f241a73c640 1 -- 172.21.15.43:0/1876519569 learned_addr learned my addr 172.21.15.43:0/1876519569 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:46.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.909+0000 7f2419f3b640 1 --2- 172.21.15.43:0/1876519569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559091c18580 0x559091c29be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:46.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.909+0000 7f241973a640 1 --2- 172.21.15.43:0/1876519569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559091c18ea0 0x559091c2a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:46.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.909+0000 7f241a73c640 1 -- 172.21.15.43:0/1876519569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559091c18ea0 msgr2=0x559091c2a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:46.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.909+0000 7f241a73c640 1 --2- 172.21.15.43:0/1876519569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559091c18ea0 0x559091c2a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.909+0000 7f241a73c640 1 -- 172.21.15.43:0/1876519569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559091c18580 msgr2=0x559091c29be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:46.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.909+0000 7f241a73c640 1 --2- 172.21.15.43:0/1876519569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559091c18580 0x559091c29be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.909+0000 7f241a73c640 1 -- 172.21.15.43:0/1876519569 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559091c273e0 con 0x559091c21a80 2024-04-21T08:47:46.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.909+0000 7f241a73c640 1 --2- 172.21.15.43:0/1876519569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c21a80 0x559091c26bb0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f240c00cca0 tx=0x7f240c014be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27533ed7f7395e13 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:46.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.909+0000 7f23fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:46.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.909+0000 7f23fd7fa640 1 -- 172.21.15.43:0/1876519569 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c03d060 con 0x559091c21a80 2024-04-21T08:47:46.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.909+0000 7f23fd7fa640 1 -- 172.21.15.43:0/1876519569 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f240c005120 con 0x559091c21a80 2024-04-21T08:47:46.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.909+0000 7f241ed8b740 1 -- 172.21.15.43:0/1876519569 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f240c0035f0 con 0x559091c21a80 2024-04-21T08:47:46.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.909+0000 7f23fd7fa640 1 -- 172.21.15.43:0/1876519569 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f240c022050 con 0x559091c21a80 2024-04-21T08:47:46.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.909+0000 7f241ed8b740 10 client.19536 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:46.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.909+0000 7f241ed8b740 1 -- 172.21.15.43:0/1876519569 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559091c98ba0 con 0x559091c21a80 2024-04-21T08:47:46.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.910+0000 7f23fd7fa640 1 -- 172.21.15.43:0/1876519569 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f240c0035f0 con 0x559091c21a80 2024-04-21T08:47:46.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.910+0000 7f23fd7fa640 1 -- 172.21.15.43:0/1876519569 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f240c02e070 con 0x559091c21a80 2024-04-21T08:47:46.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.910+0000 7f241ed8b740 10 client.19536 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:46.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.910+0000 7f241ed8b740 10 client.19536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:46.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.910+0000 7f241ed8b740 20 client.19536 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:46.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.910+0000 7f241ed8b740 1 --2- 172.21.15.43:0/1876519569 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559091c990c0 0x559091c9b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:46.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.910+0000 7f241ed8b740 4 client.19536 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:47:46.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.910+0000 7f241ed8b740 1 -- 172.21.15.43:0/1876519569 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559091b84a70 con 0x559091c990c0 2024-04-21T08:47:46.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.910+0000 7f2419f3b640 1 --2- 172.21.15.43:0/1876519569 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559091c990c0 0x559091c9b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:46.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.910+0000 7f23fd7fa640 10 client.19536 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:46.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.910+0000 7f2419f3b640 1 --2- 172.21.15.43:0/1876519569 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559091c990c0 0x559091c9b4b0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22053bff8337bc8d server_cookie=5c653408fee30af9 in_seq=0 out_seq=0 2024-04-21T08:47:46.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f23fd7fa640 1 -- 172.21.15.43:0/1876519569 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559091b84a70 con 0x559091c990c0 2024-04-21T08:47:46.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f23fd7fa640 10 client.19536 handle_command_reply: tid=0 2024-04-21T08:47:46.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f241ed8b740 1 client.19536 shutdown 2024-04-21T08:47:46.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f241ed8b740 1 -- 172.21.15.43:0/1876519569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c21a80 msgr2=0x559091c26bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:46.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f241ed8b740 1 --2- 172.21.15.43:0/1876519569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c21a80 0x559091c26bb0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f240c00cca0 tx=0x7f240c014be0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f241ed8b740 1 -- 172.21.15.43:0/1876519569 shutdown_connections 2024-04-21T08:47:46.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f241ed8b740 1 --2- 172.21.15.43:0/1876519569 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559091c990c0 0x559091c9b4b0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f241ed8b740 1 --2- 172.21.15.43:0/1876519569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559091c21a80 0x559091c26bb0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f241ed8b740 1 --2- 172.21.15.43:0/1876519569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559091c18ea0 0x559091c2a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f241ed8b740 1 --2- 172.21.15.43:0/1876519569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559091c18580 0x559091c29be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f23fd7fa640 0 client.19536 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:46.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f241ed8b740 1 -- 172.21.15.43:0/1876519569 >> 172.21.15.43:0/1876519569 conn(0x559091c12cb0 msgr2=0x559091c1eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:46.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f241ed8b740 1 -- 172.21.15.43:0/1876519569 shutdown_connections 2024-04-21T08:47:46.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f241ed8b740 1 -- 172.21.15.43:0/1876519569 wait complete. 2024-04-21T08:47:46.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f241ed8b740 20 client.19536 trim_cache size 0 max 16384 2024-04-21T08:47:46.918 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:47:46.918 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:47:46.918 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:47:46.919 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:47:46.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f241c19f640 1 -- 172.21.15.43:0/3878944502 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f24000a3d20 msgr2=0x7f24000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:46.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f241c19f640 1 --2- 172.21.15.43:0/3878944502 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f24000a3d20 0x7f24000a61e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f240c011cb0 tx=0x7f240c016000 comp rx=0 tx=0).stop 2024-04-21T08:47:46.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f241c19f640 1 -- 172.21.15.43:0/3878944502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2414101080 msgr2=0x7f24141994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:46.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f241c19f640 1 --2- 172.21.15.43:0/3878944502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2414101080 0x7f24141994d0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f2410005e30 tx=0x7f2410034780 comp rx=0 tx=0).stop 2024-04-21T08:47:46.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f241c19f640 1 -- 172.21.15.43:0/3878944502 shutdown_connections 2024-04-21T08:47:46.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f241c19f640 1 --2- 172.21.15.43:0/3878944502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2414199fa0 0x7f24141a4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f241c19f640 1 --2- 172.21.15.43:0/3878944502 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f24000a3d20 0x7f24000a61e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f241c19f640 1 --2- 172.21.15.43:0/3878944502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2414106150 0x7f2414199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f241c19f640 1 --2- 172.21.15.43:0/3878944502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2414101080 0x7f24141994d0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:46.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.911+0000 7f241c19f640 1 -- 172.21.15.43:0/3878944502 >> 172.21.15.43:0/3878944502 conn(0x7f2414078000 msgr2=0x7f24140ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:46.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.912+0000 7f241c19f640 1 -- 172.21.15.43:0/3878944502 shutdown_connections 2024-04-21T08:47:46.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:46.912+0000 7f241c19f640 1 -- 172.21.15.43:0/3878944502 wait complete. 2024-04-21T08:47:46.923 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:47:46.923 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:47:47.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.025+0000 7fc6ff352640 1 -- 172.21.15.43:0/3864852425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f8105e30 msgr2=0x7fc6f8106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.025+0000 7fc6ff352640 1 --2- 172.21.15.43:0/3864852425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f8105e30 0x7fc6f8106210 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009930 tx=0x7fc6f402bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:47.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.025+0000 7fc6ff352640 1 -- 172.21.15.43:0/3864852425 shutdown_connections 2024-04-21T08:47:47.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.025+0000 7fc6ff352640 1 --2- 172.21.15.43:0/3864852425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6f8101b60 0x7fc6f810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.025+0000 7fc6ff352640 1 --2- 172.21.15.43:0/3864852425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f81011a0 0x7fc6f8101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.025+0000 7fc6ff352640 1 --2- 172.21.15.43:0/3864852425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f8105e30 0x7fc6f8106210 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.025+0000 7fc6ff352640 1 -- 172.21.15.43:0/3864852425 >> 172.21.15.43:0/3864852425 conn(0x7fc6f8077e30 msgr2=0x7fc6f8078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.025+0000 7fc6ff352640 1 -- 172.21.15.43:0/3864852425 shutdown_connections 2024-04-21T08:47:47.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.025+0000 7fc6ff352640 1 -- 172.21.15.43:0/3864852425 wait complete. 2024-04-21T08:47:47.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.026+0000 7fc6ff352640 1 Processor -- start 2024-04-21T08:47:47.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.026+0000 7fc6ff352640 1 -- start start 2024-04-21T08:47:47.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.026+0000 7fc6ff352640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f81011a0 0x7fc6f8197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.026+0000 7fc6ff352640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6f8101b60 0x7fc6f81978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.026+0000 7fc6ff352640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f8105e30 0x7fc6f8197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.026+0000 7fc6ff352640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f81110e0 con 0x7fc6f81011a0 2024-04-21T08:47:47.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.026+0000 7fc6ff352640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f8110f60 con 0x7fc6f8105e30 2024-04-21T08:47:47.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.026+0000 7fc6ff352640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f8111260 con 0x7fc6f8101b60 2024-04-21T08:47:47.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.026+0000 7fc6fd0ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f81011a0 0x7fc6f8197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.026+0000 7fc6fd0ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f81011a0 0x7fc6f8197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57366/0 (socket says 172.21.15.43:57366) 2024-04-21T08:47:47.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.026+0000 7fc6fd0ee640 1 -- 172.21.15.43:0/3353434283 learned_addr learned my addr 172.21.15.43:0/3353434283 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.026+0000 7fc6fd8ef640 1 --2- 172.21.15.43:0/3353434283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f8105e30 0x7fc6f8197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.026+0000 7fc6fd0ee640 1 -- 172.21.15.43:0/3353434283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6f8101b60 msgr2=0x7fc6f81978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.026+0000 7fc6fd0ee640 1 --2- 172.21.15.43:0/3353434283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6f8101b60 0x7fc6f81978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.026+0000 7fc6fd0ee640 1 -- 172.21.15.43:0/3353434283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f8105e30 msgr2=0x7fc6f8197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.026+0000 7fc6fd0ee640 1 --2- 172.21.15.43:0/3353434283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f8105e30 0x7fc6f8197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.026+0000 7fc6fd0ee640 1 -- 172.21.15.43:0/3353434283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f81a6be0 con 0x7fc6f81011a0 2024-04-21T08:47:47.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.026+0000 7fc6fd0ee640 1 --2- 172.21.15.43:0/3353434283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f81011a0 0x7fc6f8197370 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009230 tx=0x7fc6f4004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.026+0000 7fc6ee7fc640 1 -- 172.21.15.43:0/3353434283 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4043020 con 0x7fc6f81011a0 2024-04-21T08:47:47.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.027+0000 7fc6ee7fc640 1 -- 172.21.15.43:0/3353434283 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6f4004350 con 0x7fc6f81011a0 2024-04-21T08:47:47.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.027+0000 7fc6ff352640 1 -- 172.21.15.43:0/3353434283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f81a6e70 con 0x7fc6f81011a0 2024-04-21T08:47:47.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.027+0000 7fc6ee7fc640 1 -- 172.21.15.43:0/3353434283 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4003b50 con 0x7fc6f81011a0 2024-04-21T08:47:47.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.027+0000 7fc6ff352640 1 -- 172.21.15.43:0/3353434283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f81a7330 con 0x7fc6f81011a0 2024-04-21T08:47:47.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.027+0000 7fc6ff352640 1 -- 172.21.15.43:0/3353434283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6f8068de0 con 0x7fc6f81011a0 2024-04-21T08:47:47.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.029+0000 7fc6ee7fc640 1 -- 172.21.15.43:0/3353434283 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc6f4046020 con 0x7fc6f81011a0 2024-04-21T08:47:47.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.029+0000 7fc6ee7fc640 1 --2- 172.21.15.43:0/3353434283 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc6e40a3d20 0x7fc6e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.029+0000 7fc6ee7fc640 1 -- 172.21.15.43:0/3353434283 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fc6f40f3120 con 0x7fc6f81011a0 2024-04-21T08:47:47.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.029+0000 7fc6fc8ed640 1 --2- 172.21.15.43:0/3353434283 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc6e40a3d20 0x7fc6e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.030+0000 7fc6fc8ed640 1 --2- 172.21.15.43:0/3353434283 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc6e40a3d20 0x7fc6e40a61e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8005fd0 tx=0x7fc6e8005850 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.033+0000 7fc6ee7fc640 1 -- 172.21.15.43:0/3353434283 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc6f404d020 con 0x7fc6f81011a0 2024-04-21T08:47:47.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.263+0000 7fc6ff352640 1 -- 172.21.15.43:0/3353434283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6f802d5c0 con 0x7fc6f81011a0 2024-04-21T08:47:47.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.264+0000 7fc6ee7fc640 1 -- 172.21.15.43:0/3353434283 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11935 (secure 0 0 0) 0x7fc6f40bf5d0 con 0x7fc6f81011a0 2024-04-21T08:47:47.264 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:47:47.265 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:46:17.283806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27556":{"gid":27556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/229985237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":229985237},{"type":"v1","addr":"172.21.15.175:6837","nonce":229985237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24425,24412,24487,24527,24491]},"id":1}]} 2024-04-21T08:47:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-04-21T08:47:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.266+0000 7fc6ff352640 1 -- 172.21.15.43:0/3353434283 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc6e40a3d20 msgr2=0x7fc6e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.266+0000 7fc6ff352640 1 --2- 172.21.15.43:0/3353434283 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc6e40a3d20 0x7fc6e40a61e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8005fd0 tx=0x7fc6e8005850 comp rx=0 tx=0).stop 2024-04-21T08:47:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.266+0000 7fc6ff352640 1 -- 172.21.15.43:0/3353434283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f81011a0 msgr2=0x7fc6f8197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.266+0000 7fc6ff352640 1 --2- 172.21.15.43:0/3353434283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f81011a0 0x7fc6f8197370 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009230 tx=0x7fc6f4004bb0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.266+0000 7fc6ff352640 1 -- 172.21.15.43:0/3353434283 shutdown_connections 2024-04-21T08:47:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.266+0000 7fc6ff352640 1 --2- 172.21.15.43:0/3353434283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f8105e30 0x7fc6f8197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.266+0000 7fc6ff352640 1 --2- 172.21.15.43:0/3353434283 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc6e40a3d20 0x7fc6e40a61e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.266+0000 7fc6ff352640 1 --2- 172.21.15.43:0/3353434283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6f8101b60 0x7fc6f81978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.266+0000 7fc6ff352640 1 --2- 172.21.15.43:0/3353434283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f81011a0 0x7fc6f8197370 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.266+0000 7fc6ff352640 1 -- 172.21.15.43:0/3353434283 >> 172.21.15.43:0/3353434283 conn(0x7fc6f8077e30 msgr2=0x7fc6f80fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.266+0000 7fc6ff352640 1 -- 172.21.15.43:0/3353434283 shutdown_connections 2024-04-21T08:47:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.266+0000 7fc6ff352640 1 -- 172.21.15.43:0/3353434283 wait complete. 2024-04-21T08:47:47.277 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:47:47.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.381+0000 7fd9a9632640 1 -- 172.21.15.43:0/1622880865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a4107260 msgr2=0x7fd9a410ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.381+0000 7fd9a9632640 1 --2- 172.21.15.43:0/1622880865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a4107260 0x7fd9a410ee30 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fd994009900 tx=0x7fd99402bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:47.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.381+0000 7fd9a9632640 1 -- 172.21.15.43:0/1622880865 shutdown_connections 2024-04-21T08:47:47.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.381+0000 7fd9a9632640 1 --2- 172.21.15.43:0/1622880865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a4107260 0x7fd9a410ee30 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.381+0000 7fd9a9632640 1 --2- 172.21.15.43:0/1622880865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9a4102b40 0x7fd9a4106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.381+0000 7fd9a9632640 1 --2- 172.21.15.43:0/1622880865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9a4102190 0x7fd9a4102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.381+0000 7fd9a9632640 1 -- 172.21.15.43:0/1622880865 >> 172.21.15.43:0/1622880865 conn(0x7fd9a40fd990 msgr2=0x7fd9a40ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.381+0000 7fd9a9632640 1 -- 172.21.15.43:0/1622880865 shutdown_connections 2024-04-21T08:47:47.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.381+0000 7fd9a9632640 1 -- 172.21.15.43:0/1622880865 wait complete. 2024-04-21T08:47:47.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.382+0000 7fd9a9632640 1 Processor -- start 2024-04-21T08:47:47.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.382+0000 7fd9a9632640 1 -- start start 2024-04-21T08:47:47.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.382+0000 7fd9a9632640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a4102190 0x7fd9a4198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.382+0000 7fd9a9632640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9a4107260 0x7fd9a4198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.382+0000 7fd9a9632640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9a419cea0 0x7fd9a419d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.382+0000 7fd9a9632640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41121b0 con 0x7fd9a4102190 2024-04-21T08:47:47.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.382+0000 7fd9a9632640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4112030 con 0x7fd9a4107260 2024-04-21T08:47:47.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.382+0000 7fd9a9632640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4112330 con 0x7fd9a419cea0 2024-04-21T08:47:47.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.382+0000 7fd9a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a4102190 0x7fd9a4198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.382+0000 7fd9a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a4102190 0x7fd9a4198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57394/0 (socket says 172.21.15.43:57394) 2024-04-21T08:47:47.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.382+0000 7fd9a2ffd640 1 -- 172.21.15.43:0/272459627 learned_addr learned my addr 172.21.15.43:0/272459627 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.382+0000 7fd9a27fc640 1 --2- 172.21.15.43:0/272459627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9a4107260 0x7fd9a4198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.382+0000 7fd9a37fe640 1 --2- 172.21.15.43:0/272459627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9a419cea0 0x7fd9a419d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.382+0000 7fd9a2ffd640 1 -- 172.21.15.43:0/272459627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9a419cea0 msgr2=0x7fd9a419d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.382+0000 7fd9a2ffd640 1 --2- 172.21.15.43:0/272459627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9a419cea0 0x7fd9a419d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.382+0000 7fd9a2ffd640 1 -- 172.21.15.43:0/272459627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9a4107260 msgr2=0x7fd9a4198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.382+0000 7fd9a2ffd640 1 --2- 172.21.15.43:0/272459627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9a4107260 0x7fd9a4198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.382+0000 7fd9a2ffd640 1 -- 172.21.15.43:0/272459627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a419d990 con 0x7fd9a4102190 2024-04-21T08:47:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.383+0000 7fd9a2ffd640 1 --2- 172.21.15.43:0/272459627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a4102190 0x7fd9a4198630 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00c920 tx=0x7fd99c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.383+0000 7fd98bfff640 1 -- 172.21.15.43:0/272459627 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c007c70 con 0x7fd9a4102190 2024-04-21T08:47:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.383+0000 7fd98bfff640 1 -- 172.21.15.43:0/272459627 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd99c007e10 con 0x7fd9a4102190 2024-04-21T08:47:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.383+0000 7fd9a9632640 1 -- 172.21.15.43:0/272459627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a419dc80 con 0x7fd9a4102190 2024-04-21T08:47:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.383+0000 7fd98bfff640 1 -- 172.21.15.43:0/272459627 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c018820 con 0x7fd9a4102190 2024-04-21T08:47:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.383+0000 7fd9a9632640 1 -- 172.21.15.43:0/272459627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a41a5470 con 0x7fd9a4102190 2024-04-21T08:47:47.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.385+0000 7fd98bfff640 1 -- 172.21.15.43:0/272459627 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd99c00f450 con 0x7fd9a4102190 2024-04-21T08:47:47.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.385+0000 7fd9ac21c740 1 Processor -- start 2024-04-21T08:47:47.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.385+0000 7fd9ac21c740 1 -- start start 2024-04-21T08:47:47.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.385+0000 7fd9ac21c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc6b330 0x55d47cc6b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.385+0000 7fd9ac21c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d47cc6bce0 0x55d47cc74820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.385+0000 7fd9ac21c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d47cc74e40 0x55d47cc752c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.385+0000 7fd9ac21c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d47cbd01f0 con 0x55d47cc6b330 2024-04-21T08:47:47.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.385+0000 7fd9ac21c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d47cb6d4b0 con 0x55d47cc6bce0 2024-04-21T08:47:47.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.385+0000 7fd9ac21c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d47cbcc1b0 con 0x55d47cc74e40 2024-04-21T08:47:47.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.386+0000 7fd9a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc6b330 0x55d47cc6b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.386+0000 7fd9a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc6b330 0x55d47cc6b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57402/0 (socket says 172.21.15.43:57402) 2024-04-21T08:47:47.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.386+0000 7fd9a37fe640 1 -- 172.21.15.43:0/991282014 learned_addr learned my addr 172.21.15.43:0/991282014 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.386+0000 7fd98bfff640 1 --2- 172.21.15.43:0/272459627 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd98c0a3d20 0x7fd98c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.386+0000 7fd9a2ffd640 1 --2- 172.21.15.43:0/991282014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d47cc6bce0 0x55d47cc74820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.386+0000 7fd9a37fe640 1 --2- 172.21.15.43:0/272459627 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd98c0a3d20 0x7fd98c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.386+0000 7fd9a27fc640 1 --2- 172.21.15.43:0/991282014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d47cc74e40 0x55d47cc752c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.386+0000 7fd9a37fe640 1 -- 172.21.15.43:0/991282014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d47cc74e40 msgr2=0x55d47cc752c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.386+0000 7fd98bfff640 1 -- 172.21.15.43:0/272459627 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fd99c0d0df0 con 0x7fd9a4102190 2024-04-21T08:47:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.386+0000 7fd9a37fe640 1 --2- 172.21.15.43:0/991282014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d47cc74e40 0x55d47cc752c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.386+0000 7fd9a37fe640 1 -- 172.21.15.43:0/991282014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d47cc6bce0 msgr2=0x55d47cc74820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.386+0000 7fd9a37fe640 1 --2- 172.21.15.43:0/991282014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d47cc6bce0 0x55d47cc74820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.386+0000 7fd9a37fe640 1 -- 172.21.15.43:0/991282014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d47cb5f1b0 con 0x55d47cc6b330 2024-04-21T08:47:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.386+0000 7fd9a27fc640 1 --2- 172.21.15.43:0/991282014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d47cc74e40 0x55d47cc752c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.386+0000 7fd9a37fe640 1 --2- 172.21.15.43:0/991282014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc6b330 0x55d47cc6b710 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fd994005ec0 tx=0x7fd994004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dacd7c2e6607da4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.386+0000 7fd9897fa640 1 -- 172.21.15.43:0/991282014 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994005410 con 0x55d47cc6b330 2024-04-21T08:47:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.386+0000 7fd9897fa640 1 -- 172.21.15.43:0/991282014 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9940055b0 con 0x55d47cc6b330 2024-04-21T08:47:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.386+0000 7fd9897fa640 1 -- 172.21.15.43:0/991282014 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9940038e0 con 0x55d47cc6b330 2024-04-21T08:47:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.386+0000 7fd9a37fe640 1 --2- 172.21.15.43:0/272459627 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd98c0a3d20 0x7fd98c0a61e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd99404d000 tx=0x7fd99404b000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.387+0000 7fd9ac21c740 1 -- 172.21.15.43:0/991282014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc6b330 msgr2=0x55d47cc6b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.387+0000 7fd9ac21c740 1 --2- 172.21.15.43:0/991282014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc6b330 0x55d47cc6b710 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fd994005ec0 tx=0x7fd994004580 comp rx=0 tx=0).stop 2024-04-21T08:47:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.387+0000 7fd9ac21c740 1 -- 172.21.15.43:0/991282014 shutdown_connections 2024-04-21T08:47:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.387+0000 7fd9ac21c740 1 --2- 172.21.15.43:0/991282014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d47cc74e40 0x55d47cc752c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.387+0000 7fd9ac21c740 1 --2- 172.21.15.43:0/991282014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d47cc6bce0 0x55d47cc74820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.387+0000 7fd9ac21c740 1 --2- 172.21.15.43:0/991282014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc6b330 0x55d47cc6b710 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.387+0000 7fd9ac21c740 1 -- 172.21.15.43:0/991282014 >> 172.21.15.43:0/991282014 conn(0x55d47cc65ba0 msgr2=0x55d47cc65f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.387+0000 7fd9ac21c740 1 -- 172.21.15.43:0/991282014 shutdown_connections 2024-04-21T08:47:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.387+0000 7fd9ac21c740 1 -- 172.21.15.43:0/991282014 wait complete. 2024-04-21T08:47:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.387+0000 7fd9ac21c740 1 -- start start 2024-04-21T08:47:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.388+0000 7fd9ac21c740 1 Processor -- start 2024-04-21T08:47:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.388+0000 7fd9ac21c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d47cc6b330 0x55d47cc7b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.388+0000 7fd9ac21c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d47cc6bce0 0x55d47cc78590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.388+0000 7fd9ac21c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc74e40 0x55d47cc78ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.388+0000 7fd9ac21c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d47cc7bdc0 con 0x55d47cc74e40 2024-04-21T08:47:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.388+0000 7fd9ac21c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d47cb89500 con 0x55d47cc6bce0 2024-04-21T08:47:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.388+0000 7fd9ac21c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d47cc7bf40 con 0x55d47cc6b330 2024-04-21T08:47:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.388+0000 7fd9ac21c740 10 client.0 _ll_register_callbacks cb 0x55d47cb57b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.388+0000 7fd9a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc74e40 0x55d47cc78ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.388+0000 7fd9a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc74e40 0x55d47cc78ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57414/0 (socket says 172.21.15.43:57414) 2024-04-21T08:47:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.388+0000 7fd9a37fe640 1 -- 172.21.15.43:0/4286671424 learned_addr learned my addr 172.21.15.43:0/4286671424 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.388+0000 7fd9a2ffd640 1 --2- 172.21.15.43:0/4286671424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d47cc6b330 0x55d47cc7b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.388+0000 7fd9a27fc640 1 --2- 172.21.15.43:0/4286671424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d47cc6bce0 0x55d47cc78590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.388+0000 7fd9a37fe640 1 -- 172.21.15.43:0/4286671424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d47cc6b330 msgr2=0x55d47cc7b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.388+0000 7fd9a37fe640 1 --2- 172.21.15.43:0/4286671424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d47cc6b330 0x55d47cc7b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.388+0000 7fd9a37fe640 1 -- 172.21.15.43:0/4286671424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d47cc6bce0 msgr2=0x55d47cc78590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.388+0000 7fd9a37fe640 1 --2- 172.21.15.43:0/4286671424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d47cc6bce0 0x55d47cc78590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.388+0000 7fd9a37fe640 1 -- 172.21.15.43:0/4286671424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d47c8e23f0 con 0x55d47cc74e40 2024-04-21T08:47:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.388+0000 7fd9a27fc640 1 --2- 172.21.15.43:0/4286671424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d47cc6bce0 0x55d47cc78590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.388+0000 7fd9a37fe640 1 --2- 172.21.15.43:0/4286671424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc74e40 0x55d47cc78ad0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fd994006010 tx=0x7fd9940058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a664bf85a726ba7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.388+0000 7fd986ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.389+0000 7fd986ff5640 1 -- 172.21.15.43:0/4286671424 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99405a070 con 0x55d47cc74e40 2024-04-21T08:47:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.389+0000 7fd986ff5640 1 -- 172.21.15.43:0/4286671424 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fd9940038c0 con 0x55d47cc74e40 2024-04-21T08:47:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.389+0000 7fd9ac21c740 1 -- 172.21.15.43:0/4286671424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9940055b0 con 0x55d47cc74e40 2024-04-21T08:47:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.389+0000 7fd986ff5640 1 -- 172.21.15.43:0/4286671424 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd994066050 con 0x55d47cc74e40 2024-04-21T08:47:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.389+0000 7fd9ac21c740 10 client.19554 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.389+0000 7fd9ac21c740 1 -- 172.21.15.43:0/4286671424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d47c9ac7d0 con 0x55d47cc74e40 2024-04-21T08:47:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.389+0000 7fd986ff5640 1 -- 172.21.15.43:0/4286671424 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9940055b0 con 0x55d47cc74e40 2024-04-21T08:47:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.389+0000 7fd986ff5640 1 -- 172.21.15.43:0/4286671424 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd994043450 con 0x55d47cc74e40 2024-04-21T08:47:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.389+0000 7fd9ac21c740 10 client.19554 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.389+0000 7fd9ac21c740 10 client.19554 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:47:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.389+0000 7fd9ac21c740 20 client.19554 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.389+0000 7fd9ac21c740 1 --2- 172.21.15.43:0/4286671424 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55d47cc88ed0 0x55d47cc8b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.389+0000 7fd9ac21c740 4 client.19554 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.389+0000 7fd9ac21c740 1 -- 172.21.15.43:0/4286671424 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d47cb6f930 con 0x55d47cc88ed0 2024-04-21T08:47:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.390+0000 7fd9a2ffd640 1 --2- 172.21.15.43:0/4286671424 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55d47cc88ed0 0x55d47cc8b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.390+0000 7fd986ff5640 10 client.19554 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:47:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.390+0000 7fd9a2ffd640 1 --2- 172.21.15.43:0/4286671424 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55d47cc88ed0 0x55d47cc8b2c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f27027447887321e server_cookie=c71a7e289c3a8ded in_seq=0 out_seq=0 2024-04-21T08:47:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.391+0000 7fd986ff5640 1 -- 172.21.15.43:0/4286671424 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d47cb6f930 con 0x55d47cc88ed0 2024-04-21T08:47:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.391+0000 7fd986ff5640 10 client.19554 handle_command_reply: tid=0 2024-04-21T08:47:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.391+0000 7fd9ac21c740 1 client.19554 shutdown 2024-04-21T08:47:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.391+0000 7fd9ac21c740 1 -- 172.21.15.43:0/4286671424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc74e40 msgr2=0x55d47cc78ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.391+0000 7fd9ac21c740 1 --2- 172.21.15.43:0/4286671424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc74e40 0x55d47cc78ad0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fd994006010 tx=0x7fd9940058b0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.391+0000 7fd9ac21c740 1 -- 172.21.15.43:0/4286671424 shutdown_connections 2024-04-21T08:47:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.391+0000 7fd9ac21c740 1 --2- 172.21.15.43:0/4286671424 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55d47cc88ed0 0x55d47cc8b2c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.391+0000 7fd9ac21c740 1 --2- 172.21.15.43:0/4286671424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc74e40 0x55d47cc78ad0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.391+0000 7fd9ac21c740 1 --2- 172.21.15.43:0/4286671424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d47cc6bce0 0x55d47cc78590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.391+0000 7fd9ac21c740 1 --2- 172.21.15.43:0/4286671424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d47cc6b330 0x55d47cc7b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.391+0000 7fd9ac21c740 1 -- 172.21.15.43:0/4286671424 >> 172.21.15.43:0/4286671424 conn(0x55d47cc65ba0 msgr2=0x55d47cc678f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.391+0000 7fd986ff5640 0 client.19554 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:47:47.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.391+0000 7fd9ac21c740 1 -- 172.21.15.43:0/4286671424 shutdown_connections 2024-04-21T08:47:47.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.391+0000 7fd9ac21c740 1 -- 172.21.15.43:0/4286671424 wait complete. 2024-04-21T08:47:47.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.391+0000 7fd9ac21c740 20 client.19554 trim_cache size 0 max 16384 2024-04-21T08:47:47.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.404+0000 7fd9ac21c740 1 Processor -- start 2024-04-21T08:47:47.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.404+0000 7fd9ac21c740 1 -- start start 2024-04-21T08:47:47.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.404+0000 7fd9ac21c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc6b330 0x55d47cc76230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.404+0000 7fd9ac21c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d47cc6bce0 0x55d47cc84cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.404+0000 7fd9ac21c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d47cc74e40 0x55d47cc7acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.404+0000 7fd9ac21c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41ad1c0 con 0x55d47cc6b330 2024-04-21T08:47:47.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.404+0000 7fd9ac21c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41aed00 con 0x55d47cc6bce0 2024-04-21T08:47:47.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.404+0000 7fd9ac21c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41ad890 con 0x55d47cc74e40 2024-04-21T08:47:47.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.405+0000 7fd9a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc6b330 0x55d47cc76230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.405+0000 7fd9a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc6b330 0x55d47cc76230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57428/0 (socket says 172.21.15.43:57428) 2024-04-21T08:47:47.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.405+0000 7fd9a2ffd640 1 -- 172.21.15.43:0/2451743561 learned_addr learned my addr 172.21.15.43:0/2451743561 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.405+0000 7fd9a27fc640 1 --2- 172.21.15.43:0/2451743561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d47cc6bce0 0x55d47cc84cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.405+0000 7fd9a2ffd640 1 -- 172.21.15.43:0/2451743561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d47cc74e40 msgr2=0x55d47cc7acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.405+0000 7fd9a37fe640 1 --2- 172.21.15.43:0/2451743561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d47cc74e40 0x55d47cc7acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.405+0000 7fd9a2ffd640 1 --2- 172.21.15.43:0/2451743561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d47cc74e40 0x55d47cc7acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.405+0000 7fd9a2ffd640 1 -- 172.21.15.43:0/2451743561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d47cc6bce0 msgr2=0x55d47cc84cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.405+0000 7fd9a2ffd640 1 --2- 172.21.15.43:0/2451743561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d47cc6bce0 0x55d47cc84cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.405+0000 7fd9a2ffd640 1 -- 172.21.15.43:0/2451743561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d47cc7bf30 con 0x55d47cc6b330 2024-04-21T08:47:47.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.405+0000 7fd9a27fc640 1 --2- 172.21.15.43:0/2451743561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d47cc6bce0 0x55d47cc84cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:47.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.405+0000 7fd9a2ffd640 1 --2- 172.21.15.43:0/2451743561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc6b330 0x55d47cc76230 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0ce720 tx=0x7fd99c087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626fbf63233ea8d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.405+0000 7fd988ff9640 1 -- 172.21.15.43:0/2451743561 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c09aa10 con 0x55d47cc6b330 2024-04-21T08:47:47.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.405+0000 7fd988ff9640 1 -- 172.21.15.43:0/2451743561 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd99c09ac70 con 0x55d47cc6b330 2024-04-21T08:47:47.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.405+0000 7fd988ff9640 1 -- 172.21.15.43:0/2451743561 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c098fe0 con 0x55d47cc6b330 2024-04-21T08:47:47.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.406+0000 7fd9ac21c740 1 -- 172.21.15.43:0/2451743561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc6b330 msgr2=0x55d47cc76230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.406+0000 7fd9ac21c740 1 --2- 172.21.15.43:0/2451743561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc6b330 0x55d47cc76230 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0ce720 tx=0x7fd99c087040 comp rx=0 tx=0).stop 2024-04-21T08:47:47.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.406+0000 7fd9ac21c740 1 -- 172.21.15.43:0/2451743561 shutdown_connections 2024-04-21T08:47:47.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.406+0000 7fd9ac21c740 1 --2- 172.21.15.43:0/2451743561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d47cc74e40 0x55d47cc7acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.406+0000 7fd9ac21c740 1 --2- 172.21.15.43:0/2451743561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d47cc6bce0 0x55d47cc84cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.406+0000 7fd9ac21c740 1 --2- 172.21.15.43:0/2451743561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc6b330 0x55d47cc76230 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.406+0000 7fd9ac21c740 1 -- 172.21.15.43:0/2451743561 >> 172.21.15.43:0/2451743561 conn(0x55d47cc65ba0 msgr2=0x55d47cc64c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.406+0000 7fd9ac21c740 1 -- 172.21.15.43:0/2451743561 shutdown_connections 2024-04-21T08:47:47.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.406+0000 7fd9ac21c740 1 -- 172.21.15.43:0/2451743561 wait complete. 2024-04-21T08:47:47.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.406+0000 7fd9ac21c740 1 -- start start 2024-04-21T08:47:47.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.406+0000 7fd9ac21c740 1 Processor -- start 2024-04-21T08:47:47.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.406+0000 7fd9ac21c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc6b330 0x55d47cc811d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.406+0000 7fd9ac21c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d47cc6bce0 0x55d47cc81710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.406+0000 7fd9ac21c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d47cc74e40 0x55d47cc79dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.406+0000 7fd9ac21c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41ae630 con 0x55d47cc6b330 2024-04-21T08:47:47.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.406+0000 7fd9ac21c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41adf60 con 0x55d47cc74e40 2024-04-21T08:47:47.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.406+0000 7fd9ac21c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41ab1c0 con 0x55d47cc6bce0 2024-04-21T08:47:47.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.407+0000 7fd9ac21c740 10 client.0 _ll_register_callbacks cb 0x7fd9940043d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:47.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.407+0000 7fd9a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc6b330 0x55d47cc811d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.407+0000 7fd9a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc6b330 0x55d47cc811d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57442/0 (socket says 172.21.15.43:57442) 2024-04-21T08:47:47.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.407+0000 7fd9a2ffd640 1 -- 172.21.15.43:0/3802181202 learned_addr learned my addr 172.21.15.43:0/3802181202 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.407+0000 7fd9a27fc640 1 --2- 172.21.15.43:0/3802181202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d47cc6bce0 0x55d47cc81710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.407+0000 7fd9a37fe640 1 --2- 172.21.15.43:0/3802181202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d47cc74e40 0x55d47cc79dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.407+0000 7fd9a2ffd640 1 -- 172.21.15.43:0/3802181202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d47cc6bce0 msgr2=0x55d47cc81710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.407+0000 7fd9a2ffd640 1 --2- 172.21.15.43:0/3802181202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d47cc6bce0 0x55d47cc81710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.407+0000 7fd9a2ffd640 1 -- 172.21.15.43:0/3802181202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d47cc74e40 msgr2=0x55d47cc79dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.407+0000 7fd9a2ffd640 1 --2- 172.21.15.43:0/3802181202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d47cc74e40 0x55d47cc79dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.407+0000 7fd9a2ffd640 1 -- 172.21.15.43:0/3802181202 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d47cc7a600 con 0x55d47cc6b330 2024-04-21T08:47:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.407+0000 7fd9a37fe640 1 --2- 172.21.15.43:0/3802181202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d47cc74e40 0x55d47cc79dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.407+0000 7fd9a2ffd640 1 --2- 172.21.15.43:0/3802181202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc6b330 0x55d47cc811d0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fd99c087010 tx=0x7fd99c0a5760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc38ad9fe93ee51a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.407+0000 7fd986ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.407+0000 7fd986ff5640 1 -- 172.21.15.43:0/3802181202 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c00f740 con 0x55d47cc6b330 2024-04-21T08:47:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.407+0000 7fd986ff5640 1 -- 172.21.15.43:0/3802181202 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fd99c067340 con 0x55d47cc6b330 2024-04-21T08:47:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.407+0000 7fd9ac21c740 1 -- 172.21.15.43:0/3802181202 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd99c09ac70 con 0x55d47cc6b330 2024-04-21T08:47:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.407+0000 7fd986ff5640 1 -- 172.21.15.43:0/3802181202 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd99c0ce3a0 con 0x55d47cc6b330 2024-04-21T08:47:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.407+0000 7fd9ac21c740 10 client.19560 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.407+0000 7fd9ac21c740 1 -- 172.21.15.43:0/3802181202 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d47cceedf0 con 0x55d47cc6b330 2024-04-21T08:47:47.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.407+0000 7fd986ff5640 1 -- 172.21.15.43:0/3802181202 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd99c09ac70 con 0x55d47cc6b330 2024-04-21T08:47:47.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.408+0000 7fd986ff5640 1 -- 172.21.15.43:0/3802181202 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd99c0870b0 con 0x55d47cc6b330 2024-04-21T08:47:47.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.408+0000 7fd9ac21c740 10 client.19560 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:47.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.408+0000 7fd9ac21c740 10 client.19560 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:47:47.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.408+0000 7fd9ac21c740 20 client.19560 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:47.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.408+0000 7fd9ac21c740 1 --2- 172.21.15.43:0/3802181202 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55d47ccef530 0x55d47ccf1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.408+0000 7fd9ac21c740 4 client.19560 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:47.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.408+0000 7fd9ac21c740 1 -- 172.21.15.43:0/3802181202 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d47cbddc10 con 0x55d47ccef530 2024-04-21T08:47:47.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.408+0000 7fd9a27fc640 1 --2- 172.21.15.43:0/3802181202 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55d47ccef530 0x55d47ccf1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.408+0000 7fd986ff5640 10 client.19560 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:47:47.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.408+0000 7fd9a27fc640 1 --2- 172.21.15.43:0/3802181202 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55d47ccef530 0x55d47ccf1950 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ec42b9977045f0c6 server_cookie=ab3399ece4b1824f in_seq=0 out_seq=0 2024-04-21T08:47:47.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.408+0000 7fd986ff5640 1 -- 172.21.15.43:0/3802181202 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d47cbddc10 con 0x55d47ccef530 2024-04-21T08:47:47.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.408+0000 7fd986ff5640 10 client.19560 handle_command_reply: tid=0 2024-04-21T08:47:47.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.408+0000 7fd9ac21c740 1 client.19560 shutdown 2024-04-21T08:47:47.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.409+0000 7fd9ac21c740 1 -- 172.21.15.43:0/3802181202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc6b330 msgr2=0x55d47cc811d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.409+0000 7fd9ac21c740 1 --2- 172.21.15.43:0/3802181202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc6b330 0x55d47cc811d0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fd99c087010 tx=0x7fd99c0a5760 comp rx=0 tx=0).stop 2024-04-21T08:47:47.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.409+0000 7fd9ac21c740 1 -- 172.21.15.43:0/3802181202 shutdown_connections 2024-04-21T08:47:47.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.409+0000 7fd9ac21c740 1 --2- 172.21.15.43:0/3802181202 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55d47ccef530 0x55d47ccf1950 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.409+0000 7fd9ac21c740 1 --2- 172.21.15.43:0/3802181202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d47cc74e40 0x55d47cc79dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.409+0000 7fd9ac21c740 1 --2- 172.21.15.43:0/3802181202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d47cc6bce0 0x55d47cc81710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.409+0000 7fd986ff5640 0 client.19560 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:47:47.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.409+0000 7fd9ac21c740 1 --2- 172.21.15.43:0/3802181202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d47cc6b330 0x55d47cc811d0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.409+0000 7fd9ac21c740 1 -- 172.21.15.43:0/3802181202 >> 172.21.15.43:0/3802181202 conn(0x55d47cc65ba0 msgr2=0x55d47cc77e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.409+0000 7fd9ac21c740 1 -- 172.21.15.43:0/3802181202 shutdown_connections 2024-04-21T08:47:47.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.409+0000 7fd9ac21c740 1 -- 172.21.15.43:0/3802181202 wait complete. 2024-04-21T08:47:47.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.409+0000 7fd9ac21c740 20 client.19560 trim_cache size 0 max 16384 2024-04-21T08:47:47.417 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:47:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.409+0000 7fd9a9632640 1 -- 172.21.15.43:0/272459627 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd98c0a3d20 msgr2=0x7fd98c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.409+0000 7fd9a9632640 1 --2- 172.21.15.43:0/272459627 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd98c0a3d20 0x7fd98c0a61e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd99404d000 tx=0x7fd99404b000 comp rx=0 tx=0).stop 2024-04-21T08:47:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.409+0000 7fd9a9632640 1 -- 172.21.15.43:0/272459627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a4102190 msgr2=0x7fd9a4198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.409+0000 7fd9a9632640 1 --2- 172.21.15.43:0/272459627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a4102190 0x7fd9a4198630 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00c920 tx=0x7fd99c00cde0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.409+0000 7fd9a9632640 1 -- 172.21.15.43:0/272459627 shutdown_connections 2024-04-21T08:47:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.409+0000 7fd9a9632640 1 --2- 172.21.15.43:0/272459627 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd98c0a3d20 0x7fd98c0a61e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.409+0000 7fd9a9632640 1 --2- 172.21.15.43:0/272459627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9a419cea0 0x7fd9a419d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.409+0000 7fd9a9632640 1 --2- 172.21.15.43:0/272459627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9a4107260 0x7fd9a4198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.409+0000 7fd9a9632640 1 --2- 172.21.15.43:0/272459627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a4102190 0x7fd9a4198630 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.409+0000 7fd9a9632640 1 -- 172.21.15.43:0/272459627 >> 172.21.15.43:0/272459627 conn(0x7fd9a40fd990 msgr2=0x7fd9a40fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.409+0000 7fd9a9632640 1 -- 172.21.15.43:0/272459627 shutdown_connections 2024-04-21T08:47:47.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.409+0000 7fd9a9632640 1 -- 172.21.15.43:0/272459627 wait complete. 2024-04-21T08:47:47.420 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:47:47.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.524+0000 7fe5209d5640 1 -- 172.21.15.43:0/3576902413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe51c107260 msgr2=0x7fe51c10ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.524+0000 7fe5209d5640 1 --2- 172.21.15.43:0/3576902413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe51c107260 0x7fe51c10ee30 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fe514009ca0 tx=0x7fe51402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.524+0000 7fe5209d5640 1 -- 172.21.15.43:0/3576902413 shutdown_connections 2024-04-21T08:47:47.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.524+0000 7fe5209d5640 1 --2- 172.21.15.43:0/3576902413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe51c107260 0x7fe51c10ee30 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.524+0000 7fe5209d5640 1 --2- 172.21.15.43:0/3576902413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe51c102b40 0x7fe51c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.524+0000 7fe5209d5640 1 --2- 172.21.15.43:0/3576902413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe51c102190 0x7fe51c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.524+0000 7fe5209d5640 1 -- 172.21.15.43:0/3576902413 >> 172.21.15.43:0/3576902413 conn(0x7fe51c0fd990 msgr2=0x7fe51c0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.524+0000 7fe5209d5640 1 -- 172.21.15.43:0/3576902413 shutdown_connections 2024-04-21T08:47:47.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.524+0000 7fe5209d5640 1 -- 172.21.15.43:0/3576902413 wait complete. 2024-04-21T08:47:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.525+0000 7fe5209d5640 1 Processor -- start 2024-04-21T08:47:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.525+0000 7fe5209d5640 1 -- start start 2024-04-21T08:47:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.525+0000 7fe5209d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe51c102190 0x7fe51c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.525+0000 7fe5209d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe51c107260 0x7fe51c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.526+0000 7fe5209d5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe51c19cff0 0x7fe51c19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.526+0000 7fe5209d5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1121d0 con 0x7fe51c107260 2024-04-21T08:47:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.526+0000 7fe5209d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c112050 con 0x7fe51c102190 2024-04-21T08:47:47.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.526+0000 7fe5209d5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c112350 con 0x7fe51c19cff0 2024-04-21T08:47:47.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.526+0000 7fe511d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe51c107260 0x7fe51c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.526+0000 7fe511d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe51c107260 0x7fe51c198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57464/0 (socket says 172.21.15.43:57464) 2024-04-21T08:47:47.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.526+0000 7fe511d9b640 1 -- 172.21.15.43:0/2604114953 learned_addr learned my addr 172.21.15.43:0/2604114953 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.526+0000 7fe51ad9d640 1 --2- 172.21.15.43:0/2604114953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe51c19cff0 0x7fe51c19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.526+0000 7fe51a59c640 1 --2- 172.21.15.43:0/2604114953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe51c102190 0x7fe51c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.526+0000 7fe511d9b640 1 -- 172.21.15.43:0/2604114953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe51c19cff0 msgr2=0x7fe51c19d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.526+0000 7fe511d9b640 1 --2- 172.21.15.43:0/2604114953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe51c19cff0 0x7fe51c19d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.526+0000 7fe511d9b640 1 -- 172.21.15.43:0/2604114953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe51c102190 msgr2=0x7fe51c198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.526+0000 7fe511d9b640 1 --2- 172.21.15.43:0/2604114953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe51c102190 0x7fe51c198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.526+0000 7fe511d9b640 1 -- 172.21.15.43:0/2604114953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51c19dae0 con 0x7fe51c107260 2024-04-21T08:47:47.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.526+0000 7fe51a59c640 1 --2- 172.21.15.43:0/2604114953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe51c102190 0x7fe51c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:47.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.526+0000 7fe511d9b640 1 --2- 172.21.15.43:0/2604114953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe51c107260 0x7fe51c198c70 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fe50400d770 tx=0x7fe50400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.526+0000 7fe513fff640 1 -- 172.21.15.43:0/2604114953 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504017070 con 0x7fe51c107260 2024-04-21T08:47:47.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.526+0000 7fe5209d5640 1 -- 172.21.15.43:0/2604114953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe51c19ddd0 con 0x7fe51c107260 2024-04-21T08:47:47.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.526+0000 7fe513fff640 1 -- 172.21.15.43:0/2604114953 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe504009c30 con 0x7fe51c107260 2024-04-21T08:47:47.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.526+0000 7fe513fff640 1 -- 172.21.15.43:0/2604114953 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50400f620 con 0x7fe51c107260 2024-04-21T08:47:47.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.526+0000 7fe5209d5640 1 -- 172.21.15.43:0/2604114953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe51c1a5610 con 0x7fe51c107260 2024-04-21T08:47:47.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.529+0000 7fe513fff640 1 -- 172.21.15.43:0/2604114953 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe50401a070 con 0x7fe51c107260 2024-04-21T08:47:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.529+0000 7fe5235ca740 1 Processor -- start 2024-04-21T08:47:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.529+0000 7fe5235ca740 1 -- start start 2024-04-21T08:47:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.529+0000 7fe5235ca740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585d74d0 0x555f585d78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.529+0000 7fe5235ca740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f585d7e80 0x555f585e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.529+0000 7fe5235ca740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f585e0a00 0x555f585e2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.529+0000 7fe5235ca740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555f5853c1f0 con 0x555f585d74d0 2024-04-21T08:47:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.529+0000 7fe5235ca740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555f584d94b0 con 0x555f585d7e80 2024-04-21T08:47:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.529+0000 7fe5235ca740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555f585381b0 con 0x555f585e0a00 2024-04-21T08:47:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.529+0000 7fe51ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585d74d0 0x555f585d78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.529+0000 7fe51ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585d74d0 0x555f585d78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57470/0 (socket says 172.21.15.43:57470) 2024-04-21T08:47:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.529+0000 7fe51ad9d640 1 -- 172.21.15.43:0/3656791865 learned_addr learned my addr 172.21.15.43:0/3656791865 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.529+0000 7fe51a59c640 1 --2- 172.21.15.43:0/3656791865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f585d7e80 0x555f585e0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.529+0000 7fe513fff640 1 --2- 172.21.15.43:0/2604114953 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe5080a3d20 0x7fe5080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.529+0000 7fe511d9b640 1 --2- 172.21.15.43:0/3656791865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f585e0a00 0x555f585e2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.530+0000 7fe51ad9d640 1 -- 172.21.15.43:0/3656791865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f585e0a00 msgr2=0x555f585e2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.530+0000 7fe51ad9d640 1 --2- 172.21.15.43:0/3656791865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f585e0a00 0x555f585e2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.530+0000 7fe51ad9d640 1 -- 172.21.15.43:0/3656791865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f585d7e80 msgr2=0x555f585e0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.530+0000 7fe51ad9d640 1 --2- 172.21.15.43:0/3656791865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f585d7e80 0x555f585e0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.530+0000 7fe51ad9d640 1 -- 172.21.15.43:0/3656791865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f584cb1b0 con 0x555f585d74d0 2024-04-21T08:47:47.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.530+0000 7fe51a59c640 1 --2- 172.21.15.43:0/3656791865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f585d7e80 0x555f585e0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:47.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.530+0000 7fe513fff640 1 -- 172.21.15.43:0/2604114953 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fe5040d0700 con 0x7fe51c107260 2024-04-21T08:47:47.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.530+0000 7fe51ad9d640 1 --2- 172.21.15.43:0/2604114953 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe5080a3d20 0x7fe5080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.530+0000 7fe51ad9d640 1 --2- 172.21.15.43:0/3656791865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585d74d0 0x555f585d78b0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fe514009ca0 tx=0x7fe514009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee354f242300d87 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.530+0000 7fe510d99640 1 -- 172.21.15.43:0/3656791865 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514039040 con 0x555f585d74d0 2024-04-21T08:47:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.530+0000 7fe510d99640 1 -- 172.21.15.43:0/3656791865 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe514003e20 con 0x555f585d74d0 2024-04-21T08:47:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.530+0000 7fe510d99640 1 -- 172.21.15.43:0/3656791865 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514004800 con 0x555f585d74d0 2024-04-21T08:47:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.531+0000 7fe5235ca740 1 -- 172.21.15.43:0/3656791865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585d74d0 msgr2=0x555f585d78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.531+0000 7fe51ad9d640 1 --2- 172.21.15.43:0/2604114953 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe5080a3d20 0x7fe5080a61e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe51404c000 tx=0x7fe514045000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.531+0000 7fe5235ca740 1 --2- 172.21.15.43:0/3656791865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585d74d0 0x555f585d78b0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fe514009ca0 tx=0x7fe514009900 comp rx=0 tx=0).stop 2024-04-21T08:47:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.531+0000 7fe5235ca740 1 -- 172.21.15.43:0/3656791865 shutdown_connections 2024-04-21T08:47:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.531+0000 7fe5235ca740 1 --2- 172.21.15.43:0/3656791865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f585e0a00 0x555f585e2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.531+0000 7fe5235ca740 1 --2- 172.21.15.43:0/3656791865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f585d7e80 0x555f585e0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.531+0000 7fe5235ca740 1 --2- 172.21.15.43:0/3656791865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585d74d0 0x555f585d78b0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.531+0000 7fe5235ca740 1 -- 172.21.15.43:0/3656791865 >> 172.21.15.43:0/3656791865 conn(0x555f585d1ba0 msgr2=0x555f585d1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.531+0000 7fe5235ca740 1 -- 172.21.15.43:0/3656791865 shutdown_connections 2024-04-21T08:47:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.531+0000 7fe5235ca740 1 -- 172.21.15.43:0/3656791865 wait complete. 2024-04-21T08:47:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.531+0000 7fe5235ca740 1 -- start start 2024-04-21T08:47:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.532+0000 7fe5235ca740 1 Processor -- start 2024-04-21T08:47:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.532+0000 7fe5235ca740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f585d74d0 0x555f585e0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.532+0000 7fe5235ca740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f585d7e80 0x555f585dd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.532+0000 7fe5235ca740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585e0a00 0x555f585ddcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.532+0000 7fe5235ca740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555f585e3970 con 0x555f585e0a00 2024-04-21T08:47:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.532+0000 7fe5235ca740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555f584f5500 con 0x555f585d7e80 2024-04-21T08:47:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.532+0000 7fe5235ca740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555f585e3af0 con 0x555f585d74d0 2024-04-21T08:47:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.532+0000 7fe5235ca740 10 client.0 _ll_register_callbacks cb 0x555f584c3b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.532+0000 7fe51ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585e0a00 0x555f585ddcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.532+0000 7fe51ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585e0a00 0x555f585ddcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57482/0 (socket says 172.21.15.43:57482) 2024-04-21T08:47:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.532+0000 7fe51ad9d640 1 -- 172.21.15.43:0/2762174184 learned_addr learned my addr 172.21.15.43:0/2762174184 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.532+0000 7fe511d9b640 1 --2- 172.21.15.43:0/2762174184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f585d7e80 0x555f585dd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.532+0000 7fe51a59c640 1 --2- 172.21.15.43:0/2762174184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f585d74d0 0x555f585e0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.532+0000 7fe51ad9d640 1 -- 172.21.15.43:0/2762174184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f585d74d0 msgr2=0x555f585e0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.532+0000 7fe51ad9d640 1 --2- 172.21.15.43:0/2762174184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f585d74d0 0x555f585e0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.532+0000 7fe51ad9d640 1 -- 172.21.15.43:0/2762174184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f585d7e80 msgr2=0x555f585dd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.532+0000 7fe51ad9d640 1 --2- 172.21.15.43:0/2762174184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f585d7e80 0x555f585dd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.532+0000 7fe51ad9d640 1 -- 172.21.15.43:0/2762174184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f5824e3f0 con 0x555f585e0a00 2024-04-21T08:47:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.532+0000 7fe51ad9d640 1 --2- 172.21.15.43:0/2762174184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585e0a00 0x555f585ddcf0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fe51402c630 tx=0x7fe514032a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4874636e80879a5c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.532+0000 7fe501ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.533+0000 7fe501ffb640 1 -- 172.21.15.43:0/2762174184 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514039070 con 0x555f585e0a00 2024-04-21T08:47:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.533+0000 7fe501ffb640 1 -- 172.21.15.43:0/2762174184 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fe5140047e0 con 0x555f585e0a00 2024-04-21T08:47:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.533+0000 7fe5235ca740 1 -- 172.21.15.43:0/2762174184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe514003e20 con 0x555f585e0a00 2024-04-21T08:47:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.533+0000 7fe501ffb640 1 -- 172.21.15.43:0/2762174184 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe514004250 con 0x555f585e0a00 2024-04-21T08:47:47.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.533+0000 7fe5235ca740 10 client.19572 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:47.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.533+0000 7fe5235ca740 1 -- 172.21.15.43:0/2762174184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555f583187d0 con 0x555f585e0a00 2024-04-21T08:47:47.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.533+0000 7fe501ffb640 1 -- 172.21.15.43:0/2762174184 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe514003e20 con 0x555f585e0a00 2024-04-21T08:47:47.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.533+0000 7fe501ffb640 1 -- 172.21.15.43:0/2762174184 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe51403f040 con 0x555f585e0a00 2024-04-21T08:47:47.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.533+0000 7fe5235ca740 10 client.19572 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:47.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.533+0000 7fe5235ca740 10 client.19572 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:47:47.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.533+0000 7fe5235ca740 20 client.19572 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:47.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.533+0000 7fe5235ca740 1 --2- 172.21.15.43:0/2762174184 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x555f585f0ae0 0x555f585f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.533+0000 7fe5235ca740 4 client.19572 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:47.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.533+0000 7fe5235ca740 1 -- 172.21.15.43:0/2762174184 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f584db930 con 0x555f585f0ae0 2024-04-21T08:47:47.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.533+0000 7fe51a59c640 1 --2- 172.21.15.43:0/2762174184 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x555f585f0ae0 0x555f585f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.534+0000 7fe51a59c640 1 --2- 172.21.15.43:0/2762174184 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x555f585f0ae0 0x555f585f2ed0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4aa724b81c1c185c server_cookie=9463047a8fec5ea9 in_seq=0 out_seq=0 2024-04-21T08:47:47.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.534+0000 7fe501ffb640 10 client.19572 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:47:47.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.535+0000 7fe501ffb640 1 -- 172.21.15.43:0/2762174184 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f584db930 con 0x555f585f0ae0 2024-04-21T08:47:47.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.535+0000 7fe501ffb640 10 client.19572 handle_command_reply: tid=0 2024-04-21T08:47:47.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.535+0000 7fe5235ca740 1 client.19572 shutdown 2024-04-21T08:47:47.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.535+0000 7fe5235ca740 1 -- 172.21.15.43:0/2762174184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585e0a00 msgr2=0x555f585ddcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.535+0000 7fe5235ca740 1 --2- 172.21.15.43:0/2762174184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585e0a00 0x555f585ddcf0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fe51402c630 tx=0x7fe514032a50 comp rx=0 tx=0).stop 2024-04-21T08:47:47.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.535+0000 7fe5235ca740 1 -- 172.21.15.43:0/2762174184 shutdown_connections 2024-04-21T08:47:47.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.535+0000 7fe5235ca740 1 --2- 172.21.15.43:0/2762174184 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x555f585f0ae0 0x555f585f2ed0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.535+0000 7fe5235ca740 1 --2- 172.21.15.43:0/2762174184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585e0a00 0x555f585ddcf0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.535+0000 7fe501ffb640 0 client.19572 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:47:47.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.535+0000 7fe5235ca740 1 --2- 172.21.15.43:0/2762174184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f585d7e80 0x555f585dd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.535+0000 7fe5235ca740 1 --2- 172.21.15.43:0/2762174184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f585d74d0 0x555f585e0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.535+0000 7fe5235ca740 1 -- 172.21.15.43:0/2762174184 >> 172.21.15.43:0/2762174184 conn(0x555f585d1ba0 msgr2=0x555f585d38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.535+0000 7fe5235ca740 1 -- 172.21.15.43:0/2762174184 shutdown_connections 2024-04-21T08:47:47.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.535+0000 7fe5235ca740 1 -- 172.21.15.43:0/2762174184 wait complete. 2024-04-21T08:47:47.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.535+0000 7fe5235ca740 20 client.19572 trim_cache size 0 max 16384 2024-04-21T08:47:47.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.548+0000 7fe5235ca740 1 Processor -- start 2024-04-21T08:47:47.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.548+0000 7fe5235ca740 1 -- start start 2024-04-21T08:47:47.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.548+0000 7fe5235ca740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f585d74d0 0x555f585d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.548+0000 7fe5235ca740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f585d7e80 0x555f585f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.548+0000 7fe5235ca740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585e0a00 0x555f585f6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.548+0000 7fe5235ca740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1af4d0 con 0x555f585e0a00 2024-04-21T08:47:47.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.548+0000 7fe5235ca740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1b1010 con 0x555f585d74d0 2024-04-21T08:47:47.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.548+0000 7fe5235ca740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1afba0 con 0x555f585d7e80 2024-04-21T08:47:47.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.548+0000 7fe51ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585e0a00 0x555f585f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.548+0000 7fe51ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585e0a00 0x555f585f6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57484/0 (socket says 172.21.15.43:57484) 2024-04-21T08:47:47.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.548+0000 7fe51ad9d640 1 -- 172.21.15.43:0/1511978330 learned_addr learned my addr 172.21.15.43:0/1511978330 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.549+0000 7fe51a59c640 1 --2- 172.21.15.43:0/1511978330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f585d74d0 0x555f585d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.549+0000 7fe511d9b640 1 --2- 172.21.15.43:0/1511978330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f585d7e80 0x555f585f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.549+0000 7fe51ad9d640 1 -- 172.21.15.43:0/1511978330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f585d7e80 msgr2=0x555f585f5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.549+0000 7fe51ad9d640 1 --2- 172.21.15.43:0/1511978330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f585d7e80 0x555f585f5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.549+0000 7fe51ad9d640 1 -- 172.21.15.43:0/1511978330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f585d74d0 msgr2=0x555f585d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.549+0000 7fe51ad9d640 1 --2- 172.21.15.43:0/1511978330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f585d74d0 0x555f585d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.549+0000 7fe51ad9d640 1 -- 172.21.15.43:0/1511978330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f585d24e0 con 0x555f585e0a00 2024-04-21T08:47:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.549+0000 7fe51ad9d640 1 --2- 172.21.15.43:0/1511978330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585e0a00 0x555f585f6430 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fe514009ba0 tx=0x7fe514032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1bd3ed697f803d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.549+0000 7fe503fff640 1 -- 172.21.15.43:0/1511978330 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514039040 con 0x555f585e0a00 2024-04-21T08:47:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.549+0000 7fe503fff640 1 -- 172.21.15.43:0/1511978330 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe514003e20 con 0x555f585e0a00 2024-04-21T08:47:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.549+0000 7fe503fff640 1 -- 172.21.15.43:0/1511978330 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514034570 con 0x555f585e0a00 2024-04-21T08:47:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.550+0000 7fe5235ca740 1 -- 172.21.15.43:0/1511978330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585e0a00 msgr2=0x555f585f6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.550+0000 7fe5235ca740 1 --2- 172.21.15.43:0/1511978330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585e0a00 0x555f585f6430 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fe514009ba0 tx=0x7fe514032c40 comp rx=0 tx=0).stop 2024-04-21T08:47:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.550+0000 7fe5235ca740 1 -- 172.21.15.43:0/1511978330 shutdown_connections 2024-04-21T08:47:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.550+0000 7fe5235ca740 1 --2- 172.21.15.43:0/1511978330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585e0a00 0x555f585f6430 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.550+0000 7fe5235ca740 1 --2- 172.21.15.43:0/1511978330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f585d7e80 0x555f585f5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.550+0000 7fe5235ca740 1 --2- 172.21.15.43:0/1511978330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f585d74d0 0x555f585d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.550+0000 7fe5235ca740 1 -- 172.21.15.43:0/1511978330 >> 172.21.15.43:0/1511978330 conn(0x555f585d1ba0 msgr2=0x555f585d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.550+0000 7fe5235ca740 1 -- 172.21.15.43:0/1511978330 shutdown_connections 2024-04-21T08:47:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.550+0000 7fe5235ca740 1 -- 172.21.15.43:0/1511978330 wait complete. 2024-04-21T08:47:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.550+0000 7fe5235ca740 1 -- start start 2024-04-21T08:47:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.550+0000 7fe5235ca740 1 Processor -- start 2024-04-21T08:47:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.550+0000 7fe5235ca740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585d74d0 0x555f585f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.550+0000 7fe5235ca740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f585d7e80 0x555f585f3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.550+0000 7fe5235ca740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f585e0a00 0x555f585f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.550+0000 7fe5235ca740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1b0940 con 0x555f585d74d0 2024-04-21T08:47:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.550+0000 7fe5235ca740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1b0270 con 0x555f585d7e80 2024-04-21T08:47:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.550+0000 7fe5235ca740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1ad4d0 con 0x555f585e0a00 2024-04-21T08:47:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.550+0000 7fe5235ca740 10 client.0 _ll_register_callbacks cb 0x7fe514005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.550+0000 7fe51a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585d74d0 0x555f585f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.550+0000 7fe51a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585d74d0 0x555f585f61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57494/0 (socket says 172.21.15.43:57494) 2024-04-21T08:47:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.550+0000 7fe51a59c640 1 -- 172.21.15.43:0/3512220367 learned_addr learned my addr 172.21.15.43:0/3512220367 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.551+0000 7fe511d9b640 1 --2- 172.21.15.43:0/3512220367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f585d7e80 0x555f585f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.551+0000 7fe51a59c640 1 -- 172.21.15.43:0/3512220367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f585e0a00 msgr2=0x555f585f3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.551+0000 7fe51a59c640 1 --2- 172.21.15.43:0/3512220367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f585e0a00 0x555f585f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.551+0000 7fe51a59c640 1 -- 172.21.15.43:0/3512220367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f585d7e80 msgr2=0x555f585f3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.551+0000 7fe51a59c640 1 --2- 172.21.15.43:0/3512220367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f585d7e80 0x555f585f3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.551+0000 7fe51a59c640 1 -- 172.21.15.43:0/3512220367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f585f4290 con 0x555f585d74d0 2024-04-21T08:47:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.551+0000 7fe51a59c640 1 --2- 172.21.15.43:0/3512220367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585d74d0 0x555f585f61c0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x555f585d6fa0 tx=0x7fe50c01e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa96835774d3abf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.551+0000 7fe501ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.551+0000 7fe501ffb640 1 -- 172.21.15.43:0/3512220367 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c014ca0 con 0x555f585d74d0 2024-04-21T08:47:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.551+0000 7fe501ffb640 1 -- 172.21.15.43:0/3512220367 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fe50c014e40 con 0x555f585d74d0 2024-04-21T08:47:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.551+0000 7fe5235ca740 1 -- 172.21.15.43:0/3512220367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe514003e20 con 0x555f585d74d0 2024-04-21T08:47:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.551+0000 7fe501ffb640 1 -- 172.21.15.43:0/3512220367 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe50c0047a0 con 0x555f585d74d0 2024-04-21T08:47:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.551+0000 7fe5235ca740 10 client.19578 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.551+0000 7fe5235ca740 1 -- 172.21.15.43:0/3512220367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555f58656c90 con 0x555f585d74d0 2024-04-21T08:47:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.551+0000 7fe501ffb640 1 -- 172.21.15.43:0/3512220367 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe514003e20 con 0x555f585d74d0 2024-04-21T08:47:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.552+0000 7fe501ffb640 1 -- 172.21.15.43:0/3512220367 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe50c025070 con 0x555f585d74d0 2024-04-21T08:47:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.552+0000 7fe5235ca740 10 client.19578 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.552+0000 7fe5235ca740 10 client.19578 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:47:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.552+0000 7fe5235ca740 20 client.19578 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.552+0000 7fe5235ca740 1 --2- 172.21.15.43:0/3512220367 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x555f58657460 0x555f586598a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.552+0000 7fe5235ca740 4 client.19578 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.552+0000 7fe5235ca740 1 -- 172.21.15.43:0/3512220367 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f58549c10 con 0x555f58657460 2024-04-21T08:47:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.552+0000 7fe511d9b640 1 --2- 172.21.15.43:0/3512220367 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x555f58657460 0x555f586598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.552+0000 7fe511d9b640 1 --2- 172.21.15.43:0/3512220367 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x555f58657460 0x555f586598a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e52d48d6eb60d05d server_cookie=7524a81b1ea02935 in_seq=0 out_seq=0 2024-04-21T08:47:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.552+0000 7fe501ffb640 10 client.19578 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:47:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.552+0000 7fe501ffb640 1 -- 172.21.15.43:0/3512220367 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f58549c10 con 0x555f58657460 2024-04-21T08:47:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.552+0000 7fe501ffb640 10 client.19578 handle_command_reply: tid=0 2024-04-21T08:47:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.552+0000 7fe5235ca740 1 client.19578 shutdown 2024-04-21T08:47:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.553+0000 7fe5235ca740 1 -- 172.21.15.43:0/3512220367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585d74d0 msgr2=0x555f585f61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.553+0000 7fe5235ca740 1 --2- 172.21.15.43:0/3512220367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585d74d0 0x555f585f61c0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x555f585d6fa0 tx=0x7fe50c01e9e0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.553+0000 7fe5235ca740 1 -- 172.21.15.43:0/3512220367 shutdown_connections 2024-04-21T08:47:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.553+0000 7fe5235ca740 1 --2- 172.21.15.43:0/3512220367 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x555f58657460 0x555f586598a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.553+0000 7fe5235ca740 1 --2- 172.21.15.43:0/3512220367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f585e0a00 0x555f585f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.553+0000 7fe5235ca740 1 --2- 172.21.15.43:0/3512220367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f585d7e80 0x555f585f3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.553+0000 7fe501ffb640 0 client.19578 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:47:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.553+0000 7fe5235ca740 1 --2- 172.21.15.43:0/3512220367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f585d74d0 0x555f585f61c0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.553+0000 7fe5235ca740 1 -- 172.21.15.43:0/3512220367 >> 172.21.15.43:0/3512220367 conn(0x555f585d1ba0 msgr2=0x555f585d37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.553+0000 7fe5235ca740 1 -- 172.21.15.43:0/3512220367 shutdown_connections 2024-04-21T08:47:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.553+0000 7fe5235ca740 1 -- 172.21.15.43:0/3512220367 wait complete. 2024-04-21T08:47:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.553+0000 7fe5235ca740 20 client.19578 trim_cache size 0 max 16384 2024-04-21T08:47:47.565 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:47:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.553+0000 7fe5209d5640 1 -- 172.21.15.43:0/2604114953 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe5080a3d20 msgr2=0x7fe5080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.553+0000 7fe5209d5640 1 --2- 172.21.15.43:0/2604114953 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe5080a3d20 0x7fe5080a61e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe51404c000 tx=0x7fe514045000 comp rx=0 tx=0).stop 2024-04-21T08:47:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.553+0000 7fe5209d5640 1 -- 172.21.15.43:0/2604114953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe51c107260 msgr2=0x7fe51c198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.553+0000 7fe5209d5640 1 --2- 172.21.15.43:0/2604114953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe51c107260 0x7fe51c198c70 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fe50400d770 tx=0x7fe50400dc30 comp rx=0 tx=0).stop 2024-04-21T08:47:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.553+0000 7fe5209d5640 1 -- 172.21.15.43:0/2604114953 shutdown_connections 2024-04-21T08:47:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.553+0000 7fe5209d5640 1 --2- 172.21.15.43:0/2604114953 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe5080a3d20 0x7fe5080a61e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.553+0000 7fe5209d5640 1 --2- 172.21.15.43:0/2604114953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe51c19cff0 0x7fe51c19d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.553+0000 7fe5209d5640 1 --2- 172.21.15.43:0/2604114953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe51c107260 0x7fe51c198c70 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.553+0000 7fe5209d5640 1 --2- 172.21.15.43:0/2604114953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe51c102190 0x7fe51c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.553+0000 7fe5209d5640 1 -- 172.21.15.43:0/2604114953 >> 172.21.15.43:0/2604114953 conn(0x7fe51c0fd990 msgr2=0x7fe51c0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.553+0000 7fe5209d5640 1 -- 172.21.15.43:0/2604114953 shutdown_connections 2024-04-21T08:47:47.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.553+0000 7fe5209d5640 1 -- 172.21.15.43:0/2604114953 wait complete. 2024-04-21T08:47:47.568 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:47:47.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.671+0000 7fa729940640 1 -- 172.21.15.43:0/1137434843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724101090 msgr2=0x7fa724101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.671+0000 7fa729940640 1 --2- 172.21.15.43:0/1137434843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724101090 0x7fa724101470 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fa714009900 tx=0x7fa71402bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:47.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.671+0000 7fa729940640 1 -- 172.21.15.43:0/1137434843 shutdown_connections 2024-04-21T08:47:47.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.671+0000 7fa729940640 1 --2- 172.21.15.43:0/1137434843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa724106160 0x7fa72410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.671+0000 7fa729940640 1 --2- 172.21.15.43:0/1137434843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa724101a40 0x7fa724105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.671+0000 7fa729940640 1 --2- 172.21.15.43:0/1137434843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724101090 0x7fa724101470 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.671+0000 7fa729940640 1 -- 172.21.15.43:0/1137434843 >> 172.21.15.43:0/1137434843 conn(0x7fa724077f50 msgr2=0x7fa724078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.671+0000 7fa729940640 1 -- 172.21.15.43:0/1137434843 shutdown_connections 2024-04-21T08:47:47.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.672+0000 7fa729940640 1 -- 172.21.15.43:0/1137434843 wait complete. 2024-04-21T08:47:47.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.672+0000 7fa729940640 1 Processor -- start 2024-04-21T08:47:47.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.672+0000 7fa729940640 1 -- start start 2024-04-21T08:47:47.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.672+0000 7fa729940640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724101090 0x7fa7241972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.672+0000 7fa729940640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa724106160 0x7fa724197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.672+0000 7fa729940640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa724197d70 0x7fa7241a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.672+0000 7fa729940640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241110a0 con 0x7fa724101090 2024-04-21T08:47:47.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.672+0000 7fa729940640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa724110f20 con 0x7fa724197d70 2024-04-21T08:47:47.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.672+0000 7fa729940640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa724111220 con 0x7fa724106160 2024-04-21T08:47:47.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.673+0000 7fa722ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724101090 0x7fa7241972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.673+0000 7fa722ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724101090 0x7fa7241972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57516/0 (socket says 172.21.15.43:57516) 2024-04-21T08:47:47.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.673+0000 7fa722ffd640 1 -- 172.21.15.43:0/2817678438 learned_addr learned my addr 172.21.15.43:0/2817678438 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.673+0000 7fa71a7fc640 1 --2- 172.21.15.43:0/2817678438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa724106160 0x7fa724197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.673+0000 7fa722ffd640 1 -- 172.21.15.43:0/2817678438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa724106160 msgr2=0x7fa724197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.673+0000 7fa7237fe640 1 --2- 172.21.15.43:0/2817678438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa724197d70 0x7fa7241a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.673+0000 7fa722ffd640 1 --2- 172.21.15.43:0/2817678438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa724106160 0x7fa724197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.673+0000 7fa722ffd640 1 -- 172.21.15.43:0/2817678438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa724197d70 msgr2=0x7fa7241a6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.673+0000 7fa722ffd640 1 --2- 172.21.15.43:0/2817678438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa724197d70 0x7fa7241a6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.673+0000 7fa722ffd640 1 -- 172.21.15.43:0/2817678438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7241a6f00 con 0x7fa724101090 2024-04-21T08:47:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.673+0000 7fa71a7fc640 1 --2- 172.21.15.43:0/2817678438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa724106160 0x7fa724197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.673+0000 7fa7237fe640 1 --2- 172.21.15.43:0/2817678438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa724197d70 0x7fa7241a6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.673+0000 7fa722ffd640 1 --2- 172.21.15.43:0/2817678438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724101090 0x7fa7241972f0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fa714005940 tx=0x7fa71402c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.673+0000 7fa72893e640 1 -- 172.21.15.43:0/2817678438 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714043020 con 0x7fa724101090 2024-04-21T08:47:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.673+0000 7fa72893e640 1 -- 172.21.15.43:0/2817678438 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa71402cc60 con 0x7fa724101090 2024-04-21T08:47:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.673+0000 7fa729940640 1 -- 172.21.15.43:0/2817678438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7241a7190 con 0x7fa724101090 2024-04-21T08:47:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.673+0000 7fa72893e640 1 -- 172.21.15.43:0/2817678438 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714003c30 con 0x7fa724101090 2024-04-21T08:47:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.673+0000 7fa729940640 1 -- 172.21.15.43:0/2817678438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7241a76b0 con 0x7fa724101090 2024-04-21T08:47:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.675+0000 7fa72893e640 1 -- 172.21.15.43:0/2817678438 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa714046020 con 0x7fa724101090 2024-04-21T08:47:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.676+0000 7fa72c60d740 1 Processor -- start 2024-04-21T08:47:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.676+0000 7fa72c60d740 1 -- start start 2024-04-21T08:47:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.676+0000 7fa72c60d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e70a09a4d0 0x55e70a09a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.676+0000 7fa72c60d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e70a09ae80 0x55e70a0a3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.676+0000 7fa72c60d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a0a3a00 0x55e70a0a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.676+0000 7fa72c60d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e709fff1f0 con 0x55e70a0a3a00 2024-04-21T08:47:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.676+0000 7fa72c60d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e709f9c4b0 con 0x55e70a09a4d0 2024-04-21T08:47:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.676+0000 7fa72c60d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e709ffb1b0 con 0x55e70a09ae80 2024-04-21T08:47:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.676+0000 7fa72893e640 1 --2- 172.21.15.43:0/2817678438 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa7100a3d20 0x7fa7100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.676+0000 7fa71a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a0a3a00 0x55e70a0a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.676+0000 7fa71a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a0a3a00 0x55e70a0a5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57532/0 (socket says 172.21.15.43:57532) 2024-04-21T08:47:47.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.676+0000 7fa71a7fc640 1 -- 172.21.15.43:0/3971816681 learned_addr learned my addr 172.21.15.43:0/3971816681 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.676+0000 7fa7237fe640 1 --2- 172.21.15.43:0/2817678438 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa7100a3d20 0x7fa7100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.676+0000 7fa7237fe640 1 --2- 172.21.15.43:0/3971816681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e70a09a4d0 0x55e70a09a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.676+0000 7fa72893e640 1 -- 172.21.15.43:0/2817678438 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fa7140f2ff0 con 0x7fa724101090 2024-04-21T08:47:47.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.676+0000 7fa722ffd640 1 --2- 172.21.15.43:0/3971816681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e70a09ae80 0x55e70a0a3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.677+0000 7fa71a7fc640 1 -- 172.21.15.43:0/3971816681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e70a09ae80 msgr2=0x55e70a0a3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.677+0000 7fa71a7fc640 1 --2- 172.21.15.43:0/3971816681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e70a09ae80 0x55e70a0a3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.677+0000 7fa71a7fc640 1 -- 172.21.15.43:0/3971816681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e70a09a4d0 msgr2=0x55e70a09a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.677+0000 7fa71a7fc640 1 --2- 172.21.15.43:0/3971816681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e70a09a4d0 0x55e70a09a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.677+0000 7fa71a7fc640 1 -- 172.21.15.43:0/3971816681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e709f8e1b0 con 0x55e70a0a3a00 2024-04-21T08:47:47.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.677+0000 7fa722ffd640 1 --2- 172.21.15.43:0/3971816681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e70a09ae80 0x55e70a0a3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:47.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.677+0000 7fa71a7fc640 1 --2- 172.21.15.43:0/3971816681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a0a3a00 0x55e70a0a5e50 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fa70c008c00 tx=0x7fa70c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5aecfa658ccaf1f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.677+0000 7fa7237fe640 1 --2- 172.21.15.43:0/2817678438 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa7100a3d20 0x7fa7100a61e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00afd0 tx=0x7fa71c00a250 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.677+0000 7fa719ffb640 1 -- 172.21.15.43:0/3971816681 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c014070 con 0x55e70a0a3a00 2024-04-21T08:47:47.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.677+0000 7fa719ffb640 1 -- 172.21.15.43:0/3971816681 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa70c0047a0 con 0x55e70a0a3a00 2024-04-21T08:47:47.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.677+0000 7fa719ffb640 1 -- 172.21.15.43:0/3971816681 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c010690 con 0x55e70a0a3a00 2024-04-21T08:47:47.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.678+0000 7fa72c60d740 1 -- 172.21.15.43:0/3971816681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a0a3a00 msgr2=0x55e70a0a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.678+0000 7fa72c60d740 1 --2- 172.21.15.43:0/3971816681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a0a3a00 0x55e70a0a5e50 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fa70c008c00 tx=0x7fa70c00e720 comp rx=0 tx=0).stop 2024-04-21T08:47:47.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.678+0000 7fa72c60d740 1 -- 172.21.15.43:0/3971816681 shutdown_connections 2024-04-21T08:47:47.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.678+0000 7fa72c60d740 1 --2- 172.21.15.43:0/3971816681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a0a3a00 0x55e70a0a5e50 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.678+0000 7fa72c60d740 1 --2- 172.21.15.43:0/3971816681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e70a09ae80 0x55e70a0a3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.678+0000 7fa72c60d740 1 --2- 172.21.15.43:0/3971816681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e70a09a4d0 0x55e70a09a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.678+0000 7fa72c60d740 1 -- 172.21.15.43:0/3971816681 >> 172.21.15.43:0/3971816681 conn(0x55e70a094ba0 msgr2=0x55e70a094f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.678+0000 7fa72c60d740 1 -- 172.21.15.43:0/3971816681 shutdown_connections 2024-04-21T08:47:47.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.678+0000 7fa72c60d740 1 -- 172.21.15.43:0/3971816681 wait complete. 2024-04-21T08:47:47.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.678+0000 7fa72c60d740 1 -- start start 2024-04-21T08:47:47.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.678+0000 7fa72c60d740 1 Processor -- start 2024-04-21T08:47:47.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.678+0000 7fa72c60d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a09a4d0 0x55e70a0a32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.678+0000 7fa72c60d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e70a09ae80 0x55e70a0a0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.678+0000 7fa72c60d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e70a0a3a00 0x55e70a0a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.678+0000 7fa72c60d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e70a0a6970 con 0x55e70a09a4d0 2024-04-21T08:47:47.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.678+0000 7fa72c60d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e709fb8500 con 0x55e70a0a3a00 2024-04-21T08:47:47.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.678+0000 7fa72c60d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e70a0a6af0 con 0x55e70a09ae80 2024-04-21T08:47:47.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.679+0000 7fa72c60d740 10 client.0 _ll_register_callbacks cb 0x55e709f86b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:47.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.679+0000 7fa722ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a09a4d0 0x55e70a0a32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.679+0000 7fa722ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a09a4d0 0x55e70a0a32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57534/0 (socket says 172.21.15.43:57534) 2024-04-21T08:47:47.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.679+0000 7fa722ffd640 1 -- 172.21.15.43:0/1315403911 learned_addr learned my addr 172.21.15.43:0/1315403911 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.679+0000 7fa71a7fc640 1 --2- 172.21.15.43:0/1315403911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e70a09ae80 0x55e70a0a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.679+0000 7fa7237fe640 1 --2- 172.21.15.43:0/1315403911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e70a0a3a00 0x55e70a0a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.679+0000 7fa722ffd640 1 -- 172.21.15.43:0/1315403911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e70a09ae80 msgr2=0x55e70a0a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.679+0000 7fa722ffd640 1 --2- 172.21.15.43:0/1315403911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e70a09ae80 0x55e70a0a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.679+0000 7fa722ffd640 1 -- 172.21.15.43:0/1315403911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e70a0a3a00 msgr2=0x55e70a0a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.679+0000 7fa722ffd640 1 --2- 172.21.15.43:0/1315403911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e70a0a3a00 0x55e70a0a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.679+0000 7fa722ffd640 1 -- 172.21.15.43:0/1315403911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e709d113f0 con 0x55e70a09a4d0 2024-04-21T08:47:47.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.679+0000 7fa7237fe640 1 --2- 172.21.15.43:0/1315403911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e70a0a3a00 0x55e70a0a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:47.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.679+0000 7fa722ffd640 1 --2- 172.21.15.43:0/1315403911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a09a4d0 0x55e70a0a32e0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fa7140bb080 tx=0x7fa7140be640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa6c33b8a1f5278 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.679+0000 7fa702ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:47.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.679+0000 7fa702ffd640 1 -- 172.21.15.43:0/1315403911 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7140c2fd0 con 0x55e70a09a4d0 2024-04-21T08:47:47.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.679+0000 7fa702ffd640 1 -- 172.21.15.43:0/1315403911 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fa7140f0540 con 0x55e70a09a4d0 2024-04-21T08:47:47.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.679+0000 7fa72c60d740 1 -- 172.21.15.43:0/1315403911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa70c0047a0 con 0x55e70a09a4d0 2024-04-21T08:47:47.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.679+0000 7fa702ffd640 1 -- 172.21.15.43:0/1315403911 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa7140f0740 con 0x55e70a09a4d0 2024-04-21T08:47:47.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.679+0000 7fa72c60d740 10 client.19590 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:47.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.679+0000 7fa72c60d740 1 -- 172.21.15.43:0/1315403911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e709ddb7d0 con 0x55e70a09a4d0 2024-04-21T08:47:47.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.680+0000 7fa702ffd640 1 -- 172.21.15.43:0/1315403911 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa70c0047a0 con 0x55e70a09a4d0 2024-04-21T08:47:47.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.680+0000 7fa702ffd640 1 -- 172.21.15.43:0/1315403911 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa7140c7400 con 0x55e70a09a4d0 2024-04-21T08:47:47.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.680+0000 7fa72c60d740 10 client.19590 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:47.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.680+0000 7fa72c60d740 10 client.19590 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:47:47.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.680+0000 7fa72c60d740 20 client.19590 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:47.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.680+0000 7fa72c60d740 1 --2- 172.21.15.43:0/1315403911 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55e70a0b3bc0 0x55e70a0b6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.680+0000 7fa72c60d740 4 client.19590 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:47.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.680+0000 7fa72c60d740 1 -- 172.21.15.43:0/1315403911 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e709f9e930 con 0x55e70a0b3bc0 2024-04-21T08:47:47.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.680+0000 7fa71a7fc640 1 --2- 172.21.15.43:0/1315403911 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55e70a0b3bc0 0x55e70a0b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.681+0000 7fa702ffd640 10 client.19590 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:47:47.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.681+0000 7fa71a7fc640 1 --2- 172.21.15.43:0/1315403911 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55e70a0b3bc0 0x55e70a0b6010 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=946b1a7e3586c7da server_cookie=dd457d0b85167b48 in_seq=0 out_seq=0 2024-04-21T08:47:47.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.681+0000 7fa702ffd640 1 -- 172.21.15.43:0/1315403911 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e709f9e930 con 0x55e70a0b3bc0 2024-04-21T08:47:47.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.681+0000 7fa702ffd640 10 client.19590 handle_command_reply: tid=0 2024-04-21T08:47:47.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.682+0000 7fa72c60d740 1 client.19590 shutdown 2024-04-21T08:47:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.682+0000 7fa72c60d740 1 -- 172.21.15.43:0/1315403911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a09a4d0 msgr2=0x55e70a0a32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.682+0000 7fa72c60d740 1 --2- 172.21.15.43:0/1315403911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a09a4d0 0x55e70a0a32e0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fa7140bb080 tx=0x7fa7140be640 comp rx=0 tx=0).stop 2024-04-21T08:47:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.682+0000 7fa72c60d740 1 -- 172.21.15.43:0/1315403911 shutdown_connections 2024-04-21T08:47:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.682+0000 7fa72c60d740 1 --2- 172.21.15.43:0/1315403911 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55e70a0b3bc0 0x55e70a0b6010 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.682+0000 7fa72c60d740 1 --2- 172.21.15.43:0/1315403911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e70a0a3a00 0x55e70a0a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.682+0000 7fa72c60d740 1 --2- 172.21.15.43:0/1315403911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e70a09ae80 0x55e70a0a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.682+0000 7fa702ffd640 0 client.19590 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:47:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.682+0000 7fa72c60d740 1 --2- 172.21.15.43:0/1315403911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a09a4d0 0x55e70a0a32e0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.682+0000 7fa72c60d740 1 -- 172.21.15.43:0/1315403911 >> 172.21.15.43:0/1315403911 conn(0x55e70a094ba0 msgr2=0x55e70a0968f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.682+0000 7fa72c60d740 1 -- 172.21.15.43:0/1315403911 shutdown_connections 2024-04-21T08:47:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.682+0000 7fa72c60d740 1 -- 172.21.15.43:0/1315403911 wait complete. 2024-04-21T08:47:47.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.682+0000 7fa72c60d740 20 client.19590 trim_cache size 0 max 16384 2024-04-21T08:47:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.695+0000 7fa72c60d740 1 Processor -- start 2024-04-21T08:47:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.695+0000 7fa72c60d740 1 -- start start 2024-04-21T08:47:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.695+0000 7fa72c60d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e70a09a4d0 0x55e70a0af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.695+0000 7fa72c60d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e70a09ae80 0x55e70a093a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.695+0000 7fa72c60d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a0a3a00 0x55e70a0a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.695+0000 7fa72c60d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241b1600 con 0x55e70a0a3a00 2024-04-21T08:47:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.695+0000 7fa72c60d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241b3140 con 0x55e70a09a4d0 2024-04-21T08:47:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.696+0000 7fa72c60d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241b1cd0 con 0x55e70a09ae80 2024-04-21T08:47:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.696+0000 7fa7237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a0a3a00 0x55e70a0a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.696+0000 7fa7237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a0a3a00 0x55e70a0a2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57536/0 (socket says 172.21.15.43:57536) 2024-04-21T08:47:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.696+0000 7fa7237fe640 1 -- 172.21.15.43:0/3074570795 learned_addr learned my addr 172.21.15.43:0/3074570795 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.696+0000 7fa722ffd640 1 --2- 172.21.15.43:0/3074570795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e70a09a4d0 0x55e70a0af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.696+0000 7fa71a7fc640 1 --2- 172.21.15.43:0/3074570795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e70a09ae80 0x55e70a093a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.696+0000 7fa7237fe640 1 -- 172.21.15.43:0/3074570795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e70a09ae80 msgr2=0x55e70a093a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.696+0000 7fa7237fe640 1 --2- 172.21.15.43:0/3074570795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e70a09ae80 0x55e70a093a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.696+0000 7fa7237fe640 1 -- 172.21.15.43:0/3074570795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e70a09a4d0 msgr2=0x55e70a0af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.696+0000 7fa7237fe640 1 --2- 172.21.15.43:0/3074570795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e70a09a4d0 0x55e70a0af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.696+0000 7fa7237fe640 1 -- 172.21.15.43:0/3074570795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e70a09ac20 con 0x55e70a0a3a00 2024-04-21T08:47:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.696+0000 7fa7237fe640 1 --2- 172.21.15.43:0/3074570795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a0a3a00 0x55e70a0a2da0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00af50 tx=0x7fa71c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae044bc0800426d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.696+0000 7fa7197fa640 1 -- 172.21.15.43:0/3074570795 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71c023070 con 0x55e70a0a3a00 2024-04-21T08:47:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.696+0000 7fa7197fa640 1 -- 172.21.15.43:0/3074570795 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa71c012230 con 0x55e70a0a3a00 2024-04-21T08:47:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.696+0000 7fa7197fa640 1 -- 172.21.15.43:0/3074570795 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71c01f8d0 con 0x55e70a0a3a00 2024-04-21T08:47:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.697+0000 7fa72c60d740 1 -- 172.21.15.43:0/3074570795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a0a3a00 msgr2=0x55e70a0a2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.697+0000 7fa72c60d740 1 --2- 172.21.15.43:0/3074570795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a0a3a00 0x55e70a0a2da0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00af50 tx=0x7fa71c005f90 comp rx=0 tx=0).stop 2024-04-21T08:47:47.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.697+0000 7fa72c60d740 1 -- 172.21.15.43:0/3074570795 shutdown_connections 2024-04-21T08:47:47.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.697+0000 7fa72c60d740 1 --2- 172.21.15.43:0/3074570795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a0a3a00 0x55e70a0a2da0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.697+0000 7fa72c60d740 1 --2- 172.21.15.43:0/3074570795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e70a09ae80 0x55e70a093a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.697+0000 7fa72c60d740 1 --2- 172.21.15.43:0/3074570795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e70a09a4d0 0x55e70a0af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.697+0000 7fa72c60d740 1 -- 172.21.15.43:0/3074570795 >> 172.21.15.43:0/3074570795 conn(0x55e70a094ba0 msgr2=0x55e70a093370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.697+0000 7fa72c60d740 1 -- 172.21.15.43:0/3074570795 shutdown_connections 2024-04-21T08:47:47.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.697+0000 7fa72c60d740 1 -- 172.21.15.43:0/3074570795 wait complete. 2024-04-21T08:47:47.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.697+0000 7fa72c60d740 1 -- start start 2024-04-21T08:47:47.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.698+0000 7fa72c60d740 1 Processor -- start 2024-04-21T08:47:47.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.698+0000 7fa72c60d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e70a09a4d0 0x55e70a09d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.698+0000 7fa72c60d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e70a09ae80 0x55e70a09dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.698+0000 7fa72c60d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a0a3a00 0x55e70a119160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.698+0000 7fa72c60d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241b2a70 con 0x55e70a0a3a00 2024-04-21T08:47:47.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.698+0000 7fa72c60d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241b23a0 con 0x55e70a09ae80 2024-04-21T08:47:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.698+0000 7fa72c60d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241af600 con 0x55e70a09a4d0 2024-04-21T08:47:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.698+0000 7fa72c60d740 10 client.0 _ll_register_callbacks cb 0x7fa7140be9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.698+0000 7fa7237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a0a3a00 0x55e70a119160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.698+0000 7fa7237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a0a3a00 0x55e70a119160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57542/0 (socket says 172.21.15.43:57542) 2024-04-21T08:47:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.698+0000 7fa7237fe640 1 -- 172.21.15.43:0/1837721825 learned_addr learned my addr 172.21.15.43:0/1837721825 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.698+0000 7fa71a7fc640 1 --2- 172.21.15.43:0/1837721825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e70a09ae80 0x55e70a09dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.698+0000 7fa722ffd640 1 --2- 172.21.15.43:0/1837721825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e70a09a4d0 0x55e70a09d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.698+0000 7fa7237fe640 1 -- 172.21.15.43:0/1837721825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e70a09a4d0 msgr2=0x55e70a09d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.698+0000 7fa7237fe640 1 --2- 172.21.15.43:0/1837721825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e70a09a4d0 0x55e70a09d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.698+0000 7fa7237fe640 1 -- 172.21.15.43:0/1837721825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e70a09ae80 msgr2=0x55e70a09dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.698+0000 7fa7237fe640 1 --2- 172.21.15.43:0/1837721825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e70a09ae80 0x55e70a09dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.698+0000 7fa7237fe640 1 -- 172.21.15.43:0/1837721825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e70a1196a0 con 0x55e70a0a3a00 2024-04-21T08:47:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.698+0000 7fa722ffd640 1 --2- 172.21.15.43:0/1837721825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e70a09a4d0 0x55e70a09d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.698+0000 7fa7237fe640 1 --2- 172.21.15.43:0/1837721825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a0a3a00 0x55e70a119160 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00ff50 tx=0x7fa71c00fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed5d9a2b83067ef server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.698+0000 7fa702ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.698+0000 7fa702ffd640 1 -- 172.21.15.43:0/1837721825 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71c023040 con 0x55e70a0a3a00 2024-04-21T08:47:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.698+0000 7fa702ffd640 1 -- 172.21.15.43:0/1837721825 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fa71c01f8e0 con 0x55e70a0a3a00 2024-04-21T08:47:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.699+0000 7fa72c60d740 1 -- 172.21.15.43:0/1837721825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa71c012230 con 0x55e70a0a3a00 2024-04-21T08:47:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.699+0000 7fa702ffd640 1 -- 172.21.15.43:0/1837721825 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa71c00fd20 con 0x55e70a0a3a00 2024-04-21T08:47:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.699+0000 7fa72c60d740 10 client.19596 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.699+0000 7fa72c60d740 1 -- 172.21.15.43:0/1837721825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e70a11a900 con 0x55e70a0a3a00 2024-04-21T08:47:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.699+0000 7fa702ffd640 1 -- 172.21.15.43:0/1837721825 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa71c012230 con 0x55e70a0a3a00 2024-04-21T08:47:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.699+0000 7fa702ffd640 1 -- 172.21.15.43:0/1837721825 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa71c02f070 con 0x55e70a0a3a00 2024-04-21T08:47:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.699+0000 7fa72c60d740 10 client.19596 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.699+0000 7fa72c60d740 10 client.19596 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:47:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.699+0000 7fa72c60d740 20 client.19596 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.699+0000 7fa72c60d740 1 --2- 172.21.15.43:0/1837721825 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55e70a11afd0 0x55e70a11d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.699+0000 7fa72c60d740 4 client.19596 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.699+0000 7fa72c60d740 1 -- 172.21.15.43:0/1837721825 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e70a00cc10 con 0x55e70a11afd0 2024-04-21T08:47:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.699+0000 7fa722ffd640 1 --2- 172.21.15.43:0/1837721825 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55e70a11afd0 0x55e70a11d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.700+0000 7fa722ffd640 1 --2- 172.21.15.43:0/1837721825 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55e70a11afd0 0x55e70a11d3f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fc66f2e924138909 server_cookie=91bbaa222850f4db in_seq=0 out_seq=0 2024-04-21T08:47:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.700+0000 7fa702ffd640 10 client.19596 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:47:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.700+0000 7fa702ffd640 1 -- 172.21.15.43:0/1837721825 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e70a00cc10 con 0x55e70a11afd0 2024-04-21T08:47:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.700+0000 7fa702ffd640 10 client.19596 handle_command_reply: tid=0 2024-04-21T08:47:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.700+0000 7fa72c60d740 1 client.19596 shutdown 2024-04-21T08:47:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.700+0000 7fa72c60d740 1 -- 172.21.15.43:0/1837721825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a0a3a00 msgr2=0x55e70a119160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.700+0000 7fa72c60d740 1 --2- 172.21.15.43:0/1837721825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a0a3a00 0x55e70a119160 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00ff50 tx=0x7fa71c00fc30 comp rx=0 tx=0).stop 2024-04-21T08:47:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.700+0000 7fa72c60d740 1 -- 172.21.15.43:0/1837721825 shutdown_connections 2024-04-21T08:47:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.700+0000 7fa72c60d740 1 --2- 172.21.15.43:0/1837721825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e70a0a3a00 0x55e70a119160 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.700+0000 7fa72c60d740 1 --2- 172.21.15.43:0/1837721825 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55e70a11afd0 0x55e70a11d3f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.700+0000 7fa72c60d740 1 --2- 172.21.15.43:0/1837721825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e70a09ae80 0x55e70a09dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.700+0000 7fa72c60d740 1 --2- 172.21.15.43:0/1837721825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e70a09a4d0 0x55e70a09d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.700+0000 7fa702ffd640 0 client.19596 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:47:47.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.700+0000 7fa72c60d740 1 -- 172.21.15.43:0/1837721825 >> 172.21.15.43:0/1837721825 conn(0x55e70a094ba0 msgr2=0x55e70a09b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.700+0000 7fa72c60d740 1 -- 172.21.15.43:0/1837721825 shutdown_connections 2024-04-21T08:47:47.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.700+0000 7fa72c60d740 1 -- 172.21.15.43:0/1837721825 wait complete. 2024-04-21T08:47:47.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.700+0000 7fa72c60d740 20 client.19596 trim_cache size 0 max 16384 2024-04-21T08:47:47.709 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:47:47.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.701+0000 7fa729940640 1 -- 172.21.15.43:0/2817678438 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa7100a3d20 msgr2=0x7fa7100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.701+0000 7fa729940640 1 --2- 172.21.15.43:0/2817678438 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa7100a3d20 0x7fa7100a61e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00afd0 tx=0x7fa71c00a250 comp rx=0 tx=0).stop 2024-04-21T08:47:47.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.701+0000 7fa729940640 1 -- 172.21.15.43:0/2817678438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724101090 msgr2=0x7fa7241972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.701+0000 7fa729940640 1 --2- 172.21.15.43:0/2817678438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724101090 0x7fa7241972f0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fa714005940 tx=0x7fa71402c860 comp rx=0 tx=0).stop 2024-04-21T08:47:47.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.701+0000 7fa729940640 1 -- 172.21.15.43:0/2817678438 shutdown_connections 2024-04-21T08:47:47.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.701+0000 7fa729940640 1 --2- 172.21.15.43:0/2817678438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa724197d70 0x7fa7241a6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.701+0000 7fa729940640 1 --2- 172.21.15.43:0/2817678438 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa7100a3d20 0x7fa7100a61e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.701+0000 7fa729940640 1 --2- 172.21.15.43:0/2817678438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa724106160 0x7fa724197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.701+0000 7fa729940640 1 --2- 172.21.15.43:0/2817678438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724101090 0x7fa7241972f0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.701+0000 7fa729940640 1 -- 172.21.15.43:0/2817678438 >> 172.21.15.43:0/2817678438 conn(0x7fa724077f50 msgr2=0x7fa7240ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.701+0000 7fa729940640 1 -- 172.21.15.43:0/2817678438 shutdown_connections 2024-04-21T08:47:47.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.701+0000 7fa729940640 1 -- 172.21.15.43:0/2817678438 wait complete. 2024-04-21T08:47:47.713 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:47:47.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.816+0000 7fbdcf752640 1 -- 172.21.15.43:0/3345867188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdc806b1f0 msgr2=0x7fbdc8106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.816+0000 7fbdcf752640 1 --2- 172.21.15.43:0/3345867188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdc806b1f0 0x7fbdc8106b90 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009900 tx=0x7fbdb802bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:47.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.816+0000 7fbdcf752640 1 -- 172.21.15.43:0/3345867188 shutdown_connections 2024-04-21T08:47:47.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.816+0000 7fbdcf752640 1 --2- 172.21.15.43:0/3345867188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdc8107270 0x7fbdc810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.816+0000 7fbdcf752640 1 --2- 172.21.15.43:0/3345867188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdc806b1f0 0x7fbdc8106b90 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.816+0000 7fbdcf752640 1 --2- 172.21.15.43:0/3345867188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdc806a840 0x7fbdc806ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.816+0000 7fbdcf752640 1 -- 172.21.15.43:0/3345867188 >> 172.21.15.43:0/3345867188 conn(0x7fbdc8077eb0 msgr2=0x7fbdc80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.816+0000 7fbdcf752640 1 -- 172.21.15.43:0/3345867188 shutdown_connections 2024-04-21T08:47:47.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.816+0000 7fbdcf752640 1 -- 172.21.15.43:0/3345867188 wait complete. 2024-04-21T08:47:47.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.817+0000 7fbdcf752640 1 Processor -- start 2024-04-21T08:47:47.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.817+0000 7fbdcf752640 1 -- start start 2024-04-21T08:47:47.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.817+0000 7fbdcf752640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdc806a840 0x7fbdc81986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.817+0000 7fbdcf752640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdc8107270 0x7fbdc8198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.817+0000 7fbdcf752640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdc819cf90 0x7fbdc819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.817+0000 7fbdcf752640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc8112190 con 0x7fbdc8107270 2024-04-21T08:47:47.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.817+0000 7fbdcf752640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc8112010 con 0x7fbdc806a840 2024-04-21T08:47:47.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.817+0000 7fbdcf752640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc8112310 con 0x7fbdc819cf90 2024-04-21T08:47:47.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.817+0000 7fbdccced640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdc8107270 0x7fbdc8198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.817+0000 7fbdccced640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdc8107270 0x7fbdc8198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57560/0 (socket says 172.21.15.43:57560) 2024-04-21T08:47:47.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.817+0000 7fbdccced640 1 -- 172.21.15.43:0/2360769948 learned_addr learned my addr 172.21.15.43:0/2360769948 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.817+0000 7fbdcdcef640 1 --2- 172.21.15.43:0/2360769948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdc819cf90 0x7fbdc819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.817+0000 7fbdcd4ee640 1 --2- 172.21.15.43:0/2360769948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdc806a840 0x7fbdc81986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.817+0000 7fbdccced640 1 -- 172.21.15.43:0/2360769948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdc819cf90 msgr2=0x7fbdc819d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.818+0000 7fbdccced640 1 --2- 172.21.15.43:0/2360769948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdc819cf90 0x7fbdc819d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.818+0000 7fbdccced640 1 -- 172.21.15.43:0/2360769948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdc806a840 msgr2=0x7fbdc81986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.818+0000 7fbdccced640 1 --2- 172.21.15.43:0/2360769948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdc806a840 0x7fbdc81986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.818+0000 7fbdccced640 1 -- 172.21.15.43:0/2360769948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc819da80 con 0x7fbdc8107270 2024-04-21T08:47:47.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.818+0000 7fbdccced640 1 --2- 172.21.15.43:0/2360769948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdc8107270 0x7fbdc8198c10 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8036040 tx=0x7fbdb80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.818+0000 7fbdbe7fc640 1 -- 172.21.15.43:0/2360769948 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8043020 con 0x7fbdc8107270 2024-04-21T08:47:47.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.818+0000 7fbdbe7fc640 1 -- 172.21.15.43:0/2360769948 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdb802cb70 con 0x7fbdc8107270 2024-04-21T08:47:47.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.818+0000 7fbdcf752640 1 -- 172.21.15.43:0/2360769948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc819dd10 con 0x7fbdc8107270 2024-04-21T08:47:47.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.818+0000 7fbdbe7fc640 1 -- 172.21.15.43:0/2360769948 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8003bf0 con 0x7fbdc8107270 2024-04-21T08:47:47.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.818+0000 7fbdcf752640 1 -- 172.21.15.43:0/2360769948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc81a5560 con 0x7fbdc8107270 2024-04-21T08:47:47.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.821+0000 7fbdbe7fc640 1 -- 172.21.15.43:0/2360769948 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbdb8046020 con 0x7fbdc8107270 2024-04-21T08:47:47.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.821+0000 7fbdd2429740 1 Processor -- start 2024-04-21T08:47:47.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.821+0000 7fbdd2429740 1 -- start start 2024-04-21T08:47:47.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.821+0000 7fbdd2429740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117cec4d0 0x55b117cec8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.821+0000 7fbdd2429740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cece80 0x55b117cf5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.821+0000 7fbdd2429740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b117cf5a00 0x55b117cf7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.821+0000 7fbdd2429740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b117c511f0 con 0x55b117cece80 2024-04-21T08:47:47.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.821+0000 7fbdd2429740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b117bee4b0 con 0x55b117cec4d0 2024-04-21T08:47:47.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.821+0000 7fbdd2429740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b117c4d1b0 con 0x55b117cf5a00 2024-04-21T08:47:47.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.821+0000 7fbdcd4ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cece80 0x55b117cf5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.821+0000 7fbdbe7fc640 1 --2- 172.21.15.43:0/2360769948 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbdb40a3da0 0x7fbdb40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.822+0000 7fbdcd4ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cece80 0x55b117cf5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57570/0 (socket says 172.21.15.43:57570) 2024-04-21T08:47:47.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.822+0000 7fbdcd4ee640 1 -- 172.21.15.43:0/2842170833 learned_addr learned my addr 172.21.15.43:0/2842170833 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.822+0000 7fbdcdcef640 1 --2- 172.21.15.43:0/2360769948 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbdb40a3da0 0x7fbdb40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.822+0000 7fbdbe7fc640 1 -- 172.21.15.43:0/2360769948 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fbdb80f2c40 con 0x7fbdc8107270 2024-04-21T08:47:47.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.822+0000 7fbdccced640 1 --2- 172.21.15.43:0/2842170833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b117cf5a00 0x55b117cf7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.822+0000 7fbdcdcef640 1 --2- 172.21.15.43:0/2842170833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117cec4d0 0x55b117cec8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.822+0000 7fbdcd4ee640 1 -- 172.21.15.43:0/2842170833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b117cf5a00 msgr2=0x55b117cf7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.822+0000 7fbdcd4ee640 1 --2- 172.21.15.43:0/2842170833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b117cf5a00 0x55b117cf7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.822+0000 7fbdcd4ee640 1 -- 172.21.15.43:0/2842170833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117cec4d0 msgr2=0x55b117cec8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.822+0000 7fbdcd4ee640 1 --2- 172.21.15.43:0/2842170833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117cec4d0 0x55b117cec8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.822+0000 7fbdcd4ee640 1 -- 172.21.15.43:0/2842170833 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b117be01b0 con 0x55b117cece80 2024-04-21T08:47:47.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.822+0000 7fbdccced640 1 --2- 172.21.15.43:0/2842170833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b117cf5a00 0x55b117cf7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:47.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.822+0000 7fbdcd4ee640 1 --2- 172.21.15.43:0/2842170833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cece80 0x55b117cf5430 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009ca0 tx=0x7fbdc40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4c549ac95c32d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.822+0000 7fbdcdcef640 1 --2- 172.21.15.43:0/2360769948 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbdb40a3da0 0x7fbdb40a6260 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009fd0 tx=0x7fbdc0009840 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.822+0000 7fbdb37fe640 1 -- 172.21.15.43:0/2842170833 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4012450 con 0x55b117cece80 2024-04-21T08:47:47.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.822+0000 7fbdb37fe640 1 -- 172.21.15.43:0/2842170833 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc4012aa0 con 0x55b117cece80 2024-04-21T08:47:47.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.822+0000 7fbdb37fe640 1 -- 172.21.15.43:0/2842170833 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc400fb60 con 0x55b117cece80 2024-04-21T08:47:47.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.823+0000 7fbdd2429740 1 -- 172.21.15.43:0/2842170833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cece80 msgr2=0x55b117cf5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.823+0000 7fbdd2429740 1 --2- 172.21.15.43:0/2842170833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cece80 0x55b117cf5430 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009ca0 tx=0x7fbdc40117b0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.823+0000 7fbdd2429740 1 -- 172.21.15.43:0/2842170833 shutdown_connections 2024-04-21T08:47:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.823+0000 7fbdd2429740 1 --2- 172.21.15.43:0/2842170833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b117cf5a00 0x55b117cf7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.823+0000 7fbdd2429740 1 --2- 172.21.15.43:0/2842170833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cece80 0x55b117cf5430 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.823+0000 7fbdd2429740 1 --2- 172.21.15.43:0/2842170833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117cec4d0 0x55b117cec8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.823+0000 7fbdd2429740 1 -- 172.21.15.43:0/2842170833 >> 172.21.15.43:0/2842170833 conn(0x55b117ce6ba0 msgr2=0x55b117ce6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.823+0000 7fbdd2429740 1 -- 172.21.15.43:0/2842170833 shutdown_connections 2024-04-21T08:47:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.823+0000 7fbdd2429740 1 -- 172.21.15.43:0/2842170833 wait complete. 2024-04-21T08:47:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.823+0000 7fbdd2429740 1 -- start start 2024-04-21T08:47:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.823+0000 7fbdd2429740 1 Processor -- start 2024-04-21T08:47:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.824+0000 7fbdd2429740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b117cec4d0 0x55b117cf5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.824+0000 7fbdd2429740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117cece80 0x55b117cf25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.824+0000 7fbdd2429740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cf5a00 0x55b117cf2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.824+0000 7fbdd2429740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b117cf8970 con 0x55b117cf5a00 2024-04-21T08:47:47.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.824+0000 7fbdd2429740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b117c0a500 con 0x55b117cece80 2024-04-21T08:47:47.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.824+0000 7fbdd2429740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b117cf8af0 con 0x55b117cec4d0 2024-04-21T08:47:47.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.824+0000 7fbdd2429740 10 client.0 _ll_register_callbacks cb 0x55b117bd8b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:47.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.824+0000 7fbdcdcef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cf5a00 0x55b117cf2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.824+0000 7fbdcdcef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cf5a00 0x55b117cf2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57584/0 (socket says 172.21.15.43:57584) 2024-04-21T08:47:47.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.824+0000 7fbdcdcef640 1 -- 172.21.15.43:0/958288284 learned_addr learned my addr 172.21.15.43:0/958288284 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.824+0000 7fbdccced640 1 --2- 172.21.15.43:0/958288284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117cece80 0x55b117cf25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.824+0000 7fbdcd4ee640 1 --2- 172.21.15.43:0/958288284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b117cec4d0 0x55b117cf5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.824+0000 7fbdcdcef640 1 -- 172.21.15.43:0/958288284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b117cec4d0 msgr2=0x55b117cf5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.824+0000 7fbdcdcef640 1 --2- 172.21.15.43:0/958288284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b117cec4d0 0x55b117cf5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.824+0000 7fbdcdcef640 1 -- 172.21.15.43:0/958288284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117cece80 msgr2=0x55b117cf25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.824+0000 7fbdcdcef640 1 --2- 172.21.15.43:0/958288284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117cece80 0x55b117cf25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.824+0000 7fbdcdcef640 1 -- 172.21.15.43:0/958288284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1179633f0 con 0x55b117cf5a00 2024-04-21T08:47:47.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.825+0000 7fbdcdcef640 1 --2- 172.21.15.43:0/958288284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cf5a00 0x55b117cf2b30 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00122e0 tx=0x7fbdc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e3e296d2d482be server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.825+0000 7fbdb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:47.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.825+0000 7fbdb0ff9640 1 -- 172.21.15.43:0/958288284 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0011b60 con 0x55b117cf5a00 2024-04-21T08:47:47.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.825+0000 7fbdb0ff9640 1 -- 172.21.15.43:0/958288284 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fbdc0011d00 con 0x55b117cf5a00 2024-04-21T08:47:47.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.825+0000 7fbdd2429740 1 -- 172.21.15.43:0/958288284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdc4012aa0 con 0x55b117cf5a00 2024-04-21T08:47:47.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.825+0000 7fbdb0ff9640 1 -- 172.21.15.43:0/958288284 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbdc0032050 con 0x55b117cf5a00 2024-04-21T08:47:47.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.825+0000 7fbdd2429740 10 client.19608 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:47.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.825+0000 7fbdd2429740 1 -- 172.21.15.43:0/958288284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b117a2d7d0 con 0x55b117cf5a00 2024-04-21T08:47:47.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.825+0000 7fbdb0ff9640 1 -- 172.21.15.43:0/958288284 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc4012aa0 con 0x55b117cf5a00 2024-04-21T08:47:47.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.825+0000 7fbdb0ff9640 1 -- 172.21.15.43:0/958288284 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fbdc001f040 con 0x55b117cf5a00 2024-04-21T08:47:47.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.825+0000 7fbdd2429740 10 client.19608 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:47.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.825+0000 7fbdd2429740 10 client.19608 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:47:47.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.825+0000 7fbdd2429740 20 client.19608 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:47.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.825+0000 7fbdd2429740 1 --2- 172.21.15.43:0/958288284 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55b117d05bb0 0x55b117d07fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.825+0000 7fbdd2429740 4 client.19608 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:47.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.825+0000 7fbdd2429740 1 -- 172.21.15.43:0/958288284 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b117bf0930 con 0x55b117d05bb0 2024-04-21T08:47:47.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.826+0000 7fbdcd4ee640 1 --2- 172.21.15.43:0/958288284 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55b117d05bb0 0x55b117d07fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.826+0000 7fbdcd4ee640 1 --2- 172.21.15.43:0/958288284 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55b117d05bb0 0x55b117d07fe0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4eaaa3810eff9d94 server_cookie=84e05ed839664647 in_seq=0 out_seq=0 2024-04-21T08:47:47.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.826+0000 7fbdb0ff9640 10 client.19608 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:47:47.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.827+0000 7fbdb0ff9640 1 -- 172.21.15.43:0/958288284 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b117bf0930 con 0x55b117d05bb0 2024-04-21T08:47:47.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.827+0000 7fbdb0ff9640 10 client.19608 handle_command_reply: tid=0 2024-04-21T08:47:47.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.827+0000 7fbdd2429740 1 client.19608 shutdown 2024-04-21T08:47:47.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.827+0000 7fbdd2429740 1 -- 172.21.15.43:0/958288284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cf5a00 msgr2=0x55b117cf2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.827+0000 7fbdd2429740 1 --2- 172.21.15.43:0/958288284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cf5a00 0x55b117cf2b30 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00122e0 tx=0x7fbdc0004a40 comp rx=0 tx=0).stop 2024-04-21T08:47:47.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.827+0000 7fbdd2429740 1 -- 172.21.15.43:0/958288284 shutdown_connections 2024-04-21T08:47:47.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.827+0000 7fbdd2429740 1 --2- 172.21.15.43:0/958288284 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55b117d05bb0 0x55b117d07fe0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.827+0000 7fbdd2429740 1 --2- 172.21.15.43:0/958288284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cf5a00 0x55b117cf2b30 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.827+0000 7fbdd2429740 1 --2- 172.21.15.43:0/958288284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117cece80 0x55b117cf25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.827+0000 7fbdb0ff9640 0 client.19608 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:47:47.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.827+0000 7fbdd2429740 1 --2- 172.21.15.43:0/958288284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b117cec4d0 0x55b117cf5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.827+0000 7fbdd2429740 1 -- 172.21.15.43:0/958288284 >> 172.21.15.43:0/958288284 conn(0x55b117ce6ba0 msgr2=0x55b117ce88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.827+0000 7fbdd2429740 1 -- 172.21.15.43:0/958288284 shutdown_connections 2024-04-21T08:47:47.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.827+0000 7fbdd2429740 1 -- 172.21.15.43:0/958288284 wait complete. 2024-04-21T08:47:47.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.827+0000 7fbdd2429740 20 client.19608 trim_cache size 0 max 16384 2024-04-21T08:47:47.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.841+0000 7fbdd2429740 1 Processor -- start 2024-04-21T08:47:47.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.841+0000 7fbdd2429740 1 -- start start 2024-04-21T08:47:47.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.841+0000 7fbdd2429740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b117cec4d0 0x55b117d01860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.841+0000 7fbdd2429740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117cece80 0x55b117cf4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.841+0000 7fbdd2429740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cf5a00 0x55b117cf5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.841+0000 7fbdd2429740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc81af460 con 0x55b117cf5a00 2024-04-21T08:47:47.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.841+0000 7fbdd2429740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc81b0fa0 con 0x55b117cece80 2024-04-21T08:47:47.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.841+0000 7fbdd2429740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc81afb30 con 0x55b117cec4d0 2024-04-21T08:47:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.841+0000 7fbdcdcef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cf5a00 0x55b117cf5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.841+0000 7fbdcdcef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cf5a00 0x55b117cf5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57600/0 (socket says 172.21.15.43:57600) 2024-04-21T08:47:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.841+0000 7fbdcdcef640 1 -- 172.21.15.43:0/2249042570 learned_addr learned my addr 172.21.15.43:0/2249042570 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.841+0000 7fbdccced640 1 --2- 172.21.15.43:0/2249042570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117cece80 0x55b117cf4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.841+0000 7fbdcdcef640 1 -- 172.21.15.43:0/2249042570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b117cec4d0 msgr2=0x55b117d01860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.841+0000 7fbdcdcef640 1 --2- 172.21.15.43:0/2249042570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b117cec4d0 0x55b117d01860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.841+0000 7fbdcdcef640 1 -- 172.21.15.43:0/2249042570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117cece80 msgr2=0x55b117cf4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.841+0000 7fbdcdcef640 1 --2- 172.21.15.43:0/2249042570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117cece80 0x55b117cf4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.841+0000 7fbdcdcef640 1 -- 172.21.15.43:0/2249042570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b117cf8210 con 0x55b117cf5a00 2024-04-21T08:47:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.841+0000 7fbdccced640 1 --2- 172.21.15.43:0/2249042570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117cece80 0x55b117cf4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.841+0000 7fbdcdcef640 1 --2- 172.21.15.43:0/2249042570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cf5a00 0x55b117cf5260 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0020540 tx=0x7fbdc0012350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c0bf87b7bc065b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.842+0000 7fbdb2ffd640 1 -- 172.21.15.43:0/2249042570 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc003a070 con 0x55b117cf5a00 2024-04-21T08:47:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.842+0000 7fbdb2ffd640 1 -- 172.21.15.43:0/2249042570 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc0004d20 con 0x55b117cf5a00 2024-04-21T08:47:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.842+0000 7fbdb2ffd640 1 -- 172.21.15.43:0/2249042570 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc00216d0 con 0x55b117cf5a00 2024-04-21T08:47:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.842+0000 7fbdd2429740 1 -- 172.21.15.43:0/2249042570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cf5a00 msgr2=0x55b117cf5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.842+0000 7fbdd2429740 1 --2- 172.21.15.43:0/2249042570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cf5a00 0x55b117cf5260 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0020540 tx=0x7fbdc0012350 comp rx=0 tx=0).stop 2024-04-21T08:47:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.842+0000 7fbdd2429740 1 -- 172.21.15.43:0/2249042570 shutdown_connections 2024-04-21T08:47:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.842+0000 7fbdd2429740 1 --2- 172.21.15.43:0/2249042570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cf5a00 0x55b117cf5260 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.842+0000 7fbdd2429740 1 --2- 172.21.15.43:0/2249042570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117cece80 0x55b117cf4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.842+0000 7fbdd2429740 1 --2- 172.21.15.43:0/2249042570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b117cec4d0 0x55b117d01860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.842+0000 7fbdd2429740 1 -- 172.21.15.43:0/2249042570 >> 172.21.15.43:0/2249042570 conn(0x55b117ce6ba0 msgr2=0x55b117ce5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.842+0000 7fbdd2429740 1 -- 172.21.15.43:0/2249042570 shutdown_connections 2024-04-21T08:47:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.842+0000 7fbdd2429740 1 -- 172.21.15.43:0/2249042570 wait complete. 2024-04-21T08:47:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.842+0000 7fbdd2429740 1 -- start start 2024-04-21T08:47:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.843+0000 7fbdd2429740 1 Processor -- start 2024-04-21T08:47:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.843+0000 7fbdd2429740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cec4d0 0x55b117cef2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.843+0000 7fbdd2429740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117cece80 0x55b117cef7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.843+0000 7fbdd2429740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b117cf5a00 0x55b117cefd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.843+0000 7fbdd2429740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc81b08d0 con 0x55b117cec4d0 2024-04-21T08:47:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.843+0000 7fbdd2429740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc81b0200 con 0x55b117cece80 2024-04-21T08:47:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.843+0000 7fbdd2429740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc81ad460 con 0x55b117cf5a00 2024-04-21T08:47:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.843+0000 7fbdd2429740 10 client.0 _ll_register_callbacks cb 0x7fbdc0012e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.843+0000 7fbdcd4ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cec4d0 0x55b117cef2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.843+0000 7fbdcd4ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cec4d0 0x55b117cef2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57602/0 (socket says 172.21.15.43:57602) 2024-04-21T08:47:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.843+0000 7fbdcd4ee640 1 -- 172.21.15.43:0/2580338277 learned_addr learned my addr 172.21.15.43:0/2580338277 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.843+0000 7fbdccced640 1 --2- 172.21.15.43:0/2580338277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117cece80 0x55b117cef7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.843+0000 7fbdcdcef640 1 --2- 172.21.15.43:0/2580338277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b117cf5a00 0x55b117cefd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.843+0000 7fbdcd4ee640 1 -- 172.21.15.43:0/2580338277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b117cf5a00 msgr2=0x55b117cefd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.843+0000 7fbdcd4ee640 1 --2- 172.21.15.43:0/2580338277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b117cf5a00 0x55b117cefd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.843+0000 7fbdcd4ee640 1 -- 172.21.15.43:0/2580338277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117cece80 msgr2=0x55b117cef7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.843+0000 7fbdcd4ee640 1 --2- 172.21.15.43:0/2580338277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117cece80 0x55b117cef7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.843+0000 7fbdcd4ee640 1 -- 172.21.15.43:0/2580338277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b117d6b120 con 0x55b117cec4d0 2024-04-21T08:47:47.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.843+0000 7fbdccced640 1 --2- 172.21.15.43:0/2580338277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117cece80 0x55b117cef7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:47.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.844+0000 7fbdcd4ee640 1 --2- 172.21.15.43:0/2580338277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cec4d0 0x55b117cef2a0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x55b117cebfa0 tx=0x7fbdc40096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed8d68a207abb15 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.844+0000 7fbdb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:47.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.844+0000 7fbdb0ff9640 1 -- 172.21.15.43:0/2580338277 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4004020 con 0x55b117cec4d0 2024-04-21T08:47:47.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.844+0000 7fbdb0ff9640 1 -- 172.21.15.43:0/2580338277 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fbdc40041c0 con 0x55b117cec4d0 2024-04-21T08:47:47.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.844+0000 7fbdd2429740 1 -- 172.21.15.43:0/2580338277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdc0004d20 con 0x55b117cec4d0 2024-04-21T08:47:47.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.844+0000 7fbdb0ff9640 1 -- 172.21.15.43:0/2580338277 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbdc4039050 con 0x55b117cec4d0 2024-04-21T08:47:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.844+0000 7fbdd2429740 10 client.19614 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.844+0000 7fbdd2429740 1 -- 172.21.15.43:0/2580338277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b117d6c330 con 0x55b117cec4d0 2024-04-21T08:47:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.844+0000 7fbdb0ff9640 1 -- 172.21.15.43:0/2580338277 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc0004d20 con 0x55b117cec4d0 2024-04-21T08:47:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.844+0000 7fbdb0ff9640 1 -- 172.21.15.43:0/2580338277 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fbdc4004950 con 0x55b117cec4d0 2024-04-21T08:47:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.844+0000 7fbdd2429740 10 client.19614 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.844+0000 7fbdd2429740 10 client.19614 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:47:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.844+0000 7fbdd2429740 20 client.19614 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.844+0000 7fbdd2429740 1 --2- 172.21.15.43:0/2580338277 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55b117d6ca60 0x55b117d6ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.844+0000 7fbdd2429740 4 client.19614 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.844+0000 7fbdd2429740 1 -- 172.21.15.43:0/2580338277 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b117cf2ec0 con 0x55b117d6ca60 2024-04-21T08:47:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.845+0000 7fbdccced640 1 --2- 172.21.15.43:0/2580338277 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55b117d6ca60 0x55b117d6ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.845+0000 7fbdccced640 1 --2- 172.21.15.43:0/2580338277 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55b117d6ca60 0x55b117d6ee80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=faae92c7cb97e16 server_cookie=253b556a1fdc9821 in_seq=0 out_seq=0 2024-04-21T08:47:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.845+0000 7fbdb0ff9640 10 client.19614 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:47:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.845+0000 7fbdb0ff9640 1 -- 172.21.15.43:0/2580338277 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b117cf2ec0 con 0x55b117d6ca60 2024-04-21T08:47:47.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.845+0000 7fbdb0ff9640 10 client.19614 handle_command_reply: tid=0 2024-04-21T08:47:47.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.845+0000 7fbdd2429740 1 client.19614 shutdown 2024-04-21T08:47:47.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.845+0000 7fbdd2429740 1 -- 172.21.15.43:0/2580338277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cec4d0 msgr2=0x55b117cef2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.845+0000 7fbdd2429740 1 --2- 172.21.15.43:0/2580338277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cec4d0 0x55b117cef2a0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x55b117cebfa0 tx=0x7fbdc40096c0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.846+0000 7fbdd2429740 1 -- 172.21.15.43:0/2580338277 shutdown_connections 2024-04-21T08:47:47.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.846+0000 7fbdd2429740 1 --2- 172.21.15.43:0/2580338277 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55b117d6ca60 0x55b117d6ee80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.846+0000 7fbdd2429740 1 --2- 172.21.15.43:0/2580338277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b117cf5a00 0x55b117cefd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.846+0000 7fbdd2429740 1 --2- 172.21.15.43:0/2580338277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b117cece80 0x55b117cef7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.846+0000 7fbdd2429740 1 --2- 172.21.15.43:0/2580338277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b117cec4d0 0x55b117cef2a0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.846+0000 7fbdb0ff9640 0 client.19614 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:47:47.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.846+0000 7fbdd2429740 1 -- 172.21.15.43:0/2580338277 >> 172.21.15.43:0/2580338277 conn(0x55b117ce6ba0 msgr2=0x55b117ced2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.846+0000 7fbdd2429740 1 -- 172.21.15.43:0/2580338277 shutdown_connections 2024-04-21T08:47:47.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.846+0000 7fbdd2429740 1 -- 172.21.15.43:0/2580338277 wait complete. 2024-04-21T08:47:47.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.846+0000 7fbdd2429740 20 client.19614 trim_cache size 0 max 16384 2024-04-21T08:47:47.854 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:47:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.846+0000 7fbdcf752640 1 -- 172.21.15.43:0/2360769948 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbdb40a3da0 msgr2=0x7fbdb40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.846+0000 7fbdcf752640 1 --2- 172.21.15.43:0/2360769948 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbdb40a3da0 0x7fbdb40a6260 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009fd0 tx=0x7fbdc0009840 comp rx=0 tx=0).stop 2024-04-21T08:47:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.846+0000 7fbdcf752640 1 -- 172.21.15.43:0/2360769948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdc8107270 msgr2=0x7fbdc8198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.846+0000 7fbdcf752640 1 --2- 172.21.15.43:0/2360769948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdc8107270 0x7fbdc8198c10 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8036040 tx=0x7fbdb80045e0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.846+0000 7fbdcf752640 1 -- 172.21.15.43:0/2360769948 shutdown_connections 2024-04-21T08:47:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.846+0000 7fbdcf752640 1 --2- 172.21.15.43:0/2360769948 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbdb40a3da0 0x7fbdb40a6260 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.846+0000 7fbdcf752640 1 --2- 172.21.15.43:0/2360769948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdc819cf90 0x7fbdc819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.846+0000 7fbdcf752640 1 --2- 172.21.15.43:0/2360769948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdc8107270 0x7fbdc8198c10 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.846+0000 7fbdcf752640 1 --2- 172.21.15.43:0/2360769948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdc806a840 0x7fbdc81986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.846+0000 7fbdcf752640 1 -- 172.21.15.43:0/2360769948 >> 172.21.15.43:0/2360769948 conn(0x7fbdc8077eb0 msgr2=0x7fbdc8108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.846+0000 7fbdcf752640 1 -- 172.21.15.43:0/2360769948 shutdown_connections 2024-04-21T08:47:47.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.846+0000 7fbdcf752640 1 -- 172.21.15.43:0/2360769948 wait complete. 2024-04-21T08:47:47.858 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:47:47.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.961+0000 7fc62f211640 1 -- 172.21.15.43:0/1235154137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc628074300 msgr2=0x7fc6280746e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.961+0000 7fc62f211640 1 --2- 172.21.15.43:0/1235154137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc628074300 0x7fc6280746e0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fc61c009900 tx=0x7fc61c02bcd0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.961+0000 7fc62f211640 1 -- 172.21.15.43:0/1235154137 shutdown_connections 2024-04-21T08:47:47.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.961+0000 7fc62f211640 1 --2- 172.21.15.43:0/1235154137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc62810e7e0 0x7fc628110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.961+0000 7fc62f211640 1 --2- 172.21.15.43:0/1235154137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc628074c20 0x7fc62810e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.961+0000 7fc62f211640 1 --2- 172.21.15.43:0/1235154137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc628074300 0x7fc6280746e0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.961+0000 7fc62f211640 1 -- 172.21.15.43:0/1235154137 >> 172.21.15.43:0/1235154137 conn(0x7fc6280fd990 msgr2=0x7fc6280ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.961+0000 7fc62f211640 1 -- 172.21.15.43:0/1235154137 shutdown_connections 2024-04-21T08:47:47.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.961+0000 7fc62f211640 1 -- 172.21.15.43:0/1235154137 wait complete. 2024-04-21T08:47:47.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.962+0000 7fc62f211640 1 Processor -- start 2024-04-21T08:47:47.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.962+0000 7fc62f211640 1 -- start start 2024-04-21T08:47:47.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.962+0000 7fc62f211640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc628074300 0x7fc6281a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.962+0000 7fc62f211640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc628074c20 0x7fc6281a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.962+0000 7fc62f211640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc62810e7e0 0x7fc62819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.962+0000 7fc62f211640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc628114390 con 0x7fc62810e7e0 2024-04-21T08:47:47.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.962+0000 7fc62f211640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc628114210 con 0x7fc628074c20 2024-04-21T08:47:47.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.962+0000 7fc62f211640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc628114510 con 0x7fc628074300 2024-04-21T08:47:47.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.962+0000 7fc62d7ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc62810e7e0 0x7fc62819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.962+0000 7fc62d7ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc62810e7e0 0x7fc62819a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57620/0 (socket says 172.21.15.43:57620) 2024-04-21T08:47:47.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.962+0000 7fc62d7ae640 1 -- 172.21.15.43:0/2918808150 learned_addr learned my addr 172.21.15.43:0/2918808150 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.962+0000 7fc627fff640 1 --2- 172.21.15.43:0/2918808150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc628074c20 0x7fc6281a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.962+0000 7fc62cfad640 1 --2- 172.21.15.43:0/2918808150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc628074300 0x7fc6281a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.963+0000 7fc62d7ae640 1 -- 172.21.15.43:0/2918808150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc628074300 msgr2=0x7fc6281a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.963+0000 7fc62d7ae640 1 --2- 172.21.15.43:0/2918808150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc628074300 0x7fc6281a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.963+0000 7fc62d7ae640 1 -- 172.21.15.43:0/2918808150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc628074c20 msgr2=0x7fc6281a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.963+0000 7fc62d7ae640 1 --2- 172.21.15.43:0/2918808150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc628074c20 0x7fc6281a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.963+0000 7fc62d7ae640 1 -- 172.21.15.43:0/2918808150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62819b030 con 0x7fc62810e7e0 2024-04-21T08:47:47.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.963+0000 7fc627fff640 1 --2- 172.21.15.43:0/2918808150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc628074c20 0x7fc6281a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:47.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.963+0000 7fc62d7ae640 1 --2- 172.21.15.43:0/2918808150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc62810e7e0 0x7fc62819a770 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fc62000eb10 tx=0x7fc62000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.963+0000 7fc625ffb640 1 -- 172.21.15.43:0/2918808150 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc620004b20 con 0x7fc62810e7e0 2024-04-21T08:47:47.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.963+0000 7fc625ffb640 1 -- 172.21.15.43:0/2918808150 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc620004cc0 con 0x7fc62810e7e0 2024-04-21T08:47:47.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.963+0000 7fc62f211640 1 -- 172.21.15.43:0/2918808150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc62819b320 con 0x7fc62810e7e0 2024-04-21T08:47:47.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.963+0000 7fc625ffb640 1 -- 172.21.15.43:0/2918808150 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc620010670 con 0x7fc62810e7e0 2024-04-21T08:47:47.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.963+0000 7fc62f211640 1 -- 172.21.15.43:0/2918808150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6281a7630 con 0x7fc62810e7e0 2024-04-21T08:47:47.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.965+0000 7fc625ffb640 1 -- 172.21.15.43:0/2918808150 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc620005020 con 0x7fc62810e7e0 2024-04-21T08:47:47.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.966+0000 7fc631e00740 1 Processor -- start 2024-04-21T08:47:47.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.966+0000 7fc631e00740 1 -- start start 2024-04-21T08:47:47.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.966+0000 7fc625ffb640 1 --2- 172.21.15.43:0/2918808150 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc6180a3d20 0x7fc6180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.966+0000 7fc631e00740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ff82544d0 0x560ff82548b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.966+0000 7fc631e00740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ff8254e80 0x560ff825d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.966+0000 7fc631e00740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff825da00 0x560ff825fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.966+0000 7fc631e00740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560ff81b91f0 con 0x560ff825da00 2024-04-21T08:47:47.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.966+0000 7fc631e00740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560ff81564b0 con 0x560ff8254e80 2024-04-21T08:47:47.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.966+0000 7fc631e00740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560ff81b51b0 con 0x560ff82544d0 2024-04-21T08:47:47.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.966+0000 7fc625ffb640 1 -- 172.21.15.43:0/2918808150 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fc6200d1160 con 0x7fc62810e7e0 2024-04-21T08:47:47.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.966+0000 7fc627fff640 1 --2- 172.21.15.43:0/2918808150 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc6180a3d20 0x7fc6180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.966+0000 7fc62d7ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff825da00 0x560ff825fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.966+0000 7fc62cfad640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ff82544d0 0x560ff82548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.966+0000 7fc62d7ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff825da00 0x560ff825fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57622/0 (socket says 172.21.15.43:57622) 2024-04-21T08:47:47.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.966+0000 7fc62d7ae640 1 -- 172.21.15.43:0/38242402 learned_addr learned my addr 172.21.15.43:0/38242402 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.967+0000 7fc627fff640 1 --2- 172.21.15.43:0/38242402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ff8254e80 0x560ff825d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.967+0000 7fc62d7ae640 1 -- 172.21.15.43:0/38242402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ff82544d0 msgr2=0x560ff82548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.967+0000 7fc62d7ae640 1 --2- 172.21.15.43:0/38242402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ff82544d0 0x560ff82548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.967+0000 7fc62d7ae640 1 -- 172.21.15.43:0/38242402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ff8254e80 msgr2=0x560ff825d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.967+0000 7fc62d7ae640 1 --2- 172.21.15.43:0/38242402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ff8254e80 0x560ff825d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.967+0000 7fc62d7ae640 1 -- 172.21.15.43:0/38242402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ff81481b0 con 0x560ff825da00 2024-04-21T08:47:47.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.967+0000 7fc627fff640 1 --2- 172.21.15.43:0/2918808150 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc6180a3d20 0x7fc6180a61e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc62819bfc0 tx=0x7fc61400bf30 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.967+0000 7fc62d7ae640 1 --2- 172.21.15.43:0/38242402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff825da00 0x560ff825fe50 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fc6200a53c0 tx=0x7fc62009c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ddda83bef2bb268 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.967+0000 7fc612ffd640 1 -- 172.21.15.43:0/38242402 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62009d380 con 0x560ff825da00 2024-04-21T08:47:47.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.967+0000 7fc612ffd640 1 -- 172.21.15.43:0/38242402 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc62009cc80 con 0x560ff825da00 2024-04-21T08:47:47.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.967+0000 7fc612ffd640 1 -- 172.21.15.43:0/38242402 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6200bd110 con 0x560ff825da00 2024-04-21T08:47:47.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.968+0000 7fc631e00740 1 -- 172.21.15.43:0/38242402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff825da00 msgr2=0x560ff825fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.968+0000 7fc631e00740 1 --2- 172.21.15.43:0/38242402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff825da00 0x560ff825fe50 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fc6200a53c0 tx=0x7fc62009c7c0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.968+0000 7fc631e00740 1 -- 172.21.15.43:0/38242402 shutdown_connections 2024-04-21T08:47:47.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.968+0000 7fc631e00740 1 --2- 172.21.15.43:0/38242402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff825da00 0x560ff825fe50 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.968+0000 7fc631e00740 1 --2- 172.21.15.43:0/38242402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ff8254e80 0x560ff825d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.968+0000 7fc631e00740 1 --2- 172.21.15.43:0/38242402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ff82544d0 0x560ff82548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.968+0000 7fc631e00740 1 -- 172.21.15.43:0/38242402 >> 172.21.15.43:0/38242402 conn(0x560ff824eba0 msgr2=0x560ff824ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.968+0000 7fc631e00740 1 -- 172.21.15.43:0/38242402 shutdown_connections 2024-04-21T08:47:47.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.968+0000 7fc631e00740 1 -- 172.21.15.43:0/38242402 wait complete. 2024-04-21T08:47:47.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.968+0000 7fc631e00740 1 -- start start 2024-04-21T08:47:47.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.968+0000 7fc631e00740 1 Processor -- start 2024-04-21T08:47:47.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.969+0000 7fc631e00740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ff82544d0 0x560ff825d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.969+0000 7fc631e00740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff8254e80 0x560ff825a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.969+0000 7fc631e00740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ff825da00 0x560ff825ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.969+0000 7fc631e00740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560ff8260970 con 0x560ff8254e80 2024-04-21T08:47:47.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.969+0000 7fc631e00740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560ff8172500 con 0x560ff82544d0 2024-04-21T08:47:47.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.969+0000 7fc631e00740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560ff8260af0 con 0x560ff825da00 2024-04-21T08:47:47.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.969+0000 7fc631e00740 10 client.0 _ll_register_callbacks cb 0x560ff8140b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:47.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.969+0000 7fc627fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff8254e80 0x560ff825a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.969+0000 7fc627fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff8254e80 0x560ff825a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57634/0 (socket says 172.21.15.43:57634) 2024-04-21T08:47:47.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.969+0000 7fc627fff640 1 -- 172.21.15.43:0/3149468325 learned_addr learned my addr 172.21.15.43:0/3149468325 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.969+0000 7fc62d7ae640 1 --2- 172.21.15.43:0/3149468325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ff825da00 0x560ff825ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.969+0000 7fc62cfad640 1 --2- 172.21.15.43:0/3149468325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ff82544d0 0x560ff825d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.969+0000 7fc627fff640 1 -- 172.21.15.43:0/3149468325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ff825da00 msgr2=0x560ff825ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.969+0000 7fc627fff640 1 --2- 172.21.15.43:0/3149468325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ff825da00 0x560ff825ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.969+0000 7fc627fff640 1 -- 172.21.15.43:0/3149468325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ff82544d0 msgr2=0x560ff825d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.969+0000 7fc627fff640 1 --2- 172.21.15.43:0/3149468325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ff82544d0 0x560ff825d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.969+0000 7fc627fff640 1 -- 172.21.15.43:0/3149468325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ff7ecb3f0 con 0x560ff8254e80 2024-04-21T08:47:47.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.969+0000 7fc62cfad640 1 --2- 172.21.15.43:0/3149468325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ff82544d0 0x560ff825d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:47.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.969+0000 7fc627fff640 1 --2- 172.21.15.43:0/3149468325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff8254e80 0x560ff825a630 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fc61401e6a0 tx=0x7fc614004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e67ac75573961e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.969+0000 7fc607fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:47.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.969+0000 7fc607fff640 1 -- 172.21.15.43:0/3149468325 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc614004020 con 0x560ff8254e80 2024-04-21T08:47:47.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.969+0000 7fc607fff640 1 -- 172.21.15.43:0/3149468325 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fc6140041c0 con 0x560ff8254e80 2024-04-21T08:47:47.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.969+0000 7fc631e00740 1 -- 172.21.15.43:0/3149468325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc62009cc80 con 0x560ff8254e80 2024-04-21T08:47:47.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.970+0000 7fc607fff640 1 -- 172.21.15.43:0/3149468325 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc6140043c0 con 0x560ff8254e80 2024-04-21T08:47:47.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.970+0000 7fc631e00740 10 client.19626 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:47.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.970+0000 7fc631e00740 1 -- 172.21.15.43:0/3149468325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560ff7f957d0 con 0x560ff8254e80 2024-04-21T08:47:47.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.970+0000 7fc607fff640 1 -- 172.21.15.43:0/3149468325 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc62009cc80 con 0x560ff8254e80 2024-04-21T08:47:47.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.970+0000 7fc607fff640 1 -- 172.21.15.43:0/3149468325 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc61401f070 con 0x560ff8254e80 2024-04-21T08:47:47.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.970+0000 7fc631e00740 10 client.19626 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:47.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.970+0000 7fc631e00740 10 client.19626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:47.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.970+0000 7fc631e00740 20 client.19626 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:47.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.970+0000 7fc631e00740 1 --2- 172.21.15.43:0/3149468325 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560ff826dbe0 0x560ff8270050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.970+0000 7fc631e00740 4 client.19626 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:47.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.970+0000 7fc631e00740 1 -- 172.21.15.43:0/3149468325 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ff8158930 con 0x560ff826dbe0 2024-04-21T08:47:47.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.971+0000 7fc62cfad640 1 --2- 172.21.15.43:0/3149468325 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560ff826dbe0 0x560ff8270050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.971+0000 7fc62cfad640 1 --2- 172.21.15.43:0/3149468325 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560ff826dbe0 0x560ff8270050 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9f3c67cff0d29a8 server_cookie=ea6dfdc98cad3b59 in_seq=0 out_seq=0 2024-04-21T08:47:47.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.971+0000 7fc607fff640 10 client.19626 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:47.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.972+0000 7fc607fff640 1 -- 172.21.15.43:0/3149468325 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560ff8158930 con 0x560ff826dbe0 2024-04-21T08:47:47.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.972+0000 7fc607fff640 10 client.19626 handle_command_reply: tid=0 2024-04-21T08:47:47.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.972+0000 7fc631e00740 1 client.19626 shutdown 2024-04-21T08:47:47.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.972+0000 7fc631e00740 1 -- 172.21.15.43:0/3149468325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff8254e80 msgr2=0x560ff825a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.972+0000 7fc631e00740 1 --2- 172.21.15.43:0/3149468325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff8254e80 0x560ff825a630 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fc61401e6a0 tx=0x7fc614004730 comp rx=0 tx=0).stop 2024-04-21T08:47:47.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.972+0000 7fc631e00740 1 -- 172.21.15.43:0/3149468325 shutdown_connections 2024-04-21T08:47:47.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.972+0000 7fc631e00740 1 --2- 172.21.15.43:0/3149468325 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560ff826dbe0 0x560ff8270050 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.972+0000 7fc631e00740 1 --2- 172.21.15.43:0/3149468325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ff825da00 0x560ff825ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.972+0000 7fc607fff640 0 client.19626 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:47.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.972+0000 7fc631e00740 1 --2- 172.21.15.43:0/3149468325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff8254e80 0x560ff825a630 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.972+0000 7fc631e00740 1 --2- 172.21.15.43:0/3149468325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ff82544d0 0x560ff825d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.972+0000 7fc631e00740 1 -- 172.21.15.43:0/3149468325 >> 172.21.15.43:0/3149468325 conn(0x560ff824eba0 msgr2=0x560ff82508f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.972+0000 7fc631e00740 1 -- 172.21.15.43:0/3149468325 shutdown_connections 2024-04-21T08:47:47.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.972+0000 7fc631e00740 1 -- 172.21.15.43:0/3149468325 wait complete. 2024-04-21T08:47:47.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.972+0000 7fc631e00740 20 client.19626 trim_cache size 0 max 16384 2024-04-21T08:47:47.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.985+0000 7fc631e00740 1 Processor -- start 2024-04-21T08:47:47.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.985+0000 7fc631e00740 1 -- start start 2024-04-21T08:47:47.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.985+0000 7fc631e00740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ff82544d0 0x560ff8269860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.985+0000 7fc631e00740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff8254e80 0x560ff8259790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.986+0000 7fc631e00740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ff825da00 0x560ff824da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.986+0000 7fc631e00740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc6281b16f0 con 0x560ff8254e80 2024-04-21T08:47:47.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.986+0000 7fc631e00740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc6281b3230 con 0x560ff825da00 2024-04-21T08:47:47.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.986+0000 7fc631e00740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc6281b1dc0 con 0x560ff82544d0 2024-04-21T08:47:47.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.986+0000 7fc627fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff8254e80 0x560ff8259790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.986+0000 7fc627fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff8254e80 0x560ff8259790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57636/0 (socket says 172.21.15.43:57636) 2024-04-21T08:47:47.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.986+0000 7fc627fff640 1 -- 172.21.15.43:0/2979446217 learned_addr learned my addr 172.21.15.43:0/2979446217 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.986+0000 7fc62d7ae640 1 --2- 172.21.15.43:0/2979446217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ff825da00 0x560ff824da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.986+0000 7fc62cfad640 1 --2- 172.21.15.43:0/2979446217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ff82544d0 0x560ff8269860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.986+0000 7fc627fff640 1 -- 172.21.15.43:0/2979446217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ff82544d0 msgr2=0x560ff8269860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.986+0000 7fc627fff640 1 --2- 172.21.15.43:0/2979446217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ff82544d0 0x560ff8269860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.986+0000 7fc627fff640 1 -- 172.21.15.43:0/2979446217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ff825da00 msgr2=0x560ff824da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.986+0000 7fc627fff640 1 --2- 172.21.15.43:0/2979446217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ff825da00 0x560ff824da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.986+0000 7fc627fff640 1 -- 172.21.15.43:0/2979446217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ff824fdd0 con 0x560ff8254e80 2024-04-21T08:47:47.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.986+0000 7fc62cfad640 1 --2- 172.21.15.43:0/2979446217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ff82544d0 0x560ff8269860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:47.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.986+0000 7fc62d7ae640 1 --2- 172.21.15.43:0/2979446217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ff825da00 0x560ff824da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:47.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.986+0000 7fc627fff640 1 --2- 172.21.15.43:0/2979446217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff8254e80 0x560ff8259790 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fc614010710 tx=0x7fc614005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b553c4cc47f956b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.986+0000 7fc6127fc640 1 -- 172.21.15.43:0/2979446217 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc614012920 con 0x560ff8254e80 2024-04-21T08:47:47.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.986+0000 7fc6127fc640 1 -- 172.21.15.43:0/2979446217 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc614024070 con 0x560ff8254e80 2024-04-21T08:47:47.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.986+0000 7fc6127fc640 1 -- 172.21.15.43:0/2979446217 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6140356a0 con 0x560ff8254e80 2024-04-21T08:47:47.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.987+0000 7fc631e00740 1 -- 172.21.15.43:0/2979446217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff8254e80 msgr2=0x560ff8259790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.987+0000 7fc631e00740 1 --2- 172.21.15.43:0/2979446217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff8254e80 0x560ff8259790 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fc614010710 tx=0x7fc614005fb0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.987+0000 7fc631e00740 1 -- 172.21.15.43:0/2979446217 shutdown_connections 2024-04-21T08:47:47.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.987+0000 7fc631e00740 1 --2- 172.21.15.43:0/2979446217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ff825da00 0x560ff824da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.987+0000 7fc631e00740 1 --2- 172.21.15.43:0/2979446217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff8254e80 0x560ff8259790 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.987+0000 7fc631e00740 1 --2- 172.21.15.43:0/2979446217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ff82544d0 0x560ff8269860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.987+0000 7fc631e00740 1 -- 172.21.15.43:0/2979446217 >> 172.21.15.43:0/2979446217 conn(0x560ff824eba0 msgr2=0x560ff824d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.987+0000 7fc631e00740 1 -- 172.21.15.43:0/2979446217 shutdown_connections 2024-04-21T08:47:47.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.987+0000 7fc631e00740 1 -- 172.21.15.43:0/2979446217 wait complete. 2024-04-21T08:47:47.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.987+0000 7fc631e00740 1 -- start start 2024-04-21T08:47:47.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.987+0000 7fc631e00740 1 Processor -- start 2024-04-21T08:47:47.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.988+0000 7fc631e00740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ff82544d0 0x560ff8261770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.988+0000 7fc631e00740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff8254e80 0x560ff8261cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.988+0000 7fc631e00740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ff825da00 0x560ff82621f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.988+0000 7fc631e00740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc6281b2b60 con 0x560ff8254e80 2024-04-21T08:47:47.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.988+0000 7fc631e00740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc6281b2490 con 0x560ff825da00 2024-04-21T08:47:47.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.988+0000 7fc631e00740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc6281af6f0 con 0x560ff82544d0 2024-04-21T08:47:47.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.988+0000 7fc631e00740 10 client.0 _ll_register_callbacks cb 0x7fc614012590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:47.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.988+0000 7fc627fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff8254e80 0x560ff8261cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.988+0000 7fc627fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff8254e80 0x560ff8261cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57642/0 (socket says 172.21.15.43:57642) 2024-04-21T08:47:47.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.988+0000 7fc627fff640 1 -- 172.21.15.43:0/387799845 learned_addr learned my addr 172.21.15.43:0/387799845 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:47.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.988+0000 7fc62d7ae640 1 --2- 172.21.15.43:0/387799845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ff825da00 0x560ff82621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.988+0000 7fc62cfad640 1 --2- 172.21.15.43:0/387799845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ff82544d0 0x560ff8261770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.988+0000 7fc627fff640 1 -- 172.21.15.43:0/387799845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ff82544d0 msgr2=0x560ff8261770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.988+0000 7fc627fff640 1 --2- 172.21.15.43:0/387799845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ff82544d0 0x560ff8261770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.988+0000 7fc627fff640 1 -- 172.21.15.43:0/387799845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ff825da00 msgr2=0x560ff82621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:47.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.988+0000 7fc627fff640 1 --2- 172.21.15.43:0/387799845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ff825da00 0x560ff82621f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.988+0000 7fc627fff640 1 -- 172.21.15.43:0/387799845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ff82d3190 con 0x560ff8254e80 2024-04-21T08:47:47.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.988+0000 7fc62cfad640 1 --2- 172.21.15.43:0/387799845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ff82544d0 0x560ff8261770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:47.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.988+0000 7fc627fff640 1 --2- 172.21.15.43:0/387799845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff8254e80 0x560ff8261cb0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fc6140105a0 tx=0x7fc614012b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abbe0302e07a2539 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:47.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.988+0000 7fc607fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:47.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.988+0000 7fc607fff640 1 -- 172.21.15.43:0/387799845 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc614010b50 con 0x560ff8254e80 2024-04-21T08:47:47.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.988+0000 7fc607fff640 1 -- 172.21.15.43:0/387799845 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fc614035440 con 0x560ff8254e80 2024-04-21T08:47:47.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.988+0000 7fc631e00740 1 -- 172.21.15.43:0/387799845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc614024070 con 0x560ff8254e80 2024-04-21T08:47:47.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.989+0000 7fc607fff640 1 -- 172.21.15.43:0/387799845 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc61404b050 con 0x560ff8254e80 2024-04-21T08:47:47.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.989+0000 7fc631e00740 10 client.19632 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:47.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.989+0000 7fc631e00740 1 -- 172.21.15.43:0/387799845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560ff82d4460 con 0x560ff8254e80 2024-04-21T08:47:47.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.989+0000 7fc607fff640 1 -- 172.21.15.43:0/387799845 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc614024070 con 0x560ff8254e80 2024-04-21T08:47:47.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.989+0000 7fc607fff640 1 -- 172.21.15.43:0/387799845 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc614028650 con 0x560ff8254e80 2024-04-21T08:47:47.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.989+0000 7fc631e00740 10 client.19632 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:47.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.989+0000 7fc631e00740 10 client.19632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:47.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.989+0000 7fc631e00740 20 client.19632 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:47.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.989+0000 7fc631e00740 1 --2- 172.21.15.43:0/387799845 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560ff82d4ae0 0x560ff82d6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:47.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.989+0000 7fc631e00740 4 client.19632 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:47.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.989+0000 7fc631e00740 1 -- 172.21.15.43:0/387799845 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560ff81c6c10 con 0x560ff82d4ae0 2024-04-21T08:47:47.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.989+0000 7fc62cfad640 1 --2- 172.21.15.43:0/387799845 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560ff82d4ae0 0x560ff82d6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:47.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.990+0000 7fc62cfad640 1 --2- 172.21.15.43:0/387799845 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560ff82d4ae0 0x560ff82d6f00 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d29f38d25c29171 server_cookie=5ddf1ca2ce271701 in_seq=0 out_seq=0 2024-04-21T08:47:47.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.990+0000 7fc607fff640 10 client.19632 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:47.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.990+0000 7fc607fff640 1 -- 172.21.15.43:0/387799845 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ff81c6c10 con 0x560ff82d4ae0 2024-04-21T08:47:47.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.990+0000 7fc607fff640 10 client.19632 handle_command_reply: tid=0 2024-04-21T08:47:47.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.990+0000 7fc631e00740 1 client.19632 shutdown 2024-04-21T08:47:47.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.990+0000 7fc631e00740 1 -- 172.21.15.43:0/387799845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff8254e80 msgr2=0x560ff8261cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.990+0000 7fc631e00740 1 --2- 172.21.15.43:0/387799845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff8254e80 0x560ff8261cb0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fc6140105a0 tx=0x7fc614012b50 comp rx=0 tx=0).stop 2024-04-21T08:47:47.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.990+0000 7fc631e00740 1 -- 172.21.15.43:0/387799845 shutdown_connections 2024-04-21T08:47:47.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.990+0000 7fc631e00740 1 --2- 172.21.15.43:0/387799845 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560ff82d4ae0 0x560ff82d6f00 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.990+0000 7fc631e00740 1 --2- 172.21.15.43:0/387799845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ff825da00 0x560ff82621f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.990+0000 7fc607fff640 0 client.19632 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:47.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.990+0000 7fc631e00740 1 --2- 172.21.15.43:0/387799845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff8254e80 0x560ff8261cb0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.990+0000 7fc631e00740 1 --2- 172.21.15.43:0/387799845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560ff82544d0 0x560ff8261770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.990+0000 7fc631e00740 1 -- 172.21.15.43:0/387799845 >> 172.21.15.43:0/387799845 conn(0x560ff824eba0 msgr2=0x560ff824dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.990+0000 7fc631e00740 1 -- 172.21.15.43:0/387799845 shutdown_connections 2024-04-21T08:47:47.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.990+0000 7fc631e00740 1 -- 172.21.15.43:0/387799845 wait complete. 2024-04-21T08:47:47.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.990+0000 7fc631e00740 20 client.19632 trim_cache size 0 max 16384 2024-04-21T08:47:47.998 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:47:47.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.991+0000 7fc62f211640 1 -- 172.21.15.43:0/2918808150 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc6180a3d20 msgr2=0x7fc6180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.991+0000 7fc62f211640 1 --2- 172.21.15.43:0/2918808150 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc6180a3d20 0x7fc6180a61e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc62819bfc0 tx=0x7fc61400bf30 comp rx=0 tx=0).stop 2024-04-21T08:47:47.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.991+0000 7fc62f211640 1 -- 172.21.15.43:0/2918808150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc62810e7e0 msgr2=0x7fc62819a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:47.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.991+0000 7fc62f211640 1 --2- 172.21.15.43:0/2918808150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc62810e7e0 0x7fc62819a770 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fc62000eb10 tx=0x7fc62000c2d0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.991+0000 7fc62f211640 1 -- 172.21.15.43:0/2918808150 shutdown_connections 2024-04-21T08:47:47.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.991+0000 7fc62f211640 1 --2- 172.21.15.43:0/2918808150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc62810e7e0 0x7fc62819a770 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.991+0000 7fc62f211640 1 --2- 172.21.15.43:0/2918808150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc628074c20 0x7fc6281a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.991+0000 7fc62f211640 1 --2- 172.21.15.43:0/2918808150 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc6180a3d20 0x7fc6180a61e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.991+0000 7fc62f211640 1 --2- 172.21.15.43:0/2918808150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc628074300 0x7fc6281a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:47.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.991+0000 7fc62f211640 1 -- 172.21.15.43:0/2918808150 >> 172.21.15.43:0/2918808150 conn(0x7fc6280fd990 msgr2=0x7fc6280fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:47.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.991+0000 7fc62f211640 1 -- 172.21.15.43:0/2918808150 shutdown_connections 2024-04-21T08:47:47.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:47.991+0000 7fc62f211640 1 -- 172.21.15.43:0/2918808150 wait complete. 2024-04-21T08:47:48.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:48 smithi043 ceph-mon[27978]: pgmap v274: 129 pgs: 129 active+clean; 5.6 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-21T08:47:48.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:48 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3353434283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:48.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:48 smithi175 ceph-mon[31920]: pgmap v274: 129 pgs: 129 active+clean; 5.6 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-21T08:47:48.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:48 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3353434283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:48.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:48 smithi112 ceph-mon[31085]: pgmap v274: 129 pgs: 129 active+clean; 5.6 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-21T08:47:48.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:48 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3353434283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:49.002 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:47:49.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:47:49.003 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:47:49.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.122+0000 7f6b0d48e640 1 -- 172.21.15.43:0/1697759585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b08102b50 msgr2=0x7f6b08106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.122+0000 7f6b0d48e640 1 --2- 172.21.15.43:0/1697759585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b08102b50 0x7f6b08106b90 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f6afc009900 tx=0x7f6afc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:49.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.122+0000 7f6b0d48e640 1 -- 172.21.15.43:0/1697759585 shutdown_connections 2024-04-21T08:47:49.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.122+0000 7f6b0d48e640 1 --2- 172.21.15.43:0/1697759585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b08107270 0x7f6b0810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.122+0000 7f6b0d48e640 1 --2- 172.21.15.43:0/1697759585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b08102b50 0x7f6b08106b90 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.122+0000 7f6b0d48e640 1 --2- 172.21.15.43:0/1697759585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b081021a0 0x7f6b08102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.122+0000 7f6b0d48e640 1 -- 172.21.15.43:0/1697759585 >> 172.21.15.43:0/1697759585 conn(0x7f6b08079350 msgr2=0x7f6b08079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.122+0000 7f6b0d48e640 1 -- 172.21.15.43:0/1697759585 shutdown_connections 2024-04-21T08:47:49.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.122+0000 7f6b0d48e640 1 -- 172.21.15.43:0/1697759585 wait complete. 2024-04-21T08:47:49.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.123+0000 7f6b0d48e640 1 Processor -- start 2024-04-21T08:47:49.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.123+0000 7f6b0d48e640 1 -- start start 2024-04-21T08:47:49.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.123+0000 7f6b0d48e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b081021a0 0x7f6b0819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.123+0000 7f6b0d48e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b08107270 0x7f6b0819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.123+0000 7f6b0d48e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b08198590 0x7f6b08198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.123+0000 7f6b0d48e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6b08112190 con 0x7f6b08198590 2024-04-21T08:47:49.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.123+0000 7f6b0d48e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6b08112010 con 0x7f6b08107270 2024-04-21T08:47:49.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.123+0000 7f6b0d48e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6b08112310 con 0x7f6b081021a0 2024-04-21T08:47:49.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.124+0000 7f6b077fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b08198590 0x7f6b08198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.124+0000 7f6b077fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b08198590 0x7f6b08198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57664/0 (socket says 172.21.15.43:57664) 2024-04-21T08:47:49.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.124+0000 7f6b077fe640 1 -- 172.21.15.43:0/1503236350 learned_addr learned my addr 172.21.15.43:0/1503236350 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.124+0000 7f6b067fc640 1 --2- 172.21.15.43:0/1503236350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b08107270 0x7f6b0819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.124+0000 7f6b06ffd640 1 --2- 172.21.15.43:0/1503236350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b081021a0 0x7f6b0819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.124+0000 7f6b077fe640 1 -- 172.21.15.43:0/1503236350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b081021a0 msgr2=0x7f6b0819e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.124+0000 7f6b077fe640 1 --2- 172.21.15.43:0/1503236350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b081021a0 0x7f6b0819e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.124+0000 7f6b077fe640 1 -- 172.21.15.43:0/1503236350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b08107270 msgr2=0x7f6b0819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.124+0000 7f6b077fe640 1 --2- 172.21.15.43:0/1503236350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b08107270 0x7f6b0819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.124+0000 7f6b077fe640 1 -- 172.21.15.43:0/1503236350 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b081990b0 con 0x7f6b08198590 2024-04-21T08:47:49.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.124+0000 7f6b077fe640 1 --2- 172.21.15.43:0/1503236350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b08198590 0x7f6b08198a40 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000eb10 tx=0x7f6b0000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.124+0000 7f6aeffff640 1 -- 172.21.15.43:0/1503236350 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b00004b20 con 0x7f6b08198590 2024-04-21T08:47:49.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.124+0000 7f6aeffff640 1 -- 172.21.15.43:0/1503236350 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b00004cc0 con 0x7f6b08198590 2024-04-21T08:47:49.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.124+0000 7f6b0d48e640 1 -- 172.21.15.43:0/1503236350 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b081993a0 con 0x7f6b08198590 2024-04-21T08:47:49.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.124+0000 7f6aeffff640 1 -- 172.21.15.43:0/1503236350 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b00010670 con 0x7f6b08198590 2024-04-21T08:47:49.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.124+0000 7f6b0d48e640 1 -- 172.21.15.43:0/1503236350 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b081a55a0 con 0x7f6b08198590 2024-04-21T08:47:49.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.126+0000 7f6aeffff640 1 -- 172.21.15.43:0/1503236350 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6b00005020 con 0x7f6b08198590 2024-04-21T08:47:49.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.127+0000 7f6b1007d740 1 Processor -- start 2024-04-21T08:47:49.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.127+0000 7f6b1007d740 1 -- start start 2024-04-21T08:47:49.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.127+0000 7f6b1007d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1be7618e0 0x55a1be761cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.127+0000 7f6b1007d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be762290 0x55a1be76a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.127+0000 7f6b1007d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1be76ae10 0x55a1be76d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.127+0000 7f6b1007d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a1be6c5f00 con 0x55a1be762290 2024-04-21T08:47:49.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.127+0000 7f6b1007d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a1be663590 con 0x55a1be76ae10 2024-04-21T08:47:49.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.127+0000 7f6b1007d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a1be6c1ec0 con 0x55a1be7618e0 2024-04-21T08:47:49.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.127+0000 7f6aeffff640 1 --2- 172.21.15.43:0/1503236350 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6af00a3d20 0x7f6af00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.127+0000 7f6b06ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be762290 0x55a1be76a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.128+0000 7f6b06ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be762290 0x55a1be76a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57668/0 (socket says 172.21.15.43:57668) 2024-04-21T08:47:49.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.128+0000 7f6b06ffd640 1 -- 172.21.15.43:0/2683195023 learned_addr learned my addr 172.21.15.43:0/2683195023 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.128+0000 7f6b077fe640 1 --2- 172.21.15.43:0/1503236350 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6af00a3d20 0x7f6af00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.128+0000 7f6b067fc640 1 --2- 172.21.15.43:0/2683195023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1be7618e0 0x55a1be761cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.128+0000 7f6b067fc640 1 --2- 172.21.15.43:0/2683195023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1be76ae10 0x55a1be76d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.128+0000 7f6aeffff640 1 -- 172.21.15.43:0/1503236350 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f6b000d1290 con 0x7f6b08198590 2024-04-21T08:47:49.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.128+0000 7f6b06ffd640 1 -- 172.21.15.43:0/2683195023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1be7618e0 msgr2=0x55a1be761cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.128+0000 7f6b06ffd640 1 --2- 172.21.15.43:0/2683195023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1be7618e0 0x55a1be761cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.128+0000 7f6b06ffd640 1 -- 172.21.15.43:0/2683195023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1be76ae10 msgr2=0x55a1be76d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.128+0000 7f6b06ffd640 1 --2- 172.21.15.43:0/2683195023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1be76ae10 0x55a1be76d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.128+0000 7f6b06ffd640 1 -- 172.21.15.43:0/2683195023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1be4a07e0 con 0x55a1be762290 2024-04-21T08:47:49.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.128+0000 7f6b067fc640 1 --2- 172.21.15.43:0/2683195023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1be76ae10 0x55a1be76d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:49.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.128+0000 7f6b077fe640 1 --2- 172.21.15.43:0/1503236350 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6af00a3d20 0x7f6af00a61e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6b000ce670 tx=0x7f6b0006f900 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.128+0000 7f6b06ffd640 1 --2- 172.21.15.43:0/2683195023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be762290 0x55a1be76a840 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f6af800baa0 tx=0x7f6af800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b7ed500d9bebc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.128+0000 7f6aed7fa640 1 -- 172.21.15.43:0/2683195023 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af8017420 con 0x55a1be762290 2024-04-21T08:47:49.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.128+0000 7f6aed7fa640 1 -- 172.21.15.43:0/2683195023 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6af8017a70 con 0x55a1be762290 2024-04-21T08:47:49.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.128+0000 7f6aed7fa640 1 -- 172.21.15.43:0/2683195023 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af800fb20 con 0x55a1be762290 2024-04-21T08:47:49.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.129+0000 7f6b1007d740 1 -- 172.21.15.43:0/2683195023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be762290 msgr2=0x55a1be76a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.129+0000 7f6b1007d740 1 --2- 172.21.15.43:0/2683195023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be762290 0x55a1be76a840 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f6af800baa0 tx=0x7f6af800bf60 comp rx=0 tx=0).stop 2024-04-21T08:47:49.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.129+0000 7f6b1007d740 1 -- 172.21.15.43:0/2683195023 shutdown_connections 2024-04-21T08:47:49.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.129+0000 7f6b1007d740 1 --2- 172.21.15.43:0/2683195023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1be76ae10 0x55a1be76d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.129+0000 7f6b1007d740 1 --2- 172.21.15.43:0/2683195023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be762290 0x55a1be76a840 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.129+0000 7f6b1007d740 1 --2- 172.21.15.43:0/2683195023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1be7618e0 0x55a1be761cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.129+0000 7f6b1007d740 1 -- 172.21.15.43:0/2683195023 >> 172.21.15.43:0/2683195023 conn(0x55a1be75bfb0 msgr2=0x55a1be75c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.129+0000 7f6b1007d740 1 -- 172.21.15.43:0/2683195023 shutdown_connections 2024-04-21T08:47:49.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.129+0000 7f6b1007d740 1 -- 172.21.15.43:0/2683195023 wait complete. 2024-04-21T08:47:49.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.129+0000 7f6b1007d740 1 -- start start 2024-04-21T08:47:49.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.130+0000 7f6b1007d740 1 Processor -- start 2024-04-21T08:47:49.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.130+0000 7f6b1007d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1be7618e0 0x55a1be767df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.130+0000 7f6b1007d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be762290 0x55a1be768330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.130+0000 7f6b1007d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1be76ae10 0x55a1be768870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.130+0000 7f6b1007d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a1be76dd80 con 0x55a1be762290 2024-04-21T08:47:49.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.130+0000 7f6b1007d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a1be67f580 con 0x55a1be76ae10 2024-04-21T08:47:49.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.130+0000 7f6b1007d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a1be76df00 con 0x55a1be7618e0 2024-04-21T08:47:49.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.130+0000 7f6b1007d740 10 client.0 _ll_register_callbacks cb 0x55a1be685650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:49.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.130+0000 7f6b06ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be762290 0x55a1be768330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.130+0000 7f6b06ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be762290 0x55a1be768330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57682/0 (socket says 172.21.15.43:57682) 2024-04-21T08:47:49.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.130+0000 7f6b06ffd640 1 -- 172.21.15.43:0/2861291454 learned_addr learned my addr 172.21.15.43:0/2861291454 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.130+0000 7f6b067fc640 1 --2- 172.21.15.43:0/2861291454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1be76ae10 0x55a1be768870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.130+0000 7f6b067fc640 1 --2- 172.21.15.43:0/2861291454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1be7618e0 0x55a1be767df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.130+0000 7f6b06ffd640 1 -- 172.21.15.43:0/2861291454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1be7618e0 msgr2=0x55a1be767df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.130+0000 7f6b06ffd640 1 --2- 172.21.15.43:0/2861291454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1be7618e0 0x55a1be767df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.130+0000 7f6b06ffd640 1 -- 172.21.15.43:0/2861291454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1be76ae10 msgr2=0x55a1be768870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.130+0000 7f6b06ffd640 1 --2- 172.21.15.43:0/2861291454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1be76ae10 0x55a1be768870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.130+0000 7f6b06ffd640 1 -- 172.21.15.43:0/2861291454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1be500f70 con 0x55a1be762290 2024-04-21T08:47:49.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.130+0000 7f6b067fc640 1 --2- 172.21.15.43:0/2861291454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1be7618e0 0x55a1be767df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:49.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.130+0000 7f6b06ffd640 1 --2- 172.21.15.43:0/2861291454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be762290 0x55a1be768330 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f6af800f800 tx=0x7f6af800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35a45ebc9e36381 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.130+0000 7f6aeaff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:49.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.131+0000 7f6aeaff5640 1 -- 172.21.15.43:0/2861291454 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af8025070 con 0x55a1be762290 2024-04-21T08:47:49.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.131+0000 7f6aeaff5640 1 -- 172.21.15.43:0/2861291454 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f6af8017620 con 0x55a1be762290 2024-04-21T08:47:49.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.131+0000 7f6b1007d740 1 -- 172.21.15.43:0/2861291454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6af8017a70 con 0x55a1be762290 2024-04-21T08:47:49.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.131+0000 7f6aeaff5640 1 -- 172.21.15.43:0/2861291454 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6af8031050 con 0x55a1be762290 2024-04-21T08:47:49.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.131+0000 7f6b1007d740 10 client.19644 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:49.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.131+0000 7f6b1007d740 1 -- 172.21.15.43:0/2861291454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a1be398170 con 0x55a1be762290 2024-04-21T08:47:49.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.131+0000 7f6aeaff5640 1 -- 172.21.15.43:0/2861291454 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6af8017a70 con 0x55a1be762290 2024-04-21T08:47:49.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.131+0000 7f6aeaff5640 1 -- 172.21.15.43:0/2861291454 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6af8005420 con 0x55a1be762290 2024-04-21T08:47:49.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.131+0000 7f6b1007d740 10 client.19644 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:49.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.131+0000 7f6b1007d740 10 client.19644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:49.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.131+0000 7f6b1007d740 20 client.19644 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:49.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.131+0000 7f6b1007d740 1 --2- 172.21.15.43:0/2861291454 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a1be77b280 0x55a1be77d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.131+0000 7f6b1007d740 4 client.19644 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:49.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.131+0000 7f6b1007d740 1 -- 172.21.15.43:0/2861291454 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1be6d3920 con 0x55a1be77b280 2024-04-21T08:47:49.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.132+0000 7f6b067fc640 1 --2- 172.21.15.43:0/2861291454 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a1be77b280 0x55a1be77d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.132+0000 7f6aeaff5640 10 client.19644 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:49.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.132+0000 7f6b067fc640 1 --2- 172.21.15.43:0/2861291454 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a1be77b280 0x55a1be77d670 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=200889e8a10a7add server_cookie=ea5e7bf818c3803a in_seq=0 out_seq=0 2024-04-21T08:47:49.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.133+0000 7f6aeaff5640 1 -- 172.21.15.43:0/2861291454 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a1be6d3920 con 0x55a1be77b280 2024-04-21T08:47:49.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.133+0000 7f6aeaff5640 10 client.19644 handle_command_reply: tid=0 2024-04-21T08:47:49.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.133+0000 7f6b1007d740 1 client.19644 shutdown 2024-04-21T08:47:49.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.133+0000 7f6b1007d740 1 -- 172.21.15.43:0/2861291454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be762290 msgr2=0x55a1be768330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.133+0000 7f6b1007d740 1 --2- 172.21.15.43:0/2861291454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be762290 0x55a1be768330 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f6af800f800 tx=0x7f6af800b7c0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.133+0000 7f6b1007d740 1 -- 172.21.15.43:0/2861291454 shutdown_connections 2024-04-21T08:47:49.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.133+0000 7f6b1007d740 1 --2- 172.21.15.43:0/2861291454 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a1be77b280 0x55a1be77d670 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.133+0000 7f6b1007d740 1 --2- 172.21.15.43:0/2861291454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1be76ae10 0x55a1be768870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.133+0000 7f6b1007d740 1 --2- 172.21.15.43:0/2861291454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be762290 0x55a1be768330 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.133+0000 7f6b1007d740 1 --2- 172.21.15.43:0/2861291454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1be7618e0 0x55a1be767df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.133+0000 7f6b1007d740 1 -- 172.21.15.43:0/2861291454 >> 172.21.15.43:0/2861291454 conn(0x55a1be75bfb0 msgr2=0x55a1be75dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.133+0000 7f6aeaff5640 0 client.19644 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:49.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.133+0000 7f6b1007d740 1 -- 172.21.15.43:0/2861291454 shutdown_connections 2024-04-21T08:47:49.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.133+0000 7f6b1007d740 1 -- 172.21.15.43:0/2861291454 wait complete. 2024-04-21T08:47:49.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.133+0000 7f6b1007d740 20 client.19644 trim_cache size 0 max 16384 2024-04-21T08:47:49.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.151+0000 7f6b1007d740 1 Processor -- start 2024-04-21T08:47:49.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.151+0000 7f6b1007d740 1 -- start start 2024-04-21T08:47:49.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.151+0000 7f6b1007d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1be7618e0 0x55a1be76a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.151+0000 7f6b1007d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1be762290 0x55a1be776c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.151+0000 7f6b1007d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be76ae10 0x55a1be7684c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.151+0000 7f6b1007d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6b081af620 con 0x55a1be76ae10 2024-04-21T08:47:49.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.151+0000 7f6b1007d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6b081b1160 con 0x55a1be762290 2024-04-21T08:47:49.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.151+0000 7f6b1007d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6b081afcf0 con 0x55a1be7618e0 2024-04-21T08:47:49.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.151+0000 7f6b067fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be76ae10 0x55a1be7684c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.151+0000 7f6b067fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be76ae10 0x55a1be7684c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57694/0 (socket says 172.21.15.43:57694) 2024-04-21T08:47:49.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.151+0000 7f6b067fc640 1 -- 172.21.15.43:0/3611866826 learned_addr learned my addr 172.21.15.43:0/3611866826 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.151+0000 7f6b06ffd640 1 --2- 172.21.15.43:0/3611866826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1be762290 0x55a1be776c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.151+0000 7f6b067fc640 1 --2- 172.21.15.43:0/3611866826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1be7618e0 0x55a1be76a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.152+0000 7f6b067fc640 1 -- 172.21.15.43:0/3611866826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1be7618e0 msgr2=0x55a1be76a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.152+0000 7f6b067fc640 1 --2- 172.21.15.43:0/3611866826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1be7618e0 0x55a1be76a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.152+0000 7f6b067fc640 1 -- 172.21.15.43:0/3611866826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1be762290 msgr2=0x55a1be776c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.152+0000 7f6b067fc640 1 --2- 172.21.15.43:0/3611866826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1be762290 0x55a1be776c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.152+0000 7f6b067fc640 1 -- 172.21.15.43:0/3611866826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1be76d590 con 0x55a1be76ae10 2024-04-21T08:47:49.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.152+0000 7f6b067fc640 1 --2- 172.21.15.43:0/3611866826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be76ae10 0x55a1be7684c0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f6afc005940 tx=0x7f6afc0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f099955e27b83a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.152+0000 7f6aecff9640 1 -- 172.21.15.43:0/3611866826 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc04f030 con 0x55a1be76ae10 2024-04-21T08:47:49.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.152+0000 7f6aecff9640 1 -- 172.21.15.43:0/3611866826 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6afc005040 con 0x55a1be76ae10 2024-04-21T08:47:49.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.152+0000 7f6aecff9640 1 -- 172.21.15.43:0/3611866826 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc0048c0 con 0x55a1be76ae10 2024-04-21T08:47:49.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.153+0000 7f6b1007d740 1 -- 172.21.15.43:0/3611866826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be76ae10 msgr2=0x55a1be7684c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.153+0000 7f6b1007d740 1 --2- 172.21.15.43:0/3611866826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be76ae10 0x55a1be7684c0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f6afc005940 tx=0x7f6afc0058b0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.153+0000 7f6b1007d740 1 -- 172.21.15.43:0/3611866826 shutdown_connections 2024-04-21T08:47:49.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.153+0000 7f6b1007d740 1 --2- 172.21.15.43:0/3611866826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be76ae10 0x55a1be7684c0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.153+0000 7f6b1007d740 1 --2- 172.21.15.43:0/3611866826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1be762290 0x55a1be776c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.153+0000 7f6b1007d740 1 --2- 172.21.15.43:0/3611866826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1be7618e0 0x55a1be76a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.153+0000 7f6b1007d740 1 -- 172.21.15.43:0/3611866826 >> 172.21.15.43:0/3611866826 conn(0x55a1be75bfb0 msgr2=0x55a1be767db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.153+0000 7f6b1007d740 1 -- 172.21.15.43:0/3611866826 shutdown_connections 2024-04-21T08:47:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.153+0000 7f6b1007d740 1 -- 172.21.15.43:0/3611866826 wait complete. 2024-04-21T08:47:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.153+0000 7f6b1007d740 1 -- start start 2024-04-21T08:47:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.153+0000 7f6b1007d740 1 Processor -- start 2024-04-21T08:47:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.153+0000 7f6b1007d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1be7618e0 0x55a1be76ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.153+0000 7f6b1007d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be762290 0x55a1be76efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.153+0000 7f6b1007d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1be76ae10 0x55a1be76f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.153+0000 7f6b1007d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6b081b0a90 con 0x55a1be762290 2024-04-21T08:47:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.153+0000 7f6b1007d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6b081b03c0 con 0x55a1be76ae10 2024-04-21T08:47:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.153+0000 7f6b1007d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6b081ad620 con 0x55a1be7618e0 2024-04-21T08:47:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.153+0000 7f6b1007d740 10 client.0 _ll_register_callbacks cb 0x7f6b000093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.153+0000 7f6b06ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be762290 0x55a1be76efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.153+0000 7f6b06ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be762290 0x55a1be76efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57702/0 (socket says 172.21.15.43:57702) 2024-04-21T08:47:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.153+0000 7f6b06ffd640 1 -- 172.21.15.43:0/2591599387 learned_addr learned my addr 172.21.15.43:0/2591599387 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.154+0000 7f6b067fc640 1 --2- 172.21.15.43:0/2591599387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1be76ae10 0x55a1be76f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.154+0000 7f6b06ffd640 1 -- 172.21.15.43:0/2591599387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1be7618e0 msgr2=0x55a1be76ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.154+0000 7f6b06ffd640 1 --2- 172.21.15.43:0/2591599387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1be7618e0 0x55a1be76ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.154+0000 7f6b06ffd640 1 -- 172.21.15.43:0/2591599387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1be76ae10 msgr2=0x55a1be76f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.154+0000 7f6b06ffd640 1 --2- 172.21.15.43:0/2591599387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1be76ae10 0x55a1be76f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.154+0000 7f6b06ffd640 1 -- 172.21.15.43:0/2591599387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1be7e0540 con 0x55a1be762290 2024-04-21T08:47:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.154+0000 7f6b06ffd640 1 --2- 172.21.15.43:0/2591599387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be762290 0x55a1be76efd0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f6af800b830 tx=0x7f6af800f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1076a1eef7286ea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.154+0000 7f6aeaff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.154+0000 7f6aeaff5640 1 -- 172.21.15.43:0/2591599387 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af8025040 con 0x55a1be762290 2024-04-21T08:47:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.154+0000 7f6aeaff5640 1 -- 172.21.15.43:0/2591599387 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f6af800fc40 con 0x55a1be762290 2024-04-21T08:47:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.154+0000 7f6b1007d740 1 -- 172.21.15.43:0/2591599387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6afc005040 con 0x55a1be762290 2024-04-21T08:47:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.154+0000 7f6aeaff5640 1 -- 172.21.15.43:0/2591599387 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6af8043050 con 0x55a1be762290 2024-04-21T08:47:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.154+0000 7f6b1007d740 10 client.19650 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.154+0000 7f6b1007d740 1 -- 172.21.15.43:0/2591599387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a1be7e1760 con 0x55a1be762290 2024-04-21T08:47:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.154+0000 7f6aeaff5640 1 -- 172.21.15.43:0/2591599387 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6afc005040 con 0x55a1be762290 2024-04-21T08:47:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.155+0000 7f6aeaff5640 1 -- 172.21.15.43:0/2591599387 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6af80179f0 con 0x55a1be762290 2024-04-21T08:47:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.155+0000 7f6b1007d740 10 client.19650 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.155+0000 7f6b1007d740 10 client.19650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.155+0000 7f6b1007d740 20 client.19650 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.155+0000 7f6b1007d740 1 --2- 172.21.15.43:0/2591599387 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a1be7e1e70 0x55a1be7e4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.155+0000 7f6b1007d740 4 client.19650 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:47:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.155+0000 7f6b1007d740 1 -- 172.21.15.43:0/2591599387 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a1be6cd790 con 0x55a1be7e1e70 2024-04-21T08:47:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.155+0000 7f6b067fc640 1 --2- 172.21.15.43:0/2591599387 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a1be7e1e70 0x55a1be7e4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.155+0000 7f6aeaff5640 10 client.19650 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.155+0000 7f6b067fc640 1 --2- 172.21.15.43:0/2591599387 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a1be7e1e70 0x55a1be7e4290 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b22032c5f755fe8 server_cookie=c8b3186c2947b193 in_seq=0 out_seq=0 2024-04-21T08:47:49.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.156+0000 7f6aeaff5640 1 -- 172.21.15.43:0/2591599387 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a1be6cd790 con 0x55a1be7e1e70 2024-04-21T08:47:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.156+0000 7f6aeaff5640 10 client.19650 handle_command_reply: tid=0 2024-04-21T08:47:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.156+0000 7f6b1007d740 1 client.19650 shutdown 2024-04-21T08:47:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.157+0000 7f6b1007d740 1 -- 172.21.15.43:0/2591599387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be762290 msgr2=0x55a1be76efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.157+0000 7f6b1007d740 1 --2- 172.21.15.43:0/2591599387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be762290 0x55a1be76efd0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f6af800b830 tx=0x7f6af800f650 comp rx=0 tx=0).stop 2024-04-21T08:47:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.157+0000 7f6b1007d740 1 -- 172.21.15.43:0/2591599387 shutdown_connections 2024-04-21T08:47:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.157+0000 7f6b1007d740 1 --2- 172.21.15.43:0/2591599387 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a1be7e1e70 0x55a1be7e4290 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.157+0000 7f6b1007d740 1 --2- 172.21.15.43:0/2591599387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1be76ae10 0x55a1be76f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.157+0000 7f6b1007d740 1 --2- 172.21.15.43:0/2591599387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a1be762290 0x55a1be76efd0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.157+0000 7f6aeaff5640 0 client.19650 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.157+0000 7f6b1007d740 1 --2- 172.21.15.43:0/2591599387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a1be7618e0 0x55a1be76ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.157+0000 7f6b1007d740 1 -- 172.21.15.43:0/2591599387 >> 172.21.15.43:0/2591599387 conn(0x55a1be75bfb0 msgr2=0x55a1be75b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.157+0000 7f6b1007d740 1 -- 172.21.15.43:0/2591599387 shutdown_connections 2024-04-21T08:47:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.157+0000 7f6b1007d740 1 -- 172.21.15.43:0/2591599387 wait complete. 2024-04-21T08:47:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.157+0000 7f6b1007d740 20 client.19650 trim_cache size 0 max 16384 2024-04-21T08:47:49.164 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:47:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:47:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "642aa83c-5851-4424-ba6f-56f9a91fbef8", 2024-04-21T08:47:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:47:49.164 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:47:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.157+0000 7f6b0d48e640 1 -- 172.21.15.43:0/1503236350 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6af00a3d20 msgr2=0x7f6af00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.157+0000 7f6b0d48e640 1 --2- 172.21.15.43:0/1503236350 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6af00a3d20 0x7f6af00a61e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6b000ce670 tx=0x7f6b0006f900 comp rx=0 tx=0).stop 2024-04-21T08:47:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.157+0000 7f6b0d48e640 1 -- 172.21.15.43:0/1503236350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b08198590 msgr2=0x7f6b08198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.157+0000 7f6b0d48e640 1 --2- 172.21.15.43:0/1503236350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b08198590 0x7f6b08198a40 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000eb10 tx=0x7f6b0000c2d0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.158+0000 7f6b0d48e640 1 -- 172.21.15.43:0/1503236350 shutdown_connections 2024-04-21T08:47:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.158+0000 7f6b0d48e640 1 --2- 172.21.15.43:0/1503236350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b08198590 0x7f6b08198a40 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.158+0000 7f6b0d48e640 1 --2- 172.21.15.43:0/1503236350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b08107270 0x7f6b0819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.158+0000 7f6b0d48e640 1 --2- 172.21.15.43:0/1503236350 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6af00a3d20 0x7f6af00a61e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.158+0000 7f6b0d48e640 1 --2- 172.21.15.43:0/1503236350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b081021a0 0x7f6b0819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.158+0000 7f6b0d48e640 1 -- 172.21.15.43:0/1503236350 >> 172.21.15.43:0/1503236350 conn(0x7f6b08079350 msgr2=0x7f6b08100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.158+0000 7f6b0d48e640 1 -- 172.21.15.43:0/1503236350 shutdown_connections 2024-04-21T08:47:49.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.158+0000 7f6b0d48e640 1 -- 172.21.15.43:0/1503236350 wait complete. 2024-04-21T08:47:49.169 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:47:49.170 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:47:49.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.274+0000 7f4815552640 1 -- 172.21.15.43:0/1630877888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481010c530 msgr2=0x7f481010e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.274+0000 7f4815552640 1 --2- 172.21.15.43:0/1630877888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481010c530 0x7f481010e9d0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f4808009ca0 tx=0x7f480802c230 comp rx=0 tx=0).stop 2024-04-21T08:47:49.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.274+0000 7f4815552640 1 -- 172.21.15.43:0/1630877888 shutdown_connections 2024-04-21T08:47:49.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.274+0000 7f4815552640 1 --2- 172.21.15.43:0/1630877888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481010c530 0x7f481010e9d0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.274+0000 7f4815552640 1 --2- 172.21.15.43:0/1630877888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4810103b10 0x7f481010bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.274+0000 7f4815552640 1 --2- 172.21.15.43:0/1630877888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48101031f0 0x7f48101035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.274+0000 7f4815552640 1 -- 172.21.15.43:0/1630877888 >> 172.21.15.43:0/1630877888 conn(0x7f48100771d0 msgr2=0x7f48100775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.274+0000 7f4815552640 1 -- 172.21.15.43:0/1630877888 shutdown_connections 2024-04-21T08:47:49.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.274+0000 7f4815552640 1 -- 172.21.15.43:0/1630877888 wait complete. 2024-04-21T08:47:49.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.275+0000 7f4815552640 1 Processor -- start 2024-04-21T08:47:49.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.275+0000 7f4815552640 1 -- start start 2024-04-21T08:47:49.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.275+0000 7f4815552640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48101031f0 0x7f48101986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.275+0000 7f4815552640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4810103b10 0x7f4810198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.275+0000 7f4815552640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f481010c530 0x7f481019cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.275+0000 7f4815552640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f48101121a0 con 0x7f48101031f0 2024-04-21T08:47:49.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.275+0000 7f4815552640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4810112020 con 0x7f481010c530 2024-04-21T08:47:49.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.275+0000 7f4815552640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4810112320 con 0x7f4810103b10 2024-04-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.275+0000 7f480effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48101031f0 0x7f48101986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.275+0000 7f480effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48101031f0 0x7f48101986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57724/0 (socket says 172.21.15.43:57724) 2024-04-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.275+0000 7f480effd640 1 -- 172.21.15.43:0/1099049214 learned_addr learned my addr 172.21.15.43:0/1099049214 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.276+0000 7f480e7fc640 1 --2- 172.21.15.43:0/1099049214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4810103b10 0x7f4810198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.276+0000 7f480f7fe640 1 --2- 172.21.15.43:0/1099049214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f481010c530 0x7f481019cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.276+0000 7f480effd640 1 -- 172.21.15.43:0/1099049214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4810103b10 msgr2=0x7f4810198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.276+0000 7f480effd640 1 --2- 172.21.15.43:0/1099049214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4810103b10 0x7f4810198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.276+0000 7f480effd640 1 -- 172.21.15.43:0/1099049214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f481010c530 msgr2=0x7f481019cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.276+0000 7f480effd640 1 --2- 172.21.15.43:0/1099049214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f481010c530 0x7f481019cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.276+0000 7f480effd640 1 -- 172.21.15.43:0/1099049214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481019d6d0 con 0x7f48101031f0 2024-04-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.276+0000 7f480e7fc640 1 --2- 172.21.15.43:0/1099049214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4810103b10 0x7f4810198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.276+0000 7f480f7fe640 1 --2- 172.21.15.43:0/1099049214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f481010c530 0x7f481019cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.276+0000 7f480effd640 1 --2- 172.21.15.43:0/1099049214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48101031f0 0x7f48101986e0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f480000d950 tx=0x7f480000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.276+0000 7f47f7fff640 1 -- 172.21.15.43:0/1099049214 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800017030 con 0x7f48101031f0 2024-04-21T08:47:49.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.276+0000 7f4815552640 1 -- 172.21.15.43:0/1099049214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481019d9c0 con 0x7f48101031f0 2024-04-21T08:47:49.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.276+0000 7f47f7fff640 1 -- 172.21.15.43:0/1099049214 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4800004020 con 0x7f48101031f0 2024-04-21T08:47:49.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.276+0000 7f47f7fff640 1 -- 172.21.15.43:0/1099049214 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48000047a0 con 0x7f48101031f0 2024-04-21T08:47:49.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.276+0000 7f4815552640 1 -- 172.21.15.43:0/1099049214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48101a5250 con 0x7f48101031f0 2024-04-21T08:47:49.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.279+0000 7f47f7fff640 1 -- 172.21.15.43:0/1099049214 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4800009a30 con 0x7f48101031f0 2024-04-21T08:47:49.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.279+0000 7f4818228740 1 Processor -- start 2024-04-21T08:47:49.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.279+0000 7f4818228740 1 -- start start 2024-04-21T08:47:49.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.279+0000 7f4818228740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a831ca580 0x563a831ca960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.279+0000 7f4818228740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831caea0 0x563a831d3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.279+0000 7f4818228740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a831d3a80 0x563a831d5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.279+0000 7f4818228740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563a8312f1e0 con 0x563a831caea0 2024-04-21T08:47:49.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.279+0000 7f4818228740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563a830cc390 con 0x563a831ca580 2024-04-21T08:47:49.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.279+0000 7f4818228740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563a8312b1a0 con 0x563a831d3a80 2024-04-21T08:47:49.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.279+0000 7f480effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831caea0 0x563a831d3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.279+0000 7f480effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831caea0 0x563a831d3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57736/0 (socket says 172.21.15.43:57736) 2024-04-21T08:47:49.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.279+0000 7f480effd640 1 -- 172.21.15.43:0/463161256 learned_addr learned my addr 172.21.15.43:0/463161256 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.280+0000 7f480f7fe640 1 --2- 172.21.15.43:0/463161256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a831ca580 0x563a831ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.280+0000 7f480e7fc640 1 --2- 172.21.15.43:0/463161256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a831d3a80 0x563a831d5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.280+0000 7f480effd640 1 -- 172.21.15.43:0/463161256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a831d3a80 msgr2=0x563a831d5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.280+0000 7f480effd640 1 --2- 172.21.15.43:0/463161256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a831d3a80 0x563a831d5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.280+0000 7f480effd640 1 -- 172.21.15.43:0/463161256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a831ca580 msgr2=0x563a831ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.280+0000 7f480effd640 1 --2- 172.21.15.43:0/463161256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a831ca580 0x563a831ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.280+0000 7f480effd640 1 -- 172.21.15.43:0/463161256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a82f0b7d0 con 0x563a831caea0 2024-04-21T08:47:49.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.280+0000 7f480e7fc640 1 --2- 172.21.15.43:0/463161256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a831d3a80 0x563a831d5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:49.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.280+0000 7f47f7fff640 1 --2- 172.21.15.43:0/1099049214 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f47f80a3d20 0x7f47f80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.280+0000 7f480effd640 1 --2- 172.21.15.43:0/463161256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831caea0 0x563a831d3540 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f48000cd9a0 tx=0x7f48000d6ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad941da2f7fdb95b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.280+0000 7f480f7fe640 1 --2- 172.21.15.43:0/1099049214 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f47f80a3d20 0x7f47f80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.280+0000 7f47f7fff640 1 -- 172.21.15.43:0/1099049214 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f48000d0750 con 0x7f48101031f0 2024-04-21T08:47:49.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.280+0000 7f47f57fa640 1 -- 172.21.15.43:0/463161256 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800098550 con 0x563a831caea0 2024-04-21T08:47:49.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.280+0000 7f47f57fa640 1 -- 172.21.15.43:0/463161256 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f480009c9b0 con 0x563a831caea0 2024-04-21T08:47:49.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.280+0000 7f47f57fa640 1 -- 172.21.15.43:0/463161256 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480009bc10 con 0x563a831caea0 2024-04-21T08:47:49.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.281+0000 7f4818228740 1 -- 172.21.15.43:0/463161256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831caea0 msgr2=0x563a831d3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.281+0000 7f4818228740 1 --2- 172.21.15.43:0/463161256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831caea0 0x563a831d3540 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f48000cd9a0 tx=0x7f48000d6ef0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.281+0000 7f480f7fe640 1 --2- 172.21.15.43:0/1099049214 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f47f80a3d20 0x7f47f80a61e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4808006010 tx=0x7f480802e040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.281+0000 7f4818228740 1 -- 172.21.15.43:0/463161256 shutdown_connections 2024-04-21T08:47:49.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.281+0000 7f4818228740 1 --2- 172.21.15.43:0/463161256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a831d3a80 0x563a831d5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.281+0000 7f4818228740 1 --2- 172.21.15.43:0/463161256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831caea0 0x563a831d3540 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.281+0000 7f4818228740 1 --2- 172.21.15.43:0/463161256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a831ca580 0x563a831ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.281+0000 7f4818228740 1 -- 172.21.15.43:0/463161256 >> 172.21.15.43:0/463161256 conn(0x563a831c4cb0 msgr2=0x563a831c5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.281+0000 7f4818228740 1 -- 172.21.15.43:0/463161256 shutdown_connections 2024-04-21T08:47:49.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.281+0000 7f4818228740 1 -- 172.21.15.43:0/463161256 wait complete. 2024-04-21T08:47:49.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.281+0000 7f4818228740 1 -- start start 2024-04-21T08:47:49.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.282+0000 7f4818228740 1 Processor -- start 2024-04-21T08:47:49.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.282+0000 7f4818228740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831ca580 0x563a831d0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.282+0000 7f4818228740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a831caea0 0x563a831d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.282+0000 7f4818228740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a831d3a80 0x563a831d1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.282+0000 7f4818228740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563a831d6870 con 0x563a831ca580 2024-04-21T08:47:49.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.282+0000 7f4818228740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563a830e82f0 con 0x563a831d3a80 2024-04-21T08:47:49.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.282+0000 7f4818228740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563a831d69f0 con 0x563a831caea0 2024-04-21T08:47:49.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.282+0000 7f4818228740 10 client.0 _ll_register_callbacks cb 0x563a830f5580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:49.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.282+0000 7f480effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831ca580 0x563a831d0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.282+0000 7f480effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831ca580 0x563a831d0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57744/0 (socket says 172.21.15.43:57744) 2024-04-21T08:47:49.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.282+0000 7f480effd640 1 -- 172.21.15.43:0/574856391 learned_addr learned my addr 172.21.15.43:0/574856391 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.282+0000 7f480f7fe640 1 --2- 172.21.15.43:0/574856391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a831d3a80 0x563a831d1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.282+0000 7f480e7fc640 1 --2- 172.21.15.43:0/574856391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a831caea0 0x563a831d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.282+0000 7f480effd640 1 -- 172.21.15.43:0/574856391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a831caea0 msgr2=0x563a831d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.282+0000 7f480effd640 1 --2- 172.21.15.43:0/574856391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a831caea0 0x563a831d1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.282+0000 7f480effd640 1 -- 172.21.15.43:0/574856391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a831d3a80 msgr2=0x563a831d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.282+0000 7f480effd640 1 --2- 172.21.15.43:0/574856391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a831d3a80 0x563a831d1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.282+0000 7f480effd640 1 -- 172.21.15.43:0/574856391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a82e413f0 con 0x563a831ca580 2024-04-21T08:47:49.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.282+0000 7f480effd640 1 --2- 172.21.15.43:0/574856391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831ca580 0x563a831d0af0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f4800005fd0 tx=0x7f48000097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e3128ee425f94c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.282+0000 7f47f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:49.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.283+0000 7f47f2ff5640 1 -- 172.21.15.43:0/574856391 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480009bdf0 con 0x563a831ca580 2024-04-21T08:47:49.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.283+0000 7f47f2ff5640 1 -- 172.21.15.43:0/574856391 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f4800009d20 con 0x563a831ca580 2024-04-21T08:47:49.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.283+0000 7f4818228740 1 -- 172.21.15.43:0/574856391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f480009c9b0 con 0x563a831ca580 2024-04-21T08:47:49.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.283+0000 7f47f2ff5640 1 -- 172.21.15.43:0/574856391 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f480009c080 con 0x563a831ca580 2024-04-21T08:47:49.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.283+0000 7f4818228740 10 client.19662 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:49.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.283+0000 7f4818228740 1 -- 172.21.15.43:0/574856391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563a82e010f0 con 0x563a831ca580 2024-04-21T08:47:49.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.283+0000 7f47f2ff5640 1 -- 172.21.15.43:0/574856391 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f480009c9b0 con 0x563a831ca580 2024-04-21T08:47:49.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.283+0000 7f47f2ff5640 1 -- 172.21.15.43:0/574856391 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f48000cdce0 con 0x563a831ca580 2024-04-21T08:47:49.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.284+0000 7f4818228740 10 client.19662 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:49.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.284+0000 7f4818228740 10 client.19662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:49.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.284+0000 7f4818228740 20 client.19662 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:49.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.284+0000 7f4818228740 1 --2- 172.21.15.43:0/574856391 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563a831e38c0 0x563a831e5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.284+0000 7f4818228740 4 client.19662 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:49.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.284+0000 7f4818228740 1 -- 172.21.15.43:0/574856391 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a8313cc00 con 0x563a831e38c0 2024-04-21T08:47:49.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.284+0000 7f480e7fc640 1 --2- 172.21.15.43:0/574856391 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563a831e38c0 0x563a831e5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.284+0000 7f47f2ff5640 10 client.19662 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:49.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.284+0000 7f480e7fc640 1 --2- 172.21.15.43:0/574856391 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563a831e38c0 0x563a831e5cb0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9600759b989000 server_cookie=a4a8f52b8e4ed72a in_seq=0 out_seq=0 2024-04-21T08:47:49.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.285+0000 7f47f2ff5640 1 -- 172.21.15.43:0/574856391 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563a8313cc00 con 0x563a831e38c0 2024-04-21T08:47:49.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.285+0000 7f47f2ff5640 10 client.19662 handle_command_reply: tid=0 2024-04-21T08:47:49.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.285+0000 7f4818228740 1 client.19662 shutdown 2024-04-21T08:47:49.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.285+0000 7f4818228740 1 -- 172.21.15.43:0/574856391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831ca580 msgr2=0x563a831d0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.285+0000 7f4818228740 1 --2- 172.21.15.43:0/574856391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831ca580 0x563a831d0af0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f4800005fd0 tx=0x7f48000097b0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.285+0000 7f4818228740 1 -- 172.21.15.43:0/574856391 shutdown_connections 2024-04-21T08:47:49.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.285+0000 7f4818228740 1 --2- 172.21.15.43:0/574856391 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563a831e38c0 0x563a831e5cb0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.285+0000 7f4818228740 1 --2- 172.21.15.43:0/574856391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a831d3a80 0x563a831d1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.285+0000 7f4818228740 1 --2- 172.21.15.43:0/574856391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a831caea0 0x563a831d1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.285+0000 7f47f2ff5640 0 client.19662 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:49.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.285+0000 7f4818228740 1 --2- 172.21.15.43:0/574856391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831ca580 0x563a831d0af0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.285+0000 7f4818228740 1 -- 172.21.15.43:0/574856391 >> 172.21.15.43:0/574856391 conn(0x563a831c4cb0 msgr2=0x563a831c6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.285+0000 7f4818228740 1 -- 172.21.15.43:0/574856391 shutdown_connections 2024-04-21T08:47:49.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.285+0000 7f4818228740 1 -- 172.21.15.43:0/574856391 wait complete. 2024-04-21T08:47:49.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.286+0000 7f4818228740 20 client.19662 trim_cache size 0 max 16384 2024-04-21T08:47:49.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.303+0000 7f4818228740 1 Processor -- start 2024-04-21T08:47:49.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.303+0000 7f4818228740 1 -- start start 2024-04-21T08:47:49.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.303+0000 7f4818228740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a831ca580 0x563a831d3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.303+0000 7f4818228740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a831caea0 0x563a831cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.303+0000 7f4818228740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831d3a80 0x563a831d1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.303+0000 7f4818228740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f48101af680 con 0x563a831d3a80 2024-04-21T08:47:49.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.304+0000 7f4818228740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f48101b11c0 con 0x563a831ca580 2024-04-21T08:47:49.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.304+0000 7f4818228740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48101afd50 con 0x563a831caea0 2024-04-21T08:47:49.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.304+0000 7f480f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831d3a80 0x563a831d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.304+0000 7f480f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831d3a80 0x563a831d1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57750/0 (socket says 172.21.15.43:57750) 2024-04-21T08:47:49.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.304+0000 7f480f7fe640 1 -- 172.21.15.43:0/3826437223 learned_addr learned my addr 172.21.15.43:0/3826437223 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.304+0000 7f480effd640 1 --2- 172.21.15.43:0/3826437223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a831ca580 0x563a831d3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.304+0000 7f480f7fe640 1 -- 172.21.15.43:0/3826437223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a831caea0 msgr2=0x563a831cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.304+0000 7f480f7fe640 1 --2- 172.21.15.43:0/3826437223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a831caea0 0x563a831cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.304+0000 7f480f7fe640 1 -- 172.21.15.43:0/3826437223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a831ca580 msgr2=0x563a831d3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.304+0000 7f480f7fe640 1 --2- 172.21.15.43:0/3826437223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a831ca580 0x563a831d3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.304+0000 7f480f7fe640 1 -- 172.21.15.43:0/3826437223 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a831cf040 con 0x563a831d3a80 2024-04-21T08:47:49.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.304+0000 7f480f7fe640 1 --2- 172.21.15.43:0/3826437223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831d3a80 0x563a831d1390 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f4808036000 tx=0x7f480803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11fb52b34df8167c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.304+0000 7f47f4ff9640 1 -- 172.21.15.43:0/3826437223 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4808039040 con 0x563a831d3a80 2024-04-21T08:47:49.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.304+0000 7f47f4ff9640 1 -- 172.21.15.43:0/3826437223 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4808052070 con 0x563a831d3a80 2024-04-21T08:47:49.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.304+0000 7f47f4ff9640 1 -- 172.21.15.43:0/3826437223 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480804d450 con 0x563a831d3a80 2024-04-21T08:47:49.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.305+0000 7f4818228740 1 -- 172.21.15.43:0/3826437223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831d3a80 msgr2=0x563a831d1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.305+0000 7f4818228740 1 --2- 172.21.15.43:0/3826437223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831d3a80 0x563a831d1390 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f4808036000 tx=0x7f480803f000 comp rx=0 tx=0).stop 2024-04-21T08:47:49.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.305+0000 7f4818228740 1 -- 172.21.15.43:0/3826437223 shutdown_connections 2024-04-21T08:47:49.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.305+0000 7f4818228740 1 --2- 172.21.15.43:0/3826437223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831d3a80 0x563a831d1390 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.305+0000 7f4818228740 1 --2- 172.21.15.43:0/3826437223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a831caea0 0x563a831cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.305+0000 7f4818228740 1 --2- 172.21.15.43:0/3826437223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a831ca580 0x563a831d3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.305+0000 7f4818228740 1 -- 172.21.15.43:0/3826437223 >> 172.21.15.43:0/3826437223 conn(0x563a831c4cb0 msgr2=0x563a831c3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.305+0000 7f4818228740 1 -- 172.21.15.43:0/3826437223 shutdown_connections 2024-04-21T08:47:49.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.305+0000 7f4818228740 1 -- 172.21.15.43:0/3826437223 wait complete. 2024-04-21T08:47:49.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.305+0000 7f4818228740 1 -- start start 2024-04-21T08:47:49.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.306+0000 7f4818228740 1 Processor -- start 2024-04-21T08:47:49.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.306+0000 7f4818228740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a831ca580 0x563a831e9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.306+0000 7f4818228740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a831caea0 0x563a831e64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.306+0000 7f4818228740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831d3a80 0x563a831e6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.306+0000 7f4818228740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f48101b0af0 con 0x563a831d3a80 2024-04-21T08:47:49.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.306+0000 7f4818228740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f48101b0420 con 0x563a831ca580 2024-04-21T08:47:49.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.306+0000 7f4818228740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48101ad680 con 0x563a831caea0 2024-04-21T08:47:49.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.306+0000 7f4818228740 10 client.0 _ll_register_callbacks cb 0x7f48000d68e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:49.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.306+0000 7f480f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831d3a80 0x563a831e6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.306+0000 7f480f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831d3a80 0x563a831e6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57764/0 (socket says 172.21.15.43:57764) 2024-04-21T08:47:49.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.306+0000 7f480f7fe640 1 -- 172.21.15.43:0/3027744445 learned_addr learned my addr 172.21.15.43:0/3027744445 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.306+0000 7f480effd640 1 --2- 172.21.15.43:0/3027744445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a831ca580 0x563a831e9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.306+0000 7f480e7fc640 1 --2- 172.21.15.43:0/3027744445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a831caea0 0x563a831e64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.306+0000 7f480f7fe640 1 -- 172.21.15.43:0/3027744445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a831caea0 msgr2=0x563a831e64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.306+0000 7f480f7fe640 1 --2- 172.21.15.43:0/3027744445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a831caea0 0x563a831e64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.306+0000 7f480f7fe640 1 -- 172.21.15.43:0/3027744445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a831ca580 msgr2=0x563a831e9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.306+0000 7f480f7fe640 1 --2- 172.21.15.43:0/3027744445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a831ca580 0x563a831e9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.306+0000 7f480f7fe640 1 -- 172.21.15.43:0/3027744445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a831e7230 con 0x563a831d3a80 2024-04-21T08:47:49.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.307+0000 7f480f7fe640 1 --2- 172.21.15.43:0/3027744445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831d3a80 0x563a831e6a00 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f480802c770 tx=0x7f48080093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7d6b2b0a1bc025 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.307+0000 7f47f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:49.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.307+0000 7f47f2ff5640 1 -- 172.21.15.43:0/3027744445 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480805a070 con 0x563a831d3a80 2024-04-21T08:47:49.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.307+0000 7f47f2ff5640 1 -- 172.21.15.43:0/3027744445 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f4808004240 con 0x563a831d3a80 2024-04-21T08:47:49.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.307+0000 7f4818228740 1 -- 172.21.15.43:0/3027744445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4808052070 con 0x563a831d3a80 2024-04-21T08:47:49.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.307+0000 7f47f2ff5640 1 -- 172.21.15.43:0/3027744445 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4808068050 con 0x563a831d3a80 2024-04-21T08:47:49.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.307+0000 7f4818228740 10 client.19668 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:49.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.307+0000 7f4818228740 1 -- 172.21.15.43:0/3027744445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563a83249b40 con 0x563a831d3a80 2024-04-21T08:47:49.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.307+0000 7f47f2ff5640 1 -- 172.21.15.43:0/3027744445 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4808052070 con 0x563a831d3a80 2024-04-21T08:47:49.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.307+0000 7f47f2ff5640 1 -- 172.21.15.43:0/3027744445 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f480805fa30 con 0x563a831d3a80 2024-04-21T08:47:49.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.307+0000 7f4818228740 10 client.19668 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:49.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.307+0000 7f4818228740 10 client.19668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:49.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.307+0000 7f4818228740 20 client.19668 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:49.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.307+0000 7f4818228740 1 --2- 172.21.15.43:0/3027744445 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563a8324a230 0x563a8324c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.307+0000 7f4818228740 4 client.19668 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:47:49.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.307+0000 7f4818228740 1 -- 172.21.15.43:0/3027744445 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563a83136a70 con 0x563a8324a230 2024-04-21T08:47:49.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.308+0000 7f480effd640 1 --2- 172.21.15.43:0/3027744445 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563a8324a230 0x563a8324c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.308+0000 7f480effd640 1 --2- 172.21.15.43:0/3027744445 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563a8324a230 0x563a8324c650 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61a7a85f85a13685 server_cookie=69533f0d8b55d6af in_seq=0 out_seq=0 2024-04-21T08:47:49.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.308+0000 7f47f2ff5640 10 client.19668 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:49.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.308+0000 7f47f2ff5640 1 -- 172.21.15.43:0/3027744445 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563a83136a70 con 0x563a8324a230 2024-04-21T08:47:49.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.308+0000 7f47f2ff5640 10 client.19668 handle_command_reply: tid=0 2024-04-21T08:47:49.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.308+0000 7f4818228740 1 client.19668 shutdown 2024-04-21T08:47:49.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.308+0000 7f4818228740 1 -- 172.21.15.43:0/3027744445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831d3a80 msgr2=0x563a831e6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.308+0000 7f4818228740 1 --2- 172.21.15.43:0/3027744445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831d3a80 0x563a831e6a00 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f480802c770 tx=0x7f48080093f0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.308+0000 7f4818228740 1 -- 172.21.15.43:0/3027744445 shutdown_connections 2024-04-21T08:47:49.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.308+0000 7f4818228740 1 --2- 172.21.15.43:0/3027744445 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563a8324a230 0x563a8324c650 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.308+0000 7f4818228740 1 --2- 172.21.15.43:0/3027744445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a831d3a80 0x563a831e6a00 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.309+0000 7f4818228740 1 --2- 172.21.15.43:0/3027744445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563a831caea0 0x563a831e64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.309+0000 7f4818228740 1 --2- 172.21.15.43:0/3027744445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563a831ca580 0x563a831e9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.309+0000 7f4818228740 1 -- 172.21.15.43:0/3027744445 >> 172.21.15.43:0/3027744445 conn(0x563a831c4cb0 msgr2=0x563a831dec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.309+0000 7f47f2ff5640 0 client.19668 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:49.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.309+0000 7f4818228740 1 -- 172.21.15.43:0/3027744445 shutdown_connections 2024-04-21T08:47:49.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.309+0000 7f4818228740 1 -- 172.21.15.43:0/3027744445 wait complete. 2024-04-21T08:47:49.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.309+0000 7f4818228740 20 client.19668 trim_cache size 0 max 16384 2024-04-21T08:47:49.316 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:47:49.316 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:47:49.316 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:47:49.316 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:47:49.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.309+0000 7f4815552640 1 -- 172.21.15.43:0/1099049214 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f47f80a3d20 msgr2=0x7f47f80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.309+0000 7f4815552640 1 --2- 172.21.15.43:0/1099049214 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f47f80a3d20 0x7f47f80a61e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4808006010 tx=0x7f480802e040 comp rx=0 tx=0).stop 2024-04-21T08:47:49.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.309+0000 7f4815552640 1 -- 172.21.15.43:0/1099049214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48101031f0 msgr2=0x7f48101986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.309+0000 7f4815552640 1 --2- 172.21.15.43:0/1099049214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48101031f0 0x7f48101986e0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f480000d950 tx=0x7f480000de10 comp rx=0 tx=0).stop 2024-04-21T08:47:49.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.309+0000 7f4815552640 1 -- 172.21.15.43:0/1099049214 shutdown_connections 2024-04-21T08:47:49.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.309+0000 7f4815552640 1 --2- 172.21.15.43:0/1099049214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f481010c530 0x7f481019cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.309+0000 7f4815552640 1 --2- 172.21.15.43:0/1099049214 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f47f80a3d20 0x7f47f80a61e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.309+0000 7f4815552640 1 --2- 172.21.15.43:0/1099049214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4810103b10 0x7f4810198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.309+0000 7f4815552640 1 --2- 172.21.15.43:0/1099049214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48101031f0 0x7f48101986e0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.309+0000 7f4815552640 1 -- 172.21.15.43:0/1099049214 >> 172.21.15.43:0/1099049214 conn(0x7f48100771d0 msgr2=0x7f48100fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.309+0000 7f4815552640 1 -- 172.21.15.43:0/1099049214 shutdown_connections 2024-04-21T08:47:49.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.309+0000 7f4815552640 1 -- 172.21.15.43:0/1099049214 wait complete. 2024-04-21T08:47:49.321 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:47:49.321 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:47:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.322+0000 7ffa6742e640 1 -- 172.21.15.43:0/3706545225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa60103b00 msgr2=0x7ffa6010bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.322+0000 7ffa6742e640 1 --2- 172.21.15.43:0/3706545225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa60103b00 0x7ffa6010bfe0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c009900 tx=0x7ffa5c02bd60 comp rx=0 tx=0).stop 2024-04-21T08:47:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.322+0000 7ffa6742e640 1 -- 172.21.15.43:0/3706545225 shutdown_connections 2024-04-21T08:47:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.322+0000 7ffa6742e640 1 --2- 172.21.15.43:0/3706545225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa6010c520 0x7ffa6010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.322+0000 7ffa6742e640 1 --2- 172.21.15.43:0/3706545225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa60103b00 0x7ffa6010bfe0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.322+0000 7ffa6742e640 1 --2- 172.21.15.43:0/3706545225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa601031e0 0x7ffa601035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.322+0000 7ffa6742e640 1 -- 172.21.15.43:0/3706545225 >> 172.21.15.43:0/3706545225 conn(0x7ffa60077fd0 msgr2=0x7ffa600783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.322+0000 7ffa6742e640 1 -- 172.21.15.43:0/3706545225 shutdown_connections 2024-04-21T08:47:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.322+0000 7ffa6742e640 1 -- 172.21.15.43:0/3706545225 wait complete. 2024-04-21T08:47:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.322+0000 7ffa6742e640 1 Processor -- start 2024-04-21T08:47:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.322+0000 7ffa6742e640 1 -- start start 2024-04-21T08:47:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.323+0000 7ffa6742e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa601031e0 0x7ffa60100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.323+0000 7ffa6742e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa60103b00 0x7ffa60101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.323+0000 7ffa6742e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa6010c520 0x7ffa601018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.323+0000 7ffa6742e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffa601121b0 con 0x7ffa601031e0 2024-04-21T08:47:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.323+0000 7ffa6742e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffa60112030 con 0x7ffa60103b00 2024-04-21T08:47:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.323+0000 7ffa6742e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa60112330 con 0x7ffa6010c520 2024-04-21T08:47:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.323+0000 7ffa651ca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa601031e0 0x7ffa60100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.323+0000 7ffa651ca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa601031e0 0x7ffa60100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57790/0 (socket says 172.21.15.43:57790) 2024-04-21T08:47:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.323+0000 7ffa651ca640 1 -- 172.21.15.43:0/4126914568 learned_addr learned my addr 172.21.15.43:0/4126914568 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.323+0000 7ffa659cb640 1 --2- 172.21.15.43:0/4126914568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa6010c520 0x7ffa601018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.323+0000 7ffa649c9640 1 --2- 172.21.15.43:0/4126914568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa60103b00 0x7ffa60101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.323+0000 7ffa651ca640 1 -- 172.21.15.43:0/4126914568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa6010c520 msgr2=0x7ffa601018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.323+0000 7ffa651ca640 1 --2- 172.21.15.43:0/4126914568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa6010c520 0x7ffa601018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.323+0000 7ffa651ca640 1 -- 172.21.15.43:0/4126914568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa60103b00 msgr2=0x7ffa60101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.323+0000 7ffa651ca640 1 --2- 172.21.15.43:0/4126914568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa60103b00 0x7ffa60101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.323+0000 7ffa651ca640 1 -- 172.21.15.43:0/4126914568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa601a6bd0 con 0x7ffa601031e0 2024-04-21T08:47:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.323+0000 7ffa649c9640 1 --2- 172.21.15.43:0/4126914568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa60103b00 0x7ffa60101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.323+0000 7ffa659cb640 1 --2- 172.21.15.43:0/4126914568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa6010c520 0x7ffa601018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.323+0000 7ffa651ca640 1 --2- 172.21.15.43:0/4126914568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa601031e0 0x7ffa60100e30 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000e9f0 tx=0x7ffa5000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.323+0000 7ffa567fc640 1 -- 172.21.15.43:0/4126914568 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5000cd80 con 0x7ffa601031e0 2024-04-21T08:47:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.324+0000 7ffa567fc640 1 -- 172.21.15.43:0/4126914568 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa500054d0 con 0x7ffa601031e0 2024-04-21T08:47:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.324+0000 7ffa6742e640 1 -- 172.21.15.43:0/4126914568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa601a6ec0 con 0x7ffa601031e0 2024-04-21T08:47:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.324+0000 7ffa567fc640 1 -- 172.21.15.43:0/4126914568 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa50010690 con 0x7ffa601031e0 2024-04-21T08:47:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.324+0000 7ffa6742e640 1 -- 172.21.15.43:0/4126914568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa601a7380 con 0x7ffa601031e0 2024-04-21T08:47:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.326+0000 7ffa567fc640 1 -- 172.21.15.43:0/4126914568 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffa50004e50 con 0x7ffa601031e0 2024-04-21T08:47:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.326+0000 7ffa6a017740 1 Processor -- start 2024-04-21T08:47:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.326+0000 7ffa6a017740 1 -- start start 2024-04-21T08:47:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.326+0000 7ffa6a017740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc4432d8d0 0x55cc4432c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.326+0000 7ffa6a017740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4432cb60 0x55cc4432cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.327+0000 7ffa6a017740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc443360e0 0x55cc443384d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.327+0000 7ffa6a017740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cc442918d0 con 0x55cc4432d8d0 2024-04-21T08:47:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.327+0000 7ffa6a017740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cc4422ee80 con 0x55cc443360e0 2024-04-21T08:47:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.327+0000 7ffa6a017740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55cc4428d890 con 0x55cc4432cb60 2024-04-21T08:47:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.327+0000 7ffa567fc640 1 --2- 172.21.15.43:0/4126914568 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ffa4c0a3e70 0x7ffa4c0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.327+0000 7ffa659cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc4432d8d0 0x55cc4432c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.327+0000 7ffa659cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc4432d8d0 0x55cc4432c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57804/0 (socket says 172.21.15.43:57804) 2024-04-21T08:47:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.327+0000 7ffa659cb640 1 -- 172.21.15.43:0/3289957337 learned_addr learned my addr 172.21.15.43:0/3289957337 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.327+0000 7ffa659cb640 1 --2- 172.21.15.43:0/4126914568 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ffa4c0a3e70 0x7ffa4c0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.327+0000 7ffa567fc640 1 -- 172.21.15.43:0/4126914568 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7ffa500d10a0 con 0x7ffa601031e0 2024-04-21T08:47:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.327+0000 7ffa651ca640 1 --2- 172.21.15.43:0/3289957337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4432cb60 0x55cc4432cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.327+0000 7ffa649c9640 1 --2- 172.21.15.43:0/3289957337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc443360e0 0x55cc443384d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.327+0000 7ffa659cb640 1 -- 172.21.15.43:0/3289957337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4432cb60 msgr2=0x55cc4432cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.327+0000 7ffa659cb640 1 --2- 172.21.15.43:0/3289957337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4432cb60 0x55cc4432cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.327+0000 7ffa659cb640 1 -- 172.21.15.43:0/3289957337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc443360e0 msgr2=0x55cc443384d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.327+0000 7ffa659cb640 1 --2- 172.21.15.43:0/3289957337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc443360e0 0x55cc443384d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.327+0000 7ffa659cb640 1 -- 172.21.15.43:0/3289957337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc44250320 con 0x55cc4432d8d0 2024-04-21T08:47:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.327+0000 7ffa649c9640 1 --2- 172.21.15.43:0/3289957337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc443360e0 0x55cc443384d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.327+0000 7ffa659cb640 1 --2- 172.21.15.43:0/3289957337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc4432d8d0 0x55cc4432c620 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7ffa58014f40 tx=0x7ffa58011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf4d2eefcb6fd11 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.327+0000 7ffa659cb640 1 --2- 172.21.15.43:0/4126914568 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ffa4c0a3e70 0x7ffa4c0a6330 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ffa58011d70 tx=0x7ffa58016000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.328+0000 7ffa4b7fe640 1 -- 172.21.15.43:0/3289957337 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa58024070 con 0x55cc4432d8d0 2024-04-21T08:47:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.328+0000 7ffa4b7fe640 1 -- 172.21.15.43:0/3289957337 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa58009cc0 con 0x55cc4432d8d0 2024-04-21T08:47:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.328+0000 7ffa4b7fe640 1 -- 172.21.15.43:0/3289957337 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa58004020 con 0x55cc4432d8d0 2024-04-21T08:47:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.328+0000 7ffa6a017740 1 -- 172.21.15.43:0/3289957337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc4432d8d0 msgr2=0x55cc4432c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.328+0000 7ffa6a017740 1 --2- 172.21.15.43:0/3289957337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc4432d8d0 0x55cc4432c620 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7ffa58014f40 tx=0x7ffa58011870 comp rx=0 tx=0).stop 2024-04-21T08:47:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.328+0000 7ffa6a017740 1 -- 172.21.15.43:0/3289957337 shutdown_connections 2024-04-21T08:47:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.328+0000 7ffa6a017740 1 --2- 172.21.15.43:0/3289957337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc443360e0 0x55cc443384d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.328+0000 7ffa6a017740 1 --2- 172.21.15.43:0/3289957337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4432cb60 0x55cc4432cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.328+0000 7ffa6a017740 1 --2- 172.21.15.43:0/3289957337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc4432d8d0 0x55cc4432c620 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.328+0000 7ffa6a017740 1 -- 172.21.15.43:0/3289957337 >> 172.21.15.43:0/3289957337 conn(0x55cc442f3c90 msgr2=0x55cc44327850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.329+0000 7ffa6a017740 1 -- 172.21.15.43:0/3289957337 shutdown_connections 2024-04-21T08:47:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.329+0000 7ffa6a017740 1 -- 172.21.15.43:0/3289957337 wait complete. 2024-04-21T08:47:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.329+0000 7ffa6a017740 1 -- start start 2024-04-21T08:47:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.329+0000 7ffa6a017740 1 Processor -- start 2024-04-21T08:47:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.329+0000 7ffa6a017740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4432cb60 0x55cc44335b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.329+0000 7ffa6a017740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc4432d8d0 0x55cc44332eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.329+0000 7ffa6a017740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc443360e0 0x55cc443335f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.329+0000 7ffa6a017740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cc44338db0 con 0x55cc443360e0 2024-04-21T08:47:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.329+0000 7ffa6a017740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cc44219eb0 con 0x55cc4432d8d0 2024-04-21T08:47:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.329+0000 7ffa6a017740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55cc44338f30 con 0x55cc4432cb60 2024-04-21T08:47:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.329+0000 7ffa6a017740 10 client.0 _ll_register_callbacks cb 0x55cc44246b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.330+0000 7ffa659cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc443360e0 0x55cc443335f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.330+0000 7ffa659cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc443360e0 0x55cc443335f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57816/0 (socket says 172.21.15.43:57816) 2024-04-21T08:47:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.330+0000 7ffa659cb640 1 -- 172.21.15.43:0/2786829649 learned_addr learned my addr 172.21.15.43:0/2786829649 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.330+0000 7ffa651ca640 1 --2- 172.21.15.43:0/2786829649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4432cb60 0x55cc44335b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.330+0000 7ffa649c9640 1 --2- 172.21.15.43:0/2786829649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc4432d8d0 0x55cc44332eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.330+0000 7ffa659cb640 1 -- 172.21.15.43:0/2786829649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4432cb60 msgr2=0x55cc44335b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.330+0000 7ffa659cb640 1 --2- 172.21.15.43:0/2786829649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4432cb60 0x55cc44335b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.330+0000 7ffa659cb640 1 -- 172.21.15.43:0/2786829649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc4432d8d0 msgr2=0x55cc44332eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.330+0000 7ffa659cb640 1 --2- 172.21.15.43:0/2786829649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc4432d8d0 0x55cc44332eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.330+0000 7ffa659cb640 1 -- 172.21.15.43:0/2786829649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc43fa4210 con 0x55cc443360e0 2024-04-21T08:47:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.330+0000 7ffa651ca640 1 --2- 172.21.15.43:0/2786829649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4432cb60 0x55cc44335b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.330+0000 7ffa649c9640 1 --2- 172.21.15.43:0/2786829649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc4432d8d0 0x55cc44332eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.330+0000 7ffa659cb640 1 --2- 172.21.15.43:0/2786829649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc443360e0 0x55cc443335f0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7ffa58003c40 tx=0x7ffa580044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4632c9ce47f40e26 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.330+0000 7ffa48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.330+0000 7ffa48ff9640 1 -- 172.21.15.43:0/2786829649 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa58024070 con 0x55cc443360e0 2024-04-21T08:47:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.330+0000 7ffa48ff9640 1 -- 172.21.15.43:0/2786829649 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7ffa580049e0 con 0x55cc443360e0 2024-04-21T08:47:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.330+0000 7ffa6a017740 1 -- 172.21.15.43:0/2786829649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa58009cc0 con 0x55cc443360e0 2024-04-21T08:47:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.330+0000 7ffa48ff9640 1 -- 172.21.15.43:0/2786829649 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffa5800fd70 con 0x55cc443360e0 2024-04-21T08:47:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.330+0000 7ffa6a017740 10 client.19680 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.330+0000 7ffa6a017740 1 -- 172.21.15.43:0/2786829649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cc44070620 con 0x55cc443360e0 2024-04-21T08:47:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.331+0000 7ffa48ff9640 1 -- 172.21.15.43:0/2786829649 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa58009cc0 con 0x55cc443360e0 2024-04-21T08:47:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.331+0000 7ffa48ff9640 1 -- 172.21.15.43:0/2786829649 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ffa5801d030 con 0x55cc443360e0 2024-04-21T08:47:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.331+0000 7ffa6a017740 10 client.19680 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.331+0000 7ffa6a017740 10 client.19680 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.331+0000 7ffa6a017740 20 client.19680 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.331+0000 7ffa6a017740 1 --2- 172.21.15.43:0/2786829649 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cc44346080 0x55cc44348470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.331+0000 7ffa6a017740 4 client.19680 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.331+0000 7ffa6a017740 1 -- 172.21.15.43:0/2786829649 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc442a5880 con 0x55cc44346080 2024-04-21T08:47:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.331+0000 7ffa651ca640 1 --2- 172.21.15.43:0/2786829649 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cc44346080 0x55cc44348470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.331+0000 7ffa651ca640 1 --2- 172.21.15.43:0/2786829649 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cc44346080 0x55cc44348470 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5422eb0a305fad70 server_cookie=19f4c71a42a47970 in_seq=0 out_seq=0 2024-04-21T08:47:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.331+0000 7ffa48ff9640 10 client.19680 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.332+0000 7ffa48ff9640 1 -- 172.21.15.43:0/2786829649 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cc442a5880 con 0x55cc44346080 2024-04-21T08:47:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.332+0000 7ffa48ff9640 10 client.19680 handle_command_reply: tid=0 2024-04-21T08:47:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.332+0000 7ffa6a017740 1 client.19680 shutdown 2024-04-21T08:47:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.332+0000 7ffa6a017740 1 -- 172.21.15.43:0/2786829649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc443360e0 msgr2=0x55cc443335f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.332+0000 7ffa6a017740 1 --2- 172.21.15.43:0/2786829649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc443360e0 0x55cc443335f0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7ffa58003c40 tx=0x7ffa580044c0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.332+0000 7ffa6a017740 1 -- 172.21.15.43:0/2786829649 shutdown_connections 2024-04-21T08:47:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.332+0000 7ffa6a017740 1 --2- 172.21.15.43:0/2786829649 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cc44346080 0x55cc44348470 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.332+0000 7ffa6a017740 1 --2- 172.21.15.43:0/2786829649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc443360e0 0x55cc443335f0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.332+0000 7ffa6a017740 1 --2- 172.21.15.43:0/2786829649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc4432d8d0 0x55cc44332eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.332+0000 7ffa48ff9640 0 client.19680 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.332+0000 7ffa6a017740 1 --2- 172.21.15.43:0/2786829649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc4432cb60 0x55cc44335b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.332+0000 7ffa6a017740 1 -- 172.21.15.43:0/2786829649 >> 172.21.15.43:0/2786829649 conn(0x55cc442f3c90 msgr2=0x55cc443295a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.332+0000 7ffa6a017740 1 -- 172.21.15.43:0/2786829649 shutdown_connections 2024-04-21T08:47:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.333+0000 7ffa6a017740 1 -- 172.21.15.43:0/2786829649 wait complete. 2024-04-21T08:47:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.333+0000 7ffa6a017740 20 client.19680 trim_cache size 0 max 16384 2024-04-21T08:47:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.348+0000 7ffa6a017740 1 Processor -- start 2024-04-21T08:47:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.348+0000 7ffa6a017740 1 -- start start 2024-04-21T08:47:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.348+0000 7ffa6a017740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc4432cb60 0x55cc44341cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.348+0000 7ffa6a017740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc4432d8d0 0x55cc4432e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.348+0000 7ffa6a017740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc443360e0 0x55cc44331eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.348+0000 7ffa6a017740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffa601b17a0 con 0x55cc4432d8d0 2024-04-21T08:47:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.348+0000 7ffa6a017740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffa601b32e0 con 0x55cc4432cb60 2024-04-21T08:47:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.348+0000 7ffa6a017740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa601b1e70 con 0x55cc443360e0 2024-04-21T08:47:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.348+0000 7ffa649c9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc4432d8d0 0x55cc4432e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.348+0000 7ffa649c9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc4432d8d0 0x55cc4432e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57820/0 (socket says 172.21.15.43:57820) 2024-04-21T08:47:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.348+0000 7ffa649c9640 1 -- 172.21.15.43:0/599713028 learned_addr learned my addr 172.21.15.43:0/599713028 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.348+0000 7ffa651ca640 1 --2- 172.21.15.43:0/599713028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc4432cb60 0x55cc44341cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.348+0000 7ffa659cb640 1 --2- 172.21.15.43:0/599713028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc443360e0 0x55cc44331eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.348+0000 7ffa649c9640 1 -- 172.21.15.43:0/599713028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc443360e0 msgr2=0x55cc44331eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.348+0000 7ffa649c9640 1 --2- 172.21.15.43:0/599713028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc443360e0 0x55cc44331eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.348+0000 7ffa649c9640 1 -- 172.21.15.43:0/599713028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc4432cb60 msgr2=0x55cc44341cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.348+0000 7ffa649c9640 1 --2- 172.21.15.43:0/599713028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc4432cb60 0x55cc44341cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.348+0000 7ffa649c9640 1 -- 172.21.15.43:0/599713028 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc4432be30 con 0x55cc4432d8d0 2024-04-21T08:47:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.349+0000 7ffa649c9640 1 --2- 172.21.15.43:0/599713028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc4432d8d0 0x55cc4432e6f0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c005ab0 tx=0x7ffa5c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410575f596f202c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.349+0000 7ffa4affd640 1 -- 172.21.15.43:0/599713028 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c042070 con 0x55cc4432d8d0 2024-04-21T08:47:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.349+0000 7ffa4affd640 1 -- 172.21.15.43:0/599713028 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa5c02cdc0 con 0x55cc4432d8d0 2024-04-21T08:47:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.349+0000 7ffa4affd640 1 -- 172.21.15.43:0/599713028 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c0046c0 con 0x55cc4432d8d0 2024-04-21T08:47:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.350+0000 7ffa6a017740 1 -- 172.21.15.43:0/599713028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc4432d8d0 msgr2=0x55cc4432e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.350+0000 7ffa6a017740 1 --2- 172.21.15.43:0/599713028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc4432d8d0 0x55cc4432e6f0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c005ab0 tx=0x7ffa5c005e80 comp rx=0 tx=0).stop 2024-04-21T08:47:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.350+0000 7ffa6a017740 1 -- 172.21.15.43:0/599713028 shutdown_connections 2024-04-21T08:47:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.350+0000 7ffa6a017740 1 --2- 172.21.15.43:0/599713028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc443360e0 0x55cc44331eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.350+0000 7ffa6a017740 1 --2- 172.21.15.43:0/599713028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc4432d8d0 0x55cc4432e6f0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.350+0000 7ffa6a017740 1 --2- 172.21.15.43:0/599713028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc4432cb60 0x55cc44341cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.350+0000 7ffa6a017740 1 -- 172.21.15.43:0/599713028 >> 172.21.15.43:0/599713028 conn(0x55cc442f3c90 msgr2=0x55cc443317a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.350+0000 7ffa6a017740 1 -- 172.21.15.43:0/599713028 shutdown_connections 2024-04-21T08:47:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.350+0000 7ffa6a017740 1 -- 172.21.15.43:0/599713028 wait complete. 2024-04-21T08:47:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.350+0000 7ffa6a017740 1 -- start start 2024-04-21T08:47:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.350+0000 7ffa6a017740 1 Processor -- start 2024-04-21T08:47:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.350+0000 7ffa6a017740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc4432cb60 0x55cc4433e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.350+0000 7ffa6a017740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc4432d8d0 0x55cc44330150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.350+0000 7ffa6a017740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc443360e0 0x55cc44330690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.350+0000 7ffa6a017740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffa601b2c10 con 0x55cc4432d8d0 2024-04-21T08:47:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.350+0000 7ffa6a017740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffa601b2540 con 0x55cc4432cb60 2024-04-21T08:47:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.350+0000 7ffa6a017740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa601af7a0 con 0x55cc443360e0 2024-04-21T08:47:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.350+0000 7ffa6a017740 10 client.0 _ll_register_callbacks cb 0x7ffa5000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.351+0000 7ffa649c9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc4432d8d0 0x55cc44330150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.351+0000 7ffa649c9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc4432d8d0 0x55cc44330150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57828/0 (socket says 172.21.15.43:57828) 2024-04-21T08:47:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.351+0000 7ffa649c9640 1 -- 172.21.15.43:0/3802423054 learned_addr learned my addr 172.21.15.43:0/3802423054 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.351+0000 7ffa651ca640 1 --2- 172.21.15.43:0/3802423054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc4432cb60 0x55cc4433e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.351+0000 7ffa659cb640 1 --2- 172.21.15.43:0/3802423054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc443360e0 0x55cc44330690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.351+0000 7ffa649c9640 1 -- 172.21.15.43:0/3802423054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc443360e0 msgr2=0x55cc44330690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.351+0000 7ffa649c9640 1 --2- 172.21.15.43:0/3802423054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc443360e0 0x55cc44330690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.351+0000 7ffa649c9640 1 -- 172.21.15.43:0/3802423054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc4432cb60 msgr2=0x55cc4433e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.351+0000 7ffa649c9640 1 --2- 172.21.15.43:0/3802423054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc4432cb60 0x55cc4433e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.351+0000 7ffa649c9640 1 -- 172.21.15.43:0/3802423054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc44330c50 con 0x55cc4432d8d0 2024-04-21T08:47:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.351+0000 7ffa649c9640 1 --2- 172.21.15.43:0/3802423054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc4432d8d0 0x55cc44330150 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c036040 tx=0x7ffa5c02cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c29989a8d46614 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.351+0000 7ffa48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.351+0000 7ffa48ff9640 1 -- 172.21.15.43:0/3802423054 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c04b020 con 0x55cc4432d8d0 2024-04-21T08:47:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.351+0000 7ffa48ff9640 1 -- 172.21.15.43:0/3802423054 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7ffa5c0035f0 con 0x55cc4432d8d0 2024-04-21T08:47:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.351+0000 7ffa6a017740 1 -- 172.21.15.43:0/3802423054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa5c02cdc0 con 0x55cc4432d8d0 2024-04-21T08:47:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.351+0000 7ffa48ff9640 1 -- 172.21.15.43:0/3802423054 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffa5c056050 con 0x55cc4432d8d0 2024-04-21T08:47:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.351+0000 7ffa6a017740 10 client.19686 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.351+0000 7ffa6a017740 1 -- 172.21.15.43:0/3802423054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cc44337710 con 0x55cc4432d8d0 2024-04-21T08:47:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.352+0000 7ffa48ff9640 1 -- 172.21.15.43:0/3802423054 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa5c02cdc0 con 0x55cc4432d8d0 2024-04-21T08:47:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.352+0000 7ffa48ff9640 1 -- 172.21.15.43:0/3802423054 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ffa5c047820 con 0x55cc4432d8d0 2024-04-21T08:47:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.352+0000 7ffa6a017740 10 client.19686 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.352+0000 7ffa6a017740 10 client.19686 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.352+0000 7ffa6a017740 20 client.19686 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.352+0000 7ffa6a017740 1 --2- 172.21.15.43:0/3802423054 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cc443ac8e0 0x55cc443aed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.352+0000 7ffa6a017740 4 client.19686 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:47:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.352+0000 7ffa6a017740 1 -- 172.21.15.43:0/3802423054 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cc442313d0 con 0x55cc443ac8e0 2024-04-21T08:47:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.352+0000 7ffa651ca640 1 --2- 172.21.15.43:0/3802423054 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cc443ac8e0 0x55cc443aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.352+0000 7ffa48ff9640 10 client.19686 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.352+0000 7ffa651ca640 1 --2- 172.21.15.43:0/3802423054 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cc443ac8e0 0x55cc443aed00 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b74f9f07bb67d989 server_cookie=3e2cbe375e46259e in_seq=0 out_seq=0 2024-04-21T08:47:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.366+0000 7ffa48ff9640 1 -- 172.21.15.43:0/3802423054 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cc442313d0 con 0x55cc443ac8e0 2024-04-21T08:47:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.366+0000 7ffa48ff9640 10 client.19686 handle_command_reply: tid=0 2024-04-21T08:47:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.366+0000 7ffa6a017740 1 client.19686 shutdown 2024-04-21T08:47:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.366+0000 7ffa6a017740 1 -- 172.21.15.43:0/3802423054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc4432d8d0 msgr2=0x55cc44330150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.366+0000 7ffa6a017740 1 --2- 172.21.15.43:0/3802423054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc4432d8d0 0x55cc44330150 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c036040 tx=0x7ffa5c02cbc0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.366+0000 7ffa6a017740 1 -- 172.21.15.43:0/3802423054 shutdown_connections 2024-04-21T08:47:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.366+0000 7ffa6a017740 1 --2- 172.21.15.43:0/3802423054 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cc443ac8e0 0x55cc443aed00 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.366+0000 7ffa6a017740 1 --2- 172.21.15.43:0/3802423054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc443360e0 0x55cc44330690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.366+0000 7ffa6a017740 1 --2- 172.21.15.43:0/3802423054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc4432d8d0 0x55cc44330150 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.366+0000 7ffa48ff9640 0 client.19686 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.366+0000 7ffa6a017740 1 --2- 172.21.15.43:0/3802423054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc4432cb60 0x55cc4433e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.366+0000 7ffa6a017740 1 -- 172.21.15.43:0/3802423054 >> 172.21.15.43:0/3802423054 conn(0x55cc442f3c90 msgr2=0x55cc44326040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.366+0000 7ffa6a017740 1 -- 172.21.15.43:0/3802423054 shutdown_connections 2024-04-21T08:47:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.367+0000 7ffa6a017740 1 -- 172.21.15.43:0/3802423054 wait complete. 2024-04-21T08:47:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.367+0000 7ffa6a017740 20 client.19686 trim_cache size 0 max 16384 2024-04-21T08:47:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:47:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:47:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:47:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:47:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.367+0000 7ffa6742e640 1 -- 172.21.15.43:0/4126914568 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ffa4c0a3e70 msgr2=0x7ffa4c0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.367+0000 7ffa6742e640 1 --2- 172.21.15.43:0/4126914568 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ffa4c0a3e70 0x7ffa4c0a6330 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ffa58011d70 tx=0x7ffa58016000 comp rx=0 tx=0).stop 2024-04-21T08:47:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.367+0000 7ffa6742e640 1 -- 172.21.15.43:0/4126914568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa601031e0 msgr2=0x7ffa60100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.367+0000 7ffa6742e640 1 --2- 172.21.15.43:0/4126914568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa601031e0 0x7ffa60100e30 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000e9f0 tx=0x7ffa5000edf0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.367+0000 7ffa6742e640 1 -- 172.21.15.43:0/4126914568 shutdown_connections 2024-04-21T08:47:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.367+0000 7ffa6742e640 1 --2- 172.21.15.43:0/4126914568 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ffa4c0a3e70 0x7ffa4c0a6330 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.367+0000 7ffa6742e640 1 --2- 172.21.15.43:0/4126914568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa6010c520 0x7ffa601018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.367+0000 7ffa6742e640 1 --2- 172.21.15.43:0/4126914568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa60103b00 0x7ffa60101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.367+0000 7ffa6742e640 1 --2- 172.21.15.43:0/4126914568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa601031e0 0x7ffa60100e30 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.367+0000 7ffa6742e640 1 -- 172.21.15.43:0/4126914568 >> 172.21.15.43:0/4126914568 conn(0x7ffa60077fd0 msgr2=0x7ffa600fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.367+0000 7ffa6742e640 1 -- 172.21.15.43:0/4126914568 shutdown_connections 2024-04-21T08:47:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:47:49.367+0000 7ffa6742e640 1 -- 172.21.15.43:0/4126914568 wait complete. 2024-04-21T08:47:49.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.428+0000 7f10668c8640 1 -- 172.21.15.43:0/3906875415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10601031d0 msgr2=0x7f10601035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.428+0000 7f10668c8640 1 --2- 172.21.15.43:0/3906875415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10601031d0 0x7f10601035b0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f1050009900 tx=0x7f105002bcd0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.428+0000 7f10668c8640 1 -- 172.21.15.43:0/3906875415 shutdown_connections 2024-04-21T08:47:49.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.428+0000 7f10668c8640 1 --2- 172.21.15.43:0/3906875415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f106010c5a0 0x7f106010e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.428+0000 7f10668c8640 1 --2- 172.21.15.43:0/3906875415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1060103af0 0x7f106010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.428+0000 7f10668c8640 1 --2- 172.21.15.43:0/3906875415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10601031d0 0x7f10601035b0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.428+0000 7f10668c8640 1 -- 172.21.15.43:0/3906875415 >> 172.21.15.43:0/3906875415 conn(0x7f10600771d0 msgr2=0x7f10600775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.428+0000 7f10668c8640 1 -- 172.21.15.43:0/3906875415 shutdown_connections 2024-04-21T08:47:49.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.428+0000 7f10668c8640 1 -- 172.21.15.43:0/3906875415 wait complete. 2024-04-21T08:47:49.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.429+0000 7f10668c8640 1 Processor -- start 2024-04-21T08:47:49.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.429+0000 7f10668c8640 1 -- start start 2024-04-21T08:47:49.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.429+0000 7f10668c8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10601031d0 0x7f1060198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.429+0000 7f10668c8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1060103af0 0x7f1060198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.429+0000 7f10668c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f106010c5a0 0x7f106019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.429+0000 7f10668c8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1060112240 con 0x7f10601031d0 2024-04-21T08:47:49.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.429+0000 7f10668c8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f10601120c0 con 0x7f106010c5a0 2024-04-21T08:47:49.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.429+0000 7f10668c8640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f10601123c0 con 0x7f1060103af0 2024-04-21T08:47:49.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.429+0000 7f105ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10601031d0 0x7f1060198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.429+0000 7f105ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10601031d0 0x7f1060198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57844/0 (socket says 172.21.15.43:57844) 2024-04-21T08:47:49.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.429+0000 7f105ffff640 1 -- 172.21.15.43:0/2941763346 learned_addr learned my addr 172.21.15.43:0/2941763346 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.429+0000 7f1064e65640 1 --2- 172.21.15.43:0/2941763346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f106010c5a0 0x7f106019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.429+0000 7f105f7fe640 1 --2- 172.21.15.43:0/2941763346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1060103af0 0x7f1060198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.430+0000 7f105ffff640 1 -- 172.21.15.43:0/2941763346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1060103af0 msgr2=0x7f1060198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.430+0000 7f105ffff640 1 --2- 172.21.15.43:0/2941763346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1060103af0 0x7f1060198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.430+0000 7f105ffff640 1 -- 172.21.15.43:0/2941763346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f106010c5a0 msgr2=0x7f106019d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.430+0000 7f105ffff640 1 --2- 172.21.15.43:0/2941763346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f106010c5a0 0x7f106019d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.430+0000 7f105ffff640 1 -- 172.21.15.43:0/2941763346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106019d750 con 0x7f10601031d0 2024-04-21T08:47:49.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.430+0000 7f1064e65640 1 --2- 172.21.15.43:0/2941763346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f106010c5a0 0x7f106019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:49.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.430+0000 7f105ffff640 1 --2- 172.21.15.43:0/2941763346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10601031d0 0x7f1060198760 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f1050005940 tx=0x7f105002d910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.430+0000 7f105d7fa640 1 -- 172.21.15.43:0/2941763346 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050043020 con 0x7f10601031d0 2024-04-21T08:47:49.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.430+0000 7f10668c8640 1 -- 172.21.15.43:0/2941763346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f106019d9e0 con 0x7f10601031d0 2024-04-21T08:47:49.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.430+0000 7f105d7fa640 1 -- 172.21.15.43:0/2941763346 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f105002dd10 con 0x7f10601031d0 2024-04-21T08:47:49.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.430+0000 7f105d7fa640 1 -- 172.21.15.43:0/2941763346 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050003b80 con 0x7f10601031d0 2024-04-21T08:47:49.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.430+0000 7f10668c8640 1 -- 172.21.15.43:0/2941763346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10601a52d0 con 0x7f10601031d0 2024-04-21T08:47:49.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.431+0000 7f10668c8640 1 -- 172.21.15.43:0/2941763346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1060069230 con 0x7f10601031d0 2024-04-21T08:47:49.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.432+0000 7f105d7fa640 1 -- 172.21.15.43:0/2941763346 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1050046020 con 0x7f10601031d0 2024-04-21T08:47:49.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.433+0000 7f105d7fa640 1 --2- 172.21.15.43:0/2941763346 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f10480a3d20 0x7f10480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.433+0000 7f105f7fe640 1 --2- 172.21.15.43:0/2941763346 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f10480a3d20 0x7f10480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.433+0000 7f105d7fa640 1 -- 172.21.15.43:0/2941763346 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f10500f2e10 con 0x7f10601031d0 2024-04-21T08:47:49.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.434+0000 7f105f7fe640 1 --2- 172.21.15.43:0/2941763346 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f10480a3d20 0x7f10480a61e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f10601006a0 tx=0x7f105400a4c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.438+0000 7f105d7fa640 1 -- 172.21.15.43:0/2941763346 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f105004d020 con 0x7f10601031d0 2024-04-21T08:47:49.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.663+0000 7f10668c8640 1 -- 172.21.15.43:0/2941763346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1060107690 con 0x7f10601031d0 2024-04-21T08:47:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.663+0000 7f105d7fa640 1 -- 172.21.15.43:0/2941763346 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11935 (secure 0 0 0) 0x7f10500bf1a0 con 0x7f10601031d0 2024-04-21T08:47:49.664 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:47:49.664 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:46:17.283806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27556":{"gid":27556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/229985237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":229985237},{"type":"v1","addr":"172.21.15.175:6837","nonce":229985237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24425,24412,24487,24527,24491]},"id":1}]} 2024-04-21T08:47:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-04-21T08:47:49.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.666+0000 7f10668c8640 1 -- 172.21.15.43:0/2941763346 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f10480a3d20 msgr2=0x7f10480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.666+0000 7f10668c8640 1 --2- 172.21.15.43:0/2941763346 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f10480a3d20 0x7f10480a61e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f10601006a0 tx=0x7f105400a4c0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.666+0000 7f10668c8640 1 -- 172.21.15.43:0/2941763346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10601031d0 msgr2=0x7f1060198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.666+0000 7f10668c8640 1 --2- 172.21.15.43:0/2941763346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10601031d0 0x7f1060198760 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f1050005940 tx=0x7f105002d910 comp rx=0 tx=0).stop 2024-04-21T08:47:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.666+0000 7f10668c8640 1 -- 172.21.15.43:0/2941763346 shutdown_connections 2024-04-21T08:47:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.666+0000 7f10668c8640 1 --2- 172.21.15.43:0/2941763346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f106010c5a0 0x7f106019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.666+0000 7f10668c8640 1 --2- 172.21.15.43:0/2941763346 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f10480a3d20 0x7f10480a61e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.666+0000 7f10668c8640 1 --2- 172.21.15.43:0/2941763346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1060103af0 0x7f1060198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.666+0000 7f10668c8640 1 --2- 172.21.15.43:0/2941763346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10601031d0 0x7f1060198760 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.666+0000 7f10668c8640 1 -- 172.21.15.43:0/2941763346 >> 172.21.15.43:0/2941763346 conn(0x7f10600771d0 msgr2=0x7f10600fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.666+0000 7f10668c8640 1 -- 172.21.15.43:0/2941763346 shutdown_connections 2024-04-21T08:47:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.666+0000 7f10668c8640 1 -- 172.21.15.43:0/2941763346 wait complete. 2024-04-21T08:47:49.679 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:47:49.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.823+0000 7f33d3a9a640 1 -- 172.21.15.43:0/3990086857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33cc1031d0 msgr2=0x7f33cc1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.823+0000 7f33d3a9a640 1 --2- 172.21.15.43:0/3990086857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33cc1031d0 0x7f33cc1035b0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009ca0 tx=0x7f33c802c230 comp rx=0 tx=0).stop 2024-04-21T08:47:49.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.823+0000 7f33d3a9a640 1 -- 172.21.15.43:0/3990086857 shutdown_connections 2024-04-21T08:47:49.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.823+0000 7f33d3a9a640 1 --2- 172.21.15.43:0/3990086857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33cc10c510 0x7f33cc10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.823+0000 7f33d3a9a640 1 --2- 172.21.15.43:0/3990086857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33cc103af0 0x7f33cc10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.823+0000 7f33d3a9a640 1 --2- 172.21.15.43:0/3990086857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33cc1031d0 0x7f33cc1035b0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.823+0000 7f33d3a9a640 1 -- 172.21.15.43:0/3990086857 >> 172.21.15.43:0/3990086857 conn(0x7f33cc077f50 msgr2=0x7f33cc078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.823+0000 7f33d3a9a640 1 -- 172.21.15.43:0/3990086857 shutdown_connections 2024-04-21T08:47:49.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.823+0000 7f33d3a9a640 1 -- 172.21.15.43:0/3990086857 wait complete. 2024-04-21T08:47:49.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.823+0000 7f33d3a9a640 1 Processor -- start 2024-04-21T08:47:49.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.823+0000 7f33d3a9a640 1 -- start start 2024-04-21T08:47:49.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.824+0000 7f33d3a9a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33cc1031d0 0x7f33cc19e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.824+0000 7f33d3a9a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33cc103af0 0x7f33cc19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.824+0000 7f33d3a9a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33cc10c510 0x7f33cc1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.824+0000 7f33d3a9a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc1121a0 con 0x7f33cc10c510 2024-04-21T08:47:49.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.824+0000 7f33d3a9a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc112020 con 0x7f33cc103af0 2024-04-21T08:47:49.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.824+0000 7f33d3a9a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc112320 con 0x7f33cc1031d0 2024-04-21T08:47:49.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.824+0000 7f33d2037640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33cc10c510 0x7f33cc1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.824+0000 7f33d2037640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33cc10c510 0x7f33cc1985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57868/0 (socket says 172.21.15.43:57868) 2024-04-21T08:47:49.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.824+0000 7f33d2037640 1 -- 172.21.15.43:0/316176561 learned_addr learned my addr 172.21.15.43:0/316176561 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.824+0000 7f33d1836640 1 --2- 172.21.15.43:0/316176561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33cc1031d0 0x7f33cc19e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.824+0000 7f33d1035640 1 --2- 172.21.15.43:0/316176561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33cc103af0 0x7f33cc19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.824+0000 7f33d2037640 1 -- 172.21.15.43:0/316176561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33cc1031d0 msgr2=0x7f33cc19e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.824+0000 7f33d2037640 1 --2- 172.21.15.43:0/316176561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33cc1031d0 0x7f33cc19e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.824+0000 7f33d2037640 1 -- 172.21.15.43:0/316176561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33cc103af0 msgr2=0x7f33cc19e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.824+0000 7f33d2037640 1 --2- 172.21.15.43:0/316176561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33cc103af0 0x7f33cc19e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.824+0000 7f33d2037640 1 -- 172.21.15.43:0/316176561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33cc198d30 con 0x7f33cc10c510 2024-04-21T08:47:49.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.824+0000 7f33d1035640 1 --2- 172.21.15.43:0/316176561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33cc103af0 0x7f33cc19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:49.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.824+0000 7f33d1836640 1 --2- 172.21.15.43:0/316176561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33cc1031d0 0x7f33cc19e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:49.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.824+0000 7f33d2037640 1 --2- 172.21.15.43:0/316176561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33cc10c510 0x7f33cc1985b0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f33c400d750 tx=0x7f33c400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.824+0000 7f33c2ffd640 1 -- 172.21.15.43:0/316176561 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4017070 con 0x7f33cc10c510 2024-04-21T08:47:49.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.824+0000 7f33d3a9a640 1 -- 172.21.15.43:0/316176561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33cc199020 con 0x7f33cc10c510 2024-04-21T08:47:49.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.824+0000 7f33c2ffd640 1 -- 172.21.15.43:0/316176561 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33c40054d0 con 0x7f33cc10c510 2024-04-21T08:47:49.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.824+0000 7f33c2ffd640 1 -- 172.21.15.43:0/316176561 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4004c60 con 0x7f33cc10c510 2024-04-21T08:47:49.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.824+0000 7f33d3a9a640 1 -- 172.21.15.43:0/316176561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33cc1a52c0 con 0x7f33cc10c510 2024-04-21T08:47:49.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.827+0000 7f33d676e740 1 Processor -- start 2024-04-21T08:47:49.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.827+0000 7f33c2ffd640 1 -- 172.21.15.43:0/316176561 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f33c4009bd0 con 0x7f33cc10c510 2024-04-21T08:47:49.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.827+0000 7f33d676e740 1 -- start start 2024-04-21T08:47:49.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.827+0000 7f33d676e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0351c250 0x557c0351c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.827+0000 7f33d676e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c0351cc00 0x557c03529840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.827+0000 7f33d676e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c03529e10 0x557c0352c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.827+0000 7f33d676e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557c034811f0 con 0x557c0351c250 2024-04-21T08:47:49.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.827+0000 7f33d676e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557c0341e4b0 con 0x557c03529e10 2024-04-21T08:47:49.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.827+0000 7f33d676e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557c0347d1b0 con 0x557c0351cc00 2024-04-21T08:47:49.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.827+0000 7f33d1035640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0351c250 0x557c0351c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.827+0000 7f33d1035640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0351c250 0x557c0351c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57872/0 (socket says 172.21.15.43:57872) 2024-04-21T08:47:49.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.828+0000 7f33d1035640 1 -- 172.21.15.43:0/746445421 learned_addr learned my addr 172.21.15.43:0/746445421 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.828+0000 7f33c2ffd640 1 --2- 172.21.15.43:0/316176561 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f33b80a3d20 0x7f33b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.828+0000 7f33d1035640 1 --2- 172.21.15.43:0/746445421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c03529e10 0x557c0352c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.828+0000 7f33d1836640 1 --2- 172.21.15.43:0/746445421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c0351cc00 0x557c03529840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.828+0000 7f33d2037640 1 --2- 172.21.15.43:0/316176561 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f33b80a3d20 0x7f33b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.828+0000 7f33d1035640 1 -- 172.21.15.43:0/746445421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c0351cc00 msgr2=0x557c03529840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.828+0000 7f33d1035640 1 --2- 172.21.15.43:0/746445421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c0351cc00 0x557c03529840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.828+0000 7f33d1035640 1 -- 172.21.15.43:0/746445421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c03529e10 msgr2=0x557c0352c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.828+0000 7f33d1035640 1 --2- 172.21.15.43:0/746445421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c03529e10 0x557c0352c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.828+0000 7f33d1035640 1 -- 172.21.15.43:0/746445421 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c034101b0 con 0x557c0351c250 2024-04-21T08:47:49.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.828+0000 7f33c2ffd640 1 -- 172.21.15.43:0/316176561 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f33c40d0600 con 0x7f33cc10c510 2024-04-21T08:47:49.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.828+0000 7f33d1035640 1 --2- 172.21.15.43:0/746445421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0351c250 0x557c0351c630 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f33bc013610 tx=0x7f33bc00b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5323d71adf705aaf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.828+0000 7f33b7fff640 1 -- 172.21.15.43:0/746445421 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc019030 con 0x557c0351c250 2024-04-21T08:47:49.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.828+0000 7f33d2037640 1 --2- 172.21.15.43:0/316176561 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f33b80a3d20 0x7f33b80a61e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f33c40cd9e0 tx=0x7f33c406ec70 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.828+0000 7f33b7fff640 1 -- 172.21.15.43:0/746445421 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33bc0054d0 con 0x557c0351c250 2024-04-21T08:47:49.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.828+0000 7f33b7fff640 1 -- 172.21.15.43:0/746445421 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc011540 con 0x557c0351c250 2024-04-21T08:47:49.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.829+0000 7f33d676e740 1 -- 172.21.15.43:0/746445421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0351c250 msgr2=0x557c0351c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.829+0000 7f33d676e740 1 --2- 172.21.15.43:0/746445421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0351c250 0x557c0351c630 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f33bc013610 tx=0x7f33bc00b6f0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.829+0000 7f33d676e740 1 -- 172.21.15.43:0/746445421 shutdown_connections 2024-04-21T08:47:49.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.829+0000 7f33d676e740 1 --2- 172.21.15.43:0/746445421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c03529e10 0x557c0352c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.829+0000 7f33d676e740 1 --2- 172.21.15.43:0/746445421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c0351cc00 0x557c03529840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.829+0000 7f33d676e740 1 --2- 172.21.15.43:0/746445421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0351c250 0x557c0351c630 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.829+0000 7f33d676e740 1 -- 172.21.15.43:0/746445421 >> 172.21.15.43:0/746445421 conn(0x557c03516ba0 msgr2=0x557c03516f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.829+0000 7f33d676e740 1 -- 172.21.15.43:0/746445421 shutdown_connections 2024-04-21T08:47:49.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.829+0000 7f33d676e740 1 -- 172.21.15.43:0/746445421 wait complete. 2024-04-21T08:47:49.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.829+0000 7f33d676e740 1 -- start start 2024-04-21T08:47:49.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.829+0000 7f33d676e740 1 Processor -- start 2024-04-21T08:47:49.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.830+0000 7f33d676e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0351c250 0x557c03529660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.830+0000 7f33d676e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c0351cc00 0x557c03526a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.830+0000 7f33d676e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c03529e10 0x557c03526fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.830+0000 7f33d676e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557c0352cd40 con 0x557c0351c250 2024-04-21T08:47:49.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.830+0000 7f33d676e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557c0343a500 con 0x557c03529e10 2024-04-21T08:47:49.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.830+0000 7f33d676e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557c0352cec0 con 0x557c0351cc00 2024-04-21T08:47:49.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.830+0000 7f33d676e740 10 client.0 _ll_register_callbacks cb 0x557c03408b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:49.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.830+0000 7f33d1035640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0351c250 0x557c03529660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.830+0000 7f33d1035640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0351c250 0x557c03529660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57876/0 (socket says 172.21.15.43:57876) 2024-04-21T08:47:49.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.830+0000 7f33d1035640 1 -- 172.21.15.43:0/3224731001 learned_addr learned my addr 172.21.15.43:0/3224731001 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.830+0000 7f33d1836640 1 --2- 172.21.15.43:0/3224731001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c0351cc00 0x557c03526a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.830+0000 7f33d1035640 1 -- 172.21.15.43:0/3224731001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c0351cc00 msgr2=0x557c03526a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.830+0000 7f33d1035640 1 --2- 172.21.15.43:0/3224731001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c0351cc00 0x557c03526a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.830+0000 7f33d1035640 1 -- 172.21.15.43:0/3224731001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c03529e10 msgr2=0x557c03526fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.830+0000 7f33d1035640 1 --2- 172.21.15.43:0/3224731001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c03529e10 0x557c03526fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.830+0000 7f33d1035640 1 -- 172.21.15.43:0/3224731001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c031933f0 con 0x557c0351c250 2024-04-21T08:47:49.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.830+0000 7f33d1035640 1 --2- 172.21.15.43:0/3224731001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0351c250 0x557c03529660 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f33bc005bc0 tx=0x7f33bc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82110b7b51cf9383 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.830+0000 7f33b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:49.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.831+0000 7f33b57fa640 1 -- 172.21.15.43:0/3224731001 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc019050 con 0x557c0351c250 2024-04-21T08:47:49.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.831+0000 7f33b57fa640 1 -- 172.21.15.43:0/3224731001 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f33bc005150 con 0x557c0351c250 2024-04-21T08:47:49.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.831+0000 7f33d676e740 1 -- 172.21.15.43:0/3224731001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33bc0054d0 con 0x557c0351c250 2024-04-21T08:47:49.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.831+0000 7f33b57fa640 1 -- 172.21.15.43:0/3224731001 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f33bc032050 con 0x557c0351c250 2024-04-21T08:47:49.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.831+0000 7f33d676e740 10 client.19704 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:49.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.831+0000 7f33d676e740 1 -- 172.21.15.43:0/3224731001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557c0325d7d0 con 0x557c0351c250 2024-04-21T08:47:49.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.831+0000 7f33b57fa640 1 -- 172.21.15.43:0/3224731001 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33bc0054d0 con 0x557c0351c250 2024-04-21T08:47:49.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.831+0000 7f33b57fa640 1 -- 172.21.15.43:0/3224731001 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f33bc02c4e0 con 0x557c0351c250 2024-04-21T08:47:49.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.831+0000 7f33d676e740 10 client.19704 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:49.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.831+0000 7f33d676e740 10 client.19704 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:47:49.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.831+0000 7f33d676e740 20 client.19704 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:49.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.831+0000 7f33d676e740 1 --2- 172.21.15.43:0/3224731001 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x557c03539f30 0x557c0353c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.831+0000 7f33d676e740 4 client.19704 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:49.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.831+0000 7f33d676e740 1 -- 172.21.15.43:0/3224731001 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c03420930 con 0x557c03539f30 2024-04-21T08:47:49.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.832+0000 7f33d1836640 1 --2- 172.21.15.43:0/3224731001 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x557c03539f30 0x557c0353c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.832+0000 7f33d1836640 1 --2- 172.21.15.43:0/3224731001 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x557c03539f30 0x557c0353c320 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4dcc6429945786e5 server_cookie=f6cec7aca7f70e4f in_seq=0 out_seq=0 2024-04-21T08:47:49.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.832+0000 7f33b57fa640 10 client.19704 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:47:49.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.833+0000 7f33b57fa640 1 -- 172.21.15.43:0/3224731001 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557c03420930 con 0x557c03539f30 2024-04-21T08:47:49.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.833+0000 7f33b57fa640 10 client.19704 handle_command_reply: tid=0 2024-04-21T08:47:49.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.833+0000 7f33d676e740 1 client.19704 shutdown 2024-04-21T08:47:49.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.833+0000 7f33d676e740 1 -- 172.21.15.43:0/3224731001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0351c250 msgr2=0x557c03529660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.833+0000 7f33d676e740 1 --2- 172.21.15.43:0/3224731001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0351c250 0x557c03529660 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f33bc005bc0 tx=0x7f33bc00b840 comp rx=0 tx=0).stop 2024-04-21T08:47:49.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.833+0000 7f33d676e740 1 -- 172.21.15.43:0/3224731001 shutdown_connections 2024-04-21T08:47:49.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.833+0000 7f33d676e740 1 --2- 172.21.15.43:0/3224731001 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x557c03539f30 0x557c0353c320 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.833+0000 7f33d676e740 1 --2- 172.21.15.43:0/3224731001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c03529e10 0x557c03526fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.833+0000 7f33b57fa640 0 client.19704 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:47:49.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.833+0000 7f33d676e740 1 --2- 172.21.15.43:0/3224731001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c0351cc00 0x557c03526a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.833+0000 7f33d676e740 1 --2- 172.21.15.43:0/3224731001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0351c250 0x557c03529660 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.833+0000 7f33d676e740 1 -- 172.21.15.43:0/3224731001 >> 172.21.15.43:0/3224731001 conn(0x557c03516ba0 msgr2=0x557c035188f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.833+0000 7f33d676e740 1 -- 172.21.15.43:0/3224731001 shutdown_connections 2024-04-21T08:47:49.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.833+0000 7f33d676e740 1 -- 172.21.15.43:0/3224731001 wait complete. 2024-04-21T08:47:49.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.833+0000 7f33d676e740 20 client.19704 trim_cache size 0 max 16384 2024-04-21T08:47:49.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.847+0000 7f33d676e740 1 Processor -- start 2024-04-21T08:47:49.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.847+0000 7f33d676e740 1 -- start start 2024-04-21T08:47:49.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.847+0000 7f33d676e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c0351c250 0x557c03535c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.847+0000 7f33d676e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c0351cc00 0x557c03521ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.847+0000 7f33d676e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c03529e10 0x557c03529190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.847+0000 7f33d676e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc1ad2b0 con 0x557c03529e10 2024-04-21T08:47:49.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.847+0000 7f33d676e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc1aedf0 con 0x557c0351cc00 2024-04-21T08:47:49.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.847+0000 7f33d676e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc1ad980 con 0x557c0351c250 2024-04-21T08:47:49.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.847+0000 7f33d1035640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c03529e10 0x557c03529190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.847+0000 7f33d1035640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c03529e10 0x557c03529190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57880/0 (socket says 172.21.15.43:57880) 2024-04-21T08:47:49.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.847+0000 7f33d1035640 1 -- 172.21.15.43:0/433463978 learned_addr learned my addr 172.21.15.43:0/433463978 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.847+0000 7f33d1836640 1 --2- 172.21.15.43:0/433463978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c0351cc00 0x557c03521ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.847+0000 7f33d1035640 1 -- 172.21.15.43:0/433463978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c0351c250 msgr2=0x557c03535c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.847+0000 7f33d1035640 1 --2- 172.21.15.43:0/433463978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c0351c250 0x557c03535c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.847+0000 7f33d1035640 1 -- 172.21.15.43:0/433463978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c0351cc00 msgr2=0x557c03521ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.847+0000 7f33d1035640 1 --2- 172.21.15.43:0/433463978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c0351cc00 0x557c03521ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.847+0000 7f33d1035640 1 -- 172.21.15.43:0/433463978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c0352ced0 con 0x557c03529e10 2024-04-21T08:47:49.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.847+0000 7f33d1035640 1 --2- 172.21.15.43:0/433463978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c03529e10 0x557c03529190 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f33bc004700 tx=0x7f33bc005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588c6571f0800a61 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.848+0000 7f33b77fe640 1 -- 172.21.15.43:0/433463978 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc019070 con 0x557c03529e10 2024-04-21T08:47:49.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.848+0000 7f33b77fe640 1 -- 172.21.15.43:0/433463978 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33bc025030 con 0x557c03529e10 2024-04-21T08:47:49.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.848+0000 7f33b77fe640 1 -- 172.21.15.43:0/433463978 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc004bf0 con 0x557c03529e10 2024-04-21T08:47:49.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.848+0000 7f33d676e740 1 -- 172.21.15.43:0/433463978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c03529e10 msgr2=0x557c03529190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.848+0000 7f33d676e740 1 --2- 172.21.15.43:0/433463978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c03529e10 0x557c03529190 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f33bc004700 tx=0x7f33bc005970 comp rx=0 tx=0).stop 2024-04-21T08:47:49.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.848+0000 7f33d676e740 1 -- 172.21.15.43:0/433463978 shutdown_connections 2024-04-21T08:47:49.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.848+0000 7f33d676e740 1 --2- 172.21.15.43:0/433463978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c03529e10 0x557c03529190 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.848+0000 7f33d676e740 1 --2- 172.21.15.43:0/433463978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c0351cc00 0x557c03521ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.848+0000 7f33d676e740 1 --2- 172.21.15.43:0/433463978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c0351c250 0x557c03535c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.848+0000 7f33d676e740 1 -- 172.21.15.43:0/433463978 >> 172.21.15.43:0/433463978 conn(0x557c03516ba0 msgr2=0x557c035217e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.848+0000 7f33d676e740 1 -- 172.21.15.43:0/433463978 shutdown_connections 2024-04-21T08:47:49.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.848+0000 7f33d676e740 1 -- 172.21.15.43:0/433463978 wait complete. 2024-04-21T08:47:49.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.849+0000 7f33d676e740 1 -- start start 2024-04-21T08:47:49.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.849+0000 7f33d676e740 1 Processor -- start 2024-04-21T08:47:49.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.849+0000 7f33d676e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c0351c250 0x557c03532660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.849+0000 7f33d676e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c0351cc00 0x557c035234a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.849+0000 7f33d676e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c03529e10 0x557c03523a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.849+0000 7f33d676e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc1ae720 con 0x557c03529e10 2024-04-21T08:47:49.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.849+0000 7f33d676e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc1ae050 con 0x557c0351c250 2024-04-21T08:47:49.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.849+0000 7f33d676e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc1ab2b0 con 0x557c0351cc00 2024-04-21T08:47:49.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.849+0000 7f33d676e740 10 client.0 _ll_register_callbacks cb 0x7f33bc005710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:49.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.849+0000 7f33d1035640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c03529e10 0x557c03523a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.849+0000 7f33d1035640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c03529e10 0x557c03523a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57894/0 (socket says 172.21.15.43:57894) 2024-04-21T08:47:49.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.849+0000 7f33d1035640 1 -- 172.21.15.43:0/4152721437 learned_addr learned my addr 172.21.15.43:0/4152721437 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.849+0000 7f33d1035640 1 --2- 172.21.15.43:0/4152721437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c0351c250 0x557c03532660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.849+0000 7f33d1836640 1 --2- 172.21.15.43:0/4152721437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c0351cc00 0x557c035234a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.850+0000 7f33d1035640 1 -- 172.21.15.43:0/4152721437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c0351cc00 msgr2=0x557c035234a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.850+0000 7f33d1035640 1 --2- 172.21.15.43:0/4152721437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c0351cc00 0x557c035234a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.850+0000 7f33d1035640 1 -- 172.21.15.43:0/4152721437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c0351c250 msgr2=0x557c03532660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.850+0000 7f33d1035640 1 --2- 172.21.15.43:0/4152721437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c0351c250 0x557c03532660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.850+0000 7f33d1035640 1 -- 172.21.15.43:0/4152721437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c03524270 con 0x557c03529e10 2024-04-21T08:47:49.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.850+0000 7f33d1035640 1 --2- 172.21.15.43:0/4152721437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c03529e10 0x557c03523a40 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f33bc006010 tx=0x7f33bc023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d867d641b0c758e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.850+0000 7f33b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:49.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.850+0000 7f33b57fa640 1 -- 172.21.15.43:0/4152721437 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc019070 con 0x557c03529e10 2024-04-21T08:47:49.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.850+0000 7f33b57fa640 1 -- 172.21.15.43:0/4152721437 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f33bc02c110 con 0x557c03529e10 2024-04-21T08:47:49.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.850+0000 7f33d676e740 1 -- 172.21.15.43:0/4152721437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33bc025030 con 0x557c03529e10 2024-04-21T08:47:49.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.850+0000 7f33b57fa640 1 -- 172.21.15.43:0/4152721437 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f33bc048050 con 0x557c03529e10 2024-04-21T08:47:49.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.850+0000 7f33d676e740 10 client.19710 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:49.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.850+0000 7f33d676e740 1 -- 172.21.15.43:0/4152721437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557c035a0260 con 0x557c03529e10 2024-04-21T08:47:49.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.850+0000 7f33b57fa640 1 -- 172.21.15.43:0/4152721437 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33bc025030 con 0x557c03529e10 2024-04-21T08:47:49.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.850+0000 7f33b57fa640 1 -- 172.21.15.43:0/4152721437 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f33bc011b70 con 0x557c03529e10 2024-04-21T08:47:49.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.850+0000 7f33d676e740 10 client.19710 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:49.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.850+0000 7f33d676e740 10 client.19710 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:47:49.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.850+0000 7f33d676e740 20 client.19710 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:49.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.850+0000 7f33d676e740 1 --2- 172.21.15.43:0/4152721437 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x557c035a08e0 0x557c035a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.850+0000 7f33d676e740 4 client.19710 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:49.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.850+0000 7f33d676e740 1 -- 172.21.15.43:0/4152721437 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557c0348ec10 con 0x557c035a08e0 2024-04-21T08:47:49.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.851+0000 7f33d1836640 1 --2- 172.21.15.43:0/4152721437 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x557c035a08e0 0x557c035a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.851+0000 7f33b57fa640 10 client.19710 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:47:49.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.851+0000 7f33d1836640 1 --2- 172.21.15.43:0/4152721437 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x557c035a08e0 0x557c035a2d00 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1092d737ee9809a server_cookie=ce89f2f8cb390b53 in_seq=0 out_seq=0 2024-04-21T08:47:49.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.851+0000 7f33b57fa640 1 -- 172.21.15.43:0/4152721437 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557c0348ec10 con 0x557c035a08e0 2024-04-21T08:47:49.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.851+0000 7f33b57fa640 10 client.19710 handle_command_reply: tid=0 2024-04-21T08:47:49.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.851+0000 7f33d676e740 1 client.19710 shutdown 2024-04-21T08:47:49.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.851+0000 7f33d676e740 1 -- 172.21.15.43:0/4152721437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c03529e10 msgr2=0x557c03523a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.851+0000 7f33d676e740 1 --2- 172.21.15.43:0/4152721437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c03529e10 0x557c03523a40 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f33bc006010 tx=0x7f33bc023040 comp rx=0 tx=0).stop 2024-04-21T08:47:49.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.852+0000 7f33d676e740 1 -- 172.21.15.43:0/4152721437 shutdown_connections 2024-04-21T08:47:49.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.852+0000 7f33d676e740 1 --2- 172.21.15.43:0/4152721437 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x557c035a08e0 0x557c035a2d00 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.852+0000 7f33d676e740 1 --2- 172.21.15.43:0/4152721437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c03529e10 0x557c03523a40 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.852+0000 7f33d676e740 1 --2- 172.21.15.43:0/4152721437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557c0351cc00 0x557c035234a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.852+0000 7f33d676e740 1 --2- 172.21.15.43:0/4152721437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c0351c250 0x557c03532660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.852+0000 7f33b57fa640 0 client.19710 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:47:49.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.852+0000 7f33d676e740 1 -- 172.21.15.43:0/4152721437 >> 172.21.15.43:0/4152721437 conn(0x557c03516ba0 msgr2=0x557c03524f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.852+0000 7f33d676e740 1 -- 172.21.15.43:0/4152721437 shutdown_connections 2024-04-21T08:47:49.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.852+0000 7f33d676e740 1 -- 172.21.15.43:0/4152721437 wait complete. 2024-04-21T08:47:49.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.852+0000 7f33d676e740 20 client.19710 trim_cache size 0 max 16384 2024-04-21T08:47:49.861 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:47:49.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.852+0000 7f33d3a9a640 1 -- 172.21.15.43:0/316176561 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f33b80a3d20 msgr2=0x7f33b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.852+0000 7f33d3a9a640 1 --2- 172.21.15.43:0/316176561 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f33b80a3d20 0x7f33b80a61e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f33c40cd9e0 tx=0x7f33c406ec70 comp rx=0 tx=0).stop 2024-04-21T08:47:49.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.852+0000 7f33d3a9a640 1 -- 172.21.15.43:0/316176561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33cc10c510 msgr2=0x7f33cc1985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.852+0000 7f33d3a9a640 1 --2- 172.21.15.43:0/316176561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33cc10c510 0x7f33cc1985b0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f33c400d750 tx=0x7f33c400dc10 comp rx=0 tx=0).stop 2024-04-21T08:47:49.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.852+0000 7f33d3a9a640 1 -- 172.21.15.43:0/316176561 shutdown_connections 2024-04-21T08:47:49.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.852+0000 7f33d3a9a640 1 --2- 172.21.15.43:0/316176561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33cc10c510 0x7f33cc1985b0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.852+0000 7f33d3a9a640 1 --2- 172.21.15.43:0/316176561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33cc103af0 0x7f33cc19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.852+0000 7f33d3a9a640 1 --2- 172.21.15.43:0/316176561 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f33b80a3d20 0x7f33b80a61e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.852+0000 7f33d3a9a640 1 --2- 172.21.15.43:0/316176561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33cc1031d0 0x7f33cc19e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.852+0000 7f33d3a9a640 1 -- 172.21.15.43:0/316176561 >> 172.21.15.43:0/316176561 conn(0x7f33cc077f50 msgr2=0x7f33cc0fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.852+0000 7f33d3a9a640 1 -- 172.21.15.43:0/316176561 shutdown_connections 2024-04-21T08:47:49.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.853+0000 7f33d3a9a640 1 -- 172.21.15.43:0/316176561 wait complete. 2024-04-21T08:47:49.864 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:47:49.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.968+0000 7f7b4570d640 1 -- 172.21.15.43:0/1506206077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b401010a0 msgr2=0x7f7b40101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.968+0000 7f7b4570d640 1 --2- 172.21.15.43:0/1506206077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b401010a0 0x7f7b40101480 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009900 tx=0x7f7b3002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:49.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.968+0000 7f7b4570d640 1 -- 172.21.15.43:0/1506206077 shutdown_connections 2024-04-21T08:47:49.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.968+0000 7f7b4570d640 1 --2- 172.21.15.43:0/1506206077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b40106170 0x7f7b4010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.968+0000 7f7b4570d640 1 --2- 172.21.15.43:0/1506206077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b40101a50 0x7f7b40105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.968+0000 7f7b4570d640 1 --2- 172.21.15.43:0/1506206077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b401010a0 0x7f7b40101480 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.968+0000 7f7b4570d640 1 -- 172.21.15.43:0/1506206077 >> 172.21.15.43:0/1506206077 conn(0x7f7b40077eb0 msgr2=0x7f7b400782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.969+0000 7f7b4570d640 1 -- 172.21.15.43:0/1506206077 shutdown_connections 2024-04-21T08:47:49.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.969+0000 7f7b4570d640 1 -- 172.21.15.43:0/1506206077 wait complete. 2024-04-21T08:47:49.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.969+0000 7f7b4570d640 1 Processor -- start 2024-04-21T08:47:49.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.969+0000 7f7b4570d640 1 -- start start 2024-04-21T08:47:49.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.969+0000 7f7b4570d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b401010a0 0x7f7b40197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.970+0000 7f7b4570d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b40106170 0x7f7b40197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.970+0000 7f7b4570d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b4019bec0 0x7f7b4019c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.970+0000 7f7b4570d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7b40111120 con 0x7f7b40106170 2024-04-21T08:47:49.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.970+0000 7f7b4570d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7b40110fa0 con 0x7f7b401010a0 2024-04-21T08:47:49.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.970+0000 7f7b4570d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7b401112a0 con 0x7f7b4019bec0 2024-04-21T08:47:49.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.970+0000 7f7b3e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b40106170 0x7f7b40197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.970+0000 7f7b3e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b40106170 0x7f7b40197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57920/0 (socket says 172.21.15.43:57920) 2024-04-21T08:47:49.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.970+0000 7f7b3e7fc640 1 -- 172.21.15.43:0/1700900440 learned_addr learned my addr 172.21.15.43:0/1700900440 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.970+0000 7f7b3effd640 1 --2- 172.21.15.43:0/1700900440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b401010a0 0x7f7b40197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.970+0000 7f7b3f7fe640 1 --2- 172.21.15.43:0/1700900440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b4019bec0 0x7f7b4019c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.970+0000 7f7b3e7fc640 1 -- 172.21.15.43:0/1700900440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b4019bec0 msgr2=0x7f7b4019c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.970+0000 7f7b3e7fc640 1 --2- 172.21.15.43:0/1700900440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b4019bec0 0x7f7b4019c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.970+0000 7f7b3e7fc640 1 -- 172.21.15.43:0/1700900440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b401010a0 msgr2=0x7f7b40197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.970+0000 7f7b3e7fc640 1 --2- 172.21.15.43:0/1700900440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b401010a0 0x7f7b40197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.970+0000 7f7b3e7fc640 1 -- 172.21.15.43:0/1700900440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4019c9b0 con 0x7f7b40106170 2024-04-21T08:47:49.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.970+0000 7f7b3e7fc640 1 --2- 172.21.15.43:0/1700900440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b40106170 0x7f7b40197b40 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f7b3400d740 tx=0x7f7b3400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.970+0000 7f7b27fff640 1 -- 172.21.15.43:0/1700900440 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34017070 con 0x7f7b40106170 2024-04-21T08:47:49.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.970+0000 7f7b27fff640 1 -- 172.21.15.43:0/1700900440 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b3400de50 con 0x7f7b40106170 2024-04-21T08:47:49.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.970+0000 7f7b4570d640 1 -- 172.21.15.43:0/1700900440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b4019cca0 con 0x7f7b40106170 2024-04-21T08:47:49.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.970+0000 7f7b27fff640 1 -- 172.21.15.43:0/1700900440 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34004ce0 con 0x7f7b40106170 2024-04-21T08:47:49.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.970+0000 7f7b4570d640 1 -- 172.21.15.43:0/1700900440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b4006a820 con 0x7f7b40106170 2024-04-21T08:47:49.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.973+0000 7f7b27fff640 1 -- 172.21.15.43:0/1700900440 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7b34004e80 con 0x7f7b40106170 2024-04-21T08:47:49.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.973+0000 7f7b483ce740 1 Processor -- start 2024-04-21T08:47:49.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.973+0000 7f7b483ce740 1 -- start start 2024-04-21T08:47:49.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.973+0000 7f7b483ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b60399d4d0 0x55b60399d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.973+0000 7f7b483ce740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b60399de80 0x55b6039a6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.973+0000 7f7b483ce740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6039a6a00 0x55b6039a8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.973+0000 7f7b483ce740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b6039021f0 con 0x55b6039a6a00 2024-04-21T08:47:49.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.973+0000 7f7b483ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b60389f4b0 con 0x55b60399d4d0 2024-04-21T08:47:49.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.973+0000 7f7b483ce740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b6038fe1b0 con 0x55b60399de80 2024-04-21T08:47:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.973+0000 7f7b27fff640 1 --2- 172.21.15.43:0/1700900440 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7b280a3d20 0x7f7b280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.973+0000 7f7b3e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6039a6a00 0x55b6039a8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.974+0000 7f7b3e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6039a6a00 0x55b6039a8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57926/0 (socket says 172.21.15.43:57926) 2024-04-21T08:47:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.974+0000 7f7b3e7fc640 1 -- 172.21.15.43:0/1856403017 learned_addr learned my addr 172.21.15.43:0/1856403017 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.974+0000 7f7b3f7fe640 1 --2- 172.21.15.43:0/1856403017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b60399d4d0 0x55b60399d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.974+0000 7f7b3f7fe640 1 --2- 172.21.15.43:0/1700900440 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7b280a3d20 0x7f7b280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.974+0000 7f7b3effd640 1 --2- 172.21.15.43:0/1856403017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b60399de80 0x55b6039a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.974+0000 7f7b27fff640 1 -- 172.21.15.43:0/1700900440 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f7b340d0520 con 0x7f7b40106170 2024-04-21T08:47:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.974+0000 7f7b3e7fc640 1 -- 172.21.15.43:0/1856403017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b60399de80 msgr2=0x55b6039a6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.974+0000 7f7b3e7fc640 1 --2- 172.21.15.43:0/1856403017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b60399de80 0x55b6039a6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.974+0000 7f7b3e7fc640 1 -- 172.21.15.43:0/1856403017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b60399d4d0 msgr2=0x55b60399d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.974+0000 7f7b3e7fc640 1 --2- 172.21.15.43:0/1856403017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b60399d4d0 0x55b60399d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.974+0000 7f7b3e7fc640 1 -- 172.21.15.43:0/1856403017 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6038911b0 con 0x55b6039a6a00 2024-04-21T08:47:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.974+0000 7f7b3f7fe640 1 --2- 172.21.15.43:0/1700900440 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7b280a3d20 0x7f7b280a61e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7b38009fd0 tx=0x7f7b38009600 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.974+0000 7f7b3e7fc640 1 --2- 172.21.15.43:0/1856403017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6039a6a00 0x55b6039a8e50 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009610 tx=0x7f7b34098520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9103363d8fb18691 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.974+0000 7f7b257fa640 1 -- 172.21.15.43:0/1856403017 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34020070 con 0x55b6039a6a00 2024-04-21T08:47:49.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.974+0000 7f7b257fa640 1 -- 172.21.15.43:0/1856403017 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b3409c150 con 0x55b6039a6a00 2024-04-21T08:47:49.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.974+0000 7f7b257fa640 1 -- 172.21.15.43:0/1856403017 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3409c850 con 0x55b6039a6a00 2024-04-21T08:47:49.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.975+0000 7f7b483ce740 1 -- 172.21.15.43:0/1856403017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6039a6a00 msgr2=0x55b6039a8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.975+0000 7f7b483ce740 1 --2- 172.21.15.43:0/1856403017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6039a6a00 0x55b6039a8e50 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009610 tx=0x7f7b34098520 comp rx=0 tx=0).stop 2024-04-21T08:47:49.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.975+0000 7f7b483ce740 1 -- 172.21.15.43:0/1856403017 shutdown_connections 2024-04-21T08:47:49.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.975+0000 7f7b483ce740 1 --2- 172.21.15.43:0/1856403017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6039a6a00 0x55b6039a8e50 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.975+0000 7f7b483ce740 1 --2- 172.21.15.43:0/1856403017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b60399de80 0x55b6039a6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.975+0000 7f7b483ce740 1 --2- 172.21.15.43:0/1856403017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b60399d4d0 0x55b60399d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.975+0000 7f7b483ce740 1 -- 172.21.15.43:0/1856403017 >> 172.21.15.43:0/1856403017 conn(0x55b603997ba0 msgr2=0x55b603997f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.975+0000 7f7b483ce740 1 -- 172.21.15.43:0/1856403017 shutdown_connections 2024-04-21T08:47:49.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.975+0000 7f7b483ce740 1 -- 172.21.15.43:0/1856403017 wait complete. 2024-04-21T08:47:49.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.975+0000 7f7b483ce740 1 -- start start 2024-04-21T08:47:49.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.975+0000 7f7b483ce740 1 Processor -- start 2024-04-21T08:47:49.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.976+0000 7f7b483ce740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b60399d4d0 0x55b6039a6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.976+0000 7f7b483ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b60399de80 0x55b6039a3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.976+0000 7f7b483ce740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6039a6a00 0x55b6039a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.976+0000 7f7b483ce740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b6039a98b0 con 0x55b6039a6a00 2024-04-21T08:47:49.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.976+0000 7f7b483ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b6038bb500 con 0x55b60399de80 2024-04-21T08:47:49.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.976+0000 7f7b483ce740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b6039a9a30 con 0x55b60399d4d0 2024-04-21T08:47:49.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.976+0000 7f7b483ce740 10 client.0 _ll_register_callbacks cb 0x55b603889b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:49.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.976+0000 7f7b3f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6039a6a00 0x55b6039a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.976+0000 7f7b3f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6039a6a00 0x55b6039a3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57928/0 (socket says 172.21.15.43:57928) 2024-04-21T08:47:49.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.976+0000 7f7b3f7fe640 1 -- 172.21.15.43:0/13078477 learned_addr learned my addr 172.21.15.43:0/13078477 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.976+0000 7f7b3effd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b60399d4d0 0x55b6039a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.976+0000 7f7b3e7fc640 1 --2- 172.21.15.43:0/13078477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b60399de80 0x55b6039a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.976+0000 7f7b3f7fe640 1 -- 172.21.15.43:0/13078477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b60399d4d0 msgr2=0x55b6039a6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.976+0000 7f7b3f7fe640 1 --2- 172.21.15.43:0/13078477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b60399d4d0 0x55b6039a6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.976+0000 7f7b3f7fe640 1 -- 172.21.15.43:0/13078477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b60399de80 msgr2=0x55b6039a3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.976+0000 7f7b3f7fe640 1 --2- 172.21.15.43:0/13078477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b60399de80 0x55b6039a3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.976+0000 7f7b3f7fe640 1 -- 172.21.15.43:0/13078477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6036143f0 con 0x55b6039a6a00 2024-04-21T08:47:49.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.976+0000 7f7b3f7fe640 1 --2- 172.21.15.43:0/13078477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6039a6a00 0x55b6039a3d20 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800ab90 tx=0x7f7b38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c8759f78256228 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.976+0000 7f7b22ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:49.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.976+0000 7f7b22ff5640 1 -- 172.21.15.43:0/13078477 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3801ebb0 con 0x55b6039a6a00 2024-04-21T08:47:49.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.976+0000 7f7b22ff5640 1 -- 172.21.15.43:0/13078477 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f7b3801ed50 con 0x55b6039a6a00 2024-04-21T08:47:49.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.976+0000 7f7b483ce740 1 -- 172.21.15.43:0/13078477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b6038a1930 con 0x55b6039a6a00 2024-04-21T08:47:49.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.977+0000 7f7b22ff5640 1 -- 172.21.15.43:0/13078477 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7b38033050 con 0x55b6039a6a00 2024-04-21T08:47:49.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.977+0000 7f7b483ce740 10 client.19722 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:49.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.977+0000 7f7b483ce740 1 -- 172.21.15.43:0/13078477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b6036de7d0 con 0x55b6039a6a00 2024-04-21T08:47:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.977+0000 7f7b22ff5640 1 -- 172.21.15.43:0/13078477 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b6038a1930 con 0x55b6039a6a00 2024-04-21T08:47:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.977+0000 7f7b22ff5640 1 -- 172.21.15.43:0/13078477 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7b3800f040 con 0x55b6039a6a00 2024-04-21T08:47:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.977+0000 7f7b483ce740 10 client.19722 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.977+0000 7f7b483ce740 10 client.19722 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:47:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.977+0000 7f7b483ce740 20 client.19722 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.977+0000 7f7b483ce740 1 --2- 172.21.15.43:0/13078477 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55b6039b6c20 0x55b6039b9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.977+0000 7f7b483ce740 4 client.19722 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.977+0000 7f7b483ce740 1 -- 172.21.15.43:0/13078477 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b60390fc10 con 0x55b6039b6c20 2024-04-21T08:47:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.977+0000 7f7b3effd640 1 --2- 172.21.15.43:0/13078477 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55b6039b6c20 0x55b6039b9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.978+0000 7f7b3effd640 1 --2- 172.21.15.43:0/13078477 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55b6039b6c20 0x55b6039b9040 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=56e9e376cf030526 server_cookie=2a069b05873e138 in_seq=0 out_seq=0 2024-04-21T08:47:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.978+0000 7f7b22ff5640 10 client.19722 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:47:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.979+0000 7f7b22ff5640 1 -- 172.21.15.43:0/13078477 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b60390fc10 con 0x55b6039b6c20 2024-04-21T08:47:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.979+0000 7f7b22ff5640 10 client.19722 handle_command_reply: tid=0 2024-04-21T08:47:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.979+0000 7f7b483ce740 1 client.19722 shutdown 2024-04-21T08:47:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.979+0000 7f7b483ce740 1 -- 172.21.15.43:0/13078477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6039a6a00 msgr2=0x55b6039a3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.979+0000 7f7b483ce740 1 --2- 172.21.15.43:0/13078477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6039a6a00 0x55b6039a3d20 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800ab90 tx=0x7f7b38004a40 comp rx=0 tx=0).stop 2024-04-21T08:47:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.979+0000 7f7b483ce740 1 -- 172.21.15.43:0/13078477 shutdown_connections 2024-04-21T08:47:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.979+0000 7f7b483ce740 1 --2- 172.21.15.43:0/13078477 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55b6039b6c20 0x55b6039b9040 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.979+0000 7f7b483ce740 1 --2- 172.21.15.43:0/13078477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6039a6a00 0x55b6039a3d20 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.979+0000 7f7b483ce740 1 --2- 172.21.15.43:0/13078477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b60399de80 0x55b6039a3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.979+0000 7f7b22ff5640 0 client.19722 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:47:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.979+0000 7f7b483ce740 1 --2- 172.21.15.43:0/13078477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b60399d4d0 0x55b6039a6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.979+0000 7f7b483ce740 1 -- 172.21.15.43:0/13078477 >> 172.21.15.43:0/13078477 conn(0x55b603997ba0 msgr2=0x55b6039998f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.979+0000 7f7b483ce740 1 -- 172.21.15.43:0/13078477 shutdown_connections 2024-04-21T08:47:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.979+0000 7f7b483ce740 1 -- 172.21.15.43:0/13078477 wait complete. 2024-04-21T08:47:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.979+0000 7f7b483ce740 20 client.19722 trim_cache size 0 max 16384 2024-04-21T08:47:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.992+0000 7f7b483ce740 1 Processor -- start 2024-04-21T08:47:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.992+0000 7f7b483ce740 1 -- start start 2024-04-21T08:47:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.992+0000 7f7b483ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b60399d4d0 0x55b6039b2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.992+0000 7f7b483ce740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b60399de80 0x55b6039af060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.992+0000 7f7b483ce740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6039a6a00 0x55b603999700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.992+0000 7f7b483ce740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7b401af490 con 0x55b60399de80 2024-04-21T08:47:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.992+0000 7f7b483ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7b401b0fd0 con 0x55b60399d4d0 2024-04-21T08:47:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.992+0000 7f7b483ce740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7b401afb60 con 0x55b6039a6a00 2024-04-21T08:47:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.993+0000 7f7b3e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b60399de80 0x55b6039af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.993+0000 7f7b3e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b60399de80 0x55b6039af060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57944/0 (socket says 172.21.15.43:57944) 2024-04-21T08:47:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.993+0000 7f7b3e7fc640 1 -- 172.21.15.43:0/9741299 learned_addr learned my addr 172.21.15.43:0/9741299 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.993+0000 7f7b3effd640 1 --2- 172.21.15.43:0/9741299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b60399d4d0 0x55b6039b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.993+0000 7f7b3e7fc640 1 -- 172.21.15.43:0/9741299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6039a6a00 msgr2=0x55b603999700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.993+0000 7f7b3f7fe640 1 --2- 172.21.15.43:0/9741299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6039a6a00 0x55b603999700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.993+0000 7f7b3e7fc640 1 --2- 172.21.15.43:0/9741299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6039a6a00 0x55b603999700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.993+0000 7f7b3e7fc640 1 -- 172.21.15.43:0/9741299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b60399d4d0 msgr2=0x55b6039b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.993+0000 7f7b3e7fc640 1 --2- 172.21.15.43:0/9741299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b60399d4d0 0x55b6039b2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.993+0000 7f7b3e7fc640 1 -- 172.21.15.43:0/9741299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6039a9950 con 0x55b60399de80 2024-04-21T08:47:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.993+0000 7f7b3effd640 1 --2- 172.21.15.43:0/9741299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b60399d4d0 0x55b6039b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.993+0000 7f7b3f7fe640 1 --2- 172.21.15.43:0/9741299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6039a6a00 0x55b603999700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.993+0000 7f7b3e7fc640 1 --2- 172.21.15.43:0/9741299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b60399de80 0x55b6039af060 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f7b3409c310 tx=0x7f7b340cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec846e5422c0a506 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.993+0000 7f7b24ff9640 1 -- 172.21.15.43:0/9741299 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34020040 con 0x55b60399de80 2024-04-21T08:47:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.993+0000 7f7b24ff9640 1 -- 172.21.15.43:0/9741299 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b340a0120 con 0x55b60399de80 2024-04-21T08:47:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.993+0000 7f7b24ff9640 1 -- 172.21.15.43:0/9741299 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3409cfc0 con 0x55b60399de80 2024-04-21T08:47:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.994+0000 7f7b483ce740 1 -- 172.21.15.43:0/9741299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b60399de80 msgr2=0x55b6039af060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.994+0000 7f7b483ce740 1 --2- 172.21.15.43:0/9741299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b60399de80 0x55b6039af060 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f7b3409c310 tx=0x7f7b340cf040 comp rx=0 tx=0).stop 2024-04-21T08:47:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.994+0000 7f7b483ce740 1 -- 172.21.15.43:0/9741299 shutdown_connections 2024-04-21T08:47:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.994+0000 7f7b483ce740 1 --2- 172.21.15.43:0/9741299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6039a6a00 0x55b603999700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.994+0000 7f7b483ce740 1 --2- 172.21.15.43:0/9741299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b60399de80 0x55b6039af060 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.994+0000 7f7b483ce740 1 --2- 172.21.15.43:0/9741299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b60399d4d0 0x55b6039b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.994+0000 7f7b483ce740 1 -- 172.21.15.43:0/9741299 >> 172.21.15.43:0/9741299 conn(0x55b603997ba0 msgr2=0x55b603998ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:50.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.994+0000 7f7b483ce740 1 -- 172.21.15.43:0/9741299 shutdown_connections 2024-04-21T08:47:50.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.994+0000 7f7b483ce740 1 -- 172.21.15.43:0/9741299 wait complete. 2024-04-21T08:47:50.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.994+0000 7f7b483ce740 1 -- start start 2024-04-21T08:47:50.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.994+0000 7f7b483ce740 1 Processor -- start 2024-04-21T08:47:50.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.994+0000 7f7b483ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b60399d4d0 0x55b6039a1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.994+0000 7f7b483ce740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b60399de80 0x55b6039a5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.994+0000 7f7b483ce740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6039a6a00 0x55b6039a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.994+0000 7f7b483ce740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7b401b0900 con 0x55b60399de80 2024-04-21T08:47:50.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.994+0000 7f7b483ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7b401b0230 con 0x55b60399d4d0 2024-04-21T08:47:50.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.994+0000 7f7b483ce740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7b401ad490 con 0x55b6039a6a00 2024-04-21T08:47:50.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.994+0000 7f7b483ce740 10 client.0 _ll_register_callbacks cb 0x7f7b3800a840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:50.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.994+0000 7f7b3e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b60399de80 0x55b6039a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.995+0000 7f7b3e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b60399de80 0x55b6039a5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57954/0 (socket says 172.21.15.43:57954) 2024-04-21T08:47:50.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.995+0000 7f7b3e7fc640 1 -- 172.21.15.43:0/693854873 learned_addr learned my addr 172.21.15.43:0/693854873 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:50.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.995+0000 7f7b3effd640 1 --2- 172.21.15.43:0/693854873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b60399d4d0 0x55b6039a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.995+0000 7f7b3f7fe640 1 --2- 172.21.15.43:0/693854873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6039a6a00 0x55b6039a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.995+0000 7f7b3e7fc640 1 -- 172.21.15.43:0/693854873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6039a6a00 msgr2=0x55b6039a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.995+0000 7f7b3e7fc640 1 --2- 172.21.15.43:0/693854873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6039a6a00 0x55b6039a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.995+0000 7f7b3e7fc640 1 -- 172.21.15.43:0/693854873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b60399d4d0 msgr2=0x55b6039a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.995+0000 7f7b3e7fc640 1 --2- 172.21.15.43:0/693854873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b60399d4d0 0x55b6039a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.995+0000 7f7b3e7fc640 1 -- 172.21.15.43:0/693854873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b603a20200 con 0x55b60399de80 2024-04-21T08:47:50.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.995+0000 7f7b3effd640 1 --2- 172.21.15.43:0/693854873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b60399d4d0 0x55b6039a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:50.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.995+0000 7f7b3e7fc640 1 --2- 172.21.15.43:0/693854873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b60399de80 0x55b6039a5690 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f7b3409be50 tx=0x7f7b34005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1510f811c61378b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:50.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.995+0000 7f7b22ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:50.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.995+0000 7f7b22ff5640 1 -- 172.21.15.43:0/693854873 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34020070 con 0x55b60399de80 2024-04-21T08:47:50.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.995+0000 7f7b22ff5640 1 -- 172.21.15.43:0/693854873 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f7b340cdad0 con 0x55b60399de80 2024-04-21T08:47:50.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.995+0000 7f7b483ce740 1 -- 172.21.15.43:0/693854873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b340a0120 con 0x55b60399de80 2024-04-21T08:47:50.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.995+0000 7f7b22ff5640 1 -- 172.21.15.43:0/693854873 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7b3407d080 con 0x55b60399de80 2024-04-21T08:47:50.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.995+0000 7f7b483ce740 10 client.19728 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:50.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.995+0000 7f7b483ce740 1 -- 172.21.15.43:0/693854873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b603a212d0 con 0x55b60399de80 2024-04-21T08:47:50.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.995+0000 7f7b22ff5640 1 -- 172.21.15.43:0/693854873 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b340a0120 con 0x55b60399de80 2024-04-21T08:47:50.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.995+0000 7f7b22ff5640 1 -- 172.21.15.43:0/693854873 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7b340d0160 con 0x55b60399de80 2024-04-21T08:47:50.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.996+0000 7f7b483ce740 10 client.19728 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:50.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.996+0000 7f7b483ce740 10 client.19728 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:47:50.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.996+0000 7f7b483ce740 20 client.19728 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:50.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.996+0000 7f7b483ce740 1 --2- 172.21.15.43:0/693854873 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55b603a214f0 0x55b603a238e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.996+0000 7f7b483ce740 4 client.19728 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:50.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.996+0000 7f7b483ce740 1 -- 172.21.15.43:0/693854873 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b603909a80 con 0x55b603a214f0 2024-04-21T08:47:50.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.996+0000 7f7b3effd640 1 --2- 172.21.15.43:0/693854873 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55b603a214f0 0x55b603a238e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.996+0000 7f7b22ff5640 10 client.19728 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:47:50.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.996+0000 7f7b3effd640 1 --2- 172.21.15.43:0/693854873 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55b603a214f0 0x55b603a238e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4e423fea85dcd028 server_cookie=c023c0bdd5fa04da in_seq=0 out_seq=0 2024-04-21T08:47:50.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b22ff5640 1 -- 172.21.15.43:0/693854873 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b603909a80 con 0x55b603a214f0 2024-04-21T08:47:50.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b22ff5640 10 client.19728 handle_command_reply: tid=0 2024-04-21T08:47:50.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b483ce740 1 client.19728 shutdown 2024-04-21T08:47:50.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b483ce740 1 -- 172.21.15.43:0/693854873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b60399de80 msgr2=0x55b6039a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b483ce740 1 --2- 172.21.15.43:0/693854873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b60399de80 0x55b6039a5690 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f7b3409be50 tx=0x7f7b34005b20 comp rx=0 tx=0).stop 2024-04-21T08:47:50.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b483ce740 1 -- 172.21.15.43:0/693854873 shutdown_connections 2024-04-21T08:47:50.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b483ce740 1 --2- 172.21.15.43:0/693854873 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55b603a214f0 0x55b603a238e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b483ce740 1 --2- 172.21.15.43:0/693854873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6039a6a00 0x55b6039a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b483ce740 1 --2- 172.21.15.43:0/693854873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b60399de80 0x55b6039a5690 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b22ff5640 0 client.19728 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:47:50.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b483ce740 1 --2- 172.21.15.43:0/693854873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b60399d4d0 0x55b6039a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b483ce740 1 -- 172.21.15.43:0/693854873 >> 172.21.15.43:0/693854873 conn(0x55b603997ba0 msgr2=0x55b60399ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:50.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b483ce740 1 -- 172.21.15.43:0/693854873 shutdown_connections 2024-04-21T08:47:50.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b483ce740 1 -- 172.21.15.43:0/693854873 wait complete. 2024-04-21T08:47:50.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b483ce740 20 client.19728 trim_cache size 0 max 16384 2024-04-21T08:47:50.005 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:47:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b4570d640 1 -- 172.21.15.43:0/1700900440 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7b280a3d20 msgr2=0x7f7b280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b4570d640 1 --2- 172.21.15.43:0/1700900440 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7b280a3d20 0x7f7b280a61e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7b38009fd0 tx=0x7f7b38009600 comp rx=0 tx=0).stop 2024-04-21T08:47:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b4570d640 1 -- 172.21.15.43:0/1700900440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b40106170 msgr2=0x7f7b40197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b4570d640 1 --2- 172.21.15.43:0/1700900440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b40106170 0x7f7b40197b40 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f7b3400d740 tx=0x7f7b3400dc00 comp rx=0 tx=0).stop 2024-04-21T08:47:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b4570d640 1 -- 172.21.15.43:0/1700900440 shutdown_connections 2024-04-21T08:47:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b4570d640 1 --2- 172.21.15.43:0/1700900440 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7b280a3d20 0x7f7b280a61e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b4570d640 1 --2- 172.21.15.43:0/1700900440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b4019bec0 0x7f7b4019c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b4570d640 1 --2- 172.21.15.43:0/1700900440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b40106170 0x7f7b40197b40 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b4570d640 1 --2- 172.21.15.43:0/1700900440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b401010a0 0x7f7b40197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.997+0000 7f7b4570d640 1 -- 172.21.15.43:0/1700900440 >> 172.21.15.43:0/1700900440 conn(0x7f7b40077eb0 msgr2=0x7f7b40107340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:50.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.998+0000 7f7b4570d640 1 -- 172.21.15.43:0/1700900440 shutdown_connections 2024-04-21T08:47:50.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:49.998+0000 7f7b4570d640 1 -- 172.21.15.43:0/1700900440 wait complete. 2024-04-21T08:47:50.009 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:47:50.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.113+0000 7fa790cf2640 1 -- 172.21.15.43:0/803145041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa78c06b1f0 msgr2=0x7fa78c106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.113+0000 7fa790cf2640 1 --2- 172.21.15.43:0/803145041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa78c06b1f0 0x7fa78c106b90 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fa780009900 tx=0x7fa78002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:50.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.113+0000 7fa790cf2640 1 -- 172.21.15.43:0/803145041 shutdown_connections 2024-04-21T08:47:50.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.113+0000 7fa790cf2640 1 --2- 172.21.15.43:0/803145041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa78c107270 0x7fa78c10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.113+0000 7fa790cf2640 1 --2- 172.21.15.43:0/803145041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa78c06b1f0 0x7fa78c106b90 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.113+0000 7fa790cf2640 1 --2- 172.21.15.43:0/803145041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa78c06a840 0x7fa78c06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.113+0000 7fa790cf2640 1 -- 172.21.15.43:0/803145041 >> 172.21.15.43:0/803145041 conn(0x7fa78c077eb0 msgr2=0x7fa78c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:50.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.113+0000 7fa790cf2640 1 -- 172.21.15.43:0/803145041 shutdown_connections 2024-04-21T08:47:50.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.113+0000 7fa790cf2640 1 -- 172.21.15.43:0/803145041 wait complete. 2024-04-21T08:47:50.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.114+0000 7fa790cf2640 1 Processor -- start 2024-04-21T08:47:50.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.114+0000 7fa790cf2640 1 -- start start 2024-04-21T08:47:50.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.114+0000 7fa790cf2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa78c06a840 0x7fa78c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.114+0000 7fa790cf2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa78c107270 0x7fa78c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.114+0000 7fa790cf2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa78c19cfb0 0x7fa78c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.114+0000 7fa790cf2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c1121b0 con 0x7fa78c19cfb0 2024-04-21T08:47:50.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.114+0000 7fa790cf2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c112030 con 0x7fa78c107270 2024-04-21T08:47:50.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.114+0000 7fa790cf2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c112330 con 0x7fa78c06a840 2024-04-21T08:47:50.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.115+0000 7fa78ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa78c19cfb0 0x7fa78c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.115+0000 7fa78ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa78c19cfb0 0x7fa78c19d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57974/0 (socket says 172.21.15.43:57974) 2024-04-21T08:47:50.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.115+0000 7fa78ad9d640 1 -- 172.21.15.43:0/4167226895 learned_addr learned my addr 172.21.15.43:0/4167226895 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:50.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.115+0000 7fa789d9b640 1 --2- 172.21.15.43:0/4167226895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa78c107270 0x7fa78c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.115+0000 7fa78a59c640 1 --2- 172.21.15.43:0/4167226895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa78c06a840 0x7fa78c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.115+0000 7fa78ad9d640 1 -- 172.21.15.43:0/4167226895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa78c06a840 msgr2=0x7fa78c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.115+0000 7fa78ad9d640 1 --2- 172.21.15.43:0/4167226895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa78c06a840 0x7fa78c1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.115+0000 7fa78ad9d640 1 -- 172.21.15.43:0/4167226895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa78c107270 msgr2=0x7fa78c198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.115+0000 7fa78ad9d640 1 --2- 172.21.15.43:0/4167226895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa78c107270 0x7fa78c198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.115+0000 7fa78ad9d640 1 -- 172.21.15.43:0/4167226895 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa78c19daa0 con 0x7fa78c19cfb0 2024-04-21T08:47:50.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.115+0000 7fa789d9b640 1 --2- 172.21.15.43:0/4167226895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa78c107270 0x7fa78c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:50.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.115+0000 7fa78ad9d640 1 --2- 172.21.15.43:0/4167226895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa78c19cfb0 0x7fa78c19d460 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fa78400eaa0 tx=0x7fa78400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:50.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.115+0000 7fa77b7fe640 1 -- 172.21.15.43:0/4167226895 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78400cdb0 con 0x7fa78c19cfb0 2024-04-21T08:47:50.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.115+0000 7fa790cf2640 1 -- 172.21.15.43:0/4167226895 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa78c19dd90 con 0x7fa78c19cfb0 2024-04-21T08:47:50.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.115+0000 7fa77b7fe640 1 -- 172.21.15.43:0/4167226895 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa784005020 con 0x7fa78c19cfb0 2024-04-21T08:47:50.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.115+0000 7fa77b7fe640 1 -- 172.21.15.43:0/4167226895 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7840116d0 con 0x7fa78c19cfb0 2024-04-21T08:47:50.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.116+0000 7fa790cf2640 1 -- 172.21.15.43:0/4167226895 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa78c1a5550 con 0x7fa78c19cfb0 2024-04-21T08:47:50.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.118+0000 7fa77b7fe640 1 -- 172.21.15.43:0/4167226895 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa7840054a0 con 0x7fa78c19cfb0 2024-04-21T08:47:50.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.118+0000 7fa7939c1740 1 Processor -- start 2024-04-21T08:47:50.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.118+0000 7fa7939c1740 1 -- start start 2024-04-21T08:47:50.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.119+0000 7fa7939c1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638038b24d0 0x5638038b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.119+0000 7fa7939c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038b2e80 0x5638038bb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.119+0000 7fa7939c1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638038bba00 0x5638038bde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.119+0000 7fa7939c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5638038171f0 con 0x5638038b2e80 2024-04-21T08:47:50.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.119+0000 7fa7939c1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5638037b44b0 con 0x5638038bba00 2024-04-21T08:47:50.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.119+0000 7fa7939c1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5638038131b0 con 0x5638038b24d0 2024-04-21T08:47:50.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.119+0000 7fa78a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038b2e80 0x5638038bb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.119+0000 7fa789d9b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638038b24d0 0x5638038b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.119+0000 7fa789d9b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638038b24d0 0x5638038b28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.43:59270/0 (socket says 172.21.15.43:59270) 2024-04-21T08:47:50.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.119+0000 7fa78a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038b2e80 0x5638038bb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57980/0 (socket says 172.21.15.43:57980) 2024-04-21T08:47:50.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.119+0000 7fa789d9b640 1 -- 172.21.15.43:0/2332640826 learned_addr learned my addr 172.21.15.43:0/2332640826 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:50.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.119+0000 7fa789d9b640 1 --2- 172.21.15.43:0/2332640826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638038bba00 0x5638038bde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.119+0000 7fa77b7fe640 1 --2- 172.21.15.43:0/4167226895 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa7740a3d20 0x7fa7740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.119+0000 7fa78a59c640 1 -- 172.21.15.43:0/2332640826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638038b24d0 msgr2=0x5638038b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.119+0000 7fa78a59c640 1 --2- 172.21.15.43:0/2332640826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638038b24d0 0x5638038b28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.119+0000 7fa78ad9d640 1 --2- 172.21.15.43:0/4167226895 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa7740a3d20 0x7fa7740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.119+0000 7fa78a59c640 1 -- 172.21.15.43:0/2332640826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638038bba00 msgr2=0x5638038bde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.119+0000 7fa78a59c640 1 --2- 172.21.15.43:0/2332640826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638038bba00 0x5638038bde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.119+0000 7fa78a59c640 1 -- 172.21.15.43:0/2332640826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638037a61b0 con 0x5638038b2e80 2024-04-21T08:47:50.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.119+0000 7fa77b7fe640 1 -- 172.21.15.43:0/4167226895 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fa7840d0ca0 con 0x7fa78c19cfb0 2024-04-21T08:47:50.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.119+0000 7fa78a59c640 1 --2- 172.21.15.43:0/2332640826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038b2e80 0x5638038bb430 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fa77c009c20 tx=0x7fa77c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecbde7901c676499 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:50.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.120+0000 7fa78ad9d640 1 --2- 172.21.15.43:0/4167226895 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa7740a3d20 0x7fa7740a61e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa7840ce080 tx=0x7fa78406c1b0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:50.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.120+0000 7fa778ff9640 1 -- 172.21.15.43:0/2332640826 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c004050 con 0x5638038b2e80 2024-04-21T08:47:50.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.120+0000 7fa778ff9640 1 -- 172.21.15.43:0/2332640826 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa77c0054d0 con 0x5638038b2e80 2024-04-21T08:47:50.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.120+0000 7fa778ff9640 1 -- 172.21.15.43:0/2332640826 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c004d90 con 0x5638038b2e80 2024-04-21T08:47:50.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.120+0000 7fa7939c1740 1 -- 172.21.15.43:0/2332640826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038b2e80 msgr2=0x5638038bb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.120+0000 7fa7939c1740 1 --2- 172.21.15.43:0/2332640826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038b2e80 0x5638038bb430 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fa77c009c20 tx=0x7fa77c011810 comp rx=0 tx=0).stop 2024-04-21T08:47:50.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.120+0000 7fa7939c1740 1 -- 172.21.15.43:0/2332640826 shutdown_connections 2024-04-21T08:47:50.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.120+0000 7fa7939c1740 1 --2- 172.21.15.43:0/2332640826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638038bba00 0x5638038bde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.120+0000 7fa7939c1740 1 --2- 172.21.15.43:0/2332640826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038b2e80 0x5638038bb430 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.120+0000 7fa7939c1740 1 --2- 172.21.15.43:0/2332640826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638038b24d0 0x5638038b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.121+0000 7fa7939c1740 1 -- 172.21.15.43:0/2332640826 >> 172.21.15.43:0/2332640826 conn(0x5638038acba0 msgr2=0x5638038acf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:50.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.121+0000 7fa7939c1740 1 -- 172.21.15.43:0/2332640826 shutdown_connections 2024-04-21T08:47:50.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.121+0000 7fa7939c1740 1 -- 172.21.15.43:0/2332640826 wait complete. 2024-04-21T08:47:50.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.121+0000 7fa7939c1740 1 -- start start 2024-04-21T08:47:50.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.121+0000 7fa7939c1740 1 Processor -- start 2024-04-21T08:47:50.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.121+0000 7fa7939c1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638038b24d0 0x5638038bb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.121+0000 7fa7939c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038b2e80 0x5638038b8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.121+0000 7fa7939c1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638038bba00 0x5638038b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.121+0000 7fa7939c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5638038be970 con 0x5638038b2e80 2024-04-21T08:47:50.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.121+0000 7fa7939c1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5638037d0500 con 0x5638038bba00 2024-04-21T08:47:50.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.121+0000 7fa7939c1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5638038beaf0 con 0x5638038b24d0 2024-04-21T08:47:50.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.121+0000 7fa7939c1740 10 client.0 _ll_register_callbacks cb 0x56380379eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:50.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.121+0000 7fa78a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038b2e80 0x5638038b8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.122+0000 7fa789d9b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638038b24d0 0x5638038bb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.122+0000 7fa78a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038b2e80 0x5638038b8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57986/0 (socket says 172.21.15.43:57986) 2024-04-21T08:47:50.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.122+0000 7fa78a59c640 1 -- 172.21.15.43:0/1300905202 learned_addr learned my addr 172.21.15.43:0/1300905202 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:50.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.122+0000 7fa789d9b640 1 --2- 172.21.15.43:0/1300905202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638038bba00 0x5638038b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.122+0000 7fa78a59c640 1 -- 172.21.15.43:0/1300905202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638038b24d0 msgr2=0x5638038bb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.122+0000 7fa78a59c640 1 --2- 172.21.15.43:0/1300905202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638038b24d0 0x5638038bb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.122+0000 7fa78a59c640 1 -- 172.21.15.43:0/1300905202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638038bba00 msgr2=0x5638038b8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.122+0000 7fa78a59c640 1 --2- 172.21.15.43:0/1300905202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638038bba00 0x5638038b8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.122+0000 7fa78a59c640 1 -- 172.21.15.43:0/1300905202 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638035293f0 con 0x5638038b2e80 2024-04-21T08:47:50.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.122+0000 7fa78a59c640 1 --2- 172.21.15.43:0/1300905202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038b2e80 0x5638038b8660 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fa77c000c00 tx=0x7fa77c004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d402bda3afd8e5b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:50.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.122+0000 7fa771ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:50.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.122+0000 7fa771ffb640 1 -- 172.21.15.43:0/1300905202 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c0046c0 con 0x5638038b2e80 2024-04-21T08:47:50.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.122+0000 7fa771ffb640 1 -- 172.21.15.43:0/1300905202 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fa77c005140 con 0x5638038b2e80 2024-04-21T08:47:50.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.122+0000 7fa7939c1740 1 -- 172.21.15.43:0/1300905202 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa77c0054d0 con 0x5638038b2e80 2024-04-21T08:47:50.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.122+0000 7fa771ffb640 1 -- 172.21.15.43:0/1300905202 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa77c032050 con 0x5638038b2e80 2024-04-21T08:47:50.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.122+0000 7fa7939c1740 10 client.19740 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:50.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.122+0000 7fa7939c1740 1 -- 172.21.15.43:0/1300905202 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5638035f37d0 con 0x5638038b2e80 2024-04-21T08:47:50.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.123+0000 7fa771ffb640 1 -- 172.21.15.43:0/1300905202 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa77c0054d0 con 0x5638038b2e80 2024-04-21T08:47:50.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.123+0000 7fa771ffb640 1 -- 172.21.15.43:0/1300905202 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa77c018070 con 0x5638038b2e80 2024-04-21T08:47:50.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.123+0000 7fa7939c1740 10 client.19740 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:50.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.123+0000 7fa7939c1740 10 client.19740 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:47:50.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.123+0000 7fa7939c1740 20 client.19740 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:50.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.123+0000 7fa7939c1740 1 --2- 172.21.15.43:0/1300905202 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5638038cbb10 0x5638038cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.123+0000 7fa7939c1740 4 client.19740 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:50.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.123+0000 7fa7939c1740 1 -- 172.21.15.43:0/1300905202 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638037b6930 con 0x5638038cbb10 2024-04-21T08:47:50.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.123+0000 7fa789d9b640 1 --2- 172.21.15.43:0/1300905202 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5638038cbb10 0x5638038cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.124+0000 7fa771ffb640 10 client.19740 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:47:50.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.124+0000 7fa789d9b640 1 --2- 172.21.15.43:0/1300905202 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5638038cbb10 0x5638038cdf00 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=448e3f441b7f6293 server_cookie=bd50ca7910f98301 in_seq=0 out_seq=0 2024-04-21T08:47:50.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.124+0000 7fa771ffb640 1 -- 172.21.15.43:0/1300905202 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5638037b6930 con 0x5638038cbb10 2024-04-21T08:47:50.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.124+0000 7fa771ffb640 10 client.19740 handle_command_reply: tid=0 2024-04-21T08:47:50.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.124+0000 7fa7939c1740 1 client.19740 shutdown 2024-04-21T08:47:50.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.124+0000 7fa7939c1740 1 -- 172.21.15.43:0/1300905202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038b2e80 msgr2=0x5638038b8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.124+0000 7fa7939c1740 1 --2- 172.21.15.43:0/1300905202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038b2e80 0x5638038b8660 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fa77c000c00 tx=0x7fa77c004dc0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.124+0000 7fa7939c1740 1 -- 172.21.15.43:0/1300905202 shutdown_connections 2024-04-21T08:47:50.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.124+0000 7fa7939c1740 1 --2- 172.21.15.43:0/1300905202 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5638038cbb10 0x5638038cdf00 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.124+0000 7fa7939c1740 1 --2- 172.21.15.43:0/1300905202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638038bba00 0x5638038b8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.124+0000 7fa7939c1740 1 --2- 172.21.15.43:0/1300905202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038b2e80 0x5638038b8660 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.124+0000 7fa771ffb640 0 client.19740 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:47:50.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.124+0000 7fa7939c1740 1 --2- 172.21.15.43:0/1300905202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638038b24d0 0x5638038bb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.124+0000 7fa7939c1740 1 -- 172.21.15.43:0/1300905202 >> 172.21.15.43:0/1300905202 conn(0x5638038acba0 msgr2=0x5638038ae8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:50.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.124+0000 7fa7939c1740 1 -- 172.21.15.43:0/1300905202 shutdown_connections 2024-04-21T08:47:50.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.124+0000 7fa7939c1740 1 -- 172.21.15.43:0/1300905202 wait complete. 2024-04-21T08:47:50.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.125+0000 7fa7939c1740 20 client.19740 trim_cache size 0 max 16384 2024-04-21T08:47:50.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.138+0000 7fa7939c1740 1 Processor -- start 2024-04-21T08:47:50.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.138+0000 7fa7939c1740 1 -- start start 2024-04-21T08:47:50.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.138+0000 7fa7939c1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638038b24d0 0x5638038c7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.138+0000 7fa7939c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038b2e80 0x5638038ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.138+0000 7fa7939c1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638038bba00 0x5638038bad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.138+0000 7fa7939c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c1af510 con 0x5638038b2e80 2024-04-21T08:47:50.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.138+0000 7fa7939c1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c1b1050 con 0x5638038b24d0 2024-04-21T08:47:50.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.138+0000 7fa7939c1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c1afbe0 con 0x5638038bba00 2024-04-21T08:47:50.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.138+0000 7fa78a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038b2e80 0x5638038ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.138+0000 7fa78a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038b2e80 0x5638038ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58002/0 (socket says 172.21.15.43:58002) 2024-04-21T08:47:50.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.138+0000 7fa78a59c640 1 -- 172.21.15.43:0/313357885 learned_addr learned my addr 172.21.15.43:0/313357885 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:50.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.138+0000 7fa789d9b640 1 --2- 172.21.15.43:0/313357885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638038bba00 0x5638038bad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.138+0000 7fa789d9b640 1 --2- 172.21.15.43:0/313357885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638038b24d0 0x5638038c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.138+0000 7fa78a59c640 1 -- 172.21.15.43:0/313357885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638038bba00 msgr2=0x5638038bad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.138+0000 7fa78a59c640 1 --2- 172.21.15.43:0/313357885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638038bba00 0x5638038bad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.138+0000 7fa78a59c640 1 -- 172.21.15.43:0/313357885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638038b24d0 msgr2=0x5638038c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.138+0000 7fa78a59c640 1 --2- 172.21.15.43:0/313357885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638038b24d0 0x5638038c7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.138+0000 7fa78a59c640 1 -- 172.21.15.43:0/313357885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638038b2c20 con 0x5638038b2e80 2024-04-21T08:47:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.138+0000 7fa789d9b640 1 --2- 172.21.15.43:0/313357885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638038b24d0 0x5638038c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.139+0000 7fa78a59c640 1 --2- 172.21.15.43:0/313357885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038b2e80 0x5638038ae690 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fa77c005e00 tx=0x7fa77c004aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86049c9d997c330a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.139+0000 7fa773fff640 1 -- 172.21.15.43:0/313357885 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c03b070 con 0x5638038b2e80 2024-04-21T08:47:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.139+0000 7fa773fff640 1 -- 172.21.15.43:0/313357885 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa77c029070 con 0x5638038b2e80 2024-04-21T08:47:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.139+0000 7fa773fff640 1 -- 172.21.15.43:0/313357885 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c0053a0 con 0x5638038b2e80 2024-04-21T08:47:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.139+0000 7fa7939c1740 1 -- 172.21.15.43:0/313357885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038b2e80 msgr2=0x5638038ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.139+0000 7fa7939c1740 1 --2- 172.21.15.43:0/313357885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038b2e80 0x5638038ae690 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fa77c005e00 tx=0x7fa77c004aa0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.139+0000 7fa7939c1740 1 -- 172.21.15.43:0/313357885 shutdown_connections 2024-04-21T08:47:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.139+0000 7fa7939c1740 1 --2- 172.21.15.43:0/313357885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638038bba00 0x5638038bad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.139+0000 7fa7939c1740 1 --2- 172.21.15.43:0/313357885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038b2e80 0x5638038ae690 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.139+0000 7fa7939c1740 1 --2- 172.21.15.43:0/313357885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638038b24d0 0x5638038c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.139+0000 7fa7939c1740 1 -- 172.21.15.43:0/313357885 >> 172.21.15.43:0/313357885 conn(0x5638038acba0 msgr2=0x5638038adf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.139+0000 7fa7939c1740 1 -- 172.21.15.43:0/313357885 shutdown_connections 2024-04-21T08:47:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.139+0000 7fa7939c1740 1 -- 172.21.15.43:0/313357885 wait complete. 2024-04-21T08:47:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.140+0000 7fa7939c1740 1 -- start start 2024-04-21T08:47:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.140+0000 7fa7939c1740 1 Processor -- start 2024-04-21T08:47:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.140+0000 7fa7939c1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638038b24d0 0x5638038c3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.140+0000 7fa7939c1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638038b2e80 0x5638038c4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.140+0000 7fa7939c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038bba00 0x5638038bef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.140+0000 7fa7939c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c1b0980 con 0x5638038bba00 2024-04-21T08:47:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.140+0000 7fa7939c1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c1b02b0 con 0x5638038b2e80 2024-04-21T08:47:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.140+0000 7fa7939c1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c1ad510 con 0x5638038b24d0 2024-04-21T08:47:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.140+0000 7fa7939c1740 10 client.0 _ll_register_callbacks cb 0x7fa77c009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.140+0000 7fa789d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038bba00 0x5638038bef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.140+0000 7fa789d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038bba00 0x5638038bef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58016/0 (socket says 172.21.15.43:58016) 2024-04-21T08:47:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.140+0000 7fa789d9b640 1 -- 172.21.15.43:0/2414087810 learned_addr learned my addr 172.21.15.43:0/2414087810 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.140+0000 7fa789d9b640 1 --2- 172.21.15.43:0/2414087810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638038b24d0 0x5638038c3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.140+0000 7fa78a59c640 1 --2- 172.21.15.43:0/2414087810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638038b2e80 0x5638038c4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.140+0000 7fa789d9b640 1 -- 172.21.15.43:0/2414087810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638038b24d0 msgr2=0x5638038c3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.140+0000 7fa789d9b640 1 --2- 172.21.15.43:0/2414087810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638038b24d0 0x5638038c3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.140+0000 7fa789d9b640 1 -- 172.21.15.43:0/2414087810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638038b2e80 msgr2=0x5638038c4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.140+0000 7fa789d9b640 1 --2- 172.21.15.43:0/2414087810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638038b2e80 0x5638038c4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.140+0000 7fa789d9b640 1 -- 172.21.15.43:0/2414087810 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638038bf700 con 0x5638038bba00 2024-04-21T08:47:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.141+0000 7fa789d9b640 1 --2- 172.21.15.43:0/2414087810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038bba00 0x5638038bef00 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fa78c06a420 tx=0x7fa78002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50dc3b2f11f38220 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.141+0000 7fa771ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.141+0000 7fa771ffb640 1 -- 172.21.15.43:0/2414087810 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7800526f0 con 0x5638038bba00 2024-04-21T08:47:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.141+0000 7fa771ffb640 1 -- 172.21.15.43:0/2414087810 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fa78003a040 con 0x5638038bba00 2024-04-21T08:47:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.141+0000 7fa7939c1740 1 -- 172.21.15.43:0/2414087810 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa77c029070 con 0x5638038bba00 2024-04-21T08:47:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.141+0000 7fa771ffb640 1 -- 172.21.15.43:0/2414087810 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa780067050 con 0x5638038bba00 2024-04-21T08:47:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.141+0000 7fa7939c1740 10 client.19746 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.141+0000 7fa7939c1740 1 -- 172.21.15.43:0/2414087810 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5638039319a0 con 0x5638038bba00 2024-04-21T08:47:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.141+0000 7fa771ffb640 1 -- 172.21.15.43:0/2414087810 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa77c029070 con 0x5638038bba00 2024-04-21T08:47:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.141+0000 7fa771ffb640 1 -- 172.21.15.43:0/2414087810 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa780043450 con 0x5638038bba00 2024-04-21T08:47:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.141+0000 7fa7939c1740 10 client.19746 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.141+0000 7fa7939c1740 10 client.19746 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:47:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.141+0000 7fa7939c1740 20 client.19746 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.141+0000 7fa7939c1740 1 --2- 172.21.15.43:0/2414087810 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5638039320a0 0x5638039344c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.141+0000 7fa7939c1740 4 client.19746 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.141+0000 7fa7939c1740 1 -- 172.21.15.43:0/2414087810 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563803824c10 con 0x5638039320a0 2024-04-21T08:47:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.142+0000 7fa78a59c640 1 --2- 172.21.15.43:0/2414087810 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5638039320a0 0x5638039344c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.142+0000 7fa78a59c640 1 --2- 172.21.15.43:0/2414087810 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5638039320a0 0x5638039344c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8619a74fe40a6093 server_cookie=20a87e24a941c8d9 in_seq=0 out_seq=0 2024-04-21T08:47:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.142+0000 7fa771ffb640 10 client.19746 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:47:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.142+0000 7fa771ffb640 1 -- 172.21.15.43:0/2414087810 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563803824c10 con 0x5638039320a0 2024-04-21T08:47:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.142+0000 7fa771ffb640 10 client.19746 handle_command_reply: tid=0 2024-04-21T08:47:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.142+0000 7fa7939c1740 1 client.19746 shutdown 2024-04-21T08:47:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.142+0000 7fa7939c1740 1 -- 172.21.15.43:0/2414087810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038bba00 msgr2=0x5638038bef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.142+0000 7fa7939c1740 1 --2- 172.21.15.43:0/2414087810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038bba00 0x5638038bef00 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fa78c06a420 tx=0x7fa78002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:50.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.142+0000 7fa7939c1740 1 -- 172.21.15.43:0/2414087810 shutdown_connections 2024-04-21T08:47:50.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.142+0000 7fa7939c1740 1 --2- 172.21.15.43:0/2414087810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5638038bba00 0x5638038bef00 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.142+0000 7fa7939c1740 1 --2- 172.21.15.43:0/2414087810 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5638039320a0 0x5638039344c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.142+0000 7fa7939c1740 1 --2- 172.21.15.43:0/2414087810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5638038b2e80 0x5638038c4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.142+0000 7fa7939c1740 1 --2- 172.21.15.43:0/2414087810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5638038b24d0 0x5638038c3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.142+0000 7fa771ffb640 0 client.19746 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:47:50.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.142+0000 7fa7939c1740 1 -- 172.21.15.43:0/2414087810 >> 172.21.15.43:0/2414087810 conn(0x5638038acba0 msgr2=0x5638038b3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:50.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.142+0000 7fa7939c1740 1 -- 172.21.15.43:0/2414087810 shutdown_connections 2024-04-21T08:47:50.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.142+0000 7fa7939c1740 1 -- 172.21.15.43:0/2414087810 wait complete. 2024-04-21T08:47:50.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.142+0000 7fa7939c1740 20 client.19746 trim_cache size 0 max 16384 2024-04-21T08:47:50.150 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:47:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.143+0000 7fa790cf2640 1 -- 172.21.15.43:0/4167226895 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa7740a3d20 msgr2=0x7fa7740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.143+0000 7fa790cf2640 1 --2- 172.21.15.43:0/4167226895 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa7740a3d20 0x7fa7740a61e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa7840ce080 tx=0x7fa78406c1b0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.143+0000 7fa790cf2640 1 -- 172.21.15.43:0/4167226895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa78c19cfb0 msgr2=0x7fa78c19d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.143+0000 7fa790cf2640 1 --2- 172.21.15.43:0/4167226895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa78c19cfb0 0x7fa78c19d460 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fa78400eaa0 tx=0x7fa78400ef60 comp rx=0 tx=0).stop 2024-04-21T08:47:50.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.143+0000 7fa790cf2640 1 -- 172.21.15.43:0/4167226895 shutdown_connections 2024-04-21T08:47:50.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.143+0000 7fa790cf2640 1 --2- 172.21.15.43:0/4167226895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa78c19cfb0 0x7fa78c19d460 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.143+0000 7fa790cf2640 1 --2- 172.21.15.43:0/4167226895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa78c107270 0x7fa78c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.143+0000 7fa790cf2640 1 --2- 172.21.15.43:0/4167226895 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa7740a3d20 0x7fa7740a61e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.143+0000 7fa790cf2640 1 --2- 172.21.15.43:0/4167226895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa78c06a840 0x7fa78c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.143+0000 7fa790cf2640 1 -- 172.21.15.43:0/4167226895 >> 172.21.15.43:0/4167226895 conn(0x7fa78c077eb0 msgr2=0x7fa78c108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:50.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.143+0000 7fa790cf2640 1 -- 172.21.15.43:0/4167226895 shutdown_connections 2024-04-21T08:47:50.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.143+0000 7fa790cf2640 1 -- 172.21.15.43:0/4167226895 wait complete. 2024-04-21T08:47:50.154 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:47:50.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.258+0000 7fa884406640 1 -- 172.21.15.43:0/774365480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa87c107250 msgr2=0x7fa87c10ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.258+0000 7fa884406640 1 --2- 172.21.15.43:0/774365480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa87c107250 0x7fa87c10ee20 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fa874009ca0 tx=0x7fa87402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.258+0000 7fa884406640 1 -- 172.21.15.43:0/774365480 shutdown_connections 2024-04-21T08:47:50.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.258+0000 7fa884406640 1 --2- 172.21.15.43:0/774365480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa87c107250 0x7fa87c10ee20 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.258+0000 7fa884406640 1 --2- 172.21.15.43:0/774365480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa87c102b30 0x7fa87c106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.258+0000 7fa884406640 1 --2- 172.21.15.43:0/774365480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa87c102180 0x7fa87c102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.258+0000 7fa884406640 1 -- 172.21.15.43:0/774365480 >> 172.21.15.43:0/774365480 conn(0x7fa87c0fd9c0 msgr2=0x7fa87c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:50.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.258+0000 7fa884406640 1 -- 172.21.15.43:0/774365480 shutdown_connections 2024-04-21T08:47:50.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.258+0000 7fa884406640 1 -- 172.21.15.43:0/774365480 wait complete. 2024-04-21T08:47:50.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.258+0000 7fa884406640 1 Processor -- start 2024-04-21T08:47:50.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.259+0000 7fa884406640 1 -- start start 2024-04-21T08:47:50.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.259+0000 7fa884406640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa87c102180 0x7fa87c19e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.259+0000 7fa884406640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa87c107250 0x7fa87c19ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.259+0000 7fa884406640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa87c198650 0x7fa87c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.259+0000 7fa884406640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c1121a0 con 0x7fa87c102180 2024-04-21T08:47:50.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.259+0000 7fa884406640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c112020 con 0x7fa87c107250 2024-04-21T08:47:50.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.259+0000 7fa884406640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c112320 con 0x7fa87c198650 2024-04-21T08:47:50.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.259+0000 7fa8821a2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa87c102180 0x7fa87c19e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.259+0000 7fa8821a2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa87c102180 0x7fa87c19e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58034/0 (socket says 172.21.15.43:58034) 2024-04-21T08:47:50.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.259+0000 7fa8821a2640 1 -- 172.21.15.43:0/3078712127 learned_addr learned my addr 172.21.15.43:0/3078712127 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:50.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.259+0000 7fa8829a3640 1 --2- 172.21.15.43:0/3078712127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa87c198650 0x7fa87c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.259+0000 7fa8819a1640 1 --2- 172.21.15.43:0/3078712127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa87c107250 0x7fa87c19ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.259+0000 7fa8821a2640 1 -- 172.21.15.43:0/3078712127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa87c198650 msgr2=0x7fa87c198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.259+0000 7fa8821a2640 1 --2- 172.21.15.43:0/3078712127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa87c198650 0x7fa87c198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.259+0000 7fa8821a2640 1 -- 172.21.15.43:0/3078712127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa87c107250 msgr2=0x7fa87c19ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.259+0000 7fa8821a2640 1 --2- 172.21.15.43:0/3078712127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa87c107250 0x7fa87c19ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.259+0000 7fa8821a2640 1 -- 172.21.15.43:0/3078712127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87c199230 con 0x7fa87c102180 2024-04-21T08:47:50.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.259+0000 7fa8821a2640 1 --2- 172.21.15.43:0/3078712127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa87c102180 0x7fa87c19e500 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fa87800d950 tx=0x7fa87800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:50.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.260+0000 7fa8737fe640 1 -- 172.21.15.43:0/3078712127 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878017030 con 0x7fa87c102180 2024-04-21T08:47:50.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.260+0000 7fa884406640 1 -- 172.21.15.43:0/3078712127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87c1a5310 con 0x7fa87c102180 2024-04-21T08:47:50.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.260+0000 7fa8737fe640 1 -- 172.21.15.43:0/3078712127 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa878004020 con 0x7fa87c102180 2024-04-21T08:47:50.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.260+0000 7fa8737fe640 1 -- 172.21.15.43:0/3078712127 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8780047a0 con 0x7fa87c102180 2024-04-21T08:47:50.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.260+0000 7fa884406640 1 -- 172.21.15.43:0/3078712127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa87c1a56c0 con 0x7fa87c102180 2024-04-21T08:47:50.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.262+0000 7fa8737fe640 1 -- 172.21.15.43:0/3078712127 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa878009a30 con 0x7fa87c102180 2024-04-21T08:47:50.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.262+0000 7fa886ff4740 1 Processor -- start 2024-04-21T08:47:50.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.262+0000 7fa886ff4740 1 -- start start 2024-04-21T08:47:50.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.263+0000 7fa886ff4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b144893b0 0x559b14489790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.263+0000 7fa886ff4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b14489d60 0x559b14492890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.263+0000 7fa886ff4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b14492e60 0x559b144932c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.263+0000 7fa886ff4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559b143ee1f0 con 0x559b14489d60 2024-04-21T08:47:50.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.263+0000 7fa886ff4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559b1438b4b0 con 0x559b144893b0 2024-04-21T08:47:50.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.263+0000 7fa886ff4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559b143ea1b0 con 0x559b14492e60 2024-04-21T08:47:50.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.263+0000 7fa8737fe640 1 --2- 172.21.15.43:0/3078712127 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa8680a3d20 0x7fa8680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.263+0000 7fa8829a3640 1 --2- 172.21.15.43:0/3078712127 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa8680a3d20 0x7fa8680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.263+0000 7fa8821a2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b14489d60 0x559b14492890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.263+0000 7fa8821a2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b14489d60 0x559b14492890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58044/0 (socket says 172.21.15.43:58044) 2024-04-21T08:47:50.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.263+0000 7fa8821a2640 1 -- 172.21.15.43:0/1266740646 learned_addr learned my addr 172.21.15.43:0/1266740646 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:50.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.263+0000 7fa8819a1640 1 --2- 172.21.15.43:0/1266740646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b14492e60 0x559b144932c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.263+0000 7fa8737fe640 1 -- 172.21.15.43:0/3078712127 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fa8780d0560 con 0x7fa87c102180 2024-04-21T08:47:50.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.263+0000 7fa8829a3640 1 --2- 172.21.15.43:0/1266740646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b144893b0 0x559b14489790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.263+0000 7fa8821a2640 1 -- 172.21.15.43:0/1266740646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b14492e60 msgr2=0x559b144932c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.263+0000 7fa8821a2640 1 --2- 172.21.15.43:0/1266740646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b14492e60 0x559b144932c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.263+0000 7fa8821a2640 1 -- 172.21.15.43:0/1266740646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b144893b0 msgr2=0x559b14489790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.263+0000 7fa8821a2640 1 --2- 172.21.15.43:0/1266740646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b144893b0 0x559b14489790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.263+0000 7fa8821a2640 1 -- 172.21.15.43:0/1266740646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b1437d1b0 con 0x559b14489d60 2024-04-21T08:47:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.263+0000 7fa8829a3640 1 --2- 172.21.15.43:0/1266740646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b144893b0 0x559b14489790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.263+0000 7fa8829a3640 1 --2- 172.21.15.43:0/3078712127 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa8680a3d20 0x7fa8680a61e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa874004af0 tx=0x7fa87402cf10 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.263+0000 7fa8821a2640 1 --2- 172.21.15.43:0/1266740646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b14489d60 0x559b14492890 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fa87809c070 tx=0x7fa87806b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f277cf85e5d75995 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.264+0000 7fa870bfe640 1 -- 172.21.15.43:0/1266740646 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87809c7e0 con 0x559b14489d60 2024-04-21T08:47:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.264+0000 7fa870bfe640 1 -- 172.21.15.43:0/1266740646 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa87809c0e0 con 0x559b14489d60 2024-04-21T08:47:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.264+0000 7fa870bfe640 1 -- 172.21.15.43:0/1266740646 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8780cdb10 con 0x559b14489d60 2024-04-21T08:47:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.264+0000 7fa886ff4740 1 -- 172.21.15.43:0/1266740646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b14489d60 msgr2=0x559b14492890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.264+0000 7fa886ff4740 1 --2- 172.21.15.43:0/1266740646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b14489d60 0x559b14492890 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fa87809c070 tx=0x7fa87806b9c0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.264+0000 7fa886ff4740 1 -- 172.21.15.43:0/1266740646 shutdown_connections 2024-04-21T08:47:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.264+0000 7fa886ff4740 1 --2- 172.21.15.43:0/1266740646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b14492e60 0x559b144932c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.264+0000 7fa886ff4740 1 --2- 172.21.15.43:0/1266740646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b14489d60 0x559b14492890 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.264+0000 7fa886ff4740 1 --2- 172.21.15.43:0/1266740646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b144893b0 0x559b14489790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.264+0000 7fa886ff4740 1 -- 172.21.15.43:0/1266740646 >> 172.21.15.43:0/1266740646 conn(0x559b14483ba0 msgr2=0x559b14483f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.264+0000 7fa886ff4740 1 -- 172.21.15.43:0/1266740646 shutdown_connections 2024-04-21T08:47:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.264+0000 7fa886ff4740 1 -- 172.21.15.43:0/1266740646 wait complete. 2024-04-21T08:47:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.265+0000 7fa886ff4740 1 -- start start 2024-04-21T08:47:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.265+0000 7fa886ff4740 1 Processor -- start 2024-04-21T08:47:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.265+0000 7fa886ff4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b144893b0 0x559b144992a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.265+0000 7fa886ff4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b14489d60 0x559b14496630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.265+0000 7fa886ff4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b14492e60 0x559b14496b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.265+0000 7fa886ff4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559b14499da0 con 0x559b144893b0 2024-04-21T08:47:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.265+0000 7fa886ff4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559b143a7500 con 0x559b14489d60 2024-04-21T08:47:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.265+0000 7fa886ff4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559b14499f20 con 0x559b14492e60 2024-04-21T08:47:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.265+0000 7fa886ff4740 10 client.0 _ll_register_callbacks cb 0x559b14375b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.265+0000 7fa8821a2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b144893b0 0x559b144992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.265+0000 7fa8821a2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b144893b0 0x559b144992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58056/0 (socket says 172.21.15.43:58056) 2024-04-21T08:47:50.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.265+0000 7fa8821a2640 1 -- 172.21.15.43:0/238281973 learned_addr learned my addr 172.21.15.43:0/238281973 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:50.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.265+0000 7fa8829a3640 1 --2- 172.21.15.43:0/238281973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b14492e60 0x559b14496b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.265+0000 7fa8819a1640 1 --2- 172.21.15.43:0/238281973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b14489d60 0x559b14496630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.266+0000 7fa8821a2640 1 -- 172.21.15.43:0/238281973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b14492e60 msgr2=0x559b14496b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.266+0000 7fa8821a2640 1 --2- 172.21.15.43:0/238281973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b14492e60 0x559b14496b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.266+0000 7fa8821a2640 1 -- 172.21.15.43:0/238281973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b14489d60 msgr2=0x559b14496630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.266+0000 7fa8821a2640 1 --2- 172.21.15.43:0/238281973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b14489d60 0x559b14496630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.266+0000 7fa8821a2640 1 -- 172.21.15.43:0/238281973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b141003f0 con 0x559b144893b0 2024-04-21T08:47:50.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.266+0000 7fa8819a1640 1 --2- 172.21.15.43:0/238281973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b14489d60 0x559b14496630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:50.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.266+0000 7fa8821a2640 1 --2- 172.21.15.43:0/238281973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b144893b0 0x559b144992a0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fa878098ab0 tx=0x7fa878004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28fb09ba4ec6a64 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:50.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.266+0000 7fa865ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:50.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.266+0000 7fa865ffb640 1 -- 172.21.15.43:0/238281973 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87809c7e0 con 0x559b144893b0 2024-04-21T08:47:50.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.266+0000 7fa865ffb640 1 -- 172.21.15.43:0/238281973 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fa8780a0120 con 0x559b144893b0 2024-04-21T08:47:50.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.266+0000 7fa886ff4740 1 -- 172.21.15.43:0/238281973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa87809c0e0 con 0x559b144893b0 2024-04-21T08:47:50.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.266+0000 7fa865ffb640 1 -- 172.21.15.43:0/238281973 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8780a0490 con 0x559b144893b0 2024-04-21T08:47:50.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.266+0000 7fa886ff4740 10 client.19758 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:50.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.266+0000 7fa886ff4740 1 -- 172.21.15.43:0/238281973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559b141ca7d0 con 0x559b144893b0 2024-04-21T08:47:50.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.266+0000 7fa865ffb640 1 -- 172.21.15.43:0/238281973 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa87809c0e0 con 0x559b144893b0 2024-04-21T08:47:50.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.266+0000 7fa865ffb640 1 -- 172.21.15.43:0/238281973 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa8780d0160 con 0x559b144893b0 2024-04-21T08:47:50.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.266+0000 7fa886ff4740 10 client.19758 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:50.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.266+0000 7fa886ff4740 10 client.19758 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:47:50.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.266+0000 7fa886ff4740 20 client.19758 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:50.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.266+0000 7fa886ff4740 1 --2- 172.21.15.43:0/238281973 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x559b144a6ed0 0x559b144a92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.266+0000 7fa886ff4740 4 client.19758 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:50.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.266+0000 7fa886ff4740 1 -- 172.21.15.43:0/238281973 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b1438d930 con 0x559b144a6ed0 2024-04-21T08:47:50.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.267+0000 7fa8819a1640 1 --2- 172.21.15.43:0/238281973 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x559b144a6ed0 0x559b144a92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.267+0000 7fa865ffb640 10 client.19758 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:47:50.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.267+0000 7fa8819a1640 1 --2- 172.21.15.43:0/238281973 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x559b144a6ed0 0x559b144a92c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=515e0db5a6353106 server_cookie=b84e38d8cdb643f8 in_seq=0 out_seq=0 2024-04-21T08:47:50.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.268+0000 7fa865ffb640 1 -- 172.21.15.43:0/238281973 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559b1438d930 con 0x559b144a6ed0 2024-04-21T08:47:50.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.268+0000 7fa865ffb640 10 client.19758 handle_command_reply: tid=0 2024-04-21T08:47:50.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.268+0000 7fa886ff4740 1 client.19758 shutdown 2024-04-21T08:47:50.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.268+0000 7fa886ff4740 1 -- 172.21.15.43:0/238281973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b144893b0 msgr2=0x559b144992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.268+0000 7fa886ff4740 1 --2- 172.21.15.43:0/238281973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b144893b0 0x559b144992a0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fa878098ab0 tx=0x7fa878004400 comp rx=0 tx=0).stop 2024-04-21T08:47:50.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.268+0000 7fa886ff4740 1 -- 172.21.15.43:0/238281973 shutdown_connections 2024-04-21T08:47:50.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.268+0000 7fa886ff4740 1 --2- 172.21.15.43:0/238281973 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x559b144a6ed0 0x559b144a92c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.268+0000 7fa886ff4740 1 --2- 172.21.15.43:0/238281973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b14492e60 0x559b14496b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.268+0000 7fa886ff4740 1 --2- 172.21.15.43:0/238281973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b14489d60 0x559b14496630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.268+0000 7fa865ffb640 0 client.19758 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:47:50.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.268+0000 7fa886ff4740 1 --2- 172.21.15.43:0/238281973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b144893b0 0x559b144992a0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.268+0000 7fa886ff4740 1 -- 172.21.15.43:0/238281973 >> 172.21.15.43:0/238281973 conn(0x559b14483ba0 msgr2=0x559b144858f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:50.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.268+0000 7fa886ff4740 1 -- 172.21.15.43:0/238281973 shutdown_connections 2024-04-21T08:47:50.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.268+0000 7fa886ff4740 1 -- 172.21.15.43:0/238281973 wait complete. 2024-04-21T08:47:50.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.269+0000 7fa886ff4740 20 client.19758 trim_cache size 0 max 16384 2024-04-21T08:47:50.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.282+0000 7fa886ff4740 1 Processor -- start 2024-04-21T08:47:50.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.282+0000 7fa886ff4740 1 -- start start 2024-04-21T08:47:50.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.282+0000 7fa886ff4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b144893b0 0x559b14494250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.282+0000 7fa886ff4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b14489d60 0x559b144954e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.282+0000 7fa886ff4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b14492e60 0x559b14498d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.282+0000 7fa886ff4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c1ad3d0 con 0x559b14489d60 2024-04-21T08:47:50.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.282+0000 7fa886ff4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c1aef10 con 0x559b144893b0 2024-04-21T08:47:50.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.282+0000 7fa886ff4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c1adaa0 con 0x559b14492e60 2024-04-21T08:47:50.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.282+0000 7fa8819a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b14489d60 0x559b144954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.282+0000 7fa8819a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b14489d60 0x559b144954e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58066/0 (socket says 172.21.15.43:58066) 2024-04-21T08:47:50.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.282+0000 7fa8819a1640 1 -- 172.21.15.43:0/3013904609 learned_addr learned my addr 172.21.15.43:0/3013904609 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:50.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.282+0000 7fa8821a2640 1 --2- 172.21.15.43:0/3013904609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b144893b0 0x559b14494250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.282+0000 7fa8819a1640 1 -- 172.21.15.43:0/3013904609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b14492e60 msgr2=0x559b14498d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.282+0000 7fa8819a1640 1 --2- 172.21.15.43:0/3013904609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b14492e60 0x559b14498d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.282+0000 7fa8819a1640 1 -- 172.21.15.43:0/3013904609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b144893b0 msgr2=0x559b14494250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.282+0000 7fa8819a1640 1 --2- 172.21.15.43:0/3013904609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b144893b0 0x559b14494250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.282+0000 7fa8819a1640 1 -- 172.21.15.43:0/3013904609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b14488560 con 0x559b14489d60 2024-04-21T08:47:50.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.282+0000 7fa8821a2640 1 --2- 172.21.15.43:0/3013904609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b144893b0 0x559b14494250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:50.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.283+0000 7fa8819a1640 1 --2- 172.21.15.43:0/3013904609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b14489d60 0x559b144954e0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0fdfe0 tx=0x7fa86c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f46e8d553a5f68b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:50.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.283+0000 7fa867fff640 1 -- 172.21.15.43:0/3013904609 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c005020 con 0x559b14489d60 2024-04-21T08:47:50.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.283+0000 7fa867fff640 1 -- 172.21.15.43:0/3013904609 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa86c023030 con 0x559b14489d60 2024-04-21T08:47:50.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.283+0000 7fa867fff640 1 -- 172.21.15.43:0/3013904609 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c01f0f0 con 0x559b14489d60 2024-04-21T08:47:50.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.283+0000 7fa886ff4740 1 -- 172.21.15.43:0/3013904609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b14489d60 msgr2=0x559b144954e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.283+0000 7fa886ff4740 1 --2- 172.21.15.43:0/3013904609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b14489d60 0x559b144954e0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0fdfe0 tx=0x7fa86c00bf10 comp rx=0 tx=0).stop 2024-04-21T08:47:50.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.283+0000 7fa886ff4740 1 -- 172.21.15.43:0/3013904609 shutdown_connections 2024-04-21T08:47:50.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.283+0000 7fa886ff4740 1 --2- 172.21.15.43:0/3013904609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b14492e60 0x559b14498d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.283+0000 7fa886ff4740 1 --2- 172.21.15.43:0/3013904609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b14489d60 0x559b144954e0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.283+0000 7fa886ff4740 1 --2- 172.21.15.43:0/3013904609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b144893b0 0x559b14494250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.283+0000 7fa886ff4740 1 -- 172.21.15.43:0/3013904609 >> 172.21.15.43:0/3013904609 conn(0x559b14483ba0 msgr2=0x559b14494dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:50.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.283+0000 7fa886ff4740 1 -- 172.21.15.43:0/3013904609 shutdown_connections 2024-04-21T08:47:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.283+0000 7fa886ff4740 1 -- 172.21.15.43:0/3013904609 wait complete. 2024-04-21T08:47:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.284+0000 7fa886ff4740 1 -- start start 2024-04-21T08:47:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.284+0000 7fa886ff4740 1 Processor -- start 2024-04-21T08:47:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.284+0000 7fa886ff4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b144893b0 0x559b144ac760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.284+0000 7fa886ff4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b14489d60 0x559b144a9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.284+0000 7fa886ff4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b14492e60 0x559b144aa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.284+0000 7fa886ff4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c1ae840 con 0x559b14492e60 2024-04-21T08:47:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.284+0000 7fa886ff4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c1ae170 con 0x559b144893b0 2024-04-21T08:47:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.284+0000 7fa886ff4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c1ab3d0 con 0x559b14489d60 2024-04-21T08:47:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.284+0000 7fa886ff4740 10 client.0 _ll_register_callbacks cb 0x7fa878008ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.284+0000 7fa8829a3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b14492e60 0x559b144aa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.284+0000 7fa8829a3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b14492e60 0x559b144aa090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58070/0 (socket says 172.21.15.43:58070) 2024-04-21T08:47:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.284+0000 7fa8829a3640 1 -- 172.21.15.43:0/1845339968 learned_addr learned my addr 172.21.15.43:0/1845339968 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.284+0000 7fa8821a2640 1 --2- 172.21.15.43:0/1845339968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b144893b0 0x559b144ac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.284+0000 7fa8829a3640 1 -- 172.21.15.43:0/1845339968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b14489d60 msgr2=0x559b144a9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.284+0000 7fa8819a1640 1 --2- 172.21.15.43:0/1845339968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b14489d60 0x559b144a9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.284+0000 7fa8829a3640 1 --2- 172.21.15.43:0/1845339968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b14489d60 0x559b144a9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.284+0000 7fa8829a3640 1 -- 172.21.15.43:0/1845339968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b144893b0 msgr2=0x559b144ac760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.284+0000 7fa8829a3640 1 --2- 172.21.15.43:0/1845339968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b144893b0 0x559b144ac760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.284+0000 7fa8829a3640 1 -- 172.21.15.43:0/1845339968 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b144aa8c0 con 0x559b14492e60 2024-04-21T08:47:50.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.284+0000 7fa8821a2640 1 --2- 172.21.15.43:0/1845339968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b144893b0 0x559b144ac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:50.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.285+0000 7fa8829a3640 1 --2- 172.21.15.43:0/1845339968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b14492e60 0x559b144aa090 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fa874009a60 tx=0x7fa8740095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f807b579a5cddb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:50.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.285+0000 7fa865ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:50.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.285+0000 7fa865ffb640 1 -- 172.21.15.43:0/1845339968 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874052020 con 0x559b14492e60 2024-04-21T08:47:50.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.285+0000 7fa865ffb640 1 -- 172.21.15.43:0/1845339968 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fa874004040 con 0x559b14492e60 2024-04-21T08:47:50.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.285+0000 7fa886ff4740 1 -- 172.21.15.43:0/1845339968 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa86c023030 con 0x559b14492e60 2024-04-21T08:47:50.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.285+0000 7fa865ffb640 1 -- 172.21.15.43:0/1845339968 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa87405d050 con 0x559b14492e60 2024-04-21T08:47:50.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.285+0000 7fa886ff4740 10 client.19764 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:50.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.285+0000 7fa886ff4740 1 -- 172.21.15.43:0/1845339968 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559b1450d180 con 0x559b14492e60 2024-04-21T08:47:50.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.285+0000 7fa865ffb640 1 -- 172.21.15.43:0/1845339968 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa86c023030 con 0x559b14492e60 2024-04-21T08:47:50.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.285+0000 7fa865ffb640 1 -- 172.21.15.43:0/1845339968 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa87403f030 con 0x559b14492e60 2024-04-21T08:47:50.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.285+0000 7fa886ff4740 10 client.19764 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:50.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.285+0000 7fa886ff4740 10 client.19764 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:47:50.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.285+0000 7fa886ff4740 20 client.19764 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:50.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.285+0000 7fa886ff4740 1 --2- 172.21.15.43:0/1845339968 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x559b1450d980 0x559b1450fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.285+0000 7fa886ff4740 4 client.19764 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:50.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.285+0000 7fa886ff4740 1 -- 172.21.15.43:0/1845339968 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b143fbc10 con 0x559b1450d980 2024-04-21T08:47:50.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.286+0000 7fa8821a2640 1 --2- 172.21.15.43:0/1845339968 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x559b1450d980 0x559b1450fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.286+0000 7fa8821a2640 1 --2- 172.21.15.43:0/1845339968 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x559b1450d980 0x559b1450fda0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=784655564201b21f server_cookie=d9c0d2ab9d8e5992 in_seq=0 out_seq=0 2024-04-21T08:47:50.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.286+0000 7fa865ffb640 10 client.19764 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:47:50.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.286+0000 7fa865ffb640 1 -- 172.21.15.43:0/1845339968 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b143fbc10 con 0x559b1450d980 2024-04-21T08:47:50.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.286+0000 7fa865ffb640 10 client.19764 handle_command_reply: tid=0 2024-04-21T08:47:50.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.286+0000 7fa886ff4740 1 client.19764 shutdown 2024-04-21T08:47:50.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.286+0000 7fa886ff4740 1 -- 172.21.15.43:0/1845339968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b14492e60 msgr2=0x559b144aa090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.286+0000 7fa886ff4740 1 --2- 172.21.15.43:0/1845339968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b14492e60 0x559b144aa090 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fa874009a60 tx=0x7fa8740095c0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.286+0000 7fa886ff4740 1 -- 172.21.15.43:0/1845339968 shutdown_connections 2024-04-21T08:47:50.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.286+0000 7fa886ff4740 1 --2- 172.21.15.43:0/1845339968 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x559b1450d980 0x559b1450fda0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.286+0000 7fa886ff4740 1 --2- 172.21.15.43:0/1845339968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b14492e60 0x559b144aa090 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.286+0000 7fa865ffb640 0 client.19764 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:47:50.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.286+0000 7fa886ff4740 1 --2- 172.21.15.43:0/1845339968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b14489d60 0x559b144a9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.286+0000 7fa886ff4740 1 --2- 172.21.15.43:0/1845339968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b144893b0 0x559b144ac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.287+0000 7fa886ff4740 1 -- 172.21.15.43:0/1845339968 >> 172.21.15.43:0/1845339968 conn(0x559b14483ba0 msgr2=0x559b144a2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:50.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.287+0000 7fa886ff4740 1 -- 172.21.15.43:0/1845339968 shutdown_connections 2024-04-21T08:47:50.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.287+0000 7fa886ff4740 1 -- 172.21.15.43:0/1845339968 wait complete. 2024-04-21T08:47:50.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.287+0000 7fa886ff4740 20 client.19764 trim_cache size 0 max 16384 2024-04-21T08:47:50.294 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:47:50.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.287+0000 7fa884406640 1 -- 172.21.15.43:0/3078712127 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa8680a3d20 msgr2=0x7fa8680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.287+0000 7fa884406640 1 --2- 172.21.15.43:0/3078712127 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa8680a3d20 0x7fa8680a61e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa874004af0 tx=0x7fa87402cf10 comp rx=0 tx=0).stop 2024-04-21T08:47:50.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.287+0000 7fa884406640 1 -- 172.21.15.43:0/3078712127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa87c102180 msgr2=0x7fa87c19e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.287+0000 7fa884406640 1 --2- 172.21.15.43:0/3078712127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa87c102180 0x7fa87c19e500 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fa87800d950 tx=0x7fa87800de10 comp rx=0 tx=0).stop 2024-04-21T08:47:50.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.287+0000 7fa884406640 1 -- 172.21.15.43:0/3078712127 shutdown_connections 2024-04-21T08:47:50.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.287+0000 7fa884406640 1 --2- 172.21.15.43:0/3078712127 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa8680a3d20 0x7fa8680a61e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.287+0000 7fa884406640 1 --2- 172.21.15.43:0/3078712127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa87c198650 0x7fa87c198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.287+0000 7fa884406640 1 --2- 172.21.15.43:0/3078712127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa87c107250 0x7fa87c19ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.287+0000 7fa884406640 1 --2- 172.21.15.43:0/3078712127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa87c102180 0x7fa87c19e500 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.287+0000 7fa884406640 1 -- 172.21.15.43:0/3078712127 >> 172.21.15.43:0/3078712127 conn(0x7fa87c0fd9c0 msgr2=0x7fa87c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:50.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.287+0000 7fa884406640 1 -- 172.21.15.43:0/3078712127 shutdown_connections 2024-04-21T08:47:50.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.287+0000 7fa884406640 1 -- 172.21.15.43:0/3078712127 wait complete. 2024-04-21T08:47:50.297 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:50 smithi043 ceph-mon[27978]: pgmap v275: 129 pgs: 129 active+clean; 5.6 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:47:50.297 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:50 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:47:50.297 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:50 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:50.297 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:50 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:47:50.297 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:50 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:50.297 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:50 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:47:50.298 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:50 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2941763346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:50.299 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:47:50.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.402+0000 7f9ebd1b2640 1 -- 172.21.15.43:0/168179770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb806b1f0 msgr2=0x7f9eb8106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.402+0000 7f9ebd1b2640 1 --2- 172.21.15.43:0/168179770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb806b1f0 0x7f9eb8106b90 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009900 tx=0x7f9eac02bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:50.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.402+0000 7f9ebd1b2640 1 -- 172.21.15.43:0/168179770 shutdown_connections 2024-04-21T08:47:50.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.402+0000 7f9ebd1b2640 1 --2- 172.21.15.43:0/168179770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9eb8107270 0x7f9eb810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.402+0000 7f9ebd1b2640 1 --2- 172.21.15.43:0/168179770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb806b1f0 0x7f9eb8106b90 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.402+0000 7f9ebd1b2640 1 --2- 172.21.15.43:0/168179770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eb806a840 0x7f9eb806ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.402+0000 7f9ebd1b2640 1 -- 172.21.15.43:0/168179770 >> 172.21.15.43:0/168179770 conn(0x7f9eb8077eb0 msgr2=0x7f9eb80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:50.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.402+0000 7f9ebd1b2640 1 -- 172.21.15.43:0/168179770 shutdown_connections 2024-04-21T08:47:50.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.402+0000 7f9ebd1b2640 1 -- 172.21.15.43:0/168179770 wait complete. 2024-04-21T08:47:50.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.403+0000 7f9ebd1b2640 1 Processor -- start 2024-04-21T08:47:50.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.403+0000 7f9ebd1b2640 1 -- start start 2024-04-21T08:47:50.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.403+0000 7f9ebd1b2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eb806a840 0x7f9eb8100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.403+0000 7f9ebd1b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb8107270 0x7f9eb81013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.403+0000 7f9ebd1b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9eb8101900 0x7f9eb81a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.403+0000 7f9ebd1b2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb8112200 con 0x7f9eb8107270 2024-04-21T08:47:50.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.403+0000 7f9ebd1b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb8112080 con 0x7f9eb8101900 2024-04-21T08:47:50.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.403+0000 7f9ebd1b2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb8112380 con 0x7f9eb806a840 2024-04-21T08:47:50.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.404+0000 7f9eb659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb8107270 0x7f9eb81013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.404+0000 7f9eb659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb8107270 0x7f9eb81013c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58096/0 (socket says 172.21.15.43:58096) 2024-04-21T08:47:50.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.404+0000 7f9eb659c640 1 -- 172.21.15.43:0/1592323396 learned_addr learned my addr 172.21.15.43:0/1592323396 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:50.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.404+0000 7f9eb6d9d640 1 --2- 172.21.15.43:0/1592323396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eb806a840 0x7f9eb8100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.404+0000 7f9eb759e640 1 --2- 172.21.15.43:0/1592323396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9eb8101900 0x7f9eb81a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.404+0000 7f9eb659c640 1 -- 172.21.15.43:0/1592323396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eb806a840 msgr2=0x7f9eb8100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.404+0000 7f9eb659c640 1 --2- 172.21.15.43:0/1592323396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eb806a840 0x7f9eb8100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.404+0000 7f9eb659c640 1 -- 172.21.15.43:0/1592323396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9eb8101900 msgr2=0x7f9eb81a6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.404+0000 7f9eb659c640 1 --2- 172.21.15.43:0/1592323396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9eb8101900 0x7f9eb81a6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.404+0000 7f9eb659c640 1 -- 172.21.15.43:0/1592323396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb81a7010 con 0x7f9eb8107270 2024-04-21T08:47:50.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.404+0000 7f9eb659c640 1 --2- 172.21.15.43:0/1592323396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb8107270 0x7f9eb81013c0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f9eac000c00 tx=0x7f9eac0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:50.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.404+0000 7f9e9ffff640 1 -- 172.21.15.43:0/1592323396 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac043020 con 0x7f9eb8107270 2024-04-21T08:47:50.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.404+0000 7f9ebd1b2640 1 -- 172.21.15.43:0/1592323396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb81a72a0 con 0x7f9eb8107270 2024-04-21T08:47:50.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.404+0000 7f9e9ffff640 1 -- 172.21.15.43:0/1592323396 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9eac02cb70 con 0x7f9eb8107270 2024-04-21T08:47:50.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.404+0000 7f9e9ffff640 1 -- 172.21.15.43:0/1592323396 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac003b80 con 0x7f9eb8107270 2024-04-21T08:47:50.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.404+0000 7f9ebd1b2640 1 -- 172.21.15.43:0/1592323396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb81a77f0 con 0x7f9eb8107270 2024-04-21T08:47:50.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.407+0000 7f9e9ffff640 1 -- 172.21.15.43:0/1592323396 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9eac046020 con 0x7f9eb8107270 2024-04-21T08:47:50.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.407+0000 7f9ebfda6740 1 Processor -- start 2024-04-21T08:47:50.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.407+0000 7f9ebfda6740 1 -- start start 2024-04-21T08:47:50.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.408+0000 7f9ebfda6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559825ddd4d0 0x559825ddd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.408+0000 7f9ebfda6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559825ddde80 0x559825de6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.408+0000 7f9ebfda6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825de6a00 0x559825de8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.408+0000 7f9ebfda6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559825d421f0 con 0x559825de6a00 2024-04-21T08:47:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.408+0000 7f9ebfda6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559825cdf4b0 con 0x559825ddde80 2024-04-21T08:47:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.408+0000 7f9ebfda6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559825d3e1b0 con 0x559825ddd4d0 2024-04-21T08:47:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.408+0000 7f9eb659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825de6a00 0x559825de8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.408+0000 7f9eb659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825de6a00 0x559825de8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58102/0 (socket says 172.21.15.43:58102) 2024-04-21T08:47:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.408+0000 7f9eb659c640 1 -- 172.21.15.43:0/681603713 learned_addr learned my addr 172.21.15.43:0/681603713 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.408+0000 7f9eb759e640 1 --2- 172.21.15.43:0/681603713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559825ddd4d0 0x559825ddd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.408+0000 7f9e9ffff640 1 --2- 172.21.15.43:0/1592323396 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9ea00a3d50 0x7f9ea00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.408+0000 7f9eb6d9d640 1 --2- 172.21.15.43:0/681603713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559825ddde80 0x559825de6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.408+0000 7f9eb659c640 1 -- 172.21.15.43:0/681603713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559825ddd4d0 msgr2=0x559825ddd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.408+0000 7f9eb659c640 1 --2- 172.21.15.43:0/681603713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559825ddd4d0 0x559825ddd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.408+0000 7f9eb659c640 1 -- 172.21.15.43:0/681603713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559825ddde80 msgr2=0x559825de6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.408+0000 7f9eb659c640 1 --2- 172.21.15.43:0/681603713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559825ddde80 0x559825de6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.408+0000 7f9eb659c640 1 -- 172.21.15.43:0/681603713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559825cd11b0 con 0x559825de6a00 2024-04-21T08:47:50.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.408+0000 7f9eb759e640 1 --2- 172.21.15.43:0/681603713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559825ddd4d0 0x559825ddd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:50.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.408+0000 7f9eb759e640 1 --2- 172.21.15.43:0/1592323396 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9ea00a3d50 0x7f9ea00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.408+0000 7f9e9ffff640 1 -- 172.21.15.43:0/1592323396 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f9eac0f2b50 con 0x7f9eb8107270 2024-04-21T08:47:50.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.408+0000 7f9eb659c640 1 --2- 172.21.15.43:0/681603713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825de6a00 0x559825de8e50 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0eff00 tx=0x7f9eac07e330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b482442156879c88 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:50.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.409+0000 7f9e9d7fa640 1 -- 172.21.15.43:0/681603713 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac0be5b0 con 0x559825de6a00 2024-04-21T08:47:50.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.409+0000 7f9eb759e640 1 --2- 172.21.15.43:0/1592323396 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9ea00a3d50 0x7f9ea00a6210 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0005fd0 tx=0x7f9eb0014040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:50.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.409+0000 7f9e9d7fa640 1 -- 172.21.15.43:0/681603713 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9eac0beee0 con 0x559825de6a00 2024-04-21T08:47:50.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.409+0000 7f9e9d7fa640 1 -- 172.21.15.43:0/681603713 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac03bb60 con 0x559825de6a00 2024-04-21T08:47:50.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.409+0000 7f9ebfda6740 1 -- 172.21.15.43:0/681603713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825de6a00 msgr2=0x559825de8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.409+0000 7f9ebfda6740 1 --2- 172.21.15.43:0/681603713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825de6a00 0x559825de8e50 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0eff00 tx=0x7f9eac07e330 comp rx=0 tx=0).stop 2024-04-21T08:47:50.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.409+0000 7f9ebfda6740 1 -- 172.21.15.43:0/681603713 shutdown_connections 2024-04-21T08:47:50.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.409+0000 7f9ebfda6740 1 --2- 172.21.15.43:0/681603713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825de6a00 0x559825de8e50 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.409+0000 7f9ebfda6740 1 --2- 172.21.15.43:0/681603713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559825ddde80 0x559825de6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.409+0000 7f9ebfda6740 1 --2- 172.21.15.43:0/681603713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559825ddd4d0 0x559825ddd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.409+0000 7f9ebfda6740 1 -- 172.21.15.43:0/681603713 >> 172.21.15.43:0/681603713 conn(0x559825dd7ba0 msgr2=0x559825dd7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:50.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.409+0000 7f9ebfda6740 1 -- 172.21.15.43:0/681603713 shutdown_connections 2024-04-21T08:47:50.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.409+0000 7f9ebfda6740 1 -- 172.21.15.43:0/681603713 wait complete. 2024-04-21T08:47:50.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.410+0000 7f9ebfda6740 1 -- start start 2024-04-21T08:47:50.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.410+0000 7f9ebfda6740 1 Processor -- start 2024-04-21T08:47:50.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.410+0000 7f9ebfda6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559825ddd4d0 0x559825de62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.410+0000 7f9ebfda6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825ddde80 0x559825de3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.410+0000 7f9ebfda6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559825de6a00 0x559825de3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.410+0000 7f9ebfda6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559825de9730 con 0x559825ddde80 2024-04-21T08:47:50.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.410+0000 7f9ebfda6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559825cfb500 con 0x559825ddd4d0 2024-04-21T08:47:50.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.410+0000 7f9ebfda6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559825de98b0 con 0x559825de6a00 2024-04-21T08:47:50.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.410+0000 7f9ebfda6740 10 client.0 _ll_register_callbacks cb 0x559825cc9b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.410+0000 7f9eb659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825ddde80 0x559825de3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.410+0000 7f9eb659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825ddde80 0x559825de3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58116/0 (socket says 172.21.15.43:58116) 2024-04-21T08:47:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.410+0000 7f9eb659c640 1 -- 172.21.15.43:0/3467418882 learned_addr learned my addr 172.21.15.43:0/3467418882 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.410+0000 7f9eb6d9d640 1 --2- 172.21.15.43:0/3467418882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559825ddd4d0 0x559825de62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.410+0000 7f9eb759e640 1 --2- 172.21.15.43:0/3467418882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559825de6a00 0x559825de3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.410+0000 7f9eb659c640 1 -- 172.21.15.43:0/3467418882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559825de6a00 msgr2=0x559825de3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.410+0000 7f9eb659c640 1 --2- 172.21.15.43:0/3467418882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559825de6a00 0x559825de3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.410+0000 7f9eb659c640 1 -- 172.21.15.43:0/3467418882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559825ddd4d0 msgr2=0x559825de62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.410+0000 7f9eb659c640 1 --2- 172.21.15.43:0/3467418882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559825ddd4d0 0x559825de62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.410+0000 7f9eb659c640 1 -- 172.21.15.43:0/3467418882 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559825a543f0 con 0x559825ddde80 2024-04-21T08:47:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.411+0000 7f9eb659c640 1 --2- 172.21.15.43:0/3467418882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825ddde80 0x559825de3660 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f9eac02bc70 tx=0x7f9eac003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3455d0e30e256b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.411+0000 7f9e9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.411+0000 7f9e9aff5640 1 -- 172.21.15.43:0/3467418882 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac03eaa0 con 0x559825ddde80 2024-04-21T08:47:50.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.411+0000 7f9e9aff5640 1 -- 172.21.15.43:0/3467418882 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f9eac003d20 con 0x559825ddde80 2024-04-21T08:47:50.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.411+0000 7f9ebfda6740 1 -- 172.21.15.43:0/3467418882 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9eac0beee0 con 0x559825ddde80 2024-04-21T08:47:50.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.411+0000 7f9e9aff5640 1 -- 172.21.15.43:0/3467418882 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9eac0be140 con 0x559825ddde80 2024-04-21T08:47:50.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.411+0000 7f9ebfda6740 10 client.19776 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:50.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.411+0000 7f9ebfda6740 1 -- 172.21.15.43:0/3467418882 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559825b1e7d0 con 0x559825ddde80 2024-04-21T08:47:50.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.411+0000 7f9e9aff5640 1 -- 172.21.15.43:0/3467418882 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9eac0beee0 con 0x559825ddde80 2024-04-21T08:47:50.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.411+0000 7f9e9aff5640 1 -- 172.21.15.43:0/3467418882 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9eac0bae50 con 0x559825ddde80 2024-04-21T08:47:50.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.411+0000 7f9ebfda6740 10 client.19776 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:50.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.411+0000 7f9ebfda6740 10 client.19776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:50.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.411+0000 7f9ebfda6740 20 client.19776 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:50.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.411+0000 7f9ebfda6740 1 --2- 172.21.15.43:0/3467418882 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559825df6710 0x559825df8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.411+0000 7f9ebfda6740 4 client.19776 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:50.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.411+0000 7f9ebfda6740 1 -- 172.21.15.43:0/3467418882 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559825ce1930 con 0x559825df6710 2024-04-21T08:47:50.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.412+0000 7f9eb6d9d640 1 --2- 172.21.15.43:0/3467418882 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559825df6710 0x559825df8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.412+0000 7f9eb6d9d640 1 --2- 172.21.15.43:0/3467418882 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559825df6710 0x559825df8b00 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd8ac1d4cb4ed62f server_cookie=944b22b3d07c87b0 in_seq=0 out_seq=0 2024-04-21T08:47:50.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.412+0000 7f9e9aff5640 10 client.19776 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:50.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.413+0000 7f9e9aff5640 1 -- 172.21.15.43:0/3467418882 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559825ce1930 con 0x559825df6710 2024-04-21T08:47:50.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.413+0000 7f9e9aff5640 10 client.19776 handle_command_reply: tid=0 2024-04-21T08:47:50.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.413+0000 7f9ebfda6740 1 client.19776 shutdown 2024-04-21T08:47:50.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.413+0000 7f9ebfda6740 1 -- 172.21.15.43:0/3467418882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825ddde80 msgr2=0x559825de3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.413+0000 7f9ebfda6740 1 --2- 172.21.15.43:0/3467418882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825ddde80 0x559825de3660 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f9eac02bc70 tx=0x7f9eac003680 comp rx=0 tx=0).stop 2024-04-21T08:47:50.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.413+0000 7f9ebfda6740 1 -- 172.21.15.43:0/3467418882 shutdown_connections 2024-04-21T08:47:50.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.413+0000 7f9ebfda6740 1 --2- 172.21.15.43:0/3467418882 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559825df6710 0x559825df8b00 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.413+0000 7f9ebfda6740 1 --2- 172.21.15.43:0/3467418882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559825de6a00 0x559825de3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.413+0000 7f9ebfda6740 1 --2- 172.21.15.43:0/3467418882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825ddde80 0x559825de3660 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.413+0000 7f9ebfda6740 1 --2- 172.21.15.43:0/3467418882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559825ddd4d0 0x559825de62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.413+0000 7f9e9aff5640 0 client.19776 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:50.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.413+0000 7f9ebfda6740 1 -- 172.21.15.43:0/3467418882 >> 172.21.15.43:0/3467418882 conn(0x559825dd7ba0 msgr2=0x559825dd98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:50.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.413+0000 7f9ebfda6740 1 -- 172.21.15.43:0/3467418882 shutdown_connections 2024-04-21T08:47:50.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.413+0000 7f9ebfda6740 1 -- 172.21.15.43:0/3467418882 wait complete. 2024-04-21T08:47:50.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.413+0000 7f9ebfda6740 20 client.19776 trim_cache size 0 max 16384 2024-04-21T08:47:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.426+0000 7f9ebfda6740 1 Processor -- start 2024-04-21T08:47:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.427+0000 7f9ebfda6740 1 -- start start 2024-04-21T08:47:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.427+0000 7f9ebfda6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559825ddd4d0 0x559825df2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.427+0000 7f9ebfda6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559825ddde80 0x559825dd9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.427+0000 7f9ebfda6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825de6a00 0x559825de5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.427+0000 7f9ebfda6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb81b1780 con 0x559825de6a00 2024-04-21T08:47:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.427+0000 7f9ebfda6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb81b32c0 con 0x559825ddde80 2024-04-21T08:47:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.427+0000 7f9ebfda6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb81b1e50 con 0x559825ddd4d0 2024-04-21T08:47:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.427+0000 7f9eb759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825de6a00 0x559825de5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.427+0000 7f9eb759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825de6a00 0x559825de5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58124/0 (socket says 172.21.15.43:58124) 2024-04-21T08:47:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.427+0000 7f9eb759e640 1 -- 172.21.15.43:0/2592213245 learned_addr learned my addr 172.21.15.43:0/2592213245 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.427+0000 7f9eb6d9d640 1 --2- 172.21.15.43:0/2592213245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559825ddd4d0 0x559825df2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.427+0000 7f9eb659c640 1 --2- 172.21.15.43:0/2592213245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559825ddde80 0x559825dd9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.427+0000 7f9eb759e640 1 -- 172.21.15.43:0/2592213245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559825ddd4d0 msgr2=0x559825df2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.427+0000 7f9eb759e640 1 --2- 172.21.15.43:0/2592213245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559825ddd4d0 0x559825df2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.427+0000 7f9eb759e640 1 -- 172.21.15.43:0/2592213245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559825ddde80 msgr2=0x559825dd9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.427+0000 7f9eb759e640 1 --2- 172.21.15.43:0/2592213245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559825ddde80 0x559825dd9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.427+0000 7f9eb759e640 1 -- 172.21.15.43:0/2592213245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559825de1bd0 con 0x559825de6a00 2024-04-21T08:47:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.427+0000 7f9eb6d9d640 1 --2- 172.21.15.43:0/2592213245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559825ddd4d0 0x559825df2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.427+0000 7f9eb759e640 1 --2- 172.21.15.43:0/2592213245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825de6a00 0x559825de5d90 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009e60 tx=0x7f9eb0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa24444fbdade2b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.427+0000 7f9e9cff9640 1 -- 172.21.15.43:0/2592213245 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0004b20 con 0x559825de6a00 2024-04-21T08:47:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.427+0000 7f9e9cff9640 1 -- 172.21.15.43:0/2592213245 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9eb0004cc0 con 0x559825de6a00 2024-04-21T08:47:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.428+0000 7f9e9cff9640 1 -- 172.21.15.43:0/2592213245 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb001dd70 con 0x559825de6a00 2024-04-21T08:47:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.428+0000 7f9ebfda6740 1 -- 172.21.15.43:0/2592213245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825de6a00 msgr2=0x559825de5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.428+0000 7f9ebfda6740 1 --2- 172.21.15.43:0/2592213245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825de6a00 0x559825de5d90 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009e60 tx=0x7f9eb0016000 comp rx=0 tx=0).stop 2024-04-21T08:47:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.428+0000 7f9ebfda6740 1 -- 172.21.15.43:0/2592213245 shutdown_connections 2024-04-21T08:47:50.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.428+0000 7f9ebfda6740 1 --2- 172.21.15.43:0/2592213245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825de6a00 0x559825de5d90 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.428+0000 7f9ebfda6740 1 --2- 172.21.15.43:0/2592213245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559825ddde80 0x559825dd9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.428+0000 7f9ebfda6740 1 --2- 172.21.15.43:0/2592213245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559825ddd4d0 0x559825df2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.428+0000 7f9ebfda6740 1 -- 172.21.15.43:0/2592213245 >> 172.21.15.43:0/2592213245 conn(0x559825dd7ba0 msgr2=0x559825dd8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:50.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.428+0000 7f9ebfda6740 1 -- 172.21.15.43:0/2592213245 shutdown_connections 2024-04-21T08:47:50.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.428+0000 7f9ebfda6740 1 -- 172.21.15.43:0/2592213245 wait complete. 2024-04-21T08:47:50.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.428+0000 7f9ebfda6740 1 -- start start 2024-04-21T08:47:50.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.428+0000 7f9ebfda6740 1 Processor -- start 2024-04-21T08:47:50.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.429+0000 7f9ebfda6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559825ddd4d0 0x559825deea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.429+0000 7f9ebfda6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559825ddde80 0x559825deef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.429+0000 7f9ebfda6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825de6a00 0x559825ddfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.429+0000 7f9ebfda6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb81b2bf0 con 0x559825de6a00 2024-04-21T08:47:50.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.429+0000 7f9ebfda6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb81b2520 con 0x559825ddd4d0 2024-04-21T08:47:50.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.429+0000 7f9ebfda6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb81af780 con 0x559825ddde80 2024-04-21T08:47:50.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.429+0000 7f9ebfda6740 10 client.0 _ll_register_callbacks cb 0x7f9eac03a340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.429+0000 7f9eb759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825de6a00 0x559825ddfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.429+0000 7f9eb759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825de6a00 0x559825ddfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58128/0 (socket says 172.21.15.43:58128) 2024-04-21T08:47:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.429+0000 7f9eb759e640 1 -- 172.21.15.43:0/3549416108 learned_addr learned my addr 172.21.15.43:0/3549416108 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.429+0000 7f9eb6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559825ddd4d0 0x559825deea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.429+0000 7f9eb659c640 1 --2- 172.21.15.43:0/3549416108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559825ddde80 0x559825deef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.429+0000 7f9eb759e640 1 -- 172.21.15.43:0/3549416108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559825ddde80 msgr2=0x559825deef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.429+0000 7f9eb759e640 1 --2- 172.21.15.43:0/3549416108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559825ddde80 0x559825deef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.429+0000 7f9eb759e640 1 -- 172.21.15.43:0/3549416108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559825ddd4d0 msgr2=0x559825deea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.429+0000 7f9eb759e640 1 --2- 172.21.15.43:0/3549416108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559825ddd4d0 0x559825deea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.429+0000 7f9eb759e640 1 -- 172.21.15.43:0/3549416108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559825de0630 con 0x559825de6a00 2024-04-21T08:47:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.429+0000 7f9eb759e640 1 --2- 172.21.15.43:0/3549416108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825de6a00 0x559825ddfe30 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f9eb001d7d0 tx=0x7f9eb0010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38cb61c67e105b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.429+0000 7f9e9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.429+0000 7f9e9aff5640 1 -- 172.21.15.43:0/3549416108 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb001d970 con 0x559825de6a00 2024-04-21T08:47:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.430+0000 7f9ebfda6740 1 -- 172.21.15.43:0/3549416108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9eb0004cc0 con 0x559825de6a00 2024-04-21T08:47:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.430+0000 7f9e9aff5640 1 -- 172.21.15.43:0/3549416108 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f9eb001db10 con 0x559825de6a00 2024-04-21T08:47:50.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.430+0000 7f9e9aff5640 1 -- 172.21.15.43:0/3549416108 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9eb001dd10 con 0x559825de6a00 2024-04-21T08:47:50.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.430+0000 7f9ebfda6740 10 client.19782 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:50.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.430+0000 7f9ebfda6740 1 -- 172.21.15.43:0/3549416108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559825e5c7e0 con 0x559825de6a00 2024-04-21T08:47:50.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.430+0000 7f9e9aff5640 1 -- 172.21.15.43:0/3549416108 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9eb0004cc0 con 0x559825de6a00 2024-04-21T08:47:50.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.430+0000 7f9e9aff5640 1 -- 172.21.15.43:0/3549416108 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9eb001b4f0 con 0x559825de6a00 2024-04-21T08:47:50.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.430+0000 7f9ebfda6740 10 client.19782 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:50.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.430+0000 7f9ebfda6740 10 client.19782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:50.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.430+0000 7f9ebfda6740 20 client.19782 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:50.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.430+0000 7f9ebfda6740 1 --2- 172.21.15.43:0/3549416108 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559825e5ce80 0x559825e5f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:50.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.430+0000 7f9ebfda6740 4 client.19782 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:50.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.430+0000 7f9ebfda6740 1 -- 172.21.15.43:0/3549416108 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559825d4fc10 con 0x559825e5ce80 2024-04-21T08:47:50.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.430+0000 7f9eb6d9d640 1 --2- 172.21.15.43:0/3549416108 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559825e5ce80 0x559825e5f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:50.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.431+0000 7f9eb6d9d640 1 --2- 172.21.15.43:0/3549416108 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559825e5ce80 0x559825e5f2a0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e10bc65e14060a1 server_cookie=c03d5d3bf875bb60 in_seq=0 out_seq=0 2024-04-21T08:47:50.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.431+0000 7f9e9aff5640 10 client.19782 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:50.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.431+0000 7f9e9aff5640 1 -- 172.21.15.43:0/3549416108 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559825d4fc10 con 0x559825e5ce80 2024-04-21T08:47:50.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.431+0000 7f9e9aff5640 10 client.19782 handle_command_reply: tid=0 2024-04-21T08:47:50.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.431+0000 7f9ebfda6740 1 client.19782 shutdown 2024-04-21T08:47:50.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.431+0000 7f9ebfda6740 1 -- 172.21.15.43:0/3549416108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825de6a00 msgr2=0x559825ddfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.431+0000 7f9ebfda6740 1 --2- 172.21.15.43:0/3549416108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825de6a00 0x559825ddfe30 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f9eb001d7d0 tx=0x7f9eb0010b30 comp rx=0 tx=0).stop 2024-04-21T08:47:50.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.431+0000 7f9ebfda6740 1 -- 172.21.15.43:0/3549416108 shutdown_connections 2024-04-21T08:47:50.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.431+0000 7f9ebfda6740 1 --2- 172.21.15.43:0/3549416108 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559825e5ce80 0x559825e5f2a0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.431+0000 7f9ebfda6740 1 --2- 172.21.15.43:0/3549416108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559825de6a00 0x559825ddfe30 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.431+0000 7f9ebfda6740 1 --2- 172.21.15.43:0/3549416108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559825ddde80 0x559825deef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.431+0000 7f9e9aff5640 0 client.19782 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:50.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.431+0000 7f9ebfda6740 1 --2- 172.21.15.43:0/3549416108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559825ddd4d0 0x559825deea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.431+0000 7f9ebfda6740 1 -- 172.21.15.43:0/3549416108 >> 172.21.15.43:0/3549416108 conn(0x559825dd7ba0 msgr2=0x559825dd9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:50.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.431+0000 7f9ebfda6740 1 -- 172.21.15.43:0/3549416108 shutdown_connections 2024-04-21T08:47:50.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.431+0000 7f9ebfda6740 1 -- 172.21.15.43:0/3549416108 wait complete. 2024-04-21T08:47:50.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.431+0000 7f9ebfda6740 20 client.19782 trim_cache size 0 max 16384 2024-04-21T08:47:50.439 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:47:50.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.432+0000 7f9ebd1b2640 1 -- 172.21.15.43:0/1592323396 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9ea00a3d50 msgr2=0x7f9ea00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.432+0000 7f9ebd1b2640 1 --2- 172.21.15.43:0/1592323396 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9ea00a3d50 0x7f9ea00a6210 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0005fd0 tx=0x7f9eb0014040 comp rx=0 tx=0).stop 2024-04-21T08:47:50.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.432+0000 7f9ebd1b2640 1 -- 172.21.15.43:0/1592323396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb8107270 msgr2=0x7f9eb81013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:50.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.432+0000 7f9ebd1b2640 1 --2- 172.21.15.43:0/1592323396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb8107270 0x7f9eb81013c0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f9eac000c00 tx=0x7f9eac0045e0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.432+0000 7f9ebd1b2640 1 -- 172.21.15.43:0/1592323396 shutdown_connections 2024-04-21T08:47:50.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.432+0000 7f9ebd1b2640 1 --2- 172.21.15.43:0/1592323396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9eb8101900 0x7f9eb81a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.432+0000 7f9ebd1b2640 1 --2- 172.21.15.43:0/1592323396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb8107270 0x7f9eb81013c0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.432+0000 7f9ebd1b2640 1 --2- 172.21.15.43:0/1592323396 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9ea00a3d50 0x7f9ea00a6210 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.432+0000 7f9ebd1b2640 1 --2- 172.21.15.43:0/1592323396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eb806a840 0x7f9eb8100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:50.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.432+0000 7f9ebd1b2640 1 -- 172.21.15.43:0/1592323396 >> 172.21.15.43:0/1592323396 conn(0x7f9eb8077eb0 msgr2=0x7f9eb8108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:50.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.432+0000 7f9ebd1b2640 1 -- 172.21.15.43:0/1592323396 shutdown_connections 2024-04-21T08:47:50.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:50.432+0000 7f9ebd1b2640 1 -- 172.21.15.43:0/1592323396 wait complete. 2024-04-21T08:47:50.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:50 smithi175 ceph-mon[31920]: pgmap v275: 129 pgs: 129 active+clean; 5.6 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:47:50.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:50 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:47:50.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:50 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:50.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:50 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:47:50.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:50 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:50.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:50 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:47:50.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:50 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2941763346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:50 smithi112 ceph-mon[31085]: pgmap v275: 129 pgs: 129 active+clean; 5.6 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:47:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:50 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:47:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:50 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:50 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:47:50.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:50 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:50.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:50 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:47:50.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:50 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2941763346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:51.443 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:47:51.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:47:51.444 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:47:51.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.563+0000 7efd628c1640 1 -- 172.21.15.43:0/3138087537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd5c107270 msgr2=0x7efd5c10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:51.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.563+0000 7efd628c1640 1 --2- 172.21.15.43:0/3138087537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd5c107270 0x7efd5c10ee40 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7efd54009ca0 tx=0x7efd5402c100 comp rx=0 tx=0).stop 2024-04-21T08:47:51.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.563+0000 7efd628c1640 1 -- 172.21.15.43:0/3138087537 shutdown_connections 2024-04-21T08:47:51.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.563+0000 7efd628c1640 1 --2- 172.21.15.43:0/3138087537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd5c107270 0x7efd5c10ee40 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.563+0000 7efd628c1640 1 --2- 172.21.15.43:0/3138087537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd5c06b1f0 0x7efd5c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.563+0000 7efd628c1640 1 --2- 172.21.15.43:0/3138087537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd5c06a840 0x7efd5c06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.563+0000 7efd628c1640 1 -- 172.21.15.43:0/3138087537 >> 172.21.15.43:0/3138087537 conn(0x7efd5c077eb0 msgr2=0x7efd5c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:51.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.563+0000 7efd628c1640 1 -- 172.21.15.43:0/3138087537 shutdown_connections 2024-04-21T08:47:51.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.563+0000 7efd628c1640 1 -- 172.21.15.43:0/3138087537 wait complete. 2024-04-21T08:47:51.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.564+0000 7efd628c1640 1 Processor -- start 2024-04-21T08:47:51.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.564+0000 7efd628c1640 1 -- start start 2024-04-21T08:47:51.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.564+0000 7efd628c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd5c06a840 0x7efd5c100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.564+0000 7efd628c1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd5c107270 0x7efd5c1013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.564+0000 7efd628c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd5c101900 0x7efd5c1a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.564+0000 7efd628c1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c1121b0 con 0x7efd5c06a840 2024-04-21T08:47:51.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.564+0000 7efd628c1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c112030 con 0x7efd5c101900 2024-04-21T08:47:51.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.564+0000 7efd628c1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c112330 con 0x7efd5c107270 2024-04-21T08:47:51.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.564+0000 7efd5bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd5c06a840 0x7efd5c100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.565+0000 7efd5bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd5c06a840 0x7efd5c100e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58136/0 (socket says 172.21.15.43:58136) 2024-04-21T08:47:51.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.565+0000 7efd5bfff640 1 -- 172.21.15.43:0/3062871484 learned_addr learned my addr 172.21.15.43:0/3062871484 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:51.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.565+0000 7efd5b7fe640 1 --2- 172.21.15.43:0/3062871484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd5c107270 0x7efd5c1013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.565+0000 7efd60e5e640 1 --2- 172.21.15.43:0/3062871484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd5c101900 0x7efd5c1a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.565+0000 7efd5bfff640 1 -- 172.21.15.43:0/3062871484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd5c107270 msgr2=0x7efd5c1013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:51.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.565+0000 7efd5bfff640 1 --2- 172.21.15.43:0/3062871484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd5c107270 0x7efd5c1013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.565+0000 7efd5bfff640 1 -- 172.21.15.43:0/3062871484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd5c101900 msgr2=0x7efd5c1a67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:51.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.565+0000 7efd5bfff640 1 --2- 172.21.15.43:0/3062871484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd5c101900 0x7efd5c1a67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.565+0000 7efd5bfff640 1 -- 172.21.15.43:0/3062871484 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5c1a6ec0 con 0x7efd5c06a840 2024-04-21T08:47:51.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.565+0000 7efd5bfff640 1 --2- 172.21.15.43:0/3062871484 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd5c06a840 0x7efd5c100e80 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7efd4c00c920 tx=0x7efd4c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:51.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.565+0000 7efd597fa640 1 -- 172.21.15.43:0/3062871484 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c007c70 con 0x7efd5c06a840 2024-04-21T08:47:51.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.565+0000 7efd628c1640 1 -- 172.21.15.43:0/3062871484 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd5c1a71b0 con 0x7efd5c06a840 2024-04-21T08:47:51.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.565+0000 7efd597fa640 1 -- 172.21.15.43:0/3062871484 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd4c007e10 con 0x7efd5c06a840 2024-04-21T08:47:51.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.565+0000 7efd597fa640 1 -- 172.21.15.43:0/3062871484 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c018820 con 0x7efd5c06a840 2024-04-21T08:47:51.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.565+0000 7efd628c1640 1 -- 172.21.15.43:0/3062871484 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd5c1a7750 con 0x7efd5c06a840 2024-04-21T08:47:51.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.568+0000 7efd597fa640 1 -- 172.21.15.43:0/3062871484 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efd4c00f450 con 0x7efd5c06a840 2024-04-21T08:47:51.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.568+0000 7efd65592740 1 Processor -- start 2024-04-21T08:47:51.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.568+0000 7efd65592740 1 -- start start 2024-04-21T08:47:51.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.568+0000 7efd65592740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a91688d8e0 0x55a91688dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.568+0000 7efd65592740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91688e290 0x55a916896840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.568+0000 7efd65592740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a916896e10 0x55a916899260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.568+0000 7efd65592740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a9167f1f00 con 0x55a916896e10 2024-04-21T08:47:51.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.568+0000 7efd65592740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a91678f590 con 0x55a91688d8e0 2024-04-21T08:47:51.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.568+0000 7efd65592740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a9167edec0 con 0x55a91688e290 2024-04-21T08:47:51.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.568+0000 7efd5b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a916896e10 0x55a916899260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.568+0000 7efd60e5e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a91688d8e0 0x55a91688dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.568+0000 7efd5b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a916896e10 0x55a916899260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58150/0 (socket says 172.21.15.43:58150) 2024-04-21T08:47:51.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.568+0000 7efd5b7fe640 1 -- 172.21.15.43:0/1800851206 learned_addr learned my addr 172.21.15.43:0/1800851206 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:51.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.568+0000 7efd597fa640 1 --2- 172.21.15.43:0/3062871484 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7efd440a3d50 0x7efd440a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.568+0000 7efd5bfff640 1 --2- 172.21.15.43:0/1800851206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91688e290 0x55a916896840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.569+0000 7efd60e5e640 1 -- 172.21.15.43:0/1800851206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91688e290 msgr2=0x55a916896840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:51.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.569+0000 7efd60e5e640 1 --2- 172.21.15.43:0/1800851206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91688e290 0x55a916896840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.569+0000 7efd60e5e640 1 -- 172.21.15.43:0/1800851206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a916896e10 msgr2=0x55a916899260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:51.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.569+0000 7efd60e5e640 1 --2- 172.21.15.43:0/1800851206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a916896e10 0x55a916899260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.569+0000 7efd60e5e640 1 -- 172.21.15.43:0/1800851206 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9165cc7e0 con 0x55a91688d8e0 2024-04-21T08:47:51.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.569+0000 7efd5b7fe640 1 --2- 172.21.15.43:0/1800851206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a916896e10 0x55a916899260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:51.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.569+0000 7efd597fa640 1 -- 172.21.15.43:0/3062871484 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7efd4c0d0dd0 con 0x7efd5c06a840 2024-04-21T08:47:51.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.569+0000 7efd60e5e640 1 --2- 172.21.15.43:0/3062871484 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7efd440a3d50 0x7efd440a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.569+0000 7efd60e5e640 1 --2- 172.21.15.43:0/1800851206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a91688d8e0 0x55a91688dcc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efd5402c710 tx=0x7efd54009a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d949e6e2a08655fe server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:51.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.569+0000 7efd60e5e640 1 --2- 172.21.15.43:0/3062871484 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7efd440a3d50 0x7efd440a6210 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efd54006010 tx=0x7efd54044000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:51.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.569+0000 7efd427fc640 1 -- 172.21.15.43:0/1800851206 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54039040 con 0x55a91688d8e0 2024-04-21T08:47:51.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.569+0000 7efd427fc640 1 -- 172.21.15.43:0/1800851206 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd54003d30 con 0x55a91688d8e0 2024-04-21T08:47:51.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.569+0000 7efd427fc640 1 -- 172.21.15.43:0/1800851206 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54034780 con 0x55a91688d8e0 2024-04-21T08:47:51.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.570+0000 7efd65592740 1 -- 172.21.15.43:0/1800851206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a91688d8e0 msgr2=0x55a91688dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:51.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.570+0000 7efd65592740 1 --2- 172.21.15.43:0/1800851206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a91688d8e0 0x55a91688dcc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efd5402c710 tx=0x7efd54009a00 comp rx=0 tx=0).stop 2024-04-21T08:47:51.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.570+0000 7efd65592740 1 -- 172.21.15.43:0/1800851206 shutdown_connections 2024-04-21T08:47:51.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.570+0000 7efd65592740 1 --2- 172.21.15.43:0/1800851206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a916896e10 0x55a916899260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.570+0000 7efd65592740 1 --2- 172.21.15.43:0/1800851206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91688e290 0x55a916896840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.570+0000 7efd65592740 1 --2- 172.21.15.43:0/1800851206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a91688d8e0 0x55a91688dcc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.570+0000 7efd65592740 1 -- 172.21.15.43:0/1800851206 >> 172.21.15.43:0/1800851206 conn(0x55a916887fb0 msgr2=0x55a916888390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:51.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.570+0000 7efd65592740 1 -- 172.21.15.43:0/1800851206 shutdown_connections 2024-04-21T08:47:51.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.570+0000 7efd65592740 1 -- 172.21.15.43:0/1800851206 wait complete. 2024-04-21T08:47:51.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.570+0000 7efd65592740 1 -- start start 2024-04-21T08:47:51.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.571+0000 7efd65592740 1 Processor -- start 2024-04-21T08:47:51.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.571+0000 7efd65592740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a91688d8e0 0x55a916893e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.571+0000 7efd65592740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91688e290 0x55a916894380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.571+0000 7efd65592740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a916896e10 0x55a9168948c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.571+0000 7efd65592740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a916899d80 con 0x55a916896e10 2024-04-21T08:47:51.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.571+0000 7efd65592740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a9167ab580 con 0x55a91688d8e0 2024-04-21T08:47:51.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.571+0000 7efd65592740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a916899f00 con 0x55a91688e290 2024-04-21T08:47:51.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.571+0000 7efd65592740 10 client.0 _ll_register_callbacks cb 0x55a9167b1650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:51.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.571+0000 7efd60e5e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a916896e10 0x55a9168948c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.571+0000 7efd5bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a91688d8e0 0x55a916893e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.571+0000 7efd5bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a91688d8e0 0x55a916893e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:47280/0 (socket says 172.21.15.43:47280) 2024-04-21T08:47:51.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.571+0000 7efd5bfff640 1 -- 172.21.15.43:0/2531649009 learned_addr learned my addr 172.21.15.43:0/2531649009 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:51.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.571+0000 7efd60e5e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a916896e10 0x55a9168948c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58164/0 (socket says 172.21.15.43:58164) 2024-04-21T08:47:51.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.571+0000 7efd5b7fe640 1 --2- 172.21.15.43:0/2531649009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91688e290 0x55a916894380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.571+0000 7efd5bfff640 1 -- 172.21.15.43:0/2531649009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91688e290 msgr2=0x55a916894380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:51.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.571+0000 7efd5bfff640 1 --2- 172.21.15.43:0/2531649009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91688e290 0x55a916894380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.571+0000 7efd5bfff640 1 -- 172.21.15.43:0/2531649009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a916896e10 msgr2=0x55a9168948c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:51.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.571+0000 7efd5bfff640 1 --2- 172.21.15.43:0/2531649009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a916896e10 0x55a9168948c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.571+0000 7efd5bfff640 1 -- 172.21.15.43:0/2531649009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a91662cf70 con 0x55a91688d8e0 2024-04-21T08:47:51.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.572+0000 7efd5bfff640 1 --2- 172.21.15.43:0/2531649009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a91688d8e0 0x55a916893e40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efd4c098db0 tx=0x7efd4c09c550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=689e6c3ccf2ff28b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:51.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.572+0000 7efd3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-21T08:47:51.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.572+0000 7efd3b7fe640 1 -- 172.21.15.43:0/2531649009 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c0a0d00 con 0x55a91688d8e0 2024-04-21T08:47:51.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.572+0000 7efd65592740 1 -- 172.21.15.43:0/2531649009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd54003d30 con 0x55a91688d8e0 2024-04-21T08:47:51.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.572+0000 7efd3b7fe640 1 -- 172.21.15.43:0/2531649009 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7efd4c00f740 con 0x55a91688d8e0 2024-04-21T08:47:51.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.572+0000 7efd3b7fe640 1 -- 172.21.15.43:0/2531649009 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efd4c0a55c0 con 0x55a91688d8e0 2024-04-21T08:47:51.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.572+0000 7efd65592740 10 client.28550 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:51.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.572+0000 7efd65592740 1 -- 172.21.15.43:0/2531649009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a9164c4170 con 0x55a91688d8e0 2024-04-21T08:47:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.572+0000 7efd3b7fe640 1 -- 172.21.15.43:0/2531649009 <== mon.2 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd54003d30 con 0x55a91688d8e0 2024-04-21T08:47:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.572+0000 7efd3b7fe640 1 -- 172.21.15.43:0/2531649009 <== mon.2 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7efd4c0a5130 con 0x55a91688d8e0 2024-04-21T08:47:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.572+0000 7efd65592740 10 client.28550 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.572+0000 7efd65592740 10 client.28550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.572+0000 7efd65592740 20 client.28550 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.572+0000 7efd65592740 1 --2- 172.21.15.43:0/2531649009 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a9168a72e0 0x55a9168a96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.572+0000 7efd65592740 4 client.28550 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.572+0000 7efd65592740 1 -- 172.21.15.43:0/2531649009 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9167ff920 con 0x55a9168a72e0 2024-04-21T08:47:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.573+0000 7efd5b7fe640 1 --2- 172.21.15.43:0/2531649009 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a9168a72e0 0x55a9168a96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.573+0000 7efd3b7fe640 10 client.28550 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.573+0000 7efd5b7fe640 1 --2- 172.21.15.43:0/2531649009 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a9168a72e0 0x55a9168a96f0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6090f3c416d24e7 server_cookie=5e963ae1b79012f3 in_seq=0 out_seq=0 2024-04-21T08:47:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.574+0000 7efd3b7fe640 1 -- 172.21.15.43:0/2531649009 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a9167ff920 con 0x55a9168a72e0 2024-04-21T08:47:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.574+0000 7efd3b7fe640 10 client.28550 handle_command_reply: tid=0 2024-04-21T08:47:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.574+0000 7efd65592740 1 client.28550 shutdown 2024-04-21T08:47:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.574+0000 7efd65592740 1 -- 172.21.15.43:0/2531649009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a91688d8e0 msgr2=0x55a916893e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.574+0000 7efd65592740 1 --2- 172.21.15.43:0/2531649009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a91688d8e0 0x55a916893e40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efd4c098db0 tx=0x7efd4c09c550 comp rx=0 tx=0).stop 2024-04-21T08:47:51.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.574+0000 7efd65592740 1 -- 172.21.15.43:0/2531649009 shutdown_connections 2024-04-21T08:47:51.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.574+0000 7efd65592740 1 --2- 172.21.15.43:0/2531649009 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a9168a72e0 0x55a9168a96f0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.574+0000 7efd65592740 1 --2- 172.21.15.43:0/2531649009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a916896e10 0x55a9168948c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.574+0000 7efd65592740 1 --2- 172.21.15.43:0/2531649009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91688e290 0x55a916894380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.574+0000 7efd3b7fe640 0 client.28550 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:51.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.574+0000 7efd65592740 1 --2- 172.21.15.43:0/2531649009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a91688d8e0 0x55a916893e40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.574+0000 7efd65592740 1 -- 172.21.15.43:0/2531649009 >> 172.21.15.43:0/2531649009 conn(0x55a916887fb0 msgr2=0x55a916889d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:51.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.574+0000 7efd65592740 1 -- 172.21.15.43:0/2531649009 shutdown_connections 2024-04-21T08:47:51.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.574+0000 7efd65592740 1 -- 172.21.15.43:0/2531649009 wait complete. 2024-04-21T08:47:51.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.574+0000 7efd65592740 20 client.28550 trim_cache size 0 max 16384 2024-04-21T08:47:51.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.592+0000 7efd65592740 1 Processor -- start 2024-04-21T08:47:51.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.592+0000 7efd65592740 1 -- start start 2024-04-21T08:47:51.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.592+0000 7efd65592740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a91688d8e0 0x55a916896570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.592+0000 7efd65592740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91688e290 0x55a9168a2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.592+0000 7efd65592740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a916896e10 0x55a916889af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.592+0000 7efd65592740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c1b1500 con 0x55a91688e290 2024-04-21T08:47:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.592+0000 7efd65592740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c1b3040 con 0x55a91688d8e0 2024-04-21T08:47:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.592+0000 7efd65592740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c1b1bd0 con 0x55a916896e10 2024-04-21T08:47:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.592+0000 7efd5b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91688e290 0x55a9168a2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.592+0000 7efd5b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91688e290 0x55a9168a2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58170/0 (socket says 172.21.15.43:58170) 2024-04-21T08:47:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.592+0000 7efd5b7fe640 1 -- 172.21.15.43:0/1232931253 learned_addr learned my addr 172.21.15.43:0/1232931253 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.593+0000 7efd5bfff640 1 --2- 172.21.15.43:0/1232931253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a91688d8e0 0x55a916896570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.593+0000 7efd60e5e640 1 --2- 172.21.15.43:0/1232931253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a916896e10 0x55a916889af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.593+0000 7efd5b7fe640 1 -- 172.21.15.43:0/1232931253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a916896e10 msgr2=0x55a916889af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.593+0000 7efd5b7fe640 1 --2- 172.21.15.43:0/1232931253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a916896e10 0x55a916889af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.593+0000 7efd5b7fe640 1 -- 172.21.15.43:0/1232931253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a91688d8e0 msgr2=0x55a916896570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.593+0000 7efd5b7fe640 1 --2- 172.21.15.43:0/1232931253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a91688d8e0 0x55a916896570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.593+0000 7efd5b7fe640 1 -- 172.21.15.43:0/1232931253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a916892390 con 0x55a91688e290 2024-04-21T08:47:51.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.593+0000 7efd5b7fe640 1 --2- 172.21.15.43:0/1232931253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91688e290 0x55a9168a2c40 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x55a9168936a0 tx=0x7efd5000dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6f92476e1caa21 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:51.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.593+0000 7efd41ffb640 1 -- 172.21.15.43:0/1232931253 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd50003d00 con 0x55a91688e290 2024-04-21T08:47:51.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.593+0000 7efd41ffb640 1 -- 172.21.15.43:0/1232931253 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd5001d8e0 con 0x55a91688e290 2024-04-21T08:47:51.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.593+0000 7efd41ffb640 1 -- 172.21.15.43:0/1232931253 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd50011d40 con 0x55a91688e290 2024-04-21T08:47:51.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.594+0000 7efd65592740 1 -- 172.21.15.43:0/1232931253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91688e290 msgr2=0x55a9168a2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:51.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.594+0000 7efd65592740 1 --2- 172.21.15.43:0/1232931253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91688e290 0x55a9168a2c40 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x55a9168936a0 tx=0x7efd5000dda0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.594+0000 7efd65592740 1 -- 172.21.15.43:0/1232931253 shutdown_connections 2024-04-21T08:47:51.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.594+0000 7efd65592740 1 --2- 172.21.15.43:0/1232931253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a916896e10 0x55a916889af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.594+0000 7efd65592740 1 --2- 172.21.15.43:0/1232931253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91688e290 0x55a9168a2c40 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.594+0000 7efd65592740 1 --2- 172.21.15.43:0/1232931253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a91688d8e0 0x55a916896570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.594+0000 7efd65592740 1 -- 172.21.15.43:0/1232931253 >> 172.21.15.43:0/1232931253 conn(0x55a916887fb0 msgr2=0x55a9168893e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:51.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.594+0000 7efd65592740 1 -- 172.21.15.43:0/1232931253 shutdown_connections 2024-04-21T08:47:51.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.594+0000 7efd65592740 1 -- 172.21.15.43:0/1232931253 wait complete. 2024-04-21T08:47:51.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.594+0000 7efd65592740 1 -- start start 2024-04-21T08:47:51.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.594+0000 7efd65592740 1 Processor -- start 2024-04-21T08:47:51.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.594+0000 7efd65592740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91688d8e0 0x55a91689f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.594+0000 7efd65592740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a91688e290 0x55a91689f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.594+0000 7efd65592740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a916896e10 0x55a91689c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.594+0000 7efd65592740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c1b2970 con 0x55a91688d8e0 2024-04-21T08:47:51.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.594+0000 7efd65592740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c1b22a0 con 0x55a91688e290 2024-04-21T08:47:51.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.594+0000 7efd65592740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c1af500 con 0x55a916896e10 2024-04-21T08:47:51.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.595+0000 7efd65592740 10 client.0 _ll_register_callbacks cb 0x7efd4c00c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:51.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.595+0000 7efd5bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91688d8e0 0x55a91689f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.595+0000 7efd5bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91688d8e0 0x55a91689f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58180/0 (socket says 172.21.15.43:58180) 2024-04-21T08:47:51.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.595+0000 7efd5bfff640 1 -- 172.21.15.43:0/4057379190 learned_addr learned my addr 172.21.15.43:0/4057379190 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:51.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.595+0000 7efd5b7fe640 1 --2- 172.21.15.43:0/4057379190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a91688e290 0x55a91689f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.595+0000 7efd60e5e640 1 --2- 172.21.15.43:0/4057379190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a916896e10 0x55a91689c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.595+0000 7efd5bfff640 1 -- 172.21.15.43:0/4057379190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a916896e10 msgr2=0x55a91689c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:51.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.595+0000 7efd5bfff640 1 --2- 172.21.15.43:0/4057379190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a916896e10 0x55a91689c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.595+0000 7efd5bfff640 1 -- 172.21.15.43:0/4057379190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a91688e290 msgr2=0x55a91689f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:51.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.595+0000 7efd5bfff640 1 --2- 172.21.15.43:0/4057379190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a91688e290 0x55a91689f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.595+0000 7efd5bfff640 1 -- 172.21.15.43:0/4057379190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a91689c810 con 0x55a91688d8e0 2024-04-21T08:47:51.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.595+0000 7efd60e5e640 1 --2- 172.21.15.43:0/4057379190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a916896e10 0x55a91689c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:51.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.595+0000 7efd5bfff640 1 --2- 172.21.15.43:0/4057379190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91688d8e0 0x55a91689f150 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7efd4c000c00 tx=0x7efd4c00fca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28547922e26899c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:51.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.595+0000 7efd3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:51.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.595+0000 7efd3b7fe640 1 -- 172.21.15.43:0/4057379190 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c0b8020 con 0x55a91688d8e0 2024-04-21T08:47:51.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.595+0000 7efd3b7fe640 1 -- 172.21.15.43:0/4057379190 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7efd4c0bb7d0 con 0x55a91688d8e0 2024-04-21T08:47:51.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.595+0000 7efd65592740 1 -- 172.21.15.43:0/4057379190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd5001d8e0 con 0x55a91688d8e0 2024-04-21T08:47:51.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.595+0000 7efd3b7fe640 1 -- 172.21.15.43:0/4057379190 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efd4c066050 con 0x55a91688d8e0 2024-04-21T08:47:51.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.595+0000 7efd65592740 10 client.19800 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:51.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.595+0000 7efd65592740 1 -- 172.21.15.43:0/4057379190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a91690e260 con 0x55a91688d8e0 2024-04-21T08:47:51.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.596+0000 7efd3b7fe640 1 -- 172.21.15.43:0/4057379190 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd5001d8e0 con 0x55a91688d8e0 2024-04-21T08:47:51.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.596+0000 7efd3b7fe640 1 -- 172.21.15.43:0/4057379190 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7efd4c067240 con 0x55a91688d8e0 2024-04-21T08:47:51.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.596+0000 7efd65592740 10 client.19800 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:51.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.596+0000 7efd65592740 10 client.19800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:51.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.596+0000 7efd65592740 20 client.19800 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:51.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.596+0000 7efd65592740 1 --2- 172.21.15.43:0/4057379190 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a91690e920 0x55a916910d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.596+0000 7efd65592740 4 client.19800 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:47:51.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.596+0000 7efd65592740 1 -- 172.21.15.43:0/4057379190 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a9167f9790 con 0x55a91690e920 2024-04-21T08:47:51.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.596+0000 7efd5b7fe640 1 --2- 172.21.15.43:0/4057379190 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a91690e920 0x55a916910d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.596+0000 7efd3b7fe640 10 client.19800 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:51.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.596+0000 7efd5b7fe640 1 --2- 172.21.15.43:0/4057379190 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a91690e920 0x55a916910d40 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b8cbd439b88effb server_cookie=bd5d81ec1686634c in_seq=0 out_seq=0 2024-04-21T08:47:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.597+0000 7efd3b7fe640 1 -- 172.21.15.43:0/4057379190 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a9167f9790 con 0x55a91690e920 2024-04-21T08:47:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.597+0000 7efd3b7fe640 10 client.19800 handle_command_reply: tid=0 2024-04-21T08:47:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.597+0000 7efd65592740 1 client.19800 shutdown 2024-04-21T08:47:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.598+0000 7efd65592740 1 -- 172.21.15.43:0/4057379190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91688d8e0 msgr2=0x55a91689f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.598+0000 7efd65592740 1 --2- 172.21.15.43:0/4057379190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91688d8e0 0x55a91689f150 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7efd4c000c00 tx=0x7efd4c00fca0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.598+0000 7efd65592740 1 -- 172.21.15.43:0/4057379190 shutdown_connections 2024-04-21T08:47:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.598+0000 7efd65592740 1 --2- 172.21.15.43:0/4057379190 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a91690e920 0x55a916910d40 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.598+0000 7efd65592740 1 --2- 172.21.15.43:0/4057379190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a916896e10 0x55a91689c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.598+0000 7efd65592740 1 --2- 172.21.15.43:0/4057379190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a91688e290 0x55a91689f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.598+0000 7efd3b7fe640 0 client.19800 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.598+0000 7efd65592740 1 --2- 172.21.15.43:0/4057379190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a91688d8e0 0x55a91689f150 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.598+0000 7efd65592740 1 -- 172.21.15.43:0/4057379190 >> 172.21.15.43:0/4057379190 conn(0x55a916887fb0 msgr2=0x55a916887010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.598+0000 7efd65592740 1 -- 172.21.15.43:0/4057379190 shutdown_connections 2024-04-21T08:47:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.598+0000 7efd65592740 1 -- 172.21.15.43:0/4057379190 wait complete. 2024-04-21T08:47:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.598+0000 7efd65592740 20 client.19800 trim_cache size 0 max 16384 2024-04-21T08:47:51.605 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:47:51.605 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:47:51.605 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "48d06675-5b3a-46c1-9208-e45471341e68", 2024-04-21T08:47:51.605 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:47:51.605 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:47:51.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.598+0000 7efd628c1640 1 -- 172.21.15.43:0/3062871484 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7efd440a3d50 msgr2=0x7efd440a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:51.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.598+0000 7efd628c1640 1 --2- 172.21.15.43:0/3062871484 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7efd440a3d50 0x7efd440a6210 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efd54006010 tx=0x7efd54044000 comp rx=0 tx=0).stop 2024-04-21T08:47:51.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.598+0000 7efd628c1640 1 -- 172.21.15.43:0/3062871484 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd5c06a840 msgr2=0x7efd5c100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:51.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.598+0000 7efd628c1640 1 --2- 172.21.15.43:0/3062871484 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd5c06a840 0x7efd5c100e80 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7efd4c00c920 tx=0x7efd4c00cde0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.599+0000 7efd628c1640 1 -- 172.21.15.43:0/3062871484 shutdown_connections 2024-04-21T08:47:51.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.599+0000 7efd628c1640 1 --2- 172.21.15.43:0/3062871484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd5c101900 0x7efd5c1a67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.599+0000 7efd628c1640 1 --2- 172.21.15.43:0/3062871484 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7efd440a3d50 0x7efd440a6210 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.599+0000 7efd628c1640 1 --2- 172.21.15.43:0/3062871484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd5c107270 0x7efd5c1013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.599+0000 7efd628c1640 1 --2- 172.21.15.43:0/3062871484 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd5c06a840 0x7efd5c100e80 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.599+0000 7efd628c1640 1 -- 172.21.15.43:0/3062871484 >> 172.21.15.43:0/3062871484 conn(0x7efd5c077eb0 msgr2=0x7efd5c0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:51.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.599+0000 7efd628c1640 1 -- 172.21.15.43:0/3062871484 shutdown_connections 2024-04-21T08:47:51.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.599+0000 7efd628c1640 1 -- 172.21.15.43:0/3062871484 wait complete. 2024-04-21T08:47:51.611 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:47:51.611 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:47:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.715+0000 7f45c152b640 1 -- 172.21.15.43:0/3984804294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45bc102b50 msgr2=0x7f45bc106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.715+0000 7f45c152b640 1 --2- 172.21.15.43:0/3984804294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45bc102b50 0x7f45bc106b90 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009900 tx=0x7f45a402bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.715+0000 7f45c152b640 1 -- 172.21.15.43:0/3984804294 shutdown_connections 2024-04-21T08:47:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.715+0000 7f45c152b640 1 --2- 172.21.15.43:0/3984804294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45bc107270 0x7f45bc10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.715+0000 7f45c152b640 1 --2- 172.21.15.43:0/3984804294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45bc102b50 0x7f45bc106b90 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.715+0000 7f45c152b640 1 --2- 172.21.15.43:0/3984804294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45bc1021a0 0x7f45bc102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.715+0000 7f45c152b640 1 -- 172.21.15.43:0/3984804294 >> 172.21.15.43:0/3984804294 conn(0x7f45bc0fda40 msgr2=0x7f45bc0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.715+0000 7f45c152b640 1 -- 172.21.15.43:0/3984804294 shutdown_connections 2024-04-21T08:47:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.715+0000 7f45c152b640 1 -- 172.21.15.43:0/3984804294 wait complete. 2024-04-21T08:47:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.716+0000 7f45c152b640 1 Processor -- start 2024-04-21T08:47:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.716+0000 7f45c152b640 1 -- start start 2024-04-21T08:47:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.716+0000 7f45c152b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45bc1021a0 0x7f45bc074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.716+0000 7f45c152b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45bc107270 0x7f45bc074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.716+0000 7f45c152b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45bc074cd0 0x7f45bc1a6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.716+0000 7f45c152b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1121b0 con 0x7f45bc074cd0 2024-04-21T08:47:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.716+0000 7f45c152b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc112030 con 0x7f45bc107270 2024-04-21T08:47:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.716+0000 7f45c152b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc112330 con 0x7f45bc1021a0 2024-04-21T08:47:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.716+0000 7f45bb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45bc074cd0 0x7f45bc1a6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.716+0000 7f45bb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45bc074cd0 0x7f45bc1a6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58194/0 (socket says 172.21.15.43:58194) 2024-04-21T08:47:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.716+0000 7f45bb7fe640 1 -- 172.21.15.43:0/397726743 learned_addr learned my addr 172.21.15.43:0/397726743 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.716+0000 7f45b3fff640 1 --2- 172.21.15.43:0/397726743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45bc107270 0x7f45bc074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.716+0000 7f45baffd640 1 --2- 172.21.15.43:0/397726743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45bc1021a0 0x7f45bc074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.716+0000 7f45bb7fe640 1 -- 172.21.15.43:0/397726743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45bc1021a0 msgr2=0x7f45bc074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.716+0000 7f45bb7fe640 1 --2- 172.21.15.43:0/397726743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45bc1021a0 0x7f45bc074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.717+0000 7f45bb7fe640 1 -- 172.21.15.43:0/397726743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45bc107270 msgr2=0x7f45bc074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.717+0000 7f45bb7fe640 1 --2- 172.21.15.43:0/397726743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45bc107270 0x7f45bc074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.717+0000 7f45bb7fe640 1 -- 172.21.15.43:0/397726743 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc1a70f0 con 0x7f45bc074cd0 2024-04-21T08:47:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.717+0000 7f45bb7fe640 1 --2- 172.21.15.43:0/397726743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45bc074cd0 0x7f45bc1a6a50 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f45b400eaa0 tx=0x7f45b400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:51.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.717+0000 7f45b8ff9640 1 -- 172.21.15.43:0/397726743 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b400cdb0 con 0x7f45bc074cd0 2024-04-21T08:47:51.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.717+0000 7f45b8ff9640 1 -- 172.21.15.43:0/397726743 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45b4005020 con 0x7f45bc074cd0 2024-04-21T08:47:51.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.717+0000 7f45c152b640 1 -- 172.21.15.43:0/397726743 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45bc1a73e0 con 0x7f45bc074cd0 2024-04-21T08:47:51.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.717+0000 7f45b8ff9640 1 -- 172.21.15.43:0/397726743 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b40116d0 con 0x7f45bc074cd0 2024-04-21T08:47:51.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.717+0000 7f45c152b640 1 -- 172.21.15.43:0/397726743 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45bc1a78b0 con 0x7f45bc074cd0 2024-04-21T08:47:51.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.720+0000 7f45b8ff9640 1 -- 172.21.15.43:0/397726743 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f45b40051d0 con 0x7f45bc074cd0 2024-04-21T08:47:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.720+0000 7f45c41fc740 1 Processor -- start 2024-04-21T08:47:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.720+0000 7f45c41fc740 1 -- start start 2024-04-21T08:47:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.720+0000 7f45c41fc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74bea580 0x560b74bea960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.720+0000 7f45c41fc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b74beaea0 0x560b74bf3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.720+0000 7f45c41fc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b74bf3a80 0x560b74bf5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.720+0000 7f45c41fc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560b74b4f1e0 con 0x560b74bea580 2024-04-21T08:47:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.720+0000 7f45c41fc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560b74aec390 con 0x560b74beaea0 2024-04-21T08:47:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.720+0000 7f45c41fc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560b74b4b1a0 con 0x560b74bf3a80 2024-04-21T08:47:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.720+0000 7f45b3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74bea580 0x560b74bea960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.721+0000 7f45b3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74bea580 0x560b74bea960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58198/0 (socket says 172.21.15.43:58198) 2024-04-21T08:47:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.721+0000 7f45b3fff640 1 -- 172.21.15.43:0/1130648037 learned_addr learned my addr 172.21.15.43:0/1130648037 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.721+0000 7f45baffd640 1 --2- 172.21.15.43:0/1130648037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b74beaea0 0x560b74bf3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.721+0000 7f45b8ff9640 1 --2- 172.21.15.43:0/397726743 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f45a80a3d20 0x7f45a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.721+0000 7f45b3fff640 1 --2- 172.21.15.43:0/1130648037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b74bf3a80 0x560b74bf5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.721+0000 7f45b3fff640 1 -- 172.21.15.43:0/1130648037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b74bf3a80 msgr2=0x560b74bf5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.721+0000 7f45b3fff640 1 --2- 172.21.15.43:0/1130648037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b74bf3a80 0x560b74bf5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.721+0000 7f45b3fff640 1 -- 172.21.15.43:0/1130648037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b74beaea0 msgr2=0x560b74bf3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.721+0000 7f45b3fff640 1 --2- 172.21.15.43:0/1130648037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b74beaea0 0x560b74bf3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.721+0000 7f45bb7fe640 1 --2- 172.21.15.43:0/397726743 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f45a80a3d20 0x7f45a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.721+0000 7f45b3fff640 1 -- 172.21.15.43:0/1130648037 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b7492b7d0 con 0x560b74bea580 2024-04-21T08:47:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.721+0000 7f45b8ff9640 1 -- 172.21.15.43:0/397726743 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f45b40d0dc0 con 0x7f45bc074cd0 2024-04-21T08:47:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.721+0000 7f45b3fff640 1 --2- 172.21.15.43:0/1130648037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74bea580 0x560b74bea960 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f45bc06b630 tx=0x7f45a4005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9ae9b25e60b5cd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.721+0000 7f45b17fa640 1 -- 172.21.15.43:0/1130648037 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a4047020 con 0x560b74bea580 2024-04-21T08:47:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.721+0000 7f45b17fa640 1 -- 172.21.15.43:0/1130648037 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45a402cc90 con 0x560b74bea580 2024-04-21T08:47:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.721+0000 7f45b17fa640 1 -- 172.21.15.43:0/1130648037 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a4004920 con 0x560b74bea580 2024-04-21T08:47:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.722+0000 7f45c41fc740 1 -- 172.21.15.43:0/1130648037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74bea580 msgr2=0x560b74bea960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.722+0000 7f45bb7fe640 1 --2- 172.21.15.43:0/397726743 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f45a80a3d20 0x7f45a80a61e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f45b40a5080 tx=0x7f45b4087040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:51.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.722+0000 7f45c41fc740 1 --2- 172.21.15.43:0/1130648037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74bea580 0x560b74bea960 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f45bc06b630 tx=0x7f45a4005b70 comp rx=0 tx=0).stop 2024-04-21T08:47:51.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.722+0000 7f45c41fc740 1 -- 172.21.15.43:0/1130648037 shutdown_connections 2024-04-21T08:47:51.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.722+0000 7f45c41fc740 1 --2- 172.21.15.43:0/1130648037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b74bf3a80 0x560b74bf5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.722+0000 7f45c41fc740 1 --2- 172.21.15.43:0/1130648037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b74beaea0 0x560b74bf3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.722+0000 7f45c41fc740 1 --2- 172.21.15.43:0/1130648037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74bea580 0x560b74bea960 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.722+0000 7f45c41fc740 1 -- 172.21.15.43:0/1130648037 >> 172.21.15.43:0/1130648037 conn(0x560b74be4cb0 msgr2=0x560b74be5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:51.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.722+0000 7f45c41fc740 1 -- 172.21.15.43:0/1130648037 shutdown_connections 2024-04-21T08:47:51.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.722+0000 7f45c41fc740 1 -- 172.21.15.43:0/1130648037 wait complete. 2024-04-21T08:47:51.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.722+0000 7f45c41fc740 1 -- start start 2024-04-21T08:47:51.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.723+0000 7f45c41fc740 1 Processor -- start 2024-04-21T08:47:51.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.723+0000 7f45c41fc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b74bea580 0x560b74bf0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.723+0000 7f45c41fc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74beaea0 0x560b74bf1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.723+0000 7f45c41fc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b74bf3a80 0x560b74bf1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.723+0000 7f45c41fc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560b74bf6810 con 0x560b74beaea0 2024-04-21T08:47:51.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.723+0000 7f45c41fc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560b74b082f0 con 0x560b74bea580 2024-04-21T08:47:51.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.723+0000 7f45c41fc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560b74bf6990 con 0x560b74bf3a80 2024-04-21T08:47:51.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.723+0000 7f45c41fc740 10 client.0 _ll_register_callbacks cb 0x560b74b15580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:51.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.723+0000 7f45baffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74beaea0 0x560b74bf1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.723+0000 7f45baffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74beaea0 0x560b74bf1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58210/0 (socket says 172.21.15.43:58210) 2024-04-21T08:47:51.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.723+0000 7f45baffd640 1 -- 172.21.15.43:0/3191416146 learned_addr learned my addr 172.21.15.43:0/3191416146 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:51.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.723+0000 7f45b3fff640 1 --2- 172.21.15.43:0/3191416146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b74bea580 0x560b74bf0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.723+0000 7f45b3fff640 1 --2- 172.21.15.43:0/3191416146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b74bf3a80 0x560b74bf1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.723+0000 7f45baffd640 1 -- 172.21.15.43:0/3191416146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b74bf3a80 msgr2=0x560b74bf1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:51.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.723+0000 7f45baffd640 1 --2- 172.21.15.43:0/3191416146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b74bf3a80 0x560b74bf1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.723+0000 7f45baffd640 1 -- 172.21.15.43:0/3191416146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b74bea580 msgr2=0x560b74bf0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:51.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.723+0000 7f45baffd640 1 --2- 172.21.15.43:0/3191416146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b74bea580 0x560b74bf0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.723+0000 7f45baffd640 1 -- 172.21.15.43:0/3191416146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b748613f0 con 0x560b74beaea0 2024-04-21T08:47:51.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.723+0000 7f45baffd640 1 --2- 172.21.15.43:0/3191416146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74beaea0 0x560b74bf1050 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00bf70 tx=0x7f45ac0128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da04005c115a9eb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:51.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.723+0000 7f45a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:51.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.724+0000 7f45a27fc640 1 -- 172.21.15.43:0/3191416146 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac01f060 con 0x560b74beaea0 2024-04-21T08:47:51.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.724+0000 7f45a27fc640 1 -- 172.21.15.43:0/3191416146 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f45ac0108a0 con 0x560b74beaea0 2024-04-21T08:47:51.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.724+0000 7f45c41fc740 1 -- 172.21.15.43:0/3191416146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45a402cc90 con 0x560b74beaea0 2024-04-21T08:47:51.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.724+0000 7f45a27fc640 1 -- 172.21.15.43:0/3191416146 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f45ac026050 con 0x560b74beaea0 2024-04-21T08:47:51.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.724+0000 7f45c41fc740 10 client.19812 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:51.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.724+0000 7f45c41fc740 1 -- 172.21.15.43:0/3191416146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560b748210f0 con 0x560b74beaea0 2024-04-21T08:47:51.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.724+0000 7f45a27fc640 1 -- 172.21.15.43:0/3191416146 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45a402cc90 con 0x560b74beaea0 2024-04-21T08:47:51.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.724+0000 7f45a27fc640 1 -- 172.21.15.43:0/3191416146 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f45ac019030 con 0x560b74beaea0 2024-04-21T08:47:51.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.724+0000 7f45c41fc740 10 client.19812 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:51.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.724+0000 7f45c41fc740 10 client.19812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:51.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.724+0000 7f45c41fc740 20 client.19812 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:51.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.724+0000 7f45c41fc740 1 --2- 172.21.15.43:0/3191416146 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560b74c03ac0 0x560b74c05ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.724+0000 7f45c41fc740 4 client.19812 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:51.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.724+0000 7f45c41fc740 1 -- 172.21.15.43:0/3191416146 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b74b5cc00 con 0x560b74c03ac0 2024-04-21T08:47:51.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.725+0000 7f45b3fff640 1 --2- 172.21.15.43:0/3191416146 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560b74c03ac0 0x560b74c05ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.725+0000 7f45b3fff640 1 --2- 172.21.15.43:0/3191416146 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560b74c03ac0 0x560b74c05ef0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae0f17bb89848e6e server_cookie=a74aa1ec0c714b02 in_seq=0 out_seq=0 2024-04-21T08:47:51.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.725+0000 7f45a27fc640 10 client.19812 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:51.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.726+0000 7f45a27fc640 1 -- 172.21.15.43:0/3191416146 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b74b5cc00 con 0x560b74c03ac0 2024-04-21T08:47:51.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.726+0000 7f45a27fc640 10 client.19812 handle_command_reply: tid=0 2024-04-21T08:47:51.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.726+0000 7f45c41fc740 1 client.19812 shutdown 2024-04-21T08:47:51.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.726+0000 7f45c41fc740 1 -- 172.21.15.43:0/3191416146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74beaea0 msgr2=0x560b74bf1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:51.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.726+0000 7f45c41fc740 1 --2- 172.21.15.43:0/3191416146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74beaea0 0x560b74bf1050 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00bf70 tx=0x7f45ac0128e0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.726+0000 7f45c41fc740 1 -- 172.21.15.43:0/3191416146 shutdown_connections 2024-04-21T08:47:51.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.726+0000 7f45c41fc740 1 --2- 172.21.15.43:0/3191416146 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560b74c03ac0 0x560b74c05ef0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.726+0000 7f45c41fc740 1 --2- 172.21.15.43:0/3191416146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b74bf3a80 0x560b74bf1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.726+0000 7f45a27fc640 0 client.19812 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:51.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.726+0000 7f45c41fc740 1 --2- 172.21.15.43:0/3191416146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74beaea0 0x560b74bf1050 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.726+0000 7f45c41fc740 1 --2- 172.21.15.43:0/3191416146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b74bea580 0x560b74bf0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.726+0000 7f45c41fc740 1 -- 172.21.15.43:0/3191416146 >> 172.21.15.43:0/3191416146 conn(0x560b74be4cb0 msgr2=0x560b74be6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:51.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.726+0000 7f45c41fc740 1 -- 172.21.15.43:0/3191416146 shutdown_connections 2024-04-21T08:47:51.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.726+0000 7f45c41fc740 1 -- 172.21.15.43:0/3191416146 wait complete. 2024-04-21T08:47:51.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.726+0000 7f45c41fc740 20 client.19812 trim_cache size 0 max 16384 2024-04-21T08:47:51.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.743+0000 7f45c41fc740 1 Processor -- start 2024-04-21T08:47:51.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.743+0000 7f45c41fc740 1 -- start start 2024-04-21T08:47:51.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.744+0000 7f45c41fc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b74bea580 0x560b74bf3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.744+0000 7f45c41fc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74beaea0 0x560b74befa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.744+0000 7f45c41fc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b74bf3a80 0x560b74bec260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.744+0000 7f45c41fc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1b1700 con 0x560b74beaea0 2024-04-21T08:47:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.744+0000 7f45c41fc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1b3240 con 0x560b74bea580 2024-04-21T08:47:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.744+0000 7f45c41fc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1b1dd0 con 0x560b74bf3a80 2024-04-21T08:47:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.744+0000 7f45baffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74beaea0 0x560b74befa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.744+0000 7f45baffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74beaea0 0x560b74befa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58218/0 (socket says 172.21.15.43:58218) 2024-04-21T08:47:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.744+0000 7f45baffd640 1 -- 172.21.15.43:0/2650724794 learned_addr learned my addr 172.21.15.43:0/2650724794 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.744+0000 7f45b3fff640 1 --2- 172.21.15.43:0/2650724794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b74bea580 0x560b74bf3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.744+0000 7f45baffd640 1 -- 172.21.15.43:0/2650724794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b74bf3a80 msgr2=0x560b74bec260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:47:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.744+0000 7f45baffd640 1 --2- 172.21.15.43:0/2650724794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b74bf3a80 0x560b74bec260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.744+0000 7f45baffd640 1 -- 172.21.15.43:0/2650724794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b74bea580 msgr2=0x560b74bf3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.744+0000 7f45baffd640 1 --2- 172.21.15.43:0/2650724794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b74bea580 0x560b74bf3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.744+0000 7f45baffd640 1 -- 172.21.15.43:0/2650724794 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b74bff6d0 con 0x560b74beaea0 2024-04-21T08:47:51.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.744+0000 7f45baffd640 1 --2- 172.21.15.43:0/2650724794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74beaea0 0x560b74befa50 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00b9b0 tx=0x7f45ac005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897572acd5a3c3e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:51.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.744+0000 7f45b0ff9640 1 -- 172.21.15.43:0/2650724794 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac000e60 con 0x560b74beaea0 2024-04-21T08:47:51.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.744+0000 7f45b0ff9640 1 -- 172.21.15.43:0/2650724794 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45ac0054f0 con 0x560b74beaea0 2024-04-21T08:47:51.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.745+0000 7f45b0ff9640 1 -- 172.21.15.43:0/2650724794 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac02e070 con 0x560b74beaea0 2024-04-21T08:47:51.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.745+0000 7f45c41fc740 1 -- 172.21.15.43:0/2650724794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74beaea0 msgr2=0x560b74befa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:51.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.745+0000 7f45c41fc740 1 --2- 172.21.15.43:0/2650724794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74beaea0 0x560b74befa50 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00b9b0 tx=0x7f45ac005c70 comp rx=0 tx=0).stop 2024-04-21T08:47:51.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.745+0000 7f45c41fc740 1 -- 172.21.15.43:0/2650724794 shutdown_connections 2024-04-21T08:47:51.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.745+0000 7f45c41fc740 1 --2- 172.21.15.43:0/2650724794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b74bf3a80 0x560b74bec260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.745+0000 7f45c41fc740 1 --2- 172.21.15.43:0/2650724794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74beaea0 0x560b74befa50 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.745+0000 7f45c41fc740 1 --2- 172.21.15.43:0/2650724794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b74bea580 0x560b74bf3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.745+0000 7f45c41fc740 1 -- 172.21.15.43:0/2650724794 >> 172.21.15.43:0/2650724794 conn(0x560b74be4cb0 msgr2=0x560b74be6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:51.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.745+0000 7f45c41fc740 1 -- 172.21.15.43:0/2650724794 shutdown_connections 2024-04-21T08:47:51.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.745+0000 7f45c41fc740 1 -- 172.21.15.43:0/2650724794 wait complete. 2024-04-21T08:47:51.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.746+0000 7f45c41fc740 1 -- start start 2024-04-21T08:47:51.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.746+0000 7f45c41fc740 1 Processor -- start 2024-04-21T08:47:51.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.746+0000 7f45c41fc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b74bea580 0x560b74bf77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.746+0000 7f45c41fc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74beaea0 0x560b74bf7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.746+0000 7f45c41fc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b74bf3a80 0x560b74bf8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.746+0000 7f45c41fc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1b2b70 con 0x560b74beaea0 2024-04-21T08:47:51.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.746+0000 7f45c41fc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1b24a0 con 0x560b74bf3a80 2024-04-21T08:47:51.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.746+0000 7f45c41fc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1af700 con 0x560b74bea580 2024-04-21T08:47:51.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.746+0000 7f45c41fc740 10 client.0 _ll_register_callbacks cb 0x7f45ac00b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:51.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.746+0000 7f45baffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74beaea0 0x560b74bf7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.746+0000 7f45baffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74beaea0 0x560b74bf7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58232/0 (socket says 172.21.15.43:58232) 2024-04-21T08:47:51.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.746+0000 7f45baffd640 1 -- 172.21.15.43:0/3622060267 learned_addr learned my addr 172.21.15.43:0/3622060267 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:47:51.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.746+0000 7f45b3fff640 1 --2- 172.21.15.43:0/3622060267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b74bf3a80 0x560b74bf8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.746+0000 7f45b3fff640 1 --2- 172.21.15.43:0/3622060267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b74bea580 0x560b74bf77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.746+0000 7f45baffd640 1 -- 172.21.15.43:0/3622060267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b74bea580 msgr2=0x560b74bf77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:51.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.746+0000 7f45baffd640 1 --2- 172.21.15.43:0/3622060267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b74bea580 0x560b74bf77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.746+0000 7f45baffd640 1 -- 172.21.15.43:0/3622060267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b74bf3a80 msgr2=0x560b74bf8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:51.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.746+0000 7f45baffd640 1 --2- 172.21.15.43:0/3622060267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b74bf3a80 0x560b74bf8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.746+0000 7f45baffd640 1 -- 172.21.15.43:0/3622060267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b74c69270 con 0x560b74beaea0 2024-04-21T08:47:51.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.746+0000 7f45baffd640 1 --2- 172.21.15.43:0/3622060267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74beaea0 0x560b74bf7d20 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f45ac000c00 tx=0x7f45ac00f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1850da8867335f8d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:51.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.746+0000 7f45a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:47:51.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.747+0000 7f45a27fc640 1 -- 172.21.15.43:0/3622060267 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac036070 con 0x560b74beaea0 2024-04-21T08:47:51.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.747+0000 7f45a27fc640 1 -- 172.21.15.43:0/3622060267 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f45ac0106a0 con 0x560b74beaea0 2024-04-21T08:47:51.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.747+0000 7f45c41fc740 1 -- 172.21.15.43:0/3622060267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45ac0054f0 con 0x560b74beaea0 2024-04-21T08:47:51.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.747+0000 7f45a27fc640 1 -- 172.21.15.43:0/3622060267 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f45ac042050 con 0x560b74beaea0 2024-04-21T08:47:51.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.747+0000 7f45c41fc740 10 client.19818 fetch_fsmap learned FSMap version 30 2024-04-21T08:47:51.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.747+0000 7f45c41fc740 1 -- 172.21.15.43:0/3622060267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560b74c6a560 con 0x560b74beaea0 2024-04-21T08:47:51.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.747+0000 7f45a27fc640 1 -- 172.21.15.43:0/3622060267 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45ac0054f0 con 0x560b74beaea0 2024-04-21T08:47:51.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.747+0000 7f45a27fc640 1 -- 172.21.15.43:0/3622060267 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f45ac019040 con 0x560b74beaea0 2024-04-21T08:47:51.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.747+0000 7f45c41fc740 10 client.19818 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:47:51.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.747+0000 7f45c41fc740 10 client.19818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:51.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.747+0000 7f45c41fc740 20 client.19818 populate_metadata read hostname 'smithi043' 2024-04-21T08:47:51.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.747+0000 7f45c41fc740 1 --2- 172.21.15.43:0/3622060267 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560b74c6abc0 0x560b74c6cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:51.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.747+0000 7f45c41fc740 4 client.19818 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:47:51.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.747+0000 7f45c41fc740 1 -- 172.21.15.43:0/3622060267 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560b74b56a70 con 0x560b74c6abc0 2024-04-21T08:47:51.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.747+0000 7f45b3fff640 1 --2- 172.21.15.43:0/3622060267 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560b74c6abc0 0x560b74c6cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:51.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.748+0000 7f45b3fff640 1 --2- 172.21.15.43:0/3622060267 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560b74c6abc0 0x560b74c6cfe0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4c833d6b469ce5f server_cookie=b4088c7aea5e526a in_seq=0 out_seq=0 2024-04-21T08:47:51.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.748+0000 7f45a27fc640 10 client.19818 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:51.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.748+0000 7f45a27fc640 1 -- 172.21.15.43:0/3622060267 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560b74b56a70 con 0x560b74c6abc0 2024-04-21T08:47:51.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.748+0000 7f45a27fc640 10 client.19818 handle_command_reply: tid=0 2024-04-21T08:47:51.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.748+0000 7f45c41fc740 1 client.19818 shutdown 2024-04-21T08:47:51.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.748+0000 7f45c41fc740 1 -- 172.21.15.43:0/3622060267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74beaea0 msgr2=0x560b74bf7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:51.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.748+0000 7f45c41fc740 1 --2- 172.21.15.43:0/3622060267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74beaea0 0x560b74bf7d20 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f45ac000c00 tx=0x7f45ac00f5f0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.748+0000 7f45c41fc740 1 -- 172.21.15.43:0/3622060267 shutdown_connections 2024-04-21T08:47:51.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.748+0000 7f45c41fc740 1 --2- 172.21.15.43:0/3622060267 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560b74c6abc0 0x560b74c6cfe0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.748+0000 7f45c41fc740 1 --2- 172.21.15.43:0/3622060267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560b74bf3a80 0x560b74bf8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.748+0000 7f45c41fc740 1 --2- 172.21.15.43:0/3622060267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560b74beaea0 0x560b74bf7d20 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.748+0000 7f45c41fc740 1 --2- 172.21.15.43:0/3622060267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b74bea580 0x560b74bf77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.748+0000 7f45a27fc640 0 client.19818 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:47:51.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.748+0000 7f45c41fc740 1 -- 172.21.15.43:0/3622060267 >> 172.21.15.43:0/3622060267 conn(0x560b74be4cb0 msgr2=0x560b74be60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:51.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.748+0000 7f45c41fc740 1 -- 172.21.15.43:0/3622060267 shutdown_connections 2024-04-21T08:47:51.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.748+0000 7f45c41fc740 1 -- 172.21.15.43:0/3622060267 wait complete. 2024-04-21T08:47:51.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.748+0000 7f45c41fc740 20 client.19818 trim_cache size 0 max 16384 2024-04-21T08:47:51.755 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:47:51.755 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:47:51.755 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-21T08:47:51.756 INFO:teuthology.orchestra.run.smithi043.stdout: "48d06675-5b3a-46c1-9208-e45471341e68": { 2024-04-21T08:47:51.756 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-21T08:47:51.756 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "48d06675-5b3a-46c1-9208-e45471341e68", 2024-04-21T08:47:51.756 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-21T08:47:51.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:47:51.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:47:51.756 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:47:51.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.749+0000 7f45c152b640 1 -- 172.21.15.43:0/397726743 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f45a80a3d20 msgr2=0x7f45a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:51.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.749+0000 7f45c152b640 1 --2- 172.21.15.43:0/397726743 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f45a80a3d20 0x7f45a80a61e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f45b40a5080 tx=0x7f45b4087040 comp rx=0 tx=0).stop 2024-04-21T08:47:51.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.749+0000 7f45c152b640 1 -- 172.21.15.43:0/397726743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45bc074cd0 msgr2=0x7f45bc1a6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:51.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.749+0000 7f45c152b640 1 --2- 172.21.15.43:0/397726743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45bc074cd0 0x7f45bc1a6a50 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f45b400eaa0 tx=0x7f45b400ef60 comp rx=0 tx=0).stop 2024-04-21T08:47:51.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.749+0000 7f45c152b640 1 -- 172.21.15.43:0/397726743 shutdown_connections 2024-04-21T08:47:51.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.749+0000 7f45c152b640 1 --2- 172.21.15.43:0/397726743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45bc074cd0 0x7f45bc1a6a50 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.749+0000 7f45c152b640 1 --2- 172.21.15.43:0/397726743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45bc107270 0x7f45bc074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.749+0000 7f45c152b640 1 --2- 172.21.15.43:0/397726743 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f45a80a3d20 0x7f45a80a61e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.749+0000 7f45c152b640 1 --2- 172.21.15.43:0/397726743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45bc1021a0 0x7f45bc074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:51.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.749+0000 7f45c152b640 1 -- 172.21.15.43:0/397726743 >> 172.21.15.43:0/397726743 conn(0x7f45bc0fda40 msgr2=0x7f45bc0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:51.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.749+0000 7f45c152b640 1 -- 172.21.15.43:0/397726743 shutdown_connections 2024-04-21T08:47:51.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:47:51.749+0000 7f45c152b640 1 -- 172.21.15.43:0/397726743 wait complete. 2024-04-21T08:47:51.760 INFO:tasks.cephfs.filesystem:scrub status for tag:48d06675-5b3a-46c1-9208-e45471341e68 - {'path': '/', 'tag': '48d06675-5b3a-46c1-9208-e45471341e68', 'options': 'recursive,force'} 2024-04-21T08:47:52.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:52 smithi175 ceph-mon[31920]: pgmap v276: 129 pgs: 129 active+clean; 1.9 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 67 KiB/s wr, 27 op/s 2024-04-21T08:47:52.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:52 smithi112 ceph-mon[31085]: pgmap v276: 129 pgs: 129 active+clean; 1.9 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 67 KiB/s wr, 27 op/s 2024-04-21T08:47:52.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:52 smithi043 ceph-mon[27978]: pgmap v276: 129 pgs: 129 active+clean; 1.9 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 67 KiB/s wr, 27 op/s 2024-04-21T08:47:53.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:53 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:47:53.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:53 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:53.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:53 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:47:53.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:53 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:53.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:53 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:47:53.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:53 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:53.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:53 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:47:53.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:53 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:53.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:53 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:47:53.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:53 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:53.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:53 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:47:53.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:53 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:54.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:54 smithi175 ceph-mon[31920]: pgmap v277: 129 pgs: 129 active+clean; 1.9 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 17 op/s 2024-04-21T08:47:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:54 smithi112 ceph-mon[31085]: pgmap v277: 129 pgs: 129 active+clean; 1.9 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 17 op/s 2024-04-21T08:47:54.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:54 smithi043 ceph-mon[27978]: pgmap v277: 129 pgs: 129 active+clean; 1.9 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 17 op/s 2024-04-21T08:47:55.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:55 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:47:55.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:55 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:47:55.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:55 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:47:56.445 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:56 smithi112 ceph-mon[31085]: pgmap v278: 129 pgs: 129 active+clean; 1.9 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 17 op/s 2024-04-21T08:47:56.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:56 smithi175 ceph-mon[31920]: pgmap v278: 129 pgs: 129 active+clean; 1.9 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 17 op/s 2024-04-21T08:47:56.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:56 smithi043 ceph-mon[27978]: pgmap v278: 129 pgs: 129 active+clean; 1.9 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 17 op/s 2024-04-21T08:47:58.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:47:58 smithi175 ceph-mon[31920]: pgmap v279: 129 pgs: 129 active+clean; 2.0 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 25 op/s 2024-04-21T08:47:58.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:47:58 smithi112 ceph-mon[31085]: pgmap v279: 129 pgs: 129 active+clean; 2.0 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 25 op/s 2024-04-21T08:47:58.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:47:58 smithi043 ceph-mon[27978]: pgmap v279: 129 pgs: 129 active+clean; 2.0 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 25 op/s 2024-04-21T08:48:00.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:00 smithi175 ceph-mon[31920]: pgmap v280: 129 pgs: 129 active+clean; 2.0 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 18 op/s 2024-04-21T08:48:00.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:00 smithi112 ceph-mon[31085]: pgmap v280: 129 pgs: 129 active+clean; 2.0 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 18 op/s 2024-04-21T08:48:00.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:00 smithi043 ceph-mon[27978]: pgmap v280: 129 pgs: 129 active+clean; 2.0 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 18 op/s 2024-04-21T08:48:02.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:02 smithi175 ceph-mon[31920]: pgmap v281: 129 pgs: 129 active+clean; 2.3 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 26 op/s 2024-04-21T08:48:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.467+0000 7fa879304640 1 -- 172.21.15.43:0/3293475063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa874102b70 msgr2=0x7fa874106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.467+0000 7fa879304640 1 --2- 172.21.15.43:0/3293475063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa874102b70 0x7fa874106bb0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fa85c009900 tx=0x7fa85c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.467+0000 7fa879304640 1 -- 172.21.15.43:0/3293475063 shutdown_connections 2024-04-21T08:48:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.467+0000 7fa879304640 1 --2- 172.21.15.43:0/3293475063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa874107290 0x7fa87410ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.467+0000 7fa879304640 1 --2- 172.21.15.43:0/3293475063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa874102b70 0x7fa874106bb0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.467+0000 7fa879304640 1 --2- 172.21.15.43:0/3293475063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8741021c0 0x7fa8741025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.467+0000 7fa879304640 1 -- 172.21.15.43:0/3293475063 >> 172.21.15.43:0/3293475063 conn(0x7fa8740fda20 msgr2=0x7fa8740ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.467+0000 7fa879304640 1 -- 172.21.15.43:0/3293475063 shutdown_connections 2024-04-21T08:48:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.467+0000 7fa879304640 1 -- 172.21.15.43:0/3293475063 wait complete. 2024-04-21T08:48:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.467+0000 7fa879304640 1 Processor -- start 2024-04-21T08:48:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.467+0000 7fa879304640 1 -- start start 2024-04-21T08:48:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.468+0000 7fa879304640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8741021c0 0x7fa8741986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.468+0000 7fa879304640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa874107290 0x7fa874198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.468+0000 7fa879304640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa87419cf70 0x7fa87419d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.468+0000 7fa879304640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa8741121b0 con 0x7fa8741021c0 2024-04-21T08:48:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.468+0000 7fa879304640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa874112030 con 0x7fa874107290 2024-04-21T08:48:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.468+0000 7fa879304640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa874112330 con 0x7fa87419cf70 2024-04-21T08:48:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.468+0000 7fa872ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8741021c0 0x7fa8741986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.468+0000 7fa872ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8741021c0 0x7fa8741986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44698/0 (socket says 172.21.15.43:44698) 2024-04-21T08:48:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.468+0000 7fa872ffd640 1 -- 172.21.15.43:0/2307642509 learned_addr learned my addr 172.21.15.43:0/2307642509 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.468+0000 7fa86bfff640 1 --2- 172.21.15.43:0/2307642509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa874107290 0x7fa874198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.468+0000 7fa8737fe640 1 --2- 172.21.15.43:0/2307642509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa87419cf70 0x7fa87419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.468+0000 7fa872ffd640 1 -- 172.21.15.43:0/2307642509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa87419cf70 msgr2=0x7fa87419d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.468+0000 7fa872ffd640 1 --2- 172.21.15.43:0/2307642509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa87419cf70 0x7fa87419d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.468+0000 7fa872ffd640 1 -- 172.21.15.43:0/2307642509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa874107290 msgr2=0x7fa874198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.468+0000 7fa872ffd640 1 --2- 172.21.15.43:0/2307642509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa874107290 0x7fa874198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.468+0000 7fa872ffd640 1 -- 172.21.15.43:0/2307642509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87419da60 con 0x7fa8741021c0 2024-04-21T08:48:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.468+0000 7fa86bfff640 1 --2- 172.21.15.43:0/2307642509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa874107290 0x7fa874198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.468+0000 7fa872ffd640 1 --2- 172.21.15.43:0/2307642509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8741021c0 0x7fa8741986b0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fa86400d950 tx=0x7fa86400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.468+0000 7fa870ff9640 1 -- 172.21.15.43:0/2307642509 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864017030 con 0x7fa8741021c0 2024-04-21T08:48:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.469+0000 7fa870ff9640 1 -- 172.21.15.43:0/2307642509 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa864004020 con 0x7fa8741021c0 2024-04-21T08:48:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.469+0000 7fa879304640 1 -- 172.21.15.43:0/2307642509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87419dd50 con 0x7fa8741021c0 2024-04-21T08:48:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.469+0000 7fa870ff9640 1 -- 172.21.15.43:0/2307642509 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8640047a0 con 0x7fa8741021c0 2024-04-21T08:48:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.469+0000 7fa879304640 1 -- 172.21.15.43:0/2307642509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8741a5510 con 0x7fa8741021c0 2024-04-21T08:48:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.472+0000 7fa870ff9640 1 -- 172.21.15.43:0/2307642509 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa864009a30 con 0x7fa8741021c0 2024-04-21T08:48:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.472+0000 7fa87bfd5740 1 Processor -- start 2024-04-21T08:48:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.472+0000 7fa87bfd5740 1 -- start start 2024-04-21T08:48:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.472+0000 7fa87bfd5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c1dd608d0 0x555c1dd5f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.472+0000 7fa87bfd5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c1dd5fb60 0x555c1dd5ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.472+0000 7fa87bfd5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd6d4f0 0x555c1dd6f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.472+0000 7fa87bfd5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555c1dcc48d0 con 0x555c1dd6d4f0 2024-04-21T08:48:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.472+0000 7fa87bfd5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555c1dc61e80 con 0x555c1dd5fb60 2024-04-21T08:48:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.472+0000 7fa87bfd5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555c1dcc0890 con 0x555c1dd608d0 2024-04-21T08:48:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.472+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd6d4f0 0x555c1dd6f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.472+0000 7fa870ff9640 1 --2- 172.21.15.43:0/2307642509 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa8600a3dc0 0x7fa8600a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.472+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd6d4f0 0x555c1dd6f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44706/0 (socket says 172.21.15.43:44706) 2024-04-21T08:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.472+0000 7fa86bfff640 1 -- 172.21.15.43:0/1573910122 learned_addr learned my addr 172.21.15.43:0/1573910122 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.472+0000 7fa872ffd640 1 --2- 172.21.15.43:0/1573910122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c1dd5fb60 0x555c1dd5ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.472+0000 7fa8737fe640 1 --2- 172.21.15.43:0/1573910122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c1dd608d0 0x555c1dd5f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.472+0000 7fa8737fe640 1 --2- 172.21.15.43:0/2307642509 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa8600a3dc0 0x7fa8600a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.472+0000 7fa870ff9640 1 -- 172.21.15.43:0/2307642509 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fa8640d0790 con 0x7fa8741021c0 2024-04-21T08:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.472+0000 7fa86bfff640 1 -- 172.21.15.43:0/1573910122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c1dd608d0 msgr2=0x555c1dd5f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.472+0000 7fa86bfff640 1 --2- 172.21.15.43:0/1573910122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c1dd608d0 0x555c1dd5f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.472+0000 7fa86bfff640 1 -- 172.21.15.43:0/1573910122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c1dd5fb60 msgr2=0x555c1dd5ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.472+0000 7fa86bfff640 1 --2- 172.21.15.43:0/1573910122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c1dd5fb60 0x555c1dd5ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.472+0000 7fa86bfff640 1 -- 172.21.15.43:0/1573910122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c1dc83320 con 0x555c1dd6d4f0 2024-04-21T08:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.472+0000 7fa86bfff640 1 --2- 172.21.15.43:0/1573910122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd6d4f0 0x555c1dd6f8e0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fa87406b5f0 tx=0x7fa85c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c4239d26edd97 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.472+0000 7fa8737fe640 1 --2- 172.21.15.43:0/2307642509 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa8600a3dc0 0x7fa8600a6280 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa86c005c40 tx=0x7fa86c005bd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.473+0000 7fa8697fa640 1 -- 172.21.15.43:0/1573910122 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c044020 con 0x555c1dd6d4f0 2024-04-21T08:48:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.473+0000 7fa8697fa640 1 -- 172.21.15.43:0/1573910122 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa85c02cc90 con 0x555c1dd6d4f0 2024-04-21T08:48:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.473+0000 7fa8697fa640 1 -- 172.21.15.43:0/1573910122 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c004920 con 0x555c1dd6d4f0 2024-04-21T08:48:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.473+0000 7fa87bfd5740 1 -- 172.21.15.43:0/1573910122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd6d4f0 msgr2=0x555c1dd6f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.473+0000 7fa87bfd5740 1 --2- 172.21.15.43:0/1573910122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd6d4f0 0x555c1dd6f8e0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fa87406b5f0 tx=0x7fa85c005e80 comp rx=0 tx=0).stop 2024-04-21T08:48:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.473+0000 7fa87bfd5740 1 -- 172.21.15.43:0/1573910122 shutdown_connections 2024-04-21T08:48:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.474+0000 7fa87bfd5740 1 --2- 172.21.15.43:0/1573910122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd6d4f0 0x555c1dd6f8e0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.474+0000 7fa87bfd5740 1 --2- 172.21.15.43:0/1573910122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c1dd5fb60 0x555c1dd5ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.474+0000 7fa87bfd5740 1 --2- 172.21.15.43:0/1573910122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c1dd608d0 0x555c1dd5f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.474+0000 7fa87bfd5740 1 -- 172.21.15.43:0/1573910122 >> 172.21.15.43:0/1573910122 conn(0x555c1dd26c90 msgr2=0x555c1dd5a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.474+0000 7fa87bfd5740 1 -- 172.21.15.43:0/1573910122 shutdown_connections 2024-04-21T08:48:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.474+0000 7fa87bfd5740 1 -- 172.21.15.43:0/1573910122 wait complete. 2024-04-21T08:48:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.474+0000 7fa87bfd5740 1 -- start start 2024-04-21T08:48:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.474+0000 7fa87bfd5740 1 Processor -- start 2024-04-21T08:48:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.474+0000 7fa87bfd5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c1dd5fb60 0x555c1dd6ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.474+0000 7fa87bfd5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c1dd608d0 0x555c1dd6a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.474+0000 7fa87bfd5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd6d4f0 0x555c1dd6a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.474+0000 7fa87bfd5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555c1dd701c0 con 0x555c1dd6d4f0 2024-04-21T08:48:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.474+0000 7fa87bfd5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555c1dc4ceb0 con 0x555c1dd608d0 2024-04-21T08:48:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.474+0000 7fa87bfd5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555c1dd70340 con 0x555c1dd5fb60 2024-04-21T08:48:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.474+0000 7fa87bfd5740 10 client.0 _ll_register_callbacks cb 0x555c1dc79b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.474+0000 7fa8737fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd6d4f0 0x555c1dd6a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.474+0000 7fa8737fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd6d4f0 0x555c1dd6a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44714/0 (socket says 172.21.15.43:44714) 2024-04-21T08:48:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.474+0000 7fa8737fe640 1 -- 172.21.15.43:0/1134248274 learned_addr learned my addr 172.21.15.43:0/1134248274 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.474+0000 7fa86bfff640 1 --2- 172.21.15.43:0/1134248274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c1dd608d0 0x555c1dd6a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.475+0000 7fa872ffd640 1 --2- 172.21.15.43:0/1134248274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c1dd5fb60 0x555c1dd6ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.475+0000 7fa8737fe640 1 -- 172.21.15.43:0/1134248274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c1dd5fb60 msgr2=0x555c1dd6ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.475+0000 7fa8737fe640 1 --2- 172.21.15.43:0/1134248274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c1dd5fb60 0x555c1dd6ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.475+0000 7fa8737fe640 1 -- 172.21.15.43:0/1134248274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c1dd608d0 msgr2=0x555c1dd6a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.475+0000 7fa8737fe640 1 --2- 172.21.15.43:0/1134248274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c1dd608d0 0x555c1dd6a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.475+0000 7fa8737fe640 1 -- 172.21.15.43:0/1134248274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c1d9d7210 con 0x555c1dd6d4f0 2024-04-21T08:48:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.475+0000 7fa8737fe640 1 --2- 172.21.15.43:0/1134248274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd6d4f0 0x555c1dd6a740 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00db90 tx=0x7fa86c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa2d5d758806f14 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.475+0000 7fa85a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.475+0000 7fa85a7fc640 1 -- 172.21.15.43:0/1134248274 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c010ca0 con 0x555c1dd6d4f0 2024-04-21T08:48:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.475+0000 7fa85a7fc640 1 -- 172.21.15.43:0/1134248274 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fa86c020440 con 0x555c1dd6d4f0 2024-04-21T08:48:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.475+0000 7fa87bfd5740 1 -- 172.21.15.43:0/1134248274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa85c02cc90 con 0x555c1dd6d4f0 2024-04-21T08:48:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.475+0000 7fa85a7fc640 1 -- 172.21.15.43:0/1134248274 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa86c032050 con 0x555c1dd6d4f0 2024-04-21T08:48:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.475+0000 7fa87bfd5740 10 client.19830 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.475+0000 7fa87bfd5740 1 -- 172.21.15.43:0/1134248274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555c1daa3620 con 0x555c1dd6d4f0 2024-04-21T08:48:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.475+0000 7fa85a7fc640 1 -- 172.21.15.43:0/1134248274 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa85c02cc90 con 0x555c1dd6d4f0 2024-04-21T08:48:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.476+0000 7fa85a7fc640 1 -- 172.21.15.43:0/1134248274 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa86c02a510 con 0x555c1dd6d4f0 2024-04-21T08:48:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.476+0000 7fa87bfd5740 10 client.19830 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.476+0000 7fa87bfd5740 10 client.19830 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.476+0000 7fa87bfd5740 20 client.19830 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.476+0000 7fa87bfd5740 1 --2- 172.21.15.43:0/1134248274 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555c1dd7d4c0 0x555c1dd7f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.476+0000 7fa87bfd5740 4 client.19830 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.476+0000 7fa87bfd5740 1 -- 172.21.15.43:0/1134248274 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c1dcd8880 con 0x555c1dd7d4c0 2024-04-21T08:48:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.476+0000 7fa872ffd640 1 --2- 172.21.15.43:0/1134248274 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555c1dd7d4c0 0x555c1dd7f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.476+0000 7fa872ffd640 1 --2- 172.21.15.43:0/1134248274 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555c1dd7d4c0 0x555c1dd7f8d0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d44974b5f3f848da server_cookie=b0762c577adc59fd in_seq=0 out_seq=0 2024-04-21T08:48:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.476+0000 7fa85a7fc640 10 client.19830 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.477+0000 7fa85a7fc640 1 -- 172.21.15.43:0/1134248274 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555c1dcd8880 con 0x555c1dd7d4c0 2024-04-21T08:48:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.477+0000 7fa85a7fc640 10 client.19830 handle_command_reply: tid=0 2024-04-21T08:48:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.477+0000 7fa87bfd5740 1 client.19830 shutdown 2024-04-21T08:48:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.477+0000 7fa87bfd5740 1 -- 172.21.15.43:0/1134248274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd6d4f0 msgr2=0x555c1dd6a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.477+0000 7fa87bfd5740 1 --2- 172.21.15.43:0/1134248274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd6d4f0 0x555c1dd6a740 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00db90 tx=0x7fa86c004a40 comp rx=0 tx=0).stop 2024-04-21T08:48:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.477+0000 7fa87bfd5740 1 -- 172.21.15.43:0/1134248274 shutdown_connections 2024-04-21T08:48:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.477+0000 7fa87bfd5740 1 --2- 172.21.15.43:0/1134248274 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555c1dd7d4c0 0x555c1dd7f8d0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.477+0000 7fa87bfd5740 1 --2- 172.21.15.43:0/1134248274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd6d4f0 0x555c1dd6a740 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.477+0000 7fa87bfd5740 1 --2- 172.21.15.43:0/1134248274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c1dd608d0 0x555c1dd6a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.477+0000 7fa87bfd5740 1 --2- 172.21.15.43:0/1134248274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c1dd5fb60 0x555c1dd6ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.477+0000 7fa85a7fc640 0 client.19830 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.477+0000 7fa87bfd5740 1 -- 172.21.15.43:0/1134248274 >> 172.21.15.43:0/1134248274 conn(0x555c1dd26c90 msgr2=0x555c1dd5c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.477+0000 7fa87bfd5740 1 -- 172.21.15.43:0/1134248274 shutdown_connections 2024-04-21T08:48:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.477+0000 7fa87bfd5740 1 -- 172.21.15.43:0/1134248274 wait complete. 2024-04-21T08:48:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.477+0000 7fa87bfd5740 20 client.19830 trim_cache size 0 max 16384 2024-04-21T08:48:02.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.492+0000 7fa87bfd5740 1 Processor -- start 2024-04-21T08:48:02.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.492+0000 7fa87bfd5740 1 -- start start 2024-04-21T08:48:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.493+0000 7fa87bfd5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c1dd5fb60 0x555c1dd790c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.493+0000 7fa87bfd5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c1dd608d0 0x555c1dd6c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.493+0000 7fa87bfd5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd6d4f0 0x555c1dd6ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.493+0000 7fa87bfd5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa8741ad310 con 0x555c1dd6d4f0 2024-04-21T08:48:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.493+0000 7fa87bfd5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa8741aee50 con 0x555c1dd5fb60 2024-04-21T08:48:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.493+0000 7fa87bfd5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8741ad9e0 con 0x555c1dd608d0 2024-04-21T08:48:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.493+0000 7fa8737fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd6d4f0 0x555c1dd6ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.493+0000 7fa8737fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd6d4f0 0x555c1dd6ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44722/0 (socket says 172.21.15.43:44722) 2024-04-21T08:48:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.493+0000 7fa8737fe640 1 -- 172.21.15.43:0/554296376 learned_addr learned my addr 172.21.15.43:0/554296376 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.493+0000 7fa872ffd640 1 --2- 172.21.15.43:0/554296376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c1dd5fb60 0x555c1dd790c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.493+0000 7fa8737fe640 1 -- 172.21.15.43:0/554296376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c1dd608d0 msgr2=0x555c1dd6c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.493+0000 7fa8737fe640 1 --2- 172.21.15.43:0/554296376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c1dd608d0 0x555c1dd6c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.493+0000 7fa8737fe640 1 -- 172.21.15.43:0/554296376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c1dd5fb60 msgr2=0x555c1dd790c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.493+0000 7fa8737fe640 1 --2- 172.21.15.43:0/554296376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c1dd5fb60 0x555c1dd790c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.493+0000 7fa8737fe640 1 -- 172.21.15.43:0/554296376 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c1dd55f30 con 0x555c1dd6d4f0 2024-04-21T08:48:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.493+0000 7fa872ffd640 1 --2- 172.21.15.43:0/554296376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c1dd5fb60 0x555c1dd790c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.493+0000 7fa8737fe640 1 --2- 172.21.15.43:0/554296376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd6d4f0 0x555c1dd6ce50 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fa86c006010 tx=0x7fa86c02af20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb43a82c40b4b39 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.494+0000 7fa868ff9640 1 -- 172.21.15.43:0/554296376 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c03b070 con 0x555c1dd6d4f0 2024-04-21T08:48:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.494+0000 7fa868ff9640 1 -- 172.21.15.43:0/554296376 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa86c010820 con 0x555c1dd6d4f0 2024-04-21T08:48:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.494+0000 7fa868ff9640 1 -- 172.21.15.43:0/554296376 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c005030 con 0x555c1dd6d4f0 2024-04-21T08:48:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.494+0000 7fa87bfd5740 1 -- 172.21.15.43:0/554296376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd6d4f0 msgr2=0x555c1dd6ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.494+0000 7fa87bfd5740 1 --2- 172.21.15.43:0/554296376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd6d4f0 0x555c1dd6ce50 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fa86c006010 tx=0x7fa86c02af20 comp rx=0 tx=0).stop 2024-04-21T08:48:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.494+0000 7fa87bfd5740 1 -- 172.21.15.43:0/554296376 shutdown_connections 2024-04-21T08:48:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.494+0000 7fa87bfd5740 1 --2- 172.21.15.43:0/554296376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd6d4f0 0x555c1dd6ce50 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.494+0000 7fa87bfd5740 1 --2- 172.21.15.43:0/554296376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c1dd608d0 0x555c1dd6c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.494+0000 7fa87bfd5740 1 --2- 172.21.15.43:0/554296376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c1dd5fb60 0x555c1dd790c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.494+0000 7fa87bfd5740 1 -- 172.21.15.43:0/554296376 >> 172.21.15.43:0/554296376 conn(0x555c1dd26c90 msgr2=0x555c1dd68ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.494+0000 7fa87bfd5740 1 -- 172.21.15.43:0/554296376 shutdown_connections 2024-04-21T08:48:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.494+0000 7fa87bfd5740 1 -- 172.21.15.43:0/554296376 wait complete. 2024-04-21T08:48:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.495+0000 7fa87bfd5740 1 -- start start 2024-04-21T08:48:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.495+0000 7fa87bfd5740 1 Processor -- start 2024-04-21T08:48:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.495+0000 7fa87bfd5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c1dd5fb60 0x555c1dd75a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.495+0000 7fa87bfd5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd608d0 0x555c1dd70dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.495+0000 7fa87bfd5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c1dd6d4f0 0x555c1dd71360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.495+0000 7fa87bfd5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa8741ae780 con 0x555c1dd608d0 2024-04-21T08:48:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.495+0000 7fa87bfd5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa8741ae0b0 con 0x555c1dd6d4f0 2024-04-21T08:48:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.495+0000 7fa87bfd5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8741ab310 con 0x555c1dd5fb60 2024-04-21T08:48:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.495+0000 7fa87bfd5740 10 client.0 _ll_register_callbacks cb 0x7fa86c00d810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.495+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd608d0 0x555c1dd70dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.495+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd608d0 0x555c1dd70dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44728/0 (socket says 172.21.15.43:44728) 2024-04-21T08:48:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.495+0000 7fa86bfff640 1 -- 172.21.15.43:0/1099948252 learned_addr learned my addr 172.21.15.43:0/1099948252 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.495+0000 7fa872ffd640 1 --2- 172.21.15.43:0/1099948252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c1dd5fb60 0x555c1dd75a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.495+0000 7fa8737fe640 1 --2- 172.21.15.43:0/1099948252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c1dd6d4f0 0x555c1dd71360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.495+0000 7fa86bfff640 1 -- 172.21.15.43:0/1099948252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c1dd5fb60 msgr2=0x555c1dd75a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.495+0000 7fa86bfff640 1 --2- 172.21.15.43:0/1099948252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c1dd5fb60 0x555c1dd75a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.495+0000 7fa86bfff640 1 -- 172.21.15.43:0/1099948252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c1dd6d4f0 msgr2=0x555c1dd71360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.495+0000 7fa86bfff640 1 --2- 172.21.15.43:0/1099948252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c1dd6d4f0 0x555c1dd71360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.495+0000 7fa86bfff640 1 -- 172.21.15.43:0/1099948252 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c1dd71b90 con 0x555c1dd608d0 2024-04-21T08:48:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.496+0000 7fa86bfff640 1 --2- 172.21.15.43:0/1099948252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd608d0 0x555c1dd70dc0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0053f0 tx=0x7fa85c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a15077f4cf06a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.496+0000 7fa85a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.496+0000 7fa85a7fc640 1 -- 172.21.15.43:0/1099948252 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c044040 con 0x555c1dd608d0 2024-04-21T08:48:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.496+0000 7fa85a7fc640 1 -- 172.21.15.43:0/1099948252 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fa85c02c9e0 con 0x555c1dd608d0 2024-04-21T08:48:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.496+0000 7fa87bfd5740 1 -- 172.21.15.43:0/1099948252 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa86c010820 con 0x555c1dd608d0 2024-04-21T08:48:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.496+0000 7fa85a7fc640 1 -- 172.21.15.43:0/1099948252 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa85c05a050 con 0x555c1dd608d0 2024-04-21T08:48:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.496+0000 7fa87bfd5740 10 client.19836 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.496+0000 7fa87bfd5740 1 -- 172.21.15.43:0/1099948252 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555c1dde3aa0 con 0x555c1dd608d0 2024-04-21T08:48:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.496+0000 7fa85a7fc640 1 -- 172.21.15.43:0/1099948252 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa86c010820 con 0x555c1dd608d0 2024-04-21T08:48:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.496+0000 7fa85a7fc640 1 -- 172.21.15.43:0/1099948252 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa85c040470 con 0x555c1dd608d0 2024-04-21T08:48:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.496+0000 7fa87bfd5740 10 client.19836 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.496+0000 7fa87bfd5740 10 client.19836 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.496+0000 7fa87bfd5740 20 client.19836 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.496+0000 7fa87bfd5740 1 --2- 172.21.15.43:0/1099948252 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555c1dde3d80 0x555c1dde6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.496+0000 7fa87bfd5740 4 client.19836 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:48:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.496+0000 7fa87bfd5740 1 -- 172.21.15.43:0/1099948252 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555c1dc643d0 con 0x555c1dde3d80 2024-04-21T08:48:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.497+0000 7fa872ffd640 1 --2- 172.21.15.43:0/1099948252 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555c1dde3d80 0x555c1dde6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.497+0000 7fa872ffd640 1 --2- 172.21.15.43:0/1099948252 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555c1dde3d80 0x555c1dde6170 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bec64dba1f1bb84a server_cookie=4b76c123952b9b1 in_seq=0 out_seq=0 2024-04-21T08:48:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.497+0000 7fa85a7fc640 10 client.19836 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa85a7fc640 1 -- 172.21.15.43:0/1099948252 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555c1dc643d0 con 0x555c1dde3d80 2024-04-21T08:48:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa85a7fc640 10 client.19836 handle_command_reply: tid=0 2024-04-21T08:48:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa87bfd5740 1 client.19836 shutdown 2024-04-21T08:48:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa87bfd5740 1 -- 172.21.15.43:0/1099948252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd608d0 msgr2=0x555c1dd70dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa87bfd5740 1 --2- 172.21.15.43:0/1099948252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd608d0 0x555c1dd70dc0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0053f0 tx=0x7fa85c005b50 comp rx=0 tx=0).stop 2024-04-21T08:48:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa87bfd5740 1 -- 172.21.15.43:0/1099948252 shutdown_connections 2024-04-21T08:48:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa87bfd5740 1 --2- 172.21.15.43:0/1099948252 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x555c1dde3d80 0x555c1dde6170 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa87bfd5740 1 --2- 172.21.15.43:0/1099948252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555c1dd6d4f0 0x555c1dd71360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa87bfd5740 1 --2- 172.21.15.43:0/1099948252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555c1dd608d0 0x555c1dd70dc0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa87bfd5740 1 --2- 172.21.15.43:0/1099948252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555c1dd5fb60 0x555c1dd75a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa85a7fc640 0 client.19836 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa87bfd5740 1 -- 172.21.15.43:0/1099948252 >> 172.21.15.43:0/1099948252 conn(0x555c1dd26c90 msgr2=0x555c1dd59040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa87bfd5740 1 -- 172.21.15.43:0/1099948252 shutdown_connections 2024-04-21T08:48:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa87bfd5740 1 -- 172.21.15.43:0/1099948252 wait complete. 2024-04-21T08:48:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa87bfd5740 20 client.19836 trim_cache size 0 max 16384 2024-04-21T08:48:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:48:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:48:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:48:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:48:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa879304640 1 -- 172.21.15.43:0/2307642509 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa8600a3dc0 msgr2=0x7fa8600a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa879304640 1 --2- 172.21.15.43:0/2307642509 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa8600a3dc0 0x7fa8600a6280 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa86c005c40 tx=0x7fa86c005bd0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa879304640 1 -- 172.21.15.43:0/2307642509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8741021c0 msgr2=0x7fa8741986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa879304640 1 --2- 172.21.15.43:0/2307642509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8741021c0 0x7fa8741986b0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fa86400d950 tx=0x7fa86400de10 comp rx=0 tx=0).stop 2024-04-21T08:48:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa879304640 1 -- 172.21.15.43:0/2307642509 shutdown_connections 2024-04-21T08:48:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa879304640 1 --2- 172.21.15.43:0/2307642509 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa8600a3dc0 0x7fa8600a6280 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa879304640 1 --2- 172.21.15.43:0/2307642509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa87419cf70 0x7fa87419d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa879304640 1 --2- 172.21.15.43:0/2307642509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa874107290 0x7fa874198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa879304640 1 --2- 172.21.15.43:0/2307642509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8741021c0 0x7fa8741986b0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa879304640 1 -- 172.21.15.43:0/2307642509 >> 172.21.15.43:0/2307642509 conn(0x7fa8740fda20 msgr2=0x7fa8740ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa879304640 1 -- 172.21.15.43:0/2307642509 shutdown_connections 2024-04-21T08:48:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:02.513+0000 7fa879304640 1 -- 172.21.15.43:0/2307642509 wait complete. 2024-04-21T08:48:02.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:02 smithi112 ceph-mon[31085]: pgmap v281: 129 pgs: 129 active+clean; 2.3 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 26 op/s 2024-04-21T08:48:02.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:02 smithi043 ceph-mon[27978]: pgmap v281: 129 pgs: 129 active+clean; 2.3 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 26 op/s 2024-04-21T08:48:04.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:04 smithi175 ceph-mon[31920]: pgmap v282: 129 pgs: 129 active+clean; 2.3 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-21T08:48:04.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:04 smithi112 ceph-mon[31085]: pgmap v282: 129 pgs: 129 active+clean; 2.3 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-21T08:48:04.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:04 smithi043 ceph-mon[27978]: pgmap v282: 129 pgs: 129 active+clean; 2.3 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-21T08:48:06.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:06 smithi175 ceph-mon[31920]: pgmap v283: 129 pgs: 129 active+clean; 2.3 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-21T08:48:06.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:06 smithi175 ceph-mon[31920]: osdmap e101: 12 total, 12 up, 12 in 2024-04-21T08:48:06.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:06 smithi112 ceph-mon[31085]: pgmap v283: 129 pgs: 129 active+clean; 2.3 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-21T08:48:06.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:06 smithi112 ceph-mon[31085]: osdmap e101: 12 total, 12 up, 12 in 2024-04-21T08:48:06.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:06 smithi043 ceph-mon[27978]: pgmap v283: 129 pgs: 129 active+clean; 2.3 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-21T08:48:06.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:06 smithi043 ceph-mon[27978]: osdmap e101: 12 total, 12 up, 12 in 2024-04-21T08:48:08.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:08 smithi175 ceph-mon[31920]: pgmap v285: 129 pgs: 129 active+clean; 2.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 108 KiB/s wr, 39 op/s 2024-04-21T08:48:08.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:08 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:48:08.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:08 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:48:08.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:08 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:48:08.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:08 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:48:08.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:08 smithi112 ceph-mon[31085]: pgmap v285: 129 pgs: 129 active+clean; 2.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 108 KiB/s wr, 39 op/s 2024-04-21T08:48:08.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:08 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:48:08.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:08 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:48:08.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:08 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:48:08.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:08 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:48:08.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:08 smithi043 ceph-mon[27978]: pgmap v285: 129 pgs: 129 active+clean; 2.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 108 KiB/s wr, 39 op/s 2024-04-21T08:48:08.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:08 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:48:08.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:08 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:48:08.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:08 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:48:08.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:08 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:48:10.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:10 smithi175 ceph-mon[31920]: pgmap v286: 129 pgs: 129 active+clean; 2.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 108 KiB/s wr, 39 op/s 2024-04-21T08:48:10.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:10 smithi175 ceph-mon[31920]: osdmap e102: 12 total, 12 up, 12 in 2024-04-21T08:48:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:10 smithi112 ceph-mon[31085]: pgmap v286: 129 pgs: 129 active+clean; 2.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 108 KiB/s wr, 39 op/s 2024-04-21T08:48:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:10 smithi112 ceph-mon[31085]: osdmap e102: 12 total, 12 up, 12 in 2024-04-21T08:48:10.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:10 smithi043 ceph-mon[27978]: pgmap v286: 129 pgs: 129 active+clean; 2.5 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 108 KiB/s wr, 39 op/s 2024-04-21T08:48:10.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:10 smithi043 ceph-mon[27978]: osdmap e102: 12 total, 12 up, 12 in 2024-04-21T08:48:12.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:12 smithi175 ceph-mon[31920]: pgmap v288: 129 pgs: 129 active+clean; 2.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 134 KiB/s wr, 48 op/s 2024-04-21T08:48:12.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:12 smithi112 ceph-mon[31085]: pgmap v288: 129 pgs: 129 active+clean; 2.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 134 KiB/s wr, 48 op/s 2024-04-21T08:48:12.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:12 smithi043 ceph-mon[27978]: pgmap v288: 129 pgs: 129 active+clean; 2.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 134 KiB/s wr, 48 op/s 2024-04-21T08:48:14.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:14 smithi175 ceph-mon[31920]: pgmap v289: 129 pgs: 129 active+clean; 2.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 134 KiB/s wr, 48 op/s 2024-04-21T08:48:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:14 smithi112 ceph-mon[31085]: pgmap v289: 129 pgs: 129 active+clean; 2.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 134 KiB/s wr, 48 op/s 2024-04-21T08:48:14.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:14 smithi043 ceph-mon[27978]: pgmap v289: 129 pgs: 129 active+clean; 2.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 134 KiB/s wr, 48 op/s 2024-04-21T08:48:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.613+0000 7f68b912b640 1 -- 172.21.15.43:0/2021830841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b4101a70 msgr2=0x7f68b4105ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.613+0000 7f68b912b640 1 --2- 172.21.15.43:0/2021830841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b4101a70 0x7f68b4105ab0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009900 tx=0x7f68a802bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.613+0000 7f68b912b640 1 -- 172.21.15.43:0/2021830841 shutdown_connections 2024-04-21T08:48:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.613+0000 7f68b912b640 1 --2- 172.21.15.43:0/2021830841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68b4106190 0x7f68b410dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.613+0000 7f68b912b640 1 --2- 172.21.15.43:0/2021830841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b4101a70 0x7f68b4105ab0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.613+0000 7f68b912b640 1 --2- 172.21.15.43:0/2021830841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68b41010c0 0x7f68b41014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.613+0000 7f68b912b640 1 -- 172.21.15.43:0/2021830841 >> 172.21.15.43:0/2021830841 conn(0x7f68b4077f60 msgr2=0x7f68b4078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.613+0000 7f68b912b640 1 -- 172.21.15.43:0/2021830841 shutdown_connections 2024-04-21T08:48:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.613+0000 7f68b912b640 1 -- 172.21.15.43:0/2021830841 wait complete. 2024-04-21T08:48:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.614+0000 7f68b912b640 1 Processor -- start 2024-04-21T08:48:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.614+0000 7f68b912b640 1 -- start start 2024-04-21T08:48:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.614+0000 7f68b912b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68b41010c0 0x7f68b4198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.614+0000 7f68b912b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68b4106190 0x7f68b4198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.614+0000 7f68b912b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b419cf20 0x7f68b419d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.614+0000 7f68b912b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f68b4111100 con 0x7f68b419cf20 2024-04-21T08:48:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.614+0000 7f68b912b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f68b4110f80 con 0x7f68b4106190 2024-04-21T08:48:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.614+0000 7f68b912b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68b4111280 con 0x7f68b41010c0 2024-04-21T08:48:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.614+0000 7f68b359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b419cf20 0x7f68b419d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.614+0000 7f68b359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b419cf20 0x7f68b419d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51856/0 (socket says 172.21.15.43:51856) 2024-04-21T08:48:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.614+0000 7f68b359e640 1 -- 172.21.15.43:0/1655374962 learned_addr learned my addr 172.21.15.43:0/1655374962 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.615+0000 7f68b2d9d640 1 --2- 172.21.15.43:0/1655374962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68b41010c0 0x7f68b4198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.615+0000 7f68b259c640 1 --2- 172.21.15.43:0/1655374962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68b4106190 0x7f68b4198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.615+0000 7f68b359e640 1 -- 172.21.15.43:0/1655374962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68b41010c0 msgr2=0x7f68b4198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.615+0000 7f68b359e640 1 --2- 172.21.15.43:0/1655374962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68b41010c0 0x7f68b4198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.615+0000 7f68b359e640 1 -- 172.21.15.43:0/1655374962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68b4106190 msgr2=0x7f68b4198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.615+0000 7f68b359e640 1 --2- 172.21.15.43:0/1655374962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68b4106190 0x7f68b4198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.615+0000 7f68b359e640 1 -- 172.21.15.43:0/1655374962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b419da10 con 0x7f68b419cf20 2024-04-21T08:48:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.615+0000 7f68b359e640 1 --2- 172.21.15.43:0/1655374962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b419cf20 0x7f68b419d3d0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00cce0 tx=0x7f68ac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.615+0000 7f689bfff640 1 -- 172.21.15.43:0/1655374962 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac00f450 con 0x7f68b419cf20 2024-04-21T08:48:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.615+0000 7f689bfff640 1 -- 172.21.15.43:0/1655374962 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68ac00faa0 con 0x7f68b419cf20 2024-04-21T08:48:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.615+0000 7f68b912b640 1 -- 172.21.15.43:0/1655374962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b419dd00 con 0x7f68b419cf20 2024-04-21T08:48:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.615+0000 7f689bfff640 1 -- 172.21.15.43:0/1655374962 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac018b70 con 0x7f68b419cf20 2024-04-21T08:48:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.615+0000 7f68b912b640 1 -- 172.21.15.43:0/1655374962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b41a5470 con 0x7f68b419cf20 2024-04-21T08:48:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.617+0000 7f689bfff640 1 -- 172.21.15.43:0/1655374962 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f68ac018d10 con 0x7f68b419cf20 2024-04-21T08:48:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.618+0000 7f68bbdfa740 1 Processor -- start 2024-04-21T08:48:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.618+0000 7f68bbdfa740 1 -- start start 2024-04-21T08:48:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.618+0000 7f68bbdfa740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ad93878d0 0x556ad9386620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.618+0000 7f68bbdfa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ad9386b60 0x556ad9386fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.618+0000 7f68bbdfa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad93900e0 0x556ad93924d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.618+0000 7f68bbdfa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556ad92eb8d0 con 0x556ad93900e0 2024-04-21T08:48:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.618+0000 7f68bbdfa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556ad9288e80 con 0x556ad9386b60 2024-04-21T08:48:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.618+0000 7f68bbdfa740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556ad92e7890 con 0x556ad93878d0 2024-04-21T08:48:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.618+0000 7f689bfff640 1 --2- 172.21.15.43:0/1655374962 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f689c0a3dc0 0x7f689c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.618+0000 7f68b259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad93900e0 0x556ad93924d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.618+0000 7f68b259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad93900e0 0x556ad93924d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51866/0 (socket says 172.21.15.43:51866) 2024-04-21T08:48:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.618+0000 7f68b259c640 1 -- 172.21.15.43:0/4044255604 learned_addr learned my addr 172.21.15.43:0/4044255604 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.618+0000 7f68b359e640 1 --2- 172.21.15.43:0/1655374962 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f689c0a3dc0 0x7f689c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.618+0000 7f68b259c640 1 --2- 172.21.15.43:0/4044255604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ad93878d0 0x556ad9386620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.618+0000 7f68b2d9d640 1 --2- 172.21.15.43:0/4044255604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ad9386b60 0x556ad9386fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.619+0000 7f689bfff640 1 -- 172.21.15.43:0/1655374962 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f68ac0d1010 con 0x7f68b419cf20 2024-04-21T08:48:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.619+0000 7f68b259c640 1 -- 172.21.15.43:0/4044255604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ad93878d0 msgr2=0x556ad9386620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.619+0000 7f68b259c640 1 --2- 172.21.15.43:0/4044255604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ad93878d0 0x556ad9386620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.619+0000 7f68b259c640 1 -- 172.21.15.43:0/4044255604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ad9386b60 msgr2=0x556ad9386fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.619+0000 7f68b259c640 1 --2- 172.21.15.43:0/4044255604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ad9386b60 0x556ad9386fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.619+0000 7f68b259c640 1 -- 172.21.15.43:0/4044255604 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad92aa320 con 0x556ad93900e0 2024-04-21T08:48:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.619+0000 7f68b2d9d640 1 --2- 172.21.15.43:0/4044255604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ad9386b60 0x556ad9386fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.619+0000 7f68b359e640 1 --2- 172.21.15.43:0/1655374962 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f689c0a3dc0 0x7f689c0a6280 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0ce3f0 tx=0x7f68ac06f680 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.619+0000 7f68b259c640 1 --2- 172.21.15.43:0/4044255604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad93900e0 0x556ad93924d0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f68a802c780 tx=0x7f68a8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9020ba65e8da72de server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.619+0000 7f68997fa640 1 -- 172.21.15.43:0/4044255604 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a804a070 con 0x556ad93900e0 2024-04-21T08:48:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.619+0000 7f68997fa640 1 -- 172.21.15.43:0/4044255604 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68a8005050 con 0x556ad93900e0 2024-04-21T08:48:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.619+0000 7f68997fa640 1 -- 172.21.15.43:0/4044255604 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8004920 con 0x556ad93900e0 2024-04-21T08:48:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.620+0000 7f68bbdfa740 1 -- 172.21.15.43:0/4044255604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad93900e0 msgr2=0x556ad93924d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.620+0000 7f68bbdfa740 1 --2- 172.21.15.43:0/4044255604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad93900e0 0x556ad93924d0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f68a802c780 tx=0x7f68a8005fd0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.620+0000 7f68bbdfa740 1 -- 172.21.15.43:0/4044255604 shutdown_connections 2024-04-21T08:48:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.620+0000 7f68bbdfa740 1 --2- 172.21.15.43:0/4044255604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad93900e0 0x556ad93924d0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.620+0000 7f68bbdfa740 1 --2- 172.21.15.43:0/4044255604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ad9386b60 0x556ad9386fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.620+0000 7f68bbdfa740 1 --2- 172.21.15.43:0/4044255604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ad93878d0 0x556ad9386620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.620+0000 7f68bbdfa740 1 -- 172.21.15.43:0/4044255604 >> 172.21.15.43:0/4044255604 conn(0x556ad934dc90 msgr2=0x556ad9381850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.620+0000 7f68bbdfa740 1 -- 172.21.15.43:0/4044255604 shutdown_connections 2024-04-21T08:48:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.620+0000 7f68bbdfa740 1 -- 172.21.15.43:0/4044255604 wait complete. 2024-04-21T08:48:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.620+0000 7f68bbdfa740 1 -- start start 2024-04-21T08:48:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.620+0000 7f68bbdfa740 1 Processor -- start 2024-04-21T08:48:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.621+0000 7f68bbdfa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad9386b60 0x556ad938fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.621+0000 7f68bbdfa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ad93878d0 0x556ad938cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.621+0000 7f68bbdfa740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ad93900e0 0x556ad938d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.621+0000 7f68bbdfa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556ad9392db0 con 0x556ad9386b60 2024-04-21T08:48:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.621+0000 7f68bbdfa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556ad9273eb0 con 0x556ad93878d0 2024-04-21T08:48:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.621+0000 7f68bbdfa740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556ad9392f30 con 0x556ad93900e0 2024-04-21T08:48:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.621+0000 7f68bbdfa740 10 client.0 _ll_register_callbacks cb 0x556ad92a0b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.621+0000 7f68b259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad9386b60 0x556ad938fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.621+0000 7f68b259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad9386b60 0x556ad938fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51882/0 (socket says 172.21.15.43:51882) 2024-04-21T08:48:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.621+0000 7f68b259c640 1 -- 172.21.15.43:0/784709594 learned_addr learned my addr 172.21.15.43:0/784709594 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.621+0000 7f68b2d9d640 1 --2- 172.21.15.43:0/784709594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ad93878d0 0x556ad938cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.621+0000 7f68b259c640 1 -- 172.21.15.43:0/784709594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ad93900e0 msgr2=0x556ad938d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.621+0000 7f68b259c640 1 --2- 172.21.15.43:0/784709594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ad93900e0 0x556ad938d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.621+0000 7f68b259c640 1 -- 172.21.15.43:0/784709594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ad93878d0 msgr2=0x556ad938cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.621+0000 7f68b259c640 1 --2- 172.21.15.43:0/784709594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ad93878d0 0x556ad938cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.621+0000 7f68b259c640 1 -- 172.21.15.43:0/784709594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad8ffe210 con 0x556ad9386b60 2024-04-21T08:48:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.621+0000 7f68b259c640 1 --2- 172.21.15.43:0/784709594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad9386b60 0x556ad938fb30 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f68a80045d0 tx=0x7f68a802cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40575a12cc58c971 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.621+0000 7f6896ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.622+0000 7f6896ff5640 1 -- 172.21.15.43:0/784709594 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8053060 con 0x556ad9386b60 2024-04-21T08:48:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.622+0000 7f6896ff5640 1 -- 172.21.15.43:0/784709594 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f68a80046c0 con 0x556ad9386b60 2024-04-21T08:48:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.622+0000 7f68bbdfa740 1 -- 172.21.15.43:0/784709594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68a8005050 con 0x556ad9386b60 2024-04-21T08:48:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.622+0000 7f6896ff5640 1 -- 172.21.15.43:0/784709594 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f68a804a040 con 0x556ad9386b60 2024-04-21T08:48:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.622+0000 7f68bbdfa740 10 client.19848 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.622+0000 7f68bbdfa740 1 -- 172.21.15.43:0/784709594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556ad90ca620 con 0x556ad9386b60 2024-04-21T08:48:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.622+0000 7f6896ff5640 1 -- 172.21.15.43:0/784709594 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68a8005050 con 0x556ad9386b60 2024-04-21T08:48:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.622+0000 7f6896ff5640 1 -- 172.21.15.43:0/784709594 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f68a8003a70 con 0x556ad9386b60 2024-04-21T08:48:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.622+0000 7f68bbdfa740 10 client.19848 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.622+0000 7f68bbdfa740 10 client.19848 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.622+0000 7f68bbdfa740 20 client.19848 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.622+0000 7f68bbdfa740 1 --2- 172.21.15.43:0/784709594 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556ad939ffc0 0x556ad93a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.622+0000 7f68bbdfa740 4 client.19848 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.622+0000 7f68bbdfa740 1 -- 172.21.15.43:0/784709594 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ad92ff880 con 0x556ad939ffc0 2024-04-21T08:48:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.623+0000 7f68b2d9d640 1 --2- 172.21.15.43:0/784709594 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556ad939ffc0 0x556ad93a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.623+0000 7f6896ff5640 10 client.19848 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.623+0000 7f68b2d9d640 1 --2- 172.21.15.43:0/784709594 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556ad939ffc0 0x556ad93a23b0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8821c03e1f23a31a server_cookie=c44ac013fde91b9e in_seq=0 out_seq=0 2024-04-21T08:48:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.624+0000 7f6896ff5640 1 -- 172.21.15.43:0/784709594 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ad92ff880 con 0x556ad939ffc0 2024-04-21T08:48:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.624+0000 7f6896ff5640 10 client.19848 handle_command_reply: tid=0 2024-04-21T08:48:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.624+0000 7f68bbdfa740 1 client.19848 shutdown 2024-04-21T08:48:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.624+0000 7f68bbdfa740 1 -- 172.21.15.43:0/784709594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad9386b60 msgr2=0x556ad938fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.624+0000 7f68bbdfa740 1 --2- 172.21.15.43:0/784709594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad9386b60 0x556ad938fb30 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f68a80045d0 tx=0x7f68a802cd90 comp rx=0 tx=0).stop 2024-04-21T08:48:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.624+0000 7f68bbdfa740 1 -- 172.21.15.43:0/784709594 shutdown_connections 2024-04-21T08:48:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.624+0000 7f68bbdfa740 1 --2- 172.21.15.43:0/784709594 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556ad939ffc0 0x556ad93a23b0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.624+0000 7f68bbdfa740 1 --2- 172.21.15.43:0/784709594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ad93900e0 0x556ad938d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.624+0000 7f68bbdfa740 1 --2- 172.21.15.43:0/784709594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ad93878d0 0x556ad938cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.624+0000 7f68bbdfa740 1 --2- 172.21.15.43:0/784709594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad9386b60 0x556ad938fb30 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.624+0000 7f6896ff5640 0 client.19848 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.624+0000 7f68bbdfa740 1 -- 172.21.15.43:0/784709594 >> 172.21.15.43:0/784709594 conn(0x556ad934dc90 msgr2=0x556ad93835a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.624+0000 7f68bbdfa740 1 -- 172.21.15.43:0/784709594 shutdown_connections 2024-04-21T08:48:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.624+0000 7f68bbdfa740 1 -- 172.21.15.43:0/784709594 wait complete. 2024-04-21T08:48:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.624+0000 7f68bbdfa740 20 client.19848 trim_cache size 0 max 16384 2024-04-21T08:48:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.639+0000 7f68bbdfa740 1 Processor -- start 2024-04-21T08:48:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.639+0000 7f68bbdfa740 1 -- start start 2024-04-21T08:48:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.639+0000 7f68bbdfa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad9386b60 0x556ad93a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.639+0000 7f68bbdfa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ad93878d0 0x556ad938bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.639+0000 7f68bbdfa740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ad93900e0 0x556ad938f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.639+0000 7f68bbdfa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41af3b0 con 0x556ad9386b60 2024-04-21T08:48:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.639+0000 7f68bbdfa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41b0ef0 con 0x556ad93878d0 2024-04-21T08:48:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.639+0000 7f68bbdfa740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41afa80 con 0x556ad93900e0 2024-04-21T08:48:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.639+0000 7f68b259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad9386b60 0x556ad93a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.639+0000 7f68b259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad9386b60 0x556ad93a57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51886/0 (socket says 172.21.15.43:51886) 2024-04-21T08:48:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.639+0000 7f68b259c640 1 -- 172.21.15.43:0/2298391160 learned_addr learned my addr 172.21.15.43:0/2298391160 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.640+0000 7f68b2d9d640 1 --2- 172.21.15.43:0/2298391160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ad93878d0 0x556ad938bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.640+0000 7f68b259c640 1 -- 172.21.15.43:0/2298391160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ad93900e0 msgr2=0x556ad938f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.640+0000 7f68b259c640 1 --2- 172.21.15.43:0/2298391160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ad93900e0 0x556ad938f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.640+0000 7f68b259c640 1 -- 172.21.15.43:0/2298391160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ad93878d0 msgr2=0x556ad938bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.640+0000 7f68b259c640 1 --2- 172.21.15.43:0/2298391160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ad93878d0 0x556ad938bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.640+0000 7f68b259c640 1 -- 172.21.15.43:0/2298391160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad937bce0 con 0x556ad9386b60 2024-04-21T08:48:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.640+0000 7f68b259c640 1 --2- 172.21.15.43:0/2298391160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad9386b60 0x556ad93a57c0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f68a802c780 tx=0x7f68a8005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07411f766c2a0d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.640+0000 7f6898ff9640 1 -- 172.21.15.43:0/2298391160 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8053040 con 0x556ad9386b60 2024-04-21T08:48:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.640+0000 7f6898ff9640 1 -- 172.21.15.43:0/2298391160 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68a8003fe0 con 0x556ad9386b60 2024-04-21T08:48:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.640+0000 7f6898ff9640 1 -- 172.21.15.43:0/2298391160 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8003850 con 0x556ad9386b60 2024-04-21T08:48:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.641+0000 7f68bbdfa740 1 -- 172.21.15.43:0/2298391160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad9386b60 msgr2=0x556ad93a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.641+0000 7f68bbdfa740 1 --2- 172.21.15.43:0/2298391160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad9386b60 0x556ad93a57c0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f68a802c780 tx=0x7f68a8005a20 comp rx=0 tx=0).stop 2024-04-21T08:48:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.641+0000 7f68bbdfa740 1 -- 172.21.15.43:0/2298391160 shutdown_connections 2024-04-21T08:48:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.641+0000 7f68bbdfa740 1 --2- 172.21.15.43:0/2298391160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ad93900e0 0x556ad938f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.641+0000 7f68bbdfa740 1 --2- 172.21.15.43:0/2298391160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ad93878d0 0x556ad938bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.641+0000 7f68bbdfa740 1 --2- 172.21.15.43:0/2298391160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad9386b60 0x556ad93a57c0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.641+0000 7f68bbdfa740 1 -- 172.21.15.43:0/2298391160 >> 172.21.15.43:0/2298391160 conn(0x556ad934dc90 msgr2=0x556ad938b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.641+0000 7f68bbdfa740 1 -- 172.21.15.43:0/2298391160 shutdown_connections 2024-04-21T08:48:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.641+0000 7f68bbdfa740 1 -- 172.21.15.43:0/2298391160 wait complete. 2024-04-21T08:48:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.641+0000 7f68bbdfa740 1 -- start start 2024-04-21T08:48:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.641+0000 7f68bbdfa740 1 Processor -- start 2024-04-21T08:48:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.641+0000 7f68bbdfa740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ad9386b60 0x556ad93fe150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.641+0000 7f68bbdfa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ad93878d0 0x556ad93fe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.641+0000 7f68bbdfa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad93900e0 0x556ad93febd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.641+0000 7f68bbdfa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41b0820 con 0x556ad93900e0 2024-04-21T08:48:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.641+0000 7f68bbdfa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41b0150 con 0x556ad93878d0 2024-04-21T08:48:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.641+0000 7f68bbdfa740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41ad3b0 con 0x556ad9386b60 2024-04-21T08:48:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.641+0000 7f68bbdfa740 10 client.0 _ll_register_callbacks cb 0x7f68a8048070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.641+0000 7f68b259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad93900e0 0x556ad93febd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.641+0000 7f68b259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad93900e0 0x556ad93febd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51888/0 (socket says 172.21.15.43:51888) 2024-04-21T08:48:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.641+0000 7f68b259c640 1 -- 172.21.15.43:0/4063749902 learned_addr learned my addr 172.21.15.43:0/4063749902 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.642+0000 7f68b259c640 1 --2- 172.21.15.43:0/4063749902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ad9386b60 0x556ad93fe150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.642+0000 7f68b2d9d640 1 --2- 172.21.15.43:0/4063749902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ad93878d0 0x556ad93fe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.642+0000 7f68b259c640 1 -- 172.21.15.43:0/4063749902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ad9386b60 msgr2=0x556ad93fe150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.642+0000 7f68b259c640 1 --2- 172.21.15.43:0/4063749902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ad9386b60 0x556ad93fe150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.642+0000 7f68b259c640 1 -- 172.21.15.43:0/4063749902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ad93878d0 msgr2=0x556ad93fe690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.642+0000 7f68b259c640 1 --2- 172.21.15.43:0/4063749902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ad93878d0 0x556ad93fe690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.642+0000 7f68b259c640 1 -- 172.21.15.43:0/4063749902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad9385e30 con 0x556ad93900e0 2024-04-21T08:48:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.642+0000 7f68b259c640 1 --2- 172.21.15.43:0/4063749902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad93900e0 0x556ad93febd0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f68a802cdd0 tx=0x7f68a8009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6058fc87e2f400bd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.642+0000 7f6896ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.642+0000 7f6896ff5640 1 -- 172.21.15.43:0/4063749902 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a806c060 con 0x556ad93900e0 2024-04-21T08:48:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.642+0000 7f6896ff5640 1 -- 172.21.15.43:0/4063749902 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f68a80035f0 con 0x556ad93900e0 2024-04-21T08:48:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.642+0000 7f68bbdfa740 1 -- 172.21.15.43:0/4063749902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68a8003fe0 con 0x556ad93900e0 2024-04-21T08:48:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.642+0000 7f6896ff5640 1 -- 172.21.15.43:0/4063749902 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f68a8053040 con 0x556ad93900e0 2024-04-21T08:48:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.642+0000 7f68bbdfa740 10 client.19854 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.642+0000 7f68bbdfa740 1 -- 172.21.15.43:0/4063749902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556ad940a3d0 con 0x556ad93900e0 2024-04-21T08:48:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.642+0000 7f6896ff5640 1 -- 172.21.15.43:0/4063749902 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68a8003fe0 con 0x556ad93900e0 2024-04-21T08:48:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.643+0000 7f6896ff5640 1 -- 172.21.15.43:0/4063749902 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f68a8004c30 con 0x556ad93900e0 2024-04-21T08:48:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.643+0000 7f68bbdfa740 10 client.19854 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.643+0000 7f68bbdfa740 10 client.19854 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.643+0000 7f68bbdfa740 20 client.19854 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.643+0000 7f68bbdfa740 1 --2- 172.21.15.43:0/4063749902 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556ad940a850 0x556ad940cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.643+0000 7f68bbdfa740 4 client.19854 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:48:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.643+0000 7f68bbdfa740 1 -- 172.21.15.43:0/4063749902 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556ad928b3d0 con 0x556ad940a850 2024-04-21T08:48:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.643+0000 7f68b2d9d640 1 --2- 172.21.15.43:0/4063749902 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556ad940a850 0x556ad940cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.643+0000 7f68b2d9d640 1 --2- 172.21.15.43:0/4063749902 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556ad940a850 0x556ad940cc40 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f77721c4febb61eb server_cookie=2ea45fc69886f605 in_seq=0 out_seq=0 2024-04-21T08:48:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.643+0000 7f6896ff5640 10 client.19854 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.659+0000 7f6896ff5640 1 -- 172.21.15.43:0/4063749902 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556ad928b3d0 con 0x556ad940a850 2024-04-21T08:48:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.659+0000 7f6896ff5640 10 client.19854 handle_command_reply: tid=0 2024-04-21T08:48:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.659+0000 7f68bbdfa740 1 client.19854 shutdown 2024-04-21T08:48:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.659+0000 7f68bbdfa740 1 -- 172.21.15.43:0/4063749902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad93900e0 msgr2=0x556ad93febd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.659+0000 7f68bbdfa740 1 --2- 172.21.15.43:0/4063749902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad93900e0 0x556ad93febd0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f68a802cdd0 tx=0x7f68a8009640 comp rx=0 tx=0).stop 2024-04-21T08:48:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.659+0000 7f68bbdfa740 1 -- 172.21.15.43:0/4063749902 shutdown_connections 2024-04-21T08:48:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.659+0000 7f68bbdfa740 1 --2- 172.21.15.43:0/4063749902 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556ad940a850 0x556ad940cc40 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.659+0000 7f68bbdfa740 1 --2- 172.21.15.43:0/4063749902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556ad93900e0 0x556ad93febd0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.659+0000 7f68bbdfa740 1 --2- 172.21.15.43:0/4063749902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ad93878d0 0x556ad93fe690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.659+0000 7f6896ff5640 0 client.19854 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.659+0000 7f68bbdfa740 1 --2- 172.21.15.43:0/4063749902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556ad9386b60 0x556ad93fe150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.659+0000 7f68bbdfa740 1 -- 172.21.15.43:0/4063749902 >> 172.21.15.43:0/4063749902 conn(0x556ad934dc90 msgr2=0x556ad939b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.659+0000 7f68bbdfa740 1 -- 172.21.15.43:0/4063749902 shutdown_connections 2024-04-21T08:48:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.659+0000 7f68bbdfa740 1 -- 172.21.15.43:0/4063749902 wait complete. 2024-04-21T08:48:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.659+0000 7f68bbdfa740 20 client.19854 trim_cache size 0 max 16384 2024-04-21T08:48:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:48:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:48:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:48:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:48:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.659+0000 7f68b912b640 1 -- 172.21.15.43:0/1655374962 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f689c0a3dc0 msgr2=0x7f689c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.660+0000 7f68b912b640 1 --2- 172.21.15.43:0/1655374962 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f689c0a3dc0 0x7f689c0a6280 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0ce3f0 tx=0x7f68ac06f680 comp rx=0 tx=0).stop 2024-04-21T08:48:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.660+0000 7f68b912b640 1 -- 172.21.15.43:0/1655374962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b419cf20 msgr2=0x7f68b419d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.660+0000 7f68b912b640 1 --2- 172.21.15.43:0/1655374962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b419cf20 0x7f68b419d3d0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00cce0 tx=0x7f68ac00a620 comp rx=0 tx=0).stop 2024-04-21T08:48:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.660+0000 7f68b912b640 1 -- 172.21.15.43:0/1655374962 shutdown_connections 2024-04-21T08:48:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.660+0000 7f68b912b640 1 --2- 172.21.15.43:0/1655374962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b419cf20 0x7f68b419d3d0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.660+0000 7f68b912b640 1 --2- 172.21.15.43:0/1655374962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68b4106190 0x7f68b4198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.660+0000 7f68b912b640 1 --2- 172.21.15.43:0/1655374962 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f689c0a3dc0 0x7f689c0a6280 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.660+0000 7f68b912b640 1 --2- 172.21.15.43:0/1655374962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68b41010c0 0x7f68b4198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.660+0000 7f68b912b640 1 -- 172.21.15.43:0/1655374962 >> 172.21.15.43:0/1655374962 conn(0x7f68b4077f60 msgr2=0x7f68b406a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.660+0000 7f68b912b640 1 -- 172.21.15.43:0/1655374962 shutdown_connections 2024-04-21T08:48:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:15.660+0000 7f68b912b640 1 -- 172.21.15.43:0/1655374962 wait complete. 2024-04-21T08:48:16.294 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:16 smithi112 ceph-mon[31085]: pgmap v290: 129 pgs: 129 active+clean; 2.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 108 KiB/s wr, 38 op/s 2024-04-21T08:48:16.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:16 smithi043 ceph-mon[27978]: pgmap v290: 129 pgs: 129 active+clean; 2.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 108 KiB/s wr, 38 op/s 2024-04-21T08:48:16.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:16 smithi175 ceph-mon[31920]: pgmap v290: 129 pgs: 129 active+clean; 2.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 108 KiB/s wr, 38 op/s 2024-04-21T08:48:18.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:18 smithi175 ceph-mon[31920]: pgmap v291: 129 pgs: 129 active+clean; 3.0 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-04-21T08:48:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:18 smithi112 ceph-mon[31085]: pgmap v291: 129 pgs: 129 active+clean; 3.0 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-04-21T08:48:18.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:18 smithi043 ceph-mon[27978]: pgmap v291: 129 pgs: 129 active+clean; 3.0 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-04-21T08:48:20.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:20 smithi175 ceph-mon[31920]: pgmap v292: 129 pgs: 129 active+clean; 3.0 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-04-21T08:48:20.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:20 smithi112 ceph-mon[31085]: pgmap v292: 129 pgs: 129 active+clean; 3.0 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-04-21T08:48:20.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:20 smithi043 ceph-mon[27978]: pgmap v292: 129 pgs: 129 active+clean; 3.0 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-04-21T08:48:21.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:48:21.761 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:48:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.880+0000 7f0ed9843640 1 -- 172.21.15.43:0/3438308975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ed41072c0 msgr2=0x7f0ed410ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.880+0000 7f0ed9843640 1 --2- 172.21.15.43:0/3438308975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ed41072c0 0x7f0ed410ee70 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009900 tx=0x7f0ecc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.880+0000 7f0ed9843640 1 -- 172.21.15.43:0/3438308975 shutdown_connections 2024-04-21T08:48:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.880+0000 7f0ed9843640 1 --2- 172.21.15.43:0/3438308975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ed41072c0 0x7f0ed410ee70 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.880+0000 7f0ed9843640 1 --2- 172.21.15.43:0/3438308975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ed406b2d0 0x7f0ed4106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.880+0000 7f0ed9843640 1 --2- 172.21.15.43:0/3438308975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ed406a920 0x7f0ed406ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.880+0000 7f0ed9843640 1 -- 172.21.15.43:0/3438308975 >> 172.21.15.43:0/3438308975 conn(0x7f0ed40771d0 msgr2=0x7f0ed40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.880+0000 7f0ed9843640 1 -- 172.21.15.43:0/3438308975 shutdown_connections 2024-04-21T08:48:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.880+0000 7f0ed9843640 1 -- 172.21.15.43:0/3438308975 wait complete. 2024-04-21T08:48:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.880+0000 7f0ed9843640 1 Processor -- start 2024-04-21T08:48:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.881+0000 7f0ed9843640 1 -- start start 2024-04-21T08:48:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.881+0000 7f0ed9843640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ed406a920 0x7f0ed4198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.881+0000 7f0ed9843640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ed41072c0 0x7f0ed4198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.881+0000 7f0ed9843640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ed419cf30 0x7f0ed419d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.881+0000 7f0ed9843640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed41121b0 con 0x7f0ed41072c0 2024-04-21T08:48:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.881+0000 7f0ed9843640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed4112030 con 0x7f0ed406a920 2024-04-21T08:48:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.881+0000 7f0ed9843640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed4112330 con 0x7f0ed419cf30 2024-04-21T08:48:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.881+0000 7f0ed27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ed41072c0 0x7f0ed4198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.881+0000 7f0ed27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ed41072c0 0x7f0ed4198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36322/0 (socket says 172.21.15.43:36322) 2024-04-21T08:48:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.881+0000 7f0ed27fc640 1 -- 172.21.15.43:0/3278927527 learned_addr learned my addr 172.21.15.43:0/3278927527 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.881+0000 7f0ed37fe640 1 --2- 172.21.15.43:0/3278927527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ed419cf30 0x7f0ed419d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.881+0000 7f0ed2ffd640 1 --2- 172.21.15.43:0/3278927527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ed406a920 0x7f0ed4198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.881+0000 7f0ed27fc640 1 -- 172.21.15.43:0/3278927527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ed419cf30 msgr2=0x7f0ed419d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.881+0000 7f0ed27fc640 1 --2- 172.21.15.43:0/3278927527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ed419cf30 0x7f0ed419d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.881+0000 7f0ed27fc640 1 -- 172.21.15.43:0/3278927527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ed406a920 msgr2=0x7f0ed4198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.881+0000 7f0ed27fc640 1 --2- 172.21.15.43:0/3278927527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ed406a920 0x7f0ed4198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.881+0000 7f0ed27fc640 1 -- 172.21.15.43:0/3278927527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed419da20 con 0x7f0ed41072c0 2024-04-21T08:48:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.882+0000 7f0ed27fc640 1 --2- 172.21.15.43:0/3278927527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ed41072c0 0x7f0ed4198bb0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800c970 tx=0x7f0ec800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.882+0000 7f0ed8841640 1 -- 172.21.15.43:0/3278927527 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8007bb0 con 0x7f0ed41072c0 2024-04-21T08:48:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.882+0000 7f0ed8841640 1 -- 172.21.15.43:0/3278927527 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ec8007d50 con 0x7f0ed41072c0 2024-04-21T08:48:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.882+0000 7f0ed9843640 1 -- 172.21.15.43:0/3278927527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed419dd10 con 0x7f0ed41072c0 2024-04-21T08:48:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.882+0000 7f0ed8841640 1 -- 172.21.15.43:0/3278927527 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec800f450 con 0x7f0ed41072c0 2024-04-21T08:48:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.882+0000 7f0ed9843640 1 -- 172.21.15.43:0/3278927527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed41a5500 con 0x7f0ed41072c0 2024-04-21T08:48:21.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.885+0000 7f0edc43c740 1 Processor -- start 2024-04-21T08:48:21.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.885+0000 7f0ed8841640 1 -- 172.21.15.43:0/3278927527 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0ec8004eb0 con 0x7f0ed41072c0 2024-04-21T08:48:21.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.885+0000 7f0edc43c740 1 -- start start 2024-04-21T08:48:21.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.885+0000 7f0edc43c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a3fe580 0x559d6a3fe960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:21.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.885+0000 7f0edc43c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d6a3feea0 0x559d6a407540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:21.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.885+0000 7f0edc43c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d6a407a80 0x559d6a409e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:21.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.885+0000 7f0edc43c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559d6a3631e0 con 0x559d6a3fe580 2024-04-21T08:48:21.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.885+0000 7f0edc43c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559d6a300390 con 0x559d6a3feea0 2024-04-21T08:48:21.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.885+0000 7f0edc43c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559d6a35f1a0 con 0x559d6a407a80 2024-04-21T08:48:21.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.885+0000 7f0ed37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a3fe580 0x559d6a3fe960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:21.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.886+0000 7f0ed37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a3fe580 0x559d6a3fe960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36326/0 (socket says 172.21.15.43:36326) 2024-04-21T08:48:21.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.886+0000 7f0ed37fe640 1 -- 172.21.15.43:0/3380209774 learned_addr learned my addr 172.21.15.43:0/3380209774 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:21.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.886+0000 7f0ed2ffd640 1 --2- 172.21.15.43:0/3380209774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d6a3feea0 0x559d6a407540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:21.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.886+0000 7f0ed27fc640 1 --2- 172.21.15.43:0/3380209774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d6a407a80 0x559d6a409e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:21.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.886+0000 7f0ed8841640 1 --2- 172.21.15.43:0/3278927527 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0ebc0a3d20 0x7f0ebc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:21.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.886+0000 7f0ed37fe640 1 -- 172.21.15.43:0/3380209774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d6a407a80 msgr2=0x559d6a409e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:21.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.886+0000 7f0ed37fe640 1 --2- 172.21.15.43:0/3380209774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d6a407a80 0x559d6a409e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.886+0000 7f0ed37fe640 1 -- 172.21.15.43:0/3380209774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d6a3feea0 msgr2=0x559d6a407540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:21.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.886+0000 7f0ed37fe640 1 --2- 172.21.15.43:0/3380209774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d6a3feea0 0x559d6a407540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.886+0000 7f0ed37fe640 1 -- 172.21.15.43:0/3380209774 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d6a13f7d0 con 0x559d6a3fe580 2024-04-21T08:48:21.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.886+0000 7f0ed2ffd640 1 --2- 172.21.15.43:0/3380209774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d6a3feea0 0x559d6a407540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:21.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.886+0000 7f0ed37fe640 1 --2- 172.21.15.43:0/3278927527 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0ebc0a3d20 0x7f0ebc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:21.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.886+0000 7f0ed8841640 1 -- 172.21.15.43:0/3278927527 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f0ec80d08c0 con 0x7f0ed41072c0 2024-04-21T08:48:21.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.886+0000 7f0ed37fe640 1 --2- 172.21.15.43:0/3380209774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a3fe580 0x559d6a3fe960 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009900 tx=0x7f0ecc02c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9412f9950335c780 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:21.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.886+0000 7f0eb9ffb640 1 -- 172.21.15.43:0/3380209774 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc046070 con 0x559d6a3fe580 2024-04-21T08:48:21.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.886+0000 7f0eb9ffb640 1 -- 172.21.15.43:0/3380209774 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ecc0054f0 con 0x559d6a3fe580 2024-04-21T08:48:21.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.886+0000 7f0eb9ffb640 1 -- 172.21.15.43:0/3380209774 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc0035f0 con 0x559d6a3fe580 2024-04-21T08:48:21.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.887+0000 7f0edc43c740 1 -- 172.21.15.43:0/3380209774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a3fe580 msgr2=0x559d6a3fe960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:21.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.887+0000 7f0ed37fe640 1 --2- 172.21.15.43:0/3278927527 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0ebc0a3d20 0x7f0ebc0a61e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc04c000 tx=0x7f0ecc049000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:21.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.887+0000 7f0edc43c740 1 --2- 172.21.15.43:0/3380209774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a3fe580 0x559d6a3fe960 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009900 tx=0x7f0ecc02c700 comp rx=0 tx=0).stop 2024-04-21T08:48:21.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.887+0000 7f0edc43c740 1 -- 172.21.15.43:0/3380209774 shutdown_connections 2024-04-21T08:48:21.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.887+0000 7f0edc43c740 1 --2- 172.21.15.43:0/3380209774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d6a407a80 0x559d6a409e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.887+0000 7f0edc43c740 1 --2- 172.21.15.43:0/3380209774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d6a3feea0 0x559d6a407540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.887+0000 7f0edc43c740 1 --2- 172.21.15.43:0/3380209774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a3fe580 0x559d6a3fe960 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.887+0000 7f0edc43c740 1 -- 172.21.15.43:0/3380209774 >> 172.21.15.43:0/3380209774 conn(0x559d6a3f8cb0 msgr2=0x559d6a3f9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:21.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.887+0000 7f0edc43c740 1 -- 172.21.15.43:0/3380209774 shutdown_connections 2024-04-21T08:48:21.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.887+0000 7f0edc43c740 1 -- 172.21.15.43:0/3380209774 wait complete. 2024-04-21T08:48:21.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.887+0000 7f0edc43c740 1 -- start start 2024-04-21T08:48:21.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.888+0000 7f0edc43c740 1 Processor -- start 2024-04-21T08:48:21.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.888+0000 7f0edc43c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d6a3fe580 0x559d6a404b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:21.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.888+0000 7f0edc43c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a3feea0 0x559d6a405080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:21.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.888+0000 7f0edc43c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d6a407a80 0x559d6a4055c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:21.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.888+0000 7f0edc43c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559d6a40a870 con 0x559d6a3feea0 2024-04-21T08:48:21.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.888+0000 7f0edc43c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559d6a31c2f0 con 0x559d6a3fe580 2024-04-21T08:48:21.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.888+0000 7f0edc43c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559d6a40a9f0 con 0x559d6a407a80 2024-04-21T08:48:21.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.888+0000 7f0edc43c740 10 client.0 _ll_register_callbacks cb 0x559d6a329580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:21.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.888+0000 7f0ed27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a3feea0 0x559d6a405080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:21.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.888+0000 7f0ed27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a3feea0 0x559d6a405080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36342/0 (socket says 172.21.15.43:36342) 2024-04-21T08:48:21.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.888+0000 7f0ed27fc640 1 -- 172.21.15.43:0/173587975 learned_addr learned my addr 172.21.15.43:0/173587975 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:21.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.888+0000 7f0ed2ffd640 1 --2- 172.21.15.43:0/173587975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d6a3fe580 0x559d6a404b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:21.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.888+0000 7f0ed37fe640 1 --2- 172.21.15.43:0/173587975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d6a407a80 0x559d6a4055c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:21.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.889+0000 7f0ed27fc640 1 -- 172.21.15.43:0/173587975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d6a407a80 msgr2=0x559d6a4055c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:21.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.889+0000 7f0ed27fc640 1 --2- 172.21.15.43:0/173587975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d6a407a80 0x559d6a4055c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.889+0000 7f0ed27fc640 1 -- 172.21.15.43:0/173587975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d6a3fe580 msgr2=0x559d6a404b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:21.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.889+0000 7f0ed27fc640 1 --2- 172.21.15.43:0/173587975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d6a3fe580 0x559d6a404b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.889+0000 7f0ed27fc640 1 -- 172.21.15.43:0/173587975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d6a0753f0 con 0x559d6a3feea0 2024-04-21T08:48:21.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.889+0000 7f0ed2ffd640 1 --2- 172.21.15.43:0/173587975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d6a3fe580 0x559d6a404b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:21.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.889+0000 7f0ed27fc640 1 --2- 172.21.15.43:0/173587975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a3feea0 0x559d6a405080 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800c940 tx=0x7f0ec8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb7b1a5ed0510ef server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:21.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.889+0000 7f0eb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:21.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.889+0000 7f0eb2ffd640 1 -- 172.21.15.43:0/173587975 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec801d040 con 0x559d6a3feea0 2024-04-21T08:48:21.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.889+0000 7f0eb2ffd640 1 -- 172.21.15.43:0/173587975 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f0ec8098b60 con 0x559d6a3feea0 2024-04-21T08:48:21.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.889+0000 7f0edc43c740 1 -- 172.21.15.43:0/173587975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ecc0054f0 con 0x559d6a3feea0 2024-04-21T08:48:21.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.889+0000 7f0eb2ffd640 1 -- 172.21.15.43:0/173587975 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0ec80a07f0 con 0x559d6a3feea0 2024-04-21T08:48:21.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.889+0000 7f0edc43c740 10 client.19866 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:21.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.889+0000 7f0edc43c740 1 -- 172.21.15.43:0/173587975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559d6a0350f0 con 0x559d6a3feea0 2024-04-21T08:48:21.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.889+0000 7f0eb2ffd640 1 -- 172.21.15.43:0/173587975 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ecc0054f0 con 0x559d6a3feea0 2024-04-21T08:48:21.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.890+0000 7f0eb2ffd640 1 -- 172.21.15.43:0/173587975 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0ec80a4c20 con 0x559d6a3feea0 2024-04-21T08:48:21.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.890+0000 7f0edc43c740 10 client.19866 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:21.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.890+0000 7f0edc43c740 10 client.19866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:21.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.890+0000 7f0edc43c740 20 client.19866 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:21.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.890+0000 7f0edc43c740 1 --2- 172.21.15.43:0/173587975 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559d6a417b50 0x559d6a419f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:21.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.890+0000 7f0edc43c740 4 client.19866 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:21.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.890+0000 7f0edc43c740 1 -- 172.21.15.43:0/173587975 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d6a370c00 con 0x559d6a417b50 2024-04-21T08:48:21.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.890+0000 7f0ed2ffd640 1 --2- 172.21.15.43:0/173587975 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559d6a417b50 0x559d6a419f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:21.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.891+0000 7f0ed2ffd640 1 --2- 172.21.15.43:0/173587975 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559d6a417b50 0x559d6a419f40 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fa067fe79971639 server_cookie=9bf9307719d79494 in_seq=0 out_seq=0 2024-04-21T08:48:21.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.891+0000 7f0eb2ffd640 10 client.19866 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.891+0000 7f0eb2ffd640 1 -- 172.21.15.43:0/173587975 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559d6a370c00 con 0x559d6a417b50 2024-04-21T08:48:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.891+0000 7f0eb2ffd640 10 client.19866 handle_command_reply: tid=0 2024-04-21T08:48:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.891+0000 7f0edc43c740 1 client.19866 shutdown 2024-04-21T08:48:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.891+0000 7f0edc43c740 1 -- 172.21.15.43:0/173587975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a3feea0 msgr2=0x559d6a405080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.892+0000 7f0edc43c740 1 --2- 172.21.15.43:0/173587975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a3feea0 0x559d6a405080 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800c940 tx=0x7f0ec8000bc0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.892+0000 7f0edc43c740 1 -- 172.21.15.43:0/173587975 shutdown_connections 2024-04-21T08:48:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.892+0000 7f0edc43c740 1 --2- 172.21.15.43:0/173587975 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559d6a417b50 0x559d6a419f40 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.892+0000 7f0edc43c740 1 --2- 172.21.15.43:0/173587975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d6a407a80 0x559d6a4055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.892+0000 7f0eb2ffd640 0 client.19866 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.892+0000 7f0edc43c740 1 --2- 172.21.15.43:0/173587975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a3feea0 0x559d6a405080 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.892+0000 7f0edc43c740 1 --2- 172.21.15.43:0/173587975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d6a3fe580 0x559d6a404b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.892+0000 7f0edc43c740 1 -- 172.21.15.43:0/173587975 >> 172.21.15.43:0/173587975 conn(0x559d6a3f8cb0 msgr2=0x559d6a3faa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:21.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.892+0000 7f0edc43c740 1 -- 172.21.15.43:0/173587975 shutdown_connections 2024-04-21T08:48:21.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.892+0000 7f0edc43c740 1 -- 172.21.15.43:0/173587975 wait complete. 2024-04-21T08:48:21.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.892+0000 7f0edc43c740 20 client.19866 trim_cache size 0 max 16384 2024-04-21T08:48:21.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.909+0000 7f0edc43c740 1 Processor -- start 2024-04-21T08:48:21.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.909+0000 7f0edc43c740 1 -- start start 2024-04-21T08:48:21.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.910+0000 7f0edc43c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a3fe580 0x559d6a407270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:21.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.910+0000 7f0edc43c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d6a3feea0 0x559d6a403a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.910+0000 7f0edc43c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d6a407a80 0x559d6a413730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.910+0000 7f0edc43c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed41af3c0 con 0x559d6a3fe580 2024-04-21T08:48:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.910+0000 7f0edc43c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed41b0f00 con 0x559d6a3feea0 2024-04-21T08:48:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.910+0000 7f0edc43c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed41afa90 con 0x559d6a407a80 2024-04-21T08:48:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.910+0000 7f0ed2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a3fe580 0x559d6a407270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.910+0000 7f0ed2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a3fe580 0x559d6a407270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36358/0 (socket says 172.21.15.43:36358) 2024-04-21T08:48:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.910+0000 7f0ed2ffd640 1 -- 172.21.15.43:0/430750578 learned_addr learned my addr 172.21.15.43:0/430750578 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.910+0000 7f0ed37fe640 1 --2- 172.21.15.43:0/430750578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d6a407a80 0x559d6a413730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.910+0000 7f0ed2ffd640 1 -- 172.21.15.43:0/430750578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d6a407a80 msgr2=0x559d6a413730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.910+0000 7f0ed2ffd640 1 --2- 172.21.15.43:0/430750578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d6a407a80 0x559d6a413730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.910+0000 7f0ed2ffd640 1 -- 172.21.15.43:0/430750578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d6a3feea0 msgr2=0x559d6a403a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.910+0000 7f0ed27fc640 1 --2- 172.21.15.43:0/430750578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d6a3feea0 0x559d6a403a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.910+0000 7f0ed2ffd640 1 --2- 172.21.15.43:0/430750578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d6a3feea0 0x559d6a403a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.910+0000 7f0ed2ffd640 1 -- 172.21.15.43:0/430750578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d6a41a2b0 con 0x559d6a3fe580 2024-04-21T08:48:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.910+0000 7f0ed37fe640 1 --2- 172.21.15.43:0/430750578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d6a407a80 0x559d6a413730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.910+0000 7f0ed27fc640 1 --2- 172.21.15.43:0/430750578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d6a3feea0 0x559d6a403a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.910+0000 7f0ed2ffd640 1 --2- 172.21.15.43:0/430750578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a3fe580 0x559d6a407270 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x559d6a41a6c0 tx=0x7f0ec4004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5da072aa2b4a01a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.910+0000 7f0eb97fa640 1 -- 172.21.15.43:0/430750578 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec401e250 con 0x559d6a3fe580 2024-04-21T08:48:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.911+0000 7f0eb97fa640 1 -- 172.21.15.43:0/430750578 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ec401e8a0 con 0x559d6a3fe580 2024-04-21T08:48:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.911+0000 7f0eb97fa640 1 -- 172.21.15.43:0/430750578 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4012e30 con 0x559d6a3fe580 2024-04-21T08:48:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.911+0000 7f0edc43c740 1 -- 172.21.15.43:0/430750578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a3fe580 msgr2=0x559d6a407270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.911+0000 7f0edc43c740 1 --2- 172.21.15.43:0/430750578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a3fe580 0x559d6a407270 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x559d6a41a6c0 tx=0x7f0ec4004360 comp rx=0 tx=0).stop 2024-04-21T08:48:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.911+0000 7f0edc43c740 1 -- 172.21.15.43:0/430750578 shutdown_connections 2024-04-21T08:48:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.911+0000 7f0edc43c740 1 --2- 172.21.15.43:0/430750578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d6a407a80 0x559d6a413730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.911+0000 7f0edc43c740 1 --2- 172.21.15.43:0/430750578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d6a3feea0 0x559d6a403a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.911+0000 7f0edc43c740 1 --2- 172.21.15.43:0/430750578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a3fe580 0x559d6a407270 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.911+0000 7f0edc43c740 1 -- 172.21.15.43:0/430750578 >> 172.21.15.43:0/430750578 conn(0x559d6a3f8cb0 msgr2=0x559d6a3f7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:21.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.911+0000 7f0edc43c740 1 -- 172.21.15.43:0/430750578 shutdown_connections 2024-04-21T08:48:21.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.911+0000 7f0edc43c740 1 -- 172.21.15.43:0/430750578 wait complete. 2024-04-21T08:48:21.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.911+0000 7f0edc43c740 1 -- start start 2024-04-21T08:48:21.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.912+0000 7f0edc43c740 1 Processor -- start 2024-04-21T08:48:21.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.912+0000 7f0edc43c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d6a3fe580 0x559d6a4762c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:21.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.912+0000 7f0edc43c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d6a3feea0 0x559d6a476800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:21.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.912+0000 7f0edc43c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a407a80 0x559d6a40bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:21.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.912+0000 7f0edc43c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed41b0830 con 0x559d6a407a80 2024-04-21T08:48:21.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.912+0000 7f0edc43c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed41b0160 con 0x559d6a3feea0 2024-04-21T08:48:21.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.912+0000 7f0edc43c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed41ad3c0 con 0x559d6a3fe580 2024-04-21T08:48:21.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.912+0000 7f0edc43c740 10 client.0 _ll_register_callbacks cb 0x7f0ec809c3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:21.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.912+0000 7f0ed37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a407a80 0x559d6a40bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:21.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.912+0000 7f0ed37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a407a80 0x559d6a40bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36366/0 (socket says 172.21.15.43:36366) 2024-04-21T08:48:21.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.912+0000 7f0ed37fe640 1 -- 172.21.15.43:0/1496607728 learned_addr learned my addr 172.21.15.43:0/1496607728 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:21.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.912+0000 7f0ed27fc640 1 --2- 172.21.15.43:0/1496607728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d6a3feea0 0x559d6a476800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:21.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.912+0000 7f0ed2ffd640 1 --2- 172.21.15.43:0/1496607728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d6a3fe580 0x559d6a4762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:21.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.912+0000 7f0ed37fe640 1 -- 172.21.15.43:0/1496607728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d6a3fe580 msgr2=0x559d6a4762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:21.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.912+0000 7f0ed37fe640 1 --2- 172.21.15.43:0/1496607728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d6a3fe580 0x559d6a4762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.912+0000 7f0ed37fe640 1 -- 172.21.15.43:0/1496607728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d6a3feea0 msgr2=0x559d6a476800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:21.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.912+0000 7f0ed37fe640 1 --2- 172.21.15.43:0/1496607728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d6a3feea0 0x559d6a476800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.912+0000 7f0ed37fe640 1 -- 172.21.15.43:0/1496607728 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d6a40c630 con 0x559d6a407a80 2024-04-21T08:48:21.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.913+0000 7f0ed37fe640 1 --2- 172.21.15.43:0/1496607728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a407a80 0x559d6a40bf40 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc036040 tx=0x7f0ecc003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efafcaa43da6558 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:21.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.913+0000 7f0eb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:21.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.913+0000 7f0eb2ffd640 1 -- 172.21.15.43:0/1496607728 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc046040 con 0x559d6a407a80 2024-04-21T08:48:21.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.913+0000 7f0eb2ffd640 1 -- 172.21.15.43:0/1496607728 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f0ecc005000 con 0x559d6a407a80 2024-04-21T08:48:21.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.913+0000 7f0edc43c740 1 -- 172.21.15.43:0/1496607728 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ec401e8a0 con 0x559d6a407a80 2024-04-21T08:48:21.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.913+0000 7f0eb2ffd640 1 -- 172.21.15.43:0/1496607728 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0ecc064050 con 0x559d6a407a80 2024-04-21T08:48:21.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.913+0000 7f0edc43c740 10 client.19872 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:21.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.913+0000 7f0edc43c740 1 -- 172.21.15.43:0/1496607728 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559d6a47eb10 con 0x559d6a407a80 2024-04-21T08:48:21.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.913+0000 7f0eb2ffd640 1 -- 172.21.15.43:0/1496607728 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ec401e8a0 con 0x559d6a407a80 2024-04-21T08:48:21.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.913+0000 7f0eb2ffd640 1 -- 172.21.15.43:0/1496607728 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0ecc004aa0 con 0x559d6a407a80 2024-04-21T08:48:21.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.913+0000 7f0edc43c740 10 client.19872 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:21.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.913+0000 7f0edc43c740 10 client.19872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:21.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.913+0000 7f0edc43c740 20 client.19872 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:21.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.913+0000 7f0edc43c740 1 --2- 172.21.15.43:0/1496607728 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559d6a47ee10 0x559d6a481230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:21.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.914+0000 7f0edc43c740 4 client.19872 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:48:21.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.914+0000 7f0edc43c740 1 -- 172.21.15.43:0/1496607728 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559d6a36aa70 con 0x559d6a47ee10 2024-04-21T08:48:21.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.914+0000 7f0ed2ffd640 1 --2- 172.21.15.43:0/1496607728 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559d6a47ee10 0x559d6a481230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:21.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.914+0000 7f0eb2ffd640 10 client.19872 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:21.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.914+0000 7f0ed2ffd640 1 --2- 172.21.15.43:0/1496607728 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559d6a47ee10 0x559d6a481230 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af7395b1dd061c9d server_cookie=6094717fe0feb8aa in_seq=0 out_seq=0 2024-04-21T08:48:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.914+0000 7f0eb2ffd640 1 -- 172.21.15.43:0/1496607728 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559d6a36aa70 con 0x559d6a47ee10 2024-04-21T08:48:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.914+0000 7f0eb2ffd640 10 client.19872 handle_command_reply: tid=0 2024-04-21T08:48:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.914+0000 7f0edc43c740 1 client.19872 shutdown 2024-04-21T08:48:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.915+0000 7f0edc43c740 1 -- 172.21.15.43:0/1496607728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a407a80 msgr2=0x559d6a40bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.915+0000 7f0edc43c740 1 --2- 172.21.15.43:0/1496607728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a407a80 0x559d6a40bf40 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc036040 tx=0x7f0ecc003a80 comp rx=0 tx=0).stop 2024-04-21T08:48:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.915+0000 7f0edc43c740 1 -- 172.21.15.43:0/1496607728 shutdown_connections 2024-04-21T08:48:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.915+0000 7f0edc43c740 1 --2- 172.21.15.43:0/1496607728 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559d6a47ee10 0x559d6a481230 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.915+0000 7f0edc43c740 1 --2- 172.21.15.43:0/1496607728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559d6a407a80 0x559d6a40bf40 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.915+0000 7f0edc43c740 1 --2- 172.21.15.43:0/1496607728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d6a3feea0 0x559d6a476800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.915+0000 7f0edc43c740 1 --2- 172.21.15.43:0/1496607728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559d6a3fe580 0x559d6a4762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.915+0000 7f0edc43c740 1 -- 172.21.15.43:0/1496607728 >> 172.21.15.43:0/1496607728 conn(0x559d6a3f8cb0 msgr2=0x559d6a3f74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.915+0000 7f0eb2ffd640 0 client.19872 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.915+0000 7f0edc43c740 1 -- 172.21.15.43:0/1496607728 shutdown_connections 2024-04-21T08:48:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.915+0000 7f0edc43c740 1 -- 172.21.15.43:0/1496607728 wait complete. 2024-04-21T08:48:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.915+0000 7f0edc43c740 20 client.19872 trim_cache size 0 max 16384 2024-04-21T08:48:21.922 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:48:21.922 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:48:21.922 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:48:21.922 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:48:21.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.915+0000 7f0ed9843640 1 -- 172.21.15.43:0/3278927527 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0ebc0a3d20 msgr2=0x7f0ebc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:21.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.915+0000 7f0ed9843640 1 --2- 172.21.15.43:0/3278927527 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0ebc0a3d20 0x7f0ebc0a61e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc04c000 tx=0x7f0ecc049000 comp rx=0 tx=0).stop 2024-04-21T08:48:21.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.915+0000 7f0ed9843640 1 -- 172.21.15.43:0/3278927527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ed41072c0 msgr2=0x7f0ed4198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:21.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.915+0000 7f0ed9843640 1 --2- 172.21.15.43:0/3278927527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ed41072c0 0x7f0ed4198bb0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800c970 tx=0x7f0ec800ce30 comp rx=0 tx=0).stop 2024-04-21T08:48:21.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.915+0000 7f0ed9843640 1 -- 172.21.15.43:0/3278927527 shutdown_connections 2024-04-21T08:48:21.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.915+0000 7f0ed9843640 1 --2- 172.21.15.43:0/3278927527 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0ebc0a3d20 0x7f0ebc0a61e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.915+0000 7f0ed9843640 1 --2- 172.21.15.43:0/3278927527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ed419cf30 0x7f0ed419d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.915+0000 7f0ed9843640 1 --2- 172.21.15.43:0/3278927527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ed41072c0 0x7f0ed4198bb0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.915+0000 7f0ed9843640 1 --2- 172.21.15.43:0/3278927527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ed406a920 0x7f0ed4198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.915+0000 7f0ed9843640 1 -- 172.21.15.43:0/3278927527 >> 172.21.15.43:0/3278927527 conn(0x7f0ed40771d0 msgr2=0x7f0ed40feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:21.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.915+0000 7f0ed9843640 1 -- 172.21.15.43:0/3278927527 shutdown_connections 2024-04-21T08:48:21.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:21.915+0000 7f0ed9843640 1 -- 172.21.15.43:0/3278927527 wait complete. 2024-04-21T08:48:21.926 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:48:21.926 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:48:22.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.029+0000 7ff6a4a31640 1 -- 172.21.15.43:0/3785595510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6a0105e30 msgr2=0x7ff6a0106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.029+0000 7ff6a4a31640 1 --2- 172.21.15.43:0/3785595510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6a0105e30 0x7ff6a0106210 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7ff690009930 tx=0x7ff69002bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:22.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.029+0000 7ff6a4a31640 1 -- 172.21.15.43:0/3785595510 shutdown_connections 2024-04-21T08:48:22.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.029+0000 7ff6a4a31640 1 --2- 172.21.15.43:0/3785595510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a0101b60 0x7ff6a010dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.029+0000 7ff6a4a31640 1 --2- 172.21.15.43:0/3785595510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a01011a0 0x7ff6a0101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.029+0000 7ff6a4a31640 1 --2- 172.21.15.43:0/3785595510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6a0105e30 0x7ff6a0106210 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.029+0000 7ff6a4a31640 1 -- 172.21.15.43:0/3785595510 >> 172.21.15.43:0/3785595510 conn(0x7ff6a0077e30 msgr2=0x7ff6a0078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.029+0000 7ff6a4a31640 1 -- 172.21.15.43:0/3785595510 shutdown_connections 2024-04-21T08:48:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.029+0000 7ff6a4a31640 1 -- 172.21.15.43:0/3785595510 wait complete. 2024-04-21T08:48:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.029+0000 7ff6a4a31640 1 Processor -- start 2024-04-21T08:48:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.030+0000 7ff6a4a31640 1 -- start start 2024-04-21T08:48:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.030+0000 7ff6a4a31640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6a01011a0 0x7ff6a0198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.030+0000 7ff6a4a31640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a0101b60 0x7ff6a0198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.030+0000 7ff6a4a31640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a0105e30 0x7ff6a019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.030+0000 7ff6a4a31640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a0112240 con 0x7ff6a01011a0 2024-04-21T08:48:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.030+0000 7ff6a4a31640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a01120c0 con 0x7ff6a0101b60 2024-04-21T08:48:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.030+0000 7ff6a4a31640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a01123c0 con 0x7ff6a0105e30 2024-04-21T08:48:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.030+0000 7ff69e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6a01011a0 0x7ff6a0198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.030+0000 7ff69e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6a01011a0 0x7ff6a0198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36370/0 (socket says 172.21.15.43:36370) 2024-04-21T08:48:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.030+0000 7ff69e59c640 1 -- 172.21.15.43:0/200233865 learned_addr learned my addr 172.21.15.43:0/200233865 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.030+0000 7ff69ed9d640 1 --2- 172.21.15.43:0/200233865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a0105e30 0x7ff6a019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.030+0000 7ff697fff640 1 --2- 172.21.15.43:0/200233865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a0101b60 0x7ff6a0198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.030+0000 7ff69e59c640 1 -- 172.21.15.43:0/200233865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a0105e30 msgr2=0x7ff6a019d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.030+0000 7ff69e59c640 1 --2- 172.21.15.43:0/200233865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a0105e30 0x7ff6a019d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.030+0000 7ff69e59c640 1 -- 172.21.15.43:0/200233865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a0101b60 msgr2=0x7ff6a0198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.030+0000 7ff69e59c640 1 --2- 172.21.15.43:0/200233865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a0101b60 0x7ff6a0198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.030+0000 7ff69e59c640 1 -- 172.21.15.43:0/200233865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a019d750 con 0x7ff6a01011a0 2024-04-21T08:48:22.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.030+0000 7ff69e59c640 1 --2- 172.21.15.43:0/200233865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6a01011a0 0x7ff6a0198760 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7ff690009230 tx=0x7ff69002c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.031+0000 7ff6977fe640 1 -- 172.21.15.43:0/200233865 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff690043020 con 0x7ff6a01011a0 2024-04-21T08:48:22.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.031+0000 7ff6977fe640 1 -- 172.21.15.43:0/200233865 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6900043d0 con 0x7ff6a01011a0 2024-04-21T08:48:22.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.031+0000 7ff6a4a31640 1 -- 172.21.15.43:0/200233865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a019d9e0 con 0x7ff6a01011a0 2024-04-21T08:48:22.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.031+0000 7ff6977fe640 1 -- 172.21.15.43:0/200233865 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff690003bc0 con 0x7ff6a01011a0 2024-04-21T08:48:22.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.031+0000 7ff6a4a31640 1 -- 172.21.15.43:0/200233865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a01a52d0 con 0x7ff6a01011a0 2024-04-21T08:48:22.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.032+0000 7ff6a4a31640 1 -- 172.21.15.43:0/200233865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a0068de0 con 0x7ff6a01011a0 2024-04-21T08:48:22.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.033+0000 7ff6977fe640 1 -- 172.21.15.43:0/200233865 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff690046020 con 0x7ff6a01011a0 2024-04-21T08:48:22.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.034+0000 7ff6977fe640 1 --2- 172.21.15.43:0/200233865 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff68c0a3d20 0x7ff68c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.034+0000 7ff6977fe640 1 -- 172.21.15.43:0/200233865 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7ff6900f2c60 con 0x7ff6a01011a0 2024-04-21T08:48:22.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.034+0000 7ff697fff640 1 --2- 172.21.15.43:0/200233865 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff68c0a3d20 0x7ff68c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.034+0000 7ff697fff640 1 --2- 172.21.15.43:0/200233865 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff68c0a3d20 0x7ff68c0a61e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0100d80 tx=0x7ff68800b390 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.038+0000 7ff6977fe640 1 -- 172.21.15.43:0/200233865 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff69004d020 con 0x7ff6a01011a0 2024-04-21T08:48:22.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.262+0000 7ff6a4a31640 1 -- 172.21.15.43:0/200233865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6a0109330 con 0x7ff6a01011a0 2024-04-21T08:48:22.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.263+0000 7ff6977fe640 1 -- 172.21.15.43:0/200233865 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11935 (secure 0 0 0) 0x7ff6900beff0 con 0x7ff6a01011a0 2024-04-21T08:48:22.262 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:48:22.263 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:46:17.283806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27556":{"gid":27556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/229985237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":229985237},{"type":"v1","addr":"172.21.15.175:6837","nonce":229985237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24425,24412,24487,24527,24491]},"id":1}]} 2024-04-21T08:48:22.263 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-04-21T08:48:22.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.265+0000 7ff6a4a31640 1 -- 172.21.15.43:0/200233865 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff68c0a3d20 msgr2=0x7ff68c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.265+0000 7ff6a4a31640 1 --2- 172.21.15.43:0/200233865 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff68c0a3d20 0x7ff68c0a61e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0100d80 tx=0x7ff68800b390 comp rx=0 tx=0).stop 2024-04-21T08:48:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.265+0000 7ff6a4a31640 1 -- 172.21.15.43:0/200233865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6a01011a0 msgr2=0x7ff6a0198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.265+0000 7ff6a4a31640 1 --2- 172.21.15.43:0/200233865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6a01011a0 0x7ff6a0198760 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7ff690009230 tx=0x7ff69002c710 comp rx=0 tx=0).stop 2024-04-21T08:48:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.265+0000 7ff6a4a31640 1 -- 172.21.15.43:0/200233865 shutdown_connections 2024-04-21T08:48:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.265+0000 7ff6a4a31640 1 --2- 172.21.15.43:0/200233865 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff68c0a3d20 0x7ff68c0a61e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.265+0000 7ff6a4a31640 1 --2- 172.21.15.43:0/200233865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a0105e30 0x7ff6a019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.265+0000 7ff6a4a31640 1 --2- 172.21.15.43:0/200233865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a0101b60 0x7ff6a0198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.265+0000 7ff6a4a31640 1 --2- 172.21.15.43:0/200233865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6a01011a0 0x7ff6a0198760 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.265+0000 7ff6a4a31640 1 -- 172.21.15.43:0/200233865 >> 172.21.15.43:0/200233865 conn(0x7ff6a0077e30 msgr2=0x7ff6a00fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.265+0000 7ff6a4a31640 1 -- 172.21.15.43:0/200233865 shutdown_connections 2024-04-21T08:48:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.265+0000 7ff6a4a31640 1 -- 172.21.15.43:0/200233865 wait complete. 2024-04-21T08:48:22.276 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:48:22.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.381+0000 7fb47fbdb640 1 -- 172.21.15.43:0/358182897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb478101090 msgr2=0x7fb478101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.381+0000 7fb47fbdb640 1 --2- 172.21.15.43:0/358182897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb478101090 0x7fb478101470 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fb470009900 tx=0x7fb47002bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:22.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.381+0000 7fb47fbdb640 1 -- 172.21.15.43:0/358182897 shutdown_connections 2024-04-21T08:48:22.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.381+0000 7fb47fbdb640 1 --2- 172.21.15.43:0/358182897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb478106160 0x7fb47810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.381+0000 7fb47fbdb640 1 --2- 172.21.15.43:0/358182897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb478101a40 0x7fb478105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.381+0000 7fb47fbdb640 1 --2- 172.21.15.43:0/358182897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb478101090 0x7fb478101470 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.381+0000 7fb47fbdb640 1 -- 172.21.15.43:0/358182897 >> 172.21.15.43:0/358182897 conn(0x7fb478077f50 msgr2=0x7fb478078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.381+0000 7fb47fbdb640 1 -- 172.21.15.43:0/358182897 shutdown_connections 2024-04-21T08:48:22.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.381+0000 7fb47fbdb640 1 -- 172.21.15.43:0/358182897 wait complete. 2024-04-21T08:48:22.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.381+0000 7fb47fbdb640 1 Processor -- start 2024-04-21T08:48:22.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.381+0000 7fb47fbdb640 1 -- start start 2024-04-21T08:48:22.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.381+0000 7fb47fbdb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb478101090 0x7fb478199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.381+0000 7fb47fbdb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb478106160 0x7fb478199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.381+0000 7fb47fbdb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb47819a0a0 0x7fb4781a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.382+0000 7fb47fbdb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb4781110c0 con 0x7fb478106160 2024-04-21T08:48:22.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.382+0000 7fb47fbdb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb478110f40 con 0x7fb47819a0a0 2024-04-21T08:48:22.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.382+0000 7fb47fbdb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb478111240 con 0x7fb478101090 2024-04-21T08:48:22.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.382+0000 7fb47d176640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb478106160 0x7fb478199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.382+0000 7fb47d176640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb478106160 0x7fb478199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36386/0 (socket says 172.21.15.43:36386) 2024-04-21T08:48:22.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.382+0000 7fb47d176640 1 -- 172.21.15.43:0/700883109 learned_addr learned my addr 172.21.15.43:0/700883109 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.382+0000 7fb47d977640 1 --2- 172.21.15.43:0/700883109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb478101090 0x7fb478199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.382+0000 7fb47d176640 1 -- 172.21.15.43:0/700883109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb478101090 msgr2=0x7fb478199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.382+0000 7fb47d176640 1 --2- 172.21.15.43:0/700883109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb478101090 0x7fb478199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.382+0000 7fb47d176640 1 -- 172.21.15.43:0/700883109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb47819a0a0 msgr2=0x7fb4781a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.382+0000 7fb47e178640 1 --2- 172.21.15.43:0/700883109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb47819a0a0 0x7fb4781a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.382+0000 7fb47d176640 1 --2- 172.21.15.43:0/700883109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb47819a0a0 0x7fb4781a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.382+0000 7fb47d176640 1 -- 172.21.15.43:0/700883109 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4781a4e20 con 0x7fb478106160 2024-04-21T08:48:22.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.382+0000 7fb47e178640 1 --2- 172.21.15.43:0/700883109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb47819a0a0 0x7fb4781a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:22.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.382+0000 7fb47d977640 1 --2- 172.21.15.43:0/700883109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb478101090 0x7fb478199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:22.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.382+0000 7fb47d176640 1 --2- 172.21.15.43:0/700883109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb478106160 0x7fb478199ac0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fb46800d740 tx=0x7fb46800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.382+0000 7fb46effd640 1 -- 172.21.15.43:0/700883109 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468017070 con 0x7fb478106160 2024-04-21T08:48:22.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.382+0000 7fb46effd640 1 -- 172.21.15.43:0/700883109 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb46800de50 con 0x7fb478106160 2024-04-21T08:48:22.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.382+0000 7fb47fbdb640 1 -- 172.21.15.43:0/700883109 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4781a5110 con 0x7fb478106160 2024-04-21T08:48:22.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.382+0000 7fb46effd640 1 -- 172.21.15.43:0/700883109 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468004ce0 con 0x7fb478106160 2024-04-21T08:48:22.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.382+0000 7fb47fbdb640 1 -- 172.21.15.43:0/700883109 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4781a5660 con 0x7fb478106160 2024-04-21T08:48:22.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.385+0000 7fb46effd640 1 -- 172.21.15.43:0/700883109 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb468004e80 con 0x7fb478106160 2024-04-21T08:48:22.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.385+0000 7fb4827cf740 1 Processor -- start 2024-04-21T08:48:22.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.385+0000 7fb4827cf740 1 -- start start 2024-04-21T08:48:22.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.385+0000 7fb4827cf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b96b6168e0 0x55b96b616cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.385+0000 7fb4827cf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b96b617290 0x55b96b61f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.385+0000 7fb4827cf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b61fe10 0x55b96b622260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.385+0000 7fb4827cf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b96b5771f0 con 0x55b96b61fe10 2024-04-21T08:48:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.385+0000 7fb4827cf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b96b5144b0 con 0x55b96b6168e0 2024-04-21T08:48:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.385+0000 7fb4827cf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b96b5731b0 con 0x55b96b617290 2024-04-21T08:48:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.386+0000 7fb47d176640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b61fe10 0x55b96b622260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.386+0000 7fb47d176640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b61fe10 0x55b96b622260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36388/0 (socket says 172.21.15.43:36388) 2024-04-21T08:48:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.386+0000 7fb47d176640 1 -- 172.21.15.43:0/2005695303 learned_addr learned my addr 172.21.15.43:0/2005695303 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.386+0000 7fb47e178640 1 --2- 172.21.15.43:0/2005695303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b96b6168e0 0x55b96b616cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.386+0000 7fb47d977640 1 --2- 172.21.15.43:0/2005695303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b96b617290 0x55b96b61f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.386+0000 7fb46effd640 1 --2- 172.21.15.43:0/700883109 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb4640a3da0 0x7fb4640a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.386+0000 7fb47e178640 1 -- 172.21.15.43:0/2005695303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b96b617290 msgr2=0x55b96b61f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.386+0000 7fb47e178640 1 --2- 172.21.15.43:0/2005695303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b96b617290 0x55b96b61f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.386+0000 7fb47e178640 1 -- 172.21.15.43:0/2005695303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b61fe10 msgr2=0x55b96b622260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.386+0000 7fb47e178640 1 --2- 172.21.15.43:0/2005695303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b61fe10 0x55b96b622260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.386+0000 7fb47e178640 1 -- 172.21.15.43:0/2005695303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b96b5061b0 con 0x55b96b6168e0 2024-04-21T08:48:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.386+0000 7fb47d176640 1 --2- 172.21.15.43:0/2005695303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b61fe10 0x55b96b622260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.386+0000 7fb47e178640 1 --2- 172.21.15.43:0/700883109 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb4640a3da0 0x7fb4640a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.386+0000 7fb46effd640 1 -- 172.21.15.43:0/700883109 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fb4680d05f0 con 0x7fb478106160 2024-04-21T08:48:22.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.386+0000 7fb47e178640 1 --2- 172.21.15.43:0/2005695303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b96b6168e0 0x55b96b616cc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb474009f20 tx=0x7fb47400f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71a6c8b2b2eb17eb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.386+0000 7fb463fff640 1 -- 172.21.15.43:0/2005695303 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb474012d00 con 0x55b96b6168e0 2024-04-21T08:48:22.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.386+0000 7fb463fff640 1 -- 172.21.15.43:0/2005695303 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb47401bd90 con 0x55b96b6168e0 2024-04-21T08:48:22.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.386+0000 7fb463fff640 1 -- 172.21.15.43:0/2005695303 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4740119c0 con 0x55b96b6168e0 2024-04-21T08:48:22.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.387+0000 7fb4827cf740 1 -- 172.21.15.43:0/2005695303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b96b6168e0 msgr2=0x55b96b616cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.387+0000 7fb4827cf740 1 --2- 172.21.15.43:0/2005695303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b96b6168e0 0x55b96b616cc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb474009f20 tx=0x7fb47400f7f0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.387+0000 7fb47e178640 1 --2- 172.21.15.43:0/700883109 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb4640a3da0 0x7fb4640a6260 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb47401f000 tx=0x7fb474020000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.387+0000 7fb4827cf740 1 -- 172.21.15.43:0/2005695303 shutdown_connections 2024-04-21T08:48:22.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.387+0000 7fb4827cf740 1 --2- 172.21.15.43:0/2005695303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b61fe10 0x55b96b622260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.387+0000 7fb4827cf740 1 --2- 172.21.15.43:0/2005695303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b96b617290 0x55b96b61f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.387+0000 7fb4827cf740 1 --2- 172.21.15.43:0/2005695303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b96b6168e0 0x55b96b616cc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.387+0000 7fb4827cf740 1 -- 172.21.15.43:0/2005695303 >> 172.21.15.43:0/2005695303 conn(0x55b96b610fb0 msgr2=0x55b96b611390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.387+0000 7fb4827cf740 1 -- 172.21.15.43:0/2005695303 shutdown_connections 2024-04-21T08:48:22.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.387+0000 7fb4827cf740 1 -- 172.21.15.43:0/2005695303 wait complete. 2024-04-21T08:48:22.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.388+0000 7fb4827cf740 1 -- start start 2024-04-21T08:48:22.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.388+0000 7fb4827cf740 1 Processor -- start 2024-04-21T08:48:22.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.388+0000 7fb4827cf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b6168e0 0x55b96b61f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.388+0000 7fb4827cf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b96b617290 0x55b96b61ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.388+0000 7fb4827cf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b96b61fe10 0x55b96b61d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.388+0000 7fb4827cf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b96b622d80 con 0x55b96b6168e0 2024-04-21T08:48:22.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.388+0000 7fb4827cf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b96b530500 con 0x55b96b61fe10 2024-04-21T08:48:22.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.388+0000 7fb4827cf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b96b622f00 con 0x55b96b617290 2024-04-21T08:48:22.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.388+0000 7fb4827cf740 10 client.0 _ll_register_callbacks cb 0x55b96b4feb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:22.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.388+0000 7fb47d977640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b6168e0 0x55b96b61f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.388+0000 7fb47d977640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b6168e0 0x55b96b61f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36396/0 (socket says 172.21.15.43:36396) 2024-04-21T08:48:22.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.388+0000 7fb47d977640 1 -- 172.21.15.43:0/3163823787 learned_addr learned my addr 172.21.15.43:0/3163823787 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.388+0000 7fb47d176640 1 --2- 172.21.15.43:0/3163823787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b96b617290 0x55b96b61ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.388+0000 7fb47e178640 1 --2- 172.21.15.43:0/3163823787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b96b61fe10 0x55b96b61d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.388+0000 7fb47d977640 1 -- 172.21.15.43:0/3163823787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b96b617290 msgr2=0x55b96b61ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.389+0000 7fb47d977640 1 --2- 172.21.15.43:0/3163823787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b96b617290 0x55b96b61ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.389+0000 7fb47d977640 1 -- 172.21.15.43:0/3163823787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b96b61fe10 msgr2=0x55b96b61d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.389+0000 7fb47d977640 1 --2- 172.21.15.43:0/3163823787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b96b61fe10 0x55b96b61d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.389+0000 7fb47d977640 1 -- 172.21.15.43:0/3163823787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b96b2893f0 con 0x55b96b6168e0 2024-04-21T08:48:22.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.389+0000 7fb47d977640 1 --2- 172.21.15.43:0/3163823787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b6168e0 0x55b96b61f700 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fb470000c00 tx=0x7fb47002ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a696a4173d98e5c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.389+0000 7fb4617fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:22.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.389+0000 7fb4617fa640 1 -- 172.21.15.43:0/3163823787 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470005030 con 0x55b96b6168e0 2024-04-21T08:48:22.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.389+0000 7fb4617fa640 1 -- 172.21.15.43:0/3163823787 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fb4700051d0 con 0x55b96b6168e0 2024-04-21T08:48:22.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.389+0000 7fb4827cf740 1 -- 172.21.15.43:0/3163823787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb47401bd90 con 0x55b96b6168e0 2024-04-21T08:48:22.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.389+0000 7fb4617fa640 1 -- 172.21.15.43:0/3163823787 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb470050050 con 0x55b96b6168e0 2024-04-21T08:48:22.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.389+0000 7fb4827cf740 10 client.19890 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:22.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.389+0000 7fb4827cf740 1 -- 172.21.15.43:0/3163823787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b96b3537d0 con 0x55b96b6168e0 2024-04-21T08:48:22.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.389+0000 7fb4617fa640 1 -- 172.21.15.43:0/3163823787 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb47401bd90 con 0x55b96b6168e0 2024-04-21T08:48:22.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.389+0000 7fb4617fa640 1 -- 172.21.15.43:0/3163823787 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb4700035f0 con 0x55b96b6168e0 2024-04-21T08:48:22.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.389+0000 7fb4827cf740 10 client.19890 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:22.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.389+0000 7fb4827cf740 10 client.19890 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:48:22.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.390+0000 7fb4827cf740 20 client.19890 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:22.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.390+0000 7fb4827cf740 1 --2- 172.21.15.43:0/3163823787 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55b96b6300a0 0x55b96b632540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.390+0000 7fb4827cf740 4 client.19890 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:22.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.390+0000 7fb4827cf740 1 -- 172.21.15.43:0/3163823787 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b96b516930 con 0x55b96b6300a0 2024-04-21T08:48:22.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.390+0000 7fb47d176640 1 --2- 172.21.15.43:0/3163823787 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55b96b6300a0 0x55b96b632540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.390+0000 7fb47d176640 1 --2- 172.21.15.43:0/3163823787 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55b96b6300a0 0x55b96b632540 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cad7ad13ffeca456 server_cookie=11d377addd5ef63c in_seq=0 out_seq=0 2024-04-21T08:48:22.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.390+0000 7fb4617fa640 10 client.19890 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:48:22.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.391+0000 7fb4617fa640 1 -- 172.21.15.43:0/3163823787 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b96b516930 con 0x55b96b6300a0 2024-04-21T08:48:22.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.391+0000 7fb4617fa640 10 client.19890 handle_command_reply: tid=0 2024-04-21T08:48:22.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.391+0000 7fb4827cf740 1 client.19890 shutdown 2024-04-21T08:48:22.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.391+0000 7fb4827cf740 1 -- 172.21.15.43:0/3163823787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b6168e0 msgr2=0x55b96b61f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.391+0000 7fb4827cf740 1 --2- 172.21.15.43:0/3163823787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b6168e0 0x55b96b61f700 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fb470000c00 tx=0x7fb47002ca10 comp rx=0 tx=0).stop 2024-04-21T08:48:22.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.392+0000 7fb4827cf740 1 -- 172.21.15.43:0/3163823787 shutdown_connections 2024-04-21T08:48:22.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.392+0000 7fb4827cf740 1 --2- 172.21.15.43:0/3163823787 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55b96b6300a0 0x55b96b632540 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.392+0000 7fb4827cf740 1 --2- 172.21.15.43:0/3163823787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b96b61fe10 0x55b96b61d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.392+0000 7fb4617fa640 0 client.19890 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:48:22.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.392+0000 7fb4827cf740 1 --2- 172.21.15.43:0/3163823787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b96b617290 0x55b96b61ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.392+0000 7fb4827cf740 1 --2- 172.21.15.43:0/3163823787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b6168e0 0x55b96b61f700 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.392+0000 7fb4827cf740 1 -- 172.21.15.43:0/3163823787 >> 172.21.15.43:0/3163823787 conn(0x55b96b610fb0 msgr2=0x55b96b612d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.392+0000 7fb4827cf740 1 -- 172.21.15.43:0/3163823787 shutdown_connections 2024-04-21T08:48:22.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.392+0000 7fb4827cf740 1 -- 172.21.15.43:0/3163823787 wait complete. 2024-04-21T08:48:22.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.392+0000 7fb4827cf740 20 client.19890 trim_cache size 0 max 16384 2024-04-21T08:48:22.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.406+0000 7fb4827cf740 1 Processor -- start 2024-04-21T08:48:22.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.406+0000 7fb4827cf740 1 -- start start 2024-04-21T08:48:22.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.406+0000 7fb4827cf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b96b6168e0 0x55b96b617e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.406+0000 7fb4827cf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b96b617290 0x55b96b61f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.406+0000 7fb4827cf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b61fe10 0x55b96b61f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.406+0000 7fb4827cf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb4781ad2d0 con 0x55b96b61fe10 2024-04-21T08:48:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.406+0000 7fb4827cf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb4781aee10 con 0x55b96b6168e0 2024-04-21T08:48:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.406+0000 7fb4827cf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb4781ad9a0 con 0x55b96b617290 2024-04-21T08:48:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.406+0000 7fb47e178640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b61fe10 0x55b96b61f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.406+0000 7fb47e178640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b61fe10 0x55b96b61f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36410/0 (socket says 172.21.15.43:36410) 2024-04-21T08:48:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.406+0000 7fb47e178640 1 -- 172.21.15.43:0/4127514882 learned_addr learned my addr 172.21.15.43:0/4127514882 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.406+0000 7fb47d176640 1 --2- 172.21.15.43:0/4127514882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b96b617290 0x55b96b61f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.406+0000 7fb47d977640 1 --2- 172.21.15.43:0/4127514882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b96b6168e0 0x55b96b617e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.406+0000 7fb47e178640 1 -- 172.21.15.43:0/4127514882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b96b617290 msgr2=0x55b96b61f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.406+0000 7fb47e178640 1 --2- 172.21.15.43:0/4127514882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b96b617290 0x55b96b61f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.406+0000 7fb47e178640 1 -- 172.21.15.43:0/4127514882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b96b6168e0 msgr2=0x55b96b617e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.406+0000 7fb47e178640 1 --2- 172.21.15.43:0/4127514882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b96b6168e0 0x55b96b617e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.406+0000 7fb47e178640 1 -- 172.21.15.43:0/4127514882 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b96b61d380 con 0x55b96b61fe10 2024-04-21T08:48:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.406+0000 7fb47e178640 1 --2- 172.21.15.43:0/4127514882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b61fe10 0x55b96b61f700 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fb474006010 tx=0x7fb474009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea57815b8480f976 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.407+0000 7fb4637fe640 1 -- 172.21.15.43:0/4127514882 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb474035070 con 0x55b96b61fe10 2024-04-21T08:48:22.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.407+0000 7fb4637fe640 1 -- 172.21.15.43:0/4127514882 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb474012450 con 0x55b96b61fe10 2024-04-21T08:48:22.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.407+0000 7fb4637fe640 1 -- 172.21.15.43:0/4127514882 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb474011e90 con 0x55b96b61fe10 2024-04-21T08:48:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.407+0000 7fb4827cf740 1 -- 172.21.15.43:0/4127514882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b61fe10 msgr2=0x55b96b61f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.407+0000 7fb4827cf740 1 --2- 172.21.15.43:0/4127514882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b61fe10 0x55b96b61f700 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fb474006010 tx=0x7fb474009670 comp rx=0 tx=0).stop 2024-04-21T08:48:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.407+0000 7fb4827cf740 1 -- 172.21.15.43:0/4127514882 shutdown_connections 2024-04-21T08:48:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.407+0000 7fb4827cf740 1 --2- 172.21.15.43:0/4127514882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b61fe10 0x55b96b61f700 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.407+0000 7fb4827cf740 1 --2- 172.21.15.43:0/4127514882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b96b617290 0x55b96b61f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.407+0000 7fb4827cf740 1 --2- 172.21.15.43:0/4127514882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b96b6168e0 0x55b96b617e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.407+0000 7fb4827cf740 1 -- 172.21.15.43:0/4127514882 >> 172.21.15.43:0/4127514882 conn(0x55b96b610fb0 msgr2=0x55b96b617750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.407+0000 7fb4827cf740 1 -- 172.21.15.43:0/4127514882 shutdown_connections 2024-04-21T08:48:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.407+0000 7fb4827cf740 1 -- 172.21.15.43:0/4127514882 wait complete. 2024-04-21T08:48:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.408+0000 7fb4827cf740 1 -- start start 2024-04-21T08:48:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.408+0000 7fb4827cf740 1 Processor -- start 2024-04-21T08:48:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.408+0000 7fb4827cf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b96b6168e0 0x55b96b619b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.408+0000 7fb4827cf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b617290 0x55b96b61a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.408+0000 7fb4827cf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b96b61fe10 0x55b96b695680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.408+0000 7fb4827cf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb4781ae740 con 0x55b96b617290 2024-04-21T08:48:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.408+0000 7fb4827cf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb4781ae070 con 0x55b96b6168e0 2024-04-21T08:48:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.408+0000 7fb4827cf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb4781ab2d0 con 0x55b96b61fe10 2024-04-21T08:48:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.408+0000 7fb4827cf740 10 client.0 _ll_register_callbacks cb 0x7fb4700044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.408+0000 7fb47d176640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b617290 0x55b96b61a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.408+0000 7fb47d176640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b617290 0x55b96b61a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36420/0 (socket says 172.21.15.43:36420) 2024-04-21T08:48:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.408+0000 7fb47d176640 1 -- 172.21.15.43:0/2600904830 learned_addr learned my addr 172.21.15.43:0/2600904830 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.408+0000 7fb47e178640 1 --2- 172.21.15.43:0/2600904830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b96b61fe10 0x55b96b695680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.408+0000 7fb47d977640 1 --2- 172.21.15.43:0/2600904830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b96b6168e0 0x55b96b619b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.408+0000 7fb47d176640 1 -- 172.21.15.43:0/2600904830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b96b61fe10 msgr2=0x55b96b695680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.408+0000 7fb47d176640 1 --2- 172.21.15.43:0/2600904830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b96b61fe10 0x55b96b695680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.408+0000 7fb47d176640 1 -- 172.21.15.43:0/2600904830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b96b6168e0 msgr2=0x55b96b619b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.408+0000 7fb47d176640 1 --2- 172.21.15.43:0/2600904830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b96b6168e0 0x55b96b619b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.409+0000 7fb47d176640 1 -- 172.21.15.43:0/2600904830 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b96b695bc0 con 0x55b96b617290 2024-04-21T08:48:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.409+0000 7fb47e178640 1 --2- 172.21.15.43:0/2600904830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b96b61fe10 0x55b96b695680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.409+0000 7fb47d176640 1 --2- 172.21.15.43:0/2600904830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b617290 0x55b96b61a050 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fb46809c100 tx=0x7fb46800d6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbf2d2b8d63107c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.409+0000 7fb4617fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.409+0000 7fb4617fa640 1 -- 172.21.15.43:0/2600904830 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468020040 con 0x55b96b617290 2024-04-21T08:48:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.409+0000 7fb4617fa640 1 -- 172.21.15.43:0/2600904830 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fb4680cdba0 con 0x55b96b617290 2024-04-21T08:48:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.409+0000 7fb4827cf740 1 -- 172.21.15.43:0/2600904830 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb474012450 con 0x55b96b617290 2024-04-21T08:48:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.409+0000 7fb4617fa640 1 -- 172.21.15.43:0/2600904830 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb46809c870 con 0x55b96b617290 2024-04-21T08:48:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.409+0000 7fb4827cf740 10 client.19896 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.409+0000 7fb4827cf740 1 -- 172.21.15.43:0/2600904830 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b96b696da0 con 0x55b96b617290 2024-04-21T08:48:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.409+0000 7fb4617fa640 1 -- 172.21.15.43:0/2600904830 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb474012450 con 0x55b96b617290 2024-04-21T08:48:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.409+0000 7fb4617fa640 1 -- 172.21.15.43:0/2600904830 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb468096ce0 con 0x55b96b617290 2024-04-21T08:48:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.409+0000 7fb4827cf740 10 client.19896 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.409+0000 7fb4827cf740 10 client.19896 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:48:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.409+0000 7fb4827cf740 20 client.19896 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.409+0000 7fb4827cf740 1 --2- 172.21.15.43:0/2600904830 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55b96b6975a0 0x55b96b6999c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.409+0000 7fb4827cf740 4 client.19896 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.409+0000 7fb4827cf740 1 -- 172.21.15.43:0/2600904830 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b96b584c10 con 0x55b96b6975a0 2024-04-21T08:48:22.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.410+0000 7fb47d977640 1 --2- 172.21.15.43:0/2600904830 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55b96b6975a0 0x55b96b6999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.410+0000 7fb47d977640 1 --2- 172.21.15.43:0/2600904830 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55b96b6975a0 0x55b96b6999c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=253ac41b55572a4a server_cookie=fbdf5f0d1c3936e6 in_seq=0 out_seq=0 2024-04-21T08:48:22.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.410+0000 7fb4617fa640 10 client.19896 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:48:22.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.410+0000 7fb4617fa640 1 -- 172.21.15.43:0/2600904830 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b96b584c10 con 0x55b96b6975a0 2024-04-21T08:48:22.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.410+0000 7fb4617fa640 10 client.19896 handle_command_reply: tid=0 2024-04-21T08:48:22.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.410+0000 7fb4827cf740 1 client.19896 shutdown 2024-04-21T08:48:22.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.410+0000 7fb4827cf740 1 -- 172.21.15.43:0/2600904830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b617290 msgr2=0x55b96b61a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.410+0000 7fb4827cf740 1 --2- 172.21.15.43:0/2600904830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b617290 0x55b96b61a050 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fb46809c100 tx=0x7fb46800d6c0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.411+0000 7fb4827cf740 1 -- 172.21.15.43:0/2600904830 shutdown_connections 2024-04-21T08:48:22.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.411+0000 7fb4827cf740 1 --2- 172.21.15.43:0/2600904830 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55b96b6975a0 0x55b96b6999c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.411+0000 7fb4827cf740 1 --2- 172.21.15.43:0/2600904830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b96b61fe10 0x55b96b695680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.411+0000 7fb4617fa640 0 client.19896 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:48:22.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.411+0000 7fb4827cf740 1 --2- 172.21.15.43:0/2600904830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b96b617290 0x55b96b61a050 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.411+0000 7fb4827cf740 1 --2- 172.21.15.43:0/2600904830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b96b6168e0 0x55b96b619b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.411+0000 7fb4827cf740 1 -- 172.21.15.43:0/2600904830 >> 172.21.15.43:0/2600904830 conn(0x55b96b610fb0 msgr2=0x55b96b617fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.411+0000 7fb4827cf740 1 -- 172.21.15.43:0/2600904830 shutdown_connections 2024-04-21T08:48:22.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.411+0000 7fb4827cf740 1 -- 172.21.15.43:0/2600904830 wait complete. 2024-04-21T08:48:22.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.411+0000 7fb4827cf740 20 client.19896 trim_cache size 0 max 16384 2024-04-21T08:48:22.417 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:48:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.411+0000 7fb47fbdb640 1 -- 172.21.15.43:0/700883109 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb4640a3da0 msgr2=0x7fb4640a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.411+0000 7fb47fbdb640 1 --2- 172.21.15.43:0/700883109 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb4640a3da0 0x7fb4640a6260 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb47401f000 tx=0x7fb474020000 comp rx=0 tx=0).stop 2024-04-21T08:48:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.411+0000 7fb47fbdb640 1 -- 172.21.15.43:0/700883109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb478106160 msgr2=0x7fb478199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.411+0000 7fb47fbdb640 1 --2- 172.21.15.43:0/700883109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb478106160 0x7fb478199ac0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fb46800d740 tx=0x7fb46800dc00 comp rx=0 tx=0).stop 2024-04-21T08:48:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.411+0000 7fb47fbdb640 1 -- 172.21.15.43:0/700883109 shutdown_connections 2024-04-21T08:48:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.411+0000 7fb47fbdb640 1 --2- 172.21.15.43:0/700883109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb47819a0a0 0x7fb4781a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.411+0000 7fb47fbdb640 1 --2- 172.21.15.43:0/700883109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb478106160 0x7fb478199ac0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.411+0000 7fb47fbdb640 1 --2- 172.21.15.43:0/700883109 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb4640a3da0 0x7fb4640a6260 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.411+0000 7fb47fbdb640 1 --2- 172.21.15.43:0/700883109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb478101090 0x7fb478199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.411+0000 7fb47fbdb640 1 -- 172.21.15.43:0/700883109 >> 172.21.15.43:0/700883109 conn(0x7fb478077f50 msgr2=0x7fb4780ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.411+0000 7fb47fbdb640 1 -- 172.21.15.43:0/700883109 shutdown_connections 2024-04-21T08:48:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.411+0000 7fb47fbdb640 1 -- 172.21.15.43:0/700883109 wait complete. 2024-04-21T08:48:22.420 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:22 smithi043 ceph-mon[27978]: pgmap v293: 129 pgs: 129 active+clean; 3.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 65 KiB/s wr, 24 op/s 2024-04-21T08:48:22.422 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:48:22.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:22 smithi175 ceph-mon[31920]: pgmap v293: 129 pgs: 129 active+clean; 3.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 65 KiB/s wr, 24 op/s 2024-04-21T08:48:22.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.526+0000 7ff803fc7640 1 -- 172.21.15.43:0/654281602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7fc102b40 msgr2=0x7ff7fc106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.526+0000 7ff803fc7640 1 --2- 172.21.15.43:0/654281602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7fc102b40 0x7ff7fc106b80 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009900 tx=0x7ff7f002bc50 comp rx=0 tx=0).stop 2024-04-21T08:48:22.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.526+0000 7ff803fc7640 1 -- 172.21.15.43:0/654281602 shutdown_connections 2024-04-21T08:48:22.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.526+0000 7ff803fc7640 1 --2- 172.21.15.43:0/654281602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7fc107260 0x7ff7fc10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.526+0000 7ff803fc7640 1 --2- 172.21.15.43:0/654281602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7fc102b40 0x7ff7fc106b80 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.526+0000 7ff803fc7640 1 --2- 172.21.15.43:0/654281602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7fc102190 0x7ff7fc102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.526+0000 7ff803fc7640 1 -- 172.21.15.43:0/654281602 >> 172.21.15.43:0/654281602 conn(0x7ff7fc0793a0 msgr2=0x7ff7fc0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.527+0000 7ff803fc7640 1 -- 172.21.15.43:0/654281602 shutdown_connections 2024-04-21T08:48:22.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.527+0000 7ff803fc7640 1 -- 172.21.15.43:0/654281602 wait complete. 2024-04-21T08:48:22.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.527+0000 7ff803fc7640 1 Processor -- start 2024-04-21T08:48:22.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.527+0000 7ff803fc7640 1 -- start start 2024-04-21T08:48:22.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.527+0000 7ff803fc7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7fc102190 0x7ff7fc198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.527+0000 7ff803fc7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7fc107260 0x7ff7fc198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.527+0000 7ff803fc7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7fc19cf40 0x7ff7fc19d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.527+0000 7ff803fc7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc112180 con 0x7ff7fc107260 2024-04-21T08:48:22.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.527+0000 7ff803fc7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc112000 con 0x7ff7fc19cf40 2024-04-21T08:48:22.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.527+0000 7ff803fc7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc112300 con 0x7ff7fc102190 2024-04-21T08:48:22.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.527+0000 7ff801562640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7fc107260 0x7ff7fc198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.528+0000 7ff801562640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7fc107260 0x7ff7fc198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36430/0 (socket says 172.21.15.43:36430) 2024-04-21T08:48:22.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.528+0000 7ff801562640 1 -- 172.21.15.43:0/1120157402 learned_addr learned my addr 172.21.15.43:0/1120157402 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.528+0000 7ff801d63640 1 --2- 172.21.15.43:0/1120157402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7fc102190 0x7ff7fc198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.528+0000 7ff802564640 1 --2- 172.21.15.43:0/1120157402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7fc19cf40 0x7ff7fc19d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.528+0000 7ff801562640 1 -- 172.21.15.43:0/1120157402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7fc102190 msgr2=0x7ff7fc198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.528+0000 7ff801562640 1 --2- 172.21.15.43:0/1120157402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7fc102190 0x7ff7fc198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.528+0000 7ff801562640 1 -- 172.21.15.43:0/1120157402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7fc19cf40 msgr2=0x7ff7fc19d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.528+0000 7ff801562640 1 --2- 172.21.15.43:0/1120157402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7fc19cf40 0x7ff7fc19d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.528+0000 7ff801562640 1 -- 172.21.15.43:0/1120157402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7fc19da30 con 0x7ff7fc107260 2024-04-21T08:48:22.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.528+0000 7ff802564640 1 --2- 172.21.15.43:0/1120157402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7fc19cf40 0x7ff7fc19d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:22.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.528+0000 7ff801562640 1 --2- 172.21.15.43:0/1120157402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7fc107260 0x7ff7fc198bc0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0005ec0 tx=0x7ff7f002ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.528+0000 7ff7faffd640 1 -- 172.21.15.43:0/1120157402 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0043020 con 0x7ff7fc107260 2024-04-21T08:48:22.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.528+0000 7ff803fc7640 1 -- 172.21.15.43:0/1120157402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7fc19dcc0 con 0x7ff7fc107260 2024-04-21T08:48:22.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.528+0000 7ff7faffd640 1 -- 172.21.15.43:0/1120157402 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7f00052b0 con 0x7ff7fc107260 2024-04-21T08:48:22.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.528+0000 7ff7faffd640 1 -- 172.21.15.43:0/1120157402 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0003c60 con 0x7ff7fc107260 2024-04-21T08:48:22.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.528+0000 7ff803fc7640 1 -- 172.21.15.43:0/1120157402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7fc1a5510 con 0x7ff7fc107260 2024-04-21T08:48:22.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.531+0000 7ff7faffd640 1 -- 172.21.15.43:0/1120157402 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff7f0046020 con 0x7ff7fc107260 2024-04-21T08:48:22.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.531+0000 7ff806bb8740 1 Processor -- start 2024-04-21T08:48:22.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.531+0000 7ff806bb8740 1 -- start start 2024-04-21T08:48:22.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.532+0000 7ff806bb8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1561f4d0 0x555f1561f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.532+0000 7ff806bb8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f1561fe80 0x555f15628430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.532+0000 7ff806bb8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f15628a00 0x555f1562ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.532+0000 7ff806bb8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555f155841f0 con 0x555f1561fe80 2024-04-21T08:48:22.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.532+0000 7ff806bb8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555f155214b0 con 0x555f15628a00 2024-04-21T08:48:22.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.532+0000 7ff806bb8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555f155801b0 con 0x555f1561f4d0 2024-04-21T08:48:22.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.532+0000 7ff801d63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f1561fe80 0x555f15628430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.532+0000 7ff801d63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f1561fe80 0x555f15628430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36442/0 (socket says 172.21.15.43:36442) 2024-04-21T08:48:22.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.532+0000 7ff801d63640 1 -- 172.21.15.43:0/545580309 learned_addr learned my addr 172.21.15.43:0/545580309 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.532+0000 7ff802564640 1 --2- 172.21.15.43:0/545580309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1561f4d0 0x555f1561f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.532+0000 7ff801562640 1 --2- 172.21.15.43:0/545580309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f15628a00 0x555f1562ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.532+0000 7ff7faffd640 1 --2- 172.21.15.43:0/1120157402 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff7e40a3d20 0x7ff7e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.532+0000 7ff801d63640 1 -- 172.21.15.43:0/545580309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1561f4d0 msgr2=0x555f1561f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.532+0000 7ff801d63640 1 --2- 172.21.15.43:0/545580309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1561f4d0 0x555f1561f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.532+0000 7ff801d63640 1 -- 172.21.15.43:0/545580309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f15628a00 msgr2=0x555f1562ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.532+0000 7ff801d63640 1 --2- 172.21.15.43:0/545580309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f15628a00 0x555f1562ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.532+0000 7ff801d63640 1 -- 172.21.15.43:0/545580309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f155131b0 con 0x555f1561fe80 2024-04-21T08:48:22.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.532+0000 7ff802564640 1 --2- 172.21.15.43:0/1120157402 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff7e40a3d20 0x7ff7e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.532+0000 7ff7faffd640 1 -- 172.21.15.43:0/1120157402 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7ff7f00f2e40 con 0x7ff7fc107260 2024-04-21T08:48:22.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.532+0000 7ff801d63640 1 --2- 172.21.15.43:0/545580309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f1561fe80 0x555f15628430 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400ad00 tx=0x7ff7f400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de6e437eb875292 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.533+0000 7ff7e3fff640 1 -- 172.21.15.43:0/545580309 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f401d070 con 0x555f1561fe80 2024-04-21T08:48:22.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.533+0000 7ff7e3fff640 1 -- 172.21.15.43:0/545580309 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7f4018af0 con 0x555f1561fe80 2024-04-21T08:48:22.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.533+0000 7ff7e3fff640 1 -- 172.21.15.43:0/545580309 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4010b30 con 0x555f1561fe80 2024-04-21T08:48:22.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.533+0000 7ff806bb8740 1 -- 172.21.15.43:0/545580309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f1561fe80 msgr2=0x555f15628430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.533+0000 7ff802564640 1 --2- 172.21.15.43:0/1120157402 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff7e40a3d20 0x7ff7e40a61e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec005cc0 tx=0x7ff7ec004a40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.533+0000 7ff806bb8740 1 --2- 172.21.15.43:0/545580309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f1561fe80 0x555f15628430 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400ad00 tx=0x7ff7f400f790 comp rx=0 tx=0).stop 2024-04-21T08:48:22.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.533+0000 7ff806bb8740 1 -- 172.21.15.43:0/545580309 shutdown_connections 2024-04-21T08:48:22.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.533+0000 7ff806bb8740 1 --2- 172.21.15.43:0/545580309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f15628a00 0x555f1562ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.533+0000 7ff806bb8740 1 --2- 172.21.15.43:0/545580309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f1561fe80 0x555f15628430 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.533+0000 7ff806bb8740 1 --2- 172.21.15.43:0/545580309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1561f4d0 0x555f1561f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.533+0000 7ff806bb8740 1 -- 172.21.15.43:0/545580309 >> 172.21.15.43:0/545580309 conn(0x555f15619ba0 msgr2=0x555f15619f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.533+0000 7ff806bb8740 1 -- 172.21.15.43:0/545580309 shutdown_connections 2024-04-21T08:48:22.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.533+0000 7ff806bb8740 1 -- 172.21.15.43:0/545580309 wait complete. 2024-04-21T08:48:22.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.534+0000 7ff806bb8740 1 -- start start 2024-04-21T08:48:22.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.534+0000 7ff806bb8740 1 Processor -- start 2024-04-21T08:48:22.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.534+0000 7ff806bb8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f1561f4d0 0x555f15628290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.534+0000 7ff806bb8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1561fe80 0x555f15625620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.534+0000 7ff806bb8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f15628a00 0x555f15625b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.534+0000 7ff806bb8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555f1562b970 con 0x555f15628a00 2024-04-21T08:48:22.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.534+0000 7ff806bb8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555f1553d500 con 0x555f1561f4d0 2024-04-21T08:48:22.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.534+0000 7ff806bb8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555f1562baf0 con 0x555f1561fe80 2024-04-21T08:48:22.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.534+0000 7ff806bb8740 10 client.0 _ll_register_callbacks cb 0x555f1550bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:22.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.534+0000 7ff802564640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f15628a00 0x555f15625b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.534+0000 7ff802564640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f15628a00 0x555f15625b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36456/0 (socket says 172.21.15.43:36456) 2024-04-21T08:48:22.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.534+0000 7ff802564640 1 -- 172.21.15.43:0/1356305262 learned_addr learned my addr 172.21.15.43:0/1356305262 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.534+0000 7ff801d63640 1 --2- 172.21.15.43:0/1356305262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f1561f4d0 0x555f15628290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.535+0000 7ff801562640 1 --2- 172.21.15.43:0/1356305262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1561fe80 0x555f15625620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.535+0000 7ff802564640 1 -- 172.21.15.43:0/1356305262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1561fe80 msgr2=0x555f15625620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.535+0000 7ff802564640 1 --2- 172.21.15.43:0/1356305262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1561fe80 0x555f15625620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.535+0000 7ff802564640 1 -- 172.21.15.43:0/1356305262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f1561f4d0 msgr2=0x555f15628290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.535+0000 7ff802564640 1 --2- 172.21.15.43:0/1356305262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f1561f4d0 0x555f15628290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.535+0000 7ff802564640 1 -- 172.21.15.43:0/1356305262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f152963f0 con 0x555f15628a00 2024-04-21T08:48:22.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.535+0000 7ff802564640 1 --2- 172.21.15.43:0/1356305262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f15628a00 0x555f15625b60 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec01ce30 tx=0x7ff7ec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fb2b9c8bee06bd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.535+0000 7ff7e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:22.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.535+0000 7ff7e17fa640 1 -- 172.21.15.43:0/1356305262 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec029060 con 0x555f15628a00 2024-04-21T08:48:22.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.535+0000 7ff7e17fa640 1 -- 172.21.15.43:0/1356305262 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7ff7ec019620 con 0x555f15628a00 2024-04-21T08:48:22.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.535+0000 7ff806bb8740 1 -- 172.21.15.43:0/1356305262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7f4018af0 con 0x555f15628a00 2024-04-21T08:48:22.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.535+0000 7ff7e17fa640 1 -- 172.21.15.43:0/1356305262 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff7ec030050 con 0x555f15628a00 2024-04-21T08:48:22.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.535+0000 7ff806bb8740 10 client.19908 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:22.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.535+0000 7ff806bb8740 1 -- 172.21.15.43:0/1356305262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555f153607d0 con 0x555f15628a00 2024-04-21T08:48:22.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.535+0000 7ff7e17fa640 1 -- 172.21.15.43:0/1356305262 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7f4018af0 con 0x555f15628a00 2024-04-21T08:48:22.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.536+0000 7ff7e17fa640 1 -- 172.21.15.43:0/1356305262 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff7ec024720 con 0x555f15628a00 2024-04-21T08:48:22.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.536+0000 7ff806bb8740 10 client.19908 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:22.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.536+0000 7ff806bb8740 10 client.19908 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:48:22.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.536+0000 7ff806bb8740 20 client.19908 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:22.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.536+0000 7ff806bb8740 1 --2- 172.21.15.43:0/1356305262 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x555f15638c60 0x555f1563b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.536+0000 7ff806bb8740 4 client.19908 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:22.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.536+0000 7ff806bb8740 1 -- 172.21.15.43:0/1356305262 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f15523930 con 0x555f15638c60 2024-04-21T08:48:22.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.536+0000 7ff801d63640 1 --2- 172.21.15.43:0/1356305262 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x555f15638c60 0x555f1563b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.536+0000 7ff801d63640 1 --2- 172.21.15.43:0/1356305262 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x555f15638c60 0x555f1563b090 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1a1dcdbb4ce4c724 server_cookie=b00919ff8af2041d in_seq=0 out_seq=0 2024-04-21T08:48:22.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.536+0000 7ff7e17fa640 10 client.19908 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:48:22.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.537+0000 7ff7e17fa640 1 -- 172.21.15.43:0/1356305262 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f15523930 con 0x555f15638c60 2024-04-21T08:48:22.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.537+0000 7ff7e17fa640 10 client.19908 handle_command_reply: tid=0 2024-04-21T08:48:22.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.537+0000 7ff806bb8740 1 client.19908 shutdown 2024-04-21T08:48:22.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.537+0000 7ff806bb8740 1 -- 172.21.15.43:0/1356305262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f15628a00 msgr2=0x555f15625b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.537+0000 7ff806bb8740 1 --2- 172.21.15.43:0/1356305262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f15628a00 0x555f15625b60 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec01ce30 tx=0x7ff7ec004ab0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.537+0000 7ff806bb8740 1 -- 172.21.15.43:0/1356305262 shutdown_connections 2024-04-21T08:48:22.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.537+0000 7ff806bb8740 1 --2- 172.21.15.43:0/1356305262 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x555f15638c60 0x555f1563b090 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.537+0000 7ff806bb8740 1 --2- 172.21.15.43:0/1356305262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f15628a00 0x555f15625b60 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.537+0000 7ff806bb8740 1 --2- 172.21.15.43:0/1356305262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1561fe80 0x555f15625620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.537+0000 7ff806bb8740 1 --2- 172.21.15.43:0/1356305262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f1561f4d0 0x555f15628290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.537+0000 7ff7e17fa640 0 client.19908 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:48:22.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.537+0000 7ff806bb8740 1 -- 172.21.15.43:0/1356305262 >> 172.21.15.43:0/1356305262 conn(0x555f15619ba0 msgr2=0x555f1561b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.537+0000 7ff806bb8740 1 -- 172.21.15.43:0/1356305262 shutdown_connections 2024-04-21T08:48:22.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.537+0000 7ff806bb8740 1 -- 172.21.15.43:0/1356305262 wait complete. 2024-04-21T08:48:22.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.538+0000 7ff806bb8740 20 client.19908 trim_cache size 0 max 16384 2024-04-21T08:48:22.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:22 smithi112 ceph-mon[31085]: pgmap v293: 129 pgs: 129 active+clean; 3.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 65 KiB/s wr, 24 op/s 2024-04-21T08:48:22.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.551+0000 7ff806bb8740 1 Processor -- start 2024-04-21T08:48:22.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.551+0000 7ff806bb8740 1 -- start start 2024-04-21T08:48:22.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.551+0000 7ff806bb8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f1561f4d0 0x555f15634860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.551+0000 7ff806bb8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f1561fe80 0x555f15618a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.551+0000 7ff806bb8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f15628a00 0x555f15627d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.551+0000 7ff806bb8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc1af3c0 con 0x555f1561fe80 2024-04-21T08:48:22.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.551+0000 7ff806bb8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc1b0f00 con 0x555f1561f4d0 2024-04-21T08:48:22.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.551+0000 7ff806bb8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc1afa90 con 0x555f15628a00 2024-04-21T08:48:22.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.551+0000 7ff801562640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f1561fe80 0x555f15618a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.551+0000 7ff801562640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f1561fe80 0x555f15618a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36462/0 (socket says 172.21.15.43:36462) 2024-04-21T08:48:22.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.551+0000 7ff801562640 1 -- 172.21.15.43:0/2962119125 learned_addr learned my addr 172.21.15.43:0/2962119125 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.551+0000 7ff801d63640 1 --2- 172.21.15.43:0/2962119125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f1561f4d0 0x555f15634860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.551+0000 7ff801562640 1 -- 172.21.15.43:0/2962119125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f15628a00 msgr2=0x555f15627d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.551+0000 7ff801562640 1 --2- 172.21.15.43:0/2962119125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f15628a00 0x555f15627d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.551+0000 7ff801562640 1 -- 172.21.15.43:0/2962119125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f1561f4d0 msgr2=0x555f15634860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.551+0000 7ff801562640 1 --2- 172.21.15.43:0/2962119125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f1561f4d0 0x555f15634860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.551+0000 7ff801562640 1 -- 172.21.15.43:0/2962119125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f15637800 con 0x555f1561fe80 2024-04-21T08:48:22.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.551+0000 7ff801d63640 1 --2- 172.21.15.43:0/2962119125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f1561f4d0 0x555f15634860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:22.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.551+0000 7ff801562640 1 --2- 172.21.15.43:0/2962119125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f1561fe80 0x555f15618a80 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7ff7f00f0770 tx=0x7ff7f00baea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ee25bafe26f8d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.552+0000 7ff7e37fe640 1 -- 172.21.15.43:0/2962119125 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f00be610 con 0x555f1561fe80 2024-04-21T08:48:22.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.552+0000 7ff7e37fe640 1 -- 172.21.15.43:0/2962119125 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7f00bf1d0 con 0x555f1561fe80 2024-04-21T08:48:22.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.552+0000 7ff7e37fe640 1 -- 172.21.15.43:0/2962119125 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f00be430 con 0x555f1561fe80 2024-04-21T08:48:22.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.552+0000 7ff806bb8740 1 -- 172.21.15.43:0/2962119125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f1561fe80 msgr2=0x555f15618a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.552+0000 7ff806bb8740 1 --2- 172.21.15.43:0/2962119125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f1561fe80 0x555f15618a80 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7ff7f00f0770 tx=0x7ff7f00baea0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.552+0000 7ff806bb8740 1 -- 172.21.15.43:0/2962119125 shutdown_connections 2024-04-21T08:48:22.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.552+0000 7ff806bb8740 1 --2- 172.21.15.43:0/2962119125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f15628a00 0x555f15627d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.552+0000 7ff806bb8740 1 --2- 172.21.15.43:0/2962119125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f1561fe80 0x555f15618a80 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.552+0000 7ff806bb8740 1 --2- 172.21.15.43:0/2962119125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f1561f4d0 0x555f15634860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.552+0000 7ff806bb8740 1 -- 172.21.15.43:0/2962119125 >> 172.21.15.43:0/2962119125 conn(0x555f15619ba0 msgr2=0x555f15618370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.552+0000 7ff806bb8740 1 -- 172.21.15.43:0/2962119125 shutdown_connections 2024-04-21T08:48:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.552+0000 7ff806bb8740 1 -- 172.21.15.43:0/2962119125 wait complete. 2024-04-21T08:48:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.553+0000 7ff806bb8740 1 -- start start 2024-04-21T08:48:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.553+0000 7ff806bb8740 1 Processor -- start 2024-04-21T08:48:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.553+0000 7ff806bb8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f1561f4d0 0x555f15631230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.553+0000 7ff806bb8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1561fe80 0x555f15622040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.553+0000 7ff806bb8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f15628a00 0x555f156225e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.553+0000 7ff806bb8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc1b0830 con 0x555f15628a00 2024-04-21T08:48:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.553+0000 7ff806bb8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc1b0160 con 0x555f1561f4d0 2024-04-21T08:48:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.553+0000 7ff806bb8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc1ad3c0 con 0x555f1561fe80 2024-04-21T08:48:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.553+0000 7ff806bb8740 10 client.0 _ll_register_callbacks cb 0x7ff7ec01cab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.553+0000 7ff802564640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f15628a00 0x555f156225e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.553+0000 7ff802564640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f15628a00 0x555f156225e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36476/0 (socket says 172.21.15.43:36476) 2024-04-21T08:48:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.553+0000 7ff802564640 1 -- 172.21.15.43:0/2822833452 learned_addr learned my addr 172.21.15.43:0/2822833452 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.553+0000 7ff801d63640 1 --2- 172.21.15.43:0/2822833452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f1561f4d0 0x555f15631230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.553+0000 7ff801562640 1 --2- 172.21.15.43:0/2822833452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1561fe80 0x555f15622040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.554+0000 7ff802564640 1 -- 172.21.15.43:0/2822833452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1561fe80 msgr2=0x555f15622040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.554+0000 7ff802564640 1 --2- 172.21.15.43:0/2822833452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1561fe80 0x555f15622040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.554+0000 7ff802564640 1 -- 172.21.15.43:0/2822833452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f1561f4d0 msgr2=0x555f15631230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.554+0000 7ff802564640 1 --2- 172.21.15.43:0/2822833452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f1561f4d0 0x555f15631230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.554+0000 7ff802564640 1 -- 172.21.15.43:0/2822833452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f15622e10 con 0x555f15628a00 2024-04-21T08:48:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.554+0000 7ff802564640 1 --2- 172.21.15.43:0/2822833452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f15628a00 0x555f156225e0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec008440 tx=0x7ff7ec008d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638031f2dc9f35e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.554+0000 7ff7e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.554+0000 7ff7e17fa640 1 -- 172.21.15.43:0/2822833452 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec039060 con 0x555f15628a00 2024-04-21T08:48:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.554+0000 7ff7e17fa640 1 -- 172.21.15.43:0/2822833452 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7ff7ec004cb0 con 0x555f15628a00 2024-04-21T08:48:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.554+0000 7ff806bb8740 1 -- 172.21.15.43:0/2822833452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7f00bf1d0 con 0x555f15628a00 2024-04-21T08:48:22.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.554+0000 7ff7e17fa640 1 -- 172.21.15.43:0/2822833452 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff7ec040050 con 0x555f15628a00 2024-04-21T08:48:22.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.554+0000 7ff806bb8740 10 client.19914 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:22.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.554+0000 7ff806bb8740 1 -- 172.21.15.43:0/2822833452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555f1569f240 con 0x555f15628a00 2024-04-21T08:48:22.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.554+0000 7ff7e17fa640 1 -- 172.21.15.43:0/2822833452 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7f00bf1d0 con 0x555f15628a00 2024-04-21T08:48:22.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.554+0000 7ff7e17fa640 1 -- 172.21.15.43:0/2822833452 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff7ec019850 con 0x555f15628a00 2024-04-21T08:48:22.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.554+0000 7ff806bb8740 10 client.19914 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:22.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.554+0000 7ff806bb8740 10 client.19914 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:48:22.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.554+0000 7ff806bb8740 20 client.19914 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:22.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.554+0000 7ff806bb8740 1 --2- 172.21.15.43:0/2822833452 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x555f1569f900 0x555f156a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.554+0000 7ff806bb8740 4 client.19914 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:22.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.554+0000 7ff806bb8740 1 -- 172.21.15.43:0/2822833452 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f15625ef0 con 0x555f1569f900 2024-04-21T08:48:22.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.555+0000 7ff801d63640 1 --2- 172.21.15.43:0/2822833452 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x555f1569f900 0x555f156a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.555+0000 7ff7e17fa640 10 client.19914 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:48:22.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.555+0000 7ff801d63640 1 --2- 172.21.15.43:0/2822833452 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x555f1569f900 0x555f156a1cf0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f09abde33be9d58d server_cookie=42b555329304c31f in_seq=0 out_seq=0 2024-04-21T08:48:22.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.555+0000 7ff7e17fa640 1 -- 172.21.15.43:0/2822833452 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f15625ef0 con 0x555f1569f900 2024-04-21T08:48:22.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.555+0000 7ff7e17fa640 10 client.19914 handle_command_reply: tid=0 2024-04-21T08:48:22.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.555+0000 7ff806bb8740 1 client.19914 shutdown 2024-04-21T08:48:22.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.555+0000 7ff806bb8740 1 -- 172.21.15.43:0/2822833452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f15628a00 msgr2=0x555f156225e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.555+0000 7ff806bb8740 1 --2- 172.21.15.43:0/2822833452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f15628a00 0x555f156225e0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec008440 tx=0x7ff7ec008d40 comp rx=0 tx=0).stop 2024-04-21T08:48:22.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.555+0000 7ff806bb8740 1 -- 172.21.15.43:0/2822833452 shutdown_connections 2024-04-21T08:48:22.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.556+0000 7ff806bb8740 1 --2- 172.21.15.43:0/2822833452 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x555f1569f900 0x555f156a1cf0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.556+0000 7ff806bb8740 1 --2- 172.21.15.43:0/2822833452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f15628a00 0x555f156225e0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.556+0000 7ff806bb8740 1 --2- 172.21.15.43:0/2822833452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555f1561fe80 0x555f15622040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.556+0000 7ff7e17fa640 0 client.19914 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:48:22.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.556+0000 7ff806bb8740 1 --2- 172.21.15.43:0/2822833452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555f1561f4d0 0x555f15631230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.556+0000 7ff806bb8740 1 -- 172.21.15.43:0/2822833452 >> 172.21.15.43:0/2822833452 conn(0x555f15619ba0 msgr2=0x555f1561b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.556+0000 7ff806bb8740 1 -- 172.21.15.43:0/2822833452 shutdown_connections 2024-04-21T08:48:22.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.556+0000 7ff806bb8740 1 -- 172.21.15.43:0/2822833452 wait complete. 2024-04-21T08:48:22.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.556+0000 7ff806bb8740 20 client.19914 trim_cache size 0 max 16384 2024-04-21T08:48:22.562 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:48:22.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.556+0000 7ff803fc7640 1 -- 172.21.15.43:0/1120157402 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff7e40a3d20 msgr2=0x7ff7e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.556+0000 7ff803fc7640 1 --2- 172.21.15.43:0/1120157402 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff7e40a3d20 0x7ff7e40a61e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec005cc0 tx=0x7ff7ec004a40 comp rx=0 tx=0).stop 2024-04-21T08:48:22.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.556+0000 7ff803fc7640 1 -- 172.21.15.43:0/1120157402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7fc107260 msgr2=0x7ff7fc198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.556+0000 7ff803fc7640 1 --2- 172.21.15.43:0/1120157402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7fc107260 0x7ff7fc198bc0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0005ec0 tx=0x7ff7f002ca30 comp rx=0 tx=0).stop 2024-04-21T08:48:22.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.556+0000 7ff803fc7640 1 -- 172.21.15.43:0/1120157402 shutdown_connections 2024-04-21T08:48:22.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.556+0000 7ff803fc7640 1 --2- 172.21.15.43:0/1120157402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7fc19cf40 0x7ff7fc19d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.556+0000 7ff803fc7640 1 --2- 172.21.15.43:0/1120157402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7fc107260 0x7ff7fc198bc0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.556+0000 7ff803fc7640 1 --2- 172.21.15.43:0/1120157402 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff7e40a3d20 0x7ff7e40a61e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.556+0000 7ff803fc7640 1 --2- 172.21.15.43:0/1120157402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7fc102190 0x7ff7fc198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.556+0000 7ff803fc7640 1 -- 172.21.15.43:0/1120157402 >> 172.21.15.43:0/1120157402 conn(0x7ff7fc0793a0 msgr2=0x7ff7fc0ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.556+0000 7ff803fc7640 1 -- 172.21.15.43:0/1120157402 shutdown_connections 2024-04-21T08:48:22.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.556+0000 7ff803fc7640 1 -- 172.21.15.43:0/1120157402 wait complete. 2024-04-21T08:48:22.568 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:48:22.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.673+0000 7f2bd4c93640 1 -- 172.21.15.43:0/3888386150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd0101aa0 msgr2=0x7f2bd0105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.673+0000 7f2bd4c93640 1 --2- 172.21.15.43:0/3888386150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd0101aa0 0x7f2bd0105a90 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009900 tx=0x7f2bb802bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:22.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.673+0000 7f2bd4c93640 1 -- 172.21.15.43:0/3888386150 shutdown_connections 2024-04-21T08:48:22.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.673+0000 7f2bd4c93640 1 --2- 172.21.15.43:0/3888386150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bd0106170 0x7f2bd010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.673+0000 7f2bd4c93640 1 --2- 172.21.15.43:0/3888386150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd0101aa0 0x7f2bd0105a90 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.673+0000 7f2bd4c93640 1 --2- 172.21.15.43:0/3888386150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bd01010f0 0x7f2bd01014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.673+0000 7f2bd4c93640 1 -- 172.21.15.43:0/3888386150 >> 172.21.15.43:0/3888386150 conn(0x7f2bd0077fb0 msgr2=0x7f2bd00783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.673+0000 7f2bd4c93640 1 -- 172.21.15.43:0/3888386150 shutdown_connections 2024-04-21T08:48:22.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.673+0000 7f2bd4c93640 1 -- 172.21.15.43:0/3888386150 wait complete. 2024-04-21T08:48:22.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.674+0000 7f2bd4c93640 1 Processor -- start 2024-04-21T08:48:22.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.674+0000 7f2bd4c93640 1 -- start start 2024-04-21T08:48:22.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.674+0000 7f2bd4c93640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bd01010f0 0x7f2bd019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.674+0000 7f2bd4c93640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd0106170 0x7f2bd019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.674+0000 7f2bd4c93640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bd0198660 0x7f2bd0198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.674+0000 7f2bd4c93640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd0111110 con 0x7f2bd0106170 2024-04-21T08:48:22.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.674+0000 7f2bd4c93640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd0110f90 con 0x7f2bd01010f0 2024-04-21T08:48:22.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.674+0000 7f2bd4c93640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd0111290 con 0x7f2bd0198660 2024-04-21T08:48:22.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.674+0000 7f2bc7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd0106170 0x7f2bd019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.674+0000 7f2bc7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd0106170 0x7f2bd019eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36492/0 (socket says 172.21.15.43:36492) 2024-04-21T08:48:22.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.674+0000 7f2bc7fff640 1 -- 172.21.15.43:0/1336144856 learned_addr learned my addr 172.21.15.43:0/1336144856 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.674+0000 7f2bced9d640 1 --2- 172.21.15.43:0/1336144856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bd0198660 0x7f2bd0198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.674+0000 7f2bce59c640 1 --2- 172.21.15.43:0/1336144856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bd01010f0 0x7f2bd019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.674+0000 7f2bc7fff640 1 -- 172.21.15.43:0/1336144856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bd0198660 msgr2=0x7f2bd0198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.674+0000 7f2bc7fff640 1 --2- 172.21.15.43:0/1336144856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bd0198660 0x7f2bd0198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.674+0000 7f2bc7fff640 1 -- 172.21.15.43:0/1336144856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bd01010f0 msgr2=0x7f2bd019e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.674+0000 7f2bc7fff640 1 --2- 172.21.15.43:0/1336144856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bd01010f0 0x7f2bd019e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.674+0000 7f2bc7fff640 1 -- 172.21.15.43:0/1336144856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd0199260 con 0x7f2bd0106170 2024-04-21T08:48:22.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.675+0000 7f2bc7fff640 1 --2- 172.21.15.43:0/1336144856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd0106170 0x7f2bd019eae0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8005ec0 tx=0x7f2bb802ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.675+0000 7f2bc77fe640 1 -- 172.21.15.43:0/1336144856 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8043020 con 0x7f2bd0106170 2024-04-21T08:48:22.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.675+0000 7f2bc77fe640 1 -- 172.21.15.43:0/1336144856 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb80052b0 con 0x7f2bd0106170 2024-04-21T08:48:22.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.675+0000 7f2bd4c93640 1 -- 172.21.15.43:0/1336144856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd01994f0 con 0x7f2bd0106170 2024-04-21T08:48:22.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.675+0000 7f2bc77fe640 1 -- 172.21.15.43:0/1336144856 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8003d40 con 0x7f2bd0106170 2024-04-21T08:48:22.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.675+0000 7f2bd4c93640 1 -- 172.21.15.43:0/1336144856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd01a5690 con 0x7f2bd0106170 2024-04-21T08:48:22.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.677+0000 7f2bc77fe640 1 -- 172.21.15.43:0/1336144856 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2bb8046020 con 0x7f2bd0106170 2024-04-21T08:48:22.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.677+0000 7f2bd7964740 1 Processor -- start 2024-04-21T08:48:22.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.677+0000 7f2bd7964740 1 -- start start 2024-04-21T08:48:22.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.678+0000 7f2bd7964740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41a9d4d0 0x559b41a9d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.678+0000 7f2bd7964740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b41a9de80 0x559b41aa6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.678+0000 7f2bd7964740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b41aa6a00 0x559b41aa8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.678+0000 7f2bd7964740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559b41a021f0 con 0x559b41a9d4d0 2024-04-21T08:48:22.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.678+0000 7f2bd7964740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559b4199f4b0 con 0x559b41aa6a00 2024-04-21T08:48:22.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.678+0000 7f2bd7964740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559b419fe1b0 con 0x559b41a9de80 2024-04-21T08:48:22.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.678+0000 7f2bced9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41a9d4d0 0x559b41a9d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.678+0000 7f2bc77fe640 1 --2- 172.21.15.43:0/1336144856 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2bbc0a3d20 0x7f2bbc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.678+0000 7f2bced9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41a9d4d0 0x559b41a9d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36504/0 (socket says 172.21.15.43:36504) 2024-04-21T08:48:22.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.678+0000 7f2bced9d640 1 -- 172.21.15.43:0/1850329059 learned_addr learned my addr 172.21.15.43:0/1850329059 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.678+0000 7f2bce59c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b41a9de80 0x559b41aa6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.678+0000 7f2bced9d640 1 --2- 172.21.15.43:0/1336144856 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2bbc0a3d20 0x7f2bbc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.678+0000 7f2bc77fe640 1 -- 172.21.15.43:0/1336144856 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f2bb80f2f90 con 0x7f2bd0106170 2024-04-21T08:48:22.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.678+0000 7f2bc7fff640 1 --2- 172.21.15.43:0/1850329059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b41aa6a00 0x559b41aa8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.678+0000 7f2bced9d640 1 -- 172.21.15.43:0/1850329059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b41a9de80 msgr2=0x559b41aa6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.678+0000 7f2bced9d640 1 --2- 172.21.15.43:0/1850329059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b41a9de80 0x559b41aa6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.678+0000 7f2bced9d640 1 -- 172.21.15.43:0/1850329059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b41aa6a00 msgr2=0x559b41aa8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.678+0000 7f2bced9d640 1 --2- 172.21.15.43:0/1850329059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b41aa6a00 0x559b41aa8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.678+0000 7f2bced9d640 1 -- 172.21.15.43:0/1850329059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b419911b0 con 0x559b41a9d4d0 2024-04-21T08:48:22.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.678+0000 7f2bce59c640 1 --2- 172.21.15.43:0/1850329059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b41a9de80 0x559b41aa6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:22.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.678+0000 7f2bced9d640 1 --2- 172.21.15.43:0/1850329059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41a9d4d0 0x559b41a9d8b0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800edd0 tx=0x7f2bc8007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37549a47c9682cde server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.678+0000 7f2bced9d640 1 --2- 172.21.15.43:0/1336144856 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2bbc0a3d20 0x7f2bbc0a61e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8007b80 tx=0x7f2bc801b000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.679+0000 7f2bc4ff9640 1 -- 172.21.15.43:0/1850329059 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8007e50 con 0x559b41a9d4d0 2024-04-21T08:48:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.679+0000 7f2bc4ff9640 1 -- 172.21.15.43:0/1850329059 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bc8009d60 con 0x559b41a9d4d0 2024-04-21T08:48:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.679+0000 7f2bc4ff9640 1 -- 172.21.15.43:0/1850329059 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc801bab0 con 0x559b41a9d4d0 2024-04-21T08:48:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.679+0000 7f2bd7964740 1 -- 172.21.15.43:0/1850329059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41a9d4d0 msgr2=0x559b41a9d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.679+0000 7f2bd7964740 1 --2- 172.21.15.43:0/1850329059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41a9d4d0 0x559b41a9d8b0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800edd0 tx=0x7f2bc8007680 comp rx=0 tx=0).stop 2024-04-21T08:48:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.679+0000 7f2bd7964740 1 -- 172.21.15.43:0/1850329059 shutdown_connections 2024-04-21T08:48:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.679+0000 7f2bd7964740 1 --2- 172.21.15.43:0/1850329059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b41aa6a00 0x559b41aa8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.679+0000 7f2bd7964740 1 --2- 172.21.15.43:0/1850329059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b41a9de80 0x559b41aa6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.679+0000 7f2bd7964740 1 --2- 172.21.15.43:0/1850329059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41a9d4d0 0x559b41a9d8b0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.679+0000 7f2bd7964740 1 -- 172.21.15.43:0/1850329059 >> 172.21.15.43:0/1850329059 conn(0x559b41a97ba0 msgr2=0x559b41a97f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.679+0000 7f2bd7964740 1 -- 172.21.15.43:0/1850329059 shutdown_connections 2024-04-21T08:48:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.679+0000 7f2bd7964740 1 -- 172.21.15.43:0/1850329059 wait complete. 2024-04-21T08:48:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.680+0000 7f2bd7964740 1 -- start start 2024-04-21T08:48:22.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.680+0000 7f2bd7964740 1 Processor -- start 2024-04-21T08:48:22.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.680+0000 7f2bd7964740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41a9d4d0 0x559b41aa6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.680+0000 7f2bd7964740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b41a9de80 0x559b41aa36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.680+0000 7f2bd7964740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b41aa6a00 0x559b41aa3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.680+0000 7f2bd7964740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559b41aa9910 con 0x559b41a9d4d0 2024-04-21T08:48:22.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.680+0000 7f2bd7964740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559b419bb500 con 0x559b41aa6a00 2024-04-21T08:48:22.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.680+0000 7f2bd7964740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559b41aa9a90 con 0x559b41a9de80 2024-04-21T08:48:22.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.680+0000 7f2bd7964740 10 client.0 _ll_register_callbacks cb 0x559b41989b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:22.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.680+0000 7f2bce59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41a9d4d0 0x559b41aa6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.680+0000 7f2bce59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41a9d4d0 0x559b41aa6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36516/0 (socket says 172.21.15.43:36516) 2024-04-21T08:48:22.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.680+0000 7f2bce59c640 1 -- 172.21.15.43:0/3180142194 learned_addr learned my addr 172.21.15.43:0/3180142194 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.680+0000 7f2bc7fff640 1 --2- 172.21.15.43:0/3180142194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b41a9de80 0x559b41aa36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.680+0000 7f2bced9d640 1 --2- 172.21.15.43:0/3180142194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b41aa6a00 0x559b41aa3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.680+0000 7f2bce59c640 1 -- 172.21.15.43:0/3180142194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b41a9de80 msgr2=0x559b41aa36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.680+0000 7f2bce59c640 1 --2- 172.21.15.43:0/3180142194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b41a9de80 0x559b41aa36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.680+0000 7f2bce59c640 1 -- 172.21.15.43:0/3180142194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b41aa6a00 msgr2=0x559b41aa3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.680+0000 7f2bce59c640 1 --2- 172.21.15.43:0/3180142194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b41aa6a00 0x559b41aa3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.680+0000 7f2bce59c640 1 -- 172.21.15.43:0/3180142194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b417143f0 con 0x559b41a9d4d0 2024-04-21T08:48:22.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.680+0000 7f2bc7fff640 1 --2- 172.21.15.43:0/3180142194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b41a9de80 0x559b41aa36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:22.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.680+0000 7f2bced9d640 1 --2- 172.21.15.43:0/3180142194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b41aa6a00 0x559b41aa3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:22.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.681+0000 7f2bce59c640 1 --2- 172.21.15.43:0/3180142194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41a9d4d0 0x559b41aa6340 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00116c0 tx=0x7f2bc0011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdc4355ac224442 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.681+0000 7f2bb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:22.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.681+0000 7f2bb5ffb640 1 -- 172.21.15.43:0/3180142194 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc000fe70 con 0x559b41a9d4d0 2024-04-21T08:48:22.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.681+0000 7f2bb5ffb640 1 -- 172.21.15.43:0/3180142194 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f2bc0005020 con 0x559b41a9d4d0 2024-04-21T08:48:22.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.681+0000 7f2bd7964740 1 -- 172.21.15.43:0/3180142194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bc8009d60 con 0x559b41a9d4d0 2024-04-21T08:48:22.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.681+0000 7f2bb5ffb640 1 -- 172.21.15.43:0/3180142194 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2bc0024050 con 0x559b41a9d4d0 2024-04-21T08:48:22.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.681+0000 7f2bd7964740 10 client.19926 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:22.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.681+0000 7f2bd7964740 1 -- 172.21.15.43:0/3180142194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559b417de7d0 con 0x559b41a9d4d0 2024-04-21T08:48:22.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.681+0000 7f2bb5ffb640 1 -- 172.21.15.43:0/3180142194 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bc8009d60 con 0x559b41a9d4d0 2024-04-21T08:48:22.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.681+0000 7f2bb5ffb640 1 -- 172.21.15.43:0/3180142194 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2bc0018030 con 0x559b41a9d4d0 2024-04-21T08:48:22.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.681+0000 7f2bd7964740 10 client.19926 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:22.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.681+0000 7f2bd7964740 10 client.19926 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:48:22.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.681+0000 7f2bd7964740 20 client.19926 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.681+0000 7f2bd7964740 1 --2- 172.21.15.43:0/3180142194 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x559b41ab6d50 0x559b41ab91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.681+0000 7f2bd7964740 4 client.19926 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.681+0000 7f2bd7964740 1 -- 172.21.15.43:0/3180142194 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b419a1930 con 0x559b41ab6d50 2024-04-21T08:48:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.682+0000 7f2bc7fff640 1 --2- 172.21.15.43:0/3180142194 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x559b41ab6d50 0x559b41ab91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.682+0000 7f2bb5ffb640 10 client.19926 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:48:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.682+0000 7f2bc7fff640 1 --2- 172.21.15.43:0/3180142194 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x559b41ab6d50 0x559b41ab91c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fd3ec48152b6e7d6 server_cookie=34508fdfd8e9efb6 in_seq=0 out_seq=0 2024-04-21T08:48:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.683+0000 7f2bb5ffb640 1 -- 172.21.15.43:0/3180142194 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559b419a1930 con 0x559b41ab6d50 2024-04-21T08:48:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.683+0000 7f2bb5ffb640 10 client.19926 handle_command_reply: tid=0 2024-04-21T08:48:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.683+0000 7f2bd7964740 1 client.19926 shutdown 2024-04-21T08:48:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.683+0000 7f2bd7964740 1 -- 172.21.15.43:0/3180142194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41a9d4d0 msgr2=0x559b41aa6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.683+0000 7f2bd7964740 1 --2- 172.21.15.43:0/3180142194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41a9d4d0 0x559b41aa6340 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00116c0 tx=0x7f2bc0011b80 comp rx=0 tx=0).stop 2024-04-21T08:48:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.683+0000 7f2bd7964740 1 -- 172.21.15.43:0/3180142194 shutdown_connections 2024-04-21T08:48:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.683+0000 7f2bd7964740 1 --2- 172.21.15.43:0/3180142194 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x559b41ab6d50 0x559b41ab91c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.683+0000 7f2bd7964740 1 --2- 172.21.15.43:0/3180142194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b41aa6a00 0x559b41aa3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.683+0000 7f2bd7964740 1 --2- 172.21.15.43:0/3180142194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b41a9de80 0x559b41aa36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.683+0000 7f2bb5ffb640 0 client.19926 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:48:22.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.683+0000 7f2bd7964740 1 --2- 172.21.15.43:0/3180142194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41a9d4d0 0x559b41aa6340 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.683+0000 7f2bd7964740 1 -- 172.21.15.43:0/3180142194 >> 172.21.15.43:0/3180142194 conn(0x559b41a97ba0 msgr2=0x559b41a998f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.683+0000 7f2bd7964740 1 -- 172.21.15.43:0/3180142194 shutdown_connections 2024-04-21T08:48:22.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.684+0000 7f2bd7964740 1 -- 172.21.15.43:0/3180142194 wait complete. 2024-04-21T08:48:22.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.684+0000 7f2bd7964740 20 client.19926 trim_cache size 0 max 16384 2024-04-21T08:48:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.697+0000 7f2bd7964740 1 Processor -- start 2024-04-21T08:48:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.697+0000 7f2bd7964740 1 -- start start 2024-04-21T08:48:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.697+0000 7f2bd7964740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b41a9d4d0 0x559b41ab2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.697+0000 7f2bd7964740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41a9de80 0x559b41aa2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.697+0000 7f2bd7964740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b41aa6a00 0x559b41aaf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.697+0000 7f2bd7964740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd01af530 con 0x559b41a9de80 2024-04-21T08:48:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.697+0000 7f2bd7964740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd01b1070 con 0x559b41a9d4d0 2024-04-21T08:48:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.697+0000 7f2bd7964740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd01afc00 con 0x559b41aa6a00 2024-04-21T08:48:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.697+0000 7f2bc7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41a9de80 0x559b41aa2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.697+0000 7f2bc7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41a9de80 0x559b41aa2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36520/0 (socket says 172.21.15.43:36520) 2024-04-21T08:48:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.697+0000 7f2bc7fff640 1 -- 172.21.15.43:0/2756823858 learned_addr learned my addr 172.21.15.43:0/2756823858 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.697+0000 7f2bced9d640 1 --2- 172.21.15.43:0/2756823858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b41aa6a00 0x559b41aaf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.697+0000 7f2bce59c640 1 --2- 172.21.15.43:0/2756823858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b41a9d4d0 0x559b41ab2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.697+0000 7f2bc7fff640 1 -- 172.21.15.43:0/2756823858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b41aa6a00 msgr2=0x559b41aaf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.697+0000 7f2bc7fff640 1 --2- 172.21.15.43:0/2756823858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b41aa6a00 0x559b41aaf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.697+0000 7f2bc7fff640 1 -- 172.21.15.43:0/2756823858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b41a9d4d0 msgr2=0x559b41ab2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.697+0000 7f2bc7fff640 1 --2- 172.21.15.43:0/2756823858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b41a9d4d0 0x559b41ab2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.697+0000 7f2bc7fff640 1 -- 172.21.15.43:0/2756823858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b41aa4050 con 0x559b41a9de80 2024-04-21T08:48:22.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.697+0000 7f2bce59c640 1 --2- 172.21.15.43:0/2756823858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b41a9d4d0 0x559b41ab2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:22.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.697+0000 7f2bced9d640 1 --2- 172.21.15.43:0/2756823858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b41aa6a00 0x559b41aaf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:22.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.698+0000 7f2bc7fff640 1 --2- 172.21.15.43:0/2756823858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41a9de80 0x559b41aa2940 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80bb590 tx=0x7f2bb80baea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb24143b0ecad734 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.698+0000 7f2bb7fff640 1 -- 172.21.15.43:0/2756823858 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb80bebc0 con 0x559b41a9de80 2024-04-21T08:48:22.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.698+0000 7f2bb7fff640 1 -- 172.21.15.43:0/2756823858 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb80c2f70 con 0x559b41a9de80 2024-04-21T08:48:22.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.698+0000 7f2bb7fff640 1 -- 172.21.15.43:0/2756823858 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8005460 con 0x559b41a9de80 2024-04-21T08:48:22.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.698+0000 7f2bd7964740 1 -- 172.21.15.43:0/2756823858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41a9de80 msgr2=0x559b41aa2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.698+0000 7f2bd7964740 1 --2- 172.21.15.43:0/2756823858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41a9de80 0x559b41aa2940 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80bb590 tx=0x7f2bb80baea0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.698+0000 7f2bd7964740 1 -- 172.21.15.43:0/2756823858 shutdown_connections 2024-04-21T08:48:22.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.698+0000 7f2bd7964740 1 --2- 172.21.15.43:0/2756823858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b41aa6a00 0x559b41aaf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.698+0000 7f2bd7964740 1 --2- 172.21.15.43:0/2756823858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41a9de80 0x559b41aa2940 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.698+0000 7f2bd7964740 1 --2- 172.21.15.43:0/2756823858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b41a9d4d0 0x559b41ab2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.698+0000 7f2bd7964740 1 -- 172.21.15.43:0/2756823858 >> 172.21.15.43:0/2756823858 conn(0x559b41a97ba0 msgr2=0x559b41a997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.698+0000 7f2bd7964740 1 -- 172.21.15.43:0/2756823858 shutdown_connections 2024-04-21T08:48:22.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.698+0000 7f2bd7964740 1 -- 172.21.15.43:0/2756823858 wait complete. 2024-04-21T08:48:22.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.699+0000 7f2bd7964740 1 -- start start 2024-04-21T08:48:22.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.699+0000 7f2bd7964740 1 Processor -- start 2024-04-21T08:48:22.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.699+0000 7f2bd7964740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b41a9d4d0 0x559b41b154b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.699+0000 7f2bd7964740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b41a9de80 0x559b41b159f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.699+0000 7f2bd7964740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41aa6a00 0x559b41b1a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.699+0000 7f2bd7964740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd01b09a0 con 0x559b41aa6a00 2024-04-21T08:48:22.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.699+0000 7f2bd7964740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd01b02d0 con 0x559b41a9d4d0 2024-04-21T08:48:22.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.699+0000 7f2bd7964740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd01ad530 con 0x559b41a9de80 2024-04-21T08:48:22.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.699+0000 7f2bd7964740 10 client.0 _ll_register_callbacks cb 0x7f2bc000cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:22.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.699+0000 7f2bced9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41aa6a00 0x559b41b1a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.699+0000 7f2bced9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41aa6a00 0x559b41b1a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36522/0 (socket says 172.21.15.43:36522) 2024-04-21T08:48:22.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.699+0000 7f2bced9d640 1 -- 172.21.15.43:0/4182941427 learned_addr learned my addr 172.21.15.43:0/4182941427 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.699+0000 7f2bc7fff640 1 --2- 172.21.15.43:0/4182941427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b41a9de80 0x559b41b159f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.699+0000 7f2bce59c640 1 --2- 172.21.15.43:0/4182941427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b41a9d4d0 0x559b41b154b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.699+0000 7f2bced9d640 1 -- 172.21.15.43:0/4182941427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b41a9de80 msgr2=0x559b41b159f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.699+0000 7f2bced9d640 1 --2- 172.21.15.43:0/4182941427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b41a9de80 0x559b41b159f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.699+0000 7f2bced9d640 1 -- 172.21.15.43:0/4182941427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b41a9d4d0 msgr2=0x559b41b154b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.699+0000 7f2bced9d640 1 --2- 172.21.15.43:0/4182941427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b41a9d4d0 0x559b41b154b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.699+0000 7f2bced9d640 1 -- 172.21.15.43:0/4182941427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b41b1a970 con 0x559b41aa6a00 2024-04-21T08:48:22.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.700+0000 7f2bced9d640 1 --2- 172.21.15.43:0/4182941427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41aa6a00 0x559b41b1a380 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800edd0 tx=0x7f2bc800e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4e2b4c84c36685 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.700+0000 7f2bb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:22.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.700+0000 7f2bb5ffb640 1 -- 172.21.15.43:0/4182941427 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8030060 con 0x559b41aa6a00 2024-04-21T08:48:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.700+0000 7f2bb5ffb640 1 -- 172.21.15.43:0/4182941427 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f2bc8009d60 con 0x559b41aa6a00 2024-04-21T08:48:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.700+0000 7f2bd7964740 1 -- 172.21.15.43:0/4182941427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bb80c2f70 con 0x559b41aa6a00 2024-04-21T08:48:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.700+0000 7f2bb5ffb640 1 -- 172.21.15.43:0/4182941427 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2bc8037050 con 0x559b41aa6a00 2024-04-21T08:48:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.700+0000 7f2bd7964740 10 client.19932 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.700+0000 7f2bd7964740 1 -- 172.21.15.43:0/4182941427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559b41b21c80 con 0x559b41aa6a00 2024-04-21T08:48:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.700+0000 7f2bb5ffb640 1 -- 172.21.15.43:0/4182941427 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb80c2f70 con 0x559b41aa6a00 2024-04-21T08:48:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.700+0000 7f2bb5ffb640 1 -- 172.21.15.43:0/4182941427 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2bc802ba80 con 0x559b41aa6a00 2024-04-21T08:48:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.700+0000 7f2bd7964740 10 client.19932 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.700+0000 7f2bd7964740 10 client.19932 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:48:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.700+0000 7f2bd7964740 20 client.19932 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.700+0000 7f2bd7964740 1 --2- 172.21.15.43:0/4182941427 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x559b41b22230 0x559b41b24650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.700+0000 7f2bd7964740 4 client.19932 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.700+0000 7f2bd7964740 1 -- 172.21.15.43:0/4182941427 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b41a0fc10 con 0x559b41b22230 2024-04-21T08:48:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.701+0000 7f2bce59c640 1 --2- 172.21.15.43:0/4182941427 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x559b41b22230 0x559b41b24650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.701+0000 7f2bce59c640 1 --2- 172.21.15.43:0/4182941427 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x559b41b22230 0x559b41b24650 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d3d4b81215b1d4fa server_cookie=9a539f6fe6c4d038 in_seq=0 out_seq=0 2024-04-21T08:48:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.701+0000 7f2bb5ffb640 10 client.19932 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:48:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.701+0000 7f2bb5ffb640 1 -- 172.21.15.43:0/4182941427 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b41a0fc10 con 0x559b41b22230 2024-04-21T08:48:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.701+0000 7f2bb5ffb640 10 client.19932 handle_command_reply: tid=0 2024-04-21T08:48:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.701+0000 7f2bd7964740 1 client.19932 shutdown 2024-04-21T08:48:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.701+0000 7f2bd7964740 1 -- 172.21.15.43:0/4182941427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41aa6a00 msgr2=0x559b41b1a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.701+0000 7f2bd7964740 1 --2- 172.21.15.43:0/4182941427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41aa6a00 0x559b41b1a380 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800edd0 tx=0x7f2bc800e710 comp rx=0 tx=0).stop 2024-04-21T08:48:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.701+0000 7f2bd7964740 1 -- 172.21.15.43:0/4182941427 shutdown_connections 2024-04-21T08:48:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.701+0000 7f2bd7964740 1 --2- 172.21.15.43:0/4182941427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559b41aa6a00 0x559b41b1a380 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.701+0000 7f2bd7964740 1 --2- 172.21.15.43:0/4182941427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b41a9de80 0x559b41b159f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.701+0000 7f2bd7964740 1 --2- 172.21.15.43:0/4182941427 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x559b41b22230 0x559b41b24650 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.701+0000 7f2bd7964740 1 --2- 172.21.15.43:0/4182941427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b41a9d4d0 0x559b41b154b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.701+0000 7f2bb5ffb640 0 client.19932 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:48:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.701+0000 7f2bd7964740 1 -- 172.21.15.43:0/4182941427 >> 172.21.15.43:0/4182941427 conn(0x559b41a97ba0 msgr2=0x559b41a9e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.701+0000 7f2bd7964740 1 -- 172.21.15.43:0/4182941427 shutdown_connections 2024-04-21T08:48:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.701+0000 7f2bd7964740 1 -- 172.21.15.43:0/4182941427 wait complete. 2024-04-21T08:48:22.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.701+0000 7f2bd7964740 20 client.19932 trim_cache size 0 max 16384 2024-04-21T08:48:22.708 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:48:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.702+0000 7f2bd4c93640 1 -- 172.21.15.43:0/1336144856 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2bbc0a3d20 msgr2=0x7f2bbc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.702+0000 7f2bd4c93640 1 --2- 172.21.15.43:0/1336144856 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2bbc0a3d20 0x7f2bbc0a61e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8007b80 tx=0x7f2bc801b000 comp rx=0 tx=0).stop 2024-04-21T08:48:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.702+0000 7f2bd4c93640 1 -- 172.21.15.43:0/1336144856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd0106170 msgr2=0x7f2bd019eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.702+0000 7f2bd4c93640 1 --2- 172.21.15.43:0/1336144856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd0106170 0x7f2bd019eae0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8005ec0 tx=0x7f2bb802ca10 comp rx=0 tx=0).stop 2024-04-21T08:48:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.702+0000 7f2bd4c93640 1 -- 172.21.15.43:0/1336144856 shutdown_connections 2024-04-21T08:48:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.702+0000 7f2bd4c93640 1 --2- 172.21.15.43:0/1336144856 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2bbc0a3d20 0x7f2bbc0a61e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.702+0000 7f2bd4c93640 1 --2- 172.21.15.43:0/1336144856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bd0198660 0x7f2bd0198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.702+0000 7f2bd4c93640 1 --2- 172.21.15.43:0/1336144856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd0106170 0x7f2bd019eae0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.702+0000 7f2bd4c93640 1 --2- 172.21.15.43:0/1336144856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bd01010f0 0x7f2bd019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.702+0000 7f2bd4c93640 1 -- 172.21.15.43:0/1336144856 >> 172.21.15.43:0/1336144856 conn(0x7f2bd0077fb0 msgr2=0x7f2bd006a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.702+0000 7f2bd4c93640 1 -- 172.21.15.43:0/1336144856 shutdown_connections 2024-04-21T08:48:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.702+0000 7f2bd4c93640 1 -- 172.21.15.43:0/1336144856 wait complete. 2024-04-21T08:48:22.712 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:48:22.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.817+0000 7f033c211640 1 -- 172.21.15.43:0/2935530821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0334107270 msgr2=0x7f033410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.817+0000 7f033c211640 1 --2- 172.21.15.43:0/2935530821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0334107270 0x7f033410ee40 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f032c009ca0 tx=0x7f032c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.817+0000 7f033c211640 1 -- 172.21.15.43:0/2935530821 shutdown_connections 2024-04-21T08:48:22.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.817+0000 7f033c211640 1 --2- 172.21.15.43:0/2935530821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0334107270 0x7f033410ee40 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.817+0000 7f033c211640 1 --2- 172.21.15.43:0/2935530821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f033406b1f0 0x7f0334106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.817+0000 7f033c211640 1 --2- 172.21.15.43:0/2935530821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f033406a840 0x7f033406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.817+0000 7f033c211640 1 -- 172.21.15.43:0/2935530821 >> 172.21.15.43:0/2935530821 conn(0x7f0334077eb0 msgr2=0x7f03340782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.817+0000 7f033c211640 1 -- 172.21.15.43:0/2935530821 shutdown_connections 2024-04-21T08:48:22.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.817+0000 7f033c211640 1 -- 172.21.15.43:0/2935530821 wait complete. 2024-04-21T08:48:22.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.818+0000 7f033c211640 1 Processor -- start 2024-04-21T08:48:22.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.818+0000 7f033c211640 1 -- start start 2024-04-21T08:48:22.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.818+0000 7f033c211640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f033406a840 0x7f03341986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.818+0000 7f033c211640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0334107270 0x7f0334198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.818+0000 7f033c211640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f033419cf90 0x7f033419d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.818+0000 7f033c211640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f03341121d0 con 0x7f033419cf90 2024-04-21T08:48:22.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.818+0000 7f033c211640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0334112050 con 0x7f0334107270 2024-04-21T08:48:22.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.818+0000 7f033c211640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0334112350 con 0x7f033406a840 2024-04-21T08:48:22.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.818+0000 7f033a7ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f033419cf90 0x7f033419d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.818+0000 7f033a7ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f033419cf90 0x7f033419d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36538/0 (socket says 172.21.15.43:36538) 2024-04-21T08:48:22.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.818+0000 7f033a7ae640 1 -- 172.21.15.43:0/508268379 learned_addr learned my addr 172.21.15.43:0/508268379 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.818+0000 7f03397ac640 1 --2- 172.21.15.43:0/508268379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0334107270 0x7f0334198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.818+0000 7f033a7ae640 1 -- 172.21.15.43:0/508268379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f033406a840 msgr2=0x7f03341986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.818+0000 7f033a7ae640 1 --2- 172.21.15.43:0/508268379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f033406a840 0x7f03341986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.818+0000 7f033a7ae640 1 -- 172.21.15.43:0/508268379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0334107270 msgr2=0x7f0334198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.818+0000 7f033a7ae640 1 --2- 172.21.15.43:0/508268379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0334107270 0x7f0334198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.818+0000 7f033a7ae640 1 -- 172.21.15.43:0/508268379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033419da80 con 0x7f033419cf90 2024-04-21T08:48:22.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.818+0000 7f03397ac640 1 --2- 172.21.15.43:0/508268379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0334107270 0x7f0334198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:22.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.819+0000 7f033a7ae640 1 --2- 172.21.15.43:0/508268379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f033419cf90 0x7f033419d440 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f032c005e30 tx=0x7f032c034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.819+0000 7f0326ffd640 1 -- 172.21.15.43:0/508268379 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c039040 con 0x7f033419cf90 2024-04-21T08:48:22.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.819+0000 7f0326ffd640 1 -- 172.21.15.43:0/508268379 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f032c044070 con 0x7f033419cf90 2024-04-21T08:48:22.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.819+0000 7f033c211640 1 -- 172.21.15.43:0/508268379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f033419dd10 con 0x7f033419cf90 2024-04-21T08:48:22.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.819+0000 7f0326ffd640 1 -- 172.21.15.43:0/508268379 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c003e60 con 0x7f033419cf90 2024-04-21T08:48:22.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.819+0000 7f033c211640 1 -- 172.21.15.43:0/508268379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03341a54d0 con 0x7f033419cf90 2024-04-21T08:48:22.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.821+0000 7f0326ffd640 1 -- 172.21.15.43:0/508268379 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f032c0324c0 con 0x7f033419cf90 2024-04-21T08:48:22.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.821+0000 7f033ee02740 1 Processor -- start 2024-04-21T08:48:22.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.822+0000 7f033ee02740 1 -- start start 2024-04-21T08:48:22.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.822+0000 7f033ee02740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a483b0 0x564c81a48790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.822+0000 7f033ee02740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c81a48d60 0x564c81a51890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.822+0000 7f033ee02740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c81a51e60 0x564c81a522c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.822+0000 7f033ee02740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564c819ad1f0 con 0x564c81a483b0 2024-04-21T08:48:22.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.822+0000 7f033ee02740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564c8194a4b0 con 0x564c81a51e60 2024-04-21T08:48:22.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.822+0000 7f033ee02740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564c819a91b0 con 0x564c81a48d60 2024-04-21T08:48:22.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.822+0000 7f0326ffd640 1 --2- 172.21.15.43:0/508268379 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f03200a3d20 0x7f03200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.822+0000 7f03397ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a483b0 0x564c81a48790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.822+0000 7f03397ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a483b0 0x564c81a48790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36550/0 (socket says 172.21.15.43:36550) 2024-04-21T08:48:22.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.822+0000 7f03397ac640 1 -- 172.21.15.43:0/1976494620 learned_addr learned my addr 172.21.15.43:0/1976494620 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.822+0000 7f033a7ae640 1 --2- 172.21.15.43:0/508268379 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f03200a3d20 0x7f03200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.822+0000 7f03397ac640 1 --2- 172.21.15.43:0/1976494620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c81a51e60 0x564c81a522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.822+0000 7f0339fad640 1 --2- 172.21.15.43:0/1976494620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c81a48d60 0x564c81a51890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.822+0000 7f0326ffd640 1 -- 172.21.15.43:0/508268379 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f032c0f2f00 con 0x7f033419cf90 2024-04-21T08:48:22.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.822+0000 7f03397ac640 1 -- 172.21.15.43:0/1976494620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c81a48d60 msgr2=0x564c81a51890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.822+0000 7f03397ac640 1 --2- 172.21.15.43:0/1976494620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c81a48d60 0x564c81a51890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.822+0000 7f03397ac640 1 -- 172.21.15.43:0/1976494620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c81a51e60 msgr2=0x564c81a522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.822+0000 7f03397ac640 1 --2- 172.21.15.43:0/1976494620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c81a51e60 0x564c81a522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.822+0000 7f03397ac640 1 -- 172.21.15.43:0/1976494620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c8193c1b0 con 0x564c81a483b0 2024-04-21T08:48:22.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.823+0000 7f03397ac640 1 --2- 172.21.15.43:0/1976494620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a483b0 0x564c81a48790 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f0330013890 tx=0x7f033000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5c170af06b1510 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.823+0000 7f033a7ae640 1 --2- 172.21.15.43:0/508268379 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f03200a3d20 0x7f03200a61e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f032c0be830 tx=0x7f032c0c1040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.823+0000 7f031ffff640 1 -- 172.21.15.43:0/1976494620 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033000aac0 con 0x564c81a483b0 2024-04-21T08:48:22.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.823+0000 7f031ffff640 1 -- 172.21.15.43:0/1976494620 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03300054d0 con 0x564c81a483b0 2024-04-21T08:48:22.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.823+0000 7f031ffff640 1 -- 172.21.15.43:0/1976494620 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03300049a0 con 0x564c81a483b0 2024-04-21T08:48:22.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.824+0000 7f033ee02740 1 -- 172.21.15.43:0/1976494620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a483b0 msgr2=0x564c81a48790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.824+0000 7f033ee02740 1 --2- 172.21.15.43:0/1976494620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a483b0 0x564c81a48790 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f0330013890 tx=0x7f033000c3a0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.824+0000 7f033ee02740 1 -- 172.21.15.43:0/1976494620 shutdown_connections 2024-04-21T08:48:22.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.824+0000 7f033ee02740 1 --2- 172.21.15.43:0/1976494620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c81a51e60 0x564c81a522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.824+0000 7f033ee02740 1 --2- 172.21.15.43:0/1976494620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c81a48d60 0x564c81a51890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.824+0000 7f033ee02740 1 --2- 172.21.15.43:0/1976494620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a483b0 0x564c81a48790 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.824+0000 7f033ee02740 1 -- 172.21.15.43:0/1976494620 >> 172.21.15.43:0/1976494620 conn(0x564c81a42ba0 msgr2=0x564c81a42f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.824+0000 7f033ee02740 1 -- 172.21.15.43:0/1976494620 shutdown_connections 2024-04-21T08:48:22.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.824+0000 7f033ee02740 1 -- 172.21.15.43:0/1976494620 wait complete. 2024-04-21T08:48:22.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.824+0000 7f033ee02740 1 -- start start 2024-04-21T08:48:22.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.824+0000 7f033ee02740 1 Processor -- start 2024-04-21T08:48:22.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.825+0000 7f033ee02740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c81a483b0 0x564c81a58190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.825+0000 7f033ee02740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a48d60 0x564c81a55590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.825+0000 7f033ee02740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c81a51e60 0x564c81a55ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.825+0000 7f033ee02740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564c81a58da0 con 0x564c81a48d60 2024-04-21T08:48:22.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.825+0000 7f033ee02740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564c81966500 con 0x564c81a51e60 2024-04-21T08:48:22.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.825+0000 7f033ee02740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564c81a58f20 con 0x564c81a483b0 2024-04-21T08:48:22.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.825+0000 7f033ee02740 10 client.0 _ll_register_callbacks cb 0x564c81934b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:22.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.825+0000 7f0339fad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a48d60 0x564c81a55590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.825+0000 7f0339fad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a48d60 0x564c81a55590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36566/0 (socket says 172.21.15.43:36566) 2024-04-21T08:48:22.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.825+0000 7f0339fad640 1 -- 172.21.15.43:0/2707572652 learned_addr learned my addr 172.21.15.43:0/2707572652 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.825+0000 7f03397ac640 1 --2- 172.21.15.43:0/2707572652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c81a51e60 0x564c81a55ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.825+0000 7f03397ac640 1 --2- 172.21.15.43:0/2707572652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c81a483b0 0x564c81a58190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.825+0000 7f0339fad640 1 -- 172.21.15.43:0/2707572652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c81a483b0 msgr2=0x564c81a58190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.825+0000 7f0339fad640 1 --2- 172.21.15.43:0/2707572652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c81a483b0 0x564c81a58190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.825+0000 7f0339fad640 1 -- 172.21.15.43:0/2707572652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c81a51e60 msgr2=0x564c81a55ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.825+0000 7f0339fad640 1 --2- 172.21.15.43:0/2707572652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c81a51e60 0x564c81a55ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.825+0000 7f0339fad640 1 -- 172.21.15.43:0/2707572652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c816bf3f0 con 0x564c81a48d60 2024-04-21T08:48:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.825+0000 7f03397ac640 1 --2- 172.21.15.43:0/2707572652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c81a51e60 0x564c81a55ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.825+0000 7f0339fad640 1 --2- 172.21.15.43:0/2707572652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a48d60 0x564c81a55590 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f03280038a0 tx=0x7f0328003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b00363d69d390d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.825+0000 7f031d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.825+0000 7f031d7fa640 1 -- 172.21.15.43:0/2707572652 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032801a470 con 0x564c81a48d60 2024-04-21T08:48:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.825+0000 7f031d7fa640 1 -- 172.21.15.43:0/2707572652 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f032801a610 con 0x564c81a48d60 2024-04-21T08:48:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.825+0000 7f033ee02740 1 -- 172.21.15.43:0/2707572652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03300054d0 con 0x564c81a48d60 2024-04-21T08:48:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.825+0000 7f031d7fa640 1 -- 172.21.15.43:0/2707572652 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f032801a810 con 0x564c81a48d60 2024-04-21T08:48:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.826+0000 7f033ee02740 10 client.19944 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:22.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.826+0000 7f033ee02740 1 -- 172.21.15.43:0/2707572652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564c817897d0 con 0x564c81a48d60 2024-04-21T08:48:22.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.826+0000 7f031d7fa640 1 -- 172.21.15.43:0/2707572652 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03300054d0 con 0x564c81a48d60 2024-04-21T08:48:22.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.826+0000 7f031d7fa640 1 -- 172.21.15.43:0/2707572652 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0328011b60 con 0x564c81a48d60 2024-04-21T08:48:22.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.826+0000 7f033ee02740 10 client.19944 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:22.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.826+0000 7f033ee02740 10 client.19944 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:48:22.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.826+0000 7f033ee02740 20 client.19944 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:22.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.826+0000 7f033ee02740 1 --2- 172.21.15.43:0/2707572652 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x564c81a660e0 0x564c81a684d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.826+0000 7f033ee02740 4 client.19944 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:22.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.826+0000 7f033ee02740 1 -- 172.21.15.43:0/2707572652 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c8194c930 con 0x564c81a660e0 2024-04-21T08:48:22.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.826+0000 7f03397ac640 1 --2- 172.21.15.43:0/2707572652 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x564c81a660e0 0x564c81a684d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.827+0000 7f031d7fa640 10 client.19944 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:48:22.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.827+0000 7f03397ac640 1 --2- 172.21.15.43:0/2707572652 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x564c81a660e0 0x564c81a684d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ff6eef591a8fd485 server_cookie=7d4371be38b5254d in_seq=0 out_seq=0 2024-04-21T08:48:22.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.827+0000 7f031d7fa640 1 -- 172.21.15.43:0/2707572652 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564c8194c930 con 0x564c81a660e0 2024-04-21T08:48:22.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.827+0000 7f031d7fa640 10 client.19944 handle_command_reply: tid=0 2024-04-21T08:48:22.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.827+0000 7f033ee02740 1 client.19944 shutdown 2024-04-21T08:48:22.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.828+0000 7f033ee02740 1 -- 172.21.15.43:0/2707572652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a48d60 msgr2=0x564c81a55590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.828+0000 7f033ee02740 1 --2- 172.21.15.43:0/2707572652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a48d60 0x564c81a55590 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f03280038a0 tx=0x7f0328003d60 comp rx=0 tx=0).stop 2024-04-21T08:48:22.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.828+0000 7f033ee02740 1 -- 172.21.15.43:0/2707572652 shutdown_connections 2024-04-21T08:48:22.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.828+0000 7f033ee02740 1 --2- 172.21.15.43:0/2707572652 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x564c81a660e0 0x564c81a684d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.828+0000 7f033ee02740 1 --2- 172.21.15.43:0/2707572652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c81a51e60 0x564c81a55ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.828+0000 7f033ee02740 1 --2- 172.21.15.43:0/2707572652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a48d60 0x564c81a55590 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.828+0000 7f031d7fa640 0 client.19944 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:48:22.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.828+0000 7f033ee02740 1 --2- 172.21.15.43:0/2707572652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c81a483b0 0x564c81a58190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.828+0000 7f033ee02740 1 -- 172.21.15.43:0/2707572652 >> 172.21.15.43:0/2707572652 conn(0x564c81a42ba0 msgr2=0x564c81a448f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.828+0000 7f033ee02740 1 -- 172.21.15.43:0/2707572652 shutdown_connections 2024-04-21T08:48:22.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.828+0000 7f033ee02740 1 -- 172.21.15.43:0/2707572652 wait complete. 2024-04-21T08:48:22.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.828+0000 7f033ee02740 20 client.19944 trim_cache size 0 max 16384 2024-04-21T08:48:22.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.841+0000 7f033ee02740 1 Processor -- start 2024-04-21T08:48:22.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.841+0000 7f033ee02740 1 -- start start 2024-04-21T08:48:22.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.841+0000 7f033ee02740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a483b0 0x564c81a53250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.841+0000 7f033ee02740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c81a48d60 0x564c81a61d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.841+0000 7f033ee02740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c81a51e60 0x564c81a54ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.841+0000 7f033ee02740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f03341ad2b0 con 0x564c81a483b0 2024-04-21T08:48:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.841+0000 7f033ee02740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f03341aedf0 con 0x564c81a48d60 2024-04-21T08:48:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.841+0000 7f033ee02740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f03341ad980 con 0x564c81a51e60 2024-04-21T08:48:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.841+0000 7f03397ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a483b0 0x564c81a53250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.841+0000 7f03397ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a483b0 0x564c81a53250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36570/0 (socket says 172.21.15.43:36570) 2024-04-21T08:48:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.841+0000 7f03397ac640 1 -- 172.21.15.43:0/1127144065 learned_addr learned my addr 172.21.15.43:0/1127144065 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.842+0000 7f03397ac640 1 -- 172.21.15.43:0/1127144065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c81a51e60 msgr2=0x564c81a54ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.842+0000 7f0339fad640 1 --2- 172.21.15.43:0/1127144065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c81a48d60 0x564c81a61d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.842+0000 7f03397ac640 1 --2- 172.21.15.43:0/1127144065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c81a51e60 0x564c81a54ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.842+0000 7f03397ac640 1 -- 172.21.15.43:0/1127144065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c81a48d60 msgr2=0x564c81a61d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.842+0000 7f03397ac640 1 --2- 172.21.15.43:0/1127144065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c81a48d60 0x564c81a61d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.842+0000 7f03397ac640 1 -- 172.21.15.43:0/1127144065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c81a48b00 con 0x564c81a483b0 2024-04-21T08:48:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.842+0000 7f0339fad640 1 --2- 172.21.15.43:0/1127144065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c81a48d60 0x564c81a61d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.842+0000 7f03397ac640 1 --2- 172.21.15.43:0/1127144065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a483b0 0x564c81a53250 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f033000a680 tx=0x7f033000ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea9da5c87d8800e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.842+0000 7f031f7fe640 1 -- 172.21.15.43:0/1127144065 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330027040 con 0x564c81a483b0 2024-04-21T08:48:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.842+0000 7f031f7fe640 1 -- 172.21.15.43:0/1127144065 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f033000cca0 con 0x564c81a483b0 2024-04-21T08:48:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.842+0000 7f031f7fe640 1 -- 172.21.15.43:0/1127144065 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330004480 con 0x564c81a483b0 2024-04-21T08:48:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.842+0000 7f033ee02740 1 -- 172.21.15.43:0/1127144065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a483b0 msgr2=0x564c81a53250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.842+0000 7f033ee02740 1 --2- 172.21.15.43:0/1127144065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a483b0 0x564c81a53250 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f033000a680 tx=0x7f033000ae50 comp rx=0 tx=0).stop 2024-04-21T08:48:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.843+0000 7f033ee02740 1 -- 172.21.15.43:0/1127144065 shutdown_connections 2024-04-21T08:48:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.843+0000 7f033ee02740 1 --2- 172.21.15.43:0/1127144065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c81a51e60 0x564c81a54ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.843+0000 7f033ee02740 1 --2- 172.21.15.43:0/1127144065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c81a48d60 0x564c81a61d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.843+0000 7f033ee02740 1 --2- 172.21.15.43:0/1127144065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a483b0 0x564c81a53250 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.843+0000 7f033ee02740 1 -- 172.21.15.43:0/1127144065 >> 172.21.15.43:0/1127144065 conn(0x564c81a42ba0 msgr2=0x564c81a53e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.843+0000 7f033ee02740 1 -- 172.21.15.43:0/1127144065 shutdown_connections 2024-04-21T08:48:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.843+0000 7f033ee02740 1 -- 172.21.15.43:0/1127144065 wait complete. 2024-04-21T08:48:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.843+0000 7f033ee02740 1 -- start start 2024-04-21T08:48:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.843+0000 7f033ee02740 1 Processor -- start 2024-04-21T08:48:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.843+0000 7f033ee02740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a483b0 0x564c81a6d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.843+0000 7f033ee02740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c81a48d60 0x564c81a6d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.843+0000 7f033ee02740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c81a51e60 0x564c81a6ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.843+0000 7f033ee02740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f03341ae720 con 0x564c81a483b0 2024-04-21T08:48:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.843+0000 7f033ee02740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f03341ae050 con 0x564c81a51e60 2024-04-21T08:48:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.843+0000 7f033ee02740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f03341ab2b0 con 0x564c81a48d60 2024-04-21T08:48:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.843+0000 7f033ee02740 10 client.0 _ll_register_callbacks cb 0x7f0328004500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.844+0000 7f03397ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a483b0 0x564c81a6d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.844+0000 7f03397ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a483b0 0x564c81a6d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36574/0 (socket says 172.21.15.43:36574) 2024-04-21T08:48:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.844+0000 7f03397ac640 1 -- 172.21.15.43:0/1323962032 learned_addr learned my addr 172.21.15.43:0/1323962032 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.844+0000 7f0339fad640 1 --2- 172.21.15.43:0/1323962032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c81a48d60 0x564c81a6d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.844+0000 7f03397ac640 1 -- 172.21.15.43:0/1323962032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c81a48d60 msgr2=0x564c81a6d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.844+0000 7f03397ac640 1 --2- 172.21.15.43:0/1323962032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c81a48d60 0x564c81a6d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.844+0000 7f03397ac640 1 -- 172.21.15.43:0/1323962032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c81a51e60 msgr2=0x564c81a6ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.844+0000 7f03397ac640 1 --2- 172.21.15.43:0/1323962032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c81a51e60 0x564c81a6ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.844+0000 7f03397ac640 1 -- 172.21.15.43:0/1323962032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c81acb6b0 con 0x564c81a483b0 2024-04-21T08:48:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.844+0000 7f03397ac640 1 --2- 172.21.15.43:0/1323962032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a483b0 0x564c81a6d340 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f033000c370 tx=0x7f033000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a0ddd02804e84e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.844+0000 7f031d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.844+0000 7f031d7fa640 1 -- 172.21.15.43:0/1323962032 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033000ab80 con 0x564c81a483b0 2024-04-21T08:48:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.844+0000 7f031d7fa640 1 -- 172.21.15.43:0/1323962032 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f0330004020 con 0x564c81a483b0 2024-04-21T08:48:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.844+0000 7f033ee02740 1 -- 172.21.15.43:0/1323962032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f033000cca0 con 0x564c81a483b0 2024-04-21T08:48:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.844+0000 7f031d7fa640 1 -- 172.21.15.43:0/1323962032 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f033003b050 con 0x564c81a483b0 2024-04-21T08:48:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.844+0000 7f033ee02740 10 client.19950 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.844+0000 7f033ee02740 1 -- 172.21.15.43:0/1323962032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564c81acc900 con 0x564c81a483b0 2024-04-21T08:48:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.844+0000 7f031d7fa640 1 -- 172.21.15.43:0/1323962032 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f033000cca0 con 0x564c81a483b0 2024-04-21T08:48:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.845+0000 7f031d7fa640 1 -- 172.21.15.43:0/1323962032 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0330004820 con 0x564c81a483b0 2024-04-21T08:48:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.845+0000 7f033ee02740 10 client.19950 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.845+0000 7f033ee02740 10 client.19950 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:48:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.845+0000 7f033ee02740 20 client.19950 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.845+0000 7f033ee02740 1 --2- 172.21.15.43:0/1323962032 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x564c81accd00 0x564c81acf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.845+0000 7f033ee02740 4 client.19950 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.845+0000 7f033ee02740 1 -- 172.21.15.43:0/1323962032 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564c819bac10 con 0x564c81accd00 2024-04-21T08:48:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.845+0000 7f0339fad640 1 --2- 172.21.15.43:0/1323962032 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x564c81accd00 0x564c81acf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.845+0000 7f031d7fa640 10 client.19950 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:48:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.845+0000 7f0339fad640 1 --2- 172.21.15.43:0/1323962032 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x564c81accd00 0x564c81acf120 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=55e9551ebc02d6af server_cookie=ce366d7775c32f0e in_seq=0 out_seq=0 2024-04-21T08:48:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f031d7fa640 1 -- 172.21.15.43:0/1323962032 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564c819bac10 con 0x564c81accd00 2024-04-21T08:48:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f031d7fa640 10 client.19950 handle_command_reply: tid=0 2024-04-21T08:48:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f033ee02740 1 client.19950 shutdown 2024-04-21T08:48:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f033ee02740 1 -- 172.21.15.43:0/1323962032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a483b0 msgr2=0x564c81a6d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f033ee02740 1 --2- 172.21.15.43:0/1323962032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a483b0 0x564c81a6d340 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f033000c370 tx=0x7f033000c960 comp rx=0 tx=0).stop 2024-04-21T08:48:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f033ee02740 1 -- 172.21.15.43:0/1323962032 shutdown_connections 2024-04-21T08:48:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f033ee02740 1 --2- 172.21.15.43:0/1323962032 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x564c81accd00 0x564c81acf120 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f033ee02740 1 --2- 172.21.15.43:0/1323962032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c81a51e60 0x564c81a6ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f031d7fa640 0 client.19950 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:48:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f033ee02740 1 --2- 172.21.15.43:0/1323962032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564c81a48d60 0x564c81a6d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f033ee02740 1 --2- 172.21.15.43:0/1323962032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c81a483b0 0x564c81a6d340 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f033ee02740 1 -- 172.21.15.43:0/1323962032 >> 172.21.15.43:0/1323962032 conn(0x564c81a42ba0 msgr2=0x564c81a56c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f033ee02740 1 -- 172.21.15.43:0/1323962032 shutdown_connections 2024-04-21T08:48:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f033ee02740 1 -- 172.21.15.43:0/1323962032 wait complete. 2024-04-21T08:48:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f033ee02740 20 client.19950 trim_cache size 0 max 16384 2024-04-21T08:48:22.851 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:48:22.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f033c211640 1 -- 172.21.15.43:0/508268379 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f03200a3d20 msgr2=0x7f03200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f033c211640 1 --2- 172.21.15.43:0/508268379 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f03200a3d20 0x7f03200a61e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f032c0be830 tx=0x7f032c0c1040 comp rx=0 tx=0).stop 2024-04-21T08:48:22.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f033c211640 1 -- 172.21.15.43:0/508268379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f033419cf90 msgr2=0x7f033419d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f033c211640 1 --2- 172.21.15.43:0/508268379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f033419cf90 0x7f033419d440 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f032c005e30 tx=0x7f032c034760 comp rx=0 tx=0).stop 2024-04-21T08:48:22.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f033c211640 1 -- 172.21.15.43:0/508268379 shutdown_connections 2024-04-21T08:48:22.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f033c211640 1 --2- 172.21.15.43:0/508268379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f033419cf90 0x7f033419d440 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f033c211640 1 --2- 172.21.15.43:0/508268379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0334107270 0x7f0334198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f033c211640 1 --2- 172.21.15.43:0/508268379 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f03200a3d20 0x7f03200a61e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f033c211640 1 --2- 172.21.15.43:0/508268379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f033406a840 0x7f03341986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.846+0000 7f033c211640 1 -- 172.21.15.43:0/508268379 >> 172.21.15.43:0/508268379 conn(0x7f0334077eb0 msgr2=0x7f0334100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.847+0000 7f033c211640 1 -- 172.21.15.43:0/508268379 shutdown_connections 2024-04-21T08:48:22.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.847+0000 7f033c211640 1 -- 172.21.15.43:0/508268379 wait complete. 2024-04-21T08:48:22.856 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:48:22.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.961+0000 7f294dce0640 1 -- 172.21.15.43:0/1516910279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2948107260 msgr2=0x7f294810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.961+0000 7f294dce0640 1 --2- 172.21.15.43:0/1516910279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2948107260 0x7f294810ee30 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f2940009900 tx=0x7f294002bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.962+0000 7f294dce0640 1 -- 172.21.15.43:0/1516910279 shutdown_connections 2024-04-21T08:48:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.962+0000 7f294dce0640 1 --2- 172.21.15.43:0/1516910279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2948107260 0x7f294810ee30 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.962+0000 7f294dce0640 1 --2- 172.21.15.43:0/1516910279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2948102b40 0x7f2948106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.962+0000 7f294dce0640 1 --2- 172.21.15.43:0/1516910279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2948102190 0x7f2948102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.962+0000 7f294dce0640 1 -- 172.21.15.43:0/1516910279 >> 172.21.15.43:0/1516910279 conn(0x7f29480fd990 msgr2=0x7f29480ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.962+0000 7f294dce0640 1 -- 172.21.15.43:0/1516910279 shutdown_connections 2024-04-21T08:48:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.962+0000 7f294dce0640 1 -- 172.21.15.43:0/1516910279 wait complete. 2024-04-21T08:48:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.962+0000 7f294dce0640 1 Processor -- start 2024-04-21T08:48:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.962+0000 7f294dce0640 1 -- start start 2024-04-21T08:48:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.963+0000 7f294dce0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2948102190 0x7f2948198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.963+0000 7f294dce0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2948107260 0x7f2948198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.963+0000 7f294dce0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294819cfd0 0x7f294819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.963+0000 7f294dce0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f29481121d0 con 0x7f294819cfd0 2024-04-21T08:48:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.963+0000 7f294dce0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2948112050 con 0x7f2948107260 2024-04-21T08:48:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.963+0000 7f294dce0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2948112350 con 0x7f2948102190 2024-04-21T08:48:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.963+0000 7f2947fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294819cfd0 0x7f294819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.963+0000 7f2947fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294819cfd0 0x7f294819d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36602/0 (socket says 172.21.15.43:36602) 2024-04-21T08:48:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.963+0000 7f2947fff640 1 -- 172.21.15.43:0/1178323268 learned_addr learned my addr 172.21.15.43:0/1178323268 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.963+0000 7f29477fe640 1 --2- 172.21.15.43:0/1178323268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2948102190 0x7f2948198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.963+0000 7f2946ffd640 1 --2- 172.21.15.43:0/1178323268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2948107260 0x7f2948198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.963+0000 7f2947fff640 1 -- 172.21.15.43:0/1178323268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2948102190 msgr2=0x7f2948198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.963+0000 7f2947fff640 1 --2- 172.21.15.43:0/1178323268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2948102190 0x7f2948198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.963+0000 7f2947fff640 1 -- 172.21.15.43:0/1178323268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2948107260 msgr2=0x7f2948198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.963+0000 7f2947fff640 1 --2- 172.21.15.43:0/1178323268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2948107260 0x7f2948198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.963+0000 7f2947fff640 1 -- 172.21.15.43:0/1178323268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294819dac0 con 0x7f294819cfd0 2024-04-21T08:48:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.963+0000 7f2947fff640 1 --2- 172.21.15.43:0/1178323268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294819cfd0 0x7f294819d480 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f2940009230 tx=0x7f294002c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.963+0000 7f2944ff9640 1 -- 172.21.15.43:0/1178323268 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940043020 con 0x7f294819cfd0 2024-04-21T08:48:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.964+0000 7f294dce0640 1 -- 172.21.15.43:0/1178323268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294819dd50 con 0x7f294819cfd0 2024-04-21T08:48:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.964+0000 7f2944ff9640 1 -- 172.21.15.43:0/1178323268 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f294002caf0 con 0x7f294819cfd0 2024-04-21T08:48:22.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.964+0000 7f2944ff9640 1 -- 172.21.15.43:0/1178323268 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940003990 con 0x7f294819cfd0 2024-04-21T08:48:22.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.964+0000 7f294dce0640 1 -- 172.21.15.43:0/1178323268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29481a5570 con 0x7f294819cfd0 2024-04-21T08:48:22.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.966+0000 7f2944ff9640 1 -- 172.21.15.43:0/1178323268 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2940046020 con 0x7f294819cfd0 2024-04-21T08:48:22.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.966+0000 7f29509ad740 1 Processor -- start 2024-04-21T08:48:22.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.966+0000 7f29509ad740 1 -- start start 2024-04-21T08:48:22.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.967+0000 7f29509ad740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39beb44d0 0x55a39beb48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.967+0000 7f29509ad740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a39beb4e80 0x55a39bebd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.967+0000 7f29509ad740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a39bebda00 0x55a39bebfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.967+0000 7f29509ad740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a39be191f0 con 0x55a39beb44d0 2024-04-21T08:48:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.967+0000 7f29509ad740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a39bdb64b0 con 0x55a39beb4e80 2024-04-21T08:48:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.967+0000 7f29509ad740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a39be151b0 con 0x55a39bebda00 2024-04-21T08:48:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.967+0000 7f2944ff9640 1 --2- 172.21.15.43:0/1178323268 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f29340a3d20 0x7f29340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.967+0000 7f2946ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39beb44d0 0x55a39beb48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.967+0000 7f2946ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39beb44d0 0x55a39beb48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36608/0 (socket says 172.21.15.43:36608) 2024-04-21T08:48:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.967+0000 7f2946ffd640 1 -- 172.21.15.43:0/1285598680 learned_addr learned my addr 172.21.15.43:0/1285598680 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.967+0000 7f2947fff640 1 --2- 172.21.15.43:0/1178323268 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f29340a3d20 0x7f29340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.967+0000 7f29477fe640 1 --2- 172.21.15.43:0/1285598680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a39beb4e80 0x55a39bebd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.967+0000 7f2946ffd640 1 --2- 172.21.15.43:0/1285598680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a39bebda00 0x55a39bebfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.967+0000 7f2944ff9640 1 -- 172.21.15.43:0/1178323268 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f29400f2d00 con 0x7f294819cfd0 2024-04-21T08:48:22.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.967+0000 7f2946ffd640 1 -- 172.21.15.43:0/1285598680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a39bebda00 msgr2=0x55a39bebfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.967+0000 7f2946ffd640 1 --2- 172.21.15.43:0/1285598680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a39bebda00 0x55a39bebfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.967+0000 7f2946ffd640 1 -- 172.21.15.43:0/1285598680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a39beb4e80 msgr2=0x55a39bebd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.967+0000 7f2946ffd640 1 --2- 172.21.15.43:0/1285598680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a39beb4e80 0x55a39bebd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.967+0000 7f2946ffd640 1 -- 172.21.15.43:0/1285598680 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a39bda81b0 con 0x55a39beb44d0 2024-04-21T08:48:22.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.967+0000 7f2946ffd640 1 --2- 172.21.15.43:0/1285598680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39beb44d0 0x55a39beb48b0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f293000bac0 tx=0x7f2930004190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59af9ee9cb17103c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.967+0000 7f2947fff640 1 --2- 172.21.15.43:0/1178323268 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f29340a3d20 0x7f29340a61e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f29400be630 tx=0x7f29400e9040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.967+0000 7f293dffb640 1 -- 172.21.15.43:0/1285598680 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2930005020 con 0x55a39beb44d0 2024-04-21T08:48:22.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.968+0000 7f293dffb640 1 -- 172.21.15.43:0/1285598680 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2930005670 con 0x55a39beb44d0 2024-04-21T08:48:22.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.968+0000 7f293dffb640 1 -- 172.21.15.43:0/1285598680 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29300106c0 con 0x55a39beb44d0 2024-04-21T08:48:22.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.968+0000 7f29509ad740 1 -- 172.21.15.43:0/1285598680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39beb44d0 msgr2=0x55a39beb48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.968+0000 7f29509ad740 1 --2- 172.21.15.43:0/1285598680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39beb44d0 0x55a39beb48b0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f293000bac0 tx=0x7f2930004190 comp rx=0 tx=0).stop 2024-04-21T08:48:22.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.968+0000 7f29509ad740 1 -- 172.21.15.43:0/1285598680 shutdown_connections 2024-04-21T08:48:22.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.968+0000 7f29509ad740 1 --2- 172.21.15.43:0/1285598680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a39bebda00 0x55a39bebfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.968+0000 7f29509ad740 1 --2- 172.21.15.43:0/1285598680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a39beb4e80 0x55a39bebd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.968+0000 7f29509ad740 1 --2- 172.21.15.43:0/1285598680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39beb44d0 0x55a39beb48b0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.968+0000 7f29509ad740 1 -- 172.21.15.43:0/1285598680 >> 172.21.15.43:0/1285598680 conn(0x55a39beaeba0 msgr2=0x55a39beaef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.968+0000 7f29509ad740 1 -- 172.21.15.43:0/1285598680 shutdown_connections 2024-04-21T08:48:22.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.968+0000 7f29509ad740 1 -- 172.21.15.43:0/1285598680 wait complete. 2024-04-21T08:48:22.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.969+0000 7f29509ad740 1 -- start start 2024-04-21T08:48:22.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.969+0000 7f29509ad740 1 Processor -- start 2024-04-21T08:48:22.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.969+0000 7f29509ad740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a39beb44d0 0x55a39bebd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.969+0000 7f29509ad740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a39beb4e80 0x55a39beba650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.969+0000 7f29509ad740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39bebda00 0x55a39bebabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.969+0000 7f29509ad740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a39bec0910 con 0x55a39bebda00 2024-04-21T08:48:22.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.969+0000 7f29509ad740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a39bdd2500 con 0x55a39beb4e80 2024-04-21T08:48:22.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.969+0000 7f29509ad740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a39bec0a90 con 0x55a39beb44d0 2024-04-21T08:48:22.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.969+0000 7f29509ad740 10 client.0 _ll_register_callbacks cb 0x55a39bda0b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:22.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.969+0000 7f2946ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39bebda00 0x55a39bebabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.969+0000 7f29477fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a39beb4e80 0x55a39beba650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.969+0000 7f2946ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39bebda00 0x55a39bebabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36618/0 (socket says 172.21.15.43:36618) 2024-04-21T08:48:22.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.969+0000 7f29477fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a39beb4e80 0x55a39beba650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:41002/0 (socket says 172.21.15.43:41002) 2024-04-21T08:48:22.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.969+0000 7f2946ffd640 1 -- 172.21.15.43:0/1032573687 learned_addr learned my addr 172.21.15.43:0/1032573687 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.969+0000 7f2946ffd640 1 --2- 172.21.15.43:0/1032573687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a39beb44d0 0x55a39bebd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.969+0000 7f2946ffd640 1 -- 172.21.15.43:0/1032573687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a39beb44d0 msgr2=0x55a39bebd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.969+0000 7f2946ffd640 1 --2- 172.21.15.43:0/1032573687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a39beb44d0 0x55a39bebd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.969+0000 7f2946ffd640 1 -- 172.21.15.43:0/1032573687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a39beb4e80 msgr2=0x55a39beba650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.969+0000 7f2946ffd640 1 --2- 172.21.15.43:0/1032573687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a39beb4e80 0x55a39beba650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.970+0000 7f2946ffd640 1 -- 172.21.15.43:0/1032573687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a39bb2b3f0 con 0x55a39bebda00 2024-04-21T08:48:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.970+0000 7f2946ffd640 1 --2- 172.21.15.43:0/1032573687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39bebda00 0x55a39bebabb0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f2930005fd0 tx=0x7f293000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb458a50b80dc6c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.970+0000 7f2926ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.970+0000 7f2926ffd640 1 -- 172.21.15.43:0/1032573687 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2930005000 con 0x55a39bebda00 2024-04-21T08:48:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.970+0000 7f2926ffd640 1 -- 172.21.15.43:0/1032573687 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f29300051a0 con 0x55a39bebda00 2024-04-21T08:48:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.970+0000 7f29509ad740 1 -- 172.21.15.43:0/1032573687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2930005670 con 0x55a39bebda00 2024-04-21T08:48:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.970+0000 7f2926ffd640 1 -- 172.21.15.43:0/1032573687 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f293002f050 con 0x55a39bebda00 2024-04-21T08:48:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.970+0000 7f29509ad740 10 client.19962 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.970+0000 7f29509ad740 1 -- 172.21.15.43:0/1032573687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a39bbf57d0 con 0x55a39bebda00 2024-04-21T08:48:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.970+0000 7f2926ffd640 1 -- 172.21.15.43:0/1032573687 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2930005670 con 0x55a39bebda00 2024-04-21T08:48:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.970+0000 7f2926ffd640 1 -- 172.21.15.43:0/1032573687 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f293001f410 con 0x55a39bebda00 2024-04-21T08:48:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.970+0000 7f29509ad740 10 client.19962 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.970+0000 7f29509ad740 10 client.19962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.970+0000 7f29509ad740 20 client.19962 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.970+0000 7f29509ad740 1 --2- 172.21.15.43:0/1032573687 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a39becda90 0x55a39becfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.970+0000 7f29509ad740 4 client.19962 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.970+0000 7f29509ad740 1 -- 172.21.15.43:0/1032573687 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a39bdb8930 con 0x55a39becda90 2024-04-21T08:48:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.971+0000 7f29477fe640 1 --2- 172.21.15.43:0/1032573687 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a39becda90 0x55a39becfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.971+0000 7f2926ffd640 10 client.19962 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.971+0000 7f29477fe640 1 --2- 172.21.15.43:0/1032573687 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a39becda90 0x55a39becfe80 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4432906396b9999 server_cookie=86ff7af541c73fd6 in_seq=0 out_seq=0 2024-04-21T08:48:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.972+0000 7f2926ffd640 1 -- 172.21.15.43:0/1032573687 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a39bdb8930 con 0x55a39becda90 2024-04-21T08:48:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.972+0000 7f2926ffd640 10 client.19962 handle_command_reply: tid=0 2024-04-21T08:48:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.972+0000 7f29509ad740 1 client.19962 shutdown 2024-04-21T08:48:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.972+0000 7f29509ad740 1 -- 172.21.15.43:0/1032573687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39bebda00 msgr2=0x55a39bebabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.972+0000 7f29509ad740 1 --2- 172.21.15.43:0/1032573687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39bebda00 0x55a39bebabb0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f2930005fd0 tx=0x7f293000b5b0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.972+0000 7f29509ad740 1 -- 172.21.15.43:0/1032573687 shutdown_connections 2024-04-21T08:48:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.972+0000 7f29509ad740 1 --2- 172.21.15.43:0/1032573687 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a39becda90 0x55a39becfe80 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.972+0000 7f29509ad740 1 --2- 172.21.15.43:0/1032573687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39bebda00 0x55a39bebabb0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.972+0000 7f29509ad740 1 --2- 172.21.15.43:0/1032573687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a39beb4e80 0x55a39beba650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.972+0000 7f29509ad740 1 --2- 172.21.15.43:0/1032573687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a39beb44d0 0x55a39bebd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.972+0000 7f2926ffd640 0 client.19962 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:22.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.972+0000 7f29509ad740 1 -- 172.21.15.43:0/1032573687 >> 172.21.15.43:0/1032573687 conn(0x55a39beaeba0 msgr2=0x55a39beb08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.972+0000 7f29509ad740 1 -- 172.21.15.43:0/1032573687 shutdown_connections 2024-04-21T08:48:22.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.972+0000 7f29509ad740 1 -- 172.21.15.43:0/1032573687 wait complete. 2024-04-21T08:48:22.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.972+0000 7f29509ad740 20 client.19962 trim_cache size 0 max 16384 2024-04-21T08:48:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.986+0000 7f29509ad740 1 Processor -- start 2024-04-21T08:48:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.986+0000 7f29509ad740 1 -- start start 2024-04-21T08:48:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.986+0000 7f29509ad740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a39beb44d0 0x55a39bec9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.986+0000 7f29509ad740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a39beb4e80 0x55a39beb5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.986+0000 7f29509ad740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39bebda00 0x55a39bebcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.986+0000 7f29509ad740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f29481af470 con 0x55a39bebda00 2024-04-21T08:48:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.986+0000 7f29509ad740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f29481b0fb0 con 0x55a39beb4e80 2024-04-21T08:48:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.986+0000 7f29509ad740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29481afb40 con 0x55a39beb44d0 2024-04-21T08:48:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.986+0000 7f2946ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39bebda00 0x55a39bebcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.986+0000 7f2946ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39bebda00 0x55a39bebcd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36634/0 (socket says 172.21.15.43:36634) 2024-04-21T08:48:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.986+0000 7f2946ffd640 1 -- 172.21.15.43:0/3506581391 learned_addr learned my addr 172.21.15.43:0/3506581391 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.986+0000 7f29477fe640 1 --2- 172.21.15.43:0/3506581391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a39beb4e80 0x55a39beb5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.986+0000 7f2946ffd640 1 --2- 172.21.15.43:0/3506581391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a39beb44d0 0x55a39bec9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.986+0000 7f2946ffd640 1 -- 172.21.15.43:0/3506581391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a39beb44d0 msgr2=0x55a39bec9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.986+0000 7f2946ffd640 1 --2- 172.21.15.43:0/3506581391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a39beb44d0 0x55a39bec9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.986+0000 7f2946ffd640 1 -- 172.21.15.43:0/3506581391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a39beb4e80 msgr2=0x55a39beb5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.986+0000 7f2946ffd640 1 --2- 172.21.15.43:0/3506581391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a39beb4e80 0x55a39beb5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.986+0000 7f2946ffd640 1 -- 172.21.15.43:0/3506581391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a39bebaf20 con 0x55a39bebda00 2024-04-21T08:48:22.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.986+0000 7f29477fe640 1 --2- 172.21.15.43:0/3506581391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a39beb4e80 0x55a39beb5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:22.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.986+0000 7f2946ffd640 1 --2- 172.21.15.43:0/3506581391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39bebda00 0x55a39bebcd80 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f293001f980 tx=0x7f293000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a7f389322e3aca server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.987+0000 7f293d7fa640 1 -- 172.21.15.43:0/3506581391 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293000b910 con 0x55a39bebda00 2024-04-21T08:48:22.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.987+0000 7f293d7fa640 1 -- 172.21.15.43:0/3506581391 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f293001fb80 con 0x55a39bebda00 2024-04-21T08:48:22.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.987+0000 7f293d7fa640 1 -- 172.21.15.43:0/3506581391 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293001f3e0 con 0x55a39bebda00 2024-04-21T08:48:22.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.987+0000 7f29509ad740 1 -- 172.21.15.43:0/3506581391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39bebda00 msgr2=0x55a39bebcd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.987+0000 7f29509ad740 1 --2- 172.21.15.43:0/3506581391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39bebda00 0x55a39bebcd80 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f293001f980 tx=0x7f293000b390 comp rx=0 tx=0).stop 2024-04-21T08:48:22.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.987+0000 7f29509ad740 1 -- 172.21.15.43:0/3506581391 shutdown_connections 2024-04-21T08:48:22.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.987+0000 7f29509ad740 1 --2- 172.21.15.43:0/3506581391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39bebda00 0x55a39bebcd80 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.987+0000 7f29509ad740 1 --2- 172.21.15.43:0/3506581391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a39beb4e80 0x55a39beb5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.987+0000 7f29509ad740 1 --2- 172.21.15.43:0/3506581391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a39beb44d0 0x55a39bec9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.987+0000 7f29509ad740 1 -- 172.21.15.43:0/3506581391 >> 172.21.15.43:0/3506581391 conn(0x55a39beaeba0 msgr2=0x55a39beb53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.987+0000 7f29509ad740 1 -- 172.21.15.43:0/3506581391 shutdown_connections 2024-04-21T08:48:22.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.987+0000 7f29509ad740 1 -- 172.21.15.43:0/3506581391 wait complete. 2024-04-21T08:48:22.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.987+0000 7f29509ad740 1 -- start start 2024-04-21T08:48:22.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.988+0000 7f29509ad740 1 Processor -- start 2024-04-21T08:48:22.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.988+0000 7f29509ad740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a39beb44d0 0x55a39bec0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.988+0000 7f29509ad740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39beb4e80 0x55a39bec1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.988+0000 7f29509ad740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a39bebda00 0x55a39bf31e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.988+0000 7f29509ad740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f29481b08e0 con 0x55a39beb4e80 2024-04-21T08:48:22.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.988+0000 7f29509ad740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f29481b0210 con 0x55a39bebda00 2024-04-21T08:48:22.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.988+0000 7f29509ad740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29481ad470 con 0x55a39beb44d0 2024-04-21T08:48:22.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.988+0000 7f29509ad740 10 client.0 _ll_register_callbacks cb 0x7f2930003c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:22.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.988+0000 7f29477fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39beb4e80 0x55a39bec1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.988+0000 7f29477fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39beb4e80 0x55a39bec1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36638/0 (socket says 172.21.15.43:36638) 2024-04-21T08:48:22.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.988+0000 7f29477fe640 1 -- 172.21.15.43:0/1036905657 learned_addr learned my addr 172.21.15.43:0/1036905657 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:22.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.988+0000 7f2946ffd640 1 --2- 172.21.15.43:0/1036905657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a39bebda00 0x55a39bf31e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.988+0000 7f29477fe640 1 -- 172.21.15.43:0/1036905657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a39beb44d0 msgr2=0x55a39bec0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.988+0000 7f29477fe640 1 --2- 172.21.15.43:0/1036905657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a39beb44d0 0x55a39bec0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.988+0000 7f29477fe640 1 -- 172.21.15.43:0/1036905657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a39bebda00 msgr2=0x55a39bf31e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.988+0000 7f29477fe640 1 --2- 172.21.15.43:0/1036905657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a39bebda00 0x55a39bf31e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.988+0000 7f29477fe640 1 -- 172.21.15.43:0/1036905657 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a39bf32440 con 0x55a39beb4e80 2024-04-21T08:48:22.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.989+0000 7f29477fe640 1 --2- 172.21.15.43:0/1036905657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39beb4e80 0x55a39bec1400 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f2938000f80 tx=0x7f2938020630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2aeae3ba9e14a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.989+0000 7f2926ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:22.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.989+0000 7f2926ffd640 1 -- 172.21.15.43:0/1036905657 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938020940 con 0x55a39beb4e80 2024-04-21T08:48:22.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.989+0000 7f2926ffd640 1 -- 172.21.15.43:0/1036905657 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f2938011d80 con 0x55a39beb4e80 2024-04-21T08:48:22.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.989+0000 7f29509ad740 1 -- 172.21.15.43:0/1036905657 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f293001fb80 con 0x55a39beb4e80 2024-04-21T08:48:22.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.989+0000 7f2926ffd640 1 -- 172.21.15.43:0/1036905657 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2938032050 con 0x55a39beb4e80 2024-04-21T08:48:22.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.989+0000 7f29509ad740 10 client.19968 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:22.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.989+0000 7f29509ad740 1 -- 172.21.15.43:0/1036905657 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a39bf337b0 con 0x55a39beb4e80 2024-04-21T08:48:22.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.989+0000 7f2926ffd640 1 -- 172.21.15.43:0/1036905657 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f293001fb80 con 0x55a39beb4e80 2024-04-21T08:48:22.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.989+0000 7f2926ffd640 1 -- 172.21.15.43:0/1036905657 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f293801e5c0 con 0x55a39beb4e80 2024-04-21T08:48:22.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.989+0000 7f29509ad740 10 client.19968 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:22.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.989+0000 7f29509ad740 10 client.19968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:22.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.989+0000 7f29509ad740 20 client.19968 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:22.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.989+0000 7f29509ad740 1 --2- 172.21.15.43:0/1036905657 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a39bf33f10 0x55a39bf36330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.989+0000 7f29509ad740 4 client.19968 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:22.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.989+0000 7f29509ad740 1 -- 172.21.15.43:0/1036905657 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a39be26c10 con 0x55a39bf33f10 2024-04-21T08:48:22.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.990+0000 7f2946ffd640 1 --2- 172.21.15.43:0/1036905657 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a39bf33f10 0x55a39bf36330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.990+0000 7f2946ffd640 1 --2- 172.21.15.43:0/1036905657 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a39bf33f10 0x55a39bf36330 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=473b290b5d1bcdd7 server_cookie=92d30668ed49f780 in_seq=0 out_seq=0 2024-04-21T08:48:22.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.990+0000 7f2926ffd640 10 client.19968 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:22.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.990+0000 7f2926ffd640 1 -- 172.21.15.43:0/1036905657 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a39be26c10 con 0x55a39bf33f10 2024-04-21T08:48:22.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.990+0000 7f2926ffd640 10 client.19968 handle_command_reply: tid=0 2024-04-21T08:48:22.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.990+0000 7f29509ad740 1 client.19968 shutdown 2024-04-21T08:48:22.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.990+0000 7f29509ad740 1 -- 172.21.15.43:0/1036905657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39beb4e80 msgr2=0x55a39bec1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.990+0000 7f29509ad740 1 --2- 172.21.15.43:0/1036905657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39beb4e80 0x55a39bec1400 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f2938000f80 tx=0x7f2938020630 comp rx=0 tx=0).stop 2024-04-21T08:48:22.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.990+0000 7f29509ad740 1 -- 172.21.15.43:0/1036905657 shutdown_connections 2024-04-21T08:48:22.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.990+0000 7f29509ad740 1 --2- 172.21.15.43:0/1036905657 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a39bf33f10 0x55a39bf36330 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.990+0000 7f29509ad740 1 --2- 172.21.15.43:0/1036905657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a39bebda00 0x55a39bf31e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.990+0000 7f2926ffd640 0 client.19968 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:22.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.990+0000 7f29509ad740 1 --2- 172.21.15.43:0/1036905657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a39beb4e80 0x55a39bec1400 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.990+0000 7f29509ad740 1 --2- 172.21.15.43:0/1036905657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a39beb44d0 0x55a39bec0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.990+0000 7f29509ad740 1 -- 172.21.15.43:0/1036905657 >> 172.21.15.43:0/1036905657 conn(0x55a39beaeba0 msgr2=0x55a39beadc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.990+0000 7f29509ad740 1 -- 172.21.15.43:0/1036905657 shutdown_connections 2024-04-21T08:48:22.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.990+0000 7f29509ad740 1 -- 172.21.15.43:0/1036905657 wait complete. 2024-04-21T08:48:22.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.991+0000 7f29509ad740 20 client.19968 trim_cache size 0 max 16384 2024-04-21T08:48:22.997 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:48:22.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.991+0000 7f294dce0640 1 -- 172.21.15.43:0/1178323268 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f29340a3d20 msgr2=0x7f29340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.991+0000 7f294dce0640 1 --2- 172.21.15.43:0/1178323268 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f29340a3d20 0x7f29340a61e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f29400be630 tx=0x7f29400e9040 comp rx=0 tx=0).stop 2024-04-21T08:48:22.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.991+0000 7f294dce0640 1 -- 172.21.15.43:0/1178323268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294819cfd0 msgr2=0x7f294819d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.991+0000 7f294dce0640 1 --2- 172.21.15.43:0/1178323268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294819cfd0 0x7f294819d480 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f2940009230 tx=0x7f294002c710 comp rx=0 tx=0).stop 2024-04-21T08:48:22.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.991+0000 7f294dce0640 1 -- 172.21.15.43:0/1178323268 shutdown_connections 2024-04-21T08:48:22.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.991+0000 7f294dce0640 1 --2- 172.21.15.43:0/1178323268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294819cfd0 0x7f294819d480 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.991+0000 7f294dce0640 1 --2- 172.21.15.43:0/1178323268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2948107260 0x7f2948198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.991+0000 7f294dce0640 1 --2- 172.21.15.43:0/1178323268 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f29340a3d20 0x7f29340a61e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.991+0000 7f294dce0640 1 --2- 172.21.15.43:0/1178323268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2948102190 0x7f2948198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.991+0000 7f294dce0640 1 -- 172.21.15.43:0/1178323268 >> 172.21.15.43:0/1178323268 conn(0x7f29480fd990 msgr2=0x7f29480fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.991+0000 7f294dce0640 1 -- 172.21.15.43:0/1178323268 shutdown_connections 2024-04-21T08:48:22.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:22.991+0000 7f294dce0640 1 -- 172.21.15.43:0/1178323268 wait complete. 2024-04-21T08:48:23.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:23 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/200233865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:23.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:23 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/200233865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:23.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:23 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/200233865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:24.002 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:48:24.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:48:24.002 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:48:24.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.127+0000 7f0f42843640 1 -- 172.21.15.43:0/1298349095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3c102b40 msgr2=0x7f0f3c106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.127+0000 7f0f42843640 1 --2- 172.21.15.43:0/1298349095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3c102b40 0x7f0f3c106b80 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c009900 tx=0x7f0f2c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:24.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.127+0000 7f0f42843640 1 -- 172.21.15.43:0/1298349095 shutdown_connections 2024-04-21T08:48:24.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.127+0000 7f0f42843640 1 --2- 172.21.15.43:0/1298349095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f3c107260 0x7f0f3c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.127+0000 7f0f42843640 1 --2- 172.21.15.43:0/1298349095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3c102b40 0x7f0f3c106b80 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.127+0000 7f0f42843640 1 --2- 172.21.15.43:0/1298349095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f3c102190 0x7f0f3c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.127+0000 7f0f42843640 1 -- 172.21.15.43:0/1298349095 >> 172.21.15.43:0/1298349095 conn(0x7f0f3c0fd990 msgr2=0x7f0f3c0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.127+0000 7f0f42843640 1 -- 172.21.15.43:0/1298349095 shutdown_connections 2024-04-21T08:48:24.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.127+0000 7f0f42843640 1 -- 172.21.15.43:0/1298349095 wait complete. 2024-04-21T08:48:24.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.128+0000 7f0f42843640 1 Processor -- start 2024-04-21T08:48:24.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.128+0000 7f0f42843640 1 -- start start 2024-04-21T08:48:24.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.128+0000 7f0f42843640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f3c102190 0x7f0f3c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.128+0000 7f0f42843640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3c107260 0x7f0f3c198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.128+0000 7f0f42843640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f3c19d020 0x7f0f3c19d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.128+0000 7f0f42843640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c112180 con 0x7f0f3c107260 2024-04-21T08:48:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.128+0000 7f0f42843640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c112000 con 0x7f0f3c19d020 2024-04-21T08:48:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.128+0000 7f0f42843640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c112300 con 0x7f0f3c102190 2024-04-21T08:48:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.128+0000 7f0f3b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3c107260 0x7f0f3c198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.128+0000 7f0f3b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3c107260 0x7f0f3c198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36666/0 (socket says 172.21.15.43:36666) 2024-04-21T08:48:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.128+0000 7f0f3b7fe640 1 -- 172.21.15.43:0/963483366 learned_addr learned my addr 172.21.15.43:0/963483366 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.128+0000 7f0f40de0640 1 --2- 172.21.15.43:0/963483366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f3c19d020 0x7f0f3c19d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.128+0000 7f0f3bfff640 1 --2- 172.21.15.43:0/963483366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f3c102190 0x7f0f3c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.129+0000 7f0f3b7fe640 1 -- 172.21.15.43:0/963483366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f3c102190 msgr2=0x7f0f3c198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.129+0000 7f0f3b7fe640 1 --2- 172.21.15.43:0/963483366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f3c102190 0x7f0f3c198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.129+0000 7f0f3b7fe640 1 -- 172.21.15.43:0/963483366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f3c19d020 msgr2=0x7f0f3c19d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.129+0000 7f0f3b7fe640 1 --2- 172.21.15.43:0/963483366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f3c19d020 0x7f0f3c19d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.129+0000 7f0f3b7fe640 1 -- 172.21.15.43:0/963483366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3c19db10 con 0x7f0f3c107260 2024-04-21T08:48:24.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.129+0000 7f0f3b7fe640 1 --2- 172.21.15.43:0/963483366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3c107260 0x7f0f3c198ca0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c036040 tx=0x7f0f2c02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.129+0000 7f0f397fa640 1 -- 172.21.15.43:0/963483366 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c043020 con 0x7f0f3c107260 2024-04-21T08:48:24.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.129+0000 7f0f397fa640 1 -- 172.21.15.43:0/963483366 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f2c02cbe0 con 0x7f0f3c107260 2024-04-21T08:48:24.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.129+0000 7f0f42843640 1 -- 172.21.15.43:0/963483366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f3c19dda0 con 0x7f0f3c107260 2024-04-21T08:48:24.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.129+0000 7f0f397fa640 1 -- 172.21.15.43:0/963483366 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c003bf0 con 0x7f0f3c107260 2024-04-21T08:48:24.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.129+0000 7f0f42843640 1 -- 172.21.15.43:0/963483366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f3c1a55c0 con 0x7f0f3c107260 2024-04-21T08:48:24.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.131+0000 7f0f397fa640 1 -- 172.21.15.43:0/963483366 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0f2c046020 con 0x7f0f3c107260 2024-04-21T08:48:24.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.132+0000 7f0f4543b740 1 Processor -- start 2024-04-21T08:48:24.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.132+0000 7f0f4543b740 1 -- start start 2024-04-21T08:48:24.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.132+0000 7f0f4543b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55984085d8e0 0x55984085dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.132+0000 7f0f4543b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55984085e290 0x559840866840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.132+0000 7f0f4543b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559840866e10 0x559840869260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.132+0000 7f0f397fa640 1 --2- 172.21.15.43:0/963483366 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0f280a3d20 0x7f0f280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.132+0000 7f0f4543b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5598407c1f00 con 0x559840866e10 2024-04-21T08:48:24.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.132+0000 7f0f4543b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55984075f590 con 0x55984085d8e0 2024-04-21T08:48:24.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.132+0000 7f0f4543b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5598407bdec0 con 0x55984085e290 2024-04-21T08:48:24.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.133+0000 7f0f40de0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55984085d8e0 0x55984085dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.133+0000 7f0f40de0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55984085d8e0 0x55984085dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:41048/0 (socket says 172.21.15.43:41048) 2024-04-21T08:48:24.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.133+0000 7f0f3b7fe640 1 --2- 172.21.15.43:0/963483366 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0f280a3d20 0x7f0f280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.133+0000 7f0f40de0640 1 -- 172.21.15.43:0/894393151 learned_addr learned my addr 172.21.15.43:0/894393151 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:24.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.133+0000 7f0f3bfff640 1 --2- 172.21.15.43:0/894393151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55984085e290 0x559840866840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.133+0000 7f0f40de0640 1 --2- 172.21.15.43:0/894393151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559840866e10 0x559840869260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.133+0000 7f0f397fa640 1 -- 172.21.15.43:0/963483366 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f0f2c0f2d60 con 0x7f0f3c107260 2024-04-21T08:48:24.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.133+0000 7f0f40de0640 1 -- 172.21.15.43:0/894393151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55984085e290 msgr2=0x559840866840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.133+0000 7f0f40de0640 1 --2- 172.21.15.43:0/894393151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55984085e290 0x559840866840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.133+0000 7f0f40de0640 1 -- 172.21.15.43:0/894393151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559840866e10 msgr2=0x559840869260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.133+0000 7f0f40de0640 1 --2- 172.21.15.43:0/894393151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559840866e10 0x559840869260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.133+0000 7f0f40de0640 1 -- 172.21.15.43:0/894393151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55984059c7e0 con 0x55984085d8e0 2024-04-21T08:48:24.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.133+0000 7f0f3bfff640 1 --2- 172.21.15.43:0/894393151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55984085e290 0x559840866840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:24.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.133+0000 7f0f3b7fe640 1 --2- 172.21.15.43:0/963483366 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0f280a3d20 0x7f0f280a61e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0be690 tx=0x7f0f2c0a9040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.133+0000 7f0f40de0640 1 --2- 172.21.15.43:0/894393151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55984085d8e0 0x55984085dcc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0f34012a20 tx=0x7f0f340099f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69c487b1a440c28b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.133+0000 7f0f327fc640 1 -- 172.21.15.43:0/894393151 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f34003970 con 0x55984085d8e0 2024-04-21T08:48:24.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.133+0000 7f0f327fc640 1 -- 172.21.15.43:0/894393151 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f34010770 con 0x55984085d8e0 2024-04-21T08:48:24.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.133+0000 7f0f327fc640 1 -- 172.21.15.43:0/894393151 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f34004350 con 0x55984085d8e0 2024-04-21T08:48:24.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.134+0000 7f0f4543b740 1 -- 172.21.15.43:0/894393151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55984085d8e0 msgr2=0x55984085dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.134+0000 7f0f4543b740 1 --2- 172.21.15.43:0/894393151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55984085d8e0 0x55984085dcc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0f34012a20 tx=0x7f0f340099f0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.134+0000 7f0f4543b740 1 -- 172.21.15.43:0/894393151 shutdown_connections 2024-04-21T08:48:24.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.134+0000 7f0f4543b740 1 --2- 172.21.15.43:0/894393151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559840866e10 0x559840869260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.134+0000 7f0f4543b740 1 --2- 172.21.15.43:0/894393151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55984085e290 0x559840866840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.134+0000 7f0f4543b740 1 --2- 172.21.15.43:0/894393151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55984085d8e0 0x55984085dcc0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.134+0000 7f0f4543b740 1 -- 172.21.15.43:0/894393151 >> 172.21.15.43:0/894393151 conn(0x559840857fb0 msgr2=0x559840858390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.134+0000 7f0f4543b740 1 -- 172.21.15.43:0/894393151 shutdown_connections 2024-04-21T08:48:24.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.134+0000 7f0f4543b740 1 -- 172.21.15.43:0/894393151 wait complete. 2024-04-21T08:48:24.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.134+0000 7f0f4543b740 1 -- start start 2024-04-21T08:48:24.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.135+0000 7f0f4543b740 1 Processor -- start 2024-04-21T08:48:24.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.135+0000 7f0f4543b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55984085d8e0 0x559840863e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.135+0000 7f0f4543b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55984085e290 0x5598408643a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.135+0000 7f0f4543b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559840866e10 0x5598408648e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.135+0000 7f0f4543b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559840869d80 con 0x559840866e10 2024-04-21T08:48:24.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.135+0000 7f0f4543b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55984077b580 con 0x55984085e290 2024-04-21T08:48:24.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.135+0000 7f0f4543b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559840869f00 con 0x55984085d8e0 2024-04-21T08:48:24.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.135+0000 7f0f4543b740 10 client.0 _ll_register_callbacks cb 0x559840781650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:24.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.135+0000 7f0f40de0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559840866e10 0x5598408648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.135+0000 7f0f40de0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559840866e10 0x5598408648e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36686/0 (socket says 172.21.15.43:36686) 2024-04-21T08:48:24.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.135+0000 7f0f40de0640 1 -- 172.21.15.43:0/970472549 learned_addr learned my addr 172.21.15.43:0/970472549 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:24.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.135+0000 7f0f40de0640 1 --2- 172.21.15.43:0/970472549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55984085d8e0 0x559840863e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.135+0000 7f0f3bfff640 1 --2- 172.21.15.43:0/970472549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55984085e290 0x5598408643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.136+0000 7f0f40de0640 1 -- 172.21.15.43:0/970472549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55984085d8e0 msgr2=0x559840863e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.136+0000 7f0f40de0640 1 --2- 172.21.15.43:0/970472549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55984085d8e0 0x559840863e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.136+0000 7f0f40de0640 1 -- 172.21.15.43:0/970472549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55984085e290 msgr2=0x5598408643a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.136+0000 7f0f40de0640 1 --2- 172.21.15.43:0/970472549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55984085e290 0x5598408643a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.136+0000 7f0f40de0640 1 -- 172.21.15.43:0/970472549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598405fcf70 con 0x559840866e10 2024-04-21T08:48:24.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.136+0000 7f0f40de0640 1 --2- 172.21.15.43:0/970472549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559840866e10 0x5598408648e0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f0f34012470 tx=0x7f0f3401b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834b5aad03310f3a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.136+0000 7f0f1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:24.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.136+0000 7f0f1b7fe640 1 -- 172.21.15.43:0/970472549 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f34010920 con 0x559840866e10 2024-04-21T08:48:24.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.136+0000 7f0f1b7fe640 1 -- 172.21.15.43:0/970472549 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f0f34010ac0 con 0x559840866e10 2024-04-21T08:48:24.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.136+0000 7f0f4543b740 1 -- 172.21.15.43:0/970472549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f34010770 con 0x559840866e10 2024-04-21T08:48:24.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.136+0000 7f0f1b7fe640 1 -- 172.21.15.43:0/970472549 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0f34033050 con 0x559840866e10 2024-04-21T08:48:24.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.136+0000 7f0f4543b740 10 client.19980 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:24.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.136+0000 7f0f4543b740 1 -- 172.21.15.43:0/970472549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559840494170 con 0x559840866e10 2024-04-21T08:48:24.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.136+0000 7f0f1b7fe640 1 -- 172.21.15.43:0/970472549 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f34010770 con 0x559840866e10 2024-04-21T08:48:24.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.137+0000 7f0f1b7fe640 1 -- 172.21.15.43:0/970472549 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0f3400f040 con 0x559840866e10 2024-04-21T08:48:24.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.137+0000 7f0f4543b740 10 client.19980 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:24.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.137+0000 7f0f4543b740 10 client.19980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:24.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.137+0000 7f0f4543b740 20 client.19980 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:24.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.137+0000 7f0f4543b740 1 --2- 172.21.15.43:0/970472549 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559840877290 0x559840879680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.137+0000 7f0f4543b740 4 client.19980 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:24.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.137+0000 7f0f4543b740 1 -- 172.21.15.43:0/970472549 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598407cf920 con 0x559840877290 2024-04-21T08:48:24.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.137+0000 7f0f3bfff640 1 --2- 172.21.15.43:0/970472549 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559840877290 0x559840879680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.137+0000 7f0f1b7fe640 10 client.19980 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:24.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.137+0000 7f0f3bfff640 1 --2- 172.21.15.43:0/970472549 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559840877290 0x559840879680 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ceab1c254d045d0 server_cookie=33b1ccc0ed2cab1d in_seq=0 out_seq=0 2024-04-21T08:48:24.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.138+0000 7f0f1b7fe640 1 -- 172.21.15.43:0/970472549 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5598407cf920 con 0x559840877290 2024-04-21T08:48:24.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.138+0000 7f0f1b7fe640 10 client.19980 handle_command_reply: tid=0 2024-04-21T08:48:24.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.138+0000 7f0f4543b740 1 client.19980 shutdown 2024-04-21T08:48:24.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.138+0000 7f0f4543b740 1 -- 172.21.15.43:0/970472549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559840866e10 msgr2=0x5598408648e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.138+0000 7f0f4543b740 1 --2- 172.21.15.43:0/970472549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559840866e10 0x5598408648e0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f0f34012470 tx=0x7f0f3401b040 comp rx=0 tx=0).stop 2024-04-21T08:48:24.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.138+0000 7f0f4543b740 1 -- 172.21.15.43:0/970472549 shutdown_connections 2024-04-21T08:48:24.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.138+0000 7f0f4543b740 1 --2- 172.21.15.43:0/970472549 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x559840877290 0x559840879680 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.138+0000 7f0f4543b740 1 --2- 172.21.15.43:0/970472549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559840866e10 0x5598408648e0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.138+0000 7f0f4543b740 1 --2- 172.21.15.43:0/970472549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55984085e290 0x5598408643a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.138+0000 7f0f1b7fe640 0 client.19980 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:24.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.138+0000 7f0f4543b740 1 --2- 172.21.15.43:0/970472549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55984085d8e0 0x559840863e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.138+0000 7f0f4543b740 1 -- 172.21.15.43:0/970472549 >> 172.21.15.43:0/970472549 conn(0x559840857fb0 msgr2=0x559840859d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.138+0000 7f0f4543b740 1 -- 172.21.15.43:0/970472549 shutdown_connections 2024-04-21T08:48:24.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.139+0000 7f0f4543b740 1 -- 172.21.15.43:0/970472549 wait complete. 2024-04-21T08:48:24.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.139+0000 7f0f4543b740 20 client.19980 trim_cache size 0 max 16384 2024-04-21T08:48:24.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.156+0000 7f0f4543b740 1 Processor -- start 2024-04-21T08:48:24.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.156+0000 7f0f4543b740 1 -- start start 2024-04-21T08:48:24.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.157+0000 7f0f4543b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55984085d8e0 0x559840866590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.157+0000 7f0f4543b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55984085e290 0x559840872c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.157+0000 7f0f4543b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559840866e10 0x55984086f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.157+0000 7f0f4543b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c1af510 con 0x55984085e290 2024-04-21T08:48:24.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.157+0000 7f0f4543b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c1b1050 con 0x559840866e10 2024-04-21T08:48:24.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.157+0000 7f0f4543b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c1afbe0 con 0x55984085d8e0 2024-04-21T08:48:24.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.157+0000 7f0f3bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55984085e290 0x559840872c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.157+0000 7f0f3bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55984085e290 0x559840872c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36690/0 (socket says 172.21.15.43:36690) 2024-04-21T08:48:24.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.157+0000 7f0f3bfff640 1 -- 172.21.15.43:0/3296276244 learned_addr learned my addr 172.21.15.43:0/3296276244 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:24.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.157+0000 7f0f40de0640 1 --2- 172.21.15.43:0/3296276244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55984085d8e0 0x559840866590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.157+0000 7f0f3bfff640 1 -- 172.21.15.43:0/3296276244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55984085d8e0 msgr2=0x559840866590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.157+0000 7f0f3bfff640 1 --2- 172.21.15.43:0/3296276244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55984085d8e0 0x559840866590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.157+0000 7f0f3bfff640 1 -- 172.21.15.43:0/3296276244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559840866e10 msgr2=0x55984086f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.157+0000 7f0f3bfff640 1 --2- 172.21.15.43:0/3296276244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559840866e10 0x55984086f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.157+0000 7f0f3bfff640 1 -- 172.21.15.43:0/3296276244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55984085e030 con 0x55984085e290 2024-04-21T08:48:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.157+0000 7f0f3bfff640 1 --2- 172.21.15.43:0/3296276244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55984085e290 0x559840872c70 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400b890 tx=0x7f0f2400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bded533f58365edd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.157+0000 7f0f31ffb640 1 -- 172.21.15.43:0/3296276244 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f24003ca0 con 0x55984085e290 2024-04-21T08:48:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.157+0000 7f0f31ffb640 1 -- 172.21.15.43:0/3296276244 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f24024020 con 0x55984085e290 2024-04-21T08:48:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.157+0000 7f0f31ffb640 1 -- 172.21.15.43:0/3296276244 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f24011880 con 0x55984085e290 2024-04-21T08:48:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.158+0000 7f0f4543b740 1 -- 172.21.15.43:0/3296276244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55984085e290 msgr2=0x559840872c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.158+0000 7f0f4543b740 1 --2- 172.21.15.43:0/3296276244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55984085e290 0x559840872c70 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400b890 tx=0x7f0f2400b390 comp rx=0 tx=0).stop 2024-04-21T08:48:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.158+0000 7f0f4543b740 1 -- 172.21.15.43:0/3296276244 shutdown_connections 2024-04-21T08:48:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.158+0000 7f0f4543b740 1 --2- 172.21.15.43:0/3296276244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559840866e10 0x55984086f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.158+0000 7f0f4543b740 1 --2- 172.21.15.43:0/3296276244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55984085e290 0x559840872c70 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.158+0000 7f0f4543b740 1 --2- 172.21.15.43:0/3296276244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55984085d8e0 0x559840866590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.158+0000 7f0f4543b740 1 -- 172.21.15.43:0/3296276244 >> 172.21.15.43:0/3296276244 conn(0x559840857fb0 msgr2=0x559840863e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.158+0000 7f0f4543b740 1 -- 172.21.15.43:0/3296276244 shutdown_connections 2024-04-21T08:48:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.158+0000 7f0f4543b740 1 -- 172.21.15.43:0/3296276244 wait complete. 2024-04-21T08:48:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.158+0000 7f0f4543b740 1 -- start start 2024-04-21T08:48:24.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.158+0000 7f0f4543b740 1 Processor -- start 2024-04-21T08:48:24.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.159+0000 7f0f4543b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55984085d8e0 0x5598408d5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.159+0000 7f0f4543b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55984085e290 0x5598408d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.159+0000 7f0f4543b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559840866e10 0x5598408d5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.159+0000 7f0f4543b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c1b0980 con 0x559840866e10 2024-04-21T08:48:24.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.159+0000 7f0f4543b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c1b02b0 con 0x55984085d8e0 2024-04-21T08:48:24.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.159+0000 7f0f4543b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c1ad510 con 0x55984085e290 2024-04-21T08:48:24.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.159+0000 7f0f4543b740 10 client.0 _ll_register_callbacks cb 0x7f0f34012510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:24.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.159+0000 7f0f40de0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559840866e10 0x5598408d5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.159+0000 7f0f3bfff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55984085e290 0x5598408d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.159+0000 7f0f40de0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559840866e10 0x5598408d5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36694/0 (socket says 172.21.15.43:36694) 2024-04-21T08:48:24.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.159+0000 7f0f40de0640 1 -- 172.21.15.43:0/2939975280 learned_addr learned my addr 172.21.15.43:0/2939975280 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:24.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.159+0000 7f0f40de0640 1 --2- 172.21.15.43:0/2939975280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55984085d8e0 0x5598408d5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.159+0000 7f0f40de0640 1 -- 172.21.15.43:0/2939975280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55984085e290 msgr2=0x5598408d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.159+0000 7f0f40de0640 1 --2- 172.21.15.43:0/2939975280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55984085e290 0x5598408d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.159+0000 7f0f40de0640 1 -- 172.21.15.43:0/2939975280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55984085d8e0 msgr2=0x5598408d5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.159+0000 7f0f40de0640 1 --2- 172.21.15.43:0/2939975280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55984085d8e0 0x5598408d5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.159+0000 7f0f40de0640 1 -- 172.21.15.43:0/2939975280 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598408e0810 con 0x559840866e10 2024-04-21T08:48:24.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.159+0000 7f0f3bfff640 1 --2- 172.21.15.43:0/2939975280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55984085e290 0x5598408d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:24.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.159+0000 7f0f40de0640 1 --2- 172.21.15.43:0/2939975280 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559840866e10 0x5598408d5fc0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f0f34009fd0 tx=0x7f0f34004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec18b621e45bd024 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.159+0000 7f0f1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:24.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.160+0000 7f0f1b7fe640 1 -- 172.21.15.43:0/2939975280 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f34003970 con 0x559840866e10 2024-04-21T08:48:24.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.160+0000 7f0f1b7fe640 1 -- 172.21.15.43:0/2939975280 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f0f34029070 con 0x559840866e10 2024-04-21T08:48:24.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.160+0000 7f0f4543b740 1 -- 172.21.15.43:0/2939975280 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f24024020 con 0x559840866e10 2024-04-21T08:48:24.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.160+0000 7f0f1b7fe640 1 -- 172.21.15.43:0/2939975280 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0f34003b10 con 0x559840866e10 2024-04-21T08:48:24.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.160+0000 7f0f4543b740 10 client.19986 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:24.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.160+0000 7f0f4543b740 1 -- 172.21.15.43:0/2939975280 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5598408e1b50 con 0x559840866e10 2024-04-21T08:48:24.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.160+0000 7f0f1b7fe640 1 -- 172.21.15.43:0/2939975280 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f24024020 con 0x559840866e10 2024-04-21T08:48:24.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.160+0000 7f0f1b7fe640 1 -- 172.21.15.43:0/2939975280 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0f34012070 con 0x559840866e10 2024-04-21T08:48:24.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.160+0000 7f0f4543b740 10 client.19986 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:24.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.160+0000 7f0f4543b740 10 client.19986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:24.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.160+0000 7f0f4543b740 20 client.19986 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:24.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.160+0000 7f0f4543b740 1 --2- 172.21.15.43:0/2939975280 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5598408e21e0 0x5598408e4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.160+0000 7f0f4543b740 4 client.19986 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:48:24.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.160+0000 7f0f4543b740 1 -- 172.21.15.43:0/2939975280 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5598407c9790 con 0x5598408e21e0 2024-04-21T08:48:24.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.160+0000 7f0f3bfff640 1 --2- 172.21.15.43:0/2939975280 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5598408e21e0 0x5598408e4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.161+0000 7f0f3bfff640 1 --2- 172.21.15.43:0/2939975280 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5598408e21e0 0x5598408e4600 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2b6c9d0b21ecc46 server_cookie=d0670048f2efdd0 in_seq=0 out_seq=0 2024-04-21T08:48:24.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.161+0000 7f0f1b7fe640 10 client.19986 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:24.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.162+0000 7f0f1b7fe640 1 -- 172.21.15.43:0/2939975280 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5598407c9790 con 0x5598408e21e0 2024-04-21T08:48:24.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.162+0000 7f0f1b7fe640 10 client.19986 handle_command_reply: tid=0 2024-04-21T08:48:24.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.162+0000 7f0f4543b740 1 client.19986 shutdown 2024-04-21T08:48:24.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.162+0000 7f0f4543b740 1 -- 172.21.15.43:0/2939975280 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559840866e10 msgr2=0x5598408d5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.162+0000 7f0f4543b740 1 --2- 172.21.15.43:0/2939975280 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559840866e10 0x5598408d5fc0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f0f34009fd0 tx=0x7f0f34004460 comp rx=0 tx=0).stop 2024-04-21T08:48:24.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.162+0000 7f0f4543b740 1 -- 172.21.15.43:0/2939975280 shutdown_connections 2024-04-21T08:48:24.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.162+0000 7f0f4543b740 1 --2- 172.21.15.43:0/2939975280 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5598408e21e0 0x5598408e4600 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.162+0000 7f0f1b7fe640 0 client.19986 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:24.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.162+0000 7f0f4543b740 1 --2- 172.21.15.43:0/2939975280 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559840866e10 0x5598408d5fc0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.162+0000 7f0f4543b740 1 --2- 172.21.15.43:0/2939975280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55984085e290 0x5598408d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.162+0000 7f0f4543b740 1 --2- 172.21.15.43:0/2939975280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55984085d8e0 0x5598408d5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.162+0000 7f0f4543b740 1 -- 172.21.15.43:0/2939975280 >> 172.21.15.43:0/2939975280 conn(0x559840857fb0 msgr2=0x559840857010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.162+0000 7f0f4543b740 1 -- 172.21.15.43:0/2939975280 shutdown_connections 2024-04-21T08:48:24.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.162+0000 7f0f4543b740 1 -- 172.21.15.43:0/2939975280 wait complete. 2024-04-21T08:48:24.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.162+0000 7f0f4543b740 20 client.19986 trim_cache size 0 max 16384 2024-04-21T08:48:24.167 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:48:24.167 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:48:24.167 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "ca6fe7c4-c598-4807-a7cf-4364c29682ed", 2024-04-21T08:48:24.167 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:48:24.167 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:48:24.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.163+0000 7f0f42843640 1 -- 172.21.15.43:0/963483366 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0f280a3d20 msgr2=0x7f0f280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.163+0000 7f0f42843640 1 --2- 172.21.15.43:0/963483366 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0f280a3d20 0x7f0f280a61e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0be690 tx=0x7f0f2c0a9040 comp rx=0 tx=0).stop 2024-04-21T08:48:24.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.163+0000 7f0f42843640 1 -- 172.21.15.43:0/963483366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3c107260 msgr2=0x7f0f3c198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.163+0000 7f0f42843640 1 --2- 172.21.15.43:0/963483366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3c107260 0x7f0f3c198ca0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c036040 tx=0x7f0f2c02c7e0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.163+0000 7f0f42843640 1 -- 172.21.15.43:0/963483366 shutdown_connections 2024-04-21T08:48:24.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.163+0000 7f0f42843640 1 --2- 172.21.15.43:0/963483366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f3c19d020 0x7f0f3c19d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.163+0000 7f0f42843640 1 --2- 172.21.15.43:0/963483366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3c107260 0x7f0f3c198ca0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.163+0000 7f0f42843640 1 --2- 172.21.15.43:0/963483366 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0f280a3d20 0x7f0f280a61e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.163+0000 7f0f42843640 1 --2- 172.21.15.43:0/963483366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f3c102190 0x7f0f3c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.163+0000 7f0f42843640 1 -- 172.21.15.43:0/963483366 >> 172.21.15.43:0/963483366 conn(0x7f0f3c0fd990 msgr2=0x7f0f3c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.163+0000 7f0f42843640 1 -- 172.21.15.43:0/963483366 shutdown_connections 2024-04-21T08:48:24.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.163+0000 7f0f42843640 1 -- 172.21.15.43:0/963483366 wait complete. 2024-04-21T08:48:24.174 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:48:24.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:48:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.279+0000 7f6cbf5b2640 1 -- 172.21.15.43:0/3384619764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb8106150 msgr2=0x7f6cb810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.279+0000 7f6cbf5b2640 1 --2- 172.21.15.43:0/3384619764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb8106150 0x7f6cb810dd20 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0009ca0 tx=0x7f6cb002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.279+0000 7f6cbf5b2640 1 -- 172.21.15.43:0/3384619764 shutdown_connections 2024-04-21T08:48:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.279+0000 7f6cbf5b2640 1 --2- 172.21.15.43:0/3384619764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb8106150 0x7f6cb810dd20 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.279+0000 7f6cbf5b2640 1 --2- 172.21.15.43:0/3384619764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb8101a30 0x7f6cb8105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.279+0000 7f6cbf5b2640 1 --2- 172.21.15.43:0/3384619764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cb8101080 0x7f6cb8101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.279+0000 7f6cbf5b2640 1 -- 172.21.15.43:0/3384619764 >> 172.21.15.43:0/3384619764 conn(0x7f6cb8077fd0 msgr2=0x7f6cb80783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.279+0000 7f6cbf5b2640 1 -- 172.21.15.43:0/3384619764 shutdown_connections 2024-04-21T08:48:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.279+0000 7f6cbf5b2640 1 -- 172.21.15.43:0/3384619764 wait complete. 2024-04-21T08:48:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.279+0000 7f6cbf5b2640 1 Processor -- start 2024-04-21T08:48:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.279+0000 7f6cbf5b2640 1 -- start start 2024-04-21T08:48:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.280+0000 7f6cbf5b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb8101080 0x7f6cb8198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.280+0000 7f6cbf5b2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cb8106150 0x7f6cb8198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.280+0000 7f6cbf5b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb819d000 0x7f6cb819d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.280+0000 7f6cbf5b2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb81121d0 con 0x7f6cb8101080 2024-04-21T08:48:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.280+0000 7f6cbf5b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb8112050 con 0x7f6cb819d000 2024-04-21T08:48:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.280+0000 7f6cbf5b2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb8112350 con 0x7f6cb8106150 2024-04-21T08:48:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.280+0000 7f6cbd34e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb8101080 0x7f6cb8198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.280+0000 7f6cbd34e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb8101080 0x7f6cb8198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36708/0 (socket says 172.21.15.43:36708) 2024-04-21T08:48:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.280+0000 7f6cbd34e640 1 -- 172.21.15.43:0/3808963141 learned_addr learned my addr 172.21.15.43:0/3808963141 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.280+0000 7f6cbcb4d640 1 --2- 172.21.15.43:0/3808963141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cb8106150 0x7f6cb8198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.280+0000 7f6cbdb4f640 1 --2- 172.21.15.43:0/3808963141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb819d000 0x7f6cb819d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.280+0000 7f6cbd34e640 1 -- 172.21.15.43:0/3808963141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cb8106150 msgr2=0x7f6cb8198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.280+0000 7f6cbd34e640 1 --2- 172.21.15.43:0/3808963141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cb8106150 0x7f6cb8198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.280+0000 7f6cbd34e640 1 -- 172.21.15.43:0/3808963141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb819d000 msgr2=0x7f6cb819d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.280+0000 7f6cbd34e640 1 --2- 172.21.15.43:0/3808963141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb819d000 0x7f6cb819d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.280+0000 7f6cbd34e640 1 -- 172.21.15.43:0/3808963141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb819daf0 con 0x7f6cb8101080 2024-04-21T08:48:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.280+0000 7f6cbcb4d640 1 --2- 172.21.15.43:0/3808963141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cb8106150 0x7f6cb8198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.280+0000 7f6cbd34e640 1 --2- 172.21.15.43:0/3808963141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb8101080 0x7f6cb8198790 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400e9f0 tx=0x7f6cb400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.281+0000 7f6cae7fc640 1 -- 172.21.15.43:0/3808963141 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb400cd80 con 0x7f6cb8101080 2024-04-21T08:48:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.281+0000 7f6cbf5b2640 1 -- 172.21.15.43:0/3808963141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb819dde0 con 0x7f6cb8101080 2024-04-21T08:48:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.281+0000 7f6cae7fc640 1 -- 172.21.15.43:0/3808963141 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cb40054d0 con 0x7f6cb8101080 2024-04-21T08:48:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.281+0000 7f6cae7fc640 1 -- 172.21.15.43:0/3808963141 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4010690 con 0x7f6cb8101080 2024-04-21T08:48:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.281+0000 7f6cbf5b2640 1 -- 172.21.15.43:0/3808963141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb81a5580 con 0x7f6cb8101080 2024-04-21T08:48:24.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.283+0000 7f6cae7fc640 1 -- 172.21.15.43:0/3808963141 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6cb4004e90 con 0x7f6cb8101080 2024-04-21T08:48:24.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.283+0000 7f6cc21a5740 1 Processor -- start 2024-04-21T08:48:24.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.283+0000 7f6cc21a5740 1 -- start start 2024-04-21T08:48:24.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.283+0000 7f6cc21a5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cd230 0x56375f9cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.283+0000 7f6cc21a5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56375f9cdb50 0x56375f9da950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.283+0000 7f6cc21a5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56375f9dae90 0x56375f9dd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.283+0000 7f6cc21a5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56375f9321e0 con 0x56375f9cd230 2024-04-21T08:48:24.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.283+0000 7f6cc21a5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56375f8cf390 con 0x56375f9dae90 2024-04-21T08:48:24.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.283+0000 7f6cc21a5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56375f92e1a0 con 0x56375f9cdb50 2024-04-21T08:48:24.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.283+0000 7f6cae7fc640 1 --2- 172.21.15.43:0/3808963141 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6ca40a3d20 0x7f6ca40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.284+0000 7f6cbdb4f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cd230 0x56375f9cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.284+0000 7f6cbdb4f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cd230 0x56375f9cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36716/0 (socket says 172.21.15.43:36716) 2024-04-21T08:48:24.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.284+0000 7f6cbdb4f640 1 -- 172.21.15.43:0/2747580103 learned_addr learned my addr 172.21.15.43:0/2747580103 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:24.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.284+0000 7f6cbdb4f640 1 --2- 172.21.15.43:0/3808963141 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6ca40a3d20 0x7f6ca40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.284+0000 7f6cae7fc640 1 -- 172.21.15.43:0/3808963141 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f6cb40d10d0 con 0x7f6cb8101080 2024-04-21T08:48:24.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.284+0000 7f6cbd34e640 1 --2- 172.21.15.43:0/2747580103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56375f9cdb50 0x56375f9da950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.284+0000 7f6cbcb4d640 1 --2- 172.21.15.43:0/2747580103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56375f9dae90 0x56375f9dd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.284+0000 7f6cbdb4f640 1 -- 172.21.15.43:0/2747580103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56375f9cdb50 msgr2=0x56375f9da950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.284+0000 7f6cbdb4f640 1 --2- 172.21.15.43:0/2747580103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56375f9cdb50 0x56375f9da950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.284+0000 7f6cbdb4f640 1 -- 172.21.15.43:0/2747580103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56375f9dae90 msgr2=0x56375f9dd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.284+0000 7f6cbdb4f640 1 --2- 172.21.15.43:0/2747580103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56375f9dae90 0x56375f9dd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.284+0000 7f6cbdb4f640 1 -- 172.21.15.43:0/2747580103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56375f70e7d0 con 0x56375f9cd230 2024-04-21T08:48:24.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.284+0000 7f6cbdb4f640 1 --2- 172.21.15.43:0/2747580103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cd230 0x56375f9cd610 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0037040 tx=0x7f6cb0032f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94762ec8641c8349 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.284+0000 7f6cbdb4f640 1 --2- 172.21.15.43:0/3808963141 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6ca40a3d20 0x7f6ca40a61e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0009ca0 tx=0x7f6cb0042000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.284+0000 7f6ca37fe640 1 -- 172.21.15.43:0/2747580103 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0051070 con 0x56375f9cd230 2024-04-21T08:48:24.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.284+0000 7f6ca37fe640 1 -- 172.21.15.43:0/2747580103 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cb00327c0 con 0x56375f9cd230 2024-04-21T08:48:24.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.284+0000 7f6ca37fe640 1 -- 172.21.15.43:0/2747580103 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0039040 con 0x56375f9cd230 2024-04-21T08:48:24.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.285+0000 7f6cc21a5740 1 -- 172.21.15.43:0/2747580103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cd230 msgr2=0x56375f9cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.285+0000 7f6cc21a5740 1 --2- 172.21.15.43:0/2747580103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cd230 0x56375f9cd610 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0037040 tx=0x7f6cb0032f70 comp rx=0 tx=0).stop 2024-04-21T08:48:24.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.285+0000 7f6cc21a5740 1 -- 172.21.15.43:0/2747580103 shutdown_connections 2024-04-21T08:48:24.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.285+0000 7f6cc21a5740 1 --2- 172.21.15.43:0/2747580103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56375f9dae90 0x56375f9dd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.285+0000 7f6cc21a5740 1 --2- 172.21.15.43:0/2747580103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56375f9cdb50 0x56375f9da950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.285+0000 7f6cc21a5740 1 --2- 172.21.15.43:0/2747580103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cd230 0x56375f9cd610 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.285+0000 7f6cc21a5740 1 -- 172.21.15.43:0/2747580103 >> 172.21.15.43:0/2747580103 conn(0x56375f9c7cb0 msgr2=0x56375f9c8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.285+0000 7f6cc21a5740 1 -- 172.21.15.43:0/2747580103 shutdown_connections 2024-04-21T08:48:24.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.285+0000 7f6cc21a5740 1 -- 172.21.15.43:0/2747580103 wait complete. 2024-04-21T08:48:24.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.285+0000 7f6cc21a5740 1 -- start start 2024-04-21T08:48:24.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.286+0000 7f6cc21a5740 1 Processor -- start 2024-04-21T08:48:24.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.286+0000 7f6cc21a5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cd230 0x56375f9d7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.286+0000 7f6cc21a5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56375f9cdb50 0x56375f9d83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.286+0000 7f6cc21a5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56375f9dae90 0x56375f9d8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.286+0000 7f6cc21a5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56375f9ddc70 con 0x56375f9cd230 2024-04-21T08:48:24.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.286+0000 7f6cc21a5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56375f8eb2f0 con 0x56375f9dae90 2024-04-21T08:48:24.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.286+0000 7f6cc21a5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56375f9dddf0 con 0x56375f9cdb50 2024-04-21T08:48:24.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.286+0000 7f6cc21a5740 10 client.0 _ll_register_callbacks cb 0x56375f8f8580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:24.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.286+0000 7f6cbd34e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cd230 0x56375f9d7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.286+0000 7f6cbd34e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cd230 0x56375f9d7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36724/0 (socket says 172.21.15.43:36724) 2024-04-21T08:48:24.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.286+0000 7f6cbd34e640 1 -- 172.21.15.43:0/675561215 learned_addr learned my addr 172.21.15.43:0/675561215 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:24.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.286+0000 7f6cbdb4f640 1 --2- 172.21.15.43:0/675561215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56375f9dae90 0x56375f9d8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.286+0000 7f6cbcb4d640 1 --2- 172.21.15.43:0/675561215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56375f9cdb50 0x56375f9d83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.286+0000 7f6cbd34e640 1 -- 172.21.15.43:0/675561215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56375f9cdb50 msgr2=0x56375f9d83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.286+0000 7f6cbd34e640 1 --2- 172.21.15.43:0/675561215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56375f9cdb50 0x56375f9d83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.286+0000 7f6cbd34e640 1 -- 172.21.15.43:0/675561215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56375f9dae90 msgr2=0x56375f9d8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.286+0000 7f6cbd34e640 1 --2- 172.21.15.43:0/675561215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56375f9dae90 0x56375f9d8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.286+0000 7f6cbd34e640 1 -- 172.21.15.43:0/675561215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56375f6443f0 con 0x56375f9cd230 2024-04-21T08:48:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.286+0000 7f6cbdb4f640 1 --2- 172.21.15.43:0/675561215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56375f9dae90 0x56375f9d8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.287+0000 7f6cbd34e640 1 --2- 172.21.15.43:0/675561215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cd230 0x56375f9d7e90 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4004ad0 tx=0x7f6cb409cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d271f919a76d7639 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.287+0000 7f6ca0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.287+0000 7f6ca0ff9640 1 -- 172.21.15.43:0/675561215 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4005680 con 0x56375f9cd230 2024-04-21T08:48:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.287+0000 7f6ca0ff9640 1 -- 172.21.15.43:0/675561215 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f6cb40bd0b0 con 0x56375f9cd230 2024-04-21T08:48:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.287+0000 7f6cc21a5740 1 -- 172.21.15.43:0/675561215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cb00327c0 con 0x56375f9cd230 2024-04-21T08:48:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.287+0000 7f6ca0ff9640 1 -- 172.21.15.43:0/675561215 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6cb40bd2b0 con 0x56375f9cd230 2024-04-21T08:48:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.287+0000 7f6cc21a5740 10 client.19998 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.287+0000 7f6cc21a5740 1 -- 172.21.15.43:0/675561215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56375f6040f0 con 0x56375f9cd230 2024-04-21T08:48:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.287+0000 7f6ca0ff9640 1 -- 172.21.15.43:0/675561215 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cb00327c0 con 0x56375f9cd230 2024-04-21T08:48:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.287+0000 7f6ca0ff9640 1 -- 172.21.15.43:0/675561215 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6cb40a5430 con 0x56375f9cd230 2024-04-21T08:48:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.287+0000 7f6cc21a5740 10 client.19998 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.287+0000 7f6cc21a5740 10 client.19998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.287+0000 7f6cc21a5740 20 client.19998 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.287+0000 7f6cc21a5740 1 --2- 172.21.15.43:0/675561215 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56375f9eaf00 0x56375f9ed330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.287+0000 7f6cc21a5740 4 client.19998 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.287+0000 7f6cc21a5740 1 -- 172.21.15.43:0/675561215 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56375f93fc00 con 0x56375f9eaf00 2024-04-21T08:48:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.288+0000 7f6cbcb4d640 1 --2- 172.21.15.43:0/675561215 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56375f9eaf00 0x56375f9ed330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.288+0000 7f6ca0ff9640 10 client.19998 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.288+0000 7f6cbcb4d640 1 --2- 172.21.15.43:0/675561215 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56375f9eaf00 0x56375f9ed330 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42b1106ee2989672 server_cookie=db22d504ab1693b7 in_seq=0 out_seq=0 2024-04-21T08:48:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.289+0000 7f6ca0ff9640 1 -- 172.21.15.43:0/675561215 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56375f93fc00 con 0x56375f9eaf00 2024-04-21T08:48:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.289+0000 7f6ca0ff9640 10 client.19998 handle_command_reply: tid=0 2024-04-21T08:48:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.289+0000 7f6cc21a5740 1 client.19998 shutdown 2024-04-21T08:48:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.289+0000 7f6cc21a5740 1 -- 172.21.15.43:0/675561215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cd230 msgr2=0x56375f9d7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.289+0000 7f6cc21a5740 1 --2- 172.21.15.43:0/675561215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cd230 0x56375f9d7e90 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4004ad0 tx=0x7f6cb409cae0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.289+0000 7f6cc21a5740 1 -- 172.21.15.43:0/675561215 shutdown_connections 2024-04-21T08:48:24.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.289+0000 7f6cc21a5740 1 --2- 172.21.15.43:0/675561215 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56375f9eaf00 0x56375f9ed330 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.289+0000 7f6cc21a5740 1 --2- 172.21.15.43:0/675561215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56375f9dae90 0x56375f9d8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.289+0000 7f6cc21a5740 1 --2- 172.21.15.43:0/675561215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56375f9cdb50 0x56375f9d83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.289+0000 7f6cc21a5740 1 --2- 172.21.15.43:0/675561215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cd230 0x56375f9d7e90 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.289+0000 7f6ca0ff9640 0 client.19998 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:24.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.289+0000 7f6cc21a5740 1 -- 172.21.15.43:0/675561215 >> 172.21.15.43:0/675561215 conn(0x56375f9c7cb0 msgr2=0x56375f9c9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.289+0000 7f6cc21a5740 1 -- 172.21.15.43:0/675561215 shutdown_connections 2024-04-21T08:48:24.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.289+0000 7f6cc21a5740 1 -- 172.21.15.43:0/675561215 wait complete. 2024-04-21T08:48:24.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.289+0000 7f6cc21a5740 20 client.19998 trim_cache size 0 max 16384 2024-04-21T08:48:24.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.306+0000 7f6cc21a5740 1 Processor -- start 2024-04-21T08:48:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.307+0000 7f6cc21a5740 1 -- start start 2024-04-21T08:48:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.307+0000 7f6cc21a5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cd230 0x56375f9da5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.307+0000 7f6cc21a5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56375f9cdb50 0x56375f9e6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.307+0000 7f6cc21a5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56375f9dae90 0x56375f9d6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.307+0000 7f6cc21a5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb81ad330 con 0x56375f9cd230 2024-04-21T08:48:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.307+0000 7f6cc21a5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb81aee70 con 0x56375f9cdb50 2024-04-21T08:48:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.307+0000 7f6cc21a5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb81ada00 con 0x56375f9dae90 2024-04-21T08:48:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.307+0000 7f6cbd34e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cd230 0x56375f9da5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.307+0000 7f6cbd34e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cd230 0x56375f9da5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36738/0 (socket says 172.21.15.43:36738) 2024-04-21T08:48:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.307+0000 7f6cbd34e640 1 -- 172.21.15.43:0/1631265461 learned_addr learned my addr 172.21.15.43:0/1631265461 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.307+0000 7f6cbd34e640 1 -- 172.21.15.43:0/1631265461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56375f9dae90 msgr2=0x56375f9d6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.307+0000 7f6cbd34e640 1 --2- 172.21.15.43:0/1631265461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56375f9dae90 0x56375f9d6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.307+0000 7f6cbd34e640 1 -- 172.21.15.43:0/1631265461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56375f9cdb50 msgr2=0x56375f9e6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.307+0000 7f6cbd34e640 1 --2- 172.21.15.43:0/1631265461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56375f9cdb50 0x56375f9e6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.307+0000 7f6cbd34e640 1 -- 172.21.15.43:0/1631265461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56375f9c21c0 con 0x56375f9cd230 2024-04-21T08:48:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.307+0000 7f6cbd34e640 1 --2- 172.21.15.43:0/1631265461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cd230 0x56375f9da5c0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4099620 tx=0x7f6cb4009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f9cc5b05c8fdfd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.307+0000 7f6ca2ffd640 1 -- 172.21.15.43:0/1631265461 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb409c5b0 con 0x56375f9cd230 2024-04-21T08:48:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.307+0000 7f6ca2ffd640 1 -- 172.21.15.43:0/1631265461 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cb40a1000 con 0x56375f9cd230 2024-04-21T08:48:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.307+0000 7f6ca2ffd640 1 -- 172.21.15.43:0/1631265461 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb40992c0 con 0x56375f9cd230 2024-04-21T08:48:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.308+0000 7f6cc21a5740 1 -- 172.21.15.43:0/1631265461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cd230 msgr2=0x56375f9da5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.308+0000 7f6cc21a5740 1 --2- 172.21.15.43:0/1631265461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cd230 0x56375f9da5c0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4099620 tx=0x7f6cb4009330 comp rx=0 tx=0).stop 2024-04-21T08:48:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.308+0000 7f6cc21a5740 1 -- 172.21.15.43:0/1631265461 shutdown_connections 2024-04-21T08:48:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.308+0000 7f6cc21a5740 1 --2- 172.21.15.43:0/1631265461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56375f9dae90 0x56375f9d6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.308+0000 7f6cc21a5740 1 --2- 172.21.15.43:0/1631265461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56375f9cdb50 0x56375f9e6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.308+0000 7f6cc21a5740 1 --2- 172.21.15.43:0/1631265461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cd230 0x56375f9da5c0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.308+0000 7f6cc21a5740 1 -- 172.21.15.43:0/1631265461 >> 172.21.15.43:0/1631265461 conn(0x56375f9c7cb0 msgr2=0x56375f9c64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.308+0000 7f6cc21a5740 1 -- 172.21.15.43:0/1631265461 shutdown_connections 2024-04-21T08:48:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.308+0000 7f6cc21a5740 1 -- 172.21.15.43:0/1631265461 wait complete. 2024-04-21T08:48:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.308+0000 7f6cc21a5740 1 -- start start 2024-04-21T08:48:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.308+0000 7f6cc21a5740 1 Processor -- start 2024-04-21T08:48:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.309+0000 7f6cc21a5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56375f9cd230 0x56375f9e3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.309+0000 7f6cc21a5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cdb50 0x56375f9de670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.309+0000 7f6cc21a5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56375f9dae90 0x56375f9debb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.309+0000 7f6cc21a5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb81ae7a0 con 0x56375f9cdb50 2024-04-21T08:48:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.309+0000 7f6cc21a5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb81ae0d0 con 0x56375f9dae90 2024-04-21T08:48:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.309+0000 7f6cc21a5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb81ab330 con 0x56375f9cd230 2024-04-21T08:48:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.309+0000 7f6cc21a5740 10 client.0 _ll_register_callbacks cb 0x7f6cb400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.309+0000 7f6cbcb4d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cdb50 0x56375f9de670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.309+0000 7f6cbcb4d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cdb50 0x56375f9de670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36746/0 (socket says 172.21.15.43:36746) 2024-04-21T08:48:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.309+0000 7f6cbcb4d640 1 -- 172.21.15.43:0/3101152180 learned_addr learned my addr 172.21.15.43:0/3101152180 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.309+0000 7f6cbdb4f640 1 --2- 172.21.15.43:0/3101152180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56375f9dae90 0x56375f9debb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.309+0000 7f6cbd34e640 1 --2- 172.21.15.43:0/3101152180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56375f9cd230 0x56375f9e3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.309+0000 7f6cbcb4d640 1 -- 172.21.15.43:0/3101152180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56375f9cd230 msgr2=0x56375f9e3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.309+0000 7f6cbcb4d640 1 --2- 172.21.15.43:0/3101152180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56375f9cd230 0x56375f9e3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.309+0000 7f6cbcb4d640 1 -- 172.21.15.43:0/3101152180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56375f9dae90 msgr2=0x56375f9debb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.309+0000 7f6cbcb4d640 1 --2- 172.21.15.43:0/3101152180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56375f9dae90 0x56375f9debb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.309+0000 7f6cbcb4d640 1 -- 172.21.15.43:0/3101152180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56375f9df0f0 con 0x56375f9cdb50 2024-04-21T08:48:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.309+0000 7f6cbcb4d640 1 --2- 172.21.15.43:0/3101152180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cdb50 0x56375f9de670 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8004650 tx=0x7f6ca801bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256bc638f7972129 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.309+0000 7f6ca0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.310+0000 7f6ca0ff9640 1 -- 172.21.15.43:0/3101152180 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca801b530 con 0x56375f9cdb50 2024-04-21T08:48:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.310+0000 7f6ca0ff9640 1 -- 172.21.15.43:0/3101152180 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f6ca801b6d0 con 0x56375f9cdb50 2024-04-21T08:48:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.310+0000 7f6cc21a5740 1 -- 172.21.15.43:0/3101152180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56375f939a70 con 0x56375f9cdb50 2024-04-21T08:48:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.310+0000 7f6ca0ff9640 1 -- 172.21.15.43:0/3101152180 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6ca802b050 con 0x56375f9cdb50 2024-04-21T08:48:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.310+0000 7f6cc21a5740 10 client.20004 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.310+0000 7f6cc21a5740 1 -- 172.21.15.43:0/3101152180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56375fa51220 con 0x56375f9cdb50 2024-04-21T08:48:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.310+0000 7f6ca0ff9640 1 -- 172.21.15.43:0/3101152180 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56375f939a70 con 0x56375f9cdb50 2024-04-21T08:48:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.310+0000 7f6ca0ff9640 1 -- 172.21.15.43:0/3101152180 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6ca8012570 con 0x56375f9cdb50 2024-04-21T08:48:24.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.310+0000 7f6cc21a5740 10 client.20004 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:24.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.310+0000 7f6cc21a5740 10 client.20004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:24.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.310+0000 7f6cc21a5740 20 client.20004 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:24.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.310+0000 7f6cc21a5740 1 --2- 172.21.15.43:0/3101152180 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56375fa519d0 0x56375fa53df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.310+0000 7f6cc21a5740 4 client.20004 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:48:24.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.310+0000 7f6cc21a5740 1 -- 172.21.15.43:0/3101152180 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56375f8b5590 con 0x56375fa519d0 2024-04-21T08:48:24.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.310+0000 7f6cbd34e640 1 --2- 172.21.15.43:0/3101152180 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56375fa519d0 0x56375fa53df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.311+0000 7f6cbd34e640 1 --2- 172.21.15.43:0/3101152180 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56375fa519d0 0x56375fa53df0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af69cdecd8f88ea1 server_cookie=1763e0f8defecc9f in_seq=0 out_seq=0 2024-04-21T08:48:24.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.311+0000 7f6ca0ff9640 10 client.20004 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:24.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.311+0000 7f6ca0ff9640 1 -- 172.21.15.43:0/3101152180 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56375f8b5590 con 0x56375fa519d0 2024-04-21T08:48:24.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.311+0000 7f6ca0ff9640 10 client.20004 handle_command_reply: tid=0 2024-04-21T08:48:24.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.311+0000 7f6cc21a5740 1 client.20004 shutdown 2024-04-21T08:48:24.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.311+0000 7f6cc21a5740 1 -- 172.21.15.43:0/3101152180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cdb50 msgr2=0x56375f9de670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.311+0000 7f6cc21a5740 1 --2- 172.21.15.43:0/3101152180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cdb50 0x56375f9de670 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8004650 tx=0x7f6ca801bf90 comp rx=0 tx=0).stop 2024-04-21T08:48:24.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.311+0000 7f6cc21a5740 1 -- 172.21.15.43:0/3101152180 shutdown_connections 2024-04-21T08:48:24.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.311+0000 7f6cc21a5740 1 --2- 172.21.15.43:0/3101152180 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56375fa519d0 0x56375fa53df0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.311+0000 7f6cc21a5740 1 --2- 172.21.15.43:0/3101152180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56375f9dae90 0x56375f9debb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.311+0000 7f6cc21a5740 1 --2- 172.21.15.43:0/3101152180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56375f9cdb50 0x56375f9de670 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.311+0000 7f6ca0ff9640 0 client.20004 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:24.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.311+0000 7f6cc21a5740 1 --2- 172.21.15.43:0/3101152180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56375f9cd230 0x56375f9e3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.311+0000 7f6cc21a5740 1 -- 172.21.15.43:0/3101152180 >> 172.21.15.43:0/3101152180 conn(0x56375f9c7cb0 msgr2=0x56375f9d86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.311+0000 7f6cc21a5740 1 -- 172.21.15.43:0/3101152180 shutdown_connections 2024-04-21T08:48:24.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.311+0000 7f6cc21a5740 1 -- 172.21.15.43:0/3101152180 wait complete. 2024-04-21T08:48:24.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.311+0000 7f6cc21a5740 20 client.20004 trim_cache size 0 max 16384 2024-04-21T08:48:24.317 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:48:24.317 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:48:24.318 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:48:24.318 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:48:24.318 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:24 smithi043 ceph-mon[27978]: pgmap v294: 129 pgs: 129 active+clean; 3.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:48:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.312+0000 7f6cbf5b2640 1 -- 172.21.15.43:0/3808963141 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6ca40a3d20 msgr2=0x7f6ca40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.312+0000 7f6cbf5b2640 1 --2- 172.21.15.43:0/3808963141 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6ca40a3d20 0x7f6ca40a61e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0009ca0 tx=0x7f6cb0042000 comp rx=0 tx=0).stop 2024-04-21T08:48:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.312+0000 7f6cbf5b2640 1 -- 172.21.15.43:0/3808963141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb8101080 msgr2=0x7f6cb8198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.312+0000 7f6cbf5b2640 1 --2- 172.21.15.43:0/3808963141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb8101080 0x7f6cb8198790 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400e9f0 tx=0x7f6cb400edf0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.312+0000 7f6cbf5b2640 1 -- 172.21.15.43:0/3808963141 shutdown_connections 2024-04-21T08:48:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.312+0000 7f6cbf5b2640 1 --2- 172.21.15.43:0/3808963141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb819d000 0x7f6cb819d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.312+0000 7f6cbf5b2640 1 --2- 172.21.15.43:0/3808963141 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6ca40a3d20 0x7f6ca40a61e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.312+0000 7f6cbf5b2640 1 --2- 172.21.15.43:0/3808963141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cb8106150 0x7f6cb8198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.312+0000 7f6cbf5b2640 1 --2- 172.21.15.43:0/3808963141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb8101080 0x7f6cb8198790 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.312+0000 7f6cbf5b2640 1 -- 172.21.15.43:0/3808963141 >> 172.21.15.43:0/3808963141 conn(0x7f6cb8077fd0 msgr2=0x7f6cb80fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.312+0000 7f6cbf5b2640 1 -- 172.21.15.43:0/3808963141 shutdown_connections 2024-04-21T08:48:24.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.312+0000 7f6cbf5b2640 1 -- 172.21.15.43:0/3808963141 wait complete. 2024-04-21T08:48:24.322 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:48:24.322 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:48:24.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.426+0000 7f611f3a4640 1 -- 172.21.15.43:0/2080353856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6118101ad0 msgr2=0x7f611810dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.426+0000 7f611f3a4640 1 --2- 172.21.15.43:0/2080353856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6118101ad0 0x7f611810dcb0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f6110009ca0 tx=0x7f611002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.426+0000 7f611f3a4640 1 -- 172.21.15.43:0/2080353856 shutdown_connections 2024-04-21T08:48:24.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.426+0000 7f611f3a4640 1 --2- 172.21.15.43:0/2080353856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6118101ad0 0x7f611810dcb0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.426+0000 7f611f3a4640 1 --2- 172.21.15.43:0/2080353856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6118101130 0x7f6118101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.426+0000 7f611f3a4640 1 --2- 172.21.15.43:0/2080353856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6118106f10 0x7f61181072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.426+0000 7f611f3a4640 1 -- 172.21.15.43:0/2080353856 >> 172.21.15.43:0/2080353856 conn(0x7f6118077e30 msgr2=0x7f6118078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.426+0000 7f611f3a4640 1 -- 172.21.15.43:0/2080353856 shutdown_connections 2024-04-21T08:48:24.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.426+0000 7f611f3a4640 1 -- 172.21.15.43:0/2080353856 wait complete. 2024-04-21T08:48:24.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.427+0000 7f611f3a4640 1 Processor -- start 2024-04-21T08:48:24.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.427+0000 7f611f3a4640 1 -- start start 2024-04-21T08:48:24.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.427+0000 7f611f3a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6118101130 0x7f6118198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.427+0000 7f611f3a4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6118101ad0 0x7f6118198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.427+0000 7f611f3a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6118106f10 0x7f611819d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.427+0000 7f611f3a4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6118111140 con 0x7f6118101130 2024-04-21T08:48:24.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.427+0000 7f611f3a4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6118110fc0 con 0x7f6118106f10 2024-04-21T08:48:24.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.427+0000 7f611f3a4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f61181112c0 con 0x7f6118101ad0 2024-04-21T08:48:24.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.427+0000 7f611d140640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6118101130 0x7f6118198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.427+0000 7f611d140640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6118101130 0x7f6118198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36752/0 (socket says 172.21.15.43:36752) 2024-04-21T08:48:24.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.427+0000 7f611d140640 1 -- 172.21.15.43:0/137622022 learned_addr learned my addr 172.21.15.43:0/137622022 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:24.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.427+0000 7f611d941640 1 --2- 172.21.15.43:0/137622022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6118106f10 0x7f611819d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.427+0000 7f611c93f640 1 --2- 172.21.15.43:0/137622022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6118101ad0 0x7f6118198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.427+0000 7f611d140640 1 -- 172.21.15.43:0/137622022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6118101ad0 msgr2=0x7f6118198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.427+0000 7f611d140640 1 --2- 172.21.15.43:0/137622022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6118101ad0 0x7f6118198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.427+0000 7f611d140640 1 -- 172.21.15.43:0/137622022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6118106f10 msgr2=0x7f611819d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.427+0000 7f611d140640 1 --2- 172.21.15.43:0/137622022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6118106f10 0x7f611819d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.427+0000 7f611d140640 1 -- 172.21.15.43:0/137622022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f611819d730 con 0x7f6118101130 2024-04-21T08:48:24.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.427+0000 7f611d941640 1 --2- 172.21.15.43:0/137622022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6118106f10 0x7f611819d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:24.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.428+0000 7f611d140640 1 --2- 172.21.15.43:0/137622022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6118101130 0x7f6118198740 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f611400b4a0 tx=0x7f611400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.428+0000 7f610e7fc640 1 -- 172.21.15.43:0/137622022 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114014070 con 0x7f6118101130 2024-04-21T08:48:24.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.428+0000 7f611f3a4640 1 -- 172.21.15.43:0/137622022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f611819da20 con 0x7f6118101130 2024-04-21T08:48:24.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.428+0000 7f610e7fc640 1 -- 172.21.15.43:0/137622022 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6114010ac0 con 0x7f6118101130 2024-04-21T08:48:24.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.428+0000 7f610e7fc640 1 -- 172.21.15.43:0/137622022 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114017c30 con 0x7f6118101130 2024-04-21T08:48:24.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.428+0000 7f611f3a4640 1 -- 172.21.15.43:0/137622022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61181a52b0 con 0x7f6118101130 2024-04-21T08:48:24.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.429+0000 7f611f3a4640 1 -- 172.21.15.43:0/137622022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6118069230 con 0x7f6118101130 2024-04-21T08:48:24.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.430+0000 7f610e7fc640 1 -- 172.21.15.43:0/137622022 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6114017070 con 0x7f6118101130 2024-04-21T08:48:24.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.430+0000 7f610e7fc640 1 --2- 172.21.15.43:0/137622022 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f61040a3d20 0x7f61040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.431+0000 7f611c93f640 1 --2- 172.21.15.43:0/137622022 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f61040a3d20 0x7f61040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.431+0000 7f610e7fc640 1 -- 172.21.15.43:0/137622022 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f61140d0330 con 0x7f6118101130 2024-04-21T08:48:24.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.431+0000 7f611c93f640 1 --2- 172.21.15.43:0/137622022 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f61040a3d20 0x7f61040a61e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f611806b720 tx=0x7f610800a4c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.434+0000 7f610e7fc640 1 -- 172.21.15.43:0/137622022 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f611409c6c0 con 0x7f6118101130 2024-04-21T08:48:24.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:24 smithi175 ceph-mon[31920]: pgmap v294: 129 pgs: 129 active+clean; 3.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:48:24.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:24 smithi112 ceph-mon[31085]: pgmap v294: 129 pgs: 129 active+clean; 3.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:48:24.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.663+0000 7f611f3a4640 1 -- 172.21.15.43:0/137622022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61181a54e0 con 0x7f6118101130 2024-04-21T08:48:24.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.664+0000 7f610e7fc640 1 -- 172.21.15.43:0/137622022 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11935 (secure 0 0 0) 0x7f611409bfc0 con 0x7f6118101130 2024-04-21T08:48:24.663 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:48:24.663 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:46:17.283806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27556":{"gid":27556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/229985237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":229985237},{"type":"v1","addr":"172.21.15.175:6837","nonce":229985237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24425,24412,24487,24527,24491]},"id":1}]} 2024-04-21T08:48:24.664 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-04-21T08:48:24.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.666+0000 7f611f3a4640 1 -- 172.21.15.43:0/137622022 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f61040a3d20 msgr2=0x7f61040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.666+0000 7f611f3a4640 1 --2- 172.21.15.43:0/137622022 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f61040a3d20 0x7f61040a61e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f611806b720 tx=0x7f610800a4c0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.666+0000 7f611f3a4640 1 -- 172.21.15.43:0/137622022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6118101130 msgr2=0x7f6118198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.666+0000 7f611f3a4640 1 --2- 172.21.15.43:0/137622022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6118101130 0x7f6118198740 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f611400b4a0 tx=0x7f611400b960 comp rx=0 tx=0).stop 2024-04-21T08:48:24.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.666+0000 7f611f3a4640 1 -- 172.21.15.43:0/137622022 shutdown_connections 2024-04-21T08:48:24.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.666+0000 7f611f3a4640 1 --2- 172.21.15.43:0/137622022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6118106f10 0x7f611819d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.666+0000 7f611f3a4640 1 --2- 172.21.15.43:0/137622022 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f61040a3d20 0x7f61040a61e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.666+0000 7f611f3a4640 1 --2- 172.21.15.43:0/137622022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6118101ad0 0x7f6118198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.666+0000 7f611f3a4640 1 --2- 172.21.15.43:0/137622022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6118101130 0x7f6118198740 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.666+0000 7f611f3a4640 1 -- 172.21.15.43:0/137622022 >> 172.21.15.43:0/137622022 conn(0x7f6118077e30 msgr2=0x7f6118105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.666+0000 7f611f3a4640 1 -- 172.21.15.43:0/137622022 shutdown_connections 2024-04-21T08:48:24.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.666+0000 7f611f3a4640 1 -- 172.21.15.43:0/137622022 wait complete. 2024-04-21T08:48:24.676 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:48:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.781+0000 7f2e0c025640 1 -- 172.21.15.43:0/1551328434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e0406b1f0 msgr2=0x7f2e04106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.781+0000 7f2e0c025640 1 --2- 172.21.15.43:0/1551328434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e0406b1f0 0x7f2e04106b90 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009900 tx=0x7f2df802bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.781+0000 7f2e0c025640 1 -- 172.21.15.43:0/1551328434 shutdown_connections 2024-04-21T08:48:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.781+0000 7f2e0c025640 1 --2- 172.21.15.43:0/1551328434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e04107270 0x7f2e0410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.781+0000 7f2e0c025640 1 --2- 172.21.15.43:0/1551328434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e0406b1f0 0x7f2e04106b90 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.781+0000 7f2e0c025640 1 --2- 172.21.15.43:0/1551328434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e0406a840 0x7f2e0406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.781+0000 7f2e0c025640 1 -- 172.21.15.43:0/1551328434 >> 172.21.15.43:0/1551328434 conn(0x7f2e04077eb0 msgr2=0x7f2e040782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.781+0000 7f2e0c025640 1 -- 172.21.15.43:0/1551328434 shutdown_connections 2024-04-21T08:48:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.781+0000 7f2e0c025640 1 -- 172.21.15.43:0/1551328434 wait complete. 2024-04-21T08:48:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.782+0000 7f2e0c025640 1 Processor -- start 2024-04-21T08:48:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.782+0000 7f2e0c025640 1 -- start start 2024-04-21T08:48:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.782+0000 7f2e0c025640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e0406a840 0x7f2e041a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.782+0000 7f2e0c025640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e04107270 0x7f2e041a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.782+0000 7f2e0c025640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e0419a740 0x7f2e0419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.782+0000 7f2e0c025640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2e04112200 con 0x7f2e04107270 2024-04-21T08:48:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.782+0000 7f2e0c025640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e04112080 con 0x7f2e0419a740 2024-04-21T08:48:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.782+0000 7f2e0c025640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2e04112380 con 0x7f2e0406a840 2024-04-21T08:48:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.782+0000 7f2e095c0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e04107270 0x7f2e041a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.782+0000 7f2e095c0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e04107270 0x7f2e041a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36776/0 (socket says 172.21.15.43:36776) 2024-04-21T08:48:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.782+0000 7f2e095c0640 1 -- 172.21.15.43:0/1437212901 learned_addr learned my addr 172.21.15.43:0/1437212901 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.782+0000 7f2e0a5c2640 1 --2- 172.21.15.43:0/1437212901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e0419a740 0x7f2e0419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.782+0000 7f2e09dc1640 1 --2- 172.21.15.43:0/1437212901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e0406a840 0x7f2e041a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.782+0000 7f2e095c0640 1 -- 172.21.15.43:0/1437212901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e0406a840 msgr2=0x7f2e041a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.782+0000 7f2e095c0640 1 --2- 172.21.15.43:0/1437212901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e0406a840 0x7f2e041a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.782+0000 7f2e095c0640 1 -- 172.21.15.43:0/1437212901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e0419a740 msgr2=0x7f2e0419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.782+0000 7f2e095c0640 1 --2- 172.21.15.43:0/1437212901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e0419a740 0x7f2e0419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.782+0000 7f2e095c0640 1 -- 172.21.15.43:0/1437212901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0419b340 con 0x7f2e04107270 2024-04-21T08:48:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.782+0000 7f2e09dc1640 1 --2- 172.21.15.43:0/1437212901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e0406a840 0x7f2e041a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.783+0000 7f2e095c0640 1 --2- 172.21.15.43:0/1437212901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e04107270 0x7f2e041a0b70 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f2df8000c00 tx=0x7f2df802c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.783+0000 7f2e02ffd640 1 -- 172.21.15.43:0/1437212901 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8043020 con 0x7f2e04107270 2024-04-21T08:48:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.783+0000 7f2e02ffd640 1 -- 172.21.15.43:0/1437212901 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2df802cc60 con 0x7f2e04107270 2024-04-21T08:48:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.783+0000 7f2e02ffd640 1 -- 172.21.15.43:0/1437212901 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8003ca0 con 0x7f2e04107270 2024-04-21T08:48:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.783+0000 7f2e0c025640 1 -- 172.21.15.43:0/1437212901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e0419b5d0 con 0x7f2e04107270 2024-04-21T08:48:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.783+0000 7f2e0c025640 1 -- 172.21.15.43:0/1437212901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e041a7720 con 0x7f2e04107270 2024-04-21T08:48:24.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.785+0000 7f2e02ffd640 1 -- 172.21.15.43:0/1437212901 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2df8046020 con 0x7f2e04107270 2024-04-21T08:48:24.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.785+0000 7f2e0ec13740 1 Processor -- start 2024-04-21T08:48:24.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.786+0000 7f2e0ec13740 1 -- start start 2024-04-21T08:48:24.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.786+0000 7f2e0ec13740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629bf88e4d0 0x5629bf88e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.786+0000 7f2e0ec13740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629bf88ee80 0x5629bf897430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.786+0000 7f2e0ec13740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf897a00 0x5629bf899e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.786+0000 7f2e0ec13740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5629bf7f31f0 con 0x5629bf897a00 2024-04-21T08:48:24.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.786+0000 7f2e0ec13740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5629bf7904b0 con 0x5629bf88ee80 2024-04-21T08:48:24.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.786+0000 7f2e0ec13740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5629bf7ef1b0 con 0x5629bf88e4d0 2024-04-21T08:48:24.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.786+0000 7f2e095c0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf897a00 0x5629bf899e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.786+0000 7f2e02ffd640 1 --2- 172.21.15.43:0/1437212901 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2dec0a3d20 0x7f2dec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.786+0000 7f2e0a5c2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629bf88e4d0 0x5629bf88e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.786+0000 7f2e09dc1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629bf88ee80 0x5629bf897430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.786+0000 7f2e095c0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf897a00 0x5629bf899e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36792/0 (socket says 172.21.15.43:36792) 2024-04-21T08:48:24.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.786+0000 7f2e095c0640 1 -- 172.21.15.43:0/344124376 learned_addr learned my addr 172.21.15.43:0/344124376 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:24.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.786+0000 7f2e02ffd640 1 -- 172.21.15.43:0/1437212901 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f2df80f2d30 con 0x7f2e04107270 2024-04-21T08:48:24.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.786+0000 7f2e0a5c2640 1 --2- 172.21.15.43:0/1437212901 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2dec0a3d20 0x7f2dec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.786+0000 7f2e09dc1640 1 -- 172.21.15.43:0/344124376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629bf88e4d0 msgr2=0x5629bf88e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.786+0000 7f2e09dc1640 1 --2- 172.21.15.43:0/344124376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629bf88e4d0 0x5629bf88e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.786+0000 7f2e09dc1640 1 -- 172.21.15.43:0/344124376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf897a00 msgr2=0x5629bf899e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.786+0000 7f2e09dc1640 1 --2- 172.21.15.43:0/344124376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf897a00 0x5629bf899e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.786+0000 7f2e09dc1640 1 -- 172.21.15.43:0/344124376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629bf7821b0 con 0x5629bf88ee80 2024-04-21T08:48:24.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.787+0000 7f2e09dc1640 1 --2- 172.21.15.43:0/344124376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629bf88ee80 0x5629bf897430 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc00bb30 tx=0x7f2dfc00bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3900ad35d30c523 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.787+0000 7f2e0a5c2640 1 --2- 172.21.15.43:0/1437212901 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2dec0a3d20 0x7f2dec0a61e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2df4005fd0 tx=0x7f2df4004ab0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.787+0000 7f2debfff640 1 -- 172.21.15.43:0/344124376 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc017420 con 0x5629bf88ee80 2024-04-21T08:48:24.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.787+0000 7f2debfff640 1 -- 172.21.15.43:0/344124376 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dfc017a70 con 0x5629bf88ee80 2024-04-21T08:48:24.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.787+0000 7f2debfff640 1 -- 172.21.15.43:0/344124376 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc00fa80 con 0x5629bf88ee80 2024-04-21T08:48:24.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.788+0000 7f2e0ec13740 1 -- 172.21.15.43:0/344124376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629bf88ee80 msgr2=0x5629bf897430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.788+0000 7f2e0ec13740 1 --2- 172.21.15.43:0/344124376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629bf88ee80 0x5629bf897430 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc00bb30 tx=0x7f2dfc00bf30 comp rx=0 tx=0).stop 2024-04-21T08:48:24.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.788+0000 7f2e0ec13740 1 -- 172.21.15.43:0/344124376 shutdown_connections 2024-04-21T08:48:24.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.788+0000 7f2e0ec13740 1 --2- 172.21.15.43:0/344124376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf897a00 0x5629bf899e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.788+0000 7f2e0ec13740 1 --2- 172.21.15.43:0/344124376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629bf88ee80 0x5629bf897430 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.788+0000 7f2e0ec13740 1 --2- 172.21.15.43:0/344124376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629bf88e4d0 0x5629bf88e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.788+0000 7f2e0ec13740 1 -- 172.21.15.43:0/344124376 >> 172.21.15.43:0/344124376 conn(0x5629bf888ba0 msgr2=0x5629bf888f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.788+0000 7f2e0ec13740 1 -- 172.21.15.43:0/344124376 shutdown_connections 2024-04-21T08:48:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.788+0000 7f2e0ec13740 1 -- 172.21.15.43:0/344124376 wait complete. 2024-04-21T08:48:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.788+0000 7f2e0ec13740 1 -- start start 2024-04-21T08:48:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.788+0000 7f2e0ec13740 1 Processor -- start 2024-04-21T08:48:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.789+0000 7f2e0ec13740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629bf88e4d0 0x5629bf897260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.789+0000 7f2e0ec13740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf88ee80 0x5629bf8945f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.789+0000 7f2e0ec13740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629bf897a00 0x5629bf894b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.789+0000 7f2e0ec13740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5629bf89a970 con 0x5629bf88ee80 2024-04-21T08:48:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.789+0000 7f2e0ec13740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5629bf7ac500 con 0x5629bf897a00 2024-04-21T08:48:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.789+0000 7f2e0ec13740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5629bf89aaf0 con 0x5629bf88e4d0 2024-04-21T08:48:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.789+0000 7f2e0ec13740 10 client.0 _ll_register_callbacks cb 0x5629bf77ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.789+0000 7f2e095c0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf88ee80 0x5629bf8945f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.789+0000 7f2e095c0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf88ee80 0x5629bf8945f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36806/0 (socket says 172.21.15.43:36806) 2024-04-21T08:48:24.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.789+0000 7f2e095c0640 1 -- 172.21.15.43:0/959048030 learned_addr learned my addr 172.21.15.43:0/959048030 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:24.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.789+0000 7f2e0a5c2640 1 --2- 172.21.15.43:0/959048030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629bf897a00 0x5629bf894b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.789+0000 7f2e09dc1640 1 --2- 172.21.15.43:0/959048030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629bf88e4d0 0x5629bf897260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.789+0000 7f2e095c0640 1 -- 172.21.15.43:0/959048030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629bf88e4d0 msgr2=0x5629bf897260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.789+0000 7f2e095c0640 1 --2- 172.21.15.43:0/959048030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629bf88e4d0 0x5629bf897260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.789+0000 7f2e095c0640 1 -- 172.21.15.43:0/959048030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629bf897a00 msgr2=0x5629bf894b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.789+0000 7f2e095c0640 1 --2- 172.21.15.43:0/959048030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629bf897a00 0x5629bf894b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.789+0000 7f2e095c0640 1 -- 172.21.15.43:0/959048030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629bf5053f0 con 0x5629bf88ee80 2024-04-21T08:48:24.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.789+0000 7f2e095c0640 1 --2- 172.21.15.43:0/959048030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf88ee80 0x5629bf8945f0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f2df80bae20 tx=0x7f2df802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c1804d1896d640 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.789+0000 7f2de97fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:24.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.789+0000 7f2de97fa640 1 -- 172.21.15.43:0/959048030 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8003e40 con 0x5629bf88ee80 2024-04-21T08:48:24.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.789+0000 7f2de97fa640 1 -- 172.21.15.43:0/959048030 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f2df80d30b0 con 0x5629bf88ee80 2024-04-21T08:48:24.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.789+0000 7f2e0ec13740 1 -- 172.21.15.43:0/959048030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dfc017a70 con 0x5629bf88ee80 2024-04-21T08:48:24.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.790+0000 7f2de97fa640 1 -- 172.21.15.43:0/959048030 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2df80bf0c0 con 0x5629bf88ee80 2024-04-21T08:48:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.790+0000 7f2e0ec13740 10 client.20022 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.790+0000 7f2e0ec13740 1 -- 172.21.15.43:0/959048030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5629bf5cf7d0 con 0x5629bf88ee80 2024-04-21T08:48:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.790+0000 7f2de97fa640 1 -- 172.21.15.43:0/959048030 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dfc017a70 con 0x5629bf88ee80 2024-04-21T08:48:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.790+0000 7f2de97fa640 1 -- 172.21.15.43:0/959048030 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2df80f02e0 con 0x5629bf88ee80 2024-04-21T08:48:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.790+0000 7f2e0ec13740 10 client.20022 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.790+0000 7f2e0ec13740 10 client.20022 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:48:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.790+0000 7f2e0ec13740 20 client.20022 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.790+0000 7f2e0ec13740 1 --2- 172.21.15.43:0/959048030 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5629bf8a7b40 0x5629bf8a9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.790+0000 7f2e0ec13740 4 client.20022 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.790+0000 7f2e0ec13740 1 -- 172.21.15.43:0/959048030 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629bf792930 con 0x5629bf8a7b40 2024-04-21T08:48:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.790+0000 7f2e09dc1640 1 --2- 172.21.15.43:0/959048030 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5629bf8a7b40 0x5629bf8a9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.791+0000 7f2e09dc1640 1 --2- 172.21.15.43:0/959048030 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5629bf8a7b40 0x5629bf8a9f80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cf2aacd9b8cca85c server_cookie=291e4fe265a4ac73 in_seq=0 out_seq=0 2024-04-21T08:48:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.791+0000 7f2de97fa640 10 client.20022 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:48:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.792+0000 7f2de97fa640 1 -- 172.21.15.43:0/959048030 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629bf792930 con 0x5629bf8a7b40 2024-04-21T08:48:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.792+0000 7f2de97fa640 10 client.20022 handle_command_reply: tid=0 2024-04-21T08:48:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.792+0000 7f2e0ec13740 1 client.20022 shutdown 2024-04-21T08:48:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.792+0000 7f2e0ec13740 1 -- 172.21.15.43:0/959048030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf88ee80 msgr2=0x5629bf8945f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.792+0000 7f2e0ec13740 1 --2- 172.21.15.43:0/959048030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf88ee80 0x5629bf8945f0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f2df80bae20 tx=0x7f2df802bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.792+0000 7f2e0ec13740 1 -- 172.21.15.43:0/959048030 shutdown_connections 2024-04-21T08:48:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.792+0000 7f2e0ec13740 1 --2- 172.21.15.43:0/959048030 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5629bf8a7b40 0x5629bf8a9f80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.792+0000 7f2e0ec13740 1 --2- 172.21.15.43:0/959048030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629bf897a00 0x5629bf894b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.792+0000 7f2e0ec13740 1 --2- 172.21.15.43:0/959048030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf88ee80 0x5629bf8945f0 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.792+0000 7f2e0ec13740 1 --2- 172.21.15.43:0/959048030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629bf88e4d0 0x5629bf897260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.792+0000 7f2de97fa640 0 client.20022 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:48:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.792+0000 7f2e0ec13740 1 -- 172.21.15.43:0/959048030 >> 172.21.15.43:0/959048030 conn(0x5629bf888ba0 msgr2=0x5629bf88a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.792+0000 7f2e0ec13740 1 -- 172.21.15.43:0/959048030 shutdown_connections 2024-04-21T08:48:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.792+0000 7f2e0ec13740 1 -- 172.21.15.43:0/959048030 wait complete. 2024-04-21T08:48:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.792+0000 7f2e0ec13740 20 client.20022 trim_cache size 0 max 16384 2024-04-21T08:48:24.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.805+0000 7f2e0ec13740 1 Processor -- start 2024-04-21T08:48:24.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.805+0000 7f2e0ec13740 1 -- start start 2024-04-21T08:48:24.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.805+0000 7f2e0ec13740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629bf88e4d0 0x5629bf8a3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.805+0000 7f2e0ec13740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf88ee80 0x5629bf887a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.805+0000 7f2e0ec13740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629bf897a00 0x5629bf896d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.805+0000 7f2e0ec13740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2e041b1690 con 0x5629bf88ee80 2024-04-21T08:48:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.805+0000 7f2e0ec13740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e041b31d0 con 0x5629bf88e4d0 2024-04-21T08:48:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.805+0000 7f2e0ec13740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2e041b1d60 con 0x5629bf897a00 2024-04-21T08:48:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.806+0000 7f2e095c0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf88ee80 0x5629bf887a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.806+0000 7f2e095c0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf88ee80 0x5629bf887a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36808/0 (socket says 172.21.15.43:36808) 2024-04-21T08:48:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.806+0000 7f2e095c0640 1 -- 172.21.15.43:0/2523357624 learned_addr learned my addr 172.21.15.43:0/2523357624 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.806+0000 7f2e09dc1640 1 --2- 172.21.15.43:0/2523357624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629bf88e4d0 0x5629bf8a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.806+0000 7f2e095c0640 1 -- 172.21.15.43:0/2523357624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629bf897a00 msgr2=0x5629bf896d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.806+0000 7f2e095c0640 1 --2- 172.21.15.43:0/2523357624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629bf897a00 0x5629bf896d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.806+0000 7f2e095c0640 1 -- 172.21.15.43:0/2523357624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629bf88e4d0 msgr2=0x5629bf8a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.806+0000 7f2e095c0640 1 --2- 172.21.15.43:0/2523357624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629bf88e4d0 0x5629bf8a3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.806+0000 7f2e095c0640 1 -- 172.21.15.43:0/2523357624 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629bf8889d0 con 0x5629bf88ee80 2024-04-21T08:48:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.806+0000 7f2e09dc1640 1 --2- 172.21.15.43:0/2523357624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629bf88e4d0 0x5629bf8a3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:24.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.806+0000 7f2e095c0640 1 --2- 172.21.15.43:0/2523357624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf88ee80 0x5629bf887a80 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f2df80be950 tx=0x7f2df8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3ac7917b1e7c6d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.806+0000 7f2deb7fe640 1 -- 172.21.15.43:0/2523357624 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df80be320 con 0x5629bf88ee80 2024-04-21T08:48:24.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.806+0000 7f2deb7fe640 1 -- 172.21.15.43:0/2523357624 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2df8003e40 con 0x5629bf88ee80 2024-04-21T08:48:24.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.806+0000 7f2deb7fe640 1 -- 172.21.15.43:0/2523357624 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df80c71a0 con 0x5629bf88ee80 2024-04-21T08:48:24.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.807+0000 7f2e0ec13740 1 -- 172.21.15.43:0/2523357624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf88ee80 msgr2=0x5629bf887a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.807+0000 7f2e0ec13740 1 --2- 172.21.15.43:0/2523357624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf88ee80 0x5629bf887a80 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f2df80be950 tx=0x7f2df8005b50 comp rx=0 tx=0).stop 2024-04-21T08:48:24.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.807+0000 7f2e0ec13740 1 -- 172.21.15.43:0/2523357624 shutdown_connections 2024-04-21T08:48:24.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.807+0000 7f2e0ec13740 1 --2- 172.21.15.43:0/2523357624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629bf897a00 0x5629bf896d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.807+0000 7f2e0ec13740 1 --2- 172.21.15.43:0/2523357624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf88ee80 0x5629bf887a80 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.807+0000 7f2e0ec13740 1 --2- 172.21.15.43:0/2523357624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629bf88e4d0 0x5629bf8a3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.807+0000 7f2e0ec13740 1 -- 172.21.15.43:0/2523357624 >> 172.21.15.43:0/2523357624 conn(0x5629bf888ba0 msgr2=0x5629bf887370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.807+0000 7f2e0ec13740 1 -- 172.21.15.43:0/2523357624 shutdown_connections 2024-04-21T08:48:24.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.807+0000 7f2e0ec13740 1 -- 172.21.15.43:0/2523357624 wait complete. 2024-04-21T08:48:24.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.807+0000 7f2e0ec13740 1 -- start start 2024-04-21T08:48:24.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.807+0000 7f2e0ec13740 1 Processor -- start 2024-04-21T08:48:24.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.807+0000 7f2e0ec13740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629bf88e4d0 0x5629bf8a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.807+0000 7f2e0ec13740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf88ee80 0x5629bf890ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.807+0000 7f2e0ec13740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629bf897a00 0x5629bf891420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.807+0000 7f2e0ec13740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2e041b2b00 con 0x5629bf88ee80 2024-04-21T08:48:24.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.807+0000 7f2e0ec13740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e041b2430 con 0x5629bf897a00 2024-04-21T08:48:24.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.807+0000 7f2e0ec13740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2e041af690 con 0x5629bf88e4d0 2024-04-21T08:48:24.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.807+0000 7f2e0ec13740 10 client.0 _ll_register_callbacks cb 0x7f2df80be860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:24.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.808+0000 7f2e095c0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf88ee80 0x5629bf890ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.808+0000 7f2e095c0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf88ee80 0x5629bf890ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36812/0 (socket says 172.21.15.43:36812) 2024-04-21T08:48:24.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.808+0000 7f2e095c0640 1 -- 172.21.15.43:0/4261902205 learned_addr learned my addr 172.21.15.43:0/4261902205 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:24.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.808+0000 7f2e09dc1640 1 --2- 172.21.15.43:0/4261902205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629bf88e4d0 0x5629bf8a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.808+0000 7f2e0a5c2640 1 --2- 172.21.15.43:0/4261902205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629bf897a00 0x5629bf891420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.808+0000 7f2e095c0640 1 -- 172.21.15.43:0/4261902205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629bf88e4d0 msgr2=0x5629bf8a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.808+0000 7f2e095c0640 1 --2- 172.21.15.43:0/4261902205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629bf88e4d0 0x5629bf8a00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.808+0000 7f2e095c0640 1 -- 172.21.15.43:0/4261902205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629bf897a00 msgr2=0x5629bf891420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.808+0000 7f2e095c0640 1 --2- 172.21.15.43:0/4261902205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629bf897a00 0x5629bf891420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.808+0000 7f2e095c0640 1 -- 172.21.15.43:0/4261902205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629bf891c50 con 0x5629bf88ee80 2024-04-21T08:48:24.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.808+0000 7f2e0a5c2640 1 --2- 172.21.15.43:0/4261902205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629bf897a00 0x5629bf891420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:24.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.808+0000 7f2e095c0640 1 --2- 172.21.15.43:0/4261902205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf88ee80 0x5629bf890ee0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f2df80bb260 tx=0x7f2df80c7770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7710f7459971ccf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.808+0000 7f2de97fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:24.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.808+0000 7f2de97fa640 1 -- 172.21.15.43:0/4261902205 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df80a1020 con 0x5629bf88ee80 2024-04-21T08:48:24.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.808+0000 7f2de97fa640 1 -- 172.21.15.43:0/4261902205 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f2df80f0220 con 0x5629bf88ee80 2024-04-21T08:48:24.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.808+0000 7f2e0ec13740 1 -- 172.21.15.43:0/4261902205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5629bf894ec0 con 0x5629bf88ee80 2024-04-21T08:48:24.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.808+0000 7f2de97fa640 1 -- 172.21.15.43:0/4261902205 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2df80be9c0 con 0x5629bf88ee80 2024-04-21T08:48:24.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.808+0000 7f2e0ec13740 10 client.20028 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:24.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.808+0000 7f2e0ec13740 1 -- 172.21.15.43:0/4261902205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5629bf891c50 con 0x5629bf88ee80 2024-04-21T08:48:24.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.808+0000 7f2de97fa640 1 -- 172.21.15.43:0/4261902205 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5629bf894ec0 con 0x5629bf88ee80 2024-04-21T08:48:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.809+0000 7f2de97fa640 1 -- 172.21.15.43:0/4261902205 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2df80ddb80 con 0x5629bf88ee80 2024-04-21T08:48:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.809+0000 7f2e0ec13740 10 client.20028 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.809+0000 7f2e0ec13740 10 client.20028 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:48:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.809+0000 7f2e0ec13740 20 client.20028 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.809+0000 7f2e0ec13740 1 --2- 172.21.15.43:0/4261902205 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5629bf90e2a0 0x5629bf910690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.809+0000 7f2e0ec13740 4 client.20028 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.809+0000 7f2e0ec13740 1 -- 172.21.15.43:0/4261902205 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629bf800c10 con 0x5629bf90e2a0 2024-04-21T08:48:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.809+0000 7f2e09dc1640 1 --2- 172.21.15.43:0/4261902205 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5629bf90e2a0 0x5629bf910690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.809+0000 7f2e09dc1640 1 --2- 172.21.15.43:0/4261902205 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5629bf90e2a0 0x5629bf910690 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e8369b41113ee41e server_cookie=4310ce9c7c971a21 in_seq=0 out_seq=0 2024-04-21T08:48:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.809+0000 7f2de97fa640 10 client.20028 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:48:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.809+0000 7f2de97fa640 1 -- 172.21.15.43:0/4261902205 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629bf800c10 con 0x5629bf90e2a0 2024-04-21T08:48:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.809+0000 7f2de97fa640 10 client.20028 handle_command_reply: tid=0 2024-04-21T08:48:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.809+0000 7f2e0ec13740 1 client.20028 shutdown 2024-04-21T08:48:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.810+0000 7f2e0ec13740 1 -- 172.21.15.43:0/4261902205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf88ee80 msgr2=0x5629bf890ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.810+0000 7f2e0ec13740 1 --2- 172.21.15.43:0/4261902205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf88ee80 0x5629bf890ee0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f2df80bb260 tx=0x7f2df80c7770 comp rx=0 tx=0).stop 2024-04-21T08:48:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.810+0000 7f2e0ec13740 1 -- 172.21.15.43:0/4261902205 shutdown_connections 2024-04-21T08:48:24.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.810+0000 7f2e0ec13740 1 --2- 172.21.15.43:0/4261902205 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x5629bf90e2a0 0x5629bf910690 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.810+0000 7f2e0ec13740 1 --2- 172.21.15.43:0/4261902205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629bf897a00 0x5629bf891420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.810+0000 7f2de97fa640 0 client.20028 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:48:24.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.810+0000 7f2e0ec13740 1 --2- 172.21.15.43:0/4261902205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629bf88ee80 0x5629bf890ee0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.810+0000 7f2e0ec13740 1 --2- 172.21.15.43:0/4261902205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629bf88e4d0 0x5629bf8a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.810+0000 7f2e0ec13740 1 -- 172.21.15.43:0/4261902205 >> 172.21.15.43:0/4261902205 conn(0x5629bf888ba0 msgr2=0x5629bf887c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.810+0000 7f2e0ec13740 1 -- 172.21.15.43:0/4261902205 shutdown_connections 2024-04-21T08:48:24.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.810+0000 7f2e0ec13740 1 -- 172.21.15.43:0/4261902205 wait complete. 2024-04-21T08:48:24.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.810+0000 7f2e0ec13740 20 client.20028 trim_cache size 0 max 16384 2024-04-21T08:48:24.816 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:48:24.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.810+0000 7f2e0c025640 1 -- 172.21.15.43:0/1437212901 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2dec0a3d20 msgr2=0x7f2dec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.810+0000 7f2e0c025640 1 --2- 172.21.15.43:0/1437212901 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2dec0a3d20 0x7f2dec0a61e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2df4005fd0 tx=0x7f2df4004ab0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.810+0000 7f2e0c025640 1 -- 172.21.15.43:0/1437212901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e04107270 msgr2=0x7f2e041a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.810+0000 7f2e0c025640 1 --2- 172.21.15.43:0/1437212901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e04107270 0x7f2e041a0b70 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f2df8000c00 tx=0x7f2df802c860 comp rx=0 tx=0).stop 2024-04-21T08:48:24.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.810+0000 7f2e0c025640 1 -- 172.21.15.43:0/1437212901 shutdown_connections 2024-04-21T08:48:24.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.810+0000 7f2e0c025640 1 --2- 172.21.15.43:0/1437212901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e0419a740 0x7f2e0419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.810+0000 7f2e0c025640 1 --2- 172.21.15.43:0/1437212901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e04107270 0x7f2e041a0b70 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.810+0000 7f2e0c025640 1 --2- 172.21.15.43:0/1437212901 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2dec0a3d20 0x7f2dec0a61e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.810+0000 7f2e0c025640 1 --2- 172.21.15.43:0/1437212901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e0406a840 0x7f2e041a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.810+0000 7f2e0c025640 1 -- 172.21.15.43:0/1437212901 >> 172.21.15.43:0/1437212901 conn(0x7f2e04077eb0 msgr2=0x7f2e0410d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.811+0000 7f2e0c025640 1 -- 172.21.15.43:0/1437212901 shutdown_connections 2024-04-21T08:48:24.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.811+0000 7f2e0c025640 1 -- 172.21.15.43:0/1437212901 wait complete. 2024-04-21T08:48:24.821 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:48:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.927+0000 7ff9d7e62640 1 -- 172.21.15.43:0/3632807206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9d0112920 msgr2=0x7ff9d0112d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.927+0000 7ff9d7e62640 1 --2- 172.21.15.43:0/3632807206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9d0112920 0x7ff9d0112d00 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009930 tx=0x7ff9cc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.927+0000 7ff9d7e62640 1 -- 172.21.15.43:0/3632807206 shutdown_connections 2024-04-21T08:48:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.927+0000 7ff9d7e62640 1 --2- 172.21.15.43:0/3632807206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9d010d2d0 0x7ff9d01197e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.927+0000 7ff9d7e62640 1 --2- 172.21.15.43:0/3632807206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9d010c910 0x7ff9d010cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.927+0000 7ff9d7e62640 1 --2- 172.21.15.43:0/3632807206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9d0112920 0x7ff9d0112d00 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.927+0000 7ff9d7e62640 1 -- 172.21.15.43:0/3632807206 >> 172.21.15.43:0/3632807206 conn(0x7ff9d006cf70 msgr2=0x7ff9d010af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.927+0000 7ff9d7e62640 1 -- 172.21.15.43:0/3632807206 shutdown_connections 2024-04-21T08:48:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.927+0000 7ff9d7e62640 1 -- 172.21.15.43:0/3632807206 wait complete. 2024-04-21T08:48:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.927+0000 7ff9d7e62640 1 Processor -- start 2024-04-21T08:48:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.927+0000 7ff9d7e62640 1 -- start start 2024-04-21T08:48:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.928+0000 7ff9d7e62640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9d010c910 0x7ff9d01a3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.928+0000 7ff9d7e62640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9d010d2d0 0x7ff9d01a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.928+0000 7ff9d7e62640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9d0112920 0x7ff9d01a7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.928+0000 7ff9d7e62640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d011cbe0 con 0x7ff9d0112920 2024-04-21T08:48:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.928+0000 7ff9d7e62640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d011ca60 con 0x7ff9d010d2d0 2024-04-21T08:48:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.928+0000 7ff9d7e62640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d011cd60 con 0x7ff9d010c910 2024-04-21T08:48:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.928+0000 7ff9d63ff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9d0112920 0x7ff9d01a7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.928+0000 7ff9d63ff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9d0112920 0x7ff9d01a7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36830/0 (socket says 172.21.15.43:36830) 2024-04-21T08:48:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.928+0000 7ff9d63ff640 1 -- 172.21.15.43:0/156403791 learned_addr learned my addr 172.21.15.43:0/156403791 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.928+0000 7ff9d53fd640 1 --2- 172.21.15.43:0/156403791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9d010d2d0 0x7ff9d01a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.928+0000 7ff9d5bfe640 1 --2- 172.21.15.43:0/156403791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9d010c910 0x7ff9d01a3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.928+0000 7ff9d63ff640 1 -- 172.21.15.43:0/156403791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9d010c910 msgr2=0x7ff9d01a3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.928+0000 7ff9d63ff640 1 --2- 172.21.15.43:0/156403791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9d010c910 0x7ff9d01a3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.928+0000 7ff9d63ff640 1 -- 172.21.15.43:0/156403791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9d010d2d0 msgr2=0x7ff9d01a3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.928+0000 7ff9d63ff640 1 --2- 172.21.15.43:0/156403791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9d010d2d0 0x7ff9d01a3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.928+0000 7ff9d63ff640 1 -- 172.21.15.43:0/156403791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d01a8130 con 0x7ff9d0112920 2024-04-21T08:48:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.928+0000 7ff9d63ff640 1 --2- 172.21.15.43:0/156403791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9d0112920 0x7ff9d01a7a00 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800eaa0 tx=0x7ff9c800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.928+0000 7ff9c6ffd640 1 -- 172.21.15.43:0/156403791 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c800cdb0 con 0x7ff9d0112920 2024-04-21T08:48:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.928+0000 7ff9c6ffd640 1 -- 172.21.15.43:0/156403791 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9c8005020 con 0x7ff9d0112920 2024-04-21T08:48:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.928+0000 7ff9d7e62640 1 -- 172.21.15.43:0/156403791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d01a8420 con 0x7ff9d0112920 2024-04-21T08:48:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.928+0000 7ff9c6ffd640 1 -- 172.21.15.43:0/156403791 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c80116d0 con 0x7ff9d0112920 2024-04-21T08:48:24.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.928+0000 7ff9d7e62640 1 -- 172.21.15.43:0/156403791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d01afcb0 con 0x7ff9d0112920 2024-04-21T08:48:24.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.931+0000 7ff9c6ffd640 1 -- 172.21.15.43:0/156403791 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff9c80051d0 con 0x7ff9d0112920 2024-04-21T08:48:24.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.931+0000 7ff9daa4c740 1 Processor -- start 2024-04-21T08:48:24.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.931+0000 7ff9daa4c740 1 -- start start 2024-04-21T08:48:24.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.932+0000 7ff9daa4c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 0x5647e21838b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.932+0000 7ff9daa4c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647e2183e80 0x5647e218c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.932+0000 7ff9daa4c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647e218ca00 0x5647e218ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.932+0000 7ff9daa4c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5647e20e81f0 con 0x5647e21834d0 2024-04-21T08:48:24.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.932+0000 7ff9c6ffd640 1 --2- 172.21.15.43:0/156403791 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff9bc0a3d20 0x7ff9bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.932+0000 7ff9daa4c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5647e20854b0 con 0x5647e2183e80 2024-04-21T08:48:24.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.932+0000 7ff9daa4c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5647e20e41b0 con 0x5647e218ca00 2024-04-21T08:48:24.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.932+0000 7ff9d53fd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 0x5647e21838b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.932+0000 7ff9d63ff640 1 --2- 172.21.15.43:0/156403791 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff9bc0a3d20 0x7ff9bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.932+0000 7ff9d53fd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 0x5647e21838b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36834/0 (socket says 172.21.15.43:36834) 2024-04-21T08:48:24.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.932+0000 7ff9d53fd640 1 -- 172.21.15.43:0/4122245627 learned_addr learned my addr 172.21.15.43:0/4122245627 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:24.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.932+0000 7ff9c6ffd640 1 -- 172.21.15.43:0/156403791 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7ff9c80d0d90 con 0x7ff9d0112920 2024-04-21T08:48:24.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.932+0000 7ff9d5bfe640 1 --2- 172.21.15.43:0/4122245627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647e2183e80 0x5647e218c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.932+0000 7ff9d53fd640 1 --2- 172.21.15.43:0/4122245627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647e218ca00 0x5647e218ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.932+0000 7ff9d53fd640 1 -- 172.21.15.43:0/4122245627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647e218ca00 msgr2=0x5647e218ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.932+0000 7ff9d53fd640 1 --2- 172.21.15.43:0/4122245627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647e218ca00 0x5647e218ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.932+0000 7ff9d53fd640 1 -- 172.21.15.43:0/4122245627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647e2183e80 msgr2=0x5647e218c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.932+0000 7ff9d53fd640 1 --2- 172.21.15.43:0/4122245627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647e2183e80 0x5647e218c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.932+0000 7ff9d53fd640 1 -- 172.21.15.43:0/4122245627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647e20771b0 con 0x5647e21834d0 2024-04-21T08:48:24.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.932+0000 7ff9d63ff640 1 --2- 172.21.15.43:0/156403791 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff9bc0a3d20 0x7ff9bc0a61e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff9c80ce170 tx=0x7ff9c80bd040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.932+0000 7ff9d53fd640 1 --2- 172.21.15.43:0/4122245627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 0x5647e21838b0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00135b0 tx=0x7ff9c0009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f4e7d1e16cd672 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.932+0000 7ff9bbfff640 1 -- 172.21.15.43:0/4122245627 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0004250 con 0x5647e21834d0 2024-04-21T08:48:24.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.933+0000 7ff9bbfff640 1 -- 172.21.15.43:0/4122245627 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9c00043f0 con 0x5647e21834d0 2024-04-21T08:48:24.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.933+0000 7ff9bbfff640 1 -- 172.21.15.43:0/4122245627 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c00119c0 con 0x5647e21834d0 2024-04-21T08:48:24.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.933+0000 7ff9daa4c740 1 -- 172.21.15.43:0/4122245627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 msgr2=0x5647e21838b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.933+0000 7ff9daa4c740 1 --2- 172.21.15.43:0/4122245627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 0x5647e21838b0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00135b0 tx=0x7ff9c0009850 comp rx=0 tx=0).stop 2024-04-21T08:48:24.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.933+0000 7ff9daa4c740 1 -- 172.21.15.43:0/4122245627 shutdown_connections 2024-04-21T08:48:24.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.933+0000 7ff9daa4c740 1 --2- 172.21.15.43:0/4122245627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647e218ca00 0x5647e218ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.933+0000 7ff9daa4c740 1 --2- 172.21.15.43:0/4122245627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647e2183e80 0x5647e218c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.933+0000 7ff9daa4c740 1 --2- 172.21.15.43:0/4122245627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 0x5647e21838b0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.933+0000 7ff9daa4c740 1 -- 172.21.15.43:0/4122245627 >> 172.21.15.43:0/4122245627 conn(0x5647e217dba0 msgr2=0x5647e217df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.933+0000 7ff9daa4c740 1 -- 172.21.15.43:0/4122245627 shutdown_connections 2024-04-21T08:48:24.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.933+0000 7ff9daa4c740 1 -- 172.21.15.43:0/4122245627 wait complete. 2024-04-21T08:48:24.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.934+0000 7ff9daa4c740 1 -- start start 2024-04-21T08:48:24.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.934+0000 7ff9daa4c740 1 Processor -- start 2024-04-21T08:48:24.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.934+0000 7ff9daa4c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 0x5647e218c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.934+0000 7ff9daa4c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647e2183e80 0x5647e21895f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.934+0000 7ff9daa4c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647e218ca00 0x5647e2189b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.934+0000 7ff9daa4c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5647e218f970 con 0x5647e21834d0 2024-04-21T08:48:24.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.934+0000 7ff9daa4c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5647e20a1500 con 0x5647e218ca00 2024-04-21T08:48:24.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.934+0000 7ff9daa4c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5647e218faf0 con 0x5647e2183e80 2024-04-21T08:48:24.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.934+0000 7ff9daa4c740 10 client.0 _ll_register_callbacks cb 0x5647e206fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:24.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.934+0000 7ff9d53fd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 0x5647e218c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.934+0000 7ff9d53fd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 0x5647e218c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36838/0 (socket says 172.21.15.43:36838) 2024-04-21T08:48:24.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.934+0000 7ff9d53fd640 1 -- 172.21.15.43:0/1862221604 learned_addr learned my addr 172.21.15.43:0/1862221604 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:24.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.935+0000 7ff9d53fd640 1 --2- 172.21.15.43:0/1862221604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647e218ca00 0x5647e2189b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.935+0000 7ff9d5bfe640 1 --2- 172.21.15.43:0/1862221604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647e2183e80 0x5647e21895f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.935+0000 7ff9d53fd640 1 -- 172.21.15.43:0/1862221604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647e2183e80 msgr2=0x5647e21895f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.935+0000 7ff9d53fd640 1 --2- 172.21.15.43:0/1862221604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647e2183e80 0x5647e21895f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.935+0000 7ff9d53fd640 1 -- 172.21.15.43:0/1862221604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647e218ca00 msgr2=0x5647e2189b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.935+0000 7ff9d53fd640 1 --2- 172.21.15.43:0/1862221604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647e218ca00 0x5647e2189b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.935+0000 7ff9d53fd640 1 -- 172.21.15.43:0/1862221604 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647e1dfa3f0 con 0x5647e21834d0 2024-04-21T08:48:24.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.935+0000 7ff9d5bfe640 1 --2- 172.21.15.43:0/1862221604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647e2183e80 0x5647e21895f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:24.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.935+0000 7ff9d53fd640 1 --2- 172.21.15.43:0/1862221604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 0x5647e218c260 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00116a0 tx=0x7ff9c0005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bdc45fd857db47a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.935+0000 7ff9b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:24.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.935+0000 7ff9b97fa640 1 -- 172.21.15.43:0/1862221604 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c00039f0 con 0x5647e21834d0 2024-04-21T08:48:24.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.935+0000 7ff9b97fa640 1 -- 172.21.15.43:0/1862221604 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7ff9c0026070 con 0x5647e21834d0 2024-04-21T08:48:24.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.935+0000 7ff9daa4c740 1 -- 172.21.15.43:0/1862221604 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9c00043f0 con 0x5647e21834d0 2024-04-21T08:48:24.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.935+0000 7ff9b97fa640 1 -- 172.21.15.43:0/1862221604 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff9c0003b90 con 0x5647e21834d0 2024-04-21T08:48:24.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.935+0000 7ff9daa4c740 10 client.20040 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:24.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.935+0000 7ff9daa4c740 1 -- 172.21.15.43:0/1862221604 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5647e1ec47d0 con 0x5647e21834d0 2024-04-21T08:48:24.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.935+0000 7ff9b97fa640 1 -- 172.21.15.43:0/1862221604 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9c00043f0 con 0x5647e21834d0 2024-04-21T08:48:24.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.936+0000 7ff9b97fa640 1 -- 172.21.15.43:0/1862221604 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff9c0029030 con 0x5647e21834d0 2024-04-21T08:48:24.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.936+0000 7ff9daa4c740 10 client.20040 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:24.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.936+0000 7ff9daa4c740 10 client.20040 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:48:24.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.936+0000 7ff9daa4c740 20 client.20040 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:24.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.936+0000 7ff9daa4c740 1 --2- 172.21.15.43:0/1862221604 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5647e219ca00 0x5647e219edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.936+0000 7ff9daa4c740 4 client.20040 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:24.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.936+0000 7ff9daa4c740 1 -- 172.21.15.43:0/1862221604 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647e2087930 con 0x5647e219ca00 2024-04-21T08:48:24.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.936+0000 7ff9d5bfe640 1 --2- 172.21.15.43:0/1862221604 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5647e219ca00 0x5647e219edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.936+0000 7ff9b97fa640 10 client.20040 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:48:24.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.936+0000 7ff9d5bfe640 1 --2- 172.21.15.43:0/1862221604 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5647e219ca00 0x5647e219edf0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=af5022292ea79bde server_cookie=ebe3b8b5b75c0910 in_seq=0 out_seq=0 2024-04-21T08:48:24.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.937+0000 7ff9b97fa640 1 -- 172.21.15.43:0/1862221604 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5647e2087930 con 0x5647e219ca00 2024-04-21T08:48:24.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.937+0000 7ff9b97fa640 10 client.20040 handle_command_reply: tid=0 2024-04-21T08:48:24.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.937+0000 7ff9daa4c740 1 client.20040 shutdown 2024-04-21T08:48:24.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.937+0000 7ff9daa4c740 1 -- 172.21.15.43:0/1862221604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 msgr2=0x5647e218c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.937+0000 7ff9daa4c740 1 --2- 172.21.15.43:0/1862221604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 0x5647e218c260 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00116a0 tx=0x7ff9c0005b90 comp rx=0 tx=0).stop 2024-04-21T08:48:24.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.937+0000 7ff9daa4c740 1 -- 172.21.15.43:0/1862221604 shutdown_connections 2024-04-21T08:48:24.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.937+0000 7ff9daa4c740 1 --2- 172.21.15.43:0/1862221604 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5647e219ca00 0x5647e219edf0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.937+0000 7ff9daa4c740 1 --2- 172.21.15.43:0/1862221604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647e218ca00 0x5647e2189b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.937+0000 7ff9daa4c740 1 --2- 172.21.15.43:0/1862221604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647e2183e80 0x5647e21895f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.937+0000 7ff9b97fa640 0 client.20040 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:48:24.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.937+0000 7ff9daa4c740 1 --2- 172.21.15.43:0/1862221604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 0x5647e218c260 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.937+0000 7ff9daa4c740 1 -- 172.21.15.43:0/1862221604 >> 172.21.15.43:0/1862221604 conn(0x5647e217dba0 msgr2=0x5647e217f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.937+0000 7ff9daa4c740 1 -- 172.21.15.43:0/1862221604 shutdown_connections 2024-04-21T08:48:24.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.937+0000 7ff9daa4c740 1 -- 172.21.15.43:0/1862221604 wait complete. 2024-04-21T08:48:24.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.938+0000 7ff9daa4c740 20 client.20040 trim_cache size 0 max 16384 2024-04-21T08:48:24.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.951+0000 7ff9daa4c740 1 Processor -- start 2024-04-21T08:48:24.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.951+0000 7ff9daa4c740 1 -- start start 2024-04-21T08:48:24.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.951+0000 7ff9daa4c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 0x5647e2198830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.951+0000 7ff9daa4c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647e2183e80 0x5647e2184260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.951+0000 7ff9daa4c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647e218ca00 0x5647e21a4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.951+0000 7ff9daa4c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d01ba050 con 0x5647e21834d0 2024-04-21T08:48:24.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.951+0000 7ff9daa4c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d01bbb90 con 0x5647e2183e80 2024-04-21T08:48:24.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.951+0000 7ff9daa4c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d01ba720 con 0x5647e218ca00 2024-04-21T08:48:24.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.951+0000 7ff9d53fd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 0x5647e2198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.951+0000 7ff9d53fd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 0x5647e2198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36854/0 (socket says 172.21.15.43:36854) 2024-04-21T08:48:24.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.951+0000 7ff9d53fd640 1 -- 172.21.15.43:0/1717416379 learned_addr learned my addr 172.21.15.43:0/1717416379 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:24.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.951+0000 7ff9d5bfe640 1 --2- 172.21.15.43:0/1717416379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647e2183e80 0x5647e2184260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.951+0000 7ff9d53fd640 1 -- 172.21.15.43:0/1717416379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647e218ca00 msgr2=0x5647e21a4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.951+0000 7ff9d53fd640 1 --2- 172.21.15.43:0/1717416379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647e218ca00 0x5647e21a4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.951+0000 7ff9d53fd640 1 -- 172.21.15.43:0/1717416379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647e2183e80 msgr2=0x5647e2184260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.951+0000 7ff9d53fd640 1 --2- 172.21.15.43:0/1717416379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647e2183e80 0x5647e2184260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.951+0000 7ff9d53fd640 1 -- 172.21.15.43:0/1717416379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647e218f850 con 0x5647e21834d0 2024-04-21T08:48:24.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.952+0000 7ff9d53fd640 1 --2- 172.21.15.43:0/1717416379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 0x5647e2198830 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0003f80 tx=0x7ff9c00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=274f00caca3e85a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.952+0000 7ff9bb7fe640 1 -- 172.21.15.43:0/1717416379 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0009c50 con 0x5647e21834d0 2024-04-21T08:48:24.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.952+0000 7ff9bb7fe640 1 -- 172.21.15.43:0/1717416379 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9c0003b90 con 0x5647e21834d0 2024-04-21T08:48:24.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.952+0000 7ff9bb7fe640 1 -- 172.21.15.43:0/1717416379 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0011020 con 0x5647e21834d0 2024-04-21T08:48:24.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.952+0000 7ff9daa4c740 1 -- 172.21.15.43:0/1717416379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 msgr2=0x5647e2198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.952+0000 7ff9daa4c740 1 --2- 172.21.15.43:0/1717416379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 0x5647e2198830 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0003f80 tx=0x7ff9c00115a0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.952+0000 7ff9daa4c740 1 -- 172.21.15.43:0/1717416379 shutdown_connections 2024-04-21T08:48:24.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.952+0000 7ff9daa4c740 1 --2- 172.21.15.43:0/1717416379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647e218ca00 0x5647e21a4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.952+0000 7ff9daa4c740 1 --2- 172.21.15.43:0/1717416379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647e2183e80 0x5647e2184260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.952+0000 7ff9daa4c740 1 --2- 172.21.15.43:0/1717416379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 0x5647e2198830 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.952+0000 7ff9daa4c740 1 -- 172.21.15.43:0/1717416379 >> 172.21.15.43:0/1717416379 conn(0x5647e217dba0 msgr2=0x5647e217f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.952+0000 7ff9daa4c740 1 -- 172.21.15.43:0/1717416379 shutdown_connections 2024-04-21T08:48:24.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.952+0000 7ff9daa4c740 1 -- 172.21.15.43:0/1717416379 wait complete. 2024-04-21T08:48:24.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.953+0000 7ff9daa4c740 1 -- start start 2024-04-21T08:48:24.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.953+0000 7ff9daa4c740 1 Processor -- start 2024-04-21T08:48:24.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.953+0000 7ff9daa4c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 0x5647e2194dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.953+0000 7ff9daa4c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647e2183e80 0x5647e2192150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.953+0000 7ff9daa4c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647e218ca00 0x5647e2192690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.953+0000 7ff9daa4c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d01bb4c0 con 0x5647e21834d0 2024-04-21T08:48:24.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.953+0000 7ff9daa4c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d01badf0 con 0x5647e218ca00 2024-04-21T08:48:24.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.953+0000 7ff9daa4c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d01b8050 con 0x5647e2183e80 2024-04-21T08:48:24.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.953+0000 7ff9daa4c740 10 client.0 _ll_register_callbacks cb 0x7ff9c0003680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:24.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.953+0000 7ff9d53fd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 0x5647e2194dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.953+0000 7ff9d53fd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 0x5647e2194dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36860/0 (socket says 172.21.15.43:36860) 2024-04-21T08:48:24.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.953+0000 7ff9d53fd640 1 -- 172.21.15.43:0/415481732 learned_addr learned my addr 172.21.15.43:0/415481732 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.953+0000 7ff9d53fd640 1 --2- 172.21.15.43:0/415481732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647e218ca00 0x5647e2192690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.953+0000 7ff9d5bfe640 1 --2- 172.21.15.43:0/415481732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647e2183e80 0x5647e2192150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.953+0000 7ff9d53fd640 1 -- 172.21.15.43:0/415481732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647e2183e80 msgr2=0x5647e2192150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.954+0000 7ff9d53fd640 1 --2- 172.21.15.43:0/415481732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647e2183e80 0x5647e2192150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.954+0000 7ff9d53fd640 1 -- 172.21.15.43:0/415481732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647e218ca00 msgr2=0x5647e2192690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.954+0000 7ff9d53fd640 1 --2- 172.21.15.43:0/415481732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647e218ca00 0x5647e2192690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.954+0000 7ff9d53fd640 1 -- 172.21.15.43:0/415481732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647e2192bd0 con 0x5647e21834d0 2024-04-21T08:48:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.954+0000 7ff9d53fd640 1 --2- 172.21.15.43:0/415481732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 0x5647e2194dc0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0005c00 tx=0x7ff9c0005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3522dc5932d6382d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.954+0000 7ff9b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.954+0000 7ff9b97fa640 1 -- 172.21.15.43:0/415481732 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0011040 con 0x5647e21834d0 2024-04-21T08:48:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.954+0000 7ff9b97fa640 1 -- 172.21.15.43:0/415481732 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7ff9c0023070 con 0x5647e21834d0 2024-04-21T08:48:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.954+0000 7ff9daa4c740 1 -- 172.21.15.43:0/415481732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9c0003b90 con 0x5647e21834d0 2024-04-21T08:48:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.954+0000 7ff9b97fa640 1 -- 172.21.15.43:0/415481732 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff9c00111e0 con 0x5647e21834d0 2024-04-21T08:48:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.954+0000 7ff9daa4c740 10 client.20046 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.954+0000 7ff9daa4c740 1 -- 172.21.15.43:0/415481732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5647e22029c0 con 0x5647e21834d0 2024-04-21T08:48:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.954+0000 7ff9b97fa640 1 -- 172.21.15.43:0/415481732 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9c0003b90 con 0x5647e21834d0 2024-04-21T08:48:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.954+0000 7ff9b97fa640 1 -- 172.21.15.43:0/415481732 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff9c0039070 con 0x5647e21834d0 2024-04-21T08:48:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.954+0000 7ff9daa4c740 10 client.20046 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.954+0000 7ff9daa4c740 10 client.20046 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:48:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.954+0000 7ff9daa4c740 20 client.20046 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.955+0000 7ff9daa4c740 1 --2- 172.21.15.43:0/415481732 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5647e22030b0 0x5647e22054d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.955+0000 7ff9daa4c740 4 client.20046 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.955+0000 7ff9daa4c740 1 -- 172.21.15.43:0/415481732 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5647e20f5c10 con 0x5647e22030b0 2024-04-21T08:48:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.955+0000 7ff9d5bfe640 1 --2- 172.21.15.43:0/415481732 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5647e22030b0 0x5647e22054d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.955+0000 7ff9d5bfe640 1 --2- 172.21.15.43:0/415481732 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5647e22030b0 0x5647e22054d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c485df0f4fda8168 server_cookie=940f8327bc9bc4fb in_seq=0 out_seq=0 2024-04-21T08:48:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.955+0000 7ff9b97fa640 10 client.20046 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:48:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.955+0000 7ff9b97fa640 1 -- 172.21.15.43:0/415481732 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5647e20f5c10 con 0x5647e22030b0 2024-04-21T08:48:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.955+0000 7ff9b97fa640 10 client.20046 handle_command_reply: tid=0 2024-04-21T08:48:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.955+0000 7ff9daa4c740 1 client.20046 shutdown 2024-04-21T08:48:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.955+0000 7ff9daa4c740 1 -- 172.21.15.43:0/415481732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 msgr2=0x5647e2194dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.955+0000 7ff9daa4c740 1 --2- 172.21.15.43:0/415481732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 0x5647e2194dc0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0005c00 tx=0x7ff9c0005b70 comp rx=0 tx=0).stop 2024-04-21T08:48:24.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.956+0000 7ff9daa4c740 1 -- 172.21.15.43:0/415481732 shutdown_connections 2024-04-21T08:48:24.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.956+0000 7ff9daa4c740 1 --2- 172.21.15.43:0/415481732 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x5647e22030b0 0x5647e22054d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.956+0000 7ff9daa4c740 1 --2- 172.21.15.43:0/415481732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647e218ca00 0x5647e2192690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.956+0000 7ff9daa4c740 1 --2- 172.21.15.43:0/415481732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647e2183e80 0x5647e2192150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.956+0000 7ff9b97fa640 0 client.20046 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:48:24.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.956+0000 7ff9daa4c740 1 --2- 172.21.15.43:0/415481732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647e21834d0 0x5647e2194dc0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.956+0000 7ff9daa4c740 1 -- 172.21.15.43:0/415481732 >> 172.21.15.43:0/415481732 conn(0x5647e217dba0 msgr2=0x5647e2197820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.956+0000 7ff9daa4c740 1 -- 172.21.15.43:0/415481732 shutdown_connections 2024-04-21T08:48:24.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.956+0000 7ff9daa4c740 1 -- 172.21.15.43:0/415481732 wait complete. 2024-04-21T08:48:24.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.956+0000 7ff9daa4c740 20 client.20046 trim_cache size 0 max 16384 2024-04-21T08:48:24.961 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:48:24.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.956+0000 7ff9d7e62640 1 -- 172.21.15.43:0/156403791 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff9bc0a3d20 msgr2=0x7ff9bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.956+0000 7ff9d7e62640 1 --2- 172.21.15.43:0/156403791 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff9bc0a3d20 0x7ff9bc0a61e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff9c80ce170 tx=0x7ff9c80bd040 comp rx=0 tx=0).stop 2024-04-21T08:48:24.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.956+0000 7ff9d7e62640 1 -- 172.21.15.43:0/156403791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9d0112920 msgr2=0x7ff9d01a7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:24.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.956+0000 7ff9d7e62640 1 --2- 172.21.15.43:0/156403791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9d0112920 0x7ff9d01a7a00 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800eaa0 tx=0x7ff9c800ef60 comp rx=0 tx=0).stop 2024-04-21T08:48:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.956+0000 7ff9d7e62640 1 -- 172.21.15.43:0/156403791 shutdown_connections 2024-04-21T08:48:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.956+0000 7ff9d7e62640 1 --2- 172.21.15.43:0/156403791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9d0112920 0x7ff9d01a7a00 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.956+0000 7ff9d7e62640 1 --2- 172.21.15.43:0/156403791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9d010d2d0 0x7ff9d01a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.956+0000 7ff9d7e62640 1 --2- 172.21.15.43:0/156403791 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff9bc0a3d20 0x7ff9bc0a61e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.956+0000 7ff9d7e62640 1 --2- 172.21.15.43:0/156403791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9d010c910 0x7ff9d01a3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.956+0000 7ff9d7e62640 1 -- 172.21.15.43:0/156403791 >> 172.21.15.43:0/156403791 conn(0x7ff9d006cf70 msgr2=0x7ff9d010f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.956+0000 7ff9d7e62640 1 -- 172.21.15.43:0/156403791 shutdown_connections 2024-04-21T08:48:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:24.956+0000 7ff9d7e62640 1 -- 172.21.15.43:0/156403791 wait complete. 2024-04-21T08:48:24.967 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:48:25.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.071+0000 7f2e9cc32640 1 -- 172.21.15.43:0/3991765950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e98103af0 msgr2=0x7f2e9810bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.071+0000 7f2e9cc32640 1 --2- 172.21.15.43:0/3991765950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e98103af0 0x7f2e9810bfd0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009900 tx=0x7f2e8c02bd60 comp rx=0 tx=0).stop 2024-04-21T08:48:25.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.071+0000 7f2e9cc32640 1 -- 172.21.15.43:0/3991765950 shutdown_connections 2024-04-21T08:48:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.071+0000 7f2e9cc32640 1 --2- 172.21.15.43:0/3991765950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e9810c510 0x7f2e9810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.071+0000 7f2e9cc32640 1 --2- 172.21.15.43:0/3991765950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e98103af0 0x7f2e9810bfd0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.071+0000 7f2e9cc32640 1 --2- 172.21.15.43:0/3991765950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e981031d0 0x7f2e981035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.071+0000 7f2e9cc32640 1 -- 172.21.15.43:0/3991765950 >> 172.21.15.43:0/3991765950 conn(0x7f2e98077f50 msgr2=0x7f2e98078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.072+0000 7f2e9cc32640 1 -- 172.21.15.43:0/3991765950 shutdown_connections 2024-04-21T08:48:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.072+0000 7f2e9cc32640 1 -- 172.21.15.43:0/3991765950 wait complete. 2024-04-21T08:48:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.072+0000 7f2e9cc32640 1 Processor -- start 2024-04-21T08:48:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.072+0000 7f2e9cc32640 1 -- start start 2024-04-21T08:48:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.072+0000 7f2e9cc32640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e981031d0 0x7f2e9819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.073+0000 7f2e9cc32640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e98103af0 0x7f2e9819eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.073+0000 7f2e9cc32640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e9810c510 0x7f2e98198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.073+0000 7f2e9cc32640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2e98112200 con 0x7f2e98103af0 2024-04-21T08:48:25.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.073+0000 7f2e9cc32640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e98112080 con 0x7f2e981031d0 2024-04-21T08:48:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.073+0000 7f2e9cc32640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2e98112380 con 0x7f2e9810c510 2024-04-21T08:48:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.073+0000 7f2e95d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e98103af0 0x7f2e9819eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.073+0000 7f2e95d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e98103af0 0x7f2e9819eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36874/0 (socket says 172.21.15.43:36874) 2024-04-21T08:48:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.073+0000 7f2e95d9b640 1 -- 172.21.15.43:0/1475810340 learned_addr learned my addr 172.21.15.43:0/1475810340 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.073+0000 7f2e9659c640 1 --2- 172.21.15.43:0/1475810340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e981031d0 0x7f2e9819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.073+0000 7f2e96d9d640 1 --2- 172.21.15.43:0/1475810340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e9810c510 0x7f2e98198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.073+0000 7f2e95d9b640 1 -- 172.21.15.43:0/1475810340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e9810c510 msgr2=0x7f2e98198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.073+0000 7f2e95d9b640 1 --2- 172.21.15.43:0/1475810340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e9810c510 0x7f2e98198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.073+0000 7f2e95d9b640 1 -- 172.21.15.43:0/1475810340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e981031d0 msgr2=0x7f2e9819e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.073+0000 7f2e95d9b640 1 --2- 172.21.15.43:0/1475810340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e981031d0 0x7f2e9819e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.073+0000 7f2e95d9b640 1 -- 172.21.15.43:0/1475810340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e98198f30 con 0x7f2e98103af0 2024-04-21T08:48:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.073+0000 7f2e9659c640 1 --2- 172.21.15.43:0/1475810340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e981031d0 0x7f2e9819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.073+0000 7f2e95d9b640 1 --2- 172.21.15.43:0/1475810340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e98103af0 0x7f2e9819eaf0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c005940 tx=0x7f2e8c02d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:25.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.073+0000 7f2e877fe640 1 -- 172.21.15.43:0/1475810340 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c043020 con 0x7f2e98103af0 2024-04-21T08:48:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.073+0000 7f2e9cc32640 1 -- 172.21.15.43:0/1475810340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e981991c0 con 0x7f2e98103af0 2024-04-21T08:48:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.073+0000 7f2e877fe640 1 -- 172.21.15.43:0/1475810340 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e8c02dd20 con 0x7f2e98103af0 2024-04-21T08:48:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.073+0000 7f2e877fe640 1 -- 172.21.15.43:0/1475810340 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c003b80 con 0x7f2e98103af0 2024-04-21T08:48:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.073+0000 7f2e9cc32640 1 -- 172.21.15.43:0/1475810340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e981a5530 con 0x7f2e98103af0 2024-04-21T08:48:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.076+0000 7f2e877fe640 1 -- 172.21.15.43:0/1475810340 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2e8c046020 con 0x7f2e98103af0 2024-04-21T08:48:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.076+0000 7f2e9f81c740 1 Processor -- start 2024-04-21T08:48:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.076+0000 7f2e9f81c740 1 -- start start 2024-04-21T08:48:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.076+0000 7f2e9f81c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1fea4d0 0x55bdb1fea8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.076+0000 7f2e9f81c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdb1feae80 0x55bdb1ff3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.076+0000 7f2e9f81c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb1ff3a00 0x55bdb1ff5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.076+0000 7f2e9f81c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bdb1f4f1f0 con 0x55bdb1fea4d0 2024-04-21T08:48:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.076+0000 7f2e9f81c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bdb1eec4b0 con 0x55bdb1feae80 2024-04-21T08:48:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.076+0000 7f2e9f81c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bdb1f4b1b0 con 0x55bdb1ff3a00 2024-04-21T08:48:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.076+0000 7f2e877fe640 1 --2- 172.21.15.43:0/1475810340 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2e800a3d20 0x7f2e800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.077+0000 7f2e96d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1fea4d0 0x55bdb1fea8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.077+0000 7f2e96d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1fea4d0 0x55bdb1fea8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36888/0 (socket says 172.21.15.43:36888) 2024-04-21T08:48:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.077+0000 7f2e96d9d640 1 -- 172.21.15.43:0/3464590986 learned_addr learned my addr 172.21.15.43:0/3464590986 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.077+0000 7f2e96d9d640 1 --2- 172.21.15.43:0/1475810340 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2e800a3d20 0x7f2e800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.077+0000 7f2e9659c640 1 --2- 172.21.15.43:0/3464590986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdb1feae80 0x55bdb1ff3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.077+0000 7f2e877fe640 1 -- 172.21.15.43:0/1475810340 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f2e8c0f2d70 con 0x7f2e98103af0 2024-04-21T08:48:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.077+0000 7f2e95d9b640 1 --2- 172.21.15.43:0/3464590986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb1ff3a00 0x55bdb1ff5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.077+0000 7f2e96d9d640 1 -- 172.21.15.43:0/3464590986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb1ff3a00 msgr2=0x55bdb1ff5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.077+0000 7f2e96d9d640 1 --2- 172.21.15.43:0/3464590986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb1ff3a00 0x55bdb1ff5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.077+0000 7f2e96d9d640 1 -- 172.21.15.43:0/3464590986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdb1feae80 msgr2=0x55bdb1ff3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.077+0000 7f2e96d9d640 1 --2- 172.21.15.43:0/3464590986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdb1feae80 0x55bdb1ff3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.077+0000 7f2e96d9d640 1 -- 172.21.15.43:0/3464590986 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdb1ede1b0 con 0x55bdb1fea4d0 2024-04-21T08:48:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.077+0000 7f2e95d9b640 1 --2- 172.21.15.43:0/3464590986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb1ff3a00 0x55bdb1ff5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:25.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.077+0000 7f2e96d9d640 1 --2- 172.21.15.43:0/3464590986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1fea4d0 0x55bdb1fea8b0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000af30 tx=0x7f2e90016700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40e6f114f0589fa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:25.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.077+0000 7f2e96d9d640 1 --2- 172.21.15.43:0/1475810340 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2e800a3d20 0x7f2e800a61e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2e90009fd0 tx=0x7f2e9001a000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:25.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.077+0000 7f2e84ff9640 1 -- 172.21.15.43:0/3464590986 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e90026070 con 0x55bdb1fea4d0 2024-04-21T08:48:25.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.077+0000 7f2e84ff9640 1 -- 172.21.15.43:0/3464590986 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e9000a590 con 0x55bdb1fea4d0 2024-04-21T08:48:25.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.077+0000 7f2e84ff9640 1 -- 172.21.15.43:0/3464590986 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e90013040 con 0x55bdb1fea4d0 2024-04-21T08:48:25.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.078+0000 7f2e9f81c740 1 -- 172.21.15.43:0/3464590986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1fea4d0 msgr2=0x55bdb1fea8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.078+0000 7f2e9f81c740 1 --2- 172.21.15.43:0/3464590986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1fea4d0 0x55bdb1fea8b0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000af30 tx=0x7f2e90016700 comp rx=0 tx=0).stop 2024-04-21T08:48:25.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.078+0000 7f2e9f81c740 1 -- 172.21.15.43:0/3464590986 shutdown_connections 2024-04-21T08:48:25.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.078+0000 7f2e9f81c740 1 --2- 172.21.15.43:0/3464590986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb1ff3a00 0x55bdb1ff5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.078+0000 7f2e9f81c740 1 --2- 172.21.15.43:0/3464590986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdb1feae80 0x55bdb1ff3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.078+0000 7f2e9f81c740 1 --2- 172.21.15.43:0/3464590986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1fea4d0 0x55bdb1fea8b0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.078+0000 7f2e9f81c740 1 -- 172.21.15.43:0/3464590986 >> 172.21.15.43:0/3464590986 conn(0x55bdb1fe4ba0 msgr2=0x55bdb1fe4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:25.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.078+0000 7f2e9f81c740 1 -- 172.21.15.43:0/3464590986 shutdown_connections 2024-04-21T08:48:25.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.078+0000 7f2e9f81c740 1 -- 172.21.15.43:0/3464590986 wait complete. 2024-04-21T08:48:25.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.079+0000 7f2e9f81c740 1 -- start start 2024-04-21T08:48:25.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.079+0000 7f2e9f81c740 1 Processor -- start 2024-04-21T08:48:25.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.079+0000 7f2e9f81c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdb1fea4d0 0x55bdb1ff3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.079+0000 7f2e9f81c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb1feae80 0x55bdb1ff0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.079+0000 7f2e9f81c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1ff3a00 0x55bdb1ff0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.079+0000 7f2e9f81c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bdb1ff6910 con 0x55bdb1ff3a00 2024-04-21T08:48:25.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.079+0000 7f2e9f81c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bdb1f08500 con 0x55bdb1fea4d0 2024-04-21T08:48:25.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.079+0000 7f2e9f81c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bdb1ff6a90 con 0x55bdb1feae80 2024-04-21T08:48:25.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.079+0000 7f2e9f81c740 10 client.0 _ll_register_callbacks cb 0x55bdb1ed6b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:25.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.079+0000 7f2e96d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1ff3a00 0x55bdb1ff0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.079+0000 7f2e96d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1ff3a00 0x55bdb1ff0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36902/0 (socket says 172.21.15.43:36902) 2024-04-21T08:48:25.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.079+0000 7f2e96d9d640 1 -- 172.21.15.43:0/345428549 learned_addr learned my addr 172.21.15.43:0/345428549 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.079+0000 7f2e9659c640 1 --2- 172.21.15.43:0/345428549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdb1fea4d0 0x55bdb1ff3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.079+0000 7f2e95d9b640 1 --2- 172.21.15.43:0/345428549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb1feae80 0x55bdb1ff0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.079+0000 7f2e96d9d640 1 -- 172.21.15.43:0/345428549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb1feae80 msgr2=0x55bdb1ff0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.079+0000 7f2e96d9d640 1 --2- 172.21.15.43:0/345428549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb1feae80 0x55bdb1ff0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.079+0000 7f2e96d9d640 1 -- 172.21.15.43:0/345428549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdb1fea4d0 msgr2=0x55bdb1ff3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.079+0000 7f2e96d9d640 1 --2- 172.21.15.43:0/345428549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdb1fea4d0 0x55bdb1ff3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.079+0000 7f2e96d9d640 1 -- 172.21.15.43:0/345428549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdb1c613f0 con 0x55bdb1ff3a00 2024-04-21T08:48:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.079+0000 7f2e95d9b640 1 --2- 172.21.15.43:0/345428549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb1feae80 0x55bdb1ff0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.079+0000 7f2e96d9d640 1 --2- 172.21.15.43:0/345428549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1ff3a00 0x55bdb1ff0c10 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000bfd0 tx=0x7f2e90009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78565157f4954f1e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.080+0000 7f2e7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.080+0000 7f2e7dffb640 1 -- 172.21.15.43:0/345428549 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e90030070 con 0x55bdb1ff3a00 2024-04-21T08:48:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.080+0000 7f2e7dffb640 1 -- 172.21.15.43:0/345428549 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f2e90021c70 con 0x55bdb1ff3a00 2024-04-21T08:48:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.080+0000 7f2e9f81c740 1 -- 172.21.15.43:0/345428549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e9000a590 con 0x55bdb1ff3a00 2024-04-21T08:48:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.080+0000 7f2e7dffb640 1 -- 172.21.15.43:0/345428549 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2e9003c050 con 0x55bdb1ff3a00 2024-04-21T08:48:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.080+0000 7f2e9f81c740 10 client.20058 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.080+0000 7f2e9f81c740 1 -- 172.21.15.43:0/345428549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bdb1d2b7d0 con 0x55bdb1ff3a00 2024-04-21T08:48:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.080+0000 7f2e7dffb640 1 -- 172.21.15.43:0/345428549 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e9000a590 con 0x55bdb1ff3a00 2024-04-21T08:48:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.080+0000 7f2e7dffb640 1 -- 172.21.15.43:0/345428549 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2e9002b460 con 0x55bdb1ff3a00 2024-04-21T08:48:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.080+0000 7f2e9f81c740 10 client.20058 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.080+0000 7f2e9f81c740 10 client.20058 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:48:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.080+0000 7f2e9f81c740 20 client.20058 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.080+0000 7f2e9f81c740 1 --2- 172.21.15.43:0/345428549 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55bdb2003ab0 0x55bdb2005ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.080+0000 7f2e9f81c740 4 client.20058 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.080+0000 7f2e9f81c740 1 -- 172.21.15.43:0/345428549 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdb1eee930 con 0x55bdb2003ab0 2024-04-21T08:48:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.081+0000 7f2e9659c640 1 --2- 172.21.15.43:0/345428549 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55bdb2003ab0 0x55bdb2005ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.081+0000 7f2e7dffb640 10 client.20058 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:48:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.081+0000 7f2e9659c640 1 --2- 172.21.15.43:0/345428549 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55bdb2003ab0 0x55bdb2005ea0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=727a1ade930d5d7c server_cookie=40306e70d684c675 in_seq=0 out_seq=0 2024-04-21T08:48:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.082+0000 7f2e7dffb640 1 -- 172.21.15.43:0/345428549 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bdb1eee930 con 0x55bdb2003ab0 2024-04-21T08:48:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.082+0000 7f2e7dffb640 10 client.20058 handle_command_reply: tid=0 2024-04-21T08:48:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.082+0000 7f2e9f81c740 1 client.20058 shutdown 2024-04-21T08:48:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.082+0000 7f2e9f81c740 1 -- 172.21.15.43:0/345428549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1ff3a00 msgr2=0x55bdb1ff0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.082+0000 7f2e9f81c740 1 --2- 172.21.15.43:0/345428549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1ff3a00 0x55bdb1ff0c10 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000bfd0 tx=0x7f2e90009c80 comp rx=0 tx=0).stop 2024-04-21T08:48:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.082+0000 7f2e9f81c740 1 -- 172.21.15.43:0/345428549 shutdown_connections 2024-04-21T08:48:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.082+0000 7f2e9f81c740 1 --2- 172.21.15.43:0/345428549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1ff3a00 0x55bdb1ff0c10 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.082+0000 7f2e9f81c740 1 --2- 172.21.15.43:0/345428549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb1feae80 0x55bdb1ff0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.082+0000 7f2e9f81c740 1 --2- 172.21.15.43:0/345428549 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55bdb2003ab0 0x55bdb2005ea0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.082+0000 7f2e9f81c740 1 --2- 172.21.15.43:0/345428549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdb1fea4d0 0x55bdb1ff3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.082+0000 7f2e9f81c740 1 -- 172.21.15.43:0/345428549 >> 172.21.15.43:0/345428549 conn(0x55bdb1fe4ba0 msgr2=0x55bdb1fe68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.082+0000 7f2e7dffb640 0 client.20058 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:48:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.082+0000 7f2e9f81c740 1 -- 172.21.15.43:0/345428549 shutdown_connections 2024-04-21T08:48:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.082+0000 7f2e9f81c740 1 -- 172.21.15.43:0/345428549 wait complete. 2024-04-21T08:48:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.082+0000 7f2e9f81c740 20 client.20058 trim_cache size 0 max 16384 2024-04-21T08:48:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.095+0000 7f2e9f81c740 1 Processor -- start 2024-04-21T08:48:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.095+0000 7f2e9f81c740 1 -- start start 2024-04-21T08:48:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.095+0000 7f2e9f81c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb1fea4d0 0x55bdb1fff830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.095+0000 7f2e9f81c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1feae80 0x55bdb1ff2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.095+0000 7f2e9f81c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdb1ff3a00 0x55bdb1ff3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.095+0000 7f2e9f81c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2e981af5a0 con 0x55bdb1feae80 2024-04-21T08:48:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.095+0000 7f2e9f81c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e981b10e0 con 0x55bdb1ff3a00 2024-04-21T08:48:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.095+0000 7f2e9f81c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2e981afc70 con 0x55bdb1fea4d0 2024-04-21T08:48:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.095+0000 7f2e95d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1feae80 0x55bdb1ff2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.096+0000 7f2e95d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1feae80 0x55bdb1ff2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36910/0 (socket says 172.21.15.43:36910) 2024-04-21T08:48:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.096+0000 7f2e95d9b640 1 -- 172.21.15.43:0/3759920209 learned_addr learned my addr 172.21.15.43:0/3759920209 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.096+0000 7f2e95d9b640 1 -- 172.21.15.43:0/3759920209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb1fea4d0 msgr2=0x55bdb1fff830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.096+0000 7f2e95d9b640 1 --2- 172.21.15.43:0/3759920209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb1fea4d0 0x55bdb1fff830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.096+0000 7f2e95d9b640 1 -- 172.21.15.43:0/3759920209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdb1ff3a00 msgr2=0x55bdb1ff3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.096+0000 7f2e95d9b640 1 --2- 172.21.15.43:0/3759920209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdb1ff3a00 0x55bdb1ff3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.096+0000 7f2e95d9b640 1 -- 172.21.15.43:0/3759920209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdb1fe54e0 con 0x55bdb1feae80 2024-04-21T08:48:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.096+0000 7f2e95d9b640 1 --2- 172.21.15.43:0/3759920209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1feae80 0x55bdb1ff2dc0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c006010 tx=0x7f2e8c0df040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c55b640cd0bb992 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.096+0000 7f2e7ffff640 1 -- 172.21.15.43:0/3759920209 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c0bf100 con 0x55bdb1feae80 2024-04-21T08:48:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.096+0000 7f2e7ffff640 1 -- 172.21.15.43:0/3759920209 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e8c0bea00 con 0x55bdb1feae80 2024-04-21T08:48:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.096+0000 7f2e7ffff640 1 -- 172.21.15.43:0/3759920209 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c0f0320 con 0x55bdb1feae80 2024-04-21T08:48:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.096+0000 7f2e9f81c740 1 -- 172.21.15.43:0/3759920209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1feae80 msgr2=0x55bdb1ff2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.097+0000 7f2e9f81c740 1 --2- 172.21.15.43:0/3759920209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1feae80 0x55bdb1ff2dc0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c006010 tx=0x7f2e8c0df040 comp rx=0 tx=0).stop 2024-04-21T08:48:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.097+0000 7f2e9f81c740 1 -- 172.21.15.43:0/3759920209 shutdown_connections 2024-04-21T08:48:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.097+0000 7f2e9f81c740 1 --2- 172.21.15.43:0/3759920209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdb1ff3a00 0x55bdb1ff3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.097+0000 7f2e9f81c740 1 --2- 172.21.15.43:0/3759920209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1feae80 0x55bdb1ff2dc0 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.097+0000 7f2e9f81c740 1 --2- 172.21.15.43:0/3759920209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb1fea4d0 0x55bdb1fff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.097+0000 7f2e9f81c740 1 -- 172.21.15.43:0/3759920209 >> 172.21.15.43:0/3759920209 conn(0x55bdb1fe4ba0 msgr2=0x55bdb1fe3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.097+0000 7f2e9f81c740 1 -- 172.21.15.43:0/3759920209 shutdown_connections 2024-04-21T08:48:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.097+0000 7f2e9f81c740 1 -- 172.21.15.43:0/3759920209 wait complete. 2024-04-21T08:48:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.097+0000 7f2e9f81c740 1 -- start start 2024-04-21T08:48:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.097+0000 7f2e9f81c740 1 Processor -- start 2024-04-21T08:48:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.097+0000 7f2e9f81c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdb1fea4d0 0x55bdb1ffbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.097+0000 7f2e9f81c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1feae80 0x55bdb1fecf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.097+0000 7f2e9f81c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb1ff3a00 0x55bdb1fed440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.097+0000 7f2e9f81c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2e981b0a10 con 0x55bdb1feae80 2024-04-21T08:48:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.097+0000 7f2e9f81c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e981b0340 con 0x55bdb1fea4d0 2024-04-21T08:48:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.097+0000 7f2e9f81c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2e981ad5a0 con 0x55bdb1ff3a00 2024-04-21T08:48:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.097+0000 7f2e9f81c740 10 client.0 _ll_register_callbacks cb 0x7f2e9000ad00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.097+0000 7f2e95d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1feae80 0x55bdb1fecf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.097+0000 7f2e95d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1feae80 0x55bdb1fecf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36918/0 (socket says 172.21.15.43:36918) 2024-04-21T08:48:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.097+0000 7f2e95d9b640 1 -- 172.21.15.43:0/3538209856 learned_addr learned my addr 172.21.15.43:0/3538209856 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.097+0000 7f2e95d9b640 1 -- 172.21.15.43:0/3538209856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb1ff3a00 msgr2=0x55bdb1fed440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.097+0000 7f2e95d9b640 1 --2- 172.21.15.43:0/3538209856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb1ff3a00 0x55bdb1fed440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.097+0000 7f2e95d9b640 1 -- 172.21.15.43:0/3538209856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdb1fea4d0 msgr2=0x55bdb1ffbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.098+0000 7f2e95d9b640 1 --2- 172.21.15.43:0/3538209856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdb1fea4d0 0x55bdb1ffbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.098+0000 7f2e95d9b640 1 -- 172.21.15.43:0/3538209856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdb1fedae0 con 0x55bdb1feae80 2024-04-21T08:48:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.098+0000 7f2e95d9b640 1 --2- 172.21.15.43:0/3538209856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1feae80 0x55bdb1fecf00 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0be700 tx=0x7f2e8c003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3cc7f663e7ae9e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.098+0000 7f2e7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.098+0000 7f2e7dffb640 1 -- 172.21.15.43:0/3538209856 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c0d7070 con 0x55bdb1feae80 2024-04-21T08:48:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.098+0000 7f2e7dffb640 1 -- 172.21.15.43:0/3538209856 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f2e8c0bb070 con 0x55bdb1feae80 2024-04-21T08:48:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.098+0000 7f2e9f81c740 1 -- 172.21.15.43:0/3538209856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e8c0bea00 con 0x55bdb1feae80 2024-04-21T08:48:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.098+0000 7f2e7dffb640 1 -- 172.21.15.43:0/3538209856 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2e8c0c2db0 con 0x55bdb1feae80 2024-04-21T08:48:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.098+0000 7f2e9f81c740 10 client.20064 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.098+0000 7f2e9f81c740 1 -- 172.21.15.43:0/3538209856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bdb2069bd0 con 0x55bdb1feae80 2024-04-21T08:48:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.098+0000 7f2e7dffb640 1 -- 172.21.15.43:0/3538209856 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e8c0bea00 con 0x55bdb1feae80 2024-04-21T08:48:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.098+0000 7f2e7dffb640 1 -- 172.21.15.43:0/3538209856 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2e8c0ddca0 con 0x55bdb1feae80 2024-04-21T08:48:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.098+0000 7f2e9f81c740 10 client.20064 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.098+0000 7f2e9f81c740 10 client.20064 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:48:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.098+0000 7f2e9f81c740 20 client.20064 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.098+0000 7f2e9f81c740 1 --2- 172.21.15.43:0/3538209856 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55bdb206a310 0x55bdb206c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.098+0000 7f2e9f81c740 4 client.20064 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.098+0000 7f2e9f81c740 1 -- 172.21.15.43:0/3538209856 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bdb1f5cc10 con 0x55bdb206a310 2024-04-21T08:48:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.099+0000 7f2e9659c640 1 --2- 172.21.15.43:0/3538209856 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55bdb206a310 0x55bdb206c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.099+0000 7f2e9659c640 1 --2- 172.21.15.43:0/3538209856 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55bdb206a310 0x55bdb206c730 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e6ce0ade27311c server_cookie=c79c7bca469783a1 in_seq=0 out_seq=0 2024-04-21T08:48:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.099+0000 7f2e7dffb640 10 client.20064 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:48:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.099+0000 7f2e7dffb640 1 -- 172.21.15.43:0/3538209856 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bdb1f5cc10 con 0x55bdb206a310 2024-04-21T08:48:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.099+0000 7f2e7dffb640 10 client.20064 handle_command_reply: tid=0 2024-04-21T08:48:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.099+0000 7f2e9f81c740 1 client.20064 shutdown 2024-04-21T08:48:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.099+0000 7f2e9f81c740 1 -- 172.21.15.43:0/3538209856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1feae80 msgr2=0x55bdb1fecf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.099+0000 7f2e9f81c740 1 --2- 172.21.15.43:0/3538209856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1feae80 0x55bdb1fecf00 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0be700 tx=0x7f2e8c003680 comp rx=0 tx=0).stop 2024-04-21T08:48:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.100+0000 7f2e9f81c740 1 -- 172.21.15.43:0/3538209856 shutdown_connections 2024-04-21T08:48:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.100+0000 7f2e9f81c740 1 --2- 172.21.15.43:0/3538209856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bdb1ff3a00 0x55bdb1fed440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.100+0000 7f2e9f81c740 1 --2- 172.21.15.43:0/3538209856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bdb1feae80 0x55bdb1fecf00 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.100+0000 7f2e9f81c740 1 --2- 172.21.15.43:0/3538209856 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55bdb206a310 0x55bdb206c730 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.100+0000 7f2e9f81c740 1 --2- 172.21.15.43:0/3538209856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdb1fea4d0 0x55bdb1ffbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.100+0000 7f2e7dffb640 0 client.20064 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:48:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.100+0000 7f2e9f81c740 1 -- 172.21.15.43:0/3538209856 >> 172.21.15.43:0/3538209856 conn(0x55bdb1fe4ba0 msgr2=0x55bdb1fe5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.100+0000 7f2e9f81c740 1 -- 172.21.15.43:0/3538209856 shutdown_connections 2024-04-21T08:48:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.100+0000 7f2e9f81c740 1 -- 172.21.15.43:0/3538209856 wait complete. 2024-04-21T08:48:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.100+0000 7f2e9f81c740 20 client.20064 trim_cache size 0 max 16384 2024-04-21T08:48:25.105 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:48:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.100+0000 7f2e9cc32640 1 -- 172.21.15.43:0/1475810340 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2e800a3d20 msgr2=0x7f2e800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.100+0000 7f2e9cc32640 1 --2- 172.21.15.43:0/1475810340 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2e800a3d20 0x7f2e800a61e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2e90009fd0 tx=0x7f2e9001a000 comp rx=0 tx=0).stop 2024-04-21T08:48:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.100+0000 7f2e9cc32640 1 -- 172.21.15.43:0/1475810340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e98103af0 msgr2=0x7f2e9819eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.100+0000 7f2e9cc32640 1 --2- 172.21.15.43:0/1475810340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e98103af0 0x7f2e9819eaf0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c005940 tx=0x7f2e8c02d920 comp rx=0 tx=0).stop 2024-04-21T08:48:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.100+0000 7f2e9cc32640 1 -- 172.21.15.43:0/1475810340 shutdown_connections 2024-04-21T08:48:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.100+0000 7f2e9cc32640 1 --2- 172.21.15.43:0/1475810340 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2e800a3d20 0x7f2e800a61e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.100+0000 7f2e9cc32640 1 --2- 172.21.15.43:0/1475810340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e9810c510 0x7f2e98198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.100+0000 7f2e9cc32640 1 --2- 172.21.15.43:0/1475810340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e98103af0 0x7f2e9819eaf0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.100+0000 7f2e9cc32640 1 --2- 172.21.15.43:0/1475810340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e981031d0 0x7f2e9819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.100+0000 7f2e9cc32640 1 -- 172.21.15.43:0/1475810340 >> 172.21.15.43:0/1475810340 conn(0x7f2e98077f50 msgr2=0x7f2e980ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.100+0000 7f2e9cc32640 1 -- 172.21.15.43:0/1475810340 shutdown_connections 2024-04-21T08:48:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.100+0000 7f2e9cc32640 1 -- 172.21.15.43:0/1475810340 wait complete. 2024-04-21T08:48:25.110 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:48:25.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.216+0000 7f55deb52640 1 -- 172.21.15.43:0/97061241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55d806a840 msgr2=0x7f55d806ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.216+0000 7f55deb52640 1 --2- 172.21.15.43:0/97061241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55d806a840 0x7f55d806ac20 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009900 tx=0x7f55d402bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:25.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.216+0000 7f55deb52640 1 -- 172.21.15.43:0/97061241 shutdown_connections 2024-04-21T08:48:25.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.216+0000 7f55deb52640 1 --2- 172.21.15.43:0/97061241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55d8107270 0x7f55d810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.216+0000 7f55deb52640 1 --2- 172.21.15.43:0/97061241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d806b1f0 0x7f55d8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.216+0000 7f55deb52640 1 --2- 172.21.15.43:0/97061241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55d806a840 0x7f55d806ac20 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.216+0000 7f55deb52640 1 -- 172.21.15.43:0/97061241 >> 172.21.15.43:0/97061241 conn(0x7f55d8077eb0 msgr2=0x7f55d80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.216+0000 7f55deb52640 1 -- 172.21.15.43:0/97061241 shutdown_connections 2024-04-21T08:48:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.216+0000 7f55deb52640 1 -- 172.21.15.43:0/97061241 wait complete. 2024-04-21T08:48:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.217+0000 7f55deb52640 1 Processor -- start 2024-04-21T08:48:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.217+0000 7f55deb52640 1 -- start start 2024-04-21T08:48:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.217+0000 7f55deb52640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55d806a840 0x7f55d8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.217+0000 7f55deb52640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d8107270 0x7f55d8198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.217+0000 7f55deb52640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55d819cfd0 0x7f55d819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.217+0000 7f55deb52640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81121d0 con 0x7f55d819cfd0 2024-04-21T08:48:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.217+0000 7f55deb52640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55d8112050 con 0x7f55d806a840 2024-04-21T08:48:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.217+0000 7f55deb52640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f55d8112350 con 0x7f55d8107270 2024-04-21T08:48:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.217+0000 7f55dd0ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55d819cfd0 0x7f55d819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.217+0000 7f55dd0ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55d819cfd0 0x7f55d819d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36946/0 (socket says 172.21.15.43:36946) 2024-04-21T08:48:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.217+0000 7f55dd0ef640 1 -- 172.21.15.43:0/3270294288 learned_addr learned my addr 172.21.15.43:0/3270294288 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.217+0000 7f55dc8ee640 1 --2- 172.21.15.43:0/3270294288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55d806a840 0x7f55d8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.217+0000 7f55cffff640 1 --2- 172.21.15.43:0/3270294288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d8107270 0x7f55d8198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.217+0000 7f55dd0ef640 1 -- 172.21.15.43:0/3270294288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d8107270 msgr2=0x7f55d8198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.217+0000 7f55dd0ef640 1 --2- 172.21.15.43:0/3270294288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d8107270 0x7f55d8198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.217+0000 7f55dd0ef640 1 -- 172.21.15.43:0/3270294288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55d806a840 msgr2=0x7f55d8198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.217+0000 7f55dd0ef640 1 --2- 172.21.15.43:0/3270294288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55d806a840 0x7f55d8198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.217+0000 7f55dd0ef640 1 -- 172.21.15.43:0/3270294288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d819dac0 con 0x7f55d819cfd0 2024-04-21T08:48:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.217+0000 7f55dc8ee640 1 --2- 172.21.15.43:0/3270294288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55d806a840 0x7f55d8198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.217+0000 7f55cffff640 1 --2- 172.21.15.43:0/3270294288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d8107270 0x7f55d8198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.217+0000 7f55dd0ef640 1 --2- 172.21.15.43:0/3270294288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55d819cfd0 0x7f55d819d480 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f55d000eb10 tx=0x7f55d000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.218+0000 7f55cdffb640 1 -- 172.21.15.43:0/3270294288 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0004b20 con 0x7f55d819cfd0 2024-04-21T08:48:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.218+0000 7f55cdffb640 1 -- 172.21.15.43:0/3270294288 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d0004cc0 con 0x7f55d819cfd0 2024-04-21T08:48:25.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.218+0000 7f55cdffb640 1 -- 172.21.15.43:0/3270294288 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0010670 con 0x7f55d819cfd0 2024-04-21T08:48:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.218+0000 7f55deb52640 1 -- 172.21.15.43:0/3270294288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d819ddb0 con 0x7f55d819cfd0 2024-04-21T08:48:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.218+0000 7f55deb52640 1 -- 172.21.15.43:0/3270294288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d81a5550 con 0x7f55d819cfd0 2024-04-21T08:48:25.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.220+0000 7f55cdffb640 1 -- 172.21.15.43:0/3270294288 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f55d0005020 con 0x7f55d819cfd0 2024-04-21T08:48:25.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.220+0000 7f55e1828740 1 Processor -- start 2024-04-21T08:48:25.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.220+0000 7f55e1828740 1 -- start start 2024-04-21T08:48:25.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.221+0000 7f55cdffb640 1 --2- 172.21.15.43:0/3270294288 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f55c00a3d20 0x7f55c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.221+0000 7f55e1828740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac244d0 0x55da5ac248b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.221+0000 7f55e1828740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5ac24e80 0x55da5ac2d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.221+0000 7f55e1828740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da5ac2da00 0x55da5ac2fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.221+0000 7f55e1828740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55da5ab891f0 con 0x55da5ac244d0 2024-04-21T08:48:25.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.221+0000 7f55e1828740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55da5ab264b0 con 0x55da5ac2da00 2024-04-21T08:48:25.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.221+0000 7f55e1828740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55da5ab851b0 con 0x55da5ac24e80 2024-04-21T08:48:25.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.221+0000 7f55cffff640 1 --2- 172.21.15.43:0/3270294288 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f55c00a3d20 0x7f55c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.221+0000 7f55dc8ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac244d0 0x55da5ac248b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.221+0000 7f55dc8ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac244d0 0x55da5ac248b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36952/0 (socket says 172.21.15.43:36952) 2024-04-21T08:48:25.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.221+0000 7f55dc8ee640 1 -- 172.21.15.43:0/2843684109 learned_addr learned my addr 172.21.15.43:0/2843684109 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:25.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.221+0000 7f55cdffb640 1 -- 172.21.15.43:0/3270294288 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f55d00d12f0 con 0x7f55d819cfd0 2024-04-21T08:48:25.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.221+0000 7f55dd0ef640 1 --2- 172.21.15.43:0/2843684109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da5ac2da00 0x55da5ac2fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.221+0000 7f55cffff640 1 --2- 172.21.15.43:0/2843684109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5ac24e80 0x55da5ac2d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.221+0000 7f55dc8ee640 1 -- 172.21.15.43:0/2843684109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5ac24e80 msgr2=0x55da5ac2d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:25.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.221+0000 7f55dc8ee640 1 --2- 172.21.15.43:0/2843684109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5ac24e80 0x55da5ac2d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.221+0000 7f55dc8ee640 1 -- 172.21.15.43:0/2843684109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da5ac2da00 msgr2=0x55da5ac2fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:25.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.221+0000 7f55dc8ee640 1 --2- 172.21.15.43:0/2843684109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da5ac2da00 0x55da5ac2fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.221+0000 7f55dc8ee640 1 -- 172.21.15.43:0/2843684109 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da5ab181b0 con 0x55da5ac244d0 2024-04-21T08:48:25.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.221+0000 7f55cffff640 1 --2- 172.21.15.43:0/3270294288 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f55c00a3d20 0x7f55c00a61e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f55c80058c0 tx=0x7f55c8009790 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:25.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.221+0000 7f55dc8ee640 1 --2- 172.21.15.43:0/2843684109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac244d0 0x55da5ac248b0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009900 tx=0x7f55d4005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61fd912d14cf508 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:25.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.222+0000 7f55beffd640 1 -- 172.21.15.43:0/2843684109 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4044020 con 0x55da5ac244d0 2024-04-21T08:48:25.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.222+0000 7f55beffd640 1 -- 172.21.15.43:0/2843684109 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d402cc90 con 0x55da5ac244d0 2024-04-21T08:48:25.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.222+0000 7f55beffd640 1 -- 172.21.15.43:0/2843684109 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d40046c0 con 0x55da5ac244d0 2024-04-21T08:48:25.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.222+0000 7f55e1828740 1 -- 172.21.15.43:0/2843684109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac244d0 msgr2=0x55da5ac248b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.222+0000 7f55e1828740 1 --2- 172.21.15.43:0/2843684109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac244d0 0x55da5ac248b0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009900 tx=0x7f55d4005e50 comp rx=0 tx=0).stop 2024-04-21T08:48:25.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.222+0000 7f55e1828740 1 -- 172.21.15.43:0/2843684109 shutdown_connections 2024-04-21T08:48:25.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.222+0000 7f55e1828740 1 --2- 172.21.15.43:0/2843684109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da5ac2da00 0x55da5ac2fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.222+0000 7f55e1828740 1 --2- 172.21.15.43:0/2843684109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5ac24e80 0x55da5ac2d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.222+0000 7f55e1828740 1 --2- 172.21.15.43:0/2843684109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac244d0 0x55da5ac248b0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.222+0000 7f55e1828740 1 -- 172.21.15.43:0/2843684109 >> 172.21.15.43:0/2843684109 conn(0x55da5ac1eba0 msgr2=0x55da5ac1ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:25.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.222+0000 7f55e1828740 1 -- 172.21.15.43:0/2843684109 shutdown_connections 2024-04-21T08:48:25.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.222+0000 7f55e1828740 1 -- 172.21.15.43:0/2843684109 wait complete. 2024-04-21T08:48:25.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.223+0000 7f55e1828740 1 -- start start 2024-04-21T08:48:25.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.223+0000 7f55e1828740 1 Processor -- start 2024-04-21T08:48:25.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.223+0000 7f55e1828740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da5ac244d0 0x55da5ac2d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.223+0000 7f55e1828740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5ac24e80 0x55da5ac2a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.223+0000 7f55e1828740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac2da00 0x55da5ac2ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.223+0000 7f55e1828740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55da5ac30910 con 0x55da5ac2da00 2024-04-21T08:48:25.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.223+0000 7f55e1828740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55da5ab42500 con 0x55da5ac244d0 2024-04-21T08:48:25.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.223+0000 7f55e1828740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55da5ac30a90 con 0x55da5ac24e80 2024-04-21T08:48:25.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.223+0000 7f55e1828740 10 client.0 _ll_register_callbacks cb 0x55da5ab10b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:25.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.223+0000 7f55dd0ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac2da00 0x55da5ac2ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.223+0000 7f55dd0ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac2da00 0x55da5ac2ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36954/0 (socket says 172.21.15.43:36954) 2024-04-21T08:48:25.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.223+0000 7f55dd0ef640 1 -- 172.21.15.43:0/2437447673 learned_addr learned my addr 172.21.15.43:0/2437447673 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:25.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.223+0000 7f55dc8ee640 1 --2- 172.21.15.43:0/2437447673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da5ac244d0 0x55da5ac2d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.224+0000 7f55cffff640 1 --2- 172.21.15.43:0/2437447673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5ac24e80 0x55da5ac2a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.224+0000 7f55dd0ef640 1 -- 172.21.15.43:0/2437447673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5ac24e80 msgr2=0x55da5ac2a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:25.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.224+0000 7f55dd0ef640 1 --2- 172.21.15.43:0/2437447673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5ac24e80 0x55da5ac2a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.224+0000 7f55dd0ef640 1 -- 172.21.15.43:0/2437447673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da5ac244d0 msgr2=0x55da5ac2d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:25.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.224+0000 7f55dd0ef640 1 --2- 172.21.15.43:0/2437447673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da5ac244d0 0x55da5ac2d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.224+0000 7f55dd0ef640 1 -- 172.21.15.43:0/2437447673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da5a89b3f0 con 0x55da5ac2da00 2024-04-21T08:48:25.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.224+0000 7f55dd0ef640 1 --2- 172.21.15.43:0/2437447673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac2da00 0x55da5ac2ace0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f55d00049d0 tx=0x7f55d009c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc70751c847344d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:25.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.224+0000 7f55b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:25.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.224+0000 7f55b7fff640 1 -- 172.21.15.43:0/2437447673 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0092300 con 0x55da5ac2da00 2024-04-21T08:48:25.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.224+0000 7f55b7fff640 1 -- 172.21.15.43:0/2437447673 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f55d00975e0 con 0x55da5ac2da00 2024-04-21T08:48:25.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.224+0000 7f55e1828740 1 -- 172.21.15.43:0/2437447673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55d402cc90 con 0x55da5ac2da00 2024-04-21T08:48:25.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.224+0000 7f55b7fff640 1 -- 172.21.15.43:0/2437447673 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f55d00a11c0 con 0x55da5ac2da00 2024-04-21T08:48:25.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.224+0000 7f55e1828740 10 client.20076 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:25.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.224+0000 7f55e1828740 1 -- 172.21.15.43:0/2437447673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55da5a9657d0 con 0x55da5ac2da00 2024-04-21T08:48:25.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.224+0000 7f55b7fff640 1 -- 172.21.15.43:0/2437447673 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d402cc90 con 0x55da5ac2da00 2024-04-21T08:48:25.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.224+0000 7f55b7fff640 1 -- 172.21.15.43:0/2437447673 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f55d00b6210 con 0x55da5ac2da00 2024-04-21T08:48:25.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.224+0000 7f55e1828740 10 client.20076 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:25.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.224+0000 7f55e1828740 10 client.20076 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:48:25.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.224+0000 7f55e1828740 20 client.20076 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:25.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.224+0000 7f55e1828740 1 --2- 172.21.15.43:0/2437447673 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55da5ac3dbc0 0x55da5ac3fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.225+0000 7f55e1828740 4 client.20076 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:25.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.225+0000 7f55e1828740 1 -- 172.21.15.43:0/2437447673 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da5ab28930 con 0x55da5ac3dbc0 2024-04-21T08:48:25.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.225+0000 7f55dc8ee640 1 --2- 172.21.15.43:0/2437447673 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55da5ac3dbc0 0x55da5ac3fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.225+0000 7f55b7fff640 10 client.20076 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:48:25.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.225+0000 7f55dc8ee640 1 --2- 172.21.15.43:0/2437447673 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55da5ac3dbc0 0x55da5ac3fff0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d0ec6a97dee5febb server_cookie=ad82aee9074a8985 in_seq=0 out_seq=0 2024-04-21T08:48:25.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.226+0000 7f55b7fff640 1 -- 172.21.15.43:0/2437447673 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da5ab28930 con 0x55da5ac3dbc0 2024-04-21T08:48:25.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.226+0000 7f55b7fff640 10 client.20076 handle_command_reply: tid=0 2024-04-21T08:48:25.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.226+0000 7f55e1828740 1 client.20076 shutdown 2024-04-21T08:48:25.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.226+0000 7f55e1828740 1 -- 172.21.15.43:0/2437447673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac2da00 msgr2=0x55da5ac2ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.226+0000 7f55e1828740 1 --2- 172.21.15.43:0/2437447673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac2da00 0x55da5ac2ace0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f55d00049d0 tx=0x7f55d009c830 comp rx=0 tx=0).stop 2024-04-21T08:48:25.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.226+0000 7f55e1828740 1 -- 172.21.15.43:0/2437447673 shutdown_connections 2024-04-21T08:48:25.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.226+0000 7f55e1828740 1 --2- 172.21.15.43:0/2437447673 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55da5ac3dbc0 0x55da5ac3fff0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.226+0000 7f55e1828740 1 --2- 172.21.15.43:0/2437447673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac2da00 0x55da5ac2ace0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.226+0000 7f55e1828740 1 --2- 172.21.15.43:0/2437447673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5ac24e80 0x55da5ac2a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.226+0000 7f55e1828740 1 --2- 172.21.15.43:0/2437447673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da5ac244d0 0x55da5ac2d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.226+0000 7f55b7fff640 0 client.20076 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:48:25.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.226+0000 7f55e1828740 1 -- 172.21.15.43:0/2437447673 >> 172.21.15.43:0/2437447673 conn(0x55da5ac1eba0 msgr2=0x55da5ac208f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:25.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.226+0000 7f55e1828740 1 -- 172.21.15.43:0/2437447673 shutdown_connections 2024-04-21T08:48:25.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.226+0000 7f55e1828740 1 -- 172.21.15.43:0/2437447673 wait complete. 2024-04-21T08:48:25.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.226+0000 7f55e1828740 20 client.20076 trim_cache size 0 max 16384 2024-04-21T08:48:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.240+0000 7f55e1828740 1 Processor -- start 2024-04-21T08:48:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.240+0000 7f55e1828740 1 -- start start 2024-04-21T08:48:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.240+0000 7f55e1828740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da5ac244d0 0x55da5ac397d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.240+0000 7f55e1828740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5ac24e80 0x55da5ac2ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.240+0000 7f55e1828740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac2da00 0x55da5ac2d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.240+0000 7f55e1828740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81af490 con 0x55da5ac2da00 2024-04-21T08:48:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.240+0000 7f55e1828740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81b0fd0 con 0x55da5ac244d0 2024-04-21T08:48:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.240+0000 7f55e1828740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81afb60 con 0x55da5ac24e80 2024-04-21T08:48:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.240+0000 7f55dd0ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac2da00 0x55da5ac2d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.240+0000 7f55dd0ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac2da00 0x55da5ac2d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36960/0 (socket says 172.21.15.43:36960) 2024-04-21T08:48:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.240+0000 7f55dd0ef640 1 -- 172.21.15.43:0/51288407 learned_addr learned my addr 172.21.15.43:0/51288407 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.240+0000 7f55cffff640 1 --2- 172.21.15.43:0/51288407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5ac24e80 0x55da5ac2ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.240+0000 7f55dd0ef640 1 -- 172.21.15.43:0/51288407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5ac24e80 msgr2=0x55da5ac2ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:25.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.240+0000 7f55dd0ef640 1 --2- 172.21.15.43:0/51288407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5ac24e80 0x55da5ac2ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.240+0000 7f55dd0ef640 1 -- 172.21.15.43:0/51288407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da5ac244d0 msgr2=0x55da5ac397d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:25.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.240+0000 7f55dc8ee640 1 --2- 172.21.15.43:0/51288407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da5ac244d0 0x55da5ac397d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.240+0000 7f55dd0ef640 1 --2- 172.21.15.43:0/51288407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da5ac244d0 0x55da5ac397d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.240+0000 7f55dd0ef640 1 -- 172.21.15.43:0/51288407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da5ac3a340 con 0x55da5ac2da00 2024-04-21T08:48:25.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.240+0000 7f55dd0ef640 1 --2- 172.21.15.43:0/51288407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac2da00 0x55da5ac2d340 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f55d00997e0 tx=0x7f55d009c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ac94ebdc1984dc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:25.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.240+0000 7f55be7fc640 1 -- 172.21.15.43:0/51288407 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d00a5a80 con 0x55da5ac2da00 2024-04-21T08:48:25.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.240+0000 7f55be7fc640 1 -- 172.21.15.43:0/51288407 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d006c040 con 0x55da5ac2da00 2024-04-21T08:48:25.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.241+0000 7f55be7fc640 1 -- 172.21.15.43:0/51288407 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d00bd110 con 0x55da5ac2da00 2024-04-21T08:48:25.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.241+0000 7f55e1828740 1 -- 172.21.15.43:0/51288407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac2da00 msgr2=0x55da5ac2d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.241+0000 7f55e1828740 1 --2- 172.21.15.43:0/51288407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac2da00 0x55da5ac2d340 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f55d00997e0 tx=0x7f55d009c8a0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.241+0000 7f55e1828740 1 -- 172.21.15.43:0/51288407 shutdown_connections 2024-04-21T08:48:25.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.241+0000 7f55e1828740 1 --2- 172.21.15.43:0/51288407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac2da00 0x55da5ac2d340 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.241+0000 7f55e1828740 1 --2- 172.21.15.43:0/51288407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5ac24e80 0x55da5ac2ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.241+0000 7f55e1828740 1 --2- 172.21.15.43:0/51288407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da5ac244d0 0x55da5ac397d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.241+0000 7f55e1828740 1 -- 172.21.15.43:0/51288407 >> 172.21.15.43:0/51288407 conn(0x55da5ac1eba0 msgr2=0x55da5ac20860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.241+0000 7f55e1828740 1 -- 172.21.15.43:0/51288407 shutdown_connections 2024-04-21T08:48:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.241+0000 7f55e1828740 1 -- 172.21.15.43:0/51288407 wait complete. 2024-04-21T08:48:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.241+0000 7f55e1828740 1 -- start start 2024-04-21T08:48:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.241+0000 7f55e1828740 1 Processor -- start 2024-04-21T08:48:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.242+0000 7f55e1828740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da5ac244d0 0x55da5ac31da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.242+0000 7f55e1828740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac24e80 0x55da5ac322e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.242+0000 7f55e1828740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5ac2da00 0x55da5aca3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.242+0000 7f55e1828740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81b0900 con 0x55da5ac24e80 2024-04-21T08:48:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.242+0000 7f55e1828740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81b0230 con 0x55da5ac244d0 2024-04-21T08:48:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.242+0000 7f55e1828740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81ad490 con 0x55da5ac2da00 2024-04-21T08:48:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.242+0000 7f55e1828740 10 client.0 _ll_register_callbacks cb 0x7f55d009c950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.242+0000 7f55cffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac24e80 0x55da5ac322e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.242+0000 7f55cffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac24e80 0x55da5ac322e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36962/0 (socket says 172.21.15.43:36962) 2024-04-21T08:48:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.242+0000 7f55cffff640 1 -- 172.21.15.43:0/3308197877 learned_addr learned my addr 172.21.15.43:0/3308197877 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.242+0000 7f55dc8ee640 1 --2- 172.21.15.43:0/3308197877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da5ac244d0 0x55da5ac31da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.242+0000 7f55dd0ef640 1 --2- 172.21.15.43:0/3308197877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5ac2da00 0x55da5aca3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.242+0000 7f55cffff640 1 -- 172.21.15.43:0/3308197877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5ac2da00 msgr2=0x55da5aca3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.242+0000 7f55cffff640 1 --2- 172.21.15.43:0/3308197877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5ac2da00 0x55da5aca3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.242+0000 7f55cffff640 1 -- 172.21.15.43:0/3308197877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da5ac244d0 msgr2=0x55da5ac31da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.242+0000 7f55cffff640 1 --2- 172.21.15.43:0/3308197877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da5ac244d0 0x55da5ac31da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.242+0000 7f55cffff640 1 -- 172.21.15.43:0/3308197877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da5aca3670 con 0x55da5ac24e80 2024-04-21T08:48:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.242+0000 7f55cffff640 1 --2- 172.21.15.43:0/3308197877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac24e80 0x55da5ac322e0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x55da5ac23fa0 tx=0x7f55c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64bd968f22da61ea server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.242+0000 7f55b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.243+0000 7f55b7fff640 1 -- 172.21.15.43:0/3308197877 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8026070 con 0x55da5ac24e80 2024-04-21T08:48:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.243+0000 7f55b7fff640 1 -- 172.21.15.43:0/3308197877 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f55c8005220 con 0x55da5ac24e80 2024-04-21T08:48:25.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.243+0000 7f55e1828740 1 -- 172.21.15.43:0/3308197877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55d006c040 con 0x55da5ac24e80 2024-04-21T08:48:25.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.243+0000 7f55b7fff640 1 -- 172.21.15.43:0/3308197877 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f55c8005420 con 0x55da5ac24e80 2024-04-21T08:48:25.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.243+0000 7f55e1828740 10 client.20082 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:25.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.243+0000 7f55e1828740 1 -- 172.21.15.43:0/3308197877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55da5aca48a0 con 0x55da5ac24e80 2024-04-21T08:48:25.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.243+0000 7f55b7fff640 1 -- 172.21.15.43:0/3308197877 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d006c040 con 0x55da5ac24e80 2024-04-21T08:48:25.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.243+0000 7f55b7fff640 1 -- 172.21.15.43:0/3308197877 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f55c800f040 con 0x55da5ac24e80 2024-04-21T08:48:25.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.243+0000 7f55e1828740 10 client.20082 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:25.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.243+0000 7f55e1828740 10 client.20082 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:48:25.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.243+0000 7f55e1828740 20 client.20082 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:25.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.243+0000 7f55e1828740 1 --2- 172.21.15.43:0/3308197877 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55da5aca5020 0x55da5aca7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.243+0000 7f55e1828740 4 client.20082 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:25.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.243+0000 7f55e1828740 1 -- 172.21.15.43:0/3308197877 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da5ab96c10 con 0x55da5aca5020 2024-04-21T08:48:25.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.244+0000 7f55dc8ee640 1 --2- 172.21.15.43:0/3308197877 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55da5aca5020 0x55da5aca7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.244+0000 7f55dc8ee640 1 --2- 172.21.15.43:0/3308197877 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55da5aca5020 0x55da5aca7440 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4e9e423db5cd31c server_cookie=4fd38138ce44ddf5 in_seq=0 out_seq=0 2024-04-21T08:48:25.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.244+0000 7f55b7fff640 10 client.20082 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:48:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.244+0000 7f55b7fff640 1 -- 172.21.15.43:0/3308197877 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da5ab96c10 con 0x55da5aca5020 2024-04-21T08:48:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.244+0000 7f55b7fff640 10 client.20082 handle_command_reply: tid=0 2024-04-21T08:48:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.244+0000 7f55e1828740 1 client.20082 shutdown 2024-04-21T08:48:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.244+0000 7f55e1828740 1 -- 172.21.15.43:0/3308197877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac24e80 msgr2=0x55da5ac322e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.244+0000 7f55e1828740 1 --2- 172.21.15.43:0/3308197877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac24e80 0x55da5ac322e0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x55da5ac23fa0 tx=0x7f55c8004730 comp rx=0 tx=0).stop 2024-04-21T08:48:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.244+0000 7f55e1828740 1 -- 172.21.15.43:0/3308197877 shutdown_connections 2024-04-21T08:48:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.244+0000 7f55e1828740 1 --2- 172.21.15.43:0/3308197877 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55da5aca5020 0x55da5aca7440 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.244+0000 7f55e1828740 1 --2- 172.21.15.43:0/3308197877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da5ac2da00 0x55da5aca3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.244+0000 7f55e1828740 1 --2- 172.21.15.43:0/3308197877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da5ac24e80 0x55da5ac322e0 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.244+0000 7f55b7fff640 0 client.20082 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:48:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.244+0000 7f55e1828740 1 --2- 172.21.15.43:0/3308197877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55da5ac244d0 0x55da5ac31da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.244+0000 7f55e1828740 1 -- 172.21.15.43:0/3308197877 >> 172.21.15.43:0/3308197877 conn(0x55da5ac1eba0 msgr2=0x55da5ac1d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.244+0000 7f55e1828740 1 -- 172.21.15.43:0/3308197877 shutdown_connections 2024-04-21T08:48:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.244+0000 7f55e1828740 1 -- 172.21.15.43:0/3308197877 wait complete. 2024-04-21T08:48:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.244+0000 7f55e1828740 20 client.20082 trim_cache size 0 max 16384 2024-04-21T08:48:25.251 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:48:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.245+0000 7f55deb52640 1 -- 172.21.15.43:0/3270294288 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f55c00a3d20 msgr2=0x7f55c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.245+0000 7f55deb52640 1 --2- 172.21.15.43:0/3270294288 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f55c00a3d20 0x7f55c00a61e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f55c80058c0 tx=0x7f55c8009790 comp rx=0 tx=0).stop 2024-04-21T08:48:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.245+0000 7f55deb52640 1 -- 172.21.15.43:0/3270294288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55d819cfd0 msgr2=0x7f55d819d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.245+0000 7f55deb52640 1 --2- 172.21.15.43:0/3270294288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55d819cfd0 0x7f55d819d480 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f55d000eb10 tx=0x7f55d000c2d0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.245+0000 7f55deb52640 1 -- 172.21.15.43:0/3270294288 shutdown_connections 2024-04-21T08:48:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.245+0000 7f55deb52640 1 --2- 172.21.15.43:0/3270294288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55d819cfd0 0x7f55d819d480 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.245+0000 7f55deb52640 1 --2- 172.21.15.43:0/3270294288 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f55c00a3d20 0x7f55c00a61e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.245+0000 7f55deb52640 1 --2- 172.21.15.43:0/3270294288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d8107270 0x7f55d8198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.245+0000 7f55deb52640 1 --2- 172.21.15.43:0/3270294288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55d806a840 0x7f55d8198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.245+0000 7f55deb52640 1 -- 172.21.15.43:0/3270294288 >> 172.21.15.43:0/3270294288 conn(0x7f55d8077eb0 msgr2=0x7f55d8108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.245+0000 7f55deb52640 1 -- 172.21.15.43:0/3270294288 shutdown_connections 2024-04-21T08:48:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.245+0000 7f55deb52640 1 -- 172.21.15.43:0/3270294288 wait complete. 2024-04-21T08:48:25.254 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:25 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:48:25.254 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:25 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:25.254 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:25 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:48:25.254 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:25 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:25.254 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:25 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:48:25.255 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:25 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/137622022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:25.256 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:48:25.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.362+0000 7f0978450640 1 -- 172.21.15.43:0/2198245168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0970102b40 msgr2=0x7f0970106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.362+0000 7f0978450640 1 --2- 172.21.15.43:0/2198245168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0970102b40 0x7f0970106b80 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f0960009900 tx=0x7f096002bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:25.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.362+0000 7f0978450640 1 -- 172.21.15.43:0/2198245168 shutdown_connections 2024-04-21T08:48:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.362+0000 7f0978450640 1 --2- 172.21.15.43:0/2198245168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0970107260 0x7f097010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.362+0000 7f0978450640 1 --2- 172.21.15.43:0/2198245168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0970102b40 0x7f0970106b80 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.362+0000 7f0978450640 1 --2- 172.21.15.43:0/2198245168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0970102190 0x7f0970102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.362+0000 7f0978450640 1 -- 172.21.15.43:0/2198245168 >> 172.21.15.43:0/2198245168 conn(0x7f09700fd990 msgr2=0x7f09700ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.362+0000 7f0978450640 1 -- 172.21.15.43:0/2198245168 shutdown_connections 2024-04-21T08:48:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.362+0000 7f0978450640 1 -- 172.21.15.43:0/2198245168 wait complete. 2024-04-21T08:48:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.362+0000 7f0978450640 1 Processor -- start 2024-04-21T08:48:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.362+0000 7f0978450640 1 -- start start 2024-04-21T08:48:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.363+0000 7f0978450640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0970102190 0x7f097019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.363+0000 7f0978450640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0970107260 0x7f097019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.363+0000 7f0978450640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09701985c0 0x7f0970198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.363+0000 7f0978450640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f09701121a0 con 0x7f0970102190 2024-04-21T08:48:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.363+0000 7f0978450640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0970112020 con 0x7f0970107260 2024-04-21T08:48:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.363+0000 7f0978450640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0970112320 con 0x7f09701985c0 2024-04-21T08:48:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.363+0000 7f09761ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0970102190 0x7f097019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.363+0000 7f09761ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0970102190 0x7f097019e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36980/0 (socket says 172.21.15.43:36980) 2024-04-21T08:48:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.363+0000 7f09761ec640 1 -- 172.21.15.43:0/4073095756 learned_addr learned my addr 172.21.15.43:0/4073095756 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.363+0000 7f09759eb640 1 --2- 172.21.15.43:0/4073095756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0970107260 0x7f097019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.363+0000 7f09769ed640 1 --2- 172.21.15.43:0/4073095756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09701985c0 0x7f0970198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.363+0000 7f09761ec640 1 -- 172.21.15.43:0/4073095756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09701985c0 msgr2=0x7f0970198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.363+0000 7f09761ec640 1 --2- 172.21.15.43:0/4073095756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09701985c0 0x7f0970198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.363+0000 7f09761ec640 1 -- 172.21.15.43:0/4073095756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0970107260 msgr2=0x7f097019e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.363+0000 7f09761ec640 1 --2- 172.21.15.43:0/4073095756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0970107260 0x7f097019e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.363+0000 7f09761ec640 1 -- 172.21.15.43:0/4073095756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09701990e0 con 0x7f0970102190 2024-04-21T08:48:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.363+0000 7f09761ec640 1 --2- 172.21.15.43:0/4073095756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0970102190 0x7f097019e410 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f096c00d950 tx=0x7f096c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.363+0000 7f09677fe640 1 -- 172.21.15.43:0/4073095756 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c017070 con 0x7f0970102190 2024-04-21T08:48:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.363+0000 7f0978450640 1 -- 172.21.15.43:0/4073095756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09701993d0 con 0x7f0970102190 2024-04-21T08:48:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.363+0000 7f09677fe640 1 -- 172.21.15.43:0/4073095756 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f096c009c30 con 0x7f0970102190 2024-04-21T08:48:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.363+0000 7f09677fe640 1 -- 172.21.15.43:0/4073095756 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c00f620 con 0x7f0970102190 2024-04-21T08:48:25.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.364+0000 7f0978450640 1 -- 172.21.15.43:0/4073095756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09701a5620 con 0x7f0970102190 2024-04-21T08:48:25.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.366+0000 7f09677fe640 1 -- 172.21.15.43:0/4073095756 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f096c010460 con 0x7f0970102190 2024-04-21T08:48:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.366+0000 7f097b03d740 1 Processor -- start 2024-04-21T08:48:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.366+0000 7f097b03d740 1 -- start start 2024-04-21T08:48:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.367+0000 7f097b03d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be19e054d0 0x55be19e058b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.367+0000 7f097b03d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be19e05e80 0x55be19e0e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.367+0000 7f097b03d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e0ea00 0x55be19e10e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.367+0000 7f097b03d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55be19d6a1f0 con 0x55be19e0ea00 2024-04-21T08:48:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.367+0000 7f097b03d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55be19d074b0 con 0x55be19e05e80 2024-04-21T08:48:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.367+0000 7f097b03d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55be19d661b0 con 0x55be19e054d0 2024-04-21T08:48:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.367+0000 7f09677fe640 1 --2- 172.21.15.43:0/4073095756 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f095c0a3d20 0x7f095c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.367+0000 7f09759eb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e0ea00 0x55be19e10e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.367+0000 7f09759eb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e0ea00 0x55be19e10e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36986/0 (socket says 172.21.15.43:36986) 2024-04-21T08:48:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.367+0000 7f09759eb640 1 -- 172.21.15.43:0/1272448039 learned_addr learned my addr 172.21.15.43:0/1272448039 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.367+0000 7f09761ec640 1 --2- 172.21.15.43:0/1272448039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be19e05e80 0x55be19e0e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.367+0000 7f09769ed640 1 --2- 172.21.15.43:0/4073095756 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f095c0a3d20 0x7f095c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.367+0000 7f09677fe640 1 -- 172.21.15.43:0/4073095756 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f096c0d08c0 con 0x7f0970102190 2024-04-21T08:48:25.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.367+0000 7f09769ed640 1 --2- 172.21.15.43:0/1272448039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be19e054d0 0x55be19e058b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.367+0000 7f09759eb640 1 -- 172.21.15.43:0/1272448039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be19e054d0 msgr2=0x55be19e058b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:25.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.367+0000 7f09759eb640 1 --2- 172.21.15.43:0/1272448039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be19e054d0 0x55be19e058b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.367+0000 7f09759eb640 1 -- 172.21.15.43:0/1272448039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be19e05e80 msgr2=0x55be19e0e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:25.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.367+0000 7f09759eb640 1 --2- 172.21.15.43:0/1272448039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be19e05e80 0x55be19e0e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.367+0000 7f09759eb640 1 -- 172.21.15.43:0/1272448039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be19cf91b0 con 0x55be19e0ea00 2024-04-21T08:48:25.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.367+0000 7f09769ed640 1 --2- 172.21.15.43:0/1272448039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be19e054d0 0x55be19e058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:25.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.368+0000 7f09759eb640 1 --2- 172.21.15.43:0/1272448039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e0ea00 0x55be19e10e50 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f0960009900 tx=0x7f096002c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89d6c7e0c0934b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:25.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.368+0000 7f09769ed640 1 --2- 172.21.15.43:0/4073095756 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f095c0a3d20 0x7f095c0a61e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0968009fd0 tx=0x7f0968009500 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:25.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.368+0000 7f0964bfe640 1 -- 172.21.15.43:0/1272448039 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0960045020 con 0x55be19e0ea00 2024-04-21T08:48:25.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.368+0000 7f0964bfe640 1 -- 172.21.15.43:0/1272448039 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f096002c7e0 con 0x55be19e0ea00 2024-04-21T08:48:25.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.368+0000 7f0964bfe640 1 -- 172.21.15.43:0/1272448039 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09600046c0 con 0x55be19e0ea00 2024-04-21T08:48:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.368+0000 7f097b03d740 1 -- 172.21.15.43:0/1272448039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e0ea00 msgr2=0x55be19e10e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.368+0000 7f097b03d740 1 --2- 172.21.15.43:0/1272448039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e0ea00 0x55be19e10e50 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f0960009900 tx=0x7f096002c610 comp rx=0 tx=0).stop 2024-04-21T08:48:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.368+0000 7f097b03d740 1 -- 172.21.15.43:0/1272448039 shutdown_connections 2024-04-21T08:48:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.368+0000 7f097b03d740 1 --2- 172.21.15.43:0/1272448039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e0ea00 0x55be19e10e50 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.368+0000 7f097b03d740 1 --2- 172.21.15.43:0/1272448039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be19e05e80 0x55be19e0e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.368+0000 7f097b03d740 1 --2- 172.21.15.43:0/1272448039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be19e054d0 0x55be19e058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.369+0000 7f097b03d740 1 -- 172.21.15.43:0/1272448039 >> 172.21.15.43:0/1272448039 conn(0x55be19dffba0 msgr2=0x55be19dfff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.369+0000 7f097b03d740 1 -- 172.21.15.43:0/1272448039 shutdown_connections 2024-04-21T08:48:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.369+0000 7f097b03d740 1 -- 172.21.15.43:0/1272448039 wait complete. 2024-04-21T08:48:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.369+0000 7f097b03d740 1 -- start start 2024-04-21T08:48:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.369+0000 7f097b03d740 1 Processor -- start 2024-04-21T08:48:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.369+0000 7f097b03d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e054d0 0x55be19e0e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.369+0000 7f097b03d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be19e05e80 0x55be19e0b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.369+0000 7f097b03d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be19e0ea00 0x55be19e0bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.369+0000 7f097b03d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55be19e11910 con 0x55be19e054d0 2024-04-21T08:48:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.369+0000 7f097b03d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55be19d23500 con 0x55be19e05e80 2024-04-21T08:48:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.369+0000 7f097b03d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55be19e11a90 con 0x55be19e0ea00 2024-04-21T08:48:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.369+0000 7f097b03d740 10 client.0 _ll_register_callbacks cb 0x55be19cf1b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.369+0000 7f09761ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e054d0 0x55be19e0e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.369+0000 7f09761ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e054d0 0x55be19e0e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37002/0 (socket says 172.21.15.43:37002) 2024-04-21T08:48:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.369+0000 7f09761ec640 1 -- 172.21.15.43:0/4142452978 learned_addr learned my addr 172.21.15.43:0/4142452978 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.370+0000 7f09759eb640 1 --2- 172.21.15.43:0/4142452978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be19e05e80 0x55be19e0b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.370+0000 7f09769ed640 1 --2- 172.21.15.43:0/4142452978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be19e0ea00 0x55be19e0bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.370+0000 7f09761ec640 1 -- 172.21.15.43:0/4142452978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be19e0ea00 msgr2=0x55be19e0bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.370+0000 7f09761ec640 1 --2- 172.21.15.43:0/4142452978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be19e0ea00 0x55be19e0bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.370+0000 7f09761ec640 1 -- 172.21.15.43:0/4142452978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be19e05e80 msgr2=0x55be19e0b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.370+0000 7f09761ec640 1 --2- 172.21.15.43:0/4142452978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be19e05e80 0x55be19e0b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.370+0000 7f09761ec640 1 -- 172.21.15.43:0/4142452978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be19a7c3f0 con 0x55be19e054d0 2024-04-21T08:48:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.370+0000 7f09759eb640 1 --2- 172.21.15.43:0/4142452978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be19e05e80 0x55be19e0b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:25.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.370+0000 7f09761ec640 1 --2- 172.21.15.43:0/4142452978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e054d0 0x55be19e0e280 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f096c0058c0 tx=0x7f096c0209a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd866a7b915c1209 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:25.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.370+0000 7f0959ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:25.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.370+0000 7f0959ffb640 1 -- 172.21.15.43:0/4142452978 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c020050 con 0x55be19e054d0 2024-04-21T08:48:25.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.370+0000 7f0959ffb640 1 -- 172.21.15.43:0/4142452978 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f096c0cde10 con 0x55be19e054d0 2024-04-21T08:48:25.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.370+0000 7f097b03d740 1 -- 172.21.15.43:0/4142452978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f096002c7e0 con 0x55be19e054d0 2024-04-21T08:48:25.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.370+0000 7f0959ffb640 1 -- 172.21.15.43:0/4142452978 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f096c0ce010 con 0x55be19e054d0 2024-04-21T08:48:25.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.370+0000 7f097b03d740 10 client.20094 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:25.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.370+0000 7f097b03d740 1 -- 172.21.15.43:0/4142452978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55be19b467d0 con 0x55be19e054d0 2024-04-21T08:48:25.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.370+0000 7f0959ffb640 1 -- 172.21.15.43:0/4142452978 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f096002c7e0 con 0x55be19e054d0 2024-04-21T08:48:25.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.370+0000 7f0959ffb640 1 -- 172.21.15.43:0/4142452978 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f096c020a10 con 0x55be19e054d0 2024-04-21T08:48:25.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.370+0000 7f097b03d740 10 client.20094 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:25.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.370+0000 7f097b03d740 10 client.20094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:25.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.371+0000 7f097b03d740 20 client.20094 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:25.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.371+0000 7f097b03d740 1 --2- 172.21.15.43:0/4142452978 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55be19e1ebd0 0x55be19e21060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.371+0000 7f097b03d740 4 client.20094 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:25.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.371+0000 7f097b03d740 1 -- 172.21.15.43:0/4142452978 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be19d09930 con 0x55be19e1ebd0 2024-04-21T08:48:25.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.371+0000 7f09759eb640 1 --2- 172.21.15.43:0/4142452978 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55be19e1ebd0 0x55be19e21060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.371+0000 7f09759eb640 1 --2- 172.21.15.43:0/4142452978 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55be19e1ebd0 0x55be19e21060 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b93f0662cc84f12 server_cookie=62d84e9742bfaa48 in_seq=0 out_seq=0 2024-04-21T08:48:25.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.371+0000 7f0959ffb640 10 client.20094 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:25.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.372+0000 7f0959ffb640 1 -- 172.21.15.43:0/4142452978 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55be19d09930 con 0x55be19e1ebd0 2024-04-21T08:48:25.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.372+0000 7f0959ffb640 10 client.20094 handle_command_reply: tid=0 2024-04-21T08:48:25.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.372+0000 7f097b03d740 1 client.20094 shutdown 2024-04-21T08:48:25.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.372+0000 7f097b03d740 1 -- 172.21.15.43:0/4142452978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e054d0 msgr2=0x55be19e0e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.372+0000 7f097b03d740 1 --2- 172.21.15.43:0/4142452978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e054d0 0x55be19e0e280 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f096c0058c0 tx=0x7f096c0209a0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.372+0000 7f097b03d740 1 -- 172.21.15.43:0/4142452978 shutdown_connections 2024-04-21T08:48:25.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.372+0000 7f097b03d740 1 --2- 172.21.15.43:0/4142452978 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55be19e1ebd0 0x55be19e21060 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.372+0000 7f097b03d740 1 --2- 172.21.15.43:0/4142452978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be19e0ea00 0x55be19e0bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.372+0000 7f097b03d740 1 --2- 172.21.15.43:0/4142452978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be19e05e80 0x55be19e0b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.372+0000 7f0959ffb640 0 client.20094 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:25.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.372+0000 7f097b03d740 1 --2- 172.21.15.43:0/4142452978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e054d0 0x55be19e0e280 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.372+0000 7f097b03d740 1 -- 172.21.15.43:0/4142452978 >> 172.21.15.43:0/4142452978 conn(0x55be19dffba0 msgr2=0x55be19e018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:25.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.373+0000 7f097b03d740 1 -- 172.21.15.43:0/4142452978 shutdown_connections 2024-04-21T08:48:25.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.373+0000 7f097b03d740 1 -- 172.21.15.43:0/4142452978 wait complete. 2024-04-21T08:48:25.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.373+0000 7f097b03d740 20 client.20094 trim_cache size 0 max 16384 2024-04-21T08:48:25.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.386+0000 7f097b03d740 1 Processor -- start 2024-04-21T08:48:25.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.386+0000 7f097b03d740 1 -- start start 2024-04-21T08:48:25.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.386+0000 7f097b03d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be19e054d0 0x55be19e1a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.386+0000 7f097b03d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e05e80 0x55be19e06ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.386+0000 7f097b03d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be19e0ea00 0x55be19e0dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.386+0000 7f097b03d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f09701af4e0 con 0x55be19e05e80 2024-04-21T08:48:25.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.386+0000 7f097b03d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f09701b1020 con 0x55be19e054d0 2024-04-21T08:48:25.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.386+0000 7f097b03d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f09701afbb0 con 0x55be19e0ea00 2024-04-21T08:48:25.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.386+0000 7f09759eb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e05e80 0x55be19e06ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.386+0000 7f09759eb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e05e80 0x55be19e06ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37004/0 (socket says 172.21.15.43:37004) 2024-04-21T08:48:25.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.386+0000 7f09759eb640 1 -- 172.21.15.43:0/2011624179 learned_addr learned my addr 172.21.15.43:0/2011624179 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:25.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.386+0000 7f09761ec640 1 --2- 172.21.15.43:0/2011624179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be19e054d0 0x55be19e1a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.386+0000 7f09759eb640 1 -- 172.21.15.43:0/2011624179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be19e0ea00 msgr2=0x55be19e0dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:25.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.386+0000 7f09769ed640 1 --2- 172.21.15.43:0/2011624179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be19e0ea00 0x55be19e0dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.387+0000 7f09759eb640 1 --2- 172.21.15.43:0/2011624179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be19e0ea00 0x55be19e0dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.387+0000 7f09759eb640 1 -- 172.21.15.43:0/2011624179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be19e054d0 msgr2=0x55be19e1a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:25.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.387+0000 7f09759eb640 1 --2- 172.21.15.43:0/2011624179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be19e054d0 0x55be19e1a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.387+0000 7f09759eb640 1 -- 172.21.15.43:0/2011624179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be19e05c20 con 0x55be19e05e80 2024-04-21T08:48:25.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.387+0000 7f09759eb640 1 --2- 172.21.15.43:0/2011624179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e05e80 0x55be19e06ba0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f0960009900 tx=0x7f09600052c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2bf5d1997f8c101 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:25.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.387+0000 7f095bfff640 1 -- 172.21.15.43:0/2011624179 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0960045040 con 0x55be19e05e80 2024-04-21T08:48:25.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.387+0000 7f095bfff640 1 -- 172.21.15.43:0/2011624179 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0960050070 con 0x55be19e05e80 2024-04-21T08:48:25.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.387+0000 7f095bfff640 1 -- 172.21.15.43:0/2011624179 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0960003830 con 0x55be19e05e80 2024-04-21T08:48:25.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.387+0000 7f097b03d740 1 -- 172.21.15.43:0/2011624179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e05e80 msgr2=0x55be19e06ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.387+0000 7f097b03d740 1 --2- 172.21.15.43:0/2011624179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e05e80 0x55be19e06ba0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f0960009900 tx=0x7f09600052c0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.387+0000 7f097b03d740 1 -- 172.21.15.43:0/2011624179 shutdown_connections 2024-04-21T08:48:25.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.387+0000 7f097b03d740 1 --2- 172.21.15.43:0/2011624179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be19e0ea00 0x55be19e0dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.387+0000 7f097b03d740 1 --2- 172.21.15.43:0/2011624179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e05e80 0x55be19e06ba0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.387+0000 7f097b03d740 1 --2- 172.21.15.43:0/2011624179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be19e054d0 0x55be19e1a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.387+0000 7f097b03d740 1 -- 172.21.15.43:0/2011624179 >> 172.21.15.43:0/2011624179 conn(0x55be19dffba0 msgr2=0x55be19e01860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:25.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.387+0000 7f097b03d740 1 -- 172.21.15.43:0/2011624179 shutdown_connections 2024-04-21T08:48:25.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.387+0000 7f097b03d740 1 -- 172.21.15.43:0/2011624179 wait complete. 2024-04-21T08:48:25.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.388+0000 7f097b03d740 1 -- start start 2024-04-21T08:48:25.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.388+0000 7f097b03d740 1 Processor -- start 2024-04-21T08:48:25.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.388+0000 7f097b03d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be19e054d0 0x55be19e17230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.388+0000 7f097b03d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e05e80 0x55be19e08100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.388+0000 7f097b03d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be19e0ea00 0x55be19e08820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.388+0000 7f097b03d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f09701b0950 con 0x55be19e05e80 2024-04-21T08:48:25.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.388+0000 7f097b03d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f09701b0280 con 0x55be19e0ea00 2024-04-21T08:48:25.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.388+0000 7f097b03d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f09701ad4e0 con 0x55be19e054d0 2024-04-21T08:48:25.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.388+0000 7f097b03d740 10 client.0 _ll_register_callbacks cb 0x7f096c09c280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:25.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.388+0000 7f09759eb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e05e80 0x55be19e08100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.388+0000 7f09759eb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e05e80 0x55be19e08100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37020/0 (socket says 172.21.15.43:37020) 2024-04-21T08:48:25.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.388+0000 7f09759eb640 1 -- 172.21.15.43:0/1330101848 learned_addr learned my addr 172.21.15.43:0/1330101848 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:25.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.388+0000 7f09761ec640 1 --2- 172.21.15.43:0/1330101848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be19e054d0 0x55be19e17230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.388+0000 7f09769ed640 1 --2- 172.21.15.43:0/1330101848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be19e0ea00 0x55be19e08820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.388+0000 7f09759eb640 1 -- 172.21.15.43:0/1330101848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be19e054d0 msgr2=0x55be19e17230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:25.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.388+0000 7f09759eb640 1 --2- 172.21.15.43:0/1330101848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be19e054d0 0x55be19e17230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.388+0000 7f09759eb640 1 -- 172.21.15.43:0/1330101848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be19e0ea00 msgr2=0x55be19e08820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:25.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.388+0000 7f09759eb640 1 --2- 172.21.15.43:0/1330101848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be19e0ea00 0x55be19e08820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.388+0000 7f09759eb640 1 -- 172.21.15.43:0/1330101848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be19e841f0 con 0x55be19e05e80 2024-04-21T08:48:25.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.389+0000 7f09759eb640 1 --2- 172.21.15.43:0/1330101848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e05e80 0x55be19e08100 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f0960005bd0 tx=0x7f0960005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ae0273919083f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:25.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.389+0000 7f0959ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:25.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.389+0000 7f0959ffb640 1 -- 172.21.15.43:0/1330101848 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0960060060 con 0x55be19e05e80 2024-04-21T08:48:25.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.389+0000 7f0959ffb640 1 -- 172.21.15.43:0/1330101848 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f09600040c0 con 0x55be19e05e80 2024-04-21T08:48:25.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.389+0000 7f097b03d740 1 -- 172.21.15.43:0/1330101848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0960050070 con 0x55be19e05e80 2024-04-21T08:48:25.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.389+0000 7f0959ffb640 1 -- 172.21.15.43:0/1330101848 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0960045040 con 0x55be19e05e80 2024-04-21T08:48:25.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.389+0000 7f097b03d740 10 client.20100 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:25.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.389+0000 7f097b03d740 1 -- 172.21.15.43:0/1330101848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55be19e85310 con 0x55be19e05e80 2024-04-21T08:48:25.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.389+0000 7f0959ffb640 1 -- 172.21.15.43:0/1330101848 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0960050070 con 0x55be19e05e80 2024-04-21T08:48:25.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.389+0000 7f0959ffb640 1 -- 172.21.15.43:0/1330101848 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f096005a070 con 0x55be19e05e80 2024-04-21T08:48:25.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.389+0000 7f097b03d740 10 client.20100 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:25.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.389+0000 7f097b03d740 10 client.20100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:25.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.389+0000 7f097b03d740 20 client.20100 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:25.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.389+0000 7f097b03d740 1 --2- 172.21.15.43:0/1330101848 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55be19e85630 0x55be19e87a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:25.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.389+0000 7f097b03d740 4 client.20100 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:25.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.389+0000 7f097b03d740 1 -- 172.21.15.43:0/1330101848 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be19d77c10 con 0x55be19e85630 2024-04-21T08:48:25.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.390+0000 7f09761ec640 1 --2- 172.21.15.43:0/1330101848 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55be19e85630 0x55be19e87a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:25.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.390+0000 7f0959ffb640 10 client.20100 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:25.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.390+0000 7f09761ec640 1 --2- 172.21.15.43:0/1330101848 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55be19e85630 0x55be19e87a20 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbb68264a77f3487 server_cookie=e89453c4a1c18501 in_seq=0 out_seq=0 2024-04-21T08:48:25.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.390+0000 7f0959ffb640 1 -- 172.21.15.43:0/1330101848 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be19d77c10 con 0x55be19e85630 2024-04-21T08:48:25.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.390+0000 7f0959ffb640 10 client.20100 handle_command_reply: tid=0 2024-04-21T08:48:25.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.390+0000 7f097b03d740 1 client.20100 shutdown 2024-04-21T08:48:25.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.390+0000 7f097b03d740 1 -- 172.21.15.43:0/1330101848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e05e80 msgr2=0x55be19e08100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.390+0000 7f097b03d740 1 --2- 172.21.15.43:0/1330101848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e05e80 0x55be19e08100 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f0960005bd0 tx=0x7f0960005fd0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.390+0000 7f097b03d740 1 -- 172.21.15.43:0/1330101848 shutdown_connections 2024-04-21T08:48:25.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.390+0000 7f097b03d740 1 --2- 172.21.15.43:0/1330101848 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55be19e85630 0x55be19e87a20 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.390+0000 7f097b03d740 1 --2- 172.21.15.43:0/1330101848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be19e0ea00 0x55be19e08820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.390+0000 7f097b03d740 1 --2- 172.21.15.43:0/1330101848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be19e05e80 0x55be19e08100 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.390+0000 7f097b03d740 1 --2- 172.21.15.43:0/1330101848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be19e054d0 0x55be19e17230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.390+0000 7f0959ffb640 0 client.20100 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:25.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.390+0000 7f097b03d740 1 -- 172.21.15.43:0/1330101848 >> 172.21.15.43:0/1330101848 conn(0x55be19dffba0 msgr2=0x55be19dfec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:25.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.390+0000 7f097b03d740 1 -- 172.21.15.43:0/1330101848 shutdown_connections 2024-04-21T08:48:25.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.390+0000 7f097b03d740 1 -- 172.21.15.43:0/1330101848 wait complete. 2024-04-21T08:48:25.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.391+0000 7f097b03d740 20 client.20100 trim_cache size 0 max 16384 2024-04-21T08:48:25.396 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:48:25.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.391+0000 7f0978450640 1 -- 172.21.15.43:0/4073095756 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f095c0a3d20 msgr2=0x7f095c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.391+0000 7f0978450640 1 --2- 172.21.15.43:0/4073095756 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f095c0a3d20 0x7f095c0a61e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0968009fd0 tx=0x7f0968009500 comp rx=0 tx=0).stop 2024-04-21T08:48:25.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.391+0000 7f0978450640 1 -- 172.21.15.43:0/4073095756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0970102190 msgr2=0x7f097019e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:25.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.391+0000 7f0978450640 1 --2- 172.21.15.43:0/4073095756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0970102190 0x7f097019e410 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f096c00d950 tx=0x7f096c00de10 comp rx=0 tx=0).stop 2024-04-21T08:48:25.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.391+0000 7f0978450640 1 -- 172.21.15.43:0/4073095756 shutdown_connections 2024-04-21T08:48:25.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.391+0000 7f0978450640 1 --2- 172.21.15.43:0/4073095756 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f095c0a3d20 0x7f095c0a61e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.391+0000 7f0978450640 1 --2- 172.21.15.43:0/4073095756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09701985c0 0x7f0970198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.391+0000 7f0978450640 1 --2- 172.21.15.43:0/4073095756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0970107260 0x7f097019e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.391+0000 7f0978450640 1 --2- 172.21.15.43:0/4073095756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0970102190 0x7f097019e410 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:25.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.391+0000 7f0978450640 1 -- 172.21.15.43:0/4073095756 >> 172.21.15.43:0/4073095756 conn(0x7f09700fd990 msgr2=0x7f09700fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:25.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.391+0000 7f0978450640 1 -- 172.21.15.43:0/4073095756 shutdown_connections 2024-04-21T08:48:25.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:25.391+0000 7f0978450640 1 -- 172.21.15.43:0/4073095756 wait complete. 2024-04-21T08:48:25.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:25 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:48:25.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:25 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:25.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:25 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:48:25.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:25 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:25.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:25 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:48:25.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:25 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/137622022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:25.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:25 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:48:25.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:25 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:25.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:25 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:48:25.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:25 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:25.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:25 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:48:25.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:25 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/137622022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:26.401 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:48:26.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:48:26.402 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:48:26.429 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:26 smithi043 ceph-mon[27978]: pgmap v295: 129 pgs: 129 active+clean; 3.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:48:26.445 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:26 smithi112 ceph-mon[31085]: pgmap v295: 129 pgs: 129 active+clean; 3.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:48:26.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:26 smithi175 ceph-mon[31920]: pgmap v295: 129 pgs: 129 active+clean; 3.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:48:26.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.524+0000 7f68a9806640 1 -- 172.21.15.43:0/907039185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68a4107270 msgr2=0x7f68a410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.524+0000 7f68a9806640 1 --2- 172.21.15.43:0/907039185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68a4107270 0x7f68a410ee40 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f689c009ca0 tx=0x7f689c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.524+0000 7f68a9806640 1 -- 172.21.15.43:0/907039185 shutdown_connections 2024-04-21T08:48:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.524+0000 7f68a9806640 1 --2- 172.21.15.43:0/907039185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68a4107270 0x7f68a410ee40 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.524+0000 7f68a9806640 1 --2- 172.21.15.43:0/907039185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a406b1f0 0x7f68a4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.524+0000 7f68a9806640 1 --2- 172.21.15.43:0/907039185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68a406a840 0x7f68a406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.524+0000 7f68a9806640 1 -- 172.21.15.43:0/907039185 >> 172.21.15.43:0/907039185 conn(0x7f68a4077eb0 msgr2=0x7f68a40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.524+0000 7f68a9806640 1 -- 172.21.15.43:0/907039185 shutdown_connections 2024-04-21T08:48:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.524+0000 7f68a9806640 1 -- 172.21.15.43:0/907039185 wait complete. 2024-04-21T08:48:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.524+0000 7f68a9806640 1 Processor -- start 2024-04-21T08:48:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.525+0000 7f68a9806640 1 -- start start 2024-04-21T08:48:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.525+0000 7f68a9806640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a406a840 0x7f68a41986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.525+0000 7f68a9806640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68a4107270 0x7f68a4198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.525+0000 7f68a9806640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68a419cf70 0x7f68a419d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.525+0000 7f68a9806640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41121d0 con 0x7f68a4107270 2024-04-21T08:48:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.525+0000 7f68a9806640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f68a4112050 con 0x7f68a406a840 2024-04-21T08:48:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.525+0000 7f68a9806640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68a4112350 con 0x7f68a419cf70 2024-04-21T08:48:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.525+0000 7f689bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68a4107270 0x7f68a4198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.525+0000 7f689bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68a4107270 0x7f68a4198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37034/0 (socket says 172.21.15.43:37034) 2024-04-21T08:48:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.525+0000 7f689bfff640 1 -- 172.21.15.43:0/2248352765 learned_addr learned my addr 172.21.15.43:0/2248352765 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.525+0000 7f68a37fe640 1 --2- 172.21.15.43:0/2248352765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68a419cf70 0x7f68a419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.525+0000 7f68a2ffd640 1 --2- 172.21.15.43:0/2248352765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a406a840 0x7f68a41986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.525+0000 7f689bfff640 1 -- 172.21.15.43:0/2248352765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68a419cf70 msgr2=0x7f68a419d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.525+0000 7f689bfff640 1 --2- 172.21.15.43:0/2248352765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68a419cf70 0x7f68a419d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.525+0000 7f689bfff640 1 -- 172.21.15.43:0/2248352765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a406a840 msgr2=0x7f68a41986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.525+0000 7f689bfff640 1 --2- 172.21.15.43:0/2248352765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a406a840 0x7f68a41986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.525+0000 7f689bfff640 1 -- 172.21.15.43:0/2248352765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a419da60 con 0x7f68a4107270 2024-04-21T08:48:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.525+0000 7f689bfff640 1 --2- 172.21.15.43:0/2248352765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68a4107270 0x7f68a4198bf0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f688c00d770 tx=0x7f688c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.526+0000 7f68a8804640 1 -- 172.21.15.43:0/2248352765 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c017070 con 0x7f68a4107270 2024-04-21T08:48:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.526+0000 7f68a8804640 1 -- 172.21.15.43:0/2248352765 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f688c009c30 con 0x7f68a4107270 2024-04-21T08:48:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.526+0000 7f68a9806640 1 -- 172.21.15.43:0/2248352765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a419dd50 con 0x7f68a4107270 2024-04-21T08:48:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.526+0000 7f68a8804640 1 -- 172.21.15.43:0/2248352765 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c00f620 con 0x7f68a4107270 2024-04-21T08:48:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.526+0000 7f68a9806640 1 -- 172.21.15.43:0/2248352765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a41a5540 con 0x7f68a4107270 2024-04-21T08:48:26.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.528+0000 7f68a8804640 1 -- 172.21.15.43:0/2248352765 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f688c01a070 con 0x7f68a4107270 2024-04-21T08:48:26.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.528+0000 7f68ac3f5740 1 Processor -- start 2024-04-21T08:48:26.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.528+0000 7f68ac3f5740 1 -- start start 2024-04-21T08:48:26.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.528+0000 7f68ac3f5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8186b58e0 0x55d8186b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.528+0000 7f68ac3f5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8186b6290 0x55d8186be840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.528+0000 7f68a8804640 1 --2- 172.21.15.43:0/2248352765 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f68900a3d20 0x7f68900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.528+0000 7f68ac3f5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186bee10 0x55d8186c1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.528+0000 7f68ac3f5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d818619f00 con 0x55d8186bee10 2024-04-21T08:48:26.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.528+0000 7f68ac3f5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d8185b7590 con 0x55d8186b6290 2024-04-21T08:48:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.528+0000 7f68ac3f5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d818615ec0 con 0x55d8186b58e0 2024-04-21T08:48:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.529+0000 7f68a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186bee10 0x55d8186c1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.529+0000 7f68a2ffd640 1 --2- 172.21.15.43:0/2248352765 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f68900a3d20 0x7f68900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.529+0000 7f68a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186bee10 0x55d8186c1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37040/0 (socket says 172.21.15.43:37040) 2024-04-21T08:48:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.529+0000 7f68a37fe640 1 -- 172.21.15.43:0/3061676840 learned_addr learned my addr 172.21.15.43:0/3061676840 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.529+0000 7f68a8804640 1 -- 172.21.15.43:0/2248352765 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f688c0d0760 con 0x7f68a4107270 2024-04-21T08:48:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.529+0000 7f68a37fe640 1 --2- 172.21.15.43:0/3061676840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8186b58e0 0x55d8186b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.529+0000 7f689bfff640 1 --2- 172.21.15.43:0/3061676840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8186b6290 0x55d8186be840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.529+0000 7f68a37fe640 1 -- 172.21.15.43:0/3061676840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8186b58e0 msgr2=0x55d8186b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.529+0000 7f68a37fe640 1 --2- 172.21.15.43:0/3061676840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8186b58e0 0x55d8186b5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.529+0000 7f68a37fe640 1 -- 172.21.15.43:0/3061676840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8186b6290 msgr2=0x55d8186be840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.529+0000 7f68a37fe640 1 --2- 172.21.15.43:0/3061676840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8186b6290 0x55d8186be840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.529+0000 7f68a37fe640 1 -- 172.21.15.43:0/3061676840 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8183f47e0 con 0x55d8186bee10 2024-04-21T08:48:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.529+0000 7f68a2ffd640 1 --2- 172.21.15.43:0/2248352765 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f68900a3d20 0x7f68900a61e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6894005fd0 tx=0x7f6894005850 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:26.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.529+0000 7f68a37fe640 1 --2- 172.21.15.43:0/3061676840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186bee10 0x55d8186c1260 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f689c037040 tx=0x7f689c02e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdad82119db99ef5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:26.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.529+0000 7f6899ffb640 1 -- 172.21.15.43:0/3061676840 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c04c070 con 0x55d8186bee10 2024-04-21T08:48:26.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.529+0000 7f6899ffb640 1 -- 172.21.15.43:0/3061676840 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f689c032e80 con 0x55d8186bee10 2024-04-21T08:48:26.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.529+0000 7f6899ffb640 1 -- 172.21.15.43:0/3061676840 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c039040 con 0x55d8186bee10 2024-04-21T08:48:26.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.530+0000 7f68ac3f5740 1 -- 172.21.15.43:0/3061676840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186bee10 msgr2=0x55d8186c1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:26.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.530+0000 7f68ac3f5740 1 --2- 172.21.15.43:0/3061676840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186bee10 0x55d8186c1260 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f689c037040 tx=0x7f689c02e040 comp rx=0 tx=0).stop 2024-04-21T08:48:26.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.530+0000 7f68ac3f5740 1 -- 172.21.15.43:0/3061676840 shutdown_connections 2024-04-21T08:48:26.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.530+0000 7f68ac3f5740 1 --2- 172.21.15.43:0/3061676840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186bee10 0x55d8186c1260 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.530+0000 7f68ac3f5740 1 --2- 172.21.15.43:0/3061676840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8186b6290 0x55d8186be840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.530+0000 7f68ac3f5740 1 --2- 172.21.15.43:0/3061676840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8186b58e0 0x55d8186b5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.530+0000 7f68ac3f5740 1 -- 172.21.15.43:0/3061676840 >> 172.21.15.43:0/3061676840 conn(0x55d8186affb0 msgr2=0x55d8186b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:26.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.530+0000 7f68ac3f5740 1 -- 172.21.15.43:0/3061676840 shutdown_connections 2024-04-21T08:48:26.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.530+0000 7f68ac3f5740 1 -- 172.21.15.43:0/3061676840 wait complete. 2024-04-21T08:48:26.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.530+0000 7f68ac3f5740 1 -- start start 2024-04-21T08:48:26.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.531+0000 7f68ac3f5740 1 Processor -- start 2024-04-21T08:48:26.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.531+0000 7f68ac3f5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186b58e0 0x55d8186bbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.531+0000 7f68ac3f5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8186b6290 0x55d8186bc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.531+0000 7f68ac3f5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8186bee10 0x55d8186bc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.531+0000 7f68ac3f5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d8186c1d20 con 0x55d8186b58e0 2024-04-21T08:48:26.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.531+0000 7f68ac3f5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d8185d3580 con 0x55d8186b6290 2024-04-21T08:48:26.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.531+0000 7f68ac3f5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d8186c1ea0 con 0x55d8186bee10 2024-04-21T08:48:26.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.531+0000 7f68ac3f5740 10 client.0 _ll_register_callbacks cb 0x55d8185d9650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:26.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.531+0000 7f68a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186b58e0 0x55d8186bbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.531+0000 7f68a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186b58e0 0x55d8186bbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37052/0 (socket says 172.21.15.43:37052) 2024-04-21T08:48:26.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.531+0000 7f68a2ffd640 1 -- 172.21.15.43:0/2411301825 learned_addr learned my addr 172.21.15.43:0/2411301825 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:26.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.531+0000 7f68a37fe640 1 --2- 172.21.15.43:0/2411301825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8186bee10 0x55d8186bc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.531+0000 7f689bfff640 1 --2- 172.21.15.43:0/2411301825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8186b6290 0x55d8186bc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.531+0000 7f68a2ffd640 1 -- 172.21.15.43:0/2411301825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8186bee10 msgr2=0x55d8186bc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:26.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.531+0000 7f68a2ffd640 1 --2- 172.21.15.43:0/2411301825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8186bee10 0x55d8186bc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.531+0000 7f68a2ffd640 1 -- 172.21.15.43:0/2411301825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8186b6290 msgr2=0x55d8186bc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:26.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.531+0000 7f68a2ffd640 1 --2- 172.21.15.43:0/2411301825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8186b6290 0x55d8186bc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.531+0000 7f68a2ffd640 1 -- 172.21.15.43:0/2411301825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d818454f70 con 0x55d8186b58e0 2024-04-21T08:48:26.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.532+0000 7f68a2ffd640 1 --2- 172.21.15.43:0/2411301825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186b58e0 0x55d8186bbe40 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f689400b9e0 tx=0x7f68940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766816647d8da840 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:26.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.532+0000 7f6882ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:26.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.532+0000 7f6882ffd640 1 -- 172.21.15.43:0/2411301825 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6894022070 con 0x55d8186b58e0 2024-04-21T08:48:26.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.532+0000 7f6882ffd640 1 -- 172.21.15.43:0/2411301825 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f689401e860 con 0x55d8186b58e0 2024-04-21T08:48:26.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.532+0000 7f68ac3f5740 1 -- 172.21.15.43:0/2411301825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f689c032e80 con 0x55d8186b58e0 2024-04-21T08:48:26.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.532+0000 7f6882ffd640 1 -- 172.21.15.43:0/2411301825 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f689402e050 con 0x55d8186b58e0 2024-04-21T08:48:26.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.532+0000 7f68ac3f5740 10 client.20112 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:26.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.532+0000 7f68ac3f5740 1 -- 172.21.15.43:0/2411301825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d8182ec170 con 0x55d8186b58e0 2024-04-21T08:48:26.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.532+0000 7f6882ffd640 1 -- 172.21.15.43:0/2411301825 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f689c032e80 con 0x55d8186b58e0 2024-04-21T08:48:26.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.532+0000 7f6882ffd640 1 -- 172.21.15.43:0/2411301825 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6894010a90 con 0x55d8186b58e0 2024-04-21T08:48:26.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.532+0000 7f68ac3f5740 10 client.20112 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:26.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.532+0000 7f68ac3f5740 10 client.20112 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:26.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.533+0000 7f68ac3f5740 20 client.20112 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:26.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.533+0000 7f68ac3f5740 1 --2- 172.21.15.43:0/2411301825 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d8186cf330 0x55d8186d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.533+0000 7f68ac3f5740 4 client.20112 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:26.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.533+0000 7f68ac3f5740 1 -- 172.21.15.43:0/2411301825 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d818627920 con 0x55d8186cf330 2024-04-21T08:48:26.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.533+0000 7f689bfff640 1 --2- 172.21.15.43:0/2411301825 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d8186cf330 0x55d8186d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.533+0000 7f6882ffd640 10 client.20112 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:26.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.533+0000 7f689bfff640 1 --2- 172.21.15.43:0/2411301825 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d8186cf330 0x55d8186d1780 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef6030baeac18ebb server_cookie=6329bb3104dbdd20 in_seq=0 out_seq=0 2024-04-21T08:48:26.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.534+0000 7f6882ffd640 1 -- 172.21.15.43:0/2411301825 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d818627920 con 0x55d8186cf330 2024-04-21T08:48:26.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.534+0000 7f6882ffd640 10 client.20112 handle_command_reply: tid=0 2024-04-21T08:48:26.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.534+0000 7f68ac3f5740 1 client.20112 shutdown 2024-04-21T08:48:26.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.534+0000 7f68ac3f5740 1 -- 172.21.15.43:0/2411301825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186b58e0 msgr2=0x55d8186bbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:26.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.534+0000 7f68ac3f5740 1 --2- 172.21.15.43:0/2411301825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186b58e0 0x55d8186bbe40 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f689400b9e0 tx=0x7f68940046c0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.534+0000 7f68ac3f5740 1 -- 172.21.15.43:0/2411301825 shutdown_connections 2024-04-21T08:48:26.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.534+0000 7f68ac3f5740 1 --2- 172.21.15.43:0/2411301825 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d8186cf330 0x55d8186d1780 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.534+0000 7f68ac3f5740 1 --2- 172.21.15.43:0/2411301825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8186bee10 0x55d8186bc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.534+0000 7f6882ffd640 0 client.20112 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:26.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.534+0000 7f68ac3f5740 1 --2- 172.21.15.43:0/2411301825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8186b6290 0x55d8186bc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.534+0000 7f68ac3f5740 1 --2- 172.21.15.43:0/2411301825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186b58e0 0x55d8186bbe40 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.534+0000 7f68ac3f5740 1 -- 172.21.15.43:0/2411301825 >> 172.21.15.43:0/2411301825 conn(0x55d8186affb0 msgr2=0x55d8186b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:26.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.534+0000 7f68ac3f5740 1 -- 172.21.15.43:0/2411301825 shutdown_connections 2024-04-21T08:48:26.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.534+0000 7f68ac3f5740 1 -- 172.21.15.43:0/2411301825 wait complete. 2024-04-21T08:48:26.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.534+0000 7f68ac3f5740 20 client.20112 trim_cache size 0 max 16384 2024-04-21T08:48:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.552+0000 7f68ac3f5740 1 Processor -- start 2024-04-21T08:48:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.552+0000 7f68ac3f5740 1 -- start start 2024-04-21T08:48:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.552+0000 7f68ac3f5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186affb0 0x55d8186cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.552+0000 7f68ac3f5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8186b58e0 0x55d8186d7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.552+0000 7f68ac3f5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8186b6290 0x55d8186d7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.552+0000 7f68ac3f5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41ad300 con 0x55d8186affb0 2024-04-21T08:48:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.552+0000 7f68ac3f5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41aee40 con 0x55d8186b6290 2024-04-21T08:48:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.552+0000 7f68ac3f5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41ad9d0 con 0x55d8186b58e0 2024-04-21T08:48:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.552+0000 7f68a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186affb0 0x55d8186cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.552+0000 7f68a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186affb0 0x55d8186cabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37058/0 (socket says 172.21.15.43:37058) 2024-04-21T08:48:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.552+0000 7f68a2ffd640 1 -- 172.21.15.43:0/887835445 learned_addr learned my addr 172.21.15.43:0/887835445 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.553+0000 7f68a2ffd640 1 -- 172.21.15.43:0/887835445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8186b58e0 msgr2=0x55d8186d7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.553+0000 7f68a2ffd640 1 --2- 172.21.15.43:0/887835445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8186b58e0 0x55d8186d7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.553+0000 7f68a2ffd640 1 -- 172.21.15.43:0/887835445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8186b6290 msgr2=0x55d8186d7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.553+0000 7f68a2ffd640 1 --2- 172.21.15.43:0/887835445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8186b6290 0x55d8186d7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.553+0000 7f68a2ffd640 1 -- 172.21.15.43:0/887835445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8186b4b20 con 0x55d8186affb0 2024-04-21T08:48:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.553+0000 7f68a2ffd640 1 --2- 172.21.15.43:0/887835445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186affb0 0x55d8186cabe0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f689400fd50 tx=0x7f689400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8bc529b64b8d1d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.553+0000 7f68997fa640 1 -- 172.21.15.43:0/887835445 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6894036030 con 0x55d8186affb0 2024-04-21T08:48:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.553+0000 7f68997fa640 1 -- 172.21.15.43:0/887835445 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f689400f840 con 0x55d8186affb0 2024-04-21T08:48:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.553+0000 7f68997fa640 1 -- 172.21.15.43:0/887835445 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6894022050 con 0x55d8186affb0 2024-04-21T08:48:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.554+0000 7f68ac3f5740 1 -- 172.21.15.43:0/887835445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186affb0 msgr2=0x55d8186cabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.554+0000 7f68ac3f5740 1 --2- 172.21.15.43:0/887835445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186affb0 0x55d8186cabe0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f689400fd50 tx=0x7f689400ba50 comp rx=0 tx=0).stop 2024-04-21T08:48:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.554+0000 7f68ac3f5740 1 -- 172.21.15.43:0/887835445 shutdown_connections 2024-04-21T08:48:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.554+0000 7f68ac3f5740 1 --2- 172.21.15.43:0/887835445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8186b6290 0x55d8186d7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.554+0000 7f68ac3f5740 1 --2- 172.21.15.43:0/887835445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8186b58e0 0x55d8186d7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.554+0000 7f68ac3f5740 1 --2- 172.21.15.43:0/887835445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186affb0 0x55d8186cabe0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.554+0000 7f68ac3f5740 1 -- 172.21.15.43:0/887835445 >> 172.21.15.43:0/887835445 conn(0x55d8186b69e0 msgr2=0x55d8186b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.554+0000 7f68ac3f5740 1 -- 172.21.15.43:0/887835445 shutdown_connections 2024-04-21T08:48:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.554+0000 7f68ac3f5740 1 -- 172.21.15.43:0/887835445 wait complete. 2024-04-21T08:48:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.554+0000 7f68ac3f5740 1 -- start start 2024-04-21T08:48:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.554+0000 7f68ac3f5740 1 Processor -- start 2024-04-21T08:48:26.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.554+0000 7f68ac3f5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8186affb0 0x55d8186c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.555+0000 7f68ac3f5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8186b58e0 0x55d8186c3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.555+0000 7f68ac3f5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186b6290 0x55d8186c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.555+0000 7f68ac3f5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41ae770 con 0x55d8186b6290 2024-04-21T08:48:26.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.555+0000 7f68ac3f5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41ae0a0 con 0x55d8186b58e0 2024-04-21T08:48:26.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.555+0000 7f68ac3f5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41ab300 con 0x55d8186affb0 2024-04-21T08:48:26.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.555+0000 7f68ac3f5740 10 client.0 _ll_register_callbacks cb 0x7f6894015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:26.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.555+0000 7f68a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186b6290 0x55d8186c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.555+0000 7f68a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186b6290 0x55d8186c3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37062/0 (socket says 172.21.15.43:37062) 2024-04-21T08:48:26.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.555+0000 7f68a37fe640 1 -- 172.21.15.43:0/2595405233 learned_addr learned my addr 172.21.15.43:0/2595405233 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:26.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.555+0000 7f68a2ffd640 1 --2- 172.21.15.43:0/2595405233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8186affb0 0x55d8186c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.555+0000 7f689bfff640 1 --2- 172.21.15.43:0/2595405233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8186b58e0 0x55d8186c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.555+0000 7f68a37fe640 1 -- 172.21.15.43:0/2595405233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8186affb0 msgr2=0x55d8186c72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:26.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.555+0000 7f68a37fe640 1 --2- 172.21.15.43:0/2595405233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8186affb0 0x55d8186c72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.555+0000 7f68a37fe640 1 -- 172.21.15.43:0/2595405233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8186b58e0 msgr2=0x55d8186c3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:26.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.555+0000 7f68a37fe640 1 --2- 172.21.15.43:0/2595405233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8186b58e0 0x55d8186c3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.555+0000 7f68a37fe640 1 -- 172.21.15.43:0/2595405233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8186cdf80 con 0x55d8186b6290 2024-04-21T08:48:26.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.555+0000 7f68a2ffd640 1 --2- 172.21.15.43:0/2595405233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8186affb0 0x55d8186c72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:26.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.555+0000 7f689bfff640 1 --2- 172.21.15.43:0/2595405233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8186b58e0 0x55d8186c3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:26.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.555+0000 7f68a37fe640 1 --2- 172.21.15.43:0/2595405233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186b6290 0x55d8186c3b90 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f689c009ca0 tx=0x7f689c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e434ce28fd0086 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:26.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.555+0000 7f6882ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:26.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.555+0000 7f6882ffd640 1 -- 172.21.15.43:0/2595405233 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c04c070 con 0x55d8186b6290 2024-04-21T08:48:26.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.555+0000 7f6882ffd640 1 -- 172.21.15.43:0/2595405233 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f689c0324c0 con 0x55d8186b6290 2024-04-21T08:48:26.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.555+0000 7f68ac3f5740 1 -- 172.21.15.43:0/2595405233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f689400f840 con 0x55d8186b6290 2024-04-21T08:48:26.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.556+0000 7f6882ffd640 1 -- 172.21.15.43:0/2595405233 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f689c05e050 con 0x55d8186b6290 2024-04-21T08:48:26.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.556+0000 7f68ac3f5740 10 client.20118 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:26.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.556+0000 7f68ac3f5740 1 -- 172.21.15.43:0/2595405233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d818736550 con 0x55d8186b6290 2024-04-21T08:48:26.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.556+0000 7f6882ffd640 1 -- 172.21.15.43:0/2595405233 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f689400f840 con 0x55d8186b6290 2024-04-21T08:48:26.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.556+0000 7f6882ffd640 1 -- 172.21.15.43:0/2595405233 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f689c059030 con 0x55d8186b6290 2024-04-21T08:48:26.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.556+0000 7f68ac3f5740 10 client.20118 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:26.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.556+0000 7f68ac3f5740 10 client.20118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:26.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.556+0000 7f68ac3f5740 20 client.20118 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:26.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.556+0000 7f68ac3f5740 1 --2- 172.21.15.43:0/2595405233 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d8186bee10 0x55d818738830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.556+0000 7f68ac3f5740 4 client.20118 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:48:26.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.556+0000 7f68ac3f5740 1 -- 172.21.15.43:0/2595405233 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d818621790 con 0x55d8186bee10 2024-04-21T08:48:26.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.556+0000 7f68a2ffd640 1 --2- 172.21.15.43:0/2595405233 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d8186bee10 0x55d818738830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.557+0000 7f68a2ffd640 1 --2- 172.21.15.43:0/2595405233 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d8186bee10 0x55d818738830 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=478eaa53306704fd server_cookie=44e4609e477614ef in_seq=0 out_seq=0 2024-04-21T08:48:26.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.557+0000 7f6882ffd640 10 client.20118 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:26.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.558+0000 7f6882ffd640 1 -- 172.21.15.43:0/2595405233 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d818621790 con 0x55d8186bee10 2024-04-21T08:48:26.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.558+0000 7f6882ffd640 10 client.20118 handle_command_reply: tid=0 2024-04-21T08:48:26.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.558+0000 7f68ac3f5740 1 client.20118 shutdown 2024-04-21T08:48:26.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.558+0000 7f68ac3f5740 1 -- 172.21.15.43:0/2595405233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186b6290 msgr2=0x55d8186c3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:26.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.558+0000 7f68ac3f5740 1 --2- 172.21.15.43:0/2595405233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186b6290 0x55d8186c3b90 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f689c009ca0 tx=0x7f689c0095c0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.558+0000 7f68ac3f5740 1 -- 172.21.15.43:0/2595405233 shutdown_connections 2024-04-21T08:48:26.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.558+0000 7f68ac3f5740 1 --2- 172.21.15.43:0/2595405233 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d8186bee10 0x55d818738830 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.558+0000 7f68ac3f5740 1 --2- 172.21.15.43:0/2595405233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8186b6290 0x55d8186c3b90 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.558+0000 7f68ac3f5740 1 --2- 172.21.15.43:0/2595405233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8186b58e0 0x55d8186c3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.558+0000 7f68ac3f5740 1 --2- 172.21.15.43:0/2595405233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d8186affb0 0x55d8186c72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.558+0000 7f68ac3f5740 1 -- 172.21.15.43:0/2595405233 >> 172.21.15.43:0/2595405233 conn(0x55d8186b69e0 msgr2=0x55d8186bbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:26.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.558+0000 7f6882ffd640 0 client.20118 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:26.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.558+0000 7f68ac3f5740 1 -- 172.21.15.43:0/2595405233 shutdown_connections 2024-04-21T08:48:26.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.558+0000 7f68ac3f5740 1 -- 172.21.15.43:0/2595405233 wait complete. 2024-04-21T08:48:26.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.558+0000 7f68ac3f5740 20 client.20118 trim_cache size 0 max 16384 2024-04-21T08:48:26.563 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:48:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:48:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "87ced35a-56f0-4c76-9063-7c81399b281e", 2024-04-21T08:48:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:48:26.564 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:48:26.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.559+0000 7f68a9806640 1 -- 172.21.15.43:0/2248352765 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f68900a3d20 msgr2=0x7f68900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:26.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.559+0000 7f68a9806640 1 --2- 172.21.15.43:0/2248352765 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f68900a3d20 0x7f68900a61e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6894005fd0 tx=0x7f6894005850 comp rx=0 tx=0).stop 2024-04-21T08:48:26.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.559+0000 7f68a9806640 1 -- 172.21.15.43:0/2248352765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68a4107270 msgr2=0x7f68a4198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:26.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.559+0000 7f68a9806640 1 --2- 172.21.15.43:0/2248352765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68a4107270 0x7f68a4198bf0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f688c00d770 tx=0x7f688c00dc30 comp rx=0 tx=0).stop 2024-04-21T08:48:26.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.559+0000 7f68a9806640 1 -- 172.21.15.43:0/2248352765 shutdown_connections 2024-04-21T08:48:26.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.559+0000 7f68a9806640 1 --2- 172.21.15.43:0/2248352765 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f68900a3d20 0x7f68900a61e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.559+0000 7f68a9806640 1 --2- 172.21.15.43:0/2248352765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68a419cf70 0x7f68a419d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.559+0000 7f68a9806640 1 --2- 172.21.15.43:0/2248352765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68a4107270 0x7f68a4198bf0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.559+0000 7f68a9806640 1 --2- 172.21.15.43:0/2248352765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a406a840 0x7f68a41986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.559+0000 7f68a9806640 1 -- 172.21.15.43:0/2248352765 >> 172.21.15.43:0/2248352765 conn(0x7f68a4077eb0 msgr2=0x7f68a40fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:26.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.559+0000 7f68a9806640 1 -- 172.21.15.43:0/2248352765 shutdown_connections 2024-04-21T08:48:26.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.559+0000 7f68a9806640 1 -- 172.21.15.43:0/2248352765 wait complete. 2024-04-21T08:48:26.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:48:26.570 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:48:26.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.714+0000 7fc7772ac640 1 -- 172.21.15.43:0/632315758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc770103ae0 msgr2=0x7fc77010bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:26.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.714+0000 7fc7772ac640 1 --2- 172.21.15.43:0/632315758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc770103ae0 0x7fc77010bfc0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fc760009900 tx=0x7fc76002bd60 comp rx=0 tx=0).stop 2024-04-21T08:48:26.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.714+0000 7fc7772ac640 1 -- 172.21.15.43:0/632315758 shutdown_connections 2024-04-21T08:48:26.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.714+0000 7fc7772ac640 1 --2- 172.21.15.43:0/632315758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc77010c500 0x7fc77010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.714+0000 7fc7772ac640 1 --2- 172.21.15.43:0/632315758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc770103ae0 0x7fc77010bfc0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.714+0000 7fc7772ac640 1 --2- 172.21.15.43:0/632315758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7701031c0 0x7fc7701035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.714+0000 7fc7772ac640 1 -- 172.21.15.43:0/632315758 >> 172.21.15.43:0/632315758 conn(0x7fc770078000 msgr2=0x7fc770078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:26.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.715+0000 7fc7772ac640 1 -- 172.21.15.43:0/632315758 shutdown_connections 2024-04-21T08:48:26.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.715+0000 7fc7772ac640 1 -- 172.21.15.43:0/632315758 wait complete. 2024-04-21T08:48:26.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.715+0000 7fc7772ac640 1 Processor -- start 2024-04-21T08:48:26.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.715+0000 7fc7772ac640 1 -- start start 2024-04-21T08:48:26.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.715+0000 7fc7772ac640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7701031c0 0x7fc770198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.715+0000 7fc7772ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc770103ae0 0x7fc770198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.715+0000 7fc7772ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc77010c500 0x7fc77019cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.715+0000 7fc7772ac640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc7701121e0 con 0x7fc77010c500 2024-04-21T08:48:26.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.715+0000 7fc7772ac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc770112060 con 0x7fc770103ae0 2024-04-21T08:48:26.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.715+0000 7fc7772ac640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc770112360 con 0x7fc7701031c0 2024-04-21T08:48:26.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.715+0000 7fc775849640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc77010c500 0x7fc77019cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.715+0000 7fc775849640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc77010c500 0x7fc77019cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45010/0 (socket says 172.21.15.43:45010) 2024-04-21T08:48:26.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.715+0000 7fc775849640 1 -- 172.21.15.43:0/541463875 learned_addr learned my addr 172.21.15.43:0/541463875 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:26.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.716+0000 7fc775048640 1 --2- 172.21.15.43:0/541463875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7701031c0 0x7fc770198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.716+0000 7fc774847640 1 --2- 172.21.15.43:0/541463875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc770103ae0 0x7fc770198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.716+0000 7fc775849640 1 -- 172.21.15.43:0/541463875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7701031c0 msgr2=0x7fc770198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:26.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.716+0000 7fc775849640 1 --2- 172.21.15.43:0/541463875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7701031c0 0x7fc770198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.716+0000 7fc775849640 1 -- 172.21.15.43:0/541463875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc770103ae0 msgr2=0x7fc770198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:26.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.716+0000 7fc775849640 1 --2- 172.21.15.43:0/541463875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc770103ae0 0x7fc770198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.716+0000 7fc775849640 1 -- 172.21.15.43:0/541463875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77019d710 con 0x7fc77010c500 2024-04-21T08:48:26.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.716+0000 7fc774847640 1 --2- 172.21.15.43:0/541463875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc770103ae0 0x7fc770198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:26.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.716+0000 7fc775849640 1 --2- 172.21.15.43:0/541463875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc77010c500 0x7fc77019cfe0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fc76800c970 tx=0x7fc76800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:26.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.716+0000 7fc7667fc640 1 -- 172.21.15.43:0/541463875 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc768007bb0 con 0x7fc77010c500 2024-04-21T08:48:26.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.716+0000 7fc7667fc640 1 -- 172.21.15.43:0/541463875 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc768007d50 con 0x7fc77010c500 2024-04-21T08:48:26.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.716+0000 7fc7772ac640 1 -- 172.21.15.43:0/541463875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc77019da00 con 0x7fc77010c500 2024-04-21T08:48:26.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.716+0000 7fc7667fc640 1 -- 172.21.15.43:0/541463875 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc768018770 con 0x7fc77010c500 2024-04-21T08:48:26.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.716+0000 7fc7772ac640 1 -- 172.21.15.43:0/541463875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7701a5290 con 0x7fc77010c500 2024-04-21T08:48:26.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.719+0000 7fc7667fc640 1 -- 172.21.15.43:0/541463875 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc76800f990 con 0x7fc77010c500 2024-04-21T08:48:26.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.719+0000 7fc779f78740 1 Processor -- start 2024-04-21T08:48:26.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.719+0000 7fc779f78740 1 -- start start 2024-04-21T08:48:26.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.719+0000 7fc779f78740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeacf7df60 0x55eeacf7e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.719+0000 7fc779f78740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeacf7e910 0x55eeacf8b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.719+0000 7fc779f78740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf8be90 0x55eeacf8e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.719+0000 7fc779f78740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55eeacee31e0 con 0x55eeacf8be90 2024-04-21T08:48:26.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.719+0000 7fc779f78740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55eeace80390 con 0x55eeacf7df60 2024-04-21T08:48:26.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.719+0000 7fc779f78740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55eeacedf1a0 con 0x55eeacf7e910 2024-04-21T08:48:26.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.719+0000 7fc7667fc640 1 --2- 172.21.15.43:0/541463875 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc75c0a3da0 0x7fc75c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.720+0000 7fc774847640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf8be90 0x55eeacf8e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.720+0000 7fc774847640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeacf7df60 0x55eeacf7e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.720+0000 7fc775048640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeacf7e910 0x55eeacf8b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.720+0000 7fc775048640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeacf7e910 0x55eeacf8b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.43:38216/0 (socket says 172.21.15.43:38216) 2024-04-21T08:48:26.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.720+0000 7fc774847640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeacf7df60 0x55eeacf7e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:34572/0 (socket says 172.21.15.43:34572) 2024-04-21T08:48:26.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.720+0000 7fc775048640 1 -- 172.21.15.43:0/1299098327 learned_addr learned my addr 172.21.15.43:0/1299098327 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:26.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.720+0000 7fc775849640 1 --2- 172.21.15.43:0/541463875 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc75c0a3da0 0x7fc75c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.720+0000 7fc7667fc640 1 -- 172.21.15.43:0/541463875 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fc7680d0bf0 con 0x7fc77010c500 2024-04-21T08:48:26.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.720+0000 7fc775048640 1 -- 172.21.15.43:0/1299098327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeacf7df60 msgr2=0x55eeacf7e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:26.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.720+0000 7fc775048640 1 --2- 172.21.15.43:0/1299098327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeacf7df60 0x55eeacf7e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.720+0000 7fc775048640 1 -- 172.21.15.43:0/1299098327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf8be90 msgr2=0x55eeacf8e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:26.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.720+0000 7fc775048640 1 --2- 172.21.15.43:0/1299098327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf8be90 0x55eeacf8e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.720+0000 7fc775048640 1 -- 172.21.15.43:0/1299098327 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeaccbf7d0 con 0x55eeacf7e910 2024-04-21T08:48:26.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.720+0000 7fc775849640 1 --2- 172.21.15.43:0/541463875 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc75c0a3da0 0x7fc75c0a6260 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc76800afd0 tx=0x7fc768087040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:26.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.720+0000 7fc775048640 1 --2- 172.21.15.43:0/1299098327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeacf7e910 0x55eeacf8b950 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00af30 tx=0x7fc76c00f850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6249f649426a392a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:26.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.720+0000 7fc75b7fe640 1 -- 172.21.15.43:0/1299098327 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c010840 con 0x55eeacf7e910 2024-04-21T08:48:26.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.720+0000 7fc75b7fe640 1 -- 172.21.15.43:0/1299098327 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc76c018430 con 0x55eeacf7e910 2024-04-21T08:48:26.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.720+0000 7fc75b7fe640 1 -- 172.21.15.43:0/1299098327 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c017640 con 0x55eeacf7e910 2024-04-21T08:48:26.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.721+0000 7fc779f78740 1 -- 172.21.15.43:0/1299098327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeacf7e910 msgr2=0x55eeacf8b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:26.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.721+0000 7fc779f78740 1 --2- 172.21.15.43:0/1299098327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeacf7e910 0x55eeacf8b950 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00af30 tx=0x7fc76c00f850 comp rx=0 tx=0).stop 2024-04-21T08:48:26.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.721+0000 7fc779f78740 1 -- 172.21.15.43:0/1299098327 shutdown_connections 2024-04-21T08:48:26.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.721+0000 7fc779f78740 1 --2- 172.21.15.43:0/1299098327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf8be90 0x55eeacf8e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.721+0000 7fc779f78740 1 --2- 172.21.15.43:0/1299098327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeacf7e910 0x55eeacf8b950 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.721+0000 7fc779f78740 1 --2- 172.21.15.43:0/1299098327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeacf7df60 0x55eeacf7e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.721+0000 7fc779f78740 1 -- 172.21.15.43:0/1299098327 >> 172.21.15.43:0/1299098327 conn(0x55eeacf78cb0 msgr2=0x55eeacf79090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:26.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.721+0000 7fc779f78740 1 -- 172.21.15.43:0/1299098327 shutdown_connections 2024-04-21T08:48:26.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.721+0000 7fc779f78740 1 -- 172.21.15.43:0/1299098327 wait complete. 2024-04-21T08:48:26.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.721+0000 7fc779f78740 1 -- start start 2024-04-21T08:48:26.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.722+0000 7fc779f78740 1 Processor -- start 2024-04-21T08:48:26.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.722+0000 7fc779f78740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeacf7df60 0x55eeacf88e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.722+0000 7fc779f78740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeacf7e910 0x55eeacf89340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.722+0000 7fc779f78740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf8be90 0x55eeacf89880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.722+0000 7fc779f78740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55eeacf8ec50 con 0x55eeacf8be90 2024-04-21T08:48:26.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.722+0000 7fc779f78740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55eeace9c2f0 con 0x55eeacf7e910 2024-04-21T08:48:26.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.722+0000 7fc779f78740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55eeacf8edd0 con 0x55eeacf7df60 2024-04-21T08:48:26.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.722+0000 7fc779f78740 10 client.0 _ll_register_callbacks cb 0x55eeacea9580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:26.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.722+0000 7fc774847640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf8be90 0x55eeacf89880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.722+0000 7fc774847640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf8be90 0x55eeacf89880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45042/0 (socket says 172.21.15.43:45042) 2024-04-21T08:48:26.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.722+0000 7fc774847640 1 -- 172.21.15.43:0/3439804525 learned_addr learned my addr 172.21.15.43:0/3439804525 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:26.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.722+0000 7fc774847640 1 --2- 172.21.15.43:0/3439804525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeacf7df60 0x55eeacf88e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.722+0000 7fc775048640 1 --2- 172.21.15.43:0/3439804525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeacf7e910 0x55eeacf89340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.722+0000 7fc774847640 1 -- 172.21.15.43:0/3439804525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeacf7df60 msgr2=0x55eeacf88e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:26.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.722+0000 7fc774847640 1 --2- 172.21.15.43:0/3439804525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeacf7df60 0x55eeacf88e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.722+0000 7fc774847640 1 -- 172.21.15.43:0/3439804525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeacf7e910 msgr2=0x55eeacf89340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:26.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.722+0000 7fc774847640 1 --2- 172.21.15.43:0/3439804525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeacf7e910 0x55eeacf89340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.722+0000 7fc774847640 1 -- 172.21.15.43:0/3439804525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeacbf53f0 con 0x55eeacf8be90 2024-04-21T08:48:26.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.723+0000 7fc774847640 1 --2- 172.21.15.43:0/3439804525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf8be90 0x55eeacf89880 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fc760009900 tx=0x7fc76002bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9edfcee8f39578dc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:26.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.723+0000 7fc758ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:26.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.723+0000 7fc758ff9640 1 -- 172.21.15.43:0/3439804525 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760003af0 con 0x55eeacf8be90 2024-04-21T08:48:26.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.723+0000 7fc758ff9640 1 -- 172.21.15.43:0/3439804525 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fc760003c90 con 0x55eeacf8be90 2024-04-21T08:48:26.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.723+0000 7fc779f78740 1 -- 172.21.15.43:0/3439804525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc76c018430 con 0x55eeacf8be90 2024-04-21T08:48:26.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.723+0000 7fc758ff9640 1 -- 172.21.15.43:0/3439804525 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc760053050 con 0x55eeacf8be90 2024-04-21T08:48:26.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.723+0000 7fc779f78740 10 client.20130 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:26.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.723+0000 7fc779f78740 1 -- 172.21.15.43:0/3439804525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eeacbb50f0 con 0x55eeacf8be90 2024-04-21T08:48:26.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.723+0000 7fc758ff9640 1 -- 172.21.15.43:0/3439804525 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc76c018430 con 0x55eeacf8be90 2024-04-21T08:48:26.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.723+0000 7fc758ff9640 1 -- 172.21.15.43:0/3439804525 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc760040040 con 0x55eeacf8be90 2024-04-21T08:48:26.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.723+0000 7fc779f78740 10 client.20130 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:26.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.723+0000 7fc779f78740 10 client.20130 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:26.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.723+0000 7fc779f78740 20 client.20130 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:26.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.723+0000 7fc779f78740 1 --2- 172.21.15.43:0/3439804525 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55eeacf9bdf0 0x55eeacf9e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.723+0000 7fc779f78740 4 client.20130 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:26.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.723+0000 7fc779f78740 1 -- 172.21.15.43:0/3439804525 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eeacef0c00 con 0x55eeacf9bdf0 2024-04-21T08:48:26.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.724+0000 7fc775048640 1 --2- 172.21.15.43:0/3439804525 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55eeacf9bdf0 0x55eeacf9e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.724+0000 7fc775048640 1 --2- 172.21.15.43:0/3439804525 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55eeacf9bdf0 0x55eeacf9e240 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2cf274c91013c66 server_cookie=dcef8b9b71b26090 in_seq=0 out_seq=0 2024-04-21T08:48:26.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.724+0000 7fc758ff9640 10 client.20130 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:26.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.725+0000 7fc758ff9640 1 -- 172.21.15.43:0/3439804525 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eeacef0c00 con 0x55eeacf9bdf0 2024-04-21T08:48:26.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.725+0000 7fc758ff9640 10 client.20130 handle_command_reply: tid=0 2024-04-21T08:48:26.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.725+0000 7fc779f78740 1 client.20130 shutdown 2024-04-21T08:48:26.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.725+0000 7fc779f78740 1 -- 172.21.15.43:0/3439804525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf8be90 msgr2=0x55eeacf89880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:26.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.725+0000 7fc779f78740 1 --2- 172.21.15.43:0/3439804525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf8be90 0x55eeacf89880 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fc760009900 tx=0x7fc76002bd60 comp rx=0 tx=0).stop 2024-04-21T08:48:26.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.725+0000 7fc779f78740 1 -- 172.21.15.43:0/3439804525 shutdown_connections 2024-04-21T08:48:26.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.725+0000 7fc779f78740 1 --2- 172.21.15.43:0/3439804525 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55eeacf9bdf0 0x55eeacf9e240 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.725+0000 7fc779f78740 1 --2- 172.21.15.43:0/3439804525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf8be90 0x55eeacf89880 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.725+0000 7fc758ff9640 0 client.20130 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:26.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.725+0000 7fc779f78740 1 --2- 172.21.15.43:0/3439804525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeacf7e910 0x55eeacf89340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.725+0000 7fc779f78740 1 --2- 172.21.15.43:0/3439804525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeacf7df60 0x55eeacf88e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.725+0000 7fc779f78740 1 -- 172.21.15.43:0/3439804525 >> 172.21.15.43:0/3439804525 conn(0x55eeacf78cb0 msgr2=0x55eeacf7aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:26.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.725+0000 7fc779f78740 1 -- 172.21.15.43:0/3439804525 shutdown_connections 2024-04-21T08:48:26.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.725+0000 7fc779f78740 1 -- 172.21.15.43:0/3439804525 wait complete. 2024-04-21T08:48:26.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.725+0000 7fc779f78740 20 client.20130 trim_cache size 0 max 16384 2024-04-21T08:48:26.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.743+0000 7fc779f78740 1 Processor -- start 2024-04-21T08:48:26.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.743+0000 7fc779f78740 1 -- start start 2024-04-21T08:48:26.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.743+0000 7fc779f78740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeacf7df60 0x55eeacf8b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.743+0000 7fc779f78740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf7e910 0x55eeacf97b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.743+0000 7fc779f78740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeacf8be90 0x55eeacf87f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.743+0000 7fc779f78740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc7701ad2d0 con 0x55eeacf7e910 2024-04-21T08:48:26.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.743+0000 7fc779f78740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc7701aee10 con 0x55eeacf7df60 2024-04-21T08:48:26.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.743+0000 7fc779f78740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc7701ad9a0 con 0x55eeacf8be90 2024-04-21T08:48:26.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.743+0000 7fc775048640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf7e910 0x55eeacf97b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.743+0000 7fc775048640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf7e910 0x55eeacf97b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45048/0 (socket says 172.21.15.43:45048) 2024-04-21T08:48:26.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.743+0000 7fc775048640 1 -- 172.21.15.43:0/2911377195 learned_addr learned my addr 172.21.15.43:0/2911377195 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:26.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.743+0000 7fc774847640 1 --2- 172.21.15.43:0/2911377195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeacf7df60 0x55eeacf8b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.743+0000 7fc774847640 1 --2- 172.21.15.43:0/2911377195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeacf8be90 0x55eeacf87f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.744+0000 7fc775048640 1 -- 172.21.15.43:0/2911377195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeacf8be90 msgr2=0x55eeacf87f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:26.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.744+0000 7fc775048640 1 --2- 172.21.15.43:0/2911377195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeacf8be90 0x55eeacf87f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.744+0000 7fc775048640 1 -- 172.21.15.43:0/2911377195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeacf7df60 msgr2=0x55eeacf8b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:26.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.744+0000 7fc775048640 1 --2- 172.21.15.43:0/2911377195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeacf7df60 0x55eeacf8b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.744+0000 7fc775048640 1 -- 172.21.15.43:0/2911377195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeacf8e280 con 0x55eeacf7e910 2024-04-21T08:48:26.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.744+0000 7fc774847640 1 --2- 172.21.15.43:0/2911377195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeacf8be90 0x55eeacf87f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:26.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.744+0000 7fc775048640 1 --2- 172.21.15.43:0/2911377195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf7e910 0x55eeacf97b80 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00af00 tx=0x7fc76c00f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee80209c81c684e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:26.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.744+0000 7fc75affd640 1 -- 172.21.15.43:0/2911377195 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c018e60 con 0x55eeacf7e910 2024-04-21T08:48:26.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.744+0000 7fc75affd640 1 -- 172.21.15.43:0/2911377195 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc76c00a970 con 0x55eeacf7e910 2024-04-21T08:48:26.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.744+0000 7fc75affd640 1 -- 172.21.15.43:0/2911377195 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c02a2f0 con 0x55eeacf7e910 2024-04-21T08:48:26.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.744+0000 7fc779f78740 1 -- 172.21.15.43:0/2911377195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf7e910 msgr2=0x55eeacf97b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:26.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.744+0000 7fc779f78740 1 --2- 172.21.15.43:0/2911377195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf7e910 0x55eeacf97b80 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00af00 tx=0x7fc76c00f850 comp rx=0 tx=0).stop 2024-04-21T08:48:26.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.744+0000 7fc779f78740 1 -- 172.21.15.43:0/2911377195 shutdown_connections 2024-04-21T08:48:26.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.744+0000 7fc779f78740 1 --2- 172.21.15.43:0/2911377195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeacf8be90 0x55eeacf87f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.744+0000 7fc779f78740 1 --2- 172.21.15.43:0/2911377195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf7e910 0x55eeacf97b80 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.744+0000 7fc779f78740 1 --2- 172.21.15.43:0/2911377195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeacf7df60 0x55eeacf8b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.744+0000 7fc779f78740 1 -- 172.21.15.43:0/2911377195 >> 172.21.15.43:0/2911377195 conn(0x55eeacf78cb0 msgr2=0x55eeacf774e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:26.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.745+0000 7fc779f78740 1 -- 172.21.15.43:0/2911377195 shutdown_connections 2024-04-21T08:48:26.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.745+0000 7fc779f78740 1 -- 172.21.15.43:0/2911377195 wait complete. 2024-04-21T08:48:26.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.745+0000 7fc779f78740 1 -- start start 2024-04-21T08:48:26.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.745+0000 7fc779f78740 1 Processor -- start 2024-04-21T08:48:26.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.745+0000 7fc779f78740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeacf7df60 0x55eeacf94280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.745+0000 7fc779f78740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeacf7e910 0x55eeacf85690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.745+0000 7fc779f78740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf8be90 0x55eeacf85bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.745+0000 7fc779f78740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc7701ae740 con 0x55eeacf8be90 2024-04-21T08:48:26.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.745+0000 7fc779f78740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc7701ae070 con 0x55eeacf7df60 2024-04-21T08:48:26.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.745+0000 7fc779f78740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc7701ab2d0 con 0x55eeacf7e910 2024-04-21T08:48:26.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.745+0000 7fc779f78740 10 client.0 _ll_register_callbacks cb 0x7fc76002ddc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:26.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.745+0000 7fc774847640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf8be90 0x55eeacf85bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.745+0000 7fc774847640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf8be90 0x55eeacf85bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45050/0 (socket says 172.21.15.43:45050) 2024-04-21T08:48:26.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.745+0000 7fc774847640 1 -- 172.21.15.43:0/3254660986 learned_addr learned my addr 172.21.15.43:0/3254660986 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:26.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.745+0000 7fc775048640 1 --2- 172.21.15.43:0/3254660986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeacf7e910 0x55eeacf85690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.746+0000 7fc774847640 1 --2- 172.21.15.43:0/3254660986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeacf7df60 0x55eeacf94280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.746+0000 7fc774847640 1 -- 172.21.15.43:0/3254660986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeacf7e910 msgr2=0x55eeacf85690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:26.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.746+0000 7fc774847640 1 --2- 172.21.15.43:0/3254660986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeacf7e910 0x55eeacf85690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.746+0000 7fc774847640 1 -- 172.21.15.43:0/3254660986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeacf7df60 msgr2=0x55eeacf94280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:26.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.746+0000 7fc774847640 1 --2- 172.21.15.43:0/3254660986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeacf7df60 0x55eeacf94280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.746+0000 7fc774847640 1 -- 172.21.15.43:0/3254660986 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeacf86190 con 0x55eeacf8be90 2024-04-21T08:48:26.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.746+0000 7fc774847640 1 --2- 172.21.15.43:0/3254660986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf8be90 0x55eeacf85bd0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fc760001eb0 tx=0x7fc760004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8180017cefc573dd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:26.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.746+0000 7fc758ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:26.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.746+0000 7fc758ff9640 1 -- 172.21.15.43:0/3254660986 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76002c700 con 0x55eeacf8be90 2024-04-21T08:48:26.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.746+0000 7fc758ff9640 1 -- 172.21.15.43:0/3254660986 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fc760043450 con 0x55eeacf8be90 2024-04-21T08:48:26.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.746+0000 7fc779f78740 1 -- 172.21.15.43:0/3254660986 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc76c00a970 con 0x55eeacf8be90 2024-04-21T08:48:26.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.746+0000 7fc758ff9640 1 -- 172.21.15.43:0/3254660986 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc760064050 con 0x55eeacf8be90 2024-04-21T08:48:26.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.746+0000 7fc779f78740 10 client.20136 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:26.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.746+0000 7fc779f78740 1 -- 172.21.15.43:0/3254660986 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eead0020f0 con 0x55eeacf8be90 2024-04-21T08:48:26.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.746+0000 7fc758ff9640 1 -- 172.21.15.43:0/3254660986 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc76c00a970 con 0x55eeacf8be90 2024-04-21T08:48:26.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.747+0000 7fc758ff9640 1 -- 172.21.15.43:0/3254660986 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc760034a80 con 0x55eeacf8be90 2024-04-21T08:48:26.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.747+0000 7fc779f78740 10 client.20136 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:26.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.747+0000 7fc779f78740 10 client.20136 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:26.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.747+0000 7fc779f78740 20 client.20136 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:26.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.747+0000 7fc779f78740 1 --2- 172.21.15.43:0/3254660986 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55eead002840 0x55eead004c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:26.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.747+0000 7fc779f78740 4 client.20136 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:48:26.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.747+0000 7fc779f78740 1 -- 172.21.15.43:0/3254660986 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eeaceeaa70 con 0x55eead002840 2024-04-21T08:48:26.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.747+0000 7fc775048640 1 --2- 172.21.15.43:0/3254660986 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55eead002840 0x55eead004c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:26.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.747+0000 7fc758ff9640 10 client.20136 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:26.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.747+0000 7fc775048640 1 --2- 172.21.15.43:0/3254660986 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55eead002840 0x55eead004c60 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41863a14a086913c server_cookie=9f8962936b2a6853 in_seq=0 out_seq=0 2024-04-21T08:48:26.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.748+0000 7fc758ff9640 1 -- 172.21.15.43:0/3254660986 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55eeaceeaa70 con 0x55eead002840 2024-04-21T08:48:26.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.748+0000 7fc758ff9640 10 client.20136 handle_command_reply: tid=0 2024-04-21T08:48:26.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.748+0000 7fc779f78740 1 client.20136 shutdown 2024-04-21T08:48:26.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.748+0000 7fc779f78740 1 -- 172.21.15.43:0/3254660986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf8be90 msgr2=0x55eeacf85bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:26.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.748+0000 7fc779f78740 1 --2- 172.21.15.43:0/3254660986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf8be90 0x55eeacf85bd0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fc760001eb0 tx=0x7fc760004340 comp rx=0 tx=0).stop 2024-04-21T08:48:26.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.748+0000 7fc779f78740 1 -- 172.21.15.43:0/3254660986 shutdown_connections 2024-04-21T08:48:26.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.748+0000 7fc779f78740 1 --2- 172.21.15.43:0/3254660986 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55eead002840 0x55eead004c60 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.748+0000 7fc779f78740 1 --2- 172.21.15.43:0/3254660986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eeacf8be90 0x55eeacf85bd0 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.748+0000 7fc779f78740 1 --2- 172.21.15.43:0/3254660986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eeacf7e910 0x55eeacf85690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.748+0000 7fc779f78740 1 --2- 172.21.15.43:0/3254660986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eeacf7df60 0x55eeacf94280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.748+0000 7fc779f78740 1 -- 172.21.15.43:0/3254660986 >> 172.21.15.43:0/3254660986 conn(0x55eeacf78cb0 msgr2=0x55eeacf774e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:26.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.748+0000 7fc758ff9640 0 client.20136 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:26.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.748+0000 7fc779f78740 1 -- 172.21.15.43:0/3254660986 shutdown_connections 2024-04-21T08:48:26.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.748+0000 7fc779f78740 1 -- 172.21.15.43:0/3254660986 wait complete. 2024-04-21T08:48:26.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.748+0000 7fc779f78740 20 client.20136 trim_cache size 0 max 16384 2024-04-21T08:48:26.754 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:48:26.754 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:48:26.754 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-21T08:48:26.754 INFO:teuthology.orchestra.run.smithi043.stdout: "87ced35a-56f0-4c76-9063-7c81399b281e": { 2024-04-21T08:48:26.754 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-21T08:48:26.755 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "87ced35a-56f0-4c76-9063-7c81399b281e", 2024-04-21T08:48:26.755 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-21T08:48:26.755 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:48:26.755 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:48:26.755 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:48:26.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.749+0000 7fc7772ac640 1 -- 172.21.15.43:0/541463875 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc75c0a3da0 msgr2=0x7fc75c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:26.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.749+0000 7fc7772ac640 1 --2- 172.21.15.43:0/541463875 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc75c0a3da0 0x7fc75c0a6260 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc76800afd0 tx=0x7fc768087040 comp rx=0 tx=0).stop 2024-04-21T08:48:26.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.749+0000 7fc7772ac640 1 -- 172.21.15.43:0/541463875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc77010c500 msgr2=0x7fc77019cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:26.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.749+0000 7fc7772ac640 1 --2- 172.21.15.43:0/541463875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc77010c500 0x7fc77019cfe0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fc76800c970 tx=0x7fc76800ce30 comp rx=0 tx=0).stop 2024-04-21T08:48:26.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.749+0000 7fc7772ac640 1 -- 172.21.15.43:0/541463875 shutdown_connections 2024-04-21T08:48:26.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.749+0000 7fc7772ac640 1 --2- 172.21.15.43:0/541463875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc77010c500 0x7fc77019cfe0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.749+0000 7fc7772ac640 1 --2- 172.21.15.43:0/541463875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc770103ae0 0x7fc770198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.749+0000 7fc7772ac640 1 --2- 172.21.15.43:0/541463875 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc75c0a3da0 0x7fc75c0a6260 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.749+0000 7fc7772ac640 1 --2- 172.21.15.43:0/541463875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7701031c0 0x7fc770198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:26.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.749+0000 7fc7772ac640 1 -- 172.21.15.43:0/541463875 >> 172.21.15.43:0/541463875 conn(0x7fc770078000 msgr2=0x7fc7700fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:26.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.749+0000 7fc7772ac640 1 -- 172.21.15.43:0/541463875 shutdown_connections 2024-04-21T08:48:26.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:26.749+0000 7fc7772ac640 1 -- 172.21.15.43:0/541463875 wait complete. 2024-04-21T08:48:26.759 INFO:tasks.cephfs.filesystem:scrub status for tag:87ced35a-56f0-4c76-9063-7c81399b281e - {'path': '/', 'tag': '87ced35a-56f0-4c76-9063-7c81399b281e', 'options': 'recursive,force'} 2024-04-21T08:48:28.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:28 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:48:28.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:28 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:28.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:28 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:48:28.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:28 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:28.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:28 smithi175 ceph-mon[31920]: pgmap v296: 129 pgs: 129 active+clean; 3.3 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-04-21T08:48:28.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:28 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:48:28.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:28 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:28.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:28 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:48:28.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:28 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:28.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:28 smithi112 ceph-mon[31085]: pgmap v296: 129 pgs: 129 active+clean; 3.3 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-04-21T08:48:28.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:28 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:48:28.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:28 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:28.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:28 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:48:28.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:28 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:28.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:28 smithi043 ceph-mon[27978]: pgmap v296: 129 pgs: 129 active+clean; 3.3 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-04-21T08:48:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.759+0000 7f316dd8f640 1 -- 172.21.15.43:0/444935591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3168102b70 msgr2=0x7f3168106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.759+0000 7f316dd8f640 1 --2- 172.21.15.43:0/444935591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3168102b70 0x7f3168106bb0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f3150009900 tx=0x7f315002bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.759+0000 7f316dd8f640 1 -- 172.21.15.43:0/444935591 shutdown_connections 2024-04-21T08:48:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.759+0000 7f316dd8f640 1 --2- 172.21.15.43:0/444935591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3168107290 0x7f316810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.759+0000 7f316dd8f640 1 --2- 172.21.15.43:0/444935591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3168102b70 0x7f3168106bb0 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.759+0000 7f316dd8f640 1 --2- 172.21.15.43:0/444935591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31681021c0 0x7f31681025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.759+0000 7f316dd8f640 1 -- 172.21.15.43:0/444935591 >> 172.21.15.43:0/444935591 conn(0x7f31680fda20 msgr2=0x7f31680ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.759+0000 7f316dd8f640 1 -- 172.21.15.43:0/444935591 shutdown_connections 2024-04-21T08:48:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.759+0000 7f316dd8f640 1 -- 172.21.15.43:0/444935591 wait complete. 2024-04-21T08:48:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.760+0000 7f316dd8f640 1 Processor -- start 2024-04-21T08:48:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.760+0000 7f316dd8f640 1 -- start start 2024-04-21T08:48:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.760+0000 7f316dd8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31681021c0 0x7f31681a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.760+0000 7f316dd8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3168107290 0x7f31681a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.760+0000 7f316dd8f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f316819a6b0 0x7f316819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.760+0000 7f316dd8f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3168112200 con 0x7f31681021c0 2024-04-21T08:48:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.760+0000 7f316dd8f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3168112080 con 0x7f3168107290 2024-04-21T08:48:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.760+0000 7f316dd8f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3168112380 con 0x7f316819a6b0 2024-04-21T08:48:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.760+0000 7f31677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31681021c0 0x7f31681a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.761+0000 7f31677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31681021c0 0x7f31681a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45060/0 (socket says 172.21.15.43:45060) 2024-04-21T08:48:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.761+0000 7f31677fe640 1 -- 172.21.15.43:0/3688188797 learned_addr learned my addr 172.21.15.43:0/3688188797 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.761+0000 7f315effd640 1 --2- 172.21.15.43:0/3688188797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3168107290 0x7f31681a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.761+0000 7f3167fff640 1 --2- 172.21.15.43:0/3688188797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f316819a6b0 0x7f316819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.761+0000 7f31677fe640 1 -- 172.21.15.43:0/3688188797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f316819a6b0 msgr2=0x7f316819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.761+0000 7f31677fe640 1 --2- 172.21.15.43:0/3688188797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f316819a6b0 0x7f316819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.761+0000 7f31677fe640 1 -- 172.21.15.43:0/3688188797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3168107290 msgr2=0x7f31681a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.761+0000 7f31677fe640 1 --2- 172.21.15.43:0/3688188797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3168107290 0x7f31681a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.761+0000 7f31677fe640 1 -- 172.21.15.43:0/3688188797 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f316819b2b0 con 0x7f31681021c0 2024-04-21T08:48:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.761+0000 7f3167fff640 1 --2- 172.21.15.43:0/3688188797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f316819a6b0 0x7f316819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.761+0000 7f31677fe640 1 --2- 172.21.15.43:0/3688188797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31681021c0 0x7f31681a05f0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f315800c890 tx=0x7f315800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.761+0000 7f31657fa640 1 -- 172.21.15.43:0/3688188797 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158004220 con 0x7f31681021c0 2024-04-21T08:48:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.761+0000 7f31657fa640 1 -- 172.21.15.43:0/3688188797 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31580043c0 con 0x7f31681021c0 2024-04-21T08:48:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.761+0000 7f316dd8f640 1 -- 172.21.15.43:0/3688188797 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31681a7390 con 0x7f31681021c0 2024-04-21T08:48:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.761+0000 7f31657fa640 1 -- 172.21.15.43:0/3688188797 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158004a00 con 0x7f31681021c0 2024-04-21T08:48:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.761+0000 7f316dd8f640 1 -- 172.21.15.43:0/3688188797 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31681a7740 con 0x7f31681021c0 2024-04-21T08:48:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.764+0000 7f31657fa640 1 -- 172.21.15.43:0/3688188797 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f31580079b0 con 0x7f31681021c0 2024-04-21T08:48:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.764+0000 7f317097e740 1 Processor -- start 2024-04-21T08:48:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.764+0000 7f317097e740 1 -- start start 2024-04-21T08:48:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.764+0000 7f317097e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871b08d0 0x562c871af620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.764+0000 7f317097e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c871afb60 0x562c871affe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.764+0000 7f317097e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c871b90e0 0x562c871bb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.764+0000 7f317097e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562c871148d0 con 0x562c871b08d0 2024-04-21T08:48:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.764+0000 7f317097e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562c870b1e80 con 0x562c871b90e0 2024-04-21T08:48:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.764+0000 7f317097e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562c87110890 con 0x562c871afb60 2024-04-21T08:48:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.764+0000 7f3167fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871b08d0 0x562c871af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.764+0000 7f3167fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871b08d0 0x562c871af620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45068/0 (socket says 172.21.15.43:45068) 2024-04-21T08:48:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.764+0000 7f3167fff640 1 -- 172.21.15.43:0/2097625182 learned_addr learned my addr 172.21.15.43:0/2097625182 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.765+0000 7f315effd640 1 --2- 172.21.15.43:0/2097625182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c871b90e0 0x562c871bb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.765+0000 7f31677fe640 1 --2- 172.21.15.43:0/2097625182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c871afb60 0x562c871affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.765+0000 7f31657fa640 1 --2- 172.21.15.43:0/3688188797 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f31540a3dc0 0x7f31540a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.765+0000 7f3167fff640 1 -- 172.21.15.43:0/2097625182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c871afb60 msgr2=0x562c871affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.765+0000 7f3167fff640 1 --2- 172.21.15.43:0/2097625182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c871afb60 0x562c871affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.765+0000 7f3167fff640 1 -- 172.21.15.43:0/2097625182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c871b90e0 msgr2=0x562c871bb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.765+0000 7f3167fff640 1 --2- 172.21.15.43:0/2097625182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c871b90e0 0x562c871bb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.765+0000 7f3167fff640 1 -- 172.21.15.43:0/2097625182 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c870d3320 con 0x562c871b08d0 2024-04-21T08:48:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.765+0000 7f31657fa640 1 -- 172.21.15.43:0/3688188797 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f31580d0970 con 0x7f31681021c0 2024-04-21T08:48:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.765+0000 7f3167fff640 1 --2- 172.21.15.43:0/3688188797 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f31540a3dc0 0x7f31540a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.765+0000 7f3167fff640 1 --2- 172.21.15.43:0/2097625182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871b08d0 0x562c871af620 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f3160009fd0 tx=0x7f3160010a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8b1864c3b43e4b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.765+0000 7f315dffb640 1 -- 172.21.15.43:0/2097625182 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3160019070 con 0x562c871b08d0 2024-04-21T08:48:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.765+0000 7f315dffb640 1 -- 172.21.15.43:0/2097625182 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3160022070 con 0x562c871b08d0 2024-04-21T08:48:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.765+0000 7f3167fff640 1 --2- 172.21.15.43:0/3688188797 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f31540a3dc0 0x7f31540a6280 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3160021000 tx=0x7f316001f000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.765+0000 7f315dffb640 1 -- 172.21.15.43:0/2097625182 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3160011b20 con 0x562c871b08d0 2024-04-21T08:48:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.766+0000 7f317097e740 1 -- 172.21.15.43:0/2097625182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871b08d0 msgr2=0x562c871af620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.766+0000 7f317097e740 1 --2- 172.21.15.43:0/2097625182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871b08d0 0x562c871af620 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f3160009fd0 tx=0x7f3160010a00 comp rx=0 tx=0).stop 2024-04-21T08:48:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.766+0000 7f317097e740 1 -- 172.21.15.43:0/2097625182 shutdown_connections 2024-04-21T08:48:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.766+0000 7f317097e740 1 --2- 172.21.15.43:0/2097625182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c871b90e0 0x562c871bb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.766+0000 7f317097e740 1 --2- 172.21.15.43:0/2097625182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c871afb60 0x562c871affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.766+0000 7f317097e740 1 --2- 172.21.15.43:0/2097625182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871b08d0 0x562c871af620 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.766+0000 7f317097e740 1 -- 172.21.15.43:0/2097625182 >> 172.21.15.43:0/2097625182 conn(0x562c87176c90 msgr2=0x562c871aa850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.766+0000 7f317097e740 1 -- 172.21.15.43:0/2097625182 shutdown_connections 2024-04-21T08:48:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.766+0000 7f317097e740 1 -- 172.21.15.43:0/2097625182 wait complete. 2024-04-21T08:48:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.766+0000 7f317097e740 1 -- start start 2024-04-21T08:48:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.767+0000 7f317097e740 1 Processor -- start 2024-04-21T08:48:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.767+0000 7f317097e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871afb60 0x562c871b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.767+0000 7f317097e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c871b08d0 0x562c871b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.767+0000 7f317097e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c871b90e0 0x562c871b6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.767+0000 7f317097e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562c871bbe10 con 0x562c871afb60 2024-04-21T08:48:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.767+0000 7f317097e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562c8709ceb0 con 0x562c871b90e0 2024-04-21T08:48:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.767+0000 7f317097e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562c871bbf90 con 0x562c871b08d0 2024-04-21T08:48:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.767+0000 7f317097e740 10 client.0 _ll_register_callbacks cb 0x562c870c9b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.767+0000 7f31677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871afb60 0x562c871b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.767+0000 7f31677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871afb60 0x562c871b8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45072/0 (socket says 172.21.15.43:45072) 2024-04-21T08:48:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.767+0000 7f31677fe640 1 -- 172.21.15.43:0/4152099498 learned_addr learned my addr 172.21.15.43:0/4152099498 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.767+0000 7f315effd640 1 --2- 172.21.15.43:0/4152099498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c871b08d0 0x562c871b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.767+0000 7f3167fff640 1 --2- 172.21.15.43:0/4152099498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c871b90e0 0x562c871b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.767+0000 7f31677fe640 1 -- 172.21.15.43:0/4152099498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c871b08d0 msgr2=0x562c871b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.767+0000 7f31677fe640 1 --2- 172.21.15.43:0/4152099498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c871b08d0 0x562c871b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.767+0000 7f31677fe640 1 -- 172.21.15.43:0/4152099498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c871b90e0 msgr2=0x562c871b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.767+0000 7f31677fe640 1 --2- 172.21.15.43:0/4152099498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c871b90e0 0x562c871b6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.767+0000 7f31677fe640 1 -- 172.21.15.43:0/4152099498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c86e27210 con 0x562c871afb60 2024-04-21T08:48:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.768+0000 7f31677fe640 1 --2- 172.21.15.43:0/4152099498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871afb60 0x562c871b8a50 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f3158005fd0 tx=0x7f3158098170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff616454a89bbae server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.768+0000 7f3146ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.768+0000 7f3146ffd640 1 -- 172.21.15.43:0/4152099498 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315801d050 con 0x562c871afb60 2024-04-21T08:48:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.768+0000 7f3146ffd640 1 -- 172.21.15.43:0/4152099498 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f315809b940 con 0x562c871afb60 2024-04-21T08:48:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.768+0000 7f317097e740 1 -- 172.21.15.43:0/4152099498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3160022070 con 0x562c871afb60 2024-04-21T08:48:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.768+0000 7f3146ffd640 1 -- 172.21.15.43:0/4152099498 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3158005290 con 0x562c871afb60 2024-04-21T08:48:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.768+0000 7f317097e740 10 client.20148 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.768+0000 7f317097e740 1 -- 172.21.15.43:0/4152099498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562c86ef3620 con 0x562c871afb60 2024-04-21T08:48:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.768+0000 7f3146ffd640 1 -- 172.21.15.43:0/4152099498 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3160022070 con 0x562c871afb60 2024-04-21T08:48:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.768+0000 7f3146ffd640 1 -- 172.21.15.43:0/4152099498 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f315809b3b0 con 0x562c871afb60 2024-04-21T08:48:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.768+0000 7f317097e740 10 client.20148 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.768+0000 7f317097e740 10 client.20148 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.768+0000 7f317097e740 20 client.20148 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.768+0000 7f317097e740 1 --2- 172.21.15.43:0/4152099498 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562c871c90b0 0x562c871cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.768+0000 7f317097e740 4 client.20148 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.769+0000 7f317097e740 1 -- 172.21.15.43:0/4152099498 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c87128880 con 0x562c871c90b0 2024-04-21T08:48:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.769+0000 7f315effd640 1 --2- 172.21.15.43:0/4152099498 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562c871c90b0 0x562c871cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.769+0000 7f3146ffd640 10 client.20148 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.769+0000 7f315effd640 1 --2- 172.21.15.43:0/4152099498 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562c871c90b0 0x562c871cb4a0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89179671b2cfe979 server_cookie=4923f55c8af493dc in_seq=0 out_seq=0 2024-04-21T08:48:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.770+0000 7f3146ffd640 1 -- 172.21.15.43:0/4152099498 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562c87128880 con 0x562c871c90b0 2024-04-21T08:48:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.770+0000 7f3146ffd640 10 client.20148 handle_command_reply: tid=0 2024-04-21T08:48:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.770+0000 7f317097e740 1 client.20148 shutdown 2024-04-21T08:48:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.770+0000 7f317097e740 1 -- 172.21.15.43:0/4152099498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871afb60 msgr2=0x562c871b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.770+0000 7f317097e740 1 --2- 172.21.15.43:0/4152099498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871afb60 0x562c871b8a50 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f3158005fd0 tx=0x7f3158098170 comp rx=0 tx=0).stop 2024-04-21T08:48:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.770+0000 7f317097e740 1 -- 172.21.15.43:0/4152099498 shutdown_connections 2024-04-21T08:48:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.770+0000 7f317097e740 1 --2- 172.21.15.43:0/4152099498 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562c871c90b0 0x562c871cb4a0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.770+0000 7f317097e740 1 --2- 172.21.15.43:0/4152099498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c871b90e0 0x562c871b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.770+0000 7f317097e740 1 --2- 172.21.15.43:0/4152099498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c871b08d0 0x562c871b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.770+0000 7f317097e740 1 --2- 172.21.15.43:0/4152099498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871afb60 0x562c871b8a50 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.770+0000 7f3146ffd640 0 client.20148 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.770+0000 7f317097e740 1 -- 172.21.15.43:0/4152099498 >> 172.21.15.43:0/4152099498 conn(0x562c87176c90 msgr2=0x562c871ac5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.770+0000 7f317097e740 1 -- 172.21.15.43:0/4152099498 shutdown_connections 2024-04-21T08:48:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.770+0000 7f317097e740 1 -- 172.21.15.43:0/4152099498 wait complete. 2024-04-21T08:48:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.770+0000 7f317097e740 20 client.20148 trim_cache size 0 max 16384 2024-04-21T08:48:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.785+0000 7f317097e740 1 Processor -- start 2024-04-21T08:48:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.785+0000 7f317097e740 1 -- start start 2024-04-21T08:48:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.785+0000 7f317097e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871afb60 0x562c871c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.785+0000 7f317097e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c871b08d0 0x562c871b8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.785+0000 7f317097e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c871b90e0 0x562c871b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.786+0000 7f317097e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f31681b1660 con 0x562c871afb60 2024-04-21T08:48:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.786+0000 7f317097e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f31681b31a0 con 0x562c871b90e0 2024-04-21T08:48:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.786+0000 7f317097e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f31681b1d30 con 0x562c871b08d0 2024-04-21T08:48:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.786+0000 7f31677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871afb60 0x562c871c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.786+0000 7f31677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871afb60 0x562c871c4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45084/0 (socket says 172.21.15.43:45084) 2024-04-21T08:48:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.786+0000 7f31677fe640 1 -- 172.21.15.43:0/1895026690 learned_addr learned my addr 172.21.15.43:0/1895026690 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.786+0000 7f3167fff640 1 --2- 172.21.15.43:0/1895026690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c871b90e0 0x562c871b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.786+0000 7f31677fe640 1 -- 172.21.15.43:0/1895026690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c871b08d0 msgr2=0x562c871b8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.786+0000 7f31677fe640 1 --2- 172.21.15.43:0/1895026690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c871b08d0 0x562c871b8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.786+0000 7f31677fe640 1 -- 172.21.15.43:0/1895026690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c871b90e0 msgr2=0x562c871b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.786+0000 7f31677fe640 1 --2- 172.21.15.43:0/1895026690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c871b90e0 0x562c871b8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.786+0000 7f31677fe640 1 -- 172.21.15.43:0/1895026690 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c871cbad0 con 0x562c871afb60 2024-04-21T08:48:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.786+0000 7f3167fff640 1 --2- 172.21.15.43:0/1895026690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c871b90e0 0x562c871b8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.786+0000 7f31677fe640 1 --2- 172.21.15.43:0/1895026690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871afb60 0x562c871c4d10 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f315809c1f0 tx=0x7f315800a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856e8c02a39c14a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.786+0000 7f315d7fa640 1 -- 172.21.15.43:0/1895026690 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315801d070 con 0x562c871afb60 2024-04-21T08:48:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.786+0000 7f315d7fa640 1 -- 172.21.15.43:0/1895026690 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f315809cbf0 con 0x562c871afb60 2024-04-21T08:48:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.786+0000 7f315d7fa640 1 -- 172.21.15.43:0/1895026690 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315809b940 con 0x562c871afb60 2024-04-21T08:48:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.787+0000 7f317097e740 1 -- 172.21.15.43:0/1895026690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871afb60 msgr2=0x562c871c4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.787+0000 7f317097e740 1 --2- 172.21.15.43:0/1895026690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871afb60 0x562c871c4d10 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f315809c1f0 tx=0x7f315800a560 comp rx=0 tx=0).stop 2024-04-21T08:48:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.787+0000 7f317097e740 1 -- 172.21.15.43:0/1895026690 shutdown_connections 2024-04-21T08:48:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.787+0000 7f317097e740 1 --2- 172.21.15.43:0/1895026690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c871b90e0 0x562c871b8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.787+0000 7f317097e740 1 --2- 172.21.15.43:0/1895026690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c871b08d0 0x562c871b8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.787+0000 7f317097e740 1 --2- 172.21.15.43:0/1895026690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871afb60 0x562c871c4d10 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.787+0000 7f317097e740 1 -- 172.21.15.43:0/1895026690 >> 172.21.15.43:0/1895026690 conn(0x562c87176c90 msgr2=0x562c871b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.787+0000 7f317097e740 1 -- 172.21.15.43:0/1895026690 shutdown_connections 2024-04-21T08:48:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.787+0000 7f317097e740 1 -- 172.21.15.43:0/1895026690 wait complete. 2024-04-21T08:48:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.787+0000 7f317097e740 1 -- start start 2024-04-21T08:48:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.788+0000 7f317097e740 1 Processor -- start 2024-04-21T08:48:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.788+0000 7f317097e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871afb60 0x562c871c1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.788+0000 7f317097e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c871b08d0 0x562c871c1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.788+0000 7f317097e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c871b90e0 0x562c871be1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.788+0000 7f317097e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f31681b2ad0 con 0x562c871afb60 2024-04-21T08:48:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.788+0000 7f317097e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f31681b2400 con 0x562c871b08d0 2024-04-21T08:48:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.788+0000 7f317097e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f31681af660 con 0x562c871b90e0 2024-04-21T08:48:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.788+0000 7f317097e740 10 client.0 _ll_register_callbacks cb 0x7f3158004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.788+0000 7f31677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871afb60 0x562c871c1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.788+0000 7f31677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871afb60 0x562c871c1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45090/0 (socket says 172.21.15.43:45090) 2024-04-21T08:48:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.788+0000 7f31677fe640 1 -- 172.21.15.43:0/551954365 learned_addr learned my addr 172.21.15.43:0/551954365 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.788+0000 7f315effd640 1 --2- 172.21.15.43:0/551954365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c871b08d0 0x562c871c1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.788+0000 7f3167fff640 1 --2- 172.21.15.43:0/551954365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c871b90e0 0x562c871be1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.788+0000 7f31677fe640 1 -- 172.21.15.43:0/551954365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c871b90e0 msgr2=0x562c871be1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.788+0000 7f31677fe640 1 --2- 172.21.15.43:0/551954365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c871b90e0 0x562c871be1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.788+0000 7f31677fe640 1 -- 172.21.15.43:0/551954365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c871b08d0 msgr2=0x562c871c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.788+0000 7f31677fe640 1 --2- 172.21.15.43:0/551954365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c871b08d0 0x562c871c1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.788+0000 7f31677fe640 1 -- 172.21.15.43:0/551954365 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c871be800 con 0x562c871afb60 2024-04-21T08:48:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.788+0000 7f31677fe640 1 --2- 172.21.15.43:0/551954365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871afb60 0x562c871c1220 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f315809c480 tx=0x7f3158098d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668e2c36e5ba5cb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.789+0000 7f3146ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.789+0000 7f3146ffd640 1 -- 172.21.15.43:0/551954365 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31580ba060 con 0x562c871afb60 2024-04-21T08:48:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.789+0000 7f3146ffd640 1 -- 172.21.15.43:0/551954365 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f3158098240 con 0x562c871afb60 2024-04-21T08:48:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.789+0000 7f317097e740 1 -- 172.21.15.43:0/551954365 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f315809cbf0 con 0x562c871afb60 2024-04-21T08:48:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.789+0000 7f3146ffd640 1 -- 172.21.15.43:0/551954365 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f315801d070 con 0x562c871afb60 2024-04-21T08:48:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.789+0000 7f317097e740 10 client.20154 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.789+0000 7f317097e740 1 -- 172.21.15.43:0/551954365 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562c8722fe40 con 0x562c871afb60 2024-04-21T08:48:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.789+0000 7f3146ffd640 1 -- 172.21.15.43:0/551954365 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f315809cbf0 con 0x562c871afb60 2024-04-21T08:48:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.789+0000 7f3146ffd640 1 -- 172.21.15.43:0/551954365 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f31580bb250 con 0x562c871afb60 2024-04-21T08:48:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.789+0000 7f317097e740 10 client.20154 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.789+0000 7f317097e740 10 client.20154 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.789+0000 7f317097e740 20 client.20154 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.789+0000 7f317097e740 1 --2- 172.21.15.43:0/551954365 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562c87230360 0x562c87232750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.789+0000 7f317097e740 4 client.20154 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:48:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.789+0000 7f317097e740 1 -- 172.21.15.43:0/551954365 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c870b43d0 con 0x562c87230360 2024-04-21T08:48:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.789+0000 7f315effd640 1 --2- 172.21.15.43:0/551954365 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562c87230360 0x562c87232750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.790+0000 7f315effd640 1 --2- 172.21.15.43:0/551954365 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562c87230360 0x562c87232750 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a14fadba66f1186b server_cookie=7627e74b92e63fc4 in_seq=0 out_seq=0 2024-04-21T08:48:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.790+0000 7f3146ffd640 10 client.20154 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.804+0000 7f3146ffd640 1 -- 172.21.15.43:0/551954365 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562c870b43d0 con 0x562c87230360 2024-04-21T08:48:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.804+0000 7f3146ffd640 10 client.20154 handle_command_reply: tid=0 2024-04-21T08:48:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.804+0000 7f317097e740 1 client.20154 shutdown 2024-04-21T08:48:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.804+0000 7f317097e740 1 -- 172.21.15.43:0/551954365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871afb60 msgr2=0x562c871c1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.804+0000 7f317097e740 1 --2- 172.21.15.43:0/551954365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871afb60 0x562c871c1220 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f315809c480 tx=0x7f3158098d90 comp rx=0 tx=0).stop 2024-04-21T08:48:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.805+0000 7f317097e740 1 -- 172.21.15.43:0/551954365 shutdown_connections 2024-04-21T08:48:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.805+0000 7f317097e740 1 --2- 172.21.15.43:0/551954365 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x562c87230360 0x562c87232750 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.805+0000 7f317097e740 1 --2- 172.21.15.43:0/551954365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c871b90e0 0x562c871be1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.805+0000 7f317097e740 1 --2- 172.21.15.43:0/551954365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c871b08d0 0x562c871c1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.805+0000 7f3146ffd640 0 client.20154 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.805+0000 7f317097e740 1 --2- 172.21.15.43:0/551954365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c871afb60 0x562c871c1220 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.805+0000 7f317097e740 1 -- 172.21.15.43:0/551954365 >> 172.21.15.43:0/551954365 conn(0x562c87176c90 msgr2=0x562c871a9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.805+0000 7f317097e740 1 -- 172.21.15.43:0/551954365 shutdown_connections 2024-04-21T08:48:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.805+0000 7f317097e740 1 -- 172.21.15.43:0/551954365 wait complete. 2024-04-21T08:48:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.805+0000 7f317097e740 20 client.20154 trim_cache size 0 max 16384 2024-04-21T08:48:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:48:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:48:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:48:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:48:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.805+0000 7f316dd8f640 1 -- 172.21.15.43:0/3688188797 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f31540a3dc0 msgr2=0x7f31540a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.805+0000 7f316dd8f640 1 --2- 172.21.15.43:0/3688188797 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f31540a3dc0 0x7f31540a6280 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3160021000 tx=0x7f316001f000 comp rx=0 tx=0).stop 2024-04-21T08:48:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.805+0000 7f316dd8f640 1 -- 172.21.15.43:0/3688188797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31681021c0 msgr2=0x7f31681a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.805+0000 7f316dd8f640 1 --2- 172.21.15.43:0/3688188797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31681021c0 0x7f31681a05f0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f315800c890 tx=0x7f315800cd50 comp rx=0 tx=0).stop 2024-04-21T08:48:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.805+0000 7f316dd8f640 1 -- 172.21.15.43:0/3688188797 shutdown_connections 2024-04-21T08:48:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.805+0000 7f316dd8f640 1 --2- 172.21.15.43:0/3688188797 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f31540a3dc0 0x7f31540a6280 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.805+0000 7f316dd8f640 1 --2- 172.21.15.43:0/3688188797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f316819a6b0 0x7f316819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.805+0000 7f316dd8f640 1 --2- 172.21.15.43:0/3688188797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3168107290 0x7f31681a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.805+0000 7f316dd8f640 1 --2- 172.21.15.43:0/3688188797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31681021c0 0x7f31681a05f0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.805+0000 7f316dd8f640 1 -- 172.21.15.43:0/3688188797 >> 172.21.15.43:0/3688188797 conn(0x7f31680fda20 msgr2=0x7f31680fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.806+0000 7f316dd8f640 1 -- 172.21.15.43:0/3688188797 shutdown_connections 2024-04-21T08:48:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:28.806+0000 7f316dd8f640 1 -- 172.21.15.43:0/3688188797 wait complete. 2024-04-21T08:48:30.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:30 smithi175 ceph-mon[31920]: pgmap v297: 129 pgs: 129 active+clean; 3.3 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-04-21T08:48:30.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:30 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:48:30.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:30 smithi112 ceph-mon[31085]: pgmap v297: 129 pgs: 129 active+clean; 3.3 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-04-21T08:48:30.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:30 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:48:30.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:30 smithi043 ceph-mon[27978]: pgmap v297: 129 pgs: 129 active+clean; 3.3 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-04-21T08:48:30.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:30 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:48:32.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:32 smithi175 ceph-mon[31920]: pgmap v298: 129 pgs: 129 active+clean; 3.6 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 24 op/s 2024-04-21T08:48:32.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:32 smithi112 ceph-mon[31085]: pgmap v298: 129 pgs: 129 active+clean; 3.6 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 24 op/s 2024-04-21T08:48:32.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:32 smithi043 ceph-mon[27978]: pgmap v298: 129 pgs: 129 active+clean; 3.6 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 24 op/s 2024-04-21T08:48:34.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:34 smithi175 ceph-mon[31920]: pgmap v299: 129 pgs: 129 active+clean; 3.6 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-04-21T08:48:34.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:34 smithi112 ceph-mon[31085]: pgmap v299: 129 pgs: 129 active+clean; 3.6 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-04-21T08:48:34.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:34 smithi043 ceph-mon[27978]: pgmap v299: 129 pgs: 129 active+clean; 3.6 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-04-21T08:48:36.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:36 smithi175 ceph-mon[31920]: pgmap v300: 129 pgs: 129 active+clean; 3.6 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-04-21T08:48:36.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:36 smithi112 ceph-mon[31085]: pgmap v300: 129 pgs: 129 active+clean; 3.6 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-04-21T08:48:36.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:36 smithi043 ceph-mon[27978]: pgmap v300: 129 pgs: 129 active+clean; 3.6 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-04-21T08:48:38.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:38 smithi175 ceph-mon[31920]: pgmap v301: 129 pgs: 129 active+clean; 3.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 24 op/s 2024-04-21T08:48:38.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:38 smithi112 ceph-mon[31085]: pgmap v301: 129 pgs: 129 active+clean; 3.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 24 op/s 2024-04-21T08:48:38.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:38 smithi043 ceph-mon[27978]: pgmap v301: 129 pgs: 129 active+clean; 3.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 24 op/s 2024-04-21T08:48:40.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:40 smithi175 ceph-mon[31920]: pgmap v302: 129 pgs: 129 active+clean; 3.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:48:40.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:40 smithi112 ceph-mon[31085]: pgmap v302: 129 pgs: 129 active+clean; 3.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:48:40.553 INFO:tasks.workunit.client.0.smithi043.stdout:375/600: open fd = 3 2024-04-21T08:48:40.554 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.554 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.554 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.554 INFO:tasks.workunit.client.0.smithi043.stdout:376/600: open fd = 3 2024-04-21T08:48:40.554 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.554 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.554 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.554 INFO:tasks.workunit.client.0.smithi043.stdout:377/600: open fd = 3 2024-04-21T08:48:40.554 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.554 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.554 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.554 INFO:tasks.workunit.client.0.smithi043.stdout:378/600: open fd = 3 2024-04-21T08:48:40.554 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.554 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.555 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.555 INFO:tasks.workunit.client.0.smithi043.stdout:379/600: open fd = 3 2024-04-21T08:48:40.555 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.555 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.555 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.555 INFO:tasks.workunit.client.0.smithi043.stdout:380/600: open fd = 3 2024-04-21T08:48:40.555 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.555 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.555 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.555 INFO:tasks.workunit.client.0.smithi043.stdout:381/600: open fd = 3 2024-04-21T08:48:40.555 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.555 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.555 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.555 INFO:tasks.workunit.client.0.smithi043.stdout:382/600: open fd = 3 2024-04-21T08:48:40.555 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.556 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.556 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.556 INFO:tasks.workunit.client.0.smithi043.stdout:383/600: open fd = 3 2024-04-21T08:48:40.556 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.556 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.556 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.556 INFO:tasks.workunit.client.0.smithi043.stdout:384/600: open fd = 3 2024-04-21T08:48:40.556 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.556 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.556 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.556 INFO:tasks.workunit.client.0.smithi043.stdout:385/600: open fd = 3 2024-04-21T08:48:40.556 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.556 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.556 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.556 INFO:tasks.workunit.client.0.smithi043.stdout:386/600: open fd = 3 2024-04-21T08:48:40.557 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.557 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.557 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.557 INFO:tasks.workunit.client.0.smithi043.stdout:387/600: open fd = 3 2024-04-21T08:48:40.557 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.557 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.557 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.557 INFO:tasks.workunit.client.0.smithi043.stdout:388/600: open fd = 3 2024-04-21T08:48:40.557 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.557 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.557 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.557 INFO:tasks.workunit.client.0.smithi043.stdout:389/600: open fd = 3 2024-04-21T08:48:40.557 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.557 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.557 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.558 INFO:tasks.workunit.client.0.smithi043.stdout:390/600: open fd = 3 2024-04-21T08:48:40.558 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.558 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.558 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.558 INFO:tasks.workunit.client.0.smithi043.stdout:391/600: open fd = 3 2024-04-21T08:48:40.558 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.558 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.558 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.558 INFO:tasks.workunit.client.0.smithi043.stdout:392/600: open fd = 3 2024-04-21T08:48:40.558 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.558 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.558 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.558 INFO:tasks.workunit.client.0.smithi043.stdout:393/600: open fd = 3 2024-04-21T08:48:40.558 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.558 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.559 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.559 INFO:tasks.workunit.client.0.smithi043.stdout:394/600: open fd = 3 2024-04-21T08:48:40.559 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.559 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.559 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.559 INFO:tasks.workunit.client.0.smithi043.stdout:395/600: open fd = 3 2024-04-21T08:48:40.559 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.559 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.559 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.559 INFO:tasks.workunit.client.0.smithi043.stdout:396/600: open fd = 3 2024-04-21T08:48:40.559 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.559 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.559 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.559 INFO:tasks.workunit.client.0.smithi043.stdout:397/600: open fd = 3 2024-04-21T08:48:40.559 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.559 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.560 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.560 INFO:tasks.workunit.client.0.smithi043.stdout:398/600: open fd = 3 2024-04-21T08:48:40.560 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.560 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.560 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.560 INFO:tasks.workunit.client.0.smithi043.stdout:399/600: open fd = 3 2024-04-21T08:48:40.560 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.560 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.560 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.560 INFO:tasks.workunit.client.0.smithi043.stdout:400/600: open fd = 3 2024-04-21T08:48:40.560 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.560 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.560 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.560 INFO:tasks.workunit.client.0.smithi043.stdout:401/600: open fd = 3 2024-04-21T08:48:40.560 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.561 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.561 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.561 INFO:tasks.workunit.client.0.smithi043.stdout:402/600: open fd = 3 2024-04-21T08:48:40.561 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.561 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.561 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.561 INFO:tasks.workunit.client.0.smithi043.stdout:403/600: open fd = 3 2024-04-21T08:48:40.561 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.561 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.561 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.561 INFO:tasks.workunit.client.0.smithi043.stdout:404/600: open fd = 3 2024-04-21T08:48:40.561 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.561 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.561 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.561 INFO:tasks.workunit.client.0.smithi043.stdout:405/600: open fd = 3 2024-04-21T08:48:40.561 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.562 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.562 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.562 INFO:tasks.workunit.client.0.smithi043.stdout:406/600: open fd = 3 2024-04-21T08:48:40.562 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.562 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.562 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.562 INFO:tasks.workunit.client.0.smithi043.stdout:407/600: open fd = 3 2024-04-21T08:48:40.562 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.562 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.562 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.562 INFO:tasks.workunit.client.0.smithi043.stdout:408/600: open fd = 3 2024-04-21T08:48:40.562 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.562 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.562 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.562 INFO:tasks.workunit.client.0.smithi043.stdout:409/600: open fd = 3 2024-04-21T08:48:40.563 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.563 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.563 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.563 INFO:tasks.workunit.client.0.smithi043.stdout:410/600: open fd = 3 2024-04-21T08:48:40.563 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.563 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.563 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.563 INFO:tasks.workunit.client.0.smithi043.stdout:411/600: open fd = 3 2024-04-21T08:48:40.563 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.563 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.563 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.563 INFO:tasks.workunit.client.0.smithi043.stdout:412/600: open fd = 3 2024-04-21T08:48:40.563 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.563 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.563 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.563 INFO:tasks.workunit.client.0.smithi043.stdout:413/600: open fd = 3 2024-04-21T08:48:40.564 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.564 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.564 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.564 INFO:tasks.workunit.client.0.smithi043.stdout:414/600: open fd = 3 2024-04-21T08:48:40.564 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.564 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.564 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.564 INFO:tasks.workunit.client.0.smithi043.stdout:415/600: open fd = 3 2024-04-21T08:48:40.564 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.564 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.564 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.564 INFO:tasks.workunit.client.0.smithi043.stdout:416/600: open fd = 3 2024-04-21T08:48:40.564 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.564 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.564 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.565 INFO:tasks.workunit.client.0.smithi043.stdout:417/600: open fd = 3 2024-04-21T08:48:40.565 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.565 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.565 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.565 INFO:tasks.workunit.client.0.smithi043.stdout:418/600: open fd = 3 2024-04-21T08:48:40.565 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.565 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.565 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.565 INFO:tasks.workunit.client.0.smithi043.stdout:419/600: open fd = 3 2024-04-21T08:48:40.565 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.565 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.565 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.565 INFO:tasks.workunit.client.0.smithi043.stdout:420/600: open fd = 3 2024-04-21T08:48:40.565 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.565 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.565 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.566 INFO:tasks.workunit.client.0.smithi043.stdout:421/600: open fd = 3 2024-04-21T08:48:40.566 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.566 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.566 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.566 INFO:tasks.workunit.client.0.smithi043.stdout:422/600: open fd = 3 2024-04-21T08:48:40.566 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.566 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.566 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.566 INFO:tasks.workunit.client.0.smithi043.stdout:423/600: open fd = 3 2024-04-21T08:48:40.566 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.566 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.566 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.566 INFO:tasks.workunit.client.0.smithi043.stdout:424/600: open fd = 3 2024-04-21T08:48:40.566 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.566 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.567 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.567 INFO:tasks.workunit.client.0.smithi043.stdout:425/600: open fd = 3 2024-04-21T08:48:40.567 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.567 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.567 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.567 INFO:tasks.workunit.client.0.smithi043.stdout:426/600: open fd = 3 2024-04-21T08:48:40.567 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.567 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.567 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.567 INFO:tasks.workunit.client.0.smithi043.stdout:427/600: open fd = 3 2024-04-21T08:48:40.567 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.567 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.567 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.567 INFO:tasks.workunit.client.0.smithi043.stdout:428/600: open fd = 3 2024-04-21T08:48:40.567 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.567 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.568 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.568 INFO:tasks.workunit.client.0.smithi043.stdout:429/600: open fd = 3 2024-04-21T08:48:40.568 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.568 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.568 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.568 INFO:tasks.workunit.client.0.smithi043.stdout:430/600: open fd = 3 2024-04-21T08:48:40.568 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.568 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.568 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.568 INFO:tasks.workunit.client.0.smithi043.stdout:431/600: open fd = 3 2024-04-21T08:48:40.568 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.568 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.568 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.568 INFO:tasks.workunit.client.0.smithi043.stdout:432/600: open fd = 3 2024-04-21T08:48:40.568 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.569 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.569 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.569 INFO:tasks.workunit.client.0.smithi043.stdout:433/600: open fd = 3 2024-04-21T08:48:40.569 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.569 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.569 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.569 INFO:tasks.workunit.client.0.smithi043.stdout:434/600: open fd = 3 2024-04-21T08:48:40.569 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.569 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.569 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.569 INFO:tasks.workunit.client.0.smithi043.stdout:435/600: open fd = 3 2024-04-21T08:48:40.569 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.569 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.569 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.569 INFO:tasks.workunit.client.0.smithi043.stdout:436/600: open fd = 3 2024-04-21T08:48:40.570 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.570 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:48:40.570 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:48:40.570 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:40 smithi043 ceph-mon[27978]: pgmap v302: 129 pgs: 129 active+clean; 3.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:48:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.905+0000 7f31fd5ce640 1 -- 172.21.15.43:0/2694343500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31f8103b00 msgr2=0x7f31f810bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.905+0000 7f31fd5ce640 1 --2- 172.21.15.43:0/2694343500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31f8103b00 0x7f31f810bfe0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f31ec009900 tx=0x7f31ec02bd60 comp rx=0 tx=0).stop 2024-04-21T08:48:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.905+0000 7f31fd5ce640 1 -- 172.21.15.43:0/2694343500 shutdown_connections 2024-04-21T08:48:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.905+0000 7f31fd5ce640 1 --2- 172.21.15.43:0/2694343500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31f810c520 0x7f31f810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.905+0000 7f31fd5ce640 1 --2- 172.21.15.43:0/2694343500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31f8103b00 0x7f31f810bfe0 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.905+0000 7f31fd5ce640 1 --2- 172.21.15.43:0/2694343500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f81031e0 0x7f31f81035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.905+0000 7f31fd5ce640 1 -- 172.21.15.43:0/2694343500 >> 172.21.15.43:0/2694343500 conn(0x7f31f8077fd0 msgr2=0x7f31f80783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.905+0000 7f31fd5ce640 1 -- 172.21.15.43:0/2694343500 shutdown_connections 2024-04-21T08:48:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.905+0000 7f31fd5ce640 1 -- 172.21.15.43:0/2694343500 wait complete. 2024-04-21T08:48:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.906+0000 7f31fd5ce640 1 Processor -- start 2024-04-21T08:48:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.906+0000 7f31fd5ce640 1 -- start start 2024-04-21T08:48:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.906+0000 7f31fd5ce640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f81031e0 0x7f31f8100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.906+0000 7f31fd5ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31f8103b00 0x7f31f8101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.906+0000 7f31fd5ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31f810c520 0x7f31f81018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.906+0000 7f31fd5ce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f31f81121b0 con 0x7f31f8103b00 2024-04-21T08:48:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.906+0000 7f31fd5ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f31f8112030 con 0x7f31f810c520 2024-04-21T08:48:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.906+0000 7f31fd5ce640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f31f8112330 con 0x7f31f81031e0 2024-04-21T08:48:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.906+0000 7f31f67fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31f8103b00 0x7f31f8101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.906+0000 7f31f67fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31f8103b00 0x7f31f8101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53498/0 (socket says 172.21.15.43:53498) 2024-04-21T08:48:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.906+0000 7f31f67fc640 1 -- 172.21.15.43:0/1810584273 learned_addr learned my addr 172.21.15.43:0/1810584273 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.906+0000 7f31f77fe640 1 --2- 172.21.15.43:0/1810584273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31f810c520 0x7f31f81018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.906+0000 7f31f6ffd640 1 --2- 172.21.15.43:0/1810584273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f81031e0 0x7f31f8100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.907+0000 7f31f67fc640 1 -- 172.21.15.43:0/1810584273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f81031e0 msgr2=0x7f31f8100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.907+0000 7f31f67fc640 1 --2- 172.21.15.43:0/1810584273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f81031e0 0x7f31f8100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.907+0000 7f31f67fc640 1 -- 172.21.15.43:0/1810584273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31f810c520 msgr2=0x7f31f81018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.907+0000 7f31f67fc640 1 --2- 172.21.15.43:0/1810584273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31f810c520 0x7f31f81018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.907+0000 7f31f67fc640 1 -- 172.21.15.43:0/1810584273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f81a6bb0 con 0x7f31f8103b00 2024-04-21T08:48:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.907+0000 7f31f67fc640 1 --2- 172.21.15.43:0/1810584273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31f8103b00 0x7f31f8101370 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f31ec005940 tx=0x7f31ec0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.907+0000 7f31dffff640 1 -- 172.21.15.43:0/1810584273 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec043020 con 0x7f31f8103b00 2024-04-21T08:48:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.907+0000 7f31dffff640 1 -- 172.21.15.43:0/1810584273 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31ec02dc30 con 0x7f31f8103b00 2024-04-21T08:48:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.907+0000 7f31dffff640 1 -- 172.21.15.43:0/1810584273 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec003a10 con 0x7f31f8103b00 2024-04-21T08:48:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.907+0000 7f31fd5ce640 1 -- 172.21.15.43:0/1810584273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f81a6e40 con 0x7f31f8103b00 2024-04-21T08:48:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.907+0000 7f31fd5ce640 1 -- 172.21.15.43:0/1810584273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f81a7350 con 0x7f31f8103b00 2024-04-21T08:48:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.909+0000 7f31dffff640 1 -- 172.21.15.43:0/1810584273 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f31ec046020 con 0x7f31f8103b00 2024-04-21T08:48:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.910+0000 7f32001bf740 1 Processor -- start 2024-04-21T08:48:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.910+0000 7f32001bf740 1 -- start start 2024-04-21T08:48:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.910+0000 7f32001bf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578f245d8d0 0x5578f245c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.910+0000 7f32001bf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f245cb60 0x5578f245cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.910+0000 7f32001bf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578f24660e0 0x5578f24684d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.910+0000 7f32001bf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5578f23c18d0 con 0x5578f245cb60 2024-04-21T08:48:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.910+0000 7f32001bf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5578f235ee80 con 0x5578f245d8d0 2024-04-21T08:48:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.910+0000 7f32001bf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5578f23bd890 con 0x5578f24660e0 2024-04-21T08:48:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.910+0000 7f31f6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f245cb60 0x5578f245cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.910+0000 7f31f6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f245cb60 0x5578f245cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53512/0 (socket says 172.21.15.43:53512) 2024-04-21T08:48:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.910+0000 7f31f6ffd640 1 -- 172.21.15.43:0/1663669372 learned_addr learned my addr 172.21.15.43:0/1663669372 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.910+0000 7f31f77fe640 1 --2- 172.21.15.43:0/1663669372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578f245d8d0 0x5578f245c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.910+0000 7f31dffff640 1 --2- 172.21.15.43:0/1810584273 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f31e00a3e70 0x7f31e00a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.910+0000 7f31f67fc640 1 --2- 172.21.15.43:0/1663669372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578f24660e0 0x5578f24684d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.910+0000 7f31f77fe640 1 --2- 172.21.15.43:0/1810584273 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f31e00a3e70 0x7f31e00a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.910+0000 7f31f6ffd640 1 -- 172.21.15.43:0/1663669372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578f24660e0 msgr2=0x5578f24684d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.910+0000 7f31f6ffd640 1 --2- 172.21.15.43:0/1663669372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578f24660e0 0x5578f24684d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.911+0000 7f31f6ffd640 1 -- 172.21.15.43:0/1663669372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578f245d8d0 msgr2=0x5578f245c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.911+0000 7f31f6ffd640 1 --2- 172.21.15.43:0/1663669372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578f245d8d0 0x5578f245c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.911+0000 7f31f6ffd640 1 -- 172.21.15.43:0/1663669372 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578f2380320 con 0x5578f245cb60 2024-04-21T08:48:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.911+0000 7f31f77fe640 1 --2- 172.21.15.43:0/1663669372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578f245d8d0 0x5578f245c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.911+0000 7f31dffff640 1 -- 172.21.15.43:0/1810584273 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f31ec0f2b30 con 0x7f31f8103b00 2024-04-21T08:48:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.911+0000 7f31f6ffd640 1 --2- 172.21.15.43:0/1663669372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f245cb60 0x5578f245cfe0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f31e800bcf0 tx=0x7f31e800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6e79d7ac4b2a64 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.911+0000 7f31f77fe640 1 --2- 172.21.15.43:0/1810584273 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f31e00a3e70 0x7f31e00a6330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009fd0 tx=0x7f31f0004ab0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.911+0000 7f31dd7fa640 1 -- 172.21.15.43:0/1663669372 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e8015070 con 0x5578f245cb60 2024-04-21T08:48:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.911+0000 7f31dd7fa640 1 -- 172.21.15.43:0/1663669372 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31e801a910 con 0x5578f245cb60 2024-04-21T08:48:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.911+0000 7f31dd7fa640 1 -- 172.21.15.43:0/1663669372 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e80118c0 con 0x5578f245cb60 2024-04-21T08:48:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.912+0000 7f32001bf740 1 -- 172.21.15.43:0/1663669372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f245cb60 msgr2=0x5578f245cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.912+0000 7f32001bf740 1 --2- 172.21.15.43:0/1663669372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f245cb60 0x5578f245cfe0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f31e800bcf0 tx=0x7f31e800f730 comp rx=0 tx=0).stop 2024-04-21T08:48:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.912+0000 7f32001bf740 1 -- 172.21.15.43:0/1663669372 shutdown_connections 2024-04-21T08:48:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.912+0000 7f32001bf740 1 --2- 172.21.15.43:0/1663669372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578f24660e0 0x5578f24684d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.912+0000 7f32001bf740 1 --2- 172.21.15.43:0/1663669372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f245cb60 0x5578f245cfe0 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.912+0000 7f32001bf740 1 --2- 172.21.15.43:0/1663669372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578f245d8d0 0x5578f245c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.912+0000 7f32001bf740 1 -- 172.21.15.43:0/1663669372 >> 172.21.15.43:0/1663669372 conn(0x5578f2423c90 msgr2=0x5578f2457850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.912+0000 7f32001bf740 1 -- 172.21.15.43:0/1663669372 shutdown_connections 2024-04-21T08:48:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.912+0000 7f32001bf740 1 -- 172.21.15.43:0/1663669372 wait complete. 2024-04-21T08:48:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.912+0000 7f32001bf740 1 -- start start 2024-04-21T08:48:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.912+0000 7f32001bf740 1 Processor -- start 2024-04-21T08:48:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.912+0000 7f32001bf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578f245cb60 0x5578f2465a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.912+0000 7f32001bf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f245d8d0 0x5578f2462e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.912+0000 7f32001bf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578f24660e0 0x5578f24633c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.912+0000 7f32001bf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5578f2468e10 con 0x5578f245d8d0 2024-04-21T08:48:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.912+0000 7f32001bf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5578f2349eb0 con 0x5578f24660e0 2024-04-21T08:48:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.912+0000 7f32001bf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5578f2468f90 con 0x5578f245cb60 2024-04-21T08:48:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.913+0000 7f32001bf740 10 client.0 _ll_register_callbacks cb 0x5578f2376b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.913+0000 7f31f67fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f245d8d0 0x5578f2462e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.913+0000 7f31f67fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f245d8d0 0x5578f2462e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53514/0 (socket says 172.21.15.43:53514) 2024-04-21T08:48:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.913+0000 7f31f67fc640 1 -- 172.21.15.43:0/215743232 learned_addr learned my addr 172.21.15.43:0/215743232 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.913+0000 7f31f6ffd640 1 --2- 172.21.15.43:0/215743232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578f245cb60 0x5578f2465a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.913+0000 7f31f77fe640 1 --2- 172.21.15.43:0/215743232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578f24660e0 0x5578f24633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.913+0000 7f31f67fc640 1 -- 172.21.15.43:0/215743232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578f245cb60 msgr2=0x5578f2465a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.913+0000 7f31f67fc640 1 --2- 172.21.15.43:0/215743232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578f245cb60 0x5578f2465a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.913+0000 7f31f67fc640 1 -- 172.21.15.43:0/215743232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578f24660e0 msgr2=0x5578f24633c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.913+0000 7f31f67fc640 1 --2- 172.21.15.43:0/215743232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578f24660e0 0x5578f24633c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.913+0000 7f31f67fc640 1 -- 172.21.15.43:0/215743232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578f20d4210 con 0x5578f245d8d0 2024-04-21T08:48:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.913+0000 7f31f67fc640 1 --2- 172.21.15.43:0/215743232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f245d8d0 0x5578f2462e20 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f31ec0bac20 tx=0x7f31ec009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9859406535c2f5a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.913+0000 7f31daff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.913+0000 7f31daff5640 1 -- 172.21.15.43:0/215743232 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec0c7430 con 0x5578f245d8d0 2024-04-21T08:48:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.913+0000 7f31daff5640 1 -- 172.21.15.43:0/215743232 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f31ec0bae30 con 0x5578f245d8d0 2024-04-21T08:48:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.913+0000 7f32001bf740 1 -- 172.21.15.43:0/215743232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31e801a910 con 0x5578f245d8d0 2024-04-21T08:48:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.913+0000 7f31daff5640 1 -- 172.21.15.43:0/215743232 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f31ec0c2b70 con 0x5578f245d8d0 2024-04-21T08:48:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.913+0000 7f32001bf740 10 client.20166 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.913+0000 7f32001bf740 1 -- 172.21.15.43:0/215743232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5578f21a0620 con 0x5578f245d8d0 2024-04-21T08:48:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.914+0000 7f31daff5640 1 -- 172.21.15.43:0/215743232 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31e801a910 con 0x5578f245d8d0 2024-04-21T08:48:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.914+0000 7f31daff5640 1 -- 172.21.15.43:0/215743232 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f31ec003bb0 con 0x5578f245d8d0 2024-04-21T08:48:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.914+0000 7f32001bf740 10 client.20166 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.914+0000 7f32001bf740 10 client.20166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.914+0000 7f32001bf740 20 client.20166 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.914+0000 7f32001bf740 1 --2- 172.21.15.43:0/215743232 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5578f2476040 0x5578f24784a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.914+0000 7f32001bf740 4 client.20166 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.914+0000 7f32001bf740 1 -- 172.21.15.43:0/215743232 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578f23d5880 con 0x5578f2476040 2024-04-21T08:48:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.914+0000 7f31f6ffd640 1 --2- 172.21.15.43:0/215743232 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5578f2476040 0x5578f24784a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.915+0000 7f31f6ffd640 1 --2- 172.21.15.43:0/215743232 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5578f2476040 0x5578f24784a0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79098e43e091b00e server_cookie=6944f330b661d1be in_seq=0 out_seq=0 2024-04-21T08:48:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.915+0000 7f31daff5640 10 client.20166 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.915+0000 7f31daff5640 1 -- 172.21.15.43:0/215743232 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578f23d5880 con 0x5578f2476040 2024-04-21T08:48:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.915+0000 7f31daff5640 10 client.20166 handle_command_reply: tid=0 2024-04-21T08:48:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.915+0000 7f32001bf740 1 client.20166 shutdown 2024-04-21T08:48:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.915+0000 7f32001bf740 1 -- 172.21.15.43:0/215743232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f245d8d0 msgr2=0x5578f2462e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.915+0000 7f32001bf740 1 --2- 172.21.15.43:0/215743232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f245d8d0 0x5578f2462e20 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f31ec0bac20 tx=0x7f31ec009620 comp rx=0 tx=0).stop 2024-04-21T08:48:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.916+0000 7f32001bf740 1 -- 172.21.15.43:0/215743232 shutdown_connections 2024-04-21T08:48:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.916+0000 7f32001bf740 1 --2- 172.21.15.43:0/215743232 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5578f2476040 0x5578f24784a0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.916+0000 7f32001bf740 1 --2- 172.21.15.43:0/215743232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578f24660e0 0x5578f24633c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.916+0000 7f32001bf740 1 --2- 172.21.15.43:0/215743232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f245d8d0 0x5578f2462e20 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.916+0000 7f31daff5640 0 client.20166 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.916+0000 7f32001bf740 1 --2- 172.21.15.43:0/215743232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578f245cb60 0x5578f2465a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.916+0000 7f32001bf740 1 -- 172.21.15.43:0/215743232 >> 172.21.15.43:0/215743232 conn(0x5578f2423c90 msgr2=0x5578f24595a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.916+0000 7f32001bf740 1 -- 172.21.15.43:0/215743232 shutdown_connections 2024-04-21T08:48:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.916+0000 7f32001bf740 1 -- 172.21.15.43:0/215743232 wait complete. 2024-04-21T08:48:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.916+0000 7f32001bf740 20 client.20166 trim_cache size 0 max 16384 2024-04-21T08:48:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.931+0000 7f32001bf740 1 Processor -- start 2024-04-21T08:48:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.931+0000 7f32001bf740 1 -- start start 2024-04-21T08:48:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.931+0000 7f32001bf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578f245cb60 0x5578f24593b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.931+0000 7f32001bf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578f245d8d0 0x5578f2465550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.931+0000 7f32001bf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f24660e0 0x5578f2465a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.931+0000 7f32001bf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f31f81b15e0 con 0x5578f24660e0 2024-04-21T08:48:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.931+0000 7f32001bf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f31f81b3120 con 0x5578f245cb60 2024-04-21T08:48:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.931+0000 7f32001bf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f31f81b1cb0 con 0x5578f245d8d0 2024-04-21T08:48:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.931+0000 7f31f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f24660e0 0x5578f2465a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.931+0000 7f31f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f24660e0 0x5578f2465a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53530/0 (socket says 172.21.15.43:53530) 2024-04-21T08:48:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.931+0000 7f31f77fe640 1 -- 172.21.15.43:0/1703224849 learned_addr learned my addr 172.21.15.43:0/1703224849 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.931+0000 7f31f6ffd640 1 --2- 172.21.15.43:0/1703224849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578f245cb60 0x5578f24593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.931+0000 7f31f67fc640 1 --2- 172.21.15.43:0/1703224849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578f245d8d0 0x5578f2465550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.931+0000 7f31f77fe640 1 -- 172.21.15.43:0/1703224849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578f245d8d0 msgr2=0x5578f2465550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.931+0000 7f31f77fe640 1 --2- 172.21.15.43:0/1703224849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578f245d8d0 0x5578f2465550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.931+0000 7f31f77fe640 1 -- 172.21.15.43:0/1703224849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578f245cb60 msgr2=0x5578f24593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.931+0000 7f31f77fe640 1 --2- 172.21.15.43:0/1703224849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578f245cb60 0x5578f24593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.931+0000 7f31f77fe640 1 -- 172.21.15.43:0/1703224849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578f2452f30 con 0x5578f24660e0 2024-04-21T08:48:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.932+0000 7f31f77fe640 1 --2- 172.21.15.43:0/1703224849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f24660e0 0x5578f2465a90 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f31f0013270 tx=0x7f31f0009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff014f4d623467c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.932+0000 7f31dcff9640 1 -- 172.21.15.43:0/1703224849 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f00109b0 con 0x5578f24660e0 2024-04-21T08:48:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.932+0000 7f31dcff9640 1 -- 172.21.15.43:0/1703224849 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31f0026070 con 0x5578f24660e0 2024-04-21T08:48:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.932+0000 7f31dcff9640 1 -- 172.21.15.43:0/1703224849 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f002bd90 con 0x5578f24660e0 2024-04-21T08:48:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.932+0000 7f32001bf740 1 -- 172.21.15.43:0/1703224849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f24660e0 msgr2=0x5578f2465a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.932+0000 7f32001bf740 1 --2- 172.21.15.43:0/1703224849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f24660e0 0x5578f2465a90 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f31f0013270 tx=0x7f31f0009c80 comp rx=0 tx=0).stop 2024-04-21T08:48:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.932+0000 7f32001bf740 1 -- 172.21.15.43:0/1703224849 shutdown_connections 2024-04-21T08:48:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.932+0000 7f32001bf740 1 --2- 172.21.15.43:0/1703224849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f24660e0 0x5578f2465a90 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.932+0000 7f32001bf740 1 --2- 172.21.15.43:0/1703224849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578f245d8d0 0x5578f2465550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.933+0000 7f32001bf740 1 --2- 172.21.15.43:0/1703224849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578f245cb60 0x5578f24593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.933+0000 7f32001bf740 1 -- 172.21.15.43:0/1703224849 >> 172.21.15.43:0/1703224849 conn(0x5578f2423c90 msgr2=0x5578f2458ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.933+0000 7f32001bf740 1 -- 172.21.15.43:0/1703224849 shutdown_connections 2024-04-21T08:48:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.933+0000 7f32001bf740 1 -- 172.21.15.43:0/1703224849 wait complete. 2024-04-21T08:48:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.933+0000 7f32001bf740 1 -- start start 2024-04-21T08:48:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.933+0000 7f32001bf740 1 Processor -- start 2024-04-21T08:48:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.933+0000 7f32001bf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578f245cb60 0x5578f246e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.933+0000 7f32001bf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578f245d8d0 0x5578f246e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.933+0000 7f32001bf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f24660e0 0x5578f24691e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.933+0000 7f32001bf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f31f81b2a50 con 0x5578f24660e0 2024-04-21T08:48:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.933+0000 7f32001bf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f31f81b2380 con 0x5578f245cb60 2024-04-21T08:48:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.933+0000 7f32001bf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f31f81af5e0 con 0x5578f245d8d0 2024-04-21T08:48:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.933+0000 7f32001bf740 10 client.0 _ll_register_callbacks cb 0x7f31ec0be3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.933+0000 7f31f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f24660e0 0x5578f24691e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.933+0000 7f31f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f24660e0 0x5578f24691e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53536/0 (socket says 172.21.15.43:53536) 2024-04-21T08:48:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.933+0000 7f31f77fe640 1 -- 172.21.15.43:0/3628294863 learned_addr learned my addr 172.21.15.43:0/3628294863 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.933+0000 7f31f6ffd640 1 --2- 172.21.15.43:0/3628294863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578f245cb60 0x5578f246e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.934+0000 7f31f67fc640 1 --2- 172.21.15.43:0/3628294863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578f245d8d0 0x5578f246e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.934+0000 7f31f77fe640 1 -- 172.21.15.43:0/3628294863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578f245d8d0 msgr2=0x5578f246e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.934+0000 7f31f77fe640 1 --2- 172.21.15.43:0/3628294863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578f245d8d0 0x5578f246e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.934+0000 7f31f77fe640 1 -- 172.21.15.43:0/3628294863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578f245cb60 msgr2=0x5578f246e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.934+0000 7f31f77fe640 1 --2- 172.21.15.43:0/3628294863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578f245cb60 0x5578f246e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.934+0000 7f31f77fe640 1 -- 172.21.15.43:0/3628294863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578f2469a10 con 0x5578f24660e0 2024-04-21T08:48:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.934+0000 7f31f6ffd640 1 --2- 172.21.15.43:0/3628294863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578f245cb60 0x5578f246e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.934+0000 7f31f77fe640 1 --2- 172.21.15.43:0/3628294863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f24660e0 0x5578f24691e0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009f30 tx=0x7f31f000ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c156edfadeeb50b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.934+0000 7f31daff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.934+0000 7f31daff5640 1 -- 172.21.15.43:0/3628294863 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f0039060 con 0x5578f24660e0 2024-04-21T08:48:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.934+0000 7f31daff5640 1 -- 172.21.15.43:0/3628294863 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f31f002b680 con 0x5578f24660e0 2024-04-21T08:48:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.934+0000 7f32001bf740 1 -- 172.21.15.43:0/3628294863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31f0026070 con 0x5578f24660e0 2024-04-21T08:48:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.934+0000 7f31daff5640 1 -- 172.21.15.43:0/3628294863 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f31f0040050 con 0x5578f24660e0 2024-04-21T08:48:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.934+0000 7f32001bf740 10 client.20172 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.934+0000 7f32001bf740 1 -- 172.21.15.43:0/3628294863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5578f24dd090 con 0x5578f24660e0 2024-04-21T08:48:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.934+0000 7f31daff5640 1 -- 172.21.15.43:0/3628294863 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31f0026070 con 0x5578f24660e0 2024-04-21T08:48:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.934+0000 7f31daff5640 1 -- 172.21.15.43:0/3628294863 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f31f0003970 con 0x5578f24660e0 2024-04-21T08:48:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.935+0000 7f32001bf740 10 client.20172 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.935+0000 7f32001bf740 10 client.20172 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.935+0000 7f32001bf740 20 client.20172 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.935+0000 7f32001bf740 1 --2- 172.21.15.43:0/3628294863 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5578f24dd700 0x5578f24dfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.935+0000 7f32001bf740 4 client.20172 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:48:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.935+0000 7f32001bf740 1 -- 172.21.15.43:0/3628294863 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5578f23613d0 con 0x5578f24dd700 2024-04-21T08:48:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.935+0000 7f31f6ffd640 1 --2- 172.21.15.43:0/3628294863 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5578f24dd700 0x5578f24dfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.935+0000 7f31f6ffd640 1 --2- 172.21.15.43:0/3628294863 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5578f24dd700 0x5578f24dfaf0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18c6c286deffab17 server_cookie=17d82f5c04ba1eb4 in_seq=0 out_seq=0 2024-04-21T08:48:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.935+0000 7f31daff5640 10 client.20172 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.949+0000 7f31daff5640 1 -- 172.21.15.43:0/3628294863 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5578f23613d0 con 0x5578f24dd700 2024-04-21T08:48:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.949+0000 7f31daff5640 10 client.20172 handle_command_reply: tid=0 2024-04-21T08:48:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.949+0000 7f32001bf740 1 client.20172 shutdown 2024-04-21T08:48:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.949+0000 7f32001bf740 1 -- 172.21.15.43:0/3628294863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f24660e0 msgr2=0x5578f24691e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.949+0000 7f32001bf740 1 --2- 172.21.15.43:0/3628294863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f24660e0 0x5578f24691e0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009f30 tx=0x7f31f000ff90 comp rx=0 tx=0).stop 2024-04-21T08:48:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.949+0000 7f32001bf740 1 -- 172.21.15.43:0/3628294863 shutdown_connections 2024-04-21T08:48:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.949+0000 7f32001bf740 1 --2- 172.21.15.43:0/3628294863 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5578f24dd700 0x5578f24dfaf0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.949+0000 7f32001bf740 1 --2- 172.21.15.43:0/3628294863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578f24660e0 0x5578f24691e0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.949+0000 7f32001bf740 1 --2- 172.21.15.43:0/3628294863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5578f245d8d0 0x5578f246e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.949+0000 7f31daff5640 0 client.20172 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.949+0000 7f32001bf740 1 --2- 172.21.15.43:0/3628294863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578f245cb60 0x5578f246e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.949+0000 7f32001bf740 1 -- 172.21.15.43:0/3628294863 >> 172.21.15.43:0/3628294863 conn(0x5578f2423c90 msgr2=0x5578f2459370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.949+0000 7f32001bf740 1 -- 172.21.15.43:0/3628294863 shutdown_connections 2024-04-21T08:48:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.949+0000 7f32001bf740 1 -- 172.21.15.43:0/3628294863 wait complete. 2024-04-21T08:48:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.949+0000 7f32001bf740 20 client.20172 trim_cache size 0 max 16384 2024-04-21T08:48:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:48:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:48:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:48:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:48:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.949+0000 7f31fd5ce640 1 -- 172.21.15.43:0/1810584273 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f31e00a3e70 msgr2=0x7f31e00a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.949+0000 7f31fd5ce640 1 --2- 172.21.15.43:0/1810584273 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f31e00a3e70 0x7f31e00a6330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009fd0 tx=0x7f31f0004ab0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.949+0000 7f31fd5ce640 1 -- 172.21.15.43:0/1810584273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31f8103b00 msgr2=0x7f31f8101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.949+0000 7f31fd5ce640 1 --2- 172.21.15.43:0/1810584273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31f8103b00 0x7f31f8101370 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f31ec005940 tx=0x7f31ec0045e0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.950+0000 7f31fd5ce640 1 -- 172.21.15.43:0/1810584273 shutdown_connections 2024-04-21T08:48:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.950+0000 7f31fd5ce640 1 --2- 172.21.15.43:0/1810584273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31f810c520 0x7f31f81018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.950+0000 7f31fd5ce640 1 --2- 172.21.15.43:0/1810584273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31f8103b00 0x7f31f8101370 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.950+0000 7f31fd5ce640 1 --2- 172.21.15.43:0/1810584273 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f31e00a3e70 0x7f31e00a6330 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.950+0000 7f31fd5ce640 1 --2- 172.21.15.43:0/1810584273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f81031e0 0x7f31f8100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.950+0000 7f31fd5ce640 1 -- 172.21.15.43:0/1810584273 >> 172.21.15.43:0/1810584273 conn(0x7f31f8077fd0 msgr2=0x7f31f80fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.950+0000 7f31fd5ce640 1 -- 172.21.15.43:0/1810584273 shutdown_connections 2024-04-21T08:48:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:41.950+0000 7f31fd5ce640 1 -- 172.21.15.43:0/1810584273 wait complete. 2024-04-21T08:48:42.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:42 smithi175 ceph-mon[31920]: pgmap v303: 129 pgs: 129 active+clean; 4.0 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:48:42.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:42 smithi112 ceph-mon[31085]: pgmap v303: 129 pgs: 129 active+clean; 4.0 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:48:42.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:42 smithi043 ceph-mon[27978]: pgmap v303: 129 pgs: 129 active+clean; 4.0 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:48:44.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:44 smithi175 ceph-mon[31920]: pgmap v304: 129 pgs: 129 active+clean; 4.0 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-21T08:48:44.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:44 smithi112 ceph-mon[31085]: pgmap v304: 129 pgs: 129 active+clean; 4.0 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-21T08:48:44.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:44 smithi043 ceph-mon[27978]: pgmap v304: 129 pgs: 129 active+clean; 4.0 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-21T08:48:46.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:46 smithi175 ceph-mon[31920]: pgmap v305: 129 pgs: 129 active+clean; 4.0 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-21T08:48:46.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:46 smithi112 ceph-mon[31085]: pgmap v305: 129 pgs: 129 active+clean; 4.0 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-21T08:48:46.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:46 smithi043 ceph-mon[27978]: pgmap v305: 129 pgs: 129 active+clean; 4.0 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-21T08:48:48.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:48 smithi175 ceph-mon[31920]: pgmap v306: 129 pgs: 129 active+clean; 4.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-21T08:48:48.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:48 smithi112 ceph-mon[31085]: pgmap v306: 129 pgs: 129 active+clean; 4.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-21T08:48:48.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:48 smithi043 ceph-mon[27978]: pgmap v306: 129 pgs: 129 active+clean; 4.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-21T08:48:50.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:50 smithi175 ceph-mon[31920]: pgmap v307: 129 pgs: 129 active+clean; 4.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:48:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:50 smithi112 ceph-mon[31085]: pgmap v307: 129 pgs: 129 active+clean; 4.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:48:50.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:50 smithi043 ceph-mon[27978]: pgmap v307: 129 pgs: 129 active+clean; 4.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:48:52.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:52 smithi175 ceph-mon[31920]: pgmap v308: 129 pgs: 129 active+clean; 4.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-21T08:48:52.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:52 smithi112 ceph-mon[31085]: pgmap v308: 129 pgs: 129 active+clean; 4.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-21T08:48:52.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:52 smithi043 ceph-mon[27978]: pgmap v308: 129 pgs: 129 active+clean; 4.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-21T08:48:54.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:54 smithi175 ceph-mon[31920]: pgmap v309: 129 pgs: 129 active+clean; 4.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:48:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:54 smithi112 ceph-mon[31085]: pgmap v309: 129 pgs: 129 active+clean; 4.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:48:54.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:54 smithi043 ceph-mon[27978]: pgmap v309: 129 pgs: 129 active+clean; 4.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:48:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.050+0000 7f33fb9bd640 1 -- 172.21.15.43:0/2054317931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33f41063f0 msgr2=0x7f33f41068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.050+0000 7f33fb9bd640 1 --2- 172.21.15.43:0/2054317931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33f41063f0 0x7f33f41068a0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f33ec009cd0 tx=0x7f33ec02c180 comp rx=0 tx=0).stop 2024-04-21T08:48:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.050+0000 7f33fb9bd640 1 -- 172.21.15.43:0/2054317931 shutdown_connections 2024-04-21T08:48:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.050+0000 7f33fb9bd640 1 --2- 172.21.15.43:0/2054317931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33f41063f0 0x7f33f41068a0 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.050+0000 7f33fb9bd640 1 --2- 172.21.15.43:0/2054317931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33f4101910 0x7f33f4105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.050+0000 7f33fb9bd640 1 --2- 172.21.15.43:0/2054317931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33f4100ff0 0x7f33f41013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.050+0000 7f33fb9bd640 1 -- 172.21.15.43:0/2054317931 >> 172.21.15.43:0/2054317931 conn(0x7f33f40771d0 msgr2=0x7f33f40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.050+0000 7f33fb9bd640 1 -- 172.21.15.43:0/2054317931 shutdown_connections 2024-04-21T08:48:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.050+0000 7f33fb9bd640 1 -- 172.21.15.43:0/2054317931 wait complete. 2024-04-21T08:48:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.050+0000 7f33fb9bd640 1 Processor -- start 2024-04-21T08:48:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.051+0000 7f33fb9bd640 1 -- start start 2024-04-21T08:48:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.051+0000 7f33fb9bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33f4100ff0 0x7f33f41a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.051+0000 7f33fb9bd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33f4101910 0x7f33f41a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.051+0000 7f33fb9bd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33f41063f0 0x7f33f419ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.051+0000 7f33fb9bd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f33f41187f0 con 0x7f33f4101910 2024-04-21T08:48:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.051+0000 7f33fb9bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f33f4118670 con 0x7f33f4100ff0 2024-04-21T08:48:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.051+0000 7f33fb9bd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33f4118970 con 0x7f33f41063f0 2024-04-21T08:48:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.051+0000 7f33f8f58640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33f4101910 0x7f33f41a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.051+0000 7f33f8f58640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33f4101910 0x7f33f41a4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60060/0 (socket says 172.21.15.43:60060) 2024-04-21T08:48:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.051+0000 7f33f8f58640 1 -- 172.21.15.43:0/1435699862 learned_addr learned my addr 172.21.15.43:0/1435699862 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.051+0000 7f33f9f5a640 1 --2- 172.21.15.43:0/1435699862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33f41063f0 0x7f33f419ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.051+0000 7f33f9759640 1 --2- 172.21.15.43:0/1435699862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33f4100ff0 0x7f33f41a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.051+0000 7f33f8f58640 1 -- 172.21.15.43:0/1435699862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33f41063f0 msgr2=0x7f33f419ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.051+0000 7f33f8f58640 1 --2- 172.21.15.43:0/1435699862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33f41063f0 0x7f33f419ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.051+0000 7f33f8f58640 1 -- 172.21.15.43:0/1435699862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33f4100ff0 msgr2=0x7f33f41a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.051+0000 7f33f8f58640 1 --2- 172.21.15.43:0/1435699862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33f4100ff0 0x7f33f41a4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.051+0000 7f33f8f58640 1 -- 172.21.15.43:0/1435699862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f419f210 con 0x7f33f4101910 2024-04-21T08:48:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.051+0000 7f33f8f58640 1 --2- 172.21.15.43:0/1435699862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33f4101910 0x7f33f41a4dd0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f33e400e9f0 tx=0x7f33e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.052+0000 7f33ea7fc640 1 -- 172.21.15.43:0/1435699862 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e400cd80 con 0x7f33f4101910 2024-04-21T08:48:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.052+0000 7f33fb9bd640 1 -- 172.21.15.43:0/1435699862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33f419f500 con 0x7f33f4101910 2024-04-21T08:48:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.052+0000 7f33ea7fc640 1 -- 172.21.15.43:0/1435699862 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33e40108e0 con 0x7f33f4101910 2024-04-21T08:48:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.052+0000 7f33ea7fc640 1 -- 172.21.15.43:0/1435699862 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e4018980 con 0x7f33f4101910 2024-04-21T08:48:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.052+0000 7f33fb9bd640 1 -- 172.21.15.43:0/1435699862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33f41ab750 con 0x7f33f4101910 2024-04-21T08:48:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.054+0000 7f33ea7fc640 1 -- 172.21.15.43:0/1435699862 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f33e4010430 con 0x7f33f4101910 2024-04-21T08:48:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.054+0000 7f33fe5b5740 1 Processor -- start 2024-04-21T08:48:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.054+0000 7f33fe5b5740 1 -- start start 2024-04-21T08:48:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.055+0000 7f33fe5b5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55994e4058d0 0x55994e404620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.055+0000 7f33fe5b5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e404b60 0x55994e404fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.055+0000 7f33fe5b5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994e40e0e0 0x55994e4104d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.055+0000 7f33fe5b5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55994e3698d0 con 0x55994e404b60 2024-04-21T08:48:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.055+0000 7f33fe5b5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55994e306e80 con 0x55994e40e0e0 2024-04-21T08:48:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.055+0000 7f33fe5b5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55994e365890 con 0x55994e4058d0 2024-04-21T08:48:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.055+0000 7f33f9759640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e404b60 0x55994e404fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.055+0000 7f33ea7fc640 1 --2- 172.21.15.43:0/1435699862 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f33e00a3dc0 0x7f33e00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.055+0000 7f33f9759640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e404b60 0x55994e404fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60070/0 (socket says 172.21.15.43:60070) 2024-04-21T08:48:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.055+0000 7f33f9759640 1 -- 172.21.15.43:0/3628978896 learned_addr learned my addr 172.21.15.43:0/3628978896 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.055+0000 7f33f9f5a640 1 --2- 172.21.15.43:0/1435699862 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f33e00a3dc0 0x7f33e00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.055+0000 7f33f8f58640 1 --2- 172.21.15.43:0/3628978896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994e40e0e0 0x55994e4104d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.055+0000 7f33f9f5a640 1 --2- 172.21.15.43:0/3628978896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55994e4058d0 0x55994e404620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.055+0000 7f33ea7fc640 1 -- 172.21.15.43:0/1435699862 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f33e40d0f90 con 0x7f33f4101910 2024-04-21T08:48:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.055+0000 7f33f9759640 1 -- 172.21.15.43:0/3628978896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55994e4058d0 msgr2=0x55994e404620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.055+0000 7f33f9759640 1 --2- 172.21.15.43:0/3628978896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55994e4058d0 0x55994e404620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.055+0000 7f33f9759640 1 -- 172.21.15.43:0/3628978896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994e40e0e0 msgr2=0x55994e4104d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.055+0000 7f33f9759640 1 --2- 172.21.15.43:0/3628978896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994e40e0e0 0x55994e4104d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.055+0000 7f33f9759640 1 -- 172.21.15.43:0/3628978896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55994e328320 con 0x55994e404b60 2024-04-21T08:48:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.055+0000 7f33f9f5a640 1 --2- 172.21.15.43:0/3628978896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55994e4058d0 0x55994e404620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.055+0000 7f33f9759640 1 --2- 172.21.15.43:0/3628978896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e404b60 0x55994e404fe0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f33f000bc90 tx=0x7f33f000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f5264fb57cc8d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.056+0000 7f33df7fe640 1 -- 172.21.15.43:0/3628978896 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f0015070 con 0x55994e404b60 2024-04-21T08:48:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.056+0000 7f33f9f5a640 1 --2- 172.21.15.43:0/1435699862 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f33e00a3dc0 0x7f33e00a6280 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f33ec005cc0 tx=0x7f33ec02e040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.056+0000 7f33df7fe640 1 -- 172.21.15.43:0/3628978896 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33f001a910 con 0x55994e404b60 2024-04-21T08:48:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.056+0000 7f33df7fe640 1 -- 172.21.15.43:0/3628978896 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f00118c0 con 0x55994e404b60 2024-04-21T08:48:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.056+0000 7f33fe5b5740 1 -- 172.21.15.43:0/3628978896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e404b60 msgr2=0x55994e404fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.056+0000 7f33fe5b5740 1 --2- 172.21.15.43:0/3628978896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e404b60 0x55994e404fe0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f33f000bc90 tx=0x7f33f000f730 comp rx=0 tx=0).stop 2024-04-21T08:48:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.057+0000 7f33fe5b5740 1 -- 172.21.15.43:0/3628978896 shutdown_connections 2024-04-21T08:48:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.057+0000 7f33fe5b5740 1 --2- 172.21.15.43:0/3628978896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994e40e0e0 0x55994e4104d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.057+0000 7f33fe5b5740 1 --2- 172.21.15.43:0/3628978896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e404b60 0x55994e404fe0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.057+0000 7f33fe5b5740 1 --2- 172.21.15.43:0/3628978896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55994e4058d0 0x55994e404620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.057+0000 7f33fe5b5740 1 -- 172.21.15.43:0/3628978896 >> 172.21.15.43:0/3628978896 conn(0x55994e3cbc90 msgr2=0x55994e3ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.057+0000 7f33fe5b5740 1 -- 172.21.15.43:0/3628978896 shutdown_connections 2024-04-21T08:48:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.057+0000 7f33fe5b5740 1 -- 172.21.15.43:0/3628978896 wait complete. 2024-04-21T08:48:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.057+0000 7f33fe5b5740 1 -- start start 2024-04-21T08:48:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.057+0000 7f33fe5b5740 1 Processor -- start 2024-04-21T08:48:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.058+0000 7f33fe5b5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994e404b60 0x55994e40da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.058+0000 7f33fe5b5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55994e4058d0 0x55994e40ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.058+0000 7f33fe5b5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e40e0e0 0x55994e40b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.058+0000 7f33fe5b5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55994e410e10 con 0x55994e40e0e0 2024-04-21T08:48:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.058+0000 7f33fe5b5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55994e2f1eb0 con 0x55994e404b60 2024-04-21T08:48:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.058+0000 7f33fe5b5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55994e410f90 con 0x55994e4058d0 2024-04-21T08:48:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.058+0000 7f33fe5b5740 10 client.0 _ll_register_callbacks cb 0x55994e31eb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.058+0000 7f33f9f5a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e40e0e0 0x55994e40b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.058+0000 7f33f9f5a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e40e0e0 0x55994e40b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60080/0 (socket says 172.21.15.43:60080) 2024-04-21T08:48:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.058+0000 7f33f9f5a640 1 -- 172.21.15.43:0/1994227819 learned_addr learned my addr 172.21.15.43:0/1994227819 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.058+0000 7f33f8f58640 1 --2- 172.21.15.43:0/1994227819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55994e4058d0 0x55994e40ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.058+0000 7f33f9759640 1 --2- 172.21.15.43:0/1994227819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994e404b60 0x55994e40da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.058+0000 7f33f9f5a640 1 -- 172.21.15.43:0/1994227819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55994e4058d0 msgr2=0x55994e40ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.058+0000 7f33f9f5a640 1 --2- 172.21.15.43:0/1994227819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55994e4058d0 0x55994e40ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.058+0000 7f33f9f5a640 1 -- 172.21.15.43:0/1994227819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994e404b60 msgr2=0x55994e40da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.058+0000 7f33f9f5a640 1 --2- 172.21.15.43:0/1994227819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994e404b60 0x55994e40da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.058+0000 7f33f9f5a640 1 -- 172.21.15.43:0/1994227819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55994e07c210 con 0x55994e40e0e0 2024-04-21T08:48:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.058+0000 7f33f9f5a640 1 --2- 172.21.15.43:0/1994227819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e40e0e0 0x55994e40b3c0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f33ec009cd0 tx=0x7f33ec02c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6c76b3161ba00e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.058+0000 7f33dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.058+0000 7f33dcff9640 1 -- 172.21.15.43:0/1994227819 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec04f070 con 0x55994e40e0e0 2024-04-21T08:48:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.058+0000 7f33dcff9640 1 -- 172.21.15.43:0/1994227819 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f33ec034990 con 0x55994e40e0e0 2024-04-21T08:48:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.058+0000 7f33fe5b5740 1 -- 172.21.15.43:0/1994227819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33f001a910 con 0x55994e40e0e0 2024-04-21T08:48:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.059+0000 7f33dcff9640 1 -- 172.21.15.43:0/1994227819 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f33ec05b050 con 0x55994e40e0e0 2024-04-21T08:48:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.059+0000 7f33fe5b5740 10 client.20184 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.059+0000 7f33fe5b5740 1 -- 172.21.15.43:0/1994227819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55994e148620 con 0x55994e40e0e0 2024-04-21T08:48:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.059+0000 7f33dcff9640 1 -- 172.21.15.43:0/1994227819 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33f001a910 con 0x55994e40e0e0 2024-04-21T08:48:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.059+0000 7f33dcff9640 1 -- 172.21.15.43:0/1994227819 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f33ec04a040 con 0x55994e40e0e0 2024-04-21T08:48:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.059+0000 7f33fe5b5740 10 client.20184 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.059+0000 7f33fe5b5740 10 client.20184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.059+0000 7f33fe5b5740 20 client.20184 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.059+0000 7f33fe5b5740 1 --2- 172.21.15.43:0/1994227819 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55994e41e0b0 0x55994e4204a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.059+0000 7f33fe5b5740 4 client.20184 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.059+0000 7f33fe5b5740 1 -- 172.21.15.43:0/1994227819 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55994e37d880 con 0x55994e41e0b0 2024-04-21T08:48:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.059+0000 7f33f9759640 1 --2- 172.21.15.43:0/1994227819 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55994e41e0b0 0x55994e4204a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.060+0000 7f33dcff9640 10 client.20184 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.060+0000 7f33f9759640 1 --2- 172.21.15.43:0/1994227819 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55994e41e0b0 0x55994e4204a0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=222a0113deb1f07a server_cookie=53362631c333fc68 in_seq=0 out_seq=0 2024-04-21T08:48:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.060+0000 7f33dcff9640 1 -- 172.21.15.43:0/1994227819 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55994e37d880 con 0x55994e41e0b0 2024-04-21T08:48:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.060+0000 7f33dcff9640 10 client.20184 handle_command_reply: tid=0 2024-04-21T08:48:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.061+0000 7f33fe5b5740 1 client.20184 shutdown 2024-04-21T08:48:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.061+0000 7f33fe5b5740 1 -- 172.21.15.43:0/1994227819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e40e0e0 msgr2=0x55994e40b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.061+0000 7f33fe5b5740 1 --2- 172.21.15.43:0/1994227819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e40e0e0 0x55994e40b3c0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f33ec009cd0 tx=0x7f33ec02c180 comp rx=0 tx=0).stop 2024-04-21T08:48:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.061+0000 7f33fe5b5740 1 -- 172.21.15.43:0/1994227819 shutdown_connections 2024-04-21T08:48:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.061+0000 7f33fe5b5740 1 --2- 172.21.15.43:0/1994227819 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55994e41e0b0 0x55994e4204a0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.061+0000 7f33fe5b5740 1 --2- 172.21.15.43:0/1994227819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e40e0e0 0x55994e40b3c0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.061+0000 7f33dcff9640 0 client.20184 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.061+0000 7f33fe5b5740 1 --2- 172.21.15.43:0/1994227819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55994e4058d0 0x55994e40ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.061+0000 7f33fe5b5740 1 --2- 172.21.15.43:0/1994227819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994e404b60 0x55994e40da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.061+0000 7f33fe5b5740 1 -- 172.21.15.43:0/1994227819 >> 172.21.15.43:0/1994227819 conn(0x55994e3cbc90 msgr2=0x55994e4015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.061+0000 7f33fe5b5740 1 -- 172.21.15.43:0/1994227819 shutdown_connections 2024-04-21T08:48:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.061+0000 7f33fe5b5740 1 -- 172.21.15.43:0/1994227819 wait complete. 2024-04-21T08:48:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.061+0000 7f33fe5b5740 20 client.20184 trim_cache size 0 max 16384 2024-04-21T08:48:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.076+0000 7f33fe5b5740 1 Processor -- start 2024-04-21T08:48:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.076+0000 7f33fe5b5740 1 -- start start 2024-04-21T08:48:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.077+0000 7f33fe5b5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994e404b60 0x55994e419d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.077+0000 7f33fe5b5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e4058d0 0x55994e40d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.077+0000 7f33fe5b5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55994e40e0e0 0x55994e40da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.077+0000 7f33fe5b5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f33f41b5a00 con 0x55994e4058d0 2024-04-21T08:48:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.077+0000 7f33fe5b5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f33f41b7540 con 0x55994e404b60 2024-04-21T08:48:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.077+0000 7f33fe5b5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33f41b60d0 con 0x55994e40e0e0 2024-04-21T08:48:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.077+0000 7f33f8f58640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e4058d0 0x55994e40d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.077+0000 7f33f8f58640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e4058d0 0x55994e40d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60092/0 (socket says 172.21.15.43:60092) 2024-04-21T08:48:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.077+0000 7f33f8f58640 1 -- 172.21.15.43:0/2993881823 learned_addr learned my addr 172.21.15.43:0/2993881823 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.077+0000 7f33f9f5a640 1 --2- 172.21.15.43:0/2993881823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55994e40e0e0 0x55994e40da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.077+0000 7f33f9759640 1 --2- 172.21.15.43:0/2993881823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994e404b60 0x55994e419d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.077+0000 7f33f8f58640 1 -- 172.21.15.43:0/2993881823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55994e40e0e0 msgr2=0x55994e40da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.077+0000 7f33f8f58640 1 --2- 172.21.15.43:0/2993881823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55994e40e0e0 0x55994e40da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.077+0000 7f33f8f58640 1 -- 172.21.15.43:0/2993881823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994e404b60 msgr2=0x55994e419d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.077+0000 7f33f8f58640 1 --2- 172.21.15.43:0/2993881823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994e404b60 0x55994e419d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.077+0000 7f33f8f58640 1 -- 172.21.15.43:0/2993881823 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55994e410940 con 0x55994e4058d0 2024-04-21T08:48:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.077+0000 7f33f9759640 1 --2- 172.21.15.43:0/2993881823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994e404b60 0x55994e419d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.077+0000 7f33f8f58640 1 --2- 172.21.15.43:0/2993881823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e4058d0 0x55994e40d550 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f33e4099480 tx=0x7f33e401ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421b82ef2ff5707e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.077+0000 7f33deffd640 1 -- 172.21.15.43:0/2993881823 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e4026040 con 0x55994e4058d0 2024-04-21T08:48:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.078+0000 7f33deffd640 1 -- 172.21.15.43:0/2993881823 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33e409d1b0 con 0x55994e4058d0 2024-04-21T08:48:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.078+0000 7f33deffd640 1 -- 172.21.15.43:0/2993881823 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e4099120 con 0x55994e4058d0 2024-04-21T08:48:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.078+0000 7f33fe5b5740 1 -- 172.21.15.43:0/2993881823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e4058d0 msgr2=0x55994e40d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.078+0000 7f33fe5b5740 1 --2- 172.21.15.43:0/2993881823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e4058d0 0x55994e40d550 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f33e4099480 tx=0x7f33e401ff90 comp rx=0 tx=0).stop 2024-04-21T08:48:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.078+0000 7f33fe5b5740 1 -- 172.21.15.43:0/2993881823 shutdown_connections 2024-04-21T08:48:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.078+0000 7f33fe5b5740 1 --2- 172.21.15.43:0/2993881823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55994e40e0e0 0x55994e40da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.078+0000 7f33fe5b5740 1 --2- 172.21.15.43:0/2993881823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e4058d0 0x55994e40d550 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.078+0000 7f33fe5b5740 1 --2- 172.21.15.43:0/2993881823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994e404b60 0x55994e419d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.078+0000 7f33fe5b5740 1 -- 172.21.15.43:0/2993881823 >> 172.21.15.43:0/2993881823 conn(0x55994e3cbc90 msgr2=0x55994e406060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.078+0000 7f33fe5b5740 1 -- 172.21.15.43:0/2993881823 shutdown_connections 2024-04-21T08:48:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.078+0000 7f33fe5b5740 1 -- 172.21.15.43:0/2993881823 wait complete. 2024-04-21T08:48:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.079+0000 7f33fe5b5740 1 -- start start 2024-04-21T08:48:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.079+0000 7f33fe5b5740 1 Processor -- start 2024-04-21T08:48:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.079+0000 7f33fe5b5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e404b60 0x55994e4166f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.079+0000 7f33fe5b5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55994e4058d0 0x55994e408110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.079+0000 7f33fe5b5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994e40e0e0 0x55994e408740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.079+0000 7f33fe5b5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f33f41b6e70 con 0x55994e404b60 2024-04-21T08:48:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.079+0000 7f33fe5b5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f33f41b67a0 con 0x55994e40e0e0 2024-04-21T08:48:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.079+0000 7f33fe5b5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33f41b3a00 con 0x55994e4058d0 2024-04-21T08:48:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.079+0000 7f33fe5b5740 10 client.0 _ll_register_callbacks cb 0x7f33e4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.079+0000 7f33f9759640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e404b60 0x55994e4166f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.079+0000 7f33f9759640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e404b60 0x55994e4166f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60098/0 (socket says 172.21.15.43:60098) 2024-04-21T08:48:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.079+0000 7f33f9759640 1 -- 172.21.15.43:0/54221323 learned_addr learned my addr 172.21.15.43:0/54221323 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.079+0000 7f33f8f58640 1 --2- 172.21.15.43:0/54221323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55994e4058d0 0x55994e408110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.079+0000 7f33f9f5a640 1 --2- 172.21.15.43:0/54221323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994e40e0e0 0x55994e408740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.080+0000 7f33f9759640 1 -- 172.21.15.43:0/54221323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55994e4058d0 msgr2=0x55994e408110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.080+0000 7f33f9759640 1 --2- 172.21.15.43:0/54221323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55994e4058d0 0x55994e408110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.080+0000 7f33f9759640 1 -- 172.21.15.43:0/54221323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994e40e0e0 msgr2=0x55994e408740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.080+0000 7f33f9759640 1 --2- 172.21.15.43:0/54221323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994e40e0e0 0x55994e408740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.080+0000 7f33f9759640 1 -- 172.21.15.43:0/54221323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55994e3faf30 con 0x55994e404b60 2024-04-21T08:48:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.080+0000 7f33f9759640 1 --2- 172.21.15.43:0/54221323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e404b60 0x55994e4166f0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f33f000bbb0 tx=0x7f33f0005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b2519b3d4f31ca server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.080+0000 7f33dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.080+0000 7f33dcff9640 1 -- 172.21.15.43:0/54221323 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f002c060 con 0x55994e404b60 2024-04-21T08:48:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.080+0000 7f33dcff9640 1 -- 172.21.15.43:0/54221323 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f33f0005450 con 0x55994e404b60 2024-04-21T08:48:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.080+0000 7f33fe5b5740 1 -- 172.21.15.43:0/54221323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33e409d1b0 con 0x55994e404b60 2024-04-21T08:48:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.080+0000 7f33dcff9640 1 -- 172.21.15.43:0/54221323 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f33f0033050 con 0x55994e404b60 2024-04-21T08:48:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.080+0000 7f33fe5b5740 10 client.20190 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.080+0000 7f33fe5b5740 1 -- 172.21.15.43:0/54221323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55994e484680 con 0x55994e404b60 2024-04-21T08:48:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.080+0000 7f33dcff9640 1 -- 172.21.15.43:0/54221323 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33e409d1b0 con 0x55994e404b60 2024-04-21T08:48:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.081+0000 7f33dcff9640 1 -- 172.21.15.43:0/54221323 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f33f00115c0 con 0x55994e404b60 2024-04-21T08:48:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.081+0000 7f33fe5b5740 10 client.20190 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.081+0000 7f33fe5b5740 10 client.20190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.081+0000 7f33fe5b5740 20 client.20190 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.081+0000 7f33fe5b5740 1 --2- 172.21.15.43:0/54221323 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55994e41e0b0 0x55994e486ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.081+0000 7f33fe5b5740 4 client.20190 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:48:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.081+0000 7f33fe5b5740 1 -- 172.21.15.43:0/54221323 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55994e3093d0 con 0x55994e41e0b0 2024-04-21T08:48:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.081+0000 7f33f8f58640 1 --2- 172.21.15.43:0/54221323 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55994e41e0b0 0x55994e486ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.082+0000 7f33dcff9640 10 client.20190 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.082+0000 7f33f8f58640 1 --2- 172.21.15.43:0/54221323 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55994e41e0b0 0x55994e486ac0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=135c2a78c260877d server_cookie=789d568060c0869c in_seq=0 out_seq=0 2024-04-21T08:48:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.096+0000 7f33dcff9640 1 -- 172.21.15.43:0/54221323 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55994e3093d0 con 0x55994e41e0b0 2024-04-21T08:48:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.096+0000 7f33dcff9640 10 client.20190 handle_command_reply: tid=0 2024-04-21T08:48:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.096+0000 7f33fe5b5740 1 client.20190 shutdown 2024-04-21T08:48:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.097+0000 7f33fe5b5740 1 -- 172.21.15.43:0/54221323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e404b60 msgr2=0x55994e4166f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.097+0000 7f33fe5b5740 1 --2- 172.21.15.43:0/54221323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e404b60 0x55994e4166f0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f33f000bbb0 tx=0x7f33f0005a70 comp rx=0 tx=0).stop 2024-04-21T08:48:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.097+0000 7f33fe5b5740 1 -- 172.21.15.43:0/54221323 shutdown_connections 2024-04-21T08:48:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.097+0000 7f33fe5b5740 1 --2- 172.21.15.43:0/54221323 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55994e41e0b0 0x55994e486ac0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.097+0000 7f33fe5b5740 1 --2- 172.21.15.43:0/54221323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55994e40e0e0 0x55994e408740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.097+0000 7f33fe5b5740 1 --2- 172.21.15.43:0/54221323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55994e4058d0 0x55994e408110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.097+0000 7f33dcff9640 0 client.20190 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.097+0000 7f33fe5b5740 1 --2- 172.21.15.43:0/54221323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55994e404b60 0x55994e4166f0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.097+0000 7f33fe5b5740 1 -- 172.21.15.43:0/54221323 >> 172.21.15.43:0/54221323 conn(0x55994e3cbc90 msgr2=0x55994e4096b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.097+0000 7f33fe5b5740 1 -- 172.21.15.43:0/54221323 shutdown_connections 2024-04-21T08:48:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.097+0000 7f33fe5b5740 1 -- 172.21.15.43:0/54221323 wait complete. 2024-04-21T08:48:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.097+0000 7f33fe5b5740 20 client.20190 trim_cache size 0 max 16384 2024-04-21T08:48:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:48:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:48:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:48:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:48:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.098+0000 7f33fb9bd640 1 -- 172.21.15.43:0/1435699862 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f33e00a3dc0 msgr2=0x7f33e00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.098+0000 7f33fb9bd640 1 --2- 172.21.15.43:0/1435699862 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f33e00a3dc0 0x7f33e00a6280 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f33ec005cc0 tx=0x7f33ec02e040 comp rx=0 tx=0).stop 2024-04-21T08:48:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.098+0000 7f33fb9bd640 1 -- 172.21.15.43:0/1435699862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33f4101910 msgr2=0x7f33f41a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.098+0000 7f33fb9bd640 1 --2- 172.21.15.43:0/1435699862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33f4101910 0x7f33f41a4dd0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f33e400e9f0 tx=0x7f33e400eeb0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.098+0000 7f33fb9bd640 1 -- 172.21.15.43:0/1435699862 shutdown_connections 2024-04-21T08:48:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.098+0000 7f33fb9bd640 1 --2- 172.21.15.43:0/1435699862 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f33e00a3dc0 0x7f33e00a6280 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.098+0000 7f33fb9bd640 1 --2- 172.21.15.43:0/1435699862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33f41063f0 0x7f33f419ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.098+0000 7f33fb9bd640 1 --2- 172.21.15.43:0/1435699862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33f4101910 0x7f33f41a4dd0 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.098+0000 7f33fb9bd640 1 --2- 172.21.15.43:0/1435699862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33f4100ff0 0x7f33f41a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.098+0000 7f33fb9bd640 1 -- 172.21.15.43:0/1435699862 >> 172.21.15.43:0/1435699862 conn(0x7f33f40771d0 msgr2=0x7f33f4103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.098+0000 7f33fb9bd640 1 -- 172.21.15.43:0/1435699862 shutdown_connections 2024-04-21T08:48:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:48:55.098+0000 7f33fb9bd640 1 -- 172.21.15.43:0/1435699862 wait complete. 2024-04-21T08:48:56.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:56 smithi175 ceph-mon[31920]: pgmap v310: 129 pgs: 129 active+clean; 4.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:48:56.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:56 smithi112 ceph-mon[31085]: pgmap v310: 129 pgs: 129 active+clean; 4.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:48:56.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:56 smithi043 ceph-mon[27978]: pgmap v310: 129 pgs: 129 active+clean; 4.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:48:56.760 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:48:56.760 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:48:56.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.879+0000 7f3e6cdf4640 1 -- 172.21.15.43:0/1450777457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e68101a30 msgr2=0x7f3e68105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:56.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.879+0000 7f3e6cdf4640 1 --2- 172.21.15.43:0/1450777457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e68101a30 0x7f3e68105a70 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009900 tx=0x7f3e5c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:56.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.879+0000 7f3e6cdf4640 1 -- 172.21.15.43:0/1450777457 shutdown_connections 2024-04-21T08:48:56.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.879+0000 7f3e6cdf4640 1 --2- 172.21.15.43:0/1450777457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e68106150 0x7f3e6810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.879+0000 7f3e6cdf4640 1 --2- 172.21.15.43:0/1450777457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e68101a30 0x7f3e68105a70 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.879+0000 7f3e6cdf4640 1 --2- 172.21.15.43:0/1450777457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e68101080 0x7f3e68101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.879+0000 7f3e6cdf4640 1 -- 172.21.15.43:0/1450777457 >> 172.21.15.43:0/1450777457 conn(0x7f3e68078000 msgr2=0x7f3e68078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:56.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.879+0000 7f3e6cdf4640 1 -- 172.21.15.43:0/1450777457 shutdown_connections 2024-04-21T08:48:56.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.879+0000 7f3e6cdf4640 1 -- 172.21.15.43:0/1450777457 wait complete. 2024-04-21T08:48:56.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.880+0000 7f3e6cdf4640 1 Processor -- start 2024-04-21T08:48:56.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.880+0000 7f3e6cdf4640 1 -- start start 2024-04-21T08:48:56.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.880+0000 7f3e6cdf4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e68101080 0x7f3e681a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:56.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.880+0000 7f3e6cdf4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e68106150 0x7f3e681a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:56.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.880+0000 7f3e6cdf4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e6819a650 0x7f3e6819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:56.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.880+0000 7f3e6cdf4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3e68112190 con 0x7f3e68106150 2024-04-21T08:48:56.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.880+0000 7f3e6cdf4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e68112010 con 0x7f3e6819a650 2024-04-21T08:48:56.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.880+0000 7f3e6cdf4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3e68112310 con 0x7f3e68101080 2024-04-21T08:48:56.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.880+0000 7f3e65d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e68106150 0x7f3e681a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:56.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.880+0000 7f3e65d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e68106150 0x7f3e681a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47336/0 (socket says 172.21.15.43:47336) 2024-04-21T08:48:56.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.880+0000 7f3e65d9b640 1 -- 172.21.15.43:0/1718432852 learned_addr learned my addr 172.21.15.43:0/1718432852 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:56.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.880+0000 7f3e6659c640 1 --2- 172.21.15.43:0/1718432852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e68101080 0x7f3e681a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:56.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.880+0000 7f3e66d9d640 1 --2- 172.21.15.43:0/1718432852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e6819a650 0x7f3e6819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:56.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.880+0000 7f3e65d9b640 1 -- 172.21.15.43:0/1718432852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e68101080 msgr2=0x7f3e681a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:56.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.880+0000 7f3e65d9b640 1 --2- 172.21.15.43:0/1718432852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e68101080 0x7f3e681a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.880+0000 7f3e65d9b640 1 -- 172.21.15.43:0/1718432852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e6819a650 msgr2=0x7f3e6819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:56.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.880+0000 7f3e65d9b640 1 --2- 172.21.15.43:0/1718432852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e6819a650 0x7f3e6819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.880+0000 7f3e65d9b640 1 -- 172.21.15.43:0/1718432852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6819b250 con 0x7f3e68106150 2024-04-21T08:48:56.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.880+0000 7f3e6659c640 1 --2- 172.21.15.43:0/1718432852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e68101080 0x7f3e681a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:56.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.881+0000 7f3e65d9b640 1 --2- 172.21.15.43:0/1718432852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e68106150 0x7f3e681a0ad0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c005ec0 tx=0x7f3e5c02ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:56.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.881+0000 7f3e577fe640 1 -- 172.21.15.43:0/1718432852 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c043020 con 0x7f3e68106150 2024-04-21T08:48:56.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.881+0000 7f3e577fe640 1 -- 172.21.15.43:0/1718432852 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e5c0052b0 con 0x7f3e68106150 2024-04-21T08:48:56.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.881+0000 7f3e6cdf4640 1 -- 172.21.15.43:0/1718432852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e6819b4e0 con 0x7f3e68106150 2024-04-21T08:48:56.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.881+0000 7f3e577fe640 1 -- 172.21.15.43:0/1718432852 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c003d40 con 0x7f3e68106150 2024-04-21T08:48:56.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.881+0000 7f3e6cdf4640 1 -- 172.21.15.43:0/1718432852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e681a76d0 con 0x7f3e68106150 2024-04-21T08:48:56.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.883+0000 7f3e577fe640 1 -- 172.21.15.43:0/1718432852 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3e5c046020 con 0x7f3e68106150 2024-04-21T08:48:56.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.883+0000 7f3e6f9e7740 1 Processor -- start 2024-04-21T08:48:56.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.884+0000 7f3e6f9e7740 1 -- start start 2024-04-21T08:48:56.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.884+0000 7f3e6f9e7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df14580 0x560d1df14960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:56.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.884+0000 7f3e6f9e7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1df14ea0 0x560d1df1d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:56.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.884+0000 7f3e6f9e7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d1df1da80 0x560d1df1fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:56.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.884+0000 7f3e6f9e7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560d1de791e0 con 0x560d1df14580 2024-04-21T08:48:56.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.884+0000 7f3e6f9e7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560d1de16390 con 0x560d1df14ea0 2024-04-21T08:48:56.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.884+0000 7f3e6f9e7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560d1de751a0 con 0x560d1df1da80 2024-04-21T08:48:56.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.884+0000 7f3e66d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df14580 0x560d1df14960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:56.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.884+0000 7f3e66d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df14580 0x560d1df14960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47340/0 (socket says 172.21.15.43:47340) 2024-04-21T08:48:56.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.884+0000 7f3e66d9d640 1 -- 172.21.15.43:0/1353566450 learned_addr learned my addr 172.21.15.43:0/1353566450 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:56.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.884+0000 7f3e6659c640 1 --2- 172.21.15.43:0/1353566450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1df14ea0 0x560d1df1d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:56.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.884+0000 7f3e65d9b640 1 --2- 172.21.15.43:0/1353566450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d1df1da80 0x560d1df1fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:56.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.884+0000 7f3e577fe640 1 --2- 172.21.15.43:0/1718432852 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3e500a3da0 0x7f3e500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:56.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.884+0000 7f3e66d9d640 1 -- 172.21.15.43:0/1353566450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d1df1da80 msgr2=0x560d1df1fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:56.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.884+0000 7f3e66d9d640 1 --2- 172.21.15.43:0/1353566450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d1df1da80 0x560d1df1fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.884+0000 7f3e66d9d640 1 -- 172.21.15.43:0/1353566450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1df14ea0 msgr2=0x560d1df1d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:56.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.884+0000 7f3e66d9d640 1 --2- 172.21.15.43:0/1353566450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1df14ea0 0x560d1df1d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.884+0000 7f3e66d9d640 1 -- 172.21.15.43:0/1353566450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d1dc557d0 con 0x560d1df14580 2024-04-21T08:48:56.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.884+0000 7f3e6659c640 1 --2- 172.21.15.43:0/1353566450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1df14ea0 0x560d1df1d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:56.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.885+0000 7f3e577fe640 1 -- 172.21.15.43:0/1718432852 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f3e5c0f2ef0 con 0x7f3e68106150 2024-04-21T08:48:56.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.885+0000 7f3e66d9d640 1 --2- 172.21.15.43:0/1718432852 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3e500a3da0 0x7f3e500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:56.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.885+0000 7f3e66d9d640 1 --2- 172.21.15.43:0/1353566450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df14580 0x560d1df14960 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f3e60007f50 tx=0x7f3e6000e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b586f86c8da65e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:56.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.885+0000 7f3e54ff9640 1 -- 172.21.15.43:0/1353566450 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e60022070 con 0x560d1df14580 2024-04-21T08:48:56.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.885+0000 7f3e54ff9640 1 -- 172.21.15.43:0/1353566450 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e60011b70 con 0x560d1df14580 2024-04-21T08:48:56.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.885+0000 7f3e54ff9640 1 -- 172.21.15.43:0/1353566450 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6001cd80 con 0x560d1df14580 2024-04-21T08:48:56.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.886+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/1353566450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df14580 msgr2=0x560d1df14960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:56.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.886+0000 7f3e66d9d640 1 --2- 172.21.15.43:0/1718432852 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3e500a3da0 0x7f3e500a6260 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3e6001f000 tx=0x7f3e60020000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:56.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.886+0000 7f3e6f9e7740 1 --2- 172.21.15.43:0/1353566450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df14580 0x560d1df14960 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f3e60007f50 tx=0x7f3e6000e7e0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.886+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/1353566450 shutdown_connections 2024-04-21T08:48:56.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.886+0000 7f3e6f9e7740 1 --2- 172.21.15.43:0/1353566450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d1df1da80 0x560d1df1fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.886+0000 7f3e6f9e7740 1 --2- 172.21.15.43:0/1353566450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1df14ea0 0x560d1df1d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.886+0000 7f3e6f9e7740 1 --2- 172.21.15.43:0/1353566450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df14580 0x560d1df14960 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.886+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/1353566450 >> 172.21.15.43:0/1353566450 conn(0x560d1df0ecb0 msgr2=0x560d1df0f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:56.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.886+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/1353566450 shutdown_connections 2024-04-21T08:48:56.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.886+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/1353566450 wait complete. 2024-04-21T08:48:56.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.886+0000 7f3e6f9e7740 1 -- start start 2024-04-21T08:48:56.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.886+0000 7f3e6f9e7740 1 Processor -- start 2024-04-21T08:48:56.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.887+0000 7f3e6f9e7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d1df14580 0x560d1df1ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:56.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.887+0000 7f3e6f9e7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1df14ea0 0x560d1df1b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:56.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.887+0000 7f3e6f9e7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df1da80 0x560d1df1b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:56.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.887+0000 7f3e6f9e7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560d1df20870 con 0x560d1df1da80 2024-04-21T08:48:56.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.887+0000 7f3e6f9e7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560d1de322f0 con 0x560d1df14ea0 2024-04-21T08:48:56.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.887+0000 7f3e6f9e7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560d1df209f0 con 0x560d1df14580 2024-04-21T08:48:56.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.887+0000 7f3e6f9e7740 10 client.0 _ll_register_callbacks cb 0x560d1de3f580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:56.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.887+0000 7f3e66d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df1da80 0x560d1df1b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:56.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.887+0000 7f3e66d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df1da80 0x560d1df1b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47344/0 (socket says 172.21.15.43:47344) 2024-04-21T08:48:56.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.887+0000 7f3e66d9d640 1 -- 172.21.15.43:0/1592934965 learned_addr learned my addr 172.21.15.43:0/1592934965 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:56.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.887+0000 7f3e65d9b640 1 --2- 172.21.15.43:0/1592934965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1df14ea0 0x560d1df1b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:56.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.887+0000 7f3e6659c640 1 --2- 172.21.15.43:0/1592934965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d1df14580 0x560d1df1ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:56.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.887+0000 7f3e66d9d640 1 -- 172.21.15.43:0/1592934965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d1df14580 msgr2=0x560d1df1ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:56.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.887+0000 7f3e66d9d640 1 --2- 172.21.15.43:0/1592934965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d1df14580 0x560d1df1ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.887+0000 7f3e66d9d640 1 -- 172.21.15.43:0/1592934965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1df14ea0 msgr2=0x560d1df1b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:56.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.887+0000 7f3e66d9d640 1 --2- 172.21.15.43:0/1592934965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1df14ea0 0x560d1df1b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.887+0000 7f3e66d9d640 1 -- 172.21.15.43:0/1592934965 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d1db8b3f0 con 0x560d1df1da80 2024-04-21T08:48:56.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.887+0000 7f3e6659c640 1 --2- 172.21.15.43:0/1592934965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d1df14580 0x560d1df1ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:56.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.887+0000 7f3e66d9d640 1 --2- 172.21.15.43:0/1592934965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df1da80 0x560d1df1b580 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f3e60011f80 tx=0x7f3e600076b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a2e950fe8004a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:56.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.887+0000 7f3e4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:56.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.887+0000 7f3e4dffb640 1 -- 172.21.15.43:0/1592934965 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e60022040 con 0x560d1df1da80 2024-04-21T08:48:56.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.887+0000 7f3e4dffb640 1 -- 172.21.15.43:0/1592934965 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f3e60011970 con 0x560d1df1da80 2024-04-21T08:48:56.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.887+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/1592934965 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e60011b70 con 0x560d1df1da80 2024-04-21T08:48:56.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.888+0000 7f3e4dffb640 1 -- 172.21.15.43:0/1592934965 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3e6003a050 con 0x560d1df1da80 2024-04-21T08:48:56.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.888+0000 7f3e6f9e7740 10 client.20202 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:56.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.888+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/1592934965 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560d1db4b0f0 con 0x560d1df1da80 2024-04-21T08:48:56.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.888+0000 7f3e4dffb640 1 -- 172.21.15.43:0/1592934965 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e60011b70 con 0x560d1df1da80 2024-04-21T08:48:56.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.888+0000 7f3e4dffb640 1 -- 172.21.15.43:0/1592934965 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3e60019040 con 0x560d1df1da80 2024-04-21T08:48:56.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.888+0000 7f3e6f9e7740 10 client.20202 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:56.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.888+0000 7f3e6f9e7740 10 client.20202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:56.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.888+0000 7f3e6f9e7740 20 client.20202 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:56.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.888+0000 7f3e6f9e7740 1 --2- 172.21.15.43:0/1592934965 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560d1df2dac0 0x560d1df2feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:56.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.888+0000 7f3e6f9e7740 4 client.20202 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:56.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.888+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/1592934965 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d1de86c00 con 0x560d1df2dac0 2024-04-21T08:48:56.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.888+0000 7f3e6659c640 1 --2- 172.21.15.43:0/1592934965 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560d1df2dac0 0x560d1df2feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:56.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.889+0000 7f3e4dffb640 10 client.20202 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:56.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.889+0000 7f3e6659c640 1 --2- 172.21.15.43:0/1592934965 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560d1df2dac0 0x560d1df2feb0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7ac225b6d5a2eaa server_cookie=6466dbf5a84be4c in_seq=0 out_seq=0 2024-04-21T08:48:56.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.890+0000 7f3e4dffb640 1 -- 172.21.15.43:0/1592934965 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d1de86c00 con 0x560d1df2dac0 2024-04-21T08:48:56.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.890+0000 7f3e4dffb640 10 client.20202 handle_command_reply: tid=0 2024-04-21T08:48:56.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.890+0000 7f3e6f9e7740 1 client.20202 shutdown 2024-04-21T08:48:56.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.890+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/1592934965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df1da80 msgr2=0x560d1df1b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:56.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.890+0000 7f3e6f9e7740 1 --2- 172.21.15.43:0/1592934965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df1da80 0x560d1df1b580 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f3e60011f80 tx=0x7f3e600076b0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.890+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/1592934965 shutdown_connections 2024-04-21T08:48:56.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.890+0000 7f3e6f9e7740 1 --2- 172.21.15.43:0/1592934965 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560d1df2dac0 0x560d1df2feb0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.890+0000 7f3e6f9e7740 1 --2- 172.21.15.43:0/1592934965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df1da80 0x560d1df1b580 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.890+0000 7f3e6f9e7740 1 --2- 172.21.15.43:0/1592934965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1df14ea0 0x560d1df1b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.890+0000 7f3e6f9e7740 1 --2- 172.21.15.43:0/1592934965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d1df14580 0x560d1df1ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.890+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/1592934965 >> 172.21.15.43:0/1592934965 conn(0x560d1df0ecb0 msgr2=0x560d1df10a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:56.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.890+0000 7f3e4dffb640 0 client.20202 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:56.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.890+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/1592934965 shutdown_connections 2024-04-21T08:48:56.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.890+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/1592934965 wait complete. 2024-04-21T08:48:56.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.890+0000 7f3e6f9e7740 20 client.20202 trim_cache size 0 max 16384 2024-04-21T08:48:56.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.909+0000 7f3e6f9e7740 1 Processor -- start 2024-04-21T08:48:56.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.909+0000 7f3e6f9e7740 1 -- start start 2024-04-21T08:48:56.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.909+0000 7f3e6f9e7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df14580 0x560d1df1d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:56.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.909+0000 7f3e6f9e7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d1df14ea0 0x560d1df29730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:56.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.909+0000 7f3e6f9e7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1df1da80 0x560d1df1b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:56.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.909+0000 7f3e6f9e7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3e681b1560 con 0x560d1df14580 2024-04-21T08:48:56.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.909+0000 7f3e6f9e7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e681b30a0 con 0x560d1df1da80 2024-04-21T08:48:56.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.909+0000 7f3e6f9e7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3e681b1c30 con 0x560d1df14ea0 2024-04-21T08:48:56.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.909+0000 7f3e6659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df14580 0x560d1df1d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:56.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.909+0000 7f3e6659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df14580 0x560d1df1d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47360/0 (socket says 172.21.15.43:47360) 2024-04-21T08:48:56.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.909+0000 7f3e6659c640 1 -- 172.21.15.43:0/3621800686 learned_addr learned my addr 172.21.15.43:0/3621800686 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:56.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.909+0000 7f3e65d9b640 1 --2- 172.21.15.43:0/3621800686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d1df14ea0 0x560d1df29730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:56.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.909+0000 7f3e66d9d640 1 --2- 172.21.15.43:0/3621800686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1df1da80 0x560d1df1b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:56.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.909+0000 7f3e6659c640 1 -- 172.21.15.43:0/3621800686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d1df14ea0 msgr2=0x560d1df29730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:56.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.909+0000 7f3e6659c640 1 --2- 172.21.15.43:0/3621800686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d1df14ea0 0x560d1df29730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.909+0000 7f3e6659c640 1 -- 172.21.15.43:0/3621800686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1df1da80 msgr2=0x560d1df1b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:56.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.909+0000 7f3e6659c640 1 --2- 172.21.15.43:0/3621800686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1df1da80 0x560d1df1b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.909+0000 7f3e6659c640 1 -- 172.21.15.43:0/3621800686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d1df14cd0 con 0x560d1df14580 2024-04-21T08:48:56.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.909+0000 7f3e6659c640 1 --2- 172.21.15.43:0/3621800686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df14580 0x560d1df1d230 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f3e58008ab0 tx=0x7f3e580084c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7bb921e3800f409 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:56.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.910+0000 7f3e4ffff640 1 -- 172.21.15.43:0/3621800686 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58004050 con 0x560d1df14580 2024-04-21T08:48:56.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.910+0000 7f3e4ffff640 1 -- 172.21.15.43:0/3621800686 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e58020070 con 0x560d1df14580 2024-04-21T08:48:56.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.910+0000 7f3e4ffff640 1 -- 172.21.15.43:0/3621800686 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5801a6d0 con 0x560d1df14580 2024-04-21T08:48:56.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.910+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/3621800686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df14580 msgr2=0x560d1df1d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:56.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.910+0000 7f3e6f9e7740 1 --2- 172.21.15.43:0/3621800686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df14580 0x560d1df1d230 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f3e58008ab0 tx=0x7f3e580084c0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.910+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/3621800686 shutdown_connections 2024-04-21T08:48:56.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.910+0000 7f3e6f9e7740 1 --2- 172.21.15.43:0/3621800686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1df1da80 0x560d1df1b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.910+0000 7f3e6f9e7740 1 --2- 172.21.15.43:0/3621800686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d1df14ea0 0x560d1df29730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.910+0000 7f3e6f9e7740 1 --2- 172.21.15.43:0/3621800686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df14580 0x560d1df1d230 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.910+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/3621800686 >> 172.21.15.43:0/3621800686 conn(0x560d1df0ecb0 msgr2=0x560d1df1aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:56.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.910+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/3621800686 shutdown_connections 2024-04-21T08:48:56.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.910+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/3621800686 wait complete. 2024-04-21T08:48:56.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.911+0000 7f3e6f9e7740 1 -- start start 2024-04-21T08:48:56.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.911+0000 7f3e6f9e7740 1 Processor -- start 2024-04-21T08:48:56.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.911+0000 7f3e6f9e7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1df14580 0x560d1df26050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:56.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.911+0000 7f3e6f9e7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df14ea0 0x560d1df174d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:56.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.911+0000 7f3e6f9e7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d1df1da80 0x560d1df17a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:56.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.911+0000 7f3e6f9e7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3e681b29d0 con 0x560d1df14ea0 2024-04-21T08:48:56.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.911+0000 7f3e6f9e7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e681b2300 con 0x560d1df14580 2024-04-21T08:48:56.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.911+0000 7f3e6f9e7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3e681af560 con 0x560d1df1da80 2024-04-21T08:48:56.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.911+0000 7f3e6f9e7740 10 client.0 _ll_register_callbacks cb 0x7f3e60020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:56.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.911+0000 7f3e65d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df14ea0 0x560d1df174d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:56.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.911+0000 7f3e65d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df14ea0 0x560d1df174d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47364/0 (socket says 172.21.15.43:47364) 2024-04-21T08:48:56.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.911+0000 7f3e65d9b640 1 -- 172.21.15.43:0/3683297701 learned_addr learned my addr 172.21.15.43:0/3683297701 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:56.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.911+0000 7f3e6659c640 1 --2- 172.21.15.43:0/3683297701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1df14580 0x560d1df26050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:56.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.911+0000 7f3e66d9d640 1 --2- 172.21.15.43:0/3683297701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d1df1da80 0x560d1df17a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:56.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.911+0000 7f3e65d9b640 1 -- 172.21.15.43:0/3683297701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d1df1da80 msgr2=0x560d1df17a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:56.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.911+0000 7f3e65d9b640 1 --2- 172.21.15.43:0/3683297701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d1df1da80 0x560d1df17a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.911+0000 7f3e65d9b640 1 -- 172.21.15.43:0/3683297701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1df14580 msgr2=0x560d1df26050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:56.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.911+0000 7f3e65d9b640 1 --2- 172.21.15.43:0/3683297701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1df14580 0x560d1df26050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.911+0000 7f3e65d9b640 1 -- 172.21.15.43:0/3683297701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d1df181f0 con 0x560d1df14ea0 2024-04-21T08:48:56.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.911+0000 7f3e65d9b640 1 --2- 172.21.15.43:0/3683297701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df14ea0 0x560d1df174d0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0be820 tx=0x7f3e5c0badc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ead845f0916af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:56.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.911+0000 7f3e4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:56.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.912+0000 7f3e4dffb640 1 -- 172.21.15.43:0/3683297701 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c0be480 con 0x560d1df14ea0 2024-04-21T08:48:56.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.912+0000 7f3e4dffb640 1 -- 172.21.15.43:0/3683297701 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f3e5c005460 con 0x560d1df14ea0 2024-04-21T08:48:56.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.912+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/3683297701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e58020070 con 0x560d1df14ea0 2024-04-21T08:48:56.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.912+0000 7f3e4dffb640 1 -- 172.21.15.43:0/3683297701 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3e5c0bf220 con 0x560d1df14ea0 2024-04-21T08:48:56.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.912+0000 7f3e6f9e7740 10 client.20208 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:56.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.912+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/3683297701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560d1df93fe0 con 0x560d1df14ea0 2024-04-21T08:48:56.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.912+0000 7f3e4dffb640 1 -- 172.21.15.43:0/3683297701 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e58020070 con 0x560d1df14ea0 2024-04-21T08:48:56.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.912+0000 7f3e4dffb640 1 -- 172.21.15.43:0/3683297701 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3e5c0ddce0 con 0x560d1df14ea0 2024-04-21T08:48:56.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.912+0000 7f3e6f9e7740 10 client.20208 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:56.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.912+0000 7f3e6f9e7740 10 client.20208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:56.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.912+0000 7f3e6f9e7740 20 client.20208 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:56.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.912+0000 7f3e6f9e7740 1 --2- 172.21.15.43:0/3683297701 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560d1df94760 0x560d1df96b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:56.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.912+0000 7f3e6f9e7740 4 client.20208 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:48:56.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.912+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/3683297701 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560d1de80a70 con 0x560d1df94760 2024-04-21T08:48:56.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.912+0000 7f3e6659c640 1 --2- 172.21.15.43:0/3683297701 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560d1df94760 0x560d1df96b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:56.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.913+0000 7f3e4dffb640 10 client.20208 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:56.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.913+0000 7f3e6659c640 1 --2- 172.21.15.43:0/3683297701 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560d1df94760 0x560d1df96b80 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=724a5155111c00c server_cookie=6c1204c9f5aa777d in_seq=0 out_seq=0 2024-04-21T08:48:56.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.913+0000 7f3e4dffb640 1 -- 172.21.15.43:0/3683297701 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560d1de80a70 con 0x560d1df94760 2024-04-21T08:48:56.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.913+0000 7f3e4dffb640 10 client.20208 handle_command_reply: tid=0 2024-04-21T08:48:56.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.913+0000 7f3e6f9e7740 1 client.20208 shutdown 2024-04-21T08:48:56.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.913+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/3683297701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df14ea0 msgr2=0x560d1df174d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:56.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.913+0000 7f3e6f9e7740 1 --2- 172.21.15.43:0/3683297701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df14ea0 0x560d1df174d0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0be820 tx=0x7f3e5c0badc0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.913+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/3683297701 shutdown_connections 2024-04-21T08:48:56.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.913+0000 7f3e6f9e7740 1 --2- 172.21.15.43:0/3683297701 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560d1df94760 0x560d1df96b80 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.913+0000 7f3e6f9e7740 1 --2- 172.21.15.43:0/3683297701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d1df1da80 0x560d1df17a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.913+0000 7f3e6f9e7740 1 --2- 172.21.15.43:0/3683297701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d1df14ea0 0x560d1df174d0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.913+0000 7f3e6f9e7740 1 --2- 172.21.15.43:0/3683297701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1df14580 0x560d1df26050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.913+0000 7f3e4dffb640 0 client.20208 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:56.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.913+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/3683297701 >> 172.21.15.43:0/3683297701 conn(0x560d1df0ecb0 msgr2=0x560d1df1b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:56.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.913+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/3683297701 shutdown_connections 2024-04-21T08:48:56.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.913+0000 7f3e6f9e7740 1 -- 172.21.15.43:0/3683297701 wait complete. 2024-04-21T08:48:56.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.913+0000 7f3e6f9e7740 20 client.20208 trim_cache size 0 max 16384 2024-04-21T08:48:56.922 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:48:56.922 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:48:56.922 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:48:56.922 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:48:56.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.914+0000 7f3e6cdf4640 1 -- 172.21.15.43:0/1718432852 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3e500a3da0 msgr2=0x7f3e500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:56.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.914+0000 7f3e6cdf4640 1 --2- 172.21.15.43:0/1718432852 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3e500a3da0 0x7f3e500a6260 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3e6001f000 tx=0x7f3e60020000 comp rx=0 tx=0).stop 2024-04-21T08:48:56.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.914+0000 7f3e6cdf4640 1 -- 172.21.15.43:0/1718432852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e68106150 msgr2=0x7f3e681a0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:56.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.914+0000 7f3e6cdf4640 1 --2- 172.21.15.43:0/1718432852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e68106150 0x7f3e681a0ad0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c005ec0 tx=0x7f3e5c02ca10 comp rx=0 tx=0).stop 2024-04-21T08:48:56.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.914+0000 7f3e6cdf4640 1 -- 172.21.15.43:0/1718432852 shutdown_connections 2024-04-21T08:48:56.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.914+0000 7f3e6cdf4640 1 --2- 172.21.15.43:0/1718432852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e6819a650 0x7f3e6819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.914+0000 7f3e6cdf4640 1 --2- 172.21.15.43:0/1718432852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e68106150 0x7f3e681a0ad0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.914+0000 7f3e6cdf4640 1 --2- 172.21.15.43:0/1718432852 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3e500a3da0 0x7f3e500a6260 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.914+0000 7f3e6cdf4640 1 --2- 172.21.15.43:0/1718432852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e68101080 0x7f3e681a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:56.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.914+0000 7f3e6cdf4640 1 -- 172.21.15.43:0/1718432852 >> 172.21.15.43:0/1718432852 conn(0x7f3e68078000 msgr2=0x7f3e680fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:56.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.914+0000 7f3e6cdf4640 1 -- 172.21.15.43:0/1718432852 shutdown_connections 2024-04-21T08:48:56.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:56.914+0000 7f3e6cdf4640 1 -- 172.21.15.43:0/1718432852 wait complete. 2024-04-21T08:48:56.925 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:48:56.925 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:48:57.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.029+0000 7fe97b643640 1 -- 172.21.15.43:0/3225047047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9741031d0 msgr2=0x7fe9741035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.029+0000 7fe97b643640 1 --2- 172.21.15.43:0/3225047047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9741031d0 0x7fe9741035b0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fe970009900 tx=0x7fe97002bcd0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.029+0000 7fe97b643640 1 -- 172.21.15.43:0/3225047047 shutdown_connections 2024-04-21T08:48:57.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.029+0000 7fe97b643640 1 --2- 172.21.15.43:0/3225047047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe97410c5a0 0x7fe97410e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.029+0000 7fe97b643640 1 --2- 172.21.15.43:0/3225047047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe974103af0 0x7fe97410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.029+0000 7fe97b643640 1 --2- 172.21.15.43:0/3225047047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9741031d0 0x7fe9741035b0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.029+0000 7fe97b643640 1 -- 172.21.15.43:0/3225047047 >> 172.21.15.43:0/3225047047 conn(0x7fe9740771d0 msgr2=0x7fe9740775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.029+0000 7fe97b643640 1 -- 172.21.15.43:0/3225047047 shutdown_connections 2024-04-21T08:48:57.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.029+0000 7fe97b643640 1 -- 172.21.15.43:0/3225047047 wait complete. 2024-04-21T08:48:57.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.030+0000 7fe97b643640 1 Processor -- start 2024-04-21T08:48:57.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.030+0000 7fe97b643640 1 -- start start 2024-04-21T08:48:57.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.030+0000 7fe97b643640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9741031d0 0x7fe9741a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.030+0000 7fe97b643640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe974103af0 0x7fe9741a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.030+0000 7fe97b643640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe97410c5a0 0x7fe97419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.030+0000 7fe97b643640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe974102200 con 0x7fe974103af0 2024-04-21T08:48:57.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.030+0000 7fe97b643640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe974102080 con 0x7fe97410c5a0 2024-04-21T08:48:57.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.030+0000 7fe97b643640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe974102380 con 0x7fe9741031d0 2024-04-21T08:48:57.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.030+0000 7fe978bde640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe974103af0 0x7fe9741a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.030+0000 7fe978bde640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe974103af0 0x7fe9741a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47390/0 (socket says 172.21.15.43:47390) 2024-04-21T08:48:57.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.030+0000 7fe978bde640 1 -- 172.21.15.43:0/3668966439 learned_addr learned my addr 172.21.15.43:0/3668966439 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.030+0000 7fe9793df640 1 --2- 172.21.15.43:0/3668966439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9741031d0 0x7fe9741a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.030+0000 7fe979be0640 1 --2- 172.21.15.43:0/3668966439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe97410c5a0 0x7fe97419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.030+0000 7fe978bde640 1 -- 172.21.15.43:0/3668966439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9741031d0 msgr2=0x7fe9741a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.030+0000 7fe978bde640 1 --2- 172.21.15.43:0/3668966439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9741031d0 0x7fe9741a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.030+0000 7fe978bde640 1 -- 172.21.15.43:0/3668966439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe97410c5a0 msgr2=0x7fe97419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.030+0000 7fe978bde640 1 --2- 172.21.15.43:0/3668966439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe97410c5a0 0x7fe97419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.030+0000 7fe978bde640 1 -- 172.21.15.43:0/3668966439 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97419aff0 con 0x7fe974103af0 2024-04-21T08:48:57.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.031+0000 7fe978bde640 1 --2- 172.21.15.43:0/3668966439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe974103af0 0x7fe9741a0bb0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fe96400e9f0 tx=0x7fe96400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.031+0000 7fe96a7fc640 1 -- 172.21.15.43:0/3668966439 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96400cd80 con 0x7fe974103af0 2024-04-21T08:48:57.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.031+0000 7fe96a7fc640 1 -- 172.21.15.43:0/3668966439 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9640108e0 con 0x7fe974103af0 2024-04-21T08:48:57.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.031+0000 7fe97b643640 1 -- 172.21.15.43:0/3668966439 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe97419b2e0 con 0x7fe974103af0 2024-04-21T08:48:57.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.031+0000 7fe96a7fc640 1 -- 172.21.15.43:0/3668966439 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964018980 con 0x7fe974103af0 2024-04-21T08:48:57.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.031+0000 7fe97b643640 1 -- 172.21.15.43:0/3668966439 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9741a75f0 con 0x7fe974103af0 2024-04-21T08:48:57.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.032+0000 7fe97b643640 1 -- 172.21.15.43:0/3668966439 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe97406ae60 con 0x7fe974103af0 2024-04-21T08:48:57.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.034+0000 7fe96a7fc640 1 -- 172.21.15.43:0/3668966439 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe964010430 con 0x7fe974103af0 2024-04-21T08:48:57.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.034+0000 7fe96a7fc640 1 --2- 172.21.15.43:0/3668966439 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe9600a3da0 0x7fe9600a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.034+0000 7fe9793df640 1 --2- 172.21.15.43:0/3668966439 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe9600a3da0 0x7fe9600a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.035+0000 7fe96a7fc640 1 -- 172.21.15.43:0/3668966439 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fe9640d1330 con 0x7fe974103af0 2024-04-21T08:48:57.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.035+0000 7fe9793df640 1 --2- 172.21.15.43:0/3668966439 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe9600a3da0 0x7fe9600a6260 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe970006010 tx=0x7fe970005e40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.038+0000 7fe96a7fc640 1 -- 172.21.15.43:0/3668966439 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe96409cf60 con 0x7fe974103af0 2024-04-21T08:48:57.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.266+0000 7fe97b643640 1 -- 172.21.15.43:0/3668966439 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9741a77c0 con 0x7fe974103af0 2024-04-21T08:48:57.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.267+0000 7fe96a7fc640 1 -- 172.21.15.43:0/3668966439 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11935 (secure 0 0 0) 0x7fe9640a1260 con 0x7fe974103af0 2024-04-21T08:48:57.266 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:48:57.266 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:46:17.283806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27556":{"gid":27556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/229985237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":229985237},{"type":"v1","addr":"172.21.15.175:6837","nonce":229985237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24425,24412,24487,24527,24491]},"id":1}]} 2024-04-21T08:48:57.267 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-04-21T08:48:57.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.269+0000 7fe97b643640 1 -- 172.21.15.43:0/3668966439 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe9600a3da0 msgr2=0x7fe9600a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.269+0000 7fe97b643640 1 --2- 172.21.15.43:0/3668966439 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe9600a3da0 0x7fe9600a6260 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe970006010 tx=0x7fe970005e40 comp rx=0 tx=0).stop 2024-04-21T08:48:57.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.269+0000 7fe97b643640 1 -- 172.21.15.43:0/3668966439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe974103af0 msgr2=0x7fe9741a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.269+0000 7fe97b643640 1 --2- 172.21.15.43:0/3668966439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe974103af0 0x7fe9741a0bb0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fe96400e9f0 tx=0x7fe96400eeb0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.269+0000 7fe97b643640 1 -- 172.21.15.43:0/3668966439 shutdown_connections 2024-04-21T08:48:57.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.269+0000 7fe97b643640 1 --2- 172.21.15.43:0/3668966439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe97410c5a0 0x7fe97419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.269+0000 7fe97b643640 1 --2- 172.21.15.43:0/3668966439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe974103af0 0x7fe9741a0bb0 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.269+0000 7fe97b643640 1 --2- 172.21.15.43:0/3668966439 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe9600a3da0 0x7fe9600a6260 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.269+0000 7fe97b643640 1 --2- 172.21.15.43:0/3668966439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9741031d0 0x7fe9741a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.269+0000 7fe97b643640 1 -- 172.21.15.43:0/3668966439 >> 172.21.15.43:0/3668966439 conn(0x7fe9740771d0 msgr2=0x7fe9740fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.269+0000 7fe97b643640 1 -- 172.21.15.43:0/3668966439 shutdown_connections 2024-04-21T08:48:57.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.269+0000 7fe97b643640 1 -- 172.21.15.43:0/3668966439 wait complete. 2024-04-21T08:48:57.280 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:48:57.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.385+0000 7f7ec3bb6640 1 -- 172.21.15.43:0/3937478267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ebc101090 msgr2=0x7f7ebc101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.385+0000 7f7ec3bb6640 1 --2- 172.21.15.43:0/3937478267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ebc101090 0x7f7ebc101470 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009900 tx=0x7f7eb802bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:57.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.385+0000 7f7ec3bb6640 1 -- 172.21.15.43:0/3937478267 shutdown_connections 2024-04-21T08:48:57.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.385+0000 7f7ec3bb6640 1 --2- 172.21.15.43:0/3937478267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ebc106160 0x7f7ebc10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.385+0000 7f7ec3bb6640 1 --2- 172.21.15.43:0/3937478267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ebc101a40 0x7f7ebc105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.385+0000 7f7ec3bb6640 1 --2- 172.21.15.43:0/3937478267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ebc101090 0x7f7ebc101470 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.385+0000 7f7ec3bb6640 1 -- 172.21.15.43:0/3937478267 >> 172.21.15.43:0/3937478267 conn(0x7f7ebc0771d0 msgr2=0x7f7ebc0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.385+0000 7f7ec3bb6640 1 -- 172.21.15.43:0/3937478267 shutdown_connections 2024-04-21T08:48:57.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.385+0000 7f7ec3bb6640 1 -- 172.21.15.43:0/3937478267 wait complete. 2024-04-21T08:48:57.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.386+0000 7f7ec3bb6640 1 Processor -- start 2024-04-21T08:48:57.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.386+0000 7f7ec3bb6640 1 -- start start 2024-04-21T08:48:57.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.386+0000 7f7ec3bb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ebc101090 0x7f7ebc19e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.386+0000 7f7ec3bb6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ebc106160 0x7f7ebc19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.386+0000 7f7ec3bb6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ebc1985e0 0x7f7ebc198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.386+0000 7f7ec3bb6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc112220 con 0x7f7ebc1985e0 2024-04-21T08:48:57.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.386+0000 7f7ec3bb6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1120a0 con 0x7f7ebc101090 2024-04-21T08:48:57.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.386+0000 7f7ec3bb6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1123a0 con 0x7f7ebc106160 2024-04-21T08:48:57.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.386+0000 7f7ec2153640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ebc1985e0 0x7f7ebc198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.386+0000 7f7ec2153640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ebc1985e0 0x7f7ebc198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47398/0 (socket says 172.21.15.43:47398) 2024-04-21T08:48:57.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.386+0000 7f7ec2153640 1 -- 172.21.15.43:0/1036785600 learned_addr learned my addr 172.21.15.43:0/1036785600 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.386+0000 7f7ec1952640 1 --2- 172.21.15.43:0/1036785600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ebc101090 0x7f7ebc19e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.386+0000 7f7ec1151640 1 --2- 172.21.15.43:0/1036785600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ebc106160 0x7f7ebc19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.387+0000 7f7ec2153640 1 -- 172.21.15.43:0/1036785600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ebc106160 msgr2=0x7f7ebc19e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.387+0000 7f7ec2153640 1 --2- 172.21.15.43:0/1036785600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ebc106160 0x7f7ebc19e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.387+0000 7f7ec2153640 1 -- 172.21.15.43:0/1036785600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ebc101090 msgr2=0x7f7ebc19e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.387+0000 7f7ec2153640 1 --2- 172.21.15.43:0/1036785600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ebc101090 0x7f7ebc19e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.387+0000 7f7ec2153640 1 -- 172.21.15.43:0/1036785600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ebc199100 con 0x7f7ebc1985e0 2024-04-21T08:48:57.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.387+0000 7f7ec2153640 1 --2- 172.21.15.43:0/1036785600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ebc1985e0 0x7f7ebc198a90 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400eaa0 tx=0x7f7eb400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.387+0000 7f7eb2ffd640 1 -- 172.21.15.43:0/1036785600 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb400cdb0 con 0x7f7ebc1985e0 2024-04-21T08:48:57.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.387+0000 7f7eb2ffd640 1 -- 172.21.15.43:0/1036785600 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7eb4005020 con 0x7f7ebc1985e0 2024-04-21T08:48:57.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.387+0000 7f7ec3bb6640 1 -- 172.21.15.43:0/1036785600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ebc1993f0 con 0x7f7ebc1985e0 2024-04-21T08:48:57.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.387+0000 7f7eb2ffd640 1 -- 172.21.15.43:0/1036785600 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb40116d0 con 0x7f7ebc1985e0 2024-04-21T08:48:57.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.387+0000 7f7ec3bb6640 1 -- 172.21.15.43:0/1036785600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ebc1a5640 con 0x7f7ebc1985e0 2024-04-21T08:48:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.389+0000 7f7eb2ffd640 1 -- 172.21.15.43:0/1036785600 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7eb40051d0 con 0x7f7ebc1985e0 2024-04-21T08:48:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.390+0000 7f7ec67ae740 1 Processor -- start 2024-04-21T08:48:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.390+0000 7f7ec67ae740 1 -- start start 2024-04-21T08:48:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.390+0000 7f7ec67ae740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 0x55c09d5bc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.390+0000 7f7ec67ae740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c09d5bce80 0x55c09d5c5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.390+0000 7f7ec67ae740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c09d5c5a00 0x55c09d5c7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.390+0000 7f7ec67ae740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c09d5211f0 con 0x55c09d5bc4d0 2024-04-21T08:48:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.390+0000 7f7ec67ae740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c09d4be4b0 con 0x55c09d5bce80 2024-04-21T08:48:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.390+0000 7f7ec67ae740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c09d51d1b0 con 0x55c09d5c5a00 2024-04-21T08:48:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.390+0000 7f7eb2ffd640 1 --2- 172.21.15.43:0/1036785600 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7ea80a3da0 0x7f7ea80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.390+0000 7f7ec1151640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 0x55c09d5bc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.390+0000 7f7ec1151640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 0x55c09d5bc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47406/0 (socket says 172.21.15.43:47406) 2024-04-21T08:48:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.390+0000 7f7ec1151640 1 -- 172.21.15.43:0/1296422457 learned_addr learned my addr 172.21.15.43:0/1296422457 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.390+0000 7f7ec1952640 1 --2- 172.21.15.43:0/1296422457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c09d5bce80 0x55c09d5c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.390+0000 7f7ec2153640 1 --2- 172.21.15.43:0/1036785600 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7ea80a3da0 0x7f7ea80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.391+0000 7f7ec1151640 1 --2- 172.21.15.43:0/1296422457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c09d5c5a00 0x55c09d5c7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.391+0000 7f7eb2ffd640 1 -- 172.21.15.43:0/1036785600 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f7eb40d0ed0 con 0x7f7ebc1985e0 2024-04-21T08:48:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.391+0000 7f7ec1151640 1 -- 172.21.15.43:0/1296422457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c09d5c5a00 msgr2=0x55c09d5c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.391+0000 7f7ec1151640 1 --2- 172.21.15.43:0/1296422457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c09d5c5a00 0x55c09d5c7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.391+0000 7f7ec1151640 1 -- 172.21.15.43:0/1296422457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c09d5bce80 msgr2=0x55c09d5c5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.391+0000 7f7ec1151640 1 --2- 172.21.15.43:0/1296422457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c09d5bce80 0x55c09d5c5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.391+0000 7f7ec1151640 1 -- 172.21.15.43:0/1296422457 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c09d4b01b0 con 0x55c09d5bc4d0 2024-04-21T08:48:57.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.391+0000 7f7ec1952640 1 --2- 172.21.15.43:0/1296422457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c09d5bce80 0x55c09d5c5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:57.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.391+0000 7f7ec1151640 1 --2- 172.21.15.43:0/1296422457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 0x55c09d5bc8b0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f7eac006aa0 tx=0x7f7eac00b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8510c6b38c899e6c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.391+0000 7f7ec2153640 1 --2- 172.21.15.43:0/1036785600 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7ea80a3da0 0x7f7ea80a6260 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7eb409c6f0 tx=0x7f7eb4087040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.391+0000 7f7ea7fff640 1 -- 172.21.15.43:0/1296422457 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac0173f0 con 0x55c09d5bc4d0 2024-04-21T08:48:57.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.391+0000 7f7ea7fff640 1 -- 172.21.15.43:0/1296422457 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7eac010040 con 0x55c09d5bc4d0 2024-04-21T08:48:57.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.391+0000 7f7ea7fff640 1 -- 172.21.15.43:0/1296422457 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac016970 con 0x55c09d5bc4d0 2024-04-21T08:48:57.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.392+0000 7f7ec67ae740 1 -- 172.21.15.43:0/1296422457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 msgr2=0x55c09d5bc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.392+0000 7f7ec67ae740 1 --2- 172.21.15.43:0/1296422457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 0x55c09d5bc8b0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f7eac006aa0 tx=0x7f7eac00b400 comp rx=0 tx=0).stop 2024-04-21T08:48:57.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.392+0000 7f7ec67ae740 1 -- 172.21.15.43:0/1296422457 shutdown_connections 2024-04-21T08:48:57.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.392+0000 7f7ec67ae740 1 --2- 172.21.15.43:0/1296422457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c09d5c5a00 0x55c09d5c7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.392+0000 7f7ec67ae740 1 --2- 172.21.15.43:0/1296422457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c09d5bce80 0x55c09d5c5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.392+0000 7f7ec67ae740 1 --2- 172.21.15.43:0/1296422457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 0x55c09d5bc8b0 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.392+0000 7f7ec67ae740 1 -- 172.21.15.43:0/1296422457 >> 172.21.15.43:0/1296422457 conn(0x55c09d5b6ba0 msgr2=0x55c09d5b6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.392+0000 7f7ec67ae740 1 -- 172.21.15.43:0/1296422457 shutdown_connections 2024-04-21T08:48:57.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.392+0000 7f7ec67ae740 1 -- 172.21.15.43:0/1296422457 wait complete. 2024-04-21T08:48:57.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.392+0000 7f7ec67ae740 1 -- start start 2024-04-21T08:48:57.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.392+0000 7f7ec67ae740 1 Processor -- start 2024-04-21T08:48:57.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.393+0000 7f7ec67ae740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 0x55c09d5c5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.393+0000 7f7ec67ae740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c09d5bce80 0x55c09d5c2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.393+0000 7f7ec67ae740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c09d5c5a00 0x55c09d5c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.393+0000 7f7ec67ae740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c09d5c8910 con 0x55c09d5bc4d0 2024-04-21T08:48:57.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.393+0000 7f7ec67ae740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c09d4da500 con 0x55c09d5c5a00 2024-04-21T08:48:57.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.393+0000 7f7ec67ae740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c09d5c8a90 con 0x55c09d5bce80 2024-04-21T08:48:57.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.393+0000 7f7ec1151640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 0x55c09d5c5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.393+0000 7f7ec1151640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 0x55c09d5c5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47414/0 (socket says 172.21.15.43:47414) 2024-04-21T08:48:57.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.393+0000 7f7ec1151640 1 -- 172.21.15.43:0/1741933443 learned_addr learned my addr 172.21.15.43:0/1741933443 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.393+0000 7f7ec67ae740 10 client.0 _ll_register_callbacks cb 0x55c09d4a8b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:57.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.393+0000 7f7ec1952640 1 --2- 172.21.15.43:0/1741933443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c09d5bce80 0x55c09d5c2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.393+0000 7f7ec1151640 1 --2- 172.21.15.43:0/1741933443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c09d5c5a00 0x55c09d5c2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.393+0000 7f7ec1151640 1 -- 172.21.15.43:0/1741933443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c09d5bce80 msgr2=0x55c09d5c2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.393+0000 7f7ec1151640 1 --2- 172.21.15.43:0/1741933443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c09d5bce80 0x55c09d5c2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.393+0000 7f7ec1151640 1 -- 172.21.15.43:0/1741933443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c09d5c5a00 msgr2=0x55c09d5c2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.393+0000 7f7ec1151640 1 --2- 172.21.15.43:0/1741933443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c09d5c5a00 0x55c09d5c2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.393+0000 7f7ec1151640 1 -- 172.21.15.43:0/1741933443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c09d2333f0 con 0x55c09d5bc4d0 2024-04-21T08:48:57.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.393+0000 7f7ec1151640 1 --2- 172.21.15.43:0/1741933443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 0x55c09d5c5300 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f7eac006cd0 tx=0x7f7eac016650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b48e3a0afb1f75 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.393+0000 7f7ea57fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:57.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.394+0000 7f7ea57fa640 1 -- 172.21.15.43:0/1741933443 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac026450 con 0x55c09d5bc4d0 2024-04-21T08:48:57.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.394+0000 7f7ea57fa640 1 -- 172.21.15.43:0/1741933443 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f7eac0265f0 con 0x55c09d5bc4d0 2024-04-21T08:48:57.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.394+0000 7f7ec67ae740 1 -- 172.21.15.43:0/1741933443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eac010040 con 0x55c09d5bc4d0 2024-04-21T08:48:57.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.394+0000 7f7ea57fa640 1 -- 172.21.15.43:0/1741933443 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7eac0059c0 con 0x55c09d5bc4d0 2024-04-21T08:48:57.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.394+0000 7f7ec67ae740 10 client.20226 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:57.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.394+0000 7f7ec67ae740 1 -- 172.21.15.43:0/1741933443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c09d2fd7d0 con 0x55c09d5bc4d0 2024-04-21T08:48:57.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.394+0000 7f7ea57fa640 1 -- 172.21.15.43:0/1741933443 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7eac010040 con 0x55c09d5bc4d0 2024-04-21T08:48:57.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.394+0000 7f7ea57fa640 1 -- 172.21.15.43:0/1741933443 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7eac006020 con 0x55c09d5bc4d0 2024-04-21T08:48:57.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.394+0000 7f7ec67ae740 10 client.20226 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:57.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.394+0000 7f7ec67ae740 10 client.20226 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:48:57.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.394+0000 7f7ec67ae740 20 client.20226 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:57.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.394+0000 7f7ec67ae740 1 --2- 172.21.15.43:0/1741933443 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55c09d5d5b30 0x55c09d5d7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.394+0000 7f7ec67ae740 4 client.20226 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:57.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.394+0000 7f7ec67ae740 1 -- 172.21.15.43:0/1741933443 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c09d4c0930 con 0x55c09d5d5b30 2024-04-21T08:48:57.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.395+0000 7f7ec1952640 1 --2- 172.21.15.43:0/1741933443 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55c09d5d5b30 0x55c09d5d7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.395+0000 7f7ec1952640 1 --2- 172.21.15.43:0/1741933443 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55c09d5d5b30 0x55c09d5d7f20 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=35174c6a44d7500b server_cookie=ba0bf9e3e858b3ef in_seq=0 out_seq=0 2024-04-21T08:48:57.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.395+0000 7f7ea57fa640 10 client.20226 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:48:57.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.396+0000 7f7ea57fa640 1 -- 172.21.15.43:0/1741933443 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c09d4c0930 con 0x55c09d5d5b30 2024-04-21T08:48:57.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.396+0000 7f7ea57fa640 10 client.20226 handle_command_reply: tid=0 2024-04-21T08:48:57.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.396+0000 7f7ec67ae740 1 client.20226 shutdown 2024-04-21T08:48:57.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.396+0000 7f7ec67ae740 1 -- 172.21.15.43:0/1741933443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 msgr2=0x55c09d5c5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.396+0000 7f7ec67ae740 1 --2- 172.21.15.43:0/1741933443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 0x55c09d5c5300 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f7eac006cd0 tx=0x7f7eac016650 comp rx=0 tx=0).stop 2024-04-21T08:48:57.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.396+0000 7f7ec67ae740 1 -- 172.21.15.43:0/1741933443 shutdown_connections 2024-04-21T08:48:57.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.396+0000 7f7ec67ae740 1 --2- 172.21.15.43:0/1741933443 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55c09d5d5b30 0x55c09d5d7f20 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.396+0000 7f7ec67ae740 1 --2- 172.21.15.43:0/1741933443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c09d5c5a00 0x55c09d5c2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.396+0000 7f7ea57fa640 0 client.20226 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:48:57.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.396+0000 7f7ec67ae740 1 --2- 172.21.15.43:0/1741933443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c09d5bce80 0x55c09d5c2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.396+0000 7f7ec67ae740 1 --2- 172.21.15.43:0/1741933443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 0x55c09d5c5300 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.396+0000 7f7ec67ae740 1 -- 172.21.15.43:0/1741933443 >> 172.21.15.43:0/1741933443 conn(0x55c09d5b6ba0 msgr2=0x55c09d5b88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.396+0000 7f7ec67ae740 1 -- 172.21.15.43:0/1741933443 shutdown_connections 2024-04-21T08:48:57.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.396+0000 7f7ec67ae740 1 -- 172.21.15.43:0/1741933443 wait complete. 2024-04-21T08:48:57.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.396+0000 7f7ec67ae740 20 client.20226 trim_cache size 0 max 16384 2024-04-21T08:48:57.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.409+0000 7f7ec67ae740 1 Processor -- start 2024-04-21T08:48:57.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.410+0000 7f7ec67ae740 1 -- start start 2024-04-21T08:48:57.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.410+0000 7f7ec67ae740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 0x55c09d5b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.410+0000 7f7ec67ae740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c09d5bce80 0x55c09d5d17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.410+0000 7f7ec67ae740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c09d5c5a00 0x55c09d5ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.410+0000 7f7ec67ae740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1af5f0 con 0x55c09d5bc4d0 2024-04-21T08:48:57.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.410+0000 7f7ec67ae740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1b1130 con 0x55c09d5bce80 2024-04-21T08:48:57.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.410+0000 7f7ec67ae740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1afcc0 con 0x55c09d5c5a00 2024-04-21T08:48:57.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.410+0000 7f7ec1151640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 0x55c09d5b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.410+0000 7f7ec1151640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 0x55c09d5b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47416/0 (socket says 172.21.15.43:47416) 2024-04-21T08:48:57.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.410+0000 7f7ec1151640 1 -- 172.21.15.43:0/1898945334 learned_addr learned my addr 172.21.15.43:0/1898945334 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.410+0000 7f7ec1952640 1 --2- 172.21.15.43:0/1898945334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c09d5bce80 0x55c09d5d17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.410+0000 7f7ec1151640 1 -- 172.21.15.43:0/1898945334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c09d5c5a00 msgr2=0x55c09d5ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.410+0000 7f7ec1151640 1 --2- 172.21.15.43:0/1898945334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c09d5c5a00 0x55c09d5ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.410+0000 7f7ec1151640 1 -- 172.21.15.43:0/1898945334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c09d5bce80 msgr2=0x55c09d5d17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.410+0000 7f7ec1151640 1 --2- 172.21.15.43:0/1898945334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c09d5bce80 0x55c09d5d17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.410+0000 7f7ec1151640 1 -- 172.21.15.43:0/1898945334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c09d5c2f80 con 0x55c09d5bc4d0 2024-04-21T08:48:57.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.410+0000 7f7ec1952640 1 --2- 172.21.15.43:0/1898945334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c09d5bce80 0x55c09d5d17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:57.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.410+0000 7f7ec1151640 1 --2- 172.21.15.43:0/1898945334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 0x55c09d5b5e40 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0164c0 tx=0x7f7eac0165c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e72725dfdb2f08 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.410+0000 7f7ea77fe640 1 -- 172.21.15.43:0/1898945334 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac03d070 con 0x55c09d5bc4d0 2024-04-21T08:48:57.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.410+0000 7f7ea77fe640 1 -- 172.21.15.43:0/1898945334 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7eac02b070 con 0x55c09d5bc4d0 2024-04-21T08:48:57.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.410+0000 7f7ea77fe640 1 -- 172.21.15.43:0/1898945334 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac026450 con 0x55c09d5bc4d0 2024-04-21T08:48:57.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.411+0000 7f7ec67ae740 1 -- 172.21.15.43:0/1898945334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 msgr2=0x55c09d5b5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.411+0000 7f7ec67ae740 1 --2- 172.21.15.43:0/1898945334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 0x55c09d5b5e40 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0164c0 tx=0x7f7eac0165c0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.411+0000 7f7ec67ae740 1 -- 172.21.15.43:0/1898945334 shutdown_connections 2024-04-21T08:48:57.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.411+0000 7f7ec67ae740 1 --2- 172.21.15.43:0/1898945334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c09d5c5a00 0x55c09d5ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.411+0000 7f7ec67ae740 1 --2- 172.21.15.43:0/1898945334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c09d5bce80 0x55c09d5d17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.411+0000 7f7ec67ae740 1 --2- 172.21.15.43:0/1898945334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 0x55c09d5b5e40 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.411+0000 7f7ec67ae740 1 -- 172.21.15.43:0/1898945334 >> 172.21.15.43:0/1898945334 conn(0x55c09d5b6ba0 msgr2=0x55c09d5b8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.411+0000 7f7ec67ae740 1 -- 172.21.15.43:0/1898945334 shutdown_connections 2024-04-21T08:48:57.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.411+0000 7f7ec67ae740 1 -- 172.21.15.43:0/1898945334 wait complete. 2024-04-21T08:48:57.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.411+0000 7f7ec67ae740 1 -- start start 2024-04-21T08:48:57.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.411+0000 7f7ec67ae740 1 Processor -- start 2024-04-21T08:48:57.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.412+0000 7f7ec67ae740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 0x55c09d5c0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.412+0000 7f7ec67ae740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c09d5bce80 0x55c09d5c4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.412+0000 7f7ec67ae740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c09d5c5a00 0x55c09d5c4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.412+0000 7f7ec67ae740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1b0a60 con 0x55c09d5bc4d0 2024-04-21T08:48:57.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.412+0000 7f7ec67ae740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1b0390 con 0x55c09d5c5a00 2024-04-21T08:48:57.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.412+0000 7f7ec67ae740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1ad5f0 con 0x55c09d5bce80 2024-04-21T08:48:57.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.412+0000 7f7ec67ae740 10 client.0 _ll_register_callbacks cb 0x7f7eac0167e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:57.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.412+0000 7f7ec1151640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 0x55c09d5c0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.412+0000 7f7ec1151640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 0x55c09d5c0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47430/0 (socket says 172.21.15.43:47430) 2024-04-21T08:48:57.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.412+0000 7f7ec1151640 1 -- 172.21.15.43:0/2421023868 learned_addr learned my addr 172.21.15.43:0/2421023868 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.412+0000 7f7ec1952640 1 --2- 172.21.15.43:0/2421023868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c09d5bce80 0x55c09d5c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.412+0000 7f7ec1151640 1 --2- 172.21.15.43:0/2421023868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c09d5c5a00 0x55c09d5c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.412+0000 7f7ec1151640 1 -- 172.21.15.43:0/2421023868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c09d5bce80 msgr2=0x55c09d5c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.412+0000 7f7ec1151640 1 --2- 172.21.15.43:0/2421023868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c09d5bce80 0x55c09d5c4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.412+0000 7f7ec1151640 1 -- 172.21.15.43:0/2421023868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c09d5c5a00 msgr2=0x55c09d5c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.412+0000 7f7ec1151640 1 --2- 172.21.15.43:0/2421023868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c09d5c5a00 0x55c09d5c4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.412+0000 7f7ec1151640 1 -- 172.21.15.43:0/2421023868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c09d5c5160 con 0x55c09d5bc4d0 2024-04-21T08:48:57.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.412+0000 7f7ec1151640 1 --2- 172.21.15.43:0/2421023868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 0x55c09d5c0150 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f7eac016460 tx=0x7f7eac004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8683b15446a731 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.412+0000 7f7ea57fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:57.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.412+0000 7f7ea57fa640 1 -- 172.21.15.43:0/2421023868 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac04a060 con 0x55c09d5bc4d0 2024-04-21T08:48:57.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.413+0000 7f7ea57fa640 1 -- 172.21.15.43:0/2421023868 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f7eac026450 con 0x55c09d5bc4d0 2024-04-21T08:48:57.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.413+0000 7f7ec67ae740 1 -- 172.21.15.43:0/2421023868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eac02b070 con 0x55c09d5bc4d0 2024-04-21T08:48:57.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.413+0000 7f7ea57fa640 1 -- 172.21.15.43:0/2421023868 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7eac051050 con 0x55c09d5bc4d0 2024-04-21T08:48:57.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.413+0000 7f7ec67ae740 10 client.20232 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:57.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.413+0000 7f7ec67ae740 1 -- 172.21.15.43:0/2421023868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c09d63fdc0 con 0x55c09d5bc4d0 2024-04-21T08:48:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.413+0000 7f7ea57fa640 1 -- 172.21.15.43:0/2421023868 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7eac02b070 con 0x55c09d5bc4d0 2024-04-21T08:48:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.413+0000 7f7ea57fa640 1 -- 172.21.15.43:0/2421023868 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7eac038440 con 0x55c09d5bc4d0 2024-04-21T08:48:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.413+0000 7f7ec67ae740 10 client.20232 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.413+0000 7f7ec67ae740 10 client.20232 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:48:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.413+0000 7f7ec67ae740 20 client.20232 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.413+0000 7f7ec67ae740 1 --2- 172.21.15.43:0/2421023868 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55c09d640420 0x55c09d642840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.413+0000 7f7ec67ae740 4 client.20232 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.413+0000 7f7ec67ae740 1 -- 172.21.15.43:0/2421023868 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c09d52ec10 con 0x55c09d640420 2024-04-21T08:48:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.413+0000 7f7ec1952640 1 --2- 172.21.15.43:0/2421023868 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55c09d640420 0x55c09d642840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.414+0000 7f7ec1952640 1 --2- 172.21.15.43:0/2421023868 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55c09d640420 0x55c09d642840 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e96c2f82d59944ca server_cookie=b17aa96c922a421a in_seq=0 out_seq=0 2024-04-21T08:48:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.414+0000 7f7ea57fa640 10 client.20232 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:48:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.414+0000 7f7ea57fa640 1 -- 172.21.15.43:0/2421023868 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c09d52ec10 con 0x55c09d640420 2024-04-21T08:48:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.414+0000 7f7ea57fa640 10 client.20232 handle_command_reply: tid=0 2024-04-21T08:48:57.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.414+0000 7f7ec67ae740 1 client.20232 shutdown 2024-04-21T08:48:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.414+0000 7f7ec67ae740 1 -- 172.21.15.43:0/2421023868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 msgr2=0x55c09d5c0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.414+0000 7f7ec67ae740 1 --2- 172.21.15.43:0/2421023868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 0x55c09d5c0150 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f7eac016460 tx=0x7f7eac004640 comp rx=0 tx=0).stop 2024-04-21T08:48:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.414+0000 7f7ec67ae740 1 -- 172.21.15.43:0/2421023868 shutdown_connections 2024-04-21T08:48:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.414+0000 7f7ec67ae740 1 --2- 172.21.15.43:0/2421023868 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55c09d640420 0x55c09d642840 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.414+0000 7f7ec67ae740 1 --2- 172.21.15.43:0/2421023868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c09d5c5a00 0x55c09d5c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.414+0000 7f7ec67ae740 1 --2- 172.21.15.43:0/2421023868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c09d5bce80 0x55c09d5c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.414+0000 7f7ec67ae740 1 --2- 172.21.15.43:0/2421023868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c09d5bc4d0 0x55c09d5c0150 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.414+0000 7f7ea57fa640 0 client.20232 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:48:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.414+0000 7f7ec67ae740 1 -- 172.21.15.43:0/2421023868 >> 172.21.15.43:0/2421023868 conn(0x55c09d5b6ba0 msgr2=0x55c09d5c0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.415+0000 7f7ec67ae740 1 -- 172.21.15.43:0/2421023868 shutdown_connections 2024-04-21T08:48:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.415+0000 7f7ec67ae740 1 -- 172.21.15.43:0/2421023868 wait complete. 2024-04-21T08:48:57.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.415+0000 7f7ec67ae740 20 client.20232 trim_cache size 0 max 16384 2024-04-21T08:48:57.421 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:48:57.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.415+0000 7f7ec3bb6640 1 -- 172.21.15.43:0/1036785600 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7ea80a3da0 msgr2=0x7f7ea80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.415+0000 7f7ec3bb6640 1 --2- 172.21.15.43:0/1036785600 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7ea80a3da0 0x7f7ea80a6260 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7eb409c6f0 tx=0x7f7eb4087040 comp rx=0 tx=0).stop 2024-04-21T08:48:57.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.415+0000 7f7ec3bb6640 1 -- 172.21.15.43:0/1036785600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ebc1985e0 msgr2=0x7f7ebc198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.415+0000 7f7ec3bb6640 1 --2- 172.21.15.43:0/1036785600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ebc1985e0 0x7f7ebc198a90 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400eaa0 tx=0x7f7eb400ef60 comp rx=0 tx=0).stop 2024-04-21T08:48:57.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.415+0000 7f7ec3bb6640 1 -- 172.21.15.43:0/1036785600 shutdown_connections 2024-04-21T08:48:57.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.415+0000 7f7ec3bb6640 1 --2- 172.21.15.43:0/1036785600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ebc1985e0 0x7f7ebc198a90 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.415+0000 7f7ec3bb6640 1 --2- 172.21.15.43:0/1036785600 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7ea80a3da0 0x7f7ea80a6260 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.415+0000 7f7ec3bb6640 1 --2- 172.21.15.43:0/1036785600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ebc106160 0x7f7ebc19e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.415+0000 7f7ec3bb6640 1 --2- 172.21.15.43:0/1036785600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ebc101090 0x7f7ebc19e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.415+0000 7f7ec3bb6640 1 -- 172.21.15.43:0/1036785600 >> 172.21.15.43:0/1036785600 conn(0x7f7ebc0771d0 msgr2=0x7f7ebc0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.415+0000 7f7ec3bb6640 1 -- 172.21.15.43:0/1036785600 shutdown_connections 2024-04-21T08:48:57.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.415+0000 7f7ec3bb6640 1 -- 172.21.15.43:0/1036785600 wait complete. 2024-04-21T08:48:57.425 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:48:57.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.530+0000 7f4249e31640 1 -- 172.21.15.43:0/3271731950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4244102b40 msgr2=0x7f4244106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.530+0000 7f4249e31640 1 --2- 172.21.15.43:0/3271731950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4244102b40 0x7f4244106b80 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f4238009900 tx=0x7f423802bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:57.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.530+0000 7f4249e31640 1 -- 172.21.15.43:0/3271731950 shutdown_connections 2024-04-21T08:48:57.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.530+0000 7f4249e31640 1 --2- 172.21.15.43:0/3271731950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4244107260 0x7f424410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.530+0000 7f4249e31640 1 --2- 172.21.15.43:0/3271731950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4244102b40 0x7f4244106b80 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.530+0000 7f4249e31640 1 --2- 172.21.15.43:0/3271731950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4244102190 0x7f4244102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.530+0000 7f4249e31640 1 -- 172.21.15.43:0/3271731950 >> 172.21.15.43:0/3271731950 conn(0x7f42440fd990 msgr2=0x7f42440ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.530+0000 7f4249e31640 1 -- 172.21.15.43:0/3271731950 shutdown_connections 2024-04-21T08:48:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.530+0000 7f4249e31640 1 -- 172.21.15.43:0/3271731950 wait complete. 2024-04-21T08:48:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.531+0000 7f4249e31640 1 Processor -- start 2024-04-21T08:48:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.531+0000 7f4249e31640 1 -- start start 2024-04-21T08:48:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.531+0000 7f4249e31640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4244102190 0x7f424419e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.531+0000 7f4249e31640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4244107260 0x7f424419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.531+0000 7f4249e31640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42441985c0 0x7f4244198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.531+0000 7f4249e31640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f42441121a0 con 0x7f4244107260 2024-04-21T08:48:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.531+0000 7f4249e31640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4244112020 con 0x7f42441985c0 2024-04-21T08:48:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.531+0000 7f4249e31640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4244112320 con 0x7f4244102190 2024-04-21T08:48:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.531+0000 7f4242ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4244107260 0x7f424419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.531+0000 7f4242ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4244107260 0x7f424419e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47446/0 (socket says 172.21.15.43:47446) 2024-04-21T08:48:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.531+0000 7f4242ffd640 1 -- 172.21.15.43:0/318408434 learned_addr learned my addr 172.21.15.43:0/318408434 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.531+0000 7f4243fff640 1 --2- 172.21.15.43:0/318408434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42441985c0 0x7f4244198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.531+0000 7f42437fe640 1 --2- 172.21.15.43:0/318408434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4244102190 0x7f424419e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.532+0000 7f4242ffd640 1 -- 172.21.15.43:0/318408434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4244102190 msgr2=0x7f424419e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.532+0000 7f4242ffd640 1 --2- 172.21.15.43:0/318408434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4244102190 0x7f424419e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.532+0000 7f4242ffd640 1 -- 172.21.15.43:0/318408434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42441985c0 msgr2=0x7f4244198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.532+0000 7f4242ffd640 1 --2- 172.21.15.43:0/318408434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42441985c0 0x7f4244198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.532+0000 7f4242ffd640 1 -- 172.21.15.43:0/318408434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42441990e0 con 0x7f4244107260 2024-04-21T08:48:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.532+0000 7f4243fff640 1 --2- 172.21.15.43:0/318408434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42441985c0 0x7f4244198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.532+0000 7f4242ffd640 1 --2- 172.21.15.43:0/318408434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4244107260 0x7f424419e950 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f4238036040 tx=0x7f423802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.532+0000 7f4240ff9640 1 -- 172.21.15.43:0/318408434 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238043020 con 0x7f4244107260 2024-04-21T08:48:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.532+0000 7f4240ff9640 1 -- 172.21.15.43:0/318408434 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f423802cbe0 con 0x7f4244107260 2024-04-21T08:48:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.532+0000 7f4249e31640 1 -- 172.21.15.43:0/318408434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4244199370 con 0x7f4244107260 2024-04-21T08:48:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.532+0000 7f4240ff9640 1 -- 172.21.15.43:0/318408434 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238003bf0 con 0x7f4244107260 2024-04-21T08:48:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.532+0000 7f4249e31640 1 -- 172.21.15.43:0/318408434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42441a55c0 con 0x7f4244107260 2024-04-21T08:48:57.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.534+0000 7f4240ff9640 1 -- 172.21.15.43:0/318408434 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4238046020 con 0x7f4244107260 2024-04-21T08:48:57.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.534+0000 7f424ca26740 1 Processor -- start 2024-04-21T08:48:57.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.534+0000 7f424ca26740 1 -- start start 2024-04-21T08:48:57.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.535+0000 7f424ca26740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339baaf4d0 0x56339baaf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.535+0000 7f424ca26740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56339baafe80 0x56339bab8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.535+0000 7f424ca26740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339bab8a00 0x56339babae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.535+0000 7f424ca26740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56339ba141f0 con 0x56339bab8a00 2024-04-21T08:48:57.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.535+0000 7f424ca26740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56339b9b14b0 con 0x56339baaf4d0 2024-04-21T08:48:57.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.535+0000 7f424ca26740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56339ba101b0 con 0x56339baafe80 2024-04-21T08:48:57.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.535+0000 7f4240ff9640 1 --2- 172.21.15.43:0/318408434 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f422c0a3d20 0x7f422c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.535+0000 7f4242ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339bab8a00 0x56339babae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.535+0000 7f4243fff640 1 --2- 172.21.15.43:0/318408434 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f422c0a3d20 0x7f422c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.535+0000 7f4242ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339bab8a00 0x56339babae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47450/0 (socket says 172.21.15.43:47450) 2024-04-21T08:48:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.535+0000 7f4242ffd640 1 -- 172.21.15.43:0/1357003849 learned_addr learned my addr 172.21.15.43:0/1357003849 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.535+0000 7f4240ff9640 1 -- 172.21.15.43:0/318408434 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f42380f2e40 con 0x7f4244107260 2024-04-21T08:48:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.535+0000 7f4243fff640 1 --2- 172.21.15.43:0/1357003849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339baaf4d0 0x56339baaf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.535+0000 7f42437fe640 1 --2- 172.21.15.43:0/1357003849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56339baafe80 0x56339bab8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.535+0000 7f4242ffd640 1 -- 172.21.15.43:0/1357003849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56339baafe80 msgr2=0x56339bab8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.535+0000 7f4242ffd640 1 --2- 172.21.15.43:0/1357003849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56339baafe80 0x56339bab8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.535+0000 7f4242ffd640 1 -- 172.21.15.43:0/1357003849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339baaf4d0 msgr2=0x56339baaf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.535+0000 7f4242ffd640 1 --2- 172.21.15.43:0/1357003849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339baaf4d0 0x56339baaf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.535+0000 7f4242ffd640 1 -- 172.21.15.43:0/1357003849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56339b9a31b0 con 0x56339bab8a00 2024-04-21T08:48:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.535+0000 7f4242ffd640 1 --2- 172.21.15.43:0/1357003849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339bab8a00 0x56339babae50 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f42380be7d0 tx=0x7f423808e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70314e8b29739081 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.535+0000 7f4243fff640 1 --2- 172.21.15.43:0/318408434 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f422c0a3d20 0x7f422c0a61e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f423c005c40 tx=0x7f423c009d50 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.535+0000 7f4229ffb640 1 -- 172.21.15.43:0/1357003849 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42380be8a0 con 0x56339bab8a00 2024-04-21T08:48:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.536+0000 7f4229ffb640 1 -- 172.21.15.43:0/1357003849 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42380bf1d0 con 0x56339bab8a00 2024-04-21T08:48:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.536+0000 7f4229ffb640 1 -- 172.21.15.43:0/1357003849 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238088840 con 0x56339bab8a00 2024-04-21T08:48:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.536+0000 7f424ca26740 1 -- 172.21.15.43:0/1357003849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339bab8a00 msgr2=0x56339babae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.536+0000 7f424ca26740 1 --2- 172.21.15.43:0/1357003849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339bab8a00 0x56339babae50 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f42380be7d0 tx=0x7f423808e3b0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.536+0000 7f424ca26740 1 -- 172.21.15.43:0/1357003849 shutdown_connections 2024-04-21T08:48:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.536+0000 7f424ca26740 1 --2- 172.21.15.43:0/1357003849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339bab8a00 0x56339babae50 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.536+0000 7f424ca26740 1 --2- 172.21.15.43:0/1357003849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56339baafe80 0x56339bab8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.536+0000 7f424ca26740 1 --2- 172.21.15.43:0/1357003849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339baaf4d0 0x56339baaf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.536+0000 7f424ca26740 1 -- 172.21.15.43:0/1357003849 >> 172.21.15.43:0/1357003849 conn(0x56339baa9ba0 msgr2=0x56339baa9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.536+0000 7f424ca26740 1 -- 172.21.15.43:0/1357003849 shutdown_connections 2024-04-21T08:48:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.536+0000 7f424ca26740 1 -- 172.21.15.43:0/1357003849 wait complete. 2024-04-21T08:48:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.537+0000 7f424ca26740 1 -- start start 2024-04-21T08:48:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.537+0000 7f424ca26740 1 Processor -- start 2024-04-21T08:48:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.537+0000 7f424ca26740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339baaf4d0 0x56339bab8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.537+0000 7f424ca26740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339baafe80 0x56339bab5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.537+0000 7f424ca26740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56339bab8a00 0x56339bab5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.537+0000 7f424ca26740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56339babb730 con 0x56339baafe80 2024-04-21T08:48:57.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.537+0000 7f424ca26740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56339b9cd500 con 0x56339baaf4d0 2024-04-21T08:48:57.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.537+0000 7f424ca26740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56339babb8b0 con 0x56339bab8a00 2024-04-21T08:48:57.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.537+0000 7f424ca26740 10 client.0 _ll_register_callbacks cb 0x56339b99bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:57.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.537+0000 7f4242ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339baafe80 0x56339bab5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.537+0000 7f4242ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339baafe80 0x56339bab5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47454/0 (socket says 172.21.15.43:47454) 2024-04-21T08:48:57.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.537+0000 7f4242ffd640 1 -- 172.21.15.43:0/594271544 learned_addr learned my addr 172.21.15.43:0/594271544 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.537+0000 7f42437fe640 1 --2- 172.21.15.43:0/594271544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339baaf4d0 0x56339bab8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.537+0000 7f4243fff640 1 --2- 172.21.15.43:0/594271544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56339bab8a00 0x56339bab5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.537+0000 7f4242ffd640 1 -- 172.21.15.43:0/594271544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56339bab8a00 msgr2=0x56339bab5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.537+0000 7f4242ffd640 1 --2- 172.21.15.43:0/594271544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56339bab8a00 0x56339bab5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.537+0000 7f4242ffd640 1 -- 172.21.15.43:0/594271544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339baaf4d0 msgr2=0x56339bab8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.537+0000 7f4242ffd640 1 --2- 172.21.15.43:0/594271544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339baaf4d0 0x56339bab8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.538+0000 7f4242ffd640 1 -- 172.21.15.43:0/594271544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56339b7263f0 con 0x56339baafe80 2024-04-21T08:48:57.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.538+0000 7f4242ffd640 1 --2- 172.21.15.43:0/594271544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339baafe80 0x56339bab5600 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f42380f31f0 tx=0x7f42380beb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4ba8cb8f404335 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.538+0000 7f4222ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:57.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.538+0000 7f4222ffd640 1 -- 172.21.15.43:0/594271544 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238083a50 con 0x56339baafe80 2024-04-21T08:48:57.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.538+0000 7f4222ffd640 1 -- 172.21.15.43:0/594271544 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f42380bcdd0 con 0x56339baafe80 2024-04-21T08:48:57.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.538+0000 7f424ca26740 1 -- 172.21.15.43:0/594271544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42380bf1d0 con 0x56339baafe80 2024-04-21T08:48:57.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.538+0000 7f4222ffd640 1 -- 172.21.15.43:0/594271544 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f42380be430 con 0x56339baafe80 2024-04-21T08:48:57.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.538+0000 7f424ca26740 10 client.20244 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:57.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.538+0000 7f424ca26740 1 -- 172.21.15.43:0/594271544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56339b7f07d0 con 0x56339baafe80 2024-04-21T08:48:57.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.538+0000 7f4222ffd640 1 -- 172.21.15.43:0/594271544 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42380bf1d0 con 0x56339baafe80 2024-04-21T08:48:57.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.538+0000 7f4222ffd640 1 -- 172.21.15.43:0/594271544 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f42380bb140 con 0x56339baafe80 2024-04-21T08:48:57.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.538+0000 7f424ca26740 10 client.20244 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:57.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.538+0000 7f424ca26740 10 client.20244 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:48:57.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.538+0000 7f424ca26740 20 client.20244 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:57.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.538+0000 7f424ca26740 1 --2- 172.21.15.43:0/594271544 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56339bac8730 0x56339bacab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.538+0000 7f424ca26740 4 client.20244 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:57.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.538+0000 7f424ca26740 1 -- 172.21.15.43:0/594271544 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56339b9b3930 con 0x56339bac8730 2024-04-21T08:48:57.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.539+0000 7f42437fe640 1 --2- 172.21.15.43:0/594271544 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56339bac8730 0x56339bacab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.539+0000 7f4222ffd640 10 client.20244 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:48:57.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.539+0000 7f42437fe640 1 --2- 172.21.15.43:0/594271544 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56339bac8730 0x56339bacab20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f0fc10512539f3bd server_cookie=7140025afdb6fbb7 in_seq=0 out_seq=0 2024-04-21T08:48:57.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.540+0000 7f4222ffd640 1 -- 172.21.15.43:0/594271544 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56339b9b3930 con 0x56339bac8730 2024-04-21T08:48:57.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.540+0000 7f4222ffd640 10 client.20244 handle_command_reply: tid=0 2024-04-21T08:48:57.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.540+0000 7f424ca26740 1 client.20244 shutdown 2024-04-21T08:48:57.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.540+0000 7f424ca26740 1 -- 172.21.15.43:0/594271544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339baafe80 msgr2=0x56339bab5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.540+0000 7f424ca26740 1 --2- 172.21.15.43:0/594271544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339baafe80 0x56339bab5600 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f42380f31f0 tx=0x7f42380beb40 comp rx=0 tx=0).stop 2024-04-21T08:48:57.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.540+0000 7f424ca26740 1 -- 172.21.15.43:0/594271544 shutdown_connections 2024-04-21T08:48:57.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.540+0000 7f424ca26740 1 --2- 172.21.15.43:0/594271544 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56339bac8730 0x56339bacab20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.540+0000 7f424ca26740 1 --2- 172.21.15.43:0/594271544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56339bab8a00 0x56339bab5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.540+0000 7f424ca26740 1 --2- 172.21.15.43:0/594271544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339baafe80 0x56339bab5600 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.540+0000 7f4222ffd640 0 client.20244 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:48:57.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.540+0000 7f424ca26740 1 --2- 172.21.15.43:0/594271544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339baaf4d0 0x56339bab8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.540+0000 7f424ca26740 1 -- 172.21.15.43:0/594271544 >> 172.21.15.43:0/594271544 conn(0x56339baa9ba0 msgr2=0x56339baab8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.540+0000 7f424ca26740 1 -- 172.21.15.43:0/594271544 shutdown_connections 2024-04-21T08:48:57.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.540+0000 7f424ca26740 1 -- 172.21.15.43:0/594271544 wait complete. 2024-04-21T08:48:57.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.540+0000 7f424ca26740 20 client.20244 trim_cache size 0 max 16384 2024-04-21T08:48:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.554+0000 7f424ca26740 1 Processor -- start 2024-04-21T08:48:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.554+0000 7f424ca26740 1 -- start start 2024-04-21T08:48:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.554+0000 7f424ca26740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339baaf4d0 0x56339bac43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.554+0000 7f424ca26740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339baafe80 0x56339bab7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.554+0000 7f424ca26740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56339bab8a00 0x56339bab8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.554+0000 7f424ca26740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f42441af610 con 0x56339baafe80 2024-04-21T08:48:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.554+0000 7f424ca26740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f42441b1150 con 0x56339baaf4d0 2024-04-21T08:48:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.554+0000 7f424ca26740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f42441afce0 con 0x56339bab8a00 2024-04-21T08:48:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.554+0000 7f4242ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339baafe80 0x56339bab7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.554+0000 7f4242ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339baafe80 0x56339bab7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47460/0 (socket says 172.21.15.43:47460) 2024-04-21T08:48:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.554+0000 7f4242ffd640 1 -- 172.21.15.43:0/3659333394 learned_addr learned my addr 172.21.15.43:0/3659333394 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.554+0000 7f42437fe640 1 --2- 172.21.15.43:0/3659333394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339baaf4d0 0x56339bac43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.554+0000 7f4243fff640 1 --2- 172.21.15.43:0/3659333394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56339bab8a00 0x56339bab8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.554+0000 7f4242ffd640 1 -- 172.21.15.43:0/3659333394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56339bab8a00 msgr2=0x56339bab8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.554+0000 7f4242ffd640 1 --2- 172.21.15.43:0/3659333394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56339bab8a00 0x56339bab8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.554+0000 7f4242ffd640 1 -- 172.21.15.43:0/3659333394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339baaf4d0 msgr2=0x56339bac43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.554+0000 7f4242ffd640 1 --2- 172.21.15.43:0/3659333394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339baaf4d0 0x56339bac43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.554+0000 7f4242ffd640 1 -- 172.21.15.43:0/3659333394 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56339bab3b50 con 0x56339baafe80 2024-04-21T08:48:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.554+0000 7f4243fff640 1 --2- 172.21.15.43:0/3659333394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56339bab8a00 0x56339bab8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.555+0000 7f4242ffd640 1 --2- 172.21.15.43:0/3659333394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339baafe80 0x56339bab7d30 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f42380be7d0 tx=0x7f42380ba770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9098901ddddb6289 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.555+0000 7f42297fa640 1 -- 172.21.15.43:0/3659333394 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42380be610 con 0x56339baafe80 2024-04-21T08:48:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.555+0000 7f42297fa640 1 -- 172.21.15.43:0/3659333394 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42380bf1d0 con 0x56339baafe80 2024-04-21T08:48:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.555+0000 7f42297fa640 1 -- 172.21.15.43:0/3659333394 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42380054d0 con 0x56339baafe80 2024-04-21T08:48:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.555+0000 7f424ca26740 1 -- 172.21.15.43:0/3659333394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339baafe80 msgr2=0x56339bab7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.555+0000 7f424ca26740 1 --2- 172.21.15.43:0/3659333394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339baafe80 0x56339bab7d30 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f42380be7d0 tx=0x7f42380ba770 comp rx=0 tx=0).stop 2024-04-21T08:48:57.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.555+0000 7f424ca26740 1 -- 172.21.15.43:0/3659333394 shutdown_connections 2024-04-21T08:48:57.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.555+0000 7f424ca26740 1 --2- 172.21.15.43:0/3659333394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56339bab8a00 0x56339bab8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.555+0000 7f424ca26740 1 --2- 172.21.15.43:0/3659333394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339baafe80 0x56339bab7d30 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.555+0000 7f424ca26740 1 --2- 172.21.15.43:0/3659333394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339baaf4d0 0x56339bac43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.555+0000 7f424ca26740 1 -- 172.21.15.43:0/3659333394 >> 172.21.15.43:0/3659333394 conn(0x56339baa9ba0 msgr2=0x56339bab0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.555+0000 7f424ca26740 1 -- 172.21.15.43:0/3659333394 shutdown_connections 2024-04-21T08:48:57.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.555+0000 7f424ca26740 1 -- 172.21.15.43:0/3659333394 wait complete. 2024-04-21T08:48:57.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.556+0000 7f424ca26740 1 -- start start 2024-04-21T08:48:57.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.556+0000 7f424ca26740 1 Processor -- start 2024-04-21T08:48:57.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.556+0000 7f424ca26740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339baaf4d0 0x56339bac0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.556+0000 7f424ca26740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339baafe80 0x56339babc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.556+0000 7f424ca26740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56339bab8a00 0x56339babc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.556+0000 7f424ca26740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f42441b0a80 con 0x56339baaf4d0 2024-04-21T08:48:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.556+0000 7f424ca26740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f42441b03b0 con 0x56339baafe80 2024-04-21T08:48:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.556+0000 7f424ca26740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f42441ad610 con 0x56339bab8a00 2024-04-21T08:48:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.556+0000 7f424ca26740 10 client.0 _ll_register_callbacks cb 0x7f42380035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.556+0000 7f42437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339baaf4d0 0x56339bac0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.556+0000 7f42437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339baaf4d0 0x56339bac0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47464/0 (socket says 172.21.15.43:47464) 2024-04-21T08:48:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.556+0000 7f42437fe640 1 -- 172.21.15.43:0/2368093180 learned_addr learned my addr 172.21.15.43:0/2368093180 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.556+0000 7f4242ffd640 1 --2- 172.21.15.43:0/2368093180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339baafe80 0x56339babc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.556+0000 7f42437fe640 1 -- 172.21.15.43:0/2368093180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56339bab8a00 msgr2=0x56339babc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.556+0000 7f42437fe640 1 --2- 172.21.15.43:0/2368093180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56339bab8a00 0x56339babc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.556+0000 7f42437fe640 1 -- 172.21.15.43:0/2368093180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339baafe80 msgr2=0x56339babc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.556+0000 7f42437fe640 1 --2- 172.21.15.43:0/2368093180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339baafe80 0x56339babc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.556+0000 7f42437fe640 1 -- 172.21.15.43:0/2368093180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56339babcee0 con 0x56339baaf4d0 2024-04-21T08:48:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.557+0000 7f42437fe640 1 --2- 172.21.15.43:0/2368093180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339baaf4d0 0x56339bac0d60 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f4234000c00 tx=0x7f42340039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6daa03d5b6208 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.557+0000 7f4222ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.557+0000 7f4222ffd640 1 -- 172.21.15.43:0/2368093180 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423401d590 con 0x56339baaf4d0 2024-04-21T08:48:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.557+0000 7f4222ffd640 1 -- 172.21.15.43:0/2368093180 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f423401d730 con 0x56339baaf4d0 2024-04-21T08:48:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.557+0000 7f424ca26740 1 -- 172.21.15.43:0/2368093180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42380bf1d0 con 0x56339baaf4d0 2024-04-21T08:48:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.557+0000 7f4222ffd640 1 -- 172.21.15.43:0/2368093180 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f423402d050 con 0x56339baaf4d0 2024-04-21T08:48:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.557+0000 7f424ca26740 10 client.20250 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.557+0000 7f424ca26740 1 -- 172.21.15.43:0/2368093180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56339bb32de0 con 0x56339baaf4d0 2024-04-21T08:48:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.557+0000 7f4222ffd640 1 -- 172.21.15.43:0/2368093180 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42380bf1d0 con 0x56339baaf4d0 2024-04-21T08:48:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.557+0000 7f4222ffd640 1 -- 172.21.15.43:0/2368093180 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4234025410 con 0x56339baaf4d0 2024-04-21T08:48:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.557+0000 7f424ca26740 10 client.20250 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.557+0000 7f424ca26740 10 client.20250 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:48:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.557+0000 7f424ca26740 20 client.20250 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.557+0000 7f424ca26740 1 --2- 172.21.15.43:0/2368093180 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56339bb335b0 0x56339bb359f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.557+0000 7f424ca26740 4 client.20250 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.557+0000 7f424ca26740 1 -- 172.21.15.43:0/2368093180 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56339ba21c10 con 0x56339bb335b0 2024-04-21T08:48:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.558+0000 7f4242ffd640 1 --2- 172.21.15.43:0/2368093180 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56339bb335b0 0x56339bb359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.558+0000 7f4242ffd640 1 --2- 172.21.15.43:0/2368093180 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56339bb335b0 0x56339bb359f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=575d17dce4205d18 server_cookie=8a6d8816104ef1e5 in_seq=0 out_seq=0 2024-04-21T08:48:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.558+0000 7f4222ffd640 10 client.20250 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:48:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.558+0000 7f4222ffd640 1 -- 172.21.15.43:0/2368093180 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56339ba21c10 con 0x56339bb335b0 2024-04-21T08:48:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.558+0000 7f4222ffd640 10 client.20250 handle_command_reply: tid=0 2024-04-21T08:48:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.558+0000 7f424ca26740 1 client.20250 shutdown 2024-04-21T08:48:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.558+0000 7f424ca26740 1 -- 172.21.15.43:0/2368093180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339baaf4d0 msgr2=0x56339bac0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.558+0000 7f424ca26740 1 --2- 172.21.15.43:0/2368093180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339baaf4d0 0x56339bac0d60 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f4234000c00 tx=0x7f42340039c0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.558+0000 7f424ca26740 1 -- 172.21.15.43:0/2368093180 shutdown_connections 2024-04-21T08:48:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.558+0000 7f424ca26740 1 --2- 172.21.15.43:0/2368093180 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x56339bb335b0 0x56339bb359f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.558+0000 7f424ca26740 1 --2- 172.21.15.43:0/2368093180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56339bab8a00 0x56339babc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.558+0000 7f424ca26740 1 --2- 172.21.15.43:0/2368093180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339baafe80 0x56339babc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.558+0000 7f424ca26740 1 --2- 172.21.15.43:0/2368093180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56339baaf4d0 0x56339bac0d60 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.558+0000 7f424ca26740 1 -- 172.21.15.43:0/2368093180 >> 172.21.15.43:0/2368093180 conn(0x56339baa9ba0 msgr2=0x56339baa8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.558+0000 7f4222ffd640 0 client.20250 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:48:57.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.558+0000 7f424ca26740 1 -- 172.21.15.43:0/2368093180 shutdown_connections 2024-04-21T08:48:57.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.558+0000 7f424ca26740 1 -- 172.21.15.43:0/2368093180 wait complete. 2024-04-21T08:48:57.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.559+0000 7f424ca26740 20 client.20250 trim_cache size 0 max 16384 2024-04-21T08:48:57.565 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:48:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.559+0000 7f4249e31640 1 -- 172.21.15.43:0/318408434 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f422c0a3d20 msgr2=0x7f422c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.559+0000 7f4249e31640 1 --2- 172.21.15.43:0/318408434 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f422c0a3d20 0x7f422c0a61e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f423c005c40 tx=0x7f423c009d50 comp rx=0 tx=0).stop 2024-04-21T08:48:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.559+0000 7f4249e31640 1 -- 172.21.15.43:0/318408434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4244107260 msgr2=0x7f424419e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.559+0000 7f4249e31640 1 --2- 172.21.15.43:0/318408434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4244107260 0x7f424419e950 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f4238036040 tx=0x7f423802c7e0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.559+0000 7f4249e31640 1 -- 172.21.15.43:0/318408434 shutdown_connections 2024-04-21T08:48:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.559+0000 7f4249e31640 1 --2- 172.21.15.43:0/318408434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42441985c0 0x7f4244198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.559+0000 7f4249e31640 1 --2- 172.21.15.43:0/318408434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4244107260 0x7f424419e950 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.559+0000 7f4249e31640 1 --2- 172.21.15.43:0/318408434 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f422c0a3d20 0x7f422c0a61e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.559+0000 7f4249e31640 1 --2- 172.21.15.43:0/318408434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4244102190 0x7f424419e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.559+0000 7f4249e31640 1 -- 172.21.15.43:0/318408434 >> 172.21.15.43:0/318408434 conn(0x7f42440fd990 msgr2=0x7f42440fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.559+0000 7f4249e31640 1 -- 172.21.15.43:0/318408434 shutdown_connections 2024-04-21T08:48:57.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.559+0000 7f4249e31640 1 -- 172.21.15.43:0/318408434 wait complete. 2024-04-21T08:48:57.569 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:48:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.674+0000 7ff08504e640 1 -- 172.21.15.43:0/448784170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0801010a0 msgr2=0x7ff080101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.674+0000 7ff08504e640 1 --2- 172.21.15.43:0/448784170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0801010a0 0x7ff080101480 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7ff070009900 tx=0x7ff07002bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.674+0000 7ff08504e640 1 -- 172.21.15.43:0/448784170 shutdown_connections 2024-04-21T08:48:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.674+0000 7ff08504e640 1 --2- 172.21.15.43:0/448784170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff080106170 0x7ff08010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.674+0000 7ff08504e640 1 --2- 172.21.15.43:0/448784170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff080101a50 0x7ff080105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.674+0000 7ff08504e640 1 --2- 172.21.15.43:0/448784170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0801010a0 0x7ff080101480 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.674+0000 7ff08504e640 1 -- 172.21.15.43:0/448784170 >> 172.21.15.43:0/448784170 conn(0x7ff080077eb0 msgr2=0x7ff0800782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.674+0000 7ff08504e640 1 -- 172.21.15.43:0/448784170 shutdown_connections 2024-04-21T08:48:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.674+0000 7ff08504e640 1 -- 172.21.15.43:0/448784170 wait complete. 2024-04-21T08:48:57.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.675+0000 7ff08504e640 1 Processor -- start 2024-04-21T08:48:57.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.675+0000 7ff08504e640 1 -- start start 2024-04-21T08:48:57.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.675+0000 7ff08504e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0801010a0 0x7ff080198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.675+0000 7ff08504e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff080106170 0x7ff080198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.675+0000 7ff08504e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff08019d010 0x7ff08019d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.675+0000 7ff08504e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff080112230 con 0x7ff0801010a0 2024-04-21T08:48:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.675+0000 7ff08504e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff0801120b0 con 0x7ff08019d010 2024-04-21T08:48:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.675+0000 7ff08504e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff0801123b0 con 0x7ff080106170 2024-04-21T08:48:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.675+0000 7ff07ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0801010a0 0x7ff080198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.675+0000 7ff07ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0801010a0 0x7ff080198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47484/0 (socket says 172.21.15.43:47484) 2024-04-21T08:48:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.675+0000 7ff07ed9d640 1 -- 172.21.15.43:0/1172550504 learned_addr learned my addr 172.21.15.43:0/1172550504 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.675+0000 7ff07e59c640 1 --2- 172.21.15.43:0/1172550504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff080106170 0x7ff080198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.675+0000 7ff07f59e640 1 --2- 172.21.15.43:0/1172550504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff08019d010 0x7ff08019d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.675+0000 7ff07ed9d640 1 -- 172.21.15.43:0/1172550504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff080106170 msgr2=0x7ff080198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.675+0000 7ff07ed9d640 1 --2- 172.21.15.43:0/1172550504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff080106170 0x7ff080198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.675+0000 7ff07ed9d640 1 -- 172.21.15.43:0/1172550504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff08019d010 msgr2=0x7ff08019d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.675+0000 7ff07ed9d640 1 --2- 172.21.15.43:0/1172550504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff08019d010 0x7ff08019d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.675+0000 7ff07ed9d640 1 -- 172.21.15.43:0/1172550504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08019db00 con 0x7ff0801010a0 2024-04-21T08:48:57.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.675+0000 7ff07f59e640 1 --2- 172.21.15.43:0/1172550504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff08019d010 0x7ff08019d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:57.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.676+0000 7ff07ed9d640 1 --2- 172.21.15.43:0/1172550504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0801010a0 0x7ff080198750 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7ff070005940 tx=0x7ff07002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.676+0000 7ff067fff640 1 -- 172.21.15.43:0/1172550504 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070043020 con 0x7ff0801010a0 2024-04-21T08:48:57.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.676+0000 7ff067fff640 1 -- 172.21.15.43:0/1172550504 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff07002cbe0 con 0x7ff0801010a0 2024-04-21T08:48:57.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.676+0000 7ff08504e640 1 -- 172.21.15.43:0/1172550504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08019dd90 con 0x7ff0801010a0 2024-04-21T08:48:57.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.676+0000 7ff067fff640 1 -- 172.21.15.43:0/1172550504 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070003bf0 con 0x7ff0801010a0 2024-04-21T08:48:57.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.676+0000 7ff08504e640 1 -- 172.21.15.43:0/1172550504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0801a55b0 con 0x7ff0801010a0 2024-04-21T08:48:57.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.678+0000 7ff067fff640 1 -- 172.21.15.43:0/1172550504 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff070046020 con 0x7ff0801010a0 2024-04-21T08:48:57.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.678+0000 7ff087c40740 1 Processor -- start 2024-04-21T08:48:57.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.679+0000 7ff087c40740 1 -- start start 2024-04-21T08:48:57.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.679+0000 7ff087c40740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568d1f793b0 0x5568d1f79790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.679+0000 7ff087c40740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f79d60 0x5568d1f82890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.679+0000 7ff087c40740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568d1f82e60 0x5568d1f832c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.679+0000 7ff087c40740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5568d1ede1f0 con 0x5568d1f79d60 2024-04-21T08:48:57.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.679+0000 7ff087c40740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5568d1e7b4b0 con 0x5568d1f793b0 2024-04-21T08:48:57.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.679+0000 7ff087c40740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5568d1eda1b0 con 0x5568d1f82e60 2024-04-21T08:48:57.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.679+0000 7ff067fff640 1 --2- 172.21.15.43:0/1172550504 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff0680a3d20 0x7ff0680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.679+0000 7ff07ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f79d60 0x5568d1f82890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.679+0000 7ff07ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f79d60 0x5568d1f82890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47488/0 (socket says 172.21.15.43:47488) 2024-04-21T08:48:57.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.679+0000 7ff07ed9d640 1 -- 172.21.15.43:0/252397530 learned_addr learned my addr 172.21.15.43:0/252397530 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.679+0000 7ff07f59e640 1 --2- 172.21.15.43:0/1172550504 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff0680a3d20 0x7ff0680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.679+0000 7ff07e59c640 1 --2- 172.21.15.43:0/252397530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568d1f82e60 0x5568d1f832c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.679+0000 7ff07f59e640 1 --2- 172.21.15.43:0/252397530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568d1f793b0 0x5568d1f79790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.679+0000 7ff067fff640 1 -- 172.21.15.43:0/1172550504 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7ff0700f2e40 con 0x7ff0801010a0 2024-04-21T08:48:57.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.679+0000 7ff07ed9d640 1 -- 172.21.15.43:0/252397530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568d1f82e60 msgr2=0x5568d1f832c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.679+0000 7ff07ed9d640 1 --2- 172.21.15.43:0/252397530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568d1f82e60 0x5568d1f832c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.679+0000 7ff07ed9d640 1 -- 172.21.15.43:0/252397530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568d1f793b0 msgr2=0x5568d1f79790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.679+0000 7ff07ed9d640 1 --2- 172.21.15.43:0/252397530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568d1f793b0 0x5568d1f79790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.679+0000 7ff07ed9d640 1 -- 172.21.15.43:0/252397530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568d1e6d1b0 con 0x5568d1f79d60 2024-04-21T08:48:57.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.680+0000 7ff07ed9d640 1 --2- 172.21.15.43:0/252397530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f79d60 0x5568d1f82890 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7ff070005ec0 tx=0x7ff070000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e3f6fa1d4abb13 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.680+0000 7ff07f59e640 1 --2- 172.21.15.43:0/1172550504 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff0680a3d20 0x7ff0680a61e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff07800afd0 tx=0x7ff07800ad00 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.680+0000 7ff0657fa640 1 -- 172.21.15.43:0/252397530 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0700bf1d0 con 0x5568d1f79d60 2024-04-21T08:48:57.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.680+0000 7ff0657fa640 1 -- 172.21.15.43:0/252397530 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff07003c350 con 0x5568d1f79d60 2024-04-21T08:48:57.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.680+0000 7ff0657fa640 1 -- 172.21.15.43:0/252397530 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0700bcdd0 con 0x5568d1f79d60 2024-04-21T08:48:57.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.680+0000 7ff087c40740 1 -- 172.21.15.43:0/252397530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f79d60 msgr2=0x5568d1f82890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.680+0000 7ff087c40740 1 --2- 172.21.15.43:0/252397530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f79d60 0x5568d1f82890 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7ff070005ec0 tx=0x7ff070000bc0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.680+0000 7ff087c40740 1 -- 172.21.15.43:0/252397530 shutdown_connections 2024-04-21T08:48:57.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.680+0000 7ff087c40740 1 --2- 172.21.15.43:0/252397530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568d1f82e60 0x5568d1f832c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.680+0000 7ff087c40740 1 --2- 172.21.15.43:0/252397530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f79d60 0x5568d1f82890 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.680+0000 7ff087c40740 1 --2- 172.21.15.43:0/252397530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568d1f793b0 0x5568d1f79790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.680+0000 7ff087c40740 1 -- 172.21.15.43:0/252397530 >> 172.21.15.43:0/252397530 conn(0x5568d1f73ba0 msgr2=0x5568d1f73f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.680+0000 7ff087c40740 1 -- 172.21.15.43:0/252397530 shutdown_connections 2024-04-21T08:48:57.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.680+0000 7ff087c40740 1 -- 172.21.15.43:0/252397530 wait complete. 2024-04-21T08:48:57.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.681+0000 7ff087c40740 1 -- start start 2024-04-21T08:48:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.681+0000 7ff087c40740 1 Processor -- start 2024-04-21T08:48:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.681+0000 7ff087c40740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f793b0 0x5568d1f89250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.681+0000 7ff087c40740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568d1f79d60 0x5568d1f865e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.681+0000 7ff087c40740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568d1f82e60 0x5568d1f86b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.681+0000 7ff087c40740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5568d1f89b60 con 0x5568d1f793b0 2024-04-21T08:48:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.681+0000 7ff087c40740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5568d1e97500 con 0x5568d1f82e60 2024-04-21T08:48:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.681+0000 7ff087c40740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5568d1f89ce0 con 0x5568d1f79d60 2024-04-21T08:48:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.681+0000 7ff087c40740 10 client.0 _ll_register_callbacks cb 0x5568d1e65b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.681+0000 7ff07ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f793b0 0x5568d1f89250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.681+0000 7ff07ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f793b0 0x5568d1f89250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47502/0 (socket says 172.21.15.43:47502) 2024-04-21T08:48:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.681+0000 7ff07ed9d640 1 -- 172.21.15.43:0/2622668618 learned_addr learned my addr 172.21.15.43:0/2622668618 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.681+0000 7ff07f59e640 1 --2- 172.21.15.43:0/2622668618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568d1f82e60 0x5568d1f86b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.681+0000 7ff07ed9d640 1 -- 172.21.15.43:0/2622668618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568d1f79d60 msgr2=0x5568d1f865e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.681+0000 7ff07e59c640 1 --2- 172.21.15.43:0/2622668618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568d1f79d60 0x5568d1f865e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.681+0000 7ff07ed9d640 1 --2- 172.21.15.43:0/2622668618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568d1f79d60 0x5568d1f865e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.681+0000 7ff07ed9d640 1 -- 172.21.15.43:0/2622668618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568d1f82e60 msgr2=0x5568d1f86b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.681+0000 7ff07ed9d640 1 --2- 172.21.15.43:0/2622668618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568d1f82e60 0x5568d1f86b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.681+0000 7ff07ed9d640 1 -- 172.21.15.43:0/2622668618 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568d1bf03f0 con 0x5568d1f793b0 2024-04-21T08:48:57.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.681+0000 7ff07f59e640 1 --2- 172.21.15.43:0/2622668618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568d1f82e60 0x5568d1f86b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:57.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.682+0000 7ff07ed9d640 1 --2- 172.21.15.43:0/2622668618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f793b0 0x5568d1f89250 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7ff0700f0330 tx=0x7ff0700f0220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a3023f2d483b59 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.682+0000 7ff062ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:57.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.682+0000 7ff062ff5640 1 -- 172.21.15.43:0/2622668618 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0700bead0 con 0x5568d1f793b0 2024-04-21T08:48:57.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.682+0000 7ff062ff5640 1 -- 172.21.15.43:0/2622668618 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7ff0700054d0 con 0x5568d1f793b0 2024-04-21T08:48:57.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.682+0000 7ff087c40740 1 -- 172.21.15.43:0/2622668618 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff07003c350 con 0x5568d1f793b0 2024-04-21T08:48:57.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.682+0000 7ff062ff5640 1 -- 172.21.15.43:0/2622668618 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff0700bf1d0 con 0x5568d1f793b0 2024-04-21T08:48:57.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.682+0000 7ff087c40740 10 client.20262 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:57.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.682+0000 7ff087c40740 1 -- 172.21.15.43:0/2622668618 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5568d1cba7d0 con 0x5568d1f793b0 2024-04-21T08:48:57.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.682+0000 7ff062ff5640 1 -- 172.21.15.43:0/2622668618 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff07003c350 con 0x5568d1f793b0 2024-04-21T08:48:57.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.682+0000 7ff062ff5640 1 -- 172.21.15.43:0/2622668618 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff0700f4070 con 0x5568d1f793b0 2024-04-21T08:48:57.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.682+0000 7ff087c40740 10 client.20262 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:57.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.682+0000 7ff087c40740 10 client.20262 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:48:57.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.682+0000 7ff087c40740 20 client.20262 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:57.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.682+0000 7ff087c40740 1 --2- 172.21.15.43:0/2622668618 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5568d1f96b70 0x5568d1f98f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.682+0000 7ff087c40740 4 client.20262 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:57.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.682+0000 7ff087c40740 1 -- 172.21.15.43:0/2622668618 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568d1e7d930 con 0x5568d1f96b70 2024-04-21T08:48:57.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.683+0000 7ff07e59c640 1 --2- 172.21.15.43:0/2622668618 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5568d1f96b70 0x5568d1f98f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.683+0000 7ff07e59c640 1 --2- 172.21.15.43:0/2622668618 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5568d1f96b70 0x5568d1f98f60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fda5cad7a8bd00c1 server_cookie=cea0d29286413a49 in_seq=0 out_seq=0 2024-04-21T08:48:57.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.683+0000 7ff062ff5640 10 client.20262 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:48:57.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.684+0000 7ff062ff5640 1 -- 172.21.15.43:0/2622668618 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568d1e7d930 con 0x5568d1f96b70 2024-04-21T08:48:57.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.684+0000 7ff062ff5640 10 client.20262 handle_command_reply: tid=0 2024-04-21T08:48:57.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.684+0000 7ff087c40740 1 client.20262 shutdown 2024-04-21T08:48:57.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.684+0000 7ff087c40740 1 -- 172.21.15.43:0/2622668618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f793b0 msgr2=0x5568d1f89250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.684+0000 7ff087c40740 1 --2- 172.21.15.43:0/2622668618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f793b0 0x5568d1f89250 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7ff0700f0330 tx=0x7ff0700f0220 comp rx=0 tx=0).stop 2024-04-21T08:48:57.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.684+0000 7ff087c40740 1 -- 172.21.15.43:0/2622668618 shutdown_connections 2024-04-21T08:48:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.684+0000 7ff087c40740 1 --2- 172.21.15.43:0/2622668618 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5568d1f96b70 0x5568d1f98f60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.684+0000 7ff087c40740 1 --2- 172.21.15.43:0/2622668618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568d1f82e60 0x5568d1f86b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.684+0000 7ff087c40740 1 --2- 172.21.15.43:0/2622668618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568d1f79d60 0x5568d1f865e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.684+0000 7ff087c40740 1 --2- 172.21.15.43:0/2622668618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f793b0 0x5568d1f89250 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.684+0000 7ff062ff5640 0 client.20262 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:48:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.684+0000 7ff087c40740 1 -- 172.21.15.43:0/2622668618 >> 172.21.15.43:0/2622668618 conn(0x5568d1f73ba0 msgr2=0x5568d1f758f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.684+0000 7ff087c40740 1 -- 172.21.15.43:0/2622668618 shutdown_connections 2024-04-21T08:48:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.684+0000 7ff087c40740 1 -- 172.21.15.43:0/2622668618 wait complete. 2024-04-21T08:48:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.684+0000 7ff087c40740 20 client.20262 trim_cache size 0 max 16384 2024-04-21T08:48:57.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.697+0000 7ff087c40740 1 Processor -- start 2024-04-21T08:48:57.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.697+0000 7ff087c40740 1 -- start start 2024-04-21T08:48:57.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.697+0000 7ff087c40740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568d1f73ba0 0x5568d1f928f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.697+0000 7ff087c40740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f793b0 0x5568d1f84d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.697+0000 7ff087c40740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568d1f79d60 0x5568d1f81800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.697+0000 7ff087c40740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff0801ad290 con 0x5568d1f793b0 2024-04-21T08:48:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.697+0000 7ff087c40740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff0801aedd0 con 0x5568d1f79d60 2024-04-21T08:48:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.697+0000 7ff087c40740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff0801ad960 con 0x5568d1f73ba0 2024-04-21T08:48:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.698+0000 7ff07e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f793b0 0x5568d1f84d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.698+0000 7ff07e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f793b0 0x5568d1f84d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47514/0 (socket says 172.21.15.43:47514) 2024-04-21T08:48:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.698+0000 7ff07e59c640 1 -- 172.21.15.43:0/134965745 learned_addr learned my addr 172.21.15.43:0/134965745 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.698+0000 7ff07e59c640 1 -- 172.21.15.43:0/134965745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568d1f73ba0 msgr2=0x5568d1f928f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.698+0000 7ff07e59c640 1 --2- 172.21.15.43:0/134965745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568d1f73ba0 0x5568d1f928f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.698+0000 7ff07e59c640 1 -- 172.21.15.43:0/134965745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568d1f79d60 msgr2=0x5568d1f81800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.698+0000 7ff07f59e640 1 --2- 172.21.15.43:0/134965745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568d1f79d60 0x5568d1f81800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.698+0000 7ff07e59c640 1 --2- 172.21.15.43:0/134965745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568d1f79d60 0x5568d1f81800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.698+0000 7ff07e59c640 1 -- 172.21.15.43:0/134965745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568d1f899d0 con 0x5568d1f793b0 2024-04-21T08:48:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.698+0000 7ff07f59e640 1 --2- 172.21.15.43:0/134965745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568d1f79d60 0x5568d1f81800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.698+0000 7ff07e59c640 1 --2- 172.21.15.43:0/134965745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f793b0 0x5568d1f84d80 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7ff074000c00 tx=0x7ff07400b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01f5bcaf50caee server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.698+0000 7ff064ff9640 1 -- 172.21.15.43:0/134965745 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074014110 con 0x5568d1f793b0 2024-04-21T08:48:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.698+0000 7ff064ff9640 1 -- 172.21.15.43:0/134965745 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0740142b0 con 0x5568d1f793b0 2024-04-21T08:48:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.698+0000 7ff064ff9640 1 -- 172.21.15.43:0/134965745 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074005380 con 0x5568d1f793b0 2024-04-21T08:48:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.699+0000 7ff087c40740 1 -- 172.21.15.43:0/134965745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f793b0 msgr2=0x5568d1f84d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.699+0000 7ff087c40740 1 --2- 172.21.15.43:0/134965745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f793b0 0x5568d1f84d80 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7ff074000c00 tx=0x7ff07400b4d0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.699+0000 7ff087c40740 1 -- 172.21.15.43:0/134965745 shutdown_connections 2024-04-21T08:48:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.699+0000 7ff087c40740 1 --2- 172.21.15.43:0/134965745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568d1f79d60 0x5568d1f81800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.699+0000 7ff087c40740 1 --2- 172.21.15.43:0/134965745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f793b0 0x5568d1f84d80 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.699+0000 7ff087c40740 1 --2- 172.21.15.43:0/134965745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568d1f73ba0 0x5568d1f928f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.699+0000 7ff087c40740 1 -- 172.21.15.43:0/134965745 >> 172.21.15.43:0/134965745 conn(0x5568d1f86c80 msgr2=0x5568d1f757f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.699+0000 7ff087c40740 1 -- 172.21.15.43:0/134965745 shutdown_connections 2024-04-21T08:48:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.699+0000 7ff087c40740 1 -- 172.21.15.43:0/134965745 wait complete. 2024-04-21T08:48:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.699+0000 7ff087c40740 1 -- start start 2024-04-21T08:48:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.699+0000 7ff087c40740 1 Processor -- start 2024-04-21T08:48:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.699+0000 7ff087c40740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568d1f73ba0 0x5568d1f885e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.699+0000 7ff087c40740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568d1f793b0 0x5568d1f88b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.699+0000 7ff087c40740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f79d60 0x5568d1ffbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.699+0000 7ff087c40740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff0801ae700 con 0x5568d1f79d60 2024-04-21T08:48:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.699+0000 7ff087c40740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff0801ae030 con 0x5568d1f73ba0 2024-04-21T08:48:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.699+0000 7ff087c40740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff0801ab290 con 0x5568d1f793b0 2024-04-21T08:48:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.699+0000 7ff087c40740 10 client.0 _ll_register_callbacks cb 0x7ff0700c7740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.699+0000 7ff07f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f79d60 0x5568d1ffbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.700+0000 7ff07f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f79d60 0x5568d1ffbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47520/0 (socket says 172.21.15.43:47520) 2024-04-21T08:48:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.700+0000 7ff07f59e640 1 -- 172.21.15.43:0/93181074 learned_addr learned my addr 172.21.15.43:0/93181074 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.700+0000 7ff07ed9d640 1 --2- 172.21.15.43:0/93181074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568d1f73ba0 0x5568d1f885e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.700+0000 7ff07e59c640 1 --2- 172.21.15.43:0/93181074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568d1f793b0 0x5568d1f88b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.700+0000 7ff07f59e640 1 -- 172.21.15.43:0/93181074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568d1f793b0 msgr2=0x5568d1f88b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.700+0000 7ff07f59e640 1 --2- 172.21.15.43:0/93181074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568d1f793b0 0x5568d1f88b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.700+0000 7ff07f59e640 1 -- 172.21.15.43:0/93181074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568d1f73ba0 msgr2=0x5568d1f885e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.700+0000 7ff07f59e640 1 --2- 172.21.15.43:0/93181074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568d1f73ba0 0x5568d1f885e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.700+0000 7ff07f59e640 1 -- 172.21.15.43:0/93181074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568d1ffc400 con 0x5568d1f79d60 2024-04-21T08:48:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.700+0000 7ff07e59c640 1 --2- 172.21.15.43:0/93181074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568d1f793b0 0x5568d1f88b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.700+0000 7ff07f59e640 1 --2- 172.21.15.43:0/93181074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f79d60 0x5568d1ffbec0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7ff07800ab60 tx=0x7ff0780012a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6007cca1c0880b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.700+0000 7ff062ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.700+0000 7ff062ff5640 1 -- 172.21.15.43:0/93181074 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07800fa50 con 0x5568d1f79d60 2024-04-21T08:48:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.700+0000 7ff062ff5640 1 -- 172.21.15.43:0/93181074 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7ff07800fbf0 con 0x5568d1f79d60 2024-04-21T08:48:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.700+0000 7ff087c40740 1 -- 172.21.15.43:0/93181074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5568d1eebc10 con 0x5568d1f79d60 2024-04-21T08:48:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.700+0000 7ff062ff5640 1 -- 172.21.15.43:0/93181074 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff078031050 con 0x5568d1f79d60 2024-04-21T08:48:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.700+0000 7ff087c40740 10 client.20268 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.700+0000 7ff087c40740 1 -- 172.21.15.43:0/93181074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5568d1ffd4d0 con 0x5568d1f79d60 2024-04-21T08:48:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.700+0000 7ff062ff5640 1 -- 172.21.15.43:0/93181074 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5568d1eebc10 con 0x5568d1f79d60 2024-04-21T08:48:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.701+0000 7ff062ff5640 1 -- 172.21.15.43:0/93181074 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff078010040 con 0x5568d1f79d60 2024-04-21T08:48:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.701+0000 7ff087c40740 10 client.20268 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.701+0000 7ff087c40740 10 client.20268 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:48:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.701+0000 7ff087c40740 20 client.20268 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.701+0000 7ff087c40740 1 --2- 172.21.15.43:0/93181074 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5568d1f82e60 0x5568d1fffce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.701+0000 7ff087c40740 4 client.20268 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.701+0000 7ff087c40740 1 -- 172.21.15.43:0/93181074 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5568d1ee5a80 con 0x5568d1f82e60 2024-04-21T08:48:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.701+0000 7ff07ed9d640 1 --2- 172.21.15.43:0/93181074 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5568d1f82e60 0x5568d1fffce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.701+0000 7ff07ed9d640 1 --2- 172.21.15.43:0/93181074 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5568d1f82e60 0x5568d1fffce0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=77dc1f084986a5b server_cookie=6a3deb4e6550e56a in_seq=0 out_seq=0 2024-04-21T08:48:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.701+0000 7ff062ff5640 10 client.20268 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:48:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff062ff5640 1 -- 172.21.15.43:0/93181074 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5568d1ee5a80 con 0x5568d1f82e60 2024-04-21T08:48:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff062ff5640 10 client.20268 handle_command_reply: tid=0 2024-04-21T08:48:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff087c40740 1 client.20268 shutdown 2024-04-21T08:48:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff087c40740 1 -- 172.21.15.43:0/93181074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f79d60 msgr2=0x5568d1ffbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff087c40740 1 --2- 172.21.15.43:0/93181074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f79d60 0x5568d1ffbec0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7ff07800ab60 tx=0x7ff0780012a0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff087c40740 1 -- 172.21.15.43:0/93181074 shutdown_connections 2024-04-21T08:48:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff087c40740 1 --2- 172.21.15.43:0/93181074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5568d1f79d60 0x5568d1ffbec0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff087c40740 1 --2- 172.21.15.43:0/93181074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5568d1f793b0 0x5568d1f88b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff087c40740 1 --2- 172.21.15.43:0/93181074 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5568d1f82e60 0x5568d1fffce0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff087c40740 1 --2- 172.21.15.43:0/93181074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5568d1f73ba0 0x5568d1f885e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff062ff5640 0 client.20268 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:48:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff087c40740 1 -- 172.21.15.43:0/93181074 >> 172.21.15.43:0/93181074 conn(0x5568d1f86c80 msgr2=0x5568d1f72370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff087c40740 1 -- 172.21.15.43:0/93181074 shutdown_connections 2024-04-21T08:48:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff087c40740 1 -- 172.21.15.43:0/93181074 wait complete. 2024-04-21T08:48:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff087c40740 20 client.20268 trim_cache size 0 max 16384 2024-04-21T08:48:57.708 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:48:57.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff08504e640 1 -- 172.21.15.43:0/1172550504 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff0680a3d20 msgr2=0x7ff0680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff08504e640 1 --2- 172.21.15.43:0/1172550504 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff0680a3d20 0x7ff0680a61e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff07800afd0 tx=0x7ff07800ad00 comp rx=0 tx=0).stop 2024-04-21T08:48:57.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff08504e640 1 -- 172.21.15.43:0/1172550504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0801010a0 msgr2=0x7ff080198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff08504e640 1 --2- 172.21.15.43:0/1172550504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0801010a0 0x7ff080198750 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7ff070005940 tx=0x7ff07002c7e0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff08504e640 1 -- 172.21.15.43:0/1172550504 shutdown_connections 2024-04-21T08:48:57.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff08504e640 1 --2- 172.21.15.43:0/1172550504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff08019d010 0x7ff08019d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff08504e640 1 --2- 172.21.15.43:0/1172550504 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff0680a3d20 0x7ff0680a61e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff08504e640 1 --2- 172.21.15.43:0/1172550504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff080106170 0x7ff080198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff08504e640 1 --2- 172.21.15.43:0/1172550504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0801010a0 0x7ff080198750 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff08504e640 1 -- 172.21.15.43:0/1172550504 >> 172.21.15.43:0/1172550504 conn(0x7ff080077eb0 msgr2=0x7ff080107340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff08504e640 1 -- 172.21.15.43:0/1172550504 shutdown_connections 2024-04-21T08:48:57.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.702+0000 7ff08504e640 1 -- 172.21.15.43:0/1172550504 wait complete. 2024-04-21T08:48:57.713 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:48:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.817+0000 7f179c670640 1 -- 172.21.15.43:0/961618269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17941010a0 msgr2=0x7f1794101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.817+0000 7f179c670640 1 --2- 172.21.15.43:0/961618269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17941010a0 0x7f1794101480 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f1790009900 tx=0x7f179002bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.817+0000 7f179c670640 1 -- 172.21.15.43:0/961618269 shutdown_connections 2024-04-21T08:48:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.817+0000 7f179c670640 1 --2- 172.21.15.43:0/961618269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1794106170 0x7f179410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.817+0000 7f179c670640 1 --2- 172.21.15.43:0/961618269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1794101a50 0x7f1794105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.817+0000 7f179c670640 1 --2- 172.21.15.43:0/961618269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17941010a0 0x7f1794101480 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.817+0000 7f179c670640 1 -- 172.21.15.43:0/961618269 >> 172.21.15.43:0/961618269 conn(0x7f1794077eb0 msgr2=0x7f17940782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.817+0000 7f179c670640 1 -- 172.21.15.43:0/961618269 shutdown_connections 2024-04-21T08:48:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.817+0000 7f179c670640 1 -- 172.21.15.43:0/961618269 wait complete. 2024-04-21T08:48:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.817+0000 7f179c670640 1 Processor -- start 2024-04-21T08:48:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.818+0000 7f179c670640 1 -- start start 2024-04-21T08:48:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.818+0000 7f179c670640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17941010a0 0x7f1794198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.818+0000 7f179c670640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1794106170 0x7f1794198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.818+0000 7f179c670640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f179419cfd0 0x7f179419d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.818+0000 7f179c670640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17941121f0 con 0x7f1794106170 2024-04-21T08:48:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.818+0000 7f179c670640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1794112070 con 0x7f17941010a0 2024-04-21T08:48:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.818+0000 7f179c670640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1794112370 con 0x7f179419cfd0 2024-04-21T08:48:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.818+0000 7f1799c0b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1794106170 0x7f1794198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.818+0000 7f1799c0b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1794106170 0x7f1794198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47546/0 (socket says 172.21.15.43:47546) 2024-04-21T08:48:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.818+0000 7f1799c0b640 1 -- 172.21.15.43:0/3146416789 learned_addr learned my addr 172.21.15.43:0/3146416789 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.818+0000 7f179a40c640 1 --2- 172.21.15.43:0/3146416789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17941010a0 0x7f1794198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.818+0000 7f179ac0d640 1 --2- 172.21.15.43:0/3146416789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f179419cfd0 0x7f179419d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.818+0000 7f1799c0b640 1 -- 172.21.15.43:0/3146416789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f179419cfd0 msgr2=0x7f179419d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.818+0000 7f1799c0b640 1 --2- 172.21.15.43:0/3146416789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f179419cfd0 0x7f179419d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.818+0000 7f1799c0b640 1 -- 172.21.15.43:0/3146416789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17941010a0 msgr2=0x7f1794198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.818+0000 7f1799c0b640 1 --2- 172.21.15.43:0/3146416789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17941010a0 0x7f1794198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.818+0000 7f1799c0b640 1 -- 172.21.15.43:0/3146416789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179419dac0 con 0x7f1794106170 2024-04-21T08:48:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.818+0000 7f1799c0b640 1 --2- 172.21.15.43:0/3146416789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1794106170 0x7f1794198c50 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f178400e9c0 tx=0x7f178400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.818+0000 7f178b7fe640 1 -- 172.21.15.43:0/3146416789 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178400cd80 con 0x7f1794106170 2024-04-21T08:48:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.819+0000 7f178b7fe640 1 -- 172.21.15.43:0/3146416789 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17840054d0 con 0x7f1794106170 2024-04-21T08:48:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.819+0000 7f179c670640 1 -- 172.21.15.43:0/3146416789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179419ddb0 con 0x7f1794106170 2024-04-21T08:48:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.819+0000 7f178b7fe640 1 -- 172.21.15.43:0/3146416789 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1784010430 con 0x7f1794106170 2024-04-21T08:48:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.819+0000 7f179c670640 1 -- 172.21.15.43:0/3146416789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17941a55f0 con 0x7f1794106170 2024-04-21T08:48:57.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.821+0000 7f178b7fe640 1 -- 172.21.15.43:0/3146416789 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1784004fd0 con 0x7f1794106170 2024-04-21T08:48:57.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.821+0000 7f179f25c740 1 Processor -- start 2024-04-21T08:48:57.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.821+0000 7f179f25c740 1 -- start start 2024-04-21T08:48:57.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.821+0000 7f179f25c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604433e2330 0x5604433e2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.821+0000 7f179f25c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433e2ce0 0x5604433eb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.821+0000 7f179f25c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604433ebe40 0x5604433ec2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.821+0000 7f179f25c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5604433471f0 con 0x5604433e2ce0 2024-04-21T08:48:57.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.821+0000 7f179f25c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5604432e44b0 con 0x5604433e2330 2024-04-21T08:48:57.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.821+0000 7f179f25c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5604433431b0 con 0x5604433ebe40 2024-04-21T08:48:57.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.822+0000 7f178b7fe640 1 --2- 172.21.15.43:0/3146416789 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f17800a3da0 0x7f17800a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.822+0000 7f179a40c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433e2ce0 0x5604433eb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.822+0000 7f179a40c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433e2ce0 0x5604433eb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47556/0 (socket says 172.21.15.43:47556) 2024-04-21T08:48:57.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.822+0000 7f179a40c640 1 -- 172.21.15.43:0/1173090187 learned_addr learned my addr 172.21.15.43:0/1173090187 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.822+0000 7f179ac0d640 1 --2- 172.21.15.43:0/3146416789 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f17800a3da0 0x7f17800a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.822+0000 7f178b7fe640 1 -- 172.21.15.43:0/3146416789 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f17840d1020 con 0x7f1794106170 2024-04-21T08:48:57.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.822+0000 7f1799c0b640 1 --2- 172.21.15.43:0/1173090187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604433ebe40 0x5604433ec2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.822+0000 7f179ac0d640 1 --2- 172.21.15.43:0/1173090187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604433e2330 0x5604433e2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.822+0000 7f179a40c640 1 -- 172.21.15.43:0/1173090187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604433ebe40 msgr2=0x5604433ec2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.822+0000 7f179a40c640 1 --2- 172.21.15.43:0/1173090187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604433ebe40 0x5604433ec2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.822+0000 7f179a40c640 1 -- 172.21.15.43:0/1173090187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604433e2330 msgr2=0x5604433e2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.822+0000 7f179a40c640 1 --2- 172.21.15.43:0/1173090187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604433e2330 0x5604433e2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.822+0000 7f179a40c640 1 -- 172.21.15.43:0/1173090187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604432d61b0 con 0x5604433e2ce0 2024-04-21T08:48:57.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.822+0000 7f179a40c640 1 --2- 172.21.15.43:0/1173090187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433e2ce0 0x5604433eb820 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f1790009900 tx=0x7f179002c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a4ec14b27d7079 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.822+0000 7f179ac0d640 1 --2- 172.21.15.43:0/3146416789 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f17800a3da0 0x7f17800a6260 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f178c005c40 tx=0x7f178c0094f0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.822+0000 7f1788bfe640 1 -- 172.21.15.43:0/1173090187 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790045020 con 0x5604433e2ce0 2024-04-21T08:48:57.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.822+0000 7f1788bfe640 1 -- 172.21.15.43:0/1173090187 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f179002c7e0 con 0x5604433e2ce0 2024-04-21T08:48:57.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.822+0000 7f1788bfe640 1 -- 172.21.15.43:0/1173090187 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17900046c0 con 0x5604433e2ce0 2024-04-21T08:48:57.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.823+0000 7f179f25c740 1 -- 172.21.15.43:0/1173090187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433e2ce0 msgr2=0x5604433eb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.823+0000 7f179f25c740 1 --2- 172.21.15.43:0/1173090187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433e2ce0 0x5604433eb820 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f1790009900 tx=0x7f179002c610 comp rx=0 tx=0).stop 2024-04-21T08:48:57.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.823+0000 7f179f25c740 1 -- 172.21.15.43:0/1173090187 shutdown_connections 2024-04-21T08:48:57.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.823+0000 7f179f25c740 1 --2- 172.21.15.43:0/1173090187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604433ebe40 0x5604433ec2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.823+0000 7f179f25c740 1 --2- 172.21.15.43:0/1173090187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433e2ce0 0x5604433eb820 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.823+0000 7f179f25c740 1 --2- 172.21.15.43:0/1173090187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604433e2330 0x5604433e2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.823+0000 7f179f25c740 1 -- 172.21.15.43:0/1173090187 >> 172.21.15.43:0/1173090187 conn(0x5604433dcba0 msgr2=0x5604433dcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.823+0000 7f179f25c740 1 -- 172.21.15.43:0/1173090187 shutdown_connections 2024-04-21T08:48:57.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.823+0000 7f179f25c740 1 -- 172.21.15.43:0/1173090187 wait complete. 2024-04-21T08:48:57.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.824+0000 7f179f25c740 1 -- start start 2024-04-21T08:48:57.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.824+0000 7f179f25c740 1 Processor -- start 2024-04-21T08:48:57.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.824+0000 7f179f25c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433e2330 0x5604433f21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.824+0000 7f179f25c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604433e2ce0 0x5604433ef580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.824+0000 7f179f25c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604433ebe40 0x5604433efac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.824+0000 7f179f25c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5604433f2d60 con 0x5604433e2330 2024-04-21T08:48:57.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.824+0000 7f179f25c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560443300500 con 0x5604433e2ce0 2024-04-21T08:48:57.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.824+0000 7f179f25c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5604433f2ee0 con 0x5604433ebe40 2024-04-21T08:48:57.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.824+0000 7f179f25c740 10 client.0 _ll_register_callbacks cb 0x5604432ceb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:57.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.824+0000 7f179a40c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433e2330 0x5604433f21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.824+0000 7f179a40c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433e2330 0x5604433f21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47572/0 (socket says 172.21.15.43:47572) 2024-04-21T08:48:57.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.824+0000 7f179a40c640 1 -- 172.21.15.43:0/286672476 learned_addr learned my addr 172.21.15.43:0/286672476 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.824+0000 7f1799c0b640 1 --2- 172.21.15.43:0/286672476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604433e2ce0 0x5604433ef580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.824+0000 7f179ac0d640 1 --2- 172.21.15.43:0/286672476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604433ebe40 0x5604433efac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.824+0000 7f179a40c640 1 -- 172.21.15.43:0/286672476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604433ebe40 msgr2=0x5604433efac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.824+0000 7f179a40c640 1 --2- 172.21.15.43:0/286672476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604433ebe40 0x5604433efac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.824+0000 7f179a40c640 1 -- 172.21.15.43:0/286672476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604433e2ce0 msgr2=0x5604433ef580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.824+0000 7f179a40c640 1 --2- 172.21.15.43:0/286672476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604433e2ce0 0x5604433ef580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.824+0000 7f179a40c640 1 -- 172.21.15.43:0/286672476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604430593f0 con 0x5604433e2330 2024-04-21T08:48:57.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.824+0000 7f1799c0b640 1 --2- 172.21.15.43:0/286672476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604433e2ce0 0x5604433ef580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:57.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.824+0000 7f179ac0d640 1 --2- 172.21.15.43:0/286672476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604433ebe40 0x5604433efac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:57.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.824+0000 7f179a40c640 1 --2- 172.21.15.43:0/286672476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433e2330 0x5604433f21f0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f17900053c0 tx=0x7f1790000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244d2737f95fa6b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.824+0000 7f177dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:57.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.825+0000 7f177dffb640 1 -- 172.21.15.43:0/286672476 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790045070 con 0x5604433e2330 2024-04-21T08:48:57.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.825+0000 7f177dffb640 1 -- 172.21.15.43:0/286672476 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f17900035f0 con 0x5604433e2330 2024-04-21T08:48:57.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.825+0000 7f179f25c740 1 -- 172.21.15.43:0/286672476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f179002c7e0 con 0x5604433e2330 2024-04-21T08:48:57.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.825+0000 7f177dffb640 1 -- 172.21.15.43:0/286672476 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1790058050 con 0x5604433e2330 2024-04-21T08:48:57.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.825+0000 7f179f25c740 10 client.20280 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:57.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.825+0000 7f179f25c740 1 -- 172.21.15.43:0/286672476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5604431237d0 con 0x5604433e2330 2024-04-21T08:48:57.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.825+0000 7f177dffb640 1 -- 172.21.15.43:0/286672476 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f179002c7e0 con 0x5604433e2330 2024-04-21T08:48:57.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.825+0000 7f177dffb640 1 -- 172.21.15.43:0/286672476 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1790041ac0 con 0x5604433e2330 2024-04-21T08:48:57.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.825+0000 7f179f25c740 10 client.20280 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:57.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.825+0000 7f179f25c740 10 client.20280 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:48:57.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.825+0000 7f179f25c740 20 client.20280 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:57.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.825+0000 7f179f25c740 1 --2- 172.21.15.43:0/286672476 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5604433ffe40 0x560443402230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.825+0000 7f179f25c740 4 client.20280 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:57.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.825+0000 7f179f25c740 1 -- 172.21.15.43:0/286672476 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604432e6930 con 0x5604433ffe40 2024-04-21T08:48:57.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.825+0000 7f1799c0b640 1 --2- 172.21.15.43:0/286672476 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5604433ffe40 0x560443402230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.826+0000 7f1799c0b640 1 --2- 172.21.15.43:0/286672476 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5604433ffe40 0x560443402230 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=962ddb9ae0a43165 server_cookie=429a5c1bbdc1fd5d in_seq=0 out_seq=0 2024-04-21T08:48:57.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.826+0000 7f177dffb640 10 client.20280 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:48:57.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.826+0000 7f177dffb640 1 -- 172.21.15.43:0/286672476 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5604432e6930 con 0x5604433ffe40 2024-04-21T08:48:57.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.826+0000 7f177dffb640 10 client.20280 handle_command_reply: tid=0 2024-04-21T08:48:57.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.827+0000 7f179f25c740 1 client.20280 shutdown 2024-04-21T08:48:57.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.827+0000 7f179f25c740 1 -- 172.21.15.43:0/286672476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433e2330 msgr2=0x5604433f21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.827+0000 7f179f25c740 1 --2- 172.21.15.43:0/286672476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433e2330 0x5604433f21f0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f17900053c0 tx=0x7f1790000bc0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.827+0000 7f179f25c740 1 -- 172.21.15.43:0/286672476 shutdown_connections 2024-04-21T08:48:57.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.827+0000 7f179f25c740 1 --2- 172.21.15.43:0/286672476 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5604433ffe40 0x560443402230 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.827+0000 7f179f25c740 1 --2- 172.21.15.43:0/286672476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604433ebe40 0x5604433efac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.827+0000 7f179f25c740 1 --2- 172.21.15.43:0/286672476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604433e2ce0 0x5604433ef580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.827+0000 7f177dffb640 0 client.20280 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:48:57.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.827+0000 7f179f25c740 1 --2- 172.21.15.43:0/286672476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433e2330 0x5604433f21f0 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.827+0000 7f179f25c740 1 -- 172.21.15.43:0/286672476 >> 172.21.15.43:0/286672476 conn(0x5604433dcba0 msgr2=0x5604433de8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.827+0000 7f179f25c740 1 -- 172.21.15.43:0/286672476 shutdown_connections 2024-04-21T08:48:57.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.827+0000 7f179f25c740 1 -- 172.21.15.43:0/286672476 wait complete. 2024-04-21T08:48:57.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.827+0000 7f179f25c740 20 client.20280 trim_cache size 0 max 16384 2024-04-21T08:48:57.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.840+0000 7f179f25c740 1 Processor -- start 2024-04-21T08:48:57.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.840+0000 7f179f25c740 1 -- start start 2024-04-21T08:48:57.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.840+0000 7f179f25c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604433e2330 0x5604433fbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.840+0000 7f179f25c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604433e2ce0 0x5604433eee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.840+0000 7f179f25c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433ebe40 0x5604433dbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.840+0000 7f179f25c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17941ad260 con 0x5604433ebe40 2024-04-21T08:48:57.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.840+0000 7f179f25c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f17941aeda0 con 0x5604433e2330 2024-04-21T08:48:57.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.840+0000 7f179f25c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f17941ad930 con 0x5604433e2ce0 2024-04-21T08:48:57.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.841+0000 7f179ac0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433ebe40 0x5604433dbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.841+0000 7f179ac0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433ebe40 0x5604433dbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47576/0 (socket says 172.21.15.43:47576) 2024-04-21T08:48:57.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.841+0000 7f179ac0d640 1 -- 172.21.15.43:0/1652614828 learned_addr learned my addr 172.21.15.43:0/1652614828 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.841+0000 7f179a40c640 1 --2- 172.21.15.43:0/1652614828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604433e2330 0x5604433fbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.841+0000 7f1799c0b640 1 --2- 172.21.15.43:0/1652614828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604433e2ce0 0x5604433eee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.841+0000 7f179ac0d640 1 -- 172.21.15.43:0/1652614828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604433e2ce0 msgr2=0x5604433eee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.841+0000 7f179ac0d640 1 --2- 172.21.15.43:0/1652614828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604433e2ce0 0x5604433eee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.841+0000 7f179ac0d640 1 -- 172.21.15.43:0/1652614828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604433e2330 msgr2=0x5604433fbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.841+0000 7f179ac0d640 1 --2- 172.21.15.43:0/1652614828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604433e2330 0x5604433fbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.841+0000 7f179ac0d640 1 -- 172.21.15.43:0/1652614828 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604433e2a80 con 0x5604433ebe40 2024-04-21T08:48:57.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.841+0000 7f179ac0d640 1 --2- 172.21.15.43:0/1652614828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433ebe40 0x5604433dbc00 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f178c010650 tx=0x7f178c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24d4fea5a41c33 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.841+0000 7f177ffff640 1 -- 172.21.15.43:0/1652614828 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c01e730 con 0x5604433ebe40 2024-04-21T08:48:57.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.841+0000 7f177ffff640 1 -- 172.21.15.43:0/1652614828 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f178c01ed80 con 0x5604433ebe40 2024-04-21T08:48:57.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.841+0000 7f177ffff640 1 -- 172.21.15.43:0/1652614828 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c028dd0 con 0x5604433ebe40 2024-04-21T08:48:57.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.842+0000 7f179f25c740 1 -- 172.21.15.43:0/1652614828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433ebe40 msgr2=0x5604433dbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.842+0000 7f179f25c740 1 --2- 172.21.15.43:0/1652614828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433ebe40 0x5604433dbc00 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f178c010650 tx=0x7f178c005bd0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.842+0000 7f179f25c740 1 -- 172.21.15.43:0/1652614828 shutdown_connections 2024-04-21T08:48:57.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.842+0000 7f179f25c740 1 --2- 172.21.15.43:0/1652614828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433ebe40 0x5604433dbc00 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.842+0000 7f179f25c740 1 --2- 172.21.15.43:0/1652614828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604433e2ce0 0x5604433eee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.842+0000 7f179f25c740 1 --2- 172.21.15.43:0/1652614828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604433e2330 0x5604433fbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.842+0000 7f179f25c740 1 -- 172.21.15.43:0/1652614828 >> 172.21.15.43:0/1652614828 conn(0x5604433dcba0 msgr2=0x5604433e9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.842+0000 7f179f25c740 1 -- 172.21.15.43:0/1652614828 shutdown_connections 2024-04-21T08:48:57.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.842+0000 7f179f25c740 1 -- 172.21.15.43:0/1652614828 wait complete. 2024-04-21T08:48:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.842+0000 7f179f25c740 1 -- start start 2024-04-21T08:48:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.842+0000 7f179f25c740 1 Processor -- start 2024-04-21T08:48:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.842+0000 7f179f25c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433e2330 0x5604433f8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.842+0000 7f179f25c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604433e2ce0 0x5604433f86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.842+0000 7f179f25c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604433ebe40 0x5604433f0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.842+0000 7f179f25c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17941ae6d0 con 0x5604433e2330 2024-04-21T08:48:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.842+0000 7f179f25c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f17941ae000 con 0x5604433e2ce0 2024-04-21T08:48:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.842+0000 7f179f25c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f17941ab260 con 0x5604433ebe40 2024-04-21T08:48:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.842+0000 7f179f25c740 10 client.0 _ll_register_callbacks cb 0x7f17900095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.842+0000 7f179a40c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433e2330 0x5604433f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.842+0000 7f179a40c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433e2330 0x5604433f8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47588/0 (socket says 172.21.15.43:47588) 2024-04-21T08:48:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.842+0000 7f179a40c640 1 -- 172.21.15.43:0/517864802 learned_addr learned my addr 172.21.15.43:0/517864802 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.843+0000 7f1799c0b640 1 --2- 172.21.15.43:0/517864802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604433e2ce0 0x5604433f86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.843+0000 7f179ac0d640 1 --2- 172.21.15.43:0/517864802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604433ebe40 0x5604433f0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.843+0000 7f179a40c640 1 -- 172.21.15.43:0/517864802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604433ebe40 msgr2=0x5604433f0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.843+0000 7f179a40c640 1 --2- 172.21.15.43:0/517864802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604433ebe40 0x5604433f0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.843+0000 7f179a40c640 1 -- 172.21.15.43:0/517864802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604433e2ce0 msgr2=0x5604433f86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.843+0000 7f179a40c640 1 --2- 172.21.15.43:0/517864802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604433e2ce0 0x5604433f86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.843+0000 7f179a40c640 1 -- 172.21.15.43:0/517864802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604433f13c0 con 0x5604433e2330 2024-04-21T08:48:57.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.843+0000 7f179ac0d640 1 --2- 172.21.15.43:0/517864802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604433ebe40 0x5604433f0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:57.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.843+0000 7f179a40c640 1 --2- 172.21.15.43:0/517864802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433e2330 0x5604433f8170 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f1790009900 tx=0x7f17900052c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6428892231e5b8cf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.843+0000 7f177dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:57.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.843+0000 7f177dffb640 1 -- 172.21.15.43:0/517864802 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790060020 con 0x5604433e2330 2024-04-21T08:48:57.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.843+0000 7f177dffb640 1 -- 172.21.15.43:0/517864802 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f179004a810 con 0x5604433e2330 2024-04-21T08:48:57.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.843+0000 7f179f25c740 1 -- 172.21.15.43:0/517864802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f178c01ed80 con 0x5604433e2330 2024-04-21T08:48:57.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.843+0000 7f177dffb640 1 -- 172.21.15.43:0/517864802 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f179006b050 con 0x5604433e2330 2024-04-21T08:48:57.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.843+0000 7f179f25c740 10 client.20286 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:57.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.843+0000 7f179f25c740 1 -- 172.21.15.43:0/517864802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560443465c20 con 0x5604433e2330 2024-04-21T08:48:57.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.843+0000 7f177dffb640 1 -- 172.21.15.43:0/517864802 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f178c01ed80 con 0x5604433e2330 2024-04-21T08:48:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.844+0000 7f177dffb640 1 -- 172.21.15.43:0/517864802 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f17900418f0 con 0x5604433e2330 2024-04-21T08:48:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.844+0000 7f179f25c740 10 client.20286 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.844+0000 7f179f25c740 10 client.20286 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:48:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.844+0000 7f179f25c740 20 client.20286 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.844+0000 7f179f25c740 1 --2- 172.21.15.43:0/517864802 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x560443466370 0x560443468790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.844+0000 7f179f25c740 4 client.20286 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.844+0000 7f179f25c740 1 -- 172.21.15.43:0/517864802 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560443354c10 con 0x560443466370 2024-04-21T08:48:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.844+0000 7f1799c0b640 1 --2- 172.21.15.43:0/517864802 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x560443466370 0x560443468790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.844+0000 7f177dffb640 10 client.20286 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:48:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.844+0000 7f1799c0b640 1 --2- 172.21.15.43:0/517864802 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x560443466370 0x560443468790 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f7f4830ecbc25fd0 server_cookie=4625beb597814f84 in_seq=0 out_seq=0 2024-04-21T08:48:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.844+0000 7f177dffb640 1 -- 172.21.15.43:0/517864802 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560443354c10 con 0x560443466370 2024-04-21T08:48:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.844+0000 7f177dffb640 10 client.20286 handle_command_reply: tid=0 2024-04-21T08:48:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.844+0000 7f179f25c740 1 client.20286 shutdown 2024-04-21T08:48:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.844+0000 7f179f25c740 1 -- 172.21.15.43:0/517864802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433e2330 msgr2=0x5604433f8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.844+0000 7f179f25c740 1 --2- 172.21.15.43:0/517864802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433e2330 0x5604433f8170 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f1790009900 tx=0x7f17900052c0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.845+0000 7f179f25c740 1 -- 172.21.15.43:0/517864802 shutdown_connections 2024-04-21T08:48:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.845+0000 7f179f25c740 1 --2- 172.21.15.43:0/517864802 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x560443466370 0x560443468790 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.845+0000 7f179f25c740 1 --2- 172.21.15.43:0/517864802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604433ebe40 0x5604433f0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.845+0000 7f179f25c740 1 --2- 172.21.15.43:0/517864802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604433e2ce0 0x5604433f86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.845+0000 7f177dffb640 0 client.20286 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:48:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.845+0000 7f179f25c740 1 --2- 172.21.15.43:0/517864802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604433e2330 0x5604433f8170 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.845+0000 7f179f25c740 1 -- 172.21.15.43:0/517864802 >> 172.21.15.43:0/517864802 conn(0x5604433dcba0 msgr2=0x5604433dba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.845+0000 7f179f25c740 1 -- 172.21.15.43:0/517864802 shutdown_connections 2024-04-21T08:48:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.845+0000 7f179f25c740 1 -- 172.21.15.43:0/517864802 wait complete. 2024-04-21T08:48:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.845+0000 7f179f25c740 20 client.20286 trim_cache size 0 max 16384 2024-04-21T08:48:57.852 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:48:57.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.845+0000 7f179c670640 1 -- 172.21.15.43:0/3146416789 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f17800a3da0 msgr2=0x7f17800a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.845+0000 7f179c670640 1 --2- 172.21.15.43:0/3146416789 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f17800a3da0 0x7f17800a6260 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f178c005c40 tx=0x7f178c0094f0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.845+0000 7f179c670640 1 -- 172.21.15.43:0/3146416789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1794106170 msgr2=0x7f1794198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.845+0000 7f179c670640 1 --2- 172.21.15.43:0/3146416789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1794106170 0x7f1794198c50 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f178400e9c0 tx=0x7f178400ee80 comp rx=0 tx=0).stop 2024-04-21T08:48:57.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.845+0000 7f179c670640 1 -- 172.21.15.43:0/3146416789 shutdown_connections 2024-04-21T08:48:57.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.845+0000 7f179c670640 1 --2- 172.21.15.43:0/3146416789 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f17800a3da0 0x7f17800a6260 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.845+0000 7f179c670640 1 --2- 172.21.15.43:0/3146416789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f179419cfd0 0x7f179419d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.845+0000 7f179c670640 1 --2- 172.21.15.43:0/3146416789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1794106170 0x7f1794198c50 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.845+0000 7f179c670640 1 --2- 172.21.15.43:0/3146416789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17941010a0 0x7f1794198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.845+0000 7f179c670640 1 -- 172.21.15.43:0/3146416789 >> 172.21.15.43:0/3146416789 conn(0x7f1794077eb0 msgr2=0x7f17940ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.845+0000 7f179c670640 1 -- 172.21.15.43:0/3146416789 shutdown_connections 2024-04-21T08:48:57.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.845+0000 7f179c670640 1 -- 172.21.15.43:0/3146416789 wait complete. 2024-04-21T08:48:57.856 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:48:57.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.960+0000 7f74a5fd0640 1 -- 172.21.15.43:0/4019757729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a006b1f0 msgr2=0x7f74a0106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.960+0000 7f74a5fd0640 1 --2- 172.21.15.43:0/4019757729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a006b1f0 0x7f74a0106b90 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f7490009900 tx=0x7f749002bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:57.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.960+0000 7f74a5fd0640 1 -- 172.21.15.43:0/4019757729 shutdown_connections 2024-04-21T08:48:57.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.960+0000 7f74a5fd0640 1 --2- 172.21.15.43:0/4019757729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74a0107270 0x7f74a010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.960+0000 7f74a5fd0640 1 --2- 172.21.15.43:0/4019757729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a006b1f0 0x7f74a0106b90 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.960+0000 7f74a5fd0640 1 --2- 172.21.15.43:0/4019757729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74a006a840 0x7f74a006ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.960+0000 7f74a5fd0640 1 -- 172.21.15.43:0/4019757729 >> 172.21.15.43:0/4019757729 conn(0x7f74a0077eb0 msgr2=0x7f74a00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.960+0000 7f74a5fd0640 1 -- 172.21.15.43:0/4019757729 shutdown_connections 2024-04-21T08:48:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.960+0000 7f74a5fd0640 1 -- 172.21.15.43:0/4019757729 wait complete. 2024-04-21T08:48:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.961+0000 7f74a5fd0640 1 Processor -- start 2024-04-21T08:48:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.961+0000 7f74a5fd0640 1 -- start start 2024-04-21T08:48:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.961+0000 7f74a5fd0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74a006a840 0x7f74a019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.961+0000 7f74a5fd0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74a0107270 0x7f74a019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.961+0000 7f74a5fd0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a01985d0 0x7f74a0198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.961+0000 7f74a5fd0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f74a0112210 con 0x7f74a01985d0 2024-04-21T08:48:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.961+0000 7f74a5fd0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f74a0112090 con 0x7f74a006a840 2024-04-21T08:48:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.961+0000 7f74a5fd0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74a0112390 con 0x7f74a0107270 2024-04-21T08:48:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.961+0000 7f749ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a01985d0 0x7f74a0198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.961+0000 7f749ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a01985d0 0x7f74a0198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47600/0 (socket says 172.21.15.43:47600) 2024-04-21T08:48:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.961+0000 7f749ffff640 1 -- 172.21.15.43:0/834395217 learned_addr learned my addr 172.21.15.43:0/834395217 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.961+0000 7f749f7fe640 1 --2- 172.21.15.43:0/834395217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74a006a840 0x7f74a019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.962+0000 7f749ffff640 1 -- 172.21.15.43:0/834395217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74a0107270 msgr2=0x7f74a019e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.962+0000 7f749effd640 1 --2- 172.21.15.43:0/834395217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74a0107270 0x7f74a019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.962+0000 7f749ffff640 1 --2- 172.21.15.43:0/834395217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74a0107270 0x7f74a019e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.962+0000 7f749ffff640 1 -- 172.21.15.43:0/834395217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74a006a840 msgr2=0x7f74a019e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.962+0000 7f749ffff640 1 --2- 172.21.15.43:0/834395217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74a006a840 0x7f74a019e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.962+0000 7f749ffff640 1 -- 172.21.15.43:0/834395217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a01990f0 con 0x7f74a01985d0 2024-04-21T08:48:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.962+0000 7f749f7fe640 1 --2- 172.21.15.43:0/834395217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74a006a840 0x7f74a019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.962+0000 7f749ffff640 1 --2- 172.21.15.43:0/834395217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a01985d0 0x7f74a0198a80 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f749800cce0 tx=0x7f749800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.962+0000 7f749cff9640 1 -- 172.21.15.43:0/834395217 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749800f450 con 0x7f74a01985d0 2024-04-21T08:48:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.962+0000 7f749cff9640 1 -- 172.21.15.43:0/834395217 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f749800faa0 con 0x7f74a01985d0 2024-04-21T08:48:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.962+0000 7f74a5fd0640 1 -- 172.21.15.43:0/834395217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a01993e0 con 0x7f74a01985d0 2024-04-21T08:48:57.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.962+0000 7f749cff9640 1 -- 172.21.15.43:0/834395217 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498018b70 con 0x7f74a01985d0 2024-04-21T08:48:57.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.962+0000 7f74a5fd0640 1 -- 172.21.15.43:0/834395217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a01a55e0 con 0x7f74a01985d0 2024-04-21T08:48:57.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.964+0000 7f749cff9640 1 -- 172.21.15.43:0/834395217 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7498018d10 con 0x7f74a01985d0 2024-04-21T08:48:57.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.964+0000 7f74a8bbd740 1 Processor -- start 2024-04-21T08:48:57.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.965+0000 7f74a8bbd740 1 -- start start 2024-04-21T08:48:57.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.965+0000 7f74a8bbd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635330 0x55c820635710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.965+0000 7f74a8bbd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c820635ce0 0x55c82063e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.965+0000 7f74a8bbd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c82063ee40 0x55c82063f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.965+0000 7f74a8bbd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c82059a1f0 con 0x55c820635330 2024-04-21T08:48:57.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.965+0000 7f74a8bbd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c8205374b0 con 0x55c82063ee40 2024-04-21T08:48:57.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.965+0000 7f74a8bbd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c8205961b0 con 0x55c820635ce0 2024-04-21T08:48:57.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.965+0000 7f749cff9640 1 --2- 172.21.15.43:0/834395217 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f74880a3d20 0x7f74880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.965+0000 7f749effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635330 0x55c820635710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.965+0000 7f749effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635330 0x55c820635710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47614/0 (socket says 172.21.15.43:47614) 2024-04-21T08:48:57.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.965+0000 7f749effd640 1 -- 172.21.15.43:0/1168014423 learned_addr learned my addr 172.21.15.43:0/1168014423 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.965+0000 7f749ffff640 1 --2- 172.21.15.43:0/834395217 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f74880a3d20 0x7f74880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.965+0000 7f749f7fe640 1 --2- 172.21.15.43:0/1168014423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c820635ce0 0x55c82063e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.965+0000 7f749effd640 1 --2- 172.21.15.43:0/1168014423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c82063ee40 0x55c82063f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.965+0000 7f749cff9640 1 -- 172.21.15.43:0/834395217 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f74980d1120 con 0x7f74a01985d0 2024-04-21T08:48:57.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.965+0000 7f749effd640 1 -- 172.21.15.43:0/1168014423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c820635ce0 msgr2=0x55c82063e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.965+0000 7f749effd640 1 --2- 172.21.15.43:0/1168014423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c820635ce0 0x55c82063e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.965+0000 7f749effd640 1 -- 172.21.15.43:0/1168014423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c82063ee40 msgr2=0x55c82063f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.965+0000 7f749effd640 1 --2- 172.21.15.43:0/1168014423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c82063ee40 0x55c82063f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.965+0000 7f749effd640 1 -- 172.21.15.43:0/1168014423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8205291b0 con 0x55c820635330 2024-04-21T08:48:57.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.965+0000 7f749ffff640 1 --2- 172.21.15.43:0/834395217 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f74880a3d20 0x7f74880a61e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f74980ce500 tx=0x7f7498092130 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.965+0000 7f749effd640 1 --2- 172.21.15.43:0/1168014423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635330 0x55c820635710 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f7490005940 tx=0x7f7490005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8768f499598355aa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.966+0000 7f7485ffb640 1 -- 172.21.15.43:0/1168014423 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7490004450 con 0x55c820635330 2024-04-21T08:48:57.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.966+0000 7f7485ffb640 1 -- 172.21.15.43:0/1168014423 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f749002cc90 con 0x55c820635330 2024-04-21T08:48:57.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.966+0000 7f7485ffb640 1 -- 172.21.15.43:0/1168014423 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7490004900 con 0x55c820635330 2024-04-21T08:48:57.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.966+0000 7f74a8bbd740 1 -- 172.21.15.43:0/1168014423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635330 msgr2=0x55c820635710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.966+0000 7f74a8bbd740 1 --2- 172.21.15.43:0/1168014423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635330 0x55c820635710 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f7490005940 tx=0x7f7490005a20 comp rx=0 tx=0).stop 2024-04-21T08:48:57.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.966+0000 7f74a8bbd740 1 -- 172.21.15.43:0/1168014423 shutdown_connections 2024-04-21T08:48:57.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.966+0000 7f74a8bbd740 1 --2- 172.21.15.43:0/1168014423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c82063ee40 0x55c82063f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.966+0000 7f74a8bbd740 1 --2- 172.21.15.43:0/1168014423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c820635ce0 0x55c82063e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.966+0000 7f74a8bbd740 1 --2- 172.21.15.43:0/1168014423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635330 0x55c820635710 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.966+0000 7f74a8bbd740 1 -- 172.21.15.43:0/1168014423 >> 172.21.15.43:0/1168014423 conn(0x55c82062fba0 msgr2=0x55c82062ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.966+0000 7f74a8bbd740 1 -- 172.21.15.43:0/1168014423 shutdown_connections 2024-04-21T08:48:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.966+0000 7f74a8bbd740 1 -- 172.21.15.43:0/1168014423 wait complete. 2024-04-21T08:48:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.967+0000 7f74a8bbd740 1 -- start start 2024-04-21T08:48:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.967+0000 7f74a8bbd740 1 Processor -- start 2024-04-21T08:48:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.967+0000 7f74a8bbd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635330 0x55c8206452d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.967+0000 7f74a8bbd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c820635ce0 0x55c820642660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.967+0000 7f74a8bbd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82063ee40 0x55c820642c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.967+0000 7f74a8bbd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c820645d60 con 0x55c820635330 2024-04-21T08:48:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.967+0000 7f74a8bbd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c820553500 con 0x55c820635ce0 2024-04-21T08:48:57.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.967+0000 7f74a8bbd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c820645ee0 con 0x55c82063ee40 2024-04-21T08:48:57.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.967+0000 7f74a8bbd740 10 client.0 _ll_register_callbacks cb 0x55c820521b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:57.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.967+0000 7f749effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635330 0x55c8206452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.967+0000 7f749effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635330 0x55c8206452d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47630/0 (socket says 172.21.15.43:47630) 2024-04-21T08:48:57.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.967+0000 7f749effd640 1 -- 172.21.15.43:0/4097721327 learned_addr learned my addr 172.21.15.43:0/4097721327 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.968+0000 7f749f7fe640 1 --2- 172.21.15.43:0/4097721327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c820635ce0 0x55c820642660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.968+0000 7f749effd640 1 --2- 172.21.15.43:0/4097721327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82063ee40 0x55c820642c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.968+0000 7f749effd640 1 -- 172.21.15.43:0/4097721327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82063ee40 msgr2=0x55c820642c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.968+0000 7f749effd640 1 --2- 172.21.15.43:0/4097721327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82063ee40 0x55c820642c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.968+0000 7f749effd640 1 -- 172.21.15.43:0/4097721327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c820635ce0 msgr2=0x55c820642660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.968+0000 7f749effd640 1 --2- 172.21.15.43:0/4097721327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c820635ce0 0x55c820642660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.968+0000 7f749effd640 1 -- 172.21.15.43:0/4097721327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8202ac3f0 con 0x55c820635330 2024-04-21T08:48:57.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.968+0000 7f749f7fe640 1 --2- 172.21.15.43:0/4097721327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c820635ce0 0x55c820642660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:57.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.968+0000 7f749effd640 1 --2- 172.21.15.43:0/4097721327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635330 0x55c8206452d0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f7490036040 tx=0x7f7490005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a784720335fe4a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.968+0000 7f747effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:57.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.968+0000 7f747effd640 1 -- 172.21.15.43:0/4097721327 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7490056060 con 0x55c820635330 2024-04-21T08:48:57.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.968+0000 7f747effd640 1 -- 172.21.15.43:0/4097721327 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f7490003800 con 0x55c820635330 2024-04-21T08:48:57.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.968+0000 7f74a8bbd740 1 -- 172.21.15.43:0/4097721327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f749002cc90 con 0x55c820635330 2024-04-21T08:48:57.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.968+0000 7f747effd640 1 -- 172.21.15.43:0/4097721327 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f749005d050 con 0x55c820635330 2024-04-21T08:48:57.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.968+0000 7f74a8bbd740 10 client.20298 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:57.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.968+0000 7f74a8bbd740 1 -- 172.21.15.43:0/4097721327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c8203767d0 con 0x55c820635330 2024-04-21T08:48:57.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.968+0000 7f747effd640 1 -- 172.21.15.43:0/4097721327 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f749002cc90 con 0x55c820635330 2024-04-21T08:48:57.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.969+0000 7f747effd640 1 -- 172.21.15.43:0/4097721327 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f749004d070 con 0x55c820635330 2024-04-21T08:48:57.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.969+0000 7f74a8bbd740 10 client.20298 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:57.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.969+0000 7f74a8bbd740 10 client.20298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:57.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.969+0000 7f74a8bbd740 20 client.20298 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:57.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.969+0000 7f74a8bbd740 1 --2- 172.21.15.43:0/4097721327 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c820652f10 0x55c820655300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.969+0000 7f74a8bbd740 4 client.20298 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.969+0000 7f74a8bbd740 1 -- 172.21.15.43:0/4097721327 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c820539930 con 0x55c820652f10 2024-04-21T08:48:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.969+0000 7f749f7fe640 1 --2- 172.21.15.43:0/4097721327 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c820652f10 0x55c820655300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.969+0000 7f747effd640 10 client.20298 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.969+0000 7f749f7fe640 1 --2- 172.21.15.43:0/4097721327 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c820652f10 0x55c820655300 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e7f166259334e18 server_cookie=e6578e9de34d906f in_seq=0 out_seq=0 2024-04-21T08:48:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.970+0000 7f747effd640 1 -- 172.21.15.43:0/4097721327 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c820539930 con 0x55c820652f10 2024-04-21T08:48:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.970+0000 7f747effd640 10 client.20298 handle_command_reply: tid=0 2024-04-21T08:48:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.970+0000 7f74a8bbd740 1 client.20298 shutdown 2024-04-21T08:48:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.970+0000 7f74a8bbd740 1 -- 172.21.15.43:0/4097721327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635330 msgr2=0x55c8206452d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.970+0000 7f74a8bbd740 1 --2- 172.21.15.43:0/4097721327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635330 0x55c8206452d0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f7490036040 tx=0x7f7490005690 comp rx=0 tx=0).stop 2024-04-21T08:48:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.970+0000 7f74a8bbd740 1 -- 172.21.15.43:0/4097721327 shutdown_connections 2024-04-21T08:48:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.970+0000 7f74a8bbd740 1 --2- 172.21.15.43:0/4097721327 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c820652f10 0x55c820655300 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.970+0000 7f74a8bbd740 1 --2- 172.21.15.43:0/4097721327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82063ee40 0x55c820642c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.970+0000 7f747effd640 0 client.20298 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.970+0000 7f74a8bbd740 1 --2- 172.21.15.43:0/4097721327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c820635ce0 0x55c820642660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.970+0000 7f74a8bbd740 1 --2- 172.21.15.43:0/4097721327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635330 0x55c8206452d0 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.970+0000 7f74a8bbd740 1 -- 172.21.15.43:0/4097721327 >> 172.21.15.43:0/4097721327 conn(0x55c82062fba0 msgr2=0x55c8206318f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.971+0000 7f74a8bbd740 1 -- 172.21.15.43:0/4097721327 shutdown_connections 2024-04-21T08:48:57.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.971+0000 7f74a8bbd740 1 -- 172.21.15.43:0/4097721327 wait complete. 2024-04-21T08:48:57.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.971+0000 7f74a8bbd740 20 client.20298 trim_cache size 0 max 16384 2024-04-21T08:48:57.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.984+0000 7f74a8bbd740 1 Processor -- start 2024-04-21T08:48:57.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.984+0000 7f74a8bbd740 1 -- start start 2024-04-21T08:48:57.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.984+0000 7f74a8bbd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c820635330 0x55c820640230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.984+0000 7f74a8bbd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635ce0 0x55c820644d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.984+0000 7f74a8bbd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82063ee40 0x55c8206452d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.984+0000 7f74a8bbd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f74a01ad1a0 con 0x55c820635ce0 2024-04-21T08:48:57.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.984+0000 7f74a8bbd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f74a01aece0 con 0x55c820635330 2024-04-21T08:48:57.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.984+0000 7f74a8bbd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74a01ad870 con 0x55c82063ee40 2024-04-21T08:48:57.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.984+0000 7f749f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635ce0 0x55c820644d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.984+0000 7f749f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635ce0 0x55c820644d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47638/0 (socket says 172.21.15.43:47638) 2024-04-21T08:48:57.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.984+0000 7f749f7fe640 1 -- 172.21.15.43:0/451569631 learned_addr learned my addr 172.21.15.43:0/451569631 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.985+0000 7f749effd640 1 --2- 172.21.15.43:0/451569631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82063ee40 0x55c8206452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.985+0000 7f749f7fe640 1 -- 172.21.15.43:0/451569631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82063ee40 msgr2=0x55c8206452d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.985+0000 7f749f7fe640 1 --2- 172.21.15.43:0/451569631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82063ee40 0x55c8206452d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.985+0000 7f749f7fe640 1 -- 172.21.15.43:0/451569631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c820635330 msgr2=0x55c820640230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.985+0000 7f749f7fe640 1 --2- 172.21.15.43:0/451569631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c820635330 0x55c820640230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.985+0000 7f749f7fe640 1 -- 172.21.15.43:0/451569631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c820640bb0 con 0x55c820635ce0 2024-04-21T08:48:57.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.985+0000 7f749effd640 1 --2- 172.21.15.43:0/451569631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82063ee40 0x55c8206452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:57.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.985+0000 7f749f7fe640 1 --2- 172.21.15.43:0/451569631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635ce0 0x55c820644d90 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x55c820641ec0 tx=0x7f7494008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d74a572b4620c1c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.985+0000 7f74857fa640 1 -- 172.21.15.43:0/451569631 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7494020070 con 0x55c820635ce0 2024-04-21T08:48:57.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.985+0000 7f74857fa640 1 -- 172.21.15.43:0/451569631 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7494015d00 con 0x55c820635ce0 2024-04-21T08:48:57.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.985+0000 7f74857fa640 1 -- 172.21.15.43:0/451569631 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74940143b0 con 0x55c820635ce0 2024-04-21T08:48:57.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.985+0000 7f74a8bbd740 1 -- 172.21.15.43:0/451569631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635ce0 msgr2=0x55c820644d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.985+0000 7f74a8bbd740 1 --2- 172.21.15.43:0/451569631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635ce0 0x55c820644d90 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x55c820641ec0 tx=0x7f7494008040 comp rx=0 tx=0).stop 2024-04-21T08:48:57.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.986+0000 7f74a8bbd740 1 -- 172.21.15.43:0/451569631 shutdown_connections 2024-04-21T08:48:57.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.986+0000 7f74a8bbd740 1 --2- 172.21.15.43:0/451569631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82063ee40 0x55c8206452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.986+0000 7f74a8bbd740 1 --2- 172.21.15.43:0/451569631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635ce0 0x55c820644d90 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.986+0000 7f74a8bbd740 1 --2- 172.21.15.43:0/451569631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c820635330 0x55c820640230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.986+0000 7f74a8bbd740 1 -- 172.21.15.43:0/451569631 >> 172.21.15.43:0/451569631 conn(0x55c82062fba0 msgr2=0x55c82064ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.986+0000 7f74a8bbd740 1 -- 172.21.15.43:0/451569631 shutdown_connections 2024-04-21T08:48:57.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.986+0000 7f74a8bbd740 1 -- 172.21.15.43:0/451569631 wait complete. 2024-04-21T08:48:57.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.986+0000 7f74a8bbd740 1 -- start start 2024-04-21T08:48:57.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.986+0000 7f74a8bbd740 1 Processor -- start 2024-04-21T08:48:57.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.986+0000 7f74a8bbd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635330 0x55c820656570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.986+0000 7f74a8bbd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c820635ce0 0x55c820656ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.986+0000 7f74a8bbd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82063ee40 0x55c8206b80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.986+0000 7f74a8bbd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f74a01ae610 con 0x55c820635330 2024-04-21T08:48:57.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.986+0000 7f74a8bbd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f74a01adf40 con 0x55c820635ce0 2024-04-21T08:48:57.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.986+0000 7f74a8bbd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74a01ab1a0 con 0x55c82063ee40 2024-04-21T08:48:57.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.986+0000 7f74a8bbd740 10 client.0 _ll_register_callbacks cb 0x7f749002c7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:57.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.986+0000 7f749effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635330 0x55c820656570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.986+0000 7f749effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635330 0x55c820656570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47654/0 (socket says 172.21.15.43:47654) 2024-04-21T08:48:57.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.986+0000 7f749effd640 1 -- 172.21.15.43:0/1508058413 learned_addr learned my addr 172.21.15.43:0/1508058413 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:57.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.987+0000 7f749f7fe640 1 --2- 172.21.15.43:0/1508058413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c820635ce0 0x55c820656ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.987+0000 7f749effd640 1 --2- 172.21.15.43:0/1508058413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82063ee40 0x55c8206b80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.987+0000 7f749effd640 1 -- 172.21.15.43:0/1508058413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82063ee40 msgr2=0x55c8206b80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.987+0000 7f749effd640 1 --2- 172.21.15.43:0/1508058413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82063ee40 0x55c8206b80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.987+0000 7f749effd640 1 -- 172.21.15.43:0/1508058413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c820635ce0 msgr2=0x55c820656ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:57.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.987+0000 7f749effd640 1 --2- 172.21.15.43:0/1508058413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c820635ce0 0x55c820656ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.987+0000 7f749effd640 1 -- 172.21.15.43:0/1508058413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8206b8700 con 0x55c820635330 2024-04-21T08:48:57.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.987+0000 7f749effd640 1 --2- 172.21.15.43:0/1508058413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635330 0x55c820656570 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f7490034460 tx=0x7f7490005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9e447782c9b285 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:57.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.987+0000 7f747effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:57.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.987+0000 7f747effd640 1 -- 172.21.15.43:0/1508058413 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7490067060 con 0x55c820635330 2024-04-21T08:48:57.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.987+0000 7f747effd640 1 -- 172.21.15.43:0/1508058413 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f749002cac0 con 0x55c820635330 2024-04-21T08:48:57.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.987+0000 7f74a8bbd740 1 -- 172.21.15.43:0/1508058413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7494015d00 con 0x55c820635330 2024-04-21T08:48:57.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.987+0000 7f747effd640 1 -- 172.21.15.43:0/1508058413 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f749006e050 con 0x55c820635330 2024-04-21T08:48:57.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.987+0000 7f74a8bbd740 10 client.20304 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:57.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.987+0000 7f74a8bbd740 1 -- 172.21.15.43:0/1508058413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c8206b9970 con 0x55c820635330 2024-04-21T08:48:57.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.988+0000 7f747effd640 1 -- 172.21.15.43:0/1508058413 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7494015d00 con 0x55c820635330 2024-04-21T08:48:57.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.988+0000 7f747effd640 1 -- 172.21.15.43:0/1508058413 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f749004d040 con 0x55c820635330 2024-04-21T08:48:57.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.988+0000 7f74a8bbd740 10 client.20304 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:57.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.988+0000 7f74a8bbd740 10 client.20304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:57.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.988+0000 7f74a8bbd740 20 client.20304 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:57.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.988+0000 7f74a8bbd740 1 --2- 172.21.15.43:0/1508058413 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c8206b9fd0 0x55c8206bc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:57.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.988+0000 7f74a8bbd740 4 client.20304 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:57.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.988+0000 7f74a8bbd740 1 -- 172.21.15.43:0/1508058413 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c8205a7c10 con 0x55c8206b9fd0 2024-04-21T08:48:57.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.988+0000 7f749f7fe640 1 --2- 172.21.15.43:0/1508058413 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c8206b9fd0 0x55c8206bc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:57.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.988+0000 7f749f7fe640 1 --2- 172.21.15.43:0/1508058413 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c8206b9fd0 0x55c8206bc3c0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=212db53cfaf4fe51 server_cookie=6357ef4a36dfb789 in_seq=0 out_seq=0 2024-04-21T08:48:57.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.988+0000 7f747effd640 10 client.20304 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:57.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f747effd640 1 -- 172.21.15.43:0/1508058413 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c8205a7c10 con 0x55c8206b9fd0 2024-04-21T08:48:57.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f747effd640 10 client.20304 handle_command_reply: tid=0 2024-04-21T08:48:57.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f74a8bbd740 1 client.20304 shutdown 2024-04-21T08:48:57.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f74a8bbd740 1 -- 172.21.15.43:0/1508058413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635330 msgr2=0x55c820656570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f74a8bbd740 1 --2- 172.21.15.43:0/1508058413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635330 0x55c820656570 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f7490034460 tx=0x7f7490005fd0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f74a8bbd740 1 -- 172.21.15.43:0/1508058413 shutdown_connections 2024-04-21T08:48:57.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f74a8bbd740 1 --2- 172.21.15.43:0/1508058413 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55c8206b9fd0 0x55c8206bc3c0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f74a8bbd740 1 --2- 172.21.15.43:0/1508058413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c82063ee40 0x55c8206b80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f74a8bbd740 1 --2- 172.21.15.43:0/1508058413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c820635ce0 0x55c820656ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f747effd640 0 client.20304 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:57.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f74a8bbd740 1 --2- 172.21.15.43:0/1508058413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c820635330 0x55c820656570 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f74a8bbd740 1 -- 172.21.15.43:0/1508058413 >> 172.21.15.43:0/1508058413 conn(0x55c82062fba0 msgr2=0x55c82062e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f74a8bbd740 1 -- 172.21.15.43:0/1508058413 shutdown_connections 2024-04-21T08:48:57.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f74a8bbd740 1 -- 172.21.15.43:0/1508058413 wait complete. 2024-04-21T08:48:57.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f74a8bbd740 20 client.20304 trim_cache size 0 max 16384 2024-04-21T08:48:57.995 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:48:57.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f74a5fd0640 1 -- 172.21.15.43:0/834395217 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f74880a3d20 msgr2=0x7f74880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f74a5fd0640 1 --2- 172.21.15.43:0/834395217 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f74880a3d20 0x7f74880a61e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f74980ce500 tx=0x7f7498092130 comp rx=0 tx=0).stop 2024-04-21T08:48:57.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f74a5fd0640 1 -- 172.21.15.43:0/834395217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a01985d0 msgr2=0x7f74a0198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f74a5fd0640 1 --2- 172.21.15.43:0/834395217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a01985d0 0x7f74a0198a80 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f749800cce0 tx=0x7f749800a620 comp rx=0 tx=0).stop 2024-04-21T08:48:57.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f74a5fd0640 1 -- 172.21.15.43:0/834395217 shutdown_connections 2024-04-21T08:48:57.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f74a5fd0640 1 --2- 172.21.15.43:0/834395217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a01985d0 0x7f74a0198a80 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f74a5fd0640 1 --2- 172.21.15.43:0/834395217 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f74880a3d20 0x7f74880a61e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f74a5fd0640 1 --2- 172.21.15.43:0/834395217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74a0107270 0x7f74a019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f74a5fd0640 1 --2- 172.21.15.43:0/834395217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74a006a840 0x7f74a019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.989+0000 7f74a5fd0640 1 -- 172.21.15.43:0/834395217 >> 172.21.15.43:0/834395217 conn(0x7f74a0077eb0 msgr2=0x7f74a0100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.990+0000 7f74a5fd0640 1 -- 172.21.15.43:0/834395217 shutdown_connections 2024-04-21T08:48:57.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:57.990+0000 7f74a5fd0640 1 -- 172.21.15.43:0/834395217 wait complete. 2024-04-21T08:48:58.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:58 smithi175 ceph-mon[31920]: pgmap v311: 129 pgs: 129 active+clean; 4.6 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 67 KiB/s wr, 24 op/s 2024-04-21T08:48:58.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:48:58 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3668966439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:58.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:58 smithi112 ceph-mon[31085]: pgmap v311: 129 pgs: 129 active+clean; 4.6 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 67 KiB/s wr, 24 op/s 2024-04-21T08:48:58.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:48:58 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3668966439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:58.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:58 smithi043 ceph-mon[27978]: pgmap v311: 129 pgs: 129 active+clean; 4.6 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 67 KiB/s wr, 24 op/s 2024-04-21T08:48:58.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:48:58 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3668966439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:59.000 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:48:59.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:48:59.001 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:48:59.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.120+0000 7ff2aad85640 1 -- 172.21.15.43:0/180191870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2a4102190 msgr2=0x7ff2a4102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.120+0000 7ff2aad85640 1 --2- 172.21.15.43:0/180191870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2a4102190 0x7ff2a4102570 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009900 tx=0x7ff2a002bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.120+0000 7ff2aad85640 1 -- 172.21.15.43:0/180191870 shutdown_connections 2024-04-21T08:48:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.120+0000 7ff2aad85640 1 --2- 172.21.15.43:0/180191870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a4107260 0x7ff2a410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.120+0000 7ff2aad85640 1 --2- 172.21.15.43:0/180191870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2a4102b40 0x7ff2a4106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.120+0000 7ff2aad85640 1 --2- 172.21.15.43:0/180191870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2a4102190 0x7ff2a4102570 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.120+0000 7ff2aad85640 1 -- 172.21.15.43:0/180191870 >> 172.21.15.43:0/180191870 conn(0x7ff2a40fd990 msgr2=0x7ff2a40ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.121+0000 7ff2aad85640 1 -- 172.21.15.43:0/180191870 shutdown_connections 2024-04-21T08:48:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.121+0000 7ff2aad85640 1 -- 172.21.15.43:0/180191870 wait complete. 2024-04-21T08:48:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.121+0000 7ff2aad85640 1 Processor -- start 2024-04-21T08:48:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.121+0000 7ff2aad85640 1 -- start start 2024-04-21T08:48:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.121+0000 7ff2aad85640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a4102190 0x7ff2a4198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.121+0000 7ff2aad85640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2a4107260 0x7ff2a4198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.121+0000 7ff2aad85640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2a419cfe0 0x7ff2a419d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.121+0000 7ff2aad85640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a41121a0 con 0x7ff2a4107260 2024-04-21T08:48:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.121+0000 7ff2aad85640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a4112020 con 0x7ff2a419cfe0 2024-04-21T08:48:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.121+0000 7ff2aad85640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a4112320 con 0x7ff2a4102190 2024-04-21T08:48:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.122+0000 7ff29bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2a4107260 0x7ff2a4198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.122+0000 7ff29bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2a4107260 0x7ff2a4198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47678/0 (socket says 172.21.15.43:47678) 2024-04-21T08:48:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.122+0000 7ff29bfff640 1 -- 172.21.15.43:0/1024489079 learned_addr learned my addr 172.21.15.43:0/1024489079 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.122+0000 7ff2a9322640 1 --2- 172.21.15.43:0/1024489079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2a419cfe0 0x7ff2a419d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.122+0000 7ff2a8b21640 1 --2- 172.21.15.43:0/1024489079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a4102190 0x7ff2a4198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.122+0000 7ff29bfff640 1 -- 172.21.15.43:0/1024489079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a4102190 msgr2=0x7ff2a4198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.122+0000 7ff29bfff640 1 --2- 172.21.15.43:0/1024489079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a4102190 0x7ff2a4198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.122+0000 7ff29bfff640 1 -- 172.21.15.43:0/1024489079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2a419cfe0 msgr2=0x7ff2a419d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.122+0000 7ff29bfff640 1 --2- 172.21.15.43:0/1024489079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2a419cfe0 0x7ff2a419d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.122+0000 7ff29bfff640 1 -- 172.21.15.43:0/1024489079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a419dad0 con 0x7ff2a4107260 2024-04-21T08:48:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.122+0000 7ff2a9322640 1 --2- 172.21.15.43:0/1024489079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2a419cfe0 0x7ff2a419d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:59.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.122+0000 7ff29bfff640 1 --2- 172.21.15.43:0/1024489079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2a4107260 0x7ff2a4198c60 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7ff29400d770 tx=0x7ff29400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.122+0000 7ff299ffb640 1 -- 172.21.15.43:0/1024489079 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294017070 con 0x7ff2a4107260 2024-04-21T08:48:59.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.122+0000 7ff299ffb640 1 -- 172.21.15.43:0/1024489079 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff294009c30 con 0x7ff2a4107260 2024-04-21T08:48:59.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.122+0000 7ff2aad85640 1 -- 172.21.15.43:0/1024489079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a419ddc0 con 0x7ff2a4107260 2024-04-21T08:48:59.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.122+0000 7ff299ffb640 1 -- 172.21.15.43:0/1024489079 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29400f620 con 0x7ff2a4107260 2024-04-21T08:48:59.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.123+0000 7ff2aad85640 1 -- 172.21.15.43:0/1024489079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a41a5600 con 0x7ff2a4107260 2024-04-21T08:48:59.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.125+0000 7ff299ffb640 1 -- 172.21.15.43:0/1024489079 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff29401a070 con 0x7ff2a4107260 2024-04-21T08:48:59.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.125+0000 7ff2ad975740 1 Processor -- start 2024-04-21T08:48:59.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.125+0000 7ff2ad975740 1 -- start start 2024-04-21T08:48:59.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.125+0000 7ff2ad975740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611f61bd8e0 0x5611f61bdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.125+0000 7ff2ad975740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611f61be290 0x5611f61c6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.125+0000 7ff2ad975740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61c6e10 0x5611f61c9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.125+0000 7ff2ad975740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5611f6121f00 con 0x5611f61c6e10 2024-04-21T08:48:59.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.126+0000 7ff2ad975740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5611f60bf590 con 0x5611f61bd8e0 2024-04-21T08:48:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.126+0000 7ff2ad975740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5611f611dec0 con 0x5611f61be290 2024-04-21T08:48:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.126+0000 7ff299ffb640 1 --2- 172.21.15.43:0/1024489079 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff28c0a3da0 0x7ff28c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.126+0000 7ff29bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61c6e10 0x5611f61c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.126+0000 7ff2a8b21640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611f61be290 0x5611f61c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.126+0000 7ff29bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61c6e10 0x5611f61c9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47680/0 (socket says 172.21.15.43:47680) 2024-04-21T08:48:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.126+0000 7ff29bfff640 1 -- 172.21.15.43:0/758181205 learned_addr learned my addr 172.21.15.43:0/758181205 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.126+0000 7ff2a9322640 1 --2- 172.21.15.43:0/758181205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611f61bd8e0 0x5611f61bdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.126+0000 7ff2a9322640 1 --2- 172.21.15.43:0/1024489079 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff28c0a3da0 0x7ff28c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.126+0000 7ff299ffb640 1 -- 172.21.15.43:0/1024489079 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7ff2940d0770 con 0x7ff2a4107260 2024-04-21T08:48:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.126+0000 7ff2a8b21640 1 -- 172.21.15.43:0/758181205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611f61bd8e0 msgr2=0x5611f61bdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.126+0000 7ff2a8b21640 1 --2- 172.21.15.43:0/758181205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611f61bd8e0 0x5611f61bdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.126+0000 7ff2a8b21640 1 -- 172.21.15.43:0/758181205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61c6e10 msgr2=0x5611f61c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.126+0000 7ff2a8b21640 1 --2- 172.21.15.43:0/758181205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61c6e10 0x5611f61c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.126+0000 7ff2a8b21640 1 -- 172.21.15.43:0/758181205 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611f5efc7e0 con 0x5611f61be290 2024-04-21T08:48:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.126+0000 7ff2a8b21640 1 --2- 172.21.15.43:0/758181205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611f61be290 0x5611f61c6840 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0000c00 tx=0x7ff2a0004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88391ccc1a410e3e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.126+0000 7ff2a9322640 1 --2- 172.21.15.43:0/1024489079 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff28c0a3da0 0x7ff28c0a6260 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff29c007310 tx=0x7ff29c0072a0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.127+0000 7ff28affd640 1 -- 172.21.15.43:0/758181205 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0045020 con 0x5611f61be290 2024-04-21T08:48:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.127+0000 7ff28affd640 1 -- 172.21.15.43:0/758181205 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2a002cc90 con 0x5611f61be290 2024-04-21T08:48:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.127+0000 7ff28affd640 1 -- 172.21.15.43:0/758181205 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00046c0 con 0x5611f61be290 2024-04-21T08:48:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.127+0000 7ff2ad975740 1 -- 172.21.15.43:0/758181205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611f61be290 msgr2=0x5611f61c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.127+0000 7ff2ad975740 1 --2- 172.21.15.43:0/758181205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611f61be290 0x5611f61c6840 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0000c00 tx=0x7ff2a0004040 comp rx=0 tx=0).stop 2024-04-21T08:48:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.127+0000 7ff2ad975740 1 -- 172.21.15.43:0/758181205 shutdown_connections 2024-04-21T08:48:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.127+0000 7ff2ad975740 1 --2- 172.21.15.43:0/758181205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61c6e10 0x5611f61c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.127+0000 7ff2ad975740 1 --2- 172.21.15.43:0/758181205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611f61be290 0x5611f61c6840 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.127+0000 7ff2ad975740 1 --2- 172.21.15.43:0/758181205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611f61bd8e0 0x5611f61bdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.127+0000 7ff2ad975740 1 -- 172.21.15.43:0/758181205 >> 172.21.15.43:0/758181205 conn(0x5611f61b7fb0 msgr2=0x5611f61b8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.127+0000 7ff2ad975740 1 -- 172.21.15.43:0/758181205 shutdown_connections 2024-04-21T08:48:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.127+0000 7ff2ad975740 1 -- 172.21.15.43:0/758181205 wait complete. 2024-04-21T08:48:59.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.128+0000 7ff2ad975740 1 -- start start 2024-04-21T08:48:59.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.128+0000 7ff2ad975740 1 Processor -- start 2024-04-21T08:48:59.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.128+0000 7ff2ad975740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61bd8e0 0x5611f61c3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.128+0000 7ff2ad975740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611f61be290 0x5611f61c4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.128+0000 7ff2ad975740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611f61c6e10 0x5611f61c4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.128+0000 7ff2ad975740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5611f61c9d20 con 0x5611f61bd8e0 2024-04-21T08:48:59.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.128+0000 7ff2ad975740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5611f60db580 con 0x5611f61be290 2024-04-21T08:48:59.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.128+0000 7ff2ad975740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5611f61c9ea0 con 0x5611f61c6e10 2024-04-21T08:48:59.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.128+0000 7ff2ad975740 10 client.0 _ll_register_callbacks cb 0x5611f60e1650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:59.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.128+0000 7ff2a8b21640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61bd8e0 0x5611f61c3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.128+0000 7ff2a8b21640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61bd8e0 0x5611f61c3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47694/0 (socket says 172.21.15.43:47694) 2024-04-21T08:48:59.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.128+0000 7ff2a8b21640 1 -- 172.21.15.43:0/231432232 learned_addr learned my addr 172.21.15.43:0/231432232 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:59.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.129+0000 7ff2a9322640 1 --2- 172.21.15.43:0/231432232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611f61c6e10 0x5611f61c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.129+0000 7ff29bfff640 1 --2- 172.21.15.43:0/231432232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611f61be290 0x5611f61c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.129+0000 7ff2a8b21640 1 -- 172.21.15.43:0/231432232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611f61c6e10 msgr2=0x5611f61c4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.129+0000 7ff2a8b21640 1 --2- 172.21.15.43:0/231432232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611f61c6e10 0x5611f61c4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.129+0000 7ff2a8b21640 1 -- 172.21.15.43:0/231432232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611f61be290 msgr2=0x5611f61c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.129+0000 7ff2a8b21640 1 --2- 172.21.15.43:0/231432232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611f61be290 0x5611f61c4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.129+0000 7ff2a8b21640 1 -- 172.21.15.43:0/231432232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611f5f5cf70 con 0x5611f61bd8e0 2024-04-21T08:48:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.129+0000 7ff29bfff640 1 --2- 172.21.15.43:0/231432232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611f61be290 0x5611f61c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.129+0000 7ff2a8b21640 1 --2- 172.21.15.43:0/231432232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61bd8e0 0x5611f61c3e10 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7ff2a002c170 tx=0x7ff2a002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd7fee673e303b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.129+0000 7ff283fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.129+0000 7ff283fff640 1 -- 172.21.15.43:0/231432232 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0045070 con 0x5611f61bd8e0 2024-04-21T08:48:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.129+0000 7ff283fff640 1 -- 172.21.15.43:0/231432232 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7ff2a00037f0 con 0x5611f61bd8e0 2024-04-21T08:48:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.129+0000 7ff2ad975740 1 -- 172.21.15.43:0/231432232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2a002cc90 con 0x5611f61bd8e0 2024-04-21T08:48:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.129+0000 7ff283fff640 1 -- 172.21.15.43:0/231432232 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff2a0059050 con 0x5611f61bd8e0 2024-04-21T08:48:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.129+0000 7ff2ad975740 10 client.20316 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.129+0000 7ff2ad975740 1 -- 172.21.15.43:0/231432232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5611f5df4170 con 0x5611f61bd8e0 2024-04-21T08:48:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.129+0000 7ff283fff640 1 -- 172.21.15.43:0/231432232 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2a002cc90 con 0x5611f61bd8e0 2024-04-21T08:48:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.130+0000 7ff283fff640 1 -- 172.21.15.43:0/231432232 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff2a0041b00 con 0x5611f61bd8e0 2024-04-21T08:48:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.130+0000 7ff2ad975740 10 client.20316 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.130+0000 7ff2ad975740 10 client.20316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.130+0000 7ff2ad975740 20 client.20316 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.130+0000 7ff2ad975740 1 --2- 172.21.15.43:0/231432232 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5611f61d71a0 0x5611f61d9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.130+0000 7ff2ad975740 4 client.20316 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.130+0000 7ff2ad975740 1 -- 172.21.15.43:0/231432232 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611f612f920 con 0x5611f61d71a0 2024-04-21T08:48:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.130+0000 7ff29bfff640 1 --2- 172.21.15.43:0/231432232 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5611f61d71a0 0x5611f61d9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.130+0000 7ff283fff640 10 client.20316 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.130+0000 7ff29bfff640 1 --2- 172.21.15.43:0/231432232 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5611f61d71a0 0x5611f61d9590 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f579303778b82479 server_cookie=55113f2e6a526ccc in_seq=0 out_seq=0 2024-04-21T08:48:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.131+0000 7ff283fff640 1 -- 172.21.15.43:0/231432232 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5611f612f920 con 0x5611f61d71a0 2024-04-21T08:48:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.131+0000 7ff283fff640 10 client.20316 handle_command_reply: tid=0 2024-04-21T08:48:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.131+0000 7ff2ad975740 1 client.20316 shutdown 2024-04-21T08:48:59.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.131+0000 7ff2ad975740 1 -- 172.21.15.43:0/231432232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61bd8e0 msgr2=0x5611f61c3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.131+0000 7ff2ad975740 1 --2- 172.21.15.43:0/231432232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61bd8e0 0x5611f61c3e10 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7ff2a002c170 tx=0x7ff2a002bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:59.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.131+0000 7ff2ad975740 1 -- 172.21.15.43:0/231432232 shutdown_connections 2024-04-21T08:48:59.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.131+0000 7ff2ad975740 1 --2- 172.21.15.43:0/231432232 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5611f61d71a0 0x5611f61d9590 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.131+0000 7ff2ad975740 1 --2- 172.21.15.43:0/231432232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611f61c6e10 0x5611f61c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.131+0000 7ff2ad975740 1 --2- 172.21.15.43:0/231432232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611f61be290 0x5611f61c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.131+0000 7ff2ad975740 1 --2- 172.21.15.43:0/231432232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61bd8e0 0x5611f61c3e10 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.131+0000 7ff283fff640 0 client.20316 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:59.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.131+0000 7ff2ad975740 1 -- 172.21.15.43:0/231432232 >> 172.21.15.43:0/231432232 conn(0x5611f61b7fb0 msgr2=0x5611f61b9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.132+0000 7ff2ad975740 1 -- 172.21.15.43:0/231432232 shutdown_connections 2024-04-21T08:48:59.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.132+0000 7ff2ad975740 1 -- 172.21.15.43:0/231432232 wait complete. 2024-04-21T08:48:59.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.132+0000 7ff2ad975740 20 client.20316 trim_cache size 0 max 16384 2024-04-21T08:48:59.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.150+0000 7ff2ad975740 1 Processor -- start 2024-04-21T08:48:59.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.150+0000 7ff2ad975740 1 -- start start 2024-04-21T08:48:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.150+0000 7ff2ad975740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611f61bd8e0 0x5611f61c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.150+0000 7ff2ad975740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611f61be290 0x5611f61d2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.150+0000 7ff2ad975740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61c6e10 0x5611f61b9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.150+0000 7ff2ad975740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a41af400 con 0x5611f61c6e10 2024-04-21T08:48:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.150+0000 7ff2ad975740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a41b0f40 con 0x5611f61be290 2024-04-21T08:48:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.150+0000 7ff2ad975740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a41afad0 con 0x5611f61bd8e0 2024-04-21T08:48:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.150+0000 7ff2a9322640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61c6e10 0x5611f61b9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.150+0000 7ff2a9322640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61c6e10 0x5611f61b9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47698/0 (socket says 172.21.15.43:47698) 2024-04-21T08:48:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.150+0000 7ff2a9322640 1 -- 172.21.15.43:0/3587634414 learned_addr learned my addr 172.21.15.43:0/3587634414 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.150+0000 7ff29bfff640 1 --2- 172.21.15.43:0/3587634414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611f61be290 0x5611f61d2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.150+0000 7ff2a8b21640 1 --2- 172.21.15.43:0/3587634414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611f61bd8e0 0x5611f61c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.150+0000 7ff2a9322640 1 -- 172.21.15.43:0/3587634414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611f61bd8e0 msgr2=0x5611f61c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.150+0000 7ff2a9322640 1 --2- 172.21.15.43:0/3587634414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611f61bd8e0 0x5611f61c6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.150+0000 7ff2a9322640 1 -- 172.21.15.43:0/3587634414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611f61be290 msgr2=0x5611f61d2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.150+0000 7ff2a9322640 1 --2- 172.21.15.43:0/3587634414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611f61be290 0x5611f61d2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.150+0000 7ff2a9322640 1 -- 172.21.15.43:0/3587634414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611f61b39d0 con 0x5611f61c6e10 2024-04-21T08:48:59.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.150+0000 7ff29bfff640 1 --2- 172.21.15.43:0/3587634414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611f61be290 0x5611f61d2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:59.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.150+0000 7ff2a9322640 1 --2- 172.21.15.43:0/3587634414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61c6e10 0x5611f61b9a90 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7ff29c010c80 tx=0x7ff29c007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596986d75882a31b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.151+0000 7ff28a7fc640 1 -- 172.21.15.43:0/3587634414 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c010cb0 con 0x5611f61c6e10 2024-04-21T08:48:59.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.151+0000 7ff28a7fc640 1 -- 172.21.15.43:0/3587634414 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff29c010e50 con 0x5611f61c6e10 2024-04-21T08:48:59.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.151+0000 7ff28a7fc640 1 -- 172.21.15.43:0/3587634414 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c01f710 con 0x5611f61c6e10 2024-04-21T08:48:59.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.151+0000 7ff2ad975740 1 -- 172.21.15.43:0/3587634414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61c6e10 msgr2=0x5611f61b9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.151+0000 7ff2ad975740 1 --2- 172.21.15.43:0/3587634414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61c6e10 0x5611f61b9a90 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7ff29c010c80 tx=0x7ff29c007620 comp rx=0 tx=0).stop 2024-04-21T08:48:59.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.151+0000 7ff2ad975740 1 -- 172.21.15.43:0/3587634414 shutdown_connections 2024-04-21T08:48:59.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.151+0000 7ff2ad975740 1 --2- 172.21.15.43:0/3587634414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61c6e10 0x5611f61b9a90 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.151+0000 7ff2ad975740 1 --2- 172.21.15.43:0/3587634414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611f61be290 0x5611f61d2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.151+0000 7ff2ad975740 1 --2- 172.21.15.43:0/3587634414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611f61bd8e0 0x5611f61c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.151+0000 7ff2ad975740 1 -- 172.21.15.43:0/3587634414 >> 172.21.15.43:0/3587634414 conn(0x5611f61b7fb0 msgr2=0x5611f61b9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.151+0000 7ff2ad975740 1 -- 172.21.15.43:0/3587634414 shutdown_connections 2024-04-21T08:48:59.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.151+0000 7ff2ad975740 1 -- 172.21.15.43:0/3587634414 wait complete. 2024-04-21T08:48:59.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.152+0000 7ff2ad975740 1 -- start start 2024-04-21T08:48:59.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.152+0000 7ff2ad975740 1 Processor -- start 2024-04-21T08:48:59.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.152+0000 7ff2ad975740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611f61bd8e0 0x5611f61cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.152+0000 7ff2ad975740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61be290 0x5611f61cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.152+0000 7ff2ad975740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611f61c6e10 0x5611f61cb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.152+0000 7ff2ad975740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a41b0870 con 0x5611f61be290 2024-04-21T08:48:59.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.152+0000 7ff2ad975740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a41b01a0 con 0x5611f61bd8e0 2024-04-21T08:48:59.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.152+0000 7ff2ad975740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a41ad400 con 0x5611f61c6e10 2024-04-21T08:48:59.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.152+0000 7ff2ad975740 10 client.0 _ll_register_callbacks cb 0x7ff294097e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:59.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.152+0000 7ff29bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61be290 0x5611f61cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.152+0000 7ff29bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61be290 0x5611f61cf2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47706/0 (socket says 172.21.15.43:47706) 2024-04-21T08:48:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.152+0000 7ff29bfff640 1 -- 172.21.15.43:0/2188762520 learned_addr learned my addr 172.21.15.43:0/2188762520 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.152+0000 7ff2a8b21640 1 --2- 172.21.15.43:0/2188762520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611f61bd8e0 0x5611f61cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.152+0000 7ff29bfff640 1 -- 172.21.15.43:0/2188762520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611f61c6e10 msgr2=0x5611f61cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.152+0000 7ff29bfff640 1 --2- 172.21.15.43:0/2188762520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611f61c6e10 0x5611f61cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.152+0000 7ff29bfff640 1 -- 172.21.15.43:0/2188762520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611f61bd8e0 msgr2=0x5611f61cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.152+0000 7ff29bfff640 1 --2- 172.21.15.43:0/2188762520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611f61bd8e0 0x5611f61cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.152+0000 7ff29bfff640 1 -- 172.21.15.43:0/2188762520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611f61cba50 con 0x5611f61be290 2024-04-21T08:48:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.153+0000 7ff29bfff640 1 --2- 172.21.15.43:0/2188762520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61be290 0x5611f61cf2f0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7ff294009610 tx=0x7ff29400de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c25abe969dc8c85 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.153+0000 7ff283fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.153+0000 7ff283fff640 1 -- 172.21.15.43:0/2188762520 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294021040 con 0x5611f61be290 2024-04-21T08:48:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.153+0000 7ff283fff640 1 -- 172.21.15.43:0/2188762520 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7ff294098900 con 0x5611f61be290 2024-04-21T08:48:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.153+0000 7ff2ad975740 1 -- 172.21.15.43:0/2188762520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff29c010e50 con 0x5611f61be290 2024-04-21T08:48:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.153+0000 7ff283fff640 1 -- 172.21.15.43:0/2188762520 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff29409c290 con 0x5611f61be290 2024-04-21T08:48:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.153+0000 7ff2ad975740 10 client.20322 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.153+0000 7ff2ad975740 1 -- 172.21.15.43:0/2188762520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5611f62422a0 con 0x5611f61be290 2024-04-21T08:48:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.153+0000 7ff283fff640 1 -- 172.21.15.43:0/2188762520 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff29c010e50 con 0x5611f61be290 2024-04-21T08:48:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.153+0000 7ff283fff640 1 -- 172.21.15.43:0/2188762520 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff294081c50 con 0x5611f61be290 2024-04-21T08:48:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.153+0000 7ff2ad975740 10 client.20322 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.153+0000 7ff2ad975740 10 client.20322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.153+0000 7ff2ad975740 20 client.20322 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.153+0000 7ff2ad975740 1 --2- 172.21.15.43:0/2188762520 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5611f62429d0 0x5611f6244df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.153+0000 7ff2ad975740 4 client.20322 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:48:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.153+0000 7ff2ad975740 1 -- 172.21.15.43:0/2188762520 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5611f6129790 con 0x5611f62429d0 2024-04-21T08:48:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.154+0000 7ff2a8b21640 1 --2- 172.21.15.43:0/2188762520 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5611f62429d0 0x5611f6244df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.154+0000 7ff2a8b21640 1 --2- 172.21.15.43:0/2188762520 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5611f62429d0 0x5611f6244df0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7fadd434dc0ee0b server_cookie=8e05a704721bf206 in_seq=0 out_seq=0 2024-04-21T08:48:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.154+0000 7ff283fff640 10 client.20322 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.155+0000 7ff283fff640 1 -- 172.21.15.43:0/2188762520 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5611f6129790 con 0x5611f62429d0 2024-04-21T08:48:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.155+0000 7ff283fff640 10 client.20322 handle_command_reply: tid=0 2024-04-21T08:48:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.155+0000 7ff2ad975740 1 client.20322 shutdown 2024-04-21T08:48:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.155+0000 7ff2ad975740 1 -- 172.21.15.43:0/2188762520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61be290 msgr2=0x5611f61cf2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.155+0000 7ff2ad975740 1 --2- 172.21.15.43:0/2188762520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61be290 0x5611f61cf2f0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7ff294009610 tx=0x7ff29400de80 comp rx=0 tx=0).stop 2024-04-21T08:48:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.155+0000 7ff2ad975740 1 -- 172.21.15.43:0/2188762520 shutdown_connections 2024-04-21T08:48:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.155+0000 7ff2ad975740 1 --2- 172.21.15.43:0/2188762520 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5611f62429d0 0x5611f6244df0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.155+0000 7ff2ad975740 1 --2- 172.21.15.43:0/2188762520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5611f61c6e10 0x5611f61cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.155+0000 7ff2ad975740 1 --2- 172.21.15.43:0/2188762520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5611f61be290 0x5611f61cf2f0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.155+0000 7ff2ad975740 1 --2- 172.21.15.43:0/2188762520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611f61bd8e0 0x5611f61cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.155+0000 7ff2ad975740 1 -- 172.21.15.43:0/2188762520 >> 172.21.15.43:0/2188762520 conn(0x5611f61b7fb0 msgr2=0x5611f61c4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.155+0000 7ff283fff640 0 client.20322 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:59.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.156+0000 7ff2ad975740 1 -- 172.21.15.43:0/2188762520 shutdown_connections 2024-04-21T08:48:59.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.156+0000 7ff2ad975740 1 -- 172.21.15.43:0/2188762520 wait complete. 2024-04-21T08:48:59.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.156+0000 7ff2ad975740 20 client.20322 trim_cache size 0 max 16384 2024-04-21T08:48:59.161 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:48:59.161 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:48:59.161 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "70ad6967-15bd-4109-952a-a16a213ec19f", 2024-04-21T08:48:59.161 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:48:59.161 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:48:59.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.156+0000 7ff2aad85640 1 -- 172.21.15.43:0/1024489079 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff28c0a3da0 msgr2=0x7ff28c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.156+0000 7ff2aad85640 1 --2- 172.21.15.43:0/1024489079 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff28c0a3da0 0x7ff28c0a6260 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff29c007310 tx=0x7ff29c0072a0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.156+0000 7ff2aad85640 1 -- 172.21.15.43:0/1024489079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2a4107260 msgr2=0x7ff2a4198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.156+0000 7ff2aad85640 1 --2- 172.21.15.43:0/1024489079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2a4107260 0x7ff2a4198c60 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7ff29400d770 tx=0x7ff29400dc30 comp rx=0 tx=0).stop 2024-04-21T08:48:59.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.156+0000 7ff2aad85640 1 -- 172.21.15.43:0/1024489079 shutdown_connections 2024-04-21T08:48:59.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.156+0000 7ff2aad85640 1 --2- 172.21.15.43:0/1024489079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2a419cfe0 0x7ff2a419d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.156+0000 7ff2aad85640 1 --2- 172.21.15.43:0/1024489079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2a4107260 0x7ff2a4198c60 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.156+0000 7ff2aad85640 1 --2- 172.21.15.43:0/1024489079 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff28c0a3da0 0x7ff28c0a6260 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.156+0000 7ff2aad85640 1 --2- 172.21.15.43:0/1024489079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a4102190 0x7ff2a4198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.156+0000 7ff2aad85640 1 -- 172.21.15.43:0/1024489079 >> 172.21.15.43:0/1024489079 conn(0x7ff2a40fd990 msgr2=0x7ff2a40fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.157+0000 7ff2aad85640 1 -- 172.21.15.43:0/1024489079 shutdown_connections 2024-04-21T08:48:59.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.157+0000 7ff2aad85640 1 -- 172.21.15.43:0/1024489079 wait complete. 2024-04-21T08:48:59.167 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:48:59.167 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:48:59.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.273+0000 7f851366d640 1 -- 172.21.15.43:0/293650616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f850c074250 msgr2=0x7f850c074630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.273+0000 7f851366d640 1 --2- 172.21.15.43:0/293650616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f850c074250 0x7f850c074630 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f8508009900 tx=0x7f850802bcd0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.273+0000 7f851366d640 1 -- 172.21.15.43:0/293650616 shutdown_connections 2024-04-21T08:48:59.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.273+0000 7f851366d640 1 --2- 172.21.15.43:0/293650616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f850c10e810 0x7f850c110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.273+0000 7f851366d640 1 --2- 172.21.15.43:0/293650616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f850c074b70 0x7f850c10e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.273+0000 7f851366d640 1 --2- 172.21.15.43:0/293650616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f850c074250 0x7f850c074630 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.273+0000 7f851366d640 1 -- 172.21.15.43:0/293650616 >> 172.21.15.43:0/293650616 conn(0x7f850c0793e0 msgr2=0x7f850c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.273+0000 7f851366d640 1 -- 172.21.15.43:0/293650616 shutdown_connections 2024-04-21T08:48:59.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.273+0000 7f851366d640 1 -- 172.21.15.43:0/293650616 wait complete. 2024-04-21T08:48:59.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.273+0000 7f851366d640 1 Processor -- start 2024-04-21T08:48:59.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.273+0000 7f851366d640 1 -- start start 2024-04-21T08:48:59.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.274+0000 7f851366d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f850c074250 0x7f850c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.274+0000 7f851366d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f850c074b70 0x7f850c1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.274+0000 7f851366d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f850c10e810 0x7f850c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.274+0000 7f851366d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f850c1143b0 con 0x7f850c074b70 2024-04-21T08:48:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.274+0000 7f851366d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f850c114230 con 0x7f850c074250 2024-04-21T08:48:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.274+0000 7f851366d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f850c114530 con 0x7f850c10e810 2024-04-21T08:48:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.274+0000 7f8510c08640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f850c074b70 0x7f850c1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.274+0000 7f8510c08640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f850c074b70 0x7f850c1a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47730/0 (socket says 172.21.15.43:47730) 2024-04-21T08:48:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.274+0000 7f8510c08640 1 -- 172.21.15.43:0/1202257256 learned_addr learned my addr 172.21.15.43:0/1202257256 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.274+0000 7f8511409640 1 --2- 172.21.15.43:0/1202257256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f850c074250 0x7f850c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.274+0000 7f8511c0a640 1 --2- 172.21.15.43:0/1202257256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f850c10e810 0x7f850c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.274+0000 7f8510c08640 1 -- 172.21.15.43:0/1202257256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f850c10e810 msgr2=0x7f850c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.274+0000 7f8510c08640 1 --2- 172.21.15.43:0/1202257256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f850c10e810 0x7f850c19a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.274+0000 7f8510c08640 1 -- 172.21.15.43:0/1202257256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f850c074250 msgr2=0x7f850c1a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.274+0000 7f8510c08640 1 --2- 172.21.15.43:0/1202257256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f850c074250 0x7f850c1a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.274+0000 7f8510c08640 1 -- 172.21.15.43:0/1202257256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850c19afb0 con 0x7f850c074b70 2024-04-21T08:48:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.274+0000 7f8510c08640 1 --2- 172.21.15.43:0/1202257256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f850c074b70 0x7f850c1a0b70 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00c920 tx=0x7f84fc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.274+0000 7f85027fc640 1 -- 172.21.15.43:0/1202257256 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc007c70 con 0x7f850c074b70 2024-04-21T08:48:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.274+0000 7f85027fc640 1 -- 172.21.15.43:0/1202257256 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84fc007e10 con 0x7f850c074b70 2024-04-21T08:48:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.274+0000 7f851366d640 1 -- 172.21.15.43:0/1202257256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850c19b2a0 con 0x7f850c074b70 2024-04-21T08:48:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.274+0000 7f85027fc640 1 -- 172.21.15.43:0/1202257256 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc018820 con 0x7f850c074b70 2024-04-21T08:48:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.274+0000 7f851366d640 1 -- 172.21.15.43:0/1202257256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f850c1a75b0 con 0x7f850c074b70 2024-04-21T08:48:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.277+0000 7f85027fc640 1 -- 172.21.15.43:0/1202257256 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f84fc00f450 con 0x7f850c074b70 2024-04-21T08:48:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.277+0000 7f8516269740 1 Processor -- start 2024-04-21T08:48:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.277+0000 7f8516269740 1 -- start start 2024-04-21T08:48:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.277+0000 7f8516269740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858af3a580 0x55858af3a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.277+0000 7f8516269740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af3aea0 0x55858af43540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.277+0000 7f8516269740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55858af43a80 0x55858af45e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.277+0000 7f8516269740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55858ae9f1e0 con 0x55858af3aea0 2024-04-21T08:48:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.277+0000 7f8516269740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55858ae3c390 con 0x55858af3a580 2024-04-21T08:48:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.277+0000 7f8516269740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55858ae9b1a0 con 0x55858af43a80 2024-04-21T08:48:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.277+0000 7f85027fc640 1 --2- 172.21.15.43:0/1202257256 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84f80a3da0 0x7f84f80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.277+0000 7f8511409640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af3aea0 0x55858af43540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.277+0000 7f8511409640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af3aea0 0x55858af43540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47740/0 (socket says 172.21.15.43:47740) 2024-04-21T08:48:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.277+0000 7f8511409640 1 -- 172.21.15.43:0/3283516328 learned_addr learned my addr 172.21.15.43:0/3283516328 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.277+0000 7f8511c0a640 1 --2- 172.21.15.43:0/3283516328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858af3a580 0x55858af3a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.278+0000 7f85027fc640 1 -- 172.21.15.43:0/1202257256 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f84fc0d0ce0 con 0x7f850c074b70 2024-04-21T08:48:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.278+0000 7f8511c0a640 1 --2- 172.21.15.43:0/1202257256 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84f80a3da0 0x7f84f80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.278+0000 7f8510c08640 1 --2- 172.21.15.43:0/3283516328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55858af43a80 0x55858af45e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.278+0000 7f8511409640 1 -- 172.21.15.43:0/3283516328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55858af43a80 msgr2=0x55858af45e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.278+0000 7f8511409640 1 --2- 172.21.15.43:0/3283516328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55858af43a80 0x55858af45e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.278+0000 7f8511409640 1 -- 172.21.15.43:0/3283516328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858af3a580 msgr2=0x55858af3a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.278+0000 7f8511409640 1 --2- 172.21.15.43:0/3283516328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858af3a580 0x55858af3a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.278+0000 7f8511409640 1 -- 172.21.15.43:0/3283516328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55858ac7b7d0 con 0x55858af3aea0 2024-04-21T08:48:59.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.278+0000 7f8511c0a640 1 --2- 172.21.15.43:0/3283516328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858af3a580 0x55858af3a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:59.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.278+0000 7f8510c08640 1 --2- 172.21.15.43:0/3283516328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55858af43a80 0x55858af45e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:59.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.278+0000 7f8511409640 1 --2- 172.21.15.43:0/3283516328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af3aea0 0x55858af43540 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f8508009900 tx=0x7f8508004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ae2aa3d02da907 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.278+0000 7f8511c0a640 1 --2- 172.21.15.43:0/1202257256 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84f80a3da0 0x7f84f80a6260 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8504009fd0 tx=0x7f8504004a40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.278+0000 7f84f77fe640 1 -- 172.21.15.43:0/3283516328 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508004230 con 0x55858af3aea0 2024-04-21T08:48:59.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.278+0000 7f84f77fe640 1 -- 172.21.15.43:0/3283516328 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85080043d0 con 0x55858af3aea0 2024-04-21T08:48:59.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.278+0000 7f84f77fe640 1 -- 172.21.15.43:0/3283516328 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508004920 con 0x55858af3aea0 2024-04-21T08:48:59.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.279+0000 7f8516269740 1 -- 172.21.15.43:0/3283516328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af3aea0 msgr2=0x55858af43540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.279+0000 7f8516269740 1 --2- 172.21.15.43:0/3283516328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af3aea0 0x55858af43540 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f8508009900 tx=0x7f8508004040 comp rx=0 tx=0).stop 2024-04-21T08:48:59.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.279+0000 7f8516269740 1 -- 172.21.15.43:0/3283516328 shutdown_connections 2024-04-21T08:48:59.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.279+0000 7f8516269740 1 --2- 172.21.15.43:0/3283516328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55858af43a80 0x55858af45e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.279+0000 7f8516269740 1 --2- 172.21.15.43:0/3283516328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af3aea0 0x55858af43540 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.279+0000 7f8516269740 1 --2- 172.21.15.43:0/3283516328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858af3a580 0x55858af3a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.279+0000 7f8516269740 1 -- 172.21.15.43:0/3283516328 >> 172.21.15.43:0/3283516328 conn(0x55858af34cb0 msgr2=0x55858af35090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.279+0000 7f8516269740 1 -- 172.21.15.43:0/3283516328 shutdown_connections 2024-04-21T08:48:59.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.279+0000 7f8516269740 1 -- 172.21.15.43:0/3283516328 wait complete. 2024-04-21T08:48:59.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.279+0000 7f8516269740 1 -- start start 2024-04-21T08:48:59.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.279+0000 7f8516269740 1 Processor -- start 2024-04-21T08:48:59.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.280+0000 7f8516269740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858af3a580 0x55858af40a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.280+0000 7f8516269740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55858af3aea0 0x55858af40f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.280+0000 7f8516269740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af43a80 0x55858af414d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.280+0000 7f8516269740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55858af46810 con 0x55858af43a80 2024-04-21T08:48:59.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.280+0000 7f8516269740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55858ae582f0 con 0x55858af3a580 2024-04-21T08:48:59.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.280+0000 7f8516269740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55858af46990 con 0x55858af3aea0 2024-04-21T08:48:59.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.280+0000 7f8516269740 10 client.0 _ll_register_callbacks cb 0x55858ae65580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:59.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.280+0000 7f8511c0a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af43a80 0x55858af414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.280+0000 7f8511c0a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af43a80 0x55858af414d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47744/0 (socket says 172.21.15.43:47744) 2024-04-21T08:48:59.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.280+0000 7f8511c0a640 1 -- 172.21.15.43:0/3568806761 learned_addr learned my addr 172.21.15.43:0/3568806761 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:59.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.280+0000 7f8511409640 1 --2- 172.21.15.43:0/3568806761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858af3a580 0x55858af40a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.280+0000 7f8510c08640 1 --2- 172.21.15.43:0/3568806761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55858af3aea0 0x55858af40f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.280+0000 7f8511c0a640 1 -- 172.21.15.43:0/3568806761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55858af3aea0 msgr2=0x55858af40f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.280+0000 7f8511c0a640 1 --2- 172.21.15.43:0/3568806761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55858af3aea0 0x55858af40f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.280+0000 7f8511c0a640 1 -- 172.21.15.43:0/3568806761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858af3a580 msgr2=0x55858af40a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.280+0000 7f8511c0a640 1 --2- 172.21.15.43:0/3568806761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858af3a580 0x55858af40a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.280+0000 7f8511c0a640 1 -- 172.21.15.43:0/3568806761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55858abb13f0 con 0x55858af43a80 2024-04-21T08:48:59.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.280+0000 7f8511c0a640 1 --2- 172.21.15.43:0/3568806761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af43a80 0x55858af414d0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f8504014840 tx=0x7f8504004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b93cc5a9a76514 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.280+0000 7f84f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:59.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.281+0000 7f84f4ff9640 1 -- 172.21.15.43:0/3568806761 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850400fb60 con 0x55858af43a80 2024-04-21T08:48:59.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.281+0000 7f84f4ff9640 1 -- 172.21.15.43:0/3568806761 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f850400fd00 con 0x55858af43a80 2024-04-21T08:48:59.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.281+0000 7f8516269740 1 -- 172.21.15.43:0/3568806761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85080043d0 con 0x55858af43a80 2024-04-21T08:48:59.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.281+0000 7f84f4ff9640 1 -- 172.21.15.43:0/3568806761 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8504020450 con 0x55858af43a80 2024-04-21T08:48:59.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.281+0000 7f8516269740 10 client.20334 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:59.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.281+0000 7f8516269740 1 -- 172.21.15.43:0/3568806761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55858ab710f0 con 0x55858af43a80 2024-04-21T08:48:59.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.281+0000 7f84f4ff9640 1 -- 172.21.15.43:0/3568806761 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85080043d0 con 0x55858af43a80 2024-04-21T08:48:59.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.281+0000 7f84f4ff9640 1 -- 172.21.15.43:0/3568806761 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f850401d040 con 0x55858af43a80 2024-04-21T08:48:59.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.281+0000 7f8516269740 10 client.20334 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:59.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.281+0000 7f8516269740 10 client.20334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:59.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.281+0000 7f8516269740 20 client.20334 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:59.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.281+0000 7f8516269740 1 --2- 172.21.15.43:0/3568806761 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55858af53a30 0x55858af55e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.281+0000 7f8516269740 4 client.20334 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:59.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.281+0000 7f8516269740 1 -- 172.21.15.43:0/3568806761 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55858aeacc00 con 0x55858af53a30 2024-04-21T08:48:59.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.281+0000 7f8511409640 1 --2- 172.21.15.43:0/3568806761 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55858af53a30 0x55858af55e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.282+0000 7f8511409640 1 --2- 172.21.15.43:0/3568806761 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55858af53a30 0x55858af55e90 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=252f0f765484838c server_cookie=c10bfee5f0ea7558 in_seq=0 out_seq=0 2024-04-21T08:48:59.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.282+0000 7f84f4ff9640 10 client.20334 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:59.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.283+0000 7f84f4ff9640 1 -- 172.21.15.43:0/3568806761 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55858aeacc00 con 0x55858af53a30 2024-04-21T08:48:59.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.283+0000 7f84f4ff9640 10 client.20334 handle_command_reply: tid=0 2024-04-21T08:48:59.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.283+0000 7f8516269740 1 client.20334 shutdown 2024-04-21T08:48:59.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.283+0000 7f8516269740 1 -- 172.21.15.43:0/3568806761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af43a80 msgr2=0x55858af414d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.283+0000 7f8516269740 1 --2- 172.21.15.43:0/3568806761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af43a80 0x55858af414d0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f8504014840 tx=0x7f8504004ab0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.283+0000 7f8516269740 1 -- 172.21.15.43:0/3568806761 shutdown_connections 2024-04-21T08:48:59.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.283+0000 7f8516269740 1 --2- 172.21.15.43:0/3568806761 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55858af53a30 0x55858af55e90 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.283+0000 7f8516269740 1 --2- 172.21.15.43:0/3568806761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af43a80 0x55858af414d0 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.283+0000 7f8516269740 1 --2- 172.21.15.43:0/3568806761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55858af3aea0 0x55858af40f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.283+0000 7f84f4ff9640 0 client.20334 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:59.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.283+0000 7f8516269740 1 --2- 172.21.15.43:0/3568806761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858af3a580 0x55858af40a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.283+0000 7f8516269740 1 -- 172.21.15.43:0/3568806761 >> 172.21.15.43:0/3568806761 conn(0x55858af34cb0 msgr2=0x55858af36a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.283+0000 7f8516269740 1 -- 172.21.15.43:0/3568806761 shutdown_connections 2024-04-21T08:48:59.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.283+0000 7f8516269740 1 -- 172.21.15.43:0/3568806761 wait complete. 2024-04-21T08:48:59.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.283+0000 7f8516269740 20 client.20334 trim_cache size 0 max 16384 2024-04-21T08:48:59.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.301+0000 7f8516269740 1 Processor -- start 2024-04-21T08:48:59.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.301+0000 7f8516269740 1 -- start start 2024-04-21T08:48:59.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.301+0000 7f8516269740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55858af3a580 0x55858af43180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.301+0000 7f8516269740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858af3aea0 0x55858af3f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.301+0000 7f8516269740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af43a80 0x55858af4f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.301+0000 7f8516269740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f850c1b16b0 con 0x55858af43a80 2024-04-21T08:48:59.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.301+0000 7f8516269740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f850c1b31f0 con 0x55858af3aea0 2024-04-21T08:48:59.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.301+0000 7f8516269740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f850c1b1d80 con 0x55858af3a580 2024-04-21T08:48:59.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.301+0000 7f8511c0a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af43a80 0x55858af4f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.301+0000 7f8511c0a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af43a80 0x55858af4f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47750/0 (socket says 172.21.15.43:47750) 2024-04-21T08:48:59.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.301+0000 7f8511c0a640 1 -- 172.21.15.43:0/2021135852 learned_addr learned my addr 172.21.15.43:0/2021135852 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:59.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.301+0000 7f8511409640 1 --2- 172.21.15.43:0/2021135852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55858af3a580 0x55858af43180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.301+0000 7f8510c08640 1 --2- 172.21.15.43:0/2021135852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858af3aea0 0x55858af3f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.301+0000 7f8511c0a640 1 -- 172.21.15.43:0/2021135852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55858af3a580 msgr2=0x55858af43180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.301+0000 7f8511c0a640 1 --2- 172.21.15.43:0/2021135852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55858af3a580 0x55858af43180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.301+0000 7f8511c0a640 1 -- 172.21.15.43:0/2021135852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858af3aea0 msgr2=0x55858af3f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.301+0000 7f8511c0a640 1 --2- 172.21.15.43:0/2021135852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858af3aea0 0x55858af3f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.301+0000 7f8511c0a640 1 -- 172.21.15.43:0/2021135852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55858af34ae0 con 0x55858af43a80 2024-04-21T08:48:59.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.301+0000 7f8510c08640 1 --2- 172.21.15.43:0/2021135852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858af3aea0 0x55858af3f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:59.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.301+0000 7f8511c0a640 1 --2- 172.21.15.43:0/2021135852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af43a80 0x55858af4f730 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f85040095e0 tx=0x7f8504009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41e1e24fa25723c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.302+0000 7f84f6ffd640 1 -- 172.21.15.43:0/2021135852 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850403b020 con 0x55858af43a80 2024-04-21T08:48:59.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.302+0000 7f84f6ffd640 1 -- 172.21.15.43:0/2021135852 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f850402aca0 con 0x55858af43a80 2024-04-21T08:48:59.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.302+0000 7f84f6ffd640 1 -- 172.21.15.43:0/2021135852 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850400faa0 con 0x55858af43a80 2024-04-21T08:48:59.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.302+0000 7f8516269740 1 -- 172.21.15.43:0/2021135852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af43a80 msgr2=0x55858af4f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.302+0000 7f8516269740 1 --2- 172.21.15.43:0/2021135852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af43a80 0x55858af4f730 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f85040095e0 tx=0x7f8504009b50 comp rx=0 tx=0).stop 2024-04-21T08:48:59.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.302+0000 7f8516269740 1 -- 172.21.15.43:0/2021135852 shutdown_connections 2024-04-21T08:48:59.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.302+0000 7f8516269740 1 --2- 172.21.15.43:0/2021135852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af43a80 0x55858af4f730 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.302+0000 7f8516269740 1 --2- 172.21.15.43:0/2021135852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858af3aea0 0x55858af3f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.302+0000 7f8516269740 1 --2- 172.21.15.43:0/2021135852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55858af3a580 0x55858af43180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.302+0000 7f8516269740 1 -- 172.21.15.43:0/2021135852 >> 172.21.15.43:0/2021135852 conn(0x55858af34cb0 msgr2=0x55858af36970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.302+0000 7f8516269740 1 -- 172.21.15.43:0/2021135852 shutdown_connections 2024-04-21T08:48:59.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.302+0000 7f8516269740 1 -- 172.21.15.43:0/2021135852 wait complete. 2024-04-21T08:48:59.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.302+0000 7f8516269740 1 -- start start 2024-04-21T08:48:59.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.303+0000 7f8516269740 1 Processor -- start 2024-04-21T08:48:59.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.303+0000 7f8516269740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858af3a580 0x55858af477c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.303+0000 7f8516269740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af3aea0 0x55858af47d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.303+0000 7f8516269740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55858af43a80 0x55858af48240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.303+0000 7f8516269740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f850c1b2b20 con 0x55858af3aea0 2024-04-21T08:48:59.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.303+0000 7f8516269740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f850c1b2450 con 0x55858af3a580 2024-04-21T08:48:59.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.303+0000 7f8516269740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f850c1af6b0 con 0x55858af43a80 2024-04-21T08:48:59.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.303+0000 7f8516269740 10 client.0 _ll_register_callbacks cb 0x7f8504009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:59.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.303+0000 7f8510c08640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af3aea0 0x55858af47d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.303+0000 7f8510c08640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af3aea0 0x55858af47d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47758/0 (socket says 172.21.15.43:47758) 2024-04-21T08:48:59.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.303+0000 7f8510c08640 1 -- 172.21.15.43:0/2923059688 learned_addr learned my addr 172.21.15.43:0/2923059688 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:59.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.303+0000 7f8511409640 1 --2- 172.21.15.43:0/2923059688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858af3a580 0x55858af477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.303+0000 7f8511c0a640 1 --2- 172.21.15.43:0/2923059688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55858af43a80 0x55858af48240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.303+0000 7f8510c08640 1 -- 172.21.15.43:0/2923059688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55858af43a80 msgr2=0x55858af48240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.303+0000 7f8510c08640 1 --2- 172.21.15.43:0/2923059688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55858af43a80 0x55858af48240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.303+0000 7f8510c08640 1 -- 172.21.15.43:0/2923059688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858af3a580 msgr2=0x55858af477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.303+0000 7f8510c08640 1 --2- 172.21.15.43:0/2923059688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858af3a580 0x55858af477c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.303+0000 7f8510c08640 1 -- 172.21.15.43:0/2923059688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55858afb9130 con 0x55858af3aea0 2024-04-21T08:48:59.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.304+0000 7f8510c08640 1 --2- 172.21.15.43:0/2923059688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af3aea0 0x55858af47d00 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0ce610 tx=0x7f84fc06b210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8881086c7c9e4c6d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.304+0000 7f84f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:59.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.304+0000 7f84f4ff9640 1 -- 172.21.15.43:0/2923059688 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc09c860 con 0x55858af3aea0 2024-04-21T08:48:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.304+0000 7f84f4ff9640 1 -- 172.21.15.43:0/2923059688 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f84fc098ed0 con 0x55858af3aea0 2024-04-21T08:48:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.304+0000 7f8516269740 1 -- 172.21.15.43:0/2923059688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f850402aca0 con 0x55858af3aea0 2024-04-21T08:48:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.304+0000 7f84f4ff9640 1 -- 172.21.15.43:0/2923059688 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f84fc0a0c10 con 0x55858af3aea0 2024-04-21T08:48:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.304+0000 7f8516269740 10 client.20340 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.304+0000 7f8516269740 1 -- 172.21.15.43:0/2923059688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55858afba440 con 0x55858af3aea0 2024-04-21T08:48:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.304+0000 7f84f4ff9640 1 -- 172.21.15.43:0/2923059688 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f850402aca0 con 0x55858af3aea0 2024-04-21T08:48:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.304+0000 7f84f4ff9640 1 -- 172.21.15.43:0/2923059688 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f84fc0a5040 con 0x55858af3aea0 2024-04-21T08:48:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.304+0000 7f8516269740 10 client.20340 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.304+0000 7f8516269740 10 client.20340 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.304+0000 7f8516269740 20 client.20340 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.304+0000 7f8516269740 1 --2- 172.21.15.43:0/2923059688 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55858afbab20 0x55858afbcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.304+0000 7f8516269740 4 client.20340 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:48:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.304+0000 7f8516269740 1 -- 172.21.15.43:0/2923059688 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55858aea6a70 con 0x55858afbab20 2024-04-21T08:48:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.305+0000 7f8511409640 1 --2- 172.21.15.43:0/2923059688 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55858afbab20 0x55858afbcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.305+0000 7f8511409640 1 --2- 172.21.15.43:0/2923059688 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55858afbab20 0x55858afbcf40 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d71b6f3537f3565 server_cookie=6df025ca2ac30367 in_seq=0 out_seq=0 2024-04-21T08:48:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.305+0000 7f84f4ff9640 10 client.20340 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.305+0000 7f84f4ff9640 1 -- 172.21.15.43:0/2923059688 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55858aea6a70 con 0x55858afbab20 2024-04-21T08:48:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.305+0000 7f84f4ff9640 10 client.20340 handle_command_reply: tid=0 2024-04-21T08:48:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.305+0000 7f8516269740 1 client.20340 shutdown 2024-04-21T08:48:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.305+0000 7f8516269740 1 -- 172.21.15.43:0/2923059688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af3aea0 msgr2=0x55858af47d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.305+0000 7f8516269740 1 --2- 172.21.15.43:0/2923059688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af3aea0 0x55858af47d00 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0ce610 tx=0x7f84fc06b210 comp rx=0 tx=0).stop 2024-04-21T08:48:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.305+0000 7f8516269740 1 -- 172.21.15.43:0/2923059688 shutdown_connections 2024-04-21T08:48:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.305+0000 7f8516269740 1 --2- 172.21.15.43:0/2923059688 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55858afbab20 0x55858afbcf40 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.305+0000 7f8516269740 1 --2- 172.21.15.43:0/2923059688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55858af43a80 0x55858af48240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.305+0000 7f8516269740 1 --2- 172.21.15.43:0/2923059688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55858af3aea0 0x55858af47d00 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.305+0000 7f8516269740 1 --2- 172.21.15.43:0/2923059688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858af3a580 0x55858af477c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.305+0000 7f84f4ff9640 0 client.20340 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:48:59.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.305+0000 7f8516269740 1 -- 172.21.15.43:0/2923059688 >> 172.21.15.43:0/2923059688 conn(0x55858af34cb0 msgr2=0x55858af40a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.305+0000 7f8516269740 1 -- 172.21.15.43:0/2923059688 shutdown_connections 2024-04-21T08:48:59.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.305+0000 7f8516269740 1 -- 172.21.15.43:0/2923059688 wait complete. 2024-04-21T08:48:59.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.305+0000 7f8516269740 20 client.20340 trim_cache size 0 max 16384 2024-04-21T08:48:59.313 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:48:59.313 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:48:59.313 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:48:59.314 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:48:59.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.306+0000 7f851366d640 1 -- 172.21.15.43:0/1202257256 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84f80a3da0 msgr2=0x7f84f80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.306+0000 7f851366d640 1 --2- 172.21.15.43:0/1202257256 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84f80a3da0 0x7f84f80a6260 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8504009fd0 tx=0x7f8504004a40 comp rx=0 tx=0).stop 2024-04-21T08:48:59.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.306+0000 7f851366d640 1 -- 172.21.15.43:0/1202257256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f850c074b70 msgr2=0x7f850c1a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.306+0000 7f851366d640 1 --2- 172.21.15.43:0/1202257256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f850c074b70 0x7f850c1a0b70 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00c920 tx=0x7f84fc00cde0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.306+0000 7f851366d640 1 -- 172.21.15.43:0/1202257256 shutdown_connections 2024-04-21T08:48:59.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.306+0000 7f851366d640 1 --2- 172.21.15.43:0/1202257256 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84f80a3da0 0x7f84f80a6260 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.306+0000 7f851366d640 1 --2- 172.21.15.43:0/1202257256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f850c10e810 0x7f850c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.306+0000 7f851366d640 1 --2- 172.21.15.43:0/1202257256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f850c074b70 0x7f850c1a0b70 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.306+0000 7f851366d640 1 --2- 172.21.15.43:0/1202257256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f850c074250 0x7f850c1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.306+0000 7f851366d640 1 -- 172.21.15.43:0/1202257256 >> 172.21.15.43:0/1202257256 conn(0x7f850c0793e0 msgr2=0x7f850c0feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.306+0000 7f851366d640 1 -- 172.21.15.43:0/1202257256 shutdown_connections 2024-04-21T08:48:59.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.306+0000 7f851366d640 1 -- 172.21.15.43:0/1202257256 wait complete. 2024-04-21T08:48:59.316 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:48:59.317 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:48:59.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.419+0000 7f29485fb640 1 -- 172.21.15.43:0/422656596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2940101ab0 msgr2=0x7f294010dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.419+0000 7f29485fb640 1 --2- 172.21.15.43:0/422656596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2940101ab0 0x7f294010dc50 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f2938009cf0 tx=0x7f293802e300 comp rx=0 tx=0).stop 2024-04-21T08:48:59.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.419+0000 7f29485fb640 1 -- 172.21.15.43:0/422656596 shutdown_connections 2024-04-21T08:48:59.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.419+0000 7f29485fb640 1 --2- 172.21.15.43:0/422656596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2940101ab0 0x7f294010dc50 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.419+0000 7f29485fb640 1 --2- 172.21.15.43:0/422656596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2940101110 0x7f2940101570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.419+0000 7f29485fb640 1 --2- 172.21.15.43:0/422656596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2940107010 0x7f29401073f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.419+0000 7f29485fb640 1 -- 172.21.15.43:0/422656596 >> 172.21.15.43:0/422656596 conn(0x7f2940077e30 msgr2=0x7f2940078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.419+0000 7f29485fb640 1 -- 172.21.15.43:0/422656596 shutdown_connections 2024-04-21T08:48:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.419+0000 7f29485fb640 1 -- 172.21.15.43:0/422656596 wait complete. 2024-04-21T08:48:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.420+0000 7f29485fb640 1 Processor -- start 2024-04-21T08:48:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.420+0000 7f29485fb640 1 -- start start 2024-04-21T08:48:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.420+0000 7f29485fb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2940101110 0x7f29401964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.420+0000 7f29485fb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2940101ab0 0x7f2940196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.420+0000 7f29485fb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2940107010 0x7f294019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.420+0000 7f29485fb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2940110030 con 0x7f2940107010 2024-04-21T08:48:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.420+0000 7f29485fb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f294010feb0 con 0x7f2940101110 2024-04-21T08:48:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.420+0000 7f29485fb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29401101b0 con 0x7f2940101ab0 2024-04-21T08:48:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.420+0000 7f2946b98640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2940107010 0x7f294019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.420+0000 7f2946b98640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2940107010 0x7f294019ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47776/0 (socket says 172.21.15.43:47776) 2024-04-21T08:48:59.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.420+0000 7f2946b98640 1 -- 172.21.15.43:0/494020366 learned_addr learned my addr 172.21.15.43:0/494020366 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:59.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.420+0000 7f2945b96640 1 --2- 172.21.15.43:0/494020366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2940101ab0 0x7f2940196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.420+0000 7f2946397640 1 --2- 172.21.15.43:0/494020366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2940101110 0x7f29401964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.421+0000 7f2946b98640 1 -- 172.21.15.43:0/494020366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2940101ab0 msgr2=0x7f2940196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.421+0000 7f2946b98640 1 --2- 172.21.15.43:0/494020366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2940101ab0 0x7f2940196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.421+0000 7f2946b98640 1 -- 172.21.15.43:0/494020366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2940101110 msgr2=0x7f29401964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.421+0000 7f2946b98640 1 --2- 172.21.15.43:0/494020366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2940101110 0x7f29401964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.421+0000 7f2946b98640 1 -- 172.21.15.43:0/494020366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294019b490 con 0x7f2940107010 2024-04-21T08:48:59.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.421+0000 7f2946b98640 1 --2- 172.21.15.43:0/494020366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2940107010 0x7f294019ad60 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f2938009cc0 tx=0x7f29380345e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.421+0000 7f29377fe640 1 -- 172.21.15.43:0/494020366 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293803b040 con 0x7f2940107010 2024-04-21T08:48:59.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.421+0000 7f29485fb640 1 -- 172.21.15.43:0/494020366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294019b720 con 0x7f2940107010 2024-04-21T08:48:59.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.421+0000 7f29377fe640 1 -- 172.21.15.43:0/494020366 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2938046070 con 0x7f2940107010 2024-04-21T08:48:59.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.421+0000 7f29377fe640 1 -- 172.21.15.43:0/494020366 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29380042a0 con 0x7f2940107010 2024-04-21T08:48:59.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.421+0000 7f29485fb640 1 -- 172.21.15.43:0/494020366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29401a3010 con 0x7f2940107010 2024-04-21T08:48:59.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.422+0000 7f29485fb640 1 -- 172.21.15.43:0/494020366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2940069230 con 0x7f2940107010 2024-04-21T08:48:59.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.423+0000 7f29377fe640 1 -- 172.21.15.43:0/494020366 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2938036440 con 0x7f2940107010 2024-04-21T08:48:59.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.424+0000 7f29377fe640 1 --2- 172.21.15.43:0/494020366 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f292c0a3d20 0x7f292c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.424+0000 7f29377fe640 1 -- 172.21.15.43:0/494020366 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f29380f51f0 con 0x7f2940107010 2024-04-21T08:48:59.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.424+0000 7f2946397640 1 --2- 172.21.15.43:0/494020366 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f292c0a3d20 0x7f292c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.424+0000 7f2946397640 1 --2- 172.21.15.43:0/494020366 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f292c0a3d20 0x7f292c0a61e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f293c005fd0 tx=0x7f293c005e20 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.428+0000 7f29377fe640 1 -- 172.21.15.43:0/494020366 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f29380c1580 con 0x7f2940107010 2024-04-21T08:48:59.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.655+0000 7f29485fb640 1 -- 172.21.15.43:0/494020366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29401092a0 con 0x7f2940107010 2024-04-21T08:48:59.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.656+0000 7f29377fe640 1 -- 172.21.15.43:0/494020366 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11935 (secure 0 0 0) 0x7f29380c0e80 con 0x7f2940107010 2024-04-21T08:48:59.656 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:48:59.656 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:46:17.283806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27556":{"gid":27556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/229985237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":229985237},{"type":"v1","addr":"172.21.15.175:6837","nonce":229985237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24425,24412,24487,24527,24491]},"id":1}]} 2024-04-21T08:48:59.656 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-04-21T08:48:59.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.658+0000 7f29485fb640 1 -- 172.21.15.43:0/494020366 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f292c0a3d20 msgr2=0x7f292c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.658+0000 7f29485fb640 1 --2- 172.21.15.43:0/494020366 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f292c0a3d20 0x7f292c0a61e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f293c005fd0 tx=0x7f293c005e20 comp rx=0 tx=0).stop 2024-04-21T08:48:59.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.658+0000 7f29485fb640 1 -- 172.21.15.43:0/494020366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2940107010 msgr2=0x7f294019ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.658+0000 7f29485fb640 1 --2- 172.21.15.43:0/494020366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2940107010 0x7f294019ad60 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f2938009cc0 tx=0x7f29380345e0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.658+0000 7f29485fb640 1 -- 172.21.15.43:0/494020366 shutdown_connections 2024-04-21T08:48:59.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.658+0000 7f29485fb640 1 --2- 172.21.15.43:0/494020366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2940107010 0x7f294019ad60 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.658+0000 7f29485fb640 1 --2- 172.21.15.43:0/494020366 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f292c0a3d20 0x7f292c0a61e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.658+0000 7f29485fb640 1 --2- 172.21.15.43:0/494020366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2940101ab0 0x7f2940196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.658+0000 7f29485fb640 1 --2- 172.21.15.43:0/494020366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2940101110 0x7f29401964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.658+0000 7f29485fb640 1 -- 172.21.15.43:0/494020366 >> 172.21.15.43:0/494020366 conn(0x7f2940077e30 msgr2=0x7f29400fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.658+0000 7f29485fb640 1 -- 172.21.15.43:0/494020366 shutdown_connections 2024-04-21T08:48:59.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.658+0000 7f29485fb640 1 -- 172.21.15.43:0/494020366 wait complete. 2024-04-21T08:48:59.669 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:48:59.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.774+0000 7f931a7f4640 1 -- 172.21.15.43:0/3699043293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f931406b1f0 msgr2=0x7f9314106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.774+0000 7f931a7f4640 1 --2- 172.21.15.43:0/3699043293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f931406b1f0 0x7f9314106b90 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f9308009900 tx=0x7f930802bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.774+0000 7f931a7f4640 1 -- 172.21.15.43:0/3699043293 shutdown_connections 2024-04-21T08:48:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.774+0000 7f931a7f4640 1 --2- 172.21.15.43:0/3699043293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9314107270 0x7f931410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.774+0000 7f931a7f4640 1 --2- 172.21.15.43:0/3699043293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f931406b1f0 0x7f9314106b90 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.774+0000 7f931a7f4640 1 --2- 172.21.15.43:0/3699043293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f931406a840 0x7f931406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.774+0000 7f931a7f4640 1 -- 172.21.15.43:0/3699043293 >> 172.21.15.43:0/3699043293 conn(0x7f9314077eb0 msgr2=0x7f93140782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.774+0000 7f931a7f4640 1 -- 172.21.15.43:0/3699043293 shutdown_connections 2024-04-21T08:48:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.774+0000 7f931a7f4640 1 -- 172.21.15.43:0/3699043293 wait complete. 2024-04-21T08:48:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.774+0000 7f931a7f4640 1 Processor -- start 2024-04-21T08:48:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.774+0000 7f931a7f4640 1 -- start start 2024-04-21T08:48:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.775+0000 7f931a7f4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f931406a840 0x7f9314198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.775+0000 7f931a7f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9314107270 0x7f9314198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.775+0000 7f931a7f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f931419cfd0 0x7f931419d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.775+0000 7f931a7f4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f93141121b0 con 0x7f931419cfd0 2024-04-21T08:48:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.775+0000 7f931a7f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9314112030 con 0x7f9314107270 2024-04-21T08:48:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.775+0000 7f931a7f4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9314112330 con 0x7f931406a840 2024-04-21T08:48:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.775+0000 7f9318d91640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f931419cfd0 0x7f931419d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.775+0000 7f9318d91640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f931419cfd0 0x7f931419d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47798/0 (socket says 172.21.15.43:47798) 2024-04-21T08:48:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.775+0000 7f9318d91640 1 -- 172.21.15.43:0/329376389 learned_addr learned my addr 172.21.15.43:0/329376389 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.775+0000 7f93137fe640 1 --2- 172.21.15.43:0/329376389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9314107270 0x7f9314198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.775+0000 7f9313fff640 1 --2- 172.21.15.43:0/329376389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f931406a840 0x7f9314198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.775+0000 7f9318d91640 1 -- 172.21.15.43:0/329376389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f931406a840 msgr2=0x7f9314198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.775+0000 7f9318d91640 1 --2- 172.21.15.43:0/329376389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f931406a840 0x7f9314198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.775+0000 7f9318d91640 1 -- 172.21.15.43:0/329376389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9314107270 msgr2=0x7f9314198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.775+0000 7f9318d91640 1 --2- 172.21.15.43:0/329376389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9314107270 0x7f9314198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.775+0000 7f9318d91640 1 -- 172.21.15.43:0/329376389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931419dac0 con 0x7f931419cfd0 2024-04-21T08:48:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.775+0000 7f9318d91640 1 --2- 172.21.15.43:0/329376389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f931419cfd0 0x7f931419d480 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f930c00eaa0 tx=0x7f930c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.775+0000 7f93117fa640 1 -- 172.21.15.43:0/329376389 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c00cdb0 con 0x7f931419cfd0 2024-04-21T08:48:59.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.776+0000 7f93117fa640 1 -- 172.21.15.43:0/329376389 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f930c005020 con 0x7f931419cfd0 2024-04-21T08:48:59.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.776+0000 7f93117fa640 1 -- 172.21.15.43:0/329376389 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c0116d0 con 0x7f931419cfd0 2024-04-21T08:48:59.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.776+0000 7f931a7f4640 1 -- 172.21.15.43:0/329376389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f931419ddb0 con 0x7f931419cfd0 2024-04-21T08:48:59.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.776+0000 7f931a7f4640 1 -- 172.21.15.43:0/329376389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93141a5520 con 0x7f931419cfd0 2024-04-21T08:48:59.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.778+0000 7f93117fa640 1 -- 172.21.15.43:0/329376389 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f930c0051d0 con 0x7f931419cfd0 2024-04-21T08:48:59.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.778+0000 7f931d3e6740 1 Processor -- start 2024-04-21T08:48:59.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.778+0000 7f931d3e6740 1 -- start start 2024-04-21T08:48:59.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.778+0000 7f931d3e6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd5103b0 0x55fcbd510790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.778+0000 7f931d3e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcbd510d60 0x55fcbd519890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.778+0000 7f931d3e6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcbd519e60 0x55fcbd51a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.778+0000 7f931d3e6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fcbd4751f0 con 0x55fcbd5103b0 2024-04-21T08:48:59.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.778+0000 7f931d3e6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fcbd4124b0 con 0x55fcbd519e60 2024-04-21T08:48:59.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.778+0000 7f931d3e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fcbd4711b0 con 0x55fcbd510d60 2024-04-21T08:48:59.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.779+0000 7f93137fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd5103b0 0x55fcbd510790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.779+0000 7f93117fa640 1 --2- 172.21.15.43:0/329376389 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f92fc0a3d20 0x7f92fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.779+0000 7f93137fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd5103b0 0x55fcbd510790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47814/0 (socket says 172.21.15.43:47814) 2024-04-21T08:48:59.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.779+0000 7f93137fe640 1 -- 172.21.15.43:0/2847114270 learned_addr learned my addr 172.21.15.43:0/2847114270 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:59.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.779+0000 7f9318d91640 1 --2- 172.21.15.43:0/329376389 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f92fc0a3d20 0x7f92fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.779+0000 7f9313fff640 1 --2- 172.21.15.43:0/2847114270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcbd510d60 0x55fcbd519890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.779+0000 7f93117fa640 1 -- 172.21.15.43:0/329376389 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f930c0d0dc0 con 0x7f931419cfd0 2024-04-21T08:48:59.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.779+0000 7f93137fe640 1 -- 172.21.15.43:0/2847114270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcbd510d60 msgr2=0x55fcbd519890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.779+0000 7f93137fe640 1 --2- 172.21.15.43:0/2847114270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcbd510d60 0x55fcbd519890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.779+0000 7f93137fe640 1 -- 172.21.15.43:0/2847114270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcbd519e60 msgr2=0x55fcbd51a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.779+0000 7f93137fe640 1 --2- 172.21.15.43:0/2847114270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcbd519e60 0x55fcbd51a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.779+0000 7f93137fe640 1 -- 172.21.15.43:0/2847114270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcbd4041b0 con 0x55fcbd5103b0 2024-04-21T08:48:59.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.779+0000 7f93137fe640 1 --2- 172.21.15.43:0/2847114270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd5103b0 0x55fcbd510790 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f931406a420 tx=0x7f930802c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9301cdb6eacc7e3c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.779+0000 7f9318d91640 1 --2- 172.21.15.43:0/329376389 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f92fc0a3d20 0x7f92fc0a61e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f930c0ce1a0 tx=0x7f930c005890 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.779+0000 7f92fa7fc640 1 -- 172.21.15.43:0/2847114270 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308044070 con 0x55fcbd5103b0 2024-04-21T08:48:59.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.779+0000 7f92fa7fc640 1 -- 172.21.15.43:0/2847114270 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f930802c7e0 con 0x55fcbd5103b0 2024-04-21T08:48:59.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.779+0000 7f92fa7fc640 1 -- 172.21.15.43:0/2847114270 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93080046c0 con 0x55fcbd5103b0 2024-04-21T08:48:59.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.780+0000 7f931d3e6740 1 -- 172.21.15.43:0/2847114270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd5103b0 msgr2=0x55fcbd510790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.780+0000 7f931d3e6740 1 --2- 172.21.15.43:0/2847114270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd5103b0 0x55fcbd510790 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f931406a420 tx=0x7f930802c6c0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.780+0000 7f931d3e6740 1 -- 172.21.15.43:0/2847114270 shutdown_connections 2024-04-21T08:48:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.780+0000 7f931d3e6740 1 --2- 172.21.15.43:0/2847114270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcbd519e60 0x55fcbd51a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.780+0000 7f931d3e6740 1 --2- 172.21.15.43:0/2847114270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcbd510d60 0x55fcbd519890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.780+0000 7f931d3e6740 1 --2- 172.21.15.43:0/2847114270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd5103b0 0x55fcbd510790 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.780+0000 7f931d3e6740 1 -- 172.21.15.43:0/2847114270 >> 172.21.15.43:0/2847114270 conn(0x55fcbd50aba0 msgr2=0x55fcbd50af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.780+0000 7f931d3e6740 1 -- 172.21.15.43:0/2847114270 shutdown_connections 2024-04-21T08:48:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.780+0000 7f931d3e6740 1 -- 172.21.15.43:0/2847114270 wait complete. 2024-04-21T08:48:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.780+0000 7f931d3e6740 1 -- start start 2024-04-21T08:48:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.781+0000 7f931d3e6740 1 Processor -- start 2024-04-21T08:48:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.781+0000 7f931d3e6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd5103b0 0x55fcbd520320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.781+0000 7f931d3e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcbd510d60 0x55fcbd51d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.781+0000 7f931d3e6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcbd519e60 0x55fcbd51dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.781+0000 7f931d3e6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fcbd520d40 con 0x55fcbd5103b0 2024-04-21T08:48:59.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.781+0000 7f931d3e6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fcbd42e500 con 0x55fcbd519e60 2024-04-21T08:48:59.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.781+0000 7f931d3e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fcbd520ec0 con 0x55fcbd510d60 2024-04-21T08:48:59.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.781+0000 7f931d3e6740 10 client.0 _ll_register_callbacks cb 0x55fcbd3fcb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:59.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.781+0000 7f93137fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd5103b0 0x55fcbd520320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.781+0000 7f93137fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd5103b0 0x55fcbd520320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47828/0 (socket says 172.21.15.43:47828) 2024-04-21T08:48:59.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.781+0000 7f93137fe640 1 -- 172.21.15.43:0/3717191802 learned_addr learned my addr 172.21.15.43:0/3717191802 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:59.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.781+0000 7f9313fff640 1 --2- 172.21.15.43:0/3717191802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcbd510d60 0x55fcbd51d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.781+0000 7f93137fe640 1 --2- 172.21.15.43:0/3717191802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcbd519e60 0x55fcbd51dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.781+0000 7f93137fe640 1 -- 172.21.15.43:0/3717191802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcbd510d60 msgr2=0x55fcbd51d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.781+0000 7f93137fe640 1 --2- 172.21.15.43:0/3717191802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcbd510d60 0x55fcbd51d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.781+0000 7f93137fe640 1 -- 172.21.15.43:0/3717191802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcbd519e60 msgr2=0x55fcbd51dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.781+0000 7f93137fe640 1 --2- 172.21.15.43:0/3717191802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcbd519e60 0x55fcbd51dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.781+0000 7f93137fe640 1 -- 172.21.15.43:0/3717191802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcbd1873f0 con 0x55fcbd5103b0 2024-04-21T08:48:59.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.781+0000 7f93137fe640 1 --2- 172.21.15.43:0/3717191802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd5103b0 0x55fcbd520320 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f9308009430 tx=0x7f93080052c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb28cf07fbc0c413 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.781+0000 7f92f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:59.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.782+0000 7f92f37fe640 1 -- 172.21.15.43:0/3717191802 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308056060 con 0x55fcbd5103b0 2024-04-21T08:48:59.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.782+0000 7f92f37fe640 1 -- 172.21.15.43:0/3717191802 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f93080035f0 con 0x55fcbd5103b0 2024-04-21T08:48:59.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.782+0000 7f931d3e6740 1 -- 172.21.15.43:0/3717191802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f930802c7e0 con 0x55fcbd5103b0 2024-04-21T08:48:59.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.782+0000 7f92f37fe640 1 -- 172.21.15.43:0/3717191802 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f930805d050 con 0x55fcbd5103b0 2024-04-21T08:48:59.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.782+0000 7f931d3e6740 10 client.20358 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:59.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.782+0000 7f931d3e6740 1 -- 172.21.15.43:0/3717191802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fcbd2517d0 con 0x55fcbd5103b0 2024-04-21T08:48:59.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.782+0000 7f92f37fe640 1 -- 172.21.15.43:0/3717191802 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f930802c7e0 con 0x55fcbd5103b0 2024-04-21T08:48:59.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.782+0000 7f92f37fe640 1 -- 172.21.15.43:0/3717191802 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f93080497b0 con 0x55fcbd5103b0 2024-04-21T08:48:59.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.782+0000 7f931d3e6740 10 client.20358 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:59.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.782+0000 7f931d3e6740 10 client.20358 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:48:59.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.782+0000 7f931d3e6740 20 client.20358 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:59.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.782+0000 7f931d3e6740 1 --2- 172.21.15.43:0/3717191802 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55fcbd52def0 0x55fcbd5302e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.782+0000 7f931d3e6740 4 client.20358 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:59.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.782+0000 7f931d3e6740 1 -- 172.21.15.43:0/3717191802 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcbd414930 con 0x55fcbd52def0 2024-04-21T08:48:59.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.782+0000 7f9313fff640 1 --2- 172.21.15.43:0/3717191802 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55fcbd52def0 0x55fcbd5302e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.783+0000 7f92f37fe640 10 client.20358 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:48:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.783+0000 7f9313fff640 1 --2- 172.21.15.43:0/3717191802 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55fcbd52def0 0x55fcbd5302e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5fdc4df5f194d409 server_cookie=b0576274b0a6d53f in_seq=0 out_seq=0 2024-04-21T08:48:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.784+0000 7f92f37fe640 1 -- 172.21.15.43:0/3717191802 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fcbd414930 con 0x55fcbd52def0 2024-04-21T08:48:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.784+0000 7f92f37fe640 10 client.20358 handle_command_reply: tid=0 2024-04-21T08:48:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.784+0000 7f931d3e6740 1 client.20358 shutdown 2024-04-21T08:48:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.784+0000 7f931d3e6740 1 -- 172.21.15.43:0/3717191802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd5103b0 msgr2=0x55fcbd520320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.784+0000 7f931d3e6740 1 --2- 172.21.15.43:0/3717191802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd5103b0 0x55fcbd520320 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f9308009430 tx=0x7f93080052c0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.784+0000 7f931d3e6740 1 -- 172.21.15.43:0/3717191802 shutdown_connections 2024-04-21T08:48:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.784+0000 7f931d3e6740 1 --2- 172.21.15.43:0/3717191802 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55fcbd52def0 0x55fcbd5302e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.784+0000 7f931d3e6740 1 --2- 172.21.15.43:0/3717191802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcbd519e60 0x55fcbd51dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.784+0000 7f92f37fe640 0 client.20358 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:48:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.784+0000 7f931d3e6740 1 --2- 172.21.15.43:0/3717191802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcbd510d60 0x55fcbd51d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.784+0000 7f931d3e6740 1 --2- 172.21.15.43:0/3717191802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd5103b0 0x55fcbd520320 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.784+0000 7f931d3e6740 1 -- 172.21.15.43:0/3717191802 >> 172.21.15.43:0/3717191802 conn(0x55fcbd50aba0 msgr2=0x55fcbd50c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.784+0000 7f931d3e6740 1 -- 172.21.15.43:0/3717191802 shutdown_connections 2024-04-21T08:48:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.784+0000 7f931d3e6740 1 -- 172.21.15.43:0/3717191802 wait complete. 2024-04-21T08:48:59.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.784+0000 7f931d3e6740 20 client.20358 trim_cache size 0 max 16384 2024-04-21T08:48:59.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.797+0000 7f931d3e6740 1 Processor -- start 2024-04-21T08:48:59.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.797+0000 7f931d3e6740 1 -- start start 2024-04-21T08:48:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.798+0000 7f931d3e6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd5103b0 0x55fcbd51b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.798+0000 7f931d3e6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcbd510d60 0x55fcbd51fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.798+0000 7f931d3e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcbd519e60 0x55fcbd520320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.798+0000 7f931d3e6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f93141ad1e0 con 0x55fcbd5103b0 2024-04-21T08:48:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.798+0000 7f931d3e6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f93141aed20 con 0x55fcbd510d60 2024-04-21T08:48:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.798+0000 7f931d3e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f93141ad8b0 con 0x55fcbd519e60 2024-04-21T08:48:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.798+0000 7f93137fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd5103b0 0x55fcbd51b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.798+0000 7f93137fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd5103b0 0x55fcbd51b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47842/0 (socket says 172.21.15.43:47842) 2024-04-21T08:48:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.798+0000 7f93137fe640 1 -- 172.21.15.43:0/4030539746 learned_addr learned my addr 172.21.15.43:0/4030539746 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.798+0000 7f9313fff640 1 --2- 172.21.15.43:0/4030539746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcbd510d60 0x55fcbd51fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.798+0000 7f93137fe640 1 -- 172.21.15.43:0/4030539746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcbd519e60 msgr2=0x55fcbd520320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.798+0000 7f93137fe640 1 --2- 172.21.15.43:0/4030539746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcbd519e60 0x55fcbd520320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.798+0000 7f93137fe640 1 -- 172.21.15.43:0/4030539746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcbd510d60 msgr2=0x55fcbd51fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.798+0000 7f93137fe640 1 --2- 172.21.15.43:0/4030539746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcbd510d60 0x55fcbd51fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.798+0000 7f93137fe640 1 -- 172.21.15.43:0/4030539746 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcbd51e080 con 0x55fcbd5103b0 2024-04-21T08:48:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.798+0000 7f93137fe640 1 --2- 172.21.15.43:0/4030539746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd5103b0 0x55fcbd51b250 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f9308005ec0 tx=0x7f93080058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd12ba243cad4b2f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.798+0000 7f92f9ffb640 1 -- 172.21.15.43:0/4030539746 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308056040 con 0x55fcbd5103b0 2024-04-21T08:48:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.798+0000 7f92f9ffb640 1 -- 172.21.15.43:0/4030539746 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9308049da0 con 0x55fcbd5103b0 2024-04-21T08:48:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.798+0000 7f92f9ffb640 1 -- 172.21.15.43:0/4030539746 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308044040 con 0x55fcbd5103b0 2024-04-21T08:48:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.799+0000 7f931d3e6740 1 -- 172.21.15.43:0/4030539746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd5103b0 msgr2=0x55fcbd51b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.799+0000 7f931d3e6740 1 --2- 172.21.15.43:0/4030539746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd5103b0 0x55fcbd51b250 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f9308005ec0 tx=0x7f93080058b0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.799+0000 7f931d3e6740 1 -- 172.21.15.43:0/4030539746 shutdown_connections 2024-04-21T08:48:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.799+0000 7f931d3e6740 1 --2- 172.21.15.43:0/4030539746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcbd519e60 0x55fcbd520320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.799+0000 7f931d3e6740 1 --2- 172.21.15.43:0/4030539746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcbd510d60 0x55fcbd51fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.799+0000 7f931d3e6740 1 --2- 172.21.15.43:0/4030539746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd5103b0 0x55fcbd51b250 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.799+0000 7f931d3e6740 1 -- 172.21.15.43:0/4030539746 >> 172.21.15.43:0/4030539746 conn(0x55fcbd50aba0 msgr2=0x55fcbd5299e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.799+0000 7f931d3e6740 1 -- 172.21.15.43:0/4030539746 shutdown_connections 2024-04-21T08:48:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.799+0000 7f931d3e6740 1 -- 172.21.15.43:0/4030539746 wait complete. 2024-04-21T08:48:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.799+0000 7f931d3e6740 1 -- start start 2024-04-21T08:48:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.799+0000 7f931d3e6740 1 Processor -- start 2024-04-21T08:48:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.800+0000 7f931d3e6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcbd5103b0 0x55fcbd5337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.800+0000 7f931d3e6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd510d60 0x55fcbd530b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.800+0000 7f931d3e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcbd519e60 0x55fcbd531190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.800+0000 7f931d3e6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f93141ae650 con 0x55fcbd510d60 2024-04-21T08:48:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.800+0000 7f931d3e6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f93141adf80 con 0x55fcbd5103b0 2024-04-21T08:48:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.800+0000 7f931d3e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f93141ab1e0 con 0x55fcbd519e60 2024-04-21T08:48:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.800+0000 7f931d3e6740 10 client.0 _ll_register_callbacks cb 0x7f93080056f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.800+0000 7f9313fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd510d60 0x55fcbd530b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.800+0000 7f9313fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd510d60 0x55fcbd530b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47846/0 (socket says 172.21.15.43:47846) 2024-04-21T08:48:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.800+0000 7f9313fff640 1 -- 172.21.15.43:0/229738095 learned_addr learned my addr 172.21.15.43:0/229738095 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.800+0000 7f93137fe640 1 --2- 172.21.15.43:0/229738095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcbd5103b0 0x55fcbd5337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.800+0000 7f93137fe640 1 --2- 172.21.15.43:0/229738095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcbd519e60 0x55fcbd531190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.800+0000 7f9313fff640 1 -- 172.21.15.43:0/229738095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcbd519e60 msgr2=0x55fcbd531190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.800+0000 7f9313fff640 1 --2- 172.21.15.43:0/229738095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcbd519e60 0x55fcbd531190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.800+0000 7f9313fff640 1 -- 172.21.15.43:0/229738095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcbd5103b0 msgr2=0x55fcbd5337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.800+0000 7f9313fff640 1 --2- 172.21.15.43:0/229738095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcbd5103b0 0x55fcbd5337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.800+0000 7f9313fff640 1 -- 172.21.15.43:0/229738095 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcbd5319c0 con 0x55fcbd510d60 2024-04-21T08:48:59.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.800+0000 7f93137fe640 1 --2- 172.21.15.43:0/229738095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcbd5103b0 0x55fcbd5337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:59.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.800+0000 7f9313fff640 1 --2- 172.21.15.43:0/229738095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd510d60 0x55fcbd530b80 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f93040058c0 tx=0x7f93040118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74f6cf4c1b9f91b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.800+0000 7f92f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:59.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.800+0000 7f92f37fe640 1 -- 172.21.15.43:0/229738095 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930401d910 con 0x55fcbd510d60 2024-04-21T08:48:59.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.801+0000 7f92f37fe640 1 -- 172.21.15.43:0/229738095 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f9304005020 con 0x55fcbd510d60 2024-04-21T08:48:59.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.801+0000 7f931d3e6740 1 -- 172.21.15.43:0/229738095 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9308049da0 con 0x55fcbd510d60 2024-04-21T08:48:59.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.801+0000 7f92f37fe640 1 -- 172.21.15.43:0/229738095 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f930402d050 con 0x55fcbd510d60 2024-04-21T08:48:59.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.801+0000 7f931d3e6740 10 client.20364 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:59.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.801+0000 7f931d3e6740 1 -- 172.21.15.43:0/229738095 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fcbd594290 con 0x55fcbd510d60 2024-04-21T08:48:59.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.801+0000 7f92f37fe640 1 -- 172.21.15.43:0/229738095 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9308049da0 con 0x55fcbd510d60 2024-04-21T08:48:59.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.801+0000 7f92f37fe640 1 -- 172.21.15.43:0/229738095 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9304025400 con 0x55fcbd510d60 2024-04-21T08:48:59.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.801+0000 7f931d3e6740 10 client.20364 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:59.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.801+0000 7f931d3e6740 10 client.20364 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:48:59.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.801+0000 7f931d3e6740 20 client.20364 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:59.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.801+0000 7f931d3e6740 1 --2- 172.21.15.43:0/229738095 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55fcbd594a90 0x55fcbd596eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.801+0000 7f931d3e6740 4 client.20364 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:59.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.801+0000 7f931d3e6740 1 -- 172.21.15.43:0/229738095 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fcbd482c10 con 0x55fcbd594a90 2024-04-21T08:48:59.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.802+0000 7f93137fe640 1 --2- 172.21.15.43:0/229738095 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55fcbd594a90 0x55fcbd596eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.802+0000 7f93137fe640 1 --2- 172.21.15.43:0/229738095 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55fcbd594a90 0x55fcbd596eb0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2f115f31febcce08 server_cookie=3cc78f43affff74d in_seq=0 out_seq=0 2024-04-21T08:48:59.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.802+0000 7f92f37fe640 10 client.20364 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:48:59.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.802+0000 7f92f37fe640 1 -- 172.21.15.43:0/229738095 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fcbd482c10 con 0x55fcbd594a90 2024-04-21T08:48:59.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.802+0000 7f92f37fe640 10 client.20364 handle_command_reply: tid=0 2024-04-21T08:48:59.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.802+0000 7f931d3e6740 1 client.20364 shutdown 2024-04-21T08:48:59.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.802+0000 7f931d3e6740 1 -- 172.21.15.43:0/229738095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd510d60 msgr2=0x55fcbd530b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.802+0000 7f931d3e6740 1 --2- 172.21.15.43:0/229738095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd510d60 0x55fcbd530b80 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f93040058c0 tx=0x7f93040118d0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.802+0000 7f931d3e6740 1 -- 172.21.15.43:0/229738095 shutdown_connections 2024-04-21T08:48:59.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.802+0000 7f931d3e6740 1 --2- 172.21.15.43:0/229738095 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55fcbd594a90 0x55fcbd596eb0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.802+0000 7f931d3e6740 1 --2- 172.21.15.43:0/229738095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fcbd519e60 0x55fcbd531190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.802+0000 7f931d3e6740 1 --2- 172.21.15.43:0/229738095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcbd510d60 0x55fcbd530b80 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.802+0000 7f92f37fe640 0 client.20364 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:48:59.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.802+0000 7f931d3e6740 1 --2- 172.21.15.43:0/229738095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcbd5103b0 0x55fcbd5337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.802+0000 7f931d3e6740 1 -- 172.21.15.43:0/229738095 >> 172.21.15.43:0/229738095 conn(0x55fcbd50aba0 msgr2=0x55fcbd50bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.803+0000 7f931d3e6740 1 -- 172.21.15.43:0/229738095 shutdown_connections 2024-04-21T08:48:59.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.803+0000 7f931d3e6740 1 -- 172.21.15.43:0/229738095 wait complete. 2024-04-21T08:48:59.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.803+0000 7f931d3e6740 20 client.20364 trim_cache size 0 max 16384 2024-04-21T08:48:59.809 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:48:59.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.803+0000 7f931a7f4640 1 -- 172.21.15.43:0/329376389 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f92fc0a3d20 msgr2=0x7f92fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.803+0000 7f931a7f4640 1 --2- 172.21.15.43:0/329376389 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f92fc0a3d20 0x7f92fc0a61e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f930c0ce1a0 tx=0x7f930c005890 comp rx=0 tx=0).stop 2024-04-21T08:48:59.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.803+0000 7f931a7f4640 1 -- 172.21.15.43:0/329376389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f931419cfd0 msgr2=0x7f931419d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.803+0000 7f931a7f4640 1 --2- 172.21.15.43:0/329376389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f931419cfd0 0x7f931419d480 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f930c00eaa0 tx=0x7f930c00ef60 comp rx=0 tx=0).stop 2024-04-21T08:48:59.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.803+0000 7f931a7f4640 1 -- 172.21.15.43:0/329376389 shutdown_connections 2024-04-21T08:48:59.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.803+0000 7f931a7f4640 1 --2- 172.21.15.43:0/329376389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f931419cfd0 0x7f931419d480 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.803+0000 7f931a7f4640 1 --2- 172.21.15.43:0/329376389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9314107270 0x7f9314198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.803+0000 7f931a7f4640 1 --2- 172.21.15.43:0/329376389 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f92fc0a3d20 0x7f92fc0a61e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.803+0000 7f931a7f4640 1 --2- 172.21.15.43:0/329376389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f931406a840 0x7f9314198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.803+0000 7f931a7f4640 1 -- 172.21.15.43:0/329376389 >> 172.21.15.43:0/329376389 conn(0x7f9314077eb0 msgr2=0x7f9314108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.803+0000 7f931a7f4640 1 -- 172.21.15.43:0/329376389 shutdown_connections 2024-04-21T08:48:59.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.803+0000 7f931a7f4640 1 -- 172.21.15.43:0/329376389 wait complete. 2024-04-21T08:48:59.814 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:48:59.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.918+0000 7f636eb75640 1 -- 172.21.15.43:0/5227641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6368106170 msgr2=0x7f636810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.918+0000 7f636eb75640 1 --2- 172.21.15.43:0/5227641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6368106170 0x7f636810dd40 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f6360009ca0 tx=0x7f636002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.918+0000 7f636eb75640 1 -- 172.21.15.43:0/5227641 shutdown_connections 2024-04-21T08:48:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.918+0000 7f636eb75640 1 --2- 172.21.15.43:0/5227641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6368106170 0x7f636810dd40 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.918+0000 7f636eb75640 1 --2- 172.21.15.43:0/5227641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6368101a50 0x7f6368105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.918+0000 7f636eb75640 1 --2- 172.21.15.43:0/5227641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63681010a0 0x7f6368101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.918+0000 7f636eb75640 1 -- 172.21.15.43:0/5227641 >> 172.21.15.43:0/5227641 conn(0x7f6368077eb0 msgr2=0x7f63680782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.918+0000 7f636eb75640 1 -- 172.21.15.43:0/5227641 shutdown_connections 2024-04-21T08:48:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.918+0000 7f636eb75640 1 -- 172.21.15.43:0/5227641 wait complete. 2024-04-21T08:48:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.919+0000 7f636eb75640 1 Processor -- start 2024-04-21T08:48:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.919+0000 7f636eb75640 1 -- start start 2024-04-21T08:48:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.919+0000 7f636eb75640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63681010a0 0x7f63681994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.919+0000 7f636eb75640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6368106170 0x7f6368199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.919+0000 7f636eb75640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6368199fe0 0x7f63681a4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.919+0000 7f636eb75640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f63681110c0 con 0x7f6368106170 2024-04-21T08:48:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.919+0000 7f636eb75640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6368110f40 con 0x7f63681010a0 2024-04-21T08:48:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.919+0000 7f636eb75640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6368111240 con 0x7f6368199fe0 2024-04-21T08:48:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.919+0000 7f6367fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6368106170 0x7f6368199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.919+0000 7f6367fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6368106170 0x7f6368199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47864/0 (socket says 172.21.15.43:47864) 2024-04-21T08:48:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.919+0000 7f6367fff640 1 -- 172.21.15.43:0/1731025890 learned_addr learned my addr 172.21.15.43:0/1731025890 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.919+0000 7f636c911640 1 --2- 172.21.15.43:0/1731025890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63681010a0 0x7f63681994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.919+0000 7f636d112640 1 --2- 172.21.15.43:0/1731025890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6368199fe0 0x7f63681a4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.919+0000 7f6367fff640 1 -- 172.21.15.43:0/1731025890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6368199fe0 msgr2=0x7f63681a4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.919+0000 7f6367fff640 1 --2- 172.21.15.43:0/1731025890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6368199fe0 0x7f63681a4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.919+0000 7f6367fff640 1 -- 172.21.15.43:0/1731025890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63681010a0 msgr2=0x7f63681994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.919+0000 7f6367fff640 1 --2- 172.21.15.43:0/1731025890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63681010a0 0x7f63681994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.919+0000 7f6367fff640 1 -- 172.21.15.43:0/1731025890 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63681a4d60 con 0x7f6368106170 2024-04-21T08:48:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.920+0000 7f6367fff640 1 --2- 172.21.15.43:0/1731025890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6368106170 0x7f6368199a00 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f635400e9c0 tx=0x7f635400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.920+0000 7f6365ffb640 1 -- 172.21.15.43:0/1731025890 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635400cd80 con 0x7f6368106170 2024-04-21T08:48:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.920+0000 7f636eb75640 1 -- 172.21.15.43:0/1731025890 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63681a4ff0 con 0x7f6368106170 2024-04-21T08:48:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.920+0000 7f6365ffb640 1 -- 172.21.15.43:0/1731025890 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63540054d0 con 0x7f6368106170 2024-04-21T08:48:59.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.920+0000 7f6365ffb640 1 -- 172.21.15.43:0/1731025890 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354010430 con 0x7f6368106170 2024-04-21T08:48:59.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.920+0000 7f636eb75640 1 -- 172.21.15.43:0/1731025890 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63681a5510 con 0x7f6368106170 2024-04-21T08:48:59.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.922+0000 7f6365ffb640 1 -- 172.21.15.43:0/1731025890 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6354004fd0 con 0x7f6368106170 2024-04-21T08:48:59.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.922+0000 7f63713fd740 1 Processor -- start 2024-04-21T08:48:59.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.923+0000 7f63713fd740 1 -- start start 2024-04-21T08:48:59.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.923+0000 7f63713fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf85244d0 0x55fbf85248b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.923+0000 7f63713fd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbf8524e80 0x55fbf852d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.923+0000 7f63713fd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbf852da00 0x55fbf852fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.923+0000 7f63713fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fbf84891f0 con 0x55fbf85244d0 2024-04-21T08:48:59.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.923+0000 7f63713fd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fbf84264b0 con 0x55fbf852da00 2024-04-21T08:48:59.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.923+0000 7f63713fd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fbf84851b0 con 0x55fbf8524e80 2024-04-21T08:48:59.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.923+0000 7f6365ffb640 1 --2- 172.21.15.43:0/1731025890 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f63580a3d20 0x7f63580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.923+0000 7f636d112640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf85244d0 0x55fbf85248b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.923+0000 7f636d112640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf85244d0 0x55fbf85248b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47870/0 (socket says 172.21.15.43:47870) 2024-04-21T08:48:59.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.923+0000 7f636d112640 1 -- 172.21.15.43:0/1996121773 learned_addr learned my addr 172.21.15.43:0/1996121773 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:59.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.923+0000 7f636d112640 1 --2- 172.21.15.43:0/1731025890 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f63580a3d20 0x7f63580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.923+0000 7f6365ffb640 1 -- 172.21.15.43:0/1731025890 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f63540d1100 con 0x7f6368106170 2024-04-21T08:48:59.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.923+0000 7f6367fff640 1 --2- 172.21.15.43:0/1996121773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbf852da00 0x55fbf852fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.923+0000 7f636c911640 1 --2- 172.21.15.43:0/1996121773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbf8524e80 0x55fbf852d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.923+0000 7f636d112640 1 -- 172.21.15.43:0/1996121773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbf8524e80 msgr2=0x55fbf852d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.923+0000 7f636d112640 1 --2- 172.21.15.43:0/1996121773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbf8524e80 0x55fbf852d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.923+0000 7f636d112640 1 -- 172.21.15.43:0/1996121773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbf852da00 msgr2=0x55fbf852fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.923+0000 7f636d112640 1 --2- 172.21.15.43:0/1996121773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbf852da00 0x55fbf852fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.923+0000 7f636d112640 1 -- 172.21.15.43:0/1996121773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbf84181b0 con 0x55fbf85244d0 2024-04-21T08:48:59.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.924+0000 7f636d112640 1 --2- 172.21.15.43:0/1996121773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf85244d0 0x55fbf85248b0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f6360009430 tx=0x7f63600326f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab76a5a14733642 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.924+0000 7f636d112640 1 --2- 172.21.15.43:0/1731025890 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f63580a3d20 0x7f63580a61e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f636002c630 tx=0x7f6360040000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.924+0000 7f6352ffd640 1 -- 172.21.15.43:0/1996121773 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360039040 con 0x55fbf85244d0 2024-04-21T08:48:59.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.924+0000 7f6352ffd640 1 -- 172.21.15.43:0/1996121773 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6360050070 con 0x55fbf85244d0 2024-04-21T08:48:59.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.924+0000 7f6352ffd640 1 -- 172.21.15.43:0/1996121773 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360034440 con 0x55fbf85244d0 2024-04-21T08:48:59.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.925+0000 7f63713fd740 1 -- 172.21.15.43:0/1996121773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf85244d0 msgr2=0x55fbf85248b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.925+0000 7f63713fd740 1 --2- 172.21.15.43:0/1996121773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf85244d0 0x55fbf85248b0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f6360009430 tx=0x7f63600326f0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.925+0000 7f63713fd740 1 -- 172.21.15.43:0/1996121773 shutdown_connections 2024-04-21T08:48:59.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.925+0000 7f63713fd740 1 --2- 172.21.15.43:0/1996121773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbf852da00 0x55fbf852fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.925+0000 7f63713fd740 1 --2- 172.21.15.43:0/1996121773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbf8524e80 0x55fbf852d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.925+0000 7f63713fd740 1 --2- 172.21.15.43:0/1996121773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf85244d0 0x55fbf85248b0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.925+0000 7f63713fd740 1 -- 172.21.15.43:0/1996121773 >> 172.21.15.43:0/1996121773 conn(0x55fbf851eba0 msgr2=0x55fbf851ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.925+0000 7f63713fd740 1 -- 172.21.15.43:0/1996121773 shutdown_connections 2024-04-21T08:48:59.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.925+0000 7f63713fd740 1 -- 172.21.15.43:0/1996121773 wait complete. 2024-04-21T08:48:59.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.925+0000 7f63713fd740 1 -- start start 2024-04-21T08:48:59.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.925+0000 7f63713fd740 1 Processor -- start 2024-04-21T08:48:59.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.925+0000 7f63713fd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbf85244d0 0x55fbf852d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.925+0000 7f63713fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf8524e80 0x55fbf852a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.925+0000 7f63713fd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbf852da00 0x55fbf852ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.925+0000 7f63713fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fbf8530910 con 0x55fbf8524e80 2024-04-21T08:48:59.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.925+0000 7f63713fd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fbf8442500 con 0x55fbf852da00 2024-04-21T08:48:59.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.925+0000 7f63713fd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fbf8530a90 con 0x55fbf85244d0 2024-04-21T08:48:59.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.925+0000 7f63713fd740 10 client.0 _ll_register_callbacks cb 0x55fbf8410b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:59.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.926+0000 7f6367fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf8524e80 0x55fbf852a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.926+0000 7f6367fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf8524e80 0x55fbf852a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47884/0 (socket says 172.21.15.43:47884) 2024-04-21T08:48:59.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.926+0000 7f6367fff640 1 -- 172.21.15.43:0/2397724407 learned_addr learned my addr 172.21.15.43:0/2397724407 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:59.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.926+0000 7f636d112640 1 --2- 172.21.15.43:0/2397724407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbf852da00 0x55fbf852ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.926+0000 7f636c911640 1 --2- 172.21.15.43:0/2397724407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbf85244d0 0x55fbf852d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.926+0000 7f6367fff640 1 -- 172.21.15.43:0/2397724407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbf85244d0 msgr2=0x55fbf852d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.926+0000 7f6367fff640 1 --2- 172.21.15.43:0/2397724407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbf85244d0 0x55fbf852d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.926+0000 7f6367fff640 1 -- 172.21.15.43:0/2397724407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbf852da00 msgr2=0x55fbf852ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.926+0000 7f6367fff640 1 --2- 172.21.15.43:0/2397724407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbf852da00 0x55fbf852ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.926+0000 7f6367fff640 1 -- 172.21.15.43:0/2397724407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbf819b3f0 con 0x55fbf8524e80 2024-04-21T08:48:59.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.926+0000 7f636c911640 1 --2- 172.21.15.43:0/2397724407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbf85244d0 0x55fbf852d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:59.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.926+0000 7f6367fff640 1 --2- 172.21.15.43:0/2397724407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf8524e80 0x55fbf852a730 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f6354099080 tx=0x7f6354098a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8cfdd670a6523a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.926+0000 7f6347fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:59.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.926+0000 7f6347fff640 1 -- 172.21.15.43:0/2397724407 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63540b5020 con 0x55fbf8524e80 2024-04-21T08:48:59.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.926+0000 7f6347fff640 1 -- 172.21.15.43:0/2397724407 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f63540bd0b0 con 0x55fbf8524e80 2024-04-21T08:48:59.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.926+0000 7f63713fd740 1 -- 172.21.15.43:0/2397724407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6360050070 con 0x55fbf8524e80 2024-04-21T08:48:59.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.926+0000 7f6347fff640 1 -- 172.21.15.43:0/2397724407 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6354066050 con 0x55fbf8524e80 2024-04-21T08:48:59.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.926+0000 7f63713fd740 10 client.20376 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:59.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.926+0000 7f63713fd740 1 -- 172.21.15.43:0/2397724407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fbf82657d0 con 0x55fbf8524e80 2024-04-21T08:48:59.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.927+0000 7f6347fff640 1 -- 172.21.15.43:0/2397724407 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6360050070 con 0x55fbf8524e80 2024-04-21T08:48:59.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.927+0000 7f6347fff640 1 -- 172.21.15.43:0/2397724407 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f63540ce650 con 0x55fbf8524e80 2024-04-21T08:48:59.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.927+0000 7f63713fd740 10 client.20376 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:59.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.927+0000 7f63713fd740 10 client.20376 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:48:59.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.927+0000 7f63713fd740 20 client.20376 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:59.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.927+0000 7f63713fd740 1 --2- 172.21.15.43:0/2397724407 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55fbf853dbd0 0x55fbf8540000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.927+0000 7f63713fd740 4 client.20376 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:59.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.927+0000 7f63713fd740 1 -- 172.21.15.43:0/2397724407 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbf8428930 con 0x55fbf853dbd0 2024-04-21T08:48:59.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.927+0000 7f636c911640 1 --2- 172.21.15.43:0/2397724407 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55fbf853dbd0 0x55fbf8540000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.927+0000 7f636c911640 1 --2- 172.21.15.43:0/2397724407 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55fbf853dbd0 0x55fbf8540000 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=66b3ffbf4be85bcd server_cookie=711cc9346d49c2f2 in_seq=0 out_seq=0 2024-04-21T08:48:59.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.927+0000 7f6347fff640 10 client.20376 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:48:59.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.928+0000 7f6347fff640 1 -- 172.21.15.43:0/2397724407 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fbf8428930 con 0x55fbf853dbd0 2024-04-21T08:48:59.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.928+0000 7f6347fff640 10 client.20376 handle_command_reply: tid=0 2024-04-21T08:48:59.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.928+0000 7f63713fd740 1 client.20376 shutdown 2024-04-21T08:48:59.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.928+0000 7f63713fd740 1 -- 172.21.15.43:0/2397724407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf8524e80 msgr2=0x55fbf852a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.928+0000 7f63713fd740 1 --2- 172.21.15.43:0/2397724407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf8524e80 0x55fbf852a730 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f6354099080 tx=0x7f6354098a30 comp rx=0 tx=0).stop 2024-04-21T08:48:59.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.928+0000 7f63713fd740 1 -- 172.21.15.43:0/2397724407 shutdown_connections 2024-04-21T08:48:59.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.928+0000 7f63713fd740 1 --2- 172.21.15.43:0/2397724407 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55fbf853dbd0 0x55fbf8540000 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.928+0000 7f63713fd740 1 --2- 172.21.15.43:0/2397724407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbf852da00 0x55fbf852ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.928+0000 7f6347fff640 0 client.20376 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:48:59.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.928+0000 7f63713fd740 1 --2- 172.21.15.43:0/2397724407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf8524e80 0x55fbf852a730 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.928+0000 7f63713fd740 1 --2- 172.21.15.43:0/2397724407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbf85244d0 0x55fbf852d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.928+0000 7f63713fd740 1 -- 172.21.15.43:0/2397724407 >> 172.21.15.43:0/2397724407 conn(0x55fbf851eba0 msgr2=0x55fbf85208f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.928+0000 7f63713fd740 1 -- 172.21.15.43:0/2397724407 shutdown_connections 2024-04-21T08:48:59.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.928+0000 7f63713fd740 1 -- 172.21.15.43:0/2397724407 wait complete. 2024-04-21T08:48:59.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.929+0000 7f63713fd740 20 client.20376 trim_cache size 0 max 16384 2024-04-21T08:48:59.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.942+0000 7f63713fd740 1 Processor -- start 2024-04-21T08:48:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.942+0000 7f63713fd740 1 -- start start 2024-04-21T08:48:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.942+0000 7f63713fd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbf85244d0 0x55fbf8520800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.942+0000 7f63713fd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbf8524e80 0x55fbf85458d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.942+0000 7f63713fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf852da00 0x55fbf8545e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.942+0000 7f63713fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f63681af4a0 con 0x55fbf852da00 2024-04-21T08:48:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.942+0000 7f63713fd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63681b0fe0 con 0x55fbf8524e80 2024-04-21T08:48:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.942+0000 7f63713fd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f63681afb70 con 0x55fbf85244d0 2024-04-21T08:48:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.942+0000 7f636d112640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf852da00 0x55fbf8545e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.942+0000 7f636d112640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf852da00 0x55fbf8545e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47896/0 (socket says 172.21.15.43:47896) 2024-04-21T08:48:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.942+0000 7f636d112640 1 -- 172.21.15.43:0/1759463463 learned_addr learned my addr 172.21.15.43:0/1759463463 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.942+0000 7f6367fff640 1 --2- 172.21.15.43:0/1759463463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbf8524e80 0x55fbf85458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.942+0000 7f636c911640 1 --2- 172.21.15.43:0/1759463463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbf85244d0 0x55fbf8520800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.942+0000 7f636d112640 1 -- 172.21.15.43:0/1759463463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbf85244d0 msgr2=0x55fbf8520800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.942+0000 7f636d112640 1 --2- 172.21.15.43:0/1759463463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbf85244d0 0x55fbf8520800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.942+0000 7f636d112640 1 -- 172.21.15.43:0/1759463463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbf8524e80 msgr2=0x55fbf85458d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.942+0000 7f636d112640 1 --2- 172.21.15.43:0/1759463463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbf8524e80 0x55fbf85458d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.942+0000 7f636d112640 1 -- 172.21.15.43:0/1759463463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbf8528c80 con 0x55fbf852da00 2024-04-21T08:48:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.942+0000 7f636c911640 1 --2- 172.21.15.43:0/1759463463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbf85244d0 0x55fbf8520800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.942+0000 7f6367fff640 1 --2- 172.21.15.43:0/1759463463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbf8524e80 0x55fbf85458d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.943+0000 7f636d112640 1 --2- 172.21.15.43:0/1759463463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf852da00 0x55fbf8545e10 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f6360006010 tx=0x7f63600324c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f361adbcd627bb6b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.943+0000 7f63527fc640 1 -- 172.21.15.43:0/1759463463 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360039070 con 0x55fbf852da00 2024-04-21T08:48:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.943+0000 7f63527fc640 1 -- 172.21.15.43:0/1759463463 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6360050040 con 0x55fbf852da00 2024-04-21T08:48:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.943+0000 7f63527fc640 1 -- 172.21.15.43:0/1759463463 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360034440 con 0x55fbf852da00 2024-04-21T08:48:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.943+0000 7f63713fd740 1 -- 172.21.15.43:0/1759463463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf852da00 msgr2=0x55fbf8545e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.943+0000 7f63713fd740 1 --2- 172.21.15.43:0/1759463463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf852da00 0x55fbf8545e10 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f6360006010 tx=0x7f63600324c0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.943+0000 7f63713fd740 1 -- 172.21.15.43:0/1759463463 shutdown_connections 2024-04-21T08:48:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.943+0000 7f63713fd740 1 --2- 172.21.15.43:0/1759463463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf852da00 0x55fbf8545e10 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.943+0000 7f63713fd740 1 --2- 172.21.15.43:0/1759463463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbf8524e80 0x55fbf85458d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.943+0000 7f63713fd740 1 --2- 172.21.15.43:0/1759463463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbf85244d0 0x55fbf8520800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.943+0000 7f63713fd740 1 -- 172.21.15.43:0/1759463463 >> 172.21.15.43:0/1759463463 conn(0x55fbf851eba0 msgr2=0x55fbf85252b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.943+0000 7f63713fd740 1 -- 172.21.15.43:0/1759463463 shutdown_connections 2024-04-21T08:48:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.943+0000 7f63713fd740 1 -- 172.21.15.43:0/1759463463 wait complete. 2024-04-21T08:48:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.944+0000 7f63713fd740 1 -- start start 2024-04-21T08:48:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.944+0000 7f63713fd740 1 Processor -- start 2024-04-21T08:48:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.944+0000 7f63713fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf85244d0 0x55fbf8531f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.944+0000 7f63713fd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbf8524e80 0x55fbf8532490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.944+0000 7f63713fd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbf852da00 0x55fbf85a1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.944+0000 7f63713fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f63681b0910 con 0x55fbf85244d0 2024-04-21T08:48:59.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.944+0000 7f63713fd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63681b0240 con 0x55fbf8524e80 2024-04-21T08:48:59.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.944+0000 7f63713fd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f63681ad4a0 con 0x55fbf852da00 2024-04-21T08:48:59.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.944+0000 7f63713fd740 10 client.0 _ll_register_callbacks cb 0x7f6354098de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:59.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.944+0000 7f636c911640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf85244d0 0x55fbf8531f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.944+0000 7f636c911640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf85244d0 0x55fbf8531f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47904/0 (socket says 172.21.15.43:47904) 2024-04-21T08:48:59.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.944+0000 7f636c911640 1 -- 172.21.15.43:0/3304645291 learned_addr learned my addr 172.21.15.43:0/3304645291 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:48:59.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.944+0000 7f636c911640 1 -- 172.21.15.43:0/3304645291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbf852da00 msgr2=0x55fbf85a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.944+0000 7f636c911640 1 --2- 172.21.15.43:0/3304645291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbf852da00 0x55fbf85a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.944+0000 7f6367fff640 1 --2- 172.21.15.43:0/3304645291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbf8524e80 0x55fbf8532490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.944+0000 7f636c911640 1 -- 172.21.15.43:0/3304645291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbf8524e80 msgr2=0x55fbf8532490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:59.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.944+0000 7f636c911640 1 --2- 172.21.15.43:0/3304645291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbf8524e80 0x55fbf8532490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.944+0000 7f636c911640 1 -- 172.21.15.43:0/3304645291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbf85a1b10 con 0x55fbf85244d0 2024-04-21T08:48:59.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.945+0000 7f636c911640 1 --2- 172.21.15.43:0/3304645291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf85244d0 0x55fbf8531f50 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f635c005d00 tx=0x7f635c01bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837076150af13370 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:59.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.945+0000 7f6347fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:48:59.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.945+0000 7f6347fff640 1 -- 172.21.15.43:0/3304645291 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c005020 con 0x55fbf85244d0 2024-04-21T08:48:59.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.945+0000 7f6347fff640 1 -- 172.21.15.43:0/3304645291 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f635c0051c0 con 0x55fbf85244d0 2024-04-21T08:48:59.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.945+0000 7f63713fd740 1 -- 172.21.15.43:0/3304645291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6360050040 con 0x55fbf85244d0 2024-04-21T08:48:59.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.945+0000 7f6347fff640 1 -- 172.21.15.43:0/3304645291 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f635c02b050 con 0x55fbf85244d0 2024-04-21T08:48:59.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.945+0000 7f63713fd740 10 client.20382 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:59.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.945+0000 7f63713fd740 1 -- 172.21.15.43:0/3304645291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fbf85a8e60 con 0x55fbf85244d0 2024-04-21T08:48:59.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.945+0000 7f6347fff640 1 -- 172.21.15.43:0/3304645291 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6360050040 con 0x55fbf85244d0 2024-04-21T08:48:59.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.945+0000 7f6347fff640 1 -- 172.21.15.43:0/3304645291 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6354016d50 con 0x55fbf85244d0 2024-04-21T08:48:59.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.945+0000 7f63713fd740 10 client.20382 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:59.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.945+0000 7f63713fd740 10 client.20382 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:48:59.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.945+0000 7f63713fd740 20 client.20382 populate_metadata read hostname 'smithi043' 2024-04-21T08:48:59.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.945+0000 7f63713fd740 1 --2- 172.21.15.43:0/3304645291 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55fbf85a9530 0x55fbf85ab950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:59.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.945+0000 7f63713fd740 4 client.20382 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:59.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.945+0000 7f63713fd740 1 -- 172.21.15.43:0/3304645291 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fbf8496c10 con 0x55fbf85a9530 2024-04-21T08:48:59.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.946+0000 7f6367fff640 1 --2- 172.21.15.43:0/3304645291 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55fbf85a9530 0x55fbf85ab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:59.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.946+0000 7f6347fff640 10 client.20382 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:48:59.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.946+0000 7f6367fff640 1 --2- 172.21.15.43:0/3304645291 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55fbf85a9530 0x55fbf85ab950 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dcfe448cfe976aee server_cookie=603eb8ffeacde352 in_seq=0 out_seq=0 2024-04-21T08:48:59.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.946+0000 7f6347fff640 1 -- 172.21.15.43:0/3304645291 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fbf8496c10 con 0x55fbf85a9530 2024-04-21T08:48:59.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.946+0000 7f6347fff640 10 client.20382 handle_command_reply: tid=0 2024-04-21T08:48:59.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.946+0000 7f63713fd740 1 client.20382 shutdown 2024-04-21T08:48:59.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.946+0000 7f63713fd740 1 -- 172.21.15.43:0/3304645291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf85244d0 msgr2=0x55fbf8531f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.946+0000 7f63713fd740 1 --2- 172.21.15.43:0/3304645291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf85244d0 0x55fbf8531f50 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f635c005d00 tx=0x7f635c01bf90 comp rx=0 tx=0).stop 2024-04-21T08:48:59.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.946+0000 7f63713fd740 1 -- 172.21.15.43:0/3304645291 shutdown_connections 2024-04-21T08:48:59.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.946+0000 7f63713fd740 1 --2- 172.21.15.43:0/3304645291 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55fbf85a9530 0x55fbf85ab950 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.947+0000 7f63713fd740 1 --2- 172.21.15.43:0/3304645291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fbf852da00 0x55fbf85a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.947+0000 7f63713fd740 1 --2- 172.21.15.43:0/3304645291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fbf8524e80 0x55fbf8532490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.947+0000 7f6347fff640 0 client.20382 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:48:59.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.947+0000 7f63713fd740 1 --2- 172.21.15.43:0/3304645291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fbf85244d0 0x55fbf8531f50 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.947+0000 7f63713fd740 1 -- 172.21.15.43:0/3304645291 >> 172.21.15.43:0/3304645291 conn(0x55fbf851eba0 msgr2=0x55fbf8520680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.947+0000 7f63713fd740 1 -- 172.21.15.43:0/3304645291 shutdown_connections 2024-04-21T08:48:59.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.947+0000 7f63713fd740 1 -- 172.21.15.43:0/3304645291 wait complete. 2024-04-21T08:48:59.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.947+0000 7f63713fd740 20 client.20382 trim_cache size 0 max 16384 2024-04-21T08:48:59.953 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:48:59.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.947+0000 7f636eb75640 1 -- 172.21.15.43:0/1731025890 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f63580a3d20 msgr2=0x7f63580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.947+0000 7f636eb75640 1 --2- 172.21.15.43:0/1731025890 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f63580a3d20 0x7f63580a61e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f636002c630 tx=0x7f6360040000 comp rx=0 tx=0).stop 2024-04-21T08:48:59.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.947+0000 7f636eb75640 1 -- 172.21.15.43:0/1731025890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6368106170 msgr2=0x7f6368199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:59.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.947+0000 7f636eb75640 1 --2- 172.21.15.43:0/1731025890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6368106170 0x7f6368199a00 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f635400e9c0 tx=0x7f635400ee80 comp rx=0 tx=0).stop 2024-04-21T08:48:59.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.947+0000 7f636eb75640 1 -- 172.21.15.43:0/1731025890 shutdown_connections 2024-04-21T08:48:59.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.947+0000 7f636eb75640 1 --2- 172.21.15.43:0/1731025890 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f63580a3d20 0x7f63580a61e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.947+0000 7f636eb75640 1 --2- 172.21.15.43:0/1731025890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6368199fe0 0x7f63681a4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.947+0000 7f636eb75640 1 --2- 172.21.15.43:0/1731025890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6368106170 0x7f6368199a00 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.947+0000 7f636eb75640 1 --2- 172.21.15.43:0/1731025890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63681010a0 0x7f63681994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:59.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.947+0000 7f636eb75640 1 -- 172.21.15.43:0/1731025890 >> 172.21.15.43:0/1731025890 conn(0x7f6368077eb0 msgr2=0x7f6368100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:59.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.947+0000 7f636eb75640 1 -- 172.21.15.43:0/1731025890 shutdown_connections 2024-04-21T08:48:59.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:48:59.947+0000 7f636eb75640 1 -- 172.21.15.43:0/1731025890 wait complete. 2024-04-21T08:48:59.958 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:49:00.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.070+0000 7f72a6e81640 1 -- 172.21.15.43:0/4072685311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72a0103af0 msgr2=0x7f72a010bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.070+0000 7f72a6e81640 1 --2- 172.21.15.43:0/4072685311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72a0103af0 0x7f72a010bfd0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f728c009900 tx=0x7f728c02bd60 comp rx=0 tx=0).stop 2024-04-21T08:49:00.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.070+0000 7f72a6e81640 1 -- 172.21.15.43:0/4072685311 shutdown_connections 2024-04-21T08:49:00.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.070+0000 7f72a6e81640 1 --2- 172.21.15.43:0/4072685311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72a010c510 0x7f72a010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.070+0000 7f72a6e81640 1 --2- 172.21.15.43:0/4072685311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72a0103af0 0x7f72a010bfd0 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.070+0000 7f72a6e81640 1 --2- 172.21.15.43:0/4072685311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a01031d0 0x7f72a01035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.070+0000 7f72a6e81640 1 -- 172.21.15.43:0/4072685311 >> 172.21.15.43:0/4072685311 conn(0x7f72a0077f50 msgr2=0x7f72a0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:00.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.070+0000 7f72a6e81640 1 -- 172.21.15.43:0/4072685311 shutdown_connections 2024-04-21T08:49:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.070+0000 7f72a6e81640 1 -- 172.21.15.43:0/4072685311 wait complete. 2024-04-21T08:49:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.071+0000 7f72a6e81640 1 Processor -- start 2024-04-21T08:49:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.071+0000 7f72a6e81640 1 -- start start 2024-04-21T08:49:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.071+0000 7f72a6e81640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a01031d0 0x7f72a01a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.071+0000 7f72a6e81640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72a0103af0 0x7f72a01a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.071+0000 7f72a6e81640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72a010c510 0x7f72a019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.071+0000 7f72a6e81640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f72a0112200 con 0x7f72a0103af0 2024-04-21T08:49:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.071+0000 7f72a6e81640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f72a0112080 con 0x7f72a010c510 2024-04-21T08:49:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.071+0000 7f72a6e81640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f72a0112380 con 0x7f72a01031d0 2024-04-21T08:49:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.071+0000 7f729ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72a0103af0 0x7f72a01a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.071+0000 7f729ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72a0103af0 0x7f72a01a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47924/0 (socket says 172.21.15.43:47924) 2024-04-21T08:49:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.071+0000 7f729ffff640 1 -- 172.21.15.43:0/2755338066 learned_addr learned my addr 172.21.15.43:0/2755338066 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.071+0000 7f72a4c1d640 1 --2- 172.21.15.43:0/2755338066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a01031d0 0x7f72a01a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.071+0000 7f72a541e640 1 --2- 172.21.15.43:0/2755338066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72a010c510 0x7f72a019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.072+0000 7f729ffff640 1 -- 172.21.15.43:0/2755338066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a01031d0 msgr2=0x7f72a01a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.072+0000 7f729ffff640 1 --2- 172.21.15.43:0/2755338066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a01031d0 0x7f72a01a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.072+0000 7f729ffff640 1 -- 172.21.15.43:0/2755338066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72a010c510 msgr2=0x7f72a019a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.072+0000 7f729ffff640 1 --2- 172.21.15.43:0/2755338066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72a010c510 0x7f72a019a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.072+0000 7f729ffff640 1 -- 172.21.15.43:0/2755338066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a019afe0 con 0x7f72a0103af0 2024-04-21T08:49:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.072+0000 7f729ffff640 1 --2- 172.21.15.43:0/2755338066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72a0103af0 0x7f72a01a0bf0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f728c005940 tx=0x7f728c02d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.072+0000 7f729dffb640 1 -- 172.21.15.43:0/2755338066 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c043020 con 0x7f72a0103af0 2024-04-21T08:49:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.072+0000 7f72a6e81640 1 -- 172.21.15.43:0/2755338066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a019b270 con 0x7f72a0103af0 2024-04-21T08:49:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.072+0000 7f729dffb640 1 -- 172.21.15.43:0/2755338066 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f728c02dd20 con 0x7f72a0103af0 2024-04-21T08:49:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.072+0000 7f729dffb640 1 -- 172.21.15.43:0/2755338066 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c003b80 con 0x7f72a0103af0 2024-04-21T08:49:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.072+0000 7f72a6e81640 1 -- 172.21.15.43:0/2755338066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a01a7570 con 0x7f72a0103af0 2024-04-21T08:49:00.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.075+0000 7f729dffb640 1 -- 172.21.15.43:0/2755338066 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f728c046020 con 0x7f72a0103af0 2024-04-21T08:49:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.075+0000 7f72a9a75740 1 Processor -- start 2024-04-21T08:49:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.075+0000 7f729dffb640 1 --2- 172.21.15.43:0/2755338066 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f72900a3d20 0x7f72900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.075+0000 7f72a9a75740 1 -- start start 2024-04-21T08:49:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.076+0000 7f72a541e640 1 --2- 172.21.15.43:0/2755338066 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f72900a3d20 0x7f72900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.076+0000 7f729dffb640 1 -- 172.21.15.43:0/2755338066 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f728c0f2d30 con 0x7f72a0103af0 2024-04-21T08:49:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.076+0000 7f72a9a75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2bf4d0 0x5566ca2bf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.076+0000 7f72a9a75740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566ca2bfe80 0x5566ca2c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.076+0000 7f72a9a75740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566ca2c8a00 0x5566ca2cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.076+0000 7f72a9a75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5566ca2241f0 con 0x5566ca2bf4d0 2024-04-21T08:49:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.076+0000 7f72a9a75740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5566ca1c14b0 con 0x5566ca2bfe80 2024-04-21T08:49:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.076+0000 7f72a9a75740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5566ca2201b0 con 0x5566ca2c8a00 2024-04-21T08:49:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.076+0000 7f72a4c1d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2bf4d0 0x5566ca2bf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.076+0000 7f72a4c1d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2bf4d0 0x5566ca2bf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47938/0 (socket says 172.21.15.43:47938) 2024-04-21T08:49:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.076+0000 7f72a4c1d640 1 -- 172.21.15.43:0/660262179 learned_addr learned my addr 172.21.15.43:0/660262179 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.076+0000 7f72a541e640 1 --2- 172.21.15.43:0/660262179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566ca2c8a00 0x5566ca2cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.076+0000 7f729ffff640 1 --2- 172.21.15.43:0/660262179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566ca2bfe80 0x5566ca2c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.076+0000 7f72a541e640 1 --2- 172.21.15.43:0/2755338066 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f72900a3d20 0x7f72900a61e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7298009ef0 tx=0x7f729800c7f0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.076+0000 7f72a4c1d640 1 -- 172.21.15.43:0/660262179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566ca2c8a00 msgr2=0x5566ca2cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.076+0000 7f72a4c1d640 1 --2- 172.21.15.43:0/660262179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566ca2c8a00 0x5566ca2cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.076+0000 7f72a4c1d640 1 -- 172.21.15.43:0/660262179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566ca2bfe80 msgr2=0x5566ca2c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.076+0000 7f72a4c1d640 1 --2- 172.21.15.43:0/660262179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566ca2bfe80 0x5566ca2c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.076+0000 7f72a4c1d640 1 -- 172.21.15.43:0/660262179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566ca1b31b0 con 0x5566ca2bf4d0 2024-04-21T08:49:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.076+0000 7f729ffff640 1 --2- 172.21.15.43:0/660262179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566ca2bfe80 0x5566ca2c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.077+0000 7f72a4c1d640 1 --2- 172.21.15.43:0/660262179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2bf4d0 0x5566ca2bf8b0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f729400bc90 tx=0x7f72940107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1031c5cc269ae33b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.077+0000 7f728affd640 1 -- 172.21.15.43:0/660262179 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294005220 con 0x5566ca2bf4d0 2024-04-21T08:49:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.077+0000 7f728affd640 1 -- 172.21.15.43:0/660262179 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f729400be40 con 0x5566ca2bf4d0 2024-04-21T08:49:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.077+0000 7f728affd640 1 -- 172.21.15.43:0/660262179 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72940116a0 con 0x5566ca2bf4d0 2024-04-21T08:49:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.078+0000 7f72a9a75740 1 -- 172.21.15.43:0/660262179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2bf4d0 msgr2=0x5566ca2bf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.078+0000 7f72a9a75740 1 --2- 172.21.15.43:0/660262179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2bf4d0 0x5566ca2bf8b0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f729400bc90 tx=0x7f72940107a0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.078+0000 7f72a9a75740 1 -- 172.21.15.43:0/660262179 shutdown_connections 2024-04-21T08:49:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.078+0000 7f72a9a75740 1 --2- 172.21.15.43:0/660262179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566ca2c8a00 0x5566ca2cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.078+0000 7f72a9a75740 1 --2- 172.21.15.43:0/660262179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566ca2bfe80 0x5566ca2c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.078+0000 7f72a9a75740 1 --2- 172.21.15.43:0/660262179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2bf4d0 0x5566ca2bf8b0 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.078+0000 7f72a9a75740 1 -- 172.21.15.43:0/660262179 >> 172.21.15.43:0/660262179 conn(0x5566ca2b9ba0 msgr2=0x5566ca2b9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.078+0000 7f72a9a75740 1 -- 172.21.15.43:0/660262179 shutdown_connections 2024-04-21T08:49:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.078+0000 7f72a9a75740 1 -- 172.21.15.43:0/660262179 wait complete. 2024-04-21T08:49:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.078+0000 7f72a9a75740 1 -- start start 2024-04-21T08:49:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.078+0000 7f72a9a75740 1 Processor -- start 2024-04-21T08:49:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.079+0000 7f72a9a75740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566ca2bf4d0 0x5566ca2c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.079+0000 7f72a9a75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2bfe80 0x5566ca2c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.079+0000 7f72a9a75740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566ca2c8a00 0x5566ca2c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.079+0000 7f72a9a75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5566ca2cb970 con 0x5566ca2bfe80 2024-04-21T08:49:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.079+0000 7f72a9a75740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5566ca1dd500 con 0x5566ca2bf4d0 2024-04-21T08:49:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.079+0000 7f72a9a75740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5566ca2cbaf0 con 0x5566ca2c8a00 2024-04-21T08:49:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.079+0000 7f72a9a75740 10 client.0 _ll_register_callbacks cb 0x5566ca1abb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.079+0000 7f729ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2bfe80 0x5566ca2c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.079+0000 7f729ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2bfe80 0x5566ca2c56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47952/0 (socket says 172.21.15.43:47952) 2024-04-21T08:49:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.079+0000 7f729ffff640 1 -- 172.21.15.43:0/2067721319 learned_addr learned my addr 172.21.15.43:0/2067721319 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.079+0000 7f72a541e640 1 --2- 172.21.15.43:0/2067721319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566ca2c8a00 0x5566ca2c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.079+0000 7f72a4c1d640 1 --2- 172.21.15.43:0/2067721319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566ca2bf4d0 0x5566ca2c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.079+0000 7f729ffff640 1 -- 172.21.15.43:0/2067721319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566ca2c8a00 msgr2=0x5566ca2c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.079+0000 7f729ffff640 1 --2- 172.21.15.43:0/2067721319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566ca2c8a00 0x5566ca2c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.079+0000 7f729ffff640 1 -- 172.21.15.43:0/2067721319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566ca2bf4d0 msgr2=0x5566ca2c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.079+0000 7f729ffff640 1 --2- 172.21.15.43:0/2067721319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566ca2bf4d0 0x5566ca2c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.079+0000 7f729ffff640 1 -- 172.21.15.43:0/2067721319 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566c9f363f0 con 0x5566ca2bfe80 2024-04-21T08:49:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.079+0000 7f729ffff640 1 --2- 172.21.15.43:0/2067721319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2bfe80 0x5566ca2c56e0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f728c000c00 tx=0x7f728c0be5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837f73183abedf5d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:00.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.079+0000 7f727ffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:00.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.080+0000 7f727ffff640 1 -- 172.21.15.43:0/2067721319 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c0be9c0 con 0x5566ca2bfe80 2024-04-21T08:49:00.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.080+0000 7f727ffff640 1 -- 172.21.15.43:0/2067721319 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f728c0bb030 con 0x5566ca2bfe80 2024-04-21T08:49:00.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.080+0000 7f72a9a75740 1 -- 172.21.15.43:0/2067721319 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5566ca1c3930 con 0x5566ca2bfe80 2024-04-21T08:49:00.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.080+0000 7f727ffff640 1 -- 172.21.15.43:0/2067721319 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f728c0c2d70 con 0x5566ca2bfe80 2024-04-21T08:49:00.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.080+0000 7f72a9a75740 10 client.20394 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:00.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.080+0000 7f72a9a75740 1 -- 172.21.15.43:0/2067721319 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5566ca0007d0 con 0x5566ca2bfe80 2024-04-21T08:49:00.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.080+0000 7f727ffff640 1 -- 172.21.15.43:0/2067721319 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5566ca1c3930 con 0x5566ca2bfe80 2024-04-21T08:49:00.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.080+0000 7f727ffff640 1 -- 172.21.15.43:0/2067721319 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f728c0ddb80 con 0x5566ca2bfe80 2024-04-21T08:49:00.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.080+0000 7f72a9a75740 10 client.20394 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:00.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.080+0000 7f72a9a75740 10 client.20394 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:49:00.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.080+0000 7f72a9a75740 20 client.20394 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:00.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.080+0000 7f72a9a75740 1 --2- 172.21.15.43:0/2067721319 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5566ca2d8ac0 0x5566ca2daeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.080+0000 7f72a9a75740 4 client.20394 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:00.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.080+0000 7f72a9a75740 1 -- 172.21.15.43:0/2067721319 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566ca231c10 con 0x5566ca2d8ac0 2024-04-21T08:49:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.081+0000 7f72a4c1d640 1 --2- 172.21.15.43:0/2067721319 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5566ca2d8ac0 0x5566ca2daeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.081+0000 7f727ffff640 10 client.20394 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:49:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.081+0000 7f72a4c1d640 1 --2- 172.21.15.43:0/2067721319 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5566ca2d8ac0 0x5566ca2daeb0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=da96e0ba64a017d server_cookie=8fc4c81f414c569f in_seq=0 out_seq=0 2024-04-21T08:49:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.082+0000 7f727ffff640 1 -- 172.21.15.43:0/2067721319 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5566ca231c10 con 0x5566ca2d8ac0 2024-04-21T08:49:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.082+0000 7f727ffff640 10 client.20394 handle_command_reply: tid=0 2024-04-21T08:49:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.082+0000 7f72a9a75740 1 client.20394 shutdown 2024-04-21T08:49:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.082+0000 7f72a9a75740 1 -- 172.21.15.43:0/2067721319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2bfe80 msgr2=0x5566ca2c56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.082+0000 7f72a9a75740 1 --2- 172.21.15.43:0/2067721319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2bfe80 0x5566ca2c56e0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f728c000c00 tx=0x7f728c0be5c0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.082+0000 7f72a9a75740 1 -- 172.21.15.43:0/2067721319 shutdown_connections 2024-04-21T08:49:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.082+0000 7f72a9a75740 1 --2- 172.21.15.43:0/2067721319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566ca2c8a00 0x5566ca2c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.082+0000 7f72a9a75740 1 --2- 172.21.15.43:0/2067721319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2bfe80 0x5566ca2c56e0 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.082+0000 7f72a9a75740 1 --2- 172.21.15.43:0/2067721319 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5566ca2d8ac0 0x5566ca2daeb0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.082+0000 7f72a9a75740 1 --2- 172.21.15.43:0/2067721319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566ca2bf4d0 0x5566ca2c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.082+0000 7f727ffff640 0 client.20394 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:49:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.082+0000 7f72a9a75740 1 -- 172.21.15.43:0/2067721319 >> 172.21.15.43:0/2067721319 conn(0x5566ca2b9ba0 msgr2=0x5566ca2bb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.082+0000 7f72a9a75740 1 -- 172.21.15.43:0/2067721319 shutdown_connections 2024-04-21T08:49:00.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.082+0000 7f72a9a75740 1 -- 172.21.15.43:0/2067721319 wait complete. 2024-04-21T08:49:00.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.083+0000 7f72a9a75740 20 client.20394 trim_cache size 0 max 16384 2024-04-21T08:49:00.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.098+0000 7f72a9a75740 1 Processor -- start 2024-04-21T08:49:00.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.098+0000 7f72a9a75740 1 -- start start 2024-04-21T08:49:00.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.099+0000 7f72a9a75740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566ca2bf4d0 0x5566ca2d4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.099+0000 7f72a9a75740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566ca2bfe80 0x5566ca2c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.099+0000 7f72a9a75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2c8a00 0x5566ca2c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.099+0000 7f72a9a75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f72a01b16e0 con 0x5566ca2c8a00 2024-04-21T08:49:00.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.099+0000 7f72a9a75740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f72a01b3220 con 0x5566ca2bfe80 2024-04-21T08:49:00.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.099+0000 7f72a9a75740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f72a01b1db0 con 0x5566ca2bf4d0 2024-04-21T08:49:00.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.099+0000 7f729ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566ca2bfe80 0x5566ca2c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.099+0000 7f729ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566ca2bfe80 0x5566ca2c7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:59348/0 (socket says 172.21.15.43:59348) 2024-04-21T08:49:00.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.099+0000 7f729ffff640 1 -- 172.21.15.43:0/2498160909 learned_addr learned my addr 172.21.15.43:0/2498160909 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:00.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.099+0000 7f72a541e640 1 --2- 172.21.15.43:0/2498160909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2c8a00 0x5566ca2c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.099+0000 7f72a4c1d640 1 --2- 172.21.15.43:0/2498160909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566ca2bf4d0 0x5566ca2d4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.099+0000 7f729ffff640 1 -- 172.21.15.43:0/2498160909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566ca2bf4d0 msgr2=0x5566ca2d4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:00.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.099+0000 7f729ffff640 1 --2- 172.21.15.43:0/2498160909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566ca2bf4d0 0x5566ca2d4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.099+0000 7f729ffff640 1 -- 172.21.15.43:0/2498160909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2c8a00 msgr2=0x5566ca2c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:00.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.099+0000 7f729ffff640 1 --2- 172.21.15.43:0/2498160909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2c8a00 0x5566ca2c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.099+0000 7f729ffff640 1 -- 172.21.15.43:0/2498160909 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566ca2c6060 con 0x5566ca2bfe80 2024-04-21T08:49:00.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.099+0000 7f729ffff640 1 --2- 172.21.15.43:0/2498160909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566ca2bfe80 0x5566ca2c7e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f728c0bae20 tx=0x7f728c003f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=415e9a65ce5bdb7d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:00.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.100+0000 7f728a7fc640 1 -- 172.21.15.43:0/2498160909 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c0be790 con 0x5566ca2bfe80 2024-04-21T08:49:00.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.100+0000 7f728a7fc640 1 -- 172.21.15.43:0/2498160909 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f728c0be9c0 con 0x5566ca2bfe80 2024-04-21T08:49:00.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.100+0000 7f728a7fc640 1 -- 172.21.15.43:0/2498160909 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c0c71a0 con 0x5566ca2bfe80 2024-04-21T08:49:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.100+0000 7f72a9a75740 1 -- 172.21.15.43:0/2498160909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566ca2bfe80 msgr2=0x5566ca2c7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.100+0000 7f72a9a75740 1 --2- 172.21.15.43:0/2498160909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566ca2bfe80 0x5566ca2c7e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f728c0bae20 tx=0x7f728c003f30 comp rx=0 tx=0).stop 2024-04-21T08:49:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.100+0000 7f72a9a75740 1 -- 172.21.15.43:0/2498160909 shutdown_connections 2024-04-21T08:49:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.100+0000 7f72a9a75740 1 --2- 172.21.15.43:0/2498160909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2c8a00 0x5566ca2c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.100+0000 7f72a9a75740 1 --2- 172.21.15.43:0/2498160909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566ca2bfe80 0x5566ca2c7e10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.100+0000 7f72a9a75740 1 --2- 172.21.15.43:0/2498160909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566ca2bf4d0 0x5566ca2d4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.100+0000 7f72a9a75740 1 -- 172.21.15.43:0/2498160909 >> 172.21.15.43:0/2498160909 conn(0x5566ca2b9ba0 msgr2=0x5566ca2bb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.101+0000 7f72a9a75740 1 -- 172.21.15.43:0/2498160909 shutdown_connections 2024-04-21T08:49:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.101+0000 7f72a9a75740 1 -- 172.21.15.43:0/2498160909 wait complete. 2024-04-21T08:49:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.101+0000 7f72a9a75740 1 -- start start 2024-04-21T08:49:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.101+0000 7f72a9a75740 1 Processor -- start 2024-04-21T08:49:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.101+0000 7f72a9a75740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566ca2bf4d0 0x5566ca2c2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.101+0000 7f72a9a75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2bfe80 0x5566ca2c2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.101+0000 7f72a9a75740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566ca2c8a00 0x5566ca2c2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.101+0000 7f72a9a75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f72a01b2b50 con 0x5566ca2bfe80 2024-04-21T08:49:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.101+0000 7f72a9a75740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f72a01b2480 con 0x5566ca2bf4d0 2024-04-21T08:49:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.101+0000 7f72a9a75740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f72a01af6e0 con 0x5566ca2c8a00 2024-04-21T08:49:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.101+0000 7f72a9a75740 10 client.0 _ll_register_callbacks cb 0x7f728c0bac60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.101+0000 7f729ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2bfe80 0x5566ca2c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.101+0000 7f729ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2bfe80 0x5566ca2c2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47968/0 (socket says 172.21.15.43:47968) 2024-04-21T08:49:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.101+0000 7f729ffff640 1 -- 172.21.15.43:0/61786193 learned_addr learned my addr 172.21.15.43:0/61786193 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.102+0000 7f72a4c1d640 1 --2- 172.21.15.43:0/61786193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566ca2bf4d0 0x5566ca2c2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.102+0000 7f72a541e640 1 --2- 172.21.15.43:0/61786193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566ca2c8a00 0x5566ca2c2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.102+0000 7f729ffff640 1 -- 172.21.15.43:0/61786193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566ca2c8a00 msgr2=0x5566ca2c2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.102+0000 7f729ffff640 1 --2- 172.21.15.43:0/61786193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566ca2c8a00 0x5566ca2c2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.102+0000 7f729ffff640 1 -- 172.21.15.43:0/61786193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566ca2bf4d0 msgr2=0x5566ca2c2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.102+0000 7f729ffff640 1 --2- 172.21.15.43:0/61786193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566ca2bf4d0 0x5566ca2c2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.102+0000 7f729ffff640 1 -- 172.21.15.43:0/61786193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566ca33e030 con 0x5566ca2bfe80 2024-04-21T08:49:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.102+0000 7f72a541e640 1 --2- 172.21.15.43:0/61786193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566ca2c8a00 0x5566ca2c2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.102+0000 7f729ffff640 1 --2- 172.21.15.43:0/61786193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2bfe80 0x5566ca2c2850 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f728c0be6c0 tx=0x7f728c003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2021f5cc2a99153a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.102+0000 7f727ffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.102+0000 7f727ffff640 1 -- 172.21.15.43:0/61786193 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c0ba910 con 0x5566ca2bfe80 2024-04-21T08:49:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.102+0000 7f727ffff640 1 -- 172.21.15.43:0/61786193 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f728c0c71a0 con 0x5566ca2bfe80 2024-04-21T08:49:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.102+0000 7f72a9a75740 1 -- 172.21.15.43:0/61786193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f728c0be9c0 con 0x5566ca2bfe80 2024-04-21T08:49:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.102+0000 7f727ffff640 1 -- 172.21.15.43:0/61786193 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f728c0f0450 con 0x5566ca2bfe80 2024-04-21T08:49:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.102+0000 7f72a9a75740 10 client.20400 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.102+0000 7f72a9a75740 1 -- 172.21.15.43:0/61786193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5566ca33e030 con 0x5566ca2bfe80 2024-04-21T08:49:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.103+0000 7f727ffff640 1 -- 172.21.15.43:0/61786193 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f728c0be9c0 con 0x5566ca2bfe80 2024-04-21T08:49:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.103+0000 7f727ffff640 1 -- 172.21.15.43:0/61786193 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f728c0bd880 con 0x5566ca2bfe80 2024-04-21T08:49:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.103+0000 7f72a9a75740 10 client.20400 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.103+0000 7f72a9a75740 10 client.20400 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:49:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.103+0000 7f72a9a75740 20 client.20400 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.103+0000 7f72a9a75740 1 --2- 172.21.15.43:0/61786193 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5566ca2d8ac0 0x5566ca341540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.103+0000 7f72a9a75740 4 client.20400 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.103+0000 7f72a9a75740 1 -- 172.21.15.43:0/61786193 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566ca22ba80 con 0x5566ca2d8ac0 2024-04-21T08:49:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.103+0000 7f72a4c1d640 1 --2- 172.21.15.43:0/61786193 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5566ca2d8ac0 0x5566ca341540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.103+0000 7f727ffff640 10 client.20400 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:49:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.103+0000 7f72a4c1d640 1 --2- 172.21.15.43:0/61786193 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5566ca2d8ac0 0x5566ca341540 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=278d734650e0d01a server_cookie=144a088a3fc5ae00 in_seq=0 out_seq=0 2024-04-21T08:49:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.104+0000 7f727ffff640 1 -- 172.21.15.43:0/61786193 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566ca22ba80 con 0x5566ca2d8ac0 2024-04-21T08:49:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.104+0000 7f727ffff640 10 client.20400 handle_command_reply: tid=0 2024-04-21T08:49:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.104+0000 7f72a9a75740 1 client.20400 shutdown 2024-04-21T08:49:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.104+0000 7f72a9a75740 1 -- 172.21.15.43:0/61786193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2bfe80 msgr2=0x5566ca2c2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.104+0000 7f72a9a75740 1 --2- 172.21.15.43:0/61786193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2bfe80 0x5566ca2c2850 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f728c0be6c0 tx=0x7f728c003680 comp rx=0 tx=0).stop 2024-04-21T08:49:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.104+0000 7f72a9a75740 1 -- 172.21.15.43:0/61786193 shutdown_connections 2024-04-21T08:49:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.104+0000 7f72a9a75740 1 --2- 172.21.15.43:0/61786193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566ca2c8a00 0x5566ca2c2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.104+0000 7f72a9a75740 1 --2- 172.21.15.43:0/61786193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566ca2bfe80 0x5566ca2c2850 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.104+0000 7f72a9a75740 1 --2- 172.21.15.43:0/61786193 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5566ca2d8ac0 0x5566ca341540 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.104+0000 7f72a9a75740 1 --2- 172.21.15.43:0/61786193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566ca2bf4d0 0x5566ca2c2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.104+0000 7f72a9a75740 1 -- 172.21.15.43:0/61786193 >> 172.21.15.43:0/61786193 conn(0x5566ca2b9ba0 msgr2=0x5566ca2b8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.104+0000 7f727ffff640 0 client.20400 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:49:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.104+0000 7f72a9a75740 1 -- 172.21.15.43:0/61786193 shutdown_connections 2024-04-21T08:49:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.104+0000 7f72a9a75740 1 -- 172.21.15.43:0/61786193 wait complete. 2024-04-21T08:49:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.104+0000 7f72a9a75740 20 client.20400 trim_cache size 0 max 16384 2024-04-21T08:49:00.110 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:49:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.105+0000 7f72a6e81640 1 -- 172.21.15.43:0/2755338066 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f72900a3d20 msgr2=0x7f72900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.105+0000 7f72a6e81640 1 --2- 172.21.15.43:0/2755338066 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f72900a3d20 0x7f72900a61e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7298009ef0 tx=0x7f729800c7f0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.105+0000 7f72a6e81640 1 -- 172.21.15.43:0/2755338066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72a0103af0 msgr2=0x7f72a01a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.105+0000 7f72a6e81640 1 --2- 172.21.15.43:0/2755338066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72a0103af0 0x7f72a01a0bf0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f728c005940 tx=0x7f728c02d920 comp rx=0 tx=0).stop 2024-04-21T08:49:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.105+0000 7f72a6e81640 1 -- 172.21.15.43:0/2755338066 shutdown_connections 2024-04-21T08:49:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.105+0000 7f72a6e81640 1 --2- 172.21.15.43:0/2755338066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72a010c510 0x7f72a019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.105+0000 7f72a6e81640 1 --2- 172.21.15.43:0/2755338066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72a0103af0 0x7f72a01a0bf0 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.105+0000 7f72a6e81640 1 --2- 172.21.15.43:0/2755338066 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f72900a3d20 0x7f72900a61e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.105+0000 7f72a6e81640 1 --2- 172.21.15.43:0/2755338066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a01031d0 0x7f72a01a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.105+0000 7f72a6e81640 1 -- 172.21.15.43:0/2755338066 >> 172.21.15.43:0/2755338066 conn(0x7f72a0077f50 msgr2=0x7f72a01085c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.105+0000 7f72a6e81640 1 -- 172.21.15.43:0/2755338066 shutdown_connections 2024-04-21T08:49:00.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.105+0000 7f72a6e81640 1 -- 172.21.15.43:0/2755338066 wait complete. 2024-04-21T08:49:00.116 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:49:00.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.222+0000 7fd853524640 1 -- 172.21.15.43:0/2609150331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd84c102190 msgr2=0x7fd84c102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.222+0000 7fd853524640 1 --2- 172.21.15.43:0/2609150331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd84c102190 0x7fd84c102570 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fd848009900 tx=0x7fd84802bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.222+0000 7fd853524640 1 -- 172.21.15.43:0/2609150331 shutdown_connections 2024-04-21T08:49:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.222+0000 7fd853524640 1 --2- 172.21.15.43:0/2609150331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd84c107260 0x7fd84c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.222+0000 7fd853524640 1 --2- 172.21.15.43:0/2609150331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd84c102b40 0x7fd84c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.222+0000 7fd853524640 1 --2- 172.21.15.43:0/2609150331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd84c102190 0x7fd84c102570 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.222+0000 7fd853524640 1 -- 172.21.15.43:0/2609150331 >> 172.21.15.43:0/2609150331 conn(0x7fd84c0793a0 msgr2=0x7fd84c0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.222+0000 7fd853524640 1 -- 172.21.15.43:0/2609150331 shutdown_connections 2024-04-21T08:49:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.222+0000 7fd853524640 1 -- 172.21.15.43:0/2609150331 wait complete. 2024-04-21T08:49:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.223+0000 7fd853524640 1 Processor -- start 2024-04-21T08:49:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.223+0000 7fd853524640 1 -- start start 2024-04-21T08:49:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.223+0000 7fd853524640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd84c102190 0x7fd84c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.223+0000 7fd853524640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd84c107260 0x7fd84c198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.223+0000 7fd853524640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd84c19cfc0 0x7fd84c19d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.223+0000 7fd853524640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1121a0 con 0x7fd84c102190 2024-04-21T08:49:00.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.223+0000 7fd853524640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c112020 con 0x7fd84c107260 2024-04-21T08:49:00.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.223+0000 7fd853524640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c112320 con 0x7fd84c19cfc0 2024-04-21T08:49:00.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.223+0000 7fd8512c0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd84c102190 0x7fd84c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.223+0000 7fd8512c0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd84c102190 0x7fd84c198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48000/0 (socket says 172.21.15.43:48000) 2024-04-21T08:49:00.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.223+0000 7fd8512c0640 1 -- 172.21.15.43:0/2670861123 learned_addr learned my addr 172.21.15.43:0/2670861123 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:00.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.223+0000 7fd850abf640 1 --2- 172.21.15.43:0/2670861123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd84c107260 0x7fd84c198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.223+0000 7fd8512c0640 1 -- 172.21.15.43:0/2670861123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd84c19cfc0 msgr2=0x7fd84c19d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.223+0000 7fd851ac1640 1 --2- 172.21.15.43:0/2670861123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd84c19cfc0 0x7fd84c19d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.223+0000 7fd8512c0640 1 --2- 172.21.15.43:0/2670861123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd84c19cfc0 0x7fd84c19d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.223+0000 7fd8512c0640 1 -- 172.21.15.43:0/2670861123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd84c107260 msgr2=0x7fd84c198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.223+0000 7fd8512c0640 1 --2- 172.21.15.43:0/2670861123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd84c107260 0x7fd84c198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.223+0000 7fd8512c0640 1 -- 172.21.15.43:0/2670861123 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84c19dab0 con 0x7fd84c102190 2024-04-21T08:49:00.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.223+0000 7fd850abf640 1 --2- 172.21.15.43:0/2670861123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd84c107260 0x7fd84c198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:00.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.224+0000 7fd8512c0640 1 --2- 172.21.15.43:0/2670861123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd84c102190 0x7fd84c198700 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fd848005940 tx=0x7fd84802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:00.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.224+0000 7fd8427fc640 1 -- 172.21.15.43:0/2670861123 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd848043020 con 0x7fd84c102190 2024-04-21T08:49:00.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.224+0000 7fd8427fc640 1 -- 172.21.15.43:0/2670861123 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd84802cbe0 con 0x7fd84c102190 2024-04-21T08:49:00.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.224+0000 7fd853524640 1 -- 172.21.15.43:0/2670861123 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd84c19dd40 con 0x7fd84c102190 2024-04-21T08:49:00.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.224+0000 7fd8427fc640 1 -- 172.21.15.43:0/2670861123 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd848003bf0 con 0x7fd84c102190 2024-04-21T08:49:00.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.224+0000 7fd853524640 1 -- 172.21.15.43:0/2670861123 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd84c1a5560 con 0x7fd84c102190 2024-04-21T08:49:00.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.226+0000 7fd8427fc640 1 -- 172.21.15.43:0/2670861123 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd848046020 con 0x7fd84c102190 2024-04-21T08:49:00.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.227+0000 7fd8561f2740 1 Processor -- start 2024-04-21T08:49:00.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.227+0000 7fd8561f2740 1 -- start start 2024-04-21T08:49:00.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.227+0000 7fd8561f2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bbfef84d0 0x557bbfef88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.227+0000 7fd8427fc640 1 --2- 172.21.15.43:0/2670861123 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd8380a3d20 0x7fd8380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.227+0000 7fd8561f2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bbfef8e80 0x557bbff01430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.227+0000 7fd8561f2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbff01a00 0x557bbff03e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.227+0000 7fd8561f2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557bbfe5d1f0 con 0x557bbff01a00 2024-04-21T08:49:00.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.227+0000 7fd8561f2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557bbfdfa4b0 con 0x557bbfef84d0 2024-04-21T08:49:00.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.227+0000 7fd8561f2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557bbfe591b0 con 0x557bbfef8e80 2024-04-21T08:49:00.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.227+0000 7fd8512c0640 1 --2- 172.21.15.43:0/2670861123 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd8380a3d20 0x7fd8380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.227+0000 7fd851ac1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbff01a00 0x557bbff03e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.227+0000 7fd850abf640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bbfef8e80 0x557bbff01430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.227+0000 7fd8427fc640 1 -- 172.21.15.43:0/2670861123 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fd8480f2e40 con 0x7fd84c102190 2024-04-21T08:49:00.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.227+0000 7fd850abf640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bbfef8e80 0x557bbff01430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.43:51548/0 (socket says 172.21.15.43:51548) 2024-04-21T08:49:00.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.227+0000 7fd851ac1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbff01a00 0x557bbff03e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48010/0 (socket says 172.21.15.43:48010) 2024-04-21T08:49:00.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.227+0000 7fd850abf640 1 -- 172.21.15.43:0/2711670786 learned_addr learned my addr 172.21.15.43:0/2711670786 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:00.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.227+0000 7fd851ac1640 1 --2- 172.21.15.43:0/2711670786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bbfef84d0 0x557bbfef88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.227+0000 7fd850abf640 1 -- 172.21.15.43:0/2711670786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bbfef84d0 msgr2=0x557bbfef88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:00.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.227+0000 7fd850abf640 1 --2- 172.21.15.43:0/2711670786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bbfef84d0 0x557bbfef88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.227+0000 7fd850abf640 1 -- 172.21.15.43:0/2711670786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbff01a00 msgr2=0x557bbff03e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:00.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.227+0000 7fd850abf640 1 --2- 172.21.15.43:0/2711670786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbff01a00 0x557bbff03e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.227+0000 7fd850abf640 1 -- 172.21.15.43:0/2711670786 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bbfdec1b0 con 0x557bbfef8e80 2024-04-21T08:49:00.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.227+0000 7fd851ac1640 1 --2- 172.21.15.43:0/2711670786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbff01a00 0x557bbff03e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:00.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.228+0000 7fd8512c0640 1 --2- 172.21.15.43:0/2670861123 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd8380a3d20 0x7fd8380a61e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd8480f0770 tx=0x7fd84803ac10 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:00.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.228+0000 7fd850abf640 1 --2- 172.21.15.43:0/2711670786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bbfef8e80 0x557bbff01430 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00bc80 tx=0x7fd83c00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a8e69374d028fe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:00.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.228+0000 7fd8377fe640 1 -- 172.21.15.43:0/2711670786 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c011440 con 0x557bbfef8e80 2024-04-21T08:49:00.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.228+0000 7fd8377fe640 1 -- 172.21.15.43:0/2711670786 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd83c011a90 con 0x557bbfef8e80 2024-04-21T08:49:00.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.228+0000 7fd8377fe640 1 -- 172.21.15.43:0/2711670786 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c01b950 con 0x557bbfef8e80 2024-04-21T08:49:00.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.229+0000 7fd8561f2740 1 -- 172.21.15.43:0/2711670786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bbfef8e80 msgr2=0x557bbff01430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.229+0000 7fd8561f2740 1 --2- 172.21.15.43:0/2711670786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bbfef8e80 0x557bbff01430 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00bc80 tx=0x7fd83c00f730 comp rx=0 tx=0).stop 2024-04-21T08:49:00.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.229+0000 7fd8561f2740 1 -- 172.21.15.43:0/2711670786 shutdown_connections 2024-04-21T08:49:00.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.229+0000 7fd8561f2740 1 --2- 172.21.15.43:0/2711670786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbff01a00 0x557bbff03e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.229+0000 7fd8561f2740 1 --2- 172.21.15.43:0/2711670786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bbfef8e80 0x557bbff01430 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.229+0000 7fd8561f2740 1 --2- 172.21.15.43:0/2711670786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bbfef84d0 0x557bbfef88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.229+0000 7fd8561f2740 1 -- 172.21.15.43:0/2711670786 >> 172.21.15.43:0/2711670786 conn(0x557bbfef2ba0 msgr2=0x557bbfef2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:00.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.229+0000 7fd8561f2740 1 -- 172.21.15.43:0/2711670786 shutdown_connections 2024-04-21T08:49:00.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.229+0000 7fd8561f2740 1 -- 172.21.15.43:0/2711670786 wait complete. 2024-04-21T08:49:00.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.229+0000 7fd8561f2740 1 -- start start 2024-04-21T08:49:00.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.229+0000 7fd8561f2740 1 Processor -- start 2024-04-21T08:49:00.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.229+0000 7fd8561f2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bbfef84d0 0x557bbff01310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.229+0000 7fd8561f2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbfef8e80 0x557bbfefe6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.229+0000 7fd8561f2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bbff01a00 0x557bbfefec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.229+0000 7fd8561f2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557bbff04970 con 0x557bbfef8e80 2024-04-21T08:49:00.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.229+0000 7fd8561f2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557bbfe16500 con 0x557bbff01a00 2024-04-21T08:49:00.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.229+0000 7fd8561f2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557bbff04af0 con 0x557bbfef84d0 2024-04-21T08:49:00.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.229+0000 7fd8561f2740 10 client.0 _ll_register_callbacks cb 0x557bbfde4b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:00.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.230+0000 7fd850abf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbfef8e80 0x557bbfefe6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.230+0000 7fd850abf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbfef8e80 0x557bbfefe6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48022/0 (socket says 172.21.15.43:48022) 2024-04-21T08:49:00.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.230+0000 7fd850abf640 1 -- 172.21.15.43:0/1354483346 learned_addr learned my addr 172.21.15.43:0/1354483346 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:00.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.230+0000 7fd851ac1640 1 --2- 172.21.15.43:0/1354483346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bbfef84d0 0x557bbff01310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.230+0000 7fd851ac1640 1 --2- 172.21.15.43:0/1354483346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bbff01a00 0x557bbfefec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.230+0000 7fd850abf640 1 -- 172.21.15.43:0/1354483346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bbfef84d0 msgr2=0x557bbff01310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:00.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.230+0000 7fd850abf640 1 --2- 172.21.15.43:0/1354483346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bbfef84d0 0x557bbff01310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.230+0000 7fd850abf640 1 -- 172.21.15.43:0/1354483346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bbff01a00 msgr2=0x557bbfefec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:00.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.230+0000 7fd850abf640 1 --2- 172.21.15.43:0/1354483346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bbff01a00 0x557bbfefec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.230+0000 7fd850abf640 1 -- 172.21.15.43:0/1354483346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bbfb6f3f0 con 0x557bbfef8e80 2024-04-21T08:49:00.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.230+0000 7fd851ac1640 1 --2- 172.21.15.43:0/1354483346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bbfef84d0 0x557bbff01310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:00.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.230+0000 7fd850abf640 1 --2- 172.21.15.43:0/1354483346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbfef8e80 0x557bbfefe6a0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fd83c01a000 tx=0x7fd83c00b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77019ae7b8bd4161 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:00.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.230+0000 7fd834ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:00.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.230+0000 7fd834ff9640 1 -- 172.21.15.43:0/1354483346 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c02a060 con 0x557bbfef8e80 2024-04-21T08:49:00.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.230+0000 7fd834ff9640 1 -- 172.21.15.43:0/1354483346 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fd83c004250 con 0x557bbfef8e80 2024-04-21T08:49:00.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.230+0000 7fd8561f2740 1 -- 172.21.15.43:0/1354483346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd83c011a90 con 0x557bbfef8e80 2024-04-21T08:49:00.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.230+0000 7fd834ff9640 1 -- 172.21.15.43:0/1354483346 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd83c031050 con 0x557bbfef8e80 2024-04-21T08:49:00.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.230+0000 7fd8561f2740 10 client.20412 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:00.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.230+0000 7fd8561f2740 1 -- 172.21.15.43:0/1354483346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557bbfc397d0 con 0x557bbfef8e80 2024-04-21T08:49:00.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.231+0000 7fd834ff9640 1 -- 172.21.15.43:0/1354483346 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd83c011a90 con 0x557bbfef8e80 2024-04-21T08:49:00.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.231+0000 7fd834ff9640 1 -- 172.21.15.43:0/1354483346 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd83c01b9c0 con 0x557bbfef8e80 2024-04-21T08:49:00.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.231+0000 7fd8561f2740 10 client.20412 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:00.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.231+0000 7fd8561f2740 10 client.20412 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:49:00.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.231+0000 7fd8561f2740 20 client.20412 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:00.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.231+0000 7fd8561f2740 1 --2- 172.21.15.43:0/1354483346 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x557bbff11be0 0x557bbff13fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.231+0000 7fd8561f2740 4 client.20412 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:00.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.231+0000 7fd8561f2740 1 -- 172.21.15.43:0/1354483346 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bbfdfc930 con 0x557bbff11be0 2024-04-21T08:49:00.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.231+0000 7fd851ac1640 1 --2- 172.21.15.43:0/1354483346 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x557bbff11be0 0x557bbff13fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.231+0000 7fd834ff9640 10 client.20412 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:49:00.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.231+0000 7fd851ac1640 1 --2- 172.21.15.43:0/1354483346 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x557bbff11be0 0x557bbff13fd0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28cd333ecc8a78b7 server_cookie=2c4f6b4472d9c00d in_seq=0 out_seq=0 2024-04-21T08:49:00.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.232+0000 7fd834ff9640 1 -- 172.21.15.43:0/1354483346 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557bbfdfc930 con 0x557bbff11be0 2024-04-21T08:49:00.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.232+0000 7fd834ff9640 10 client.20412 handle_command_reply: tid=0 2024-04-21T08:49:00.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.232+0000 7fd8561f2740 1 client.20412 shutdown 2024-04-21T08:49:00.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.232+0000 7fd8561f2740 1 -- 172.21.15.43:0/1354483346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbfef8e80 msgr2=0x557bbfefe6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.232+0000 7fd8561f2740 1 --2- 172.21.15.43:0/1354483346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbfef8e80 0x557bbfefe6a0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fd83c01a000 tx=0x7fd83c00b9f0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.232+0000 7fd8561f2740 1 -- 172.21.15.43:0/1354483346 shutdown_connections 2024-04-21T08:49:00.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.232+0000 7fd8561f2740 1 --2- 172.21.15.43:0/1354483346 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x557bbff11be0 0x557bbff13fd0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.232+0000 7fd8561f2740 1 --2- 172.21.15.43:0/1354483346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bbff01a00 0x557bbfefec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.232+0000 7fd8561f2740 1 --2- 172.21.15.43:0/1354483346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbfef8e80 0x557bbfefe6a0 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.232+0000 7fd8561f2740 1 --2- 172.21.15.43:0/1354483346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bbfef84d0 0x557bbff01310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.232+0000 7fd834ff9640 0 client.20412 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:49:00.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.232+0000 7fd8561f2740 1 -- 172.21.15.43:0/1354483346 >> 172.21.15.43:0/1354483346 conn(0x557bbfef2ba0 msgr2=0x557bbfef48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:00.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.232+0000 7fd8561f2740 1 -- 172.21.15.43:0/1354483346 shutdown_connections 2024-04-21T08:49:00.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.232+0000 7fd8561f2740 1 -- 172.21.15.43:0/1354483346 wait complete. 2024-04-21T08:49:00.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.233+0000 7fd8561f2740 20 client.20412 trim_cache size 0 max 16384 2024-04-21T08:49:00.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.246+0000 7fd8561f2740 1 Processor -- start 2024-04-21T08:49:00.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.246+0000 7fd8561f2740 1 -- start start 2024-04-21T08:49:00.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.246+0000 7fd8561f2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbfef84d0 0x557bbff0d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.246+0000 7fd8561f2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bbfef8e80 0x557bbfefd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.246+0000 7fd8561f2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bbff01a00 0x557bbff00dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.246+0000 7fd8561f2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1af450 con 0x557bbfef84d0 2024-04-21T08:49:00.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.246+0000 7fd8561f2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1b0f90 con 0x557bbfef8e80 2024-04-21T08:49:00.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.246+0000 7fd8561f2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1afb20 con 0x557bbff01a00 2024-04-21T08:49:00.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.246+0000 7fd851ac1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbfef84d0 0x557bbff0d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.246+0000 7fd851ac1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbfef84d0 0x557bbff0d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48030/0 (socket says 172.21.15.43:48030) 2024-04-21T08:49:00.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.246+0000 7fd851ac1640 1 -- 172.21.15.43:0/4102703140 learned_addr learned my addr 172.21.15.43:0/4102703140 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:00.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.246+0000 7fd851ac1640 1 -- 172.21.15.43:0/4102703140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bbff01a00 msgr2=0x557bbff00dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:00.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.246+0000 7fd851ac1640 1 --2- 172.21.15.43:0/4102703140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bbff01a00 0x557bbff00dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.246+0000 7fd851ac1640 1 -- 172.21.15.43:0/4102703140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bbfef8e80 msgr2=0x557bbfefd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:00.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.246+0000 7fd850abf640 1 --2- 172.21.15.43:0/4102703140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bbfef8e80 0x557bbfefd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.246+0000 7fd851ac1640 1 --2- 172.21.15.43:0/4102703140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bbfef8e80 0x557bbfefd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.246+0000 7fd851ac1640 1 -- 172.21.15.43:0/4102703140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bbfef8c20 con 0x557bbfef84d0 2024-04-21T08:49:00.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.247+0000 7fd851ac1640 1 --2- 172.21.15.43:0/4102703140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbfef84d0 0x557bbff0d860 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fd84400adf0 tx=0x7fd844004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d675fff663ee68 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:00.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.247+0000 7fd836ffd640 1 -- 172.21.15.43:0/4102703140 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd844021130 con 0x557bbfef84d0 2024-04-21T08:49:00.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.247+0000 7fd836ffd640 1 -- 172.21.15.43:0/4102703140 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd844027020 con 0x557bbfef84d0 2024-04-21T08:49:00.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.247+0000 7fd836ffd640 1 -- 172.21.15.43:0/4102703140 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8440234a0 con 0x557bbfef84d0 2024-04-21T08:49:00.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.247+0000 7fd8561f2740 1 -- 172.21.15.43:0/4102703140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbfef84d0 msgr2=0x557bbff0d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.247+0000 7fd8561f2740 1 --2- 172.21.15.43:0/4102703140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbfef84d0 0x557bbff0d860 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fd84400adf0 tx=0x7fd844004a40 comp rx=0 tx=0).stop 2024-04-21T08:49:00.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.247+0000 7fd8561f2740 1 -- 172.21.15.43:0/4102703140 shutdown_connections 2024-04-21T08:49:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.247+0000 7fd8561f2740 1 --2- 172.21.15.43:0/4102703140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bbff01a00 0x557bbff00dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.247+0000 7fd8561f2740 1 --2- 172.21.15.43:0/4102703140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bbfef8e80 0x557bbfefd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.247+0000 7fd8561f2740 1 --2- 172.21.15.43:0/4102703140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbfef84d0 0x557bbff0d860 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.247+0000 7fd8561f2740 1 -- 172.21.15.43:0/4102703140 >> 172.21.15.43:0/4102703140 conn(0x557bbfef2ba0 msgr2=0x557bbfefcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.247+0000 7fd8561f2740 1 -- 172.21.15.43:0/4102703140 shutdown_connections 2024-04-21T08:49:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.247+0000 7fd8561f2740 1 -- 172.21.15.43:0/4102703140 wait complete. 2024-04-21T08:49:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.248+0000 7fd8561f2740 1 -- start start 2024-04-21T08:49:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.248+0000 7fd8561f2740 1 Processor -- start 2024-04-21T08:49:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.248+0000 7fd8561f2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bbfef84d0 0x557bbff04d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.248+0000 7fd8561f2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bbfef8e80 0x557bbff05270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.248+0000 7fd8561f2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbff01a00 0x557bbff057d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.248+0000 7fd8561f2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1b08c0 con 0x557bbff01a00 2024-04-21T08:49:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.248+0000 7fd8561f2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1b01f0 con 0x557bbfef84d0 2024-04-21T08:49:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.248+0000 7fd8561f2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1ad450 con 0x557bbfef8e80 2024-04-21T08:49:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.248+0000 7fd8561f2740 10 client.0 _ll_register_callbacks cb 0x7fd84400af10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.248+0000 7fd851ac1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbff01a00 0x557bbff057d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.248+0000 7fd851ac1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbff01a00 0x557bbff057d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48046/0 (socket says 172.21.15.43:48046) 2024-04-21T08:49:00.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.248+0000 7fd851ac1640 1 -- 172.21.15.43:0/3610000036 learned_addr learned my addr 172.21.15.43:0/3610000036 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:00.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.248+0000 7fd851ac1640 1 --2- 172.21.15.43:0/3610000036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bbfef84d0 0x557bbff04d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.248+0000 7fd850abf640 1 --2- 172.21.15.43:0/3610000036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bbfef8e80 0x557bbff05270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.249+0000 7fd851ac1640 1 -- 172.21.15.43:0/3610000036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bbfef8e80 msgr2=0x557bbff05270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:00.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.249+0000 7fd851ac1640 1 --2- 172.21.15.43:0/3610000036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bbfef8e80 0x557bbff05270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.249+0000 7fd851ac1640 1 -- 172.21.15.43:0/3610000036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bbfef84d0 msgr2=0x557bbff04d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:00.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.249+0000 7fd851ac1640 1 --2- 172.21.15.43:0/3610000036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bbfef84d0 0x557bbff04d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.249+0000 7fd851ac1640 1 -- 172.21.15.43:0/3610000036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bbff760d0 con 0x557bbff01a00 2024-04-21T08:49:00.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.249+0000 7fd851ac1640 1 --2- 172.21.15.43:0/3610000036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbff01a00 0x557bbff057d0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fd844021310 tx=0x7fd844025040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2fb1c22dc15f4b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:00.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.249+0000 7fd834ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:00.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.249+0000 7fd834ff9640 1 -- 172.21.15.43:0/3610000036 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd844031020 con 0x557bbff01a00 2024-04-21T08:49:00.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.249+0000 7fd834ff9640 1 -- 172.21.15.43:0/3610000036 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fd84401b040 con 0x557bbff01a00 2024-04-21T08:49:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.249+0000 7fd8561f2740 1 -- 172.21.15.43:0/3610000036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd844027020 con 0x557bbff01a00 2024-04-21T08:49:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.249+0000 7fd834ff9640 1 -- 172.21.15.43:0/3610000036 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd844023880 con 0x557bbff01a00 2024-04-21T08:49:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.249+0000 7fd8561f2740 10 client.20418 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.249+0000 7fd8561f2740 1 -- 172.21.15.43:0/3610000036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557bbff773d0 con 0x557bbff01a00 2024-04-21T08:49:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.249+0000 7fd834ff9640 1 -- 172.21.15.43:0/3610000036 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd844027020 con 0x557bbff01a00 2024-04-21T08:49:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.249+0000 7fd834ff9640 1 -- 172.21.15.43:0/3610000036 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd84402d820 con 0x557bbff01a00 2024-04-21T08:49:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.250+0000 7fd8561f2740 10 client.20418 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.250+0000 7fd8561f2740 10 client.20418 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:49:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.250+0000 7fd8561f2740 20 client.20418 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.250+0000 7fd8561f2740 1 --2- 172.21.15.43:0/3610000036 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x557bbff77a40 0x557bbff79e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.250+0000 7fd8561f2740 4 client.20418 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.250+0000 7fd8561f2740 1 -- 172.21.15.43:0/3610000036 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557bbfe6ac10 con 0x557bbff77a40 2024-04-21T08:49:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.250+0000 7fd850abf640 1 --2- 172.21.15.43:0/3610000036 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x557bbff77a40 0x557bbff79e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.250+0000 7fd834ff9640 10 client.20418 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:49:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.250+0000 7fd850abf640 1 --2- 172.21.15.43:0/3610000036 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x557bbff77a40 0x557bbff79e60 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38ba7191ce0347b1 server_cookie=dbec0eaeddc33da9 in_seq=0 out_seq=0 2024-04-21T08:49:00.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.250+0000 7fd834ff9640 1 -- 172.21.15.43:0/3610000036 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557bbfe6ac10 con 0x557bbff77a40 2024-04-21T08:49:00.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.250+0000 7fd834ff9640 10 client.20418 handle_command_reply: tid=0 2024-04-21T08:49:00.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.250+0000 7fd8561f2740 1 client.20418 shutdown 2024-04-21T08:49:00.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.251+0000 7fd8561f2740 1 -- 172.21.15.43:0/3610000036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbff01a00 msgr2=0x557bbff057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.251+0000 7fd8561f2740 1 --2- 172.21.15.43:0/3610000036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbff01a00 0x557bbff057d0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fd844021310 tx=0x7fd844025040 comp rx=0 tx=0).stop 2024-04-21T08:49:00.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.251+0000 7fd8561f2740 1 -- 172.21.15.43:0/3610000036 shutdown_connections 2024-04-21T08:49:00.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.251+0000 7fd8561f2740 1 --2- 172.21.15.43:0/3610000036 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x557bbff77a40 0x557bbff79e60 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.251+0000 7fd8561f2740 1 --2- 172.21.15.43:0/3610000036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bbff01a00 0x557bbff057d0 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.251+0000 7fd834ff9640 0 client.20418 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:49:00.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.251+0000 7fd8561f2740 1 --2- 172.21.15.43:0/3610000036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bbfef8e80 0x557bbff05270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.251+0000 7fd8561f2740 1 --2- 172.21.15.43:0/3610000036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bbfef84d0 0x557bbff04d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.251+0000 7fd8561f2740 1 -- 172.21.15.43:0/3610000036 >> 172.21.15.43:0/3610000036 conn(0x557bbfef2ba0 msgr2=0x557bbfef3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:00.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.251+0000 7fd8561f2740 1 -- 172.21.15.43:0/3610000036 shutdown_connections 2024-04-21T08:49:00.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.251+0000 7fd8561f2740 1 -- 172.21.15.43:0/3610000036 wait complete. 2024-04-21T08:49:00.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.251+0000 7fd8561f2740 20 client.20418 trim_cache size 0 max 16384 2024-04-21T08:49:00.257 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:49:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.251+0000 7fd853524640 1 -- 172.21.15.43:0/2670861123 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd8380a3d20 msgr2=0x7fd8380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.251+0000 7fd853524640 1 --2- 172.21.15.43:0/2670861123 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd8380a3d20 0x7fd8380a61e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd8480f0770 tx=0x7fd84803ac10 comp rx=0 tx=0).stop 2024-04-21T08:49:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.251+0000 7fd853524640 1 -- 172.21.15.43:0/2670861123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd84c102190 msgr2=0x7fd84c198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.251+0000 7fd853524640 1 --2- 172.21.15.43:0/2670861123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd84c102190 0x7fd84c198700 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fd848005940 tx=0x7fd84802c7e0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.251+0000 7fd853524640 1 -- 172.21.15.43:0/2670861123 shutdown_connections 2024-04-21T08:49:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.251+0000 7fd853524640 1 --2- 172.21.15.43:0/2670861123 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd8380a3d20 0x7fd8380a61e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.251+0000 7fd853524640 1 --2- 172.21.15.43:0/2670861123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd84c19cfc0 0x7fd84c19d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.251+0000 7fd853524640 1 --2- 172.21.15.43:0/2670861123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd84c107260 0x7fd84c198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.251+0000 7fd853524640 1 --2- 172.21.15.43:0/2670861123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd84c102190 0x7fd84c198700 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.251+0000 7fd853524640 1 -- 172.21.15.43:0/2670861123 >> 172.21.15.43:0/2670861123 conn(0x7fd84c0793a0 msgr2=0x7fd84c0fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.251+0000 7fd853524640 1 -- 172.21.15.43:0/2670861123 shutdown_connections 2024-04-21T08:49:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.251+0000 7fd853524640 1 -- 172.21.15.43:0/2670861123 wait complete. 2024-04-21T08:49:00.261 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:00 smithi043 ceph-mon[27978]: pgmap v312: 129 pgs: 129 active+clean; 4.6 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:49:00.261 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:00 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:49:00.261 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:00 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:00.261 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:00 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:49:00.261 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:00 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:00.261 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:00 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:49:00.261 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:00 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/494020366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:49:00.262 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:49:00.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.367+0000 7f8517998640 1 -- 172.21.15.43:0/242168062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85101031d0 msgr2=0x7f85101035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.367+0000 7f8517998640 1 --2- 172.21.15.43:0/242168062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85101031d0 0x7f85101035b0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f850c009ca0 tx=0x7f850c02c230 comp rx=0 tx=0).stop 2024-04-21T08:49:00.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.367+0000 7f8517998640 1 -- 172.21.15.43:0/242168062 shutdown_connections 2024-04-21T08:49:00.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.367+0000 7f8517998640 1 --2- 172.21.15.43:0/242168062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f851010c510 0x7f851010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.367+0000 7f8517998640 1 --2- 172.21.15.43:0/242168062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8510103af0 0x7f851010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.367+0000 7f8517998640 1 --2- 172.21.15.43:0/242168062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85101031d0 0x7f85101035b0 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.367+0000 7f8517998640 1 -- 172.21.15.43:0/242168062 >> 172.21.15.43:0/242168062 conn(0x7f8510077f50 msgr2=0x7f8510078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:00.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.367+0000 7f8517998640 1 -- 172.21.15.43:0/242168062 shutdown_connections 2024-04-21T08:49:00.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.367+0000 7f8517998640 1 -- 172.21.15.43:0/242168062 wait complete. 2024-04-21T08:49:00.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.367+0000 7f8517998640 1 Processor -- start 2024-04-21T08:49:00.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.367+0000 7f8517998640 1 -- start start 2024-04-21T08:49:00.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.368+0000 7f8517998640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85101031d0 0x7f8510100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.368+0000 7f8517998640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8510103af0 0x7f8510101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.368+0000 7f8517998640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f851010c510 0x7f85101018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.368+0000 7f8517998640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f85101121a0 con 0x7f85101031d0 2024-04-21T08:49:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.368+0000 7f8517998640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8510112020 con 0x7f8510103af0 2024-04-21T08:49:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.368+0000 7f8517998640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8510112320 con 0x7f851010c510 2024-04-21T08:49:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.368+0000 7f8515734640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85101031d0 0x7f8510100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.368+0000 7f8515734640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85101031d0 0x7f8510100e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48068/0 (socket says 172.21.15.43:48068) 2024-04-21T08:49:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.368+0000 7f8515734640 1 -- 172.21.15.43:0/114947377 learned_addr learned my addr 172.21.15.43:0/114947377 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.368+0000 7f8514f33640 1 --2- 172.21.15.43:0/114947377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8510103af0 0x7f8510101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.368+0000 7f8515f35640 1 --2- 172.21.15.43:0/114947377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f851010c510 0x7f85101018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.368+0000 7f8515734640 1 -- 172.21.15.43:0/114947377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f851010c510 msgr2=0x7f85101018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.368+0000 7f8515734640 1 --2- 172.21.15.43:0/114947377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f851010c510 0x7f85101018a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.368+0000 7f8515734640 1 -- 172.21.15.43:0/114947377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8510103af0 msgr2=0x7f8510101360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.368+0000 7f8515734640 1 --2- 172.21.15.43:0/114947377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8510103af0 0x7f8510101360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.368+0000 7f8515734640 1 -- 172.21.15.43:0/114947377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85101a6cf0 con 0x7f85101031d0 2024-04-21T08:49:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.368+0000 7f8515734640 1 --2- 172.21.15.43:0/114947377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85101031d0 0x7f8510100e20 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f850c005e30 tx=0x7f850c035ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.368+0000 7f85067fc640 1 -- 172.21.15.43:0/114947377 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c039040 con 0x7f85101031d0 2024-04-21T08:49:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.369+0000 7f85067fc640 1 -- 172.21.15.43:0/114947377 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f850c044070 con 0x7f85101031d0 2024-04-21T08:49:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.369+0000 7f8517998640 1 -- 172.21.15.43:0/114947377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85101a6f80 con 0x7f85101031d0 2024-04-21T08:49:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.369+0000 7f85067fc640 1 -- 172.21.15.43:0/114947377 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c032880 con 0x7f85101031d0 2024-04-21T08:49:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.369+0000 7f8517998640 1 -- 172.21.15.43:0/114947377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85101a73e0 con 0x7f85101031d0 2024-04-21T08:49:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.371+0000 7f85067fc640 1 -- 172.21.15.43:0/114947377 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f850c032ae0 con 0x7f85101031d0 2024-04-21T08:49:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.371+0000 7f851a582740 1 Processor -- start 2024-04-21T08:49:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.371+0000 7f851a582740 1 -- start start 2024-04-21T08:49:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.371+0000 7f851a582740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2a2aff4d0 0x55d2a2aff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.372+0000 7f851a582740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2affe80 0x55d2a2b08430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.372+0000 7f851a582740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a2b08a00 0x55d2a2b0ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.372+0000 7f85067fc640 1 --2- 172.21.15.43:0/114947377 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84fc0a3d20 0x7f84fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.372+0000 7f851a582740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d2a2a641f0 con 0x55d2a2affe80 2024-04-21T08:49:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.372+0000 7f851a582740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d2a2a014b0 con 0x55d2a2aff4d0 2024-04-21T08:49:00.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.372+0000 7f851a582740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d2a2a601b0 con 0x55d2a2b08a00 2024-04-21T08:49:00.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.372+0000 7f8515734640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2affe80 0x55d2a2b08430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.372+0000 7f8515f35640 1 --2- 172.21.15.43:0/114947377 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84fc0a3d20 0x7f84fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.372+0000 7f8515734640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2affe80 0x55d2a2b08430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48078/0 (socket says 172.21.15.43:48078) 2024-04-21T08:49:00.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.372+0000 7f8515734640 1 -- 172.21.15.43:0/1330426507 learned_addr learned my addr 172.21.15.43:0/1330426507 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:00.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.372+0000 7f8514f33640 1 --2- 172.21.15.43:0/1330426507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a2b08a00 0x55d2a2b0ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.372+0000 7f85067fc640 1 -- 172.21.15.43:0/114947377 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f850c0f2fe0 con 0x7f85101031d0 2024-04-21T08:49:00.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.372+0000 7f8515f35640 1 --2- 172.21.15.43:0/1330426507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2a2aff4d0 0x55d2a2aff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.372+0000 7f8514f33640 1 -- 172.21.15.43:0/1330426507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2a2aff4d0 msgr2=0x55d2a2aff8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:00.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.372+0000 7f8514f33640 1 --2- 172.21.15.43:0/1330426507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2a2aff4d0 0x55d2a2aff8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.372+0000 7f8514f33640 1 -- 172.21.15.43:0/1330426507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2affe80 msgr2=0x55d2a2b08430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:00.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.372+0000 7f8514f33640 1 --2- 172.21.15.43:0/1330426507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2affe80 0x55d2a2b08430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.372+0000 7f8514f33640 1 -- 172.21.15.43:0/1330426507 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a29f31b0 con 0x55d2a2b08a00 2024-04-21T08:49:00.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.372+0000 7f8515734640 1 --2- 172.21.15.43:0/1330426507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2affe80 0x55d2a2b08430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:00.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.372+0000 7f8515f35640 1 --2- 172.21.15.43:0/114947377 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84fc0a3d20 0x7f84fc0a61e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8508005d80 tx=0x7f8508013830 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:00.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.372+0000 7f8514f33640 1 --2- 172.21.15.43:0/1330426507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a2b08a00 0x55d2a2b0ae50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8500009b70 tx=0x7f8500011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=222274857f96a567 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:00.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.373+0000 7f84fb7fe640 1 -- 172.21.15.43:0/1330426507 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500004050 con 0x55d2a2b08a00 2024-04-21T08:49:00.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.373+0000 7f84fb7fe640 1 -- 172.21.15.43:0/1330426507 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85000054d0 con 0x55d2a2b08a00 2024-04-21T08:49:00.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.373+0000 7f84fb7fe640 1 -- 172.21.15.43:0/1330426507 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500004cd0 con 0x55d2a2b08a00 2024-04-21T08:49:00.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.373+0000 7f851a582740 1 -- 172.21.15.43:0/1330426507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a2b08a00 msgr2=0x55d2a2b0ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.373+0000 7f851a582740 1 --2- 172.21.15.43:0/1330426507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a2b08a00 0x55d2a2b0ae50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8500009b70 tx=0x7f8500011810 comp rx=0 tx=0).stop 2024-04-21T08:49:00.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.373+0000 7f851a582740 1 -- 172.21.15.43:0/1330426507 shutdown_connections 2024-04-21T08:49:00.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.373+0000 7f851a582740 1 --2- 172.21.15.43:0/1330426507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a2b08a00 0x55d2a2b0ae50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.373+0000 7f851a582740 1 --2- 172.21.15.43:0/1330426507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2affe80 0x55d2a2b08430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.373+0000 7f851a582740 1 --2- 172.21.15.43:0/1330426507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2a2aff4d0 0x55d2a2aff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.374+0000 7f851a582740 1 -- 172.21.15.43:0/1330426507 >> 172.21.15.43:0/1330426507 conn(0x55d2a2af9ba0 msgr2=0x55d2a2af9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:00.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.374+0000 7f851a582740 1 -- 172.21.15.43:0/1330426507 shutdown_connections 2024-04-21T08:49:00.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.374+0000 7f851a582740 1 -- 172.21.15.43:0/1330426507 wait complete. 2024-04-21T08:49:00.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.374+0000 7f851a582740 1 -- start start 2024-04-21T08:49:00.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.374+0000 7f851a582740 1 Processor -- start 2024-04-21T08:49:00.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.374+0000 7f851a582740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a2aff4d0 0x55d2a2b08260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.374+0000 7f851a582740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2a2affe80 0x55d2a2b055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.374+0000 7f851a582740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2b08a00 0x55d2a2b05b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.374+0000 7f851a582740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d2a2b0b970 con 0x55d2a2b08a00 2024-04-21T08:49:00.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.374+0000 7f851a582740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d2a2a1d500 con 0x55d2a2affe80 2024-04-21T08:49:00.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.374+0000 7f851a582740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d2a2b0baf0 con 0x55d2a2aff4d0 2024-04-21T08:49:00.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.374+0000 7f851a582740 10 client.0 _ll_register_callbacks cb 0x55d2a29ebb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:00.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.375+0000 7f8515f35640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2b08a00 0x55d2a2b05b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.375+0000 7f8515f35640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2b08a00 0x55d2a2b05b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48090/0 (socket says 172.21.15.43:48090) 2024-04-21T08:49:00.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.375+0000 7f8515f35640 1 -- 172.21.15.43:0/4224824007 learned_addr learned my addr 172.21.15.43:0/4224824007 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:00.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.375+0000 7f8515734640 1 --2- 172.21.15.43:0/4224824007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a2aff4d0 0x55d2a2b08260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.375+0000 7f8514f33640 1 --2- 172.21.15.43:0/4224824007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2a2affe80 0x55d2a2b055f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.375+0000 7f8515f35640 1 -- 172.21.15.43:0/4224824007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a2aff4d0 msgr2=0x55d2a2b08260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:00.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.375+0000 7f8515f35640 1 --2- 172.21.15.43:0/4224824007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a2aff4d0 0x55d2a2b08260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.375+0000 7f8515f35640 1 -- 172.21.15.43:0/4224824007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2a2affe80 msgr2=0x55d2a2b055f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:00.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.375+0000 7f8515f35640 1 --2- 172.21.15.43:0/4224824007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2a2affe80 0x55d2a2b055f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.375+0000 7f8515f35640 1 -- 172.21.15.43:0/4224824007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2a27763f0 con 0x55d2a2b08a00 2024-04-21T08:49:00.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.375+0000 7f8515734640 1 --2- 172.21.15.43:0/4224824007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a2aff4d0 0x55d2a2b08260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:00.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.375+0000 7f8515f35640 1 --2- 172.21.15.43:0/4224824007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2b08a00 0x55d2a2b05b30 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f8508011bb0 tx=0x7f8508019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17052045c6ff7f80 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:00.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.375+0000 7f84f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:00.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.375+0000 7f84f8ff9640 1 -- 172.21.15.43:0/4224824007 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508023030 con 0x55d2a2b08a00 2024-04-21T08:49:00.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.375+0000 7f84f8ff9640 1 -- 172.21.15.43:0/4224824007 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f850800fa10 con 0x55d2a2b08a00 2024-04-21T08:49:00.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.375+0000 7f851a582740 1 -- 172.21.15.43:0/4224824007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85000054d0 con 0x55d2a2b08a00 2024-04-21T08:49:00.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.375+0000 7f84f8ff9640 1 -- 172.21.15.43:0/4224824007 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8508030050 con 0x55d2a2b08a00 2024-04-21T08:49:00.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.375+0000 7f851a582740 10 client.20430 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:00.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.375+0000 7f851a582740 1 -- 172.21.15.43:0/4224824007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d2a28407d0 con 0x55d2a2b08a00 2024-04-21T08:49:00.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.376+0000 7f84f8ff9640 1 -- 172.21.15.43:0/4224824007 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85000054d0 con 0x55d2a2b08a00 2024-04-21T08:49:00.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.376+0000 7f84f8ff9640 1 -- 172.21.15.43:0/4224824007 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f850801fcb0 con 0x55d2a2b08a00 2024-04-21T08:49:00.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.376+0000 7f851a582740 10 client.20430 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:00.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.376+0000 7f851a582740 10 client.20430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:00.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.376+0000 7f851a582740 20 client.20430 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:00.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.376+0000 7f851a582740 1 --2- 172.21.15.43:0/4224824007 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d2a2b18bc0 0x55d2a2b1b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.376+0000 7f851a582740 4 client.20430 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:00.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.376+0000 7f851a582740 1 -- 172.21.15.43:0/4224824007 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2a2a03930 con 0x55d2a2b18bc0 2024-04-21T08:49:00.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.376+0000 7f8515734640 1 --2- 172.21.15.43:0/4224824007 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d2a2b18bc0 0x55d2a2b1b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.377+0000 7f84f8ff9640 10 client.20430 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:00.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.377+0000 7f8515734640 1 --2- 172.21.15.43:0/4224824007 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d2a2b18bc0 0x55d2a2b1b040 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bec4782dae502c0 server_cookie=5236ce7acf81fb7c in_seq=0 out_seq=0 2024-04-21T08:49:00.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.377+0000 7f84f8ff9640 1 -- 172.21.15.43:0/4224824007 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d2a2a03930 con 0x55d2a2b18bc0 2024-04-21T08:49:00.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.377+0000 7f84f8ff9640 10 client.20430 handle_command_reply: tid=0 2024-04-21T08:49:00.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.377+0000 7f851a582740 1 client.20430 shutdown 2024-04-21T08:49:00.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.377+0000 7f851a582740 1 -- 172.21.15.43:0/4224824007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2b08a00 msgr2=0x55d2a2b05b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.377+0000 7f851a582740 1 --2- 172.21.15.43:0/4224824007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2b08a00 0x55d2a2b05b30 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f8508011bb0 tx=0x7f8508019000 comp rx=0 tx=0).stop 2024-04-21T08:49:00.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.377+0000 7f851a582740 1 -- 172.21.15.43:0/4224824007 shutdown_connections 2024-04-21T08:49:00.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.377+0000 7f851a582740 1 --2- 172.21.15.43:0/4224824007 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d2a2b18bc0 0x55d2a2b1b040 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.377+0000 7f851a582740 1 --2- 172.21.15.43:0/4224824007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2b08a00 0x55d2a2b05b30 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.378+0000 7f84f8ff9640 0 client.20430 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:00.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.378+0000 7f851a582740 1 --2- 172.21.15.43:0/4224824007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2a2affe80 0x55d2a2b055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.378+0000 7f851a582740 1 --2- 172.21.15.43:0/4224824007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a2aff4d0 0x55d2a2b08260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.378+0000 7f851a582740 1 -- 172.21.15.43:0/4224824007 >> 172.21.15.43:0/4224824007 conn(0x55d2a2af9ba0 msgr2=0x55d2a2afb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:00.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.378+0000 7f851a582740 1 -- 172.21.15.43:0/4224824007 shutdown_connections 2024-04-21T08:49:00.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.378+0000 7f851a582740 1 -- 172.21.15.43:0/4224824007 wait complete. 2024-04-21T08:49:00.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.378+0000 7f851a582740 20 client.20430 trim_cache size 0 max 16384 2024-04-21T08:49:00.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.391+0000 7f851a582740 1 Processor -- start 2024-04-21T08:49:00.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.391+0000 7f851a582740 1 -- start start 2024-04-21T08:49:00.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.391+0000 7f851a582740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2a2af9ba0 0x55d2a2b14830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.391+0000 7f851a582740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2aff4d0 0x55d2a2b07d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.391+0000 7f851a582740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a2affe80 0x55d2a2b08260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.391+0000 7f851a582740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f85101b1740 con 0x55d2a2aff4d0 2024-04-21T08:49:00.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.391+0000 7f851a582740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85101b3280 con 0x55d2a2af9ba0 2024-04-21T08:49:00.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.391+0000 7f851a582740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f85101b1e10 con 0x55d2a2affe80 2024-04-21T08:49:00.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.391+0000 7f8514f33640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2aff4d0 0x55d2a2b07d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.391+0000 7f8514f33640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2aff4d0 0x55d2a2b07d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48104/0 (socket says 172.21.15.43:48104) 2024-04-21T08:49:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.391+0000 7f8514f33640 1 -- 172.21.15.43:0/1354916428 learned_addr learned my addr 172.21.15.43:0/1354916428 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.391+0000 7f8515734640 1 --2- 172.21.15.43:0/1354916428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2a2af9ba0 0x55d2a2b14830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.392+0000 7f8515f35640 1 --2- 172.21.15.43:0/1354916428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a2affe80 0x55d2a2b08260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.392+0000 7f8514f33640 1 -- 172.21.15.43:0/1354916428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a2affe80 msgr2=0x55d2a2b08260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.392+0000 7f8514f33640 1 --2- 172.21.15.43:0/1354916428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a2affe80 0x55d2a2b08260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.392+0000 7f8514f33640 1 -- 172.21.15.43:0/1354916428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2a2af9ba0 msgr2=0x55d2a2b14830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.392+0000 7f8514f33640 1 --2- 172.21.15.43:0/1354916428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2a2af9ba0 0x55d2a2b14830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.392+0000 7f8514f33640 1 -- 172.21.15.43:0/1354916428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a2b03b40 con 0x55d2a2aff4d0 2024-04-21T08:49:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.392+0000 7f8515f35640 1 --2- 172.21.15.43:0/1354916428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a2affe80 0x55d2a2b08260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.392+0000 7f8514f33640 1 --2- 172.21.15.43:0/1354916428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2aff4d0 0x55d2a2b07d20 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f8500009b40 tx=0x7f8500009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6444b0fea26f1429 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.392+0000 7f84faffd640 1 -- 172.21.15.43:0/1354916428 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500028020 con 0x55d2a2aff4d0 2024-04-21T08:49:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.392+0000 7f84faffd640 1 -- 172.21.15.43:0/1354916428 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f850001f040 con 0x55d2a2aff4d0 2024-04-21T08:49:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.392+0000 7f84faffd640 1 -- 172.21.15.43:0/1354916428 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500003ab0 con 0x55d2a2aff4d0 2024-04-21T08:49:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.392+0000 7f851a582740 1 -- 172.21.15.43:0/1354916428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2aff4d0 msgr2=0x55d2a2b07d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.393+0000 7f851a582740 1 --2- 172.21.15.43:0/1354916428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2aff4d0 0x55d2a2b07d20 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f8500009b40 tx=0x7f8500009890 comp rx=0 tx=0).stop 2024-04-21T08:49:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.393+0000 7f851a582740 1 -- 172.21.15.43:0/1354916428 shutdown_connections 2024-04-21T08:49:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.393+0000 7f851a582740 1 --2- 172.21.15.43:0/1354916428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a2affe80 0x55d2a2b08260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.393+0000 7f851a582740 1 --2- 172.21.15.43:0/1354916428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2aff4d0 0x55d2a2b07d20 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.393+0000 7f851a582740 1 --2- 172.21.15.43:0/1354916428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2a2af9ba0 0x55d2a2b14830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.393+0000 7f851a582740 1 -- 172.21.15.43:0/1354916428 >> 172.21.15.43:0/1354916428 conn(0x55d2a2b05c90 msgr2=0x55d2a2af8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.393+0000 7f851a582740 1 -- 172.21.15.43:0/1354916428 shutdown_connections 2024-04-21T08:49:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.393+0000 7f851a582740 1 -- 172.21.15.43:0/1354916428 wait complete. 2024-04-21T08:49:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.393+0000 7f851a582740 1 -- start start 2024-04-21T08:49:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.393+0000 7f851a582740 1 Processor -- start 2024-04-21T08:49:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.393+0000 7f851a582740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2af9ba0 0x55d2a2b02510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.393+0000 7f851a582740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a2aff4d0 0x55d2a2b02a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.393+0000 7f851a582740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2a2affe80 0x55d2a2b02f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.393+0000 7f851a582740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f85101b2bb0 con 0x55d2a2af9ba0 2024-04-21T08:49:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.393+0000 7f851a582740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85101b24e0 con 0x55d2a2affe80 2024-04-21T08:49:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.393+0000 7f851a582740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f85101af740 con 0x55d2a2aff4d0 2024-04-21T08:49:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.393+0000 7f851a582740 10 client.0 _ll_register_callbacks cb 0x7f8508011880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.393+0000 7f8515734640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2af9ba0 0x55d2a2b02510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.394+0000 7f8515734640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2af9ba0 0x55d2a2b02510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48110/0 (socket says 172.21.15.43:48110) 2024-04-21T08:49:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.394+0000 7f8515734640 1 -- 172.21.15.43:0/438923787 learned_addr learned my addr 172.21.15.43:0/438923787 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.394+0000 7f8514f33640 1 --2- 172.21.15.43:0/438923787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a2aff4d0 0x55d2a2b02a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.394+0000 7f8515f35640 1 --2- 172.21.15.43:0/438923787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2a2affe80 0x55d2a2b02f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.394+0000 7f8515734640 1 -- 172.21.15.43:0/438923787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a2aff4d0 msgr2=0x55d2a2b02a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.394+0000 7f8515734640 1 --2- 172.21.15.43:0/438923787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a2aff4d0 0x55d2a2b02a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.394+0000 7f8515734640 1 -- 172.21.15.43:0/438923787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2a2affe80 msgr2=0x55d2a2b02f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.394+0000 7f8515734640 1 --2- 172.21.15.43:0/438923787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2a2affe80 0x55d2a2b02f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.394+0000 7f8515734640 1 -- 172.21.15.43:0/438923787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2a2b7e3d0 con 0x55d2a2af9ba0 2024-04-21T08:49:00.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.394+0000 7f8515734640 1 --2- 172.21.15.43:0/438923787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2af9ba0 0x55d2a2b02510 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f850c0f04d0 tx=0x7f850c0c2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e3c2eaaa95f8a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:00.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.394+0000 7f84f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:00.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.394+0000 7f84f8ff9640 1 -- 172.21.15.43:0/438923787 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c0bec70 con 0x55d2a2af9ba0 2024-04-21T08:49:00.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.394+0000 7f84f8ff9640 1 -- 172.21.15.43:0/438923787 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f850c0bcf70 con 0x55d2a2af9ba0 2024-04-21T08:49:00.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.394+0000 7f851a582740 1 -- 172.21.15.43:0/438923787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f850001f040 con 0x55d2a2af9ba0 2024-04-21T08:49:00.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.394+0000 7f84f8ff9640 1 -- 172.21.15.43:0/438923787 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f850c0c3020 con 0x55d2a2af9ba0 2024-04-21T08:49:00.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.394+0000 7f851a582740 10 client.20436 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:00.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.394+0000 7f851a582740 1 -- 172.21.15.43:0/438923787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d2a2b7f620 con 0x55d2a2af9ba0 2024-04-21T08:49:00.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.394+0000 7f84f8ff9640 1 -- 172.21.15.43:0/438923787 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f850001f040 con 0x55d2a2af9ba0 2024-04-21T08:49:00.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.395+0000 7f84f8ff9640 1 -- 172.21.15.43:0/438923787 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f850c03c030 con 0x55d2a2af9ba0 2024-04-21T08:49:00.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.395+0000 7f851a582740 10 client.20436 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:00.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.395+0000 7f851a582740 10 client.20436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:00.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.395+0000 7f851a582740 20 client.20436 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:00.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.395+0000 7f851a582740 1 --2- 172.21.15.43:0/438923787 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d2a2b08a00 0x55d2a2b81d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:00.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.395+0000 7f851a582740 4 client.20436 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:00.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.395+0000 7f851a582740 1 -- 172.21.15.43:0/438923787 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d2a2a71c10 con 0x55d2a2b08a00 2024-04-21T08:49:00.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.395+0000 7f8514f33640 1 --2- 172.21.15.43:0/438923787 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d2a2b08a00 0x55d2a2b81d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:00.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.395+0000 7f8514f33640 1 --2- 172.21.15.43:0/438923787 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d2a2b08a00 0x55d2a2b81d60 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18a11c351001a13c server_cookie=3cab183196d755fc in_seq=0 out_seq=0 2024-04-21T08:49:00.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.395+0000 7f84f8ff9640 10 client.20436 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:00.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f84f8ff9640 1 -- 172.21.15.43:0/438923787 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d2a2a71c10 con 0x55d2a2b08a00 2024-04-21T08:49:00.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f84f8ff9640 10 client.20436 handle_command_reply: tid=0 2024-04-21T08:49:00.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f851a582740 1 client.20436 shutdown 2024-04-21T08:49:00.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f851a582740 1 -- 172.21.15.43:0/438923787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2af9ba0 msgr2=0x55d2a2b02510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f851a582740 1 --2- 172.21.15.43:0/438923787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2af9ba0 0x55d2a2b02510 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f850c0f04d0 tx=0x7f850c0c2040 comp rx=0 tx=0).stop 2024-04-21T08:49:00.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f851a582740 1 -- 172.21.15.43:0/438923787 shutdown_connections 2024-04-21T08:49:00.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f851a582740 1 --2- 172.21.15.43:0/438923787 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d2a2b08a00 0x55d2a2b81d60 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f851a582740 1 --2- 172.21.15.43:0/438923787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2a2affe80 0x55d2a2b02f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f851a582740 1 --2- 172.21.15.43:0/438923787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a2aff4d0 0x55d2a2b02a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f851a582740 1 --2- 172.21.15.43:0/438923787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d2a2af9ba0 0x55d2a2b02510 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f84f8ff9640 0 client.20436 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:00.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f851a582740 1 -- 172.21.15.43:0/438923787 >> 172.21.15.43:0/438923787 conn(0x55d2a2b05c90 msgr2=0x55d2a2b00260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:00.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f851a582740 1 -- 172.21.15.43:0/438923787 shutdown_connections 2024-04-21T08:49:00.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f851a582740 1 -- 172.21.15.43:0/438923787 wait complete. 2024-04-21T08:49:00.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f851a582740 20 client.20436 trim_cache size 0 max 16384 2024-04-21T08:49:00.403 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:49:00.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f8517998640 1 -- 172.21.15.43:0/114947377 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84fc0a3d20 msgr2=0x7f84fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f8517998640 1 --2- 172.21.15.43:0/114947377 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84fc0a3d20 0x7f84fc0a61e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8508005d80 tx=0x7f8508013830 comp rx=0 tx=0).stop 2024-04-21T08:49:00.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f8517998640 1 -- 172.21.15.43:0/114947377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85101031d0 msgr2=0x7f8510100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:00.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f8517998640 1 --2- 172.21.15.43:0/114947377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85101031d0 0x7f8510100e20 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f850c005e30 tx=0x7f850c035ea0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f8517998640 1 -- 172.21.15.43:0/114947377 shutdown_connections 2024-04-21T08:49:00.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f8517998640 1 --2- 172.21.15.43:0/114947377 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84fc0a3d20 0x7f84fc0a61e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f8517998640 1 --2- 172.21.15.43:0/114947377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f851010c510 0x7f85101018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f8517998640 1 --2- 172.21.15.43:0/114947377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8510103af0 0x7f8510101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f8517998640 1 --2- 172.21.15.43:0/114947377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85101031d0 0x7f8510100e20 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:00.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f8517998640 1 -- 172.21.15.43:0/114947377 >> 172.21.15.43:0/114947377 conn(0x7f8510077f50 msgr2=0x7f85100ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:00.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.396+0000 7f8517998640 1 -- 172.21.15.43:0/114947377 shutdown_connections 2024-04-21T08:49:00.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:00.397+0000 7f8517998640 1 -- 172.21.15.43:0/114947377 wait complete. 2024-04-21T08:49:00.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:00 smithi175 ceph-mon[31920]: pgmap v312: 129 pgs: 129 active+clean; 4.6 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:49:00.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:00 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:49:00.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:00 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:00.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:00 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:49:00.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:00 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:00.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:00 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:49:00.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:00 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/494020366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:49:00.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:00 smithi112 ceph-mon[31085]: pgmap v312: 129 pgs: 129 active+clean; 4.6 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:49:00.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:00 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:49:00.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:00 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:00.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:00 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:49:00.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:00 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:00.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:00 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:49:00.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:00 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/494020366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:49:01.407 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:49:01.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:49:01.407 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:49:01.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.527+0000 7f381f493640 1 -- 172.21.15.43:0/1241925919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3818106170 msgr2=0x7f381810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:01.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.527+0000 7f381f493640 1 --2- 172.21.15.43:0/1241925919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3818106170 0x7f381810dd40 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f3810009cc0 tx=0x7f381002e310 comp rx=0 tx=0).stop 2024-04-21T08:49:01.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.527+0000 7f381f493640 1 -- 172.21.15.43:0/1241925919 shutdown_connections 2024-04-21T08:49:01.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.527+0000 7f381f493640 1 --2- 172.21.15.43:0/1241925919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3818106170 0x7f381810dd40 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.527+0000 7f381f493640 1 --2- 172.21.15.43:0/1241925919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3818101a50 0x7f3818105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.527+0000 7f381f493640 1 --2- 172.21.15.43:0/1241925919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38181010a0 0x7f3818101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.527+0000 7f381f493640 1 -- 172.21.15.43:0/1241925919 >> 172.21.15.43:0/1241925919 conn(0x7f3818077eb0 msgr2=0x7f38180782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:01.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.528+0000 7f381f493640 1 -- 172.21.15.43:0/1241925919 shutdown_connections 2024-04-21T08:49:01.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.528+0000 7f381f493640 1 -- 172.21.15.43:0/1241925919 wait complete. 2024-04-21T08:49:01.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.528+0000 7f381f493640 1 Processor -- start 2024-04-21T08:49:01.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.528+0000 7f381f493640 1 -- start start 2024-04-21T08:49:01.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.528+0000 7f381f493640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38181010a0 0x7f3818196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.528+0000 7f381f493640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3818106170 0x7f3818196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.528+0000 7f381f493640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f381819ada0 0x7f381819b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.528+0000 7f381f493640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f381810ffd0 con 0x7f38181010a0 2024-04-21T08:49:01.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.528+0000 7f381f493640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f381810fe50 con 0x7f3818106170 2024-04-21T08:49:01.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.528+0000 7f381f493640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3818110150 con 0x7f381819ada0 2024-04-21T08:49:01.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.529+0000 7f381d22f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38181010a0 0x7f3818196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.529+0000 7f381d22f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38181010a0 0x7f3818196530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48124/0 (socket says 172.21.15.43:48124) 2024-04-21T08:49:01.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.529+0000 7f381d22f640 1 -- 172.21.15.43:0/3033862725 learned_addr learned my addr 172.21.15.43:0/3033862725 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:01.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.529+0000 7f381da30640 1 --2- 172.21.15.43:0/3033862725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f381819ada0 0x7f381819b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.529+0000 7f381ca2e640 1 --2- 172.21.15.43:0/3033862725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3818106170 0x7f3818196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.529+0000 7f381d22f640 1 -- 172.21.15.43:0/3033862725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f381819ada0 msgr2=0x7f381819b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:01.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.529+0000 7f381d22f640 1 --2- 172.21.15.43:0/3033862725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f381819ada0 0x7f381819b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.529+0000 7f381d22f640 1 -- 172.21.15.43:0/3033862725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3818106170 msgr2=0x7f3818196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:01.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.529+0000 7f381d22f640 1 --2- 172.21.15.43:0/3033862725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3818106170 0x7f3818196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.529+0000 7f381d22f640 1 -- 172.21.15.43:0/3033862725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f381819b890 con 0x7f38181010a0 2024-04-21T08:49:01.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.529+0000 7f381d22f640 1 --2- 172.21.15.43:0/3033862725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38181010a0 0x7f3818196530 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f381400e9f0 tx=0x7f381400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:01.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.529+0000 7f380e7fc640 1 -- 172.21.15.43:0/3033862725 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381400cd80 con 0x7f38181010a0 2024-04-21T08:49:01.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.529+0000 7f381f493640 1 -- 172.21.15.43:0/3033862725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f381819bb80 con 0x7f38181010a0 2024-04-21T08:49:01.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.529+0000 7f380e7fc640 1 -- 172.21.15.43:0/3033862725 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38140108e0 con 0x7f38181010a0 2024-04-21T08:49:01.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.529+0000 7f380e7fc640 1 -- 172.21.15.43:0/3033862725 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814018980 con 0x7f38181010a0 2024-04-21T08:49:01.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.529+0000 7f381f493640 1 -- 172.21.15.43:0/3033862725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38181a3370 con 0x7f38181010a0 2024-04-21T08:49:01.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.532+0000 7f380e7fc640 1 -- 172.21.15.43:0/3033862725 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3814010430 con 0x7f38181010a0 2024-04-21T08:49:01.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.532+0000 7f3822166740 1 Processor -- start 2024-04-21T08:49:01.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.532+0000 7f3822166740 1 -- start start 2024-04-21T08:49:01.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.532+0000 7f3822166740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629fb5768e0 0x5629fb576cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.532+0000 7f3822166740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629fb577290 0x5629fb57f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.532+0000 7f3822166740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb57fe10 0x5629fb582260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.532+0000 7f3822166740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5629fb4daf00 con 0x5629fb57fe10 2024-04-21T08:49:01.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.532+0000 7f3822166740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5629fb478590 con 0x5629fb5768e0 2024-04-21T08:49:01.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.532+0000 7f3822166740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5629fb4d6ec0 con 0x5629fb577290 2024-04-21T08:49:01.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.532+0000 7f380e7fc640 1 --2- 172.21.15.43:0/3033862725 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f38040a3d20 0x7f38040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.532+0000 7f381ca2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb57fe10 0x5629fb582260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.533+0000 7f381ca2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb57fe10 0x5629fb582260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48136/0 (socket says 172.21.15.43:48136) 2024-04-21T08:49:01.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.533+0000 7f381ca2e640 1 -- 172.21.15.43:0/2079752818 learned_addr learned my addr 172.21.15.43:0/2079752818 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:01.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.533+0000 7f381da30640 1 --2- 172.21.15.43:0/3033862725 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f38040a3d20 0x7f38040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.533+0000 7f381d22f640 1 --2- 172.21.15.43:0/2079752818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629fb577290 0x5629fb57f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.533+0000 7f381da30640 1 --2- 172.21.15.43:0/2079752818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629fb5768e0 0x5629fb576cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.533+0000 7f380e7fc640 1 -- 172.21.15.43:0/3033862725 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f38140d0fb0 con 0x7f38181010a0 2024-04-21T08:49:01.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.533+0000 7f381ca2e640 1 -- 172.21.15.43:0/2079752818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629fb577290 msgr2=0x5629fb57f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:01.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.533+0000 7f381ca2e640 1 --2- 172.21.15.43:0/2079752818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629fb577290 0x5629fb57f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.533+0000 7f381ca2e640 1 -- 172.21.15.43:0/2079752818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629fb5768e0 msgr2=0x5629fb576cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:01.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.533+0000 7f381ca2e640 1 --2- 172.21.15.43:0/2079752818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629fb5768e0 0x5629fb576cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.533+0000 7f381ca2e640 1 -- 172.21.15.43:0/2079752818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629fb2b57e0 con 0x5629fb57fe10 2024-04-21T08:49:01.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.533+0000 7f381da30640 1 --2- 172.21.15.43:0/2079752818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629fb5768e0 0x5629fb576cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:01.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.533+0000 7f381ca2e640 1 --2- 172.21.15.43:0/2079752818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb57fe10 0x5629fb582260 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f380800bbc0 tx=0x7f380800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170ef5e1d22612a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:01.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.533+0000 7f381da30640 1 --2- 172.21.15.43:0/3033862725 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f38040a3d20 0x7f38040a61e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3810004af0 tx=0x7f3810030040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:01.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.533+0000 7f38037fe640 1 -- 172.21.15.43:0/2079752818 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808015070 con 0x5629fb57fe10 2024-04-21T08:49:01.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.533+0000 7f38037fe640 1 -- 172.21.15.43:0/2079752818 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38080054d0 con 0x5629fb57fe10 2024-04-21T08:49:01.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.533+0000 7f38037fe640 1 -- 172.21.15.43:0/2079752818 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808011440 con 0x5629fb57fe10 2024-04-21T08:49:01.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.534+0000 7f3822166740 1 -- 172.21.15.43:0/2079752818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb57fe10 msgr2=0x5629fb582260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:01.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.534+0000 7f3822166740 1 --2- 172.21.15.43:0/2079752818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb57fe10 0x5629fb582260 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f380800bbc0 tx=0x7f380800f730 comp rx=0 tx=0).stop 2024-04-21T08:49:01.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.534+0000 7f3822166740 1 -- 172.21.15.43:0/2079752818 shutdown_connections 2024-04-21T08:49:01.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.534+0000 7f3822166740 1 --2- 172.21.15.43:0/2079752818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb57fe10 0x5629fb582260 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.534+0000 7f3822166740 1 --2- 172.21.15.43:0/2079752818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629fb577290 0x5629fb57f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.534+0000 7f3822166740 1 --2- 172.21.15.43:0/2079752818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629fb5768e0 0x5629fb576cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.534+0000 7f3822166740 1 -- 172.21.15.43:0/2079752818 >> 172.21.15.43:0/2079752818 conn(0x5629fb570fb0 msgr2=0x5629fb571390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:01.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.534+0000 7f3822166740 1 -- 172.21.15.43:0/2079752818 shutdown_connections 2024-04-21T08:49:01.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.534+0000 7f3822166740 1 -- 172.21.15.43:0/2079752818 wait complete. 2024-04-21T08:49:01.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.534+0000 7f3822166740 1 -- start start 2024-04-21T08:49:01.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.534+0000 7f3822166740 1 Processor -- start 2024-04-21T08:49:01.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.535+0000 7f3822166740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629fb5768e0 0x5629fb57cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.535+0000 7f3822166740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629fb577290 0x5629fb57d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.535+0000 7f3822166740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb57fe10 0x5629fb57d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.535+0000 7f3822166740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5629fb582d80 con 0x5629fb57fe10 2024-04-21T08:49:01.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.535+0000 7f3822166740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5629fb494580 con 0x5629fb5768e0 2024-04-21T08:49:01.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.535+0000 7f3822166740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5629fb582f00 con 0x5629fb577290 2024-04-21T08:49:01.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.535+0000 7f3822166740 10 client.0 _ll_register_callbacks cb 0x5629fb49a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:01.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.535+0000 7f381da30640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb57fe10 0x5629fb57d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.535+0000 7f381da30640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb57fe10 0x5629fb57d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48150/0 (socket says 172.21.15.43:48150) 2024-04-21T08:49:01.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.535+0000 7f381da30640 1 -- 172.21.15.43:0/1330812070 learned_addr learned my addr 172.21.15.43:0/1330812070 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:01.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.535+0000 7f381d22f640 1 --2- 172.21.15.43:0/1330812070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629fb5768e0 0x5629fb57cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.535+0000 7f381ca2e640 1 --2- 172.21.15.43:0/1330812070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629fb577290 0x5629fb57d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.535+0000 7f381da30640 1 -- 172.21.15.43:0/1330812070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629fb577290 msgr2=0x5629fb57d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:01.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.535+0000 7f381da30640 1 --2- 172.21.15.43:0/1330812070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629fb577290 0x5629fb57d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.535+0000 7f381da30640 1 -- 172.21.15.43:0/1330812070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629fb5768e0 msgr2=0x5629fb57cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:01.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.535+0000 7f381da30640 1 --2- 172.21.15.43:0/1330812070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629fb5768e0 0x5629fb57cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.535+0000 7f381da30640 1 -- 172.21.15.43:0/1330812070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629fb315f70 con 0x5629fb57fe10 2024-04-21T08:49:01.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.535+0000 7f381ca2e640 1 --2- 172.21.15.43:0/1330812070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629fb577290 0x5629fb57d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:01.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.536+0000 7f381da30640 1 --2- 172.21.15.43:0/1330812070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb57fe10 0x5629fb57d7d0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f3810003bf0 tx=0x7f381002e310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5298e9539f9a87f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:01.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.536+0000 7f3800ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:01.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.536+0000 7f3800ff9640 1 -- 172.21.15.43:0/1330812070 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810053020 con 0x5629fb57fe10 2024-04-21T08:49:01.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.536+0000 7f3800ff9640 1 -- 172.21.15.43:0/1330812070 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f3810004450 con 0x5629fb57fe10 2024-04-21T08:49:01.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.536+0000 7f3822166740 1 -- 172.21.15.43:0/1330812070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38080054d0 con 0x5629fb57fe10 2024-04-21T08:49:01.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.536+0000 7f3800ff9640 1 -- 172.21.15.43:0/1330812070 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f381005e050 con 0x5629fb57fe10 2024-04-21T08:49:01.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.536+0000 7f3822166740 10 client.20448 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:01.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.536+0000 7f3822166740 1 -- 172.21.15.43:0/1330812070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5629fb1ad170 con 0x5629fb57fe10 2024-04-21T08:49:01.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.536+0000 7f3800ff9640 1 -- 172.21.15.43:0/1330812070 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38080054d0 con 0x5629fb57fe10 2024-04-21T08:49:01.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.536+0000 7f3800ff9640 1 -- 172.21.15.43:0/1330812070 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3810041070 con 0x5629fb57fe10 2024-04-21T08:49:01.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.536+0000 7f3822166740 10 client.20448 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:01.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.536+0000 7f3822166740 10 client.20448 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:01.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.536+0000 7f3822166740 20 client.20448 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:01.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.536+0000 7f3822166740 1 --2- 172.21.15.43:0/1330812070 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5629fb590350 0x5629fb5927f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.536+0000 7f3822166740 4 client.20448 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:01.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.536+0000 7f3822166740 1 -- 172.21.15.43:0/1330812070 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629fb4e8920 con 0x5629fb590350 2024-04-21T08:49:01.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.537+0000 7f381d22f640 1 --2- 172.21.15.43:0/1330812070 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5629fb590350 0x5629fb5927f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.537+0000 7f3800ff9640 10 client.20448 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:01.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.537+0000 7f381d22f640 1 --2- 172.21.15.43:0/1330812070 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5629fb590350 0x5629fb5927f0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9712f297dda04e72 server_cookie=7b9627e78fd4c66e in_seq=0 out_seq=0 2024-04-21T08:49:01.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.538+0000 7f3800ff9640 1 -- 172.21.15.43:0/1330812070 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629fb4e8920 con 0x5629fb590350 2024-04-21T08:49:01.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.538+0000 7f3800ff9640 10 client.20448 handle_command_reply: tid=0 2024-04-21T08:49:01.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.538+0000 7f3822166740 1 client.20448 shutdown 2024-04-21T08:49:01.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.538+0000 7f3822166740 1 -- 172.21.15.43:0/1330812070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb57fe10 msgr2=0x5629fb57d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:01.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.538+0000 7f3822166740 1 --2- 172.21.15.43:0/1330812070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb57fe10 0x5629fb57d7d0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f3810003bf0 tx=0x7f381002e310 comp rx=0 tx=0).stop 2024-04-21T08:49:01.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.538+0000 7f3822166740 1 -- 172.21.15.43:0/1330812070 shutdown_connections 2024-04-21T08:49:01.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.538+0000 7f3822166740 1 --2- 172.21.15.43:0/1330812070 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5629fb590350 0x5629fb5927f0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.538+0000 7f3822166740 1 --2- 172.21.15.43:0/1330812070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb57fe10 0x5629fb57d7d0 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.538+0000 7f3822166740 1 --2- 172.21.15.43:0/1330812070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629fb577290 0x5629fb57d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.538+0000 7f3822166740 1 --2- 172.21.15.43:0/1330812070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629fb5768e0 0x5629fb57cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.538+0000 7f3800ff9640 0 client.20448 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:01.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.538+0000 7f3822166740 1 -- 172.21.15.43:0/1330812070 >> 172.21.15.43:0/1330812070 conn(0x5629fb570fb0 msgr2=0x5629fb572d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:01.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.538+0000 7f3822166740 1 -- 172.21.15.43:0/1330812070 shutdown_connections 2024-04-21T08:49:01.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.538+0000 7f3822166740 1 -- 172.21.15.43:0/1330812070 wait complete. 2024-04-21T08:49:01.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.538+0000 7f3822166740 20 client.20448 trim_cache size 0 max 16384 2024-04-21T08:49:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.556+0000 7f3822166740 1 Processor -- start 2024-04-21T08:49:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.556+0000 7f3822166740 1 -- start start 2024-04-21T08:49:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.556+0000 7f3822166740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629fb570fb0 0x5629fb58bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.556+0000 7f3822166740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb5768e0 0x5629fb57d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.556+0000 7f3822166740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629fb577290 0x5629fb5980c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.556+0000 7f3822166740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f38181ad250 con 0x5629fb5768e0 2024-04-21T08:49:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.556+0000 7f3822166740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38181aed90 con 0x5629fb577290 2024-04-21T08:49:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.556+0000 7f3822166740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f38181ad920 con 0x5629fb570fb0 2024-04-21T08:49:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.556+0000 7f381ca2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb5768e0 0x5629fb57d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.556+0000 7f381ca2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb5768e0 0x5629fb57d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48162/0 (socket says 172.21.15.43:48162) 2024-04-21T08:49:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.556+0000 7f381ca2e640 1 -- 172.21.15.43:0/762173306 learned_addr learned my addr 172.21.15.43:0/762173306 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.557+0000 7f381d22f640 1 --2- 172.21.15.43:0/762173306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629fb570fb0 0x5629fb58bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.557+0000 7f381ca2e640 1 -- 172.21.15.43:0/762173306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629fb570fb0 msgr2=0x5629fb58bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.557+0000 7f381ca2e640 1 --2- 172.21.15.43:0/762173306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629fb570fb0 0x5629fb58bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.557+0000 7f381ca2e640 1 -- 172.21.15.43:0/762173306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629fb577290 msgr2=0x5629fb5980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.557+0000 7f381da30640 1 --2- 172.21.15.43:0/762173306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629fb577290 0x5629fb5980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.557+0000 7f381ca2e640 1 --2- 172.21.15.43:0/762173306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629fb577290 0x5629fb5980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.557+0000 7f381ca2e640 1 -- 172.21.15.43:0/762173306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629fb582200 con 0x5629fb5768e0 2024-04-21T08:49:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.557+0000 7f381ca2e640 1 --2- 172.21.15.43:0/762173306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb5768e0 0x5629fb57d420 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f3808018000 tx=0x7f3808005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6e7bd3ff50ac8b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.557+0000 7f3802ffd640 1 -- 172.21.15.43:0/762173306 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808026070 con 0x5629fb5768e0 2024-04-21T08:49:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.557+0000 7f3802ffd640 1 -- 172.21.15.43:0/762173306 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38080054f0 con 0x5629fb5768e0 2024-04-21T08:49:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.557+0000 7f3802ffd640 1 -- 172.21.15.43:0/762173306 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808015070 con 0x5629fb5768e0 2024-04-21T08:49:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.558+0000 7f3822166740 1 -- 172.21.15.43:0/762173306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb5768e0 msgr2=0x5629fb57d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.558+0000 7f3822166740 1 --2- 172.21.15.43:0/762173306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb5768e0 0x5629fb57d420 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f3808018000 tx=0x7f3808005cf0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.558+0000 7f3822166740 1 -- 172.21.15.43:0/762173306 shutdown_connections 2024-04-21T08:49:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.558+0000 7f3822166740 1 --2- 172.21.15.43:0/762173306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629fb577290 0x5629fb5980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.558+0000 7f3822166740 1 --2- 172.21.15.43:0/762173306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb5768e0 0x5629fb57d420 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.558+0000 7f3822166740 1 --2- 172.21.15.43:0/762173306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629fb570fb0 0x5629fb58bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.558+0000 7f3822166740 1 -- 172.21.15.43:0/762173306 >> 172.21.15.43:0/762173306 conn(0x5629fb57b990 msgr2=0x5629fb57cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.558+0000 7f3822166740 1 -- 172.21.15.43:0/762173306 shutdown_connections 2024-04-21T08:49:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.558+0000 7f3822166740 1 -- 172.21.15.43:0/762173306 wait complete. 2024-04-21T08:49:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.558+0000 7f3822166740 1 -- start start 2024-04-21T08:49:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.558+0000 7f3822166740 1 Processor -- start 2024-04-21T08:49:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.559+0000 7f3822166740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629fb570fb0 0x5629fb588300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.559+0000 7f3822166740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb5768e0 0x5629fb584680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.559+0000 7f3822166740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629fb577290 0x5629fb584bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.559+0000 7f3822166740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f38181ae6c0 con 0x5629fb5768e0 2024-04-21T08:49:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.559+0000 7f3822166740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38181adff0 con 0x5629fb577290 2024-04-21T08:49:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.559+0000 7f3822166740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f38181ab250 con 0x5629fb570fb0 2024-04-21T08:49:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.559+0000 7f3822166740 10 client.0 _ll_register_callbacks cb 0x7f3814005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.559+0000 7f381ca2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb5768e0 0x5629fb584680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.559+0000 7f381ca2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb5768e0 0x5629fb584680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48166/0 (socket says 172.21.15.43:48166) 2024-04-21T08:49:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.559+0000 7f381ca2e640 1 -- 172.21.15.43:0/4160476858 learned_addr learned my addr 172.21.15.43:0/4160476858 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.559+0000 7f381d22f640 1 --2- 172.21.15.43:0/4160476858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629fb570fb0 0x5629fb588300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.559+0000 7f381da30640 1 --2- 172.21.15.43:0/4160476858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629fb577290 0x5629fb584bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.559+0000 7f381ca2e640 1 -- 172.21.15.43:0/4160476858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629fb570fb0 msgr2=0x5629fb588300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.559+0000 7f381ca2e640 1 --2- 172.21.15.43:0/4160476858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629fb570fb0 0x5629fb588300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.559+0000 7f381ca2e640 1 -- 172.21.15.43:0/4160476858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629fb577290 msgr2=0x5629fb584bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.559+0000 7f381ca2e640 1 --2- 172.21.15.43:0/4160476858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629fb577290 0x5629fb584bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.559+0000 7f381ca2e640 1 -- 172.21.15.43:0/4160476858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629fb5851b0 con 0x5629fb5768e0 2024-04-21T08:49:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.559+0000 7f381d22f640 1 --2- 172.21.15.43:0/4160476858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629fb570fb0 0x5629fb588300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.559+0000 7f381ca2e640 1 --2- 172.21.15.43:0/4160476858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb5768e0 0x5629fb584680 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f3808005d80 tx=0x7f380800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7448d25660b0d007 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.559+0000 7f3800ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.559+0000 7f3800ff9640 1 -- 172.21.15.43:0/4160476858 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808026040 con 0x5629fb5768e0 2024-04-21T08:49:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.559+0000 7f3800ff9640 1 -- 172.21.15.43:0/4160476858 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f3808004ac0 con 0x5629fb5768e0 2024-04-21T08:49:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.559+0000 7f3822166740 1 -- 172.21.15.43:0/4160476858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38080054f0 con 0x5629fb5768e0 2024-04-21T08:49:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.560+0000 7f3800ff9640 1 -- 172.21.15.43:0/4160476858 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3808015040 con 0x5629fb5768e0 2024-04-21T08:49:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.560+0000 7f3822166740 10 client.20454 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.560+0000 7f3822166740 1 -- 172.21.15.43:0/4160476858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5629fb5fbb70 con 0x5629fb5768e0 2024-04-21T08:49:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.560+0000 7f3800ff9640 1 -- 172.21.15.43:0/4160476858 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38080054f0 con 0x5629fb5768e0 2024-04-21T08:49:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.560+0000 7f3800ff9640 1 -- 172.21.15.43:0/4160476858 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3808034030 con 0x5629fb5768e0 2024-04-21T08:49:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.560+0000 7f3822166740 10 client.20454 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.560+0000 7f3822166740 10 client.20454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.560+0000 7f3822166740 20 client.20454 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.560+0000 7f3822166740 1 --2- 172.21.15.43:0/4160476858 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5629fb57fe10 0x5629fb5fe100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.560+0000 7f3822166740 4 client.20454 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:49:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.560+0000 7f3822166740 1 -- 172.21.15.43:0/4160476858 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5629fb4e2790 con 0x5629fb57fe10 2024-04-21T08:49:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.560+0000 7f381d22f640 1 --2- 172.21.15.43:0/4160476858 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5629fb57fe10 0x5629fb5fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.561+0000 7f3800ff9640 10 client.20454 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.561+0000 7f381d22f640 1 --2- 172.21.15.43:0/4160476858 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5629fb57fe10 0x5629fb5fe100 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b95f9ed803ef3dc6 server_cookie=cac102d3e040adbf in_seq=0 out_seq=0 2024-04-21T08:49:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.562+0000 7f3800ff9640 1 -- 172.21.15.43:0/4160476858 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5629fb4e2790 con 0x5629fb57fe10 2024-04-21T08:49:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.562+0000 7f3800ff9640 10 client.20454 handle_command_reply: tid=0 2024-04-21T08:49:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.562+0000 7f3822166740 1 client.20454 shutdown 2024-04-21T08:49:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.562+0000 7f3822166740 1 -- 172.21.15.43:0/4160476858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb5768e0 msgr2=0x5629fb584680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.562+0000 7f3822166740 1 --2- 172.21.15.43:0/4160476858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb5768e0 0x5629fb584680 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f3808005d80 tx=0x7f380800b910 comp rx=0 tx=0).stop 2024-04-21T08:49:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.562+0000 7f3822166740 1 -- 172.21.15.43:0/4160476858 shutdown_connections 2024-04-21T08:49:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.562+0000 7f3822166740 1 --2- 172.21.15.43:0/4160476858 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5629fb57fe10 0x5629fb5fe100 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.562+0000 7f3822166740 1 --2- 172.21.15.43:0/4160476858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629fb577290 0x5629fb584bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.562+0000 7f3822166740 1 --2- 172.21.15.43:0/4160476858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629fb5768e0 0x5629fb584680 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.562+0000 7f3822166740 1 --2- 172.21.15.43:0/4160476858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629fb570fb0 0x5629fb588300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.562+0000 7f3822166740 1 -- 172.21.15.43:0/4160476858 >> 172.21.15.43:0/4160476858 conn(0x5629fb57b990 msgr2=0x5629fb57d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.562+0000 7f3800ff9640 0 client.20454 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.562+0000 7f3822166740 1 -- 172.21.15.43:0/4160476858 shutdown_connections 2024-04-21T08:49:01.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.562+0000 7f3822166740 1 -- 172.21.15.43:0/4160476858 wait complete. 2024-04-21T08:49:01.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.562+0000 7f3822166740 20 client.20454 trim_cache size 0 max 16384 2024-04-21T08:49:01.569 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:49:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:49:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "e814190c-dd38-482c-978b-8e4944dd11d9", 2024-04-21T08:49:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:49:01.569 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:49:01.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.563+0000 7f381f493640 1 -- 172.21.15.43:0/3033862725 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f38040a3d20 msgr2=0x7f38040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:01.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.563+0000 7f381f493640 1 --2- 172.21.15.43:0/3033862725 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f38040a3d20 0x7f38040a61e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3810004af0 tx=0x7f3810030040 comp rx=0 tx=0).stop 2024-04-21T08:49:01.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.563+0000 7f381f493640 1 -- 172.21.15.43:0/3033862725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38181010a0 msgr2=0x7f3818196530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:01.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.563+0000 7f381f493640 1 --2- 172.21.15.43:0/3033862725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38181010a0 0x7f3818196530 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f381400e9f0 tx=0x7f381400eeb0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.563+0000 7f381f493640 1 -- 172.21.15.43:0/3033862725 shutdown_connections 2024-04-21T08:49:01.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.563+0000 7f381f493640 1 --2- 172.21.15.43:0/3033862725 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f38040a3d20 0x7f38040a61e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.563+0000 7f381f493640 1 --2- 172.21.15.43:0/3033862725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f381819ada0 0x7f381819b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.563+0000 7f381f493640 1 --2- 172.21.15.43:0/3033862725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3818106170 0x7f3818196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.563+0000 7f381f493640 1 --2- 172.21.15.43:0/3033862725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38181010a0 0x7f3818196530 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.563+0000 7f381f493640 1 -- 172.21.15.43:0/3033862725 >> 172.21.15.43:0/3033862725 conn(0x7f3818077eb0 msgr2=0x7f38180fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:01.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.563+0000 7f381f493640 1 -- 172.21.15.43:0/3033862725 shutdown_connections 2024-04-21T08:49:01.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.563+0000 7f381f493640 1 -- 172.21.15.43:0/3033862725 wait complete. 2024-04-21T08:49:01.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:49:01.574 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:49:01.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.678+0000 7fb8a739f640 1 -- 172.21.15.43:0/777187204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a0101820 msgr2=0x7fb8a0105860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:01.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.678+0000 7fb8a739f640 1 --2- 172.21.15.43:0/777187204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a0101820 0x7fb8a0105860 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7fb890009900 tx=0x7fb89002bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:01.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.679+0000 7fb8a739f640 1 -- 172.21.15.43:0/777187204 shutdown_connections 2024-04-21T08:49:01.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.679+0000 7fb8a739f640 1 --2- 172.21.15.43:0/777187204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8a0105f40 0x7fb8a010db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.679+0000 7fb8a739f640 1 --2- 172.21.15.43:0/777187204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a0101820 0x7fb8a0105860 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.679+0000 7fb8a739f640 1 --2- 172.21.15.43:0/777187204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8a0100e70 0x7fb8a0101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.679+0000 7fb8a739f640 1 -- 172.21.15.43:0/777187204 >> 172.21.15.43:0/777187204 conn(0x7fb8a0077a30 msgr2=0x7fb8a00fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:01.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.679+0000 7fb8a739f640 1 -- 172.21.15.43:0/777187204 shutdown_connections 2024-04-21T08:49:01.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.679+0000 7fb8a739f640 1 -- 172.21.15.43:0/777187204 wait complete. 2024-04-21T08:49:01.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.679+0000 7fb8a739f640 1 Processor -- start 2024-04-21T08:49:01.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.679+0000 7fb8a739f640 1 -- start start 2024-04-21T08:49:01.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.679+0000 7fb8a739f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a0100e70 0x7fb8a01984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.679+0000 7fb8a739f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8a0105f40 0x7fb8a0198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.680+0000 7fb8a739f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8a019cdb0 0x7fb8a019d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.680+0000 7fb8a739f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a0110ea0 con 0x7fb8a0100e70 2024-04-21T08:49:01.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.680+0000 7fb8a739f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a0110d20 con 0x7fb8a0105f40 2024-04-21T08:49:01.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.680+0000 7fb8a739f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a0111020 con 0x7fb8a019cdb0 2024-04-21T08:49:01.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.680+0000 7fb8a639d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a0100e70 0x7fb8a01984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.680+0000 7fb8a639d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a0100e70 0x7fb8a01984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48188/0 (socket says 172.21.15.43:48188) 2024-04-21T08:49:01.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.680+0000 7fb8a639d640 1 -- 172.21.15.43:0/848341440 learned_addr learned my addr 172.21.15.43:0/848341440 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:01.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.680+0000 7fb8a6b9e640 1 --2- 172.21.15.43:0/848341440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8a019cdb0 0x7fb8a019d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.680+0000 7fb8a5b9c640 1 --2- 172.21.15.43:0/848341440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8a0105f40 0x7fb8a0198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.680+0000 7fb8a639d640 1 -- 172.21.15.43:0/848341440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8a019cdb0 msgr2=0x7fb8a019d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:01.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.680+0000 7fb8a639d640 1 --2- 172.21.15.43:0/848341440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8a019cdb0 0x7fb8a019d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.680+0000 7fb8a639d640 1 -- 172.21.15.43:0/848341440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8a0105f40 msgr2=0x7fb8a0198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:01.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.680+0000 7fb8a639d640 1 --2- 172.21.15.43:0/848341440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8a0105f40 0x7fb8a0198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.680+0000 7fb8a639d640 1 -- 172.21.15.43:0/848341440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a019d8a0 con 0x7fb8a0100e70 2024-04-21T08:49:01.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.680+0000 7fb8a5b9c640 1 --2- 172.21.15.43:0/848341440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8a0105f40 0x7fb8a0198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:01.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.680+0000 7fb8a639d640 1 --2- 172.21.15.43:0/848341440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a0100e70 0x7fb8a01984f0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00e9f0 tx=0x7fb89c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:01.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.680+0000 7fb8977fe640 1 -- 172.21.15.43:0/848341440 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c00cd80 con 0x7fb8a0100e70 2024-04-21T08:49:01.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.680+0000 7fb8a739f640 1 -- 172.21.15.43:0/848341440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a019db90 con 0x7fb8a0100e70 2024-04-21T08:49:01.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.680+0000 7fb8977fe640 1 -- 172.21.15.43:0/848341440 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb89c0054d0 con 0x7fb8a0100e70 2024-04-21T08:49:01.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.680+0000 7fb8977fe640 1 -- 172.21.15.43:0/848341440 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c010690 con 0x7fb8a0100e70 2024-04-21T08:49:01.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.680+0000 7fb8a739f640 1 -- 172.21.15.43:0/848341440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a01a5330 con 0x7fb8a0100e70 2024-04-21T08:49:01.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.683+0000 7fb8977fe640 1 -- 172.21.15.43:0/848341440 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb89c004e90 con 0x7fb8a0100e70 2024-04-21T08:49:01.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.683+0000 7fb8ab1ea740 1 Processor -- start 2024-04-21T08:49:01.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.683+0000 7fb8ab1ea740 1 -- start start 2024-04-21T08:49:01.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.683+0000 7fb8ab1ea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c266710 0x55a53c266af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.683+0000 7fb8ab1ea740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a53c267030 0x55a53c26f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.683+0000 7fb8ab1ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a53c26fc10 0x55a53c272000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.683+0000 7fb8ab1ea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a53c1cb1e0 con 0x55a53c266710 2024-04-21T08:49:01.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.683+0000 7fb8ab1ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a53c168390 con 0x55a53c26fc10 2024-04-21T08:49:01.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.683+0000 7fb8ab1ea740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a53c1c71a0 con 0x55a53c267030 2024-04-21T08:49:01.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.683+0000 7fb8977fe640 1 --2- 172.21.15.43:0/848341440 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb88c0a3d20 0x7fb88c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.684+0000 7fb8a6b9e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c266710 0x55a53c266af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.684+0000 7fb8a6b9e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c266710 0x55a53c266af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48198/0 (socket says 172.21.15.43:48198) 2024-04-21T08:49:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.684+0000 7fb8a6b9e640 1 -- 172.21.15.43:0/1498369032 learned_addr learned my addr 172.21.15.43:0/1498369032 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.684+0000 7fb8a6b9e640 1 --2- 172.21.15.43:0/848341440 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb88c0a3d20 0x7fb88c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.684+0000 7fb8a5b9c640 1 --2- 172.21.15.43:0/1498369032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a53c26fc10 0x55a53c272000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.684+0000 7fb8977fe640 1 -- 172.21.15.43:0/848341440 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fb89c0d10d0 con 0x7fb8a0100e70 2024-04-21T08:49:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.684+0000 7fb8a639d640 1 --2- 172.21.15.43:0/1498369032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a53c267030 0x55a53c26f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.684+0000 7fb8a6b9e640 1 -- 172.21.15.43:0/1498369032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a53c267030 msgr2=0x55a53c26f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.684+0000 7fb8a6b9e640 1 --2- 172.21.15.43:0/1498369032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a53c267030 0x55a53c26f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.684+0000 7fb8a6b9e640 1 -- 172.21.15.43:0/1498369032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a53c26fc10 msgr2=0x55a53c272000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.684+0000 7fb8a6b9e640 1 --2- 172.21.15.43:0/1498369032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a53c26fc10 0x55a53c272000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.684+0000 7fb8a6b9e640 1 -- 172.21.15.43:0/1498369032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a53bfa77d0 con 0x55a53c266710 2024-04-21T08:49:01.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.684+0000 7fb8a639d640 1 --2- 172.21.15.43:0/1498369032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a53c267030 0x55a53c26f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:01.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.684+0000 7fb8a6b9e640 1 --2- 172.21.15.43:0/1498369032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c266710 0x55a53c266af0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fb89800a780 tx=0x7fb8980117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ac62ca8e659bc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:01.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.684+0000 7fb8a6b9e640 1 --2- 172.21.15.43:0/848341440 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb88c0a3d20 0x7fb88c0a61e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb898005cc0 tx=0x7fb89801c000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:01.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.684+0000 7fb894bfe640 1 -- 172.21.15.43:0/1498369032 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb898024070 con 0x55a53c266710 2024-04-21T08:49:01.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.684+0000 7fb894bfe640 1 -- 172.21.15.43:0/1498369032 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb898003e20 con 0x55a53c266710 2024-04-21T08:49:01.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.684+0000 7fb894bfe640 1 -- 172.21.15.43:0/1498369032 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb898004660 con 0x55a53c266710 2024-04-21T08:49:01.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.685+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/1498369032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c266710 msgr2=0x55a53c266af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:01.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.685+0000 7fb8ab1ea740 1 --2- 172.21.15.43:0/1498369032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c266710 0x55a53c266af0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fb89800a780 tx=0x7fb8980117b0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.685+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/1498369032 shutdown_connections 2024-04-21T08:49:01.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.685+0000 7fb8ab1ea740 1 --2- 172.21.15.43:0/1498369032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a53c26fc10 0x55a53c272000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.685+0000 7fb8ab1ea740 1 --2- 172.21.15.43:0/1498369032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a53c267030 0x55a53c26f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.685+0000 7fb8ab1ea740 1 --2- 172.21.15.43:0/1498369032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c266710 0x55a53c266af0 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.685+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/1498369032 >> 172.21.15.43:0/1498369032 conn(0x55a53c260e40 msgr2=0x55a53c261220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.685+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/1498369032 shutdown_connections 2024-04-21T08:49:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.685+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/1498369032 wait complete. 2024-04-21T08:49:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.686+0000 7fb8ab1ea740 1 -- start start 2024-04-21T08:49:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.686+0000 7fb8ab1ea740 1 Processor -- start 2024-04-21T08:49:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.686+0000 7fb8ab1ea740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a53c266710 0x55a53c26cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.686+0000 7fb8ab1ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a53c267030 0x55a53c26d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.686+0000 7fb8ab1ea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c26fc10 0x55a53c26d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.686+0000 7fb8ab1ea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a53c2729a0 con 0x55a53c26fc10 2024-04-21T08:49:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.686+0000 7fb8ab1ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a53c1842f0 con 0x55a53c267030 2024-04-21T08:49:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.686+0000 7fb8ab1ea740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a53c272b20 con 0x55a53c266710 2024-04-21T08:49:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.686+0000 7fb8ab1ea740 10 client.0 _ll_register_callbacks cb 0x55a53c191580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.686+0000 7fb8a6b9e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c26fc10 0x55a53c26d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.686+0000 7fb8a6b9e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c26fc10 0x55a53c26d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48204/0 (socket says 172.21.15.43:48204) 2024-04-21T08:49:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.686+0000 7fb8a6b9e640 1 -- 172.21.15.43:0/3560072881 learned_addr learned my addr 172.21.15.43:0/3560072881 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.686+0000 7fb8a5b9c640 1 --2- 172.21.15.43:0/3560072881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a53c267030 0x55a53c26d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.686+0000 7fb8a639d640 1 --2- 172.21.15.43:0/3560072881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a53c266710 0x55a53c26cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.686+0000 7fb8a6b9e640 1 -- 172.21.15.43:0/3560072881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a53c266710 msgr2=0x55a53c26cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.686+0000 7fb8a6b9e640 1 --2- 172.21.15.43:0/3560072881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a53c266710 0x55a53c26cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.686+0000 7fb8a6b9e640 1 -- 172.21.15.43:0/3560072881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a53c267030 msgr2=0x55a53c26d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.686+0000 7fb8a6b9e640 1 --2- 172.21.15.43:0/3560072881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a53c267030 0x55a53c26d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.686+0000 7fb8a6b9e640 1 -- 172.21.15.43:0/3560072881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a53bedd3f0 con 0x55a53c26fc10 2024-04-21T08:49:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.687+0000 7fb8a6b9e640 1 --2- 172.21.15.43:0/3560072881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c26fc10 0x55a53c26d6b0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fb898011fd0 tx=0x7fb89800a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78a92fcf749c42f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.687+0000 7fb889ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.687+0000 7fb889ffb640 1 -- 172.21.15.43:0/3560072881 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb898035060 con 0x55a53c26fc10 2024-04-21T08:49:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.687+0000 7fb889ffb640 1 -- 172.21.15.43:0/3560072881 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fb898013240 con 0x55a53c26fc10 2024-04-21T08:49:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.687+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/3560072881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb898003e20 con 0x55a53c26fc10 2024-04-21T08:49:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.687+0000 7fb889ffb640 1 -- 172.21.15.43:0/3560072881 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb898024050 con 0x55a53c26fc10 2024-04-21T08:49:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.687+0000 7fb8ab1ea740 10 client.20466 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.687+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/3560072881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a53be9d0f0 con 0x55a53c26fc10 2024-04-21T08:49:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.687+0000 7fb889ffb640 1 -- 172.21.15.43:0/3560072881 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb898003e20 con 0x55a53c26fc10 2024-04-21T08:49:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.687+0000 7fb889ffb640 1 -- 172.21.15.43:0/3560072881 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb898012450 con 0x55a53c26fc10 2024-04-21T08:49:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.687+0000 7fb8ab1ea740 10 client.20466 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:01.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.687+0000 7fb8ab1ea740 10 client.20466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:01.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.687+0000 7fb8ab1ea740 20 client.20466 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:01.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.687+0000 7fb8ab1ea740 1 --2- 172.21.15.43:0/3560072881 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a53c27fa00 0x55a53c281df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.687+0000 7fb8ab1ea740 4 client.20466 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:01.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.687+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/3560072881 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a53c1d8c00 con 0x55a53c27fa00 2024-04-21T08:49:01.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.688+0000 7fb8a639d640 1 --2- 172.21.15.43:0/3560072881 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a53c27fa00 0x55a53c281df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.688+0000 7fb8a639d640 1 --2- 172.21.15.43:0/3560072881 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a53c27fa00 0x55a53c281df0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bea423d519ef957 server_cookie=cdd7e9cb7d101af5 in_seq=0 out_seq=0 2024-04-21T08:49:01.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.688+0000 7fb889ffb640 10 client.20466 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:01.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.689+0000 7fb889ffb640 1 -- 172.21.15.43:0/3560072881 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a53c1d8c00 con 0x55a53c27fa00 2024-04-21T08:49:01.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.689+0000 7fb889ffb640 10 client.20466 handle_command_reply: tid=0 2024-04-21T08:49:01.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.689+0000 7fb8ab1ea740 1 client.20466 shutdown 2024-04-21T08:49:01.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.689+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/3560072881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c26fc10 msgr2=0x55a53c26d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:01.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.689+0000 7fb8ab1ea740 1 --2- 172.21.15.43:0/3560072881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c26fc10 0x55a53c26d6b0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fb898011fd0 tx=0x7fb89800a540 comp rx=0 tx=0).stop 2024-04-21T08:49:01.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.689+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/3560072881 shutdown_connections 2024-04-21T08:49:01.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.689+0000 7fb8ab1ea740 1 --2- 172.21.15.43:0/3560072881 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a53c27fa00 0x55a53c281df0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.689+0000 7fb8ab1ea740 1 --2- 172.21.15.43:0/3560072881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c26fc10 0x55a53c26d6b0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.689+0000 7fb8ab1ea740 1 --2- 172.21.15.43:0/3560072881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a53c267030 0x55a53c26d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.689+0000 7fb8ab1ea740 1 --2- 172.21.15.43:0/3560072881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a53c266710 0x55a53c26cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.689+0000 7fb889ffb640 0 client.20466 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:01.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.689+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/3560072881 >> 172.21.15.43:0/3560072881 conn(0x55a53c260e40 msgr2=0x55a53c260120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:01.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.689+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/3560072881 shutdown_connections 2024-04-21T08:49:01.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.689+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/3560072881 wait complete. 2024-04-21T08:49:01.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.689+0000 7fb8ab1ea740 20 client.20466 trim_cache size 0 max 16384 2024-04-21T08:49:01.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.707+0000 7fb8ab1ea740 1 Processor -- start 2024-04-21T08:49:01.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.707+0000 7fb8ab1ea740 1 -- start start 2024-04-21T08:49:01.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.707+0000 7fb8ab1ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a53c266710 0x55a53c27b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.707+0000 7fb8ab1ea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c267030 0x55a53c26d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.707+0000 7fb8ab1ea740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a53c26fc10 0x55a53c2876c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.707+0000 7fb8ab1ea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a01af2f0 con 0x55a53c267030 2024-04-21T08:49:01.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.707+0000 7fb8ab1ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a01b0e30 con 0x55a53c266710 2024-04-21T08:49:01.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.707+0000 7fb8ab1ea740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a01af9c0 con 0x55a53c26fc10 2024-04-21T08:49:01.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.707+0000 7fb8a5b9c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c267030 0x55a53c26d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.707+0000 7fb8a5b9c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c267030 0x55a53c26d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48220/0 (socket says 172.21.15.43:48220) 2024-04-21T08:49:01.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.707+0000 7fb8a5b9c640 1 -- 172.21.15.43:0/1287950826 learned_addr learned my addr 172.21.15.43:0/1287950826 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:01.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.707+0000 7fb8a639d640 1 --2- 172.21.15.43:0/1287950826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a53c266710 0x55a53c27b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.707+0000 7fb8a5b9c640 1 -- 172.21.15.43:0/1287950826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a53c26fc10 msgr2=0x55a53c2876c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:01.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.707+0000 7fb8a5b9c640 1 --2- 172.21.15.43:0/1287950826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a53c26fc10 0x55a53c2876c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.707+0000 7fb8a5b9c640 1 -- 172.21.15.43:0/1287950826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a53c266710 msgr2=0x55a53c27b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:01.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.707+0000 7fb8a5b9c640 1 --2- 172.21.15.43:0/1287950826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a53c266710 0x55a53c27b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.707+0000 7fb8a5b9c640 1 -- 172.21.15.43:0/1287950826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a53c266e60 con 0x55a53c267030 2024-04-21T08:49:01.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.707+0000 7fb8a5b9c640 1 --2- 172.21.15.43:0/1287950826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c267030 0x55a53c26d4d0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0074d30 tx=0x7fb890004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db25879b0d40f515 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:01.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.707+0000 7fb88bfff640 1 -- 172.21.15.43:0/1287950826 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb890004330 con 0x55a53c267030 2024-04-21T08:49:01.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.708+0000 7fb88bfff640 1 -- 172.21.15.43:0/1287950826 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8900044d0 con 0x55a53c267030 2024-04-21T08:49:01.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.708+0000 7fb88bfff640 1 -- 172.21.15.43:0/1287950826 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb890004920 con 0x55a53c267030 2024-04-21T08:49:01.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.708+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/1287950826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c267030 msgr2=0x55a53c26d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:01.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.708+0000 7fb8ab1ea740 1 --2- 172.21.15.43:0/1287950826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c267030 0x55a53c26d4d0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0074d30 tx=0x7fb890004180 comp rx=0 tx=0).stop 2024-04-21T08:49:01.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.708+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/1287950826 shutdown_connections 2024-04-21T08:49:01.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.708+0000 7fb8ab1ea740 1 --2- 172.21.15.43:0/1287950826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a53c26fc10 0x55a53c2876c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.708+0000 7fb8ab1ea740 1 --2- 172.21.15.43:0/1287950826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c267030 0x55a53c26d4d0 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.708+0000 7fb8ab1ea740 1 --2- 172.21.15.43:0/1287950826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a53c266710 0x55a53c27b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.708+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/1287950826 >> 172.21.15.43:0/1287950826 conn(0x55a53c260e40 msgr2=0x55a53c260120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:01.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.708+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/1287950826 shutdown_connections 2024-04-21T08:49:01.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.708+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/1287950826 wait complete. 2024-04-21T08:49:01.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.708+0000 7fb8ab1ea740 1 -- start start 2024-04-21T08:49:01.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.708+0000 7fb8ab1ea740 1 Processor -- start 2024-04-21T08:49:01.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.709+0000 7fb8ab1ea740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a53c266710 0x55a53c277da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.709+0000 7fb8ab1ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a53c267030 0x55a53c2782e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.709+0000 7fb8ab1ea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c26fc10 0x55a53c274d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.709+0000 7fb8ab1ea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a01b0760 con 0x55a53c26fc10 2024-04-21T08:49:01.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.709+0000 7fb8ab1ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a01b0090 con 0x55a53c267030 2024-04-21T08:49:01.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.709+0000 7fb8ab1ea740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a01ad2f0 con 0x55a53c266710 2024-04-21T08:49:01.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.709+0000 7fb8ab1ea740 10 client.0 _ll_register_callbacks cb 0x7fb89c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:01.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.709+0000 7fb8a6b9e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c26fc10 0x55a53c274d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.709+0000 7fb8a6b9e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c26fc10 0x55a53c274d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48232/0 (socket says 172.21.15.43:48232) 2024-04-21T08:49:01.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.709+0000 7fb8a6b9e640 1 -- 172.21.15.43:0/681734248 learned_addr learned my addr 172.21.15.43:0/681734248 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:01.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.709+0000 7fb8a639d640 1 --2- 172.21.15.43:0/681734248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a53c266710 0x55a53c277da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.709+0000 7fb8a5b9c640 1 --2- 172.21.15.43:0/681734248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a53c267030 0x55a53c2782e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.709+0000 7fb8a6b9e640 1 -- 172.21.15.43:0/681734248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a53c266710 msgr2=0x55a53c277da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:01.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.709+0000 7fb8a6b9e640 1 --2- 172.21.15.43:0/681734248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a53c266710 0x55a53c277da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.709+0000 7fb8a6b9e640 1 -- 172.21.15.43:0/681734248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a53c267030 msgr2=0x55a53c2782e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:01.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.709+0000 7fb8a6b9e640 1 --2- 172.21.15.43:0/681734248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a53c267030 0x55a53c2782e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.709+0000 7fb8a6b9e640 1 -- 172.21.15.43:0/681734248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a53c275360 con 0x55a53c26fc10 2024-04-21T08:49:01.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.709+0000 7fb8a639d640 1 --2- 172.21.15.43:0/681734248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a53c266710 0x55a53c277da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:01.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.709+0000 7fb8a5b9c640 1 --2- 172.21.15.43:0/681734248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a53c267030 0x55a53c2782e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:01.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.709+0000 7fb8a6b9e640 1 --2- 172.21.15.43:0/681734248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c26fc10 0x55a53c274d70 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fb898012940 tx=0x7fb898012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b58cb4197c247e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:01.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.709+0000 7fb889ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:01.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.709+0000 7fb889ffb640 1 -- 172.21.15.43:0/681734248 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb898035050 con 0x55a53c26fc10 2024-04-21T08:49:01.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.709+0000 7fb889ffb640 1 -- 172.21.15.43:0/681734248 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fb898004c10 con 0x55a53c26fc10 2024-04-21T08:49:01.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.709+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/681734248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8900044d0 con 0x55a53c26fc10 2024-04-21T08:49:01.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.710+0000 7fb889ffb640 1 -- 172.21.15.43:0/681734248 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb898049050 con 0x55a53c26fc10 2024-04-21T08:49:01.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.710+0000 7fb8ab1ea740 10 client.20472 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:01.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.710+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/681734248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a53c2eac20 con 0x55a53c26fc10 2024-04-21T08:49:01.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.710+0000 7fb889ffb640 1 -- 172.21.15.43:0/681734248 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8900044d0 con 0x55a53c26fc10 2024-04-21T08:49:01.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.710+0000 7fb889ffb640 1 -- 172.21.15.43:0/681734248 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb89803f030 con 0x55a53c26fc10 2024-04-21T08:49:01.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.710+0000 7fb8ab1ea740 10 client.20472 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:01.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.710+0000 7fb8ab1ea740 10 client.20472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:01.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.710+0000 7fb8ab1ea740 20 client.20472 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:01.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.710+0000 7fb8ab1ea740 1 --2- 172.21.15.43:0/681734248 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a53c2eb340 0x55a53c2ed760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:01.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.710+0000 7fb8ab1ea740 4 client.20472 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:49:01.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.710+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/681734248 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a53c1d2a70 con 0x55a53c2eb340 2024-04-21T08:49:01.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.710+0000 7fb8a639d640 1 --2- 172.21.15.43:0/681734248 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a53c2eb340 0x55a53c2ed760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:01.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.711+0000 7fb889ffb640 10 client.20472 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:01.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.711+0000 7fb8a639d640 1 --2- 172.21.15.43:0/681734248 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a53c2eb340 0x55a53c2ed760 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64454ca1f240bbed server_cookie=3cb217bfada2df0f in_seq=0 out_seq=0 2024-04-21T08:49:01.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.711+0000 7fb889ffb640 1 -- 172.21.15.43:0/681734248 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a53c1d2a70 con 0x55a53c2eb340 2024-04-21T08:49:01.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.711+0000 7fb889ffb640 10 client.20472 handle_command_reply: tid=0 2024-04-21T08:49:01.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.711+0000 7fb8ab1ea740 1 client.20472 shutdown 2024-04-21T08:49:01.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.711+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/681734248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c26fc10 msgr2=0x55a53c274d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:01.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.711+0000 7fb8ab1ea740 1 --2- 172.21.15.43:0/681734248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c26fc10 0x55a53c274d70 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fb898012940 tx=0x7fb898012760 comp rx=0 tx=0).stop 2024-04-21T08:49:01.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.711+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/681734248 shutdown_connections 2024-04-21T08:49:01.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.711+0000 7fb8ab1ea740 1 --2- 172.21.15.43:0/681734248 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a53c2eb340 0x55a53c2ed760 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.711+0000 7fb8ab1ea740 1 --2- 172.21.15.43:0/681734248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53c26fc10 0x55a53c274d70 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.711+0000 7fb8ab1ea740 1 --2- 172.21.15.43:0/681734248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a53c267030 0x55a53c2782e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.711+0000 7fb8ab1ea740 1 --2- 172.21.15.43:0/681734248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a53c266710 0x55a53c277da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.711+0000 7fb889ffb640 0 client.20472 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:01.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.711+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/681734248 >> 172.21.15.43:0/681734248 conn(0x55a53c260e40 msgr2=0x55a53c262140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:01.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.711+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/681734248 shutdown_connections 2024-04-21T08:49:01.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.711+0000 7fb8ab1ea740 1 -- 172.21.15.43:0/681734248 wait complete. 2024-04-21T08:49:01.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.711+0000 7fb8ab1ea740 20 client.20472 trim_cache size 0 max 16384 2024-04-21T08:49:01.720 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:49:01.720 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:49:01.720 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-21T08:49:01.720 INFO:teuthology.orchestra.run.smithi043.stdout: "e814190c-dd38-482c-978b-8e4944dd11d9": { 2024-04-21T08:49:01.720 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-21T08:49:01.720 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "e814190c-dd38-482c-978b-8e4944dd11d9", 2024-04-21T08:49:01.720 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-21T08:49:01.720 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:49:01.720 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:49:01.720 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:49:01.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.712+0000 7fb8a739f640 1 -- 172.21.15.43:0/848341440 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb88c0a3d20 msgr2=0x7fb88c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:01.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.712+0000 7fb8a739f640 1 --2- 172.21.15.43:0/848341440 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb88c0a3d20 0x7fb88c0a61e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb898005cc0 tx=0x7fb89801c000 comp rx=0 tx=0).stop 2024-04-21T08:49:01.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.712+0000 7fb8a739f640 1 -- 172.21.15.43:0/848341440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a0100e70 msgr2=0x7fb8a01984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:01.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.712+0000 7fb8a739f640 1 --2- 172.21.15.43:0/848341440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a0100e70 0x7fb8a01984f0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00e9f0 tx=0x7fb89c00edf0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.712+0000 7fb8a739f640 1 -- 172.21.15.43:0/848341440 shutdown_connections 2024-04-21T08:49:01.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.712+0000 7fb8a739f640 1 --2- 172.21.15.43:0/848341440 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb88c0a3d20 0x7fb88c0a61e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.712+0000 7fb8a739f640 1 --2- 172.21.15.43:0/848341440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8a019cdb0 0x7fb8a019d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.712+0000 7fb8a739f640 1 --2- 172.21.15.43:0/848341440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8a0105f40 0x7fb8a0198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.712+0000 7fb8a739f640 1 --2- 172.21.15.43:0/848341440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a0100e70 0x7fb8a01984f0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:01.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.712+0000 7fb8a739f640 1 -- 172.21.15.43:0/848341440 >> 172.21.15.43:0/848341440 conn(0x7fb8a0077a30 msgr2=0x7fb8a00694a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:01.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.712+0000 7fb8a739f640 1 -- 172.21.15.43:0/848341440 shutdown_connections 2024-04-21T08:49:01.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:01.712+0000 7fb8a739f640 1 -- 172.21.15.43:0/848341440 wait complete. 2024-04-21T08:49:01.723 INFO:tasks.cephfs.filesystem:scrub status for tag:e814190c-dd38-482c-978b-8e4944dd11d9 - {'path': '/', 'tag': 'e814190c-dd38-482c-978b-8e4944dd11d9', 'options': 'recursive,force'} 2024-04-21T08:49:02.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:02 smithi175 ceph-mon[31920]: pgmap v313: 129 pgs: 129 active+clean; 4.9 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 106 KiB/s wr, 40 op/s 2024-04-21T08:49:02.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:02 smithi112 ceph-mon[31085]: pgmap v313: 129 pgs: 129 active+clean; 4.9 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 106 KiB/s wr, 40 op/s 2024-04-21T08:49:02.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:02 smithi043 ceph-mon[27978]: pgmap v313: 129 pgs: 129 active+clean; 4.9 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 106 KiB/s wr, 40 op/s 2024-04-21T08:49:03.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:03 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:49:03.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:03 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:03.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:03 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:49:03.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:03 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:03.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:03 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:49:03.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:03 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:03.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:03 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:49:03.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:03 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:03.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:03 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:49:03.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:03 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:03.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:03 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:49:03.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:03 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:04.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:04 smithi175 ceph-mon[31920]: pgmap v314: 129 pgs: 129 active+clean; 4.9 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 88 KiB/s wr, 33 op/s 2024-04-21T08:49:04.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:04 smithi112 ceph-mon[31085]: pgmap v314: 129 pgs: 129 active+clean; 4.9 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 88 KiB/s wr, 33 op/s 2024-04-21T08:49:04.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:04 smithi043 ceph-mon[27978]: pgmap v314: 129 pgs: 129 active+clean; 4.9 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 88 KiB/s wr, 33 op/s 2024-04-21T08:49:05.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:05 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:49:05.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:05 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:49:05.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:05 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:49:06.463 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:06 smithi043 ceph-mon[27978]: pgmap v315: 129 pgs: 129 active+clean; 4.9 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 88 KiB/s wr, 33 op/s 2024-04-21T08:49:06.463 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:06 smithi043 ceph-mon[27978]: osdmap e103: 12 total, 12 up, 12 in 2024-04-21T08:49:06.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:06 smithi175 ceph-mon[31920]: pgmap v315: 129 pgs: 129 active+clean; 4.9 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 88 KiB/s wr, 33 op/s 2024-04-21T08:49:06.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:06 smithi175 ceph-mon[31920]: osdmap e103: 12 total, 12 up, 12 in 2024-04-21T08:49:06.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:06 smithi112 ceph-mon[31085]: pgmap v315: 129 pgs: 129 active+clean; 4.9 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 88 KiB/s wr, 33 op/s 2024-04-21T08:49:06.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:06 smithi112 ceph-mon[31085]: osdmap e103: 12 total, 12 up, 12 in 2024-04-21T08:49:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.188+0000 7f841f196640 1 -- 172.21.15.43:0/3272306478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8418106170 msgr2=0x7f841810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.188+0000 7f841f196640 1 --2- 172.21.15.43:0/3272306478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8418106170 0x7f841810dd40 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f8410009ca0 tx=0x7f841002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.188+0000 7f841f196640 1 -- 172.21.15.43:0/3272306478 shutdown_connections 2024-04-21T08:49:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.188+0000 7f841f196640 1 --2- 172.21.15.43:0/3272306478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8418106170 0x7f841810dd40 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.188+0000 7f841f196640 1 --2- 172.21.15.43:0/3272306478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8418101aa0 0x7f8418105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.188+0000 7f841f196640 1 --2- 172.21.15.43:0/3272306478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84181010f0 0x7f84181014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.188+0000 7f841f196640 1 -- 172.21.15.43:0/3272306478 >> 172.21.15.43:0/3272306478 conn(0x7f8418077fd0 msgr2=0x7f84180783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.188+0000 7f841f196640 1 -- 172.21.15.43:0/3272306478 shutdown_connections 2024-04-21T08:49:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.188+0000 7f841f196640 1 -- 172.21.15.43:0/3272306478 wait complete. 2024-04-21T08:49:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.188+0000 7f841f196640 1 Processor -- start 2024-04-21T08:49:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.188+0000 7f841f196640 1 -- start start 2024-04-21T08:49:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.189+0000 7f841f196640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84181010f0 0x7f8418102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.189+0000 7f841f196640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8418106170 0x7f8418103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.189+0000 7f841f196640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84181035b0 0x7f84181a6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.189+0000 7f841f196640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f84181110e0 con 0x7f84181035b0 2024-04-21T08:49:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.189+0000 7f841f196640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8418110f60 con 0x7f8418106170 2024-04-21T08:49:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.189+0000 7f841f196640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8418111260 con 0x7f84181010f0 2024-04-21T08:49:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.189+0000 7f841d733640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84181035b0 0x7f84181a6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.189+0000 7f841d733640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84181035b0 0x7f84181a6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49110/0 (socket says 172.21.15.43:49110) 2024-04-21T08:49:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.189+0000 7f841d733640 1 -- 172.21.15.43:0/1304946619 learned_addr learned my addr 172.21.15.43:0/1304946619 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.189+0000 7f841cf32640 1 --2- 172.21.15.43:0/1304946619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84181010f0 0x7f8418102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.189+0000 7f8417fff640 1 --2- 172.21.15.43:0/1304946619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8418106170 0x7f8418103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.189+0000 7f841d733640 1 -- 172.21.15.43:0/1304946619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84181010f0 msgr2=0x7f8418102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.189+0000 7f841d733640 1 --2- 172.21.15.43:0/1304946619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84181010f0 0x7f8418102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.189+0000 7f841d733640 1 -- 172.21.15.43:0/1304946619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8418106170 msgr2=0x7f8418103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.189+0000 7f841d733640 1 --2- 172.21.15.43:0/1304946619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8418106170 0x7f8418103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.189+0000 7f841d733640 1 -- 172.21.15.43:0/1304946619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84181a6f70 con 0x7f84181035b0 2024-04-21T08:49:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.189+0000 7f841d733640 1 --2- 172.21.15.43:0/1304946619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84181035b0 0x7f84181a6870 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f8410005e30 tx=0x7f84100346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.189+0000 7f8415ffb640 1 -- 172.21.15.43:0/1304946619 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410041020 con 0x7f84181035b0 2024-04-21T08:49:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.189+0000 7f8415ffb640 1 -- 172.21.15.43:0/1304946619 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8410032970 con 0x7f84181035b0 2024-04-21T08:49:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.189+0000 7f841f196640 1 -- 172.21.15.43:0/1304946619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84181a7200 con 0x7f84181035b0 2024-04-21T08:49:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.189+0000 7f8415ffb640 1 -- 172.21.15.43:0/1304946619 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410039040 con 0x7f84181035b0 2024-04-21T08:49:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.190+0000 7f841f196640 1 -- 172.21.15.43:0/1304946619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84181a76f0 con 0x7f84181035b0 2024-04-21T08:49:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.192+0000 7f8421d86740 1 Processor -- start 2024-04-21T08:49:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.192+0000 7f8415ffb640 1 -- 172.21.15.43:0/1304946619 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8410044020 con 0x7f84181035b0 2024-04-21T08:49:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.192+0000 7f8421d86740 1 -- start start 2024-04-21T08:49:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.192+0000 7f8421d86740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55871c0378d0 0x55871c036620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.192+0000 7f8421d86740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c036b60 0x55871c036fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.192+0000 7f8421d86740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871c0400e0 0x55871c0424d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.192+0000 7f8421d86740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55871bf9b8d0 con 0x55871c036b60 2024-04-21T08:49:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.192+0000 7f8421d86740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55871bf38e80 con 0x55871c0400e0 2024-04-21T08:49:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.192+0000 7f8421d86740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55871bf97890 con 0x55871c0378d0 2024-04-21T08:49:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.192+0000 7f841cf32640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c036b60 0x55871c036fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.192+0000 7f841cf32640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c036b60 0x55871c036fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49112/0 (socket says 172.21.15.43:49112) 2024-04-21T08:49:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.192+0000 7f841cf32640 1 -- 172.21.15.43:0/286876253 learned_addr learned my addr 172.21.15.43:0/286876253 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.192+0000 7f8417fff640 1 --2- 172.21.15.43:0/286876253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871c0400e0 0x55871c0424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.193+0000 7f8417fff640 1 --2- 172.21.15.43:0/286876253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55871c0378d0 0x55871c036620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.193+0000 7f841cf32640 1 -- 172.21.15.43:0/286876253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55871c0378d0 msgr2=0x55871c036620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.193+0000 7f841cf32640 1 --2- 172.21.15.43:0/286876253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55871c0378d0 0x55871c036620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.193+0000 7f841cf32640 1 -- 172.21.15.43:0/286876253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871c0400e0 msgr2=0x55871c0424d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.193+0000 7f841cf32640 1 --2- 172.21.15.43:0/286876253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871c0400e0 0x55871c0424d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.193+0000 7f841cf32640 1 -- 172.21.15.43:0/286876253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55871bf5a320 con 0x55871c036b60 2024-04-21T08:49:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.193+0000 7f8415ffb640 1 --2- 172.21.15.43:0/1304946619 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84080a3df0 0x7f84080a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.193+0000 7f8417fff640 1 --2- 172.21.15.43:0/1304946619 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84080a3df0 0x7f84080a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.193+0000 7f841cf32640 1 --2- 172.21.15.43:0/286876253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c036b60 0x55871c036fe0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f840c009d20 tx=0x7f840c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be13dca51bada49 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.193+0000 7f8415ffb640 1 -- 172.21.15.43:0/1304946619 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9671+0+0 (secure 0 0 0) 0x7f84100f26d0 con 0x7f84181035b0 2024-04-21T08:49:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.193+0000 7f8402ffd640 1 -- 172.21.15.43:0/286876253 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c01e070 con 0x55871c036b60 2024-04-21T08:49:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.193+0000 7f8402ffd640 1 -- 172.21.15.43:0/286876253 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f840c009e10 con 0x55871c036b60 2024-04-21T08:49:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.193+0000 7f8402ffd640 1 -- 172.21.15.43:0/286876253 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c0187a0 con 0x55871c036b60 2024-04-21T08:49:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.194+0000 7f8421d86740 1 -- 172.21.15.43:0/286876253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c036b60 msgr2=0x55871c036fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.194+0000 7f8417fff640 1 --2- 172.21.15.43:0/1304946619 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84080a3df0 0x7f84080a62b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x55871c0329e0 tx=0x7f84040046c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.194+0000 7f8421d86740 1 --2- 172.21.15.43:0/286876253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c036b60 0x55871c036fe0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f840c009d20 tx=0x7f840c011810 comp rx=0 tx=0).stop 2024-04-21T08:49:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.194+0000 7f8421d86740 1 -- 172.21.15.43:0/286876253 shutdown_connections 2024-04-21T08:49:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.194+0000 7f8421d86740 1 --2- 172.21.15.43:0/286876253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871c0400e0 0x55871c0424d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.194+0000 7f8421d86740 1 --2- 172.21.15.43:0/286876253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c036b60 0x55871c036fe0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.194+0000 7f8421d86740 1 --2- 172.21.15.43:0/286876253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55871c0378d0 0x55871c036620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.194+0000 7f8421d86740 1 -- 172.21.15.43:0/286876253 >> 172.21.15.43:0/286876253 conn(0x55871bffdc90 msgr2=0x55871c031850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.194+0000 7f8421d86740 1 -- 172.21.15.43:0/286876253 shutdown_connections 2024-04-21T08:49:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.194+0000 7f8421d86740 1 -- 172.21.15.43:0/286876253 wait complete. 2024-04-21T08:49:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.194+0000 7f8421d86740 1 -- start start 2024-04-21T08:49:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.194+0000 7f8421d86740 1 Processor -- start 2024-04-21T08:49:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.195+0000 7f8421d86740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c036b60 0x55871c03fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.195+0000 7f8421d86740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55871c0378d0 0x55871c03ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.195+0000 7f8421d86740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871c0400e0 0x55871c03d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.195+0000 7f8421d86740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55871c042e10 con 0x55871c036b60 2024-04-21T08:49:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.195+0000 7f8421d86740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55871bf23eb0 con 0x55871c0400e0 2024-04-21T08:49:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.195+0000 7f8421d86740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55871c042f90 con 0x55871c0378d0 2024-04-21T08:49:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.195+0000 7f8421d86740 10 client.0 _ll_register_callbacks cb 0x55871bf50b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.195+0000 7f841cf32640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c036b60 0x55871c03fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.195+0000 7f8417fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55871c0378d0 0x55871c03ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.195+0000 7f841cf32640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c036b60 0x55871c03fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49124/0 (socket says 172.21.15.43:49124) 2024-04-21T08:49:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.195+0000 7f841d733640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871c0400e0 0x55871c03d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.195+0000 7f841cf32640 1 -- 172.21.15.43:0/2194976641 learned_addr learned my addr 172.21.15.43:0/2194976641 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.195+0000 7f841cf32640 1 -- 172.21.15.43:0/2194976641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55871c0378d0 msgr2=0x55871c03ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.195+0000 7f841cf32640 1 --2- 172.21.15.43:0/2194976641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55871c0378d0 0x55871c03ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.195+0000 7f841cf32640 1 -- 172.21.15.43:0/2194976641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871c0400e0 msgr2=0x55871c03d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.195+0000 7f841cf32640 1 --2- 172.21.15.43:0/2194976641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871c0400e0 0x55871c03d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.195+0000 7f841cf32640 1 -- 172.21.15.43:0/2194976641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55871bcae210 con 0x55871c036b60 2024-04-21T08:49:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.195+0000 7f841d733640 1 --2- 172.21.15.43:0/2194976641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871c0400e0 0x55871c03d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.195+0000 7f841cf32640 1 --2- 172.21.15.43:0/2194976641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c036b60 0x55871c03fa70 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f840c009cf0 tx=0x7f840c018c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02bab0d252ad0d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.195+0000 7f83f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.196+0000 7f83f7fff640 1 -- 172.21.15.43:0/2194976641 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c01e040 con 0x55871c036b60 2024-04-21T08:49:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.196+0000 7f83f7fff640 1 -- 172.21.15.43:0/2194976641 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9671+0+0 (secure 0 0 0) 0x7f840c018df0 con 0x55871c036b60 2024-04-21T08:49:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.196+0000 7f8421d86740 1 -- 172.21.15.43:0/2194976641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f840c009e10 con 0x55871c036b60 2024-04-21T08:49:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.196+0000 7f83f7fff640 1 -- 172.21.15.43:0/2194976641 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f840c02f050 con 0x55871c036b60 2024-04-21T08:49:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.196+0000 7f8421d86740 10 client.20484 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.196+0000 7f8421d86740 1 -- 172.21.15.43:0/2194976641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55871bd7a620 con 0x55871c036b60 2024-04-21T08:49:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.196+0000 7f83f7fff640 1 -- 172.21.15.43:0/2194976641 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f840c009e10 con 0x55871c036b60 2024-04-21T08:49:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.196+0000 7f83f7fff640 1 -- 172.21.15.43:0/2194976641 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f840c02a030 con 0x55871c036b60 2024-04-21T08:49:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.196+0000 7f8421d86740 10 client.20484 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.196+0000 7f8421d86740 10 client.20484 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.196+0000 7f8421d86740 20 client.20484 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.196+0000 7f8421d86740 1 --2- 172.21.15.43:0/2194976641 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55871c050010 0x55871c052400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.196+0000 7f8421d86740 4 client.20484 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.196+0000 7f8421d86740 1 -- 172.21.15.43:0/2194976641 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55871bfaf880 con 0x55871c050010 2024-04-21T08:49:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.197+0000 7f8417fff640 1 --2- 172.21.15.43:0/2194976641 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55871c050010 0x55871c052400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.197+0000 7f8417fff640 1 --2- 172.21.15.43:0/2194976641 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55871c050010 0x55871c052400 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6c504c8066978a9 server_cookie=e3bf78f750431825 in_seq=0 out_seq=0 2024-04-21T08:49:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.197+0000 7f83f7fff640 10 client.20484 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.198+0000 7f83f7fff640 1 -- 172.21.15.43:0/2194976641 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55871bfaf880 con 0x55871c050010 2024-04-21T08:49:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.198+0000 7f83f7fff640 10 client.20484 handle_command_reply: tid=0 2024-04-21T08:49:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.198+0000 7f8421d86740 1 client.20484 shutdown 2024-04-21T08:49:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.198+0000 7f8421d86740 1 -- 172.21.15.43:0/2194976641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c036b60 msgr2=0x55871c03fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.198+0000 7f8421d86740 1 --2- 172.21.15.43:0/2194976641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c036b60 0x55871c03fa70 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f840c009cf0 tx=0x7f840c018c00 comp rx=0 tx=0).stop 2024-04-21T08:49:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.198+0000 7f8421d86740 1 -- 172.21.15.43:0/2194976641 shutdown_connections 2024-04-21T08:49:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.198+0000 7f8421d86740 1 --2- 172.21.15.43:0/2194976641 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55871c050010 0x55871c052400 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.198+0000 7f8421d86740 1 --2- 172.21.15.43:0/2194976641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871c0400e0 0x55871c03d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.198+0000 7f8421d86740 1 --2- 172.21.15.43:0/2194976641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55871c0378d0 0x55871c03ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.198+0000 7f8421d86740 1 --2- 172.21.15.43:0/2194976641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c036b60 0x55871c03fa70 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.198+0000 7f8421d86740 1 -- 172.21.15.43:0/2194976641 >> 172.21.15.43:0/2194976641 conn(0x55871bffdc90 msgr2=0x55871c0335a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.198+0000 7f83f7fff640 0 client.20484 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.198+0000 7f8421d86740 1 -- 172.21.15.43:0/2194976641 shutdown_connections 2024-04-21T08:49:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.198+0000 7f8421d86740 1 -- 172.21.15.43:0/2194976641 wait complete. 2024-04-21T08:49:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.198+0000 7f8421d86740 20 client.20484 trim_cache size 0 max 16384 2024-04-21T08:49:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.213+0000 7f8421d86740 1 Processor -- start 2024-04-21T08:49:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.213+0000 7f8421d86740 1 -- start start 2024-04-21T08:49:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.213+0000 7f8421d86740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55871c036b60 0x55871c04bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.213+0000 7f8421d86740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c0378d0 0x55871c03f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.213+0000 7f8421d86740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871c0400e0 0x55871c03fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.213+0000 7f8421d86740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f84181b17a0 con 0x55871c0378d0 2024-04-21T08:49:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.213+0000 7f8421d86740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f84181b32e0 con 0x55871c0400e0 2024-04-21T08:49:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.213+0000 7f8421d86740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f84181b1e70 con 0x55871c036b60 2024-04-21T08:49:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.213+0000 7f8417fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c0378d0 0x55871c03f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.214+0000 7f8417fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c0378d0 0x55871c03f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49126/0 (socket says 172.21.15.43:49126) 2024-04-21T08:49:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.214+0000 7f8417fff640 1 -- 172.21.15.43:0/1598027466 learned_addr learned my addr 172.21.15.43:0/1598027466 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.214+0000 7f841d733640 1 --2- 172.21.15.43:0/1598027466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871c0400e0 0x55871c03fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.214+0000 7f841cf32640 1 --2- 172.21.15.43:0/1598027466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55871c036b60 0x55871c04bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.214+0000 7f8417fff640 1 -- 172.21.15.43:0/1598027466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55871c036b60 msgr2=0x55871c04bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.214+0000 7f8417fff640 1 --2- 172.21.15.43:0/1598027466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55871c036b60 0x55871c04bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.214+0000 7f8417fff640 1 -- 172.21.15.43:0/1598027466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871c0400e0 msgr2=0x55871c03fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.214+0000 7f8417fff640 1 --2- 172.21.15.43:0/1598027466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871c0400e0 0x55871c03fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.214+0000 7f8417fff640 1 -- 172.21.15.43:0/1598027466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55871c035e30 con 0x55871c0378d0 2024-04-21T08:49:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.214+0000 7f841d733640 1 --2- 172.21.15.43:0/1598027466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871c0400e0 0x55871c03fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.214+0000 7f8417fff640 1 --2- 172.21.15.43:0/1598027466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c0378d0 0x55871c03f530 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f84181045c0 tx=0x7f8404013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195b3445001cc69b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.214+0000 7f84027fc640 1 -- 172.21.15.43:0/1598027466 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404010ce0 con 0x55871c0378d0 2024-04-21T08:49:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.214+0000 7f84027fc640 1 -- 172.21.15.43:0/1598027466 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8404010e80 con 0x55871c0378d0 2024-04-21T08:49:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.214+0000 7f84027fc640 1 -- 172.21.15.43:0/1598027466 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84040047a0 con 0x55871c0378d0 2024-04-21T08:49:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.215+0000 7f8421d86740 1 -- 172.21.15.43:0/1598027466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c0378d0 msgr2=0x55871c03f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.215+0000 7f8421d86740 1 --2- 172.21.15.43:0/1598027466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c0378d0 0x55871c03f530 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f84181045c0 tx=0x7f8404013040 comp rx=0 tx=0).stop 2024-04-21T08:49:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.215+0000 7f8421d86740 1 -- 172.21.15.43:0/1598027466 shutdown_connections 2024-04-21T08:49:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.215+0000 7f8421d86740 1 --2- 172.21.15.43:0/1598027466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871c0400e0 0x55871c03fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.215+0000 7f8421d86740 1 --2- 172.21.15.43:0/1598027466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c0378d0 0x55871c03f530 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.215+0000 7f8421d86740 1 --2- 172.21.15.43:0/1598027466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55871c036b60 0x55871c04bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.215+0000 7f8421d86740 1 -- 172.21.15.43:0/1598027466 >> 172.21.15.43:0/1598027466 conn(0x55871bffdc90 msgr2=0x55871c031380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.215+0000 7f8421d86740 1 -- 172.21.15.43:0/1598027466 shutdown_connections 2024-04-21T08:49:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.215+0000 7f8421d86740 1 -- 172.21.15.43:0/1598027466 wait complete. 2024-04-21T08:49:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.215+0000 7f8421d86740 1 -- start start 2024-04-21T08:49:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.215+0000 7f8421d86740 1 Processor -- start 2024-04-21T08:49:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.216+0000 7f8421d86740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871c036b60 0x55871c048220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.216+0000 7f8421d86740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c0378d0 0x55871c048760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.216+0000 7f8421d86740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55871c0400e0 0x55871c0392b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.216+0000 7f8421d86740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f84181b2c10 con 0x55871c0378d0 2024-04-21T08:49:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.216+0000 7f8421d86740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f84181b2540 con 0x55871c036b60 2024-04-21T08:49:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.216+0000 7f8421d86740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f84181af7a0 con 0x55871c0400e0 2024-04-21T08:49:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.216+0000 7f8421d86740 10 client.0 _ll_register_callbacks cb 0x7f840c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.216+0000 7f8417fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c0378d0 0x55871c048760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.216+0000 7f841cf32640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871c036b60 0x55871c048220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.216+0000 7f8417fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c0378d0 0x55871c048760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49132/0 (socket says 172.21.15.43:49132) 2024-04-21T08:49:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.216+0000 7f8417fff640 1 -- 172.21.15.43:0/2518967732 learned_addr learned my addr 172.21.15.43:0/2518967732 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.216+0000 7f841d733640 1 --2- 172.21.15.43:0/2518967732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55871c0400e0 0x55871c0392b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.216+0000 7f841cf32640 1 -- 172.21.15.43:0/2518967732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55871c0400e0 msgr2=0x55871c0392b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.216+0000 7f841cf32640 1 --2- 172.21.15.43:0/2518967732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55871c0400e0 0x55871c0392b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.216+0000 7f841cf32640 1 -- 172.21.15.43:0/2518967732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c0378d0 msgr2=0x55871c048760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.216+0000 7f841cf32640 1 --2- 172.21.15.43:0/2518967732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c0378d0 0x55871c048760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.216+0000 7f841cf32640 1 -- 172.21.15.43:0/2518967732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55871c039ae0 con 0x55871c036b60 2024-04-21T08:49:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.216+0000 7f8417fff640 1 --2- 172.21.15.43:0/2518967732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c0378d0 0x55871c048760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.217+0000 7f841cf32640 1 --2- 172.21.15.43:0/2518967732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871c036b60 0x55871c048220 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f840c0047b0 tx=0x7f840c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b8fd93413bd11ca server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.217+0000 7f83f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-21T08:49:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.217+0000 7f83f7fff640 1 -- 172.21.15.43:0/2518967732 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c037020 con 0x55871c036b60 2024-04-21T08:49:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.217+0000 7f83f7fff640 1 -- 172.21.15.43:0/2518967732 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9671+0+0 (secure 0 0 0) 0x7f840c019410 con 0x55871c036b60 2024-04-21T08:49:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.217+0000 7f8421d86740 1 -- 172.21.15.43:0/2518967732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55871bf3b3d0 con 0x55871c036b60 2024-04-21T08:49:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.217+0000 7f83f7fff640 1 -- 172.21.15.43:0/2518967732 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f840c03f050 con 0x55871c036b60 2024-04-21T08:49:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.217+0000 7f8421d86740 10 client.29111 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.217+0000 7f8421d86740 1 -- 172.21.15.43:0/2518967732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55871c0b5ed0 con 0x55871c036b60 2024-04-21T08:49:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.217+0000 7f83f7fff640 1 -- 172.21.15.43:0/2518967732 <== mon.2 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55871bf3b3d0 con 0x55871c036b60 2024-04-21T08:49:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.217+0000 7f83f7fff640 1 -- 172.21.15.43:0/2518967732 <== mon.2 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f840c02a040 con 0x55871c036b60 2024-04-21T08:49:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.217+0000 7f8421d86740 10 client.29111 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.217+0000 7f8421d86740 10 client.29111 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.217+0000 7f8421d86740 20 client.29111 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.217+0000 7f8421d86740 1 --2- 172.21.15.43:0/2518967732 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55871c0b6520 0x55871c0b8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.217+0000 7f8421d86740 4 client.29111 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:49:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.217+0000 7f8421d86740 1 -- 172.21.15.43:0/2518967732 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55871bfa92f0 con 0x55871c0b6520 2024-04-21T08:49:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.217+0000 7f8417fff640 1 --2- 172.21.15.43:0/2518967732 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55871c0b6520 0x55871c0b8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.218+0000 7f83f7fff640 10 client.29111 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.218+0000 7f8417fff640 1 --2- 172.21.15.43:0/2518967732 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55871c0b6520 0x55871c0b8910 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52f25455fb35be8a server_cookie=4b2d1e0f516db10b in_seq=0 out_seq=0 2024-04-21T08:49:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.232+0000 7f83f7fff640 1 -- 172.21.15.43:0/2518967732 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55871bfa92f0 con 0x55871c0b6520 2024-04-21T08:49:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.232+0000 7f83f7fff640 10 client.29111 handle_command_reply: tid=0 2024-04-21T08:49:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.232+0000 7f8421d86740 1 client.29111 shutdown 2024-04-21T08:49:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.233+0000 7f8421d86740 1 -- 172.21.15.43:0/2518967732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871c036b60 msgr2=0x55871c048220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.233+0000 7f8421d86740 1 --2- 172.21.15.43:0/2518967732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871c036b60 0x55871c048220 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f840c0047b0 tx=0x7f840c009c20 comp rx=0 tx=0).stop 2024-04-21T08:49:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.233+0000 7f8421d86740 1 -- 172.21.15.43:0/2518967732 shutdown_connections 2024-04-21T08:49:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.233+0000 7f8421d86740 1 --2- 172.21.15.43:0/2518967732 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55871c0b6520 0x55871c0b8910 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.233+0000 7f8421d86740 1 --2- 172.21.15.43:0/2518967732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55871c0400e0 0x55871c0392b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.233+0000 7f8421d86740 1 --2- 172.21.15.43:0/2518967732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55871c0378d0 0x55871c048760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.233+0000 7f8421d86740 1 --2- 172.21.15.43:0/2518967732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871c036b60 0x55871c048220 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.233+0000 7f83f7fff640 0 client.29111 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.233+0000 7f8421d86740 1 -- 172.21.15.43:0/2518967732 >> 172.21.15.43:0/2518967732 conn(0x55871bffdc90 msgr2=0x55871c037ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.233+0000 7f8421d86740 1 -- 172.21.15.43:0/2518967732 shutdown_connections 2024-04-21T08:49:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.233+0000 7f8421d86740 1 -- 172.21.15.43:0/2518967732 wait complete. 2024-04-21T08:49:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.233+0000 7f8421d86740 20 client.29111 trim_cache size 0 max 16384 2024-04-21T08:49:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:49:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:49:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:49:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:49:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.233+0000 7f841f196640 1 -- 172.21.15.43:0/1304946619 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84080a3df0 msgr2=0x7f84080a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.233+0000 7f841f196640 1 --2- 172.21.15.43:0/1304946619 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84080a3df0 0x7f84080a62b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x55871c0329e0 tx=0x7f84040046c0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.233+0000 7f841f196640 1 -- 172.21.15.43:0/1304946619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84181035b0 msgr2=0x7f84181a6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.233+0000 7f841f196640 1 --2- 172.21.15.43:0/1304946619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84181035b0 0x7f84181a6870 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f8410005e30 tx=0x7f84100346d0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.233+0000 7f841f196640 1 -- 172.21.15.43:0/1304946619 shutdown_connections 2024-04-21T08:49:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.233+0000 7f841f196640 1 --2- 172.21.15.43:0/1304946619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84181035b0 0x7f84181a6870 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.233+0000 7f841f196640 1 --2- 172.21.15.43:0/1304946619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8418106170 0x7f8418103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.233+0000 7f841f196640 1 --2- 172.21.15.43:0/1304946619 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84080a3df0 0x7f84080a62b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.233+0000 7f841f196640 1 --2- 172.21.15.43:0/1304946619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84181010f0 0x7f8418102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.233+0000 7f841f196640 1 -- 172.21.15.43:0/1304946619 >> 172.21.15.43:0/1304946619 conn(0x7f8418077fd0 msgr2=0x7f8418069880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.234+0000 7f841f196640 1 -- 172.21.15.43:0/1304946619 shutdown_connections 2024-04-21T08:49:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:08.234+0000 7f841f196640 1 -- 172.21.15.43:0/1304946619 wait complete. 2024-04-21T08:49:08.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:08 smithi175 ceph-mon[31920]: pgmap v317: 129 pgs: 129 active+clean; 5.1 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 100 KiB/s wr, 39 op/s 2024-04-21T08:49:08.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:08 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:49:08.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:08 smithi112 ceph-mon[31085]: pgmap v317: 129 pgs: 129 active+clean; 5.1 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 100 KiB/s wr, 39 op/s 2024-04-21T08:49:08.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:08 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:49:08.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:08 smithi043 ceph-mon[27978]: pgmap v317: 129 pgs: 129 active+clean; 5.1 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 100 KiB/s wr, 39 op/s 2024-04-21T08:49:08.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:08 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:49:10.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:10 smithi175 ceph-mon[31920]: pgmap v318: 129 pgs: 129 active+clean; 5.1 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 100 KiB/s wr, 39 op/s 2024-04-21T08:49:10.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:10 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:49:10.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:10 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:49:10.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:10 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:49:10.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:10 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:49:10.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:10 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:49:10.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:10 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:49:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:10 smithi112 ceph-mon[31085]: pgmap v318: 129 pgs: 129 active+clean; 5.1 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 100 KiB/s wr, 39 op/s 2024-04-21T08:49:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:10 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:49:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:10 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:49:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:10 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:49:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:10 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:49:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:10 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:49:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:10 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:49:10.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:10 smithi043 ceph-mon[27978]: pgmap v318: 129 pgs: 129 active+clean; 5.1 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 100 KiB/s wr, 39 op/s 2024-04-21T08:49:10.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:10 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:49:10.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:10 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:49:10.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:10 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:49:10.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:10 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:49:10.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:10 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:49:10.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:10 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:49:11.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:11 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:11 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:11 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:11 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:11 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:11 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:11 smithi175 ceph-mon[31920]: Adjusting osd_memory_target on smithi175 to 176.2M 2024-04-21T08:49:11.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:11 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:11 smithi175 ceph-mon[31920]: Unable to set osd_memory_target on smithi175 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-21T08:49:11.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:11 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:11 smithi175 ceph-mon[31920]: Adjusting osd_memory_target on smithi112 to 176.2M 2024-04-21T08:49:11.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:11 smithi175 ceph-mon[31920]: Unable to set osd_memory_target on smithi112 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-04-21T08:49:11.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:11 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:11 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:11 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:11 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:11 smithi175 ceph-mon[31920]: Adjusting osd_memory_target on smithi043 to 177.2M 2024-04-21T08:49:11.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:11 smithi175 ceph-mon[31920]: Unable to set osd_memory_target on smithi043 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-21T08:49:11.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:11 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:49:11.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:11 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:49:11.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:11 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:49:11.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:11 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:11 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:11 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:11 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:11 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:11 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:11 smithi112 ceph-mon[31085]: Adjusting osd_memory_target on smithi175 to 176.2M 2024-04-21T08:49:11.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:11 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:11 smithi112 ceph-mon[31085]: Unable to set osd_memory_target on smithi175 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-21T08:49:11.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:11 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:11 smithi112 ceph-mon[31085]: Adjusting osd_memory_target on smithi112 to 176.2M 2024-04-21T08:49:11.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:11 smithi112 ceph-mon[31085]: Unable to set osd_memory_target on smithi112 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-04-21T08:49:11.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:11 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:11 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:11 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:11 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:11 smithi112 ceph-mon[31085]: Adjusting osd_memory_target on smithi043 to 177.2M 2024-04-21T08:49:11.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:11 smithi112 ceph-mon[31085]: Unable to set osd_memory_target on smithi043 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-21T08:49:11.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:11 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:49:11.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:11 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:49:11.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:11 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:49:11.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:11 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:11 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:11 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:11 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:11 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:11 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:11 smithi043 ceph-mon[27978]: Adjusting osd_memory_target on smithi175 to 176.2M 2024-04-21T08:49:11.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:11 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:11 smithi043 ceph-mon[27978]: Unable to set osd_memory_target on smithi175 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-21T08:49:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:11 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:11 smithi043 ceph-mon[27978]: Adjusting osd_memory_target on smithi112 to 176.2M 2024-04-21T08:49:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:11 smithi043 ceph-mon[27978]: Unable to set osd_memory_target on smithi112 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-04-21T08:49:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:11 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:11 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:11 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:11 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:11 smithi043 ceph-mon[27978]: Adjusting osd_memory_target on smithi043 to 177.2M 2024-04-21T08:49:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:11 smithi043 ceph-mon[27978]: Unable to set osd_memory_target on smithi043 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-21T08:49:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:11 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:49:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:11 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:49:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:11 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:49:12.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:12 smithi175 ceph-mon[31920]: pgmap v319: 129 pgs: 129 active+clean; 5.3 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 20 op/s 2024-04-21T08:49:12.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:12 smithi112 ceph-mon[31085]: pgmap v319: 129 pgs: 129 active+clean; 5.3 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 20 op/s 2024-04-21T08:49:12.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:12 smithi043 ceph-mon[27978]: pgmap v319: 129 pgs: 129 active+clean; 5.3 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 20 op/s 2024-04-21T08:49:14.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:14 smithi175 ceph-mon[31920]: pgmap v320: 129 pgs: 129 active+clean; 5.3 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 20 op/s 2024-04-21T08:49:14.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:14 smithi112 ceph-mon[31085]: pgmap v320: 129 pgs: 129 active+clean; 5.3 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 20 op/s 2024-04-21T08:49:14.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:14 smithi043 ceph-mon[27978]: pgmap v320: 129 pgs: 129 active+clean; 5.3 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 20 op/s 2024-04-21T08:49:16.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:16 smithi043 ceph-mon[27978]: osdmap e104: 12 total, 12 up, 12 in 2024-04-21T08:49:16.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:16 smithi043 ceph-mon[27978]: pgmap v322: 129 pgs: 129 active+clean; 5.3 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-04-21T08:49:16.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:16 smithi175 ceph-mon[31920]: osdmap e104: 12 total, 12 up, 12 in 2024-04-21T08:49:16.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:16 smithi175 ceph-mon[31920]: pgmap v322: 129 pgs: 129 active+clean; 5.3 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-04-21T08:49:16.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:16 smithi112 ceph-mon[31085]: osdmap e104: 12 total, 12 up, 12 in 2024-04-21T08:49:16.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:16 smithi112 ceph-mon[31085]: pgmap v322: 129 pgs: 129 active+clean; 5.3 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-04-21T08:49:18.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:18 smithi175 ceph-mon[31920]: pgmap v323: 129 pgs: 129 active+clean; 5.5 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 19 op/s 2024-04-21T08:49:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:18 smithi112 ceph-mon[31085]: pgmap v323: 129 pgs: 129 active+clean; 5.5 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 19 op/s 2024-04-21T08:49:18.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:18 smithi043 ceph-mon[27978]: pgmap v323: 129 pgs: 129 active+clean; 5.5 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 19 op/s 2024-04-21T08:49:20.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:20 smithi175 ceph-mon[31920]: pgmap v324: 129 pgs: 129 active+clean; 5.5 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 19 op/s 2024-04-21T08:49:20.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:20 smithi112 ceph-mon[31085]: pgmap v324: 129 pgs: 129 active+clean; 5.5 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 19 op/s 2024-04-21T08:49:20.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:20 smithi043 ceph-mon[27978]: pgmap v324: 129 pgs: 129 active+clean; 5.5 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 19 op/s 2024-04-21T08:49:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.333+0000 7f29cf175640 1 -- 172.21.15.43:0/2592166618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c806b120 msgr2=0x7f29c810edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.333+0000 7f29cf175640 1 --2- 172.21.15.43:0/2592166618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c806b120 0x7f29c810edf0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f29c0009cd0 tx=0x7f29c002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.333+0000 7f29cf175640 1 -- 172.21.15.43:0/2592166618 shutdown_connections 2024-04-21T08:49:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.333+0000 7f29cf175640 1 --2- 172.21.15.43:0/2592166618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c806b120 0x7f29c810edf0 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.333+0000 7f29cf175640 1 --2- 172.21.15.43:0/2592166618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c806a780 0x7f29c806abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.333+0000 7f29cf175640 1 --2- 172.21.15.43:0/2592166618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29c8107fe0 0x7f29c81083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.333+0000 7f29cf175640 1 -- 172.21.15.43:0/2592166618 >> 172.21.15.43:0/2592166618 conn(0x7f29c8076c50 msgr2=0x7f29c8077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.333+0000 7f29cf175640 1 -- 172.21.15.43:0/2592166618 shutdown_connections 2024-04-21T08:49:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.333+0000 7f29cf175640 1 -- 172.21.15.43:0/2592166618 wait complete. 2024-04-21T08:49:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.334+0000 7f29cf175640 1 Processor -- start 2024-04-21T08:49:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.334+0000 7f29cf175640 1 -- start start 2024-04-21T08:49:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.334+0000 7f29cf175640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c806a780 0x7f29c8198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.334+0000 7f29cf175640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c806b120 0x7f29c8198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.334+0000 7f29cf175640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29c8107fe0 0x7f29c819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.334+0000 7f29cf175640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f29c8112180 con 0x7f29c806b120 2024-04-21T08:49:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.334+0000 7f29cf175640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f29c8112000 con 0x7f29c8107fe0 2024-04-21T08:49:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.334+0000 7f29cf175640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29c8112300 con 0x7f29c806a780 2024-04-21T08:49:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.334+0000 7f29c7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c806b120 0x7f29c8198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.334+0000 7f29c7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c806b120 0x7f29c8198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36108/0 (socket says 172.21.15.43:36108) 2024-04-21T08:49:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.334+0000 7f29c7fff640 1 -- 172.21.15.43:0/2192680451 learned_addr learned my addr 172.21.15.43:0/2192680451 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.334+0000 7f29cd712640 1 --2- 172.21.15.43:0/2192680451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29c8107fe0 0x7f29c819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.335+0000 7f29ccf11640 1 --2- 172.21.15.43:0/2192680451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c806a780 0x7f29c8198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.335+0000 7f29c7fff640 1 -- 172.21.15.43:0/2192680451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c806a780 msgr2=0x7f29c8198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.335+0000 7f29c7fff640 1 --2- 172.21.15.43:0/2192680451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c806a780 0x7f29c8198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.335+0000 7f29c7fff640 1 -- 172.21.15.43:0/2192680451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29c8107fe0 msgr2=0x7f29c819cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.335+0000 7f29c7fff640 1 --2- 172.21.15.43:0/2192680451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29c8107fe0 0x7f29c819cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.335+0000 7f29c7fff640 1 -- 172.21.15.43:0/2192680451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c819d630 con 0x7f29c806b120 2024-04-21T08:49:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.335+0000 7f29ccf11640 1 --2- 172.21.15.43:0/2192680451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c806a780 0x7f29c8198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.335+0000 7f29c7fff640 1 --2- 172.21.15.43:0/2192680451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c806b120 0x7f29c8198b80 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f29b800c890 tx=0x7f29b800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.335+0000 7f29c5ffb640 1 -- 172.21.15.43:0/2192680451 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b8004220 con 0x7f29c806b120 2024-04-21T08:49:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.335+0000 7f29c5ffb640 1 -- 172.21.15.43:0/2192680451 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29b80043c0 con 0x7f29c806b120 2024-04-21T08:49:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.335+0000 7f29c5ffb640 1 -- 172.21.15.43:0/2192680451 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b8004a00 con 0x7f29c806b120 2024-04-21T08:49:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.335+0000 7f29cf175640 1 -- 172.21.15.43:0/2192680451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29c819d920 con 0x7f29c806b120 2024-04-21T08:49:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.335+0000 7f29cf175640 1 -- 172.21.15.43:0/2192680451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29c81a51b0 con 0x7f29c806b120 2024-04-21T08:49:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.338+0000 7f29c5ffb640 1 -- 172.21.15.43:0/2192680451 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f29b80079b0 con 0x7f29c806b120 2024-04-21T08:49:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.338+0000 7f29d1d65740 1 Processor -- start 2024-04-21T08:49:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.338+0000 7f29d1d65740 1 -- start start 2024-04-21T08:49:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.338+0000 7f29c5ffb640 1 --2- 172.21.15.43:0/2192680451 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f29b40a3dc0 0x7f29b40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.339+0000 7f29d1d65740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e18d0 0x563ca93e0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.339+0000 7f29d1d65740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563ca93e0b60 0x563ca93e0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.339+0000 7f29d1d65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ca93ea0e0 0x563ca93ec4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.339+0000 7f29d1d65740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563ca93458d0 con 0x563ca93e18d0 2024-04-21T08:49:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.339+0000 7f29d1d65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563ca92e2e80 con 0x563ca93ea0e0 2024-04-21T08:49:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.339+0000 7f29d1d65740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563ca9341890 con 0x563ca93e0b60 2024-04-21T08:49:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.339+0000 7f29cd712640 1 --2- 172.21.15.43:0/2192680451 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f29b40a3dc0 0x7f29b40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.339+0000 7f29ccf11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e18d0 0x563ca93e0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.339+0000 7f29c5ffb640 1 -- 172.21.15.43:0/2192680451 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f29b80d0a20 con 0x7f29c806b120 2024-04-21T08:49:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.339+0000 7f29ccf11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e18d0 0x563ca93e0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36116/0 (socket says 172.21.15.43:36116) 2024-04-21T08:49:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.339+0000 7f29ccf11640 1 -- 172.21.15.43:0/580769506 learned_addr learned my addr 172.21.15.43:0/580769506 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.339+0000 7f29c7fff640 1 --2- 172.21.15.43:0/580769506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563ca93e0b60 0x563ca93e0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.339+0000 7f29cd712640 1 --2- 172.21.15.43:0/580769506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ca93ea0e0 0x563ca93ec4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.339+0000 7f29ccf11640 1 -- 172.21.15.43:0/580769506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563ca93e0b60 msgr2=0x563ca93e0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.339+0000 7f29ccf11640 1 --2- 172.21.15.43:0/580769506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563ca93e0b60 0x563ca93e0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.339+0000 7f29ccf11640 1 -- 172.21.15.43:0/580769506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ca93ea0e0 msgr2=0x563ca93ec4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.339+0000 7f29ccf11640 1 --2- 172.21.15.43:0/580769506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ca93ea0e0 0x563ca93ec4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.339+0000 7f29ccf11640 1 -- 172.21.15.43:0/580769506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ca9304320 con 0x563ca93e18d0 2024-04-21T08:49:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.339+0000 7f29ccf11640 1 --2- 172.21.15.43:0/580769506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e18d0 0x563ca93e0620 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009bd0 tx=0x7f29b000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc00039b99ed378 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.339+0000 7f29cd712640 1 --2- 172.21.15.43:0/2192680451 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f29b40a3dc0 0x7f29b40a6280 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f29c0006010 tx=0x7f29c0032bc0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.339+0000 7f29beffd640 1 -- 172.21.15.43:0/580769506 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b0016070 con 0x563ca93e18d0 2024-04-21T08:49:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.340+0000 7f29beffd640 1 -- 172.21.15.43:0/580769506 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29b00054d0 con 0x563ca93e18d0 2024-04-21T08:49:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.340+0000 7f29beffd640 1 -- 172.21.15.43:0/580769506 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b0011470 con 0x563ca93e18d0 2024-04-21T08:49:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.340+0000 7f29d1d65740 1 -- 172.21.15.43:0/580769506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e18d0 msgr2=0x563ca93e0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.340+0000 7f29d1d65740 1 --2- 172.21.15.43:0/580769506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e18d0 0x563ca93e0620 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009bd0 tx=0x7f29b000e6c0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.340+0000 7f29d1d65740 1 -- 172.21.15.43:0/580769506 shutdown_connections 2024-04-21T08:49:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.340+0000 7f29d1d65740 1 --2- 172.21.15.43:0/580769506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ca93ea0e0 0x563ca93ec4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.340+0000 7f29d1d65740 1 --2- 172.21.15.43:0/580769506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563ca93e0b60 0x563ca93e0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.340+0000 7f29d1d65740 1 --2- 172.21.15.43:0/580769506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e18d0 0x563ca93e0620 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.340+0000 7f29d1d65740 1 -- 172.21.15.43:0/580769506 >> 172.21.15.43:0/580769506 conn(0x563ca93a7c90 msgr2=0x563ca93db850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.340+0000 7f29d1d65740 1 -- 172.21.15.43:0/580769506 shutdown_connections 2024-04-21T08:49:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.340+0000 7f29d1d65740 1 -- 172.21.15.43:0/580769506 wait complete. 2024-04-21T08:49:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.341+0000 7f29d1d65740 1 -- start start 2024-04-21T08:49:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.341+0000 7f29d1d65740 1 Processor -- start 2024-04-21T08:49:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.341+0000 7f29d1d65740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e0b60 0x563ca93e9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.341+0000 7f29d1d65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ca93e18d0 0x563ca93e6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.341+0000 7f29d1d65740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563ca93ea0e0 0x563ca93e7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.341+0000 7f29d1d65740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563ca93f1220 con 0x563ca93e0b60 2024-04-21T08:49:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.341+0000 7f29d1d65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563ca92cdeb0 con 0x563ca93e18d0 2024-04-21T08:49:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.341+0000 7f29d1d65740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563ca93f13a0 con 0x563ca93ea0e0 2024-04-21T08:49:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.341+0000 7f29d1d65740 10 client.0 _ll_register_callbacks cb 0x563ca92fab40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.341+0000 7f29ccf11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e0b60 0x563ca93e9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.341+0000 7f29ccf11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e0b60 0x563ca93e9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36122/0 (socket says 172.21.15.43:36122) 2024-04-21T08:49:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.341+0000 7f29ccf11640 1 -- 172.21.15.43:0/2461289128 learned_addr learned my addr 172.21.15.43:0/2461289128 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.342+0000 7f29c7fff640 1 --2- 172.21.15.43:0/2461289128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ca93e18d0 0x563ca93e6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.342+0000 7f29cd712640 1 --2- 172.21.15.43:0/2461289128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563ca93ea0e0 0x563ca93e7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.342+0000 7f29ccf11640 1 -- 172.21.15.43:0/2461289128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563ca93ea0e0 msgr2=0x563ca93e7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.342+0000 7f29ccf11640 1 --2- 172.21.15.43:0/2461289128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563ca93ea0e0 0x563ca93e7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.342+0000 7f29ccf11640 1 -- 172.21.15.43:0/2461289128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ca93e18d0 msgr2=0x563ca93e6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.342+0000 7f29ccf11640 1 --2- 172.21.15.43:0/2461289128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ca93e18d0 0x563ca93e6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.342+0000 7f29ccf11640 1 -- 172.21.15.43:0/2461289128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ca9058210 con 0x563ca93e0b60 2024-04-21T08:49:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.342+0000 7f29cd712640 1 --2- 172.21.15.43:0/2461289128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563ca93ea0e0 0x563ca93e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.342+0000 7f29c7fff640 1 --2- 172.21.15.43:0/2461289128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ca93e18d0 0x563ca93e6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.342+0000 7f29ccf11640 1 --2- 172.21.15.43:0/2461289128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e0b60 0x563ca93e9a50 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f29b0000c00 tx=0x7f29b00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83f242d91c9dd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.342+0000 7f29a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.342+0000 7f29a7fff640 1 -- 172.21.15.43:0/2461289128 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b0028060 con 0x563ca93e0b60 2024-04-21T08:49:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.342+0000 7f29a7fff640 1 -- 172.21.15.43:0/2461289128 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f29b0004ee0 con 0x563ca93e0b60 2024-04-21T08:49:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.342+0000 7f29d1d65740 1 -- 172.21.15.43:0/2461289128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29b00054d0 con 0x563ca93e0b60 2024-04-21T08:49:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.342+0000 7f29a7fff640 1 -- 172.21.15.43:0/2461289128 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f29b0016040 con 0x563ca93e0b60 2024-04-21T08:49:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.342+0000 7f29d1d65740 10 client.20502 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.342+0000 7f29d1d65740 1 -- 172.21.15.43:0/2461289128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563ca9124620 con 0x563ca93e0b60 2024-04-21T08:49:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.342+0000 7f29a7fff640 1 -- 172.21.15.43:0/2461289128 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29b00054d0 con 0x563ca93e0b60 2024-04-21T08:49:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.343+0000 7f29a7fff640 1 -- 172.21.15.43:0/2461289128 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f29b000f040 con 0x563ca93e0b60 2024-04-21T08:49:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.343+0000 7f29d1d65740 10 client.20502 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.343+0000 7f29d1d65740 10 client.20502 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.343+0000 7f29d1d65740 20 client.20502 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.343+0000 7f29d1d65740 1 --2- 172.21.15.43:0/2461289128 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563ca93fe460 0x563ca9400850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.343+0000 7f29d1d65740 4 client.20502 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.343+0000 7f29d1d65740 1 -- 172.21.15.43:0/2461289128 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ca9359880 con 0x563ca93fe460 2024-04-21T08:49:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.343+0000 7f29c7fff640 1 --2- 172.21.15.43:0/2461289128 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563ca93fe460 0x563ca9400850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.343+0000 7f29a7fff640 10 client.20502 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.343+0000 7f29c7fff640 1 --2- 172.21.15.43:0/2461289128 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563ca93fe460 0x563ca9400850 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d24e16d4c0491b3 server_cookie=dab596a0a4955e5f in_seq=0 out_seq=0 2024-04-21T08:49:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.344+0000 7f29a7fff640 1 -- 172.21.15.43:0/2461289128 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563ca9359880 con 0x563ca93fe460 2024-04-21T08:49:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.344+0000 7f29a7fff640 10 client.20502 handle_command_reply: tid=0 2024-04-21T08:49:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.344+0000 7f29d1d65740 1 client.20502 shutdown 2024-04-21T08:49:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.344+0000 7f29d1d65740 1 -- 172.21.15.43:0/2461289128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e0b60 msgr2=0x563ca93e9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.344+0000 7f29d1d65740 1 --2- 172.21.15.43:0/2461289128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e0b60 0x563ca93e9a50 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f29b0000c00 tx=0x7f29b00048f0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.344+0000 7f29d1d65740 1 -- 172.21.15.43:0/2461289128 shutdown_connections 2024-04-21T08:49:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.344+0000 7f29d1d65740 1 --2- 172.21.15.43:0/2461289128 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563ca93fe460 0x563ca9400850 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.345+0000 7f29d1d65740 1 --2- 172.21.15.43:0/2461289128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563ca93ea0e0 0x563ca93e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.345+0000 7f29d1d65740 1 --2- 172.21.15.43:0/2461289128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ca93e18d0 0x563ca93e6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.345+0000 7f29a7fff640 0 client.20502 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.345+0000 7f29d1d65740 1 --2- 172.21.15.43:0/2461289128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e0b60 0x563ca93e9a50 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.345+0000 7f29d1d65740 1 -- 172.21.15.43:0/2461289128 >> 172.21.15.43:0/2461289128 conn(0x563ca93a7c90 msgr2=0x563ca93dd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.345+0000 7f29d1d65740 1 -- 172.21.15.43:0/2461289128 shutdown_connections 2024-04-21T08:49:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.345+0000 7f29d1d65740 1 -- 172.21.15.43:0/2461289128 wait complete. 2024-04-21T08:49:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.345+0000 7f29d1d65740 20 client.20502 trim_cache size 0 max 16384 2024-04-21T08:49:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.360+0000 7f29d1d65740 1 Processor -- start 2024-04-21T08:49:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.360+0000 7f29d1d65740 1 -- start start 2024-04-21T08:49:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.360+0000 7f29d1d65740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563ca93e0b60 0x563ca93e04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.360+0000 7f29d1d65740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e18d0 0x563ca93da750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.360+0000 7f29d1d65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ca93ea0e0 0x563ca93e9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.360+0000 7f29d1d65740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f29c81ad390 con 0x563ca93e18d0 2024-04-21T08:49:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.360+0000 7f29d1d65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f29c81aeed0 con 0x563ca93ea0e0 2024-04-21T08:49:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.360+0000 7f29d1d65740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29c81ada60 con 0x563ca93e0b60 2024-04-21T08:49:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.360+0000 7f29c7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e18d0 0x563ca93da750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.360+0000 7f29c7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e18d0 0x563ca93da750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36124/0 (socket says 172.21.15.43:36124) 2024-04-21T08:49:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.360+0000 7f29c7fff640 1 -- 172.21.15.43:0/619194970 learned_addr learned my addr 172.21.15.43:0/619194970 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.360+0000 7f29ccf11640 1 --2- 172.21.15.43:0/619194970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563ca93e0b60 0x563ca93e04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.360+0000 7f29cd712640 1 --2- 172.21.15.43:0/619194970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ca93ea0e0 0x563ca93e9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.360+0000 7f29c7fff640 1 -- 172.21.15.43:0/619194970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563ca93e0b60 msgr2=0x563ca93e04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.360+0000 7f29c7fff640 1 --2- 172.21.15.43:0/619194970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563ca93e0b60 0x563ca93e04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.361+0000 7f29c7fff640 1 -- 172.21.15.43:0/619194970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ca93ea0e0 msgr2=0x563ca93e9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.361+0000 7f29c7fff640 1 --2- 172.21.15.43:0/619194970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ca93ea0e0 0x563ca93e9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.361+0000 7f29c7fff640 1 -- 172.21.15.43:0/619194970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ca93dfe30 con 0x563ca93e18d0 2024-04-21T08:49:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.361+0000 7f29cd712640 1 --2- 172.21.15.43:0/619194970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ca93ea0e0 0x563ca93e9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.361+0000 7f29c7fff640 1 --2- 172.21.15.43:0/619194970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e18d0 0x563ca93da750 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f29b8005fd0 tx=0x7f29b8005350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bd6ac88266c38a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.361+0000 7f29be7fc640 1 -- 172.21.15.43:0/619194970 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b801d040 con 0x563ca93e18d0 2024-04-21T08:49:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.361+0000 7f29be7fc640 1 -- 172.21.15.43:0/619194970 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29b80a0950 con 0x563ca93e18d0 2024-04-21T08:49:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.361+0000 7f29be7fc640 1 -- 172.21.15.43:0/619194970 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b80cdfd0 con 0x563ca93e18d0 2024-04-21T08:49:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.361+0000 7f29d1d65740 1 -- 172.21.15.43:0/619194970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e18d0 msgr2=0x563ca93da750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.361+0000 7f29d1d65740 1 --2- 172.21.15.43:0/619194970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e18d0 0x563ca93da750 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f29b8005fd0 tx=0x7f29b8005350 comp rx=0 tx=0).stop 2024-04-21T08:49:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.362+0000 7f29d1d65740 1 -- 172.21.15.43:0/619194970 shutdown_connections 2024-04-21T08:49:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.362+0000 7f29d1d65740 1 --2- 172.21.15.43:0/619194970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ca93ea0e0 0x563ca93e9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.362+0000 7f29d1d65740 1 --2- 172.21.15.43:0/619194970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e18d0 0x563ca93da750 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.362+0000 7f29d1d65740 1 --2- 172.21.15.43:0/619194970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563ca93e0b60 0x563ca93e04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.362+0000 7f29d1d65740 1 -- 172.21.15.43:0/619194970 >> 172.21.15.43:0/619194970 conn(0x563ca93a7c90 msgr2=0x563ca93da040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.362+0000 7f29d1d65740 1 -- 172.21.15.43:0/619194970 shutdown_connections 2024-04-21T08:49:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.362+0000 7f29d1d65740 1 -- 172.21.15.43:0/619194970 wait complete. 2024-04-21T08:49:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.362+0000 7f29d1d65740 1 -- start start 2024-04-21T08:49:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.362+0000 7f29d1d65740 1 Processor -- start 2024-04-21T08:49:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.362+0000 7f29d1d65740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563ca93e0b60 0x563ca93e3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.362+0000 7f29d1d65740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e18d0 0x563ca93e4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.362+0000 7f29d1d65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ca93ea0e0 0x563ca93e4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.362+0000 7f29d1d65740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f29c81ae800 con 0x563ca93e18d0 2024-04-21T08:49:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.362+0000 7f29d1d65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f29c81ae130 con 0x563ca93ea0e0 2024-04-21T08:49:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.362+0000 7f29d1d65740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29c81ab390 con 0x563ca93e0b60 2024-04-21T08:49:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.362+0000 7f29d1d65740 10 client.0 _ll_register_callbacks cb 0x7f29b0009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.362+0000 7f29c7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e18d0 0x563ca93e4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.363+0000 7f29c7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e18d0 0x563ca93e4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36126/0 (socket says 172.21.15.43:36126) 2024-04-21T08:49:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.363+0000 7f29c7fff640 1 -- 172.21.15.43:0/555636973 learned_addr learned my addr 172.21.15.43:0/555636973 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.363+0000 7f29ccf11640 1 --2- 172.21.15.43:0/555636973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563ca93e0b60 0x563ca93e3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.363+0000 7f29cd712640 1 --2- 172.21.15.43:0/555636973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ca93ea0e0 0x563ca93e4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.363+0000 7f29c7fff640 1 -- 172.21.15.43:0/555636973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563ca93e0b60 msgr2=0x563ca93e3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.363+0000 7f29c7fff640 1 --2- 172.21.15.43:0/555636973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563ca93e0b60 0x563ca93e3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.363+0000 7f29c7fff640 1 -- 172.21.15.43:0/555636973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ca93ea0e0 msgr2=0x563ca93e4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.363+0000 7f29c7fff640 1 --2- 172.21.15.43:0/555636973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ca93ea0e0 0x563ca93e4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.363+0000 7f29c7fff640 1 -- 172.21.15.43:0/555636973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ca9463790 con 0x563ca93e18d0 2024-04-21T08:49:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.363+0000 7f29ccf11640 1 --2- 172.21.15.43:0/555636973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563ca93e0b60 0x563ca93e3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.363+0000 7f29c7fff640 1 --2- 172.21.15.43:0/555636973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e18d0 0x563ca93e4120 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f29b809c2a0 tx=0x7f29b809c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7fef326865c2e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.363+0000 7f29a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.363+0000 7f29a7fff640 1 -- 172.21.15.43:0/555636973 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b8084020 con 0x563ca93e18d0 2024-04-21T08:49:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.363+0000 7f29a7fff640 1 -- 172.21.15.43:0/555636973 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f29b80cdfd0 con 0x563ca93e18d0 2024-04-21T08:49:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.363+0000 7f29d1d65740 1 -- 172.21.15.43:0/555636973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29b80a0950 con 0x563ca93e18d0 2024-04-21T08:49:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.363+0000 7f29a7fff640 1 -- 172.21.15.43:0/555636973 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f29b801d040 con 0x563ca93e18d0 2024-04-21T08:49:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.363+0000 7f29d1d65740 10 client.20508 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.363+0000 7f29d1d65740 1 -- 172.21.15.43:0/555636973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563ca9464a80 con 0x563ca93e18d0 2024-04-21T08:49:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.363+0000 7f29a7fff640 1 -- 172.21.15.43:0/555636973 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29b80a0950 con 0x563ca93e18d0 2024-04-21T08:49:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.364+0000 7f29a7fff640 1 -- 172.21.15.43:0/555636973 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f29b80803f0 con 0x563ca93e18d0 2024-04-21T08:49:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.364+0000 7f29d1d65740 10 client.20508 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.364+0000 7f29d1d65740 10 client.20508 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.364+0000 7f29d1d65740 20 client.20508 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.364+0000 7f29d1d65740 1 --2- 172.21.15.43:0/555636973 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563ca94651e0 0x563ca9467600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.364+0000 7f29d1d65740 4 client.20508 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:49:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.364+0000 7f29d1d65740 1 -- 172.21.15.43:0/555636973 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ca92e53d0 con 0x563ca94651e0 2024-04-21T08:49:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.364+0000 7f29ccf11640 1 --2- 172.21.15.43:0/555636973 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563ca94651e0 0x563ca9467600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.364+0000 7f29a7fff640 10 client.20508 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.364+0000 7f29ccf11640 1 --2- 172.21.15.43:0/555636973 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563ca94651e0 0x563ca9467600 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4fa58f2cbe58a9b server_cookie=c13295603b12a469 in_seq=0 out_seq=0 2024-04-21T08:49:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.380+0000 7f29a7fff640 1 -- 172.21.15.43:0/555636973 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563ca92e53d0 con 0x563ca94651e0 2024-04-21T08:49:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.380+0000 7f29a7fff640 10 client.20508 handle_command_reply: tid=0 2024-04-21T08:49:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.380+0000 7f29d1d65740 1 client.20508 shutdown 2024-04-21T08:49:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.380+0000 7f29d1d65740 1 -- 172.21.15.43:0/555636973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e18d0 msgr2=0x563ca93e4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.380+0000 7f29d1d65740 1 --2- 172.21.15.43:0/555636973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e18d0 0x563ca93e4120 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f29b809c2a0 tx=0x7f29b809c720 comp rx=0 tx=0).stop 2024-04-21T08:49:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.380+0000 7f29d1d65740 1 -- 172.21.15.43:0/555636973 shutdown_connections 2024-04-21T08:49:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.380+0000 7f29d1d65740 1 --2- 172.21.15.43:0/555636973 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x563ca94651e0 0x563ca9467600 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.380+0000 7f29d1d65740 1 --2- 172.21.15.43:0/555636973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ca93ea0e0 0x563ca93e4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.380+0000 7f29d1d65740 1 --2- 172.21.15.43:0/555636973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ca93e18d0 0x563ca93e4120 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.380+0000 7f29d1d65740 1 --2- 172.21.15.43:0/555636973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563ca93e0b60 0x563ca93e3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.380+0000 7f29a7fff640 0 client.20508 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.380+0000 7f29d1d65740 1 -- 172.21.15.43:0/555636973 >> 172.21.15.43:0/555636973 conn(0x563ca93a7c90 msgr2=0x563ca93da8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.380+0000 7f29d1d65740 1 -- 172.21.15.43:0/555636973 shutdown_connections 2024-04-21T08:49:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.380+0000 7f29d1d65740 1 -- 172.21.15.43:0/555636973 wait complete. 2024-04-21T08:49:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.380+0000 7f29d1d65740 20 client.20508 trim_cache size 0 max 16384 2024-04-21T08:49:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:49:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:49:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:49:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:49:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.381+0000 7f29cf175640 1 -- 172.21.15.43:0/2192680451 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f29b40a3dc0 msgr2=0x7f29b40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.381+0000 7f29cf175640 1 --2- 172.21.15.43:0/2192680451 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f29b40a3dc0 0x7f29b40a6280 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f29c0006010 tx=0x7f29c0032bc0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.381+0000 7f29cf175640 1 -- 172.21.15.43:0/2192680451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c806b120 msgr2=0x7f29c8198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.381+0000 7f29cf175640 1 --2- 172.21.15.43:0/2192680451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c806b120 0x7f29c8198b80 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f29b800c890 tx=0x7f29b800cd50 comp rx=0 tx=0).stop 2024-04-21T08:49:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.381+0000 7f29cf175640 1 -- 172.21.15.43:0/2192680451 shutdown_connections 2024-04-21T08:49:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.381+0000 7f29cf175640 1 --2- 172.21.15.43:0/2192680451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29c8107fe0 0x7f29c819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.381+0000 7f29cf175640 1 --2- 172.21.15.43:0/2192680451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c806b120 0x7f29c8198b80 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.381+0000 7f29cf175640 1 --2- 172.21.15.43:0/2192680451 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f29b40a3dc0 0x7f29b40a6280 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.381+0000 7f29cf175640 1 --2- 172.21.15.43:0/2192680451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c806a780 0x7f29c8198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.381+0000 7f29cf175640 1 -- 172.21.15.43:0/2192680451 >> 172.21.15.43:0/2192680451 conn(0x7f29c8076c50 msgr2=0x7f29c80ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.381+0000 7f29cf175640 1 -- 172.21.15.43:0/2192680451 shutdown_connections 2024-04-21T08:49:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:21.381+0000 7f29cf175640 1 -- 172.21.15.43:0/2192680451 wait complete. 2024-04-21T08:49:22.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:22 smithi175 ceph-mon[31920]: pgmap v325: 129 pgs: 129 active+clean; 5.7 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:49:22.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:22 smithi112 ceph-mon[31085]: pgmap v325: 129 pgs: 129 active+clean; 5.7 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:49:22.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:22 smithi043 ceph-mon[27978]: pgmap v325: 129 pgs: 129 active+clean; 5.7 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:49:24.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:24 smithi175 ceph-mon[31920]: pgmap v326: 129 pgs: 129 active+clean; 5.7 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:49:24.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:24 smithi112 ceph-mon[31085]: pgmap v326: 129 pgs: 129 active+clean; 5.7 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:49:24.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:24 smithi043 ceph-mon[27978]: pgmap v326: 129 pgs: 129 active+clean; 5.7 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:49:26.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:26 smithi175 ceph-mon[31920]: pgmap v327: 129 pgs: 129 active+clean; 5.7 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:49:26.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:26 smithi112 ceph-mon[31085]: pgmap v327: 129 pgs: 129 active+clean; 5.7 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:49:26.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:26 smithi043 ceph-mon[27978]: pgmap v327: 129 pgs: 129 active+clean; 5.7 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:49:28.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:28 smithi175 ceph-mon[31920]: pgmap v328: 129 pgs: 129 active+clean; 2.0 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-21T08:49:28.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:28 smithi112 ceph-mon[31085]: pgmap v328: 129 pgs: 129 active+clean; 2.0 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-21T08:49:28.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:28 smithi043 ceph-mon[27978]: pgmap v328: 129 pgs: 129 active+clean; 2.0 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-21T08:49:30.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:30 smithi175 ceph-mon[31920]: pgmap v329: 129 pgs: 129 active+clean; 2.0 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:49:30.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:30 smithi112 ceph-mon[31085]: pgmap v329: 129 pgs: 129 active+clean; 2.0 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:49:30.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:30 smithi043 ceph-mon[27978]: pgmap v329: 129 pgs: 129 active+clean; 2.0 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:49:31.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:49:31.724 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:49:31.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.844+0000 7fe1fdb96640 1 -- 172.21.15.43:0/375206638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1f8102190 msgr2=0x7fe1f8102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:31.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.844+0000 7fe1fdb96640 1 --2- 172.21.15.43:0/375206638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1f8102190 0x7fe1f8102570 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8009ca0 tx=0x7fe1e802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.844+0000 7fe1fdb96640 1 -- 172.21.15.43:0/375206638 shutdown_connections 2024-04-21T08:49:31.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.844+0000 7fe1fdb96640 1 --2- 172.21.15.43:0/375206638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1f8107260 0x7fe1f810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.844+0000 7fe1fdb96640 1 --2- 172.21.15.43:0/375206638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1f8102b40 0x7fe1f8106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.844+0000 7fe1fdb96640 1 --2- 172.21.15.43:0/375206638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1f8102190 0x7fe1f8102570 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.844+0000 7fe1fdb96640 1 -- 172.21.15.43:0/375206638 >> 172.21.15.43:0/375206638 conn(0x7fe1f80fda30 msgr2=0x7fe1f80ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:31.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.844+0000 7fe1fdb96640 1 -- 172.21.15.43:0/375206638 shutdown_connections 2024-04-21T08:49:31.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.844+0000 7fe1fdb96640 1 -- 172.21.15.43:0/375206638 wait complete. 2024-04-21T08:49:31.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.844+0000 7fe1fdb96640 1 Processor -- start 2024-04-21T08:49:31.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.845+0000 7fe1fdb96640 1 -- start start 2024-04-21T08:49:31.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.845+0000 7fe1fdb96640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1f8102190 0x7fe1f8074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:31.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.845+0000 7fe1fdb96640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1f8107260 0x7fe1f8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:31.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.845+0000 7fe1fdb96640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1f8074cd0 0x7fe1f81a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:31.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.845+0000 7fe1fdb96640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81121c0 con 0x7fe1f8102190 2024-04-21T08:49:31.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.845+0000 7fe1fdb96640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f8112040 con 0x7fe1f8074cd0 2024-04-21T08:49:31.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.845+0000 7fe1fdb96640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f8112340 con 0x7fe1f8107260 2024-04-21T08:49:31.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.845+0000 7fe1f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1f8102190 0x7fe1f8074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:31.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.845+0000 7fe1f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1f8102190 0x7fe1f8074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47420/0 (socket says 172.21.15.43:47420) 2024-04-21T08:49:31.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.845+0000 7fe1f77fe640 1 -- 172.21.15.43:0/2280784978 learned_addr learned my addr 172.21.15.43:0/2280784978 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:31.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.845+0000 7fe1f6ffd640 1 --2- 172.21.15.43:0/2280784978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1f8107260 0x7fe1f8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:31.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.845+0000 7fe1f7fff640 1 --2- 172.21.15.43:0/2280784978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1f8074cd0 0x7fe1f81a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:31.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.845+0000 7fe1f77fe640 1 -- 172.21.15.43:0/2280784978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1f8107260 msgr2=0x7fe1f8074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:31.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.845+0000 7fe1f77fe640 1 --2- 172.21.15.43:0/2280784978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1f8107260 0x7fe1f8074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.845+0000 7fe1f77fe640 1 -- 172.21.15.43:0/2280784978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1f8074cd0 msgr2=0x7fe1f81a6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:31.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.845+0000 7fe1f77fe640 1 --2- 172.21.15.43:0/2280784978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1f8074cd0 0x7fe1f81a6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.845+0000 7fe1f77fe640 1 -- 172.21.15.43:0/2280784978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f81a7050 con 0x7fe1f8102190 2024-04-21T08:49:31.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.845+0000 7fe1f7fff640 1 --2- 172.21.15.43:0/2280784978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1f8074cd0 0x7fe1f81a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:31.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.846+0000 7fe1f77fe640 1 --2- 172.21.15.43:0/2280784978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1f8102190 0x7fe1f8074250 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8005e30 tx=0x7fe1e8034780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:31.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.846+0000 7fe1f4ff9640 1 -- 172.21.15.43:0/2280784978 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8039040 con 0x7fe1f8102190 2024-04-21T08:49:31.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.846+0000 7fe1fdb96640 1 -- 172.21.15.43:0/2280784978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f81a72e0 con 0x7fe1f8102190 2024-04-21T08:49:31.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.846+0000 7fe1f4ff9640 1 -- 172.21.15.43:0/2280784978 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e8044070 con 0x7fe1f8102190 2024-04-21T08:49:31.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.846+0000 7fe1f4ff9640 1 -- 172.21.15.43:0/2280784978 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8003e60 con 0x7fe1f8102190 2024-04-21T08:49:31.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.846+0000 7fe1fdb96640 1 -- 172.21.15.43:0/2280784978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f81a77f0 con 0x7fe1f8102190 2024-04-21T08:49:31.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.848+0000 7fe1f4ff9640 1 -- 172.21.15.43:0/2280784978 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe1e80324c0 con 0x7fe1f8102190 2024-04-21T08:49:31.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.848+0000 7fe200789740 1 Processor -- start 2024-04-21T08:49:31.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.848+0000 7fe200789740 1 -- start start 2024-04-21T08:49:31.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.849+0000 7fe200789740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f340dc580 0x556f340dc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:31.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.849+0000 7fe200789740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f340dcea0 0x556f340e5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:31.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.849+0000 7fe200789740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340e5a80 0x556f340e7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:31.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.849+0000 7fe200789740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556f340411e0 con 0x556f340e5a80 2024-04-21T08:49:31.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.849+0000 7fe200789740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556f33fde390 con 0x556f340dc580 2024-04-21T08:49:31.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.849+0000 7fe200789740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556f3403d1a0 con 0x556f340dcea0 2024-04-21T08:49:31.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.849+0000 7fe1f4ff9640 1 --2- 172.21.15.43:0/2280784978 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe1e40a3d50 0x7fe1e40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:31.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.849+0000 7fe1f6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340e5a80 0x556f340e7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:31.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.849+0000 7fe1f7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f340dc580 0x556f340dc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:31.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.849+0000 7fe1f6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340e5a80 0x556f340e7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47434/0 (socket says 172.21.15.43:47434) 2024-04-21T08:49:31.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.849+0000 7fe1f6ffd640 1 -- 172.21.15.43:0/1019247287 learned_addr learned my addr 172.21.15.43:0/1019247287 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:31.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.849+0000 7fe1f7fff640 1 --2- 172.21.15.43:0/2280784978 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe1e40a3d50 0x7fe1e40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:31.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.849+0000 7fe1f77fe640 1 --2- 172.21.15.43:0/1019247287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f340dcea0 0x556f340e5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:31.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.849+0000 7fe1f4ff9640 1 -- 172.21.15.43:0/2280784978 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fe1e80f2e10 con 0x7fe1f8102190 2024-04-21T08:49:31.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.849+0000 7fe1f6ffd640 1 -- 172.21.15.43:0/1019247287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f340dcea0 msgr2=0x556f340e5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:31.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.849+0000 7fe1f6ffd640 1 --2- 172.21.15.43:0/1019247287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f340dcea0 0x556f340e5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.849+0000 7fe1f6ffd640 1 -- 172.21.15.43:0/1019247287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f340dc580 msgr2=0x556f340dc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:31.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.849+0000 7fe1f6ffd640 1 --2- 172.21.15.43:0/1019247287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f340dc580 0x556f340dc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.849+0000 7fe1f6ffd640 1 -- 172.21.15.43:0/1019247287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f33e1d7d0 con 0x556f340e5a80 2024-04-21T08:49:31.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.849+0000 7fe1f7fff640 1 --2- 172.21.15.43:0/1019247287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f340dc580 0x556f340dc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:31.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.849+0000 7fe1f6ffd640 1 --2- 172.21.15.43:0/1019247287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340e5a80 0x556f340e7e70 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000b6c0 tx=0x7fe1e000bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a803912464505b95 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:31.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.849+0000 7fe1f7fff640 1 --2- 172.21.15.43:0/2280784978 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe1e40a3d50 0x7fe1e40a6210 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0005fd0 tx=0x7fe1f0005850 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:31.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.850+0000 7fe1edffb640 1 -- 172.21.15.43:0/1019247287 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0003b20 con 0x556f340e5a80 2024-04-21T08:49:31.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.850+0000 7fe1edffb640 1 -- 172.21.15.43:0/1019247287 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e0015070 con 0x556f340e5a80 2024-04-21T08:49:31.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.850+0000 7fe1edffb640 1 -- 172.21.15.43:0/1019247287 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0004c00 con 0x556f340e5a80 2024-04-21T08:49:31.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.850+0000 7fe200789740 1 -- 172.21.15.43:0/1019247287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340e5a80 msgr2=0x556f340e7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:31.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.850+0000 7fe200789740 1 --2- 172.21.15.43:0/1019247287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340e5a80 0x556f340e7e70 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000b6c0 tx=0x7fe1e000bac0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.850+0000 7fe200789740 1 -- 172.21.15.43:0/1019247287 shutdown_connections 2024-04-21T08:49:31.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.850+0000 7fe200789740 1 --2- 172.21.15.43:0/1019247287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340e5a80 0x556f340e7e70 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.850+0000 7fe200789740 1 --2- 172.21.15.43:0/1019247287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f340dcea0 0x556f340e5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.850+0000 7fe200789740 1 --2- 172.21.15.43:0/1019247287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f340dc580 0x556f340dc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.850+0000 7fe200789740 1 -- 172.21.15.43:0/1019247287 >> 172.21.15.43:0/1019247287 conn(0x556f340d6cb0 msgr2=0x556f340d7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:31.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.850+0000 7fe200789740 1 -- 172.21.15.43:0/1019247287 shutdown_connections 2024-04-21T08:49:31.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.850+0000 7fe200789740 1 -- 172.21.15.43:0/1019247287 wait complete. 2024-04-21T08:49:31.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.851+0000 7fe200789740 1 -- start start 2024-04-21T08:49:31.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.851+0000 7fe200789740 1 Processor -- start 2024-04-21T08:49:31.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.851+0000 7fe200789740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f340dc580 0x556f340e2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:31.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.851+0000 7fe200789740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f340dcea0 0x556f340e3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:31.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.851+0000 7fe200789740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340e5a80 0x556f340e3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:31.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.851+0000 7fe200789740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556f340e8870 con 0x556f340e5a80 2024-04-21T08:49:31.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.851+0000 7fe200789740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556f33ffa2f0 con 0x556f340dcea0 2024-04-21T08:49:31.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.851+0000 7fe200789740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556f340e89f0 con 0x556f340dc580 2024-04-21T08:49:31.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.851+0000 7fe200789740 10 client.0 _ll_register_callbacks cb 0x556f34007580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:31.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.851+0000 7fe1f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340e5a80 0x556f340e3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:31.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.851+0000 7fe1f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340e5a80 0x556f340e3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47448/0 (socket says 172.21.15.43:47448) 2024-04-21T08:49:31.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.851+0000 7fe1f7fff640 1 -- 172.21.15.43:0/2483385672 learned_addr learned my addr 172.21.15.43:0/2483385672 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:31.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.851+0000 7fe1f77fe640 1 --2- 172.21.15.43:0/2483385672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f340dc580 0x556f340e2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:31.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.852+0000 7fe1f6ffd640 1 --2- 172.21.15.43:0/2483385672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f340dcea0 0x556f340e3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:31.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.852+0000 7fe1f7fff640 1 -- 172.21.15.43:0/2483385672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f340dc580 msgr2=0x556f340e2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:31.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.852+0000 7fe1f7fff640 1 --2- 172.21.15.43:0/2483385672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f340dc580 0x556f340e2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.852+0000 7fe1f7fff640 1 -- 172.21.15.43:0/2483385672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f340dcea0 msgr2=0x556f340e3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:31.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.852+0000 7fe1f7fff640 1 --2- 172.21.15.43:0/2483385672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f340dcea0 0x556f340e3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.852+0000 7fe1f7fff640 1 -- 172.21.15.43:0/2483385672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f33d533f0 con 0x556f340e5a80 2024-04-21T08:49:31.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.852+0000 7fe1f7fff640 1 --2- 172.21.15.43:0/2483385672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340e5a80 0x556f340e3550 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0011f50 tx=0x7fe1f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94a8440913afd94 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:31.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.852+0000 7fe1d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:31.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.852+0000 7fe1d6ffd640 1 -- 172.21.15.43:0/2483385672 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0026070 con 0x556f340e5a80 2024-04-21T08:49:31.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.852+0000 7fe1d6ffd640 1 -- 172.21.15.43:0/2483385672 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fe1f000fbc0 con 0x556f340e5a80 2024-04-21T08:49:31.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.852+0000 7fe200789740 1 -- 172.21.15.43:0/2483385672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1e0015070 con 0x556f340e5a80 2024-04-21T08:49:31.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.852+0000 7fe1d6ffd640 1 -- 172.21.15.43:0/2483385672 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe1f0031050 con 0x556f340e5a80 2024-04-21T08:49:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.852+0000 7fe200789740 10 client.20520 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.852+0000 7fe200789740 1 -- 172.21.15.43:0/2483385672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556f33d130f0 con 0x556f340e5a80 2024-04-21T08:49:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.852+0000 7fe1d6ffd640 1 -- 172.21.15.43:0/2483385672 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e0015070 con 0x556f340e5a80 2024-04-21T08:49:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.853+0000 7fe1d6ffd640 1 -- 172.21.15.43:0/2483385672 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe1f00048a0 con 0x556f340e5a80 2024-04-21T08:49:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.853+0000 7fe200789740 10 client.20520 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.853+0000 7fe200789740 10 client.20520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.853+0000 7fe200789740 20 client.20520 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.853+0000 7fe200789740 1 --2- 172.21.15.43:0/2483385672 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556f340f5aa0 0x556f340f7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.853+0000 7fe200789740 4 client.20520 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.853+0000 7fe200789740 1 -- 172.21.15.43:0/2483385672 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f3404ec00 con 0x556f340f5aa0 2024-04-21T08:49:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.853+0000 7fe1f77fe640 1 --2- 172.21.15.43:0/2483385672 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556f340f5aa0 0x556f340f7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.854+0000 7fe1f77fe640 1 --2- 172.21.15.43:0/2483385672 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556f340f5aa0 0x556f340f7f20 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bee8aaca3eafb50 server_cookie=bd6ea9d962ea34df in_seq=0 out_seq=0 2024-04-21T08:49:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.854+0000 7fe1d6ffd640 10 client.20520 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.854+0000 7fe1d6ffd640 1 -- 172.21.15.43:0/2483385672 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556f3404ec00 con 0x556f340f5aa0 2024-04-21T08:49:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.854+0000 7fe1d6ffd640 10 client.20520 handle_command_reply: tid=0 2024-04-21T08:49:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.854+0000 7fe200789740 1 client.20520 shutdown 2024-04-21T08:49:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.855+0000 7fe200789740 1 -- 172.21.15.43:0/2483385672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340e5a80 msgr2=0x556f340e3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.855+0000 7fe200789740 1 --2- 172.21.15.43:0/2483385672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340e5a80 0x556f340e3550 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0011f50 tx=0x7fe1f00046c0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.855+0000 7fe200789740 1 -- 172.21.15.43:0/2483385672 shutdown_connections 2024-04-21T08:49:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.855+0000 7fe200789740 1 --2- 172.21.15.43:0/2483385672 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556f340f5aa0 0x556f340f7f20 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.855+0000 7fe200789740 1 --2- 172.21.15.43:0/2483385672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340e5a80 0x556f340e3550 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.855+0000 7fe200789740 1 --2- 172.21.15.43:0/2483385672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f340dcea0 0x556f340e3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.855+0000 7fe1d6ffd640 0 client.20520 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.855+0000 7fe200789740 1 --2- 172.21.15.43:0/2483385672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f340dc580 0x556f340e2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.855+0000 7fe200789740 1 -- 172.21.15.43:0/2483385672 >> 172.21.15.43:0/2483385672 conn(0x556f340d6cb0 msgr2=0x556f340d8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.855+0000 7fe200789740 1 -- 172.21.15.43:0/2483385672 shutdown_connections 2024-04-21T08:49:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.855+0000 7fe200789740 1 -- 172.21.15.43:0/2483385672 wait complete. 2024-04-21T08:49:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.855+0000 7fe200789740 20 client.20520 trim_cache size 0 max 16384 2024-04-21T08:49:31.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.872+0000 7fe200789740 1 Processor -- start 2024-04-21T08:49:31.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.872+0000 7fe200789740 1 -- start start 2024-04-21T08:49:31.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.873+0000 7fe200789740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340dc580 0x556f340e5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:31.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.873+0000 7fe200789740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f340dcea0 0x556f340e1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:31.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.873+0000 7fe200789740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f340e5a80 0x556f340f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:31.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.873+0000 7fe200789740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81b1730 con 0x556f340dc580 2024-04-21T08:49:31.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.873+0000 7fe200789740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81b3270 con 0x556f340e5a80 2024-04-21T08:49:31.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.873+0000 7fe200789740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81b1e00 con 0x556f340dcea0 2024-04-21T08:49:31.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.873+0000 7fe1f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340dc580 0x556f340e5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:31.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.873+0000 7fe1f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340dc580 0x556f340e5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47456/0 (socket says 172.21.15.43:47456) 2024-04-21T08:49:31.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.873+0000 7fe1f77fe640 1 -- 172.21.15.43:0/676196353 learned_addr learned my addr 172.21.15.43:0/676196353 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:31.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.873+0000 7fe1f6ffd640 1 --2- 172.21.15.43:0/676196353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f340dcea0 0x556f340e1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:31.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.873+0000 7fe1f7fff640 1 --2- 172.21.15.43:0/676196353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f340e5a80 0x556f340f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:31.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.873+0000 7fe1f77fe640 1 -- 172.21.15.43:0/676196353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f340dcea0 msgr2=0x556f340e1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:31.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.873+0000 7fe1f77fe640 1 --2- 172.21.15.43:0/676196353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f340dcea0 0x556f340e1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.873+0000 7fe1f77fe640 1 -- 172.21.15.43:0/676196353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f340e5a80 msgr2=0x556f340f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:31.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.873+0000 7fe1f77fe640 1 --2- 172.21.15.43:0/676196353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f340e5a80 0x556f340f1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.873+0000 7fe1f77fe640 1 -- 172.21.15.43:0/676196353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f340db900 con 0x556f340dc580 2024-04-21T08:49:31.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.873+0000 7fe1f7fff640 1 --2- 172.21.15.43:0/676196353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f340e5a80 0x556f340f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:31.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.873+0000 7fe1f77fe640 1 --2- 172.21.15.43:0/676196353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340dc580 0x556f340e5200 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fe1e80bb470 tx=0x7fe1e8045e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a2c3c9013f3994 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:31.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.873+0000 7fe1ed7fa640 1 -- 172.21.15.43:0/676196353 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e80beaa0 con 0x556f340dc580 2024-04-21T08:49:31.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.873+0000 7fe1ed7fa640 1 -- 172.21.15.43:0/676196353 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e80c2e50 con 0x556f340dc580 2024-04-21T08:49:31.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.873+0000 7fe1ed7fa640 1 -- 172.21.15.43:0/676196353 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e80bce30 con 0x556f340dc580 2024-04-21T08:49:31.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.874+0000 7fe200789740 1 -- 172.21.15.43:0/676196353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340dc580 msgr2=0x556f340e5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:31.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.874+0000 7fe200789740 1 --2- 172.21.15.43:0/676196353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340dc580 0x556f340e5200 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fe1e80bb470 tx=0x7fe1e8045e80 comp rx=0 tx=0).stop 2024-04-21T08:49:31.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.874+0000 7fe200789740 1 -- 172.21.15.43:0/676196353 shutdown_connections 2024-04-21T08:49:31.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.874+0000 7fe200789740 1 --2- 172.21.15.43:0/676196353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f340e5a80 0x556f340f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.874+0000 7fe200789740 1 --2- 172.21.15.43:0/676196353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f340dcea0 0x556f340e1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.874+0000 7fe200789740 1 --2- 172.21.15.43:0/676196353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340dc580 0x556f340e5200 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.874+0000 7fe200789740 1 -- 172.21.15.43:0/676196353 >> 172.21.15.43:0/676196353 conn(0x556f340d6cb0 msgr2=0x556f340d5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:31.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.874+0000 7fe200789740 1 -- 172.21.15.43:0/676196353 shutdown_connections 2024-04-21T08:49:31.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.874+0000 7fe200789740 1 -- 172.21.15.43:0/676196353 wait complete. 2024-04-21T08:49:31.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.874+0000 7fe200789740 1 -- start start 2024-04-21T08:49:31.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.875+0000 7fe200789740 1 Processor -- start 2024-04-21T08:49:31.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.875+0000 7fe200789740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f340dc580 0x556f340edc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:31.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.875+0000 7fe200789740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f340dcea0 0x556f340ee180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:31.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.875+0000 7fe200789740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340e5a80 0x556f340eac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:31.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.875+0000 7fe200789740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81b2ba0 con 0x556f340e5a80 2024-04-21T08:49:31.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.875+0000 7fe200789740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81b24d0 con 0x556f340dcea0 2024-04-21T08:49:31.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.875+0000 7fe200789740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81af730 con 0x556f340dc580 2024-04-21T08:49:31.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.875+0000 7fe200789740 10 client.0 _ll_register_callbacks cb 0x7fe1f0014f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:31.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.875+0000 7fe1f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340e5a80 0x556f340eac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:31.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.875+0000 7fe1f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340e5a80 0x556f340eac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47462/0 (socket says 172.21.15.43:47462) 2024-04-21T08:49:31.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.875+0000 7fe1f7fff640 1 -- 172.21.15.43:0/3735312786 learned_addr learned my addr 172.21.15.43:0/3735312786 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:31.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.875+0000 7fe1f77fe640 1 --2- 172.21.15.43:0/3735312786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f340dc580 0x556f340edc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:31.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.875+0000 7fe1f6ffd640 1 --2- 172.21.15.43:0/3735312786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f340dcea0 0x556f340ee180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:31.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.875+0000 7fe1f7fff640 1 -- 172.21.15.43:0/3735312786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f340dc580 msgr2=0x556f340edc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:31.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.875+0000 7fe1f7fff640 1 --2- 172.21.15.43:0/3735312786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f340dc580 0x556f340edc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.875+0000 7fe1f7fff640 1 -- 172.21.15.43:0/3735312786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f340dcea0 msgr2=0x556f340ee180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:31.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.875+0000 7fe1f7fff640 1 --2- 172.21.15.43:0/3735312786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f340dcea0 0x556f340ee180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.875+0000 7fe1f7fff640 1 -- 172.21.15.43:0/3735312786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f340eb420 con 0x556f340e5a80 2024-04-21T08:49:31.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.875+0000 7fe1f6ffd640 1 --2- 172.21.15.43:0/3735312786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f340dcea0 0x556f340ee180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:31.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.875+0000 7fe1f77fe640 1 --2- 172.21.15.43:0/3735312786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f340dc580 0x556f340edc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:31.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.876+0000 7fe1f7fff640 1 --2- 172.21.15.43:0/3735312786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340e5a80 0x556f340eac10 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00058c0 tx=0x7fe1f000c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab93bdd832dd259f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:31.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.876+0000 7fe1d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:31.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.876+0000 7fe1d6ffd640 1 -- 172.21.15.43:0/3735312786 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0026040 con 0x556f340e5a80 2024-04-21T08:49:31.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.876+0000 7fe1d6ffd640 1 -- 172.21.15.43:0/3735312786 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fe1f0004940 con 0x556f340e5a80 2024-04-21T08:49:31.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.876+0000 7fe200789740 1 -- 172.21.15.43:0/3735312786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1e80c2e50 con 0x556f340e5a80 2024-04-21T08:49:31.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.876+0000 7fe1d6ffd640 1 -- 172.21.15.43:0/3735312786 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe1f0011ce0 con 0x556f340e5a80 2024-04-21T08:49:31.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.876+0000 7fe200789740 10 client.20526 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:31.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.876+0000 7fe200789740 1 -- 172.21.15.43:0/3735312786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556f3415cb50 con 0x556f340e5a80 2024-04-21T08:49:31.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.876+0000 7fe1d6ffd640 1 -- 172.21.15.43:0/3735312786 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e80c2e50 con 0x556f340e5a80 2024-04-21T08:49:31.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.876+0000 7fe1d6ffd640 1 -- 172.21.15.43:0/3735312786 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe1f0020c00 con 0x556f340e5a80 2024-04-21T08:49:31.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.876+0000 7fe200789740 10 client.20526 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:31.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.876+0000 7fe200789740 10 client.20526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:31.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.876+0000 7fe200789740 20 client.20526 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:31.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.876+0000 7fe200789740 1 --2- 172.21.15.43:0/3735312786 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556f3415d320 0x556f3415f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:31.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.876+0000 7fe200789740 4 client.20526 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:49:31.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.876+0000 7fe200789740 1 -- 172.21.15.43:0/3735312786 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556f34048a70 con 0x556f3415d320 2024-04-21T08:49:31.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.877+0000 7fe1f77fe640 1 --2- 172.21.15.43:0/3735312786 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556f3415d320 0x556f3415f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:31.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.877+0000 7fe1d6ffd640 10 client.20526 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:31.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.877+0000 7fe1f77fe640 1 --2- 172.21.15.43:0/3735312786 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556f3415d320 0x556f3415f740 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=198dfc400b20065d server_cookie=79682229e19722eb in_seq=0 out_seq=0 2024-04-21T08:49:31.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.877+0000 7fe1d6ffd640 1 -- 172.21.15.43:0/3735312786 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556f34048a70 con 0x556f3415d320 2024-04-21T08:49:31.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.877+0000 7fe1d6ffd640 10 client.20526 handle_command_reply: tid=0 2024-04-21T08:49:31.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.877+0000 7fe200789740 1 client.20526 shutdown 2024-04-21T08:49:31.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.877+0000 7fe200789740 1 -- 172.21.15.43:0/3735312786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340e5a80 msgr2=0x556f340eac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:31.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.877+0000 7fe200789740 1 --2- 172.21.15.43:0/3735312786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340e5a80 0x556f340eac10 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00058c0 tx=0x7fe1f000c570 comp rx=0 tx=0).stop 2024-04-21T08:49:31.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.877+0000 7fe200789740 1 -- 172.21.15.43:0/3735312786 shutdown_connections 2024-04-21T08:49:31.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.877+0000 7fe200789740 1 --2- 172.21.15.43:0/3735312786 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556f3415d320 0x556f3415f740 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.877+0000 7fe200789740 1 --2- 172.21.15.43:0/3735312786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f340e5a80 0x556f340eac10 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.877+0000 7fe200789740 1 --2- 172.21.15.43:0/3735312786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f340dcea0 0x556f340ee180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.877+0000 7fe1d6ffd640 0 client.20526 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:31.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.877+0000 7fe200789740 1 --2- 172.21.15.43:0/3735312786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556f340dc580 0x556f340edc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.877+0000 7fe200789740 1 -- 172.21.15.43:0/3735312786 >> 172.21.15.43:0/3735312786 conn(0x556f340d6cb0 msgr2=0x556f340e2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:31.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.878+0000 7fe200789740 1 -- 172.21.15.43:0/3735312786 shutdown_connections 2024-04-21T08:49:31.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.878+0000 7fe200789740 1 -- 172.21.15.43:0/3735312786 wait complete. 2024-04-21T08:49:31.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.878+0000 7fe200789740 20 client.20526 trim_cache size 0 max 16384 2024-04-21T08:49:31.884 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:49:31.884 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:49:31.884 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:49:31.884 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:49:31.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.878+0000 7fe1fdb96640 1 -- 172.21.15.43:0/2280784978 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe1e40a3d50 msgr2=0x7fe1e40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:31.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.878+0000 7fe1fdb96640 1 --2- 172.21.15.43:0/2280784978 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe1e40a3d50 0x7fe1e40a6210 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0005fd0 tx=0x7fe1f0005850 comp rx=0 tx=0).stop 2024-04-21T08:49:31.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.878+0000 7fe1fdb96640 1 -- 172.21.15.43:0/2280784978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1f8102190 msgr2=0x7fe1f8074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:31.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.878+0000 7fe1fdb96640 1 --2- 172.21.15.43:0/2280784978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1f8102190 0x7fe1f8074250 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8005e30 tx=0x7fe1e8034780 comp rx=0 tx=0).stop 2024-04-21T08:49:31.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.878+0000 7fe1fdb96640 1 -- 172.21.15.43:0/2280784978 shutdown_connections 2024-04-21T08:49:31.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.878+0000 7fe1fdb96640 1 --2- 172.21.15.43:0/2280784978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1f8074cd0 0x7fe1f81a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.878+0000 7fe1fdb96640 1 --2- 172.21.15.43:0/2280784978 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe1e40a3d50 0x7fe1e40a6210 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.878+0000 7fe1fdb96640 1 --2- 172.21.15.43:0/2280784978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1f8107260 0x7fe1f8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.878+0000 7fe1fdb96640 1 --2- 172.21.15.43:0/2280784978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1f8102190 0x7fe1f8074250 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.878+0000 7fe1fdb96640 1 -- 172.21.15.43:0/2280784978 >> 172.21.15.43:0/2280784978 conn(0x7fe1f80fda30 msgr2=0x7fe1f80fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:31.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.878+0000 7fe1fdb96640 1 -- 172.21.15.43:0/2280784978 shutdown_connections 2024-04-21T08:49:31.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.878+0000 7fe1fdb96640 1 -- 172.21.15.43:0/2280784978 wait complete. 2024-04-21T08:49:31.889 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:49:31.889 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:49:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.992+0000 7f7c9c184640 1 -- 172.21.15.43:0/3211513168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c94102200 msgr2=0x7f7c941025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.992+0000 7f7c9c184640 1 --2- 172.21.15.43:0/3211513168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c94102200 0x7f7c941025e0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009900 tx=0x7f7c9002bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.992+0000 7f7c9c184640 1 -- 172.21.15.43:0/3211513168 shutdown_connections 2024-04-21T08:49:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.992+0000 7f7c9c184640 1 --2- 172.21.15.43:0/3211513168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c941072d0 0x7f7c9410eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.992+0000 7f7c9c184640 1 --2- 172.21.15.43:0/3211513168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c94102bb0 0x7f7c94106bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.992+0000 7f7c9c184640 1 --2- 172.21.15.43:0/3211513168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c94102200 0x7f7c941025e0 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.992+0000 7f7c9c184640 1 -- 172.21.15.43:0/3211513168 >> 172.21.15.43:0/3211513168 conn(0x7f7c940fda60 msgr2=0x7f7c940ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.992+0000 7f7c9c184640 1 -- 172.21.15.43:0/3211513168 shutdown_connections 2024-04-21T08:49:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.992+0000 7f7c9c184640 1 -- 172.21.15.43:0/3211513168 wait complete. 2024-04-21T08:49:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.993+0000 7f7c9c184640 1 Processor -- start 2024-04-21T08:49:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.993+0000 7f7c9c184640 1 -- start start 2024-04-21T08:49:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.993+0000 7f7c9c184640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c94102200 0x7f7c94198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.993+0000 7f7c9c184640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c941072d0 0x7f7c94198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.993+0000 7f7c9c184640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9419cff0 0x7f7c9419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.993+0000 7f7c9c184640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7c94112210 con 0x7f7c9419cff0 2024-04-21T08:49:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.993+0000 7f7c9c184640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7c94112090 con 0x7f7c941072d0 2024-04-21T08:49:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.993+0000 7f7c9c184640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7c94112390 con 0x7f7c94102200 2024-04-21T08:49:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.993+0000 7f7c9a721640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9419cff0 0x7f7c9419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.993+0000 7f7c9a721640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9419cff0 0x7f7c9419d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47474/0 (socket says 172.21.15.43:47474) 2024-04-21T08:49:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.993+0000 7f7c9a721640 1 -- 172.21.15.43:0/1660898022 learned_addr learned my addr 172.21.15.43:0/1660898022 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.994+0000 7f7c9971f640 1 --2- 172.21.15.43:0/1660898022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c941072d0 0x7f7c94198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.994+0000 7f7c99f20640 1 --2- 172.21.15.43:0/1660898022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c94102200 0x7f7c94198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.994+0000 7f7c9a721640 1 -- 172.21.15.43:0/1660898022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c94102200 msgr2=0x7f7c94198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.994+0000 7f7c9a721640 1 --2- 172.21.15.43:0/1660898022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c94102200 0x7f7c94198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.994+0000 7f7c9a721640 1 -- 172.21.15.43:0/1660898022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c941072d0 msgr2=0x7f7c94198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.994+0000 7f7c9a721640 1 --2- 172.21.15.43:0/1660898022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c941072d0 0x7f7c94198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.994+0000 7f7c9a721640 1 -- 172.21.15.43:0/1660898022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9419dae0 con 0x7f7c9419cff0 2024-04-21T08:49:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.994+0000 7f7c9971f640 1 --2- 172.21.15.43:0/1660898022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c941072d0 0x7f7c94198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.994+0000 7f7c9a721640 1 --2- 172.21.15.43:0/1660898022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9419cff0 0x7f7c9419d4a0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c00cce0 tx=0x7f7c8c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.994+0000 7f7c8affd640 1 -- 172.21.15.43:0/1660898022 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c00f450 con 0x7f7c9419cff0 2024-04-21T08:49:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.994+0000 7f7c8affd640 1 -- 172.21.15.43:0/1660898022 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c8c00faa0 con 0x7f7c9419cff0 2024-04-21T08:49:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.994+0000 7f7c9c184640 1 -- 172.21.15.43:0/1660898022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9419ddd0 con 0x7f7c9419cff0 2024-04-21T08:49:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.994+0000 7f7c8affd640 1 -- 172.21.15.43:0/1660898022 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c018b70 con 0x7f7c9419cff0 2024-04-21T08:49:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.994+0000 7f7c9c184640 1 -- 172.21.15.43:0/1660898022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c941a55c0 con 0x7f7c9419cff0 2024-04-21T08:49:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.995+0000 7f7c9c184640 1 -- 172.21.15.43:0/1660898022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c94069230 con 0x7f7c9419cff0 2024-04-21T08:49:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.997+0000 7f7c8affd640 1 -- 172.21.15.43:0/1660898022 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7c8c018d10 con 0x7f7c9419cff0 2024-04-21T08:49:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.998+0000 7f7c8affd640 1 --2- 172.21.15.43:0/1660898022 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7c800a3da0 0x7f7c800a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:31.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.998+0000 7f7c99f20640 1 --2- 172.21.15.43:0/1660898022 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7c800a3da0 0x7f7c800a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:31.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.998+0000 7f7c8affd640 1 -- 172.21.15.43:0/1660898022 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f7c8c0d10d0 con 0x7f7c9419cff0 2024-04-21T08:49:31.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:31.998+0000 7f7c99f20640 1 --2- 172.21.15.43:0/1660898022 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7c800a3da0 0x7f7c800a6260 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7c90036040 tx=0x7f7c90000bc0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.001+0000 7f7c8affd640 1 -- 172.21.15.43:0/1660898022 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7c8c014030 con 0x7f7c9419cff0 2024-04-21T08:49:32.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.232+0000 7f7c9c184640 1 -- 172.21.15.43:0/1660898022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c94199a80 con 0x7f7c9419cff0 2024-04-21T08:49:32.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.233+0000 7f7c8affd640 1 -- 172.21.15.43:0/1660898022 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11935 (secure 0 0 0) 0x7f7c94199a80 con 0x7f7c9419cff0 2024-04-21T08:49:32.233 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:49:32.233 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:46:17.283806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27556":{"gid":27556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/229985237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":229985237},{"type":"v1","addr":"172.21.15.175:6837","nonce":229985237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24425,24412,24487,24527,24491]},"id":1}]} 2024-04-21T08:49:32.233 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-04-21T08:49:32.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.235+0000 7f7c9c184640 1 -- 172.21.15.43:0/1660898022 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7c800a3da0 msgr2=0x7f7c800a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.235+0000 7f7c9c184640 1 --2- 172.21.15.43:0/1660898022 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7c800a3da0 0x7f7c800a6260 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7c90036040 tx=0x7f7c90000bc0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.235+0000 7f7c9c184640 1 -- 172.21.15.43:0/1660898022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9419cff0 msgr2=0x7f7c9419d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.235+0000 7f7c9c184640 1 --2- 172.21.15.43:0/1660898022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9419cff0 0x7f7c9419d4a0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c00cce0 tx=0x7f7c8c00a620 comp rx=0 tx=0).stop 2024-04-21T08:49:32.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.235+0000 7f7c9c184640 1 -- 172.21.15.43:0/1660898022 shutdown_connections 2024-04-21T08:49:32.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.235+0000 7f7c9c184640 1 --2- 172.21.15.43:0/1660898022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9419cff0 0x7f7c9419d4a0 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.235+0000 7f7c9c184640 1 --2- 172.21.15.43:0/1660898022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c941072d0 0x7f7c94198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.235+0000 7f7c9c184640 1 --2- 172.21.15.43:0/1660898022 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7c800a3da0 0x7f7c800a6260 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.235+0000 7f7c9c184640 1 --2- 172.21.15.43:0/1660898022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c94102200 0x7f7c94198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.235+0000 7f7c9c184640 1 -- 172.21.15.43:0/1660898022 >> 172.21.15.43:0/1660898022 conn(0x7f7c940fda60 msgr2=0x7f7c940fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.235+0000 7f7c9c184640 1 -- 172.21.15.43:0/1660898022 shutdown_connections 2024-04-21T08:49:32.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.235+0000 7f7c9c184640 1 -- 172.21.15.43:0/1660898022 wait complete. 2024-04-21T08:49:32.246 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:49:32.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.351+0000 7fae2332b640 1 -- 172.21.15.43:0/1981897683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae1c106160 msgr2=0x7fae1c10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.351+0000 7fae2332b640 1 --2- 172.21.15.43:0/1981897683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae1c106160 0x7fae1c10dd30 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fae14009ca0 tx=0x7fae1402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.351+0000 7fae2332b640 1 -- 172.21.15.43:0/1981897683 shutdown_connections 2024-04-21T08:49:32.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.351+0000 7fae2332b640 1 --2- 172.21.15.43:0/1981897683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae1c106160 0x7fae1c10dd30 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.351+0000 7fae2332b640 1 --2- 172.21.15.43:0/1981897683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae1c101a40 0x7fae1c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.351+0000 7fae2332b640 1 --2- 172.21.15.43:0/1981897683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae1c101090 0x7fae1c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.351+0000 7fae2332b640 1 -- 172.21.15.43:0/1981897683 >> 172.21.15.43:0/1981897683 conn(0x7fae1c077f50 msgr2=0x7fae1c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.351+0000 7fae2332b640 1 -- 172.21.15.43:0/1981897683 shutdown_connections 2024-04-21T08:49:32.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.351+0000 7fae2332b640 1 -- 172.21.15.43:0/1981897683 wait complete. 2024-04-21T08:49:32.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.352+0000 7fae2332b640 1 Processor -- start 2024-04-21T08:49:32.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.352+0000 7fae2332b640 1 -- start start 2024-04-21T08:49:32.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.352+0000 7fae2332b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae1c101090 0x7fae1c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.352+0000 7fae2332b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae1c106160 0x7fae1c198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.352+0000 7fae2332b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae1c19cfc0 0x7fae1c19d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.352+0000 7fae2332b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c1110b0 con 0x7fae1c101090 2024-04-21T08:49:32.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.352+0000 7fae2332b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c110f30 con 0x7fae1c19cfc0 2024-04-21T08:49:32.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.352+0000 7fae2332b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c111230 con 0x7fae1c106160 2024-04-21T08:49:32.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.352+0000 7fae210c7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae1c101090 0x7fae1c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.352+0000 7fae210c7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae1c101090 0x7fae1c198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47498/0 (socket says 172.21.15.43:47498) 2024-04-21T08:49:32.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.352+0000 7fae210c7640 1 -- 172.21.15.43:0/1738590273 learned_addr learned my addr 172.21.15.43:0/1738590273 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:32.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.352+0000 7fae208c6640 1 --2- 172.21.15.43:0/1738590273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae1c106160 0x7fae1c198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.352+0000 7fae218c8640 1 --2- 172.21.15.43:0/1738590273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae1c19cfc0 0x7fae1c19d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.352+0000 7fae210c7640 1 -- 172.21.15.43:0/1738590273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae1c106160 msgr2=0x7fae1c198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.352+0000 7fae210c7640 1 --2- 172.21.15.43:0/1738590273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae1c106160 0x7fae1c198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.352+0000 7fae210c7640 1 -- 172.21.15.43:0/1738590273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae1c19cfc0 msgr2=0x7fae1c19d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.352+0000 7fae210c7640 1 --2- 172.21.15.43:0/1738590273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae1c19cfc0 0x7fae1c19d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.352+0000 7fae210c7640 1 -- 172.21.15.43:0/1738590273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae1c19dab0 con 0x7fae1c101090 2024-04-21T08:49:32.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.352+0000 7fae210c7640 1 --2- 172.21.15.43:0/1738590273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae1c101090 0x7fae1c198700 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fae1800d950 tx=0x7fae1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.353+0000 7fae127fc640 1 -- 172.21.15.43:0/1738590273 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18017030 con 0x7fae1c101090 2024-04-21T08:49:32.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.353+0000 7fae2332b640 1 -- 172.21.15.43:0/1738590273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae1c19dda0 con 0x7fae1c101090 2024-04-21T08:49:32.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.353+0000 7fae127fc640 1 -- 172.21.15.43:0/1738590273 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae18004020 con 0x7fae1c101090 2024-04-21T08:49:32.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.353+0000 7fae127fc640 1 -- 172.21.15.43:0/1738590273 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae180047a0 con 0x7fae1c101090 2024-04-21T08:49:32.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.353+0000 7fae2332b640 1 -- 172.21.15.43:0/1738590273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae1c1a5590 con 0x7fae1c101090 2024-04-21T08:49:32.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.355+0000 7fae127fc640 1 -- 172.21.15.43:0/1738590273 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fae18009a30 con 0x7fae1c101090 2024-04-21T08:49:32.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.355+0000 7fae25ff8740 1 Processor -- start 2024-04-21T08:49:32.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.355+0000 7fae25ff8740 1 -- start start 2024-04-21T08:49:32.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.355+0000 7fae25ff8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b70165e4d0 0x55b70165e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.355+0000 7fae25ff8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b70165ee80 0x55b701667430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.355+0000 7fae25ff8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b701667a00 0x55b701669e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.355+0000 7fae25ff8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b7015c31f0 con 0x55b701667a00 2024-04-21T08:49:32.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.355+0000 7fae25ff8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b7015604b0 con 0x55b70165ee80 2024-04-21T08:49:32.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.355+0000 7fae25ff8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b7015bf1b0 con 0x55b70165e4d0 2024-04-21T08:49:32.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.356+0000 7fae208c6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b701667a00 0x55b701669e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.356+0000 7fae208c6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b701667a00 0x55b701669e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47516/0 (socket says 172.21.15.43:47516) 2024-04-21T08:49:32.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.356+0000 7fae208c6640 1 -- 172.21.15.43:0/3657528594 learned_addr learned my addr 172.21.15.43:0/3657528594 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:32.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.356+0000 7fae210c7640 1 --2- 172.21.15.43:0/3657528594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b70165ee80 0x55b701667430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.356+0000 7fae127fc640 1 --2- 172.21.15.43:0/1738590273 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fae080a3d20 0x7fae080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.356+0000 7fae218c8640 1 --2- 172.21.15.43:0/3657528594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b70165e4d0 0x55b70165e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.356+0000 7fae208c6640 1 -- 172.21.15.43:0/3657528594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b70165e4d0 msgr2=0x55b70165e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.356+0000 7fae208c6640 1 --2- 172.21.15.43:0/3657528594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b70165e4d0 0x55b70165e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.356+0000 7fae208c6640 1 -- 172.21.15.43:0/3657528594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b70165ee80 msgr2=0x55b701667430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.356+0000 7fae208c6640 1 --2- 172.21.15.43:0/3657528594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b70165ee80 0x55b701667430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.356+0000 7fae208c6640 1 -- 172.21.15.43:0/3657528594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7015521b0 con 0x55b701667a00 2024-04-21T08:49:32.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.356+0000 7fae218c8640 1 --2- 172.21.15.43:0/1738590273 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fae080a3d20 0x7fae080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.356+0000 7fae127fc640 1 -- 172.21.15.43:0/1738590273 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fae180d06a0 con 0x7fae1c101090 2024-04-21T08:49:32.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.356+0000 7fae208c6640 1 --2- 172.21.15.43:0/3657528594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b701667a00 0x55b701669e50 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00bbd0 tx=0x7fae0c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c72d2d4fb3149b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.356+0000 7fae077fe640 1 -- 172.21.15.43:0/3657528594 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0c01c480 con 0x55b701667a00 2024-04-21T08:49:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.356+0000 7fae077fe640 1 -- 172.21.15.43:0/3657528594 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae0c01cad0 con 0x55b701667a00 2024-04-21T08:49:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.356+0000 7fae077fe640 1 -- 172.21.15.43:0/3657528594 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0c011a90 con 0x55b701667a00 2024-04-21T08:49:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.357+0000 7fae25ff8740 1 -- 172.21.15.43:0/3657528594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b701667a00 msgr2=0x55b701669e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.357+0000 7fae218c8640 1 --2- 172.21.15.43:0/1738590273 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fae080a3d20 0x7fae080a61e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fae14005e30 tx=0x7fae1402e040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.357+0000 7fae25ff8740 1 --2- 172.21.15.43:0/3657528594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b701667a00 0x55b701669e50 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00bbd0 tx=0x7fae0c0107a0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.357+0000 7fae25ff8740 1 -- 172.21.15.43:0/3657528594 shutdown_connections 2024-04-21T08:49:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.357+0000 7fae25ff8740 1 --2- 172.21.15.43:0/3657528594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b701667a00 0x55b701669e50 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.357+0000 7fae25ff8740 1 --2- 172.21.15.43:0/3657528594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b70165ee80 0x55b701667430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.357+0000 7fae25ff8740 1 --2- 172.21.15.43:0/3657528594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b70165e4d0 0x55b70165e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.357+0000 7fae25ff8740 1 -- 172.21.15.43:0/3657528594 >> 172.21.15.43:0/3657528594 conn(0x55b701658ba0 msgr2=0x55b701658f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.357+0000 7fae25ff8740 1 -- 172.21.15.43:0/3657528594 shutdown_connections 2024-04-21T08:49:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.357+0000 7fae25ff8740 1 -- 172.21.15.43:0/3657528594 wait complete. 2024-04-21T08:49:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.357+0000 7fae25ff8740 1 -- start start 2024-04-21T08:49:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.357+0000 7fae25ff8740 1 Processor -- start 2024-04-21T08:49:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.358+0000 7fae25ff8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b70165e4d0 0x55b701671a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.358+0000 7fae25ff8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b70165ee80 0x55b701671fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.358+0000 7fae25ff8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b701667a00 0x55b70166be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.358+0000 7fae25ff8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b70166a970 con 0x55b70165ee80 2024-04-21T08:49:32.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.358+0000 7fae25ff8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b70157c500 con 0x55b70165e4d0 2024-04-21T08:49:32.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.358+0000 7fae25ff8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b70166aaf0 con 0x55b701667a00 2024-04-21T08:49:32.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.358+0000 7fae25ff8740 10 client.0 _ll_register_callbacks cb 0x55b70154ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:32.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.358+0000 7fae208c6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b70165ee80 0x55b701671fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.358+0000 7fae208c6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b70165ee80 0x55b701671fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47518/0 (socket says 172.21.15.43:47518) 2024-04-21T08:49:32.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.358+0000 7fae208c6640 1 -- 172.21.15.43:0/3472978671 learned_addr learned my addr 172.21.15.43:0/3472978671 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:32.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.358+0000 7fae210c7640 1 --2- 172.21.15.43:0/3472978671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b70165e4d0 0x55b701671a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.358+0000 7fae218c8640 1 --2- 172.21.15.43:0/3472978671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b701667a00 0x55b70166be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.358+0000 7fae208c6640 1 -- 172.21.15.43:0/3472978671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b701667a00 msgr2=0x55b70166be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.358+0000 7fae208c6640 1 --2- 172.21.15.43:0/3472978671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b701667a00 0x55b70166be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.358+0000 7fae208c6640 1 -- 172.21.15.43:0/3472978671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b70165e4d0 msgr2=0x55b701671a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.358+0000 7fae208c6640 1 --2- 172.21.15.43:0/3472978671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b70165e4d0 0x55b701671a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.358+0000 7fae208c6640 1 -- 172.21.15.43:0/3472978671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7012d53f0 con 0x55b70165ee80 2024-04-21T08:49:32.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.358+0000 7fae208c6640 1 --2- 172.21.15.43:0/3472978671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b70165ee80 0x55b701671fb0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fae0c011630 tx=0x7fae0c01c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281fd7306452b1fa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.358+0000 7fae04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:32.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.359+0000 7fae04ff9640 1 -- 172.21.15.43:0/3472978671 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0c00b7e0 con 0x55b70165ee80 2024-04-21T08:49:32.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.359+0000 7fae04ff9640 1 -- 172.21.15.43:0/3472978671 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fae0c00bd80 con 0x55b70165ee80 2024-04-21T08:49:32.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.359+0000 7fae25ff8740 1 -- 172.21.15.43:0/3472978671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae0c01cad0 con 0x55b70165ee80 2024-04-21T08:49:32.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.359+0000 7fae04ff9640 1 -- 172.21.15.43:0/3472978671 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fae0c030050 con 0x55b70165ee80 2024-04-21T08:49:32.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.359+0000 7fae25ff8740 10 client.20544 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:32.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.359+0000 7fae25ff8740 1 -- 172.21.15.43:0/3472978671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b70139f7d0 con 0x55b70165ee80 2024-04-21T08:49:32.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.359+0000 7fae04ff9640 1 -- 172.21.15.43:0/3472978671 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae0c01cad0 con 0x55b70165ee80 2024-04-21T08:49:32.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.359+0000 7fae04ff9640 1 -- 172.21.15.43:0/3472978671 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fae0c0035f0 con 0x55b70165ee80 2024-04-21T08:49:32.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.359+0000 7fae25ff8740 10 client.20544 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.359+0000 7fae25ff8740 10 client.20544 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:49:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.359+0000 7fae25ff8740 20 client.20544 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.359+0000 7fae25ff8740 1 --2- 172.21.15.43:0/3472978671 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55b70167be70 0x55b70167e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.359+0000 7fae25ff8740 4 client.20544 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.359+0000 7fae25ff8740 1 -- 172.21.15.43:0/3472978671 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b701562930 con 0x55b70167be70 2024-04-21T08:49:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.360+0000 7fae210c7640 1 --2- 172.21.15.43:0/3472978671 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55b70167be70 0x55b70167e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.360+0000 7fae210c7640 1 --2- 172.21.15.43:0/3472978671 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55b70167be70 0x55b70167e260 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c26f78aebefd85df server_cookie=11aab0527391831f in_seq=0 out_seq=0 2024-04-21T08:49:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.360+0000 7fae04ff9640 10 client.20544 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:49:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.361+0000 7fae04ff9640 1 -- 172.21.15.43:0/3472978671 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b701562930 con 0x55b70167be70 2024-04-21T08:49:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.361+0000 7fae04ff9640 10 client.20544 handle_command_reply: tid=0 2024-04-21T08:49:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.361+0000 7fae25ff8740 1 client.20544 shutdown 2024-04-21T08:49:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.361+0000 7fae25ff8740 1 -- 172.21.15.43:0/3472978671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b70165ee80 msgr2=0x55b701671fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.361+0000 7fae25ff8740 1 --2- 172.21.15.43:0/3472978671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b70165ee80 0x55b701671fb0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fae0c011630 tx=0x7fae0c01c6b0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.361+0000 7fae25ff8740 1 -- 172.21.15.43:0/3472978671 shutdown_connections 2024-04-21T08:49:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.361+0000 7fae25ff8740 1 --2- 172.21.15.43:0/3472978671 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55b70167be70 0x55b70167e260 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.361+0000 7fae25ff8740 1 --2- 172.21.15.43:0/3472978671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b701667a00 0x55b70166be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.361+0000 7fae04ff9640 0 client.20544 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:49:32.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.361+0000 7fae25ff8740 1 --2- 172.21.15.43:0/3472978671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b70165ee80 0x55b701671fb0 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.361+0000 7fae25ff8740 1 --2- 172.21.15.43:0/3472978671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b70165e4d0 0x55b701671a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.361+0000 7fae25ff8740 1 -- 172.21.15.43:0/3472978671 >> 172.21.15.43:0/3472978671 conn(0x55b701658ba0 msgr2=0x55b70165a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.361+0000 7fae25ff8740 1 -- 172.21.15.43:0/3472978671 shutdown_connections 2024-04-21T08:49:32.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.361+0000 7fae25ff8740 1 -- 172.21.15.43:0/3472978671 wait complete. 2024-04-21T08:49:32.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.361+0000 7fae25ff8740 20 client.20544 trim_cache size 0 max 16384 2024-04-21T08:49:32.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.374+0000 7fae25ff8740 1 Processor -- start 2024-04-21T08:49:32.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.375+0000 7fae25ff8740 1 -- start start 2024-04-21T08:49:32.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.375+0000 7fae25ff8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b70165e4d0 0x55b701671e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.375+0000 7fae25ff8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b70165ee80 0x55b701657a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.375+0000 7fae25ff8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b701667a00 0x55b701683860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.375+0000 7fae25ff8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c1ad3f0 con 0x55b70165ee80 2024-04-21T08:49:32.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.375+0000 7fae25ff8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c1aef30 con 0x55b701667a00 2024-04-21T08:49:32.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.375+0000 7fae25ff8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c1adac0 con 0x55b70165e4d0 2024-04-21T08:49:32.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.375+0000 7fae208c6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b70165ee80 0x55b701657a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.375+0000 7fae208c6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b70165ee80 0x55b701657a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47534/0 (socket says 172.21.15.43:47534) 2024-04-21T08:49:32.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.375+0000 7fae208c6640 1 -- 172.21.15.43:0/3029277667 learned_addr learned my addr 172.21.15.43:0/3029277667 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:32.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.375+0000 7fae210c7640 1 --2- 172.21.15.43:0/3029277667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b70165e4d0 0x55b701671e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.375+0000 7fae208c6640 1 -- 172.21.15.43:0/3029277667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b70165e4d0 msgr2=0x55b701671e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.375+0000 7fae208c6640 1 --2- 172.21.15.43:0/3029277667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b70165e4d0 0x55b701671e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.375+0000 7fae208c6640 1 -- 172.21.15.43:0/3029277667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b701667a00 msgr2=0x55b701683860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.375+0000 7fae218c8640 1 --2- 172.21.15.43:0/3029277667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b701667a00 0x55b701683860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.375+0000 7fae208c6640 1 --2- 172.21.15.43:0/3029277667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b701667a00 0x55b701683860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.375+0000 7fae208c6640 1 -- 172.21.15.43:0/3029277667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b70165ebe0 con 0x55b70165ee80 2024-04-21T08:49:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.375+0000 7fae210c7640 1 --2- 172.21.15.43:0/3029277667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b70165e4d0 0x55b701671e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.375+0000 7fae218c8640 1 --2- 172.21.15.43:0/3029277667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b701667a00 0x55b701683860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.375+0000 7fae208c6640 1 --2- 172.21.15.43:0/3029277667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b70165ee80 0x55b701657a80 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fae0c003990 tx=0x7fae0c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9acf1460b774b78 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.375+0000 7fae06ffd640 1 -- 172.21.15.43:0/3029277667 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0c01cab0 con 0x55b70165ee80 2024-04-21T08:49:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.375+0000 7fae06ffd640 1 -- 172.21.15.43:0/3029277667 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae0c004480 con 0x55b70165ee80 2024-04-21T08:49:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.375+0000 7fae06ffd640 1 -- 172.21.15.43:0/3029277667 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0c003c00 con 0x55b70165ee80 2024-04-21T08:49:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.376+0000 7fae25ff8740 1 -- 172.21.15.43:0/3029277667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b70165ee80 msgr2=0x55b701657a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.376+0000 7fae25ff8740 1 --2- 172.21.15.43:0/3029277667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b70165ee80 0x55b701657a80 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fae0c003990 tx=0x7fae0c005780 comp rx=0 tx=0).stop 2024-04-21T08:49:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.376+0000 7fae25ff8740 1 -- 172.21.15.43:0/3029277667 shutdown_connections 2024-04-21T08:49:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.376+0000 7fae25ff8740 1 --2- 172.21.15.43:0/3029277667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b701667a00 0x55b701683860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.376+0000 7fae25ff8740 1 --2- 172.21.15.43:0/3029277667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b70165ee80 0x55b701657a80 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.376+0000 7fae25ff8740 1 --2- 172.21.15.43:0/3029277667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b70165e4d0 0x55b701671e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.376+0000 7fae25ff8740 1 -- 172.21.15.43:0/3029277667 >> 172.21.15.43:0/3029277667 conn(0x55b701658ba0 msgr2=0x55b701657370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.376+0000 7fae25ff8740 1 -- 172.21.15.43:0/3029277667 shutdown_connections 2024-04-21T08:49:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.376+0000 7fae25ff8740 1 -- 172.21.15.43:0/3029277667 wait complete. 2024-04-21T08:49:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.376+0000 7fae25ff8740 1 -- start start 2024-04-21T08:49:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.376+0000 7fae25ff8740 1 Processor -- start 2024-04-21T08:49:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.377+0000 7fae25ff8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b70165e4d0 0x55b701677e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.377+0000 7fae25ff8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b70165ee80 0x55b701675220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.377+0000 7fae25ff8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b701667a00 0x55b701675760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.377+0000 7fae25ff8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c1ae860 con 0x55b70165ee80 2024-04-21T08:49:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.377+0000 7fae25ff8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c1ae190 con 0x55b70165e4d0 2024-04-21T08:49:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.377+0000 7fae25ff8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c1ab3f0 con 0x55b701667a00 2024-04-21T08:49:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.377+0000 7fae25ff8740 10 client.0 _ll_register_callbacks cb 0x7fae0c005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.377+0000 7fae208c6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b70165ee80 0x55b701675220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.377+0000 7fae208c6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b70165ee80 0x55b701675220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47548/0 (socket says 172.21.15.43:47548) 2024-04-21T08:49:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.377+0000 7fae208c6640 1 -- 172.21.15.43:0/601499510 learned_addr learned my addr 172.21.15.43:0/601499510 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.377+0000 7fae218c8640 1 --2- 172.21.15.43:0/601499510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b701667a00 0x55b701675760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.377+0000 7fae210c7640 1 --2- 172.21.15.43:0/601499510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b70165e4d0 0x55b701677e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.377+0000 7fae208c6640 1 -- 172.21.15.43:0/601499510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b701667a00 msgr2=0x55b701675760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.377+0000 7fae208c6640 1 --2- 172.21.15.43:0/601499510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b701667a00 0x55b701675760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.377+0000 7fae208c6640 1 -- 172.21.15.43:0/601499510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b70165e4d0 msgr2=0x55b701677e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.377+0000 7fae208c6640 1 --2- 172.21.15.43:0/601499510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b70165e4d0 0x55b701677e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.377+0000 7fae208c6640 1 -- 172.21.15.43:0/601499510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b701675ca0 con 0x55b70165ee80 2024-04-21T08:49:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.377+0000 7fae210c7640 1 --2- 172.21.15.43:0/601499510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b70165e4d0 0x55b701677e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.377+0000 7fae208c6640 1 --2- 172.21.15.43:0/601499510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b70165ee80 0x55b701675220 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fae0c003b00 tx=0x7fae0c010d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5474d02e304d562c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.377+0000 7fae04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.378+0000 7fae04ff9640 1 -- 172.21.15.43:0/601499510 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0c01cab0 con 0x55b70165ee80 2024-04-21T08:49:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.378+0000 7fae04ff9640 1 -- 172.21.15.43:0/601499510 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fae0c03a040 con 0x55b70165ee80 2024-04-21T08:49:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.378+0000 7fae25ff8740 1 -- 172.21.15.43:0/601499510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b7015d0c10 con 0x55b70165ee80 2024-04-21T08:49:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.378+0000 7fae04ff9640 1 -- 172.21.15.43:0/601499510 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fae0c0035f0 con 0x55b70165ee80 2024-04-21T08:49:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.378+0000 7fae25ff8740 10 client.20550 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.378+0000 7fae25ff8740 1 -- 172.21.15.43:0/601499510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b7016e1d60 con 0x55b70165ee80 2024-04-21T08:49:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.378+0000 7fae04ff9640 1 -- 172.21.15.43:0/601499510 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b7015d0c10 con 0x55b70165ee80 2024-04-21T08:49:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.378+0000 7fae04ff9640 1 -- 172.21.15.43:0/601499510 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fae0c011930 con 0x55b70165ee80 2024-04-21T08:49:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.378+0000 7fae25ff8740 10 client.20550 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.378+0000 7fae25ff8740 10 client.20550 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:49:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.378+0000 7fae25ff8740 20 client.20550 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.378+0000 7fae25ff8740 1 --2- 172.21.15.43:0/601499510 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55b7016e2430 0x55b7016e4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.378+0000 7fae25ff8740 4 client.20550 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.378+0000 7fae25ff8740 1 -- 172.21.15.43:0/601499510 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b7015caa80 con 0x55b7016e2430 2024-04-21T08:49:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.378+0000 7fae210c7640 1 --2- 172.21.15.43:0/601499510 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55b7016e2430 0x55b7016e4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.379+0000 7fae210c7640 1 --2- 172.21.15.43:0/601499510 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55b7016e2430 0x55b7016e4850 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=20d1069c72ce620d server_cookie=a0d59877cac5eb35 in_seq=0 out_seq=0 2024-04-21T08:49:32.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.379+0000 7fae04ff9640 10 client.20550 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:49:32.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.379+0000 7fae04ff9640 1 -- 172.21.15.43:0/601499510 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b7015caa80 con 0x55b7016e2430 2024-04-21T08:49:32.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.379+0000 7fae04ff9640 10 client.20550 handle_command_reply: tid=0 2024-04-21T08:49:32.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.379+0000 7fae25ff8740 1 client.20550 shutdown 2024-04-21T08:49:32.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.379+0000 7fae25ff8740 1 -- 172.21.15.43:0/601499510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b70165ee80 msgr2=0x55b701675220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.379+0000 7fae25ff8740 1 --2- 172.21.15.43:0/601499510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b70165ee80 0x55b701675220 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fae0c003b00 tx=0x7fae0c010d40 comp rx=0 tx=0).stop 2024-04-21T08:49:32.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.379+0000 7fae25ff8740 1 -- 172.21.15.43:0/601499510 shutdown_connections 2024-04-21T08:49:32.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.379+0000 7fae25ff8740 1 --2- 172.21.15.43:0/601499510 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55b7016e2430 0x55b7016e4850 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.379+0000 7fae25ff8740 1 --2- 172.21.15.43:0/601499510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b701667a00 0x55b701675760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.379+0000 7fae25ff8740 1 --2- 172.21.15.43:0/601499510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b70165ee80 0x55b701675220 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.379+0000 7fae04ff9640 0 client.20550 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:49:32.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.379+0000 7fae25ff8740 1 --2- 172.21.15.43:0/601499510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b70165e4d0 0x55b701677e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.379+0000 7fae25ff8740 1 -- 172.21.15.43:0/601499510 >> 172.21.15.43:0/601499510 conn(0x55b701658ba0 msgr2=0x55b701664070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.379+0000 7fae25ff8740 1 -- 172.21.15.43:0/601499510 shutdown_connections 2024-04-21T08:49:32.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.379+0000 7fae25ff8740 1 -- 172.21.15.43:0/601499510 wait complete. 2024-04-21T08:49:32.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.379+0000 7fae25ff8740 20 client.20550 trim_cache size 0 max 16384 2024-04-21T08:49:32.386 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:49:32.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.380+0000 7fae2332b640 1 -- 172.21.15.43:0/1738590273 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fae080a3d20 msgr2=0x7fae080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.380+0000 7fae2332b640 1 --2- 172.21.15.43:0/1738590273 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fae080a3d20 0x7fae080a61e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fae14005e30 tx=0x7fae1402e040 comp rx=0 tx=0).stop 2024-04-21T08:49:32.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.380+0000 7fae2332b640 1 -- 172.21.15.43:0/1738590273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae1c101090 msgr2=0x7fae1c198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.380+0000 7fae2332b640 1 --2- 172.21.15.43:0/1738590273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae1c101090 0x7fae1c198700 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fae1800d950 tx=0x7fae1800de10 comp rx=0 tx=0).stop 2024-04-21T08:49:32.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.380+0000 7fae2332b640 1 -- 172.21.15.43:0/1738590273 shutdown_connections 2024-04-21T08:49:32.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.380+0000 7fae2332b640 1 --2- 172.21.15.43:0/1738590273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae1c19cfc0 0x7fae1c19d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.380+0000 7fae2332b640 1 --2- 172.21.15.43:0/1738590273 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fae080a3d20 0x7fae080a61e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.380+0000 7fae2332b640 1 --2- 172.21.15.43:0/1738590273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae1c106160 0x7fae1c198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.380+0000 7fae2332b640 1 --2- 172.21.15.43:0/1738590273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae1c101090 0x7fae1c198700 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.380+0000 7fae2332b640 1 -- 172.21.15.43:0/1738590273 >> 172.21.15.43:0/1738590273 conn(0x7fae1c077f50 msgr2=0x7fae1c069480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.380+0000 7fae2332b640 1 -- 172.21.15.43:0/1738590273 shutdown_connections 2024-04-21T08:49:32.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.380+0000 7fae2332b640 1 -- 172.21.15.43:0/1738590273 wait complete. 2024-04-21T08:49:32.390 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:49:32.411 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:32 smithi043 ceph-mon[27978]: pgmap v330: 129 pgs: 129 active+clean; 2.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-21T08:49:32.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:32 smithi175 ceph-mon[31920]: pgmap v330: 129 pgs: 129 active+clean; 2.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-21T08:49:32.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.496+0000 7f52af7f4640 1 -- 172.21.15.43:0/3697325166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52a806a840 msgr2=0x7f52a806ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.496+0000 7f52af7f4640 1 --2- 172.21.15.43:0/3697325166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52a806a840 0x7f52a806ac20 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009ca0 tx=0x7f52a002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.496+0000 7f52af7f4640 1 -- 172.21.15.43:0/3697325166 shutdown_connections 2024-04-21T08:49:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.496+0000 7f52af7f4640 1 --2- 172.21.15.43:0/3697325166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52a8107270 0x7f52a810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.496+0000 7f52af7f4640 1 --2- 172.21.15.43:0/3697325166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52a806b1f0 0x7f52a8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.496+0000 7f52af7f4640 1 --2- 172.21.15.43:0/3697325166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52a806a840 0x7f52a806ac20 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.496+0000 7f52af7f4640 1 -- 172.21.15.43:0/3697325166 >> 172.21.15.43:0/3697325166 conn(0x7f52a8077eb0 msgr2=0x7f52a80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.496+0000 7f52af7f4640 1 -- 172.21.15.43:0/3697325166 shutdown_connections 2024-04-21T08:49:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.496+0000 7f52af7f4640 1 -- 172.21.15.43:0/3697325166 wait complete. 2024-04-21T08:49:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.497+0000 7f52af7f4640 1 Processor -- start 2024-04-21T08:49:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.497+0000 7f52af7f4640 1 -- start start 2024-04-21T08:49:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.497+0000 7f52af7f4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52a806a840 0x7f52a8100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.497+0000 7f52af7f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52a8107270 0x7f52a81013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.497+0000 7f52af7f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52a8101900 0x7f52a81a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.497+0000 7f52af7f4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f52a81121b0 con 0x7f52a8101900 2024-04-21T08:49:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.497+0000 7f52af7f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f52a8112030 con 0x7f52a8107270 2024-04-21T08:49:32.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.497+0000 7f52af7f4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f52a8112330 con 0x7f52a806a840 2024-04-21T08:49:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.497+0000 7f52add91640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52a8101900 0x7f52a81a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.497+0000 7f52add91640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52a8101900 0x7f52a81a68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47578/0 (socket says 172.21.15.43:47578) 2024-04-21T08:49:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.497+0000 7f52add91640 1 -- 172.21.15.43:0/2107014066 learned_addr learned my addr 172.21.15.43:0/2107014066 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.497+0000 7f52acd8f640 1 --2- 172.21.15.43:0/2107014066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52a8107270 0x7f52a81013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.497+0000 7f52ad590640 1 --2- 172.21.15.43:0/2107014066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52a806a840 0x7f52a8100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.497+0000 7f52add91640 1 -- 172.21.15.43:0/2107014066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52a806a840 msgr2=0x7f52a8100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.497+0000 7f52add91640 1 --2- 172.21.15.43:0/2107014066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52a806a840 0x7f52a8100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.497+0000 7f52add91640 1 -- 172.21.15.43:0/2107014066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52a8107270 msgr2=0x7f52a81013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.497+0000 7f52add91640 1 --2- 172.21.15.43:0/2107014066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52a8107270 0x7f52a81013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.497+0000 7f52add91640 1 -- 172.21.15.43:0/2107014066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a81a6fa0 con 0x7f52a8101900 2024-04-21T08:49:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.498+0000 7f52add91640 1 --2- 172.21.15.43:0/2107014066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52a8101900 0x7f52a81a68a0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f529800e970 tx=0x7f529800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.498+0000 7f52a67fc640 1 -- 172.21.15.43:0/2107014066 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529800ccc0 con 0x7f52a8101900 2024-04-21T08:49:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.498+0000 7f52a67fc640 1 -- 172.21.15.43:0/2107014066 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f529800ce60 con 0x7f52a8101900 2024-04-21T08:49:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.498+0000 7f52af7f4640 1 -- 172.21.15.43:0/2107014066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52a81a7290 con 0x7f52a8101900 2024-04-21T08:49:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.498+0000 7f52a67fc640 1 -- 172.21.15.43:0/2107014066 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298010430 con 0x7f52a8101900 2024-04-21T08:49:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.498+0000 7f52af7f4640 1 -- 172.21.15.43:0/2107014066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52a81a7790 con 0x7f52a8101900 2024-04-21T08:49:32.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.500+0000 7f52a67fc640 1 -- 172.21.15.43:0/2107014066 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5298004f90 con 0x7f52a8101900 2024-04-21T08:49:32.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.500+0000 7f52b23e3740 1 Processor -- start 2024-04-21T08:49:32.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.501+0000 7f52b23e3740 1 -- start start 2024-04-21T08:49:32.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.501+0000 7f52b23e3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc64d0 0x55b150dc68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.501+0000 7f52b23e3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b150dc6e80 0x55b150dcf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.501+0000 7f52b23e3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b150dcfa00 0x55b150dd1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.501+0000 7f52b23e3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b150d2b1f0 con 0x55b150dc64d0 2024-04-21T08:49:32.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.501+0000 7f52b23e3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b150cc84b0 con 0x55b150dc6e80 2024-04-21T08:49:32.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.501+0000 7f52b23e3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b150d271b0 con 0x55b150dcfa00 2024-04-21T08:49:32.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.501+0000 7f52acd8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc64d0 0x55b150dc68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.501+0000 7f52acd8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc64d0 0x55b150dc68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47584/0 (socket says 172.21.15.43:47584) 2024-04-21T08:49:32.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.501+0000 7f52acd8f640 1 -- 172.21.15.43:0/704798013 learned_addr learned my addr 172.21.15.43:0/704798013 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:32.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.501+0000 7f52a67fc640 1 --2- 172.21.15.43:0/2107014066 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f52900a3d50 0x7f52900a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.501+0000 7f52ad590640 1 --2- 172.21.15.43:0/704798013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b150dc6e80 0x55b150dcf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.501+0000 7f52acd8f640 1 --2- 172.21.15.43:0/704798013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b150dcfa00 0x55b150dd1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.501+0000 7f52acd8f640 1 -- 172.21.15.43:0/704798013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b150dcfa00 msgr2=0x55b150dd1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.501+0000 7f52add91640 1 --2- 172.21.15.43:0/2107014066 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f52900a3d50 0x7f52900a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.501+0000 7f52acd8f640 1 --2- 172.21.15.43:0/704798013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b150dcfa00 0x55b150dd1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.501+0000 7f52acd8f640 1 -- 172.21.15.43:0/704798013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b150dc6e80 msgr2=0x55b150dcf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.501+0000 7f52acd8f640 1 --2- 172.21.15.43:0/704798013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b150dc6e80 0x55b150dcf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.501+0000 7f52acd8f640 1 -- 172.21.15.43:0/704798013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b150cba1b0 con 0x55b150dc64d0 2024-04-21T08:49:32.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.501+0000 7f52a67fc640 1 -- 172.21.15.43:0/2107014066 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f52980d0f60 con 0x7f52a8101900 2024-04-21T08:49:32.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.501+0000 7f52acd8f640 1 --2- 172.21.15.43:0/704798013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc64d0 0x55b150dc68b0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f529c00be50 tx=0x7f529c00b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36af4987014c116a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.502+0000 7f528f7fe640 1 -- 172.21.15.43:0/704798013 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c012e60 con 0x55b150dc64d0 2024-04-21T08:49:32.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.502+0000 7f52add91640 1 --2- 172.21.15.43:0/2107014066 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f52900a3d50 0x7f52900a6210 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f52980994b0 tx=0x7f529806e520 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.502+0000 7f528f7fe640 1 -- 172.21.15.43:0/704798013 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f529c0047a0 con 0x55b150dc64d0 2024-04-21T08:49:32.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.502+0000 7f528f7fe640 1 -- 172.21.15.43:0/704798013 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c0052a0 con 0x55b150dc64d0 2024-04-21T08:49:32.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.502+0000 7f52b23e3740 1 -- 172.21.15.43:0/704798013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc64d0 msgr2=0x55b150dc68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.502+0000 7f52b23e3740 1 --2- 172.21.15.43:0/704798013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc64d0 0x55b150dc68b0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f529c00be50 tx=0x7f529c00b650 comp rx=0 tx=0).stop 2024-04-21T08:49:32.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.502+0000 7f52b23e3740 1 -- 172.21.15.43:0/704798013 shutdown_connections 2024-04-21T08:49:32.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.502+0000 7f52b23e3740 1 --2- 172.21.15.43:0/704798013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b150dcfa00 0x55b150dd1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.502+0000 7f52b23e3740 1 --2- 172.21.15.43:0/704798013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b150dc6e80 0x55b150dcf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.502+0000 7f52b23e3740 1 --2- 172.21.15.43:0/704798013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc64d0 0x55b150dc68b0 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.502+0000 7f52b23e3740 1 -- 172.21.15.43:0/704798013 >> 172.21.15.43:0/704798013 conn(0x55b150dc0ba0 msgr2=0x55b150dc0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.503+0000 7f52b23e3740 1 -- 172.21.15.43:0/704798013 shutdown_connections 2024-04-21T08:49:32.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.503+0000 7f52b23e3740 1 -- 172.21.15.43:0/704798013 wait complete. 2024-04-21T08:49:32.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.503+0000 7f52b23e3740 1 -- start start 2024-04-21T08:49:32.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.503+0000 7f52b23e3740 1 Processor -- start 2024-04-21T08:49:32.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.503+0000 7f52b23e3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc64d0 0x55b150dcf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.503+0000 7f52b23e3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b150dc6e80 0x55b150dcc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.503+0000 7f52b23e3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b150dcfa00 0x55b150dccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.503+0000 7f52b23e3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b150dd2910 con 0x55b150dc64d0 2024-04-21T08:49:32.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.503+0000 7f52b23e3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b150ce4500 con 0x55b150dcfa00 2024-04-21T08:49:32.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.503+0000 7f52b23e3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b150dd2a90 con 0x55b150dc6e80 2024-04-21T08:49:32.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.503+0000 7f52b23e3740 10 client.0 _ll_register_callbacks cb 0x55b150cb2b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:32.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.503+0000 7f52acd8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc64d0 0x55b150dcf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.503+0000 7f52acd8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc64d0 0x55b150dcf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47590/0 (socket says 172.21.15.43:47590) 2024-04-21T08:49:32.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.503+0000 7f52acd8f640 1 -- 172.21.15.43:0/395294989 learned_addr learned my addr 172.21.15.43:0/395294989 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:32.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.504+0000 7f52acd8f640 1 -- 172.21.15.43:0/395294989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b150dc6e80 msgr2=0x55b150dcc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.504+0000 7f52acd8f640 1 --2- 172.21.15.43:0/395294989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b150dc6e80 0x55b150dcc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.504+0000 7f52acd8f640 1 -- 172.21.15.43:0/395294989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b150dcfa00 msgr2=0x55b150dccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.504+0000 7f52acd8f640 1 --2- 172.21.15.43:0/395294989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b150dcfa00 0x55b150dccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.504+0000 7f52acd8f640 1 -- 172.21.15.43:0/395294989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b150a3d3f0 con 0x55b150dc64d0 2024-04-21T08:49:32.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.504+0000 7f52acd8f640 1 --2- 172.21.15.43:0/395294989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc64d0 0x55b150dcf340 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f529c004bd0 tx=0x7f529c004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b887fff6f2041a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.504+0000 7f528cff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:32.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.504+0000 7f528cff9640 1 -- 172.21.15.43:0/395294989 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c012e60 con 0x55b150dc64d0 2024-04-21T08:49:32.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.504+0000 7f528cff9640 1 -- 172.21.15.43:0/395294989 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f529c004ff0 con 0x55b150dc64d0 2024-04-21T08:49:32.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.504+0000 7f52b23e3740 1 -- 172.21.15.43:0/395294989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f529c0047a0 con 0x55b150dc64d0 2024-04-21T08:49:32.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.504+0000 7f528cff9640 1 -- 172.21.15.43:0/395294989 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f529c0051f0 con 0x55b150dc64d0 2024-04-21T08:49:32.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.504+0000 7f52b23e3740 10 client.20562 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:32.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.504+0000 7f52b23e3740 1 -- 172.21.15.43:0/395294989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b150b077d0 con 0x55b150dc64d0 2024-04-21T08:49:32.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.504+0000 7f528cff9640 1 -- 172.21.15.43:0/395294989 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f529c0047a0 con 0x55b150dc64d0 2024-04-21T08:49:32.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.504+0000 7f528cff9640 1 -- 172.21.15.43:0/395294989 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f529c026420 con 0x55b150dc64d0 2024-04-21T08:49:32.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.505+0000 7f52b23e3740 10 client.20562 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:32.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.505+0000 7f52b23e3740 10 client.20562 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:49:32.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.505+0000 7f52b23e3740 20 client.20562 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:32.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.505+0000 7f52b23e3740 1 --2- 172.21.15.43:0/395294989 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55b150ddfb70 0x55b150de1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.505+0000 7f52b23e3740 4 client.20562 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:32.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.505+0000 7f52b23e3740 1 -- 172.21.15.43:0/395294989 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b150cca930 con 0x55b150ddfb70 2024-04-21T08:49:32.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.505+0000 7f52ad590640 1 --2- 172.21.15.43:0/395294989 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55b150ddfb70 0x55b150de1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.505+0000 7f528cff9640 10 client.20562 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:49:32.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.505+0000 7f52ad590640 1 --2- 172.21.15.43:0/395294989 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55b150ddfb70 0x55b150de1f60 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=57c772d66f6c5624 server_cookie=829f5113c88cbbaa in_seq=0 out_seq=0 2024-04-21T08:49:32.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.506+0000 7f528cff9640 1 -- 172.21.15.43:0/395294989 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b150cca930 con 0x55b150ddfb70 2024-04-21T08:49:32.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.506+0000 7f528cff9640 10 client.20562 handle_command_reply: tid=0 2024-04-21T08:49:32.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.506+0000 7f52b23e3740 1 client.20562 shutdown 2024-04-21T08:49:32.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.506+0000 7f52b23e3740 1 -- 172.21.15.43:0/395294989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc64d0 msgr2=0x55b150dcf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.506+0000 7f52b23e3740 1 --2- 172.21.15.43:0/395294989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc64d0 0x55b150dcf340 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f529c004bd0 tx=0x7f529c004c00 comp rx=0 tx=0).stop 2024-04-21T08:49:32.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.506+0000 7f52b23e3740 1 -- 172.21.15.43:0/395294989 shutdown_connections 2024-04-21T08:49:32.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.506+0000 7f52b23e3740 1 --2- 172.21.15.43:0/395294989 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55b150ddfb70 0x55b150de1f60 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.506+0000 7f52b23e3740 1 --2- 172.21.15.43:0/395294989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b150dcfa00 0x55b150dccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.506+0000 7f52b23e3740 1 --2- 172.21.15.43:0/395294989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b150dc6e80 0x55b150dcc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.506+0000 7f52b23e3740 1 --2- 172.21.15.43:0/395294989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc64d0 0x55b150dcf340 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.506+0000 7f528cff9640 0 client.20562 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:49:32.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.506+0000 7f52b23e3740 1 -- 172.21.15.43:0/395294989 >> 172.21.15.43:0/395294989 conn(0x55b150dc0ba0 msgr2=0x55b150dc28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.506+0000 7f52b23e3740 1 -- 172.21.15.43:0/395294989 shutdown_connections 2024-04-21T08:49:32.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.506+0000 7f52b23e3740 1 -- 172.21.15.43:0/395294989 wait complete. 2024-04-21T08:49:32.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.506+0000 7f52b23e3740 20 client.20562 trim_cache size 0 max 16384 2024-04-21T08:49:32.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.520+0000 7f52b23e3740 1 Processor -- start 2024-04-21T08:49:32.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.520+0000 7f52b23e3740 1 -- start start 2024-04-21T08:49:32.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.520+0000 7f52b23e3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b150dc64d0 0x55b150ddb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.520+0000 7f52b23e3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc6e80 0x55b150dcee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.520+0000 7f52b23e3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b150dcfa00 0x55b150dcf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.520+0000 7f52b23e3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f52a81b1740 con 0x55b150dc6e80 2024-04-21T08:49:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.520+0000 7f52b23e3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f52a81b3280 con 0x55b150dc64d0 2024-04-21T08:49:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.520+0000 7f52b23e3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f52a81b1e10 con 0x55b150dcfa00 2024-04-21T08:49:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.520+0000 7f52ad590640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc6e80 0x55b150dcee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.520+0000 7f52ad590640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc6e80 0x55b150dcee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47598/0 (socket says 172.21.15.43:47598) 2024-04-21T08:49:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.520+0000 7f52ad590640 1 -- 172.21.15.43:0/2178013296 learned_addr learned my addr 172.21.15.43:0/2178013296 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.520+0000 7f52acd8f640 1 --2- 172.21.15.43:0/2178013296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b150dcfa00 0x55b150dcf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.520+0000 7f52ad590640 1 -- 172.21.15.43:0/2178013296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b150dcfa00 msgr2=0x55b150dcf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.520+0000 7f52ad590640 1 --2- 172.21.15.43:0/2178013296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b150dcfa00 0x55b150dcf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.520+0000 7f52ad590640 1 -- 172.21.15.43:0/2178013296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b150dc64d0 msgr2=0x55b150ddb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.520+0000 7f52ad590640 1 --2- 172.21.15.43:0/2178013296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b150dc64d0 0x55b150ddb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.520+0000 7f52ad590640 1 -- 172.21.15.43:0/2178013296 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b150dcac20 con 0x55b150dc6e80 2024-04-21T08:49:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.520+0000 7f52acd8f640 1 --2- 172.21.15.43:0/2178013296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b150dcfa00 0x55b150dcf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.521+0000 7f52ad590640 1 --2- 172.21.15.43:0/2178013296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc6e80 0x55b150dcee00 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009920 tx=0x7f52a002e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de96aa2970ba30c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.521+0000 7f528effd640 1 -- 172.21.15.43:0/2178013296 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0039040 con 0x55b150dc6e80 2024-04-21T08:49:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.521+0000 7f528effd640 1 -- 172.21.15.43:0/2178013296 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a004b020 con 0x55b150dc6e80 2024-04-21T08:49:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.521+0000 7f528effd640 1 -- 172.21.15.43:0/2178013296 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a00324c0 con 0x55b150dc6e80 2024-04-21T08:49:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.521+0000 7f52b23e3740 1 -- 172.21.15.43:0/2178013296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc6e80 msgr2=0x55b150dcee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.521+0000 7f52b23e3740 1 --2- 172.21.15.43:0/2178013296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc6e80 0x55b150dcee00 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009920 tx=0x7f52a002e040 comp rx=0 tx=0).stop 2024-04-21T08:49:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.521+0000 7f52b23e3740 1 -- 172.21.15.43:0/2178013296 shutdown_connections 2024-04-21T08:49:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.521+0000 7f52b23e3740 1 --2- 172.21.15.43:0/2178013296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b150dcfa00 0x55b150dcf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.521+0000 7f52b23e3740 1 --2- 172.21.15.43:0/2178013296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc6e80 0x55b150dcee00 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.521+0000 7f52b23e3740 1 --2- 172.21.15.43:0/2178013296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b150dc64d0 0x55b150ddb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.521+0000 7f52b23e3740 1 -- 172.21.15.43:0/2178013296 >> 172.21.15.43:0/2178013296 conn(0x55b150dc0ba0 msgr2=0x55b150ddace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.521+0000 7f52b23e3740 1 -- 172.21.15.43:0/2178013296 shutdown_connections 2024-04-21T08:49:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.522+0000 7f52b23e3740 1 -- 172.21.15.43:0/2178013296 wait complete. 2024-04-21T08:49:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.522+0000 7f52b23e3740 1 -- start start 2024-04-21T08:49:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.522+0000 7f52b23e3740 1 Processor -- start 2024-04-21T08:49:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.522+0000 7f52b23e3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc64d0 0x55b150de4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.522+0000 7f52b23e3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b150dc6e80 0x55b150de5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.522+0000 7f52b23e3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b150dcfa00 0x55b150de24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.522+0000 7f52b23e3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f52a81b2bb0 con 0x55b150dc64d0 2024-04-21T08:49:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.522+0000 7f52b23e3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f52a81b24e0 con 0x55b150dcfa00 2024-04-21T08:49:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.522+0000 7f52b23e3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f52a81af740 con 0x55b150dc6e80 2024-04-21T08:49:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.522+0000 7f52b23e3740 10 client.0 _ll_register_callbacks cb 0x7f529c012230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.522+0000 7f52acd8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc64d0 0x55b150de4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.522+0000 7f52acd8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc64d0 0x55b150de4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47602/0 (socket says 172.21.15.43:47602) 2024-04-21T08:49:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.522+0000 7f52acd8f640 1 -- 172.21.15.43:0/1531355677 learned_addr learned my addr 172.21.15.43:0/1531355677 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.522+0000 7f52ad590640 1 --2- 172.21.15.43:0/1531355677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b150dc6e80 0x55b150de5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.523+0000 7f52acd8f640 1 -- 172.21.15.43:0/1531355677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b150dc6e80 msgr2=0x55b150de5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.523+0000 7f52acd8f640 1 --2- 172.21.15.43:0/1531355677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b150dc6e80 0x55b150de5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.523+0000 7f52acd8f640 1 -- 172.21.15.43:0/1531355677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b150dcfa00 msgr2=0x55b150de24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.523+0000 7f52acd8f640 1 --2- 172.21.15.43:0/1531355677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b150dcfa00 0x55b150de24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.523+0000 7f52acd8f640 1 -- 172.21.15.43:0/1531355677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b150de2ce0 con 0x55b150dc64d0 2024-04-21T08:49:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.523+0000 7f52ad590640 1 --2- 172.21.15.43:0/1531355677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b150dc6e80 0x55b150de5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.523+0000 7f52acd8f640 1 --2- 172.21.15.43:0/1531355677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc64d0 0x55b150de4f50 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f529c00be50 tx=0x7f529c005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ea3e2bab89f4f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.523+0000 7f528cff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.523+0000 7f528cff9640 1 -- 172.21.15.43:0/1531355677 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c012b90 con 0x55b150dc64d0 2024-04-21T08:49:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.523+0000 7f528cff9640 1 -- 172.21.15.43:0/1531355677 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f529c022040 con 0x55b150dc64d0 2024-04-21T08:49:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.523+0000 7f52b23e3740 1 -- 172.21.15.43:0/1531355677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52a004b020 con 0x55b150dc64d0 2024-04-21T08:49:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.523+0000 7f528cff9640 1 -- 172.21.15.43:0/1531355677 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f529c03b050 con 0x55b150dc64d0 2024-04-21T08:49:32.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.523+0000 7f52b23e3740 10 client.20568 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:32.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.523+0000 7f52b23e3740 1 -- 172.21.15.43:0/1531355677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b150e458d0 con 0x55b150dc64d0 2024-04-21T08:49:32.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.523+0000 7f528cff9640 1 -- 172.21.15.43:0/1531355677 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a004b020 con 0x55b150dc64d0 2024-04-21T08:49:32.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.523+0000 7f528cff9640 1 -- 172.21.15.43:0/1531355677 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f529c010760 con 0x55b150dc64d0 2024-04-21T08:49:32.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.523+0000 7f52b23e3740 10 client.20568 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:32.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.523+0000 7f52b23e3740 10 client.20568 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:49:32.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.523+0000 7f52b23e3740 20 client.20568 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:32.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.524+0000 7f52b23e3740 1 --2- 172.21.15.43:0/1531355677 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55b150e45ee0 0x55b150e482d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.524+0000 7f52b23e3740 4 client.20568 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:32.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.524+0000 7f52b23e3740 1 -- 172.21.15.43:0/1531355677 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b150d38c10 con 0x55b150e45ee0 2024-04-21T08:49:32.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.524+0000 7f52ad590640 1 --2- 172.21.15.43:0/1531355677 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55b150e45ee0 0x55b150e482d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.524+0000 7f52ad590640 1 --2- 172.21.15.43:0/1531355677 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55b150e45ee0 0x55b150e482d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=516b145dd4615749 server_cookie=bdab1d74e38c3290 in_seq=0 out_seq=0 2024-04-21T08:49:32.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.524+0000 7f528cff9640 10 client.20568 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:49:32.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.524+0000 7f528cff9640 1 -- 172.21.15.43:0/1531355677 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b150d38c10 con 0x55b150e45ee0 2024-04-21T08:49:32.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.524+0000 7f528cff9640 10 client.20568 handle_command_reply: tid=0 2024-04-21T08:49:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.524+0000 7f52b23e3740 1 client.20568 shutdown 2024-04-21T08:49:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.525+0000 7f52b23e3740 1 -- 172.21.15.43:0/1531355677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc64d0 msgr2=0x55b150de4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.525+0000 7f52b23e3740 1 --2- 172.21.15.43:0/1531355677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc64d0 0x55b150de4f50 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f529c00be50 tx=0x7f529c005ae0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.525+0000 7f52b23e3740 1 -- 172.21.15.43:0/1531355677 shutdown_connections 2024-04-21T08:49:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.525+0000 7f52b23e3740 1 --2- 172.21.15.43:0/1531355677 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55b150e45ee0 0x55b150e482d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.525+0000 7f52b23e3740 1 --2- 172.21.15.43:0/1531355677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b150dcfa00 0x55b150de24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.525+0000 7f52b23e3740 1 --2- 172.21.15.43:0/1531355677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b150dc6e80 0x55b150de5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.525+0000 7f528cff9640 0 client.20568 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:49:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.525+0000 7f52b23e3740 1 --2- 172.21.15.43:0/1531355677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b150dc64d0 0x55b150de4f50 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.525+0000 7f52b23e3740 1 -- 172.21.15.43:0/1531355677 >> 172.21.15.43:0/1531355677 conn(0x55b150dc0ba0 msgr2=0x55b150ddb570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.525+0000 7f52b23e3740 1 -- 172.21.15.43:0/1531355677 shutdown_connections 2024-04-21T08:49:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.525+0000 7f52b23e3740 1 -- 172.21.15.43:0/1531355677 wait complete. 2024-04-21T08:49:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.525+0000 7f52b23e3740 20 client.20568 trim_cache size 0 max 16384 2024-04-21T08:49:32.531 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:49:32.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.525+0000 7f52af7f4640 1 -- 172.21.15.43:0/2107014066 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f52900a3d50 msgr2=0x7f52900a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.525+0000 7f52af7f4640 1 --2- 172.21.15.43:0/2107014066 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f52900a3d50 0x7f52900a6210 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f52980994b0 tx=0x7f529806e520 comp rx=0 tx=0).stop 2024-04-21T08:49:32.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.525+0000 7f52af7f4640 1 -- 172.21.15.43:0/2107014066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52a8101900 msgr2=0x7f52a81a68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.525+0000 7f52af7f4640 1 --2- 172.21.15.43:0/2107014066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52a8101900 0x7f52a81a68a0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f529800e970 tx=0x7f529800ed70 comp rx=0 tx=0).stop 2024-04-21T08:49:32.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.525+0000 7f52af7f4640 1 -- 172.21.15.43:0/2107014066 shutdown_connections 2024-04-21T08:49:32.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.525+0000 7f52af7f4640 1 --2- 172.21.15.43:0/2107014066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52a8101900 0x7f52a81a68a0 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.525+0000 7f52af7f4640 1 --2- 172.21.15.43:0/2107014066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52a8107270 0x7f52a81013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.525+0000 7f52af7f4640 1 --2- 172.21.15.43:0/2107014066 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f52900a3d50 0x7f52900a6210 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.525+0000 7f52af7f4640 1 --2- 172.21.15.43:0/2107014066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52a806a840 0x7f52a8100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.525+0000 7f52af7f4640 1 -- 172.21.15.43:0/2107014066 >> 172.21.15.43:0/2107014066 conn(0x7f52a8077eb0 msgr2=0x7f52a8108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.525+0000 7f52af7f4640 1 -- 172.21.15.43:0/2107014066 shutdown_connections 2024-04-21T08:49:32.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.525+0000 7f52af7f4640 1 -- 172.21.15.43:0/2107014066 wait complete. 2024-04-21T08:49:32.535 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:49:32.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:32 smithi112 ceph-mon[31085]: pgmap v330: 129 pgs: 129 active+clean; 2.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-21T08:49:32.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.679+0000 7ffae8dce640 1 -- 172.21.15.43:0/1554033520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffae41014a0 msgr2=0x7ffae4106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.679+0000 7ffae8dce640 1 --2- 172.21.15.43:0/1554033520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffae41014a0 0x7ffae4106bd0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009900 tx=0x7ffacc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:32.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.680+0000 7ffae8dce640 1 -- 172.21.15.43:0/1554033520 shutdown_connections 2024-04-21T08:49:32.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.680+0000 7ffae8dce640 1 --2- 172.21.15.43:0/1554033520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffae41072b0 0x7ffae410eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.680+0000 7ffae8dce640 1 --2- 172.21.15.43:0/1554033520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffae41014a0 0x7ffae4106bd0 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.680+0000 7ffae8dce640 1 --2- 172.21.15.43:0/1554033520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffae4100af0 0x7ffae4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.680+0000 7ffae8dce640 1 -- 172.21.15.43:0/1554033520 >> 172.21.15.43:0/1554033520 conn(0x7ffae40760e0 msgr2=0x7ffae40764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.680+0000 7ffae8dce640 1 -- 172.21.15.43:0/1554033520 shutdown_connections 2024-04-21T08:49:32.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.680+0000 7ffae8dce640 1 -- 172.21.15.43:0/1554033520 wait complete. 2024-04-21T08:49:32.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.680+0000 7ffae8dce640 1 Processor -- start 2024-04-21T08:49:32.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.681+0000 7ffae8dce640 1 -- start start 2024-04-21T08:49:32.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.681+0000 7ffae8dce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffae4100af0 0x7ffae4101a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.681+0000 7ffae8dce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffae41072b0 0x7ffae4101fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.681+0000 7ffae8dce640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffae41024e0 0x7ffae41a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.681+0000 7ffae8dce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffae4112210 con 0x7ffae4100af0 2024-04-21T08:49:32.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.681+0000 7ffae8dce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffae4112090 con 0x7ffae41072b0 2024-04-21T08:49:32.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.681+0000 7ffae8dce640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffae4112390 con 0x7ffae41024e0 2024-04-21T08:49:32.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.681+0000 7ffae259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffae4100af0 0x7ffae4101a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.681+0000 7ffae259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffae4100af0 0x7ffae4101a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47622/0 (socket says 172.21.15.43:47622) 2024-04-21T08:49:32.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.681+0000 7ffae259c640 1 -- 172.21.15.43:0/1217490876 learned_addr learned my addr 172.21.15.43:0/1217490876 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:32.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.681+0000 7ffad9d9b640 1 --2- 172.21.15.43:0/1217490876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffae41072b0 0x7ffae4101fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.681+0000 7ffae2d9d640 1 --2- 172.21.15.43:0/1217490876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffae41024e0 0x7ffae41a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.681+0000 7ffae259c640 1 -- 172.21.15.43:0/1217490876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffae41024e0 msgr2=0x7ffae41a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.681+0000 7ffae259c640 1 --2- 172.21.15.43:0/1217490876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffae41024e0 0x7ffae41a6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.681+0000 7ffae259c640 1 -- 172.21.15.43:0/1217490876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffae41072b0 msgr2=0x7ffae4101fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.681+0000 7ffae259c640 1 --2- 172.21.15.43:0/1217490876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffae41072b0 0x7ffae4101fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.681+0000 7ffae259c640 1 -- 172.21.15.43:0/1217490876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae41a7020 con 0x7ffae4100af0 2024-04-21T08:49:32.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.681+0000 7ffad9d9b640 1 --2- 172.21.15.43:0/1217490876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffae41072b0 0x7ffae4101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:32.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.681+0000 7ffae259c640 1 --2- 172.21.15.43:0/1217490876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffae4100af0 0x7ffae4101a60 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7ffad400c920 tx=0x7ffad400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.681+0000 7ffadbfff640 1 -- 172.21.15.43:0/1217490876 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad4007c70 con 0x7ffae4100af0 2024-04-21T08:49:32.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.681+0000 7ffae8dce640 1 -- 172.21.15.43:0/1217490876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae41a7310 con 0x7ffae4100af0 2024-04-21T08:49:32.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.681+0000 7ffadbfff640 1 -- 172.21.15.43:0/1217490876 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffad4007e10 con 0x7ffae4100af0 2024-04-21T08:49:32.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.681+0000 7ffadbfff640 1 -- 172.21.15.43:0/1217490876 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad4018820 con 0x7ffae4100af0 2024-04-21T08:49:32.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.682+0000 7ffae8dce640 1 -- 172.21.15.43:0/1217490876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae41a7880 con 0x7ffae4100af0 2024-04-21T08:49:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.684+0000 7ffadbfff640 1 -- 172.21.15.43:0/1217490876 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffad400f450 con 0x7ffae4100af0 2024-04-21T08:49:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.684+0000 7ffaeb9c3740 1 Processor -- start 2024-04-21T08:49:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.684+0000 7ffaeb9c3740 1 -- start start 2024-04-21T08:49:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.684+0000 7ffaeb9c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa72804d0 0x556fa72808b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.684+0000 7ffaeb9c3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fa7280e80 0x556fa7289430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.684+0000 7ffadbfff640 1 --2- 172.21.15.43:0/1217490876 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ffad00a3d50 0x7ffad00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.684+0000 7ffaeb9c3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fa7289a00 0x556fa728be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.684+0000 7ffaeb9c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556fa71e51f0 con 0x556fa72804d0 2024-04-21T08:49:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.684+0000 7ffaeb9c3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556fa71824b0 con 0x556fa7289a00 2024-04-21T08:49:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.684+0000 7ffaeb9c3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556fa71e11b0 con 0x556fa7280e80 2024-04-21T08:49:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.684+0000 7ffae2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa72804d0 0x556fa72808b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.685+0000 7ffad9d9b640 1 --2- 172.21.15.43:0/1217490876 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ffad00a3d50 0x7ffad00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.685+0000 7ffae2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa72804d0 0x556fa72808b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47632/0 (socket says 172.21.15.43:47632) 2024-04-21T08:49:32.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.685+0000 7ffae2d9d640 1 -- 172.21.15.43:0/1167681390 learned_addr learned my addr 172.21.15.43:0/1167681390 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:32.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.685+0000 7ffadbfff640 1 -- 172.21.15.43:0/1217490876 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7ffad40d0dc0 con 0x7ffae4100af0 2024-04-21T08:49:32.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.685+0000 7ffae259c640 1 --2- 172.21.15.43:0/1167681390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fa7280e80 0x556fa7289430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.685+0000 7ffae2d9d640 1 --2- 172.21.15.43:0/1167681390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fa7289a00 0x556fa728be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.685+0000 7ffae2d9d640 1 -- 172.21.15.43:0/1167681390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fa7280e80 msgr2=0x556fa7289430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.685+0000 7ffae2d9d640 1 --2- 172.21.15.43:0/1167681390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fa7280e80 0x556fa7289430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.685+0000 7ffae2d9d640 1 -- 172.21.15.43:0/1167681390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fa7289a00 msgr2=0x556fa728be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.685+0000 7ffae2d9d640 1 --2- 172.21.15.43:0/1167681390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fa7289a00 0x556fa728be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.685+0000 7ffae2d9d640 1 -- 172.21.15.43:0/1167681390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fa71741b0 con 0x556fa72804d0 2024-04-21T08:49:32.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.685+0000 7ffad9d9b640 1 --2- 172.21.15.43:0/1217490876 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ffad00a3d50 0x7ffad00a6210 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ffacc006010 tx=0x7ffacc005a20 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.685+0000 7ffae2d9d640 1 --2- 172.21.15.43:0/1167681390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa72804d0 0x556fa72808b0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0049c0 tx=0x7ffadc009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17c1e6db17586fc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.685+0000 7ffad8d99640 1 -- 172.21.15.43:0/1167681390 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc003b70 con 0x556fa72804d0 2024-04-21T08:49:32.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.685+0000 7ffad8d99640 1 -- 172.21.15.43:0/1167681390 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffadc017070 con 0x556fa72804d0 2024-04-21T08:49:32.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.685+0000 7ffad8d99640 1 -- 172.21.15.43:0/1167681390 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc011820 con 0x556fa72804d0 2024-04-21T08:49:32.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.686+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/1167681390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa72804d0 msgr2=0x556fa72808b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.686+0000 7ffaeb9c3740 1 --2- 172.21.15.43:0/1167681390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa72804d0 0x556fa72808b0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0049c0 tx=0x7ffadc009a50 comp rx=0 tx=0).stop 2024-04-21T08:49:32.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.686+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/1167681390 shutdown_connections 2024-04-21T08:49:32.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.686+0000 7ffaeb9c3740 1 --2- 172.21.15.43:0/1167681390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fa7289a00 0x556fa728be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.686+0000 7ffaeb9c3740 1 --2- 172.21.15.43:0/1167681390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fa7280e80 0x556fa7289430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.686+0000 7ffaeb9c3740 1 --2- 172.21.15.43:0/1167681390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa72804d0 0x556fa72808b0 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.686+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/1167681390 >> 172.21.15.43:0/1167681390 conn(0x556fa727aba0 msgr2=0x556fa727af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.686+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/1167681390 shutdown_connections 2024-04-21T08:49:32.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.686+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/1167681390 wait complete. 2024-04-21T08:49:32.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.686+0000 7ffaeb9c3740 1 -- start start 2024-04-21T08:49:32.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.687+0000 7ffaeb9c3740 1 Processor -- start 2024-04-21T08:49:32.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.687+0000 7ffaeb9c3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fa72804d0 0x556fa7289300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.687+0000 7ffaeb9c3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fa7280e80 0x556fa7286690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.687+0000 7ffaeb9c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa7289a00 0x556fa7286c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.687+0000 7ffaeb9c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556fa728c970 con 0x556fa7289a00 2024-04-21T08:49:32.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.687+0000 7ffaeb9c3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556fa719e500 con 0x556fa7280e80 2024-04-21T08:49:32.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.687+0000 7ffaeb9c3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556fa728caf0 con 0x556fa72804d0 2024-04-21T08:49:32.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.687+0000 7ffaeb9c3740 10 client.0 _ll_register_callbacks cb 0x556fa716cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:32.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.687+0000 7ffae2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa7289a00 0x556fa7286c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.687+0000 7ffae2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa7289a00 0x556fa7286c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47640/0 (socket says 172.21.15.43:47640) 2024-04-21T08:49:32.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.687+0000 7ffae2d9d640 1 -- 172.21.15.43:0/1337575220 learned_addr learned my addr 172.21.15.43:0/1337575220 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:32.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.687+0000 7ffae259c640 1 --2- 172.21.15.43:0/1337575220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fa72804d0 0x556fa7289300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.687+0000 7ffad9d9b640 1 --2- 172.21.15.43:0/1337575220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fa7280e80 0x556fa7286690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.687+0000 7ffae2d9d640 1 -- 172.21.15.43:0/1337575220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fa72804d0 msgr2=0x556fa7289300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.687+0000 7ffae2d9d640 1 --2- 172.21.15.43:0/1337575220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fa72804d0 0x556fa7289300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.687+0000 7ffae2d9d640 1 -- 172.21.15.43:0/1337575220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fa7280e80 msgr2=0x556fa7286690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.687+0000 7ffae2d9d640 1 --2- 172.21.15.43:0/1337575220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fa7280e80 0x556fa7286690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.687+0000 7ffae2d9d640 1 -- 172.21.15.43:0/1337575220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fa6ef73f0 con 0x556fa7289a00 2024-04-21T08:49:32.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.687+0000 7ffae2d9d640 1 --2- 172.21.15.43:0/1337575220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa7289a00 0x556fa7286c10 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7ffadc005cc0 tx=0x7ffadc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee93c29493501db server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.687+0000 7ffac9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:32.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.688+0000 7ffac9ffb640 1 -- 172.21.15.43:0/1337575220 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc029060 con 0x556fa7289a00 2024-04-21T08:49:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.688+0000 7ffac9ffb640 1 -- 172.21.15.43:0/1337575220 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7ffadc004020 con 0x556fa7289a00 2024-04-21T08:49:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.688+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/1337575220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffadc017070 con 0x556fa7289a00 2024-04-21T08:49:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.688+0000 7ffac9ffb640 1 -- 172.21.15.43:0/1337575220 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffadc030050 con 0x556fa7289a00 2024-04-21T08:49:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.688+0000 7ffaeb9c3740 10 client.20580 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.688+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/1337575220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556fa6fc17d0 con 0x556fa7289a00 2024-04-21T08:49:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.688+0000 7ffac9ffb640 1 -- 172.21.15.43:0/1337575220 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffadc017070 con 0x556fa7289a00 2024-04-21T08:49:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.688+0000 7ffac9ffb640 1 -- 172.21.15.43:0/1337575220 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ffadc011d20 con 0x556fa7289a00 2024-04-21T08:49:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.688+0000 7ffaeb9c3740 10 client.20580 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.688+0000 7ffaeb9c3740 10 client.20580 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:49:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.688+0000 7ffaeb9c3740 20 client.20580 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.688+0000 7ffaeb9c3740 1 --2- 172.21.15.43:0/1337575220 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x556fa7299bf0 0x556fa729bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.688+0000 7ffaeb9c3740 4 client.20580 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.688+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/1337575220 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fa7184930 con 0x556fa7299bf0 2024-04-21T08:49:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.689+0000 7ffae259c640 1 --2- 172.21.15.43:0/1337575220 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x556fa7299bf0 0x556fa729bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.689+0000 7ffae259c640 1 --2- 172.21.15.43:0/1337575220 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x556fa7299bf0 0x556fa729bfe0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b009636b30993fb0 server_cookie=931de988d528ff0d in_seq=0 out_seq=0 2024-04-21T08:49:32.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.689+0000 7ffac9ffb640 10 client.20580 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:49:32.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.690+0000 7ffac9ffb640 1 -- 172.21.15.43:0/1337575220 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556fa7184930 con 0x556fa7299bf0 2024-04-21T08:49:32.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.690+0000 7ffac9ffb640 10 client.20580 handle_command_reply: tid=0 2024-04-21T08:49:32.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.690+0000 7ffaeb9c3740 1 client.20580 shutdown 2024-04-21T08:49:32.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.690+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/1337575220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa7289a00 msgr2=0x556fa7286c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.690+0000 7ffaeb9c3740 1 --2- 172.21.15.43:0/1337575220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa7289a00 0x556fa7286c10 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7ffadc005cc0 tx=0x7ffadc003ba0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.690+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/1337575220 shutdown_connections 2024-04-21T08:49:32.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.690+0000 7ffaeb9c3740 1 --2- 172.21.15.43:0/1337575220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa7289a00 0x556fa7286c10 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.690+0000 7ffaeb9c3740 1 --2- 172.21.15.43:0/1337575220 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x556fa7299bf0 0x556fa729bfe0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.690+0000 7ffaeb9c3740 1 --2- 172.21.15.43:0/1337575220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fa7280e80 0x556fa7286690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.690+0000 7ffaeb9c3740 1 --2- 172.21.15.43:0/1337575220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fa72804d0 0x556fa7289300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.690+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/1337575220 >> 172.21.15.43:0/1337575220 conn(0x556fa727aba0 msgr2=0x556fa727c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.690+0000 7ffac9ffb640 0 client.20580 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:49:32.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.690+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/1337575220 shutdown_connections 2024-04-21T08:49:32.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.690+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/1337575220 wait complete. 2024-04-21T08:49:32.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.690+0000 7ffaeb9c3740 20 client.20580 trim_cache size 0 max 16384 2024-04-21T08:49:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.703+0000 7ffaeb9c3740 1 Processor -- start 2024-04-21T08:49:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.703+0000 7ffaeb9c3740 1 -- start start 2024-04-21T08:49:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.703+0000 7ffaeb9c3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fa72804d0 0x556fa7295830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.703+0000 7ffaeb9c3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fa7280e80 0x556fa72857f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.703+0000 7ffaeb9c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa7289a00 0x556fa7279a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.703+0000 7ffaeb9c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffae41b1710 con 0x556fa7289a00 2024-04-21T08:49:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.703+0000 7ffaeb9c3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffae41b3250 con 0x556fa72804d0 2024-04-21T08:49:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.703+0000 7ffaeb9c3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffae41b1de0 con 0x556fa7280e80 2024-04-21T08:49:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.704+0000 7ffae2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa7289a00 0x556fa7279a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.704+0000 7ffae2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa7289a00 0x556fa7279a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47656/0 (socket says 172.21.15.43:47656) 2024-04-21T08:49:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.704+0000 7ffae2d9d640 1 -- 172.21.15.43:0/1099810636 learned_addr learned my addr 172.21.15.43:0/1099810636 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.704+0000 7ffae2d9d640 1 -- 172.21.15.43:0/1099810636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fa7280e80 msgr2=0x556fa72857f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.704+0000 7ffae2d9d640 1 --2- 172.21.15.43:0/1099810636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fa7280e80 0x556fa72857f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.704+0000 7ffae259c640 1 --2- 172.21.15.43:0/1099810636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fa72804d0 0x556fa7295830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.704+0000 7ffae2d9d640 1 -- 172.21.15.43:0/1099810636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fa72804d0 msgr2=0x556fa7295830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.704+0000 7ffae2d9d640 1 --2- 172.21.15.43:0/1099810636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fa72804d0 0x556fa7295830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.704+0000 7ffae2d9d640 1 -- 172.21.15.43:0/1099810636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fa72871f0 con 0x556fa7289a00 2024-04-21T08:49:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.704+0000 7ffae2d9d640 1 --2- 172.21.15.43:0/1099810636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa7289a00 0x556fa7279a80 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0134f0 tx=0x7ffadc009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3371b71012e4d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.704+0000 7ffacbfff640 1 -- 172.21.15.43:0/1099810636 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc029070 con 0x556fa7289a00 2024-04-21T08:49:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.704+0000 7ffacbfff640 1 -- 172.21.15.43:0/1099810636 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffadc017030 con 0x556fa7289a00 2024-04-21T08:49:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.704+0000 7ffacbfff640 1 -- 172.21.15.43:0/1099810636 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc01bdc0 con 0x556fa7289a00 2024-04-21T08:49:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.705+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/1099810636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa7289a00 msgr2=0x556fa7279a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.705+0000 7ffaeb9c3740 1 --2- 172.21.15.43:0/1099810636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa7289a00 0x556fa7279a80 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0134f0 tx=0x7ffadc009500 comp rx=0 tx=0).stop 2024-04-21T08:49:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.705+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/1099810636 shutdown_connections 2024-04-21T08:49:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.705+0000 7ffaeb9c3740 1 --2- 172.21.15.43:0/1099810636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa7289a00 0x556fa7279a80 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.705+0000 7ffaeb9c3740 1 --2- 172.21.15.43:0/1099810636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fa7280e80 0x556fa72857f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.705+0000 7ffaeb9c3740 1 --2- 172.21.15.43:0/1099810636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fa72804d0 0x556fa7295830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.705+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/1099810636 >> 172.21.15.43:0/1099810636 conn(0x556fa727aba0 msgr2=0x556fa7279370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.705+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/1099810636 shutdown_connections 2024-04-21T08:49:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.705+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/1099810636 wait complete. 2024-04-21T08:49:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.705+0000 7ffaeb9c3740 1 -- start start 2024-04-21T08:49:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.705+0000 7ffaeb9c3740 1 Processor -- start 2024-04-21T08:49:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.705+0000 7ffaeb9c3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fa72804d0 0x556fa7291d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.705+0000 7ffaeb9c3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fa7280e80 0x556fa7292280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.705+0000 7ffaeb9c3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa7289a00 0x556fa72829a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.705+0000 7ffaeb9c3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffae41b2b80 con 0x556fa7289a00 2024-04-21T08:49:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.705+0000 7ffaeb9c3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffae41b24b0 con 0x556fa7280e80 2024-04-21T08:49:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.705+0000 7ffaeb9c3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffae41af710 con 0x556fa72804d0 2024-04-21T08:49:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.705+0000 7ffaeb9c3740 10 client.0 _ll_register_callbacks cb 0x7ffadc009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.705+0000 7ffae2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa7289a00 0x556fa72829a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.706+0000 7ffae2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa7289a00 0x556fa72829a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47658/0 (socket says 172.21.15.43:47658) 2024-04-21T08:49:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.706+0000 7ffae2d9d640 1 -- 172.21.15.43:0/3509470557 learned_addr learned my addr 172.21.15.43:0/3509470557 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.706+0000 7ffad9d9b640 1 --2- 172.21.15.43:0/3509470557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fa7280e80 0x556fa7292280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.706+0000 7ffae259c640 1 --2- 172.21.15.43:0/3509470557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fa72804d0 0x556fa7291d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.706+0000 7ffae2d9d640 1 -- 172.21.15.43:0/3509470557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fa72804d0 msgr2=0x556fa7291d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.706+0000 7ffae2d9d640 1 --2- 172.21.15.43:0/3509470557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fa72804d0 0x556fa7291d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.706+0000 7ffae2d9d640 1 -- 172.21.15.43:0/3509470557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fa7280e80 msgr2=0x556fa7292280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.706+0000 7ffae2d9d640 1 --2- 172.21.15.43:0/3509470557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fa7280e80 0x556fa7292280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.706+0000 7ffae2d9d640 1 -- 172.21.15.43:0/3509470557 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fa7283090 con 0x556fa7289a00 2024-04-21T08:49:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.706+0000 7ffae2d9d640 1 --2- 172.21.15.43:0/3509470557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa7289a00 0x556fa72829a0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7ffadc003be0 tx=0x7ffadc009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f1157875ae9f8e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.706+0000 7ffac9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.706+0000 7ffac9ffb640 1 -- 172.21.15.43:0/3509470557 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc029040 con 0x556fa7289a00 2024-04-21T08:49:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.706+0000 7ffac9ffb640 1 -- 172.21.15.43:0/3509470557 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7ffadc01bc10 con 0x556fa7289a00 2024-04-21T08:49:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.706+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/3509470557 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffadc017030 con 0x556fa7289a00 2024-04-21T08:49:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.706+0000 7ffac9ffb640 1 -- 172.21.15.43:0/3509470557 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffadc01be50 con 0x556fa7289a00 2024-04-21T08:49:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.706+0000 7ffaeb9c3740 10 client.20586 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.706+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/3509470557 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556fa72ff790 con 0x556fa7289a00 2024-04-21T08:49:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.707+0000 7ffac9ffb640 1 -- 172.21.15.43:0/3509470557 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffadc017030 con 0x556fa7289a00 2024-04-21T08:49:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.707+0000 7ffac9ffb640 1 -- 172.21.15.43:0/3509470557 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ffadc011790 con 0x556fa7289a00 2024-04-21T08:49:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.707+0000 7ffaeb9c3740 10 client.20586 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.707+0000 7ffaeb9c3740 10 client.20586 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:49:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.707+0000 7ffaeb9c3740 20 client.20586 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.707+0000 7ffaeb9c3740 1 --2- 172.21.15.43:0/3509470557 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x556fa72ffc90 0x556fa73020b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.707+0000 7ffaeb9c3740 4 client.20586 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.707+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/3509470557 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556fa71f2c10 con 0x556fa72ffc90 2024-04-21T08:49:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.707+0000 7ffae259c640 1 --2- 172.21.15.43:0/3509470557 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x556fa72ffc90 0x556fa73020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.707+0000 7ffac9ffb640 10 client.20586 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:49:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.707+0000 7ffae259c640 1 --2- 172.21.15.43:0/3509470557 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x556fa72ffc90 0x556fa73020b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ce1342656ca8ef8 server_cookie=9fde68ab05dc739b in_seq=0 out_seq=0 2024-04-21T08:49:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.707+0000 7ffac9ffb640 1 -- 172.21.15.43:0/3509470557 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556fa71f2c10 con 0x556fa72ffc90 2024-04-21T08:49:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.707+0000 7ffac9ffb640 10 client.20586 handle_command_reply: tid=0 2024-04-21T08:49:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffaeb9c3740 1 client.20586 shutdown 2024-04-21T08:49:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/3509470557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa7289a00 msgr2=0x556fa72829a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffaeb9c3740 1 --2- 172.21.15.43:0/3509470557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa7289a00 0x556fa72829a0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7ffadc003be0 tx=0x7ffadc009a50 comp rx=0 tx=0).stop 2024-04-21T08:49:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/3509470557 shutdown_connections 2024-04-21T08:49:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffaeb9c3740 1 --2- 172.21.15.43:0/3509470557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fa7289a00 0x556fa72829a0 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffaeb9c3740 1 --2- 172.21.15.43:0/3509470557 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x556fa72ffc90 0x556fa73020b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffaeb9c3740 1 --2- 172.21.15.43:0/3509470557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556fa7280e80 0x556fa7292280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffaeb9c3740 1 --2- 172.21.15.43:0/3509470557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556fa72804d0 0x556fa7291d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/3509470557 >> 172.21.15.43:0/3509470557 conn(0x556fa727aba0 msgr2=0x556fa7279370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffac9ffb640 0 client.20586 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:49:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/3509470557 shutdown_connections 2024-04-21T08:49:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffaeb9c3740 1 -- 172.21.15.43:0/3509470557 wait complete. 2024-04-21T08:49:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffaeb9c3740 20 client.20586 trim_cache size 0 max 16384 2024-04-21T08:49:32.714 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:49:32.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffae8dce640 1 -- 172.21.15.43:0/1217490876 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ffad00a3d50 msgr2=0x7ffad00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffae8dce640 1 --2- 172.21.15.43:0/1217490876 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ffad00a3d50 0x7ffad00a6210 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ffacc006010 tx=0x7ffacc005a20 comp rx=0 tx=0).stop 2024-04-21T08:49:32.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffae8dce640 1 -- 172.21.15.43:0/1217490876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffae4100af0 msgr2=0x7ffae4101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffae8dce640 1 --2- 172.21.15.43:0/1217490876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffae4100af0 0x7ffae4101a60 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7ffad400c920 tx=0x7ffad400cde0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffae8dce640 1 -- 172.21.15.43:0/1217490876 shutdown_connections 2024-04-21T08:49:32.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffae8dce640 1 --2- 172.21.15.43:0/1217490876 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ffad00a3d50 0x7ffad00a6210 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffae8dce640 1 --2- 172.21.15.43:0/1217490876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffae41024e0 0x7ffae41a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffae8dce640 1 --2- 172.21.15.43:0/1217490876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffae41072b0 0x7ffae4101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffae8dce640 1 --2- 172.21.15.43:0/1217490876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffae4100af0 0x7ffae4101a60 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffae8dce640 1 -- 172.21.15.43:0/1217490876 >> 172.21.15.43:0/1217490876 conn(0x7ffae40760e0 msgr2=0x7ffae40feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffae8dce640 1 -- 172.21.15.43:0/1217490876 shutdown_connections 2024-04-21T08:49:32.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.708+0000 7ffae8dce640 1 -- 172.21.15.43:0/1217490876 wait complete. 2024-04-21T08:49:32.719 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:49:32.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.823+0000 7f80d345c640 1 -- 172.21.15.43:0/1432325638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80cc102b40 msgr2=0x7f80cc106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.823+0000 7f80d345c640 1 --2- 172.21.15.43:0/1432325638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80cc102b40 0x7f80cc106b80 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009900 tx=0x7f80c802bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:32.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.823+0000 7f80d345c640 1 -- 172.21.15.43:0/1432325638 shutdown_connections 2024-04-21T08:49:32.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.823+0000 7f80d345c640 1 --2- 172.21.15.43:0/1432325638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80cc107260 0x7f80cc10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.823+0000 7f80d345c640 1 --2- 172.21.15.43:0/1432325638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80cc102b40 0x7f80cc106b80 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.823+0000 7f80d345c640 1 --2- 172.21.15.43:0/1432325638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80cc102190 0x7f80cc102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.823+0000 7f80d345c640 1 -- 172.21.15.43:0/1432325638 >> 172.21.15.43:0/1432325638 conn(0x7f80cc0793a0 msgr2=0x7f80cc0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.823+0000 7f80d345c640 1 -- 172.21.15.43:0/1432325638 shutdown_connections 2024-04-21T08:49:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.823+0000 7f80d345c640 1 -- 172.21.15.43:0/1432325638 wait complete. 2024-04-21T08:49:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.824+0000 7f80d345c640 1 Processor -- start 2024-04-21T08:49:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.824+0000 7f80d345c640 1 -- start start 2024-04-21T08:49:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.824+0000 7f80d345c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80cc102190 0x7f80cc074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.824+0000 7f80d345c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80cc107260 0x7f80cc074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.824+0000 7f80d345c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80cc074cd0 0x7f80cc1a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.824+0000 7f80d345c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc1121a0 con 0x7f80cc074cd0 2024-04-21T08:49:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.824+0000 7f80d345c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc112020 con 0x7f80cc107260 2024-04-21T08:49:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.824+0000 7f80d345c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc112320 con 0x7f80cc102190 2024-04-21T08:49:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.824+0000 7f80d19f9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80cc074cd0 0x7f80cc1a69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.824+0000 7f80d19f9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80cc074cd0 0x7f80cc1a69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47670/0 (socket says 172.21.15.43:47670) 2024-04-21T08:49:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.824+0000 7f80d19f9640 1 -- 172.21.15.43:0/1980797182 learned_addr learned my addr 172.21.15.43:0/1980797182 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:32.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.824+0000 7f80d09f7640 1 --2- 172.21.15.43:0/1980797182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80cc107260 0x7f80cc074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.824+0000 7f80d11f8640 1 --2- 172.21.15.43:0/1980797182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80cc102190 0x7f80cc074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.825+0000 7f80d19f9640 1 -- 172.21.15.43:0/1980797182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80cc102190 msgr2=0x7f80cc074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.825+0000 7f80d19f9640 1 --2- 172.21.15.43:0/1980797182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80cc102190 0x7f80cc074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.825+0000 7f80d19f9640 1 -- 172.21.15.43:0/1980797182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80cc107260 msgr2=0x7f80cc074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.825+0000 7f80d19f9640 1 --2- 172.21.15.43:0/1980797182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80cc107260 0x7f80cc074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.825+0000 7f80d19f9640 1 -- 172.21.15.43:0/1980797182 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80cc1a7070 con 0x7f80cc074cd0 2024-04-21T08:49:32.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.825+0000 7f80d11f8640 1 --2- 172.21.15.43:0/1980797182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80cc102190 0x7f80cc074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:32.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.825+0000 7f80d19f9640 1 --2- 172.21.15.43:0/1980797182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80cc074cd0 0x7f80cc1a69d0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f80c400eb10 tx=0x7f80c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.825+0000 7f80be7fc640 1 -- 172.21.15.43:0/1980797182 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4004b20 con 0x7f80cc074cd0 2024-04-21T08:49:32.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.825+0000 7f80d345c640 1 -- 172.21.15.43:0/1980797182 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80cc1a7360 con 0x7f80cc074cd0 2024-04-21T08:49:32.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.825+0000 7f80be7fc640 1 -- 172.21.15.43:0/1980797182 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80c4004cc0 con 0x7f80cc074cd0 2024-04-21T08:49:32.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.825+0000 7f80be7fc640 1 -- 172.21.15.43:0/1980797182 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4010670 con 0x7f80cc074cd0 2024-04-21T08:49:32.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.825+0000 7f80d345c640 1 -- 172.21.15.43:0/1980797182 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80cc1a7880 con 0x7f80cc074cd0 2024-04-21T08:49:32.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.828+0000 7f80be7fc640 1 -- 172.21.15.43:0/1980797182 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f80c4005020 con 0x7f80cc074cd0 2024-04-21T08:49:32.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.828+0000 7f80d6052740 1 Processor -- start 2024-04-21T08:49:32.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.828+0000 7f80d6052740 1 -- start start 2024-04-21T08:49:32.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.828+0000 7f80d6052740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570f59064d0 0x5570f59068b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.828+0000 7f80d6052740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f5906e80 0x5570f590f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.828+0000 7f80d6052740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570f590fa00 0x5570f5911e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.828+0000 7f80d6052740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5570f586b1f0 con 0x5570f5906e80 2024-04-21T08:49:32.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.828+0000 7f80d6052740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5570f58084b0 con 0x5570f59064d0 2024-04-21T08:49:32.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.828+0000 7f80d6052740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5570f58671b0 con 0x5570f590fa00 2024-04-21T08:49:32.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.829+0000 7f80be7fc640 1 --2- 172.21.15.43:0/1980797182 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f80b80a3d20 0x7f80b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.829+0000 7f80d11f8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f5906e80 0x5570f590f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.829+0000 7f80d11f8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f5906e80 0x5570f590f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47680/0 (socket says 172.21.15.43:47680) 2024-04-21T08:49:32.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.829+0000 7f80d11f8640 1 -- 172.21.15.43:0/1785158719 learned_addr learned my addr 172.21.15.43:0/1785158719 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:32.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.829+0000 7f80d19f9640 1 --2- 172.21.15.43:0/1980797182 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f80b80a3d20 0x7f80b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.829+0000 7f80d09f7640 1 --2- 172.21.15.43:0/1785158719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570f59064d0 0x5570f59068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.829+0000 7f80d09f7640 1 --2- 172.21.15.43:0/1785158719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570f590fa00 0x5570f5911e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.829+0000 7f80be7fc640 1 -- 172.21.15.43:0/1980797182 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f80c40d11d0 con 0x7f80cc074cd0 2024-04-21T08:49:32.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.829+0000 7f80d11f8640 1 -- 172.21.15.43:0/1785158719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570f590fa00 msgr2=0x5570f5911e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.829+0000 7f80d11f8640 1 --2- 172.21.15.43:0/1785158719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570f590fa00 0x5570f5911e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.829+0000 7f80d11f8640 1 -- 172.21.15.43:0/1785158719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570f59064d0 msgr2=0x5570f59068b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.829+0000 7f80d11f8640 1 --2- 172.21.15.43:0/1785158719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570f59064d0 0x5570f59068b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.829+0000 7f80d11f8640 1 -- 172.21.15.43:0/1785158719 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570f57fa1b0 con 0x5570f5906e80 2024-04-21T08:49:32.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.829+0000 7f80d11f8640 1 --2- 172.21.15.43:0/1785158719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f5906e80 0x5570f590f430 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f80c000bb70 tx=0x7f80c000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51afdbc21a4b2677 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.829+0000 7f80d19f9640 1 --2- 172.21.15.43:0/1980797182 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f80b80a3d20 0x7f80b80a61e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f80c40ce5b0 tx=0x7f80c40bd040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.829+0000 7f80b77fe640 1 -- 172.21.15.43:0/1785158719 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c0017620 con 0x5570f5906e80 2024-04-21T08:49:32.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.829+0000 7f80b77fe640 1 -- 172.21.15.43:0/1785158719 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80c0017c70 con 0x5570f5906e80 2024-04-21T08:49:32.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.829+0000 7f80b77fe640 1 -- 172.21.15.43:0/1785158719 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c000fb40 con 0x5570f5906e80 2024-04-21T08:49:32.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.830+0000 7f80d6052740 1 -- 172.21.15.43:0/1785158719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f5906e80 msgr2=0x5570f590f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.830+0000 7f80d6052740 1 --2- 172.21.15.43:0/1785158719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f5906e80 0x5570f590f430 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f80c000bb70 tx=0x7f80c000d710 comp rx=0 tx=0).stop 2024-04-21T08:49:32.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.830+0000 7f80d6052740 1 -- 172.21.15.43:0/1785158719 shutdown_connections 2024-04-21T08:49:32.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.830+0000 7f80d6052740 1 --2- 172.21.15.43:0/1785158719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570f590fa00 0x5570f5911e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.830+0000 7f80d6052740 1 --2- 172.21.15.43:0/1785158719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f5906e80 0x5570f590f430 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.830+0000 7f80d6052740 1 --2- 172.21.15.43:0/1785158719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570f59064d0 0x5570f59068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.830+0000 7f80d6052740 1 -- 172.21.15.43:0/1785158719 >> 172.21.15.43:0/1785158719 conn(0x5570f5900ba0 msgr2=0x5570f5900f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.830+0000 7f80d6052740 1 -- 172.21.15.43:0/1785158719 shutdown_connections 2024-04-21T08:49:32.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.830+0000 7f80d6052740 1 -- 172.21.15.43:0/1785158719 wait complete. 2024-04-21T08:49:32.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.830+0000 7f80d6052740 1 -- start start 2024-04-21T08:49:32.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.830+0000 7f80d6052740 1 Processor -- start 2024-04-21T08:49:32.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.831+0000 7f80d6052740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f59064d0 0x5570f590f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.831+0000 7f80d6052740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570f5906e80 0x5570f590c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.831+0000 7f80d6052740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570f590fa00 0x5570f590cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.831+0000 7f80d6052740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5570f5912970 con 0x5570f59064d0 2024-04-21T08:49:32.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.831+0000 7f80d6052740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5570f5824500 con 0x5570f590fa00 2024-04-21T08:49:32.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.831+0000 7f80d6052740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5570f5912af0 con 0x5570f5906e80 2024-04-21T08:49:32.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.831+0000 7f80d6052740 10 client.0 _ll_register_callbacks cb 0x5570f57f2b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:32.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.831+0000 7f80d09f7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f59064d0 0x5570f590f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.831+0000 7f80d09f7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f59064d0 0x5570f590f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47682/0 (socket says 172.21.15.43:47682) 2024-04-21T08:49:32.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.831+0000 7f80d09f7640 1 -- 172.21.15.43:0/1658827653 learned_addr learned my addr 172.21.15.43:0/1658827653 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:32.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.831+0000 7f80d09f7640 1 --2- 172.21.15.43:0/1658827653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570f590fa00 0x5570f590cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.831+0000 7f80d11f8640 1 --2- 172.21.15.43:0/1658827653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570f5906e80 0x5570f590c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.831+0000 7f80d09f7640 1 -- 172.21.15.43:0/1658827653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570f5906e80 msgr2=0x5570f590c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.831+0000 7f80d09f7640 1 --2- 172.21.15.43:0/1658827653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570f5906e80 0x5570f590c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.831+0000 7f80d09f7640 1 -- 172.21.15.43:0/1658827653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570f590fa00 msgr2=0x5570f590cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.831+0000 7f80d09f7640 1 --2- 172.21.15.43:0/1658827653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570f590fa00 0x5570f590cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.831+0000 7f80d09f7640 1 -- 172.21.15.43:0/1658827653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570f557d3f0 con 0x5570f59064d0 2024-04-21T08:49:32.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.831+0000 7f80d11f8640 1 --2- 172.21.15.43:0/1658827653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570f5906e80 0x5570f590c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:32.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.832+0000 7f80d09f7640 1 --2- 172.21.15.43:0/1658827653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f59064d0 0x5570f590f320 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f80cc06b820 tx=0x7f80c802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3205b50b629806 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.832+0000 7f80b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:32.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.832+0000 7f80b4ff9640 1 -- 172.21.15.43:0/1658827653 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c8005350 con 0x5570f59064d0 2024-04-21T08:49:32.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.832+0000 7f80b4ff9640 1 -- 172.21.15.43:0/1658827653 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f80c80054f0 con 0x5570f59064d0 2024-04-21T08:49:32.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.832+0000 7f80d6052740 1 -- 172.21.15.43:0/1658827653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80c0017c70 con 0x5570f59064d0 2024-04-21T08:49:32.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.832+0000 7f80b4ff9640 1 -- 172.21.15.43:0/1658827653 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f80c8055050 con 0x5570f59064d0 2024-04-21T08:49:32.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.832+0000 7f80d6052740 10 client.20598 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:32.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.832+0000 7f80d6052740 1 -- 172.21.15.43:0/1658827653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5570f56477d0 con 0x5570f59064d0 2024-04-21T08:49:32.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.832+0000 7f80b4ff9640 1 -- 172.21.15.43:0/1658827653 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80c0017c70 con 0x5570f59064d0 2024-04-21T08:49:32.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.832+0000 7f80b4ff9640 1 -- 172.21.15.43:0/1658827653 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f80c80039e0 con 0x5570f59064d0 2024-04-21T08:49:32.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.832+0000 7f80d6052740 10 client.20598 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:32.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.832+0000 7f80d6052740 10 client.20598 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:49:32.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.832+0000 7f80d6052740 20 client.20598 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:32.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.832+0000 7f80d6052740 1 --2- 172.21.15.43:0/1658827653 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5570f591fb60 0x5570f5921f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.832+0000 7f80d6052740 4 client.20598 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:32.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.832+0000 7f80d6052740 1 -- 172.21.15.43:0/1658827653 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570f580a930 con 0x5570f591fb60 2024-04-21T08:49:32.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.833+0000 7f80d11f8640 1 --2- 172.21.15.43:0/1658827653 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5570f591fb60 0x5570f5921f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.833+0000 7f80d11f8640 1 --2- 172.21.15.43:0/1658827653 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5570f591fb60 0x5570f5921f90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d6fdcb380476f4c server_cookie=edd2060c37d1a98c in_seq=0 out_seq=0 2024-04-21T08:49:32.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.833+0000 7f80b4ff9640 10 client.20598 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:49:32.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.834+0000 7f80b4ff9640 1 -- 172.21.15.43:0/1658827653 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5570f580a930 con 0x5570f591fb60 2024-04-21T08:49:32.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.834+0000 7f80b4ff9640 10 client.20598 handle_command_reply: tid=0 2024-04-21T08:49:32.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.834+0000 7f80d6052740 1 client.20598 shutdown 2024-04-21T08:49:32.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.834+0000 7f80d6052740 1 -- 172.21.15.43:0/1658827653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f59064d0 msgr2=0x5570f590f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.834+0000 7f80d6052740 1 --2- 172.21.15.43:0/1658827653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f59064d0 0x5570f590f320 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f80cc06b820 tx=0x7f80c802bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:32.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.834+0000 7f80d6052740 1 -- 172.21.15.43:0/1658827653 shutdown_connections 2024-04-21T08:49:32.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.834+0000 7f80d6052740 1 --2- 172.21.15.43:0/1658827653 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5570f591fb60 0x5570f5921f90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.834+0000 7f80d6052740 1 --2- 172.21.15.43:0/1658827653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570f590fa00 0x5570f590cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.834+0000 7f80b4ff9640 0 client.20598 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:49:32.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.834+0000 7f80d6052740 1 --2- 172.21.15.43:0/1658827653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570f5906e80 0x5570f590c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.834+0000 7f80d6052740 1 --2- 172.21.15.43:0/1658827653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f59064d0 0x5570f590f320 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.834+0000 7f80d6052740 1 -- 172.21.15.43:0/1658827653 >> 172.21.15.43:0/1658827653 conn(0x5570f5900ba0 msgr2=0x5570f59028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.834+0000 7f80d6052740 1 -- 172.21.15.43:0/1658827653 shutdown_connections 2024-04-21T08:49:32.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.834+0000 7f80d6052740 1 -- 172.21.15.43:0/1658827653 wait complete. 2024-04-21T08:49:32.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.834+0000 7f80d6052740 20 client.20598 trim_cache size 0 max 16384 2024-04-21T08:49:32.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.848+0000 7f80d6052740 1 Processor -- start 2024-04-21T08:49:32.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.848+0000 7f80d6052740 1 -- start start 2024-04-21T08:49:32.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.849+0000 7f80d6052740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570f59064d0 0x5570f591b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.849+0000 7f80d6052740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570f5906e80 0x5570f590ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.849+0000 7f80d6052740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f590fa00 0x5570f590f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.849+0000 7f80d6052740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc1b1760 con 0x5570f590fa00 2024-04-21T08:49:32.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.849+0000 7f80d6052740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc1b32a0 con 0x5570f59064d0 2024-04-21T08:49:32.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.849+0000 7f80d6052740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc1b1e30 con 0x5570f5906e80 2024-04-21T08:49:32.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.849+0000 7f80d09f7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f590fa00 0x5570f590f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.849+0000 7f80d09f7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f590fa00 0x5570f590f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47690/0 (socket says 172.21.15.43:47690) 2024-04-21T08:49:32.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.849+0000 7f80d09f7640 1 -- 172.21.15.43:0/4127159762 learned_addr learned my addr 172.21.15.43:0/4127159762 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:32.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.849+0000 7f80d09f7640 1 --2- 172.21.15.43:0/4127159762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570f59064d0 0x5570f591b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.849+0000 7f80d11f8640 1 --2- 172.21.15.43:0/4127159762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570f5906e80 0x5570f590ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.849+0000 7f80d09f7640 1 -- 172.21.15.43:0/4127159762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570f5906e80 msgr2=0x5570f590ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.849+0000 7f80d09f7640 1 --2- 172.21.15.43:0/4127159762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570f5906e80 0x5570f590ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.849+0000 7f80d09f7640 1 -- 172.21.15.43:0/4127159762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570f59064d0 msgr2=0x5570f591b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.849+0000 7f80d09f7640 1 --2- 172.21.15.43:0/4127159762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570f59064d0 0x5570f591b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.849+0000 7f80d09f7640 1 -- 172.21.15.43:0/4127159762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570f5905790 con 0x5570f590fa00 2024-04-21T08:49:32.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.849+0000 7f80d11f8640 1 --2- 172.21.15.43:0/4127159762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570f5906e80 0x5570f590ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:32.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.849+0000 7f80d09f7640 1 --2- 172.21.15.43:0/4127159762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f590fa00 0x5570f590f320 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f80c802c170 tx=0x7f80c8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b3760ecf4ed57 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.849+0000 7f80b6ffd640 1 -- 172.21.15.43:0/4127159762 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c8047030 con 0x5570f590fa00 2024-04-21T08:49:32.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.850+0000 7f80b6ffd640 1 -- 172.21.15.43:0/4127159762 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80c8005850 con 0x5570f590fa00 2024-04-21T08:49:32.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.850+0000 7f80b6ffd640 1 -- 172.21.15.43:0/4127159762 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c8000e60 con 0x5570f590fa00 2024-04-21T08:49:32.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.850+0000 7f80d6052740 1 -- 172.21.15.43:0/4127159762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f590fa00 msgr2=0x5570f590f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.850+0000 7f80d6052740 1 --2- 172.21.15.43:0/4127159762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f590fa00 0x5570f590f320 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f80c802c170 tx=0x7f80c8005fd0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.850+0000 7f80d6052740 1 -- 172.21.15.43:0/4127159762 shutdown_connections 2024-04-21T08:49:32.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.850+0000 7f80d6052740 1 --2- 172.21.15.43:0/4127159762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f590fa00 0x5570f590f320 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.850+0000 7f80d6052740 1 --2- 172.21.15.43:0/4127159762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570f5906e80 0x5570f590ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.850+0000 7f80d6052740 1 --2- 172.21.15.43:0/4127159762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570f59064d0 0x5570f591b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.850+0000 7f80d6052740 1 -- 172.21.15.43:0/4127159762 >> 172.21.15.43:0/4127159762 conn(0x5570f5900ba0 msgr2=0x5570f58ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.850+0000 7f80d6052740 1 -- 172.21.15.43:0/4127159762 shutdown_connections 2024-04-21T08:49:32.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.850+0000 7f80d6052740 1 -- 172.21.15.43:0/4127159762 wait complete. 2024-04-21T08:49:32.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.850+0000 7f80d6052740 1 -- start start 2024-04-21T08:49:32.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.851+0000 7f80d6052740 1 Processor -- start 2024-04-21T08:49:32.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.851+0000 7f80d6052740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570f59064d0 0x5570f59092e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.851+0000 7f80d6052740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f5906e80 0x5570f5909820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.851+0000 7f80d6052740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570f590fa00 0x5570f5909d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.851+0000 7f80d6052740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc1b2bd0 con 0x5570f5906e80 2024-04-21T08:49:32.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.851+0000 7f80d6052740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc1b2500 con 0x5570f590fa00 2024-04-21T08:49:32.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.851+0000 7f80d6052740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc1af760 con 0x5570f59064d0 2024-04-21T08:49:32.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.851+0000 7f80d6052740 10 client.0 _ll_register_callbacks cb 0x7f80c802c9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:32.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.851+0000 7f80d11f8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f5906e80 0x5570f5909820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.851+0000 7f80d11f8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f5906e80 0x5570f5909820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47696/0 (socket says 172.21.15.43:47696) 2024-04-21T08:49:32.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.851+0000 7f80d11f8640 1 -- 172.21.15.43:0/658087872 learned_addr learned my addr 172.21.15.43:0/658087872 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:32.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.851+0000 7f80d09f7640 1 --2- 172.21.15.43:0/658087872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570f590fa00 0x5570f5909d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.851+0000 7f80d09f7640 1 --2- 172.21.15.43:0/658087872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570f59064d0 0x5570f59092e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.851+0000 7f80d11f8640 1 -- 172.21.15.43:0/658087872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570f59064d0 msgr2=0x5570f59092e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.851+0000 7f80d11f8640 1 --2- 172.21.15.43:0/658087872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570f59064d0 0x5570f59092e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.851+0000 7f80d11f8640 1 -- 172.21.15.43:0/658087872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570f590fa00 msgr2=0x5570f5909d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.851+0000 7f80d11f8640 1 --2- 172.21.15.43:0/658087872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570f590fa00 0x5570f5909d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.851+0000 7f80d11f8640 1 -- 172.21.15.43:0/658087872 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570f59850d0 con 0x5570f5906e80 2024-04-21T08:49:32.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.851+0000 7f80d09f7640 1 --2- 172.21.15.43:0/658087872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570f59064d0 0x5570f59092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:32.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.851+0000 7f80d11f8640 1 --2- 172.21.15.43:0/658087872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f5906e80 0x5570f5909820 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f80c00058c0 tx=0x7f80c0005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31129a1bb43d5415 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.851+0000 7f80b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:32.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.852+0000 7f80b4ff9640 1 -- 172.21.15.43:0/658087872 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c002c020 con 0x5570f5906e80 2024-04-21T08:49:32.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.852+0000 7f80b4ff9640 1 -- 172.21.15.43:0/658087872 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f80c0028bc0 con 0x5570f5906e80 2024-04-21T08:49:32.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.852+0000 7f80d6052740 1 -- 172.21.15.43:0/658087872 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80c8005850 con 0x5570f5906e80 2024-04-21T08:49:32.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.852+0000 7f80b4ff9640 1 -- 172.21.15.43:0/658087872 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f80c0037050 con 0x5570f5906e80 2024-04-21T08:49:32.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.852+0000 7f80d6052740 10 client.20604 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:32.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.852+0000 7f80d6052740 1 -- 172.21.15.43:0/658087872 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5570f59862c0 con 0x5570f5906e80 2024-04-21T08:49:32.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.852+0000 7f80b4ff9640 1 -- 172.21.15.43:0/658087872 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80c8005850 con 0x5570f5906e80 2024-04-21T08:49:32.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.852+0000 7f80b4ff9640 1 -- 172.21.15.43:0/658087872 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f80c000f420 con 0x5570f5906e80 2024-04-21T08:49:32.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.852+0000 7f80d6052740 10 client.20604 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:32.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.852+0000 7f80d6052740 10 client.20604 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:49:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.852+0000 7f80d6052740 20 client.20604 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.852+0000 7f80d6052740 1 --2- 172.21.15.43:0/658087872 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5570f5986760 0x5570f5988b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.852+0000 7f80d6052740 4 client.20604 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.852+0000 7f80d6052740 1 -- 172.21.15.43:0/658087872 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570f5878c10 con 0x5570f5986760 2024-04-21T08:49:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.853+0000 7f80d09f7640 1 --2- 172.21.15.43:0/658087872 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5570f5986760 0x5570f5988b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.853+0000 7f80b4ff9640 10 client.20604 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:49:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.853+0000 7f80d09f7640 1 --2- 172.21.15.43:0/658087872 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5570f5986760 0x5570f5988b50 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f33faa10e471a790 server_cookie=d502ecd6120ff36 in_seq=0 out_seq=0 2024-04-21T08:49:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.853+0000 7f80b4ff9640 1 -- 172.21.15.43:0/658087872 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570f5878c10 con 0x5570f5986760 2024-04-21T08:49:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.853+0000 7f80b4ff9640 10 client.20604 handle_command_reply: tid=0 2024-04-21T08:49:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.853+0000 7f80d6052740 1 client.20604 shutdown 2024-04-21T08:49:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.853+0000 7f80d6052740 1 -- 172.21.15.43:0/658087872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f5906e80 msgr2=0x5570f5909820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.853+0000 7f80d6052740 1 --2- 172.21.15.43:0/658087872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f5906e80 0x5570f5909820 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f80c00058c0 tx=0x7f80c0005bf0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.853+0000 7f80d6052740 1 -- 172.21.15.43:0/658087872 shutdown_connections 2024-04-21T08:49:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.853+0000 7f80d6052740 1 --2- 172.21.15.43:0/658087872 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x5570f5986760 0x5570f5988b50 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.853+0000 7f80d6052740 1 --2- 172.21.15.43:0/658087872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5570f590fa00 0x5570f5909d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.853+0000 7f80d6052740 1 --2- 172.21.15.43:0/658087872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570f5906e80 0x5570f5909820 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.853+0000 7f80b4ff9640 0 client.20604 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:49:32.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.853+0000 7f80d6052740 1 --2- 172.21.15.43:0/658087872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570f59064d0 0x5570f59092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.853+0000 7f80d6052740 1 -- 172.21.15.43:0/658087872 >> 172.21.15.43:0/658087872 conn(0x5570f5900ba0 msgr2=0x5570f5901f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.853+0000 7f80d6052740 1 -- 172.21.15.43:0/658087872 shutdown_connections 2024-04-21T08:49:32.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.853+0000 7f80d6052740 1 -- 172.21.15.43:0/658087872 wait complete. 2024-04-21T08:49:32.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.854+0000 7f80d6052740 20 client.20604 trim_cache size 0 max 16384 2024-04-21T08:49:32.860 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:49:32.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.854+0000 7f80d345c640 1 -- 172.21.15.43:0/1980797182 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f80b80a3d20 msgr2=0x7f80b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.854+0000 7f80d345c640 1 --2- 172.21.15.43:0/1980797182 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f80b80a3d20 0x7f80b80a61e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f80c40ce5b0 tx=0x7f80c40bd040 comp rx=0 tx=0).stop 2024-04-21T08:49:32.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.854+0000 7f80d345c640 1 -- 172.21.15.43:0/1980797182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80cc074cd0 msgr2=0x7f80cc1a69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.854+0000 7f80d345c640 1 --2- 172.21.15.43:0/1980797182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80cc074cd0 0x7f80cc1a69d0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f80c400eb10 tx=0x7f80c400c2d0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.854+0000 7f80d345c640 1 -- 172.21.15.43:0/1980797182 shutdown_connections 2024-04-21T08:49:32.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.854+0000 7f80d345c640 1 --2- 172.21.15.43:0/1980797182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80cc074cd0 0x7f80cc1a69d0 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.854+0000 7f80d345c640 1 --2- 172.21.15.43:0/1980797182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80cc107260 0x7f80cc074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.854+0000 7f80d345c640 1 --2- 172.21.15.43:0/1980797182 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f80b80a3d20 0x7f80b80a61e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.854+0000 7f80d345c640 1 --2- 172.21.15.43:0/1980797182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80cc102190 0x7f80cc074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.854+0000 7f80d345c640 1 -- 172.21.15.43:0/1980797182 >> 172.21.15.43:0/1980797182 conn(0x7f80cc0793a0 msgr2=0x7f80cc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.854+0000 7f80d345c640 1 -- 172.21.15.43:0/1980797182 shutdown_connections 2024-04-21T08:49:32.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.854+0000 7f80d345c640 1 -- 172.21.15.43:0/1980797182 wait complete. 2024-04-21T08:49:32.864 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:49:32.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.969+0000 7f704ed9d640 1 -- 172.21.15.43:0/2524154139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7048103af0 msgr2=0x7f704810bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.969+0000 7f704ed9d640 1 --2- 172.21.15.43:0/2524154139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7048103af0 0x7f704810bfd0 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f7034009900 tx=0x7f703402bd60 comp rx=0 tx=0).stop 2024-04-21T08:49:32.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.969+0000 7f704ed9d640 1 -- 172.21.15.43:0/2524154139 shutdown_connections 2024-04-21T08:49:32.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.969+0000 7f704ed9d640 1 --2- 172.21.15.43:0/2524154139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f704810c510 0x7f704810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.969+0000 7f704ed9d640 1 --2- 172.21.15.43:0/2524154139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7048103af0 0x7f704810bfd0 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.969+0000 7f704ed9d640 1 --2- 172.21.15.43:0/2524154139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70481031d0 0x7f70481035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.969+0000 7f704ed9d640 1 -- 172.21.15.43:0/2524154139 >> 172.21.15.43:0/2524154139 conn(0x7f7048077f50 msgr2=0x7f7048078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.969+0000 7f704ed9d640 1 -- 172.21.15.43:0/2524154139 shutdown_connections 2024-04-21T08:49:32.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.969+0000 7f704ed9d640 1 -- 172.21.15.43:0/2524154139 wait complete. 2024-04-21T08:49:32.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.970+0000 7f704ed9d640 1 Processor -- start 2024-04-21T08:49:32.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.970+0000 7f704ed9d640 1 -- start start 2024-04-21T08:49:32.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.970+0000 7f704ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70481031d0 0x7f704819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.970+0000 7f704ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7048103af0 0x7f704819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.970+0000 7f704ed9d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f704810c510 0x7f7048198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.970+0000 7f704ed9d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7048112180 con 0x7f7048103af0 2024-04-21T08:49:32.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.970+0000 7f704ed9d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7048112000 con 0x7f70481031d0 2024-04-21T08:49:32.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.970+0000 7f704ed9d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7048112300 con 0x7f704810c510 2024-04-21T08:49:32.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.970+0000 7f7047fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7048103af0 0x7f704819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.970+0000 7f7047fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7048103af0 0x7f704819e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47708/0 (socket says 172.21.15.43:47708) 2024-04-21T08:49:32.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.970+0000 7f7047fff640 1 -- 172.21.15.43:0/2008734208 learned_addr learned my addr 172.21.15.43:0/2008734208 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:32.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.971+0000 7f704cb39640 1 --2- 172.21.15.43:0/2008734208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70481031d0 0x7f704819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.971+0000 7f704d33a640 1 --2- 172.21.15.43:0/2008734208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f704810c510 0x7f7048198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.971+0000 7f7047fff640 1 -- 172.21.15.43:0/2008734208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f704810c510 msgr2=0x7f7048198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.971+0000 7f7047fff640 1 --2- 172.21.15.43:0/2008734208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f704810c510 0x7f7048198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.971+0000 7f7047fff640 1 -- 172.21.15.43:0/2008734208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70481031d0 msgr2=0x7f704819e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.971+0000 7f7047fff640 1 --2- 172.21.15.43:0/2008734208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70481031d0 0x7f704819e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.971+0000 7f7047fff640 1 -- 172.21.15.43:0/2008734208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7048198d10 con 0x7f7048103af0 2024-04-21T08:49:32.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.971+0000 7f704cb39640 1 --2- 172.21.15.43:0/2008734208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70481031d0 0x7f704819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:32.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.971+0000 7f7047fff640 1 --2- 172.21.15.43:0/2008734208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7048103af0 0x7f704819e920 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f7034005940 tx=0x7f703402d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.971+0000 7f7045ffb640 1 -- 172.21.15.43:0/2008734208 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7034043020 con 0x7f7048103af0 2024-04-21T08:49:32.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.971+0000 7f7045ffb640 1 -- 172.21.15.43:0/2008734208 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f703402dd20 con 0x7f7048103af0 2024-04-21T08:49:32.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.971+0000 7f704ed9d640 1 -- 172.21.15.43:0/2008734208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7048198fa0 con 0x7f7048103af0 2024-04-21T08:49:32.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.971+0000 7f7045ffb640 1 -- 172.21.15.43:0/2008734208 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7034003b80 con 0x7f7048103af0 2024-04-21T08:49:32.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.971+0000 7f704ed9d640 1 -- 172.21.15.43:0/2008734208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70481a52a0 con 0x7f7048103af0 2024-04-21T08:49:32.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.973+0000 7f7045ffb640 1 -- 172.21.15.43:0/2008734208 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7034046020 con 0x7f7048103af0 2024-04-21T08:49:32.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.974+0000 7f705198f740 1 Processor -- start 2024-04-21T08:49:32.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.974+0000 7f705198f740 1 -- start start 2024-04-21T08:49:32.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.974+0000 7f705198f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56006901a4d0 0x56006901a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.974+0000 7f705198f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56006901ae80 0x560069023430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.974+0000 7f705198f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560069023a00 0x560069025e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.974+0000 7f705198f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560068f7f1f0 con 0x56006901ae80 2024-04-21T08:49:32.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.974+0000 7f705198f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560068f1c4b0 con 0x56006901a4d0 2024-04-21T08:49:32.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.974+0000 7f705198f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560068f7b1b0 con 0x560069023a00 2024-04-21T08:49:32.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.974+0000 7f7045ffb640 1 --2- 172.21.15.43:0/2008734208 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f70380a3da0 0x7f70380a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.974+0000 7f704cb39640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56006901ae80 0x560069023430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.974+0000 7f704cb39640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56006901ae80 0x560069023430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47722/0 (socket says 172.21.15.43:47722) 2024-04-21T08:49:32.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.974+0000 7f704cb39640 1 -- 172.21.15.43:0/4178279238 learned_addr learned my addr 172.21.15.43:0/4178279238 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:32.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.974+0000 7f704d33a640 1 --2- 172.21.15.43:0/4178279238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56006901a4d0 0x56006901a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.974+0000 7f704d33a640 1 --2- 172.21.15.43:0/2008734208 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f70380a3da0 0x7f70380a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.974+0000 7f7045ffb640 1 -- 172.21.15.43:0/2008734208 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f70340f2da0 con 0x7f7048103af0 2024-04-21T08:49:32.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.974+0000 7f7047fff640 1 --2- 172.21.15.43:0/4178279238 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560069023a00 0x560069025e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.974+0000 7f704cb39640 1 -- 172.21.15.43:0/4178279238 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560069023a00 msgr2=0x560069025e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.974+0000 7f704cb39640 1 --2- 172.21.15.43:0/4178279238 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560069023a00 0x560069025e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.974+0000 7f704cb39640 1 -- 172.21.15.43:0/4178279238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56006901a4d0 msgr2=0x56006901a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.974+0000 7f704cb39640 1 --2- 172.21.15.43:0/4178279238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56006901a4d0 0x56006901a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.974+0000 7f704cb39640 1 -- 172.21.15.43:0/4178279238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560068f0e1b0 con 0x56006901ae80 2024-04-21T08:49:32.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.974+0000 7f704d33a640 1 --2- 172.21.15.43:0/4178279238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56006901a4d0 0x56006901a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:32.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.975+0000 7f704cb39640 1 --2- 172.21.15.43:0/4178279238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56006901ae80 0x560069023430 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f703c00bd40 tx=0x7f703c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5533ad5092a1a8c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.975+0000 7f704d33a640 1 --2- 172.21.15.43:0/2008734208 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f70380a3da0 0x7f70380a6260 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7040009370 tx=0x7f7040009240 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.975+0000 7f7032ffd640 1 -- 172.21.15.43:0/4178279238 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c015070 con 0x56006901ae80 2024-04-21T08:49:32.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.975+0000 7f7032ffd640 1 -- 172.21.15.43:0/4178279238 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f703c00be30 con 0x56006901ae80 2024-04-21T08:49:32.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.975+0000 7f7032ffd640 1 -- 172.21.15.43:0/4178279238 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c011940 con 0x56006901ae80 2024-04-21T08:49:32.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.975+0000 7f705198f740 1 -- 172.21.15.43:0/4178279238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56006901ae80 msgr2=0x560069023430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.975+0000 7f705198f740 1 --2- 172.21.15.43:0/4178279238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56006901ae80 0x560069023430 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f703c00bd40 tx=0x7f703c00f730 comp rx=0 tx=0).stop 2024-04-21T08:49:32.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.975+0000 7f705198f740 1 -- 172.21.15.43:0/4178279238 shutdown_connections 2024-04-21T08:49:32.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.975+0000 7f705198f740 1 --2- 172.21.15.43:0/4178279238 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560069023a00 0x560069025e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.975+0000 7f705198f740 1 --2- 172.21.15.43:0/4178279238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56006901ae80 0x560069023430 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.975+0000 7f705198f740 1 --2- 172.21.15.43:0/4178279238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56006901a4d0 0x56006901a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.975+0000 7f705198f740 1 -- 172.21.15.43:0/4178279238 >> 172.21.15.43:0/4178279238 conn(0x560069014ba0 msgr2=0x560069014f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.975+0000 7f705198f740 1 -- 172.21.15.43:0/4178279238 shutdown_connections 2024-04-21T08:49:32.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.976+0000 7f705198f740 1 -- 172.21.15.43:0/4178279238 wait complete. 2024-04-21T08:49:32.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.976+0000 7f705198f740 1 -- start start 2024-04-21T08:49:32.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.976+0000 7f705198f740 1 Processor -- start 2024-04-21T08:49:32.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.976+0000 7f705198f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56006901a4d0 0x560069023070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.976+0000 7f705198f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56006901ae80 0x560069020660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.976+0000 7f705198f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560069023a00 0x560069020ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.976+0000 7f705198f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560069026970 con 0x560069023a00 2024-04-21T08:49:32.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.977+0000 7f705198f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560068f38500 con 0x56006901a4d0 2024-04-21T08:49:32.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.977+0000 7f705198f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560069026af0 con 0x56006901ae80 2024-04-21T08:49:32.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.977+0000 7f705198f740 10 client.0 _ll_register_callbacks cb 0x560068f06b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:32.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.977+0000 7f704d33a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560069023a00 0x560069020ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.977+0000 7f7047fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56006901ae80 0x560069020660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.977+0000 7f704d33a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560069023a00 0x560069020ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47728/0 (socket says 172.21.15.43:47728) 2024-04-21T08:49:32.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.977+0000 7f704d33a640 1 -- 172.21.15.43:0/3588820473 learned_addr learned my addr 172.21.15.43:0/3588820473 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:32.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.977+0000 7f704cb39640 1 --2- 172.21.15.43:0/3588820473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56006901a4d0 0x560069023070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.977+0000 7f704d33a640 1 -- 172.21.15.43:0/3588820473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56006901ae80 msgr2=0x560069020660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.977+0000 7f704d33a640 1 --2- 172.21.15.43:0/3588820473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56006901ae80 0x560069020660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.977+0000 7f704d33a640 1 -- 172.21.15.43:0/3588820473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56006901a4d0 msgr2=0x560069023070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.977+0000 7f704d33a640 1 --2- 172.21.15.43:0/3588820473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56006901a4d0 0x560069023070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.977+0000 7f704d33a640 1 -- 172.21.15.43:0/3588820473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560068c913f0 con 0x560069023a00 2024-04-21T08:49:32.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.977+0000 7f704d33a640 1 --2- 172.21.15.43:0/3588820473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560069023a00 0x560069020ba0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f7040013c30 tx=0x7f7040004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a14b93d44c977b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.977+0000 7f7027fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:32.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.977+0000 7f7027fff640 1 -- 172.21.15.43:0/3588820473 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040023020 con 0x560069023a00 2024-04-21T08:49:32.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.977+0000 7f7027fff640 1 -- 172.21.15.43:0/3588820473 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f7040011470 con 0x560069023a00 2024-04-21T08:49:32.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.977+0000 7f705198f740 1 -- 172.21.15.43:0/3588820473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f703c00be30 con 0x560069023a00 2024-04-21T08:49:32.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.977+0000 7f7027fff640 1 -- 172.21.15.43:0/3588820473 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f704002e050 con 0x560069023a00 2024-04-21T08:49:32.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.977+0000 7f705198f740 10 client.20616 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:32.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.977+0000 7f705198f740 1 -- 172.21.15.43:0/3588820473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560068d5b7d0 con 0x560069023a00 2024-04-21T08:49:32.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.978+0000 7f7027fff640 1 -- 172.21.15.43:0/3588820473 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f703c00be30 con 0x560069023a00 2024-04-21T08:49:32.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.978+0000 7f7027fff640 1 -- 172.21.15.43:0/3588820473 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f704001f4a0 con 0x560069023a00 2024-04-21T08:49:32.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.978+0000 7f705198f740 10 client.20616 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:32.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.978+0000 7f705198f740 10 client.20616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:32.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.978+0000 7f705198f740 20 client.20616 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:32.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.978+0000 7f705198f740 1 --2- 172.21.15.43:0/3588820473 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56006903c510 0x56006903e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.978+0000 7f705198f740 4 client.20616 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:32.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.978+0000 7f705198f740 1 -- 172.21.15.43:0/3588820473 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560068f1e930 con 0x56006903c510 2024-04-21T08:49:32.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.978+0000 7f704cb39640 1 --2- 172.21.15.43:0/3588820473 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56006903c510 0x56006903e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.978+0000 7f7027fff640 10 client.20616 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:32.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.978+0000 7f704cb39640 1 --2- 172.21.15.43:0/3588820473 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56006903c510 0x56006903e940 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96018be5270b049b server_cookie=19c6ae2c1e8f7a75 in_seq=0 out_seq=0 2024-04-21T08:49:32.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.979+0000 7f7027fff640 1 -- 172.21.15.43:0/3588820473 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560068f1e930 con 0x56006903c510 2024-04-21T08:49:32.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.979+0000 7f7027fff640 10 client.20616 handle_command_reply: tid=0 2024-04-21T08:49:32.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.979+0000 7f705198f740 1 client.20616 shutdown 2024-04-21T08:49:32.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.979+0000 7f705198f740 1 -- 172.21.15.43:0/3588820473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560069023a00 msgr2=0x560069020ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.979+0000 7f705198f740 1 --2- 172.21.15.43:0/3588820473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560069023a00 0x560069020ba0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f7040013c30 tx=0x7f7040004a40 comp rx=0 tx=0).stop 2024-04-21T08:49:32.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.979+0000 7f705198f740 1 -- 172.21.15.43:0/3588820473 shutdown_connections 2024-04-21T08:49:32.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.979+0000 7f705198f740 1 --2- 172.21.15.43:0/3588820473 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x56006903c510 0x56006903e940 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.979+0000 7f705198f740 1 --2- 172.21.15.43:0/3588820473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560069023a00 0x560069020ba0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.979+0000 7f705198f740 1 --2- 172.21.15.43:0/3588820473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56006901ae80 0x560069020660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.979+0000 7f705198f740 1 --2- 172.21.15.43:0/3588820473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56006901a4d0 0x560069023070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.979+0000 7f705198f740 1 -- 172.21.15.43:0/3588820473 >> 172.21.15.43:0/3588820473 conn(0x560069014ba0 msgr2=0x5600690168f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:32.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.979+0000 7f7027fff640 0 client.20616 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:32.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.979+0000 7f705198f740 1 -- 172.21.15.43:0/3588820473 shutdown_connections 2024-04-21T08:49:32.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.979+0000 7f705198f740 1 -- 172.21.15.43:0/3588820473 wait complete. 2024-04-21T08:49:32.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.980+0000 7f705198f740 20 client.20616 trim_cache size 0 max 16384 2024-04-21T08:49:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.993+0000 7f705198f740 1 Processor -- start 2024-04-21T08:49:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.993+0000 7f705198f740 1 -- start start 2024-04-21T08:49:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.993+0000 7f705198f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560069014ba0 0x560069022d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.993+0000 7f705198f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56006901a4d0 0x560069044210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.993+0000 7f705198f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56006901ae80 0x560069044750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.993+0000 7f705198f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f70481ad3b0 con 0x56006901ae80 2024-04-21T08:49:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.993+0000 7f705198f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f70481aeef0 con 0x56006901a4d0 2024-04-21T08:49:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.993+0000 7f705198f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f70481ada80 con 0x560069014ba0 2024-04-21T08:49:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.993+0000 7f704d33a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56006901ae80 0x560069044750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.993+0000 7f704d33a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56006901ae80 0x560069044750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47742/0 (socket says 172.21.15.43:47742) 2024-04-21T08:49:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.993+0000 7f704d33a640 1 -- 172.21.15.43:0/1576323175 learned_addr learned my addr 172.21.15.43:0/1576323175 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.993+0000 7f704d33a640 1 -- 172.21.15.43:0/1576323175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560069014ba0 msgr2=0x560069022d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.993+0000 7f704d33a640 1 --2- 172.21.15.43:0/1576323175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560069014ba0 0x560069022d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.993+0000 7f704d33a640 1 -- 172.21.15.43:0/1576323175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56006901a4d0 msgr2=0x560069044210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.993+0000 7f7047fff640 1 --2- 172.21.15.43:0/1576323175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56006901a4d0 0x560069044210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.993+0000 7f704d33a640 1 --2- 172.21.15.43:0/1576323175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56006901a4d0 0x560069044210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.993+0000 7f704d33a640 1 -- 172.21.15.43:0/1576323175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560069023690 con 0x56006901ae80 2024-04-21T08:49:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.993+0000 7f7047fff640 1 --2- 172.21.15.43:0/1576323175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56006901a4d0 0x560069044210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.994+0000 7f704d33a640 1 --2- 172.21.15.43:0/1576323175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56006901ae80 0x560069044750 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f7040015040 tx=0x7f7040009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de4464ed3f91ced server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.994+0000 7f70327fc640 1 -- 172.21.15.43:0/1576323175 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040036020 con 0x56006901ae80 2024-04-21T08:49:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.994+0000 7f70327fc640 1 -- 172.21.15.43:0/1576323175 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7040018040 con 0x56006901ae80 2024-04-21T08:49:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.994+0000 7f70327fc640 1 -- 172.21.15.43:0/1576323175 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040023070 con 0x56006901ae80 2024-04-21T08:49:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.994+0000 7f705198f740 1 -- 172.21.15.43:0/1576323175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56006901ae80 msgr2=0x560069044750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.994+0000 7f705198f740 1 --2- 172.21.15.43:0/1576323175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56006901ae80 0x560069044750 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f7040015040 tx=0x7f7040009ba0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.994+0000 7f705198f740 1 -- 172.21.15.43:0/1576323175 shutdown_connections 2024-04-21T08:49:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.994+0000 7f705198f740 1 --2- 172.21.15.43:0/1576323175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56006901ae80 0x560069044750 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.994+0000 7f705198f740 1 --2- 172.21.15.43:0/1576323175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56006901a4d0 0x560069044210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.994+0000 7f705198f740 1 --2- 172.21.15.43:0/1576323175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560069014ba0 0x560069022d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.994+0000 7f705198f740 1 -- 172.21.15.43:0/1576323175 >> 172.21.15.43:0/1576323175 conn(0x560069020d00 msgr2=0x560069013c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.994+0000 7f705198f740 1 -- 172.21.15.43:0/1576323175 shutdown_connections 2024-04-21T08:49:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.994+0000 7f705198f740 1 -- 172.21.15.43:0/1576323175 wait complete. 2024-04-21T08:49:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.995+0000 7f705198f740 1 -- start start 2024-04-21T08:49:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.995+0000 7f705198f740 1 Processor -- start 2024-04-21T08:49:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.995+0000 7f705198f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560069014ba0 0x560069034b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.995+0000 7f705198f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56006901a4d0 0x560069035070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.995+0000 7f705198f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56006901ae80 0x5600690355b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.995+0000 7f705198f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f70481ae820 con 0x56006901a4d0 2024-04-21T08:49:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.995+0000 7f705198f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f70481ae150 con 0x56006901ae80 2024-04-21T08:49:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.995+0000 7f705198f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f70481ab3b0 con 0x560069014ba0 2024-04-21T08:49:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.995+0000 7f705198f740 10 client.0 _ll_register_callbacks cb 0x7f70400138b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.995+0000 7f7047fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56006901a4d0 0x560069035070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.995+0000 7f7047fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56006901a4d0 0x560069035070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47754/0 (socket says 172.21.15.43:47754) 2024-04-21T08:49:33.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.995+0000 7f7047fff640 1 -- 172.21.15.43:0/2186486474 learned_addr learned my addr 172.21.15.43:0/2186486474 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:33.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.995+0000 7f704d33a640 1 --2- 172.21.15.43:0/2186486474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56006901ae80 0x5600690355b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:33.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.996+0000 7f704cb39640 1 --2- 172.21.15.43:0/2186486474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560069014ba0 0x560069034b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:33.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.996+0000 7f7047fff640 1 -- 172.21.15.43:0/2186486474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560069014ba0 msgr2=0x560069034b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:33.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.996+0000 7f7047fff640 1 --2- 172.21.15.43:0/2186486474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560069014ba0 0x560069034b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:33.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.996+0000 7f7047fff640 1 -- 172.21.15.43:0/2186486474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56006901ae80 msgr2=0x5600690355b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:33.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.996+0000 7f7047fff640 1 --2- 172.21.15.43:0/2186486474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56006901ae80 0x5600690355b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:33.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.996+0000 7f7047fff640 1 -- 172.21.15.43:0/2186486474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600690a1c80 con 0x56006901a4d0 2024-04-21T08:49:33.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.996+0000 7f704d33a640 1 --2- 172.21.15.43:0/2186486474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56006901ae80 0x5600690355b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:33.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.996+0000 7f704cb39640 1 --2- 172.21.15.43:0/2186486474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560069014ba0 0x560069034b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:33.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.996+0000 7f7047fff640 1 --2- 172.21.15.43:0/2186486474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56006901a4d0 0x560069035070 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f7034006010 tx=0x7f70340035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6ea45cea177ac0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:33.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.996+0000 7f7027fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:33.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.996+0000 7f7027fff640 1 -- 172.21.15.43:0/2186486474 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70340bf0d0 con 0x56006901a4d0 2024-04-21T08:49:33.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.996+0000 7f7027fff640 1 -- 172.21.15.43:0/2186486474 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f70340f02f0 con 0x56006901a4d0 2024-04-21T08:49:33.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.996+0000 7f705198f740 1 -- 172.21.15.43:0/2186486474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7040018040 con 0x56006901a4d0 2024-04-21T08:49:33.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.996+0000 7f7027fff640 1 -- 172.21.15.43:0/2186486474 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f70340f04f0 con 0x56006901a4d0 2024-04-21T08:49:33.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.996+0000 7f705198f740 10 client.20622 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:33.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.996+0000 7f705198f740 1 -- 172.21.15.43:0/2186486474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5600690a2ef0 con 0x56006901a4d0 2024-04-21T08:49:33.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.996+0000 7f7027fff640 1 -- 172.21.15.43:0/2186486474 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7040018040 con 0x56006901a4d0 2024-04-21T08:49:33.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.997+0000 7f7027fff640 1 -- 172.21.15.43:0/2186486474 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f70340c71b0 con 0x56006901a4d0 2024-04-21T08:49:33.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.997+0000 7f705198f740 10 client.20622 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:33.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.997+0000 7f705198f740 10 client.20622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:33.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.997+0000 7f705198f740 20 client.20622 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:33.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.997+0000 7f705198f740 1 --2- 172.21.15.43:0/2186486474 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560069023a00 0x5600690a5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:33.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.997+0000 7f705198f740 4 client.20622 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:33.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.997+0000 7f705198f740 1 -- 172.21.15.43:0/2186486474 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560068f8cc10 con 0x560069023a00 2024-04-21T08:49:33.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.997+0000 7f704cb39640 1 --2- 172.21.15.43:0/2186486474 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560069023a00 0x5600690a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:33.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.997+0000 7f7027fff640 10 client.20622 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:33.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.997+0000 7f704cb39640 1 --2- 172.21.15.43:0/2186486474 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560069023a00 0x5600690a5700 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95dbb7c5408f94da server_cookie=f5bf2d5d5d7dc3bd in_seq=0 out_seq=0 2024-04-21T08:49:33.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.997+0000 7f7027fff640 1 -- 172.21.15.43:0/2186486474 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560068f8cc10 con 0x560069023a00 2024-04-21T08:49:33.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.997+0000 7f7027fff640 10 client.20622 handle_command_reply: tid=0 2024-04-21T08:49:33.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.997+0000 7f705198f740 1 client.20622 shutdown 2024-04-21T08:49:33.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.998+0000 7f705198f740 1 -- 172.21.15.43:0/2186486474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56006901a4d0 msgr2=0x560069035070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:33.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.998+0000 7f705198f740 1 --2- 172.21.15.43:0/2186486474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56006901a4d0 0x560069035070 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f7034006010 tx=0x7f70340035f0 comp rx=0 tx=0).stop 2024-04-21T08:49:33.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.998+0000 7f705198f740 1 -- 172.21.15.43:0/2186486474 shutdown_connections 2024-04-21T08:49:33.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.998+0000 7f705198f740 1 --2- 172.21.15.43:0/2186486474 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560069023a00 0x5600690a5700 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:33.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.998+0000 7f705198f740 1 --2- 172.21.15.43:0/2186486474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56006901ae80 0x5600690355b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:33.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.998+0000 7f705198f740 1 --2- 172.21.15.43:0/2186486474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56006901a4d0 0x560069035070 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:33.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.998+0000 7f7027fff640 0 client.20622 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:33.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.998+0000 7f705198f740 1 --2- 172.21.15.43:0/2186486474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560069014ba0 0x560069034b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:33.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.998+0000 7f705198f740 1 -- 172.21.15.43:0/2186486474 >> 172.21.15.43:0/2186486474 conn(0x560069020d00 msgr2=0x560069015ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:33.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.998+0000 7f705198f740 1 -- 172.21.15.43:0/2186486474 shutdown_connections 2024-04-21T08:49:33.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.998+0000 7f705198f740 1 -- 172.21.15.43:0/2186486474 wait complete. 2024-04-21T08:49:33.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.998+0000 7f705198f740 20 client.20622 trim_cache size 0 max 16384 2024-04-21T08:49:33.004 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:49:33.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.998+0000 7f704ed9d640 1 -- 172.21.15.43:0/2008734208 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f70380a3da0 msgr2=0x7f70380a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:33.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.998+0000 7f704ed9d640 1 --2- 172.21.15.43:0/2008734208 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f70380a3da0 0x7f70380a6260 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7040009370 tx=0x7f7040009240 comp rx=0 tx=0).stop 2024-04-21T08:49:33.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.998+0000 7f704ed9d640 1 -- 172.21.15.43:0/2008734208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7048103af0 msgr2=0x7f704819e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:33.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.998+0000 7f704ed9d640 1 --2- 172.21.15.43:0/2008734208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7048103af0 0x7f704819e920 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f7034005940 tx=0x7f703402d920 comp rx=0 tx=0).stop 2024-04-21T08:49:33.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.999+0000 7f704ed9d640 1 -- 172.21.15.43:0/2008734208 shutdown_connections 2024-04-21T08:49:33.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.999+0000 7f704ed9d640 1 --2- 172.21.15.43:0/2008734208 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f70380a3da0 0x7f70380a6260 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:33.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.999+0000 7f704ed9d640 1 --2- 172.21.15.43:0/2008734208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f704810c510 0x7f7048198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:33.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.999+0000 7f704ed9d640 1 --2- 172.21.15.43:0/2008734208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7048103af0 0x7f704819e920 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:33.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.999+0000 7f704ed9d640 1 --2- 172.21.15.43:0/2008734208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70481031d0 0x7f704819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:33.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.999+0000 7f704ed9d640 1 -- 172.21.15.43:0/2008734208 >> 172.21.15.43:0/2008734208 conn(0x7f7048077f50 msgr2=0x7f70480fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:33.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.999+0000 7f704ed9d640 1 -- 172.21.15.43:0/2008734208 shutdown_connections 2024-04-21T08:49:33.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:32.999+0000 7f704ed9d640 1 -- 172.21.15.43:0/2008734208 wait complete. 2024-04-21T08:49:33.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:33 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1660898022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:49:33.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:33 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1660898022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:49:33.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:33 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1660898022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:49:34.010 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:49:34.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:49:34.010 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:49:34.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.130+0000 7f652457e640 1 -- 172.21.15.43:0/1048405319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f651c106170 msgr2=0x7f651c10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.130+0000 7f652457e640 1 --2- 172.21.15.43:0/1048405319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f651c106170 0x7f651c10dd40 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f6514009ca0 tx=0x7f651402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.130+0000 7f652457e640 1 -- 172.21.15.43:0/1048405319 shutdown_connections 2024-04-21T08:49:34.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.130+0000 7f652457e640 1 --2- 172.21.15.43:0/1048405319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f651c106170 0x7f651c10dd40 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.130+0000 7f652457e640 1 --2- 172.21.15.43:0/1048405319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f651c101a50 0x7f651c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.130+0000 7f652457e640 1 --2- 172.21.15.43:0/1048405319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f651c1010a0 0x7f651c101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.130+0000 7f652457e640 1 -- 172.21.15.43:0/1048405319 >> 172.21.15.43:0/1048405319 conn(0x7f651c077eb0 msgr2=0x7f651c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.130+0000 7f652457e640 1 -- 172.21.15.43:0/1048405319 shutdown_connections 2024-04-21T08:49:34.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.130+0000 7f652457e640 1 -- 172.21.15.43:0/1048405319 wait complete. 2024-04-21T08:49:34.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.131+0000 7f652457e640 1 Processor -- start 2024-04-21T08:49:34.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.131+0000 7f652457e640 1 -- start start 2024-04-21T08:49:34.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.131+0000 7f652457e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f651c1010a0 0x7f651c1994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.131+0000 7f652457e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f651c106170 0x7f651c1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.131+0000 7f652457e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f651c199f70 0x7f651c1a4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.131+0000 7f652457e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f651c1110e0 con 0x7f651c199f70 2024-04-21T08:49:34.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.131+0000 7f652457e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f651c110f60 con 0x7f651c106170 2024-04-21T08:49:34.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.131+0000 7f652457e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f651c111260 con 0x7f651c1010a0 2024-04-21T08:49:34.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.131+0000 7f6522b1b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f651c199f70 0x7f651c1a4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.131+0000 7f6522b1b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f651c199f70 0x7f651c1a4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47770/0 (socket says 172.21.15.43:47770) 2024-04-21T08:49:34.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.131+0000 7f6522b1b640 1 -- 172.21.15.43:0/2835429301 learned_addr learned my addr 172.21.15.43:0/2835429301 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:34.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.131+0000 7f6521b19640 1 --2- 172.21.15.43:0/2835429301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f651c106170 0x7f651c1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.131+0000 7f652231a640 1 --2- 172.21.15.43:0/2835429301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f651c1010a0 0x7f651c1994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.131+0000 7f6522b1b640 1 -- 172.21.15.43:0/2835429301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f651c1010a0 msgr2=0x7f651c1994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.131+0000 7f6522b1b640 1 --2- 172.21.15.43:0/2835429301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f651c1010a0 0x7f651c1994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.131+0000 7f6522b1b640 1 -- 172.21.15.43:0/2835429301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f651c106170 msgr2=0x7f651c1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.131+0000 7f6522b1b640 1 --2- 172.21.15.43:0/2835429301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f651c106170 0x7f651c1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.131+0000 7f6522b1b640 1 -- 172.21.15.43:0/2835429301 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f651c1a4d20 con 0x7f651c199f70 2024-04-21T08:49:34.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.131+0000 7f6521b19640 1 --2- 172.21.15.43:0/2835429301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f651c106170 0x7f651c1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:34.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.132+0000 7f6522b1b640 1 --2- 172.21.15.43:0/2835429301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f651c199f70 0x7f651c1a4520 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f6514005e30 tx=0x7f6514034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.132+0000 7f65137fe640 1 -- 172.21.15.43:0/2835429301 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6514039040 con 0x7f651c199f70 2024-04-21T08:49:34.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.132+0000 7f65137fe640 1 -- 172.21.15.43:0/2835429301 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6514044070 con 0x7f651c199f70 2024-04-21T08:49:34.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.132+0000 7f65137fe640 1 -- 172.21.15.43:0/2835429301 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6514003e60 con 0x7f651c199f70 2024-04-21T08:49:34.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.132+0000 7f652457e640 1 -- 172.21.15.43:0/2835429301 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f651c1a4fb0 con 0x7f651c199f70 2024-04-21T08:49:34.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.132+0000 7f652457e640 1 -- 172.21.15.43:0/2835429301 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f651c1a5470 con 0x7f651c199f70 2024-04-21T08:49:34.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.134+0000 7f65137fe640 1 -- 172.21.15.43:0/2835429301 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f65140324c0 con 0x7f651c199f70 2024-04-21T08:49:34.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.135+0000 7f6527169740 1 Processor -- start 2024-04-21T08:49:34.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.135+0000 7f6527169740 1 -- start start 2024-04-21T08:49:34.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.135+0000 7f6527169740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed7612590 0x561ed7612970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.135+0000 7f65137fe640 1 --2- 172.21.15.43:0/2835429301 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f65080a3d20 0x7f65080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.135+0000 7f6527169740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 0x561ed7624420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.135+0000 7f6527169740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ed7624960 0x561ed7626d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.135+0000 7f6527169740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561ed7576f00 con 0x561ed7612eb0 2024-04-21T08:49:34.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.135+0000 7f6527169740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561ed7514590 con 0x561ed7624960 2024-04-21T08:49:34.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.135+0000 7f6527169740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561ed7572ec0 con 0x561ed7612590 2024-04-21T08:49:34.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.136+0000 7f652231a640 1 --2- 172.21.15.43:0/2835429301 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f65080a3d20 0x7f65080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.136+0000 7f6521b19640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 0x561ed7624420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.136+0000 7f6521b19640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 0x561ed7624420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47784/0 (socket says 172.21.15.43:47784) 2024-04-21T08:49:34.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.136+0000 7f6521b19640 1 -- 172.21.15.43:0/3334851847 learned_addr learned my addr 172.21.15.43:0/3334851847 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:34.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.136+0000 7f65137fe640 1 -- 172.21.15.43:0/2835429301 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f65140f2ed0 con 0x7f651c199f70 2024-04-21T08:49:34.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.136+0000 7f6521b19640 1 --2- 172.21.15.43:0/3334851847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed7612590 0x561ed7612970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.136+0000 7f6522b1b640 1 --2- 172.21.15.43:0/3334851847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ed7624960 0x561ed7626d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.136+0000 7f6521b19640 1 -- 172.21.15.43:0/3334851847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed7612590 msgr2=0x561ed7612970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.136+0000 7f6521b19640 1 --2- 172.21.15.43:0/3334851847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed7612590 0x561ed7612970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.136+0000 7f6521b19640 1 -- 172.21.15.43:0/3334851847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ed7624960 msgr2=0x561ed7626d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.136+0000 7f6521b19640 1 --2- 172.21.15.43:0/3334851847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ed7624960 0x561ed7626d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.136+0000 7f6521b19640 1 -- 172.21.15.43:0/3334851847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ed73517e0 con 0x561ed7612eb0 2024-04-21T08:49:34.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.136+0000 7f652231a640 1 --2- 172.21.15.43:0/2835429301 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f65080a3d20 0x7f65080a61e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6518005fd0 tx=0x7f6518005850 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.136+0000 7f6521b19640 1 --2- 172.21.15.43:0/3334851847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 0x561ed7624420 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f650c013920 tx=0x7f650c00b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6613671d715837 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.136+0000 7f6510bfe640 1 -- 172.21.15.43:0/3334851847 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c004030 con 0x561ed7612eb0 2024-04-21T08:49:34.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.137+0000 7f6510bfe640 1 -- 172.21.15.43:0/3334851847 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f650c00e960 con 0x561ed7612eb0 2024-04-21T08:49:34.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.137+0000 7f6510bfe640 1 -- 172.21.15.43:0/3334851847 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c003850 con 0x561ed7612eb0 2024-04-21T08:49:34.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.137+0000 7f6527169740 1 -- 172.21.15.43:0/3334851847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 msgr2=0x561ed7624420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.137+0000 7f6527169740 1 --2- 172.21.15.43:0/3334851847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 0x561ed7624420 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f650c013920 tx=0x7f650c00b440 comp rx=0 tx=0).stop 2024-04-21T08:49:34.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.137+0000 7f6527169740 1 -- 172.21.15.43:0/3334851847 shutdown_connections 2024-04-21T08:49:34.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.137+0000 7f6527169740 1 --2- 172.21.15.43:0/3334851847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ed7624960 0x561ed7626d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.137+0000 7f6527169740 1 --2- 172.21.15.43:0/3334851847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 0x561ed7624420 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.137+0000 7f6527169740 1 --2- 172.21.15.43:0/3334851847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed7612590 0x561ed7612970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.137+0000 7f6527169740 1 -- 172.21.15.43:0/3334851847 >> 172.21.15.43:0/3334851847 conn(0x561ed760cfb0 msgr2=0x561ed760d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.137+0000 7f6527169740 1 -- 172.21.15.43:0/3334851847 shutdown_connections 2024-04-21T08:49:34.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.137+0000 7f6527169740 1 -- 172.21.15.43:0/3334851847 wait complete. 2024-04-21T08:49:34.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.138+0000 7f6527169740 1 -- start start 2024-04-21T08:49:34.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.138+0000 7f6527169740 1 Processor -- start 2024-04-21T08:49:34.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.138+0000 7f6527169740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed7612590 0x561ed7621980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.138+0000 7f6527169740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 0x561ed7621ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.138+0000 7f6527169740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ed7624960 0x561ed7622400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.138+0000 7f6527169740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561ed7627530 con 0x561ed7612eb0 2024-04-21T08:49:34.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.138+0000 7f6527169740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561ed7530580 con 0x561ed7624960 2024-04-21T08:49:34.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.138+0000 7f6527169740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561ed76276b0 con 0x561ed7612590 2024-04-21T08:49:34.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.138+0000 7f6527169740 10 client.0 _ll_register_callbacks cb 0x561ed7536650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:34.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.138+0000 7f6521b19640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 0x561ed7621ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.138+0000 7f6521b19640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 0x561ed7621ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47792/0 (socket says 172.21.15.43:47792) 2024-04-21T08:49:34.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.138+0000 7f6521b19640 1 -- 172.21.15.43:0/77093372 learned_addr learned my addr 172.21.15.43:0/77093372 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:34.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.138+0000 7f652231a640 1 --2- 172.21.15.43:0/77093372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed7612590 0x561ed7621980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.138+0000 7f6522b1b640 1 --2- 172.21.15.43:0/77093372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ed7624960 0x561ed7622400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.138+0000 7f6521b19640 1 -- 172.21.15.43:0/77093372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed7612590 msgr2=0x561ed7621980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.138+0000 7f6521b19640 1 --2- 172.21.15.43:0/77093372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed7612590 0x561ed7621980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.138+0000 7f6521b19640 1 -- 172.21.15.43:0/77093372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ed7624960 msgr2=0x561ed7622400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.138+0000 7f6521b19640 1 --2- 172.21.15.43:0/77093372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ed7624960 0x561ed7622400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.139+0000 7f6521b19640 1 -- 172.21.15.43:0/77093372 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ed73b1f70 con 0x561ed7612eb0 2024-04-21T08:49:34.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.139+0000 7f6521b19640 1 --2- 172.21.15.43:0/77093372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 0x561ed7621ec0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f650c00c010 tx=0x7f650c013300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3390510bc13149a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.139+0000 7f6505ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:34.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.139+0000 7f6505ffb640 1 -- 172.21.15.43:0/77093372 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c013590 con 0x561ed7612eb0 2024-04-21T08:49:34.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.139+0000 7f6505ffb640 1 -- 172.21.15.43:0/77093372 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f650c027450 con 0x561ed7612eb0 2024-04-21T08:49:34.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.139+0000 7f6527169740 1 -- 172.21.15.43:0/77093372 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f650c00e960 con 0x561ed7612eb0 2024-04-21T08:49:34.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.139+0000 7f6505ffb640 1 -- 172.21.15.43:0/77093372 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f650c030050 con 0x561ed7612eb0 2024-04-21T08:49:34.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.139+0000 7f6527169740 10 client.20634 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:34.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.139+0000 7f6527169740 1 -- 172.21.15.43:0/77093372 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561ed7249170 con 0x561ed7612eb0 2024-04-21T08:49:34.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.139+0000 7f6505ffb640 1 -- 172.21.15.43:0/77093372 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f650c00e960 con 0x561ed7612eb0 2024-04-21T08:49:34.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.139+0000 7f6505ffb640 1 -- 172.21.15.43:0/77093372 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f650c011660 con 0x561ed7612eb0 2024-04-21T08:49:34.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.139+0000 7f6527169740 10 client.20634 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:34.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.139+0000 7f6527169740 10 client.20634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:34.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.139+0000 7f6527169740 20 client.20634 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:34.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.139+0000 7f6527169740 1 --2- 172.21.15.43:0/77093372 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561ed76349c0 0x561ed7636db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.139+0000 7f6527169740 4 client.20634 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:34.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.139+0000 7f6527169740 1 -- 172.21.15.43:0/77093372 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ed7584920 con 0x561ed76349c0 2024-04-21T08:49:34.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.140+0000 7f652231a640 1 --2- 172.21.15.43:0/77093372 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561ed76349c0 0x561ed7636db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.140+0000 7f6505ffb640 10 client.20634 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:34.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.140+0000 7f652231a640 1 --2- 172.21.15.43:0/77093372 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561ed76349c0 0x561ed7636db0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80e68a833b8827a server_cookie=cd01021cc6629482 in_seq=0 out_seq=0 2024-04-21T08:49:34.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.141+0000 7f6505ffb640 1 -- 172.21.15.43:0/77093372 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561ed7584920 con 0x561ed76349c0 2024-04-21T08:49:34.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.141+0000 7f6505ffb640 10 client.20634 handle_command_reply: tid=0 2024-04-21T08:49:34.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.141+0000 7f6527169740 1 client.20634 shutdown 2024-04-21T08:49:34.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.141+0000 7f6527169740 1 -- 172.21.15.43:0/77093372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 msgr2=0x561ed7621ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.141+0000 7f6527169740 1 --2- 172.21.15.43:0/77093372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 0x561ed7621ec0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f650c00c010 tx=0x7f650c013300 comp rx=0 tx=0).stop 2024-04-21T08:49:34.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.141+0000 7f6527169740 1 -- 172.21.15.43:0/77093372 shutdown_connections 2024-04-21T08:49:34.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.141+0000 7f6527169740 1 --2- 172.21.15.43:0/77093372 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561ed76349c0 0x561ed7636db0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.141+0000 7f6527169740 1 --2- 172.21.15.43:0/77093372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ed7624960 0x561ed7622400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.141+0000 7f6527169740 1 --2- 172.21.15.43:0/77093372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 0x561ed7621ec0 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.141+0000 7f6527169740 1 --2- 172.21.15.43:0/77093372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed7612590 0x561ed7621980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.141+0000 7f6505ffb640 0 client.20634 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:34.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.141+0000 7f6527169740 1 -- 172.21.15.43:0/77093372 >> 172.21.15.43:0/77093372 conn(0x561ed760cfb0 msgr2=0x561ed760ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.141+0000 7f6527169740 1 -- 172.21.15.43:0/77093372 shutdown_connections 2024-04-21T08:49:34.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.141+0000 7f6527169740 1 -- 172.21.15.43:0/77093372 wait complete. 2024-04-21T08:49:34.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.141+0000 7f6527169740 20 client.20634 trim_cache size 0 max 16384 2024-04-21T08:49:34.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.160+0000 7f6527169740 1 Processor -- start 2024-04-21T08:49:34.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.160+0000 7f6527169740 1 -- start start 2024-04-21T08:49:34.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.160+0000 7f6527169740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ed7612590 0x561ed76082d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.160+0000 7f6527169740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 0x561ed760be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.160+0000 7f6527169740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed7624960 0x561ed7639dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.160+0000 7f6527169740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f651c1ad220 con 0x561ed7612eb0 2024-04-21T08:49:34.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.160+0000 7f6527169740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f651c1aed60 con 0x561ed7612590 2024-04-21T08:49:34.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.160+0000 7f6527169740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f651c1ad8f0 con 0x561ed7624960 2024-04-21T08:49:34.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.160+0000 7f6521b19640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 0x561ed760be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.160+0000 7f6521b19640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 0x561ed760be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47804/0 (socket says 172.21.15.43:47804) 2024-04-21T08:49:34.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.160+0000 7f6521b19640 1 -- 172.21.15.43:0/882905567 learned_addr learned my addr 172.21.15.43:0/882905567 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:34.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.160+0000 7f652231a640 1 --2- 172.21.15.43:0/882905567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ed7612590 0x561ed76082d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.160+0000 7f6522b1b640 1 --2- 172.21.15.43:0/882905567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed7624960 0x561ed7639dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.160+0000 7f6521b19640 1 -- 172.21.15.43:0/882905567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed7624960 msgr2=0x561ed7639dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.160+0000 7f6521b19640 1 --2- 172.21.15.43:0/882905567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed7624960 0x561ed7639dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.160+0000 7f6521b19640 1 -- 172.21.15.43:0/882905567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ed7612590 msgr2=0x561ed76082d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.160+0000 7f6521b19640 1 --2- 172.21.15.43:0/882905567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ed7612590 0x561ed76082d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.160+0000 7f6521b19640 1 -- 172.21.15.43:0/882905567 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ed76172e0 con 0x561ed7612eb0 2024-04-21T08:49:34.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.161+0000 7f6521b19640 1 --2- 172.21.15.43:0/882905567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 0x561ed760be90 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f650c01e410 tx=0x7f650c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5cb8c2bb5c50cf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.161+0000 7f6507fff640 1 -- 172.21.15.43:0/882905567 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c039070 con 0x561ed7612eb0 2024-04-21T08:49:34.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.161+0000 7f6507fff640 1 -- 172.21.15.43:0/882905567 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f650c00e960 con 0x561ed7612eb0 2024-04-21T08:49:34.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.161+0000 7f6507fff640 1 -- 172.21.15.43:0/882905567 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c0052d0 con 0x561ed7612eb0 2024-04-21T08:49:34.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.162+0000 7f6527169740 1 -- 172.21.15.43:0/882905567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 msgr2=0x561ed760be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.162+0000 7f6527169740 1 --2- 172.21.15.43:0/882905567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 0x561ed760be90 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f650c01e410 tx=0x7f650c00bf10 comp rx=0 tx=0).stop 2024-04-21T08:49:34.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.162+0000 7f6527169740 1 -- 172.21.15.43:0/882905567 shutdown_connections 2024-04-21T08:49:34.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.162+0000 7f6527169740 1 --2- 172.21.15.43:0/882905567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed7624960 0x561ed7639dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.162+0000 7f6527169740 1 --2- 172.21.15.43:0/882905567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 0x561ed760be90 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.162+0000 7f6527169740 1 --2- 172.21.15.43:0/882905567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ed7612590 0x561ed76082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.162+0000 7f6527169740 1 -- 172.21.15.43:0/882905567 >> 172.21.15.43:0/882905567 conn(0x561ed760cfb0 msgr2=0x561ed760b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.162+0000 7f6527169740 1 -- 172.21.15.43:0/882905567 shutdown_connections 2024-04-21T08:49:34.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.162+0000 7f6527169740 1 -- 172.21.15.43:0/882905567 wait complete. 2024-04-21T08:49:34.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.162+0000 7f6527169740 1 -- start start 2024-04-21T08:49:34.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.162+0000 7f6527169740 1 Processor -- start 2024-04-21T08:49:34.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.162+0000 7f6527169740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed7612590 0x561ed7637a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.162+0000 7f6527169740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 0x561ed7637f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.162+0000 7f6527169740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ed7624960 0x561ed76384c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.162+0000 7f6527169740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f651c1ae690 con 0x561ed7612eb0 2024-04-21T08:49:34.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.162+0000 7f6527169740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f651c1adfc0 con 0x561ed7624960 2024-04-21T08:49:34.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.162+0000 7f6527169740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f651c1ab220 con 0x561ed7612590 2024-04-21T08:49:34.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.162+0000 7f6527169740 10 client.0 _ll_register_callbacks cb 0x7f650c00be60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:34.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.162+0000 7f6521b19640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 0x561ed7637f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.162+0000 7f6521b19640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 0x561ed7637f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47816/0 (socket says 172.21.15.43:47816) 2024-04-21T08:49:34.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.162+0000 7f6521b19640 1 -- 172.21.15.43:0/1953378726 learned_addr learned my addr 172.21.15.43:0/1953378726 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:34.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.163+0000 7f6522b1b640 1 --2- 172.21.15.43:0/1953378726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ed7624960 0x561ed76384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.163+0000 7f6521b19640 1 -- 172.21.15.43:0/1953378726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed7612590 msgr2=0x561ed7637a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.163+0000 7f6521b19640 1 --2- 172.21.15.43:0/1953378726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed7612590 0x561ed7637a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.163+0000 7f6521b19640 1 -- 172.21.15.43:0/1953378726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ed7624960 msgr2=0x561ed76384c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.163+0000 7f6521b19640 1 --2- 172.21.15.43:0/1953378726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ed7624960 0x561ed76384c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.163+0000 7f6521b19640 1 -- 172.21.15.43:0/1953378726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ed7699df0 con 0x561ed7612eb0 2024-04-21T08:49:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.163+0000 7f6521b19640 1 --2- 172.21.15.43:0/1953378726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 0x561ed7637f80 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f650c00bc00 tx=0x7f650c00ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62020957df602691 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.163+0000 7f6505ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.163+0000 7f6505ffb640 1 -- 172.21.15.43:0/1953378726 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c039070 con 0x561ed7612eb0 2024-04-21T08:49:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.163+0000 7f6505ffb640 1 -- 172.21.15.43:0/1953378726 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f650c011640 con 0x561ed7612eb0 2024-04-21T08:49:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.163+0000 7f6527169740 1 -- 172.21.15.43:0/1953378726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f650c00e960 con 0x561ed7612eb0 2024-04-21T08:49:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.163+0000 7f6505ffb640 1 -- 172.21.15.43:0/1953378726 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f650c046050 con 0x561ed7612eb0 2024-04-21T08:49:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.163+0000 7f6527169740 10 client.20640 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.163+0000 7f6527169740 1 -- 172.21.15.43:0/1953378726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561ed769b0f0 con 0x561ed7612eb0 2024-04-21T08:49:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.163+0000 7f6505ffb640 1 -- 172.21.15.43:0/1953378726 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f650c00e960 con 0x561ed7612eb0 2024-04-21T08:49:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.164+0000 7f6505ffb640 1 -- 172.21.15.43:0/1953378726 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f650c004820 con 0x561ed7612eb0 2024-04-21T08:49:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.164+0000 7f6527169740 10 client.20640 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.164+0000 7f6527169740 10 client.20640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.164+0000 7f6527169740 20 client.20640 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.164+0000 7f6527169740 1 --2- 172.21.15.43:0/1953378726 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561ed769b7d0 0x561ed769dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.164+0000 7f6527169740 4 client.20640 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:49:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.164+0000 7f6527169740 1 -- 172.21.15.43:0/1953378726 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561ed757e790 con 0x561ed769b7d0 2024-04-21T08:49:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.164+0000 7f652231a640 1 --2- 172.21.15.43:0/1953378726 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561ed769b7d0 0x561ed769dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.164+0000 7f6505ffb640 10 client.20640 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.164+0000 7f652231a640 1 --2- 172.21.15.43:0/1953378726 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561ed769b7d0 0x561ed769dbf0 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f417d5f341633f67 server_cookie=8eaab0eb95e20475 in_seq=0 out_seq=0 2024-04-21T08:49:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.165+0000 7f6505ffb640 1 -- 172.21.15.43:0/1953378726 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561ed757e790 con 0x561ed769b7d0 2024-04-21T08:49:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.165+0000 7f6505ffb640 10 client.20640 handle_command_reply: tid=0 2024-04-21T08:49:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.165+0000 7f6527169740 1 client.20640 shutdown 2024-04-21T08:49:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.166+0000 7f6527169740 1 -- 172.21.15.43:0/1953378726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 msgr2=0x561ed7637f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.166+0000 7f6527169740 1 --2- 172.21.15.43:0/1953378726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 0x561ed7637f80 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f650c00bc00 tx=0x7f650c00ed20 comp rx=0 tx=0).stop 2024-04-21T08:49:34.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.166+0000 7f6527169740 1 -- 172.21.15.43:0/1953378726 shutdown_connections 2024-04-21T08:49:34.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.166+0000 7f6527169740 1 --2- 172.21.15.43:0/1953378726 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561ed769b7d0 0x561ed769dbf0 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.166+0000 7f6527169740 1 --2- 172.21.15.43:0/1953378726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ed7624960 0x561ed76384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.166+0000 7f6527169740 1 --2- 172.21.15.43:0/1953378726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ed7612eb0 0x561ed7637f80 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.166+0000 7f6527169740 1 --2- 172.21.15.43:0/1953378726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561ed7612590 0x561ed7637a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.166+0000 7f6505ffb640 0 client.20640 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:34.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.166+0000 7f6527169740 1 -- 172.21.15.43:0/1953378726 >> 172.21.15.43:0/1953378726 conn(0x561ed760cfb0 msgr2=0x561ed760c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.166+0000 7f6527169740 1 -- 172.21.15.43:0/1953378726 shutdown_connections 2024-04-21T08:49:34.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.166+0000 7f6527169740 1 -- 172.21.15.43:0/1953378726 wait complete. 2024-04-21T08:49:34.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.166+0000 7f6527169740 20 client.20640 trim_cache size 0 max 16384 2024-04-21T08:49:34.170 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:49:34.171 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:49:34.171 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "997973e3-d1f7-46ad-8149-2982d88a5f57", 2024-04-21T08:49:34.171 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:49:34.171 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:49:34.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.166+0000 7f652457e640 1 -- 172.21.15.43:0/2835429301 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f65080a3d20 msgr2=0x7f65080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.166+0000 7f652457e640 1 --2- 172.21.15.43:0/2835429301 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f65080a3d20 0x7f65080a61e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6518005fd0 tx=0x7f6518005850 comp rx=0 tx=0).stop 2024-04-21T08:49:34.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.166+0000 7f652457e640 1 -- 172.21.15.43:0/2835429301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f651c199f70 msgr2=0x7f651c1a4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.166+0000 7f652457e640 1 --2- 172.21.15.43:0/2835429301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f651c199f70 0x7f651c1a4520 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f6514005e30 tx=0x7f6514034760 comp rx=0 tx=0).stop 2024-04-21T08:49:34.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.166+0000 7f652457e640 1 -- 172.21.15.43:0/2835429301 shutdown_connections 2024-04-21T08:49:34.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.166+0000 7f652457e640 1 --2- 172.21.15.43:0/2835429301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f651c199f70 0x7f651c1a4520 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.166+0000 7f652457e640 1 --2- 172.21.15.43:0/2835429301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f651c106170 0x7f651c1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.166+0000 7f652457e640 1 --2- 172.21.15.43:0/2835429301 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f65080a3d20 0x7f65080a61e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.166+0000 7f652457e640 1 --2- 172.21.15.43:0/2835429301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f651c1010a0 0x7f651c1994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.166+0000 7f652457e640 1 -- 172.21.15.43:0/2835429301 >> 172.21.15.43:0/2835429301 conn(0x7f651c077eb0 msgr2=0x7f651c0fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.167+0000 7f652457e640 1 -- 172.21.15.43:0/2835429301 shutdown_connections 2024-04-21T08:49:34.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.167+0000 7f652457e640 1 -- 172.21.15.43:0/2835429301 wait complete. 2024-04-21T08:49:34.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:49:34.178 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:49:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.282+0000 7fe44e0c1640 1 -- 172.21.15.43:0/953269641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4481031b0 msgr2=0x7fe448103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.282+0000 7fe44e0c1640 1 --2- 172.21.15.43:0/953269641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4481031b0 0x7fe448103590 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7fe438009900 tx=0x7fe43802bcd0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.282+0000 7fe44e0c1640 1 -- 172.21.15.43:0/953269641 shutdown_connections 2024-04-21T08:49:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.282+0000 7fe44e0c1640 1 --2- 172.21.15.43:0/953269641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe44810c580 0x7fe44810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.282+0000 7fe44e0c1640 1 --2- 172.21.15.43:0/953269641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe448103ad0 0x7fe44810bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.282+0000 7fe44e0c1640 1 --2- 172.21.15.43:0/953269641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4481031b0 0x7fe448103590 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.282+0000 7fe44e0c1640 1 -- 172.21.15.43:0/953269641 >> 172.21.15.43:0/953269641 conn(0x7fe4480771d0 msgr2=0x7fe4480775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.282+0000 7fe44e0c1640 1 -- 172.21.15.43:0/953269641 shutdown_connections 2024-04-21T08:49:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.282+0000 7fe44e0c1640 1 -- 172.21.15.43:0/953269641 wait complete. 2024-04-21T08:49:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.283+0000 7fe44e0c1640 1 Processor -- start 2024-04-21T08:49:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.283+0000 7fe44e0c1640 1 -- start start 2024-04-21T08:49:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.283+0000 7fe44e0c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4481031b0 0x7fe448198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.283+0000 7fe44e0c1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe448103ad0 0x7fe448198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.283+0000 7fe44e0c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe44810c580 0x7fe44819cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.283+0000 7fe44e0c1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe4481121d0 con 0x7fe4481031b0 2024-04-21T08:49:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.283+0000 7fe44e0c1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe448112050 con 0x7fe44810c580 2024-04-21T08:49:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.283+0000 7fe44e0c1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe448112350 con 0x7fe448103ad0 2024-04-21T08:49:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.283+0000 7fe4477fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4481031b0 0x7fe448198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.283+0000 7fe4477fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4481031b0 0x7fe448198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47830/0 (socket says 172.21.15.43:47830) 2024-04-21T08:49:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.283+0000 7fe4477fe640 1 -- 172.21.15.43:0/891398271 learned_addr learned my addr 172.21.15.43:0/891398271 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.284+0000 7fe4477fe640 1 -- 172.21.15.43:0/891398271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe448103ad0 msgr2=0x7fe448198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.284+0000 7fe446ffd640 1 --2- 172.21.15.43:0/891398271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe448103ad0 0x7fe448198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.284+0000 7fe447fff640 1 --2- 172.21.15.43:0/891398271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe44810c580 0x7fe44819cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.284+0000 7fe4477fe640 1 --2- 172.21.15.43:0/891398271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe448103ad0 0x7fe448198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.284+0000 7fe4477fe640 1 -- 172.21.15.43:0/891398271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe44810c580 msgr2=0x7fe44819cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.284+0000 7fe4477fe640 1 --2- 172.21.15.43:0/891398271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe44810c580 0x7fe44819cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.284+0000 7fe4477fe640 1 -- 172.21.15.43:0/891398271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44819d720 con 0x7fe4481031b0 2024-04-21T08:49:34.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.284+0000 7fe447fff640 1 --2- 172.21.15.43:0/891398271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe44810c580 0x7fe44819cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:34.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.284+0000 7fe4477fe640 1 --2- 172.21.15.43:0/891398271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4481031b0 0x7fe448198730 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fe438005940 tx=0x7fe43802d910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.284+0000 7fe444ff9640 1 -- 172.21.15.43:0/891398271 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438043020 con 0x7fe4481031b0 2024-04-21T08:49:34.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.284+0000 7fe444ff9640 1 -- 172.21.15.43:0/891398271 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe43802dd10 con 0x7fe4481031b0 2024-04-21T08:49:34.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.284+0000 7fe44e0c1640 1 -- 172.21.15.43:0/891398271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44819d9b0 con 0x7fe4481031b0 2024-04-21T08:49:34.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.284+0000 7fe444ff9640 1 -- 172.21.15.43:0/891398271 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438003b80 con 0x7fe4481031b0 2024-04-21T08:49:34.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.284+0000 7fe44e0c1640 1 -- 172.21.15.43:0/891398271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4481a52a0 con 0x7fe4481031b0 2024-04-21T08:49:34.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.286+0000 7fe444ff9640 1 -- 172.21.15.43:0/891398271 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe438046020 con 0x7fe4481031b0 2024-04-21T08:49:34.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.287+0000 7fe450d8d740 1 Processor -- start 2024-04-21T08:49:34.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.287+0000 7fe450d8d740 1 -- start start 2024-04-21T08:49:34.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.287+0000 7fe450d8d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e4932230 0x5613e4932610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.287+0000 7fe450d8d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613e4932b50 0x5613e493f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.287+0000 7fe450d8d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613e493fe90 0x5613e4942280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.287+0000 7fe450d8d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5613e48971e0 con 0x5613e4932230 2024-04-21T08:49:34.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.287+0000 7fe450d8d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5613e4834390 con 0x5613e4932b50 2024-04-21T08:49:34.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.287+0000 7fe450d8d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5613e48931a0 con 0x5613e493fe90 2024-04-21T08:49:34.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.287+0000 7fe444ff9640 1 --2- 172.21.15.43:0/891398271 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe4300a3d20 0x7fe4300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.287+0000 7fe447fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e4932230 0x5613e4932610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.287+0000 7fe447fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e4932230 0x5613e4932610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47840/0 (socket says 172.21.15.43:47840) 2024-04-21T08:49:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.287+0000 7fe4477fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613e4932b50 0x5613e493f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.287+0000 7fe447fff640 1 -- 172.21.15.43:0/1450078977 learned_addr learned my addr 172.21.15.43:0/1450078977 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.287+0000 7fe447fff640 1 --2- 172.21.15.43:0/891398271 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe4300a3d20 0x7fe4300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.287+0000 7fe444ff9640 1 -- 172.21.15.43:0/891398271 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fe4380f2e50 con 0x7fe4481031b0 2024-04-21T08:49:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.287+0000 7fe446ffd640 1 --2- 172.21.15.43:0/1450078977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613e493fe90 0x5613e4942280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.287+0000 7fe447fff640 1 -- 172.21.15.43:0/1450078977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613e493fe90 msgr2=0x5613e4942280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.287+0000 7fe447fff640 1 --2- 172.21.15.43:0/1450078977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613e493fe90 0x5613e4942280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.287+0000 7fe447fff640 1 -- 172.21.15.43:0/1450078977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613e4932b50 msgr2=0x5613e493f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.287+0000 7fe447fff640 1 --2- 172.21.15.43:0/1450078977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613e4932b50 0x5613e493f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.287+0000 7fe447fff640 1 -- 172.21.15.43:0/1450078977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613e46737d0 con 0x5613e4932230 2024-04-21T08:49:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.288+0000 7fe447fff640 1 --2- 172.21.15.43:0/1450078977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e4932230 0x5613e4932610 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fe440012ce0 tx=0x7fe44000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa72b309f6ea749 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.288+0000 7fe447fff640 1 --2- 172.21.15.43:0/891398271 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe4300a3d20 0x7fe4300a61e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe44000afd0 tx=0x7fe44001a000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.288+0000 7fe42dffb640 1 -- 172.21.15.43:0/1450078977 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440019cb0 con 0x5613e4932230 2024-04-21T08:49:34.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.288+0000 7fe42dffb640 1 -- 172.21.15.43:0/1450078977 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe440019e50 con 0x5613e4932230 2024-04-21T08:49:34.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.288+0000 7fe42dffb640 1 -- 172.21.15.43:0/1450078977 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440011640 con 0x5613e4932230 2024-04-21T08:49:34.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.288+0000 7fe450d8d740 1 -- 172.21.15.43:0/1450078977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e4932230 msgr2=0x5613e4932610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.288+0000 7fe450d8d740 1 --2- 172.21.15.43:0/1450078977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e4932230 0x5613e4932610 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fe440012ce0 tx=0x7fe44000f730 comp rx=0 tx=0).stop 2024-04-21T08:49:34.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.289+0000 7fe450d8d740 1 -- 172.21.15.43:0/1450078977 shutdown_connections 2024-04-21T08:49:34.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.289+0000 7fe450d8d740 1 --2- 172.21.15.43:0/1450078977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613e493fe90 0x5613e4942280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.289+0000 7fe450d8d740 1 --2- 172.21.15.43:0/1450078977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613e4932b50 0x5613e493f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.289+0000 7fe450d8d740 1 --2- 172.21.15.43:0/1450078977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e4932230 0x5613e4932610 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.289+0000 7fe450d8d740 1 -- 172.21.15.43:0/1450078977 >> 172.21.15.43:0/1450078977 conn(0x5613e492ccb0 msgr2=0x5613e492d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.289+0000 7fe450d8d740 1 -- 172.21.15.43:0/1450078977 shutdown_connections 2024-04-21T08:49:34.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.289+0000 7fe450d8d740 1 -- 172.21.15.43:0/1450078977 wait complete. 2024-04-21T08:49:34.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.289+0000 7fe450d8d740 1 -- start start 2024-04-21T08:49:34.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.289+0000 7fe450d8d740 1 Processor -- start 2024-04-21T08:49:34.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.289+0000 7fe450d8d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e4932230 0x5613e493ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.289+0000 7fe450d8d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613e4932b50 0x5613e493d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.289+0000 7fe450d8d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613e493fe90 0x5613e493d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.289+0000 7fe450d8d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5613e4942c70 con 0x5613e4932230 2024-04-21T08:49:34.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.289+0000 7fe450d8d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5613e48502f0 con 0x5613e4932b50 2024-04-21T08:49:34.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.289+0000 7fe450d8d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5613e4942df0 con 0x5613e493fe90 2024-04-21T08:49:34.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.289+0000 7fe450d8d740 10 client.0 _ll_register_callbacks cb 0x5613e485d580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:34.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.289+0000 7fe4477fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e4932230 0x5613e493ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.290+0000 7fe4477fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e4932230 0x5613e493ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47858/0 (socket says 172.21.15.43:47858) 2024-04-21T08:49:34.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.290+0000 7fe4477fe640 1 -- 172.21.15.43:0/2915237429 learned_addr learned my addr 172.21.15.43:0/2915237429 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:34.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.290+0000 7fe446ffd640 1 --2- 172.21.15.43:0/2915237429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613e4932b50 0x5613e493d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.290+0000 7fe4477fe640 1 -- 172.21.15.43:0/2915237429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613e493fe90 msgr2=0x5613e493d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.290+0000 7fe4477fe640 1 --2- 172.21.15.43:0/2915237429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613e493fe90 0x5613e493d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.290+0000 7fe4477fe640 1 -- 172.21.15.43:0/2915237429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613e4932b50 msgr2=0x5613e493d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.290+0000 7fe4477fe640 1 --2- 172.21.15.43:0/2915237429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613e4932b50 0x5613e493d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.290+0000 7fe4477fe640 1 -- 172.21.15.43:0/2915237429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613e45a93f0 con 0x5613e4932230 2024-04-21T08:49:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.290+0000 7fe4477fe640 1 --2- 172.21.15.43:0/2915237429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e4932230 0x5613e493ce60 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fe438005ec0 tx=0x7fe43803b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4995a1c0d88a0a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.290+0000 7fe426ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.290+0000 7fe426ffd640 1 -- 172.21.15.43:0/2915237429 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43803c350 con 0x5613e4932230 2024-04-21T08:49:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.290+0000 7fe426ffd640 1 -- 172.21.15.43:0/2915237429 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fe4380f0400 con 0x5613e4932230 2024-04-21T08:49:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.290+0000 7fe450d8d740 1 -- 172.21.15.43:0/2915237429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe440019e50 con 0x5613e4932230 2024-04-21T08:49:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.290+0000 7fe426ffd640 1 -- 172.21.15.43:0/2915237429 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe4380895f0 con 0x5613e4932230 2024-04-21T08:49:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.290+0000 7fe450d8d740 10 client.20652 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.290+0000 7fe450d8d740 1 -- 172.21.15.43:0/2915237429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5613e45690f0 con 0x5613e4932230 2024-04-21T08:49:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.290+0000 7fe426ffd640 1 -- 172.21.15.43:0/2915237429 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe440019e50 con 0x5613e4932230 2024-04-21T08:49:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.291+0000 7fe426ffd640 1 -- 172.21.15.43:0/2915237429 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe4380bb150 con 0x5613e4932230 2024-04-21T08:49:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.291+0000 7fe450d8d740 10 client.20652 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.291+0000 7fe450d8d740 10 client.20652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.291+0000 7fe450d8d740 20 client.20652 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.291+0000 7fe450d8d740 1 --2- 172.21.15.43:0/2915237429 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5613e494fed0 0x5613e4952390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.291+0000 7fe450d8d740 4 client.20652 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.291+0000 7fe450d8d740 1 -- 172.21.15.43:0/2915237429 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613e48a4c00 con 0x5613e494fed0 2024-04-21T08:49:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.291+0000 7fe446ffd640 1 --2- 172.21.15.43:0/2915237429 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5613e494fed0 0x5613e4952390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.291+0000 7fe446ffd640 1 --2- 172.21.15.43:0/2915237429 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5613e494fed0 0x5613e4952390 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=318d759b361305ee server_cookie=a4364bf640938cbd in_seq=0 out_seq=0 2024-04-21T08:49:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.291+0000 7fe426ffd640 10 client.20652 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.292+0000 7fe426ffd640 1 -- 172.21.15.43:0/2915237429 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5613e48a4c00 con 0x5613e494fed0 2024-04-21T08:49:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.292+0000 7fe426ffd640 10 client.20652 handle_command_reply: tid=0 2024-04-21T08:49:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.292+0000 7fe450d8d740 1 client.20652 shutdown 2024-04-21T08:49:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.292+0000 7fe450d8d740 1 -- 172.21.15.43:0/2915237429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e4932230 msgr2=0x5613e493ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.292+0000 7fe450d8d740 1 --2- 172.21.15.43:0/2915237429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e4932230 0x5613e493ce60 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fe438005ec0 tx=0x7fe43803b940 comp rx=0 tx=0).stop 2024-04-21T08:49:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.292+0000 7fe450d8d740 1 -- 172.21.15.43:0/2915237429 shutdown_connections 2024-04-21T08:49:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.292+0000 7fe450d8d740 1 --2- 172.21.15.43:0/2915237429 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5613e494fed0 0x5613e4952390 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.292+0000 7fe450d8d740 1 --2- 172.21.15.43:0/2915237429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613e493fe90 0x5613e493d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.292+0000 7fe450d8d740 1 --2- 172.21.15.43:0/2915237429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613e4932b50 0x5613e493d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.292+0000 7fe426ffd640 0 client.20652 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:34.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.292+0000 7fe450d8d740 1 --2- 172.21.15.43:0/2915237429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e4932230 0x5613e493ce60 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.292+0000 7fe450d8d740 1 -- 172.21.15.43:0/2915237429 >> 172.21.15.43:0/2915237429 conn(0x5613e492ccb0 msgr2=0x5613e492ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.292+0000 7fe450d8d740 1 -- 172.21.15.43:0/2915237429 shutdown_connections 2024-04-21T08:49:34.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.292+0000 7fe450d8d740 1 -- 172.21.15.43:0/2915237429 wait complete. 2024-04-21T08:49:34.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.292+0000 7fe450d8d740 20 client.20652 trim_cache size 0 max 16384 2024-04-21T08:49:34.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.310+0000 7fe450d8d740 1 Processor -- start 2024-04-21T08:49:34.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.310+0000 7fe450d8d740 1 -- start start 2024-04-21T08:49:34.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.310+0000 7fe450d8d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613e4932230 0x5613e493f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.310+0000 7fe450d8d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613e4932b50 0x5613e494bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.310+0000 7fe450d8d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e493fe90 0x5613e492d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.310+0000 7fe450d8d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe4481ad350 con 0x5613e493fe90 2024-04-21T08:49:34.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.310+0000 7fe450d8d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4481aee90 con 0x5613e4932b50 2024-04-21T08:49:34.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.310+0000 7fe450d8d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe4481ada20 con 0x5613e4932230 2024-04-21T08:49:34.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.310+0000 7fe447fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e493fe90 0x5613e492d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.310+0000 7fe447fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e493fe90 0x5613e492d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47860/0 (socket says 172.21.15.43:47860) 2024-04-21T08:49:34.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.310+0000 7fe447fff640 1 -- 172.21.15.43:0/2702634125 learned_addr learned my addr 172.21.15.43:0/2702634125 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:34.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.311+0000 7fe4477fe640 1 --2- 172.21.15.43:0/2702634125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613e4932230 0x5613e493f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.311+0000 7fe446ffd640 1 --2- 172.21.15.43:0/2702634125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613e4932b50 0x5613e494bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.311+0000 7fe447fff640 1 -- 172.21.15.43:0/2702634125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613e4932230 msgr2=0x5613e493f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.311+0000 7fe447fff640 1 --2- 172.21.15.43:0/2702634125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613e4932230 0x5613e493f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.311+0000 7fe447fff640 1 -- 172.21.15.43:0/2702634125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613e4932b50 msgr2=0x5613e494bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.311+0000 7fe447fff640 1 --2- 172.21.15.43:0/2702634125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613e4932b50 0x5613e494bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.311+0000 7fe447fff640 1 -- 172.21.15.43:0/2702634125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613e493b3b0 con 0x5613e493fe90 2024-04-21T08:49:34.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.311+0000 7fe447fff640 1 --2- 172.21.15.43:0/2702634125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e493fe90 0x5613e492d090 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7fe440004a70 tx=0x7fe440004aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef806e51f3a32bb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.311+0000 7fe42d7fa640 1 -- 172.21.15.43:0/2702634125 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440031070 con 0x5613e493fe90 2024-04-21T08:49:34.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.311+0000 7fe42d7fa640 1 -- 172.21.15.43:0/2702634125 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe44000aaf0 con 0x5613e493fe90 2024-04-21T08:49:34.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.311+0000 7fe42d7fa640 1 -- 172.21.15.43:0/2702634125 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44002cb10 con 0x5613e493fe90 2024-04-21T08:49:34.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.312+0000 7fe450d8d740 1 -- 172.21.15.43:0/2702634125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e493fe90 msgr2=0x5613e492d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.312+0000 7fe450d8d740 1 --2- 172.21.15.43:0/2702634125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e493fe90 0x5613e492d090 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7fe440004a70 tx=0x7fe440004aa0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.312+0000 7fe450d8d740 1 -- 172.21.15.43:0/2702634125 shutdown_connections 2024-04-21T08:49:34.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.312+0000 7fe450d8d740 1 --2- 172.21.15.43:0/2702634125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e493fe90 0x5613e492d090 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.312+0000 7fe450d8d740 1 --2- 172.21.15.43:0/2702634125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613e4932b50 0x5613e494bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.312+0000 7fe450d8d740 1 --2- 172.21.15.43:0/2702634125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613e4932230 0x5613e493f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.312+0000 7fe450d8d740 1 -- 172.21.15.43:0/2702634125 >> 172.21.15.43:0/2702634125 conn(0x5613e492ccb0 msgr2=0x5613e492e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.312+0000 7fe450d8d740 1 -- 172.21.15.43:0/2702634125 shutdown_connections 2024-04-21T08:49:34.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.312+0000 7fe450d8d740 1 -- 172.21.15.43:0/2702634125 wait complete. 2024-04-21T08:49:34.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:34 smithi043 ceph-mon[27978]: pgmap v331: 129 pgs: 129 active+clean; 2.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:49:34.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.312+0000 7fe450d8d740 1 -- start start 2024-04-21T08:49:34.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.312+0000 7fe450d8d740 1 Processor -- start 2024-04-21T08:49:34.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.312+0000 7fe450d8d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613e4932230 0x5613e4943a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.312+0000 7fe450d8d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613e4932b50 0x5613e4943f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.312+0000 7fe450d8d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e493fe90 0x5613e49444d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.312+0000 7fe450d8d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe4481ae7c0 con 0x5613e493fe90 2024-04-21T08:49:34.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.312+0000 7fe450d8d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4481ae0f0 con 0x5613e4932b50 2024-04-21T08:49:34.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.312+0000 7fe450d8d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe4481ab350 con 0x5613e4932230 2024-04-21T08:49:34.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.312+0000 7fe450d8d740 10 client.0 _ll_register_callbacks cb 0x7fe4380057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:34.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.313+0000 7fe447fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e493fe90 0x5613e49444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.313+0000 7fe447fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e493fe90 0x5613e49444d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47862/0 (socket says 172.21.15.43:47862) 2024-04-21T08:49:34.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.313+0000 7fe447fff640 1 -- 172.21.15.43:0/466953563 learned_addr learned my addr 172.21.15.43:0/466953563 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:34.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.313+0000 7fe446ffd640 1 --2- 172.21.15.43:0/466953563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613e4932b50 0x5613e4943f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.313+0000 7fe4477fe640 1 --2- 172.21.15.43:0/466953563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613e4932230 0x5613e4943a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.313+0000 7fe447fff640 1 -- 172.21.15.43:0/466953563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613e4932230 msgr2=0x5613e4943a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.313+0000 7fe447fff640 1 --2- 172.21.15.43:0/466953563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613e4932230 0x5613e4943a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.313+0000 7fe447fff640 1 -- 172.21.15.43:0/466953563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613e4932b50 msgr2=0x5613e4943f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.313+0000 7fe447fff640 1 --2- 172.21.15.43:0/466953563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613e4932b50 0x5613e4943f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.313+0000 7fe447fff640 1 -- 172.21.15.43:0/466953563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613e49b5510 con 0x5613e493fe90 2024-04-21T08:49:34.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.313+0000 7fe447fff640 1 --2- 172.21.15.43:0/466953563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e493fe90 0x5613e49444d0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fe44000a5e0 tx=0x7fe440012440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3844c47b746cbb37 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.313+0000 7fe426ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:34.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.313+0000 7fe426ffd640 1 -- 172.21.15.43:0/466953563 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440031030 con 0x5613e493fe90 2024-04-21T08:49:34.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.313+0000 7fe426ffd640 1 -- 172.21.15.43:0/466953563 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fe440029040 con 0x5613e493fe90 2024-04-21T08:49:34.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.313+0000 7fe450d8d740 1 -- 172.21.15.43:0/466953563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe44000aaf0 con 0x5613e493fe90 2024-04-21T08:49:34.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.313+0000 7fe426ffd640 1 -- 172.21.15.43:0/466953563 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe440019cc0 con 0x5613e493fe90 2024-04-21T08:49:34.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.313+0000 7fe450d8d740 10 client.20658 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:34.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.313+0000 7fe450d8d740 1 -- 172.21.15.43:0/466953563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5613e49b6730 con 0x5613e493fe90 2024-04-21T08:49:34.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.314+0000 7fe426ffd640 1 -- 172.21.15.43:0/466953563 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe44000aaf0 con 0x5613e493fe90 2024-04-21T08:49:34.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.314+0000 7fe426ffd640 1 -- 172.21.15.43:0/466953563 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe44001b070 con 0x5613e493fe90 2024-04-21T08:49:34.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.314+0000 7fe450d8d740 10 client.20658 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:34.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.314+0000 7fe450d8d740 10 client.20658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:34.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.314+0000 7fe450d8d740 20 client.20658 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:34.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.314+0000 7fe450d8d740 1 --2- 172.21.15.43:0/466953563 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5613e49b6c50 0x5613e49b9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.314+0000 7fe450d8d740 4 client.20658 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:49:34.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.314+0000 7fe450d8d740 1 -- 172.21.15.43:0/466953563 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5613e489ea70 con 0x5613e49b6c50 2024-04-21T08:49:34.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.314+0000 7fe4477fe640 1 --2- 172.21.15.43:0/466953563 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5613e49b6c50 0x5613e49b9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.314+0000 7fe4477fe640 1 --2- 172.21.15.43:0/466953563 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5613e49b6c50 0x5613e49b9040 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=755ec75bed6b753b server_cookie=b8dcbb40e3a32e9a in_seq=0 out_seq=0 2024-04-21T08:49:34.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.314+0000 7fe426ffd640 10 client.20658 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:34.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.315+0000 7fe426ffd640 1 -- 172.21.15.43:0/466953563 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5613e489ea70 con 0x5613e49b6c50 2024-04-21T08:49:34.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.315+0000 7fe426ffd640 10 client.20658 handle_command_reply: tid=0 2024-04-21T08:49:34.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.315+0000 7fe450d8d740 1 client.20658 shutdown 2024-04-21T08:49:34.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.315+0000 7fe450d8d740 1 -- 172.21.15.43:0/466953563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e493fe90 msgr2=0x5613e49444d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.315+0000 7fe450d8d740 1 --2- 172.21.15.43:0/466953563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e493fe90 0x5613e49444d0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fe44000a5e0 tx=0x7fe440012440 comp rx=0 tx=0).stop 2024-04-21T08:49:34.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.315+0000 7fe450d8d740 1 -- 172.21.15.43:0/466953563 shutdown_connections 2024-04-21T08:49:34.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.315+0000 7fe450d8d740 1 --2- 172.21.15.43:0/466953563 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5613e49b6c50 0x5613e49b9040 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.315+0000 7fe450d8d740 1 --2- 172.21.15.43:0/466953563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613e493fe90 0x5613e49444d0 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.315+0000 7fe450d8d740 1 --2- 172.21.15.43:0/466953563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613e4932b50 0x5613e4943f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.315+0000 7fe426ffd640 0 client.20658 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:34.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.315+0000 7fe450d8d740 1 --2- 172.21.15.43:0/466953563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613e4932230 0x5613e4943a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.315+0000 7fe450d8d740 1 -- 172.21.15.43:0/466953563 >> 172.21.15.43:0/466953563 conn(0x5613e492ccb0 msgr2=0x5613e492e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.315+0000 7fe450d8d740 1 -- 172.21.15.43:0/466953563 shutdown_connections 2024-04-21T08:49:34.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.315+0000 7fe450d8d740 1 -- 172.21.15.43:0/466953563 wait complete. 2024-04-21T08:49:34.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.315+0000 7fe450d8d740 20 client.20658 trim_cache size 0 max 16384 2024-04-21T08:49:34.322 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:49:34.322 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:49:34.322 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:49:34.322 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:49:34.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.315+0000 7fe44e0c1640 1 -- 172.21.15.43:0/891398271 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe4300a3d20 msgr2=0x7fe4300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.315+0000 7fe44e0c1640 1 --2- 172.21.15.43:0/891398271 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe4300a3d20 0x7fe4300a61e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe44000afd0 tx=0x7fe44001a000 comp rx=0 tx=0).stop 2024-04-21T08:49:34.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.315+0000 7fe44e0c1640 1 -- 172.21.15.43:0/891398271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4481031b0 msgr2=0x7fe448198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.315+0000 7fe44e0c1640 1 --2- 172.21.15.43:0/891398271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4481031b0 0x7fe448198730 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fe438005940 tx=0x7fe43802d910 comp rx=0 tx=0).stop 2024-04-21T08:49:34.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.316+0000 7fe44e0c1640 1 -- 172.21.15.43:0/891398271 shutdown_connections 2024-04-21T08:49:34.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.316+0000 7fe44e0c1640 1 --2- 172.21.15.43:0/891398271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe44810c580 0x7fe44819cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.316+0000 7fe44e0c1640 1 --2- 172.21.15.43:0/891398271 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe4300a3d20 0x7fe4300a61e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.316+0000 7fe44e0c1640 1 --2- 172.21.15.43:0/891398271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe448103ad0 0x7fe448198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.316+0000 7fe44e0c1640 1 --2- 172.21.15.43:0/891398271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4481031b0 0x7fe448198730 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.316+0000 7fe44e0c1640 1 -- 172.21.15.43:0/891398271 >> 172.21.15.43:0/891398271 conn(0x7fe4480771d0 msgr2=0x7fe4480fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.316+0000 7fe44e0c1640 1 -- 172.21.15.43:0/891398271 shutdown_connections 2024-04-21T08:49:34.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.316+0000 7fe44e0c1640 1 -- 172.21.15.43:0/891398271 wait complete. 2024-04-21T08:49:34.326 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:49:34.326 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:49:34.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.429+0000 7f97dbd2b640 1 -- 172.21.15.43:0/430295264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97d4069a80 msgr2=0x7f97d410eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.429+0000 7f97dbd2b640 1 --2- 172.21.15.43:0/430295264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97d4069a80 0x7f97d410eda0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009ca0 tx=0x7f97cc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.429+0000 7f97dbd2b640 1 -- 172.21.15.43:0/430295264 shutdown_connections 2024-04-21T08:49:34.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.429+0000 7f97dbd2b640 1 --2- 172.21.15.43:0/430295264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97d4069a80 0x7f97d410eda0 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.430+0000 7f97dbd2b640 1 --2- 172.21.15.43:0/430295264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97d40690e0 0x7f97d4069540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.430+0000 7f97dbd2b640 1 --2- 172.21.15.43:0/430295264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97d4107fe0 0x7f97d41083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.430+0000 7f97dbd2b640 1 -- 172.21.15.43:0/430295264 >> 172.21.15.43:0/430295264 conn(0x7f97d4077e30 msgr2=0x7f97d4078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.430+0000 7f97dbd2b640 1 -- 172.21.15.43:0/430295264 shutdown_connections 2024-04-21T08:49:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.430+0000 7f97dbd2b640 1 -- 172.21.15.43:0/430295264 wait complete. 2024-04-21T08:49:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.430+0000 7f97dbd2b640 1 Processor -- start 2024-04-21T08:49:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.430+0000 7f97dbd2b640 1 -- start start 2024-04-21T08:49:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.430+0000 7f97dbd2b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97d40690e0 0x7f97d4101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.430+0000 7f97dbd2b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97d4069a80 0x7f97d41017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.430+0000 7f97dbd2b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97d4107fe0 0x7f97d4101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.430+0000 7f97dbd2b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f97d4112170 con 0x7f97d40690e0 2024-04-21T08:49:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.430+0000 7f97dbd2b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f97d4111ff0 con 0x7f97d4069a80 2024-04-21T08:49:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.430+0000 7f97dbd2b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f97d41122f0 con 0x7f97d4107fe0 2024-04-21T08:49:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.431+0000 7f97d9ac7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97d40690e0 0x7f97d4101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.431+0000 7f97d9ac7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97d40690e0 0x7f97d4101270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47876/0 (socket says 172.21.15.43:47876) 2024-04-21T08:49:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.431+0000 7f97d9ac7640 1 -- 172.21.15.43:0/3899165181 learned_addr learned my addr 172.21.15.43:0/3899165181 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.431+0000 7f97d92c6640 1 --2- 172.21.15.43:0/3899165181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97d4069a80 0x7f97d41017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.431+0000 7f97d9ac7640 1 -- 172.21.15.43:0/3899165181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97d4107fe0 msgr2=0x7f97d4101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.431+0000 7f97d9ac7640 1 --2- 172.21.15.43:0/3899165181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97d4107fe0 0x7f97d4101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.431+0000 7f97d9ac7640 1 -- 172.21.15.43:0/3899165181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97d4069a80 msgr2=0x7f97d41017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.431+0000 7f97d9ac7640 1 --2- 172.21.15.43:0/3899165181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97d4069a80 0x7f97d41017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.431+0000 7f97d9ac7640 1 -- 172.21.15.43:0/3899165181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d41a6b90 con 0x7f97d40690e0 2024-04-21T08:49:34.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.431+0000 7f97d9ac7640 1 --2- 172.21.15.43:0/3899165181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97d40690e0 0x7f97d4101270 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f97d000d770 tx=0x7f97d000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.431+0000 7f97caffd640 1 -- 172.21.15.43:0/3899165181 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0017070 con 0x7f97d40690e0 2024-04-21T08:49:34.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.431+0000 7f97dbd2b640 1 -- 172.21.15.43:0/3899165181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d41a6e80 con 0x7f97d40690e0 2024-04-21T08:49:34.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.431+0000 7f97caffd640 1 -- 172.21.15.43:0/3899165181 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97d00054d0 con 0x7f97d40690e0 2024-04-21T08:49:34.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.431+0000 7f97caffd640 1 -- 172.21.15.43:0/3899165181 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0004c30 con 0x7f97d40690e0 2024-04-21T08:49:34.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.431+0000 7f97dbd2b640 1 -- 172.21.15.43:0/3899165181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d41a7340 con 0x7f97d40690e0 2024-04-21T08:49:34.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.432+0000 7f97dbd2b640 1 -- 172.21.15.43:0/3899165181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97d4068de0 con 0x7f97d40690e0 2024-04-21T08:49:34.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.434+0000 7f97caffd640 1 -- 172.21.15.43:0/3899165181 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f97d0004dd0 con 0x7f97d40690e0 2024-04-21T08:49:34.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.434+0000 7f97caffd640 1 --2- 172.21.15.43:0/3899165181 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f97c00a3d20 0x7f97c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.434+0000 7f97d92c6640 1 --2- 172.21.15.43:0/3899165181 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f97c00a3d20 0x7f97c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.434+0000 7f97caffd640 1 -- 172.21.15.43:0/3899165181 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f97d00d0af0 con 0x7f97d40690e0 2024-04-21T08:49:34.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.435+0000 7f97d92c6640 1 --2- 172.21.15.43:0/3899165181 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f97c00a3d20 0x7f97c00a61e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f97d4102d00 tx=0x7f97c4005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.442+0000 7f97caffd640 1 -- 172.21.15.43:0/3899165181 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f97d009cd70 con 0x7f97d40690e0 2024-04-21T08:49:34.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:34 smithi175 ceph-mon[31920]: pgmap v331: 129 pgs: 129 active+clean; 2.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:49:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.476+0000 7fa8ddfd5640 1 -- 172.21.15.43:0/631573336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8d8103200 msgr2=0x7fa8d81035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.476+0000 7fa8ddfd5640 1 --2- 172.21.15.43:0/631573336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8d8103200 0x7fa8d81035e0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009900 tx=0x7fa8c002bcd0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.477+0000 7fa8ddfd5640 1 -- 172.21.15.43:0/631573336 shutdown_connections 2024-04-21T08:49:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.477+0000 7fa8ddfd5640 1 --2- 172.21.15.43:0/631573336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8d810c540 0x7fa8d810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.477+0000 7fa8ddfd5640 1 --2- 172.21.15.43:0/631573336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8d8103b20 0x7fa8d810c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.477+0000 7fa8ddfd5640 1 --2- 172.21.15.43:0/631573336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8d8103200 0x7fa8d81035e0 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.477+0000 7fa8ddfd5640 1 -- 172.21.15.43:0/631573336 >> 172.21.15.43:0/631573336 conn(0x7fa8d8077fd0 msgr2=0x7fa8d80783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.477+0000 7fa8ddfd5640 1 -- 172.21.15.43:0/631573336 shutdown_connections 2024-04-21T08:49:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.477+0000 7fa8ddfd5640 1 -- 172.21.15.43:0/631573336 wait complete. 2024-04-21T08:49:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.477+0000 7fa8ddfd5640 1 Processor -- start 2024-04-21T08:49:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.477+0000 7fa8ddfd5640 1 -- start start 2024-04-21T08:49:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.478+0000 7fa8ddfd5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8d8103200 0x7fa8d8198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.478+0000 7fa8ddfd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8d8103b20 0x7fa8d8198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.478+0000 7fa8ddfd5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8d810c540 0x7fa8d819cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.478+0000 7fa8ddfd5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d81121f0 con 0x7fa8d8103200 2024-04-21T08:49:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.478+0000 7fa8ddfd5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d8112070 con 0x7fa8d8103b20 2024-04-21T08:49:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.478+0000 7fa8ddfd5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d8112370 con 0x7fa8d810c540 2024-04-21T08:49:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.478+0000 7fa8d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8d8103200 0x7fa8d8198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.478+0000 7fa8d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8d8103200 0x7fa8d8198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47886/0 (socket says 172.21.15.43:47886) 2024-04-21T08:49:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.478+0000 7fa8d77fe640 1 -- 172.21.15.43:0/1083276758 learned_addr learned my addr 172.21.15.43:0/1083276758 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.478+0000 7fa8d7fff640 1 --2- 172.21.15.43:0/1083276758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8d810c540 0x7fa8d819cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.478+0000 7fa8d6ffd640 1 --2- 172.21.15.43:0/1083276758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8d8103b20 0x7fa8d8198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.478+0000 7fa8d77fe640 1 -- 172.21.15.43:0/1083276758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8d810c540 msgr2=0x7fa8d819cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.478+0000 7fa8d77fe640 1 --2- 172.21.15.43:0/1083276758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8d810c540 0x7fa8d819cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.478+0000 7fa8d77fe640 1 -- 172.21.15.43:0/1083276758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8d8103b20 msgr2=0x7fa8d8198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.478+0000 7fa8d77fe640 1 --2- 172.21.15.43:0/1083276758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8d8103b20 0x7fa8d8198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.478+0000 7fa8d77fe640 1 -- 172.21.15.43:0/1083276758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d819d710 con 0x7fa8d8103200 2024-04-21T08:49:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.478+0000 7fa8d6ffd640 1 --2- 172.21.15.43:0/1083276758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8d8103b20 0x7fa8d8198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.478+0000 7fa8d77fe640 1 --2- 172.21.15.43:0/1083276758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8d8103200 0x7fa8d8198770 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0000c00 tx=0x7fa8c002c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.478+0000 7fa8d4ff9640 1 -- 172.21.15.43:0/1083276758 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0043020 con 0x7fa8d8103200 2024-04-21T08:49:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.478+0000 7fa8d4ff9640 1 -- 172.21.15.43:0/1083276758 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8c002dc60 con 0x7fa8d8103200 2024-04-21T08:49:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.478+0000 7fa8d4ff9640 1 -- 172.21.15.43:0/1083276758 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c00039e0 con 0x7fa8d8103200 2024-04-21T08:49:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.479+0000 7fa8ddfd5640 1 -- 172.21.15.43:0/1083276758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d819d9a0 con 0x7fa8d8103200 2024-04-21T08:49:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.479+0000 7fa8ddfd5640 1 -- 172.21.15.43:0/1083276758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d81a5290 con 0x7fa8d8103200 2024-04-21T08:49:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.481+0000 7fa8d4ff9640 1 -- 172.21.15.43:0/1083276758 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa8c0046020 con 0x7fa8d8103200 2024-04-21T08:49:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.481+0000 7fa8e0bc6740 1 Processor -- start 2024-04-21T08:49:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.481+0000 7fa8e0bc6740 1 -- start start 2024-04-21T08:49:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.481+0000 7fa8e0bc6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f54baee8d0 0x55f54baed620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.481+0000 7fa8e0bc6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baedb60 0x55f54baedfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.481+0000 7fa8e0bc6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f54baf70e0 0x55f54baf94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.481+0000 7fa8e0bc6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f54ba528d0 con 0x55f54baedb60 2024-04-21T08:49:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.481+0000 7fa8e0bc6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f54b9efe80 con 0x55f54baee8d0 2024-04-21T08:49:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.481+0000 7fa8e0bc6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f54ba4e890 con 0x55f54baf70e0 2024-04-21T08:49:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.481+0000 7fa8d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baedb60 0x55f54baedfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.481+0000 7fa8d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baedb60 0x55f54baedfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47898/0 (socket says 172.21.15.43:47898) 2024-04-21T08:49:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.481+0000 7fa8d77fe640 1 -- 172.21.15.43:0/2800035423 learned_addr learned my addr 172.21.15.43:0/2800035423 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.482+0000 7fa8d7fff640 1 --2- 172.21.15.43:0/2800035423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f54baee8d0 0x55f54baed620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.482+0000 7fa8d4ff9640 1 --2- 172.21.15.43:0/1083276758 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa8c40a3e40 0x7fa8c40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.482+0000 7fa8d6ffd640 1 --2- 172.21.15.43:0/2800035423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f54baf70e0 0x55f54baf94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.482+0000 7fa8d77fe640 1 -- 172.21.15.43:0/2800035423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f54baf70e0 msgr2=0x55f54baf94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.482+0000 7fa8d77fe640 1 --2- 172.21.15.43:0/2800035423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f54baf70e0 0x55f54baf94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.482+0000 7fa8d77fe640 1 -- 172.21.15.43:0/2800035423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f54baee8d0 msgr2=0x55f54baed620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.482+0000 7fa8d77fe640 1 --2- 172.21.15.43:0/2800035423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f54baee8d0 0x55f54baed620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.482+0000 7fa8d77fe640 1 -- 172.21.15.43:0/2800035423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f54ba11320 con 0x55f54baedb60 2024-04-21T08:49:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.482+0000 7fa8d7fff640 1 --2- 172.21.15.43:0/1083276758 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa8c40a3e40 0x7fa8c40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.482+0000 7fa8d4ff9640 1 -- 172.21.15.43:0/1083276758 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fa8c00f2ad0 con 0x7fa8d8103200 2024-04-21T08:49:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.482+0000 7fa8d77fe640 1 --2- 172.21.15.43:0/2800035423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baedb60 0x55f54baedfe0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00efe80 tx=0x7fa8c0088d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8676b5099cf686b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.482+0000 7fa8cdffb640 1 -- 172.21.15.43:0/2800035423 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c00be2a0 con 0x55f54baedb60 2024-04-21T08:49:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.482+0000 7fa8cdffb640 1 -- 172.21.15.43:0/2800035423 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8c002de10 con 0x55f54baedb60 2024-04-21T08:49:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.482+0000 7fa8cdffb640 1 -- 172.21.15.43:0/2800035423 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0090450 con 0x55f54baedb60 2024-04-21T08:49:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.483+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/2800035423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baedb60 msgr2=0x55f54baedfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.483+0000 7fa8d7fff640 1 --2- 172.21.15.43:0/1083276758 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa8c40a3e40 0x7fa8c40a6300 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0005fd0 tx=0x7fa8d0004a40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.483+0000 7fa8e0bc6740 1 --2- 172.21.15.43:0/2800035423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baedb60 0x55f54baedfe0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00efe80 tx=0x7fa8c0088d60 comp rx=0 tx=0).stop 2024-04-21T08:49:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.483+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/2800035423 shutdown_connections 2024-04-21T08:49:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.483+0000 7fa8e0bc6740 1 --2- 172.21.15.43:0/2800035423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f54baf70e0 0x55f54baf94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.483+0000 7fa8e0bc6740 1 --2- 172.21.15.43:0/2800035423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baedb60 0x55f54baedfe0 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.483+0000 7fa8e0bc6740 1 --2- 172.21.15.43:0/2800035423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f54baee8d0 0x55f54baed620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.483+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/2800035423 >> 172.21.15.43:0/2800035423 conn(0x55f54bab4c90 msgr2=0x55f54bae8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.483+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/2800035423 shutdown_connections 2024-04-21T08:49:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.483+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/2800035423 wait complete. 2024-04-21T08:49:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.483+0000 7fa8e0bc6740 1 -- start start 2024-04-21T08:49:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.484+0000 7fa8e0bc6740 1 Processor -- start 2024-04-21T08:49:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.484+0000 7fa8e0bc6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baedb60 0x55f54baf6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.484+0000 7fa8e0bc6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f54baee8d0 0x55f54baf3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.484+0000 7fa8e0bc6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f54baf70e0 0x55f54baf42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.484+0000 7fa8e0bc6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f54baf9bd0 con 0x55f54baedb60 2024-04-21T08:49:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.484+0000 7fa8e0bc6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f54b9daeb0 con 0x55f54baee8d0 2024-04-21T08:49:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.484+0000 7fa8e0bc6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f54baf9d50 con 0x55f54baf70e0 2024-04-21T08:49:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.484+0000 7fa8e0bc6740 10 client.0 _ll_register_callbacks cb 0x55f54ba07b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.484+0000 7fa8d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baedb60 0x55f54baf6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.484+0000 7fa8d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baedb60 0x55f54baf6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47908/0 (socket says 172.21.15.43:47908) 2024-04-21T08:49:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.484+0000 7fa8d77fe640 1 -- 172.21.15.43:0/4050491936 learned_addr learned my addr 172.21.15.43:0/4050491936 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.484+0000 7fa8d6ffd640 1 --2- 172.21.15.43:0/4050491936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f54baee8d0 0x55f54baf3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.484+0000 7fa8d7fff640 1 --2- 172.21.15.43:0/4050491936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f54baf70e0 0x55f54baf42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.484+0000 7fa8d77fe640 1 -- 172.21.15.43:0/4050491936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f54baf70e0 msgr2=0x55f54baf42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.484+0000 7fa8d77fe640 1 --2- 172.21.15.43:0/4050491936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f54baf70e0 0x55f54baf42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.484+0000 7fa8d77fe640 1 -- 172.21.15.43:0/4050491936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f54baee8d0 msgr2=0x55f54baf3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.484+0000 7fa8d77fe640 1 --2- 172.21.15.43:0/4050491936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f54baee8d0 0x55f54baf3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.485+0000 7fa8d77fe640 1 -- 172.21.15.43:0/4050491936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f54b765210 con 0x55f54baedb60 2024-04-21T08:49:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.485+0000 7fa8d77fe640 1 --2- 172.21.15.43:0/4050491936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baedb60 0x55f54baf6a00 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00be460 tx=0x7fa8c00056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40e746411c02a80 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.485+0000 7fa8b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.485+0000 7fa8b6ffd640 1 -- 172.21.15.43:0/4050491936 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c00910b0 con 0x55f54baedb60 2024-04-21T08:49:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.485+0000 7fa8b6ffd640 1 -- 172.21.15.43:0/4050491936 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fa8c00d2ec0 con 0x55f54baedb60 2024-04-21T08:49:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.485+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/4050491936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f54ba66880 con 0x55f54baedb60 2024-04-21T08:49:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.485+0000 7fa8b6ffd640 1 -- 172.21.15.43:0/4050491936 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8c00be0c0 con 0x55f54baedb60 2024-04-21T08:49:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.485+0000 7fa8e0bc6740 10 client.20676 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.485+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/4050491936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f54b831620 con 0x55f54baedb60 2024-04-21T08:49:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.485+0000 7fa8b6ffd640 1 -- 172.21.15.43:0/4050491936 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f54ba66880 con 0x55f54baedb60 2024-04-21T08:49:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.485+0000 7fa8b6ffd640 1 -- 172.21.15.43:0/4050491936 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa8c00badd0 con 0x55f54baedb60 2024-04-21T08:49:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.485+0000 7fa8e0bc6740 10 client.20676 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.485+0000 7fa8e0bc6740 10 client.20676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.486+0000 7fa8e0bc6740 20 client.20676 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.486+0000 7fa8e0bc6740 1 --2- 172.21.15.43:0/4050491936 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f54bb06be0 0x55f54bb08fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.486+0000 7fa8e0bc6740 4 client.20676 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.486+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/4050491936 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f54b9f23d0 con 0x55f54bb06be0 2024-04-21T08:49:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.486+0000 7fa8d6ffd640 1 --2- 172.21.15.43:0/4050491936 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f54bb06be0 0x55f54bb08fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.486+0000 7fa8b6ffd640 10 client.20676 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.486+0000 7fa8d6ffd640 1 --2- 172.21.15.43:0/4050491936 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f54bb06be0 0x55f54bb08fd0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=139fcdfe729b1cd9 server_cookie=38a77d0d15688a2b in_seq=0 out_seq=0 2024-04-21T08:49:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.487+0000 7fa8b6ffd640 1 -- 172.21.15.43:0/4050491936 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f54b9f23d0 con 0x55f54bb06be0 2024-04-21T08:49:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.487+0000 7fa8b6ffd640 10 client.20676 handle_command_reply: tid=0 2024-04-21T08:49:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.487+0000 7fa8e0bc6740 1 client.20676 shutdown 2024-04-21T08:49:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.487+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/4050491936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baedb60 msgr2=0x55f54baf6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.487+0000 7fa8e0bc6740 1 --2- 172.21.15.43:0/4050491936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baedb60 0x55f54baf6a00 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00be460 tx=0x7fa8c00056e0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.487+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/4050491936 shutdown_connections 2024-04-21T08:49:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.487+0000 7fa8e0bc6740 1 --2- 172.21.15.43:0/4050491936 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f54bb06be0 0x55f54bb08fd0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.487+0000 7fa8e0bc6740 1 --2- 172.21.15.43:0/4050491936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f54baf70e0 0x55f54baf42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.487+0000 7fa8e0bc6740 1 --2- 172.21.15.43:0/4050491936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f54baee8d0 0x55f54baf3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.487+0000 7fa8b6ffd640 0 client.20676 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.487+0000 7fa8e0bc6740 1 --2- 172.21.15.43:0/4050491936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baedb60 0x55f54baf6a00 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.487+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/4050491936 >> 172.21.15.43:0/4050491936 conn(0x55f54bab4c90 msgr2=0x55f54baea5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.487+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/4050491936 shutdown_connections 2024-04-21T08:49:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.487+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/4050491936 wait complete. 2024-04-21T08:49:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.487+0000 7fa8e0bc6740 20 client.20676 trim_cache size 0 max 16384 2024-04-21T08:49:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.502+0000 7fa8e0bc6740 1 Processor -- start 2024-04-21T08:49:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.502+0000 7fa8e0bc6740 1 -- start start 2024-04-21T08:49:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.502+0000 7fa8e0bc6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f54bab4c90 0x55f54baef5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.502+0000 7fa8e0bc6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f54baedb60 0x55f54bb02980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.502+0000 7fa8e0bc6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baee8d0 0x55f54baf64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.502+0000 7fa8e0bc6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d81af490 con 0x55f54baee8d0 2024-04-21T08:49:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.502+0000 7fa8e0bc6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d81b0fd0 con 0x55f54bab4c90 2024-04-21T08:49:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.502+0000 7fa8e0bc6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d81afb60 con 0x55f54baedb60 2024-04-21T08:49:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.502+0000 7fa8d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baee8d0 0x55f54baf64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.502+0000 7fa8d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baee8d0 0x55f54baf64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47920/0 (socket says 172.21.15.43:47920) 2024-04-21T08:49:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.502+0000 7fa8d7fff640 1 -- 172.21.15.43:0/2753933128 learned_addr learned my addr 172.21.15.43:0/2753933128 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.503+0000 7fa8d77fe640 1 --2- 172.21.15.43:0/2753933128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f54bab4c90 0x55f54baef5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.503+0000 7fa8d6ffd640 1 --2- 172.21.15.43:0/2753933128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f54baedb60 0x55f54bb02980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.503+0000 7fa8d7fff640 1 -- 172.21.15.43:0/2753933128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f54baedb60 msgr2=0x55f54bb02980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.503+0000 7fa8d7fff640 1 --2- 172.21.15.43:0/2753933128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f54baedb60 0x55f54bb02980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.503+0000 7fa8d7fff640 1 -- 172.21.15.43:0/2753933128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f54bab4c90 msgr2=0x55f54baef5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.503+0000 7fa8d7fff640 1 --2- 172.21.15.43:0/2753933128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f54bab4c90 0x55f54baef5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.503+0000 7fa8d7fff640 1 -- 172.21.15.43:0/2753933128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f54baece30 con 0x55f54baee8d0 2024-04-21T08:49:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.503+0000 7fa8d6ffd640 1 --2- 172.21.15.43:0/2753933128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f54baedb60 0x55f54bb02980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.503+0000 7fa8d7fff640 1 --2- 172.21.15.43:0/2753933128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baee8d0 0x55f54baf64c0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fa8d001e770 tx=0x7fa8d0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7556ac0548e6d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.503+0000 7fa8cd7fa640 1 -- 172.21.15.43:0/2753933128 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d002c070 con 0x55f54baee8d0 2024-04-21T08:49:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.503+0000 7fa8cd7fa640 1 -- 172.21.15.43:0/2753933128 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8d001cad0 con 0x55f54baee8d0 2024-04-21T08:49:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.503+0000 7fa8cd7fa640 1 -- 172.21.15.43:0/2753933128 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0027b80 con 0x55f54baee8d0 2024-04-21T08:49:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.504+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/2753933128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baee8d0 msgr2=0x55f54baf64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.504+0000 7fa8e0bc6740 1 --2- 172.21.15.43:0/2753933128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baee8d0 0x55f54baf64c0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fa8d001e770 tx=0x7fa8d0015000 comp rx=0 tx=0).stop 2024-04-21T08:49:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.504+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/2753933128 shutdown_connections 2024-04-21T08:49:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.504+0000 7fa8e0bc6740 1 --2- 172.21.15.43:0/2753933128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baee8d0 0x55f54baf64c0 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.504+0000 7fa8e0bc6740 1 --2- 172.21.15.43:0/2753933128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f54baedb60 0x55f54bb02980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.504+0000 7fa8e0bc6740 1 --2- 172.21.15.43:0/2753933128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f54bab4c90 0x55f54baef5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.504+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/2753933128 >> 172.21.15.43:0/2753933128 conn(0x55f54baf9840 msgr2=0x55f54baeeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.504+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/2753933128 shutdown_connections 2024-04-21T08:49:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.504+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/2753933128 wait complete. 2024-04-21T08:49:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.504+0000 7fa8e0bc6740 1 -- start start 2024-04-21T08:49:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.505+0000 7fa8e0bc6740 1 Processor -- start 2024-04-21T08:49:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.505+0000 7fa8e0bc6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f54bab4c90 0x55f54baf0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.505+0000 7fa8e0bc6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baedb60 0x55f54baf1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.505+0000 7fa8e0bc6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f54baee8d0 0x55f54baf1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.505+0000 7fa8e0bc6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d81b0900 con 0x55f54baedb60 2024-04-21T08:49:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.505+0000 7fa8e0bc6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d81b0230 con 0x55f54baee8d0 2024-04-21T08:49:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.505+0000 7fa8e0bc6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d81ad490 con 0x55f54bab4c90 2024-04-21T08:49:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.505+0000 7fa8e0bc6740 10 client.0 _ll_register_callbacks cb 0x7fa8c00baa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.505+0000 7fa8d6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baedb60 0x55f54baf1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.505+0000 7fa8d6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baedb60 0x55f54baf1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47930/0 (socket says 172.21.15.43:47930) 2024-04-21T08:49:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.505+0000 7fa8d6ffd640 1 -- 172.21.15.43:0/4096413478 learned_addr learned my addr 172.21.15.43:0/4096413478 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.505+0000 7fa8d77fe640 1 --2- 172.21.15.43:0/4096413478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f54bab4c90 0x55f54baf0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.513 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.505+0000 7fa8d7fff640 1 --2- 172.21.15.43:0/4096413478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f54baee8d0 0x55f54baf1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.513 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.506+0000 7fa8d6ffd640 1 -- 172.21.15.43:0/4096413478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f54bab4c90 msgr2=0x55f54baf0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.513 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.506+0000 7fa8d6ffd640 1 --2- 172.21.15.43:0/4096413478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f54bab4c90 0x55f54baf0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.506+0000 7fa8d6ffd640 1 -- 172.21.15.43:0/4096413478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f54baee8d0 msgr2=0x55f54baf1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.506+0000 7fa8d6ffd640 1 --2- 172.21.15.43:0/4096413478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f54baee8d0 0x55f54baf1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.506+0000 7fa8d6ffd640 1 -- 172.21.15.43:0/4096413478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f54bb6c110 con 0x55f54baedb60 2024-04-21T08:49:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.506+0000 7fa8d6ffd640 1 --2- 172.21.15.43:0/4096413478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baedb60 0x55f54baf1150 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8004700 tx=0x7fa8c8012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68a17494dd8d4b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.506+0000 7fa8b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.506+0000 7fa8b6ffd640 1 -- 172.21.15.43:0/4096413478 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8004460 con 0x55f54baedb60 2024-04-21T08:49:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.506+0000 7fa8b6ffd640 1 -- 172.21.15.43:0/4096413478 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fa8c80035f0 con 0x55f54baedb60 2024-04-21T08:49:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.506+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/4096413478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8d001cad0 con 0x55f54baedb60 2024-04-21T08:49:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.506+0000 7fa8b6ffd640 1 -- 172.21.15.43:0/4096413478 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8c802d050 con 0x55f54baedb60 2024-04-21T08:49:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.506+0000 7fa8e0bc6740 10 client.20682 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.506+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/4096413478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f54bb6d3d0 con 0x55f54baedb60 2024-04-21T08:49:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.506+0000 7fa8b6ffd640 1 -- 172.21.15.43:0/4096413478 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8d001cad0 con 0x55f54baedb60 2024-04-21T08:49:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.506+0000 7fa8b6ffd640 1 -- 172.21.15.43:0/4096413478 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa8c801e560 con 0x55f54baedb60 2024-04-21T08:49:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.506+0000 7fa8e0bc6740 10 client.20682 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.507+0000 7fa8e0bc6740 10 client.20682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.507+0000 7fa8e0bc6740 20 client.20682 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.507+0000 7fa8e0bc6740 1 --2- 172.21.15.43:0/4096413478 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f54baf70e0 0x55f54bb6fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.507+0000 7fa8e0bc6740 4 client.20682 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:49:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.507+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/4096413478 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f54ba602f0 con 0x55f54baf70e0 2024-04-21T08:49:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.507+0000 7fa8d77fe640 1 --2- 172.21.15.43:0/4096413478 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f54baf70e0 0x55f54bb6fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.507+0000 7fa8b6ffd640 10 client.20682 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.507+0000 7fa8d77fe640 1 --2- 172.21.15.43:0/4096413478 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f54baf70e0 0x55f54bb6fbc0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51aa65b84d502b33 server_cookie=454f7ee036e89a71 in_seq=0 out_seq=0 2024-04-21T08:49:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.521+0000 7fa8b6ffd640 1 -- 172.21.15.43:0/4096413478 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f54ba602f0 con 0x55f54baf70e0 2024-04-21T08:49:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.521+0000 7fa8b6ffd640 10 client.20682 handle_command_reply: tid=0 2024-04-21T08:49:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.521+0000 7fa8e0bc6740 1 client.20682 shutdown 2024-04-21T08:49:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.522+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/4096413478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baedb60 msgr2=0x55f54baf1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.522+0000 7fa8e0bc6740 1 --2- 172.21.15.43:0/4096413478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baedb60 0x55f54baf1150 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8004700 tx=0x7fa8c8012820 comp rx=0 tx=0).stop 2024-04-21T08:49:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.522+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/4096413478 shutdown_connections 2024-04-21T08:49:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.522+0000 7fa8e0bc6740 1 --2- 172.21.15.43:0/4096413478 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f54baf70e0 0x55f54bb6fbc0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.522+0000 7fa8e0bc6740 1 --2- 172.21.15.43:0/4096413478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f54baee8d0 0x55f54baf1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.522+0000 7fa8e0bc6740 1 --2- 172.21.15.43:0/4096413478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f54baedb60 0x55f54baf1150 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.522+0000 7fa8e0bc6740 1 --2- 172.21.15.43:0/4096413478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f54bab4c90 0x55f54baf0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.522+0000 7fa8b6ffd640 0 client.20682 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.522+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/4096413478 >> 172.21.15.43:0/4096413478 conn(0x55f54baf9840 msgr2=0x55f54baf2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.522+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/4096413478 shutdown_connections 2024-04-21T08:49:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.522+0000 7fa8e0bc6740 1 -- 172.21.15.43:0/4096413478 wait complete. 2024-04-21T08:49:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.522+0000 7fa8e0bc6740 20 client.20682 trim_cache size 0 max 16384 2024-04-21T08:49:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:49:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:49:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:49:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:49:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.522+0000 7fa8ddfd5640 1 -- 172.21.15.43:0/1083276758 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa8c40a3e40 msgr2=0x7fa8c40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.522+0000 7fa8ddfd5640 1 --2- 172.21.15.43:0/1083276758 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa8c40a3e40 0x7fa8c40a6300 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0005fd0 tx=0x7fa8d0004a40 comp rx=0 tx=0).stop 2024-04-21T08:49:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.522+0000 7fa8ddfd5640 1 -- 172.21.15.43:0/1083276758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8d8103200 msgr2=0x7fa8d8198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.522+0000 7fa8ddfd5640 1 --2- 172.21.15.43:0/1083276758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8d8103200 0x7fa8d8198770 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0000c00 tx=0x7fa8c002c760 comp rx=0 tx=0).stop 2024-04-21T08:49:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.523+0000 7fa8ddfd5640 1 -- 172.21.15.43:0/1083276758 shutdown_connections 2024-04-21T08:49:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.523+0000 7fa8ddfd5640 1 --2- 172.21.15.43:0/1083276758 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa8c40a3e40 0x7fa8c40a6300 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.523+0000 7fa8ddfd5640 1 --2- 172.21.15.43:0/1083276758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8d810c540 0x7fa8d819cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.523+0000 7fa8ddfd5640 1 --2- 172.21.15.43:0/1083276758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8d8103b20 0x7fa8d8198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.523+0000 7fa8ddfd5640 1 --2- 172.21.15.43:0/1083276758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8d8103200 0x7fa8d8198770 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.523+0000 7fa8ddfd5640 1 -- 172.21.15.43:0/1083276758 >> 172.21.15.43:0/1083276758 conn(0x7fa8d8077fd0 msgr2=0x7fa8d80fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.523+0000 7fa8ddfd5640 1 -- 172.21.15.43:0/1083276758 shutdown_connections 2024-04-21T08:49:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:34.523+0000 7fa8ddfd5640 1 -- 172.21.15.43:0/1083276758 wait complete. 2024-04-21T08:49:34.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:34 smithi112 ceph-mon[31085]: pgmap v331: 129 pgs: 129 active+clean; 2.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:49:34.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.677+0000 7f97dbd2b640 1 -- 172.21.15.43:0/3899165181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97d41027a0 con 0x7f97d40690e0 2024-04-21T08:49:34.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.678+0000 7f97caffd640 1 -- 172.21.15.43:0/3899165181 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11935 (secure 0 0 0) 0x7f97d009c670 con 0x7f97d40690e0 2024-04-21T08:49:34.677 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:49:34.678 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:46:17.283806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27556":{"gid":27556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/229985237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":229985237},{"type":"v1","addr":"172.21.15.175:6837","nonce":229985237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24425,24412,24487,24527,24491]},"id":1}]} 2024-04-21T08:49:34.678 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-04-21T08:49:34.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.680+0000 7f97dbd2b640 1 -- 172.21.15.43:0/3899165181 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f97c00a3d20 msgr2=0x7f97c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.680+0000 7f97dbd2b640 1 --2- 172.21.15.43:0/3899165181 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f97c00a3d20 0x7f97c00a61e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f97d4102d00 tx=0x7f97c4005f90 comp rx=0 tx=0).stop 2024-04-21T08:49:34.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.680+0000 7f97dbd2b640 1 -- 172.21.15.43:0/3899165181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97d40690e0 msgr2=0x7f97d4101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.680+0000 7f97dbd2b640 1 --2- 172.21.15.43:0/3899165181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97d40690e0 0x7f97d4101270 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f97d000d770 tx=0x7f97d000dc30 comp rx=0 tx=0).stop 2024-04-21T08:49:34.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.680+0000 7f97dbd2b640 1 -- 172.21.15.43:0/3899165181 shutdown_connections 2024-04-21T08:49:34.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.680+0000 7f97dbd2b640 1 --2- 172.21.15.43:0/3899165181 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f97c00a3d20 0x7f97c00a61e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.680+0000 7f97dbd2b640 1 --2- 172.21.15.43:0/3899165181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97d4107fe0 0x7f97d4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.680+0000 7f97dbd2b640 1 --2- 172.21.15.43:0/3899165181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97d4069a80 0x7f97d41017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.680+0000 7f97dbd2b640 1 --2- 172.21.15.43:0/3899165181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97d40690e0 0x7f97d4101270 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.680+0000 7f97dbd2b640 1 -- 172.21.15.43:0/3899165181 >> 172.21.15.43:0/3899165181 conn(0x7f97d4077e30 msgr2=0x7f97d40ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.680+0000 7f97dbd2b640 1 -- 172.21.15.43:0/3899165181 shutdown_connections 2024-04-21T08:49:34.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.680+0000 7f97dbd2b640 1 -- 172.21.15.43:0/3899165181 wait complete. 2024-04-21T08:49:34.690 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:49:34.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.841+0000 7f22854a0640 1 -- 172.21.15.43:0/2555063878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2280107260 msgr2=0x7f228010ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.841+0000 7f22854a0640 1 --2- 172.21.15.43:0/2555063878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2280107260 0x7f228010ee30 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f2278009ca0 tx=0x7f227802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.841+0000 7f22854a0640 1 -- 172.21.15.43:0/2555063878 shutdown_connections 2024-04-21T08:49:34.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.841+0000 7f22854a0640 1 --2- 172.21.15.43:0/2555063878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2280107260 0x7f228010ee30 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.841+0000 7f22854a0640 1 --2- 172.21.15.43:0/2555063878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2280102b40 0x7f2280106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.841+0000 7f22854a0640 1 --2- 172.21.15.43:0/2555063878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2280102190 0x7f2280102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.841+0000 7f22854a0640 1 -- 172.21.15.43:0/2555063878 >> 172.21.15.43:0/2555063878 conn(0x7f22800fd990 msgr2=0x7f22800ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.841+0000 7f22854a0640 1 -- 172.21.15.43:0/2555063878 shutdown_connections 2024-04-21T08:49:34.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.841+0000 7f22854a0640 1 -- 172.21.15.43:0/2555063878 wait complete. 2024-04-21T08:49:34.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.842+0000 7f22854a0640 1 Processor -- start 2024-04-21T08:49:34.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.842+0000 7f22854a0640 1 -- start start 2024-04-21T08:49:34.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.842+0000 7f22854a0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2280102190 0x7f2280198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.842+0000 7f22854a0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2280107260 0x7f2280198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.842+0000 7f22854a0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f228019cf50 0x7f228019d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.842+0000 7f22854a0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f22801121b0 con 0x7f2280107260 2024-04-21T08:49:34.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.842+0000 7f22854a0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2280112030 con 0x7f228019cf50 2024-04-21T08:49:34.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.842+0000 7f22854a0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2280112330 con 0x7f2280102190 2024-04-21T08:49:34.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.842+0000 7f2277fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2280107260 0x7f2280198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.842+0000 7f2277fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2280107260 0x7f2280198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47958/0 (socket says 172.21.15.43:47958) 2024-04-21T08:49:34.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.842+0000 7f2277fff640 1 -- 172.21.15.43:0/400734548 learned_addr learned my addr 172.21.15.43:0/400734548 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:34.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.842+0000 7f2277fff640 1 -- 172.21.15.43:0/400734548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2280102190 msgr2=0x7f2280198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.842+0000 7f2277fff640 1 --2- 172.21.15.43:0/400734548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2280102190 0x7f2280198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.842+0000 7f2277fff640 1 -- 172.21.15.43:0/400734548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f228019cf50 msgr2=0x7f228019d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.842+0000 7f2277fff640 1 --2- 172.21.15.43:0/400734548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f228019cf50 0x7f228019d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.842+0000 7f2277fff640 1 -- 172.21.15.43:0/400734548 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228019da40 con 0x7f2280107260 2024-04-21T08:49:34.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.843+0000 7f2277fff640 1 --2- 172.21.15.43:0/400734548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2280107260 0x7f2280198bd0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f226800d770 tx=0x7f226800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.843+0000 7f227cff9640 1 -- 172.21.15.43:0/400734548 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268017070 con 0x7f2280107260 2024-04-21T08:49:34.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.843+0000 7f227cff9640 1 -- 172.21.15.43:0/400734548 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2268009c30 con 0x7f2280107260 2024-04-21T08:49:34.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.843+0000 7f22854a0640 1 -- 172.21.15.43:0/400734548 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f228019dd30 con 0x7f2280107260 2024-04-21T08:49:34.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.843+0000 7f227cff9640 1 -- 172.21.15.43:0/400734548 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226800f620 con 0x7f2280107260 2024-04-21T08:49:34.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.843+0000 7f22854a0640 1 -- 172.21.15.43:0/400734548 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22801a5520 con 0x7f2280107260 2024-04-21T08:49:34.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.845+0000 7f227cff9640 1 -- 172.21.15.43:0/400734548 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f226801a070 con 0x7f2280107260 2024-04-21T08:49:34.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.845+0000 7f2288168740 1 Processor -- start 2024-04-21T08:49:34.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.845+0000 7f2288168740 1 -- start start 2024-04-21T08:49:34.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.846+0000 7f2288168740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56037c26a4d0 0x56037c26a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.846+0000 7f2288168740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037c26ae80 0x56037c273430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.846+0000 7f2288168740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c273a00 0x56037c275e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.846+0000 7f2288168740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56037c1cf1f0 con 0x56037c273a00 2024-04-21T08:49:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.846+0000 7f2288168740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56037c16c4b0 con 0x56037c26ae80 2024-04-21T08:49:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.846+0000 7f2288168740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56037c1cb1b0 con 0x56037c26a4d0 2024-04-21T08:49:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.846+0000 7f2277fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c273a00 0x56037c275e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.846+0000 7f227cff9640 1 --2- 172.21.15.43:0/400734548 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f226c0a3d20 0x7f226c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.846+0000 7f2277fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c273a00 0x56037c275e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47962/0 (socket says 172.21.15.43:47962) 2024-04-21T08:49:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.846+0000 7f2277fff640 1 -- 172.21.15.43:0/345316866 learned_addr learned my addr 172.21.15.43:0/345316866 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.846+0000 7f227effd640 1 --2- 172.21.15.43:0/345316866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037c26ae80 0x56037c273430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.846+0000 7f227f7fe640 1 --2- 172.21.15.43:0/400734548 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f226c0a3d20 0x7f226c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.846+0000 7f227f7fe640 1 --2- 172.21.15.43:0/345316866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56037c26a4d0 0x56037c26a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.846+0000 7f227cff9640 1 -- 172.21.15.43:0/400734548 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f22680d07b0 con 0x7f2280107260 2024-04-21T08:49:34.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.846+0000 7f2277fff640 1 -- 172.21.15.43:0/345316866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56037c26a4d0 msgr2=0x56037c26a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.846+0000 7f2277fff640 1 --2- 172.21.15.43:0/345316866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56037c26a4d0 0x56037c26a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.846+0000 7f2277fff640 1 -- 172.21.15.43:0/345316866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037c26ae80 msgr2=0x56037c273430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.846+0000 7f2277fff640 1 --2- 172.21.15.43:0/345316866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037c26ae80 0x56037c273430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.846+0000 7f2277fff640 1 -- 172.21.15.43:0/345316866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56037c15e1b0 con 0x56037c273a00 2024-04-21T08:49:34.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.846+0000 7f227effd640 1 --2- 172.21.15.43:0/345316866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037c26ae80 0x56037c273430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:34.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.847+0000 7f2277fff640 1 --2- 172.21.15.43:0/345316866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c273a00 0x56037c275e50 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f2268098010 tx=0x7f226800df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abc00322e72a5a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.847+0000 7f227f7fe640 1 --2- 172.21.15.43:0/400734548 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f226c0a3d20 0x7f226c0a61e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2278009a60 tx=0x7f2278005df0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.847+0000 7f22757fa640 1 -- 172.21.15.43:0/345316866 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268021070 con 0x56037c273a00 2024-04-21T08:49:34.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.847+0000 7f22757fa640 1 -- 172.21.15.43:0/345316866 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2268009de0 con 0x56037c273a00 2024-04-21T08:49:34.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.847+0000 7f22757fa640 1 -- 172.21.15.43:0/345316866 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268098080 con 0x56037c273a00 2024-04-21T08:49:34.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.847+0000 7f2288168740 1 -- 172.21.15.43:0/345316866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c273a00 msgr2=0x56037c275e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.847+0000 7f2288168740 1 --2- 172.21.15.43:0/345316866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c273a00 0x56037c275e50 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f2268098010 tx=0x7f226800df10 comp rx=0 tx=0).stop 2024-04-21T08:49:34.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.848+0000 7f2288168740 1 -- 172.21.15.43:0/345316866 shutdown_connections 2024-04-21T08:49:34.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.848+0000 7f2288168740 1 --2- 172.21.15.43:0/345316866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c273a00 0x56037c275e50 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.848+0000 7f2288168740 1 --2- 172.21.15.43:0/345316866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037c26ae80 0x56037c273430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.848+0000 7f2288168740 1 --2- 172.21.15.43:0/345316866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56037c26a4d0 0x56037c26a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.848+0000 7f2288168740 1 -- 172.21.15.43:0/345316866 >> 172.21.15.43:0/345316866 conn(0x56037c264ba0 msgr2=0x56037c264f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.848+0000 7f2288168740 1 -- 172.21.15.43:0/345316866 shutdown_connections 2024-04-21T08:49:34.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.848+0000 7f2288168740 1 -- 172.21.15.43:0/345316866 wait complete. 2024-04-21T08:49:34.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.848+0000 7f2288168740 1 -- start start 2024-04-21T08:49:34.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.848+0000 7f2288168740 1 Processor -- start 2024-04-21T08:49:34.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.848+0000 7f2288168740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c26a4d0 0x56037c2733a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.848+0000 7f2288168740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56037c26ae80 0x56037c270730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.848+0000 7f2288168740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037c273a00 0x56037c270e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.848+0000 7f2288168740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56037c2768b0 con 0x56037c26a4d0 2024-04-21T08:49:34.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.848+0000 7f2288168740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56037c188500 con 0x56037c273a00 2024-04-21T08:49:34.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.848+0000 7f2288168740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56037c276a30 con 0x56037c26ae80 2024-04-21T08:49:34.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.848+0000 7f2288168740 10 client.0 _ll_register_callbacks cb 0x56037c156b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:34.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.848+0000 7f227effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c26a4d0 0x56037c2733a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.848+0000 7f227effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c26a4d0 0x56037c2733a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47974/0 (socket says 172.21.15.43:47974) 2024-04-21T08:49:34.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.848+0000 7f227effd640 1 -- 172.21.15.43:0/32818941 learned_addr learned my addr 172.21.15.43:0/32818941 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:34.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.849+0000 7f2277fff640 1 --2- 172.21.15.43:0/32818941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56037c26ae80 0x56037c270730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.849+0000 7f227effd640 1 -- 172.21.15.43:0/32818941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56037c26ae80 msgr2=0x56037c270730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.849+0000 7f227effd640 1 --2- 172.21.15.43:0/32818941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56037c26ae80 0x56037c270730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.849+0000 7f227effd640 1 -- 172.21.15.43:0/32818941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037c273a00 msgr2=0x56037c270e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.849+0000 7f227f7fe640 1 --2- 172.21.15.43:0/32818941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037c273a00 0x56037c270e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.849+0000 7f227effd640 1 --2- 172.21.15.43:0/32818941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037c273a00 0x56037c270e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.849+0000 7f227effd640 1 -- 172.21.15.43:0/32818941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56037bee13f0 con 0x56037c26a4d0 2024-04-21T08:49:34.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.849+0000 7f227effd640 1 --2- 172.21.15.43:0/32818941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c26a4d0 0x56037c2733a0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f227000f640 tx=0x7f227000fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7cd6499bbdcac3e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.849+0000 7f22667fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:34.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.849+0000 7f22667fc640 1 -- 172.21.15.43:0/32818941 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227001d060 con 0x56037c26a4d0 2024-04-21T08:49:34.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.849+0000 7f22667fc640 1 -- 172.21.15.43:0/32818941 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f227000d9d0 con 0x56037c26a4d0 2024-04-21T08:49:34.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.849+0000 7f2288168740 1 -- 172.21.15.43:0/32818941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2268009de0 con 0x56037c26a4d0 2024-04-21T08:49:34.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.849+0000 7f22667fc640 1 -- 172.21.15.43:0/32818941 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2270024050 con 0x56037c26a4d0 2024-04-21T08:49:34.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.849+0000 7f2288168740 10 client.20694 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:34.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.849+0000 7f2288168740 1 -- 172.21.15.43:0/32818941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56037bfab7d0 con 0x56037c26a4d0 2024-04-21T08:49:34.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.849+0000 7f22667fc640 1 -- 172.21.15.43:0/32818941 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2268009de0 con 0x56037c26a4d0 2024-04-21T08:49:34.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.850+0000 7f22667fc640 1 -- 172.21.15.43:0/32818941 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2270016340 con 0x56037c26a4d0 2024-04-21T08:49:34.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.850+0000 7f2288168740 10 client.20694 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:34.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.850+0000 7f2288168740 10 client.20694 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:49:34.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.850+0000 7f2288168740 20 client.20694 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:34.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.850+0000 7f2288168740 1 --2- 172.21.15.43:0/32818941 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x56037c283c70 0x56037c2860e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.850+0000 7f2288168740 4 client.20694 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:34.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.850+0000 7f2288168740 1 -- 172.21.15.43:0/32818941 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56037c16e930 con 0x56037c283c70 2024-04-21T08:49:34.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.850+0000 7f2277fff640 1 --2- 172.21.15.43:0/32818941 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x56037c283c70 0x56037c2860e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.850+0000 7f2277fff640 1 --2- 172.21.15.43:0/32818941 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x56037c283c70 0x56037c2860e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7ab08de34fa6345b server_cookie=6150b799ef4a4aef in_seq=0 out_seq=0 2024-04-21T08:49:34.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.850+0000 7f22667fc640 10 client.20694 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:49:34.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.851+0000 7f22667fc640 1 -- 172.21.15.43:0/32818941 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56037c16e930 con 0x56037c283c70 2024-04-21T08:49:34.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.851+0000 7f22667fc640 10 client.20694 handle_command_reply: tid=0 2024-04-21T08:49:34.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.851+0000 7f2288168740 1 client.20694 shutdown 2024-04-21T08:49:34.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.851+0000 7f2288168740 1 -- 172.21.15.43:0/32818941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c26a4d0 msgr2=0x56037c2733a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.851+0000 7f2288168740 1 --2- 172.21.15.43:0/32818941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c26a4d0 0x56037c2733a0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f227000f640 tx=0x7f227000fb00 comp rx=0 tx=0).stop 2024-04-21T08:49:34.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.851+0000 7f2288168740 1 -- 172.21.15.43:0/32818941 shutdown_connections 2024-04-21T08:49:34.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.851+0000 7f2288168740 1 --2- 172.21.15.43:0/32818941 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x56037c283c70 0x56037c2860e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.851+0000 7f2288168740 1 --2- 172.21.15.43:0/32818941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037c273a00 0x56037c270e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.851+0000 7f2288168740 1 --2- 172.21.15.43:0/32818941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56037c26ae80 0x56037c270730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.851+0000 7f22667fc640 0 client.20694 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:49:34.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.851+0000 7f2288168740 1 --2- 172.21.15.43:0/32818941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c26a4d0 0x56037c2733a0 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.851+0000 7f2288168740 1 -- 172.21.15.43:0/32818941 >> 172.21.15.43:0/32818941 conn(0x56037c264ba0 msgr2=0x56037c2668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.851+0000 7f2288168740 1 -- 172.21.15.43:0/32818941 shutdown_connections 2024-04-21T08:49:34.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.851+0000 7f2288168740 1 -- 172.21.15.43:0/32818941 wait complete. 2024-04-21T08:49:34.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.851+0000 7f2288168740 20 client.20694 trim_cache size 0 max 16384 2024-04-21T08:49:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.864+0000 7f2288168740 1 Processor -- start 2024-04-21T08:49:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.865+0000 7f2288168740 1 -- start start 2024-04-21T08:49:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.865+0000 7f2288168740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56037c26a4d0 0x56037c27f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.865+0000 7f2288168740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037c26ae80 0x56037c26f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.865+0000 7f2288168740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c273a00 0x56037c263a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.865+0000 7f2288168740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f22801af550 con 0x56037c273a00 2024-04-21T08:49:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.865+0000 7f2288168740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f22801b1090 con 0x56037c26ae80 2024-04-21T08:49:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.865+0000 7f2288168740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22801afc20 con 0x56037c26a4d0 2024-04-21T08:49:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.865+0000 7f227f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c273a00 0x56037c263a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.865+0000 7f227f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c273a00 0x56037c263a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47986/0 (socket says 172.21.15.43:47986) 2024-04-21T08:49:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.865+0000 7f227f7fe640 1 -- 172.21.15.43:0/155953808 learned_addr learned my addr 172.21.15.43:0/155953808 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.865+0000 7f227effd640 1 --2- 172.21.15.43:0/155953808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56037c26a4d0 0x56037c27f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.865+0000 7f2277fff640 1 --2- 172.21.15.43:0/155953808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037c26ae80 0x56037c26f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.865+0000 7f227f7fe640 1 -- 172.21.15.43:0/155953808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56037c26a4d0 msgr2=0x56037c27f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.865+0000 7f227f7fe640 1 --2- 172.21.15.43:0/155953808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56037c26a4d0 0x56037c27f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.865+0000 7f227f7fe640 1 -- 172.21.15.43:0/155953808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037c26ae80 msgr2=0x56037c26f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.865+0000 7f227f7fe640 1 --2- 172.21.15.43:0/155953808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037c26ae80 0x56037c26f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.865+0000 7f227f7fe640 1 -- 172.21.15.43:0/155953808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56037c2711e0 con 0x56037c273a00 2024-04-21T08:49:34.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.865+0000 7f2277fff640 1 --2- 172.21.15.43:0/155953808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037c26ae80 0x56037c26f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:34.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.865+0000 7f227effd640 1 --2- 172.21.15.43:0/155953808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56037c26a4d0 0x56037c27f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:34.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.865+0000 7f227f7fe640 1 --2- 172.21.15.43:0/155953808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c273a00 0x56037c263a80 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f2278005c80 tx=0x7f227803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa00a403f82030ba server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.866+0000 7f2274ff9640 1 -- 172.21.15.43:0/155953808 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278039040 con 0x56037c273a00 2024-04-21T08:49:34.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.866+0000 7f2274ff9640 1 -- 172.21.15.43:0/155953808 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2278052070 con 0x56037c273a00 2024-04-21T08:49:34.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.866+0000 7f2274ff9640 1 -- 172.21.15.43:0/155953808 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22780346a0 con 0x56037c273a00 2024-04-21T08:49:34.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.866+0000 7f2288168740 1 -- 172.21.15.43:0/155953808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c273a00 msgr2=0x56037c263a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.866+0000 7f2288168740 1 --2- 172.21.15.43:0/155953808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c273a00 0x56037c263a80 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f2278005c80 tx=0x7f227803c000 comp rx=0 tx=0).stop 2024-04-21T08:49:34.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.866+0000 7f2288168740 1 -- 172.21.15.43:0/155953808 shutdown_connections 2024-04-21T08:49:34.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.866+0000 7f2288168740 1 --2- 172.21.15.43:0/155953808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c273a00 0x56037c263a80 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.866+0000 7f2288168740 1 --2- 172.21.15.43:0/155953808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037c26ae80 0x56037c26f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.866+0000 7f2288168740 1 --2- 172.21.15.43:0/155953808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56037c26a4d0 0x56037c27f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.866+0000 7f2288168740 1 -- 172.21.15.43:0/155953808 >> 172.21.15.43:0/155953808 conn(0x56037c264ba0 msgr2=0x56037c263370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.866+0000 7f2288168740 1 -- 172.21.15.43:0/155953808 shutdown_connections 2024-04-21T08:49:34.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.866+0000 7f2288168740 1 -- 172.21.15.43:0/155953808 wait complete. 2024-04-21T08:49:34.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.867+0000 7f2288168740 1 -- start start 2024-04-21T08:49:34.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.867+0000 7f2288168740 1 Processor -- start 2024-04-21T08:49:34.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.867+0000 7f2288168740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037c26a4d0 0x56037c27c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.867+0000 7f2288168740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56037c26ae80 0x56037c2773d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.867+0000 7f2288168740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c273a00 0x56037c277910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.867+0000 7f2288168740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f22801b09c0 con 0x56037c273a00 2024-04-21T08:49:34.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.867+0000 7f2288168740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f22801b02f0 con 0x56037c26a4d0 2024-04-21T08:49:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.867+0000 7f2288168740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22801ad550 con 0x56037c26ae80 2024-04-21T08:49:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.867+0000 7f2288168740 10 client.0 _ll_register_callbacks cb 0x7f227000ae10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.867+0000 7f227f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c273a00 0x56037c277910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.867+0000 7f227f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c273a00 0x56037c277910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47994/0 (socket says 172.21.15.43:47994) 2024-04-21T08:49:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.867+0000 7f227f7fe640 1 -- 172.21.15.43:0/3405430248 learned_addr learned my addr 172.21.15.43:0/3405430248 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.867+0000 7f227effd640 1 --2- 172.21.15.43:0/3405430248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037c26a4d0 0x56037c27c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.867+0000 7f2277fff640 1 --2- 172.21.15.43:0/3405430248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56037c26ae80 0x56037c2773d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.867+0000 7f227f7fe640 1 -- 172.21.15.43:0/3405430248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56037c26ae80 msgr2=0x56037c2773d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.867+0000 7f227f7fe640 1 --2- 172.21.15.43:0/3405430248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56037c26ae80 0x56037c2773d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.867+0000 7f227f7fe640 1 -- 172.21.15.43:0/3405430248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037c26a4d0 msgr2=0x56037c27c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.867+0000 7f227f7fe640 1 --2- 172.21.15.43:0/3405430248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037c26a4d0 0x56037c27c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.867+0000 7f227f7fe640 1 -- 172.21.15.43:0/3405430248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56037c278110 con 0x56037c273a00 2024-04-21T08:49:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.868+0000 7f227f7fe640 1 --2- 172.21.15.43:0/3405430248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c273a00 0x56037c277910 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f2278033000 tx=0x7f2278032870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99834b508fbad7a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.868+0000 7f22667fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.868+0000 7f22667fc640 1 -- 172.21.15.43:0/3405430248 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278039040 con 0x56037c273a00 2024-04-21T08:49:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.868+0000 7f22667fc640 1 -- 172.21.15.43:0/3405430248 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f2278042680 con 0x56037c273a00 2024-04-21T08:49:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.868+0000 7f2288168740 1 -- 172.21.15.43:0/3405430248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2278052070 con 0x56037c273a00 2024-04-21T08:49:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.868+0000 7f22667fc640 1 -- 172.21.15.43:0/3405430248 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2278064050 con 0x56037c273a00 2024-04-21T08:49:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.868+0000 7f2288168740 10 client.20700 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.868+0000 7f2288168740 1 -- 172.21.15.43:0/3405430248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56037c2ee5d0 con 0x56037c273a00 2024-04-21T08:49:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.868+0000 7f22667fc640 1 -- 172.21.15.43:0/3405430248 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2278052070 con 0x56037c273a00 2024-04-21T08:49:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.868+0000 7f22667fc640 1 -- 172.21.15.43:0/3405430248 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f227804c070 con 0x56037c273a00 2024-04-21T08:49:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.868+0000 7f2288168740 10 client.20700 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.868+0000 7f2288168740 10 client.20700 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:49:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.868+0000 7f2288168740 20 client.20700 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.868+0000 7f2288168740 1 --2- 172.21.15.43:0/3405430248 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x56037c2eeda0 0x56037c2f11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.868+0000 7f2288168740 4 client.20700 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.868+0000 7f2288168740 1 -- 172.21.15.43:0/3405430248 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56037c1dcc10 con 0x56037c2eeda0 2024-04-21T08:49:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.869+0000 7f227effd640 1 --2- 172.21.15.43:0/3405430248 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x56037c2eeda0 0x56037c2f11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.869+0000 7f227effd640 1 --2- 172.21.15.43:0/3405430248 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x56037c2eeda0 0x56037c2f11c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c17ed3c3d509566 server_cookie=c2d4a1bb203cc964 in_seq=0 out_seq=0 2024-04-21T08:49:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.869+0000 7f22667fc640 10 client.20700 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:49:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.869+0000 7f22667fc640 1 -- 172.21.15.43:0/3405430248 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56037c1dcc10 con 0x56037c2eeda0 2024-04-21T08:49:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.869+0000 7f22667fc640 10 client.20700 handle_command_reply: tid=0 2024-04-21T08:49:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.869+0000 7f2288168740 1 client.20700 shutdown 2024-04-21T08:49:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.869+0000 7f2288168740 1 -- 172.21.15.43:0/3405430248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c273a00 msgr2=0x56037c277910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.869+0000 7f2288168740 1 --2- 172.21.15.43:0/3405430248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c273a00 0x56037c277910 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f2278033000 tx=0x7f2278032870 comp rx=0 tx=0).stop 2024-04-21T08:49:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.870+0000 7f2288168740 1 -- 172.21.15.43:0/3405430248 shutdown_connections 2024-04-21T08:49:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.870+0000 7f2288168740 1 --2- 172.21.15.43:0/3405430248 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x56037c2eeda0 0x56037c2f11c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.870+0000 7f2288168740 1 --2- 172.21.15.43:0/3405430248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56037c273a00 0x56037c277910 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.870+0000 7f2288168740 1 --2- 172.21.15.43:0/3405430248 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56037c26ae80 0x56037c2773d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.870+0000 7f22667fc640 0 client.20700 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:49:34.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.870+0000 7f2288168740 1 --2- 172.21.15.43:0/3405430248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037c26a4d0 0x56037c27c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.870+0000 7f2288168740 1 -- 172.21.15.43:0/3405430248 >> 172.21.15.43:0/3405430248 conn(0x56037c264ba0 msgr2=0x56037c26f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.870+0000 7f2288168740 1 -- 172.21.15.43:0/3405430248 shutdown_connections 2024-04-21T08:49:34.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.870+0000 7f2288168740 1 -- 172.21.15.43:0/3405430248 wait complete. 2024-04-21T08:49:34.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.870+0000 7f2288168740 20 client.20700 trim_cache size 0 max 16384 2024-04-21T08:49:34.876 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:49:34.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.870+0000 7f22854a0640 1 -- 172.21.15.43:0/400734548 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f226c0a3d20 msgr2=0x7f226c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.870+0000 7f22854a0640 1 --2- 172.21.15.43:0/400734548 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f226c0a3d20 0x7f226c0a61e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2278009a60 tx=0x7f2278005df0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.870+0000 7f22854a0640 1 -- 172.21.15.43:0/400734548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2280107260 msgr2=0x7f2280198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.870+0000 7f22854a0640 1 --2- 172.21.15.43:0/400734548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2280107260 0x7f2280198bd0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f226800d770 tx=0x7f226800dc30 comp rx=0 tx=0).stop 2024-04-21T08:49:34.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.870+0000 7f22854a0640 1 -- 172.21.15.43:0/400734548 shutdown_connections 2024-04-21T08:49:34.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.870+0000 7f22854a0640 1 --2- 172.21.15.43:0/400734548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f228019cf50 0x7f228019d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.870+0000 7f22854a0640 1 --2- 172.21.15.43:0/400734548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2280107260 0x7f2280198bd0 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.870+0000 7f22854a0640 1 --2- 172.21.15.43:0/400734548 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f226c0a3d20 0x7f226c0a61e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.870+0000 7f22854a0640 1 --2- 172.21.15.43:0/400734548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2280102190 0x7f2280198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.870+0000 7f22854a0640 1 -- 172.21.15.43:0/400734548 >> 172.21.15.43:0/400734548 conn(0x7f22800fd990 msgr2=0x7f22800ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.870+0000 7f22854a0640 1 -- 172.21.15.43:0/400734548 shutdown_connections 2024-04-21T08:49:34.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.870+0000 7f22854a0640 1 -- 172.21.15.43:0/400734548 wait complete. 2024-04-21T08:49:34.881 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:49:34.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.984+0000 7f325777e640 1 -- 172.21.15.43:0/2311911902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3250102b40 msgr2=0x7f3250106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.984+0000 7f325777e640 1 --2- 172.21.15.43:0/2311911902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3250102b40 0x7f3250106b80 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f3240009900 tx=0x7f324002bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:34.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.985+0000 7f325777e640 1 -- 172.21.15.43:0/2311911902 shutdown_connections 2024-04-21T08:49:34.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.985+0000 7f325777e640 1 --2- 172.21.15.43:0/2311911902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3250107260 0x7f325010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.985+0000 7f325777e640 1 --2- 172.21.15.43:0/2311911902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3250102b40 0x7f3250106b80 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.985+0000 7f325777e640 1 --2- 172.21.15.43:0/2311911902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3250102190 0x7f3250102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.985+0000 7f325777e640 1 -- 172.21.15.43:0/2311911902 >> 172.21.15.43:0/2311911902 conn(0x7f32500fd990 msgr2=0x7f32500ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.985+0000 7f325777e640 1 -- 172.21.15.43:0/2311911902 shutdown_connections 2024-04-21T08:49:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.985+0000 7f325777e640 1 -- 172.21.15.43:0/2311911902 wait complete. 2024-04-21T08:49:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.985+0000 7f325777e640 1 Processor -- start 2024-04-21T08:49:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.985+0000 7f325777e640 1 -- start start 2024-04-21T08:49:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.985+0000 7f325777e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3250102190 0x7f3250198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.985+0000 7f325777e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3250107260 0x7f3250198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.985+0000 7f325777e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f325019d020 0x7f325019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.985+0000 7f325777e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3250112180 con 0x7f3250102190 2024-04-21T08:49:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.985+0000 7f325777e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3250112000 con 0x7f3250107260 2024-04-21T08:49:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.985+0000 7f325777e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3250112300 con 0x7f325019d020 2024-04-21T08:49:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.986+0000 7f325551a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3250102190 0x7f3250198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.986+0000 7f325551a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3250102190 0x7f3250198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48014/0 (socket says 172.21.15.43:48014) 2024-04-21T08:49:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.986+0000 7f325551a640 1 -- 172.21.15.43:0/2808918980 learned_addr learned my addr 172.21.15.43:0/2808918980 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.986+0000 7f3254d19640 1 --2- 172.21.15.43:0/2808918980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3250107260 0x7f3250198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.986+0000 7f3255d1b640 1 --2- 172.21.15.43:0/2808918980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f325019d020 0x7f325019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.986+0000 7f325551a640 1 -- 172.21.15.43:0/2808918980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f325019d020 msgr2=0x7f325019d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.986+0000 7f325551a640 1 --2- 172.21.15.43:0/2808918980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f325019d020 0x7f325019d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.986+0000 7f325551a640 1 -- 172.21.15.43:0/2808918980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3250107260 msgr2=0x7f3250198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.986+0000 7f325551a640 1 --2- 172.21.15.43:0/2808918980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3250107260 0x7f3250198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.986+0000 7f325551a640 1 -- 172.21.15.43:0/2808918980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f325019db10 con 0x7f3250102190 2024-04-21T08:49:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.986+0000 7f325551a640 1 --2- 172.21.15.43:0/2808918980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3250102190 0x7f3250198760 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f324c00c890 tx=0x7f324c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.986+0000 7f32467fc640 1 -- 172.21.15.43:0/2808918980 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c004220 con 0x7f3250102190 2024-04-21T08:49:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.986+0000 7f32467fc640 1 -- 172.21.15.43:0/2808918980 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f324c0043c0 con 0x7f3250102190 2024-04-21T08:49:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.986+0000 7f325777e640 1 -- 172.21.15.43:0/2808918980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f325019de00 con 0x7f3250102190 2024-04-21T08:49:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.986+0000 7f32467fc640 1 -- 172.21.15.43:0/2808918980 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c004a00 con 0x7f3250102190 2024-04-21T08:49:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.986+0000 7f325777e640 1 -- 172.21.15.43:0/2808918980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32501a55c0 con 0x7f3250102190 2024-04-21T08:49:34.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.989+0000 7f32467fc640 1 -- 172.21.15.43:0/2808918980 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f324c0079b0 con 0x7f3250102190 2024-04-21T08:49:34.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.989+0000 7f325a36d740 1 Processor -- start 2024-04-21T08:49:34.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.989+0000 7f325a36d740 1 -- start start 2024-04-21T08:49:34.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.989+0000 7f325a36d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30e9250 0x55eec30e9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.989+0000 7f325a36d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eec30e9c00 0x55eec30f6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.989+0000 7f325a36d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eec30f6e10 0x55eec30f9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.989+0000 7f325a36d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55eec304e1f0 con 0x55eec30e9250 2024-04-21T08:49:34.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.989+0000 7f325a36d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55eec2feb4b0 con 0x55eec30f6e10 2024-04-21T08:49:34.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.989+0000 7f325a36d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55eec304a1b0 con 0x55eec30e9c00 2024-04-21T08:49:34.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.989+0000 7f32467fc640 1 --2- 172.21.15.43:0/2808918980 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f323c0a3d20 0x7f323c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.989+0000 7f3255d1b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30e9250 0x55eec30e9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.989+0000 7f3255d1b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30e9250 0x55eec30e9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48018/0 (socket says 172.21.15.43:48018) 2024-04-21T08:49:34.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.989+0000 7f3255d1b640 1 -- 172.21.15.43:0/751985812 learned_addr learned my addr 172.21.15.43:0/751985812 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:34.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.990+0000 7f3254d19640 1 --2- 172.21.15.43:0/751985812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eec30f6e10 0x55eec30f9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.990+0000 7f3255d1b640 1 --2- 172.21.15.43:0/2808918980 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f323c0a3d20 0x7f323c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.990+0000 7f32467fc640 1 -- 172.21.15.43:0/2808918980 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f324c0d0a20 con 0x7f3250102190 2024-04-21T08:49:34.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.990+0000 7f325551a640 1 --2- 172.21.15.43:0/751985812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eec30e9c00 0x55eec30f6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:34.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.990+0000 7f3255d1b640 1 -- 172.21.15.43:0/751985812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eec30e9c00 msgr2=0x55eec30f6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.990+0000 7f3255d1b640 1 --2- 172.21.15.43:0/751985812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eec30e9c00 0x55eec30f6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.990+0000 7f3255d1b640 1 -- 172.21.15.43:0/751985812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eec30f6e10 msgr2=0x55eec30f9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:34.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.990+0000 7f3255d1b640 1 --2- 172.21.15.43:0/751985812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eec30f6e10 0x55eec30f9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.990+0000 7f3255d1b640 1 -- 172.21.15.43:0/751985812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eec2fdd1b0 con 0x55eec30e9250 2024-04-21T08:49:34.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.990+0000 7f3255d1b640 1 --2- 172.21.15.43:0/751985812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30e9250 0x55eec30e9630 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f3248014940 tx=0x7f3248014e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728221d9b0f1d312 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.990+0000 7f323b7fe640 1 -- 172.21.15.43:0/751985812 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3248014050 con 0x55eec30e9250 2024-04-21T08:49:34.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.990+0000 7f323b7fe640 1 -- 172.21.15.43:0/751985812 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32480141f0 con 0x55eec30e9250 2024-04-21T08:49:34.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.990+0000 7f323b7fe640 1 -- 172.21.15.43:0/751985812 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3248013640 con 0x55eec30e9250 2024-04-21T08:49:34.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.991+0000 7f325a36d740 1 -- 172.21.15.43:0/751985812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30e9250 msgr2=0x55eec30e9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:34.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.991+0000 7f325a36d740 1 --2- 172.21.15.43:0/751985812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30e9250 0x55eec30e9630 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f3248014940 tx=0x7f3248014e00 comp rx=0 tx=0).stop 2024-04-21T08:49:34.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.991+0000 7f3255d1b640 1 --2- 172.21.15.43:0/2808918980 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f323c0a3d20 0x7f323c0a61e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3248021000 tx=0x7f3248020000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:34.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.991+0000 7f325a36d740 1 -- 172.21.15.43:0/751985812 shutdown_connections 2024-04-21T08:49:34.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.991+0000 7f325a36d740 1 --2- 172.21.15.43:0/751985812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eec30f6e10 0x55eec30f9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.991+0000 7f325a36d740 1 --2- 172.21.15.43:0/751985812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eec30e9c00 0x55eec30f6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.991+0000 7f325a36d740 1 --2- 172.21.15.43:0/751985812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30e9250 0x55eec30e9630 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:34.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.991+0000 7f325a36d740 1 -- 172.21.15.43:0/751985812 >> 172.21.15.43:0/751985812 conn(0x55eec30e3ba0 msgr2=0x55eec30e3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:34.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.991+0000 7f325a36d740 1 -- 172.21.15.43:0/751985812 shutdown_connections 2024-04-21T08:49:34.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.991+0000 7f325a36d740 1 -- 172.21.15.43:0/751985812 wait complete. 2024-04-21T08:49:34.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.991+0000 7f325a36d740 1 -- start start 2024-04-21T08:49:34.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.991+0000 7f325a36d740 1 Processor -- start 2024-04-21T08:49:34.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.992+0000 7f325a36d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eec30e9250 0x55eec30f6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:34.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.992+0000 7f325a36d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30e9c00 0x55eec30f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.992+0000 7f325a36d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eec30f6e10 0x55eec30f4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.992+0000 7f325a36d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55eec30f9d40 con 0x55eec30e9c00 2024-04-21T08:49:35.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.992+0000 7f325a36d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55eec3007500 con 0x55eec30f6e10 2024-04-21T08:49:35.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.992+0000 7f325a36d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55eec30f9ec0 con 0x55eec30e9250 2024-04-21T08:49:35.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.992+0000 7f325a36d740 10 client.0 _ll_register_callbacks cb 0x55eec2fd5b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:35.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.992+0000 7f3254d19640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30e9c00 0x55eec30f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.992+0000 7f3254d19640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30e9c00 0x55eec30f3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48028/0 (socket says 172.21.15.43:48028) 2024-04-21T08:49:35.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.992+0000 7f3254d19640 1 -- 172.21.15.43:0/1657744072 learned_addr learned my addr 172.21.15.43:0/1657744072 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:35.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.992+0000 7f325551a640 1 --2- 172.21.15.43:0/1657744072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eec30e9250 0x55eec30f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.992+0000 7f3255d1b640 1 --2- 172.21.15.43:0/1657744072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eec30f6e10 0x55eec30f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.992+0000 7f3254d19640 1 -- 172.21.15.43:0/1657744072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eec30e9250 msgr2=0x55eec30f6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.992+0000 7f3254d19640 1 --2- 172.21.15.43:0/1657744072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eec30e9250 0x55eec30f6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.992+0000 7f3254d19640 1 -- 172.21.15.43:0/1657744072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eec30f6e10 msgr2=0x55eec30f4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.992+0000 7f3254d19640 1 --2- 172.21.15.43:0/1657744072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eec30f6e10 0x55eec30f4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.992+0000 7f3254d19640 1 -- 172.21.15.43:0/1657744072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eec2d603f0 con 0x55eec30e9c00 2024-04-21T08:49:35.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.992+0000 7f3254d19640 1 --2- 172.21.15.43:0/1657744072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30e9c00 0x55eec30f3ac0 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f3240005ab0 tx=0x7f324002c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544ff9bccbeb44b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:35.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.992+0000 7f3238ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:35.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.993+0000 7f3238ff9640 1 -- 172.21.15.43:0/1657744072 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3240005230 con 0x55eec30e9c00 2024-04-21T08:49:35.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.993+0000 7f3238ff9640 1 -- 172.21.15.43:0/1657744072 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f32400053d0 con 0x55eec30e9c00 2024-04-21T08:49:35.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.993+0000 7f325a36d740 1 -- 172.21.15.43:0/1657744072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32480141f0 con 0x55eec30e9c00 2024-04-21T08:49:35.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.993+0000 7f3238ff9640 1 -- 172.21.15.43:0/1657744072 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f32400055d0 con 0x55eec30e9c00 2024-04-21T08:49:35.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.993+0000 7f325a36d740 10 client.20712 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:35.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.993+0000 7f325a36d740 1 -- 172.21.15.43:0/1657744072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eec2e2a7d0 con 0x55eec30e9c00 2024-04-21T08:49:35.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.993+0000 7f3238ff9640 1 -- 172.21.15.43:0/1657744072 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32480141f0 con 0x55eec30e9c00 2024-04-21T08:49:35.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.993+0000 7f3238ff9640 1 -- 172.21.15.43:0/1657744072 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3240003870 con 0x55eec30e9c00 2024-04-21T08:49:35.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.993+0000 7f325a36d740 10 client.20712 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:35.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.993+0000 7f325a36d740 10 client.20712 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:49:35.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.993+0000 7f325a36d740 20 client.20712 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:35.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.993+0000 7f325a36d740 1 --2- 172.21.15.43:0/1657744072 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55eec3106fe0 0x55eec3109430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.993+0000 7f325a36d740 4 client.20712 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:35.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.993+0000 7f325a36d740 1 -- 172.21.15.43:0/1657744072 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eec2fed930 con 0x55eec3106fe0 2024-04-21T08:49:35.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.994+0000 7f325551a640 1 --2- 172.21.15.43:0/1657744072 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55eec3106fe0 0x55eec3109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.994+0000 7f325551a640 1 --2- 172.21.15.43:0/1657744072 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55eec3106fe0 0x55eec3109430 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1ce1cec3e6146fef server_cookie=bea8c86afb5b80b0 in_seq=0 out_seq=0 2024-04-21T08:49:35.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.994+0000 7f3238ff9640 10 client.20712 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:49:35.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.995+0000 7f3238ff9640 1 -- 172.21.15.43:0/1657744072 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eec2fed930 con 0x55eec3106fe0 2024-04-21T08:49:35.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.995+0000 7f3238ff9640 10 client.20712 handle_command_reply: tid=0 2024-04-21T08:49:35.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.995+0000 7f325a36d740 1 client.20712 shutdown 2024-04-21T08:49:35.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.995+0000 7f325a36d740 1 -- 172.21.15.43:0/1657744072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30e9c00 msgr2=0x55eec30f3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.995+0000 7f325a36d740 1 --2- 172.21.15.43:0/1657744072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30e9c00 0x55eec30f3ac0 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f3240005ab0 tx=0x7f324002c9f0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.995+0000 7f325a36d740 1 -- 172.21.15.43:0/1657744072 shutdown_connections 2024-04-21T08:49:35.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.995+0000 7f325a36d740 1 --2- 172.21.15.43:0/1657744072 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55eec3106fe0 0x55eec3109430 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.995+0000 7f325a36d740 1 --2- 172.21.15.43:0/1657744072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eec30f6e10 0x55eec30f4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.995+0000 7f3238ff9640 0 client.20712 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:49:35.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.995+0000 7f325a36d740 1 --2- 172.21.15.43:0/1657744072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30e9c00 0x55eec30f3ac0 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.995+0000 7f325a36d740 1 --2- 172.21.15.43:0/1657744072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eec30e9250 0x55eec30f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.995+0000 7f325a36d740 1 -- 172.21.15.43:0/1657744072 >> 172.21.15.43:0/1657744072 conn(0x55eec30e3ba0 msgr2=0x55eec30e58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:35.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.995+0000 7f325a36d740 1 -- 172.21.15.43:0/1657744072 shutdown_connections 2024-04-21T08:49:35.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.995+0000 7f325a36d740 1 -- 172.21.15.43:0/1657744072 wait complete. 2024-04-21T08:49:35.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:34.995+0000 7f325a36d740 20 client.20712 trim_cache size 0 max 16384 2024-04-21T08:49:35.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.008+0000 7f325a36d740 1 Processor -- start 2024-04-21T08:49:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.009+0000 7f325a36d740 1 -- start start 2024-04-21T08:49:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.009+0000 7f325a36d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eec30e9250 0x55eec3102c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.009+0000 7f325a36d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eec30e9c00 0x55eec30f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.009+0000 7f325a36d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30f6e10 0x55eec30f6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.009+0000 7f325a36d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f32501ad3a0 con 0x55eec30f6e10 2024-04-21T08:49:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.009+0000 7f325a36d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f32501aeee0 con 0x55eec30e9c00 2024-04-21T08:49:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.009+0000 7f325a36d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32501ada70 con 0x55eec30e9250 2024-04-21T08:49:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.009+0000 7f3255d1b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30f6e10 0x55eec30f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.009+0000 7f3255d1b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30f6e10 0x55eec30f6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48038/0 (socket says 172.21.15.43:48038) 2024-04-21T08:49:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.009+0000 7f3255d1b640 1 -- 172.21.15.43:0/2239982025 learned_addr learned my addr 172.21.15.43:0/2239982025 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.009+0000 7f3254d19640 1 --2- 172.21.15.43:0/2239982025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eec30e9c00 0x55eec30f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.009+0000 7f325551a640 1 --2- 172.21.15.43:0/2239982025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eec30e9250 0x55eec3102c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.009+0000 7f3255d1b640 1 -- 172.21.15.43:0/2239982025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eec30e9250 msgr2=0x55eec3102c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.009+0000 7f3255d1b640 1 --2- 172.21.15.43:0/2239982025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eec30e9250 0x55eec3102c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.009+0000 7f3255d1b640 1 -- 172.21.15.43:0/2239982025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eec30e9c00 msgr2=0x55eec30f61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.009+0000 7f3255d1b640 1 --2- 172.21.15.43:0/2239982025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eec30e9c00 0x55eec30f61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.009+0000 7f3255d1b640 1 -- 172.21.15.43:0/2239982025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eec30f2a70 con 0x55eec30f6e10 2024-04-21T08:49:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.009+0000 7f3255d1b640 1 --2- 172.21.15.43:0/2239982025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30f6e10 0x55eec30f6730 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f3248014940 tx=0x7f3248017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aba6516db9dc3c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.009+0000 7f323affd640 1 -- 172.21.15.43:0/2239982025 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3248009b00 con 0x55eec30f6e10 2024-04-21T08:49:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.010+0000 7f323affd640 1 -- 172.21.15.43:0/2239982025 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3248014490 con 0x55eec30f6e10 2024-04-21T08:49:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.010+0000 7f323affd640 1 -- 172.21.15.43:0/2239982025 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3248014050 con 0x55eec30f6e10 2024-04-21T08:49:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.010+0000 7f325a36d740 1 -- 172.21.15.43:0/2239982025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30f6e10 msgr2=0x55eec30f6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.010+0000 7f325a36d740 1 --2- 172.21.15.43:0/2239982025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30f6e10 0x55eec30f6730 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f3248014940 tx=0x7f3248017000 comp rx=0 tx=0).stop 2024-04-21T08:49:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.010+0000 7f325a36d740 1 -- 172.21.15.43:0/2239982025 shutdown_connections 2024-04-21T08:49:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.010+0000 7f325a36d740 1 --2- 172.21.15.43:0/2239982025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30f6e10 0x55eec30f6730 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.010+0000 7f325a36d740 1 --2- 172.21.15.43:0/2239982025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eec30e9c00 0x55eec30f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.010+0000 7f325a36d740 1 --2- 172.21.15.43:0/2239982025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eec30e9250 0x55eec3102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.010+0000 7f325a36d740 1 -- 172.21.15.43:0/2239982025 >> 172.21.15.43:0/2239982025 conn(0x55eec30e3ba0 msgr2=0x55eec30e2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.010+0000 7f325a36d740 1 -- 172.21.15.43:0/2239982025 shutdown_connections 2024-04-21T08:49:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.010+0000 7f325a36d740 1 -- 172.21.15.43:0/2239982025 wait complete. 2024-04-21T08:49:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.010+0000 7f325a36d740 1 -- start start 2024-04-21T08:49:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.011+0000 7f325a36d740 1 Processor -- start 2024-04-21T08:49:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.011+0000 7f325a36d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eec30e9250 0x55eec30f0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.011+0000 7f325a36d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30e9c00 0x55eec30f1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.011+0000 7f325a36d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eec30f6e10 0x55eec316c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.011+0000 7f325a36d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f32501ae810 con 0x55eec30e9c00 2024-04-21T08:49:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.011+0000 7f325a36d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f32501ae140 con 0x55eec30e9250 2024-04-21T08:49:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.011+0000 7f325a36d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32501ab3a0 con 0x55eec30f6e10 2024-04-21T08:49:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.011+0000 7f325a36d740 10 client.0 _ll_register_callbacks cb 0x7f32400043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.011+0000 7f3254d19640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30e9c00 0x55eec30f1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.011+0000 7f3254d19640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30e9c00 0x55eec30f1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48054/0 (socket says 172.21.15.43:48054) 2024-04-21T08:49:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.011+0000 7f3254d19640 1 -- 172.21.15.43:0/858025871 learned_addr learned my addr 172.21.15.43:0/858025871 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.011+0000 7f3255d1b640 1 --2- 172.21.15.43:0/858025871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eec30f6e10 0x55eec316c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.011+0000 7f325551a640 1 --2- 172.21.15.43:0/858025871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eec30e9250 0x55eec30f0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.011+0000 7f3254d19640 1 -- 172.21.15.43:0/858025871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eec30f6e10 msgr2=0x55eec316c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.011+0000 7f3254d19640 1 --2- 172.21.15.43:0/858025871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eec30f6e10 0x55eec316c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.011+0000 7f3254d19640 1 -- 172.21.15.43:0/858025871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eec30e9250 msgr2=0x55eec30f0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.011+0000 7f3254d19640 1 --2- 172.21.15.43:0/858025871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eec30e9250 0x55eec30f0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.011+0000 7f3254d19640 1 -- 172.21.15.43:0/858025871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eec316cab0 con 0x55eec30e9c00 2024-04-21T08:49:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.012+0000 7f3254d19640 1 --2- 172.21.15.43:0/858025871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30e9c00 0x55eec30f1310 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f3240003d10 tx=0x7f324002c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82391bf45dd241a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.012+0000 7f3238ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.012+0000 7f3238ff9640 1 -- 172.21.15.43:0/858025871 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3240000e60 con 0x55eec30e9c00 2024-04-21T08:49:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.012+0000 7f3238ff9640 1 -- 172.21.15.43:0/858025871 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f3240034760 con 0x55eec30e9c00 2024-04-21T08:49:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.012+0000 7f325a36d740 1 -- 172.21.15.43:0/858025871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3248014490 con 0x55eec30e9c00 2024-04-21T08:49:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.012+0000 7f3238ff9640 1 -- 172.21.15.43:0/858025871 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f324002c5d0 con 0x55eec30e9c00 2024-04-21T08:49:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.012+0000 7f325a36d740 10 client.20718 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.012+0000 7f325a36d740 1 -- 172.21.15.43:0/858025871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eec316dcc0 con 0x55eec30e9c00 2024-04-21T08:49:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.012+0000 7f3238ff9640 1 -- 172.21.15.43:0/858025871 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3248014490 con 0x55eec30e9c00 2024-04-21T08:49:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.012+0000 7f3238ff9640 1 -- 172.21.15.43:0/858025871 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3240055800 con 0x55eec30e9c00 2024-04-21T08:49:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.012+0000 7f325a36d740 10 client.20718 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.012+0000 7f325a36d740 10 client.20718 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:49:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.012+0000 7f325a36d740 20 client.20718 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.012+0000 7f325a36d740 1 --2- 172.21.15.43:0/858025871 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55eec316e490 0x55eec31708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.012+0000 7f325a36d740 4 client.20718 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.012+0000 7f325a36d740 1 -- 172.21.15.43:0/858025871 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eec305bc10 con 0x55eec316e490 2024-04-21T08:49:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.013+0000 7f325551a640 1 --2- 172.21.15.43:0/858025871 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55eec316e490 0x55eec31708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.013+0000 7f325551a640 1 --2- 172.21.15.43:0/858025871 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55eec316e490 0x55eec31708b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6a7924f7c4f10b80 server_cookie=7b25a72c5d6fdeed in_seq=0 out_seq=0 2024-04-21T08:49:35.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.013+0000 7f3238ff9640 10 client.20718 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:49:35.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.013+0000 7f3238ff9640 1 -- 172.21.15.43:0/858025871 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eec305bc10 con 0x55eec316e490 2024-04-21T08:49:35.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.013+0000 7f3238ff9640 10 client.20718 handle_command_reply: tid=0 2024-04-21T08:49:35.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.013+0000 7f325a36d740 1 client.20718 shutdown 2024-04-21T08:49:35.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.013+0000 7f325a36d740 1 -- 172.21.15.43:0/858025871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30e9c00 msgr2=0x55eec30f1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.013+0000 7f325a36d740 1 --2- 172.21.15.43:0/858025871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30e9c00 0x55eec30f1310 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f3240003d10 tx=0x7f324002c2b0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.013+0000 7f325a36d740 1 -- 172.21.15.43:0/858025871 shutdown_connections 2024-04-21T08:49:35.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.013+0000 7f325a36d740 1 --2- 172.21.15.43:0/858025871 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55eec316e490 0x55eec31708b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.014+0000 7f325a36d740 1 --2- 172.21.15.43:0/858025871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eec30f6e10 0x55eec316c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.014+0000 7f325a36d740 1 --2- 172.21.15.43:0/858025871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eec30e9c00 0x55eec30f1310 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.014+0000 7f325a36d740 1 --2- 172.21.15.43:0/858025871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eec30e9250 0x55eec30f0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.014+0000 7f3238ff9640 0 client.20718 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:49:35.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.014+0000 7f325a36d740 1 -- 172.21.15.43:0/858025871 >> 172.21.15.43:0/858025871 conn(0x55eec30e3ba0 msgr2=0x55eec30e4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:35.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.014+0000 7f325a36d740 1 -- 172.21.15.43:0/858025871 shutdown_connections 2024-04-21T08:49:35.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.014+0000 7f325a36d740 1 -- 172.21.15.43:0/858025871 wait complete. 2024-04-21T08:49:35.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.014+0000 7f325a36d740 20 client.20718 trim_cache size 0 max 16384 2024-04-21T08:49:35.020 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:49:35.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.014+0000 7f325777e640 1 -- 172.21.15.43:0/2808918980 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f323c0a3d20 msgr2=0x7f323c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.014+0000 7f325777e640 1 --2- 172.21.15.43:0/2808918980 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f323c0a3d20 0x7f323c0a61e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3248021000 tx=0x7f3248020000 comp rx=0 tx=0).stop 2024-04-21T08:49:35.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.014+0000 7f325777e640 1 -- 172.21.15.43:0/2808918980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3250102190 msgr2=0x7f3250198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.014+0000 7f325777e640 1 --2- 172.21.15.43:0/2808918980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3250102190 0x7f3250198760 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f324c00c890 tx=0x7f324c00cd50 comp rx=0 tx=0).stop 2024-04-21T08:49:35.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.014+0000 7f325777e640 1 -- 172.21.15.43:0/2808918980 shutdown_connections 2024-04-21T08:49:35.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.014+0000 7f325777e640 1 --2- 172.21.15.43:0/2808918980 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f323c0a3d20 0x7f323c0a61e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.014+0000 7f325777e640 1 --2- 172.21.15.43:0/2808918980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f325019d020 0x7f325019d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.014+0000 7f325777e640 1 --2- 172.21.15.43:0/2808918980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3250107260 0x7f3250198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.014+0000 7f325777e640 1 --2- 172.21.15.43:0/2808918980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3250102190 0x7f3250198760 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.014+0000 7f325777e640 1 -- 172.21.15.43:0/2808918980 >> 172.21.15.43:0/2808918980 conn(0x7f32500fd990 msgr2=0x7f32500fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:35.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.014+0000 7f325777e640 1 -- 172.21.15.43:0/2808918980 shutdown_connections 2024-04-21T08:49:35.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.014+0000 7f325777e640 1 -- 172.21.15.43:0/2808918980 wait complete. 2024-04-21T08:49:35.025 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:49:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.130+0000 7feff7bb1640 1 -- 172.21.15.43:0/1382969156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feff01010a0 msgr2=0x7feff0101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.130+0000 7feff7bb1640 1 --2- 172.21.15.43:0/1382969156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feff01010a0 0x7feff0101480 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fefec009900 tx=0x7fefec02bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.130+0000 7feff7bb1640 1 -- 172.21.15.43:0/1382969156 shutdown_connections 2024-04-21T08:49:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.130+0000 7feff7bb1640 1 --2- 172.21.15.43:0/1382969156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feff0106170 0x7feff010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.130+0000 7feff7bb1640 1 --2- 172.21.15.43:0/1382969156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feff0101a50 0x7feff0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.130+0000 7feff7bb1640 1 --2- 172.21.15.43:0/1382969156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feff01010a0 0x7feff0101480 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.130+0000 7feff7bb1640 1 -- 172.21.15.43:0/1382969156 >> 172.21.15.43:0/1382969156 conn(0x7feff0077eb0 msgr2=0x7feff00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.130+0000 7feff7bb1640 1 -- 172.21.15.43:0/1382969156 shutdown_connections 2024-04-21T08:49:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.130+0000 7feff7bb1640 1 -- 172.21.15.43:0/1382969156 wait complete. 2024-04-21T08:49:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.130+0000 7feff7bb1640 1 Processor -- start 2024-04-21T08:49:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.130+0000 7feff7bb1640 1 -- start start 2024-04-21T08:49:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.131+0000 7feff7bb1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feff01010a0 0x7feff0199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.131+0000 7feff7bb1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feff0106170 0x7feff0199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.131+0000 7feff7bb1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feff019a020 0x7feff01a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.131+0000 7feff7bb1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feff0111120 con 0x7feff01010a0 2024-04-21T08:49:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.131+0000 7feff7bb1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feff0110fa0 con 0x7feff0106170 2024-04-21T08:49:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.131+0000 7feff7bb1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feff01112a0 con 0x7feff019a020 2024-04-21T08:49:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.131+0000 7feff594d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feff01010a0 0x7feff0199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.131+0000 7feff594d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feff01010a0 0x7feff0199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48080/0 (socket says 172.21.15.43:48080) 2024-04-21T08:49:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.131+0000 7feff594d640 1 -- 172.21.15.43:0/320232654 learned_addr learned my addr 172.21.15.43:0/320232654 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.131+0000 7feff614e640 1 --2- 172.21.15.43:0/320232654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feff019a020 0x7feff01a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.131+0000 7feff514c640 1 --2- 172.21.15.43:0/320232654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feff0106170 0x7feff0199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.131+0000 7feff594d640 1 -- 172.21.15.43:0/320232654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feff019a020 msgr2=0x7feff01a4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.131+0000 7feff594d640 1 --2- 172.21.15.43:0/320232654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feff019a020 0x7feff01a4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.131+0000 7feff594d640 1 -- 172.21.15.43:0/320232654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feff0106170 msgr2=0x7feff0199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.131+0000 7feff594d640 1 --2- 172.21.15.43:0/320232654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feff0106170 0x7feff0199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.131+0000 7feff594d640 1 -- 172.21.15.43:0/320232654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff01a4da0 con 0x7feff01010a0 2024-04-21T08:49:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.131+0000 7feff514c640 1 --2- 172.21.15.43:0/320232654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feff0106170 0x7feff0199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.131+0000 7feff594d640 1 --2- 172.21.15.43:0/320232654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feff01010a0 0x7feff0199500 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fefec000c00 tx=0x7fefec02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.131+0000 7fefe6ffd640 1 -- 172.21.15.43:0/320232654 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec043020 con 0x7feff01010a0 2024-04-21T08:49:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.131+0000 7feff7bb1640 1 -- 172.21.15.43:0/320232654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff01a5030 con 0x7feff01010a0 2024-04-21T08:49:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.131+0000 7fefe6ffd640 1 -- 172.21.15.43:0/320232654 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefec0052b0 con 0x7feff01010a0 2024-04-21T08:49:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.131+0000 7fefe6ffd640 1 -- 172.21.15.43:0/320232654 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec003c60 con 0x7feff01010a0 2024-04-21T08:49:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.131+0000 7feff7bb1640 1 -- 172.21.15.43:0/320232654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff01a5580 con 0x7feff01010a0 2024-04-21T08:49:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.134+0000 7fefe6ffd640 1 -- 172.21.15.43:0/320232654 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fefec046020 con 0x7feff01010a0 2024-04-21T08:49:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.134+0000 7feffa79c740 1 Processor -- start 2024-04-21T08:49:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.134+0000 7feffa79c740 1 -- start start 2024-04-21T08:49:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.134+0000 7feffa79c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625bd12b4d0 0x5625bd12b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.134+0000 7feffa79c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd12be80 0x5625bd134430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.134+0000 7feffa79c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625bd134a00 0x5625bd136e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.134+0000 7feffa79c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5625bd0901f0 con 0x5625bd12be80 2024-04-21T08:49:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.134+0000 7feffa79c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5625bd02d4b0 con 0x5625bd134a00 2024-04-21T08:49:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.134+0000 7feffa79c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5625bd08c1b0 con 0x5625bd12b4d0 2024-04-21T08:49:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.134+0000 7feff594d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd12be80 0x5625bd134430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.134+0000 7feff594d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd12be80 0x5625bd134430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48084/0 (socket says 172.21.15.43:48084) 2024-04-21T08:49:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.135+0000 7feff594d640 1 -- 172.21.15.43:0/1338774776 learned_addr learned my addr 172.21.15.43:0/1338774776 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.135+0000 7feff514c640 1 --2- 172.21.15.43:0/1338774776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625bd134a00 0x5625bd136e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.135+0000 7feff614e640 1 --2- 172.21.15.43:0/1338774776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625bd12b4d0 0x5625bd12b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.135+0000 7fefe6ffd640 1 --2- 172.21.15.43:0/320232654 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fefdc0a3d20 0x7fefdc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.135+0000 7feff614e640 1 --2- 172.21.15.43:0/320232654 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fefdc0a3d20 0x7fefdc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.135+0000 7feff594d640 1 -- 172.21.15.43:0/1338774776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625bd12b4d0 msgr2=0x5625bd12b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.135+0000 7feff594d640 1 --2- 172.21.15.43:0/1338774776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625bd12b4d0 0x5625bd12b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.135+0000 7feff594d640 1 -- 172.21.15.43:0/1338774776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625bd134a00 msgr2=0x5625bd136e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.135+0000 7feff594d640 1 --2- 172.21.15.43:0/1338774776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625bd134a00 0x5625bd136e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.135+0000 7feff594d640 1 -- 172.21.15.43:0/1338774776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625bd01f1b0 con 0x5625bd12be80 2024-04-21T08:49:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.135+0000 7fefe6ffd640 1 -- 172.21.15.43:0/320232654 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fefec0f2d50 con 0x7feff01010a0 2024-04-21T08:49:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.135+0000 7feff594d640 1 --2- 172.21.15.43:0/1338774776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd12be80 0x5625bd134430 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7fefec0f0100 tx=0x7fefec009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f28b4da366570 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.135+0000 7feff614e640 1 --2- 172.21.15.43:0/320232654 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fefdc0a3d20 0x7fefdc0a61e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fefe8005fd0 tx=0x7fefe8004ab0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.135+0000 7fefdbfff640 1 -- 172.21.15.43:0/1338774776 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec0be7b0 con 0x5625bd12be80 2024-04-21T08:49:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.135+0000 7fefdbfff640 1 -- 172.21.15.43:0/1338774776 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefec02fea0 con 0x5625bd12be80 2024-04-21T08:49:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.135+0000 7fefdbfff640 1 -- 172.21.15.43:0/1338774776 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec02cca0 con 0x5625bd12be80 2024-04-21T08:49:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.136+0000 7feffa79c740 1 -- 172.21.15.43:0/1338774776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd12be80 msgr2=0x5625bd134430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.136+0000 7feffa79c740 1 --2- 172.21.15.43:0/1338774776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd12be80 0x5625bd134430 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7fefec0f0100 tx=0x7fefec009260 comp rx=0 tx=0).stop 2024-04-21T08:49:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.136+0000 7feffa79c740 1 -- 172.21.15.43:0/1338774776 shutdown_connections 2024-04-21T08:49:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.136+0000 7feffa79c740 1 --2- 172.21.15.43:0/1338774776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625bd134a00 0x5625bd136e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.136+0000 7feffa79c740 1 --2- 172.21.15.43:0/1338774776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd12be80 0x5625bd134430 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.136+0000 7feffa79c740 1 --2- 172.21.15.43:0/1338774776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625bd12b4d0 0x5625bd12b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.136+0000 7feffa79c740 1 -- 172.21.15.43:0/1338774776 >> 172.21.15.43:0/1338774776 conn(0x5625bd125ba0 msgr2=0x5625bd125f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.136+0000 7feffa79c740 1 -- 172.21.15.43:0/1338774776 shutdown_connections 2024-04-21T08:49:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.136+0000 7feffa79c740 1 -- 172.21.15.43:0/1338774776 wait complete. 2024-04-21T08:49:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.137+0000 7feffa79c740 1 -- start start 2024-04-21T08:49:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.137+0000 7feffa79c740 1 Processor -- start 2024-04-21T08:49:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.137+0000 7feffa79c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625bd12b4d0 0x5625bd134270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.137+0000 7feffa79c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625bd12be80 0x5625bd131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.137+0000 7feffa79c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd134a00 0x5625bd131b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.137+0000 7feffa79c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5625bd137730 con 0x5625bd134a00 2024-04-21T08:49:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.137+0000 7feffa79c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5625bd049500 con 0x5625bd12be80 2024-04-21T08:49:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.137+0000 7feffa79c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5625bd1378b0 con 0x5625bd12b4d0 2024-04-21T08:49:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.137+0000 7feffa79c740 10 client.0 _ll_register_callbacks cb 0x5625bd017b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.137+0000 7feff614e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd134a00 0x5625bd131b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.137+0000 7feff614e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd134a00 0x5625bd131b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48100/0 (socket says 172.21.15.43:48100) 2024-04-21T08:49:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.137+0000 7feff614e640 1 -- 172.21.15.43:0/3715173009 learned_addr learned my addr 172.21.15.43:0/3715173009 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.137+0000 7feff594d640 1 --2- 172.21.15.43:0/3715173009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625bd12b4d0 0x5625bd134270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.137+0000 7feff514c640 1 --2- 172.21.15.43:0/3715173009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625bd12be80 0x5625bd131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.138+0000 7feff614e640 1 -- 172.21.15.43:0/3715173009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625bd12b4d0 msgr2=0x5625bd134270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.138+0000 7feff614e640 1 --2- 172.21.15.43:0/3715173009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625bd12b4d0 0x5625bd134270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.138+0000 7feff614e640 1 -- 172.21.15.43:0/3715173009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625bd12be80 msgr2=0x5625bd131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.138+0000 7feff614e640 1 --2- 172.21.15.43:0/3715173009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625bd12be80 0x5625bd131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.138+0000 7feff614e640 1 -- 172.21.15.43:0/3715173009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625bcda23f0 con 0x5625bd134a00 2024-04-21T08:49:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.138+0000 7feff614e640 1 --2- 172.21.15.43:0/3715173009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd134a00 0x5625bd131b40 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7fefe8010920 tx=0x7fefe8009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc02e6c374f020c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:35.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.138+0000 7fefd97fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:35.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.138+0000 7fefd97fa640 1 -- 172.21.15.43:0/3715173009 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe801dd40 con 0x5625bd134a00 2024-04-21T08:49:35.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.138+0000 7fefd97fa640 1 -- 172.21.15.43:0/3715173009 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fefe8004b20 con 0x5625bd134a00 2024-04-21T08:49:35.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.138+0000 7feffa79c740 1 -- 172.21.15.43:0/3715173009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefec02fea0 con 0x5625bd134a00 2024-04-21T08:49:35.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.138+0000 7fefd97fa640 1 -- 172.21.15.43:0/3715173009 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fefe802f050 con 0x5625bd134a00 2024-04-21T08:49:35.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.138+0000 7feffa79c740 10 client.20730 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:35.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.138+0000 7feffa79c740 1 -- 172.21.15.43:0/3715173009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5625bce6c7d0 con 0x5625bd134a00 2024-04-21T08:49:35.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.138+0000 7fefd97fa640 1 -- 172.21.15.43:0/3715173009 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefec02fea0 con 0x5625bd134a00 2024-04-21T08:49:35.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.139+0000 7fefd97fa640 1 -- 172.21.15.43:0/3715173009 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fefe800f040 con 0x5625bd134a00 2024-04-21T08:49:35.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.139+0000 7feffa79c740 10 client.20730 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:35.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.139+0000 7feffa79c740 10 client.20730 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:49:35.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.139+0000 7feffa79c740 20 client.20730 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:35.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.139+0000 7feffa79c740 1 --2- 172.21.15.43:0/3715173009 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5625bd1448d0 0x5625bd146cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.139+0000 7feffa79c740 4 client.20730 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:35.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.139+0000 7feffa79c740 1 -- 172.21.15.43:0/3715173009 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625bd02f930 con 0x5625bd1448d0 2024-04-21T08:49:35.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.139+0000 7feff594d640 1 --2- 172.21.15.43:0/3715173009 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5625bd1448d0 0x5625bd146cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.139+0000 7fefd97fa640 10 client.20730 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:49:35.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.139+0000 7feff594d640 1 --2- 172.21.15.43:0/3715173009 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5625bd1448d0 0x5625bd146cf0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ccc79f30d82d79b5 server_cookie=dfc186afb5e074e2 in_seq=0 out_seq=0 2024-04-21T08:49:35.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.140+0000 7fefd97fa640 1 -- 172.21.15.43:0/3715173009 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5625bd02f930 con 0x5625bd1448d0 2024-04-21T08:49:35.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.140+0000 7fefd97fa640 10 client.20730 handle_command_reply: tid=0 2024-04-21T08:49:35.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.140+0000 7feffa79c740 1 client.20730 shutdown 2024-04-21T08:49:35.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.140+0000 7feffa79c740 1 -- 172.21.15.43:0/3715173009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd134a00 msgr2=0x5625bd131b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.140+0000 7feffa79c740 1 --2- 172.21.15.43:0/3715173009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd134a00 0x5625bd131b40 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7fefe8010920 tx=0x7fefe8009600 comp rx=0 tx=0).stop 2024-04-21T08:49:35.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.140+0000 7feffa79c740 1 -- 172.21.15.43:0/3715173009 shutdown_connections 2024-04-21T08:49:35.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.140+0000 7feffa79c740 1 --2- 172.21.15.43:0/3715173009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd134a00 0x5625bd131b40 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.140+0000 7feffa79c740 1 --2- 172.21.15.43:0/3715173009 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5625bd1448d0 0x5625bd146cf0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.140+0000 7feffa79c740 1 --2- 172.21.15.43:0/3715173009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625bd12be80 0x5625bd131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.140+0000 7feffa79c740 1 --2- 172.21.15.43:0/3715173009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625bd12b4d0 0x5625bd134270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.140+0000 7fefd97fa640 0 client.20730 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:49:35.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.140+0000 7feffa79c740 1 -- 172.21.15.43:0/3715173009 >> 172.21.15.43:0/3715173009 conn(0x5625bd125ba0 msgr2=0x5625bd1278f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:35.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.140+0000 7feffa79c740 1 -- 172.21.15.43:0/3715173009 shutdown_connections 2024-04-21T08:49:35.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.140+0000 7feffa79c740 1 -- 172.21.15.43:0/3715173009 wait complete. 2024-04-21T08:49:35.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.141+0000 7feffa79c740 20 client.20730 trim_cache size 0 max 16384 2024-04-21T08:49:35.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.154+0000 7feffa79c740 1 Processor -- start 2024-04-21T08:49:35.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.154+0000 7feffa79c740 1 -- start start 2024-04-21T08:49:35.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.154+0000 7feffa79c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd125ba0 0x5625bd140430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.154+0000 7feffa79c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625bd12b4d0 0x5625bd133d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.154+0000 7feffa79c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625bd12be80 0x5625bd134270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.154+0000 7feffa79c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feff01af4f0 con 0x5625bd125ba0 2024-04-21T08:49:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.154+0000 7feffa79c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feff01b1030 con 0x5625bd12be80 2024-04-21T08:49:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.154+0000 7feffa79c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feff01afbc0 con 0x5625bd12b4d0 2024-04-21T08:49:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.154+0000 7feff594d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd125ba0 0x5625bd140430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.154+0000 7feff594d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd125ba0 0x5625bd140430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48106/0 (socket says 172.21.15.43:48106) 2024-04-21T08:49:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.154+0000 7feff594d640 1 -- 172.21.15.43:0/400503189 learned_addr learned my addr 172.21.15.43:0/400503189 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.154+0000 7feff514c640 1 --2- 172.21.15.43:0/400503189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625bd12b4d0 0x5625bd133d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.154+0000 7feff614e640 1 --2- 172.21.15.43:0/400503189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625bd12be80 0x5625bd134270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.154+0000 7feff594d640 1 -- 172.21.15.43:0/400503189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625bd12b4d0 msgr2=0x5625bd133d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.154+0000 7feff594d640 1 --2- 172.21.15.43:0/400503189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625bd12b4d0 0x5625bd133d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.155+0000 7feff594d640 1 -- 172.21.15.43:0/400503189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625bd12be80 msgr2=0x5625bd134270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.155+0000 7feff594d640 1 --2- 172.21.15.43:0/400503189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625bd12be80 0x5625bd134270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.155+0000 7feff594d640 1 -- 172.21.15.43:0/400503189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625bd132140 con 0x5625bd125ba0 2024-04-21T08:49:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.155+0000 7feff614e640 1 --2- 172.21.15.43:0/400503189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625bd12be80 0x5625bd134270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.155+0000 7feff514c640 1 --2- 172.21.15.43:0/400503189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625bd12b4d0 0x5625bd133d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.155+0000 7feff594d640 1 --2- 172.21.15.43:0/400503189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd125ba0 0x5625bd140430 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7fefec005ab0 tx=0x7fefec005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a3005d32511133 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.155+0000 7fefdb7fe640 1 -- 172.21.15.43:0/400503189 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec04c1b0 con 0x5625bd125ba0 2024-04-21T08:49:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.155+0000 7fefdb7fe640 1 -- 172.21.15.43:0/400503189 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefec0bf0e0 con 0x5625bd125ba0 2024-04-21T08:49:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.155+0000 7fefdb7fe640 1 -- 172.21.15.43:0/400503189 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec0be340 con 0x5625bd125ba0 2024-04-21T08:49:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.155+0000 7feffa79c740 1 -- 172.21.15.43:0/400503189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd125ba0 msgr2=0x5625bd140430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.155+0000 7feffa79c740 1 --2- 172.21.15.43:0/400503189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd125ba0 0x5625bd140430 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7fefec005ab0 tx=0x7fefec005bd0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.155+0000 7feffa79c740 1 -- 172.21.15.43:0/400503189 shutdown_connections 2024-04-21T08:49:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.155+0000 7feffa79c740 1 --2- 172.21.15.43:0/400503189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625bd12be80 0x5625bd134270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.155+0000 7feffa79c740 1 --2- 172.21.15.43:0/400503189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625bd12b4d0 0x5625bd133d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.155+0000 7feffa79c740 1 --2- 172.21.15.43:0/400503189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd125ba0 0x5625bd140430 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.156+0000 7feffa79c740 1 -- 172.21.15.43:0/400503189 >> 172.21.15.43:0/400503189 conn(0x5625bd131ca0 msgr2=0x5625bd12cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.156+0000 7feffa79c740 1 -- 172.21.15.43:0/400503189 shutdown_connections 2024-04-21T08:49:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.156+0000 7feffa79c740 1 -- 172.21.15.43:0/400503189 wait complete. 2024-04-21T08:49:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.156+0000 7feffa79c740 1 -- start start 2024-04-21T08:49:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.156+0000 7feffa79c740 1 Processor -- start 2024-04-21T08:49:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.156+0000 7feffa79c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd125ba0 0x5625bd138590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.156+0000 7feffa79c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625bd12b4d0 0x5625bd138ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.156+0000 7feffa79c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625bd12be80 0x5625bd139010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.156+0000 7feffa79c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feff01b0960 con 0x5625bd125ba0 2024-04-21T08:49:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.156+0000 7feffa79c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feff01b0290 con 0x5625bd12be80 2024-04-21T08:49:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.156+0000 7feffa79c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feff01ad4f0 con 0x5625bd12b4d0 2024-04-21T08:49:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.156+0000 7feffa79c740 10 client.0 _ll_register_callbacks cb 0x7fefe80105f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.156+0000 7feff594d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd125ba0 0x5625bd138590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.156+0000 7feff594d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd125ba0 0x5625bd138590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48116/0 (socket says 172.21.15.43:48116) 2024-04-21T08:49:35.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.157+0000 7feff594d640 1 -- 172.21.15.43:0/1258444266 learned_addr learned my addr 172.21.15.43:0/1258444266 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:35.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.157+0000 7feff614e640 1 --2- 172.21.15.43:0/1258444266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625bd12be80 0x5625bd139010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.157+0000 7feff514c640 1 --2- 172.21.15.43:0/1258444266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625bd12b4d0 0x5625bd138ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.157+0000 7feff594d640 1 -- 172.21.15.43:0/1258444266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625bd12b4d0 msgr2=0x5625bd138ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.157+0000 7feff594d640 1 --2- 172.21.15.43:0/1258444266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625bd12b4d0 0x5625bd138ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.157+0000 7feff594d640 1 -- 172.21.15.43:0/1258444266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625bd12be80 msgr2=0x5625bd139010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.157+0000 7feff594d640 1 --2- 172.21.15.43:0/1258444266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625bd12be80 0x5625bd139010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.157+0000 7feff594d640 1 -- 172.21.15.43:0/1258444266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625bd1aa080 con 0x5625bd125ba0 2024-04-21T08:49:35.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.157+0000 7feff594d640 1 --2- 172.21.15.43:0/1258444266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd125ba0 0x5625bd138590 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7fefec0bcc90 tx=0x7fefec005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1e34e6d5d15485 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:35.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.157+0000 7fefd97fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.160+0000 7fefd97fa640 1 -- 172.21.15.43:0/1258444266 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec0be520 con 0x5625bd125ba0 2024-04-21T08:49:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.160+0000 7fefd97fa640 1 -- 172.21.15.43:0/1258444266 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fefec005460 con 0x5625bd125ba0 2024-04-21T08:49:35.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.160+0000 7feffa79c740 1 -- 172.21.15.43:0/1258444266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefec0bf0e0 con 0x5625bd125ba0 2024-04-21T08:49:35.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.160+0000 7fefd97fa640 1 -- 172.21.15.43:0/1258444266 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fefec0be340 con 0x5625bd125ba0 2024-04-21T08:49:35.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.160+0000 7feffa79c740 10 client.20736 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:35.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.160+0000 7feffa79c740 1 -- 172.21.15.43:0/1258444266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5625bd1ab2d0 con 0x5625bd125ba0 2024-04-21T08:49:35.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.160+0000 7fefd97fa640 1 -- 172.21.15.43:0/1258444266 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefec0bf0e0 con 0x5625bd125ba0 2024-04-21T08:49:35.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.160+0000 7fefd97fa640 1 -- 172.21.15.43:0/1258444266 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fefec0c71c0 con 0x5625bd125ba0 2024-04-21T08:49:35.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.160+0000 7feffa79c740 10 client.20736 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:35.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.160+0000 7feffa79c740 10 client.20736 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:49:35.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.160+0000 7feffa79c740 20 client.20736 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:35.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.160+0000 7feffa79c740 1 --2- 172.21.15.43:0/1258444266 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5625bd134a00 0x5625bd1ad620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.160+0000 7feffa79c740 4 client.20736 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:35.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.160+0000 7feffa79c740 1 -- 172.21.15.43:0/1258444266 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5625bd09dc10 con 0x5625bd134a00 2024-04-21T08:49:35.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.160+0000 7feff514c640 1 --2- 172.21.15.43:0/1258444266 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5625bd134a00 0x5625bd1ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.161+0000 7feff514c640 1 --2- 172.21.15.43:0/1258444266 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5625bd134a00 0x5625bd1ad620 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d156084d893dee server_cookie=602054bc54b7223 in_seq=0 out_seq=0 2024-04-21T08:49:35.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.161+0000 7fefd97fa640 10 client.20736 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:49:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.161+0000 7fefd97fa640 1 -- 172.21.15.43:0/1258444266 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5625bd09dc10 con 0x5625bd134a00 2024-04-21T08:49:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.161+0000 7fefd97fa640 10 client.20736 handle_command_reply: tid=0 2024-04-21T08:49:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.161+0000 7feffa79c740 1 client.20736 shutdown 2024-04-21T08:49:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.161+0000 7feffa79c740 1 -- 172.21.15.43:0/1258444266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd125ba0 msgr2=0x5625bd138590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.161+0000 7feffa79c740 1 --2- 172.21.15.43:0/1258444266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd125ba0 0x5625bd138590 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7fefec0bcc90 tx=0x7fefec005720 comp rx=0 tx=0).stop 2024-04-21T08:49:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.161+0000 7feffa79c740 1 -- 172.21.15.43:0/1258444266 shutdown_connections 2024-04-21T08:49:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.161+0000 7feffa79c740 1 --2- 172.21.15.43:0/1258444266 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5625bd134a00 0x5625bd1ad620 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.161+0000 7feffa79c740 1 --2- 172.21.15.43:0/1258444266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625bd12be80 0x5625bd139010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.161+0000 7fefd97fa640 0 client.20736 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:49:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.161+0000 7feffa79c740 1 --2- 172.21.15.43:0/1258444266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5625bd12b4d0 0x5625bd138ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.161+0000 7feffa79c740 1 --2- 172.21.15.43:0/1258444266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5625bd125ba0 0x5625bd138590 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.161+0000 7feffa79c740 1 -- 172.21.15.43:0/1258444266 >> 172.21.15.43:0/1258444266 conn(0x5625bd131ca0 msgr2=0x5625bd124370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.161+0000 7feffa79c740 1 -- 172.21.15.43:0/1258444266 shutdown_connections 2024-04-21T08:49:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.162+0000 7feffa79c740 1 -- 172.21.15.43:0/1258444266 wait complete. 2024-04-21T08:49:35.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.162+0000 7feffa79c740 20 client.20736 trim_cache size 0 max 16384 2024-04-21T08:49:35.165 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:49:35.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.162+0000 7feff7bb1640 1 -- 172.21.15.43:0/320232654 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fefdc0a3d20 msgr2=0x7fefdc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.162+0000 7feff7bb1640 1 --2- 172.21.15.43:0/320232654 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fefdc0a3d20 0x7fefdc0a61e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fefe8005fd0 tx=0x7fefe8004ab0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.162+0000 7feff7bb1640 1 -- 172.21.15.43:0/320232654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feff01010a0 msgr2=0x7feff0199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.162+0000 7feff7bb1640 1 --2- 172.21.15.43:0/320232654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feff01010a0 0x7feff0199500 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fefec000c00 tx=0x7fefec02c7e0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.162+0000 7feff7bb1640 1 -- 172.21.15.43:0/320232654 shutdown_connections 2024-04-21T08:49:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.162+0000 7feff7bb1640 1 --2- 172.21.15.43:0/320232654 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fefdc0a3d20 0x7fefdc0a61e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.162+0000 7feff7bb1640 1 --2- 172.21.15.43:0/320232654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feff019a020 0x7feff01a4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.162+0000 7feff7bb1640 1 --2- 172.21.15.43:0/320232654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feff0106170 0x7feff0199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.162+0000 7feff7bb1640 1 --2- 172.21.15.43:0/320232654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feff01010a0 0x7feff0199500 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.162+0000 7feff7bb1640 1 -- 172.21.15.43:0/320232654 >> 172.21.15.43:0/320232654 conn(0x7feff0077eb0 msgr2=0x7feff0107340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.162+0000 7feff7bb1640 1 -- 172.21.15.43:0/320232654 shutdown_connections 2024-04-21T08:49:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.162+0000 7feff7bb1640 1 -- 172.21.15.43:0/320232654 wait complete. 2024-04-21T08:49:35.173 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:49:35.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.277+0000 7fef5d746640 1 -- 172.21.15.43:0/2685363451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef58102b40 msgr2=0x7fef58106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.277+0000 7fef5d746640 1 --2- 172.21.15.43:0/2685363451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef58102b40 0x7fef58106b80 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7fef4c009900 tx=0x7fef4c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:35.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.277+0000 7fef5d746640 1 -- 172.21.15.43:0/2685363451 shutdown_connections 2024-04-21T08:49:35.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.277+0000 7fef5d746640 1 --2- 172.21.15.43:0/2685363451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef58107260 0x7fef5810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.277+0000 7fef5d746640 1 --2- 172.21.15.43:0/2685363451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef58102b40 0x7fef58106b80 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.277+0000 7fef5d746640 1 --2- 172.21.15.43:0/2685363451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef58102190 0x7fef58102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.277+0000 7fef5d746640 1 -- 172.21.15.43:0/2685363451 >> 172.21.15.43:0/2685363451 conn(0x7fef580fd990 msgr2=0x7fef580ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:35.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.277+0000 7fef5d746640 1 -- 172.21.15.43:0/2685363451 shutdown_connections 2024-04-21T08:49:35.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.277+0000 7fef5d746640 1 -- 172.21.15.43:0/2685363451 wait complete. 2024-04-21T08:49:35.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.278+0000 7fef5d746640 1 Processor -- start 2024-04-21T08:49:35.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.278+0000 7fef5d746640 1 -- start start 2024-04-21T08:49:35.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.278+0000 7fef5d746640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef58102190 0x7fef581986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.278+0000 7fef5d746640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef58107260 0x7fef58198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.278+0000 7fef5d746640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef5819cfa0 0x7fef5819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.278+0000 7fef5d746640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fef58112180 con 0x7fef5819cfa0 2024-04-21T08:49:35.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.278+0000 7fef5d746640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fef58112000 con 0x7fef58107260 2024-04-21T08:49:35.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.278+0000 7fef5d746640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fef58112300 con 0x7fef58102190 2024-04-21T08:49:35.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.278+0000 7fef577fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef5819cfa0 0x7fef5819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.278+0000 7fef577fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef5819cfa0 0x7fef5819d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48142/0 (socket says 172.21.15.43:48142) 2024-04-21T08:49:35.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.278+0000 7fef577fe640 1 -- 172.21.15.43:0/2249635379 learned_addr learned my addr 172.21.15.43:0/2249635379 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:35.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.279+0000 7fef56ffd640 1 --2- 172.21.15.43:0/2249635379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef58102190 0x7fef581986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.279+0000 7fef567fc640 1 --2- 172.21.15.43:0/2249635379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef58107260 0x7fef58198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.279+0000 7fef577fe640 1 -- 172.21.15.43:0/2249635379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef58102190 msgr2=0x7fef581986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.279+0000 7fef577fe640 1 --2- 172.21.15.43:0/2249635379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef58102190 0x7fef581986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.279+0000 7fef577fe640 1 -- 172.21.15.43:0/2249635379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef58107260 msgr2=0x7fef58198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.279+0000 7fef577fe640 1 --2- 172.21.15.43:0/2249635379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef58107260 0x7fef58198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.279+0000 7fef577fe640 1 -- 172.21.15.43:0/2249635379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5819da90 con 0x7fef5819cfa0 2024-04-21T08:49:35.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.279+0000 7fef567fc640 1 --2- 172.21.15.43:0/2249635379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef58107260 0x7fef58198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:35.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.279+0000 7fef577fe640 1 --2- 172.21.15.43:0/2249635379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef5819cfa0 0x7fef5819d450 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fef5000eaa0 tx=0x7fef5000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:35.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.279+0000 7fef3ffff640 1 -- 172.21.15.43:0/2249635379 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5000cdb0 con 0x7fef5819cfa0 2024-04-21T08:49:35.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.279+0000 7fef5d746640 1 -- 172.21.15.43:0/2249635379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef5819dd80 con 0x7fef5819cfa0 2024-04-21T08:49:35.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.279+0000 7fef3ffff640 1 -- 172.21.15.43:0/2249635379 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef50005020 con 0x7fef5819cfa0 2024-04-21T08:49:35.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.279+0000 7fef3ffff640 1 -- 172.21.15.43:0/2249635379 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef500116d0 con 0x7fef5819cfa0 2024-04-21T08:49:35.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.279+0000 7fef5d746640 1 -- 172.21.15.43:0/2249635379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef581a5570 con 0x7fef5819cfa0 2024-04-21T08:49:35.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.281+0000 7fef3ffff640 1 -- 172.21.15.43:0/2249635379 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fef500051d0 con 0x7fef5819cfa0 2024-04-21T08:49:35.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.282+0000 7fef6040b740 1 Processor -- start 2024-04-21T08:49:35.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.282+0000 7fef6040b740 1 -- start start 2024-04-21T08:49:35.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.282+0000 7fef3ffff640 1 --2- 172.21.15.43:0/2249635379 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fef400a3d20 0x7fef400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.282+0000 7fef6040b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c80e5f74d0 0x55c80e5f78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.282+0000 7fef6040b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e5f7e80 0x55c80e600430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.282+0000 7fef6040b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80e600a00 0x55c80e602e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.282+0000 7fef6040b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c80e55c1f0 con 0x55c80e5f7e80 2024-04-21T08:49:35.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.282+0000 7fef6040b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c80e4f94b0 con 0x55c80e5f74d0 2024-04-21T08:49:35.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.282+0000 7fef6040b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c80e5581b0 con 0x55c80e600a00 2024-04-21T08:49:35.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.282+0000 7fef567fc640 1 --2- 172.21.15.43:0/2249635379 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fef400a3d20 0x7fef400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.282+0000 7fef56ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c80e5f74d0 0x55c80e5f78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.282+0000 7fef3ffff640 1 -- 172.21.15.43:0/2249635379 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fef500d0ed0 con 0x7fef5819cfa0 2024-04-21T08:49:35.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.282+0000 7fef567fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e5f7e80 0x55c80e600430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.282+0000 7fef56ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c80e5f74d0 0x55c80e5f78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:40204/0 (socket says 172.21.15.43:40204) 2024-04-21T08:49:35.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.282+0000 7fef56ffd640 1 -- 172.21.15.43:0/3395794582 learned_addr learned my addr 172.21.15.43:0/3395794582 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:35.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.282+0000 7fef577fe640 1 --2- 172.21.15.43:0/3395794582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80e600a00 0x55c80e602e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.282+0000 7fef56ffd640 1 -- 172.21.15.43:0/3395794582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80e600a00 msgr2=0x55c80e602e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.282+0000 7fef56ffd640 1 --2- 172.21.15.43:0/3395794582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80e600a00 0x55c80e602e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.282+0000 7fef56ffd640 1 -- 172.21.15.43:0/3395794582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e5f7e80 msgr2=0x55c80e600430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.282+0000 7fef56ffd640 1 --2- 172.21.15.43:0/3395794582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e5f7e80 0x55c80e600430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.283+0000 7fef56ffd640 1 -- 172.21.15.43:0/3395794582 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c80e4eb1b0 con 0x55c80e5f74d0 2024-04-21T08:49:35.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.283+0000 7fef567fc640 1 --2- 172.21.15.43:0/3395794582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e5f7e80 0x55c80e600430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:35.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.283+0000 7fef567fc640 1 --2- 172.21.15.43:0/2249635379 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fef400a3d20 0x7fef400a61e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fef4c005770 tx=0x7fef4c02c650 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:35.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.283+0000 7fef56ffd640 1 --2- 172.21.15.43:0/3395794582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c80e5f74d0 0x55c80e5f78b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fef4800cc60 tx=0x7fef4800e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0d49b2168dac55a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:35.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.283+0000 7fef3d7fa640 1 -- 172.21.15.43:0/3395794582 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48014070 con 0x55c80e5f74d0 2024-04-21T08:49:35.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.283+0000 7fef3d7fa640 1 -- 172.21.15.43:0/3395794582 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef480047a0 con 0x55c80e5f74d0 2024-04-21T08:49:35.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.283+0000 7fef3d7fa640 1 -- 172.21.15.43:0/3395794582 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48010650 con 0x55c80e5f74d0 2024-04-21T08:49:35.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.284+0000 7fef6040b740 1 -- 172.21.15.43:0/3395794582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c80e5f74d0 msgr2=0x55c80e5f78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.284+0000 7fef6040b740 1 --2- 172.21.15.43:0/3395794582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c80e5f74d0 0x55c80e5f78b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fef4800cc60 tx=0x7fef4800e720 comp rx=0 tx=0).stop 2024-04-21T08:49:35.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.284+0000 7fef6040b740 1 -- 172.21.15.43:0/3395794582 shutdown_connections 2024-04-21T08:49:35.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.284+0000 7fef6040b740 1 --2- 172.21.15.43:0/3395794582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80e600a00 0x55c80e602e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.284+0000 7fef6040b740 1 --2- 172.21.15.43:0/3395794582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e5f7e80 0x55c80e600430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.284+0000 7fef6040b740 1 --2- 172.21.15.43:0/3395794582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c80e5f74d0 0x55c80e5f78b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.284+0000 7fef6040b740 1 -- 172.21.15.43:0/3395794582 >> 172.21.15.43:0/3395794582 conn(0x55c80e5f1ba0 msgr2=0x55c80e5f1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:35.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.284+0000 7fef6040b740 1 -- 172.21.15.43:0/3395794582 shutdown_connections 2024-04-21T08:49:35.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.284+0000 7fef6040b740 1 -- 172.21.15.43:0/3395794582 wait complete. 2024-04-21T08:49:35.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.284+0000 7fef6040b740 1 -- start start 2024-04-21T08:49:35.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.284+0000 7fef6040b740 1 Processor -- start 2024-04-21T08:49:35.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.284+0000 7fef6040b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80e5f74d0 0x55c80e600350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.284+0000 7fef6040b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c80e5f7e80 0x55c80e5fd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.284+0000 7fef6040b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e600a00 0x55c80e5fdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.284+0000 7fef6040b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c80e603970 con 0x55c80e600a00 2024-04-21T08:49:35.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.284+0000 7fef6040b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c80e515500 con 0x55c80e5f7e80 2024-04-21T08:49:35.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.284+0000 7fef6040b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c80e603af0 con 0x55c80e5f74d0 2024-04-21T08:49:35.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.284+0000 7fef6040b740 10 client.0 _ll_register_callbacks cb 0x55c80e4e3b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:35.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.284+0000 7fef577fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e600a00 0x55c80e5fdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.285+0000 7fef577fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e600a00 0x55c80e5fdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48154/0 (socket says 172.21.15.43:48154) 2024-04-21T08:49:35.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.285+0000 7fef577fe640 1 -- 172.21.15.43:0/4141636195 learned_addr learned my addr 172.21.15.43:0/4141636195 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:35.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.285+0000 7fef56ffd640 1 --2- 172.21.15.43:0/4141636195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80e5f74d0 0x55c80e600350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.285+0000 7fef567fc640 1 --2- 172.21.15.43:0/4141636195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c80e5f7e80 0x55c80e5fd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.285+0000 7fef577fe640 1 -- 172.21.15.43:0/4141636195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80e5f74d0 msgr2=0x55c80e600350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.285+0000 7fef577fe640 1 --2- 172.21.15.43:0/4141636195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80e5f74d0 0x55c80e600350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.285+0000 7fef577fe640 1 -- 172.21.15.43:0/4141636195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c80e5f7e80 msgr2=0x55c80e5fd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.285+0000 7fef577fe640 1 --2- 172.21.15.43:0/4141636195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c80e5f7e80 0x55c80e5fd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.285+0000 7fef577fe640 1 -- 172.21.15.43:0/4141636195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c80e26e3f0 con 0x55c80e600a00 2024-04-21T08:49:35.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.285+0000 7fef577fe640 1 --2- 172.21.15.43:0/4141636195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e600a00 0x55c80e5fdcf0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fef5009c930 tx=0x7fef500b6260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c66022688970378 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:35.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.285+0000 7fef3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:35.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.285+0000 7fef3aff5640 1 -- 172.21.15.43:0/4141636195 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5001f040 con 0x55c80e600a00 2024-04-21T08:49:35.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.285+0000 7fef3aff5640 1 -- 172.21.15.43:0/4141636195 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fef500ce420 con 0x55c80e600a00 2024-04-21T08:49:35.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.285+0000 7fef6040b740 1 -- 172.21.15.43:0/4141636195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef480047a0 con 0x55c80e600a00 2024-04-21T08:49:35.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.285+0000 7fef3aff5640 1 -- 172.21.15.43:0/4141636195 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fef500ce620 con 0x55c80e600a00 2024-04-21T08:49:35.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.285+0000 7fef6040b740 10 client.20748 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:35.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.285+0000 7fef6040b740 1 -- 172.21.15.43:0/4141636195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c80e3387d0 con 0x55c80e600a00 2024-04-21T08:49:35.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.286+0000 7fef3aff5640 1 -- 172.21.15.43:0/4141636195 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef480047a0 con 0x55c80e600a00 2024-04-21T08:49:35.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.286+0000 7fef3aff5640 1 -- 172.21.15.43:0/4141636195 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fef5000f040 con 0x55c80e600a00 2024-04-21T08:49:35.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.286+0000 7fef6040b740 10 client.20748 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:35.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.286+0000 7fef6040b740 10 client.20748 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:49:35.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.286+0000 7fef6040b740 20 client.20748 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:35.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.286+0000 7fef6040b740 1 --2- 172.21.15.43:0/4141636195 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55c80e610ba0 0x55c80e612fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.286+0000 7fef6040b740 4 client.20748 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:35.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.286+0000 7fef6040b740 1 -- 172.21.15.43:0/4141636195 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c80e4fb930 con 0x55c80e610ba0 2024-04-21T08:49:35.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.286+0000 7fef56ffd640 1 --2- 172.21.15.43:0/4141636195 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55c80e610ba0 0x55c80e612fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.286+0000 7fef3aff5640 10 client.20748 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:49:35.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.286+0000 7fef56ffd640 1 --2- 172.21.15.43:0/4141636195 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55c80e610ba0 0x55c80e612fd0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4dddd400059c9e78 server_cookie=744417fb73f660b4 in_seq=0 out_seq=0 2024-04-21T08:49:35.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.287+0000 7fef3aff5640 1 -- 172.21.15.43:0/4141636195 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c80e4fb930 con 0x55c80e610ba0 2024-04-21T08:49:35.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.287+0000 7fef3aff5640 10 client.20748 handle_command_reply: tid=0 2024-04-21T08:49:35.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.287+0000 7fef6040b740 1 client.20748 shutdown 2024-04-21T08:49:35.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.287+0000 7fef6040b740 1 -- 172.21.15.43:0/4141636195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e600a00 msgr2=0x55c80e5fdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.287+0000 7fef6040b740 1 --2- 172.21.15.43:0/4141636195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e600a00 0x55c80e5fdcf0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fef5009c930 tx=0x7fef500b6260 comp rx=0 tx=0).stop 2024-04-21T08:49:35.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.287+0000 7fef6040b740 1 -- 172.21.15.43:0/4141636195 shutdown_connections 2024-04-21T08:49:35.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.287+0000 7fef6040b740 1 --2- 172.21.15.43:0/4141636195 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55c80e610ba0 0x55c80e612fd0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.287+0000 7fef6040b740 1 --2- 172.21.15.43:0/4141636195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e600a00 0x55c80e5fdcf0 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.287+0000 7fef3aff5640 0 client.20748 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:49:35.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.287+0000 7fef6040b740 1 --2- 172.21.15.43:0/4141636195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c80e5f7e80 0x55c80e5fd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.287+0000 7fef6040b740 1 --2- 172.21.15.43:0/4141636195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80e5f74d0 0x55c80e600350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.287+0000 7fef6040b740 1 -- 172.21.15.43:0/4141636195 >> 172.21.15.43:0/4141636195 conn(0x55c80e5f1ba0 msgr2=0x55c80e5f38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:35.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.287+0000 7fef6040b740 1 -- 172.21.15.43:0/4141636195 shutdown_connections 2024-04-21T08:49:35.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.287+0000 7fef6040b740 1 -- 172.21.15.43:0/4141636195 wait complete. 2024-04-21T08:49:35.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.288+0000 7fef6040b740 20 client.20748 trim_cache size 0 max 16384 2024-04-21T08:49:35.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.301+0000 7fef6040b740 1 Processor -- start 2024-04-21T08:49:35.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.301+0000 7fef6040b740 1 -- start start 2024-04-21T08:49:35.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.301+0000 7fef6040b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e5f74d0 0x55c80e60c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.301+0000 7fef6040b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80e5f7e80 0x55c80e5f3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.301+0000 7fef6040b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c80e600a00 0x55c80e5ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.301+0000 7fef6040b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fef581af580 con 0x55c80e5f74d0 2024-04-21T08:49:35.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.301+0000 7fef6040b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fef581b10c0 con 0x55c80e600a00 2024-04-21T08:49:35.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.301+0000 7fef6040b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fef581afc50 con 0x55c80e5f7e80 2024-04-21T08:49:35.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.301+0000 7fef56ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e5f74d0 0x55c80e60c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.301+0000 7fef56ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e5f74d0 0x55c80e60c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48164/0 (socket says 172.21.15.43:48164) 2024-04-21T08:49:35.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.301+0000 7fef56ffd640 1 -- 172.21.15.43:0/437027169 learned_addr learned my addr 172.21.15.43:0/437027169 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:35.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.301+0000 7fef577fe640 1 --2- 172.21.15.43:0/437027169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c80e600a00 0x55c80e5ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.301+0000 7fef567fc640 1 --2- 172.21.15.43:0/437027169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80e5f7e80 0x55c80e5f3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.301+0000 7fef56ffd640 1 -- 172.21.15.43:0/437027169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80e5f7e80 msgr2=0x55c80e5f3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.301+0000 7fef56ffd640 1 --2- 172.21.15.43:0/437027169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80e5f7e80 0x55c80e5f3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.301+0000 7fef56ffd640 1 -- 172.21.15.43:0/437027169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c80e600a00 msgr2=0x55c80e5ffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.301+0000 7fef56ffd640 1 --2- 172.21.15.43:0/437027169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c80e600a00 0x55c80e5ffe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.301+0000 7fef56ffd640 1 -- 172.21.15.43:0/437027169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c80e5f7c20 con 0x55c80e5f74d0 2024-04-21T08:49:35.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.301+0000 7fef567fc640 1 --2- 172.21.15.43:0/437027169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80e5f7e80 0x55c80e5f3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:35.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.301+0000 7fef577fe640 1 --2- 172.21.15.43:0/437027169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c80e600a00 0x55c80e5ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:35.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.302+0000 7fef56ffd640 1 --2- 172.21.15.43:0/437027169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e5f74d0 0x55c80e60c830 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fef4800cb80 tx=0x7fef4800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d532cb589ec07c42 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:35.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.302+0000 7fef3cff9640 1 -- 172.21.15.43:0/437027169 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48014030 con 0x55c80e5f74d0 2024-04-21T08:49:35.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.302+0000 7fef3cff9640 1 -- 172.21.15.43:0/437027169 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef480054d0 con 0x55c80e5f74d0 2024-04-21T08:49:35.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.302+0000 7fef3cff9640 1 -- 172.21.15.43:0/437027169 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef480262e0 con 0x55c80e5f74d0 2024-04-21T08:49:35.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.302+0000 7fef6040b740 1 -- 172.21.15.43:0/437027169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e5f74d0 msgr2=0x55c80e60c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.302+0000 7fef6040b740 1 --2- 172.21.15.43:0/437027169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e5f74d0 0x55c80e60c830 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fef4800cb80 tx=0x7fef4800e720 comp rx=0 tx=0).stop 2024-04-21T08:49:35.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.302+0000 7fef6040b740 1 -- 172.21.15.43:0/437027169 shutdown_connections 2024-04-21T08:49:35.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.302+0000 7fef6040b740 1 --2- 172.21.15.43:0/437027169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c80e600a00 0x55c80e5ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.302+0000 7fef6040b740 1 --2- 172.21.15.43:0/437027169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80e5f7e80 0x55c80e5f3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.302+0000 7fef6040b740 1 --2- 172.21.15.43:0/437027169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e5f74d0 0x55c80e60c830 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.302+0000 7fef6040b740 1 -- 172.21.15.43:0/437027169 >> 172.21.15.43:0/437027169 conn(0x55c80e5f1ba0 msgr2=0x55c80e5f2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:35.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.302+0000 7fef6040b740 1 -- 172.21.15.43:0/437027169 shutdown_connections 2024-04-21T08:49:35.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.302+0000 7fef6040b740 1 -- 172.21.15.43:0/437027169 wait complete. 2024-04-21T08:49:35.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.303+0000 7fef6040b740 1 -- start start 2024-04-21T08:49:35.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.303+0000 7fef6040b740 1 Processor -- start 2024-04-21T08:49:35.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.303+0000 7fef6040b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e5f74d0 0x55c80e609170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.303+0000 7fef6040b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c80e5f7e80 0x55c80e6044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.303+0000 7fef6040b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80e600a00 0x55c80e604a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.303+0000 7fef6040b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fef581b09f0 con 0x55c80e5f74d0 2024-04-21T08:49:35.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.303+0000 7fef6040b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fef581b0320 con 0x55c80e5f7e80 2024-04-21T08:49:35.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.303+0000 7fef6040b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fef581ad580 con 0x55c80e600a00 2024-04-21T08:49:35.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.303+0000 7fef6040b740 10 client.0 _ll_register_callbacks cb 0x7fef5009c3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:35.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.303+0000 7fef56ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e5f74d0 0x55c80e609170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.303+0000 7fef56ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e5f74d0 0x55c80e609170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48176/0 (socket says 172.21.15.43:48176) 2024-04-21T08:49:35.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.303+0000 7fef56ffd640 1 -- 172.21.15.43:0/2232911286 learned_addr learned my addr 172.21.15.43:0/2232911286 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:35.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.303+0000 7fef567fc640 1 --2- 172.21.15.43:0/2232911286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c80e5f7e80 0x55c80e6044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.303+0000 7fef56ffd640 1 -- 172.21.15.43:0/2232911286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80e600a00 msgr2=0x55c80e604a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.303+0000 7fef577fe640 1 --2- 172.21.15.43:0/2232911286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80e600a00 0x55c80e604a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.303+0000 7fef56ffd640 1 --2- 172.21.15.43:0/2232911286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80e600a00 0x55c80e604a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.303+0000 7fef56ffd640 1 -- 172.21.15.43:0/2232911286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c80e5f7e80 msgr2=0x55c80e6044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.303+0000 7fef56ffd640 1 --2- 172.21.15.43:0/2232911286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c80e5f7e80 0x55c80e6044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.304+0000 7fef56ffd640 1 -- 172.21.15.43:0/2232911286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c80e605270 con 0x55c80e5f74d0 2024-04-21T08:49:35.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.304+0000 7fef567fc640 1 --2- 172.21.15.43:0/2232911286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c80e5f7e80 0x55c80e6044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:35.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.304+0000 7fef56ffd640 1 --2- 172.21.15.43:0/2232911286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e5f74d0 0x55c80e609170 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fef4800cfa0 tx=0x7fef48005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbc075010191165 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:35.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.304+0000 7fef3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:35.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.304+0000 7fef3aff5640 1 -- 172.21.15.43:0/2232911286 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48014030 con 0x55c80e5f74d0 2024-04-21T08:49:35.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.304+0000 7fef3aff5640 1 -- 172.21.15.43:0/2232911286 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fef480052d0 con 0x55c80e5f74d0 2024-04-21T08:49:35.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.304+0000 7fef6040b740 1 -- 172.21.15.43:0/2232911286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef480054d0 con 0x55c80e5f74d0 2024-04-21T08:49:35.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.304+0000 7fef3aff5640 1 -- 172.21.15.43:0/2232911286 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fef4800c710 con 0x55c80e5f74d0 2024-04-21T08:49:35.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.304+0000 7fef6040b740 10 client.20754 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:35.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.304+0000 7fef6040b740 1 -- 172.21.15.43:0/2232911286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c80e677130 con 0x55c80e5f74d0 2024-04-21T08:49:35.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.304+0000 7fef3aff5640 1 -- 172.21.15.43:0/2232911286 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef480054d0 con 0x55c80e5f74d0 2024-04-21T08:49:35.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.304+0000 7fef3aff5640 1 -- 172.21.15.43:0/2232911286 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fef48036030 con 0x55c80e5f74d0 2024-04-21T08:49:35.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.304+0000 7fef6040b740 10 client.20754 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:35.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.304+0000 7fef6040b740 10 client.20754 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:49:35.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.304+0000 7fef6040b740 20 client.20754 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:35.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.304+0000 7fef6040b740 1 --2- 172.21.15.43:0/2232911286 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55c80e677640 0x55c80e679a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.304+0000 7fef6040b740 4 client.20754 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:35.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.304+0000 7fef6040b740 1 -- 172.21.15.43:0/2232911286 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c80e569c10 con 0x55c80e677640 2024-04-21T08:49:35.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.305+0000 7fef567fc640 1 --2- 172.21.15.43:0/2232911286 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55c80e677640 0x55c80e679a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.305+0000 7fef567fc640 1 --2- 172.21.15.43:0/2232911286 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55c80e677640 0x55c80e679a30 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a9f68da6cbc3e128 server_cookie=8c42e6a3c75eb12d in_seq=0 out_seq=0 2024-04-21T08:49:35.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.305+0000 7fef3aff5640 10 client.20754 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:49:35.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.305+0000 7fef3aff5640 1 -- 172.21.15.43:0/2232911286 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c80e569c10 con 0x55c80e677640 2024-04-21T08:49:35.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.305+0000 7fef3aff5640 10 client.20754 handle_command_reply: tid=0 2024-04-21T08:49:35.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.305+0000 7fef6040b740 1 client.20754 shutdown 2024-04-21T08:49:35.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.305+0000 7fef6040b740 1 -- 172.21.15.43:0/2232911286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e5f74d0 msgr2=0x55c80e609170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.305+0000 7fef6040b740 1 --2- 172.21.15.43:0/2232911286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e5f74d0 0x55c80e609170 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fef4800cfa0 tx=0x7fef48005c90 comp rx=0 tx=0).stop 2024-04-21T08:49:35.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.306+0000 7fef6040b740 1 -- 172.21.15.43:0/2232911286 shutdown_connections 2024-04-21T08:49:35.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.306+0000 7fef6040b740 1 --2- 172.21.15.43:0/2232911286 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55c80e677640 0x55c80e679a30 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.306+0000 7fef6040b740 1 --2- 172.21.15.43:0/2232911286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c80e600a00 0x55c80e604a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.306+0000 7fef6040b740 1 --2- 172.21.15.43:0/2232911286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c80e5f7e80 0x55c80e6044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.306+0000 7fef3aff5640 0 client.20754 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:49:35.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.306+0000 7fef6040b740 1 --2- 172.21.15.43:0/2232911286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c80e5f74d0 0x55c80e609170 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.306+0000 7fef6040b740 1 -- 172.21.15.43:0/2232911286 >> 172.21.15.43:0/2232911286 conn(0x55c80e5f1ba0 msgr2=0x55c80e5f37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:35.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.306+0000 7fef6040b740 1 -- 172.21.15.43:0/2232911286 shutdown_connections 2024-04-21T08:49:35.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.306+0000 7fef6040b740 1 -- 172.21.15.43:0/2232911286 wait complete. 2024-04-21T08:49:35.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.306+0000 7fef6040b740 20 client.20754 trim_cache size 0 max 16384 2024-04-21T08:49:35.312 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:49:35.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.306+0000 7fef5d746640 1 -- 172.21.15.43:0/2249635379 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fef400a3d20 msgr2=0x7fef400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.306+0000 7fef5d746640 1 --2- 172.21.15.43:0/2249635379 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fef400a3d20 0x7fef400a61e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fef4c005770 tx=0x7fef4c02c650 comp rx=0 tx=0).stop 2024-04-21T08:49:35.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.306+0000 7fef5d746640 1 -- 172.21.15.43:0/2249635379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef5819cfa0 msgr2=0x7fef5819d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.306+0000 7fef5d746640 1 --2- 172.21.15.43:0/2249635379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef5819cfa0 0x7fef5819d450 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fef5000eaa0 tx=0x7fef5000ef60 comp rx=0 tx=0).stop 2024-04-21T08:49:35.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.306+0000 7fef5d746640 1 -- 172.21.15.43:0/2249635379 shutdown_connections 2024-04-21T08:49:35.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.306+0000 7fef5d746640 1 --2- 172.21.15.43:0/2249635379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef5819cfa0 0x7fef5819d450 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.306+0000 7fef5d746640 1 --2- 172.21.15.43:0/2249635379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef58107260 0x7fef58198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.306+0000 7fef5d746640 1 --2- 172.21.15.43:0/2249635379 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fef400a3d20 0x7fef400a61e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.306+0000 7fef5d746640 1 --2- 172.21.15.43:0/2249635379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef58102190 0x7fef581986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.306+0000 7fef5d746640 1 -- 172.21.15.43:0/2249635379 >> 172.21.15.43:0/2249635379 conn(0x7fef580fd990 msgr2=0x7fef580fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:35.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.306+0000 7fef5d746640 1 -- 172.21.15.43:0/2249635379 shutdown_connections 2024-04-21T08:49:35.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.306+0000 7fef5d746640 1 -- 172.21.15.43:0/2249635379 wait complete. 2024-04-21T08:49:35.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:35 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:49:35.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:35 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:35.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:35 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:49:35.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:35 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:35.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:35 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:49:35.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:35 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3899165181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:49:35.317 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:49:35.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.421+0000 7efd0cfd5640 1 -- 172.21.15.43:0/2141295998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd0810c510 msgr2=0x7efd0810e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.421+0000 7efd0cfd5640 1 --2- 172.21.15.43:0/2141295998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd0810c510 0x7efd0810e9b0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7efd00009ca0 tx=0x7efd0002c230 comp rx=0 tx=0).stop 2024-04-21T08:49:35.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.422+0000 7efd0cfd5640 1 -- 172.21.15.43:0/2141295998 shutdown_connections 2024-04-21T08:49:35.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.422+0000 7efd0cfd5640 1 --2- 172.21.15.43:0/2141295998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd0810c510 0x7efd0810e9b0 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.422+0000 7efd0cfd5640 1 --2- 172.21.15.43:0/2141295998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd08103af0 0x7efd0810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.422+0000 7efd0cfd5640 1 --2- 172.21.15.43:0/2141295998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd081031d0 0x7efd081035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.422+0000 7efd0cfd5640 1 -- 172.21.15.43:0/2141295998 >> 172.21.15.43:0/2141295998 conn(0x7efd08077f50 msgr2=0x7efd08078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:35.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.422+0000 7efd0cfd5640 1 -- 172.21.15.43:0/2141295998 shutdown_connections 2024-04-21T08:49:35.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.422+0000 7efd0cfd5640 1 -- 172.21.15.43:0/2141295998 wait complete. 2024-04-21T08:49:35.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.422+0000 7efd0cfd5640 1 Processor -- start 2024-04-21T08:49:35.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.422+0000 7efd0cfd5640 1 -- start start 2024-04-21T08:49:35.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.423+0000 7efd0cfd5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd081031d0 0x7efd081986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.423+0000 7efd0cfd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd08103af0 0x7efd08198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.423+0000 7efd0cfd5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd0810c510 0x7efd0819cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.423+0000 7efd0cfd5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efd08112180 con 0x7efd0810c510 2024-04-21T08:49:35.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.423+0000 7efd0cfd5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd08112000 con 0x7efd08103af0 2024-04-21T08:49:35.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.423+0000 7efd0cfd5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efd08112300 con 0x7efd081031d0 2024-04-21T08:49:35.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.423+0000 7efd06d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd0810c510 0x7efd0819cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.423+0000 7efd06d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd0810c510 0x7efd0819cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48186/0 (socket says 172.21.15.43:48186) 2024-04-21T08:49:35.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.423+0000 7efd06d9d640 1 -- 172.21.15.43:0/465809094 learned_addr learned my addr 172.21.15.43:0/465809094 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:35.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.423+0000 7efd0659c640 1 --2- 172.21.15.43:0/465809094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd081031d0 0x7efd081986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.423+0000 7efd05d9b640 1 --2- 172.21.15.43:0/465809094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd08103af0 0x7efd08198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.423+0000 7efd06d9d640 1 -- 172.21.15.43:0/465809094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd081031d0 msgr2=0x7efd081986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.423+0000 7efd06d9d640 1 --2- 172.21.15.43:0/465809094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd081031d0 0x7efd081986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.423+0000 7efd06d9d640 1 -- 172.21.15.43:0/465809094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd08103af0 msgr2=0x7efd08198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.423+0000 7efd06d9d640 1 --2- 172.21.15.43:0/465809094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd08103af0 0x7efd08198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.423+0000 7efd06d9d640 1 -- 172.21.15.43:0/465809094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0819d6d0 con 0x7efd0810c510 2024-04-21T08:49:35.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.423+0000 7efd0659c640 1 --2- 172.21.15.43:0/465809094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd081031d0 0x7efd081986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:35.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.423+0000 7efd06d9d640 1 --2- 172.21.15.43:0/465809094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd0810c510 0x7efd0819cfa0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7efd0002c840 tx=0x7efd00003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:35.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.423+0000 7efcff7fe640 1 -- 172.21.15.43:0/465809094 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00039040 con 0x7efd0810c510 2024-04-21T08:49:35.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.423+0000 7efcff7fe640 1 -- 172.21.15.43:0/465809094 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd00044070 con 0x7efd0810c510 2024-04-21T08:49:35.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.423+0000 7efd0cfd5640 1 -- 172.21.15.43:0/465809094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd0819d960 con 0x7efd0810c510 2024-04-21T08:49:35.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.423+0000 7efcff7fe640 1 -- 172.21.15.43:0/465809094 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd000329c0 con 0x7efd0810c510 2024-04-21T08:49:35.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.423+0000 7efd0cfd5640 1 -- 172.21.15.43:0/465809094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd081a5250 con 0x7efd0810c510 2024-04-21T08:49:35.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.425+0000 7efcff7fe640 1 -- 172.21.15.43:0/465809094 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efd00032b60 con 0x7efd0810c510 2024-04-21T08:49:35.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.426+0000 7efd0fbc8740 1 Processor -- start 2024-04-21T08:49:35.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.426+0000 7efd0fbc8740 1 -- start start 2024-04-21T08:49:35.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.426+0000 7efd0fbc8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55861aa094d0 0x55861aa098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.426+0000 7efd0fbc8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa09e80 0x55861aa12430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.426+0000 7efd0fbc8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55861aa12a00 0x55861aa14e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.426+0000 7efd0fbc8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55861a96e1f0 con 0x55861aa09e80 2024-04-21T08:49:35.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.426+0000 7efcff7fe640 1 --2- 172.21.15.43:0/465809094 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7efcf40a3da0 0x7efcf40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.426+0000 7efd0fbc8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55861a90b4b0 con 0x55861aa12a00 2024-04-21T08:49:35.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.426+0000 7efd0fbc8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55861a96a1b0 con 0x55861aa094d0 2024-04-21T08:49:35.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.426+0000 7efd0659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa09e80 0x55861aa12430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.426+0000 7efd0659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa09e80 0x55861aa12430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48196/0 (socket says 172.21.15.43:48196) 2024-04-21T08:49:35.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.426+0000 7efd0659c640 1 -- 172.21.15.43:0/2664481084 learned_addr learned my addr 172.21.15.43:0/2664481084 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.426+0000 7efcff7fe640 1 -- 172.21.15.43:0/465809094 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7efd000f3070 con 0x7efd0810c510 2024-04-21T08:49:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.427+0000 7efd06d9d640 1 --2- 172.21.15.43:0/465809094 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7efcf40a3da0 0x7efcf40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.427+0000 7efd05d9b640 1 --2- 172.21.15.43:0/2664481084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55861aa094d0 0x55861aa098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.427+0000 7efd05d9b640 1 --2- 172.21.15.43:0/2664481084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55861aa12a00 0x55861aa14e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.427+0000 7efd0659c640 1 -- 172.21.15.43:0/2664481084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55861aa094d0 msgr2=0x55861aa098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.427+0000 7efd0659c640 1 --2- 172.21.15.43:0/2664481084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55861aa094d0 0x55861aa098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.427+0000 7efd0659c640 1 -- 172.21.15.43:0/2664481084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55861aa12a00 msgr2=0x55861aa14e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.427+0000 7efd0659c640 1 --2- 172.21.15.43:0/2664481084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55861aa12a00 0x55861aa14e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.427+0000 7efd0659c640 1 -- 172.21.15.43:0/2664481084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55861a8fd1b0 con 0x55861aa09e80 2024-04-21T08:49:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.427+0000 7efd05d9b640 1 --2- 172.21.15.43:0/2664481084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55861aa12a00 0x55861aa14e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.427+0000 7efd06d9d640 1 --2- 172.21.15.43:0/465809094 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7efcf40a3da0 0x7efcf40a6260 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efd000f0560 tx=0x7efd0002c230 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.427+0000 7efd0659c640 1 --2- 172.21.15.43:0/2664481084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa09e80 0x55861aa12430 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7efcf000cca0 tx=0x7efcf000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78920ee797da8956 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.427+0000 7efcfcff9640 1 -- 172.21.15.43:0/2664481084 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0005220 con 0x55861aa09e80 2024-04-21T08:49:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.427+0000 7efcfcff9640 1 -- 172.21.15.43:0/2664481084 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcf0004c50 con 0x55861aa09e80 2024-04-21T08:49:35.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.427+0000 7efcfcff9640 1 -- 172.21.15.43:0/2664481084 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf00107f0 con 0x55861aa09e80 2024-04-21T08:49:35.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.428+0000 7efd0fbc8740 1 -- 172.21.15.43:0/2664481084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa09e80 msgr2=0x55861aa12430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.428+0000 7efd0fbc8740 1 --2- 172.21.15.43:0/2664481084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa09e80 0x55861aa12430 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7efcf000cca0 tx=0x7efcf000e720 comp rx=0 tx=0).stop 2024-04-21T08:49:35.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.428+0000 7efd0fbc8740 1 -- 172.21.15.43:0/2664481084 shutdown_connections 2024-04-21T08:49:35.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.428+0000 7efd0fbc8740 1 --2- 172.21.15.43:0/2664481084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55861aa12a00 0x55861aa14e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.428+0000 7efd0fbc8740 1 --2- 172.21.15.43:0/2664481084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa09e80 0x55861aa12430 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.428+0000 7efd0fbc8740 1 --2- 172.21.15.43:0/2664481084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55861aa094d0 0x55861aa098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.428+0000 7efd0fbc8740 1 -- 172.21.15.43:0/2664481084 >> 172.21.15.43:0/2664481084 conn(0x55861aa03ba0 msgr2=0x55861aa03f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:35.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.428+0000 7efd0fbc8740 1 -- 172.21.15.43:0/2664481084 shutdown_connections 2024-04-21T08:49:35.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.428+0000 7efd0fbc8740 1 -- 172.21.15.43:0/2664481084 wait complete. 2024-04-21T08:49:35.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.428+0000 7efd0fbc8740 1 -- start start 2024-04-21T08:49:35.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.428+0000 7efd0fbc8740 1 Processor -- start 2024-04-21T08:49:35.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.429+0000 7efd0fbc8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55861aa094d0 0x55861aa12290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.429+0000 7efd0fbc8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55861aa09e80 0x55861aa0f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.429+0000 7efd0fbc8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa12a00 0x55861aa0fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.429+0000 7efd0fbc8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55861aa15970 con 0x55861aa12a00 2024-04-21T08:49:35.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.429+0000 7efd0fbc8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55861a927500 con 0x55861aa094d0 2024-04-21T08:49:35.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.429+0000 7efd0fbc8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55861aa15af0 con 0x55861aa09e80 2024-04-21T08:49:35.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.429+0000 7efd0fbc8740 10 client.0 _ll_register_callbacks cb 0x55861a8f5b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:35.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.429+0000 7efd05d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa12a00 0x55861aa0fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.429+0000 7efd05d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa12a00 0x55861aa0fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48202/0 (socket says 172.21.15.43:48202) 2024-04-21T08:49:35.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.429+0000 7efd05d9b640 1 -- 172.21.15.43:0/3584481225 learned_addr learned my addr 172.21.15.43:0/3584481225 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:35.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.429+0000 7efd0659c640 1 --2- 172.21.15.43:0/3584481225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55861aa09e80 0x55861aa0f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.429+0000 7efd05d9b640 1 --2- 172.21.15.43:0/3584481225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55861aa094d0 0x55861aa12290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.429+0000 7efd05d9b640 1 -- 172.21.15.43:0/3584481225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55861aa09e80 msgr2=0x55861aa0f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.429+0000 7efd05d9b640 1 --2- 172.21.15.43:0/3584481225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55861aa09e80 0x55861aa0f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.429+0000 7efd05d9b640 1 -- 172.21.15.43:0/3584481225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55861aa094d0 msgr2=0x55861aa12290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.429+0000 7efd05d9b640 1 --2- 172.21.15.43:0/3584481225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55861aa094d0 0x55861aa12290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.429+0000 7efd05d9b640 1 -- 172.21.15.43:0/3584481225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55861a6803f0 con 0x55861aa12a00 2024-04-21T08:49:35.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.429+0000 7efd05d9b640 1 --2- 172.21.15.43:0/3584481225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa12a00 0x55861aa0fb60 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7efcf800fe70 tx=0x7efcf800f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd88be5903099d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:35.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.429+0000 7efcedffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:35.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.429+0000 7efcedffb640 1 -- 172.21.15.43:0/3584481225 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf800ce70 con 0x55861aa12a00 2024-04-21T08:49:35.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.429+0000 7efcedffb640 1 -- 172.21.15.43:0/3584481225 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7efcf8005020 con 0x55861aa12a00 2024-04-21T08:49:35.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.429+0000 7efd0fbc8740 1 -- 172.21.15.43:0/3584481225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcf0004c50 con 0x55861aa12a00 2024-04-21T08:49:35.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.430+0000 7efcedffb640 1 -- 172.21.15.43:0/3584481225 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efcf8025050 con 0x55861aa12a00 2024-04-21T08:49:35.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.430+0000 7efd0fbc8740 10 client.20766 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:35.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.430+0000 7efd0fbc8740 1 -- 172.21.15.43:0/3584481225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55861a74a7d0 con 0x55861aa12a00 2024-04-21T08:49:35.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.430+0000 7efcedffb640 1 -- 172.21.15.43:0/3584481225 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcf0004c50 con 0x55861aa12a00 2024-04-21T08:49:35.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.430+0000 7efcedffb640 1 -- 172.21.15.43:0/3584481225 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7efcf8011440 con 0x55861aa12a00 2024-04-21T08:49:35.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.430+0000 7efd0fbc8740 10 client.20766 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:35.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.430+0000 7efd0fbc8740 10 client.20766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:35.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.430+0000 7efd0fbc8740 20 client.20766 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:35.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.430+0000 7efd0fbc8740 1 --2- 172.21.15.43:0/3584481225 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55861aa22c00 0x55861aa25030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.430+0000 7efd0fbc8740 4 client.20766 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:35.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.430+0000 7efd0fbc8740 1 -- 172.21.15.43:0/3584481225 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55861a90d930 con 0x55861aa22c00 2024-04-21T08:49:35.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.430+0000 7efd0659c640 1 --2- 172.21.15.43:0/3584481225 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55861aa22c00 0x55861aa25030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.431+0000 7efd0659c640 1 --2- 172.21.15.43:0/3584481225 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55861aa22c00 0x55861aa25030 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eb152fc1cbba0d0 server_cookie=bc4123f3403ccd2e in_seq=0 out_seq=0 2024-04-21T08:49:35.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.431+0000 7efcedffb640 10 client.20766 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:35.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.431+0000 7efcedffb640 1 -- 172.21.15.43:0/3584481225 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55861a90d930 con 0x55861aa22c00 2024-04-21T08:49:35.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.431+0000 7efcedffb640 10 client.20766 handle_command_reply: tid=0 2024-04-21T08:49:35.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.432+0000 7efd0fbc8740 1 client.20766 shutdown 2024-04-21T08:49:35.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.432+0000 7efd0fbc8740 1 -- 172.21.15.43:0/3584481225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa12a00 msgr2=0x55861aa0fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.432+0000 7efd0fbc8740 1 --2- 172.21.15.43:0/3584481225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa12a00 0x55861aa0fb60 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7efcf800fe70 tx=0x7efcf800f890 comp rx=0 tx=0).stop 2024-04-21T08:49:35.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.432+0000 7efd0fbc8740 1 -- 172.21.15.43:0/3584481225 shutdown_connections 2024-04-21T08:49:35.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.432+0000 7efd0fbc8740 1 --2- 172.21.15.43:0/3584481225 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55861aa22c00 0x55861aa25030 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.432+0000 7efd0fbc8740 1 --2- 172.21.15.43:0/3584481225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa12a00 0x55861aa0fb60 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.432+0000 7efcedffb640 0 client.20766 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:35.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.432+0000 7efd0fbc8740 1 --2- 172.21.15.43:0/3584481225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55861aa09e80 0x55861aa0f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.432+0000 7efd0fbc8740 1 --2- 172.21.15.43:0/3584481225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55861aa094d0 0x55861aa12290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.432+0000 7efd0fbc8740 1 -- 172.21.15.43:0/3584481225 >> 172.21.15.43:0/3584481225 conn(0x55861aa03ba0 msgr2=0x55861aa058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:35.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.432+0000 7efd0fbc8740 1 -- 172.21.15.43:0/3584481225 shutdown_connections 2024-04-21T08:49:35.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.432+0000 7efd0fbc8740 1 -- 172.21.15.43:0/3584481225 wait complete. 2024-04-21T08:49:35.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.432+0000 7efd0fbc8740 20 client.20766 trim_cache size 0 max 16384 2024-04-21T08:49:35.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.445+0000 7efd0fbc8740 1 Processor -- start 2024-04-21T08:49:35.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.445+0000 7efd0fbc8740 1 -- start start 2024-04-21T08:49:35.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.445+0000 7efd0fbc8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55861aa094d0 0x55861aa1e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.445+0000 7efd0fbc8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55861aa09e80 0x55861aa1b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.446+0000 7efd0fbc8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa12a00 0x55861aa0e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.446+0000 7efd0fbc8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efd081af530 con 0x55861aa12a00 2024-04-21T08:49:35.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.446+0000 7efd0fbc8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd081b1070 con 0x55861aa09e80 2024-04-21T08:49:35.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.446+0000 7efd0fbc8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efd081afc00 con 0x55861aa094d0 2024-04-21T08:49:35.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.446+0000 7efd05d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa12a00 0x55861aa0e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.446+0000 7efd05d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa12a00 0x55861aa0e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48210/0 (socket says 172.21.15.43:48210) 2024-04-21T08:49:35.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.446+0000 7efd05d9b640 1 -- 172.21.15.43:0/1302478583 learned_addr learned my addr 172.21.15.43:0/1302478583 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:35.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.446+0000 7efd0659c640 1 --2- 172.21.15.43:0/1302478583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55861aa09e80 0x55861aa1b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.446+0000 7efd05d9b640 1 --2- 172.21.15.43:0/1302478583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55861aa094d0 0x55861aa1e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.446+0000 7efd05d9b640 1 -- 172.21.15.43:0/1302478583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55861aa094d0 msgr2=0x55861aa1e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.446+0000 7efd05d9b640 1 --2- 172.21.15.43:0/1302478583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55861aa094d0 0x55861aa1e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.446+0000 7efd05d9b640 1 -- 172.21.15.43:0/1302478583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55861aa09e80 msgr2=0x55861aa1b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.446+0000 7efd05d9b640 1 --2- 172.21.15.43:0/1302478583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55861aa09e80 0x55861aa1b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.446+0000 7efd05d9b640 1 -- 172.21.15.43:0/1302478583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55861aa217a0 con 0x55861aa12a00 2024-04-21T08:49:35.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.446+0000 7efd05d9b640 1 --2- 172.21.15.43:0/1302478583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa12a00 0x55861aa0e780 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7efcf800cc30 tx=0x7efcf800f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c893264fc7271fb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:35.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.446+0000 7efceffff640 1 -- 172.21.15.43:0/1302478583 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf8011440 con 0x55861aa12a00 2024-04-21T08:49:35.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.446+0000 7efceffff640 1 -- 172.21.15.43:0/1302478583 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcf80115e0 con 0x55861aa12a00 2024-04-21T08:49:35.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.446+0000 7efceffff640 1 -- 172.21.15.43:0/1302478583 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf8005290 con 0x55861aa12a00 2024-04-21T08:49:35.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.447+0000 7efd0fbc8740 1 -- 172.21.15.43:0/1302478583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa12a00 msgr2=0x55861aa0e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.447+0000 7efd0fbc8740 1 --2- 172.21.15.43:0/1302478583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa12a00 0x55861aa0e780 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7efcf800cc30 tx=0x7efcf800f440 comp rx=0 tx=0).stop 2024-04-21T08:49:35.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.447+0000 7efd0fbc8740 1 -- 172.21.15.43:0/1302478583 shutdown_connections 2024-04-21T08:49:35.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.447+0000 7efd0fbc8740 1 --2- 172.21.15.43:0/1302478583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa12a00 0x55861aa0e780 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.447+0000 7efd0fbc8740 1 --2- 172.21.15.43:0/1302478583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55861aa09e80 0x55861aa1b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.447+0000 7efd0fbc8740 1 --2- 172.21.15.43:0/1302478583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55861aa094d0 0x55861aa1e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.447+0000 7efd0fbc8740 1 -- 172.21.15.43:0/1302478583 >> 172.21.15.43:0/1302478583 conn(0x55861aa03ba0 msgr2=0x55861aa02370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:35.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.447+0000 7efd0fbc8740 1 -- 172.21.15.43:0/1302478583 shutdown_connections 2024-04-21T08:49:35.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.447+0000 7efd0fbc8740 1 -- 172.21.15.43:0/1302478583 wait complete. 2024-04-21T08:49:35.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.447+0000 7efd0fbc8740 1 -- start start 2024-04-21T08:49:35.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.447+0000 7efd0fbc8740 1 Processor -- start 2024-04-21T08:49:35.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.447+0000 7efd0fbc8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa094d0 0x55861aa84550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.447+0000 7efd0fbc8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55861aa09e80 0x55861aa84a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.447+0000 7efd0fbc8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55861aa12a00 0x55861aa110c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.447+0000 7efd0fbc8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efd081b09a0 con 0x55861aa094d0 2024-04-21T08:49:35.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.447+0000 7efd0fbc8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd081b02d0 con 0x55861aa09e80 2024-04-21T08:49:35.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.447+0000 7efd0fbc8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efd081ad530 con 0x55861aa12a00 2024-04-21T08:49:35.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.448+0000 7efd0fbc8740 10 client.0 _ll_register_callbacks cb 0x7efcf8012a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:35.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.448+0000 7efd05d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa094d0 0x55861aa84550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.448+0000 7efd05d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa094d0 0x55861aa84550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48226/0 (socket says 172.21.15.43:48226) 2024-04-21T08:49:35.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.448+0000 7efd05d9b640 1 -- 172.21.15.43:0/55330984 learned_addr learned my addr 172.21.15.43:0/55330984 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:35.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.448+0000 7efd0659c640 1 --2- 172.21.15.43:0/55330984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55861aa09e80 0x55861aa84a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.448+0000 7efd05d9b640 1 --2- 172.21.15.43:0/55330984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55861aa12a00 0x55861aa110c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.448+0000 7efd05d9b640 1 -- 172.21.15.43:0/55330984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55861aa12a00 msgr2=0x55861aa110c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.448+0000 7efd05d9b640 1 --2- 172.21.15.43:0/55330984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55861aa12a00 0x55861aa110c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.448+0000 7efd05d9b640 1 -- 172.21.15.43:0/55330984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55861aa09e80 msgr2=0x55861aa84a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:35.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.448+0000 7efd05d9b640 1 --2- 172.21.15.43:0/55330984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55861aa09e80 0x55861aa84a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.448+0000 7efd05d9b640 1 -- 172.21.15.43:0/55330984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55861aa118f0 con 0x55861aa094d0 2024-04-21T08:49:35.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.448+0000 7efd05d9b640 1 --2- 172.21.15.43:0/55330984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa094d0 0x55861aa84550 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7efcf8005700 tx=0x7efcf8005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31944fe939ee18e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:35.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.448+0000 7efcedffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:35.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.448+0000 7efcedffb640 1 -- 172.21.15.43:0/55330984 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf803d060 con 0x55861aa094d0 2024-04-21T08:49:35.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.448+0000 7efcedffb640 1 -- 172.21.15.43:0/55330984 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7efcf8004390 con 0x55861aa094d0 2024-04-21T08:49:35.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.448+0000 7efd0fbc8740 1 -- 172.21.15.43:0/55330984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcf80115e0 con 0x55861aa094d0 2024-04-21T08:49:35.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.448+0000 7efcedffb640 1 -- 172.21.15.43:0/55330984 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efcf8044050 con 0x55861aa094d0 2024-04-21T08:49:35.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.448+0000 7efd0fbc8740 10 client.20772 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:35.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.448+0000 7efd0fbc8740 1 -- 172.21.15.43:0/55330984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55861aa8ce00 con 0x55861aa094d0 2024-04-21T08:49:35.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.449+0000 7efcedffb640 1 -- 172.21.15.43:0/55330984 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcf80115e0 con 0x55861aa094d0 2024-04-21T08:49:35.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.449+0000 7efcedffb640 1 -- 172.21.15.43:0/55330984 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7efcf8035070 con 0x55861aa094d0 2024-04-21T08:49:35.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.449+0000 7efd0fbc8740 10 client.20772 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:35.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.449+0000 7efd0fbc8740 10 client.20772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:35.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.449+0000 7efd0fbc8740 20 client.20772 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:35.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.449+0000 7efd0fbc8740 1 --2- 172.21.15.43:0/55330984 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55861aa8d4a0 0x55861aa8f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:35.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.449+0000 7efd0fbc8740 4 client.20772 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:35.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.449+0000 7efd0fbc8740 1 -- 172.21.15.43:0/55330984 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55861a97bc10 con 0x55861aa8d4a0 2024-04-21T08:49:35.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.449+0000 7efd0659c640 1 --2- 172.21.15.43:0/55330984 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55861aa8d4a0 0x55861aa8f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:35.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.449+0000 7efcedffb640 10 client.20772 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:35.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.449+0000 7efd0659c640 1 --2- 172.21.15.43:0/55330984 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55861aa8d4a0 0x55861aa8f8c0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b725bfbd422e56d server_cookie=ab8d4518e1348163 in_seq=0 out_seq=0 2024-04-21T08:49:35.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efcedffb640 1 -- 172.21.15.43:0/55330984 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55861a97bc10 con 0x55861aa8d4a0 2024-04-21T08:49:35.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efcedffb640 10 client.20772 handle_command_reply: tid=0 2024-04-21T08:49:35.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efd0fbc8740 1 client.20772 shutdown 2024-04-21T08:49:35.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efd0fbc8740 1 -- 172.21.15.43:0/55330984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa094d0 msgr2=0x55861aa84550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efd0fbc8740 1 --2- 172.21.15.43:0/55330984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa094d0 0x55861aa84550 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7efcf8005700 tx=0x7efcf8005970 comp rx=0 tx=0).stop 2024-04-21T08:49:35.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efd0fbc8740 1 -- 172.21.15.43:0/55330984 shutdown_connections 2024-04-21T08:49:35.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efd0fbc8740 1 --2- 172.21.15.43:0/55330984 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55861aa8d4a0 0x55861aa8f8c0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efd0fbc8740 1 --2- 172.21.15.43:0/55330984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55861aa12a00 0x55861aa110c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efcedffb640 0 client.20772 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:35.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efd0fbc8740 1 --2- 172.21.15.43:0/55330984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55861aa09e80 0x55861aa84a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efd0fbc8740 1 --2- 172.21.15.43:0/55330984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55861aa094d0 0x55861aa84550 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efd0fbc8740 1 -- 172.21.15.43:0/55330984 >> 172.21.15.43:0/55330984 conn(0x55861aa03ba0 msgr2=0x55861aa0a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:35.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efd0fbc8740 1 -- 172.21.15.43:0/55330984 shutdown_connections 2024-04-21T08:49:35.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efd0fbc8740 1 -- 172.21.15.43:0/55330984 wait complete. 2024-04-21T08:49:35.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efd0fbc8740 20 client.20772 trim_cache size 0 max 16384 2024-04-21T08:49:35.458 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:49:35.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efd0cfd5640 1 -- 172.21.15.43:0/465809094 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7efcf40a3da0 msgr2=0x7efcf40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efd0cfd5640 1 --2- 172.21.15.43:0/465809094 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7efcf40a3da0 0x7efcf40a6260 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efd000f0560 tx=0x7efd0002c230 comp rx=0 tx=0).stop 2024-04-21T08:49:35.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efd0cfd5640 1 -- 172.21.15.43:0/465809094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd0810c510 msgr2=0x7efd0819cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:35.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efd0cfd5640 1 --2- 172.21.15.43:0/465809094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd0810c510 0x7efd0819cfa0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7efd0002c840 tx=0x7efd00003c20 comp rx=0 tx=0).stop 2024-04-21T08:49:35.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efd0cfd5640 1 -- 172.21.15.43:0/465809094 shutdown_connections 2024-04-21T08:49:35.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efd0cfd5640 1 --2- 172.21.15.43:0/465809094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd0810c510 0x7efd0819cfa0 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efd0cfd5640 1 --2- 172.21.15.43:0/465809094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd08103af0 0x7efd08198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efd0cfd5640 1 --2- 172.21.15.43:0/465809094 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7efcf40a3da0 0x7efcf40a6260 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efd0cfd5640 1 --2- 172.21.15.43:0/465809094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd081031d0 0x7efd081986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:35.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.450+0000 7efd0cfd5640 1 -- 172.21.15.43:0/465809094 >> 172.21.15.43:0/465809094 conn(0x7efd08077f50 msgr2=0x7efd080fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:35.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.451+0000 7efd0cfd5640 1 -- 172.21.15.43:0/465809094 shutdown_connections 2024-04-21T08:49:35.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:35.451+0000 7efd0cfd5640 1 -- 172.21.15.43:0/465809094 wait complete. 2024-04-21T08:49:35.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:35 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:49:35.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:35 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:35.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:35 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:49:35.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:35 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:35.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:35 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:49:35.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:35 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3899165181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:49:35.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:35 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:49:35.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:35 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:35.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:35 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:49:35.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:35 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:35.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:35 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:49:35.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:35 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3899165181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:49:36.461 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:49:36.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:49:36.462 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:49:36.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:36 smithi175 ceph-mon[31920]: pgmap v332: 129 pgs: 129 active+clean; 2.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:49:36.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:36 smithi043 ceph-mon[27978]: pgmap v332: 129 pgs: 129 active+clean; 2.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:49:36.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:36 smithi112 ceph-mon[31085]: pgmap v332: 129 pgs: 129 active+clean; 2.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:49:36.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.586+0000 7fcb342e0640 1 -- 172.21.15.43:0/2993530233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb2c101a40 msgr2=0x7fcb2c105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:36.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.586+0000 7fcb342e0640 1 --2- 172.21.15.43:0/2993530233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb2c101a40 0x7fcb2c105a80 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009900 tx=0x7fcb2802bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:36.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.586+0000 7fcb342e0640 1 -- 172.21.15.43:0/2993530233 shutdown_connections 2024-04-21T08:49:36.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.586+0000 7fcb342e0640 1 --2- 172.21.15.43:0/2993530233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb2c106160 0x7fcb2c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.586+0000 7fcb342e0640 1 --2- 172.21.15.43:0/2993530233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb2c101a40 0x7fcb2c105a80 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.586+0000 7fcb342e0640 1 --2- 172.21.15.43:0/2993530233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb2c101090 0x7fcb2c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.586+0000 7fcb342e0640 1 -- 172.21.15.43:0/2993530233 >> 172.21.15.43:0/2993530233 conn(0x7fcb2c077f50 msgr2=0x7fcb2c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.586+0000 7fcb342e0640 1 -- 172.21.15.43:0/2993530233 shutdown_connections 2024-04-21T08:49:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.586+0000 7fcb342e0640 1 -- 172.21.15.43:0/2993530233 wait complete. 2024-04-21T08:49:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.587+0000 7fcb342e0640 1 Processor -- start 2024-04-21T08:49:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.587+0000 7fcb342e0640 1 -- start start 2024-04-21T08:49:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.587+0000 7fcb342e0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb2c101090 0x7fcb2c1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.587+0000 7fcb342e0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb2c106160 0x7fcb2c1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.587+0000 7fcb342e0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb2c19a720 0x7fcb2c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.587+0000 7fcb342e0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c1110d0 con 0x7fcb2c101090 2024-04-21T08:49:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.587+0000 7fcb342e0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c110f50 con 0x7fcb2c19a720 2024-04-21T08:49:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.587+0000 7fcb342e0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c111250 con 0x7fcb2c106160 2024-04-21T08:49:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.587+0000 7fcb3207c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb2c101090 0x7fcb2c1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.587+0000 7fcb3207c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb2c101090 0x7fcb2c1a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48242/0 (socket says 172.21.15.43:48242) 2024-04-21T08:49:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.587+0000 7fcb3207c640 1 -- 172.21.15.43:0/3496239483 learned_addr learned my addr 172.21.15.43:0/3496239483 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.587+0000 7fcb3287d640 1 --2- 172.21.15.43:0/3496239483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb2c19a720 0x7fcb2c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.587+0000 7fcb3187b640 1 --2- 172.21.15.43:0/3496239483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb2c106160 0x7fcb2c1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.587+0000 7fcb3207c640 1 -- 172.21.15.43:0/3496239483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb2c106160 msgr2=0x7fcb2c1a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.587+0000 7fcb3207c640 1 --2- 172.21.15.43:0/3496239483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb2c106160 0x7fcb2c1a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.587+0000 7fcb3207c640 1 -- 172.21.15.43:0/3496239483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb2c19a720 msgr2=0x7fcb2c19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.587+0000 7fcb3207c640 1 --2- 172.21.15.43:0/3496239483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb2c19a720 0x7fcb2c19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.587+0000 7fcb3207c640 1 -- 172.21.15.43:0/3496239483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2c19b320 con 0x7fcb2c101090 2024-04-21T08:49:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.588+0000 7fcb3207c640 1 --2- 172.21.15.43:0/3496239483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb2c101090 0x7fcb2c1a0660 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000c920 tx=0x7fcb2000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.588+0000 7fcb1f7fe640 1 -- 172.21.15.43:0/3496239483 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb20007c70 con 0x7fcb2c101090 2024-04-21T08:49:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.588+0000 7fcb342e0640 1 -- 172.21.15.43:0/3496239483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2c1a7400 con 0x7fcb2c101090 2024-04-21T08:49:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.588+0000 7fcb1f7fe640 1 -- 172.21.15.43:0/3496239483 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb20007e10 con 0x7fcb2c101090 2024-04-21T08:49:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.588+0000 7fcb1f7fe640 1 -- 172.21.15.43:0/3496239483 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb20018820 con 0x7fcb2c101090 2024-04-21T08:49:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.588+0000 7fcb342e0640 1 -- 172.21.15.43:0/3496239483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb2c1a7860 con 0x7fcb2c101090 2024-04-21T08:49:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.590+0000 7fcb1f7fe640 1 -- 172.21.15.43:0/3496239483 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcb2000f450 con 0x7fcb2c101090 2024-04-21T08:49:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.591+0000 7fcb36faf740 1 Processor -- start 2024-04-21T08:49:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.591+0000 7fcb36faf740 1 -- start start 2024-04-21T08:49:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.591+0000 7fcb36faf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a30a438e0 0x561a30a43cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.591+0000 7fcb36faf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a30a44290 0x561a30a4c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.591+0000 7fcb36faf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a4ce10 0x561a30a4f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.591+0000 7fcb36faf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561a309a7f00 con 0x561a30a4ce10 2024-04-21T08:49:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.591+0000 7fcb36faf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561a30945590 con 0x561a30a438e0 2024-04-21T08:49:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.591+0000 7fcb36faf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561a309a3ec0 con 0x561a30a44290 2024-04-21T08:49:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.591+0000 7fcb3187b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a4ce10 0x561a30a4f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.591+0000 7fcb3187b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a4ce10 0x561a30a4f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48250/0 (socket says 172.21.15.43:48250) 2024-04-21T08:49:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.591+0000 7fcb3187b640 1 -- 172.21.15.43:0/3919973674 learned_addr learned my addr 172.21.15.43:0/3919973674 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.592+0000 7fcb3287d640 1 --2- 172.21.15.43:0/3919973674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a30a438e0 0x561a30a43cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.592+0000 7fcb3207c640 1 --2- 172.21.15.43:0/3919973674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a30a44290 0x561a30a4c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.592+0000 7fcb1f7fe640 1 --2- 172.21.15.43:0/3496239483 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcb180a3d20 0x7fcb180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.592+0000 7fcb3187b640 1 -- 172.21.15.43:0/3919973674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a30a44290 msgr2=0x561a30a4c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.592+0000 7fcb3187b640 1 --2- 172.21.15.43:0/3919973674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a30a44290 0x561a30a4c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.592+0000 7fcb3187b640 1 -- 172.21.15.43:0/3919973674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a30a438e0 msgr2=0x561a30a43cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.592+0000 7fcb3187b640 1 --2- 172.21.15.43:0/3919973674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a30a438e0 0x561a30a43cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.592+0000 7fcb3187b640 1 -- 172.21.15.43:0/3919973674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a307827e0 con 0x561a30a4ce10 2024-04-21T08:49:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.592+0000 7fcb3287d640 1 --2- 172.21.15.43:0/3496239483 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcb180a3d20 0x7fcb180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.592+0000 7fcb1f7fe640 1 -- 172.21.15.43:0/3496239483 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fcb200d0d40 con 0x7fcb2c101090 2024-04-21T08:49:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.592+0000 7fcb3187b640 1 --2- 172.21.15.43:0/3919973674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a4ce10 0x561a30a4f260 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7fcb28000c00 tx=0x7fcb2802c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4791a978b9d4f657 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.592+0000 7fcb1cbfe640 1 -- 172.21.15.43:0/3919973674 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb28043020 con 0x561a30a4ce10 2024-04-21T08:49:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.592+0000 7fcb1cbfe640 1 -- 172.21.15.43:0/3919973674 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb2802c7e0 con 0x561a30a4ce10 2024-04-21T08:49:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.592+0000 7fcb1cbfe640 1 -- 172.21.15.43:0/3919973674 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb280055a0 con 0x561a30a4ce10 2024-04-21T08:49:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.593+0000 7fcb36faf740 1 -- 172.21.15.43:0/3919973674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a4ce10 msgr2=0x561a30a4f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.593+0000 7fcb3287d640 1 --2- 172.21.15.43:0/3496239483 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcb180a3d20 0x7fcb180a61e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcb24005fd0 tx=0x7fcb24004a40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.593+0000 7fcb36faf740 1 --2- 172.21.15.43:0/3919973674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a4ce10 0x561a30a4f260 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7fcb28000c00 tx=0x7fcb2802c5d0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.593+0000 7fcb36faf740 1 -- 172.21.15.43:0/3919973674 shutdown_connections 2024-04-21T08:49:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.593+0000 7fcb36faf740 1 --2- 172.21.15.43:0/3919973674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a4ce10 0x561a30a4f260 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.593+0000 7fcb36faf740 1 --2- 172.21.15.43:0/3919973674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a30a44290 0x561a30a4c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.593+0000 7fcb36faf740 1 --2- 172.21.15.43:0/3919973674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a30a438e0 0x561a30a43cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.593+0000 7fcb36faf740 1 -- 172.21.15.43:0/3919973674 >> 172.21.15.43:0/3919973674 conn(0x561a30a3dfb0 msgr2=0x561a30a3e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:36.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.593+0000 7fcb36faf740 1 -- 172.21.15.43:0/3919973674 shutdown_connections 2024-04-21T08:49:36.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.593+0000 7fcb36faf740 1 -- 172.21.15.43:0/3919973674 wait complete. 2024-04-21T08:49:36.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.594+0000 7fcb36faf740 1 -- start start 2024-04-21T08:49:36.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.594+0000 7fcb36faf740 1 Processor -- start 2024-04-21T08:49:36.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.594+0000 7fcb36faf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a30a438e0 0x561a30a49da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.594+0000 7fcb36faf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a44290 0x561a30a4a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.594+0000 7fcb36faf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a30a4ce10 0x561a30a4a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.594+0000 7fcb36faf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561a30a4fd20 con 0x561a30a44290 2024-04-21T08:49:36.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.594+0000 7fcb36faf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561a30961580 con 0x561a30a4ce10 2024-04-21T08:49:36.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.594+0000 7fcb36faf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561a30a4fea0 con 0x561a30a438e0 2024-04-21T08:49:36.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.594+0000 7fcb36faf740 10 client.0 _ll_register_callbacks cb 0x561a30967650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:36.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.594+0000 7fcb3187b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a44290 0x561a30a4a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.594+0000 7fcb3187b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a44290 0x561a30a4a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48252/0 (socket says 172.21.15.43:48252) 2024-04-21T08:49:36.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.594+0000 7fcb3187b640 1 -- 172.21.15.43:0/4019132216 learned_addr learned my addr 172.21.15.43:0/4019132216 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:36.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.594+0000 7fcb3207c640 1 --2- 172.21.15.43:0/4019132216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a30a438e0 0x561a30a49da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.594+0000 7fcb3287d640 1 --2- 172.21.15.43:0/4019132216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a30a4ce10 0x561a30a4a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.594+0000 7fcb3187b640 1 -- 172.21.15.43:0/4019132216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a30a438e0 msgr2=0x561a30a49da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:36.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.594+0000 7fcb3187b640 1 --2- 172.21.15.43:0/4019132216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a30a438e0 0x561a30a49da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.594+0000 7fcb3187b640 1 -- 172.21.15.43:0/4019132216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a30a4ce10 msgr2=0x561a30a4a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:36.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.594+0000 7fcb3187b640 1 --2- 172.21.15.43:0/4019132216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a30a4ce10 0x561a30a4a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.594+0000 7fcb3187b640 1 -- 172.21.15.43:0/4019132216 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a307e2f70 con 0x561a30a44290 2024-04-21T08:49:36.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.595+0000 7fcb3187b640 1 --2- 172.21.15.43:0/4019132216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a44290 0x561a30a4a2e0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7fcb280056a0 tx=0x7fcb280056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a5e41491acce6b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:36.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.595+0000 7fcb15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:36.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.595+0000 7fcb15ffb640 1 -- 172.21.15.43:0/4019132216 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2804c070 con 0x561a30a44290 2024-04-21T08:49:36.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.595+0000 7fcb15ffb640 1 -- 172.21.15.43:0/4019132216 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fcb280035f0 con 0x561a30a44290 2024-04-21T08:49:36.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.595+0000 7fcb36faf740 1 -- 172.21.15.43:0/4019132216 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb2802c7e0 con 0x561a30a44290 2024-04-21T08:49:36.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.595+0000 7fcb15ffb640 1 -- 172.21.15.43:0/4019132216 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcb28058050 con 0x561a30a44290 2024-04-21T08:49:36.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.595+0000 7fcb36faf740 10 client.20784 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:36.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.595+0000 7fcb36faf740 1 -- 172.21.15.43:0/4019132216 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561a3067a170 con 0x561a30a44290 2024-04-21T08:49:36.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.595+0000 7fcb15ffb640 1 -- 172.21.15.43:0/4019132216 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb2802c7e0 con 0x561a30a44290 2024-04-21T08:49:36.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.595+0000 7fcb15ffb640 1 -- 172.21.15.43:0/4019132216 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcb28048810 con 0x561a30a44290 2024-04-21T08:49:36.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.596+0000 7fcb36faf740 10 client.20784 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:36.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.596+0000 7fcb36faf740 10 client.20784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:36.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.596+0000 7fcb36faf740 20 client.20784 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:36.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.596+0000 7fcb36faf740 1 --2- 172.21.15.43:0/4019132216 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561a30a5d120 0x561a30a5f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.596+0000 7fcb36faf740 4 client.20784 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:36.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.596+0000 7fcb36faf740 1 -- 172.21.15.43:0/4019132216 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a309b5920 con 0x561a30a5d120 2024-04-21T08:49:36.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.596+0000 7fcb3207c640 1 --2- 172.21.15.43:0/4019132216 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561a30a5d120 0x561a30a5f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.596+0000 7fcb15ffb640 10 client.20784 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.596+0000 7fcb3207c640 1 --2- 172.21.15.43:0/4019132216 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561a30a5d120 0x561a30a5f510 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c445f78989e84410 server_cookie=de8a31060b0bf424 in_seq=0 out_seq=0 2024-04-21T08:49:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.597+0000 7fcb15ffb640 1 -- 172.21.15.43:0/4019132216 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a309b5920 con 0x561a30a5d120 2024-04-21T08:49:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.597+0000 7fcb15ffb640 10 client.20784 handle_command_reply: tid=0 2024-04-21T08:49:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.597+0000 7fcb36faf740 1 client.20784 shutdown 2024-04-21T08:49:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.597+0000 7fcb36faf740 1 -- 172.21.15.43:0/4019132216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a44290 msgr2=0x561a30a4a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.597+0000 7fcb36faf740 1 --2- 172.21.15.43:0/4019132216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a44290 0x561a30a4a2e0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7fcb280056a0 tx=0x7fcb280056d0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.597+0000 7fcb36faf740 1 -- 172.21.15.43:0/4019132216 shutdown_connections 2024-04-21T08:49:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.597+0000 7fcb36faf740 1 --2- 172.21.15.43:0/4019132216 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561a30a5d120 0x561a30a5f510 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.597+0000 7fcb36faf740 1 --2- 172.21.15.43:0/4019132216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a30a4ce10 0x561a30a4a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.597+0000 7fcb36faf740 1 --2- 172.21.15.43:0/4019132216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a44290 0x561a30a4a2e0 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.597+0000 7fcb15ffb640 0 client.20784 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.597+0000 7fcb36faf740 1 --2- 172.21.15.43:0/4019132216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a30a438e0 0x561a30a49da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.597+0000 7fcb36faf740 1 -- 172.21.15.43:0/4019132216 >> 172.21.15.43:0/4019132216 conn(0x561a30a3dfb0 msgr2=0x561a30a3fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.597+0000 7fcb36faf740 1 -- 172.21.15.43:0/4019132216 shutdown_connections 2024-04-21T08:49:36.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.597+0000 7fcb36faf740 1 -- 172.21.15.43:0/4019132216 wait complete. 2024-04-21T08:49:36.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.597+0000 7fcb36faf740 20 client.20784 trim_cache size 0 max 16384 2024-04-21T08:49:36.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.616+0000 7fcb36faf740 1 Processor -- start 2024-04-21T08:49:36.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.616+0000 7fcb36faf740 1 -- start start 2024-04-21T08:49:36.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.616+0000 7fcb36faf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a30a3dfb0 0x561a30a4c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.616+0000 7fcb36faf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a30a438e0 0x561a30a48ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.616+0000 7fcb36faf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a44290 0x561a30a62530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.616+0000 7fcb36faf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c1b16c0 con 0x561a30a44290 2024-04-21T08:49:36.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.616+0000 7fcb36faf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c1b3200 con 0x561a30a3dfb0 2024-04-21T08:49:36.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.616+0000 7fcb36faf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c1b1d90 con 0x561a30a438e0 2024-04-21T08:49:36.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.616+0000 7fcb3287d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a44290 0x561a30a62530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.616+0000 7fcb3287d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a44290 0x561a30a62530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48258/0 (socket says 172.21.15.43:48258) 2024-04-21T08:49:36.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.616+0000 7fcb3287d640 1 -- 172.21.15.43:0/985833845 learned_addr learned my addr 172.21.15.43:0/985833845 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:36.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.616+0000 7fcb3207c640 1 --2- 172.21.15.43:0/985833845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a30a3dfb0 0x561a30a4c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.616+0000 7fcb3187b640 1 --2- 172.21.15.43:0/985833845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a30a438e0 0x561a30a48ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.616+0000 7fcb3287d640 1 -- 172.21.15.43:0/985833845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a30a438e0 msgr2=0x561a30a48ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:36.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.616+0000 7fcb3287d640 1 --2- 172.21.15.43:0/985833845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a30a438e0 0x561a30a48ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.616+0000 7fcb3287d640 1 -- 172.21.15.43:0/985833845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a30a3dfb0 msgr2=0x561a30a4c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:36.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.616+0000 7fcb3287d640 1 --2- 172.21.15.43:0/985833845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a30a3dfb0 0x561a30a4c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.616+0000 7fcb3287d640 1 -- 172.21.15.43:0/985833845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a30a40180 con 0x561a30a44290 2024-04-21T08:49:36.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.616+0000 7fcb3207c640 1 --2- 172.21.15.43:0/985833845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a30a3dfb0 0x561a30a4c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:36.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.617+0000 7fcb3287d640 1 --2- 172.21.15.43:0/985833845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a44290 0x561a30a62530 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fcb2401f7d0 tx=0x7fcb24015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db83fa29fd49752 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:36.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.617+0000 7fcb17fff640 1 -- 172.21.15.43:0/985833845 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24013880 con 0x561a30a44290 2024-04-21T08:49:36.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.617+0000 7fcb17fff640 1 -- 172.21.15.43:0/985833845 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb24017070 con 0x561a30a44290 2024-04-21T08:49:36.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.617+0000 7fcb17fff640 1 -- 172.21.15.43:0/985833845 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2401e450 con 0x561a30a44290 2024-04-21T08:49:36.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.617+0000 7fcb36faf740 1 -- 172.21.15.43:0/985833845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a44290 msgr2=0x561a30a62530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:36.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.617+0000 7fcb36faf740 1 --2- 172.21.15.43:0/985833845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a44290 0x561a30a62530 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fcb2401f7d0 tx=0x7fcb24015000 comp rx=0 tx=0).stop 2024-04-21T08:49:36.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.618+0000 7fcb36faf740 1 -- 172.21.15.43:0/985833845 shutdown_connections 2024-04-21T08:49:36.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.618+0000 7fcb36faf740 1 --2- 172.21.15.43:0/985833845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a44290 0x561a30a62530 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.618+0000 7fcb36faf740 1 --2- 172.21.15.43:0/985833845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a30a438e0 0x561a30a48ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.618+0000 7fcb36faf740 1 --2- 172.21.15.43:0/985833845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a30a3dfb0 0x561a30a4c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.618+0000 7fcb36faf740 1 -- 172.21.15.43:0/985833845 >> 172.21.15.43:0/985833845 conn(0x561a30a449e0 msgr2=0x561a30a58980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:36.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.618+0000 7fcb36faf740 1 -- 172.21.15.43:0/985833845 shutdown_connections 2024-04-21T08:49:36.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.618+0000 7fcb36faf740 1 -- 172.21.15.43:0/985833845 wait complete. 2024-04-21T08:49:36.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.618+0000 7fcb36faf740 1 -- start start 2024-04-21T08:49:36.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.618+0000 7fcb36faf740 1 Processor -- start 2024-04-21T08:49:36.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.618+0000 7fcb36faf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a3dfb0 0x561a30a60860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.618+0000 7fcb36faf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a30a438e0 0x561a30a60da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.618+0000 7fcb36faf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a30a44290 0x561a30abc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.618+0000 7fcb36faf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c1b2b30 con 0x561a30a3dfb0 2024-04-21T08:49:36.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.618+0000 7fcb36faf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c1b2460 con 0x561a30a438e0 2024-04-21T08:49:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.618+0000 7fcb36faf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c1af6c0 con 0x561a30a44290 2024-04-21T08:49:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.618+0000 7fcb36faf740 10 client.0 _ll_register_callbacks cb 0x7fcb28005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.618+0000 7fcb3207c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a3dfb0 0x561a30a60860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.618+0000 7fcb3207c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a3dfb0 0x561a30a60860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48268/0 (socket says 172.21.15.43:48268) 2024-04-21T08:49:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.618+0000 7fcb3207c640 1 -- 172.21.15.43:0/3373230388 learned_addr learned my addr 172.21.15.43:0/3373230388 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.619+0000 7fcb3287d640 1 --2- 172.21.15.43:0/3373230388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a30a44290 0x561a30abc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.619+0000 7fcb3187b640 1 --2- 172.21.15.43:0/3373230388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a30a438e0 0x561a30a60da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.619+0000 7fcb3207c640 1 -- 172.21.15.43:0/3373230388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a30a44290 msgr2=0x561a30abc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.619+0000 7fcb3207c640 1 --2- 172.21.15.43:0/3373230388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a30a44290 0x561a30abc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.619+0000 7fcb3207c640 1 -- 172.21.15.43:0/3373230388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a30a438e0 msgr2=0x561a30a60da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.619+0000 7fcb3207c640 1 --2- 172.21.15.43:0/3373230388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a30a438e0 0x561a30a60da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.619+0000 7fcb3207c640 1 -- 172.21.15.43:0/3373230388 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a30abc930 con 0x561a30a3dfb0 2024-04-21T08:49:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.619+0000 7fcb3207c640 1 --2- 172.21.15.43:0/3373230388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a3dfb0 0x561a30a60860 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7fcb20099290 tx=0x7fcb200bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175a5b6bfce70ad3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:36.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.619+0000 7fcb15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:36.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.619+0000 7fcb15ffb640 1 -- 172.21.15.43:0/3373230388 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb200a5530 con 0x561a30a3dfb0 2024-04-21T08:49:36.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.619+0000 7fcb15ffb640 1 -- 172.21.15.43:0/3373230388 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fcb2000fa90 con 0x561a30a3dfb0 2024-04-21T08:49:36.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.619+0000 7fcb36faf740 1 -- 172.21.15.43:0/3373230388 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb24017070 con 0x561a30a3dfb0 2024-04-21T08:49:36.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.619+0000 7fcb15ffb640 1 -- 172.21.15.43:0/3373230388 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcb2000f740 con 0x561a30a3dfb0 2024-04-21T08:49:36.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.619+0000 7fcb36faf740 10 client.20790 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:36.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.619+0000 7fcb36faf740 1 -- 172.21.15.43:0/3373230388 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561a30ac3d70 con 0x561a30a3dfb0 2024-04-21T08:49:36.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.619+0000 7fcb15ffb640 1 -- 172.21.15.43:0/3373230388 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb24017070 con 0x561a30a3dfb0 2024-04-21T08:49:36.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.620+0000 7fcb15ffb640 1 -- 172.21.15.43:0/3373230388 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcb200870b0 con 0x561a30a3dfb0 2024-04-21T08:49:36.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.620+0000 7fcb36faf740 10 client.20790 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:36.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.620+0000 7fcb36faf740 10 client.20790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:36.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.620+0000 7fcb36faf740 20 client.20790 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:36.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.620+0000 7fcb36faf740 1 --2- 172.21.15.43:0/3373230388 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561a30a4ce10 0x561a30ac6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.620+0000 7fcb36faf740 4 client.20790 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:49:36.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.620+0000 7fcb36faf740 1 -- 172.21.15.43:0/3373230388 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561a309af790 con 0x561a30a4ce10 2024-04-21T08:49:36.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.620+0000 7fcb3187b640 1 --2- 172.21.15.43:0/3373230388 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561a30a4ce10 0x561a30ac6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.620+0000 7fcb3187b640 1 --2- 172.21.15.43:0/3373230388 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561a30a4ce10 0x561a30ac6480 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6bad4c1ece4954c server_cookie=8376c79a487bd2e8 in_seq=0 out_seq=0 2024-04-21T08:49:36.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.620+0000 7fcb15ffb640 10 client.20790 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:36.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.621+0000 7fcb15ffb640 1 -- 172.21.15.43:0/3373230388 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561a309af790 con 0x561a30a4ce10 2024-04-21T08:49:36.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.621+0000 7fcb15ffb640 10 client.20790 handle_command_reply: tid=0 2024-04-21T08:49:36.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.621+0000 7fcb36faf740 1 client.20790 shutdown 2024-04-21T08:49:36.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.622+0000 7fcb36faf740 1 -- 172.21.15.43:0/3373230388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a3dfb0 msgr2=0x561a30a60860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:36.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.622+0000 7fcb36faf740 1 --2- 172.21.15.43:0/3373230388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a3dfb0 0x561a30a60860 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7fcb20099290 tx=0x7fcb200bd040 comp rx=0 tx=0).stop 2024-04-21T08:49:36.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.622+0000 7fcb36faf740 1 -- 172.21.15.43:0/3373230388 shutdown_connections 2024-04-21T08:49:36.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.622+0000 7fcb36faf740 1 --2- 172.21.15.43:0/3373230388 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561a30a4ce10 0x561a30ac6480 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.622+0000 7fcb36faf740 1 --2- 172.21.15.43:0/3373230388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561a30a44290 0x561a30abc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.622+0000 7fcb15ffb640 0 client.20790 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:36.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.622+0000 7fcb36faf740 1 --2- 172.21.15.43:0/3373230388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a30a438e0 0x561a30a60da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.622+0000 7fcb36faf740 1 --2- 172.21.15.43:0/3373230388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a30a3dfb0 0x561a30a60860 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.622+0000 7fcb36faf740 1 -- 172.21.15.43:0/3373230388 >> 172.21.15.43:0/3373230388 conn(0x561a30a449e0 msgr2=0x561a30a3f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:36.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.622+0000 7fcb36faf740 1 -- 172.21.15.43:0/3373230388 shutdown_connections 2024-04-21T08:49:36.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.622+0000 7fcb36faf740 1 -- 172.21.15.43:0/3373230388 wait complete. 2024-04-21T08:49:36.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.622+0000 7fcb36faf740 20 client.20790 trim_cache size 0 max 16384 2024-04-21T08:49:36.728 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:49:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:49:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "08d99fe7-86e2-4f2d-a241-768a1509fb91", 2024-04-21T08:49:36.728 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:49:36.729 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:49:36.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.622+0000 7fcb342e0640 1 -- 172.21.15.43:0/3496239483 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcb180a3d20 msgr2=0x7fcb180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:36.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.622+0000 7fcb342e0640 1 --2- 172.21.15.43:0/3496239483 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcb180a3d20 0x7fcb180a61e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcb24005fd0 tx=0x7fcb24004a40 comp rx=0 tx=0).stop 2024-04-21T08:49:36.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.622+0000 7fcb342e0640 1 -- 172.21.15.43:0/3496239483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb2c101090 msgr2=0x7fcb2c1a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:36.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.622+0000 7fcb342e0640 1 --2- 172.21.15.43:0/3496239483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb2c101090 0x7fcb2c1a0660 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000c920 tx=0x7fcb2000cde0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.622+0000 7fcb342e0640 1 -- 172.21.15.43:0/3496239483 shutdown_connections 2024-04-21T08:49:36.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.622+0000 7fcb342e0640 1 --2- 172.21.15.43:0/3496239483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb2c19a720 0x7fcb2c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.622+0000 7fcb342e0640 1 --2- 172.21.15.43:0/3496239483 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcb180a3d20 0x7fcb180a61e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.622+0000 7fcb342e0640 1 --2- 172.21.15.43:0/3496239483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb2c106160 0x7fcb2c1a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.622+0000 7fcb342e0640 1 --2- 172.21.15.43:0/3496239483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb2c101090 0x7fcb2c1a0660 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.622+0000 7fcb342e0640 1 -- 172.21.15.43:0/3496239483 >> 172.21.15.43:0/3496239483 conn(0x7fcb2c077f50 msgr2=0x7fcb2c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:36.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.623+0000 7fcb342e0640 1 -- 172.21.15.43:0/3496239483 shutdown_connections 2024-04-21T08:49:36.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.623+0000 7fcb342e0640 1 -- 172.21.15.43:0/3496239483 wait complete. 2024-04-21T08:49:36.731 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:49:36.731 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:49:36.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.882+0000 7f320beb3640 1 -- 172.21.15.43:0/2947255690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32041031c0 msgr2=0x7f32041035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:36.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.882+0000 7f320beb3640 1 --2- 172.21.15.43:0/2947255690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32041031c0 0x7f32041035a0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f3200009900 tx=0x7f320002bd60 comp rx=0 tx=0).stop 2024-04-21T08:49:36.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.882+0000 7f320beb3640 1 -- 172.21.15.43:0/2947255690 shutdown_connections 2024-04-21T08:49:36.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.882+0000 7f320beb3640 1 --2- 172.21.15.43:0/2947255690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f320410c500 0x7f320410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.882+0000 7f320beb3640 1 --2- 172.21.15.43:0/2947255690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3204103ae0 0x7f320410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.882+0000 7f320beb3640 1 --2- 172.21.15.43:0/2947255690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32041031c0 0x7f32041035a0 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.882+0000 7f320beb3640 1 -- 172.21.15.43:0/2947255690 >> 172.21.15.43:0/2947255690 conn(0x7f3204078000 msgr2=0x7f3204078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:36.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.882+0000 7f320beb3640 1 -- 172.21.15.43:0/2947255690 shutdown_connections 2024-04-21T08:49:36.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.882+0000 7f320beb3640 1 -- 172.21.15.43:0/2947255690 wait complete. 2024-04-21T08:49:36.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.883+0000 7f320beb3640 1 Processor -- start 2024-04-21T08:49:36.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.883+0000 7f320beb3640 1 -- start start 2024-04-21T08:49:36.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.883+0000 7f320beb3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32041031c0 0x7f3204198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.883+0000 7f320beb3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3204103ae0 0x7f3204198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.883+0000 7f320beb3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f320410c500 0x7f320419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.883+0000 7f320beb3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3204112170 con 0x7f3204103ae0 2024-04-21T08:49:36.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.883+0000 7f320beb3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3204111ff0 con 0x7f320410c500 2024-04-21T08:49:36.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.883+0000 7f320beb3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32041122f0 con 0x7f32041031c0 2024-04-21T08:49:36.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.883+0000 7f320944e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3204103ae0 0x7f3204198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.883+0000 7f320944e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3204103ae0 0x7f3204198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59120/0 (socket says 172.21.15.43:59120) 2024-04-21T08:49:36.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.883+0000 7f320944e640 1 -- 172.21.15.43:0/384884807 learned_addr learned my addr 172.21.15.43:0/384884807 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:36.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.883+0000 7f3209c4f640 1 --2- 172.21.15.43:0/384884807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32041031c0 0x7f3204198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.884+0000 7f320a450640 1 --2- 172.21.15.43:0/384884807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f320410c500 0x7f320419cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.884+0000 7f320944e640 1 -- 172.21.15.43:0/384884807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32041031c0 msgr2=0x7f3204198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:36.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.884+0000 7f320944e640 1 --2- 172.21.15.43:0/384884807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32041031c0 0x7f3204198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.884+0000 7f320944e640 1 -- 172.21.15.43:0/384884807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f320410c500 msgr2=0x7f320419cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:36.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.884+0000 7f320944e640 1 --2- 172.21.15.43:0/384884807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f320410c500 0x7f320419cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.884+0000 7f320944e640 1 -- 172.21.15.43:0/384884807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320419d6b0 con 0x7f3204103ae0 2024-04-21T08:49:36.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.884+0000 7f320944e640 1 --2- 172.21.15.43:0/384884807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3204103ae0 0x7f3204198c50 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f31f400d950 tx=0x7f31f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:36.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.884+0000 7f31faffd640 1 -- 172.21.15.43:0/384884807 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f4017070 con 0x7f3204103ae0 2024-04-21T08:49:36.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.884+0000 7f31faffd640 1 -- 172.21.15.43:0/384884807 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31f4009c30 con 0x7f3204103ae0 2024-04-21T08:49:36.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.884+0000 7f320beb3640 1 -- 172.21.15.43:0/384884807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320419d9a0 con 0x7f3204103ae0 2024-04-21T08:49:36.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.884+0000 7f31faffd640 1 -- 172.21.15.43:0/384884807 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f400f620 con 0x7f3204103ae0 2024-04-21T08:49:36.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.884+0000 7f320beb3640 1 -- 172.21.15.43:0/384884807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32041a5230 con 0x7f3204103ae0 2024-04-21T08:49:36.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.886+0000 7f31faffd640 1 -- 172.21.15.43:0/384884807 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f31f4010460 con 0x7f3204103ae0 2024-04-21T08:49:36.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.887+0000 7f320eb7e740 1 Processor -- start 2024-04-21T08:49:36.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.887+0000 7f31faffd640 1 --2- 172.21.15.43:0/384884807 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f31f00a3d20 0x7f31f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.887+0000 7f320eb7e740 1 -- start start 2024-04-21T08:49:36.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.887+0000 7f320eb7e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557eea4d3580 0x557eea4d3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.887+0000 7f320eb7e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eea4d3ea0 0x557eea4dc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.887+0000 7f320a450640 1 --2- 172.21.15.43:0/384884807 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f31f00a3d20 0x7f31f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.887+0000 7f320eb7e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4dca80 0x557eea4dee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.887+0000 7f320eb7e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557eea4381e0 con 0x557eea4dca80 2024-04-21T08:49:36.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.887+0000 7f320eb7e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557eea3d5390 con 0x557eea4d3ea0 2024-04-21T08:49:36.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.887+0000 7f31faffd640 1 -- 172.21.15.43:0/384884807 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f31f40d0880 con 0x7f3204103ae0 2024-04-21T08:49:36.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.887+0000 7f320eb7e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557eea4341a0 con 0x557eea4d3580 2024-04-21T08:49:36.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.888+0000 7f320a450640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4dca80 0x557eea4dee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.888+0000 7f320a450640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4dca80 0x557eea4dee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59132/0 (socket says 172.21.15.43:59132) 2024-04-21T08:49:36.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.888+0000 7f320a450640 1 -- 172.21.15.43:0/540071845 learned_addr learned my addr 172.21.15.43:0/540071845 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:36.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.888+0000 7f320944e640 1 --2- 172.21.15.43:0/540071845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eea4d3ea0 0x557eea4dc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.888+0000 7f3209c4f640 1 --2- 172.21.15.43:0/540071845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557eea4d3580 0x557eea4d3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.888+0000 7f320a450640 1 -- 172.21.15.43:0/540071845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557eea4d3580 msgr2=0x557eea4d3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.888+0000 7f320a450640 1 --2- 172.21.15.43:0/540071845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557eea4d3580 0x557eea4d3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.888+0000 7f320a450640 1 -- 172.21.15.43:0/540071845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eea4d3ea0 msgr2=0x557eea4dc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.888+0000 7f320a450640 1 --2- 172.21.15.43:0/540071845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eea4d3ea0 0x557eea4dc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.888+0000 7f320a450640 1 -- 172.21.15.43:0/540071845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eea2147d0 con 0x557eea4dca80 2024-04-21T08:49:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.888+0000 7f320944e640 1 --2- 172.21.15.43:0/540071845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eea4d3ea0 0x557eea4dc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.888+0000 7f320a450640 1 --2- 172.21.15.43:0/384884807 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f31f00a3d20 0x7f31f00a61e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f31fc015b70 tx=0x7f31fc009d90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.888+0000 7f320a450640 1 --2- 172.21.15.43:0/540071845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4dca80 0x557eea4dee70 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f31fc015fd0 tx=0x7f31fc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de6195ca37fb545 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.888+0000 7f31effff640 1 -- 172.21.15.43:0/540071845 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc012e10 con 0x557eea4dca80 2024-04-21T08:49:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.889+0000 7f31effff640 1 -- 172.21.15.43:0/540071845 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31fc010c10 con 0x557eea4dca80 2024-04-21T08:49:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.889+0000 7f31effff640 1 -- 172.21.15.43:0/540071845 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc005390 con 0x557eea4dca80 2024-04-21T08:49:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.889+0000 7f320eb7e740 1 -- 172.21.15.43:0/540071845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4dca80 msgr2=0x557eea4dee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.889+0000 7f320eb7e740 1 --2- 172.21.15.43:0/540071845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4dca80 0x557eea4dee70 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f31fc015fd0 tx=0x7f31fc01c000 comp rx=0 tx=0).stop 2024-04-21T08:49:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.889+0000 7f320eb7e740 1 -- 172.21.15.43:0/540071845 shutdown_connections 2024-04-21T08:49:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.889+0000 7f320eb7e740 1 --2- 172.21.15.43:0/540071845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4dca80 0x557eea4dee70 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.889+0000 7f320eb7e740 1 --2- 172.21.15.43:0/540071845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eea4d3ea0 0x557eea4dc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.889+0000 7f320eb7e740 1 --2- 172.21.15.43:0/540071845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557eea4d3580 0x557eea4d3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.889+0000 7f320eb7e740 1 -- 172.21.15.43:0/540071845 >> 172.21.15.43:0/540071845 conn(0x557eea4cdcb0 msgr2=0x557eea4ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.889+0000 7f320eb7e740 1 -- 172.21.15.43:0/540071845 shutdown_connections 2024-04-21T08:49:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.889+0000 7f320eb7e740 1 -- 172.21.15.43:0/540071845 wait complete. 2024-04-21T08:49:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.890+0000 7f320eb7e740 1 -- start start 2024-04-21T08:49:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.890+0000 7f320eb7e740 1 Processor -- start 2024-04-21T08:49:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.890+0000 7f320eb7e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4d3580 0x557eea4d9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.890+0000 7f320eb7e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557eea4d3ea0 0x557eea4da020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.890+0000 7f320eb7e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eea4dca80 0x557eea4da560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.890+0000 7f320eb7e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557eea4e3c20 con 0x557eea4d3580 2024-04-21T08:49:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.890+0000 7f320eb7e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557eea3f12f0 con 0x557eea4dca80 2024-04-21T08:49:36.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.890+0000 7f320eb7e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557eea4e3da0 con 0x557eea4d3ea0 2024-04-21T08:49:36.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.890+0000 7f320eb7e740 10 client.0 _ll_register_callbacks cb 0x557eea3fe580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:36.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.890+0000 7f3209c4f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4d3580 0x557eea4d9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.890+0000 7f3209c4f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4d3580 0x557eea4d9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59142/0 (socket says 172.21.15.43:59142) 2024-04-21T08:49:36.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.891+0000 7f3209c4f640 1 -- 172.21.15.43:0/1967221964 learned_addr learned my addr 172.21.15.43:0/1967221964 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:36.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.891+0000 7f320a450640 1 --2- 172.21.15.43:0/1967221964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eea4dca80 0x557eea4da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.891+0000 7f320944e640 1 --2- 172.21.15.43:0/1967221964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557eea4d3ea0 0x557eea4da020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.891+0000 7f3209c4f640 1 -- 172.21.15.43:0/1967221964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557eea4d3ea0 msgr2=0x557eea4da020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:36.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.891+0000 7f3209c4f640 1 --2- 172.21.15.43:0/1967221964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557eea4d3ea0 0x557eea4da020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.891+0000 7f3209c4f640 1 -- 172.21.15.43:0/1967221964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eea4dca80 msgr2=0x557eea4da560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:36.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.891+0000 7f3209c4f640 1 --2- 172.21.15.43:0/1967221964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eea4dca80 0x557eea4da560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.891+0000 7f3209c4f640 1 -- 172.21.15.43:0/1967221964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eea14a3f0 con 0x557eea4d3580 2024-04-21T08:49:36.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.891+0000 7f320944e640 1 --2- 172.21.15.43:0/1967221964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557eea4d3ea0 0x557eea4da020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:36.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.891+0000 7f3209c4f640 1 --2- 172.21.15.43:0/1967221964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4d3580 0x557eea4d9ae0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f3200009900 tx=0x7f320002db30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf18eab4fd514887 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:36.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.891+0000 7f31ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:36.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.891+0000 7f31ed7fa640 1 -- 172.21.15.43:0/1967221964 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3200044030 con 0x557eea4d3580 2024-04-21T08:49:36.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.891+0000 7f31ed7fa640 1 -- 172.21.15.43:0/1967221964 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f320003a040 con 0x557eea4d3580 2024-04-21T08:49:36.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.891+0000 7f320eb7e740 1 -- 172.21.15.43:0/1967221964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31fc010c10 con 0x557eea4d3580 2024-04-21T08:49:36.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.891+0000 7f31ed7fa640 1 -- 172.21.15.43:0/1967221964 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3200051050 con 0x557eea4d3580 2024-04-21T08:49:36.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.891+0000 7f320eb7e740 10 client.20802 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:36.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.891+0000 7f320eb7e740 1 -- 172.21.15.43:0/1967221964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557eea10a0f0 con 0x557eea4d3580 2024-04-21T08:49:36.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.892+0000 7f31ed7fa640 1 -- 172.21.15.43:0/1967221964 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31fc010c10 con 0x557eea4d3580 2024-04-21T08:49:36.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.892+0000 7f31ed7fa640 1 -- 172.21.15.43:0/1967221964 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f32000035f0 con 0x557eea4d3580 2024-04-21T08:49:36.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.892+0000 7f320eb7e740 10 client.20802 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:36.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.892+0000 7f320eb7e740 10 client.20802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:36.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.892+0000 7f320eb7e740 20 client.20802 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:36.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.892+0000 7f320eb7e740 1 --2- 172.21.15.43:0/1967221964 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557eea4f0ef0 0x557eea4f3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.892+0000 7f320eb7e740 4 client.20802 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.892+0000 7f320eb7e740 1 -- 172.21.15.43:0/1967221964 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557eea445c00 con 0x557eea4f0ef0 2024-04-21T08:49:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.892+0000 7f320944e640 1 --2- 172.21.15.43:0/1967221964 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557eea4f0ef0 0x557eea4f3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.893+0000 7f31ed7fa640 10 client.20802 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.893+0000 7f320944e640 1 --2- 172.21.15.43:0/1967221964 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557eea4f0ef0 0x557eea4f3360 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=168018710d893c65 server_cookie=e2b941895619c74c in_seq=0 out_seq=0 2024-04-21T08:49:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.893+0000 7f31ed7fa640 1 -- 172.21.15.43:0/1967221964 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557eea445c00 con 0x557eea4f0ef0 2024-04-21T08:49:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.893+0000 7f31ed7fa640 10 client.20802 handle_command_reply: tid=0 2024-04-21T08:49:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.893+0000 7f320eb7e740 1 client.20802 shutdown 2024-04-21T08:49:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.893+0000 7f320eb7e740 1 -- 172.21.15.43:0/1967221964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4d3580 msgr2=0x557eea4d9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.893+0000 7f320eb7e740 1 --2- 172.21.15.43:0/1967221964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4d3580 0x557eea4d9ae0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f3200009900 tx=0x7f320002db30 comp rx=0 tx=0).stop 2024-04-21T08:49:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.894+0000 7f320eb7e740 1 -- 172.21.15.43:0/1967221964 shutdown_connections 2024-04-21T08:49:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.894+0000 7f320eb7e740 1 --2- 172.21.15.43:0/1967221964 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557eea4f0ef0 0x557eea4f3360 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.894+0000 7f320eb7e740 1 --2- 172.21.15.43:0/1967221964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eea4dca80 0x557eea4da560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.894+0000 7f320eb7e740 1 --2- 172.21.15.43:0/1967221964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557eea4d3ea0 0x557eea4da020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.894+0000 7f320eb7e740 1 --2- 172.21.15.43:0/1967221964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4d3580 0x557eea4d9ae0 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.894+0000 7f320eb7e740 1 -- 172.21.15.43:0/1967221964 >> 172.21.15.43:0/1967221964 conn(0x557eea4cdcb0 msgr2=0x557eea4cfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:36.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.894+0000 7f31ed7fa640 0 client.20802 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:36.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.894+0000 7f320eb7e740 1 -- 172.21.15.43:0/1967221964 shutdown_connections 2024-04-21T08:49:36.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.894+0000 7f320eb7e740 1 -- 172.21.15.43:0/1967221964 wait complete. 2024-04-21T08:49:36.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.894+0000 7f320eb7e740 20 client.20802 trim_cache size 0 max 16384 2024-04-21T08:49:36.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.924+0000 7f320eb7e740 1 Processor -- start 2024-04-21T08:49:36.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.924+0000 7f320eb7e740 1 -- start start 2024-04-21T08:49:36.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.924+0000 7f320eb7e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4d3580 0x557eea4dc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.924+0000 7f320eb7e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eea4d3ea0 0x557eea4ecb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.925+0000 7f320eb7e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557eea4dca80 0x557eea4ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.925+0000 7f320eb7e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f32041ad290 con 0x557eea4d3580 2024-04-21T08:49:36.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.925+0000 7f320eb7e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f32041aedd0 con 0x557eea4d3ea0 2024-04-21T08:49:36.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.925+0000 7f320eb7e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32041ad960 con 0x557eea4dca80 2024-04-21T08:49:36.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.925+0000 7f3209c4f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4d3580 0x557eea4dc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.925+0000 7f3209c4f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4d3580 0x557eea4dc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59158/0 (socket says 172.21.15.43:59158) 2024-04-21T08:49:36.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.925+0000 7f3209c4f640 1 -- 172.21.15.43:0/915161788 learned_addr learned my addr 172.21.15.43:0/915161788 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:36.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.925+0000 7f320a450640 1 --2- 172.21.15.43:0/915161788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557eea4dca80 0x557eea4ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.925+0000 7f320944e640 1 --2- 172.21.15.43:0/915161788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eea4d3ea0 0x557eea4ecb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.925+0000 7f3209c4f640 1 -- 172.21.15.43:0/915161788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557eea4dca80 msgr2=0x557eea4ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:36.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.925+0000 7f3209c4f640 1 --2- 172.21.15.43:0/915161788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557eea4dca80 0x557eea4ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.925+0000 7f3209c4f640 1 -- 172.21.15.43:0/915161788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eea4d3ea0 msgr2=0x557eea4ecb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:36.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.925+0000 7f3209c4f640 1 --2- 172.21.15.43:0/915161788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eea4d3ea0 0x557eea4ecb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.925+0000 7f3209c4f640 1 -- 172.21.15.43:0/915161788 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eea4f36d0 con 0x557eea4d3580 2024-04-21T08:49:36.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.925+0000 7f3209c4f640 1 --2- 172.21.15.43:0/915161788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4d3580 0x557eea4dc210 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f3200000c00 tx=0x7f320002bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698699a9618bdbfc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:36.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.925+0000 7f31ef7fe640 1 -- 172.21.15.43:0/915161788 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320004e070 con 0x557eea4d3580 2024-04-21T08:49:36.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.925+0000 7f31ef7fe640 1 -- 172.21.15.43:0/915161788 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3200005700 con 0x557eea4d3580 2024-04-21T08:49:36.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.925+0000 7f31ef7fe640 1 -- 172.21.15.43:0/915161788 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3200044070 con 0x557eea4d3580 2024-04-21T08:49:36.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.926+0000 7f320eb7e740 1 -- 172.21.15.43:0/915161788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4d3580 msgr2=0x557eea4dc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:36.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.926+0000 7f320eb7e740 1 --2- 172.21.15.43:0/915161788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4d3580 0x557eea4dc210 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f3200000c00 tx=0x7f320002bd60 comp rx=0 tx=0).stop 2024-04-21T08:49:36.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.926+0000 7f320eb7e740 1 -- 172.21.15.43:0/915161788 shutdown_connections 2024-04-21T08:49:36.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.926+0000 7f320eb7e740 1 --2- 172.21.15.43:0/915161788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557eea4dca80 0x557eea4ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.926+0000 7f320eb7e740 1 --2- 172.21.15.43:0/915161788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eea4d3ea0 0x557eea4ecb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.926+0000 7f320eb7e740 1 --2- 172.21.15.43:0/915161788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4d3580 0x557eea4dc210 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.926+0000 7f320eb7e740 1 -- 172.21.15.43:0/915161788 >> 172.21.15.43:0/915161788 conn(0x557eea4cdcb0 msgr2=0x557eea4cc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:36.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.926+0000 7f320eb7e740 1 -- 172.21.15.43:0/915161788 shutdown_connections 2024-04-21T08:49:36.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.926+0000 7f320eb7e740 1 -- 172.21.15.43:0/915161788 wait complete. 2024-04-21T08:49:36.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.927+0000 7f320eb7e740 1 -- start start 2024-04-21T08:49:36.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.927+0000 7f320eb7e740 1 Processor -- start 2024-04-21T08:49:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.927+0000 7f320eb7e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eea4d3580 0x557eea4e4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.927+0000 7f320eb7e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4d3ea0 0x557eea4e4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.927+0000 7f320eb7e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557eea4dca80 0x557eea4e53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.927+0000 7f320eb7e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f32041ae700 con 0x557eea4d3ea0 2024-04-21T08:49:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.927+0000 7f320eb7e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f32041ae030 con 0x557eea4d3580 2024-04-21T08:49:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.927+0000 7f320eb7e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32041ab290 con 0x557eea4dca80 2024-04-21T08:49:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.927+0000 7f320eb7e740 10 client.0 _ll_register_callbacks cb 0x7f3200009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.927+0000 7f320944e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4d3ea0 0x557eea4e4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.927+0000 7f320944e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4d3ea0 0x557eea4e4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59174/0 (socket says 172.21.15.43:59174) 2024-04-21T08:49:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.927+0000 7f320944e640 1 -- 172.21.15.43:0/271161634 learned_addr learned my addr 172.21.15.43:0/271161634 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.927+0000 7f320a450640 1 --2- 172.21.15.43:0/271161634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557eea4dca80 0x557eea4e53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.928+0000 7f3209c4f640 1 --2- 172.21.15.43:0/271161634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eea4d3580 0x557eea4e4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.928+0000 7f320944e640 1 -- 172.21.15.43:0/271161634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557eea4dca80 msgr2=0x557eea4e53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.928+0000 7f320944e640 1 --2- 172.21.15.43:0/271161634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557eea4dca80 0x557eea4e53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.928+0000 7f320944e640 1 -- 172.21.15.43:0/271161634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eea4d3580 msgr2=0x557eea4e4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.928+0000 7f320944e640 1 --2- 172.21.15.43:0/271161634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eea4d3580 0x557eea4e4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.928+0000 7f320944e640 1 -- 172.21.15.43:0/271161634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eea5564a0 con 0x557eea4d3ea0 2024-04-21T08:49:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.928+0000 7f3209c4f640 1 --2- 172.21.15.43:0/271161634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eea4d3580 0x557eea4e4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.928+0000 7f320944e640 1 --2- 172.21.15.43:0/271161634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4d3ea0 0x557eea4e4ea0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f31f4000c00 tx=0x7f31f4009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5270bbd784ba437 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.928+0000 7f31ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.928+0000 7f31ed7fa640 1 -- 172.21.15.43:0/271161634 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f4020040 con 0x557eea4d3ea0 2024-04-21T08:49:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.928+0000 7f31ed7fa640 1 -- 172.21.15.43:0/271161634 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f31f4098a70 con 0x557eea4d3ea0 2024-04-21T08:49:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.928+0000 7f320eb7e740 1 -- 172.21.15.43:0/271161634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3200005700 con 0x557eea4d3ea0 2024-04-21T08:49:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.928+0000 7f31ed7fa640 1 -- 172.21.15.43:0/271161634 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f31f40b5b80 con 0x557eea4d3ea0 2024-04-21T08:49:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.928+0000 7f320eb7e740 10 client.20808 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.928+0000 7f320eb7e740 1 -- 172.21.15.43:0/271161634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557eea557710 con 0x557eea4d3ea0 2024-04-21T08:49:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.928+0000 7f31ed7fa640 1 -- 172.21.15.43:0/271161634 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3200005700 con 0x557eea4d3ea0 2024-04-21T08:49:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.929+0000 7f31ed7fa640 1 -- 172.21.15.43:0/271161634 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f31f4098150 con 0x557eea4d3ea0 2024-04-21T08:49:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.929+0000 7f320eb7e740 10 client.20808 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.929+0000 7f320eb7e740 10 client.20808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.929+0000 7f320eb7e740 20 client.20808 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.929+0000 7f320eb7e740 1 --2- 172.21.15.43:0/271161634 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557eea557e40 0x557eea55a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.929+0000 7f320eb7e740 4 client.20808 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:49:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.929+0000 7f320eb7e740 1 -- 172.21.15.43:0/271161634 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557eea43fa70 con 0x557eea557e40 2024-04-21T08:49:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.929+0000 7f3209c4f640 1 --2- 172.21.15.43:0/271161634 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557eea557e40 0x557eea55a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.929+0000 7f31ed7fa640 10 client.20808 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.929+0000 7f3209c4f640 1 --2- 172.21.15.43:0/271161634 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557eea557e40 0x557eea55a260 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c0832f8ac764083 server_cookie=ef0122a0ab0feb30 in_seq=0 out_seq=0 2024-04-21T08:49:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.930+0000 7f31ed7fa640 1 -- 172.21.15.43:0/271161634 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557eea43fa70 con 0x557eea557e40 2024-04-21T08:49:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.930+0000 7f31ed7fa640 10 client.20808 handle_command_reply: tid=0 2024-04-21T08:49:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.930+0000 7f320eb7e740 1 client.20808 shutdown 2024-04-21T08:49:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.930+0000 7f320eb7e740 1 -- 172.21.15.43:0/271161634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4d3ea0 msgr2=0x557eea4e4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:36.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.930+0000 7f320eb7e740 1 --2- 172.21.15.43:0/271161634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4d3ea0 0x557eea4e4ea0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f31f4000c00 tx=0x7f31f4009f30 comp rx=0 tx=0).stop 2024-04-21T08:49:36.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.930+0000 7f320eb7e740 1 -- 172.21.15.43:0/271161634 shutdown_connections 2024-04-21T08:49:36.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.930+0000 7f320eb7e740 1 --2- 172.21.15.43:0/271161634 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557eea557e40 0x557eea55a260 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.930+0000 7f320eb7e740 1 --2- 172.21.15.43:0/271161634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557eea4dca80 0x557eea4e53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.930+0000 7f31ed7fa640 0 client.20808 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:36.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.930+0000 7f320eb7e740 1 --2- 172.21.15.43:0/271161634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eea4d3ea0 0x557eea4e4ea0 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.930+0000 7f320eb7e740 1 --2- 172.21.15.43:0/271161634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557eea4d3580 0x557eea4e4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.930+0000 7f320eb7e740 1 -- 172.21.15.43:0/271161634 >> 172.21.15.43:0/271161634 conn(0x557eea4cdcb0 msgr2=0x557eea4cf970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:36.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.930+0000 7f320eb7e740 1 -- 172.21.15.43:0/271161634 shutdown_connections 2024-04-21T08:49:36.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.930+0000 7f320eb7e740 1 -- 172.21.15.43:0/271161634 wait complete. 2024-04-21T08:49:36.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.930+0000 7f320eb7e740 20 client.20808 trim_cache size 0 max 16384 2024-04-21T08:49:36.937 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:49:36.937 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:49:36.937 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-21T08:49:36.937 INFO:teuthology.orchestra.run.smithi043.stdout: "08d99fe7-86e2-4f2d-a241-768a1509fb91": { 2024-04-21T08:49:36.937 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-21T08:49:36.937 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "08d99fe7-86e2-4f2d-a241-768a1509fb91", 2024-04-21T08:49:36.937 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-21T08:49:36.937 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:49:36.937 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:49:36.937 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:49:36.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.930+0000 7f320beb3640 1 -- 172.21.15.43:0/384884807 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f31f00a3d20 msgr2=0x7f31f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:36.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.930+0000 7f320beb3640 1 --2- 172.21.15.43:0/384884807 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f31f00a3d20 0x7f31f00a61e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f31fc015b70 tx=0x7f31fc009d90 comp rx=0 tx=0).stop 2024-04-21T08:49:36.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.930+0000 7f320beb3640 1 -- 172.21.15.43:0/384884807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3204103ae0 msgr2=0x7f3204198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:36.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.930+0000 7f320beb3640 1 --2- 172.21.15.43:0/384884807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3204103ae0 0x7f3204198c50 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f31f400d950 tx=0x7f31f400de10 comp rx=0 tx=0).stop 2024-04-21T08:49:36.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.931+0000 7f320beb3640 1 -- 172.21.15.43:0/384884807 shutdown_connections 2024-04-21T08:49:36.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.931+0000 7f320beb3640 1 --2- 172.21.15.43:0/384884807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f320410c500 0x7f320419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.931+0000 7f320beb3640 1 --2- 172.21.15.43:0/384884807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3204103ae0 0x7f3204198c50 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.931+0000 7f320beb3640 1 --2- 172.21.15.43:0/384884807 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f31f00a3d20 0x7f31f00a61e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.931+0000 7f320beb3640 1 --2- 172.21.15.43:0/384884807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32041031c0 0x7f3204198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.931+0000 7f320beb3640 1 -- 172.21.15.43:0/384884807 >> 172.21.15.43:0/384884807 conn(0x7f3204078000 msgr2=0x7f32040fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:36.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.931+0000 7f320beb3640 1 -- 172.21.15.43:0/384884807 shutdown_connections 2024-04-21T08:49:36.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:49:36.931+0000 7f320beb3640 1 -- 172.21.15.43:0/384884807 wait complete. 2024-04-21T08:49:36.941 INFO:tasks.cephfs.filesystem:scrub status for tag:08d99fe7-86e2-4f2d-a241-768a1509fb91 - {'path': '/', 'tag': '08d99fe7-86e2-4f2d-a241-768a1509fb91', 'options': 'recursive,force'} 2024-04-21T08:49:38.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:38 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:49:38.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:38 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:38.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:38 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:49:38.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:38 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:38.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:38 smithi175 ceph-mon[31920]: pgmap v333: 129 pgs: 129 active+clean; 2.4 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 26 op/s 2024-04-21T08:49:38.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:38 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:49:38.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:38 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:38.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:38 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:49:38.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:38 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:38.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:38 smithi112 ceph-mon[31085]: pgmap v333: 129 pgs: 129 active+clean; 2.4 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 26 op/s 2024-04-21T08:49:38.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:38 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:49:38.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:38 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:38.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:38 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:49:38.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:38 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:38.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:38 smithi043 ceph-mon[27978]: pgmap v333: 129 pgs: 129 active+clean; 2.4 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 26 op/s 2024-04-21T08:49:40.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:40 smithi175 ceph-mon[31920]: pgmap v334: 129 pgs: 129 active+clean; 2.4 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-21T08:49:40.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:40 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:49:40.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:40 smithi112 ceph-mon[31085]: pgmap v334: 129 pgs: 129 active+clean; 2.4 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-21T08:49:40.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:40 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:49:40.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:40 smithi043 ceph-mon[27978]: pgmap v334: 129 pgs: 129 active+clean; 2.4 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-21T08:49:40.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:40 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:49:42.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:42 smithi175 ceph-mon[31920]: pgmap v335: 129 pgs: 129 active+clean; 2.6 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 57 KiB/s wr, 24 op/s 2024-04-21T08:49:42.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:42 smithi112 ceph-mon[31085]: pgmap v335: 129 pgs: 129 active+clean; 2.6 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 57 KiB/s wr, 24 op/s 2024-04-21T08:49:42.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:42 smithi043 ceph-mon[27978]: pgmap v335: 129 pgs: 129 active+clean; 2.6 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 57 KiB/s wr, 24 op/s 2024-04-21T08:49:43.087 INFO:tasks.workunit.client.0.smithi043.stdout:437/600: open fd = 3 2024-04-21T08:49:43.087 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.088 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.088 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.088 INFO:tasks.workunit.client.0.smithi043.stdout:438/600: open fd = 3 2024-04-21T08:49:43.088 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.088 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.088 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.089 INFO:tasks.workunit.client.0.smithi043.stdout:439/600: open fd = 3 2024-04-21T08:49:43.089 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.089 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.089 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.089 INFO:tasks.workunit.client.0.smithi043.stdout:440/600: open fd = 3 2024-04-21T08:49:43.089 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.089 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.089 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.090 INFO:tasks.workunit.client.0.smithi043.stdout:441/600: open fd = 3 2024-04-21T08:49:43.090 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.090 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.090 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.090 INFO:tasks.workunit.client.0.smithi043.stdout:442/600: open fd = 3 2024-04-21T08:49:43.090 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.090 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.090 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.091 INFO:tasks.workunit.client.0.smithi043.stdout:443/600: open fd = 3 2024-04-21T08:49:43.091 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.091 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.091 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.091 INFO:tasks.workunit.client.0.smithi043.stdout:444/600: open fd = 3 2024-04-21T08:49:43.091 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.091 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.091 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.091 INFO:tasks.workunit.client.0.smithi043.stdout:445/600: open fd = 3 2024-04-21T08:49:43.091 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.091 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.091 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.091 INFO:tasks.workunit.client.0.smithi043.stdout:446/600: open fd = 3 2024-04-21T08:49:43.092 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.092 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.092 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.092 INFO:tasks.workunit.client.0.smithi043.stdout:447/600: open fd = 3 2024-04-21T08:49:43.092 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.092 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.092 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.092 INFO:tasks.workunit.client.0.smithi043.stdout:448/600: open fd = 3 2024-04-21T08:49:43.092 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.092 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.092 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.092 INFO:tasks.workunit.client.0.smithi043.stdout:449/600: open fd = 3 2024-04-21T08:49:43.092 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.092 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.092 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.093 INFO:tasks.workunit.client.0.smithi043.stdout:450/600: open fd = 3 2024-04-21T08:49:43.093 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.093 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.093 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.093 INFO:tasks.workunit.client.0.smithi043.stdout:451/600: open fd = 3 2024-04-21T08:49:43.093 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.093 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.093 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.093 INFO:tasks.workunit.client.0.smithi043.stdout:452/600: open fd = 3 2024-04-21T08:49:43.093 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.093 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.093 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.093 INFO:tasks.workunit.client.0.smithi043.stdout:453/600: open fd = 3 2024-04-21T08:49:43.093 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.093 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.094 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.094 INFO:tasks.workunit.client.0.smithi043.stdout:454/600: open fd = 3 2024-04-21T08:49:43.094 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.094 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.094 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.094 INFO:tasks.workunit.client.0.smithi043.stdout:455/600: open fd = 3 2024-04-21T08:49:43.094 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.094 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.094 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.094 INFO:tasks.workunit.client.0.smithi043.stdout:456/600: open fd = 3 2024-04-21T08:49:43.094 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.094 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.094 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.095 INFO:tasks.workunit.client.0.smithi043.stdout:457/600: open fd = 3 2024-04-21T08:49:43.095 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.095 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.095 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.095 INFO:tasks.workunit.client.0.smithi043.stdout:458/600: open fd = 3 2024-04-21T08:49:43.095 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.095 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.095 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.095 INFO:tasks.workunit.client.0.smithi043.stdout:459/600: open fd = 3 2024-04-21T08:49:43.095 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.095 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.095 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.095 INFO:tasks.workunit.client.0.smithi043.stdout:460/600: open fd = 3 2024-04-21T08:49:43.095 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.095 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.095 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.096 INFO:tasks.workunit.client.0.smithi043.stdout:461/600: open fd = 3 2024-04-21T08:49:43.096 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.096 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.096 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.096 INFO:tasks.workunit.client.0.smithi043.stdout:462/600: open fd = 3 2024-04-21T08:49:43.096 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.096 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.096 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.096 INFO:tasks.workunit.client.0.smithi043.stdout:463/600: open fd = 3 2024-04-21T08:49:43.096 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.096 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.096 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.096 INFO:tasks.workunit.client.0.smithi043.stdout:464/600: open fd = 3 2024-04-21T08:49:43.096 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.096 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.097 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.097 INFO:tasks.workunit.client.0.smithi043.stdout:465/600: open fd = 3 2024-04-21T08:49:43.097 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.097 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.097 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.097 INFO:tasks.workunit.client.0.smithi043.stdout:466/600: open fd = 3 2024-04-21T08:49:43.097 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.097 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.097 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.097 INFO:tasks.workunit.client.0.smithi043.stdout:467/600: open fd = 3 2024-04-21T08:49:43.097 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.097 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.097 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.097 INFO:tasks.workunit.client.0.smithi043.stdout:468/600: open fd = 3 2024-04-21T08:49:43.097 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.097 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.098 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.098 INFO:tasks.workunit.client.0.smithi043.stdout:469/600: open fd = 3 2024-04-21T08:49:43.098 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.098 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.098 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.098 INFO:tasks.workunit.client.0.smithi043.stdout:470/600: open fd = 3 2024-04-21T08:49:43.098 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.098 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.098 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.098 INFO:tasks.workunit.client.0.smithi043.stdout:471/600: open fd = 3 2024-04-21T08:49:43.098 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.098 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.098 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.098 INFO:tasks.workunit.client.0.smithi043.stdout:472/600: open fd = 3 2024-04-21T08:49:43.099 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.099 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.099 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.099 INFO:tasks.workunit.client.0.smithi043.stdout:473/600: open fd = 3 2024-04-21T08:49:43.099 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.099 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.099 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.099 INFO:tasks.workunit.client.0.smithi043.stdout:474/600: open fd = 3 2024-04-21T08:49:43.099 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.099 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.099 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.099 INFO:tasks.workunit.client.0.smithi043.stdout:475/600: open fd = 3 2024-04-21T08:49:43.099 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.099 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.099 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.100 INFO:tasks.workunit.client.0.smithi043.stdout:476/600: open fd = 3 2024-04-21T08:49:43.100 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.100 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.100 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.100 INFO:tasks.workunit.client.0.smithi043.stdout:477/600: open fd = 3 2024-04-21T08:49:43.100 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.100 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.100 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.100 INFO:tasks.workunit.client.0.smithi043.stdout:478/600: open fd = 3 2024-04-21T08:49:43.100 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.100 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.100 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.100 INFO:tasks.workunit.client.0.smithi043.stdout:479/600: open fd = 3 2024-04-21T08:49:43.100 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.101 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.101 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.101 INFO:tasks.workunit.client.0.smithi043.stdout:480/600: open fd = 3 2024-04-21T08:49:43.101 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.101 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.101 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.101 INFO:tasks.workunit.client.0.smithi043.stdout:481/600: open fd = 3 2024-04-21T08:49:43.101 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.101 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.101 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.101 INFO:tasks.workunit.client.0.smithi043.stdout:482/600: open fd = 3 2024-04-21T08:49:43.101 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.101 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.101 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.101 INFO:tasks.workunit.client.0.smithi043.stdout:483/600: open fd = 3 2024-04-21T08:49:43.101 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.102 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.102 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.102 INFO:tasks.workunit.client.0.smithi043.stdout:484/600: open fd = 3 2024-04-21T08:49:43.102 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.102 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.102 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.102 INFO:tasks.workunit.client.0.smithi043.stdout:485/600: open fd = 3 2024-04-21T08:49:43.102 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.102 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.102 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.102 INFO:tasks.workunit.client.0.smithi043.stdout:486/600: open fd = 3 2024-04-21T08:49:43.102 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.103 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.103 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.103 INFO:tasks.workunit.client.0.smithi043.stdout:487/600: open fd = 3 2024-04-21T08:49:43.103 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.103 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.104 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.104 INFO:tasks.workunit.client.0.smithi043.stdout:488/600: open fd = 3 2024-04-21T08:49:43.104 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.104 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.104 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.104 INFO:tasks.workunit.client.0.smithi043.stdout:489/600: open fd = 3 2024-04-21T08:49:43.104 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.105 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.105 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.105 INFO:tasks.workunit.client.0.smithi043.stdout:490/600: open fd = 3 2024-04-21T08:49:43.105 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.105 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.105 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.106 INFO:tasks.workunit.client.0.smithi043.stdout:491/600: open fd = 3 2024-04-21T08:49:43.106 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.106 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.106 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.106 INFO:tasks.workunit.client.0.smithi043.stdout:492/600: open fd = 3 2024-04-21T08:49:43.106 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.106 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.106 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.106 INFO:tasks.workunit.client.0.smithi043.stdout:493/600: open fd = 3 2024-04-21T08:49:43.107 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.107 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.107 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.107 INFO:tasks.workunit.client.0.smithi043.stdout:494/600: open fd = 3 2024-04-21T08:49:43.107 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.107 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.107 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.107 INFO:tasks.workunit.client.0.smithi043.stdout:495/600: open fd = 3 2024-04-21T08:49:43.107 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.107 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.107 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.108 INFO:tasks.workunit.client.0.smithi043.stdout:496/600: open fd = 3 2024-04-21T08:49:43.108 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.108 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.108 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.108 INFO:tasks.workunit.client.0.smithi043.stdout:497/600: open fd = 3 2024-04-21T08:49:43.108 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.108 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.108 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:43.108 INFO:tasks.workunit.client.0.smithi043.stdout:498/600: open fd = 3 2024-04-21T08:49:43.108 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.108 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:49:43.108 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:49:44.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:44 smithi175 ceph-mon[31920]: pgmap v336: 129 pgs: 129 active+clean; 2.6 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 18 op/s 2024-04-21T08:49:44.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:44 smithi112 ceph-mon[31085]: pgmap v336: 129 pgs: 129 active+clean; 2.6 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 18 op/s 2024-04-21T08:49:44.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:44 smithi043 ceph-mon[27978]: pgmap v336: 129 pgs: 129 active+clean; 2.6 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 18 op/s 2024-04-21T08:49:46.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:46 smithi175 ceph-mon[31920]: pgmap v337: 129 pgs: 129 active+clean; 2.6 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 18 op/s 2024-04-21T08:49:46.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:46 smithi112 ceph-mon[31085]: pgmap v337: 129 pgs: 129 active+clean; 2.6 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 18 op/s 2024-04-21T08:49:46.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:46 smithi043 ceph-mon[27978]: pgmap v337: 129 pgs: 129 active+clean; 2.6 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 18 op/s 2024-04-21T08:49:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.623+0000 7f9f528f1640 1 -- 172.21.15.43:0/200843260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f4c102b80 msgr2=0x7f9f4c106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.623+0000 7f9f528f1640 1 --2- 172.21.15.43:0/200843260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f4c102b80 0x7f9f4c106bc0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009900 tx=0x7f9f4002bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.623+0000 7f9f528f1640 1 -- 172.21.15.43:0/200843260 shutdown_connections 2024-04-21T08:49:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.623+0000 7f9f528f1640 1 --2- 172.21.15.43:0/200843260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f4c1072a0 0x7f9f4c10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.623+0000 7f9f528f1640 1 --2- 172.21.15.43:0/200843260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f4c102b80 0x7f9f4c106bc0 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.623+0000 7f9f528f1640 1 --2- 172.21.15.43:0/200843260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f4c1021d0 0x7f9f4c1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.623+0000 7f9f528f1640 1 -- 172.21.15.43:0/200843260 >> 172.21.15.43:0/200843260 conn(0x7f9f4c0793e0 msgr2=0x7f9f4c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.623+0000 7f9f528f1640 1 -- 172.21.15.43:0/200843260 shutdown_connections 2024-04-21T08:49:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.623+0000 7f9f528f1640 1 -- 172.21.15.43:0/200843260 wait complete. 2024-04-21T08:49:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.623+0000 7f9f528f1640 1 Processor -- start 2024-04-21T08:49:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.623+0000 7f9f528f1640 1 -- start start 2024-04-21T08:49:47.625 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.624+0000 7f9f528f1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f4c1021d0 0x7f9f4c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.624+0000 7f9f528f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f4c1072a0 0x7f9f4c198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.624+0000 7f9f528f1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f4c19cf00 0x7f9f4c19d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.624+0000 7f9f528f1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c1121c0 con 0x7f9f4c19cf00 2024-04-21T08:49:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.624+0000 7f9f528f1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c112040 con 0x7f9f4c1072a0 2024-04-21T08:49:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.624+0000 7f9f528f1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c112340 con 0x7f9f4c1021d0 2024-04-21T08:49:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.624+0000 7f9f50e8e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f4c19cf00 0x7f9f4c19d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.624+0000 7f9f50e8e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f4c19cf00 0x7f9f4c19d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47604/0 (socket says 172.21.15.43:47604) 2024-04-21T08:49:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.624+0000 7f9f50e8e640 1 -- 172.21.15.43:0/3541992748 learned_addr learned my addr 172.21.15.43:0/3541992748 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.624+0000 7f9f4b7fe640 1 --2- 172.21.15.43:0/3541992748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f4c1072a0 0x7f9f4c198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.624+0000 7f9f4bfff640 1 --2- 172.21.15.43:0/3541992748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f4c1021d0 0x7f9f4c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.624+0000 7f9f50e8e640 1 -- 172.21.15.43:0/3541992748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f4c1021d0 msgr2=0x7f9f4c198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.624+0000 7f9f50e8e640 1 --2- 172.21.15.43:0/3541992748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f4c1021d0 0x7f9f4c198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.624+0000 7f9f50e8e640 1 -- 172.21.15.43:0/3541992748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f4c1072a0 msgr2=0x7f9f4c198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.624+0000 7f9f50e8e640 1 --2- 172.21.15.43:0/3541992748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f4c1072a0 0x7f9f4c198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.624+0000 7f9f50e8e640 1 -- 172.21.15.43:0/3541992748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4c19d9f0 con 0x7f9f4c19cf00 2024-04-21T08:49:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.625+0000 7f9f50e8e640 1 --2- 172.21.15.43:0/3541992748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f4c19cf00 0x7f9f4c19d3b0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400cab0 tx=0x7f9f4400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.625+0000 7f9f497fa640 1 -- 172.21.15.43:0/3541992748 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4400ae20 con 0x7f9f4c19cf00 2024-04-21T08:49:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.625+0000 7f9f497fa640 1 -- 172.21.15.43:0/3541992748 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f4400f900 con 0x7f9f4c19cf00 2024-04-21T08:49:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.625+0000 7f9f528f1640 1 -- 172.21.15.43:0/3541992748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4c19dce0 con 0x7f9f4c19cf00 2024-04-21T08:49:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.625+0000 7f9f497fa640 1 -- 172.21.15.43:0/3541992748 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f440189f0 con 0x7f9f4c19cf00 2024-04-21T08:49:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.625+0000 7f9f528f1640 1 -- 172.21.15.43:0/3541992748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f4c1a5450 con 0x7f9f4c19cf00 2024-04-21T08:49:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.627+0000 7f9f497fa640 1 -- 172.21.15.43:0/3541992748 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9f4400fab0 con 0x7f9f4c19cf00 2024-04-21T08:49:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.628+0000 7f9f555b7740 1 Processor -- start 2024-04-21T08:49:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.628+0000 7f9f555b7740 1 -- start start 2024-04-21T08:49:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.628+0000 7f9f555b7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1c42b48d0 0x55d1c42b3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.628+0000 7f9f555b7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42b3b60 0x55d1c42b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.628+0000 7f9f555b7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1c42c14f0 0x55d1c42c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.628+0000 7f9f555b7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d1c42188d0 con 0x55d1c42b3b60 2024-04-21T08:49:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.628+0000 7f9f555b7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d1c41b5e80 con 0x55d1c42b48d0 2024-04-21T08:49:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.628+0000 7f9f555b7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d1c4214890 con 0x55d1c42c14f0 2024-04-21T08:49:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.628+0000 7f9f497fa640 1 --2- 172.21.15.43:0/3541992748 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9f340a3dc0 0x7f9f340a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.628+0000 7f9f4bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42b3b60 0x55d1c42b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.628+0000 7f9f4bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42b3b60 0x55d1c42b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47606/0 (socket says 172.21.15.43:47606) 2024-04-21T08:49:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.628+0000 7f9f4bfff640 1 -- 172.21.15.43:0/257251560 learned_addr learned my addr 172.21.15.43:0/257251560 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.628+0000 7f9f50e8e640 1 --2- 172.21.15.43:0/3541992748 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9f340a3dc0 0x7f9f340a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.628+0000 7f9f4b7fe640 1 --2- 172.21.15.43:0/257251560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1c42b48d0 0x55d1c42b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.628+0000 7f9f4b7fe640 1 --2- 172.21.15.43:0/257251560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1c42c14f0 0x55d1c42c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.628+0000 7f9f497fa640 1 -- 172.21.15.43:0/3541992748 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f9f440d13a0 con 0x7f9f4c19cf00 2024-04-21T08:49:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.629+0000 7f9f4bfff640 1 -- 172.21.15.43:0/257251560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1c42c14f0 msgr2=0x55d1c42c38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.629+0000 7f9f4bfff640 1 --2- 172.21.15.43:0/257251560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1c42c14f0 0x55d1c42c38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.629+0000 7f9f4bfff640 1 -- 172.21.15.43:0/257251560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1c42b48d0 msgr2=0x55d1c42b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.629+0000 7f9f4bfff640 1 --2- 172.21.15.43:0/257251560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1c42b48d0 0x55d1c42b3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.629+0000 7f9f4bfff640 1 -- 172.21.15.43:0/257251560 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1c41d7320 con 0x55d1c42b3b60 2024-04-21T08:49:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.629+0000 7f9f4b7fe640 1 --2- 172.21.15.43:0/257251560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1c42c14f0 0x55d1c42c38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.629+0000 7f9f4bfff640 1 --2- 172.21.15.43:0/257251560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42b3b60 0x55d1c42b3fe0 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009c80 tx=0x7f9f3c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9590ba184b3bc15 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.629+0000 7f9f50e8e640 1 --2- 172.21.15.43:0/3541992748 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9f340a3dc0 0x7f9f340a6280 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9f440ce780 tx=0x7f9f4406c800 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.629+0000 7f9f327fc640 1 -- 172.21.15.43:0/257251560 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c012450 con 0x55d1c42b3b60 2024-04-21T08:49:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.629+0000 7f9f327fc640 1 -- 172.21.15.43:0/257251560 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f3c012aa0 con 0x55d1c42b3b60 2024-04-21T08:49:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.629+0000 7f9f327fc640 1 -- 172.21.15.43:0/257251560 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c00fae0 con 0x55d1c42b3b60 2024-04-21T08:49:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.630+0000 7f9f555b7740 1 -- 172.21.15.43:0/257251560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42b3b60 msgr2=0x55d1c42b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.630+0000 7f9f555b7740 1 --2- 172.21.15.43:0/257251560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42b3b60 0x55d1c42b3fe0 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009c80 tx=0x7f9f3c0117b0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.630+0000 7f9f555b7740 1 -- 172.21.15.43:0/257251560 shutdown_connections 2024-04-21T08:49:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.630+0000 7f9f555b7740 1 --2- 172.21.15.43:0/257251560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1c42c14f0 0x55d1c42c38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.630+0000 7f9f555b7740 1 --2- 172.21.15.43:0/257251560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42b3b60 0x55d1c42b3fe0 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.630+0000 7f9f555b7740 1 --2- 172.21.15.43:0/257251560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1c42b48d0 0x55d1c42b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.630+0000 7f9f555b7740 1 -- 172.21.15.43:0/257251560 >> 172.21.15.43:0/257251560 conn(0x55d1c427ac90 msgr2=0x55d1c42ae850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.630+0000 7f9f555b7740 1 -- 172.21.15.43:0/257251560 shutdown_connections 2024-04-21T08:49:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.630+0000 7f9f555b7740 1 -- 172.21.15.43:0/257251560 wait complete. 2024-04-21T08:49:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.630+0000 7f9f555b7740 1 -- start start 2024-04-21T08:49:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.630+0000 7f9f555b7740 1 Processor -- start 2024-04-21T08:49:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.631+0000 7f9f555b7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1c42b3b60 0x55d1c42c0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.631+0000 7f9f555b7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1c42b48d0 0x55d1c42be1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.631+0000 7f9f555b7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42c14f0 0x55d1c42be740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.631+0000 7f9f555b7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d1c42c4220 con 0x55d1c42c14f0 2024-04-21T08:49:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.631+0000 7f9f555b7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d1c41a0eb0 con 0x55d1c42b48d0 2024-04-21T08:49:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.631+0000 7f9f555b7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d1c42c43a0 con 0x55d1c42b3b60 2024-04-21T08:49:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.631+0000 7f9f555b7740 10 client.0 _ll_register_callbacks cb 0x55d1c41cdb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.631+0000 7f9f4b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42c14f0 0x55d1c42be740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.631+0000 7f9f4b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42c14f0 0x55d1c42be740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47624/0 (socket says 172.21.15.43:47624) 2024-04-21T08:49:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.631+0000 7f9f4b7fe640 1 -- 172.21.15.43:0/2937251426 learned_addr learned my addr 172.21.15.43:0/2937251426 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.631+0000 7f9f4bfff640 1 --2- 172.21.15.43:0/2937251426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1c42b48d0 0x55d1c42be1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.631+0000 7f9f4b7fe640 1 --2- 172.21.15.43:0/2937251426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1c42b3b60 0x55d1c42c0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.631+0000 7f9f4b7fe640 1 -- 172.21.15.43:0/2937251426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1c42b3b60 msgr2=0x55d1c42c0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.631+0000 7f9f4b7fe640 1 --2- 172.21.15.43:0/2937251426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1c42b3b60 0x55d1c42c0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.631+0000 7f9f4b7fe640 1 -- 172.21.15.43:0/2937251426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1c42b48d0 msgr2=0x55d1c42be1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.631+0000 7f9f4b7fe640 1 --2- 172.21.15.43:0/2937251426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1c42b48d0 0x55d1c42be1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.631+0000 7f9f4b7fe640 1 -- 172.21.15.43:0/2937251426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1c3f2b210 con 0x55d1c42c14f0 2024-04-21T08:49:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.631+0000 7f9f4b7fe640 1 --2- 172.21.15.43:0/2937251426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42c14f0 0x55d1c42be740 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009900 tx=0x7f9f4002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38b213518387dad server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.631+0000 7f9f2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.632+0000 7f9f2b7fe640 1 -- 172.21.15.43:0/2937251426 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f400050f0 con 0x55d1c42c14f0 2024-04-21T08:49:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.632+0000 7f9f2b7fe640 1 -- 172.21.15.43:0/2937251426 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f9f4003a040 con 0x55d1c42c14f0 2024-04-21T08:49:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.632+0000 7f9f555b7740 1 -- 172.21.15.43:0/2937251426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f3c012aa0 con 0x55d1c42c14f0 2024-04-21T08:49:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.632+0000 7f9f2b7fe640 1 -- 172.21.15.43:0/2937251426 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9f40054050 con 0x55d1c42c14f0 2024-04-21T08:49:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.632+0000 7f9f555b7740 10 client.20820 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.632+0000 7f9f555b7740 1 -- 172.21.15.43:0/2937251426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d1c3ff7620 con 0x55d1c42c14f0 2024-04-21T08:49:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.632+0000 7f9f2b7fe640 1 -- 172.21.15.43:0/2937251426 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f3c012aa0 con 0x55d1c42c14f0 2024-04-21T08:49:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.632+0000 7f9f2b7fe640 1 -- 172.21.15.43:0/2937251426 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9f400036f0 con 0x55d1c42c14f0 2024-04-21T08:49:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.632+0000 7f9f555b7740 10 client.20820 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.632+0000 7f9f555b7740 10 client.20820 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.632+0000 7f9f555b7740 20 client.20820 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.632+0000 7f9f555b7740 1 --2- 172.21.15.43:0/2937251426 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d1c42d13e0 0x55d1c42d37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.632+0000 7f9f555b7740 4 client.20820 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.632+0000 7f9f555b7740 1 -- 172.21.15.43:0/2937251426 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1c422c880 con 0x55d1c42d13e0 2024-04-21T08:49:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.633+0000 7f9f4bfff640 1 --2- 172.21.15.43:0/2937251426 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d1c42d13e0 0x55d1c42d37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.633+0000 7f9f2b7fe640 10 client.20820 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.633+0000 7f9f4bfff640 1 --2- 172.21.15.43:0/2937251426 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d1c42d13e0 0x55d1c42d37d0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3925c03ed64a8d3 server_cookie=2773b1c99b6c2c83 in_seq=0 out_seq=0 2024-04-21T08:49:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.634+0000 7f9f2b7fe640 1 -- 172.21.15.43:0/2937251426 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d1c422c880 con 0x55d1c42d13e0 2024-04-21T08:49:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.634+0000 7f9f2b7fe640 10 client.20820 handle_command_reply: tid=0 2024-04-21T08:49:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.634+0000 7f9f555b7740 1 client.20820 shutdown 2024-04-21T08:49:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.634+0000 7f9f555b7740 1 -- 172.21.15.43:0/2937251426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42c14f0 msgr2=0x55d1c42be740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.634+0000 7f9f555b7740 1 --2- 172.21.15.43:0/2937251426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42c14f0 0x55d1c42be740 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009900 tx=0x7f9f4002bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.634+0000 7f9f555b7740 1 -- 172.21.15.43:0/2937251426 shutdown_connections 2024-04-21T08:49:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.634+0000 7f9f555b7740 1 --2- 172.21.15.43:0/2937251426 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d1c42d13e0 0x55d1c42d37d0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.634+0000 7f9f555b7740 1 --2- 172.21.15.43:0/2937251426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42c14f0 0x55d1c42be740 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.634+0000 7f9f555b7740 1 --2- 172.21.15.43:0/2937251426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1c42b48d0 0x55d1c42be1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.634+0000 7f9f2b7fe640 0 client.20820 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.634+0000 7f9f555b7740 1 --2- 172.21.15.43:0/2937251426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1c42b3b60 0x55d1c42c0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.634+0000 7f9f555b7740 1 -- 172.21.15.43:0/2937251426 >> 172.21.15.43:0/2937251426 conn(0x55d1c427ac90 msgr2=0x55d1c42b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.634+0000 7f9f555b7740 1 -- 172.21.15.43:0/2937251426 shutdown_connections 2024-04-21T08:49:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.634+0000 7f9f555b7740 1 -- 172.21.15.43:0/2937251426 wait complete. 2024-04-21T08:49:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.634+0000 7f9f555b7740 20 client.20820 trim_cache size 0 max 16384 2024-04-21T08:49:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.650+0000 7f9f555b7740 1 Processor -- start 2024-04-21T08:49:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.650+0000 7f9f555b7740 1 -- start start 2024-04-21T08:49:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.650+0000 7f9f555b7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1c42b3b60 0x55d1c42cd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.650+0000 7f9f555b7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1c42b48d0 0x55d1c42bd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.650+0000 7f9f555b7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42c14f0 0x55d1c42d90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.650+0000 7f9f555b7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c1ad200 con 0x55d1c42c14f0 2024-04-21T08:49:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.650+0000 7f9f555b7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c1aed40 con 0x55d1c42b3b60 2024-04-21T08:49:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.650+0000 7f9f555b7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c1ad8d0 con 0x55d1c42b48d0 2024-04-21T08:49:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.650+0000 7f9f4b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42c14f0 0x55d1c42d90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.650+0000 7f9f4b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42c14f0 0x55d1c42d90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47642/0 (socket says 172.21.15.43:47642) 2024-04-21T08:49:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.650+0000 7f9f4b7fe640 1 -- 172.21.15.43:0/3911241061 learned_addr learned my addr 172.21.15.43:0/3911241061 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.650+0000 7f9f4bfff640 1 --2- 172.21.15.43:0/3911241061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1c42b48d0 0x55d1c42bd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.650+0000 7f9f4b7fe640 1 --2- 172.21.15.43:0/3911241061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1c42b3b60 0x55d1c42cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.650+0000 7f9f4b7fe640 1 -- 172.21.15.43:0/3911241061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1c42b48d0 msgr2=0x55d1c42bd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.650+0000 7f9f4b7fe640 1 --2- 172.21.15.43:0/3911241061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1c42b48d0 0x55d1c42bd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.650+0000 7f9f4b7fe640 1 -- 172.21.15.43:0/3911241061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1c42b3b60 msgr2=0x55d1c42cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.650+0000 7f9f4b7fe640 1 --2- 172.21.15.43:0/3911241061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1c42b3b60 0x55d1c42cd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.650+0000 7f9f4b7fe640 1 -- 172.21.15.43:0/3911241061 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1c42b2e30 con 0x55d1c42c14f0 2024-04-21T08:49:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.650+0000 7f9f4b7fe640 1 --2- 172.21.15.43:0/3911241061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42c14f0 0x55d1c42d90c0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009620 tx=0x7f9f40005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb34b4d8b5fdfc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.651+0000 7f9f31ffb640 1 -- 172.21.15.43:0/3911241061 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4002ca80 con 0x55d1c42c14f0 2024-04-21T08:49:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.651+0000 7f9f31ffb640 1 -- 172.21.15.43:0/3911241061 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f400054a0 con 0x55d1c42c14f0 2024-04-21T08:49:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.651+0000 7f9f31ffb640 1 -- 172.21.15.43:0/3911241061 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40004b10 con 0x55d1c42c14f0 2024-04-21T08:49:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.651+0000 7f9f555b7740 1 -- 172.21.15.43:0/3911241061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42c14f0 msgr2=0x55d1c42d90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.651+0000 7f9f555b7740 1 --2- 172.21.15.43:0/3911241061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42c14f0 0x55d1c42d90c0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009620 tx=0x7f9f40005fd0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.651+0000 7f9f555b7740 1 -- 172.21.15.43:0/3911241061 shutdown_connections 2024-04-21T08:49:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.651+0000 7f9f555b7740 1 --2- 172.21.15.43:0/3911241061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42c14f0 0x55d1c42d90c0 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.651+0000 7f9f555b7740 1 --2- 172.21.15.43:0/3911241061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1c42b48d0 0x55d1c42bd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.651+0000 7f9f555b7740 1 --2- 172.21.15.43:0/3911241061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1c42b3b60 0x55d1c42cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.651+0000 7f9f555b7740 1 -- 172.21.15.43:0/3911241061 >> 172.21.15.43:0/3911241061 conn(0x55d1c427ac90 msgr2=0x55d1c42ae380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.651+0000 7f9f555b7740 1 -- 172.21.15.43:0/3911241061 shutdown_connections 2024-04-21T08:49:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.651+0000 7f9f555b7740 1 -- 172.21.15.43:0/3911241061 wait complete. 2024-04-21T08:49:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.652+0000 7f9f555b7740 1 -- start start 2024-04-21T08:49:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.652+0000 7f9f555b7740 1 Processor -- start 2024-04-21T08:49:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.652+0000 7f9f555b7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42b3b60 0x55d1c42c5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.652+0000 7f9f555b7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1c42b48d0 0x55d1c42c5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.652+0000 7f9f555b7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1c42c14f0 0x55d1c42c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.652+0000 7f9f555b7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c1ae670 con 0x55d1c42b3b60 2024-04-21T08:49:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.652+0000 7f9f555b7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c1adfa0 con 0x55d1c42c14f0 2024-04-21T08:49:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.652+0000 7f9f555b7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c1ab200 con 0x55d1c42b48d0 2024-04-21T08:49:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.652+0000 7f9f555b7740 10 client.0 _ll_register_callbacks cb 0x7f9f40009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.652+0000 7f9f4b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42b3b60 0x55d1c42c5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.652+0000 7f9f4b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42b3b60 0x55d1c42c5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47660/0 (socket says 172.21.15.43:47660) 2024-04-21T08:49:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.652+0000 7f9f4b7fe640 1 -- 172.21.15.43:0/3747936163 learned_addr learned my addr 172.21.15.43:0/3747936163 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:49:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.653+0000 7f9f4bfff640 1 --2- 172.21.15.43:0/3747936163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1c42b48d0 0x55d1c42c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.653+0000 7f9f4b7fe640 1 -- 172.21.15.43:0/3747936163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1c42b48d0 msgr2=0x55d1c42c5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.653+0000 7f9f4b7fe640 1 --2- 172.21.15.43:0/3747936163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1c42b48d0 0x55d1c42c5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.653+0000 7f9f4b7fe640 1 -- 172.21.15.43:0/3747936163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1c42c14f0 msgr2=0x55d1c42c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.653+0000 7f9f4b7fe640 1 --2- 172.21.15.43:0/3747936163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1c42c14f0 0x55d1c42c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.653+0000 7f9f4b7fe640 1 -- 172.21.15.43:0/3747936163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1c42a9f30 con 0x55d1c42b3b60 2024-04-21T08:49:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.653+0000 7f9f4b7fe640 1 --2- 172.21.15.43:0/3747936163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42b3b60 0x55d1c42c5010 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f9f4002c420 tx=0x7f9f40009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1479238f68370ca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.653+0000 7f9f2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:49:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.653+0000 7f9f2b7fe640 1 -- 172.21.15.43:0/3747936163 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40005650 con 0x55d1c42b3b60 2024-04-21T08:49:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.653+0000 7f9f555b7740 1 -- 172.21.15.43:0/3747936163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f400054a0 con 0x55d1c42b3b60 2024-04-21T08:49:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.653+0000 7f9f2b7fe640 1 -- 172.21.15.43:0/3747936163 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f9f40003800 con 0x55d1c42b3b60 2024-04-21T08:49:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.653+0000 7f9f2b7fe640 1 -- 172.21.15.43:0/3747936163 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9f400046c0 con 0x55d1c42b3b60 2024-04-21T08:49:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.653+0000 7f9f555b7740 10 client.20826 fetch_fsmap learned FSMap version 30 2024-04-21T08:49:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.653+0000 7f9f555b7740 1 -- 172.21.15.43:0/3747936163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d1c4337990 con 0x55d1c42b3b60 2024-04-21T08:49:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.653+0000 7f9f2b7fe640 1 -- 172.21.15.43:0/3747936163 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f400054a0 con 0x55d1c42b3b60 2024-04-21T08:49:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.654+0000 7f9f2b7fe640 1 -- 172.21.15.43:0/3747936163 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9f400348d0 con 0x55d1c42b3b60 2024-04-21T08:49:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.654+0000 7f9f555b7740 10 client.20826 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:49:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.654+0000 7f9f555b7740 10 client.20826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.654+0000 7f9f555b7740 20 client.20826 populate_metadata read hostname 'smithi043' 2024-04-21T08:49:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.654+0000 7f9f555b7740 1 --2- 172.21.15.43:0/3747936163 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d1c4337f10 0x55d1c433a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.654+0000 7f9f555b7740 4 client.20826 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:49:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.654+0000 7f9f555b7740 1 -- 172.21.15.43:0/3747936163 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d1c41b83d0 con 0x55d1c4337f10 2024-04-21T08:49:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.654+0000 7f9f4bfff640 1 --2- 172.21.15.43:0/3747936163 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d1c4337f10 0x55d1c433a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.654+0000 7f9f4bfff640 1 --2- 172.21.15.43:0/3747936163 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d1c4337f10 0x55d1c433a300 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85938033973fbdf2 server_cookie=d216a4a650d0339e in_seq=0 out_seq=0 2024-04-21T08:49:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.654+0000 7f9f2b7fe640 10 client.20826 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.670+0000 7f9f2b7fe640 1 -- 172.21.15.43:0/3747936163 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d1c41b83d0 con 0x55d1c4337f10 2024-04-21T08:49:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.670+0000 7f9f2b7fe640 10 client.20826 handle_command_reply: tid=0 2024-04-21T08:49:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.670+0000 7f9f555b7740 1 client.20826 shutdown 2024-04-21T08:49:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.671+0000 7f9f555b7740 1 -- 172.21.15.43:0/3747936163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42b3b60 msgr2=0x55d1c42c5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.671+0000 7f9f555b7740 1 --2- 172.21.15.43:0/3747936163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42b3b60 0x55d1c42c5010 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f9f4002c420 tx=0x7f9f40009590 comp rx=0 tx=0).stop 2024-04-21T08:49:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.671+0000 7f9f555b7740 1 -- 172.21.15.43:0/3747936163 shutdown_connections 2024-04-21T08:49:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.671+0000 7f9f555b7740 1 --2- 172.21.15.43:0/3747936163 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d1c4337f10 0x55d1c433a300 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.671+0000 7f9f555b7740 1 --2- 172.21.15.43:0/3747936163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1c42c14f0 0x55d1c42c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.671+0000 7f9f555b7740 1 --2- 172.21.15.43:0/3747936163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d1c42b48d0 0x55d1c42c5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.671+0000 7f9f2b7fe640 0 client.20826 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:49:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.671+0000 7f9f555b7740 1 --2- 172.21.15.43:0/3747936163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1c42b3b60 0x55d1c42c5010 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.671+0000 7f9f555b7740 1 -- 172.21.15.43:0/3747936163 >> 172.21.15.43:0/3747936163 conn(0x55d1c427ac90 msgr2=0x55d1c42bc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.671+0000 7f9f555b7740 1 -- 172.21.15.43:0/3747936163 shutdown_connections 2024-04-21T08:49:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.671+0000 7f9f555b7740 1 -- 172.21.15.43:0/3747936163 wait complete. 2024-04-21T08:49:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.671+0000 7f9f555b7740 20 client.20826 trim_cache size 0 max 16384 2024-04-21T08:49:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:49:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:49:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:49:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:49:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.671+0000 7f9f528f1640 1 -- 172.21.15.43:0/3541992748 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9f340a3dc0 msgr2=0x7f9f340a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.671+0000 7f9f528f1640 1 --2- 172.21.15.43:0/3541992748 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9f340a3dc0 0x7f9f340a6280 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9f440ce780 tx=0x7f9f4406c800 comp rx=0 tx=0).stop 2024-04-21T08:49:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.671+0000 7f9f528f1640 1 -- 172.21.15.43:0/3541992748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f4c19cf00 msgr2=0x7f9f4c19d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.671+0000 7f9f528f1640 1 --2- 172.21.15.43:0/3541992748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f4c19cf00 0x7f9f4c19d3b0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400cab0 tx=0x7f9f4400cf70 comp rx=0 tx=0).stop 2024-04-21T08:49:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.672+0000 7f9f528f1640 1 -- 172.21.15.43:0/3541992748 shutdown_connections 2024-04-21T08:49:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.672+0000 7f9f528f1640 1 --2- 172.21.15.43:0/3541992748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f4c19cf00 0x7f9f4c19d3b0 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.672+0000 7f9f528f1640 1 --2- 172.21.15.43:0/3541992748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f4c1072a0 0x7f9f4c198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.672+0000 7f9f528f1640 1 --2- 172.21.15.43:0/3541992748 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9f340a3dc0 0x7f9f340a6280 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.672+0000 7f9f528f1640 1 --2- 172.21.15.43:0/3541992748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f4c1021d0 0x7f9f4c198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.672+0000 7f9f528f1640 1 -- 172.21.15.43:0/3541992748 >> 172.21.15.43:0/3541992748 conn(0x7f9f4c0793e0 msgr2=0x7f9f4c0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.672+0000 7f9f528f1640 1 -- 172.21.15.43:0/3541992748 shutdown_connections 2024-04-21T08:49:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:49:47.672+0000 7f9f528f1640 1 -- 172.21.15.43:0/3541992748 wait complete. 2024-04-21T08:49:48.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:48 smithi175 ceph-mon[31920]: pgmap v338: 129 pgs: 129 active+clean; 2.7 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 24 op/s 2024-04-21T08:49:48.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:48 smithi112 ceph-mon[31085]: pgmap v338: 129 pgs: 129 active+clean; 2.7 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 24 op/s 2024-04-21T08:49:48.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:48 smithi043 ceph-mon[27978]: pgmap v338: 129 pgs: 129 active+clean; 2.7 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 24 op/s 2024-04-21T08:49:50.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:50 smithi175 ceph-mon[31920]: pgmap v339: 129 pgs: 129 active+clean; 2.7 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-21T08:49:50.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:50 smithi112 ceph-mon[31085]: pgmap v339: 129 pgs: 129 active+clean; 2.7 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-21T08:49:50.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:50 smithi043 ceph-mon[27978]: pgmap v339: 129 pgs: 129 active+clean; 2.7 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-21T08:49:52.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:52 smithi175 ceph-mon[31920]: pgmap v340: 129 pgs: 129 active+clean; 3.0 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:49:52.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:52 smithi112 ceph-mon[31085]: pgmap v340: 129 pgs: 129 active+clean; 3.0 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:49:52.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:52 smithi043 ceph-mon[27978]: pgmap v340: 129 pgs: 129 active+clean; 3.0 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:49:54.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:54 smithi175 ceph-mon[31920]: pgmap v341: 129 pgs: 129 active+clean; 3.0 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:49:54.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:54 smithi112 ceph-mon[31085]: pgmap v341: 129 pgs: 129 active+clean; 3.0 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:49:54.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:54 smithi043 ceph-mon[27978]: pgmap v341: 129 pgs: 129 active+clean; 3.0 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:49:56.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:56 smithi175 ceph-mon[31920]: pgmap v342: 129 pgs: 129 active+clean; 3.0 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:49:56.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:56 smithi112 ceph-mon[31085]: pgmap v342: 129 pgs: 129 active+clean; 3.0 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:49:56.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:56 smithi043 ceph-mon[27978]: pgmap v342: 129 pgs: 129 active+clean; 3.0 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:49:58.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:49:58 smithi175 ceph-mon[31920]: pgmap v343: 129 pgs: 129 active+clean; 3.2 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-21T08:49:58.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:49:58 smithi112 ceph-mon[31085]: pgmap v343: 129 pgs: 129 active+clean; 3.2 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-21T08:49:58.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:49:58 smithi043 ceph-mon[27978]: pgmap v343: 129 pgs: 129 active+clean; 3.2 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-21T08:50:00.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:00 smithi175 ceph-mon[31920]: pgmap v344: 129 pgs: 129 active+clean; 3.2 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:50:00.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:00 smithi175 ceph-mon[31920]: overall HEALTH_OK 2024-04-21T08:50:00.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:00 smithi112 ceph-mon[31085]: pgmap v344: 129 pgs: 129 active+clean; 3.2 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:50:00.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:00 smithi112 ceph-mon[31085]: overall HEALTH_OK 2024-04-21T08:50:00.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:00 smithi043 ceph-mon[27978]: pgmap v344: 129 pgs: 129 active+clean; 3.2 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:50:00.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:00 smithi043 ceph-mon[27978]: overall HEALTH_OK 2024-04-21T08:50:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.771+0000 7fc3f0fe6640 1 -- 172.21.15.43:0/3023302104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3ec100d60 msgr2=0x7fc3ec101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.771+0000 7fc3f0fe6640 1 --2- 172.21.15.43:0/3023302104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3ec100d60 0x7fc3ec101140 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009900 tx=0x7fc3dc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.772+0000 7fc3f0fe6640 1 -- 172.21.15.43:0/3023302104 shutdown_connections 2024-04-21T08:50:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.772+0000 7fc3f0fe6640 1 --2- 172.21.15.43:0/3023302104 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3ec10e910 0x7fc3ec110d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.772+0000 7fc3f0fe6640 1 --2- 172.21.15.43:0/3023302104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3ec101680 0x7fc3ec10e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.772+0000 7fc3f0fe6640 1 --2- 172.21.15.43:0/3023302104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3ec100d60 0x7fc3ec101140 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.772+0000 7fc3f0fe6640 1 -- 172.21.15.43:0/3023302104 >> 172.21.15.43:0/3023302104 conn(0x7fc3ec077f70 msgr2=0x7fc3ec078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.772+0000 7fc3f0fe6640 1 -- 172.21.15.43:0/3023302104 shutdown_connections 2024-04-21T08:50:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.772+0000 7fc3f0fe6640 1 -- 172.21.15.43:0/3023302104 wait complete. 2024-04-21T08:50:00.774 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.772+0000 7fc3f0fe6640 1 Processor -- start 2024-04-21T08:50:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.773+0000 7fc3f0fe6640 1 -- start start 2024-04-21T08:50:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.773+0000 7fc3f0fe6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3ec100d60 0x7fc3ec1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.773+0000 7fc3f0fe6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3ec101680 0x7fc3ec1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.773+0000 7fc3f0fe6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3ec10e910 0x7fc3ec19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.773+0000 7fc3f0fe6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec114430 con 0x7fc3ec101680 2024-04-21T08:50:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.773+0000 7fc3f0fe6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1142b0 con 0x7fc3ec10e910 2024-04-21T08:50:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.773+0000 7fc3f0fe6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1145b0 con 0x7fc3ec100d60 2024-04-21T08:50:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.773+0000 7fc3e3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3ec101680 0x7fc3ec1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.773+0000 7fc3e3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3ec101680 0x7fc3ec1a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36622/0 (socket says 172.21.15.43:36622) 2024-04-21T08:50:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.773+0000 7fc3e3fff640 1 -- 172.21.15.43:0/159307446 learned_addr learned my addr 172.21.15.43:0/159307446 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.773+0000 7fc3ead9d640 1 --2- 172.21.15.43:0/159307446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3ec10e910 0x7fc3ec19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.773+0000 7fc3ea59c640 1 --2- 172.21.15.43:0/159307446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3ec100d60 0x7fc3ec1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.773+0000 7fc3e3fff640 1 -- 172.21.15.43:0/159307446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3ec100d60 msgr2=0x7fc3ec1a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.773+0000 7fc3e3fff640 1 --2- 172.21.15.43:0/159307446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3ec100d60 0x7fc3ec1a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.773+0000 7fc3e3fff640 1 -- 172.21.15.43:0/159307446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3ec10e910 msgr2=0x7fc3ec19a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.773+0000 7fc3e3fff640 1 --2- 172.21.15.43:0/159307446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3ec10e910 0x7fc3ec19a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.773+0000 7fc3e3fff640 1 -- 172.21.15.43:0/159307446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ec19afe0 con 0x7fc3ec101680 2024-04-21T08:50:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.773+0000 7fc3e3fff640 1 --2- 172.21.15.43:0/159307446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3ec101680 0x7fc3ec1a0be0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400d950 tx=0x7fc3d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.774+0000 7fc3e37fe640 1 -- 172.21.15.43:0/159307446 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4017070 con 0x7fc3ec101680 2024-04-21T08:50:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.774+0000 7fc3e37fe640 1 -- 172.21.15.43:0/159307446 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3d4009c30 con 0x7fc3ec101680 2024-04-21T08:50:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.774+0000 7fc3f0fe6640 1 -- 172.21.15.43:0/159307446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ec19b2d0 con 0x7fc3ec101680 2024-04-21T08:50:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.774+0000 7fc3e37fe640 1 -- 172.21.15.43:0/159307446 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d400f620 con 0x7fc3ec101680 2024-04-21T08:50:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.774+0000 7fc3f0fe6640 1 -- 172.21.15.43:0/159307446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ec1a74f0 con 0x7fc3ec101680 2024-04-21T08:50:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.776+0000 7fc3e37fe640 1 -- 172.21.15.43:0/159307446 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc3d4010460 con 0x7fc3ec101680 2024-04-21T08:50:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.776+0000 7fc3f3bd3740 1 Processor -- start 2024-04-21T08:50:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.777+0000 7fc3f3bd3740 1 -- start start 2024-04-21T08:50:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.777+0000 7fc3e37fe640 1 --2- 172.21.15.43:0/159307446 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc3d80a3dc0 0x7fc3d80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.777+0000 7fc3f3bd3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609bfb6b8d0 0x5609bfb6a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.777+0000 7fc3f3bd3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609bfb6ab60 0x5609bfb6afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.777+0000 7fc3f3bd3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb740e0 0x5609bfb764d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.777+0000 7fc3f3bd3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5609bfacf8d0 con 0x5609bfb740e0 2024-04-21T08:50:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.777+0000 7fc3f3bd3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5609bfa6ce80 con 0x5609bfb6ab60 2024-04-21T08:50:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.777+0000 7fc3f3bd3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5609bfacb890 con 0x5609bfb6b8d0 2024-04-21T08:50:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.777+0000 7fc3ead9d640 1 --2- 172.21.15.43:0/159307446 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc3d80a3dc0 0x7fc3d80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.777+0000 7fc3e37fe640 1 -- 172.21.15.43:0/159307446 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fc3d40d0810 con 0x7fc3ec101680 2024-04-21T08:50:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.777+0000 7fc3ead9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb740e0 0x5609bfb764d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.777+0000 7fc3ead9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb740e0 0x5609bfb764d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36636/0 (socket says 172.21.15.43:36636) 2024-04-21T08:50:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.777+0000 7fc3ead9d640 1 -- 172.21.15.43:0/1990248665 learned_addr learned my addr 172.21.15.43:0/1990248665 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.777+0000 7fc3ea59c640 1 --2- 172.21.15.43:0/1990248665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609bfb6b8d0 0x5609bfb6a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.777+0000 7fc3e3fff640 1 --2- 172.21.15.43:0/1990248665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609bfb6ab60 0x5609bfb6afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.778+0000 7fc3ea59c640 1 -- 172.21.15.43:0/1990248665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609bfb6ab60 msgr2=0x5609bfb6afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.778+0000 7fc3ea59c640 1 --2- 172.21.15.43:0/1990248665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609bfb6ab60 0x5609bfb6afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.778+0000 7fc3ea59c640 1 -- 172.21.15.43:0/1990248665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb740e0 msgr2=0x5609bfb764d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.778+0000 7fc3ea59c640 1 --2- 172.21.15.43:0/1990248665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb740e0 0x5609bfb764d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.778+0000 7fc3ea59c640 1 -- 172.21.15.43:0/1990248665 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609bfa8e320 con 0x5609bfb6b8d0 2024-04-21T08:50:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.778+0000 7fc3ead9d640 1 --2- 172.21.15.43:0/1990248665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb740e0 0x5609bfb764d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:50:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.778+0000 7fc3ead9d640 1 --2- 172.21.15.43:0/159307446 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc3d80a3dc0 0x7fc3d80a6280 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4011c10 tx=0x7fc3e4009ed0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.778+0000 7fc3ea59c640 1 --2- 172.21.15.43:0/1990248665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609bfb6b8d0 0x5609bfb6a620 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc000c00 tx=0x7fc3dc02c610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e324c038770034f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.778+0000 7fc3e0ff9640 1 -- 172.21.15.43:0/1990248665 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc045020 con 0x5609bfb6b8d0 2024-04-21T08:50:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.778+0000 7fc3e0ff9640 1 -- 172.21.15.43:0/1990248665 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3dc02c7e0 con 0x5609bfb6b8d0 2024-04-21T08:50:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.778+0000 7fc3e0ff9640 1 -- 172.21.15.43:0/1990248665 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc0055a0 con 0x5609bfb6b8d0 2024-04-21T08:50:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.779+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/1990248665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609bfb6b8d0 msgr2=0x5609bfb6a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.779+0000 7fc3f3bd3740 1 --2- 172.21.15.43:0/1990248665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609bfb6b8d0 0x5609bfb6a620 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc000c00 tx=0x7fc3dc02c610 comp rx=0 tx=0).stop 2024-04-21T08:50:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.779+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/1990248665 shutdown_connections 2024-04-21T08:50:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.779+0000 7fc3f3bd3740 1 --2- 172.21.15.43:0/1990248665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb740e0 0x5609bfb764d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.779+0000 7fc3f3bd3740 1 --2- 172.21.15.43:0/1990248665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609bfb6ab60 0x5609bfb6afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.779+0000 7fc3f3bd3740 1 --2- 172.21.15.43:0/1990248665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609bfb6b8d0 0x5609bfb6a620 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.779+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/1990248665 >> 172.21.15.43:0/1990248665 conn(0x5609bfb31c90 msgr2=0x5609bfb65850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.779+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/1990248665 shutdown_connections 2024-04-21T08:50:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.779+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/1990248665 wait complete. 2024-04-21T08:50:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.779+0000 7fc3f3bd3740 1 -- start start 2024-04-21T08:50:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.779+0000 7fc3f3bd3740 1 Processor -- start 2024-04-21T08:50:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.780+0000 7fc3f3bd3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb6ab60 0x5609bfb73a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.780+0000 7fc3f3bd3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609bfb6b8d0 0x5609bfb70de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.780+0000 7fc3f3bd3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609bfb740e0 0x5609bfb71340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.780+0000 7fc3f3bd3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5609bfb76db0 con 0x5609bfb6ab60 2024-04-21T08:50:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.780+0000 7fc3f3bd3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5609bfa57eb0 con 0x5609bfb740e0 2024-04-21T08:50:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.780+0000 7fc3f3bd3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5609bfb76f30 con 0x5609bfb6b8d0 2024-04-21T08:50:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.780+0000 7fc3f3bd3740 10 client.0 _ll_register_callbacks cb 0x5609bfa84b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.780+0000 7fc3ea59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb6ab60 0x5609bfb73a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.780+0000 7fc3ea59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb6ab60 0x5609bfb73a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36644/0 (socket says 172.21.15.43:36644) 2024-04-21T08:50:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.780+0000 7fc3e3fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609bfb6b8d0 0x5609bfb70de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.780+0000 7fc3ea59c640 1 -- 172.21.15.43:0/3122229076 learned_addr learned my addr 172.21.15.43:0/3122229076 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.780+0000 7fc3ead9d640 1 --2- 172.21.15.43:0/3122229076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609bfb740e0 0x5609bfb71340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.780+0000 7fc3ea59c640 1 -- 172.21.15.43:0/3122229076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609bfb6b8d0 msgr2=0x5609bfb70de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.780+0000 7fc3ea59c640 1 --2- 172.21.15.43:0/3122229076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609bfb6b8d0 0x5609bfb70de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.780+0000 7fc3ea59c640 1 -- 172.21.15.43:0/3122229076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609bfb740e0 msgr2=0x5609bfb71340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.780+0000 7fc3ea59c640 1 --2- 172.21.15.43:0/3122229076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609bfb740e0 0x5609bfb71340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.780+0000 7fc3ea59c640 1 -- 172.21.15.43:0/3122229076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609bf7e2210 con 0x5609bfb6ab60 2024-04-21T08:50:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.780+0000 7fc3ea59c640 1 --2- 172.21.15.43:0/3122229076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb6ab60 0x5609bfb73a50 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc036040 tx=0x7fc3dc02cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb9b306ea4a79ab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.780+0000 7fc3d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.781+0000 7fc3d1ffb640 1 -- 172.21.15.43:0/3122229076 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc045070 con 0x5609bfb6ab60 2024-04-21T08:50:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.781+0000 7fc3d1ffb640 1 -- 172.21.15.43:0/3122229076 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fc3dc0035f0 con 0x5609bfb6ab60 2024-04-21T08:50:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.781+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/3122229076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3dc02c7e0 con 0x5609bfb6ab60 2024-04-21T08:50:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.781+0000 7fc3d1ffb640 1 -- 172.21.15.43:0/3122229076 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc3dc059050 con 0x5609bfb6ab60 2024-04-21T08:50:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.781+0000 7fc3f3bd3740 10 client.20838 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.781+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/3122229076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5609bf8ae620 con 0x5609bfb6ab60 2024-04-21T08:50:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.781+0000 7fc3d1ffb640 1 -- 172.21.15.43:0/3122229076 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3dc02c7e0 con 0x5609bfb6ab60 2024-04-21T08:50:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.781+0000 7fc3d1ffb640 1 -- 172.21.15.43:0/3122229076 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc3dc041870 con 0x5609bfb6ab60 2024-04-21T08:50:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.781+0000 7fc3f3bd3740 10 client.20838 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.781+0000 7fc3f3bd3740 10 client.20838 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.781+0000 7fc3f3bd3740 20 client.20838 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.781+0000 7fc3f3bd3740 1 --2- 172.21.15.43:0/3122229076 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5609bfb83ee0 0x5609bfb862d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.781+0000 7fc3f3bd3740 4 client.20838 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.781+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/3122229076 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609bfae3880 con 0x5609bfb83ee0 2024-04-21T08:50:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.782+0000 7fc3e3fff640 1 --2- 172.21.15.43:0/3122229076 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5609bfb83ee0 0x5609bfb862d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.782+0000 7fc3d1ffb640 10 client.20838 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.782+0000 7fc3e3fff640 1 --2- 172.21.15.43:0/3122229076 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5609bfb83ee0 0x5609bfb862d0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5724a4def153fb server_cookie=b12bc2ee1b68e5b8 in_seq=0 out_seq=0 2024-04-21T08:50:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.783+0000 7fc3d1ffb640 1 -- 172.21.15.43:0/3122229076 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5609bfae3880 con 0x5609bfb83ee0 2024-04-21T08:50:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.783+0000 7fc3d1ffb640 10 client.20838 handle_command_reply: tid=0 2024-04-21T08:50:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.783+0000 7fc3f3bd3740 1 client.20838 shutdown 2024-04-21T08:50:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.783+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/3122229076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb6ab60 msgr2=0x5609bfb73a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.783+0000 7fc3f3bd3740 1 --2- 172.21.15.43:0/3122229076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb6ab60 0x5609bfb73a50 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc036040 tx=0x7fc3dc02cc40 comp rx=0 tx=0).stop 2024-04-21T08:50:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.783+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/3122229076 shutdown_connections 2024-04-21T08:50:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.783+0000 7fc3f3bd3740 1 --2- 172.21.15.43:0/3122229076 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5609bfb83ee0 0x5609bfb862d0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.783+0000 7fc3f3bd3740 1 --2- 172.21.15.43:0/3122229076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609bfb740e0 0x5609bfb71340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.783+0000 7fc3f3bd3740 1 --2- 172.21.15.43:0/3122229076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609bfb6b8d0 0x5609bfb70de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.783+0000 7fc3f3bd3740 1 --2- 172.21.15.43:0/3122229076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb6ab60 0x5609bfb73a50 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.783+0000 7fc3d1ffb640 0 client.20838 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.783+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/3122229076 >> 172.21.15.43:0/3122229076 conn(0x5609bfb31c90 msgr2=0x5609bfb675a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.783+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/3122229076 shutdown_connections 2024-04-21T08:50:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.783+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/3122229076 wait complete. 2024-04-21T08:50:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.783+0000 7fc3f3bd3740 20 client.20838 trim_cache size 0 max 16384 2024-04-21T08:50:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.798+0000 7fc3f3bd3740 1 Processor -- start 2024-04-21T08:50:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.798+0000 7fc3f3bd3740 1 -- start start 2024-04-21T08:50:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.798+0000 7fc3f3bd3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609bfb6ab60 0x5609bfb64750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.798+0000 7fc3f3bd3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb6b8d0 0x5609bfb73510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.798+0000 7fc3f3bd3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609bfb740e0 0x5609bfb73a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.798+0000 7fc3f3bd3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1b15d0 con 0x5609bfb6b8d0 2024-04-21T08:50:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.798+0000 7fc3f3bd3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1b3110 con 0x5609bfb740e0 2024-04-21T08:50:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.798+0000 7fc3f3bd3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1b1ca0 con 0x5609bfb6ab60 2024-04-21T08:50:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.798+0000 7fc3e3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb6b8d0 0x5609bfb73510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.798+0000 7fc3e3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb6b8d0 0x5609bfb73510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36646/0 (socket says 172.21.15.43:36646) 2024-04-21T08:50:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.798+0000 7fc3e3fff640 1 -- 172.21.15.43:0/1869206320 learned_addr learned my addr 172.21.15.43:0/1869206320 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.799+0000 7fc3e3fff640 1 -- 172.21.15.43:0/1869206320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609bfb6ab60 msgr2=0x5609bfb64750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.799+0000 7fc3e3fff640 1 --2- 172.21.15.43:0/1869206320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609bfb6ab60 0x5609bfb64750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.799+0000 7fc3ead9d640 1 --2- 172.21.15.43:0/1869206320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609bfb740e0 0x5609bfb73a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.799+0000 7fc3e3fff640 1 -- 172.21.15.43:0/1869206320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609bfb740e0 msgr2=0x5609bfb73a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.799+0000 7fc3e3fff640 1 --2- 172.21.15.43:0/1869206320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609bfb740e0 0x5609bfb73a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.799+0000 7fc3e3fff640 1 -- 172.21.15.43:0/1869206320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609bfb69e30 con 0x5609bfb6b8d0 2024-04-21T08:50:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.799+0000 7fc3ead9d640 1 --2- 172.21.15.43:0/1869206320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609bfb740e0 0x5609bfb73a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.799+0000 7fc3e3fff640 1 --2- 172.21.15.43:0/1869206320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb6b8d0 0x5609bfb73510 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4000c00 tx=0x7fc3d400df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a63bc55aa4a9bc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.799+0000 7fc3d3fff640 1 -- 172.21.15.43:0/1869206320 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d409c3e0 con 0x5609bfb6b8d0 2024-04-21T08:50:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.799+0000 7fc3d3fff640 1 -- 172.21.15.43:0/1869206320 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3d40a06e0 con 0x5609bfb6b8d0 2024-04-21T08:50:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.799+0000 7fc3d3fff640 1 -- 172.21.15.43:0/1869206320 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d40cdd60 con 0x5609bfb6b8d0 2024-04-21T08:50:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.799+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/1869206320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb6b8d0 msgr2=0x5609bfb73510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.799+0000 7fc3f3bd3740 1 --2- 172.21.15.43:0/1869206320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb6b8d0 0x5609bfb73510 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4000c00 tx=0x7fc3d400df40 comp rx=0 tx=0).stop 2024-04-21T08:50:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.799+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/1869206320 shutdown_connections 2024-04-21T08:50:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.799+0000 7fc3f3bd3740 1 --2- 172.21.15.43:0/1869206320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609bfb740e0 0x5609bfb73a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.799+0000 7fc3f3bd3740 1 --2- 172.21.15.43:0/1869206320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb6b8d0 0x5609bfb73510 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.799+0000 7fc3f3bd3740 1 --2- 172.21.15.43:0/1869206320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609bfb6ab60 0x5609bfb64750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.799+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/1869206320 >> 172.21.15.43:0/1869206320 conn(0x5609bfb31c90 msgr2=0x5609bfb64040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.800+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/1869206320 shutdown_connections 2024-04-21T08:50:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.800+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/1869206320 wait complete. 2024-04-21T08:50:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.800+0000 7fc3f3bd3740 1 -- start start 2024-04-21T08:50:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.800+0000 7fc3f3bd3740 1 Processor -- start 2024-04-21T08:50:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.800+0000 7fc3f3bd3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb6ab60 0x5609bfb7c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.800+0000 7fc3f3bd3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609bfb6b8d0 0x5609bfb7c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.800+0000 7fc3f3bd3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609bfb740e0 0x5609bfb79190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.800+0000 7fc3f3bd3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1b2a40 con 0x5609bfb6ab60 2024-04-21T08:50:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.800+0000 7fc3f3bd3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1b2370 con 0x5609bfb740e0 2024-04-21T08:50:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.800+0000 7fc3f3bd3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1af5d0 con 0x5609bfb6b8d0 2024-04-21T08:50:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.800+0000 7fc3f3bd3740 10 client.0 _ll_register_callbacks cb 0x7fc3e4011760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.800+0000 7fc3ea59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb6ab60 0x5609bfb7c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.800+0000 7fc3ea59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb6ab60 0x5609bfb7c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36658/0 (socket says 172.21.15.43:36658) 2024-04-21T08:50:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.800+0000 7fc3ea59c640 1 -- 172.21.15.43:0/2970732524 learned_addr learned my addr 172.21.15.43:0/2970732524 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.800+0000 7fc3ead9d640 1 --2- 172.21.15.43:0/2970732524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609bfb740e0 0x5609bfb79190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.800+0000 7fc3e3fff640 1 --2- 172.21.15.43:0/2970732524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609bfb6b8d0 0x5609bfb7c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.801+0000 7fc3ea59c640 1 -- 172.21.15.43:0/2970732524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609bfb6b8d0 msgr2=0x5609bfb7c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.801+0000 7fc3ea59c640 1 --2- 172.21.15.43:0/2970732524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609bfb6b8d0 0x5609bfb7c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.801+0000 7fc3ea59c640 1 -- 172.21.15.43:0/2970732524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609bfb740e0 msgr2=0x5609bfb79190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.801+0000 7fc3ea59c640 1 --2- 172.21.15.43:0/2970732524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609bfb740e0 0x5609bfb79190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.801+0000 7fc3ea59c640 1 -- 172.21.15.43:0/2970732524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609bfb79780 con 0x5609bfb6ab60 2024-04-21T08:50:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.801+0000 7fc3e3fff640 1 --2- 172.21.15.43:0/2970732524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609bfb6b8d0 0x5609bfb7c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.801+0000 7fc3ea59c640 1 --2- 172.21.15.43:0/2970732524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb6ab60 0x5609bfb7c1c0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009900 tx=0x7fc3dc009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d845c65a38af13fe server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.801+0000 7fc3d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.801+0000 7fc3d1ffb640 1 -- 172.21.15.43:0/2970732524 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc056030 con 0x5609bfb6ab60 2024-04-21T08:50:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.801+0000 7fc3d1ffb640 1 -- 172.21.15.43:0/2970732524 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fc3dc041480 con 0x5609bfb6ab60 2024-04-21T08:50:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.801+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/2970732524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3d40a06e0 con 0x5609bfb6ab60 2024-04-21T08:50:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.801+0000 7fc3d1ffb640 1 -- 172.21.15.43:0/2970732524 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc3dc066050 con 0x5609bfb6ab60 2024-04-21T08:50:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.801+0000 7fc3f3bd3740 10 client.20844 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.801+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/2970732524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5609bfbea9e0 con 0x5609bfb6ab60 2024-04-21T08:50:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.801+0000 7fc3d1ffb640 1 -- 172.21.15.43:0/2970732524 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3d40a06e0 con 0x5609bfb6ab60 2024-04-21T08:50:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.802+0000 7fc3d1ffb640 1 -- 172.21.15.43:0/2970732524 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc3dc034770 con 0x5609bfb6ab60 2024-04-21T08:50:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.802+0000 7fc3f3bd3740 10 client.20844 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.802+0000 7fc3f3bd3740 10 client.20844 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.802+0000 7fc3f3bd3740 20 client.20844 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.802+0000 7fc3f3bd3740 1 --2- 172.21.15.43:0/2970732524 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5609bfbeb160 0x5609bfbed580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.802+0000 7fc3f3bd3740 4 client.20844 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:50:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.802+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/2970732524 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5609bfa6f3d0 con 0x5609bfbeb160 2024-04-21T08:50:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.802+0000 7fc3e3fff640 1 --2- 172.21.15.43:0/2970732524 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5609bfbeb160 0x5609bfbed580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.802+0000 7fc3e3fff640 1 --2- 172.21.15.43:0/2970732524 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5609bfbeb160 0x5609bfbed580 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4c796cae3de3e4e server_cookie=b5efe522d99becee in_seq=0 out_seq=0 2024-04-21T08:50:00.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.802+0000 7fc3d1ffb640 10 client.20844 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.819+0000 7fc3d1ffb640 1 -- 172.21.15.43:0/2970732524 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5609bfa6f3d0 con 0x5609bfbeb160 2024-04-21T08:50:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.819+0000 7fc3d1ffb640 10 client.20844 handle_command_reply: tid=0 2024-04-21T08:50:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.819+0000 7fc3f3bd3740 1 client.20844 shutdown 2024-04-21T08:50:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.819+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/2970732524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb6ab60 msgr2=0x5609bfb7c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.819+0000 7fc3f3bd3740 1 --2- 172.21.15.43:0/2970732524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb6ab60 0x5609bfb7c1c0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009900 tx=0x7fc3dc009670 comp rx=0 tx=0).stop 2024-04-21T08:50:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.819+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/2970732524 shutdown_connections 2024-04-21T08:50:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.819+0000 7fc3f3bd3740 1 --2- 172.21.15.43:0/2970732524 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5609bfbeb160 0x5609bfbed580 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.819+0000 7fc3f3bd3740 1 --2- 172.21.15.43:0/2970732524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609bfb740e0 0x5609bfb79190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.819+0000 7fc3d1ffb640 0 client.20844 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.819+0000 7fc3f3bd3740 1 --2- 172.21.15.43:0/2970732524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5609bfb6b8d0 0x5609bfb7c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.819+0000 7fc3f3bd3740 1 --2- 172.21.15.43:0/2970732524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609bfb6ab60 0x5609bfb7c1c0 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.819+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/2970732524 >> 172.21.15.43:0/2970732524 conn(0x5609bfb31c90 msgr2=0x5609bfb648d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.819+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/2970732524 shutdown_connections 2024-04-21T08:50:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.819+0000 7fc3f3bd3740 1 -- 172.21.15.43:0/2970732524 wait complete. 2024-04-21T08:50:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.819+0000 7fc3f3bd3740 20 client.20844 trim_cache size 0 max 16384 2024-04-21T08:50:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:50:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:50:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:50:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:50:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.820+0000 7fc3f0fe6640 1 -- 172.21.15.43:0/159307446 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc3d80a3dc0 msgr2=0x7fc3d80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.820+0000 7fc3f0fe6640 1 --2- 172.21.15.43:0/159307446 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc3d80a3dc0 0x7fc3d80a6280 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4011c10 tx=0x7fc3e4009ed0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.820+0000 7fc3f0fe6640 1 -- 172.21.15.43:0/159307446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3ec101680 msgr2=0x7fc3ec1a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.820+0000 7fc3f0fe6640 1 --2- 172.21.15.43:0/159307446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3ec101680 0x7fc3ec1a0be0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400d950 tx=0x7fc3d400de10 comp rx=0 tx=0).stop 2024-04-21T08:50:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.820+0000 7fc3f0fe6640 1 -- 172.21.15.43:0/159307446 shutdown_connections 2024-04-21T08:50:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.820+0000 7fc3f0fe6640 1 --2- 172.21.15.43:0/159307446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3ec10e910 0x7fc3ec19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.820+0000 7fc3f0fe6640 1 --2- 172.21.15.43:0/159307446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3ec101680 0x7fc3ec1a0be0 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.820+0000 7fc3f0fe6640 1 --2- 172.21.15.43:0/159307446 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc3d80a3dc0 0x7fc3d80a6280 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.820+0000 7fc3f0fe6640 1 --2- 172.21.15.43:0/159307446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3ec100d60 0x7fc3ec1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.820+0000 7fc3f0fe6640 1 -- 172.21.15.43:0/159307446 >> 172.21.15.43:0/159307446 conn(0x7fc3ec077f70 msgr2=0x7fc3ec10fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.820+0000 7fc3f0fe6640 1 -- 172.21.15.43:0/159307446 shutdown_connections 2024-04-21T08:50:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:00.820+0000 7fc3f0fe6640 1 -- 172.21.15.43:0/159307446 wait complete. 2024-04-21T08:50:02.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:02 smithi175 ceph-mon[31920]: pgmap v345: 129 pgs: 129 active+clean; 3.4 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 112 KiB/s wr, 41 op/s 2024-04-21T08:50:02.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:02 smithi112 ceph-mon[31085]: pgmap v345: 129 pgs: 129 active+clean; 3.4 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 112 KiB/s wr, 41 op/s 2024-04-21T08:50:02.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:02 smithi043 ceph-mon[27978]: pgmap v345: 129 pgs: 129 active+clean; 3.4 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 112 KiB/s wr, 41 op/s 2024-04-21T08:50:04.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:04 smithi175 ceph-mon[31920]: pgmap v346: 129 pgs: 129 active+clean; 3.4 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 88 KiB/s wr, 32 op/s 2024-04-21T08:50:04.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:04 smithi112 ceph-mon[31085]: pgmap v346: 129 pgs: 129 active+clean; 3.4 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 88 KiB/s wr, 32 op/s 2024-04-21T08:50:04.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:04 smithi043 ceph-mon[27978]: pgmap v346: 129 pgs: 129 active+clean; 3.4 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 88 KiB/s wr, 32 op/s 2024-04-21T08:50:05.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:05 smithi112 ceph-mon[31085]: osdmap e105: 12 total, 12 up, 12 in 2024-04-21T08:50:05.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:05 smithi043 ceph-mon[27978]: osdmap e105: 12 total, 12 up, 12 in 2024-04-21T08:50:05.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:05 smithi175 ceph-mon[31920]: osdmap e105: 12 total, 12 up, 12 in 2024-04-21T08:50:06.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:06 smithi112 ceph-mon[31085]: pgmap v348: 129 pgs: 129 active+clean; 3.4 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-21T08:50:06.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:06 smithi043 ceph-mon[27978]: pgmap v348: 129 pgs: 129 active+clean; 3.4 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-21T08:50:06.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:06 smithi175 ceph-mon[31920]: pgmap v348: 129 pgs: 129 active+clean; 3.4 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-21T08:50:06.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:50:06.942 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:50:07.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.062+0000 7f01d1293640 1 -- 172.21.15.43:0/1569185413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01cc101a50 msgr2=0x7f01cc105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.062+0000 7f01d1293640 1 --2- 172.21.15.43:0/1569185413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01cc101a50 0x7f01cc105a90 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f01c0009900 tx=0x7f01c002bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:07.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.063+0000 7f01d1293640 1 -- 172.21.15.43:0/1569185413 shutdown_connections 2024-04-21T08:50:07.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.063+0000 7f01d1293640 1 --2- 172.21.15.43:0/1569185413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01cc106170 0x7f01cc10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.063+0000 7f01d1293640 1 --2- 172.21.15.43:0/1569185413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01cc101a50 0x7f01cc105a90 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.063+0000 7f01d1293640 1 --2- 172.21.15.43:0/1569185413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01cc1010a0 0x7f01cc101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.063+0000 7f01d1293640 1 -- 172.21.15.43:0/1569185413 >> 172.21.15.43:0/1569185413 conn(0x7f01cc077f70 msgr2=0x7f01cc078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.063+0000 7f01d1293640 1 -- 172.21.15.43:0/1569185413 shutdown_connections 2024-04-21T08:50:07.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.063+0000 7f01d1293640 1 -- 172.21.15.43:0/1569185413 wait complete. 2024-04-21T08:50:07.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.063+0000 7f01d1293640 1 Processor -- start 2024-04-21T08:50:07.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.063+0000 7f01d1293640 1 -- start start 2024-04-21T08:50:07.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.064+0000 7f01d1293640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01cc1010a0 0x7f01cc199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.064+0000 7f01d1293640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01cc106170 0x7f01cc199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.064+0000 7f01d1293640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01cc199fe0 0x7f01cc1a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.064+0000 7f01d1293640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc1110b0 con 0x7f01cc106170 2024-04-21T08:50:07.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.064+0000 7f01d1293640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc110f30 con 0x7f01cc1010a0 2024-04-21T08:50:07.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.064+0000 7f01d1293640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc111230 con 0x7f01cc199fe0 2024-04-21T08:50:07.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.064+0000 7f01ca7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01cc106170 0x7f01cc199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.064+0000 7f01ca7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01cc106170 0x7f01cc199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39370/0 (socket says 172.21.15.43:39370) 2024-04-21T08:50:07.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.064+0000 7f01ca7fc640 1 -- 172.21.15.43:0/37705658 learned_addr learned my addr 172.21.15.43:0/37705658 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:07.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.064+0000 7f01cb7fe640 1 --2- 172.21.15.43:0/37705658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01cc199fe0 0x7f01cc1a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.064+0000 7f01caffd640 1 --2- 172.21.15.43:0/37705658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01cc1010a0 0x7f01cc199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.064+0000 7f01ca7fc640 1 -- 172.21.15.43:0/37705658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01cc199fe0 msgr2=0x7f01cc1a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.064+0000 7f01ca7fc640 1 --2- 172.21.15.43:0/37705658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01cc199fe0 0x7f01cc1a4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.064+0000 7f01ca7fc640 1 -- 172.21.15.43:0/37705658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01cc1010a0 msgr2=0x7f01cc199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.064+0000 7f01ca7fc640 1 --2- 172.21.15.43:0/37705658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01cc1010a0 0x7f01cc199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.064+0000 7f01ca7fc640 1 -- 172.21.15.43:0/37705658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01cc1a4d60 con 0x7f01cc106170 2024-04-21T08:50:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.064+0000 7f01ca7fc640 1 --2- 172.21.15.43:0/37705658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01cc106170 0x7f01cc199a50 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f01c0036040 tx=0x7f01c00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.064+0000 7f01b3fff640 1 -- 172.21.15.43:0/37705658 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c0043020 con 0x7f01cc106170 2024-04-21T08:50:07.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.065+0000 7f01d1293640 1 -- 172.21.15.43:0/37705658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01cc1a4ff0 con 0x7f01cc106170 2024-04-21T08:50:07.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.065+0000 7f01b3fff640 1 -- 172.21.15.43:0/37705658 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01c002ca70 con 0x7f01cc106170 2024-04-21T08:50:07.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.065+0000 7f01b3fff640 1 -- 172.21.15.43:0/37705658 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c0003ac0 con 0x7f01cc106170 2024-04-21T08:50:07.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.065+0000 7f01d1293640 1 -- 172.21.15.43:0/37705658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01cc1a5500 con 0x7f01cc106170 2024-04-21T08:50:07.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.067+0000 7f01b3fff640 1 -- 172.21.15.43:0/37705658 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f01c0046020 con 0x7f01cc106170 2024-04-21T08:50:07.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.067+0000 7f01d3f65740 1 Processor -- start 2024-04-21T08:50:07.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.068+0000 7f01d3f65740 1 -- start start 2024-04-21T08:50:07.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.068+0000 7f01d3f65740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115264f580 0x56115264f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.068+0000 7f01d3f65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56115264fea0 0x561152658540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.068+0000 7f01d3f65740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561152658a80 0x56115265ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.068+0000 7f01d3f65740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5611525b41e0 con 0x561152658a80 2024-04-21T08:50:07.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.068+0000 7f01d3f65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561152551390 con 0x56115264fea0 2024-04-21T08:50:07.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.068+0000 7f01d3f65740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5611525b01a0 con 0x56115264f580 2024-04-21T08:50:07.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.068+0000 7f01b3fff640 1 --2- 172.21.15.43:0/37705658 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f01b40a3d20 0x7f01b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.068+0000 7f01caffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56115264fea0 0x561152658540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.068+0000 7f01ca7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561152658a80 0x56115265ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.068+0000 7f01caffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56115264fea0 0x561152658540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:52490/0 (socket says 172.21.15.43:52490) 2024-04-21T08:50:07.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.068+0000 7f01caffd640 1 -- 172.21.15.43:0/247210211 learned_addr learned my addr 172.21.15.43:0/247210211 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:07.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.068+0000 7f01cb7fe640 1 --2- 172.21.15.43:0/37705658 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f01b40a3d20 0x7f01b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.068+0000 7f01cb7fe640 1 --2- 172.21.15.43:0/247210211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115264f580 0x56115264f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.068+0000 7f01b3fff640 1 -- 172.21.15.43:0/37705658 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7f01c00f2bc0 con 0x7f01cc106170 2024-04-21T08:50:07.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.068+0000 7f01caffd640 1 -- 172.21.15.43:0/247210211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115264f580 msgr2=0x56115264f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.068+0000 7f01caffd640 1 --2- 172.21.15.43:0/247210211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115264f580 0x56115264f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.068+0000 7f01caffd640 1 -- 172.21.15.43:0/247210211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561152658a80 msgr2=0x56115265ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.068+0000 7f01caffd640 1 --2- 172.21.15.43:0/247210211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561152658a80 0x56115265ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.068+0000 7f01caffd640 1 -- 172.21.15.43:0/247210211 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611523907d0 con 0x56115264fea0 2024-04-21T08:50:07.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.068+0000 7f01ca7fc640 1 --2- 172.21.15.43:0/247210211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561152658a80 0x56115265ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:50:07.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.069+0000 7f01cb7fe640 1 --2- 172.21.15.43:0/37705658 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f01b40a3d20 0x7f01b40a61e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f01c4005fd0 tx=0x7f01c4009e80 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.069+0000 7f01caffd640 1 --2- 172.21.15.43:0/247210211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56115264fea0 0x561152658540 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f01bc009c70 tx=0x7f01bc013ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9812fc73aceb83c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.069+0000 7f01b17fa640 1 -- 172.21.15.43:0/247210211 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc021070 con 0x56115264fea0 2024-04-21T08:50:07.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.069+0000 7f01b17fa640 1 -- 172.21.15.43:0/247210211 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01bc00ee40 con 0x56115264fea0 2024-04-21T08:50:07.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.069+0000 7f01b17fa640 1 -- 172.21.15.43:0/247210211 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc01cda0 con 0x56115264fea0 2024-04-21T08:50:07.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.070+0000 7f01d3f65740 1 -- 172.21.15.43:0/247210211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56115264fea0 msgr2=0x561152658540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.070+0000 7f01d3f65740 1 --2- 172.21.15.43:0/247210211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56115264fea0 0x561152658540 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f01bc009c70 tx=0x7f01bc013ba0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.070+0000 7f01d3f65740 1 -- 172.21.15.43:0/247210211 shutdown_connections 2024-04-21T08:50:07.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.070+0000 7f01d3f65740 1 --2- 172.21.15.43:0/247210211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561152658a80 0x56115265ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.070+0000 7f01d3f65740 1 --2- 172.21.15.43:0/247210211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56115264fea0 0x561152658540 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.070+0000 7f01d3f65740 1 --2- 172.21.15.43:0/247210211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115264f580 0x56115264f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.070+0000 7f01d3f65740 1 -- 172.21.15.43:0/247210211 >> 172.21.15.43:0/247210211 conn(0x561152649cb0 msgr2=0x56115264a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.070+0000 7f01d3f65740 1 -- 172.21.15.43:0/247210211 shutdown_connections 2024-04-21T08:50:07.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.070+0000 7f01d3f65740 1 -- 172.21.15.43:0/247210211 wait complete. 2024-04-21T08:50:07.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.070+0000 7f01d3f65740 1 -- start start 2024-04-21T08:50:07.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.070+0000 7f01d3f65740 1 Processor -- start 2024-04-21T08:50:07.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.070+0000 7f01d3f65740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56115264f580 0x561152655ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.070+0000 7f01d3f65740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115264fea0 0x561152656020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.070+0000 7f01d3f65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561152658a80 0x561152656560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.070+0000 7f01d3f65740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56115265b870 con 0x56115264f580 2024-04-21T08:50:07.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.070+0000 7f01d3f65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56115256d2f0 con 0x561152658a80 2024-04-21T08:50:07.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.070+0000 7f01d3f65740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56115265b9f0 con 0x56115264fea0 2024-04-21T08:50:07.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.070+0000 7f01d3f65740 10 client.0 _ll_register_callbacks cb 0x56115257a580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:07.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.070+0000 7f01caffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56115264f580 0x561152655ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.070+0000 7f01caffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56115264f580 0x561152655ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39384/0 (socket says 172.21.15.43:39384) 2024-04-21T08:50:07.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.070+0000 7f01caffd640 1 -- 172.21.15.43:0/533811525 learned_addr learned my addr 172.21.15.43:0/533811525 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:07.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.071+0000 7f01cb7fe640 1 --2- 172.21.15.43:0/533811525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561152658a80 0x561152656560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.071+0000 7f01ca7fc640 1 --2- 172.21.15.43:0/533811525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115264fea0 0x561152656020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.071+0000 7f01caffd640 1 -- 172.21.15.43:0/533811525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115264fea0 msgr2=0x561152656020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.071+0000 7f01caffd640 1 --2- 172.21.15.43:0/533811525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115264fea0 0x561152656020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.071+0000 7f01caffd640 1 -- 172.21.15.43:0/533811525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561152658a80 msgr2=0x561152656560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.071+0000 7f01caffd640 1 --2- 172.21.15.43:0/533811525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561152658a80 0x561152656560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.071+0000 7f01caffd640 1 -- 172.21.15.43:0/533811525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611522c63f0 con 0x56115264f580 2024-04-21T08:50:07.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.071+0000 7f01caffd640 1 --2- 172.21.15.43:0/533811525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56115264f580 0x561152655ae0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f01bc01c920 tx=0x7f01bc00ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f111891a283db8ed server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.071+0000 7f01aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:07.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.071+0000 7f01aeff5640 1 -- 172.21.15.43:0/533811525 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc021040 con 0x56115264f580 2024-04-21T08:50:07.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.071+0000 7f01aeff5640 1 -- 172.21.15.43:0/533811525 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7f01bc01cb20 con 0x56115264f580 2024-04-21T08:50:07.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.071+0000 7f01d3f65740 1 -- 172.21.15.43:0/533811525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5611525c1c00 con 0x56115264f580 2024-04-21T08:50:07.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.071+0000 7f01aeff5640 1 -- 172.21.15.43:0/533811525 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f01bc033050 con 0x56115264f580 2024-04-21T08:50:07.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.071+0000 7f01d3f65740 10 client.20856 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:07.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.071+0000 7f01d3f65740 1 -- 172.21.15.43:0/533811525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5611522860f0 con 0x56115264f580 2024-04-21T08:50:07.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.071+0000 7f01aeff5640 1 -- 172.21.15.43:0/533811525 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5611525c1c00 con 0x56115264f580 2024-04-21T08:50:07.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.072+0000 7f01aeff5640 1 -- 172.21.15.43:0/533811525 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f01bc024070 con 0x56115264f580 2024-04-21T08:50:07.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.072+0000 7f01d3f65740 10 client.20856 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:07.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.072+0000 7f01d3f65740 10 client.20856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:07.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.072+0000 7f01d3f65740 20 client.20856 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:07.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.072+0000 7f01d3f65740 1 --2- 172.21.15.43:0/533811525 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561152668a40 0x56115266ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.072+0000 7f01d3f65740 4 client.20856 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:07.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.072+0000 7f01d3f65740 1 -- 172.21.15.43:0/533811525 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611525bba70 con 0x561152668a40 2024-04-21T08:50:07.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.072+0000 7f01ca7fc640 1 --2- 172.21.15.43:0/533811525 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561152668a40 0x56115266ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.072+0000 7f01ca7fc640 1 --2- 172.21.15.43:0/533811525 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561152668a40 0x56115266ae30 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e90b1e6b8dcb8ec1 server_cookie=3a065b3586837cc8 in_seq=0 out_seq=0 2024-04-21T08:50:07.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.072+0000 7f01aeff5640 10 client.20856 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:07.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.073+0000 7f01aeff5640 1 -- 172.21.15.43:0/533811525 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5611525bba70 con 0x561152668a40 2024-04-21T08:50:07.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.073+0000 7f01aeff5640 10 client.20856 handle_command_reply: tid=0 2024-04-21T08:50:07.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.073+0000 7f01d3f65740 1 client.20856 shutdown 2024-04-21T08:50:07.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.073+0000 7f01d3f65740 1 -- 172.21.15.43:0/533811525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56115264f580 msgr2=0x561152655ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.073+0000 7f01d3f65740 1 --2- 172.21.15.43:0/533811525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56115264f580 0x561152655ae0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f01bc01c920 tx=0x7f01bc00ebc0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.073+0000 7f01d3f65740 1 -- 172.21.15.43:0/533811525 shutdown_connections 2024-04-21T08:50:07.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.073+0000 7f01d3f65740 1 --2- 172.21.15.43:0/533811525 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x561152668a40 0x56115266ae30 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.073+0000 7f01d3f65740 1 --2- 172.21.15.43:0/533811525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561152658a80 0x561152656560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.073+0000 7f01aeff5640 0 client.20856 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:07.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.073+0000 7f01d3f65740 1 --2- 172.21.15.43:0/533811525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115264fea0 0x561152656020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.073+0000 7f01d3f65740 1 --2- 172.21.15.43:0/533811525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56115264f580 0x561152655ae0 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.073+0000 7f01d3f65740 1 -- 172.21.15.43:0/533811525 >> 172.21.15.43:0/533811525 conn(0x561152649cb0 msgr2=0x56115264ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.073+0000 7f01d3f65740 1 -- 172.21.15.43:0/533811525 shutdown_connections 2024-04-21T08:50:07.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.073+0000 7f01d3f65740 1 -- 172.21.15.43:0/533811525 wait complete. 2024-04-21T08:50:07.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.073+0000 7f01d3f65740 20 client.20856 trim_cache size 0 max 16384 2024-04-21T08:50:07.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.091+0000 7f01d3f65740 1 Processor -- start 2024-04-21T08:50:07.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.091+0000 7f01d3f65740 1 -- start start 2024-04-21T08:50:07.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.091+0000 7f01d3f65740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115264f580 0x561152658210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.091+0000 7f01d3f65740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56115264fea0 0x561152664730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.091+0000 7f01d3f65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561152658a80 0x561152670490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.091+0000 7f01d3f65740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc1ad3d0 con 0x56115264fea0 2024-04-21T08:50:07.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.091+0000 7f01d3f65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc1aef10 con 0x561152658a80 2024-04-21T08:50:07.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.091+0000 7f01d3f65740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc1adaa0 con 0x56115264f580 2024-04-21T08:50:07.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.091+0000 7f01ca7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56115264fea0 0x561152664730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.092+0000 7f01ca7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56115264fea0 0x561152664730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39390/0 (socket says 172.21.15.43:39390) 2024-04-21T08:50:07.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.092+0000 7f01ca7fc640 1 -- 172.21.15.43:0/155115042 learned_addr learned my addr 172.21.15.43:0/155115042 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:07.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.092+0000 7f01ca7fc640 1 -- 172.21.15.43:0/155115042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115264f580 msgr2=0x561152658210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.092+0000 7f01ca7fc640 1 --2- 172.21.15.43:0/155115042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115264f580 0x561152658210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.092+0000 7f01ca7fc640 1 -- 172.21.15.43:0/155115042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561152658a80 msgr2=0x561152670490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.092+0000 7f01ca7fc640 1 --2- 172.21.15.43:0/155115042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561152658a80 0x561152670490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.092+0000 7f01ca7fc640 1 -- 172.21.15.43:0/155115042 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561152654030 con 0x56115264fea0 2024-04-21T08:50:07.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.092+0000 7f01ca7fc640 1 --2- 172.21.15.43:0/155115042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56115264fea0 0x561152664730 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f01c00be4d0 tx=0x7f01c00a9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e48ebc4f1bc5370 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.092+0000 7f01b0ff9640 1 -- 172.21.15.43:0/155115042 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c002cc20 con 0x56115264fea0 2024-04-21T08:50:07.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.092+0000 7f01b0ff9640 1 -- 172.21.15.43:0/155115042 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01c00be7d0 con 0x56115264fea0 2024-04-21T08:50:07.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.092+0000 7f01b0ff9640 1 -- 172.21.15.43:0/155115042 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c00bae40 con 0x56115264fea0 2024-04-21T08:50:07.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.093+0000 7f01d3f65740 1 -- 172.21.15.43:0/155115042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56115264fea0 msgr2=0x561152664730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.093+0000 7f01d3f65740 1 --2- 172.21.15.43:0/155115042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56115264fea0 0x561152664730 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f01c00be4d0 tx=0x7f01c00a9040 comp rx=0 tx=0).stop 2024-04-21T08:50:07.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.093+0000 7f01d3f65740 1 -- 172.21.15.43:0/155115042 shutdown_connections 2024-04-21T08:50:07.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.093+0000 7f01d3f65740 1 --2- 172.21.15.43:0/155115042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561152658a80 0x561152670490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.093+0000 7f01d3f65740 1 --2- 172.21.15.43:0/155115042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56115264fea0 0x561152664730 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.093+0000 7f01d3f65740 1 --2- 172.21.15.43:0/155115042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115264f580 0x561152658210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.093+0000 7f01d3f65740 1 -- 172.21.15.43:0/155115042 >> 172.21.15.43:0/155115042 conn(0x561152649cb0 msgr2=0x561152654b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.093+0000 7f01d3f65740 1 -- 172.21.15.43:0/155115042 shutdown_connections 2024-04-21T08:50:07.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.093+0000 7f01d3f65740 1 -- 172.21.15.43:0/155115042 wait complete. 2024-04-21T08:50:07.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.093+0000 7f01d3f65740 1 -- start start 2024-04-21T08:50:07.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.093+0000 7f01d3f65740 1 Processor -- start 2024-04-21T08:50:07.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.093+0000 7f01d3f65740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115264f580 0x561152652710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.093+0000 7f01d3f65740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56115264fea0 0x561152652c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.094+0000 7f01d3f65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561152658a80 0x561152653190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.094+0000 7f01d3f65740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc1ae840 con 0x56115264fea0 2024-04-21T08:50:07.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.094+0000 7f01d3f65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc1ae170 con 0x561152658a80 2024-04-21T08:50:07.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.094+0000 7f01d3f65740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc1ab3d0 con 0x56115264f580 2024-04-21T08:50:07.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.094+0000 7f01d3f65740 10 client.0 _ll_register_callbacks cb 0x7f01bc01f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:07.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.094+0000 7f01ca7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56115264fea0 0x561152652c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.094+0000 7f01ca7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56115264fea0 0x561152652c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39396/0 (socket says 172.21.15.43:39396) 2024-04-21T08:50:07.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.094+0000 7f01ca7fc640 1 -- 172.21.15.43:0/3744301849 learned_addr learned my addr 172.21.15.43:0/3744301849 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:07.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.094+0000 7f01caffd640 1 --2- 172.21.15.43:0/3744301849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115264f580 0x561152652710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.094+0000 7f01cb7fe640 1 --2- 172.21.15.43:0/3744301849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561152658a80 0x561152653190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.094+0000 7f01ca7fc640 1 -- 172.21.15.43:0/3744301849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115264f580 msgr2=0x561152652710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.094+0000 7f01ca7fc640 1 --2- 172.21.15.43:0/3744301849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115264f580 0x561152652710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.094+0000 7f01ca7fc640 1 -- 172.21.15.43:0/3744301849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561152658a80 msgr2=0x561152653190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.094+0000 7f01ca7fc640 1 --2- 172.21.15.43:0/3744301849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561152658a80 0x561152653190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.094+0000 7f01ca7fc640 1 -- 172.21.15.43:0/3744301849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611526cdd00 con 0x56115264fea0 2024-04-21T08:50:07.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.094+0000 7f01cb7fe640 1 --2- 172.21.15.43:0/3744301849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561152658a80 0x561152653190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:07.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.094+0000 7f01ca7fc640 1 --2- 172.21.15.43:0/3744301849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56115264fea0 0x561152652c50 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f01c00bac30 tx=0x7f01c00035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559b83d2d36a7267 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.094+0000 7f01aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:07.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.094+0000 7f01aeff5640 1 -- 172.21.15.43:0/3744301849 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c002cc20 con 0x56115264fea0 2024-04-21T08:50:07.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.094+0000 7f01aeff5640 1 -- 172.21.15.43:0/3744301849 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7f01c00bae40 con 0x56115264fea0 2024-04-21T08:50:07.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.094+0000 7f01d3f65740 1 -- 172.21.15.43:0/3744301849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01c00be7d0 con 0x56115264fea0 2024-04-21T08:50:07.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.094+0000 7f01aeff5640 1 -- 172.21.15.43:0/3744301849 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f01c00a5050 con 0x56115264fea0 2024-04-21T08:50:07.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.094+0000 7f01d3f65740 10 client.20862 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:07.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.094+0000 7f01d3f65740 1 -- 172.21.15.43:0/3744301849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5611526cef50 con 0x56115264fea0 2024-04-21T08:50:07.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.095+0000 7f01aeff5640 1 -- 172.21.15.43:0/3744301849 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01c00be7d0 con 0x56115264fea0 2024-04-21T08:50:07.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.095+0000 7f01aeff5640 1 -- 172.21.15.43:0/3744301849 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f01c00f0060 con 0x56115264fea0 2024-04-21T08:50:07.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.095+0000 7f01d3f65740 10 client.20862 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:07.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.095+0000 7f01d3f65740 10 client.20862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:07.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.095+0000 7f01d3f65740 20 client.20862 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:07.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.095+0000 7f01d3f65740 1 --2- 172.21.15.43:0/3744301849 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5611526cf630 0x5611526d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.095+0000 7f01d3f65740 4 client.20862 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:50:07.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.095+0000 7f01d3f65740 1 -- 172.21.15.43:0/3744301849 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561152537590 con 0x5611526cf630 2024-04-21T08:50:07.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.095+0000 7f01caffd640 1 --2- 172.21.15.43:0/3744301849 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5611526cf630 0x5611526d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.095+0000 7f01caffd640 1 --2- 172.21.15.43:0/3744301849 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5611526cf630 0x5611526d1a50 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61fdb27525a92194 server_cookie=10c6b14086239101 in_seq=0 out_seq=0 2024-04-21T08:50:07.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.095+0000 7f01aeff5640 10 client.20862 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:07.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.096+0000 7f01aeff5640 1 -- 172.21.15.43:0/3744301849 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561152537590 con 0x5611526cf630 2024-04-21T08:50:07.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.096+0000 7f01aeff5640 10 client.20862 handle_command_reply: tid=0 2024-04-21T08:50:07.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.096+0000 7f01d3f65740 1 client.20862 shutdown 2024-04-21T08:50:07.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.096+0000 7f01d3f65740 1 -- 172.21.15.43:0/3744301849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56115264fea0 msgr2=0x561152652c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.096+0000 7f01d3f65740 1 --2- 172.21.15.43:0/3744301849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56115264fea0 0x561152652c50 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f01c00bac30 tx=0x7f01c00035f0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.096+0000 7f01d3f65740 1 -- 172.21.15.43:0/3744301849 shutdown_connections 2024-04-21T08:50:07.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.096+0000 7f01d3f65740 1 --2- 172.21.15.43:0/3744301849 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5611526cf630 0x5611526d1a50 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.096+0000 7f01d3f65740 1 --2- 172.21.15.43:0/3744301849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561152658a80 0x561152653190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.096+0000 7f01d3f65740 1 --2- 172.21.15.43:0/3744301849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56115264fea0 0x561152652c50 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.096+0000 7f01aeff5640 0 client.20862 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:07.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.096+0000 7f01d3f65740 1 --2- 172.21.15.43:0/3744301849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115264f580 0x561152652710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.096+0000 7f01d3f65740 1 -- 172.21.15.43:0/3744301849 >> 172.21.15.43:0/3744301849 conn(0x561152649cb0 msgr2=0x561152654280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.096+0000 7f01d3f65740 1 -- 172.21.15.43:0/3744301849 shutdown_connections 2024-04-21T08:50:07.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.096+0000 7f01d3f65740 1 -- 172.21.15.43:0/3744301849 wait complete. 2024-04-21T08:50:07.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.096+0000 7f01d3f65740 20 client.20862 trim_cache size 0 max 16384 2024-04-21T08:50:07.126 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:50:07.126 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:50:07.126 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:50:07.126 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:50:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.096+0000 7f01d1293640 1 -- 172.21.15.43:0/37705658 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f01b40a3d20 msgr2=0x7f01b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.096+0000 7f01d1293640 1 --2- 172.21.15.43:0/37705658 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f01b40a3d20 0x7f01b40a61e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f01c4005fd0 tx=0x7f01c4009e80 comp rx=0 tx=0).stop 2024-04-21T08:50:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.096+0000 7f01d1293640 1 -- 172.21.15.43:0/37705658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01cc106170 msgr2=0x7f01cc199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.096+0000 7f01d1293640 1 --2- 172.21.15.43:0/37705658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01cc106170 0x7f01cc199a50 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f01c0036040 tx=0x7f01c00045e0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.097+0000 7f01d1293640 1 -- 172.21.15.43:0/37705658 shutdown_connections 2024-04-21T08:50:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.097+0000 7f01d1293640 1 --2- 172.21.15.43:0/37705658 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f01b40a3d20 0x7f01b40a61e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.097+0000 7f01d1293640 1 --2- 172.21.15.43:0/37705658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01cc199fe0 0x7f01cc1a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.097+0000 7f01d1293640 1 --2- 172.21.15.43:0/37705658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01cc106170 0x7f01cc199a50 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.097+0000 7f01d1293640 1 --2- 172.21.15.43:0/37705658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01cc1010a0 0x7f01cc199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.097+0000 7f01d1293640 1 -- 172.21.15.43:0/37705658 >> 172.21.15.43:0/37705658 conn(0x7f01cc077f70 msgr2=0x7f01cc0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.097+0000 7f01d1293640 1 -- 172.21.15.43:0/37705658 shutdown_connections 2024-04-21T08:50:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.097+0000 7f01d1293640 1 -- 172.21.15.43:0/37705658 wait complete. 2024-04-21T08:50:07.129 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:50:07.129 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:50:07.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.232+0000 7fcd1564e640 1 -- 172.21.15.43:0/3163408582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd10074b20 msgr2=0x7fcd1010e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.232+0000 7fcd1564e640 1 --2- 172.21.15.43:0/3163408582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd10074b20 0x7fcd1010e320 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7fccf8009900 tx=0x7fccf802bcd0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.232+0000 7fcd1564e640 1 -- 172.21.15.43:0/3163408582 shutdown_connections 2024-04-21T08:50:07.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.232+0000 7fcd1564e640 1 --2- 172.21.15.43:0/3163408582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd1010e860 0x7fcd10110c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.232+0000 7fcd1564e640 1 --2- 172.21.15.43:0/3163408582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd10074b20 0x7fcd1010e320 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.232+0000 7fcd1564e640 1 --2- 172.21.15.43:0/3163408582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd10074200 0x7fcd100745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.232+0000 7fcd1564e640 1 -- 172.21.15.43:0/3163408582 >> 172.21.15.43:0/3163408582 conn(0x7fcd100793a0 msgr2=0x7fcd100797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.232+0000 7fcd1564e640 1 -- 172.21.15.43:0/3163408582 shutdown_connections 2024-04-21T08:50:07.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.232+0000 7fcd1564e640 1 -- 172.21.15.43:0/3163408582 wait complete. 2024-04-21T08:50:07.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.233+0000 7fcd1564e640 1 Processor -- start 2024-04-21T08:50:07.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.233+0000 7fcd1564e640 1 -- start start 2024-04-21T08:50:07.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.233+0000 7fcd1564e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd10074200 0x7fcd101a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.233+0000 7fcd1564e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd10074b20 0x7fcd101a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.233+0000 7fcd1564e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd1010e860 0x7fcd1019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.233+0000 7fcd1564e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101143e0 con 0x7fcd10074200 2024-04-21T08:50:07.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.233+0000 7fcd1564e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcd10114260 con 0x7fcd10074b20 2024-04-21T08:50:07.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.233+0000 7fcd1564e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcd10114560 con 0x7fcd1010e860 2024-04-21T08:50:07.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.233+0000 7fcd0effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd10074200 0x7fcd101a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.233+0000 7fcd0effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd10074200 0x7fcd101a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39408/0 (socket says 172.21.15.43:39408) 2024-04-21T08:50:07.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.233+0000 7fcd0effd640 1 -- 172.21.15.43:0/3226730246 learned_addr learned my addr 172.21.15.43:0/3226730246 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:07.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.233+0000 7fcd0e7fc640 1 --2- 172.21.15.43:0/3226730246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd10074b20 0x7fcd101a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.233+0000 7fcd0f7fe640 1 --2- 172.21.15.43:0/3226730246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd1010e860 0x7fcd1019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.233+0000 7fcd0effd640 1 -- 172.21.15.43:0/3226730246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd1010e860 msgr2=0x7fcd1019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.234+0000 7fcd0effd640 1 --2- 172.21.15.43:0/3226730246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd1010e860 0x7fcd1019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.234+0000 7fcd0effd640 1 -- 172.21.15.43:0/3226730246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd10074b20 msgr2=0x7fcd101a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.234+0000 7fcd0effd640 1 --2- 172.21.15.43:0/3226730246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd10074b20 0x7fcd101a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.234+0000 7fcd0effd640 1 -- 172.21.15.43:0/3226730246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1019af50 con 0x7fcd10074200 2024-04-21T08:50:07.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.234+0000 7fcd0e7fc640 1 --2- 172.21.15.43:0/3226730246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd10074b20 0x7fcd101a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:07.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.234+0000 7fcd0f7fe640 1 --2- 172.21.15.43:0/3226730246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd1010e860 0x7fcd1019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:07.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.234+0000 7fcd0effd640 1 --2- 172.21.15.43:0/3226730246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd10074200 0x7fcd101a0610 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000cce0 tx=0x7fcd0000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.234+0000 7fcd07fff640 1 -- 172.21.15.43:0/3226730246 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0000f650 con 0x7fcd10074200 2024-04-21T08:50:07.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.234+0000 7fcd07fff640 1 -- 172.21.15.43:0/3226730246 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd0000fca0 con 0x7fcd10074200 2024-04-21T08:50:07.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.234+0000 7fcd07fff640 1 -- 172.21.15.43:0/3226730246 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00018b40 con 0x7fcd10074200 2024-04-21T08:50:07.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.234+0000 7fcd1564e640 1 -- 172.21.15.43:0/3226730246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd1019b240 con 0x7fcd10074200 2024-04-21T08:50:07.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.234+0000 7fcd1564e640 1 -- 172.21.15.43:0/3226730246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd101a7460 con 0x7fcd10074200 2024-04-21T08:50:07.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.235+0000 7fcd1564e640 1 -- 172.21.15.43:0/3226730246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd10075940 con 0x7fcd10074200 2024-04-21T08:50:07.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.236+0000 7fcd07fff640 1 -- 172.21.15.43:0/3226730246 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcd0000f7f0 con 0x7fcd10074200 2024-04-21T08:50:07.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.237+0000 7fcd07fff640 1 --2- 172.21.15.43:0/3226730246 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fccfc0a3d20 0x7fccfc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.237+0000 7fcd0e7fc640 1 --2- 172.21.15.43:0/3226730246 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fccfc0a3d20 0x7fccfc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.237+0000 7fcd07fff640 1 -- 172.21.15.43:0/3226730246 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7fcd000d1270 con 0x7fcd10074200 2024-04-21T08:50:07.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.237+0000 7fcd0e7fc640 1 --2- 172.21.15.43:0/3226730246 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fccfc0a3d20 0x7fccfc0a61e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcd1019bad0 tx=0x7fccf8005a20 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.241+0000 7fcd07fff640 1 -- 172.21.15.43:0/3226730246 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd00014030 con 0x7fcd10074200 2024-04-21T08:50:07.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.473+0000 7fcd1564e640 1 -- 172.21.15.43:0/3226730246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd101099e0 con 0x7fcd10074200 2024-04-21T08:50:07.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.473+0000 7fcd07fff640 1 -- 172.21.15.43:0/3226730246 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11935 (secure 0 0 0) 0x7fcd0009d430 con 0x7fcd10074200 2024-04-21T08:50:07.473 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:50:07.473 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:46:17.283806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27556":{"gid":27556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/229985237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":229985237},{"type":"v1","addr":"172.21.15.175:6837","nonce":229985237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24425,24412,24487,24527,24491]},"id":1}]} 2024-04-21T08:50:07.473 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-04-21T08:50:07.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.476+0000 7fcd1564e640 1 -- 172.21.15.43:0/3226730246 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fccfc0a3d20 msgr2=0x7fccfc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.476+0000 7fcd1564e640 1 --2- 172.21.15.43:0/3226730246 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fccfc0a3d20 0x7fccfc0a61e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcd1019bad0 tx=0x7fccf8005a20 comp rx=0 tx=0).stop 2024-04-21T08:50:07.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.476+0000 7fcd1564e640 1 -- 172.21.15.43:0/3226730246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd10074200 msgr2=0x7fcd101a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.476+0000 7fcd1564e640 1 --2- 172.21.15.43:0/3226730246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd10074200 0x7fcd101a0610 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000cce0 tx=0x7fcd0000a620 comp rx=0 tx=0).stop 2024-04-21T08:50:07.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.476+0000 7fcd1564e640 1 -- 172.21.15.43:0/3226730246 shutdown_connections 2024-04-21T08:50:07.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.476+0000 7fcd1564e640 1 --2- 172.21.15.43:0/3226730246 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fccfc0a3d20 0x7fccfc0a61e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.476+0000 7fcd1564e640 1 --2- 172.21.15.43:0/3226730246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd1010e860 0x7fcd1019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.476+0000 7fcd1564e640 1 --2- 172.21.15.43:0/3226730246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd10074b20 0x7fcd101a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.476+0000 7fcd1564e640 1 --2- 172.21.15.43:0/3226730246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd10074200 0x7fcd101a0610 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.476+0000 7fcd1564e640 1 -- 172.21.15.43:0/3226730246 >> 172.21.15.43:0/3226730246 conn(0x7fcd100793a0 msgr2=0x7fcd101001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.476+0000 7fcd1564e640 1 -- 172.21.15.43:0/3226730246 shutdown_connections 2024-04-21T08:50:07.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.476+0000 7fcd1564e640 1 -- 172.21.15.43:0/3226730246 wait complete. 2024-04-21T08:50:07.486 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:50:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.592+0000 7fcff30e4640 1 -- 172.21.15.43:0/1420374212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfec1010a0 msgr2=0x7fcfec101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.592+0000 7fcff30e4640 1 --2- 172.21.15.43:0/1420374212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfec1010a0 0x7fcfec101480 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4009900 tx=0x7fcfd402bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.592+0000 7fcff30e4640 1 -- 172.21.15.43:0/1420374212 shutdown_connections 2024-04-21T08:50:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.592+0000 7fcff30e4640 1 --2- 172.21.15.43:0/1420374212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfec106170 0x7fcfec10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.592+0000 7fcff30e4640 1 --2- 172.21.15.43:0/1420374212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfec101a50 0x7fcfec105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.592+0000 7fcff30e4640 1 --2- 172.21.15.43:0/1420374212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfec1010a0 0x7fcfec101480 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.592+0000 7fcff30e4640 1 -- 172.21.15.43:0/1420374212 >> 172.21.15.43:0/1420374212 conn(0x7fcfec0771d0 msgr2=0x7fcfec0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.592+0000 7fcff30e4640 1 -- 172.21.15.43:0/1420374212 shutdown_connections 2024-04-21T08:50:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.592+0000 7fcff30e4640 1 -- 172.21.15.43:0/1420374212 wait complete. 2024-04-21T08:50:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.593+0000 7fcff30e4640 1 Processor -- start 2024-04-21T08:50:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.593+0000 7fcff30e4640 1 -- start start 2024-04-21T08:50:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.593+0000 7fcff30e4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfec1010a0 0x7fcfec102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.593+0000 7fcff30e4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfec106170 0x7fcfec103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.593+0000 7fcff30e4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfec1035b0 0x7fcfec1a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.593+0000 7fcff30e4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec111100 con 0x7fcfec1035b0 2024-04-21T08:50:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.593+0000 7fcff30e4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec110f80 con 0x7fcfec106170 2024-04-21T08:50:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.593+0000 7fcff30e4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec111280 con 0x7fcfec1010a0 2024-04-21T08:50:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.593+0000 7fcff1681640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfec1035b0 0x7fcfec1a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.593+0000 7fcff1681640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfec1035b0 0x7fcfec1a69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39418/0 (socket says 172.21.15.43:39418) 2024-04-21T08:50:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.593+0000 7fcff1681640 1 -- 172.21.15.43:0/3354679768 learned_addr learned my addr 172.21.15.43:0/3354679768 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.593+0000 7fcfebfff640 1 --2- 172.21.15.43:0/3354679768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfec106170 0x7fcfec103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.593+0000 7fcff0e80640 1 --2- 172.21.15.43:0/3354679768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfec1010a0 0x7fcfec102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.593+0000 7fcff1681640 1 -- 172.21.15.43:0/3354679768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfec1010a0 msgr2=0x7fcfec102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.593+0000 7fcff1681640 1 --2- 172.21.15.43:0/3354679768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfec1010a0 0x7fcfec102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.593+0000 7fcff1681640 1 -- 172.21.15.43:0/3354679768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfec106170 msgr2=0x7fcfec103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.593+0000 7fcff1681640 1 --2- 172.21.15.43:0/3354679768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfec106170 0x7fcfec103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.593+0000 7fcff1681640 1 -- 172.21.15.43:0/3354679768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfec1a70c0 con 0x7fcfec1035b0 2024-04-21T08:50:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.593+0000 7fcff0e80640 1 --2- 172.21.15.43:0/3354679768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfec1010a0 0x7fcfec102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.594+0000 7fcff1681640 1 --2- 172.21.15.43:0/3354679768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfec1035b0 0x7fcfec1a69c0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7fcfe400c970 tx=0x7fcfe400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.594+0000 7fcfe9ffb640 1 -- 172.21.15.43:0/3354679768 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4007bb0 con 0x7fcfec1035b0 2024-04-21T08:50:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.594+0000 7fcff30e4640 1 -- 172.21.15.43:0/3354679768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfec1a73b0 con 0x7fcfec1035b0 2024-04-21T08:50:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.594+0000 7fcfe9ffb640 1 -- 172.21.15.43:0/3354679768 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfe4007d50 con 0x7fcfec1035b0 2024-04-21T08:50:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.594+0000 7fcfe9ffb640 1 -- 172.21.15.43:0/3354679768 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4004e60 con 0x7fcfec1035b0 2024-04-21T08:50:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.594+0000 7fcff30e4640 1 -- 172.21.15.43:0/3354679768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfec1a78b0 con 0x7fcfec1035b0 2024-04-21T08:50:07.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.596+0000 7fcfe9ffb640 1 -- 172.21.15.43:0/3354679768 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcfe4005020 con 0x7fcfec1035b0 2024-04-21T08:50:07.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.596+0000 7fcff5db4740 1 Processor -- start 2024-04-21T08:50:07.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.596+0000 7fcff5db4740 1 -- start start 2024-04-21T08:50:07.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.597+0000 7fcff5db4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d0afa64d0 0x560d0afa68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.597+0000 7fcff5db4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0afa6e80 0x560d0afaf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.597+0000 7fcff5db4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afafa00 0x560d0afb1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.597+0000 7fcff5db4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560d0af0b1f0 con 0x560d0afafa00 2024-04-21T08:50:07.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.597+0000 7fcff5db4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560d0aea84b0 con 0x560d0afa64d0 2024-04-21T08:50:07.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.597+0000 7fcff5db4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560d0af071b0 con 0x560d0afa6e80 2024-04-21T08:50:07.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.597+0000 7fcfe9ffb640 1 --2- 172.21.15.43:0/3354679768 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcfd80a3d50 0x7fcfd80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.597+0000 7fcfebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afafa00 0x560d0afb1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.597+0000 7fcfebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afafa00 0x560d0afb1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39420/0 (socket says 172.21.15.43:39420) 2024-04-21T08:50:07.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.597+0000 7fcfebfff640 1 -- 172.21.15.43:0/2820142941 learned_addr learned my addr 172.21.15.43:0/2820142941 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:07.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.597+0000 7fcfebfff640 1 --2- 172.21.15.43:0/2820142941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d0afa64d0 0x560d0afa68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.597+0000 7fcff0e80640 1 --2- 172.21.15.43:0/2820142941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0afa6e80 0x560d0afaf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.597+0000 7fcff1681640 1 --2- 172.21.15.43:0/3354679768 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcfd80a3d50 0x7fcfd80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.597+0000 7fcfe9ffb640 1 -- 172.21.15.43:0/3354679768 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7fcfe40d0870 con 0x7fcfec1035b0 2024-04-21T08:50:07.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.597+0000 7fcfebfff640 1 -- 172.21.15.43:0/2820142941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0afa6e80 msgr2=0x560d0afaf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.597+0000 7fcfebfff640 1 --2- 172.21.15.43:0/2820142941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0afa6e80 0x560d0afaf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.597+0000 7fcfebfff640 1 -- 172.21.15.43:0/2820142941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d0afa64d0 msgr2=0x560d0afa68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.597+0000 7fcfebfff640 1 --2- 172.21.15.43:0/2820142941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d0afa64d0 0x560d0afa68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.597+0000 7fcfebfff640 1 -- 172.21.15.43:0/2820142941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d0ae9a1b0 con 0x560d0afafa00 2024-04-21T08:50:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.597+0000 7fcfebfff640 1 --2- 172.21.15.43:0/2820142941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afafa00 0x560d0afb1e50 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc012af0 tx=0x7fcfdc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd04f7a0ce2a5a54 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.598+0000 7fcff1681640 1 --2- 172.21.15.43:0/3354679768 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcfd80a3d50 0x7fcfd80a6210 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcfe40a4ad0 tx=0x7fcfe4066ee0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.598+0000 7fcfe2ffd640 1 -- 172.21.15.43:0/2820142941 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc004b20 con 0x560d0afafa00 2024-04-21T08:50:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.598+0000 7fcfe2ffd640 1 -- 172.21.15.43:0/2820142941 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfdc004cc0 con 0x560d0afafa00 2024-04-21T08:50:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.598+0000 7fcfe2ffd640 1 -- 172.21.15.43:0/2820142941 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc0054c0 con 0x560d0afafa00 2024-04-21T08:50:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.598+0000 7fcff5db4740 1 -- 172.21.15.43:0/2820142941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afafa00 msgr2=0x560d0afb1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.598+0000 7fcff5db4740 1 --2- 172.21.15.43:0/2820142941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afafa00 0x560d0afb1e50 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc012af0 tx=0x7fcfdc004ab0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.598+0000 7fcff5db4740 1 -- 172.21.15.43:0/2820142941 shutdown_connections 2024-04-21T08:50:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.598+0000 7fcff5db4740 1 --2- 172.21.15.43:0/2820142941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afafa00 0x560d0afb1e50 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.598+0000 7fcff5db4740 1 --2- 172.21.15.43:0/2820142941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0afa6e80 0x560d0afaf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.598+0000 7fcff5db4740 1 --2- 172.21.15.43:0/2820142941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d0afa64d0 0x560d0afa68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.598+0000 7fcff5db4740 1 -- 172.21.15.43:0/2820142941 >> 172.21.15.43:0/2820142941 conn(0x560d0afa0ba0 msgr2=0x560d0afa0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.598+0000 7fcff5db4740 1 -- 172.21.15.43:0/2820142941 shutdown_connections 2024-04-21T08:50:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.598+0000 7fcff5db4740 1 -- 172.21.15.43:0/2820142941 wait complete. 2024-04-21T08:50:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.599+0000 7fcff5db4740 1 -- start start 2024-04-21T08:50:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.599+0000 7fcff5db4740 1 Processor -- start 2024-04-21T08:50:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.599+0000 7fcff5db4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d0afa64d0 0x560d0afaf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.599+0000 7fcff5db4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0afa6e80 0x560d0afac690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.599+0000 7fcff5db4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afafa00 0x560d0afacc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.599+0000 7fcff5db4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560d0afb2970 con 0x560d0afafa00 2024-04-21T08:50:07.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.599+0000 7fcff5db4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560d0aec4500 con 0x560d0afa64d0 2024-04-21T08:50:07.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.599+0000 7fcff5db4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560d0afb2af0 con 0x560d0afa6e80 2024-04-21T08:50:07.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.599+0000 7fcff5db4740 10 client.0 _ll_register_callbacks cb 0x560d0ae92b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:07.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.599+0000 7fcfebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afafa00 0x560d0afacc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.599+0000 7fcfebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afafa00 0x560d0afacc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39436/0 (socket says 172.21.15.43:39436) 2024-04-21T08:50:07.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.599+0000 7fcfebfff640 1 -- 172.21.15.43:0/3276009947 learned_addr learned my addr 172.21.15.43:0/3276009947 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:07.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.600+0000 7fcfebfff640 1 --2- 172.21.15.43:0/3276009947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d0afa64d0 0x560d0afaf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.600+0000 7fcff0e80640 1 --2- 172.21.15.43:0/3276009947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0afa6e80 0x560d0afac690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.600+0000 7fcfebfff640 1 -- 172.21.15.43:0/3276009947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0afa6e80 msgr2=0x560d0afac690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.600+0000 7fcfebfff640 1 --2- 172.21.15.43:0/3276009947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0afa6e80 0x560d0afac690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.600+0000 7fcfebfff640 1 -- 172.21.15.43:0/3276009947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d0afa64d0 msgr2=0x560d0afaf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.600+0000 7fcfebfff640 1 --2- 172.21.15.43:0/3276009947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d0afa64d0 0x560d0afaf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.600+0000 7fcfebfff640 1 -- 172.21.15.43:0/3276009947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d0ac1d3f0 con 0x560d0afafa00 2024-04-21T08:50:07.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.600+0000 7fcfebfff640 1 --2- 172.21.15.43:0/3276009947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afafa00 0x560d0afacc10 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc002230 tx=0x7fcfdc01b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498ce6825cb0bb44 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.600+0000 7fcfcbfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:07.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.600+0000 7fcfcbfff640 1 -- 172.21.15.43:0/3276009947 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc004b20 con 0x560d0afafa00 2024-04-21T08:50:07.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.600+0000 7fcff5db4740 1 -- 172.21.15.43:0/3276009947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfdc004cc0 con 0x560d0afafa00 2024-04-21T08:50:07.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.600+0000 7fcfcbfff640 1 -- 172.21.15.43:0/3276009947 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7fcfdc005440 con 0x560d0afafa00 2024-04-21T08:50:07.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.600+0000 7fcfcbfff640 1 -- 172.21.15.43:0/3276009947 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcfdc010cc0 con 0x560d0afafa00 2024-04-21T08:50:07.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.600+0000 7fcff5db4740 10 client.20880 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:07.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.600+0000 7fcff5db4740 1 -- 172.21.15.43:0/3276009947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560d0ace77d0 con 0x560d0afafa00 2024-04-21T08:50:07.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.600+0000 7fcfcbfff640 1 -- 172.21.15.43:0/3276009947 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfdc004cc0 con 0x560d0afafa00 2024-04-21T08:50:07.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.601+0000 7fcfcbfff640 1 -- 172.21.15.43:0/3276009947 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcfdc01fab0 con 0x560d0afafa00 2024-04-21T08:50:07.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.601+0000 7fcff5db4740 10 client.20880 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:07.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.601+0000 7fcff5db4740 10 client.20880 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:50:07.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.601+0000 7fcff5db4740 20 client.20880 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:07.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.601+0000 7fcff5db4740 1 --2- 172.21.15.43:0/3276009947 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x560d0afbfa80 0x560d0afc1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.601+0000 7fcff5db4740 4 client.20880 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:07.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.601+0000 7fcff5db4740 1 -- 172.21.15.43:0/3276009947 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d0aeaa930 con 0x560d0afbfa80 2024-04-21T08:50:07.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.601+0000 7fcff0e80640 1 --2- 172.21.15.43:0/3276009947 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x560d0afbfa80 0x560d0afc1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.601+0000 7fcfcbfff640 10 client.20880 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:50:07.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.601+0000 7fcff0e80640 1 --2- 172.21.15.43:0/3276009947 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x560d0afbfa80 0x560d0afc1e70 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=40174a2cf1b17eb3 server_cookie=f6612820efa24b66 in_seq=0 out_seq=0 2024-04-21T08:50:07.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.602+0000 7fcfcbfff640 1 -- 172.21.15.43:0/3276009947 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d0aeaa930 con 0x560d0afbfa80 2024-04-21T08:50:07.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.602+0000 7fcfcbfff640 10 client.20880 handle_command_reply: tid=0 2024-04-21T08:50:07.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.602+0000 7fcff5db4740 1 client.20880 shutdown 2024-04-21T08:50:07.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.602+0000 7fcff5db4740 1 -- 172.21.15.43:0/3276009947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afafa00 msgr2=0x560d0afacc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.602+0000 7fcff5db4740 1 --2- 172.21.15.43:0/3276009947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afafa00 0x560d0afacc10 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc002230 tx=0x7fcfdc01b040 comp rx=0 tx=0).stop 2024-04-21T08:50:07.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.602+0000 7fcff5db4740 1 -- 172.21.15.43:0/3276009947 shutdown_connections 2024-04-21T08:50:07.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.602+0000 7fcff5db4740 1 --2- 172.21.15.43:0/3276009947 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x560d0afbfa80 0x560d0afc1e70 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.602+0000 7fcff5db4740 1 --2- 172.21.15.43:0/3276009947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afafa00 0x560d0afacc10 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.603+0000 7fcff5db4740 1 --2- 172.21.15.43:0/3276009947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0afa6e80 0x560d0afac690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.603+0000 7fcff5db4740 1 --2- 172.21.15.43:0/3276009947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d0afa64d0 0x560d0afaf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.603+0000 7fcfcbfff640 0 client.20880 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:50:07.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.603+0000 7fcff5db4740 1 -- 172.21.15.43:0/3276009947 >> 172.21.15.43:0/3276009947 conn(0x560d0afa0ba0 msgr2=0x560d0afa28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.603+0000 7fcff5db4740 1 -- 172.21.15.43:0/3276009947 shutdown_connections 2024-04-21T08:50:07.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.603+0000 7fcff5db4740 1 -- 172.21.15.43:0/3276009947 wait complete. 2024-04-21T08:50:07.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.603+0000 7fcff5db4740 20 client.20880 trim_cache size 0 max 16384 2024-04-21T08:50:07.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.616+0000 7fcff5db4740 1 Processor -- start 2024-04-21T08:50:07.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.616+0000 7fcff5db4740 1 -- start start 2024-04-21T08:50:07.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.616+0000 7fcff5db4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0afa64d0 0x560d0afabe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.616+0000 7fcff5db4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afa6e80 0x560d0afaedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.616+0000 7fcff5db4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d0afafa00 0x560d0afaf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.616+0000 7fcff5db4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec1b1900 con 0x560d0afa6e80 2024-04-21T08:50:07.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.616+0000 7fcff5db4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec1b3440 con 0x560d0afafa00 2024-04-21T08:50:07.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.616+0000 7fcff5db4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec1b1fd0 con 0x560d0afa64d0 2024-04-21T08:50:07.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.616+0000 7fcff0e80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afa6e80 0x560d0afaedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.616+0000 7fcff0e80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afa6e80 0x560d0afaedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39446/0 (socket says 172.21.15.43:39446) 2024-04-21T08:50:07.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.616+0000 7fcff0e80640 1 -- 172.21.15.43:0/3734979985 learned_addr learned my addr 172.21.15.43:0/3734979985 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:07.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.616+0000 7fcfebfff640 1 --2- 172.21.15.43:0/3734979985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d0afafa00 0x560d0afaf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.616+0000 7fcff0e80640 1 -- 172.21.15.43:0/3734979985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0afa64d0 msgr2=0x560d0afabe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.616+0000 7fcff0e80640 1 --2- 172.21.15.43:0/3734979985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0afa64d0 0x560d0afabe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.617+0000 7fcff0e80640 1 -- 172.21.15.43:0/3734979985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d0afafa00 msgr2=0x560d0afaf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.617+0000 7fcff0e80640 1 --2- 172.21.15.43:0/3734979985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d0afafa00 0x560d0afaf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.617+0000 7fcff0e80640 1 -- 172.21.15.43:0/3734979985 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d0af9c470 con 0x560d0afa6e80 2024-04-21T08:50:07.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.617+0000 7fcfebfff640 1 --2- 172.21.15.43:0/3734979985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d0afafa00 0x560d0afaf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:07.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.617+0000 7fcff0e80640 1 --2- 172.21.15.43:0/3734979985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afa6e80 0x560d0afaedc0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fcfd402c170 tx=0x7fcfd4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887d142cf57d338e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.617+0000 7fcfe27fc640 1 -- 172.21.15.43:0/3734979985 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4047070 con 0x560d0afa6e80 2024-04-21T08:50:07.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.617+0000 7fcfe27fc640 1 -- 172.21.15.43:0/3734979985 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd402cc90 con 0x560d0afa6e80 2024-04-21T08:50:07.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.617+0000 7fcfe27fc640 1 -- 172.21.15.43:0/3734979985 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd40046c0 con 0x560d0afa6e80 2024-04-21T08:50:07.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.617+0000 7fcff5db4740 1 -- 172.21.15.43:0/3734979985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afa6e80 msgr2=0x560d0afaedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.617+0000 7fcff5db4740 1 --2- 172.21.15.43:0/3734979985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afa6e80 0x560d0afaedc0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fcfd402c170 tx=0x7fcfd4009260 comp rx=0 tx=0).stop 2024-04-21T08:50:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.617+0000 7fcff5db4740 1 -- 172.21.15.43:0/3734979985 shutdown_connections 2024-04-21T08:50:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.617+0000 7fcff5db4740 1 --2- 172.21.15.43:0/3734979985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d0afafa00 0x560d0afaf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.617+0000 7fcff5db4740 1 --2- 172.21.15.43:0/3734979985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afa6e80 0x560d0afaedc0 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.617+0000 7fcff5db4740 1 --2- 172.21.15.43:0/3734979985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0afa64d0 0x560d0afabe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.617+0000 7fcff5db4740 1 -- 172.21.15.43:0/3734979985 >> 172.21.15.43:0/3734979985 conn(0x560d0afa0ba0 msgr2=0x560d0afa7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.618+0000 7fcff5db4740 1 -- 172.21.15.43:0/3734979985 shutdown_connections 2024-04-21T08:50:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.618+0000 7fcff5db4740 1 -- 172.21.15.43:0/3734979985 wait complete. 2024-04-21T08:50:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.618+0000 7fcff5db4740 1 -- start start 2024-04-21T08:50:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.618+0000 7fcff5db4740 1 Processor -- start 2024-04-21T08:50:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.618+0000 7fcff5db4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0afa64d0 0x560d0afb7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.618+0000 7fcff5db4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d0afa6e80 0x560d0afb3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.618+0000 7fcff5db4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afafa00 0x560d0afb38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.618+0000 7fcff5db4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec1b2d70 con 0x560d0afafa00 2024-04-21T08:50:07.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.618+0000 7fcff5db4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec1b26a0 con 0x560d0afa6e80 2024-04-21T08:50:07.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.618+0000 7fcff5db4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec1af900 con 0x560d0afa64d0 2024-04-21T08:50:07.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.618+0000 7fcff5db4740 10 client.0 _ll_register_callbacks cb 0x7fcfdc0129c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:07.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.618+0000 7fcfebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afafa00 0x560d0afb38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.618+0000 7fcfebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afafa00 0x560d0afb38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39462/0 (socket says 172.21.15.43:39462) 2024-04-21T08:50:07.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.618+0000 7fcfebfff640 1 -- 172.21.15.43:0/3487181747 learned_addr learned my addr 172.21.15.43:0/3487181747 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:07.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.619+0000 7fcfebfff640 1 --2- 172.21.15.43:0/3487181747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0afa64d0 0x560d0afb7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.619+0000 7fcff0e80640 1 --2- 172.21.15.43:0/3487181747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d0afa6e80 0x560d0afb3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.619+0000 7fcfebfff640 1 -- 172.21.15.43:0/3487181747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0afa64d0 msgr2=0x560d0afb7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.619+0000 7fcfebfff640 1 --2- 172.21.15.43:0/3487181747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0afa64d0 0x560d0afb7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.619+0000 7fcfebfff640 1 -- 172.21.15.43:0/3487181747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d0afa6e80 msgr2=0x560d0afb3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.619+0000 7fcfebfff640 1 --2- 172.21.15.43:0/3487181747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d0afa6e80 0x560d0afb3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.619+0000 7fcfebfff640 1 -- 172.21.15.43:0/3487181747 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d0afb3f90 con 0x560d0afafa00 2024-04-21T08:50:07.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.619+0000 7fcfebfff640 1 --2- 172.21.15.43:0/3487181747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afafa00 0x560d0afb38a0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009860 tx=0x7fcfdc009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2796af418d3dd957 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.619+0000 7fcfcbfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:07.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.619+0000 7fcfcbfff640 1 -- 172.21.15.43:0/3487181747 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc01d630 con 0x560d0afafa00 2024-04-21T08:50:07.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.619+0000 7fcfcbfff640 1 -- 172.21.15.43:0/3487181747 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7fcfdc01ec90 con 0x560d0afafa00 2024-04-21T08:50:07.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.619+0000 7fcff5db4740 1 -- 172.21.15.43:0/3487181747 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfd402cc90 con 0x560d0afafa00 2024-04-21T08:50:07.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.619+0000 7fcfcbfff640 1 -- 172.21.15.43:0/3487181747 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcfdc040050 con 0x560d0afafa00 2024-04-21T08:50:07.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.619+0000 7fcff5db4740 10 client.20886 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:07.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.619+0000 7fcff5db4740 1 -- 172.21.15.43:0/3487181747 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560d0b025b80 con 0x560d0afafa00 2024-04-21T08:50:07.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.619+0000 7fcfcbfff640 1 -- 172.21.15.43:0/3487181747 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd402cc90 con 0x560d0afafa00 2024-04-21T08:50:07.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.620+0000 7fcfcbfff640 1 -- 172.21.15.43:0/3487181747 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcfdc01fb30 con 0x560d0afafa00 2024-04-21T08:50:07.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.620+0000 7fcff5db4740 10 client.20886 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:07.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.620+0000 7fcff5db4740 10 client.20886 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:50:07.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.620+0000 7fcff5db4740 20 client.20886 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:07.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.620+0000 7fcff5db4740 1 --2- 172.21.15.43:0/3487181747 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x560d0b026270 0x560d0b028690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.620+0000 7fcff5db4740 4 client.20886 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:50:07.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.620+0000 7fcff5db4740 1 -- 172.21.15.43:0/3487181747 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d0af18c10 con 0x560d0b026270 2024-04-21T08:50:07.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.620+0000 7fcff0e80640 1 --2- 172.21.15.43:0/3487181747 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x560d0b026270 0x560d0b028690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.620+0000 7fcfcbfff640 10 client.20886 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:50:07.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.620+0000 7fcff0e80640 1 --2- 172.21.15.43:0/3487181747 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x560d0b026270 0x560d0b028690 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4163dd6181282c5b server_cookie=7d04d0589a038d36 in_seq=0 out_seq=0 2024-04-21T08:50:07.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.620+0000 7fcfcbfff640 1 -- 172.21.15.43:0/3487181747 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d0af18c10 con 0x560d0b026270 2024-04-21T08:50:07.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.620+0000 7fcfcbfff640 10 client.20886 handle_command_reply: tid=0 2024-04-21T08:50:07.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.620+0000 7fcff5db4740 1 client.20886 shutdown 2024-04-21T08:50:07.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.621+0000 7fcff5db4740 1 -- 172.21.15.43:0/3487181747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afafa00 msgr2=0x560d0afb38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.621+0000 7fcff5db4740 1 --2- 172.21.15.43:0/3487181747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afafa00 0x560d0afb38a0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009860 tx=0x7fcfdc009dd0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.621+0000 7fcff5db4740 1 -- 172.21.15.43:0/3487181747 shutdown_connections 2024-04-21T08:50:07.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.621+0000 7fcff5db4740 1 --2- 172.21.15.43:0/3487181747 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x560d0b026270 0x560d0b028690 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.621+0000 7fcff5db4740 1 --2- 172.21.15.43:0/3487181747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d0afafa00 0x560d0afb38a0 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.621+0000 7fcff5db4740 1 --2- 172.21.15.43:0/3487181747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d0afa6e80 0x560d0afb3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.621+0000 7fcfcbfff640 0 client.20886 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:50:07.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.621+0000 7fcff5db4740 1 --2- 172.21.15.43:0/3487181747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0afa64d0 0x560d0afb7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.621+0000 7fcff5db4740 1 -- 172.21.15.43:0/3487181747 >> 172.21.15.43:0/3487181747 conn(0x560d0afa0ba0 msgr2=0x560d0afa1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.621+0000 7fcff5db4740 1 -- 172.21.15.43:0/3487181747 shutdown_connections 2024-04-21T08:50:07.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.621+0000 7fcff5db4740 1 -- 172.21.15.43:0/3487181747 wait complete. 2024-04-21T08:50:07.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.621+0000 7fcff5db4740 20 client.20886 trim_cache size 0 max 16384 2024-04-21T08:50:07.632 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:50:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.621+0000 7fcff30e4640 1 -- 172.21.15.43:0/3354679768 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcfd80a3d50 msgr2=0x7fcfd80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.621+0000 7fcff30e4640 1 --2- 172.21.15.43:0/3354679768 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcfd80a3d50 0x7fcfd80a6210 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcfe40a4ad0 tx=0x7fcfe4066ee0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.621+0000 7fcff30e4640 1 -- 172.21.15.43:0/3354679768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfec1035b0 msgr2=0x7fcfec1a69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.621+0000 7fcff30e4640 1 --2- 172.21.15.43:0/3354679768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfec1035b0 0x7fcfec1a69c0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7fcfe400c970 tx=0x7fcfe400ce30 comp rx=0 tx=0).stop 2024-04-21T08:50:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.621+0000 7fcff30e4640 1 -- 172.21.15.43:0/3354679768 shutdown_connections 2024-04-21T08:50:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.621+0000 7fcff30e4640 1 --2- 172.21.15.43:0/3354679768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfec1035b0 0x7fcfec1a69c0 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.621+0000 7fcff30e4640 1 --2- 172.21.15.43:0/3354679768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfec106170 0x7fcfec103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.621+0000 7fcff30e4640 1 --2- 172.21.15.43:0/3354679768 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fcfd80a3d50 0x7fcfd80a6210 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.621+0000 7fcff30e4640 1 --2- 172.21.15.43:0/3354679768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfec1010a0 0x7fcfec102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.621+0000 7fcff30e4640 1 -- 172.21.15.43:0/3354679768 >> 172.21.15.43:0/3354679768 conn(0x7fcfec0771d0 msgr2=0x7fcfec104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.622+0000 7fcff30e4640 1 -- 172.21.15.43:0/3354679768 shutdown_connections 2024-04-21T08:50:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.622+0000 7fcff30e4640 1 -- 172.21.15.43:0/3354679768 wait complete. 2024-04-21T08:50:07.635 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:50:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.739+0000 7ff712196640 1 -- 172.21.15.43:0/1884724934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70c10e960 msgr2=0x7ff70c110d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.739+0000 7ff712196640 1 --2- 172.21.15.43:0/1884724934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70c10e960 0x7ff70c110d50 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7ff704009ca0 tx=0x7ff70402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.739+0000 7ff712196640 1 -- 172.21.15.43:0/1884724934 shutdown_connections 2024-04-21T08:50:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.739+0000 7ff712196640 1 --2- 172.21.15.43:0/1884724934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70c10e960 0x7ff70c110d50 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.739+0000 7ff712196640 1 --2- 172.21.15.43:0/1884724934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff70c06aef0 0x7ff70c10e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.739+0000 7ff712196640 1 --2- 172.21.15.43:0/1884724934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff70c06a5d0 0x7ff70c06a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.739+0000 7ff712196640 1 -- 172.21.15.43:0/1884724934 >> 172.21.15.43:0/1884724934 conn(0x7ff70c077eb0 msgr2=0x7ff70c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.739+0000 7ff712196640 1 -- 172.21.15.43:0/1884724934 shutdown_connections 2024-04-21T08:50:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.739+0000 7ff712196640 1 -- 172.21.15.43:0/1884724934 wait complete. 2024-04-21T08:50:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.740+0000 7ff712196640 1 Processor -- start 2024-04-21T08:50:07.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.740+0000 7ff712196640 1 -- start start 2024-04-21T08:50:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.740+0000 7ff712196640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff70c06a5d0 0x7ff70c1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.740+0000 7ff712196640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70c06aef0 0x7ff70c1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.740+0000 7ff712196640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff70c10e960 0x7ff70c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.740+0000 7ff712196640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c1143b0 con 0x7ff70c06aef0 2024-04-21T08:50:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.740+0000 7ff712196640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c114230 con 0x7ff70c06a5d0 2024-04-21T08:50:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.740+0000 7ff712196640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c114530 con 0x7ff70c10e960 2024-04-21T08:50:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.740+0000 7ff70affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70c06aef0 0x7ff70c1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.740+0000 7ff70affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70c06aef0 0x7ff70c1a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39470/0 (socket says 172.21.15.43:39470) 2024-04-21T08:50:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.740+0000 7ff70affd640 1 -- 172.21.15.43:0/3682602626 learned_addr learned my addr 172.21.15.43:0/3682602626 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.741+0000 7ff70bfff640 1 --2- 172.21.15.43:0/3682602626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff70c10e960 0x7ff70c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.741+0000 7ff70b7fe640 1 --2- 172.21.15.43:0/3682602626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff70c06a5d0 0x7ff70c1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.741+0000 7ff70affd640 1 -- 172.21.15.43:0/3682602626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff70c10e960 msgr2=0x7ff70c19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.741+0000 7ff70affd640 1 --2- 172.21.15.43:0/3682602626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff70c10e960 0x7ff70c19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.741+0000 7ff70affd640 1 -- 172.21.15.43:0/3682602626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff70c06a5d0 msgr2=0x7ff70c1a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.741+0000 7ff70affd640 1 --2- 172.21.15.43:0/3682602626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff70c06a5d0 0x7ff70c1a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.741+0000 7ff70affd640 1 -- 172.21.15.43:0/3682602626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff70c19afd0 con 0x7ff70c06aef0 2024-04-21T08:50:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.741+0000 7ff70affd640 1 --2- 172.21.15.43:0/3682602626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70c06aef0 0x7ff70c1a0b90 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7ff70000d950 tx=0x7ff70000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.741+0000 7ff708ff9640 1 -- 172.21.15.43:0/3682602626 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700017070 con 0x7ff70c06aef0 2024-04-21T08:50:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.741+0000 7ff708ff9640 1 -- 172.21.15.43:0/3682602626 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff700009c30 con 0x7ff70c06aef0 2024-04-21T08:50:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.741+0000 7ff708ff9640 1 -- 172.21.15.43:0/3682602626 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70000f620 con 0x7ff70c06aef0 2024-04-21T08:50:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.741+0000 7ff712196640 1 -- 172.21.15.43:0/3682602626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff70c19b2c0 con 0x7ff70c06aef0 2024-04-21T08:50:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.741+0000 7ff712196640 1 -- 172.21.15.43:0/3682602626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff70c1a75d0 con 0x7ff70c06aef0 2024-04-21T08:50:07.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.743+0000 7ff708ff9640 1 -- 172.21.15.43:0/3682602626 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff700010460 con 0x7ff70c06aef0 2024-04-21T08:50:07.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.744+0000 7ff714d83740 1 Processor -- start 2024-04-21T08:50:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.744+0000 7ff714d83740 1 -- start start 2024-04-21T08:50:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.744+0000 7ff714d83740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6615474d0 0x55a6615478b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.744+0000 7ff714d83740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a661547e80 0x55a661550430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.744+0000 7ff714d83740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a661550a00 0x55a661552e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.744+0000 7ff714d83740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a6614ac1f0 con 0x55a6615474d0 2024-04-21T08:50:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.744+0000 7ff714d83740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a6614494b0 con 0x55a661550a00 2024-04-21T08:50:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.744+0000 7ff714d83740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a6614a81b0 con 0x55a661547e80 2024-04-21T08:50:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.744+0000 7ff70bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6615474d0 0x55a6615478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.744+0000 7ff70bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6615474d0 0x55a6615478b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39482/0 (socket says 172.21.15.43:39482) 2024-04-21T08:50:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.744+0000 7ff70bfff640 1 -- 172.21.15.43:0/3985792504 learned_addr learned my addr 172.21.15.43:0/3985792504 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.744+0000 7ff708ff9640 1 --2- 172.21.15.43:0/3682602626 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff6f40a3d20 0x7ff6f40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.744+0000 7ff70b7fe640 1 --2- 172.21.15.43:0/3985792504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a661547e80 0x55a661550430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.744+0000 7ff70affd640 1 --2- 172.21.15.43:0/3985792504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a661550a00 0x55a661552e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.744+0000 7ff70bfff640 1 --2- 172.21.15.43:0/3682602626 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff6f40a3d20 0x7ff6f40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.744+0000 7ff708ff9640 1 -- 172.21.15.43:0/3682602626 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7ff7000d07e0 con 0x7ff70c06aef0 2024-04-21T08:50:07.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.744+0000 7ff70bfff640 1 -- 172.21.15.43:0/3985792504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a661547e80 msgr2=0x55a661550430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.744+0000 7ff70bfff640 1 --2- 172.21.15.43:0/3985792504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a661547e80 0x55a661550430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.744+0000 7ff70bfff640 1 -- 172.21.15.43:0/3985792504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a661550a00 msgr2=0x55a661552e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.744+0000 7ff70bfff640 1 --2- 172.21.15.43:0/3985792504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a661550a00 0x55a661552e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.744+0000 7ff70bfff640 1 -- 172.21.15.43:0/3985792504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a66143b1b0 con 0x55a6615474d0 2024-04-21T08:50:07.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.745+0000 7ff70bfff640 1 --2- 172.21.15.43:0/3985792504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6615474d0 0x55a6615478b0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7ff704009430 tx=0x7ff704032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1835f40e9bb4399 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.745+0000 7ff6f1ffb640 1 -- 172.21.15.43:0/3985792504 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff704050070 con 0x55a6615474d0 2024-04-21T08:50:07.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.745+0000 7ff6f1ffb640 1 -- 172.21.15.43:0/3985792504 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff704003b70 con 0x55a6615474d0 2024-04-21T08:50:07.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.745+0000 7ff6f1ffb640 1 -- 172.21.15.43:0/3985792504 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff704039040 con 0x55a6615474d0 2024-04-21T08:50:07.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.745+0000 7ff70bfff640 1 --2- 172.21.15.43:0/3682602626 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff6f40a3d20 0x7ff6f40a61e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff70404a000 tx=0x7ff704048000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.745+0000 7ff714d83740 1 -- 172.21.15.43:0/3985792504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6615474d0 msgr2=0x55a6615478b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.745+0000 7ff714d83740 1 --2- 172.21.15.43:0/3985792504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6615474d0 0x55a6615478b0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7ff704009430 tx=0x7ff704032bc0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.746+0000 7ff714d83740 1 -- 172.21.15.43:0/3985792504 shutdown_connections 2024-04-21T08:50:07.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.746+0000 7ff714d83740 1 --2- 172.21.15.43:0/3985792504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a661550a00 0x55a661552e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.746+0000 7ff714d83740 1 --2- 172.21.15.43:0/3985792504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a661547e80 0x55a661550430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.746+0000 7ff714d83740 1 --2- 172.21.15.43:0/3985792504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6615474d0 0x55a6615478b0 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.746+0000 7ff714d83740 1 -- 172.21.15.43:0/3985792504 >> 172.21.15.43:0/3985792504 conn(0x55a661541ba0 msgr2=0x55a661541f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.746+0000 7ff714d83740 1 -- 172.21.15.43:0/3985792504 shutdown_connections 2024-04-21T08:50:07.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.746+0000 7ff714d83740 1 -- 172.21.15.43:0/3985792504 wait complete. 2024-04-21T08:50:07.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.746+0000 7ff714d83740 1 -- start start 2024-04-21T08:50:07.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.746+0000 7ff714d83740 1 Processor -- start 2024-04-21T08:50:07.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.746+0000 7ff714d83740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6615474d0 0x55a661550310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.746+0000 7ff714d83740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a661547e80 0x55a66154d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.746+0000 7ff714d83740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a661550a00 0x55a66154dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.746+0000 7ff714d83740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a661553970 con 0x55a6615474d0 2024-04-21T08:50:07.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.746+0000 7ff714d83740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a661465500 con 0x55a661547e80 2024-04-21T08:50:07.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.746+0000 7ff714d83740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a661553af0 con 0x55a661550a00 2024-04-21T08:50:07.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.746+0000 7ff714d83740 10 client.0 _ll_register_callbacks cb 0x55a661433b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:07.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.747+0000 7ff70b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6615474d0 0x55a661550310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.747+0000 7ff70b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6615474d0 0x55a661550310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39486/0 (socket says 172.21.15.43:39486) 2024-04-21T08:50:07.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.747+0000 7ff70b7fe640 1 -- 172.21.15.43:0/1880787052 learned_addr learned my addr 172.21.15.43:0/1880787052 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:07.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.747+0000 7ff70affd640 1 --2- 172.21.15.43:0/1880787052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a661547e80 0x55a66154d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.747+0000 7ff70bfff640 1 --2- 172.21.15.43:0/1880787052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a661550a00 0x55a66154dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.747+0000 7ff70b7fe640 1 -- 172.21.15.43:0/1880787052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a661550a00 msgr2=0x55a66154dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.747+0000 7ff70b7fe640 1 --2- 172.21.15.43:0/1880787052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a661550a00 0x55a66154dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.747+0000 7ff70b7fe640 1 -- 172.21.15.43:0/1880787052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a661547e80 msgr2=0x55a66154d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.747+0000 7ff70b7fe640 1 --2- 172.21.15.43:0/1880787052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a661547e80 0x55a66154d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.747+0000 7ff70b7fe640 1 -- 172.21.15.43:0/1880787052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6611be3f0 con 0x55a6615474d0 2024-04-21T08:50:07.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.747+0000 7ff70bfff640 1 --2- 172.21.15.43:0/1880787052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a661550a00 0x55a66154dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:07.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.747+0000 7ff70b7fe640 1 --2- 172.21.15.43:0/1880787052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6615474d0 0x55a661550310 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc012700 tx=0x7ff6fc012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf9b7a9492d5078 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.747+0000 7ff6eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:07.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.747+0000 7ff6eaffd640 1 -- 172.21.15.43:0/1880787052 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc004020 con 0x55a6615474d0 2024-04-21T08:50:07.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.747+0000 7ff6eaffd640 1 -- 172.21.15.43:0/1880787052 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7ff6fc0041c0 con 0x55a6615474d0 2024-04-21T08:50:07.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.747+0000 7ff714d83740 1 -- 172.21.15.43:0/1880787052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff704003b70 con 0x55a6615474d0 2024-04-21T08:50:07.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.747+0000 7ff6eaffd640 1 -- 172.21.15.43:0/1880787052 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff6fc0043c0 con 0x55a6615474d0 2024-04-21T08:50:07.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.747+0000 7ff714d83740 10 client.20898 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:07.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.747+0000 7ff714d83740 1 -- 172.21.15.43:0/1880787052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a6612887d0 con 0x55a6615474d0 2024-04-21T08:50:07.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.748+0000 7ff6eaffd640 1 -- 172.21.15.43:0/1880787052 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff704003b70 con 0x55a6615474d0 2024-04-21T08:50:07.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.748+0000 7ff6eaffd640 1 -- 172.21.15.43:0/1880787052 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff6fc003a10 con 0x55a6615474d0 2024-04-21T08:50:07.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.748+0000 7ff714d83740 10 client.20898 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:07.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.748+0000 7ff714d83740 10 client.20898 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:50:07.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.748+0000 7ff714d83740 20 client.20898 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:07.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.748+0000 7ff714d83740 1 --2- 172.21.15.43:0/1880787052 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55a661560cd0 0x55a661563120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.748+0000 7ff714d83740 4 client.20898 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:07.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.748+0000 7ff714d83740 1 -- 172.21.15.43:0/1880787052 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a66144b930 con 0x55a661560cd0 2024-04-21T08:50:07.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.748+0000 7ff70affd640 1 --2- 172.21.15.43:0/1880787052 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55a661560cd0 0x55a661563120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.748+0000 7ff6eaffd640 10 client.20898 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:50:07.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.748+0000 7ff70affd640 1 --2- 172.21.15.43:0/1880787052 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55a661560cd0 0x55a661563120 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4b3080a70ddbf8ff server_cookie=313c54a51ac19c7f in_seq=0 out_seq=0 2024-04-21T08:50:07.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.749+0000 7ff6eaffd640 1 -- 172.21.15.43:0/1880787052 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a66144b930 con 0x55a661560cd0 2024-04-21T08:50:07.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.749+0000 7ff6eaffd640 10 client.20898 handle_command_reply: tid=0 2024-04-21T08:50:07.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.749+0000 7ff714d83740 1 client.20898 shutdown 2024-04-21T08:50:07.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.749+0000 7ff714d83740 1 -- 172.21.15.43:0/1880787052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6615474d0 msgr2=0x55a661550310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.749+0000 7ff714d83740 1 --2- 172.21.15.43:0/1880787052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6615474d0 0x55a661550310 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc012700 tx=0x7ff6fc012bc0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.749+0000 7ff714d83740 1 -- 172.21.15.43:0/1880787052 shutdown_connections 2024-04-21T08:50:07.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.749+0000 7ff714d83740 1 --2- 172.21.15.43:0/1880787052 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55a661560cd0 0x55a661563120 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.749+0000 7ff714d83740 1 --2- 172.21.15.43:0/1880787052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a661550a00 0x55a66154dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.749+0000 7ff714d83740 1 --2- 172.21.15.43:0/1880787052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a661547e80 0x55a66154d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.749+0000 7ff6eaffd640 0 client.20898 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:50:07.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.749+0000 7ff714d83740 1 --2- 172.21.15.43:0/1880787052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6615474d0 0x55a661550310 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.749+0000 7ff714d83740 1 -- 172.21.15.43:0/1880787052 >> 172.21.15.43:0/1880787052 conn(0x55a661541ba0 msgr2=0x55a6615438f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.750+0000 7ff714d83740 1 -- 172.21.15.43:0/1880787052 shutdown_connections 2024-04-21T08:50:07.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.750+0000 7ff714d83740 1 -- 172.21.15.43:0/1880787052 wait complete. 2024-04-21T08:50:07.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.750+0000 7ff714d83740 20 client.20898 trim_cache size 0 max 16384 2024-04-21T08:50:07.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.763+0000 7ff714d83740 1 Processor -- start 2024-04-21T08:50:07.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.763+0000 7ff714d83740 1 -- start start 2024-04-21T08:50:07.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.763+0000 7ff714d83740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6615474d0 0x55a66155c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.763+0000 7ff714d83740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a661547e80 0x55a661540a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.763+0000 7ff714d83740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a661550a00 0x55a6615689f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.763+0000 7ff714d83740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c1b18c0 con 0x55a6615474d0 2024-04-21T08:50:07.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.763+0000 7ff714d83740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c1b3400 con 0x55a661547e80 2024-04-21T08:50:07.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.763+0000 7ff714d83740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c1b1f90 con 0x55a661550a00 2024-04-21T08:50:07.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.763+0000 7ff70b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6615474d0 0x55a66155c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.763+0000 7ff70b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6615474d0 0x55a66155c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39488/0 (socket says 172.21.15.43:39488) 2024-04-21T08:50:07.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.763+0000 7ff70b7fe640 1 -- 172.21.15.43:0/2447896739 learned_addr learned my addr 172.21.15.43:0/2447896739 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:07.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.763+0000 7ff70b7fe640 1 -- 172.21.15.43:0/2447896739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a661550a00 msgr2=0x55a6615689f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.763+0000 7ff70b7fe640 1 --2- 172.21.15.43:0/2447896739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a661550a00 0x55a6615689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.763+0000 7ff70b7fe640 1 -- 172.21.15.43:0/2447896739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a661547e80 msgr2=0x55a661540a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.763+0000 7ff70b7fe640 1 --2- 172.21.15.43:0/2447896739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a661547e80 0x55a661540a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.763+0000 7ff70b7fe640 1 -- 172.21.15.43:0/2447896739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a661549270 con 0x55a6615474d0 2024-04-21T08:50:07.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.764+0000 7ff70b7fe640 1 --2- 172.21.15.43:0/2447896739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6615474d0 0x55a66155c890 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc005780 tx=0x7ff6fc005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696eb4af20777bfa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.764+0000 7ff6f17fa640 1 -- 172.21.15.43:0/2447896739 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc01c5d0 con 0x55a6615474d0 2024-04-21T08:50:07.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.764+0000 7ff6f17fa640 1 -- 172.21.15.43:0/2447896739 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6fc00fdd0 con 0x55a6615474d0 2024-04-21T08:50:07.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.764+0000 7ff6f17fa640 1 -- 172.21.15.43:0/2447896739 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc01ce20 con 0x55a6615474d0 2024-04-21T08:50:07.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.764+0000 7ff714d83740 1 -- 172.21.15.43:0/2447896739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6615474d0 msgr2=0x55a66155c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.764+0000 7ff714d83740 1 --2- 172.21.15.43:0/2447896739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6615474d0 0x55a66155c890 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc005780 tx=0x7ff6fc005db0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.764+0000 7ff714d83740 1 -- 172.21.15.43:0/2447896739 shutdown_connections 2024-04-21T08:50:07.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.764+0000 7ff714d83740 1 --2- 172.21.15.43:0/2447896739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a661550a00 0x55a6615689f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.764+0000 7ff714d83740 1 --2- 172.21.15.43:0/2447896739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a661547e80 0x55a661540a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.764+0000 7ff714d83740 1 --2- 172.21.15.43:0/2447896739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6615474d0 0x55a66155c890 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.764+0000 7ff714d83740 1 -- 172.21.15.43:0/2447896739 >> 172.21.15.43:0/2447896739 conn(0x55a661541ba0 msgr2=0x55a661540370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.764+0000 7ff714d83740 1 -- 172.21.15.43:0/2447896739 shutdown_connections 2024-04-21T08:50:07.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.764+0000 7ff714d83740 1 -- 172.21.15.43:0/2447896739 wait complete. 2024-04-21T08:50:07.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.765+0000 7ff714d83740 1 -- start start 2024-04-21T08:50:07.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.765+0000 7ff714d83740 1 Processor -- start 2024-04-21T08:50:07.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.765+0000 7ff714d83740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6615474d0 0x55a6615548a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.765+0000 7ff714d83740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a661547e80 0x55a661554de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.765+0000 7ff714d83740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a661550a00 0x55a661555320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.765+0000 7ff714d83740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c1b2d30 con 0x55a661547e80 2024-04-21T08:50:07.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.765+0000 7ff714d83740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c1b2660 con 0x55a661550a00 2024-04-21T08:50:07.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.765+0000 7ff714d83740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c1af8c0 con 0x55a6615474d0 2024-04-21T08:50:07.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.765+0000 7ff714d83740 10 client.0 _ll_register_callbacks cb 0x7ff6fc009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:07.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.765+0000 7ff70affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a661547e80 0x55a661554de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.765+0000 7ff70affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a661547e80 0x55a661554de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39500/0 (socket says 172.21.15.43:39500) 2024-04-21T08:50:07.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.765+0000 7ff70affd640 1 -- 172.21.15.43:0/4091923136 learned_addr learned my addr 172.21.15.43:0/4091923136 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:07.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.766+0000 7ff70b7fe640 1 --2- 172.21.15.43:0/4091923136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6615474d0 0x55a6615548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.766+0000 7ff70affd640 1 -- 172.21.15.43:0/4091923136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6615474d0 msgr2=0x55a6615548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.766+0000 7ff70affd640 1 --2- 172.21.15.43:0/4091923136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6615474d0 0x55a6615548a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.766+0000 7ff70bfff640 1 --2- 172.21.15.43:0/4091923136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a661550a00 0x55a661555320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.766+0000 7ff70affd640 1 -- 172.21.15.43:0/4091923136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a661550a00 msgr2=0x55a661555320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.766+0000 7ff70affd640 1 --2- 172.21.15.43:0/4091923136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a661550a00 0x55a661555320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.766+0000 7ff70affd640 1 -- 172.21.15.43:0/4091923136 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6615c6330 con 0x55a661547e80 2024-04-21T08:50:07.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.766+0000 7ff70bfff640 1 --2- 172.21.15.43:0/4091923136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a661550a00 0x55a661555320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:07.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.766+0000 7ff70affd640 1 --2- 172.21.15.43:0/4091923136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a661547e80 0x55a661554de0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7ff7000cdab0 tx=0x7ff700066ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be110f63abd6901 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.766+0000 7ff6eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:07.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.766+0000 7ff6eaffd640 1 -- 172.21.15.43:0/4091923136 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700020040 con 0x55a661547e80 2024-04-21T08:50:07.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.766+0000 7ff6eaffd640 1 -- 172.21.15.43:0/4091923136 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7ff7000cdcd0 con 0x55a661547e80 2024-04-21T08:50:07.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.766+0000 7ff714d83740 1 -- 172.21.15.43:0/4091923136 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6fc00fdd0 con 0x55a661547e80 2024-04-21T08:50:07.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.766+0000 7ff6eaffd640 1 -- 172.21.15.43:0/4091923136 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff7000b5050 con 0x55a661547e80 2024-04-21T08:50:07.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.766+0000 7ff714d83740 10 client.20904 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:07.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.766+0000 7ff714d83740 1 -- 172.21.15.43:0/4091923136 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a6615c7690 con 0x55a661547e80 2024-04-21T08:50:07.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.766+0000 7ff6eaffd640 1 -- 172.21.15.43:0/4091923136 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6fc00fdd0 con 0x55a661547e80 2024-04-21T08:50:07.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.766+0000 7ff6eaffd640 1 -- 172.21.15.43:0/4091923136 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff70009b7a0 con 0x55a661547e80 2024-04-21T08:50:07.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.766+0000 7ff714d83740 10 client.20904 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:07.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.766+0000 7ff714d83740 10 client.20904 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:50:07.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.766+0000 7ff714d83740 20 client.20904 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:07.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.766+0000 7ff714d83740 1 --2- 172.21.15.43:0/4091923136 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55a6615c7df0 0x55a6615ca210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.766+0000 7ff714d83740 4 client.20904 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:50:07.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.766+0000 7ff714d83740 1 -- 172.21.15.43:0/4091923136 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a6614b9c10 con 0x55a6615c7df0 2024-04-21T08:50:07.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.767+0000 7ff70b7fe640 1 --2- 172.21.15.43:0/4091923136 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55a6615c7df0 0x55a6615ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.767+0000 7ff6eaffd640 10 client.20904 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:50:07.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.767+0000 7ff70b7fe640 1 --2- 172.21.15.43:0/4091923136 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55a6615c7df0 0x55a6615ca210 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=acb3ea09fcc889e server_cookie=ece12ee9341c8f77 in_seq=0 out_seq=0 2024-04-21T08:50:07.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff6eaffd640 1 -- 172.21.15.43:0/4091923136 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a6614b9c10 con 0x55a6615c7df0 2024-04-21T08:50:07.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff6eaffd640 10 client.20904 handle_command_reply: tid=0 2024-04-21T08:50:07.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff714d83740 1 client.20904 shutdown 2024-04-21T08:50:07.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff714d83740 1 -- 172.21.15.43:0/4091923136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a661547e80 msgr2=0x55a661554de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff714d83740 1 --2- 172.21.15.43:0/4091923136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a661547e80 0x55a661554de0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7ff7000cdab0 tx=0x7ff700066ea0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff714d83740 1 -- 172.21.15.43:0/4091923136 shutdown_connections 2024-04-21T08:50:07.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff714d83740 1 --2- 172.21.15.43:0/4091923136 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x55a6615c7df0 0x55a6615ca210 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff714d83740 1 --2- 172.21.15.43:0/4091923136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a661550a00 0x55a661555320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff714d83740 1 --2- 172.21.15.43:0/4091923136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a661547e80 0x55a661554de0 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff6eaffd640 0 client.20904 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:50:07.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff714d83740 1 --2- 172.21.15.43:0/4091923136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a6615474d0 0x55a6615548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff714d83740 1 -- 172.21.15.43:0/4091923136 >> 172.21.15.43:0/4091923136 conn(0x55a661541ba0 msgr2=0x55a661548af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff714d83740 1 -- 172.21.15.43:0/4091923136 shutdown_connections 2024-04-21T08:50:07.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff714d83740 1 -- 172.21.15.43:0/4091923136 wait complete. 2024-04-21T08:50:07.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff714d83740 20 client.20904 trim_cache size 0 max 16384 2024-04-21T08:50:07.773 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:50:07.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff712196640 1 -- 172.21.15.43:0/3682602626 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff6f40a3d20 msgr2=0x7ff6f40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff712196640 1 --2- 172.21.15.43:0/3682602626 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff6f40a3d20 0x7ff6f40a61e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff70404a000 tx=0x7ff704048000 comp rx=0 tx=0).stop 2024-04-21T08:50:07.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff712196640 1 -- 172.21.15.43:0/3682602626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70c06aef0 msgr2=0x7ff70c1a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff712196640 1 --2- 172.21.15.43:0/3682602626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70c06aef0 0x7ff70c1a0b90 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7ff70000d950 tx=0x7ff70000de10 comp rx=0 tx=0).stop 2024-04-21T08:50:07.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff712196640 1 -- 172.21.15.43:0/3682602626 shutdown_connections 2024-04-21T08:50:07.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff712196640 1 --2- 172.21.15.43:0/3682602626 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff6f40a3d20 0x7ff6f40a61e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff712196640 1 --2- 172.21.15.43:0/3682602626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff70c10e960 0x7ff70c19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff712196640 1 --2- 172.21.15.43:0/3682602626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70c06aef0 0x7ff70c1a0b90 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff712196640 1 --2- 172.21.15.43:0/3682602626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff70c06a5d0 0x7ff70c1a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.768+0000 7ff712196640 1 -- 172.21.15.43:0/3682602626 >> 172.21.15.43:0/3682602626 conn(0x7ff70c077eb0 msgr2=0x7ff70c10fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.769+0000 7ff712196640 1 -- 172.21.15.43:0/3682602626 shutdown_connections 2024-04-21T08:50:07.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.769+0000 7ff712196640 1 -- 172.21.15.43:0/3682602626 wait complete. 2024-04-21T08:50:07.779 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:50:07.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.884+0000 7f800625c640 1 -- 172.21.15.43:0/947725971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80001072e0 msgr2=0x7f800010eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.884+0000 7f800625c640 1 --2- 172.21.15.43:0/947725971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80001072e0 0x7f800010eeb0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009ca0 tx=0x7f7ff802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.884+0000 7f800625c640 1 -- 172.21.15.43:0/947725971 shutdown_connections 2024-04-21T08:50:07.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.884+0000 7f800625c640 1 --2- 172.21.15.43:0/947725971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80001072e0 0x7f800010eeb0 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.884+0000 7f800625c640 1 --2- 172.21.15.43:0/947725971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8000102bc0 0x7f8000106c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.884+0000 7f800625c640 1 --2- 172.21.15.43:0/947725971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8000102210 0x7f80001025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.884+0000 7f800625c640 1 -- 172.21.15.43:0/947725971 >> 172.21.15.43:0/947725971 conn(0x7f80000fdab0 msgr2=0x7f80000ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.884+0000 7f800625c640 1 -- 172.21.15.43:0/947725971 shutdown_connections 2024-04-21T08:50:07.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.884+0000 7f800625c640 1 -- 172.21.15.43:0/947725971 wait complete. 2024-04-21T08:50:07.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.884+0000 7f800625c640 1 Processor -- start 2024-04-21T08:50:07.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.885+0000 7f800625c640 1 -- start start 2024-04-21T08:50:07.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.885+0000 7f800625c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8000102210 0x7f8000074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.885+0000 7f800625c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80001072e0 0x7f8000074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.885+0000 7f800625c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8000074cd0 0x7f80001a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.885+0000 7f800625c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8000112230 con 0x7f80001072e0 2024-04-21T08:50:07.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.885+0000 7f800625c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80001120b0 con 0x7f8000074cd0 2024-04-21T08:50:07.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.885+0000 7f800625c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80001123b0 con 0x7f8000102210 2024-04-21T08:50:07.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.885+0000 7f7ffeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80001072e0 0x7f8000074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.885+0000 7f7ffeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80001072e0 0x7f8000074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39518/0 (socket says 172.21.15.43:39518) 2024-04-21T08:50:07.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.885+0000 7f7ffeffd640 1 -- 172.21.15.43:0/1003865973 learned_addr learned my addr 172.21.15.43:0/1003865973 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:07.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.885+0000 7f7ffffff640 1 --2- 172.21.15.43:0/1003865973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8000074cd0 0x7f80001a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.885+0000 7f7fff7fe640 1 --2- 172.21.15.43:0/1003865973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8000102210 0x7f8000074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.885+0000 7f7ffeffd640 1 -- 172.21.15.43:0/1003865973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8000102210 msgr2=0x7f8000074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.885+0000 7f7ffeffd640 1 --2- 172.21.15.43:0/1003865973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8000102210 0x7f8000074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.885+0000 7f7ffeffd640 1 -- 172.21.15.43:0/1003865973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8000074cd0 msgr2=0x7f80001a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.885+0000 7f7ffeffd640 1 --2- 172.21.15.43:0/1003865973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8000074cd0 0x7f80001a6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.885+0000 7f7ffeffd640 1 -- 172.21.15.43:0/1003865973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80001a7020 con 0x7f80001072e0 2024-04-21T08:50:07.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.885+0000 7f7ffeffd640 1 --2- 172.21.15.43:0/1003865973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80001072e0 0x7f8000074790 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400d770 tx=0x7f7ff400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.886+0000 7f7ffcff9640 1 -- 172.21.15.43:0/1003865973 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4017070 con 0x7f80001072e0 2024-04-21T08:50:07.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.886+0000 7f7ffcff9640 1 -- 172.21.15.43:0/1003865973 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ff4009c30 con 0x7f80001072e0 2024-04-21T08:50:07.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.886+0000 7f800625c640 1 -- 172.21.15.43:0/1003865973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80001a7310 con 0x7f80001072e0 2024-04-21T08:50:07.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.886+0000 7f7ffcff9640 1 -- 172.21.15.43:0/1003865973 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff400f620 con 0x7f80001072e0 2024-04-21T08:50:07.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.886+0000 7f800625c640 1 -- 172.21.15.43:0/1003865973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80001a7860 con 0x7f80001072e0 2024-04-21T08:50:07.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.888+0000 7f7ffcff9640 1 -- 172.21.15.43:0/1003865973 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7ff401a070 con 0x7f80001072e0 2024-04-21T08:50:07.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.888+0000 7f8008e54740 1 Processor -- start 2024-04-21T08:50:07.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.888+0000 7f8008e54740 1 -- start start 2024-04-21T08:50:07.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.888+0000 7f8008e54740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647a14454d0 0x5647a14458b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.888+0000 7f8008e54740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a1445e80 0x5647a144e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.888+0000 7f8008e54740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647a144ea00 0x5647a1450e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.888+0000 7f8008e54740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5647a13aa1f0 con 0x5647a1445e80 2024-04-21T08:50:07.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.888+0000 7f8008e54740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5647a13474b0 con 0x5647a144ea00 2024-04-21T08:50:07.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.888+0000 7f8008e54740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5647a13a61b0 con 0x5647a14454d0 2024-04-21T08:50:07.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.888+0000 7f7ffcff9640 1 --2- 172.21.15.43:0/1003865973 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7fe80a3d50 0x7f7fe80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.888+0000 7f7fff7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a1445e80 0x5647a144e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.888+0000 7f7fff7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a1445e80 0x5647a144e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39522/0 (socket says 172.21.15.43:39522) 2024-04-21T08:50:07.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.888+0000 7f7fff7fe640 1 -- 172.21.15.43:0/172745827 learned_addr learned my addr 172.21.15.43:0/172745827 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:07.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.889+0000 7f7ffffff640 1 --2- 172.21.15.43:0/1003865973 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7fe80a3d50 0x7f7fe80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.889+0000 7f7ffcff9640 1 -- 172.21.15.43:0/1003865973 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7f7ff40d0700 con 0x7f80001072e0 2024-04-21T08:50:07.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.889+0000 7f7ffeffd640 1 --2- 172.21.15.43:0/172745827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647a144ea00 0x5647a1450e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.889+0000 7f7ffffff640 1 --2- 172.21.15.43:0/172745827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647a14454d0 0x5647a14458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.889+0000 7f7fff7fe640 1 -- 172.21.15.43:0/172745827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647a14454d0 msgr2=0x5647a14458b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.889+0000 7f7fff7fe640 1 --2- 172.21.15.43:0/172745827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647a14454d0 0x5647a14458b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.889+0000 7f7fff7fe640 1 -- 172.21.15.43:0/172745827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647a144ea00 msgr2=0x5647a1450e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.889+0000 7f7fff7fe640 1 --2- 172.21.15.43:0/172745827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647a144ea00 0x5647a1450e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.889+0000 7f7fff7fe640 1 -- 172.21.15.43:0/172745827 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647a13391b0 con 0x5647a1445e80 2024-04-21T08:50:07.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.889+0000 7f7fff7fe640 1 --2- 172.21.15.43:0/172745827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a1445e80 0x5647a144e430 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000cc60 tx=0x7f7ff000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1be0f0902bdc84 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.889+0000 7f7ffffff640 1 --2- 172.21.15.43:0/1003865973 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7fe80a3d50 0x7f7fe80a6210 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8004af0 tx=0x7f7ff802e040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.889+0000 7f7fe5ffb640 1 -- 172.21.15.43:0/172745827 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0014070 con 0x5647a1445e80 2024-04-21T08:50:07.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.889+0000 7f7fe5ffb640 1 -- 172.21.15.43:0/172745827 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ff00047a0 con 0x5647a1445e80 2024-04-21T08:50:07.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.889+0000 7f7fe5ffb640 1 -- 172.21.15.43:0/172745827 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0010690 con 0x5647a1445e80 2024-04-21T08:50:07.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.890+0000 7f8008e54740 1 -- 172.21.15.43:0/172745827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a1445e80 msgr2=0x5647a144e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.890+0000 7f8008e54740 1 --2- 172.21.15.43:0/172745827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a1445e80 0x5647a144e430 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000cc60 tx=0x7f7ff000e720 comp rx=0 tx=0).stop 2024-04-21T08:50:07.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.890+0000 7f8008e54740 1 -- 172.21.15.43:0/172745827 shutdown_connections 2024-04-21T08:50:07.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.890+0000 7f8008e54740 1 --2- 172.21.15.43:0/172745827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647a144ea00 0x5647a1450e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.890+0000 7f8008e54740 1 --2- 172.21.15.43:0/172745827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a1445e80 0x5647a144e430 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.890+0000 7f8008e54740 1 --2- 172.21.15.43:0/172745827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647a14454d0 0x5647a14458b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.890+0000 7f8008e54740 1 -- 172.21.15.43:0/172745827 >> 172.21.15.43:0/172745827 conn(0x5647a143fba0 msgr2=0x5647a143ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.890+0000 7f8008e54740 1 -- 172.21.15.43:0/172745827 shutdown_connections 2024-04-21T08:50:07.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.890+0000 7f8008e54740 1 -- 172.21.15.43:0/172745827 wait complete. 2024-04-21T08:50:07.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.890+0000 7f8008e54740 1 -- start start 2024-04-21T08:50:07.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.890+0000 7f8008e54740 1 Processor -- start 2024-04-21T08:50:07.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.891+0000 7f8008e54740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647a14454d0 0x5647a144e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.891+0000 7f8008e54740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647a1445e80 0x5647a144b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.891+0000 7f8008e54740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a144ea00 0x5647a144bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.891+0000 7f8008e54740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5647a1451970 con 0x5647a144ea00 2024-04-21T08:50:07.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.891+0000 7f8008e54740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5647a1363500 con 0x5647a14454d0 2024-04-21T08:50:07.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.891+0000 7f8008e54740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5647a1451af0 con 0x5647a1445e80 2024-04-21T08:50:07.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.891+0000 7f8008e54740 10 client.0 _ll_register_callbacks cb 0x5647a1331b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:07.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.891+0000 7f7ffffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a144ea00 0x5647a144bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.891+0000 7f7ffffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a144ea00 0x5647a144bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39538/0 (socket says 172.21.15.43:39538) 2024-04-21T08:50:07.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.891+0000 7f7ffffff640 1 -- 172.21.15.43:0/635440090 learned_addr learned my addr 172.21.15.43:0/635440090 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:07.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.891+0000 7f7fff7fe640 1 --2- 172.21.15.43:0/635440090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647a14454d0 0x5647a144e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.891+0000 7f7ffeffd640 1 --2- 172.21.15.43:0/635440090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647a1445e80 0x5647a144b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.891+0000 7f7ffffff640 1 -- 172.21.15.43:0/635440090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647a1445e80 msgr2=0x5647a144b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.891+0000 7f7ffffff640 1 --2- 172.21.15.43:0/635440090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647a1445e80 0x5647a144b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.891+0000 7f7ffffff640 1 -- 172.21.15.43:0/635440090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647a14454d0 msgr2=0x5647a144e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.891+0000 7f7ffffff640 1 --2- 172.21.15.43:0/635440090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647a14454d0 0x5647a144e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.891+0000 7f7ffffff640 1 -- 172.21.15.43:0/635440090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647a10bc3f0 con 0x5647a144ea00 2024-04-21T08:50:07.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.891+0000 7f7ffffff640 1 --2- 172.21.15.43:0/635440090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a144ea00 0x5647a144bbd0 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8036000 tx=0x7f7ff802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1173c4fc49447a0b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.891+0000 7f7fdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:07.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.891+0000 7f7fdeffd640 1 -- 172.21.15.43:0/635440090 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8051020 con 0x5647a144ea00 2024-04-21T08:50:07.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.891+0000 7f7fdeffd640 1 -- 172.21.15.43:0/635440090 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7f7ff8004450 con 0x5647a144ea00 2024-04-21T08:50:07.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.892+0000 7f8008e54740 1 -- 172.21.15.43:0/635440090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ff00047a0 con 0x5647a144ea00 2024-04-21T08:50:07.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.892+0000 7f7fdeffd640 1 -- 172.21.15.43:0/635440090 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7ff805c050 con 0x5647a144ea00 2024-04-21T08:50:07.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.892+0000 7f8008e54740 10 client.20916 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:07.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.892+0000 7f8008e54740 1 -- 172.21.15.43:0/635440090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5647a11867d0 con 0x5647a144ea00 2024-04-21T08:50:07.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.892+0000 7f7fdeffd640 1 -- 172.21.15.43:0/635440090 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ff00047a0 con 0x5647a144ea00 2024-04-21T08:50:07.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.892+0000 7f7fdeffd640 1 -- 172.21.15.43:0/635440090 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7ff803f070 con 0x5647a144ea00 2024-04-21T08:50:07.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.892+0000 7f8008e54740 10 client.20916 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:07.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.892+0000 7f8008e54740 10 client.20916 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:50:07.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.892+0000 7f8008e54740 20 client.20916 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:07.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.892+0000 7f8008e54740 1 --2- 172.21.15.43:0/635440090 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5647a145ec70 0x5647a14610a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.892+0000 7f8008e54740 4 client.20916 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:07.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.892+0000 7f8008e54740 1 -- 172.21.15.43:0/635440090 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647a1349930 con 0x5647a145ec70 2024-04-21T08:50:07.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.892+0000 7f7fff7fe640 1 --2- 172.21.15.43:0/635440090 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5647a145ec70 0x5647a14610a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.893+0000 7f7fff7fe640 1 --2- 172.21.15.43:0/635440090 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5647a145ec70 0x5647a14610a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=17c54f11567f2998 server_cookie=123b0651794ef04 in_seq=0 out_seq=0 2024-04-21T08:50:07.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.893+0000 7f7fdeffd640 10 client.20916 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:50:07.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.893+0000 7f7fdeffd640 1 -- 172.21.15.43:0/635440090 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5647a1349930 con 0x5647a145ec70 2024-04-21T08:50:07.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.893+0000 7f7fdeffd640 10 client.20916 handle_command_reply: tid=0 2024-04-21T08:50:07.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.893+0000 7f8008e54740 1 client.20916 shutdown 2024-04-21T08:50:07.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.894+0000 7f8008e54740 1 -- 172.21.15.43:0/635440090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a144ea00 msgr2=0x5647a144bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.894+0000 7f8008e54740 1 --2- 172.21.15.43:0/635440090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a144ea00 0x5647a144bbd0 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8036000 tx=0x7f7ff802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.894+0000 7f8008e54740 1 -- 172.21.15.43:0/635440090 shutdown_connections 2024-04-21T08:50:07.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.894+0000 7f8008e54740 1 --2- 172.21.15.43:0/635440090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a144ea00 0x5647a144bbd0 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.894+0000 7f8008e54740 1 --2- 172.21.15.43:0/635440090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647a1445e80 0x5647a144b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.894+0000 7f8008e54740 1 --2- 172.21.15.43:0/635440090 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5647a145ec70 0x5647a14610a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.894+0000 7f8008e54740 1 --2- 172.21.15.43:0/635440090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647a14454d0 0x5647a144e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.894+0000 7f8008e54740 1 -- 172.21.15.43:0/635440090 >> 172.21.15.43:0/635440090 conn(0x5647a143fba0 msgr2=0x5647a14418f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.894+0000 7f7fdeffd640 0 client.20916 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:50:07.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.894+0000 7f8008e54740 1 -- 172.21.15.43:0/635440090 shutdown_connections 2024-04-21T08:50:07.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.894+0000 7f8008e54740 1 -- 172.21.15.43:0/635440090 wait complete. 2024-04-21T08:50:07.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.894+0000 7f8008e54740 20 client.20916 trim_cache size 0 max 16384 2024-04-21T08:50:07.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.907+0000 7f8008e54740 1 Processor -- start 2024-04-21T08:50:07.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.907+0000 7f8008e54740 1 -- start start 2024-04-21T08:50:07.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.907+0000 7f8008e54740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647a14454d0 0x5647a145a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.907+0000 7f8008e54740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a1445e80 0x5647a144dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.907+0000 7f8008e54740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647a144ea00 0x5647a144e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.907+0000 7f8008e54740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f80001b17e0 con 0x5647a1445e80 2024-04-21T08:50:07.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.907+0000 7f8008e54740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80001b3320 con 0x5647a14454d0 2024-04-21T08:50:07.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.907+0000 7f8008e54740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80001b1eb0 con 0x5647a144ea00 2024-04-21T08:50:07.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.907+0000 7f7ffeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a1445e80 0x5647a144dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.907+0000 7f7ffeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a1445e80 0x5647a144dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39546/0 (socket says 172.21.15.43:39546) 2024-04-21T08:50:07.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.907+0000 7f7ffeffd640 1 -- 172.21.15.43:0/396179787 learned_addr learned my addr 172.21.15.43:0/396179787 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:07.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.907+0000 7f7ffffff640 1 --2- 172.21.15.43:0/396179787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647a144ea00 0x5647a144e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.908+0000 7f7fff7fe640 1 --2- 172.21.15.43:0/396179787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647a14454d0 0x5647a145a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.908+0000 7f7ffeffd640 1 -- 172.21.15.43:0/396179787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647a144ea00 msgr2=0x5647a144e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.908+0000 7f7ffeffd640 1 --2- 172.21.15.43:0/396179787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647a144ea00 0x5647a144e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.908+0000 7f7ffeffd640 1 -- 172.21.15.43:0/396179787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647a14454d0 msgr2=0x5647a145a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.908+0000 7f7ffeffd640 1 --2- 172.21.15.43:0/396179787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647a14454d0 0x5647a145a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.908+0000 7f7ffeffd640 1 -- 172.21.15.43:0/396179787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647a143f380 con 0x5647a1445e80 2024-04-21T08:50:07.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.908+0000 7f7ffffff640 1 --2- 172.21.15.43:0/396179787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647a144ea00 0x5647a144e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:07.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.908+0000 7f7fff7fe640 1 --2- 172.21.15.43:0/396179787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647a14454d0 0x5647a145a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:07.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.908+0000 7f7ffeffd640 1 --2- 172.21.15.43:0/396179787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a1445e80 0x5647a144dda0 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009610 tx=0x7f7ff409bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96942c5dd98b532c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.908+0000 7f7fe57fa640 1 -- 172.21.15.43:0/396179787 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff40b5070 con 0x5647a1445e80 2024-04-21T08:50:07.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.908+0000 7f7fe57fa640 1 -- 172.21.15.43:0/396179787 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ff40a05d0 con 0x5647a1445e80 2024-04-21T08:50:07.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.908+0000 7f7fe57fa640 1 -- 172.21.15.43:0/396179787 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4021040 con 0x5647a1445e80 2024-04-21T08:50:07.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.908+0000 7f8008e54740 1 -- 172.21.15.43:0/396179787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a1445e80 msgr2=0x5647a144dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.908+0000 7f8008e54740 1 --2- 172.21.15.43:0/396179787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a1445e80 0x5647a144dda0 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009610 tx=0x7f7ff409bb80 comp rx=0 tx=0).stop 2024-04-21T08:50:07.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.908+0000 7f8008e54740 1 -- 172.21.15.43:0/396179787 shutdown_connections 2024-04-21T08:50:07.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.908+0000 7f8008e54740 1 --2- 172.21.15.43:0/396179787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647a144ea00 0x5647a144e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.908+0000 7f8008e54740 1 --2- 172.21.15.43:0/396179787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a1445e80 0x5647a144dda0 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.908+0000 7f8008e54740 1 --2- 172.21.15.43:0/396179787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647a14454d0 0x5647a145a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.908+0000 7f8008e54740 1 -- 172.21.15.43:0/396179787 >> 172.21.15.43:0/396179787 conn(0x5647a143fba0 msgr2=0x5647a14462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.909+0000 7f8008e54740 1 -- 172.21.15.43:0/396179787 shutdown_connections 2024-04-21T08:50:07.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.909+0000 7f8008e54740 1 -- 172.21.15.43:0/396179787 wait complete. 2024-04-21T08:50:07.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.909+0000 7f8008e54740 1 -- start start 2024-04-21T08:50:07.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.909+0000 7f8008e54740 1 Processor -- start 2024-04-21T08:50:07.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.909+0000 7f8008e54740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647a14454d0 0x5647a1456f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.909+0000 7f8008e54740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a1445e80 0x5647a1447f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.909+0000 7f8008e54740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647a144ea00 0x5647a1448440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.909+0000 7f8008e54740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f80001b2c50 con 0x5647a1445e80 2024-04-21T08:50:07.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.909+0000 7f8008e54740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80001b2580 con 0x5647a144ea00 2024-04-21T08:50:07.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.909+0000 7f8008e54740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80001af7e0 con 0x5647a14454d0 2024-04-21T08:50:07.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.909+0000 7f8008e54740 10 client.0 _ll_register_callbacks cb 0x7f7ff80328a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:07.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.909+0000 7f7ffeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a1445e80 0x5647a1447f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.909+0000 7f7ffeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a1445e80 0x5647a1447f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39556/0 (socket says 172.21.15.43:39556) 2024-04-21T08:50:07.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.909+0000 7f7ffeffd640 1 -- 172.21.15.43:0/3967882591 learned_addr learned my addr 172.21.15.43:0/3967882591 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:07.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.910+0000 7f7fff7fe640 1 --2- 172.21.15.43:0/3967882591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647a14454d0 0x5647a1456f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.910+0000 7f7ffffff640 1 --2- 172.21.15.43:0/3967882591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647a144ea00 0x5647a1448440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.910+0000 7f7ffeffd640 1 -- 172.21.15.43:0/3967882591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647a14454d0 msgr2=0x5647a1456f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.910+0000 7f7ffeffd640 1 --2- 172.21.15.43:0/3967882591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647a14454d0 0x5647a1456f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.910+0000 7f7ffeffd640 1 -- 172.21.15.43:0/3967882591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647a144ea00 msgr2=0x5647a1448440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:07.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.910+0000 7f7ffeffd640 1 --2- 172.21.15.43:0/3967882591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647a144ea00 0x5647a1448440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.910+0000 7f7ffeffd640 1 -- 172.21.15.43:0/3967882591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647a1448980 con 0x5647a1445e80 2024-04-21T08:50:07.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.910+0000 7f7ffeffd640 1 --2- 172.21.15.43:0/3967882591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a1445e80 0x5647a1447f00 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4098bf0 tx=0x7f7ff409bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66cf2d303c045193 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:07.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.910+0000 7f7fdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:07.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.910+0000 7f7fdeffd640 1 -- 172.21.15.43:0/3967882591 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4080060 con 0x5647a1445e80 2024-04-21T08:50:07.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.910+0000 7f7fdeffd640 1 -- 172.21.15.43:0/3967882591 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7f7ff40a0370 con 0x5647a1445e80 2024-04-21T08:50:07.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.910+0000 7f8008e54740 1 -- 172.21.15.43:0/3967882591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ff40a05d0 con 0x5647a1445e80 2024-04-21T08:50:07.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.910+0000 7f7fdeffd640 1 -- 172.21.15.43:0/3967882591 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7ff4021040 con 0x5647a1445e80 2024-04-21T08:50:07.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.910+0000 7f8008e54740 10 client.20922 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:07.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.910+0000 7f8008e54740 1 -- 172.21.15.43:0/3967882591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5647a14c4dc0 con 0x5647a1445e80 2024-04-21T08:50:07.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.910+0000 7f7fdeffd640 1 -- 172.21.15.43:0/3967882591 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ff40a05d0 con 0x5647a1445e80 2024-04-21T08:50:07.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.911+0000 7f7fdeffd640 1 -- 172.21.15.43:0/3967882591 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7ff4098000 con 0x5647a1445e80 2024-04-21T08:50:07.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.911+0000 7f8008e54740 10 client.20922 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:07.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.911+0000 7f8008e54740 10 client.20922 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:50:07.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.911+0000 7f8008e54740 20 client.20922 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:07.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.911+0000 7f8008e54740 1 --2- 172.21.15.43:0/3967882591 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5647a14c5460 0x5647a14c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:07.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.911+0000 7f8008e54740 4 client.20922 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:50:07.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.911+0000 7f8008e54740 1 -- 172.21.15.43:0/3967882591 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5647a13b7c10 con 0x5647a14c5460 2024-04-21T08:50:07.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.911+0000 7f7fff7fe640 1 --2- 172.21.15.43:0/3967882591 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5647a14c5460 0x5647a14c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:07.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.911+0000 7f7fff7fe640 1 --2- 172.21.15.43:0/3967882591 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5647a14c5460 0x5647a14c7880 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f1b8939b52fd155 server_cookie=51410909c46e14d6 in_seq=0 out_seq=0 2024-04-21T08:50:07.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.911+0000 7f7fdeffd640 10 client.20922 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:50:07.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.911+0000 7f7fdeffd640 1 -- 172.21.15.43:0/3967882591 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5647a13b7c10 con 0x5647a14c5460 2024-04-21T08:50:07.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.911+0000 7f7fdeffd640 10 client.20922 handle_command_reply: tid=0 2024-04-21T08:50:07.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.911+0000 7f8008e54740 1 client.20922 shutdown 2024-04-21T08:50:07.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.912+0000 7f8008e54740 1 -- 172.21.15.43:0/3967882591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a1445e80 msgr2=0x5647a1447f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.912+0000 7f8008e54740 1 --2- 172.21.15.43:0/3967882591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a1445e80 0x5647a1447f00 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4098bf0 tx=0x7f7ff409bcb0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.912+0000 7f8008e54740 1 -- 172.21.15.43:0/3967882591 shutdown_connections 2024-04-21T08:50:07.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.912+0000 7f8008e54740 1 --2- 172.21.15.43:0/3967882591 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x5647a14c5460 0x5647a14c7880 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.912+0000 7f8008e54740 1 --2- 172.21.15.43:0/3967882591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647a144ea00 0x5647a1448440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.912+0000 7f8008e54740 1 --2- 172.21.15.43:0/3967882591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5647a1445e80 0x5647a1447f00 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.912+0000 7f8008e54740 1 --2- 172.21.15.43:0/3967882591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5647a14454d0 0x5647a1456f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.912+0000 7f7fdeffd640 0 client.20922 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:50:07.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.912+0000 7f8008e54740 1 -- 172.21.15.43:0/3967882591 >> 172.21.15.43:0/3967882591 conn(0x5647a143fba0 msgr2=0x5647a1449b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.912+0000 7f8008e54740 1 -- 172.21.15.43:0/3967882591 shutdown_connections 2024-04-21T08:50:07.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.912+0000 7f8008e54740 1 -- 172.21.15.43:0/3967882591 wait complete. 2024-04-21T08:50:07.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.912+0000 7f8008e54740 20 client.20922 trim_cache size 0 max 16384 2024-04-21T08:50:07.924 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:50:07.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.912+0000 7f800625c640 1 -- 172.21.15.43:0/1003865973 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7fe80a3d50 msgr2=0x7f7fe80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.912+0000 7f800625c640 1 --2- 172.21.15.43:0/1003865973 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7fe80a3d50 0x7f7fe80a6210 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8004af0 tx=0x7f7ff802e040 comp rx=0 tx=0).stop 2024-04-21T08:50:07.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.912+0000 7f800625c640 1 -- 172.21.15.43:0/1003865973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80001072e0 msgr2=0x7f8000074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:07.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.912+0000 7f800625c640 1 --2- 172.21.15.43:0/1003865973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80001072e0 0x7f8000074790 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400d770 tx=0x7f7ff400dc30 comp rx=0 tx=0).stop 2024-04-21T08:50:07.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.912+0000 7f800625c640 1 -- 172.21.15.43:0/1003865973 shutdown_connections 2024-04-21T08:50:07.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.912+0000 7f800625c640 1 --2- 172.21.15.43:0/1003865973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8000074cd0 0x7f80001a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.912+0000 7f800625c640 1 --2- 172.21.15.43:0/1003865973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80001072e0 0x7f8000074790 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.912+0000 7f800625c640 1 --2- 172.21.15.43:0/1003865973 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7fe80a3d50 0x7f7fe80a6210 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.912+0000 7f800625c640 1 --2- 172.21.15.43:0/1003865973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8000102210 0x7f8000074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:07.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.912+0000 7f800625c640 1 -- 172.21.15.43:0/1003865973 >> 172.21.15.43:0/1003865973 conn(0x7f80000fdab0 msgr2=0x7f80000ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:07.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.913+0000 7f800625c640 1 -- 172.21.15.43:0/1003865973 shutdown_connections 2024-04-21T08:50:07.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:07.913+0000 7f800625c640 1 -- 172.21.15.43:0/1003865973 wait complete. 2024-04-21T08:50:07.926 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:50:08.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.032+0000 7f54231b2640 1 -- 172.21.15.43:0/2221221203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f541c102190 msgr2=0x7f541c102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.032+0000 7f54231b2640 1 --2- 172.21.15.43:0/2221221203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f541c102190 0x7f541c102570 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f5418009900 tx=0x7f541802bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.032+0000 7f54231b2640 1 -- 172.21.15.43:0/2221221203 shutdown_connections 2024-04-21T08:50:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.032+0000 7f54231b2640 1 --2- 172.21.15.43:0/2221221203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f541c107260 0x7f541c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.032+0000 7f54231b2640 1 --2- 172.21.15.43:0/2221221203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f541c102b40 0x7f541c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.032+0000 7f54231b2640 1 --2- 172.21.15.43:0/2221221203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f541c102190 0x7f541c102570 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.032+0000 7f54231b2640 1 -- 172.21.15.43:0/2221221203 >> 172.21.15.43:0/2221221203 conn(0x7f541c0793a0 msgr2=0x7f541c0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.032+0000 7f54231b2640 1 -- 172.21.15.43:0/2221221203 shutdown_connections 2024-04-21T08:50:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.032+0000 7f54231b2640 1 -- 172.21.15.43:0/2221221203 wait complete. 2024-04-21T08:50:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.033+0000 7f54231b2640 1 Processor -- start 2024-04-21T08:50:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.033+0000 7f54231b2640 1 -- start start 2024-04-21T08:50:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.033+0000 7f54231b2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f541c102190 0x7f541c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.033+0000 7f54231b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f541c107260 0x7f541c198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.033+0000 7f54231b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f541c19cfd0 0x7f541c19d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.033+0000 7f54231b2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1121d0 con 0x7f541c107260 2024-04-21T08:50:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.033+0000 7f54231b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f541c112050 con 0x7f541c19cfd0 2024-04-21T08:50:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.033+0000 7f54231b2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f541c112350 con 0x7f541c102190 2024-04-21T08:50:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.033+0000 7f5413fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f541c107260 0x7f541c198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.033+0000 7f5413fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f541c107260 0x7f541c198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39582/0 (socket says 172.21.15.43:39582) 2024-04-21T08:50:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.033+0000 7f5413fff640 1 -- 172.21.15.43:0/913055240 learned_addr learned my addr 172.21.15.43:0/913055240 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.034+0000 7f5420f4e640 1 --2- 172.21.15.43:0/913055240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f541c102190 0x7f541c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.034+0000 7f542174f640 1 --2- 172.21.15.43:0/913055240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f541c19cfd0 0x7f541c19d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.034+0000 7f5413fff640 1 -- 172.21.15.43:0/913055240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f541c102190 msgr2=0x7f541c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.034+0000 7f5413fff640 1 --2- 172.21.15.43:0/913055240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f541c102190 0x7f541c198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.034+0000 7f5413fff640 1 -- 172.21.15.43:0/913055240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f541c19cfd0 msgr2=0x7f541c19d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.034+0000 7f5413fff640 1 --2- 172.21.15.43:0/913055240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f541c19cfd0 0x7f541c19d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.034+0000 7f5413fff640 1 -- 172.21.15.43:0/913055240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541c19dac0 con 0x7f541c107260 2024-04-21T08:50:08.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.034+0000 7f542174f640 1 --2- 172.21.15.43:0/913055240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f541c19cfd0 0x7f541c19d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:08.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.034+0000 7f5413fff640 1 --2- 172.21.15.43:0/913055240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f541c107260 0x7f541c198c50 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f540c00c970 tx=0x7f540c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:08.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.034+0000 7f5411ffb640 1 -- 172.21.15.43:0/913055240 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c014070 con 0x7f541c107260 2024-04-21T08:50:08.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.034+0000 7f54231b2640 1 -- 172.21.15.43:0/913055240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f541c19ddb0 con 0x7f541c107260 2024-04-21T08:50:08.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.034+0000 7f5411ffb640 1 -- 172.21.15.43:0/913055240 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f540c004440 con 0x7f541c107260 2024-04-21T08:50:08.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.034+0000 7f5411ffb640 1 -- 172.21.15.43:0/913055240 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c003e50 con 0x7f541c107260 2024-04-21T08:50:08.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.034+0000 7f54231b2640 1 -- 172.21.15.43:0/913055240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f541c1a55a0 con 0x7f541c107260 2024-04-21T08:50:08.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.036+0000 7f5411ffb640 1 -- 172.21.15.43:0/913055240 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f540c017070 con 0x7f541c107260 2024-04-21T08:50:08.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.037+0000 7f5425da6740 1 Processor -- start 2024-04-21T08:50:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.037+0000 7f5425da6740 1 -- start start 2024-04-21T08:50:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.037+0000 7f5411ffb640 1 --2- 172.21.15.43:0/913055240 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f54040a3ec0 0x7f54040a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.037+0000 7f5425da6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be767ab3b0 0x55be767ab790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.037+0000 7f5425da6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be767abd60 0x55be767b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.037+0000 7f542174f640 1 --2- 172.21.15.43:0/913055240 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f54040a3ec0 0x7f54040a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.037+0000 7f5425da6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767b4e60 0x55be767b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.037+0000 7f5425da6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55be767101f0 con 0x55be767b4e60 2024-04-21T08:50:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.037+0000 7f5425da6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55be766ad4b0 con 0x55be767ab3b0 2024-04-21T08:50:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.037+0000 7f5425da6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55be7670c1b0 con 0x55be767abd60 2024-04-21T08:50:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.037+0000 7f5411ffb640 1 -- 172.21.15.43:0/913055240 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7f540c0d08c0 con 0x7f541c107260 2024-04-21T08:50:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.037+0000 7f542174f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767b4e60 0x55be767b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.037+0000 7f542174f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767b4e60 0x55be767b52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39594/0 (socket says 172.21.15.43:39594) 2024-04-21T08:50:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.037+0000 7f542174f640 1 -- 172.21.15.43:0/567577333 learned_addr learned my addr 172.21.15.43:0/567577333 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.037+0000 7f5420f4e640 1 --2- 172.21.15.43:0/567577333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be767ab3b0 0x55be767ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.037+0000 7f5413fff640 1 --2- 172.21.15.43:0/567577333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be767abd60 0x55be767b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.037+0000 7f542174f640 1 --2- 172.21.15.43:0/913055240 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f54040a3ec0 0x7f54040a6380 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5414008fd0 tx=0x7f5414014a30 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:08.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.038+0000 7f542174f640 1 -- 172.21.15.43:0/567577333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be767abd60 msgr2=0x55be767b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:08.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.038+0000 7f542174f640 1 --2- 172.21.15.43:0/567577333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be767abd60 0x55be767b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.038+0000 7f542174f640 1 -- 172.21.15.43:0/567577333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be767ab3b0 msgr2=0x55be767ab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:08.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.038+0000 7f542174f640 1 --2- 172.21.15.43:0/567577333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be767ab3b0 0x55be767ab790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.038+0000 7f542174f640 1 -- 172.21.15.43:0/567577333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be7669f1b0 con 0x55be767b4e60 2024-04-21T08:50:08.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.038+0000 7f5420f4e640 1 --2- 172.21.15.43:0/567577333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be767ab3b0 0x55be767ab790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:50:08.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.038+0000 7f542174f640 1 --2- 172.21.15.43:0/567577333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767b4e60 0x55be767b52c0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f5414011790 tx=0x7f541401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b0fa87ed5dc02d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:08.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.038+0000 7f5402ffd640 1 -- 172.21.15.43:0/567577333 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5414023070 con 0x55be767b4e60 2024-04-21T08:50:08.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.038+0000 7f5402ffd640 1 -- 172.21.15.43:0/567577333 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5414014230 con 0x55be767b4e60 2024-04-21T08:50:08.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.038+0000 7f5402ffd640 1 -- 172.21.15.43:0/567577333 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541401ebd0 con 0x55be767b4e60 2024-04-21T08:50:08.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.039+0000 7f5425da6740 1 -- 172.21.15.43:0/567577333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767b4e60 msgr2=0x55be767b52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:08.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.039+0000 7f5425da6740 1 --2- 172.21.15.43:0/567577333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767b4e60 0x55be767b52c0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f5414011790 tx=0x7f541401b000 comp rx=0 tx=0).stop 2024-04-21T08:50:08.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.039+0000 7f5425da6740 1 -- 172.21.15.43:0/567577333 shutdown_connections 2024-04-21T08:50:08.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.039+0000 7f5425da6740 1 --2- 172.21.15.43:0/567577333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767b4e60 0x55be767b52c0 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.039+0000 7f5425da6740 1 --2- 172.21.15.43:0/567577333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be767abd60 0x55be767b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.039+0000 7f5425da6740 1 --2- 172.21.15.43:0/567577333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be767ab3b0 0x55be767ab790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.039+0000 7f5425da6740 1 -- 172.21.15.43:0/567577333 >> 172.21.15.43:0/567577333 conn(0x55be767a5ba0 msgr2=0x55be767a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:08.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.039+0000 7f5425da6740 1 -- 172.21.15.43:0/567577333 shutdown_connections 2024-04-21T08:50:08.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.039+0000 7f5425da6740 1 -- 172.21.15.43:0/567577333 wait complete. 2024-04-21T08:50:08.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.039+0000 7f5425da6740 1 -- start start 2024-04-21T08:50:08.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.039+0000 7f5425da6740 1 Processor -- start 2024-04-21T08:50:08.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.039+0000 7f5425da6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be767ab3b0 0x55be767bb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.039+0000 7f5425da6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767abd60 0x55be767b8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.039+0000 7f5425da6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be767b4e60 0x55be767b8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.039+0000 7f5425da6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55be767bbd40 con 0x55be767abd60 2024-04-21T08:50:08.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.039+0000 7f5425da6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55be766c9500 con 0x55be767ab3b0 2024-04-21T08:50:08.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.039+0000 7f5425da6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55be767bbec0 con 0x55be767b4e60 2024-04-21T08:50:08.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.039+0000 7f5425da6740 10 client.0 _ll_register_callbacks cb 0x55be76697b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:08.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.040+0000 7f5413fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767abd60 0x55be767b8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.040+0000 7f5413fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767abd60 0x55be767b8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39604/0 (socket says 172.21.15.43:39604) 2024-04-21T08:50:08.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.040+0000 7f5413fff640 1 -- 172.21.15.43:0/4281197971 learned_addr learned my addr 172.21.15.43:0/4281197971 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:08.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.040+0000 7f5420f4e640 1 --2- 172.21.15.43:0/4281197971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be767ab3b0 0x55be767bb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.040+0000 7f542174f640 1 --2- 172.21.15.43:0/4281197971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be767b4e60 0x55be767b8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.040+0000 7f5413fff640 1 -- 172.21.15.43:0/4281197971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be767b4e60 msgr2=0x55be767b8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:08.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.040+0000 7f5413fff640 1 --2- 172.21.15.43:0/4281197971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be767b4e60 0x55be767b8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.040+0000 7f5413fff640 1 -- 172.21.15.43:0/4281197971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be767ab3b0 msgr2=0x55be767bb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:08.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.040+0000 7f5413fff640 1 --2- 172.21.15.43:0/4281197971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be767ab3b0 0x55be767bb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.040+0000 7f5413fff640 1 -- 172.21.15.43:0/4281197971 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be764223f0 con 0x55be767abd60 2024-04-21T08:50:08.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.040+0000 7f5413fff640 1 --2- 172.21.15.43:0/4281197971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767abd60 0x55be767b8660 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f540c0cdbf0 tx=0x7f540c09be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a8b5efb7e1e370 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:08.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.040+0000 7f53fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:08.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.040+0000 7f53fbfff640 1 -- 172.21.15.43:0/4281197971 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c0a0790 con 0x55be767abd60 2024-04-21T08:50:08.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.040+0000 7f53fbfff640 1 -- 172.21.15.43:0/4281197971 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7f540c0cde10 con 0x55be767abd60 2024-04-21T08:50:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.040+0000 7f5425da6740 1 -- 172.21.15.43:0/4281197971 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5414014230 con 0x55be767abd60 2024-04-21T08:50:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.040+0000 7f53fbfff640 1 -- 172.21.15.43:0/4281197971 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f540c0a5050 con 0x55be767abd60 2024-04-21T08:50:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.040+0000 7f5425da6740 10 client.20934 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.040+0000 7f5425da6740 1 -- 172.21.15.43:0/4281197971 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55be764ec7d0 con 0x55be767abd60 2024-04-21T08:50:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.040+0000 7f53fbfff640 1 -- 172.21.15.43:0/4281197971 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5414014230 con 0x55be767abd60 2024-04-21T08:50:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.041+0000 7f53fbfff640 1 -- 172.21.15.43:0/4281197971 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f540c0d03c0 con 0x55be767abd60 2024-04-21T08:50:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.041+0000 7f5425da6740 10 client.20934 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.041+0000 7f5425da6740 10 client.20934 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:50:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.041+0000 7f5425da6740 20 client.20934 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.041+0000 7f5425da6740 1 --2- 172.21.15.43:0/4281197971 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55be767c8f10 0x55be767cb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.041+0000 7f5425da6740 4 client.20934 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.041+0000 7f5425da6740 1 -- 172.21.15.43:0/4281197971 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be766af930 con 0x55be767c8f10 2024-04-21T08:50:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.041+0000 7f5420f4e640 1 --2- 172.21.15.43:0/4281197971 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55be767c8f10 0x55be767cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.041+0000 7f53fbfff640 10 client.20934 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:50:08.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.041+0000 7f5420f4e640 1 --2- 172.21.15.43:0/4281197971 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55be767c8f10 0x55be767cb360 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a5212f08b6aba887 server_cookie=30a299a143a1949 in_seq=0 out_seq=0 2024-04-21T08:50:08.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.042+0000 7f53fbfff640 1 -- 172.21.15.43:0/4281197971 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55be766af930 con 0x55be767c8f10 2024-04-21T08:50:08.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.042+0000 7f53fbfff640 10 client.20934 handle_command_reply: tid=0 2024-04-21T08:50:08.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.042+0000 7f5425da6740 1 client.20934 shutdown 2024-04-21T08:50:08.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.042+0000 7f5425da6740 1 -- 172.21.15.43:0/4281197971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767abd60 msgr2=0x55be767b8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:08.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.042+0000 7f5425da6740 1 --2- 172.21.15.43:0/4281197971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767abd60 0x55be767b8660 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f540c0cdbf0 tx=0x7f540c09be30 comp rx=0 tx=0).stop 2024-04-21T08:50:08.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.042+0000 7f5425da6740 1 -- 172.21.15.43:0/4281197971 shutdown_connections 2024-04-21T08:50:08.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.042+0000 7f5425da6740 1 --2- 172.21.15.43:0/4281197971 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55be767c8f10 0x55be767cb360 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.042+0000 7f5425da6740 1 --2- 172.21.15.43:0/4281197971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be767b4e60 0x55be767b8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.042+0000 7f5425da6740 1 --2- 172.21.15.43:0/4281197971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767abd60 0x55be767b8660 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.042+0000 7f5425da6740 1 --2- 172.21.15.43:0/4281197971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be767ab3b0 0x55be767bb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.042+0000 7f53fbfff640 0 client.20934 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:50:08.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.042+0000 7f5425da6740 1 -- 172.21.15.43:0/4281197971 >> 172.21.15.43:0/4281197971 conn(0x55be767a5ba0 msgr2=0x55be767a78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:08.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.042+0000 7f5425da6740 1 -- 172.21.15.43:0/4281197971 shutdown_connections 2024-04-21T08:50:08.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.042+0000 7f5425da6740 1 -- 172.21.15.43:0/4281197971 wait complete. 2024-04-21T08:50:08.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.043+0000 7f5425da6740 20 client.20934 trim_cache size 0 max 16384 2024-04-21T08:50:08.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.056+0000 7f5425da6740 1 Processor -- start 2024-04-21T08:50:08.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.056+0000 7f5425da6740 1 -- start start 2024-04-21T08:50:08.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.056+0000 7f5425da6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767ab3b0 0x55be767c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.056+0000 7f5425da6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be767abd60 0x55be767b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.056+0000 7f5425da6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be767b4e60 0x55be767b6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.056+0000 7f5425da6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1ad3c0 con 0x55be767ab3b0 2024-04-21T08:50:08.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.056+0000 7f5425da6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1aef00 con 0x55be767b4e60 2024-04-21T08:50:08.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.056+0000 7f5425da6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1ada90 con 0x55be767abd60 2024-04-21T08:50:08.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.056+0000 7f5420f4e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767ab3b0 0x55be767c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.056+0000 7f5420f4e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767ab3b0 0x55be767c4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39616/0 (socket says 172.21.15.43:39616) 2024-04-21T08:50:08.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.056+0000 7f5420f4e640 1 -- 172.21.15.43:0/4180117677 learned_addr learned my addr 172.21.15.43:0/4180117677 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:08.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.056+0000 7f5420f4e640 1 -- 172.21.15.43:0/4180117677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be767abd60 msgr2=0x55be767b77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:08.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.056+0000 7f5420f4e640 1 --2- 172.21.15.43:0/4180117677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be767abd60 0x55be767b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.056+0000 7f5420f4e640 1 -- 172.21.15.43:0/4180117677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be767b4e60 msgr2=0x55be767b6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:08.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.056+0000 7f5420f4e640 1 --2- 172.21.15.43:0/4180117677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be767b4e60 0x55be767b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.056+0000 7f5420f4e640 1 -- 172.21.15.43:0/4180117677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be767bbd90 con 0x55be767ab3b0 2024-04-21T08:50:08.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.057+0000 7f5420f4e640 1 --2- 172.21.15.43:0/4180117677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767ab3b0 0x55be767c4c40 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f5418005940 tx=0x7f54180041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd91ca8c66cd41c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:08.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.057+0000 7f54027fc640 1 -- 172.21.15.43:0/4180117677 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541802c7e0 con 0x55be767ab3b0 2024-04-21T08:50:08.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.057+0000 7f54027fc640 1 -- 172.21.15.43:0/4180117677 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5418005040 con 0x55be767ab3b0 2024-04-21T08:50:08.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.057+0000 7f54027fc640 1 -- 172.21.15.43:0/4180117677 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5418004bf0 con 0x55be767ab3b0 2024-04-21T08:50:08.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.057+0000 7f5425da6740 1 -- 172.21.15.43:0/4180117677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767ab3b0 msgr2=0x55be767c4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:08.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.057+0000 7f5425da6740 1 --2- 172.21.15.43:0/4180117677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767ab3b0 0x55be767c4c40 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f5418005940 tx=0x7f54180041e0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.057+0000 7f5425da6740 1 -- 172.21.15.43:0/4180117677 shutdown_connections 2024-04-21T08:50:08.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.057+0000 7f5425da6740 1 --2- 172.21.15.43:0/4180117677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be767b4e60 0x55be767b6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.057+0000 7f5425da6740 1 --2- 172.21.15.43:0/4180117677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be767abd60 0x55be767b77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.057+0000 7f5425da6740 1 --2- 172.21.15.43:0/4180117677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767ab3b0 0x55be767c4c40 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.057+0000 7f5425da6740 1 -- 172.21.15.43:0/4180117677 >> 172.21.15.43:0/4180117677 conn(0x55be767a5ba0 msgr2=0x55be767a6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:08.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.057+0000 7f5425da6740 1 -- 172.21.15.43:0/4180117677 shutdown_connections 2024-04-21T08:50:08.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.057+0000 7f5425da6740 1 -- 172.21.15.43:0/4180117677 wait complete. 2024-04-21T08:50:08.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.058+0000 7f5425da6740 1 -- start start 2024-04-21T08:50:08.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.058+0000 7f5425da6740 1 Processor -- start 2024-04-21T08:50:08.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.058+0000 7f5425da6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be767ab3b0 0x55be767c13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.058+0000 7f5425da6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be767abd60 0x55be767b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.058+0000 7f5425da6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767b4e60 0x55be767b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.058+0000 7f5425da6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1ae830 con 0x55be767b4e60 2024-04-21T08:50:08.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.058+0000 7f5425da6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1ae160 con 0x55be767ab3b0 2024-04-21T08:50:08.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.058+0000 7f5425da6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1ab3c0 con 0x55be767abd60 2024-04-21T08:50:08.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.058+0000 7f5425da6740 10 client.0 _ll_register_callbacks cb 0x7f540c09c280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:08.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.058+0000 7f542174f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767b4e60 0x55be767b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.058+0000 7f542174f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767b4e60 0x55be767b5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39620/0 (socket says 172.21.15.43:39620) 2024-04-21T08:50:08.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.058+0000 7f542174f640 1 -- 172.21.15.43:0/3932214995 learned_addr learned my addr 172.21.15.43:0/3932214995 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:08.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.058+0000 7f5420f4e640 1 --2- 172.21.15.43:0/3932214995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be767ab3b0 0x55be767c13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.058+0000 7f5413fff640 1 --2- 172.21.15.43:0/3932214995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be767abd60 0x55be767b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.059+0000 7f542174f640 1 -- 172.21.15.43:0/3932214995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be767abd60 msgr2=0x55be767b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:08.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.059+0000 7f542174f640 1 --2- 172.21.15.43:0/3932214995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be767abd60 0x55be767b53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.059+0000 7f542174f640 1 -- 172.21.15.43:0/3932214995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be767ab3b0 msgr2=0x55be767c13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:08.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.059+0000 7f542174f640 1 --2- 172.21.15.43:0/3932214995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be767ab3b0 0x55be767c13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.059+0000 7f542174f640 1 -- 172.21.15.43:0/3932214995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be767b5f40 con 0x55be767b4e60 2024-04-21T08:50:08.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.059+0000 7f5413fff640 1 --2- 172.21.15.43:0/3932214995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be767abd60 0x55be767b53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:08.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.059+0000 7f542174f640 1 --2- 172.21.15.43:0/3932214995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767b4e60 0x55be767b5900 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f5414021040 tx=0x7f5414014300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387a4bbf681da116 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:08.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.059+0000 7f53fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:08.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.059+0000 7f53fbfff640 1 -- 172.21.15.43:0/3932214995 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541402e060 con 0x55be767b4e60 2024-04-21T08:50:08.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.059+0000 7f53fbfff640 1 -- 172.21.15.43:0/3932214995 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7f541400fbf0 con 0x55be767b4e60 2024-04-21T08:50:08.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.059+0000 7f5425da6740 1 -- 172.21.15.43:0/3932214995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5418005040 con 0x55be767b4e60 2024-04-21T08:50:08.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.059+0000 7f53fbfff640 1 -- 172.21.15.43:0/3932214995 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5414035050 con 0x55be767b4e60 2024-04-21T08:50:08.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.059+0000 7f5425da6740 10 client.20940 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:08.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.059+0000 7f5425da6740 1 -- 172.21.15.43:0/3932214995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55be7682f4a0 con 0x55be767b4e60 2024-04-21T08:50:08.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.059+0000 7f53fbfff640 1 -- 172.21.15.43:0/3932214995 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5418005040 con 0x55be767b4e60 2024-04-21T08:50:08.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.059+0000 7f53fbfff640 1 -- 172.21.15.43:0/3932214995 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f541401e450 con 0x55be767b4e60 2024-04-21T08:50:08.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.059+0000 7f5425da6740 10 client.20940 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:08.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.059+0000 7f5425da6740 10 client.20940 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:50:08.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.059+0000 7f5425da6740 20 client.20940 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:08.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.059+0000 7f5425da6740 1 --2- 172.21.15.43:0/3932214995 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55be7682f7d0 0x55be76831bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.060+0000 7f5425da6740 4 client.20940 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:50:08.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.060+0000 7f5425da6740 1 -- 172.21.15.43:0/3932214995 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be7671dc10 con 0x55be7682f7d0 2024-04-21T08:50:08.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.060+0000 7f5420f4e640 1 --2- 172.21.15.43:0/3932214995 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55be7682f7d0 0x55be76831bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.060+0000 7f5420f4e640 1 --2- 172.21.15.43:0/3932214995 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55be7682f7d0 0x55be76831bc0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5194f6f54f705dbf server_cookie=574e8dc9ef44ba8b in_seq=0 out_seq=0 2024-04-21T08:50:08.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.060+0000 7f53fbfff640 10 client.20940 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:50:08.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.060+0000 7f53fbfff640 1 -- 172.21.15.43:0/3932214995 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be7671dc10 con 0x55be7682f7d0 2024-04-21T08:50:08.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.060+0000 7f53fbfff640 10 client.20940 handle_command_reply: tid=0 2024-04-21T08:50:08.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.060+0000 7f5425da6740 1 client.20940 shutdown 2024-04-21T08:50:08.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.061+0000 7f5425da6740 1 -- 172.21.15.43:0/3932214995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767b4e60 msgr2=0x55be767b5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:08.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.061+0000 7f5425da6740 1 --2- 172.21.15.43:0/3932214995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767b4e60 0x55be767b5900 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f5414021040 tx=0x7f5414014300 comp rx=0 tx=0).stop 2024-04-21T08:50:08.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.061+0000 7f5425da6740 1 -- 172.21.15.43:0/3932214995 shutdown_connections 2024-04-21T08:50:08.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.061+0000 7f5425da6740 1 --2- 172.21.15.43:0/3932214995 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55be7682f7d0 0x55be76831bc0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.061+0000 7f53fbfff640 0 client.20940 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:50:08.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.061+0000 7f5425da6740 1 --2- 172.21.15.43:0/3932214995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be767b4e60 0x55be767b5900 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.061+0000 7f5425da6740 1 --2- 172.21.15.43:0/3932214995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be767abd60 0x55be767b53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.061+0000 7f5425da6740 1 --2- 172.21.15.43:0/3932214995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be767ab3b0 0x55be767c13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.061+0000 7f5425da6740 1 -- 172.21.15.43:0/3932214995 >> 172.21.15.43:0/3932214995 conn(0x55be767a5ba0 msgr2=0x55be767a7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:08.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.061+0000 7f5425da6740 1 -- 172.21.15.43:0/3932214995 shutdown_connections 2024-04-21T08:50:08.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.061+0000 7f5425da6740 1 -- 172.21.15.43:0/3932214995 wait complete. 2024-04-21T08:50:08.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.061+0000 7f5425da6740 20 client.20940 trim_cache size 0 max 16384 2024-04-21T08:50:08.067 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:50:08.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.061+0000 7f54231b2640 1 -- 172.21.15.43:0/913055240 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f54040a3ec0 msgr2=0x7f54040a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:08.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.061+0000 7f54231b2640 1 --2- 172.21.15.43:0/913055240 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f54040a3ec0 0x7f54040a6380 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5414008fd0 tx=0x7f5414014a30 comp rx=0 tx=0).stop 2024-04-21T08:50:08.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.061+0000 7f54231b2640 1 -- 172.21.15.43:0/913055240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f541c107260 msgr2=0x7f541c198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:08.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.061+0000 7f54231b2640 1 --2- 172.21.15.43:0/913055240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f541c107260 0x7f541c198c50 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f540c00c970 tx=0x7f540c00ce30 comp rx=0 tx=0).stop 2024-04-21T08:50:08.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.062+0000 7f54231b2640 1 -- 172.21.15.43:0/913055240 shutdown_connections 2024-04-21T08:50:08.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.062+0000 7f54231b2640 1 --2- 172.21.15.43:0/913055240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f541c19cfd0 0x7f541c19d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.062+0000 7f54231b2640 1 --2- 172.21.15.43:0/913055240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f541c107260 0x7f541c198c50 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.062+0000 7f54231b2640 1 --2- 172.21.15.43:0/913055240 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f54040a3ec0 0x7f54040a6380 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.062+0000 7f54231b2640 1 --2- 172.21.15.43:0/913055240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f541c102190 0x7f541c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.062+0000 7f54231b2640 1 -- 172.21.15.43:0/913055240 >> 172.21.15.43:0/913055240 conn(0x7f541c0793a0 msgr2=0x7f541c0fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:08.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.062+0000 7f54231b2640 1 -- 172.21.15.43:0/913055240 shutdown_connections 2024-04-21T08:50:08.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.062+0000 7f54231b2640 1 -- 172.21.15.43:0/913055240 wait complete. 2024-04-21T08:50:08.072 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:50:08.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.177+0000 7fa2e188c640 1 -- 172.21.15.43:0/513491431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2dc101a50 msgr2=0x7fa2dc105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:08.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.177+0000 7fa2e188c640 1 --2- 172.21.15.43:0/513491431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2dc101a50 0x7fa2dc105a90 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4009900 tx=0x7fa2c402bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:08.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.177+0000 7fa2e188c640 1 -- 172.21.15.43:0/513491431 shutdown_connections 2024-04-21T08:50:08.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.177+0000 7fa2e188c640 1 --2- 172.21.15.43:0/513491431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2dc106170 0x7fa2dc10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.177+0000 7fa2e188c640 1 --2- 172.21.15.43:0/513491431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2dc101a50 0x7fa2dc105a90 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.177+0000 7fa2e188c640 1 --2- 172.21.15.43:0/513491431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2dc1010a0 0x7fa2dc101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.177+0000 7fa2e188c640 1 -- 172.21.15.43:0/513491431 >> 172.21.15.43:0/513491431 conn(0x7fa2dc077eb0 msgr2=0x7fa2dc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:08.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.177+0000 7fa2e188c640 1 -- 172.21.15.43:0/513491431 shutdown_connections 2024-04-21T08:50:08.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.177+0000 7fa2e188c640 1 -- 172.21.15.43:0/513491431 wait complete. 2024-04-21T08:50:08.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.177+0000 7fa2e188c640 1 Processor -- start 2024-04-21T08:50:08.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.177+0000 7fa2e188c640 1 -- start start 2024-04-21T08:50:08.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.178+0000 7fa2e188c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2dc1010a0 0x7fa2dc199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.178+0000 7fa2e188c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2dc106170 0x7fa2dc199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.178+0000 7fa2e188c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2dc19a040 0x7fa2dc1a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.178+0000 7fa2e188c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc1110b0 con 0x7fa2dc1010a0 2024-04-21T08:50:08.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.178+0000 7fa2e188c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc110f30 con 0x7fa2dc19a040 2024-04-21T08:50:08.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.178+0000 7fa2e188c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc111230 con 0x7fa2dc106170 2024-04-21T08:50:08.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.178+0000 7fa2daffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2dc1010a0 0x7fa2dc199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.178+0000 7fa2daffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2dc1010a0 0x7fa2dc199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39644/0 (socket says 172.21.15.43:39644) 2024-04-21T08:50:08.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.178+0000 7fa2daffd640 1 -- 172.21.15.43:0/2882512508 learned_addr learned my addr 172.21.15.43:0/2882512508 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:08.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.178+0000 7fa2db7fe640 1 --2- 172.21.15.43:0/2882512508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2dc19a040 0x7fa2dc1a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.178+0000 7fa2daffd640 1 -- 172.21.15.43:0/2882512508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2dc106170 msgr2=0x7fa2dc199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:08.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.178+0000 7fa2daffd640 1 --2- 172.21.15.43:0/2882512508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2dc106170 0x7fa2dc199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.178+0000 7fa2daffd640 1 -- 172.21.15.43:0/2882512508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2dc19a040 msgr2=0x7fa2dc1a45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:08.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.178+0000 7fa2daffd640 1 --2- 172.21.15.43:0/2882512508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2dc19a040 0x7fa2dc1a45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.178+0000 7fa2daffd640 1 -- 172.21.15.43:0/2882512508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2dc1a4df0 con 0x7fa2dc1010a0 2024-04-21T08:50:08.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.178+0000 7fa2daffd640 1 --2- 172.21.15.43:0/2882512508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2dc1010a0 0x7fa2dc199570 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00e9f0 tx=0x7fa2cc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:08.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.178+0000 7fa2e088a640 1 -- 172.21.15.43:0/2882512508 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00cd80 con 0x7fa2dc1010a0 2024-04-21T08:50:08.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.179+0000 7fa2e088a640 1 -- 172.21.15.43:0/2882512508 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2cc0108e0 con 0x7fa2dc1010a0 2024-04-21T08:50:08.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.179+0000 7fa2e188c640 1 -- 172.21.15.43:0/2882512508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2dc1a50e0 con 0x7fa2dc1010a0 2024-04-21T08:50:08.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.179+0000 7fa2e088a640 1 -- 172.21.15.43:0/2882512508 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc018980 con 0x7fa2dc1010a0 2024-04-21T08:50:08.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.179+0000 7fa2e188c640 1 -- 172.21.15.43:0/2882512508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2dc1a5630 con 0x7fa2dc1010a0 2024-04-21T08:50:08.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.181+0000 7fa2e088a640 1 -- 172.21.15.43:0/2882512508 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa2cc010430 con 0x7fa2dc1010a0 2024-04-21T08:50:08.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.181+0000 7fa2e4122740 1 Processor -- start 2024-04-21T08:50:08.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.181+0000 7fa2e4122740 1 -- start start 2024-04-21T08:50:08.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.181+0000 7fa2e4122740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dbf3b0 0x55f5e4dbf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.181+0000 7fa2e4122740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5e4dbfd60 0x55f5e4dc8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.181+0000 7fa2e088a640 1 --2- 172.21.15.43:0/2882512508 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa2c80a3d20 0x7fa2c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.181+0000 7fa2e4122740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5e4dc8e60 0x55f5e4dc92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.181+0000 7fa2e4122740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f5e4d241f0 con 0x55f5e4dbf3b0 2024-04-21T08:50:08.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.181+0000 7fa2e4122740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f5e4cc14b0 con 0x55f5e4dc8e60 2024-04-21T08:50:08.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.181+0000 7fa2e4122740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f5e4d201b0 con 0x55f5e4dbfd60 2024-04-21T08:50:08.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.182+0000 7fa2db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dbf3b0 0x55f5e4dbf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.182+0000 7fa2daffd640 1 --2- 172.21.15.43:0/2882512508 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa2c80a3d20 0x7fa2c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.182+0000 7fa2db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dbf3b0 0x55f5e4dbf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39654/0 (socket says 172.21.15.43:39654) 2024-04-21T08:50:08.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.182+0000 7fa2db7fe640 1 -- 172.21.15.43:0/2608767199 learned_addr learned my addr 172.21.15.43:0/2608767199 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:08.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.182+0000 7fa2e088a640 1 -- 172.21.15.43:0/2882512508 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7fa2cc0d10c0 con 0x7fa2dc1010a0 2024-04-21T08:50:08.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.182+0000 7fa2db7fe640 1 --2- 172.21.15.43:0/2608767199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5e4dc8e60 0x55f5e4dc92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.182+0000 7fa2d27fc640 1 --2- 172.21.15.43:0/2608767199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5e4dbfd60 0x55f5e4dc8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.182+0000 7fa2db7fe640 1 -- 172.21.15.43:0/2608767199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5e4dbfd60 msgr2=0x55f5e4dc8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:08.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.182+0000 7fa2db7fe640 1 --2- 172.21.15.43:0/2608767199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5e4dbfd60 0x55f5e4dc8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.182+0000 7fa2db7fe640 1 -- 172.21.15.43:0/2608767199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5e4dc8e60 msgr2=0x55f5e4dc92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:08.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.182+0000 7fa2db7fe640 1 --2- 172.21.15.43:0/2608767199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5e4dc8e60 0x55f5e4dc92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.182+0000 7fa2db7fe640 1 -- 172.21.15.43:0/2608767199 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5e4cb31b0 con 0x55f5e4dbf3b0 2024-04-21T08:50:08.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.182+0000 7fa2d27fc640 1 --2- 172.21.15.43:0/2608767199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5e4dbfd60 0x55f5e4dc8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:08.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.182+0000 7fa2daffd640 1 --2- 172.21.15.43:0/2882512508 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa2c80a3d20 0x7fa2c80a61e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc09cb70 tx=0x7fa2cc00cf20 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:08.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.182+0000 7fa2db7fe640 1 --2- 172.21.15.43:0/2608767199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dbf3b0 0x55f5e4dbf790 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4012650 tx=0x7fa2d4008730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50844f067ab08491 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:08.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.182+0000 7fa2d1ffb640 1 -- 172.21.15.43:0/2608767199 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4018030 con 0x55f5e4dbf3b0 2024-04-21T08:50:08.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.182+0000 7fa2d1ffb640 1 -- 172.21.15.43:0/2608767199 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2d4003e20 con 0x55f5e4dbf3b0 2024-04-21T08:50:08.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.182+0000 7fa2d1ffb640 1 -- 172.21.15.43:0/2608767199 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4004280 con 0x55f5e4dbf3b0 2024-04-21T08:50:08.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.183+0000 7fa2e4122740 1 -- 172.21.15.43:0/2608767199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dbf3b0 msgr2=0x55f5e4dbf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:08.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.183+0000 7fa2e4122740 1 --2- 172.21.15.43:0/2608767199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dbf3b0 0x55f5e4dbf790 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4012650 tx=0x7fa2d4008730 comp rx=0 tx=0).stop 2024-04-21T08:50:08.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.183+0000 7fa2e4122740 1 -- 172.21.15.43:0/2608767199 shutdown_connections 2024-04-21T08:50:08.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.183+0000 7fa2e4122740 1 --2- 172.21.15.43:0/2608767199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5e4dc8e60 0x55f5e4dc92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.183+0000 7fa2e4122740 1 --2- 172.21.15.43:0/2608767199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5e4dbfd60 0x55f5e4dc8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.183+0000 7fa2e4122740 1 --2- 172.21.15.43:0/2608767199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dbf3b0 0x55f5e4dbf790 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.183+0000 7fa2e4122740 1 -- 172.21.15.43:0/2608767199 >> 172.21.15.43:0/2608767199 conn(0x55f5e4db9ba0 msgr2=0x55f5e4db9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:08.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.183+0000 7fa2e4122740 1 -- 172.21.15.43:0/2608767199 shutdown_connections 2024-04-21T08:50:08.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.183+0000 7fa2e4122740 1 -- 172.21.15.43:0/2608767199 wait complete. 2024-04-21T08:50:08.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.183+0000 7fa2e4122740 1 -- start start 2024-04-21T08:50:08.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.183+0000 7fa2e4122740 1 Processor -- start 2024-04-21T08:50:08.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.184+0000 7fa2e4122740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5e4dbf3b0 0x55f5e4dcf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.184+0000 7fa2e4122740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5e4dbfd60 0x55f5e4dcc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.184+0000 7fa2e4122740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dc8e60 0x55f5e4dccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.184+0000 7fa2e4122740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f5e4dcfda0 con 0x55f5e4dc8e60 2024-04-21T08:50:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.184+0000 7fa2e4122740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f5e4cdd500 con 0x55f5e4dbfd60 2024-04-21T08:50:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.184+0000 7fa2e4122740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f5e4dcff20 con 0x55f5e4dbf3b0 2024-04-21T08:50:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.184+0000 7fa2e4122740 10 client.0 _ll_register_callbacks cb 0x55f5e4cabb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.184+0000 7fa2db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dc8e60 0x55f5e4dccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.184+0000 7fa2db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dc8e60 0x55f5e4dccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39662/0 (socket says 172.21.15.43:39662) 2024-04-21T08:50:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.184+0000 7fa2db7fe640 1 -- 172.21.15.43:0/2585508861 learned_addr learned my addr 172.21.15.43:0/2585508861 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.184+0000 7fa2db7fe640 1 --2- 172.21.15.43:0/2585508861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5e4dbf3b0 0x55f5e4dcf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.184+0000 7fa2d27fc640 1 --2- 172.21.15.43:0/2585508861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5e4dbfd60 0x55f5e4dcc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.184+0000 7fa2db7fe640 1 -- 172.21.15.43:0/2585508861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5e4dbf3b0 msgr2=0x55f5e4dcf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.184+0000 7fa2db7fe640 1 --2- 172.21.15.43:0/2585508861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5e4dbf3b0 0x55f5e4dcf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.184+0000 7fa2db7fe640 1 -- 172.21.15.43:0/2585508861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5e4dbfd60 msgr2=0x55f5e4dcc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:08.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.184+0000 7fa2db7fe640 1 --2- 172.21.15.43:0/2585508861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5e4dbfd60 0x55f5e4dcc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.184+0000 7fa2db7fe640 1 -- 172.21.15.43:0/2585508861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5e4a363f0 con 0x55f5e4dc8e60 2024-04-21T08:50:08.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.184+0000 7fa2db7fe640 1 --2- 172.21.15.43:0/2585508861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dc8e60 0x55f5e4dccb00 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4010db0 tx=0x7fa2d4010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e69cb902ddb79b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:08.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.184+0000 7fa2baffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:08.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.184+0000 7fa2baffd640 1 -- 172.21.15.43:0/2585508861 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4018050 con 0x55f5e4dc8e60 2024-04-21T08:50:08.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.185+0000 7fa2baffd640 1 -- 172.21.15.43:0/2585508861 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7fa2d4003b80 con 0x55f5e4dc8e60 2024-04-21T08:50:08.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.185+0000 7fa2e4122740 1 -- 172.21.15.43:0/2585508861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2d4003e20 con 0x55f5e4dc8e60 2024-04-21T08:50:08.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.185+0000 7fa2baffd640 1 -- 172.21.15.43:0/2585508861 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa2d4034050 con 0x55f5e4dc8e60 2024-04-21T08:50:08.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.185+0000 7fa2e4122740 10 client.20952 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:08.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.185+0000 7fa2e4122740 1 -- 172.21.15.43:0/2585508861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f5e4b007d0 con 0x55f5e4dc8e60 2024-04-21T08:50:08.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.185+0000 7fa2baffd640 1 -- 172.21.15.43:0/2585508861 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2d4003e20 con 0x55f5e4dc8e60 2024-04-21T08:50:08.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.185+0000 7fa2baffd640 1 -- 172.21.15.43:0/2585508861 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa2d401d840 con 0x55f5e4dc8e60 2024-04-21T08:50:08.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.185+0000 7fa2e4122740 10 client.20952 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:08.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.185+0000 7fa2e4122740 10 client.20952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:08.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.185+0000 7fa2e4122740 20 client.20952 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:08.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.185+0000 7fa2e4122740 1 --2- 172.21.15.43:0/2585508861 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f5e4ddceb0 0x55f5e4ddf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.185+0000 7fa2e4122740 4 client.20952 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:08.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.185+0000 7fa2e4122740 1 -- 172.21.15.43:0/2585508861 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5e4cc3930 con 0x55f5e4ddceb0 2024-04-21T08:50:08.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.185+0000 7fa2d27fc640 1 --2- 172.21.15.43:0/2585508861 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f5e4ddceb0 0x55f5e4ddf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.186+0000 7fa2baffd640 10 client.20952 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:08.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.186+0000 7fa2d27fc640 1 --2- 172.21.15.43:0/2585508861 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f5e4ddceb0 0x55f5e4ddf2a0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b39cfc8140b1e0b server_cookie=59a803bfe47db810 in_seq=0 out_seq=0 2024-04-21T08:50:08.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.187+0000 7fa2baffd640 1 -- 172.21.15.43:0/2585508861 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f5e4cc3930 con 0x55f5e4ddceb0 2024-04-21T08:50:08.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.187+0000 7fa2baffd640 10 client.20952 handle_command_reply: tid=0 2024-04-21T08:50:08.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.187+0000 7fa2e4122740 1 client.20952 shutdown 2024-04-21T08:50:08.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.187+0000 7fa2e4122740 1 -- 172.21.15.43:0/2585508861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dc8e60 msgr2=0x55f5e4dccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:08.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.187+0000 7fa2e4122740 1 --2- 172.21.15.43:0/2585508861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dc8e60 0x55f5e4dccb00 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4010db0 tx=0x7fa2d4010cb0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.187+0000 7fa2e4122740 1 -- 172.21.15.43:0/2585508861 shutdown_connections 2024-04-21T08:50:08.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.187+0000 7fa2e4122740 1 --2- 172.21.15.43:0/2585508861 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f5e4ddceb0 0x55f5e4ddf2a0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.187+0000 7fa2e4122740 1 --2- 172.21.15.43:0/2585508861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dc8e60 0x55f5e4dccb00 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.187+0000 7fa2e4122740 1 --2- 172.21.15.43:0/2585508861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5e4dbfd60 0x55f5e4dcc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.187+0000 7fa2e4122740 1 --2- 172.21.15.43:0/2585508861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5e4dbf3b0 0x55f5e4dcf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.187+0000 7fa2baffd640 0 client.20952 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:08.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.187+0000 7fa2e4122740 1 -- 172.21.15.43:0/2585508861 >> 172.21.15.43:0/2585508861 conn(0x55f5e4db9ba0 msgr2=0x55f5e4dbb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:08.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.187+0000 7fa2e4122740 1 -- 172.21.15.43:0/2585508861 shutdown_connections 2024-04-21T08:50:08.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.187+0000 7fa2e4122740 1 -- 172.21.15.43:0/2585508861 wait complete. 2024-04-21T08:50:08.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.187+0000 7fa2e4122740 20 client.20952 trim_cache size 0 max 16384 2024-04-21T08:50:08.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.200+0000 7fa2e4122740 1 Processor -- start 2024-04-21T08:50:08.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.200+0000 7fa2e4122740 1 -- start start 2024-04-21T08:50:08.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.201+0000 7fa2e4122740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dbf3b0 0x55f5e4dca250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.201+0000 7fa2e4122740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5e4dbfd60 0x55f5e4dd8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.201+0000 7fa2e4122740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5e4dc8e60 0x55f5e4dc7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.201+0000 7fa2e4122740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc1ad4a0 con 0x55f5e4dbf3b0 2024-04-21T08:50:08.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.201+0000 7fa2e4122740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc1aefe0 con 0x55f5e4dbfd60 2024-04-21T08:50:08.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.201+0000 7fa2e4122740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc1adb70 con 0x55f5e4dc8e60 2024-04-21T08:50:08.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.201+0000 7fa2db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dbf3b0 0x55f5e4dca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.201+0000 7fa2db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dbf3b0 0x55f5e4dca250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39668/0 (socket says 172.21.15.43:39668) 2024-04-21T08:50:08.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.201+0000 7fa2db7fe640 1 -- 172.21.15.43:0/3050256675 learned_addr learned my addr 172.21.15.43:0/3050256675 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:08.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.201+0000 7fa2db7fe640 1 -- 172.21.15.43:0/3050256675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5e4dc8e60 msgr2=0x55f5e4dc7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:08.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.201+0000 7fa2db7fe640 1 --2- 172.21.15.43:0/3050256675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5e4dc8e60 0x55f5e4dc7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.201+0000 7fa2db7fe640 1 -- 172.21.15.43:0/3050256675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5e4dbfd60 msgr2=0x55f5e4dd8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:08.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.201+0000 7fa2db7fe640 1 --2- 172.21.15.43:0/3050256675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5e4dbfd60 0x55f5e4dd8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.201+0000 7fa2db7fe640 1 -- 172.21.15.43:0/3050256675 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5e4ddf830 con 0x55f5e4dbf3b0 2024-04-21T08:50:08.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.201+0000 7fa2db7fe640 1 --2- 172.21.15.43:0/3050256675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dbf3b0 0x55f5e4dca250 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4010ad0 tx=0x7fa2d4014fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3567c4ad09a41b16 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:08.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.201+0000 7fa2d17fa640 1 -- 172.21.15.43:0/3050256675 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4018070 con 0x55f5e4dbf3b0 2024-04-21T08:50:08.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.201+0000 7fa2d17fa640 1 -- 172.21.15.43:0/3050256675 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2d4019040 con 0x55f5e4dbf3b0 2024-04-21T08:50:08.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.201+0000 7fa2d17fa640 1 -- 172.21.15.43:0/3050256675 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4003dc0 con 0x55f5e4dbf3b0 2024-04-21T08:50:08.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.202+0000 7fa2e4122740 1 -- 172.21.15.43:0/3050256675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dbf3b0 msgr2=0x55f5e4dca250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:08.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.202+0000 7fa2e4122740 1 --2- 172.21.15.43:0/3050256675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dbf3b0 0x55f5e4dca250 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4010ad0 tx=0x7fa2d4014fc0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.202+0000 7fa2e4122740 1 -- 172.21.15.43:0/3050256675 shutdown_connections 2024-04-21T08:50:08.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.202+0000 7fa2e4122740 1 --2- 172.21.15.43:0/3050256675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5e4dc8e60 0x55f5e4dc7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.202+0000 7fa2e4122740 1 --2- 172.21.15.43:0/3050256675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5e4dbfd60 0x55f5e4dd8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.202+0000 7fa2e4122740 1 --2- 172.21.15.43:0/3050256675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dbf3b0 0x55f5e4dca250 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.202+0000 7fa2e4122740 1 -- 172.21.15.43:0/3050256675 >> 172.21.15.43:0/3050256675 conn(0x55f5e4db9ba0 msgr2=0x55f5e4dbb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:08.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.202+0000 7fa2e4122740 1 -- 172.21.15.43:0/3050256675 shutdown_connections 2024-04-21T08:50:08.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.202+0000 7fa2e4122740 1 -- 172.21.15.43:0/3050256675 wait complete. 2024-04-21T08:50:08.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.202+0000 7fa2e4122740 1 -- start start 2024-04-21T08:50:08.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.202+0000 7fa2e4122740 1 Processor -- start 2024-04-21T08:50:08.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2e4122740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5e4dbf3b0 0x55f5e4dd51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2e4122740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dbfd60 0x55f5e4dd5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2e4122740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5e4dc8e60 0x55f5e4dca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2e4122740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc1ae910 con 0x55f5e4dbfd60 2024-04-21T08:50:08.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2e4122740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc1ae240 con 0x55f5e4dc8e60 2024-04-21T08:50:08.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2e4122740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc1ab4a0 con 0x55f5e4dbf3b0 2024-04-21T08:50:08.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2e4122740 10 client.0 _ll_register_callbacks cb 0x7fa2d4004180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:08.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2d27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dbfd60 0x55f5e4dd5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2d27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dbfd60 0x55f5e4dd5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39674/0 (socket says 172.21.15.43:39674) 2024-04-21T08:50:08.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2d27fc640 1 -- 172.21.15.43:0/2437120204 learned_addr learned my addr 172.21.15.43:0/2437120204 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:08.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2db7fe640 1 --2- 172.21.15.43:0/2437120204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5e4dbf3b0 0x55f5e4dd51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2d27fc640 1 -- 172.21.15.43:0/2437120204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5e4dbf3b0 msgr2=0x55f5e4dd51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:08.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2d27fc640 1 --2- 172.21.15.43:0/2437120204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5e4dbf3b0 0x55f5e4dd51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2d27fc640 1 -- 172.21.15.43:0/2437120204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5e4dc8e60 msgr2=0x55f5e4dca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:08.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2d27fc640 1 --2- 172.21.15.43:0/2437120204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5e4dc8e60 0x55f5e4dca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2d27fc640 1 -- 172.21.15.43:0/2437120204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5e4dcadc0 con 0x55f5e4dbfd60 2024-04-21T08:50:08.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2db7fe640 1 --2- 172.21.15.43:0/2437120204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5e4dbf3b0 0x55f5e4dd51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:08.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2d27fc640 1 --2- 172.21.15.43:0/2437120204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dbfd60 0x55f5e4dd5720 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4005ec0 tx=0x7fa2c40043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b997ecf864541a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:08.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2baffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:08.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2baffd640 1 -- 172.21.15.43:0/2437120204 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c404f060 con 0x55f5e4dbfd60 2024-04-21T08:50:08.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2baffd640 1 -- 172.21.15.43:0/2437120204 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7fa2c403a040 con 0x55f5e4dbfd60 2024-04-21T08:50:08.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2e4122740 1 -- 172.21.15.43:0/2437120204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2d4019040 con 0x55f5e4dbfd60 2024-04-21T08:50:08.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2baffd640 1 -- 172.21.15.43:0/2437120204 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa2c4056050 con 0x55f5e4dbfd60 2024-04-21T08:50:08.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2e4122740 10 client.20958 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:08.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.203+0000 7fa2e4122740 1 -- 172.21.15.43:0/2437120204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f5e4e42be0 con 0x55f5e4dbfd60 2024-04-21T08:50:08.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.204+0000 7fa2baffd640 1 -- 172.21.15.43:0/2437120204 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2d4019040 con 0x55f5e4dbfd60 2024-04-21T08:50:08.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.204+0000 7fa2baffd640 1 -- 172.21.15.43:0/2437120204 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa2c4004750 con 0x55f5e4dbfd60 2024-04-21T08:50:08.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.204+0000 7fa2e4122740 10 client.20958 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:08.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.204+0000 7fa2e4122740 10 client.20958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:08.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.204+0000 7fa2e4122740 20 client.20958 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:08.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.204+0000 7fa2e4122740 1 --2- 172.21.15.43:0/2437120204 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f5e4e43310 0x55f5e4e45730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:08.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.204+0000 7fa2e4122740 4 client.20958 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:50:08.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.204+0000 7fa2e4122740 1 -- 172.21.15.43:0/2437120204 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f5e4d31c10 con 0x55f5e4e43310 2024-04-21T08:50:08.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.204+0000 7fa2db7fe640 1 --2- 172.21.15.43:0/2437120204 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f5e4e43310 0x55f5e4e45730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:08.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.204+0000 7fa2baffd640 10 client.20958 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:08.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.204+0000 7fa2db7fe640 1 --2- 172.21.15.43:0/2437120204 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f5e4e43310 0x55f5e4e45730 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdd715747029a118 server_cookie=155e318a6094891b in_seq=0 out_seq=0 2024-04-21T08:50:08.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.205+0000 7fa2baffd640 1 -- 172.21.15.43:0/2437120204 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f5e4d31c10 con 0x55f5e4e43310 2024-04-21T08:50:08.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.205+0000 7fa2baffd640 10 client.20958 handle_command_reply: tid=0 2024-04-21T08:50:08.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.205+0000 7fa2e4122740 1 client.20958 shutdown 2024-04-21T08:50:08.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.205+0000 7fa2e4122740 1 -- 172.21.15.43:0/2437120204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dbfd60 msgr2=0x55f5e4dd5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:08.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.205+0000 7fa2e4122740 1 --2- 172.21.15.43:0/2437120204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dbfd60 0x55f5e4dd5720 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4005ec0 tx=0x7fa2c40043c0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.205+0000 7fa2e4122740 1 -- 172.21.15.43:0/2437120204 shutdown_connections 2024-04-21T08:50:08.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.205+0000 7fa2e4122740 1 --2- 172.21.15.43:0/2437120204 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55f5e4e43310 0x55f5e4e45730 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.205+0000 7fa2e4122740 1 --2- 172.21.15.43:0/2437120204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f5e4dc8e60 0x55f5e4dca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.205+0000 7fa2baffd640 0 client.20958 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:08.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.205+0000 7fa2e4122740 1 --2- 172.21.15.43:0/2437120204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5e4dbfd60 0x55f5e4dd5720 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.205+0000 7fa2e4122740 1 --2- 172.21.15.43:0/2437120204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5e4dbf3b0 0x55f5e4dd51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.205+0000 7fa2e4122740 1 -- 172.21.15.43:0/2437120204 >> 172.21.15.43:0/2437120204 conn(0x55f5e4db9ba0 msgr2=0x55f5e4dcdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:08.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.205+0000 7fa2e4122740 1 -- 172.21.15.43:0/2437120204 shutdown_connections 2024-04-21T08:50:08.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.205+0000 7fa2e4122740 1 -- 172.21.15.43:0/2437120204 wait complete. 2024-04-21T08:50:08.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.205+0000 7fa2e4122740 20 client.20958 trim_cache size 0 max 16384 2024-04-21T08:50:08.213 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:50:08.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.205+0000 7fa2e188c640 1 -- 172.21.15.43:0/2882512508 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa2c80a3d20 msgr2=0x7fa2c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:08.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.205+0000 7fa2e188c640 1 --2- 172.21.15.43:0/2882512508 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa2c80a3d20 0x7fa2c80a61e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc09cb70 tx=0x7fa2cc00cf20 comp rx=0 tx=0).stop 2024-04-21T08:50:08.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.205+0000 7fa2e188c640 1 -- 172.21.15.43:0/2882512508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2dc1010a0 msgr2=0x7fa2dc199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:08.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.205+0000 7fa2e188c640 1 --2- 172.21.15.43:0/2882512508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2dc1010a0 0x7fa2dc199570 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00e9f0 tx=0x7fa2cc00eeb0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.206+0000 7fa2e188c640 1 -- 172.21.15.43:0/2882512508 shutdown_connections 2024-04-21T08:50:08.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.206+0000 7fa2e188c640 1 --2- 172.21.15.43:0/2882512508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2dc19a040 0x7fa2dc1a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.206+0000 7fa2e188c640 1 --2- 172.21.15.43:0/2882512508 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa2c80a3d20 0x7fa2c80a61e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.206+0000 7fa2e188c640 1 --2- 172.21.15.43:0/2882512508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2dc106170 0x7fa2dc199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.206+0000 7fa2e188c640 1 --2- 172.21.15.43:0/2882512508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2dc1010a0 0x7fa2dc199570 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:08.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.206+0000 7fa2e188c640 1 -- 172.21.15.43:0/2882512508 >> 172.21.15.43:0/2882512508 conn(0x7fa2dc077eb0 msgr2=0x7fa2dc0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:08.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.206+0000 7fa2e188c640 1 -- 172.21.15.43:0/2882512508 shutdown_connections 2024-04-21T08:50:08.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:08.206+0000 7fa2e188c640 1 -- 172.21.15.43:0/2882512508 wait complete. 2024-04-21T08:50:08.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:08 smithi112 ceph-mon[31085]: pgmap v349: 129 pgs: 129 active+clean; 3.6 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 38 op/s 2024-04-21T08:50:08.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:08 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3226730246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:50:08.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:08 smithi043 ceph-mon[27978]: pgmap v349: 129 pgs: 129 active+clean; 3.6 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 38 op/s 2024-04-21T08:50:08.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:08 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3226730246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:50:08.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:08 smithi175 ceph-mon[31920]: pgmap v349: 129 pgs: 129 active+clean; 3.6 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 38 op/s 2024-04-21T08:50:08.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:08 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3226730246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:50:09.219 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:50:09.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:50:09.219 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:50:09.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.340+0000 7febc672b640 1 -- 172.21.15.43:0/2532410027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febc0102190 msgr2=0x7febc0102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:09.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.340+0000 7febc672b640 1 --2- 172.21.15.43:0/2532410027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febc0102190 0x7febc0102570 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7febb0009900 tx=0x7febb002bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:09.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.340+0000 7febc672b640 1 -- 172.21.15.43:0/2532410027 shutdown_connections 2024-04-21T08:50:09.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.340+0000 7febc672b640 1 --2- 172.21.15.43:0/2532410027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febc0107260 0x7febc010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.340+0000 7febc672b640 1 --2- 172.21.15.43:0/2532410027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febc0102b40 0x7febc0106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.340+0000 7febc672b640 1 --2- 172.21.15.43:0/2532410027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febc0102190 0x7febc0102570 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.340+0000 7febc672b640 1 -- 172.21.15.43:0/2532410027 >> 172.21.15.43:0/2532410027 conn(0x7febc00fd990 msgr2=0x7febc00ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:09.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.340+0000 7febc672b640 1 -- 172.21.15.43:0/2532410027 shutdown_connections 2024-04-21T08:50:09.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.340+0000 7febc672b640 1 -- 172.21.15.43:0/2532410027 wait complete. 2024-04-21T08:50:09.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.341+0000 7febc672b640 1 Processor -- start 2024-04-21T08:50:09.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.341+0000 7febc672b640 1 -- start start 2024-04-21T08:50:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.341+0000 7febc672b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febc0102190 0x7febc019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.341+0000 7febc672b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febc0107260 0x7febc019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.341+0000 7febc672b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febc0198610 0x7febc0198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.341+0000 7febc672b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7febc01121c0 con 0x7febc0198610 2024-04-21T08:50:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.341+0000 7febc672b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7febc0112040 con 0x7febc0102190 2024-04-21T08:50:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.341+0000 7febc672b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7febc0112340 con 0x7febc0107260 2024-04-21T08:50:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.341+0000 7febc4cc8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febc0198610 0x7febc0198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.341+0000 7febc4cc8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febc0198610 0x7febc0198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39698/0 (socket says 172.21.15.43:39698) 2024-04-21T08:50:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.341+0000 7febc4cc8640 1 -- 172.21.15.43:0/245783442 learned_addr learned my addr 172.21.15.43:0/245783442 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.341+0000 7febbffff640 1 --2- 172.21.15.43:0/245783442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febc0102190 0x7febc019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.341+0000 7febbf7fe640 1 --2- 172.21.15.43:0/245783442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febc0107260 0x7febc019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.342+0000 7febc4cc8640 1 -- 172.21.15.43:0/245783442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febc0107260 msgr2=0x7febc019e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:09.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.342+0000 7febc4cc8640 1 --2- 172.21.15.43:0/245783442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febc0107260 0x7febc019e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.342+0000 7febc4cc8640 1 -- 172.21.15.43:0/245783442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febc0102190 msgr2=0x7febc019e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:09.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.342+0000 7febc4cc8640 1 --2- 172.21.15.43:0/245783442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febc0102190 0x7febc019e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.342+0000 7febc4cc8640 1 -- 172.21.15.43:0/245783442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc0199130 con 0x7febc0198610 2024-04-21T08:50:09.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.342+0000 7febc4cc8640 1 --2- 172.21.15.43:0/245783442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febc0198610 0x7febc0198ac0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7febb800eaa0 tx=0x7febb800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:09.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.342+0000 7febbd7fa640 1 -- 172.21.15.43:0/245783442 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb800cdb0 con 0x7febc0198610 2024-04-21T08:50:09.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.342+0000 7febbd7fa640 1 -- 172.21.15.43:0/245783442 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febb8005020 con 0x7febc0198610 2024-04-21T08:50:09.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.342+0000 7febc672b640 1 -- 172.21.15.43:0/245783442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc0199420 con 0x7febc0198610 2024-04-21T08:50:09.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.342+0000 7febbd7fa640 1 -- 172.21.15.43:0/245783442 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb80116d0 con 0x7febc0198610 2024-04-21T08:50:09.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.342+0000 7febc672b640 1 -- 172.21.15.43:0/245783442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc01a55d0 con 0x7febc0198610 2024-04-21T08:50:09.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.344+0000 7febbd7fa640 1 -- 172.21.15.43:0/245783442 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7febb80051d0 con 0x7febc0198610 2024-04-21T08:50:09.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.345+0000 7febc93f8740 1 Processor -- start 2024-04-21T08:50:09.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.345+0000 7febc93f8740 1 -- start start 2024-04-21T08:50:09.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.345+0000 7febc93f8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86b8e0 0x5586ab86bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.345+0000 7febc93f8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586ab86c290 0x5586ab874840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.345+0000 7febc93f8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586ab874e10 0x5586ab877260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.345+0000 7febc93f8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5586ab7cff00 con 0x5586ab86b8e0 2024-04-21T08:50:09.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.345+0000 7febc93f8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5586ab76d590 con 0x5586ab86c290 2024-04-21T08:50:09.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.345+0000 7febc93f8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5586ab7cbec0 con 0x5586ab874e10 2024-04-21T08:50:09.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.345+0000 7febbf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86b8e0 0x5586ab86bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.345+0000 7febbf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86b8e0 0x5586ab86bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39708/0 (socket says 172.21.15.43:39708) 2024-04-21T08:50:09.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.345+0000 7febbf7fe640 1 -- 172.21.15.43:0/107189859 learned_addr learned my addr 172.21.15.43:0/107189859 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:09.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.345+0000 7febbffff640 1 --2- 172.21.15.43:0/107189859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586ab86c290 0x5586ab874840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.345+0000 7febbd7fa640 1 --2- 172.21.15.43:0/245783442 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7feba80a3d20 0x7feba80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.345+0000 7febbf7fe640 1 --2- 172.21.15.43:0/107189859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586ab874e10 0x5586ab877260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.345+0000 7febbf7fe640 1 -- 172.21.15.43:0/107189859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586ab874e10 msgr2=0x5586ab877260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:09.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.345+0000 7febbf7fe640 1 --2- 172.21.15.43:0/107189859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586ab874e10 0x5586ab877260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.345+0000 7febbf7fe640 1 -- 172.21.15.43:0/107189859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586ab86c290 msgr2=0x5586ab874840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:09.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.345+0000 7febbf7fe640 1 --2- 172.21.15.43:0/107189859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586ab86c290 0x5586ab874840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.345+0000 7febbf7fe640 1 -- 172.21.15.43:0/107189859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586ab5aa7e0 con 0x5586ab86b8e0 2024-04-21T08:50:09.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.345+0000 7febc4cc8640 1 --2- 172.21.15.43:0/245783442 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7feba80a3d20 0x7feba80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.346+0000 7febbf7fe640 1 --2- 172.21.15.43:0/107189859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86b8e0 0x5586ab86bcc0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7febb400bb20 tx=0x7febb40041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e2d411a419a01b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:09.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.346+0000 7febbd7fa640 1 -- 172.21.15.43:0/245783442 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7febb80d0d70 con 0x7febc0198610 2024-04-21T08:50:09.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.346+0000 7feba67fc640 1 -- 172.21.15.43:0/107189859 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb4005020 con 0x5586ab86b8e0 2024-04-21T08:50:09.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.346+0000 7feba67fc640 1 -- 172.21.15.43:0/107189859 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febb4005670 con 0x5586ab86b8e0 2024-04-21T08:50:09.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.346+0000 7feba67fc640 1 -- 172.21.15.43:0/107189859 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb40106c0 con 0x5586ab86b8e0 2024-04-21T08:50:09.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.347+0000 7febc93f8740 1 -- 172.21.15.43:0/107189859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86b8e0 msgr2=0x5586ab86bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:09.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.347+0000 7febc4cc8640 1 --2- 172.21.15.43:0/245783442 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7feba80a3d20 0x7feba80a61e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7febb80ce0f0 tx=0x7febb808cf90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:09.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.347+0000 7febc93f8740 1 --2- 172.21.15.43:0/107189859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86b8e0 0x5586ab86bcc0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7febb400bb20 tx=0x7febb40041d0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.347+0000 7febc93f8740 1 -- 172.21.15.43:0/107189859 shutdown_connections 2024-04-21T08:50:09.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.347+0000 7febc93f8740 1 --2- 172.21.15.43:0/107189859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586ab874e10 0x5586ab877260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.347+0000 7febc93f8740 1 --2- 172.21.15.43:0/107189859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586ab86c290 0x5586ab874840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.347+0000 7febc93f8740 1 --2- 172.21.15.43:0/107189859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86b8e0 0x5586ab86bcc0 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.347+0000 7febc93f8740 1 -- 172.21.15.43:0/107189859 >> 172.21.15.43:0/107189859 conn(0x5586ab865fb0 msgr2=0x5586ab866390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:09.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.347+0000 7febc93f8740 1 -- 172.21.15.43:0/107189859 shutdown_connections 2024-04-21T08:50:09.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.347+0000 7febc93f8740 1 -- 172.21.15.43:0/107189859 wait complete. 2024-04-21T08:50:09.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.347+0000 7febc93f8740 1 -- start start 2024-04-21T08:50:09.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.347+0000 7febc93f8740 1 Processor -- start 2024-04-21T08:50:09.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.347+0000 7febc93f8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586ab86b8e0 0x5586ab871e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.348+0000 7febc93f8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86c290 0x5586ab872340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.348+0000 7febc93f8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586ab874e10 0x5586ab872880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.348+0000 7febc93f8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5586ab877d20 con 0x5586ab86c290 2024-04-21T08:50:09.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.348+0000 7febc93f8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5586ab789580 con 0x5586ab874e10 2024-04-21T08:50:09.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.348+0000 7febc93f8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5586ab877ea0 con 0x5586ab86b8e0 2024-04-21T08:50:09.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.348+0000 7febc93f8740 10 client.0 _ll_register_callbacks cb 0x5586ab78f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:09.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.348+0000 7febbffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86c290 0x5586ab872340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.348+0000 7febbffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86c290 0x5586ab872340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39712/0 (socket says 172.21.15.43:39712) 2024-04-21T08:50:09.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.348+0000 7febbffff640 1 -- 172.21.15.43:0/360151423 learned_addr learned my addr 172.21.15.43:0/360151423 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:09.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.348+0000 7febbf7fe640 1 --2- 172.21.15.43:0/360151423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586ab874e10 0x5586ab872880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.348+0000 7febbf7fe640 1 --2- 172.21.15.43:0/360151423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586ab86b8e0 0x5586ab871e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.348+0000 7febbffff640 1 -- 172.21.15.43:0/360151423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586ab86b8e0 msgr2=0x5586ab871e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:09.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.348+0000 7febbffff640 1 --2- 172.21.15.43:0/360151423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586ab86b8e0 0x5586ab871e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.348+0000 7febbffff640 1 -- 172.21.15.43:0/360151423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586ab874e10 msgr2=0x5586ab872880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:09.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.348+0000 7febbffff640 1 --2- 172.21.15.43:0/360151423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586ab874e10 0x5586ab872880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.348+0000 7febbffff640 1 -- 172.21.15.43:0/360151423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586ab60af70 con 0x5586ab86c290 2024-04-21T08:50:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.348+0000 7febbffff640 1 --2- 172.21.15.43:0/360151423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86c290 0x5586ab872340 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7febb0009900 tx=0x7febb002ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6664d3b9317e8fc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.348+0000 7feb9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.348+0000 7feb9f7fe640 1 -- 172.21.15.43:0/360151423 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0005030 con 0x5586ab86c290 2024-04-21T08:50:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.348+0000 7feb9f7fe640 1 -- 172.21.15.43:0/360151423 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7febb00051d0 con 0x5586ab86c290 2024-04-21T08:50:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.348+0000 7febc93f8740 1 -- 172.21.15.43:0/360151423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febb4005670 con 0x5586ab86c290 2024-04-21T08:50:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.349+0000 7feb9f7fe640 1 -- 172.21.15.43:0/360151423 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7febb00053d0 con 0x5586ab86c290 2024-04-21T08:50:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.349+0000 7febc93f8740 10 client.20970 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.349+0000 7febc93f8740 1 -- 172.21.15.43:0/360151423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5586ab4a2170 con 0x5586ab86c290 2024-04-21T08:50:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.349+0000 7feb9f7fe640 1 -- 172.21.15.43:0/360151423 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febb4005670 con 0x5586ab86c290 2024-04-21T08:50:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.349+0000 7feb9f7fe640 1 -- 172.21.15.43:0/360151423 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7febb0003710 con 0x5586ab86c290 2024-04-21T08:50:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.349+0000 7febc93f8740 10 client.20970 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.349+0000 7febc93f8740 10 client.20970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.349+0000 7febc93f8740 20 client.20970 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.349+0000 7febc93f8740 1 --2- 172.21.15.43:0/360151423 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5586ab8853a0 0x5586ab8877c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.349+0000 7febc93f8740 4 client.20970 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.349+0000 7febc93f8740 1 -- 172.21.15.43:0/360151423 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586ab7dd920 con 0x5586ab8853a0 2024-04-21T08:50:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.349+0000 7febbf7fe640 1 --2- 172.21.15.43:0/360151423 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5586ab8853a0 0x5586ab8877c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.350+0000 7febbf7fe640 1 --2- 172.21.15.43:0/360151423 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5586ab8853a0 0x5586ab8877c0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4543af1ba85a61e server_cookie=40a3b17ef32652f7 in_seq=0 out_seq=0 2024-04-21T08:50:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.350+0000 7feb9f7fe640 10 client.20970 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.350+0000 7feb9f7fe640 1 -- 172.21.15.43:0/360151423 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586ab7dd920 con 0x5586ab8853a0 2024-04-21T08:50:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.350+0000 7feb9f7fe640 10 client.20970 handle_command_reply: tid=0 2024-04-21T08:50:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.350+0000 7febc93f8740 1 client.20970 shutdown 2024-04-21T08:50:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.351+0000 7febc93f8740 1 -- 172.21.15.43:0/360151423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86c290 msgr2=0x5586ab872340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.351+0000 7febc93f8740 1 --2- 172.21.15.43:0/360151423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86c290 0x5586ab872340 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7febb0009900 tx=0x7febb002ca10 comp rx=0 tx=0).stop 2024-04-21T08:50:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.351+0000 7febc93f8740 1 -- 172.21.15.43:0/360151423 shutdown_connections 2024-04-21T08:50:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.351+0000 7febc93f8740 1 --2- 172.21.15.43:0/360151423 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5586ab8853a0 0x5586ab8877c0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.351+0000 7febc93f8740 1 --2- 172.21.15.43:0/360151423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586ab874e10 0x5586ab872880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.351+0000 7febc93f8740 1 --2- 172.21.15.43:0/360151423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86c290 0x5586ab872340 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.351+0000 7feb9f7fe640 0 client.20970 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.351+0000 7febc93f8740 1 --2- 172.21.15.43:0/360151423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586ab86b8e0 0x5586ab871e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.351+0000 7febc93f8740 1 -- 172.21.15.43:0/360151423 >> 172.21.15.43:0/360151423 conn(0x5586ab865fb0 msgr2=0x5586ab867d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.351+0000 7febc93f8740 1 -- 172.21.15.43:0/360151423 shutdown_connections 2024-04-21T08:50:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.351+0000 7febc93f8740 1 -- 172.21.15.43:0/360151423 wait complete. 2024-04-21T08:50:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.351+0000 7febc93f8740 20 client.20970 trim_cache size 0 max 16384 2024-04-21T08:50:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.369+0000 7febc93f8740 1 Processor -- start 2024-04-21T08:50:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.369+0000 7febc93f8740 1 -- start start 2024-04-21T08:50:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.369+0000 7febc93f8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586ab86b8e0 0x5586ab880be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.369+0000 7febc93f8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86c290 0x5586ab8726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.369+0000 7febc93f8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586ab874e10 0x5586ab88d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.369+0000 7febc93f8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7febc01af720 con 0x5586ab86c290 2024-04-21T08:50:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.369+0000 7febc93f8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7febc01b1260 con 0x5586ab86b8e0 2024-04-21T08:50:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.369+0000 7febc93f8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7febc01afdf0 con 0x5586ab874e10 2024-04-21T08:50:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.369+0000 7febbffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86c290 0x5586ab8726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.369+0000 7febbffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86c290 0x5586ab8726a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39724/0 (socket says 172.21.15.43:39724) 2024-04-21T08:50:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.369+0000 7febbffff640 1 -- 172.21.15.43:0/2608047605 learned_addr learned my addr 172.21.15.43:0/2608047605 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.369+0000 7febbf7fe640 1 --2- 172.21.15.43:0/2608047605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586ab874e10 0x5586ab88d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.369+0000 7febbf7fe640 1 --2- 172.21.15.43:0/2608047605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586ab86b8e0 0x5586ab880be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.369+0000 7febbffff640 1 -- 172.21.15.43:0/2608047605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586ab874e10 msgr2=0x5586ab88d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.369+0000 7febbffff640 1 --2- 172.21.15.43:0/2608047605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586ab874e10 0x5586ab88d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.369+0000 7febbffff640 1 -- 172.21.15.43:0/2608047605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586ab86b8e0 msgr2=0x5586ab880be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.369+0000 7febbffff640 1 --2- 172.21.15.43:0/2608047605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586ab86b8e0 0x5586ab880be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.369+0000 7febbffff640 1 -- 172.21.15.43:0/2608047605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586ab86d560 con 0x5586ab86c290 2024-04-21T08:50:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.369+0000 7febbf7fe640 1 --2- 172.21.15.43:0/2608047605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586ab874e10 0x5586ab88d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.370+0000 7febbffff640 1 --2- 172.21.15.43:0/2608047605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86c290 0x5586ab8726a0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7febb0006010 tx=0x7febb002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e39f859670288b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.370+0000 7feba5ffb640 1 -- 172.21.15.43:0/2608047605 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0000e60 con 0x5586ab86c290 2024-04-21T08:50:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.370+0000 7feba5ffb640 1 -- 172.21.15.43:0/2608047605 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febb0005850 con 0x5586ab86c290 2024-04-21T08:50:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.370+0000 7feba5ffb640 1 -- 172.21.15.43:0/2608047605 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0034790 con 0x5586ab86c290 2024-04-21T08:50:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.370+0000 7febc93f8740 1 -- 172.21.15.43:0/2608047605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86c290 msgr2=0x5586ab8726a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.370+0000 7febc93f8740 1 --2- 172.21.15.43:0/2608047605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86c290 0x5586ab8726a0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7febb0006010 tx=0x7febb002bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.370+0000 7febc93f8740 1 -- 172.21.15.43:0/2608047605 shutdown_connections 2024-04-21T08:50:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.370+0000 7febc93f8740 1 --2- 172.21.15.43:0/2608047605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586ab874e10 0x5586ab88d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.370+0000 7febc93f8740 1 --2- 172.21.15.43:0/2608047605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86c290 0x5586ab8726a0 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.370+0000 7febc93f8740 1 --2- 172.21.15.43:0/2608047605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586ab86b8e0 0x5586ab880be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.370+0000 7febc93f8740 1 -- 172.21.15.43:0/2608047605 >> 172.21.15.43:0/2608047605 conn(0x5586ab865fb0 msgr2=0x5586ab865010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.371+0000 7febc93f8740 1 -- 172.21.15.43:0/2608047605 shutdown_connections 2024-04-21T08:50:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.371+0000 7febc93f8740 1 -- 172.21.15.43:0/2608047605 wait complete. 2024-04-21T08:50:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.371+0000 7febc93f8740 1 -- start start 2024-04-21T08:50:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.371+0000 7febc93f8740 1 Processor -- start 2024-04-21T08:50:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.371+0000 7febc93f8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86b8e0 0x5586ab87d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.371+0000 7febc93f8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586ab86c290 0x5586ab87a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.371+0000 7febc93f8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586ab874e10 0x5586ab87a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.371+0000 7febc93f8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7febc01b0b90 con 0x5586ab86b8e0 2024-04-21T08:50:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.371+0000 7febc93f8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7febc01b04c0 con 0x5586ab874e10 2024-04-21T08:50:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.371+0000 7febc93f8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7febc01ad720 con 0x5586ab86c290 2024-04-21T08:50:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.371+0000 7febc93f8740 10 client.0 _ll_register_callbacks cb 0x7febb0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.371+0000 7febbf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86b8e0 0x5586ab87d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.371+0000 7febbf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86b8e0 0x5586ab87d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39734/0 (socket says 172.21.15.43:39734) 2024-04-21T08:50:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.371+0000 7febbf7fe640 1 -- 172.21.15.43:0/1211935001 learned_addr learned my addr 172.21.15.43:0/1211935001 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.372+0000 7febbffff640 1 --2- 172.21.15.43:0/1211935001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586ab86c290 0x5586ab87a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.372+0000 7febbf7fe640 1 -- 172.21.15.43:0/1211935001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586ab86c290 msgr2=0x5586ab87a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.372+0000 7febbf7fe640 1 --2- 172.21.15.43:0/1211935001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586ab86c290 0x5586ab87a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.372+0000 7febbf7fe640 1 -- 172.21.15.43:0/1211935001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586ab874e10 msgr2=0x5586ab87a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.372+0000 7febbf7fe640 1 --2- 172.21.15.43:0/1211935001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586ab874e10 0x5586ab87a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.372+0000 7febbf7fe640 1 -- 172.21.15.43:0/1211935001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586ab87ab40 con 0x5586ab86b8e0 2024-04-21T08:50:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.372+0000 7febbf7fe640 1 --2- 172.21.15.43:0/1211935001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86b8e0 0x5586ab87d2e0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7febb4013000 tx=0x7febb400b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43a141711170b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.372+0000 7feb9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:09.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.372+0000 7feb9f7fe640 1 -- 172.21.15.43:0/1211935001 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb40049d0 con 0x5586ab86b8e0 2024-04-21T08:50:09.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.372+0000 7feb9f7fe640 1 -- 172.21.15.43:0/1211935001 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7febb402e2d0 con 0x5586ab86b8e0 2024-04-21T08:50:09.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.372+0000 7febc93f8740 1 -- 172.21.15.43:0/1211935001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febb0005850 con 0x5586ab86b8e0 2024-04-21T08:50:09.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.372+0000 7feb9f7fe640 1 -- 172.21.15.43:0/1211935001 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7febb4004b70 con 0x5586ab86b8e0 2024-04-21T08:50:09.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.372+0000 7febc93f8740 10 client.20976 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:09.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.372+0000 7febc93f8740 1 -- 172.21.15.43:0/1211935001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5586ab8ec6a0 con 0x5586ab86b8e0 2024-04-21T08:50:09.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.372+0000 7feb9f7fe640 1 -- 172.21.15.43:0/1211935001 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febb0005850 con 0x5586ab86b8e0 2024-04-21T08:50:09.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.373+0000 7feb9f7fe640 1 -- 172.21.15.43:0/1211935001 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7febb40053c0 con 0x5586ab86b8e0 2024-04-21T08:50:09.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.373+0000 7febc93f8740 10 client.20976 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:09.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.373+0000 7febc93f8740 10 client.20976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:09.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.373+0000 7febc93f8740 20 client.20976 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:09.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.373+0000 7febc93f8740 1 --2- 172.21.15.43:0/1211935001 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5586ab8ecc60 0x5586ab8ef080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.373+0000 7febc93f8740 4 client.20976 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:50:09.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.373+0000 7febc93f8740 1 -- 172.21.15.43:0/1211935001 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5586ab7d7790 con 0x5586ab8ecc60 2024-04-21T08:50:09.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.373+0000 7febbffff640 1 --2- 172.21.15.43:0/1211935001 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5586ab8ecc60 0x5586ab8ef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.373+0000 7feb9f7fe640 10 client.20976 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:09.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.373+0000 7febbffff640 1 --2- 172.21.15.43:0/1211935001 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5586ab8ecc60 0x5586ab8ef080 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccb9e0ce4bca589 server_cookie=89b6d259dc7f34d2 in_seq=0 out_seq=0 2024-04-21T08:50:09.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.375+0000 7feb9f7fe640 1 -- 172.21.15.43:0/1211935001 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5586ab7d7790 con 0x5586ab8ecc60 2024-04-21T08:50:09.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.375+0000 7feb9f7fe640 10 client.20976 handle_command_reply: tid=0 2024-04-21T08:50:09.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.375+0000 7febc93f8740 1 client.20976 shutdown 2024-04-21T08:50:09.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.375+0000 7febc93f8740 1 -- 172.21.15.43:0/1211935001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86b8e0 msgr2=0x5586ab87d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:09.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.375+0000 7febc93f8740 1 --2- 172.21.15.43:0/1211935001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86b8e0 0x5586ab87d2e0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7febb4013000 tx=0x7febb400b5d0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.375+0000 7febc93f8740 1 -- 172.21.15.43:0/1211935001 shutdown_connections 2024-04-21T08:50:09.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.375+0000 7febc93f8740 1 --2- 172.21.15.43:0/1211935001 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5586ab8ecc60 0x5586ab8ef080 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.375+0000 7febc93f8740 1 --2- 172.21.15.43:0/1211935001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586ab874e10 0x5586ab87a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.375+0000 7febc93f8740 1 --2- 172.21.15.43:0/1211935001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5586ab86c290 0x5586ab87a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.375+0000 7febc93f8740 1 --2- 172.21.15.43:0/1211935001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586ab86b8e0 0x5586ab87d2e0 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.375+0000 7febc93f8740 1 -- 172.21.15.43:0/1211935001 >> 172.21.15.43:0/1211935001 conn(0x5586ab865fb0 msgr2=0x5586ab864780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:09.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.375+0000 7feb9f7fe640 0 client.20976 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:09.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.375+0000 7febc93f8740 1 -- 172.21.15.43:0/1211935001 shutdown_connections 2024-04-21T08:50:09.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.375+0000 7febc93f8740 1 -- 172.21.15.43:0/1211935001 wait complete. 2024-04-21T08:50:09.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.375+0000 7febc93f8740 20 client.20976 trim_cache size 0 max 16384 2024-04-21T08:50:09.380 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:50:09.380 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:50:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "2d11eadb-cae9-4f59-bf19-a10df13b0d39", 2024-04-21T08:50:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:50:09.381 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:50:09.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.376+0000 7febc672b640 1 -- 172.21.15.43:0/245783442 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7feba80a3d20 msgr2=0x7feba80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:09.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.376+0000 7febc672b640 1 --2- 172.21.15.43:0/245783442 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7feba80a3d20 0x7feba80a61e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7febb80ce0f0 tx=0x7febb808cf90 comp rx=0 tx=0).stop 2024-04-21T08:50:09.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.376+0000 7febc672b640 1 -- 172.21.15.43:0/245783442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febc0198610 msgr2=0x7febc0198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:09.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.376+0000 7febc672b640 1 --2- 172.21.15.43:0/245783442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febc0198610 0x7febc0198ac0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7febb800eaa0 tx=0x7febb800ef60 comp rx=0 tx=0).stop 2024-04-21T08:50:09.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.376+0000 7febc672b640 1 -- 172.21.15.43:0/245783442 shutdown_connections 2024-04-21T08:50:09.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.376+0000 7febc672b640 1 --2- 172.21.15.43:0/245783442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febc0198610 0x7febc0198ac0 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.376+0000 7febc672b640 1 --2- 172.21.15.43:0/245783442 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7feba80a3d20 0x7feba80a61e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.376+0000 7febc672b640 1 --2- 172.21.15.43:0/245783442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febc0107260 0x7febc019e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.376+0000 7febc672b640 1 --2- 172.21.15.43:0/245783442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febc0102190 0x7febc019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.376+0000 7febc672b640 1 -- 172.21.15.43:0/245783442 >> 172.21.15.43:0/245783442 conn(0x7febc00fd990 msgr2=0x7febc00fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:09.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.376+0000 7febc672b640 1 -- 172.21.15.43:0/245783442 shutdown_connections 2024-04-21T08:50:09.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.376+0000 7febc672b640 1 -- 172.21.15.43:0/245783442 wait complete. 2024-04-21T08:50:09.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:50:09.387 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:50:09.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.492+0000 7f856a1f4640 1 -- 172.21.15.43:0/4000030530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8564101a40 msgr2=0x7f856410dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.492+0000 7f856a1f4640 1 --2- 172.21.15.43:0/4000030530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8564101a40 0x7f856410dca0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f855c009ca0 tx=0x7f855c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.492+0000 7f856a1f4640 1 -- 172.21.15.43:0/4000030530 shutdown_connections 2024-04-21T08:50:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.492+0000 7f856a1f4640 1 --2- 172.21.15.43:0/4000030530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8564101a40 0x7f856410dca0 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.492+0000 7f856a1f4640 1 --2- 172.21.15.43:0/4000030530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85641010a0 0x7f8564101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.492+0000 7f856a1f4640 1 --2- 172.21.15.43:0/4000030530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8564105dd0 0x7f85641061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.492+0000 7f856a1f4640 1 -- 172.21.15.43:0/4000030530 >> 172.21.15.43:0/4000030530 conn(0x7f8564077c40 msgr2=0x7f8564078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.492+0000 7f856a1f4640 1 -- 172.21.15.43:0/4000030530 shutdown_connections 2024-04-21T08:50:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.492+0000 7f856a1f4640 1 -- 172.21.15.43:0/4000030530 wait complete. 2024-04-21T08:50:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.492+0000 7f856a1f4640 1 Processor -- start 2024-04-21T08:50:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.492+0000 7f856a1f4640 1 -- start start 2024-04-21T08:50:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.493+0000 7f856a1f4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85641010a0 0x7f85640fec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.493+0000 7f856a1f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8564101a40 0x7f85640ff160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.493+0000 7f856a1f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8564105dd0 0x7f85640ff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.493+0000 7f856a1f4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8564111180 con 0x7f8564105dd0 2024-04-21T08:50:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.493+0000 7f856a1f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8564111000 con 0x7f8564101a40 2024-04-21T08:50:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.493+0000 7f856a1f4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8564111300 con 0x7f85641010a0 2024-04-21T08:50:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.493+0000 7f8563fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8564105dd0 0x7f85640ff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.493+0000 7f8563fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8564105dd0 0x7f85640ff6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39746/0 (socket says 172.21.15.43:39746) 2024-04-21T08:50:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.493+0000 7f8563fff640 1 -- 172.21.15.43:0/212517097 learned_addr learned my addr 172.21.15.43:0/212517097 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.493+0000 7f85637fe640 1 --2- 172.21.15.43:0/212517097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85641010a0 0x7f85640fec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.493+0000 7f8562ffd640 1 --2- 172.21.15.43:0/212517097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8564101a40 0x7f85640ff160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.493+0000 7f8563fff640 1 -- 172.21.15.43:0/212517097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85641010a0 msgr2=0x7f85640fec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.493+0000 7f8563fff640 1 --2- 172.21.15.43:0/212517097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85641010a0 0x7f85640fec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.493+0000 7f8563fff640 1 -- 172.21.15.43:0/212517097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8564101a40 msgr2=0x7f85640ff160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.493+0000 7f8563fff640 1 --2- 172.21.15.43:0/212517097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8564101a40 0x7f85640ff160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.493+0000 7f8563fff640 1 -- 172.21.15.43:0/212517097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85641a6c10 con 0x7f8564105dd0 2024-04-21T08:50:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.493+0000 7f8562ffd640 1 --2- 172.21.15.43:0/212517097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8564101a40 0x7f85640ff160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.493+0000 7f85637fe640 1 --2- 172.21.15.43:0/212517097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85641010a0 0x7f85640fec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.493+0000 7f8563fff640 1 --2- 172.21.15.43:0/212517097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8564105dd0 0x7f85640ff6a0 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f855c005e30 tx=0x7f855c034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.493+0000 7f8560ff9640 1 -- 172.21.15.43:0/212517097 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c039040 con 0x7f8564105dd0 2024-04-21T08:50:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.493+0000 7f856a1f4640 1 -- 172.21.15.43:0/212517097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85641a6ea0 con 0x7f8564105dd0 2024-04-21T08:50:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.493+0000 7f8560ff9640 1 -- 172.21.15.43:0/212517097 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f855c044070 con 0x7f8564105dd0 2024-04-21T08:50:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.494+0000 7f8560ff9640 1 -- 172.21.15.43:0/212517097 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c003e60 con 0x7f8564105dd0 2024-04-21T08:50:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.494+0000 7f856a1f4640 1 -- 172.21.15.43:0/212517097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85641a73b0 con 0x7f8564105dd0 2024-04-21T08:50:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.496+0000 7f8560ff9640 1 -- 172.21.15.43:0/212517097 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f855c0324c0 con 0x7f8564105dd0 2024-04-21T08:50:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.496+0000 7f856cde8740 1 Processor -- start 2024-04-21T08:50:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.496+0000 7f856cde8740 1 -- start start 2024-04-21T08:50:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.496+0000 7f856cde8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a469f32580 0x55a469f32960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.496+0000 7f856cde8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a469f32ea0 0x55a469f3b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.496+0000 7f856cde8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f3ba80 0x55a469f3de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.496+0000 7f856cde8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a469e971e0 con 0x55a469f3ba80 2024-04-21T08:50:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.496+0000 7f856cde8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a469e34390 con 0x55a469f32ea0 2024-04-21T08:50:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.496+0000 7f856cde8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a469e931a0 con 0x55a469f32580 2024-04-21T08:50:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.496+0000 7f8560ff9640 1 --2- 172.21.15.43:0/212517097 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f854c0a3d20 0x7f854c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.496+0000 7f8562ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f3ba80 0x55a469f3de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.496+0000 7f8562ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f3ba80 0x55a469f3de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39754/0 (socket says 172.21.15.43:39754) 2024-04-21T08:50:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.496+0000 7f8562ffd640 1 -- 172.21.15.43:0/869871687 learned_addr learned my addr 172.21.15.43:0/869871687 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.496+0000 7f8563fff640 1 --2- 172.21.15.43:0/212517097 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f854c0a3d20 0x7f854c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.496+0000 7f8562ffd640 1 --2- 172.21.15.43:0/869871687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a469f32580 0x55a469f32960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.497+0000 7f85637fe640 1 --2- 172.21.15.43:0/869871687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a469f32ea0 0x55a469f3b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.497+0000 7f8560ff9640 1 -- 172.21.15.43:0/212517097 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7f855c0f2ea0 con 0x7f8564105dd0 2024-04-21T08:50:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.497+0000 7f8562ffd640 1 -- 172.21.15.43:0/869871687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a469f32580 msgr2=0x55a469f32960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.497+0000 7f8562ffd640 1 --2- 172.21.15.43:0/869871687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a469f32580 0x55a469f32960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.497+0000 7f8562ffd640 1 -- 172.21.15.43:0/869871687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a469f32ea0 msgr2=0x55a469f3b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.497+0000 7f8562ffd640 1 --2- 172.21.15.43:0/869871687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a469f32ea0 0x55a469f3b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.497+0000 7f8562ffd640 1 -- 172.21.15.43:0/869871687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a469c737d0 con 0x55a469f3ba80 2024-04-21T08:50:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.497+0000 7f85637fe640 1 --2- 172.21.15.43:0/869871687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a469f32ea0 0x55a469f3b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.497+0000 7f8563fff640 1 --2- 172.21.15.43:0/212517097 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f854c0a3d20 0x7f854c0a61e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f855c0f0330 tx=0x7f855c089570 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.497+0000 7f8562ffd640 1 --2- 172.21.15.43:0/869871687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f3ba80 0x55a469f3de70 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f8558012820 tx=0x7f855800d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcadf5504ed48d9f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.497+0000 7f8549ffb640 1 -- 172.21.15.43:0/869871687 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558012e60 con 0x55a469f3ba80 2024-04-21T08:50:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.497+0000 7f8549ffb640 1 -- 172.21.15.43:0/869871687 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f855801d430 con 0x55a469f3ba80 2024-04-21T08:50:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.497+0000 7f8549ffb640 1 -- 172.21.15.43:0/869871687 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85580105d0 con 0x55a469f3ba80 2024-04-21T08:50:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.498+0000 7f856cde8740 1 -- 172.21.15.43:0/869871687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f3ba80 msgr2=0x55a469f3de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.498+0000 7f856cde8740 1 --2- 172.21.15.43:0/869871687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f3ba80 0x55a469f3de70 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f8558012820 tx=0x7f855800d3c0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.498+0000 7f856cde8740 1 -- 172.21.15.43:0/869871687 shutdown_connections 2024-04-21T08:50:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.498+0000 7f856cde8740 1 --2- 172.21.15.43:0/869871687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f3ba80 0x55a469f3de70 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.498+0000 7f856cde8740 1 --2- 172.21.15.43:0/869871687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a469f32ea0 0x55a469f3b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.498+0000 7f856cde8740 1 --2- 172.21.15.43:0/869871687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a469f32580 0x55a469f32960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.498+0000 7f856cde8740 1 -- 172.21.15.43:0/869871687 >> 172.21.15.43:0/869871687 conn(0x55a469f2ccb0 msgr2=0x55a469f2d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.498+0000 7f856cde8740 1 -- 172.21.15.43:0/869871687 shutdown_connections 2024-04-21T08:50:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.498+0000 7f856cde8740 1 -- 172.21.15.43:0/869871687 wait complete. 2024-04-21T08:50:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.498+0000 7f856cde8740 1 -- start start 2024-04-21T08:50:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.498+0000 7f856cde8740 1 Processor -- start 2024-04-21T08:50:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.499+0000 7f856cde8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a469f32580 0x55a469f38a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.499+0000 7f856cde8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a469f32ea0 0x55a469f38fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.499+0000 7f856cde8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f3ba80 0x55a469f39510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.499+0000 7f856cde8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a469f3e870 con 0x55a469f3ba80 2024-04-21T08:50:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.499+0000 7f856cde8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a469e502f0 con 0x55a469f32ea0 2024-04-21T08:50:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.499+0000 7f856cde8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a469f3e9f0 con 0x55a469f32580 2024-04-21T08:50:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.499+0000 7f856cde8740 10 client.0 _ll_register_callbacks cb 0x55a469e5d580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.499+0000 7f8562ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f3ba80 0x55a469f39510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.499+0000 7f8562ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f3ba80 0x55a469f39510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39758/0 (socket says 172.21.15.43:39758) 2024-04-21T08:50:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.499+0000 7f8562ffd640 1 -- 172.21.15.43:0/1649093232 learned_addr learned my addr 172.21.15.43:0/1649093232 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.499+0000 7f85637fe640 1 --2- 172.21.15.43:0/1649093232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a469f32ea0 0x55a469f38fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.499+0000 7f8562ffd640 1 --2- 172.21.15.43:0/1649093232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a469f32580 0x55a469f38a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.499+0000 7f8562ffd640 1 -- 172.21.15.43:0/1649093232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a469f32580 msgr2=0x55a469f38a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.499+0000 7f8562ffd640 1 --2- 172.21.15.43:0/1649093232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a469f32580 0x55a469f38a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.499+0000 7f8562ffd640 1 -- 172.21.15.43:0/1649093232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a469f32ea0 msgr2=0x55a469f38fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.499+0000 7f8562ffd640 1 --2- 172.21.15.43:0/1649093232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a469f32ea0 0x55a469f38fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.499+0000 7f8562ffd640 1 -- 172.21.15.43:0/1649093232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a469ba93f0 con 0x55a469f3ba80 2024-04-21T08:50:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.499+0000 7f8562ffd640 1 --2- 172.21.15.43:0/1649093232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f3ba80 0x55a469f39510 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f85580058c0 tx=0x7f8558014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1684ad9de732626c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.499+0000 7f8542ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.499+0000 7f8542ffd640 1 -- 172.21.15.43:0/1649093232 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855801d5e0 con 0x55a469f3ba80 2024-04-21T08:50:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.499+0000 7f8542ffd640 1 -- 172.21.15.43:0/1649093232 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7f855801d780 con 0x55a469f3ba80 2024-04-21T08:50:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.499+0000 7f856cde8740 1 -- 172.21.15.43:0/1649093232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f855801d430 con 0x55a469f3ba80 2024-04-21T08:50:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.500+0000 7f8542ffd640 1 -- 172.21.15.43:0/1649093232 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f855801d9a0 con 0x55a469f3ba80 2024-04-21T08:50:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.500+0000 7f856cde8740 10 client.20988 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.500+0000 7f856cde8740 1 -- 172.21.15.43:0/1649093232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a469b690f0 con 0x55a469f3ba80 2024-04-21T08:50:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.500+0000 7f8542ffd640 1 -- 172.21.15.43:0/1649093232 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f855801d430 con 0x55a469f3ba80 2024-04-21T08:50:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.500+0000 7f8542ffd640 1 -- 172.21.15.43:0/1649093232 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f855802b750 con 0x55a469f3ba80 2024-04-21T08:50:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.500+0000 7f856cde8740 10 client.20988 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.500+0000 7f856cde8740 10 client.20988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.500+0000 7f856cde8740 20 client.20988 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.500+0000 7f856cde8740 1 --2- 172.21.15.43:0/1649093232 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a469f4b9a0 0x55a469f4dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.500+0000 7f856cde8740 4 client.20988 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:09.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.500+0000 7f856cde8740 1 -- 172.21.15.43:0/1649093232 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a469ea4c00 con 0x55a469f4b9a0 2024-04-21T08:50:09.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.500+0000 7f85637fe640 1 --2- 172.21.15.43:0/1649093232 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a469f4b9a0 0x55a469f4dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.501+0000 7f8542ffd640 10 client.20988 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:09.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.501+0000 7f85637fe640 1 --2- 172.21.15.43:0/1649093232 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a469f4b9a0 0x55a469f4dd90 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aac1917e6973802 server_cookie=6a345e5cff6e9311 in_seq=0 out_seq=0 2024-04-21T08:50:09.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.501+0000 7f8542ffd640 1 -- 172.21.15.43:0/1649093232 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a469ea4c00 con 0x55a469f4b9a0 2024-04-21T08:50:09.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.501+0000 7f8542ffd640 10 client.20988 handle_command_reply: tid=0 2024-04-21T08:50:09.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.501+0000 7f856cde8740 1 client.20988 shutdown 2024-04-21T08:50:09.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.502+0000 7f856cde8740 1 -- 172.21.15.43:0/1649093232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f3ba80 msgr2=0x55a469f39510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:09.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.502+0000 7f856cde8740 1 --2- 172.21.15.43:0/1649093232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f3ba80 0x55a469f39510 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f85580058c0 tx=0x7f8558014040 comp rx=0 tx=0).stop 2024-04-21T08:50:09.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.502+0000 7f856cde8740 1 -- 172.21.15.43:0/1649093232 shutdown_connections 2024-04-21T08:50:09.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.502+0000 7f856cde8740 1 --2- 172.21.15.43:0/1649093232 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a469f4b9a0 0x55a469f4dd90 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.502+0000 7f856cde8740 1 --2- 172.21.15.43:0/1649093232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f3ba80 0x55a469f39510 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.502+0000 7f856cde8740 1 --2- 172.21.15.43:0/1649093232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a469f32ea0 0x55a469f38fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.502+0000 7f856cde8740 1 --2- 172.21.15.43:0/1649093232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a469f32580 0x55a469f38a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.502+0000 7f856cde8740 1 -- 172.21.15.43:0/1649093232 >> 172.21.15.43:0/1649093232 conn(0x55a469f2ccb0 msgr2=0x55a469f2ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:09.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.502+0000 7f8542ffd640 0 client.20988 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:09.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.502+0000 7f856cde8740 1 -- 172.21.15.43:0/1649093232 shutdown_connections 2024-04-21T08:50:09.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.502+0000 7f856cde8740 1 -- 172.21.15.43:0/1649093232 wait complete. 2024-04-21T08:50:09.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.502+0000 7f856cde8740 20 client.20988 trim_cache size 0 max 16384 2024-04-21T08:50:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.519+0000 7f856cde8740 1 Processor -- start 2024-04-21T08:50:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.519+0000 7f856cde8740 1 -- start start 2024-04-21T08:50:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.519+0000 7f856cde8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a469f32580 0x55a469f3b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.520+0000 7f856cde8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a469f32ea0 0x55a469f47790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.520+0000 7f856cde8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f3ba80 0x55a469f37a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.520+0000 7f856cde8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f85641b17d0 con 0x55a469f3ba80 2024-04-21T08:50:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.520+0000 7f856cde8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85641b3310 con 0x55a469f32ea0 2024-04-21T08:50:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.520+0000 7f856cde8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f85641b1ea0 con 0x55a469f32580 2024-04-21T08:50:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.520+0000 7f8562ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f3ba80 0x55a469f37a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.520+0000 7f8562ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f3ba80 0x55a469f37a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39774/0 (socket says 172.21.15.43:39774) 2024-04-21T08:50:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.520+0000 7f8562ffd640 1 -- 172.21.15.43:0/3448367884 learned_addr learned my addr 172.21.15.43:0/3448367884 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.520+0000 7f8562ffd640 1 --2- 172.21.15.43:0/3448367884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a469f32580 0x55a469f3b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.520+0000 7f85637fe640 1 --2- 172.21.15.43:0/3448367884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a469f32ea0 0x55a469f47790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.520+0000 7f8562ffd640 1 -- 172.21.15.43:0/3448367884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a469f32580 msgr2=0x55a469f3b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.520+0000 7f8562ffd640 1 --2- 172.21.15.43:0/3448367884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a469f32580 0x55a469f3b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.520+0000 7f8562ffd640 1 -- 172.21.15.43:0/3448367884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a469f32ea0 msgr2=0x55a469f47790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.520+0000 7f8562ffd640 1 --2- 172.21.15.43:0/3448367884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a469f32ea0 0x55a469f47790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.520+0000 7f8562ffd640 1 -- 172.21.15.43:0/3448367884 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a469f34690 con 0x55a469f3ba80 2024-04-21T08:50:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.520+0000 7f8562ffd640 1 --2- 172.21.15.43:0/3448367884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f3ba80 0x55a469f37a70 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f855800dac0 tx=0x7f8558000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c025d77de838d85c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.520+0000 7f85497fa640 1 -- 172.21.15.43:0/3448367884 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855801de60 con 0x55a469f3ba80 2024-04-21T08:50:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.520+0000 7f85497fa640 1 -- 172.21.15.43:0/3448367884 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f855801db40 con 0x55a469f3ba80 2024-04-21T08:50:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.520+0000 7f85497fa640 1 -- 172.21.15.43:0/3448367884 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558012070 con 0x55a469f3ba80 2024-04-21T08:50:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.521+0000 7f856cde8740 1 -- 172.21.15.43:0/3448367884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f3ba80 msgr2=0x55a469f37a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.521+0000 7f856cde8740 1 --2- 172.21.15.43:0/3448367884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f3ba80 0x55a469f37a70 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f855800dac0 tx=0x7f8558000bc0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.521+0000 7f856cde8740 1 -- 172.21.15.43:0/3448367884 shutdown_connections 2024-04-21T08:50:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.521+0000 7f856cde8740 1 --2- 172.21.15.43:0/3448367884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f3ba80 0x55a469f37a70 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.521+0000 7f856cde8740 1 --2- 172.21.15.43:0/3448367884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a469f32ea0 0x55a469f47790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.521+0000 7f856cde8740 1 --2- 172.21.15.43:0/3448367884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a469f32580 0x55a469f3b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.521+0000 7f856cde8740 1 -- 172.21.15.43:0/3448367884 >> 172.21.15.43:0/3448367884 conn(0x55a469f2ccb0 msgr2=0x55a469f37360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.521+0000 7f856cde8740 1 -- 172.21.15.43:0/3448367884 shutdown_connections 2024-04-21T08:50:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.521+0000 7f856cde8740 1 -- 172.21.15.43:0/3448367884 wait complete. 2024-04-21T08:50:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.521+0000 7f856cde8740 1 -- start start 2024-04-21T08:50:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.522+0000 7f856cde8740 1 Processor -- start 2024-04-21T08:50:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.522+0000 7f856cde8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f32580 0x55a469f36730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.522+0000 7f856cde8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a469f32ea0 0x55a469f43c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.522+0000 7f856cde8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a469f3ba80 0x55a469f34d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.522+0000 7f856cde8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f85641b2c40 con 0x55a469f32580 2024-04-21T08:50:09.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.522+0000 7f856cde8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85641b2570 con 0x55a469f3ba80 2024-04-21T08:50:09.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.522+0000 7f856cde8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f85641af7d0 con 0x55a469f32ea0 2024-04-21T08:50:09.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.522+0000 7f856cde8740 10 client.0 _ll_register_callbacks cb 0x7f855800dde0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:09.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.522+0000 7f8562ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f32580 0x55a469f36730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.522+0000 7f8562ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f32580 0x55a469f36730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39790/0 (socket says 172.21.15.43:39790) 2024-04-21T08:50:09.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.522+0000 7f8562ffd640 1 -- 172.21.15.43:0/1557283033 learned_addr learned my addr 172.21.15.43:0/1557283033 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:09.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.522+0000 7f8562ffd640 1 --2- 172.21.15.43:0/1557283033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a469f3ba80 0x55a469f34d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.522+0000 7f85637fe640 1 --2- 172.21.15.43:0/1557283033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a469f32ea0 0x55a469f43c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.522+0000 7f8562ffd640 1 -- 172.21.15.43:0/1557283033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a469f32ea0 msgr2=0x55a469f43c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:09.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.522+0000 7f8562ffd640 1 --2- 172.21.15.43:0/1557283033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a469f32ea0 0x55a469f43c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.522+0000 7f8562ffd640 1 -- 172.21.15.43:0/1557283033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a469f3ba80 msgr2=0x55a469f34d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:09.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.522+0000 7f8562ffd640 1 --2- 172.21.15.43:0/1557283033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a469f3ba80 0x55a469f34d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.522+0000 7f8562ffd640 1 -- 172.21.15.43:0/1557283033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a469f35240 con 0x55a469f32580 2024-04-21T08:50:09.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.522+0000 7f8562ffd640 1 --2- 172.21.15.43:0/1557283033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f32580 0x55a469f36730 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f855800aee0 tx=0x7f855800dfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d437104505416d1d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:09.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.522+0000 7f8542ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:09.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.522+0000 7f8542ffd640 1 -- 172.21.15.43:0/1557283033 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855800a860 con 0x55a469f32580 2024-04-21T08:50:09.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.522+0000 7f8542ffd640 1 -- 172.21.15.43:0/1557283033 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7f855800aa00 con 0x55a469f32580 2024-04-21T08:50:09.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.523+0000 7f856cde8740 1 -- 172.21.15.43:0/1557283033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f855801db40 con 0x55a469f32580 2024-04-21T08:50:09.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.523+0000 7f8542ffd640 1 -- 172.21.15.43:0/1557283033 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f855804d050 con 0x55a469f32580 2024-04-21T08:50:09.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.523+0000 7f856cde8740 10 client.20994 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:09.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.523+0000 7f856cde8740 1 -- 172.21.15.43:0/1557283033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a469fb11d0 con 0x55a469f32580 2024-04-21T08:50:09.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.523+0000 7f8542ffd640 1 -- 172.21.15.43:0/1557283033 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f855801db40 con 0x55a469f32580 2024-04-21T08:50:09.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.523+0000 7f8542ffd640 1 -- 172.21.15.43:0/1557283033 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f855802b700 con 0x55a469f32580 2024-04-21T08:50:09.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.523+0000 7f856cde8740 10 client.20994 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:09.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.523+0000 7f856cde8740 10 client.20994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:09.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.523+0000 7f856cde8740 20 client.20994 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:09.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.523+0000 7f856cde8740 1 --2- 172.21.15.43:0/1557283033 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a469fb1830 0x55a469fb3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:09.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.523+0000 7f856cde8740 4 client.20994 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:50:09.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.523+0000 7f856cde8740 1 -- 172.21.15.43:0/1557283033 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a469e9ea70 con 0x55a469fb1830 2024-04-21T08:50:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.523+0000 7f85637fe640 1 --2- 172.21.15.43:0/1557283033 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a469fb1830 0x55a469fb3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.524+0000 7f8542ffd640 10 client.20994 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.524+0000 7f85637fe640 1 --2- 172.21.15.43:0/1557283033 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a469fb1830 0x55a469fb3c20 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79da9e1318172fba server_cookie=c505604ab3d7915 in_seq=0 out_seq=0 2024-04-21T08:50:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.524+0000 7f8542ffd640 1 -- 172.21.15.43:0/1557283033 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a469e9ea70 con 0x55a469fb1830 2024-04-21T08:50:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.524+0000 7f8542ffd640 10 client.20994 handle_command_reply: tid=0 2024-04-21T08:50:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.524+0000 7f856cde8740 1 client.20994 shutdown 2024-04-21T08:50:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.524+0000 7f856cde8740 1 -- 172.21.15.43:0/1557283033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f32580 msgr2=0x55a469f36730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.524+0000 7f856cde8740 1 --2- 172.21.15.43:0/1557283033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f32580 0x55a469f36730 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f855800aee0 tx=0x7f855800dfa0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.524+0000 7f856cde8740 1 -- 172.21.15.43:0/1557283033 shutdown_connections 2024-04-21T08:50:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.524+0000 7f856cde8740 1 --2- 172.21.15.43:0/1557283033 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55a469fb1830 0x55a469fb3c20 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.524+0000 7f856cde8740 1 --2- 172.21.15.43:0/1557283033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a469f3ba80 0x55a469f34d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.524+0000 7f856cde8740 1 --2- 172.21.15.43:0/1557283033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a469f32ea0 0x55a469f43c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.524+0000 7f8542ffd640 0 client.20994 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:09.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.524+0000 7f856cde8740 1 --2- 172.21.15.43:0/1557283033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a469f32580 0x55a469f36730 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.524+0000 7f856cde8740 1 -- 172.21.15.43:0/1557283033 >> 172.21.15.43:0/1557283033 conn(0x55a469f2ccb0 msgr2=0x55a469f2b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:09.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.524+0000 7f856cde8740 1 -- 172.21.15.43:0/1557283033 shutdown_connections 2024-04-21T08:50:09.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.524+0000 7f856cde8740 1 -- 172.21.15.43:0/1557283033 wait complete. 2024-04-21T08:50:09.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.524+0000 7f856cde8740 20 client.20994 trim_cache size 0 max 16384 2024-04-21T08:50:09.531 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:50:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:50:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-21T08:50:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "2d11eadb-cae9-4f59-bf19-a10df13b0d39": { 2024-04-21T08:50:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-21T08:50:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "2d11eadb-cae9-4f59-bf19-a10df13b0d39", 2024-04-21T08:50:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-21T08:50:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:50:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:50:09.532 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:50:09.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.525+0000 7f856a1f4640 1 -- 172.21.15.43:0/212517097 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f854c0a3d20 msgr2=0x7f854c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:09.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.525+0000 7f856a1f4640 1 --2- 172.21.15.43:0/212517097 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f854c0a3d20 0x7f854c0a61e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f855c0f0330 tx=0x7f855c089570 comp rx=0 tx=0).stop 2024-04-21T08:50:09.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.525+0000 7f856a1f4640 1 -- 172.21.15.43:0/212517097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8564105dd0 msgr2=0x7f85640ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:09.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.525+0000 7f856a1f4640 1 --2- 172.21.15.43:0/212517097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8564105dd0 0x7f85640ff6a0 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f855c005e30 tx=0x7f855c034760 comp rx=0 tx=0).stop 2024-04-21T08:50:09.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.525+0000 7f856a1f4640 1 -- 172.21.15.43:0/212517097 shutdown_connections 2024-04-21T08:50:09.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.525+0000 7f856a1f4640 1 --2- 172.21.15.43:0/212517097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8564105dd0 0x7f85640ff6a0 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.525+0000 7f856a1f4640 1 --2- 172.21.15.43:0/212517097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8564101a40 0x7f85640ff160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.525+0000 7f856a1f4640 1 --2- 172.21.15.43:0/212517097 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f854c0a3d20 0x7f854c0a61e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.525+0000 7f856a1f4640 1 --2- 172.21.15.43:0/212517097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85641010a0 0x7f85640fec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:09.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.525+0000 7f856a1f4640 1 -- 172.21.15.43:0/212517097 >> 172.21.15.43:0/212517097 conn(0x7f8564077c40 msgr2=0x7f856406a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:09.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.525+0000 7f856a1f4640 1 -- 172.21.15.43:0/212517097 shutdown_connections 2024-04-21T08:50:09.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:09.525+0000 7f856a1f4640 1 -- 172.21.15.43:0/212517097 wait complete. 2024-04-21T08:50:09.535 INFO:tasks.cephfs.filesystem:scrub status for tag:2d11eadb-cae9-4f59-bf19-a10df13b0d39 - {'path': '/', 'tag': '2d11eadb-cae9-4f59-bf19-a10df13b0d39', 'options': 'recursive,force'} 2024-04-21T08:50:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:10 smithi112 ceph-mon[31085]: pgmap v350: 129 pgs: 129 active+clean; 3.6 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 38 op/s 2024-04-21T08:50:10.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:10 smithi112 ceph-mon[31085]: osdmap e106: 12 total, 12 up, 12 in 2024-04-21T08:50:10.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:10 smithi043 ceph-mon[27978]: pgmap v350: 129 pgs: 129 active+clean; 3.6 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 38 op/s 2024-04-21T08:50:10.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:10 smithi043 ceph-mon[27978]: osdmap e106: 12 total, 12 up, 12 in 2024-04-21T08:50:10.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:10 smithi175 ceph-mon[31920]: pgmap v350: 129 pgs: 129 active+clean; 3.6 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 38 op/s 2024-04-21T08:50:10.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:10 smithi175 ceph-mon[31920]: osdmap e106: 12 total, 12 up, 12 in 2024-04-21T08:50:11.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:11 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:50:11.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:11 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:50:11.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:11 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:50:11.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:11 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:50:11.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:11 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:50:11.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:11 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:50:11.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:11 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:50:11.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:11 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:50:11.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:11 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:50:11.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:11 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:50:11.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:11 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:50:11.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:11 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:50:11.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:11 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:50:11.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:11 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:50:11.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:11 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:50:11.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:11 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:50:11.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:11 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:50:11.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:11 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:50:11.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:11 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:50:11.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:11 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:50:11.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:11 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:50:11.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:11 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:50:11.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:11 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:50:11.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:11 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:50:11.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:11 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:50:11.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:11 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:50:11.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:11 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:50:11.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:11 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:50:11.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:11 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:50:11.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:11 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:50:11.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:11 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:50:11.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:11 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:50:11.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:11 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:50:12.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:12 smithi112 ceph-mon[31085]: pgmap v352: 129 pgs: 129 active+clean; 3.8 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 53 KiB/s wr, 21 op/s 2024-04-21T08:50:12.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:12 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:50:12.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:12 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:50:12.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:12 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:50:12.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:12 smithi043 ceph-mon[27978]: pgmap v352: 129 pgs: 129 active+clean; 3.8 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 53 KiB/s wr, 21 op/s 2024-04-21T08:50:12.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:12 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:50:12.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:12 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:50:12.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:12 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:50:12.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:12 smithi175 ceph-mon[31920]: pgmap v352: 129 pgs: 129 active+clean; 3.8 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 53 KiB/s wr, 21 op/s 2024-04-21T08:50:12.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:12 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:50:12.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:12 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:50:12.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:12 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:50:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.920+0000 7f543e5ce640 1 -- 172.21.15.43:0/3859000456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f543810c520 msgr2=0x7f543810e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.920+0000 7f543e5ce640 1 --2- 172.21.15.43:0/3859000456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f543810c520 0x7f543810e9c0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f5430009ca0 tx=0x7f543002c230 comp rx=0 tx=0).stop 2024-04-21T08:50:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.920+0000 7f543e5ce640 1 -- 172.21.15.43:0/3859000456 shutdown_connections 2024-04-21T08:50:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.920+0000 7f543e5ce640 1 --2- 172.21.15.43:0/3859000456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f543810c520 0x7f543810e9c0 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.920+0000 7f543e5ce640 1 --2- 172.21.15.43:0/3859000456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5438103b00 0x7f543810bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.920+0000 7f543e5ce640 1 --2- 172.21.15.43:0/3859000456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54381031e0 0x7f54381035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.920+0000 7f543e5ce640 1 -- 172.21.15.43:0/3859000456 >> 172.21.15.43:0/3859000456 conn(0x7f5438077fd0 msgr2=0x7f54380783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.920+0000 7f543e5ce640 1 -- 172.21.15.43:0/3859000456 shutdown_connections 2024-04-21T08:50:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.920+0000 7f543e5ce640 1 -- 172.21.15.43:0/3859000456 wait complete. 2024-04-21T08:50:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.921+0000 7f543e5ce640 1 Processor -- start 2024-04-21T08:50:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.921+0000 7f543e5ce640 1 -- start start 2024-04-21T08:50:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.921+0000 7f543e5ce640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54381031e0 0x7f5438100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.921+0000 7f543e5ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5438103b00 0x7f5438101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.921+0000 7f543e5ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f543810c520 0x7f54381018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.921+0000 7f543e5ce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5438112190 con 0x7f5438103b00 2024-04-21T08:50:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.921+0000 7f543e5ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5438112010 con 0x7f543810c520 2024-04-21T08:50:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.921+0000 7f543e5ce640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5438112310 con 0x7f54381031e0 2024-04-21T08:50:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.921+0000 7f54377fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5438103b00 0x7f5438101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.921+0000 7f54377fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5438103b00 0x7f5438101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39816/0 (socket says 172.21.15.43:39816) 2024-04-21T08:50:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.921+0000 7f54377fe640 1 -- 172.21.15.43:0/4046372142 learned_addr learned my addr 172.21.15.43:0/4046372142 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.922+0000 7f543cb6b640 1 --2- 172.21.15.43:0/4046372142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f543810c520 0x7f54381018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.922+0000 7f5437fff640 1 --2- 172.21.15.43:0/4046372142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54381031e0 0x7f5438100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.922+0000 7f54377fe640 1 -- 172.21.15.43:0/4046372142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54381031e0 msgr2=0x7f5438100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.922+0000 7f54377fe640 1 --2- 172.21.15.43:0/4046372142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54381031e0 0x7f5438100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.922+0000 7f54377fe640 1 -- 172.21.15.43:0/4046372142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f543810c520 msgr2=0x7f54381018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.922+0000 7f54377fe640 1 --2- 172.21.15.43:0/4046372142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f543810c520 0x7f54381018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.922+0000 7f54377fe640 1 -- 172.21.15.43:0/4046372142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54381a6b70 con 0x7f5438103b00 2024-04-21T08:50:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.922+0000 7f543cb6b640 1 --2- 172.21.15.43:0/4046372142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f543810c520 0x7f54381018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.922+0000 7f54377fe640 1 --2- 172.21.15.43:0/4046372142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5438103b00 0x7f5438101370 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f542800e9f0 tx=0x7f542800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.922+0000 7f54357fa640 1 -- 172.21.15.43:0/4046372142 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542800cd80 con 0x7f5438103b00 2024-04-21T08:50:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.922+0000 7f54357fa640 1 -- 172.21.15.43:0/4046372142 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54280108e0 con 0x7f5438103b00 2024-04-21T08:50:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.922+0000 7f543e5ce640 1 -- 172.21.15.43:0/4046372142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54381a6e60 con 0x7f5438103b00 2024-04-21T08:50:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.922+0000 7f54357fa640 1 -- 172.21.15.43:0/4046372142 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428018980 con 0x7f5438103b00 2024-04-21T08:50:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.922+0000 7f543e5ce640 1 -- 172.21.15.43:0/4046372142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54381a7320 con 0x7f5438103b00 2024-04-21T08:50:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.924+0000 7f54357fa640 1 -- 172.21.15.43:0/4046372142 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5428010430 con 0x7f5438103b00 2024-04-21T08:50:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.925+0000 7f54411be740 1 Processor -- start 2024-04-21T08:50:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.925+0000 7f54411be740 1 -- start start 2024-04-21T08:50:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.925+0000 7f54357fa640 1 --2- 172.21.15.43:0/4046372142 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f54240a3dc0 0x7f54240a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.925+0000 7f54411be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775ddf78d0 0x55775ddf6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.925+0000 7f54411be740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf6b60 0x55775ddf6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.925+0000 7f54411be740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55775de000e0 0x55775de024d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.925+0000 7f54411be740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55775dd5b8d0 con 0x55775ddf6b60 2024-04-21T08:50:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.925+0000 7f54411be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55775dcf8e80 con 0x55775ddf78d0 2024-04-21T08:50:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.925+0000 7f54411be740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55775dd57890 con 0x55775de000e0 2024-04-21T08:50:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.925+0000 7f543cb6b640 1 --2- 172.21.15.43:0/4046372142 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f54240a3dc0 0x7f54240a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.925+0000 7f54357fa640 1 -- 172.21.15.43:0/4046372142 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f54280d1060 con 0x7f5438103b00 2024-04-21T08:50:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.925+0000 7f54377fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf6b60 0x55775ddf6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.925+0000 7f54377fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf6b60 0x55775ddf6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39820/0 (socket says 172.21.15.43:39820) 2024-04-21T08:50:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.925+0000 7f54377fe640 1 -- 172.21.15.43:0/3922535474 learned_addr learned my addr 172.21.15.43:0/3922535474 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.925+0000 7f543cb6b640 1 --2- 172.21.15.43:0/3922535474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55775de000e0 0x55775de024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.926+0000 7f5437fff640 1 --2- 172.21.15.43:0/3922535474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775ddf78d0 0x55775ddf6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.926+0000 7f54377fe640 1 -- 172.21.15.43:0/3922535474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55775de000e0 msgr2=0x55775de024d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.926+0000 7f54377fe640 1 --2- 172.21.15.43:0/3922535474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55775de000e0 0x55775de024d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.926+0000 7f54377fe640 1 -- 172.21.15.43:0/3922535474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775ddf78d0 msgr2=0x55775ddf6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.926+0000 7f54377fe640 1 --2- 172.21.15.43:0/3922535474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775ddf78d0 0x55775ddf6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.926+0000 7f54377fe640 1 -- 172.21.15.43:0/3922535474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55775dd1a320 con 0x55775ddf6b60 2024-04-21T08:50:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.926+0000 7f543cb6b640 1 --2- 172.21.15.43:0/4046372142 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f54240a3dc0 0x7f54240a6280 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5430006010 tx=0x7f54300323c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.926+0000 7f54377fe640 1 --2- 172.21.15.43:0/3922535474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf6b60 0x55775ddf6fe0 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f5428098fe0 tx=0x7f54280107e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a47f3c26a9c546 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.926+0000 7f542e7fc640 1 -- 172.21.15.43:0/3922535474 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54280a0f30 con 0x55775ddf6b60 2024-04-21T08:50:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.926+0000 7f542e7fc640 1 -- 172.21.15.43:0/3922535474 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54280a57f0 con 0x55775ddf6b60 2024-04-21T08:50:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.926+0000 7f542e7fc640 1 -- 172.21.15.43:0/3922535474 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54280ce5b0 con 0x55775ddf6b60 2024-04-21T08:50:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.927+0000 7f54411be740 1 -- 172.21.15.43:0/3922535474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf6b60 msgr2=0x55775ddf6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.927+0000 7f54411be740 1 --2- 172.21.15.43:0/3922535474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf6b60 0x55775ddf6fe0 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f5428098fe0 tx=0x7f54280107e0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.927+0000 7f54411be740 1 -- 172.21.15.43:0/3922535474 shutdown_connections 2024-04-21T08:50:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.927+0000 7f54411be740 1 --2- 172.21.15.43:0/3922535474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55775de000e0 0x55775de024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.927+0000 7f54411be740 1 --2- 172.21.15.43:0/3922535474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf6b60 0x55775ddf6fe0 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.927+0000 7f54411be740 1 --2- 172.21.15.43:0/3922535474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775ddf78d0 0x55775ddf6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.927+0000 7f54411be740 1 -- 172.21.15.43:0/3922535474 >> 172.21.15.43:0/3922535474 conn(0x55775ddbdc90 msgr2=0x55775ddf1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.927+0000 7f54411be740 1 -- 172.21.15.43:0/3922535474 shutdown_connections 2024-04-21T08:50:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.927+0000 7f54411be740 1 -- 172.21.15.43:0/3922535474 wait complete. 2024-04-21T08:50:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.927+0000 7f54411be740 1 -- start start 2024-04-21T08:50:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.928+0000 7f54411be740 1 Processor -- start 2024-04-21T08:50:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.928+0000 7f54411be740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf6b60 0x55775ddffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.928+0000 7f54411be740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55775ddf78d0 0x55775ddfce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.928+0000 7f54411be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775de000e0 0x55775ddfd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.928+0000 7f54411be740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55775de02db0 con 0x55775ddf6b60 2024-04-21T08:50:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.928+0000 7f54411be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55775dce3eb0 con 0x55775de000e0 2024-04-21T08:50:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.928+0000 7f54411be740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55775de02f30 con 0x55775ddf78d0 2024-04-21T08:50:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.928+0000 7f54411be740 10 client.0 _ll_register_callbacks cb 0x55775dd10b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.928+0000 7f5437fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf6b60 0x55775ddffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.928+0000 7f5437fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf6b60 0x55775ddffaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39834/0 (socket says 172.21.15.43:39834) 2024-04-21T08:50:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.928+0000 7f5437fff640 1 -- 172.21.15.43:0/1354624304 learned_addr learned my addr 172.21.15.43:0/1354624304 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.928+0000 7f54377fe640 1 --2- 172.21.15.43:0/1354624304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55775ddf78d0 0x55775ddfce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.928+0000 7f543cb6b640 1 --2- 172.21.15.43:0/1354624304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775de000e0 0x55775ddfd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.928+0000 7f5437fff640 1 -- 172.21.15.43:0/1354624304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55775ddf78d0 msgr2=0x55775ddfce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.928+0000 7f5437fff640 1 --2- 172.21.15.43:0/1354624304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55775ddf78d0 0x55775ddfce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.928+0000 7f5437fff640 1 -- 172.21.15.43:0/1354624304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775de000e0 msgr2=0x55775ddfd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.928+0000 7f5437fff640 1 --2- 172.21.15.43:0/1354624304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775de000e0 0x55775ddfd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.928+0000 7f5437fff640 1 -- 172.21.15.43:0/1354624304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55775da6e210 con 0x55775ddf6b60 2024-04-21T08:50:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.928+0000 7f5437fff640 1 --2- 172.21.15.43:0/1354624304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf6b60 0x55775ddffaf0 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f5420012720 tx=0x7f5420012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da5df4f1338abf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.928+0000 7f54177fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.929+0000 7f54177fe640 1 -- 172.21.15.43:0/1354624304 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5420011bb0 con 0x55775ddf6b60 2024-04-21T08:50:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.929+0000 7f54177fe640 1 -- 172.21.15.43:0/1354624304 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f5420011d50 con 0x55775ddf6b60 2024-04-21T08:50:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.929+0000 7f54411be740 1 -- 172.21.15.43:0/1354624304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54280a57f0 con 0x55775ddf6b60 2024-04-21T08:50:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.929+0000 7f54177fe640 1 -- 172.21.15.43:0/1354624304 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f542000f460 con 0x55775ddf6b60 2024-04-21T08:50:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.929+0000 7f54411be740 10 client.21006 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.929+0000 7f54411be740 1 -- 172.21.15.43:0/1354624304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55775db3a620 con 0x55775ddf6b60 2024-04-21T08:50:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.929+0000 7f54177fe640 1 -- 172.21.15.43:0/1354624304 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54280a57f0 con 0x55775ddf6b60 2024-04-21T08:50:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.929+0000 7f54177fe640 1 -- 172.21.15.43:0/1354624304 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f542001c7c0 con 0x55775ddf6b60 2024-04-21T08:50:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.929+0000 7f54411be740 10 client.21006 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.929+0000 7f54411be740 10 client.21006 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.929+0000 7f54411be740 20 client.21006 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.929+0000 7f54411be740 1 --2- 172.21.15.43:0/1354624304 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55775de101c0 0x55775de125d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.929+0000 7f54411be740 4 client.21006 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.929+0000 7f54411be740 1 -- 172.21.15.43:0/1354624304 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55775dd6f880 con 0x55775de101c0 2024-04-21T08:50:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.930+0000 7f54377fe640 1 --2- 172.21.15.43:0/1354624304 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55775de101c0 0x55775de125d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.930+0000 7f54377fe640 1 --2- 172.21.15.43:0/1354624304 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55775de101c0 0x55775de125d0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50d914cc74660224 server_cookie=10c5323bc1b58c33 in_seq=0 out_seq=0 2024-04-21T08:50:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.930+0000 7f54177fe640 10 client.21006 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.931+0000 7f54177fe640 1 -- 172.21.15.43:0/1354624304 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55775dd6f880 con 0x55775de101c0 2024-04-21T08:50:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.931+0000 7f54177fe640 10 client.21006 handle_command_reply: tid=0 2024-04-21T08:50:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.931+0000 7f54411be740 1 client.21006 shutdown 2024-04-21T08:50:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.931+0000 7f54411be740 1 -- 172.21.15.43:0/1354624304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf6b60 msgr2=0x55775ddffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.931+0000 7f54411be740 1 --2- 172.21.15.43:0/1354624304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf6b60 0x55775ddffaf0 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f5420012720 tx=0x7f5420012be0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.931+0000 7f54411be740 1 -- 172.21.15.43:0/1354624304 shutdown_connections 2024-04-21T08:50:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.931+0000 7f54411be740 1 --2- 172.21.15.43:0/1354624304 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55775de101c0 0x55775de125d0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.931+0000 7f54411be740 1 --2- 172.21.15.43:0/1354624304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775de000e0 0x55775ddfd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.931+0000 7f54411be740 1 --2- 172.21.15.43:0/1354624304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55775ddf78d0 0x55775ddfce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.931+0000 7f54411be740 1 --2- 172.21.15.43:0/1354624304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf6b60 0x55775ddffaf0 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.931+0000 7f54411be740 1 -- 172.21.15.43:0/1354624304 >> 172.21.15.43:0/1354624304 conn(0x55775ddbdc90 msgr2=0x55775ddf35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.931+0000 7f54177fe640 0 client.21006 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.931+0000 7f54411be740 1 -- 172.21.15.43:0/1354624304 shutdown_connections 2024-04-21T08:50:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.931+0000 7f54411be740 1 -- 172.21.15.43:0/1354624304 wait complete. 2024-04-21T08:50:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.931+0000 7f54411be740 20 client.21006 trim_cache size 0 max 16384 2024-04-21T08:50:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.946+0000 7f54411be740 1 Processor -- start 2024-04-21T08:50:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.946+0000 7f54411be740 1 -- start start 2024-04-21T08:50:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.946+0000 7f54411be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775ddf6b60 0x55775de0bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.946+0000 7f54411be740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf78d0 0x55775ddff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.946+0000 7f54411be740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55775de000e0 0x55775ddffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.946+0000 7f54411be740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f54381b1700 con 0x55775ddf78d0 2024-04-21T08:50:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.946+0000 7f54411be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54381b3240 con 0x55775ddf6b60 2024-04-21T08:50:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.946+0000 7f54411be740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54381b1dd0 con 0x55775de000e0 2024-04-21T08:50:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.946+0000 7f54377fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf78d0 0x55775ddff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.946+0000 7f54377fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf78d0 0x55775ddff5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39836/0 (socket says 172.21.15.43:39836) 2024-04-21T08:50:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.946+0000 7f54377fe640 1 -- 172.21.15.43:0/3982021924 learned_addr learned my addr 172.21.15.43:0/3982021924 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.946+0000 7f5437fff640 1 --2- 172.21.15.43:0/3982021924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775ddf6b60 0x55775de0bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.947+0000 7f543cb6b640 1 --2- 172.21.15.43:0/3982021924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55775de000e0 0x55775ddffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.947+0000 7f54377fe640 1 -- 172.21.15.43:0/3982021924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55775de000e0 msgr2=0x55775ddffaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.947+0000 7f54377fe640 1 --2- 172.21.15.43:0/3982021924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55775de000e0 0x55775ddffaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.947+0000 7f54377fe640 1 -- 172.21.15.43:0/3982021924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775ddf6b60 msgr2=0x55775de0bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.947+0000 7f54377fe640 1 --2- 172.21.15.43:0/3982021924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775ddf6b60 0x55775de0bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.947+0000 7f54377fe640 1 -- 172.21.15.43:0/3982021924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55775ddf5e30 con 0x55775ddf78d0 2024-04-21T08:50:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.947+0000 7f5437fff640 1 --2- 172.21.15.43:0/3982021924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775ddf6b60 0x55775de0bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.947+0000 7f54377fe640 1 --2- 172.21.15.43:0/3982021924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf78d0 0x55775ddff5b0 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f54280a10f0 tx=0x7f5428098f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb8ec08843eeed2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.947+0000 7f542dffb640 1 -- 172.21.15.43:0/3982021924 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428010a90 con 0x55775ddf78d0 2024-04-21T08:50:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.947+0000 7f542dffb640 1 -- 172.21.15.43:0/3982021924 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54280bd0b0 con 0x55775ddf78d0 2024-04-21T08:50:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.947+0000 7f542dffb640 1 -- 172.21.15.43:0/3982021924 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428087350 con 0x55775ddf78d0 2024-04-21T08:50:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.947+0000 7f54411be740 1 -- 172.21.15.43:0/3982021924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf78d0 msgr2=0x55775ddff5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.947+0000 7f54411be740 1 --2- 172.21.15.43:0/3982021924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf78d0 0x55775ddff5b0 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f54280a10f0 tx=0x7f5428098f50 comp rx=0 tx=0).stop 2024-04-21T08:50:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.948+0000 7f54411be740 1 -- 172.21.15.43:0/3982021924 shutdown_connections 2024-04-21T08:50:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.948+0000 7f54411be740 1 --2- 172.21.15.43:0/3982021924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55775de000e0 0x55775ddffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.948+0000 7f54411be740 1 --2- 172.21.15.43:0/3982021924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf78d0 0x55775ddff5b0 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.948+0000 7f54411be740 1 --2- 172.21.15.43:0/3982021924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775ddf6b60 0x55775de0bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.948+0000 7f54411be740 1 -- 172.21.15.43:0/3982021924 >> 172.21.15.43:0/3982021924 conn(0x55775ddbdc90 msgr2=0x55775ddf8060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.948+0000 7f54411be740 1 -- 172.21.15.43:0/3982021924 shutdown_connections 2024-04-21T08:50:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.948+0000 7f54411be740 1 -- 172.21.15.43:0/3982021924 wait complete. 2024-04-21T08:50:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.948+0000 7f54411be740 1 -- start start 2024-04-21T08:50:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.948+0000 7f54411be740 1 Processor -- start 2024-04-21T08:50:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.948+0000 7f54411be740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf6b60 0x55775de081c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.948+0000 7f54411be740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55775ddf78d0 0x55775de08700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.948+0000 7f54411be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775de000e0 0x55775ddf9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.949+0000 7f54411be740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f54381b2b70 con 0x55775ddf6b60 2024-04-21T08:50:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.949+0000 7f54411be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54381b24a0 con 0x55775de000e0 2024-04-21T08:50:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.949+0000 7f54411be740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54381af700 con 0x55775ddf78d0 2024-04-21T08:50:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.949+0000 7f54411be740 10 client.0 _ll_register_callbacks cb 0x7f5420009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.949+0000 7f5437fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf6b60 0x55775de081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.949+0000 7f5437fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf6b60 0x55775de081c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39846/0 (socket says 172.21.15.43:39846) 2024-04-21T08:50:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.949+0000 7f5437fff640 1 -- 172.21.15.43:0/3302027298 learned_addr learned my addr 172.21.15.43:0/3302027298 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.949+0000 7f543cb6b640 1 --2- 172.21.15.43:0/3302027298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775de000e0 0x55775ddf9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.949+0000 7f54377fe640 1 --2- 172.21.15.43:0/3302027298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55775ddf78d0 0x55775de08700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.949+0000 7f5437fff640 1 -- 172.21.15.43:0/3302027298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55775ddf78d0 msgr2=0x55775de08700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.949+0000 7f5437fff640 1 --2- 172.21.15.43:0/3302027298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55775ddf78d0 0x55775de08700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.949+0000 7f5437fff640 1 -- 172.21.15.43:0/3302027298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775de000e0 msgr2=0x55775ddf9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.949+0000 7f5437fff640 1 --2- 172.21.15.43:0/3302027298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775de000e0 0x55775ddf9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.949+0000 7f5437fff640 1 -- 172.21.15.43:0/3302027298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55775ddf9760 con 0x55775ddf6b60 2024-04-21T08:50:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.949+0000 7f54377fe640 1 --2- 172.21.15.43:0/3302027298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55775ddf78d0 0x55775de08700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.949+0000 7f543cb6b640 1 --2- 172.21.15.43:0/3302027298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775de000e0 0x55775ddf9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.949+0000 7f5437fff640 1 --2- 172.21.15.43:0/3302027298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf6b60 0x55775de081c0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f542001ce20 tx=0x7f54200056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9f51224a93d637 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.949+0000 7f54177fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.949+0000 7f54177fe640 1 -- 172.21.15.43:0/3302027298 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542002e030 con 0x55775ddf6b60 2024-04-21T08:50:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.949+0000 7f54177fe640 1 -- 172.21.15.43:0/3302027298 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f5420018050 con 0x55775ddf6b60 2024-04-21T08:50:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.949+0000 7f54411be740 1 -- 172.21.15.43:0/3302027298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54280bd0b0 con 0x55775ddf6b60 2024-04-21T08:50:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.950+0000 7f54177fe640 1 -- 172.21.15.43:0/3302027298 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f542003a050 con 0x55775ddf6b60 2024-04-21T08:50:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.950+0000 7f54411be740 10 client.21012 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.950+0000 7f54411be740 1 -- 172.21.15.43:0/3302027298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55775de76070 con 0x55775ddf6b60 2024-04-21T08:50:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.950+0000 7f54177fe640 1 -- 172.21.15.43:0/3302027298 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54280bd0b0 con 0x55775ddf6b60 2024-04-21T08:50:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.950+0000 7f54177fe640 1 -- 172.21.15.43:0/3302027298 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f542000fcd0 con 0x55775ddf6b60 2024-04-21T08:50:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.950+0000 7f54411be740 10 client.21012 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.950+0000 7f54411be740 10 client.21012 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.950+0000 7f54411be740 20 client.21012 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.950+0000 7f54411be740 1 --2- 172.21.15.43:0/3302027298 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55775de76820 0x55775de78c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.950+0000 7f54411be740 4 client.21012 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:50:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.950+0000 7f54411be740 1 -- 172.21.15.43:0/3302027298 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55775dcfb3d0 con 0x55775de76820 2024-04-21T08:50:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.950+0000 7f54377fe640 1 --2- 172.21.15.43:0/3302027298 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55775de76820 0x55775de78c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.951+0000 7f54377fe640 1 --2- 172.21.15.43:0/3302027298 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55775de76820 0x55775de78c40 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=547f564215f41259 server_cookie=3390886fdd997891 in_seq=0 out_seq=0 2024-04-21T08:50:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.951+0000 7f54177fe640 10 client.21012 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.966+0000 7f54177fe640 1 -- 172.21.15.43:0/3302027298 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55775dcfb3d0 con 0x55775de76820 2024-04-21T08:50:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.966+0000 7f54177fe640 10 client.21012 handle_command_reply: tid=0 2024-04-21T08:50:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.966+0000 7f54411be740 1 client.21012 shutdown 2024-04-21T08:50:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.967+0000 7f54411be740 1 -- 172.21.15.43:0/3302027298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf6b60 msgr2=0x55775de081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.967+0000 7f54411be740 1 --2- 172.21.15.43:0/3302027298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf6b60 0x55775de081c0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f542001ce20 tx=0x7f54200056b0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.967+0000 7f54411be740 1 -- 172.21.15.43:0/3302027298 shutdown_connections 2024-04-21T08:50:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.967+0000 7f54411be740 1 --2- 172.21.15.43:0/3302027298 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55775de76820 0x55775de78c40 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.967+0000 7f54411be740 1 --2- 172.21.15.43:0/3302027298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55775de000e0 0x55775ddf9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.967+0000 7f54411be740 1 --2- 172.21.15.43:0/3302027298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55775ddf78d0 0x55775de08700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.967+0000 7f54177fe640 0 client.21012 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.967+0000 7f54411be740 1 --2- 172.21.15.43:0/3302027298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55775ddf6b60 0x55775de081c0 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.967+0000 7f54411be740 1 -- 172.21.15.43:0/3302027298 >> 172.21.15.43:0/3302027298 conn(0x55775ddbdc90 msgr2=0x55775ddf7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.967+0000 7f54411be740 1 -- 172.21.15.43:0/3302027298 shutdown_connections 2024-04-21T08:50:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.967+0000 7f54411be740 1 -- 172.21.15.43:0/3302027298 wait complete. 2024-04-21T08:50:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.967+0000 7f54411be740 20 client.21012 trim_cache size 0 max 16384 2024-04-21T08:50:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:50:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:50:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:50:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:50:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.967+0000 7f543e5ce640 1 -- 172.21.15.43:0/4046372142 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f54240a3dc0 msgr2=0x7f54240a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.967+0000 7f543e5ce640 1 --2- 172.21.15.43:0/4046372142 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f54240a3dc0 0x7f54240a6280 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5430006010 tx=0x7f54300323c0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.967+0000 7f543e5ce640 1 -- 172.21.15.43:0/4046372142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5438103b00 msgr2=0x7f5438101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.967+0000 7f543e5ce640 1 --2- 172.21.15.43:0/4046372142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5438103b00 0x7f5438101370 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f542800e9f0 tx=0x7f542800eeb0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.968+0000 7f543e5ce640 1 -- 172.21.15.43:0/4046372142 shutdown_connections 2024-04-21T08:50:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.968+0000 7f543e5ce640 1 --2- 172.21.15.43:0/4046372142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f543810c520 0x7f54381018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.968+0000 7f543e5ce640 1 --2- 172.21.15.43:0/4046372142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5438103b00 0x7f5438101370 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.968+0000 7f543e5ce640 1 --2- 172.21.15.43:0/4046372142 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f54240a3dc0 0x7f54240a6280 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.968+0000 7f543e5ce640 1 --2- 172.21.15.43:0/4046372142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54381031e0 0x7f5438100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.968+0000 7f543e5ce640 1 -- 172.21.15.43:0/4046372142 >> 172.21.15.43:0/4046372142 conn(0x7f5438077fd0 msgr2=0x7f54380fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.968+0000 7f543e5ce640 1 -- 172.21.15.43:0/4046372142 shutdown_connections 2024-04-21T08:50:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:13.968+0000 7f543e5ce640 1 -- 172.21.15.43:0/4046372142 wait complete. 2024-04-21T08:50:14.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:14 smithi112 ceph-mon[31085]: pgmap v353: 129 pgs: 129 active+clean; 3.8 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 48 KiB/s wr, 19 op/s 2024-04-21T08:50:14.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:14 smithi043 ceph-mon[27978]: pgmap v353: 129 pgs: 129 active+clean; 3.8 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 48 KiB/s wr, 19 op/s 2024-04-21T08:50:14.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:14 smithi175 ceph-mon[31920]: pgmap v353: 129 pgs: 129 active+clean; 3.8 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 48 KiB/s wr, 19 op/s 2024-04-21T08:50:15.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:15 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:50:15.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:15 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:50:15.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:15 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:50:16.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:16 smithi112 ceph-mon[31085]: pgmap v354: 129 pgs: 129 active+clean; 3.8 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-21T08:50:16.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:16 smithi043 ceph-mon[27978]: pgmap v354: 129 pgs: 129 active+clean; 3.8 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-21T08:50:16.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:16 smithi175 ceph-mon[31920]: pgmap v354: 129 pgs: 129 active+clean; 3.8 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-21T08:50:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:18 smithi112 ceph-mon[31085]: pgmap v355: 129 pgs: 129 active+clean; 4.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-04-21T08:50:18.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:18 smithi043 ceph-mon[27978]: pgmap v355: 129 pgs: 129 active+clean; 4.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-04-21T08:50:18.717 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:18 smithi175 ceph-mon[31920]: pgmap v355: 129 pgs: 129 active+clean; 4.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-04-21T08:50:20.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:20 smithi112 ceph-mon[31085]: pgmap v356: 129 pgs: 129 active+clean; 4.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-04-21T08:50:20.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:20 smithi043 ceph-mon[27978]: pgmap v356: 129 pgs: 129 active+clean; 4.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-04-21T08:50:20.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:20 smithi175 ceph-mon[31920]: pgmap v356: 129 pgs: 129 active+clean; 4.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-04-21T08:50:22.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:22 smithi112 ceph-mon[31085]: pgmap v357: 129 pgs: 129 active+clean; 4.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 65 KiB/s wr, 25 op/s 2024-04-21T08:50:22.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:22 smithi043 ceph-mon[27978]: pgmap v357: 129 pgs: 129 active+clean; 4.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 65 KiB/s wr, 25 op/s 2024-04-21T08:50:22.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:22 smithi175 ceph-mon[31920]: pgmap v357: 129 pgs: 129 active+clean; 4.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 65 KiB/s wr, 25 op/s 2024-04-21T08:50:24.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:24 smithi112 ceph-mon[31085]: pgmap v358: 129 pgs: 129 active+clean; 4.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:50:24.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:24 smithi043 ceph-mon[27978]: pgmap v358: 129 pgs: 129 active+clean; 4.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:50:24.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:24 smithi175 ceph-mon[31920]: pgmap v358: 129 pgs: 129 active+clean; 4.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:50:26.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:26 smithi112 ceph-mon[31085]: pgmap v359: 129 pgs: 129 active+clean; 4.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:50:26.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:26 smithi043 ceph-mon[27978]: pgmap v359: 129 pgs: 129 active+clean; 4.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:50:26.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:26 smithi175 ceph-mon[31920]: pgmap v359: 129 pgs: 129 active+clean; 4.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:50:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.066+0000 7fe4df124640 1 -- 172.21.15.43:0/1600813520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4d8103180 msgr2=0x7fe4d8103560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.066+0000 7fe4df124640 1 --2- 172.21.15.43:0/1600813520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4d8103180 0x7fe4d8103560 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009900 tx=0x7fe4c002bcd0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.066+0000 7fe4df124640 1 -- 172.21.15.43:0/1600813520 shutdown_connections 2024-04-21T08:50:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.066+0000 7fe4df124640 1 --2- 172.21.15.43:0/1600813520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4d810c550 0x7fe4d810e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.066+0000 7fe4df124640 1 --2- 172.21.15.43:0/1600813520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4d8103aa0 0x7fe4d810bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.066+0000 7fe4df124640 1 --2- 172.21.15.43:0/1600813520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4d8103180 0x7fe4d8103560 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.066+0000 7fe4df124640 1 -- 172.21.15.43:0/1600813520 >> 172.21.15.43:0/1600813520 conn(0x7fe4d8077fd0 msgr2=0x7fe4d80783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.066+0000 7fe4df124640 1 -- 172.21.15.43:0/1600813520 shutdown_connections 2024-04-21T08:50:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.066+0000 7fe4df124640 1 -- 172.21.15.43:0/1600813520 wait complete. 2024-04-21T08:50:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.067+0000 7fe4df124640 1 Processor -- start 2024-04-21T08:50:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.067+0000 7fe4df124640 1 -- start start 2024-04-21T08:50:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.067+0000 7fe4df124640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4d8103180 0x7fe4d8198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.067+0000 7fe4df124640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4d8103aa0 0x7fe4d8198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.067+0000 7fe4df124640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4d810c550 0x7fe4d819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.067+0000 7fe4df124640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d81121b0 con 0x7fe4d8103180 2024-04-21T08:50:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.067+0000 7fe4df124640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d8112030 con 0x7fe4d810c550 2024-04-21T08:50:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.067+0000 7fe4df124640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d8112330 con 0x7fe4d8103aa0 2024-04-21T08:50:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.067+0000 7fe4dcec0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4d8103180 0x7fe4d8198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.068+0000 7fe4dcec0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4d8103180 0x7fe4d8198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41862/0 (socket says 172.21.15.43:41862) 2024-04-21T08:50:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.068+0000 7fe4dcec0640 1 -- 172.21.15.43:0/616218939 learned_addr learned my addr 172.21.15.43:0/616218939 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.068+0000 7fe4dd6c1640 1 --2- 172.21.15.43:0/616218939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4d810c550 0x7fe4d819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.068+0000 7fe4d7fff640 1 --2- 172.21.15.43:0/616218939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4d8103aa0 0x7fe4d8198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.068+0000 7fe4dcec0640 1 -- 172.21.15.43:0/616218939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4d8103aa0 msgr2=0x7fe4d8198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.068+0000 7fe4dcec0640 1 --2- 172.21.15.43:0/616218939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4d8103aa0 0x7fe4d8198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.068+0000 7fe4dcec0640 1 -- 172.21.15.43:0/616218939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4d810c550 msgr2=0x7fe4d819cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.068+0000 7fe4dcec0640 1 --2- 172.21.15.43:0/616218939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4d810c550 0x7fe4d819cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.068+0000 7fe4dcec0640 1 -- 172.21.15.43:0/616218939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d819d620 con 0x7fe4d8103180 2024-04-21T08:50:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.068+0000 7fe4dcec0640 1 --2- 172.21.15.43:0/616218939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4d8103180 0x7fe4d8198630 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0005940 tx=0x7fe4c00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.068+0000 7fe4d5ffb640 1 -- 172.21.15.43:0/616218939 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0043020 con 0x7fe4d8103180 2024-04-21T08:50:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.068+0000 7fe4d5ffb640 1 -- 172.21.15.43:0/616218939 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4c002dba0 con 0x7fe4d8103180 2024-04-21T08:50:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.068+0000 7fe4d5ffb640 1 -- 172.21.15.43:0/616218939 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0003850 con 0x7fe4d8103180 2024-04-21T08:50:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.068+0000 7fe4df124640 1 -- 172.21.15.43:0/616218939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d819d8b0 con 0x7fe4d8103180 2024-04-21T08:50:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.069+0000 7fe4df124640 1 -- 172.21.15.43:0/616218939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d81a51a0 con 0x7fe4d8103180 2024-04-21T08:50:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.071+0000 7fe4d5ffb640 1 -- 172.21.15.43:0/616218939 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe4c0046020 con 0x7fe4d8103180 2024-04-21T08:50:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.071+0000 7fe4e1df2740 1 Processor -- start 2024-04-21T08:50:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.071+0000 7fe4e1df2740 1 -- start start 2024-04-21T08:50:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.071+0000 7fe4e1df2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd6398d0 0x5630dd638620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.071+0000 7fe4e1df2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630dd638b60 0x5630dd638fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.072+0000 7fe4e1df2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630dd6420e0 0x5630dd6444d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.072+0000 7fe4e1df2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5630dd59d8d0 con 0x5630dd6398d0 2024-04-21T08:50:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.072+0000 7fe4e1df2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5630dd53ae80 con 0x5630dd638b60 2024-04-21T08:50:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.072+0000 7fe4e1df2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5630dd599890 con 0x5630dd6420e0 2024-04-21T08:50:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.072+0000 7fe4dd6c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd6398d0 0x5630dd638620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.072+0000 7fe4dd6c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd6398d0 0x5630dd638620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41866/0 (socket says 172.21.15.43:41866) 2024-04-21T08:50:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.072+0000 7fe4dd6c1640 1 -- 172.21.15.43:0/3493915705 learned_addr learned my addr 172.21.15.43:0/3493915705 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.072+0000 7fe4dcec0640 1 --2- 172.21.15.43:0/3493915705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630dd638b60 0x5630dd638fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.072+0000 7fe4d5ffb640 1 --2- 172.21.15.43:0/616218939 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe4c40a3e40 0x7fe4c40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.072+0000 7fe4d7fff640 1 --2- 172.21.15.43:0/3493915705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630dd6420e0 0x5630dd6444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.072+0000 7fe4dd6c1640 1 -- 172.21.15.43:0/3493915705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630dd6420e0 msgr2=0x5630dd6444d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.072+0000 7fe4dd6c1640 1 --2- 172.21.15.43:0/3493915705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630dd6420e0 0x5630dd6444d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.072+0000 7fe4dd6c1640 1 -- 172.21.15.43:0/3493915705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630dd638b60 msgr2=0x5630dd638fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.072+0000 7fe4dd6c1640 1 --2- 172.21.15.43:0/3493915705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630dd638b60 0x5630dd638fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.072+0000 7fe4dd6c1640 1 -- 172.21.15.43:0/3493915705 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630dd55c320 con 0x5630dd6398d0 2024-04-21T08:50:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.072+0000 7fe4d7fff640 1 --2- 172.21.15.43:0/3493915705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630dd6420e0 0x5630dd6444d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.072+0000 7fe4d5ffb640 1 -- 172.21.15.43:0/616218939 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fe4c00f2ae0 con 0x7fe4d8103180 2024-04-21T08:50:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.072+0000 7fe4dd6c1640 1 --2- 172.21.15.43:0/616218939 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe4c40a3e40 0x7fe4c40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.072+0000 7fe4dd6c1640 1 --2- 172.21.15.43:0/3493915705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd6398d0 0x5630dd638620 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0011590 tx=0x7fe4d0011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc740eb11d14866e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.073+0000 7fe4ceffd640 1 -- 172.21.15.43:0/3493915705 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0017050 con 0x5630dd6398d0 2024-04-21T08:50:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.073+0000 7fe4ceffd640 1 -- 172.21.15.43:0/3493915705 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d001bb90 con 0x5630dd6398d0 2024-04-21T08:50:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.073+0000 7fe4ceffd640 1 -- 172.21.15.43:0/3493915705 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0025d80 con 0x5630dd6398d0 2024-04-21T08:50:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.073+0000 7fe4e1df2740 1 -- 172.21.15.43:0/3493915705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd6398d0 msgr2=0x5630dd638620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.073+0000 7fe4e1df2740 1 --2- 172.21.15.43:0/3493915705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd6398d0 0x5630dd638620 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0011590 tx=0x7fe4d0011a50 comp rx=0 tx=0).stop 2024-04-21T08:50:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.073+0000 7fe4dd6c1640 1 --2- 172.21.15.43:0/616218939 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe4c40a3e40 0x7fe4c40a6300 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0022000 tx=0x7fe4d0023000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.074+0000 7fe4e1df2740 1 -- 172.21.15.43:0/3493915705 shutdown_connections 2024-04-21T08:50:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.074+0000 7fe4e1df2740 1 --2- 172.21.15.43:0/3493915705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630dd6420e0 0x5630dd6444d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.074+0000 7fe4e1df2740 1 --2- 172.21.15.43:0/3493915705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630dd638b60 0x5630dd638fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.074+0000 7fe4e1df2740 1 --2- 172.21.15.43:0/3493915705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd6398d0 0x5630dd638620 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.074+0000 7fe4e1df2740 1 -- 172.21.15.43:0/3493915705 >> 172.21.15.43:0/3493915705 conn(0x5630dd5ffc90 msgr2=0x5630dd633850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.074+0000 7fe4e1df2740 1 -- 172.21.15.43:0/3493915705 shutdown_connections 2024-04-21T08:50:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.074+0000 7fe4e1df2740 1 -- 172.21.15.43:0/3493915705 wait complete. 2024-04-21T08:50:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.074+0000 7fe4e1df2740 1 -- start start 2024-04-21T08:50:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.074+0000 7fe4e1df2740 1 Processor -- start 2024-04-21T08:50:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.074+0000 7fe4e1df2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630dd638b60 0x5630dd641a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.074+0000 7fe4e1df2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630dd6398d0 0x5630dd63edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.074+0000 7fe4e1df2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd6420e0 0x5630dd63f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.074+0000 7fe4e1df2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5630dd644e10 con 0x5630dd6420e0 2024-04-21T08:50:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.074+0000 7fe4e1df2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5630dd525eb0 con 0x5630dd638b60 2024-04-21T08:50:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.074+0000 7fe4e1df2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5630dd644f90 con 0x5630dd6398d0 2024-04-21T08:50:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.074+0000 7fe4e1df2740 10 client.0 _ll_register_callbacks cb 0x5630dd552b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.075+0000 7fe4dd6c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd6420e0 0x5630dd63f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.075+0000 7fe4dd6c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd6420e0 0x5630dd63f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41876/0 (socket says 172.21.15.43:41876) 2024-04-21T08:50:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.075+0000 7fe4dd6c1640 1 -- 172.21.15.43:0/3666266070 learned_addr learned my addr 172.21.15.43:0/3666266070 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.075+0000 7fe4d7fff640 1 --2- 172.21.15.43:0/3666266070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630dd6398d0 0x5630dd63edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.075+0000 7fe4dcec0640 1 --2- 172.21.15.43:0/3666266070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630dd638b60 0x5630dd641a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.075+0000 7fe4dd6c1640 1 -- 172.21.15.43:0/3666266070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630dd6398d0 msgr2=0x5630dd63edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.075+0000 7fe4dd6c1640 1 --2- 172.21.15.43:0/3666266070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630dd6398d0 0x5630dd63edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.075+0000 7fe4dd6c1640 1 -- 172.21.15.43:0/3666266070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630dd638b60 msgr2=0x5630dd641a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.075+0000 7fe4dd6c1640 1 --2- 172.21.15.43:0/3666266070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630dd638b60 0x5630dd641a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.075+0000 7fe4dd6c1640 1 -- 172.21.15.43:0/3666266070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630dd2b0210 con 0x5630dd6420e0 2024-04-21T08:50:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.075+0000 7fe4dd6c1640 1 --2- 172.21.15.43:0/3666266070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd6420e0 0x5630dd63f370 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009c50 tx=0x7fe4d0009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdbf1904c30dec55 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.075+0000 7fe4b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.075+0000 7fe4b7fff640 1 -- 172.21.15.43:0/3666266070 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0031020 con 0x5630dd6420e0 2024-04-21T08:50:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.075+0000 7fe4e1df2740 1 -- 172.21.15.43:0/3666266070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4d001bb90 con 0x5630dd6420e0 2024-04-21T08:50:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.075+0000 7fe4b7fff640 1 -- 172.21.15.43:0/3666266070 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fe4d001b990 con 0x5630dd6420e0 2024-04-21T08:50:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.076+0000 7fe4b7fff640 1 -- 172.21.15.43:0/3666266070 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe4d0017020 con 0x5630dd6420e0 2024-04-21T08:50:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.076+0000 7fe4e1df2740 10 client.21024 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.076+0000 7fe4e1df2740 1 -- 172.21.15.43:0/3666266070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5630dd37c620 con 0x5630dd6420e0 2024-04-21T08:50:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.076+0000 7fe4b7fff640 1 -- 172.21.15.43:0/3666266070 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d001bb90 con 0x5630dd6420e0 2024-04-21T08:50:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.076+0000 7fe4b7fff640 1 -- 172.21.15.43:0/3666266070 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe4d002c070 con 0x5630dd6420e0 2024-04-21T08:50:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.076+0000 7fe4e1df2740 10 client.21024 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.076+0000 7fe4e1df2740 10 client.21024 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.076+0000 7fe4e1df2740 20 client.21024 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.076+0000 7fe4e1df2740 1 --2- 172.21.15.43:0/3666266070 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5630dd652070 0x5630dd654460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.076+0000 7fe4e1df2740 4 client.21024 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.076+0000 7fe4e1df2740 1 -- 172.21.15.43:0/3666266070 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630dd5b1880 con 0x5630dd652070 2024-04-21T08:50:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.076+0000 7fe4dcec0640 1 --2- 172.21.15.43:0/3666266070 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5630dd652070 0x5630dd654460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.077+0000 7fe4dcec0640 1 --2- 172.21.15.43:0/3666266070 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5630dd652070 0x5630dd654460 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbf592daff2658e3 server_cookie=e0ecf85bc79a55cd in_seq=0 out_seq=0 2024-04-21T08:50:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.077+0000 7fe4b7fff640 10 client.21024 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.077+0000 7fe4b7fff640 1 -- 172.21.15.43:0/3666266070 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5630dd5b1880 con 0x5630dd652070 2024-04-21T08:50:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.077+0000 7fe4b7fff640 10 client.21024 handle_command_reply: tid=0 2024-04-21T08:50:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.077+0000 7fe4e1df2740 1 client.21024 shutdown 2024-04-21T08:50:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.078+0000 7fe4e1df2740 1 -- 172.21.15.43:0/3666266070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd6420e0 msgr2=0x5630dd63f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.078+0000 7fe4e1df2740 1 --2- 172.21.15.43:0/3666266070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd6420e0 0x5630dd63f370 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009c50 tx=0x7fe4d0009c80 comp rx=0 tx=0).stop 2024-04-21T08:50:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.078+0000 7fe4e1df2740 1 -- 172.21.15.43:0/3666266070 shutdown_connections 2024-04-21T08:50:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.078+0000 7fe4e1df2740 1 --2- 172.21.15.43:0/3666266070 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5630dd652070 0x5630dd654460 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.078+0000 7fe4e1df2740 1 --2- 172.21.15.43:0/3666266070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd6420e0 0x5630dd63f370 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.078+0000 7fe4e1df2740 1 --2- 172.21.15.43:0/3666266070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630dd6398d0 0x5630dd63edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.078+0000 7fe4b7fff640 0 client.21024 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.078+0000 7fe4e1df2740 1 --2- 172.21.15.43:0/3666266070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630dd638b60 0x5630dd641a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.078+0000 7fe4e1df2740 1 -- 172.21.15.43:0/3666266070 >> 172.21.15.43:0/3666266070 conn(0x5630dd5ffc90 msgr2=0x5630dd6355a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.078+0000 7fe4e1df2740 1 -- 172.21.15.43:0/3666266070 shutdown_connections 2024-04-21T08:50:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.078+0000 7fe4e1df2740 1 -- 172.21.15.43:0/3666266070 wait complete. 2024-04-21T08:50:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.078+0000 7fe4e1df2740 20 client.21024 trim_cache size 0 max 16384 2024-04-21T08:50:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.093+0000 7fe4e1df2740 1 Processor -- start 2024-04-21T08:50:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.093+0000 7fe4e1df2740 1 -- start start 2024-04-21T08:50:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.093+0000 7fe4e1df2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd638b60 0x5630dd64dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.093+0000 7fe4e1df2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630dd6398d0 0x5630dd64a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.093+0000 7fe4e1df2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630dd6420e0 0x5630dd63ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.093+0000 7fe4e1df2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d81af480 con 0x5630dd638b60 2024-04-21T08:50:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.093+0000 7fe4e1df2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d81b0fc0 con 0x5630dd6398d0 2024-04-21T08:50:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.093+0000 7fe4e1df2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d81afb50 con 0x5630dd6420e0 2024-04-21T08:50:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.093+0000 7fe4dcec0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd638b60 0x5630dd64dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.093+0000 7fe4dcec0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd638b60 0x5630dd64dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41882/0 (socket says 172.21.15.43:41882) 2024-04-21T08:50:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.093+0000 7fe4dcec0640 1 -- 172.21.15.43:0/1639409855 learned_addr learned my addr 172.21.15.43:0/1639409855 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.094+0000 7fe4dd6c1640 1 --2- 172.21.15.43:0/1639409855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630dd6420e0 0x5630dd63ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.094+0000 7fe4d7fff640 1 --2- 172.21.15.43:0/1639409855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630dd6398d0 0x5630dd64a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.094+0000 7fe4dcec0640 1 -- 172.21.15.43:0/1639409855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630dd6420e0 msgr2=0x5630dd63ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.094+0000 7fe4dcec0640 1 --2- 172.21.15.43:0/1639409855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630dd6420e0 0x5630dd63ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.094+0000 7fe4dcec0640 1 -- 172.21.15.43:0/1639409855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630dd6398d0 msgr2=0x5630dd64a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.094+0000 7fe4dcec0640 1 --2- 172.21.15.43:0/1639409855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630dd6398d0 0x5630dd64a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.094+0000 7fe4dcec0640 1 -- 172.21.15.43:0/1639409855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630dd62f120 con 0x5630dd638b60 2024-04-21T08:50:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.094+0000 7fe4dcec0640 1 --2- 172.21.15.43:0/1639409855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd638b60 0x5630dd64dd10 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00be6a0 tx=0x7fe4c0089040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5c22af42624569 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.094+0000 7fe4ce7fc640 1 -- 172.21.15.43:0/1639409855 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c00be710 con 0x5630dd638b60 2024-04-21T08:50:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.094+0000 7fe4ce7fc640 1 -- 172.21.15.43:0/1639409855 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4c00c2ac0 con 0x5630dd638b60 2024-04-21T08:50:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.094+0000 7fe4ce7fc640 1 -- 172.21.15.43:0/1639409855 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c00c6ef0 con 0x5630dd638b60 2024-04-21T08:50:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.095+0000 7fe4e1df2740 1 -- 172.21.15.43:0/1639409855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd638b60 msgr2=0x5630dd64dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.095+0000 7fe4e1df2740 1 --2- 172.21.15.43:0/1639409855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd638b60 0x5630dd64dd10 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00be6a0 tx=0x7fe4c0089040 comp rx=0 tx=0).stop 2024-04-21T08:50:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.095+0000 7fe4e1df2740 1 -- 172.21.15.43:0/1639409855 shutdown_connections 2024-04-21T08:50:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.095+0000 7fe4e1df2740 1 --2- 172.21.15.43:0/1639409855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630dd6420e0 0x5630dd63ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.095+0000 7fe4e1df2740 1 --2- 172.21.15.43:0/1639409855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630dd6398d0 0x5630dd64a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.095+0000 7fe4e1df2740 1 --2- 172.21.15.43:0/1639409855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd638b60 0x5630dd64dd10 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.095+0000 7fe4e1df2740 1 -- 172.21.15.43:0/1639409855 >> 172.21.15.43:0/1639409855 conn(0x5630dd5ffc90 msgr2=0x5630dd63d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.095+0000 7fe4e1df2740 1 -- 172.21.15.43:0/1639409855 shutdown_connections 2024-04-21T08:50:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.095+0000 7fe4e1df2740 1 -- 172.21.15.43:0/1639409855 wait complete. 2024-04-21T08:50:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.095+0000 7fe4e1df2740 1 -- start start 2024-04-21T08:50:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.095+0000 7fe4e1df2740 1 Processor -- start 2024-04-21T08:50:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.096+0000 7fe4e1df2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630dd638b60 0x5630dd6b0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.096+0000 7fe4e1df2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630dd6398d0 0x5630dd6b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.096+0000 7fe4e1df2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd6420e0 0x5630dd6b5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.096+0000 7fe4e1df2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d81b08f0 con 0x5630dd6420e0 2024-04-21T08:50:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.096+0000 7fe4e1df2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d81b0220 con 0x5630dd6398d0 2024-04-21T08:50:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.096+0000 7fe4e1df2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d81ad480 con 0x5630dd638b60 2024-04-21T08:50:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.096+0000 7fe4e1df2740 10 client.0 _ll_register_callbacks cb 0x7fe4c00be4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.096+0000 7fe4dd6c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd6420e0 0x5630dd6b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.096+0000 7fe4dd6c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd6420e0 0x5630dd6b5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41896/0 (socket says 172.21.15.43:41896) 2024-04-21T08:50:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.096+0000 7fe4dd6c1640 1 -- 172.21.15.43:0/302654507 learned_addr learned my addr 172.21.15.43:0/302654507 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.096+0000 7fe4d7fff640 1 --2- 172.21.15.43:0/302654507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630dd6398d0 0x5630dd6b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.096+0000 7fe4dcec0640 1 --2- 172.21.15.43:0/302654507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630dd638b60 0x5630dd6b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.096+0000 7fe4dd6c1640 1 -- 172.21.15.43:0/302654507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630dd638b60 msgr2=0x5630dd6b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.096+0000 7fe4dd6c1640 1 --2- 172.21.15.43:0/302654507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630dd638b60 0x5630dd6b0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.096+0000 7fe4dd6c1640 1 -- 172.21.15.43:0/302654507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630dd6398d0 msgr2=0x5630dd6b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.096+0000 7fe4dd6c1640 1 --2- 172.21.15.43:0/302654507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630dd6398d0 0x5630dd6b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.096+0000 7fe4dd6c1640 1 -- 172.21.15.43:0/302654507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630dd6548c0 con 0x5630dd6420e0 2024-04-21T08:50:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.096+0000 7fe4dcec0640 1 --2- 172.21.15.43:0/302654507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630dd638b60 0x5630dd6b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.096+0000 7fe4dd6c1640 1 --2- 172.21.15.43:0/302654507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd6420e0 0x5630dd6b5370 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000eb70 tx=0x7fe4d0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678cfb56935f9cf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.096+0000 7fe4b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.096+0000 7fe4b7fff640 1 -- 172.21.15.43:0/302654507 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0031070 con 0x5630dd6420e0 2024-04-21T08:50:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.096+0000 7fe4b7fff640 1 -- 172.21.15.43:0/302654507 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fe4d0025640 con 0x5630dd6420e0 2024-04-21T08:50:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.096+0000 7fe4e1df2740 1 -- 172.21.15.43:0/302654507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4c00c2ac0 con 0x5630dd6420e0 2024-04-21T08:50:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.097+0000 7fe4b7fff640 1 -- 172.21.15.43:0/302654507 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe4d0025840 con 0x5630dd6420e0 2024-04-21T08:50:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.097+0000 7fe4e1df2740 10 client.21030 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.097+0000 7fe4e1df2740 1 -- 172.21.15.43:0/302654507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5630dd637e30 con 0x5630dd6420e0 2024-04-21T08:50:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.097+0000 7fe4b7fff640 1 -- 172.21.15.43:0/302654507 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4c00c2ac0 con 0x5630dd6420e0 2024-04-21T08:50:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.097+0000 7fe4b7fff640 1 -- 172.21.15.43:0/302654507 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe4d002c040 con 0x5630dd6420e0 2024-04-21T08:50:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.097+0000 7fe4e1df2740 10 client.21030 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.097+0000 7fe4e1df2740 10 client.21030 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.097+0000 7fe4e1df2740 20 client.21030 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.097+0000 7fe4e1df2740 1 --2- 172.21.15.43:0/302654507 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5630dd652070 0x5630dd6bf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.097+0000 7fe4e1df2740 4 client.21030 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:50:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.097+0000 7fe4e1df2740 1 -- 172.21.15.43:0/302654507 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630dd53d3d0 con 0x5630dd652070 2024-04-21T08:50:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.097+0000 7fe4dcec0640 1 --2- 172.21.15.43:0/302654507 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5630dd652070 0x5630dd6bf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.098+0000 7fe4b7fff640 10 client.21030 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.098+0000 7fe4dcec0640 1 --2- 172.21.15.43:0/302654507 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5630dd652070 0x5630dd6bf060 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=487dd82c5a0051ac server_cookie=2449b963b72e29c8 in_seq=0 out_seq=0 2024-04-21T08:50:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.112+0000 7fe4b7fff640 1 -- 172.21.15.43:0/302654507 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5630dd53d3d0 con 0x5630dd652070 2024-04-21T08:50:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.112+0000 7fe4b7fff640 10 client.21030 handle_command_reply: tid=0 2024-04-21T08:50:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.112+0000 7fe4e1df2740 1 client.21030 shutdown 2024-04-21T08:50:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.113+0000 7fe4e1df2740 1 -- 172.21.15.43:0/302654507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd6420e0 msgr2=0x5630dd6b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.113+0000 7fe4e1df2740 1 --2- 172.21.15.43:0/302654507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd6420e0 0x5630dd6b5370 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000eb70 tx=0x7fe4d0009580 comp rx=0 tx=0).stop 2024-04-21T08:50:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.113+0000 7fe4e1df2740 1 -- 172.21.15.43:0/302654507 shutdown_connections 2024-04-21T08:50:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.113+0000 7fe4e1df2740 1 --2- 172.21.15.43:0/302654507 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5630dd652070 0x5630dd6bf060 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.113+0000 7fe4e1df2740 1 --2- 172.21.15.43:0/302654507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630dd6420e0 0x5630dd6b5370 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.113+0000 7fe4b7fff640 0 client.21030 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.113+0000 7fe4e1df2740 1 --2- 172.21.15.43:0/302654507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630dd6398d0 0x5630dd6b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.113+0000 7fe4e1df2740 1 --2- 172.21.15.43:0/302654507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630dd638b60 0x5630dd6b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.113+0000 7fe4e1df2740 1 -- 172.21.15.43:0/302654507 >> 172.21.15.43:0/302654507 conn(0x5630dd5ffc90 msgr2=0x5630dd634be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.113+0000 7fe4e1df2740 1 -- 172.21.15.43:0/302654507 shutdown_connections 2024-04-21T08:50:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.113+0000 7fe4e1df2740 1 -- 172.21.15.43:0/302654507 wait complete. 2024-04-21T08:50:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.113+0000 7fe4e1df2740 20 client.21030 trim_cache size 0 max 16384 2024-04-21T08:50:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:50:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:50:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:50:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:50:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.114+0000 7fe4df124640 1 -- 172.21.15.43:0/616218939 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe4c40a3e40 msgr2=0x7fe4c40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.114+0000 7fe4df124640 1 --2- 172.21.15.43:0/616218939 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe4c40a3e40 0x7fe4c40a6300 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0022000 tx=0x7fe4d0023000 comp rx=0 tx=0).stop 2024-04-21T08:50:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.114+0000 7fe4df124640 1 -- 172.21.15.43:0/616218939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4d8103180 msgr2=0x7fe4d8198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.114+0000 7fe4df124640 1 --2- 172.21.15.43:0/616218939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4d8103180 0x7fe4d8198630 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0005940 tx=0x7fe4c00045e0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.114+0000 7fe4df124640 1 -- 172.21.15.43:0/616218939 shutdown_connections 2024-04-21T08:50:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.114+0000 7fe4df124640 1 --2- 172.21.15.43:0/616218939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4d810c550 0x7fe4d819cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.114+0000 7fe4df124640 1 --2- 172.21.15.43:0/616218939 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe4c40a3e40 0x7fe4c40a6300 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.114+0000 7fe4df124640 1 --2- 172.21.15.43:0/616218939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4d8103aa0 0x7fe4d8198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.114+0000 7fe4df124640 1 --2- 172.21.15.43:0/616218939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4d8103180 0x7fe4d8198630 unknown :-1 s=CLOSED pgs=2233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.114+0000 7fe4df124640 1 -- 172.21.15.43:0/616218939 >> 172.21.15.43:0/616218939 conn(0x7fe4d8077fd0 msgr2=0x7fe4d80fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.114+0000 7fe4df124640 1 -- 172.21.15.43:0/616218939 shutdown_connections 2024-04-21T08:50:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:27.114+0000 7fe4df124640 1 -- 172.21.15.43:0/616218939 wait complete. 2024-04-21T08:50:28.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:28 smithi112 ceph-mon[31085]: pgmap v360: 129 pgs: 129 active+clean; 4.4 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-21T08:50:28.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:28 smithi043 ceph-mon[27978]: pgmap v360: 129 pgs: 129 active+clean; 4.4 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-21T08:50:28.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:28 smithi175 ceph-mon[31920]: pgmap v360: 129 pgs: 129 active+clean; 4.4 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-21T08:50:30.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:30 smithi112 ceph-mon[31085]: pgmap v361: 129 pgs: 129 active+clean; 4.4 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-21T08:50:30.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:30 smithi043 ceph-mon[27978]: pgmap v361: 129 pgs: 129 active+clean; 4.4 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-21T08:50:30.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:30 smithi175 ceph-mon[31920]: pgmap v361: 129 pgs: 129 active+clean; 4.4 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-21T08:50:32.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:32 smithi043 ceph-mon[27978]: pgmap v362: 129 pgs: 129 active+clean; 4.7 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-21T08:50:32.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:32 smithi175 ceph-mon[31920]: pgmap v362: 129 pgs: 129 active+clean; 4.7 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-21T08:50:32.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:32 smithi112 ceph-mon[31085]: pgmap v362: 129 pgs: 129 active+clean; 4.7 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-21T08:50:34.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:34 smithi043 ceph-mon[27978]: pgmap v363: 129 pgs: 129 active+clean; 4.7 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:50:34.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:34 smithi175 ceph-mon[31920]: pgmap v363: 129 pgs: 129 active+clean; 4.7 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:50:34.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:34 smithi112 ceph-mon[31085]: pgmap v363: 129 pgs: 129 active+clean; 4.7 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:50:36.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:36 smithi043 ceph-mon[27978]: pgmap v364: 129 pgs: 129 active+clean; 4.7 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:50:36.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:36 smithi175 ceph-mon[31920]: pgmap v364: 129 pgs: 129 active+clean; 4.7 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:50:36.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:36 smithi112 ceph-mon[31085]: pgmap v364: 129 pgs: 129 active+clean; 4.7 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:50:38.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:38 smithi043 ceph-mon[27978]: pgmap v365: 129 pgs: 129 active+clean; 4.9 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-21T08:50:38.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:38 smithi175 ceph-mon[31920]: pgmap v365: 129 pgs: 129 active+clean; 4.9 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-21T08:50:38.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:38 smithi112 ceph-mon[31085]: pgmap v365: 129 pgs: 129 active+clean; 4.9 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-21T08:50:39.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:50:39.537 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:50:39.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.657+0000 7f307a406640 1 -- 172.21.15.43:0/527739317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3074100d40 msgr2=0x7f3074101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.657+0000 7f307a406640 1 --2- 172.21.15.43:0/527739317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3074100d40 0x7f3074101120 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f3064009900 tx=0x7f306402bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.657+0000 7f307a406640 1 -- 172.21.15.43:0/527739317 shutdown_connections 2024-04-21T08:50:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.657+0000 7f307a406640 1 --2- 172.21.15.43:0/527739317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f307410e8f0 0x7f3074110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.657+0000 7f307a406640 1 --2- 172.21.15.43:0/527739317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3074101660 0x7f307410e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.657+0000 7f307a406640 1 --2- 172.21.15.43:0/527739317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3074100d40 0x7f3074101120 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.657+0000 7f307a406640 1 -- 172.21.15.43:0/527739317 >> 172.21.15.43:0/527739317 conn(0x7f3074078000 msgr2=0x7f3074078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.657+0000 7f307a406640 1 -- 172.21.15.43:0/527739317 shutdown_connections 2024-04-21T08:50:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.657+0000 7f307a406640 1 -- 172.21.15.43:0/527739317 wait complete. 2024-04-21T08:50:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.657+0000 7f307a406640 1 Processor -- start 2024-04-21T08:50:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.657+0000 7f307a406640 1 -- start start 2024-04-21T08:50:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.658+0000 7f307a406640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3074100d40 0x7f30741a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.658+0000 7f307a406640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3074101660 0x7f30741a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.658+0000 7f307a406640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f307410e8f0 0x7f307419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.658+0000 7f307a406640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3074114420 con 0x7f307410e8f0 2024-04-21T08:50:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.658+0000 7f307a406640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f30741142a0 con 0x7f3074100d40 2024-04-21T08:50:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.658+0000 7f307a406640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f30741145a0 con 0x7f3074101660 2024-04-21T08:50:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.658+0000 7f30789a3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f307410e8f0 0x7f307419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.658+0000 7f30737fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3074101660 0x7f30741a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.658+0000 7f30789a3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f307410e8f0 0x7f307419a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49782/0 (socket says 172.21.15.43:49782) 2024-04-21T08:50:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.658+0000 7f30789a3640 1 -- 172.21.15.43:0/3133410135 learned_addr learned my addr 172.21.15.43:0/3133410135 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.658+0000 7f3073fff640 1 --2- 172.21.15.43:0/3133410135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3074100d40 0x7f30741a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.658+0000 7f30789a3640 1 -- 172.21.15.43:0/3133410135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3074101660 msgr2=0x7f30741a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.658+0000 7f30789a3640 1 --2- 172.21.15.43:0/3133410135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3074101660 0x7f30741a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.658+0000 7f30789a3640 1 -- 172.21.15.43:0/3133410135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3074100d40 msgr2=0x7f30741a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.658+0000 7f30789a3640 1 --2- 172.21.15.43:0/3133410135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3074100d40 0x7f30741a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.658+0000 7f30789a3640 1 -- 172.21.15.43:0/3133410135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307419af50 con 0x7f307410e8f0 2024-04-21T08:50:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.658+0000 7f30789a3640 1 --2- 172.21.15.43:0/3133410135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f307410e8f0 0x7f307419a780 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f306c00cad0 tx=0x7f306c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.659+0000 7f30717fa640 1 -- 172.21.15.43:0/3133410135 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c00ad60 con 0x7f307410e8f0 2024-04-21T08:50:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.659+0000 7f30717fa640 1 -- 172.21.15.43:0/3133410135 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f306c003e40 con 0x7f307410e8f0 2024-04-21T08:50:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.659+0000 7f307a406640 1 -- 172.21.15.43:0/3133410135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f307419b240 con 0x7f307410e8f0 2024-04-21T08:50:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.659+0000 7f30717fa640 1 -- 172.21.15.43:0/3133410135 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c00f6b0 con 0x7f307410e8f0 2024-04-21T08:50:39.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.659+0000 7f307a406640 1 -- 172.21.15.43:0/3133410135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30741a7460 con 0x7f307410e8f0 2024-04-21T08:50:39.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.661+0000 7f30717fa640 1 -- 172.21.15.43:0/3133410135 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f306c003990 con 0x7f307410e8f0 2024-04-21T08:50:39.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.661+0000 7f307cff4740 1 Processor -- start 2024-04-21T08:50:39.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.661+0000 7f307cff4740 1 -- start start 2024-04-21T08:50:39.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.662+0000 7f307cff4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebbca6a580 0x55ebbca6a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:39.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.662+0000 7f307cff4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbca6aea0 0x55ebbca73540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:39.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.662+0000 7f307cff4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca73a80 0x55ebbca75e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:39.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.662+0000 7f307cff4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ebbc9cf1e0 con 0x55ebbca73a80 2024-04-21T08:50:39.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.662+0000 7f307cff4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ebbc96c390 con 0x55ebbca6a580 2024-04-21T08:50:39.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.662+0000 7f307cff4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ebbc9cb1a0 con 0x55ebbca6aea0 2024-04-21T08:50:39.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.662+0000 7f30737fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca73a80 0x55ebbca75e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:39.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.662+0000 7f30717fa640 1 --2- 172.21.15.43:0/3133410135 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f305c0a3d20 0x7f305c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:39.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.662+0000 7f30737fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca73a80 0x55ebbca75e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49792/0 (socket says 172.21.15.43:49792) 2024-04-21T08:50:39.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.662+0000 7f30737fe640 1 -- 172.21.15.43:0/3978046602 learned_addr learned my addr 172.21.15.43:0/3978046602 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:39.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.662+0000 7f3073fff640 1 --2- 172.21.15.43:0/3978046602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbca6aea0 0x55ebbca73540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:39.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.662+0000 7f30737fe640 1 --2- 172.21.15.43:0/3978046602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebbca6a580 0x55ebbca6a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:39.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.662+0000 7f30789a3640 1 --2- 172.21.15.43:0/3133410135 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f305c0a3d20 0x7f305c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:39.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.662+0000 7f30737fe640 1 -- 172.21.15.43:0/3978046602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbca6aea0 msgr2=0x55ebbca73540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:39.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.662+0000 7f30737fe640 1 --2- 172.21.15.43:0/3978046602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbca6aea0 0x55ebbca73540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.662+0000 7f30737fe640 1 -- 172.21.15.43:0/3978046602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebbca6a580 msgr2=0x55ebbca6a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:39.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.662+0000 7f30737fe640 1 --2- 172.21.15.43:0/3978046602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebbca6a580 0x55ebbca6a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.662+0000 7f30737fe640 1 -- 172.21.15.43:0/3978046602 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebbc7ab7d0 con 0x55ebbca73a80 2024-04-21T08:50:39.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.662+0000 7f30717fa640 1 -- 172.21.15.43:0/3133410135 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f306c0d0ba0 con 0x7f307410e8f0 2024-04-21T08:50:39.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.662+0000 7f30737fe640 1 --2- 172.21.15.43:0/3978046602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca73a80 0x55ebbca75e70 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f3068009fa0 tx=0x7f3068009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be4a5a370dce6cd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:39.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.663+0000 7f30789a3640 1 --2- 172.21.15.43:0/3133410135 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f305c0a3d20 0x7f305c0a61e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f306c09c6b0 tx=0x7f306c05b040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:39.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.663+0000 7f305a7fc640 1 -- 172.21.15.43:0/3978046602 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068017470 con 0x55ebbca73a80 2024-04-21T08:50:39.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.663+0000 7f305a7fc640 1 -- 172.21.15.43:0/3978046602 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f306801c070 con 0x55ebbca73a80 2024-04-21T08:50:39.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.663+0000 7f305a7fc640 1 -- 172.21.15.43:0/3978046602 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30680158b0 con 0x55ebbca73a80 2024-04-21T08:50:39.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.663+0000 7f307cff4740 1 -- 172.21.15.43:0/3978046602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca73a80 msgr2=0x55ebbca75e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:39.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.663+0000 7f307cff4740 1 --2- 172.21.15.43:0/3978046602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca73a80 0x55ebbca75e70 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f3068009fa0 tx=0x7f3068009580 comp rx=0 tx=0).stop 2024-04-21T08:50:39.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.663+0000 7f307cff4740 1 -- 172.21.15.43:0/3978046602 shutdown_connections 2024-04-21T08:50:39.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.663+0000 7f307cff4740 1 --2- 172.21.15.43:0/3978046602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca73a80 0x55ebbca75e70 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.663+0000 7f307cff4740 1 --2- 172.21.15.43:0/3978046602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbca6aea0 0x55ebbca73540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.663+0000 7f307cff4740 1 --2- 172.21.15.43:0/3978046602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebbca6a580 0x55ebbca6a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.663+0000 7f307cff4740 1 -- 172.21.15.43:0/3978046602 >> 172.21.15.43:0/3978046602 conn(0x55ebbca64cb0 msgr2=0x55ebbca65090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:39.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.663+0000 7f307cff4740 1 -- 172.21.15.43:0/3978046602 shutdown_connections 2024-04-21T08:50:39.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.663+0000 7f307cff4740 1 -- 172.21.15.43:0/3978046602 wait complete. 2024-04-21T08:50:39.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.664+0000 7f307cff4740 1 -- start start 2024-04-21T08:50:39.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.664+0000 7f307cff4740 1 Processor -- start 2024-04-21T08:50:39.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.664+0000 7f307cff4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebbca6a580 0x55ebbca70aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:39.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.664+0000 7f307cff4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca6aea0 0x55ebbca70fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:39.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.664+0000 7f307cff4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbca73a80 0x55ebbca71520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:39.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.664+0000 7f307cff4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ebbca76870 con 0x55ebbca6aea0 2024-04-21T08:50:39.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.664+0000 7f307cff4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ebbc9882f0 con 0x55ebbca6a580 2024-04-21T08:50:39.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.664+0000 7f307cff4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ebbca769f0 con 0x55ebbca73a80 2024-04-21T08:50:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.664+0000 7f307cff4740 10 client.0 _ll_register_callbacks cb 0x55ebbc995580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.664+0000 7f3073fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca6aea0 0x55ebbca70fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.664+0000 7f3073fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca6aea0 0x55ebbca70fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49794/0 (socket says 172.21.15.43:49794) 2024-04-21T08:50:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.664+0000 7f3073fff640 1 -- 172.21.15.43:0/1535528935 learned_addr learned my addr 172.21.15.43:0/1535528935 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.664+0000 7f30737fe640 1 --2- 172.21.15.43:0/1535528935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbca73a80 0x55ebbca71520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.665+0000 7f30737fe640 1 --2- 172.21.15.43:0/1535528935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebbca6a580 0x55ebbca70aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.665+0000 7f3073fff640 1 -- 172.21.15.43:0/1535528935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbca73a80 msgr2=0x55ebbca71520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.665+0000 7f3073fff640 1 --2- 172.21.15.43:0/1535528935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbca73a80 0x55ebbca71520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.665+0000 7f3073fff640 1 -- 172.21.15.43:0/1535528935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebbca6a580 msgr2=0x55ebbca70aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.665+0000 7f3073fff640 1 --2- 172.21.15.43:0/1535528935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebbca6a580 0x55ebbca70aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.665+0000 7f3073fff640 1 -- 172.21.15.43:0/1535528935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebbc6e13f0 con 0x55ebbca6aea0 2024-04-21T08:50:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.665+0000 7f30737fe640 1 --2- 172.21.15.43:0/1535528935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebbca6a580 0x55ebbca70aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.665+0000 7f3073fff640 1 --2- 172.21.15.43:0/1535528935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca6aea0 0x55ebbca70fe0 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f3064009340 tx=0x7f3064004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58e0461874ddfb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.665+0000 7f30537fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.665+0000 7f30537fe640 1 -- 172.21.15.43:0/1535528935 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064005030 con 0x55ebbca6aea0 2024-04-21T08:50:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.665+0000 7f30537fe640 1 -- 172.21.15.43:0/1535528935 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f30640051d0 con 0x55ebbca6aea0 2024-04-21T08:50:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.665+0000 7f307cff4740 1 -- 172.21.15.43:0/1535528935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f306801c070 con 0x55ebbca6aea0 2024-04-21T08:50:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.665+0000 7f30537fe640 1 -- 172.21.15.43:0/1535528935 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f30640053d0 con 0x55ebbca6aea0 2024-04-21T08:50:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.665+0000 7f307cff4740 10 client.21042 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.665+0000 7f307cff4740 1 -- 172.21.15.43:0/1535528935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ebbc6a10f0 con 0x55ebbca6aea0 2024-04-21T08:50:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.666+0000 7f30537fe640 1 -- 172.21.15.43:0/1535528935 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f306801c070 con 0x55ebbca6aea0 2024-04-21T08:50:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.666+0000 7f30537fe640 1 -- 172.21.15.43:0/1535528935 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f30640038a0 con 0x55ebbca6aea0 2024-04-21T08:50:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.666+0000 7f307cff4740 10 client.21042 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.666+0000 7f307cff4740 10 client.21042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.666+0000 7f307cff4740 20 client.21042 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.666+0000 7f307cff4740 1 --2- 172.21.15.43:0/1535528935 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ebbca83a50 0x55ebbca85e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.666+0000 7f307cff4740 4 client.21042 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.666+0000 7f307cff4740 1 -- 172.21.15.43:0/1535528935 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebbc9dcc00 con 0x55ebbca83a50 2024-04-21T08:50:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.666+0000 7f30737fe640 1 --2- 172.21.15.43:0/1535528935 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ebbca83a50 0x55ebbca85e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.666+0000 7f30737fe640 1 --2- 172.21.15.43:0/1535528935 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ebbca83a50 0x55ebbca85e80 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e0a2bf92372258f server_cookie=9d50899eafb691df in_seq=0 out_seq=0 2024-04-21T08:50:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.666+0000 7f30537fe640 10 client.21042 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.667+0000 7f30537fe640 1 -- 172.21.15.43:0/1535528935 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ebbc9dcc00 con 0x55ebbca83a50 2024-04-21T08:50:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.667+0000 7f30537fe640 10 client.21042 handle_command_reply: tid=0 2024-04-21T08:50:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.667+0000 7f307cff4740 1 client.21042 shutdown 2024-04-21T08:50:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.667+0000 7f307cff4740 1 -- 172.21.15.43:0/1535528935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca6aea0 msgr2=0x55ebbca70fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.667+0000 7f307cff4740 1 --2- 172.21.15.43:0/1535528935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca6aea0 0x55ebbca70fe0 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f3064009340 tx=0x7f3064004270 comp rx=0 tx=0).stop 2024-04-21T08:50:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.668+0000 7f307cff4740 1 -- 172.21.15.43:0/1535528935 shutdown_connections 2024-04-21T08:50:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.668+0000 7f307cff4740 1 --2- 172.21.15.43:0/1535528935 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ebbca83a50 0x55ebbca85e80 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.668+0000 7f307cff4740 1 --2- 172.21.15.43:0/1535528935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbca73a80 0x55ebbca71520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.668+0000 7f307cff4740 1 --2- 172.21.15.43:0/1535528935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca6aea0 0x55ebbca70fe0 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.668+0000 7f30537fe640 0 client.21042 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.668+0000 7f307cff4740 1 --2- 172.21.15.43:0/1535528935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebbca6a580 0x55ebbca70aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.668+0000 7f307cff4740 1 -- 172.21.15.43:0/1535528935 >> 172.21.15.43:0/1535528935 conn(0x55ebbca64cb0 msgr2=0x55ebbca66a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.668+0000 7f307cff4740 1 -- 172.21.15.43:0/1535528935 shutdown_connections 2024-04-21T08:50:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.668+0000 7f307cff4740 1 -- 172.21.15.43:0/1535528935 wait complete. 2024-04-21T08:50:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.668+0000 7f307cff4740 20 client.21042 trim_cache size 0 max 16384 2024-04-21T08:50:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.685+0000 7f307cff4740 1 Processor -- start 2024-04-21T08:50:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.685+0000 7f307cff4740 1 -- start start 2024-04-21T08:50:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.686+0000 7f307cff4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebbca6a580 0x55ebbca731d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.686+0000 7f307cff4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca6aea0 0x55ebbca7f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.686+0000 7f307cff4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbca73a80 0x55ebbca63bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.686+0000 7f307cff4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f30741b17e0 con 0x55ebbca6aea0 2024-04-21T08:50:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.686+0000 7f307cff4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f30741b3320 con 0x55ebbca6a580 2024-04-21T08:50:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.686+0000 7f307cff4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f30741b1eb0 con 0x55ebbca73a80 2024-04-21T08:50:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.686+0000 7f3073fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca6aea0 0x55ebbca7f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.686+0000 7f3073fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca6aea0 0x55ebbca7f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49798/0 (socket says 172.21.15.43:49798) 2024-04-21T08:50:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.686+0000 7f3073fff640 1 -- 172.21.15.43:0/146240732 learned_addr learned my addr 172.21.15.43:0/146240732 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.686+0000 7f30737fe640 1 --2- 172.21.15.43:0/146240732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbca73a80 0x55ebbca63bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.686+0000 7f30737fe640 1 --2- 172.21.15.43:0/146240732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebbca6a580 0x55ebbca731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.686+0000 7f3073fff640 1 -- 172.21.15.43:0/146240732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbca73a80 msgr2=0x55ebbca63bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.686+0000 7f3073fff640 1 --2- 172.21.15.43:0/146240732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbca73a80 0x55ebbca63bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.686+0000 7f3073fff640 1 -- 172.21.15.43:0/146240732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebbca6a580 msgr2=0x55ebbca731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.686+0000 7f3073fff640 1 --2- 172.21.15.43:0/146240732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebbca6a580 0x55ebbca731d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.686+0000 7f3073fff640 1 -- 172.21.15.43:0/146240732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebbca647e0 con 0x55ebbca6aea0 2024-04-21T08:50:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.686+0000 7f30737fe640 1 --2- 172.21.15.43:0/146240732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebbca6a580 0x55ebbca731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.686+0000 7f3073fff640 1 --2- 172.21.15.43:0/146240732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca6aea0 0x55ebbca7f730 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f306402c170 tx=0x7f306402c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0d141fe1169eea server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.686+0000 7f3059ffb640 1 -- 172.21.15.43:0/146240732 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064005850 con 0x55ebbca6aea0 2024-04-21T08:50:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.686+0000 7f3059ffb640 1 -- 172.21.15.43:0/146240732 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3064034440 con 0x55ebbca6aea0 2024-04-21T08:50:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.686+0000 7f3059ffb640 1 -- 172.21.15.43:0/146240732 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064034e10 con 0x55ebbca6aea0 2024-04-21T08:50:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.687+0000 7f307cff4740 1 -- 172.21.15.43:0/146240732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca6aea0 msgr2=0x55ebbca7f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.687+0000 7f307cff4740 1 --2- 172.21.15.43:0/146240732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca6aea0 0x55ebbca7f730 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f306402c170 tx=0x7f306402c630 comp rx=0 tx=0).stop 2024-04-21T08:50:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.687+0000 7f307cff4740 1 -- 172.21.15.43:0/146240732 shutdown_connections 2024-04-21T08:50:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.687+0000 7f307cff4740 1 --2- 172.21.15.43:0/146240732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbca73a80 0x55ebbca63bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.687+0000 7f307cff4740 1 --2- 172.21.15.43:0/146240732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca6aea0 0x55ebbca7f730 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.687+0000 7f307cff4740 1 --2- 172.21.15.43:0/146240732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebbca6a580 0x55ebbca731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.687+0000 7f307cff4740 1 -- 172.21.15.43:0/146240732 >> 172.21.15.43:0/146240732 conn(0x55ebbca64cb0 msgr2=0x55ebbca634e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.687+0000 7f307cff4740 1 -- 172.21.15.43:0/146240732 shutdown_connections 2024-04-21T08:50:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.687+0000 7f307cff4740 1 -- 172.21.15.43:0/146240732 wait complete. 2024-04-21T08:50:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.687+0000 7f307cff4740 1 -- start start 2024-04-21T08:50:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.688+0000 7f307cff4740 1 Processor -- start 2024-04-21T08:50:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.688+0000 7f307cff4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca6a580 0x55ebbca7c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.688+0000 7f307cff4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbca6aea0 0x55ebbca6d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.688+0000 7f307cff4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebbca73a80 0x55ebbca6dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.688+0000 7f307cff4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f30741b2c50 con 0x55ebbca6a580 2024-04-21T08:50:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.688+0000 7f307cff4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f30741b2580 con 0x55ebbca73a80 2024-04-21T08:50:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.688+0000 7f307cff4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f30741af7e0 con 0x55ebbca6aea0 2024-04-21T08:50:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.688+0000 7f307cff4740 10 client.0 _ll_register_callbacks cb 0x7f306402cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.688+0000 7f30737fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca6a580 0x55ebbca7c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.688+0000 7f30737fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca6a580 0x55ebbca7c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49814/0 (socket says 172.21.15.43:49814) 2024-04-21T08:50:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.688+0000 7f30737fe640 1 -- 172.21.15.43:0/1428704099 learned_addr learned my addr 172.21.15.43:0/1428704099 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.688+0000 7f30737fe640 1 --2- 172.21.15.43:0/1428704099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebbca73a80 0x55ebbca6dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.688+0000 7f3073fff640 1 --2- 172.21.15.43:0/1428704099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbca6aea0 0x55ebbca6d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.688+0000 7f30737fe640 1 -- 172.21.15.43:0/1428704099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbca6aea0 msgr2=0x55ebbca6d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.688+0000 7f30737fe640 1 --2- 172.21.15.43:0/1428704099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbca6aea0 0x55ebbca6d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.688+0000 7f30737fe640 1 -- 172.21.15.43:0/1428704099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebbca73a80 msgr2=0x55ebbca6dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.688+0000 7f30737fe640 1 --2- 172.21.15.43:0/1428704099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebbca73a80 0x55ebbca6dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.688+0000 7f30737fe640 1 -- 172.21.15.43:0/1428704099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebbcae8fc0 con 0x55ebbca6a580 2024-04-21T08:50:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.688+0000 7f30737fe640 1 --2- 172.21.15.43:0/1428704099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca6a580 0x55ebbca7c180 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f3068009fa0 tx=0x7f3068026040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cda38d2822a90c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.688+0000 7f30537fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.689+0000 7f30537fe640 1 -- 172.21.15.43:0/1428704099 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068034030 con 0x55ebbca6a580 2024-04-21T08:50:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.689+0000 7f30537fe640 1 -- 172.21.15.43:0/1428704099 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f306802ede0 con 0x55ebbca6a580 2024-04-21T08:50:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.689+0000 7f307cff4740 1 -- 172.21.15.43:0/1428704099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ebbc9d6a70 con 0x55ebbca6a580 2024-04-21T08:50:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.689+0000 7f30537fe640 1 -- 172.21.15.43:0/1428704099 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f306803d050 con 0x55ebbca6a580 2024-04-21T08:50:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.689+0000 7f307cff4740 10 client.21048 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.689+0000 7f307cff4740 1 -- 172.21.15.43:0/1428704099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ebbcaea0e0 con 0x55ebbca6a580 2024-04-21T08:50:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.689+0000 7f30537fe640 1 -- 172.21.15.43:0/1428704099 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ebbc9d6a70 con 0x55ebbca6a580 2024-04-21T08:50:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.689+0000 7f30537fe640 1 -- 172.21.15.43:0/1428704099 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3068017550 con 0x55ebbca6a580 2024-04-21T08:50:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.689+0000 7f307cff4740 10 client.21048 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.689+0000 7f307cff4740 10 client.21048 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.689+0000 7f307cff4740 20 client.21048 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.689+0000 7f307cff4740 1 --2- 172.21.15.43:0/1428704099 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ebbcaea530 0x55ebbcaec950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.689+0000 7f307cff4740 4 client.21048 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:50:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.689+0000 7f307cff4740 1 -- 172.21.15.43:0/1428704099 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ebbc952590 con 0x55ebbcaea530 2024-04-21T08:50:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.690+0000 7f3073fff640 1 --2- 172.21.15.43:0/1428704099 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ebbcaea530 0x55ebbcaec950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.690+0000 7f3073fff640 1 --2- 172.21.15.43:0/1428704099 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ebbcaea530 0x55ebbcaec950 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af6b074df9afc7ac server_cookie=bfc57d877e557191 in_seq=0 out_seq=0 2024-04-21T08:50:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.690+0000 7f30537fe640 10 client.21048 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.690+0000 7f30537fe640 1 -- 172.21.15.43:0/1428704099 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ebbc952590 con 0x55ebbcaea530 2024-04-21T08:50:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.690+0000 7f30537fe640 10 client.21048 handle_command_reply: tid=0 2024-04-21T08:50:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.690+0000 7f307cff4740 1 client.21048 shutdown 2024-04-21T08:50:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.690+0000 7f307cff4740 1 -- 172.21.15.43:0/1428704099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca6a580 msgr2=0x55ebbca7c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.690+0000 7f307cff4740 1 --2- 172.21.15.43:0/1428704099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca6a580 0x55ebbca7c180 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f3068009fa0 tx=0x7f3068026040 comp rx=0 tx=0).stop 2024-04-21T08:50:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.690+0000 7f307cff4740 1 -- 172.21.15.43:0/1428704099 shutdown_connections 2024-04-21T08:50:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.690+0000 7f307cff4740 1 --2- 172.21.15.43:0/1428704099 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55ebbcaea530 0x55ebbcaec950 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.690+0000 7f307cff4740 1 --2- 172.21.15.43:0/1428704099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebbca73a80 0x55ebbca6dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.690+0000 7f307cff4740 1 --2- 172.21.15.43:0/1428704099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ebbca6aea0 0x55ebbca6d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.690+0000 7f30537fe640 0 client.21048 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.690+0000 7f307cff4740 1 --2- 172.21.15.43:0/1428704099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ebbca6a580 0x55ebbca7c180 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.690+0000 7f307cff4740 1 -- 172.21.15.43:0/1428704099 >> 172.21.15.43:0/1428704099 conn(0x55ebbca64cb0 msgr2=0x55ebbca712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.690+0000 7f307cff4740 1 -- 172.21.15.43:0/1428704099 shutdown_connections 2024-04-21T08:50:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.691+0000 7f307cff4740 1 -- 172.21.15.43:0/1428704099 wait complete. 2024-04-21T08:50:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.691+0000 7f307cff4740 20 client.21048 trim_cache size 0 max 16384 2024-04-21T08:50:39.698 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:50:39.698 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:50:39.698 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:50:39.698 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:50:39.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.691+0000 7f307a406640 1 -- 172.21.15.43:0/3133410135 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f305c0a3d20 msgr2=0x7f305c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:39.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.691+0000 7f307a406640 1 --2- 172.21.15.43:0/3133410135 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f305c0a3d20 0x7f305c0a61e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f306c09c6b0 tx=0x7f306c05b040 comp rx=0 tx=0).stop 2024-04-21T08:50:39.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.691+0000 7f307a406640 1 -- 172.21.15.43:0/3133410135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f307410e8f0 msgr2=0x7f307419a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:39.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.691+0000 7f307a406640 1 --2- 172.21.15.43:0/3133410135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f307410e8f0 0x7f307419a780 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f306c00cad0 tx=0x7f306c00ced0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.691+0000 7f307a406640 1 -- 172.21.15.43:0/3133410135 shutdown_connections 2024-04-21T08:50:39.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.691+0000 7f307a406640 1 --2- 172.21.15.43:0/3133410135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f307410e8f0 0x7f307419a780 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.691+0000 7f307a406640 1 --2- 172.21.15.43:0/3133410135 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f305c0a3d20 0x7f305c0a61e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.691+0000 7f307a406640 1 --2- 172.21.15.43:0/3133410135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3074101660 0x7f30741a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.691+0000 7f307a406640 1 --2- 172.21.15.43:0/3133410135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3074100d40 0x7f30741a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.691+0000 7f307a406640 1 -- 172.21.15.43:0/3133410135 >> 172.21.15.43:0/3133410135 conn(0x7f3074078000 msgr2=0x7f307410ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:39.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.691+0000 7f307a406640 1 -- 172.21.15.43:0/3133410135 shutdown_connections 2024-04-21T08:50:39.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.691+0000 7f307a406640 1 -- 172.21.15.43:0/3133410135 wait complete. 2024-04-21T08:50:39.703 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:50:39.703 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:50:39.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.805+0000 7f60b366d640 1 -- 172.21.15.43:0/3045403537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ac1021a0 msgr2=0x7f60ac102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:39.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.805+0000 7f60b366d640 1 --2- 172.21.15.43:0/3045403537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ac1021a0 0x7f60ac102580 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f609c009900 tx=0x7f609c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:39.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.806+0000 7f60b366d640 1 -- 172.21.15.43:0/3045403537 shutdown_connections 2024-04-21T08:50:39.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.806+0000 7f60b366d640 1 --2- 172.21.15.43:0/3045403537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac107270 0x7f60ac10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.806+0000 7f60b366d640 1 --2- 172.21.15.43:0/3045403537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60ac102b50 0x7f60ac106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.806+0000 7f60b366d640 1 --2- 172.21.15.43:0/3045403537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ac1021a0 0x7f60ac102580 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.806+0000 7f60b366d640 1 -- 172.21.15.43:0/3045403537 >> 172.21.15.43:0/3045403537 conn(0x7f60ac0793a0 msgr2=0x7f60ac0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:39.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.806+0000 7f60b366d640 1 -- 172.21.15.43:0/3045403537 shutdown_connections 2024-04-21T08:50:39.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.806+0000 7f60b366d640 1 -- 172.21.15.43:0/3045403537 wait complete. 2024-04-21T08:50:39.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.806+0000 7f60b366d640 1 Processor -- start 2024-04-21T08:50:39.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.806+0000 7f60b366d640 1 -- start start 2024-04-21T08:50:39.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.806+0000 7f60b366d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60ac1021a0 0x7f60ac198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:39.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.806+0000 7f60b366d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ac107270 0x7f60ac198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:39.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.806+0000 7f60b366d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac19d000 0x7f60ac19d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:39.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.806+0000 7f60b366d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac112220 con 0x7f60ac107270 2024-04-21T08:50:39.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.806+0000 7f60b366d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac1120a0 con 0x7f60ac19d000 2024-04-21T08:50:39.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.806+0000 7f60b366d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac1123a0 con 0x7f60ac1021a0 2024-04-21T08:50:39.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.807+0000 7f60b0c08640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ac107270 0x7f60ac198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:39.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.807+0000 7f60b0c08640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ac107270 0x7f60ac198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49846/0 (socket says 172.21.15.43:49846) 2024-04-21T08:50:39.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.807+0000 7f60b0c08640 1 -- 172.21.15.43:0/3123942233 learned_addr learned my addr 172.21.15.43:0/3123942233 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:39.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.807+0000 7f60b1409640 1 --2- 172.21.15.43:0/3123942233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60ac1021a0 0x7f60ac198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:39.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.807+0000 7f60b0c08640 1 -- 172.21.15.43:0/3123942233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60ac1021a0 msgr2=0x7f60ac198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:39.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.807+0000 7f60b0c08640 1 --2- 172.21.15.43:0/3123942233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60ac1021a0 0x7f60ac198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.807+0000 7f60b0c08640 1 -- 172.21.15.43:0/3123942233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac19d000 msgr2=0x7f60ac19d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:39.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.807+0000 7f60b0c08640 1 --2- 172.21.15.43:0/3123942233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac19d000 0x7f60ac19d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:39.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.807+0000 7f60b0c08640 1 -- 172.21.15.43:0/3123942233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ac19daf0 con 0x7f60ac107270 2024-04-21T08:50:39.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.807+0000 7f60b1409640 1 --2- 172.21.15.43:0/3123942233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60ac1021a0 0x7f60ac198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:39.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.807+0000 7f60b0c08640 1 --2- 172.21.15.43:0/3123942233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ac107270 0x7f60ac198c80 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f60a800e9f0 tx=0x7f60a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:39.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.807+0000 7f60a27fc640 1 -- 172.21.15.43:0/3123942233 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a800ccc0 con 0x7f60ac107270 2024-04-21T08:50:39.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.807+0000 7f60a27fc640 1 -- 172.21.15.43:0/3123942233 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60a80054d0 con 0x7f60ac107270 2024-04-21T08:50:39.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.807+0000 7f60b366d640 1 -- 172.21.15.43:0/3123942233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ac19dde0 con 0x7f60ac107270 2024-04-21T08:50:39.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.807+0000 7f60a27fc640 1 -- 172.21.15.43:0/3123942233 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8010430 con 0x7f60ac107270 2024-04-21T08:50:39.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.807+0000 7f60b366d640 1 -- 172.21.15.43:0/3123942233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60ac1a5580 con 0x7f60ac107270 2024-04-21T08:50:39.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.808+0000 7f60b366d640 1 -- 172.21.15.43:0/3123942233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60ac069230 con 0x7f60ac107270 2024-04-21T08:50:39.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.809+0000 7f60a27fc640 1 -- 172.21.15.43:0/3123942233 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f60a80046c0 con 0x7f60ac107270 2024-04-21T08:50:39.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.810+0000 7f60a27fc640 1 --2- 172.21.15.43:0/3123942233 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f60980a3d20 0x7f60980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:39.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.810+0000 7f60a27fc640 1 -- 172.21.15.43:0/3123942233 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f60a80d12d0 con 0x7f60ac107270 2024-04-21T08:50:39.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.810+0000 7f60b1409640 1 --2- 172.21.15.43:0/3123942233 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f60980a3d20 0x7f60980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:39.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.810+0000 7f60b1409640 1 --2- 172.21.15.43:0/3123942233 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f60980a3d20 0x7f60980a61e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f609c036040 tx=0x7f609c005fd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:39.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:39.814+0000 7f60a27fc640 1 -- 172.21.15.43:0/3123942233 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60a8053ff0 con 0x7f60ac107270 2024-04-21T08:50:40.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.043+0000 7f60b366d640 1 -- 172.21.15.43:0/3123942233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60ac199a90 con 0x7f60ac107270 2024-04-21T08:50:40.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.044+0000 7f60a27fc640 1 -- 172.21.15.43:0/3123942233 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11935 (secure 0 0 0) 0x7f60ac199a90 con 0x7f60ac107270 2024-04-21T08:50:40.044 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:50:40.044 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:46:17.283806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27556":{"gid":27556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/229985237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":229985237},{"type":"v1","addr":"172.21.15.175:6837","nonce":229985237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24425,24412,24487,24527,24491]},"id":1}]} 2024-04-21T08:50:40.045 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-04-21T08:50:40.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.045+0000 7f60b366d640 1 -- 172.21.15.43:0/3123942233 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f60980a3d20 msgr2=0x7f60980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.046+0000 7f60b366d640 1 --2- 172.21.15.43:0/3123942233 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f60980a3d20 0x7f60980a61e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f609c036040 tx=0x7f609c005fd0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.046+0000 7f60b366d640 1 -- 172.21.15.43:0/3123942233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ac107270 msgr2=0x7f60ac198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.046+0000 7f60b366d640 1 --2- 172.21.15.43:0/3123942233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ac107270 0x7f60ac198c80 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f60a800e9f0 tx=0x7f60a800eeb0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.046+0000 7f60b366d640 1 -- 172.21.15.43:0/3123942233 shutdown_connections 2024-04-21T08:50:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.046+0000 7f60b366d640 1 --2- 172.21.15.43:0/3123942233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60ac19d000 0x7f60ac19d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.046+0000 7f60b366d640 1 --2- 172.21.15.43:0/3123942233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ac107270 0x7f60ac198c80 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.046+0000 7f60b366d640 1 --2- 172.21.15.43:0/3123942233 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f60980a3d20 0x7f60980a61e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.046+0000 7f60b366d640 1 --2- 172.21.15.43:0/3123942233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60ac1021a0 0x7f60ac198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.046+0000 7f60b366d640 1 -- 172.21.15.43:0/3123942233 >> 172.21.15.43:0/3123942233 conn(0x7f60ac0793a0 msgr2=0x7f60ac0fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.046+0000 7f60b366d640 1 -- 172.21.15.43:0/3123942233 shutdown_connections 2024-04-21T08:50:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.046+0000 7f60b366d640 1 -- 172.21.15.43:0/3123942233 wait complete. 2024-04-21T08:50:40.057 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:50:40.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.161+0000 7f65e8792640 1 -- 172.21.15.43:0/2053879223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65e0103af0 msgr2=0x7f65e010bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.161+0000 7f65e8792640 1 --2- 172.21.15.43:0/2053879223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65e0103af0 0x7f65e010bfd0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f65d0009900 tx=0x7f65d002bd60 comp rx=0 tx=0).stop 2024-04-21T08:50:40.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.161+0000 7f65e8792640 1 -- 172.21.15.43:0/2053879223 shutdown_connections 2024-04-21T08:50:40.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.161+0000 7f65e8792640 1 --2- 172.21.15.43:0/2053879223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65e010c510 0x7f65e010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.161+0000 7f65e8792640 1 --2- 172.21.15.43:0/2053879223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65e0103af0 0x7f65e010bfd0 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.161+0000 7f65e8792640 1 --2- 172.21.15.43:0/2053879223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e01031d0 0x7f65e01035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.161+0000 7f65e8792640 1 -- 172.21.15.43:0/2053879223 >> 172.21.15.43:0/2053879223 conn(0x7f65e0077f50 msgr2=0x7f65e0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.161+0000 7f65e8792640 1 -- 172.21.15.43:0/2053879223 shutdown_connections 2024-04-21T08:50:40.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.161+0000 7f65e8792640 1 -- 172.21.15.43:0/2053879223 wait complete. 2024-04-21T08:50:40.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.162+0000 7f65e8792640 1 Processor -- start 2024-04-21T08:50:40.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.162+0000 7f65e8792640 1 -- start start 2024-04-21T08:50:40.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.162+0000 7f65e8792640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65e01031d0 0x7f65e01a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.162+0000 7f65e8792640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e0103af0 0x7f65e01a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.162+0000 7f65e8792640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65e010c510 0x7f65e019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.162+0000 7f65e8792640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f65e01121a0 con 0x7f65e01031d0 2024-04-21T08:50:40.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.162+0000 7f65e8792640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f65e0112020 con 0x7f65e0103af0 2024-04-21T08:50:40.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.162+0000 7f65e8792640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65e0112320 con 0x7f65e010c510 2024-04-21T08:50:40.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.162+0000 7f65e652e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65e01031d0 0x7f65e01a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.162+0000 7f65e652e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65e01031d0 0x7f65e01a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49860/0 (socket says 172.21.15.43:49860) 2024-04-21T08:50:40.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.162+0000 7f65e652e640 1 -- 172.21.15.43:0/2485113888 learned_addr learned my addr 172.21.15.43:0/2485113888 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.162+0000 7f65e6d2f640 1 --2- 172.21.15.43:0/2485113888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65e010c510 0x7f65e019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.162+0000 7f65e5d2d640 1 --2- 172.21.15.43:0/2485113888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e0103af0 0x7f65e01a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.162+0000 7f65e652e640 1 -- 172.21.15.43:0/2485113888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65e010c510 msgr2=0x7f65e019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.162+0000 7f65e652e640 1 --2- 172.21.15.43:0/2485113888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65e010c510 0x7f65e019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.162+0000 7f65e652e640 1 -- 172.21.15.43:0/2485113888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e0103af0 msgr2=0x7f65e01a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.162+0000 7f65e652e640 1 --2- 172.21.15.43:0/2485113888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e0103af0 0x7f65e01a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.162+0000 7f65e652e640 1 -- 172.21.15.43:0/2485113888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e019af50 con 0x7f65e01031d0 2024-04-21T08:50:40.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.162+0000 7f65e652e640 1 --2- 172.21.15.43:0/2485113888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65e01031d0 0x7f65e01a0610 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00d950 tx=0x7f65dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.163+0000 7f65d77fe640 1 -- 172.21.15.43:0/2485113888 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc017070 con 0x7f65e01031d0 2024-04-21T08:50:40.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.163+0000 7f65d77fe640 1 -- 172.21.15.43:0/2485113888 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65dc009c30 con 0x7f65e01031d0 2024-04-21T08:50:40.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.163+0000 7f65e8792640 1 -- 172.21.15.43:0/2485113888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e019b240 con 0x7f65e01031d0 2024-04-21T08:50:40.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.163+0000 7f65d77fe640 1 -- 172.21.15.43:0/2485113888 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc00f620 con 0x7f65e01031d0 2024-04-21T08:50:40.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.163+0000 7f65e8792640 1 -- 172.21.15.43:0/2485113888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e01a7460 con 0x7f65e01031d0 2024-04-21T08:50:40.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.165+0000 7f65d77fe640 1 -- 172.21.15.43:0/2485113888 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f65dc010460 con 0x7f65e01031d0 2024-04-21T08:50:40.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.165+0000 7f65d77fe640 1 --2- 172.21.15.43:0/2485113888 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f65cc0a3d20 0x7f65cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.166+0000 7f65e5d2d640 1 --2- 172.21.15.43:0/2485113888 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f65cc0a3d20 0x7f65cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.166+0000 7f65d77fe640 1 -- 172.21.15.43:0/2485113888 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f65dc0d0970 con 0x7f65e01031d0 2024-04-21T08:50:40.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.166+0000 7f65eb378740 1 Processor -- start 2024-04-21T08:50:40.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.166+0000 7f65e5d2d640 1 --2- 172.21.15.43:0/2485113888 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f65cc0a3d20 0x7f65cc0a61e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f65e019bad0 tx=0x7f65d002c7c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.166+0000 7f65eb378740 1 -- start start 2024-04-21T08:50:40.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.167+0000 7f65eb378740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f7aaeb4d0 0x557f7aaeb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.167+0000 7f65eb378740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f7aaebe80 0x557f7aaf4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.167+0000 7f65eb378740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaf4a00 0x557f7aaf6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.167+0000 7f65eb378740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557f7aa501f0 con 0x557f7aaf4a00 2024-04-21T08:50:40.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.167+0000 7f65eb378740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557f7a9ed4b0 con 0x557f7aaeb4d0 2024-04-21T08:50:40.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.167+0000 7f65eb378740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557f7aa4c1b0 con 0x557f7aaebe80 2024-04-21T08:50:40.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.167+0000 7f65e6d2f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaf4a00 0x557f7aaf6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.167+0000 7f65e6d2f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaf4a00 0x557f7aaf6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49876/0 (socket says 172.21.15.43:49876) 2024-04-21T08:50:40.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.167+0000 7f65e6d2f640 1 -- 172.21.15.43:0/3321646348 learned_addr learned my addr 172.21.15.43:0/3321646348 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.167+0000 7f65e652e640 1 --2- 172.21.15.43:0/3321646348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f7aaeb4d0 0x557f7aaeb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.167+0000 7f65e5d2d640 1 --2- 172.21.15.43:0/3321646348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f7aaebe80 0x557f7aaf4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.167+0000 7f65e6d2f640 1 -- 172.21.15.43:0/3321646348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f7aaebe80 msgr2=0x557f7aaf4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.167+0000 7f65e6d2f640 1 --2- 172.21.15.43:0/3321646348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f7aaebe80 0x557f7aaf4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.167+0000 7f65e6d2f640 1 -- 172.21.15.43:0/3321646348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f7aaeb4d0 msgr2=0x557f7aaeb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.167+0000 7f65e6d2f640 1 --2- 172.21.15.43:0/3321646348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f7aaeb4d0 0x557f7aaeb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.167+0000 7f65e6d2f640 1 -- 172.21.15.43:0/3321646348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f7a9df1b0 con 0x557f7aaf4a00 2024-04-21T08:50:40.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.167+0000 7f65e5d2d640 1 --2- 172.21.15.43:0/3321646348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f7aaebe80 0x557f7aaf4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:40.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.167+0000 7f65e652e640 1 --2- 172.21.15.43:0/3321646348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f7aaeb4d0 0x557f7aaeb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:40.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.167+0000 7f65e6d2f640 1 --2- 172.21.15.43:0/3321646348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaf4a00 0x557f7aaf6e50 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f65d8009e60 tx=0x7f65d8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d7c46947e429b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.167+0000 7f65d4ff9640 1 -- 172.21.15.43:0/3321646348 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d8016070 con 0x557f7aaf4a00 2024-04-21T08:50:40.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.167+0000 7f65d4ff9640 1 -- 172.21.15.43:0/3321646348 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65d8004d20 con 0x557f7aaf4a00 2024-04-21T08:50:40.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.167+0000 7f65d4ff9640 1 -- 172.21.15.43:0/3321646348 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d80127d0 con 0x557f7aaf4a00 2024-04-21T08:50:40.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.168+0000 7f65eb378740 1 -- 172.21.15.43:0/3321646348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaf4a00 msgr2=0x557f7aaf6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.168+0000 7f65eb378740 1 --2- 172.21.15.43:0/3321646348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaf4a00 0x557f7aaf6e50 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f65d8009e60 tx=0x7f65d8010740 comp rx=0 tx=0).stop 2024-04-21T08:50:40.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.168+0000 7f65eb378740 1 -- 172.21.15.43:0/3321646348 shutdown_connections 2024-04-21T08:50:40.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.168+0000 7f65eb378740 1 --2- 172.21.15.43:0/3321646348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaf4a00 0x557f7aaf6e50 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.168+0000 7f65eb378740 1 --2- 172.21.15.43:0/3321646348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f7aaebe80 0x557f7aaf4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.168+0000 7f65eb378740 1 --2- 172.21.15.43:0/3321646348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f7aaeb4d0 0x557f7aaeb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.168+0000 7f65eb378740 1 -- 172.21.15.43:0/3321646348 >> 172.21.15.43:0/3321646348 conn(0x557f7aae5ba0 msgr2=0x557f7aae5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.168+0000 7f65eb378740 1 -- 172.21.15.43:0/3321646348 shutdown_connections 2024-04-21T08:50:40.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.168+0000 7f65eb378740 1 -- 172.21.15.43:0/3321646348 wait complete. 2024-04-21T08:50:40.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.169+0000 7f65eb378740 1 -- start start 2024-04-21T08:50:40.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.169+0000 7f65eb378740 1 Processor -- start 2024-04-21T08:50:40.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.169+0000 7f65eb378740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaeb4d0 0x557f7aaf41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.169+0000 7f65eb378740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f7aaebe80 0x557f7aaf1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.169+0000 7f65eb378740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f7aaf4a00 0x557f7aaf1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.169+0000 7f65eb378740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557f7aaf7970 con 0x557f7aaeb4d0 2024-04-21T08:50:40.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.169+0000 7f65eb378740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557f7aa09500 con 0x557f7aaebe80 2024-04-21T08:50:40.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.169+0000 7f65eb378740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557f7aaf7af0 con 0x557f7aaf4a00 2024-04-21T08:50:40.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.169+0000 7f65eb378740 10 client.0 _ll_register_callbacks cb 0x557f7a9d7b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:40.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.169+0000 7f65e652e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaeb4d0 0x557f7aaf41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.169+0000 7f65e652e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaeb4d0 0x557f7aaf41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49888/0 (socket says 172.21.15.43:49888) 2024-04-21T08:50:40.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.169+0000 7f65e652e640 1 -- 172.21.15.43:0/1058005721 learned_addr learned my addr 172.21.15.43:0/1058005721 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.169+0000 7f65e6d2f640 1 --2- 172.21.15.43:0/1058005721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f7aaf4a00 0x557f7aaf1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.169+0000 7f65e5d2d640 1 --2- 172.21.15.43:0/1058005721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f7aaebe80 0x557f7aaf1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.170+0000 7f65e652e640 1 -- 172.21.15.43:0/1058005721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f7aaf4a00 msgr2=0x557f7aaf1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.170+0000 7f65e652e640 1 --2- 172.21.15.43:0/1058005721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f7aaf4a00 0x557f7aaf1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.170+0000 7f65e652e640 1 -- 172.21.15.43:0/1058005721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f7aaebe80 msgr2=0x557f7aaf1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.170+0000 7f65e652e640 1 --2- 172.21.15.43:0/1058005721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f7aaebe80 0x557f7aaf1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.170+0000 7f65e652e640 1 -- 172.21.15.43:0/1058005721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f7a7623f0 con 0x557f7aaeb4d0 2024-04-21T08:50:40.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.170+0000 7f65e652e640 1 --2- 172.21.15.43:0/1058005721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaeb4d0 0x557f7aaf41f0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f65dc09c190 tx=0x7f65dc09cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69b33b3145442d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.170+0000 7f65c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:40.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.170+0000 7f65c9ffb640 1 -- 172.21.15.43:0/1058005721 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc020040 con 0x557f7aaeb4d0 2024-04-21T08:50:40.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.170+0000 7f65c9ffb640 1 -- 172.21.15.43:0/1058005721 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f65dc09b8e0 con 0x557f7aaeb4d0 2024-04-21T08:50:40.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.170+0000 7f65eb378740 1 -- 172.21.15.43:0/1058005721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65d8004d20 con 0x557f7aaeb4d0 2024-04-21T08:50:40.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.170+0000 7f65c9ffb640 1 -- 172.21.15.43:0/1058005721 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f65dc0a0840 con 0x557f7aaeb4d0 2024-04-21T08:50:40.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.170+0000 7f65eb378740 10 client.21066 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:40.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.170+0000 7f65eb378740 1 -- 172.21.15.43:0/1058005721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557f7a82c7d0 con 0x557f7aaeb4d0 2024-04-21T08:50:40.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.170+0000 7f65c9ffb640 1 -- 172.21.15.43:0/1058005721 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65d8004d20 con 0x557f7aaeb4d0 2024-04-21T08:50:40.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.170+0000 7f65c9ffb640 1 -- 172.21.15.43:0/1058005721 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f65dc010870 con 0x557f7aaeb4d0 2024-04-21T08:50:40.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.171+0000 7f65eb378740 10 client.21066 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:40.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.171+0000 7f65eb378740 10 client.21066 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:50:40.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.171+0000 7f65eb378740 20 client.21066 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:40.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.171+0000 7f65eb378740 1 --2- 172.21.15.43:0/1058005721 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x557f7ab04a60 0x557f7ab06e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.171+0000 7f65eb378740 4 client.21066 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:40.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.171+0000 7f65eb378740 1 -- 172.21.15.43:0/1058005721 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f7a9ef930 con 0x557f7ab04a60 2024-04-21T08:50:40.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.171+0000 7f65e5d2d640 1 --2- 172.21.15.43:0/1058005721 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x557f7ab04a60 0x557f7ab06e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.171+0000 7f65e5d2d640 1 --2- 172.21.15.43:0/1058005721 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x557f7ab04a60 0x557f7ab06e50 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a49132a55ee5c8b0 server_cookie=1f81d4ba8b639107 in_seq=0 out_seq=0 2024-04-21T08:50:40.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.171+0000 7f65c9ffb640 10 client.21066 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:50:40.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.172+0000 7f65c9ffb640 1 -- 172.21.15.43:0/1058005721 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557f7a9ef930 con 0x557f7ab04a60 2024-04-21T08:50:40.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.172+0000 7f65c9ffb640 10 client.21066 handle_command_reply: tid=0 2024-04-21T08:50:40.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.172+0000 7f65eb378740 1 client.21066 shutdown 2024-04-21T08:50:40.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.172+0000 7f65eb378740 1 -- 172.21.15.43:0/1058005721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaeb4d0 msgr2=0x557f7aaf41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.172+0000 7f65eb378740 1 --2- 172.21.15.43:0/1058005721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaeb4d0 0x557f7aaf41f0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f65dc09c190 tx=0x7f65dc09cc50 comp rx=0 tx=0).stop 2024-04-21T08:50:40.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.172+0000 7f65eb378740 1 -- 172.21.15.43:0/1058005721 shutdown_connections 2024-04-21T08:50:40.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.172+0000 7f65eb378740 1 --2- 172.21.15.43:0/1058005721 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x557f7ab04a60 0x557f7ab06e50 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.172+0000 7f65eb378740 1 --2- 172.21.15.43:0/1058005721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f7aaf4a00 0x557f7aaf1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.172+0000 7f65c9ffb640 0 client.21066 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:50:40.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.172+0000 7f65eb378740 1 --2- 172.21.15.43:0/1058005721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f7aaebe80 0x557f7aaf1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.172+0000 7f65eb378740 1 --2- 172.21.15.43:0/1058005721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaeb4d0 0x557f7aaf41f0 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.172+0000 7f65eb378740 1 -- 172.21.15.43:0/1058005721 >> 172.21.15.43:0/1058005721 conn(0x557f7aae5ba0 msgr2=0x557f7aae78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.172+0000 7f65eb378740 1 -- 172.21.15.43:0/1058005721 shutdown_connections 2024-04-21T08:50:40.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.172+0000 7f65eb378740 1 -- 172.21.15.43:0/1058005721 wait complete. 2024-04-21T08:50:40.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.172+0000 7f65eb378740 20 client.21066 trim_cache size 0 max 16384 2024-04-21T08:50:40.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.201+0000 7f65eb378740 1 Processor -- start 2024-04-21T08:50:40.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.201+0000 7f65eb378740 1 -- start start 2024-04-21T08:50:40.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.201+0000 7f65eb378740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f7aaeb4d0 0x557f7aaf3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.201+0000 7f65eb378740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f7aaebe80 0x557f7aaf41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.201+0000 7f65eb378740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaf4a00 0x557f7ab0c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.201+0000 7f65eb378740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f65e01b17d0 con 0x557f7aaf4a00 2024-04-21T08:50:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.201+0000 7f65eb378740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f65e01b3310 con 0x557f7aaebe80 2024-04-21T08:50:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.201+0000 7f65eb378740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65e01b1ea0 con 0x557f7aaeb4d0 2024-04-21T08:50:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.201+0000 7f65e6d2f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaf4a00 0x557f7ab0c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.201+0000 7f65e6d2f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaf4a00 0x557f7ab0c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49896/0 (socket says 172.21.15.43:49896) 2024-04-21T08:50:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.201+0000 7f65e6d2f640 1 -- 172.21.15.43:0/834801829 learned_addr learned my addr 172.21.15.43:0/834801829 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.201+0000 7f65e5d2d640 1 --2- 172.21.15.43:0/834801829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f7aaebe80 0x557f7aaf41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.201+0000 7f65e652e640 1 --2- 172.21.15.43:0/834801829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f7aaeb4d0 0x557f7aaf3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.201+0000 7f65e6d2f640 1 -- 172.21.15.43:0/834801829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f7aaeb4d0 msgr2=0x557f7aaf3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.201+0000 7f65e6d2f640 1 --2- 172.21.15.43:0/834801829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f7aaeb4d0 0x557f7aaf3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.201+0000 7f65e6d2f640 1 -- 172.21.15.43:0/834801829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f7aaebe80 msgr2=0x557f7aaf41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.201+0000 7f65e6d2f640 1 --2- 172.21.15.43:0/834801829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f7aaebe80 0x557f7aaf41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.201+0000 7f65e6d2f640 1 -- 172.21.15.43:0/834801829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f7aaebc20 con 0x557f7aaf4a00 2024-04-21T08:50:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.202+0000 7f65e6d2f640 1 --2- 172.21.15.43:0/834801829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaf4a00 0x557f7ab0c680 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f65d8009be0 tx=0x7f65d8005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d4b4d59e09537c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.202+0000 7f65cbfff640 1 -- 172.21.15.43:0/834801829 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d8016040 con 0x557f7aaf4a00 2024-04-21T08:50:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.202+0000 7f65cbfff640 1 -- 172.21.15.43:0/834801829 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65d8028070 con 0x557f7aaf4a00 2024-04-21T08:50:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.202+0000 7f65cbfff640 1 -- 172.21.15.43:0/834801829 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d8023a50 con 0x557f7aaf4a00 2024-04-21T08:50:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.203+0000 7f65eb378740 1 -- 172.21.15.43:0/834801829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaf4a00 msgr2=0x557f7ab0c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.203+0000 7f65eb378740 1 --2- 172.21.15.43:0/834801829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaf4a00 0x557f7ab0c680 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f65d8009be0 tx=0x7f65d8005a70 comp rx=0 tx=0).stop 2024-04-21T08:50:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.203+0000 7f65eb378740 1 -- 172.21.15.43:0/834801829 shutdown_connections 2024-04-21T08:50:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.203+0000 7f65eb378740 1 --2- 172.21.15.43:0/834801829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaf4a00 0x557f7ab0c680 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.203+0000 7f65eb378740 1 --2- 172.21.15.43:0/834801829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f7aaebe80 0x557f7aaf41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.203+0000 7f65eb378740 1 --2- 172.21.15.43:0/834801829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f7aaeb4d0 0x557f7aaf3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.203+0000 7f65eb378740 1 -- 172.21.15.43:0/834801829 >> 172.21.15.43:0/834801829 conn(0x557f7aae5ba0 msgr2=0x557f7aae4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.203+0000 7f65eb378740 1 -- 172.21.15.43:0/834801829 shutdown_connections 2024-04-21T08:50:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.203+0000 7f65eb378740 1 -- 172.21.15.43:0/834801829 wait complete. 2024-04-21T08:50:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.203+0000 7f65eb378740 1 -- start start 2024-04-21T08:50:40.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.203+0000 7f65eb378740 1 Processor -- start 2024-04-21T08:50:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.203+0000 7f65eb378740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaeb4d0 0x557f7aafd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.203+0000 7f65eb378740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f7aaebe80 0x557f7aafa410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.203+0000 7f65eb378740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f7aaf4a00 0x557f7aafa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.203+0000 7f65eb378740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f65e01b2c40 con 0x557f7aaeb4d0 2024-04-21T08:50:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.203+0000 7f65eb378740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f65e01b2570 con 0x557f7aaebe80 2024-04-21T08:50:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.203+0000 7f65eb378740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65e01af7d0 con 0x557f7aaf4a00 2024-04-21T08:50:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.203+0000 7f65eb378740 10 client.0 _ll_register_callbacks cb 0x7f65dc09bdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.203+0000 7f65e652e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaeb4d0 0x557f7aafd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.204+0000 7f65e652e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaeb4d0 0x557f7aafd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49904/0 (socket says 172.21.15.43:49904) 2024-04-21T08:50:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.204+0000 7f65e652e640 1 -- 172.21.15.43:0/841971491 learned_addr learned my addr 172.21.15.43:0/841971491 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.204+0000 7f65e5d2d640 1 --2- 172.21.15.43:0/841971491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f7aaebe80 0x557f7aafa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.204+0000 7f65e652e640 1 -- 172.21.15.43:0/841971491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f7aaf4a00 msgr2=0x557f7aafa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.204+0000 7f65e652e640 1 --2- 172.21.15.43:0/841971491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f7aaf4a00 0x557f7aafa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.204+0000 7f65e652e640 1 -- 172.21.15.43:0/841971491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f7aaebe80 msgr2=0x557f7aafa410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.204+0000 7f65e652e640 1 --2- 172.21.15.43:0/841971491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f7aaebe80 0x557f7aafa410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.204+0000 7f65e652e640 1 -- 172.21.15.43:0/841971491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f7aafb100 con 0x557f7aaeb4d0 2024-04-21T08:50:40.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.204+0000 7f65e652e640 1 --2- 172.21.15.43:0/841971491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaeb4d0 0x557f7aafd080 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f65dc098010 tx=0x7f65dc010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20f1c4c33dcdc65 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.204+0000 7f65c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:40.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.204+0000 7f65c9ffb640 1 -- 172.21.15.43:0/841971491 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc020070 con 0x557f7aaeb4d0 2024-04-21T08:50:40.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.204+0000 7f65c9ffb640 1 -- 172.21.15.43:0/841971491 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f65dc010750 con 0x557f7aaeb4d0 2024-04-21T08:50:40.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.204+0000 7f65eb378740 1 -- 172.21.15.43:0/841971491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65d8028070 con 0x557f7aaeb4d0 2024-04-21T08:50:40.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.204+0000 7f65c9ffb640 1 -- 172.21.15.43:0/841971491 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f65dc010950 con 0x557f7aaeb4d0 2024-04-21T08:50:40.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.204+0000 7f65eb378740 10 client.21072 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:40.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.204+0000 7f65eb378740 1 -- 172.21.15.43:0/841971491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557f7ab6ae80 con 0x557f7aaeb4d0 2024-04-21T08:50:40.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.204+0000 7f65c9ffb640 1 -- 172.21.15.43:0/841971491 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65d8028070 con 0x557f7aaeb4d0 2024-04-21T08:50:40.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.205+0000 7f65c9ffb640 1 -- 172.21.15.43:0/841971491 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f65dc098350 con 0x557f7aaeb4d0 2024-04-21T08:50:40.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.205+0000 7f65eb378740 10 client.21072 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:40.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.205+0000 7f65eb378740 10 client.21072 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:50:40.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.205+0000 7f65eb378740 20 client.21072 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:40.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.205+0000 7f65eb378740 1 --2- 172.21.15.43:0/841971491 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x557f7ab6b620 0x557f7ab6da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.205+0000 7f65eb378740 4 client.21072 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:50:40.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.205+0000 7f65eb378740 1 -- 172.21.15.43:0/841971491 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f7aa5dc10 con 0x557f7ab6b620 2024-04-21T08:50:40.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.205+0000 7f65e5d2d640 1 --2- 172.21.15.43:0/841971491 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x557f7ab6b620 0x557f7ab6da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.205+0000 7f65c9ffb640 10 client.21072 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:50:40.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.205+0000 7f65e5d2d640 1 --2- 172.21.15.43:0/841971491 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x557f7ab6b620 0x557f7ab6da40 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c6827fa8a90f7c51 server_cookie=a9d262a13f4ba0ec in_seq=0 out_seq=0 2024-04-21T08:50:40.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.206+0000 7f65c9ffb640 1 -- 172.21.15.43:0/841971491 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f7aa5dc10 con 0x557f7ab6b620 2024-04-21T08:50:40.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.206+0000 7f65c9ffb640 10 client.21072 handle_command_reply: tid=0 2024-04-21T08:50:40.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.206+0000 7f65eb378740 1 client.21072 shutdown 2024-04-21T08:50:40.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.206+0000 7f65eb378740 1 -- 172.21.15.43:0/841971491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaeb4d0 msgr2=0x557f7aafd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.206+0000 7f65eb378740 1 --2- 172.21.15.43:0/841971491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaeb4d0 0x557f7aafd080 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f65dc098010 tx=0x7f65dc010cb0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.206+0000 7f65eb378740 1 -- 172.21.15.43:0/841971491 shutdown_connections 2024-04-21T08:50:40.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.206+0000 7f65eb378740 1 --2- 172.21.15.43:0/841971491 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x557f7ab6b620 0x557f7ab6da40 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.206+0000 7f65eb378740 1 --2- 172.21.15.43:0/841971491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557f7aaf4a00 0x557f7aafa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.206+0000 7f65eb378740 1 --2- 172.21.15.43:0/841971491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f7aaebe80 0x557f7aafa410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.206+0000 7f65c9ffb640 0 client.21072 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:50:40.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.206+0000 7f65eb378740 1 --2- 172.21.15.43:0/841971491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f7aaeb4d0 0x557f7aafd080 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.206+0000 7f65eb378740 1 -- 172.21.15.43:0/841971491 >> 172.21.15.43:0/841971491 conn(0x557f7aae5ba0 msgr2=0x557f7aae6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.206+0000 7f65eb378740 1 -- 172.21.15.43:0/841971491 shutdown_connections 2024-04-21T08:50:40.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.206+0000 7f65eb378740 1 -- 172.21.15.43:0/841971491 wait complete. 2024-04-21T08:50:40.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.206+0000 7f65eb378740 20 client.21072 trim_cache size 0 max 16384 2024-04-21T08:50:40.213 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:50:40.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.207+0000 7f65e8792640 1 -- 172.21.15.43:0/2485113888 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f65cc0a3d20 msgr2=0x7f65cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.207+0000 7f65e8792640 1 --2- 172.21.15.43:0/2485113888 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f65cc0a3d20 0x7f65cc0a61e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f65e019bad0 tx=0x7f65d002c7c0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.207+0000 7f65e8792640 1 -- 172.21.15.43:0/2485113888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65e01031d0 msgr2=0x7f65e01a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.207+0000 7f65e8792640 1 --2- 172.21.15.43:0/2485113888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65e01031d0 0x7f65e01a0610 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00d950 tx=0x7f65dc00de10 comp rx=0 tx=0).stop 2024-04-21T08:50:40.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.207+0000 7f65e8792640 1 -- 172.21.15.43:0/2485113888 shutdown_connections 2024-04-21T08:50:40.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.207+0000 7f65e8792640 1 --2- 172.21.15.43:0/2485113888 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f65cc0a3d20 0x7f65cc0a61e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.207+0000 7f65e8792640 1 --2- 172.21.15.43:0/2485113888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65e010c510 0x7f65e019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.207+0000 7f65e8792640 1 --2- 172.21.15.43:0/2485113888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e0103af0 0x7f65e01a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.207+0000 7f65e8792640 1 --2- 172.21.15.43:0/2485113888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65e01031d0 0x7f65e01a0610 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.207+0000 7f65e8792640 1 -- 172.21.15.43:0/2485113888 >> 172.21.15.43:0/2485113888 conn(0x7f65e0077f50 msgr2=0x7f65e0108480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.207+0000 7f65e8792640 1 -- 172.21.15.43:0/2485113888 shutdown_connections 2024-04-21T08:50:40.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.207+0000 7f65e8792640 1 -- 172.21.15.43:0/2485113888 wait complete. 2024-04-21T08:50:40.218 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:50:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.219+0000 7fc49f1db640 1 -- 172.21.15.43:0/3113100048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc498101a50 msgr2=0x7fc498105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.219+0000 7fc49f1db640 1 --2- 172.21.15.43:0/3113100048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc498101a50 0x7fc498105a90 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7fc484009900 tx=0x7fc48402bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.219+0000 7fc49f1db640 1 -- 172.21.15.43:0/3113100048 shutdown_connections 2024-04-21T08:50:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.219+0000 7fc49f1db640 1 --2- 172.21.15.43:0/3113100048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc498106170 0x7fc49810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.219+0000 7fc49f1db640 1 --2- 172.21.15.43:0/3113100048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc498101a50 0x7fc498105a90 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.219+0000 7fc49f1db640 1 --2- 172.21.15.43:0/3113100048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4981010a0 0x7fc498101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.219+0000 7fc49f1db640 1 -- 172.21.15.43:0/3113100048 >> 172.21.15.43:0/3113100048 conn(0x7fc4980fc920 msgr2=0x7fc4980fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.219+0000 7fc49f1db640 1 -- 172.21.15.43:0/3113100048 shutdown_connections 2024-04-21T08:50:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.219+0000 7fc49f1db640 1 -- 172.21.15.43:0/3113100048 wait complete. 2024-04-21T08:50:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.220+0000 7fc49f1db640 1 Processor -- start 2024-04-21T08:50:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.220+0000 7fc49f1db640 1 -- start start 2024-04-21T08:50:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.220+0000 7fc49f1db640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4981010a0 0x7fc498102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.220+0000 7fc49f1db640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc498106170 0x7fc498103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.220+0000 7fc49f1db640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4981035b0 0x7fc4981a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.220+0000 7fc49f1db640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4981110b0 con 0x7fc4981010a0 2024-04-21T08:50:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.220+0000 7fc49f1db640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc498110f30 con 0x7fc4981035b0 2024-04-21T08:50:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.220+0000 7fc49f1db640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc498111230 con 0x7fc498106170 2024-04-21T08:50:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.220+0000 7fc49cf77640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4981010a0 0x7fc498102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.220+0000 7fc49cf77640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4981010a0 0x7fc498102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49920/0 (socket says 172.21.15.43:49920) 2024-04-21T08:50:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.220+0000 7fc49cf77640 1 -- 172.21.15.43:0/2707507595 learned_addr learned my addr 172.21.15.43:0/2707507595 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.220+0000 7fc497fff640 1 --2- 172.21.15.43:0/2707507595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc498106170 0x7fc498103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.221+0000 7fc49d778640 1 --2- 172.21.15.43:0/2707507595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4981035b0 0x7fc4981a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.221+0000 7fc49cf77640 1 -- 172.21.15.43:0/2707507595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc498106170 msgr2=0x7fc498103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.221+0000 7fc49cf77640 1 --2- 172.21.15.43:0/2707507595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc498106170 0x7fc498103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.221+0000 7fc49cf77640 1 -- 172.21.15.43:0/2707507595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4981035b0 msgr2=0x7fc4981a6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.221+0000 7fc49cf77640 1 --2- 172.21.15.43:0/2707507595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4981035b0 0x7fc4981a6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.221+0000 7fc49cf77640 1 -- 172.21.15.43:0/2707507595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4981a7010 con 0x7fc4981010a0 2024-04-21T08:50:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.221+0000 7fc49cf77640 1 --2- 172.21.15.43:0/2707507595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4981010a0 0x7fc498102b30 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00d950 tx=0x7fc48c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.221+0000 7fc495ffb640 1 -- 172.21.15.43:0/2707507595 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c017070 con 0x7fc4981010a0 2024-04-21T08:50:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.221+0000 7fc49f1db640 1 -- 172.21.15.43:0/2707507595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4981a7300 con 0x7fc4981010a0 2024-04-21T08:50:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.221+0000 7fc495ffb640 1 -- 172.21.15.43:0/2707507595 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc48c009c30 con 0x7fc4981010a0 2024-04-21T08:50:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.221+0000 7fc495ffb640 1 -- 172.21.15.43:0/2707507595 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c00f620 con 0x7fc4981010a0 2024-04-21T08:50:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.221+0000 7fc49f1db640 1 -- 172.21.15.43:0/2707507595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4981a7870 con 0x7fc4981010a0 2024-04-21T08:50:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.224+0000 7fc495ffb640 1 -- 172.21.15.43:0/2707507595 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc48c010460 con 0x7fc4981010a0 2024-04-21T08:50:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.224+0000 7fc4a1dc5740 1 Processor -- start 2024-04-21T08:50:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.225+0000 7fc4a1dc5740 1 -- start start 2024-04-21T08:50:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.225+0000 7fc4a1dc5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565210928d0 0x556521091620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.225+0000 7fc4a1dc5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556521091b60 0x556521091fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.225+0000 7fc4a1dc5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55652109b0e0 0x55652109d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.225+0000 7fc4a1dc5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556520ff68d0 con 0x5565210928d0 2024-04-21T08:50:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.225+0000 7fc4a1dc5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556520f93e80 con 0x55652109b0e0 2024-04-21T08:50:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.225+0000 7fc4a1dc5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556520ff2890 con 0x556521091b60 2024-04-21T08:50:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.225+0000 7fc49d778640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565210928d0 0x556521091620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.225+0000 7fc495ffb640 1 --2- 172.21.15.43:0/2707507595 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc4880a3dc0 0x7fc4880a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.225+0000 7fc49cf77640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556521091b60 0x556521091fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.225+0000 7fc49d778640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565210928d0 0x556521091620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49928/0 (socket says 172.21.15.43:49928) 2024-04-21T08:50:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.225+0000 7fc49d778640 1 -- 172.21.15.43:0/901138342 learned_addr learned my addr 172.21.15.43:0/901138342 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.225+0000 7fc497fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55652109b0e0 0x55652109d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.225+0000 7fc49d778640 1 --2- 172.21.15.43:0/2707507595 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc4880a3dc0 0x7fc4880a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.225+0000 7fc495ffb640 1 -- 172.21.15.43:0/2707507595 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fc48c0d0860 con 0x7fc4981010a0 2024-04-21T08:50:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.225+0000 7fc49d778640 1 -- 172.21.15.43:0/901138342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556521091b60 msgr2=0x556521091fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.225+0000 7fc49d778640 1 --2- 172.21.15.43:0/901138342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556521091b60 0x556521091fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.225+0000 7fc49d778640 1 -- 172.21.15.43:0/901138342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55652109b0e0 msgr2=0x55652109d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.225+0000 7fc49d778640 1 --2- 172.21.15.43:0/901138342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55652109b0e0 0x55652109d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.225+0000 7fc49d778640 1 -- 172.21.15.43:0/901138342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556520fb5320 con 0x5565210928d0 2024-04-21T08:50:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.226+0000 7fc49d778640 1 --2- 172.21.15.43:0/901138342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565210928d0 0x556521091620 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7fc490009270 tx=0x7fc490010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7eb293380120d3b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.226+0000 7fc49d778640 1 --2- 172.21.15.43:0/2707507595 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc4880a3dc0 0x7fc4880a6280 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc490010c40 tx=0x7fc49001d000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.226+0000 7fc482ffd640 1 -- 172.21.15.43:0/901138342 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc490022070 con 0x5565210928d0 2024-04-21T08:50:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.226+0000 7fc482ffd640 1 -- 172.21.15.43:0/901138342 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc490009e40 con 0x5565210928d0 2024-04-21T08:50:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.226+0000 7fc482ffd640 1 -- 172.21.15.43:0/901138342 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc490004030 con 0x5565210928d0 2024-04-21T08:50:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.227+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/901138342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565210928d0 msgr2=0x556521091620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.227+0000 7fc4a1dc5740 1 --2- 172.21.15.43:0/901138342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565210928d0 0x556521091620 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7fc490009270 tx=0x7fc490010740 comp rx=0 tx=0).stop 2024-04-21T08:50:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.227+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/901138342 shutdown_connections 2024-04-21T08:50:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.227+0000 7fc4a1dc5740 1 --2- 172.21.15.43:0/901138342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55652109b0e0 0x55652109d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.227+0000 7fc4a1dc5740 1 --2- 172.21.15.43:0/901138342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556521091b60 0x556521091fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.227+0000 7fc4a1dc5740 1 --2- 172.21.15.43:0/901138342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565210928d0 0x556521091620 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.227+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/901138342 >> 172.21.15.43:0/901138342 conn(0x556521058c90 msgr2=0x55652108c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.227+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/901138342 shutdown_connections 2024-04-21T08:50:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.227+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/901138342 wait complete. 2024-04-21T08:50:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.227+0000 7fc4a1dc5740 1 -- start start 2024-04-21T08:50:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.227+0000 7fc4a1dc5740 1 Processor -- start 2024-04-21T08:50:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.228+0000 7fc4a1dc5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556521091b60 0x55652109aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.228+0000 7fc4a1dc5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565210928d0 0x556521097e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.228+0000 7fc4a1dc5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55652109b0e0 0x556521098480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.228+0000 7fc4a1dc5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55652109ddb0 con 0x5565210928d0 2024-04-21T08:50:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.228+0000 7fc4a1dc5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556520f7eeb0 con 0x55652109b0e0 2024-04-21T08:50:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.228+0000 7fc4a1dc5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55652109df30 con 0x556521091b60 2024-04-21T08:50:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.228+0000 7fc4a1dc5740 10 client.0 _ll_register_callbacks cb 0x556520fabb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.228+0000 7fc497fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565210928d0 0x556521097e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.228+0000 7fc497fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565210928d0 0x556521097e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49930/0 (socket says 172.21.15.43:49930) 2024-04-21T08:50:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.228+0000 7fc497fff640 1 -- 172.21.15.43:0/3684171647 learned_addr learned my addr 172.21.15.43:0/3684171647 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.228+0000 7fc49cf77640 1 --2- 172.21.15.43:0/3684171647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556521091b60 0x55652109aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.228+0000 7fc49d778640 1 --2- 172.21.15.43:0/3684171647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55652109b0e0 0x556521098480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.228+0000 7fc497fff640 1 -- 172.21.15.43:0/3684171647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556521091b60 msgr2=0x55652109aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.228+0000 7fc497fff640 1 --2- 172.21.15.43:0/3684171647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556521091b60 0x55652109aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.228+0000 7fc497fff640 1 -- 172.21.15.43:0/3684171647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55652109b0e0 msgr2=0x556521098480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.228+0000 7fc497fff640 1 --2- 172.21.15.43:0/3684171647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55652109b0e0 0x556521098480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.228+0000 7fc497fff640 1 -- 172.21.15.43:0/3684171647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556520d09210 con 0x5565210928d0 2024-04-21T08:50:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.228+0000 7fc49cf77640 1 --2- 172.21.15.43:0/3684171647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556521091b60 0x55652109aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.229+0000 7fc497fff640 1 --2- 172.21.15.43:0/3684171647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565210928d0 0x556521097e70 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7fc484009900 tx=0x7fc48402ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ba6b3262bf319b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.229+0000 7fc477fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.229+0000 7fc477fff640 1 -- 172.21.15.43:0/3684171647 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484005030 con 0x5565210928d0 2024-04-21T08:50:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.229+0000 7fc477fff640 1 -- 172.21.15.43:0/3684171647 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fc4840051d0 con 0x5565210928d0 2024-04-21T08:50:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.229+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/3684171647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55652100a880 con 0x5565210928d0 2024-04-21T08:50:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.229+0000 7fc477fff640 1 -- 172.21.15.43:0/3684171647 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc484052050 con 0x5565210928d0 2024-04-21T08:50:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.229+0000 7fc4a1dc5740 10 client.21084 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.229+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/3684171647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556520dd5620 con 0x5565210928d0 2024-04-21T08:50:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.229+0000 7fc477fff640 1 -- 172.21.15.43:0/3684171647 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55652100a880 con 0x5565210928d0 2024-04-21T08:50:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.229+0000 7fc477fff640 1 -- 172.21.15.43:0/3684171647 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc4840035f0 con 0x5565210928d0 2024-04-21T08:50:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.229+0000 7fc4a1dc5740 10 client.21084 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.229+0000 7fc4a1dc5740 10 client.21084 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.229+0000 7fc4a1dc5740 20 client.21084 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.229+0000 7fc4a1dc5740 1 --2- 172.21.15.43:0/3684171647 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5565210ab0d0 0x5565210ad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.229+0000 7fc4a1dc5740 4 client.21084 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.229+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/3684171647 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556520f963d0 con 0x5565210ab0d0 2024-04-21T08:50:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.230+0000 7fc49cf77640 1 --2- 172.21.15.43:0/3684171647 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5565210ab0d0 0x5565210ad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.230+0000 7fc49cf77640 1 --2- 172.21.15.43:0/3684171647 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5565210ab0d0 0x5565210ad4e0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=108647b7fd8435ca server_cookie=1e60f2b8643af3a5 in_seq=0 out_seq=0 2024-04-21T08:50:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.230+0000 7fc477fff640 10 client.21084 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.231+0000 7fc477fff640 1 -- 172.21.15.43:0/3684171647 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556520f963d0 con 0x5565210ab0d0 2024-04-21T08:50:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.231+0000 7fc477fff640 10 client.21084 handle_command_reply: tid=0 2024-04-21T08:50:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.231+0000 7fc4a1dc5740 1 client.21084 shutdown 2024-04-21T08:50:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.231+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/3684171647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565210928d0 msgr2=0x556521097e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.231+0000 7fc4a1dc5740 1 --2- 172.21.15.43:0/3684171647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565210928d0 0x556521097e70 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7fc484009900 tx=0x7fc48402ca10 comp rx=0 tx=0).stop 2024-04-21T08:50:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.231+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/3684171647 shutdown_connections 2024-04-21T08:50:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.231+0000 7fc4a1dc5740 1 --2- 172.21.15.43:0/3684171647 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5565210ab0d0 0x5565210ad4e0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.231+0000 7fc4a1dc5740 1 --2- 172.21.15.43:0/3684171647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55652109b0e0 0x556521098480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.231+0000 7fc4a1dc5740 1 --2- 172.21.15.43:0/3684171647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565210928d0 0x556521097e70 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.231+0000 7fc477fff640 0 client.21084 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.231+0000 7fc4a1dc5740 1 --2- 172.21.15.43:0/3684171647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556521091b60 0x55652109aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.231+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/3684171647 >> 172.21.15.43:0/3684171647 conn(0x556521058c90 msgr2=0x55652108e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.231+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/3684171647 shutdown_connections 2024-04-21T08:50:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.231+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/3684171647 wait complete. 2024-04-21T08:50:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.231+0000 7fc4a1dc5740 20 client.21084 trim_cache size 0 max 16384 2024-04-21T08:50:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.247+0000 7fc4a1dc5740 1 Processor -- start 2024-04-21T08:50:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.247+0000 7fc4a1dc5740 1 -- start start 2024-04-21T08:50:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.247+0000 7fc4a1dc5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556521091b60 0x5565210a6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.247+0000 7fc4a1dc5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565210928d0 0x5565210b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.247+0000 7fc4a1dc5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55652109b0e0 0x5565210b3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.247+0000 7fc4a1dc5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4981b16c0 con 0x5565210928d0 2024-04-21T08:50:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.247+0000 7fc4a1dc5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc4981b3200 con 0x556521091b60 2024-04-21T08:50:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.247+0000 7fc4a1dc5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc4981b1d90 con 0x55652109b0e0 2024-04-21T08:50:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.247+0000 7fc497fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565210928d0 0x5565210b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.247+0000 7fc497fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565210928d0 0x5565210b2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49940/0 (socket says 172.21.15.43:49940) 2024-04-21T08:50:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.247+0000 7fc497fff640 1 -- 172.21.15.43:0/1760097546 learned_addr learned my addr 172.21.15.43:0/1760097546 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.247+0000 7fc49cf77640 1 --2- 172.21.15.43:0/1760097546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556521091b60 0x5565210a6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.247+0000 7fc49d778640 1 --2- 172.21.15.43:0/1760097546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55652109b0e0 0x5565210b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.247+0000 7fc497fff640 1 -- 172.21.15.43:0/1760097546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55652109b0e0 msgr2=0x5565210b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.247+0000 7fc497fff640 1 --2- 172.21.15.43:0/1760097546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55652109b0e0 0x5565210b3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.247+0000 7fc497fff640 1 -- 172.21.15.43:0/1760097546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556521091b60 msgr2=0x5565210a6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.247+0000 7fc497fff640 1 --2- 172.21.15.43:0/1760097546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556521091b60 0x5565210a6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.247+0000 7fc497fff640 1 -- 172.21.15.43:0/1760097546 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556521090e30 con 0x5565210928d0 2024-04-21T08:50:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.248+0000 7fc497fff640 1 --2- 172.21.15.43:0/1760097546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565210928d0 0x5565210b2df0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7fc484009850 tx=0x7fc48402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b6c70064ade044 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.248+0000 7fc4827fc640 1 -- 172.21.15.43:0/1760097546 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484000e60 con 0x5565210928d0 2024-04-21T08:50:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.248+0000 7fc4827fc640 1 -- 172.21.15.43:0/1760097546 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4840035f0 con 0x5565210928d0 2024-04-21T08:50:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.248+0000 7fc4827fc640 1 -- 172.21.15.43:0/1760097546 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484004920 con 0x5565210928d0 2024-04-21T08:50:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.248+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/1760097546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565210928d0 msgr2=0x5565210b2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.248+0000 7fc4a1dc5740 1 --2- 172.21.15.43:0/1760097546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565210928d0 0x5565210b2df0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7fc484009850 tx=0x7fc48402bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.249+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/1760097546 shutdown_connections 2024-04-21T08:50:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.249+0000 7fc4a1dc5740 1 --2- 172.21.15.43:0/1760097546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55652109b0e0 0x5565210b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.249+0000 7fc4a1dc5740 1 --2- 172.21.15.43:0/1760097546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565210928d0 0x5565210b2df0 unknown :-1 s=CLOSED pgs=2256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.249+0000 7fc4a1dc5740 1 --2- 172.21.15.43:0/1760097546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556521091b60 0x5565210a6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.249+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/1760097546 >> 172.21.15.43:0/1760097546 conn(0x556521058c90 msgr2=0x556521093030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.249+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/1760097546 shutdown_connections 2024-04-21T08:50:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.249+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/1760097546 wait complete. 2024-04-21T08:50:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.249+0000 7fc4a1dc5740 1 -- start start 2024-04-21T08:50:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.249+0000 7fc4a1dc5740 1 Processor -- start 2024-04-21T08:50:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.249+0000 7fc4a1dc5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556521091b60 0x5565210a33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.249+0000 7fc4a1dc5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565210928d0 0x5565210950f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.249+0000 7fc4a1dc5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55652109b0e0 0x556521095630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.249+0000 7fc4a1dc5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4981b2b30 con 0x55652109b0e0 2024-04-21T08:50:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.249+0000 7fc4a1dc5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc4981b2460 con 0x5565210928d0 2024-04-21T08:50:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.249+0000 7fc4a1dc5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc4981af6c0 con 0x556521091b60 2024-04-21T08:50:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.249+0000 7fc4a1dc5740 10 client.0 _ll_register_callbacks cb 0x7fc484009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.250+0000 7fc49d778640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55652109b0e0 0x556521095630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.250+0000 7fc49d778640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55652109b0e0 0x556521095630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49952/0 (socket says 172.21.15.43:49952) 2024-04-21T08:50:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.250+0000 7fc49d778640 1 -- 172.21.15.43:0/1731699427 learned_addr learned my addr 172.21.15.43:0/1731699427 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.250+0000 7fc497fff640 1 --2- 172.21.15.43:0/1731699427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565210928d0 0x5565210950f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.250+0000 7fc49cf77640 1 --2- 172.21.15.43:0/1731699427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556521091b60 0x5565210a33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.250+0000 7fc49d778640 1 -- 172.21.15.43:0/1731699427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556521091b60 msgr2=0x5565210a33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.250+0000 7fc49d778640 1 --2- 172.21.15.43:0/1731699427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556521091b60 0x5565210a33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.250+0000 7fc49d778640 1 -- 172.21.15.43:0/1731699427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565210928d0 msgr2=0x5565210950f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.250+0000 7fc49d778640 1 --2- 172.21.15.43:0/1731699427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565210928d0 0x5565210950f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.250+0000 7fc49d778640 1 -- 172.21.15.43:0/1731699427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556521095c20 con 0x55652109b0e0 2024-04-21T08:50:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.250+0000 7fc49d778640 1 --2- 172.21.15.43:0/1731699427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55652109b0e0 0x556521095630 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fc490009270 tx=0x7fc490009a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10bae610f9c186ba server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.250+0000 7fc477fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.250+0000 7fc477fff640 1 -- 172.21.15.43:0/1731699427 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc490022030 con 0x55652109b0e0 2024-04-21T08:50:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.250+0000 7fc477fff640 1 -- 172.21.15.43:0/1731699427 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fc490003970 con 0x55652109b0e0 2024-04-21T08:50:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.250+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/1731699427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4840035f0 con 0x55652109b0e0 2024-04-21T08:50:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.250+0000 7fc477fff640 1 -- 172.21.15.43:0/1731699427 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc49001b040 con 0x55652109b0e0 2024-04-21T08:50:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.250+0000 7fc4a1dc5740 10 client.21090 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.250+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/1731699427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55652109c710 con 0x55652109b0e0 2024-04-21T08:50:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.251+0000 7fc477fff640 1 -- 172.21.15.43:0/1731699427 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4840035f0 con 0x55652109b0e0 2024-04-21T08:50:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.251+0000 7fc477fff640 1 -- 172.21.15.43:0/1731699427 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc49002f460 con 0x55652109b0e0 2024-04-21T08:50:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.251+0000 7fc4a1dc5740 10 client.21090 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.251+0000 7fc4a1dc5740 10 client.21090 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.251+0000 7fc4a1dc5740 20 client.21090 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.251+0000 7fc4a1dc5740 1 --2- 172.21.15.43:0/1731699427 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5565210ab0d0 0x5565211138e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.251+0000 7fc4a1dc5740 4 client.21090 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:50:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.251+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/1731699427 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5565210042f0 con 0x5565210ab0d0 2024-04-21T08:50:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.251+0000 7fc49cf77640 1 --2- 172.21.15.43:0/1731699427 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5565210ab0d0 0x5565211138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.252+0000 7fc477fff640 10 client.21090 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.252+0000 7fc49cf77640 1 --2- 172.21.15.43:0/1731699427 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5565210ab0d0 0x5565211138e0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc02ef3ef949c6f7 server_cookie=1c75587bc63366b0 in_seq=0 out_seq=0 2024-04-21T08:50:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.267+0000 7fc477fff640 1 -- 172.21.15.43:0/1731699427 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5565210042f0 con 0x5565210ab0d0 2024-04-21T08:50:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.267+0000 7fc477fff640 10 client.21090 handle_command_reply: tid=0 2024-04-21T08:50:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.267+0000 7fc4a1dc5740 1 client.21090 shutdown 2024-04-21T08:50:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.267+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/1731699427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55652109b0e0 msgr2=0x556521095630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.267+0000 7fc4a1dc5740 1 --2- 172.21.15.43:0/1731699427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55652109b0e0 0x556521095630 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fc490009270 tx=0x7fc490009a40 comp rx=0 tx=0).stop 2024-04-21T08:50:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.267+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/1731699427 shutdown_connections 2024-04-21T08:50:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.267+0000 7fc4a1dc5740 1 --2- 172.21.15.43:0/1731699427 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5565210ab0d0 0x5565211138e0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.267+0000 7fc4a1dc5740 1 --2- 172.21.15.43:0/1731699427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55652109b0e0 0x556521095630 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.267+0000 7fc477fff640 0 client.21090 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.267+0000 7fc4a1dc5740 1 --2- 172.21.15.43:0/1731699427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565210928d0 0x5565210950f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.267+0000 7fc4a1dc5740 1 --2- 172.21.15.43:0/1731699427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556521091b60 0x5565210a33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.267+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/1731699427 >> 172.21.15.43:0/1731699427 conn(0x556521058c90 msgr2=0x556521096700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.267+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/1731699427 shutdown_connections 2024-04-21T08:50:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.267+0000 7fc4a1dc5740 1 -- 172.21.15.43:0/1731699427 wait complete. 2024-04-21T08:50:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.267+0000 7fc4a1dc5740 20 client.21090 trim_cache size 0 max 16384 2024-04-21T08:50:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:50:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:50:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:50:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:50:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.268+0000 7fc49f1db640 1 -- 172.21.15.43:0/2707507595 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc4880a3dc0 msgr2=0x7fc4880a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.268+0000 7fc49f1db640 1 --2- 172.21.15.43:0/2707507595 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc4880a3dc0 0x7fc4880a6280 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc490010c40 tx=0x7fc49001d000 comp rx=0 tx=0).stop 2024-04-21T08:50:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.268+0000 7fc49f1db640 1 -- 172.21.15.43:0/2707507595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4981010a0 msgr2=0x7fc498102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.268+0000 7fc49f1db640 1 --2- 172.21.15.43:0/2707507595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4981010a0 0x7fc498102b30 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00d950 tx=0x7fc48c00de10 comp rx=0 tx=0).stop 2024-04-21T08:50:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.268+0000 7fc49f1db640 1 -- 172.21.15.43:0/2707507595 shutdown_connections 2024-04-21T08:50:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.268+0000 7fc49f1db640 1 --2- 172.21.15.43:0/2707507595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4981035b0 0x7fc4981a6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.268+0000 7fc49f1db640 1 --2- 172.21.15.43:0/2707507595 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc4880a3dc0 0x7fc4880a6280 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.268+0000 7fc49f1db640 1 --2- 172.21.15.43:0/2707507595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc498106170 0x7fc498103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.268+0000 7fc49f1db640 1 --2- 172.21.15.43:0/2707507595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4981010a0 0x7fc498102b30 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.268+0000 7fc49f1db640 1 -- 172.21.15.43:0/2707507595 >> 172.21.15.43:0/2707507595 conn(0x7fc4980fc920 msgr2=0x7fc49806ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.268+0000 7fc49f1db640 1 -- 172.21.15.43:0/2707507595 shutdown_connections 2024-04-21T08:50:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:40.268+0000 7fc49f1db640 1 -- 172.21.15.43:0/2707507595 wait complete. 2024-04-21T08:50:40.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.327+0000 7fb0a9af1640 1 -- 172.21.15.43:0/1702229786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0a406b1f0 msgr2=0x7fb0a4106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.327+0000 7fb0a9af1640 1 --2- 172.21.15.43:0/1702229786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0a406b1f0 0x7fb0a4106b90 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7fb098009900 tx=0x7fb09802bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:40.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.327+0000 7fb0a9af1640 1 -- 172.21.15.43:0/1702229786 shutdown_connections 2024-04-21T08:50:40.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.327+0000 7fb0a9af1640 1 --2- 172.21.15.43:0/1702229786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a4107270 0x7fb0a410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.327+0000 7fb0a9af1640 1 --2- 172.21.15.43:0/1702229786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0a406b1f0 0x7fb0a4106b90 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.327+0000 7fb0a9af1640 1 --2- 172.21.15.43:0/1702229786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0a406a840 0x7fb0a406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.327+0000 7fb0a9af1640 1 -- 172.21.15.43:0/1702229786 >> 172.21.15.43:0/1702229786 conn(0x7fb0a4077eb0 msgr2=0x7fb0a40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.327+0000 7fb0a9af1640 1 -- 172.21.15.43:0/1702229786 shutdown_connections 2024-04-21T08:50:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.327+0000 7fb0a9af1640 1 -- 172.21.15.43:0/1702229786 wait complete. 2024-04-21T08:50:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.327+0000 7fb0a9af1640 1 Processor -- start 2024-04-21T08:50:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.328+0000 7fb0a9af1640 1 -- start start 2024-04-21T08:50:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.328+0000 7fb0a9af1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0a406a840 0x7fb0a4198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.328+0000 7fb0a9af1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a4107270 0x7fb0a4198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.328+0000 7fb0a9af1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0a419cf50 0x7fb0a419d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.328+0000 7fb0a9af1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a4112190 con 0x7fb0a406a840 2024-04-21T08:50:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.328+0000 7fb0a9af1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a4112010 con 0x7fb0a4107270 2024-04-21T08:50:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.328+0000 7fb0a9af1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a4112310 con 0x7fb0a419cf50 2024-04-21T08:50:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.328+0000 7fb0a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0a406a840 0x7fb0a4198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.328+0000 7fb0a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0a406a840 0x7fb0a4198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49974/0 (socket says 172.21.15.43:49974) 2024-04-21T08:50:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.328+0000 7fb0a37fe640 1 -- 172.21.15.43:0/711514135 learned_addr learned my addr 172.21.15.43:0/711514135 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.328+0000 7fb0a3fff640 1 --2- 172.21.15.43:0/711514135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0a419cf50 0x7fb0a419d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.328+0000 7fb0a2ffd640 1 --2- 172.21.15.43:0/711514135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a4107270 0x7fb0a4198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.328+0000 7fb0a37fe640 1 -- 172.21.15.43:0/711514135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0a419cf50 msgr2=0x7fb0a419d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.328+0000 7fb0a37fe640 1 --2- 172.21.15.43:0/711514135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0a419cf50 0x7fb0a419d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.328+0000 7fb0a37fe640 1 -- 172.21.15.43:0/711514135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a4107270 msgr2=0x7fb0a4198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.328+0000 7fb0a37fe640 1 --2- 172.21.15.43:0/711514135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a4107270 0x7fb0a4198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.328+0000 7fb0a37fe640 1 -- 172.21.15.43:0/711514135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a419da40 con 0x7fb0a406a840 2024-04-21T08:50:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.328+0000 7fb0a37fe640 1 --2- 172.21.15.43:0/711514135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0a406a840 0x7fb0a4198690 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7fb09400c920 tx=0x7fb09400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.329+0000 7fb0a0ff9640 1 -- 172.21.15.43:0/711514135 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb094007c70 con 0x7fb0a406a840 2024-04-21T08:50:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.329+0000 7fb0a9af1640 1 -- 172.21.15.43:0/711514135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a419dd30 con 0x7fb0a406a840 2024-04-21T08:50:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.329+0000 7fb0a0ff9640 1 -- 172.21.15.43:0/711514135 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb094007e10 con 0x7fb0a406a840 2024-04-21T08:50:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.329+0000 7fb0a0ff9640 1 -- 172.21.15.43:0/711514135 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb094018820 con 0x7fb0a406a840 2024-04-21T08:50:40.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.329+0000 7fb0a9af1640 1 -- 172.21.15.43:0/711514135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a41a54f0 con 0x7fb0a406a840 2024-04-21T08:50:40.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.331+0000 7fb0a0ff9640 1 -- 172.21.15.43:0/711514135 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb09400f450 con 0x7fb0a406a840 2024-04-21T08:50:40.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.331+0000 7fb0ac7b6740 1 Processor -- start 2024-04-21T08:50:40.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.331+0000 7fb0ac7b6740 1 -- start start 2024-04-21T08:50:40.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.332+0000 7fb0ac7b6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bd5d4c4d0 0x557bd5d4c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.332+0000 7fb0ac7b6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d4ce80 0x557bd5d55430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.332+0000 7fb0ac7b6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd5d55a00 0x557bd5d57e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.332+0000 7fb0ac7b6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557bd5cb11f0 con 0x557bd5d4ce80 2024-04-21T08:50:40.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.332+0000 7fb0ac7b6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557bd5c4e4b0 con 0x557bd5d55a00 2024-04-21T08:50:40.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.332+0000 7fb0ac7b6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557bd5cad1b0 con 0x557bd5d4c4d0 2024-04-21T08:50:40.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.332+0000 7fb0a0ff9640 1 --2- 172.21.15.43:0/711514135 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb08c0a3d20 0x7fb08c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.332+0000 7fb0a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d4ce80 0x557bd5d55430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.332+0000 7fb0a2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd5d55a00 0x557bd5d57e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.332+0000 7fb0a3fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bd5d4c4d0 0x557bd5d4c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.332+0000 7fb0a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d4ce80 0x557bd5d55430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49976/0 (socket says 172.21.15.43:49976) 2024-04-21T08:50:40.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.332+0000 7fb0a2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd5d55a00 0x557bd5d57e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:36568/0 (socket says 172.21.15.43:36568) 2024-04-21T08:50:40.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.332+0000 7fb0a3fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bd5d4c4d0 0x557bd5d4c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.43:39534/0 (socket says 172.21.15.43:39534) 2024-04-21T08:50:40.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.332+0000 7fb0a37fe640 1 -- 172.21.15.43:0/878438750 learned_addr learned my addr 172.21.15.43:0/878438750 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.332+0000 7fb0a3fff640 1 --2- 172.21.15.43:0/711514135 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb08c0a3d20 0x7fb08c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.332+0000 7fb0a0ff9640 1 -- 172.21.15.43:0/711514135 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fb0940d0cf0 con 0x7fb0a406a840 2024-04-21T08:50:40.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.332+0000 7fb0a37fe640 1 -- 172.21.15.43:0/878438750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bd5d4c4d0 msgr2=0x557bd5d4c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.332+0000 7fb0a37fe640 1 --2- 172.21.15.43:0/878438750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bd5d4c4d0 0x557bd5d4c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.332+0000 7fb0a37fe640 1 -- 172.21.15.43:0/878438750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd5d55a00 msgr2=0x557bd5d57e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.332+0000 7fb0a37fe640 1 --2- 172.21.15.43:0/878438750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd5d55a00 0x557bd5d57e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.332+0000 7fb0a37fe640 1 -- 172.21.15.43:0/878438750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bd5c401b0 con 0x557bd5d4ce80 2024-04-21T08:50:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.333+0000 7fb0a37fe640 1 --2- 172.21.15.43:0/878438750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d4ce80 0x557bd5d55430 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7fb0940ce020 tx=0x7fb09400fe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218937ed69491734 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.333+0000 7fb0a3fff640 1 --2- 172.21.15.43:0/711514135 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb08c0a3d20 0x7fb08c0a61e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb09c009990 tx=0x7fb09c0096a0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.333+0000 7fb089ffb640 1 -- 172.21.15.43:0/878438750 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0940a5480 con 0x557bd5d4ce80 2024-04-21T08:50:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.333+0000 7fb089ffb640 1 -- 172.21.15.43:0/878438750 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb09409ab10 con 0x557bd5d4ce80 2024-04-21T08:50:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.333+0000 7fb089ffb640 1 -- 172.21.15.43:0/878438750 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0940ce240 con 0x557bd5d4ce80 2024-04-21T08:50:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.333+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/878438750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d4ce80 msgr2=0x557bd5d55430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.333+0000 7fb0ac7b6740 1 --2- 172.21.15.43:0/878438750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d4ce80 0x557bd5d55430 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7fb0940ce020 tx=0x7fb09400fe60 comp rx=0 tx=0).stop 2024-04-21T08:50:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.333+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/878438750 shutdown_connections 2024-04-21T08:50:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.333+0000 7fb0ac7b6740 1 --2- 172.21.15.43:0/878438750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd5d55a00 0x557bd5d57e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.333+0000 7fb0ac7b6740 1 --2- 172.21.15.43:0/878438750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d4ce80 0x557bd5d55430 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.333+0000 7fb0ac7b6740 1 --2- 172.21.15.43:0/878438750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bd5d4c4d0 0x557bd5d4c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.333+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/878438750 >> 172.21.15.43:0/878438750 conn(0x557bd5d46ba0 msgr2=0x557bd5d46f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.334+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/878438750 shutdown_connections 2024-04-21T08:50:40.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.334+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/878438750 wait complete. 2024-04-21T08:50:40.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.334+0000 7fb0ac7b6740 1 -- start start 2024-04-21T08:50:40.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.334+0000 7fb0ac7b6740 1 Processor -- start 2024-04-21T08:50:40.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.334+0000 7fb0ac7b6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd5d4c4d0 0x557bd5d55300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.334+0000 7fb0ac7b6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bd5d4ce80 0x557bd5d52690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.334+0000 7fb0ac7b6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d55a00 0x557bd5d52c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.334+0000 7fb0ac7b6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557bd5d58910 con 0x557bd5d55a00 2024-04-21T08:50:40.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.334+0000 7fb0ac7b6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557bd5c6a500 con 0x557bd5d4c4d0 2024-04-21T08:50:40.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.334+0000 7fb0ac7b6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557bd5d58a90 con 0x557bd5d4ce80 2024-04-21T08:50:40.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.334+0000 7fb0ac7b6740 10 client.0 _ll_register_callbacks cb 0x557bd5c38b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:40.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.334+0000 7fb0a3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d55a00 0x557bd5d52c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.334+0000 7fb0a3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d55a00 0x557bd5d52c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49988/0 (socket says 172.21.15.43:49988) 2024-04-21T08:50:40.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.335+0000 7fb0a3fff640 1 -- 172.21.15.43:0/4175407510 learned_addr learned my addr 172.21.15.43:0/4175407510 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.335+0000 7fb0a37fe640 1 --2- 172.21.15.43:0/4175407510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd5d4c4d0 0x557bd5d55300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.335+0000 7fb0a2ffd640 1 --2- 172.21.15.43:0/4175407510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bd5d4ce80 0x557bd5d52690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.335+0000 7fb0a3fff640 1 -- 172.21.15.43:0/4175407510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bd5d4ce80 msgr2=0x557bd5d52690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.335+0000 7fb0a3fff640 1 --2- 172.21.15.43:0/4175407510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bd5d4ce80 0x557bd5d52690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.335+0000 7fb0a3fff640 1 -- 172.21.15.43:0/4175407510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd5d4c4d0 msgr2=0x557bd5d55300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.335+0000 7fb0a3fff640 1 --2- 172.21.15.43:0/4175407510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd5d4c4d0 0x557bd5d55300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.335+0000 7fb0a3fff640 1 -- 172.21.15.43:0/4175407510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bd59c33f0 con 0x557bd5d55a00 2024-04-21T08:50:40.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.335+0000 7fb0a3fff640 1 --2- 172.21.15.43:0/4175407510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d55a00 0x557bd5d52c10 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7fb09c003cf0 tx=0x7fb09c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619f630047bd4a61 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.335+0000 7fb082ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:40.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.335+0000 7fb082ffd640 1 -- 172.21.15.43:0/4175407510 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c003d20 con 0x557bd5d55a00 2024-04-21T08:50:40.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.335+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/4175407510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb09409ab10 con 0x557bd5d55a00 2024-04-21T08:50:40.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.335+0000 7fb082ffd640 1 -- 172.21.15.43:0/4175407510 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fb09c004040 con 0x557bd5d55a00 2024-04-21T08:50:40.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.335+0000 7fb082ffd640 1 -- 172.21.15.43:0/4175407510 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb09c004240 con 0x557bd5d55a00 2024-04-21T08:50:40.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.335+0000 7fb0ac7b6740 10 client.21102 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:40.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.335+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/4175407510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557bd5a8d7d0 con 0x557bd5d55a00 2024-04-21T08:50:40.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.336+0000 7fb082ffd640 1 -- 172.21.15.43:0/4175407510 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb09409ab10 con 0x557bd5d55a00 2024-04-21T08:50:40.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.336+0000 7fb082ffd640 1 -- 172.21.15.43:0/4175407510 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb09c010830 con 0x557bd5d55a00 2024-04-21T08:50:40.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.336+0000 7fb0ac7b6740 10 client.21102 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:40.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.336+0000 7fb0ac7b6740 10 client.21102 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:50:40.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.336+0000 7fb0ac7b6740 20 client.21102 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:40.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.336+0000 7fb0ac7b6740 1 --2- 172.21.15.43:0/4175407510 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x557bd5d65c40 0x557bd5d680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.336+0000 7fb0ac7b6740 4 client.21102 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:40.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.336+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/4175407510 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bd5c50930 con 0x557bd5d65c40 2024-04-21T08:50:40.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.336+0000 7fb0a37fe640 1 --2- 172.21.15.43:0/4175407510 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x557bd5d65c40 0x557bd5d680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.336+0000 7fb0a37fe640 1 --2- 172.21.15.43:0/4175407510 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x557bd5d65c40 0x557bd5d680b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=abc02de77d0d78c8 server_cookie=5e3532f8747a1ec in_seq=0 out_seq=0 2024-04-21T08:50:40.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.336+0000 7fb082ffd640 10 client.21102 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:50:40.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.337+0000 7fb082ffd640 1 -- 172.21.15.43:0/4175407510 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557bd5c50930 con 0x557bd5d65c40 2024-04-21T08:50:40.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.337+0000 7fb082ffd640 10 client.21102 handle_command_reply: tid=0 2024-04-21T08:50:40.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.337+0000 7fb0ac7b6740 1 client.21102 shutdown 2024-04-21T08:50:40.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.337+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/4175407510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d55a00 msgr2=0x557bd5d52c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.337+0000 7fb0ac7b6740 1 --2- 172.21.15.43:0/4175407510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d55a00 0x557bd5d52c10 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7fb09c003cf0 tx=0x7fb09c004a40 comp rx=0 tx=0).stop 2024-04-21T08:50:40.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.337+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/4175407510 shutdown_connections 2024-04-21T08:50:40.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.337+0000 7fb0ac7b6740 1 --2- 172.21.15.43:0/4175407510 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x557bd5d65c40 0x557bd5d680b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.337+0000 7fb0ac7b6740 1 --2- 172.21.15.43:0/4175407510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d55a00 0x557bd5d52c10 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.337+0000 7fb0ac7b6740 1 --2- 172.21.15.43:0/4175407510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bd5d4ce80 0x557bd5d52690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.337+0000 7fb082ffd640 0 client.21102 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:50:40.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.337+0000 7fb0ac7b6740 1 --2- 172.21.15.43:0/4175407510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd5d4c4d0 0x557bd5d55300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.337+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/4175407510 >> 172.21.15.43:0/4175407510 conn(0x557bd5d46ba0 msgr2=0x557bd5d488f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.337+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/4175407510 shutdown_connections 2024-04-21T08:50:40.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.337+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/4175407510 wait complete. 2024-04-21T08:50:40.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.338+0000 7fb0ac7b6740 20 client.21102 trim_cache size 0 max 16384 2024-04-21T08:50:40.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.351+0000 7fb0ac7b6740 1 Processor -- start 2024-04-21T08:50:40.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.351+0000 7fb0ac7b6740 1 -- start start 2024-04-21T08:50:40.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.351+0000 7fb0ac7b6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d4c4d0 0x557bd5d617d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.351+0000 7fb0ac7b6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd5d4ce80 0x557bd5d48700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.351+0000 7fb0ac7b6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bd5d55a00 0x557bd5d54dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.351+0000 7fb0ac7b6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a41af470 con 0x557bd5d4c4d0 2024-04-21T08:50:40.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.351+0000 7fb0ac7b6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a41b0fb0 con 0x557bd5d4ce80 2024-04-21T08:50:40.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.351+0000 7fb0ac7b6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a41afb40 con 0x557bd5d55a00 2024-04-21T08:50:40.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.351+0000 7fb0a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d4c4d0 0x557bd5d617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.351+0000 7fb0a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d4c4d0 0x557bd5d617d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50004/0 (socket says 172.21.15.43:50004) 2024-04-21T08:50:40.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.351+0000 7fb0a37fe640 1 -- 172.21.15.43:0/1813053539 learned_addr learned my addr 172.21.15.43:0/1813053539 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.351+0000 7fb0a2ffd640 1 --2- 172.21.15.43:0/1813053539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd5d4ce80 0x557bd5d48700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.351+0000 7fb0a3fff640 1 --2- 172.21.15.43:0/1813053539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bd5d55a00 0x557bd5d54dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.351+0000 7fb0a37fe640 1 -- 172.21.15.43:0/1813053539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bd5d55a00 msgr2=0x557bd5d54dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.351+0000 7fb0a37fe640 1 --2- 172.21.15.43:0/1813053539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bd5d55a00 0x557bd5d54dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.351+0000 7fb0a37fe640 1 -- 172.21.15.43:0/1813053539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd5d4ce80 msgr2=0x557bd5d48700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.351+0000 7fb0a37fe640 1 --2- 172.21.15.43:0/1813053539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd5d4ce80 0x557bd5d48700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.351+0000 7fb0a37fe640 1 -- 172.21.15.43:0/1813053539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bd5d47dd0 con 0x557bd5d4c4d0 2024-04-21T08:50:40.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.351+0000 7fb0a2ffd640 1 --2- 172.21.15.43:0/1813053539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd5d4ce80 0x557bd5d48700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:40.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.351+0000 7fb0a3fff640 1 --2- 172.21.15.43:0/1813053539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bd5d55a00 0x557bd5d54dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:40.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.351+0000 7fb0a37fe640 1 --2- 172.21.15.43:0/1813053539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d4c4d0 0x557bd5d617d0 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7fb09409c9d0 tx=0x7fb0940d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=105ea7992b200fab server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.352+0000 7fb0897fa640 1 -- 172.21.15.43:0/1813053539 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0940bb070 con 0x557bd5d4c4d0 2024-04-21T08:50:40.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.352+0000 7fb0897fa640 1 -- 172.21.15.43:0/1813053539 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb09400fa90 con 0x557bd5d4c4d0 2024-04-21T08:50:40.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.352+0000 7fb0897fa640 1 -- 172.21.15.43:0/1813053539 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb094098e80 con 0x557bd5d4c4d0 2024-04-21T08:50:40.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.352+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/1813053539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d4c4d0 msgr2=0x557bd5d617d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.352+0000 7fb0ac7b6740 1 --2- 172.21.15.43:0/1813053539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d4c4d0 0x557bd5d617d0 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7fb09409c9d0 tx=0x7fb0940d0040 comp rx=0 tx=0).stop 2024-04-21T08:50:40.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.352+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/1813053539 shutdown_connections 2024-04-21T08:50:40.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.352+0000 7fb0ac7b6740 1 --2- 172.21.15.43:0/1813053539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bd5d55a00 0x557bd5d54dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.352+0000 7fb0ac7b6740 1 --2- 172.21.15.43:0/1813053539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd5d4ce80 0x557bd5d48700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.352+0000 7fb0ac7b6740 1 --2- 172.21.15.43:0/1813053539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d4c4d0 0x557bd5d617d0 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.352+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/1813053539 >> 172.21.15.43:0/1813053539 conn(0x557bd5d46ba0 msgr2=0x557bd5d47ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.352+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/1813053539 shutdown_connections 2024-04-21T08:50:40.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.352+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/1813053539 wait complete. 2024-04-21T08:50:40.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.352+0000 7fb0ac7b6740 1 -- start start 2024-04-21T08:50:40.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.353+0000 7fb0ac7b6740 1 Processor -- start 2024-04-21T08:50:40.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.353+0000 7fb0ac7b6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bd5d4c4d0 0x557bd5d5ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.353+0000 7fb0ac7b6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd5d4ce80 0x557bd5d4ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.353+0000 7fb0ac7b6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d55a00 0x557bd5d4f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.353+0000 7fb0ac7b6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a41b08e0 con 0x557bd5d55a00 2024-04-21T08:50:40.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.353+0000 7fb0ac7b6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a41b0210 con 0x557bd5d4ce80 2024-04-21T08:50:40.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.353+0000 7fb0ac7b6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a41ad470 con 0x557bd5d4c4d0 2024-04-21T08:50:40.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.353+0000 7fb0ac7b6740 10 client.0 _ll_register_callbacks cb 0x7fb09c009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:40.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.353+0000 7fb0a3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d55a00 0x557bd5d4f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.353+0000 7fb0a3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d55a00 0x557bd5d4f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50018/0 (socket says 172.21.15.43:50018) 2024-04-21T08:50:40.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.353+0000 7fb0a3fff640 1 -- 172.21.15.43:0/2633081843 learned_addr learned my addr 172.21.15.43:0/2633081843 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.353+0000 7fb0a2ffd640 1 --2- 172.21.15.43:0/2633081843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd5d4ce80 0x557bd5d4ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.353+0000 7fb0a37fe640 1 --2- 172.21.15.43:0/2633081843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bd5d4c4d0 0x557bd5d5ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.353+0000 7fb0a3fff640 1 -- 172.21.15.43:0/2633081843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bd5d4c4d0 msgr2=0x557bd5d5ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.353+0000 7fb0a3fff640 1 --2- 172.21.15.43:0/2633081843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bd5d4c4d0 0x557bd5d5ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.353+0000 7fb0a3fff640 1 -- 172.21.15.43:0/2633081843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd5d4ce80 msgr2=0x557bd5d4ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.353+0000 7fb0a3fff640 1 --2- 172.21.15.43:0/2633081843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd5d4ce80 0x557bd5d4ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.353+0000 7fb0a3fff640 1 -- 172.21.15.43:0/2633081843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bd5d4f9d0 con 0x557bd5d55a00 2024-04-21T08:50:40.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.353+0000 7fb0a3fff640 1 --2- 172.21.15.43:0/2633081843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d55a00 0x557bd5d4f460 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7fb09c010e90 tx=0x7fb09c029660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd61747c3cead2c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.353+0000 7fb082ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:40.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.354+0000 7fb082ffd640 1 -- 172.21.15.43:0/2633081843 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c003d20 con 0x557bd5d55a00 2024-04-21T08:50:40.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.354+0000 7fb082ffd640 1 -- 172.21.15.43:0/2633081843 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fb09c003ec0 con 0x557bd5d55a00 2024-04-21T08:50:40.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.354+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/2633081843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb09400fa90 con 0x557bd5d55a00 2024-04-21T08:50:40.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.354+0000 7fb082ffd640 1 -- 172.21.15.43:0/2633081843 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb09c004ab0 con 0x557bd5d55a00 2024-04-21T08:50:40.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.354+0000 7fb0ac7b6740 10 client.21108 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:40.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.354+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/2633081843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557bd5dd0290 con 0x557bd5d55a00 2024-04-21T08:50:40.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.354+0000 7fb082ffd640 1 -- 172.21.15.43:0/2633081843 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb09400fa90 con 0x557bd5d55a00 2024-04-21T08:50:40.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.354+0000 7fb082ffd640 1 -- 172.21.15.43:0/2633081843 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb09c010570 con 0x557bd5d55a00 2024-04-21T08:50:40.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.354+0000 7fb0ac7b6740 10 client.21108 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:40.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.354+0000 7fb0ac7b6740 10 client.21108 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:50:40.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.354+0000 7fb0ac7b6740 20 client.21108 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:40.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.354+0000 7fb0ac7b6740 1 --2- 172.21.15.43:0/2633081843 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x557bd5dd09b0 0x557bd5dd2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.354+0000 7fb0ac7b6740 4 client.21108 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:50:40.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.354+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/2633081843 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557bd5cbec10 con 0x557bd5dd09b0 2024-04-21T08:50:40.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.354+0000 7fb0a37fe640 1 --2- 172.21.15.43:0/2633081843 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x557bd5dd09b0 0x557bd5dd2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.355+0000 7fb0a37fe640 1 --2- 172.21.15.43:0/2633081843 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x557bd5dd09b0 0x557bd5dd2dd0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=63f02e3c4b17c019 server_cookie=7558b504d047a30f in_seq=0 out_seq=0 2024-04-21T08:50:40.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.355+0000 7fb082ffd640 10 client.21108 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:50:40.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.355+0000 7fb082ffd640 1 -- 172.21.15.43:0/2633081843 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557bd5cbec10 con 0x557bd5dd09b0 2024-04-21T08:50:40.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.355+0000 7fb082ffd640 10 client.21108 handle_command_reply: tid=0 2024-04-21T08:50:40.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.355+0000 7fb0ac7b6740 1 client.21108 shutdown 2024-04-21T08:50:40.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.355+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/2633081843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d55a00 msgr2=0x557bd5d4f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.355+0000 7fb0ac7b6740 1 --2- 172.21.15.43:0/2633081843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d55a00 0x557bd5d4f460 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7fb09c010e90 tx=0x7fb09c029660 comp rx=0 tx=0).stop 2024-04-21T08:50:40.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.355+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/2633081843 shutdown_connections 2024-04-21T08:50:40.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.355+0000 7fb0ac7b6740 1 --2- 172.21.15.43:0/2633081843 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x557bd5dd09b0 0x557bd5dd2dd0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.355+0000 7fb0ac7b6740 1 --2- 172.21.15.43:0/2633081843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bd5d55a00 0x557bd5d4f460 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.355+0000 7fb0ac7b6740 1 --2- 172.21.15.43:0/2633081843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd5d4ce80 0x557bd5d4ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.355+0000 7fb082ffd640 0 client.21108 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:50:40.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.355+0000 7fb0ac7b6740 1 --2- 172.21.15.43:0/2633081843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bd5d4c4d0 0x557bd5d5ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.355+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/2633081843 >> 172.21.15.43:0/2633081843 conn(0x557bd5d46ba0 msgr2=0x557bd5d45c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.355+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/2633081843 shutdown_connections 2024-04-21T08:50:40.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.355+0000 7fb0ac7b6740 1 -- 172.21.15.43:0/2633081843 wait complete. 2024-04-21T08:50:40.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.355+0000 7fb0ac7b6740 20 client.21108 trim_cache size 0 max 16384 2024-04-21T08:50:40.363 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:50:40.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.356+0000 7fb0a9af1640 1 -- 172.21.15.43:0/711514135 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb08c0a3d20 msgr2=0x7fb08c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.356+0000 7fb0a9af1640 1 --2- 172.21.15.43:0/711514135 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb08c0a3d20 0x7fb08c0a61e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb09c009990 tx=0x7fb09c0096a0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.356+0000 7fb0a9af1640 1 -- 172.21.15.43:0/711514135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0a406a840 msgr2=0x7fb0a4198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.356+0000 7fb0a9af1640 1 --2- 172.21.15.43:0/711514135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0a406a840 0x7fb0a4198690 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7fb09400c920 tx=0x7fb09400cde0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.356+0000 7fb0a9af1640 1 -- 172.21.15.43:0/711514135 shutdown_connections 2024-04-21T08:50:40.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.356+0000 7fb0a9af1640 1 --2- 172.21.15.43:0/711514135 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb08c0a3d20 0x7fb08c0a61e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.356+0000 7fb0a9af1640 1 --2- 172.21.15.43:0/711514135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0a419cf50 0x7fb0a419d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.356+0000 7fb0a9af1640 1 --2- 172.21.15.43:0/711514135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0a4107270 0x7fb0a4198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.356+0000 7fb0a9af1640 1 --2- 172.21.15.43:0/711514135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0a406a840 0x7fb0a4198690 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.356+0000 7fb0a9af1640 1 -- 172.21.15.43:0/711514135 >> 172.21.15.43:0/711514135 conn(0x7fb0a4077eb0 msgr2=0x7fb0a4100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.356+0000 7fb0a9af1640 1 -- 172.21.15.43:0/711514135 shutdown_connections 2024-04-21T08:50:40.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.356+0000 7fb0a9af1640 1 -- 172.21.15.43:0/711514135 wait complete. 2024-04-21T08:50:40.367 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:50:40.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.511+0000 7f3b116ba640 1 -- 172.21.15.43:0/293334401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b0c10c510 msgr2=0x7f3b0c10e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.511+0000 7f3b116ba640 1 --2- 172.21.15.43:0/293334401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b0c10c510 0x7f3b0c10e9b0 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009ca0 tx=0x7f3b0402c230 comp rx=0 tx=0).stop 2024-04-21T08:50:40.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.511+0000 7f3b116ba640 1 -- 172.21.15.43:0/293334401 shutdown_connections 2024-04-21T08:50:40.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.511+0000 7f3b116ba640 1 --2- 172.21.15.43:0/293334401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b0c10c510 0x7f3b0c10e9b0 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.511+0000 7f3b116ba640 1 --2- 172.21.15.43:0/293334401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b0c103af0 0x7f3b0c10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.511+0000 7f3b116ba640 1 --2- 172.21.15.43:0/293334401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b0c1031d0 0x7f3b0c1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.512+0000 7f3b116ba640 1 -- 172.21.15.43:0/293334401 >> 172.21.15.43:0/293334401 conn(0x7f3b0c077f50 msgr2=0x7f3b0c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.512+0000 7f3b116ba640 1 -- 172.21.15.43:0/293334401 shutdown_connections 2024-04-21T08:50:40.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.512+0000 7f3b116ba640 1 -- 172.21.15.43:0/293334401 wait complete. 2024-04-21T08:50:40.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.512+0000 7f3b116ba640 1 Processor -- start 2024-04-21T08:50:40.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.512+0000 7f3b116ba640 1 -- start start 2024-04-21T08:50:40.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.512+0000 7f3b116ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b0c1031d0 0x7f3b0c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.512+0000 7f3b116ba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b0c103af0 0x7f3b0c198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.512+0000 7f3b116ba640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b0c10c510 0x7f3b0c19cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.512+0000 7f3b116ba640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c112180 con 0x7f3b0c103af0 2024-04-21T08:50:40.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.512+0000 7f3b116ba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c112000 con 0x7f3b0c1031d0 2024-04-21T08:50:40.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.512+0000 7f3b116ba640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c112300 con 0x7f3b0c10c510 2024-04-21T08:50:40.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.513+0000 7f3b0a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b0c103af0 0x7f3b0c198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.513+0000 7f3b0a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b0c103af0 0x7f3b0c198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50028/0 (socket says 172.21.15.43:50028) 2024-04-21T08:50:40.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.513+0000 7f3b0a7fc640 1 -- 172.21.15.43:0/2147445270 learned_addr learned my addr 172.21.15.43:0/2147445270 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.513+0000 7f3b0affd640 1 --2- 172.21.15.43:0/2147445270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b0c1031d0 0x7f3b0c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.513+0000 7f3b0b7fe640 1 --2- 172.21.15.43:0/2147445270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b0c10c510 0x7f3b0c19cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.513+0000 7f3b0a7fc640 1 -- 172.21.15.43:0/2147445270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b0c10c510 msgr2=0x7f3b0c19cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.513+0000 7f3b0a7fc640 1 --2- 172.21.15.43:0/2147445270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b0c10c510 0x7f3b0c19cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.513+0000 7f3b0a7fc640 1 -- 172.21.15.43:0/2147445270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b0c1031d0 msgr2=0x7f3b0c198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.513+0000 7f3b0a7fc640 1 --2- 172.21.15.43:0/2147445270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b0c1031d0 0x7f3b0c198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.513+0000 7f3b0a7fc640 1 -- 172.21.15.43:0/2147445270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0c19d700 con 0x7f3b0c103af0 2024-04-21T08:50:40.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.513+0000 7f3b0affd640 1 --2- 172.21.15.43:0/2147445270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b0c1031d0 0x7f3b0c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:40.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.513+0000 7f3b0b7fe640 1 --2- 172.21.15.43:0/2147445270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b0c10c510 0x7f3b0c19cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:40.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.513+0000 7f3b0a7fc640 1 --2- 172.21.15.43:0/2147445270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b0c103af0 0x7f3b0c198ca0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000d950 tx=0x7f3b0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.513+0000 7f3af3fff640 1 -- 172.21.15.43:0/2147445270 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b00017070 con 0x7f3b0c103af0 2024-04-21T08:50:40.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.513+0000 7f3af3fff640 1 -- 172.21.15.43:0/2147445270 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b00009c30 con 0x7f3b0c103af0 2024-04-21T08:50:40.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.513+0000 7f3b116ba640 1 -- 172.21.15.43:0/2147445270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b0c19d9f0 con 0x7f3b0c103af0 2024-04-21T08:50:40.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.513+0000 7f3af3fff640 1 -- 172.21.15.43:0/2147445270 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0000f620 con 0x7f3b0c103af0 2024-04-21T08:50:40.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.513+0000 7f3b116ba640 1 -- 172.21.15.43:0/2147445270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b0c1a5280 con 0x7f3b0c103af0 2024-04-21T08:50:40.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.515+0000 7f3af3fff640 1 -- 172.21.15.43:0/2147445270 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3b00010460 con 0x7f3b0c103af0 2024-04-21T08:50:40.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.516+0000 7f3b14385740 1 Processor -- start 2024-04-21T08:50:40.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.516+0000 7f3b14385740 1 -- start start 2024-04-21T08:50:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.516+0000 7f3b14385740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b939ff84d0 0x55b939ff88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.516+0000 7f3b14385740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b939ff8e80 0x55b93a001430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.516+0000 7f3b14385740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b93a001a00 0x55b93a003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.516+0000 7f3b14385740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b939f5d1f0 con 0x55b939ff84d0 2024-04-21T08:50:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.516+0000 7f3b14385740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b939efa4b0 con 0x55b93a001a00 2024-04-21T08:50:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.516+0000 7f3b14385740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b939f591b0 con 0x55b939ff8e80 2024-04-21T08:50:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.516+0000 7f3b0b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b939ff84d0 0x55b939ff88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.516+0000 7f3af3fff640 1 --2- 172.21.15.43:0/2147445270 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3af40a3d20 0x7f3af40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.516+0000 7f3b0b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b939ff84d0 0x55b939ff88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50030/0 (socket says 172.21.15.43:50030) 2024-04-21T08:50:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.516+0000 7f3b0b7fe640 1 -- 172.21.15.43:0/1535598497 learned_addr learned my addr 172.21.15.43:0/1535598497 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.516+0000 7f3b0a7fc640 1 --2- 172.21.15.43:0/1535598497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b93a001a00 0x55b93a003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.516+0000 7f3b0affd640 1 --2- 172.21.15.43:0/1535598497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b939ff8e80 0x55b93a001430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.516+0000 7f3b0b7fe640 1 --2- 172.21.15.43:0/2147445270 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3af40a3d20 0x7f3af40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.516+0000 7f3af3fff640 1 -- 172.21.15.43:0/2147445270 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f3b000d08e0 con 0x7f3b0c103af0 2024-04-21T08:50:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.516+0000 7f3b0b7fe640 1 -- 172.21.15.43:0/1535598497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b939ff8e80 msgr2=0x55b93a001430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.516+0000 7f3b0b7fe640 1 --2- 172.21.15.43:0/1535598497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b939ff8e80 0x55b93a001430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.516+0000 7f3b0b7fe640 1 -- 172.21.15.43:0/1535598497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b93a001a00 msgr2=0x55b93a003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.516+0000 7f3b0b7fe640 1 --2- 172.21.15.43:0/1535598497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b93a001a00 0x55b93a003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.516+0000 7f3b0b7fe640 1 -- 172.21.15.43:0/1535598497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b939eec1b0 con 0x55b939ff84d0 2024-04-21T08:50:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.516+0000 7f3b0affd640 1 --2- 172.21.15.43:0/1535598497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b939ff8e80 0x55b93a001430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.517+0000 7f3b0b7fe640 1 --2- 172.21.15.43:0/1535598497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b939ff84d0 0x55b939ff88b0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009430 tx=0x7f3b04003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91204eec7bbc3645 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.517+0000 7f3af17fa640 1 -- 172.21.15.43:0/1535598497 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b040046d0 con 0x55b939ff84d0 2024-04-21T08:50:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.517+0000 7f3af17fa640 1 -- 172.21.15.43:0/1535598497 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b04032af0 con 0x55b939ff84d0 2024-04-21T08:50:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.517+0000 7f3af17fa640 1 -- 172.21.15.43:0/1535598497 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04039070 con 0x55b939ff84d0 2024-04-21T08:50:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.518+0000 7f3b14385740 1 -- 172.21.15.43:0/1535598497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b939ff84d0 msgr2=0x55b939ff88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.518+0000 7f3b14385740 1 --2- 172.21.15.43:0/1535598497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b939ff84d0 0x55b939ff88b0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009430 tx=0x7f3b04003b70 comp rx=0 tx=0).stop 2024-04-21T08:50:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.518+0000 7f3b0b7fe640 1 --2- 172.21.15.43:0/2147445270 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3af40a3d20 0x7f3af40a61e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3b0404c000 tx=0x7f3b04048000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.518+0000 7f3b14385740 1 -- 172.21.15.43:0/1535598497 shutdown_connections 2024-04-21T08:50:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.518+0000 7f3b14385740 1 --2- 172.21.15.43:0/1535598497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b93a001a00 0x55b93a003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.518+0000 7f3b14385740 1 --2- 172.21.15.43:0/1535598497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b939ff8e80 0x55b93a001430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.518+0000 7f3b14385740 1 --2- 172.21.15.43:0/1535598497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b939ff84d0 0x55b939ff88b0 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.518+0000 7f3b14385740 1 -- 172.21.15.43:0/1535598497 >> 172.21.15.43:0/1535598497 conn(0x55b939ff2ba0 msgr2=0x55b939ff2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.518+0000 7f3b14385740 1 -- 172.21.15.43:0/1535598497 shutdown_connections 2024-04-21T08:50:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.518+0000 7f3b14385740 1 -- 172.21.15.43:0/1535598497 wait complete. 2024-04-21T08:50:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.518+0000 7f3b14385740 1 -- start start 2024-04-21T08:50:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.518+0000 7f3b14385740 1 Processor -- start 2024-04-21T08:50:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.518+0000 7f3b14385740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b939ff84d0 0x55b93a001360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.518+0000 7f3b14385740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b939ff8e80 0x55b939ffe6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.518+0000 7f3b14385740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b93a001a00 0x55b939ffed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.518+0000 7f3b14385740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b93a004970 con 0x55b93a001a00 2024-04-21T08:50:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.518+0000 7f3b14385740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b939f16500 con 0x55b939ff84d0 2024-04-21T08:50:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.518+0000 7f3b14385740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b93a004af0 con 0x55b939ff8e80 2024-04-21T08:50:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.518+0000 7f3b14385740 10 client.0 _ll_register_callbacks cb 0x55b939ee4b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.518+0000 7f3b0b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b93a001a00 0x55b939ffed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.518+0000 7f3b0b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b93a001a00 0x55b939ffed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50044/0 (socket says 172.21.15.43:50044) 2024-04-21T08:50:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.518+0000 7f3b0b7fe640 1 -- 172.21.15.43:0/2051780503 learned_addr learned my addr 172.21.15.43:0/2051780503 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.519+0000 7f3b0a7fc640 1 --2- 172.21.15.43:0/2051780503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b939ff8e80 0x55b939ffe6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.519+0000 7f3b0b7fe640 1 -- 172.21.15.43:0/2051780503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b939ff8e80 msgr2=0x55b939ffe6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.519+0000 7f3b0b7fe640 1 --2- 172.21.15.43:0/2051780503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b939ff8e80 0x55b939ffe6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.519+0000 7f3b0affd640 1 --2- 172.21.15.43:0/2051780503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b939ff84d0 0x55b93a001360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.519+0000 7f3b0b7fe640 1 -- 172.21.15.43:0/2051780503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b939ff84d0 msgr2=0x55b93a001360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.519+0000 7f3b0b7fe640 1 --2- 172.21.15.43:0/2051780503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b939ff84d0 0x55b93a001360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.519+0000 7f3b0b7fe640 1 -- 172.21.15.43:0/2051780503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b939c6f3f0 con 0x55b93a001a00 2024-04-21T08:50:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.519+0000 7f3b0b7fe640 1 --2- 172.21.15.43:0/2051780503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b93a001a00 0x55b939ffed00 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f3b04005fe0 tx=0x7f3b04003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400a26493487b463 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.519+0000 7f3aeeff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.519+0000 7f3aeeff5640 1 -- 172.21.15.43:0/2051780503 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0405a060 con 0x55b93a001a00 2024-04-21T08:50:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.519+0000 7f3aeeff5640 1 -- 172.21.15.43:0/2051780503 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f3b040044d0 con 0x55b93a001a00 2024-04-21T08:50:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.519+0000 7f3b14385740 1 -- 172.21.15.43:0/2051780503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b04032af0 con 0x55b93a001a00 2024-04-21T08:50:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.519+0000 7f3aeeff5640 1 -- 172.21.15.43:0/2051780503 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3b04061050 con 0x55b93a001a00 2024-04-21T08:50:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.519+0000 7f3b14385740 10 client.21120 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.519+0000 7f3b14385740 1 -- 172.21.15.43:0/2051780503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b939d397d0 con 0x55b93a001a00 2024-04-21T08:50:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.519+0000 7f3aeeff5640 1 -- 172.21.15.43:0/2051780503 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b04032af0 con 0x55b93a001a00 2024-04-21T08:50:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.520+0000 7f3aeeff5640 1 -- 172.21.15.43:0/2051780503 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3b040434a0 con 0x55b93a001a00 2024-04-21T08:50:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.520+0000 7f3b14385740 10 client.21120 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.520+0000 7f3b14385740 10 client.21120 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:50:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.520+0000 7f3b14385740 20 client.21120 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.520+0000 7f3b14385740 1 --2- 172.21.15.43:0/2051780503 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55b93a011b10 0x55b93a013f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.520+0000 7f3b14385740 4 client.21120 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.520+0000 7f3b14385740 1 -- 172.21.15.43:0/2051780503 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b939efc930 con 0x55b93a011b10 2024-04-21T08:50:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.520+0000 7f3b0affd640 1 --2- 172.21.15.43:0/2051780503 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55b93a011b10 0x55b93a013f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.520+0000 7f3aeeff5640 10 client.21120 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:50:40.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.520+0000 7f3b0affd640 1 --2- 172.21.15.43:0/2051780503 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55b93a011b10 0x55b93a013f00 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cead9ea71662b5c3 server_cookie=472e1aa72af878e2 in_seq=0 out_seq=0 2024-04-21T08:50:40.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.521+0000 7f3aeeff5640 1 -- 172.21.15.43:0/2051780503 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b939efc930 con 0x55b93a011b10 2024-04-21T08:50:40.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.521+0000 7f3aeeff5640 10 client.21120 handle_command_reply: tid=0 2024-04-21T08:50:40.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.521+0000 7f3b14385740 1 client.21120 shutdown 2024-04-21T08:50:40.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.521+0000 7f3b14385740 1 -- 172.21.15.43:0/2051780503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b93a001a00 msgr2=0x55b939ffed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.521+0000 7f3b14385740 1 --2- 172.21.15.43:0/2051780503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b93a001a00 0x55b939ffed00 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f3b04005fe0 tx=0x7f3b04003c60 comp rx=0 tx=0).stop 2024-04-21T08:50:40.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.521+0000 7f3b14385740 1 -- 172.21.15.43:0/2051780503 shutdown_connections 2024-04-21T08:50:40.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.521+0000 7f3b14385740 1 --2- 172.21.15.43:0/2051780503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b93a001a00 0x55b939ffed00 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.521+0000 7f3b14385740 1 --2- 172.21.15.43:0/2051780503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b939ff8e80 0x55b939ffe6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.521+0000 7f3b14385740 1 --2- 172.21.15.43:0/2051780503 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55b93a011b10 0x55b93a013f00 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.521+0000 7f3b14385740 1 --2- 172.21.15.43:0/2051780503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b939ff84d0 0x55b93a001360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.521+0000 7f3b14385740 1 -- 172.21.15.43:0/2051780503 >> 172.21.15.43:0/2051780503 conn(0x55b939ff2ba0 msgr2=0x55b939ff48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.521+0000 7f3aeeff5640 0 client.21120 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:50:40.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.521+0000 7f3b14385740 1 -- 172.21.15.43:0/2051780503 shutdown_connections 2024-04-21T08:50:40.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.521+0000 7f3b14385740 1 -- 172.21.15.43:0/2051780503 wait complete. 2024-04-21T08:50:40.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.522+0000 7f3b14385740 20 client.21120 trim_cache size 0 max 16384 2024-04-21T08:50:40.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.535+0000 7f3b14385740 1 Processor -- start 2024-04-21T08:50:40.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.535+0000 7f3b14385740 1 -- start start 2024-04-21T08:50:40.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.535+0000 7f3b14385740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b939ff84d0 0x55b93a00d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.535+0000 7f3b14385740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b939ff8e80 0x55b93a000e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.535+0000 7f3b14385740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b93a001a00 0x55b93a001360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.535+0000 7f3b14385740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c1af5a0 con 0x55b939ff8e80 2024-04-21T08:50:40.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.535+0000 7f3b14385740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c1b10e0 con 0x55b93a001a00 2024-04-21T08:50:40.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.535+0000 7f3b14385740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c1afc70 con 0x55b939ff84d0 2024-04-21T08:50:40.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.535+0000 7f3b0a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b939ff8e80 0x55b93a000e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.535+0000 7f3b0a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b939ff8e80 0x55b93a000e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50052/0 (socket says 172.21.15.43:50052) 2024-04-21T08:50:40.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.535+0000 7f3b0a7fc640 1 -- 172.21.15.43:0/2778686017 learned_addr learned my addr 172.21.15.43:0/2778686017 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.535+0000 7f3b0affd640 1 --2- 172.21.15.43:0/2778686017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b939ff84d0 0x55b93a00d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.535+0000 7f3b0b7fe640 1 --2- 172.21.15.43:0/2778686017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b93a001a00 0x55b93a001360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.535+0000 7f3b0a7fc640 1 -- 172.21.15.43:0/2778686017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b939ff84d0 msgr2=0x55b93a00d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.535+0000 7f3b0a7fc640 1 --2- 172.21.15.43:0/2778686017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b939ff84d0 0x55b93a00d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.535+0000 7f3b0a7fc640 1 -- 172.21.15.43:0/2778686017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b93a001a00 msgr2=0x55b93a001360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.535+0000 7f3b0a7fc640 1 --2- 172.21.15.43:0/2778686017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b93a001a00 0x55b93a001360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.535+0000 7f3b0a7fc640 1 -- 172.21.15.43:0/2778686017 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b939ff8c20 con 0x55b939ff8e80 2024-04-21T08:50:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.535+0000 7f3b0b7fe640 1 --2- 172.21.15.43:0/2778686017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b93a001a00 0x55b93a001360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.535+0000 7f3b0a7fc640 1 --2- 172.21.15.43:0/2778686017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b939ff8e80 0x55b93a000e20 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f3b000a4b40 tx=0x7f3b00010f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6a465924b88f88 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.535+0000 7f3af0ff9640 1 -- 172.21.15.43:0/2778686017 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b00020040 con 0x55b939ff8e80 2024-04-21T08:50:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.536+0000 7f3af0ff9640 1 -- 172.21.15.43:0/2778686017 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b0009c400 con 0x55b939ff8e80 2024-04-21T08:50:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.536+0000 7f3af0ff9640 1 -- 172.21.15.43:0/2778686017 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b00098c00 con 0x55b939ff8e80 2024-04-21T08:50:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.536+0000 7f3b14385740 1 -- 172.21.15.43:0/2778686017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b939ff8e80 msgr2=0x55b93a000e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.536+0000 7f3b14385740 1 --2- 172.21.15.43:0/2778686017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b939ff8e80 0x55b93a000e20 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f3b000a4b40 tx=0x7f3b00010f80 comp rx=0 tx=0).stop 2024-04-21T08:50:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.536+0000 7f3b14385740 1 -- 172.21.15.43:0/2778686017 shutdown_connections 2024-04-21T08:50:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.536+0000 7f3b14385740 1 --2- 172.21.15.43:0/2778686017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b93a001a00 0x55b93a001360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.536+0000 7f3b14385740 1 --2- 172.21.15.43:0/2778686017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b939ff8e80 0x55b93a000e20 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.536+0000 7f3b14385740 1 --2- 172.21.15.43:0/2778686017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b939ff84d0 0x55b93a00d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.536+0000 7f3b14385740 1 -- 172.21.15.43:0/2778686017 >> 172.21.15.43:0/2778686017 conn(0x55b939ff2ba0 msgr2=0x55b939ffd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.536+0000 7f3b14385740 1 -- 172.21.15.43:0/2778686017 shutdown_connections 2024-04-21T08:50:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.536+0000 7f3b14385740 1 -- 172.21.15.43:0/2778686017 wait complete. 2024-04-21T08:50:40.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.536+0000 7f3b14385740 1 -- start start 2024-04-21T08:50:40.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.536+0000 7f3b14385740 1 Processor -- start 2024-04-21T08:50:40.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.537+0000 7f3b14385740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b939ff84d0 0x55b93a014b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.537+0000 7f3b14385740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b939ff8e80 0x55b93a015050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.537+0000 7f3b14385740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b93a001a00 0x55b93a015590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.537+0000 7f3b14385740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c1b0a10 con 0x55b93a001a00 2024-04-21T08:50:40.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.537+0000 7f3b14385740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c1b0340 con 0x55b939ff8e80 2024-04-21T08:50:40.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.537+0000 7f3b14385740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c1ad5a0 con 0x55b939ff84d0 2024-04-21T08:50:40.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.537+0000 7f3b14385740 10 client.0 _ll_register_callbacks cb 0x7f3b04004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:40.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.537+0000 7f3b0b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b93a001a00 0x55b93a015590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.537+0000 7f3b0b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b93a001a00 0x55b93a015590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50058/0 (socket says 172.21.15.43:50058) 2024-04-21T08:50:40.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.537+0000 7f3b0b7fe640 1 -- 172.21.15.43:0/1905467844 learned_addr learned my addr 172.21.15.43:0/1905467844 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.537+0000 7f3b0affd640 1 --2- 172.21.15.43:0/1905467844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b939ff84d0 0x55b93a014b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.537+0000 7f3b0a7fc640 1 --2- 172.21.15.43:0/1905467844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b939ff8e80 0x55b93a015050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.537+0000 7f3b0b7fe640 1 -- 172.21.15.43:0/1905467844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b939ff84d0 msgr2=0x55b93a014b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.537+0000 7f3b0b7fe640 1 --2- 172.21.15.43:0/1905467844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b939ff84d0 0x55b93a014b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.537+0000 7f3b0b7fe640 1 -- 172.21.15.43:0/1905467844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b939ff8e80 msgr2=0x55b93a015050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.537+0000 7f3b0b7fe640 1 --2- 172.21.15.43:0/1905467844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b939ff8e80 0x55b93a015050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.537+0000 7f3b0b7fe640 1 -- 172.21.15.43:0/1905467844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b93a07b2d0 con 0x55b93a001a00 2024-04-21T08:50:40.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.537+0000 7f3b0a7fc640 1 --2- 172.21.15.43:0/1905467844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b939ff8e80 0x55b93a015050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:40.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.537+0000 7f3b0b7fe640 1 --2- 172.21.15.43:0/1905467844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b93a001a00 0x55b93a015590 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f3b0402c810 tx=0x7f3b04043d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4516ee10b4a9a50 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.537+0000 7f3aeeff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:40.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.537+0000 7f3aeeff5640 1 -- 172.21.15.43:0/1905467844 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04039040 con 0x55b93a001a00 2024-04-21T08:50:40.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.537+0000 7f3aeeff5640 1 -- 172.21.15.43:0/1905467844 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f3b04032850 con 0x55b93a001a00 2024-04-21T08:50:40.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.537+0000 7f3b14385740 1 -- 172.21.15.43:0/1905467844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b0009c400 con 0x55b93a001a00 2024-04-21T08:50:40.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.538+0000 7f3aeeff5640 1 -- 172.21.15.43:0/1905467844 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3b0405a030 con 0x55b93a001a00 2024-04-21T08:50:40.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.538+0000 7f3b14385740 10 client.21126 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:40.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.538+0000 7f3b14385740 1 -- 172.21.15.43:0/1905467844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b93a07c500 con 0x55b93a001a00 2024-04-21T08:50:40.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.538+0000 7f3aeeff5640 1 -- 172.21.15.43:0/1905467844 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b0009c400 con 0x55b93a001a00 2024-04-21T08:50:40.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.538+0000 7f3aeeff5640 1 -- 172.21.15.43:0/1905467844 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3b040348d0 con 0x55b93a001a00 2024-04-21T08:50:40.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.538+0000 7f3b14385740 10 client.21126 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:40.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.538+0000 7f3b14385740 10 client.21126 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:50:40.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.538+0000 7f3b14385740 20 client.21126 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:40.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.538+0000 7f3b14385740 1 --2- 172.21.15.43:0/1905467844 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55b93a07ca80 0x55b93a07ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.538+0000 7f3b14385740 4 client.21126 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:50:40.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.538+0000 7f3b14385740 1 -- 172.21.15.43:0/1905467844 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b939f6ac10 con 0x55b93a07ca80 2024-04-21T08:50:40.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.538+0000 7f3b0affd640 1 --2- 172.21.15.43:0/1905467844 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55b93a07ca80 0x55b93a07ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.539+0000 7f3aeeff5640 10 client.21126 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:50:40.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.539+0000 7f3b0affd640 1 --2- 172.21.15.43:0/1905467844 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55b93a07ca80 0x55b93a07ee70 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=460b7a799efee211 server_cookie=7495dd45638e466c in_seq=0 out_seq=0 2024-04-21T08:50:40.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.539+0000 7f3aeeff5640 1 -- 172.21.15.43:0/1905467844 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b939f6ac10 con 0x55b93a07ca80 2024-04-21T08:50:40.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.539+0000 7f3aeeff5640 10 client.21126 handle_command_reply: tid=0 2024-04-21T08:50:40.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.539+0000 7f3b14385740 1 client.21126 shutdown 2024-04-21T08:50:40.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.539+0000 7f3b14385740 1 -- 172.21.15.43:0/1905467844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b93a001a00 msgr2=0x55b93a015590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.539+0000 7f3b14385740 1 --2- 172.21.15.43:0/1905467844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b93a001a00 0x55b93a015590 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f3b0402c810 tx=0x7f3b04043d40 comp rx=0 tx=0).stop 2024-04-21T08:50:40.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.539+0000 7f3b14385740 1 -- 172.21.15.43:0/1905467844 shutdown_connections 2024-04-21T08:50:40.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.539+0000 7f3b14385740 1 --2- 172.21.15.43:0/1905467844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b93a001a00 0x55b93a015590 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.539+0000 7f3b14385740 1 --2- 172.21.15.43:0/1905467844 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55b93a07ca80 0x55b93a07ee70 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.539+0000 7f3b14385740 1 --2- 172.21.15.43:0/1905467844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b939ff8e80 0x55b93a015050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.539+0000 7f3b14385740 1 --2- 172.21.15.43:0/1905467844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b939ff84d0 0x55b93a014b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.539+0000 7f3aeeff5640 0 client.21126 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:50:40.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.539+0000 7f3b14385740 1 -- 172.21.15.43:0/1905467844 >> 172.21.15.43:0/1905467844 conn(0x55b939ff2ba0 msgr2=0x55b939ff1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.539+0000 7f3b14385740 1 -- 172.21.15.43:0/1905467844 shutdown_connections 2024-04-21T08:50:40.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.539+0000 7f3b14385740 1 -- 172.21.15.43:0/1905467844 wait complete. 2024-04-21T08:50:40.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.539+0000 7f3b14385740 20 client.21126 trim_cache size 0 max 16384 2024-04-21T08:50:40.547 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:50:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.540+0000 7f3b116ba640 1 -- 172.21.15.43:0/2147445270 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3af40a3d20 msgr2=0x7f3af40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.540+0000 7f3b116ba640 1 --2- 172.21.15.43:0/2147445270 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3af40a3d20 0x7f3af40a61e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3b0404c000 tx=0x7f3b04048000 comp rx=0 tx=0).stop 2024-04-21T08:50:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.540+0000 7f3b116ba640 1 -- 172.21.15.43:0/2147445270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b0c103af0 msgr2=0x7f3b0c198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.540+0000 7f3b116ba640 1 --2- 172.21.15.43:0/2147445270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b0c103af0 0x7f3b0c198ca0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000d950 tx=0x7f3b0000de10 comp rx=0 tx=0).stop 2024-04-21T08:50:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.540+0000 7f3b116ba640 1 -- 172.21.15.43:0/2147445270 shutdown_connections 2024-04-21T08:50:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.540+0000 7f3b116ba640 1 --2- 172.21.15.43:0/2147445270 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3af40a3d20 0x7f3af40a61e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.540+0000 7f3b116ba640 1 --2- 172.21.15.43:0/2147445270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b0c10c510 0x7f3b0c19cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.540+0000 7f3b116ba640 1 --2- 172.21.15.43:0/2147445270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b0c103af0 0x7f3b0c198ca0 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.540+0000 7f3b116ba640 1 --2- 172.21.15.43:0/2147445270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b0c1031d0 0x7f3b0c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.540+0000 7f3b116ba640 1 -- 172.21.15.43:0/2147445270 >> 172.21.15.43:0/2147445270 conn(0x7f3b0c077f50 msgr2=0x7f3b0c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.540+0000 7f3b116ba640 1 -- 172.21.15.43:0/2147445270 shutdown_connections 2024-04-21T08:50:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.540+0000 7f3b116ba640 1 -- 172.21.15.43:0/2147445270 wait complete. 2024-04-21T08:50:40.551 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:50:40.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:40 smithi043 ceph-mon[27978]: pgmap v366: 129 pgs: 129 active+clean; 4.9 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:50:40.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:40 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3123942233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:50:40.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.655+0000 7fceb4248640 1 -- 172.21.15.43:0/3637089164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceac107260 msgr2=0x7fceac10ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.655+0000 7fceb4248640 1 --2- 172.21.15.43:0/3637089164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceac107260 0x7fceac10ee30 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7fcea4009ca0 tx=0x7fcea402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.655+0000 7fceb4248640 1 -- 172.21.15.43:0/3637089164 shutdown_connections 2024-04-21T08:50:40.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.655+0000 7fceb4248640 1 --2- 172.21.15.43:0/3637089164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceac107260 0x7fceac10ee30 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.655+0000 7fceb4248640 1 --2- 172.21.15.43:0/3637089164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceac102b40 0x7fceac106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.655+0000 7fceb4248640 1 --2- 172.21.15.43:0/3637089164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceac102190 0x7fceac102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.655+0000 7fceb4248640 1 -- 172.21.15.43:0/3637089164 >> 172.21.15.43:0/3637089164 conn(0x7fceac0793a0 msgr2=0x7fceac0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.655+0000 7fceb4248640 1 -- 172.21.15.43:0/3637089164 shutdown_connections 2024-04-21T08:50:40.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.655+0000 7fceb4248640 1 -- 172.21.15.43:0/3637089164 wait complete. 2024-04-21T08:50:40.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.656+0000 7fceb4248640 1 Processor -- start 2024-04-21T08:50:40.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.656+0000 7fceb4248640 1 -- start start 2024-04-21T08:50:40.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.656+0000 7fceb4248640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceac102190 0x7fceac074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.656+0000 7fceb4248640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceac107260 0x7fceac074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.656+0000 7fceb4248640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceac074cd0 0x7fceac1a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.656+0000 7fceb4248640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fceac1121b0 con 0x7fceac074cd0 2024-04-21T08:50:40.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.656+0000 7fceb4248640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fceac112030 con 0x7fceac102190 2024-04-21T08:50:40.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.656+0000 7fceb4248640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fceac112330 con 0x7fceac107260 2024-04-21T08:50:40.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.656+0000 7fceb27e5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceac074cd0 0x7fceac1a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.657+0000 7fceb27e5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceac074cd0 0x7fceac1a68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50080/0 (socket says 172.21.15.43:50080) 2024-04-21T08:50:40.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.657+0000 7fceb27e5640 1 -- 172.21.15.43:0/1227161218 learned_addr learned my addr 172.21.15.43:0/1227161218 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.657+0000 7fceb1fe4640 1 --2- 172.21.15.43:0/1227161218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceac102190 0x7fceac074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.657+0000 7fceb17e3640 1 --2- 172.21.15.43:0/1227161218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceac107260 0x7fceac074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.657+0000 7fceb27e5640 1 -- 172.21.15.43:0/1227161218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceac107260 msgr2=0x7fceac074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.657+0000 7fceb27e5640 1 --2- 172.21.15.43:0/1227161218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceac107260 0x7fceac074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.657+0000 7fceb27e5640 1 -- 172.21.15.43:0/1227161218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceac102190 msgr2=0x7fceac074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.657+0000 7fceb27e5640 1 --2- 172.21.15.43:0/1227161218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceac102190 0x7fceac074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.657+0000 7fceb27e5640 1 -- 172.21.15.43:0/1227161218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceac1a6f60 con 0x7fceac074cd0 2024-04-21T08:50:40.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.657+0000 7fceb27e5640 1 --2- 172.21.15.43:0/1227161218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceac074cd0 0x7fceac1a68c0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7fcea40098d0 tx=0x7fcea4035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.657+0000 7fcea2ffd640 1 -- 172.21.15.43:0/1227161218 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4039040 con 0x7fceac074cd0 2024-04-21T08:50:40.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.657+0000 7fceb4248640 1 -- 172.21.15.43:0/1227161218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceac1a71f0 con 0x7fceac074cd0 2024-04-21T08:50:40.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.657+0000 7fcea2ffd640 1 -- 172.21.15.43:0/1227161218 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcea4044070 con 0x7fceac074cd0 2024-04-21T08:50:40.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.657+0000 7fcea2ffd640 1 -- 172.21.15.43:0/1227161218 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea40042a0 con 0x7fceac074cd0 2024-04-21T08:50:40.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.657+0000 7fceb4248640 1 -- 172.21.15.43:0/1227161218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceac1a76e0 con 0x7fceac074cd0 2024-04-21T08:50:40.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.659+0000 7fcea2ffd640 1 -- 172.21.15.43:0/1227161218 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcea4004440 con 0x7fceac074cd0 2024-04-21T08:50:40.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.660+0000 7fceb6e30740 1 Processor -- start 2024-04-21T08:50:40.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.660+0000 7fceb6e30740 1 -- start start 2024-04-21T08:50:40.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.660+0000 7fceb6e30740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016e5834d0 0x56016e5838b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.660+0000 7fceb6e30740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016e583e80 0x56016e58c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.660+0000 7fceb6e30740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e58ca00 0x56016e58ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.660+0000 7fceb6e30740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56016e4e81f0 con 0x56016e58ca00 2024-04-21T08:50:40.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.660+0000 7fceb6e30740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56016e4854b0 con 0x56016e5834d0 2024-04-21T08:50:40.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.660+0000 7fceb6e30740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56016e4e41b0 con 0x56016e583e80 2024-04-21T08:50:40.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.660+0000 7fcea2ffd640 1 --2- 172.21.15.43:0/1227161218 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fce980a3d20 0x7fce980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.660+0000 7fceb17e3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e58ca00 0x56016e58ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.660+0000 7fceb17e3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e58ca00 0x56016e58ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50096/0 (socket says 172.21.15.43:50096) 2024-04-21T08:50:40.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.660+0000 7fceb17e3640 1 -- 172.21.15.43:0/826771338 learned_addr learned my addr 172.21.15.43:0/826771338 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.660+0000 7fceb27e5640 1 --2- 172.21.15.43:0/1227161218 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fce980a3d20 0x7fce980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.660+0000 7fceb17e3640 1 --2- 172.21.15.43:0/826771338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016e5834d0 0x56016e5838b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.660+0000 7fceb1fe4640 1 --2- 172.21.15.43:0/826771338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016e583e80 0x56016e58c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.660+0000 7fcea2ffd640 1 -- 172.21.15.43:0/1227161218 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fcea40f3090 con 0x7fceac074cd0 2024-04-21T08:50:40.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.661+0000 7fceb17e3640 1 -- 172.21.15.43:0/826771338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016e583e80 msgr2=0x56016e58c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.661+0000 7fceb17e3640 1 --2- 172.21.15.43:0/826771338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016e583e80 0x56016e58c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.661+0000 7fceb17e3640 1 -- 172.21.15.43:0/826771338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016e5834d0 msgr2=0x56016e5838b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.661+0000 7fceb17e3640 1 --2- 172.21.15.43:0/826771338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016e5834d0 0x56016e5838b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.661+0000 7fceb17e3640 1 -- 172.21.15.43:0/826771338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56016e4771b0 con 0x56016e58ca00 2024-04-21T08:50:40.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.661+0000 7fceb17e3640 1 --2- 172.21.15.43:0/826771338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e58ca00 0x56016e58ee50 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7fce9c00efd0 tx=0x7fce9c00e5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8919b173cb0485 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.661+0000 7fceb27e5640 1 --2- 172.21.15.43:0/1227161218 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fce980a3d20 0x7fce980a61e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcea40be9c0 tx=0x7fcea4041040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.661+0000 7fce97fff640 1 -- 172.21.15.43:0/826771338 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c018070 con 0x56016e58ca00 2024-04-21T08:50:40.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.661+0000 7fce97fff640 1 -- 172.21.15.43:0/826771338 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce9c0054d0 con 0x56016e58ca00 2024-04-21T08:50:40.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.661+0000 7fce97fff640 1 -- 172.21.15.43:0/826771338 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c004ca0 con 0x56016e58ca00 2024-04-21T08:50:40.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.662+0000 7fceb6e30740 1 -- 172.21.15.43:0/826771338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e58ca00 msgr2=0x56016e58ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.662+0000 7fceb6e30740 1 --2- 172.21.15.43:0/826771338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e58ca00 0x56016e58ee50 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7fce9c00efd0 tx=0x7fce9c00e5b0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.662+0000 7fceb6e30740 1 -- 172.21.15.43:0/826771338 shutdown_connections 2024-04-21T08:50:40.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.662+0000 7fceb6e30740 1 --2- 172.21.15.43:0/826771338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e58ca00 0x56016e58ee50 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.662+0000 7fceb6e30740 1 --2- 172.21.15.43:0/826771338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016e583e80 0x56016e58c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.662+0000 7fceb6e30740 1 --2- 172.21.15.43:0/826771338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016e5834d0 0x56016e5838b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.662+0000 7fceb6e30740 1 -- 172.21.15.43:0/826771338 >> 172.21.15.43:0/826771338 conn(0x56016e57dba0 msgr2=0x56016e57df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.662+0000 7fceb6e30740 1 -- 172.21.15.43:0/826771338 shutdown_connections 2024-04-21T08:50:40.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.662+0000 7fceb6e30740 1 -- 172.21.15.43:0/826771338 wait complete. 2024-04-21T08:50:40.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.662+0000 7fceb6e30740 1 -- start start 2024-04-21T08:50:40.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.662+0000 7fceb6e30740 1 Processor -- start 2024-04-21T08:50:40.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.663+0000 7fceb6e30740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016e5834d0 0x56016e58c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.663+0000 7fceb6e30740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016e583e80 0x56016e5896b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.663+0000 7fceb6e30740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e58ca00 0x56016e589c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.663+0000 7fceb6e30740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56016e58f970 con 0x56016e58ca00 2024-04-21T08:50:40.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.663+0000 7fceb6e30740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56016e4a1500 con 0x56016e583e80 2024-04-21T08:50:40.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.663+0000 7fceb6e30740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56016e58faf0 con 0x56016e5834d0 2024-04-21T08:50:40.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.663+0000 7fceb6e30740 10 client.0 _ll_register_callbacks cb 0x56016e46fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:40.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.663+0000 7fceb17e3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e58ca00 0x56016e589c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.663+0000 7fceb17e3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e58ca00 0x56016e589c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50112/0 (socket says 172.21.15.43:50112) 2024-04-21T08:50:40.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.663+0000 7fceb17e3640 1 -- 172.21.15.43:0/3693452995 learned_addr learned my addr 172.21.15.43:0/3693452995 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.663+0000 7fceb1fe4640 1 --2- 172.21.15.43:0/3693452995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016e583e80 0x56016e5896b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.663+0000 7fceb17e3640 1 --2- 172.21.15.43:0/3693452995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016e5834d0 0x56016e58c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.663+0000 7fceb17e3640 1 -- 172.21.15.43:0/3693452995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016e5834d0 msgr2=0x56016e58c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.663+0000 7fceb17e3640 1 --2- 172.21.15.43:0/3693452995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016e5834d0 0x56016e58c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.663+0000 7fceb17e3640 1 -- 172.21.15.43:0/3693452995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016e583e80 msgr2=0x56016e5896b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.663+0000 7fceb17e3640 1 --2- 172.21.15.43:0/3693452995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016e583e80 0x56016e5896b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.663+0000 7fceb17e3640 1 -- 172.21.15.43:0/3693452995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56016e1fa3f0 con 0x56016e58ca00 2024-04-21T08:50:40.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.663+0000 7fceb17e3640 1 --2- 172.21.15.43:0/3693452995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e58ca00 0x56016e589c50 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7fce9c00e3a0 tx=0x7fce9c004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca679e0d87fb9db server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.663+0000 7fce957fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:40.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.663+0000 7fce957fa640 1 -- 172.21.15.43:0/3693452995 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c018040 con 0x56016e58ca00 2024-04-21T08:50:40.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.663+0000 7fce957fa640 1 -- 172.21.15.43:0/3693452995 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fce9c0108a0 con 0x56016e58ca00 2024-04-21T08:50:40.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.663+0000 7fceb6e30740 1 -- 172.21.15.43:0/3693452995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce9c0054d0 con 0x56016e58ca00 2024-04-21T08:50:40.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.664+0000 7fce957fa640 1 -- 172.21.15.43:0/3693452995 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fce9c010aa0 con 0x56016e58ca00 2024-04-21T08:50:40.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.664+0000 7fceb6e30740 10 client.21138 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:40.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.664+0000 7fceb6e30740 1 -- 172.21.15.43:0/3693452995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56016e2c47d0 con 0x56016e58ca00 2024-04-21T08:50:40.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.664+0000 7fce957fa640 1 -- 172.21.15.43:0/3693452995 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce9c0054d0 con 0x56016e58ca00 2024-04-21T08:50:40.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.664+0000 7fce957fa640 1 -- 172.21.15.43:0/3693452995 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fce9c029020 con 0x56016e58ca00 2024-04-21T08:50:40.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.664+0000 7fceb6e30740 10 client.21138 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:40.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.664+0000 7fceb6e30740 10 client.21138 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:50:40.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.664+0000 7fceb6e30740 20 client.21138 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:40.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.664+0000 7fceb6e30740 1 --2- 172.21.15.43:0/3693452995 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x56016e59cb50 0x56016e59ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.664+0000 7fceb6e30740 4 client.21138 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:40.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.664+0000 7fceb6e30740 1 -- 172.21.15.43:0/3693452995 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56016e487930 con 0x56016e59cb50 2024-04-21T08:50:40.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.664+0000 7fceb1fe4640 1 --2- 172.21.15.43:0/3693452995 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x56016e59cb50 0x56016e59ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.665+0000 7fceb1fe4640 1 --2- 172.21.15.43:0/3693452995 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x56016e59cb50 0x56016e59ef40 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c7ceff5e2e86489d server_cookie=67a77476a7020052 in_seq=0 out_seq=0 2024-04-21T08:50:40.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.665+0000 7fce957fa640 10 client.21138 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:50:40.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.665+0000 7fce957fa640 1 -- 172.21.15.43:0/3693452995 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56016e487930 con 0x56016e59cb50 2024-04-21T08:50:40.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.665+0000 7fce957fa640 10 client.21138 handle_command_reply: tid=0 2024-04-21T08:50:40.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.666+0000 7fceb6e30740 1 client.21138 shutdown 2024-04-21T08:50:40.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.666+0000 7fceb6e30740 1 -- 172.21.15.43:0/3693452995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e58ca00 msgr2=0x56016e589c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.666+0000 7fceb6e30740 1 --2- 172.21.15.43:0/3693452995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e58ca00 0x56016e589c50 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7fce9c00e3a0 tx=0x7fce9c004840 comp rx=0 tx=0).stop 2024-04-21T08:50:40.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.666+0000 7fceb6e30740 1 -- 172.21.15.43:0/3693452995 shutdown_connections 2024-04-21T08:50:40.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.666+0000 7fceb6e30740 1 --2- 172.21.15.43:0/3693452995 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x56016e59cb50 0x56016e59ef40 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.666+0000 7fceb6e30740 1 --2- 172.21.15.43:0/3693452995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e58ca00 0x56016e589c50 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.666+0000 7fce957fa640 0 client.21138 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:50:40.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.666+0000 7fceb6e30740 1 --2- 172.21.15.43:0/3693452995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016e583e80 0x56016e5896b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.666+0000 7fceb6e30740 1 --2- 172.21.15.43:0/3693452995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016e5834d0 0x56016e58c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.666+0000 7fceb6e30740 1 -- 172.21.15.43:0/3693452995 >> 172.21.15.43:0/3693452995 conn(0x56016e57dba0 msgr2=0x56016e57f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.666+0000 7fceb6e30740 1 -- 172.21.15.43:0/3693452995 shutdown_connections 2024-04-21T08:50:40.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.666+0000 7fceb6e30740 1 -- 172.21.15.43:0/3693452995 wait complete. 2024-04-21T08:50:40.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.666+0000 7fceb6e30740 20 client.21138 trim_cache size 0 max 16384 2024-04-21T08:50:40.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.679+0000 7fceb6e30740 1 Processor -- start 2024-04-21T08:50:40.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.679+0000 7fceb6e30740 1 -- start start 2024-04-21T08:50:40.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.680+0000 7fceb6e30740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016e5834d0 0x56016e598830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.680+0000 7fceb6e30740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e583e80 0x56016e57f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.680+0000 7fceb6e30740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016e58ca00 0x56016e58bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.680+0000 7fceb6e30740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fceac1b16a0 con 0x56016e583e80 2024-04-21T08:50:40.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.680+0000 7fceb6e30740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fceac1b31e0 con 0x56016e5834d0 2024-04-21T08:50:40.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.680+0000 7fceb6e30740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fceac1b1d70 con 0x56016e58ca00 2024-04-21T08:50:40.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.680+0000 7fceb1fe4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e583e80 0x56016e57f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.680+0000 7fceb1fe4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e583e80 0x56016e57f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50124/0 (socket says 172.21.15.43:50124) 2024-04-21T08:50:40.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.680+0000 7fceb1fe4640 1 -- 172.21.15.43:0/21363581 learned_addr learned my addr 172.21.15.43:0/21363581 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.680+0000 7fceb17e3640 1 --2- 172.21.15.43:0/21363581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016e58ca00 0x56016e58bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.680+0000 7fceb1fe4640 1 -- 172.21.15.43:0/21363581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016e58ca00 msgr2=0x56016e58bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.680+0000 7fceb1fe4640 1 --2- 172.21.15.43:0/21363581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016e58ca00 0x56016e58bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.680+0000 7fceb1fe4640 1 -- 172.21.15.43:0/21363581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016e5834d0 msgr2=0x56016e598830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.680+0000 7fceb1fe4640 1 --2- 172.21.15.43:0/21363581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016e5834d0 0x56016e598830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.680+0000 7fceb1fe4640 1 -- 172.21.15.43:0/21363581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56016e5993a0 con 0x56016e583e80 2024-04-21T08:50:40.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.680+0000 7fceb1fe4640 1 --2- 172.21.15.43:0/21363581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e583e80 0x56016e57f700 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x56016e57ec10 tx=0x7fcea8013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b43ce6d943c536b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.680+0000 7fce977fe640 1 -- 172.21.15.43:0/21363581 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea801d430 con 0x56016e583e80 2024-04-21T08:50:40.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.680+0000 7fce977fe640 1 -- 172.21.15.43:0/21363581 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcea8011440 con 0x56016e583e80 2024-04-21T08:50:40.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.680+0000 7fce977fe640 1 -- 172.21.15.43:0/21363581 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea800f680 con 0x56016e583e80 2024-04-21T08:50:40.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.681+0000 7fceb6e30740 1 -- 172.21.15.43:0/21363581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e583e80 msgr2=0x56016e57f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.681+0000 7fceb6e30740 1 --2- 172.21.15.43:0/21363581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e583e80 0x56016e57f700 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x56016e57ec10 tx=0x7fcea8013830 comp rx=0 tx=0).stop 2024-04-21T08:50:40.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.681+0000 7fceb6e30740 1 -- 172.21.15.43:0/21363581 shutdown_connections 2024-04-21T08:50:40.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.681+0000 7fceb6e30740 1 --2- 172.21.15.43:0/21363581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016e58ca00 0x56016e58bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.681+0000 7fceb6e30740 1 --2- 172.21.15.43:0/21363581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e583e80 0x56016e57f700 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.681+0000 7fceb6e30740 1 --2- 172.21.15.43:0/21363581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016e5834d0 0x56016e598830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.681+0000 7fceb6e30740 1 -- 172.21.15.43:0/21363581 >> 172.21.15.43:0/21363581 conn(0x56016e57dba0 msgr2=0x56016e57eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.681+0000 7fceb6e30740 1 -- 172.21.15.43:0/21363581 shutdown_connections 2024-04-21T08:50:40.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.681+0000 7fceb6e30740 1 -- 172.21.15.43:0/21363581 wait complete. 2024-04-21T08:50:40.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.681+0000 7fceb6e30740 1 -- start start 2024-04-21T08:50:40.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.681+0000 7fceb6e30740 1 Processor -- start 2024-04-21T08:50:40.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.682+0000 7fceb6e30740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e5834d0 0x56016e595010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.682+0000 7fceb6e30740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016e583e80 0x56016e585fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.682+0000 7fceb6e30740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016e58ca00 0x56016e586520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.682+0000 7fceb6e30740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fceac1b2b10 con 0x56016e5834d0 2024-04-21T08:50:40.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.682+0000 7fceb6e30740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fceac1b2440 con 0x56016e583e80 2024-04-21T08:50:40.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.682+0000 7fceb6e30740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fceac1af6a0 con 0x56016e58ca00 2024-04-21T08:50:40.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.682+0000 7fceb6e30740 10 client.0 _ll_register_callbacks cb 0x7fce9c0056f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:40.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.682+0000 7fceb17e3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e5834d0 0x56016e595010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.682+0000 7fceb17e3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e5834d0 0x56016e595010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50132/0 (socket says 172.21.15.43:50132) 2024-04-21T08:50:40.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.682+0000 7fceb17e3640 1 -- 172.21.15.43:0/4281081766 learned_addr learned my addr 172.21.15.43:0/4281081766 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.682+0000 7fceb1fe4640 1 --2- 172.21.15.43:0/4281081766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016e583e80 0x56016e585fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.682+0000 7fceb17e3640 1 -- 172.21.15.43:0/4281081766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016e58ca00 msgr2=0x56016e586520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.682+0000 7fceb17e3640 1 --2- 172.21.15.43:0/4281081766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016e58ca00 0x56016e586520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.682+0000 7fceb17e3640 1 -- 172.21.15.43:0/4281081766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016e583e80 msgr2=0x56016e585fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.682+0000 7fceb17e3640 1 --2- 172.21.15.43:0/4281081766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016e583e80 0x56016e585fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.682+0000 7fceb17e3640 1 -- 172.21.15.43:0/4281081766 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56016e586c60 con 0x56016e5834d0 2024-04-21T08:50:40.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.682+0000 7fceb1fe4640 1 --2- 172.21.15.43:0/4281081766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016e583e80 0x56016e585fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:40.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.682+0000 7fceb17e3640 1 --2- 172.21.15.43:0/4281081766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e5834d0 0x56016e595010 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7fce9c012fd0 tx=0x7fce9c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1357ffed9f66d8a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.682+0000 7fce957fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:40.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.682+0000 7fce957fa640 1 -- 172.21.15.43:0/4281081766 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c018040 con 0x56016e5834d0 2024-04-21T08:50:40.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.683+0000 7fce957fa640 1 -- 172.21.15.43:0/4281081766 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fce9c0108a0 con 0x56016e5834d0 2024-04-21T08:50:40.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.683+0000 7fceb6e30740 1 -- 172.21.15.43:0/4281081766 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcea8011440 con 0x56016e5834d0 2024-04-21T08:50:40.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.683+0000 7fce957fa640 1 -- 172.21.15.43:0/4281081766 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fce9c045050 con 0x56016e5834d0 2024-04-21T08:50:40.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.683+0000 7fceb6e30740 10 client.21144 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:40.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.683+0000 7fceb6e30740 1 -- 172.21.15.43:0/4281081766 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56016e602cf0 con 0x56016e5834d0 2024-04-21T08:50:40.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.683+0000 7fce957fa640 1 -- 172.21.15.43:0/4281081766 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcea8011440 con 0x56016e5834d0 2024-04-21T08:50:40.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.683+0000 7fce957fa640 1 -- 172.21.15.43:0/4281081766 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fce9c029040 con 0x56016e5834d0 2024-04-21T08:50:40.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.683+0000 7fceb6e30740 10 client.21144 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:40.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.683+0000 7fceb6e30740 10 client.21144 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:50:40.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.683+0000 7fceb6e30740 20 client.21144 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:40.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.683+0000 7fceb6e30740 1 --2- 172.21.15.43:0/4281081766 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x56016e603380 0x56016e6057a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.683+0000 7fceb6e30740 4 client.21144 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:50:40.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.683+0000 7fceb6e30740 1 -- 172.21.15.43:0/4281081766 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56016e4f5c10 con 0x56016e603380 2024-04-21T08:50:40.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.683+0000 7fceb1fe4640 1 --2- 172.21.15.43:0/4281081766 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x56016e603380 0x56016e6057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.684+0000 7fce957fa640 10 client.21144 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:50:40.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.684+0000 7fceb1fe4640 1 --2- 172.21.15.43:0/4281081766 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x56016e603380 0x56016e6057a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c3b43fddcedb4a server_cookie=7daddd044c5cf9f6 in_seq=0 out_seq=0 2024-04-21T08:50:40.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.684+0000 7fce957fa640 1 -- 172.21.15.43:0/4281081766 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56016e4f5c10 con 0x56016e603380 2024-04-21T08:50:40.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.684+0000 7fce957fa640 10 client.21144 handle_command_reply: tid=0 2024-04-21T08:50:40.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.684+0000 7fceb6e30740 1 client.21144 shutdown 2024-04-21T08:50:40.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.684+0000 7fceb6e30740 1 -- 172.21.15.43:0/4281081766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e5834d0 msgr2=0x56016e595010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.684+0000 7fceb6e30740 1 --2- 172.21.15.43:0/4281081766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e5834d0 0x56016e595010 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7fce9c012fd0 tx=0x7fce9c0058a0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.684+0000 7fceb6e30740 1 -- 172.21.15.43:0/4281081766 shutdown_connections 2024-04-21T08:50:40.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.684+0000 7fceb6e30740 1 --2- 172.21.15.43:0/4281081766 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x56016e603380 0x56016e6057a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.684+0000 7fceb6e30740 1 --2- 172.21.15.43:0/4281081766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56016e58ca00 0x56016e586520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.684+0000 7fce957fa640 0 client.21144 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:50:40.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.684+0000 7fceb6e30740 1 --2- 172.21.15.43:0/4281081766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56016e583e80 0x56016e585fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.684+0000 7fceb6e30740 1 --2- 172.21.15.43:0/4281081766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56016e5834d0 0x56016e595010 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.684+0000 7fceb6e30740 1 -- 172.21.15.43:0/4281081766 >> 172.21.15.43:0/4281081766 conn(0x56016e57dba0 msgr2=0x56016e57f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.684+0000 7fceb6e30740 1 -- 172.21.15.43:0/4281081766 shutdown_connections 2024-04-21T08:50:40.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.684+0000 7fceb6e30740 1 -- 172.21.15.43:0/4281081766 wait complete. 2024-04-21T08:50:40.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.685+0000 7fceb6e30740 20 client.21144 trim_cache size 0 max 16384 2024-04-21T08:50:40.694 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:50:40.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.685+0000 7fceb4248640 1 -- 172.21.15.43:0/1227161218 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fce980a3d20 msgr2=0x7fce980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.685+0000 7fceb4248640 1 --2- 172.21.15.43:0/1227161218 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fce980a3d20 0x7fce980a61e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcea40be9c0 tx=0x7fcea4041040 comp rx=0 tx=0).stop 2024-04-21T08:50:40.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.685+0000 7fceb4248640 1 -- 172.21.15.43:0/1227161218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceac074cd0 msgr2=0x7fceac1a68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.685+0000 7fceb4248640 1 --2- 172.21.15.43:0/1227161218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceac074cd0 0x7fceac1a68c0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7fcea40098d0 tx=0x7fcea4035d60 comp rx=0 tx=0).stop 2024-04-21T08:50:40.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.685+0000 7fceb4248640 1 -- 172.21.15.43:0/1227161218 shutdown_connections 2024-04-21T08:50:40.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.685+0000 7fceb4248640 1 --2- 172.21.15.43:0/1227161218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceac074cd0 0x7fceac1a68c0 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.685+0000 7fceb4248640 1 --2- 172.21.15.43:0/1227161218 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fce980a3d20 0x7fce980a61e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.685+0000 7fceb4248640 1 --2- 172.21.15.43:0/1227161218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceac107260 0x7fceac074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.685+0000 7fceb4248640 1 --2- 172.21.15.43:0/1227161218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceac102190 0x7fceac074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.685+0000 7fceb4248640 1 -- 172.21.15.43:0/1227161218 >> 172.21.15.43:0/1227161218 conn(0x7fceac0793a0 msgr2=0x7fceac0fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.685+0000 7fceb4248640 1 -- 172.21.15.43:0/1227161218 shutdown_connections 2024-04-21T08:50:40.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.685+0000 7fceb4248640 1 -- 172.21.15.43:0/1227161218 wait complete. 2024-04-21T08:50:40.697 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:50:40.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:40 smithi175 ceph-mon[31920]: pgmap v366: 129 pgs: 129 active+clean; 4.9 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:50:40.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:40 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3123942233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:50:40.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:40 smithi112 ceph-mon[31085]: pgmap v366: 129 pgs: 129 active+clean; 4.9 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:50:40.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:40 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3123942233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:50:40.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.840+0000 7ff2299bd640 1 -- 172.21.15.43:0/594240527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff224106160 msgr2=0x7ff22410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.841+0000 7ff2299bd640 1 --2- 172.21.15.43:0/594240527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff224106160 0x7ff22410dd30 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009900 tx=0x7ff21c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:40.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.841+0000 7ff2299bd640 1 -- 172.21.15.43:0/594240527 shutdown_connections 2024-04-21T08:50:40.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.841+0000 7ff2299bd640 1 --2- 172.21.15.43:0/594240527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff224106160 0x7ff22410dd30 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.841+0000 7ff2299bd640 1 --2- 172.21.15.43:0/594240527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff224101a40 0x7ff224105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.841+0000 7ff2299bd640 1 --2- 172.21.15.43:0/594240527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff224101090 0x7ff224101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.841+0000 7ff2299bd640 1 -- 172.21.15.43:0/594240527 >> 172.21.15.43:0/594240527 conn(0x7ff224077f50 msgr2=0x7ff224078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.841+0000 7ff2299bd640 1 -- 172.21.15.43:0/594240527 shutdown_connections 2024-04-21T08:50:40.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.841+0000 7ff2299bd640 1 -- 172.21.15.43:0/594240527 wait complete. 2024-04-21T08:50:40.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.841+0000 7ff2299bd640 1 Processor -- start 2024-04-21T08:50:40.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.841+0000 7ff2299bd640 1 -- start start 2024-04-21T08:50:40.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.842+0000 7ff2299bd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff224101090 0x7ff224199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.842+0000 7ff2299bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff224106160 0x7ff224199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.842+0000 7ff2299bd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff22419a040 0x7ff2241a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.842+0000 7ff2299bd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff2241110b0 con 0x7ff224101090 2024-04-21T08:50:40.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.842+0000 7ff2299bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff224110f30 con 0x7ff224106160 2024-04-21T08:50:40.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.842+0000 7ff2299bd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff224111230 con 0x7ff22419a040 2024-04-21T08:50:40.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.842+0000 7ff222ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff224101090 0x7ff224199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.842+0000 7ff222ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff224101090 0x7ff224199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50146/0 (socket says 172.21.15.43:50146) 2024-04-21T08:50:40.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.842+0000 7ff222ffd640 1 -- 172.21.15.43:0/2286497662 learned_addr learned my addr 172.21.15.43:0/2286497662 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.842+0000 7ff2237fe640 1 --2- 172.21.15.43:0/2286497662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff22419a040 0x7ff2241a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.842+0000 7ff2227fc640 1 --2- 172.21.15.43:0/2286497662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff224106160 0x7ff224199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.842+0000 7ff222ffd640 1 -- 172.21.15.43:0/2286497662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff22419a040 msgr2=0x7ff2241a45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.842+0000 7ff222ffd640 1 --2- 172.21.15.43:0/2286497662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff22419a040 0x7ff2241a45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.842+0000 7ff222ffd640 1 -- 172.21.15.43:0/2286497662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff224106160 msgr2=0x7ff224199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.842+0000 7ff222ffd640 1 --2- 172.21.15.43:0/2286497662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff224106160 0x7ff224199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.842+0000 7ff222ffd640 1 -- 172.21.15.43:0/2286497662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2241a4df0 con 0x7ff224101090 2024-04-21T08:50:40.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.842+0000 7ff2227fc640 1 --2- 172.21.15.43:0/2286497662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff224106160 0x7ff224199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:40.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.842+0000 7ff222ffd640 1 --2- 172.21.15.43:0/2286497662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff224101090 0x7ff224199570 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7ff21400ebc0 tx=0x7ff21400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.842+0000 7ff2289bb640 1 -- 172.21.15.43:0/2286497662 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214010630 con 0x7ff224101090 2024-04-21T08:50:40.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.842+0000 7ff2289bb640 1 -- 172.21.15.43:0/2286497662 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff214010c80 con 0x7ff224101090 2024-04-21T08:50:40.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.842+0000 7ff2299bd640 1 -- 172.21.15.43:0/2286497662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2241a50e0 con 0x7ff224101090 2024-04-21T08:50:40.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.842+0000 7ff2289bb640 1 -- 172.21.15.43:0/2286497662 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214018b60 con 0x7ff224101090 2024-04-21T08:50:40.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.842+0000 7ff2299bd640 1 -- 172.21.15.43:0/2286497662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2241a5630 con 0x7ff224101090 2024-04-21T08:50:40.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.845+0000 7ff2289bb640 1 -- 172.21.15.43:0/2286497662 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff214018d00 con 0x7ff224101090 2024-04-21T08:50:40.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.845+0000 7ff22c5b1740 1 Processor -- start 2024-04-21T08:50:40.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.845+0000 7ff22c5b1740 1 -- start start 2024-04-21T08:50:40.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.845+0000 7ff22c5b1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb291dd3b0 0x55bb291dd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.845+0000 7ff22c5b1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291ddd60 0x55bb291e6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.845+0000 7ff22c5b1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb291e6e60 0x55bb291e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.845+0000 7ff22c5b1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bb291421f0 con 0x55bb291ddd60 2024-04-21T08:50:40.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.845+0000 7ff22c5b1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bb290df4b0 con 0x55bb291dd3b0 2024-04-21T08:50:40.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.845+0000 7ff22c5b1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bb2913e1b0 con 0x55bb291e6e60 2024-04-21T08:50:40.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.845+0000 7ff2289bb640 1 --2- 172.21.15.43:0/2286497662 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff20c0a3d20 0x7ff20c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.845+0000 7ff222ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291ddd60 0x55bb291e6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.845+0000 7ff222ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291ddd60 0x55bb291e6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50150/0 (socket says 172.21.15.43:50150) 2024-04-21T08:50:40.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.845+0000 7ff222ffd640 1 -- 172.21.15.43:0/1993649671 learned_addr learned my addr 172.21.15.43:0/1993649671 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.845+0000 7ff2237fe640 1 --2- 172.21.15.43:0/2286497662 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff20c0a3d20 0x7ff20c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.845+0000 7ff2237fe640 1 --2- 172.21.15.43:0/1993649671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb291dd3b0 0x55bb291dd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.846+0000 7ff2227fc640 1 --2- 172.21.15.43:0/1993649671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb291e6e60 0x55bb291e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.846+0000 7ff2289bb640 1 -- 172.21.15.43:0/2286497662 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7ff2140d12b0 con 0x7ff224101090 2024-04-21T08:50:40.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.846+0000 7ff222ffd640 1 -- 172.21.15.43:0/1993649671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb291e6e60 msgr2=0x55bb291e72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.846+0000 7ff222ffd640 1 --2- 172.21.15.43:0/1993649671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb291e6e60 0x55bb291e72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.846+0000 7ff222ffd640 1 -- 172.21.15.43:0/1993649671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb291dd3b0 msgr2=0x55bb291dd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.846+0000 7ff222ffd640 1 --2- 172.21.15.43:0/1993649671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb291dd3b0 0x55bb291dd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.846+0000 7ff222ffd640 1 -- 172.21.15.43:0/1993649671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb290d11b0 con 0x55bb291ddd60 2024-04-21T08:50:40.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.846+0000 7ff222ffd640 1 --2- 172.21.15.43:0/1993649671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291ddd60 0x55bb291e6890 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7ff21409cdc0 tx=0x7ff21409caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1655beaa595e01 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.846+0000 7ff2237fe640 1 --2- 172.21.15.43:0/2286497662 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff20c0a3d20 0x7ff20c0a61e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff21c005ab0 tx=0x7ff21c005fd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.846+0000 7ff209ffb640 1 -- 172.21.15.43:0/1993649671 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2140a0040 con 0x55bb291ddd60 2024-04-21T08:50:40.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.846+0000 7ff209ffb640 1 -- 172.21.15.43:0/1993649671 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff214018420 con 0x55bb291ddd60 2024-04-21T08:50:40.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.846+0000 7ff209ffb640 1 -- 172.21.15.43:0/1993649671 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2140ce860 con 0x55bb291ddd60 2024-04-21T08:50:40.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.847+0000 7ff22c5b1740 1 -- 172.21.15.43:0/1993649671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291ddd60 msgr2=0x55bb291e6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.847+0000 7ff22c5b1740 1 --2- 172.21.15.43:0/1993649671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291ddd60 0x55bb291e6890 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7ff21409cdc0 tx=0x7ff21409caa0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.847+0000 7ff22c5b1740 1 -- 172.21.15.43:0/1993649671 shutdown_connections 2024-04-21T08:50:40.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.847+0000 7ff22c5b1740 1 --2- 172.21.15.43:0/1993649671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb291e6e60 0x55bb291e72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.847+0000 7ff22c5b1740 1 --2- 172.21.15.43:0/1993649671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291ddd60 0x55bb291e6890 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.847+0000 7ff22c5b1740 1 --2- 172.21.15.43:0/1993649671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb291dd3b0 0x55bb291dd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.847+0000 7ff22c5b1740 1 -- 172.21.15.43:0/1993649671 >> 172.21.15.43:0/1993649671 conn(0x55bb291d7ba0 msgr2=0x55bb291d7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.847+0000 7ff22c5b1740 1 -- 172.21.15.43:0/1993649671 shutdown_connections 2024-04-21T08:50:40.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.847+0000 7ff22c5b1740 1 -- 172.21.15.43:0/1993649671 wait complete. 2024-04-21T08:50:40.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.847+0000 7ff22c5b1740 1 -- start start 2024-04-21T08:50:40.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.847+0000 7ff22c5b1740 1 Processor -- start 2024-04-21T08:50:40.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.848+0000 7ff22c5b1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291dd3b0 0x55bb291ed2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.848+0000 7ff22c5b1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb291ddd60 0x55bb291ea630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.848+0000 7ff22c5b1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb291e6e60 0x55bb291eab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.848+0000 7ff22c5b1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bb291edd40 con 0x55bb291dd3b0 2024-04-21T08:50:40.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.848+0000 7ff22c5b1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bb290fb500 con 0x55bb291e6e60 2024-04-21T08:50:40.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.848+0000 7ff22c5b1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bb291edec0 con 0x55bb291ddd60 2024-04-21T08:50:40.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.848+0000 7ff22c5b1740 10 client.0 _ll_register_callbacks cb 0x55bb290c9b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:40.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.848+0000 7ff222ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291dd3b0 0x55bb291ed2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.848+0000 7ff222ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291dd3b0 0x55bb291ed2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50154/0 (socket says 172.21.15.43:50154) 2024-04-21T08:50:40.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.848+0000 7ff222ffd640 1 -- 172.21.15.43:0/3100802684 learned_addr learned my addr 172.21.15.43:0/3100802684 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.848+0000 7ff2227fc640 1 --2- 172.21.15.43:0/3100802684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb291ddd60 0x55bb291ea630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.848+0000 7ff2237fe640 1 --2- 172.21.15.43:0/3100802684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb291e6e60 0x55bb291eab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.848+0000 7ff222ffd640 1 -- 172.21.15.43:0/3100802684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb291ddd60 msgr2=0x55bb291ea630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.848+0000 7ff222ffd640 1 --2- 172.21.15.43:0/3100802684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb291ddd60 0x55bb291ea630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.848+0000 7ff222ffd640 1 -- 172.21.15.43:0/3100802684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb291e6e60 msgr2=0x55bb291eab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.848+0000 7ff222ffd640 1 --2- 172.21.15.43:0/3100802684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb291e6e60 0x55bb291eab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.848+0000 7ff222ffd640 1 -- 172.21.15.43:0/3100802684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb28e543f0 con 0x55bb291dd3b0 2024-04-21T08:50:40.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.848+0000 7ff2237fe640 1 --2- 172.21.15.43:0/3100802684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb291e6e60 0x55bb291eab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:40.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.848+0000 7ff222ffd640 1 --2- 172.21.15.43:0/3100802684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291dd3b0 0x55bb291ed2a0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7ff21409cdc0 tx=0x7ff214000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f1e24fc8bc6f71 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.848+0000 7ff202ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:40.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.848+0000 7ff202ffd640 1 -- 172.21.15.43:0/3100802684 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2140a11e0 con 0x55bb291dd3b0 2024-04-21T08:50:40.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.848+0000 7ff202ffd640 1 -- 172.21.15.43:0/3100802684 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7ff2140a5610 con 0x55bb291dd3b0 2024-04-21T08:50:40.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.848+0000 7ff22c5b1740 1 -- 172.21.15.43:0/3100802684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bb290e1930 con 0x55bb291dd3b0 2024-04-21T08:50:40.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.848+0000 7ff202ffd640 1 -- 172.21.15.43:0/3100802684 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff21409d530 con 0x55bb291dd3b0 2024-04-21T08:50:40.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.849+0000 7ff22c5b1740 10 client.21156 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:40.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.849+0000 7ff22c5b1740 1 -- 172.21.15.43:0/3100802684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bb28f1e7d0 con 0x55bb291dd3b0 2024-04-21T08:50:40.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.849+0000 7ff202ffd640 1 -- 172.21.15.43:0/3100802684 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bb290e1930 con 0x55bb291dd3b0 2024-04-21T08:50:40.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.849+0000 7ff202ffd640 1 -- 172.21.15.43:0/3100802684 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff214098b80 con 0x55bb291dd3b0 2024-04-21T08:50:40.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.849+0000 7ff22c5b1740 10 client.21156 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:40.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.849+0000 7ff22c5b1740 10 client.21156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.849+0000 7ff22c5b1740 20 client.21156 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.849+0000 7ff22c5b1740 1 --2- 172.21.15.43:0/3100802684 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55bb291fae70 0x55bb291fd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.849+0000 7ff22c5b1740 4 client.21156 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.849+0000 7ff22c5b1740 1 -- 172.21.15.43:0/3100802684 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb2914fc10 con 0x55bb291fae70 2024-04-21T08:50:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.849+0000 7ff2227fc640 1 --2- 172.21.15.43:0/3100802684 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55bb291fae70 0x55bb291fd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.850+0000 7ff202ffd640 10 client.21156 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.850+0000 7ff2227fc640 1 --2- 172.21.15.43:0/3100802684 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55bb291fae70 0x55bb291fd260 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28d9754795e02a5e server_cookie=bd6c001305448dfb in_seq=0 out_seq=0 2024-04-21T08:50:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.850+0000 7ff202ffd640 1 -- 172.21.15.43:0/3100802684 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bb2914fc10 con 0x55bb291fae70 2024-04-21T08:50:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.850+0000 7ff202ffd640 10 client.21156 handle_command_reply: tid=0 2024-04-21T08:50:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.850+0000 7ff22c5b1740 1 client.21156 shutdown 2024-04-21T08:50:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.851+0000 7ff22c5b1740 1 -- 172.21.15.43:0/3100802684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291dd3b0 msgr2=0x55bb291ed2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.851+0000 7ff22c5b1740 1 --2- 172.21.15.43:0/3100802684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291dd3b0 0x55bb291ed2a0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7ff21409cdc0 tx=0x7ff214000f40 comp rx=0 tx=0).stop 2024-04-21T08:50:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.851+0000 7ff22c5b1740 1 -- 172.21.15.43:0/3100802684 shutdown_connections 2024-04-21T08:50:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.851+0000 7ff22c5b1740 1 --2- 172.21.15.43:0/3100802684 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55bb291fae70 0x55bb291fd260 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.851+0000 7ff22c5b1740 1 --2- 172.21.15.43:0/3100802684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb291e6e60 0x55bb291eab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.851+0000 7ff202ffd640 0 client.21156 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:40.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.851+0000 7ff22c5b1740 1 --2- 172.21.15.43:0/3100802684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb291ddd60 0x55bb291ea630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.851+0000 7ff22c5b1740 1 --2- 172.21.15.43:0/3100802684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291dd3b0 0x55bb291ed2a0 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.851+0000 7ff22c5b1740 1 -- 172.21.15.43:0/3100802684 >> 172.21.15.43:0/3100802684 conn(0x55bb291d7ba0 msgr2=0x55bb291d98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.851+0000 7ff22c5b1740 1 -- 172.21.15.43:0/3100802684 shutdown_connections 2024-04-21T08:50:40.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.851+0000 7ff22c5b1740 1 -- 172.21.15.43:0/3100802684 wait complete. 2024-04-21T08:50:40.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.851+0000 7ff22c5b1740 20 client.21156 trim_cache size 0 max 16384 2024-04-21T08:50:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.864+0000 7ff22c5b1740 1 Processor -- start 2024-04-21T08:50:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.864+0000 7ff22c5b1740 1 -- start start 2024-04-21T08:50:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.864+0000 7ff22c5b1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb291dd3b0 0x55bb291e8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.864+0000 7ff22c5b1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291ddd60 0x55bb291f6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.864+0000 7ff22c5b1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb291e6e60 0x55bb291ecd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.864+0000 7ff22c5b1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff2241ad3c0 con 0x55bb291ddd60 2024-04-21T08:50:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.864+0000 7ff22c5b1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2241aef00 con 0x55bb291dd3b0 2024-04-21T08:50:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.864+0000 7ff22c5b1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff2241ada90 con 0x55bb291e6e60 2024-04-21T08:50:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.864+0000 7ff2227fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291ddd60 0x55bb291f6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.864+0000 7ff2227fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291ddd60 0x55bb291f6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50156/0 (socket says 172.21.15.43:50156) 2024-04-21T08:50:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.864+0000 7ff2227fc640 1 -- 172.21.15.43:0/3542027444 learned_addr learned my addr 172.21.15.43:0/3542027444 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.865+0000 7ff222ffd640 1 --2- 172.21.15.43:0/3542027444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb291dd3b0 0x55bb291e8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.865+0000 7ff2237fe640 1 --2- 172.21.15.43:0/3542027444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb291e6e60 0x55bb291ecd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.865+0000 7ff2227fc640 1 -- 172.21.15.43:0/3542027444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb291e6e60 msgr2=0x55bb291ecd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.865+0000 7ff2227fc640 1 --2- 172.21.15.43:0/3542027444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb291e6e60 0x55bb291ecd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.865+0000 7ff2227fc640 1 -- 172.21.15.43:0/3542027444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb291dd3b0 msgr2=0x55bb291e8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.865+0000 7ff2227fc640 1 --2- 172.21.15.43:0/3542027444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb291dd3b0 0x55bb291e8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.865+0000 7ff2227fc640 1 -- 172.21.15.43:0/3542027444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb291eaf00 con 0x55bb291ddd60 2024-04-21T08:50:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.865+0000 7ff222ffd640 1 --2- 172.21.15.43:0/3542027444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb291dd3b0 0x55bb291e8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.865+0000 7ff2237fe640 1 --2- 172.21.15.43:0/3542027444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb291e6e60 0x55bb291ecd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.865+0000 7ff2227fc640 1 --2- 172.21.15.43:0/3542027444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291ddd60 0x55bb291f6860 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7ff218005d80 tx=0x7ff218005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d779d759a27fa86 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.865+0000 7ff2097fa640 1 -- 172.21.15.43:0/3542027444 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff218014b20 con 0x55bb291ddd60 2024-04-21T08:50:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.865+0000 7ff2097fa640 1 -- 172.21.15.43:0/3542027444 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff218021070 con 0x55bb291ddd60 2024-04-21T08:50:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.865+0000 7ff2097fa640 1 -- 172.21.15.43:0/3542027444 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21801b800 con 0x55bb291ddd60 2024-04-21T08:50:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.866+0000 7ff22c5b1740 1 -- 172.21.15.43:0/3542027444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291ddd60 msgr2=0x55bb291f6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.866+0000 7ff22c5b1740 1 --2- 172.21.15.43:0/3542027444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291ddd60 0x55bb291f6860 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7ff218005d80 tx=0x7ff218005d10 comp rx=0 tx=0).stop 2024-04-21T08:50:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.866+0000 7ff22c5b1740 1 -- 172.21.15.43:0/3542027444 shutdown_connections 2024-04-21T08:50:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.866+0000 7ff22c5b1740 1 --2- 172.21.15.43:0/3542027444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb291e6e60 0x55bb291ecd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.866+0000 7ff22c5b1740 1 --2- 172.21.15.43:0/3542027444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291ddd60 0x55bb291f6860 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.866+0000 7ff22c5b1740 1 --2- 172.21.15.43:0/3542027444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb291dd3b0 0x55bb291e8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.866+0000 7ff22c5b1740 1 -- 172.21.15.43:0/3542027444 >> 172.21.15.43:0/3542027444 conn(0x55bb291d7ba0 msgr2=0x55bb291f6150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.866+0000 7ff22c5b1740 1 -- 172.21.15.43:0/3542027444 shutdown_connections 2024-04-21T08:50:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.866+0000 7ff22c5b1740 1 -- 172.21.15.43:0/3542027444 wait complete. 2024-04-21T08:50:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.866+0000 7ff22c5b1740 1 -- start start 2024-04-21T08:50:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.866+0000 7ff22c5b1740 1 Processor -- start 2024-04-21T08:50:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.866+0000 7ff22c5b1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb291dd3b0 0x55bb291fdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.866+0000 7ff22c5b1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291ddd60 0x55bb291fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.867+0000 7ff22c5b1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb291e6e60 0x55bb291fe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.867+0000 7ff22c5b1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff2241ae830 con 0x55bb291ddd60 2024-04-21T08:50:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.867+0000 7ff22c5b1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2241ae160 con 0x55bb291e6e60 2024-04-21T08:50:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.867+0000 7ff22c5b1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff2241ab3c0 con 0x55bb291dd3b0 2024-04-21T08:50:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.867+0000 7ff22c5b1740 10 client.0 _ll_register_callbacks cb 0x7ff214005a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.867+0000 7ff2227fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291ddd60 0x55bb291fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.867+0000 7ff2227fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291ddd60 0x55bb291fe2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50158/0 (socket says 172.21.15.43:50158) 2024-04-21T08:50:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.867+0000 7ff2227fc640 1 -- 172.21.15.43:0/3823495953 learned_addr learned my addr 172.21.15.43:0/3823495953 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.867+0000 7ff222ffd640 1 --2- 172.21.15.43:0/3823495953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb291dd3b0 0x55bb291fdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.867+0000 7ff2227fc640 1 -- 172.21.15.43:0/3823495953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb291dd3b0 msgr2=0x55bb291fdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.867+0000 7ff2227fc640 1 --2- 172.21.15.43:0/3823495953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb291dd3b0 0x55bb291fdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.867+0000 7ff2227fc640 1 -- 172.21.15.43:0/3823495953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb291e6e60 msgr2=0x55bb291fe7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.867+0000 7ff2237fe640 1 --2- 172.21.15.43:0/3823495953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb291e6e60 0x55bb291fe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.867+0000 7ff2227fc640 1 --2- 172.21.15.43:0/3823495953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb291e6e60 0x55bb291fe7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.867+0000 7ff2227fc640 1 -- 172.21.15.43:0/3823495953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb29260100 con 0x55bb291ddd60 2024-04-21T08:50:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.867+0000 7ff2227fc640 1 --2- 172.21.15.43:0/3823495953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291ddd60 0x55bb291fe2a0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7ff21800bce0 tx=0x7ff218014220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54e18ccdf46d96c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.867+0000 7ff202ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.867+0000 7ff202ffd640 1 -- 172.21.15.43:0/3823495953 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff218004030 con 0x55bb291ddd60 2024-04-21T08:50:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.867+0000 7ff202ffd640 1 -- 172.21.15.43:0/3823495953 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7ff2180041d0 con 0x55bb291ddd60 2024-04-21T08:50:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.867+0000 7ff22c5b1740 1 -- 172.21.15.43:0/3823495953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff218021070 con 0x55bb291ddd60 2024-04-21T08:50:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.867+0000 7ff202ffd640 1 -- 172.21.15.43:0/3823495953 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff218036050 con 0x55bb291ddd60 2024-04-21T08:50:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.867+0000 7ff22c5b1740 10 client.21162 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.867+0000 7ff22c5b1740 1 -- 172.21.15.43:0/3823495953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bb29261330 con 0x55bb291ddd60 2024-04-21T08:50:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.868+0000 7ff202ffd640 1 -- 172.21.15.43:0/3823495953 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff218021070 con 0x55bb291ddd60 2024-04-21T08:50:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.868+0000 7ff202ffd640 1 -- 172.21.15.43:0/3823495953 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff218003ad0 con 0x55bb291ddd60 2024-04-21T08:50:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.868+0000 7ff22c5b1740 10 client.21162 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.868+0000 7ff22c5b1740 10 client.21162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.868+0000 7ff22c5b1740 20 client.21162 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.868+0000 7ff22c5b1740 1 --2- 172.21.15.43:0/3823495953 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55bb29261a20 0x55bb29263e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.868+0000 7ff22c5b1740 4 client.21162 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:50:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.868+0000 7ff22c5b1740 1 -- 172.21.15.43:0/3823495953 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb29149a80 con 0x55bb29261a20 2024-04-21T08:50:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.868+0000 7ff222ffd640 1 --2- 172.21.15.43:0/3823495953 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55bb29261a20 0x55bb29263e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.869+0000 7ff222ffd640 1 --2- 172.21.15.43:0/3823495953 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55bb29261a20 0x55bb29263e40 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3664aa23bc2a5580 server_cookie=e02d57bc81048a13 in_seq=0 out_seq=0 2024-04-21T08:50:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.869+0000 7ff202ffd640 10 client.21162 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.869+0000 7ff202ffd640 1 -- 172.21.15.43:0/3823495953 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb29149a80 con 0x55bb29261a20 2024-04-21T08:50:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.869+0000 7ff202ffd640 10 client.21162 handle_command_reply: tid=0 2024-04-21T08:50:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.869+0000 7ff22c5b1740 1 client.21162 shutdown 2024-04-21T08:50:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.869+0000 7ff22c5b1740 1 -- 172.21.15.43:0/3823495953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291ddd60 msgr2=0x55bb291fe2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.869+0000 7ff22c5b1740 1 --2- 172.21.15.43:0/3823495953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291ddd60 0x55bb291fe2a0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7ff21800bce0 tx=0x7ff218014220 comp rx=0 tx=0).stop 2024-04-21T08:50:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.869+0000 7ff22c5b1740 1 -- 172.21.15.43:0/3823495953 shutdown_connections 2024-04-21T08:50:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.869+0000 7ff22c5b1740 1 --2- 172.21.15.43:0/3823495953 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55bb29261a20 0x55bb29263e40 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.869+0000 7ff22c5b1740 1 --2- 172.21.15.43:0/3823495953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bb291e6e60 0x55bb291fe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.869+0000 7ff202ffd640 0 client.21162 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.869+0000 7ff22c5b1740 1 --2- 172.21.15.43:0/3823495953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb291ddd60 0x55bb291fe2a0 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.869+0000 7ff22c5b1740 1 --2- 172.21.15.43:0/3823495953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb291dd3b0 0x55bb291fdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.869+0000 7ff22c5b1740 1 -- 172.21.15.43:0/3823495953 >> 172.21.15.43:0/3823495953 conn(0x55bb291d7ba0 msgr2=0x55bb291f69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.869+0000 7ff22c5b1740 1 -- 172.21.15.43:0/3823495953 shutdown_connections 2024-04-21T08:50:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.869+0000 7ff22c5b1740 1 -- 172.21.15.43:0/3823495953 wait complete. 2024-04-21T08:50:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.869+0000 7ff22c5b1740 20 client.21162 trim_cache size 0 max 16384 2024-04-21T08:50:40.876 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:50:40.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.870+0000 7ff2299bd640 1 -- 172.21.15.43:0/2286497662 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff20c0a3d20 msgr2=0x7ff20c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.870+0000 7ff2299bd640 1 --2- 172.21.15.43:0/2286497662 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff20c0a3d20 0x7ff20c0a61e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff21c005ab0 tx=0x7ff21c005fd0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.870+0000 7ff2299bd640 1 -- 172.21.15.43:0/2286497662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff224101090 msgr2=0x7ff224199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:40.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.870+0000 7ff2299bd640 1 --2- 172.21.15.43:0/2286497662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff224101090 0x7ff224199570 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7ff21400ebc0 tx=0x7ff21400c6a0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.870+0000 7ff2299bd640 1 -- 172.21.15.43:0/2286497662 shutdown_connections 2024-04-21T08:50:40.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.870+0000 7ff2299bd640 1 --2- 172.21.15.43:0/2286497662 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff20c0a3d20 0x7ff20c0a61e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.870+0000 7ff2299bd640 1 --2- 172.21.15.43:0/2286497662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff22419a040 0x7ff2241a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.870+0000 7ff2299bd640 1 --2- 172.21.15.43:0/2286497662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff224106160 0x7ff224199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.870+0000 7ff2299bd640 1 --2- 172.21.15.43:0/2286497662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff224101090 0x7ff224199570 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:40.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.870+0000 7ff2299bd640 1 -- 172.21.15.43:0/2286497662 >> 172.21.15.43:0/2286497662 conn(0x7ff224077f50 msgr2=0x7ff2240fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:40.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.870+0000 7ff2299bd640 1 -- 172.21.15.43:0/2286497662 shutdown_connections 2024-04-21T08:50:40.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:40.870+0000 7ff2299bd640 1 -- 172.21.15.43:0/2286497662 wait complete. 2024-04-21T08:50:41.882 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:50:41.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:50:41.882 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:50:42.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.001+0000 7f9e064ba640 1 -- 172.21.15.43:0/2048327284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e00107270 msgr2=0x7f9e0010ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:42.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.001+0000 7f9e064ba640 1 --2- 172.21.15.43:0/2048327284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e00107270 0x7f9e0010ee40 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f9df8009ca0 tx=0x7f9df802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.001+0000 7f9e064ba640 1 -- 172.21.15.43:0/2048327284 shutdown_connections 2024-04-21T08:50:42.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.001+0000 7f9e064ba640 1 --2- 172.21.15.43:0/2048327284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e00107270 0x7f9e0010ee40 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.001+0000 7f9e064ba640 1 --2- 172.21.15.43:0/2048327284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e0006b1f0 0x7f9e00106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.001+0000 7f9e064ba640 1 --2- 172.21.15.43:0/2048327284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e0006a840 0x7f9e0006ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.001+0000 7f9e064ba640 1 -- 172.21.15.43:0/2048327284 >> 172.21.15.43:0/2048327284 conn(0x7f9e00077eb0 msgr2=0x7f9e000782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:42.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.001+0000 7f9e064ba640 1 -- 172.21.15.43:0/2048327284 shutdown_connections 2024-04-21T08:50:42.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.001+0000 7f9e064ba640 1 -- 172.21.15.43:0/2048327284 wait complete. 2024-04-21T08:50:42.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.002+0000 7f9e064ba640 1 Processor -- start 2024-04-21T08:50:42.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.002+0000 7f9e064ba640 1 -- start start 2024-04-21T08:50:42.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.002+0000 7f9e064ba640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e0006a840 0x7f9e001986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.002+0000 7f9e064ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e00107270 0x7f9e00198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.002+0000 7f9e064ba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e0019cf90 0x7f9e0019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.002+0000 7f9e064ba640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e001121d0 con 0x7f9e0019cf90 2024-04-21T08:50:42.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.002+0000 7f9e064ba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9e00112050 con 0x7f9e00107270 2024-04-21T08:50:42.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.002+0000 7f9e064ba640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9e00112350 con 0x7f9e0006a840 2024-04-21T08:50:42.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.003+0000 7f9e04a57640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e0019cf90 0x7f9e0019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.003+0000 7f9e04a57640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e0019cf90 0x7f9e0019d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50172/0 (socket says 172.21.15.43:50172) 2024-04-21T08:50:42.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.003+0000 7f9e04a57640 1 -- 172.21.15.43:0/115142709 learned_addr learned my addr 172.21.15.43:0/115142709 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:42.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.003+0000 7f9dfffff640 1 --2- 172.21.15.43:0/115142709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e0006a840 0x7f9e001986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.003+0000 7f9dff7fe640 1 --2- 172.21.15.43:0/115142709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e00107270 0x7f9e00198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.003+0000 7f9e04a57640 1 -- 172.21.15.43:0/115142709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e0006a840 msgr2=0x7f9e001986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:42.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.003+0000 7f9e04a57640 1 --2- 172.21.15.43:0/115142709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e0006a840 0x7f9e001986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.003+0000 7f9e04a57640 1 -- 172.21.15.43:0/115142709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e00107270 msgr2=0x7f9e00198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:42.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.003+0000 7f9e04a57640 1 --2- 172.21.15.43:0/115142709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e00107270 0x7f9e00198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.003+0000 7f9e04a57640 1 -- 172.21.15.43:0/115142709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0019da80 con 0x7f9e0019cf90 2024-04-21T08:50:42.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.003+0000 7f9dff7fe640 1 --2- 172.21.15.43:0/115142709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e00107270 0x7f9e00198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:42.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.003+0000 7f9e04a57640 1 --2- 172.21.15.43:0/115142709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e0019cf90 0x7f9e0019d440 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f9df8005e30 tx=0x7f9df8034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:42.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.003+0000 7f9dfd7fa640 1 -- 172.21.15.43:0/115142709 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df8041020 con 0x7f9e0019cf90 2024-04-21T08:50:42.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.003+0000 7f9e064ba640 1 -- 172.21.15.43:0/115142709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e0019dd10 con 0x7f9e0019cf90 2024-04-21T08:50:42.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.003+0000 7f9dfd7fa640 1 -- 172.21.15.43:0/115142709 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9df8032970 con 0x7f9e0019cf90 2024-04-21T08:50:42.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.003+0000 7f9dfd7fa640 1 -- 172.21.15.43:0/115142709 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df8039040 con 0x7f9e0019cf90 2024-04-21T08:50:42.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.003+0000 7f9e064ba640 1 -- 172.21.15.43:0/115142709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e001a54d0 con 0x7f9e0019cf90 2024-04-21T08:50:42.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.006+0000 7f9dfd7fa640 1 -- 172.21.15.43:0/115142709 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9df8044020 con 0x7f9e0019cf90 2024-04-21T08:50:42.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.006+0000 7f9e09184740 1 Processor -- start 2024-04-21T08:50:42.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.006+0000 7f9e09184740 1 -- start start 2024-04-21T08:50:42.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.006+0000 7f9e09184740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cdf7daf8e0 0x55cdf7dafcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.006+0000 7f9e09184740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdf7db0290 0x55cdf7db8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.006+0000 7f9e09184740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7db8e10 0x55cdf7dbb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.006+0000 7f9e09184740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cdf7d13f00 con 0x55cdf7db8e10 2024-04-21T08:50:42.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.006+0000 7f9e09184740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cdf7cb1590 con 0x55cdf7db0290 2024-04-21T08:50:42.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.006+0000 7f9e09184740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55cdf7d0fec0 con 0x55cdf7daf8e0 2024-04-21T08:50:42.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.007+0000 7f9dff7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7db8e10 0x55cdf7dbb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.007+0000 7f9dfd7fa640 1 --2- 172.21.15.43:0/115142709 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9de80a3d20 0x7f9de80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.007+0000 7f9dff7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7db8e10 0x55cdf7dbb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50180/0 (socket says 172.21.15.43:50180) 2024-04-21T08:50:42.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.007+0000 7f9dff7fe640 1 -- 172.21.15.43:0/2329201761 learned_addr learned my addr 172.21.15.43:0/2329201761 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:42.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.007+0000 7f9dfffff640 1 --2- 172.21.15.43:0/2329201761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdf7db0290 0x55cdf7db8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.007+0000 7f9dff7fe640 1 --2- 172.21.15.43:0/2329201761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cdf7daf8e0 0x55cdf7dafcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.007+0000 7f9e04a57640 1 --2- 172.21.15.43:0/115142709 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9de80a3d20 0x7f9de80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.007+0000 7f9dfd7fa640 1 -- 172.21.15.43:0/115142709 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f9df80f29b0 con 0x7f9e0019cf90 2024-04-21T08:50:42.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.007+0000 7f9dff7fe640 1 -- 172.21.15.43:0/2329201761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cdf7daf8e0 msgr2=0x55cdf7dafcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:42.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.007+0000 7f9dff7fe640 1 --2- 172.21.15.43:0/2329201761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cdf7daf8e0 0x55cdf7dafcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.007+0000 7f9dff7fe640 1 -- 172.21.15.43:0/2329201761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdf7db0290 msgr2=0x55cdf7db8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:42.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.007+0000 7f9dff7fe640 1 --2- 172.21.15.43:0/2329201761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdf7db0290 0x55cdf7db8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.007+0000 7f9dff7fe640 1 -- 172.21.15.43:0/2329201761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdf7aee7e0 con 0x55cdf7db8e10 2024-04-21T08:50:42.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.007+0000 7f9dff7fe640 1 --2- 172.21.15.43:0/2329201761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7db8e10 0x55cdf7dbb260 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f9df40149a0 tx=0x7f9df400c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abee91fbc7aca2bf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:42.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.007+0000 7f9e04a57640 1 --2- 172.21.15.43:0/115142709 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9de80a3d20 0x7f9de80a61e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9df8006010 tx=0x7f9df8091130 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:42.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.007+0000 7f9de67fc640 1 -- 172.21.15.43:0/2329201761 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df40142d0 con 0x55cdf7db8e10 2024-04-21T08:50:42.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.007+0000 7f9de67fc640 1 -- 172.21.15.43:0/2329201761 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9df40054d0 con 0x55cdf7db8e10 2024-04-21T08:50:42.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.007+0000 7f9de67fc640 1 -- 172.21.15.43:0/2329201761 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df4004e50 con 0x55cdf7db8e10 2024-04-21T08:50:42.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.008+0000 7f9e09184740 1 -- 172.21.15.43:0/2329201761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7db8e10 msgr2=0x55cdf7dbb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:42.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.008+0000 7f9e09184740 1 --2- 172.21.15.43:0/2329201761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7db8e10 0x55cdf7dbb260 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f9df40149a0 tx=0x7f9df400c4d0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.008+0000 7f9e09184740 1 -- 172.21.15.43:0/2329201761 shutdown_connections 2024-04-21T08:50:42.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.008+0000 7f9e09184740 1 --2- 172.21.15.43:0/2329201761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7db8e10 0x55cdf7dbb260 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.008+0000 7f9e09184740 1 --2- 172.21.15.43:0/2329201761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdf7db0290 0x55cdf7db8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.008+0000 7f9e09184740 1 --2- 172.21.15.43:0/2329201761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cdf7daf8e0 0x55cdf7dafcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.008+0000 7f9e09184740 1 -- 172.21.15.43:0/2329201761 >> 172.21.15.43:0/2329201761 conn(0x55cdf7da9fb0 msgr2=0x55cdf7daa390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:42.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.008+0000 7f9e09184740 1 -- 172.21.15.43:0/2329201761 shutdown_connections 2024-04-21T08:50:42.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.008+0000 7f9e09184740 1 -- 172.21.15.43:0/2329201761 wait complete. 2024-04-21T08:50:42.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.009+0000 7f9e09184740 1 -- start start 2024-04-21T08:50:42.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.009+0000 7f9e09184740 1 Processor -- start 2024-04-21T08:50:42.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.009+0000 7f9e09184740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7daf8e0 0x55cdf7db5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.009+0000 7f9e09184740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cdf7db0290 0x55cdf7db62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.009+0000 7f9e09184740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdf7db8e10 0x55cdf7db6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.009+0000 7f9e09184740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cdf7dbbd20 con 0x55cdf7daf8e0 2024-04-21T08:50:42.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.009+0000 7f9e09184740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cdf7ccd580 con 0x55cdf7db8e10 2024-04-21T08:50:42.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.009+0000 7f9e09184740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55cdf7dbbea0 con 0x55cdf7db0290 2024-04-21T08:50:42.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.009+0000 7f9e09184740 10 client.0 _ll_register_callbacks cb 0x55cdf7cd3650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:42.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.009+0000 7f9dff7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7daf8e0 0x55cdf7db5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.009+0000 7f9dff7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7daf8e0 0x55cdf7db5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50182/0 (socket says 172.21.15.43:50182) 2024-04-21T08:50:42.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.009+0000 7f9dff7fe640 1 -- 172.21.15.43:0/1596791419 learned_addr learned my addr 172.21.15.43:0/1596791419 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:42.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.010+0000 7f9dfffff640 1 --2- 172.21.15.43:0/1596791419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cdf7db0290 0x55cdf7db62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.010+0000 7f9dff7fe640 1 -- 172.21.15.43:0/1596791419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cdf7db0290 msgr2=0x55cdf7db62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:42.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.010+0000 7f9dff7fe640 1 --2- 172.21.15.43:0/1596791419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cdf7db0290 0x55cdf7db62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.010+0000 7f9dff7fe640 1 -- 172.21.15.43:0/1596791419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdf7db8e10 msgr2=0x55cdf7db6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:42.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.010+0000 7f9dff7fe640 1 --2- 172.21.15.43:0/1596791419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdf7db8e10 0x55cdf7db6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.010+0000 7f9dff7fe640 1 -- 172.21.15.43:0/1596791419 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdf7b4ef70 con 0x55cdf7daf8e0 2024-04-21T08:50:42.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.010+0000 7f9dff7fe640 1 --2- 172.21.15.43:0/1596791419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7daf8e0 0x55cdf7db5db0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f9df40058c0 tx=0x7f9df4004ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5807981155fed5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:42.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.010+0000 7f9ddf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:42.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.010+0000 7f9ddf7fe640 1 -- 172.21.15.43:0/1596791419 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df400f960 con 0x55cdf7daf8e0 2024-04-21T08:50:42.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.010+0000 7f9e09184740 1 -- 172.21.15.43:0/1596791419 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9df40054d0 con 0x55cdf7daf8e0 2024-04-21T08:50:42.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.010+0000 7f9ddf7fe640 1 -- 172.21.15.43:0/1596791419 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f9df400fb00 con 0x55cdf7daf8e0 2024-04-21T08:50:42.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.010+0000 7f9ddf7fe640 1 -- 172.21.15.43:0/1596791419 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9df402f050 con 0x55cdf7daf8e0 2024-04-21T08:50:42.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.010+0000 7f9e09184740 10 client.21174 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:42.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.010+0000 7f9e09184740 1 -- 172.21.15.43:0/1596791419 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cdf79e6170 con 0x55cdf7daf8e0 2024-04-21T08:50:42.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.010+0000 7f9ddf7fe640 1 -- 172.21.15.43:0/1596791419 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9df40054d0 con 0x55cdf7daf8e0 2024-04-21T08:50:42.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.011+0000 7f9ddf7fe640 1 -- 172.21.15.43:0/1596791419 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9df40124e0 con 0x55cdf7daf8e0 2024-04-21T08:50:42.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.011+0000 7f9e09184740 10 client.21174 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:42.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.011+0000 7f9e09184740 10 client.21174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:42.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.011+0000 7f9e09184740 20 client.21174 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:42.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.011+0000 7f9e09184740 1 --2- 172.21.15.43:0/1596791419 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cdf7dc92b0 0x55cdf7dcb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.011+0000 7f9e09184740 4 client.21174 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:42.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.011+0000 7f9e09184740 1 -- 172.21.15.43:0/1596791419 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdf7d21920 con 0x55cdf7dc92b0 2024-04-21T08:50:42.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.011+0000 7f9dfffff640 1 --2- 172.21.15.43:0/1596791419 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cdf7dc92b0 0x55cdf7dcb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.011+0000 7f9dfffff640 1 --2- 172.21.15.43:0/1596791419 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cdf7dc92b0 0x55cdf7dcb6a0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da9d4bbf2f3e5497 server_cookie=e2776cca13348ad3 in_seq=0 out_seq=0 2024-04-21T08:50:42.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.011+0000 7f9ddf7fe640 10 client.21174 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:42.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.012+0000 7f9ddf7fe640 1 -- 172.21.15.43:0/1596791419 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cdf7d21920 con 0x55cdf7dc92b0 2024-04-21T08:50:42.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.012+0000 7f9ddf7fe640 10 client.21174 handle_command_reply: tid=0 2024-04-21T08:50:42.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.012+0000 7f9e09184740 1 client.21174 shutdown 2024-04-21T08:50:42.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.012+0000 7f9e09184740 1 -- 172.21.15.43:0/1596791419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7daf8e0 msgr2=0x55cdf7db5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:42.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.012+0000 7f9e09184740 1 --2- 172.21.15.43:0/1596791419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7daf8e0 0x55cdf7db5db0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f9df40058c0 tx=0x7f9df4004ae0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.012+0000 7f9e09184740 1 -- 172.21.15.43:0/1596791419 shutdown_connections 2024-04-21T08:50:42.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.012+0000 7f9e09184740 1 --2- 172.21.15.43:0/1596791419 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cdf7dc92b0 0x55cdf7dcb6a0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.012+0000 7f9e09184740 1 --2- 172.21.15.43:0/1596791419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdf7db8e10 0x55cdf7db6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.012+0000 7f9e09184740 1 --2- 172.21.15.43:0/1596791419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cdf7db0290 0x55cdf7db62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.012+0000 7f9ddf7fe640 0 client.21174 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:42.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.012+0000 7f9e09184740 1 --2- 172.21.15.43:0/1596791419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7daf8e0 0x55cdf7db5db0 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.012+0000 7f9e09184740 1 -- 172.21.15.43:0/1596791419 >> 172.21.15.43:0/1596791419 conn(0x55cdf7da9fb0 msgr2=0x55cdf7dabd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:42.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.013+0000 7f9e09184740 1 -- 172.21.15.43:0/1596791419 shutdown_connections 2024-04-21T08:50:42.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.013+0000 7f9e09184740 1 -- 172.21.15.43:0/1596791419 wait complete. 2024-04-21T08:50:42.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.013+0000 7f9e09184740 20 client.21174 trim_cache size 0 max 16384 2024-04-21T08:50:42.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.030+0000 7f9e09184740 1 Processor -- start 2024-04-21T08:50:42.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.030+0000 7f9e09184740 1 -- start start 2024-04-21T08:50:42.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.031+0000 7f9e09184740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdf7daf8e0 0x55cdf7db84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.031+0000 7f9e09184740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7db0290 0x55cdf7dc4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.031+0000 7f9e09184740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cdf7db8e10 0x55cdf7db4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.031+0000 7f9e09184740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e001af4a0 con 0x55cdf7db0290 2024-04-21T08:50:42.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.031+0000 7f9e09184740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9e001b0fe0 con 0x55cdf7daf8e0 2024-04-21T08:50:42.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.031+0000 7f9e09184740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9e001afb70 con 0x55cdf7db8e10 2024-04-21T08:50:42.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.031+0000 7f9dfffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7db0290 0x55cdf7dc4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.031+0000 7f9dfffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7db0290 0x55cdf7dc4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50192/0 (socket says 172.21.15.43:50192) 2024-04-21T08:50:42.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.031+0000 7f9dfffff640 1 -- 172.21.15.43:0/210726932 learned_addr learned my addr 172.21.15.43:0/210726932 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:42.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.031+0000 7f9dff7fe640 1 --2- 172.21.15.43:0/210726932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdf7daf8e0 0x55cdf7db84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.031+0000 7f9dfffff640 1 -- 172.21.15.43:0/210726932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cdf7db8e10 msgr2=0x55cdf7db4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:42.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.031+0000 7f9dff7fe640 1 --2- 172.21.15.43:0/210726932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cdf7db8e10 0x55cdf7db4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.031+0000 7f9dfffff640 1 --2- 172.21.15.43:0/210726932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cdf7db8e10 0x55cdf7db4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.031+0000 7f9dfffff640 1 -- 172.21.15.43:0/210726932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdf7daf8e0 msgr2=0x55cdf7db84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:42.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.031+0000 7f9dfffff640 1 --2- 172.21.15.43:0/210726932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdf7daf8e0 0x55cdf7db84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.031+0000 7f9dfffff640 1 -- 172.21.15.43:0/210726932 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdf7daa7f0 con 0x55cdf7db0290 2024-04-21T08:50:42.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.031+0000 7f9dff7fe640 1 --2- 172.21.15.43:0/210726932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cdf7db8e10 0x55cdf7db4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:42.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.031+0000 7f9dfffff640 1 --2- 172.21.15.43:0/210726932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7db0290 0x55cdf7dc4c40 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f9df0000c00 tx=0x7f9df0013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e1a41ae2360475 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:42.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.031+0000 7f9de5ffb640 1 -- 172.21.15.43:0/210726932 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df001d430 con 0x55cdf7db0290 2024-04-21T08:50:42.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.032+0000 7f9de5ffb640 1 -- 172.21.15.43:0/210726932 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9df0011440 con 0x55cdf7db0290 2024-04-21T08:50:42.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.032+0000 7f9de5ffb640 1 -- 172.21.15.43:0/210726932 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df001c630 con 0x55cdf7db0290 2024-04-21T08:50:42.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.032+0000 7f9e09184740 1 -- 172.21.15.43:0/210726932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7db0290 msgr2=0x55cdf7dc4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:42.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.032+0000 7f9e09184740 1 --2- 172.21.15.43:0/210726932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7db0290 0x55cdf7dc4c40 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f9df0000c00 tx=0x7f9df0013830 comp rx=0 tx=0).stop 2024-04-21T08:50:42.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.032+0000 7f9e09184740 1 -- 172.21.15.43:0/210726932 shutdown_connections 2024-04-21T08:50:42.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.032+0000 7f9e09184740 1 --2- 172.21.15.43:0/210726932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cdf7db8e10 0x55cdf7db4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.032+0000 7f9e09184740 1 --2- 172.21.15.43:0/210726932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7db0290 0x55cdf7dc4c40 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.032+0000 7f9e09184740 1 --2- 172.21.15.43:0/210726932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdf7daf8e0 0x55cdf7db84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.032+0000 7f9e09184740 1 -- 172.21.15.43:0/210726932 >> 172.21.15.43:0/210726932 conn(0x55cdf7da9fb0 msgr2=0x55cdf7da8780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:42.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.032+0000 7f9e09184740 1 -- 172.21.15.43:0/210726932 shutdown_connections 2024-04-21T08:50:42.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.032+0000 7f9e09184740 1 -- 172.21.15.43:0/210726932 wait complete. 2024-04-21T08:50:42.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.033+0000 7f9e09184740 1 -- start start 2024-04-21T08:50:42.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.033+0000 7f9e09184740 1 Processor -- start 2024-04-21T08:50:42.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.033+0000 7f9e09184740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7daf8e0 0x55cdf7dbcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.033+0000 7f9e09184740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cdf7db0290 0x55cdf7dbd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.033+0000 7f9e09184740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdf7db8e10 0x55cdf7dbd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.033+0000 7f9e09184740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e001b0910 con 0x55cdf7daf8e0 2024-04-21T08:50:42.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.033+0000 7f9e09184740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9e001b0240 con 0x55cdf7db8e10 2024-04-21T08:50:42.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.033+0000 7f9e09184740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9e001ad4a0 con 0x55cdf7db0290 2024-04-21T08:50:42.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.033+0000 7f9e09184740 10 client.0 _ll_register_callbacks cb 0x7f9df000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:42.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.033+0000 7f9dff7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7daf8e0 0x55cdf7dbcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.033+0000 7f9dff7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7daf8e0 0x55cdf7dbcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50202/0 (socket says 172.21.15.43:50202) 2024-04-21T08:50:42.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.033+0000 7f9dff7fe640 1 -- 172.21.15.43:0/3186882814 learned_addr learned my addr 172.21.15.43:0/3186882814 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:42.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.034+0000 7f9dff7fe640 1 -- 172.21.15.43:0/3186882814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cdf7db0290 msgr2=0x55cdf7dbd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:42.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.034+0000 7f9dfffff640 1 --2- 172.21.15.43:0/3186882814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cdf7db0290 0x55cdf7dbd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.034+0000 7f9dff7fe640 1 --2- 172.21.15.43:0/3186882814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cdf7db0290 0x55cdf7dbd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.034+0000 7f9dff7fe640 1 -- 172.21.15.43:0/3186882814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdf7db8e10 msgr2=0x55cdf7dbd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:42.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.034+0000 7f9dff7fe640 1 --2- 172.21.15.43:0/3186882814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdf7db8e10 0x55cdf7dbd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.034+0000 7f9dff7fe640 1 -- 172.21.15.43:0/3186882814 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdf7e2e5e0 con 0x55cdf7daf8e0 2024-04-21T08:50:42.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.034+0000 7f9dfffff640 1 --2- 172.21.15.43:0/3186882814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cdf7db0290 0x55cdf7dbd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:42.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.034+0000 7f9dff7fe640 1 --2- 172.21.15.43:0/3186882814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7daf8e0 0x55cdf7dbcca0 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f9df4005fd0 tx=0x7f9df40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69615b7dc7f75f1d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:42.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.034+0000 7f9ddf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:42.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.034+0000 7f9ddf7fe640 1 -- 172.21.15.43:0/3186882814 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df4012450 con 0x55cdf7daf8e0 2024-04-21T08:50:42.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.034+0000 7f9ddf7fe640 1 -- 172.21.15.43:0/3186882814 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f9df40125f0 con 0x55cdf7daf8e0 2024-04-21T08:50:42.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.034+0000 7f9e09184740 1 -- 172.21.15.43:0/3186882814 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9df0011440 con 0x55cdf7daf8e0 2024-04-21T08:50:42.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.034+0000 7f9ddf7fe640 1 -- 172.21.15.43:0/3186882814 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9df4044050 con 0x55cdf7daf8e0 2024-04-21T08:50:42.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.034+0000 7f9e09184740 10 client.21180 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:42.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.034+0000 7f9e09184740 1 -- 172.21.15.43:0/3186882814 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cdf7e2f890 con 0x55cdf7daf8e0 2024-04-21T08:50:42.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.034+0000 7f9ddf7fe640 1 -- 172.21.15.43:0/3186882814 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9df0011440 con 0x55cdf7daf8e0 2024-04-21T08:50:42.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.034+0000 7f9ddf7fe640 1 -- 172.21.15.43:0/3186882814 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9df400fb00 con 0x55cdf7daf8e0 2024-04-21T08:50:42.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.034+0000 7f9e09184740 10 client.21180 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:42.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.035+0000 7f9e09184740 10 client.21180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:42.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.035+0000 7f9e09184740 20 client.21180 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:42.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.035+0000 7f9e09184740 1 --2- 172.21.15.43:0/3186882814 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cdf7e2ff30 0x55cdf7e32350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.035+0000 7f9e09184740 4 client.21180 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:50:42.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.035+0000 7f9e09184740 1 -- 172.21.15.43:0/3186882814 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cdf7d1b790 con 0x55cdf7e2ff30 2024-04-21T08:50:42.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.035+0000 7f9dfffff640 1 --2- 172.21.15.43:0/3186882814 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cdf7e2ff30 0x55cdf7e32350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.035+0000 7f9dfffff640 1 --2- 172.21.15.43:0/3186882814 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cdf7e2ff30 0x55cdf7e32350 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b1a6e29dc04ba30 server_cookie=37c9b00cd5644748 in_seq=0 out_seq=0 2024-04-21T08:50:42.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.035+0000 7f9ddf7fe640 10 client.21180 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:42.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.036+0000 7f9ddf7fe640 1 -- 172.21.15.43:0/3186882814 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cdf7d1b790 con 0x55cdf7e2ff30 2024-04-21T08:50:42.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.036+0000 7f9ddf7fe640 10 client.21180 handle_command_reply: tid=0 2024-04-21T08:50:42.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.036+0000 7f9e09184740 1 client.21180 shutdown 2024-04-21T08:50:42.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.037+0000 7f9e09184740 1 -- 172.21.15.43:0/3186882814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7daf8e0 msgr2=0x55cdf7dbcca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:42.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.037+0000 7f9e09184740 1 --2- 172.21.15.43:0/3186882814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7daf8e0 0x55cdf7dbcca0 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f9df4005fd0 tx=0x7f9df40048f0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.037+0000 7f9e09184740 1 -- 172.21.15.43:0/3186882814 shutdown_connections 2024-04-21T08:50:42.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.037+0000 7f9e09184740 1 --2- 172.21.15.43:0/3186882814 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cdf7e2ff30 0x55cdf7e32350 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.037+0000 7f9e09184740 1 --2- 172.21.15.43:0/3186882814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdf7db8e10 0x55cdf7dbd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.037+0000 7f9e09184740 1 --2- 172.21.15.43:0/3186882814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cdf7db0290 0x55cdf7dbd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.037+0000 7f9ddf7fe640 0 client.21180 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:42.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.037+0000 7f9e09184740 1 --2- 172.21.15.43:0/3186882814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cdf7daf8e0 0x55cdf7dbcca0 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.037+0000 7f9e09184740 1 -- 172.21.15.43:0/3186882814 >> 172.21.15.43:0/3186882814 conn(0x55cdf7da9fb0 msgr2=0x55cdf7dab3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:42.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.037+0000 7f9e09184740 1 -- 172.21.15.43:0/3186882814 shutdown_connections 2024-04-21T08:50:42.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.037+0000 7f9e09184740 1 -- 172.21.15.43:0/3186882814 wait complete. 2024-04-21T08:50:42.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.037+0000 7f9e09184740 20 client.21180 trim_cache size 0 max 16384 2024-04-21T08:50:42.042 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:50:42.042 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:50:42.043 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "4c2cd53e-4a67-460f-9de0-5ddc4d75e01a", 2024-04-21T08:50:42.043 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:50:42.043 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:50:42.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.037+0000 7f9e064ba640 1 -- 172.21.15.43:0/115142709 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9de80a3d20 msgr2=0x7f9de80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:42.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.037+0000 7f9e064ba640 1 --2- 172.21.15.43:0/115142709 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9de80a3d20 0x7f9de80a61e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9df8006010 tx=0x7f9df8091130 comp rx=0 tx=0).stop 2024-04-21T08:50:42.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.037+0000 7f9e064ba640 1 -- 172.21.15.43:0/115142709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e0019cf90 msgr2=0x7f9e0019d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:42.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.037+0000 7f9e064ba640 1 --2- 172.21.15.43:0/115142709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e0019cf90 0x7f9e0019d440 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f9df8005e30 tx=0x7f9df8034670 comp rx=0 tx=0).stop 2024-04-21T08:50:42.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.038+0000 7f9e064ba640 1 -- 172.21.15.43:0/115142709 shutdown_connections 2024-04-21T08:50:42.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.038+0000 7f9e064ba640 1 --2- 172.21.15.43:0/115142709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e0019cf90 0x7f9e0019d440 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.038+0000 7f9e064ba640 1 --2- 172.21.15.43:0/115142709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e00107270 0x7f9e00198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.038+0000 7f9e064ba640 1 --2- 172.21.15.43:0/115142709 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f9de80a3d20 0x7f9de80a61e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.038+0000 7f9e064ba640 1 --2- 172.21.15.43:0/115142709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e0006a840 0x7f9e001986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.038+0000 7f9e064ba640 1 -- 172.21.15.43:0/115142709 >> 172.21.15.43:0/115142709 conn(0x7f9e00077eb0 msgr2=0x7f9e00100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:42.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.038+0000 7f9e064ba640 1 -- 172.21.15.43:0/115142709 shutdown_connections 2024-04-21T08:50:42.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.038+0000 7f9e064ba640 1 -- 172.21.15.43:0/115142709 wait complete. 2024-04-21T08:50:42.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:50:42.049 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:50:42.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.153+0000 7f2408a32640 1 -- 172.21.15.43:0/3643100776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2404101a50 msgr2=0x7f2404105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:42.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.153+0000 7f2408a32640 1 --2- 172.21.15.43:0/3643100776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2404101a50 0x7f2404105a90 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f23f8009900 tx=0x7f23f802bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:42.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.153+0000 7f2408a32640 1 -- 172.21.15.43:0/3643100776 shutdown_connections 2024-04-21T08:50:42.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.153+0000 7f2408a32640 1 --2- 172.21.15.43:0/3643100776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2404106170 0x7f240410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.153+0000 7f2408a32640 1 --2- 172.21.15.43:0/3643100776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2404101a50 0x7f2404105a90 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.153+0000 7f2408a32640 1 --2- 172.21.15.43:0/3643100776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24041010a0 0x7f2404101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.153+0000 7f2408a32640 1 -- 172.21.15.43:0/3643100776 >> 172.21.15.43:0/3643100776 conn(0x7f2404077f70 msgr2=0x7f2404078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:42.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.153+0000 7f2408a32640 1 -- 172.21.15.43:0/3643100776 shutdown_connections 2024-04-21T08:50:42.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.153+0000 7f2408a32640 1 -- 172.21.15.43:0/3643100776 wait complete. 2024-04-21T08:50:42.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.154+0000 7f2408a32640 1 Processor -- start 2024-04-21T08:50:42.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.154+0000 7f2408a32640 1 -- start start 2024-04-21T08:50:42.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.154+0000 7f2408a32640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24041010a0 0x7f24041987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.154+0000 7f2408a32640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2404106170 0x7f2404198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.154+0000 7f2408a32640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f240419bfb0 0x7f240419c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.154+0000 7f2408a32640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24041110b0 con 0x7f24041010a0 2024-04-21T08:50:42.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.154+0000 7f2408a32640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2404110f30 con 0x7f240419bfb0 2024-04-21T08:50:42.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.154+0000 7f2408a32640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2404111230 con 0x7f2404106170 2024-04-21T08:50:42.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.154+0000 7f240259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24041010a0 0x7f24041987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.154+0000 7f240259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24041010a0 0x7f24041987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50230/0 (socket says 172.21.15.43:50230) 2024-04-21T08:50:42.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.154+0000 7f240259c640 1 -- 172.21.15.43:0/3422257851 learned_addr learned my addr 172.21.15.43:0/3422257851 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:42.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.154+0000 7f2402d9d640 1 --2- 172.21.15.43:0/3422257851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f240419bfb0 0x7f240419c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.154+0000 7f240259c640 1 -- 172.21.15.43:0/3422257851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2404106170 msgr2=0x7f2404198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:42.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.154+0000 7f240259c640 1 --2- 172.21.15.43:0/3422257851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2404106170 0x7f2404198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.154+0000 7f240259c640 1 -- 172.21.15.43:0/3422257851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f240419bfb0 msgr2=0x7f240419c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:42.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.154+0000 7f240259c640 1 --2- 172.21.15.43:0/3422257851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f240419bfb0 0x7f240419c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.154+0000 7f240259c640 1 -- 172.21.15.43:0/3422257851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240419caa0 con 0x7f24041010a0 2024-04-21T08:50:42.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.155+0000 7f240259c640 1 --2- 172.21.15.43:0/3422257851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24041010a0 0x7f24041987a0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f23f400d770 tx=0x7f23f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:42.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.155+0000 7f23f37fe640 1 -- 172.21.15.43:0/3422257851 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4017070 con 0x7f24041010a0 2024-04-21T08:50:42.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.155+0000 7f23f37fe640 1 -- 172.21.15.43:0/3422257851 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23f40054d0 con 0x7f24041010a0 2024-04-21T08:50:42.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.155+0000 7f2408a32640 1 -- 172.21.15.43:0/3422257851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f240419cd90 con 0x7f24041010a0 2024-04-21T08:50:42.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.155+0000 7f23f37fe640 1 -- 172.21.15.43:0/3422257851 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4004c30 con 0x7f24041010a0 2024-04-21T08:50:42.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.155+0000 7f2408a32640 1 -- 172.21.15.43:0/3422257851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24041a55e0 con 0x7f24041010a0 2024-04-21T08:50:42.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.157+0000 7f23f37fe640 1 -- 172.21.15.43:0/3422257851 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f23f4004e10 con 0x7f24041010a0 2024-04-21T08:50:42.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.157+0000 7f240b61a740 1 Processor -- start 2024-04-21T08:50:42.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.157+0000 7f240b61a740 1 -- start start 2024-04-21T08:50:42.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.158+0000 7f240b61a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796cc580 0x5574796cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.158+0000 7f240b61a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574796ccea0 0x5574796d5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.158+0000 7f240b61a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574796d5a80 0x5574796d7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.158+0000 7f240b61a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5574796311e0 con 0x5574796cc580 2024-04-21T08:50:42.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.158+0000 7f240b61a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5574795ce390 con 0x5574796d5a80 2024-04-21T08:50:42.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.158+0000 7f240b61a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55747962d1a0 con 0x5574796ccea0 2024-04-21T08:50:42.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.158+0000 7f23f37fe640 1 --2- 172.21.15.43:0/3422257851 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f23ec0a3d20 0x7f23ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.158+0000 7f2402d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796cc580 0x5574796cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.158+0000 7f2402d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796cc580 0x5574796cc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50232/0 (socket says 172.21.15.43:50232) 2024-04-21T08:50:42.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.158+0000 7f2402d9d640 1 -- 172.21.15.43:0/3416460970 learned_addr learned my addr 172.21.15.43:0/3416460970 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:42.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.158+0000 7f2402d9d640 1 --2- 172.21.15.43:0/3422257851 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f23ec0a3d20 0x7f23ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.158+0000 7f2401d9b640 1 --2- 172.21.15.43:0/3416460970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574796d5a80 0x5574796d7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.158+0000 7f23f37fe640 1 -- 172.21.15.43:0/3422257851 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f23f40d07a0 con 0x7f24041010a0 2024-04-21T08:50:42.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.158+0000 7f240259c640 1 --2- 172.21.15.43:0/3416460970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574796ccea0 0x5574796d5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.158+0000 7f2402d9d640 1 -- 172.21.15.43:0/3416460970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574796ccea0 msgr2=0x5574796d5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:42.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.158+0000 7f2402d9d640 1 --2- 172.21.15.43:0/3416460970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574796ccea0 0x5574796d5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.158+0000 7f2402d9d640 1 -- 172.21.15.43:0/3416460970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574796d5a80 msgr2=0x5574796d7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:42.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.158+0000 7f2402d9d640 1 --2- 172.21.15.43:0/3416460970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574796d5a80 0x5574796d7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.158+0000 7f2402d9d640 1 -- 172.21.15.43:0/3416460970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55747940d7d0 con 0x5574796cc580 2024-04-21T08:50:42.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.158+0000 7f2402d9d640 1 --2- 172.21.15.43:0/3416460970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796cc580 0x5574796cc960 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f23fc013e70 tx=0x7f23fc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55dbb2803ddab950 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:42.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.158+0000 7f2402d9d640 1 --2- 172.21.15.43:0/3422257851 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f23ec0a3d20 0x7f23ec0a61e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f23fc010ca0 tx=0x7f23fc015000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.159+0000 7f23f0ff9640 1 -- 172.21.15.43:0/3416460970 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc00ae10 con 0x5574796cc580 2024-04-21T08:50:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.159+0000 7f23f0ff9640 1 -- 172.21.15.43:0/3416460970 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23fc00bb70 con 0x5574796cc580 2024-04-21T08:50:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.159+0000 7f23f0ff9640 1 -- 172.21.15.43:0/3416460970 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc00e440 con 0x5574796cc580 2024-04-21T08:50:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.159+0000 7f240b61a740 1 -- 172.21.15.43:0/3416460970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796cc580 msgr2=0x5574796cc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.159+0000 7f240b61a740 1 --2- 172.21.15.43:0/3416460970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796cc580 0x5574796cc960 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f23fc013e70 tx=0x7f23fc0107a0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.159+0000 7f240b61a740 1 -- 172.21.15.43:0/3416460970 shutdown_connections 2024-04-21T08:50:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.159+0000 7f240b61a740 1 --2- 172.21.15.43:0/3416460970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574796d5a80 0x5574796d7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.159+0000 7f240b61a740 1 --2- 172.21.15.43:0/3416460970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574796ccea0 0x5574796d5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.159+0000 7f240b61a740 1 --2- 172.21.15.43:0/3416460970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796cc580 0x5574796cc960 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.159+0000 7f240b61a740 1 -- 172.21.15.43:0/3416460970 >> 172.21.15.43:0/3416460970 conn(0x5574796c6cb0 msgr2=0x5574796c7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.159+0000 7f240b61a740 1 -- 172.21.15.43:0/3416460970 shutdown_connections 2024-04-21T08:50:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.159+0000 7f240b61a740 1 -- 172.21.15.43:0/3416460970 wait complete. 2024-04-21T08:50:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.160+0000 7f240b61a740 1 -- start start 2024-04-21T08:50:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.160+0000 7f240b61a740 1 Processor -- start 2024-04-21T08:50:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.160+0000 7f240b61a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574796cc580 0x5574796d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.160+0000 7f240b61a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796ccea0 0x5574796d3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.160+0000 7f240b61a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574796d5a80 0x5574796d35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.160+0000 7f240b61a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5574796d8810 con 0x5574796ccea0 2024-04-21T08:50:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.160+0000 7f240b61a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5574795ea2f0 con 0x5574796cc580 2024-04-21T08:50:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.160+0000 7f240b61a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5574796d8990 con 0x5574796d5a80 2024-04-21T08:50:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.160+0000 7f240b61a740 10 client.0 _ll_register_callbacks cb 0x5574795f7580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.160+0000 7f2401d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796ccea0 0x5574796d3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.160+0000 7f2401d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796ccea0 0x5574796d3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50246/0 (socket says 172.21.15.43:50246) 2024-04-21T08:50:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.160+0000 7f2401d9b640 1 -- 172.21.15.43:0/3633428957 learned_addr learned my addr 172.21.15.43:0/3633428957 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.160+0000 7f2402d9d640 1 --2- 172.21.15.43:0/3633428957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574796d5a80 0x5574796d35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.160+0000 7f240259c640 1 --2- 172.21.15.43:0/3633428957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574796cc580 0x5574796d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.160+0000 7f2401d9b640 1 -- 172.21.15.43:0/3633428957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574796d5a80 msgr2=0x5574796d35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.160+0000 7f2401d9b640 1 --2- 172.21.15.43:0/3633428957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574796d5a80 0x5574796d35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.160+0000 7f2401d9b640 1 -- 172.21.15.43:0/3633428957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574796cc580 msgr2=0x5574796d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.160+0000 7f2401d9b640 1 --2- 172.21.15.43:0/3633428957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574796cc580 0x5574796d2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.160+0000 7f2401d9b640 1 -- 172.21.15.43:0/3633428957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574793433f0 con 0x5574796ccea0 2024-04-21T08:50:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.161+0000 7f2401d9b640 1 --2- 172.21.15.43:0/3633428957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796ccea0 0x5574796d3060 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f23f8005ab0 tx=0x7f23f802ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9212b17fccfd17 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.161+0000 7f23e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.161+0000 7f23e9ffb640 1 -- 172.21.15.43:0/3633428957 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f8044070 con 0x5574796ccea0 2024-04-21T08:50:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.161+0000 7f23e9ffb640 1 -- 172.21.15.43:0/3633428957 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f23f80037f0 con 0x5574796ccea0 2024-04-21T08:50:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.161+0000 7f240b61a740 1 -- 172.21.15.43:0/3633428957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55747963ec00 con 0x5574796ccea0 2024-04-21T08:50:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.161+0000 7f23e9ffb640 1 -- 172.21.15.43:0/3633428957 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f23f8050050 con 0x5574796ccea0 2024-04-21T08:50:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.161+0000 7f240b61a740 10 client.21192 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.161+0000 7f240b61a740 1 -- 172.21.15.43:0/3633428957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5574793030f0 con 0x5574796ccea0 2024-04-21T08:50:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.161+0000 7f23e9ffb640 1 -- 172.21.15.43:0/3633428957 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55747963ec00 con 0x5574796ccea0 2024-04-21T08:50:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.161+0000 7f23e9ffb640 1 -- 172.21.15.43:0/3633428957 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f23f8034740 con 0x5574796ccea0 2024-04-21T08:50:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.161+0000 7f240b61a740 10 client.21192 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.161+0000 7f240b61a740 10 client.21192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.161+0000 7f240b61a740 20 client.21192 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.161+0000 7f240b61a740 1 --2- 172.21.15.43:0/3633428957 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5574796e5930 0x5574796e7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.161+0000 7f240b61a740 4 client.21192 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.161+0000 7f240b61a740 1 -- 172.21.15.43:0/3633428957 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557479638a70 con 0x5574796e5930 2024-04-21T08:50:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.162+0000 7f240259c640 1 --2- 172.21.15.43:0/3633428957 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5574796e5930 0x5574796e7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.162+0000 7f23e9ffb640 10 client.21192 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.162+0000 7f240259c640 1 --2- 172.21.15.43:0/3633428957 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5574796e5930 0x5574796e7d80 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ef4cc5bb7578e98 server_cookie=1068f5977403d6c7 in_seq=0 out_seq=0 2024-04-21T08:50:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.163+0000 7f23e9ffb640 1 -- 172.21.15.43:0/3633428957 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557479638a70 con 0x5574796e5930 2024-04-21T08:50:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.163+0000 7f23e9ffb640 10 client.21192 handle_command_reply: tid=0 2024-04-21T08:50:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.163+0000 7f240b61a740 1 client.21192 shutdown 2024-04-21T08:50:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.163+0000 7f240b61a740 1 -- 172.21.15.43:0/3633428957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796ccea0 msgr2=0x5574796d3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.163+0000 7f240b61a740 1 --2- 172.21.15.43:0/3633428957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796ccea0 0x5574796d3060 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f23f8005ab0 tx=0x7f23f802ca10 comp rx=0 tx=0).stop 2024-04-21T08:50:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.163+0000 7f240b61a740 1 -- 172.21.15.43:0/3633428957 shutdown_connections 2024-04-21T08:50:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.163+0000 7f240b61a740 1 --2- 172.21.15.43:0/3633428957 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5574796e5930 0x5574796e7d80 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.163+0000 7f240b61a740 1 --2- 172.21.15.43:0/3633428957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574796d5a80 0x5574796d35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.163+0000 7f240b61a740 1 --2- 172.21.15.43:0/3633428957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796ccea0 0x5574796d3060 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.163+0000 7f23e9ffb640 0 client.21192 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.163+0000 7f240b61a740 1 --2- 172.21.15.43:0/3633428957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574796cc580 0x5574796d2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.163+0000 7f240b61a740 1 -- 172.21.15.43:0/3633428957 >> 172.21.15.43:0/3633428957 conn(0x5574796c6cb0 msgr2=0x5574796c8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.163+0000 7f240b61a740 1 -- 172.21.15.43:0/3633428957 shutdown_connections 2024-04-21T08:50:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.163+0000 7f240b61a740 1 -- 172.21.15.43:0/3633428957 wait complete. 2024-04-21T08:50:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.163+0000 7f240b61a740 20 client.21192 trim_cache size 0 max 16384 2024-04-21T08:50:42.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.181+0000 7f240b61a740 1 Processor -- start 2024-04-21T08:50:42.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.181+0000 7f240b61a740 1 -- start start 2024-04-21T08:50:42.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.181+0000 7f240b61a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574796cc580 0x5574796d5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.181+0000 7f240b61a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574796ccea0 0x5574796d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.181+0000 7f240b61a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796d5a80 0x5574796e16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.181+0000 7f240b61a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24041af570 con 0x5574796d5a80 2024-04-21T08:50:42.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.181+0000 7f240b61a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24041b10b0 con 0x5574796ccea0 2024-04-21T08:50:42.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.181+0000 7f240b61a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f24041afc40 con 0x5574796cc580 2024-04-21T08:50:42.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.181+0000 7f2402d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796d5a80 0x5574796e16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.181+0000 7f2402d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796d5a80 0x5574796e16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50248/0 (socket says 172.21.15.43:50248) 2024-04-21T08:50:42.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.181+0000 7f2402d9d640 1 -- 172.21.15.43:0/1512531242 learned_addr learned my addr 172.21.15.43:0/1512531242 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:42.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.181+0000 7f2401d9b640 1 --2- 172.21.15.43:0/1512531242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574796ccea0 0x5574796d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.181+0000 7f240259c640 1 --2- 172.21.15.43:0/1512531242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574796cc580 0x5574796d5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.181+0000 7f2402d9d640 1 -- 172.21.15.43:0/1512531242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574796cc580 msgr2=0x5574796d5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.181+0000 7f2402d9d640 1 --2- 172.21.15.43:0/1512531242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574796cc580 0x5574796d5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.181+0000 7f2402d9d640 1 -- 172.21.15.43:0/1512531242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574796ccea0 msgr2=0x5574796d1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.181+0000 7f2402d9d640 1 --2- 172.21.15.43:0/1512531242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574796ccea0 0x5574796d1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.181+0000 7f2402d9d640 1 -- 172.21.15.43:0/1512531242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574796c67e0 con 0x5574796d5a80 2024-04-21T08:50:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.181+0000 7f2401d9b640 1 --2- 172.21.15.43:0/1512531242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574796ccea0 0x5574796d1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.181+0000 7f2402d9d640 1 --2- 172.21.15.43:0/1512531242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796d5a80 0x5574796e16d0 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f23fc004400 tx=0x7f23fc021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d177e85968925db9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.181+0000 7f23ebfff640 1 -- 172.21.15.43:0/1512531242 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc036070 con 0x5574796d5a80 2024-04-21T08:50:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.182+0000 7f23ebfff640 1 -- 172.21.15.43:0/1512531242 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23fc00ac10 con 0x5574796d5a80 2024-04-21T08:50:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.182+0000 7f23ebfff640 1 -- 172.21.15.43:0/1512531242 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc003bd0 con 0x5574796d5a80 2024-04-21T08:50:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.182+0000 7f240b61a740 1 -- 172.21.15.43:0/1512531242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796d5a80 msgr2=0x5574796e16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.182+0000 7f240b61a740 1 --2- 172.21.15.43:0/1512531242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796d5a80 0x5574796e16d0 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f23fc004400 tx=0x7f23fc021040 comp rx=0 tx=0).stop 2024-04-21T08:50:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.182+0000 7f240b61a740 1 -- 172.21.15.43:0/1512531242 shutdown_connections 2024-04-21T08:50:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.182+0000 7f240b61a740 1 --2- 172.21.15.43:0/1512531242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796d5a80 0x5574796e16d0 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.182+0000 7f240b61a740 1 --2- 172.21.15.43:0/1512531242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574796ccea0 0x5574796d1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.182+0000 7f240b61a740 1 --2- 172.21.15.43:0/1512531242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574796cc580 0x5574796d5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.182+0000 7f240b61a740 1 -- 172.21.15.43:0/1512531242 >> 172.21.15.43:0/1512531242 conn(0x5574796c6cb0 msgr2=0x5574796c8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.182+0000 7f240b61a740 1 -- 172.21.15.43:0/1512531242 shutdown_connections 2024-04-21T08:50:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.182+0000 7f240b61a740 1 -- 172.21.15.43:0/1512531242 wait complete. 2024-04-21T08:50:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.182+0000 7f240b61a740 1 -- start start 2024-04-21T08:50:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.182+0000 7f240b61a740 1 Processor -- start 2024-04-21T08:50:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.183+0000 7f240b61a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796cc580 0x5574797441c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.183+0000 7f240b61a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574796ccea0 0x557479744700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.183+0000 7f240b61a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574796d5a80 0x5574797491e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.183+0000 7f240b61a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24041b09e0 con 0x5574796cc580 2024-04-21T08:50:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.183+0000 7f240b61a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24041b0310 con 0x5574796d5a80 2024-04-21T08:50:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.183+0000 7f240b61a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f24041ad570 con 0x5574796ccea0 2024-04-21T08:50:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.183+0000 7f240b61a740 10 client.0 _ll_register_callbacks cb 0x7f23f80056f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.183+0000 7f240259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796cc580 0x5574797441c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.183+0000 7f240259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796cc580 0x5574797441c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50250/0 (socket says 172.21.15.43:50250) 2024-04-21T08:50:42.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.183+0000 7f240259c640 1 -- 172.21.15.43:0/278754620 learned_addr learned my addr 172.21.15.43:0/278754620 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:42.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.183+0000 7f2401d9b640 1 --2- 172.21.15.43:0/278754620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574796ccea0 0x557479744700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.183+0000 7f2402d9d640 1 --2- 172.21.15.43:0/278754620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574796d5a80 0x5574797491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.183+0000 7f240259c640 1 -- 172.21.15.43:0/278754620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574796ccea0 msgr2=0x557479744700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:42.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.183+0000 7f240259c640 1 --2- 172.21.15.43:0/278754620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574796ccea0 0x557479744700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.183+0000 7f240259c640 1 -- 172.21.15.43:0/278754620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574796d5a80 msgr2=0x5574797491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:42.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.183+0000 7f240259c640 1 --2- 172.21.15.43:0/278754620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574796d5a80 0x5574797491e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.183+0000 7f240259c640 1 -- 172.21.15.43:0/278754620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574797498d0 con 0x5574796cc580 2024-04-21T08:50:42.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.183+0000 7f2402d9d640 1 --2- 172.21.15.43:0/278754620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574796d5a80 0x5574797491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:42.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.183+0000 7f240259c640 1 --2- 172.21.15.43:0/278754620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796cc580 0x5574797441c0 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f23f40cdb30 tx=0x7f23f40986b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69e75ed602967aa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:42.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.183+0000 7f23e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:42.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.183+0000 7f23e9ffb640 1 -- 172.21.15.43:0/278754620 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4020040 con 0x5574796cc580 2024-04-21T08:50:42.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.183+0000 7f23e9ffb640 1 -- 172.21.15.43:0/278754620 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f23f40cdd50 con 0x5574796cc580 2024-04-21T08:50:42.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.183+0000 7f240b61a740 1 -- 172.21.15.43:0/278754620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23fc00ac10 con 0x5574796cc580 2024-04-21T08:50:42.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.184+0000 7f23e9ffb640 1 -- 172.21.15.43:0/278754620 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f23f40a4f90 con 0x5574796cc580 2024-04-21T08:50:42.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.184+0000 7f240b61a740 10 client.21198 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:42.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.184+0000 7f240b61a740 1 -- 172.21.15.43:0/278754620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557479750d70 con 0x5574796cc580 2024-04-21T08:50:42.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.184+0000 7f23e9ffb640 1 -- 172.21.15.43:0/278754620 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23fc00ac10 con 0x5574796cc580 2024-04-21T08:50:42.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.184+0000 7f23e9ffb640 1 -- 172.21.15.43:0/278754620 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f23f4063050 con 0x5574796cc580 2024-04-21T08:50:42.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.184+0000 7f240b61a740 10 client.21198 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:42.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.184+0000 7f240b61a740 10 client.21198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:42.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.184+0000 7f240b61a740 20 client.21198 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:42.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.184+0000 7f240b61a740 1 --2- 172.21.15.43:0/278754620 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557479751470 0x557479753890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:42.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.184+0000 7f240b61a740 4 client.21198 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:50:42.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.184+0000 7f240b61a740 1 -- 172.21.15.43:0/278754620 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5574795b4590 con 0x557479751470 2024-04-21T08:50:42.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.184+0000 7f2401d9b640 1 --2- 172.21.15.43:0/278754620 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557479751470 0x557479753890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:42.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.184+0000 7f23e9ffb640 10 client.21198 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:42.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.184+0000 7f2401d9b640 1 --2- 172.21.15.43:0/278754620 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557479751470 0x557479753890 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4222bf2fe5e1dcaa server_cookie=a089241c9521760 in_seq=0 out_seq=0 2024-04-21T08:50:42.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.185+0000 7f23e9ffb640 1 -- 172.21.15.43:0/278754620 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5574795b4590 con 0x557479751470 2024-04-21T08:50:42.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.185+0000 7f23e9ffb640 10 client.21198 handle_command_reply: tid=0 2024-04-21T08:50:42.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.185+0000 7f240b61a740 1 client.21198 shutdown 2024-04-21T08:50:42.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.185+0000 7f240b61a740 1 -- 172.21.15.43:0/278754620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796cc580 msgr2=0x5574797441c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:42.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.185+0000 7f240b61a740 1 --2- 172.21.15.43:0/278754620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796cc580 0x5574797441c0 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f23f40cdb30 tx=0x7f23f40986b0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.185+0000 7f240b61a740 1 -- 172.21.15.43:0/278754620 shutdown_connections 2024-04-21T08:50:42.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.185+0000 7f240b61a740 1 --2- 172.21.15.43:0/278754620 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557479751470 0x557479753890 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.185+0000 7f240b61a740 1 --2- 172.21.15.43:0/278754620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574796d5a80 0x5574797491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.185+0000 7f23e9ffb640 0 client.21198 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:42.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.185+0000 7f240b61a740 1 --2- 172.21.15.43:0/278754620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5574796ccea0 0x557479744700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.185+0000 7f240b61a740 1 --2- 172.21.15.43:0/278754620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574796cc580 0x5574797441c0 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.185+0000 7f240b61a740 1 -- 172.21.15.43:0/278754620 >> 172.21.15.43:0/278754620 conn(0x5574796c6cb0 msgr2=0x5574796c54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:42.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.185+0000 7f240b61a740 1 -- 172.21.15.43:0/278754620 shutdown_connections 2024-04-21T08:50:42.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.185+0000 7f240b61a740 1 -- 172.21.15.43:0/278754620 wait complete. 2024-04-21T08:50:42.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.185+0000 7f240b61a740 20 client.21198 trim_cache size 0 max 16384 2024-04-21T08:50:42.193 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:50:42.193 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:50:42.193 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-21T08:50:42.193 INFO:teuthology.orchestra.run.smithi043.stdout: "4c2cd53e-4a67-460f-9de0-5ddc4d75e01a": { 2024-04-21T08:50:42.193 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-21T08:50:42.193 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "4c2cd53e-4a67-460f-9de0-5ddc4d75e01a", 2024-04-21T08:50:42.193 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-21T08:50:42.193 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:50:42.193 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:50:42.194 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:50:42.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.185+0000 7f2408a32640 1 -- 172.21.15.43:0/3422257851 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f23ec0a3d20 msgr2=0x7f23ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:42.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.185+0000 7f2408a32640 1 --2- 172.21.15.43:0/3422257851 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f23ec0a3d20 0x7f23ec0a61e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f23fc010ca0 tx=0x7f23fc015000 comp rx=0 tx=0).stop 2024-04-21T08:50:42.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.185+0000 7f2408a32640 1 -- 172.21.15.43:0/3422257851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24041010a0 msgr2=0x7f24041987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:42.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.185+0000 7f2408a32640 1 --2- 172.21.15.43:0/3422257851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24041010a0 0x7f24041987a0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f23f400d770 tx=0x7f23f400dc30 comp rx=0 tx=0).stop 2024-04-21T08:50:42.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.186+0000 7f2408a32640 1 -- 172.21.15.43:0/3422257851 shutdown_connections 2024-04-21T08:50:42.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.186+0000 7f2408a32640 1 --2- 172.21.15.43:0/3422257851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f240419bfb0 0x7f240419c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.186+0000 7f2408a32640 1 --2- 172.21.15.43:0/3422257851 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f23ec0a3d20 0x7f23ec0a61e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.186+0000 7f2408a32640 1 --2- 172.21.15.43:0/3422257851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2404106170 0x7f2404198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.186+0000 7f2408a32640 1 --2- 172.21.15.43:0/3422257851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24041010a0 0x7f24041987a0 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:42.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.186+0000 7f2408a32640 1 -- 172.21.15.43:0/3422257851 >> 172.21.15.43:0/3422257851 conn(0x7f2404077f70 msgr2=0x7f240406a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:42.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.186+0000 7f2408a32640 1 -- 172.21.15.43:0/3422257851 shutdown_connections 2024-04-21T08:50:42.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:50:42.186+0000 7f2408a32640 1 -- 172.21.15.43:0/3422257851 wait complete. 2024-04-21T08:50:42.197 INFO:tasks.cephfs.filesystem:scrub status for tag:4c2cd53e-4a67-460f-9de0-5ddc4d75e01a - {'path': '/', 'tag': '4c2cd53e-4a67-460f-9de0-5ddc4d75e01a', 'options': 'recursive,force'} 2024-04-21T08:50:42.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:42 smithi043 ceph-mon[27978]: pgmap v367: 129 pgs: 129 active+clean; 5.1 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 65 KiB/s wr, 24 op/s 2024-04-21T08:50:42.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:42 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:50:42.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:42 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:50:42.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:42 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:50:42.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:42 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:50:42.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:42 smithi175 ceph-mon[31920]: pgmap v367: 129 pgs: 129 active+clean; 5.1 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 65 KiB/s wr, 24 op/s 2024-04-21T08:50:42.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:42 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:50:42.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:42 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:50:42.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:42 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:50:42.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:42 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:50:42.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:42 smithi112 ceph-mon[31085]: pgmap v367: 129 pgs: 129 active+clean; 5.1 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 65 KiB/s wr, 24 op/s 2024-04-21T08:50:42.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:42 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:50:42.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:42 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:50:42.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:42 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:50:42.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:42 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:50:44.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:44 smithi175 ceph-mon[31920]: pgmap v368: 129 pgs: 129 active+clean; 5.1 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:50:44.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:44 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:50:44.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:44 smithi112 ceph-mon[31085]: pgmap v368: 129 pgs: 129 active+clean; 5.1 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:50:44.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:44 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:50:44.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:44 smithi043 ceph-mon[27978]: pgmap v368: 129 pgs: 129 active+clean; 5.1 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:50:44.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:44 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:50:45.606 INFO:tasks.workunit.client.0.smithi043.stdout:499/600: open fd = 3 2024-04-21T08:50:45.606 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.606 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.606 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.606 INFO:tasks.workunit.client.0.smithi043.stdout:500/600: open fd = 3 2024-04-21T08:50:45.606 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.607 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.607 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.607 INFO:tasks.workunit.client.0.smithi043.stdout:501/600: open fd = 3 2024-04-21T08:50:45.607 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.607 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.607 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.607 INFO:tasks.workunit.client.0.smithi043.stdout:502/600: open fd = 3 2024-04-21T08:50:45.607 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.607 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.607 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.607 INFO:tasks.workunit.client.0.smithi043.stdout:503/600: open fd = 3 2024-04-21T08:50:45.607 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.607 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.607 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.607 INFO:tasks.workunit.client.0.smithi043.stdout:504/600: open fd = 3 2024-04-21T08:50:45.608 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.608 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.608 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.608 INFO:tasks.workunit.client.0.smithi043.stdout:505/600: open fd = 3 2024-04-21T08:50:45.608 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.608 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.608 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.608 INFO:tasks.workunit.client.0.smithi043.stdout:506/600: open fd = 3 2024-04-21T08:50:45.608 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.608 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.608 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.608 INFO:tasks.workunit.client.0.smithi043.stdout:507/600: open fd = 3 2024-04-21T08:50:45.608 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.608 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.608 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.609 INFO:tasks.workunit.client.0.smithi043.stdout:508/600: open fd = 3 2024-04-21T08:50:45.609 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.609 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.609 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.609 INFO:tasks.workunit.client.0.smithi043.stdout:509/600: open fd = 3 2024-04-21T08:50:45.609 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.609 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.609 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.609 INFO:tasks.workunit.client.0.smithi043.stdout:510/600: open fd = 3 2024-04-21T08:50:45.609 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.609 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.609 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.609 INFO:tasks.workunit.client.0.smithi043.stdout:511/600: open fd = 3 2024-04-21T08:50:45.609 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.609 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.610 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.610 INFO:tasks.workunit.client.0.smithi043.stdout:512/600: open fd = 3 2024-04-21T08:50:45.610 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.610 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.610 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.610 INFO:tasks.workunit.client.0.smithi043.stdout:513/600: open fd = 3 2024-04-21T08:50:45.610 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.610 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.610 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.610 INFO:tasks.workunit.client.0.smithi043.stdout:514/600: open fd = 3 2024-04-21T08:50:45.610 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.610 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.610 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.610 INFO:tasks.workunit.client.0.smithi043.stdout:515/600: open fd = 3 2024-04-21T08:50:45.611 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.611 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.611 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.611 INFO:tasks.workunit.client.0.smithi043.stdout:516/600: open fd = 3 2024-04-21T08:50:45.611 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.611 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.611 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.611 INFO:tasks.workunit.client.0.smithi043.stdout:517/600: open fd = 3 2024-04-21T08:50:45.611 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.611 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.611 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.611 INFO:tasks.workunit.client.0.smithi043.stdout:518/600: open fd = 3 2024-04-21T08:50:45.611 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.611 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.611 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.612 INFO:tasks.workunit.client.0.smithi043.stdout:519/600: open fd = 3 2024-04-21T08:50:45.612 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.612 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.612 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.612 INFO:tasks.workunit.client.0.smithi043.stdout:520/600: open fd = 3 2024-04-21T08:50:45.612 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.612 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.612 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.612 INFO:tasks.workunit.client.0.smithi043.stdout:521/600: open fd = 3 2024-04-21T08:50:45.612 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.612 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.612 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.612 INFO:tasks.workunit.client.0.smithi043.stdout:522/600: open fd = 3 2024-04-21T08:50:45.612 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.612 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.613 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.613 INFO:tasks.workunit.client.0.smithi043.stdout:523/600: open fd = 3 2024-04-21T08:50:45.613 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.613 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.613 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.613 INFO:tasks.workunit.client.0.smithi043.stdout:524/600: open fd = 3 2024-04-21T08:50:45.613 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.613 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.613 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.613 INFO:tasks.workunit.client.0.smithi043.stdout:525/600: open fd = 3 2024-04-21T08:50:45.613 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.613 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.613 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.613 INFO:tasks.workunit.client.0.smithi043.stdout:526/600: open fd = 3 2024-04-21T08:50:45.613 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.614 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.614 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.614 INFO:tasks.workunit.client.0.smithi043.stdout:527/600: open fd = 3 2024-04-21T08:50:45.614 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.614 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.614 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.614 INFO:tasks.workunit.client.0.smithi043.stdout:528/600: open fd = 3 2024-04-21T08:50:45.614 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.614 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.614 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.614 INFO:tasks.workunit.client.0.smithi043.stdout:529/600: open fd = 3 2024-04-21T08:50:45.614 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.614 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.614 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.614 INFO:tasks.workunit.client.0.smithi043.stdout:530/600: open fd = 3 2024-04-21T08:50:45.615 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.615 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.615 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.615 INFO:tasks.workunit.client.0.smithi043.stdout:531/600: open fd = 3 2024-04-21T08:50:45.615 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.615 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.615 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.615 INFO:tasks.workunit.client.0.smithi043.stdout:532/600: open fd = 3 2024-04-21T08:50:45.615 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.615 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.615 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.615 INFO:tasks.workunit.client.0.smithi043.stdout:533/600: open fd = 3 2024-04-21T08:50:45.615 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.615 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.615 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.615 INFO:tasks.workunit.client.0.smithi043.stdout:534/600: open fd = 3 2024-04-21T08:50:45.616 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.616 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.616 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.616 INFO:tasks.workunit.client.0.smithi043.stdout:535/600: open fd = 3 2024-04-21T08:50:45.616 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.616 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.616 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.616 INFO:tasks.workunit.client.0.smithi043.stdout:536/600: open fd = 3 2024-04-21T08:50:45.616 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.616 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.616 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.616 INFO:tasks.workunit.client.0.smithi043.stdout:537/600: open fd = 3 2024-04-21T08:50:45.616 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.616 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.616 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.617 INFO:tasks.workunit.client.0.smithi043.stdout:538/600: open fd = 3 2024-04-21T08:50:45.617 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.617 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.617 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.617 INFO:tasks.workunit.client.0.smithi043.stdout:539/600: open fd = 3 2024-04-21T08:50:45.617 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.617 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.617 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.617 INFO:tasks.workunit.client.0.smithi043.stdout:540/600: open fd = 3 2024-04-21T08:50:45.617 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.617 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.617 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.617 INFO:tasks.workunit.client.0.smithi043.stdout:541/600: open fd = 3 2024-04-21T08:50:45.617 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.617 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.618 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.618 INFO:tasks.workunit.client.0.smithi043.stdout:542/600: open fd = 3 2024-04-21T08:50:45.618 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.618 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.618 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.618 INFO:tasks.workunit.client.0.smithi043.stdout:543/600: open fd = 3 2024-04-21T08:50:45.618 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.618 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.618 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.618 INFO:tasks.workunit.client.0.smithi043.stdout:544/600: open fd = 3 2024-04-21T08:50:45.618 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.618 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.618 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.618 INFO:tasks.workunit.client.0.smithi043.stdout:545/600: open fd = 3 2024-04-21T08:50:45.618 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.619 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.619 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.619 INFO:tasks.workunit.client.0.smithi043.stdout:546/600: open fd = 3 2024-04-21T08:50:45.619 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.619 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.619 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.619 INFO:tasks.workunit.client.0.smithi043.stdout:547/600: open fd = 3 2024-04-21T08:50:45.619 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.619 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.619 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.619 INFO:tasks.workunit.client.0.smithi043.stdout:548/600: open fd = 3 2024-04-21T08:50:45.619 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.619 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.619 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.619 INFO:tasks.workunit.client.0.smithi043.stdout:549/600: open fd = 3 2024-04-21T08:50:45.620 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.620 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.620 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.620 INFO:tasks.workunit.client.0.smithi043.stdout:550/600: open fd = 3 2024-04-21T08:50:45.620 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.620 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.620 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.620 INFO:tasks.workunit.client.0.smithi043.stdout:551/600: open fd = 3 2024-04-21T08:50:45.620 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.620 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.620 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.620 INFO:tasks.workunit.client.0.smithi043.stdout:552/600: open fd = 3 2024-04-21T08:50:45.620 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.620 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.620 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.621 INFO:tasks.workunit.client.0.smithi043.stdout:553/600: open fd = 3 2024-04-21T08:50:45.621 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.621 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.621 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.621 INFO:tasks.workunit.client.0.smithi043.stdout:554/600: open fd = 3 2024-04-21T08:50:45.621 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.621 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.621 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.621 INFO:tasks.workunit.client.0.smithi043.stdout:555/600: open fd = 3 2024-04-21T08:50:45.621 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.621 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.621 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.621 INFO:tasks.workunit.client.0.smithi043.stdout:556/600: open fd = 3 2024-04-21T08:50:45.621 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.621 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.622 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.622 INFO:tasks.workunit.client.0.smithi043.stdout:557/600: open fd = 3 2024-04-21T08:50:45.622 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.622 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.622 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.622 INFO:tasks.workunit.client.0.smithi043.stdout:558/600: open fd = 3 2024-04-21T08:50:45.622 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.622 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.622 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.622 INFO:tasks.workunit.client.0.smithi043.stdout:559/600: open fd = 3 2024-04-21T08:50:45.622 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.622 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.622 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:45.622 INFO:tasks.workunit.client.0.smithi043.stdout:560/600: open fd = 3 2024-04-21T08:50:45.622 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.623 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:50:45.623 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:50:46.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:46 smithi175 ceph-mon[31920]: pgmap v369: 129 pgs: 129 active+clean; 5.1 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:50:46.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:46 smithi112 ceph-mon[31085]: pgmap v369: 129 pgs: 129 active+clean; 5.1 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:50:46.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:46 smithi043 ceph-mon[27978]: pgmap v369: 129 pgs: 129 active+clean; 5.1 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:50:48.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:48 smithi175 ceph-mon[31920]: pgmap v370: 129 pgs: 129 active+clean; 5.3 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-04-21T08:50:48.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:48 smithi112 ceph-mon[31085]: pgmap v370: 129 pgs: 129 active+clean; 5.3 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-04-21T08:50:48.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:48 smithi043 ceph-mon[27978]: pgmap v370: 129 pgs: 129 active+clean; 5.3 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-04-21T08:50:50.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:50 smithi175 ceph-mon[31920]: pgmap v371: 129 pgs: 129 active+clean; 5.3 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:50:50.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:50 smithi112 ceph-mon[31085]: pgmap v371: 129 pgs: 129 active+clean; 5.3 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:50:50.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:50 smithi043 ceph-mon[27978]: pgmap v371: 129 pgs: 129 active+clean; 5.3 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:50:52.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:52 smithi175 ceph-mon[31920]: pgmap v372: 129 pgs: 129 active+clean; 5.5 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:50:52.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:52 smithi112 ceph-mon[31085]: pgmap v372: 129 pgs: 129 active+clean; 5.5 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:50:52.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:52 smithi043 ceph-mon[27978]: pgmap v372: 129 pgs: 129 active+clean; 5.5 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:50:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.369+0000 7fc840132640 1 -- 172.21.15.43:0/3830311296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8381010f0 msgr2=0x7fc8381014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.369+0000 7fc840132640 1 --2- 172.21.15.43:0/3830311296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8381010f0 0x7fc8381014d0 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7fc830009900 tx=0x7fc83002bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.369+0000 7fc840132640 1 -- 172.21.15.43:0/3830311296 shutdown_connections 2024-04-21T08:50:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.369+0000 7fc840132640 1 --2- 172.21.15.43:0/3830311296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc838106170 0x7fc83810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.369+0000 7fc840132640 1 --2- 172.21.15.43:0/3830311296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc838101aa0 0x7fc838105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.369+0000 7fc840132640 1 --2- 172.21.15.43:0/3830311296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8381010f0 0x7fc8381014d0 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.369+0000 7fc840132640 1 -- 172.21.15.43:0/3830311296 >> 172.21.15.43:0/3830311296 conn(0x7fc838077fd0 msgr2=0x7fc8380783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.369+0000 7fc840132640 1 -- 172.21.15.43:0/3830311296 shutdown_connections 2024-04-21T08:50:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.369+0000 7fc840132640 1 -- 172.21.15.43:0/3830311296 wait complete. 2024-04-21T08:50:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.370+0000 7fc840132640 1 Processor -- start 2024-04-21T08:50:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.370+0000 7fc840132640 1 -- start start 2024-04-21T08:50:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.370+0000 7fc840132640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8381010f0 0x7fc838198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.370+0000 7fc840132640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc838106170 0x7fc838198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.370+0000 7fc840132640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc83819cfb0 0x7fc83819d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.370+0000 7fc840132640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc8381121e0 con 0x7fc83819cfb0 2024-04-21T08:50:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.370+0000 7fc840132640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc838112060 con 0x7fc838106170 2024-04-21T08:50:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.370+0000 7fc840132640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc838112360 con 0x7fc8381010f0 2024-04-21T08:50:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.370+0000 7fc83e6cf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc83819cfb0 0x7fc83819d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.370+0000 7fc83e6cf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc83819cfb0 0x7fc83819d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45440/0 (socket says 172.21.15.43:45440) 2024-04-21T08:50:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.370+0000 7fc83e6cf640 1 -- 172.21.15.43:0/3504946791 learned_addr learned my addr 172.21.15.43:0/3504946791 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.370+0000 7fc83d6cd640 1 --2- 172.21.15.43:0/3504946791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc838106170 0x7fc838198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.370+0000 7fc83dece640 1 --2- 172.21.15.43:0/3504946791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8381010f0 0x7fc838198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.370+0000 7fc83e6cf640 1 -- 172.21.15.43:0/3504946791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8381010f0 msgr2=0x7fc838198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.370+0000 7fc83e6cf640 1 --2- 172.21.15.43:0/3504946791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8381010f0 0x7fc838198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.370+0000 7fc83e6cf640 1 -- 172.21.15.43:0/3504946791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc838106170 msgr2=0x7fc838198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.370+0000 7fc83e6cf640 1 --2- 172.21.15.43:0/3504946791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc838106170 0x7fc838198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.370+0000 7fc83e6cf640 1 -- 172.21.15.43:0/3504946791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83819daa0 con 0x7fc83819cfb0 2024-04-21T08:50:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.371+0000 7fc83d6cd640 1 --2- 172.21.15.43:0/3504946791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc838106170 0x7fc838198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.371+0000 7fc83e6cf640 1 --2- 172.21.15.43:0/3504946791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc83819cfb0 0x7fc83819d460 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7fc83400eaa0 tx=0x7fc83400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.371+0000 7fc82effd640 1 -- 172.21.15.43:0/3504946791 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83400cdb0 con 0x7fc83819cfb0 2024-04-21T08:50:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.371+0000 7fc82effd640 1 -- 172.21.15.43:0/3504946791 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc834005020 con 0x7fc83819cfb0 2024-04-21T08:50:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.371+0000 7fc840132640 1 -- 172.21.15.43:0/3504946791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc83819dd90 con 0x7fc83819cfb0 2024-04-21T08:50:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.371+0000 7fc82effd640 1 -- 172.21.15.43:0/3504946791 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8340116d0 con 0x7fc83819cfb0 2024-04-21T08:50:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.371+0000 7fc840132640 1 -- 172.21.15.43:0/3504946791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8381a55d0 con 0x7fc83819cfb0 2024-04-21T08:50:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.373+0000 7fc82effd640 1 -- 172.21.15.43:0/3504946791 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc8340051d0 con 0x7fc83819cfb0 2024-04-21T08:50:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.374+0000 7fc842dfe740 1 Processor -- start 2024-04-21T08:50:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.374+0000 7fc842dfe740 1 -- start start 2024-04-21T08:50:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.374+0000 7fc842dfe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579e4e788d0 0x5579e4e77620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.374+0000 7fc842dfe740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5579e4e77b60 0x5579e4e77fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.374+0000 7fc842dfe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e854f0 0x5579e4e878e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.374+0000 7fc842dfe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5579e4ddc8d0 con 0x5579e4e854f0 2024-04-21T08:50:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.374+0000 7fc842dfe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5579e4d79e80 con 0x5579e4e788d0 2024-04-21T08:50:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.374+0000 7fc842dfe740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5579e4dd8890 con 0x5579e4e77b60 2024-04-21T08:50:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.374+0000 7fc82effd640 1 --2- 172.21.15.43:0/3504946791 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc8240a3e40 0x7fc8240a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.374+0000 7fc83d6cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e854f0 0x5579e4e878e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.375+0000 7fc83d6cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e854f0 0x5579e4e878e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45456/0 (socket says 172.21.15.43:45456) 2024-04-21T08:50:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.375+0000 7fc83d6cd640 1 -- 172.21.15.43:0/2632094798 learned_addr learned my addr 172.21.15.43:0/2632094798 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.375+0000 7fc83e6cf640 1 --2- 172.21.15.43:0/3504946791 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc8240a3e40 0x7fc8240a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.375+0000 7fc83dece640 1 --2- 172.21.15.43:0/2632094798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5579e4e77b60 0x5579e4e77fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.375+0000 7fc83d6cd640 1 --2- 172.21.15.43:0/2632094798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579e4e788d0 0x5579e4e77620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.375+0000 7fc82effd640 1 -- 172.21.15.43:0/3504946791 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fc8340d0d70 con 0x7fc83819cfb0 2024-04-21T08:50:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.375+0000 7fc83d6cd640 1 -- 172.21.15.43:0/2632094798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5579e4e77b60 msgr2=0x5579e4e77fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.375+0000 7fc83d6cd640 1 --2- 172.21.15.43:0/2632094798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5579e4e77b60 0x5579e4e77fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.375+0000 7fc83d6cd640 1 -- 172.21.15.43:0/2632094798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579e4e788d0 msgr2=0x5579e4e77620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.375+0000 7fc83d6cd640 1 --2- 172.21.15.43:0/2632094798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579e4e788d0 0x5579e4e77620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.375+0000 7fc83d6cd640 1 -- 172.21.15.43:0/2632094798 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579e4d9b320 con 0x5579e4e854f0 2024-04-21T08:50:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.375+0000 7fc83dece640 1 --2- 172.21.15.43:0/2632094798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5579e4e77b60 0x5579e4e77fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:50:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.375+0000 7fc83d6cd640 1 --2- 172.21.15.43:0/2632094798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e854f0 0x5579e4e878e0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7fc8280149d0 tx=0x7fc82800b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6630b24ff5542c31 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.375+0000 7fc83e6cf640 1 --2- 172.21.15.43:0/3504946791 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc8240a3e40 0x7fc8240a6300 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc83400cfd0 tx=0x7fc834087040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.375+0000 7fc823fff640 1 -- 172.21.15.43:0/2632094798 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828004030 con 0x5579e4e854f0 2024-04-21T08:50:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.375+0000 7fc823fff640 1 -- 172.21.15.43:0/2632094798 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc82800fc70 con 0x5579e4e854f0 2024-04-21T08:50:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.375+0000 7fc823fff640 1 -- 172.21.15.43:0/2632094798 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828003850 con 0x5579e4e854f0 2024-04-21T08:50:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.376+0000 7fc842dfe740 1 -- 172.21.15.43:0/2632094798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e854f0 msgr2=0x5579e4e878e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.376+0000 7fc842dfe740 1 --2- 172.21.15.43:0/2632094798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e854f0 0x5579e4e878e0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7fc8280149d0 tx=0x7fc82800b550 comp rx=0 tx=0).stop 2024-04-21T08:50:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.376+0000 7fc842dfe740 1 -- 172.21.15.43:0/2632094798 shutdown_connections 2024-04-21T08:50:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.376+0000 7fc842dfe740 1 --2- 172.21.15.43:0/2632094798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e854f0 0x5579e4e878e0 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.376+0000 7fc842dfe740 1 --2- 172.21.15.43:0/2632094798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5579e4e77b60 0x5579e4e77fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.376+0000 7fc842dfe740 1 --2- 172.21.15.43:0/2632094798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579e4e788d0 0x5579e4e77620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.376+0000 7fc842dfe740 1 -- 172.21.15.43:0/2632094798 >> 172.21.15.43:0/2632094798 conn(0x5579e4e3ec90 msgr2=0x5579e4e72850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.376+0000 7fc842dfe740 1 -- 172.21.15.43:0/2632094798 shutdown_connections 2024-04-21T08:50:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.376+0000 7fc842dfe740 1 -- 172.21.15.43:0/2632094798 wait complete. 2024-04-21T08:50:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.377+0000 7fc842dfe740 1 -- start start 2024-04-21T08:50:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.377+0000 7fc842dfe740 1 Processor -- start 2024-04-21T08:50:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.377+0000 7fc842dfe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579e4e77b60 0x5579e4e84ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.377+0000 7fc842dfe740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5579e4e788d0 0x5579e4e82230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.377+0000 7fc842dfe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e854f0 0x5579e4e827d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.377+0000 7fc842dfe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5579e4e88220 con 0x5579e4e854f0 2024-04-21T08:50:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.377+0000 7fc842dfe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5579e4d64eb0 con 0x5579e4e77b60 2024-04-21T08:50:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.377+0000 7fc842dfe740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5579e4e883a0 con 0x5579e4e788d0 2024-04-21T08:50:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.377+0000 7fc842dfe740 10 client.0 _ll_register_callbacks cb 0x5579e4d91b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.377+0000 7fc83d6cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e854f0 0x5579e4e827d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.377+0000 7fc83d6cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e854f0 0x5579e4e827d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45472/0 (socket says 172.21.15.43:45472) 2024-04-21T08:50:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.377+0000 7fc83d6cd640 1 -- 172.21.15.43:0/1157708180 learned_addr learned my addr 172.21.15.43:0/1157708180 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.377+0000 7fc83d6cd640 1 --2- 172.21.15.43:0/1157708180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579e4e77b60 0x5579e4e84ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.378+0000 7fc83dece640 1 --2- 172.21.15.43:0/1157708180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5579e4e788d0 0x5579e4e82230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.378+0000 7fc83d6cd640 1 -- 172.21.15.43:0/1157708180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5579e4e788d0 msgr2=0x5579e4e82230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.378+0000 7fc83d6cd640 1 --2- 172.21.15.43:0/1157708180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5579e4e788d0 0x5579e4e82230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.378+0000 7fc83d6cd640 1 -- 172.21.15.43:0/1157708180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579e4e77b60 msgr2=0x5579e4e84ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.378+0000 7fc83d6cd640 1 --2- 172.21.15.43:0/1157708180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579e4e77b60 0x5579e4e84ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.378+0000 7fc83d6cd640 1 -- 172.21.15.43:0/1157708180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579e4aef210 con 0x5579e4e854f0 2024-04-21T08:50:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.378+0000 7fc83d6cd640 1 --2- 172.21.15.43:0/1157708180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e854f0 0x5579e4e827d0 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7fc828014fd0 tx=0x7fc8280043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac1e73d7564c128 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.378+0000 7fc8217fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.378+0000 7fc8217fa640 1 -- 172.21.15.43:0/1157708180 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82800f7c0 con 0x5579e4e854f0 2024-04-21T08:50:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.378+0000 7fc8217fa640 1 -- 172.21.15.43:0/1157708180 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fc82800f960 con 0x5579e4e854f0 2024-04-21T08:50:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.378+0000 7fc842dfe740 1 -- 172.21.15.43:0/1157708180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc82800fc70 con 0x5579e4e854f0 2024-04-21T08:50:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.378+0000 7fc8217fa640 1 -- 172.21.15.43:0/1157708180 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc82801f3f0 con 0x5579e4e854f0 2024-04-21T08:50:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.378+0000 7fc842dfe740 10 client.21210 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.378+0000 7fc842dfe740 1 -- 172.21.15.43:0/1157708180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5579e4bbb620 con 0x5579e4e854f0 2024-04-21T08:50:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.378+0000 7fc8217fa640 1 -- 172.21.15.43:0/1157708180 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc82800fc70 con 0x5579e4e854f0 2024-04-21T08:50:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.378+0000 7fc8217fa640 1 -- 172.21.15.43:0/1157708180 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc8280123f0 con 0x5579e4e854f0 2024-04-21T08:50:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.379+0000 7fc842dfe740 10 client.21210 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.379+0000 7fc842dfe740 10 client.21210 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.379+0000 7fc842dfe740 20 client.21210 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.379+0000 7fc842dfe740 1 --2- 172.21.15.43:0/1157708180 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5579e4e95470 0x5579e4e97860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.379+0000 7fc842dfe740 4 client.21210 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.379+0000 7fc842dfe740 1 -- 172.21.15.43:0/1157708180 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579e4df0880 con 0x5579e4e95470 2024-04-21T08:50:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.379+0000 7fc83dece640 1 --2- 172.21.15.43:0/1157708180 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5579e4e95470 0x5579e4e97860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.379+0000 7fc83dece640 1 --2- 172.21.15.43:0/1157708180 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5579e4e95470 0x5579e4e97860 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a135575b652183da server_cookie=184b257079e1be97 in_seq=0 out_seq=0 2024-04-21T08:50:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.379+0000 7fc8217fa640 10 client.21210 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.380+0000 7fc8217fa640 1 -- 172.21.15.43:0/1157708180 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5579e4df0880 con 0x5579e4e95470 2024-04-21T08:50:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.380+0000 7fc8217fa640 10 client.21210 handle_command_reply: tid=0 2024-04-21T08:50:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.380+0000 7fc842dfe740 1 client.21210 shutdown 2024-04-21T08:50:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.380+0000 7fc842dfe740 1 -- 172.21.15.43:0/1157708180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e854f0 msgr2=0x5579e4e827d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.380+0000 7fc842dfe740 1 --2- 172.21.15.43:0/1157708180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e854f0 0x5579e4e827d0 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7fc828014fd0 tx=0x7fc8280043f0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.380+0000 7fc842dfe740 1 -- 172.21.15.43:0/1157708180 shutdown_connections 2024-04-21T08:50:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.380+0000 7fc842dfe740 1 --2- 172.21.15.43:0/1157708180 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5579e4e95470 0x5579e4e97860 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.380+0000 7fc842dfe740 1 --2- 172.21.15.43:0/1157708180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e854f0 0x5579e4e827d0 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.380+0000 7fc842dfe740 1 --2- 172.21.15.43:0/1157708180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5579e4e788d0 0x5579e4e82230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.380+0000 7fc842dfe740 1 --2- 172.21.15.43:0/1157708180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579e4e77b60 0x5579e4e84ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.380+0000 7fc8217fa640 0 client.21210 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.380+0000 7fc842dfe740 1 -- 172.21.15.43:0/1157708180 >> 172.21.15.43:0/1157708180 conn(0x5579e4e3ec90 msgr2=0x5579e4e745a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.381+0000 7fc842dfe740 1 -- 172.21.15.43:0/1157708180 shutdown_connections 2024-04-21T08:50:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.381+0000 7fc842dfe740 1 -- 172.21.15.43:0/1157708180 wait complete. 2024-04-21T08:50:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.381+0000 7fc842dfe740 20 client.21210 trim_cache size 0 max 16384 2024-04-21T08:50:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.396+0000 7fc842dfe740 1 Processor -- start 2024-04-21T08:50:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.396+0000 7fc842dfe740 1 -- start start 2024-04-21T08:50:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.396+0000 7fc842dfe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e77b60 0x5579e4e84960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.396+0000 7fc842dfe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579e4e788d0 0x5579e4e84ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.396+0000 7fc842dfe740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5579e4e854f0 0x5579e4e868e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.396+0000 7fc842dfe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc8381ad390 con 0x5579e4e77b60 2024-04-21T08:50:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.396+0000 7fc842dfe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc8381aeed0 con 0x5579e4e788d0 2024-04-21T08:50:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.396+0000 7fc842dfe740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc8381ada60 con 0x5579e4e854f0 2024-04-21T08:50:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.396+0000 7fc83d6cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e77b60 0x5579e4e84960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.396+0000 7fc83d6cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e77b60 0x5579e4e84960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45484/0 (socket says 172.21.15.43:45484) 2024-04-21T08:50:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.396+0000 7fc83d6cd640 1 -- 172.21.15.43:0/3818764984 learned_addr learned my addr 172.21.15.43:0/3818764984 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.396+0000 7fc83d6cd640 1 --2- 172.21.15.43:0/3818764984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5579e4e854f0 0x5579e4e868e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.396+0000 7fc83dece640 1 --2- 172.21.15.43:0/3818764984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579e4e788d0 0x5579e4e84ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.396+0000 7fc83d6cd640 1 -- 172.21.15.43:0/3818764984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5579e4e854f0 msgr2=0x5579e4e868e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.396+0000 7fc83d6cd640 1 --2- 172.21.15.43:0/3818764984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5579e4e854f0 0x5579e4e868e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.396+0000 7fc83d6cd640 1 -- 172.21.15.43:0/3818764984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579e4e788d0 msgr2=0x5579e4e84ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.396+0000 7fc83d6cd640 1 --2- 172.21.15.43:0/3818764984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579e4e788d0 0x5579e4e84ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.396+0000 7fc83d6cd640 1 -- 172.21.15.43:0/3818764984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579e4e76e30 con 0x5579e4e77b60 2024-04-21T08:50:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.396+0000 7fc83dece640 1 --2- 172.21.15.43:0/3818764984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579e4e788d0 0x5579e4e84ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.397+0000 7fc83d6cd640 1 --2- 172.21.15.43:0/3818764984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e77b60 0x5579e4e84960 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7fc828017040 tx=0x7fc828014320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30822dcaa3109117 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.397+0000 7fc8237fe640 1 -- 172.21.15.43:0/3818764984 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828024070 con 0x5579e4e77b60 2024-04-21T08:50:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.397+0000 7fc8237fe640 1 -- 172.21.15.43:0/3818764984 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc828004460 con 0x5579e4e77b60 2024-04-21T08:50:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.397+0000 7fc8237fe640 1 -- 172.21.15.43:0/3818764984 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8280123f0 con 0x5579e4e77b60 2024-04-21T08:50:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.397+0000 7fc842dfe740 1 -- 172.21.15.43:0/3818764984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e77b60 msgr2=0x5579e4e84960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.397+0000 7fc842dfe740 1 --2- 172.21.15.43:0/3818764984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e77b60 0x5579e4e84960 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7fc828017040 tx=0x7fc828014320 comp rx=0 tx=0).stop 2024-04-21T08:50:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.397+0000 7fc842dfe740 1 -- 172.21.15.43:0/3818764984 shutdown_connections 2024-04-21T08:50:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.397+0000 7fc842dfe740 1 --2- 172.21.15.43:0/3818764984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5579e4e854f0 0x5579e4e868e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.397+0000 7fc842dfe740 1 --2- 172.21.15.43:0/3818764984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579e4e788d0 0x5579e4e84ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.397+0000 7fc842dfe740 1 --2- 172.21.15.43:0/3818764984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e77b60 0x5579e4e84960 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.397+0000 7fc842dfe740 1 -- 172.21.15.43:0/3818764984 >> 172.21.15.43:0/3818764984 conn(0x5579e4e3ec90 msgr2=0x5579e4e80b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.397+0000 7fc842dfe740 1 -- 172.21.15.43:0/3818764984 shutdown_connections 2024-04-21T08:50:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.397+0000 7fc842dfe740 1 -- 172.21.15.43:0/3818764984 wait complete. 2024-04-21T08:50:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.398+0000 7fc842dfe740 1 -- start start 2024-04-21T08:50:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.398+0000 7fc842dfe740 1 Processor -- start 2024-04-21T08:50:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.398+0000 7fc842dfe740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5579e4e77b60 0x5579e4e96310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.398+0000 7fc842dfe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579e4e788d0 0x5579e4e96850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.398+0000 7fc842dfe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e854f0 0x5579e4e96d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.398+0000 7fc842dfe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc8381ae800 con 0x5579e4e854f0 2024-04-21T08:50:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.398+0000 7fc842dfe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc8381ae130 con 0x5579e4e788d0 2024-04-21T08:50:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.398+0000 7fc842dfe740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc8381ab390 con 0x5579e4e77b60 2024-04-21T08:50:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.398+0000 7fc842dfe740 10 client.0 _ll_register_callbacks cb 0x7fc828014410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.398+0000 7fc83d6cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e854f0 0x5579e4e96d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.398+0000 7fc83d6cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e854f0 0x5579e4e96d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45486/0 (socket says 172.21.15.43:45486) 2024-04-21T08:50:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.398+0000 7fc83d6cd640 1 -- 172.21.15.43:0/2217804623 learned_addr learned my addr 172.21.15.43:0/2217804623 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:50:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.398+0000 7fc83d6cd640 1 --2- 172.21.15.43:0/2217804623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5579e4e77b60 0x5579e4e96310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.399+0000 7fc83dece640 1 --2- 172.21.15.43:0/2217804623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579e4e788d0 0x5579e4e96850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.399+0000 7fc83d6cd640 1 -- 172.21.15.43:0/2217804623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5579e4e77b60 msgr2=0x5579e4e96310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.399+0000 7fc83d6cd640 1 --2- 172.21.15.43:0/2217804623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5579e4e77b60 0x5579e4e96310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.399+0000 7fc83d6cd640 1 -- 172.21.15.43:0/2217804623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579e4e788d0 msgr2=0x5579e4e96850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.399+0000 7fc83d6cd640 1 --2- 172.21.15.43:0/2217804623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579e4e788d0 0x5579e4e96850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.399+0000 7fc83d6cd640 1 -- 172.21.15.43:0/2217804623 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579e4e87d50 con 0x5579e4e854f0 2024-04-21T08:50:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.399+0000 7fc83dece640 1 --2- 172.21.15.43:0/2217804623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579e4e788d0 0x5579e4e96850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.399+0000 7fc83d6cd640 1 --2- 172.21.15.43:0/2217804623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e854f0 0x5579e4e96d90 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7fc828000c00 tx=0x7fc828005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01724ff1ea1e01d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.399+0000 7fc8217fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:50:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.399+0000 7fc8217fa640 1 -- 172.21.15.43:0/2217804623 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828024070 con 0x5579e4e854f0 2024-04-21T08:50:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.399+0000 7fc8217fa640 1 -- 172.21.15.43:0/2217804623 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fc828005030 con 0x5579e4e854f0 2024-04-21T08:50:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.399+0000 7fc842dfe740 1 -- 172.21.15.43:0/2217804623 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc828004460 con 0x5579e4e854f0 2024-04-21T08:50:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.399+0000 7fc8217fa640 1 -- 172.21.15.43:0/2217804623 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc828005230 con 0x5579e4e854f0 2024-04-21T08:50:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.399+0000 7fc842dfe740 10 client.21216 fetch_fsmap learned FSMap version 30 2024-04-21T08:50:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.399+0000 7fc842dfe740 1 -- 172.21.15.43:0/2217804623 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5579e4efb6e0 con 0x5579e4e854f0 2024-04-21T08:50:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.399+0000 7fc8217fa640 1 -- 172.21.15.43:0/2217804623 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc828004460 con 0x5579e4e854f0 2024-04-21T08:50:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.399+0000 7fc8217fa640 1 -- 172.21.15.43:0/2217804623 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc828004810 con 0x5579e4e854f0 2024-04-21T08:50:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.399+0000 7fc842dfe740 10 client.21216 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:50:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.399+0000 7fc842dfe740 10 client.21216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.399+0000 7fc842dfe740 20 client.21216 populate_metadata read hostname 'smithi043' 2024-04-21T08:50:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.400+0000 7fc842dfe740 1 --2- 172.21.15.43:0/2217804623 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5579e4efbac0 0x5579e4efdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.400+0000 7fc842dfe740 4 client.21216 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:50:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.400+0000 7fc842dfe740 1 -- 172.21.15.43:0/2217804623 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5579e4d7c3d0 con 0x5579e4efbac0 2024-04-21T08:50:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.400+0000 7fc83dece640 1 --2- 172.21.15.43:0/2217804623 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5579e4efbac0 0x5579e4efdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.400+0000 7fc8217fa640 10 client.21216 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.400+0000 7fc83dece640 1 --2- 172.21.15.43:0/2217804623 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5579e4efbac0 0x5579e4efdee0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cba338fc635b0fdc server_cookie=afce11b144a2fba in_seq=0 out_seq=0 2024-04-21T08:50:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.414+0000 7fc8217fa640 1 -- 172.21.15.43:0/2217804623 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5579e4d7c3d0 con 0x5579e4efbac0 2024-04-21T08:50:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.414+0000 7fc8217fa640 10 client.21216 handle_command_reply: tid=0 2024-04-21T08:50:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.414+0000 7fc842dfe740 1 client.21216 shutdown 2024-04-21T08:50:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.414+0000 7fc842dfe740 1 -- 172.21.15.43:0/2217804623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e854f0 msgr2=0x5579e4e96d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.414+0000 7fc842dfe740 1 --2- 172.21.15.43:0/2217804623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e854f0 0x5579e4e96d90 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7fc828000c00 tx=0x7fc828005970 comp rx=0 tx=0).stop 2024-04-21T08:50:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.414+0000 7fc842dfe740 1 -- 172.21.15.43:0/2217804623 shutdown_connections 2024-04-21T08:50:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.414+0000 7fc842dfe740 1 --2- 172.21.15.43:0/2217804623 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5579e4efbac0 0x5579e4efdee0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.414+0000 7fc842dfe740 1 --2- 172.21.15.43:0/2217804623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579e4e854f0 0x5579e4e96d90 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.414+0000 7fc8217fa640 0 client.21216 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:50:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.414+0000 7fc842dfe740 1 --2- 172.21.15.43:0/2217804623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579e4e788d0 0x5579e4e96850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.414+0000 7fc842dfe740 1 --2- 172.21.15.43:0/2217804623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5579e4e77b60 0x5579e4e96310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.414+0000 7fc842dfe740 1 -- 172.21.15.43:0/2217804623 >> 172.21.15.43:0/2217804623 conn(0x5579e4e3ec90 msgr2=0x5579e4e80f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.414+0000 7fc842dfe740 1 -- 172.21.15.43:0/2217804623 shutdown_connections 2024-04-21T08:50:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.414+0000 7fc842dfe740 1 -- 172.21.15.43:0/2217804623 wait complete. 2024-04-21T08:50:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.414+0000 7fc842dfe740 20 client.21216 trim_cache size 0 max 16384 2024-04-21T08:50:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:50:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:50:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:50:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:50:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.415+0000 7fc840132640 1 -- 172.21.15.43:0/3504946791 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc8240a3e40 msgr2=0x7fc8240a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.415+0000 7fc840132640 1 --2- 172.21.15.43:0/3504946791 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc8240a3e40 0x7fc8240a6300 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc83400cfd0 tx=0x7fc834087040 comp rx=0 tx=0).stop 2024-04-21T08:50:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.415+0000 7fc840132640 1 -- 172.21.15.43:0/3504946791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc83819cfb0 msgr2=0x7fc83819d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.415+0000 7fc840132640 1 --2- 172.21.15.43:0/3504946791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc83819cfb0 0x7fc83819d460 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7fc83400eaa0 tx=0x7fc83400ef60 comp rx=0 tx=0).stop 2024-04-21T08:50:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.415+0000 7fc840132640 1 -- 172.21.15.43:0/3504946791 shutdown_connections 2024-04-21T08:50:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.415+0000 7fc840132640 1 --2- 172.21.15.43:0/3504946791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc83819cfb0 0x7fc83819d460 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.415+0000 7fc840132640 1 --2- 172.21.15.43:0/3504946791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc838106170 0x7fc838198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.415+0000 7fc840132640 1 --2- 172.21.15.43:0/3504946791 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc8240a3e40 0x7fc8240a6300 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.415+0000 7fc840132640 1 --2- 172.21.15.43:0/3504946791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8381010f0 0x7fc838198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.415+0000 7fc840132640 1 -- 172.21.15.43:0/3504946791 >> 172.21.15.43:0/3504946791 conn(0x7fc838077fd0 msgr2=0x7fc8380fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.415+0000 7fc840132640 1 -- 172.21.15.43:0/3504946791 shutdown_connections 2024-04-21T08:50:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:50:53.415+0000 7fc840132640 1 -- 172.21.15.43:0/3504946791 wait complete. 2024-04-21T08:50:54.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:54 smithi175 ceph-mon[31920]: pgmap v373: 129 pgs: 129 active+clean; 5.5 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-04-21T08:50:54.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:54 smithi112 ceph-mon[31085]: pgmap v373: 129 pgs: 129 active+clean; 5.5 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-04-21T08:50:54.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:54 smithi043 ceph-mon[27978]: pgmap v373: 129 pgs: 129 active+clean; 5.5 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-04-21T08:50:56.597 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:56 smithi112 ceph-mon[31085]: pgmap v374: 129 pgs: 129 active+clean; 5.5 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-04-21T08:50:56.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:56 smithi175 ceph-mon[31920]: pgmap v374: 129 pgs: 129 active+clean; 5.5 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-04-21T08:50:56.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:56 smithi043 ceph-mon[27978]: pgmap v374: 129 pgs: 129 active+clean; 5.5 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-04-21T08:50:58.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:50:58 smithi175 ceph-mon[31920]: pgmap v375: 129 pgs: 129 active+clean; 1.7 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 59 KiB/s wr, 24 op/s 2024-04-21T08:50:58.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:50:58 smithi112 ceph-mon[31085]: pgmap v375: 129 pgs: 129 active+clean; 1.7 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 59 KiB/s wr, 24 op/s 2024-04-21T08:50:58.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:50:58 smithi043 ceph-mon[27978]: pgmap v375: 129 pgs: 129 active+clean; 1.7 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 59 KiB/s wr, 24 op/s 2024-04-21T08:51:00.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:00 smithi175 ceph-mon[31920]: pgmap v376: 129 pgs: 129 active+clean; 1.7 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-04-21T08:51:00.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:00 smithi112 ceph-mon[31085]: pgmap v376: 129 pgs: 129 active+clean; 1.7 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-04-21T08:51:00.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:00 smithi043 ceph-mon[27978]: pgmap v376: 129 pgs: 129 active+clean; 1.7 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-04-21T08:51:02.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:02 smithi175 ceph-mon[31920]: pgmap v377: 129 pgs: 129 active+clean; 1.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 105 KiB/s wr, 40 op/s 2024-04-21T08:51:02.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:02 smithi112 ceph-mon[31085]: pgmap v377: 129 pgs: 129 active+clean; 1.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 105 KiB/s wr, 40 op/s 2024-04-21T08:51:02.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:02 smithi043 ceph-mon[27978]: pgmap v377: 129 pgs: 129 active+clean; 1.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 105 KiB/s wr, 40 op/s 2024-04-21T08:51:04.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:04 smithi175 ceph-mon[31920]: pgmap v378: 129 pgs: 129 active+clean; 1.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 87 KiB/s wr, 32 op/s 2024-04-21T08:51:04.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:04 smithi112 ceph-mon[31085]: pgmap v378: 129 pgs: 129 active+clean; 1.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 87 KiB/s wr, 32 op/s 2024-04-21T08:51:04.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:04 smithi043 ceph-mon[27978]: pgmap v378: 129 pgs: 129 active+clean; 1.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 87 KiB/s wr, 32 op/s 2024-04-21T08:51:05.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:05 smithi175 ceph-mon[31920]: osdmap e107: 12 total, 12 up, 12 in 2024-04-21T08:51:05.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:05 smithi112 ceph-mon[31085]: osdmap e107: 12 total, 12 up, 12 in 2024-04-21T08:51:05.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:05 smithi043 ceph-mon[27978]: osdmap e107: 12 total, 12 up, 12 in 2024-04-21T08:51:06.517 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.515+0000 7fe918104640 1 -- 172.21.15.43:0/3035724016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe910107290 msgr2=0x7fe91010ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:06.517 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.515+0000 7fe918104640 1 --2- 172.21.15.43:0/3035724016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe910107290 0x7fe91010ee60 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fe900009ca0 tx=0x7fe90002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.515+0000 7fe918104640 1 -- 172.21.15.43:0/3035724016 shutdown_connections 2024-04-21T08:51:06.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.515+0000 7fe918104640 1 --2- 172.21.15.43:0/3035724016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe910107290 0x7fe91010ee60 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.515+0000 7fe918104640 1 --2- 172.21.15.43:0/3035724016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe910102b70 0x7fe910106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.515+0000 7fe918104640 1 --2- 172.21.15.43:0/3035724016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9101021c0 0x7fe9101025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.515+0000 7fe918104640 1 -- 172.21.15.43:0/3035724016 >> 172.21.15.43:0/3035724016 conn(0x7fe9100fda20 msgr2=0x7fe9100ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:06.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.515+0000 7fe918104640 1 -- 172.21.15.43:0/3035724016 shutdown_connections 2024-04-21T08:51:06.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.515+0000 7fe918104640 1 -- 172.21.15.43:0/3035724016 wait complete. 2024-04-21T08:51:06.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.516+0000 7fe918104640 1 Processor -- start 2024-04-21T08:51:06.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.516+0000 7fe918104640 1 -- start start 2024-04-21T08:51:06.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.516+0000 7fe918104640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9101021c0 0x7fe910074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:06.518 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.516+0000 7fe918104640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe910107290 0x7fe910074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.516+0000 7fe918104640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe910074cd0 0x7fe9101a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.516+0000 7fe918104640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe9101121e0 con 0x7fe910074cd0 2024-04-21T08:51:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.516+0000 7fe918104640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe910112060 con 0x7fe9101021c0 2024-04-21T08:51:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.516+0000 7fe918104640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe910112360 con 0x7fe910107290 2024-04-21T08:51:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.516+0000 7fe9166a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe910074cd0 0x7fe9101a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.516+0000 7fe9166a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe910074cd0 0x7fe9101a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33880/0 (socket says 172.21.15.43:33880) 2024-04-21T08:51:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.516+0000 7fe9166a1640 1 -- 172.21.15.43:0/2491161848 learned_addr learned my addr 172.21.15.43:0/2491161848 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.516+0000 7fe915ea0640 1 --2- 172.21.15.43:0/2491161848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9101021c0 0x7fe910074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.516+0000 7fe91569f640 1 --2- 172.21.15.43:0/2491161848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe910107290 0x7fe910074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.517+0000 7fe9166a1640 1 -- 172.21.15.43:0/2491161848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe910107290 msgr2=0x7fe910074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.517+0000 7fe9166a1640 1 --2- 172.21.15.43:0/2491161848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe910107290 0x7fe910074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.517+0000 7fe9166a1640 1 -- 172.21.15.43:0/2491161848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9101021c0 msgr2=0x7fe910074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.517+0000 7fe9166a1640 1 --2- 172.21.15.43:0/2491161848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9101021c0 0x7fe910074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.517+0000 7fe9166a1640 1 -- 172.21.15.43:0/2491161848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9101a6e90 con 0x7fe910074cd0 2024-04-21T08:51:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.517+0000 7fe91569f640 1 --2- 172.21.15.43:0/2491161848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe910107290 0x7fe910074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.517+0000 7fe9166a1640 1 --2- 172.21.15.43:0/2491161848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe910074cd0 0x7fe9101a67f0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7fe900005e30 tx=0x7fe900034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.517+0000 7fe90effd640 1 -- 172.21.15.43:0/2491161848 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe900039040 con 0x7fe910074cd0 2024-04-21T08:51:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.517+0000 7fe90effd640 1 -- 172.21.15.43:0/2491161848 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe900044070 con 0x7fe910074cd0 2024-04-21T08:51:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.517+0000 7fe918104640 1 -- 172.21.15.43:0/2491161848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9101a7120 con 0x7fe910074cd0 2024-04-21T08:51:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.517+0000 7fe90effd640 1 -- 172.21.15.43:0/2491161848 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe900003e60 con 0x7fe910074cd0 2024-04-21T08:51:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.517+0000 7fe918104640 1 -- 172.21.15.43:0/2491161848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9101a7610 con 0x7fe910074cd0 2024-04-21T08:51:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.520+0000 7fe90effd640 1 -- 172.21.15.43:0/2491161848 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe9000324c0 con 0x7fe910074cd0 2024-04-21T08:51:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.520+0000 7fe91add4740 1 Processor -- start 2024-04-21T08:51:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.520+0000 7fe91add4740 1 -- start start 2024-04-21T08:51:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.520+0000 7fe91add4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d96e92b8d0 0x55d96e92a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.520+0000 7fe91add4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d96e92ab60 0x55d96e92afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.520+0000 7fe91add4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e9340e0 0x55d96e9364d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.520+0000 7fe91add4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d96e88f8d0 con 0x55d96e9340e0 2024-04-21T08:51:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.520+0000 7fe91add4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d96e82ce80 con 0x55d96e92ab60 2024-04-21T08:51:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.520+0000 7fe91add4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d96e88b890 con 0x55d96e92b8d0 2024-04-21T08:51:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.520+0000 7fe90effd640 1 --2- 172.21.15.43:0/2491161848 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe8f80a3dc0 0x7fe8f80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.520+0000 7fe91569f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e9340e0 0x55d96e9364d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.521+0000 7fe91569f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e9340e0 0x55d96e9364d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33886/0 (socket says 172.21.15.43:33886) 2024-04-21T08:51:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.521+0000 7fe91569f640 1 -- 172.21.15.43:0/3153019240 learned_addr learned my addr 172.21.15.43:0/3153019240 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.521+0000 7fe9166a1640 1 --2- 172.21.15.43:0/2491161848 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe8f80a3dc0 0x7fe8f80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.521+0000 7fe915ea0640 1 --2- 172.21.15.43:0/3153019240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d96e92ab60 0x55d96e92afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.521+0000 7fe91569f640 1 --2- 172.21.15.43:0/3153019240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d96e92b8d0 0x55d96e92a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.521+0000 7fe90effd640 1 -- 172.21.15.43:0/2491161848 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9671+0+0 (secure 0 0 0) 0x7fe9000f2e10 con 0x7fe910074cd0 2024-04-21T08:51:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.521+0000 7fe91569f640 1 -- 172.21.15.43:0/3153019240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d96e92b8d0 msgr2=0x55d96e92a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.521+0000 7fe91569f640 1 --2- 172.21.15.43:0/3153019240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d96e92b8d0 0x55d96e92a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.521+0000 7fe91569f640 1 -- 172.21.15.43:0/3153019240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d96e92ab60 msgr2=0x55d96e92afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.521+0000 7fe91569f640 1 --2- 172.21.15.43:0/3153019240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d96e92ab60 0x55d96e92afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.521+0000 7fe91569f640 1 -- 172.21.15.43:0/3153019240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d96e84e320 con 0x55d96e9340e0 2024-04-21T08:51:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.521+0000 7fe91569f640 1 --2- 172.21.15.43:0/3153019240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e9340e0 0x55d96e9364d0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7fe904008fc0 tx=0x7fe904008580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14493b49d68257da server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.521+0000 7fe9166a1640 1 --2- 172.21.15.43:0/2491161848 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe8f80a3dc0 0x7fe8f80a6280 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe9000f02a0 tx=0x7fe900091530 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.521+0000 7fe8f7fff640 1 -- 172.21.15.43:0/3153019240 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe904018020 con 0x55d96e9340e0 2024-04-21T08:51:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.521+0000 7fe8f7fff640 1 -- 172.21.15.43:0/3153019240 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe90400cc00 con 0x55d96e9340e0 2024-04-21T08:51:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.521+0000 7fe8f7fff640 1 -- 172.21.15.43:0/3153019240 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90400fb90 con 0x55d96e9340e0 2024-04-21T08:51:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.522+0000 7fe91add4740 1 -- 172.21.15.43:0/3153019240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e9340e0 msgr2=0x55d96e9364d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.522+0000 7fe91add4740 1 --2- 172.21.15.43:0/3153019240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e9340e0 0x55d96e9364d0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7fe904008fc0 tx=0x7fe904008580 comp rx=0 tx=0).stop 2024-04-21T08:51:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.522+0000 7fe91add4740 1 -- 172.21.15.43:0/3153019240 shutdown_connections 2024-04-21T08:51:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.522+0000 7fe91add4740 1 --2- 172.21.15.43:0/3153019240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e9340e0 0x55d96e9364d0 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.522+0000 7fe91add4740 1 --2- 172.21.15.43:0/3153019240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d96e92ab60 0x55d96e92afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.522+0000 7fe91add4740 1 --2- 172.21.15.43:0/3153019240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d96e92b8d0 0x55d96e92a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.522+0000 7fe91add4740 1 -- 172.21.15.43:0/3153019240 >> 172.21.15.43:0/3153019240 conn(0x55d96e8f1c90 msgr2=0x55d96e925850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.522+0000 7fe91add4740 1 -- 172.21.15.43:0/3153019240 shutdown_connections 2024-04-21T08:51:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.522+0000 7fe91add4740 1 -- 172.21.15.43:0/3153019240 wait complete. 2024-04-21T08:51:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.522+0000 7fe91add4740 1 -- start start 2024-04-21T08:51:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.523+0000 7fe91add4740 1 Processor -- start 2024-04-21T08:51:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.523+0000 7fe91add4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d96e92ab60 0x55d96e933ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.523+0000 7fe91add4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d96e92b8d0 0x55d96e930e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.523+0000 7fe91add4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e9340e0 0x55d96e931480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.523+0000 7fe91add4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d96e936e10 con 0x55d96e9340e0 2024-04-21T08:51:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.523+0000 7fe91add4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d96e817eb0 con 0x55d96e92b8d0 2024-04-21T08:51:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.523+0000 7fe91add4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d96e936f90 con 0x55d96e92ab60 2024-04-21T08:51:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.523+0000 7fe91add4740 10 client.0 _ll_register_callbacks cb 0x55d96e844b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.523+0000 7fe91569f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e9340e0 0x55d96e931480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.523+0000 7fe91569f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e9340e0 0x55d96e931480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33892/0 (socket says 172.21.15.43:33892) 2024-04-21T08:51:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.523+0000 7fe91569f640 1 -- 172.21.15.43:0/1513798832 learned_addr learned my addr 172.21.15.43:0/1513798832 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.523+0000 7fe91569f640 1 --2- 172.21.15.43:0/1513798832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d96e92ab60 0x55d96e933ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.523+0000 7fe915ea0640 1 --2- 172.21.15.43:0/1513798832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d96e92b8d0 0x55d96e930e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.524+0000 7fe91569f640 1 -- 172.21.15.43:0/1513798832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d96e92ab60 msgr2=0x55d96e933ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.524+0000 7fe91569f640 1 --2- 172.21.15.43:0/1513798832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d96e92ab60 0x55d96e933ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.524+0000 7fe91569f640 1 -- 172.21.15.43:0/1513798832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d96e92b8d0 msgr2=0x55d96e930e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.524+0000 7fe91569f640 1 --2- 172.21.15.43:0/1513798832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d96e92b8d0 0x55d96e930e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.524+0000 7fe91569f640 1 -- 172.21.15.43:0/1513798832 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d96e5a2210 con 0x55d96e9340e0 2024-04-21T08:51:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.524+0000 7fe91569f640 1 --2- 172.21.15.43:0/1513798832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e9340e0 0x55d96e931480 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7fe904001eb0 tx=0x7fe90400f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e3de453a9cdb2e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.524+0000 7fe8f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.524+0000 7fe8f57fa640 1 -- 172.21.15.43:0/1513798832 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe904018070 con 0x55d96e9340e0 2024-04-21T08:51:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.524+0000 7fe91add4740 1 -- 172.21.15.43:0/1513798832 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe90400cc00 con 0x55d96e9340e0 2024-04-21T08:51:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.524+0000 7fe8f57fa640 1 -- 172.21.15.43:0/1513798832 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9671+0+0 (secure 0 0 0) 0x7fe904004050 con 0x55d96e9340e0 2024-04-21T08:51:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.524+0000 7fe8f57fa640 1 -- 172.21.15.43:0/1513798832 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe90402e050 con 0x55d96e9340e0 2024-04-21T08:51:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.524+0000 7fe91add4740 10 client.21228 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.524+0000 7fe91add4740 1 -- 172.21.15.43:0/1513798832 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d96e66e620 con 0x55d96e9340e0 2024-04-21T08:51:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.524+0000 7fe8f57fa640 1 -- 172.21.15.43:0/1513798832 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe90400cc00 con 0x55d96e9340e0 2024-04-21T08:51:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.525+0000 7fe8f57fa640 1 -- 172.21.15.43:0/1513798832 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe904005290 con 0x55d96e9340e0 2024-04-21T08:51:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.525+0000 7fe91add4740 10 client.21228 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.525+0000 7fe91add4740 10 client.21228 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.525+0000 7fe91add4740 20 client.21228 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.525+0000 7fe91add4740 1 --2- 172.21.15.43:0/1513798832 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d96e944070 0x55d96e946460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.525+0000 7fe91add4740 4 client.21228 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.525+0000 7fe91add4740 1 -- 172.21.15.43:0/1513798832 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d96e8a3880 con 0x55d96e944070 2024-04-21T08:51:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.525+0000 7fe915ea0640 1 --2- 172.21.15.43:0/1513798832 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d96e944070 0x55d96e946460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.525+0000 7fe8f57fa640 10 client.21228 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.525+0000 7fe915ea0640 1 --2- 172.21.15.43:0/1513798832 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d96e944070 0x55d96e946460 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76eb3714bd2f6d4c server_cookie=3a8f3782b6ced43c in_seq=0 out_seq=0 2024-04-21T08:51:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.526+0000 7fe8f57fa640 1 -- 172.21.15.43:0/1513798832 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d96e8a3880 con 0x55d96e944070 2024-04-21T08:51:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.526+0000 7fe8f57fa640 10 client.21228 handle_command_reply: tid=0 2024-04-21T08:51:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.526+0000 7fe91add4740 1 client.21228 shutdown 2024-04-21T08:51:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.526+0000 7fe91add4740 1 -- 172.21.15.43:0/1513798832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e9340e0 msgr2=0x55d96e931480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.526+0000 7fe91add4740 1 --2- 172.21.15.43:0/1513798832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e9340e0 0x55d96e931480 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7fe904001eb0 tx=0x7fe90400f830 comp rx=0 tx=0).stop 2024-04-21T08:51:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.526+0000 7fe91add4740 1 -- 172.21.15.43:0/1513798832 shutdown_connections 2024-04-21T08:51:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.526+0000 7fe91add4740 1 --2- 172.21.15.43:0/1513798832 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d96e944070 0x55d96e946460 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.526+0000 7fe91add4740 1 --2- 172.21.15.43:0/1513798832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e9340e0 0x55d96e931480 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.526+0000 7fe8f57fa640 0 client.21228 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.526+0000 7fe91add4740 1 --2- 172.21.15.43:0/1513798832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d96e92b8d0 0x55d96e930e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.526+0000 7fe91add4740 1 --2- 172.21.15.43:0/1513798832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d96e92ab60 0x55d96e933ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.526+0000 7fe91add4740 1 -- 172.21.15.43:0/1513798832 >> 172.21.15.43:0/1513798832 conn(0x55d96e8f1c90 msgr2=0x55d96e9275a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.526+0000 7fe91add4740 1 -- 172.21.15.43:0/1513798832 shutdown_connections 2024-04-21T08:51:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.526+0000 7fe91add4740 1 -- 172.21.15.43:0/1513798832 wait complete. 2024-04-21T08:51:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.527+0000 7fe91add4740 20 client.21228 trim_cache size 0 max 16384 2024-04-21T08:51:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.542+0000 7fe91add4740 1 Processor -- start 2024-04-21T08:51:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.542+0000 7fe91add4740 1 -- start start 2024-04-21T08:51:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.542+0000 7fe91add4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d96e92ab60 0x55d96e93fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.542+0000 7fe91add4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e92b8d0 0x55d96e9335a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.542+0000 7fe91add4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d96e9340e0 0x55d96e933ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.542+0000 7fe91add4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe9101b16f0 con 0x55d96e92b8d0 2024-04-21T08:51:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.542+0000 7fe91add4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe9101b3230 con 0x55d96e92ab60 2024-04-21T08:51:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.542+0000 7fe91add4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9101b1dc0 con 0x55d96e9340e0 2024-04-21T08:51:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.542+0000 7fe915ea0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e92b8d0 0x55d96e9335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.542+0000 7fe915ea0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e92b8d0 0x55d96e9335a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33900/0 (socket says 172.21.15.43:33900) 2024-04-21T08:51:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.542+0000 7fe915ea0640 1 -- 172.21.15.43:0/2987959529 learned_addr learned my addr 172.21.15.43:0/2987959529 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.542+0000 7fe91569f640 1 --2- 172.21.15.43:0/2987959529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d96e92ab60 0x55d96e93fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.542+0000 7fe915ea0640 1 -- 172.21.15.43:0/2987959529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d96e9340e0 msgr2=0x55d96e933ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.542+0000 7fe915ea0640 1 --2- 172.21.15.43:0/2987959529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d96e9340e0 0x55d96e933ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.542+0000 7fe915ea0640 1 -- 172.21.15.43:0/2987959529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d96e92ab60 msgr2=0x55d96e93fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.542+0000 7fe915ea0640 1 --2- 172.21.15.43:0/2987959529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d96e92ab60 0x55d96e93fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.542+0000 7fe915ea0640 1 -- 172.21.15.43:0/2987959529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d96e929e30 con 0x55d96e92b8d0 2024-04-21T08:51:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.542+0000 7fe91569f640 1 --2- 172.21.15.43:0/2987959529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d96e92ab60 0x55d96e93fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.542+0000 7fe915ea0640 1 --2- 172.21.15.43:0/2987959529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e92b8d0 0x55d96e9335a0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7fe908000c00 tx=0x7fe90800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b43dfaf3d16ebdb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.543+0000 7fe8f77fe640 1 -- 172.21.15.43:0/2987959529 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe908003de0 con 0x55d96e92b8d0 2024-04-21T08:51:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.543+0000 7fe8f77fe640 1 -- 172.21.15.43:0/2987959529 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe90801f5a0 con 0x55d96e92b8d0 2024-04-21T08:51:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.543+0000 7fe8f77fe640 1 -- 172.21.15.43:0/2987959529 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe908011af0 con 0x55d96e92b8d0 2024-04-21T08:51:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.543+0000 7fe91add4740 1 -- 172.21.15.43:0/2987959529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e92b8d0 msgr2=0x55d96e9335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.543+0000 7fe91add4740 1 --2- 172.21.15.43:0/2987959529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e92b8d0 0x55d96e9335a0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7fe908000c00 tx=0x7fe90800b390 comp rx=0 tx=0).stop 2024-04-21T08:51:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.543+0000 7fe91add4740 1 -- 172.21.15.43:0/2987959529 shutdown_connections 2024-04-21T08:51:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.543+0000 7fe91add4740 1 --2- 172.21.15.43:0/2987959529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d96e9340e0 0x55d96e933ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.543+0000 7fe91add4740 1 --2- 172.21.15.43:0/2987959529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e92b8d0 0x55d96e9335a0 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.543+0000 7fe91add4740 1 --2- 172.21.15.43:0/2987959529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d96e92ab60 0x55d96e93fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.543+0000 7fe91add4740 1 -- 172.21.15.43:0/2987959529 >> 172.21.15.43:0/2987959529 conn(0x55d96e8f1c90 msgr2=0x55d96e925380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.543+0000 7fe91add4740 1 -- 172.21.15.43:0/2987959529 shutdown_connections 2024-04-21T08:51:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.543+0000 7fe91add4740 1 -- 172.21.15.43:0/2987959529 wait complete. 2024-04-21T08:51:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.544+0000 7fe91add4740 1 -- start start 2024-04-21T08:51:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.544+0000 7fe91add4740 1 Processor -- start 2024-04-21T08:51:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.544+0000 7fe91add4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e92ab60 0x55d96e93c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.544+0000 7fe91add4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d96e92b8d0 0x55d96e93c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.544+0000 7fe91add4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d96e9340e0 0x55d96e92e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.544+0000 7fe91add4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe9101b2b60 con 0x55d96e92ab60 2024-04-21T08:51:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.544+0000 7fe91add4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe9101b2490 con 0x55d96e92b8d0 2024-04-21T08:51:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.544+0000 7fe91add4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9101af6f0 con 0x55d96e9340e0 2024-04-21T08:51:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.544+0000 7fe91add4740 10 client.0 _ll_register_callbacks cb 0x7fe90800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.544+0000 7fe91569f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e92ab60 0x55d96e93c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.544+0000 7fe91569f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e92ab60 0x55d96e93c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33910/0 (socket says 172.21.15.43:33910) 2024-04-21T08:51:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.544+0000 7fe91569f640 1 -- 172.21.15.43:0/1177896641 learned_addr learned my addr 172.21.15.43:0/1177896641 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.544+0000 7fe915ea0640 1 --2- 172.21.15.43:0/1177896641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d96e92b8d0 0x55d96e93c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.545+0000 7fe91569f640 1 --2- 172.21.15.43:0/1177896641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d96e9340e0 0x55d96e92e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.545+0000 7fe91569f640 1 -- 172.21.15.43:0/1177896641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d96e9340e0 msgr2=0x55d96e92e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.545+0000 7fe91569f640 1 --2- 172.21.15.43:0/1177896641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d96e9340e0 0x55d96e92e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.545+0000 7fe91569f640 1 -- 172.21.15.43:0/1177896641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d96e92b8d0 msgr2=0x55d96e93c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.545+0000 7fe91569f640 1 --2- 172.21.15.43:0/1177896641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d96e92b8d0 0x55d96e93c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.545+0000 7fe91569f640 1 -- 172.21.15.43:0/1177896641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d96e92e830 con 0x55d96e92ab60 2024-04-21T08:51:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.545+0000 7fe915ea0640 1 --2- 172.21.15.43:0/1177896641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d96e92b8d0 0x55d96e93c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.545+0000 7fe91569f640 1 --2- 172.21.15.43:0/1177896641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e92ab60 0x55d96e93c220 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fe904006010 tx=0x7fe90400f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd598c96cf0bac8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.545+0000 7fe8f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.545+0000 7fe8f57fa640 1 -- 172.21.15.43:0/1177896641 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe904018070 con 0x55d96e92ab60 2024-04-21T08:51:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.545+0000 7fe8f57fa640 1 -- 172.21.15.43:0/1177896641 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9671+0+0 (secure 0 0 0) 0x7fe904012070 con 0x55d96e92ab60 2024-04-21T08:51:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.545+0000 7fe91add4740 1 -- 172.21.15.43:0/1177896641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe90801f5a0 con 0x55d96e92ab60 2024-04-21T08:51:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.545+0000 7fe8f57fa640 1 -- 172.21.15.43:0/1177896641 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe904012270 con 0x55d96e92ab60 2024-04-21T08:51:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.545+0000 7fe91add4740 10 client.21234 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.545+0000 7fe91add4740 1 -- 172.21.15.43:0/1177896641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d96e934ea0 con 0x55d96e92ab60 2024-04-21T08:51:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.545+0000 7fe8f57fa640 1 -- 172.21.15.43:0/1177896641 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe90801f5a0 con 0x55d96e92ab60 2024-04-21T08:51:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.545+0000 7fe8f57fa640 1 -- 172.21.15.43:0/1177896641 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe9040051d0 con 0x55d96e92ab60 2024-04-21T08:51:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.546+0000 7fe91add4740 10 client.21234 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.546+0000 7fe91add4740 10 client.21234 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.546+0000 7fe91add4740 20 client.21234 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.546+0000 7fe91add4740 1 --2- 172.21.15.43:0/1177896641 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d96e9aa580 0x55d96e9ac9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.546+0000 7fe91add4740 4 client.21234 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:51:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.546+0000 7fe91add4740 1 -- 172.21.15.43:0/1177896641 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d96e82f3d0 con 0x55d96e9aa580 2024-04-21T08:51:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.546+0000 7fe915ea0640 1 --2- 172.21.15.43:0/1177896641 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d96e9aa580 0x55d96e9ac9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.546+0000 7fe8f57fa640 10 client.21234 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.546+0000 7fe915ea0640 1 --2- 172.21.15.43:0/1177896641 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d96e9aa580 0x55d96e9ac9a0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc3331d719530d9 server_cookie=384e305ac959485c in_seq=0 out_seq=0 2024-04-21T08:51:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.561+0000 7fe8f57fa640 1 -- 172.21.15.43:0/1177896641 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d96e82f3d0 con 0x55d96e9aa580 2024-04-21T08:51:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.561+0000 7fe8f57fa640 10 client.21234 handle_command_reply: tid=0 2024-04-21T08:51:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.561+0000 7fe91add4740 1 client.21234 shutdown 2024-04-21T08:51:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.561+0000 7fe91add4740 1 -- 172.21.15.43:0/1177896641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e92ab60 msgr2=0x55d96e93c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.561+0000 7fe91add4740 1 --2- 172.21.15.43:0/1177896641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e92ab60 0x55d96e93c220 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fe904006010 tx=0x7fe90400f8a0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.561+0000 7fe91add4740 1 -- 172.21.15.43:0/1177896641 shutdown_connections 2024-04-21T08:51:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.561+0000 7fe91add4740 1 --2- 172.21.15.43:0/1177896641 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d96e9aa580 0x55d96e9ac9a0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.561+0000 7fe91add4740 1 --2- 172.21.15.43:0/1177896641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d96e9340e0 0x55d96e92e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.561+0000 7fe91add4740 1 --2- 172.21.15.43:0/1177896641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d96e92b8d0 0x55d96e93c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.561+0000 7fe91add4740 1 --2- 172.21.15.43:0/1177896641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d96e92ab60 0x55d96e93c220 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.561+0000 7fe91add4740 1 -- 172.21.15.43:0/1177896641 >> 172.21.15.43:0/1177896641 conn(0x55d96e8f1c90 msgr2=0x55d96e92f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.561+0000 7fe8f57fa640 0 client.21234 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.561+0000 7fe91add4740 1 -- 172.21.15.43:0/1177896641 shutdown_connections 2024-04-21T08:51:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.561+0000 7fe91add4740 1 -- 172.21.15.43:0/1177896641 wait complete. 2024-04-21T08:51:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.561+0000 7fe91add4740 20 client.21234 trim_cache size 0 max 16384 2024-04-21T08:51:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:51:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:51:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:51:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:51:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.562+0000 7fe918104640 1 -- 172.21.15.43:0/2491161848 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe8f80a3dc0 msgr2=0x7fe8f80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.562+0000 7fe918104640 1 --2- 172.21.15.43:0/2491161848 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe8f80a3dc0 0x7fe8f80a6280 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe9000f02a0 tx=0x7fe900091530 comp rx=0 tx=0).stop 2024-04-21T08:51:06.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.562+0000 7fe918104640 1 -- 172.21.15.43:0/2491161848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe910074cd0 msgr2=0x7fe9101a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:06.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.562+0000 7fe918104640 1 --2- 172.21.15.43:0/2491161848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe910074cd0 0x7fe9101a67f0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7fe900005e30 tx=0x7fe900034760 comp rx=0 tx=0).stop 2024-04-21T08:51:06.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.562+0000 7fe918104640 1 -- 172.21.15.43:0/2491161848 shutdown_connections 2024-04-21T08:51:06.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.562+0000 7fe918104640 1 --2- 172.21.15.43:0/2491161848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe910074cd0 0x7fe9101a67f0 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.562+0000 7fe918104640 1 --2- 172.21.15.43:0/2491161848 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe8f80a3dc0 0x7fe8f80a6280 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.562+0000 7fe918104640 1 --2- 172.21.15.43:0/2491161848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe910107290 0x7fe910074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.562+0000 7fe918104640 1 --2- 172.21.15.43:0/2491161848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9101021c0 0x7fe910074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.562+0000 7fe918104640 1 -- 172.21.15.43:0/2491161848 >> 172.21.15.43:0/2491161848 conn(0x7fe9100fda20 msgr2=0x7fe9100fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:06.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.562+0000 7fe918104640 1 -- 172.21.15.43:0/2491161848 shutdown_connections 2024-04-21T08:51:06.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:06.562+0000 7fe918104640 1 -- 172.21.15.43:0/2491161848 wait complete. 2024-04-21T08:51:06.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:06 smithi043 ceph-mon[27978]: pgmap v380: 129 pgs: 129 active+clean; 1.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-21T08:51:06.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:06 smithi175 ceph-mon[31920]: pgmap v380: 129 pgs: 129 active+clean; 1.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-21T08:51:06.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:06 smithi112 ceph-mon[31085]: pgmap v380: 129 pgs: 129 active+clean; 1.9 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-21T08:51:08.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:08 smithi175 ceph-mon[31920]: pgmap v381: 129 pgs: 129 active+clean; 2.1 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 35 op/s 2024-04-21T08:51:08.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:08 smithi112 ceph-mon[31085]: pgmap v381: 129 pgs: 129 active+clean; 2.1 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 35 op/s 2024-04-21T08:51:08.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:08 smithi043 ceph-mon[27978]: pgmap v381: 129 pgs: 129 active+clean; 2.1 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 35 op/s 2024-04-21T08:51:10.719 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:10 smithi175 ceph-mon[31920]: pgmap v382: 129 pgs: 129 active+clean; 2.1 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 35 op/s 2024-04-21T08:51:10.806 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:10 smithi175 ceph-mon[31920]: osdmap e108: 12 total, 12 up, 12 in 2024-04-21T08:51:10.806 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:10 smithi112 ceph-mon[31085]: pgmap v382: 129 pgs: 129 active+clean; 2.1 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 35 op/s 2024-04-21T08:51:10.806 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:10 smithi112 ceph-mon[31085]: osdmap e108: 12 total, 12 up, 12 in 2024-04-21T08:51:10.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:10 smithi043 ceph-mon[27978]: pgmap v382: 129 pgs: 129 active+clean; 2.1 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 35 op/s 2024-04-21T08:51:10.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:10 smithi043 ceph-mon[27978]: osdmap e108: 12 total, 12 up, 12 in 2024-04-21T08:51:12.198 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:51:12.198 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:51:12.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.305+0000 7faa1cd2b640 1 -- 172.21.15.43:0/2627849568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa1810c500 msgr2=0x7faa1810e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.305+0000 7faa1cd2b640 1 --2- 172.21.15.43:0/2627849568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa1810c500 0x7faa1810e9a0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7faa10009ca0 tx=0x7faa1002c230 comp rx=0 tx=0).stop 2024-04-21T08:51:12.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.305+0000 7faa1cd2b640 1 -- 172.21.15.43:0/2627849568 shutdown_connections 2024-04-21T08:51:12.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.305+0000 7faa1cd2b640 1 --2- 172.21.15.43:0/2627849568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa1810c500 0x7faa1810e9a0 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.305+0000 7faa1cd2b640 1 --2- 172.21.15.43:0/2627849568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa18103ae0 0x7faa1810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.305+0000 7faa1cd2b640 1 --2- 172.21.15.43:0/2627849568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa181031c0 0x7faa181035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.305+0000 7faa1cd2b640 1 -- 172.21.15.43:0/2627849568 >> 172.21.15.43:0/2627849568 conn(0x7faa18078000 msgr2=0x7faa18078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:12.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.305+0000 7faa1cd2b640 1 -- 172.21.15.43:0/2627849568 shutdown_connections 2024-04-21T08:51:12.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.305+0000 7faa1cd2b640 1 -- 172.21.15.43:0/2627849568 wait complete. 2024-04-21T08:51:12.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.306+0000 7faa1cd2b640 1 Processor -- start 2024-04-21T08:51:12.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.306+0000 7faa1cd2b640 1 -- start start 2024-04-21T08:51:12.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.306+0000 7faa1cd2b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa181031c0 0x7faa181986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.306+0000 7faa1cd2b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa18103ae0 0x7faa18198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.306+0000 7faa1cd2b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa1810c500 0x7faa1819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.306+0000 7faa1cd2b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa18112170 con 0x7faa1810c500 2024-04-21T08:51:12.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.306+0000 7faa1cd2b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faa18111ff0 con 0x7faa181031c0 2024-04-21T08:51:12.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.306+0000 7faa1cd2b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faa181122f0 con 0x7faa18103ae0 2024-04-21T08:51:12.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.306+0000 7faa16d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa1810c500 0x7faa1819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.306+0000 7faa16d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa1810c500 0x7faa1819cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51270/0 (socket says 172.21.15.43:51270) 2024-04-21T08:51:12.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.306+0000 7faa16d9d640 1 -- 172.21.15.43:0/423183845 learned_addr learned my addr 172.21.15.43:0/423183845 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:12.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.306+0000 7faa0dd9b640 1 --2- 172.21.15.43:0/423183845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa18103ae0 0x7faa18198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.306+0000 7faa1659c640 1 --2- 172.21.15.43:0/423183845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa181031c0 0x7faa181986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.307+0000 7faa16d9d640 1 -- 172.21.15.43:0/423183845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa18103ae0 msgr2=0x7faa18198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.307+0000 7faa16d9d640 1 --2- 172.21.15.43:0/423183845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa18103ae0 0x7faa18198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.307+0000 7faa16d9d640 1 -- 172.21.15.43:0/423183845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa181031c0 msgr2=0x7faa181986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.307+0000 7faa16d9d640 1 --2- 172.21.15.43:0/423183845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa181031c0 0x7faa181986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.307+0000 7faa16d9d640 1 -- 172.21.15.43:0/423183845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1819d6a0 con 0x7faa1810c500 2024-04-21T08:51:12.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.307+0000 7faa16d9d640 1 --2- 172.21.15.43:0/423183845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa1810c500 0x7faa1819cf70 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7faa10005e30 tx=0x7faa10035dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:12.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.307+0000 7faa0ffff640 1 -- 172.21.15.43:0/423183845 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10039040 con 0x7faa1810c500 2024-04-21T08:51:12.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.307+0000 7faa0ffff640 1 -- 172.21.15.43:0/423183845 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa10044070 con 0x7faa1810c500 2024-04-21T08:51:12.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.307+0000 7faa1cd2b640 1 -- 172.21.15.43:0/423183845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa1819d930 con 0x7faa1810c500 2024-04-21T08:51:12.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.307+0000 7faa0ffff640 1 -- 172.21.15.43:0/423183845 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa100328a0 con 0x7faa1810c500 2024-04-21T08:51:12.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.307+0000 7faa1cd2b640 1 -- 172.21.15.43:0/423183845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa181a5220 con 0x7faa1810c500 2024-04-21T08:51:12.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.310+0000 7faa0ffff640 1 -- 172.21.15.43:0/423183845 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faa10032a40 con 0x7faa1810c500 2024-04-21T08:51:12.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.310+0000 7faa1f9fb740 1 Processor -- start 2024-04-21T08:51:12.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.310+0000 7faa1f9fb740 1 -- start start 2024-04-21T08:51:12.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.310+0000 7faa1f9fb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f70f60 0x55e4e0f71340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.310+0000 7faa1f9fb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4e0f71910 0x55e4e0f7e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.310+0000 7faa1f9fb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4e0f7ee90 0x55e4e0f81280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.310+0000 7faa1f9fb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e4e0ed61e0 con 0x55e4e0f70f60 2024-04-21T08:51:12.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.310+0000 7faa1f9fb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e4e0e73390 con 0x55e4e0f71910 2024-04-21T08:51:12.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.310+0000 7faa1f9fb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e4e0ed21a0 con 0x55e4e0f7ee90 2024-04-21T08:51:12.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.310+0000 7faa0dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f70f60 0x55e4e0f71340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.310+0000 7faa0dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f70f60 0x55e4e0f71340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51276/0 (socket says 172.21.15.43:51276) 2024-04-21T08:51:12.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.310+0000 7faa0dd9b640 1 -- 172.21.15.43:0/4111884421 learned_addr learned my addr 172.21.15.43:0/4111884421 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:12.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.310+0000 7faa1659c640 1 --2- 172.21.15.43:0/4111884421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4e0f71910 0x55e4e0f7e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.310+0000 7faa0dd9b640 1 --2- 172.21.15.43:0/4111884421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4e0f7ee90 0x55e4e0f81280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.311+0000 7faa0dd9b640 1 -- 172.21.15.43:0/4111884421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4e0f7ee90 msgr2=0x55e4e0f81280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:12.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.311+0000 7faa0dd9b640 1 --2- 172.21.15.43:0/4111884421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4e0f7ee90 0x55e4e0f81280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.311+0000 7faa0dd9b640 1 -- 172.21.15.43:0/4111884421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4e0f71910 msgr2=0x55e4e0f7e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:12.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.311+0000 7faa0dd9b640 1 --2- 172.21.15.43:0/4111884421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4e0f71910 0x55e4e0f7e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.311+0000 7faa0ffff640 1 --2- 172.21.15.43:0/423183845 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faa040a3d20 0x7faa040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.311+0000 7faa0dd9b640 1 -- 172.21.15.43:0/4111884421 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4e0cb27d0 con 0x55e4e0f70f60 2024-04-21T08:51:12.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.311+0000 7faa0ffff640 1 -- 172.21.15.43:0/423183845 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7faa100f2e30 con 0x7faa1810c500 2024-04-21T08:51:12.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.311+0000 7faa0dd9b640 1 --2- 172.21.15.43:0/4111884421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f70f60 0x55e4e0f71340 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7faa00009f70 tx=0x7faa00005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f2e0f07552e0e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:12.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.311+0000 7faa0dd9b640 1 --2- 172.21.15.43:0/423183845 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faa040a3d20 0x7faa040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.311+0000 7faa0cd99640 1 -- 172.21.15.43:0/4111884421 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa00004250 con 0x55e4e0f70f60 2024-04-21T08:51:12.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.311+0000 7faa0cd99640 1 -- 172.21.15.43:0/4111884421 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa00003b20 con 0x55e4e0f70f60 2024-04-21T08:51:12.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.311+0000 7faa0cd99640 1 -- 172.21.15.43:0/4111884421 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa00011e20 con 0x55e4e0f70f60 2024-04-21T08:51:12.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.312+0000 7faa1f9fb740 1 -- 172.21.15.43:0/4111884421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f70f60 msgr2=0x55e4e0f71340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.312+0000 7faa1f9fb740 1 --2- 172.21.15.43:0/4111884421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f70f60 0x55e4e0f71340 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7faa00009f70 tx=0x7faa00005850 comp rx=0 tx=0).stop 2024-04-21T08:51:12.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.312+0000 7faa0dd9b640 1 --2- 172.21.15.43:0/423183845 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faa040a3d20 0x7faa040a61e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7faa0001e000 tx=0x7faa0001f4a0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:12.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.312+0000 7faa1f9fb740 1 -- 172.21.15.43:0/4111884421 shutdown_connections 2024-04-21T08:51:12.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.312+0000 7faa1f9fb740 1 --2- 172.21.15.43:0/4111884421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4e0f7ee90 0x55e4e0f81280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.312+0000 7faa1f9fb740 1 --2- 172.21.15.43:0/4111884421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4e0f71910 0x55e4e0f7e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.312+0000 7faa1f9fb740 1 --2- 172.21.15.43:0/4111884421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f70f60 0x55e4e0f71340 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.312+0000 7faa1f9fb740 1 -- 172.21.15.43:0/4111884421 >> 172.21.15.43:0/4111884421 conn(0x55e4e0f6bcb0 msgr2=0x55e4e0f6c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:12.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.312+0000 7faa1f9fb740 1 -- 172.21.15.43:0/4111884421 shutdown_connections 2024-04-21T08:51:12.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.312+0000 7faa1f9fb740 1 -- 172.21.15.43:0/4111884421 wait complete. 2024-04-21T08:51:12.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.312+0000 7faa1f9fb740 1 -- start start 2024-04-21T08:51:12.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.312+0000 7faa1f9fb740 1 Processor -- start 2024-04-21T08:51:12.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.313+0000 7faa1f9fb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4e0f70f60 0x55e4e0f7be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.313+0000 7faa1f9fb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f71910 0x55e4e0f7c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.313+0000 7faa1f9fb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4e0f7ee90 0x55e4e0f7c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.313+0000 7faa1f9fb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e4e0f81c50 con 0x55e4e0f71910 2024-04-21T08:51:12.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.313+0000 7faa1f9fb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e4e0e8f2f0 con 0x55e4e0f70f60 2024-04-21T08:51:12.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.313+0000 7faa1f9fb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e4e0f81dd0 con 0x55e4e0f7ee90 2024-04-21T08:51:12.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.313+0000 7faa1f9fb740 10 client.0 _ll_register_callbacks cb 0x55e4e0e9c580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:12.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.313+0000 7faa0dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f71910 0x55e4e0f7c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.313+0000 7faa0dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f71910 0x55e4e0f7c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51278/0 (socket says 172.21.15.43:51278) 2024-04-21T08:51:12.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.313+0000 7faa0dd9b640 1 -- 172.21.15.43:0/2450493869 learned_addr learned my addr 172.21.15.43:0/2450493869 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:12.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.313+0000 7faa1659c640 1 --2- 172.21.15.43:0/2450493869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4e0f70f60 0x55e4e0f7be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.313+0000 7faa16d9d640 1 --2- 172.21.15.43:0/2450493869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4e0f7ee90 0x55e4e0f7c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.313+0000 7faa0dd9b640 1 -- 172.21.15.43:0/2450493869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4e0f7ee90 msgr2=0x55e4e0f7c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:12.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.313+0000 7faa0dd9b640 1 --2- 172.21.15.43:0/2450493869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4e0f7ee90 0x55e4e0f7c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.313+0000 7faa0dd9b640 1 -- 172.21.15.43:0/2450493869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4e0f70f60 msgr2=0x55e4e0f7be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:12.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.313+0000 7faa0dd9b640 1 --2- 172.21.15.43:0/2450493869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4e0f70f60 0x55e4e0f7be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.313+0000 7faa0dd9b640 1 -- 172.21.15.43:0/2450493869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4e0be83f0 con 0x55e4e0f71910 2024-04-21T08:51:12.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.313+0000 7faa1659c640 1 --2- 172.21.15.43:0/2450493869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4e0f70f60 0x55e4e0f7be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:12.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.313+0000 7faa0dd9b640 1 --2- 172.21.15.43:0/2450493869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f71910 0x55e4e0f7c340 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7faa00003820 tx=0x7faa0001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f551eefdd6fedd32 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:12.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.313+0000 7fa9fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:12.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.313+0000 7fa9fdffb640 1 -- 172.21.15.43:0/2450493869 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0003b060 con 0x55e4e0f71910 2024-04-21T08:51:12.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.314+0000 7fa9fdffb640 1 -- 172.21.15.43:0/2450493869 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7faa00003cd0 con 0x55e4e0f71910 2024-04-21T08:51:12.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.314+0000 7faa1f9fb740 1 -- 172.21.15.43:0/2450493869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa00003b20 con 0x55e4e0f71910 2024-04-21T08:51:12.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.314+0000 7fa9fdffb640 1 -- 172.21.15.43:0/2450493869 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faa00042050 con 0x55e4e0f71910 2024-04-21T08:51:12.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.314+0000 7faa1f9fb740 10 client.21246 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:12.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.314+0000 7faa1f9fb740 1 -- 172.21.15.43:0/2450493869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e4e0ba80f0 con 0x55e4e0f71910 2024-04-21T08:51:12.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.314+0000 7fa9fdffb640 1 -- 172.21.15.43:0/2450493869 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa00003b20 con 0x55e4e0f71910 2024-04-21T08:51:12.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.314+0000 7fa9fdffb640 1 -- 172.21.15.43:0/2450493869 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7faa0001d020 con 0x55e4e0f71910 2024-04-21T08:51:12.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.314+0000 7faa1f9fb740 10 client.21246 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:12.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.314+0000 7faa1f9fb740 10 client.21246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:12.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.314+0000 7faa1f9fb740 20 client.21246 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:12.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.314+0000 7faa1f9fb740 1 --2- 172.21.15.43:0/2450493869 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e4e0f8ed90 0x55e4e0f91180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.314+0000 7faa1f9fb740 4 client.21246 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:12.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.314+0000 7faa1f9fb740 1 -- 172.21.15.43:0/2450493869 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4e0ee3c00 con 0x55e4e0f8ed90 2024-04-21T08:51:12.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.314+0000 7faa1659c640 1 --2- 172.21.15.43:0/2450493869 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e4e0f8ed90 0x55e4e0f91180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.315+0000 7fa9fdffb640 10 client.21246 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:12.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.315+0000 7faa1659c640 1 --2- 172.21.15.43:0/2450493869 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e4e0f8ed90 0x55e4e0f91180 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=266c0ed966d64215 server_cookie=ff143e0b179da99c in_seq=0 out_seq=0 2024-04-21T08:51:12.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.315+0000 7fa9fdffb640 1 -- 172.21.15.43:0/2450493869 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e4e0ee3c00 con 0x55e4e0f8ed90 2024-04-21T08:51:12.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.315+0000 7fa9fdffb640 10 client.21246 handle_command_reply: tid=0 2024-04-21T08:51:12.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.316+0000 7faa1f9fb740 1 client.21246 shutdown 2024-04-21T08:51:12.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.316+0000 7faa1f9fb740 1 -- 172.21.15.43:0/2450493869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f71910 msgr2=0x55e4e0f7c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.316+0000 7faa1f9fb740 1 --2- 172.21.15.43:0/2450493869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f71910 0x55e4e0f7c340 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7faa00003820 tx=0x7faa0001a000 comp rx=0 tx=0).stop 2024-04-21T08:51:12.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.316+0000 7faa1f9fb740 1 -- 172.21.15.43:0/2450493869 shutdown_connections 2024-04-21T08:51:12.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.316+0000 7faa1f9fb740 1 --2- 172.21.15.43:0/2450493869 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e4e0f8ed90 0x55e4e0f91180 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.316+0000 7faa1f9fb740 1 --2- 172.21.15.43:0/2450493869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4e0f7ee90 0x55e4e0f7c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.316+0000 7faa1f9fb740 1 --2- 172.21.15.43:0/2450493869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f71910 0x55e4e0f7c340 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.316+0000 7fa9fdffb640 0 client.21246 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:12.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.316+0000 7faa1f9fb740 1 --2- 172.21.15.43:0/2450493869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4e0f70f60 0x55e4e0f7be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.316+0000 7faa1f9fb740 1 -- 172.21.15.43:0/2450493869 >> 172.21.15.43:0/2450493869 conn(0x55e4e0f6bcb0 msgr2=0x55e4e0f6da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:12.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.316+0000 7faa1f9fb740 1 -- 172.21.15.43:0/2450493869 shutdown_connections 2024-04-21T08:51:12.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.316+0000 7faa1f9fb740 1 -- 172.21.15.43:0/2450493869 wait complete. 2024-04-21T08:51:12.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.316+0000 7faa1f9fb740 20 client.21246 trim_cache size 0 max 16384 2024-04-21T08:51:12.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.333+0000 7faa1f9fb740 1 Processor -- start 2024-04-21T08:51:12.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.334+0000 7faa1f9fb740 1 -- start start 2024-04-21T08:51:12.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.334+0000 7faa1f9fb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4e0f70f60 0x55e4e0f7e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.334+0000 7faa1f9fb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f71910 0x55e4e0f8ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.334+0000 7faa1f9fb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4e0f7ee90 0x55e4e0f7c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.334+0000 7faa1f9fb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa181ad310 con 0x55e4e0f71910 2024-04-21T08:51:12.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.334+0000 7faa1f9fb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faa181aee50 con 0x55e4e0f70f60 2024-04-21T08:51:12.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.334+0000 7faa1f9fb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faa181ad9e0 con 0x55e4e0f7ee90 2024-04-21T08:51:12.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.334+0000 7faa0dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f71910 0x55e4e0f8ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.334+0000 7faa0dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f71910 0x55e4e0f8ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51282/0 (socket says 172.21.15.43:51282) 2024-04-21T08:51:12.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.334+0000 7faa0dd9b640 1 -- 172.21.15.43:0/238456593 learned_addr learned my addr 172.21.15.43:0/238456593 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:12.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.334+0000 7faa1659c640 1 --2- 172.21.15.43:0/238456593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4e0f70f60 0x55e4e0f7e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.334+0000 7faa16d9d640 1 --2- 172.21.15.43:0/238456593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4e0f7ee90 0x55e4e0f7c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.334+0000 7faa0dd9b640 1 -- 172.21.15.43:0/238456593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4e0f7ee90 msgr2=0x55e4e0f7c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:12.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.334+0000 7faa0dd9b640 1 --2- 172.21.15.43:0/238456593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4e0f7ee90 0x55e4e0f7c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.334+0000 7faa0dd9b640 1 -- 172.21.15.43:0/238456593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4e0f70f60 msgr2=0x55e4e0f7e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:12.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.334+0000 7faa0dd9b640 1 --2- 172.21.15.43:0/238456593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4e0f70f60 0x55e4e0f7e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.334+0000 7faa0dd9b640 1 -- 172.21.15.43:0/238456593 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4e0f70670 con 0x55e4e0f71910 2024-04-21T08:51:12.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.334+0000 7faa1659c640 1 --2- 172.21.15.43:0/238456593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4e0f70f60 0x55e4e0f7e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:12.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.335+0000 7faa0dd9b640 1 --2- 172.21.15.43:0/238456593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f71910 0x55e4e0f8ab80 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7faa0001f9a0 tx=0x7faa0001faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f57c52879441f9a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:12.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.335+0000 7fa9fffff640 1 -- 172.21.15.43:0/238456593 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0003b070 con 0x55e4e0f71910 2024-04-21T08:51:12.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.335+0000 7fa9fffff640 1 -- 172.21.15.43:0/238456593 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa00032040 con 0x55e4e0f71910 2024-04-21T08:51:12.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.335+0000 7fa9fffff640 1 -- 172.21.15.43:0/238456593 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa00005240 con 0x55e4e0f71910 2024-04-21T08:51:12.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.335+0000 7faa1f9fb740 1 -- 172.21.15.43:0/238456593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f71910 msgr2=0x55e4e0f8ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.335+0000 7faa1f9fb740 1 --2- 172.21.15.43:0/238456593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f71910 0x55e4e0f8ab80 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7faa0001f9a0 tx=0x7faa0001faf0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.335+0000 7faa1f9fb740 1 -- 172.21.15.43:0/238456593 shutdown_connections 2024-04-21T08:51:12.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.336+0000 7faa1f9fb740 1 --2- 172.21.15.43:0/238456593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4e0f7ee90 0x55e4e0f7c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.336+0000 7faa1f9fb740 1 --2- 172.21.15.43:0/238456593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f71910 0x55e4e0f8ab80 unknown :-1 s=CLOSED pgs=2310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.336+0000 7faa1f9fb740 1 --2- 172.21.15.43:0/238456593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4e0f70f60 0x55e4e0f7e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.336+0000 7faa1f9fb740 1 -- 172.21.15.43:0/238456593 >> 172.21.15.43:0/238456593 conn(0x55e4e0f6bcb0 msgr2=0x55e4e0f7bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:12.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.336+0000 7faa1f9fb740 1 -- 172.21.15.43:0/238456593 shutdown_connections 2024-04-21T08:51:12.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.336+0000 7faa1f9fb740 1 -- 172.21.15.43:0/238456593 wait complete. 2024-04-21T08:51:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.336+0000 7faa1f9fb740 1 -- start start 2024-04-21T08:51:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.336+0000 7faa1f9fb740 1 Processor -- start 2024-04-21T08:51:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.336+0000 7faa1f9fb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4e0f70f60 0x55e4e0f87480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.336+0000 7faa1f9fb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f71910 0x55e4e0f78810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.336+0000 7faa1f9fb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4e0f7ee90 0x55e4e0f78d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.336+0000 7faa1f9fb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa181ae780 con 0x55e4e0f71910 2024-04-21T08:51:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.336+0000 7faa1f9fb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faa181ae0b0 con 0x55e4e0f70f60 2024-04-21T08:51:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.336+0000 7faa1f9fb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faa181ab310 con 0x55e4e0f7ee90 2024-04-21T08:51:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.336+0000 7faa1f9fb740 10 client.0 _ll_register_callbacks cb 0x7faa080097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.336+0000 7faa0dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f71910 0x55e4e0f78810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.336+0000 7faa1659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4e0f70f60 0x55e4e0f87480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.336+0000 7faa1659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4e0f70f60 0x55e4e0f87480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:60108/0 (socket says 172.21.15.43:60108) 2024-04-21T08:51:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.336+0000 7faa1659c640 1 -- 172.21.15.43:0/2359493088 learned_addr learned my addr 172.21.15.43:0/2359493088 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:12.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.337+0000 7faa16d9d640 1 --2- 172.21.15.43:0/2359493088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4e0f7ee90 0x55e4e0f78d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.337+0000 7faa1659c640 1 -- 172.21.15.43:0/2359493088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4e0f7ee90 msgr2=0x55e4e0f78d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:12.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.337+0000 7faa1659c640 1 --2- 172.21.15.43:0/2359493088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4e0f7ee90 0x55e4e0f78d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.337+0000 7faa1659c640 1 -- 172.21.15.43:0/2359493088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f71910 msgr2=0x55e4e0f78810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:12.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.337+0000 7faa1659c640 1 --2- 172.21.15.43:0/2359493088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f71910 0x55e4e0f78810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.337+0000 7faa1659c640 1 -- 172.21.15.43:0/2359493088 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4e0f79580 con 0x55e4e0f70f60 2024-04-21T08:51:12.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.337+0000 7faa1659c640 1 --2- 172.21.15.43:0/2359493088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4e0f70f60 0x55e4e0f87480 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55e4e0f91880 tx=0x7faa08012b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da7168d82d7288a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:12.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.337+0000 7fa9fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-21T08:51:12.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.337+0000 7fa9fdffb640 1 -- 172.21.15.43:0/2359493088 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08004230 con 0x55e4e0f70f60 2024-04-21T08:51:12.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.337+0000 7faa1f9fb740 1 -- 172.21.15.43:0/2359493088 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa00032040 con 0x55e4e0f70f60 2024-04-21T08:51:12.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.337+0000 7fa9fdffb640 1 -- 172.21.15.43:0/2359493088 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7faa080043d0 con 0x55e4e0f70f60 2024-04-21T08:51:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.337+0000 7fa9fdffb640 1 -- 172.21.15.43:0/2359493088 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faa08030050 con 0x55e4e0f70f60 2024-04-21T08:51:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.337+0000 7faa1f9fb740 10 client.29672 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.337+0000 7faa1f9fb740 1 -- 172.21.15.43:0/2359493088 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e4e0ff5250 con 0x55e4e0f70f60 2024-04-21T08:51:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.338+0000 7fa9fdffb640 1 -- 172.21.15.43:0/2359493088 <== mon.2 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa00032040 con 0x55e4e0f70f60 2024-04-21T08:51:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.338+0000 7fa9fdffb640 1 -- 172.21.15.43:0/2359493088 <== mon.2 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7faa08003aa0 con 0x55e4e0f70f60 2024-04-21T08:51:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.338+0000 7faa1f9fb740 10 client.29672 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.338+0000 7faa1f9fb740 10 client.29672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.338+0000 7faa1f9fb740 20 client.29672 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.338+0000 7faa1f9fb740 1 --2- 172.21.15.43:0/2359493088 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e4e0ff5970 0x55e4e0ff7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.338+0000 7faa1f9fb740 4 client.29672 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:51:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.338+0000 7faa1f9fb740 1 -- 172.21.15.43:0/2359493088 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e4e0edda70 con 0x55e4e0ff5970 2024-04-21T08:51:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.338+0000 7faa0dd9b640 1 --2- 172.21.15.43:0/2359493088 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e4e0ff5970 0x55e4e0ff7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.338+0000 7faa0dd9b640 1 --2- 172.21.15.43:0/2359493088 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e4e0ff5970 0x55e4e0ff7d90 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94578864ac1b3e41 server_cookie=10c03e5b377aea23 in_seq=0 out_seq=0 2024-04-21T08:51:12.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.338+0000 7fa9fdffb640 10 client.29672 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:12.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.339+0000 7fa9fdffb640 1 -- 172.21.15.43:0/2359493088 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e4e0edda70 con 0x55e4e0ff5970 2024-04-21T08:51:12.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.339+0000 7fa9fdffb640 10 client.29672 handle_command_reply: tid=0 2024-04-21T08:51:12.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.339+0000 7faa1f9fb740 1 client.29672 shutdown 2024-04-21T08:51:12.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.339+0000 7faa1f9fb740 1 -- 172.21.15.43:0/2359493088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4e0f70f60 msgr2=0x55e4e0f87480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.339+0000 7faa1f9fb740 1 --2- 172.21.15.43:0/2359493088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4e0f70f60 0x55e4e0f87480 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55e4e0f91880 tx=0x7faa08012b10 comp rx=0 tx=0).stop 2024-04-21T08:51:12.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.339+0000 7faa1f9fb740 1 -- 172.21.15.43:0/2359493088 shutdown_connections 2024-04-21T08:51:12.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.339+0000 7faa1f9fb740 1 --2- 172.21.15.43:0/2359493088 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55e4e0ff5970 0x55e4e0ff7d90 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.339+0000 7faa1f9fb740 1 --2- 172.21.15.43:0/2359493088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e4e0f7ee90 0x55e4e0f78d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.339+0000 7faa1f9fb740 1 --2- 172.21.15.43:0/2359493088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4e0f71910 0x55e4e0f78810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.339+0000 7faa1f9fb740 1 --2- 172.21.15.43:0/2359493088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4e0f70f60 0x55e4e0f87480 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.339+0000 7fa9fdffb640 0 client.29672 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:12.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.339+0000 7faa1f9fb740 1 -- 172.21.15.43:0/2359493088 >> 172.21.15.43:0/2359493088 conn(0x55e4e0f6bcb0 msgr2=0x55e4e0f6ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:12.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.339+0000 7faa1f9fb740 1 -- 172.21.15.43:0/2359493088 shutdown_connections 2024-04-21T08:51:12.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.339+0000 7faa1f9fb740 1 -- 172.21.15.43:0/2359493088 wait complete. 2024-04-21T08:51:12.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.339+0000 7faa1f9fb740 20 client.29672 trim_cache size 0 max 16384 2024-04-21T08:51:12.345 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:51:12.345 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:51:12.345 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:51:12.345 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:51:12.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.340+0000 7faa1cd2b640 1 -- 172.21.15.43:0/423183845 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faa040a3d20 msgr2=0x7faa040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.340+0000 7faa1cd2b640 1 --2- 172.21.15.43:0/423183845 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faa040a3d20 0x7faa040a61e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7faa0001e000 tx=0x7faa0001f4a0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.340+0000 7faa1cd2b640 1 -- 172.21.15.43:0/423183845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa1810c500 msgr2=0x7faa1819cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.340+0000 7faa1cd2b640 1 --2- 172.21.15.43:0/423183845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa1810c500 0x7faa1819cf70 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7faa10005e30 tx=0x7faa10035dd0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.340+0000 7faa1cd2b640 1 -- 172.21.15.43:0/423183845 shutdown_connections 2024-04-21T08:51:12.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.340+0000 7faa1cd2b640 1 --2- 172.21.15.43:0/423183845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa1810c500 0x7faa1819cf70 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.340+0000 7faa1cd2b640 1 --2- 172.21.15.43:0/423183845 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7faa040a3d20 0x7faa040a61e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.340+0000 7faa1cd2b640 1 --2- 172.21.15.43:0/423183845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa18103ae0 0x7faa18198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.340+0000 7faa1cd2b640 1 --2- 172.21.15.43:0/423183845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa181031c0 0x7faa181986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.340+0000 7faa1cd2b640 1 -- 172.21.15.43:0/423183845 >> 172.21.15.43:0/423183845 conn(0x7faa18078000 msgr2=0x7faa180fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:12.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.340+0000 7faa1cd2b640 1 -- 172.21.15.43:0/423183845 shutdown_connections 2024-04-21T08:51:12.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.340+0000 7faa1cd2b640 1 -- 172.21.15.43:0/423183845 wait complete. 2024-04-21T08:51:12.355 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:51:12.355 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:51:12.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.462+0000 7f712cf9f640 1 -- 172.21.15.43:0/1112903668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7128107260 msgr2=0x7f712810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.462+0000 7f712cf9f640 1 --2- 172.21.15.43:0/1112903668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7128107260 0x7f712810ee30 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f7120009ca0 tx=0x7f712002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.462+0000 7f712cf9f640 1 -- 172.21.15.43:0/1112903668 shutdown_connections 2024-04-21T08:51:12.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.462+0000 7f712cf9f640 1 --2- 172.21.15.43:0/1112903668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7128107260 0x7f712810ee30 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.462+0000 7f712cf9f640 1 --2- 172.21.15.43:0/1112903668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f712806b200 0x7f7128106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.462+0000 7f712cf9f640 1 --2- 172.21.15.43:0/1112903668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f712806a850 0x7f712806ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.462+0000 7f712cf9f640 1 -- 172.21.15.43:0/1112903668 >> 172.21.15.43:0/1112903668 conn(0x7f7128077c20 msgr2=0x7f7128078030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:12.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.462+0000 7f712cf9f640 1 -- 172.21.15.43:0/1112903668 shutdown_connections 2024-04-21T08:51:12.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.462+0000 7f712cf9f640 1 -- 172.21.15.43:0/1112903668 wait complete. 2024-04-21T08:51:12.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.463+0000 7f712cf9f640 1 Processor -- start 2024-04-21T08:51:12.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.463+0000 7f712cf9f640 1 -- start start 2024-04-21T08:51:12.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.463+0000 7f712cf9f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f712806a850 0x7f7128101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.463+0000 7f712cf9f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7128107260 0x7f71281017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.463+0000 7f712cf9f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7128101cf0 0x7f71281a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.463+0000 7f712cf9f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f71281121d0 con 0x7f7128107260 2024-04-21T08:51:12.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.463+0000 7f712cf9f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7128112050 con 0x7f7128101cf0 2024-04-21T08:51:12.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.463+0000 7f712cf9f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7128112350 con 0x7f712806a850 2024-04-21T08:51:12.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.463+0000 7f7125d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7128107260 0x7f71281017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.463+0000 7f7125d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7128107260 0x7f71281017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51306/0 (socket says 172.21.15.43:51306) 2024-04-21T08:51:12.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.463+0000 7f7125d9b640 1 -- 172.21.15.43:0/3979604342 learned_addr learned my addr 172.21.15.43:0/3979604342 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:12.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.463+0000 7f712659c640 1 --2- 172.21.15.43:0/3979604342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f712806a850 0x7f7128101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.463+0000 7f7126d9d640 1 --2- 172.21.15.43:0/3979604342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7128101cf0 0x7f71281a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.464+0000 7f7125d9b640 1 -- 172.21.15.43:0/3979604342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f712806a850 msgr2=0x7f7128101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.464+0000 7f7125d9b640 1 --2- 172.21.15.43:0/3979604342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f712806a850 0x7f7128101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.464+0000 7f7125d9b640 1 -- 172.21.15.43:0/3979604342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7128101cf0 msgr2=0x7f71281a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.464+0000 7f7125d9b640 1 --2- 172.21.15.43:0/3979604342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7128101cf0 0x7f71281a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.464+0000 7f7125d9b640 1 -- 172.21.15.43:0/3979604342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71281a7000 con 0x7f7128107260 2024-04-21T08:51:12.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.464+0000 7f7125d9b640 1 --2- 172.21.15.43:0/3979604342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7128107260 0x7f71281017b0 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f711c00e9f0 tx=0x7f711c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:12.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.464+0000 7f71177fe640 1 -- 172.21.15.43:0/3979604342 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c00ccc0 con 0x7f7128107260 2024-04-21T08:51:12.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.464+0000 7f712cf9f640 1 -- 172.21.15.43:0/3979604342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71281a72f0 con 0x7f7128107260 2024-04-21T08:51:12.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.464+0000 7f71177fe640 1 -- 172.21.15.43:0/3979604342 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f711c0054d0 con 0x7f7128107260 2024-04-21T08:51:12.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.464+0000 7f71177fe640 1 -- 172.21.15.43:0/3979604342 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c010430 con 0x7f7128107260 2024-04-21T08:51:12.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.464+0000 7f712cf9f640 1 -- 172.21.15.43:0/3979604342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71281a7840 con 0x7f7128107260 2024-04-21T08:51:12.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.465+0000 7f712cf9f640 1 -- 172.21.15.43:0/3979604342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7128103660 con 0x7f7128107260 2024-04-21T08:51:12.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.466+0000 7f71177fe640 1 -- 172.21.15.43:0/3979604342 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f711c0046c0 con 0x7f7128107260 2024-04-21T08:51:12.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.467+0000 7f71177fe640 1 --2- 172.21.15.43:0/3979604342 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f71100a3d50 0x7f71100a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.467+0000 7f712659c640 1 --2- 172.21.15.43:0/3979604342 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f71100a3d50 0x7f71100a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.467+0000 7f71177fe640 1 -- 172.21.15.43:0/3979604342 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f711c0d10f0 con 0x7f7128107260 2024-04-21T08:51:12.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.468+0000 7f712659c640 1 --2- 172.21.15.43:0/3979604342 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f71100a3d50 0x7f71100a6210 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7118009bd0 tx=0x7f7118005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:12.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.471+0000 7f71177fe640 1 -- 172.21.15.43:0/3979604342 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f711c053ff0 con 0x7f7128107260 2024-04-21T08:51:12.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:12 smithi043 ceph-mon[27978]: pgmap v384: 129 pgs: 129 active+clean; 2.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 63 KiB/s wr, 23 op/s 2024-04-21T08:51:12.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:12 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:51:12.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.700+0000 7f712cf9f640 1 -- 172.21.15.43:0/3979604342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7128069230 con 0x7f7128107260 2024-04-21T08:51:12.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.701+0000 7f71177fe640 1 -- 172.21.15.43:0/3979604342 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11935 (secure 0 0 0) 0x7f711c09d350 con 0x7f7128107260 2024-04-21T08:51:12.700 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:51:12.701 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:46:17.283806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27556":{"gid":27556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/229985237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":229985237},{"type":"v1","addr":"172.21.15.175:6837","nonce":229985237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24425,24412,24487,24527,24491]},"id":1}]} 2024-04-21T08:51:12.701 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-04-21T08:51:12.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.703+0000 7f712cf9f640 1 -- 172.21.15.43:0/3979604342 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f71100a3d50 msgr2=0x7f71100a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.703+0000 7f712cf9f640 1 --2- 172.21.15.43:0/3979604342 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f71100a3d50 0x7f71100a6210 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7118009bd0 tx=0x7f7118005f90 comp rx=0 tx=0).stop 2024-04-21T08:51:12.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.703+0000 7f712cf9f640 1 -- 172.21.15.43:0/3979604342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7128107260 msgr2=0x7f71281017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.703+0000 7f712cf9f640 1 --2- 172.21.15.43:0/3979604342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7128107260 0x7f71281017b0 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f711c00e9f0 tx=0x7f711c00eeb0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.703+0000 7f712cf9f640 1 -- 172.21.15.43:0/3979604342 shutdown_connections 2024-04-21T08:51:12.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.703+0000 7f712cf9f640 1 --2- 172.21.15.43:0/3979604342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7128101cf0 0x7f71281a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.703+0000 7f712cf9f640 1 --2- 172.21.15.43:0/3979604342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7128107260 0x7f71281017b0 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.703+0000 7f712cf9f640 1 --2- 172.21.15.43:0/3979604342 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f71100a3d50 0x7f71100a6210 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.703+0000 7f712cf9f640 1 --2- 172.21.15.43:0/3979604342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f712806a850 0x7f7128101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.703+0000 7f712cf9f640 1 -- 172.21.15.43:0/3979604342 >> 172.21.15.43:0/3979604342 conn(0x7f7128077c20 msgr2=0x7f71280ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:12.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.703+0000 7f712cf9f640 1 -- 172.21.15.43:0/3979604342 shutdown_connections 2024-04-21T08:51:12.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.703+0000 7f712cf9f640 1 -- 172.21.15.43:0/3979604342 wait complete. 2024-04-21T08:51:12.714 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:51:12.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:12 smithi175 ceph-mon[31920]: pgmap v384: 129 pgs: 129 active+clean; 2.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 63 KiB/s wr, 23 op/s 2024-04-21T08:51:12.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:12 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:51:12.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:12 smithi112 ceph-mon[31085]: pgmap v384: 129 pgs: 129 active+clean; 2.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 63 KiB/s wr, 23 op/s 2024-04-21T08:51:12.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:12 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:51:12.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.818+0000 7fe0d388c640 1 -- 172.21.15.43:0/937582503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0cc107250 msgr2=0x7fe0cc10ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.818+0000 7fe0d388c640 1 --2- 172.21.15.43:0/937582503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0cc107250 0x7fe0cc10ee20 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4009ca0 tx=0x7fe0c402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.818+0000 7fe0d388c640 1 -- 172.21.15.43:0/937582503 shutdown_connections 2024-04-21T08:51:12.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.818+0000 7fe0d388c640 1 --2- 172.21.15.43:0/937582503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0cc107250 0x7fe0cc10ee20 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.818+0000 7fe0d388c640 1 --2- 172.21.15.43:0/937582503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0cc102b30 0x7fe0cc106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.818+0000 7fe0d388c640 1 --2- 172.21.15.43:0/937582503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0cc102180 0x7fe0cc102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.818+0000 7fe0d388c640 1 -- 172.21.15.43:0/937582503 >> 172.21.15.43:0/937582503 conn(0x7fe0cc0fd9c0 msgr2=0x7fe0cc0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:12.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.818+0000 7fe0d388c640 1 -- 172.21.15.43:0/937582503 shutdown_connections 2024-04-21T08:51:12.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.818+0000 7fe0d388c640 1 -- 172.21.15.43:0/937582503 wait complete. 2024-04-21T08:51:12.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.819+0000 7fe0d388c640 1 Processor -- start 2024-04-21T08:51:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.819+0000 7fe0d388c640 1 -- start start 2024-04-21T08:51:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.819+0000 7fe0d388c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0cc102180 0x7fe0cc198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.819+0000 7fe0d388c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0cc107250 0x7fe0cc198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.819+0000 7fe0d388c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0cc19cf70 0x7fe0cc19d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.819+0000 7fe0d388c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc1121c0 con 0x7fe0cc19cf70 2024-04-21T08:51:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.819+0000 7fe0d388c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc112040 con 0x7fe0cc102180 2024-04-21T08:51:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.819+0000 7fe0d388c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc112340 con 0x7fe0cc107250 2024-04-21T08:51:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.819+0000 7fe0d1e29640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0cc19cf70 0x7fe0cc19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.819+0000 7fe0d1e29640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0cc19cf70 0x7fe0cc19d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51328/0 (socket says 172.21.15.43:51328) 2024-04-21T08:51:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.819+0000 7fe0d1e29640 1 -- 172.21.15.43:0/2044580886 learned_addr learned my addr 172.21.15.43:0/2044580886 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.819+0000 7fe0d1628640 1 --2- 172.21.15.43:0/2044580886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0cc102180 0x7fe0cc198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.819+0000 7fe0d1e29640 1 -- 172.21.15.43:0/2044580886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0cc107250 msgr2=0x7fe0cc198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.819+0000 7fe0d0e27640 1 --2- 172.21.15.43:0/2044580886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0cc107250 0x7fe0cc198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.819+0000 7fe0d1e29640 1 --2- 172.21.15.43:0/2044580886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0cc107250 0x7fe0cc198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.819+0000 7fe0d1e29640 1 -- 172.21.15.43:0/2044580886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0cc102180 msgr2=0x7fe0cc198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.819+0000 7fe0d1e29640 1 --2- 172.21.15.43:0/2044580886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0cc102180 0x7fe0cc198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.820+0000 7fe0d1e29640 1 -- 172.21.15.43:0/2044580886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0cc19da60 con 0x7fe0cc19cf70 2024-04-21T08:51:12.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.820+0000 7fe0d1628640 1 --2- 172.21.15.43:0/2044580886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0cc102180 0x7fe0cc198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:12.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.820+0000 7fe0d1e29640 1 --2- 172.21.15.43:0/2044580886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0cc19cf70 0x7fe0cc19d420 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4005e30 tx=0x7fe0c4034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:12.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.820+0000 7fe0c27fc640 1 -- 172.21.15.43:0/2044580886 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4039040 con 0x7fe0cc19cf70 2024-04-21T08:51:12.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.820+0000 7fe0c27fc640 1 -- 172.21.15.43:0/2044580886 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0c4044070 con 0x7fe0cc19cf70 2024-04-21T08:51:12.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.820+0000 7fe0d388c640 1 -- 172.21.15.43:0/2044580886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0cc19dcf0 con 0x7fe0cc19cf70 2024-04-21T08:51:12.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.820+0000 7fe0c27fc640 1 -- 172.21.15.43:0/2044580886 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4003e60 con 0x7fe0cc19cf70 2024-04-21T08:51:12.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.820+0000 7fe0d388c640 1 -- 172.21.15.43:0/2044580886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0cc1a54b0 con 0x7fe0cc19cf70 2024-04-21T08:51:12.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.822+0000 7fe0c27fc640 1 -- 172.21.15.43:0/2044580886 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe0c40324c0 con 0x7fe0cc19cf70 2024-04-21T08:51:12.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.823+0000 7fe0d6483740 1 Processor -- start 2024-04-21T08:51:12.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.823+0000 7fe0d6483740 1 -- start start 2024-04-21T08:51:12.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.823+0000 7fe0d6483740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4d5cc44d0 0x55b4d5cc48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.823+0000 7fe0d6483740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5cc4e80 0x55b4d5ccd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.823+0000 7fe0d6483740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4d5ccda00 0x55b4d5ccfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.823+0000 7fe0d6483740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b4d5c291f0 con 0x55b4d5cc4e80 2024-04-21T08:51:12.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.823+0000 7fe0d6483740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b4d5bc64b0 con 0x55b4d5ccda00 2024-04-21T08:51:12.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.823+0000 7fe0d6483740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b4d5c251b0 con 0x55b4d5cc44d0 2024-04-21T08:51:12.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.823+0000 7fe0d1628640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5cc4e80 0x55b4d5ccd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.823+0000 7fe0d1628640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5cc4e80 0x55b4d5ccd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51342/0 (socket says 172.21.15.43:51342) 2024-04-21T08:51:12.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.823+0000 7fe0d1628640 1 -- 172.21.15.43:0/1970092934 learned_addr learned my addr 172.21.15.43:0/1970092934 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:12.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.823+0000 7fe0c27fc640 1 --2- 172.21.15.43:0/2044580886 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe0b80a3d20 0x7fe0b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.823+0000 7fe0d0e27640 1 --2- 172.21.15.43:0/1970092934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4d5cc44d0 0x55b4d5cc48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.823+0000 7fe0d0e27640 1 --2- 172.21.15.43:0/1970092934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4d5ccda00 0x55b4d5ccfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.823+0000 7fe0d1e29640 1 --2- 172.21.15.43:0/2044580886 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe0b80a3d20 0x7fe0b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.823+0000 7fe0d1628640 1 -- 172.21.15.43:0/1970092934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4d5cc44d0 msgr2=0x55b4d5cc48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:12.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.823+0000 7fe0d1628640 1 --2- 172.21.15.43:0/1970092934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4d5cc44d0 0x55b4d5cc48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.823+0000 7fe0d1628640 1 -- 172.21.15.43:0/1970092934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4d5ccda00 msgr2=0x55b4d5ccfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:12.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.823+0000 7fe0d1628640 1 --2- 172.21.15.43:0/1970092934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4d5ccda00 0x55b4d5ccfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.823+0000 7fe0c27fc640 1 -- 172.21.15.43:0/2044580886 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fe0c40f2ea0 con 0x7fe0cc19cf70 2024-04-21T08:51:12.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.823+0000 7fe0d1628640 1 -- 172.21.15.43:0/1970092934 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4d5bb81b0 con 0x55b4d5cc4e80 2024-04-21T08:51:12.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.824+0000 7fe0d1628640 1 --2- 172.21.15.43:0/1970092934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5cc4e80 0x55b4d5ccd430 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800bd40 tx=0x7fe0c800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707c3152912534f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:12.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.824+0000 7fe0d1e29640 1 --2- 172.21.15.43:0/2044580886 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe0b80a3d20 0x7fe0b80a61e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe0c40f0390 tx=0x7fe0c40df040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:12.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.824+0000 7fe0b77fe640 1 -- 172.21.15.43:0/1970092934 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8015070 con 0x55b4d5cc4e80 2024-04-21T08:51:12.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.824+0000 7fe0b77fe640 1 -- 172.21.15.43:0/1970092934 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0c80118f0 con 0x55b4d5cc4e80 2024-04-21T08:51:12.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.824+0000 7fe0b77fe640 1 -- 172.21.15.43:0/1970092934 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8010870 con 0x55b4d5cc4e80 2024-04-21T08:51:12.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.824+0000 7fe0d6483740 1 -- 172.21.15.43:0/1970092934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5cc4e80 msgr2=0x55b4d5ccd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.824+0000 7fe0d6483740 1 --2- 172.21.15.43:0/1970092934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5cc4e80 0x55b4d5ccd430 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800bd40 tx=0x7fe0c800f730 comp rx=0 tx=0).stop 2024-04-21T08:51:12.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.825+0000 7fe0d6483740 1 -- 172.21.15.43:0/1970092934 shutdown_connections 2024-04-21T08:51:12.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.825+0000 7fe0d6483740 1 --2- 172.21.15.43:0/1970092934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4d5ccda00 0x55b4d5ccfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.825+0000 7fe0d6483740 1 --2- 172.21.15.43:0/1970092934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5cc4e80 0x55b4d5ccd430 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.825+0000 7fe0d6483740 1 --2- 172.21.15.43:0/1970092934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4d5cc44d0 0x55b4d5cc48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.825+0000 7fe0d6483740 1 -- 172.21.15.43:0/1970092934 >> 172.21.15.43:0/1970092934 conn(0x55b4d5cbeba0 msgr2=0x55b4d5cbef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:12.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.825+0000 7fe0d6483740 1 -- 172.21.15.43:0/1970092934 shutdown_connections 2024-04-21T08:51:12.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.825+0000 7fe0d6483740 1 -- 172.21.15.43:0/1970092934 wait complete. 2024-04-21T08:51:12.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.825+0000 7fe0d6483740 1 -- start start 2024-04-21T08:51:12.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.825+0000 7fe0d6483740 1 Processor -- start 2024-04-21T08:51:12.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.825+0000 7fe0d6483740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4d5cc44d0 0x55b4d5ccd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.825+0000 7fe0d6483740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4d5cc4e80 0x55b4d5cca720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.825+0000 7fe0d6483740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5ccda00 0x55b4d5ccad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.825+0000 7fe0d6483740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b4d5cd0970 con 0x55b4d5ccda00 2024-04-21T08:51:12.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.825+0000 7fe0d6483740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b4d5be2500 con 0x55b4d5cc44d0 2024-04-21T08:51:12.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.825+0000 7fe0d6483740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b4d5cd0af0 con 0x55b4d5cc4e80 2024-04-21T08:51:12.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.826+0000 7fe0d6483740 10 client.0 _ll_register_callbacks cb 0x55b4d5bb0b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:12.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.826+0000 7fe0d0e27640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5ccda00 0x55b4d5ccad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.826+0000 7fe0d0e27640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5ccda00 0x55b4d5ccad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51346/0 (socket says 172.21.15.43:51346) 2024-04-21T08:51:12.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.826+0000 7fe0d0e27640 1 -- 172.21.15.43:0/3373168149 learned_addr learned my addr 172.21.15.43:0/3373168149 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:12.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.826+0000 7fe0d0e27640 1 --2- 172.21.15.43:0/3373168149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4d5cc44d0 0x55b4d5ccd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.826+0000 7fe0d1628640 1 --2- 172.21.15.43:0/3373168149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4d5cc4e80 0x55b4d5cca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.826+0000 7fe0d0e27640 1 -- 172.21.15.43:0/3373168149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4d5cc4e80 msgr2=0x55b4d5cca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:12.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.826+0000 7fe0d0e27640 1 --2- 172.21.15.43:0/3373168149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4d5cc4e80 0x55b4d5cca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.826+0000 7fe0d0e27640 1 -- 172.21.15.43:0/3373168149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4d5cc44d0 msgr2=0x55b4d5ccd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:12.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.826+0000 7fe0d0e27640 1 --2- 172.21.15.43:0/3373168149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4d5cc44d0 0x55b4d5ccd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.826+0000 7fe0d0e27640 1 -- 172.21.15.43:0/3373168149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4d593b3f0 con 0x55b4d5ccda00 2024-04-21T08:51:12.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.826+0000 7fe0d0e27640 1 --2- 172.21.15.43:0/3373168149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5ccda00 0x55b4d5ccad30 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc012f20 tx=0x7fe0bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee619dd6dc2a388 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:12.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.826+0000 7fe0b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:12.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.826+0000 7fe0b4ff9640 1 -- 172.21.15.43:0/3373168149 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc005020 con 0x55b4d5ccda00 2024-04-21T08:51:12.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.826+0000 7fe0d6483740 1 -- 172.21.15.43:0/3373168149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0c80118f0 con 0x55b4d5ccda00 2024-04-21T08:51:12.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.826+0000 7fe0b4ff9640 1 -- 172.21.15.43:0/3373168149 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fe0bc0051c0 con 0x55b4d5ccda00 2024-04-21T08:51:12.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.826+0000 7fe0b4ff9640 1 -- 172.21.15.43:0/3373168149 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe0bc0053c0 con 0x55b4d5ccda00 2024-04-21T08:51:12.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.826+0000 7fe0d6483740 10 client.21270 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:12.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.826+0000 7fe0d6483740 1 -- 172.21.15.43:0/3373168149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b4d5a057d0 con 0x55b4d5ccda00 2024-04-21T08:51:12.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.827+0000 7fe0b4ff9640 1 -- 172.21.15.43:0/3373168149 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0c80118f0 con 0x55b4d5ccda00 2024-04-21T08:51:12.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.827+0000 7fe0b4ff9640 1 -- 172.21.15.43:0/3373168149 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe0bc01f440 con 0x55b4d5ccda00 2024-04-21T08:51:12.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.827+0000 7fe0d6483740 10 client.21270 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:12.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.827+0000 7fe0d6483740 10 client.21270 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:51:12.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.827+0000 7fe0d6483740 20 client.21270 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:12.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.827+0000 7fe0d6483740 1 --2- 172.21.15.43:0/3373168149 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55b4d5cddd70 0x55b4d5ce0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.827+0000 7fe0d6483740 4 client.21270 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:12.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.827+0000 7fe0d6483740 1 -- 172.21.15.43:0/3373168149 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4d5bc8930 con 0x55b4d5cddd70 2024-04-21T08:51:12.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.827+0000 7fe0d1628640 1 --2- 172.21.15.43:0/3373168149 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55b4d5cddd70 0x55b4d5ce0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.827+0000 7fe0b4ff9640 10 client.21270 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:51:12.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.827+0000 7fe0d1628640 1 --2- 172.21.15.43:0/3373168149 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55b4d5cddd70 0x55b4d5ce0160 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=832c7b7d2e0efb3a server_cookie=b1e0f3f007dd1706 in_seq=0 out_seq=0 2024-04-21T08:51:12.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.828+0000 7fe0b4ff9640 1 -- 172.21.15.43:0/3373168149 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b4d5bc8930 con 0x55b4d5cddd70 2024-04-21T08:51:12.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.828+0000 7fe0b4ff9640 10 client.21270 handle_command_reply: tid=0 2024-04-21T08:51:12.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.828+0000 7fe0d6483740 1 client.21270 shutdown 2024-04-21T08:51:12.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.828+0000 7fe0d6483740 1 -- 172.21.15.43:0/3373168149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5ccda00 msgr2=0x55b4d5ccad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.828+0000 7fe0d6483740 1 --2- 172.21.15.43:0/3373168149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5ccda00 0x55b4d5ccad30 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc012f20 tx=0x7fe0bc0046c0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.828+0000 7fe0d6483740 1 -- 172.21.15.43:0/3373168149 shutdown_connections 2024-04-21T08:51:12.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.828+0000 7fe0d6483740 1 --2- 172.21.15.43:0/3373168149 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55b4d5cddd70 0x55b4d5ce0160 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.828+0000 7fe0d6483740 1 --2- 172.21.15.43:0/3373168149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5ccda00 0x55b4d5ccad30 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.828+0000 7fe0d6483740 1 --2- 172.21.15.43:0/3373168149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4d5cc4e80 0x55b4d5cca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.828+0000 7fe0b4ff9640 0 client.21270 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:51:12.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.828+0000 7fe0d6483740 1 --2- 172.21.15.43:0/3373168149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4d5cc44d0 0x55b4d5ccd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.828+0000 7fe0d6483740 1 -- 172.21.15.43:0/3373168149 >> 172.21.15.43:0/3373168149 conn(0x55b4d5cbeba0 msgr2=0x55b4d5cc08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:12.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.828+0000 7fe0d6483740 1 -- 172.21.15.43:0/3373168149 shutdown_connections 2024-04-21T08:51:12.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.828+0000 7fe0d6483740 1 -- 172.21.15.43:0/3373168149 wait complete. 2024-04-21T08:51:12.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.829+0000 7fe0d6483740 20 client.21270 trim_cache size 0 max 16384 2024-04-21T08:51:12.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.842+0000 7fe0d6483740 1 Processor -- start 2024-04-21T08:51:12.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.842+0000 7fe0d6483740 1 -- start start 2024-04-21T08:51:12.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.842+0000 7fe0d6483740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4d5cc44d0 0x55b4d5cd9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.842+0000 7fe0d6483740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4d5cc4e80 0x55b4d5cc9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.842+0000 7fe0d6483740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5ccda00 0x55b4d5ccce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.842+0000 7fe0d6483740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc1af460 con 0x55b4d5ccda00 2024-04-21T08:51:12.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.842+0000 7fe0d6483740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc1b0fa0 con 0x55b4d5cc44d0 2024-04-21T08:51:12.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.842+0000 7fe0d6483740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc1afb30 con 0x55b4d5cc4e80 2024-04-21T08:51:12.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.842+0000 7fe0d0e27640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5ccda00 0x55b4d5ccce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.842+0000 7fe0d0e27640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5ccda00 0x55b4d5ccce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51354/0 (socket says 172.21.15.43:51354) 2024-04-21T08:51:12.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.842+0000 7fe0d0e27640 1 -- 172.21.15.43:0/3766539417 learned_addr learned my addr 172.21.15.43:0/3766539417 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:12.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.842+0000 7fe0d0e27640 1 --2- 172.21.15.43:0/3766539417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4d5cc44d0 0x55b4d5cd9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.842+0000 7fe0d1628640 1 --2- 172.21.15.43:0/3766539417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4d5cc4e80 0x55b4d5cc9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.842+0000 7fe0d0e27640 1 -- 172.21.15.43:0/3766539417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4d5cc4e80 msgr2=0x55b4d5cc9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:12.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.842+0000 7fe0d0e27640 1 --2- 172.21.15.43:0/3766539417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4d5cc4e80 0x55b4d5cc9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.842+0000 7fe0d0e27640 1 -- 172.21.15.43:0/3766539417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4d5cc44d0 msgr2=0x55b4d5cd9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:12.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.842+0000 7fe0d0e27640 1 --2- 172.21.15.43:0/3766539417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4d5cc44d0 0x55b4d5cd9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.842+0000 7fe0d0e27640 1 -- 172.21.15.43:0/3766539417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4d5cc4c20 con 0x55b4d5ccda00 2024-04-21T08:51:12.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.843+0000 7fe0d0e27640 1 --2- 172.21.15.43:0/3766539417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5ccda00 0x55b4d5ccce50 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc00f480 tx=0x7fe0bc012f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b2f5341737db63 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:12.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.843+0000 7fe0b6ffd640 1 -- 172.21.15.43:0/3766539417 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc035070 con 0x55b4d5ccda00 2024-04-21T08:51:12.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.843+0000 7fe0b6ffd640 1 -- 172.21.15.43:0/3766539417 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0bc01fcd0 con 0x55b4d5ccda00 2024-04-21T08:51:12.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.843+0000 7fe0b6ffd640 1 -- 172.21.15.43:0/3766539417 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc030b60 con 0x55b4d5ccda00 2024-04-21T08:51:12.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.843+0000 7fe0d6483740 1 -- 172.21.15.43:0/3766539417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5ccda00 msgr2=0x55b4d5ccce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.843+0000 7fe0d6483740 1 --2- 172.21.15.43:0/3766539417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5ccda00 0x55b4d5ccce50 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc00f480 tx=0x7fe0bc012f50 comp rx=0 tx=0).stop 2024-04-21T08:51:12.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.843+0000 7fe0d6483740 1 -- 172.21.15.43:0/3766539417 shutdown_connections 2024-04-21T08:51:12.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.843+0000 7fe0d6483740 1 --2- 172.21.15.43:0/3766539417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5ccda00 0x55b4d5ccce50 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.843+0000 7fe0d6483740 1 --2- 172.21.15.43:0/3766539417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4d5cc4e80 0x55b4d5cc9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.843+0000 7fe0d6483740 1 --2- 172.21.15.43:0/3766539417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4d5cc44d0 0x55b4d5cd9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.843+0000 7fe0d6483740 1 -- 172.21.15.43:0/3766539417 >> 172.21.15.43:0/3766539417 conn(0x55b4d5cbeba0 msgr2=0x55b4d5cbdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:12.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.843+0000 7fe0d6483740 1 -- 172.21.15.43:0/3766539417 shutdown_connections 2024-04-21T08:51:12.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.843+0000 7fe0d6483740 1 -- 172.21.15.43:0/3766539417 wait complete. 2024-04-21T08:51:12.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.844+0000 7fe0d6483740 1 -- start start 2024-04-21T08:51:12.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.844+0000 7fe0d6483740 1 Processor -- start 2024-04-21T08:51:12.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.844+0000 7fe0d6483740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4d5cc44d0 0x55b4d5cd5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.844+0000 7fe0d6483740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4d5cc4e80 0x55b4d5cc6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.844+0000 7fe0d6483740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5ccda00 0x55b4d5cc74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.844+0000 7fe0d6483740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc1b08d0 con 0x55b4d5ccda00 2024-04-21T08:51:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.844+0000 7fe0d6483740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc1b0200 con 0x55b4d5cc4e80 2024-04-21T08:51:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.844+0000 7fe0d6483740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc1ad460 con 0x55b4d5cc44d0 2024-04-21T08:51:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.844+0000 7fe0d6483740 10 client.0 _ll_register_callbacks cb 0x7fe0bc012b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.844+0000 7fe0d0e27640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5ccda00 0x55b4d5cc74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.844+0000 7fe0d1628640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4d5cc4e80 0x55b4d5cc6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.844+0000 7fe0d0e27640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5ccda00 0x55b4d5cc74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51370/0 (socket says 172.21.15.43:51370) 2024-04-21T08:51:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.844+0000 7fe0d0e27640 1 -- 172.21.15.43:0/1546523929 learned_addr learned my addr 172.21.15.43:0/1546523929 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.845+0000 7fe0d1628640 1 -- 172.21.15.43:0/1546523929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4d5cc44d0 msgr2=0x55b4d5cd5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.845+0000 7fe0d1628640 1 --2- 172.21.15.43:0/1546523929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4d5cc44d0 0x55b4d5cd5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.845+0000 7fe0d1628640 1 -- 172.21.15.43:0/1546523929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5ccda00 msgr2=0x55b4d5cc74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.845+0000 7fe0d1628640 1 --2- 172.21.15.43:0/1546523929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5ccda00 0x55b4d5cc74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.845+0000 7fe0d1628640 1 -- 172.21.15.43:0/1546523929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4d5cc7a30 con 0x55b4d5cc4e80 2024-04-21T08:51:12.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.845+0000 7fe0d0e27640 1 --2- 172.21.15.43:0/1546523929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5ccda00 0x55b4d5cc74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:12.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.845+0000 7fe0d1628640 1 --2- 172.21.15.43:0/1546523929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4d5cc4e80 0x55b4d5cc6fb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55b4d5cbfc10 tx=0x7fe0c800b3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcb14ada0490a3e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:12.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.845+0000 7fe0b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-21T08:51:12.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.846+0000 7fe0b4ff9640 1 -- 172.21.15.43:0/1546523929 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c802e060 con 0x55b4d5cc4e80 2024-04-21T08:51:12.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.846+0000 7fe0d6483740 1 -- 172.21.15.43:0/1546523929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0bc01fcd0 con 0x55b4d5cc4e80 2024-04-21T08:51:12.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.846+0000 7fe0b4ff9640 1 -- 172.21.15.43:0/1546523929 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fe0c8005510 con 0x55b4d5cc4e80 2024-04-21T08:51:12.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.846+0000 7fe0b4ff9640 1 -- 172.21.15.43:0/1546523929 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe0c8035050 con 0x55b4d5cc4e80 2024-04-21T08:51:12.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.846+0000 7fe0d6483740 10 client.29693 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:12.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.846+0000 7fe0d6483740 1 -- 172.21.15.43:0/1546523929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b4d5d43fc0 con 0x55b4d5cc4e80 2024-04-21T08:51:12.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.846+0000 7fe0b4ff9640 1 -- 172.21.15.43:0/1546523929 <== mon.2 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0bc01fcd0 con 0x55b4d5cc4e80 2024-04-21T08:51:12.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.846+0000 7fe0b4ff9640 1 -- 172.21.15.43:0/1546523929 <== mon.2 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe0c80048f0 con 0x55b4d5cc4e80 2024-04-21T08:51:12.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.846+0000 7fe0d6483740 10 client.29693 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:12.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.846+0000 7fe0d6483740 10 client.29693 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:51:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.846+0000 7fe0d6483740 20 client.29693 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.846+0000 7fe0d6483740 1 --2- 172.21.15.43:0/1546523929 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55b4d5d44490 0x55b4d5d468b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.846+0000 7fe0d6483740 4 client.29693 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:51:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.846+0000 7fe0d6483740 1 -- 172.21.15.43:0/1546523929 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b4d5c36c10 con 0x55b4d5d44490 2024-04-21T08:51:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.847+0000 7fe0d0e27640 1 --2- 172.21.15.43:0/1546523929 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55b4d5d44490 0x55b4d5d468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.847+0000 7fe0d0e27640 1 --2- 172.21.15.43:0/1546523929 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55b4d5d44490 0x55b4d5d468b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=57ea0397a97d32b7 server_cookie=d56ab095c93738c3 in_seq=0 out_seq=0 2024-04-21T08:51:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.847+0000 7fe0b4ff9640 10 client.29693 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:51:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.847+0000 7fe0b4ff9640 1 -- 172.21.15.43:0/1546523929 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b4d5c36c10 con 0x55b4d5d44490 2024-04-21T08:51:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.847+0000 7fe0b4ff9640 10 client.29693 handle_command_reply: tid=0 2024-04-21T08:51:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.847+0000 7fe0d6483740 1 client.29693 shutdown 2024-04-21T08:51:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.847+0000 7fe0d6483740 1 -- 172.21.15.43:0/1546523929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4d5cc4e80 msgr2=0x55b4d5cc6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.847+0000 7fe0d6483740 1 --2- 172.21.15.43:0/1546523929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4d5cc4e80 0x55b4d5cc6fb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55b4d5cbfc10 tx=0x7fe0c800b3a0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.847+0000 7fe0d6483740 1 -- 172.21.15.43:0/1546523929 shutdown_connections 2024-04-21T08:51:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.847+0000 7fe0d6483740 1 --2- 172.21.15.43:0/1546523929 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55b4d5d44490 0x55b4d5d468b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.847+0000 7fe0d6483740 1 --2- 172.21.15.43:0/1546523929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b4d5ccda00 0x55b4d5cc74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.847+0000 7fe0b4ff9640 0 client.29693 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:51:12.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.847+0000 7fe0d6483740 1 --2- 172.21.15.43:0/1546523929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4d5cc4e80 0x55b4d5cc6fb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.847+0000 7fe0d6483740 1 --2- 172.21.15.43:0/1546523929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4d5cc44d0 0x55b4d5cd5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.847+0000 7fe0d6483740 1 -- 172.21.15.43:0/1546523929 >> 172.21.15.43:0/1546523929 conn(0x55b4d5cbeba0 msgr2=0x55b4d5cc52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:12.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.848+0000 7fe0d6483740 1 -- 172.21.15.43:0/1546523929 shutdown_connections 2024-04-21T08:51:12.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.848+0000 7fe0d6483740 1 -- 172.21.15.43:0/1546523929 wait complete. 2024-04-21T08:51:12.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.848+0000 7fe0d6483740 20 client.29693 trim_cache size 0 max 16384 2024-04-21T08:51:12.853 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:51:12.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.848+0000 7fe0d388c640 1 -- 172.21.15.43:0/2044580886 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe0b80a3d20 msgr2=0x7fe0b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.848+0000 7fe0d388c640 1 --2- 172.21.15.43:0/2044580886 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe0b80a3d20 0x7fe0b80a61e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe0c40f0390 tx=0x7fe0c40df040 comp rx=0 tx=0).stop 2024-04-21T08:51:12.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.848+0000 7fe0d388c640 1 -- 172.21.15.43:0/2044580886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0cc19cf70 msgr2=0x7fe0cc19d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.848+0000 7fe0d388c640 1 --2- 172.21.15.43:0/2044580886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0cc19cf70 0x7fe0cc19d420 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4005e30 tx=0x7fe0c4034760 comp rx=0 tx=0).stop 2024-04-21T08:51:12.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.848+0000 7fe0d388c640 1 -- 172.21.15.43:0/2044580886 shutdown_connections 2024-04-21T08:51:12.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.848+0000 7fe0d388c640 1 --2- 172.21.15.43:0/2044580886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0cc19cf70 0x7fe0cc19d420 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.848+0000 7fe0d388c640 1 --2- 172.21.15.43:0/2044580886 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe0b80a3d20 0x7fe0b80a61e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.848+0000 7fe0d388c640 1 --2- 172.21.15.43:0/2044580886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0cc107250 0x7fe0cc198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.848+0000 7fe0d388c640 1 --2- 172.21.15.43:0/2044580886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0cc102180 0x7fe0cc198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.848+0000 7fe0d388c640 1 -- 172.21.15.43:0/2044580886 >> 172.21.15.43:0/2044580886 conn(0x7fe0cc0fd9c0 msgr2=0x7fe0cc0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:12.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.848+0000 7fe0d388c640 1 -- 172.21.15.43:0/2044580886 shutdown_connections 2024-04-21T08:51:12.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.849+0000 7fe0d388c640 1 -- 172.21.15.43:0/2044580886 wait complete. 2024-04-21T08:51:12.859 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:51:12.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.963+0000 7f4806481640 1 -- 172.21.15.43:0/4250288811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4800107270 msgr2=0x7f480010ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.963+0000 7f4806481640 1 --2- 172.21.15.43:0/4250288811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4800107270 0x7f480010ee40 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009ca0 tx=0x7f47f802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.963+0000 7f4806481640 1 -- 172.21.15.43:0/4250288811 shutdown_connections 2024-04-21T08:51:12.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.963+0000 7f4806481640 1 --2- 172.21.15.43:0/4250288811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4800107270 0x7f480010ee40 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.963+0000 7f4806481640 1 --2- 172.21.15.43:0/4250288811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f480006b1f0 0x7f4800106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.963+0000 7f4806481640 1 --2- 172.21.15.43:0/4250288811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f480006a840 0x7f480006ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.963+0000 7f4806481640 1 -- 172.21.15.43:0/4250288811 >> 172.21.15.43:0/4250288811 conn(0x7f4800077eb0 msgr2=0x7f48000782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:12.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.963+0000 7f4806481640 1 -- 172.21.15.43:0/4250288811 shutdown_connections 2024-04-21T08:51:12.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.963+0000 7f4806481640 1 -- 172.21.15.43:0/4250288811 wait complete. 2024-04-21T08:51:12.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.964+0000 7f4806481640 1 Processor -- start 2024-04-21T08:51:12.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.964+0000 7f4806481640 1 -- start start 2024-04-21T08:51:12.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.964+0000 7f4806481640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f480006a840 0x7f48001986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.964+0000 7f4806481640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4800107270 0x7f4800198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.964+0000 7f4806481640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f480019cf70 0x7f480019d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.964+0000 7f4806481640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f48001121d0 con 0x7f480019cf70 2024-04-21T08:51:12.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.964+0000 7f4806481640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4800112050 con 0x7f480006a840 2024-04-21T08:51:12.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.964+0000 7f4806481640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4800112350 con 0x7f4800107270 2024-04-21T08:51:12.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.964+0000 7f4804a1e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f480019cf70 0x7f480019d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.964+0000 7f4804a1e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f480019cf70 0x7f480019d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51392/0 (socket says 172.21.15.43:51392) 2024-04-21T08:51:12.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.964+0000 7f4804a1e640 1 -- 172.21.15.43:0/2937574096 learned_addr learned my addr 172.21.15.43:0/2937574096 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:12.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.964+0000 7f47ff7fe640 1 --2- 172.21.15.43:0/2937574096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4800107270 0x7f4800198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.965+0000 7f47fffff640 1 --2- 172.21.15.43:0/2937574096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f480006a840 0x7f48001986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.965+0000 7f4804a1e640 1 -- 172.21.15.43:0/2937574096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4800107270 msgr2=0x7f4800198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.965+0000 7f4804a1e640 1 --2- 172.21.15.43:0/2937574096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4800107270 0x7f4800198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.965+0000 7f4804a1e640 1 -- 172.21.15.43:0/2937574096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f480006a840 msgr2=0x7f48001986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.965+0000 7f4804a1e640 1 --2- 172.21.15.43:0/2937574096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f480006a840 0x7f48001986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.965+0000 7f4804a1e640 1 -- 172.21.15.43:0/2937574096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480019da60 con 0x7f480019cf70 2024-04-21T08:51:12.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.965+0000 7f4804a1e640 1 --2- 172.21.15.43:0/2937574096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f480019cf70 0x7f480019d420 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f47f8005e30 tx=0x7f47f80346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:12.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.965+0000 7f47fd7fa640 1 -- 172.21.15.43:0/2937574096 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8041020 con 0x7f480019cf70 2024-04-21T08:51:12.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.965+0000 7f4806481640 1 -- 172.21.15.43:0/2937574096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480019dcf0 con 0x7f480019cf70 2024-04-21T08:51:12.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.965+0000 7f47fd7fa640 1 -- 172.21.15.43:0/2937574096 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47f8032970 con 0x7f480019cf70 2024-04-21T08:51:12.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.965+0000 7f47fd7fa640 1 -- 172.21.15.43:0/2937574096 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8039040 con 0x7f480019cf70 2024-04-21T08:51:12.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.965+0000 7f4806481640 1 -- 172.21.15.43:0/2937574096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48001a54c0 con 0x7f480019cf70 2024-04-21T08:51:12.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.967+0000 7f47fd7fa640 1 -- 172.21.15.43:0/2937574096 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f47f8044020 con 0x7f480019cf70 2024-04-21T08:51:12.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.968+0000 7f4809078740 1 Processor -- start 2024-04-21T08:51:12.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.968+0000 7f4809078740 1 -- start start 2024-04-21T08:51:12.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.968+0000 7f4809078740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f136304d0 0x562f136308b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.968+0000 7f4809078740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562f13630e80 0x562f13639430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.968+0000 7f4809078740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13639a00 0x562f1363be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.968+0000 7f4809078740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562f135951f0 con 0x562f13639a00 2024-04-21T08:51:12.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.968+0000 7f4809078740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562f135324b0 con 0x562f136304d0 2024-04-21T08:51:12.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.968+0000 7f4809078740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562f135911b0 con 0x562f13630e80 2024-04-21T08:51:12.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.968+0000 7f47ff7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13639a00 0x562f1363be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.968+0000 7f47ff7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f136304d0 0x562f136308b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.968+0000 7f47ff7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f136304d0 0x562f136308b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.43:60222/0 (socket says 172.21.15.43:60222) 2024-04-21T08:51:12.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.968+0000 7f47ff7fe640 1 -- 172.21.15.43:0/1355483192 learned_addr learned my addr 172.21.15.43:0/1355483192 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:12.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.968+0000 7f47fd7fa640 1 --2- 172.21.15.43:0/2937574096 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f47e80a3d20 0x7f47e80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.968+0000 7f47fffff640 1 --2- 172.21.15.43:0/1355483192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562f13630e80 0x562f13639430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.968+0000 7f47ff7fe640 1 -- 172.21.15.43:0/1355483192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562f13630e80 msgr2=0x562f13639430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:12.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.968+0000 7f47ff7fe640 1 --2- 172.21.15.43:0/1355483192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562f13630e80 0x562f13639430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.968+0000 7f47ff7fe640 1 -- 172.21.15.43:0/1355483192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13639a00 msgr2=0x562f1363be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:12.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.968+0000 7f47ff7fe640 1 --2- 172.21.15.43:0/1355483192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13639a00 0x562f1363be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.968+0000 7f47ff7fe640 1 -- 172.21.15.43:0/1355483192 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f135241b0 con 0x562f136304d0 2024-04-21T08:51:12.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.968+0000 7f4804a1e640 1 --2- 172.21.15.43:0/2937574096 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f47e80a3d20 0x7f47e80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.969+0000 7f47fd7fa640 1 -- 172.21.15.43:0/2937574096 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f47f80f27a0 con 0x7f480019cf70 2024-04-21T08:51:12.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.969+0000 7f47ff7fe640 1 --2- 172.21.15.43:0/1355483192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f136304d0 0x562f136308b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f47f0010950 tx=0x7f47f0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a0d8ec597fc0a9e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:12.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.969+0000 7f47e67fc640 1 -- 172.21.15.43:0/1355483192 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f0016070 con 0x562f136304d0 2024-04-21T08:51:12.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.969+0000 7f47e67fc640 1 -- 172.21.15.43:0/1355483192 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47f000fc70 con 0x562f136304d0 2024-04-21T08:51:12.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.969+0000 7f47e67fc640 1 -- 172.21.15.43:0/1355483192 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f0019040 con 0x562f136304d0 2024-04-21T08:51:12.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.969+0000 7f4804a1e640 1 --2- 172.21.15.43:0/2937574096 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f47e80a3d20 0x7f47e80a61e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f47f80efc90 tx=0x7f47f808fe70 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:12.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.969+0000 7f4809078740 1 -- 172.21.15.43:0/1355483192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f136304d0 msgr2=0x562f136308b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.969+0000 7f4809078740 1 --2- 172.21.15.43:0/1355483192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f136304d0 0x562f136308b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f47f0010950 tx=0x7f47f0005f90 comp rx=0 tx=0).stop 2024-04-21T08:51:12.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.969+0000 7f4809078740 1 -- 172.21.15.43:0/1355483192 shutdown_connections 2024-04-21T08:51:12.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.969+0000 7f4809078740 1 --2- 172.21.15.43:0/1355483192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13639a00 0x562f1363be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.969+0000 7f4809078740 1 --2- 172.21.15.43:0/1355483192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562f13630e80 0x562f13639430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.969+0000 7f4809078740 1 --2- 172.21.15.43:0/1355483192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f136304d0 0x562f136308b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.970+0000 7f4809078740 1 -- 172.21.15.43:0/1355483192 >> 172.21.15.43:0/1355483192 conn(0x562f1362aba0 msgr2=0x562f1362af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:12.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.970+0000 7f4809078740 1 -- 172.21.15.43:0/1355483192 shutdown_connections 2024-04-21T08:51:12.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.970+0000 7f4809078740 1 -- 172.21.15.43:0/1355483192 wait complete. 2024-04-21T08:51:12.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.970+0000 7f4809078740 1 -- start start 2024-04-21T08:51:12.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.970+0000 7f4809078740 1 Processor -- start 2024-04-21T08:51:12.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.970+0000 7f4809078740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f136304d0 0x562f136392d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.970+0000 7f4809078740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13630e80 0x562f13636660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.970+0000 7f4809078740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562f13639a00 0x562f13636bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.970+0000 7f4809078740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562f1363c970 con 0x562f13630e80 2024-04-21T08:51:12.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.970+0000 7f4809078740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562f1354e500 con 0x562f136304d0 2024-04-21T08:51:12.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.970+0000 7f4809078740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562f1363caf0 con 0x562f13639a00 2024-04-21T08:51:12.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.970+0000 7f4809078740 10 client.0 _ll_register_callbacks cb 0x562f1351cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:12.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.970+0000 7f47fffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13630e80 0x562f13636660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.970+0000 7f47fffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13630e80 0x562f13636660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51414/0 (socket says 172.21.15.43:51414) 2024-04-21T08:51:12.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.970+0000 7f47fffff640 1 -- 172.21.15.43:0/1784769999 learned_addr learned my addr 172.21.15.43:0/1784769999 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:12.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.971+0000 7f47ff7fe640 1 --2- 172.21.15.43:0/1784769999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f136304d0 0x562f136392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.971+0000 7f47ff7fe640 1 --2- 172.21.15.43:0/1784769999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562f13639a00 0x562f13636bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.971+0000 7f47fffff640 1 -- 172.21.15.43:0/1784769999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562f13639a00 msgr2=0x562f13636bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:12.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.971+0000 7f47fffff640 1 --2- 172.21.15.43:0/1784769999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562f13639a00 0x562f13636bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.971+0000 7f47fffff640 1 -- 172.21.15.43:0/1784769999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f136304d0 msgr2=0x562f136392d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:12.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.971+0000 7f47fffff640 1 --2- 172.21.15.43:0/1784769999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f136304d0 0x562f136392d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.971+0000 7f47fffff640 1 -- 172.21.15.43:0/1784769999 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f132a73f0 con 0x562f13630e80 2024-04-21T08:51:12.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.971+0000 7f47ff7fe640 1 --2- 172.21.15.43:0/1784769999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f136304d0 0x562f136392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:12.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.971+0000 7f47fffff640 1 --2- 172.21.15.43:0/1784769999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13630e80 0x562f13636660 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f47f4013710 tx=0x7f47f4013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403db965a7b7d055 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:12.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.971+0000 7f47df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:12.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.971+0000 7f47df7fe640 1 -- 172.21.15.43:0/1784769999 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f4004460 con 0x562f13630e80 2024-04-21T08:51:12.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.971+0000 7f47df7fe640 1 -- 172.21.15.43:0/1784769999 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f47f40119c0 con 0x562f13630e80 2024-04-21T08:51:12.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.971+0000 7f4809078740 1 -- 172.21.15.43:0/1784769999 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47f000fc70 con 0x562f13630e80 2024-04-21T08:51:12.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.971+0000 7f47df7fe640 1 -- 172.21.15.43:0/1784769999 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f47f4011bc0 con 0x562f13630e80 2024-04-21T08:51:12.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.971+0000 7f4809078740 10 client.21288 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:12.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.971+0000 7f4809078740 1 -- 172.21.15.43:0/1784769999 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562f133717d0 con 0x562f13630e80 2024-04-21T08:51:12.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.971+0000 7f47df7fe640 1 -- 172.21.15.43:0/1784769999 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47f000fc70 con 0x562f13630e80 2024-04-21T08:51:12.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.972+0000 7f47df7fe640 1 -- 172.21.15.43:0/1784769999 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f47f401a070 con 0x562f13630e80 2024-04-21T08:51:12.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.972+0000 7f4809078740 10 client.21288 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:12.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.972+0000 7f4809078740 10 client.21288 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:51:12.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.972+0000 7f4809078740 20 client.21288 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:12.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.972+0000 7f4809078740 1 --2- 172.21.15.43:0/1784769999 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x562f13649d00 0x562f1364c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.972+0000 7f4809078740 4 client.21288 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:12.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.972+0000 7f4809078740 1 -- 172.21.15.43:0/1784769999 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f13534930 con 0x562f13649d00 2024-04-21T08:51:12.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.972+0000 7f47ff7fe640 1 --2- 172.21.15.43:0/1784769999 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x562f13649d00 0x562f1364c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.972+0000 7f47df7fe640 10 client.21288 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:51:12.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.972+0000 7f47ff7fe640 1 --2- 172.21.15.43:0/1784769999 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x562f13649d00 0x562f1364c120 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6286c133efd2e0b1 server_cookie=e2d9e86394ebd217 in_seq=0 out_seq=0 2024-04-21T08:51:12.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.973+0000 7f47df7fe640 1 -- 172.21.15.43:0/1784769999 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562f13534930 con 0x562f13649d00 2024-04-21T08:51:12.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.973+0000 7f47df7fe640 10 client.21288 handle_command_reply: tid=0 2024-04-21T08:51:12.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.973+0000 7f4809078740 1 client.21288 shutdown 2024-04-21T08:51:12.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.973+0000 7f4809078740 1 -- 172.21.15.43:0/1784769999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13630e80 msgr2=0x562f13636660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.973+0000 7f4809078740 1 --2- 172.21.15.43:0/1784769999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13630e80 0x562f13636660 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f47f4013710 tx=0x7f47f4013bd0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.974+0000 7f4809078740 1 -- 172.21.15.43:0/1784769999 shutdown_connections 2024-04-21T08:51:12.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.974+0000 7f4809078740 1 --2- 172.21.15.43:0/1784769999 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x562f13649d00 0x562f1364c120 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.974+0000 7f4809078740 1 --2- 172.21.15.43:0/1784769999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562f13639a00 0x562f13636bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.974+0000 7f4809078740 1 --2- 172.21.15.43:0/1784769999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13630e80 0x562f13636660 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.974+0000 7f47df7fe640 0 client.21288 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:51:12.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.974+0000 7f4809078740 1 --2- 172.21.15.43:0/1784769999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f136304d0 0x562f136392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.974+0000 7f4809078740 1 -- 172.21.15.43:0/1784769999 >> 172.21.15.43:0/1784769999 conn(0x562f1362aba0 msgr2=0x562f1362c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:12.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.974+0000 7f4809078740 1 -- 172.21.15.43:0/1784769999 shutdown_connections 2024-04-21T08:51:12.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.974+0000 7f4809078740 1 -- 172.21.15.43:0/1784769999 wait complete. 2024-04-21T08:51:12.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.974+0000 7f4809078740 20 client.21288 trim_cache size 0 max 16384 2024-04-21T08:51:12.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.987+0000 7f4809078740 1 Processor -- start 2024-04-21T08:51:12.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.987+0000 7f4809078740 1 -- start start 2024-04-21T08:51:12.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.987+0000 7f4809078740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f136304d0 0x562f13645860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.987+0000 7f4809078740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13630e80 0x562f13631ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.987+0000 7f4809078740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562f13639a00 0x562f13638d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.987+0000 7f4809078740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f48001af460 con 0x562f13630e80 2024-04-21T08:51:12.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.987+0000 7f4809078740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f48001b0fa0 con 0x562f136304d0 2024-04-21T08:51:12.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.987+0000 7f4809078740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48001afb30 con 0x562f13639a00 2024-04-21T08:51:12.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.987+0000 7f47fffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13630e80 0x562f13631ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.987+0000 7f47fffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13630e80 0x562f13631ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51422/0 (socket says 172.21.15.43:51422) 2024-04-21T08:51:12.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.987+0000 7f47fffff640 1 -- 172.21.15.43:0/3407512382 learned_addr learned my addr 172.21.15.43:0/3407512382 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:12.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.987+0000 7f47ff7fe640 1 --2- 172.21.15.43:0/3407512382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f136304d0 0x562f13645860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.987+0000 7f47fffff640 1 -- 172.21.15.43:0/3407512382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562f13639a00 msgr2=0x562f13638d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:51:12.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.987+0000 7f47fffff640 1 --2- 172.21.15.43:0/3407512382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562f13639a00 0x562f13638d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.988+0000 7f47fffff640 1 -- 172.21.15.43:0/3407512382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f136304d0 msgr2=0x562f13645860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:12.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.988+0000 7f47fffff640 1 --2- 172.21.15.43:0/3407512382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f136304d0 0x562f13645860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.988+0000 7f47fffff640 1 -- 172.21.15.43:0/3407512382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f13630c20 con 0x562f13630e80 2024-04-21T08:51:12.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.988+0000 7f47fffff640 1 --2- 172.21.15.43:0/3407512382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13630e80 0x562f13631ae0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f47f4001eb0 tx=0x7f47f4011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b898ed179d8a59d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:12.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.988+0000 7f47e5ffb640 1 -- 172.21.15.43:0/3407512382 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f402f020 con 0x562f13630e80 2024-04-21T08:51:12.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.988+0000 7f47e5ffb640 1 -- 172.21.15.43:0/3407512382 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47f4004710 con 0x562f13630e80 2024-04-21T08:51:12.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.988+0000 7f47e5ffb640 1 -- 172.21.15.43:0/3407512382 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f4003d50 con 0x562f13630e80 2024-04-21T08:51:12.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.988+0000 7f4809078740 1 -- 172.21.15.43:0/3407512382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13630e80 msgr2=0x562f13631ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.988+0000 7f4809078740 1 --2- 172.21.15.43:0/3407512382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13630e80 0x562f13631ae0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f47f4001eb0 tx=0x7f47f4011ac0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.988+0000 7f4809078740 1 -- 172.21.15.43:0/3407512382 shutdown_connections 2024-04-21T08:51:12.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.988+0000 7f4809078740 1 --2- 172.21.15.43:0/3407512382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562f13639a00 0x562f13638d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.988+0000 7f4809078740 1 --2- 172.21.15.43:0/3407512382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13630e80 0x562f13631ae0 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.988+0000 7f4809078740 1 --2- 172.21.15.43:0/3407512382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f136304d0 0x562f13645860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.988+0000 7f4809078740 1 -- 172.21.15.43:0/3407512382 >> 172.21.15.43:0/3407512382 conn(0x562f1362aba0 msgr2=0x562f136313d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:12.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.988+0000 7f4809078740 1 -- 172.21.15.43:0/3407512382 shutdown_connections 2024-04-21T08:51:12.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.989+0000 7f4809078740 1 -- 172.21.15.43:0/3407512382 wait complete. 2024-04-21T08:51:12.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.989+0000 7f4809078740 1 -- start start 2024-04-21T08:51:12.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.989+0000 7f4809078740 1 Processor -- start 2024-04-21T08:51:12.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.989+0000 7f4809078740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f136304d0 0x562f13633470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.989+0000 7f4809078740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13630e80 0x562f136339b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.989+0000 7f4809078740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562f13639a00 0x562f13633ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.989+0000 7f4809078740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f48001b08d0 con 0x562f13630e80 2024-04-21T08:51:12.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.989+0000 7f4809078740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f48001b0200 con 0x562f136304d0 2024-04-21T08:51:12.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.989+0000 7f4809078740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48001ad460 con 0x562f13639a00 2024-04-21T08:51:12.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.989+0000 7f4809078740 10 client.0 _ll_register_callbacks cb 0x7f47f4009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:12.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.989+0000 7f47fffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13630e80 0x562f136339b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.989+0000 7f47fffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13630e80 0x562f136339b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51432/0 (socket says 172.21.15.43:51432) 2024-04-21T08:51:12.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.989+0000 7f47fffff640 1 -- 172.21.15.43:0/2089537232 learned_addr learned my addr 172.21.15.43:0/2089537232 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:12.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.989+0000 7f47ff7fe640 1 --2- 172.21.15.43:0/2089537232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f136304d0 0x562f13633470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.990+0000 7f47ff7fe640 1 --2- 172.21.15.43:0/2089537232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562f13639a00 0x562f13633ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.990+0000 7f47fffff640 1 -- 172.21.15.43:0/2089537232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562f13639a00 msgr2=0x562f13633ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:12.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.990+0000 7f47fffff640 1 --2- 172.21.15.43:0/2089537232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562f13639a00 0x562f13633ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.990+0000 7f47fffff640 1 -- 172.21.15.43:0/2089537232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f136304d0 msgr2=0x562f13633470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:12.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.990+0000 7f47fffff640 1 --2- 172.21.15.43:0/2089537232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f136304d0 0x562f13633470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.990+0000 7f47fffff640 1 -- 172.21.15.43:0/2089537232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f136af330 con 0x562f13630e80 2024-04-21T08:51:12.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.990+0000 7f47fffff640 1 --2- 172.21.15.43:0/2089537232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13630e80 0x562f136339b0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f47f40092a0 tx=0x7f47f40118c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa72ff051dc00279 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:12.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.990+0000 7f47df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:12.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.990+0000 7f47df7fe640 1 -- 172.21.15.43:0/2089537232 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f4003c50 con 0x562f13630e80 2024-04-21T08:51:12.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.990+0000 7f47df7fe640 1 -- 172.21.15.43:0/2089537232 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f47f4003df0 con 0x562f13630e80 2024-04-21T08:51:12.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.990+0000 7f4809078740 1 -- 172.21.15.43:0/2089537232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47f4004710 con 0x562f13630e80 2024-04-21T08:51:12.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.990+0000 7f47df7fe640 1 -- 172.21.15.43:0/2089537232 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f47f4045050 con 0x562f13630e80 2024-04-21T08:51:12.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.990+0000 7f4809078740 10 client.21294 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:12.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.990+0000 7f4809078740 1 -- 172.21.15.43:0/2089537232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562f136b0630 con 0x562f13630e80 2024-04-21T08:51:12.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.990+0000 7f47df7fe640 1 -- 172.21.15.43:0/2089537232 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47f4004710 con 0x562f13630e80 2024-04-21T08:51:12.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.991+0000 7f47df7fe640 1 -- 172.21.15.43:0/2089537232 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f47f401a040 con 0x562f13630e80 2024-04-21T08:51:12.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.991+0000 7f4809078740 10 client.21294 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:12.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.991+0000 7f4809078740 10 client.21294 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:51:12.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.991+0000 7f4809078740 20 client.21294 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:12.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.991+0000 7f4809078740 1 --2- 172.21.15.43:0/2089537232 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x562f136b0cf0 0x562f136b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:12.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.991+0000 7f4809078740 4 client.21294 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:51:12.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.991+0000 7f4809078740 1 -- 172.21.15.43:0/2089537232 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562f135a2c10 con 0x562f136b0cf0 2024-04-21T08:51:12.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.991+0000 7f47ff7fe640 1 --2- 172.21.15.43:0/2089537232 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x562f136b0cf0 0x562f136b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:12.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.991+0000 7f47ff7fe640 1 --2- 172.21.15.43:0/2089537232 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x562f136b0cf0 0x562f136b3110 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1550af79f31e7ec8 server_cookie=a154812be0894bbf in_seq=0 out_seq=0 2024-04-21T08:51:12.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.991+0000 7f47df7fe640 10 client.21294 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:51:12.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.991+0000 7f47df7fe640 1 -- 172.21.15.43:0/2089537232 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562f135a2c10 con 0x562f136b0cf0 2024-04-21T08:51:12.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.991+0000 7f47df7fe640 10 client.21294 handle_command_reply: tid=0 2024-04-21T08:51:12.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.991+0000 7f4809078740 1 client.21294 shutdown 2024-04-21T08:51:12.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.992+0000 7f4809078740 1 -- 172.21.15.43:0/2089537232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13630e80 msgr2=0x562f136339b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:12.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.992+0000 7f4809078740 1 --2- 172.21.15.43:0/2089537232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13630e80 0x562f136339b0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f47f40092a0 tx=0x7f47f40118c0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.992+0000 7f4809078740 1 -- 172.21.15.43:0/2089537232 shutdown_connections 2024-04-21T08:51:12.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.992+0000 7f4809078740 1 --2- 172.21.15.43:0/2089537232 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x562f136b0cf0 0x562f136b3110 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.992+0000 7f4809078740 1 --2- 172.21.15.43:0/2089537232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562f13639a00 0x562f13633ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.992+0000 7f4809078740 1 --2- 172.21.15.43:0/2089537232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f13630e80 0x562f136339b0 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.992+0000 7f47df7fe640 0 client.21294 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:51:12.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.992+0000 7f4809078740 1 --2- 172.21.15.43:0/2089537232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f136304d0 0x562f13633470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:12.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.992+0000 7f4809078740 1 -- 172.21.15.43:0/2089537232 >> 172.21.15.43:0/2089537232 conn(0x562f1362aba0 msgr2=0x562f1362c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:12.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.992+0000 7f4809078740 1 -- 172.21.15.43:0/2089537232 shutdown_connections 2024-04-21T08:51:12.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.992+0000 7f4809078740 1 -- 172.21.15.43:0/2089537232 wait complete. 2024-04-21T08:51:12.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.992+0000 7f4809078740 20 client.21294 trim_cache size 0 max 16384 2024-04-21T08:51:12.999 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:51:13.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.992+0000 7f4806481640 1 -- 172.21.15.43:0/2937574096 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f47e80a3d20 msgr2=0x7f47e80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.992+0000 7f4806481640 1 --2- 172.21.15.43:0/2937574096 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f47e80a3d20 0x7f47e80a61e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f47f80efc90 tx=0x7f47f808fe70 comp rx=0 tx=0).stop 2024-04-21T08:51:13.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.992+0000 7f4806481640 1 -- 172.21.15.43:0/2937574096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f480019cf70 msgr2=0x7f480019d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.992+0000 7f4806481640 1 --2- 172.21.15.43:0/2937574096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f480019cf70 0x7f480019d420 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f47f8005e30 tx=0x7f47f80346d0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.992+0000 7f4806481640 1 -- 172.21.15.43:0/2937574096 shutdown_connections 2024-04-21T08:51:13.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.992+0000 7f4806481640 1 --2- 172.21.15.43:0/2937574096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f480019cf70 0x7f480019d420 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.992+0000 7f4806481640 1 --2- 172.21.15.43:0/2937574096 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f47e80a3d20 0x7f47e80a61e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.992+0000 7f4806481640 1 --2- 172.21.15.43:0/2937574096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4800107270 0x7f4800198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.992+0000 7f4806481640 1 --2- 172.21.15.43:0/2937574096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f480006a840 0x7f48001986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.992+0000 7f4806481640 1 -- 172.21.15.43:0/2937574096 >> 172.21.15.43:0/2937574096 conn(0x7f4800077eb0 msgr2=0x7f48000fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:13.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.992+0000 7f4806481640 1 -- 172.21.15.43:0/2937574096 shutdown_connections 2024-04-21T08:51:13.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:12.993+0000 7f4806481640 1 -- 172.21.15.43:0/2937574096 wait complete. 2024-04-21T08:51:13.003 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:51:13.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.107+0000 7f8be4614640 1 -- 172.21.15.43:0/1618051904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bdc107260 msgr2=0x7f8bdc10ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.107+0000 7f8be4614640 1 --2- 172.21.15.43:0/1618051904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bdc107260 0x7f8bdc10ee30 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4009ca0 tx=0x7f8bd402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.107+0000 7f8be4614640 1 -- 172.21.15.43:0/1618051904 shutdown_connections 2024-04-21T08:51:13.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.107+0000 7f8be4614640 1 --2- 172.21.15.43:0/1618051904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bdc107260 0x7f8bdc10ee30 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.107+0000 7f8be4614640 1 --2- 172.21.15.43:0/1618051904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bdc102b40 0x7f8bdc106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.107+0000 7f8be4614640 1 --2- 172.21.15.43:0/1618051904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bdc102190 0x7f8bdc102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.107+0000 7f8be4614640 1 -- 172.21.15.43:0/1618051904 >> 172.21.15.43:0/1618051904 conn(0x7f8bdc0793a0 msgr2=0x7f8bdc0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:13.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.107+0000 7f8be4614640 1 -- 172.21.15.43:0/1618051904 shutdown_connections 2024-04-21T08:51:13.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.107+0000 7f8be4614640 1 -- 172.21.15.43:0/1618051904 wait complete. 2024-04-21T08:51:13.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.108+0000 7f8be4614640 1 Processor -- start 2024-04-21T08:51:13.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.108+0000 7f8be4614640 1 -- start start 2024-04-21T08:51:13.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.108+0000 7f8be4614640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bdc102190 0x7f8bdc198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.108+0000 7f8be4614640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bdc107260 0x7f8bdc198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.108+0000 7f8be4614640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bdc19cff0 0x7f8bdc19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.108+0000 7f8be4614640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc1121d0 con 0x7f8bdc19cff0 2024-04-21T08:51:13.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.108+0000 7f8be4614640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc112050 con 0x7f8bdc102190 2024-04-21T08:51:13.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.108+0000 7f8be4614640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc112350 con 0x7f8bdc107260 2024-04-21T08:51:13.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.108+0000 7f8be2bb1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bdc19cff0 0x7f8bdc19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.108+0000 7f8be2bb1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bdc19cff0 0x7f8bdc19d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51440/0 (socket says 172.21.15.43:51440) 2024-04-21T08:51:13.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.108+0000 7f8be2bb1640 1 -- 172.21.15.43:0/1967368064 learned_addr learned my addr 172.21.15.43:0/1967368064 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:13.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.108+0000 7f8be23b0640 1 --2- 172.21.15.43:0/1967368064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bdc102190 0x7f8bdc198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.108+0000 7f8be1baf640 1 --2- 172.21.15.43:0/1967368064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bdc107260 0x7f8bdc198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.108+0000 7f8be2bb1640 1 -- 172.21.15.43:0/1967368064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bdc107260 msgr2=0x7f8bdc198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.108+0000 7f8be2bb1640 1 --2- 172.21.15.43:0/1967368064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bdc107260 0x7f8bdc198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.108+0000 7f8be2bb1640 1 -- 172.21.15.43:0/1967368064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bdc102190 msgr2=0x7f8bdc198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.108+0000 7f8be2bb1640 1 --2- 172.21.15.43:0/1967368064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bdc102190 0x7f8bdc198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.108+0000 7f8be2bb1640 1 -- 172.21.15.43:0/1967368064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bdc19dae0 con 0x7f8bdc19cff0 2024-04-21T08:51:13.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.108+0000 7f8be2bb1640 1 --2- 172.21.15.43:0/1967368064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bdc19cff0 0x7f8bdc19d4a0 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4005e30 tx=0x7f8bd4032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:13.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.109+0000 7f8bd37fe640 1 -- 172.21.15.43:0/1967368064 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4041020 con 0x7f8bdc19cff0 2024-04-21T08:51:13.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.109+0000 7f8bd37fe640 1 -- 172.21.15.43:0/1967368064 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bd4032970 con 0x7f8bdc19cff0 2024-04-21T08:51:13.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.109+0000 7f8be4614640 1 -- 172.21.15.43:0/1967368064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bdc19dd70 con 0x7f8bdc19cff0 2024-04-21T08:51:13.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.109+0000 7f8bd37fe640 1 -- 172.21.15.43:0/1967368064 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4039040 con 0x7f8bdc19cff0 2024-04-21T08:51:13.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.109+0000 7f8be4614640 1 -- 172.21.15.43:0/1967368064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bdc1a5530 con 0x7f8bdc19cff0 2024-04-21T08:51:13.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.111+0000 7f8bd37fe640 1 -- 172.21.15.43:0/1967368064 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8bd4044020 con 0x7f8bdc19cff0 2024-04-21T08:51:13.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.111+0000 7f8be7206740 1 Processor -- start 2024-04-21T08:51:13.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.111+0000 7f8be7206740 1 -- start start 2024-04-21T08:51:13.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.112+0000 7f8be7206740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6174c14d0 0x55b6174c18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.112+0000 7f8be7206740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174c1e80 0x55b6174ca430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.112+0000 7f8be7206740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6174caa00 0x55b6174cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.112+0000 7f8be7206740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b6174261f0 con 0x55b6174c1e80 2024-04-21T08:51:13.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.112+0000 7f8be7206740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b6173c34b0 con 0x55b6174c14d0 2024-04-21T08:51:13.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.112+0000 7f8be7206740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b6174221b0 con 0x55b6174caa00 2024-04-21T08:51:13.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.112+0000 7f8be23b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174c1e80 0x55b6174ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.112+0000 7f8be23b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174c1e80 0x55b6174ca430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51444/0 (socket says 172.21.15.43:51444) 2024-04-21T08:51:13.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.112+0000 7f8be23b0640 1 -- 172.21.15.43:0/2599105720 learned_addr learned my addr 172.21.15.43:0/2599105720 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:13.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.112+0000 7f8be1baf640 1 --2- 172.21.15.43:0/2599105720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6174c14d0 0x55b6174c18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.112+0000 7f8be1baf640 1 --2- 172.21.15.43:0/2599105720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6174caa00 0x55b6174cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.112+0000 7f8be23b0640 1 -- 172.21.15.43:0/2599105720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6174caa00 msgr2=0x55b6174cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:13.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.112+0000 7f8be23b0640 1 --2- 172.21.15.43:0/2599105720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6174caa00 0x55b6174cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.112+0000 7f8be23b0640 1 -- 172.21.15.43:0/2599105720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6174c14d0 msgr2=0x55b6174c18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:13.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.112+0000 7f8be23b0640 1 --2- 172.21.15.43:0/2599105720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6174c14d0 0x55b6174c18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.112+0000 7f8bd37fe640 1 --2- 172.21.15.43:0/1967368064 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8bc80a3d20 0x7f8bc80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.112+0000 7f8be23b0640 1 -- 172.21.15.43:0/2599105720 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6173b51b0 con 0x55b6174c1e80 2024-04-21T08:51:13.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.112+0000 7f8be1baf640 1 --2- 172.21.15.43:0/2599105720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6174c14d0 0x55b6174c18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:13.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.112+0000 7f8be2bb1640 1 --2- 172.21.15.43:0/1967368064 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8bc80a3d20 0x7f8bc80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.112+0000 7f8be23b0640 1 --2- 172.21.15.43:0/2599105720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174c1e80 0x55b6174ca430 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800dcb0 tx=0x7f8bd800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5478b5462fa8d720 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:13.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.113+0000 7f8bd37fe640 1 -- 172.21.15.43:0/1967368064 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f8bd40f29f0 con 0x7f8bdc19cff0 2024-04-21T08:51:13.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.113+0000 7f8bd0bfe640 1 -- 172.21.15.43:0/2599105720 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8011440 con 0x55b6174c1e80 2024-04-21T08:51:13.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.113+0000 7f8bd0bfe640 1 -- 172.21.15.43:0/2599105720 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bd8011a90 con 0x55b6174c1e80 2024-04-21T08:51:13.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.113+0000 7f8bd0bfe640 1 -- 172.21.15.43:0/2599105720 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd801ba70 con 0x55b6174c1e80 2024-04-21T08:51:13.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.113+0000 7f8be7206740 1 -- 172.21.15.43:0/2599105720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174c1e80 msgr2=0x55b6174ca430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.113+0000 7f8be7206740 1 --2- 172.21.15.43:0/2599105720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174c1e80 0x55b6174ca430 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800dcb0 tx=0x7f8bd800f730 comp rx=0 tx=0).stop 2024-04-21T08:51:13.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.113+0000 7f8be2bb1640 1 --2- 172.21.15.43:0/1967368064 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8bc80a3d20 0x7f8bc80a61e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40a7ba0 tx=0x7f8bd40ba9f0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:13.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.114+0000 7f8be7206740 1 -- 172.21.15.43:0/2599105720 shutdown_connections 2024-04-21T08:51:13.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.114+0000 7f8be7206740 1 --2- 172.21.15.43:0/2599105720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6174caa00 0x55b6174cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.114+0000 7f8be7206740 1 --2- 172.21.15.43:0/2599105720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174c1e80 0x55b6174ca430 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.114+0000 7f8be7206740 1 --2- 172.21.15.43:0/2599105720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6174c14d0 0x55b6174c18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.114+0000 7f8be7206740 1 -- 172.21.15.43:0/2599105720 >> 172.21.15.43:0/2599105720 conn(0x55b6174bbba0 msgr2=0x55b6174bbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:13.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.114+0000 7f8be7206740 1 -- 172.21.15.43:0/2599105720 shutdown_connections 2024-04-21T08:51:13.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.114+0000 7f8be7206740 1 -- 172.21.15.43:0/2599105720 wait complete. 2024-04-21T08:51:13.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.114+0000 7f8be7206740 1 -- start start 2024-04-21T08:51:13.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.114+0000 7f8be7206740 1 Processor -- start 2024-04-21T08:51:13.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.115+0000 7f8be7206740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174c14d0 0x55b6174ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.115+0000 7f8be7206740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6174c1e80 0x55b6174c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.115+0000 7f8be7206740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6174caa00 0x55b6174c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.115+0000 7f8be7206740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b6174cd970 con 0x55b6174c14d0 2024-04-21T08:51:13.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.115+0000 7f8be7206740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b6173df500 con 0x55b6174caa00 2024-04-21T08:51:13.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.115+0000 7f8be7206740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b6174cdaf0 con 0x55b6174c1e80 2024-04-21T08:51:13.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.115+0000 7f8be7206740 10 client.0 _ll_register_callbacks cb 0x55b6173adb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:13.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.115+0000 7f8be1baf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174c14d0 0x55b6174ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.115+0000 7f8be1baf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174c14d0 0x55b6174ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51446/0 (socket says 172.21.15.43:51446) 2024-04-21T08:51:13.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.115+0000 7f8be1baf640 1 -- 172.21.15.43:0/1434294182 learned_addr learned my addr 172.21.15.43:0/1434294182 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:13.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.115+0000 7f8be23b0640 1 --2- 172.21.15.43:0/1434294182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6174c1e80 0x55b6174c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.115+0000 7f8be1baf640 1 -- 172.21.15.43:0/1434294182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6174c1e80 msgr2=0x55b6174c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:13.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.115+0000 7f8be1baf640 1 --2- 172.21.15.43:0/1434294182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6174c1e80 0x55b6174c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.115+0000 7f8be1baf640 1 -- 172.21.15.43:0/1434294182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6174caa00 msgr2=0x55b6174c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:13.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.115+0000 7f8be1baf640 1 --2- 172.21.15.43:0/1434294182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6174caa00 0x55b6174c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.115+0000 7f8be1baf640 1 -- 172.21.15.43:0/1434294182 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6171383f0 con 0x55b6174c14d0 2024-04-21T08:51:13.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.115+0000 7f8be23b0640 1 --2- 172.21.15.43:0/1434294182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6174c1e80 0x55b6174c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:13.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.115+0000 7f8be1baf640 1 --2- 172.21.15.43:0/1434294182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174c14d0 0x55b6174ca2f0 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc011780 tx=0x7f8bcc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b80d850e1f05847 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:13.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.115+0000 7f8bc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:13.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.115+0000 7f8bc5ffb640 1 -- 172.21.15.43:0/1434294182 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc00ed40 con 0x55b6174c14d0 2024-04-21T08:51:13.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.115+0000 7f8be7206740 1 -- 172.21.15.43:0/1434294182 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bd8011a90 con 0x55b6174c14d0 2024-04-21T08:51:13.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.115+0000 7f8bc5ffb640 1 -- 172.21.15.43:0/1434294182 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f8bcc005020 con 0x55b6174c14d0 2024-04-21T08:51:13.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.115+0000 7f8bc5ffb640 1 -- 172.21.15.43:0/1434294182 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8bcc026050 con 0x55b6174c14d0 2024-04-21T08:51:13.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.116+0000 7f8be7206740 10 client.21306 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:13.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.116+0000 7f8be7206740 1 -- 172.21.15.43:0/1434294182 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b6172027d0 con 0x55b6174c14d0 2024-04-21T08:51:13.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.116+0000 7f8bc5ffb640 1 -- 172.21.15.43:0/1434294182 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bd8011a90 con 0x55b6174c14d0 2024-04-21T08:51:13.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.116+0000 7f8bc5ffb640 1 -- 172.21.15.43:0/1434294182 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8bcc01d480 con 0x55b6174c14d0 2024-04-21T08:51:13.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.116+0000 7f8be7206740 10 client.21306 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:13.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.116+0000 7f8be7206740 10 client.21306 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:51:13.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.116+0000 7f8be7206740 20 client.21306 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:13.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.116+0000 7f8be7206740 1 --2- 172.21.15.43:0/1434294182 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55b6174dac80 0x55b6174dd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.116+0000 7f8be7206740 4 client.21306 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:13.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.116+0000 7f8be7206740 1 -- 172.21.15.43:0/1434294182 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6173c5930 con 0x55b6174dac80 2024-04-21T08:51:13.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.116+0000 7f8be23b0640 1 --2- 172.21.15.43:0/1434294182 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55b6174dac80 0x55b6174dd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.117+0000 7f8be23b0640 1 --2- 172.21.15.43:0/1434294182 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55b6174dac80 0x55b6174dd070 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=10c451eecead4ef0 server_cookie=8e2dbc9d80bf7b69 in_seq=0 out_seq=0 2024-04-21T08:51:13.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.117+0000 7f8bc5ffb640 10 client.21306 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:51:13.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.117+0000 7f8bc5ffb640 1 -- 172.21.15.43:0/1434294182 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b6173c5930 con 0x55b6174dac80 2024-04-21T08:51:13.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.117+0000 7f8bc5ffb640 10 client.21306 handle_command_reply: tid=0 2024-04-21T08:51:13.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.117+0000 7f8be7206740 1 client.21306 shutdown 2024-04-21T08:51:13.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.117+0000 7f8be7206740 1 -- 172.21.15.43:0/1434294182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174c14d0 msgr2=0x55b6174ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.117+0000 7f8be7206740 1 --2- 172.21.15.43:0/1434294182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174c14d0 0x55b6174ca2f0 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc011780 tx=0x7f8bcc011c40 comp rx=0 tx=0).stop 2024-04-21T08:51:13.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.117+0000 7f8be7206740 1 -- 172.21.15.43:0/1434294182 shutdown_connections 2024-04-21T08:51:13.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.117+0000 7f8be7206740 1 --2- 172.21.15.43:0/1434294182 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55b6174dac80 0x55b6174dd070 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.117+0000 7f8be7206740 1 --2- 172.21.15.43:0/1434294182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6174caa00 0x55b6174c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.117+0000 7f8be7206740 1 --2- 172.21.15.43:0/1434294182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6174c1e80 0x55b6174c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.117+0000 7f8bc5ffb640 0 client.21306 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:51:13.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.117+0000 7f8be7206740 1 --2- 172.21.15.43:0/1434294182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174c14d0 0x55b6174ca2f0 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.117+0000 7f8be7206740 1 -- 172.21.15.43:0/1434294182 >> 172.21.15.43:0/1434294182 conn(0x55b6174bbba0 msgr2=0x55b6174bd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:13.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.118+0000 7f8be7206740 1 -- 172.21.15.43:0/1434294182 shutdown_connections 2024-04-21T08:51:13.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.118+0000 7f8be7206740 1 -- 172.21.15.43:0/1434294182 wait complete. 2024-04-21T08:51:13.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.118+0000 7f8be7206740 20 client.21306 trim_cache size 0 max 16384 2024-04-21T08:51:13.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.131+0000 7f8be7206740 1 Processor -- start 2024-04-21T08:51:13.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.131+0000 7f8be7206740 1 -- start start 2024-04-21T08:51:13.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.131+0000 7f8be7206740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6174c14d0 0x55b6174d6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.131+0000 7f8be7206740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174c1e80 0x55b6174baa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.131+0000 7f8be7206740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6174caa00 0x55b6174c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.131+0000 7f8be7206740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc1af5e0 con 0x55b6174c1e80 2024-04-21T08:51:13.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.131+0000 7f8be7206740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc1b1120 con 0x55b6174caa00 2024-04-21T08:51:13.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.131+0000 7f8be7206740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc1afcb0 con 0x55b6174c14d0 2024-04-21T08:51:13.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.131+0000 7f8be23b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174c1e80 0x55b6174baa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.131+0000 7f8be23b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174c1e80 0x55b6174baa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51454/0 (socket says 172.21.15.43:51454) 2024-04-21T08:51:13.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.131+0000 7f8be23b0640 1 -- 172.21.15.43:0/479606913 learned_addr learned my addr 172.21.15.43:0/479606913 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:13.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.131+0000 7f8be1baf640 1 --2- 172.21.15.43:0/479606913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6174caa00 0x55b6174c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.131+0000 7f8be23b0640 1 -- 172.21.15.43:0/479606913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6174c14d0 msgr2=0x55b6174d6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:13.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.131+0000 7f8be23b0640 1 --2- 172.21.15.43:0/479606913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6174c14d0 0x55b6174d6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.131+0000 7f8be23b0640 1 -- 172.21.15.43:0/479606913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6174caa00 msgr2=0x55b6174c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:13.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.131+0000 7f8be23b0640 1 --2- 172.21.15.43:0/479606913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6174caa00 0x55b6174c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.131+0000 7f8be23b0640 1 -- 172.21.15.43:0/479606913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6174c6630 con 0x55b6174c1e80 2024-04-21T08:51:13.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.132+0000 7f8be23b0640 1 --2- 172.21.15.43:0/479606913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174c1e80 0x55b6174baa80 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8014040 tx=0x7f8bd801b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154a1eb71524dd63 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:13.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.132+0000 7f8bc7fff640 1 -- 172.21.15.43:0/479606913 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8029020 con 0x55b6174c1e80 2024-04-21T08:51:13.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.132+0000 7f8bc7fff640 1 -- 172.21.15.43:0/479606913 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bd801b530 con 0x55b6174c1e80 2024-04-21T08:51:13.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.132+0000 7f8bc7fff640 1 -- 172.21.15.43:0/479606913 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8011440 con 0x55b6174c1e80 2024-04-21T08:51:13.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.132+0000 7f8be7206740 1 -- 172.21.15.43:0/479606913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174c1e80 msgr2=0x55b6174baa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.132+0000 7f8be7206740 1 --2- 172.21.15.43:0/479606913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174c1e80 0x55b6174baa80 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8014040 tx=0x7f8bd801b6f0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.132+0000 7f8be7206740 1 -- 172.21.15.43:0/479606913 shutdown_connections 2024-04-21T08:51:13.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.132+0000 7f8be7206740 1 --2- 172.21.15.43:0/479606913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6174caa00 0x55b6174c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.132+0000 7f8be7206740 1 --2- 172.21.15.43:0/479606913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174c1e80 0x55b6174baa80 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.132+0000 7f8be7206740 1 --2- 172.21.15.43:0/479606913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6174c14d0 0x55b6174d6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.132+0000 7f8be7206740 1 -- 172.21.15.43:0/479606913 >> 172.21.15.43:0/479606913 conn(0x55b6174bbba0 msgr2=0x55b6174ba370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.133+0000 7f8be7206740 1 -- 172.21.15.43:0/479606913 shutdown_connections 2024-04-21T08:51:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.133+0000 7f8be7206740 1 -- 172.21.15.43:0/479606913 wait complete. 2024-04-21T08:51:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.133+0000 7f8be7206740 1 -- start start 2024-04-21T08:51:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.133+0000 7f8be7206740 1 Processor -- start 2024-04-21T08:51:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.133+0000 7f8be7206740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6174c14d0 0x55b6174d3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.133+0000 7f8be7206740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6174c1e80 0x55b6174c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.133+0000 7f8be7206740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174caa00 0x55b6174c4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.133+0000 7f8be7206740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc1b0a50 con 0x55b6174caa00 2024-04-21T08:51:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.133+0000 7f8be7206740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc1b0380 con 0x55b6174c14d0 2024-04-21T08:51:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.133+0000 7f8be7206740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc1ad5e0 con 0x55b6174c1e80 2024-04-21T08:51:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.133+0000 7f8be7206740 10 client.0 _ll_register_callbacks cb 0x7f8bcc009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:13.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.133+0000 7f8be1baf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174caa00 0x55b6174c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.133+0000 7f8be1baf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174caa00 0x55b6174c4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51464/0 (socket says 172.21.15.43:51464) 2024-04-21T08:51:13.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.133+0000 7f8be1baf640 1 -- 172.21.15.43:0/1101916019 learned_addr learned my addr 172.21.15.43:0/1101916019 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:13.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.133+0000 7f8be1baf640 1 --2- 172.21.15.43:0/1101916019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6174c14d0 0x55b6174d3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.133+0000 7f8be23b0640 1 --2- 172.21.15.43:0/1101916019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6174c1e80 0x55b6174c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.133+0000 7f8be1baf640 1 -- 172.21.15.43:0/1101916019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6174c1e80 msgr2=0x55b6174c3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:13.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.134+0000 7f8be1baf640 1 --2- 172.21.15.43:0/1101916019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6174c1e80 0x55b6174c3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.134+0000 7f8be1baf640 1 -- 172.21.15.43:0/1101916019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6174c14d0 msgr2=0x55b6174d3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:13.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.134+0000 7f8be1baf640 1 --2- 172.21.15.43:0/1101916019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6174c14d0 0x55b6174d3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.134+0000 7f8be1baf640 1 -- 172.21.15.43:0/1101916019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6174c7f70 con 0x55b6174caa00 2024-04-21T08:51:13.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.134+0000 7f8be23b0640 1 --2- 172.21.15.43:0/1101916019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6174c1e80 0x55b6174c3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:13.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.134+0000 7f8be1baf640 1 --2- 172.21.15.43:0/1101916019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174caa00 0x55b6174c4470 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc011f40 tx=0x7f8bcc011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3e0e5e554c3b1e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:13.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.134+0000 7f8bc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:13.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.134+0000 7f8bc5ffb640 1 -- 172.21.15.43:0/1101916019 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc023040 con 0x55b6174caa00 2024-04-21T08:51:13.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.134+0000 7f8bc5ffb640 1 -- 172.21.15.43:0/1101916019 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f8bcc01dc50 con 0x55b6174caa00 2024-04-21T08:51:13.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.134+0000 7f8be7206740 1 -- 172.21.15.43:0/1101916019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b617433c10 con 0x55b6174caa00 2024-04-21T08:51:13.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.134+0000 7f8bc5ffb640 1 -- 172.21.15.43:0/1101916019 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8bcc00eb40 con 0x55b6174caa00 2024-04-21T08:51:13.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.134+0000 7f8be7206740 10 client.21312 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:13.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.134+0000 7f8be7206740 1 -- 172.21.15.43:0/1101916019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b617545300 con 0x55b6174caa00 2024-04-21T08:51:13.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.134+0000 7f8bc5ffb640 1 -- 172.21.15.43:0/1101916019 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b617433c10 con 0x55b6174caa00 2024-04-21T08:51:13.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.134+0000 7f8bc5ffb640 1 -- 172.21.15.43:0/1101916019 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8bcc0052e0 con 0x55b6174caa00 2024-04-21T08:51:13.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.134+0000 7f8be7206740 10 client.21312 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:13.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.134+0000 7f8be7206740 10 client.21312 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:51:13.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.134+0000 7f8be7206740 20 client.21312 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:13.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.134+0000 7f8be7206740 1 --2- 172.21.15.43:0/1101916019 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55b617545940 0x55b617547d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.134+0000 7f8be7206740 4 client.21312 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:51:13.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.134+0000 7f8be7206740 1 -- 172.21.15.43:0/1101916019 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b61742da80 con 0x55b617545940 2024-04-21T08:51:13.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.135+0000 7f8be23b0640 1 --2- 172.21.15.43:0/1101916019 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55b617545940 0x55b617547d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.136+0000 7f8be23b0640 1 --2- 172.21.15.43:0/1101916019 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55b617545940 0x55b617547d60 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=360feae445cd8e4e server_cookie=220663539aad16d6 in_seq=0 out_seq=0 2024-04-21T08:51:13.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.136+0000 7f8bc5ffb640 10 client.21312 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:51:13.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.136+0000 7f8bc5ffb640 1 -- 172.21.15.43:0/1101916019 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b61742da80 con 0x55b617545940 2024-04-21T08:51:13.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.136+0000 7f8bc5ffb640 10 client.21312 handle_command_reply: tid=0 2024-04-21T08:51:13.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.136+0000 7f8be7206740 1 client.21312 shutdown 2024-04-21T08:51:13.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.136+0000 7f8be7206740 1 -- 172.21.15.43:0/1101916019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174caa00 msgr2=0x55b6174c4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.136+0000 7f8be7206740 1 --2- 172.21.15.43:0/1101916019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174caa00 0x55b6174c4470 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc011f40 tx=0x7f8bcc011ed0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.136+0000 7f8be7206740 1 -- 172.21.15.43:0/1101916019 shutdown_connections 2024-04-21T08:51:13.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.136+0000 7f8be7206740 1 --2- 172.21.15.43:0/1101916019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6174caa00 0x55b6174c4470 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.136+0000 7f8be7206740 1 --2- 172.21.15.43:0/1101916019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b6174c1e80 0x55b6174c3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.136+0000 7f8be7206740 1 --2- 172.21.15.43:0/1101916019 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x55b617545940 0x55b617547d60 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.137+0000 7f8bc5ffb640 0 client.21312 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:51:13.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.136+0000 7f8be7206740 1 --2- 172.21.15.43:0/1101916019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b6174c14d0 0x55b6174d3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.137+0000 7f8be7206740 1 -- 172.21.15.43:0/1101916019 >> 172.21.15.43:0/1101916019 conn(0x55b6174bbba0 msgr2=0x55b6174bac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:13.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.137+0000 7f8be7206740 1 -- 172.21.15.43:0/1101916019 shutdown_connections 2024-04-21T08:51:13.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.137+0000 7f8be7206740 1 -- 172.21.15.43:0/1101916019 wait complete. 2024-04-21T08:51:13.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.137+0000 7f8be7206740 20 client.21312 trim_cache size 0 max 16384 2024-04-21T08:51:13.143 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:51:13.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.137+0000 7f8be4614640 1 -- 172.21.15.43:0/1967368064 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8bc80a3d20 msgr2=0x7f8bc80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.137+0000 7f8be4614640 1 --2- 172.21.15.43:0/1967368064 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8bc80a3d20 0x7f8bc80a61e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40a7ba0 tx=0x7f8bd40ba9f0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.137+0000 7f8be4614640 1 -- 172.21.15.43:0/1967368064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bdc19cff0 msgr2=0x7f8bdc19d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.137+0000 7f8be4614640 1 --2- 172.21.15.43:0/1967368064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bdc19cff0 0x7f8bdc19d4a0 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4005e30 tx=0x7f8bd4032420 comp rx=0 tx=0).stop 2024-04-21T08:51:13.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.137+0000 7f8be4614640 1 -- 172.21.15.43:0/1967368064 shutdown_connections 2024-04-21T08:51:13.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.137+0000 7f8be4614640 1 --2- 172.21.15.43:0/1967368064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bdc19cff0 0x7f8bdc19d4a0 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.137+0000 7f8be4614640 1 --2- 172.21.15.43:0/1967368064 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8bc80a3d20 0x7f8bc80a61e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.137+0000 7f8be4614640 1 --2- 172.21.15.43:0/1967368064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bdc107260 0x7f8bdc198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.137+0000 7f8be4614640 1 --2- 172.21.15.43:0/1967368064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bdc102190 0x7f8bdc198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.137+0000 7f8be4614640 1 -- 172.21.15.43:0/1967368064 >> 172.21.15.43:0/1967368064 conn(0x7f8bdc0793a0 msgr2=0x7f8bdc0fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:13.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.137+0000 7f8be4614640 1 -- 172.21.15.43:0/1967368064 shutdown_connections 2024-04-21T08:51:13.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.137+0000 7f8be4614640 1 -- 172.21.15.43:0/1967368064 wait complete. 2024-04-21T08:51:13.147 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:51:13.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.253+0000 7f84a2806640 1 -- 172.21.15.43:0/4116144099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f849c107270 msgr2=0x7f849c10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.253+0000 7f84a2806640 1 --2- 172.21.15.43:0/4116144099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f849c107270 0x7f849c10ee40 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f8494009ca0 tx=0x7f849402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.253+0000 7f84a2806640 1 -- 172.21.15.43:0/4116144099 shutdown_connections 2024-04-21T08:51:13.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.253+0000 7f84a2806640 1 --2- 172.21.15.43:0/4116144099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f849c107270 0x7f849c10ee40 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.253+0000 7f84a2806640 1 --2- 172.21.15.43:0/4116144099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f849c06b1f0 0x7f849c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.253+0000 7f84a2806640 1 --2- 172.21.15.43:0/4116144099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f849c06a840 0x7f849c06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.253+0000 7f84a2806640 1 -- 172.21.15.43:0/4116144099 >> 172.21.15.43:0/4116144099 conn(0x7f849c077eb0 msgr2=0x7f849c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:13.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.253+0000 7f84a2806640 1 -- 172.21.15.43:0/4116144099 shutdown_connections 2024-04-21T08:51:13.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.253+0000 7f84a2806640 1 -- 172.21.15.43:0/4116144099 wait complete. 2024-04-21T08:51:13.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.253+0000 7f84a2806640 1 Processor -- start 2024-04-21T08:51:13.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.254+0000 7f84a2806640 1 -- start start 2024-04-21T08:51:13.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.254+0000 7f84a2806640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f849c06a840 0x7f849c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.254+0000 7f84a2806640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f849c107270 0x7f849c198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.254+0000 7f84a2806640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f849c19cf70 0x7f849c19d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.254+0000 7f84a2806640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1121d0 con 0x7f849c107270 2024-04-21T08:51:13.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.254+0000 7f84a2806640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f849c112050 con 0x7f849c06a840 2024-04-21T08:51:13.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.254+0000 7f84a2806640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f849c112350 con 0x7f849c19cf70 2024-04-21T08:51:13.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.254+0000 7f849b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f849c107270 0x7f849c198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.254+0000 7f849b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f849c107270 0x7f849c198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51474/0 (socket says 172.21.15.43:51474) 2024-04-21T08:51:13.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.254+0000 7f849b7fe640 1 -- 172.21.15.43:0/1797105572 learned_addr learned my addr 172.21.15.43:0/1797105572 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:13.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.254+0000 7f849bfff640 1 --2- 172.21.15.43:0/1797105572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f849c06a840 0x7f849c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.254+0000 7f84a0da3640 1 --2- 172.21.15.43:0/1797105572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f849c19cf70 0x7f849c19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.254+0000 7f849b7fe640 1 -- 172.21.15.43:0/1797105572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f849c19cf70 msgr2=0x7f849c19d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.254+0000 7f849b7fe640 1 --2- 172.21.15.43:0/1797105572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f849c19cf70 0x7f849c19d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.254+0000 7f849b7fe640 1 -- 172.21.15.43:0/1797105572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f849c06a840 msgr2=0x7f849c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.254+0000 7f849b7fe640 1 --2- 172.21.15.43:0/1797105572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f849c06a840 0x7f849c1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.254+0000 7f849b7fe640 1 -- 172.21.15.43:0/1797105572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849c19da60 con 0x7f849c107270 2024-04-21T08:51:13.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.254+0000 7f849bfff640 1 --2- 172.21.15.43:0/1797105572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f849c06a840 0x7f849c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:13.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.255+0000 7f849b7fe640 1 --2- 172.21.15.43:0/1797105572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f849c107270 0x7f849c198bf0 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f849000d770 tx=0x7f849000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:13.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.255+0000 7f84997fa640 1 -- 172.21.15.43:0/1797105572 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490017070 con 0x7f849c107270 2024-04-21T08:51:13.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.255+0000 7f84997fa640 1 -- 172.21.15.43:0/1797105572 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8490009c30 con 0x7f849c107270 2024-04-21T08:51:13.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.255+0000 7f84997fa640 1 -- 172.21.15.43:0/1797105572 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849000f620 con 0x7f849c107270 2024-04-21T08:51:13.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.255+0000 7f84a2806640 1 -- 172.21.15.43:0/1797105572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f849c19dd50 con 0x7f849c107270 2024-04-21T08:51:13.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.255+0000 7f84a2806640 1 -- 172.21.15.43:0/1797105572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f849c1a54f0 con 0x7f849c107270 2024-04-21T08:51:13.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.258+0000 7f84997fa640 1 -- 172.21.15.43:0/1797105572 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f849001a070 con 0x7f849c107270 2024-04-21T08:51:13.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.258+0000 7f84a53fc740 1 Processor -- start 2024-04-21T08:51:13.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.258+0000 7f84a53fc740 1 -- start start 2024-04-21T08:51:13.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.258+0000 7f84a53fc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e95eaa4d0 0x556e95eaa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.258+0000 7f84a53fc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e95eaae80 0x556e95eb3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.258+0000 7f84a53fc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 0x556e95eb5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.258+0000 7f84a53fc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556e95e0f1f0 con 0x556e95eb3a00 2024-04-21T08:51:13.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.258+0000 7f84a53fc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556e95dac4b0 con 0x556e95eaa4d0 2024-04-21T08:51:13.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.258+0000 7f84a53fc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556e95e0b1b0 con 0x556e95eaae80 2024-04-21T08:51:13.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.258+0000 7f849b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 0x556e95eb5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.258+0000 7f849b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 0x556e95eb5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51480/0 (socket says 172.21.15.43:51480) 2024-04-21T08:51:13.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.258+0000 7f849b7fe640 1 -- 172.21.15.43:0/1137815941 learned_addr learned my addr 172.21.15.43:0/1137815941 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:13.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.258+0000 7f84997fa640 1 --2- 172.21.15.43:0/1797105572 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84840a3d20 0x7f84840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.258+0000 7f84a0da3640 1 --2- 172.21.15.43:0/1137815941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e95eaa4d0 0x556e95eaa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.258+0000 7f849bfff640 1 --2- 172.21.15.43:0/1137815941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e95eaae80 0x556e95eb3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.259+0000 7f84a0da3640 1 --2- 172.21.15.43:0/1797105572 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84840a3d20 0x7f84840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.259+0000 7f84997fa640 1 -- 172.21.15.43:0/1797105572 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f84900d07b0 con 0x7f849c107270 2024-04-21T08:51:13.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.259+0000 7f849b7fe640 1 -- 172.21.15.43:0/1137815941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e95eaae80 msgr2=0x556e95eb3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:13.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.259+0000 7f849b7fe640 1 --2- 172.21.15.43:0/1137815941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e95eaae80 0x556e95eb3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.259+0000 7f849b7fe640 1 -- 172.21.15.43:0/1137815941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e95eaa4d0 msgr2=0x556e95eaa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:13.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.259+0000 7f849b7fe640 1 --2- 172.21.15.43:0/1137815941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e95eaa4d0 0x556e95eaa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.259+0000 7f849b7fe640 1 -- 172.21.15.43:0/1137815941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e95d9e1b0 con 0x556e95eb3a00 2024-04-21T08:51:13.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.259+0000 7f84a0da3640 1 --2- 172.21.15.43:0/1137815941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e95eaa4d0 0x556e95eaa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:13.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.259+0000 7f849b7fe640 1 --2- 172.21.15.43:0/1137815941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 0x556e95eb5e50 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f849009c030 tx=0x7f849000df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa7232ead53a7e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:13.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.259+0000 7f84a0da3640 1 --2- 172.21.15.43:0/1797105572 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84840a3d20 0x7f84840a61e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8494005e30 tx=0x7f849402c0f0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:13.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.259+0000 7f84827fc640 1 -- 172.21.15.43:0/1137815941 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490021050 con 0x556e95eb3a00 2024-04-21T08:51:13.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.259+0000 7f84827fc640 1 -- 172.21.15.43:0/1137815941 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f849009ca30 con 0x556e95eb3a00 2024-04-21T08:51:13.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.259+0000 7f84827fc640 1 -- 172.21.15.43:0/1137815941 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849009c590 con 0x556e95eb3a00 2024-04-21T08:51:13.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.260+0000 7f84a53fc740 1 -- 172.21.15.43:0/1137815941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 msgr2=0x556e95eb5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.260+0000 7f84a53fc740 1 --2- 172.21.15.43:0/1137815941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 0x556e95eb5e50 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f849009c030 tx=0x7f849000df10 comp rx=0 tx=0).stop 2024-04-21T08:51:13.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.260+0000 7f84a53fc740 1 -- 172.21.15.43:0/1137815941 shutdown_connections 2024-04-21T08:51:13.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.260+0000 7f84a53fc740 1 --2- 172.21.15.43:0/1137815941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 0x556e95eb5e50 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.260+0000 7f84a53fc740 1 --2- 172.21.15.43:0/1137815941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e95eaae80 0x556e95eb3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.260+0000 7f84a53fc740 1 --2- 172.21.15.43:0/1137815941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e95eaa4d0 0x556e95eaa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.260+0000 7f84a53fc740 1 -- 172.21.15.43:0/1137815941 >> 172.21.15.43:0/1137815941 conn(0x556e95ea4ba0 msgr2=0x556e95ea4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:13.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.260+0000 7f84a53fc740 1 -- 172.21.15.43:0/1137815941 shutdown_connections 2024-04-21T08:51:13.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.260+0000 7f84a53fc740 1 -- 172.21.15.43:0/1137815941 wait complete. 2024-04-21T08:51:13.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.260+0000 7f84a53fc740 1 -- start start 2024-04-21T08:51:13.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.260+0000 7f84a53fc740 1 Processor -- start 2024-04-21T08:51:13.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.261+0000 7f84a53fc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e95eaa4d0 0x556e95eb3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.261+0000 7f84a53fc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e95eaae80 0x556e95eb06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.261+0000 7f84a53fc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 0x556e95eb0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.261+0000 7f84a53fc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556e95eb68b0 con 0x556e95eb3a00 2024-04-21T08:51:13.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.261+0000 7f84a53fc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556e95dc8500 con 0x556e95eaae80 2024-04-21T08:51:13.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.261+0000 7f84a53fc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556e95eb6a30 con 0x556e95eaa4d0 2024-04-21T08:51:13.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.261+0000 7f84a53fc740 10 client.0 _ll_register_callbacks cb 0x556e95d96b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:13.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.261+0000 7f84a0da3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 0x556e95eb0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.261+0000 7f84a0da3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 0x556e95eb0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51484/0 (socket says 172.21.15.43:51484) 2024-04-21T08:51:13.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.261+0000 7f849bfff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e95eaa4d0 0x556e95eb3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.261+0000 7f84a0da3640 1 -- 172.21.15.43:0/1927370915 learned_addr learned my addr 172.21.15.43:0/1927370915 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:13.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.261+0000 7f849b7fe640 1 --2- 172.21.15.43:0/1927370915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e95eaae80 0x556e95eb06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.261+0000 7f84a0da3640 1 -- 172.21.15.43:0/1927370915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e95eaa4d0 msgr2=0x556e95eb3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:13.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.261+0000 7f84a0da3640 1 --2- 172.21.15.43:0/1927370915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e95eaa4d0 0x556e95eb3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.261+0000 7f84a0da3640 1 -- 172.21.15.43:0/1927370915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e95eaae80 msgr2=0x556e95eb06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:13.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.261+0000 7f84a0da3640 1 --2- 172.21.15.43:0/1927370915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e95eaae80 0x556e95eb06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.261+0000 7f84a0da3640 1 -- 172.21.15.43:0/1927370915 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e95b213f0 con 0x556e95eb3a00 2024-04-21T08:51:13.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.261+0000 7f84a0da3640 1 --2- 172.21.15.43:0/1927370915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 0x556e95eb0d00 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f8494036000 tx=0x7f84940095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75747e86906d271a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:13.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.261+0000 7f847b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:13.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.261+0000 7f847b7fe640 1 -- 172.21.15.43:0/1927370915 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8494039040 con 0x556e95eb3a00 2024-04-21T08:51:13.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.262+0000 7f847b7fe640 1 -- 172.21.15.43:0/1927370915 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f8494004040 con 0x556e95eb3a00 2024-04-21T08:51:13.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.262+0000 7f84a53fc740 1 -- 172.21.15.43:0/1927370915 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f849009ca30 con 0x556e95eb3a00 2024-04-21T08:51:13.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.262+0000 7f847b7fe640 1 -- 172.21.15.43:0/1927370915 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8494058050 con 0x556e95eb3a00 2024-04-21T08:51:13.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.262+0000 7f84a53fc740 10 client.21324 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:13.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.262+0000 7f84a53fc740 1 -- 172.21.15.43:0/1927370915 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556e95beb7d0 con 0x556e95eb3a00 2024-04-21T08:51:13.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.262+0000 7f847b7fe640 1 -- 172.21.15.43:0/1927370915 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f849009ca30 con 0x556e95eb3a00 2024-04-21T08:51:13.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.262+0000 7f847b7fe640 1 -- 172.21.15.43:0/1927370915 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f84940345d0 con 0x556e95eb3a00 2024-04-21T08:51:13.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.262+0000 7f84a53fc740 10 client.21324 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:13.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.262+0000 7f84a53fc740 10 client.21324 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:51:13.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.262+0000 7f84a53fc740 20 client.21324 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:13.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.262+0000 7f84a53fc740 1 --2- 172.21.15.43:0/1927370915 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x556e95ec3be0 0x556e95ec6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.262+0000 7f84a53fc740 4 client.21324 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:13.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.262+0000 7f84a53fc740 1 -- 172.21.15.43:0/1927370915 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e95dae930 con 0x556e95ec3be0 2024-04-21T08:51:13.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.262+0000 7f849bfff640 1 --2- 172.21.15.43:0/1927370915 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x556e95ec3be0 0x556e95ec6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.263+0000 7f849bfff640 1 --2- 172.21.15.43:0/1927370915 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x556e95ec3be0 0x556e95ec6040 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=90a93e61ebda8e42 server_cookie=88966cb2751d1abf in_seq=0 out_seq=0 2024-04-21T08:51:13.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.263+0000 7f847b7fe640 10 client.21324 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:51:13.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.263+0000 7f847b7fe640 1 -- 172.21.15.43:0/1927370915 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e95dae930 con 0x556e95ec3be0 2024-04-21T08:51:13.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.263+0000 7f847b7fe640 10 client.21324 handle_command_reply: tid=0 2024-04-21T08:51:13.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.263+0000 7f84a53fc740 1 client.21324 shutdown 2024-04-21T08:51:13.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.264+0000 7f84a53fc740 1 -- 172.21.15.43:0/1927370915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 msgr2=0x556e95eb0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.264+0000 7f84a53fc740 1 --2- 172.21.15.43:0/1927370915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 0x556e95eb0d00 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f8494036000 tx=0x7f84940095c0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.264+0000 7f84a53fc740 1 -- 172.21.15.43:0/1927370915 shutdown_connections 2024-04-21T08:51:13.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.264+0000 7f84a53fc740 1 --2- 172.21.15.43:0/1927370915 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x556e95ec3be0 0x556e95ec6040 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.264+0000 7f84a53fc740 1 --2- 172.21.15.43:0/1927370915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 0x556e95eb0d00 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.264+0000 7f84a53fc740 1 --2- 172.21.15.43:0/1927370915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e95eaae80 0x556e95eb06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.264+0000 7f847b7fe640 0 client.21324 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:51:13.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.264+0000 7f84a53fc740 1 --2- 172.21.15.43:0/1927370915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e95eaa4d0 0x556e95eb3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.264+0000 7f84a53fc740 1 -- 172.21.15.43:0/1927370915 >> 172.21.15.43:0/1927370915 conn(0x556e95ea4ba0 msgr2=0x556e95ea68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:13.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.264+0000 7f84a53fc740 1 -- 172.21.15.43:0/1927370915 shutdown_connections 2024-04-21T08:51:13.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.264+0000 7f84a53fc740 1 -- 172.21.15.43:0/1927370915 wait complete. 2024-04-21T08:51:13.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.264+0000 7f84a53fc740 20 client.21324 trim_cache size 0 max 16384 2024-04-21T08:51:13.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.277+0000 7f84a53fc740 1 Processor -- start 2024-04-21T08:51:13.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.277+0000 7f84a53fc740 1 -- start start 2024-04-21T08:51:13.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.277+0000 7f84a53fc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e95eaa4d0 0x556e95ebf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.277+0000 7f84a53fc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e95eaae80 0x556e95ea3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.277+0000 7f84a53fc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 0x556e95eb2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.277+0000 7f84a53fc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1af4d0 con 0x556e95eb3a00 2024-04-21T08:51:13.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.277+0000 7f84a53fc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1b1010 con 0x556e95eaa4d0 2024-04-21T08:51:13.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.277+0000 7f84a53fc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1afba0 con 0x556e95eaae80 2024-04-21T08:51:13.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.277+0000 7f84a0da3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 0x556e95eb2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.277+0000 7f84a0da3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 0x556e95eb2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51496/0 (socket says 172.21.15.43:51496) 2024-04-21T08:51:13.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.277+0000 7f84a0da3640 1 -- 172.21.15.43:0/3568812408 learned_addr learned my addr 172.21.15.43:0/3568812408 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:13.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.277+0000 7f849b7fe640 1 --2- 172.21.15.43:0/3568812408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e95eaae80 0x556e95ea3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.277+0000 7f84a0da3640 1 -- 172.21.15.43:0/3568812408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e95eaae80 msgr2=0x556e95ea3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:13.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.277+0000 7f849bfff640 1 --2- 172.21.15.43:0/3568812408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e95eaa4d0 0x556e95ebf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.277+0000 7f84a0da3640 1 --2- 172.21.15.43:0/3568812408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e95eaae80 0x556e95ea3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.277+0000 7f84a0da3640 1 -- 172.21.15.43:0/3568812408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e95eaa4d0 msgr2=0x556e95ebf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:13.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.277+0000 7f84a0da3640 1 --2- 172.21.15.43:0/3568812408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e95eaa4d0 0x556e95ebf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.277+0000 7f84a0da3640 1 -- 172.21.15.43:0/3568812408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e95eac2c0 con 0x556e95eb3a00 2024-04-21T08:51:13.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.277+0000 7f849b7fe640 1 --2- 172.21.15.43:0/3568812408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e95eaae80 0x556e95ea3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:13.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.277+0000 7f849bfff640 1 --2- 172.21.15.43:0/3568812408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e95eaa4d0 0x556e95ebf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:13.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.278+0000 7f84a0da3640 1 --2- 172.21.15.43:0/3568812408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 0x556e95eb2e20 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f8494006010 tx=0x7f8494005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b14aedca977697 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:13.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.278+0000 7f8481ffb640 1 -- 172.21.15.43:0/3568812408 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8494061070 con 0x556e95eb3a00 2024-04-21T08:51:13.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.278+0000 7f8481ffb640 1 -- 172.21.15.43:0/3568812408 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8494032bc0 con 0x556e95eb3a00 2024-04-21T08:51:13.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.278+0000 7f8481ffb640 1 -- 172.21.15.43:0/3568812408 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8494039030 con 0x556e95eb3a00 2024-04-21T08:51:13.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.278+0000 7f84a53fc740 1 -- 172.21.15.43:0/3568812408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 msgr2=0x556e95eb2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.278+0000 7f84a53fc740 1 --2- 172.21.15.43:0/3568812408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 0x556e95eb2e20 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f8494006010 tx=0x7f8494005d80 comp rx=0 tx=0).stop 2024-04-21T08:51:13.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.278+0000 7f84a53fc740 1 -- 172.21.15.43:0/3568812408 shutdown_connections 2024-04-21T08:51:13.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.278+0000 7f84a53fc740 1 --2- 172.21.15.43:0/3568812408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 0x556e95eb2e20 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.278+0000 7f84a53fc740 1 --2- 172.21.15.43:0/3568812408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e95eaae80 0x556e95ea3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.278+0000 7f84a53fc740 1 --2- 172.21.15.43:0/3568812408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e95eaa4d0 0x556e95ebf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.278+0000 7f84a53fc740 1 -- 172.21.15.43:0/3568812408 >> 172.21.15.43:0/3568812408 conn(0x556e95ea4ba0 msgr2=0x556e95ea3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:13.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.278+0000 7f84a53fc740 1 -- 172.21.15.43:0/3568812408 shutdown_connections 2024-04-21T08:51:13.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.278+0000 7f84a53fc740 1 -- 172.21.15.43:0/3568812408 wait complete. 2024-04-21T08:51:13.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.279+0000 7f84a53fc740 1 -- start start 2024-04-21T08:51:13.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.279+0000 7f84a53fc740 1 Processor -- start 2024-04-21T08:51:13.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.279+0000 7f84a53fc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e95eaa4d0 0x556e95ebc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.279+0000 7f84a53fc740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e95eaae80 0x556e95eb7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.279+0000 7f84a53fc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 0x556e95eb7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.279+0000 7f84a53fc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1b0940 con 0x556e95eb3a00 2024-04-21T08:51:13.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.279+0000 7f84a53fc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1b0270 con 0x556e95eaa4d0 2024-04-21T08:51:13.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.279+0000 7f84a53fc740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1ad520 con 0x556e95eaae80 2024-04-21T08:51:13.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.279+0000 7f84a53fc740 10 client.0 _ll_register_callbacks cb 0x7f84940327f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:13.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.279+0000 7f84a0da3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 0x556e95eb7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.279+0000 7f84a0da3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 0x556e95eb7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51510/0 (socket says 172.21.15.43:51510) 2024-04-21T08:51:13.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.279+0000 7f84a0da3640 1 -- 172.21.15.43:0/238875607 learned_addr learned my addr 172.21.15.43:0/238875607 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:13.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.279+0000 7f849b7fe640 1 --2- 172.21.15.43:0/238875607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e95eaae80 0x556e95eb7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.279+0000 7f849bfff640 1 --2- 172.21.15.43:0/238875607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e95eaa4d0 0x556e95ebc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.280+0000 7f84a0da3640 1 -- 172.21.15.43:0/238875607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e95eaae80 msgr2=0x556e95eb7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:13.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.280+0000 7f84a0da3640 1 --2- 172.21.15.43:0/238875607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e95eaae80 0x556e95eb7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.280+0000 7f84a0da3640 1 -- 172.21.15.43:0/238875607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e95eaa4d0 msgr2=0x556e95ebc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.280+0000 7f84a0da3640 1 --2- 172.21.15.43:0/238875607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e95eaa4d0 0x556e95ebc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.280+0000 7f84a0da3640 1 -- 172.21.15.43:0/238875607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e95f291d0 con 0x556e95eb3a00 2024-04-21T08:51:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.280+0000 7f84a0da3640 1 --2- 172.21.15.43:0/238875607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 0x556e95eb7c50 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f8494034970 tx=0x7f8494009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b430ec6522a6ff server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.280+0000 7f847b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.280+0000 7f847b7fe640 1 -- 172.21.15.43:0/238875607 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8494061070 con 0x556e95eb3a00 2024-04-21T08:51:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.280+0000 7f847b7fe640 1 -- 172.21.15.43:0/238875607 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f8494034650 con 0x556e95eb3a00 2024-04-21T08:51:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.280+0000 7f84a53fc740 1 -- 172.21.15.43:0/238875607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8494032bc0 con 0x556e95eb3a00 2024-04-21T08:51:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.280+0000 7f847b7fe640 1 -- 172.21.15.43:0/238875607 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8494039070 con 0x556e95eb3a00 2024-04-21T08:51:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.280+0000 7f84a53fc740 10 client.21330 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.280+0000 7f84a53fc740 1 -- 172.21.15.43:0/238875607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556e95f2a2e0 con 0x556e95eb3a00 2024-04-21T08:51:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.280+0000 7f847b7fe640 1 -- 172.21.15.43:0/238875607 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8494032bc0 con 0x556e95eb3a00 2024-04-21T08:51:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.280+0000 7f847b7fe640 1 -- 172.21.15.43:0/238875607 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8494064030 con 0x556e95eb3a00 2024-04-21T08:51:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.280+0000 7f84a53fc740 10 client.21330 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:13.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.280+0000 7f84a53fc740 10 client.21330 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:51:13.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.280+0000 7f84a53fc740 20 client.21330 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:13.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.280+0000 7f84a53fc740 1 --2- 172.21.15.43:0/238875607 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x556e95f2a8e0 0x556e95f2ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.280+0000 7f84a53fc740 4 client.21330 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:51:13.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.281+0000 7f84a53fc740 1 -- 172.21.15.43:0/238875607 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e95e1cc10 con 0x556e95f2a8e0 2024-04-21T08:51:13.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.281+0000 7f849bfff640 1 --2- 172.21.15.43:0/238875607 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x556e95f2a8e0 0x556e95f2ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.281+0000 7f849bfff640 1 --2- 172.21.15.43:0/238875607 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x556e95f2a8e0 0x556e95f2ccd0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9368f31ed0c186b6 server_cookie=53b44591f1b72ea9 in_seq=0 out_seq=0 2024-04-21T08:51:13.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.281+0000 7f847b7fe640 10 client.21330 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:51:13.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.281+0000 7f847b7fe640 1 -- 172.21.15.43:0/238875607 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e95e1cc10 con 0x556e95f2a8e0 2024-04-21T08:51:13.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.281+0000 7f847b7fe640 10 client.21330 handle_command_reply: tid=0 2024-04-21T08:51:13.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.281+0000 7f84a53fc740 1 client.21330 shutdown 2024-04-21T08:51:13.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.281+0000 7f84a53fc740 1 -- 172.21.15.43:0/238875607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 msgr2=0x556e95eb7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.281+0000 7f84a53fc740 1 --2- 172.21.15.43:0/238875607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 0x556e95eb7c50 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f8494034970 tx=0x7f8494009930 comp rx=0 tx=0).stop 2024-04-21T08:51:13.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.281+0000 7f84a53fc740 1 -- 172.21.15.43:0/238875607 shutdown_connections 2024-04-21T08:51:13.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.281+0000 7f84a53fc740 1 --2- 172.21.15.43:0/238875607 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x556e95f2a8e0 0x556e95f2ccd0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.281+0000 7f84a53fc740 1 --2- 172.21.15.43:0/238875607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556e95eb3a00 0x556e95eb7c50 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.281+0000 7f84a53fc740 1 --2- 172.21.15.43:0/238875607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556e95eaae80 0x556e95eb7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.281+0000 7f84a53fc740 1 --2- 172.21.15.43:0/238875607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e95eaa4d0 0x556e95ebc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.281+0000 7f84a53fc740 1 -- 172.21.15.43:0/238875607 >> 172.21.15.43:0/238875607 conn(0x556e95ea4ba0 msgr2=0x556e95ea3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:13.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.281+0000 7f847b7fe640 0 client.21330 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:51:13.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.281+0000 7f84a53fc740 1 -- 172.21.15.43:0/238875607 shutdown_connections 2024-04-21T08:51:13.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.282+0000 7f84a53fc740 1 -- 172.21.15.43:0/238875607 wait complete. 2024-04-21T08:51:13.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.282+0000 7f84a53fc740 20 client.21330 trim_cache size 0 max 16384 2024-04-21T08:51:13.288 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:51:13.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.282+0000 7f84a2806640 1 -- 172.21.15.43:0/1797105572 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84840a3d20 msgr2=0x7f84840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.282+0000 7f84a2806640 1 --2- 172.21.15.43:0/1797105572 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84840a3d20 0x7f84840a61e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8494005e30 tx=0x7f849402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.282+0000 7f84a2806640 1 -- 172.21.15.43:0/1797105572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f849c107270 msgr2=0x7f849c198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.282+0000 7f84a2806640 1 --2- 172.21.15.43:0/1797105572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f849c107270 0x7f849c198bf0 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f849000d770 tx=0x7f849000dc30 comp rx=0 tx=0).stop 2024-04-21T08:51:13.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.282+0000 7f84a2806640 1 -- 172.21.15.43:0/1797105572 shutdown_connections 2024-04-21T08:51:13.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.282+0000 7f84a2806640 1 --2- 172.21.15.43:0/1797105572 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84840a3d20 0x7f84840a61e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.282+0000 7f84a2806640 1 --2- 172.21.15.43:0/1797105572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f849c19cf70 0x7f849c19d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.282+0000 7f84a2806640 1 --2- 172.21.15.43:0/1797105572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f849c107270 0x7f849c198bf0 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.282+0000 7f84a2806640 1 --2- 172.21.15.43:0/1797105572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f849c06a840 0x7f849c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.282+0000 7f84a2806640 1 -- 172.21.15.43:0/1797105572 >> 172.21.15.43:0/1797105572 conn(0x7f849c077eb0 msgr2=0x7f849c0fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:13.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.282+0000 7f84a2806640 1 -- 172.21.15.43:0/1797105572 shutdown_connections 2024-04-21T08:51:13.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.282+0000 7f84a2806640 1 -- 172.21.15.43:0/1797105572 wait complete. 2024-04-21T08:51:13.292 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:51:13.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.398+0000 7fad06f8f640 1 -- 172.21.15.43:0/1464674856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad00101b40 msgr2=0x7fad0010dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.398+0000 7fad06f8f640 1 --2- 172.21.15.43:0/1464674856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad00101b40 0x7fad0010dc80 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7facf8009cd0 tx=0x7facf802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.398+0000 7fad06f8f640 1 -- 172.21.15.43:0/1464674856 shutdown_connections 2024-04-21T08:51:13.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.398+0000 7fad06f8f640 1 --2- 172.21.15.43:0/1464674856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad00101b40 0x7fad0010dc80 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.398+0000 7fad06f8f640 1 --2- 172.21.15.43:0/1464674856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad001011a0 0x7fad00101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.398+0000 7fad06f8f640 1 --2- 172.21.15.43:0/1464674856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad00107030 0x7fad00107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.398+0000 7fad06f8f640 1 -- 172.21.15.43:0/1464674856 >> 172.21.15.43:0/1464674856 conn(0x7fad00076c50 msgr2=0x7fad00077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:13.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.398+0000 7fad06f8f640 1 -- 172.21.15.43:0/1464674856 shutdown_connections 2024-04-21T08:51:13.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.398+0000 7fad06f8f640 1 -- 172.21.15.43:0/1464674856 wait complete. 2024-04-21T08:51:13.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.399+0000 7fad06f8f640 1 Processor -- start 2024-04-21T08:51:13.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.399+0000 7fad06f8f640 1 -- start start 2024-04-21T08:51:13.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.399+0000 7fad06f8f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad001011a0 0x7fad0019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.399+0000 7fad06f8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad00101b40 0x7fad0019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.399+0000 7fad06f8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad00107030 0x7fad001985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.399+0000 7fad06f8f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fad001110e0 con 0x7fad00107030 2024-04-21T08:51:13.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.399+0000 7fad06f8f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fad00110f60 con 0x7fad00101b40 2024-04-21T08:51:13.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.399+0000 7fad06f8f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fad00111260 con 0x7fad001011a0 2024-04-21T08:51:13.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.399+0000 7fad0552c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad00107030 0x7fad001985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.399+0000 7fad0552c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad00107030 0x7fad001985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51534/0 (socket says 172.21.15.43:51534) 2024-04-21T08:51:13.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.399+0000 7fad0552c640 1 -- 172.21.15.43:0/4081012194 learned_addr learned my addr 172.21.15.43:0/4081012194 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:13.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.399+0000 7facf7fff640 1 --2- 172.21.15.43:0/4081012194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad00101b40 0x7fad0019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.399+0000 7fad0552c640 1 -- 172.21.15.43:0/4081012194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad001011a0 msgr2=0x7fad0019e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.399+0000 7fad0552c640 1 --2- 172.21.15.43:0/4081012194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad001011a0 0x7fad0019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.399+0000 7fad0552c640 1 -- 172.21.15.43:0/4081012194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad00101b40 msgr2=0x7fad0019e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.399+0000 7fad0552c640 1 --2- 172.21.15.43:0/4081012194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad00101b40 0x7fad0019e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.399+0000 7fad0552c640 1 -- 172.21.15.43:0/4081012194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad00198d50 con 0x7fad00107030 2024-04-21T08:51:13.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.400+0000 7fad0552c640 1 --2- 172.21.15.43:0/4081012194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad00107030 0x7fad001985d0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7facf80098d0 tx=0x7facf8032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:13.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.400+0000 7facf5ffb640 1 -- 172.21.15.43:0/4081012194 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8041020 con 0x7fad00107030 2024-04-21T08:51:13.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.400+0000 7facf5ffb640 1 -- 172.21.15.43:0/4081012194 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facf8032970 con 0x7fad00107030 2024-04-21T08:51:13.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.400+0000 7fad06f8f640 1 -- 172.21.15.43:0/4081012194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad00198fe0 con 0x7fad00107030 2024-04-21T08:51:13.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.400+0000 7facf5ffb640 1 -- 172.21.15.43:0/4081012194 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8039040 con 0x7fad00107030 2024-04-21T08:51:13.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.400+0000 7fad06f8f640 1 -- 172.21.15.43:0/4081012194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad001a52e0 con 0x7fad00107030 2024-04-21T08:51:13.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.402+0000 7facf5ffb640 1 -- 172.21.15.43:0/4081012194 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7facf8044020 con 0x7fad00107030 2024-04-21T08:51:13.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.403+0000 7fad09b7f740 1 Processor -- start 2024-04-21T08:51:13.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.403+0000 7fad09b7f740 1 -- start start 2024-04-21T08:51:13.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.403+0000 7fad09b7f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be3e28b4d0 0x55be3e28b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.403+0000 7fad09b7f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3e28be80 0x55be3e294430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.403+0000 7fad09b7f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e294a00 0x55be3e296e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.403+0000 7fad09b7f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55be3e1f01f0 con 0x55be3e294a00 2024-04-21T08:51:13.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.403+0000 7fad09b7f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55be3e18d4b0 con 0x55be3e28be80 2024-04-21T08:51:13.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.403+0000 7fad09b7f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55be3e1ec1b0 con 0x55be3e28b4d0 2024-04-21T08:51:13.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.403+0000 7facf5ffb640 1 --2- 172.21.15.43:0/4081012194 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7face80a3da0 0x7face80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.403+0000 7facf7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e294a00 0x55be3e296e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.403+0000 7facf7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e294a00 0x55be3e296e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51542/0 (socket says 172.21.15.43:51542) 2024-04-21T08:51:13.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.403+0000 7facf7fff640 1 -- 172.21.15.43:0/85964851 learned_addr learned my addr 172.21.15.43:0/85964851 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.403+0000 7fad04d2b640 1 --2- 172.21.15.43:0/85964851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3e28be80 0x55be3e294430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.403+0000 7fad0552c640 1 --2- 172.21.15.43:0/4081012194 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7face80a3da0 0x7face80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.403+0000 7facf7fff640 1 --2- 172.21.15.43:0/85964851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be3e28b4d0 0x55be3e28b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.403+0000 7facf5ffb640 1 -- 172.21.15.43:0/4081012194 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7facf80f2940 con 0x7fad00107030 2024-04-21T08:51:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.404+0000 7facf7fff640 1 -- 172.21.15.43:0/85964851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be3e28b4d0 msgr2=0x55be3e28b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.404+0000 7facf7fff640 1 --2- 172.21.15.43:0/85964851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be3e28b4d0 0x55be3e28b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.404+0000 7facf7fff640 1 -- 172.21.15.43:0/85964851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3e28be80 msgr2=0x55be3e294430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.404+0000 7facf7fff640 1 --2- 172.21.15.43:0/85964851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3e28be80 0x55be3e294430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.404+0000 7facf7fff640 1 -- 172.21.15.43:0/85964851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be3e17f1b0 con 0x55be3e294a00 2024-04-21T08:51:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.404+0000 7fad04d2b640 1 --2- 172.21.15.43:0/85964851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3e28be80 0x55be3e294430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.404+0000 7fad0552c640 1 --2- 172.21.15.43:0/4081012194 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7face80a3da0 0x7face80a6260 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7facf80be270 tx=0x7facf80ee040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.404+0000 7facf7fff640 1 --2- 172.21.15.43:0/85964851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e294a00 0x55be3e296e50 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7facf0009460 tx=0x7facf0009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab64489ddc2894ab server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.404+0000 7face6ffd640 1 -- 172.21.15.43:0/85964851 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf00140d0 con 0x55be3e294a00 2024-04-21T08:51:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.404+0000 7face6ffd640 1 -- 172.21.15.43:0/85964851 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facf0014270 con 0x55be3e294a00 2024-04-21T08:51:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.404+0000 7face6ffd640 1 -- 172.21.15.43:0/85964851 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf00047a0 con 0x55be3e294a00 2024-04-21T08:51:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.405+0000 7fad09b7f740 1 -- 172.21.15.43:0/85964851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e294a00 msgr2=0x55be3e296e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.405+0000 7fad09b7f740 1 --2- 172.21.15.43:0/85964851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e294a00 0x55be3e296e50 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7facf0009460 tx=0x7facf0009610 comp rx=0 tx=0).stop 2024-04-21T08:51:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.405+0000 7fad09b7f740 1 -- 172.21.15.43:0/85964851 shutdown_connections 2024-04-21T08:51:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.405+0000 7fad09b7f740 1 --2- 172.21.15.43:0/85964851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e294a00 0x55be3e296e50 unknown :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.405+0000 7fad09b7f740 1 --2- 172.21.15.43:0/85964851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3e28be80 0x55be3e294430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.405+0000 7fad09b7f740 1 --2- 172.21.15.43:0/85964851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be3e28b4d0 0x55be3e28b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.405+0000 7fad09b7f740 1 -- 172.21.15.43:0/85964851 >> 172.21.15.43:0/85964851 conn(0x55be3e285ba0 msgr2=0x55be3e285f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.405+0000 7fad09b7f740 1 -- 172.21.15.43:0/85964851 shutdown_connections 2024-04-21T08:51:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.405+0000 7fad09b7f740 1 -- 172.21.15.43:0/85964851 wait complete. 2024-04-21T08:51:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.405+0000 7fad09b7f740 1 -- start start 2024-04-21T08:51:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.405+0000 7fad09b7f740 1 Processor -- start 2024-04-21T08:51:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.406+0000 7fad09b7f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e28b4d0 0x55be3e294300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.406+0000 7fad09b7f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3e28be80 0x55be3e291690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.406+0000 7fad09b7f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be3e294a00 0x55be3e291c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.406+0000 7fad09b7f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55be3e297910 con 0x55be3e28b4d0 2024-04-21T08:51:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.406+0000 7fad09b7f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55be3e1a9500 con 0x55be3e28be80 2024-04-21T08:51:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.406+0000 7fad09b7f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55be3e297a90 con 0x55be3e294a00 2024-04-21T08:51:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.406+0000 7facf7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e28b4d0 0x55be3e294300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.406+0000 7fad09b7f740 10 client.0 _ll_register_callbacks cb 0x55be3e177b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.406+0000 7facf7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e28b4d0 0x55be3e294300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51544/0 (socket says 172.21.15.43:51544) 2024-04-21T08:51:13.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.406+0000 7facf7fff640 1 -- 172.21.15.43:0/2966211378 learned_addr learned my addr 172.21.15.43:0/2966211378 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:13.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.406+0000 7facf7fff640 1 --2- 172.21.15.43:0/2966211378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be3e294a00 0x55be3e291c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.406+0000 7fad04d2b640 1 --2- 172.21.15.43:0/2966211378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3e28be80 0x55be3e291690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.406+0000 7facf7fff640 1 -- 172.21.15.43:0/2966211378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be3e294a00 msgr2=0x55be3e291c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:13.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.406+0000 7facf7fff640 1 --2- 172.21.15.43:0/2966211378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be3e294a00 0x55be3e291c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.406+0000 7facf7fff640 1 -- 172.21.15.43:0/2966211378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3e28be80 msgr2=0x55be3e291690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:13.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.406+0000 7facf7fff640 1 --2- 172.21.15.43:0/2966211378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3e28be80 0x55be3e291690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.406+0000 7facf7fff640 1 -- 172.21.15.43:0/2966211378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be3df023f0 con 0x55be3e28b4d0 2024-04-21T08:51:13.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.406+0000 7fad04d2b640 1 --2- 172.21.15.43:0/2966211378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3e28be80 0x55be3e291690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:13.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.406+0000 7facf7fff640 1 --2- 172.21.15.43:0/2966211378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e28b4d0 0x55be3e294300 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7facf0006010 tx=0x7facf0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54473a7e4f87011d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:13.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.406+0000 7facdffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:13.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.406+0000 7facdffff640 1 -- 172.21.15.43:0/2966211378 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf00140d0 con 0x55be3e28b4d0 2024-04-21T08:51:13.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.406+0000 7facdffff640 1 -- 172.21.15.43:0/2966211378 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7facf00046c0 con 0x55be3e28b4d0 2024-04-21T08:51:13.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.406+0000 7fad09b7f740 1 -- 172.21.15.43:0/2966211378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facf0014270 con 0x55be3e28b4d0 2024-04-21T08:51:13.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.407+0000 7facdffff640 1 -- 172.21.15.43:0/2966211378 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7facf0034050 con 0x55be3e28b4d0 2024-04-21T08:51:13.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.407+0000 7fad09b7f740 10 client.21342 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:13.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.407+0000 7fad09b7f740 1 -- 172.21.15.43:0/2966211378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55be3dfcc7d0 con 0x55be3e28b4d0 2024-04-21T08:51:13.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.407+0000 7facdffff640 1 -- 172.21.15.43:0/2966211378 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facf0014270 con 0x55be3e28b4d0 2024-04-21T08:51:13.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.407+0000 7facdffff640 1 -- 172.21.15.43:0/2966211378 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7facf001f5f0 con 0x55be3e28b4d0 2024-04-21T08:51:13.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.407+0000 7fad09b7f740 10 client.21342 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:13.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.407+0000 7fad09b7f740 10 client.21342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:13.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.407+0000 7fad09b7f740 20 client.21342 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.407+0000 7fad09b7f740 1 --2- 172.21.15.43:0/2966211378 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55be3e2a4ab0 0x55be3e2a6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.407+0000 7fad09b7f740 4 client.21342 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.407+0000 7fad09b7f740 1 -- 172.21.15.43:0/2966211378 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be3e18f930 con 0x55be3e2a4ab0 2024-04-21T08:51:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.407+0000 7fad04d2b640 1 --2- 172.21.15.43:0/2966211378 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55be3e2a4ab0 0x55be3e2a6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.408+0000 7fad04d2b640 1 --2- 172.21.15.43:0/2966211378 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55be3e2a4ab0 0x55be3e2a6ea0 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47b3c224b6501c32 server_cookie=cceb0dc584016e46 in_seq=0 out_seq=0 2024-04-21T08:51:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.408+0000 7facdffff640 10 client.21342 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.408+0000 7facdffff640 1 -- 172.21.15.43:0/2966211378 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55be3e18f930 con 0x55be3e2a4ab0 2024-04-21T08:51:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.408+0000 7facdffff640 10 client.21342 handle_command_reply: tid=0 2024-04-21T08:51:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.408+0000 7fad09b7f740 1 client.21342 shutdown 2024-04-21T08:51:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.408+0000 7fad09b7f740 1 -- 172.21.15.43:0/2966211378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e28b4d0 msgr2=0x55be3e294300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.408+0000 7fad09b7f740 1 --2- 172.21.15.43:0/2966211378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e28b4d0 0x55be3e294300 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7facf0006010 tx=0x7facf0001e50 comp rx=0 tx=0).stop 2024-04-21T08:51:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.408+0000 7fad09b7f740 1 -- 172.21.15.43:0/2966211378 shutdown_connections 2024-04-21T08:51:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.408+0000 7fad09b7f740 1 --2- 172.21.15.43:0/2966211378 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55be3e2a4ab0 0x55be3e2a6ea0 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.408+0000 7fad09b7f740 1 --2- 172.21.15.43:0/2966211378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be3e294a00 0x55be3e291c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.409+0000 7facdffff640 0 client.21342 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:13.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.409+0000 7fad09b7f740 1 --2- 172.21.15.43:0/2966211378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3e28be80 0x55be3e291690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.409+0000 7fad09b7f740 1 --2- 172.21.15.43:0/2966211378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e28b4d0 0x55be3e294300 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.409+0000 7fad09b7f740 1 -- 172.21.15.43:0/2966211378 >> 172.21.15.43:0/2966211378 conn(0x55be3e285ba0 msgr2=0x55be3e2878f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:13.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.409+0000 7fad09b7f740 1 -- 172.21.15.43:0/2966211378 shutdown_connections 2024-04-21T08:51:13.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.409+0000 7fad09b7f740 1 -- 172.21.15.43:0/2966211378 wait complete. 2024-04-21T08:51:13.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.409+0000 7fad09b7f740 20 client.21342 trim_cache size 0 max 16384 2024-04-21T08:51:13.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.422+0000 7fad09b7f740 1 Processor -- start 2024-04-21T08:51:13.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.422+0000 7fad09b7f740 1 -- start start 2024-04-21T08:51:13.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.422+0000 7fad09b7f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be3e28b4d0 0x55be3e2a0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.422+0000 7fad09b7f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3e28be80 0x55be3e293dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.422+0000 7fad09b7f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e294a00 0x55be3e294300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.422+0000 7fad09b7f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fad001af4e0 con 0x55be3e294a00 2024-04-21T08:51:13.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.422+0000 7fad09b7f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fad001b1020 con 0x55be3e28be80 2024-04-21T08:51:13.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.422+0000 7fad09b7f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fad001afbb0 con 0x55be3e28b4d0 2024-04-21T08:51:13.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.422+0000 7facf7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e294a00 0x55be3e294300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.422+0000 7facf7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e294a00 0x55be3e294300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51546/0 (socket says 172.21.15.43:51546) 2024-04-21T08:51:13.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.422+0000 7facf7fff640 1 -- 172.21.15.43:0/1370488986 learned_addr learned my addr 172.21.15.43:0/1370488986 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:13.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.422+0000 7fad04d2b640 1 --2- 172.21.15.43:0/1370488986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3e28be80 0x55be3e293dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.422+0000 7facf7fff640 1 --2- 172.21.15.43:0/1370488986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be3e28b4d0 0x55be3e2a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.423+0000 7facf7fff640 1 -- 172.21.15.43:0/1370488986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be3e28b4d0 msgr2=0x55be3e2a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:13.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.423+0000 7facf7fff640 1 --2- 172.21.15.43:0/1370488986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be3e28b4d0 0x55be3e2a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.423+0000 7facf7fff640 1 -- 172.21.15.43:0/1370488986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3e28be80 msgr2=0x55be3e293dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:13.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.423+0000 7facf7fff640 1 --2- 172.21.15.43:0/1370488986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3e28be80 0x55be3e293dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.423+0000 7facf7fff640 1 -- 172.21.15.43:0/1370488986 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be3e28c990 con 0x55be3e294a00 2024-04-21T08:51:13.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.423+0000 7facf7fff640 1 --2- 172.21.15.43:0/1370488986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e294a00 0x55be3e294300 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7facf001fb60 tx=0x7facf0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e3b94fb9a7dc4f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:13.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.423+0000 7face67fc640 1 -- 172.21.15.43:0/1370488986 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf001fbe0 con 0x55be3e294a00 2024-04-21T08:51:13.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.423+0000 7face67fc640 1 -- 172.21.15.43:0/1370488986 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facf00140d0 con 0x55be3e294a00 2024-04-21T08:51:13.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.423+0000 7face67fc640 1 -- 172.21.15.43:0/1370488986 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf00035f0 con 0x55be3e294a00 2024-04-21T08:51:13.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.423+0000 7fad09b7f740 1 -- 172.21.15.43:0/1370488986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e294a00 msgr2=0x55be3e294300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.424+0000 7fad09b7f740 1 --2- 172.21.15.43:0/1370488986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e294a00 0x55be3e294300 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7facf001fb60 tx=0x7facf0000bc0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.424+0000 7fad09b7f740 1 -- 172.21.15.43:0/1370488986 shutdown_connections 2024-04-21T08:51:13.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.424+0000 7fad09b7f740 1 --2- 172.21.15.43:0/1370488986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e294a00 0x55be3e294300 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.424+0000 7fad09b7f740 1 --2- 172.21.15.43:0/1370488986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3e28be80 0x55be3e293dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.424+0000 7fad09b7f740 1 --2- 172.21.15.43:0/1370488986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be3e28b4d0 0x55be3e2a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.424+0000 7fad09b7f740 1 -- 172.21.15.43:0/1370488986 >> 172.21.15.43:0/1370488986 conn(0x55be3e285ba0 msgr2=0x55be3e284370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:13.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.424+0000 7fad09b7f740 1 -- 172.21.15.43:0/1370488986 shutdown_connections 2024-04-21T08:51:13.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.424+0000 7fad09b7f740 1 -- 172.21.15.43:0/1370488986 wait complete. 2024-04-21T08:51:13.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.424+0000 7fad09b7f740 1 -- start start 2024-04-21T08:51:13.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.424+0000 7fad09b7f740 1 Processor -- start 2024-04-21T08:51:13.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.424+0000 7fad09b7f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be3e28b4d0 0x55be3e29cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.424+0000 7fad09b7f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e28be80 0x55be3e29d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.424+0000 7fad09b7f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3e294a00 0x55be3e28da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.424+0000 7fad09b7f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fad001b0950 con 0x55be3e28be80 2024-04-21T08:51:13.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.424+0000 7fad09b7f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fad001b0280 con 0x55be3e294a00 2024-04-21T08:51:13.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.424+0000 7fad09b7f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fad001ad4e0 con 0x55be3e28b4d0 2024-04-21T08:51:13.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.424+0000 7fad09b7f740 10 client.0 _ll_register_callbacks cb 0x7facf0014750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:13.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.424+0000 7fad04d2b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e28be80 0x55be3e29d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.424+0000 7fad04d2b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e28be80 0x55be3e29d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51558/0 (socket says 172.21.15.43:51558) 2024-04-21T08:51:13.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.424+0000 7fad04d2b640 1 -- 172.21.15.43:0/1115866517 learned_addr learned my addr 172.21.15.43:0/1115866517 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:13.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.425+0000 7facf7fff640 1 --2- 172.21.15.43:0/1115866517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3e294a00 0x55be3e28da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.425+0000 7facf7fff640 1 --2- 172.21.15.43:0/1115866517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be3e28b4d0 0x55be3e29cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.425+0000 7fad04d2b640 1 -- 172.21.15.43:0/1115866517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be3e28b4d0 msgr2=0x55be3e29cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:13.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.425+0000 7fad04d2b640 1 --2- 172.21.15.43:0/1115866517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be3e28b4d0 0x55be3e29cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.425+0000 7fad04d2b640 1 -- 172.21.15.43:0/1115866517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3e294a00 msgr2=0x55be3e28da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:13.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.425+0000 7fad04d2b640 1 --2- 172.21.15.43:0/1115866517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3e294a00 0x55be3e28da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.425+0000 7fad04d2b640 1 -- 172.21.15.43:0/1115866517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be3e28e090 con 0x55be3e28be80 2024-04-21T08:51:13.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.425+0000 7fad04d2b640 1 --2- 172.21.15.43:0/1115866517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e28be80 0x55be3e29d220 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7facfc005d20 tx=0x7facfc00bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7509b873ce71b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:13.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.425+0000 7facdffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:13.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.425+0000 7facdffff640 1 -- 172.21.15.43:0/1115866517 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc0035f0 con 0x55be3e28be80 2024-04-21T08:51:13.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.425+0000 7facdffff640 1 -- 172.21.15.43:0/1115866517 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7facfc003790 con 0x55be3e28be80 2024-04-21T08:51:13.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.425+0000 7fad09b7f740 1 -- 172.21.15.43:0/1115866517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facf00140d0 con 0x55be3e28be80 2024-04-21T08:51:13.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.425+0000 7facdffff640 1 -- 172.21.15.43:0/1115866517 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7facfc02c050 con 0x55be3e28be80 2024-04-21T08:51:13.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.425+0000 7fad09b7f740 10 client.21348 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:13.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.425+0000 7fad09b7f740 1 -- 172.21.15.43:0/1115866517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55be3e30a780 con 0x55be3e28be80 2024-04-21T08:51:13.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.425+0000 7facdffff640 1 -- 172.21.15.43:0/1115866517 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facf00140d0 con 0x55be3e28be80 2024-04-21T08:51:13.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.426+0000 7facdffff640 1 -- 172.21.15.43:0/1115866517 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7facfc021020 con 0x55be3e28be80 2024-04-21T08:51:13.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.426+0000 7fad09b7f740 10 client.21348 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:13.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.426+0000 7fad09b7f740 10 client.21348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:13.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.426+0000 7fad09b7f740 20 client.21348 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:13.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.426+0000 7fad09b7f740 1 --2- 172.21.15.43:0/1115866517 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55be3e30aed0 0x55be3e30d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:13.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.426+0000 7fad09b7f740 4 client.21348 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:51:13.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.426+0000 7fad09b7f740 1 -- 172.21.15.43:0/1115866517 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be3e1fdc10 con 0x55be3e30aed0 2024-04-21T08:51:13.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.426+0000 7facf7fff640 1 --2- 172.21.15.43:0/1115866517 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55be3e30aed0 0x55be3e30d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:13.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.426+0000 7facf7fff640 1 --2- 172.21.15.43:0/1115866517 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55be3e30aed0 0x55be3e30d2f0 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54c7cbacbefbc492 server_cookie=b127b967895e5153 in_seq=0 out_seq=0 2024-04-21T08:51:13.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.426+0000 7facdffff640 10 client.21348 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:13.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.426+0000 7facdffff640 1 -- 172.21.15.43:0/1115866517 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be3e1fdc10 con 0x55be3e30aed0 2024-04-21T08:51:13.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.426+0000 7facdffff640 10 client.21348 handle_command_reply: tid=0 2024-04-21T08:51:13.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.426+0000 7fad09b7f740 1 client.21348 shutdown 2024-04-21T08:51:13.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.427+0000 7fad09b7f740 1 -- 172.21.15.43:0/1115866517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e28be80 msgr2=0x55be3e29d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.427+0000 7fad09b7f740 1 --2- 172.21.15.43:0/1115866517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e28be80 0x55be3e29d220 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7facfc005d20 tx=0x7facfc00bbd0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.427+0000 7fad09b7f740 1 -- 172.21.15.43:0/1115866517 shutdown_connections 2024-04-21T08:51:13.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.427+0000 7fad09b7f740 1 --2- 172.21.15.43:0/1115866517 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55be3e30aed0 0x55be3e30d2f0 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.427+0000 7fad09b7f740 1 --2- 172.21.15.43:0/1115866517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3e294a00 0x55be3e28da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.427+0000 7facdffff640 0 client.21348 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:13.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.427+0000 7fad09b7f740 1 --2- 172.21.15.43:0/1115866517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be3e28be80 0x55be3e29d220 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.427+0000 7fad09b7f740 1 --2- 172.21.15.43:0/1115866517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be3e28b4d0 0x55be3e29cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.427+0000 7fad09b7f740 1 -- 172.21.15.43:0/1115866517 >> 172.21.15.43:0/1115866517 conn(0x55be3e285ba0 msgr2=0x55be3e284c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:13.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.427+0000 7fad09b7f740 1 -- 172.21.15.43:0/1115866517 shutdown_connections 2024-04-21T08:51:13.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.427+0000 7fad09b7f740 1 -- 172.21.15.43:0/1115866517 wait complete. 2024-04-21T08:51:13.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.427+0000 7fad09b7f740 20 client.21348 trim_cache size 0 max 16384 2024-04-21T08:51:13.436 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:51:13.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.427+0000 7fad06f8f640 1 -- 172.21.15.43:0/4081012194 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7face80a3da0 msgr2=0x7face80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.427+0000 7fad06f8f640 1 --2- 172.21.15.43:0/4081012194 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7face80a3da0 0x7face80a6260 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7facf80be270 tx=0x7facf80ee040 comp rx=0 tx=0).stop 2024-04-21T08:51:13.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.427+0000 7fad06f8f640 1 -- 172.21.15.43:0/4081012194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad00107030 msgr2=0x7fad001985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:13.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.427+0000 7fad06f8f640 1 --2- 172.21.15.43:0/4081012194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad00107030 0x7fad001985d0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7facf80098d0 tx=0x7facf8032420 comp rx=0 tx=0).stop 2024-04-21T08:51:13.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.427+0000 7fad06f8f640 1 -- 172.21.15.43:0/4081012194 shutdown_connections 2024-04-21T08:51:13.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.427+0000 7fad06f8f640 1 --2- 172.21.15.43:0/4081012194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad00107030 0x7fad001985d0 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.427+0000 7fad06f8f640 1 --2- 172.21.15.43:0/4081012194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad00101b40 0x7fad0019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.427+0000 7fad06f8f640 1 --2- 172.21.15.43:0/4081012194 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7face80a3da0 0x7face80a6260 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.427+0000 7fad06f8f640 1 --2- 172.21.15.43:0/4081012194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad001011a0 0x7fad0019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:13.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.427+0000 7fad06f8f640 1 -- 172.21.15.43:0/4081012194 >> 172.21.15.43:0/4081012194 conn(0x7fad00076c50 msgr2=0x7fad00100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:13.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.428+0000 7fad06f8f640 1 -- 172.21.15.43:0/4081012194 shutdown_connections 2024-04-21T08:51:13.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:13.428+0000 7fad06f8f640 1 -- 172.21.15.43:0/4081012194 wait complete. 2024-04-21T08:51:13.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:13 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:51:13.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:13 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:51:13.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:13 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:51:13.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:13 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3979604342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:51:13.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:13 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:51:13.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:13 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:51:13.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:13 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:51:13.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:13 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3979604342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:51:13.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:13 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:51:13.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:13 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:51:13.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:13 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:51:13.795 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:13 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3979604342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:51:14.440 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:51:14.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:51:14.440 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:51:14.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.561+0000 7fa2d913d640 1 -- 172.21.15.43:0/419291682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d4106160 msgr2=0x7fa2d410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:14.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.561+0000 7fa2d913d640 1 --2- 172.21.15.43:0/419291682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d4106160 0x7fa2d410dd30 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009ca0 tx=0x7fa2cc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.561+0000 7fa2d913d640 1 -- 172.21.15.43:0/419291682 shutdown_connections 2024-04-21T08:51:14.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.561+0000 7fa2d913d640 1 --2- 172.21.15.43:0/419291682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d4106160 0x7fa2d410dd30 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.562+0000 7fa2d913d640 1 --2- 172.21.15.43:0/419291682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2d4101a40 0x7fa2d4105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.562+0000 7fa2d913d640 1 --2- 172.21.15.43:0/419291682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2d4101090 0x7fa2d4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.562+0000 7fa2d913d640 1 -- 172.21.15.43:0/419291682 >> 172.21.15.43:0/419291682 conn(0x7fa2d4077f50 msgr2=0x7fa2d4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:14.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.562+0000 7fa2d913d640 1 -- 172.21.15.43:0/419291682 shutdown_connections 2024-04-21T08:51:14.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.562+0000 7fa2d913d640 1 -- 172.21.15.43:0/419291682 wait complete. 2024-04-21T08:51:14.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.562+0000 7fa2d913d640 1 Processor -- start 2024-04-21T08:51:14.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.563+0000 7fa2d913d640 1 -- start start 2024-04-21T08:51:14.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.563+0000 7fa2d913d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2d4101090 0x7fa2d4102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.563+0000 7fa2d913d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2d4106160 0x7fa2d4103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.563+0000 7fa2d913d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d41035a0 0x7fa2d41a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.563+0000 7fa2d913d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41110b0 con 0x7fa2d41035a0 2024-04-21T08:51:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.563+0000 7fa2d913d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d4110f30 con 0x7fa2d4101090 2024-04-21T08:51:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.563+0000 7fa2d913d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d4111230 con 0x7fa2d4106160 2024-04-21T08:51:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.563+0000 7fa2d359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d41035a0 0x7fa2d41a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.563+0000 7fa2d359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d41035a0 0x7fa2d41a68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51576/0 (socket says 172.21.15.43:51576) 2024-04-21T08:51:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.563+0000 7fa2d359e640 1 -- 172.21.15.43:0/3539994134 learned_addr learned my addr 172.21.15.43:0/3539994134 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.563+0000 7fa2d2d9d640 1 --2- 172.21.15.43:0/3539994134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2d4101090 0x7fa2d4102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.563+0000 7fa2d259c640 1 --2- 172.21.15.43:0/3539994134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2d4106160 0x7fa2d4103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.563+0000 7fa2d359e640 1 -- 172.21.15.43:0/3539994134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2d4106160 msgr2=0x7fa2d4103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.563+0000 7fa2d359e640 1 --2- 172.21.15.43:0/3539994134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2d4106160 0x7fa2d4103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.563+0000 7fa2d359e640 1 -- 172.21.15.43:0/3539994134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2d4101090 msgr2=0x7fa2d4102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.563+0000 7fa2d359e640 1 --2- 172.21.15.43:0/3539994134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2d4101090 0x7fa2d4102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.563+0000 7fa2d359e640 1 -- 172.21.15.43:0/3539994134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d41a6ff0 con 0x7fa2d41035a0 2024-04-21T08:51:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.563+0000 7fa2d359e640 1 --2- 172.21.15.43:0/3539994134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d41035a0 0x7fa2d41a68f0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc005e30 tx=0x7fa2cc034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:14.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.564+0000 7fa2bbfff640 1 -- 172.21.15.43:0/3539994134 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc039040 con 0x7fa2d41035a0 2024-04-21T08:51:14.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.564+0000 7fa2bbfff640 1 -- 172.21.15.43:0/3539994134 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2cc044070 con 0x7fa2d41035a0 2024-04-21T08:51:14.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.564+0000 7fa2d913d640 1 -- 172.21.15.43:0/3539994134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d41a7280 con 0x7fa2d41035a0 2024-04-21T08:51:14.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.564+0000 7fa2bbfff640 1 -- 172.21.15.43:0/3539994134 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc003e60 con 0x7fa2d41035a0 2024-04-21T08:51:14.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.564+0000 7fa2d913d640 1 -- 172.21.15.43:0/3539994134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d41a7770 con 0x7fa2d41035a0 2024-04-21T08:51:14.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.566+0000 7fa2bbfff640 1 -- 172.21.15.43:0/3539994134 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa2cc0324c0 con 0x7fa2d41035a0 2024-04-21T08:51:14.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.566+0000 7fa2dbe07740 1 Processor -- start 2024-04-21T08:51:14.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.566+0000 7fa2dbe07740 1 -- start start 2024-04-21T08:51:14.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.567+0000 7fa2dbe07740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb4780d8e0 0x55cb4780dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.567+0000 7fa2dbe07740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb4780e290 0x55cb47816840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.567+0000 7fa2dbe07740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb47816e10 0x55cb47819260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.567+0000 7fa2dbe07740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cb47771f00 con 0x55cb4780e290 2024-04-21T08:51:14.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.567+0000 7fa2bbfff640 1 --2- 172.21.15.43:0/3539994134 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa2bc0a3d50 0x7fa2bc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.567+0000 7fa2dbe07740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cb4770f590 con 0x55cb47816e10 2024-04-21T08:51:14.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.567+0000 7fa2dbe07740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55cb4776dec0 con 0x55cb4780d8e0 2024-04-21T08:51:14.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.567+0000 7fa2d2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb4780e290 0x55cb47816840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.567+0000 7fa2d359e640 1 --2- 172.21.15.43:0/3539994134 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa2bc0a3d50 0x7fa2bc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.567+0000 7fa2d2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb4780e290 0x55cb47816840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51584/0 (socket says 172.21.15.43:51584) 2024-04-21T08:51:14.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.567+0000 7fa2d2d9d640 1 -- 172.21.15.43:0/2240450055 learned_addr learned my addr 172.21.15.43:0/2240450055 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:14.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.567+0000 7fa2d259c640 1 --2- 172.21.15.43:0/2240450055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb4780d8e0 0x55cb4780dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.567+0000 7fa2d259c640 1 --2- 172.21.15.43:0/2240450055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb47816e10 0x55cb47819260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.567+0000 7fa2bbfff640 1 -- 172.21.15.43:0/3539994134 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fa2cc0f2cc0 con 0x7fa2d41035a0 2024-04-21T08:51:14.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.567+0000 7fa2d2d9d640 1 -- 172.21.15.43:0/2240450055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb4780d8e0 msgr2=0x55cb4780dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:14.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.567+0000 7fa2d2d9d640 1 --2- 172.21.15.43:0/2240450055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb4780d8e0 0x55cb4780dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.567+0000 7fa2d2d9d640 1 -- 172.21.15.43:0/2240450055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb47816e10 msgr2=0x55cb47819260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:14.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.567+0000 7fa2d2d9d640 1 --2- 172.21.15.43:0/2240450055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb47816e10 0x55cb47819260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.567+0000 7fa2d2d9d640 1 -- 172.21.15.43:0/2240450055 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb4754c7e0 con 0x55cb4780e290 2024-04-21T08:51:14.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.567+0000 7fa2d359e640 1 --2- 172.21.15.43:0/3539994134 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa2bc0a3d50 0x7fa2bc0a6210 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0bb320 tx=0x7fa2cc091440 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:14.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.567+0000 7fa2d2d9d640 1 --2- 172.21.15.43:0/2240450055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb4780e290 0x55cb47816840 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400bd10 tx=0x7fa2c40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95953d6f9b9635b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:14.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.568+0000 7fa2b97fa640 1 -- 172.21.15.43:0/2240450055 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4004030 con 0x55cb4780e290 2024-04-21T08:51:14.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.568+0000 7fa2b97fa640 1 -- 172.21.15.43:0/2240450055 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2c40035f0 con 0x55cb4780e290 2024-04-21T08:51:14.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.568+0000 7fa2b97fa640 1 -- 172.21.15.43:0/2240450055 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c40116a0 con 0x55cb4780e290 2024-04-21T08:51:14.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.568+0000 7fa2dbe07740 1 -- 172.21.15.43:0/2240450055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb4780e290 msgr2=0x55cb47816840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:14.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.568+0000 7fa2dbe07740 1 --2- 172.21.15.43:0/2240450055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb4780e290 0x55cb47816840 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400bd10 tx=0x7fa2c40107a0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.568+0000 7fa2dbe07740 1 -- 172.21.15.43:0/2240450055 shutdown_connections 2024-04-21T08:51:14.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.568+0000 7fa2dbe07740 1 --2- 172.21.15.43:0/2240450055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb47816e10 0x55cb47819260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.568+0000 7fa2dbe07740 1 --2- 172.21.15.43:0/2240450055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb4780e290 0x55cb47816840 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.568+0000 7fa2dbe07740 1 --2- 172.21.15.43:0/2240450055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb4780d8e0 0x55cb4780dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.568+0000 7fa2dbe07740 1 -- 172.21.15.43:0/2240450055 >> 172.21.15.43:0/2240450055 conn(0x55cb47807fb0 msgr2=0x55cb47808390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:14.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.568+0000 7fa2dbe07740 1 -- 172.21.15.43:0/2240450055 shutdown_connections 2024-04-21T08:51:14.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.568+0000 7fa2dbe07740 1 -- 172.21.15.43:0/2240450055 wait complete. 2024-04-21T08:51:14.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.569+0000 7fa2dbe07740 1 -- start start 2024-04-21T08:51:14.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.569+0000 7fa2dbe07740 1 Processor -- start 2024-04-21T08:51:14.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.569+0000 7fa2dbe07740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb4780d8e0 0x55cb47813e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.569+0000 7fa2dbe07740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb4780e290 0x55cb478143a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.569+0000 7fa2dbe07740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb47816e10 0x55cb478148e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.569+0000 7fa2dbe07740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cb47819d80 con 0x55cb47816e10 2024-04-21T08:51:14.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.569+0000 7fa2dbe07740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cb4772b580 con 0x55cb4780e290 2024-04-21T08:51:14.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.569+0000 7fa2dbe07740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55cb47819f00 con 0x55cb4780d8e0 2024-04-21T08:51:14.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.569+0000 7fa2dbe07740 10 client.0 _ll_register_callbacks cb 0x55cb47731650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:14.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.569+0000 7fa2d259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb47816e10 0x55cb478148e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.569+0000 7fa2d259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb47816e10 0x55cb478148e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51594/0 (socket says 172.21.15.43:51594) 2024-04-21T08:51:14.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.569+0000 7fa2d259c640 1 -- 172.21.15.43:0/3225204718 learned_addr learned my addr 172.21.15.43:0/3225204718 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:14.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.569+0000 7fa2d259c640 1 --2- 172.21.15.43:0/3225204718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb4780d8e0 0x55cb47813e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.569+0000 7fa2d2d9d640 1 --2- 172.21.15.43:0/3225204718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb4780e290 0x55cb478143a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.570+0000 7fa2d259c640 1 -- 172.21.15.43:0/3225204718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb4780d8e0 msgr2=0x55cb47813e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:14.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.570+0000 7fa2d259c640 1 --2- 172.21.15.43:0/3225204718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb4780d8e0 0x55cb47813e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.570+0000 7fa2d259c640 1 -- 172.21.15.43:0/3225204718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb4780e290 msgr2=0x55cb478143a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:14.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.570+0000 7fa2d259c640 1 --2- 172.21.15.43:0/3225204718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb4780e290 0x55cb478143a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.570+0000 7fa2d259c640 1 -- 172.21.15.43:0/3225204718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb475acf70 con 0x55cb47816e10 2024-04-21T08:51:14.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.570+0000 7fa2d2d9d640 1 --2- 172.21.15.43:0/3225204718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb4780e290 0x55cb478143a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:14.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.570+0000 7fa2d259c640 1 --2- 172.21.15.43:0/3225204718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb47816e10 0x55cb478148e0 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7fa2d406b6e0 tx=0x7fa2c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424d8728d69415d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:14.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.570+0000 7fa2b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:14.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.570+0000 7fa2b6ff5640 1 -- 172.21.15.43:0/3225204718 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8004020 con 0x55cb47816e10 2024-04-21T08:51:14.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.570+0000 7fa2dbe07740 1 -- 172.21.15.43:0/3225204718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2c40035f0 con 0x55cb47816e10 2024-04-21T08:51:14.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.570+0000 7fa2b6ff5640 1 -- 172.21.15.43:0/3225204718 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fa2c80041c0 con 0x55cb47816e10 2024-04-21T08:51:14.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.570+0000 7fa2b6ff5640 1 -- 172.21.15.43:0/3225204718 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa2c80043c0 con 0x55cb47816e10 2024-04-21T08:51:14.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.570+0000 7fa2dbe07740 10 client.21360 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:14.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.570+0000 7fa2dbe07740 1 -- 172.21.15.43:0/3225204718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cb47444170 con 0x55cb47816e10 2024-04-21T08:51:14.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.570+0000 7fa2b6ff5640 1 -- 172.21.15.43:0/3225204718 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2c40035f0 con 0x55cb47816e10 2024-04-21T08:51:14.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.571+0000 7fa2b6ff5640 1 -- 172.21.15.43:0/3225204718 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa2c8004a40 con 0x55cb47816e10 2024-04-21T08:51:14.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.571+0000 7fa2dbe07740 10 client.21360 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:14.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.571+0000 7fa2dbe07740 10 client.21360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:14.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.571+0000 7fa2dbe07740 20 client.21360 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:14.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.571+0000 7fa2dbe07740 1 --2- 172.21.15.43:0/3225204718 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cb478273d0 0x55cb47829820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.571+0000 7fa2dbe07740 4 client.21360 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:14.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.571+0000 7fa2dbe07740 1 -- 172.21.15.43:0/3225204718 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb4777f920 con 0x55cb478273d0 2024-04-21T08:51:14.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.571+0000 7fa2d2d9d640 1 --2- 172.21.15.43:0/3225204718 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cb478273d0 0x55cb47829820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.571+0000 7fa2b6ff5640 10 client.21360 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:14.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.571+0000 7fa2d2d9d640 1 --2- 172.21.15.43:0/3225204718 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cb478273d0 0x55cb47829820 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f05eded6077a037 server_cookie=7cf4e732711b7484 in_seq=0 out_seq=0 2024-04-21T08:51:14.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.572+0000 7fa2b6ff5640 1 -- 172.21.15.43:0/3225204718 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb4777f920 con 0x55cb478273d0 2024-04-21T08:51:14.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.572+0000 7fa2b6ff5640 10 client.21360 handle_command_reply: tid=0 2024-04-21T08:51:14.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.572+0000 7fa2dbe07740 1 client.21360 shutdown 2024-04-21T08:51:14.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.572+0000 7fa2dbe07740 1 -- 172.21.15.43:0/3225204718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb47816e10 msgr2=0x55cb478148e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:14.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.572+0000 7fa2dbe07740 1 --2- 172.21.15.43:0/3225204718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb47816e10 0x55cb478148e0 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7fa2d406b6e0 tx=0x7fa2c80046c0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.572+0000 7fa2dbe07740 1 -- 172.21.15.43:0/3225204718 shutdown_connections 2024-04-21T08:51:14.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.572+0000 7fa2dbe07740 1 --2- 172.21.15.43:0/3225204718 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cb478273d0 0x55cb47829820 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.572+0000 7fa2dbe07740 1 --2- 172.21.15.43:0/3225204718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb47816e10 0x55cb478148e0 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.572+0000 7fa2b6ff5640 0 client.21360 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:14.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.572+0000 7fa2dbe07740 1 --2- 172.21.15.43:0/3225204718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb4780e290 0x55cb478143a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.572+0000 7fa2dbe07740 1 --2- 172.21.15.43:0/3225204718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb4780d8e0 0x55cb47813e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.572+0000 7fa2dbe07740 1 -- 172.21.15.43:0/3225204718 >> 172.21.15.43:0/3225204718 conn(0x55cb47807fb0 msgr2=0x55cb47809d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:14.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.572+0000 7fa2dbe07740 1 -- 172.21.15.43:0/3225204718 shutdown_connections 2024-04-21T08:51:14.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.572+0000 7fa2dbe07740 1 -- 172.21.15.43:0/3225204718 wait complete. 2024-04-21T08:51:14.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.572+0000 7fa2dbe07740 20 client.21360 trim_cache size 0 max 16384 2024-04-21T08:51:14.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.590+0000 7fa2dbe07740 1 Processor -- start 2024-04-21T08:51:14.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.590+0000 7fa2dbe07740 1 -- start start 2024-04-21T08:51:14.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.590+0000 7fa2dbe07740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb4780d8e0 0x55cb47816590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.590+0000 7fa2dbe07740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb4780e290 0x55cb47822c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.590+0000 7fa2dbe07740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb47816e10 0x55cb47806e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.590+0000 7fa2dbe07740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41b17a0 con 0x55cb47816e10 2024-04-21T08:51:14.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.590+0000 7fa2dbe07740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41b32e0 con 0x55cb4780d8e0 2024-04-21T08:51:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.590+0000 7fa2dbe07740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41b1e70 con 0x55cb4780e290 2024-04-21T08:51:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.591+0000 7fa2d259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb47816e10 0x55cb47806e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.591+0000 7fa2d259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb47816e10 0x55cb47806e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51596/0 (socket says 172.21.15.43:51596) 2024-04-21T08:51:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.591+0000 7fa2d259c640 1 -- 172.21.15.43:0/3817523138 learned_addr learned my addr 172.21.15.43:0/3817523138 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.591+0000 7fa2d259c640 1 --2- 172.21.15.43:0/3817523138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb4780d8e0 0x55cb47816590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.591+0000 7fa2d259c640 1 -- 172.21.15.43:0/3817523138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb4780e290 msgr2=0x55cb47822c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.591+0000 7fa2d259c640 1 --2- 172.21.15.43:0/3817523138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb4780e290 0x55cb47822c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.591+0000 7fa2d259c640 1 -- 172.21.15.43:0/3817523138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb4780d8e0 msgr2=0x55cb47816590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.591+0000 7fa2d259c640 1 --2- 172.21.15.43:0/3817523138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb4780d8e0 0x55cb47816590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.591+0000 7fa2d259c640 1 -- 172.21.15.43:0/3817523138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb478123b0 con 0x55cb47816e10 2024-04-21T08:51:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.591+0000 7fa2d259c640 1 --2- 172.21.15.43:0/3817523138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb47816e10 0x55cb47806e90 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7fa2c80129f0 tx=0x7fa2c8014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977cbc159e4fc5a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.591+0000 7fa2b8ff9640 1 -- 172.21.15.43:0/3817523138 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8012700 con 0x55cb47816e10 2024-04-21T08:51:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.591+0000 7fa2b8ff9640 1 -- 172.21.15.43:0/3817523138 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2c8004be0 con 0x55cb47816e10 2024-04-21T08:51:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.591+0000 7fa2b8ff9640 1 -- 172.21.15.43:0/3817523138 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8005200 con 0x55cb47816e10 2024-04-21T08:51:14.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.592+0000 7fa2dbe07740 1 -- 172.21.15.43:0/3817523138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb47816e10 msgr2=0x55cb47806e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:14.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.592+0000 7fa2dbe07740 1 --2- 172.21.15.43:0/3817523138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb47816e10 0x55cb47806e90 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7fa2c80129f0 tx=0x7fa2c8014040 comp rx=0 tx=0).stop 2024-04-21T08:51:14.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.592+0000 7fa2dbe07740 1 -- 172.21.15.43:0/3817523138 shutdown_connections 2024-04-21T08:51:14.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.592+0000 7fa2dbe07740 1 --2- 172.21.15.43:0/3817523138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb47816e10 0x55cb47806e90 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.592+0000 7fa2dbe07740 1 --2- 172.21.15.43:0/3817523138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb4780e290 0x55cb47822c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.592+0000 7fa2dbe07740 1 --2- 172.21.15.43:0/3817523138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb4780d8e0 0x55cb47816590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.592+0000 7fa2dbe07740 1 -- 172.21.15.43:0/3817523138 >> 172.21.15.43:0/3817523138 conn(0x55cb47807fb0 msgr2=0x55cb47806780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:14.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.592+0000 7fa2dbe07740 1 -- 172.21.15.43:0/3817523138 shutdown_connections 2024-04-21T08:51:14.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.592+0000 7fa2dbe07740 1 -- 172.21.15.43:0/3817523138 wait complete. 2024-04-21T08:51:14.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.592+0000 7fa2dbe07740 1 -- start start 2024-04-21T08:51:14.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.592+0000 7fa2dbe07740 1 Processor -- start 2024-04-21T08:51:14.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.592+0000 7fa2dbe07740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb4780d8e0 0x55cb4781f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.592+0000 7fa2dbe07740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb4780e290 0x55cb4781a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.592+0000 7fa2dbe07740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb47816e10 0x55cb4781afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.592+0000 7fa2dbe07740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41b2c10 con 0x55cb47816e10 2024-04-21T08:51:14.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.592+0000 7fa2dbe07740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41b2540 con 0x55cb4780d8e0 2024-04-21T08:51:14.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.592+0000 7fa2dbe07740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41af7a0 con 0x55cb4780e290 2024-04-21T08:51:14.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.592+0000 7fa2dbe07740 10 client.0 _ll_register_callbacks cb 0x7fa2c80124e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:14.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.592+0000 7fa2d259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb47816e10 0x55cb4781afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.593+0000 7fa2d259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb47816e10 0x55cb4781afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51606/0 (socket says 172.21.15.43:51606) 2024-04-21T08:51:14.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.593+0000 7fa2d259c640 1 -- 172.21.15.43:0/829459204 learned_addr learned my addr 172.21.15.43:0/829459204 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:14.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.593+0000 7fa2d259c640 1 --2- 172.21.15.43:0/829459204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb4780d8e0 0x55cb4781f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.593+0000 7fa2d259c640 1 -- 172.21.15.43:0/829459204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb4780e290 msgr2=0x55cb4781a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:14.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.593+0000 7fa2d259c640 1 --2- 172.21.15.43:0/829459204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb4780e290 0x55cb4781a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.593+0000 7fa2d259c640 1 -- 172.21.15.43:0/829459204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb4780d8e0 msgr2=0x55cb4781f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:14.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.593+0000 7fa2d259c640 1 --2- 172.21.15.43:0/829459204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb4780d8e0 0x55cb4781f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.593+0000 7fa2d259c640 1 -- 172.21.15.43:0/829459204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb4781b7f0 con 0x55cb47816e10 2024-04-21T08:51:14.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.593+0000 7fa2d259c640 1 --2- 172.21.15.43:0/829459204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb47816e10 0x55cb4781afc0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8004e10 tx=0x7fa2c80122c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eefd04db0ea33f95 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:14.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.593+0000 7fa2b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:14.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.593+0000 7fa2b6ff5640 1 -- 172.21.15.43:0/829459204 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8034030 con 0x55cb47816e10 2024-04-21T08:51:14.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.593+0000 7fa2b6ff5640 1 -- 172.21.15.43:0/829459204 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fa2c80046c0 con 0x55cb47816e10 2024-04-21T08:51:14.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.593+0000 7fa2dbe07740 1 -- 172.21.15.43:0/829459204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2c8004be0 con 0x55cb47816e10 2024-04-21T08:51:14.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.593+0000 7fa2b6ff5640 1 -- 172.21.15.43:0/829459204 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa2c80120d0 con 0x55cb47816e10 2024-04-21T08:51:14.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.593+0000 7fa2dbe07740 10 client.21366 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:14.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.593+0000 7fa2dbe07740 1 -- 172.21.15.43:0/829459204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cb4788dad0 con 0x55cb47816e10 2024-04-21T08:51:14.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.593+0000 7fa2b6ff5640 1 -- 172.21.15.43:0/829459204 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2c8004be0 con 0x55cb47816e10 2024-04-21T08:51:14.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.594+0000 7fa2b6ff5640 1 -- 172.21.15.43:0/829459204 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa2c803a060 con 0x55cb47816e10 2024-04-21T08:51:14.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.594+0000 7fa2dbe07740 10 client.21366 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:14.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.594+0000 7fa2dbe07740 10 client.21366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:14.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.594+0000 7fa2dbe07740 20 client.21366 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:14.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.594+0000 7fa2dbe07740 1 --2- 172.21.15.43:0/829459204 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cb4788e0d0 0x55cb478904c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.594+0000 7fa2dbe07740 4 client.21366 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:51:14.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.594+0000 7fa2dbe07740 1 -- 172.21.15.43:0/829459204 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cb47779790 con 0x55cb4788e0d0 2024-04-21T08:51:14.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.594+0000 7fa2d2d9d640 1 --2- 172.21.15.43:0/829459204 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cb4788e0d0 0x55cb478904c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.594+0000 7fa2d2d9d640 1 --2- 172.21.15.43:0/829459204 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cb4788e0d0 0x55cb478904c0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=185e97f37319e189 server_cookie=3c3d94336ca3f27a in_seq=0 out_seq=0 2024-04-21T08:51:14.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.594+0000 7fa2b6ff5640 10 client.21366 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:14.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.596+0000 7fa2b6ff5640 1 -- 172.21.15.43:0/829459204 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cb47779790 con 0x55cb4788e0d0 2024-04-21T08:51:14.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.596+0000 7fa2b6ff5640 10 client.21366 handle_command_reply: tid=0 2024-04-21T08:51:14.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.596+0000 7fa2dbe07740 1 client.21366 shutdown 2024-04-21T08:51:14.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.596+0000 7fa2dbe07740 1 -- 172.21.15.43:0/829459204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb47816e10 msgr2=0x55cb4781afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:14.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.596+0000 7fa2dbe07740 1 --2- 172.21.15.43:0/829459204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb47816e10 0x55cb4781afc0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8004e10 tx=0x7fa2c80122c0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.596+0000 7fa2dbe07740 1 -- 172.21.15.43:0/829459204 shutdown_connections 2024-04-21T08:51:14.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.596+0000 7fa2dbe07740 1 --2- 172.21.15.43:0/829459204 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55cb4788e0d0 0x55cb478904c0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.596+0000 7fa2dbe07740 1 --2- 172.21.15.43:0/829459204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb47816e10 0x55cb4781afc0 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.596+0000 7fa2dbe07740 1 --2- 172.21.15.43:0/829459204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cb4780e290 0x55cb4781a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.596+0000 7fa2b6ff5640 0 client.21366 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:14.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.596+0000 7fa2dbe07740 1 --2- 172.21.15.43:0/829459204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb4780d8e0 0x55cb4781f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.596+0000 7fa2dbe07740 1 -- 172.21.15.43:0/829459204 >> 172.21.15.43:0/829459204 conn(0x55cb47807fb0 msgr2=0x55cb47807010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:14.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.596+0000 7fa2dbe07740 1 -- 172.21.15.43:0/829459204 shutdown_connections 2024-04-21T08:51:14.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.596+0000 7fa2dbe07740 1 -- 172.21.15.43:0/829459204 wait complete. 2024-04-21T08:51:14.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.596+0000 7fa2dbe07740 20 client.21366 trim_cache size 0 max 16384 2024-04-21T08:51:14.601 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:51:14.601 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-21T08:51:14.601 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "558da244-decc-4429-99ac-6c6e74c50138", 2024-04-21T08:51:14.601 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-21T08:51:14.602 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:51:14.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.597+0000 7fa2d913d640 1 -- 172.21.15.43:0/3539994134 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa2bc0a3d50 msgr2=0x7fa2bc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:14.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.597+0000 7fa2d913d640 1 --2- 172.21.15.43:0/3539994134 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa2bc0a3d50 0x7fa2bc0a6210 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0bb320 tx=0x7fa2cc091440 comp rx=0 tx=0).stop 2024-04-21T08:51:14.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.597+0000 7fa2d913d640 1 -- 172.21.15.43:0/3539994134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d41035a0 msgr2=0x7fa2d41a68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:14.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.597+0000 7fa2d913d640 1 --2- 172.21.15.43:0/3539994134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d41035a0 0x7fa2d41a68f0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc005e30 tx=0x7fa2cc034760 comp rx=0 tx=0).stop 2024-04-21T08:51:14.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.597+0000 7fa2d913d640 1 -- 172.21.15.43:0/3539994134 shutdown_connections 2024-04-21T08:51:14.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.597+0000 7fa2d913d640 1 --2- 172.21.15.43:0/3539994134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d41035a0 0x7fa2d41a68f0 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.597+0000 7fa2d913d640 1 --2- 172.21.15.43:0/3539994134 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa2bc0a3d50 0x7fa2bc0a6210 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.597+0000 7fa2d913d640 1 --2- 172.21.15.43:0/3539994134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2d4106160 0x7fa2d4103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.597+0000 7fa2d913d640 1 --2- 172.21.15.43:0/3539994134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2d4101090 0x7fa2d4102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.597+0000 7fa2d913d640 1 -- 172.21.15.43:0/3539994134 >> 172.21.15.43:0/3539994134 conn(0x7fa2d4077f50 msgr2=0x7fa2d40694c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:14.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.597+0000 7fa2d913d640 1 -- 172.21.15.43:0/3539994134 shutdown_connections 2024-04-21T08:51:14.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.597+0000 7fa2d913d640 1 -- 172.21.15.43:0/3539994134 wait complete. 2024-04-21T08:51:14.609 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:51:14.609 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:51:14.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.714+0000 7f840da4e640 1 -- 172.21.15.43:0/1700107968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84081021a0 msgr2=0x7f8408102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:14.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.714+0000 7f840da4e640 1 --2- 172.21.15.43:0/1700107968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84081021a0 0x7f8408102580 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f83f8009900 tx=0x7f83f802bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:14.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.714+0000 7f840da4e640 1 -- 172.21.15.43:0/1700107968 shutdown_connections 2024-04-21T08:51:14.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.714+0000 7f840da4e640 1 --2- 172.21.15.43:0/1700107968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8408107270 0x7f840810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.714+0000 7f840da4e640 1 --2- 172.21.15.43:0/1700107968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8408102b50 0x7f8408106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.714+0000 7f840da4e640 1 --2- 172.21.15.43:0/1700107968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84081021a0 0x7f8408102580 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.714+0000 7f840da4e640 1 -- 172.21.15.43:0/1700107968 >> 172.21.15.43:0/1700107968 conn(0x7f84080fda40 msgr2=0x7f84080ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.715+0000 7f840da4e640 1 -- 172.21.15.43:0/1700107968 shutdown_connections 2024-04-21T08:51:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.715+0000 7f840da4e640 1 -- 172.21.15.43:0/1700107968 wait complete. 2024-04-21T08:51:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.715+0000 7f840da4e640 1 Processor -- start 2024-04-21T08:51:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.715+0000 7f840da4e640 1 -- start start 2024-04-21T08:51:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.715+0000 7f840da4e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84081021a0 0x7f84081987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.715+0000 7f840da4e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8408107270 0x7f8408198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.715+0000 7f840da4e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f840819d020 0x7f840819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.715+0000 7f840da4e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f84081121d0 con 0x7f8408107270 2024-04-21T08:51:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.715+0000 7f840da4e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8408112050 con 0x7f84081021a0 2024-04-21T08:51:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.715+0000 7f840da4e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8408112350 con 0x7f840819d020 2024-04-21T08:51:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.715+0000 7f84067fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8408107270 0x7f8408198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.716+0000 7f84067fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8408107270 0x7f8408198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51624/0 (socket says 172.21.15.43:51624) 2024-04-21T08:51:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.716+0000 7f84067fc640 1 -- 172.21.15.43:0/3468883371 learned_addr learned my addr 172.21.15.43:0/3468883371 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.716+0000 7f84077fe640 1 --2- 172.21.15.43:0/3468883371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f840819d020 0x7f840819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.716+0000 7f8406ffd640 1 --2- 172.21.15.43:0/3468883371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84081021a0 0x7f84081987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.716+0000 7f84067fc640 1 -- 172.21.15.43:0/3468883371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f840819d020 msgr2=0x7f840819d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.716+0000 7f84067fc640 1 --2- 172.21.15.43:0/3468883371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f840819d020 0x7f840819d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.716+0000 7f84067fc640 1 -- 172.21.15.43:0/3468883371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84081021a0 msgr2=0x7f84081987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.716+0000 7f84067fc640 1 --2- 172.21.15.43:0/3468883371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84081021a0 0x7f84081987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.716+0000 7f84067fc640 1 -- 172.21.15.43:0/3468883371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840819db10 con 0x7f8408107270 2024-04-21T08:51:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.716+0000 7f8406ffd640 1 --2- 172.21.15.43:0/3468883371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84081021a0 0x7f84081987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.716+0000 7f84067fc640 1 --2- 172.21.15.43:0/3468883371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8408107270 0x7f8408198cf0 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00c970 tx=0x7f83fc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.716+0000 7f840ca4c640 1 -- 172.21.15.43:0/3468883371 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc007bb0 con 0x7f8408107270 2024-04-21T08:51:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.716+0000 7f840ca4c640 1 -- 172.21.15.43:0/3468883371 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83fc007d50 con 0x7f8408107270 2024-04-21T08:51:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.716+0000 7f840da4e640 1 -- 172.21.15.43:0/3468883371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f840819de00 con 0x7f8408107270 2024-04-21T08:51:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.716+0000 7f840ca4c640 1 -- 172.21.15.43:0/3468883371 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc018730 con 0x7f8408107270 2024-04-21T08:51:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.716+0000 7f840da4e640 1 -- 172.21.15.43:0/3468883371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84081a55a0 con 0x7f8408107270 2024-04-21T08:51:14.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.718+0000 7f840ca4c640 1 -- 172.21.15.43:0/3468883371 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f83fc00f990 con 0x7f8408107270 2024-04-21T08:51:14.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.719+0000 7f8410643740 1 Processor -- start 2024-04-21T08:51:14.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.719+0000 7f8410643740 1 -- start start 2024-04-21T08:51:14.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.719+0000 7f8410643740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608875a3580 0x5608875a3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.719+0000 7f8410643740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875a3ea0 0x5608875ac540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.719+0000 7f8410643740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608875aca80 0x5608875aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.719+0000 7f8410643740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5608875081e0 con 0x5608875a3ea0 2024-04-21T08:51:14.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.719+0000 7f8410643740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5608874a5390 con 0x5608875aca80 2024-04-21T08:51:14.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.719+0000 7f8410643740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5608875041a0 con 0x5608875a3580 2024-04-21T08:51:14.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.719+0000 7f840ca4c640 1 --2- 172.21.15.43:0/3468883371 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f83f00a3d20 0x7f83f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.719+0000 7f8406ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875a3ea0 0x5608875ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.719+0000 7f8406ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875a3ea0 0x5608875ac540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51634/0 (socket says 172.21.15.43:51634) 2024-04-21T08:51:14.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.719+0000 7f8406ffd640 1 -- 172.21.15.43:0/2110584954 learned_addr learned my addr 172.21.15.43:0/2110584954 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:14.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.719+0000 7f84077fe640 1 --2- 172.21.15.43:0/3468883371 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f83f00a3d20 0x7f83f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.719+0000 7f840ca4c640 1 -- 172.21.15.43:0/3468883371 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f83fc0d0d20 con 0x7f8408107270 2024-04-21T08:51:14.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.719+0000 7f84067fc640 1 --2- 172.21.15.43:0/2110584954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608875aca80 0x5608875aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.719+0000 7f84077fe640 1 --2- 172.21.15.43:0/2110584954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608875a3580 0x5608875a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.719+0000 7f8406ffd640 1 -- 172.21.15.43:0/2110584954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608875a3580 msgr2=0x5608875a3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:14.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.719+0000 7f8406ffd640 1 --2- 172.21.15.43:0/2110584954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608875a3580 0x5608875a3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.719+0000 7f8406ffd640 1 -- 172.21.15.43:0/2110584954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608875aca80 msgr2=0x5608875aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:14.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.719+0000 7f8406ffd640 1 --2- 172.21.15.43:0/2110584954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608875aca80 0x5608875aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.719+0000 7f8406ffd640 1 -- 172.21.15.43:0/2110584954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608872e47d0 con 0x5608875a3ea0 2024-04-21T08:51:14.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.719+0000 7f84067fc640 1 --2- 172.21.15.43:0/2110584954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608875aca80 0x5608875aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:14.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.720+0000 7f84077fe640 1 --2- 172.21.15.43:0/3468883371 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f83f00a3d20 0x7f83f00a61e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8400005c40 tx=0x7f84000094f0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:14.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.720+0000 7f8406ffd640 1 --2- 172.21.15.43:0/2110584954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875a3ea0 0x5608875ac540 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f83f8036040 tx=0x7f83f802c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1e602a4a005d04 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:14.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.720+0000 7f83edffb640 1 -- 172.21.15.43:0/2110584954 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8043020 con 0x5608875a3ea0 2024-04-21T08:51:14.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.720+0000 7f83edffb640 1 -- 172.21.15.43:0/2110584954 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83f802c7e0 con 0x5608875a3ea0 2024-04-21T08:51:14.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.720+0000 7f83edffb640 1 -- 172.21.15.43:0/2110584954 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f80055d0 con 0x5608875a3ea0 2024-04-21T08:51:14.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.721+0000 7f8410643740 1 -- 172.21.15.43:0/2110584954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875a3ea0 msgr2=0x5608875ac540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:14.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.721+0000 7f8410643740 1 --2- 172.21.15.43:0/2110584954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875a3ea0 0x5608875ac540 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f83f8036040 tx=0x7f83f802c680 comp rx=0 tx=0).stop 2024-04-21T08:51:14.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.721+0000 7f8410643740 1 -- 172.21.15.43:0/2110584954 shutdown_connections 2024-04-21T08:51:14.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.721+0000 7f8410643740 1 --2- 172.21.15.43:0/2110584954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608875aca80 0x5608875aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.721+0000 7f8410643740 1 --2- 172.21.15.43:0/2110584954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875a3ea0 0x5608875ac540 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.721+0000 7f8410643740 1 --2- 172.21.15.43:0/2110584954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608875a3580 0x5608875a3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.721+0000 7f8410643740 1 -- 172.21.15.43:0/2110584954 >> 172.21.15.43:0/2110584954 conn(0x56088759dcb0 msgr2=0x56088759e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:14.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.721+0000 7f8410643740 1 -- 172.21.15.43:0/2110584954 shutdown_connections 2024-04-21T08:51:14.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.721+0000 7f8410643740 1 -- 172.21.15.43:0/2110584954 wait complete. 2024-04-21T08:51:14.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.721+0000 7f8410643740 1 -- start start 2024-04-21T08:51:14.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.721+0000 7f8410643740 1 Processor -- start 2024-04-21T08:51:14.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.721+0000 7f8410643740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608875a3580 0x5608875a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.721+0000 7f8410643740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608875a3ea0 0x5608875a9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.721+0000 7f8410643740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875aca80 0x5608875aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.721+0000 7f8410643740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5608875af810 con 0x5608875aca80 2024-04-21T08:51:14.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.721+0000 7f8410643740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5608874c12f0 con 0x5608875a3580 2024-04-21T08:51:14.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.721+0000 7f8410643740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5608875af990 con 0x5608875a3ea0 2024-04-21T08:51:14.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.722+0000 7f8410643740 10 client.0 _ll_register_callbacks cb 0x5608874ce580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:14.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.722+0000 7f84077fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875aca80 0x5608875aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.722+0000 7f84077fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875aca80 0x5608875aa520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51650/0 (socket says 172.21.15.43:51650) 2024-04-21T08:51:14.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.722+0000 7f84077fe640 1 -- 172.21.15.43:0/1451588864 learned_addr learned my addr 172.21.15.43:0/1451588864 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:14.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.722+0000 7f84067fc640 1 --2- 172.21.15.43:0/1451588864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608875a3ea0 0x5608875a9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.722+0000 7f8406ffd640 1 --2- 172.21.15.43:0/1451588864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608875a3580 0x5608875a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.722+0000 7f84077fe640 1 -- 172.21.15.43:0/1451588864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608875a3ea0 msgr2=0x5608875a9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:14.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.722+0000 7f84077fe640 1 --2- 172.21.15.43:0/1451588864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608875a3ea0 0x5608875a9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.722+0000 7f84077fe640 1 -- 172.21.15.43:0/1451588864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608875a3580 msgr2=0x5608875a9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:14.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.722+0000 7f84077fe640 1 --2- 172.21.15.43:0/1451588864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608875a3580 0x5608875a9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.722+0000 7f84077fe640 1 -- 172.21.15.43:0/1451588864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56088721a3f0 con 0x5608875aca80 2024-04-21T08:51:14.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.722+0000 7f8406ffd640 1 --2- 172.21.15.43:0/1451588864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608875a3580 0x5608875a9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:14.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.722+0000 7f84077fe640 1 --2- 172.21.15.43:0/1451588864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875aca80 0x5608875aa520 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7f8400010800 tx=0x7f8400004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5b7d7e43add70f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:14.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.722+0000 7f83e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:14.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.722+0000 7f83e6ffd640 1 -- 172.21.15.43:0/1451588864 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400025070 con 0x5608875aca80 2024-04-21T08:51:14.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.722+0000 7f83e6ffd640 1 -- 172.21.15.43:0/1451588864 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f840000cad0 con 0x5608875aca80 2024-04-21T08:51:14.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.722+0000 7f8410643740 1 -- 172.21.15.43:0/1451588864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83f802c7e0 con 0x5608875aca80 2024-04-21T08:51:14.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.722+0000 7f83e6ffd640 1 -- 172.21.15.43:0/1451588864 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f840000ccd0 con 0x5608875aca80 2024-04-21T08:51:14.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.722+0000 7f8410643740 10 client.21378 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:14.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.722+0000 7f8410643740 1 -- 172.21.15.43:0/1451588864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5608871da0f0 con 0x5608875aca80 2024-04-21T08:51:14.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.723+0000 7f83e6ffd640 1 -- 172.21.15.43:0/1451588864 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83f802c7e0 con 0x5608875aca80 2024-04-21T08:51:14.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.723+0000 7f83e6ffd640 1 -- 172.21.15.43:0/1451588864 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f840001e3c0 con 0x5608875aca80 2024-04-21T08:51:14.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.723+0000 7f8410643740 10 client.21378 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:14.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.723+0000 7f8410643740 10 client.21378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:14.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.723+0000 7f8410643740 20 client.21378 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:14.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.723+0000 7f8410643740 1 --2- 172.21.15.43:0/1451588864 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5608875bc9f0 0x5608875bee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.723+0000 7f8410643740 4 client.21378 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:14.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.723+0000 7f8410643740 1 -- 172.21.15.43:0/1451588864 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560887515c00 con 0x5608875bc9f0 2024-04-21T08:51:14.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.723+0000 7f8406ffd640 1 --2- 172.21.15.43:0/1451588864 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5608875bc9f0 0x5608875bee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.724+0000 7f83e6ffd640 10 client.21378 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:14.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.724+0000 7f8406ffd640 1 --2- 172.21.15.43:0/1451588864 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5608875bc9f0 0x5608875bee30 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ece6f3cfa9896d9e server_cookie=bed207052250a904 in_seq=0 out_seq=0 2024-04-21T08:51:14.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.724+0000 7f83e6ffd640 1 -- 172.21.15.43:0/1451588864 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560887515c00 con 0x5608875bc9f0 2024-04-21T08:51:14.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.724+0000 7f83e6ffd640 10 client.21378 handle_command_reply: tid=0 2024-04-21T08:51:14.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.724+0000 7f8410643740 1 client.21378 shutdown 2024-04-21T08:51:14.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.724+0000 7f8410643740 1 -- 172.21.15.43:0/1451588864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875aca80 msgr2=0x5608875aa520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:14.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.724+0000 7f8410643740 1 --2- 172.21.15.43:0/1451588864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875aca80 0x5608875aa520 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7f8400010800 tx=0x7f8400004a40 comp rx=0 tx=0).stop 2024-04-21T08:51:14.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.724+0000 7f8410643740 1 -- 172.21.15.43:0/1451588864 shutdown_connections 2024-04-21T08:51:14.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.724+0000 7f8410643740 1 --2- 172.21.15.43:0/1451588864 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5608875bc9f0 0x5608875bee30 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.724+0000 7f8410643740 1 --2- 172.21.15.43:0/1451588864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875aca80 0x5608875aa520 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.724+0000 7f8410643740 1 --2- 172.21.15.43:0/1451588864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608875a3ea0 0x5608875a9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.724+0000 7f8410643740 1 --2- 172.21.15.43:0/1451588864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608875a3580 0x5608875a9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.724+0000 7f8410643740 1 -- 172.21.15.43:0/1451588864 >> 172.21.15.43:0/1451588864 conn(0x56088759dcb0 msgr2=0x56088759fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:14.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.724+0000 7f83e6ffd640 0 client.21378 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:14.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.725+0000 7f8410643740 1 -- 172.21.15.43:0/1451588864 shutdown_connections 2024-04-21T08:51:14.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.725+0000 7f8410643740 1 -- 172.21.15.43:0/1451588864 wait complete. 2024-04-21T08:51:14.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.725+0000 7f8410643740 20 client.21378 trim_cache size 0 max 16384 2024-04-21T08:51:14.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.742+0000 7f8410643740 1 Processor -- start 2024-04-21T08:51:14.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.742+0000 7f8410643740 1 -- start start 2024-04-21T08:51:14.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.742+0000 7f8410643740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608875a3580 0x5608875b86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.742+0000 7f8410643740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875a3ea0 0x5608875ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.742+0000 7f8410643740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608875aca80 0x5608875c4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.742+0000 7f8410643740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f84081af5b0 con 0x5608875a3ea0 2024-04-21T08:51:14.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.742+0000 7f8410643740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f84081b10f0 con 0x5608875a3580 2024-04-21T08:51:14.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.742+0000 7f8410643740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f84081afc80 con 0x5608875aca80 2024-04-21T08:51:14.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.743+0000 7f84067fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875a3ea0 0x5608875ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.743+0000 7f84067fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875a3ea0 0x5608875ab8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51666/0 (socket says 172.21.15.43:51666) 2024-04-21T08:51:14.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.743+0000 7f84067fc640 1 -- 172.21.15.43:0/3792953545 learned_addr learned my addr 172.21.15.43:0/3792953545 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:14.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.743+0000 7f84077fe640 1 --2- 172.21.15.43:0/3792953545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608875aca80 0x5608875c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.743+0000 7f8406ffd640 1 --2- 172.21.15.43:0/3792953545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608875a3580 0x5608875b86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.743+0000 7f84067fc640 1 -- 172.21.15.43:0/3792953545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608875aca80 msgr2=0x5608875c4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:14.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.743+0000 7f84067fc640 1 --2- 172.21.15.43:0/3792953545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608875aca80 0x5608875c4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.743+0000 7f84067fc640 1 -- 172.21.15.43:0/3792953545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608875a3580 msgr2=0x5608875b86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:14.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.743+0000 7f84067fc640 1 --2- 172.21.15.43:0/3792953545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608875a3580 0x5608875b86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.743+0000 7f84067fc640 1 -- 172.21.15.43:0/3792953545 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608875aee70 con 0x5608875a3ea0 2024-04-21T08:51:14.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.743+0000 7f84067fc640 1 --2- 172.21.15.43:0/3792953545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875a3ea0 0x5608875ab8d0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f83fc0ce0b0 tx=0x7f83fc087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5463f5e58be73f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:14.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.743+0000 7f83ed7fa640 1 -- 172.21.15.43:0/3792953545 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc0a0c50 con 0x5608875a3ea0 2024-04-21T08:51:14.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.743+0000 7f83ed7fa640 1 -- 172.21.15.43:0/3792953545 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83fc0a5510 con 0x5608875a3ea0 2024-04-21T08:51:14.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.743+0000 7f83ed7fa640 1 -- 172.21.15.43:0/3792953545 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc09de30 con 0x5608875a3ea0 2024-04-21T08:51:14.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.744+0000 7f8410643740 1 -- 172.21.15.43:0/3792953545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875a3ea0 msgr2=0x5608875ab8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:14.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.744+0000 7f8410643740 1 --2- 172.21.15.43:0/3792953545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875a3ea0 0x5608875ab8d0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f83fc0ce0b0 tx=0x7f83fc087040 comp rx=0 tx=0).stop 2024-04-21T08:51:14.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.744+0000 7f8410643740 1 -- 172.21.15.43:0/3792953545 shutdown_connections 2024-04-21T08:51:14.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.744+0000 7f8410643740 1 --2- 172.21.15.43:0/3792953545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608875aca80 0x5608875c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.744+0000 7f8410643740 1 --2- 172.21.15.43:0/3792953545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875a3ea0 0x5608875ab8d0 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.744+0000 7f8410643740 1 --2- 172.21.15.43:0/3792953545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608875a3580 0x5608875b86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.744+0000 7f8410643740 1 -- 172.21.15.43:0/3792953545 >> 172.21.15.43:0/3792953545 conn(0x56088759dcb0 msgr2=0x5608875ab1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:14.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.744+0000 7f8410643740 1 -- 172.21.15.43:0/3792953545 shutdown_connections 2024-04-21T08:51:14.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.744+0000 7f8410643740 1 -- 172.21.15.43:0/3792953545 wait complete. 2024-04-21T08:51:14.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.744+0000 7f8410643740 1 -- start start 2024-04-21T08:51:14.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.744+0000 7f8410643740 1 Processor -- start 2024-04-21T08:51:14.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.744+0000 7f8410643740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608875a3580 0x5608875b0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.745+0000 7f8410643740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608875a3ea0 0x5608875b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.745+0000 7f8410643740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875aca80 0x5608875b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.745+0000 7f8410643740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f84081b0a20 con 0x5608875aca80 2024-04-21T08:51:14.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.745+0000 7f8410643740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f84081b0350 con 0x5608875a3ea0 2024-04-21T08:51:14.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.745+0000 7f8410643740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f84081ad5b0 con 0x5608875a3580 2024-04-21T08:51:14.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.745+0000 7f8410643740 10 client.0 _ll_register_callbacks cb 0x7f8400009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:14.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.745+0000 7f84077fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875aca80 0x5608875b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.745+0000 7f84077fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875aca80 0x5608875b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51670/0 (socket says 172.21.15.43:51670) 2024-04-21T08:51:14.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.745+0000 7f84077fe640 1 -- 172.21.15.43:0/2376058871 learned_addr learned my addr 172.21.15.43:0/2376058871 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:14.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.745+0000 7f8406ffd640 1 --2- 172.21.15.43:0/2376058871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608875a3580 0x5608875b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.745+0000 7f84067fc640 1 --2- 172.21.15.43:0/2376058871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608875a3ea0 0x5608875b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.745+0000 7f84077fe640 1 -- 172.21.15.43:0/2376058871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608875a3580 msgr2=0x5608875b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:14.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.745+0000 7f84077fe640 1 --2- 172.21.15.43:0/2376058871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608875a3580 0x5608875b0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.745+0000 7f84077fe640 1 -- 172.21.15.43:0/2376058871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608875a3ea0 msgr2=0x5608875b0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:14.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.745+0000 7f84077fe640 1 --2- 172.21.15.43:0/2376058871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608875a3ea0 0x5608875b0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.745+0000 7f84077fe640 1 -- 172.21.15.43:0/2376058871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560887621f70 con 0x5608875aca80 2024-04-21T08:51:14.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.745+0000 7f84077fe640 1 --2- 172.21.15.43:0/2376058871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875aca80 0x5608875b0fd0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f8400005c10 tx=0x7f840001e5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b5fe2de5b0c423 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:14.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.745+0000 7f83e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:14.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.745+0000 7f83e6ffd640 1 -- 172.21.15.43:0/2376058871 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400039020 con 0x5608875aca80 2024-04-21T08:51:14.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.745+0000 7f83e6ffd640 1 -- 172.21.15.43:0/2376058871 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f8400004ab0 con 0x5608875aca80 2024-04-21T08:51:14.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.745+0000 7f8410643740 1 -- 172.21.15.43:0/2376058871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83fc0a5510 con 0x5608875aca80 2024-04-21T08:51:14.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.745+0000 7f83e6ffd640 1 -- 172.21.15.43:0/2376058871 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8400044050 con 0x5608875aca80 2024-04-21T08:51:14.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.745+0000 7f8410643740 10 client.21384 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:14.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.745+0000 7f8410643740 1 -- 172.21.15.43:0/2376058871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560887623180 con 0x5608875aca80 2024-04-21T08:51:14.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.746+0000 7f83e6ffd640 1 -- 172.21.15.43:0/2376058871 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83fc0a5510 con 0x5608875aca80 2024-04-21T08:51:14.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.746+0000 7f83e6ffd640 1 -- 172.21.15.43:0/2376058871 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f840001ebc0 con 0x5608875aca80 2024-04-21T08:51:14.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.746+0000 7f8410643740 10 client.21384 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:14.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.746+0000 7f8410643740 10 client.21384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:14.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.746+0000 7f8410643740 20 client.21384 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:14.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.746+0000 7f8410643740 1 --2- 172.21.15.43:0/2376058871 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560887623920 0x560887625d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:14.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.746+0000 7f8410643740 4 client.21384 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:51:14.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.746+0000 7f8410643740 1 -- 172.21.15.43:0/2376058871 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56088750fa70 con 0x560887623920 2024-04-21T08:51:14.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.746+0000 7f8406ffd640 1 --2- 172.21.15.43:0/2376058871 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560887623920 0x560887625d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:14.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.747+0000 7f83e6ffd640 10 client.21384 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:14.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.747+0000 7f8406ffd640 1 --2- 172.21.15.43:0/2376058871 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560887623920 0x560887625d40 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6ef1578391ba72b server_cookie=4bc30492b1b969c in_seq=0 out_seq=0 2024-04-21T08:51:14.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.747+0000 7f83e6ffd640 1 -- 172.21.15.43:0/2376058871 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56088750fa70 con 0x560887623920 2024-04-21T08:51:14.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.747+0000 7f83e6ffd640 10 client.21384 handle_command_reply: tid=0 2024-04-21T08:51:14.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.747+0000 7f8410643740 1 client.21384 shutdown 2024-04-21T08:51:14.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.747+0000 7f8410643740 1 -- 172.21.15.43:0/2376058871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875aca80 msgr2=0x5608875b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:14.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.747+0000 7f8410643740 1 --2- 172.21.15.43:0/2376058871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875aca80 0x5608875b0fd0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f8400005c10 tx=0x7f840001e5f0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.747+0000 7f8410643740 1 -- 172.21.15.43:0/2376058871 shutdown_connections 2024-04-21T08:51:14.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.747+0000 7f8410643740 1 --2- 172.21.15.43:0/2376058871 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x560887623920 0x560887625d40 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.747+0000 7f8410643740 1 --2- 172.21.15.43:0/2376058871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5608875aca80 0x5608875b0fd0 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.747+0000 7f8410643740 1 --2- 172.21.15.43:0/2376058871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608875a3ea0 0x5608875b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.747+0000 7f83e6ffd640 0 client.21384 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:14.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.747+0000 7f8410643740 1 --2- 172.21.15.43:0/2376058871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5608875a3580 0x5608875b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.747+0000 7f8410643740 1 -- 172.21.15.43:0/2376058871 >> 172.21.15.43:0/2376058871 conn(0x56088759dcb0 msgr2=0x56088759cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:14.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.747+0000 7f8410643740 1 -- 172.21.15.43:0/2376058871 shutdown_connections 2024-04-21T08:51:14.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.747+0000 7f8410643740 1 -- 172.21.15.43:0/2376058871 wait complete. 2024-04-21T08:51:14.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.747+0000 7f8410643740 20 client.21384 trim_cache size 0 max 16384 2024-04-21T08:51:14.753 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:51:14.753 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:51:14.753 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-21T08:51:14.753 INFO:teuthology.orchestra.run.smithi043.stdout: "558da244-decc-4429-99ac-6c6e74c50138": { 2024-04-21T08:51:14.753 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-21T08:51:14.753 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "558da244-decc-4429-99ac-6c6e74c50138", 2024-04-21T08:51:14.754 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-04-21T08:51:14.754 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:14.754 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:14.754 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:51:14.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.748+0000 7f840da4e640 1 -- 172.21.15.43:0/3468883371 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f83f00a3d20 msgr2=0x7f83f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:14.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.748+0000 7f840da4e640 1 --2- 172.21.15.43:0/3468883371 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f83f00a3d20 0x7f83f00a61e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8400005c40 tx=0x7f84000094f0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.748+0000 7f840da4e640 1 -- 172.21.15.43:0/3468883371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8408107270 msgr2=0x7f8408198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:14.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.748+0000 7f840da4e640 1 --2- 172.21.15.43:0/3468883371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8408107270 0x7f8408198cf0 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00c970 tx=0x7f83fc00ce30 comp rx=0 tx=0).stop 2024-04-21T08:51:14.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.748+0000 7f840da4e640 1 -- 172.21.15.43:0/3468883371 shutdown_connections 2024-04-21T08:51:14.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.748+0000 7f840da4e640 1 --2- 172.21.15.43:0/3468883371 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f83f00a3d20 0x7f83f00a61e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.748+0000 7f840da4e640 1 --2- 172.21.15.43:0/3468883371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f840819d020 0x7f840819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.748+0000 7f840da4e640 1 --2- 172.21.15.43:0/3468883371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8408107270 0x7f8408198cf0 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.748+0000 7f840da4e640 1 --2- 172.21.15.43:0/3468883371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84081021a0 0x7f84081987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:14.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.748+0000 7f840da4e640 1 -- 172.21.15.43:0/3468883371 >> 172.21.15.43:0/3468883371 conn(0x7f84080fda40 msgr2=0x7f84080fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:14.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.748+0000 7f840da4e640 1 -- 172.21.15.43:0/3468883371 shutdown_connections 2024-04-21T08:51:14.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:14.748+0000 7f840da4e640 1 -- 172.21.15.43:0/3468883371 wait complete. 2024-04-21T08:51:14.757 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:14 smithi043 ceph-mon[27978]: pgmap v385: 129 pgs: 129 active+clean; 2.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-21T08:51:14.758 INFO:tasks.cephfs.filesystem:scrub status for tag:558da244-decc-4429-99ac-6c6e74c50138 - {'path': '/', 'tag': '558da244-decc-4429-99ac-6c6e74c50138', 'options': 'recursive,force'} 2024-04-21T08:51:14.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:14 smithi175 ceph-mon[31920]: pgmap v385: 129 pgs: 129 active+clean; 2.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-21T08:51:15.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:14 smithi112 ceph-mon[31085]: pgmap v385: 129 pgs: 129 active+clean; 2.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-21T08:51:15.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:15 smithi043 ceph-mon[27978]: scrub queued for path: / 2024-04-21T08:51:15.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:15 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:15.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:15 smithi043 ceph-mon[27978]: scrub summary: active paths [/] 2024-04-21T08:51:15.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:15 smithi043 ceph-mon[27978]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:15.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:15 smithi175 ceph-mon[31920]: scrub queued for path: / 2024-04-21T08:51:15.965 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:15 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:15.965 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:15 smithi175 ceph-mon[31920]: scrub summary: active paths [/] 2024-04-21T08:51:15.965 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:15 smithi175 ceph-mon[31920]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:16.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:15 smithi112 ceph-mon[31085]: scrub queued for path: / 2024-04-21T08:51:16.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:15 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:16.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:15 smithi112 ceph-mon[31085]: scrub summary: active paths [/] 2024-04-21T08:51:16.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:15 smithi112 ceph-mon[31085]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:16.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:16 smithi043 ceph-mon[27978]: pgmap v386: 129 pgs: 129 active+clean; 2.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-21T08:51:16.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:16 smithi175 ceph-mon[31920]: pgmap v386: 129 pgs: 129 active+clean; 2.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-21T08:51:17.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:16 smithi112 ceph-mon[31085]: pgmap v386: 129 pgs: 129 active+clean; 2.4 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-21T08:51:18.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:18 smithi175 ceph-mon[31920]: pgmap v387: 129 pgs: 129 active+clean; 2.6 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 20 op/s 2024-04-21T08:51:19.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:18 smithi112 ceph-mon[31085]: pgmap v387: 129 pgs: 129 active+clean; 2.6 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 20 op/s 2024-04-21T08:51:19.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:18 smithi043 ceph-mon[27978]: pgmap v387: 129 pgs: 129 active+clean; 2.6 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 20 op/s 2024-04-21T08:51:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.660+0000 7fe52cd2c640 1 -- 172.21.15.43:0/2916201334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52806a7c0 msgr2=0x7fe52806ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.660+0000 7fe52cd2c640 1 --2- 172.21.15.43:0/2916201334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52806a7c0 0x7fe52806ac20 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009900 tx=0x7fe51c02bcd0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.661+0000 7fe52cd2c640 1 -- 172.21.15.43:0/2916201334 shutdown_connections 2024-04-21T08:51:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.661+0000 7fe52cd2c640 1 --2- 172.21.15.43:0/2916201334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe52806b160 0x7fe52810edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.661+0000 7fe52cd2c640 1 --2- 172.21.15.43:0/2916201334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52806a7c0 0x7fe52806ac20 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.661+0000 7fe52cd2c640 1 --2- 172.21.15.43:0/2916201334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe528108020 0x7fe528108400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.661+0000 7fe52cd2c640 1 -- 172.21.15.43:0/2916201334 >> 172.21.15.43:0/2916201334 conn(0x7fe528076c50 msgr2=0x7fe528077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.661+0000 7fe52cd2c640 1 -- 172.21.15.43:0/2916201334 shutdown_connections 2024-04-21T08:51:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.661+0000 7fe52cd2c640 1 -- 172.21.15.43:0/2916201334 wait complete. 2024-04-21T08:51:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.661+0000 7fe52cd2c640 1 Processor -- start 2024-04-21T08:51:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.662+0000 7fe52cd2c640 1 -- start start 2024-04-21T08:51:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.662+0000 7fe52cd2c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52806a7c0 0x7fe528198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.662+0000 7fe52cd2c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe52806b160 0x7fe528198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.662+0000 7fe52cd2c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe528108020 0x7fe52819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.662+0000 7fe52cd2c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe528112210 con 0x7fe52806a7c0 2024-04-21T08:51:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.662+0000 7fe52cd2c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe528112090 con 0x7fe528108020 2024-04-21T08:51:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.662+0000 7fe52cd2c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe528112390 con 0x7fe52806b160 2024-04-21T08:51:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.662+0000 7fe52659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52806a7c0 0x7fe528198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.662+0000 7fe52659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52806a7c0 0x7fe528198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52244/0 (socket says 172.21.15.43:52244) 2024-04-21T08:51:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.662+0000 7fe52659c640 1 -- 172.21.15.43:0/62328078 learned_addr learned my addr 172.21.15.43:0/62328078 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.662+0000 7fe526d9d640 1 --2- 172.21.15.43:0/62328078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe528108020 0x7fe52819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.662+0000 7fe525d9b640 1 --2- 172.21.15.43:0/62328078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe52806b160 0x7fe528198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.662+0000 7fe52659c640 1 -- 172.21.15.43:0/62328078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe52806b160 msgr2=0x7fe528198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.662+0000 7fe52659c640 1 --2- 172.21.15.43:0/62328078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe52806b160 0x7fe528198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.662+0000 7fe52659c640 1 -- 172.21.15.43:0/62328078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe528108020 msgr2=0x7fe52819cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.662+0000 7fe52659c640 1 --2- 172.21.15.43:0/62328078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe528108020 0x7fe52819cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.662+0000 7fe52659c640 1 -- 172.21.15.43:0/62328078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52819d680 con 0x7fe52806a7c0 2024-04-21T08:51:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.662+0000 7fe525d9b640 1 --2- 172.21.15.43:0/62328078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe52806b160 0x7fe528198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.662+0000 7fe526d9d640 1 --2- 172.21.15.43:0/62328078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe528108020 0x7fe52819cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.663+0000 7fe52659c640 1 --2- 172.21.15.43:0/62328078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52806a7c0 0x7fe528198690 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7fe51800d950 tx=0x7fe51800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.663+0000 7fe5177fe640 1 -- 172.21.15.43:0/62328078 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518017030 con 0x7fe52806a7c0 2024-04-21T08:51:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.663+0000 7fe5177fe640 1 -- 172.21.15.43:0/62328078 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe518004020 con 0x7fe52806a7c0 2024-04-21T08:51:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.663+0000 7fe5177fe640 1 -- 172.21.15.43:0/62328078 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5180047a0 con 0x7fe52806a7c0 2024-04-21T08:51:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.663+0000 7fe52cd2c640 1 -- 172.21.15.43:0/62328078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52819d970 con 0x7fe52806a7c0 2024-04-21T08:51:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.663+0000 7fe52cd2c640 1 -- 172.21.15.43:0/62328078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5281a5200 con 0x7fe52806a7c0 2024-04-21T08:51:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.665+0000 7fe5177fe640 1 -- 172.21.15.43:0/62328078 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe518009a30 con 0x7fe52806a7c0 2024-04-21T08:51:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.665+0000 7fe52f9f1740 1 Processor -- start 2024-04-21T08:51:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.666+0000 7fe52f9f1740 1 -- start start 2024-04-21T08:51:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.666+0000 7fe52f9f1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f58d0 0x5565654f4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.666+0000 7fe52f9f1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565654f4b60 0x5565654f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.666+0000 7fe52f9f1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565654fe0e0 0x5565655004d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.666+0000 7fe5177fe640 1 --2- 172.21.15.43:0/62328078 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe5100a3dc0 0x7fe5100a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.666+0000 7fe52f9f1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5565654598d0 con 0x5565654f58d0 2024-04-21T08:51:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.666+0000 7fe52f9f1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5565653f6e80 con 0x5565654f4b60 2024-04-21T08:51:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.666+0000 7fe52f9f1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556565455890 con 0x5565654fe0e0 2024-04-21T08:51:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.666+0000 7fe526d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f58d0 0x5565654f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.666+0000 7fe525d9b640 1 --2- 172.21.15.43:0/62328078 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe5100a3dc0 0x7fe5100a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.666+0000 7fe526d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f58d0 0x5565654f4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52246/0 (socket says 172.21.15.43:52246) 2024-04-21T08:51:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.666+0000 7fe526d9d640 1 -- 172.21.15.43:0/1735993465 learned_addr learned my addr 172.21.15.43:0/1735993465 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.666+0000 7fe52659c640 1 --2- 172.21.15.43:0/1735993465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565654f4b60 0x5565654f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.666+0000 7fe5177fe640 1 -- 172.21.15.43:0/62328078 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fe5180d06f0 con 0x7fe52806a7c0 2024-04-21T08:51:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.666+0000 7fe526d9d640 1 -- 172.21.15.43:0/1735993465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565654fe0e0 msgr2=0x5565655004d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.666+0000 7fe526d9d640 1 --2- 172.21.15.43:0/1735993465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565654fe0e0 0x5565655004d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.667+0000 7fe526d9d640 1 -- 172.21.15.43:0/1735993465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565654f4b60 msgr2=0x5565654f4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.667+0000 7fe526d9d640 1 --2- 172.21.15.43:0/1735993465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565654f4b60 0x5565654f4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.667+0000 7fe526d9d640 1 -- 172.21.15.43:0/1735993465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556565418320 con 0x5565654f58d0 2024-04-21T08:51:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.667+0000 7fe526d9d640 1 --2- 172.21.15.43:0/1735993465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f58d0 0x5565654f4620 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7fe520012410 tx=0x7fe5200128d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8daafc357676a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.667+0000 7fe525d9b640 1 --2- 172.21.15.43:0/62328078 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe5100a3dc0 0x7fe5100a6280 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe51c006010 tx=0x7fe51c005a20 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.667+0000 7fe514ff9640 1 -- 172.21.15.43:0/1735993465 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5200136c0 con 0x5565654f58d0 2024-04-21T08:51:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.667+0000 7fe514ff9640 1 -- 172.21.15.43:0/1735993465 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe520013d10 con 0x5565654f58d0 2024-04-21T08:51:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.667+0000 7fe514ff9640 1 -- 172.21.15.43:0/1735993465 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52001bdb0 con 0x5565654f58d0 2024-04-21T08:51:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.668+0000 7fe52f9f1740 1 -- 172.21.15.43:0/1735993465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f58d0 msgr2=0x5565654f4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.668+0000 7fe52f9f1740 1 --2- 172.21.15.43:0/1735993465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f58d0 0x5565654f4620 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7fe520012410 tx=0x7fe5200128d0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.668+0000 7fe52f9f1740 1 -- 172.21.15.43:0/1735993465 shutdown_connections 2024-04-21T08:51:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.668+0000 7fe52f9f1740 1 --2- 172.21.15.43:0/1735993465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565654fe0e0 0x5565655004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.668+0000 7fe52f9f1740 1 --2- 172.21.15.43:0/1735993465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565654f4b60 0x5565654f4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.668+0000 7fe52f9f1740 1 --2- 172.21.15.43:0/1735993465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f58d0 0x5565654f4620 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.668+0000 7fe52f9f1740 1 -- 172.21.15.43:0/1735993465 >> 172.21.15.43:0/1735993465 conn(0x5565654bbc90 msgr2=0x5565654ef850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.668+0000 7fe52f9f1740 1 -- 172.21.15.43:0/1735993465 shutdown_connections 2024-04-21T08:51:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.668+0000 7fe52f9f1740 1 -- 172.21.15.43:0/1735993465 wait complete. 2024-04-21T08:51:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.668+0000 7fe52f9f1740 1 -- start start 2024-04-21T08:51:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.668+0000 7fe52f9f1740 1 Processor -- start 2024-04-21T08:51:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.668+0000 7fe52f9f1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f4b60 0x5565654fdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.669+0000 7fe52f9f1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565654f58d0 0x5565654faea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.669+0000 7fe52f9f1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565654fe0e0 0x5565654fb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.669+0000 7fe52f9f1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556565500e10 con 0x5565654f4b60 2024-04-21T08:51:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.669+0000 7fe52f9f1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5565653e1eb0 con 0x5565654fe0e0 2024-04-21T08:51:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.669+0000 7fe52f9f1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556565500f90 con 0x5565654f58d0 2024-04-21T08:51:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.669+0000 7fe52f9f1740 10 client.0 _ll_register_callbacks cb 0x55656540eb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.669+0000 7fe52659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f4b60 0x5565654fdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.669+0000 7fe52659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f4b60 0x5565654fdb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52256/0 (socket says 172.21.15.43:52256) 2024-04-21T08:51:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.669+0000 7fe52659c640 1 -- 172.21.15.43:0/2395984682 learned_addr learned my addr 172.21.15.43:0/2395984682 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.669+0000 7fe526d9d640 1 --2- 172.21.15.43:0/2395984682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565654fe0e0 0x5565654fb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.669+0000 7fe525d9b640 1 --2- 172.21.15.43:0/2395984682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565654f58d0 0x5565654faea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.669+0000 7fe52659c640 1 -- 172.21.15.43:0/2395984682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565654f58d0 msgr2=0x5565654faea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.669+0000 7fe52659c640 1 --2- 172.21.15.43:0/2395984682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565654f58d0 0x5565654faea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.669+0000 7fe52659c640 1 -- 172.21.15.43:0/2395984682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565654fe0e0 msgr2=0x5565654fb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.669+0000 7fe52659c640 1 --2- 172.21.15.43:0/2395984682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565654fe0e0 0x5565654fb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.669+0000 7fe52659c640 1 -- 172.21.15.43:0/2395984682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55656516c210 con 0x5565654f4b60 2024-04-21T08:51:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.669+0000 7fe526d9d640 1 --2- 172.21.15.43:0/2395984682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565654fe0e0 0x5565654fb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.669+0000 7fe52659c640 1 --2- 172.21.15.43:0/2395984682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f4b60 0x5565654fdb10 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7fe5180cda20 tx=0x7fe518021c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=851ecdd13e3bc581 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.669+0000 7fe50dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.669+0000 7fe50dffb640 1 -- 172.21.15.43:0/2395984682 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5180b4020 con 0x5565654f4b60 2024-04-21T08:51:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.669+0000 7fe50dffb640 1 -- 172.21.15.43:0/2395984682 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fe5180041d0 con 0x5565654f4b60 2024-04-21T08:51:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.669+0000 7fe52f9f1740 1 -- 172.21.15.43:0/2395984682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe520013d10 con 0x5565654f4b60 2024-04-21T08:51:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.669+0000 7fe50dffb640 1 -- 172.21.15.43:0/2395984682 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe5180056a0 con 0x5565654f4b60 2024-04-21T08:51:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.670+0000 7fe52f9f1740 10 client.21396 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.670+0000 7fe52f9f1740 1 -- 172.21.15.43:0/2395984682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556565238620 con 0x5565654f4b60 2024-04-21T08:51:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.670+0000 7fe50dffb640 1 -- 172.21.15.43:0/2395984682 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe520013d10 con 0x5565654f4b60 2024-04-21T08:51:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.670+0000 7fe50dffb640 1 -- 172.21.15.43:0/2395984682 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe518098880 con 0x5565654f4b60 2024-04-21T08:51:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.670+0000 7fe52f9f1740 10 client.21396 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.670+0000 7fe52f9f1740 10 client.21396 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.670+0000 7fe52f9f1740 20 client.21396 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.670+0000 7fe52f9f1740 1 --2- 172.21.15.43:0/2395984682 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55656550e190 0x556565510580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.670+0000 7fe52f9f1740 4 client.21396 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.670+0000 7fe52f9f1740 1 -- 172.21.15.43:0/2395984682 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55656546d880 con 0x55656550e190 2024-04-21T08:51:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.670+0000 7fe525d9b640 1 --2- 172.21.15.43:0/2395984682 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55656550e190 0x556565510580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.671+0000 7fe525d9b640 1 --2- 172.21.15.43:0/2395984682 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55656550e190 0x556565510580 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df58c6bb7d5eda49 server_cookie=4c14bff41c3aad2b in_seq=0 out_seq=0 2024-04-21T08:51:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.671+0000 7fe50dffb640 10 client.21396 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.671+0000 7fe50dffb640 1 -- 172.21.15.43:0/2395984682 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55656546d880 con 0x55656550e190 2024-04-21T08:51:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.671+0000 7fe50dffb640 10 client.21396 handle_command_reply: tid=0 2024-04-21T08:51:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.672+0000 7fe52f9f1740 1 client.21396 shutdown 2024-04-21T08:51:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.672+0000 7fe52f9f1740 1 -- 172.21.15.43:0/2395984682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f4b60 msgr2=0x5565654fdb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.672+0000 7fe52f9f1740 1 --2- 172.21.15.43:0/2395984682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f4b60 0x5565654fdb10 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7fe5180cda20 tx=0x7fe518021c50 comp rx=0 tx=0).stop 2024-04-21T08:51:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.672+0000 7fe52f9f1740 1 -- 172.21.15.43:0/2395984682 shutdown_connections 2024-04-21T08:51:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.672+0000 7fe52f9f1740 1 --2- 172.21.15.43:0/2395984682 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55656550e190 0x556565510580 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.672+0000 7fe52f9f1740 1 --2- 172.21.15.43:0/2395984682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565654fe0e0 0x5565654fb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.672+0000 7fe52f9f1740 1 --2- 172.21.15.43:0/2395984682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565654f58d0 0x5565654faea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.672+0000 7fe50dffb640 0 client.21396 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.672+0000 7fe52f9f1740 1 --2- 172.21.15.43:0/2395984682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f4b60 0x5565654fdb10 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.672+0000 7fe52f9f1740 1 -- 172.21.15.43:0/2395984682 >> 172.21.15.43:0/2395984682 conn(0x5565654bbc90 msgr2=0x5565654f15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.672+0000 7fe52f9f1740 1 -- 172.21.15.43:0/2395984682 shutdown_connections 2024-04-21T08:51:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.672+0000 7fe52f9f1740 1 -- 172.21.15.43:0/2395984682 wait complete. 2024-04-21T08:51:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.672+0000 7fe52f9f1740 20 client.21396 trim_cache size 0 max 16384 2024-04-21T08:51:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.687+0000 7fe52f9f1740 1 Processor -- start 2024-04-21T08:51:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.687+0000 7fe52f9f1740 1 -- start start 2024-04-21T08:51:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.687+0000 7fe52f9f1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f4b60 0x556565509d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.687+0000 7fe52f9f1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565654f58d0 0x556565515e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.687+0000 7fe52f9f1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565654fe0e0 0x556565516380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.687+0000 7fe52f9f1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe5281af500 con 0x5565654f4b60 2024-04-21T08:51:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.687+0000 7fe52f9f1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe5281b1040 con 0x5565654fe0e0 2024-04-21T08:51:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.687+0000 7fe52f9f1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe5281afbd0 con 0x5565654f58d0 2024-04-21T08:51:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.687+0000 7fe52659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f4b60 0x556565509d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.687+0000 7fe52659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f4b60 0x556565509d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52266/0 (socket says 172.21.15.43:52266) 2024-04-21T08:51:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.687+0000 7fe52659c640 1 -- 172.21.15.43:0/2754994751 learned_addr learned my addr 172.21.15.43:0/2754994751 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.687+0000 7fe526d9d640 1 --2- 172.21.15.43:0/2754994751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565654fe0e0 0x556565516380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.688+0000 7fe525d9b640 1 --2- 172.21.15.43:0/2754994751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565654f58d0 0x556565515e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.688+0000 7fe52659c640 1 -- 172.21.15.43:0/2754994751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565654f58d0 msgr2=0x556565515e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.688+0000 7fe52659c640 1 --2- 172.21.15.43:0/2754994751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565654f58d0 0x556565515e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.688+0000 7fe52659c640 1 -- 172.21.15.43:0/2754994751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565654fe0e0 msgr2=0x556565516380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.688+0000 7fe52659c640 1 --2- 172.21.15.43:0/2754994751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565654fe0e0 0x556565516380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.688+0000 7fe52659c640 1 -- 172.21.15.43:0/2754994751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565654fb950 con 0x5565654f4b60 2024-04-21T08:51:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.688+0000 7fe526d9d640 1 --2- 172.21.15.43:0/2754994751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565654fe0e0 0x556565516380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.688+0000 7fe525d9b640 1 --2- 172.21.15.43:0/2754994751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565654f58d0 0x556565515e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.688+0000 7fe52659c640 1 --2- 172.21.15.43:0/2754994751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f4b60 0x556565509d40 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7fe5180a4950 tx=0x7fe51809a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c642171ca0f19f97 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.688+0000 7fe50ffff640 1 -- 172.21.15.43:0/2754994751 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5180984b0 con 0x5565654f4b60 2024-04-21T08:51:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.688+0000 7fe50ffff640 1 -- 172.21.15.43:0/2754994751 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe51807f070 con 0x5565654f4b60 2024-04-21T08:51:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.688+0000 7fe50ffff640 1 -- 172.21.15.43:0/2754994751 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51809bb70 con 0x5565654f4b60 2024-04-21T08:51:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.688+0000 7fe52f9f1740 1 -- 172.21.15.43:0/2754994751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f4b60 msgr2=0x556565509d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.688+0000 7fe52f9f1740 1 --2- 172.21.15.43:0/2754994751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f4b60 0x556565509d40 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7fe5180a4950 tx=0x7fe51809a720 comp rx=0 tx=0).stop 2024-04-21T08:51:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.689+0000 7fe52f9f1740 1 -- 172.21.15.43:0/2754994751 shutdown_connections 2024-04-21T08:51:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.689+0000 7fe52f9f1740 1 --2- 172.21.15.43:0/2754994751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565654fe0e0 0x556565516380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.689+0000 7fe52f9f1740 1 --2- 172.21.15.43:0/2754994751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565654f58d0 0x556565515e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.689+0000 7fe52f9f1740 1 --2- 172.21.15.43:0/2754994751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f4b60 0x556565509d40 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.689+0000 7fe52f9f1740 1 -- 172.21.15.43:0/2754994751 >> 172.21.15.43:0/2754994751 conn(0x5565654bbc90 msgr2=0x5565654f6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.689+0000 7fe52f9f1740 1 -- 172.21.15.43:0/2754994751 shutdown_connections 2024-04-21T08:51:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.689+0000 7fe52f9f1740 1 -- 172.21.15.43:0/2754994751 wait complete. 2024-04-21T08:51:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.689+0000 7fe52f9f1740 1 -- start start 2024-04-21T08:51:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.689+0000 7fe52f9f1740 1 Processor -- start 2024-04-21T08:51:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.689+0000 7fe52f9f1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f4b60 0x5565654f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.689+0000 7fe52f9f1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565654f58d0 0x5565654f80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.689+0000 7fe52f9f1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565654fe0e0 0x5565654f85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.689+0000 7fe52f9f1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe5281b0970 con 0x5565654f4b60 2024-04-21T08:51:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.689+0000 7fe52f9f1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe5281b02a0 con 0x5565654fe0e0 2024-04-21T08:51:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.689+0000 7fe52f9f1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe5281ad500 con 0x5565654f58d0 2024-04-21T08:51:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.689+0000 7fe52f9f1740 10 client.0 _ll_register_callbacks cb 0x7fe518005cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.689+0000 7fe52659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f4b60 0x5565654f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.689+0000 7fe52659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f4b60 0x5565654f7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52282/0 (socket says 172.21.15.43:52282) 2024-04-21T08:51:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.689+0000 7fe52659c640 1 -- 172.21.15.43:0/302581539 learned_addr learned my addr 172.21.15.43:0/302581539 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.689+0000 7fe526d9d640 1 --2- 172.21.15.43:0/302581539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565654fe0e0 0x5565654f85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.690+0000 7fe52659c640 1 -- 172.21.15.43:0/302581539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565654f58d0 msgr2=0x5565654f80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.690+0000 7fe525d9b640 1 --2- 172.21.15.43:0/302581539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565654f58d0 0x5565654f80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.690+0000 7fe52659c640 1 --2- 172.21.15.43:0/302581539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565654f58d0 0x5565654f80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.690+0000 7fe52659c640 1 -- 172.21.15.43:0/302581539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565654fe0e0 msgr2=0x5565654f85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.690+0000 7fe52659c640 1 --2- 172.21.15.43:0/302581539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565654fe0e0 0x5565654f85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.690+0000 7fe52659c640 1 -- 172.21.15.43:0/302581539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556565573700 con 0x5565654f4b60 2024-04-21T08:51:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.690+0000 7fe526d9d640 1 --2- 172.21.15.43:0/302581539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565654fe0e0 0x5565654f85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.690+0000 7fe52659c640 1 --2- 172.21.15.43:0/302581539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f4b60 0x5565654f7b70 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7fe51809c3d0 tx=0x7fe51809c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb86ac2cbf0beb9e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.690+0000 7fe50dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.690+0000 7fe50dffb640 1 -- 172.21.15.43:0/302581539 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5180984b0 con 0x5565654f4b60 2024-04-21T08:51:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.690+0000 7fe50dffb640 1 -- 172.21.15.43:0/302581539 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fe5180a0170 con 0x5565654f4b60 2024-04-21T08:51:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.690+0000 7fe52f9f1740 1 -- 172.21.15.43:0/302581539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe51807f070 con 0x5565654f4b60 2024-04-21T08:51:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.690+0000 7fe50dffb640 1 -- 172.21.15.43:0/302581539 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe5180056a0 con 0x5565654f4b60 2024-04-21T08:51:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.690+0000 7fe52f9f1740 10 client.21402 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.690+0000 7fe52f9f1740 1 -- 172.21.15.43:0/302581539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556565573700 con 0x5565654f4b60 2024-04-21T08:51:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.690+0000 7fe50dffb640 1 -- 172.21.15.43:0/302581539 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe51807f070 con 0x5565654f4b60 2024-04-21T08:51:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.690+0000 7fe50dffb640 1 -- 172.21.15.43:0/302581539 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe518097f60 con 0x5565654f4b60 2024-04-21T08:51:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.690+0000 7fe52f9f1740 10 client.21402 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.690+0000 7fe52f9f1740 10 client.21402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.690+0000 7fe52f9f1740 20 client.21402 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.691+0000 7fe52f9f1740 1 --2- 172.21.15.43:0/302581539 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556565574c50 0x556565577070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.691+0000 7fe52f9f1740 4 client.21402 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:51:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.691+0000 7fe52f9f1740 1 -- 172.21.15.43:0/302581539 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5565653f93d0 con 0x556565574c50 2024-04-21T08:51:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.691+0000 7fe525d9b640 1 --2- 172.21.15.43:0/302581539 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556565574c50 0x556565577070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.691+0000 7fe525d9b640 1 --2- 172.21.15.43:0/302581539 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556565574c50 0x556565577070 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e42862bd1e36fdcd server_cookie=3660e7e9ceb082b8 in_seq=0 out_seq=0 2024-04-21T08:51:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.691+0000 7fe50dffb640 10 client.21402 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.706+0000 7fe50dffb640 1 -- 172.21.15.43:0/302581539 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5565653f93d0 con 0x556565574c50 2024-04-21T08:51:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.706+0000 7fe50dffb640 10 client.21402 handle_command_reply: tid=0 2024-04-21T08:51:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.706+0000 7fe52f9f1740 1 client.21402 shutdown 2024-04-21T08:51:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.706+0000 7fe52f9f1740 1 -- 172.21.15.43:0/302581539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f4b60 msgr2=0x5565654f7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.706+0000 7fe52f9f1740 1 --2- 172.21.15.43:0/302581539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f4b60 0x5565654f7b70 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7fe51809c3d0 tx=0x7fe51809c280 comp rx=0 tx=0).stop 2024-04-21T08:51:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.706+0000 7fe52f9f1740 1 -- 172.21.15.43:0/302581539 shutdown_connections 2024-04-21T08:51:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.706+0000 7fe52f9f1740 1 --2- 172.21.15.43:0/302581539 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x556565574c50 0x556565577070 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.706+0000 7fe52f9f1740 1 --2- 172.21.15.43:0/302581539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565654fe0e0 0x5565654f85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.706+0000 7fe52f9f1740 1 --2- 172.21.15.43:0/302581539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5565654f58d0 0x5565654f80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.706+0000 7fe52f9f1740 1 --2- 172.21.15.43:0/302581539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565654f4b60 0x5565654f7b70 unknown :-1 s=CLOSED pgs=2358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.706+0000 7fe50dffb640 0 client.21402 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.706+0000 7fe52f9f1740 1 -- 172.21.15.43:0/302581539 >> 172.21.15.43:0/302581539 conn(0x5565654bbc90 msgr2=0x5565654f0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.706+0000 7fe52f9f1740 1 -- 172.21.15.43:0/302581539 shutdown_connections 2024-04-21T08:51:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.706+0000 7fe52f9f1740 1 -- 172.21.15.43:0/302581539 wait complete. 2024-04-21T08:51:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.706+0000 7fe52f9f1740 20 client.21402 trim_cache size 0 max 16384 2024-04-21T08:51:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-04-21T08:51:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-04-21T08:51:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-04-21T08:51:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-04-21T08:51:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.707+0000 7fe52cd2c640 1 -- 172.21.15.43:0/62328078 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe5100a3dc0 msgr2=0x7fe5100a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.707+0000 7fe52cd2c640 1 --2- 172.21.15.43:0/62328078 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe5100a3dc0 0x7fe5100a6280 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe51c006010 tx=0x7fe51c005a20 comp rx=0 tx=0).stop 2024-04-21T08:51:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.707+0000 7fe52cd2c640 1 -- 172.21.15.43:0/62328078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52806a7c0 msgr2=0x7fe528198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.707+0000 7fe52cd2c640 1 --2- 172.21.15.43:0/62328078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52806a7c0 0x7fe528198690 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7fe51800d950 tx=0x7fe51800de10 comp rx=0 tx=0).stop 2024-04-21T08:51:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.707+0000 7fe52cd2c640 1 -- 172.21.15.43:0/62328078 shutdown_connections 2024-04-21T08:51:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.707+0000 7fe52cd2c640 1 --2- 172.21.15.43:0/62328078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe528108020 0x7fe52819cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.707+0000 7fe52cd2c640 1 --2- 172.21.15.43:0/62328078 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe5100a3dc0 0x7fe5100a6280 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.707+0000 7fe52cd2c640 1 --2- 172.21.15.43:0/62328078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe52806b160 0x7fe528198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.707+0000 7fe52cd2c640 1 --2- 172.21.15.43:0/62328078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52806a7c0 0x7fe528198690 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.707+0000 7fe52cd2c640 1 -- 172.21.15.43:0/62328078 >> 172.21.15.43:0/62328078 conn(0x7fe528076c50 msgr2=0x7fe5280ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.707+0000 7fe52cd2c640 1 -- 172.21.15.43:0/62328078 shutdown_connections 2024-04-21T08:51:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-04-21T08:51:19.707+0000 7fe52cd2c640 1 -- 172.21.15.43:0/62328078 wait complete. 2024-04-21T08:51:19.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:19 smithi175 ceph-mon[31920]: scrub summary: idle 2024-04-21T08:51:20.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:19 smithi112 ceph-mon[31085]: scrub summary: idle 2024-04-21T08:51:20.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:19 smithi043 ceph-mon[27978]: scrub summary: idle 2024-04-21T08:51:20.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:20 smithi175 ceph-mon[31920]: pgmap v388: 129 pgs: 129 active+clean; 2.6 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 20 op/s 2024-04-21T08:51:21.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:20 smithi112 ceph-mon[31085]: pgmap v388: 129 pgs: 129 active+clean; 2.6 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 20 op/s 2024-04-21T08:51:21.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:20 smithi043 ceph-mon[27978]: pgmap v388: 129 pgs: 129 active+clean; 2.6 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 20 op/s 2024-04-21T08:51:22.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:22 smithi175 ceph-mon[31920]: pgmap v389: 129 pgs: 129 active+clean; 2.8 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 72 KiB/s wr, 28 op/s 2024-04-21T08:51:23.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:22 smithi112 ceph-mon[31085]: pgmap v389: 129 pgs: 129 active+clean; 2.8 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 72 KiB/s wr, 28 op/s 2024-04-21T08:51:23.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:22 smithi043 ceph-mon[27978]: pgmap v389: 129 pgs: 129 active+clean; 2.8 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 72 KiB/s wr, 28 op/s 2024-04-21T08:51:24.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:24 smithi175 ceph-mon[31920]: pgmap v390: 129 pgs: 129 active+clean; 2.8 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:51:25.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:24 smithi112 ceph-mon[31085]: pgmap v390: 129 pgs: 129 active+clean; 2.8 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:51:25.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:24 smithi043 ceph-mon[27978]: pgmap v390: 129 pgs: 129 active+clean; 2.8 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:51:25.951 INFO:tasks.workunit.client.0.smithi043.stdout:561/600: open fd = 3 2024-04-21T08:51:25.951 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.951 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.951 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.951 INFO:tasks.workunit.client.0.smithi043.stdout:562/600: open fd = 3 2024-04-21T08:51:25.951 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.951 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.951 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.951 INFO:tasks.workunit.client.0.smithi043.stdout:563/600: open fd = 3 2024-04-21T08:51:25.951 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.951 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.951 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.951 INFO:tasks.workunit.client.0.smithi043.stdout:564/600: open fd = 3 2024-04-21T08:51:25.952 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.952 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.952 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.952 INFO:tasks.workunit.client.0.smithi043.stdout:565/600: open fd = 3 2024-04-21T08:51:25.952 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.952 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.952 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.952 INFO:tasks.workunit.client.0.smithi043.stdout:566/600: open fd = 3 2024-04-21T08:51:25.952 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.952 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.952 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.952 INFO:tasks.workunit.client.0.smithi043.stdout:567/600: open fd = 3 2024-04-21T08:51:25.952 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.952 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.952 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.952 INFO:tasks.workunit.client.0.smithi043.stdout:568/600: open fd = 3 2024-04-21T08:51:25.953 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.953 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.953 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.953 INFO:tasks.workunit.client.0.smithi043.stdout:569/600: open fd = 3 2024-04-21T08:51:25.953 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.953 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.953 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.953 INFO:tasks.workunit.client.0.smithi043.stdout:570/600: open fd = 3 2024-04-21T08:51:25.953 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.953 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.953 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.953 INFO:tasks.workunit.client.0.smithi043.stdout:571/600: open fd = 3 2024-04-21T08:51:25.953 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.953 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.954 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.954 INFO:tasks.workunit.client.0.smithi043.stdout:572/600: open fd = 3 2024-04-21T08:51:25.954 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.954 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.954 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.954 INFO:tasks.workunit.client.0.smithi043.stdout:573/600: open fd = 3 2024-04-21T08:51:25.954 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.954 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.954 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.954 INFO:tasks.workunit.client.0.smithi043.stdout:574/600: open fd = 3 2024-04-21T08:51:25.954 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.954 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.954 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.954 INFO:tasks.workunit.client.0.smithi043.stdout:575/600: open fd = 3 2024-04-21T08:51:25.954 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.955 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.955 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.955 INFO:tasks.workunit.client.0.smithi043.stdout:576/600: open fd = 3 2024-04-21T08:51:25.955 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.955 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.955 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.955 INFO:tasks.workunit.client.0.smithi043.stdout:577/600: open fd = 3 2024-04-21T08:51:25.955 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.955 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.955 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.955 INFO:tasks.workunit.client.0.smithi043.stdout:578/600: open fd = 3 2024-04-21T08:51:25.955 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.955 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.955 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.955 INFO:tasks.workunit.client.0.smithi043.stdout:579/600: open fd = 3 2024-04-21T08:51:25.955 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.956 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.956 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.956 INFO:tasks.workunit.client.0.smithi043.stdout:580/600: open fd = 3 2024-04-21T08:51:25.956 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.956 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.956 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.956 INFO:tasks.workunit.client.0.smithi043.stdout:581/600: open fd = 3 2024-04-21T08:51:25.956 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.956 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.956 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.956 INFO:tasks.workunit.client.0.smithi043.stdout:582/600: open fd = 3 2024-04-21T08:51:25.956 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.956 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.956 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.956 INFO:tasks.workunit.client.0.smithi043.stdout:583/600: open fd = 3 2024-04-21T08:51:25.956 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.957 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.957 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.957 INFO:tasks.workunit.client.0.smithi043.stdout:584/600: open fd = 3 2024-04-21T08:51:25.957 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.957 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.957 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.957 INFO:tasks.workunit.client.0.smithi043.stdout:585/600: open fd = 3 2024-04-21T08:51:25.957 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.957 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.957 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.957 INFO:tasks.workunit.client.0.smithi043.stdout:586/600: open fd = 3 2024-04-21T08:51:25.957 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.957 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.957 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.957 INFO:tasks.workunit.client.0.smithi043.stdout:587/600: open fd = 3 2024-04-21T08:51:25.957 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.957 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.958 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.958 INFO:tasks.workunit.client.0.smithi043.stdout:588/600: open fd = 3 2024-04-21T08:51:25.958 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.958 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.958 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.958 INFO:tasks.workunit.client.0.smithi043.stdout:589/600: open fd = 3 2024-04-21T08:51:25.958 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.958 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.958 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.958 INFO:tasks.workunit.client.0.smithi043.stdout:590/600: open fd = 3 2024-04-21T08:51:25.958 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.958 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.958 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.958 INFO:tasks.workunit.client.0.smithi043.stdout:591/600: open fd = 3 2024-04-21T08:51:25.959 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.959 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.959 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.959 INFO:tasks.workunit.client.0.smithi043.stdout:592/600: open fd = 3 2024-04-21T08:51:25.959 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.959 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.959 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.959 INFO:tasks.workunit.client.0.smithi043.stdout:593/600: open fd = 3 2024-04-21T08:51:25.959 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.959 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.959 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.959 INFO:tasks.workunit.client.0.smithi043.stdout:594/600: open fd = 3 2024-04-21T08:51:25.959 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.959 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.959 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.959 INFO:tasks.workunit.client.0.smithi043.stdout:595/600: open fd = 3 2024-04-21T08:51:25.960 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.960 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.960 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.960 INFO:tasks.workunit.client.0.smithi043.stdout:596/600: open fd = 3 2024-04-21T08:51:25.960 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.960 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.960 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.960 INFO:tasks.workunit.client.0.smithi043.stdout:597/600: open fd = 3 2024-04-21T08:51:25.960 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.960 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.960 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.960 INFO:tasks.workunit.client.0.smithi043.stdout:598/600: open fd = 3 2024-04-21T08:51:25.960 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.960 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.960 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.960 INFO:tasks.workunit.client.0.smithi043.stdout:599/600: open fd = 3 2024-04-21T08:51:25.961 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.961 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.961 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.961 INFO:tasks.workunit.client.0.smithi043.stdout:600/600: open fd = 3 2024-04-21T08:51:25.961 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.961 INFO:tasks.workunit.client.0.smithi043.stdout:write ret = 32 2024-04-21T08:51:25.961 INFO:tasks.workunit.client.0.smithi043.stdout:pread ret = 64 2024-04-21T08:51:25.961 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo OK 2024-04-21T08:51:25.962 INFO:tasks.workunit.client.0.smithi043.stdout:OK 2024-04-21T08:51:25.962 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-21T08:51:25.963 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-21T08:51:26.008 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-04-21T08:51:26.008 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-21T08:51:26.307 DEBUG:teuthology.parallel:result is None 2024-04-21T08:51:26.308 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-21T08:51:26.344 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-21T08:51:26.345 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-21T08:51:26.390 INFO:tasks.quiescer:joining Quiescers 2024-04-21T08:51:26.391 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:51:26.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.494+0000 7f6b16a14640 1 -- 172.21.15.43:0/235583424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b10106180 msgr2=0x7f6b1010dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:26.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.494+0000 7f6b16a14640 1 --2- 172.21.15.43:0/235583424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b10106180 0x7f6b1010dd50 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009ca0 tx=0x7f6b0802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.494+0000 7f6b16a14640 1 -- 172.21.15.43:0/235583424 shutdown_connections 2024-04-21T08:51:26.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.494+0000 7f6b16a14640 1 --2- 172.21.15.43:0/235583424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b10106180 0x7f6b1010dd50 unknown :-1 s=CLOSED pgs=2359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.494+0000 7f6b16a14640 1 --2- 172.21.15.43:0/235583424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b10101a60 0x7f6b10105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.494+0000 7f6b16a14640 1 --2- 172.21.15.43:0/235583424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b101010b0 0x7f6b10101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.494+0000 7f6b16a14640 1 -- 172.21.15.43:0/235583424 >> 172.21.15.43:0/235583424 conn(0x7f6b100771d0 msgr2=0x7f6b100775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:26.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.494+0000 7f6b16a14640 1 -- 172.21.15.43:0/235583424 shutdown_connections 2024-04-21T08:51:26.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.494+0000 7f6b16a14640 1 -- 172.21.15.43:0/235583424 wait complete. 2024-04-21T08:51:26.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.495+0000 7f6b16a14640 1 Processor -- start 2024-04-21T08:51:26.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.495+0000 7f6b16a14640 1 -- start start 2024-04-21T08:51:26.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.495+0000 7f6b16a14640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b101010b0 0x7f6b101994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:26.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.495+0000 7f6b16a14640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b10106180 0x7f6b10199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:26.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.495+0000 7f6b16a14640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b10199fc0 0x7f6b101a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:26.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.495+0000 7f6b16a14640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6b101110f0 con 0x7f6b10199fc0 2024-04-21T08:51:26.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.495+0000 7f6b16a14640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6b10110f70 con 0x7f6b101010b0 2024-04-21T08:51:26.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.495+0000 7f6b16a14640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6b10111270 con 0x7f6b10106180 2024-04-21T08:51:26.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.495+0000 7f6b14fb1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b10199fc0 0x7f6b101a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:26.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.495+0000 7f6b14fb1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b10199fc0 0x7f6b101a4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52304/0 (socket says 172.21.15.43:52304) 2024-04-21T08:51:26.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.495+0000 7f6b14fb1640 1 -- 172.21.15.43:0/2662286593 learned_addr learned my addr 172.21.15.43:0/2662286593 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:26.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.495+0000 7f6b0f7fe640 1 --2- 172.21.15.43:0/2662286593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b10106180 0x7f6b10199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:26.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.495+0000 7f6b0ffff640 1 --2- 172.21.15.43:0/2662286593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b101010b0 0x7f6b101994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:26.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.495+0000 7f6b14fb1640 1 -- 172.21.15.43:0/2662286593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b10106180 msgr2=0x7f6b10199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:26.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.495+0000 7f6b14fb1640 1 --2- 172.21.15.43:0/2662286593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b10106180 0x7f6b10199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.495+0000 7f6b14fb1640 1 -- 172.21.15.43:0/2662286593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b101010b0 msgr2=0x7f6b101994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:26.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.495+0000 7f6b14fb1640 1 --2- 172.21.15.43:0/2662286593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b101010b0 0x7f6b101994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.495+0000 7f6b14fb1640 1 -- 172.21.15.43:0/2662286593 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b101a4d70 con 0x7f6b10199fc0 2024-04-21T08:51:26.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.495+0000 7f6b0ffff640 1 --2- 172.21.15.43:0/2662286593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b101010b0 0x7f6b101994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:26.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.496+0000 7f6b14fb1640 1 --2- 172.21.15.43:0/2662286593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b10199fc0 0x7f6b101a4570 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f6b08005e30 tx=0x7f6b08032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:26.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.496+0000 7f6b0d7fa640 1 -- 172.21.15.43:0/2662286593 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08041020 con 0x7f6b10199fc0 2024-04-21T08:51:26.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.496+0000 7f6b0d7fa640 1 -- 172.21.15.43:0/2662286593 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b08032970 con 0x7f6b10199fc0 2024-04-21T08:51:26.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.496+0000 7f6b16a14640 1 -- 172.21.15.43:0/2662286593 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b101a5000 con 0x7f6b10199fc0 2024-04-21T08:51:26.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.496+0000 7f6b0d7fa640 1 -- 172.21.15.43:0/2662286593 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08039040 con 0x7f6b10199fc0 2024-04-21T08:51:26.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.496+0000 7f6b16a14640 1 -- 172.21.15.43:0/2662286593 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b101a54f0 con 0x7f6b10199fc0 2024-04-21T08:51:26.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.497+0000 7f6b16a14640 1 -- 172.21.15.43:0/2662286593 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b10069230 con 0x7f6b10199fc0 2024-04-21T08:51:26.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.498+0000 7f6b0d7fa640 1 -- 172.21.15.43:0/2662286593 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6b08044020 con 0x7f6b10199fc0 2024-04-21T08:51:26.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.499+0000 7f6b0d7fa640 1 --2- 172.21.15.43:0/2662286593 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6af80a3d20 0x7f6af80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:26.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.499+0000 7f6b0ffff640 1 --2- 172.21.15.43:0/2662286593 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6af80a3d20 0x7f6af80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:26.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.499+0000 7f6b0d7fa640 1 -- 172.21.15.43:0/2662286593 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f6b080f2d30 con 0x7f6b10199fc0 2024-04-21T08:51:26.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.499+0000 7f6b0ffff640 1 --2- 172.21.15.43:0/2662286593 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6af80a3d20 0x7f6af80a61e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009b50 tx=0x7f6b00005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:26.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.503+0000 7f6b0d7fa640 1 -- 172.21.15.43:0/2662286593 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6b0804b020 con 0x7f6b10199fc0 2024-04-21T08:51:26.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.731+0000 7f6b16a14640 1 -- 172.21.15.43:0/2662286593 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b10109410 con 0x7f6b10199fc0 2024-04-21T08:51:26.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.732+0000 7f6b0d7fa640 1 -- 172.21.15.43:0/2662286593 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11935 (secure 0 0 0) 0x7f6b080bf0c0 con 0x7f6b10199fc0 2024-04-21T08:51:26.732 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:51:26.732 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:46:17.283806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27556":{"gid":27556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/229985237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":229985237},{"type":"v1","addr":"172.21.15.175:6837","nonce":229985237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24425,24412,24487,24527,24491]},"id":1}]} 2024-04-21T08:51:26.732 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-04-21T08:51:26.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.735+0000 7f6b16a14640 1 -- 172.21.15.43:0/2662286593 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6af80a3d20 msgr2=0x7f6af80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:26.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.735+0000 7f6b16a14640 1 --2- 172.21.15.43:0/2662286593 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6af80a3d20 0x7f6af80a61e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009b50 tx=0x7f6b00005f90 comp rx=0 tx=0).stop 2024-04-21T08:51:26.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.735+0000 7f6b16a14640 1 -- 172.21.15.43:0/2662286593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b10199fc0 msgr2=0x7f6b101a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:26.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.735+0000 7f6b16a14640 1 --2- 172.21.15.43:0/2662286593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b10199fc0 0x7f6b101a4570 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f6b08005e30 tx=0x7f6b08032420 comp rx=0 tx=0).stop 2024-04-21T08:51:26.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.735+0000 7f6b16a14640 1 -- 172.21.15.43:0/2662286593 shutdown_connections 2024-04-21T08:51:26.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.735+0000 7f6b16a14640 1 --2- 172.21.15.43:0/2662286593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b10199fc0 0x7f6b101a4570 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.735+0000 7f6b16a14640 1 --2- 172.21.15.43:0/2662286593 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6af80a3d20 0x7f6af80a61e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.735+0000 7f6b16a14640 1 --2- 172.21.15.43:0/2662286593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b10106180 0x7f6b10199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.735+0000 7f6b16a14640 1 --2- 172.21.15.43:0/2662286593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b101010b0 0x7f6b101994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.735+0000 7f6b16a14640 1 -- 172.21.15.43:0/2662286593 >> 172.21.15.43:0/2662286593 conn(0x7f6b100771d0 msgr2=0x7f6b100fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.735+0000 7f6b16a14640 1 -- 172.21.15.43:0/2662286593 shutdown_connections 2024-04-21T08:51:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.735+0000 7f6b16a14640 1 -- 172.21.15.43:0/2662286593 wait complete. 2024-04-21T08:51:26.746 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db --cancel --all' 2024-04-21T08:51:26.746 DEBUG:teuthology.orchestra.run.smithi043:> timeout 45 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24527 quiesce db --cancel --all 2024-04-21T08:51:26.846 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:26 smithi112 ceph-mon[31085]: pgmap v391: 129 pgs: 129 active+clean; 2.8 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:51:26.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.854+0000 7f1a4916d640 1 -- 172.21.15.43:0/956049852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a44102190 msgr2=0x7f1a44102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:26.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.854+0000 7f1a4916d640 1 --2- 172.21.15.43:0/956049852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a44102190 0x7f1a44102570 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f1a34009900 tx=0x7f1a3402bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:26.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.854+0000 7f1a4916d640 1 -- 172.21.15.43:0/956049852 shutdown_connections 2024-04-21T08:51:26.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.854+0000 7f1a4916d640 1 --2- 172.21.15.43:0/956049852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a44107260 0x7f1a4410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.854+0000 7f1a4916d640 1 --2- 172.21.15.43:0/956049852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a44102b40 0x7f1a44106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.854+0000 7f1a4916d640 1 --2- 172.21.15.43:0/956049852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a44102190 0x7f1a44102570 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.854+0000 7f1a4916d640 1 -- 172.21.15.43:0/956049852 >> 172.21.15.43:0/956049852 conn(0x7f1a440fd990 msgr2=0x7f1a440ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:26.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.855+0000 7f1a4916d640 1 -- 172.21.15.43:0/956049852 shutdown_connections 2024-04-21T08:51:26.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.855+0000 7f1a4916d640 1 -- 172.21.15.43:0/956049852 wait complete. 2024-04-21T08:51:26.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.855+0000 7f1a4916d640 1 Processor -- start 2024-04-21T08:51:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.855+0000 7f1a4916d640 1 -- start start 2024-04-21T08:51:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.856+0000 7f1a4916d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a44102190 0x7f1a4419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.856+0000 7f1a4916d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a44107260 0x7f1a4419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.856+0000 7f1a4916d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a44198600 0x7f1a44198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.856+0000 7f1a4916d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a44112220 con 0x7f1a44107260 2024-04-21T08:51:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.856+0000 7f1a4916d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a441120a0 con 0x7f1a44198600 2024-04-21T08:51:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.856+0000 7f1a4916d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1a441123a0 con 0x7f1a44102190 2024-04-21T08:51:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.856+0000 7f1a4259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a44107260 0x7f1a4419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.856+0000 7f1a4259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a44107260 0x7f1a4419e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42858/0 (socket says 172.21.15.43:42858) 2024-04-21T08:51:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.856+0000 7f1a4259c640 1 -- 172.21.15.43:0/307767252 learned_addr learned my addr 172.21.15.43:0/307767252 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.856+0000 7f1a42d9d640 1 --2- 172.21.15.43:0/307767252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a44102190 0x7f1a4419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.856+0000 7f1a4359e640 1 --2- 172.21.15.43:0/307767252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a44198600 0x7f1a44198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.856+0000 7f1a4259c640 1 -- 172.21.15.43:0/307767252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a44102190 msgr2=0x7f1a4419e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.856+0000 7f1a4259c640 1 --2- 172.21.15.43:0/307767252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a44102190 0x7f1a4419e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.856+0000 7f1a4259c640 1 -- 172.21.15.43:0/307767252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a44198600 msgr2=0x7f1a44198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.856+0000 7f1a4259c640 1 --2- 172.21.15.43:0/307767252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a44198600 0x7f1a44198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.856+0000 7f1a4259c640 1 -- 172.21.15.43:0/307767252 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a44199120 con 0x7f1a44107260 2024-04-21T08:51:26.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.856+0000 7f1a4259c640 1 --2- 172.21.15.43:0/307767252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a44107260 0x7f1a4419e940 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f1a3800d770 tx=0x7f1a3800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:26.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.856+0000 7f1a2bfff640 1 -- 172.21.15.43:0/307767252 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38017070 con 0x7f1a44107260 2024-04-21T08:51:26.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.856+0000 7f1a4916d640 1 -- 172.21.15.43:0/307767252 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a44199410 con 0x7f1a44107260 2024-04-21T08:51:26.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.856+0000 7f1a2bfff640 1 -- 172.21.15.43:0/307767252 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a38009c30 con 0x7f1a44107260 2024-04-21T08:51:26.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.856+0000 7f1a2bfff640 1 -- 172.21.15.43:0/307767252 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3800f620 con 0x7f1a44107260 2024-04-21T08:51:26.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.857+0000 7f1a4916d640 1 -- 172.21.15.43:0/307767252 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a441a5660 con 0x7f1a44107260 2024-04-21T08:51:26.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.859+0000 7f1a2bfff640 1 -- 172.21.15.43:0/307767252 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1a3801a070 con 0x7f1a44107260 2024-04-21T08:51:26.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.859+0000 7f1a4be35740 1 Processor -- start 2024-04-21T08:51:26.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.859+0000 7f1a4be35740 1 -- start start 2024-04-21T08:51:26.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.859+0000 7f1a4be35740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b80ed10 0x55d68b80f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:26.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.859+0000 7f1a4be35740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d68b80f6c0 0x55d68b817c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:26.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.859+0000 7f1a4be35740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68b818240 0x55d68b81a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:26.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.859+0000 7f1a4be35740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d68b774000 con 0x55d68b80ed10 2024-04-21T08:51:26.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.860+0000 7f1a4be35740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d68b710b20 con 0x55d68b818240 2024-04-21T08:51:26.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.860+0000 7f1a4be35740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d68b76ffc0 con 0x55d68b80f6c0 2024-04-21T08:51:26.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.860+0000 7f1a4359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b80ed10 0x55d68b80f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:26.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.860+0000 7f1a4359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b80ed10 0x55d68b80f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42868/0 (socket says 172.21.15.43:42868) 2024-04-21T08:51:26.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.860+0000 7f1a4359e640 1 -- 172.21.15.43:0/2599177971 learned_addr learned my addr 172.21.15.43:0/2599177971 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:26.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.860+0000 7f1a2bfff640 1 --2- 172.21.15.43:0/307767252 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1a2c0a3d20 0x7f1a2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:26.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.860+0000 7f1a42d9d640 1 --2- 172.21.15.43:0/2599177971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d68b80f6c0 0x55d68b817c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:26.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.860+0000 7f1a4359e640 1 --2- 172.21.15.43:0/307767252 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1a2c0a3d20 0x7f1a2c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:26.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.860+0000 7f1a2bfff640 1 -- 172.21.15.43:0/307767252 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f1a380d0820 con 0x7f1a44107260 2024-04-21T08:51:26.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.860+0000 7f1a4259c640 1 --2- 172.21.15.43:0/2599177971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68b818240 0x55d68b81a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:26.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.860+0000 7f1a4359e640 1 -- 172.21.15.43:0/2599177971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d68b80f6c0 msgr2=0x55d68b817c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:26.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.860+0000 7f1a4359e640 1 --2- 172.21.15.43:0/2599177971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d68b80f6c0 0x55d68b817c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.860+0000 7f1a4359e640 1 -- 172.21.15.43:0/2599177971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68b818240 msgr2=0x55d68b81a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:26.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.860+0000 7f1a4359e640 1 --2- 172.21.15.43:0/2599177971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68b818240 0x55d68b81a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.860+0000 7f1a4359e640 1 -- 172.21.15.43:0/2599177971 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d68b72aea0 con 0x55d68b80ed10 2024-04-21T08:51:26.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.860+0000 7f1a4259c640 1 --2- 172.21.15.43:0/2599177971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68b818240 0x55d68b81a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:26.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.860+0000 7f1a4359e640 1 --2- 172.21.15.43:0/2599177971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b80ed10 0x55d68b80f0f0 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0134b0 tx=0x7f1a3c013970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7f70bc8789bbc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:26.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.860+0000 7f1a297fa640 1 -- 172.21.15.43:0/2599177971 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c022070 con 0x55d68b80ed10 2024-04-21T08:51:26.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.861+0000 7f1a297fa640 1 -- 172.21.15.43:0/2599177971 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a3c003e50 con 0x55d68b80ed10 2024-04-21T08:51:26.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.861+0000 7f1a297fa640 1 -- 172.21.15.43:0/2599177971 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c01b840 con 0x55d68b80ed10 2024-04-21T08:51:26.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.861+0000 7f1a4359e640 1 --2- 172.21.15.43:0/307767252 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1a2c0a3d20 0x7f1a2c0a61e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c01f000 tx=0x7f1a3c019000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:26.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.861+0000 7f1a4be35740 1 -- 172.21.15.43:0/2599177971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b80ed10 msgr2=0x55d68b80f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:26.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.861+0000 7f1a4be35740 1 --2- 172.21.15.43:0/2599177971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b80ed10 0x55d68b80f0f0 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0134b0 tx=0x7f1a3c013970 comp rx=0 tx=0).stop 2024-04-21T08:51:26.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.861+0000 7f1a4be35740 1 -- 172.21.15.43:0/2599177971 shutdown_connections 2024-04-21T08:51:26.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.861+0000 7f1a4be35740 1 --2- 172.21.15.43:0/2599177971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68b818240 0x55d68b81a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.861+0000 7f1a4be35740 1 --2- 172.21.15.43:0/2599177971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d68b80f6c0 0x55d68b817c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.861+0000 7f1a4be35740 1 --2- 172.21.15.43:0/2599177971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b80ed10 0x55d68b80f0f0 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.861+0000 7f1a4be35740 1 -- 172.21.15.43:0/2599177971 >> 172.21.15.43:0/2599177971 conn(0x55d68b8093e0 msgr2=0x55d68b8097c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:26.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.861+0000 7f1a4be35740 1 -- 172.21.15.43:0/2599177971 shutdown_connections 2024-04-21T08:51:26.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.861+0000 7f1a4be35740 1 -- 172.21.15.43:0/2599177971 wait complete. 2024-04-21T08:51:26.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.862+0000 7f1a4be35740 1 -- start start 2024-04-21T08:51:26.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.862+0000 7f1a4be35740 1 Processor -- start 2024-04-21T08:51:26.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.862+0000 7f1a4be35740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b80ed10 0x55d68b815140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:26.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.862+0000 7f1a4be35740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68b80f6c0 0x55d68b815680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:26.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.862+0000 7f1a4be35740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d68b818240 0x55d68b815bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:26.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.862+0000 7f1a4be35740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d68b81b0c0 con 0x55d68b80ed10 2024-04-21T08:51:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.862+0000 7f1a4be35740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d68b70e510 con 0x55d68b80f6c0 2024-04-21T08:51:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.862+0000 7f1a4be35740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d68b81b240 con 0x55d68b818240 2024-04-21T08:51:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.862+0000 7f1a4be35740 10 client.0 _ll_register_callbacks cb 0x55d68b70df00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.862+0000 7f1a42d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b80ed10 0x55d68b815140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.862+0000 7f1a42d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b80ed10 0x55d68b815140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42880/0 (socket says 172.21.15.43:42880) 2024-04-21T08:51:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.862+0000 7f1a42d9d640 1 -- 172.21.15.43:0/345162322 learned_addr learned my addr 172.21.15.43:0/345162322 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.862+0000 7f1a4259c640 1 --2- 172.21.15.43:0/345162322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68b80f6c0 0x55d68b815680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.862+0000 7f1a4359e640 1 --2- 172.21.15.43:0/345162322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d68b818240 0x55d68b815bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.862+0000 7f1a42d9d640 1 -- 172.21.15.43:0/345162322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d68b818240 msgr2=0x55d68b815bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.862+0000 7f1a42d9d640 1 --2- 172.21.15.43:0/345162322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d68b818240 0x55d68b815bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.862+0000 7f1a42d9d640 1 -- 172.21.15.43:0/345162322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68b80f6c0 msgr2=0x55d68b815680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.862+0000 7f1a42d9d640 1 --2- 172.21.15.43:0/345162322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68b80f6c0 0x55d68b815680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.862+0000 7f1a42d9d640 1 -- 172.21.15.43:0/345162322 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d68b552a20 con 0x55d68b80ed10 2024-04-21T08:51:26.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.862+0000 7f1a4259c640 1 --2- 172.21.15.43:0/345162322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68b80f6c0 0x55d68b815680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:26.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.863+0000 7f1a42d9d640 1 --2- 172.21.15.43:0/345162322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b80ed10 0x55d68b815140 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7f1a34009900 tx=0x7f1a3402ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff3873fda9e4eb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:26.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.863+0000 7f1a26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:26.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.863+0000 7f1a26ff5640 1 -- 172.21.15.43:0/345162322 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a34005230 con 0x55d68b80ed10 2024-04-21T08:51:26.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.863+0000 7f1a26ff5640 1 -- 172.21.15.43:0/345162322 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f1a340053d0 con 0x55d68b80ed10 2024-04-21T08:51:26.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.863+0000 7f1a4be35740 1 -- 172.21.15.43:0/345162322 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a3c003e50 con 0x55d68b80ed10 2024-04-21T08:51:26.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.863+0000 7f1a26ff5640 1 -- 172.21.15.43:0/345162322 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1a3402c5d0 con 0x55d68b80ed10 2024-04-21T08:51:26.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.863+0000 7f1a4be35740 10 client.21420 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:26.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.863+0000 7f1a4be35740 1 -- 172.21.15.43:0/345162322 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d68b5b31b0 con 0x55d68b80ed10 2024-04-21T08:51:26.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.863+0000 7f1a26ff5640 1 -- 172.21.15.43:0/345162322 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a3c003e50 con 0x55d68b80ed10 2024-04-21T08:51:26.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.863+0000 7f1a26ff5640 1 -- 172.21.15.43:0/345162322 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1a34003780 con 0x55d68b80ed10 2024-04-21T08:51:26.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.863+0000 7f1a4be35740 10 client.21420 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:26.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.863+0000 7f1a4be35740 10 client.21420 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-21T08:51:26.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.863+0000 7f1a4be35740 20 client.21420 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:26.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.863+0000 7f1a4be35740 1 --2- 172.21.15.43:0/345162322 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d68b8285d0 0x55d68b82aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:26.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.863+0000 7f1a4be35740 4 client.21420 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:26.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.863+0000 7f1a4be35740 1 -- 172.21.15.43:0/345162322 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d68b781a20 con 0x55d68b8285d0 2024-04-21T08:51:26.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.864+0000 7f1a4259c640 1 --2- 172.21.15.43:0/345162322 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d68b8285d0 0x55d68b82aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:26.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.864+0000 7f1a4259c640 1 --2- 172.21.15.43:0/345162322 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d68b8285d0 0x55d68b82aa20 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7d794daf580a71 server_cookie=ff25c992171613c in_seq=0 out_seq=0 2024-04-21T08:51:26.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.864+0000 7f1a26ff5640 10 client.21420 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:26.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.865+0000 7f1a26ff5640 1 -- 172.21.15.43:0/345162322 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d68b781a20 con 0x55d68b8285d0 2024-04-21T08:51:26.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.865+0000 7f1a26ff5640 10 client.21420 handle_command_reply: tid=0 2024-04-21T08:51:26.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.865+0000 7f1a4be35740 1 client.21420 shutdown 2024-04-21T08:51:26.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.865+0000 7f1a4be35740 1 -- 172.21.15.43:0/345162322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b80ed10 msgr2=0x55d68b815140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:26.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.865+0000 7f1a4be35740 1 --2- 172.21.15.43:0/345162322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b80ed10 0x55d68b815140 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7f1a34009900 tx=0x7f1a3402ca10 comp rx=0 tx=0).stop 2024-04-21T08:51:26.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.865+0000 7f1a4be35740 1 -- 172.21.15.43:0/345162322 shutdown_connections 2024-04-21T08:51:26.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.865+0000 7f1a4be35740 1 --2- 172.21.15.43:0/345162322 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d68b8285d0 0x55d68b82aa20 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.865+0000 7f1a4be35740 1 --2- 172.21.15.43:0/345162322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d68b818240 0x55d68b815bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.865+0000 7f1a26ff5640 0 client.21420 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:26.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.865+0000 7f1a4be35740 1 --2- 172.21.15.43:0/345162322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68b80f6c0 0x55d68b815680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.865+0000 7f1a4be35740 1 --2- 172.21.15.43:0/345162322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b80ed10 0x55d68b815140 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.865+0000 7f1a4be35740 1 -- 172.21.15.43:0/345162322 >> 172.21.15.43:0/345162322 conn(0x55d68b8093e0 msgr2=0x55d68b808680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:26.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.865+0000 7f1a4be35740 1 -- 172.21.15.43:0/345162322 shutdown_connections 2024-04-21T08:51:26.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.865+0000 7f1a4be35740 1 -- 172.21.15.43:0/345162322 wait complete. 2024-04-21T08:51:26.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.865+0000 7f1a4be35740 20 client.21420 trim_cache size 0 max 16384 2024-04-21T08:51:26.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.883+0000 7f1a4be35740 1 Processor -- start 2024-04-21T08:51:26.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.883+0000 7f1a4be35740 1 -- start start 2024-04-21T08:51:26.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.883+0000 7f1a4be35740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b8093e0 0x55d68b808460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:26.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.883+0000 7f1a4be35740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d68b80ed10 0x55d68b824030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:26.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.883+0000 7f1a4be35740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68b80f6c0 0x55d68b813a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:26.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.883+0000 7f1a4be35740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a441af6c0 con 0x55d68b8093e0 2024-04-21T08:51:26.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.883+0000 7f1a4be35740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a441b1200 con 0x55d68b80f6c0 2024-04-21T08:51:26.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.883+0000 7f1a4be35740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1a441afd90 con 0x55d68b80ed10 2024-04-21T08:51:26.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.883+0000 7f1a42d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b8093e0 0x55d68b808460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:26.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.883+0000 7f1a42d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b8093e0 0x55d68b808460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42888/0 (socket says 172.21.15.43:42888) 2024-04-21T08:51:26.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.883+0000 7f1a42d9d640 1 -- 172.21.15.43:0/408700449 learned_addr learned my addr 172.21.15.43:0/408700449 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:26.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.883+0000 7f1a4359e640 1 --2- 172.21.15.43:0/408700449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68b80f6c0 0x55d68b813a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:26.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.883+0000 7f1a42d9d640 1 -- 172.21.15.43:0/408700449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d68b80ed10 msgr2=0x55d68b824030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:26.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.883+0000 7f1a42d9d640 1 --2- 172.21.15.43:0/408700449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d68b80ed10 0x55d68b824030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.883+0000 7f1a42d9d640 1 -- 172.21.15.43:0/408700449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68b80f6c0 msgr2=0x55d68b813a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:26.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.883+0000 7f1a42d9d640 1 --2- 172.21.15.43:0/408700449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68b80f6c0 0x55d68b813a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.883+0000 7f1a42d9d640 1 -- 172.21.15.43:0/408700449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d68b80f460 con 0x55d68b8093e0 2024-04-21T08:51:26.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.883+0000 7f1a42d9d640 1 --2- 172.21.15.43:0/408700449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b8093e0 0x55d68b808460 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f1a34009850 tx=0x7f1a34005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a22fd82d5d861dd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:26.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.884+0000 7f1a28ff9640 1 -- 172.21.15.43:0/408700449 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a340035f0 con 0x55d68b8093e0 2024-04-21T08:51:26.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.884+0000 7f1a28ff9640 1 -- 172.21.15.43:0/408700449 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a340054d0 con 0x55d68b8093e0 2024-04-21T08:51:26.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.884+0000 7f1a28ff9640 1 -- 172.21.15.43:0/408700449 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a340046c0 con 0x55d68b8093e0 2024-04-21T08:51:26.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.884+0000 7f1a4be35740 1 -- 172.21.15.43:0/408700449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b8093e0 msgr2=0x55d68b808460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:26.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.884+0000 7f1a4be35740 1 --2- 172.21.15.43:0/408700449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b8093e0 0x55d68b808460 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f1a34009850 tx=0x7f1a34005c00 comp rx=0 tx=0).stop 2024-04-21T08:51:26.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.884+0000 7f1a4be35740 1 -- 172.21.15.43:0/408700449 shutdown_connections 2024-04-21T08:51:26.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.884+0000 7f1a4be35740 1 --2- 172.21.15.43:0/408700449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68b80f6c0 0x55d68b813a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.884+0000 7f1a4be35740 1 --2- 172.21.15.43:0/408700449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d68b80ed10 0x55d68b824030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.884+0000 7f1a4be35740 1 --2- 172.21.15.43:0/408700449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b8093e0 0x55d68b808460 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.884+0000 7f1a4be35740 1 -- 172.21.15.43:0/408700449 >> 172.21.15.43:0/408700449 conn(0x55d68b810b10 msgr2=0x55d68b80b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:26.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.884+0000 7f1a4be35740 1 -- 172.21.15.43:0/408700449 shutdown_connections 2024-04-21T08:51:26.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.884+0000 7f1a4be35740 1 -- 172.21.15.43:0/408700449 wait complete. 2024-04-21T08:51:26.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.884+0000 7f1a4be35740 1 -- start start 2024-04-21T08:51:26.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.884+0000 7f1a4be35740 1 Processor -- start 2024-04-21T08:51:26.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.885+0000 7f1a4be35740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68b8093e0 0x55d68b820570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:26.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.885+0000 7f1a4be35740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b80ed10 0x55d68b820ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:26.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.885+0000 7f1a4be35740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d68b80f6c0 0x55d68b81cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:26.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.885+0000 7f1a4be35740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a441b0b30 con 0x55d68b80ed10 2024-04-21T08:51:26.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.885+0000 7f1a4be35740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a441b0460 con 0x55d68b8093e0 2024-04-21T08:51:26.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.885+0000 7f1a4be35740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1a441ad6c0 con 0x55d68b80f6c0 2024-04-21T08:51:26.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.885+0000 7f1a4be35740 10 client.0 _ll_register_callbacks cb 0x7f1a340044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:26.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.885+0000 7f1a4259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b80ed10 0x55d68b820ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:26.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.885+0000 7f1a4259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b80ed10 0x55d68b820ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42890/0 (socket says 172.21.15.43:42890) 2024-04-21T08:51:26.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.885+0000 7f1a4259c640 1 -- 172.21.15.43:0/4062709479 learned_addr learned my addr 172.21.15.43:0/4062709479 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:26.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.885+0000 7f1a42d9d640 1 --2- 172.21.15.43:0/4062709479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68b8093e0 0x55d68b820570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:26.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.885+0000 7f1a4359e640 1 --2- 172.21.15.43:0/4062709479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d68b80f6c0 0x55d68b81cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:26.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.885+0000 7f1a4259c640 1 -- 172.21.15.43:0/4062709479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d68b80f6c0 msgr2=0x55d68b81cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:26.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.885+0000 7f1a4259c640 1 --2- 172.21.15.43:0/4062709479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d68b80f6c0 0x55d68b81cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.885+0000 7f1a4259c640 1 -- 172.21.15.43:0/4062709479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68b8093e0 msgr2=0x55d68b820570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:26.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.885+0000 7f1a4259c640 1 --2- 172.21.15.43:0/4062709479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68b8093e0 0x55d68b820570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.885+0000 7f1a4259c640 1 -- 172.21.15.43:0/4062709479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d68b446310 con 0x55d68b80ed10 2024-04-21T08:51:26.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.885+0000 7f1a4359e640 1 --2- 172.21.15.43:0/4062709479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d68b80f6c0 0x55d68b81cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:26.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.885+0000 7f1a4259c640 1 --2- 172.21.15.43:0/4062709479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b80ed10 0x55d68b820ab0 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f1a3809c0a0 tx=0x7f1a3800df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168b77d3e2a2780f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:26.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.885+0000 7f1a26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:26.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.886+0000 7f1a26ff5640 1 -- 172.21.15.43:0/4062709479 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38021040 con 0x55d68b80ed10 2024-04-21T08:51:26.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.886+0000 7f1a26ff5640 1 -- 172.21.15.43:0/4062709479 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f1a38098a10 con 0x55d68b80ed10 2024-04-21T08:51:26.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.886+0000 7f1a4be35740 1 -- 172.21.15.43:0/4062709479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a340054d0 con 0x55d68b80ed10 2024-04-21T08:51:26.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.886+0000 7f1a26ff5640 1 -- 172.21.15.43:0/4062709479 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1a380a0750 con 0x55d68b80ed10 2024-04-21T08:51:26.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.886+0000 7f1a4be35740 10 client.21426 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:26.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.886+0000 7f1a4be35740 1 -- 172.21.15.43:0/4062709479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d68b88f620 con 0x55d68b80ed10 2024-04-21T08:51:26.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.886+0000 7f1a26ff5640 1 -- 172.21.15.43:0/4062709479 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a340054d0 con 0x55d68b80ed10 2024-04-21T08:51:26.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.886+0000 7f1a26ff5640 1 -- 172.21.15.43:0/4062709479 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1a380b5b20 con 0x55d68b80ed10 2024-04-21T08:51:26.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.886+0000 7f1a4be35740 10 client.21426 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:26.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.886+0000 7f1a4be35740 10 client.21426 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-21T08:51:26.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.886+0000 7f1a4be35740 20 client.21426 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:26.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.886+0000 7f1a4be35740 1 --2- 172.21.15.43:0/4062709479 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d68b818240 0x55d68b891de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:26.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.886+0000 7f1a4be35740 4 client.21426 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-04-21T08:51:26.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.886+0000 7f1a4be35740 1 -- 172.21.15.43:0/4062709479 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55d68b77b890 con 0x55d68b818240 2024-04-21T08:51:26.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.886+0000 7f1a42d9d640 1 --2- 172.21.15.43:0/4062709479 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d68b818240 0x55d68b891de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:26.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.887+0000 7f1a42d9d640 1 --2- 172.21.15.43:0/4062709479 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d68b818240 0x55d68b891de0 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cb93a8b3681c4f2 server_cookie=e86ec79260296867 in_seq=0 out_seq=0 2024-04-21T08:51:26.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.887+0000 7f1a26ff5640 10 client.21426 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:26.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.887+0000 7f1a26ff5640 1 -- 172.21.15.43:0/4062709479 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55d68b77b890 con 0x55d68b818240 2024-04-21T08:51:26.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.887+0000 7f1a26ff5640 10 client.21426 handle_command_reply: tid=0 2024-04-21T08:51:26.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.887+0000 7f1a4be35740 1 client.21426 shutdown 2024-04-21T08:51:26.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.887+0000 7f1a4be35740 1 -- 172.21.15.43:0/4062709479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b80ed10 msgr2=0x55d68b820ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:26.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.887+0000 7f1a4be35740 1 --2- 172.21.15.43:0/4062709479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b80ed10 0x55d68b820ab0 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f1a3809c0a0 tx=0x7f1a3800df10 comp rx=0 tx=0).stop 2024-04-21T08:51:26.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.887+0000 7f1a4be35740 1 -- 172.21.15.43:0/4062709479 shutdown_connections 2024-04-21T08:51:26.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.887+0000 7f1a4be35740 1 --2- 172.21.15.43:0/4062709479 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x55d68b818240 0x55d68b891de0 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.887+0000 7f1a4be35740 1 --2- 172.21.15.43:0/4062709479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d68b80f6c0 0x55d68b81cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.887+0000 7f1a26ff5640 0 client.21426 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:26.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.887+0000 7f1a4be35740 1 --2- 172.21.15.43:0/4062709479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d68b80ed10 0x55d68b820ab0 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.887+0000 7f1a4be35740 1 --2- 172.21.15.43:0/4062709479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68b8093e0 0x55d68b820570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.887+0000 7f1a4be35740 1 -- 172.21.15.43:0/4062709479 >> 172.21.15.43:0/4062709479 conn(0x55d68b810b10 msgr2=0x55d68b80aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:26.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.887+0000 7f1a4be35740 1 -- 172.21.15.43:0/4062709479 shutdown_connections 2024-04-21T08:51:26.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.887+0000 7f1a4be35740 1 -- 172.21.15.43:0/4062709479 wait complete. 2024-04-21T08:51:26.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.887+0000 7f1a4be35740 20 client.21426 trim_cache size 0 max 16384 2024-04-21T08:51:26.899 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:51:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 30, 2024-04-21T08:51:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "leader": 24527, 2024-04-21T08:51:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: "set_version": 12, 2024-04-21T08:51:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: "sets": {} 2024-04-21T08:51:26.900 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:51:26.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.888+0000 7f1a4916d640 1 -- 172.21.15.43:0/307767252 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1a2c0a3d20 msgr2=0x7f1a2c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:26.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.888+0000 7f1a4916d640 1 --2- 172.21.15.43:0/307767252 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1a2c0a3d20 0x7f1a2c0a61e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c01f000 tx=0x7f1a3c019000 comp rx=0 tx=0).stop 2024-04-21T08:51:26.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.888+0000 7f1a4916d640 1 -- 172.21.15.43:0/307767252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a44107260 msgr2=0x7f1a4419e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:26.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.888+0000 7f1a4916d640 1 --2- 172.21.15.43:0/307767252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a44107260 0x7f1a4419e940 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f1a3800d770 tx=0x7f1a3800dc30 comp rx=0 tx=0).stop 2024-04-21T08:51:26.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.888+0000 7f1a4916d640 1 -- 172.21.15.43:0/307767252 shutdown_connections 2024-04-21T08:51:26.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.888+0000 7f1a4916d640 1 --2- 172.21.15.43:0/307767252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a44198600 0x7f1a44198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.888+0000 7f1a4916d640 1 --2- 172.21.15.43:0/307767252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a44107260 0x7f1a4419e940 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.888+0000 7f1a4916d640 1 --2- 172.21.15.43:0/307767252 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f1a2c0a3d20 0x7f1a2c0a61e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.888+0000 7f1a4916d640 1 --2- 172.21.15.43:0/307767252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a44102190 0x7f1a4419e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:26.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.888+0000 7f1a4916d640 1 -- 172.21.15.43:0/307767252 >> 172.21.15.43:0/307767252 conn(0x7f1a440fd990 msgr2=0x7f1a440fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:26.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.888+0000 7f1a4916d640 1 -- 172.21.15.43:0/307767252 shutdown_connections 2024-04-21T08:51:26.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:26.888+0000 7f1a4916d640 1 -- 172.21.15.43:0/307767252 wait complete. 2024-04-21T08:51:26.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:26 smithi043 ceph-mon[27978]: pgmap v391: 129 pgs: 129 active+clean; 2.8 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:51:26.906 INFO:tasks.quiescer:done joining Quiescers 2024-04-21T08:51:26.907 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-21T08:51:26.990 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi043.front.sepia.ceph.com 2024-04-21T08:51:26.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:26 smithi175 ceph-mon[31920]: pgmap v391: 129 pgs: 129 active+clean; 2.8 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:51:26.999 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-21T08:51:27.049 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-21T08:51:27.049 INFO:teuthology.task.exec:Executing custom commands... 2024-04-21T08:51:27.049 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-04-21T08:51:27.050 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-21T08:51:27.077 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-21T08:51:27.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-21T08:51:27.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.288+0000 7fe04ed40640 1 -- 172.21.15.43:0/148549972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe048101a90 msgr2=0x7fe048105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:27.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.288+0000 7fe04ed40640 1 --2- 172.21.15.43:0/148549972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe048101a90 0x7fe048105ad0 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7fe034009900 tx=0x7fe03402bc50 comp rx=0 tx=0).stop 2024-04-21T08:51:27.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.288+0000 7fe04ed40640 1 -- 172.21.15.43:0/148549972 shutdown_connections 2024-04-21T08:51:27.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.288+0000 7fe04ed40640 1 --2- 172.21.15.43:0/148549972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0481061b0 0x7fe04810dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:27.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.288+0000 7fe04ed40640 1 --2- 172.21.15.43:0/148549972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe048101a90 0x7fe048105ad0 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:27.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.288+0000 7fe04ed40640 1 --2- 172.21.15.43:0/148549972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0481010e0 0x7fe0481014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:27.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.288+0000 7fe04ed40640 1 -- 172.21.15.43:0/148549972 >> 172.21.15.43:0/148549972 conn(0x7fe0480771d0 msgr2=0x7fe0480775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:27.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.288+0000 7fe04ed40640 1 -- 172.21.15.43:0/148549972 shutdown_connections 2024-04-21T08:51:27.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.288+0000 7fe04ed40640 1 -- 172.21.15.43:0/148549972 wait complete. 2024-04-21T08:51:27.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.289+0000 7fe04ed40640 1 Processor -- start 2024-04-21T08:51:27.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.289+0000 7fe04ed40640 1 -- start start 2024-04-21T08:51:27.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.289+0000 7fe04ed40640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0481010e0 0x7fe048198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:27.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.289+0000 7fe04ed40640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0481061b0 0x7fe048198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:27.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.289+0000 7fe04ed40640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe04819cff0 0x7fe04819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:27.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.289+0000 7fe04ed40640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe048112270 con 0x7fe0481061b0 2024-04-21T08:51:27.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.289+0000 7fe04ed40640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe0481120f0 con 0x7fe0481010e0 2024-04-21T08:51:27.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.289+0000 7fe04ed40640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe0481123f0 con 0x7fe04819cff0 2024-04-21T08:51:27.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.289+0000 7fe047fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0481061b0 0x7fe048198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:27.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.289+0000 7fe047fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0481061b0 0x7fe048198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42908/0 (socket says 172.21.15.43:42908) 2024-04-21T08:51:27.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.289+0000 7fe047fff640 1 -- 172.21.15.43:0/4126371008 learned_addr learned my addr 172.21.15.43:0/4126371008 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:27.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.289+0000 7fe04cadc640 1 --2- 172.21.15.43:0/4126371008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0481010e0 0x7fe048198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:27.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.289+0000 7fe04d2dd640 1 --2- 172.21.15.43:0/4126371008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe04819cff0 0x7fe04819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:27.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.289+0000 7fe047fff640 1 -- 172.21.15.43:0/4126371008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe04819cff0 msgr2=0x7fe04819d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:27.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.289+0000 7fe047fff640 1 --2- 172.21.15.43:0/4126371008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe04819cff0 0x7fe04819d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:27.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.289+0000 7fe047fff640 1 -- 172.21.15.43:0/4126371008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0481010e0 msgr2=0x7fe048198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:27.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.289+0000 7fe047fff640 1 --2- 172.21.15.43:0/4126371008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0481010e0 0x7fe048198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:27.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.289+0000 7fe047fff640 1 -- 172.21.15.43:0/4126371008 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04819dae0 con 0x7fe0481061b0 2024-04-21T08:51:27.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.289+0000 7fe04cadc640 1 --2- 172.21.15.43:0/4126371008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0481010e0 0x7fe048198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:27.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.290+0000 7fe047fff640 1 --2- 172.21.15.43:0/4126371008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0481061b0 0x7fe048198c70 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7fe034005ab0 tx=0x7fe0340045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:27.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.290+0000 7fe045ffb640 1 -- 172.21.15.43:0/4126371008 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe034043020 con 0x7fe0481061b0 2024-04-21T08:51:27.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.290+0000 7fe045ffb640 1 -- 172.21.15.43:0/4126371008 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe03402cb90 con 0x7fe0481061b0 2024-04-21T08:51:27.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.290+0000 7fe045ffb640 1 -- 172.21.15.43:0/4126371008 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe034003bc0 con 0x7fe0481061b0 2024-04-21T08:51:27.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.290+0000 7fe04ed40640 1 -- 172.21.15.43:0/4126371008 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe04819dd70 con 0x7fe0481061b0 2024-04-21T08:51:27.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.290+0000 7fe04ed40640 1 -- 172.21.15.43:0/4126371008 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0481a5540 con 0x7fe0481061b0 2024-04-21T08:51:27.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.291+0000 7fe04ed40640 1 -- 172.21.15.43:0/4126371008 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe04804fce0 con 0x7fe0481061b0 2024-04-21T08:51:27.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.292+0000 7fe045ffb640 1 -- 172.21.15.43:0/4126371008 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe034046020 con 0x7fe0481061b0 2024-04-21T08:51:27.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.293+0000 7fe045ffb640 1 --2- 172.21.15.43:0/4126371008 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe0380a3dc0 0x7fe0380a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:27.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.293+0000 7fe045ffb640 1 -- 172.21.15.43:0/4126371008 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fe0340f33d0 con 0x7fe0481061b0 2024-04-21T08:51:27.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.293+0000 7fe04cadc640 1 --2- 172.21.15.43:0/4126371008 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe0380a3dc0 0x7fe0380a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:27.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.293+0000 7fe04cadc640 1 --2- 172.21.15.43:0/4126371008 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe0380a3dc0 0x7fe0380a6280 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe03c0058c0 tx=0x7fe03c005850 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:27.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.297+0000 7fe045ffb640 1 -- 172.21.15.43:0/4126371008 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe03404d020 con 0x7fe0481061b0 2024-04-21T08:51:27.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.510+0000 7fe04ed40640 1 -- 172.21.15.43:0/4126371008 --> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fe048101a90 con 0x7fe0380a3dc0 2024-04-21T08:51:27.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.517+0000 7fe045ffb640 1 -- 172.21.15.43:0/4126371008 <== mgr.15546 v2:172.21.15.43:6800/1482273906 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fe048101a90 con 0x7fe0380a3dc0 2024-04-21T08:51:27.517 INFO:teuthology.orchestra.run.smithi043.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-21T00:00:00", "created": "2024-04-21T08:39:00", "first": "2024-04-21T08:40:00", "last": "2024-04-21T08:51:00", "last_pruned": "2024-04-21T08:51:00", "created_count": 12, "pruned_count": 6, "active": true} 2024-04-21T08:51:27.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.519+0000 7fe04ed40640 1 -- 172.21.15.43:0/4126371008 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe0380a3dc0 msgr2=0x7fe0380a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:27.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.519+0000 7fe04ed40640 1 --2- 172.21.15.43:0/4126371008 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe0380a3dc0 0x7fe0380a6280 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe03c0058c0 tx=0x7fe03c005850 comp rx=0 tx=0).stop 2024-04-21T08:51:27.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.519+0000 7fe04ed40640 1 -- 172.21.15.43:0/4126371008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0481061b0 msgr2=0x7fe048198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.519+0000 7fe04ed40640 1 --2- 172.21.15.43:0/4126371008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0481061b0 0x7fe048198c70 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7fe034005ab0 tx=0x7fe0340045e0 comp rx=0 tx=0).stop 2024-04-21T08:51:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.520+0000 7fe04ed40640 1 -- 172.21.15.43:0/4126371008 shutdown_connections 2024-04-21T08:51:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.520+0000 7fe04ed40640 1 --2- 172.21.15.43:0/4126371008 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fe0380a3dc0 0x7fe0380a6280 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.520+0000 7fe04ed40640 1 --2- 172.21.15.43:0/4126371008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe04819cff0 0x7fe04819d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.520+0000 7fe04ed40640 1 --2- 172.21.15.43:0/4126371008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0481061b0 0x7fe048198c70 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.520+0000 7fe04ed40640 1 --2- 172.21.15.43:0/4126371008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0481010e0 0x7fe048198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.520+0000 7fe04ed40640 1 -- 172.21.15.43:0/4126371008 >> 172.21.15.43:0/4126371008 conn(0x7fe0480771d0 msgr2=0x7fe0480fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.520+0000 7fe04ed40640 1 -- 172.21.15.43:0/4126371008 shutdown_connections 2024-04-21T08:51:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.520+0000 7fe04ed40640 1 -- 172.21.15.43:0/4126371008 wait complete. 2024-04-21T08:51:27.530 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-21T08:51:27.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.636+0000 7f0270950640 1 -- 172.21.15.43:0/2091402518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f026c101aa0 msgr2=0x7f026c105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:27.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.636+0000 7f0270950640 1 --2- 172.21.15.43:0/2091402518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f026c101aa0 0x7f026c105a90 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f0254009900 tx=0x7f025402bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:27.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.636+0000 7f0270950640 1 -- 172.21.15.43:0/2091402518 shutdown_connections 2024-04-21T08:51:27.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.636+0000 7f0270950640 1 --2- 172.21.15.43:0/2091402518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f026c106170 0x7f026c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:27.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.636+0000 7f0270950640 1 --2- 172.21.15.43:0/2091402518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f026c101aa0 0x7f026c105a90 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:27.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.636+0000 7f0270950640 1 --2- 172.21.15.43:0/2091402518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f026c1010f0 0x7f026c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:27.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.636+0000 7f0270950640 1 -- 172.21.15.43:0/2091402518 >> 172.21.15.43:0/2091402518 conn(0x7f026c077fd0 msgr2=0x7f026c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:27.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.636+0000 7f0270950640 1 -- 172.21.15.43:0/2091402518 shutdown_connections 2024-04-21T08:51:27.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.636+0000 7f0270950640 1 -- 172.21.15.43:0/2091402518 wait complete. 2024-04-21T08:51:27.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.637+0000 7f0270950640 1 Processor -- start 2024-04-21T08:51:27.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.637+0000 7f0270950640 1 -- start start 2024-04-21T08:51:27.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.637+0000 7f0270950640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f026c1010f0 0x7f026c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:27.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.637+0000 7f0270950640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f026c106170 0x7f026c198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:27.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.637+0000 7f0270950640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f026c19cf80 0x7f026c19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:27.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.637+0000 7f0270950640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f026c111090 con 0x7f026c19cf80 2024-04-21T08:51:27.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.637+0000 7f0270950640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f026c110f10 con 0x7f026c1010f0 2024-04-21T08:51:27.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.637+0000 7f0270950640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f026c111210 con 0x7f026c106170 2024-04-21T08:51:27.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.637+0000 7f026ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f026c19cf80 0x7f026c19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:27.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.637+0000 7f026ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f026c19cf80 0x7f026c19d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42926/0 (socket says 172.21.15.43:42926) 2024-04-21T08:51:27.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.637+0000 7f026ad9d640 1 -- 172.21.15.43:0/219781751 learned_addr learned my addr 172.21.15.43:0/219781751 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:27.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.637+0000 7f026a59c640 1 --2- 172.21.15.43:0/219781751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f026c1010f0 0x7f026c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:27.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.637+0000 7f0263fff640 1 --2- 172.21.15.43:0/219781751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f026c106170 0x7f026c198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:27.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.637+0000 7f026ad9d640 1 -- 172.21.15.43:0/219781751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f026c106170 msgr2=0x7f026c198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:27.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.637+0000 7f026ad9d640 1 --2- 172.21.15.43:0/219781751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f026c106170 0x7f026c198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:27.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.637+0000 7f026ad9d640 1 -- 172.21.15.43:0/219781751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f026c1010f0 msgr2=0x7f026c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:27.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.637+0000 7f026ad9d640 1 --2- 172.21.15.43:0/219781751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f026c1010f0 0x7f026c1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:27.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.637+0000 7f026ad9d640 1 -- 172.21.15.43:0/219781751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f026c19da70 con 0x7f026c19cf80 2024-04-21T08:51:27.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.637+0000 7f026ad9d640 1 --2- 172.21.15.43:0/219781751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f026c19cf80 0x7f026c19d430 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f026400eaa0 tx=0x7f026400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:27.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.638+0000 7f02637fe640 1 -- 172.21.15.43:0/219781751 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0264004d20 con 0x7f026c19cf80 2024-04-21T08:51:27.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.638+0000 7f02637fe640 1 -- 172.21.15.43:0/219781751 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0264005020 con 0x7f026c19cf80 2024-04-21T08:51:27.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.638+0000 7f0270950640 1 -- 172.21.15.43:0/219781751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f026c19dd60 con 0x7f026c19cf80 2024-04-21T08:51:27.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.638+0000 7f02637fe640 1 -- 172.21.15.43:0/219781751 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02640116b0 con 0x7f026c19cf80 2024-04-21T08:51:27.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.638+0000 7f0270950640 1 -- 172.21.15.43:0/219781751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f026c1a5520 con 0x7f026c19cf80 2024-04-21T08:51:27.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.639+0000 7f0270950640 1 -- 172.21.15.43:0/219781751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f026c04fce0 con 0x7f026c19cf80 2024-04-21T08:51:27.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.640+0000 7f02637fe640 1 -- 172.21.15.43:0/219781751 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f02640051d0 con 0x7f026c19cf80 2024-04-21T08:51:27.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.640+0000 7f02637fe640 1 --2- 172.21.15.43:0/219781751 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f02580a3e40 0x7f02580a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:27.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.641+0000 7f026a59c640 1 --2- 172.21.15.43:0/219781751 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f02580a3e40 0x7f02580a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:27.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.641+0000 7f02637fe640 1 -- 172.21.15.43:0/219781751 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f02640d10c0 con 0x7f026c19cf80 2024-04-21T08:51:27.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.641+0000 7f026a59c640 1 --2- 172.21.15.43:0/219781751 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f02580a3e40 0x7f02580a6300 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f025c00abe0 tx=0x7f025c00a2d0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:27.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.644+0000 7f02637fe640 1 -- 172.21.15.43:0/219781751 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0264016030 con 0x7f026c19cf80 2024-04-21T08:51:27.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:27 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2662286593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:51:27.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.857+0000 7f0270950640 1 -- 172.21.15.43:0/219781751 --> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f026c101aa0 con 0x7f02580a3e40 2024-04-21T08:51:27.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.864+0000 7f02637fe640 1 -- 172.21.15.43:0/219781751 <== mgr.15546 v2:172.21.15.43:6800/1482273906 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f026c101aa0 con 0x7f02580a3e40 2024-04-21T08:51:27.864 INFO:teuthology.orchestra.run.smithi043.stdout:/ 1m 6m3h 2024-04-21T08:51:27.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.867+0000 7f0270950640 1 -- 172.21.15.43:0/219781751 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f02580a3e40 msgr2=0x7f02580a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:27.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.867+0000 7f0270950640 1 --2- 172.21.15.43:0/219781751 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f02580a3e40 0x7f02580a6300 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f025c00abe0 tx=0x7f025c00a2d0 comp rx=0 tx=0).stop 2024-04-21T08:51:27.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.867+0000 7f0270950640 1 -- 172.21.15.43:0/219781751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f026c19cf80 msgr2=0x7f026c19d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:27.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.867+0000 7f0270950640 1 --2- 172.21.15.43:0/219781751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f026c19cf80 0x7f026c19d430 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f026400eaa0 tx=0x7f026400eea0 comp rx=0 tx=0).stop 2024-04-21T08:51:27.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.867+0000 7f0270950640 1 -- 172.21.15.43:0/219781751 shutdown_connections 2024-04-21T08:51:27.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.867+0000 7f0270950640 1 --2- 172.21.15.43:0/219781751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f026c19cf80 0x7f026c19d430 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:27.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.867+0000 7f0270950640 1 --2- 172.21.15.43:0/219781751 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f02580a3e40 0x7f02580a6300 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:27.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.867+0000 7f0270950640 1 --2- 172.21.15.43:0/219781751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f026c106170 0x7f026c198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:27.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.867+0000 7f0270950640 1 --2- 172.21.15.43:0/219781751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f026c1010f0 0x7f026c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:27.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.867+0000 7f0270950640 1 -- 172.21.15.43:0/219781751 >> 172.21.15.43:0/219781751 conn(0x7f026c077fd0 msgr2=0x7f026c069910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:27.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.867+0000 7f0270950640 1 -- 172.21.15.43:0/219781751 shutdown_connections 2024-04-21T08:51:27.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:27.867+0000 7f0270950640 1 -- 172.21.15.43:0/219781751 wait complete. 2024-04-21T08:51:27.878 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-21T08:51:27.888 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-21T08:51:27.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:27 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2662286593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:51:28.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:27 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2662286593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:51:28.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:28 smithi175 ceph-mon[31920]: pgmap v392: 129 pgs: 129 active+clean; 3.0 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-21T08:51:28.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:28 smithi175 ceph-mon[31920]: from='client.21432 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:51:29.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:28 smithi112 ceph-mon[31085]: pgmap v392: 129 pgs: 129 active+clean; 3.0 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-21T08:51:29.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:28 smithi112 ceph-mon[31085]: from='client.21432 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:51:29.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:28 smithi043 ceph-mon[27978]: pgmap v392: 129 pgs: 129 active+clean; 3.0 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-21T08:51:29.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:28 smithi043 ceph-mon[27978]: from='client.21432 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:51:29.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:29 smithi175 ceph-mon[31920]: from='client.21438 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:51:30.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:29 smithi112 ceph-mon[31085]: from='client.21438 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:51:30.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:29 smithi043 ceph-mon[27978]: from='client.21438 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:51:30.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:30 smithi175 ceph-mon[31920]: pgmap v393: 129 pgs: 129 active+clean; 3.0 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 14 op/s 2024-04-21T08:51:31.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:30 smithi112 ceph-mon[31085]: pgmap v393: 129 pgs: 129 active+clean; 3.0 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 14 op/s 2024-04-21T08:51:31.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:30 smithi043 ceph-mon[27978]: pgmap v393: 129 pgs: 129 active+clean; 3.0 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 14 op/s 2024-04-21T08:51:32.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:32 smithi175 ceph-mon[31920]: pgmap v394: 129 pgs: 129 active+clean; 3.0 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-21T08:51:33.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:32 smithi112 ceph-mon[31085]: pgmap v394: 129 pgs: 129 active+clean; 3.0 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-21T08:51:33.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:32 smithi043 ceph-mon[27978]: pgmap v394: 129 pgs: 129 active+clean; 3.0 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-21T08:51:34.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:34 smithi175 ceph-mon[31920]: pgmap v395: 129 pgs: 129 active+clean; 3.0 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-21T08:51:35.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:34 smithi112 ceph-mon[31085]: pgmap v395: 129 pgs: 129 active+clean; 3.0 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-21T08:51:35.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:34 smithi043 ceph-mon[27978]: pgmap v395: 129 pgs: 129 active+clean; 3.0 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-21T08:51:36.864 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:36 smithi043 ceph-mon[27978]: pgmap v396: 129 pgs: 129 active+clean; 3.0 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-21T08:51:36.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:36 smithi175 ceph-mon[31920]: pgmap v396: 129 pgs: 129 active+clean; 3.0 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-21T08:51:37.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:36 smithi112 ceph-mon[31085]: pgmap v396: 129 pgs: 129 active+clean; 3.0 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-21T08:51:38.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:38 smithi175 ceph-mon[31920]: pgmap v397: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 KiB/s wr, 18 op/s 2024-04-21T08:51:39.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:38 smithi112 ceph-mon[31085]: pgmap v397: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 KiB/s wr, 18 op/s 2024-04-21T08:51:39.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:38 smithi043 ceph-mon[27978]: pgmap v397: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 KiB/s wr, 18 op/s 2024-04-21T08:51:40.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:40 smithi175 ceph-mon[31920]: pgmap v398: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.1 KiB/s wr, 12 op/s 2024-04-21T08:51:41.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:40 smithi112 ceph-mon[31085]: pgmap v398: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.1 KiB/s wr, 12 op/s 2024-04-21T08:51:41.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:40 smithi043 ceph-mon[27978]: pgmap v398: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.1 KiB/s wr, 12 op/s 2024-04-21T08:51:42.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:42 smithi175 ceph-mon[31920]: pgmap v399: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 KiB/s wr, 17 op/s 2024-04-21T08:51:43.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:42 smithi112 ceph-mon[31085]: pgmap v399: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 KiB/s wr, 17 op/s 2024-04-21T08:51:43.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:42 smithi043 ceph-mon[27978]: pgmap v399: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 KiB/s wr, 17 op/s 2024-04-21T08:51:44.759 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:51:44.759 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:51:44.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.878+0000 7f443d54b640 1 -- 172.21.15.43:0/2799949446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44381031c0 msgr2=0x7f44381035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:44.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.878+0000 7f443d54b640 1 --2- 172.21.15.43:0/2799949446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44381031c0 0x7f44381035a0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f4428009ca0 tx=0x7f442802c230 comp rx=0 tx=0).stop 2024-04-21T08:51:44.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.878+0000 7f443d54b640 1 -- 172.21.15.43:0/2799949446 shutdown_connections 2024-04-21T08:51:44.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.878+0000 7f443d54b640 1 --2- 172.21.15.43:0/2799949446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f443810c500 0x7f443810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.878+0000 7f443d54b640 1 --2- 172.21.15.43:0/2799949446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4438103ae0 0x7f443810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.878+0000 7f443d54b640 1 --2- 172.21.15.43:0/2799949446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44381031c0 0x7f44381035a0 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.878+0000 7f443d54b640 1 -- 172.21.15.43:0/2799949446 >> 172.21.15.43:0/2799949446 conn(0x7f4438078000 msgr2=0x7f4438078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:44.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.878+0000 7f443d54b640 1 -- 172.21.15.43:0/2799949446 shutdown_connections 2024-04-21T08:51:44.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.878+0000 7f443d54b640 1 -- 172.21.15.43:0/2799949446 wait complete. 2024-04-21T08:51:44.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.878+0000 7f443d54b640 1 Processor -- start 2024-04-21T08:51:44.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.879+0000 7f443d54b640 1 -- start start 2024-04-21T08:51:44.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.879+0000 7f443d54b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44381031c0 0x7f44381986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:44.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.879+0000 7f443d54b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4438103ae0 0x7f4438198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:44.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.879+0000 7f443d54b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f443810c500 0x7f443819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:44.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.879+0000 7f443d54b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4438112170 con 0x7f44381031c0 2024-04-21T08:51:44.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.879+0000 7f443d54b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4438111ff0 con 0x7f4438103ae0 2024-04-21T08:51:44.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.879+0000 7f443d54b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f44381122f0 con 0x7f443810c500 2024-04-21T08:51:44.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.879+0000 7f4436ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44381031c0 0x7f44381986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:44.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.879+0000 7f4436ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44381031c0 0x7f44381986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51196/0 (socket says 172.21.15.43:51196) 2024-04-21T08:51:44.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.879+0000 7f4436ffd640 1 -- 172.21.15.43:0/1543142169 learned_addr learned my addr 172.21.15.43:0/1543142169 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:44.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.879+0000 7f4436ffd640 1 -- 172.21.15.43:0/1543142169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f443810c500 msgr2=0x7f443819cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:44.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.879+0000 7f4436ffd640 1 --2- 172.21.15.43:0/1543142169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f443810c500 0x7f443819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.879+0000 7f4436ffd640 1 -- 172.21.15.43:0/1543142169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4438103ae0 msgr2=0x7f4438198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:44.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.879+0000 7f44367fc640 1 --2- 172.21.15.43:0/1543142169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4438103ae0 0x7f4438198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:44.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.879+0000 7f4436ffd640 1 --2- 172.21.15.43:0/1543142169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4438103ae0 0x7f4438198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.879+0000 7f4436ffd640 1 -- 172.21.15.43:0/1543142169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443819d6a0 con 0x7f44381031c0 2024-04-21T08:51:44.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.880+0000 7f4436ffd640 1 --2- 172.21.15.43:0/1543142169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44381031c0 0x7f44381986b0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f4428005e30 tx=0x7f4428003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:44.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.880+0000 7f441ffff640 1 -- 172.21.15.43:0/1543142169 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4428039040 con 0x7f44381031c0 2024-04-21T08:51:44.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.880+0000 7f443d54b640 1 -- 172.21.15.43:0/1543142169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443819d930 con 0x7f44381031c0 2024-04-21T08:51:44.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.880+0000 7f441ffff640 1 -- 172.21.15.43:0/1543142169 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4428044070 con 0x7f44381031c0 2024-04-21T08:51:44.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.880+0000 7f441ffff640 1 -- 172.21.15.43:0/1543142169 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44280329a0 con 0x7f44381031c0 2024-04-21T08:51:44.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.880+0000 7f443d54b640 1 -- 172.21.15.43:0/1543142169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44381a5220 con 0x7f44381031c0 2024-04-21T08:51:44.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.882+0000 7f441ffff640 1 -- 172.21.15.43:0/1543142169 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4428032b40 con 0x7f44381031c0 2024-04-21T08:51:44.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.882+0000 7f4440213740 1 Processor -- start 2024-04-21T08:51:44.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.882+0000 7f4440213740 1 -- start start 2024-04-21T08:51:44.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.883+0000 7f4440213740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b31499580 0x557b31499960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.883+0000 7f4440213740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b31499ea0 0x557b314a2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.883+0000 7f4440213740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b314a2a80 0x557b314a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.883+0000 7f4440213740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557b313fe1e0 con 0x557b314a2a80 2024-04-21T08:51:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.883+0000 7f4440213740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557b3139b390 con 0x557b31499ea0 2024-04-21T08:51:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.883+0000 7f4440213740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557b313fa1a0 con 0x557b31499580 2024-04-21T08:51:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.883+0000 7f441ffff640 1 --2- 172.21.15.43:0/1543142169 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f44200a3d20 0x7f44200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.883+0000 7f44367fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b314a2a80 0x557b314a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.883+0000 7f44367fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b314a2a80 0x557b314a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51206/0 (socket says 172.21.15.43:51206) 2024-04-21T08:51:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.883+0000 7f44367fc640 1 -- 172.21.15.43:0/671481062 learned_addr learned my addr 172.21.15.43:0/671481062 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.883+0000 7f4436ffd640 1 --2- 172.21.15.43:0/671481062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b31499ea0 0x557b314a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.883+0000 7f44377fe640 1 --2- 172.21.15.43:0/1543142169 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f44200a3d20 0x7f44200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.883+0000 7f44377fe640 1 --2- 172.21.15.43:0/671481062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b31499580 0x557b31499960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.883+0000 7f441ffff640 1 -- 172.21.15.43:0/1543142169 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f44280f3180 con 0x7f44381031c0 2024-04-21T08:51:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.883+0000 7f44367fc640 1 -- 172.21.15.43:0/671481062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b31499580 msgr2=0x557b31499960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.883+0000 7f44367fc640 1 --2- 172.21.15.43:0/671481062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b31499580 0x557b31499960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.883+0000 7f44367fc640 1 -- 172.21.15.43:0/671481062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b31499ea0 msgr2=0x557b314a2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.883+0000 7f44367fc640 1 --2- 172.21.15.43:0/671481062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b31499ea0 0x557b314a2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.883+0000 7f44367fc640 1 -- 172.21.15.43:0/671481062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b311da7d0 con 0x557b314a2a80 2024-04-21T08:51:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.883+0000 7f44377fe640 1 --2- 172.21.15.43:0/671481062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b31499580 0x557b31499960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.884+0000 7f44367fc640 1 --2- 172.21.15.43:0/671481062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b314a2a80 0x557b314a4e70 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f442c005bc0 tx=0x7f442c009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19ba79abdc62b4d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.884+0000 7f44377fe640 1 --2- 172.21.15.43:0/1543142169 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f44200a3d20 0x7f44200a61e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4430013400 tx=0x7f4430005d50 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.884+0000 7f441d7fa640 1 -- 172.21.15.43:0/671481062 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c016070 con 0x557b314a2a80 2024-04-21T08:51:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.884+0000 7f441d7fa640 1 -- 172.21.15.43:0/671481062 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f442c0054d0 con 0x557b314a2a80 2024-04-21T08:51:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.884+0000 7f441d7fa640 1 -- 172.21.15.43:0/671481062 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c004d70 con 0x557b314a2a80 2024-04-21T08:51:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.884+0000 7f4440213740 1 -- 172.21.15.43:0/671481062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b314a2a80 msgr2=0x557b314a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.884+0000 7f4440213740 1 --2- 172.21.15.43:0/671481062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b314a2a80 0x557b314a4e70 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f442c005bc0 tx=0x7f442c009f30 comp rx=0 tx=0).stop 2024-04-21T08:51:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.885+0000 7f4440213740 1 -- 172.21.15.43:0/671481062 shutdown_connections 2024-04-21T08:51:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.885+0000 7f4440213740 1 --2- 172.21.15.43:0/671481062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b314a2a80 0x557b314a4e70 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.885+0000 7f4440213740 1 --2- 172.21.15.43:0/671481062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b31499ea0 0x557b314a2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.885+0000 7f4440213740 1 --2- 172.21.15.43:0/671481062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b31499580 0x557b31499960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.885+0000 7f4440213740 1 -- 172.21.15.43:0/671481062 >> 172.21.15.43:0/671481062 conn(0x557b31493cb0 msgr2=0x557b31494090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:44.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.885+0000 7f4440213740 1 -- 172.21.15.43:0/671481062 shutdown_connections 2024-04-21T08:51:44.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.885+0000 7f4440213740 1 -- 172.21.15.43:0/671481062 wait complete. 2024-04-21T08:51:44.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.885+0000 7f4440213740 1 -- start start 2024-04-21T08:51:44.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.885+0000 7f4440213740 1 Processor -- start 2024-04-21T08:51:44.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.885+0000 7f4440213740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b31499580 0x557b3149faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:44.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.885+0000 7f4440213740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b31499ea0 0x557b3149ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:44.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.885+0000 7f4440213740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b314a2a80 0x557b314a0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:44.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.885+0000 7f4440213740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557b314a5870 con 0x557b314a2a80 2024-04-21T08:51:44.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.885+0000 7f4440213740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557b313b72f0 con 0x557b31499ea0 2024-04-21T08:51:44.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.885+0000 7f4440213740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557b314a59f0 con 0x557b31499580 2024-04-21T08:51:44.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.885+0000 7f4440213740 10 client.0 _ll_register_callbacks cb 0x557b313c4580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:44.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.885+0000 7f44377fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b314a2a80 0x557b314a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:44.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.885+0000 7f44377fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b314a2a80 0x557b314a0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51218/0 (socket says 172.21.15.43:51218) 2024-04-21T08:51:44.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.885+0000 7f44377fe640 1 -- 172.21.15.43:0/1670802009 learned_addr learned my addr 172.21.15.43:0/1670802009 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:44.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.886+0000 7f44367fc640 1 --2- 172.21.15.43:0/1670802009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b31499ea0 0x557b3149ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:44.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.886+0000 7f4436ffd640 1 --2- 172.21.15.43:0/1670802009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b31499580 0x557b3149faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:44.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.886+0000 7f44377fe640 1 -- 172.21.15.43:0/1670802009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b31499580 msgr2=0x557b3149faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:44.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.886+0000 7f44377fe640 1 --2- 172.21.15.43:0/1670802009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b31499580 0x557b3149faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.886+0000 7f44377fe640 1 -- 172.21.15.43:0/1670802009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b31499ea0 msgr2=0x557b3149ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:44.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.886+0000 7f44377fe640 1 --2- 172.21.15.43:0/1670802009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b31499ea0 0x557b3149ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.886+0000 7f44377fe640 1 -- 172.21.15.43:0/1670802009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b311103f0 con 0x557b314a2a80 2024-04-21T08:51:44.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.886+0000 7f44367fc640 1 --2- 172.21.15.43:0/1670802009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b31499ea0 0x557b3149ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:44.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.886+0000 7f4436ffd640 1 --2- 172.21.15.43:0/1670802009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b31499580 0x557b3149faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:44.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.886+0000 7f44377fe640 1 --2- 172.21.15.43:0/1670802009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b314a2a80 0x557b314a0520 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f4430010840 tx=0x7f44300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cff2b18b4d10218 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:44.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.886+0000 7f441aff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:44.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.886+0000 7f441aff5640 1 -- 172.21.15.43:0/1670802009 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430019050 con 0x557b314a2a80 2024-04-21T08:51:44.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.886+0000 7f441aff5640 1 -- 172.21.15.43:0/1670802009 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f443000fa60 con 0x557b314a2a80 2024-04-21T08:51:44.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.886+0000 7f4440213740 1 -- 172.21.15.43:0/1670802009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f442c0054d0 con 0x557b314a2a80 2024-04-21T08:51:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.886+0000 7f441aff5640 1 -- 172.21.15.43:0/1670802009 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f443002c050 con 0x557b314a2a80 2024-04-21T08:51:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.886+0000 7f4440213740 10 client.21450 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.886+0000 7f4440213740 1 -- 172.21.15.43:0/1670802009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557b310d00f0 con 0x557b314a2a80 2024-04-21T08:51:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.887+0000 7f441aff5640 1 -- 172.21.15.43:0/1670802009 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f442c0054d0 con 0x557b314a2a80 2024-04-21T08:51:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.887+0000 7f441aff5640 1 -- 172.21.15.43:0/1670802009 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f443000f040 con 0x557b314a2a80 2024-04-21T08:51:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.887+0000 7f4440213740 10 client.21450 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.887+0000 7f4440213740 10 client.21450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.887+0000 7f4440213740 20 client.21450 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.887+0000 7f4440213740 1 --2- 172.21.15.43:0/1670802009 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557b314b2a70 0x557b314b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.887+0000 7f4440213740 4 client.21450 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.887+0000 7f4440213740 1 -- 172.21.15.43:0/1670802009 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b3140bc00 con 0x557b314b2a70 2024-04-21T08:51:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.887+0000 7f4436ffd640 1 --2- 172.21.15.43:0/1670802009 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557b314b2a70 0x557b314b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.887+0000 7f441aff5640 10 client.21450 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.887+0000 7f4436ffd640 1 --2- 172.21.15.43:0/1670802009 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557b314b2a70 0x557b314b4ed0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e500e969a9d9886 server_cookie=9149edc6d40df119 in_seq=0 out_seq=0 2024-04-21T08:51:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.888+0000 7f441aff5640 1 -- 172.21.15.43:0/1670802009 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b3140bc00 con 0x557b314b2a70 2024-04-21T08:51:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.888+0000 7f441aff5640 10 client.21450 handle_command_reply: tid=0 2024-04-21T08:51:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.888+0000 7f4440213740 1 client.21450 shutdown 2024-04-21T08:51:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.888+0000 7f4440213740 1 -- 172.21.15.43:0/1670802009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b314a2a80 msgr2=0x557b314a0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.888+0000 7f4440213740 1 --2- 172.21.15.43:0/1670802009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b314a2a80 0x557b314a0520 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f4430010840 tx=0x7f44300046c0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.888+0000 7f4440213740 1 -- 172.21.15.43:0/1670802009 shutdown_connections 2024-04-21T08:51:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.888+0000 7f4440213740 1 --2- 172.21.15.43:0/1670802009 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557b314b2a70 0x557b314b4ed0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.888+0000 7f4440213740 1 --2- 172.21.15.43:0/1670802009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b314a2a80 0x557b314a0520 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.888+0000 7f4440213740 1 --2- 172.21.15.43:0/1670802009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b31499ea0 0x557b3149ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.888+0000 7f4440213740 1 --2- 172.21.15.43:0/1670802009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b31499580 0x557b3149faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.888+0000 7f441aff5640 0 client.21450 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.888+0000 7f4440213740 1 -- 172.21.15.43:0/1670802009 >> 172.21.15.43:0/1670802009 conn(0x557b31493cb0 msgr2=0x557b31495a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.888+0000 7f4440213740 1 -- 172.21.15.43:0/1670802009 shutdown_connections 2024-04-21T08:51:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.889+0000 7f4440213740 1 -- 172.21.15.43:0/1670802009 wait complete. 2024-04-21T08:51:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.889+0000 7f4440213740 20 client.21450 trim_cache size 0 max 16384 2024-04-21T08:51:44.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.906+0000 7f4440213740 1 Processor -- start 2024-04-21T08:51:44.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.907+0000 7f4440213740 1 -- start start 2024-04-21T08:51:44.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.907+0000 7f4440213740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b31499580 0x557b314ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:44.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.907+0000 7f4440213740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b31499ea0 0x557b314a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:44.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.907+0000 7f4440213740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b314a2a80 0x557b314ba7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:44.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.907+0000 7f4440213740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f44381af5a0 con 0x557b31499580 2024-04-21T08:51:44.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.907+0000 7f4440213740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f44381b10e0 con 0x557b314a2a80 2024-04-21T08:51:44.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.907+0000 7f4440213740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f44381afc70 con 0x557b31499ea0 2024-04-21T08:51:44.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.907+0000 7f4436ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b31499580 0x557b314ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:44.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.907+0000 7f4436ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b31499580 0x557b314ae730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51222/0 (socket says 172.21.15.43:51222) 2024-04-21T08:51:44.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.907+0000 7f4436ffd640 1 -- 172.21.15.43:0/3449124460 learned_addr learned my addr 172.21.15.43:0/3449124460 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:44.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.907+0000 7f44367fc640 1 --2- 172.21.15.43:0/3449124460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b31499ea0 0x557b314a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:44.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.907+0000 7f44377fe640 1 --2- 172.21.15.43:0/3449124460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b314a2a80 0x557b314ba7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:44.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.907+0000 7f4436ffd640 1 -- 172.21.15.43:0/3449124460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b31499ea0 msgr2=0x557b314a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:44.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.907+0000 7f4436ffd640 1 --2- 172.21.15.43:0/3449124460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b31499ea0 0x557b314a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.907+0000 7f4436ffd640 1 -- 172.21.15.43:0/3449124460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b314a2a80 msgr2=0x557b314ba7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:44.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.907+0000 7f4436ffd640 1 --2- 172.21.15.43:0/3449124460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b314a2a80 0x557b314ba7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.907+0000 7f4436ffd640 1 -- 172.21.15.43:0/3449124460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b314a4e70 con 0x557b31499580 2024-04-21T08:51:44.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.907+0000 7f44367fc640 1 --2- 172.21.15.43:0/3449124460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b31499ea0 0x557b314a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:44.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.907+0000 7f44377fe640 1 --2- 172.21.15.43:0/3449124460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b314a2a80 0x557b314ba7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:44.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.907+0000 7f4436ffd640 1 --2- 172.21.15.43:0/3449124460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b31499580 0x557b314ae730 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f44280bb7e0 tx=0x7f44280f5650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05056eb13b72991 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:44.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.907+0000 7f441cff9640 1 -- 172.21.15.43:0/3449124460 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44280bee10 con 0x557b31499580 2024-04-21T08:51:44.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.908+0000 7f441cff9640 1 -- 172.21.15.43:0/3449124460 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44280c31c0 con 0x557b31499580 2024-04-21T08:51:44.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.908+0000 7f441cff9640 1 -- 172.21.15.43:0/3449124460 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44280f0730 con 0x557b31499580 2024-04-21T08:51:44.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.908+0000 7f4440213740 1 -- 172.21.15.43:0/3449124460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b31499580 msgr2=0x557b314ae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:44.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.908+0000 7f4440213740 1 --2- 172.21.15.43:0/3449124460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b31499580 0x557b314ae730 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f44280bb7e0 tx=0x7f44280f5650 comp rx=0 tx=0).stop 2024-04-21T08:51:44.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.908+0000 7f4440213740 1 -- 172.21.15.43:0/3449124460 shutdown_connections 2024-04-21T08:51:44.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.908+0000 7f4440213740 1 --2- 172.21.15.43:0/3449124460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b314a2a80 0x557b314ba7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.908+0000 7f4440213740 1 --2- 172.21.15.43:0/3449124460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b31499ea0 0x557b314a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.908+0000 7f4440213740 1 --2- 172.21.15.43:0/3449124460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b31499580 0x557b314ae730 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.908+0000 7f4440213740 1 -- 172.21.15.43:0/3449124460 >> 172.21.15.43:0/3449124460 conn(0x557b31493cb0 msgr2=0x557b314a11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:44.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.908+0000 7f4440213740 1 -- 172.21.15.43:0/3449124460 shutdown_connections 2024-04-21T08:51:44.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.908+0000 7f4440213740 1 -- 172.21.15.43:0/3449124460 wait complete. 2024-04-21T08:51:44.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.908+0000 7f4440213740 1 -- start start 2024-04-21T08:51:44.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.909+0000 7f4440213740 1 Processor -- start 2024-04-21T08:51:44.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.909+0000 7f4440213740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b31499580 0x557b314a6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:44.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.909+0000 7f4440213740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b31499ea0 0x557b314a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:44.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.909+0000 7f4440213740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b314a2a80 0x557b314a70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:44.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.909+0000 7f4440213740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f44381b0a10 con 0x557b31499580 2024-04-21T08:51:44.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.909+0000 7f4440213740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f44381b0340 con 0x557b31499ea0 2024-04-21T08:51:44.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.909+0000 7f4440213740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f44381ad5a0 con 0x557b314a2a80 2024-04-21T08:51:44.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.909+0000 7f4440213740 10 client.0 _ll_register_callbacks cb 0x7f443000c600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:44.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.909+0000 7f4436ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b31499580 0x557b314a6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:44.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.909+0000 7f4436ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b31499580 0x557b314a6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51232/0 (socket says 172.21.15.43:51232) 2024-04-21T08:51:44.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.909+0000 7f4436ffd640 1 -- 172.21.15.43:0/3435614656 learned_addr learned my addr 172.21.15.43:0/3435614656 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:44.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.909+0000 7f44377fe640 1 --2- 172.21.15.43:0/3435614656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b314a2a80 0x557b314a70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:44.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.909+0000 7f44367fc640 1 --2- 172.21.15.43:0/3435614656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b31499ea0 0x557b314a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:44.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.909+0000 7f4436ffd640 1 -- 172.21.15.43:0/3435614656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b314a2a80 msgr2=0x557b314a70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:44.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.909+0000 7f4436ffd640 1 --2- 172.21.15.43:0/3435614656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b314a2a80 0x557b314a70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.909+0000 7f4436ffd640 1 -- 172.21.15.43:0/3435614656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b31499ea0 msgr2=0x557b314a6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:44.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.909+0000 7f4436ffd640 1 --2- 172.21.15.43:0/3435614656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b31499ea0 0x557b314a6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.909+0000 7f4436ffd640 1 -- 172.21.15.43:0/3435614656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b31518010 con 0x557b31499580 2024-04-21T08:51:44.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.909+0000 7f4436ffd640 1 --2- 172.21.15.43:0/3435614656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b31499580 0x557b314a6650 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f4428037040 tx=0x7f44280bef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b6882ae273100f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:44.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.909+0000 7f441aff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:44.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.909+0000 7f441aff5640 1 -- 172.21.15.43:0/3435614656 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44280bb480 con 0x557b31499580 2024-04-21T08:51:44.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.910+0000 7f441aff5640 1 -- 172.21.15.43:0/3435614656 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f4428009890 con 0x557b31499580 2024-04-21T08:51:44.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.910+0000 7f4440213740 1 -- 172.21.15.43:0/3435614656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44280c31c0 con 0x557b31499580 2024-04-21T08:51:44.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.910+0000 7f441aff5640 1 -- 172.21.15.43:0/3435614656 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f44280bb620 con 0x557b31499580 2024-04-21T08:51:44.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.910+0000 7f4440213740 10 client.21456 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:44.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.910+0000 7f4440213740 1 -- 172.21.15.43:0/3435614656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557b31519270 con 0x557b31499580 2024-04-21T08:51:44.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.910+0000 7f441aff5640 1 -- 172.21.15.43:0/3435614656 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44280c31c0 con 0x557b31499580 2024-04-21T08:51:44.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.910+0000 7f441aff5640 1 -- 172.21.15.43:0/3435614656 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f442803c030 con 0x557b31499580 2024-04-21T08:51:44.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.910+0000 7f4440213740 10 client.21456 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:44.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.910+0000 7f4440213740 10 client.21456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:44.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.910+0000 7f4440213740 20 client.21456 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:44.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.910+0000 7f4440213740 1 --2- 172.21.15.43:0/3435614656 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557b31519920 0x557b3151bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:44.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.910+0000 7f4440213740 4 client.21456 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:51:44.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.910+0000 7f4440213740 1 -- 172.21.15.43:0/3435614656 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557b31405a70 con 0x557b31519920 2024-04-21T08:51:44.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.910+0000 7f44367fc640 1 --2- 172.21.15.43:0/3435614656 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557b31519920 0x557b3151bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:44.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.911+0000 7f441aff5640 10 client.21456 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:44.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.911+0000 7f44367fc640 1 --2- 172.21.15.43:0/3435614656 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557b31519920 0x557b3151bd40 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73521bebda0794c1 server_cookie=2cf80ce16e4bcf1c in_seq=0 out_seq=0 2024-04-21T08:51:44.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.911+0000 7f441aff5640 1 -- 172.21.15.43:0/3435614656 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557b31405a70 con 0x557b31519920 2024-04-21T08:51:44.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.911+0000 7f441aff5640 10 client.21456 handle_command_reply: tid=0 2024-04-21T08:51:44.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.911+0000 7f4440213740 1 client.21456 shutdown 2024-04-21T08:51:44.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.911+0000 7f4440213740 1 -- 172.21.15.43:0/3435614656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b31499580 msgr2=0x557b314a6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:44.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.911+0000 7f4440213740 1 --2- 172.21.15.43:0/3435614656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b31499580 0x557b314a6650 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f4428037040 tx=0x7f44280bef50 comp rx=0 tx=0).stop 2024-04-21T08:51:44.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.911+0000 7f4440213740 1 -- 172.21.15.43:0/3435614656 shutdown_connections 2024-04-21T08:51:44.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.911+0000 7f4440213740 1 --2- 172.21.15.43:0/3435614656 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x557b31519920 0x557b3151bd40 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.911+0000 7f4440213740 1 --2- 172.21.15.43:0/3435614656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557b314a2a80 0x557b314a70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.911+0000 7f4440213740 1 --2- 172.21.15.43:0/3435614656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b31499ea0 0x557b314a6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.911+0000 7f441aff5640 0 client.21456 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:44.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.911+0000 7f4440213740 1 --2- 172.21.15.43:0/3435614656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b31499580 0x557b314a6650 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.911+0000 7f4440213740 1 -- 172.21.15.43:0/3435614656 >> 172.21.15.43:0/3435614656 conn(0x557b31493cb0 msgr2=0x557b31492d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:44.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.911+0000 7f4440213740 1 -- 172.21.15.43:0/3435614656 shutdown_connections 2024-04-21T08:51:44.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.911+0000 7f4440213740 1 -- 172.21.15.43:0/3435614656 wait complete. 2024-04-21T08:51:44.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.911+0000 7f4440213740 20 client.21456 trim_cache size 0 max 16384 2024-04-21T08:51:44.918 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:51:44.918 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-21T08:51:44.918 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-21T08:51:44.918 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:51:44.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.912+0000 7f443d54b640 1 -- 172.21.15.43:0/1543142169 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f44200a3d20 msgr2=0x7f44200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:44.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.912+0000 7f443d54b640 1 --2- 172.21.15.43:0/1543142169 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f44200a3d20 0x7f44200a61e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4430013400 tx=0x7f4430005d50 comp rx=0 tx=0).stop 2024-04-21T08:51:44.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.912+0000 7f443d54b640 1 -- 172.21.15.43:0/1543142169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44381031c0 msgr2=0x7f44381986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:44.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.912+0000 7f443d54b640 1 --2- 172.21.15.43:0/1543142169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44381031c0 0x7f44381986b0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f4428005e30 tx=0x7f4428003c20 comp rx=0 tx=0).stop 2024-04-21T08:51:44.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.912+0000 7f443d54b640 1 -- 172.21.15.43:0/1543142169 shutdown_connections 2024-04-21T08:51:44.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.912+0000 7f443d54b640 1 --2- 172.21.15.43:0/1543142169 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f44200a3d20 0x7f44200a61e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.912+0000 7f443d54b640 1 --2- 172.21.15.43:0/1543142169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f443810c500 0x7f443819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.912+0000 7f443d54b640 1 --2- 172.21.15.43:0/1543142169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4438103ae0 0x7f4438198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.912+0000 7f443d54b640 1 --2- 172.21.15.43:0/1543142169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44381031c0 0x7f44381986b0 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:44.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.912+0000 7f443d54b640 1 -- 172.21.15.43:0/1543142169 >> 172.21.15.43:0/1543142169 conn(0x7f4438078000 msgr2=0x7f44380fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:44.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.912+0000 7f443d54b640 1 -- 172.21.15.43:0/1543142169 shutdown_connections 2024-04-21T08:51:44.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:44.912+0000 7f443d54b640 1 -- 172.21.15.43:0/1543142169 wait complete. 2024-04-21T08:51:44.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:44 smithi043 ceph-mon[27978]: pgmap v400: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-04-21T08:51:44.923 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:51:44.923 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:51:44.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:44 smithi175 ceph-mon[31920]: pgmap v400: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-04-21T08:51:45.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.025+0000 7f8f466c8640 1 -- 172.21.15.43:0/2347815739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f40069aa0 msgr2=0x7f8f4010ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.025+0000 7f8f466c8640 1 --2- 172.21.15.43:0/2347815739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f40069aa0 0x7f8f4010ede0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009ca0 tx=0x7f8f3802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.026+0000 7f8f466c8640 1 -- 172.21.15.43:0/2347815739 shutdown_connections 2024-04-21T08:51:45.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.026+0000 7f8f466c8640 1 --2- 172.21.15.43:0/2347815739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f40069aa0 0x7f8f4010ede0 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.026+0000 7f8f466c8640 1 --2- 172.21.15.43:0/2347815739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f400690e0 0x7f8f40069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.026+0000 7f8f466c8640 1 --2- 172.21.15.43:0/2347815739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f40106f50 0x7f8f40107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.026+0000 7f8f466c8640 1 -- 172.21.15.43:0/2347815739 >> 172.21.15.43:0/2347815739 conn(0x7f8f40077e30 msgr2=0x7f8f40078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.026+0000 7f8f466c8640 1 -- 172.21.15.43:0/2347815739 shutdown_connections 2024-04-21T08:51:45.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.026+0000 7f8f466c8640 1 -- 172.21.15.43:0/2347815739 wait complete. 2024-04-21T08:51:45.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.026+0000 7f8f466c8640 1 Processor -- start 2024-04-21T08:51:45.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.026+0000 7f8f466c8640 1 -- start start 2024-04-21T08:51:45.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.027+0000 7f8f466c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f400690e0 0x7f8f40198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.027+0000 7f8f466c8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f40069aa0 0x7f8f40198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.027+0000 7f8f466c8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f40106f50 0x7f8f4019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.027+0000 7f8f466c8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8f40112150 con 0x7f8f40069aa0 2024-04-21T08:51:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.027+0000 7f8f466c8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f40111fd0 con 0x7f8f400690e0 2024-04-21T08:51:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.027+0000 7f8f466c8640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8f401122d0 con 0x7f8f40106f50 2024-04-21T08:51:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.027+0000 7f8f3f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f40069aa0 0x7f8f40198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.027+0000 7f8f3f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f40069aa0 0x7f8f40198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51248/0 (socket says 172.21.15.43:51248) 2024-04-21T08:51:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.027+0000 7f8f3f7fe640 1 -- 172.21.15.43:0/1358128324 learned_addr learned my addr 172.21.15.43:0/1358128324 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.027+0000 7f8f44c65640 1 --2- 172.21.15.43:0/1358128324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f40106f50 0x7f8f4019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.027+0000 7f8f3ffff640 1 --2- 172.21.15.43:0/1358128324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f400690e0 0x7f8f40198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.027+0000 7f8f3f7fe640 1 -- 172.21.15.43:0/1358128324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f40106f50 msgr2=0x7f8f4019cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.027+0000 7f8f3f7fe640 1 --2- 172.21.15.43:0/1358128324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f40106f50 0x7f8f4019cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.027+0000 7f8f3f7fe640 1 -- 172.21.15.43:0/1358128324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f400690e0 msgr2=0x7f8f40198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.027+0000 7f8f3f7fe640 1 --2- 172.21.15.43:0/1358128324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f400690e0 0x7f8f40198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.027+0000 7f8f3f7fe640 1 -- 172.21.15.43:0/1358128324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4019d5d0 con 0x7f8f40069aa0 2024-04-21T08:51:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.027+0000 7f8f3ffff640 1 --2- 172.21.15.43:0/1358128324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f400690e0 0x7f8f40198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.027+0000 7f8f44c65640 1 --2- 172.21.15.43:0/1358128324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f40106f50 0x7f8f4019cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:45.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.027+0000 7f8f3f7fe640 1 --2- 172.21.15.43:0/1358128324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f40069aa0 0x7f8f40198b70 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400d950 tx=0x7f8f3400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.027+0000 7f8f3d7fa640 1 -- 172.21.15.43:0/1358128324 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34017070 con 0x7f8f40069aa0 2024-04-21T08:51:45.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.027+0000 7f8f466c8640 1 -- 172.21.15.43:0/1358128324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f4019d8c0 con 0x7f8f40069aa0 2024-04-21T08:51:45.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.027+0000 7f8f3d7fa640 1 -- 172.21.15.43:0/1358128324 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f34009c30 con 0x7f8f40069aa0 2024-04-21T08:51:45.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.027+0000 7f8f3d7fa640 1 -- 172.21.15.43:0/1358128324 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400f620 con 0x7f8f40069aa0 2024-04-21T08:51:45.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.027+0000 7f8f466c8640 1 -- 172.21.15.43:0/1358128324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f401a5150 con 0x7f8f40069aa0 2024-04-21T08:51:45.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.028+0000 7f8f466c8640 1 -- 172.21.15.43:0/1358128324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f40068de0 con 0x7f8f40069aa0 2024-04-21T08:51:45.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.029+0000 7f8f3d7fa640 1 -- 172.21.15.43:0/1358128324 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8f34010460 con 0x7f8f40069aa0 2024-04-21T08:51:45.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.030+0000 7f8f3d7fa640 1 --2- 172.21.15.43:0/1358128324 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8f280a3d20 0x7f8f280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.030+0000 7f8f3d7fa640 1 -- 172.21.15.43:0/1358128324 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f8f340d0930 con 0x7f8f40069aa0 2024-04-21T08:51:45.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.030+0000 7f8f3ffff640 1 --2- 172.21.15.43:0/1358128324 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8f280a3d20 0x7f8f280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.031+0000 7f8f3ffff640 1 --2- 172.21.15.43:0/1358128324 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8f280a3d20 0x7f8f280a61e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009b40 tx=0x7f8f30005e50 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.035+0000 7f8f3d7fa640 1 -- 172.21.15.43:0/1358128324 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f3409cbb0 con 0x7f8f40069aa0 2024-04-21T08:51:45.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:44 smithi112 ceph-mon[31085]: pgmap v400: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-04-21T08:51:45.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.260+0000 7f8f466c8640 1 -- 172.21.15.43:0/1358128324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f4002d5c0 con 0x7f8f40069aa0 2024-04-21T08:51:45.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.261+0000 7f8f3d7fa640 1 -- 172.21.15.43:0/1358128324 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11935 (secure 0 0 0) 0x7f8f34009de0 con 0x7f8f40069aa0 2024-04-21T08:51:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:51:45.261 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:46:17.283806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27556":{"gid":27556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/229985237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":229985237},{"type":"v1","addr":"172.21.15.175:6837","nonce":229985237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24425,24412,24487,24527,24491]},"id":1}]} 2024-04-21T08:51:45.261 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-04-21T08:51:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.263+0000 7f8f466c8640 1 -- 172.21.15.43:0/1358128324 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8f280a3d20 msgr2=0x7f8f280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.263+0000 7f8f466c8640 1 --2- 172.21.15.43:0/1358128324 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8f280a3d20 0x7f8f280a61e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009b40 tx=0x7f8f30005e50 comp rx=0 tx=0).stop 2024-04-21T08:51:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.263+0000 7f8f466c8640 1 -- 172.21.15.43:0/1358128324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f40069aa0 msgr2=0x7f8f40198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.263+0000 7f8f466c8640 1 --2- 172.21.15.43:0/1358128324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f40069aa0 0x7f8f40198b70 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400d950 tx=0x7f8f3400de10 comp rx=0 tx=0).stop 2024-04-21T08:51:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.263+0000 7f8f466c8640 1 -- 172.21.15.43:0/1358128324 shutdown_connections 2024-04-21T08:51:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.263+0000 7f8f466c8640 1 --2- 172.21.15.43:0/1358128324 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f8f280a3d20 0x7f8f280a61e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.263+0000 7f8f466c8640 1 --2- 172.21.15.43:0/1358128324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f40106f50 0x7f8f4019cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.263+0000 7f8f466c8640 1 --2- 172.21.15.43:0/1358128324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f40069aa0 0x7f8f40198b70 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.263+0000 7f8f466c8640 1 --2- 172.21.15.43:0/1358128324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f400690e0 0x7f8f40198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.263+0000 7f8f466c8640 1 -- 172.21.15.43:0/1358128324 >> 172.21.15.43:0/1358128324 conn(0x7f8f40077e30 msgr2=0x7f8f400fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.263+0000 7f8f466c8640 1 -- 172.21.15.43:0/1358128324 shutdown_connections 2024-04-21T08:51:45.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.263+0000 7f8f466c8640 1 -- 172.21.15.43:0/1358128324 wait complete. 2024-04-21T08:51:45.275 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:51:45.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.380+0000 7f0d04e81640 1 -- 172.21.15.43:0/1654358236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d0010e960 msgr2=0x7f0d00110d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.380+0000 7f0d04e81640 1 --2- 172.21.15.43:0/1654358236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d0010e960 0x7f0d00110d50 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009ca0 tx=0x7f0cf802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.380+0000 7f0d04e81640 1 -- 172.21.15.43:0/1654358236 shutdown_connections 2024-04-21T08:51:45.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.380+0000 7f0d04e81640 1 --2- 172.21.15.43:0/1654358236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d0010e960 0x7f0d00110d50 unknown :-1 s=CLOSED pgs=2379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.380+0000 7f0d04e81640 1 --2- 172.21.15.43:0/1654358236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d0006aef0 0x7f0d0010e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.380+0000 7f0d04e81640 1 --2- 172.21.15.43:0/1654358236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0006a5d0 0x7f0d0006a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.380+0000 7f0d04e81640 1 -- 172.21.15.43:0/1654358236 >> 172.21.15.43:0/1654358236 conn(0x7f0d00077eb0 msgr2=0x7f0d000782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.380+0000 7f0d04e81640 1 -- 172.21.15.43:0/1654358236 shutdown_connections 2024-04-21T08:51:45.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.380+0000 7f0d04e81640 1 -- 172.21.15.43:0/1654358236 wait complete. 2024-04-21T08:51:45.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.381+0000 7f0d04e81640 1 Processor -- start 2024-04-21T08:51:45.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.381+0000 7f0d04e81640 1 -- start start 2024-04-21T08:51:45.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.381+0000 7f0d04e81640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0006a5d0 0x7f0d001a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.381+0000 7f0d04e81640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d0006aef0 0x7f0d001a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.381+0000 7f0d04e81640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d0010e960 0x7f0d0019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.381+0000 7f0d04e81640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0d00114390 con 0x7f0d0010e960 2024-04-21T08:51:45.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.381+0000 7f0d04e81640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0d00114210 con 0x7f0d0006a5d0 2024-04-21T08:51:45.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.381+0000 7f0d04e81640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d00114510 con 0x7f0d0006aef0 2024-04-21T08:51:45.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.381+0000 7f0cfed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d0010e960 0x7f0d0019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.381+0000 7f0cfed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d0010e960 0x7f0d0019a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51258/0 (socket says 172.21.15.43:51258) 2024-04-21T08:51:45.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.381+0000 7f0cfed9d640 1 -- 172.21.15.43:0/918212854 learned_addr learned my addr 172.21.15.43:0/918212854 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.382+0000 7f0cfe59c640 1 --2- 172.21.15.43:0/918212854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0006a5d0 0x7f0d001a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.382+0000 7f0cfed9d640 1 -- 172.21.15.43:0/918212854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d0006aef0 msgr2=0x7f0d001a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.382+0000 7f0cfed9d640 1 --2- 172.21.15.43:0/918212854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d0006aef0 0x7f0d001a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.382+0000 7f0cfed9d640 1 -- 172.21.15.43:0/918212854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0006a5d0 msgr2=0x7f0d001a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.382+0000 7f0cfed9d640 1 --2- 172.21.15.43:0/918212854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0006a5d0 0x7f0d001a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.382+0000 7f0cfed9d640 1 -- 172.21.15.43:0/918212854 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0019af90 con 0x7f0d0010e960 2024-04-21T08:51:45.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.382+0000 7f0cfed9d640 1 --2- 172.21.15.43:0/918212854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d0010e960 0x7f0d0019a6d0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8033000 tx=0x7f0cf80346e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.382+0000 7f0cf77fe640 1 -- 172.21.15.43:0/918212854 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8041020 con 0x7f0d0010e960 2024-04-21T08:51:45.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.382+0000 7f0cf77fe640 1 -- 172.21.15.43:0/918212854 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf8032970 con 0x7f0d0010e960 2024-04-21T08:51:45.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.382+0000 7f0d04e81640 1 -- 172.21.15.43:0/918212854 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d0019b220 con 0x7f0d0010e960 2024-04-21T08:51:45.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.382+0000 7f0cf77fe640 1 -- 172.21.15.43:0/918212854 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8039040 con 0x7f0d0010e960 2024-04-21T08:51:45.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.382+0000 7f0d04e81640 1 -- 172.21.15.43:0/918212854 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d001a73b0 con 0x7f0d0010e960 2024-04-21T08:51:45.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.384+0000 7f0cf77fe640 1 -- 172.21.15.43:0/918212854 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0cf8044020 con 0x7f0d0010e960 2024-04-21T08:51:45.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.384+0000 7f0d07a70740 1 Processor -- start 2024-04-21T08:51:45.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.385+0000 7f0d07a70740 1 -- start start 2024-04-21T08:51:45.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.385+0000 7f0d07a70740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f71a41c4d0 0x55f71a41c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.385+0000 7f0d07a70740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a41ce80 0x55f71a425430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.385+0000 7f0d07a70740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f71a425a00 0x55f71a427e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.385+0000 7f0d07a70740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f71a3811f0 con 0x55f71a41ce80 2024-04-21T08:51:45.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.385+0000 7f0d07a70740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f71a31e4b0 con 0x55f71a41c4d0 2024-04-21T08:51:45.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.385+0000 7f0d07a70740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f71a37d1b0 con 0x55f71a425a00 2024-04-21T08:51:45.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.385+0000 7f0cf77fe640 1 --2- 172.21.15.43:0/918212854 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0cec0a3d20 0x7f0cec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.385+0000 7f0cfe59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a41ce80 0x55f71a425430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.385+0000 7f0cfe59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a41ce80 0x55f71a425430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51272/0 (socket says 172.21.15.43:51272) 2024-04-21T08:51:45.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.385+0000 7f0cfe59c640 1 -- 172.21.15.43:0/1727070893 learned_addr learned my addr 172.21.15.43:0/1727070893 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.385+0000 7f0cfed9d640 1 --2- 172.21.15.43:0/918212854 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0cec0a3d20 0x7f0cec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.385+0000 7f0cf7fff640 1 --2- 172.21.15.43:0/1727070893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f71a425a00 0x55f71a427e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.385+0000 7f0cf7fff640 1 --2- 172.21.15.43:0/1727070893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f71a41c4d0 0x55f71a41c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.385+0000 7f0cf77fe640 1 -- 172.21.15.43:0/918212854 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f0cf80f2960 con 0x7f0d0010e960 2024-04-21T08:51:45.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.385+0000 7f0cfe59c640 1 -- 172.21.15.43:0/1727070893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f71a425a00 msgr2=0x55f71a427e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.385+0000 7f0cfe59c640 1 --2- 172.21.15.43:0/1727070893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f71a425a00 0x55f71a427e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.385+0000 7f0cfe59c640 1 -- 172.21.15.43:0/1727070893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f71a41c4d0 msgr2=0x55f71a41c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.385+0000 7f0cfe59c640 1 --2- 172.21.15.43:0/1727070893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f71a41c4d0 0x55f71a41c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.385+0000 7f0cfe59c640 1 -- 172.21.15.43:0/1727070893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f71a3101b0 con 0x55f71a41ce80 2024-04-21T08:51:45.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.386+0000 7f0cfe59c640 1 --2- 172.21.15.43:0/1727070893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a41ce80 0x55f71a425430 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000dcb0 tx=0x7f0cf000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=243f4f0ddfc16ae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.386+0000 7f0cfed9d640 1 --2- 172.21.15.43:0/918212854 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0cec0a3d20 0x7f0cec0a61e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80be2f0 tx=0x7f0cf8088040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.386+0000 7f0cf4ff9640 1 -- 172.21.15.43:0/1727070893 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0011440 con 0x55f71a41ce80 2024-04-21T08:51:45.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.386+0000 7f0cf4ff9640 1 -- 172.21.15.43:0/1727070893 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf0011a90 con 0x55f71a41ce80 2024-04-21T08:51:45.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.386+0000 7f0cf4ff9640 1 -- 172.21.15.43:0/1727070893 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0010990 con 0x55f71a41ce80 2024-04-21T08:51:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.386+0000 7f0d07a70740 1 -- 172.21.15.43:0/1727070893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a41ce80 msgr2=0x55f71a425430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.386+0000 7f0d07a70740 1 --2- 172.21.15.43:0/1727070893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a41ce80 0x55f71a425430 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000dcb0 tx=0x7f0cf000f730 comp rx=0 tx=0).stop 2024-04-21T08:51:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.387+0000 7f0d07a70740 1 -- 172.21.15.43:0/1727070893 shutdown_connections 2024-04-21T08:51:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.387+0000 7f0d07a70740 1 --2- 172.21.15.43:0/1727070893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f71a425a00 0x55f71a427e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.387+0000 7f0d07a70740 1 --2- 172.21.15.43:0/1727070893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a41ce80 0x55f71a425430 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.387+0000 7f0d07a70740 1 --2- 172.21.15.43:0/1727070893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f71a41c4d0 0x55f71a41c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.387+0000 7f0d07a70740 1 -- 172.21.15.43:0/1727070893 >> 172.21.15.43:0/1727070893 conn(0x55f71a416ba0 msgr2=0x55f71a416f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.387+0000 7f0d07a70740 1 -- 172.21.15.43:0/1727070893 shutdown_connections 2024-04-21T08:51:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.387+0000 7f0d07a70740 1 -- 172.21.15.43:0/1727070893 wait complete. 2024-04-21T08:51:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.387+0000 7f0d07a70740 1 -- start start 2024-04-21T08:51:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.387+0000 7f0d07a70740 1 Processor -- start 2024-04-21T08:51:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.387+0000 7f0d07a70740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f71a41c4d0 0x55f71a4252f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.387+0000 7f0d07a70740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f71a41ce80 0x55f71a422680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.387+0000 7f0d07a70740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a425a00 0x55f71a422c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.387+0000 7f0d07a70740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f71a428970 con 0x55f71a425a00 2024-04-21T08:51:45.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.387+0000 7f0d07a70740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f71a33a500 con 0x55f71a41ce80 2024-04-21T08:51:45.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.387+0000 7f0d07a70740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f71a428af0 con 0x55f71a41c4d0 2024-04-21T08:51:45.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.387+0000 7f0d07a70740 10 client.0 _ll_register_callbacks cb 0x55f71a308b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:45.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.387+0000 7f0cf7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a425a00 0x55f71a422c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.387+0000 7f0cf7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a425a00 0x55f71a422c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51280/0 (socket says 172.21.15.43:51280) 2024-04-21T08:51:45.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.388+0000 7f0cf7fff640 1 -- 172.21.15.43:0/3728159543 learned_addr learned my addr 172.21.15.43:0/3728159543 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.388+0000 7f0cf7fff640 1 --2- 172.21.15.43:0/3728159543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f71a41c4d0 0x55f71a4252f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.388+0000 7f0cfe59c640 1 --2- 172.21.15.43:0/3728159543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f71a41ce80 0x55f71a422680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.388+0000 7f0cf7fff640 1 -- 172.21.15.43:0/3728159543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f71a41c4d0 msgr2=0x55f71a4252f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.388+0000 7f0cf7fff640 1 --2- 172.21.15.43:0/3728159543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f71a41c4d0 0x55f71a4252f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.388+0000 7f0cf7fff640 1 -- 172.21.15.43:0/3728159543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f71a41ce80 msgr2=0x55f71a422680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.388+0000 7f0cf7fff640 1 --2- 172.21.15.43:0/3728159543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f71a41ce80 0x55f71a422680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.388+0000 7f0cf7fff640 1 -- 172.21.15.43:0/3728159543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f71a0933f0 con 0x55f71a425a00 2024-04-21T08:51:45.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.388+0000 7f0cf7fff640 1 --2- 172.21.15.43:0/3728159543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a425a00 0x55f71a422c00 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8012980 tx=0x7f0ce8005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2161b580a61fde2f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.388+0000 7f0ce5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.388+0000 7f0ce5ffb640 1 -- 172.21.15.43:0/3728159543 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8010ae0 con 0x55f71a425a00 2024-04-21T08:51:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.388+0000 7f0ce5ffb640 1 -- 172.21.15.43:0/3728159543 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f0ce8010c80 con 0x55f71a425a00 2024-04-21T08:51:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.388+0000 7f0d07a70740 1 -- 172.21.15.43:0/3728159543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf0011a90 con 0x55f71a425a00 2024-04-21T08:51:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.388+0000 7f0ce5ffb640 1 -- 172.21.15.43:0/3728159543 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0ce8010e80 con 0x55f71a425a00 2024-04-21T08:51:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.388+0000 7f0d07a70740 10 client.21474 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.388+0000 7f0d07a70740 1 -- 172.21.15.43:0/3728159543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f71a15d7d0 con 0x55f71a425a00 2024-04-21T08:51:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.388+0000 7f0ce5ffb640 1 -- 172.21.15.43:0/3728159543 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf0011a90 con 0x55f71a425a00 2024-04-21T08:51:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.389+0000 7f0ce5ffb640 1 -- 172.21.15.43:0/3728159543 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0ce80047a0 con 0x55f71a425a00 2024-04-21T08:51:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.389+0000 7f0d07a70740 10 client.21474 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.389+0000 7f0d07a70740 10 client.21474 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:51:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.389+0000 7f0d07a70740 20 client.21474 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.389+0000 7f0d07a70740 1 --2- 172.21.15.43:0/3728159543 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55f71a435bd0 0x55f71a438030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.389+0000 7f0d07a70740 4 client.21474 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.389+0000 7f0d07a70740 1 -- 172.21.15.43:0/3728159543 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f71a320930 con 0x55f71a435bd0 2024-04-21T08:51:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.389+0000 7f0cfe59c640 1 --2- 172.21.15.43:0/3728159543 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55f71a435bd0 0x55f71a438030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.389+0000 7f0cfe59c640 1 --2- 172.21.15.43:0/3728159543 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55f71a435bd0 0x55f71a438030 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f8bfb909031ee8e4 server_cookie=16271a9169e67aef in_seq=0 out_seq=0 2024-04-21T08:51:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.389+0000 7f0ce5ffb640 10 client.21474 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:51:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.390+0000 7f0ce5ffb640 1 -- 172.21.15.43:0/3728159543 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f71a320930 con 0x55f71a435bd0 2024-04-21T08:51:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.390+0000 7f0ce5ffb640 10 client.21474 handle_command_reply: tid=0 2024-04-21T08:51:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.390+0000 7f0d07a70740 1 client.21474 shutdown 2024-04-21T08:51:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.390+0000 7f0d07a70740 1 -- 172.21.15.43:0/3728159543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a425a00 msgr2=0x55f71a422c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.390+0000 7f0d07a70740 1 --2- 172.21.15.43:0/3728159543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a425a00 0x55f71a422c00 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8012980 tx=0x7f0ce8005ed0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.390+0000 7f0d07a70740 1 -- 172.21.15.43:0/3728159543 shutdown_connections 2024-04-21T08:51:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.390+0000 7f0d07a70740 1 --2- 172.21.15.43:0/3728159543 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55f71a435bd0 0x55f71a438030 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.390+0000 7f0d07a70740 1 --2- 172.21.15.43:0/3728159543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a425a00 0x55f71a422c00 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.390+0000 7f0d07a70740 1 --2- 172.21.15.43:0/3728159543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f71a41ce80 0x55f71a422680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.390+0000 7f0ce5ffb640 0 client.21474 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:51:45.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.390+0000 7f0d07a70740 1 --2- 172.21.15.43:0/3728159543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f71a41c4d0 0x55f71a4252f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.390+0000 7f0d07a70740 1 -- 172.21.15.43:0/3728159543 >> 172.21.15.43:0/3728159543 conn(0x55f71a416ba0 msgr2=0x55f71a4188f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.390+0000 7f0d07a70740 1 -- 172.21.15.43:0/3728159543 shutdown_connections 2024-04-21T08:51:45.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.390+0000 7f0d07a70740 1 -- 172.21.15.43:0/3728159543 wait complete. 2024-04-21T08:51:45.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.390+0000 7f0d07a70740 20 client.21474 trim_cache size 0 max 16384 2024-04-21T08:51:45.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.404+0000 7f0d07a70740 1 Processor -- start 2024-04-21T08:51:45.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.404+0000 7f0d07a70740 1 -- start start 2024-04-21T08:51:45.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.404+0000 7f0d07a70740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f71a41c4d0 0x55f71a415e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.404+0000 7f0d07a70740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a41ce80 0x55f71a431860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.404+0000 7f0d07a70740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f71a425a00 0x55f71a424db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.404+0000 7f0d07a70740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0d001b1780 con 0x55f71a41ce80 2024-04-21T08:51:45.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.404+0000 7f0d07a70740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0d001b32c0 con 0x55f71a41c4d0 2024-04-21T08:51:45.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.404+0000 7f0d07a70740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d001b1e50 con 0x55f71a425a00 2024-04-21T08:51:45.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.404+0000 7f0cfe59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a41ce80 0x55f71a431860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.404+0000 7f0cfe59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a41ce80 0x55f71a431860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51292/0 (socket says 172.21.15.43:51292) 2024-04-21T08:51:45.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.404+0000 7f0cfe59c640 1 -- 172.21.15.43:0/3469773015 learned_addr learned my addr 172.21.15.43:0/3469773015 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.404+0000 7f0cf7fff640 1 --2- 172.21.15.43:0/3469773015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f71a41c4d0 0x55f71a415e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.405+0000 7f0cfe59c640 1 -- 172.21.15.43:0/3469773015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f71a425a00 msgr2=0x55f71a424db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.405+0000 7f0cfe59c640 1 --2- 172.21.15.43:0/3469773015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f71a425a00 0x55f71a424db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.405+0000 7f0cfe59c640 1 -- 172.21.15.43:0/3469773015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f71a41c4d0 msgr2=0x55f71a415e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.405+0000 7f0cfe59c640 1 --2- 172.21.15.43:0/3469773015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f71a41c4d0 0x55f71a415e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.405+0000 7f0cfe59c640 1 -- 172.21.15.43:0/3469773015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f71a420bd0 con 0x55f71a41ce80 2024-04-21T08:51:45.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.405+0000 7f0cfe59c640 1 --2- 172.21.15.43:0/3469773015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a41ce80 0x55f71a431860 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000dbd0 tx=0x7f0cf0010610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2d2ef00418362e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.405+0000 7f0ce7fff640 1 -- 172.21.15.43:0/3469773015 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0004040 con 0x55f71a41ce80 2024-04-21T08:51:45.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.405+0000 7f0ce7fff640 1 -- 172.21.15.43:0/3469773015 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf002d070 con 0x55f71a41ce80 2024-04-21T08:51:45.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.405+0000 7f0ce7fff640 1 -- 172.21.15.43:0/3469773015 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0004680 con 0x55f71a41ce80 2024-04-21T08:51:45.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.405+0000 7f0d07a70740 1 -- 172.21.15.43:0/3469773015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a41ce80 msgr2=0x55f71a431860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.405+0000 7f0d07a70740 1 --2- 172.21.15.43:0/3469773015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a41ce80 0x55f71a431860 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000dbd0 tx=0x7f0cf0010610 comp rx=0 tx=0).stop 2024-04-21T08:51:45.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.405+0000 7f0d07a70740 1 -- 172.21.15.43:0/3469773015 shutdown_connections 2024-04-21T08:51:45.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.405+0000 7f0d07a70740 1 --2- 172.21.15.43:0/3469773015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f71a425a00 0x55f71a424db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.405+0000 7f0d07a70740 1 --2- 172.21.15.43:0/3469773015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a41ce80 0x55f71a431860 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.405+0000 7f0d07a70740 1 --2- 172.21.15.43:0/3469773015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f71a41c4d0 0x55f71a415e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.405+0000 7f0d07a70740 1 -- 172.21.15.43:0/3469773015 >> 172.21.15.43:0/3469773015 conn(0x55f71a416ba0 msgr2=0x55f71a418880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.406+0000 7f0d07a70740 1 -- 172.21.15.43:0/3469773015 shutdown_connections 2024-04-21T08:51:45.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.406+0000 7f0d07a70740 1 -- 172.21.15.43:0/3469773015 wait complete. 2024-04-21T08:51:45.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.406+0000 7f0d07a70740 1 -- start start 2024-04-21T08:51:45.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.406+0000 7f0d07a70740 1 Processor -- start 2024-04-21T08:51:45.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.406+0000 7f0d07a70740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f71a41c4d0 0x55f71a41f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.406+0000 7f0d07a70740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f71a41ce80 0x55f71a41f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.406+0000 7f0d07a70740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a425a00 0x55f71a41fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.406+0000 7f0d07a70740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0d001b2bf0 con 0x55f71a425a00 2024-04-21T08:51:45.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.406+0000 7f0d07a70740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0d001b2520 con 0x55f71a41ce80 2024-04-21T08:51:45.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.406+0000 7f0d07a70740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d001af780 con 0x55f71a41c4d0 2024-04-21T08:51:45.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.406+0000 7f0d07a70740 10 client.0 _ll_register_callbacks cb 0x7f0ce8012590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:45.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.406+0000 7f0cf7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a425a00 0x55f71a41fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.406+0000 7f0cf7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a425a00 0x55f71a41fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51302/0 (socket says 172.21.15.43:51302) 2024-04-21T08:51:45.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.406+0000 7f0cf7fff640 1 -- 172.21.15.43:0/878481719 learned_addr learned my addr 172.21.15.43:0/878481719 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.406+0000 7f0cf7fff640 1 -- 172.21.15.43:0/878481719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f71a41c4d0 msgr2=0x55f71a41f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.406+0000 7f0cf7fff640 1 --2- 172.21.15.43:0/878481719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f71a41c4d0 0x55f71a41f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.406+0000 7f0cfe59c640 1 --2- 172.21.15.43:0/878481719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f71a41ce80 0x55f71a41f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.406+0000 7f0cf7fff640 1 -- 172.21.15.43:0/878481719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f71a41ce80 msgr2=0x55f71a41f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.406+0000 7f0cf7fff640 1 --2- 172.21.15.43:0/878481719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f71a41ce80 0x55f71a41f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.406+0000 7f0cf7fff640 1 -- 172.21.15.43:0/878481719 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f71a49b170 con 0x55f71a425a00 2024-04-21T08:51:45.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.407+0000 7f0cf7fff640 1 --2- 172.21.15.43:0/878481719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a425a00 0x55f71a41fd30 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80126a0 tx=0x7f0ce8012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3e3d81f9780da5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.407+0000 7f0ce5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:45.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.407+0000 7f0ce5ffb640 1 -- 172.21.15.43:0/878481719 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8004de0 con 0x55f71a425a00 2024-04-21T08:51:45.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.407+0000 7f0ce5ffb640 1 -- 172.21.15.43:0/878481719 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f0ce8010ae0 con 0x55f71a425a00 2024-04-21T08:51:45.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.407+0000 7f0d07a70740 1 -- 172.21.15.43:0/878481719 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf002d070 con 0x55f71a425a00 2024-04-21T08:51:45.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.407+0000 7f0ce5ffb640 1 -- 172.21.15.43:0/878481719 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0ce8039050 con 0x55f71a425a00 2024-04-21T08:51:45.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.407+0000 7f0d07a70740 10 client.21480 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:45.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.407+0000 7f0d07a70740 1 -- 172.21.15.43:0/878481719 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f71a49c410 con 0x55f71a425a00 2024-04-21T08:51:45.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.407+0000 7f0ce5ffb640 1 -- 172.21.15.43:0/878481719 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf002d070 con 0x55f71a425a00 2024-04-21T08:51:45.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.407+0000 7f0ce5ffb640 1 -- 172.21.15.43:0/878481719 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0ce8034030 con 0x55f71a425a00 2024-04-21T08:51:45.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.407+0000 7f0d07a70740 10 client.21480 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:45.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.407+0000 7f0d07a70740 10 client.21480 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:51:45.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.407+0000 7f0d07a70740 20 client.21480 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:45.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.407+0000 7f0d07a70740 1 --2- 172.21.15.43:0/878481719 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55f71a49cad0 0x55f71a49eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.407+0000 7f0d07a70740 4 client.21480 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:51:45.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.407+0000 7f0d07a70740 1 -- 172.21.15.43:0/878481719 --> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f71a38ec10 con 0x55f71a49cad0 2024-04-21T08:51:45.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.408+0000 7f0cfe59c640 1 --2- 172.21.15.43:0/878481719 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55f71a49cad0 0x55f71a49eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.408+0000 7f0cfe59c640 1 --2- 172.21.15.43:0/878481719 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55f71a49cad0 0x55f71a49eef0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bde4006e9023bd0f server_cookie=5c1abb7ecdad94c9 in_seq=0 out_seq=0 2024-04-21T08:51:45.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.408+0000 7f0ce5ffb640 10 client.21480 ms_handle_connect on v2:172.21.15.175:6832/187945366 2024-04-21T08:51:45.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.408+0000 7f0ce5ffb640 1 -- 172.21.15.43:0/878481719 <== mds.4 v2:172.21.15.175:6832/187945366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f71a38ec10 con 0x55f71a49cad0 2024-04-21T08:51:45.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.408+0000 7f0ce5ffb640 10 client.21480 handle_command_reply: tid=0 2024-04-21T08:51:45.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.408+0000 7f0d07a70740 1 client.21480 shutdown 2024-04-21T08:51:45.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.408+0000 7f0d07a70740 1 -- 172.21.15.43:0/878481719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a425a00 msgr2=0x55f71a41fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.408+0000 7f0d07a70740 1 --2- 172.21.15.43:0/878481719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a425a00 0x55f71a41fd30 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80126a0 tx=0x7f0ce8012450 comp rx=0 tx=0).stop 2024-04-21T08:51:45.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.408+0000 7f0d07a70740 1 -- 172.21.15.43:0/878481719 shutdown_connections 2024-04-21T08:51:45.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.408+0000 7f0d07a70740 1 --2- 172.21.15.43:0/878481719 >> [v2:172.21.15.175:6832/187945366,v1:172.21.15.175:6833/187945366] conn(0x55f71a49cad0 0x55f71a49eef0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.408+0000 7f0d07a70740 1 --2- 172.21.15.43:0/878481719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f71a425a00 0x55f71a41fd30 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.408+0000 7f0ce5ffb640 0 client.21480 ms_handle_reset on v2:172.21.15.175:6832/187945366 2024-04-21T08:51:45.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.408+0000 7f0d07a70740 1 --2- 172.21.15.43:0/878481719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f71a41ce80 0x55f71a41f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.408+0000 7f0d07a70740 1 --2- 172.21.15.43:0/878481719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f71a41c4d0 0x55f71a41f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.408+0000 7f0d07a70740 1 -- 172.21.15.43:0/878481719 >> 172.21.15.43:0/878481719 conn(0x55f71a416ba0 msgr2=0x55f71a420e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.408+0000 7f0d07a70740 1 -- 172.21.15.43:0/878481719 shutdown_connections 2024-04-21T08:51:45.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.408+0000 7f0d07a70740 1 -- 172.21.15.43:0/878481719 wait complete. 2024-04-21T08:51:45.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.409+0000 7f0d07a70740 20 client.21480 trim_cache size 0 max 16384 2024-04-21T08:51:45.415 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:51:45.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.409+0000 7f0d04e81640 1 -- 172.21.15.43:0/918212854 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0cec0a3d20 msgr2=0x7f0cec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.409+0000 7f0d04e81640 1 --2- 172.21.15.43:0/918212854 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0cec0a3d20 0x7f0cec0a61e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80be2f0 tx=0x7f0cf8088040 comp rx=0 tx=0).stop 2024-04-21T08:51:45.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.409+0000 7f0d04e81640 1 -- 172.21.15.43:0/918212854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d0010e960 msgr2=0x7f0d0019a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.409+0000 7f0d04e81640 1 --2- 172.21.15.43:0/918212854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d0010e960 0x7f0d0019a6d0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8033000 tx=0x7f0cf80346e0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.409+0000 7f0d04e81640 1 -- 172.21.15.43:0/918212854 shutdown_connections 2024-04-21T08:51:45.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.409+0000 7f0d04e81640 1 --2- 172.21.15.43:0/918212854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d0010e960 0x7f0d0019a6d0 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.409+0000 7f0d04e81640 1 --2- 172.21.15.43:0/918212854 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f0cec0a3d20 0x7f0cec0a61e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.409+0000 7f0d04e81640 1 --2- 172.21.15.43:0/918212854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d0006aef0 0x7f0d001a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.409+0000 7f0d04e81640 1 --2- 172.21.15.43:0/918212854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0006a5d0 0x7f0d001a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.409+0000 7f0d04e81640 1 -- 172.21.15.43:0/918212854 >> 172.21.15.43:0/918212854 conn(0x7f0d00077eb0 msgr2=0x7f0d0010fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.409+0000 7f0d04e81640 1 -- 172.21.15.43:0/918212854 shutdown_connections 2024-04-21T08:51:45.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.409+0000 7f0d04e81640 1 -- 172.21.15.43:0/918212854 wait complete. 2024-04-21T08:51:45.420 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:51:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.524+0000 7f05ec49a640 1 -- 172.21.15.43:0/285827961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05e4101aa0 msgr2=0x7f05e4105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.524+0000 7f05ec49a640 1 --2- 172.21.15.43:0/285827961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05e4101aa0 0x7f05e4105a90 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009900 tx=0x7f05d402bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.524+0000 7f05ec49a640 1 -- 172.21.15.43:0/285827961 shutdown_connections 2024-04-21T08:51:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.524+0000 7f05ec49a640 1 --2- 172.21.15.43:0/285827961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05e4106170 0x7f05e410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.524+0000 7f05ec49a640 1 --2- 172.21.15.43:0/285827961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05e4101aa0 0x7f05e4105a90 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.524+0000 7f05ec49a640 1 --2- 172.21.15.43:0/285827961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05e41010f0 0x7f05e41014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.524+0000 7f05ec49a640 1 -- 172.21.15.43:0/285827961 >> 172.21.15.43:0/285827961 conn(0x7f05e4077fb0 msgr2=0x7f05e40783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.524+0000 7f05ec49a640 1 -- 172.21.15.43:0/285827961 shutdown_connections 2024-04-21T08:51:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.524+0000 7f05ec49a640 1 -- 172.21.15.43:0/285827961 wait complete. 2024-04-21T08:51:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.525+0000 7f05ec49a640 1 Processor -- start 2024-04-21T08:51:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.525+0000 7f05ec49a640 1 -- start start 2024-04-21T08:51:45.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.525+0000 7f05ec49a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05e41010f0 0x7f05e419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.525+0000 7f05ec49a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05e4106170 0x7f05e419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.525+0000 7f05ec49a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05e4198630 0x7f05e4198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.525+0000 7f05ec49a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f05e41121f0 con 0x7f05e4198630 2024-04-21T08:51:45.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.525+0000 7f05ec49a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f05e4112070 con 0x7f05e4106170 2024-04-21T08:51:45.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.525+0000 7f05ec49a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05e4112370 con 0x7f05e41010f0 2024-04-21T08:51:45.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.525+0000 7f05eaa37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05e4198630 0x7f05e4198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.525+0000 7f05eaa37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05e4198630 0x7f05e4198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51310/0 (socket says 172.21.15.43:51310) 2024-04-21T08:51:45.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.525+0000 7f05eaa37640 1 -- 172.21.15.43:0/64687500 learned_addr learned my addr 172.21.15.43:0/64687500 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.525+0000 7f05ea236640 1 --2- 172.21.15.43:0/64687500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05e41010f0 0x7f05e419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.525+0000 7f05e9a35640 1 --2- 172.21.15.43:0/64687500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05e4106170 0x7f05e419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.525+0000 7f05eaa37640 1 -- 172.21.15.43:0/64687500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05e41010f0 msgr2=0x7f05e419e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.525+0000 7f05eaa37640 1 --2- 172.21.15.43:0/64687500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05e41010f0 0x7f05e419e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.525+0000 7f05eaa37640 1 -- 172.21.15.43:0/64687500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05e4106170 msgr2=0x7f05e419e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.525+0000 7f05eaa37640 1 --2- 172.21.15.43:0/64687500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05e4106170 0x7f05e419e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.525+0000 7f05eaa37640 1 -- 172.21.15.43:0/64687500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e4199150 con 0x7f05e4198630 2024-04-21T08:51:45.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.525+0000 7f05eaa37640 1 --2- 172.21.15.43:0/64687500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05e4198630 0x7f05e4198ae0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00cab0 tx=0x7f05dc00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.526+0000 7f05db7fe640 1 -- 172.21.15.43:0/64687500 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc00adc0 con 0x7f05e4198630 2024-04-21T08:51:45.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.526+0000 7f05db7fe640 1 -- 172.21.15.43:0/64687500 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05dc00f900 con 0x7f05e4198630 2024-04-21T08:51:45.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.526+0000 7f05ec49a640 1 -- 172.21.15.43:0/64687500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05e4199440 con 0x7f05e4198630 2024-04-21T08:51:45.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.526+0000 7f05db7fe640 1 -- 172.21.15.43:0/64687500 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc0189a0 con 0x7f05e4198630 2024-04-21T08:51:45.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.526+0000 7f05ec49a640 1 -- 172.21.15.43:0/64687500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05e41a55a0 con 0x7f05e4198630 2024-04-21T08:51:45.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.528+0000 7f05db7fe640 1 -- 172.21.15.43:0/64687500 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f05dc00f450 con 0x7f05e4198630 2024-04-21T08:51:45.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.528+0000 7f05ef16f740 1 Processor -- start 2024-04-21T08:51:45.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.529+0000 7f05ef16f740 1 -- start start 2024-04-21T08:51:45.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.529+0000 7f05ef16f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bd5a904d0 0x560bd5a908b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.529+0000 7f05ef16f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a90e80 0x560bd5a99430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.529+0000 7f05ef16f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bd5a99a00 0x560bd5a9be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.529+0000 7f05ef16f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560bd59f51f0 con 0x560bd5a90e80 2024-04-21T08:51:45.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.529+0000 7f05ef16f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560bd59924b0 con 0x560bd5a99a00 2024-04-21T08:51:45.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.529+0000 7f05ef16f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560bd59f11b0 con 0x560bd5a904d0 2024-04-21T08:51:45.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.529+0000 7f05ea236640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a90e80 0x560bd5a99430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.529+0000 7f05ea236640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a90e80 0x560bd5a99430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51326/0 (socket says 172.21.15.43:51326) 2024-04-21T08:51:45.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.529+0000 7f05ea236640 1 -- 172.21.15.43:0/4279302794 learned_addr learned my addr 172.21.15.43:0/4279302794 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.529+0000 7f05db7fe640 1 --2- 172.21.15.43:0/64687500 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f05d00a3d20 0x7f05d00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.529+0000 7f05e9a35640 1 --2- 172.21.15.43:0/4279302794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bd5a99a00 0x560bd5a9be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.529+0000 7f05e9a35640 1 --2- 172.21.15.43:0/4279302794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bd5a904d0 0x560bd5a908b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.529+0000 7f05eaa37640 1 --2- 172.21.15.43:0/64687500 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f05d00a3d20 0x7f05d00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.529+0000 7f05db7fe640 1 -- 172.21.15.43:0/64687500 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f05dc0d0e60 con 0x7f05e4198630 2024-04-21T08:51:45.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.529+0000 7f05ea236640 1 -- 172.21.15.43:0/4279302794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bd5a904d0 msgr2=0x560bd5a908b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.529+0000 7f05ea236640 1 --2- 172.21.15.43:0/4279302794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bd5a904d0 0x560bd5a908b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.529+0000 7f05ea236640 1 -- 172.21.15.43:0/4279302794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bd5a99a00 msgr2=0x560bd5a9be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.529+0000 7f05ea236640 1 --2- 172.21.15.43:0/4279302794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bd5a99a00 0x560bd5a9be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.529+0000 7f05ea236640 1 -- 172.21.15.43:0/4279302794 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bd59841b0 con 0x560bd5a90e80 2024-04-21T08:51:45.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.529+0000 7f05e9a35640 1 --2- 172.21.15.43:0/4279302794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bd5a904d0 0x560bd5a908b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:45.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.530+0000 7f05ea236640 1 --2- 172.21.15.43:0/4279302794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a90e80 0x560bd5a99430 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7f05e000b760 tx=0x7f05e000bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da53ec200e1a11b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.530+0000 7f05eaa37640 1 --2- 172.21.15.43:0/64687500 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f05d00a3d20 0x7f05d00a61e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f05dc0ce240 tx=0x7f05dc091e70 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.530+0000 7f05d8bfe640 1 -- 172.21.15.43:0/4279302794 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0003dd0 con 0x560bd5a90e80 2024-04-21T08:51:45.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.530+0000 7f05d8bfe640 1 -- 172.21.15.43:0/4279302794 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e0015070 con 0x560bd5a90e80 2024-04-21T08:51:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.530+0000 7f05d8bfe640 1 -- 172.21.15.43:0/4279302794 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0004d50 con 0x560bd5a90e80 2024-04-21T08:51:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.531+0000 7f05ef16f740 1 -- 172.21.15.43:0/4279302794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a90e80 msgr2=0x560bd5a99430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.531+0000 7f05ef16f740 1 --2- 172.21.15.43:0/4279302794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a90e80 0x560bd5a99430 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7f05e000b760 tx=0x7f05e000bc20 comp rx=0 tx=0).stop 2024-04-21T08:51:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.531+0000 7f05ef16f740 1 -- 172.21.15.43:0/4279302794 shutdown_connections 2024-04-21T08:51:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.531+0000 7f05ef16f740 1 --2- 172.21.15.43:0/4279302794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bd5a99a00 0x560bd5a9be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.531+0000 7f05ef16f740 1 --2- 172.21.15.43:0/4279302794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a90e80 0x560bd5a99430 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.531+0000 7f05ef16f740 1 --2- 172.21.15.43:0/4279302794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bd5a904d0 0x560bd5a908b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.531+0000 7f05ef16f740 1 -- 172.21.15.43:0/4279302794 >> 172.21.15.43:0/4279302794 conn(0x560bd5a8aba0 msgr2=0x560bd5a8af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.531+0000 7f05ef16f740 1 -- 172.21.15.43:0/4279302794 shutdown_connections 2024-04-21T08:51:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.531+0000 7f05ef16f740 1 -- 172.21.15.43:0/4279302794 wait complete. 2024-04-21T08:51:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.531+0000 7f05ef16f740 1 -- start start 2024-04-21T08:51:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.531+0000 7f05ef16f740 1 Processor -- start 2024-04-21T08:51:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.531+0000 7f05ef16f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a904d0 0x560bd5a99380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.531+0000 7f05ef16f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bd5a90e80 0x560bd5a96710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.531+0000 7f05ef16f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bd5a99a00 0x560bd5a96d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.531+0000 7f05ef16f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560bd5a9c970 con 0x560bd5a904d0 2024-04-21T08:51:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.531+0000 7f05ef16f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560bd59ae500 con 0x560bd5a99a00 2024-04-21T08:51:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.531+0000 7f05ef16f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560bd5a9caf0 con 0x560bd5a90e80 2024-04-21T08:51:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.531+0000 7f05ef16f740 10 client.0 _ll_register_callbacks cb 0x560bd597cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.531+0000 7f05e9a35640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a904d0 0x560bd5a99380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.532+0000 7f05e9a35640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a904d0 0x560bd5a99380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51338/0 (socket says 172.21.15.43:51338) 2024-04-21T08:51:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.532+0000 7f05e9a35640 1 -- 172.21.15.43:0/255158343 learned_addr learned my addr 172.21.15.43:0/255158343 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.532+0000 7f05e9a35640 1 -- 172.21.15.43:0/255158343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bd5a90e80 msgr2=0x560bd5a96710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.532+0000 7f05ea236640 1 --2- 172.21.15.43:0/255158343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bd5a90e80 0x560bd5a96710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.532+0000 7f05e9a35640 1 --2- 172.21.15.43:0/255158343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bd5a90e80 0x560bd5a96710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.532+0000 7f05e9a35640 1 -- 172.21.15.43:0/255158343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bd5a99a00 msgr2=0x560bd5a96d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.532+0000 7f05e9a35640 1 --2- 172.21.15.43:0/255158343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bd5a99a00 0x560bd5a96d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.532+0000 7f05e9a35640 1 -- 172.21.15.43:0/255158343 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bd57073f0 con 0x560bd5a904d0 2024-04-21T08:51:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.532+0000 7f05ea236640 1 --2- 172.21.15.43:0/255158343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bd5a90e80 0x560bd5a96710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.532+0000 7f05e9a35640 1 --2- 172.21.15.43:0/255158343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a904d0 0x560bd5a99380 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f05e4068e50 tx=0x7f05d402c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c8f97ae3c4d636 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.532+0000 7f05cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.532+0000 7f05cdffb640 1 -- 172.21.15.43:0/255158343 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4047070 con 0x560bd5a904d0 2024-04-21T08:51:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.532+0000 7f05cdffb640 1 -- 172.21.15.43:0/255158343 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f05d4005040 con 0x560bd5a904d0 2024-04-21T08:51:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.532+0000 7f05ef16f740 1 -- 172.21.15.43:0/255158343 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05e0015070 con 0x560bd5a904d0 2024-04-21T08:51:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.532+0000 7f05cdffb640 1 -- 172.21.15.43:0/255158343 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f05d4053050 con 0x560bd5a904d0 2024-04-21T08:51:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.532+0000 7f05ef16f740 10 client.21492 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.532+0000 7f05ef16f740 1 -- 172.21.15.43:0/255158343 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560bd57d17d0 con 0x560bd5a904d0 2024-04-21T08:51:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.533+0000 7f05cdffb640 1 -- 172.21.15.43:0/255158343 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e0015070 con 0x560bd5a904d0 2024-04-21T08:51:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.533+0000 7f05cdffb640 1 -- 172.21.15.43:0/255158343 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f05d4040040 con 0x560bd5a904d0 2024-04-21T08:51:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.533+0000 7f05ef16f740 10 client.21492 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.533+0000 7f05ef16f740 10 client.21492 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:51:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.533+0000 7f05ef16f740 20 client.21492 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.533+0000 7f05ef16f740 1 --2- 172.21.15.43:0/255158343 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x560bd5aa9be0 0x560bd5aac030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.533+0000 7f05ef16f740 4 client.21492 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:45.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.533+0000 7f05ef16f740 1 -- 172.21.15.43:0/255158343 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bd5994930 con 0x560bd5aa9be0 2024-04-21T08:51:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.533+0000 7f05ea236640 1 --2- 172.21.15.43:0/255158343 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x560bd5aa9be0 0x560bd5aac030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.534+0000 7f05ea236640 1 --2- 172.21.15.43:0/255158343 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x560bd5aa9be0 0x560bd5aac030 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=873743445d6793e5 server_cookie=6a48ccd079db9b09 in_seq=0 out_seq=0 2024-04-21T08:51:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.534+0000 7f05cdffb640 10 client.21492 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:51:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.534+0000 7f05cdffb640 1 -- 172.21.15.43:0/255158343 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560bd5994930 con 0x560bd5aa9be0 2024-04-21T08:51:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.534+0000 7f05cdffb640 10 client.21492 handle_command_reply: tid=0 2024-04-21T08:51:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.534+0000 7f05ef16f740 1 client.21492 shutdown 2024-04-21T08:51:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.534+0000 7f05ef16f740 1 -- 172.21.15.43:0/255158343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a904d0 msgr2=0x560bd5a99380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.534+0000 7f05ef16f740 1 --2- 172.21.15.43:0/255158343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a904d0 0x560bd5a99380 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f05e4068e50 tx=0x7f05d402c2b0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.534+0000 7f05ef16f740 1 -- 172.21.15.43:0/255158343 shutdown_connections 2024-04-21T08:51:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.534+0000 7f05ef16f740 1 --2- 172.21.15.43:0/255158343 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x560bd5aa9be0 0x560bd5aac030 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.534+0000 7f05ef16f740 1 --2- 172.21.15.43:0/255158343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bd5a99a00 0x560bd5a96d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.534+0000 7f05ef16f740 1 --2- 172.21.15.43:0/255158343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bd5a90e80 0x560bd5a96710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.534+0000 7f05cdffb640 0 client.21492 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:51:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.534+0000 7f05ef16f740 1 --2- 172.21.15.43:0/255158343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a904d0 0x560bd5a99380 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.534+0000 7f05ef16f740 1 -- 172.21.15.43:0/255158343 >> 172.21.15.43:0/255158343 conn(0x560bd5a8aba0 msgr2=0x560bd5a8c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.535+0000 7f05ef16f740 1 -- 172.21.15.43:0/255158343 shutdown_connections 2024-04-21T08:51:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.535+0000 7f05ef16f740 1 -- 172.21.15.43:0/255158343 wait complete. 2024-04-21T08:51:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.535+0000 7f05ef16f740 20 client.21492 trim_cache size 0 max 16384 2024-04-21T08:51:45.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.548+0000 7f05ef16f740 1 Processor -- start 2024-04-21T08:51:45.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.548+0000 7f05ef16f740 1 -- start start 2024-04-21T08:51:45.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.548+0000 7f05ef16f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bd5a904d0 0x560bd5a98e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.548+0000 7f05ef16f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a90e80 0x560bd5a99380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.548+0000 7f05ef16f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bd5a99a00 0x560bd5ab1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.548+0000 7f05ef16f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f05e41af4f0 con 0x560bd5a90e80 2024-04-21T08:51:45.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.548+0000 7f05ef16f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f05e41b1030 con 0x560bd5a904d0 2024-04-21T08:51:45.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.548+0000 7f05ef16f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05e41afbc0 con 0x560bd5a99a00 2024-04-21T08:51:45.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.549+0000 7f05ea236640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a90e80 0x560bd5a99380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.549+0000 7f05ea236640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a90e80 0x560bd5a99380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51356/0 (socket says 172.21.15.43:51356) 2024-04-21T08:51:45.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.549+0000 7f05ea236640 1 -- 172.21.15.43:0/895957201 learned_addr learned my addr 172.21.15.43:0/895957201 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.549+0000 7f05e9a35640 1 --2- 172.21.15.43:0/895957201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bd5a904d0 0x560bd5a98e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.549+0000 7f05e9a35640 1 --2- 172.21.15.43:0/895957201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bd5a99a00 0x560bd5ab1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.549+0000 7f05ea236640 1 -- 172.21.15.43:0/895957201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bd5a99a00 msgr2=0x560bd5ab1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.549+0000 7f05ea236640 1 --2- 172.21.15.43:0/895957201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bd5a99a00 0x560bd5ab1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.549+0000 7f05ea236640 1 -- 172.21.15.43:0/895957201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bd5a904d0 msgr2=0x560bd5a98e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.549+0000 7f05ea236640 1 --2- 172.21.15.43:0/895957201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bd5a904d0 0x560bd5a98e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.549+0000 7f05ea236640 1 -- 172.21.15.43:0/895957201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bd5a97090 con 0x560bd5a90e80 2024-04-21T08:51:45.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.549+0000 7f05ea236640 1 --2- 172.21.15.43:0/895957201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a90e80 0x560bd5a99380 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f05e0006010 tx=0x7f05e00042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75bfc744ac69a9f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.549+0000 7f05cffff640 1 -- 172.21.15.43:0/895957201 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e00270f0 con 0x560bd5a90e80 2024-04-21T08:51:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.549+0000 7f05cffff640 1 -- 172.21.15.43:0/895957201 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e0015030 con 0x560bd5a90e80 2024-04-21T08:51:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.549+0000 7f05cffff640 1 -- 172.21.15.43:0/895957201 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0011820 con 0x560bd5a90e80 2024-04-21T08:51:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.550+0000 7f05ef16f740 1 -- 172.21.15.43:0/895957201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a90e80 msgr2=0x560bd5a99380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.550+0000 7f05ef16f740 1 --2- 172.21.15.43:0/895957201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a90e80 0x560bd5a99380 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f05e0006010 tx=0x7f05e00042b0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.550+0000 7f05ef16f740 1 -- 172.21.15.43:0/895957201 shutdown_connections 2024-04-21T08:51:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.550+0000 7f05ef16f740 1 --2- 172.21.15.43:0/895957201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bd5a99a00 0x560bd5ab1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.550+0000 7f05ef16f740 1 --2- 172.21.15.43:0/895957201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a90e80 0x560bd5a99380 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.550+0000 7f05ef16f740 1 --2- 172.21.15.43:0/895957201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bd5a904d0 0x560bd5a98e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.550+0000 7f05ef16f740 1 -- 172.21.15.43:0/895957201 >> 172.21.15.43:0/895957201 conn(0x560bd5a8aba0 msgr2=0x560bd5a91b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.550+0000 7f05ef16f740 1 -- 172.21.15.43:0/895957201 shutdown_connections 2024-04-21T08:51:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.550+0000 7f05ef16f740 1 -- 172.21.15.43:0/895957201 wait complete. 2024-04-21T08:51:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.550+0000 7f05ef16f740 1 -- start start 2024-04-21T08:51:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.550+0000 7f05ef16f740 1 Processor -- start 2024-04-21T08:51:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.550+0000 7f05ef16f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bd5a904d0 0x560bd5aa1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.550+0000 7f05ef16f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bd5a90e80 0x560bd5aa21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.550+0000 7f05ef16f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a99a00 0x560bd5a9d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.550+0000 7f05ef16f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f05e41b0960 con 0x560bd5a99a00 2024-04-21T08:51:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.550+0000 7f05ef16f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f05e41b0290 con 0x560bd5a90e80 2024-04-21T08:51:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.550+0000 7f05ef16f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05e41ad4f0 con 0x560bd5a904d0 2024-04-21T08:51:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.550+0000 7f05ef16f740 10 client.0 _ll_register_callbacks cb 0x7f05d4004530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.551+0000 7f05e9a35640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a99a00 0x560bd5a9d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.551+0000 7f05e9a35640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a99a00 0x560bd5a9d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51368/0 (socket says 172.21.15.43:51368) 2024-04-21T08:51:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.551+0000 7f05e9a35640 1 -- 172.21.15.43:0/4248822014 learned_addr learned my addr 172.21.15.43:0/4248822014 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.551+0000 7f05e9a35640 1 --2- 172.21.15.43:0/4248822014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bd5a904d0 0x560bd5aa1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.551+0000 7f05ea236640 1 --2- 172.21.15.43:0/4248822014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bd5a90e80 0x560bd5aa21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.551+0000 7f05e9a35640 1 -- 172.21.15.43:0/4248822014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bd5a904d0 msgr2=0x560bd5aa1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.551+0000 7f05e9a35640 1 --2- 172.21.15.43:0/4248822014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bd5a904d0 0x560bd5aa1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.551+0000 7f05e9a35640 1 -- 172.21.15.43:0/4248822014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bd5a90e80 msgr2=0x560bd5aa21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.551+0000 7f05e9a35640 1 --2- 172.21.15.43:0/4248822014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bd5a90e80 0x560bd5aa21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.551+0000 7f05e9a35640 1 -- 172.21.15.43:0/4248822014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bd5a90c20 con 0x560bd5a99a00 2024-04-21T08:51:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.551+0000 7f05e9a35640 1 --2- 172.21.15.43:0/4248822014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a99a00 0x560bd5a9d150 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f05d402c6d0 tx=0x7f05d4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3271038669798f6b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.551+0000 7f05cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.551+0000 7f05cdffb640 1 -- 172.21.15.43:0/4248822014 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4047040 con 0x560bd5a99a00 2024-04-21T08:51:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.551+0000 7f05cdffb640 1 -- 172.21.15.43:0/4248822014 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f05d40048c0 con 0x560bd5a99a00 2024-04-21T08:51:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.551+0000 7f05ef16f740 1 -- 172.21.15.43:0/4248822014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05e0015030 con 0x560bd5a99a00 2024-04-21T08:51:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.551+0000 7f05cdffb640 1 -- 172.21.15.43:0/4248822014 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f05d4003e60 con 0x560bd5a99a00 2024-04-21T08:51:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.551+0000 7f05ef16f740 10 client.21498 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.551+0000 7f05ef16f740 1 -- 172.21.15.43:0/4248822014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560bd5b0fb10 con 0x560bd5a99a00 2024-04-21T08:51:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.551+0000 7f05cdffb640 1 -- 172.21.15.43:0/4248822014 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e0015030 con 0x560bd5a99a00 2024-04-21T08:51:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.552+0000 7f05cdffb640 1 -- 172.21.15.43:0/4248822014 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f05d4060030 con 0x560bd5a99a00 2024-04-21T08:51:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.552+0000 7f05ef16f740 10 client.21498 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.552+0000 7f05ef16f740 10 client.21498 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:51:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.552+0000 7f05ef16f740 20 client.21498 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.552+0000 7f05ef16f740 1 --2- 172.21.15.43:0/4248822014 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x560bd5b101d0 0x560bd5b125c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.552+0000 7f05ef16f740 4 client.21498 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:51:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.552+0000 7f05ef16f740 1 -- 172.21.15.43:0/4248822014 --> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560bd5a02c10 con 0x560bd5b101d0 2024-04-21T08:51:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.552+0000 7f05ea236640 1 --2- 172.21.15.43:0/4248822014 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x560bd5b101d0 0x560bd5b125c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.552+0000 7f05ea236640 1 --2- 172.21.15.43:0/4248822014 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x560bd5b101d0 0x560bd5b125c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=24fbd4b28ea37c55 server_cookie=1c778013e115a0a2 in_seq=0 out_seq=0 2024-04-21T08:51:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.552+0000 7f05cdffb640 10 client.21498 ms_handle_connect on v2:172.21.15.112:6832/1516781885 2024-04-21T08:51:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05cdffb640 1 -- 172.21.15.43:0/4248822014 <== mds.3 v2:172.21.15.112:6832/1516781885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560bd5a02c10 con 0x560bd5b101d0 2024-04-21T08:51:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05cdffb640 10 client.21498 handle_command_reply: tid=0 2024-04-21T08:51:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05ef16f740 1 client.21498 shutdown 2024-04-21T08:51:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05ef16f740 1 -- 172.21.15.43:0/4248822014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a99a00 msgr2=0x560bd5a9d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05ef16f740 1 --2- 172.21.15.43:0/4248822014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a99a00 0x560bd5a9d150 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f05d402c6d0 tx=0x7f05d4009260 comp rx=0 tx=0).stop 2024-04-21T08:51:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05ef16f740 1 -- 172.21.15.43:0/4248822014 shutdown_connections 2024-04-21T08:51:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05ef16f740 1 --2- 172.21.15.43:0/4248822014 >> [v2:172.21.15.112:6832/1516781885,v1:172.21.15.112:6833/1516781885] conn(0x560bd5b101d0 0x560bd5b125c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05ef16f740 1 --2- 172.21.15.43:0/4248822014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560bd5a99a00 0x560bd5a9d150 unknown :-1 s=CLOSED pgs=2390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05ef16f740 1 --2- 172.21.15.43:0/4248822014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560bd5a90e80 0x560bd5aa21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05cdffb640 0 client.21498 ms_handle_reset on v2:172.21.15.112:6832/1516781885 2024-04-21T08:51:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05ef16f740 1 --2- 172.21.15.43:0/4248822014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560bd5a904d0 0x560bd5aa1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05ef16f740 1 -- 172.21.15.43:0/4248822014 >> 172.21.15.43:0/4248822014 conn(0x560bd5a8aba0 msgr2=0x560bd5a8bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05ef16f740 1 -- 172.21.15.43:0/4248822014 shutdown_connections 2024-04-21T08:51:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05ef16f740 1 -- 172.21.15.43:0/4248822014 wait complete. 2024-04-21T08:51:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05ef16f740 20 client.21498 trim_cache size 0 max 16384 2024-04-21T08:51:45.559 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:51:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05ec49a640 1 -- 172.21.15.43:0/64687500 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f05d00a3d20 msgr2=0x7f05d00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05ec49a640 1 --2- 172.21.15.43:0/64687500 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f05d00a3d20 0x7f05d00a61e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f05dc0ce240 tx=0x7f05dc091e70 comp rx=0 tx=0).stop 2024-04-21T08:51:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05ec49a640 1 -- 172.21.15.43:0/64687500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05e4198630 msgr2=0x7f05e4198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05ec49a640 1 --2- 172.21.15.43:0/64687500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05e4198630 0x7f05e4198ae0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00cab0 tx=0x7f05dc00cf70 comp rx=0 tx=0).stop 2024-04-21T08:51:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05ec49a640 1 -- 172.21.15.43:0/64687500 shutdown_connections 2024-04-21T08:51:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05ec49a640 1 --2- 172.21.15.43:0/64687500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05e4198630 0x7f05e4198ae0 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05ec49a640 1 --2- 172.21.15.43:0/64687500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05e4106170 0x7f05e419e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05ec49a640 1 --2- 172.21.15.43:0/64687500 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f05d00a3d20 0x7f05d00a61e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05ec49a640 1 --2- 172.21.15.43:0/64687500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05e41010f0 0x7f05e419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05ec49a640 1 -- 172.21.15.43:0/64687500 >> 172.21.15.43:0/64687500 conn(0x7f05e4077fb0 msgr2=0x7f05e40ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.553+0000 7f05ec49a640 1 -- 172.21.15.43:0/64687500 shutdown_connections 2024-04-21T08:51:45.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.554+0000 7f05ec49a640 1 -- 172.21.15.43:0/64687500 wait complete. 2024-04-21T08:51:45.564 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:51:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.670+0000 7fcee88f2640 1 -- 172.21.15.43:0/1134578135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee4106160 msgr2=0x7fcee410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.670+0000 7fcee88f2640 1 --2- 172.21.15.43:0/1134578135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee4106160 0x7fcee410dd30 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7fced4009ca0 tx=0x7fced402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.670+0000 7fcee88f2640 1 -- 172.21.15.43:0/1134578135 shutdown_connections 2024-04-21T08:51:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.670+0000 7fcee88f2640 1 --2- 172.21.15.43:0/1134578135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee4106160 0x7fcee410dd30 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.670+0000 7fcee88f2640 1 --2- 172.21.15.43:0/1134578135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcee4101a40 0x7fcee4105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.670+0000 7fcee88f2640 1 --2- 172.21.15.43:0/1134578135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcee4101090 0x7fcee4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.670+0000 7fcee88f2640 1 -- 172.21.15.43:0/1134578135 >> 172.21.15.43:0/1134578135 conn(0x7fcee4077f50 msgr2=0x7fcee4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.670+0000 7fcee88f2640 1 -- 172.21.15.43:0/1134578135 shutdown_connections 2024-04-21T08:51:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.670+0000 7fcee88f2640 1 -- 172.21.15.43:0/1134578135 wait complete. 2024-04-21T08:51:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.671+0000 7fcee88f2640 1 Processor -- start 2024-04-21T08:51:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.671+0000 7fcee88f2640 1 -- start start 2024-04-21T08:51:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.671+0000 7fcee88f2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee4101090 0x7fcee4198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.671+0000 7fcee88f2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcee4106160 0x7fcee4198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.671+0000 7fcee88f2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcee419cee0 0x7fcee419d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.671+0000 7fcee88f2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcee41121e0 con 0x7fcee4101090 2024-04-21T08:51:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.671+0000 7fcee88f2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcee4112060 con 0x7fcee4106160 2024-04-21T08:51:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.671+0000 7fcee88f2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcee4112360 con 0x7fcee419cee0 2024-04-21T08:51:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.671+0000 7fcee259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee4101090 0x7fcee4198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.671+0000 7fcee259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee4101090 0x7fcee4198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51394/0 (socket says 172.21.15.43:51394) 2024-04-21T08:51:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.671+0000 7fcee259c640 1 -- 172.21.15.43:0/2115913364 learned_addr learned my addr 172.21.15.43:0/2115913364 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.671+0000 7fcee1d9b640 1 --2- 172.21.15.43:0/2115913364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcee4106160 0x7fcee4198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.671+0000 7fcee259c640 1 -- 172.21.15.43:0/2115913364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcee419cee0 msgr2=0x7fcee419d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.671+0000 7fcee259c640 1 --2- 172.21.15.43:0/2115913364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcee419cee0 0x7fcee419d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.671+0000 7fcee259c640 1 -- 172.21.15.43:0/2115913364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcee4106160 msgr2=0x7fcee4198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.671+0000 7fcee259c640 1 --2- 172.21.15.43:0/2115913364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcee4106160 0x7fcee4198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.671+0000 7fcee259c640 1 -- 172.21.15.43:0/2115913364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee419d9d0 con 0x7fcee4101090 2024-04-21T08:51:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.671+0000 7fcee1d9b640 1 --2- 172.21.15.43:0/2115913364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcee4106160 0x7fcee4198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.671+0000 7fcee259c640 1 --2- 172.21.15.43:0/2115913364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee4101090 0x7fcee4198620 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00c890 tx=0x7fcedc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.672+0000 7fcedb7fe640 1 -- 172.21.15.43:0/2115913364 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcedc004220 con 0x7fcee4101090 2024-04-21T08:51:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.672+0000 7fcee88f2640 1 -- 172.21.15.43:0/2115913364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee419dcc0 con 0x7fcee4101090 2024-04-21T08:51:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.672+0000 7fcedb7fe640 1 -- 172.21.15.43:0/2115913364 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcedc0043c0 con 0x7fcee4101090 2024-04-21T08:51:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.672+0000 7fcedb7fe640 1 -- 172.21.15.43:0/2115913364 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcedc004a00 con 0x7fcee4101090 2024-04-21T08:51:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.672+0000 7fcee88f2640 1 -- 172.21.15.43:0/2115913364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee41a54b0 con 0x7fcee4101090 2024-04-21T08:51:45.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.674+0000 7fcedb7fe640 1 -- 172.21.15.43:0/2115913364 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcedc0079b0 con 0x7fcee4101090 2024-04-21T08:51:45.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.674+0000 7fceeb5c1740 1 Processor -- start 2024-04-21T08:51:45.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.674+0000 7fceeb5c1740 1 -- start start 2024-04-21T08:51:45.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.674+0000 7fceeb5c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4330 0x555ed0da4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.674+0000 7fceeb5c1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ed0da4ce0 0x555ed0dad820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.674+0000 7fceeb5c1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ed0dade40 0x555ed0dae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.674+0000 7fceeb5c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555ed0d091f0 con 0x555ed0da4330 2024-04-21T08:51:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.674+0000 7fceeb5c1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555ed0ca64b0 con 0x555ed0da4ce0 2024-04-21T08:51:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.674+0000 7fceeb5c1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555ed0d051b0 con 0x555ed0dade40 2024-04-21T08:51:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.675+0000 7fcee2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4330 0x555ed0da4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.675+0000 7fcee2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4330 0x555ed0da4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51404/0 (socket says 172.21.15.43:51404) 2024-04-21T08:51:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.675+0000 7fcee2d9d640 1 -- 172.21.15.43:0/1682461041 learned_addr learned my addr 172.21.15.43:0/1682461041 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.675+0000 7fcedb7fe640 1 --2- 172.21.15.43:0/2115913364 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fced00a3d20 0x7fced00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.675+0000 7fcee2d9d640 1 --2- 172.21.15.43:0/2115913364 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fced00a3d20 0x7fced00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.675+0000 7fcee1d9b640 1 --2- 172.21.15.43:0/1682461041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ed0dade40 0x555ed0dae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.675+0000 7fcedb7fe640 1 -- 172.21.15.43:0/2115913364 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fcedc0d0a20 con 0x7fcee4101090 2024-04-21T08:51:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.675+0000 7fcee259c640 1 --2- 172.21.15.43:0/1682461041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ed0da4ce0 0x555ed0dad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.675+0000 7fcee2d9d640 1 -- 172.21.15.43:0/1682461041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ed0dade40 msgr2=0x555ed0dae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.675+0000 7fcee2d9d640 1 --2- 172.21.15.43:0/1682461041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ed0dade40 0x555ed0dae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.675+0000 7fcee2d9d640 1 -- 172.21.15.43:0/1682461041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ed0da4ce0 msgr2=0x555ed0dad820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.675+0000 7fcee2d9d640 1 --2- 172.21.15.43:0/1682461041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ed0da4ce0 0x555ed0dad820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.675+0000 7fcee2d9d640 1 -- 172.21.15.43:0/1682461041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ed0c981b0 con 0x555ed0da4330 2024-04-21T08:51:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.675+0000 7fcee2d9d640 1 --2- 172.21.15.43:0/1682461041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4330 0x555ed0da4710 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7fced4009dc0 tx=0x7fced4032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed69c542408790bd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.675+0000 7fced8ff9640 1 -- 172.21.15.43:0/1682461041 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced4032840 con 0x555ed0da4330 2024-04-21T08:51:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.675+0000 7fced8ff9640 1 -- 172.21.15.43:0/1682461041 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fced403a070 con 0x555ed0da4330 2024-04-21T08:51:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.675+0000 7fced8ff9640 1 -- 172.21.15.43:0/1682461041 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced4039070 con 0x555ed0da4330 2024-04-21T08:51:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.676+0000 7fceeb5c1740 1 -- 172.21.15.43:0/1682461041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4330 msgr2=0x555ed0da4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.676+0000 7fcee2d9d640 1 --2- 172.21.15.43:0/2115913364 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fced00a3d20 0x7fced00a61e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fced404a000 tx=0x7fced4048000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.676+0000 7fceeb5c1740 1 --2- 172.21.15.43:0/1682461041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4330 0x555ed0da4710 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7fced4009dc0 tx=0x7fced4032fc0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.676+0000 7fceeb5c1740 1 -- 172.21.15.43:0/1682461041 shutdown_connections 2024-04-21T08:51:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.676+0000 7fceeb5c1740 1 --2- 172.21.15.43:0/1682461041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ed0dade40 0x555ed0dae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.676+0000 7fceeb5c1740 1 --2- 172.21.15.43:0/1682461041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ed0da4ce0 0x555ed0dad820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.676+0000 7fceeb5c1740 1 --2- 172.21.15.43:0/1682461041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4330 0x555ed0da4710 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.676+0000 7fceeb5c1740 1 -- 172.21.15.43:0/1682461041 >> 172.21.15.43:0/1682461041 conn(0x555ed0d9eba0 msgr2=0x555ed0d9ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.676+0000 7fceeb5c1740 1 -- 172.21.15.43:0/1682461041 shutdown_connections 2024-04-21T08:51:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.676+0000 7fceeb5c1740 1 -- 172.21.15.43:0/1682461041 wait complete. 2024-04-21T08:51:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.677+0000 7fceeb5c1740 1 -- start start 2024-04-21T08:51:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.677+0000 7fceeb5c1740 1 Processor -- start 2024-04-21T08:51:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.677+0000 7fceeb5c1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ed0da4330 0x555ed0db4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.677+0000 7fceeb5c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4ce0 0x555ed0db1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.677+0000 7fceeb5c1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ed0dade40 0x555ed0db1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.677+0000 7fceeb5c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555ed0db4dc0 con 0x555ed0da4ce0 2024-04-21T08:51:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.677+0000 7fceeb5c1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555ed0cc2500 con 0x555ed0da4330 2024-04-21T08:51:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.677+0000 7fceeb5c1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555ed0db4f40 con 0x555ed0dade40 2024-04-21T08:51:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.677+0000 7fceeb5c1740 10 client.0 _ll_register_callbacks cb 0x555ed0c90b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.677+0000 7fcee1d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4ce0 0x555ed0db1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.677+0000 7fcee1d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4ce0 0x555ed0db1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51416/0 (socket says 172.21.15.43:51416) 2024-04-21T08:51:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.677+0000 7fcee1d9b640 1 -- 172.21.15.43:0/3647672866 learned_addr learned my addr 172.21.15.43:0/3647672866 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.677+0000 7fcee2d9d640 1 --2- 172.21.15.43:0/3647672866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ed0dade40 0x555ed0db1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.677+0000 7fcee259c640 1 --2- 172.21.15.43:0/3647672866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ed0da4330 0x555ed0db4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.677+0000 7fcee1d9b640 1 -- 172.21.15.43:0/3647672866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ed0dade40 msgr2=0x555ed0db1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.677+0000 7fcee1d9b640 1 --2- 172.21.15.43:0/3647672866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ed0dade40 0x555ed0db1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.677+0000 7fcee1d9b640 1 -- 172.21.15.43:0/3647672866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ed0da4330 msgr2=0x555ed0db4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.677+0000 7fcee1d9b640 1 --2- 172.21.15.43:0/3647672866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ed0da4330 0x555ed0db4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.677+0000 7fcee1d9b640 1 -- 172.21.15.43:0/3647672866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ed0a1b3f0 con 0x555ed0da4ce0 2024-04-21T08:51:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.677+0000 7fcee259c640 1 --2- 172.21.15.43:0/3647672866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ed0da4330 0x555ed0db4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.678+0000 7fcee1d9b640 1 --2- 172.21.15.43:0/3647672866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4ce0 0x555ed0db1620 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fcecc010770 tx=0x7fcecc010c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6347d933f6ec131d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.678+0000 7fcec9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.678+0000 7fcec9ffb640 1 -- 172.21.15.43:0/3647672866 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc00ece0 con 0x555ed0da4ce0 2024-04-21T08:51:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.678+0000 7fceeb5c1740 1 -- 172.21.15.43:0/3647672866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fced403a070 con 0x555ed0da4ce0 2024-04-21T08:51:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.678+0000 7fcec9ffb640 1 -- 172.21.15.43:0/3647672866 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fcecc004020 con 0x555ed0da4ce0 2024-04-21T08:51:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.678+0000 7fcec9ffb640 1 -- 172.21.15.43:0/3647672866 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcecc023050 con 0x555ed0da4ce0 2024-04-21T08:51:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.678+0000 7fceeb5c1740 10 client.21510 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.678+0000 7fceeb5c1740 1 -- 172.21.15.43:0/3647672866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555ed0ae57d0 con 0x555ed0da4ce0 2024-04-21T08:51:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.678+0000 7fcec9ffb640 1 -- 172.21.15.43:0/3647672866 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fced403a070 con 0x555ed0da4ce0 2024-04-21T08:51:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.678+0000 7fcec9ffb640 1 -- 172.21.15.43:0/3647672866 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcecc0047a0 con 0x555ed0da4ce0 2024-04-21T08:51:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.678+0000 7fceeb5c1740 10 client.21510 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.678+0000 7fceeb5c1740 10 client.21510 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:51:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.678+0000 7fceeb5c1740 20 client.21510 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.678+0000 7fceeb5c1740 1 --2- 172.21.15.43:0/3647672866 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x555ed0dc1e20 0x555ed0dc4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.678+0000 7fceeb5c1740 4 client.21510 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.678+0000 7fceeb5c1740 1 -- 172.21.15.43:0/3647672866 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ed0ca8930 con 0x555ed0dc1e20 2024-04-21T08:51:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.679+0000 7fcee259c640 1 --2- 172.21.15.43:0/3647672866 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x555ed0dc1e20 0x555ed0dc4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.679+0000 7fcee259c640 1 --2- 172.21.15.43:0/3647672866 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x555ed0dc1e20 0x555ed0dc4250 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=275fad5cd75f1dec server_cookie=6d441630ed0b357d in_seq=0 out_seq=0 2024-04-21T08:51:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.679+0000 7fcec9ffb640 10 client.21510 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:51:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.680+0000 7fcec9ffb640 1 -- 172.21.15.43:0/3647672866 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555ed0ca8930 con 0x555ed0dc1e20 2024-04-21T08:51:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.680+0000 7fcec9ffb640 10 client.21510 handle_command_reply: tid=0 2024-04-21T08:51:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.680+0000 7fceeb5c1740 1 client.21510 shutdown 2024-04-21T08:51:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.680+0000 7fceeb5c1740 1 -- 172.21.15.43:0/3647672866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4ce0 msgr2=0x555ed0db1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.680+0000 7fceeb5c1740 1 --2- 172.21.15.43:0/3647672866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4ce0 0x555ed0db1620 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fcecc010770 tx=0x7fcecc010c30 comp rx=0 tx=0).stop 2024-04-21T08:51:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.680+0000 7fceeb5c1740 1 -- 172.21.15.43:0/3647672866 shutdown_connections 2024-04-21T08:51:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.680+0000 7fceeb5c1740 1 --2- 172.21.15.43:0/3647672866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ed0dade40 0x555ed0db1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.680+0000 7fceeb5c1740 1 --2- 172.21.15.43:0/3647672866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4ce0 0x555ed0db1620 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.680+0000 7fceeb5c1740 1 --2- 172.21.15.43:0/3647672866 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x555ed0dc1e20 0x555ed0dc4250 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.680+0000 7fceeb5c1740 1 --2- 172.21.15.43:0/3647672866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ed0da4330 0x555ed0db4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.680+0000 7fceeb5c1740 1 -- 172.21.15.43:0/3647672866 >> 172.21.15.43:0/3647672866 conn(0x555ed0d9eba0 msgr2=0x555ed0da08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.680+0000 7fcec9ffb640 0 client.21510 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:51:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.680+0000 7fceeb5c1740 1 -- 172.21.15.43:0/3647672866 shutdown_connections 2024-04-21T08:51:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.680+0000 7fceeb5c1740 1 -- 172.21.15.43:0/3647672866 wait complete. 2024-04-21T08:51:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.680+0000 7fceeb5c1740 20 client.21510 trim_cache size 0 max 16384 2024-04-21T08:51:45.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.694+0000 7fceeb5c1740 1 Processor -- start 2024-04-21T08:51:45.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.694+0000 7fceeb5c1740 1 -- start start 2024-04-21T08:51:45.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.694+0000 7fceeb5c1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ed0da4330 0x555ed0dbdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.694+0000 7fceeb5c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4ce0 0x555ed0daf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.694+0000 7fceeb5c1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ed0dade40 0x555ed0db0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.694+0000 7fceeb5c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcee41ad280 con 0x555ed0da4ce0 2024-04-21T08:51:45.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.694+0000 7fceeb5c1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcee41aedc0 con 0x555ed0da4330 2024-04-21T08:51:45.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.694+0000 7fceeb5c1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcee41ad950 con 0x555ed0dade40 2024-04-21T08:51:45.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.694+0000 7fcee1d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4ce0 0x555ed0daf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.694+0000 7fcee1d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4ce0 0x555ed0daf230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51424/0 (socket says 172.21.15.43:51424) 2024-04-21T08:51:45.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.694+0000 7fcee1d9b640 1 -- 172.21.15.43:0/4099963546 learned_addr learned my addr 172.21.15.43:0/4099963546 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.694+0000 7fcee1d9b640 1 -- 172.21.15.43:0/4099963546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ed0dade40 msgr2=0x555ed0db0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.694+0000 7fcee1d9b640 1 --2- 172.21.15.43:0/4099963546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ed0dade40 0x555ed0db0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.694+0000 7fcee1d9b640 1 -- 172.21.15.43:0/4099963546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ed0da4330 msgr2=0x555ed0dbdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.694+0000 7fcee1d9b640 1 --2- 172.21.15.43:0/4099963546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ed0da4330 0x555ed0dbdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.694+0000 7fcee1d9b640 1 -- 172.21.15.43:0/4099963546 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ed0db4ca0 con 0x555ed0da4ce0 2024-04-21T08:51:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.694+0000 7fcee1d9b640 1 --2- 172.21.15.43:0/4099963546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4ce0 0x555ed0daf230 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fcecc004b90 tx=0x7fcecc010e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221e2426c936ccec server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.694+0000 7fcecbfff640 1 -- 172.21.15.43:0/4099963546 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc004840 con 0x555ed0da4ce0 2024-04-21T08:51:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.694+0000 7fcecbfff640 1 -- 172.21.15.43:0/4099963546 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcecc02c070 con 0x555ed0da4ce0 2024-04-21T08:51:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.694+0000 7fcecbfff640 1 -- 172.21.15.43:0/4099963546 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc031470 con 0x555ed0da4ce0 2024-04-21T08:51:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.695+0000 7fceeb5c1740 1 -- 172.21.15.43:0/4099963546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4ce0 msgr2=0x555ed0daf230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.695+0000 7fceeb5c1740 1 --2- 172.21.15.43:0/4099963546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4ce0 0x555ed0daf230 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fcecc004b90 tx=0x7fcecc010e80 comp rx=0 tx=0).stop 2024-04-21T08:51:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.695+0000 7fceeb5c1740 1 -- 172.21.15.43:0/4099963546 shutdown_connections 2024-04-21T08:51:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.695+0000 7fceeb5c1740 1 --2- 172.21.15.43:0/4099963546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ed0dade40 0x555ed0db0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.695+0000 7fceeb5c1740 1 --2- 172.21.15.43:0/4099963546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4ce0 0x555ed0daf230 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.695+0000 7fceeb5c1740 1 --2- 172.21.15.43:0/4099963546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ed0da4330 0x555ed0dbdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.695+0000 7fceeb5c1740 1 -- 172.21.15.43:0/4099963546 >> 172.21.15.43:0/4099963546 conn(0x555ed0d9eba0 msgr2=0x555ed0d9d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.695+0000 7fceeb5c1740 1 -- 172.21.15.43:0/4099963546 shutdown_connections 2024-04-21T08:51:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.695+0000 7fceeb5c1740 1 -- 172.21.15.43:0/4099963546 wait complete. 2024-04-21T08:51:45.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.695+0000 7fceeb5c1740 1 -- start start 2024-04-21T08:51:45.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.695+0000 7fceeb5c1740 1 Processor -- start 2024-04-21T08:51:45.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fceeb5c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4330 0x555ed0dba390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fceeb5c1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ed0da4ce0 0x555ed0db2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fceeb5c1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ed0dade40 0x555ed0db31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fceeb5c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcee41ae6f0 con 0x555ed0da4330 2024-04-21T08:51:45.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fceeb5c1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcee41ae020 con 0x555ed0dade40 2024-04-21T08:51:45.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fceeb5c1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcee41ab280 con 0x555ed0da4ce0 2024-04-21T08:51:45.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fceeb5c1740 10 client.0 _ll_register_callbacks cb 0x7fcecc00cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:45.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fcee259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4330 0x555ed0dba390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fcee259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4330 0x555ed0dba390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51434/0 (socket says 172.21.15.43:51434) 2024-04-21T08:51:45.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fcee259c640 1 -- 172.21.15.43:0/2305400544 learned_addr learned my addr 172.21.15.43:0/2305400544 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fcee2d9d640 1 --2- 172.21.15.43:0/2305400544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ed0dade40 0x555ed0db31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fcee1d9b640 1 --2- 172.21.15.43:0/2305400544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ed0da4ce0 0x555ed0db2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fcee259c640 1 -- 172.21.15.43:0/2305400544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ed0da4ce0 msgr2=0x555ed0db2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fcee259c640 1 --2- 172.21.15.43:0/2305400544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ed0da4ce0 0x555ed0db2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fcee259c640 1 -- 172.21.15.43:0/2305400544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ed0dade40 msgr2=0x555ed0db31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fcee259c640 1 --2- 172.21.15.43:0/2305400544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ed0dade40 0x555ed0db31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fcee259c640 1 -- 172.21.15.43:0/2305400544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ed0db3700 con 0x555ed0da4330 2024-04-21T08:51:45.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fcee2d9d640 1 --2- 172.21.15.43:0/2305400544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ed0dade40 0x555ed0db31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:45.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fcee259c640 1 --2- 172.21.15.43:0/2305400544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4330 0x555ed0dba390 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7fcedc098f70 tx=0x7fcedc005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866488facb2a1d4e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fcec9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:45.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fcec9ffb640 1 -- 172.21.15.43:0/2305400544 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcedc01d040 con 0x555ed0da4330 2024-04-21T08:51:45.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fcec9ffb640 1 -- 172.21.15.43:0/2305400544 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fcedc0cdfd0 con 0x555ed0da4330 2024-04-21T08:51:45.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fceeb5c1740 1 -- 172.21.15.43:0/2305400544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcecc02c070 con 0x555ed0da4330 2024-04-21T08:51:45.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fcec9ffb640 1 -- 172.21.15.43:0/2305400544 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcedc0a5210 con 0x555ed0da4330 2024-04-21T08:51:45.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.696+0000 7fceeb5c1740 10 client.21516 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:45.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.697+0000 7fceeb5c1740 1 -- 172.21.15.43:0/2305400544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555ed0e28050 con 0x555ed0da4330 2024-04-21T08:51:45.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.697+0000 7fcec9ffb640 1 -- 172.21.15.43:0/2305400544 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcecc02c070 con 0x555ed0da4330 2024-04-21T08:51:45.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.697+0000 7fcec9ffb640 1 -- 172.21.15.43:0/2305400544 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcedc085b70 con 0x555ed0da4330 2024-04-21T08:51:45.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.697+0000 7fceeb5c1740 10 client.21516 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:45.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.697+0000 7fceeb5c1740 10 client.21516 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-21T08:51:45.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.697+0000 7fceeb5c1740 20 client.21516 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:45.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.697+0000 7fceeb5c1740 1 --2- 172.21.15.43:0/2305400544 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x555ed0e287a0 0x555ed0e2abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.697+0000 7fceeb5c1740 4 client.21516 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:51:45.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.697+0000 7fceeb5c1740 1 -- 172.21.15.43:0/2305400544 --> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ed0d16c10 con 0x555ed0e287a0 2024-04-21T08:51:45.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.697+0000 7fcee1d9b640 1 --2- 172.21.15.43:0/2305400544 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x555ed0e287a0 0x555ed0e2abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.698+0000 7fcec9ffb640 10 client.21516 ms_handle_connect on v2:172.21.15.175:6838/3664054463 2024-04-21T08:51:45.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.698+0000 7fcee1d9b640 1 --2- 172.21.15.43:0/2305400544 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x555ed0e287a0 0x555ed0e2abc0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eddb3fbbf3016676 server_cookie=9f01c72e5f72cd96 in_seq=0 out_seq=0 2024-04-21T08:51:45.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.698+0000 7fcec9ffb640 1 -- 172.21.15.43:0/2305400544 <== mds.2 v2:172.21.15.175:6838/3664054463 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ed0d16c10 con 0x555ed0e287a0 2024-04-21T08:51:45.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.698+0000 7fcec9ffb640 10 client.21516 handle_command_reply: tid=0 2024-04-21T08:51:45.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.698+0000 7fceeb5c1740 1 client.21516 shutdown 2024-04-21T08:51:45.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.698+0000 7fceeb5c1740 1 -- 172.21.15.43:0/2305400544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4330 msgr2=0x555ed0dba390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.698+0000 7fceeb5c1740 1 --2- 172.21.15.43:0/2305400544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4330 0x555ed0dba390 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7fcedc098f70 tx=0x7fcedc005760 comp rx=0 tx=0).stop 2024-04-21T08:51:45.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.698+0000 7fceeb5c1740 1 -- 172.21.15.43:0/2305400544 shutdown_connections 2024-04-21T08:51:45.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.698+0000 7fceeb5c1740 1 --2- 172.21.15.43:0/2305400544 >> [v2:172.21.15.175:6838/3664054463,v1:172.21.15.175:6839/3664054463] conn(0x555ed0e287a0 0x555ed0e2abc0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.698+0000 7fceeb5c1740 1 --2- 172.21.15.43:0/2305400544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ed0dade40 0x555ed0db31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.698+0000 7fcec9ffb640 0 client.21516 ms_handle_reset on v2:172.21.15.175:6838/3664054463 2024-04-21T08:51:45.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.698+0000 7fceeb5c1740 1 --2- 172.21.15.43:0/2305400544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555ed0da4ce0 0x555ed0db2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.698+0000 7fceeb5c1740 1 --2- 172.21.15.43:0/2305400544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ed0da4330 0x555ed0dba390 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.698+0000 7fceeb5c1740 1 -- 172.21.15.43:0/2305400544 >> 172.21.15.43:0/2305400544 conn(0x555ed0d9eba0 msgr2=0x555ed0d9d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.698+0000 7fceeb5c1740 1 -- 172.21.15.43:0/2305400544 shutdown_connections 2024-04-21T08:51:45.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.698+0000 7fceeb5c1740 1 -- 172.21.15.43:0/2305400544 wait complete. 2024-04-21T08:51:45.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.698+0000 7fceeb5c1740 20 client.21516 trim_cache size 0 max 16384 2024-04-21T08:51:45.705 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:51:45.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.698+0000 7fcee88f2640 1 -- 172.21.15.43:0/2115913364 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fced00a3d20 msgr2=0x7fced00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.698+0000 7fcee88f2640 1 --2- 172.21.15.43:0/2115913364 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fced00a3d20 0x7fced00a61e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fced404a000 tx=0x7fced4048000 comp rx=0 tx=0).stop 2024-04-21T08:51:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.698+0000 7fcee88f2640 1 -- 172.21.15.43:0/2115913364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee4101090 msgr2=0x7fcee4198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.698+0000 7fcee88f2640 1 --2- 172.21.15.43:0/2115913364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee4101090 0x7fcee4198620 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00c890 tx=0x7fcedc00cd50 comp rx=0 tx=0).stop 2024-04-21T08:51:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.699+0000 7fcee88f2640 1 -- 172.21.15.43:0/2115913364 shutdown_connections 2024-04-21T08:51:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.699+0000 7fcee88f2640 1 --2- 172.21.15.43:0/2115913364 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fced00a3d20 0x7fced00a61e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.699+0000 7fcee88f2640 1 --2- 172.21.15.43:0/2115913364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcee419cee0 0x7fcee419d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.699+0000 7fcee88f2640 1 --2- 172.21.15.43:0/2115913364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcee4106160 0x7fcee4198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.699+0000 7fcee88f2640 1 --2- 172.21.15.43:0/2115913364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee4101090 0x7fcee4198620 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.699+0000 7fcee88f2640 1 -- 172.21.15.43:0/2115913364 >> 172.21.15.43:0/2115913364 conn(0x7fcee4077f50 msgr2=0x7fcee40fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.699+0000 7fcee88f2640 1 -- 172.21.15.43:0/2115913364 shutdown_connections 2024-04-21T08:51:45.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.699+0000 7fcee88f2640 1 -- 172.21.15.43:0/2115913364 wait complete. 2024-04-21T08:51:45.709 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:51:45.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.814+0000 7fc31e093640 1 -- 172.21.15.43:0/1374502010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc318102190 msgr2=0x7fc318102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.814+0000 7fc31e093640 1 --2- 172.21.15.43:0/1374502010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc318102190 0x7fc318102570 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7fc308009900 tx=0x7fc30802bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:45.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.814+0000 7fc31e093640 1 -- 172.21.15.43:0/1374502010 shutdown_connections 2024-04-21T08:51:45.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.814+0000 7fc31e093640 1 --2- 172.21.15.43:0/1374502010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc318107260 0x7fc31810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.814+0000 7fc31e093640 1 --2- 172.21.15.43:0/1374502010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc318102b40 0x7fc318106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.814+0000 7fc31e093640 1 --2- 172.21.15.43:0/1374502010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc318102190 0x7fc318102570 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.814+0000 7fc31e093640 1 -- 172.21.15.43:0/1374502010 >> 172.21.15.43:0/1374502010 conn(0x7fc3180fd990 msgr2=0x7fc3180ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.814+0000 7fc31e093640 1 -- 172.21.15.43:0/1374502010 shutdown_connections 2024-04-21T08:51:45.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.814+0000 7fc31e093640 1 -- 172.21.15.43:0/1374502010 wait complete. 2024-04-21T08:51:45.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.814+0000 7fc31e093640 1 Processor -- start 2024-04-21T08:51:45.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.815+0000 7fc31e093640 1 -- start start 2024-04-21T08:51:45.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.815+0000 7fc31e093640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc318102190 0x7fc31819e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.815+0000 7fc31e093640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc318107260 0x7fc31819e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.815+0000 7fc31e093640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc318198610 0x7fc318198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.815+0000 7fc31e093640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc3181121d0 con 0x7fc318198610 2024-04-21T08:51:45.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.815+0000 7fc31e093640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc318112050 con 0x7fc318102190 2024-04-21T08:51:45.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.815+0000 7fc31e093640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc318112350 con 0x7fc318107260 2024-04-21T08:51:45.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.815+0000 7fc317fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc318198610 0x7fc318198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.815+0000 7fc317fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc318198610 0x7fc318198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51456/0 (socket says 172.21.15.43:51456) 2024-04-21T08:51:45.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.815+0000 7fc317fff640 1 -- 172.21.15.43:0/3891481411 learned_addr learned my addr 172.21.15.43:0/3891481411 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.815+0000 7fc316ffd640 1 --2- 172.21.15.43:0/3891481411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc318107260 0x7fc31819e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.815+0000 7fc3177fe640 1 --2- 172.21.15.43:0/3891481411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc318102190 0x7fc31819e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.815+0000 7fc317fff640 1 -- 172.21.15.43:0/3891481411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc318107260 msgr2=0x7fc31819e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.815+0000 7fc317fff640 1 --2- 172.21.15.43:0/3891481411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc318107260 0x7fc31819e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.815+0000 7fc317fff640 1 -- 172.21.15.43:0/3891481411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc318102190 msgr2=0x7fc31819e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.815+0000 7fc317fff640 1 --2- 172.21.15.43:0/3891481411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc318102190 0x7fc31819e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.815+0000 7fc317fff640 1 -- 172.21.15.43:0/3891481411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc318199130 con 0x7fc318198610 2024-04-21T08:51:45.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.815+0000 7fc3177fe640 1 --2- 172.21.15.43:0/3891481411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc318102190 0x7fc31819e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:45.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.816+0000 7fc317fff640 1 --2- 172.21.15.43:0/3891481411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc318198610 0x7fc318198ac0 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7fc31000cad0 tx=0x7fc31000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.816+0000 7fc314ff9640 1 -- 172.21.15.43:0/3891481411 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31000ae20 con 0x7fc318198610 2024-04-21T08:51:45.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.816+0000 7fc31e093640 1 -- 172.21.15.43:0/3891481411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc318199420 con 0x7fc318198610 2024-04-21T08:51:45.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.816+0000 7fc314ff9640 1 -- 172.21.15.43:0/3891481411 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc310005020 con 0x7fc318198610 2024-04-21T08:51:45.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.816+0000 7fc314ff9640 1 -- 172.21.15.43:0/3891481411 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31000f6b0 con 0x7fc318198610 2024-04-21T08:51:45.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.816+0000 7fc31e093640 1 -- 172.21.15.43:0/3891481411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3181a56c0 con 0x7fc318198610 2024-04-21T08:51:45.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.818+0000 7fc314ff9640 1 -- 172.21.15.43:0/3891481411 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc3100051d0 con 0x7fc318198610 2024-04-21T08:51:45.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.818+0000 7fc320d65740 1 Processor -- start 2024-04-21T08:51:45.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.818+0000 7fc320d65740 1 -- start start 2024-04-21T08:51:45.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.818+0000 7fc320d65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d13d5914d0 0x55d13d5918b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.819+0000 7fc320d65740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d591e80 0x55d13d59a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.819+0000 7fc320d65740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d13d59aa00 0x55d13d59ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.819+0000 7fc320d65740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d13d4f61f0 con 0x55d13d591e80 2024-04-21T08:51:45.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.819+0000 7fc320d65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d13d4934b0 con 0x55d13d5914d0 2024-04-21T08:51:45.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.819+0000 7fc320d65740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d13d4f21b0 con 0x55d13d59aa00 2024-04-21T08:51:45.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.819+0000 7fc3177fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d591e80 0x55d13d59a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.819+0000 7fc314ff9640 1 --2- 172.21.15.43:0/3891481411 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc3000a3d20 0x7fc3000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.819+0000 7fc3177fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d591e80 0x55d13d59a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51458/0 (socket says 172.21.15.43:51458) 2024-04-21T08:51:45.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.819+0000 7fc3177fe640 1 -- 172.21.15.43:0/1885926285 learned_addr learned my addr 172.21.15.43:0/1885926285 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.819+0000 7fc317fff640 1 --2- 172.21.15.43:0/3891481411 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc3000a3d20 0x7fc3000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.819+0000 7fc316ffd640 1 --2- 172.21.15.43:0/1885926285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d13d59aa00 0x55d13d59ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.819+0000 7fc3177fe640 1 -- 172.21.15.43:0/1885926285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d13d59aa00 msgr2=0x55d13d59ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.819+0000 7fc316ffd640 1 --2- 172.21.15.43:0/1885926285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d13d5914d0 0x55d13d5918b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.819+0000 7fc3177fe640 1 --2- 172.21.15.43:0/1885926285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d13d59aa00 0x55d13d59ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.819+0000 7fc3177fe640 1 -- 172.21.15.43:0/1885926285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d13d5914d0 msgr2=0x55d13d5918b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.819+0000 7fc314ff9640 1 -- 172.21.15.43:0/3891481411 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fc3100d0c10 con 0x7fc318198610 2024-04-21T08:51:45.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.819+0000 7fc3177fe640 1 --2- 172.21.15.43:0/1885926285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d13d5914d0 0x55d13d5918b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.819+0000 7fc3177fe640 1 -- 172.21.15.43:0/1885926285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d13d4851b0 con 0x55d13d591e80 2024-04-21T08:51:45.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.819+0000 7fc316ffd640 1 --2- 172.21.15.43:0/1885926285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d13d5914d0 0x55d13d5918b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:45.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.819+0000 7fc3177fe640 1 --2- 172.21.15.43:0/1885926285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d591e80 0x55d13d59a430 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7fc308000c00 tx=0x7fc308005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd47d32397d195d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.819+0000 7fc317fff640 1 --2- 172.21.15.43:0/3891481411 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc3000a3d20 0x7fc3000a61e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc31009c470 tx=0x7fc310087040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.819+0000 7fc2fdffb640 1 -- 172.21.15.43:0/1885926285 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308044020 con 0x55d13d591e80 2024-04-21T08:51:45.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.820+0000 7fc2fdffb640 1 -- 172.21.15.43:0/1885926285 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc30802cc90 con 0x55d13d591e80 2024-04-21T08:51:45.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.820+0000 7fc2fdffb640 1 -- 172.21.15.43:0/1885926285 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308004920 con 0x55d13d591e80 2024-04-21T08:51:45.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.820+0000 7fc320d65740 1 -- 172.21.15.43:0/1885926285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d591e80 msgr2=0x55d13d59a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.820+0000 7fc320d65740 1 --2- 172.21.15.43:0/1885926285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d591e80 0x55d13d59a430 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7fc308000c00 tx=0x7fc308005e80 comp rx=0 tx=0).stop 2024-04-21T08:51:45.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.820+0000 7fc320d65740 1 -- 172.21.15.43:0/1885926285 shutdown_connections 2024-04-21T08:51:45.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.820+0000 7fc320d65740 1 --2- 172.21.15.43:0/1885926285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d13d59aa00 0x55d13d59ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.820+0000 7fc320d65740 1 --2- 172.21.15.43:0/1885926285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d591e80 0x55d13d59a430 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.820+0000 7fc320d65740 1 --2- 172.21.15.43:0/1885926285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d13d5914d0 0x55d13d5918b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.820+0000 7fc320d65740 1 -- 172.21.15.43:0/1885926285 >> 172.21.15.43:0/1885926285 conn(0x55d13d58bba0 msgr2=0x55d13d58bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.820+0000 7fc320d65740 1 -- 172.21.15.43:0/1885926285 shutdown_connections 2024-04-21T08:51:45.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.820+0000 7fc320d65740 1 -- 172.21.15.43:0/1885926285 wait complete. 2024-04-21T08:51:45.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.821+0000 7fc320d65740 1 -- start start 2024-04-21T08:51:45.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.821+0000 7fc320d65740 1 Processor -- start 2024-04-21T08:51:45.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.821+0000 7fc320d65740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d13d5914d0 0x55d13d59a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.821+0000 7fc320d65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d13d591e80 0x55d13d597630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.821+0000 7fc320d65740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d59aa00 0x55d13d597b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.821+0000 7fc320d65740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d13d59d910 con 0x55d13d59aa00 2024-04-21T08:51:45.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.821+0000 7fc320d65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d13d4af500 con 0x55d13d591e80 2024-04-21T08:51:45.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.821+0000 7fc320d65740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d13d59da90 con 0x55d13d5914d0 2024-04-21T08:51:45.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.821+0000 7fc320d65740 10 client.0 _ll_register_callbacks cb 0x55d13d47db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:45.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.821+0000 7fc316ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d59aa00 0x55d13d597b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.821+0000 7fc316ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d59aa00 0x55d13d597b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51466/0 (socket says 172.21.15.43:51466) 2024-04-21T08:51:45.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.821+0000 7fc316ffd640 1 -- 172.21.15.43:0/1363722024 learned_addr learned my addr 172.21.15.43:0/1363722024 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.821+0000 7fc316ffd640 1 --2- 172.21.15.43:0/1363722024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d13d5914d0 0x55d13d59a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.821+0000 7fc3177fe640 1 --2- 172.21.15.43:0/1363722024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d13d591e80 0x55d13d597630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.821+0000 7fc316ffd640 1 -- 172.21.15.43:0/1363722024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d13d5914d0 msgr2=0x55d13d59a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.821+0000 7fc316ffd640 1 --2- 172.21.15.43:0/1363722024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d13d5914d0 0x55d13d59a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.821+0000 7fc316ffd640 1 -- 172.21.15.43:0/1363722024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d13d591e80 msgr2=0x55d13d597630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.821+0000 7fc316ffd640 1 --2- 172.21.15.43:0/1363722024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d13d591e80 0x55d13d597630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.821+0000 7fc316ffd640 1 -- 172.21.15.43:0/1363722024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d13d2083f0 con 0x55d13d59aa00 2024-04-21T08:51:45.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.822+0000 7fc316ffd640 1 --2- 172.21.15.43:0/1363722024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d59aa00 0x55d13d597b70 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7fc30c0127b0 tx=0x7fc30c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a27247756ff910 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.822+0000 7fc2f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:45.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.822+0000 7fc2f6ffd640 1 -- 172.21.15.43:0/1363722024 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c004030 con 0x55d13d59aa00 2024-04-21T08:51:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.822+0000 7fc2f6ffd640 1 -- 172.21.15.43:0/1363722024 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fc30c0041d0 con 0x55d13d59aa00 2024-04-21T08:51:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.822+0000 7fc320d65740 1 -- 172.21.15.43:0/1363722024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc30802cc90 con 0x55d13d59aa00 2024-04-21T08:51:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.822+0000 7fc2f6ffd640 1 -- 172.21.15.43:0/1363722024 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc30c027050 con 0x55d13d59aa00 2024-04-21T08:51:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.822+0000 7fc320d65740 10 client.21528 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.822+0000 7fc320d65740 1 -- 172.21.15.43:0/1363722024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d13d2d27d0 con 0x55d13d59aa00 2024-04-21T08:51:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.822+0000 7fc2f6ffd640 1 -- 172.21.15.43:0/1363722024 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc30802cc90 con 0x55d13d59aa00 2024-04-21T08:51:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.822+0000 7fc2f6ffd640 1 -- 172.21.15.43:0/1363722024 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc30c005460 con 0x55d13d59aa00 2024-04-21T08:51:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.822+0000 7fc320d65740 10 client.21528 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.822+0000 7fc320d65740 10 client.21528 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:51:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.822+0000 7fc320d65740 20 client.21528 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.822+0000 7fc320d65740 1 --2- 172.21.15.43:0/1363722024 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55d13d5aaaa0 0x55d13d5aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.822+0000 7fc320d65740 4 client.21528 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.822+0000 7fc320d65740 1 -- 172.21.15.43:0/1363722024 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d13d495930 con 0x55d13d5aaaa0 2024-04-21T08:51:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.823+0000 7fc3177fe640 1 --2- 172.21.15.43:0/1363722024 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55d13d5aaaa0 0x55d13d5aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.823+0000 7fc3177fe640 1 --2- 172.21.15.43:0/1363722024 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55d13d5aaaa0 0x55d13d5aced0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0d1239241ab5479 server_cookie=4e99bb646355efc6 in_seq=0 out_seq=0 2024-04-21T08:51:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.823+0000 7fc2f6ffd640 10 client.21528 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:51:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.824+0000 7fc2f6ffd640 1 -- 172.21.15.43:0/1363722024 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d13d495930 con 0x55d13d5aaaa0 2024-04-21T08:51:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.824+0000 7fc2f6ffd640 10 client.21528 handle_command_reply: tid=0 2024-04-21T08:51:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.824+0000 7fc320d65740 1 client.21528 shutdown 2024-04-21T08:51:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.824+0000 7fc320d65740 1 -- 172.21.15.43:0/1363722024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d59aa00 msgr2=0x55d13d597b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.824+0000 7fc320d65740 1 --2- 172.21.15.43:0/1363722024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d59aa00 0x55d13d597b70 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7fc30c0127b0 tx=0x7fc30c004730 comp rx=0 tx=0).stop 2024-04-21T08:51:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.824+0000 7fc320d65740 1 -- 172.21.15.43:0/1363722024 shutdown_connections 2024-04-21T08:51:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.824+0000 7fc320d65740 1 --2- 172.21.15.43:0/1363722024 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55d13d5aaaa0 0x55d13d5aced0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.824+0000 7fc320d65740 1 --2- 172.21.15.43:0/1363722024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d59aa00 0x55d13d597b70 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.824+0000 7fc320d65740 1 --2- 172.21.15.43:0/1363722024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d13d591e80 0x55d13d597630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.824+0000 7fc2f6ffd640 0 client.21528 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:51:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.824+0000 7fc320d65740 1 --2- 172.21.15.43:0/1363722024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d13d5914d0 0x55d13d59a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.824+0000 7fc320d65740 1 -- 172.21.15.43:0/1363722024 >> 172.21.15.43:0/1363722024 conn(0x55d13d58bba0 msgr2=0x55d13d58d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.824+0000 7fc320d65740 1 -- 172.21.15.43:0/1363722024 shutdown_connections 2024-04-21T08:51:45.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.824+0000 7fc320d65740 1 -- 172.21.15.43:0/1363722024 wait complete. 2024-04-21T08:51:45.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.824+0000 7fc320d65740 20 client.21528 trim_cache size 0 max 16384 2024-04-21T08:51:45.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.837+0000 7fc320d65740 1 Processor -- start 2024-04-21T08:51:45.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.837+0000 7fc320d65740 1 -- start start 2024-04-21T08:51:45.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.838+0000 7fc320d65740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d13d5914d0 0x55d13d5a6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.838+0000 7fc320d65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d13d591e80 0x55d13d596790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.838+0000 7fc320d65740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d59aa00 0x55d13d58d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.838+0000 7fc320d65740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc3181af680 con 0x55d13d59aa00 2024-04-21T08:51:45.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.838+0000 7fc320d65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3181b11c0 con 0x55d13d591e80 2024-04-21T08:51:45.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.838+0000 7fc320d65740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3181afd50 con 0x55d13d5914d0 2024-04-21T08:51:45.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.838+0000 7fc316ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d59aa00 0x55d13d58d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.838+0000 7fc316ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d59aa00 0x55d13d58d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51476/0 (socket says 172.21.15.43:51476) 2024-04-21T08:51:45.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.838+0000 7fc316ffd640 1 -- 172.21.15.43:0/1659853712 learned_addr learned my addr 172.21.15.43:0/1659853712 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.838+0000 7fc316ffd640 1 -- 172.21.15.43:0/1659853712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d13d5914d0 msgr2=0x55d13d5a6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.838+0000 7fc316ffd640 1 --2- 172.21.15.43:0/1659853712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d13d5914d0 0x55d13d5a6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.838+0000 7fc316ffd640 1 -- 172.21.15.43:0/1659853712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d13d591e80 msgr2=0x55d13d596790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.838+0000 7fc316ffd640 1 --2- 172.21.15.43:0/1659853712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d13d591e80 0x55d13d596790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.838+0000 7fc316ffd640 1 -- 172.21.15.43:0/1659853712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d13d595b80 con 0x55d13d59aa00 2024-04-21T08:51:45.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.838+0000 7fc316ffd640 1 --2- 172.21.15.43:0/1659853712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d59aa00 0x55d13d58d700 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7fc30c004990 tx=0x7fc30c00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df17653bf5946cab server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.838+0000 7fc2fd7fa640 1 -- 172.21.15.43:0/1659853712 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c02f020 con 0x55d13d59aa00 2024-04-21T08:51:45.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.838+0000 7fc2fd7fa640 1 -- 172.21.15.43:0/1659853712 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc30c0120e0 con 0x55d13d59aa00 2024-04-21T08:51:45.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.838+0000 7fc2fd7fa640 1 -- 172.21.15.43:0/1659853712 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c004eb0 con 0x55d13d59aa00 2024-04-21T08:51:45.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.839+0000 7fc320d65740 1 -- 172.21.15.43:0/1659853712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d59aa00 msgr2=0x55d13d58d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.839+0000 7fc320d65740 1 --2- 172.21.15.43:0/1659853712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d59aa00 0x55d13d58d700 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7fc30c004990 tx=0x7fc30c00b410 comp rx=0 tx=0).stop 2024-04-21T08:51:45.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.839+0000 7fc320d65740 1 -- 172.21.15.43:0/1659853712 shutdown_connections 2024-04-21T08:51:45.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.839+0000 7fc320d65740 1 --2- 172.21.15.43:0/1659853712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d59aa00 0x55d13d58d700 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.839+0000 7fc320d65740 1 --2- 172.21.15.43:0/1659853712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d13d591e80 0x55d13d596790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.839+0000 7fc320d65740 1 --2- 172.21.15.43:0/1659853712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d13d5914d0 0x55d13d5a6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.839+0000 7fc320d65740 1 -- 172.21.15.43:0/1659853712 >> 172.21.15.43:0/1659853712 conn(0x55d13d58bba0 msgr2=0x55d13d58cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.839+0000 7fc320d65740 1 -- 172.21.15.43:0/1659853712 shutdown_connections 2024-04-21T08:51:45.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.839+0000 7fc320d65740 1 -- 172.21.15.43:0/1659853712 wait complete. 2024-04-21T08:51:45.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.839+0000 7fc320d65740 1 -- start start 2024-04-21T08:51:45.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.839+0000 7fc320d65740 1 Processor -- start 2024-04-21T08:51:45.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.840+0000 7fc320d65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d13d5914d0 0x55d13d5a2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.840+0000 7fc320d65740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d13d591e80 0x55d13d5a31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.840+0000 7fc320d65740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d59aa00 0x55d13d59e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.840+0000 7fc320d65740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc3181b0af0 con 0x55d13d59aa00 2024-04-21T08:51:45.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.840+0000 7fc320d65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3181b0420 con 0x55d13d5914d0 2024-04-21T08:51:45.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.840+0000 7fc320d65740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3181ad680 con 0x55d13d591e80 2024-04-21T08:51:45.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.840+0000 7fc320d65740 10 client.0 _ll_register_callbacks cb 0x7fc30c0123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:45.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.840+0000 7fc316ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d59aa00 0x55d13d59e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.840+0000 7fc316ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d59aa00 0x55d13d59e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51482/0 (socket says 172.21.15.43:51482) 2024-04-21T08:51:45.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.840+0000 7fc316ffd640 1 -- 172.21.15.43:0/1209343900 learned_addr learned my addr 172.21.15.43:0/1209343900 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.840+0000 7fc316ffd640 1 --2- 172.21.15.43:0/1209343900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d13d5914d0 0x55d13d5a2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.840+0000 7fc3177fe640 1 --2- 172.21.15.43:0/1209343900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d13d591e80 0x55d13d5a31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.840+0000 7fc316ffd640 1 -- 172.21.15.43:0/1209343900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d13d591e80 msgr2=0x55d13d5a31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.840+0000 7fc316ffd640 1 --2- 172.21.15.43:0/1209343900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d13d591e80 0x55d13d5a31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.840+0000 7fc316ffd640 1 -- 172.21.15.43:0/1209343900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d13d5914d0 msgr2=0x55d13d5a2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.840+0000 7fc316ffd640 1 --2- 172.21.15.43:0/1209343900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d13d5914d0 0x55d13d5a2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.840+0000 7fc316ffd640 1 -- 172.21.15.43:0/1209343900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d13d59e9d0 con 0x55d13d59aa00 2024-04-21T08:51:45.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.840+0000 7fc316ffd640 1 --2- 172.21.15.43:0/1209343900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d59aa00 0x55d13d59e1a0 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x55d13d59f280 tx=0x7fc30c010eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2df855275d8759b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.840+0000 7fc2f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:45.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.841+0000 7fc2f6ffd640 1 -- 172.21.15.43:0/1209343900 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c02f070 con 0x55d13d59aa00 2024-04-21T08:51:45.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.841+0000 7fc2f6ffd640 1 -- 172.21.15.43:0/1209343900 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fc30c0109a0 con 0x55d13d59aa00 2024-04-21T08:51:45.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.841+0000 7fc320d65740 1 -- 172.21.15.43:0/1209343900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc30c0120e0 con 0x55d13d59aa00 2024-04-21T08:51:45.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.841+0000 7fc2f6ffd640 1 -- 172.21.15.43:0/1209343900 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc30c010ba0 con 0x55d13d59aa00 2024-04-21T08:51:45.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.841+0000 7fc320d65740 10 client.21534 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:45.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.841+0000 7fc320d65740 1 -- 172.21.15.43:0/1209343900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d13d610c10 con 0x55d13d59aa00 2024-04-21T08:51:45.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.841+0000 7fc2f6ffd640 1 -- 172.21.15.43:0/1209343900 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc30c0120e0 con 0x55d13d59aa00 2024-04-21T08:51:45.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.841+0000 7fc2f6ffd640 1 -- 172.21.15.43:0/1209343900 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc30c01f540 con 0x55d13d59aa00 2024-04-21T08:51:45.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.841+0000 7fc320d65740 10 client.21534 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:45.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.841+0000 7fc320d65740 10 client.21534 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-21T08:51:45.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.841+0000 7fc320d65740 20 client.21534 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:45.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.841+0000 7fc320d65740 1 --2- 172.21.15.43:0/1209343900 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55d13d6112e0 0x55d13d613700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.841+0000 7fc320d65740 4 client.21534 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:51:45.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.841+0000 7fc320d65740 1 -- 172.21.15.43:0/1209343900 --> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d13d503c10 con 0x55d13d6112e0 2024-04-21T08:51:45.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.841+0000 7fc3177fe640 1 --2- 172.21.15.43:0/1209343900 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55d13d6112e0 0x55d13d613700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.842+0000 7fc3177fe640 1 --2- 172.21.15.43:0/1209343900 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55d13d6112e0 0x55d13d613700 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7d3290b710e2db2 server_cookie=cbe24e573d848bc5 in_seq=0 out_seq=0 2024-04-21T08:51:45.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.842+0000 7fc2f6ffd640 10 client.21534 ms_handle_connect on v2:172.21.15.112:6836/3054508488 2024-04-21T08:51:45.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.842+0000 7fc2f6ffd640 1 -- 172.21.15.43:0/1209343900 <== mds.1 v2:172.21.15.112:6836/3054508488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d13d503c10 con 0x55d13d6112e0 2024-04-21T08:51:45.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.842+0000 7fc2f6ffd640 10 client.21534 handle_command_reply: tid=0 2024-04-21T08:51:45.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.842+0000 7fc320d65740 1 client.21534 shutdown 2024-04-21T08:51:45.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.842+0000 7fc320d65740 1 -- 172.21.15.43:0/1209343900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d59aa00 msgr2=0x55d13d59e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.842+0000 7fc320d65740 1 --2- 172.21.15.43:0/1209343900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d59aa00 0x55d13d59e1a0 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x55d13d59f280 tx=0x7fc30c010eb0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.842+0000 7fc320d65740 1 -- 172.21.15.43:0/1209343900 shutdown_connections 2024-04-21T08:51:45.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.842+0000 7fc320d65740 1 --2- 172.21.15.43:0/1209343900 >> [v2:172.21.15.112:6836/3054508488,v1:172.21.15.112:6837/3054508488] conn(0x55d13d6112e0 0x55d13d613700 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.842+0000 7fc320d65740 1 --2- 172.21.15.43:0/1209343900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d13d59aa00 0x55d13d59e1a0 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.842+0000 7fc2f6ffd640 0 client.21534 ms_handle_reset on v2:172.21.15.112:6836/3054508488 2024-04-21T08:51:45.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.842+0000 7fc320d65740 1 --2- 172.21.15.43:0/1209343900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d13d591e80 0x55d13d5a31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.842+0000 7fc320d65740 1 --2- 172.21.15.43:0/1209343900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d13d5914d0 0x55d13d5a2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.842+0000 7fc320d65740 1 -- 172.21.15.43:0/1209343900 >> 172.21.15.43:0/1209343900 conn(0x55d13d58bba0 msgr2=0x55d13d58cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.842+0000 7fc320d65740 1 -- 172.21.15.43:0/1209343900 shutdown_connections 2024-04-21T08:51:45.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.842+0000 7fc320d65740 1 -- 172.21.15.43:0/1209343900 wait complete. 2024-04-21T08:51:45.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.842+0000 7fc320d65740 20 client.21534 trim_cache size 0 max 16384 2024-04-21T08:51:45.848 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:51:45.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.843+0000 7fc31e093640 1 -- 172.21.15.43:0/3891481411 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc3000a3d20 msgr2=0x7fc3000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.843+0000 7fc31e093640 1 --2- 172.21.15.43:0/3891481411 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc3000a3d20 0x7fc3000a61e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc31009c470 tx=0x7fc310087040 comp rx=0 tx=0).stop 2024-04-21T08:51:45.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.843+0000 7fc31e093640 1 -- 172.21.15.43:0/3891481411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc318198610 msgr2=0x7fc318198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.843+0000 7fc31e093640 1 --2- 172.21.15.43:0/3891481411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc318198610 0x7fc318198ac0 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7fc31000cad0 tx=0x7fc31000cf90 comp rx=0 tx=0).stop 2024-04-21T08:51:45.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.843+0000 7fc31e093640 1 -- 172.21.15.43:0/3891481411 shutdown_connections 2024-04-21T08:51:45.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.843+0000 7fc31e093640 1 --2- 172.21.15.43:0/3891481411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc318198610 0x7fc318198ac0 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.843+0000 7fc31e093640 1 --2- 172.21.15.43:0/3891481411 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc3000a3d20 0x7fc3000a61e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.843+0000 7fc31e093640 1 --2- 172.21.15.43:0/3891481411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc318107260 0x7fc31819e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.843+0000 7fc31e093640 1 --2- 172.21.15.43:0/3891481411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc318102190 0x7fc31819e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.843+0000 7fc31e093640 1 -- 172.21.15.43:0/3891481411 >> 172.21.15.43:0/3891481411 conn(0x7fc3180fd990 msgr2=0x7fc3180fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.843+0000 7fc31e093640 1 -- 172.21.15.43:0/3891481411 shutdown_connections 2024-04-21T08:51:45.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.843+0000 7fc31e093640 1 -- 172.21.15.43:0/3891481411 wait complete. 2024-04-21T08:51:45.852 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:45 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1358128324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:51:45.854 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:51:45.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.959+0000 7fa586681640 1 -- 172.21.15.43:0/240549240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa580101a50 msgr2=0x7fa580105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.959+0000 7fa586681640 1 --2- 172.21.15.43:0/240549240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa580101a50 0x7fa580105a90 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7fa574009900 tx=0x7fa57402bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:45.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.959+0000 7fa586681640 1 -- 172.21.15.43:0/240549240 shutdown_connections 2024-04-21T08:51:45.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.959+0000 7fa586681640 1 --2- 172.21.15.43:0/240549240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa580106170 0x7fa58010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.959+0000 7fa586681640 1 --2- 172.21.15.43:0/240549240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa580101a50 0x7fa580105a90 unknown :-1 s=CLOSED pgs=2403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.959+0000 7fa586681640 1 --2- 172.21.15.43:0/240549240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5801010a0 0x7fa580101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.959+0000 7fa586681640 1 -- 172.21.15.43:0/240549240 >> 172.21.15.43:0/240549240 conn(0x7fa580077eb0 msgr2=0x7fa5800782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.959+0000 7fa586681640 1 -- 172.21.15.43:0/240549240 shutdown_connections 2024-04-21T08:51:45.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.959+0000 7fa586681640 1 -- 172.21.15.43:0/240549240 wait complete. 2024-04-21T08:51:45.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.959+0000 7fa586681640 1 Processor -- start 2024-04-21T08:51:45.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.959+0000 7fa586681640 1 -- start start 2024-04-21T08:51:45.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.960+0000 7fa586681640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5801010a0 0x7fa58019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.960+0000 7fa586681640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa580106170 0x7fa58019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.960+0000 7fa586681640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5801986a0 0x7fa580198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.960+0000 7fa586681640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa580111110 con 0x7fa580106170 2024-04-21T08:51:45.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.960+0000 7fa586681640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa580110f90 con 0x7fa5801986a0 2024-04-21T08:51:45.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.960+0000 7fa586681640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa580111290 con 0x7fa5801010a0 2024-04-21T08:51:45.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.960+0000 7fa57f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa580106170 0x7fa58019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.960+0000 7fa57f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa580106170 0x7fa58019eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51502/0 (socket says 172.21.15.43:51502) 2024-04-21T08:51:45.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.960+0000 7fa57f7fe640 1 -- 172.21.15.43:0/3271969741 learned_addr learned my addr 172.21.15.43:0/3271969741 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.960+0000 7fa584c1e640 1 --2- 172.21.15.43:0/3271969741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5801986a0 0x7fa580198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.960+0000 7fa57ffff640 1 --2- 172.21.15.43:0/3271969741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5801010a0 0x7fa58019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.960+0000 7fa57f7fe640 1 -- 172.21.15.43:0/3271969741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5801010a0 msgr2=0x7fa58019e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.960+0000 7fa57f7fe640 1 --2- 172.21.15.43:0/3271969741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5801010a0 0x7fa58019e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.960+0000 7fa57f7fe640 1 -- 172.21.15.43:0/3271969741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5801986a0 msgr2=0x7fa580198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.960+0000 7fa57f7fe640 1 --2- 172.21.15.43:0/3271969741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5801986a0 0x7fa580198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.960+0000 7fa57f7fe640 1 -- 172.21.15.43:0/3271969741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5801992a0 con 0x7fa580106170 2024-04-21T08:51:45.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.960+0000 7fa57ffff640 1 --2- 172.21.15.43:0/3271969741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5801010a0 0x7fa58019e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:45.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.960+0000 7fa57f7fe640 1 --2- 172.21.15.43:0/3271969741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa580106170 0x7fa58019eb20 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7fa574036040 tx=0x7fa57402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.961+0000 7fa57d7fa640 1 -- 172.21.15.43:0/3271969741 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa574043020 con 0x7fa580106170 2024-04-21T08:51:45.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.961+0000 7fa57d7fa640 1 -- 172.21.15.43:0/3271969741 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa57402cbe0 con 0x7fa580106170 2024-04-21T08:51:45.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.961+0000 7fa586681640 1 -- 172.21.15.43:0/3271969741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa580199530 con 0x7fa580106170 2024-04-21T08:51:45.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.961+0000 7fa57d7fa640 1 -- 172.21.15.43:0/3271969741 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa574003bf0 con 0x7fa580106170 2024-04-21T08:51:45.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.961+0000 7fa586681640 1 -- 172.21.15.43:0/3271969741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5801a5720 con 0x7fa580106170 2024-04-21T08:51:45.965 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:45 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1358128324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:51:45.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.963+0000 7fa57d7fa640 1 -- 172.21.15.43:0/3271969741 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa574046020 con 0x7fa580106170 2024-04-21T08:51:45.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.963+0000 7fa589274740 1 Processor -- start 2024-04-21T08:51:45.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.963+0000 7fa589274740 1 -- start start 2024-04-21T08:51:45.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.963+0000 7fa57d7fa640 1 --2- 172.21.15.43:0/3271969741 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa5680a3da0 0x7fa5680a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.963+0000 7fa589274740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615e2f564d0 0x5615e2f568b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.963+0000 7fa589274740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615e2f56e80 0x5615e2f5f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.963+0000 7fa589274740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f5fa00 0x5615e2f61e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.963+0000 7fa589274740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5615e2ebb1f0 con 0x5615e2f5fa00 2024-04-21T08:51:45.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.963+0000 7fa589274740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5615e2e584b0 con 0x5615e2f564d0 2024-04-21T08:51:45.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.963+0000 7fa589274740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5615e2eb71b0 con 0x5615e2f56e80 2024-04-21T08:51:45.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.964+0000 7fa584c1e640 1 --2- 172.21.15.43:0/3271969741 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa5680a3da0 0x7fa5680a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.964+0000 7fa57d7fa640 1 -- 172.21.15.43:0/3271969741 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fa5740f2cb0 con 0x7fa580106170 2024-04-21T08:51:45.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.964+0000 7fa584c1e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f5fa00 0x5615e2f61e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.964+0000 7fa584c1e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f5fa00 0x5615e2f61e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51506/0 (socket says 172.21.15.43:51506) 2024-04-21T08:51:45.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.964+0000 7fa584c1e640 1 -- 172.21.15.43:0/4066631809 learned_addr learned my addr 172.21.15.43:0/4066631809 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.964+0000 7fa57ffff640 1 --2- 172.21.15.43:0/4066631809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615e2f564d0 0x5615e2f568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.964+0000 7fa57f7fe640 1 --2- 172.21.15.43:0/4066631809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615e2f56e80 0x5615e2f5f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.964+0000 7fa584c1e640 1 -- 172.21.15.43:0/4066631809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615e2f56e80 msgr2=0x5615e2f5f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.964+0000 7fa584c1e640 1 --2- 172.21.15.43:0/4066631809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615e2f56e80 0x5615e2f5f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.964+0000 7fa584c1e640 1 -- 172.21.15.43:0/4066631809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615e2f564d0 msgr2=0x5615e2f568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.964+0000 7fa584c1e640 1 --2- 172.21.15.43:0/4066631809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615e2f564d0 0x5615e2f568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.964+0000 7fa584c1e640 1 -- 172.21.15.43:0/4066631809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615e2e4a1b0 con 0x5615e2f5fa00 2024-04-21T08:51:45.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.964+0000 7fa584c1e640 1 --2- 172.21.15.43:0/3271969741 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa5680a3da0 0x7fa5680a6260 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa578012c80 tx=0x7fa578009e90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.964+0000 7fa584c1e640 1 --2- 172.21.15.43:0/4066631809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f5fa00 0x5615e2f61e50 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7fa578009fc0 tx=0x7fa57801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f71d87198abab55 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.964+0000 7fa5667fc640 1 -- 172.21.15.43:0/4066631809 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578024070 con 0x5615e2f5fa00 2024-04-21T08:51:45.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.964+0000 7fa5667fc640 1 -- 172.21.15.43:0/4066631809 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa57800dbe0 con 0x5615e2f5fa00 2024-04-21T08:51:45.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.964+0000 7fa5667fc640 1 -- 172.21.15.43:0/4066631809 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57801e3f0 con 0x5615e2f5fa00 2024-04-21T08:51:45.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.965+0000 7fa589274740 1 -- 172.21.15.43:0/4066631809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f5fa00 msgr2=0x5615e2f61e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.965+0000 7fa589274740 1 --2- 172.21.15.43:0/4066631809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f5fa00 0x5615e2f61e50 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7fa578009fc0 tx=0x7fa57801c000 comp rx=0 tx=0).stop 2024-04-21T08:51:45.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.965+0000 7fa589274740 1 -- 172.21.15.43:0/4066631809 shutdown_connections 2024-04-21T08:51:45.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.965+0000 7fa589274740 1 --2- 172.21.15.43:0/4066631809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f5fa00 0x5615e2f61e50 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.965+0000 7fa589274740 1 --2- 172.21.15.43:0/4066631809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615e2f56e80 0x5615e2f5f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.965+0000 7fa589274740 1 --2- 172.21.15.43:0/4066631809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615e2f564d0 0x5615e2f568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.965+0000 7fa589274740 1 -- 172.21.15.43:0/4066631809 >> 172.21.15.43:0/4066631809 conn(0x5615e2f50ba0 msgr2=0x5615e2f50f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.965+0000 7fa589274740 1 -- 172.21.15.43:0/4066631809 shutdown_connections 2024-04-21T08:51:45.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.965+0000 7fa589274740 1 -- 172.21.15.43:0/4066631809 wait complete. 2024-04-21T08:51:45.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.965+0000 7fa589274740 1 -- start start 2024-04-21T08:51:45.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.966+0000 7fa589274740 1 Processor -- start 2024-04-21T08:51:45.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.966+0000 7fa589274740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f564d0 0x5615e2f5f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.966+0000 7fa589274740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615e2f56e80 0x5615e2f5c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.966+0000 7fa589274740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615e2f5fa00 0x5615e2f5cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.966+0000 7fa589274740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5615e2f62910 con 0x5615e2f564d0 2024-04-21T08:51:45.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.966+0000 7fa589274740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5615e2e74500 con 0x5615e2f5fa00 2024-04-21T08:51:45.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.966+0000 7fa589274740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5615e2f62a90 con 0x5615e2f56e80 2024-04-21T08:51:45.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.966+0000 7fa589274740 10 client.0 _ll_register_callbacks cb 0x5615e2e42b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:45.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.966+0000 7fa57ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f564d0 0x5615e2f5f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.966+0000 7fa57ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f564d0 0x5615e2f5f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51512/0 (socket says 172.21.15.43:51512) 2024-04-21T08:51:45.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.966+0000 7fa57ffff640 1 -- 172.21.15.43:0/410694006 learned_addr learned my addr 172.21.15.43:0/410694006 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.966+0000 7fa57f7fe640 1 --2- 172.21.15.43:0/410694006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615e2f56e80 0x5615e2f5c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.966+0000 7fa584c1e640 1 --2- 172.21.15.43:0/410694006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615e2f5fa00 0x5615e2f5cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.966+0000 7fa57ffff640 1 -- 172.21.15.43:0/410694006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615e2f56e80 msgr2=0x5615e2f5c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.966+0000 7fa57ffff640 1 --2- 172.21.15.43:0/410694006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615e2f56e80 0x5615e2f5c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.966+0000 7fa57ffff640 1 -- 172.21.15.43:0/410694006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615e2f5fa00 msgr2=0x5615e2f5cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.966+0000 7fa57ffff640 1 --2- 172.21.15.43:0/410694006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615e2f5fa00 0x5615e2f5cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.966+0000 7fa57ffff640 1 -- 172.21.15.43:0/410694006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615e2bcd3f0 con 0x5615e2f564d0 2024-04-21T08:51:45.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.966+0000 7fa584c1e640 1 --2- 172.21.15.43:0/410694006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615e2f5fa00 0x5615e2f5cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:45.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.966+0000 7fa57ffff640 1 --2- 172.21.15.43:0/410694006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f564d0 0x5615e2f5f280 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fa570010500 tx=0x7fa570010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26355ec15aead147 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.966+0000 7fa55f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:45.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.967+0000 7fa55f7fe640 1 -- 172.21.15.43:0/410694006 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570018070 con 0x5615e2f564d0 2024-04-21T08:51:45.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.967+0000 7fa55f7fe640 1 -- 172.21.15.43:0/410694006 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fa570014640 con 0x5615e2f564d0 2024-04-21T08:51:45.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.967+0000 7fa589274740 1 -- 172.21.15.43:0/410694006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa57800dbe0 con 0x5615e2f564d0 2024-04-21T08:51:45.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.967+0000 7fa55f7fe640 1 -- 172.21.15.43:0/410694006 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa570024050 con 0x5615e2f564d0 2024-04-21T08:51:45.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.967+0000 7fa589274740 10 client.21546 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:45.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.967+0000 7fa589274740 1 -- 172.21.15.43:0/410694006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5615e2c977d0 con 0x5615e2f564d0 2024-04-21T08:51:45.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.967+0000 7fa55f7fe640 1 -- 172.21.15.43:0/410694006 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa57800dbe0 con 0x5615e2f564d0 2024-04-21T08:51:45.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.967+0000 7fa55f7fe640 1 -- 172.21.15.43:0/410694006 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa570012970 con 0x5615e2f564d0 2024-04-21T08:51:45.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.967+0000 7fa589274740 10 client.21546 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:45.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.967+0000 7fa589274740 10 client.21546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:45.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.967+0000 7fa589274740 20 client.21546 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:45.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.967+0000 7fa589274740 1 --2- 172.21.15.43:0/410694006 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5615e2f6fbc0 0x5615e2f72040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.967+0000 7fa589274740 4 client.21546 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:45.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.967+0000 7fa589274740 1 -- 172.21.15.43:0/410694006 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615e2e5a930 con 0x5615e2f6fbc0 2024-04-21T08:51:45.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.968+0000 7fa57f7fe640 1 --2- 172.21.15.43:0/410694006 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5615e2f6fbc0 0x5615e2f72040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.968+0000 7fa57f7fe640 1 --2- 172.21.15.43:0/410694006 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5615e2f6fbc0 0x5615e2f72040 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20bf789c721155fb server_cookie=207c32c9f1e46f5d in_seq=0 out_seq=0 2024-04-21T08:51:45.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.968+0000 7fa55f7fe640 10 client.21546 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:45.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.969+0000 7fa55f7fe640 1 -- 172.21.15.43:0/410694006 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5615e2e5a930 con 0x5615e2f6fbc0 2024-04-21T08:51:45.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.969+0000 7fa55f7fe640 10 client.21546 handle_command_reply: tid=0 2024-04-21T08:51:45.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.969+0000 7fa589274740 1 client.21546 shutdown 2024-04-21T08:51:45.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.969+0000 7fa589274740 1 -- 172.21.15.43:0/410694006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f564d0 msgr2=0x5615e2f5f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.969+0000 7fa589274740 1 --2- 172.21.15.43:0/410694006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f564d0 0x5615e2f5f280 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fa570010500 tx=0x7fa570010900 comp rx=0 tx=0).stop 2024-04-21T08:51:45.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.969+0000 7fa589274740 1 -- 172.21.15.43:0/410694006 shutdown_connections 2024-04-21T08:51:45.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.969+0000 7fa589274740 1 --2- 172.21.15.43:0/410694006 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5615e2f6fbc0 0x5615e2f72040 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.969+0000 7fa589274740 1 --2- 172.21.15.43:0/410694006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615e2f5fa00 0x5615e2f5cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.969+0000 7fa589274740 1 --2- 172.21.15.43:0/410694006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615e2f56e80 0x5615e2f5c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.969+0000 7fa55f7fe640 0 client.21546 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:45.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.969+0000 7fa589274740 1 --2- 172.21.15.43:0/410694006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f564d0 0x5615e2f5f280 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.969+0000 7fa589274740 1 -- 172.21.15.43:0/410694006 >> 172.21.15.43:0/410694006 conn(0x5615e2f50ba0 msgr2=0x5615e2f528f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.969+0000 7fa589274740 1 -- 172.21.15.43:0/410694006 shutdown_connections 2024-04-21T08:51:45.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.969+0000 7fa589274740 1 -- 172.21.15.43:0/410694006 wait complete. 2024-04-21T08:51:45.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.969+0000 7fa589274740 20 client.21546 trim_cache size 0 max 16384 2024-04-21T08:51:45.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.983+0000 7fa589274740 1 Processor -- start 2024-04-21T08:51:45.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.983+0000 7fa589274740 1 -- start start 2024-04-21T08:51:45.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.983+0000 7fa589274740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615e2f564d0 0x5615e2f6b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.983+0000 7fa589274740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f56e80 0x5615e2f4f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.983+0000 7fa589274740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615e2f5fa00 0x5615e2f5ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.983+0000 7fa589274740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa5801af4b0 con 0x5615e2f56e80 2024-04-21T08:51:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.983+0000 7fa589274740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5801b0ff0 con 0x5615e2f564d0 2024-04-21T08:51:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.983+0000 7fa589274740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa5801afb80 con 0x5615e2f5fa00 2024-04-21T08:51:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.983+0000 7fa57f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f56e80 0x5615e2f4f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.983+0000 7fa57f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f56e80 0x5615e2f4f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51518/0 (socket says 172.21.15.43:51518) 2024-04-21T08:51:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.983+0000 7fa57f7fe640 1 -- 172.21.15.43:0/2563165578 learned_addr learned my addr 172.21.15.43:0/2563165578 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.983+0000 7fa57f7fe640 1 -- 172.21.15.43:0/2563165578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615e2f5fa00 msgr2=0x5615e2f5ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.983+0000 7fa57ffff640 1 --2- 172.21.15.43:0/2563165578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615e2f564d0 0x5615e2f6b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.983+0000 7fa57f7fe640 1 --2- 172.21.15.43:0/2563165578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615e2f5fa00 0x5615e2f5ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.983+0000 7fa57f7fe640 1 -- 172.21.15.43:0/2563165578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615e2f564d0 msgr2=0x5615e2f6b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.983+0000 7fa57f7fe640 1 --2- 172.21.15.43:0/2563165578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615e2f564d0 0x5615e2f6b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.983+0000 7fa57f7fe640 1 -- 172.21.15.43:0/2563165578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615e2f50220 con 0x5615e2f56e80 2024-04-21T08:51:45.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.983+0000 7fa57f7fe640 1 --2- 172.21.15.43:0/2563165578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f56e80 0x5615e2f4f370 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7fa574006010 tx=0x7fa5740036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9258ff81d8a4061 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.984+0000 7fa565ffb640 1 -- 172.21.15.43:0/2563165578 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5740c2cf0 con 0x5615e2f56e80 2024-04-21T08:51:45.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.984+0000 7fa565ffb640 1 -- 172.21.15.43:0/2563165578 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5740c75b0 con 0x5615e2f56e80 2024-04-21T08:51:45.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.984+0000 7fa565ffb640 1 -- 172.21.15.43:0/2563165578 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5740b9170 con 0x5615e2f56e80 2024-04-21T08:51:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.984+0000 7fa589274740 1 -- 172.21.15.43:0/2563165578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f56e80 msgr2=0x5615e2f4f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.984+0000 7fa589274740 1 --2- 172.21.15.43:0/2563165578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f56e80 0x5615e2f4f370 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7fa574006010 tx=0x7fa5740036f0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.984+0000 7fa589274740 1 -- 172.21.15.43:0/2563165578 shutdown_connections 2024-04-21T08:51:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.984+0000 7fa589274740 1 --2- 172.21.15.43:0/2563165578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615e2f5fa00 0x5615e2f5ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.984+0000 7fa589274740 1 --2- 172.21.15.43:0/2563165578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f56e80 0x5615e2f4f370 unknown :-1 s=CLOSED pgs=2407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.984+0000 7fa589274740 1 --2- 172.21.15.43:0/2563165578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615e2f564d0 0x5615e2f6b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.984+0000 7fa589274740 1 -- 172.21.15.43:0/2563165578 >> 172.21.15.43:0/2563165578 conn(0x5615e2f50ba0 msgr2=0x5615e2f57af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.984+0000 7fa589274740 1 -- 172.21.15.43:0/2563165578 shutdown_connections 2024-04-21T08:51:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.984+0000 7fa589274740 1 -- 172.21.15.43:0/2563165578 wait complete. 2024-04-21T08:51:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.985+0000 7fa589274740 1 -- start start 2024-04-21T08:51:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.985+0000 7fa589274740 1 Processor -- start 2024-04-21T08:51:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.985+0000 7fa589274740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615e2f564d0 0x5615e2f59430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.985+0000 7fa589274740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f56e80 0x5615e2f59970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.985+0000 7fa589274740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615e2f5fa00 0x5615e2f59eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.985+0000 7fa589274740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa5801b0920 con 0x5615e2f56e80 2024-04-21T08:51:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.985+0000 7fa589274740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5801b0250 con 0x5615e2f564d0 2024-04-21T08:51:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.985+0000 7fa589274740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa5801ad4b0 con 0x5615e2f5fa00 2024-04-21T08:51:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.985+0000 7fa589274740 10 client.0 _ll_register_callbacks cb 0x7fa570007e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.985+0000 7fa57f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f56e80 0x5615e2f59970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.985+0000 7fa57f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f56e80 0x5615e2f59970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51520/0 (socket says 172.21.15.43:51520) 2024-04-21T08:51:45.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.985+0000 7fa57f7fe640 1 -- 172.21.15.43:0/63388112 learned_addr learned my addr 172.21.15.43:0/63388112 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:45.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.985+0000 7fa57ffff640 1 --2- 172.21.15.43:0/63388112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615e2f564d0 0x5615e2f59430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.986+0000 7fa584c1e640 1 --2- 172.21.15.43:0/63388112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615e2f5fa00 0x5615e2f59eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.986+0000 7fa57f7fe640 1 -- 172.21.15.43:0/63388112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615e2f5fa00 msgr2=0x5615e2f59eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.986+0000 7fa57f7fe640 1 --2- 172.21.15.43:0/63388112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615e2f5fa00 0x5615e2f59eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.986+0000 7fa57f7fe640 1 -- 172.21.15.43:0/63388112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615e2f564d0 msgr2=0x5615e2f59430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:45.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.986+0000 7fa57f7fe640 1 --2- 172.21.15.43:0/63388112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615e2f564d0 0x5615e2f59430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.986+0000 7fa57f7fe640 1 -- 172.21.15.43:0/63388112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615e2fd5320 con 0x5615e2f56e80 2024-04-21T08:51:45.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.986+0000 7fa57f7fe640 1 --2- 172.21.15.43:0/63388112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f56e80 0x5615e2f59970 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7fa574005ec0 tx=0x7fa5740c2cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e65b3abf475243 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:45.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.986+0000 7fa55f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-21T08:51:45.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.986+0000 7fa55f7fe640 1 -- 172.21.15.43:0/63388112 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa574003d90 con 0x5615e2f56e80 2024-04-21T08:51:45.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.986+0000 7fa55f7fe640 1 -- 172.21.15.43:0/63388112 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fa5740b9170 con 0x5615e2f56e80 2024-04-21T08:51:45.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.986+0000 7fa589274740 1 -- 172.21.15.43:0/63388112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5740c75b0 con 0x5615e2f56e80 2024-04-21T08:51:45.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.986+0000 7fa55f7fe640 1 -- 172.21.15.43:0/63388112 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa5740054d0 con 0x5615e2f56e80 2024-04-21T08:51:45.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.986+0000 7fa589274740 10 client.21552 fetch_fsmap learned FSMap version 30 2024-04-21T08:51:45.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.986+0000 7fa589274740 1 -- 172.21.15.43:0/63388112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5615e2fd65e0 con 0x5615e2f56e80 2024-04-21T08:51:45.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.986+0000 7fa55f7fe640 1 -- 172.21.15.43:0/63388112 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5740c75b0 con 0x5615e2f56e80 2024-04-21T08:51:45.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.986+0000 7fa55f7fe640 1 -- 172.21.15.43:0/63388112 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa57409fce0 con 0x5615e2f56e80 2024-04-21T08:51:45.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.986+0000 7fa589274740 10 client.21552 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:51:45.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.986+0000 7fa589274740 10 client.21552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:45.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.986+0000 7fa589274740 20 client.21552 populate_metadata read hostname 'smithi043' 2024-04-21T08:51:45.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.987+0000 7fa589274740 1 --2- 172.21.15.43:0/63388112 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5615e2fd6ab0 0x5615e2fd8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:45.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.987+0000 7fa589274740 4 client.21552 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:51:45.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.987+0000 7fa589274740 1 -- 172.21.15.43:0/63388112 --> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5615e2ec8c10 con 0x5615e2fd6ab0 2024-04-21T08:51:45.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.987+0000 7fa57ffff640 1 --2- 172.21.15.43:0/63388112 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5615e2fd6ab0 0x5615e2fd8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:45.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.987+0000 7fa55f7fe640 10 client.21552 ms_handle_connect on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:45.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.987+0000 7fa57ffff640 1 --2- 172.21.15.43:0/63388112 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5615e2fd6ab0 0x5615e2fd8ea0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8748562b1f3a623 server_cookie=92b45f28b4d79d3b in_seq=0 out_seq=0 2024-04-21T08:51:45.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.987+0000 7fa55f7fe640 1 -- 172.21.15.43:0/63388112 <== mds.0 v2:172.21.15.112:6838/2143575300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5615e2ec8c10 con 0x5615e2fd6ab0 2024-04-21T08:51:45.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.987+0000 7fa55f7fe640 10 client.21552 handle_command_reply: tid=0 2024-04-21T08:51:45.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.987+0000 7fa589274740 1 client.21552 shutdown 2024-04-21T08:51:45.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.988+0000 7fa589274740 1 -- 172.21.15.43:0/63388112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f56e80 msgr2=0x5615e2f59970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.988+0000 7fa589274740 1 --2- 172.21.15.43:0/63388112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f56e80 0x5615e2f59970 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7fa574005ec0 tx=0x7fa5740c2cf0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.988+0000 7fa589274740 1 -- 172.21.15.43:0/63388112 shutdown_connections 2024-04-21T08:51:45.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.988+0000 7fa589274740 1 --2- 172.21.15.43:0/63388112 >> [v2:172.21.15.112:6838/2143575300,v1:172.21.15.112:6839/2143575300] conn(0x5615e2fd6ab0 0x5615e2fd8ea0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.988+0000 7fa589274740 1 --2- 172.21.15.43:0/63388112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5615e2f5fa00 0x5615e2f59eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.988+0000 7fa55f7fe640 0 client.21552 ms_handle_reset on v2:172.21.15.112:6838/2143575300 2024-04-21T08:51:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.988+0000 7fa589274740 1 --2- 172.21.15.43:0/63388112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615e2f56e80 0x5615e2f59970 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.988+0000 7fa589274740 1 --2- 172.21.15.43:0/63388112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615e2f564d0 0x5615e2f59430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.988+0000 7fa589274740 1 -- 172.21.15.43:0/63388112 >> 172.21.15.43:0/63388112 conn(0x5615e2f50ba0 msgr2=0x5615e2f5be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.988+0000 7fa589274740 1 -- 172.21.15.43:0/63388112 shutdown_connections 2024-04-21T08:51:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.988+0000 7fa589274740 1 -- 172.21.15.43:0/63388112 wait complete. 2024-04-21T08:51:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.988+0000 7fa589274740 20 client.21552 trim_cache size 0 max 16384 2024-04-21T08:51:45.995 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-21T08:51:45.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.988+0000 7fa586681640 1 -- 172.21.15.43:0/3271969741 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa5680a3da0 msgr2=0x7fa5680a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.988+0000 7fa586681640 1 --2- 172.21.15.43:0/3271969741 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa5680a3da0 0x7fa5680a6260 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa578012c80 tx=0x7fa578009e90 comp rx=0 tx=0).stop 2024-04-21T08:51:45.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.988+0000 7fa586681640 1 -- 172.21.15.43:0/3271969741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa580106170 msgr2=0x7fa58019eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.988+0000 7fa586681640 1 --2- 172.21.15.43:0/3271969741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa580106170 0x7fa58019eb20 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7fa574036040 tx=0x7fa57402c7e0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.988+0000 7fa586681640 1 -- 172.21.15.43:0/3271969741 shutdown_connections 2024-04-21T08:51:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.988+0000 7fa586681640 1 --2- 172.21.15.43:0/3271969741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5801986a0 0x7fa580198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.988+0000 7fa586681640 1 --2- 172.21.15.43:0/3271969741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa580106170 0x7fa58019eb20 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.988+0000 7fa586681640 1 --2- 172.21.15.43:0/3271969741 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fa5680a3da0 0x7fa5680a6260 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.988+0000 7fa586681640 1 --2- 172.21.15.43:0/3271969741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5801010a0 0x7fa58019e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.988+0000 7fa586681640 1 -- 172.21.15.43:0/3271969741 >> 172.21.15.43:0/3271969741 conn(0x7fa580077eb0 msgr2=0x7fa58006aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.988+0000 7fa586681640 1 -- 172.21.15.43:0/3271969741 shutdown_connections 2024-04-21T08:51:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:45.988+0000 7fa586681640 1 -- 172.21.15.43:0/3271969741 wait complete. 2024-04-21T08:51:45.999 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/thrasher.py", line 43, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-04-21T08:51:46.000 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: 2024-04-21T08:51:46.000 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-21T08:51:46.012 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:51:46.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:45 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1358128324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:51:46.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.115+0000 7fb557959640 1 -- 172.21.15.43:0/3691713628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb550101ab0 msgr2=0x7fb55010dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:46.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.115+0000 7fb557959640 1 --2- 172.21.15.43:0/3691713628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb550101ab0 0x7fb55010dc50 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7fb548009cd0 tx=0x7fb54802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.115+0000 7fb557959640 1 -- 172.21.15.43:0/3691713628 shutdown_connections 2024-04-21T08:51:46.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.115+0000 7fb557959640 1 --2- 172.21.15.43:0/3691713628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb550101ab0 0x7fb55010dc50 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.115+0000 7fb557959640 1 --2- 172.21.15.43:0/3691713628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb550101110 0x7fb550101570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.115+0000 7fb557959640 1 --2- 172.21.15.43:0/3691713628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb550107010 0x7fb5501073f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.115+0000 7fb557959640 1 -- 172.21.15.43:0/3691713628 >> 172.21.15.43:0/3691713628 conn(0x7fb550077e30 msgr2=0x7fb550078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:46.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.115+0000 7fb557959640 1 -- 172.21.15.43:0/3691713628 shutdown_connections 2024-04-21T08:51:46.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.115+0000 7fb557959640 1 -- 172.21.15.43:0/3691713628 wait complete. 2024-04-21T08:51:46.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.116+0000 7fb557959640 1 Processor -- start 2024-04-21T08:51:46.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.116+0000 7fb557959640 1 -- start start 2024-04-21T08:51:46.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.116+0000 7fb557959640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb550101110 0x7fb550198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:46.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.116+0000 7fb557959640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb550101ab0 0x7fb550198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:46.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.116+0000 7fb557959640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb550107010 0x7fb55019d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:46.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.116+0000 7fb557959640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb5501110d0 con 0x7fb550101110 2024-04-21T08:51:46.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.116+0000 7fb557959640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb550110f50 con 0x7fb550101ab0 2024-04-21T08:51:46.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.116+0000 7fb557959640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb550111250 con 0x7fb550107010 2024-04-21T08:51:46.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.116+0000 7fb5556f5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb550101110 0x7fb550198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:46.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.116+0000 7fb5556f5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb550101110 0x7fb550198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51536/0 (socket says 172.21.15.43:51536) 2024-04-21T08:51:46.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.116+0000 7fb5556f5640 1 -- 172.21.15.43:0/2249484367 learned_addr learned my addr 172.21.15.43:0/2249484367 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:46.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.116+0000 7fb554ef4640 1 --2- 172.21.15.43:0/2249484367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb550101ab0 0x7fb550198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:46.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.116+0000 7fb555ef6640 1 --2- 172.21.15.43:0/2249484367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb550107010 0x7fb55019d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:46.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.116+0000 7fb5556f5640 1 -- 172.21.15.43:0/2249484367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb550107010 msgr2=0x7fb55019d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.116+0000 7fb5556f5640 1 --2- 172.21.15.43:0/2249484367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb550107010 0x7fb55019d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.116+0000 7fb5556f5640 1 -- 172.21.15.43:0/2249484367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb550101ab0 msgr2=0x7fb550198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.116+0000 7fb5556f5640 1 --2- 172.21.15.43:0/2249484367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb550101ab0 0x7fb550198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.116+0000 7fb5556f5640 1 -- 172.21.15.43:0/2249484367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb55019d730 con 0x7fb550101110 2024-04-21T08:51:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.117+0000 7fb5556f5640 1 --2- 172.21.15.43:0/2249484367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb550101110 0x7fb550198740 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00d950 tx=0x7fb54c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.117+0000 7fb5467fc640 1 -- 172.21.15.43:0/2249484367 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c017070 con 0x7fb550101110 2024-04-21T08:51:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.117+0000 7fb557959640 1 -- 172.21.15.43:0/2249484367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb55019da20 con 0x7fb550101110 2024-04-21T08:51:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.117+0000 7fb5467fc640 1 -- 172.21.15.43:0/2249484367 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb54c009c30 con 0x7fb550101110 2024-04-21T08:51:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.117+0000 7fb5467fc640 1 -- 172.21.15.43:0/2249484367 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c00f620 con 0x7fb550101110 2024-04-21T08:51:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.117+0000 7fb557959640 1 -- 172.21.15.43:0/2249484367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5501a52b0 con 0x7fb550101110 2024-04-21T08:51:46.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.118+0000 7fb557959640 1 -- 172.21.15.43:0/2249484367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb550069230 con 0x7fb550101110 2024-04-21T08:51:46.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.119+0000 7fb5467fc640 1 -- 172.21.15.43:0/2249484367 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb54c010460 con 0x7fb550101110 2024-04-21T08:51:46.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.119+0000 7fb5467fc640 1 --2- 172.21.15.43:0/2249484367 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb53c0a3d20 0x7fb53c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:46.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.120+0000 7fb5467fc640 1 -- 172.21.15.43:0/2249484367 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fb54c0d0880 con 0x7fb550101110 2024-04-21T08:51:46.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.120+0000 7fb554ef4640 1 --2- 172.21.15.43:0/2249484367 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb53c0a3d20 0x7fb53c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:46.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.120+0000 7fb554ef4640 1 --2- 172.21.15.43:0/2249484367 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb53c0a3d20 0x7fb53c0a61e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb540005fd0 tx=0x7fb540005850 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:46.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.124+0000 7fb5467fc640 1 -- 172.21.15.43:0/2249484367 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb54c09cb00 con 0x7fb550101110 2024-04-21T08:51:46.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.314+0000 7fb557959640 1 -- 172.21.15.43:0/2249484367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb550199a90 con 0x7fb550101110 2024-04-21T08:51:46.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.315+0000 7fb5467fc640 1 -- 172.21.15.43:0/2249484367 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+25419 (secure 0 0 0) 0x7fb54c009de0 con 0x7fb550101110 2024-04-21T08:51:46.315 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:51:46.316 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":108,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:51:10.066155+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/126475132":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/4288951239":"2024-04-22T08:38:54.881882+0000","172.21.15.43:6801/1577030772":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/2382670190":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/1174411284":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/1577030772":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]},{"pool":4,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:51:46.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.317+0000 7fb557959640 1 -- 172.21.15.43:0/2249484367 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb53c0a3d20 msgr2=0x7fb53c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:46.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.317+0000 7fb557959640 1 --2- 172.21.15.43:0/2249484367 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb53c0a3d20 0x7fb53c0a61e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb540005fd0 tx=0x7fb540005850 comp rx=0 tx=0).stop 2024-04-21T08:51:46.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.317+0000 7fb557959640 1 -- 172.21.15.43:0/2249484367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb550101110 msgr2=0x7fb550198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:46.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.317+0000 7fb557959640 1 --2- 172.21.15.43:0/2249484367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb550101110 0x7fb550198740 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00d950 tx=0x7fb54c00de10 comp rx=0 tx=0).stop 2024-04-21T08:51:46.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.318+0000 7fb557959640 1 -- 172.21.15.43:0/2249484367 shutdown_connections 2024-04-21T08:51:46.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.318+0000 7fb557959640 1 --2- 172.21.15.43:0/2249484367 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fb53c0a3d20 0x7fb53c0a61e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.318+0000 7fb557959640 1 --2- 172.21.15.43:0/2249484367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb550107010 0x7fb55019d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.318+0000 7fb557959640 1 --2- 172.21.15.43:0/2249484367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb550101ab0 0x7fb550198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.318+0000 7fb557959640 1 --2- 172.21.15.43:0/2249484367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb550101110 0x7fb550198740 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.318+0000 7fb557959640 1 -- 172.21.15.43:0/2249484367 >> 172.21.15.43:0/2249484367 conn(0x7fb550077e30 msgr2=0x7fb55006aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:46.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.318+0000 7fb557959640 1 -- 172.21.15.43:0/2249484367 shutdown_connections 2024-04-21T08:51:46.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.318+0000 7fb557959640 1 -- 172.21.15.43:0/2249484367 wait complete. 2024-04-21T08:51:46.329 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:39.981405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:22.034751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:22.780795+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 107, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:25.984261+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 107, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:51:46.329 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:51:46.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.435+0000 7feae433d640 1 -- 172.21.15.43:0/2205671535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feadc101a40 msgr2=0x7feadc105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:46.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.435+0000 7feae433d640 1 --2- 172.21.15.43:0/2205671535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feadc101a40 0x7feadc105a80 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7feacc009900 tx=0x7feacc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:46.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.435+0000 7feae433d640 1 -- 172.21.15.43:0/2205671535 shutdown_connections 2024-04-21T08:51:46.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.435+0000 7feae433d640 1 --2- 172.21.15.43:0/2205671535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feadc106160 0x7feadc10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.435+0000 7feae433d640 1 --2- 172.21.15.43:0/2205671535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feadc101a40 0x7feadc105a80 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.435+0000 7feae433d640 1 --2- 172.21.15.43:0/2205671535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feadc101090 0x7feadc101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.435+0000 7feae433d640 1 -- 172.21.15.43:0/2205671535 >> 172.21.15.43:0/2205671535 conn(0x7feadc077f50 msgr2=0x7feadc078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.435+0000 7feae433d640 1 -- 172.21.15.43:0/2205671535 shutdown_connections 2024-04-21T08:51:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.435+0000 7feae433d640 1 -- 172.21.15.43:0/2205671535 wait complete. 2024-04-21T08:51:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.435+0000 7feae433d640 1 Processor -- start 2024-04-21T08:51:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.435+0000 7feae433d640 1 -- start start 2024-04-21T08:51:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.436+0000 7feae433d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feadc101090 0x7feadc1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.436+0000 7feae433d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feadc106160 0x7feadc199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.436+0000 7feae433d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feadc19a070 0x7feadc1a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.436+0000 7feae433d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feadc111080 con 0x7feadc106160 2024-04-21T08:51:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.436+0000 7feae433d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feadc110f00 con 0x7feadc101090 2024-04-21T08:51:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.436+0000 7feae433d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feadc111200 con 0x7feadc19a070 2024-04-21T08:51:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.436+0000 7feae18d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feadc106160 0x7feadc199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:46.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.436+0000 7feae18d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feadc106160 0x7feadc199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51552/0 (socket says 172.21.15.43:51552) 2024-04-21T08:51:46.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.436+0000 7feae18d8640 1 -- 172.21.15.43:0/3277078677 learned_addr learned my addr 172.21.15.43:0/3277078677 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:46.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.436+0000 7feae20d9640 1 --2- 172.21.15.43:0/3277078677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feadc101090 0x7feadc1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:46.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.436+0000 7feae28da640 1 --2- 172.21.15.43:0/3277078677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feadc19a070 0x7feadc1a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:46.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.436+0000 7feae18d8640 1 -- 172.21.15.43:0/3277078677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feadc19a070 msgr2=0x7feadc1a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:46.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.436+0000 7feae18d8640 1 --2- 172.21.15.43:0/3277078677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feadc19a070 0x7feadc1a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.436+0000 7feae18d8640 1 -- 172.21.15.43:0/3277078677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feadc101090 msgr2=0x7feadc1995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:46.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.436+0000 7feae18d8640 1 --2- 172.21.15.43:0/3277078677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feadc101090 0x7feadc1995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.436+0000 7feae18d8640 1 -- 172.21.15.43:0/3277078677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feadc1a4df0 con 0x7feadc106160 2024-04-21T08:51:46.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.436+0000 7feae28da640 1 --2- 172.21.15.43:0/3277078677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feadc19a070 0x7feadc1a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:46.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.436+0000 7feae18d8640 1 --2- 172.21.15.43:0/3277078677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feadc106160 0x7feadc199ae0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7feacc036040 tx=0x7feacc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:46.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.436+0000 7fead37fe640 1 -- 172.21.15.43:0/3277078677 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc043020 con 0x7feadc106160 2024-04-21T08:51:46.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.436+0000 7fead37fe640 1 -- 172.21.15.43:0/3277078677 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feacc02cb70 con 0x7feadc106160 2024-04-21T08:51:46.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.436+0000 7feae433d640 1 -- 172.21.15.43:0/3277078677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feadc1a5080 con 0x7feadc106160 2024-04-21T08:51:46.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.436+0000 7fead37fe640 1 -- 172.21.15.43:0/3277078677 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc003bf0 con 0x7feadc106160 2024-04-21T08:51:46.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.436+0000 7feae433d640 1 -- 172.21.15.43:0/3277078677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feadc1a5620 con 0x7feadc106160 2024-04-21T08:51:46.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.437+0000 7feae433d640 1 -- 172.21.15.43:0/3277078677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feadc068d70 con 0x7feadc106160 2024-04-21T08:51:46.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.439+0000 7fead37fe640 1 -- 172.21.15.43:0/3277078677 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feacc046020 con 0x7feadc106160 2024-04-21T08:51:46.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.439+0000 7fead37fe640 1 --2- 172.21.15.43:0/3277078677 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7feac80a3da0 0x7feac80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:46.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.439+0000 7feae20d9640 1 --2- 172.21.15.43:0/3277078677 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7feac80a3da0 0x7feac80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:46.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.439+0000 7fead37fe640 1 -- 172.21.15.43:0/3277078677 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7feacc0f2c60 con 0x7feadc106160 2024-04-21T08:51:46.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.440+0000 7feae20d9640 1 --2- 172.21.15.43:0/3277078677 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7feac80a3da0 0x7feac80a6260 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fead800bbe0 tx=0x7fead800b390 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:46.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.444+0000 7fead37fe640 1 -- 172.21.15.43:0/3277078677 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feacc04d020 con 0x7feadc106160 2024-04-21T08:51:46.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.632+0000 7feae433d640 1 -- 172.21.15.43:0/3277078677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feadc109390 con 0x7feadc106160 2024-04-21T08:51:46.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.633+0000 7fead37fe640 1 -- 172.21.15.43:0/3277078677 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) ==== 93+0+10 (secure 0 0 0) 0x7feacc0beff0 con 0x7feadc106160 2024-04-21T08:51:46.632 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-21T08:51:46.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.635+0000 7feae433d640 1 -- 172.21.15.43:0/3277078677 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7feac80a3da0 msgr2=0x7feac80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.635+0000 7feae433d640 1 --2- 172.21.15.43:0/3277078677 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7feac80a3da0 0x7feac80a6260 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fead800bbe0 tx=0x7fead800b390 comp rx=0 tx=0).stop 2024-04-21T08:51:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.635+0000 7feae433d640 1 -- 172.21.15.43:0/3277078677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feadc106160 msgr2=0x7feadc199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.635+0000 7feae433d640 1 --2- 172.21.15.43:0/3277078677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feadc106160 0x7feadc199ae0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7feacc036040 tx=0x7feacc0045e0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.635+0000 7feae433d640 1 -- 172.21.15.43:0/3277078677 shutdown_connections 2024-04-21T08:51:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.635+0000 7feae433d640 1 --2- 172.21.15.43:0/3277078677 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7feac80a3da0 0x7feac80a6260 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.635+0000 7feae433d640 1 --2- 172.21.15.43:0/3277078677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feadc19a070 0x7feadc1a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.635+0000 7feae433d640 1 --2- 172.21.15.43:0/3277078677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feadc106160 0x7feadc199ae0 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.635+0000 7feae433d640 1 --2- 172.21.15.43:0/3277078677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feadc101090 0x7feadc1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.635+0000 7feae433d640 1 -- 172.21.15.43:0/3277078677 >> 172.21.15.43:0/3277078677 conn(0x7feadc077f50 msgr2=0x7feadc0ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.635+0000 7feae433d640 1 -- 172.21.15.43:0/3277078677 shutdown_connections 2024-04-21T08:51:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.635+0000 7feae433d640 1 -- 172.21.15.43:0/3277078677 wait complete. 2024-04-21T08:51:46.646 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:51:46.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.751+0000 7f2a7d18e640 1 -- 172.21.15.43:0/1891332271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a781010a0 msgr2=0x7f2a78101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:46.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.751+0000 7f2a7d18e640 1 --2- 172.21.15.43:0/1891332271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a781010a0 0x7f2a78101480 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009900 tx=0x7f2a6802bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:46.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.751+0000 7f2a7d18e640 1 -- 172.21.15.43:0/1891332271 shutdown_connections 2024-04-21T08:51:46.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.751+0000 7f2a7d18e640 1 --2- 172.21.15.43:0/1891332271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a78106170 0x7f2a7810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.751+0000 7f2a7d18e640 1 --2- 172.21.15.43:0/1891332271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a78101a50 0x7f2a78105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.751+0000 7f2a7d18e640 1 --2- 172.21.15.43:0/1891332271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a781010a0 0x7f2a78101480 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.751+0000 7f2a7d18e640 1 -- 172.21.15.43:0/1891332271 >> 172.21.15.43:0/1891332271 conn(0x7f2a78077eb0 msgr2=0x7f2a780782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:46.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.751+0000 7f2a7d18e640 1 -- 172.21.15.43:0/1891332271 shutdown_connections 2024-04-21T08:51:46.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.751+0000 7f2a7d18e640 1 -- 172.21.15.43:0/1891332271 wait complete. 2024-04-21T08:51:46.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.752+0000 7f2a7d18e640 1 Processor -- start 2024-04-21T08:51:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.752+0000 7f2a7d18e640 1 -- start start 2024-04-21T08:51:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.752+0000 7f2a7d18e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a781010a0 0x7f2a78102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.753+0000 7f2a7d18e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a78106170 0x7f2a78103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.753+0000 7f2a7d18e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a781035b0 0x7f2a781a6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.753+0000 7f2a7d18e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781110e0 con 0x7f2a78106170 2024-04-21T08:51:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.753+0000 7f2a7d18e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2a78110f60 con 0x7f2a781035b0 2024-04-21T08:51:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.753+0000 7f2a7d18e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2a78111260 con 0x7f2a781010a0 2024-04-21T08:51:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.753+0000 7f2a6e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a78106170 0x7f2a78103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.753+0000 7f2a6e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a78106170 0x7f2a78103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45090/0 (socket says 172.21.15.43:45090) 2024-04-21T08:51:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.753+0000 7f2a6e59c640 1 -- 172.21.15.43:0/1540994855 learned_addr learned my addr 172.21.15.43:0/1540994855 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.753+0000 7f2a76d9d640 1 --2- 172.21.15.43:0/1540994855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a781010a0 0x7f2a78102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.753+0000 7f2a7759e640 1 --2- 172.21.15.43:0/1540994855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a781035b0 0x7f2a781a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.753+0000 7f2a6e59c640 1 -- 172.21.15.43:0/1540994855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a781010a0 msgr2=0x7f2a78102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.753+0000 7f2a6e59c640 1 --2- 172.21.15.43:0/1540994855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a781010a0 0x7f2a78102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.753+0000 7f2a6e59c640 1 -- 172.21.15.43:0/1540994855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a781035b0 msgr2=0x7f2a781a6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.753+0000 7f2a6e59c640 1 --2- 172.21.15.43:0/1540994855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a781035b0 0x7f2a781a6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.753+0000 7f2a6e59c640 1 -- 172.21.15.43:0/1540994855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a781a6f90 con 0x7f2a78106170 2024-04-21T08:51:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.753+0000 7f2a76d9d640 1 --2- 172.21.15.43:0/1540994855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a781010a0 0x7f2a78102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.753+0000 7f2a6e59c640 1 --2- 172.21.15.43:0/1540994855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a78106170 0x7f2a78103070 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000d770 tx=0x7f2a6000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.753+0000 7f2a74d99640 1 -- 172.21.15.43:0/1540994855 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a60017070 con 0x7f2a78106170 2024-04-21T08:51:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.753+0000 7f2a7d18e640 1 -- 172.21.15.43:0/1540994855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a781a7280 con 0x7f2a78106170 2024-04-21T08:51:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.753+0000 7f2a74d99640 1 -- 172.21.15.43:0/1540994855 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a60009c30 con 0x7f2a78106170 2024-04-21T08:51:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.753+0000 7f2a74d99640 1 -- 172.21.15.43:0/1540994855 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6000f620 con 0x7f2a78106170 2024-04-21T08:51:46.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.753+0000 7f2a7d18e640 1 -- 172.21.15.43:0/1540994855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a781a77d0 con 0x7f2a78106170 2024-04-21T08:51:46.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.755+0000 7f2a7d18e640 1 -- 172.21.15.43:0/1540994855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a780691c0 con 0x7f2a78106170 2024-04-21T08:51:46.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.756+0000 7f2a74d99640 1 -- 172.21.15.43:0/1540994855 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2a6001a070 con 0x7f2a78106170 2024-04-21T08:51:46.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.756+0000 7f2a74d99640 1 --2- 172.21.15.43:0/1540994855 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2a640a3d50 0x7f2a640a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:46.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.757+0000 7f2a76d9d640 1 --2- 172.21.15.43:0/1540994855 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2a640a3d50 0x7f2a640a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:46.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.757+0000 7f2a74d99640 1 -- 172.21.15.43:0/1540994855 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f2a600d07f0 con 0x7f2a78106170 2024-04-21T08:51:46.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.757+0000 7f2a76d9d640 1 --2- 172.21.15.43:0/1540994855 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2a640a3d50 0x7f2a640a6210 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2a68036040 tx=0x7f2a68000bc0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:46.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.761+0000 7f2a74d99640 1 -- 172.21.15.43:0/1540994855 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a6009ca70 con 0x7f2a78106170 2024-04-21T08:51:46.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.947+0000 7f2a7d18e640 1 -- 172.21.15.43:0/1540994855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2a7810a3b0 con 0x7f2a78106170 2024-04-21T08:51:46.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.947+0000 7f2a74d99640 1 -- 172.21.15.43:0/1540994855 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) ==== 104+0+11 (secure 0 0 0) 0x7f2a6009c370 con 0x7f2a78106170 2024-04-21T08:51:46.949 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:51:46.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.949+0000 7f2a7d18e640 1 -- 172.21.15.43:0/1540994855 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2a640a3d50 msgr2=0x7f2a640a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:46.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.949+0000 7f2a7d18e640 1 --2- 172.21.15.43:0/1540994855 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2a640a3d50 0x7f2a640a6210 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2a68036040 tx=0x7f2a68000bc0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.949+0000 7f2a7d18e640 1 -- 172.21.15.43:0/1540994855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a78106170 msgr2=0x7f2a78103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:46.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.949+0000 7f2a7d18e640 1 --2- 172.21.15.43:0/1540994855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a78106170 0x7f2a78103070 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000d770 tx=0x7f2a6000dc30 comp rx=0 tx=0).stop 2024-04-21T08:51:46.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.949+0000 7f2a7d18e640 1 -- 172.21.15.43:0/1540994855 shutdown_connections 2024-04-21T08:51:46.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.949+0000 7f2a7d18e640 1 --2- 172.21.15.43:0/1540994855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a781035b0 0x7f2a781a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.949+0000 7f2a7d18e640 1 --2- 172.21.15.43:0/1540994855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a78106170 0x7f2a78103070 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.949+0000 7f2a7d18e640 1 --2- 172.21.15.43:0/1540994855 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f2a640a3d50 0x7f2a640a6210 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.949+0000 7f2a7d18e640 1 --2- 172.21.15.43:0/1540994855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a781010a0 0x7f2a78102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:46.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.949+0000 7f2a7d18e640 1 -- 172.21.15.43:0/1540994855 >> 172.21.15.43:0/1540994855 conn(0x7f2a78077eb0 msgr2=0x7f2a7806a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:46.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.950+0000 7f2a7d18e640 1 -- 172.21.15.43:0/1540994855 shutdown_connections 2024-04-21T08:51:46.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:46.950+0000 7f2a7d18e640 1 -- 172.21.15.43:0/1540994855 wait complete. 2024-04-21T08:51:46.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:46 smithi043 ceph-mon[27978]: pgmap v401: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-04-21T08:51:46.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:46 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2249484367' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:51:46.961 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:51:46.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:46 smithi175 ceph-mon[31920]: pgmap v401: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-04-21T08:51:46.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:46 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2249484367' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:51:47.046 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:46 smithi112 ceph-mon[31085]: pgmap v401: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-04-21T08:51:47.046 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:46 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2249484367' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:51:47.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.067+0000 7f7c468eb640 1 -- 172.21.15.43:0/3453065213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c40103af0 msgr2=0x7f7c4010bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:47.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.067+0000 7f7c468eb640 1 --2- 172.21.15.43:0/3453065213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c40103af0 0x7f7c4010bfd0 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f7c34009900 tx=0x7f7c3402bd60 comp rx=0 tx=0).stop 2024-04-21T08:51:47.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.067+0000 7f7c468eb640 1 -- 172.21.15.43:0/3453065213 shutdown_connections 2024-04-21T08:51:47.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.067+0000 7f7c468eb640 1 --2- 172.21.15.43:0/3453065213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c4010c510 0x7f7c4010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.067+0000 7f7c468eb640 1 --2- 172.21.15.43:0/3453065213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c40103af0 0x7f7c4010bfd0 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.067+0000 7f7c468eb640 1 --2- 172.21.15.43:0/3453065213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c401031d0 0x7f7c401035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.067+0000 7f7c468eb640 1 -- 172.21.15.43:0/3453065213 >> 172.21.15.43:0/3453065213 conn(0x7f7c40077f50 msgr2=0x7f7c40078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:47.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.067+0000 7f7c468eb640 1 -- 172.21.15.43:0/3453065213 shutdown_connections 2024-04-21T08:51:47.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.067+0000 7f7c468eb640 1 -- 172.21.15.43:0/3453065213 wait complete. 2024-04-21T08:51:47.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.067+0000 7f7c468eb640 1 Processor -- start 2024-04-21T08:51:47.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.067+0000 7f7c468eb640 1 -- start start 2024-04-21T08:51:47.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.068+0000 7f7c468eb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c401031d0 0x7f7c401a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:47.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.068+0000 7f7c468eb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c40103af0 0x7f7c401a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:47.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.068+0000 7f7c468eb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c4010c510 0x7f7c4019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:47.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.068+0000 7f7c468eb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7c401121a0 con 0x7f7c4010c510 2024-04-21T08:51:47.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.068+0000 7f7c468eb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7c40112020 con 0x7f7c401031d0 2024-04-21T08:51:47.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.068+0000 7f7c468eb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7c40112320 con 0x7f7c40103af0 2024-04-21T08:51:47.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.068+0000 7f7c44e88640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c4010c510 0x7f7c4019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:47.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.068+0000 7f7c44e88640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c4010c510 0x7f7c4019a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45100/0 (socket says 172.21.15.43:45100) 2024-04-21T08:51:47.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.068+0000 7f7c44e88640 1 -- 172.21.15.43:0/3597748838 learned_addr learned my addr 172.21.15.43:0/3597748838 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:47.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.068+0000 7f7c3f7fe640 1 --2- 172.21.15.43:0/3597748838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c40103af0 0x7f7c401a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:47.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.068+0000 7f7c3ffff640 1 --2- 172.21.15.43:0/3597748838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c401031d0 0x7f7c401a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:47.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.068+0000 7f7c44e88640 1 -- 172.21.15.43:0/3597748838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c40103af0 msgr2=0x7f7c401a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:47.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.068+0000 7f7c44e88640 1 --2- 172.21.15.43:0/3597748838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c40103af0 0x7f7c401a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.068+0000 7f7c44e88640 1 -- 172.21.15.43:0/3597748838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c401031d0 msgr2=0x7f7c401a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:47.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.068+0000 7f7c44e88640 1 --2- 172.21.15.43:0/3597748838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c401031d0 0x7f7c401a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.068+0000 7f7c44e88640 1 -- 172.21.15.43:0/3597748838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4019b080 con 0x7f7c4010c510 2024-04-21T08:51:47.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.068+0000 7f7c3ffff640 1 --2- 172.21.15.43:0/3597748838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c401031d0 0x7f7c401a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:47.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.068+0000 7f7c44e88640 1 --2- 172.21.15.43:0/3597748838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c4010c510 0x7f7c4019a7c0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800cad0 tx=0x7f7c3800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:47.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.069+0000 7f7c3d7fa640 1 -- 172.21.15.43:0/3597748838 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3800ae20 con 0x7f7c4010c510 2024-04-21T08:51:47.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.069+0000 7f7c3d7fa640 1 -- 172.21.15.43:0/3597748838 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c38005020 con 0x7f7c4010c510 2024-04-21T08:51:47.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.069+0000 7f7c468eb640 1 -- 172.21.15.43:0/3597748838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c4019b370 con 0x7f7c4010c510 2024-04-21T08:51:47.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.069+0000 7f7c3d7fa640 1 -- 172.21.15.43:0/3597748838 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3800f6b0 con 0x7f7c4010c510 2024-04-21T08:51:47.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.069+0000 7f7c468eb640 1 -- 172.21.15.43:0/3597748838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c401a7630 con 0x7f7c4010c510 2024-04-21T08:51:47.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.070+0000 7f7c468eb640 1 -- 172.21.15.43:0/3597748838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c4019b6b0 con 0x7f7c4010c510 2024-04-21T08:51:47.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.071+0000 7f7c3d7fa640 1 -- 172.21.15.43:0/3597748838 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7c380051d0 con 0x7f7c4010c510 2024-04-21T08:51:47.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.072+0000 7f7c3d7fa640 1 --2- 172.21.15.43:0/3597748838 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7c280a3d20 0x7f7c280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:47.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.072+0000 7f7c3ffff640 1 --2- 172.21.15.43:0/3597748838 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7c280a3d20 0x7f7c280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:47.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.072+0000 7f7c3d7fa640 1 -- 172.21.15.43:0/3597748838 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f7c380d0c00 con 0x7f7c4010c510 2024-04-21T08:51:47.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.072+0000 7f7c3ffff640 1 --2- 172.21.15.43:0/3597748838 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7c280a3d20 0x7f7c280a61e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7c30005fd0 tx=0x7f7c3000b5f0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:47.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.076+0000 7f7c3d7fa640 1 -- 172.21.15.43:0/3597748838 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7c380d7080 con 0x7f7c4010c510 2024-04-21T08:51:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.263+0000 7f7c468eb640 1 -- 172.21.15.43:0/3597748838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7c401a7800 con 0x7f7c4010c510 2024-04-21T08:51:47.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.264+0000 7f7c3d7fa640 1 -- 172.21.15.43:0/3597748838 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v108) ==== 100+0+11 (secure 0 0 0) 0x7f7c38014030 con 0x7f7c4010c510 2024-04-21T08:51:47.264 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-21T08:51:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.266+0000 7f7c468eb640 1 -- 172.21.15.43:0/3597748838 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7c280a3d20 msgr2=0x7f7c280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.266+0000 7f7c468eb640 1 --2- 172.21.15.43:0/3597748838 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7c280a3d20 0x7f7c280a61e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7c30005fd0 tx=0x7f7c3000b5f0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.266+0000 7f7c468eb640 1 -- 172.21.15.43:0/3597748838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c4010c510 msgr2=0x7f7c4019a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.266+0000 7f7c468eb640 1 --2- 172.21.15.43:0/3597748838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c4010c510 0x7f7c4019a7c0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800cad0 tx=0x7f7c3800cf90 comp rx=0 tx=0).stop 2024-04-21T08:51:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.266+0000 7f7c468eb640 1 -- 172.21.15.43:0/3597748838 shutdown_connections 2024-04-21T08:51:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.266+0000 7f7c468eb640 1 --2- 172.21.15.43:0/3597748838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c4010c510 0x7f7c4019a7c0 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.266+0000 7f7c468eb640 1 --2- 172.21.15.43:0/3597748838 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7c280a3d20 0x7f7c280a61e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.266+0000 7f7c468eb640 1 --2- 172.21.15.43:0/3597748838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c40103af0 0x7f7c401a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.266+0000 7f7c468eb640 1 --2- 172.21.15.43:0/3597748838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c401031d0 0x7f7c401a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.266+0000 7f7c468eb640 1 -- 172.21.15.43:0/3597748838 >> 172.21.15.43:0/3597748838 conn(0x7f7c40077f50 msgr2=0x7f7c401084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.266+0000 7f7c468eb640 1 -- 172.21.15.43:0/3597748838 shutdown_connections 2024-04-21T08:51:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.266+0000 7f7c468eb640 1 -- 172.21.15.43:0/3597748838 wait complete. 2024-04-21T08:51:47.277 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:51:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.384+0000 7f5ad3b4b640 1 -- 172.21.15.43:0/2061474284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc106170 msgr2=0x7f5acc10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.384+0000 7f5ad3b4b640 1 --2- 172.21.15.43:0/2061474284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc106170 0x7f5acc10dd40 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f5abc009900 tx=0x7f5abc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.384+0000 7f5ad3b4b640 1 -- 172.21.15.43:0/2061474284 shutdown_connections 2024-04-21T08:51:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.384+0000 7f5ad3b4b640 1 --2- 172.21.15.43:0/2061474284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc106170 0x7f5acc10dd40 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.384+0000 7f5ad3b4b640 1 --2- 172.21.15.43:0/2061474284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5acc101aa0 0x7f5acc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.384+0000 7f5ad3b4b640 1 --2- 172.21.15.43:0/2061474284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5acc1010f0 0x7f5acc1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.384+0000 7f5ad3b4b640 1 -- 172.21.15.43:0/2061474284 >> 172.21.15.43:0/2061474284 conn(0x7f5acc077fb0 msgr2=0x7f5acc0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.384+0000 7f5ad3b4b640 1 -- 172.21.15.43:0/2061474284 shutdown_connections 2024-04-21T08:51:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.384+0000 7f5ad3b4b640 1 -- 172.21.15.43:0/2061474284 wait complete. 2024-04-21T08:51:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.385+0000 7f5ad3b4b640 1 Processor -- start 2024-04-21T08:51:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.385+0000 7f5ad3b4b640 1 -- start start 2024-04-21T08:51:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.385+0000 7f5ad3b4b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5acc1010f0 0x7f5acc1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.385+0000 7f5ad3b4b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc106170 0x7f5acc1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.385+0000 7f5ad3b4b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5acc19a740 0x7f5acc19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.385+0000 7f5ad3b4b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc1110e0 con 0x7f5acc106170 2024-04-21T08:51:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.385+0000 7f5ad3b4b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc110f60 con 0x7f5acc1010f0 2024-04-21T08:51:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.385+0000 7f5ad3b4b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc111260 con 0x7f5acc19a740 2024-04-21T08:51:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.385+0000 7f5ad10e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc106170 0x7f5acc1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.385+0000 7f5ad10e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc106170 0x7f5acc1a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45118/0 (socket says 172.21.15.43:45118) 2024-04-21T08:51:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.385+0000 7f5ad10e6640 1 -- 172.21.15.43:0/2201092360 learned_addr learned my addr 172.21.15.43:0/2201092360 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.385+0000 7f5ad18e7640 1 --2- 172.21.15.43:0/2201092360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5acc1010f0 0x7f5acc1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.386+0000 7f5ad20e8640 1 --2- 172.21.15.43:0/2201092360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5acc19a740 0x7f5acc19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.386+0000 7f5ad10e6640 1 -- 172.21.15.43:0/2201092360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5acc19a740 msgr2=0x7f5acc19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.386+0000 7f5ad10e6640 1 --2- 172.21.15.43:0/2201092360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5acc19a740 0x7f5acc19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.386+0000 7f5ad10e6640 1 -- 172.21.15.43:0/2201092360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5acc1010f0 msgr2=0x7f5acc1a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.386+0000 7f5ad10e6640 1 --2- 172.21.15.43:0/2201092360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5acc1010f0 0x7f5acc1a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.386+0000 7f5ad10e6640 1 -- 172.21.15.43:0/2201092360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5acc19b340 con 0x7f5acc106170 2024-04-21T08:51:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.386+0000 7f5ad18e7640 1 --2- 172.21.15.43:0/2201092360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5acc1010f0 0x7f5acc1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.386+0000 7f5ad20e8640 1 --2- 172.21.15.43:0/2201092360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5acc19a740 0x7f5acc19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.386+0000 7f5ad10e6640 1 --2- 172.21.15.43:0/2201092360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc106170 0x7f5acc1a0bc0 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000c970 tx=0x7f5ac000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.386+0000 7f5acaffd640 1 -- 172.21.15.43:0/2201092360 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0007bb0 con 0x7f5acc106170 2024-04-21T08:51:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.386+0000 7f5acaffd640 1 -- 172.21.15.43:0/2201092360 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ac0007d50 con 0x7f5acc106170 2024-04-21T08:51:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.386+0000 7f5acaffd640 1 -- 172.21.15.43:0/2201092360 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0018730 con 0x7f5acc106170 2024-04-21T08:51:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.386+0000 7f5ad3b4b640 1 -- 172.21.15.43:0/2201092360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5acc1a7420 con 0x7f5acc106170 2024-04-21T08:51:47.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.386+0000 7f5ad3b4b640 1 -- 172.21.15.43:0/2201092360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5acc1a77d0 con 0x7f5acc106170 2024-04-21T08:51:47.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.387+0000 7f5ad3b4b640 1 -- 172.21.15.43:0/2201092360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5acc102570 con 0x7f5acc106170 2024-04-21T08:51:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.388+0000 7f5acaffd640 1 -- 172.21.15.43:0/2201092360 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5ac000f990 con 0x7f5acc106170 2024-04-21T08:51:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.389+0000 7f5acaffd640 1 --2- 172.21.15.43:0/2201092360 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5ab40a3d20 0x7f5ab40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.389+0000 7f5acaffd640 1 -- 172.21.15.43:0/2201092360 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f5ac00d1080 con 0x7f5acc106170 2024-04-21T08:51:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.389+0000 7f5ad18e7640 1 --2- 172.21.15.43:0/2201092360 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5ab40a3d20 0x7f5ab40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.389+0000 7f5ad18e7640 1 --2- 172.21.15.43:0/2201092360 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5ab40a3d20 0x7f5ab40a61e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4005fd0 tx=0x7f5ac4009970 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:47.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.393+0000 7f5acaffd640 1 -- 172.21.15.43:0/2201092360 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5ac0014030 con 0x7f5acc106170 2024-04-21T08:51:47.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.583+0000 7f5ad3b4b640 1 -- 172.21.15.43:0/2201092360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5acc0684e0 con 0x7f5acc106170 2024-04-21T08:51:47.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.583+0000 7f5acaffd640 1 -- 172.21.15.43:0/2201092360 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v108) ==== 103+0+11 (secure 0 0 0) 0x7f5ac009d300 con 0x7f5acc106170 2024-04-21T08:51:47.583 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:51:47.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.585+0000 7f5ad3b4b640 1 -- 172.21.15.43:0/2201092360 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5ab40a3d20 msgr2=0x7f5ab40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:47.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.585+0000 7f5ad3b4b640 1 --2- 172.21.15.43:0/2201092360 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5ab40a3d20 0x7f5ab40a61e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4005fd0 tx=0x7f5ac4009970 comp rx=0 tx=0).stop 2024-04-21T08:51:47.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.585+0000 7f5ad3b4b640 1 -- 172.21.15.43:0/2201092360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc106170 msgr2=0x7f5acc1a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:47.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.585+0000 7f5ad3b4b640 1 --2- 172.21.15.43:0/2201092360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc106170 0x7f5acc1a0bc0 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000c970 tx=0x7f5ac000ce30 comp rx=0 tx=0).stop 2024-04-21T08:51:47.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.585+0000 7f5ad3b4b640 1 -- 172.21.15.43:0/2201092360 shutdown_connections 2024-04-21T08:51:47.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.585+0000 7f5ad3b4b640 1 --2- 172.21.15.43:0/2201092360 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f5ab40a3d20 0x7f5ab40a61e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.585+0000 7f5ad3b4b640 1 --2- 172.21.15.43:0/2201092360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5acc19a740 0x7f5acc19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.585+0000 7f5ad3b4b640 1 --2- 172.21.15.43:0/2201092360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc106170 0x7f5acc1a0bc0 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.585+0000 7f5ad3b4b640 1 --2- 172.21.15.43:0/2201092360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5acc1010f0 0x7f5acc1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.585+0000 7f5ad3b4b640 1 -- 172.21.15.43:0/2201092360 >> 172.21.15.43:0/2201092360 conn(0x7f5acc077fb0 msgr2=0x7f5acc100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:47.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.585+0000 7f5ad3b4b640 1 -- 172.21.15.43:0/2201092360 shutdown_connections 2024-04-21T08:51:47.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.585+0000 7f5ad3b4b640 1 -- 172.21.15.43:0/2201092360 wait complete. 2024-04-21T08:51:47.597 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-21T08:51:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.700+0000 7f61081f4640 1 -- 172.21.15.43:0/371614953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6100103230 msgr2=0x7f6100103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.700+0000 7f61081f4640 1 --2- 172.21.15.43:0/371614953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6100103230 0x7f6100103610 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7f60fc009900 tx=0x7f60fc02bcd0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.700+0000 7f61081f4640 1 -- 172.21.15.43:0/371614953 shutdown_connections 2024-04-21T08:51:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.700+0000 7f61081f4640 1 --2- 172.21.15.43:0/371614953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f610010c570 0x7f610010e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.700+0000 7f61081f4640 1 --2- 172.21.15.43:0/371614953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6100103b50 0x7f610010c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.700+0000 7f61081f4640 1 --2- 172.21.15.43:0/371614953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6100103230 0x7f6100103610 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.700+0000 7f61081f4640 1 -- 172.21.15.43:0/371614953 >> 172.21.15.43:0/371614953 conn(0x7f6100077fb0 msgr2=0x7f61000783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.700+0000 7f61081f4640 1 -- 172.21.15.43:0/371614953 shutdown_connections 2024-04-21T08:51:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.700+0000 7f61081f4640 1 -- 172.21.15.43:0/371614953 wait complete. 2024-04-21T08:51:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.701+0000 7f61081f4640 1 Processor -- start 2024-04-21T08:51:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.701+0000 7f61081f4640 1 -- start start 2024-04-21T08:51:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.701+0000 7f61081f4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6100103230 0x7f6100198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.701+0000 7f61081f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6100103b50 0x7f6100198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.701+0000 7f61081f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f610010c570 0x7f610019cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.701+0000 7f61081f4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f61001121b0 con 0x7f610010c570 2024-04-21T08:51:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.701+0000 7f61081f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6100112030 con 0x7f6100103b50 2024-04-21T08:51:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.701+0000 7f61081f4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6100112330 con 0x7f6100103230 2024-04-21T08:51:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.701+0000 7f6106791640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f610010c570 0x7f610019cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.701+0000 7f6106791640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f610010c570 0x7f610019cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45136/0 (socket says 172.21.15.43:45136) 2024-04-21T08:51:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.701+0000 7f6106791640 1 -- 172.21.15.43:0/3207501411 learned_addr learned my addr 172.21.15.43:0/3207501411 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:51:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.701+0000 7f610578f640 1 --2- 172.21.15.43:0/3207501411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6100103b50 0x7f6100198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.702+0000 7f6105f90640 1 --2- 172.21.15.43:0/3207501411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6100103230 0x7f6100198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.702+0000 7f6106791640 1 -- 172.21.15.43:0/3207501411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6100103230 msgr2=0x7f6100198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.702+0000 7f6106791640 1 --2- 172.21.15.43:0/3207501411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6100103230 0x7f6100198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.702+0000 7f6106791640 1 -- 172.21.15.43:0/3207501411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6100103b50 msgr2=0x7f6100198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.702+0000 7f6106791640 1 --2- 172.21.15.43:0/3207501411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6100103b50 0x7f6100198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.702+0000 7f6106791640 1 -- 172.21.15.43:0/3207501411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f610019d720 con 0x7f610010c570 2024-04-21T08:51:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.702+0000 7f6106791640 1 --2- 172.21.15.43:0/3207501411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f610010c570 0x7f610019cff0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f60f800ea60 tx=0x7f60f800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.702+0000 7f60f6ffd640 1 -- 172.21.15.43:0/3207501411 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f800ce40 con 0x7f610010c570 2024-04-21T08:51:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.702+0000 7f60f6ffd640 1 -- 172.21.15.43:0/3207501411 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60f8005020 con 0x7f610010c570 2024-04-21T08:51:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.702+0000 7f60f6ffd640 1 -- 172.21.15.43:0/3207501411 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8010670 con 0x7f610010c570 2024-04-21T08:51:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.702+0000 7f61081f4640 1 -- 172.21.15.43:0/3207501411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f610019da10 con 0x7f610010c570 2024-04-21T08:51:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.702+0000 7f61081f4640 1 -- 172.21.15.43:0/3207501411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61001a52a0 con 0x7f610010c570 2024-04-21T08:51:47.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.703+0000 7f61081f4640 1 -- 172.21.15.43:0/3207501411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61000691c0 con 0x7f610010c570 2024-04-21T08:51:47.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.704+0000 7f60f6ffd640 1 -- 172.21.15.43:0/3207501411 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f60f80054b0 con 0x7f610010c570 2024-04-21T08:51:47.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.705+0000 7f60f6ffd640 1 --2- 172.21.15.43:0/3207501411 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f60ec0a3d20 0x7f60ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:47.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.705+0000 7f6105f90640 1 --2- 172.21.15.43:0/3207501411 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f60ec0a3d20 0x7f60ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.705+0000 7f60f6ffd640 1 -- 172.21.15.43:0/3207501411 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f60f80d1340 con 0x7f610010c570 2024-04-21T08:51:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.705+0000 7f6105f90640 1 --2- 172.21.15.43:0/3207501411 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f60ec0a3d20 0x7f60ec0a61e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f60fc000c00 tx=0x7f60fc004040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:47.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.708+0000 7f60f6ffd640 1 -- 172.21.15.43:0/3207501411 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60f8053ff0 con 0x7f610010c570 2024-04-21T08:51:47.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.947+0000 7f61081f4640 1 -- 172.21.15.43:0/3207501411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f6100107160 con 0x7f610010c570 2024-04-21T08:51:47.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.955+0000 7f60f6ffd640 1 -- 172.21.15.43:0/3207501411 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746062 (secure 0 0 0) 0x7f60f809d5c0 con 0x7f610010c570 2024-04-21T08:51:47.964 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:51:47.965 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:51:47.965 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 26, 2024-04-21T08:51:47.965 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:47.965 INFO:teuthology.orchestra.run.smithi043.stdout: "active_gid": 15546, 2024-04-21T08:51:47.965 INFO:teuthology.orchestra.run.smithi043.stdout: "active_name": "x", 2024-04-21T08:51:47.965 INFO:teuthology.orchestra.run.smithi043.stdout: "active_addrs": { 2024-04-21T08:51:47.965 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-04-21T08:51:47.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:47.965 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-04-21T08:51:47.965 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6800", 2024-04-21T08:51:47.965 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1482273906 2024-04-21T08:51:47.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:47.966 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-21T08:51:47.966 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6801", 2024-04-21T08:51:47.966 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1482273906 2024-04-21T08:51:47.966 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:47.966 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-21T08:51:47.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.966 INFO:teuthology.orchestra.run.smithi043.stdout: "active_addr": "172.21.15.43:6801/1482273906", 2024-04-21T08:51:47.966 INFO:teuthology.orchestra.run.smithi043.stdout: "active_change": "2024-04-21T08:38:54.882186+0000", 2024-04-21T08:51:47.966 INFO:teuthology.orchestra.run.smithi043.stdout: "active_mgr_features": 4540701547738038271, 2024-04-21T08:51:47.966 INFO:teuthology.orchestra.run.smithi043.stdout: "available": true, 2024-04-21T08:51:47.966 INFO:teuthology.orchestra.run.smithi043.stdout: "standbys": [ 2024-04-21T08:51:47.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:47.966 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 25048, 2024-04-21T08:51:47.967 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "z", 2024-04-21T08:51:47.967 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_features": 4540701547738038271, 2024-04-21T08:51:47.967 INFO:teuthology.orchestra.run.smithi043.stdout: "available_modules": [ 2024-04-21T08:51:47.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:47.967 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "alerts", 2024-04-21T08:51:47.967 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:47.967 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:47.967 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:47.967 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-21T08:51:47.967 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-21T08:51:47.967 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:47.967 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.967 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.968 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-21T08:51:47.968 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.968 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.968 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:47.968 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How frequently to reexamine health status", 2024-04-21T08:51:47.968 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:47.968 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.968 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.968 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:47.968 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:47.968 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:47.968 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.968 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.969 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:47.969 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.969 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.969 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:47.969 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:47.969 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:47.969 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:47.969 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:47.969 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:47.969 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:47.969 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:47.969 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:47.969 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:47.969 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.969 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.970 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:47.970 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:47.970 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:47.970 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.970 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.970 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:47.970 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.970 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.970 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:47.970 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:47.970 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:47.970 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.971 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.971 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:47.971 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:47.971 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:47.971 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.971 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.971 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:47.971 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.971 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.971 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:47.971 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:47.971 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:47.971 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:47.972 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:47.972 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:47.972 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:47.972 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:47.972 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:47.972 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:47.972 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.972 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.972 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:47.972 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:47.972 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:47.972 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.972 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.973 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:47.973 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.973 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.973 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:47.973 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:47.973 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:47.973 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.973 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.973 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_destination": { 2024-04-21T08:51:47.973 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_destination", 2024-04-21T08:51:47.973 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:47.973 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.973 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.974 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:47.974 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.974 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.974 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:47.974 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email address to send alerts to", 2024-04-21T08:51:47.974 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:47.974 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.974 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.974 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_from_name": { 2024-04-21T08:51:47.974 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_from_name", 2024-04-21T08:51:47.974 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:47.974 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.974 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.975 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "Ceph", 2024-04-21T08:51:47.975 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.975 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.975 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:47.975 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email From: name", 2024-04-21T08:51:47.975 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:47.975 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.975 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.975 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_host": { 2024-04-21T08:51:47.975 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_host", 2024-04-21T08:51:47.975 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:47.975 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.975 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.975 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:47.975 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.975 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.976 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:47.976 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP server", 2024-04-21T08:51:47.976 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:47.976 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.976 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.976 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_password": { 2024-04-21T08:51:47.976 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_password", 2024-04-21T08:51:47.976 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:47.976 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.976 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.976 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:47.976 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.976 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.976 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:47.977 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Password to authenticate with", 2024-04-21T08:51:47.977 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:47.977 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.977 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.977 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_port": { 2024-04-21T08:51:47.977 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_port", 2024-04-21T08:51:47.977 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:47.977 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.977 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.978 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "465", 2024-04-21T08:51:47.978 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.978 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.978 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:47.978 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP port", 2024-04-21T08:51:47.978 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:47.978 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.978 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.978 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_sender": { 2024-04-21T08:51:47.978 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_sender", 2024-04-21T08:51:47.978 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:47.978 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.978 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.979 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:47.979 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.979 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.979 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:47.979 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP envelope sender", 2024-04-21T08:51:47.979 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:47.979 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.979 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.979 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_ssl": { 2024-04-21T08:51:47.979 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_ssl", 2024-04-21T08:51:47.979 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:47.979 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.979 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.980 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:47.980 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.980 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.980 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:47.980 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-21T08:51:47.980 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:47.980 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.980 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.980 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_user": { 2024-04-21T08:51:47.980 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_user", 2024-04-21T08:51:47.980 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:47.980 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.980 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.980 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:47.981 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.981 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.981 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:47.981 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "User to authenticate as", 2024-04-21T08:51:47.981 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:47.981 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.981 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.981 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:47.981 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:47.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:47.981 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "balancer", 2024-04-21T08:51:47.981 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:47.981 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:47.981 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:47.982 INFO:teuthology.orchestra.run.smithi043.stdout: "active": { 2024-04-21T08:51:47.982 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "active", 2024-04-21T08:51:47.982 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:47.982 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.982 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.982 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:47.982 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.982 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.982 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:47.982 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically balance PGs across cluster", 2024-04-21T08:51:47.982 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:47.982 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.982 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.983 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_time": { 2024-04-21T08:51:47.983 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_time", 2024-04-21T08:51:47.983 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:47.983 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.983 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.983 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0000", 2024-04-21T08:51:47.983 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.983 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.983 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:47.983 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "beginning time of day to automatically balance", 2024-04-21T08:51:47.983 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T08:51:47.983 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.983 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.983 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_weekday": { 2024-04-21T08:51:47.984 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_weekday", 2024-04-21T08:51:47.984 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-21T08:51:47.984 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.984 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.984 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-21T08:51:47.984 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-04-21T08:51:47.984 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-04-21T08:51:47.984 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:47.984 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-21T08:51:47.984 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T08:51:47.984 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.984 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.984 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_max_iterations": { 2024-04-21T08:51:47.984 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_max_iterations", 2024-04-21T08:51:47.985 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-21T08:51:47.985 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.985 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.985 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "25", 2024-04-21T08:51:47.985 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-04-21T08:51:47.985 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "250", 2024-04-21T08:51:47.985 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:47.985 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-21T08:51:47.985 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:47.985 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.985 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.985 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_metrics": { 2024-04-21T08:51:47.986 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_metrics", 2024-04-21T08:51:47.986 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:47.986 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.986 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.986 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "pgs,objects,bytes", 2024-04-21T08:51:47.986 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.986 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.986 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:47.986 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-21T08:51:47.986 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-21T08:51:47.986 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.987 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.987 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_step": { 2024-04-21T08:51:47.987 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_step", 2024-04-21T08:51:47.987 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-21T08:51:47.987 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.987 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.987 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.5", 2024-04-21T08:51:47.987 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0.001", 2024-04-21T08:51:47.987 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "0.999", 2024-04-21T08:51:47.987 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:47.987 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "aggressiveness of optimization", 2024-04-21T08:51:47.987 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-21T08:51:47.987 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.987 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.988 INFO:teuthology.orchestra.run.smithi043.stdout: "end_time": { 2024-04-21T08:51:47.988 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_time", 2024-04-21T08:51:47.988 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:47.988 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.988 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.988 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2359", 2024-04-21T08:51:47.988 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.988 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.988 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:47.988 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ending time of day to automatically balance", 2024-04-21T08:51:47.988 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T08:51:47.988 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.989 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.989 INFO:teuthology.orchestra.run.smithi043.stdout: "end_weekday": { 2024-04-21T08:51:47.989 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_weekday", 2024-04-21T08:51:47.989 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-21T08:51:47.989 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.989 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.989 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-21T08:51:47.989 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-04-21T08:51:47.989 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-04-21T08:51:47.989 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:47.989 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-21T08:51:47.989 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T08:51:47.989 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.990 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.990 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:47.990 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:47.990 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:47.990 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.990 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.990 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:47.990 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.990 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.990 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:47.990 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:47.990 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:47.991 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:47.991 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:47.991 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:47.991 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:47.991 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:47.991 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:47.991 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:47.991 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.991 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.991 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:47.991 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:47.991 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:47.991 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.991 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.992 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:47.992 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.992 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.992 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:47.992 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:47.992 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:47.992 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.993 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.993 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:47.993 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:47.993 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:47.993 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.993 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.993 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:47.993 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.993 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.993 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:47.993 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:47.993 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:47.993 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:47.994 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:47.994 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:47.994 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:47.994 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:47.994 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:47.994 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:47.994 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.994 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.994 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:47.994 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:47.994 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:47.994 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.994 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.995 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:47.995 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.995 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.995 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:47.995 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:47.995 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:47.995 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.995 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.995 INFO:teuthology.orchestra.run.smithi043.stdout: "min_score": { 2024-04-21T08:51:47.995 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_score", 2024-04-21T08:51:47.995 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-21T08:51:47.995 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.995 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.996 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-21T08:51:47.996 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.996 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.996 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:47.996 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-21T08:51:47.996 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:47.996 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.996 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.996 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-04-21T08:51:47.996 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-04-21T08:51:47.996 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:47.996 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.996 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.996 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "upmap", 2024-04-21T08:51:47.996 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.997 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.997 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:47.997 INFO:teuthology.orchestra.run.smithi043.stdout: "crush-compat", 2024-04-21T08:51:47.997 INFO:teuthology.orchestra.run.smithi043.stdout: "none", 2024-04-21T08:51:47.997 INFO:teuthology.orchestra.run.smithi043.stdout: "read", 2024-04-21T08:51:47.997 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap", 2024-04-21T08:51:47.997 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap-read" 2024-04-21T08:51:47.997 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:47.997 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Balancer mode", 2024-04-21T08:51:47.997 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:47.997 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.997 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.997 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ids": { 2024-04-21T08:51:47.997 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_ids", 2024-04-21T08:51:47.998 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:47.998 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.998 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.998 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:47.998 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.998 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.998 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:47.998 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-21T08:51:47.998 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:47.998 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.998 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.998 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-21T08:51:47.999 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-21T08:51:47.999 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:47.999 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:47.999 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:47.999 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-21T08:51:47.999 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:47.999 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:47.999 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:47.999 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-21T08:51:47.999 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:47.999 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:47.999 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:47.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:47.999 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_deviation": { 2024-04-21T08:51:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_deviation", 2024-04-21T08:51:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-04-21T08:51:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-04-21T08:51:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-21T08:51:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-21T08:51:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_optimizations": { 2024-04-21T08:51:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_optimizations", 2024-04-21T08:51:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-21T08:51:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-21T08:51:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-21T08:51:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.002 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm", 2024-04-21T08:51:48.002 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.002 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.002 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.002 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_down_multiplier": { 2024-04-21T08:51:48.002 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_down_multiplier", 2024-04-21T08:51:48.002 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-21T08:51:48.002 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.002 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.002 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-04-21T08:51:48.002 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.002 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-21T08:51:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_refresh_rate": { 2024-04-21T08:51:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_refresh_rate", 2024-04-21T08:51:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "20", 2024-04-21T08:51:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-21T08:51:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_starting_port": { 2024-04-21T08:51:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_starting_port", 2024-04-21T08:51:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4721", 2024-04-21T08:51:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-21T08:51:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.005 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.005 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.005 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_ptrace": { 2024-04-21T08:51:48.005 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_ptrace", 2024-04-21T08:51:48.005 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.005 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.005 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.005 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.005 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.005 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-21T08:51:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-21T08:51:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_interval": { 2024-04-21T08:51:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_interval", 2024-04-21T08:51:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-21T08:51:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-21T08:51:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_memory_target_ratio": { 2024-04-21T08:51:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_memory_target_ratio", 2024-04-21T08:51:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-21T08:51:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.7", 2024-04-21T08:51:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-21T08:51:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm_log_destination": { 2024-04-21T08:51:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm_log_destination", 2024-04-21T08:51:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: "file", 2024-04-21T08:51:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: "file,syslog", 2024-04-21T08:51:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: "syslog" 2024-04-21T08:51:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-21T08:51:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: "cgroups_split": { 2024-04-21T08:51:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cgroups_split", 2024-04-21T08:51:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-21T08:51:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: "config_checks_enabled": { 2024-04-21T08:51:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_checks_enabled", 2024-04-21T08:51:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-21T08:51:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "config_dashboard": { 2024-04-21T08:51:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_dashboard", 2024-04-21T08:51:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-21T08:51:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_alertmanager": { 2024-04-21T08:51:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_alertmanager", 2024-04-21T08:51:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-21T08:51:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-04-21T08:51:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_base": { 2024-04-21T08:51:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_base", 2024-04-21T08:51:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-21T08:51:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Container image name, without the tag", 2024-04-21T08:51:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_elasticsearch": { 2024-04-21T08:51:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_elasticsearch", 2024-04-21T08:51:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-21T08:51:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "elasticsearch container image", 2024-04-21T08:51:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_grafana": { 2024-04-21T08:51:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_grafana", 2024-04-21T08:51:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-21T08:51:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-04-21T08:51:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_haproxy": { 2024-04-21T08:51:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_haproxy", 2024-04-21T08:51:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-21T08:51:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "HAproxy container image", 2024-04-21T08:51:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_agent": { 2024-04-21T08:51:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_agent", 2024-04-21T08:51:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-21T08:51:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger agent container image", 2024-04-21T08:51:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_collector": { 2024-04-21T08:51:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_collector", 2024-04-21T08:51:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-21T08:51:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger collector container image", 2024-04-21T08:51:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_query": { 2024-04-21T08:51:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_query", 2024-04-21T08:51:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-21T08:51:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger query container image", 2024-04-21T08:51:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_keepalived": { 2024-04-21T08:51:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_keepalived", 2024-04-21T08:51:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-21T08:51:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Keepalived container image", 2024-04-21T08:51:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_loki": { 2024-04-21T08:51:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_loki", 2024-04-21T08:51:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-21T08:51:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Loki container image", 2024-04-21T08:51:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_node_exporter": { 2024-04-21T08:51:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_node_exporter", 2024-04-21T08:51:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-21T08:51:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-04-21T08:51:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_nvmeof": { 2024-04-21T08:51:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_nvmeof", 2024-04-21T08:51:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-21T08:51:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Nvme-of container image", 2024-04-21T08:51:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_prometheus": { 2024-04-21T08:51:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_prometheus", 2024-04-21T08:51:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-21T08:51:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-04-21T08:51:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_promtail": { 2024-04-21T08:51:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_promtail", 2024-04-21T08:51:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-21T08:51:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Promtail container image", 2024-04-21T08:51:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_samba": { 2024-04-21T08:51:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_samba", 2024-04-21T08:51:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-21T08:51:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Samba/SMB container image", 2024-04-21T08:51:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_snmp_gateway": { 2024-04-21T08:51:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_snmp_gateway", 2024-04-21T08:51:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-21T08:51:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SNMP Gateway container image", 2024-04-21T08:51:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: "container_init": { 2024-04-21T08:51:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_init", 2024-04-21T08:51:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Run podman/docker with `--init`", 2024-04-21T08:51:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: "daemon_cache_timeout": { 2024-04-21T08:51:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "daemon_cache_timeout", 2024-04-21T08:51:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-21T08:51:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-21T08:51:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: "default_cephadm_command_timeout": { 2024-04-21T08:51:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_cephadm_command_timeout", 2024-04-21T08:51:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "900", 2024-04-21T08:51:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-21T08:51:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: "default_registry": { 2024-04-21T08:51:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_registry", 2024-04-21T08:51:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io", 2024-04-21T08:51:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-21T08:51:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: "device_cache_timeout": { 2024-04-21T08:51:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_cache_timeout", 2024-04-21T08:51:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1800", 2024-04-21T08:51:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache device inventory", 2024-04-21T08:51:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: "device_enhanced_scan": { 2024-04-21T08:51:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_enhanced_scan", 2024-04-21T08:51:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-21T08:51:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "facts_cache_timeout": { 2024-04-21T08:51:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "facts_cache_timeout", 2024-04-21T08:51:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-21T08:51:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache host facts data", 2024-04-21T08:51:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: "grafana_dashboards_path": { 2024-04-21T08:51:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "grafana_dashboards_path", 2024-04-21T08:51:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-21T08:51:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-21T08:51:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: "host_check_interval": { 2024-04-21T08:51:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "host_check_interval", 2024-04-21T08:51:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-21T08:51:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to perform a host check", 2024-04-21T08:51:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: "hw_monitoring": { 2024-04-21T08:51:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hw_monitoring", 2024-04-21T08:51:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-21T08:51:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "inventory_list_all": { 2024-04-21T08:51:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "inventory_list_all", 2024-04-21T08:51:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-21T08:51:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: "log_refresh_metadata": { 2024-04-21T08:51:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_refresh_metadata", 2024-04-21T08:51:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-21T08:51:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-21T08:51:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-21T08:51:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-21T08:51:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-21T08:51:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-21T08:51:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-21T08:51:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "*", 2024-04-21T08:51:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-21T08:51:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: "max_count_per_host": { 2024-04-21T08:51:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_count_per_host", 2024-04-21T08:51:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-21T08:51:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of daemons per service per host", 2024-04-21T08:51:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "max_osd_draining_count": { 2024-04-21T08:51:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_osd_draining_count", 2024-04-21T08:51:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-21T08:51:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-21T08:51:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: "migration_current": { 2024-04-21T08:51:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "migration_current", 2024-04-21T08:51:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "internal - do not modify", 2024-04-21T08:51:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-04-21T08:51:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-04-21T08:51:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "root", 2024-04-21T08:51:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm-package", 2024-04-21T08:51:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: "root" 2024-04-21T08:51:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "mode for remote execution of cephadm", 2024-04-21T08:51:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: "oob_default_addr": { 2024-04-21T08:51:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "oob_default_addr", 2024-04-21T08:51:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "169.254.1.1", 2024-04-21T08:51:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-21T08:51:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: "prometheus_alerts_path": { 2024-04-21T08:51:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus_alerts_path", 2024-04-21T08:51:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-21T08:51:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-21T08:51:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_insecure": { 2024-04-21T08:51:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_insecure", 2024-04-21T08:51:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-21T08:51:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_password": { 2024-04-21T08:51:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_password", 2024-04-21T08:51:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-21T08:51:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_url": { 2024-04-21T08:51:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_url", 2024-04-21T08:51:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.054 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.054 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.054 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.054 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-21T08:51:48.054 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.054 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.054 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.054 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_username": { 2024-04-21T08:51:48.054 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_username", 2024-04-21T08:51:48.054 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.054 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.054 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.054 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.054 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.055 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.055 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.055 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-21T08:51:48.055 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.055 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.055 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.055 INFO:teuthology.orchestra.run.smithi043.stdout: "secure_monitoring_stack": { 2024-04-21T08:51:48.055 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "secure_monitoring_stack", 2024-04-21T08:51:48.055 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.055 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.055 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.055 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.055 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.055 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-21T08:51:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "service_discovery_port": { 2024-04-21T08:51:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "service_discovery_port", 2024-04-21T08:51:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8765", 2024-04-21T08:51:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "cephadm service discovery port", 2024-04-21T08:51:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: "ssh_config_file": { 2024-04-21T08:51:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssh_config_file", 2024-04-21T08:51:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-21T08:51:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: "use_agent": { 2024-04-21T08:51:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_agent", 2024-04-21T08:51:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-21T08:51:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: "use_repo_digest": { 2024-04-21T08:51:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_repo_digest", 2024-04-21T08:51:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-21T08:51:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_failed_host_check": { 2024-04-21T08:51:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_failed_host_check", 2024-04-21T08:51:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if the host check fails", 2024-04-21T08:51:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_daemons": { 2024-04-21T08:51:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_daemons", 2024-04-21T08:51:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-21T08:51:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_hosts": { 2024-04-21T08:51:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_hosts", 2024-04-21T08:51:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-21T08:51:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crash", 2024-04-21T08:51:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: "retain_interval": { 2024-04-21T08:51:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retain_interval", 2024-04-21T08:51:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "31536000", 2024-04-21T08:51:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-21T08:51:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_recent_interval": { 2024-04-21T08:51:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_recent_interval", 2024-04-21T08:51:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1209600", 2024-04-21T08:51:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-21T08:51:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dashboard", 2024-04-21T08:51:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-21T08:51:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-21T08:51:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-21T08:51:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_HOST": { 2024-04-21T08:51:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-21T08:51:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-21T08:51:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-21T08:51:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_ENABLED": { 2024-04-21T08:51:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_ENABLED", 2024-04-21T08:51:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-21T08:51:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-21T08:51:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: "ENABLE_BROWSABLE_API": { 2024-04-21T08:51:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-21T08:51:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-21T08:51:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-21T08:51:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-21T08:51:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-21T08:51:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-21T08:51:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-21T08:51:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-21T08:51:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-21T08:51:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-21T08:51:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-21T08:51:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-21T08:51:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-21T08:51:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-21T08:51:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-21T08:51:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-21T08:51:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-21T08:51:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_PASSWORD": { 2024-04-21T08:51:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-21T08:51:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-04-21T08:51:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-21T08:51:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-21T08:51:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_URL": { 2024-04-21T08:51:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_URL", 2024-04-21T08:51:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_USERNAME": { 2024-04-21T08:51:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-21T08:51:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-04-21T08:51:48.086 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.086 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.086 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.086 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.086 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.086 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.086 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.086 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-21T08:51:48.086 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-21T08:51:48.086 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.086 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.086 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-21T08:51:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-21T08:51:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-21T08:51:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-21T08:51:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-21T08:51:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-21T08:51:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-21T08:51:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-21T08:51:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "[]", 2024-04-21T08:51:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: "MULTICLUSTER_CONFIG": { 2024-04-21T08:51:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-21T08:51:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "{}", 2024-04-21T08:51:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_HOST": { 2024-04-21T08:51:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-21T08:51:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-21T08:51:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-21T08:51:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.094 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.094 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.094 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.094 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.094 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.094 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.094 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.094 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.094 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.094 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-21T08:51:48.094 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-21T08:51:48.094 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.094 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.094 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.095 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.095 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.095 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.095 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.095 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.095 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.095 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.095 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.095 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-21T08:51:48.095 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-21T08:51:48.095 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.095 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.095 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.095 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.096 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.096 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.096 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.096 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.096 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.096 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.096 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.096 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-21T08:51:48.096 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-21T08:51:48.096 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.096 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.096 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.096 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.096 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.096 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.097 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.097 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.097 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.097 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.097 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.097 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-21T08:51:48.097 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-21T08:51:48.097 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.097 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.097 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.097 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.097 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.097 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.097 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.097 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.098 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.098 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.098 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.098 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-21T08:51:48.098 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-21T08:51:48.098 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.098 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.098 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.098 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.098 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.098 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.098 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.098 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.098 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.099 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.099 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.099 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-21T08:51:48.099 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-21T08:51:48.099 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.099 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.099 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.099 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.099 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.099 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.099 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.099 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.099 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.099 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.099 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.100 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-21T08:51:48.100 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-21T08:51:48.100 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.100 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.100 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.100 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.100 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.100 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.100 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.100 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.100 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.100 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.100 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.100 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_ENABLED": { 2024-04-21T08:51:48.101 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-21T08:51:48.101 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.101 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.101 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.101 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.101 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.101 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.101 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.101 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.101 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.101 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.101 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.101 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-21T08:51:48.101 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-21T08:51:48.101 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.101 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.102 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.102 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-21T08:51:48.102 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.102 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.102 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.102 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.102 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.102 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.102 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.102 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-21T08:51:48.102 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-21T08:51:48.102 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.102 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.102 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.103 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-21T08:51:48.103 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.103 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.103 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.103 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.103 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.103 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.103 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.103 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-21T08:51:48.103 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-21T08:51:48.103 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.103 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.103 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.103 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8", 2024-04-21T08:51:48.103 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.103 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-21T08:51:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-21T08:51:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "45", 2024-04-21T08:51:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ACCESS_KEY": { 2024-04-21T08:51:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-21T08:51:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-21T08:51:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-21T08:51:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-04-21T08:51:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SECRET_KEY": { 2024-04-21T08:51:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-21T08:51:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SSL_VERIFY": { 2024-04-21T08:51:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-21T08:51:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "UNSAFE_TLS_v1_2": { 2024-04-21T08:51:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-21T08:51:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-21T08:51:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-21T08:51:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-21T08:51:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-21T08:51:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-21T08:51:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-21T08:51:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-21T08:51:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-21T08:51:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-04-21T08:51:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "cross_origin_url": { 2024-04-21T08:51:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cross_origin_url", 2024-04-21T08:51:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "crt_file": { 2024-04-21T08:51:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crt_file", 2024-04-21T08:51:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: "debug": { 2024-04-21T08:51:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "debug", 2024-04-21T08:51:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable/disable debug options", 2024-04-21T08:51:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.115 INFO:teuthology.orchestra.run.smithi043.stdout: "jwt_token_ttl": { 2024-04-21T08:51:48.115 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "jwt_token_ttl", 2024-04-21T08:51:48.115 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.115 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.115 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.115 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "28800", 2024-04-21T08:51:48.115 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.115 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.115 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.115 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.115 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.115 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.115 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.116 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-04-21T08:51:48.116 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-04-21T08:51:48.116 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.116 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.116 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.116 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.116 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.116 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.116 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.116 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.116 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.116 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.116 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.116 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.117 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.117 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.117 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.117 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.117 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.117 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.117 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.117 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.117 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.117 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.117 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.117 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.117 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.117 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.117 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.118 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.118 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.118 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.118 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.118 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.118 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.118 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.118 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.118 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.118 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.118 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.118 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.118 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.119 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.119 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.119 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.119 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.119 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.119 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.119 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.119 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.119 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.119 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.119 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.119 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.119 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.119 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.120 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.120 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.120 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.120 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.120 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.120 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.120 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.120 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.120 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.120 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.120 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.120 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.120 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.120 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.121 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.121 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.121 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.121 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.121 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.121 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.121 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.121 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.121 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.121 INFO:teuthology.orchestra.run.smithi043.stdout: "motd": { 2024-04-21T08:51:48.121 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "motd", 2024-04-21T08:51:48.121 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.121 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.121 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.122 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.122 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.122 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.122 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.122 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "The message of the day", 2024-04-21T08:51:48.122 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.122 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.122 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.122 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect_resolve_ip_addr": { 2024-04-21T08:51:48.122 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "redirect_resolve_ip_addr", 2024-04-21T08:51:48.122 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.122 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.122 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.122 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.123 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.123 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.123 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.123 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.123 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.123 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.123 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.123 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-04-21T08:51:48.123 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-04-21T08:51:48.123 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.123 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.123 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.123 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-04-21T08:51:48.123 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.123 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.124 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.124 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.124 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.124 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.124 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.124 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-04-21T08:51:48.124 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-04-21T08:51:48.124 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.124 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.124 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.124 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8080", 2024-04-21T08:51:48.124 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.124 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.124 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.125 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.125 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.125 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.125 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.125 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-04-21T08:51:48.125 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-04-21T08:51:48.125 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.125 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.125 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.125 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.125 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.125 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.125 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.125 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.126 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.126 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.126 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.126 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl_server_port": { 2024-04-21T08:51:48.126 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl_server_port", 2024-04-21T08:51:48.126 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.126 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.126 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.126 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8443", 2024-04-21T08:51:48.126 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.126 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.126 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.126 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.126 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.127 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.127 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.127 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-04-21T08:51:48.127 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-04-21T08:51:48.127 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.127 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.127 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.127 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "redirect", 2024-04-21T08:51:48.127 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.127 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.127 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.127 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.127 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect" 2024-04-21T08:51:48.127 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.127 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.127 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.128 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.128 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.128 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-04-21T08:51:48.128 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-04-21T08:51:48.128 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.128 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.128 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.128 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-04-21T08:51:48.128 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-04-21T08:51:48.128 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-04-21T08:51:48.128 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.128 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.128 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.128 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.128 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.129 INFO:teuthology.orchestra.run.smithi043.stdout: "url_prefix": { 2024-04-21T08:51:48.129 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url_prefix", 2024-04-21T08:51:48.129 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.129 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.129 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.129 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.129 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.129 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.129 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.129 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.129 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.129 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.129 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.129 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.129 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.130 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "devicehealth", 2024-04-21T08:51:48.130 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.130 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.130 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.130 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_monitoring": { 2024-04-21T08:51:48.130 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_monitoring", 2024-04-21T08:51:48.130 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.130 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.130 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.130 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.130 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.130 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.130 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.130 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "monitor device health metrics", 2024-04-21T08:51:48.130 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.131 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.131 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.131 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.131 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.131 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.131 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.131 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.131 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.131 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.131 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.131 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.131 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.131 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.131 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.131 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.132 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.132 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.132 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.132 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.132 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.132 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.132 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.132 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.132 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.132 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.132 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.132 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.132 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.132 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.132 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.133 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.133 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.133 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.133 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.133 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.133 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.133 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.133 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.133 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.133 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.133 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.133 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.133 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.133 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.134 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.134 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.134 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.134 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.134 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.134 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.134 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.134 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.134 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.134 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.134 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.134 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.134 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.135 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.135 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.135 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.135 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.135 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.135 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.135 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.135 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.135 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.135 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.135 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.135 INFO:teuthology.orchestra.run.smithi043.stdout: "mark_out_threshold": { 2024-04-21T08:51:48.135 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mark_out_threshold", 2024-04-21T08:51:48.135 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.135 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.136 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.136 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2419200", 2024-04-21T08:51:48.136 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.136 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.136 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.136 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-21T08:51:48.136 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.136 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.136 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.136 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": { 2024-04-21T08:51:48.136 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_name", 2024-04-21T08:51:48.136 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.137 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.137 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.137 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "device_health_metrics", 2024-04-21T08:51:48.137 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.137 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.137 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.137 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-21T08:51:48.137 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.137 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.137 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.137 INFO:teuthology.orchestra.run.smithi043.stdout: "retention_period": { 2024-04-21T08:51:48.137 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retention_period", 2024-04-21T08:51:48.137 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.137 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.138 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.138 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15552000", 2024-04-21T08:51:48.138 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.138 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.138 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.138 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain device health metrics", 2024-04-21T08:51:48.138 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.138 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.138 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.138 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_frequency": { 2024-04-21T08:51:48.138 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_frequency", 2024-04-21T08:51:48.138 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.138 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.139 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.139 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-04-21T08:51:48.139 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.139 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.139 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.139 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-21T08:51:48.139 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.139 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.139 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.139 INFO:teuthology.orchestra.run.smithi043.stdout: "self_heal": { 2024-04-21T08:51:48.139 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "self_heal", 2024-04-21T08:51:48.139 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.139 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.139 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.140 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.140 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.140 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.140 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.140 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-21T08:51:48.140 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.140 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.140 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.140 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-21T08:51:48.140 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-21T08:51:48.140 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.140 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.140 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.140 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-21T08:51:48.140 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.141 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.141 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.141 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and check device health", 2024-04-21T08:51:48.141 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.141 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.141 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.141 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_threshold": { 2024-04-21T08:51:48.141 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_threshold", 2024-04-21T08:51:48.141 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.141 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.141 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.141 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7257600", 2024-04-21T08:51:48.141 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.141 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.141 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.142 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-21T08:51:48.142 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.142 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.142 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.142 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.142 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.142 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "diskprediction_local", 2024-04-21T08:51:48.142 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.142 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.142 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.142 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.142 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.142 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.143 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.143 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.143 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.143 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.143 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.143 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.143 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.143 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.143 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.143 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.143 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.143 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.143 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.143 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.143 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.143 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.144 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.144 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.144 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.144 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.144 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.144 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.144 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.144 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.144 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.144 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.144 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.144 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.144 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.144 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.145 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.145 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.145 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.145 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.145 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.145 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.145 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.145 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.145 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.145 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.145 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.145 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.145 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.145 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.145 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.146 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.146 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.146 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.146 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.146 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.146 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.146 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.146 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.146 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.146 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.146 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.146 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.146 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.146 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.147 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.147 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.147 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.147 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.147 INFO:teuthology.orchestra.run.smithi043.stdout: "predict_interval": { 2024-04-21T08:51:48.147 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predict_interval", 2024-04-21T08:51:48.147 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.147 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.147 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.147 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-04-21T08:51:48.147 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.147 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.147 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.147 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.147 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.148 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.148 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.148 INFO:teuthology.orchestra.run.smithi043.stdout: "predictor_model": { 2024-04-21T08:51:48.148 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predictor_model", 2024-04-21T08:51:48.148 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.148 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.148 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.148 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "prophetstor", 2024-04-21T08:51:48.148 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.148 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.148 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.148 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.148 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.149 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.149 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.149 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-21T08:51:48.149 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-21T08:51:48.149 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.149 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.149 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.149 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-21T08:51:48.149 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.149 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.149 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.149 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.149 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.149 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.149 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.150 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.150 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.150 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "influx", 2024-04-21T08:51:48.150 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": false, 2024-04-21T08:51:48.150 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "influxdb python module not found", 2024-04-21T08:51:48.150 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.150 INFO:teuthology.orchestra.run.smithi043.stdout: "batch_size": { 2024-04-21T08:51:48.150 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "batch_size", 2024-04-21T08:51:48.150 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.150 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.150 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.150 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5000", 2024-04-21T08:51:48.151 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.151 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.151 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.151 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-21T08:51:48.151 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.151 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.151 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.151 INFO:teuthology.orchestra.run.smithi043.stdout: "database": { 2024-04-21T08:51:48.151 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "database", 2024-04-21T08:51:48.151 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.151 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.151 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.151 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "ceph", 2024-04-21T08:51:48.151 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.152 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.152 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.152 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-21T08:51:48.152 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.152 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.152 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.152 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": { 2024-04-21T08:51:48.152 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hostname", 2024-04-21T08:51:48.152 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.152 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.152 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.152 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.152 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.152 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.153 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.153 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server hostname", 2024-04-21T08:51:48.153 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.153 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.153 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.153 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-21T08:51:48.153 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-21T08:51:48.153 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.153 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.153 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.153 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "30", 2024-04-21T08:51:48.153 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "5", 2024-04-21T08:51:48.153 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.153 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.153 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-21T08:51:48.154 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.154 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.154 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.154 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.154 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.154 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.154 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.154 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.154 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.154 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.154 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.154 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.154 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.154 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.155 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.155 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.155 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.155 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.155 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.155 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.155 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.155 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.155 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.155 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.155 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.155 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.155 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.155 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.155 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.156 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.156 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.156 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.156 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.156 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.156 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.156 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.156 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.156 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.156 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.156 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.156 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.156 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.156 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.156 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.157 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.157 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.157 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.157 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.157 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.157 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.157 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.157 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.157 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.157 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.157 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.157 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.157 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.157 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.157 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.157 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.158 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.158 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.158 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.158 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.158 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.158 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.158 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.158 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.158 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.158 INFO:teuthology.orchestra.run.smithi043.stdout: "password": { 2024-04-21T08:51:48.158 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "password", 2024-04-21T08:51:48.158 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.158 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.158 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.159 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.159 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.159 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.159 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.159 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "password of InfluxDB server user", 2024-04-21T08:51:48.159 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.159 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.159 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.159 INFO:teuthology.orchestra.run.smithi043.stdout: "port": { 2024-04-21T08:51:48.159 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "port", 2024-04-21T08:51:48.159 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.159 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.159 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.159 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8086", 2024-04-21T08:51:48.160 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.160 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.160 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.160 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server port", 2024-04-21T08:51:48.160 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.160 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.160 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.160 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-04-21T08:51:48.160 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-04-21T08:51:48.160 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.160 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.160 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.160 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "false", 2024-04-21T08:51:48.160 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.161 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.161 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.161 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T08:51:48.161 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.161 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.161 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.161 INFO:teuthology.orchestra.run.smithi043.stdout: "threads": { 2024-04-21T08:51:48.161 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threads", 2024-04-21T08:51:48.161 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.161 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.161 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.161 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-04-21T08:51:48.161 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-04-21T08:51:48.161 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "32", 2024-04-21T08:51:48.161 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.162 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-21T08:51:48.162 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.162 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.162 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.162 INFO:teuthology.orchestra.run.smithi043.stdout: "username": { 2024-04-21T08:51:48.162 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "username", 2024-04-21T08:51:48.162 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.162 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.162 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.162 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.162 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.162 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.162 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.163 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "username of InfluxDB server user", 2024-04-21T08:51:48.163 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.163 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.163 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.163 INFO:teuthology.orchestra.run.smithi043.stdout: "verify_ssl": { 2024-04-21T08:51:48.163 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "verify_ssl", 2024-04-21T08:51:48.163 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.163 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.163 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.163 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "true", 2024-04-21T08:51:48.163 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.163 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.163 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.163 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T08:51:48.164 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.164 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.164 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.164 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.164 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.164 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "insights", 2024-04-21T08:51:48.164 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.164 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.164 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.164 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.164 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.164 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.164 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.164 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.165 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.165 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.165 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.165 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.165 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.165 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.165 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.165 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.165 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.165 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.165 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.165 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.165 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.165 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.165 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.166 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.166 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.166 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.166 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.166 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.166 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.166 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.166 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.166 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.166 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.166 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.166 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.166 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.167 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.167 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.167 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.167 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.167 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.167 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.167 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.167 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.167 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.167 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.167 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.167 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.167 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.167 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.167 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.168 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.168 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.168 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.168 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.168 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.168 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.168 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.168 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.168 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.168 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.168 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.168 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.168 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.168 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.169 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.169 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.169 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.169 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.169 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.169 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.169 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "iostat", 2024-04-21T08:51:48.169 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.169 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.169 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.169 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.169 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.170 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.170 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.170 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.170 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.170 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.170 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.170 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.170 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.170 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.170 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.170 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.170 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.170 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.170 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.171 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.171 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.171 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.171 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.171 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.171 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.171 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.171 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.171 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.171 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.171 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.171 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.171 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.172 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.172 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.172 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.172 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.172 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.172 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.172 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.172 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.172 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.172 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.172 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.172 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.172 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.173 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.173 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.173 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.173 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.173 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.173 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.173 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.173 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.173 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.173 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.173 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.173 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.173 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.173 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.174 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.174 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.174 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.174 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.174 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.174 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.174 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.174 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.174 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.174 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.174 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.174 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.175 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "k8sevents", 2024-04-21T08:51:48.175 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.175 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.175 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.175 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_event_retention_days": { 2024-04-21T08:51:48.175 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ceph_event_retention_days", 2024-04-21T08:51:48.175 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.175 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.175 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.175 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7", 2024-04-21T08:51:48.175 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.175 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.176 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.176 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-21T08:51:48.176 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.176 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.176 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.176 INFO:teuthology.orchestra.run.smithi043.stdout: "config_check_secs": { 2024-04-21T08:51:48.176 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_check_secs", 2024-04-21T08:51:48.176 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.176 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.176 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.176 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-21T08:51:48.176 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "10", 2024-04-21T08:51:48.176 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.177 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.177 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-21T08:51:48.177 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.177 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.177 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.177 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.177 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.177 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.177 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.177 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.177 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.177 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.177 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.177 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.178 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.178 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.178 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.178 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.178 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.178 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.178 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.178 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.178 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.178 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.178 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.178 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.179 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.179 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.179 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.179 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.179 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.179 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.179 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.179 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.179 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.179 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.179 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.179 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.179 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.180 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.180 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.180 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.180 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.180 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.180 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.180 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.180 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.180 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.180 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.180 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.180 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.180 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.180 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.180 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.181 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.181 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.181 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.181 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.181 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.181 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.181 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.181 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.181 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.181 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.181 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.181 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.181 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.181 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.181 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.182 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.182 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.182 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.182 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.182 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "localpool", 2024-04-21T08:51:48.182 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.182 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.182 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.182 INFO:teuthology.orchestra.run.smithi043.stdout: "failure_domain": { 2024-04-21T08:51:48.182 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "failure_domain", 2024-04-21T08:51:48.182 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.182 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.183 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.183 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "host", 2024-04-21T08:51:48.183 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.183 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.183 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.183 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "failure domain for any created local pool", 2024-04-21T08:51:48.183 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-21T08:51:48.183 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.183 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.183 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.183 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.183 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.183 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.184 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.184 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.184 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.184 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.184 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.184 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.184 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.184 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.184 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.184 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.184 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.184 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.184 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.184 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.184 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.184 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.185 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.185 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.185 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.185 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.185 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.185 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.185 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.185 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.185 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.185 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.185 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.185 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.185 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.186 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.186 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.186 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.186 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.186 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.186 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.186 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.186 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.186 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.186 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.186 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.186 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.186 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.186 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.187 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.187 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.187 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.187 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.187 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.187 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.187 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.187 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.187 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.187 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.187 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.187 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.187 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.188 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.188 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.188 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.188 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.188 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.188 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.188 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": { 2024-04-21T08:51:48.188 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_size", 2024-04-21T08:51:48.188 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.188 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.188 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.188 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.188 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.188 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.189 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.189 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default min_size for any created local pool", 2024-04-21T08:51:48.189 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-21T08:51:48.189 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.189 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.189 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rep": { 2024-04-21T08:51:48.189 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "num_rep", 2024-04-21T08:51:48.189 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.189 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.189 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.189 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3", 2024-04-21T08:51:48.189 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.189 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.189 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.190 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default replica count for any created local pool", 2024-04-21T08:51:48.190 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.190 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.190 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.190 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": { 2024-04-21T08:51:48.190 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_num", 2024-04-21T08:51:48.190 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.190 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.190 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.190 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "128", 2024-04-21T08:51:48.190 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.190 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.191 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.191 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default pg_num for any created local pool", 2024-04-21T08:51:48.191 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.191 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.191 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.191 INFO:teuthology.orchestra.run.smithi043.stdout: "prefix": { 2024-04-21T08:51:48.191 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prefix", 2024-04-21T08:51:48.191 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.191 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.191 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.191 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.191 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.191 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.191 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.192 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name prefix for any created local pool", 2024-04-21T08:51:48.192 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.192 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.192 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.192 INFO:teuthology.orchestra.run.smithi043.stdout: "subtree": { 2024-04-21T08:51:48.192 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "subtree", 2024-04-21T08:51:48.192 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.192 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.192 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.192 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "rack", 2024-04-21T08:51:48.192 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.192 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.192 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.193 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-21T08:51:48.193 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-21T08:51:48.193 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.193 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.193 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.193 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.193 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mds_autoscaler", 2024-04-21T08:51:48.193 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.193 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.193 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.193 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.193 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.194 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.194 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.194 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.194 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.194 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.194 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.194 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.194 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.194 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.194 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.194 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.194 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.194 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.194 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.194 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.195 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.195 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.195 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.195 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.195 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.195 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.195 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.195 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.195 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.195 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.195 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.195 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.195 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.195 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.196 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.196 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.196 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.196 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.196 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.196 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.196 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.196 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.196 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.196 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.196 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.196 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.196 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.196 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.197 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.197 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.197 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.197 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.197 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.197 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.197 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.197 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.197 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.197 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.197 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.197 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.197 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.197 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.198 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.198 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.198 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.198 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.198 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.198 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.198 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.198 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.198 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.198 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirroring", 2024-04-21T08:51:48.198 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.198 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.198 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.199 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.199 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.199 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.199 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.199 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.199 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.199 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.199 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.199 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.199 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.199 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.199 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.199 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.199 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.199 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.200 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.200 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.200 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.200 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.200 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.200 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.200 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.200 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.200 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.200 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.200 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.201 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.201 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.201 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.201 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.201 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.201 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.201 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.201 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.201 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.201 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.201 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.201 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.201 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.201 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.202 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.202 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.202 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.202 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.202 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.202 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.202 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.202 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.202 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.202 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.202 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.202 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.202 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.203 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.203 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.203 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.203 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.203 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.203 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.203 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.203 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.203 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.203 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.203 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.203 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.203 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.203 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.203 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.204 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "nfs", 2024-04-21T08:51:48.204 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.204 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.204 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.204 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.204 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.204 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.204 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.204 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.204 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.204 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.204 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.204 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.204 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.205 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.205 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.205 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.205 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.205 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.205 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.205 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.205 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.205 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.205 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.205 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.205 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.205 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.206 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.206 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.206 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.206 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.206 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.206 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.206 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.206 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.206 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.206 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.206 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.206 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.206 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.206 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.207 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.207 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.207 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.207 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.207 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.207 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.207 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.207 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.207 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.207 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.207 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.207 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.207 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.207 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.208 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.208 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.208 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.208 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.208 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.208 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.208 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.208 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.208 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.208 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.208 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.208 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.208 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.208 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.209 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.209 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.209 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.209 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-04-21T08:51:48.209 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.209 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.209 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.209 INFO:teuthology.orchestra.run.smithi043.stdout: "fail_fs": { 2024-04-21T08:51:48.209 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "fail_fs", 2024-04-21T08:51:48.209 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.209 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.209 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.209 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.210 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.210 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.210 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.210 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-21T08:51:48.210 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.210 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.210 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.210 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.210 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.210 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.210 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.210 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.210 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.211 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.211 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.211 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.211 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.211 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.211 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.211 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.211 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.211 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.211 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.211 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.211 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.211 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.211 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.212 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.212 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.212 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.212 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.212 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.212 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.212 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.212 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.212 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.212 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.212 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.212 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.212 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.212 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.213 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.213 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.213 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.213 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.213 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.213 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.213 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.213 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.213 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.213 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.213 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.213 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.213 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.214 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.214 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.214 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.214 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.214 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.214 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.214 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.214 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.214 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.214 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.214 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.214 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.214 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.215 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.215 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.215 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.215 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.215 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.215 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.215 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator": { 2024-04-21T08:51:48.215 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-04-21T08:51:48.215 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.215 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.215 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.215 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.215 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.216 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.216 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.216 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm", 2024-04-21T08:51:48.216 INFO:teuthology.orchestra.run.smithi043.stdout: "rook", 2024-04-21T08:51:48.216 INFO:teuthology.orchestra.run.smithi043.stdout: "test_orchestrator" 2024-04-21T08:51:48.216 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.216 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Orchestrator backend", 2024-04-21T08:51:48.216 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.216 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.216 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.216 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.216 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.217 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_perf_query", 2024-04-21T08:51:48.217 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.217 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.217 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.217 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.217 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.217 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.217 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.217 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.217 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.217 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.217 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.217 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.217 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.217 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.218 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.218 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.218 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.218 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.218 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.218 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.218 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.218 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.218 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.218 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.218 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.218 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.218 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.219 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.219 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.219 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.219 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.219 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.219 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.219 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.219 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.219 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.219 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.219 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.219 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.219 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.220 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.220 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.220 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.220 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.220 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.220 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.220 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.220 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.220 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.220 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.220 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.220 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.220 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.220 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.220 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.221 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.221 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.221 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.221 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.221 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.221 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.221 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.221 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.221 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.221 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.221 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.221 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.221 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.221 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.222 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.222 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.222 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_support", 2024-04-21T08:51:48.222 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.222 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.222 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.222 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.222 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.222 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.222 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.222 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.222 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.222 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.223 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.223 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.223 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.223 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.223 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.223 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.223 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.223 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.223 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.223 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.223 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.223 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.223 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.223 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.224 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.224 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.224 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.224 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.224 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.224 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.224 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.224 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.224 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.224 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.224 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.224 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.224 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.224 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.224 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.225 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.225 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.225 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.225 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.225 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.225 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.225 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.225 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.225 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.225 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.225 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.225 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.225 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.225 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.226 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.226 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.226 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.226 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.226 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.226 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.226 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.226 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.226 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.226 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.226 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.226 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.226 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.226 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.227 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.227 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.227 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.227 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.227 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_autoscaler", 2024-04-21T08:51:48.227 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.227 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.227 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.227 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.227 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.227 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.227 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.228 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.228 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.228 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.228 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.228 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.228 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.228 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.228 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.228 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.228 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.228 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.228 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.228 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.229 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.229 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.229 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.229 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.229 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.229 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.229 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.229 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.229 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.229 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.229 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.229 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.229 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.229 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.230 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.230 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.230 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.230 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.230 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.230 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.230 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.230 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.230 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.230 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.230 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.230 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.230 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.231 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.231 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.231 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.231 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.231 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.231 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.231 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.231 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.231 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.231 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.231 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.231 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.231 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.232 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.232 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.232 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.232 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.232 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.232 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.232 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.232 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.232 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.232 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-21T08:51:48.232 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-21T08:51:48.232 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.232 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.233 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.233 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-21T08:51:48.233 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.233 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.233 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.233 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.233 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.233 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.233 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.233 INFO:teuthology.orchestra.run.smithi043.stdout: "threshold": { 2024-04-21T08:51:48.233 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threshold", 2024-04-21T08:51:48.233 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-21T08:51:48.233 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.233 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.234 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-04-21T08:51:48.234 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1.0", 2024-04-21T08:51:48.234 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.234 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.234 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "scaling threshold", 2024-04-21T08:51:48.234 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-21T08:51:48.234 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.234 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.234 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.234 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.234 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "progress", 2024-04-21T08:51:48.234 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.234 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.235 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.235 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_pg_recovery_event": { 2024-04-21T08:51:48.235 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_pg_recovery_event", 2024-04-21T08:51:48.235 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.235 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.235 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.235 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.235 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.235 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.235 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.235 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-21T08:51:48.235 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.235 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.235 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.236 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-04-21T08:51:48.236 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-04-21T08:51:48.236 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.236 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.236 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.236 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.236 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.236 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.236 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.236 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.236 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.236 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.236 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.237 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.237 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.237 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.237 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.237 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.237 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.237 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.237 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.237 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.237 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.237 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.237 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.237 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.237 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.237 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.240 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.240 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.240 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.240 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.240 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.240 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.240 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.240 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.240 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.240 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.240 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.240 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.240 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.241 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.241 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.241 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.241 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.241 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.241 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.241 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.241 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.241 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.241 INFO:teuthology.orchestra.run.smithi043.stdout: "max_completed_events": { 2024-04-21T08:51:48.241 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_completed_events", 2024-04-21T08:51:48.241 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.241 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.242 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.242 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "50", 2024-04-21T08:51:48.242 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.242 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.242 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.242 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "number of past completed events to remember", 2024-04-21T08:51:48.242 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.242 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.242 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.242 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-21T08:51:48.242 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-21T08:51:48.242 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.242 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.243 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.243 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-04-21T08:51:48.243 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.243 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.243 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.243 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long the module is going to sleep", 2024-04-21T08:51:48.243 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.243 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.243 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.243 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.243 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.243 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus", 2024-04-21T08:51:48.243 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.244 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.244 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.244 INFO:teuthology.orchestra.run.smithi043.stdout: "cache": { 2024-04-21T08:51:48.244 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cache", 2024-04-21T08:51:48.244 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.244 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.244 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.244 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.244 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.244 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.244 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.244 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.244 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.244 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.244 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.245 INFO:teuthology.orchestra.run.smithi043.stdout: "exclude_perf_counters": { 2024-04-21T08:51:48.245 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "exclude_perf_counters", 2024-04-21T08:51:48.245 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.245 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.245 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.245 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.245 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.245 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.245 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.245 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-21T08:51:48.245 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-21T08:51:48.245 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.245 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.245 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.245 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.246 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.246 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.246 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.246 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.246 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.246 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.246 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.246 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.246 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.246 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.246 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.246 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.246 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.246 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.247 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.247 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.247 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.247 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.247 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.247 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.247 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.247 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.247 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.247 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.247 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.247 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.247 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.247 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.247 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.248 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.248 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.248 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.248 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.248 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.248 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.248 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.248 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.248 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.248 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.248 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.248 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.248 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.248 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.249 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.249 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.249 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.249 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.249 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.249 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.249 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.249 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.249 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.249 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.249 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.249 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.249 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.249 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.250 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.250 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.250 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.250 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.250 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.250 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.250 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.250 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools": { 2024-04-21T08:51:48.250 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools", 2024-04-21T08:51:48.250 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.250 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.250 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.250 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.251 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.251 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.251 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.251 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.251 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.251 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.251 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.251 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-21T08:51:48.251 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-21T08:51:48.251 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.251 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.251 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.251 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "300", 2024-04-21T08:51:48.251 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.252 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.252 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.252 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.252 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.252 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.252 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.252 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_interval": { 2024-04-21T08:51:48.252 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_interval", 2024-04-21T08:51:48.252 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-21T08:51:48.252 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.252 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.253 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15.0", 2024-04-21T08:51:48.253 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.253 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.253 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.253 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.253 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.253 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.253 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.253 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-04-21T08:51:48.253 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-04-21T08:51:48.253 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.253 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.253 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.253 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-04-21T08:51:48.253 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.254 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.254 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.254 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-21T08:51:48.254 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.254 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.254 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.254 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-04-21T08:51:48.254 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-04-21T08:51:48.254 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.254 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.254 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.254 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "9283", 2024-04-21T08:51:48.255 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.255 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.255 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.255 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-21T08:51:48.255 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.255 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.255 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.255 INFO:teuthology.orchestra.run.smithi043.stdout: "stale_cache_strategy": { 2024-04-21T08:51:48.255 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stale_cache_strategy", 2024-04-21T08:51:48.255 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.255 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.255 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.255 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "log", 2024-04-21T08:51:48.255 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.255 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.256 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.256 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.256 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.256 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.256 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.256 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-04-21T08:51:48.256 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-04-21T08:51:48.256 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.256 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.256 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.256 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "default", 2024-04-21T08:51:48.256 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.256 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.257 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.257 INFO:teuthology.orchestra.run.smithi043.stdout: "default", 2024-04-21T08:51:48.257 INFO:teuthology.orchestra.run.smithi043.stdout: "error" 2024-04-21T08:51:48.257 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.257 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.257 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.257 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.257 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.257 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-04-21T08:51:48.257 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-04-21T08:51:48.257 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.257 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.257 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.257 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-04-21T08:51:48.258 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-04-21T08:51:48.258 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-04-21T08:51:48.258 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.258 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.258 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.258 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.258 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.258 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.258 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.258 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_support", 2024-04-21T08:51:48.258 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.258 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.259 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.259 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.259 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.259 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.259 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.259 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.259 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.259 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.259 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.259 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.259 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.259 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.259 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.259 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.260 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.260 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.260 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.260 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.260 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.260 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.260 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.260 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.260 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.260 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.260 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.260 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.260 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.260 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.261 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.261 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.261 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.261 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.261 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.261 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.261 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.261 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.261 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.261 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.261 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.261 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.261 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.262 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.262 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.262 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.262 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.262 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.262 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.262 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.262 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.262 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.262 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.262 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.262 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.262 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.262 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.263 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.263 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.263 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.263 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.263 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.263 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.263 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.263 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.263 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.263 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.263 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.263 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.263 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_snap_create": { 2024-04-21T08:51:48.263 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_snap_create", 2024-04-21T08:51:48.264 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.264 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.264 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.264 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-21T08:51:48.264 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.264 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.264 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.264 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.264 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.264 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.264 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.264 INFO:teuthology.orchestra.run.smithi043.stdout: "mirror_snapshot_schedule": { 2024-04-21T08:51:48.264 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirror_snapshot_schedule", 2024-04-21T08:51:48.264 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.265 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.265 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.265 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.265 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.265 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.265 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.265 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.265 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.265 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.265 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.265 INFO:teuthology.orchestra.run.smithi043.stdout: "trash_purge_schedule": { 2024-04-21T08:51:48.266 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "trash_purge_schedule", 2024-04-21T08:51:48.266 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.266 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.266 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.266 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.266 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.266 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.266 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.266 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.266 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.266 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.266 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.266 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.266 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.267 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "restful", 2024-04-21T08:51:48.267 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.267 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.267 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.267 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_auth": { 2024-04-21T08:51:48.267 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_auth", 2024-04-21T08:51:48.267 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.267 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.267 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.267 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.267 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.267 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.267 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.268 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.268 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.268 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.268 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.268 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-04-21T08:51:48.268 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-04-21T08:51:48.268 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.268 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.268 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.268 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.268 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.268 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.268 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.268 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.269 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.269 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.269 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.269 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.269 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.269 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.269 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.269 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.269 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.269 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.269 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.269 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.269 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.269 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.270 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.270 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.270 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.270 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.270 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.270 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.270 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.270 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.270 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.270 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.270 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.270 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.270 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.271 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.271 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.271 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.271 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.271 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.271 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.271 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.271 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.271 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.271 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.271 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.271 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.271 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.272 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.272 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.272 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.272 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.272 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.272 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.272 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.272 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.272 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.272 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.272 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.272 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.272 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.272 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.273 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.273 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.273 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.273 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.273 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.273 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.273 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.273 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.273 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.273 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.273 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.273 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.273 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.273 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.274 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.274 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-04-21T08:51:48.274 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-04-21T08:51:48.274 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.274 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.274 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.274 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.274 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.274 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.274 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.274 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.274 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.274 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.275 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.275 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-04-21T08:51:48.275 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-04-21T08:51:48.275 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.275 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.275 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.275 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.275 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.275 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.275 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.275 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.275 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.275 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.275 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.275 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.276 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.276 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rgw", 2024-04-21T08:51:48.276 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.276 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.276 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.276 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.276 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.276 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.276 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.276 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.276 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.276 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.276 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.277 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.277 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.277 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.277 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.277 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.277 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.277 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.277 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.277 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.277 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.277 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.277 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.277 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.277 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.278 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.278 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.278 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.278 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.278 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.278 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.278 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.278 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.278 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.278 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.278 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.278 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.278 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.278 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.279 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.279 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.279 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.279 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.279 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.279 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.279 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.279 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.279 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.279 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.279 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.279 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.279 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.280 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.280 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.280 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.280 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.280 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.280 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.280 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.280 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.280 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.280 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.280 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.280 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.280 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.281 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.281 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.281 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.281 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.281 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.281 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.281 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rook", 2024-04-21T08:51:48.281 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.281 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.281 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.281 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.281 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.281 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.281 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "storage_class": { 2024-04-21T08:51:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "storage_class", 2024-04-21T08:51:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "local", 2024-04-21T08:51:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-21T08:51:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "selftest", 2024-04-21T08:51:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "roption1": { 2024-04-21T08:51:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption1", 2024-04-21T08:51:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.293 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.293 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.293 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.293 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.293 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.293 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.293 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.293 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.293 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.293 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.293 INFO:teuthology.orchestra.run.smithi043.stdout: "roption2": { 2024-04-21T08:51:48.293 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption2", 2024-04-21T08:51:48.294 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.294 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.294 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.294 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "xyz", 2024-04-21T08:51:48.294 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.294 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.294 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.294 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.294 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.294 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.294 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.294 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption1": { 2024-04-21T08:51:48.294 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption1", 2024-04-21T08:51:48.295 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.295 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.295 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.295 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.295 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.295 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.295 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.295 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.295 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.295 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.295 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.295 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption2": { 2024-04-21T08:51:48.295 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption2", 2024-04-21T08:51:48.295 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.296 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.296 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.296 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.296 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.296 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.296 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.296 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.296 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.296 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.296 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.296 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption3": { 2024-04-21T08:51:48.296 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption3", 2024-04-21T08:51:48.296 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-21T08:51:48.297 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.297 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.297 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.297 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.297 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.297 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.297 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.297 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.297 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.297 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.297 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption4": { 2024-04-21T08:51:48.297 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption4", 2024-04-21T08:51:48.297 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.297 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.297 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.298 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.298 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.298 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.298 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.298 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.298 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.298 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.298 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.298 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption5": { 2024-04-21T08:51:48.298 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption5", 2024-04-21T08:51:48.298 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.298 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.298 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.299 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.299 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.299 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.299 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.299 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.299 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.299 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.299 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.299 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption6": { 2024-04-21T08:51:48.299 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption6", 2024-04-21T08:51:48.299 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.299 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.299 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.299 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.300 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.300 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.300 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.300 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.300 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.300 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.300 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.300 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption7": { 2024-04-21T08:51:48.300 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption7", 2024-04-21T08:51:48.300 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.300 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.300 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.300 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.301 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-04-21T08:51:48.301 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "42", 2024-04-21T08:51:48.301 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.301 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.301 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.301 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.301 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.301 INFO:teuthology.orchestra.run.smithi043.stdout: "testkey": { 2024-04-21T08:51:48.301 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testkey", 2024-04-21T08:51:48.301 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.301 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.301 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.301 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.301 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.301 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.301 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.302 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.302 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.302 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.302 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.302 INFO:teuthology.orchestra.run.smithi043.stdout: "testlkey": { 2024-04-21T08:51:48.302 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testlkey", 2024-04-21T08:51:48.302 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.302 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.302 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.302 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.302 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.302 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.302 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.302 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.303 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.303 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.303 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.303 INFO:teuthology.orchestra.run.smithi043.stdout: "testnewline": { 2024-04-21T08:51:48.303 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testnewline", 2024-04-21T08:51:48.303 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.303 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.303 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.303 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.303 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.303 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.303 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.303 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.303 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.304 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.304 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.304 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.304 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.304 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snap_schedule", 2024-04-21T08:51:48.304 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.304 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.304 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.304 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_m_granularity": { 2024-04-21T08:51:48.304 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_m_granularity", 2024-04-21T08:51:48.304 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.304 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.304 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.305 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.305 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.305 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.305 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.305 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow minute scheduled snapshots", 2024-04-21T08:51:48.305 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.305 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.305 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.305 INFO:teuthology.orchestra.run.smithi043.stdout: "dump_on_update": { 2024-04-21T08:51:48.305 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dump_on_update", 2024-04-21T08:51:48.305 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.305 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.305 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.306 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.306 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.306 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.306 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.306 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "dump database to debug log on update", 2024-04-21T08:51:48.306 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.306 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.306 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.306 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.306 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.306 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.306 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.307 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.307 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.307 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.307 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.307 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.307 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.307 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.307 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.307 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.307 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.307 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.307 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.307 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.307 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.307 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.307 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.308 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.308 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.308 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.308 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.308 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.308 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.308 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.308 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.308 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.308 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.308 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.308 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.308 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.309 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.309 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.309 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.309 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.309 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.309 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.309 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.309 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.309 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.309 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.309 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.309 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.309 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.309 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.309 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.310 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.310 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.310 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.310 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.310 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.310 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.310 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.310 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.310 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.310 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.310 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.310 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.310 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.311 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.311 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.311 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.311 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.311 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.311 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.311 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.311 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stats", 2024-04-21T08:51:48.311 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.311 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.311 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.311 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.311 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.311 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.312 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.312 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.312 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.312 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.312 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.312 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.312 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.312 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.312 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.312 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.312 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.312 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.312 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.312 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.312 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.313 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.313 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.313 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.313 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.313 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.313 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.313 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.313 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.313 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.313 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.313 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.313 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.313 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.313 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.314 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.314 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.314 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.314 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.314 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.314 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.314 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.314 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.314 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.314 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.314 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.314 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.314 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.315 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.315 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.315 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.315 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.315 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.315 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.315 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.315 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.315 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.315 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.315 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.315 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.315 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.316 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.316 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.316 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.316 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.316 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.316 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.316 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.316 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.316 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.316 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.316 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "status", 2024-04-21T08:51:48.316 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.317 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.317 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.317 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.317 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.317 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.317 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.317 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.317 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.317 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.317 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.317 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.317 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.317 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.317 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.318 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.318 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.318 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.318 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.318 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.318 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.318 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.318 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.318 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.321 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.321 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.321 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.321 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.321 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.322 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.322 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.322 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.322 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.322 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.322 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.322 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.322 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.322 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.322 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.322 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.322 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.323 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.323 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.323 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.323 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.323 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.323 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.323 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.323 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.323 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.323 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.323 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.323 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.323 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.323 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.323 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.324 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.324 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.324 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.324 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.324 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.324 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.324 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.324 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.324 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.324 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.324 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.324 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.324 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.324 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.324 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.325 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telegraf", 2024-04-21T08:51:48.325 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.325 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.325 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.325 INFO:teuthology.orchestra.run.smithi043.stdout: "address": { 2024-04-21T08:51:48.325 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "address", 2024-04-21T08:51:48.325 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.325 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.325 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.325 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-21T08:51:48.325 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.325 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.325 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.326 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.326 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.326 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.326 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.326 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-21T08:51:48.326 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-21T08:51:48.326 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.326 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.326 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.326 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15", 2024-04-21T08:51:48.326 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.326 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.326 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.326 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.327 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.327 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.327 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.327 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.327 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.327 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.327 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.327 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.327 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.327 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.327 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.327 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.327 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.327 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.327 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.328 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.328 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.328 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.328 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.328 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.328 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.328 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.328 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.328 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.328 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.328 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.328 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.328 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.328 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.328 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.329 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.329 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.329 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.329 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.329 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.329 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.329 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.329 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.329 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.329 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.329 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.329 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.329 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.329 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.330 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.330 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.330 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.330 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.330 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.330 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.330 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.330 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.330 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.330 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.330 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.330 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.330 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.331 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.331 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.331 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.331 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.331 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.331 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.331 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.331 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.331 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.331 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.331 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.331 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.331 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.331 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.332 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telemetry", 2024-04-21T08:51:48.332 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.332 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.332 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.332 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_basic": { 2024-04-21T08:51:48.332 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_basic", 2024-04-21T08:51:48.332 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.332 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.332 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.332 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.332 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.332 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.332 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.332 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-21T08:51:48.332 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.332 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.333 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.333 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_crash": { 2024-04-21T08:51:48.333 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_crash", 2024-04-21T08:51:48.333 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.333 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.333 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.333 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.333 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.333 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.333 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.333 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-21T08:51:48.333 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.333 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.334 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.334 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_device": { 2024-04-21T08:51:48.334 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_device", 2024-04-21T08:51:48.334 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.334 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.334 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.334 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.334 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.334 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.334 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.334 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-21T08:51:48.334 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.334 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.335 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.335 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_ident": { 2024-04-21T08:51:48.335 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_ident", 2024-04-21T08:51:48.335 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.335 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.335 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.335 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.335 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.335 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.335 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.335 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-21T08:51:48.335 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.335 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.335 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.336 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_perf": { 2024-04-21T08:51:48.336 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_perf", 2024-04-21T08:51:48.336 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.336 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.336 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.336 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.336 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.336 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.336 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.336 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-21T08:51:48.336 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.336 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.336 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.337 INFO:teuthology.orchestra.run.smithi043.stdout: "contact": { 2024-04-21T08:51:48.337 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "contact", 2024-04-21T08:51:48.337 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.337 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.337 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.337 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.337 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.337 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.337 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.337 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.337 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.337 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.337 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.337 INFO:teuthology.orchestra.run.smithi043.stdout: "description": { 2024-04-21T08:51:48.338 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "description", 2024-04-21T08:51:48.338 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.338 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.338 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.338 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.338 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.338 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.338 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.338 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.338 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.338 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.338 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.338 INFO:teuthology.orchestra.run.smithi043.stdout: "device_url": { 2024-04-21T08:51:48.338 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_url", 2024-04-21T08:51:48.339 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.339 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.339 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.339 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-21T08:51:48.339 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.339 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.339 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.339 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.339 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.339 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.339 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.339 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-04-21T08:51:48.339 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-04-21T08:51:48.340 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.340 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.340 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.340 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.340 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.340 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.340 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.340 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.340 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.340 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.340 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.340 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-21T08:51:48.340 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-21T08:51:48.340 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.340 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.341 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.341 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "24", 2024-04-21T08:51:48.341 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "8", 2024-04-21T08:51:48.341 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.341 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.341 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.341 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.341 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.341 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.341 INFO:teuthology.orchestra.run.smithi043.stdout: "last_opt_revision": { 2024-04-21T08:51:48.341 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "last_opt_revision", 2024-04-21T08:51:48.341 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.341 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.341 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.342 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1", 2024-04-21T08:51:48.342 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.342 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.342 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.342 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.342 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.342 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.342 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.342 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard": { 2024-04-21T08:51:48.342 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard", 2024-04-21T08:51:48.342 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.342 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.342 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.343 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.343 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.343 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.343 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.343 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.343 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.343 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.343 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.343 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard_description": { 2024-04-21T08:51:48.343 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard_description", 2024-04-21T08:51:48.343 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.343 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.343 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.343 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.343 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.344 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.344 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.344 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.344 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.344 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.344 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.344 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.344 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.344 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.344 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.344 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.344 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.344 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.344 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.345 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.345 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.345 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.345 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.345 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.345 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.345 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.345 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.345 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.345 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.345 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.345 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.345 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.345 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.346 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.346 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.346 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.346 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.346 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.346 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.346 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.346 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.346 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.346 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.346 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.346 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.346 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.347 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.347 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.347 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.347 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.347 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.347 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.347 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.347 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.347 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.347 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.347 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.347 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.347 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.347 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.347 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.347 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.348 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.348 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.348 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.348 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.348 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.348 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.348 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.348 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.348 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.348 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.348 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.348 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.348 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.348 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.348 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.349 INFO:teuthology.orchestra.run.smithi043.stdout: "organization": { 2024-04-21T08:51:48.349 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "organization", 2024-04-21T08:51:48.349 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.349 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.349 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.349 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.349 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.349 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.349 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.349 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.349 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.349 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.349 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.349 INFO:teuthology.orchestra.run.smithi043.stdout: "proxy": { 2024-04-21T08:51:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "proxy", 2024-04-21T08:51:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: "url": { 2024-04-21T08:51:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url", 2024-04-21T08:51:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-21T08:51:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "test_orchestrator", 2024-04-21T08:51:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "volumes", 2024-04-21T08:51:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_clones": { 2024-04-21T08:51:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_clones", 2024-04-21T08:51:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4", 2024-04-21T08:51:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-21T08:51:48.362 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.362 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.362 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.362 INFO:teuthology.orchestra.run.smithi043.stdout: "periodic_async_work": { 2024-04-21T08:51:48.362 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "periodic_async_work", 2024-04-21T08:51:48.362 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.362 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.362 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.362 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.362 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.362 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.362 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.362 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Periodically check for async work", 2024-04-21T08:51:48.362 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.363 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.363 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.363 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_delay": { 2024-04-21T08:51:48.363 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_delay", 2024-04-21T08:51:48.363 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.363 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.363 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.363 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-21T08:51:48.363 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.363 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.363 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.363 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-21T08:51:48.363 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.363 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.364 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.364 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_no_wait": { 2024-04-21T08:51:48.364 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_no_wait", 2024-04-21T08:51:48.364 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.364 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.364 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.364 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.364 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.364 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.364 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.364 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-21T08:51:48.364 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.364 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.364 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.364 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.365 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.365 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix", 2024-04-21T08:51:48.365 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.365 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.365 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.365 INFO:teuthology.orchestra.run.smithi043.stdout: "discovery_interval": { 2024-04-21T08:51:48.365 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "discovery_interval", 2024-04-21T08:51:48.365 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-21T08:51:48.365 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.365 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.365 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "100", 2024-04-21T08:51:48.365 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.365 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.365 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.365 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.365 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.366 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.366 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.366 INFO:teuthology.orchestra.run.smithi043.stdout: "identifier": { 2024-04-21T08:51:48.366 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "identifier", 2024-04-21T08:51:48.366 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.366 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.366 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.366 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.366 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.366 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.366 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.366 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.366 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.367 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.367 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.367 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-21T08:51:48.367 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-21T08:51:48.367 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.367 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.367 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.367 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-21T08:51:48.367 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.367 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.367 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.367 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.367 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.367 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.368 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.368 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.368 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.368 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.368 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.368 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.368 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.368 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.368 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.368 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.368 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.368 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.368 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.368 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.369 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.369 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.369 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.369 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.369 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.369 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.369 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.369 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.369 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.369 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.369 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.369 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.369 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.370 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.370 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.370 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.370 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.370 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.370 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.370 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.370 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.370 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.370 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.370 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.370 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.370 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.371 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.371 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.371 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.371 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.371 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.371 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.371 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.371 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.371 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.371 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.371 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.371 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.371 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.371 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.372 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.372 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.372 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.372 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.372 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.372 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.372 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.372 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.372 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.372 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.372 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.372 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.372 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.373 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_host": { 2024-04-21T08:51:48.373 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_host", 2024-04-21T08:51:48.373 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.373 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.373 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.373 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.373 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.373 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.373 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.373 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.373 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.373 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.373 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.374 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_port": { 2024-04-21T08:51:48.374 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_port", 2024-04-21T08:51:48.374 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.374 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.374 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.374 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10051", 2024-04-21T08:51:48.374 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.374 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.374 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.374 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.374 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.374 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.374 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_sender": { 2024-04-21T08:51:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_sender", 2024-04-21T08:51:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-21T08:51:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-21T08:51:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 25145, 2024-04-21T08:51:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "y", 2024-04-21T08:51:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_features": 4540701547738038271, 2024-04-21T08:51:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: "available_modules": [ 2024-04-21T08:51:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "alerts", 2024-04-21T08:51:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-21T08:51:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-21T08:51:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-21T08:51:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How frequently to reexamine health status", 2024-04-21T08:51:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_destination": { 2024-04-21T08:51:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_destination", 2024-04-21T08:51:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email address to send alerts to", 2024-04-21T08:51:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_from_name": { 2024-04-21T08:51:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_from_name", 2024-04-21T08:51:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "Ceph", 2024-04-21T08:51:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email From: name", 2024-04-21T08:51:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_host": { 2024-04-21T08:51:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_host", 2024-04-21T08:51:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP server", 2024-04-21T08:51:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_password": { 2024-04-21T08:51:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_password", 2024-04-21T08:51:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Password to authenticate with", 2024-04-21T08:51:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_port": { 2024-04-21T08:51:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_port", 2024-04-21T08:51:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "465", 2024-04-21T08:51:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP port", 2024-04-21T08:51:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.387 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.387 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.387 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_sender": { 2024-04-21T08:51:48.387 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_sender", 2024-04-21T08:51:48.387 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.387 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.387 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.387 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.387 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.387 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.387 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.387 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP envelope sender", 2024-04-21T08:51:48.387 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.387 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.388 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.388 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_ssl": { 2024-04-21T08:51:48.388 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_ssl", 2024-04-21T08:51:48.388 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.388 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.388 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.388 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.388 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.388 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.388 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.388 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-21T08:51:48.388 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.388 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.389 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.389 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_user": { 2024-04-21T08:51:48.389 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_user", 2024-04-21T08:51:48.389 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.389 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.389 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.389 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.389 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.389 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.389 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.389 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "User to authenticate as", 2024-04-21T08:51:48.389 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.389 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.389 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.389 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.390 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.390 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "balancer", 2024-04-21T08:51:48.390 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.390 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.390 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.390 INFO:teuthology.orchestra.run.smithi043.stdout: "active": { 2024-04-21T08:51:48.390 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "active", 2024-04-21T08:51:48.390 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.390 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.390 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.390 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.390 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.390 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.391 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.391 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically balance PGs across cluster", 2024-04-21T08:51:48.391 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.391 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.391 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.391 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_time": { 2024-04-21T08:51:48.391 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_time", 2024-04-21T08:51:48.391 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.391 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.391 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.391 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0000", 2024-04-21T08:51:48.391 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.391 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.391 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.391 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "beginning time of day to automatically balance", 2024-04-21T08:51:48.392 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T08:51:48.392 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.392 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.392 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_weekday": { 2024-04-21T08:51:48.392 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_weekday", 2024-04-21T08:51:48.392 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-21T08:51:48.392 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.392 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.392 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-21T08:51:48.392 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-04-21T08:51:48.392 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-04-21T08:51:48.392 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.392 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-21T08:51:48.392 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T08:51:48.392 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.393 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.393 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_max_iterations": { 2024-04-21T08:51:48.393 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_max_iterations", 2024-04-21T08:51:48.393 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-21T08:51:48.393 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.393 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.393 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "25", 2024-04-21T08:51:48.393 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-04-21T08:51:48.393 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "250", 2024-04-21T08:51:48.393 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.393 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-21T08:51:48.393 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.393 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.393 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.394 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_metrics": { 2024-04-21T08:51:48.394 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_metrics", 2024-04-21T08:51:48.394 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.394 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.394 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.394 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "pgs,objects,bytes", 2024-04-21T08:51:48.394 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.394 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.394 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.394 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-21T08:51:48.394 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-21T08:51:48.394 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.394 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.395 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_step": { 2024-04-21T08:51:48.395 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_step", 2024-04-21T08:51:48.395 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-21T08:51:48.395 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.395 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.395 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.5", 2024-04-21T08:51:48.395 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0.001", 2024-04-21T08:51:48.395 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "0.999", 2024-04-21T08:51:48.395 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.395 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "aggressiveness of optimization", 2024-04-21T08:51:48.395 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-21T08:51:48.395 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.395 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.395 INFO:teuthology.orchestra.run.smithi043.stdout: "end_time": { 2024-04-21T08:51:48.396 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_time", 2024-04-21T08:51:48.396 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.396 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.396 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.396 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2359", 2024-04-21T08:51:48.396 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.396 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.396 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.396 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ending time of day to automatically balance", 2024-04-21T08:51:48.396 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T08:51:48.396 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.396 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.396 INFO:teuthology.orchestra.run.smithi043.stdout: "end_weekday": { 2024-04-21T08:51:48.396 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_weekday", 2024-04-21T08:51:48.396 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-21T08:51:48.397 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.397 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.397 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-21T08:51:48.397 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-04-21T08:51:48.397 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-04-21T08:51:48.397 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.397 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-21T08:51:48.397 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T08:51:48.397 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.397 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.397 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.397 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.397 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.397 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.398 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.398 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.398 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.398 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.398 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.398 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.398 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.398 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.398 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.398 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.398 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.398 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.398 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.398 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.398 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.399 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.399 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.399 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.399 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.399 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.399 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.399 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.399 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.399 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.399 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.399 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.399 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.399 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.399 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.400 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.400 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.400 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.400 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.400 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.400 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.400 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.400 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.400 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.400 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.400 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.400 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.400 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.400 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.400 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.401 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.401 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.401 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.401 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.401 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.401 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.401 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.401 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.401 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.401 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.401 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.401 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.401 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.401 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.401 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.402 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.402 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.402 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.402 INFO:teuthology.orchestra.run.smithi043.stdout: "min_score": { 2024-04-21T08:51:48.402 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_score", 2024-04-21T08:51:48.402 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-21T08:51:48.402 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.402 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.402 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-21T08:51:48.402 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.402 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.402 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.402 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-21T08:51:48.402 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.402 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.403 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.403 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-04-21T08:51:48.403 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-04-21T08:51:48.403 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.403 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.403 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.403 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "upmap", 2024-04-21T08:51:48.403 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.403 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.403 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.403 INFO:teuthology.orchestra.run.smithi043.stdout: "crush-compat", 2024-04-21T08:51:48.403 INFO:teuthology.orchestra.run.smithi043.stdout: "none", 2024-04-21T08:51:48.403 INFO:teuthology.orchestra.run.smithi043.stdout: "read", 2024-04-21T08:51:48.403 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap", 2024-04-21T08:51:48.403 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap-read" 2024-04-21T08:51:48.403 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.404 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Balancer mode", 2024-04-21T08:51:48.404 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.404 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.404 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.404 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ids": { 2024-04-21T08:51:48.404 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_ids", 2024-04-21T08:51:48.404 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.404 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.404 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.404 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.404 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.404 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.404 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.404 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-21T08:51:48.404 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.405 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.405 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.405 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-21T08:51:48.405 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-21T08:51:48.405 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.405 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.405 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.405 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-21T08:51:48.405 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.405 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.405 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.405 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-21T08:51:48.405 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.405 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.405 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.406 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_deviation": { 2024-04-21T08:51:48.406 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_deviation", 2024-04-21T08:51:48.406 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.406 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.406 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.406 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-04-21T08:51:48.406 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-04-21T08:51:48.406 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.406 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.406 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-21T08:51:48.406 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-21T08:51:48.406 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.406 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.406 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_optimizations": { 2024-04-21T08:51:48.407 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_optimizations", 2024-04-21T08:51:48.407 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-21T08:51:48.407 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.407 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.407 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-21T08:51:48.407 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.407 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.407 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.407 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-21T08:51:48.407 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.407 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.407 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.407 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.407 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.408 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm", 2024-04-21T08:51:48.408 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.408 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.408 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.408 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_down_multiplier": { 2024-04-21T08:51:48.408 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_down_multiplier", 2024-04-21T08:51:48.408 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-21T08:51:48.408 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.408 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.408 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-04-21T08:51:48.408 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.408 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.408 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.408 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-21T08:51:48.408 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.408 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.409 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.409 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_refresh_rate": { 2024-04-21T08:51:48.409 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_refresh_rate", 2024-04-21T08:51:48.409 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.409 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.409 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.409 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "20", 2024-04-21T08:51:48.409 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.409 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.409 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.409 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-21T08:51:48.409 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.409 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.409 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.410 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_starting_port": { 2024-04-21T08:51:48.410 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_starting_port", 2024-04-21T08:51:48.410 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.410 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.410 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.410 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4721", 2024-04-21T08:51:48.410 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.410 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.410 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.410 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-21T08:51:48.410 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.410 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.410 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.410 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_ptrace": { 2024-04-21T08:51:48.411 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_ptrace", 2024-04-21T08:51:48.411 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.411 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.411 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.411 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.411 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.411 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.411 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.411 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-21T08:51:48.411 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-21T08:51:48.411 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.411 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.411 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_interval": { 2024-04-21T08:51:48.411 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_interval", 2024-04-21T08:51:48.411 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.412 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.412 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.412 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-21T08:51:48.412 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.412 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.412 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.412 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-21T08:51:48.412 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.412 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.412 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.412 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_memory_target_ratio": { 2024-04-21T08:51:48.412 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_memory_target_ratio", 2024-04-21T08:51:48.412 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-21T08:51:48.412 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.412 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.413 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.7", 2024-04-21T08:51:48.413 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.413 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.413 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.413 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-21T08:51:48.413 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.413 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.413 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.413 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm_log_destination": { 2024-04-21T08:51:48.413 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm_log_destination", 2024-04-21T08:51:48.413 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.413 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.413 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.413 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.413 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.414 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.414 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.414 INFO:teuthology.orchestra.run.smithi043.stdout: "file", 2024-04-21T08:51:48.414 INFO:teuthology.orchestra.run.smithi043.stdout: "file,syslog", 2024-04-21T08:51:48.414 INFO:teuthology.orchestra.run.smithi043.stdout: "syslog" 2024-04-21T08:51:48.414 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.414 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-21T08:51:48.414 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.414 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.414 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.414 INFO:teuthology.orchestra.run.smithi043.stdout: "cgroups_split": { 2024-04-21T08:51:48.414 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cgroups_split", 2024-04-21T08:51:48.414 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.414 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.414 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.415 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.415 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.415 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.415 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.415 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-21T08:51:48.415 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.415 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.415 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.415 INFO:teuthology.orchestra.run.smithi043.stdout: "config_checks_enabled": { 2024-04-21T08:51:48.415 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_checks_enabled", 2024-04-21T08:51:48.415 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.415 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.415 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.415 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.415 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.415 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.416 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.416 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-21T08:51:48.416 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.416 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.416 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.416 INFO:teuthology.orchestra.run.smithi043.stdout: "config_dashboard": { 2024-04-21T08:51:48.416 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_dashboard", 2024-04-21T08:51:48.416 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.416 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.416 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.416 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.416 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.416 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.416 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.416 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-21T08:51:48.416 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.417 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.417 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.417 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_alertmanager": { 2024-04-21T08:51:48.417 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_alertmanager", 2024-04-21T08:51:48.417 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.417 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.417 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.417 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-21T08:51:48.417 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.417 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.417 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.417 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-04-21T08:51:48.417 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.417 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.418 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.418 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_base": { 2024-04-21T08:51:48.418 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_base", 2024-04-21T08:51:48.418 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.418 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.418 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.418 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-21T08:51:48.418 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.418 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.418 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.418 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Container image name, without the tag", 2024-04-21T08:51:48.418 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.418 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.418 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.419 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_elasticsearch": { 2024-04-21T08:51:48.419 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_elasticsearch", 2024-04-21T08:51:48.419 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.419 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.419 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.419 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-21T08:51:48.419 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.419 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.419 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.419 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "elasticsearch container image", 2024-04-21T08:51:48.419 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.419 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.419 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.419 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_grafana": { 2024-04-21T08:51:48.420 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_grafana", 2024-04-21T08:51:48.420 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.420 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.420 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.420 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-21T08:51:48.420 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.420 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.420 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.420 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-04-21T08:51:48.420 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.420 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.420 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.420 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_haproxy": { 2024-04-21T08:51:48.420 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_haproxy", 2024-04-21T08:51:48.421 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.421 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.421 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.421 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-21T08:51:48.421 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.421 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.421 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.421 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "HAproxy container image", 2024-04-21T08:51:48.421 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.421 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.421 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.421 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_agent": { 2024-04-21T08:51:48.421 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_agent", 2024-04-21T08:51:48.422 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.422 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.422 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.422 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-21T08:51:48.422 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.422 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.422 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.422 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger agent container image", 2024-04-21T08:51:48.422 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.422 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.422 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.422 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_collector": { 2024-04-21T08:51:48.423 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_collector", 2024-04-21T08:51:48.423 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.423 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.423 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.423 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-21T08:51:48.423 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.423 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.423 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.423 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger collector container image", 2024-04-21T08:51:48.423 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.423 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.423 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.423 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_query": { 2024-04-21T08:51:48.423 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_query", 2024-04-21T08:51:48.424 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.424 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.424 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.424 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-21T08:51:48.424 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.424 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.424 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.424 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger query container image", 2024-04-21T08:51:48.424 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.424 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.424 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.424 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_keepalived": { 2024-04-21T08:51:48.424 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_keepalived", 2024-04-21T08:51:48.424 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.425 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.425 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.425 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-21T08:51:48.425 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.425 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.425 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.425 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Keepalived container image", 2024-04-21T08:51:48.425 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.425 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.425 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.425 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_loki": { 2024-04-21T08:51:48.425 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_loki", 2024-04-21T08:51:48.425 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.425 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.426 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.426 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-21T08:51:48.426 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.426 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.426 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.426 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Loki container image", 2024-04-21T08:51:48.426 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.426 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.426 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.426 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_node_exporter": { 2024-04-21T08:51:48.426 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_node_exporter", 2024-04-21T08:51:48.426 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.426 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.427 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.427 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-21T08:51:48.427 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.427 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.427 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.427 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-04-21T08:51:48.427 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.427 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.427 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.427 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_nvmeof": { 2024-04-21T08:51:48.427 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_nvmeof", 2024-04-21T08:51:48.427 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.427 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.428 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.428 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-21T08:51:48.428 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.428 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.428 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.428 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Nvme-of container image", 2024-04-21T08:51:48.428 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.428 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.428 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.428 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_prometheus": { 2024-04-21T08:51:48.428 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_prometheus", 2024-04-21T08:51:48.428 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.428 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.428 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.429 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-21T08:51:48.429 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.429 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.429 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.429 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-04-21T08:51:48.429 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.429 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.429 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.429 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_promtail": { 2024-04-21T08:51:48.429 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_promtail", 2024-04-21T08:51:48.429 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.429 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.429 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.429 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-21T08:51:48.429 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.430 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.430 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.430 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Promtail container image", 2024-04-21T08:51:48.430 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.430 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.430 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.430 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_samba": { 2024-04-21T08:51:48.430 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_samba", 2024-04-21T08:51:48.430 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.430 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.430 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.430 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-21T08:51:48.430 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.431 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.431 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.431 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Samba/SMB container image", 2024-04-21T08:51:48.431 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.431 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.431 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.431 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_snmp_gateway": { 2024-04-21T08:51:48.431 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_snmp_gateway", 2024-04-21T08:51:48.431 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.431 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.431 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.431 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-21T08:51:48.432 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.432 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.432 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.432 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SNMP Gateway container image", 2024-04-21T08:51:48.432 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.432 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.432 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.432 INFO:teuthology.orchestra.run.smithi043.stdout: "container_init": { 2024-04-21T08:51:48.432 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_init", 2024-04-21T08:51:48.432 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.432 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.432 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.432 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.432 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.433 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.433 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.433 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Run podman/docker with `--init`", 2024-04-21T08:51:48.433 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.433 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.433 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.433 INFO:teuthology.orchestra.run.smithi043.stdout: "daemon_cache_timeout": { 2024-04-21T08:51:48.433 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "daemon_cache_timeout", 2024-04-21T08:51:48.433 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.433 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.433 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.433 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-21T08:51:48.433 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.433 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.434 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.434 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-21T08:51:48.434 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.434 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.434 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.434 INFO:teuthology.orchestra.run.smithi043.stdout: "default_cephadm_command_timeout": { 2024-04-21T08:51:48.434 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_cephadm_command_timeout", 2024-04-21T08:51:48.434 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.434 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.434 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.434 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "900", 2024-04-21T08:51:48.434 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.434 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.434 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.435 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-21T08:51:48.435 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.435 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.435 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.435 INFO:teuthology.orchestra.run.smithi043.stdout: "default_registry": { 2024-04-21T08:51:48.435 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_registry", 2024-04-21T08:51:48.435 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.435 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.435 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.435 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io", 2024-04-21T08:51:48.435 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.435 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.435 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.435 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-21T08:51:48.436 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.436 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.436 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.436 INFO:teuthology.orchestra.run.smithi043.stdout: "device_cache_timeout": { 2024-04-21T08:51:48.436 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_cache_timeout", 2024-04-21T08:51:48.436 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.436 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.436 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.436 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1800", 2024-04-21T08:51:48.436 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.436 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.436 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.436 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache device inventory", 2024-04-21T08:51:48.436 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.437 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.437 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.437 INFO:teuthology.orchestra.run.smithi043.stdout: "device_enhanced_scan": { 2024-04-21T08:51:48.437 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_enhanced_scan", 2024-04-21T08:51:48.437 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.437 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.437 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.437 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.437 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.437 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.437 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.437 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-21T08:51:48.437 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.437 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.438 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.438 INFO:teuthology.orchestra.run.smithi043.stdout: "facts_cache_timeout": { 2024-04-21T08:51:48.438 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "facts_cache_timeout", 2024-04-21T08:51:48.438 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.438 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.438 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.438 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-21T08:51:48.438 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.438 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.438 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.438 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache host facts data", 2024-04-21T08:51:48.438 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.438 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.439 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.439 INFO:teuthology.orchestra.run.smithi043.stdout: "grafana_dashboards_path": { 2024-04-21T08:51:48.439 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "grafana_dashboards_path", 2024-04-21T08:51:48.439 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.439 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.439 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.439 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-21T08:51:48.439 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.439 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.439 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.439 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-21T08:51:48.439 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.439 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.439 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.440 INFO:teuthology.orchestra.run.smithi043.stdout: "host_check_interval": { 2024-04-21T08:51:48.440 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "host_check_interval", 2024-04-21T08:51:48.440 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.440 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.440 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.440 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-21T08:51:48.440 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.440 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.440 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.440 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to perform a host check", 2024-04-21T08:51:48.440 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.440 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.440 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.441 INFO:teuthology.orchestra.run.smithi043.stdout: "hw_monitoring": { 2024-04-21T08:51:48.441 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hw_monitoring", 2024-04-21T08:51:48.441 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.441 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.441 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.441 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.441 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.441 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.441 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.441 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-21T08:51:48.441 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.441 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.441 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.441 INFO:teuthology.orchestra.run.smithi043.stdout: "inventory_list_all": { 2024-04-21T08:51:48.442 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "inventory_list_all", 2024-04-21T08:51:48.442 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.442 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.442 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.442 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.442 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.442 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.442 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.442 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-21T08:51:48.442 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.442 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.442 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.442 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.443 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.443 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.443 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.443 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.443 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.443 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.443 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.443 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.443 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.443 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.443 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.443 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.443 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.443 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.443 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.444 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.444 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.444 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.444 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.444 INFO:teuthology.orchestra.run.smithi043.stdout: "log_refresh_metadata": { 2024-04-21T08:51:48.444 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_refresh_metadata", 2024-04-21T08:51:48.444 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.444 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.444 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.444 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.444 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.444 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.444 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.444 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-21T08:51:48.445 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.445 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.445 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.445 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.445 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.445 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.445 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.445 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.445 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.445 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.445 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.445 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.445 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-21T08:51:48.446 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.446 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.446 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.446 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.446 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.446 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.446 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.446 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.446 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.446 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.446 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.446 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.446 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.446 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.447 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.447 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.447 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.447 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.447 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.447 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.447 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.447 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.447 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.447 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.447 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.447 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.447 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.448 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.448 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.448 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.448 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.448 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.448 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.448 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.448 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.448 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.448 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-21T08:51:48.448 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-21T08:51:48.448 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.448 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.448 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.449 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.449 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.449 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.449 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.449 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-21T08:51:48.449 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.449 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.449 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.449 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-21T08:51:48.449 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-21T08:51:48.449 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.449 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.449 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.449 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "*", 2024-04-21T08:51:48.450 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.450 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.450 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.450 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-21T08:51:48.450 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.450 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.450 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.450 INFO:teuthology.orchestra.run.smithi043.stdout: "max_count_per_host": { 2024-04-21T08:51:48.450 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_count_per_host", 2024-04-21T08:51:48.450 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.450 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.450 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.450 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-21T08:51:48.451 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.451 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.451 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.451 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of daemons per service per host", 2024-04-21T08:51:48.451 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.451 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.451 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.451 INFO:teuthology.orchestra.run.smithi043.stdout: "max_osd_draining_count": { 2024-04-21T08:51:48.451 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_osd_draining_count", 2024-04-21T08:51:48.451 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.451 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.451 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.451 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-21T08:51:48.451 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.451 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.451 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.452 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-21T08:51:48.452 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.452 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.452 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.452 INFO:teuthology.orchestra.run.smithi043.stdout: "migration_current": { 2024-04-21T08:51:48.452 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "migration_current", 2024-04-21T08:51:48.452 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.452 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.452 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.452 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.452 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.452 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.452 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.452 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "internal - do not modify", 2024-04-21T08:51:48.452 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.453 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.453 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.453 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-04-21T08:51:48.453 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-04-21T08:51:48.453 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.453 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.453 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.453 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "root", 2024-04-21T08:51:48.453 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.453 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.453 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.453 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm-package", 2024-04-21T08:51:48.453 INFO:teuthology.orchestra.run.smithi043.stdout: "root" 2024-04-21T08:51:48.453 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.453 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "mode for remote execution of cephadm", 2024-04-21T08:51:48.453 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.454 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.454 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.454 INFO:teuthology.orchestra.run.smithi043.stdout: "oob_default_addr": { 2024-04-21T08:51:48.454 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "oob_default_addr", 2024-04-21T08:51:48.454 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.454 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.454 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.454 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "169.254.1.1", 2024-04-21T08:51:48.454 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.454 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.454 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.454 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-21T08:51:48.454 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.455 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.455 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.455 INFO:teuthology.orchestra.run.smithi043.stdout: "prometheus_alerts_path": { 2024-04-21T08:51:48.455 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus_alerts_path", 2024-04-21T08:51:48.455 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.455 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.455 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.455 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-21T08:51:48.455 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.455 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.455 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.455 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-21T08:51:48.455 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.456 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.456 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.456 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_insecure": { 2024-04-21T08:51:48.456 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_insecure", 2024-04-21T08:51:48.456 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.456 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.456 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.456 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.456 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.456 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.456 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.456 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-21T08:51:48.456 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.456 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.457 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.457 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_password": { 2024-04-21T08:51:48.457 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_password", 2024-04-21T08:51:48.457 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.457 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.457 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.457 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.457 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.457 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.457 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.457 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-21T08:51:48.457 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.457 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.457 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.458 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_url": { 2024-04-21T08:51:48.458 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_url", 2024-04-21T08:51:48.458 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.458 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.458 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.458 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.458 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.458 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.458 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.458 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-21T08:51:48.458 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.458 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.458 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.458 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_username": { 2024-04-21T08:51:48.459 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_username", 2024-04-21T08:51:48.459 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.459 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.459 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.459 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.459 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.459 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.459 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.459 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-21T08:51:48.459 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.459 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.459 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.459 INFO:teuthology.orchestra.run.smithi043.stdout: "secure_monitoring_stack": { 2024-04-21T08:51:48.459 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "secure_monitoring_stack", 2024-04-21T08:51:48.459 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.459 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.460 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.460 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.460 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.460 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.460 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.460 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-21T08:51:48.460 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.460 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.460 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.460 INFO:teuthology.orchestra.run.smithi043.stdout: "service_discovery_port": { 2024-04-21T08:51:48.460 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "service_discovery_port", 2024-04-21T08:51:48.460 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.460 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.460 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.460 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8765", 2024-04-21T08:51:48.461 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.461 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.461 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.461 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "cephadm service discovery port", 2024-04-21T08:51:48.461 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.461 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.461 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.461 INFO:teuthology.orchestra.run.smithi043.stdout: "ssh_config_file": { 2024-04-21T08:51:48.461 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssh_config_file", 2024-04-21T08:51:48.461 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.461 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.461 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.461 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.461 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.462 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.462 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.462 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-21T08:51:48.462 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.462 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.462 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.462 INFO:teuthology.orchestra.run.smithi043.stdout: "use_agent": { 2024-04-21T08:51:48.462 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_agent", 2024-04-21T08:51:48.462 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.462 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.462 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.462 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.462 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.462 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.463 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.463 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-21T08:51:48.463 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.463 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.463 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.463 INFO:teuthology.orchestra.run.smithi043.stdout: "use_repo_digest": { 2024-04-21T08:51:48.463 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_repo_digest", 2024-04-21T08:51:48.463 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.463 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.463 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.463 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.463 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.463 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.463 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.464 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-21T08:51:48.464 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.464 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.464 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.464 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_failed_host_check": { 2024-04-21T08:51:48.464 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_failed_host_check", 2024-04-21T08:51:48.464 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.464 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.464 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.464 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.464 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.464 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.464 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.464 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if the host check fails", 2024-04-21T08:51:48.464 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.465 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.465 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.465 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_daemons": { 2024-04-21T08:51:48.465 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_daemons", 2024-04-21T08:51:48.465 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.465 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.465 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.465 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.465 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.465 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.465 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.465 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-21T08:51:48.465 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.465 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.465 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.466 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_hosts": { 2024-04-21T08:51:48.466 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_hosts", 2024-04-21T08:51:48.466 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.466 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.466 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.466 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.466 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.466 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.466 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.466 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-21T08:51:48.466 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.466 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.466 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.467 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.467 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.467 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crash", 2024-04-21T08:51:48.467 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.467 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.467 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.467 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.467 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.467 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.467 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.467 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.467 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.467 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.468 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.468 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.468 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.468 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.468 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.468 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.468 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.468 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.468 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.468 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.468 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.468 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.468 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.468 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.468 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.469 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.469 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.469 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.469 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.469 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.469 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.469 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.469 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.469 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.469 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.469 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.469 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.469 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.469 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.469 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.470 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.470 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.470 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.470 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.470 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.470 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.470 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.470 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.470 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.470 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.470 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.470 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.470 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.470 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "retain_interval": { 2024-04-21T08:51:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retain_interval", 2024-04-21T08:51:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "31536000", 2024-04-21T08:51:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-21T08:51:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_recent_interval": { 2024-04-21T08:51:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_recent_interval", 2024-04-21T08:51:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1209600", 2024-04-21T08:51:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-21T08:51:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dashboard", 2024-04-21T08:51:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-21T08:51:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-21T08:51:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-21T08:51:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_HOST": { 2024-04-21T08:51:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-21T08:51:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-21T08:51:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-21T08:51:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_ENABLED": { 2024-04-21T08:51:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_ENABLED", 2024-04-21T08:51:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-21T08:51:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-21T08:51:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "ENABLE_BROWSABLE_API": { 2024-04-21T08:51:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-21T08:51:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-21T08:51:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-21T08:51:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-21T08:51:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-21T08:51:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.482 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.482 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.482 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.482 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.482 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.482 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.482 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.482 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.482 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.482 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-21T08:51:48.482 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-21T08:51:48.482 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.482 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.482 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.483 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.483 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.483 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.483 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.483 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.483 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.483 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.483 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.483 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-21T08:51:48.483 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-21T08:51:48.483 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.483 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.483 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.484 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.484 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.484 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.484 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.484 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.484 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.484 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.484 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.484 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-21T08:51:48.484 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-21T08:51:48.484 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.484 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.484 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.484 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.484 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.485 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.485 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.485 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.485 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.485 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.485 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.485 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-21T08:51:48.485 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-21T08:51:48.485 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.485 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.485 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.485 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.485 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.486 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.486 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.486 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.486 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.486 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.486 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.486 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-21T08:51:48.486 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-21T08:51:48.486 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.486 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.486 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.486 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.486 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.486 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.487 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.487 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.487 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.487 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.487 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.487 INFO:teuthology.orchestra.run.smithi043.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-21T08:51:48.487 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-21T08:51:48.487 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.487 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.487 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.487 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.487 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.487 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.487 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.488 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.488 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.488 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.488 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.488 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_PASSWORD": { 2024-04-21T08:51:48.488 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-21T08:51:48.488 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.488 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.488 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.488 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-04-21T08:51:48.488 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.488 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.488 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.488 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.488 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.489 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.489 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.489 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-21T08:51:48.489 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-21T08:51:48.489 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.489 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.489 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.489 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.489 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.489 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.489 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.489 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.489 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.489 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.490 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.490 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_URL": { 2024-04-21T08:51:48.490 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_URL", 2024-04-21T08:51:48.490 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.490 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.490 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.490 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.490 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.490 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.490 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.490 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.490 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.490 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.490 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.491 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_USERNAME": { 2024-04-21T08:51:48.491 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-21T08:51:48.491 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.491 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.491 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.491 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-04-21T08:51:48.491 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.491 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.491 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.491 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.491 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.491 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.492 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.492 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-21T08:51:48.492 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-21T08:51:48.492 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.492 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.492 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.492 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.492 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.492 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.492 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.492 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.492 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.492 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.492 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.493 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-21T08:51:48.493 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-21T08:51:48.493 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.493 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.493 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.493 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.493 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.493 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.493 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.493 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.493 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.493 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.493 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.494 INFO:teuthology.orchestra.run.smithi043.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-21T08:51:48.494 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-21T08:51:48.494 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.494 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.494 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.494 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.494 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.494 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.494 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.494 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.494 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.494 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.494 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.494 INFO:teuthology.orchestra.run.smithi043.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-21T08:51:48.495 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-21T08:51:48.495 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.495 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.495 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.495 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.495 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.495 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.495 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.495 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.495 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.495 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.495 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.495 INFO:teuthology.orchestra.run.smithi043.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-21T08:51:48.496 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-21T08:51:48.496 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.496 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.496 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.496 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "[]", 2024-04-21T08:51:48.496 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.496 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.496 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.496 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.496 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.496 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.496 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.496 INFO:teuthology.orchestra.run.smithi043.stdout: "MULTICLUSTER_CONFIG": { 2024-04-21T08:51:48.496 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-21T08:51:48.497 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.497 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.497 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.497 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "{}", 2024-04-21T08:51:48.497 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.497 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.497 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.497 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.497 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.497 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.497 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.497 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_HOST": { 2024-04-21T08:51:48.497 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-21T08:51:48.497 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.498 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.498 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.498 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.498 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.498 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.498 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.498 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.498 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.498 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.498 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.498 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-21T08:51:48.498 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-21T08:51:48.498 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.498 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.499 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.499 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.499 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.499 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.499 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.499 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.499 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.499 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.499 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.499 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-21T08:51:48.499 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-21T08:51:48.499 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.499 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.499 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.499 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.499 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.500 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.500 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.500 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.500 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.500 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.500 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.500 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-21T08:51:48.500 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-21T08:51:48.500 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.500 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.500 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.500 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.500 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.500 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.501 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.501 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.501 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.501 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.501 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.501 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-21T08:51:48.501 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-21T08:51:48.501 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.501 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.501 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.501 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.501 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.501 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.501 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.502 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.502 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.502 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.502 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.502 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-21T08:51:48.502 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-21T08:51:48.502 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.502 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.502 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.502 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.502 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.502 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.502 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.502 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.503 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.503 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.503 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.503 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-21T08:51:48.503 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-21T08:51:48.503 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.503 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.503 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.503 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.503 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.503 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.503 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.503 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.503 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.503 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.504 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.504 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-21T08:51:48.504 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-21T08:51:48.504 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.504 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.504 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.504 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.504 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.504 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.504 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.504 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.504 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.504 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.504 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.504 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-21T08:51:48.505 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-21T08:51:48.505 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.505 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.505 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.505 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.505 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.505 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.505 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.505 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.505 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.505 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.505 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.505 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_ENABLED": { 2024-04-21T08:51:48.505 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-21T08:51:48.505 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.506 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.506 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.506 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.506 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.506 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.506 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.506 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.506 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.506 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.506 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.506 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-21T08:51:48.506 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-21T08:51:48.506 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.506 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.506 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.507 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-21T08:51:48.507 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.507 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.507 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.507 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.507 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.507 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.507 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.507 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-21T08:51:48.507 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-21T08:51:48.507 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.507 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.507 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.507 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-21T08:51:48.507 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.508 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.508 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.508 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.508 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.508 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.508 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.508 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-21T08:51:48.508 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-21T08:51:48.508 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.508 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.508 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.508 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8", 2024-04-21T08:51:48.508 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.508 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.508 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.509 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.509 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.509 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.509 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.509 INFO:teuthology.orchestra.run.smithi043.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-21T08:51:48.509 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-21T08:51:48.509 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.509 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.509 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.509 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "45", 2024-04-21T08:51:48.509 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.509 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.509 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.509 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.509 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.509 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.510 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.510 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ACCESS_KEY": { 2024-04-21T08:51:48.510 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-21T08:51:48.510 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.510 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.510 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.510 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.510 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.510 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.510 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.510 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.510 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.510 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.511 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.511 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-21T08:51:48.511 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-21T08:51:48.511 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.511 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.511 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.511 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-04-21T08:51:48.511 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.511 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.511 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.511 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.511 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.511 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.511 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.512 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SECRET_KEY": { 2024-04-21T08:51:48.512 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-21T08:51:48.512 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.512 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.512 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.512 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.512 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.512 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.512 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.512 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.512 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.512 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.512 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.513 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SSL_VERIFY": { 2024-04-21T08:51:48.513 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-21T08:51:48.513 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.513 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.513 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.513 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.513 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.513 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.513 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.513 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.513 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.513 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.513 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.513 INFO:teuthology.orchestra.run.smithi043.stdout: "UNSAFE_TLS_v1_2": { 2024-04-21T08:51:48.514 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-21T08:51:48.514 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.514 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.514 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.514 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.514 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.514 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.514 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.514 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.514 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.514 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.514 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.514 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-21T08:51:48.514 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-21T08:51:48.515 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.515 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.515 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.515 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-21T08:51:48.515 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.515 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.515 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.515 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.515 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.515 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.515 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.515 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-21T08:51:48.515 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-21T08:51:48.515 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.515 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.516 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.516 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-21T08:51:48.516 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.516 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.516 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.516 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.516 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.516 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.516 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.516 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-21T08:51:48.516 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-21T08:51:48.516 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.516 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.516 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.517 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-04-21T08:51:48.517 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.517 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.517 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.517 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.517 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.517 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.517 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.517 INFO:teuthology.orchestra.run.smithi043.stdout: "cross_origin_url": { 2024-04-21T08:51:48.517 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cross_origin_url", 2024-04-21T08:51:48.517 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.517 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.517 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.518 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.518 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.518 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.518 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.518 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.518 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.518 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.518 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.518 INFO:teuthology.orchestra.run.smithi043.stdout: "crt_file": { 2024-04-21T08:51:48.518 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crt_file", 2024-04-21T08:51:48.518 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.518 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.518 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.519 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.519 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.519 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.519 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.519 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.519 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.519 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.519 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.519 INFO:teuthology.orchestra.run.smithi043.stdout: "debug": { 2024-04-21T08:51:48.519 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "debug", 2024-04-21T08:51:48.519 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.519 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.519 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.519 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.520 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.520 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.520 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.520 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable/disable debug options", 2024-04-21T08:51:48.520 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.520 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.520 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.520 INFO:teuthology.orchestra.run.smithi043.stdout: "jwt_token_ttl": { 2024-04-21T08:51:48.520 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "jwt_token_ttl", 2024-04-21T08:51:48.520 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.520 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.520 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.520 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "28800", 2024-04-21T08:51:48.520 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.521 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.521 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.521 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.521 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.521 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.521 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.521 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-04-21T08:51:48.521 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-04-21T08:51:48.521 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.521 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.521 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.521 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.521 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.521 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.522 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.522 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.522 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.522 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.522 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.522 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.522 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.522 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.522 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.522 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.522 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.522 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.522 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.522 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.523 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.523 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.523 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.523 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.523 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.523 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.523 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.523 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.523 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.523 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.525 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.525 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.525 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.526 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.526 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.526 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.526 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.526 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.526 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.526 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.526 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.526 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.526 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.526 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.526 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.526 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.527 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.527 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.527 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.527 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.527 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.527 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.527 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.527 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.527 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.527 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.527 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.527 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.527 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.527 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.528 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.528 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.528 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.528 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.528 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.528 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.528 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.528 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.528 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.528 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.528 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.528 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.528 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.529 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.529 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.529 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.529 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.529 INFO:teuthology.orchestra.run.smithi043.stdout: "motd": { 2024-04-21T08:51:48.529 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "motd", 2024-04-21T08:51:48.529 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.529 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.529 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.529 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.529 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.529 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.529 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.529 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "The message of the day", 2024-04-21T08:51:48.530 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.530 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.530 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.530 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect_resolve_ip_addr": { 2024-04-21T08:51:48.530 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "redirect_resolve_ip_addr", 2024-04-21T08:51:48.530 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.530 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.530 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.530 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.530 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.530 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.530 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.530 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.531 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.531 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.531 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.531 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-04-21T08:51:48.531 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-04-21T08:51:48.531 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.531 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.531 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.531 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-04-21T08:51:48.531 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.531 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.531 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.531 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.531 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.531 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.532 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.532 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-04-21T08:51:48.532 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-04-21T08:51:48.532 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.532 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.532 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.532 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8080", 2024-04-21T08:51:48.532 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.532 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.532 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.532 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.532 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.532 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.532 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.533 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-04-21T08:51:48.533 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-04-21T08:51:48.533 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.533 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.533 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.533 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.533 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.533 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.533 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.533 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.533 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.533 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.533 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.533 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl_server_port": { 2024-04-21T08:51:48.534 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl_server_port", 2024-04-21T08:51:48.534 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.534 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.534 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.534 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8443", 2024-04-21T08:51:48.534 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.534 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.534 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.534 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.534 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.534 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.534 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.534 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-04-21T08:51:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-04-21T08:51:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "redirect", 2024-04-21T08:51:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect" 2024-04-21T08:51:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.536 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-04-21T08:51:48.536 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-04-21T08:51:48.536 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.536 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.536 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.536 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-04-21T08:51:48.536 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-04-21T08:51:48.536 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-04-21T08:51:48.536 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.536 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.536 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.536 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.536 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: "url_prefix": { 2024-04-21T08:51:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url_prefix", 2024-04-21T08:51:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "devicehealth", 2024-04-21T08:51:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_monitoring": { 2024-04-21T08:51:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_monitoring", 2024-04-21T08:51:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "monitor device health metrics", 2024-04-21T08:51:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: "mark_out_threshold": { 2024-04-21T08:51:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mark_out_threshold", 2024-04-21T08:51:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2419200", 2024-04-21T08:51:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-21T08:51:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": { 2024-04-21T08:51:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_name", 2024-04-21T08:51:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "device_health_metrics", 2024-04-21T08:51:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-21T08:51:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: "retention_period": { 2024-04-21T08:51:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retention_period", 2024-04-21T08:51:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15552000", 2024-04-21T08:51:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain device health metrics", 2024-04-21T08:51:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_frequency": { 2024-04-21T08:51:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_frequency", 2024-04-21T08:51:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-04-21T08:51:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-21T08:51:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: "self_heal": { 2024-04-21T08:51:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "self_heal", 2024-04-21T08:51:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-21T08:51:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-21T08:51:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-21T08:51:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-21T08:51:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and check device health", 2024-04-21T08:51:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_threshold": { 2024-04-21T08:51:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_threshold", 2024-04-21T08:51:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7257600", 2024-04-21T08:51:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-21T08:51:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "diskprediction_local", 2024-04-21T08:51:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "predict_interval": { 2024-04-21T08:51:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predict_interval", 2024-04-21T08:51:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-04-21T08:51:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "predictor_model": { 2024-04-21T08:51:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predictor_model", 2024-04-21T08:51:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "prophetstor", 2024-04-21T08:51:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-21T08:51:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-21T08:51:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-21T08:51:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "influx", 2024-04-21T08:51:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": false, 2024-04-21T08:51:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "influxdb python module not found", 2024-04-21T08:51:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: "batch_size": { 2024-04-21T08:51:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "batch_size", 2024-04-21T08:51:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5000", 2024-04-21T08:51:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-21T08:51:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "database": { 2024-04-21T08:51:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "database", 2024-04-21T08:51:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "ceph", 2024-04-21T08:51:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-21T08:51:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": { 2024-04-21T08:51:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hostname", 2024-04-21T08:51:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server hostname", 2024-04-21T08:51:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-21T08:51:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-21T08:51:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "30", 2024-04-21T08:51:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "5", 2024-04-21T08:51:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-21T08:51:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: "password": { 2024-04-21T08:51:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "password", 2024-04-21T08:51:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "password of InfluxDB server user", 2024-04-21T08:51:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "port": { 2024-04-21T08:51:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "port", 2024-04-21T08:51:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8086", 2024-04-21T08:51:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server port", 2024-04-21T08:51:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-04-21T08:51:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-04-21T08:51:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "false", 2024-04-21T08:51:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T08:51:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: "threads": { 2024-04-21T08:51:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threads", 2024-04-21T08:51:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-04-21T08:51:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-04-21T08:51:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "32", 2024-04-21T08:51:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-21T08:51:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: "username": { 2024-04-21T08:51:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "username", 2024-04-21T08:51:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "username of InfluxDB server user", 2024-04-21T08:51:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: "verify_ssl": { 2024-04-21T08:51:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "verify_ssl", 2024-04-21T08:51:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "true", 2024-04-21T08:51:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T08:51:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "insights", 2024-04-21T08:51:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "iostat", 2024-04-21T08:51:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "k8sevents", 2024-04-21T08:51:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_event_retention_days": { 2024-04-21T08:51:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ceph_event_retention_days", 2024-04-21T08:51:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7", 2024-04-21T08:51:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-21T08:51:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: "config_check_secs": { 2024-04-21T08:51:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_check_secs", 2024-04-21T08:51:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-21T08:51:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "10", 2024-04-21T08:51:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-21T08:51:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.588 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.588 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.588 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.588 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.588 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.588 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.588 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.588 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.588 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.588 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.588 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.588 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.588 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.588 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "localpool", 2024-04-21T08:51:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: "failure_domain": { 2024-04-21T08:51:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "failure_domain", 2024-04-21T08:51:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "host", 2024-04-21T08:51:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "failure domain for any created local pool", 2024-04-21T08:51:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-21T08:51:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.592 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.592 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.592 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.592 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.592 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.592 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.592 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.592 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.592 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.592 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": { 2024-04-21T08:51:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_size", 2024-04-21T08:51:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default min_size for any created local pool", 2024-04-21T08:51:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-21T08:51:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rep": { 2024-04-21T08:51:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "num_rep", 2024-04-21T08:51:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3", 2024-04-21T08:51:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default replica count for any created local pool", 2024-04-21T08:51:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": { 2024-04-21T08:51:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_num", 2024-04-21T08:51:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "128", 2024-04-21T08:51:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default pg_num for any created local pool", 2024-04-21T08:51:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: "prefix": { 2024-04-21T08:51:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prefix", 2024-04-21T08:51:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name prefix for any created local pool", 2024-04-21T08:51:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: "subtree": { 2024-04-21T08:51:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "subtree", 2024-04-21T08:51:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "rack", 2024-04-21T08:51:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-21T08:51:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-21T08:51:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mds_autoscaler", 2024-04-21T08:51:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirroring", 2024-04-21T08:51:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "nfs", 2024-04-21T08:51:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-04-21T08:51:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "fail_fs": { 2024-04-21T08:51:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "fail_fs", 2024-04-21T08:51:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-21T08:51:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator": { 2024-04-21T08:51:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-04-21T08:51:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm", 2024-04-21T08:51:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: "rook", 2024-04-21T08:51:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: "test_orchestrator" 2024-04-21T08:51:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Orchestrator backend", 2024-04-21T08:51:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_perf_query", 2024-04-21T08:51:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_support", 2024-04-21T08:51:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_autoscaler", 2024-04-21T08:51:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.634 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.634 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.634 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.634 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.634 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.634 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.634 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.634 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.634 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.634 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.634 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.634 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.634 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.634 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.635 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.635 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.635 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.635 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.635 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.635 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.635 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.635 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.635 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.635 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.635 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.635 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.635 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.635 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.636 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.636 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.636 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.636 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.636 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.636 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.636 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.636 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.636 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.636 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.636 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.636 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.636 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.636 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.637 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.637 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.637 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.637 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.637 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.637 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.637 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.637 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.637 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.637 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.637 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.637 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.637 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.637 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.638 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.638 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.638 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.638 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.638 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.638 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.638 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-21T08:51:48.638 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-21T08:51:48.638 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.638 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.638 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.638 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-21T08:51:48.638 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.638 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.639 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.639 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.639 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.639 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.639 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.639 INFO:teuthology.orchestra.run.smithi043.stdout: "threshold": { 2024-04-21T08:51:48.639 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threshold", 2024-04-21T08:51:48.639 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-21T08:51:48.639 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.639 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.639 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-04-21T08:51:48.639 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1.0", 2024-04-21T08:51:48.639 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.639 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.640 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "scaling threshold", 2024-04-21T08:51:48.640 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-21T08:51:48.640 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.640 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.640 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.640 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.640 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "progress", 2024-04-21T08:51:48.640 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.640 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.640 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.640 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_pg_recovery_event": { 2024-04-21T08:51:48.640 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_pg_recovery_event", 2024-04-21T08:51:48.640 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.641 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.641 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.641 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.641 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.641 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.641 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.641 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-21T08:51:48.641 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.641 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.641 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.641 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-04-21T08:51:48.641 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-04-21T08:51:48.641 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.641 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.641 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.642 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.642 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.642 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.642 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.642 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.642 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.642 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.642 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.642 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.642 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.642 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.642 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.642 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.643 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.643 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.643 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.643 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.643 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.643 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.643 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.643 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.643 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.643 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.643 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.643 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.643 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.643 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.643 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.644 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.644 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.644 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.644 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.644 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.644 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.644 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.644 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.644 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.644 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.644 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.644 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.644 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.645 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.645 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.645 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.645 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.645 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.645 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.645 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.645 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.645 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.645 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.645 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.645 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.645 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.645 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.645 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.646 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.646 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.646 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.646 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.646 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.646 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.646 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.646 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.646 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.646 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.646 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.646 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.646 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.647 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.647 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.647 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.647 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.647 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.647 INFO:teuthology.orchestra.run.smithi043.stdout: "max_completed_events": { 2024-04-21T08:51:48.647 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_completed_events", 2024-04-21T08:51:48.647 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.647 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.647 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.647 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "50", 2024-04-21T08:51:48.647 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.647 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.648 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.648 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "number of past completed events to remember", 2024-04-21T08:51:48.648 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.648 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.648 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.648 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-21T08:51:48.648 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-21T08:51:48.648 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.648 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.648 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.648 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-04-21T08:51:48.648 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.648 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.648 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.649 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long the module is going to sleep", 2024-04-21T08:51:48.649 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.649 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.649 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.649 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.649 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.649 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus", 2024-04-21T08:51:48.649 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.649 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.649 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.649 INFO:teuthology.orchestra.run.smithi043.stdout: "cache": { 2024-04-21T08:51:48.649 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cache", 2024-04-21T08:51:48.650 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.650 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.650 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.650 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.650 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.650 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.650 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.650 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.650 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.650 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.650 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.650 INFO:teuthology.orchestra.run.smithi043.stdout: "exclude_perf_counters": { 2024-04-21T08:51:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "exclude_perf_counters", 2024-04-21T08:51:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-21T08:51:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-21T08:51:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools": { 2024-04-21T08:51:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools", 2024-04-21T08:51:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-21T08:51:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-21T08:51:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "300", 2024-04-21T08:51:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_interval": { 2024-04-21T08:51:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_interval", 2024-04-21T08:51:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-21T08:51:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15.0", 2024-04-21T08:51:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-04-21T08:51:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-04-21T08:51:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-04-21T08:51:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-21T08:51:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-04-21T08:51:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-04-21T08:51:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "9283", 2024-04-21T08:51:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-21T08:51:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: "stale_cache_strategy": { 2024-04-21T08:51:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stale_cache_strategy", 2024-04-21T08:51:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "log", 2024-04-21T08:51:48.662 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.662 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.662 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.662 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.662 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.662 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.662 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.662 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-04-21T08:51:48.662 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-04-21T08:51:48.662 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.662 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.662 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.662 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "default", 2024-04-21T08:51:48.663 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.663 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.663 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.663 INFO:teuthology.orchestra.run.smithi043.stdout: "default", 2024-04-21T08:51:48.663 INFO:teuthology.orchestra.run.smithi043.stdout: "error" 2024-04-21T08:51:48.663 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.663 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.663 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.663 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.663 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.663 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-04-21T08:51:48.663 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-04-21T08:51:48.664 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.664 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.664 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.664 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-04-21T08:51:48.664 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-04-21T08:51:48.664 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-04-21T08:51:48.664 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.664 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.664 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.664 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.664 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.664 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.664 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.665 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_support", 2024-04-21T08:51:48.665 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.665 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.665 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.665 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.665 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.665 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.665 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.665 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.665 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.665 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.665 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.665 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.665 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.665 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.665 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.666 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.666 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.666 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.666 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.666 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.666 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.666 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.666 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.666 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.666 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.666 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.666 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.666 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.667 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.667 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.667 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.667 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.667 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.667 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.667 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.667 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.667 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.667 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.667 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.667 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.667 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.667 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.668 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.668 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.668 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.668 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.668 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.668 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.668 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.668 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.668 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.668 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.668 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.668 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.668 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.668 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.668 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.669 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.669 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.669 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.669 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.669 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.669 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.669 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.669 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.669 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.669 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.669 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.669 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.669 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_snap_create": { 2024-04-21T08:51:48.670 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_snap_create", 2024-04-21T08:51:48.670 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.670 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.670 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.670 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-21T08:51:48.670 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.670 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.670 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.670 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.670 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.670 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.670 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.670 INFO:teuthology.orchestra.run.smithi043.stdout: "mirror_snapshot_schedule": { 2024-04-21T08:51:48.671 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirror_snapshot_schedule", 2024-04-21T08:51:48.671 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.671 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.671 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.671 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.671 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.671 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.671 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.671 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.671 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.671 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.671 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.671 INFO:teuthology.orchestra.run.smithi043.stdout: "trash_purge_schedule": { 2024-04-21T08:51:48.672 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "trash_purge_schedule", 2024-04-21T08:51:48.672 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.672 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.672 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.672 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.672 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.672 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.672 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.672 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.672 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.672 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.672 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.672 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.672 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.673 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "restful", 2024-04-21T08:51:48.673 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.673 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.673 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.673 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_auth": { 2024-04-21T08:51:48.673 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_auth", 2024-04-21T08:51:48.673 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.673 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.673 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.673 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.673 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.673 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.673 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.674 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.674 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.674 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.674 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.674 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-04-21T08:51:48.674 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-04-21T08:51:48.674 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.674 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.674 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.674 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.674 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.674 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.674 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.675 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.675 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.675 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.675 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.675 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.675 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.675 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.675 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.675 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.675 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.675 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.675 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.675 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.675 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.676 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.676 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.676 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.676 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.676 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.676 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.676 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.676 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.676 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.676 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.676 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.676 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.676 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.677 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.677 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.677 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.677 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.677 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.677 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.677 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.677 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.677 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.677 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.677 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.677 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.677 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.677 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.678 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.678 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.678 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.678 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.678 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.678 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.678 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.678 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.678 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.678 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.678 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.678 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.678 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.678 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.679 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.679 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.679 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.679 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.679 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.679 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.679 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.679 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.679 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.679 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.679 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.679 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.679 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.679 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.680 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.680 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-04-21T08:51:48.680 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-04-21T08:51:48.680 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.680 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.680 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.680 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.680 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.680 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.680 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.680 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.680 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.680 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.681 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.681 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-04-21T08:51:48.681 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-04-21T08:51:48.681 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.681 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.681 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.681 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.681 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.681 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.681 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.681 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.681 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.681 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.682 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.682 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.682 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.682 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rgw", 2024-04-21T08:51:48.682 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.682 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.682 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.682 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.682 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.682 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.682 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.682 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.683 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.683 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.683 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.683 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.683 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.683 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.683 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.683 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.683 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.683 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.683 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.683 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.683 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.683 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.684 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.684 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.684 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.684 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.684 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.684 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.684 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.684 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.684 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.684 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.684 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.684 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.684 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.685 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.685 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.685 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.685 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.685 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.685 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.685 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.685 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.685 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.685 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.685 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.685 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.685 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.686 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.686 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.686 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.686 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.686 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.686 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.686 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.686 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.686 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.686 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.686 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.686 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.686 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.687 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.687 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.687 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.687 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.687 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.687 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.687 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.687 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.687 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.687 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.687 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rook", 2024-04-21T08:51:48.687 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.687 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.688 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.688 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.688 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.688 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.688 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.688 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.688 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.688 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.688 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.688 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.688 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.688 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.688 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.688 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.688 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.689 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.689 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.689 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.689 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.689 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.689 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.689 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.689 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.689 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.689 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.689 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.689 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.689 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.690 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.690 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.690 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.690 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.690 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.690 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.690 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.690 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.690 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.690 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.690 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.690 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.690 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.690 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.691 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.691 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.691 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.691 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.691 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.691 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.691 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.691 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.691 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.691 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.691 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.691 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.691 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.691 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.692 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.692 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.692 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.692 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.692 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.692 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.692 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.692 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.692 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.692 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.692 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.692 INFO:teuthology.orchestra.run.smithi043.stdout: "storage_class": { 2024-04-21T08:51:48.692 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "storage_class", 2024-04-21T08:51:48.693 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.693 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.693 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.693 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "local", 2024-04-21T08:51:48.693 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.693 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.693 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.693 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-21T08:51:48.693 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.693 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.693 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.693 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.693 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.694 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "selftest", 2024-04-21T08:51:48.694 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.694 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.694 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.694 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.694 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.694 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.694 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.694 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.694 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.694 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.694 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.695 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.695 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.695 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.695 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.695 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.695 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.695 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.695 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.695 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.695 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.695 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.695 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.695 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.695 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.696 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.696 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.696 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.696 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.696 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.696 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.696 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.696 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.696 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.696 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.696 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.696 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.696 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.696 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.697 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.697 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.697 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.697 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.697 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.697 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.697 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.697 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.697 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.697 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.697 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.697 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.697 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.698 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.698 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.698 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.698 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.698 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.698 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.698 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.698 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.698 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.698 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.698 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.698 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.698 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.699 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.699 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.699 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.699 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.699 INFO:teuthology.orchestra.run.smithi043.stdout: "roption1": { 2024-04-21T08:51:48.699 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption1", 2024-04-21T08:51:48.699 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.699 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.699 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.699 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.699 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.699 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.699 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.700 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.700 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.700 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.700 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.700 INFO:teuthology.orchestra.run.smithi043.stdout: "roption2": { 2024-04-21T08:51:48.700 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption2", 2024-04-21T08:51:48.700 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.700 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.700 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.700 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "xyz", 2024-04-21T08:51:48.700 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.700 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.700 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.700 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.701 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.701 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.701 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.701 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption1": { 2024-04-21T08:51:48.701 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption1", 2024-04-21T08:51:48.701 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.701 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.701 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.701 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.701 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.701 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.701 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.701 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.701 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption2": { 2024-04-21T08:51:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption2", 2024-04-21T08:51:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.703 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.703 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.703 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption3": { 2024-04-21T08:51:48.703 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption3", 2024-04-21T08:51:48.703 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-21T08:51:48.703 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.703 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.703 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.703 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.703 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.703 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.703 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.703 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.704 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.704 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.704 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption4": { 2024-04-21T08:51:48.704 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption4", 2024-04-21T08:51:48.704 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.704 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.704 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.704 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.704 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.704 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.704 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.704 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.704 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.704 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.705 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.705 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption5": { 2024-04-21T08:51:48.705 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption5", 2024-04-21T08:51:48.705 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.705 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.705 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.705 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.705 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.705 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.705 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.705 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.705 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.705 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.705 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.706 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption6": { 2024-04-21T08:51:48.706 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption6", 2024-04-21T08:51:48.706 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.706 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.706 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.706 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.706 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.706 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.706 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.706 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.706 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.706 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.706 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.707 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption7": { 2024-04-21T08:51:48.707 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption7", 2024-04-21T08:51:48.707 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.707 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.707 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.707 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.707 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-04-21T08:51:48.707 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "42", 2024-04-21T08:51:48.707 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.707 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.707 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.707 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.707 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.708 INFO:teuthology.orchestra.run.smithi043.stdout: "testkey": { 2024-04-21T08:51:48.708 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testkey", 2024-04-21T08:51:48.708 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.708 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.708 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.708 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.708 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.708 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.708 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.708 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.708 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.708 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.708 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.709 INFO:teuthology.orchestra.run.smithi043.stdout: "testlkey": { 2024-04-21T08:51:48.709 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testlkey", 2024-04-21T08:51:48.709 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.709 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.709 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.709 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.709 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.709 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.709 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.709 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.709 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.709 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.709 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.709 INFO:teuthology.orchestra.run.smithi043.stdout: "testnewline": { 2024-04-21T08:51:48.710 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testnewline", 2024-04-21T08:51:48.710 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.710 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.710 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.710 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.710 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.710 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.710 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.710 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.710 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.710 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.710 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.710 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.711 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.711 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snap_schedule", 2024-04-21T08:51:48.711 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.711 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.711 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.711 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_m_granularity": { 2024-04-21T08:51:48.711 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_m_granularity", 2024-04-21T08:51:48.711 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.711 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.711 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.711 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.711 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.712 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.712 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.712 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow minute scheduled snapshots", 2024-04-21T08:51:48.712 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.712 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.712 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.712 INFO:teuthology.orchestra.run.smithi043.stdout: "dump_on_update": { 2024-04-21T08:51:48.712 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dump_on_update", 2024-04-21T08:51:48.712 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.712 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.712 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.712 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.712 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.713 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.713 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.713 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "dump database to debug log on update", 2024-04-21T08:51:48.713 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.713 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.713 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.713 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.713 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.713 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.713 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.713 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.713 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.713 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.714 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.714 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.714 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.714 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.714 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.714 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.714 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.714 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.714 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.714 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.714 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.714 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.714 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.714 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.715 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.715 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.715 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.715 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.715 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.715 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.715 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.715 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.715 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.715 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.715 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.715 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.716 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.716 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.716 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.716 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.716 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.716 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.716 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.716 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.716 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.716 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.716 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.716 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.716 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.716 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.716 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.717 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.717 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.717 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.717 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.717 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.717 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.717 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.717 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.717 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.717 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.717 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.717 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.717 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.717 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.718 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.718 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.718 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.718 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.718 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.718 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.718 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stats", 2024-04-21T08:51:48.718 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.718 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.718 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.718 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.718 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.719 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.719 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.719 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.719 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.719 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.719 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.719 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.719 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.719 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.719 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.719 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.719 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.719 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.719 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.720 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.720 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.720 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.720 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.720 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.720 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.720 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.720 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.720 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.720 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.720 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.720 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.720 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.721 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.721 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.721 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.721 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.721 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.721 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.721 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.721 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.721 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.721 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.721 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.721 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.721 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.721 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.722 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.722 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.722 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.722 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.722 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.722 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.722 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.722 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.722 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.722 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.722 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.722 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.722 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.722 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.723 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.723 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.723 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.723 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.723 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.723 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.723 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.723 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.723 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.723 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.723 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.723 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "status", 2024-04-21T08:51:48.723 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.724 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.724 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.724 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.724 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.724 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.724 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.724 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.724 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.724 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.724 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.724 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.724 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.724 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.725 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.725 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.725 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.725 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.725 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.725 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.725 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.725 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.725 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.725 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.725 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.725 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.725 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.726 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.726 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.726 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.726 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.726 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.726 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.726 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.726 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.726 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.726 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.726 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.726 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.726 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.726 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.727 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.727 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.727 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.727 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.727 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.727 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.727 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.727 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.727 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.727 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.727 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.727 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.727 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.727 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.727 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.728 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.728 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.728 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.728 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.728 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.728 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.729 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.730 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.730 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.730 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.730 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.730 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.730 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.730 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.730 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.730 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telegraf", 2024-04-21T08:51:48.730 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.730 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.730 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.730 INFO:teuthology.orchestra.run.smithi043.stdout: "address": { 2024-04-21T08:51:48.731 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "address", 2024-04-21T08:51:48.731 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.731 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.731 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.731 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-21T08:51:48.731 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.731 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.731 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.731 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.731 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.731 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.731 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.731 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-21T08:51:48.732 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-21T08:51:48.732 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.732 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.732 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.732 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15", 2024-04-21T08:51:48.732 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.732 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.732 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.732 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.732 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.732 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.732 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.732 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.732 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.733 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.733 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.733 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.733 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.733 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.733 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.733 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.733 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.733 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.733 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.733 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.733 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.733 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.733 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.733 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.734 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.734 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.734 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.734 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.734 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.734 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.734 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.734 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.734 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.734 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.734 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.734 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.735 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.735 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.735 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.735 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.735 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.735 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.735 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.735 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.735 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.735 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.735 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.735 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.735 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.735 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.736 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.736 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.736 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.736 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.736 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.736 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.736 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.736 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.736 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.736 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.736 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.736 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.736 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.736 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.737 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.737 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.737 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.737 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.737 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.737 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.737 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.737 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.737 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.737 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.737 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.737 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telemetry", 2024-04-21T08:51:48.737 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.737 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.738 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.738 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_basic": { 2024-04-21T08:51:48.738 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_basic", 2024-04-21T08:51:48.738 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.738 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.738 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.738 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.738 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.738 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.738 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.738 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-21T08:51:48.738 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.738 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.738 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.739 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_crash": { 2024-04-21T08:51:48.739 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_crash", 2024-04-21T08:51:48.739 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.739 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.739 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.739 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.739 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.739 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.739 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.739 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-21T08:51:48.739 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.739 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.739 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.740 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_device": { 2024-04-21T08:51:48.740 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_device", 2024-04-21T08:51:48.740 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.740 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.740 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.740 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.740 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.740 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.740 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.740 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-21T08:51:48.740 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.740 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.740 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.741 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_ident": { 2024-04-21T08:51:48.741 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_ident", 2024-04-21T08:51:48.741 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.741 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.741 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.741 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.741 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.741 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.741 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.741 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-21T08:51:48.741 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.741 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.741 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.742 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_perf": { 2024-04-21T08:51:48.742 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_perf", 2024-04-21T08:51:48.742 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.742 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.742 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.742 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.742 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.742 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.742 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.742 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-21T08:51:48.742 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.742 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.742 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.742 INFO:teuthology.orchestra.run.smithi043.stdout: "contact": { 2024-04-21T08:51:48.743 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "contact", 2024-04-21T08:51:48.743 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.743 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.743 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.743 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.743 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.743 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.743 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.743 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.743 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.743 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.743 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.743 INFO:teuthology.orchestra.run.smithi043.stdout: "description": { 2024-04-21T08:51:48.743 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "description", 2024-04-21T08:51:48.744 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.744 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.744 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.744 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.744 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.744 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.744 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.744 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.744 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.744 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.744 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.744 INFO:teuthology.orchestra.run.smithi043.stdout: "device_url": { 2024-04-21T08:51:48.744 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_url", 2024-04-21T08:51:48.744 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.745 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.745 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.745 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-21T08:51:48.745 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.745 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.745 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.745 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.745 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.745 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.745 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.745 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-04-21T08:51:48.745 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-04-21T08:51:48.745 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.745 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.746 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.746 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.746 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.746 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.746 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.746 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.746 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.746 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.746 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.746 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-21T08:51:48.746 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-21T08:51:48.746 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.746 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.746 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.747 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "24", 2024-04-21T08:51:48.747 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "8", 2024-04-21T08:51:48.747 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.747 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.747 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.747 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.747 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.747 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.747 INFO:teuthology.orchestra.run.smithi043.stdout: "last_opt_revision": { 2024-04-21T08:51:48.747 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "last_opt_revision", 2024-04-21T08:51:48.747 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.747 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.747 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.747 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1", 2024-04-21T08:51:48.748 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.748 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.748 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.748 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.748 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.748 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.748 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.748 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard": { 2024-04-21T08:51:48.748 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard", 2024-04-21T08:51:48.748 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.748 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.748 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.748 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.748 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.749 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.749 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.749 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.749 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.749 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.749 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.749 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard_description": { 2024-04-21T08:51:48.749 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard_description", 2024-04-21T08:51:48.749 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.749 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.749 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.749 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.749 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.749 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.750 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.750 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.750 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.750 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.750 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.750 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.750 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.750 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.750 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.750 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.750 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.750 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.750 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.750 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.751 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.751 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.751 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.751 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.751 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.751 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.751 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.751 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.751 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.751 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.751 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.751 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.751 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.751 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.752 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.752 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.752 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.752 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.752 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.752 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.752 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.752 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.752 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.752 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.752 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.752 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.752 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.752 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.752 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.753 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.753 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.753 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.753 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.753 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.753 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.753 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.753 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.753 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.753 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.753 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.753 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.753 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.753 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.753 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.754 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.754 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.754 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.754 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.754 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.754 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.754 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.754 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.754 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.754 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.754 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.754 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.754 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.755 INFO:teuthology.orchestra.run.smithi043.stdout: "organization": { 2024-04-21T08:51:48.755 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "organization", 2024-04-21T08:51:48.755 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.755 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.755 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.755 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.755 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.755 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.755 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.755 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.755 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.755 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.755 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.755 INFO:teuthology.orchestra.run.smithi043.stdout: "proxy": { 2024-04-21T08:51:48.756 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "proxy", 2024-04-21T08:51:48.756 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.756 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.756 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.756 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.756 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.756 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.756 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.756 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.756 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.756 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.756 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.757 INFO:teuthology.orchestra.run.smithi043.stdout: "url": { 2024-04-21T08:51:48.757 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url", 2024-04-21T08:51:48.757 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.757 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.757 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.757 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-21T08:51:48.757 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.757 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.757 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.757 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.757 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.757 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.757 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.757 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.757 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.758 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "test_orchestrator", 2024-04-21T08:51:48.758 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.758 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.758 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.758 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.758 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.758 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.758 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.758 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.758 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.758 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.758 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.758 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.759 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.759 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.759 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.759 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.759 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.759 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.759 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.759 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.759 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.759 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.759 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.759 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.759 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.759 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.759 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.760 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.760 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.760 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.760 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.760 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.760 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.760 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.760 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.760 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.760 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.760 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.760 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.760 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.761 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.761 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.761 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.761 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.761 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.761 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.761 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.761 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.761 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.761 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.761 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.761 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.761 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.761 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.761 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.761 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.762 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.762 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.762 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.762 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.762 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.762 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.762 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.762 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.762 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.762 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.762 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.762 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.762 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.762 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.763 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.763 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "volumes", 2024-04-21T08:51:48.763 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.763 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.763 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.763 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.763 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.763 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.763 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.763 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.763 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.763 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.763 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.763 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.764 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.764 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.764 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.764 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.764 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.764 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.764 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.764 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.764 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.764 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.764 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.764 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.764 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.765 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.765 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.765 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.765 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.765 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.765 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.765 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.765 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.765 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.765 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.765 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.765 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.765 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.765 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.766 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.766 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.766 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.766 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.766 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.766 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.766 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.766 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.766 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.766 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.766 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.766 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.766 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.766 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.767 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.767 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.767 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.767 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.767 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.767 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.767 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.767 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.767 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.767 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.767 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.767 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.767 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.767 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.767 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.767 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.768 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_clones": { 2024-04-21T08:51:48.768 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_clones", 2024-04-21T08:51:48.768 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.768 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.768 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.768 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4", 2024-04-21T08:51:48.768 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.768 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.768 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.768 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-21T08:51:48.768 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.768 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.768 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.768 INFO:teuthology.orchestra.run.smithi043.stdout: "periodic_async_work": { 2024-04-21T08:51:48.769 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "periodic_async_work", 2024-04-21T08:51:48.769 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.769 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.769 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.769 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.769 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.769 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.769 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.769 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Periodically check for async work", 2024-04-21T08:51:48.769 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.769 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.769 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.769 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_delay": { 2024-04-21T08:51:48.769 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_delay", 2024-04-21T08:51:48.770 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.770 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.770 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.770 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-21T08:51:48.770 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.770 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.770 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.770 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-21T08:51:48.770 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.770 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.770 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.770 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_no_wait": { 2024-04-21T08:51:48.770 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_no_wait", 2024-04-21T08:51:48.770 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.771 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.771 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.771 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.771 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.771 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.771 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.771 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-21T08:51:48.771 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.771 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.771 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.771 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.771 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.771 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix", 2024-04-21T08:51:48.772 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.772 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.772 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.772 INFO:teuthology.orchestra.run.smithi043.stdout: "discovery_interval": { 2024-04-21T08:51:48.772 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "discovery_interval", 2024-04-21T08:51:48.772 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-21T08:51:48.772 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.772 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.772 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "100", 2024-04-21T08:51:48.772 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.772 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.772 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.772 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.772 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.772 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.773 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.773 INFO:teuthology.orchestra.run.smithi043.stdout: "identifier": { 2024-04-21T08:51:48.773 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "identifier", 2024-04-21T08:51:48.773 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.773 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.773 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.773 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.773 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.773 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.773 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.773 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.773 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.773 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.773 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.774 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-21T08:51:48.774 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-21T08:51:48.774 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.774 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.774 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.774 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-21T08:51:48.774 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.774 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.774 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.774 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.774 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.774 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.774 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.775 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.775 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.775 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.775 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.775 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.775 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.775 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.775 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.775 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.775 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.775 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.775 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.775 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.775 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.775 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.775 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.775 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.776 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.776 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.776 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.776 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.776 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.776 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.776 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.776 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.776 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.776 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.776 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.776 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.776 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.776 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_host": { 2024-04-21T08:51:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_host", 2024-04-21T08:51:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_port": { 2024-04-21T08:51:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_port", 2024-04-21T08:51:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10051", 2024-04-21T08:51:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_sender": { 2024-04-21T08:51:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_sender", 2024-04-21T08:51:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-21T08:51:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-21T08:51:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: "modules": [ 2024-04-21T08:51:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm", 2024-04-21T08:51:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: "dashboard", 2024-04-21T08:51:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: "iostat", 2024-04-21T08:51:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: "nfs", 2024-04-21T08:51:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: "restful", 2024-04-21T08:51:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_schedule" 2024-04-21T08:51:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: "available_modules": [ 2024-04-21T08:51:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "alerts", 2024-04-21T08:51:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-21T08:51:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-21T08:51:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-21T08:51:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How frequently to reexamine health status", 2024-04-21T08:51:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_destination": { 2024-04-21T08:51:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_destination", 2024-04-21T08:51:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.790 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.790 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email address to send alerts to", 2024-04-21T08:51:48.790 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.790 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.790 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.790 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_from_name": { 2024-04-21T08:51:48.790 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_from_name", 2024-04-21T08:51:48.790 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.790 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.790 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.790 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "Ceph", 2024-04-21T08:51:48.790 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email From: name", 2024-04-21T08:51:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_host": { 2024-04-21T08:51:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_host", 2024-04-21T08:51:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP server", 2024-04-21T08:51:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_password": { 2024-04-21T08:51:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_password", 2024-04-21T08:51:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Password to authenticate with", 2024-04-21T08:51:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_port": { 2024-04-21T08:51:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_port", 2024-04-21T08:51:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "465", 2024-04-21T08:51:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP port", 2024-04-21T08:51:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_sender": { 2024-04-21T08:51:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_sender", 2024-04-21T08:51:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP envelope sender", 2024-04-21T08:51:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_ssl": { 2024-04-21T08:51:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_ssl", 2024-04-21T08:51:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-21T08:51:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_user": { 2024-04-21T08:51:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_user", 2024-04-21T08:51:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "User to authenticate as", 2024-04-21T08:51:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "balancer", 2024-04-21T08:51:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: "active": { 2024-04-21T08:51:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "active", 2024-04-21T08:51:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically balance PGs across cluster", 2024-04-21T08:51:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_time": { 2024-04-21T08:51:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_time", 2024-04-21T08:51:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0000", 2024-04-21T08:51:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "beginning time of day to automatically balance", 2024-04-21T08:51:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T08:51:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_weekday": { 2024-04-21T08:51:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_weekday", 2024-04-21T08:51:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-21T08:51:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-21T08:51:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-04-21T08:51:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-04-21T08:51:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-21T08:51:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T08:51:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_max_iterations": { 2024-04-21T08:51:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_max_iterations", 2024-04-21T08:51:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-21T08:51:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "25", 2024-04-21T08:51:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-04-21T08:51:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "250", 2024-04-21T08:51:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-21T08:51:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_metrics": { 2024-04-21T08:51:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_metrics", 2024-04-21T08:51:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "pgs,objects,bytes", 2024-04-21T08:51:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-21T08:51:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-21T08:51:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_step": { 2024-04-21T08:51:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_step", 2024-04-21T08:51:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-21T08:51:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.5", 2024-04-21T08:51:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0.001", 2024-04-21T08:51:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "0.999", 2024-04-21T08:51:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "aggressiveness of optimization", 2024-04-21T08:51:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-21T08:51:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: "end_time": { 2024-04-21T08:51:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_time", 2024-04-21T08:51:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2359", 2024-04-21T08:51:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ending time of day to automatically balance", 2024-04-21T08:51:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T08:51:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: "end_weekday": { 2024-04-21T08:51:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_weekday", 2024-04-21T08:51:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-21T08:51:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-21T08:51:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-04-21T08:51:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-04-21T08:51:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-21T08:51:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T08:51:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: "min_score": { 2024-04-21T08:51:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_score", 2024-04-21T08:51:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-21T08:51:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-21T08:51:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-21T08:51:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-04-21T08:51:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-04-21T08:51:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "upmap", 2024-04-21T08:51:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: "crush-compat", 2024-04-21T08:51:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: "none", 2024-04-21T08:51:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: "read", 2024-04-21T08:51:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap", 2024-04-21T08:51:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap-read" 2024-04-21T08:51:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Balancer mode", 2024-04-21T08:51:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ids": { 2024-04-21T08:51:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_ids", 2024-04-21T08:51:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-21T08:51:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-21T08:51:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-21T08:51:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-21T08:51:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-21T08:51:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_deviation": { 2024-04-21T08:51:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_deviation", 2024-04-21T08:51:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-04-21T08:51:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-04-21T08:51:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-21T08:51:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-21T08:51:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_optimizations": { 2024-04-21T08:51:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_optimizations", 2024-04-21T08:51:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-21T08:51:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-21T08:51:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-21T08:51:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm", 2024-04-21T08:51:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_down_multiplier": { 2024-04-21T08:51:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_down_multiplier", 2024-04-21T08:51:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-21T08:51:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-04-21T08:51:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-21T08:51:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_refresh_rate": { 2024-04-21T08:51:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_refresh_rate", 2024-04-21T08:51:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "20", 2024-04-21T08:51:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-21T08:51:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_starting_port": { 2024-04-21T08:51:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_starting_port", 2024-04-21T08:51:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4721", 2024-04-21T08:51:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-21T08:51:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_ptrace": { 2024-04-21T08:51:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_ptrace", 2024-04-21T08:51:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-21T08:51:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-21T08:51:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_interval": { 2024-04-21T08:51:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_interval", 2024-04-21T08:51:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-21T08:51:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-21T08:51:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_memory_target_ratio": { 2024-04-21T08:51:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_memory_target_ratio", 2024-04-21T08:51:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-21T08:51:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.7", 2024-04-21T08:51:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-21T08:51:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm_log_destination": { 2024-04-21T08:51:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm_log_destination", 2024-04-21T08:51:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: "file", 2024-04-21T08:51:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: "file,syslog", 2024-04-21T08:51:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: "syslog" 2024-04-21T08:51:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-21T08:51:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: "cgroups_split": { 2024-04-21T08:51:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cgroups_split", 2024-04-21T08:51:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-21T08:51:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "config_checks_enabled": { 2024-04-21T08:51:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_checks_enabled", 2024-04-21T08:51:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-21T08:51:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "config_dashboard": { 2024-04-21T08:51:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_dashboard", 2024-04-21T08:51:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-21T08:51:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_alertmanager": { 2024-04-21T08:51:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_alertmanager", 2024-04-21T08:51:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-21T08:51:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-04-21T08:51:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_base": { 2024-04-21T08:51:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_base", 2024-04-21T08:51:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-21T08:51:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Container image name, without the tag", 2024-04-21T08:51:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_elasticsearch": { 2024-04-21T08:51:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_elasticsearch", 2024-04-21T08:51:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-21T08:51:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "elasticsearch container image", 2024-04-21T08:51:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_grafana": { 2024-04-21T08:51:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_grafana", 2024-04-21T08:51:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-21T08:51:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-04-21T08:51:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_haproxy": { 2024-04-21T08:51:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_haproxy", 2024-04-21T08:51:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-21T08:51:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "HAproxy container image", 2024-04-21T08:51:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_agent": { 2024-04-21T08:51:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_agent", 2024-04-21T08:51:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-21T08:51:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger agent container image", 2024-04-21T08:51:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_collector": { 2024-04-21T08:51:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_collector", 2024-04-21T08:51:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-21T08:51:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger collector container image", 2024-04-21T08:51:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_query": { 2024-04-21T08:51:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_query", 2024-04-21T08:51:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-21T08:51:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger query container image", 2024-04-21T08:51:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_keepalived": { 2024-04-21T08:51:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_keepalived", 2024-04-21T08:51:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-21T08:51:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Keepalived container image", 2024-04-21T08:51:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_loki": { 2024-04-21T08:51:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_loki", 2024-04-21T08:51:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-21T08:51:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Loki container image", 2024-04-21T08:51:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_node_exporter": { 2024-04-21T08:51:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_node_exporter", 2024-04-21T08:51:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-21T08:51:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-04-21T08:51:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_nvmeof": { 2024-04-21T08:51:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_nvmeof", 2024-04-21T08:51:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-21T08:51:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Nvme-of container image", 2024-04-21T08:51:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_prometheus": { 2024-04-21T08:51:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_prometheus", 2024-04-21T08:51:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-21T08:51:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-04-21T08:51:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_promtail": { 2024-04-21T08:51:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_promtail", 2024-04-21T08:51:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-21T08:51:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Promtail container image", 2024-04-21T08:51:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_samba": { 2024-04-21T08:51:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_samba", 2024-04-21T08:51:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-21T08:51:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Samba/SMB container image", 2024-04-21T08:51:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_snmp_gateway": { 2024-04-21T08:51:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_snmp_gateway", 2024-04-21T08:51:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-21T08:51:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SNMP Gateway container image", 2024-04-21T08:51:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "container_init": { 2024-04-21T08:51:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_init", 2024-04-21T08:51:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Run podman/docker with `--init`", 2024-04-21T08:51:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: "daemon_cache_timeout": { 2024-04-21T08:51:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "daemon_cache_timeout", 2024-04-21T08:51:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-21T08:51:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-21T08:51:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: "default_cephadm_command_timeout": { 2024-04-21T08:51:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_cephadm_command_timeout", 2024-04-21T08:51:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "900", 2024-04-21T08:51:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-21T08:51:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: "default_registry": { 2024-04-21T08:51:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_registry", 2024-04-21T08:51:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io", 2024-04-21T08:51:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-21T08:51:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: "device_cache_timeout": { 2024-04-21T08:51:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_cache_timeout", 2024-04-21T08:51:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1800", 2024-04-21T08:51:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache device inventory", 2024-04-21T08:51:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: "device_enhanced_scan": { 2024-04-21T08:51:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_enhanced_scan", 2024-04-21T08:51:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-21T08:51:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: "facts_cache_timeout": { 2024-04-21T08:51:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "facts_cache_timeout", 2024-04-21T08:51:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-21T08:51:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache host facts data", 2024-04-21T08:51:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: "grafana_dashboards_path": { 2024-04-21T08:51:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "grafana_dashboards_path", 2024-04-21T08:51:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-21T08:51:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-21T08:51:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: "host_check_interval": { 2024-04-21T08:51:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "host_check_interval", 2024-04-21T08:51:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-21T08:51:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to perform a host check", 2024-04-21T08:51:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: "hw_monitoring": { 2024-04-21T08:51:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hw_monitoring", 2024-04-21T08:51:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-21T08:51:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: "inventory_list_all": { 2024-04-21T08:51:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "inventory_list_all", 2024-04-21T08:51:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-21T08:51:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: "log_refresh_metadata": { 2024-04-21T08:51:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_refresh_metadata", 2024-04-21T08:51:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-21T08:51:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-21T08:51:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-21T08:51:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-21T08:51:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-21T08:51:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.858 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-21T08:51:48.858 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-21T08:51:48.858 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.858 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.858 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.858 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "*", 2024-04-21T08:51:48.858 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.858 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.858 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.858 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-21T08:51:48.858 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.858 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.859 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.859 INFO:teuthology.orchestra.run.smithi043.stdout: "max_count_per_host": { 2024-04-21T08:51:48.859 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_count_per_host", 2024-04-21T08:51:48.859 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.859 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.859 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.859 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-21T08:51:48.859 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.860 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.860 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.860 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of daemons per service per host", 2024-04-21T08:51:48.860 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.860 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.860 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.860 INFO:teuthology.orchestra.run.smithi043.stdout: "max_osd_draining_count": { 2024-04-21T08:51:48.860 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_osd_draining_count", 2024-04-21T08:51:48.860 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.861 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.861 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.861 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-21T08:51:48.861 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.861 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.861 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.861 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-21T08:51:48.861 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.861 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.861 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.862 INFO:teuthology.orchestra.run.smithi043.stdout: "migration_current": { 2024-04-21T08:51:48.862 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "migration_current", 2024-04-21T08:51:48.862 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.862 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.862 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.862 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.862 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.862 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.862 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.862 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "internal - do not modify", 2024-04-21T08:51:48.862 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.862 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.862 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-04-21T08:51:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-04-21T08:51:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "root", 2024-04-21T08:51:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm-package", 2024-04-21T08:51:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: "root" 2024-04-21T08:51:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "mode for remote execution of cephadm", 2024-04-21T08:51:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: "oob_default_addr": { 2024-04-21T08:51:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "oob_default_addr", 2024-04-21T08:51:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "169.254.1.1", 2024-04-21T08:51:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-21T08:51:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: "prometheus_alerts_path": { 2024-04-21T08:51:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus_alerts_path", 2024-04-21T08:51:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-21T08:51:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-21T08:51:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_insecure": { 2024-04-21T08:51:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_insecure", 2024-04-21T08:51:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-21T08:51:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_password": { 2024-04-21T08:51:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_password", 2024-04-21T08:51:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-21T08:51:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_url": { 2024-04-21T08:51:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_url", 2024-04-21T08:51:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-21T08:51:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_username": { 2024-04-21T08:51:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_username", 2024-04-21T08:51:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-21T08:51:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: "secure_monitoring_stack": { 2024-04-21T08:51:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "secure_monitoring_stack", 2024-04-21T08:51:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-21T08:51:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: "service_discovery_port": { 2024-04-21T08:51:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "service_discovery_port", 2024-04-21T08:51:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8765", 2024-04-21T08:51:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "cephadm service discovery port", 2024-04-21T08:51:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: "ssh_config_file": { 2024-04-21T08:51:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssh_config_file", 2024-04-21T08:51:48.872 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.872 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.872 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.872 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.872 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.872 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.872 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.872 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-21T08:51:48.872 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.872 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.872 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.872 INFO:teuthology.orchestra.run.smithi043.stdout: "use_agent": { 2024-04-21T08:51:48.872 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_agent", 2024-04-21T08:51:48.872 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-21T08:51:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: "use_repo_digest": { 2024-04-21T08:51:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_repo_digest", 2024-04-21T08:51:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-21T08:51:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_failed_host_check": { 2024-04-21T08:51:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_failed_host_check", 2024-04-21T08:51:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if the host check fails", 2024-04-21T08:51:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_daemons": { 2024-04-21T08:51:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_daemons", 2024-04-21T08:51:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-21T08:51:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_hosts": { 2024-04-21T08:51:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_hosts", 2024-04-21T08:51:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-21T08:51:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crash", 2024-04-21T08:51:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: "retain_interval": { 2024-04-21T08:51:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retain_interval", 2024-04-21T08:51:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "31536000", 2024-04-21T08:51:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-21T08:51:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_recent_interval": { 2024-04-21T08:51:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_recent_interval", 2024-04-21T08:51:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1209600", 2024-04-21T08:51:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-21T08:51:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dashboard", 2024-04-21T08:51:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-21T08:51:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-21T08:51:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-21T08:51:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_HOST": { 2024-04-21T08:51:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-21T08:51:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.887 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.887 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.887 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.887 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.887 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.887 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.887 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-21T08:51:48.888 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-21T08:51:48.888 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.888 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.888 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.888 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.888 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.888 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.888 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.888 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.888 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.889 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.889 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.889 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_ENABLED": { 2024-04-21T08:51:48.889 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_ENABLED", 2024-04-21T08:51:48.889 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.889 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.889 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.889 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.889 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.890 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.890 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.890 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.890 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.890 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.890 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.890 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-21T08:51:48.890 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-21T08:51:48.891 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.891 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.891 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.891 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.891 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.891 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.891 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.891 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.891 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.891 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.891 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.892 INFO:teuthology.orchestra.run.smithi043.stdout: "ENABLE_BROWSABLE_API": { 2024-04-21T08:51:48.892 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-21T08:51:48.892 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.892 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.892 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.892 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.892 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.892 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-21T08:51:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-21T08:51:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.894 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.894 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.894 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.894 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.894 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.894 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.894 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.894 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.895 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-21T08:51:48.895 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-21T08:51:48.895 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.895 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.895 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.895 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.896 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.896 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.896 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.896 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.896 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.896 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.896 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.896 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-21T08:51:48.897 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-21T08:51:48.897 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.897 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.897 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.897 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.897 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.897 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.897 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.897 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.898 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.898 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.898 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.898 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-21T08:51:48.898 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-21T08:51:48.898 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.898 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.899 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.899 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.899 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.899 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.899 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.899 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.899 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.899 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.899 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.900 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-21T08:51:48.900 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-21T08:51:48.900 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.900 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.900 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.900 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.900 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.901 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.901 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.901 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.901 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.901 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.901 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.901 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-21T08:51:48.902 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-21T08:51:48.902 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.902 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.902 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.902 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.902 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.902 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.902 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.902 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.903 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.903 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.903 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.903 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-21T08:51:48.903 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-21T08:51:48.903 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.903 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.903 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.904 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.904 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.904 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.904 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.904 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.904 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.904 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.904 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.905 INFO:teuthology.orchestra.run.smithi043.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-21T08:51:48.905 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-21T08:51:48.905 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.905 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.905 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.905 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.905 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.905 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.906 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.906 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.906 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.906 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.906 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.906 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_PASSWORD": { 2024-04-21T08:51:48.906 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-21T08:51:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-04-21T08:51:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.908 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.908 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.908 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-21T08:51:48.908 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-21T08:51:48.908 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.908 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.908 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.909 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.909 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.909 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.909 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.909 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.909 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.909 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.909 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.910 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_URL": { 2024-04-21T08:51:48.910 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_URL", 2024-04-21T08:51:48.910 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.910 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.910 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.910 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.910 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.910 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.910 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.911 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.911 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.911 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.911 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.911 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_USERNAME": { 2024-04-21T08:51:48.911 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-21T08:51:48.911 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.911 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.911 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.912 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-04-21T08:51:48.912 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.912 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.912 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.912 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.912 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.912 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.912 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.912 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-21T08:51:48.913 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-21T08:51:48.913 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.913 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.913 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.913 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.913 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.913 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.913 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.913 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.914 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.914 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.914 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.914 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-21T08:51:48.914 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-21T08:51:48.914 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.914 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.914 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.915 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.915 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.915 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.915 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.915 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.915 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.915 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.915 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.915 INFO:teuthology.orchestra.run.smithi043.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-21T08:51:48.916 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-21T08:51:48.916 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.916 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.916 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.916 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.916 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.916 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.916 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.916 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.916 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-21T08:51:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-21T08:51:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.918 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.918 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.918 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.918 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.918 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.918 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.918 INFO:teuthology.orchestra.run.smithi043.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-21T08:51:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-21T08:51:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "[]", 2024-04-21T08:51:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: "MULTICLUSTER_CONFIG": { 2024-04-21T08:51:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-21T08:51:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "{}", 2024-04-21T08:51:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.921 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.921 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.921 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.921 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.921 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.921 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.921 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_HOST": { 2024-04-21T08:51:48.921 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-21T08:51:48.921 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.922 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.922 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.922 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.922 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.922 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.922 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.922 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.922 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.922 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.923 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.923 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-21T08:51:48.923 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-21T08:51:48.923 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.923 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.923 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.923 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.923 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.923 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.924 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.924 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.924 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.924 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.924 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.924 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-21T08:51:48.924 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-21T08:51:48.924 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.924 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.926 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-21T08:51:48.926 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-21T08:51:48.926 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.926 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.926 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.926 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.926 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.926 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.926 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.926 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.927 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.927 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.927 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.927 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-21T08:51:48.927 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-21T08:51:48.927 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.927 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.927 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.927 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-21T08:51:48.929 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-21T08:51:48.929 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.929 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.929 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.929 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.929 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.929 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.929 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.929 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.929 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.929 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.930 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.930 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-21T08:51:48.930 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-21T08:51:48.930 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.930 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.930 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.930 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.931 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.931 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.931 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.931 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.931 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.931 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.931 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.931 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-21T08:51:48.932 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-21T08:51:48.932 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.932 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.932 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.932 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.932 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.932 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.932 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.932 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.932 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.932 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.933 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.933 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-21T08:51:48.933 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-21T08:51:48.933 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.933 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.933 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.933 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.933 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.933 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_ENABLED": { 2024-04-21T08:51:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-21T08:51:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-21T08:51:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-21T08:51:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-21T08:51:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-21T08:51:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-21T08:51:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-21T08:51:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-21T08:51:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-21T08:51:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8", 2024-04-21T08:51:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-21T08:51:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-21T08:51:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "45", 2024-04-21T08:51:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ACCESS_KEY": { 2024-04-21T08:51:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-21T08:51:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-21T08:51:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-21T08:51:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-04-21T08:51:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SECRET_KEY": { 2024-04-21T08:51:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-21T08:51:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SSL_VERIFY": { 2024-04-21T08:51:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-21T08:51:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: "UNSAFE_TLS_v1_2": { 2024-04-21T08:51:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-21T08:51:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-21T08:51:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-21T08:51:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-21T08:51:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-21T08:51:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-21T08:51:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-21T08:51:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-21T08:51:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-21T08:51:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-04-21T08:51:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: "cross_origin_url": { 2024-04-21T08:51:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cross_origin_url", 2024-04-21T08:51:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: "crt_file": { 2024-04-21T08:51:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crt_file", 2024-04-21T08:51:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: "debug": { 2024-04-21T08:51:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "debug", 2024-04-21T08:51:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable/disable debug options", 2024-04-21T08:51:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "jwt_token_ttl": { 2024-04-21T08:51:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "jwt_token_ttl", 2024-04-21T08:51:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "28800", 2024-04-21T08:51:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-04-21T08:51:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-04-21T08:51:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: "motd": { 2024-04-21T08:51:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "motd", 2024-04-21T08:51:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "The message of the day", 2024-04-21T08:51:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect_resolve_ip_addr": { 2024-04-21T08:51:48.958 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "redirect_resolve_ip_addr", 2024-04-21T08:51:48.958 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.958 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.958 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.958 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.958 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.958 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.958 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.958 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.958 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.958 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.958 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-04-21T08:51:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-04-21T08:51:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-04-21T08:51:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-04-21T08:51:48.960 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-04-21T08:51:48.960 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.960 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.960 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.960 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8080", 2024-04-21T08:51:48.960 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.960 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.960 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.960 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.960 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.960 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.960 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-04-21T08:51:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-04-21T08:51:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl_server_port": { 2024-04-21T08:51:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl_server_port", 2024-04-21T08:51:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8443", 2024-04-21T08:51:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-04-21T08:51:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-04-21T08:51:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "redirect", 2024-04-21T08:51:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect" 2024-04-21T08:51:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-04-21T08:51:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-04-21T08:51:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-04-21T08:51:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-04-21T08:51:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-04-21T08:51:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: "url_prefix": { 2024-04-21T08:51:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url_prefix", 2024-04-21T08:51:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "devicehealth", 2024-04-21T08:51:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_monitoring": { 2024-04-21T08:51:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_monitoring", 2024-04-21T08:51:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "monitor device health metrics", 2024-04-21T08:51:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: "mark_out_threshold": { 2024-04-21T08:51:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mark_out_threshold", 2024-04-21T08:51:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2419200", 2024-04-21T08:51:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-21T08:51:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": { 2024-04-21T08:51:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_name", 2024-04-21T08:51:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "device_health_metrics", 2024-04-21T08:51:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-21T08:51:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: "retention_period": { 2024-04-21T08:51:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retention_period", 2024-04-21T08:51:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15552000", 2024-04-21T08:51:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.975 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.975 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain device health metrics", 2024-04-21T08:51:48.975 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.975 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.975 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.975 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_frequency": { 2024-04-21T08:51:48.975 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_frequency", 2024-04-21T08:51:48.975 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.975 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.975 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-04-21T08:51:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-21T08:51:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: "self_heal": { 2024-04-21T08:51:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "self_heal", 2024-04-21T08:51:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-21T08:51:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-21T08:51:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-21T08:51:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-21T08:51:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and check device health", 2024-04-21T08:51:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_threshold": { 2024-04-21T08:51:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_threshold", 2024-04-21T08:51:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7257600", 2024-04-21T08:51:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-21T08:51:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "diskprediction_local", 2024-04-21T08:51:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: "predict_interval": { 2024-04-21T08:51:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predict_interval", 2024-04-21T08:51:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-04-21T08:51:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: "predictor_model": { 2024-04-21T08:51:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predictor_model", 2024-04-21T08:51:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "prophetstor", 2024-04-21T08:51:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-21T08:51:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-21T08:51:48.987 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.987 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.987 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.987 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-04-21T08:51:48.987 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.987 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.987 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.987 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.987 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.987 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.987 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.987 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.987 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:48.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:48.987 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "influx", 2024-04-21T08:51:48.988 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": false, 2024-04-21T08:51:48.988 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "influxdb python module not found", 2024-04-21T08:51:48.988 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:48.988 INFO:teuthology.orchestra.run.smithi043.stdout: "batch_size": { 2024-04-21T08:51:48.988 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "batch_size", 2024-04-21T08:51:48.988 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.988 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.988 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.988 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5000", 2024-04-21T08:51:48.988 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.988 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.988 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.988 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-21T08:51:48.988 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.988 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.989 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.989 INFO:teuthology.orchestra.run.smithi043.stdout: "database": { 2024-04-21T08:51:48.989 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "database", 2024-04-21T08:51:48.989 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.989 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.989 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.989 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "ceph", 2024-04-21T08:51:48.989 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.989 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.989 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.989 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-21T08:51:48.989 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.990 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.990 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.990 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": { 2024-04-21T08:51:48.990 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hostname", 2024-04-21T08:51:48.990 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.990 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.990 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.990 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.990 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.990 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.990 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.990 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server hostname", 2024-04-21T08:51:48.990 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.991 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.991 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.991 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-21T08:51:48.991 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-21T08:51:48.991 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:48.991 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.991 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.991 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "30", 2024-04-21T08:51:48.991 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "5", 2024-04-21T08:51:48.991 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.991 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.991 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-21T08:51:48.991 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.991 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.991 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.992 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:48.992 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:48.992 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.992 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.992 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.992 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.992 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.992 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.992 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.992 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.992 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.992 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.992 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.992 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.993 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.993 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.993 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.993 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.993 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.993 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.993 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:48.993 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:48.993 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.993 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.993 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.993 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.993 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.993 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.993 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.993 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.994 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.994 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.994 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.994 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:48.994 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:48.994 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.994 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.994 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.994 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:48.994 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.994 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.994 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:48.994 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:48.995 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:48.995 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:48.995 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:48.995 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:48.995 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:48.995 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:48.995 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.995 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.995 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.995 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.995 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:48.995 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:48.995 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:48.995 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.996 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:48.996 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:48.996 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.996 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.996 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.996 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:48.996 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.996 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.996 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.996 INFO:teuthology.orchestra.run.smithi043.stdout: "password": { 2024-04-21T08:51:48.996 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "password", 2024-04-21T08:51:48.996 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.996 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.996 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.997 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:48.997 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.997 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.997 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.997 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "password of InfluxDB server user", 2024-04-21T08:51:48.997 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.997 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.997 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.997 INFO:teuthology.orchestra.run.smithi043.stdout: "port": { 2024-04-21T08:51:48.997 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "port", 2024-04-21T08:51:48.997 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.997 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.997 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.997 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8086", 2024-04-21T08:51:48.997 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.998 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.998 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.998 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server port", 2024-04-21T08:51:48.998 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.998 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.998 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.998 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-04-21T08:51:48.998 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-04-21T08:51:48.998 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:48.998 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.998 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.998 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "false", 2024-04-21T08:51:48.998 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:48.999 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:48.999 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:48.999 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T08:51:48.999 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:48.999 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:48.999 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:48.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:48.999 INFO:teuthology.orchestra.run.smithi043.stdout: "threads": { 2024-04-21T08:51:48.999 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threads", 2024-04-21T08:51:48.999 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:48.999 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:48.999 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:48.999 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-04-21T08:51:48.999 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-04-21T08:51:48.999 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "32", 2024-04-21T08:51:48.999 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.000 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-21T08:51:49.000 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.000 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.000 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.000 INFO:teuthology.orchestra.run.smithi043.stdout: "username": { 2024-04-21T08:51:49.000 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "username", 2024-04-21T08:51:49.000 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.000 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.000 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.000 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.000 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.000 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.000 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.000 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "username of InfluxDB server user", 2024-04-21T08:51:49.000 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.001 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.001 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.001 INFO:teuthology.orchestra.run.smithi043.stdout: "verify_ssl": { 2024-04-21T08:51:49.001 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "verify_ssl", 2024-04-21T08:51:49.001 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.001 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.001 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.001 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "true", 2024-04-21T08:51:49.001 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.001 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.001 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.001 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T08:51:49.001 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.001 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.001 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.002 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.002 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.002 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "insights", 2024-04-21T08:51:49.002 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.002 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.002 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.002 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.002 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.002 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.002 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.002 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.002 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.002 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.003 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.003 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.003 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.003 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.003 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.003 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.003 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.003 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.003 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.003 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.003 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.003 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.003 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.004 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.004 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.004 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.004 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.004 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.004 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.004 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.004 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.004 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.004 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.004 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.004 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.004 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.004 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.005 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.005 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.005 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.005 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.005 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.005 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.005 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.005 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.005 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.005 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.005 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.005 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.005 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.005 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.006 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.006 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.006 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.006 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.006 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.006 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.006 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.006 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.006 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.006 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.006 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.006 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.007 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.007 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.007 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.007 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.007 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.007 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.007 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.007 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.007 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "iostat", 2024-04-21T08:51:49.007 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.007 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.007 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.007 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.008 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.008 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.008 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.008 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.008 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.008 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.008 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.008 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.008 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.008 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.008 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.008 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.008 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "k8sevents", 2024-04-21T08:51:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_event_retention_days": { 2024-04-21T08:51:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ceph_event_retention_days", 2024-04-21T08:51:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7", 2024-04-21T08:51:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-21T08:51:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: "config_check_secs": { 2024-04-21T08:51:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_check_secs", 2024-04-21T08:51:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-21T08:51:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "10", 2024-04-21T08:51:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-21T08:51:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.020 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.020 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.020 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.020 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.020 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.020 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.020 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.020 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.020 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.020 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.020 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.020 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.021 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "localpool", 2024-04-21T08:51:49.021 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.021 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.021 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.021 INFO:teuthology.orchestra.run.smithi043.stdout: "failure_domain": { 2024-04-21T08:51:49.021 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "failure_domain", 2024-04-21T08:51:49.021 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.021 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.021 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.021 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "host", 2024-04-21T08:51:49.021 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.021 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.021 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.021 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "failure domain for any created local pool", 2024-04-21T08:51:49.022 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-21T08:51:49.022 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.022 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.022 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.022 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.022 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.022 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.022 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.022 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.022 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.022 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.022 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.023 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.023 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.023 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.023 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.023 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.023 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.023 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.023 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.023 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.023 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.023 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.023 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.024 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.024 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.024 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.024 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.024 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.024 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.024 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.024 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.024 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.024 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.024 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.024 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.024 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.024 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.025 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.025 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.025 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.025 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.025 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.025 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.025 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.025 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.025 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.025 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.025 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.025 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.025 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.025 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.025 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.026 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.026 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.026 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.026 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.026 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.026 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.026 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.026 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.026 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.026 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.026 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.026 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.027 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.027 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.027 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.027 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.027 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": { 2024-04-21T08:51:49.027 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_size", 2024-04-21T08:51:49.027 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:49.027 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.027 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.027 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.027 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.027 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.027 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.027 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default min_size for any created local pool", 2024-04-21T08:51:49.028 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-21T08:51:49.028 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.028 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.028 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rep": { 2024-04-21T08:51:49.028 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "num_rep", 2024-04-21T08:51:49.028 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:49.028 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.028 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.028 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3", 2024-04-21T08:51:49.028 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.028 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.028 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.028 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default replica count for any created local pool", 2024-04-21T08:51:49.028 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.029 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.029 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.029 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": { 2024-04-21T08:51:49.029 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_num", 2024-04-21T08:51:49.029 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:49.029 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.029 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.029 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "128", 2024-04-21T08:51:49.029 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.029 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.029 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.029 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default pg_num for any created local pool", 2024-04-21T08:51:49.029 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.030 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.030 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.030 INFO:teuthology.orchestra.run.smithi043.stdout: "prefix": { 2024-04-21T08:51:49.030 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prefix", 2024-04-21T08:51:49.030 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.030 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.030 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.030 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.030 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.030 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.030 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.030 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name prefix for any created local pool", 2024-04-21T08:51:49.031 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.031 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.031 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.031 INFO:teuthology.orchestra.run.smithi043.stdout: "subtree": { 2024-04-21T08:51:49.031 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "subtree", 2024-04-21T08:51:49.031 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.031 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.031 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.031 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "rack", 2024-04-21T08:51:49.031 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.031 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.031 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.031 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-21T08:51:49.031 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-21T08:51:49.031 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.032 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.032 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.032 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.032 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mds_autoscaler", 2024-04-21T08:51:49.032 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.032 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.032 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.032 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.032 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.032 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.032 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.032 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.032 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.032 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.033 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.033 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.033 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.033 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.033 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.033 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.033 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.033 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.033 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.033 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.033 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.033 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.033 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.033 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.034 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.034 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.034 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.034 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.034 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.034 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.034 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.034 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.034 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.034 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.034 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.034 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.034 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.035 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.035 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.035 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.035 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.035 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.035 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.035 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.035 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.035 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.035 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.035 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.035 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.035 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.035 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.035 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.036 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.036 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.036 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.036 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.036 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.036 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.036 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.036 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.036 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.036 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.036 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.036 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.036 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.036 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.037 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.037 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.037 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.037 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirroring", 2024-04-21T08:51:49.037 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.037 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.037 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.037 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.037 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.037 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.037 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.038 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.038 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.038 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.038 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.038 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.038 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.038 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.038 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.038 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.038 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.038 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.038 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.038 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.038 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.038 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.039 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.039 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.039 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.039 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.039 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.039 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.039 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.039 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.039 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.039 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.039 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.039 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.039 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.039 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.040 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.040 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.040 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.040 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.040 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.040 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.040 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.040 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.040 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.040 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.040 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.040 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.040 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.040 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.040 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.041 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.041 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.041 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.041 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.041 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.041 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.041 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.041 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.041 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.041 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.041 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.041 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.041 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.041 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.042 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.042 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.042 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.042 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.042 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.042 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.042 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "nfs", 2024-04-21T08:51:49.042 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.042 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.042 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.042 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.042 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.043 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.043 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.043 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.043 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.043 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.043 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.043 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.043 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.043 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.043 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.043 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.043 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.043 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.043 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.044 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.044 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.044 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.044 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.044 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.044 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.044 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.044 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.044 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.044 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.044 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.045 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.045 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.045 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.045 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.045 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.045 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.045 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.046 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.046 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.046 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.046 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.046 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.046 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.046 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.046 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.046 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.046 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.046 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.046 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.047 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.047 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.047 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.047 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.047 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.047 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.047 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.047 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.047 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.047 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.047 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.047 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.047 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.048 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.048 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.048 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.048 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.048 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.048 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.048 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.048 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.048 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.048 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-04-21T08:51:49.048 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.048 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.049 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.049 INFO:teuthology.orchestra.run.smithi043.stdout: "fail_fs": { 2024-04-21T08:51:49.049 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "fail_fs", 2024-04-21T08:51:49.049 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.049 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.049 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.049 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.049 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.049 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.049 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.049 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-21T08:51:49.049 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.049 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.049 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.050 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.050 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.050 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.050 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.050 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.050 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.050 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.050 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.050 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.050 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.050 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.050 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.050 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.050 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.051 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.051 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.051 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.051 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.051 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.051 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.051 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.051 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.051 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.051 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.051 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.051 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.051 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.051 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.052 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.052 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.052 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.052 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.052 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.052 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.052 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.052 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.052 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.052 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.052 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.052 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.052 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.052 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.053 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.053 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.053 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.053 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.053 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.053 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.053 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.053 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.053 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.053 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.053 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.053 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.053 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.053 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.054 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.054 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.054 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.054 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.054 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.054 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.054 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.054 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.054 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.054 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.054 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator": { 2024-04-21T08:51:49.054 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-04-21T08:51:49.055 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.055 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.055 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.055 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.055 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.055 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.055 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.055 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm", 2024-04-21T08:51:49.055 INFO:teuthology.orchestra.run.smithi043.stdout: "rook", 2024-04-21T08:51:49.055 INFO:teuthology.orchestra.run.smithi043.stdout: "test_orchestrator" 2024-04-21T08:51:49.055 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.055 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Orchestrator backend", 2024-04-21T08:51:49.055 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.055 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.055 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.056 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.056 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.056 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_perf_query", 2024-04-21T08:51:49.056 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.056 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.056 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.056 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.056 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.056 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.056 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.056 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.056 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.056 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.057 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.057 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.057 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.057 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.057 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.057 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.057 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.057 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.057 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.057 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.057 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.057 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.057 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.057 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.058 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.058 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.058 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.058 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.058 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.058 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.058 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.058 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.058 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.058 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.058 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.058 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.058 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.059 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.059 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.059 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.059 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.059 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.059 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.059 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.059 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.059 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.059 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.059 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.059 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.059 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.059 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.059 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.060 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.060 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.060 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.060 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.060 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.060 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.060 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.060 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.060 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.060 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.060 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.060 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.060 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.061 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.061 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.061 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.061 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.061 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.061 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.061 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_support", 2024-04-21T08:51:49.061 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.061 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.061 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.061 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.061 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.062 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.062 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.062 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.062 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.062 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.062 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.062 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.062 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.062 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.062 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.062 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.062 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.062 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.062 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.063 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.063 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.063 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.063 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.063 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.063 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.063 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.063 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.063 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.063 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.063 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.063 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.063 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.063 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.063 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.064 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.064 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.064 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.064 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.064 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.064 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.064 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.064 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.064 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.064 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.064 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.064 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.064 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.064 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.064 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.065 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.065 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.065 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.065 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.065 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.065 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.065 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.065 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.065 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.065 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.065 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.065 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.065 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.066 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.066 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.066 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.066 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.066 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.066 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.066 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.066 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.066 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.066 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_autoscaler", 2024-04-21T08:51:49.066 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.066 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.066 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.067 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.067 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.067 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.067 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.067 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.067 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.067 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.067 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.067 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.067 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.067 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.067 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.067 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.067 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.067 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.067 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.068 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.068 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.068 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.068 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.068 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.068 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.068 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.068 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.068 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.068 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.068 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.068 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.068 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.068 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.068 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.069 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.069 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.069 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.069 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.069 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.069 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.069 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.069 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.069 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.069 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.069 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.069 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.069 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.069 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.069 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.070 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.070 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.070 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.070 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.070 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.070 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.070 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.070 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.070 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.070 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.070 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.070 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.071 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.071 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.071 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.071 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.071 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.071 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.071 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.071 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.071 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-21T08:51:49.071 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-21T08:51:49.071 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:49.071 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.071 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.071 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-21T08:51:49.072 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.072 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.072 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.072 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.072 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.072 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.072 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.072 INFO:teuthology.orchestra.run.smithi043.stdout: "threshold": { 2024-04-21T08:51:49.072 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threshold", 2024-04-21T08:51:49.072 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-21T08:51:49.072 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.072 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.072 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-04-21T08:51:49.072 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1.0", 2024-04-21T08:51:49.072 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.072 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.073 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "scaling threshold", 2024-04-21T08:51:49.073 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-21T08:51:49.073 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.073 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.073 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.073 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.073 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "progress", 2024-04-21T08:51:49.073 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.073 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.073 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.073 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_pg_recovery_event": { 2024-04-21T08:51:49.073 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_pg_recovery_event", 2024-04-21T08:51:49.073 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.073 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.074 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.074 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.074 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.074 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.074 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.074 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-21T08:51:49.074 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.074 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.074 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.074 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-04-21T08:51:49.074 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-04-21T08:51:49.074 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.074 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.075 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.075 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:49.075 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.075 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.075 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.075 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.075 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.075 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.075 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.075 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.075 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.075 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.075 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.076 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.076 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.076 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.076 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.076 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.076 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.076 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.076 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.076 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.076 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.076 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.076 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.076 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.076 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.076 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.077 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.077 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.077 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.077 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.077 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.077 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.077 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.077 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.077 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.077 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.077 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.077 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.077 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.077 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.078 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.078 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.078 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.078 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.078 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.078 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.078 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.078 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.078 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.078 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.078 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.078 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.078 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.078 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.079 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.079 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.079 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.079 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.079 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.079 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.079 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.079 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.079 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.079 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.079 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.079 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.079 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.079 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.080 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.080 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.080 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.080 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.080 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.080 INFO:teuthology.orchestra.run.smithi043.stdout: "max_completed_events": { 2024-04-21T08:51:49.080 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_completed_events", 2024-04-21T08:51:49.080 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:49.080 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.080 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.080 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "50", 2024-04-21T08:51:49.080 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.080 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.080 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.081 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "number of past completed events to remember", 2024-04-21T08:51:49.081 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.081 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.081 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.081 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-04-21T08:51:49.081 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-04-21T08:51:49.081 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:49.081 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.081 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.081 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-04-21T08:51:49.081 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.081 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.081 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.081 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long the module is going to sleep", 2024-04-21T08:51:49.082 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.082 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.082 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.082 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.082 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.082 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus", 2024-04-21T08:51:49.082 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.082 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.082 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.082 INFO:teuthology.orchestra.run.smithi043.stdout: "cache": { 2024-04-21T08:51:49.082 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cache", 2024-04-21T08:51:49.082 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.083 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.083 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.083 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:49.083 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.083 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.083 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.083 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.083 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.083 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.083 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.083 INFO:teuthology.orchestra.run.smithi043.stdout: "exclude_perf_counters": { 2024-04-21T08:51:49.083 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "exclude_perf_counters", 2024-04-21T08:51:49.083 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.083 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.084 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.084 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:49.084 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.084 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.084 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.084 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-21T08:51:49.084 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-21T08:51:49.084 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.084 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.084 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.084 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.084 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.084 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.085 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.085 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.085 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.085 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.085 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.085 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.085 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.085 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.085 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.085 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.085 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.085 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.085 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.085 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.085 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.086 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.086 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.086 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.086 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.086 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.086 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.086 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.086 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.086 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.086 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.086 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.086 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.088 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.088 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.088 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.088 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.088 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.088 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.088 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.088 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.088 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.088 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.088 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.088 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.088 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.088 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools": { 2024-04-21T08:51:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools", 2024-04-21T08:51:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-21T08:51:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-21T08:51:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "300", 2024-04-21T08:51:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_interval": { 2024-04-21T08:51:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_interval", 2024-04-21T08:51:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-21T08:51:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15.0", 2024-04-21T08:51:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-04-21T08:51:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-04-21T08:51:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-04-21T08:51:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-21T08:51:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-04-21T08:51:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-04-21T08:51:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "9283", 2024-04-21T08:51:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-21T08:51:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: "stale_cache_strategy": { 2024-04-21T08:51:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stale_cache_strategy", 2024-04-21T08:51:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "log", 2024-04-21T08:51:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-04-21T08:51:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-04-21T08:51:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "default", 2024-04-21T08:51:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: "default", 2024-04-21T08:51:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: "error" 2024-04-21T08:51:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-04-21T08:51:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-04-21T08:51:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-04-21T08:51:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-04-21T08:51:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-04-21T08:51:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_support", 2024-04-21T08:51:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.098 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.098 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.098 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.098 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.098 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.098 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.098 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.098 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.098 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.098 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.098 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.098 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.098 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.098 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.102 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.102 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.102 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.102 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.102 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.102 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.102 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.102 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.102 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.102 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_snap_create": { 2024-04-21T08:51:49.102 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_snap_create", 2024-04-21T08:51:49.102 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:49.102 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.103 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.103 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-04-21T08:51:49.103 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.103 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.103 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.103 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.103 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.103 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.103 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.103 INFO:teuthology.orchestra.run.smithi043.stdout: "mirror_snapshot_schedule": { 2024-04-21T08:51:49.103 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirror_snapshot_schedule", 2024-04-21T08:51:49.103 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.103 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.103 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.104 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.104 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.104 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.104 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.104 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.104 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.104 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.104 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.104 INFO:teuthology.orchestra.run.smithi043.stdout: "trash_purge_schedule": { 2024-04-21T08:51:49.104 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "trash_purge_schedule", 2024-04-21T08:51:49.104 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.104 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.104 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.105 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.105 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.105 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.105 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.105 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.105 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.105 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.105 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.105 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.105 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.105 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "restful", 2024-04-21T08:51:49.105 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_auth": { 2024-04-21T08:51:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_auth", 2024-04-21T08:51:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-04-21T08:51:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-04-21T08:51:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-04-21T08:51:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-04-21T08:51:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-04-21T08:51:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-04-21T08:51:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rgw", 2024-04-21T08:51:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.119 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.119 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.119 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.119 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.119 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.119 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.119 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.119 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.119 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.119 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.119 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.119 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.119 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rook", 2024-04-21T08:51:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: "storage_class": { 2024-04-21T08:51:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "storage_class", 2024-04-21T08:51:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "local", 2024-04-21T08:51:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-21T08:51:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "selftest", 2024-04-21T08:51:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.130 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.130 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.130 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.130 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.130 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.130 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.130 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.130 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.130 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.130 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.130 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: "roption1": { 2024-04-21T08:51:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption1", 2024-04-21T08:51:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: "roption2": { 2024-04-21T08:51:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption2", 2024-04-21T08:51:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "xyz", 2024-04-21T08:51:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption1": { 2024-04-21T08:51:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption1", 2024-04-21T08:51:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.135 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.135 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.135 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.135 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.135 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.135 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.135 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.135 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.135 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.137 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption2": { 2024-04-21T08:51:49.137 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption2", 2024-04-21T08:51:49.137 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption3": { 2024-04-21T08:51:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption3", 2024-04-21T08:51:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-04-21T08:51:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption4": { 2024-04-21T08:51:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption4", 2024-04-21T08:51:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption5": { 2024-04-21T08:51:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption5", 2024-04-21T08:51:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption6": { 2024-04-21T08:51:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption6", 2024-04-21T08:51:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption7": { 2024-04-21T08:51:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption7", 2024-04-21T08:51:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-04-21T08:51:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "42", 2024-04-21T08:51:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "testkey": { 2024-04-21T08:51:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testkey", 2024-04-21T08:51:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "testlkey": { 2024-04-21T08:51:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testlkey", 2024-04-21T08:51:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "testnewline": { 2024-04-21T08:51:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testnewline", 2024-04-21T08:51:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snap_schedule", 2024-04-21T08:51:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_m_granularity": { 2024-04-21T08:51:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_m_granularity", 2024-04-21T08:51:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow minute scheduled snapshots", 2024-04-21T08:51:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "dump_on_update": { 2024-04-21T08:51:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dump_on_update", 2024-04-21T08:51:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "dump database to debug log on update", 2024-04-21T08:51:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stats", 2024-04-21T08:51:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "status", 2024-04-21T08:51:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.161 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.161 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.161 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.161 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.161 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.161 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.161 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.161 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.161 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.161 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.161 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.161 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.161 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.161 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.162 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.162 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.162 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.162 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.162 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.162 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.162 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.162 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.162 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.162 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.162 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.162 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.162 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telegraf", 2024-04-21T08:51:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: "address": { 2024-04-21T08:51:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "address", 2024-04-21T08:51:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-21T08:51:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-21T08:51:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-21T08:51:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.165 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15", 2024-04-21T08:51:49.165 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.165 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.165 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.165 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.165 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.165 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.165 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.165 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.165 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.165 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.165 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.165 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.166 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.166 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.166 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.166 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.166 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.166 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.166 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.166 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.166 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.166 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.166 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.166 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.166 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telemetry", 2024-04-21T08:51:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_basic": { 2024-04-21T08:51:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_basic", 2024-04-21T08:51:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-21T08:51:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_crash": { 2024-04-21T08:51:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_crash", 2024-04-21T08:51:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:49.172 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.172 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.172 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.172 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-21T08:51:49.172 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.172 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.172 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.172 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_device": { 2024-04-21T08:51:49.172 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_device", 2024-04-21T08:51:49.172 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.172 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.172 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.172 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:49.172 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.173 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.173 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.173 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-21T08:51:49.173 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.173 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.173 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.173 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_ident": { 2024-04-21T08:51:49.173 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_ident", 2024-04-21T08:51:49.173 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.173 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.173 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.173 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.173 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-21T08:51:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_perf": { 2024-04-21T08:51:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_perf", 2024-04-21T08:51:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-21T08:51:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: "contact": { 2024-04-21T08:51:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "contact", 2024-04-21T08:51:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "description": { 2024-04-21T08:51:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "description", 2024-04-21T08:51:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "device_url": { 2024-04-21T08:51:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_url", 2024-04-21T08:51:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-21T08:51:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-04-21T08:51:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-04-21T08:51:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-21T08:51:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-21T08:51:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "24", 2024-04-21T08:51:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "8", 2024-04-21T08:51:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: "last_opt_revision": { 2024-04-21T08:51:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "last_opt_revision", 2024-04-21T08:51:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1", 2024-04-21T08:51:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard": { 2024-04-21T08:51:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard", 2024-04-21T08:51:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard_description": { 2024-04-21T08:51:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard_description", 2024-04-21T08:51:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.183 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.183 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.183 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.183 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.183 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.183 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.183 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.183 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.183 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.183 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.183 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.183 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.183 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.183 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.183 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: "organization": { 2024-04-21T08:51:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "organization", 2024-04-21T08:51:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: "proxy": { 2024-04-21T08:51:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "proxy", 2024-04-21T08:51:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: "url": { 2024-04-21T08:51:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url", 2024-04-21T08:51:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-21T08:51:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.190 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.190 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.190 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.190 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.190 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.190 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "test_orchestrator", 2024-04-21T08:51:49.190 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.190 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.190 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.190 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.190 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.190 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.190 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "volumes", 2024-04-21T08:51:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_clones": { 2024-04-21T08:51:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_clones", 2024-04-21T08:51:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4", 2024-04-21T08:51:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-21T08:51:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "periodic_async_work": { 2024-04-21T08:51:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "periodic_async_work", 2024-04-21T08:51:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Periodically check for async work", 2024-04-21T08:51:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_delay": { 2024-04-21T08:51:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_delay", 2024-04-21T08:51:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-04-21T08:51:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-21T08:51:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_no_wait": { 2024-04-21T08:51:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_no_wait", 2024-04-21T08:51:49.203 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.203 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.203 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.203 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-04-21T08:51:49.203 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.203 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.203 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.203 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-21T08:51:49.203 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.203 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.203 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.203 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.203 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.203 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix", 2024-04-21T08:51:49.203 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-04-21T08:51:49.204 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-04-21T08:51:49.204 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-04-21T08:51:49.204 INFO:teuthology.orchestra.run.smithi043.stdout: "discovery_interval": { 2024-04-21T08:51:49.204 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "discovery_interval", 2024-04-21T08:51:49.204 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-04-21T08:51:49.204 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.204 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.204 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "100", 2024-04-21T08:51:49.204 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.204 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.204 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.204 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.204 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.204 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.204 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.205 INFO:teuthology.orchestra.run.smithi043.stdout: "identifier": { 2024-04-21T08:51:49.205 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "identifier", 2024-04-21T08:51:49.205 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.205 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.205 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.205 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.205 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.205 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.205 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.205 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.205 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.205 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.205 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.206 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-04-21T08:51:49.206 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-04-21T08:51:49.206 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-04-21T08:51:49.206 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.206 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.206 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-04-21T08:51:49.206 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.206 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.206 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.206 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.206 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.206 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.206 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.206 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-04-21T08:51:49.207 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-04-21T08:51:49.207 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.207 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.207 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.207 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.207 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.207 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.207 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.207 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.207 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.207 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.207 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.207 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.207 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.207 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.207 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.208 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.208 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.208 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.208 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-04-21T08:51:49.208 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-04-21T08:51:49.208 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.208 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.208 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.208 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.208 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.208 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.208 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.208 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.208 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.209 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.209 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.209 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-04-21T08:51:49.209 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:49.209 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.209 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.209 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.209 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-04-21T08:51:49.209 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.209 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.209 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-04-21T08:51:49.209 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-04-21T08:51:49.209 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-04-21T08:51:49.209 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-04-21T08:51:49.209 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-04-21T08:51:49.209 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-04-21T08:51:49.210 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-04-21T08:51:49.210 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.210 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.210 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.210 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.210 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.210 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-04-21T08:51:49.210 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-04-21T08:51:49.210 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-04-21T08:51:49.210 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.210 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-04-21T08:51:49.210 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-04-21T08:51:49.210 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.210 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.211 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.211 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.211 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.211 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.211 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.211 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_host": { 2024-04-21T08:51:49.211 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_host", 2024-04-21T08:51:49.211 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.211 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.211 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.211 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-04-21T08:51:49.211 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.211 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.211 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.211 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.212 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.212 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.212 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.212 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_port": { 2024-04-21T08:51:49.212 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_port", 2024-04-21T08:51:49.212 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-04-21T08:51:49.212 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.212 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.212 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10051", 2024-04-21T08:51:49.212 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.212 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.212 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.212 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.213 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.213 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.213 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.213 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_sender": { 2024-04-21T08:51:49.213 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_sender", 2024-04-21T08:51:49.213 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-04-21T08:51:49.213 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-04-21T08:51:49.213 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-04-21T08:51:49.213 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-21T08:51:49.213 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-04-21T08:51:49.213 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-04-21T08:51:49.213 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-04-21T08:51:49.213 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-04-21T08:51:49.213 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-04-21T08:51:49.213 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-04-21T08:51:49.213 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-04-21T08:51:49.214 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.214 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.214 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.214 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.214 INFO:teuthology.orchestra.run.smithi043.stdout: "services": { 2024-04-21T08:51:49.214 INFO:teuthology.orchestra.run.smithi043.stdout: "dashboard": "https://172.21.15.43:8443/" 2024-04-21T08:51:49.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.214 INFO:teuthology.orchestra.run.smithi043.stdout: "always_on_modules": { 2024-04-21T08:51:49.214 INFO:teuthology.orchestra.run.smithi043.stdout: "octopus": [ 2024-04-21T08:51:49.214 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-04-21T08:51:49.214 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-04-21T08:51:49.214 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-04-21T08:51:49.214 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-04-21T08:51:49.214 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-04-21T08:51:49.215 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-04-21T08:51:49.215 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-04-21T08:51:49.215 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-04-21T08:51:49.215 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-04-21T08:51:49.215 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-04-21T08:51:49.215 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.215 INFO:teuthology.orchestra.run.smithi043.stdout: "pacific": [ 2024-04-21T08:51:49.215 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-04-21T08:51:49.215 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-04-21T08:51:49.215 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-04-21T08:51:49.215 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-04-21T08:51:49.215 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-04-21T08:51:49.215 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-04-21T08:51:49.215 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-04-21T08:51:49.216 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-04-21T08:51:49.216 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-04-21T08:51:49.216 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-04-21T08:51:49.216 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.216 INFO:teuthology.orchestra.run.smithi043.stdout: "quincy": [ 2024-04-21T08:51:49.216 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-04-21T08:51:49.216 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-04-21T08:51:49.216 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-04-21T08:51:49.216 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-04-21T08:51:49.216 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-04-21T08:51:49.216 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-04-21T08:51:49.216 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-04-21T08:51:49.216 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-04-21T08:51:49.216 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-04-21T08:51:49.216 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-04-21T08:51:49.216 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.217 INFO:teuthology.orchestra.run.smithi043.stdout: "reef": [ 2024-04-21T08:51:49.217 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-04-21T08:51:49.217 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-04-21T08:51:49.217 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-04-21T08:51:49.217 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-04-21T08:51:49.217 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-04-21T08:51:49.217 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-04-21T08:51:49.217 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-04-21T08:51:49.217 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-04-21T08:51:49.217 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-04-21T08:51:49.217 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-04-21T08:51:49.217 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-04-21T08:51:49.217 INFO:teuthology.orchestra.run.smithi043.stdout: "squid": [ 2024-04-21T08:51:49.217 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-04-21T08:51:49.217 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-04-21T08:51:49.217 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-04-21T08:51:49.218 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-04-21T08:51:49.218 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-04-21T08:51:49.218 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-04-21T08:51:49.218 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-04-21T08:51:49.218 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-04-21T08:51:49.218 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-04-21T08:51:49.218 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-04-21T08:51:49.218 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-21T08:51:49.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.218 INFO:teuthology.orchestra.run.smithi043.stdout: "last_failure_osd_epoch": 95, 2024-04-21T08:51:49.218 INFO:teuthology.orchestra.run.smithi043.stdout: "active_clients": [ 2024-04-21T08:51:49.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.218 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "devicehealth", 2024-04-21T08:51:49.218 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-04-21T08:51:49.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.218 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-04-21T08:51:49.219 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-04-21T08:51:49.219 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3520103489 2024-04-21T08:51:49.219 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.219 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-21T08:51:49.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.219 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "libcephsqlite", 2024-04-21T08:51:49.219 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-04-21T08:51:49.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.219 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-04-21T08:51:49.219 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-04-21T08:51:49.219 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1538119529 2024-04-21T08:51:49.219 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.219 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-21T08:51:49.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.220 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_support", 2024-04-21T08:51:49.220 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-04-21T08:51:49.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.220 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-04-21T08:51:49.220 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-04-21T08:51:49.220 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2377838201 2024-04-21T08:51:49.220 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.220 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-21T08:51:49.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.220 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snap_schedule", 2024-04-21T08:51:49.220 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-04-21T08:51:49.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.220 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-04-21T08:51:49.220 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-04-21T08:51:49.220 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3770274727 2024-04-21T08:51:49.221 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.221 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-21T08:51:49.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.221 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snap_schedule", 2024-04-21T08:51:49.221 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-04-21T08:51:49.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.221 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-04-21T08:51:49.221 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-04-21T08:51:49.221 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1992995006 2024-04-21T08:51:49.221 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.221 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-21T08:51:49.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:49.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.221 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "volumes", 2024-04-21T08:51:49.221 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-04-21T08:51:49.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-21T08:51:49.222 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-04-21T08:51:49.222 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-04-21T08:51:49.222 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 308223731 2024-04-21T08:51:49.222 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.222 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-21T08:51:49.222 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:49.222 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-21T08:51:49.222 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:51:49.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.961+0000 7f61081f4640 1 -- 172.21.15.43:0/3207501411 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f60ec0a3d20 msgr2=0x7f60ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:49.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.961+0000 7f61081f4640 1 --2- 172.21.15.43:0/3207501411 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f60ec0a3d20 0x7f60ec0a61e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f60fc000c00 tx=0x7f60fc004040 comp rx=0 tx=0).stop 2024-04-21T08:51:49.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.961+0000 7f61081f4640 1 -- 172.21.15.43:0/3207501411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f610010c570 msgr2=0x7f610019cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:49.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.961+0000 7f61081f4640 1 --2- 172.21.15.43:0/3207501411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f610010c570 0x7f610019cff0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f60f800ea60 tx=0x7f60f800ef20 comp rx=0 tx=0).stop 2024-04-21T08:51:49.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.961+0000 7f61081f4640 1 -- 172.21.15.43:0/3207501411 shutdown_connections 2024-04-21T08:51:49.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.961+0000 7f61081f4640 1 --2- 172.21.15.43:0/3207501411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f610010c570 0x7f610019cff0 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:49.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.961+0000 7f61081f4640 1 --2- 172.21.15.43:0/3207501411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6100103b50 0x7f6100198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:49.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.961+0000 7f61081f4640 1 --2- 172.21.15.43:0/3207501411 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f60ec0a3d20 0x7f60ec0a61e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:49.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.961+0000 7f61081f4640 1 --2- 172.21.15.43:0/3207501411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6100103230 0x7f6100198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:49.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.961+0000 7f61081f4640 1 -- 172.21.15.43:0/3207501411 >> 172.21.15.43:0/3207501411 conn(0x7f6100077fb0 msgr2=0x7f61000fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:49.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.961+0000 7f61081f4640 1 -- 172.21.15.43:0/3207501411 shutdown_connections 2024-04-21T08:51:49.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:51:47.961+0000 7f61081f4640 1 -- 172.21.15.43:0/3207501411 wait complete. 2024-04-21T08:51:49.224 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:47 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3277078677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:51:49.225 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:47 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1540994855' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:51:49.225 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:47 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3597748838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:51:49.225 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:47 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2201092360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:51:49.225 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:47 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3277078677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:51:49.225 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:47 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1540994855' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:51:49.225 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:47 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3597748838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:51:49.225 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:47 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2201092360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:51:49.225 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:48 smithi112 ceph-mon[31085]: pgmap v402: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-04-21T08:51:49.225 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:48 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3207501411' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-21T08:51:49.225 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:47 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3277078677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:51:49.225 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:47 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1540994855' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:51:49.225 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:47 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3597748838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:51:49.225 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:47 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2201092360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:51:49.226 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:48 smithi175 ceph-mon[31920]: pgmap v402: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-04-21T08:51:49.226 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:48 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3207501411' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-21T08:51:49.265 DEBUG:tasks.check_counter:Getting stats from a 2024-04-21T08:51:49.266 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-21T08:51:49.266 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:48 smithi043 ceph-mon[27978]: pgmap v402: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-04-21T08:51:49.266 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:48 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3207501411' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-21T08:51:49.526 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:51:50.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:50 smithi175 ceph-mon[31920]: pgmap v403: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-21T08:51:50.978 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:50 smithi043 ceph-mon[27978]: pgmap v403: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-21T08:51:51.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:50 smithi112 ceph-mon[31085]: pgmap v403: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-21T08:51:51.428 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:51:51.428 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:51:51.428 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 1801, 2024-04-21T08:51:51.428 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 1803, 2024-04-21T08:51:51.428 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 540088, 2024-04-21T08:51:51.428 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 572719, 2024-04-21T08:51:51.428 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 6, 2024-04-21T08:51:51.428 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 2, 2024-04-21T08:51:51.428 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.192051777, 2024-04-21T08:51:51.428 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.061561079, 2024-04-21T08:51:51.429 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.051100908, 2024-04-21T08:51:51.429 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.061101433, 2024-04-21T08:51:51.429 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:51.429 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1801, 2024-04-21T08:51:51.429 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.071075383, 2024-04-21T08:51:51.429 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000039464 2024-04-21T08:51:51.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.429 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:51.429 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.429 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.429 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.429 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T08:51:51.429 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T08:51:51.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.430 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:51:51.430 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 825, 2024-04-21T08:51:51.430 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 1013, 2024-04-21T08:51:51.430 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 205438, 2024-04-21T08:51:51.430 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 758654, 2024-04-21T08:51:51.430 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 7, 2024-04-21T08:51:51.430 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 5, 2024-04-21T08:51:51.430 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.107677491, 2024-04-21T08:51:51.430 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.035808158, 2024-04-21T08:51:51.430 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.024886863, 2024-04-21T08:51:51.430 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.036121399, 2024-04-21T08:51:51.430 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:51.430 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1011, 2024-04-21T08:51:51.430 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.042565656, 2024-04-21T08:51:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000042102 2024-04-21T08:51:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-21T08:51:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 498144 2024-04-21T08:51:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:51:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 287, 2024-04-21T08:51:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 273, 2024-04-21T08:51:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 1651004, 2024-04-21T08:51:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 158158, 2024-04-21T08:51:51.431 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 3, 2024-04-21T08:51:51.432 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 2, 2024-04-21T08:51:51.432 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.050789369, 2024-04-21T08:51:51.432 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.017411538, 2024-04-21T08:51:51.432 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.036780594, 2024-04-21T08:51:51.432 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.005321599, 2024-04-21T08:51:51.432 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:51.432 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 272, 2024-04-21T08:51:51.432 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.013626110, 2024-04-21T08:51:51.432 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000050095 2024-04-21T08:51:51.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.432 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:51.432 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.432 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.432 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.433 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 1650736, 2024-04-21T08:51:51.433 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 157840 2024-04-21T08:51:51.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.433 INFO:teuthology.orchestra.run.smithi043.stdout: "cct": { 2024-04-21T08:51:51.433 INFO:teuthology.orchestra.run.smithi043.stdout: "total_workers": 1, 2024-04-21T08:51:51.433 INFO:teuthology.orchestra.run.smithi043.stdout: "unhealthy_workers": 0 2024-04-21T08:51:51.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.433 INFO:teuthology.orchestra.run.smithi043.stdout: "finisher-MDSRank": { 2024-04-21T08:51:51.433 INFO:teuthology.orchestra.run.smithi043.stdout: "queue_len": 0, 2024-04-21T08:51:51.433 INFO:teuthology.orchestra.run.smithi043.stdout: "complete_latency": { 2024-04-21T08:51:51.433 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 4067, 2024-04-21T08:51:51.433 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.136260610, 2024-04-21T08:51:51.433 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000033503 2024-04-21T08:51:51.434 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:51.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.434 INFO:teuthology.orchestra.run.smithi043.stdout: "finisher-PurgeQueue": { 2024-04-21T08:51:51.434 INFO:teuthology.orchestra.run.smithi043.stdout: "queue_len": 0, 2024-04-21T08:51:51.434 INFO:teuthology.orchestra.run.smithi043.stdout: "complete_latency": { 2024-04-21T08:51:51.434 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.434 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.434 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.434 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:51.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.434 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": { 2024-04-21T08:51:51.434 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-04-21T08:51:51.434 INFO:teuthology.orchestra.run.smithi043.stdout: "reply": 0, 2024-04-21T08:51:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: "reply_latency": { 2024-04-21T08:51:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: "slow_reply": 0, 2024-04-21T08:51:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: "forward": 0, 2024-04-21T08:51:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_fetch_complete": 0, 2024-04-21T08:51:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_fetch_keys": 0, 2024-04-21T08:51:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_commit": 0, 2024-04-21T08:51:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_split": 0, 2024-04-21T08:51:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_merge": 0, 2024-04-21T08:51:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes": 0, 2024-04-21T08:51:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_top": 0, 2024-04-21T08:51:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_bottom": 0, 2024-04-21T08:51:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_pin_tail": 0, 2024-04-21T08:51:51.436 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_pinned": 0, 2024-04-21T08:51:51.436 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_expired": 0, 2024-04-21T08:51:51.436 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_with_caps": 0, 2024-04-21T08:51:51.436 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 0, 2024-04-21T08:51:51.436 INFO:teuthology.orchestra.run.smithi043.stdout: "subtrees": 1, 2024-04-21T08:51:51.436 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse": 0, 2024-04-21T08:51:51.436 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_hit": 0, 2024-04-21T08:51:51.436 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_forward": 0, 2024-04-21T08:51:51.436 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_discover": 0, 2024-04-21T08:51:51.436 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:51:51.436 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_remote_ino": 0, 2024-04-21T08:51:51.436 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_lock": 0, 2024-04-21T08:51:51.436 INFO:teuthology.orchestra.run.smithi043.stdout: "load_cent": 0, 2024-04-21T08:51:51.436 INFO:teuthology.orchestra.run.smithi043.stdout: "q": 0, 2024-04-21T08:51:51.436 INFO:teuthology.orchestra.run.smithi043.stdout: "exported": 0, 2024-04-21T08:51:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: "exported_inodes": 0, 2024-04-21T08:51:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: "imported": 0, 2024-04-21T08:51:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: "imported_inodes": 0, 2024-04-21T08:51:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_dir_fetch": 0, 2024-04-21T08:51:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_backtrace_fetch": 0, 2024-04-21T08:51:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_peer_discover": 0, 2024-04-21T08:51:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rfiles": 0, 2024-04-21T08:51:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rbytes": 0, 2024-04-21T08:51:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rsnaps": 0, 2024-04-21T08:51:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T08:51:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_set_tag": 0, 2024-04-21T08:51:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:51:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:51:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:51:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:51:51.437 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:51:51.438 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_file_inodes": 0, 2024-04-21T08:51:51.438 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:51:51.438 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T08:51:51.438 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_grant": 0, 2024-04-21T08:51:51.438 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:51:51.438 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:51:51.438 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:51:51.438 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_caps": 0, 2024-04-21T08:51:51.438 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_caps_dirty": 0, 2024-04-21T08:51:51.438 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_cap_release": 0, 2024-04-21T08:51:51.438 INFO:teuthology.orchestra.run.smithi043.stdout: "process_request_cap_release": 0 2024-04-21T08:51:51.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.438 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cache": { 2024-04-21T08:51:51.438 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_update": 0, 2024-04-21T08:51:51.438 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_update_receipt": 0, 2024-04-21T08:51:51.439 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_try_discover": 0, 2024-04-21T08:51:51.439 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_send_discover": 0, 2024-04-21T08:51:51.439 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_handle_discover": 0, 2024-04-21T08:51:51.439 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays": 0, 2024-04-21T08:51:51.439 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays_delayed": 0, 2024-04-21T08:51:51.439 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:51:51.439 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_created": 0, 2024-04-21T08:51:51.439 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_enqueued": 0, 2024-04-21T08:51:51.439 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_reintegrated": 0, 2024-04-21T08:51:51.439 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_migrated": 0, 2024-04-21T08:51:51.439 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_processing": 0, 2024-04-21T08:51:51.439 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:51:51.439 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:51:51.439 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_started": 0, 2024-04-21T08:51:51.439 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_completed": 0, 2024-04-21T08:51:51.439 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_quiesce_path": 0, 2024-04-21T08:51:51.439 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_quiesce_inode": 0, 2024-04-21T08:51:51.440 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:51:51.440 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_exportdir": 0, 2024-04-21T08:51:51.440 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_flush": 0, 2024-04-21T08:51:51.440 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:51:51.440 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_fragstats": 0, 2024-04-21T08:51:51.440 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_inodestats": 0 2024-04-21T08:51:51.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.440 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_log": { 2024-04-21T08:51:51.440 INFO:teuthology.orchestra.run.smithi043.stdout: "evlrg": 0, 2024-04-21T08:51:51.440 INFO:teuthology.orchestra.run.smithi043.stdout: "evadd": 0, 2024-04-21T08:51:51.440 INFO:teuthology.orchestra.run.smithi043.stdout: "evex": 0, 2024-04-21T08:51:51.440 INFO:teuthology.orchestra.run.smithi043.stdout: "evtrm": 0, 2024-04-21T08:51:51.440 INFO:teuthology.orchestra.run.smithi043.stdout: "ev": 6, 2024-04-21T08:51:51.440 INFO:teuthology.orchestra.run.smithi043.stdout: "evexg": 0, 2024-04-21T08:51:51.440 INFO:teuthology.orchestra.run.smithi043.stdout: "evexd": 0, 2024-04-21T08:51:51.440 INFO:teuthology.orchestra.run.smithi043.stdout: "segadd": 0, 2024-04-21T08:51:51.441 INFO:teuthology.orchestra.run.smithi043.stdout: "segex": 0, 2024-04-21T08:51:51.441 INFO:teuthology.orchestra.run.smithi043.stdout: "segtrm": 0, 2024-04-21T08:51:51.441 INFO:teuthology.orchestra.run.smithi043.stdout: "seg": 2, 2024-04-21T08:51:51.441 INFO:teuthology.orchestra.run.smithi043.stdout: "segmjr": 2, 2024-04-21T08:51:51.441 INFO:teuthology.orchestra.run.smithi043.stdout: "segexg": 0, 2024-04-21T08:51:51.441 INFO:teuthology.orchestra.run.smithi043.stdout: "segexd": 0, 2024-04-21T08:51:51.441 INFO:teuthology.orchestra.run.smithi043.stdout: "expos": 4194304, 2024-04-21T08:51:51.441 INFO:teuthology.orchestra.run.smithi043.stdout: "wrpos": 4196123, 2024-04-21T08:51:51.441 INFO:teuthology.orchestra.run.smithi043.stdout: "rdpos": 4195732, 2024-04-21T08:51:51.441 INFO:teuthology.orchestra.run.smithi043.stdout: "jlat": { 2024-04-21T08:51:51.441 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.441 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.441 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.441 INFO:teuthology.orchestra.run.smithi043.stdout: "replayed": 6 2024-04-21T08:51:51.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.442 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_mem": { 2024-04-21T08:51:51.442 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 3, 2024-04-21T08:51:51.442 INFO:teuthology.orchestra.run.smithi043.stdout: "ino+": 3, 2024-04-21T08:51:51.442 INFO:teuthology.orchestra.run.smithi043.stdout: "ino-": 0, 2024-04-21T08:51:51.442 INFO:teuthology.orchestra.run.smithi043.stdout: "dir": 1, 2024-04-21T08:51:51.442 INFO:teuthology.orchestra.run.smithi043.stdout: "dir+": 1, 2024-04-21T08:51:51.442 INFO:teuthology.orchestra.run.smithi043.stdout: "dir-": 0, 2024-04-21T08:51:51.442 INFO:teuthology.orchestra.run.smithi043.stdout: "dn": 0, 2024-04-21T08:51:51.442 INFO:teuthology.orchestra.run.smithi043.stdout: "dn+": 0, 2024-04-21T08:51:51.442 INFO:teuthology.orchestra.run.smithi043.stdout: "dn-": 0, 2024-04-21T08:51:51.442 INFO:teuthology.orchestra.run.smithi043.stdout: "cap": 0, 2024-04-21T08:51:51.442 INFO:teuthology.orchestra.run.smithi043.stdout: "cap+": 0, 2024-04-21T08:51:51.442 INFO:teuthology.orchestra.run.smithi043.stdout: "cap-": 0, 2024-04-21T08:51:51.443 INFO:teuthology.orchestra.run.smithi043.stdout: "rss": 49848, 2024-04-21T08:51:51.443 INFO:teuthology.orchestra.run.smithi043.stdout: "heap": 215416 2024-04-21T08:51:51.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.443 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_server": { 2024-04-21T08:51:51.443 INFO:teuthology.orchestra.run.smithi043.stdout: "dispatch_client_request": 0, 2024-04-21T08:51:51.443 INFO:teuthology.orchestra.run.smithi043.stdout: "dispatch_server_request": 0, 2024-04-21T08:51:51.443 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_request": 0, 2024-04-21T08:51:51.443 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_session": 0, 2024-04-21T08:51:51.443 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_peer_request": 0, 2024-04-21T08:51:51.443 INFO:teuthology.orchestra.run.smithi043.stdout: "req_create_latency": { 2024-04-21T08:51:51.443 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.443 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.443 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.443 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getattr_latency": { 2024-04-21T08:51:51.444 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.444 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.444 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.444 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getfilelock_latency": { 2024-04-21T08:51:51.444 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.444 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.444 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.444 INFO:teuthology.orchestra.run.smithi043.stdout: "req_link_latency": { 2024-04-21T08:51:51.444 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.444 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.444 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.444 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookup_latency": { 2024-04-21T08:51:51.444 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookuphash_latency": { 2024-04-21T08:51:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupino_latency": { 2024-04-21T08:51:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupname_latency": { 2024-04-21T08:51:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.445 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.446 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupparent_latency": { 2024-04-21T08:51:51.446 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.446 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.446 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.446 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupsnap_latency": { 2024-04-21T08:51:51.446 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.446 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.446 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.446 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lssnap_latency": { 2024-04-21T08:51:51.446 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.446 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.447 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.447 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mkdir_latency": { 2024-04-21T08:51:51.447 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.447 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.447 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.447 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mknod_latency": { 2024-04-21T08:51:51.448 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.448 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.448 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.448 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mksnap_latency": { 2024-04-21T08:51:51.448 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.448 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.448 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.448 INFO:teuthology.orchestra.run.smithi043.stdout: "req_open_latency": { 2024-04-21T08:51:51.448 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.448 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.448 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.448 INFO:teuthology.orchestra.run.smithi043.stdout: "req_readdir_latency": { 2024-04-21T08:51:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rename_latency": { 2024-04-21T08:51:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: "req_renamesnap_latency": { 2024-04-21T08:51:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: "req_snapdiff_latency": { 2024-04-21T08:51:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.450 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.450 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.450 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmdir_latency": { 2024-04-21T08:51:51.450 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.450 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.450 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.450 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmsnap_latency": { 2024-04-21T08:51:51.450 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.450 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.450 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.451 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmxattr_latency": { 2024-04-21T08:51:51.451 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.451 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.451 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.451 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setattr_latency": { 2024-04-21T08:51:51.451 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.451 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.451 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.451 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setdirlayout_latency": { 2024-04-21T08:51:51.451 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.451 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.451 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.452 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setfilelock_latency": { 2024-04-21T08:51:51.452 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.452 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.452 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.452 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setlayout_latency": { 2024-04-21T08:51:51.452 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.452 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.452 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.452 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setxattr_latency": { 2024-04-21T08:51:51.452 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.452 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.453 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.453 INFO:teuthology.orchestra.run.smithi043.stdout: "req_symlink_latency": { 2024-04-21T08:51:51.453 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.453 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.453 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.453 INFO:teuthology.orchestra.run.smithi043.stdout: "req_unlink_latency": { 2024-04-21T08:51:51.453 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.453 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.453 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.453 INFO:teuthology.orchestra.run.smithi043.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:51:51.453 INFO:teuthology.orchestra.run.smithi043.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:51:51.454 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getvxattr_latency": { 2024-04-21T08:51:51.454 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.454 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.454 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.454 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:51.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.454 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_sessions": { 2024-04-21T08:51:51.454 INFO:teuthology.orchestra.run.smithi043.stdout: "session_count": 2, 2024-04-21T08:51:51.454 INFO:teuthology.orchestra.run.smithi043.stdout: "session_add": 3, 2024-04-21T08:51:51.454 INFO:teuthology.orchestra.run.smithi043.stdout: "session_remove": 1, 2024-04-21T08:51:51.454 INFO:teuthology.orchestra.run.smithi043.stdout: "sessions_open": 2, 2024-04-21T08:51:51.454 INFO:teuthology.orchestra.run.smithi043.stdout: "sessions_stale": 0, 2024-04-21T08:51:51.454 INFO:teuthology.orchestra.run.smithi043.stdout: "total_load": 0, 2024-04-21T08:51:51.455 INFO:teuthology.orchestra.run.smithi043.stdout: "average_load": 0, 2024-04-21T08:51:51.455 INFO:teuthology.orchestra.run.smithi043.stdout: "avg_session_uptime": 1967, 2024-04-21T08:51:51.455 INFO:teuthology.orchestra.run.smithi043.stdout: "mdthresh_evicted": 0 2024-04-21T08:51:51.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.455 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool": { 2024-04-21T08:51:51.455 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:51:51.455 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_items": 0, 2024-04-21T08:51:51.455 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:51:51.455 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:51:51.455 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:51:51.455 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:51:51.455 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:51:51.455 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:51:51.455 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:51:51.455 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:51:51.456 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:51:51.456 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:51:51.456 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:51:51.456 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:51:51.456 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:51:51.456 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_items": 0, 2024-04-21T08:51:51.456 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:51:51.456 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_items": 0, 2024-04-21T08:51:51.456 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:51:51.456 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:51:51.456 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:51:51.456 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:51:51.456 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:51:51.456 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:51:51.457 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:51:51.457 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_items": 0, 2024-04-21T08:51:51.457 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:51:51.457 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:51:51.457 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:51:51.457 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_items": 0, 2024-04-21T08:51:51.457 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_bytes": 0, 2024-04-21T08:51:51.457 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_items": 0, 2024-04-21T08:51:51.457 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:51:51.457 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:51:51.457 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:51:51.457 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:51:51.457 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_bytes": 190441, 2024-04-21T08:51:51.457 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_items": 53, 2024-04-21T08:51:51.457 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:51:51.458 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_items": 0, 2024-04-21T08:51:51.458 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bytes": 0, 2024-04-21T08:51:51.458 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_items": 0, 2024-04-21T08:51:51.458 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:51:51.458 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_items": 0, 2024-04-21T08:51:51.458 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:51:51.458 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_items": 0, 2024-04-21T08:51:51.458 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_bytes": 9800, 2024-04-21T08:51:51.458 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_items": 188, 2024-04-21T08:51:51.458 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:51:51.458 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:51:51.458 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_bytes": 0, 2024-04-21T08:51:51.458 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_items": 0, 2024-04-21T08:51:51.459 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_bytes": 8512, 2024-04-21T08:51:51.459 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_items": 13, 2024-04-21T08:51:51.459 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_bytes": 0, 2024-04-21T08:51:51.459 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_items": 0, 2024-04-21T08:51:51.459 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_bytes": 0, 2024-04-21T08:51:51.459 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_items": 0 2024-04-21T08:51:51.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.459 INFO:teuthology.orchestra.run.smithi043.stdout: "objecter": { 2024-04-21T08:51:51.459 INFO:teuthology.orchestra.run.smithi043.stdout: "op_active": 0, 2024-04-21T08:51:51.459 INFO:teuthology.orchestra.run.smithi043.stdout: "op_laggy": 0, 2024-04-21T08:51:51.459 INFO:teuthology.orchestra.run.smithi043.stdout: "op_send": 2621, 2024-04-21T08:51:51.459 INFO:teuthology.orchestra.run.smithi043.stdout: "op_send_bytes": 21, 2024-04-21T08:51:51.459 INFO:teuthology.orchestra.run.smithi043.stdout: "op_resend": 3, 2024-04-21T08:51:51.459 INFO:teuthology.orchestra.run.smithi043.stdout: "op_reply": 2621, 2024-04-21T08:51:51.459 INFO:teuthology.orchestra.run.smithi043.stdout: "op_latency": { 2024-04-21T08:51:51.460 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 2621, 2024-04-21T08:51:51.460 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 1.738978904, 2024-04-21T08:51:51.460 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000663479 2024-04-21T08:51:51.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.460 INFO:teuthology.orchestra.run.smithi043.stdout: "op_inflight": 0, 2024-04-21T08:51:51.460 INFO:teuthology.orchestra.run.smithi043.stdout: "oplen_avg": { 2024-04-21T08:51:51.460 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 2621, 2024-04-21T08:51:51.460 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 2622 2024-04-21T08:51:51.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.460 INFO:teuthology.orchestra.run.smithi043.stdout: "op": 2621, 2024-04-21T08:51:51.460 INFO:teuthology.orchestra.run.smithi043.stdout: "op_r": 2621, 2024-04-21T08:51:51.460 INFO:teuthology.orchestra.run.smithi043.stdout: "op_w": 0, 2024-04-21T08:51:51.460 INFO:teuthology.orchestra.run.smithi043.stdout: "op_rmw": 0, 2024-04-21T08:51:51.460 INFO:teuthology.orchestra.run.smithi043.stdout: "op_pg": 0, 2024-04-21T08:51:51.461 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_stat": 1714, 2024-04-21T08:51:51.461 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_create": 0, 2024-04-21T08:51:51.461 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_read": 905, 2024-04-21T08:51:51.461 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_write": 0, 2024-04-21T08:51:51.461 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_writefull": 0, 2024-04-21T08:51:51.461 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_writesame": 0, 2024-04-21T08:51:51.461 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_append": 0, 2024-04-21T08:51:51.461 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_zero": 0, 2024-04-21T08:51:51.461 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_truncate": 0, 2024-04-21T08:51:51.461 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_delete": 0, 2024-04-21T08:51:51.461 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_mapext": 0, 2024-04-21T08:51:51.461 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_sparse_read": 0, 2024-04-21T08:51:51.461 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_clonerange": 0, 2024-04-21T08:51:51.461 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_getxattr": 1, 2024-04-21T08:51:51.462 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_setxattr": 0, 2024-04-21T08:51:51.462 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:51:51.462 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_rmxattr": 0, 2024-04-21T08:51:51.462 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:51:51.462 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_call": 0, 2024-04-21T08:51:51.462 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_watch": 0, 2024-04-21T08:51:51.462 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_notify": 0, 2024-04-21T08:51:51.462 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:51:51.462 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_pgls": 0, 2024-04-21T08:51:51.462 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:51:51.462 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_other": 0, 2024-04-21T08:51:51.462 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_active": 0, 2024-04-21T08:51:51.462 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_send": 0, 2024-04-21T08:51:51.462 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_resend": 0, 2024-04-21T08:51:51.462 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_ping": 0, 2024-04-21T08:51:51.463 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_active": 0, 2024-04-21T08:51:51.463 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_send": 0, 2024-04-21T08:51:51.463 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_resend": 0, 2024-04-21T08:51:51.463 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_active": 0, 2024-04-21T08:51:51.463 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_send": 0, 2024-04-21T08:51:51.463 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_resend": 0, 2024-04-21T08:51:51.463 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_active": 0, 2024-04-21T08:51:51.463 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_send": 0, 2024-04-21T08:51:51.463 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_resend": 0, 2024-04-21T08:51:51.463 INFO:teuthology.orchestra.run.smithi043.stdout: "command_active": 0, 2024-04-21T08:51:51.463 INFO:teuthology.orchestra.run.smithi043.stdout: "command_send": 0, 2024-04-21T08:51:51.463 INFO:teuthology.orchestra.run.smithi043.stdout: "command_resend": 0, 2024-04-21T08:51:51.463 INFO:teuthology.orchestra.run.smithi043.stdout: "map_epoch": 108, 2024-04-21T08:51:51.463 INFO:teuthology.orchestra.run.smithi043.stdout: "map_full": 0, 2024-04-21T08:51:51.464 INFO:teuthology.orchestra.run.smithi043.stdout: "map_inc": 18, 2024-04-21T08:51:51.464 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_sessions": 5, 2024-04-21T08:51:51.464 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_session_open": 5, 2024-04-21T08:51:51.464 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_session_close": 0, 2024-04-21T08:51:51.464 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_laggy": 0, 2024-04-21T08:51:51.464 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_wr": 0, 2024-04-21T08:51:51.464 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_rd": 2, 2024-04-21T08:51:51.464 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_del": 0 2024-04-21T08:51:51.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.464 INFO:teuthology.orchestra.run.smithi043.stdout: "oft": { 2024-04-21T08:51:51.464 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_objs": 0, 2024-04-21T08:51:51.464 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:51:51.464 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_updates": 0, 2024-04-21T08:51:51.464 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_removes": 0 2024-04-21T08:51:51.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.465 INFO:teuthology.orchestra.run.smithi043.stdout: "osd-slow-ops": { 2024-04-21T08:51:51.465 INFO:teuthology.orchestra.run.smithi043.stdout: "slow_ops_count": 0 2024-04-21T08:51:51.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.465 INFO:teuthology.orchestra.run.smithi043.stdout: "purge_queue": { 2024-04-21T08:51:51.465 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_ops": 0, 2024-04-21T08:51:51.465 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T08:51:51.465 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing": 0, 2024-04-21T08:51:51.465 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_high_water": 0, 2024-04-21T08:51:51.465 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executed_ops": 0, 2024-04-21T08:51:51.465 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executed": 0, 2024-04-21T08:51:51.465 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_item_in_journal": 0 2024-04-21T08:51:51.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.465 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:51:51.466 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-21T08:51:51.466 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-21T08:51:51.466 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-21T08:51:51.466 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 2909, 2024-04-21T08:51:51.466 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 2117090, 2024-04-21T08:51:51.466 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:51.466 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 2909, 2024-04-21T08:51:51.466 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-21T08:51:51.466 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-21T08:51:51.466 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 2909, 2024-04-21T08:51:51.466 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 2117090, 2024-04-21T08:51:51.466 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-21T08:51:51.466 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.466 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.466 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.467 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:51.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.467 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-objecter_bytes": { 2024-04-21T08:51:51.467 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-21T08:51:51.467 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-21T08:51:51.467 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-21T08:51:51.467 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-21T08:51:51.467 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-21T08:51:51.467 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:51.467 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:51.467 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 2621, 2024-04-21T08:51:51.467 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 1824, 2024-04-21T08:51:51.467 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 2621, 2024-04-21T08:51:51.467 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 1824, 2024-04-21T08:51:51.468 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-21T08:51:51.468 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.468 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.468 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.468 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:51.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.468 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-objecter_ops": { 2024-04-21T08:51:51.468 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-21T08:51:51.468 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 1024, 2024-04-21T08:51:51.468 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-21T08:51:51.468 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-21T08:51:51.468 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-21T08:51:51.468 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:51.468 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:51.469 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 2621, 2024-04-21T08:51:51.469 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 2621, 2024-04-21T08:51:51.469 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 2621, 2024-04-21T08:51:51.469 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 2621, 2024-04-21T08:51:51.469 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-21T08:51:51.469 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.469 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.469 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.469 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:51.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.469 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:51:51.469 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-21T08:51:51.469 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 3758096384, 2024-04-21T08:51:51.469 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-21T08:51:51.469 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-21T08:51:51.470 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-21T08:51:51.470 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:51.470 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:51.470 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-21T08:51:51.470 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-21T08:51:51.470 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 0, 2024-04-21T08:51:51.470 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 0, 2024-04-21T08:51:51.470 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-21T08:51:51.470 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.470 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.470 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.470 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:51.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:51.470 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-write_buf_throttle-0x560a756206e0": { 2024-04-21T08:51:51.471 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-21T08:51:51.471 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 3758096384, 2024-04-21T08:51:51.471 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-21T08:51:51.471 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-21T08:51:51.471 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-21T08:51:51.471 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:51.471 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:51.471 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-21T08:51:51.471 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-21T08:51:51.471 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 0, 2024-04-21T08:51:51.471 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 0, 2024-04-21T08:51:51.471 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-21T08:51:51.471 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:51.471 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:51.471 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:51.472 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:51.472 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:51.472 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:51:52.164 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-21T08:51:52.164 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-21T08:51:52.164 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-04-21T08:51:52.164 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-04-21T08:51:52.164 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-04-21T08:51:52.164 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:51:52.164 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:51:52.165 DEBUG:tasks.check_counter:Getting stats from d 2024-04-21T08:51:52.165 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-21T08:51:52.419 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:51:52.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:52 smithi043 ceph-mon[27978]: pgmap v404: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-04-21T08:51:52.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:52 smithi175 ceph-mon[31920]: pgmap v404: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-04-21T08:51:53.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:52 smithi112 ceph-mon[31085]: pgmap v404: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-04-21T08:51:54.375 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:51:54.376 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:51:54.376 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 267, 2024-04-21T08:51:54.376 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 256, 2024-04-21T08:51:54.376 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 1602624, 2024-04-21T08:51:54.376 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 157840, 2024-04-21T08:51:54.376 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 4, 2024-04-21T08:51:54.376 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 0, 2024-04-21T08:51:54.376 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.052459365, 2024-04-21T08:51:54.376 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.017638976, 2024-04-21T08:51:54.376 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.037000331, 2024-04-21T08:51:54.376 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.005484263, 2024-04-21T08:51:54.376 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:54.376 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 255, 2024-04-21T08:51:54.376 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.013233051, 2024-04-21T08:51:54.376 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000051894 2024-04-21T08:51:54.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:54.377 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:54.377 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:54.377 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:54.377 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:54.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:54.377 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 1602624, 2024-04-21T08:51:54.377 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 157840 2024-04-21T08:51:54.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:54.377 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:51:54.377 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 2, 2024-04-21T08:51:54.377 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 196, 2024-04-21T08:51:54.377 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 320, 2024-04-21T08:51:54.377 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 190576, 2024-04-21T08:51:54.378 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 5, 2024-04-21T08:51:54.378 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 3, 2024-04-21T08:51:54.378 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.018375225, 2024-04-21T08:51:54.378 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.013175228, 2024-04-21T08:51:54.378 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.002080615, 2024-04-21T08:51:54.378 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T08:51:54.378 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:54.378 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 193, 2024-04-21T08:51:54.378 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.016476499, 2024-04-21T08:51:54.378 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000085370 2024-04-21T08:51:54.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:54.378 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:54.378 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:54.378 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:54.379 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:54.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:54.379 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T08:51:54.379 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 190576 2024-04-21T08:51:54.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:54.379 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:51:54.379 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 4, 2024-04-21T08:51:54.379 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 4, 2024-04-21T08:51:54.379 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 2992, 2024-04-21T08:51:54.379 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 448, 2024-04-21T08:51:54.379 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 2, 2024-04-21T08:51:54.379 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 1, 2024-04-21T08:51:54.379 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.000762673, 2024-04-21T08:51:54.379 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.000038775, 2024-04-21T08:51:54.379 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.000653879, 2024-04-21T08:51:54.380 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T08:51:54.380 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:54.380 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 3, 2024-04-21T08:51:54.380 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000797329, 2024-04-21T08:51:54.380 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000265776 2024-04-21T08:51:54.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:54.380 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:54.380 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:54.380 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:54.380 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:54.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:54.380 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-21T08:51:54.380 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 448 2024-04-21T08:51:54.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:54.380 INFO:teuthology.orchestra.run.smithi043.stdout: "cct": { 2024-04-21T08:51:54.381 INFO:teuthology.orchestra.run.smithi043.stdout: "total_workers": 0, 2024-04-21T08:51:54.381 INFO:teuthology.orchestra.run.smithi043.stdout: "unhealthy_workers": 0 2024-04-21T08:51:54.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:54.381 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool": { 2024-04-21T08:51:54.381 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:51:54.381 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_items": 0, 2024-04-21T08:51:54.381 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:51:54.381 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:51:54.381 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:51:54.381 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:51:54.381 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:51:54.381 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:51:54.381 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:51:54.381 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:51:54.381 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:51:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:51:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:51:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:51:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:51:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_items": 0, 2024-04-21T08:51:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:51:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_items": 0, 2024-04-21T08:51:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:51:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:51:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:51:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:51:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:51:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:51:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:51:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_items": 0, 2024-04-21T08:51:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:51:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:51:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:51:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_items": 0, 2024-04-21T08:51:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_bytes": 0, 2024-04-21T08:51:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_items": 0, 2024-04-21T08:51:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:51:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:51:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:51:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:51:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_bytes": 148754, 2024-04-21T08:51:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_items": 31, 2024-04-21T08:51:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:51:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_items": 0, 2024-04-21T08:51:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bytes": 0, 2024-04-21T08:51:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_items": 0, 2024-04-21T08:51:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:51:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_items": 0, 2024-04-21T08:51:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:51:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_items": 0, 2024-04-21T08:51:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_bytes": 0, 2024-04-21T08:51:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_items": 0, 2024-04-21T08:51:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:51:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:51:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_bytes": 0, 2024-04-21T08:51:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_items": 0, 2024-04-21T08:51:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_bytes": 928, 2024-04-21T08:51:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_items": 2, 2024-04-21T08:51:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_bytes": 0, 2024-04-21T08:51:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_items": 0, 2024-04-21T08:51:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_bytes": 0, 2024-04-21T08:51:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_items": 0 2024-04-21T08:51:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:51:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-21T08:51:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-21T08:51:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-21T08:51:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 269, 2024-04-21T08:51:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 1564954, 2024-04-21T08:51:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 269, 2024-04-21T08:51:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-21T08:51:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-21T08:51:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 269, 2024-04-21T08:51:54.386 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 1564954, 2024-04-21T08:51:54.386 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-21T08:51:54.386 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:54.386 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:54.386 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:54.386 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:54.386 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:54.386 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:51:54.948 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:54 smithi043 ceph-mon[27978]: pgmap v405: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-21T08:51:54.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:54 smithi175 ceph-mon[31920]: pgmap v405: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-21T08:51:55.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:54 smithi112 ceph-mon[31085]: pgmap v405: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-21T08:51:55.091 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-21T08:51:55.091 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-21T08:51:55.091 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-21T08:51:55.091 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-21T08:51:55.091 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-21T08:51:55.091 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-21T08:51:55.091 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-21T08:51:55.092 DEBUG:tasks.check_counter:Getting stats from g 2024-04-21T08:51:55.092 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-21T08:51:55.347 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:51:56.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:56 smithi175 ceph-mon[31920]: pgmap v406: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-21T08:51:56.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:56 smithi043 ceph-mon[27978]: pgmap v406: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-21T08:51:57.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:56 smithi112 ceph-mon[31085]: pgmap v406: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-21T08:51:57.282 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:51:57.283 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:51:57.283 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 906, 2024-04-21T08:51:57.283 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 908, 2024-04-21T08:51:57.283 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 304361, 2024-04-21T08:51:57.283 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 288109, 2024-04-21T08:51:57.283 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 7, 2024-04-21T08:51:57.283 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 2, 2024-04-21T08:51:57.283 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.106866267, 2024-04-21T08:51:57.283 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.044922237, 2024-04-21T08:51:57.283 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.022583550, 2024-04-21T08:51:57.283 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.027866596, 2024-04-21T08:51:57.283 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:57.284 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 906, 2024-04-21T08:51:57.284 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.050507908, 2024-04-21T08:51:57.284 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000055748 2024-04-21T08:51:57.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.284 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:57.284 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.284 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.284 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.284 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T08:51:57.284 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T08:51:57.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.284 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:51:57.285 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 1985, 2024-04-21T08:51:57.285 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 1969, 2024-04-21T08:51:57.285 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 2086921, 2024-04-21T08:51:57.285 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 698163, 2024-04-21T08:51:57.285 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 4, 2024-04-21T08:51:57.285 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 3, 2024-04-21T08:51:57.285 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.163990739, 2024-04-21T08:51:57.285 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.049452178, 2024-04-21T08:51:57.285 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.065374940, 2024-04-21T08:51:57.285 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.049021799, 2024-04-21T08:51:57.285 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:57.285 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1969, 2024-04-21T08:51:57.285 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.064977533, 2024-04-21T08:51:57.285 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000033000 2024-04-21T08:51:57.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.286 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:57.286 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.286 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.286 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.286 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 1654880, 2024-04-21T08:51:57.286 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 160368 2024-04-21T08:51:57.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.286 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:51:57.286 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 5, 2024-04-21T08:51:57.286 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 196, 2024-04-21T08:51:57.286 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 1668, 2024-04-21T08:51:57.286 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 500453, 2024-04-21T08:51:57.286 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 8, 2024-04-21T08:51:57.287 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 5, 2024-04-21T08:51:57.287 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.015406798, 2024-04-21T08:51:57.287 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.010172256, 2024-04-21T08:51:57.287 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.002719531, 2024-04-21T08:51:57.287 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000059548, 2024-04-21T08:51:57.287 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:57.287 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 193, 2024-04-21T08:51:57.287 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.012351710, 2024-04-21T08:51:57.287 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000063998 2024-04-21T08:51:57.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.287 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:57.287 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.287 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.287 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.288 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T08:51:57.288 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 499488 2024-04-21T08:51:57.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.288 INFO:teuthology.orchestra.run.smithi043.stdout: "cct": { 2024-04-21T08:51:57.288 INFO:teuthology.orchestra.run.smithi043.stdout: "total_workers": 1, 2024-04-21T08:51:57.288 INFO:teuthology.orchestra.run.smithi043.stdout: "unhealthy_workers": 0 2024-04-21T08:51:57.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.288 INFO:teuthology.orchestra.run.smithi043.stdout: "finisher-MDSRank": { 2024-04-21T08:51:57.288 INFO:teuthology.orchestra.run.smithi043.stdout: "queue_len": 0, 2024-04-21T08:51:57.288 INFO:teuthology.orchestra.run.smithi043.stdout: "complete_latency": { 2024-04-21T08:51:57.288 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 4400, 2024-04-21T08:51:57.288 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.114299641, 2024-04-21T08:51:57.288 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000025977 2024-04-21T08:51:57.288 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:57.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.289 INFO:teuthology.orchestra.run.smithi043.stdout: "finisher-PurgeQueue": { 2024-04-21T08:51:57.289 INFO:teuthology.orchestra.run.smithi043.stdout: "queue_len": 0, 2024-04-21T08:51:57.289 INFO:teuthology.orchestra.run.smithi043.stdout: "complete_latency": { 2024-04-21T08:51:57.289 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.289 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.289 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.289 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:57.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.289 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": { 2024-04-21T08:51:57.289 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-04-21T08:51:57.289 INFO:teuthology.orchestra.run.smithi043.stdout: "reply": 0, 2024-04-21T08:51:57.289 INFO:teuthology.orchestra.run.smithi043.stdout: "reply_latency": { 2024-04-21T08:51:57.289 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.289 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.290 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.290 INFO:teuthology.orchestra.run.smithi043.stdout: "slow_reply": 0, 2024-04-21T08:51:57.290 INFO:teuthology.orchestra.run.smithi043.stdout: "forward": 0, 2024-04-21T08:51:57.290 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_fetch_complete": 0, 2024-04-21T08:51:57.290 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_fetch_keys": 0, 2024-04-21T08:51:57.290 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_commit": 0, 2024-04-21T08:51:57.290 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_split": 0, 2024-04-21T08:51:57.290 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_merge": 0, 2024-04-21T08:51:57.290 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes": 0, 2024-04-21T08:51:57.290 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_top": 0, 2024-04-21T08:51:57.290 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_bottom": 0, 2024-04-21T08:51:57.290 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_pin_tail": 0, 2024-04-21T08:51:57.290 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_pinned": 0, 2024-04-21T08:51:57.290 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_expired": 0, 2024-04-21T08:51:57.291 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_with_caps": 0, 2024-04-21T08:51:57.291 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 0, 2024-04-21T08:51:57.291 INFO:teuthology.orchestra.run.smithi043.stdout: "subtrees": 1, 2024-04-21T08:51:57.291 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse": 0, 2024-04-21T08:51:57.291 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_hit": 0, 2024-04-21T08:51:57.291 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_forward": 0, 2024-04-21T08:51:57.291 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_discover": 0, 2024-04-21T08:51:57.291 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:51:57.291 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_remote_ino": 0, 2024-04-21T08:51:57.291 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_lock": 0, 2024-04-21T08:51:57.291 INFO:teuthology.orchestra.run.smithi043.stdout: "load_cent": 0, 2024-04-21T08:51:57.291 INFO:teuthology.orchestra.run.smithi043.stdout: "q": 0, 2024-04-21T08:51:57.291 INFO:teuthology.orchestra.run.smithi043.stdout: "exported": 0, 2024-04-21T08:51:57.291 INFO:teuthology.orchestra.run.smithi043.stdout: "exported_inodes": 0, 2024-04-21T08:51:57.291 INFO:teuthology.orchestra.run.smithi043.stdout: "imported": 0, 2024-04-21T08:51:57.292 INFO:teuthology.orchestra.run.smithi043.stdout: "imported_inodes": 0, 2024-04-21T08:51:57.292 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_dir_fetch": 0, 2024-04-21T08:51:57.292 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_backtrace_fetch": 0, 2024-04-21T08:51:57.292 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_peer_discover": 0, 2024-04-21T08:51:57.292 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rfiles": 0, 2024-04-21T08:51:57.292 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rbytes": 0, 2024-04-21T08:51:57.292 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rsnaps": 0, 2024-04-21T08:51:57.292 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T08:51:57.292 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_set_tag": 0, 2024-04-21T08:51:57.292 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:51:57.292 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:51:57.292 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:51:57.292 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:51:57.292 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:51:57.292 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_file_inodes": 0, 2024-04-21T08:51:57.293 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:51:57.293 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T08:51:57.293 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_grant": 0, 2024-04-21T08:51:57.293 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:51:57.293 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:51:57.293 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:51:57.293 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_caps": 0, 2024-04-21T08:51:57.293 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_caps_dirty": 0, 2024-04-21T08:51:57.293 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_cap_release": 0, 2024-04-21T08:51:57.293 INFO:teuthology.orchestra.run.smithi043.stdout: "process_request_cap_release": 0 2024-04-21T08:51:57.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.293 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cache": { 2024-04-21T08:51:57.293 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_update": 0, 2024-04-21T08:51:57.293 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_update_receipt": 0, 2024-04-21T08:51:57.294 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_try_discover": 0, 2024-04-21T08:51:57.294 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_send_discover": 0, 2024-04-21T08:51:57.294 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_handle_discover": 0, 2024-04-21T08:51:57.294 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays": 0, 2024-04-21T08:51:57.294 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays_delayed": 0, 2024-04-21T08:51:57.294 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:51:57.294 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_created": 0, 2024-04-21T08:51:57.294 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_enqueued": 0, 2024-04-21T08:51:57.294 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_reintegrated": 0, 2024-04-21T08:51:57.294 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_migrated": 0, 2024-04-21T08:51:57.294 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_processing": 0, 2024-04-21T08:51:57.294 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:51:57.294 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:51:57.294 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_started": 0, 2024-04-21T08:51:57.294 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_completed": 0, 2024-04-21T08:51:57.295 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_quiesce_path": 0, 2024-04-21T08:51:57.295 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_quiesce_inode": 0, 2024-04-21T08:51:57.295 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:51:57.295 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_exportdir": 0, 2024-04-21T08:51:57.295 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_flush": 0, 2024-04-21T08:51:57.295 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:51:57.295 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_fragstats": 0, 2024-04-21T08:51:57.295 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_inodestats": 0 2024-04-21T08:51:57.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.295 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_log": { 2024-04-21T08:51:57.295 INFO:teuthology.orchestra.run.smithi043.stdout: "evlrg": 0, 2024-04-21T08:51:57.295 INFO:teuthology.orchestra.run.smithi043.stdout: "evadd": 0, 2024-04-21T08:51:57.295 INFO:teuthology.orchestra.run.smithi043.stdout: "evex": 0, 2024-04-21T08:51:57.295 INFO:teuthology.orchestra.run.smithi043.stdout: "evtrm": 0, 2024-04-21T08:51:57.295 INFO:teuthology.orchestra.run.smithi043.stdout: "ev": 5, 2024-04-21T08:51:57.296 INFO:teuthology.orchestra.run.smithi043.stdout: "evexg": 0, 2024-04-21T08:51:57.296 INFO:teuthology.orchestra.run.smithi043.stdout: "evexd": 0, 2024-04-21T08:51:57.296 INFO:teuthology.orchestra.run.smithi043.stdout: "segadd": 0, 2024-04-21T08:51:57.296 INFO:teuthology.orchestra.run.smithi043.stdout: "segex": 0, 2024-04-21T08:51:57.296 INFO:teuthology.orchestra.run.smithi043.stdout: "segtrm": 0, 2024-04-21T08:51:57.296 INFO:teuthology.orchestra.run.smithi043.stdout: "seg": 2, 2024-04-21T08:51:57.296 INFO:teuthology.orchestra.run.smithi043.stdout: "segmjr": 2, 2024-04-21T08:51:57.296 INFO:teuthology.orchestra.run.smithi043.stdout: "segexg": 0, 2024-04-21T08:51:57.296 INFO:teuthology.orchestra.run.smithi043.stdout: "segexd": 0, 2024-04-21T08:51:57.296 INFO:teuthology.orchestra.run.smithi043.stdout: "expos": 4194304, 2024-04-21T08:51:57.296 INFO:teuthology.orchestra.run.smithi043.stdout: "wrpos": 4195815, 2024-04-21T08:51:57.296 INFO:teuthology.orchestra.run.smithi043.stdout: "rdpos": 4195674, 2024-04-21T08:51:57.296 INFO:teuthology.orchestra.run.smithi043.stdout: "jlat": { 2024-04-21T08:51:57.296 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.296 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.297 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.297 INFO:teuthology.orchestra.run.smithi043.stdout: "replayed": 5 2024-04-21T08:51:57.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.297 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_mem": { 2024-04-21T08:51:57.297 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 3, 2024-04-21T08:51:57.297 INFO:teuthology.orchestra.run.smithi043.stdout: "ino+": 3, 2024-04-21T08:51:57.297 INFO:teuthology.orchestra.run.smithi043.stdout: "ino-": 0, 2024-04-21T08:51:57.297 INFO:teuthology.orchestra.run.smithi043.stdout: "dir": 1, 2024-04-21T08:51:57.297 INFO:teuthology.orchestra.run.smithi043.stdout: "dir+": 1, 2024-04-21T08:51:57.297 INFO:teuthology.orchestra.run.smithi043.stdout: "dir-": 0, 2024-04-21T08:51:57.297 INFO:teuthology.orchestra.run.smithi043.stdout: "dn": 0, 2024-04-21T08:51:57.297 INFO:teuthology.orchestra.run.smithi043.stdout: "dn+": 0, 2024-04-21T08:51:57.297 INFO:teuthology.orchestra.run.smithi043.stdout: "dn-": 0, 2024-04-21T08:51:57.297 INFO:teuthology.orchestra.run.smithi043.stdout: "cap": 0, 2024-04-21T08:51:57.297 INFO:teuthology.orchestra.run.smithi043.stdout: "cap+": 0, 2024-04-21T08:51:57.298 INFO:teuthology.orchestra.run.smithi043.stdout: "cap-": 0, 2024-04-21T08:51:57.298 INFO:teuthology.orchestra.run.smithi043.stdout: "rss": 50552, 2024-04-21T08:51:57.298 INFO:teuthology.orchestra.run.smithi043.stdout: "heap": 215416 2024-04-21T08:51:57.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.298 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_server": { 2024-04-21T08:51:57.298 INFO:teuthology.orchestra.run.smithi043.stdout: "dispatch_client_request": 0, 2024-04-21T08:51:57.298 INFO:teuthology.orchestra.run.smithi043.stdout: "dispatch_server_request": 0, 2024-04-21T08:51:57.298 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_request": 0, 2024-04-21T08:51:57.298 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_session": 0, 2024-04-21T08:51:57.298 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_peer_request": 0, 2024-04-21T08:51:57.298 INFO:teuthology.orchestra.run.smithi043.stdout: "req_create_latency": { 2024-04-21T08:51:57.298 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.298 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.298 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.299 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getattr_latency": { 2024-04-21T08:51:57.299 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.299 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.299 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.299 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getfilelock_latency": { 2024-04-21T08:51:57.299 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.299 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.299 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.299 INFO:teuthology.orchestra.run.smithi043.stdout: "req_link_latency": { 2024-04-21T08:51:57.299 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.299 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.299 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.300 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookup_latency": { 2024-04-21T08:51:57.300 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.300 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.300 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.300 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookuphash_latency": { 2024-04-21T08:51:57.300 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.300 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.300 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.300 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupino_latency": { 2024-04-21T08:51:57.300 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.300 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.300 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.301 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupname_latency": { 2024-04-21T08:51:57.301 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.301 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.301 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.301 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupparent_latency": { 2024-04-21T08:51:57.301 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.301 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.301 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.301 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupsnap_latency": { 2024-04-21T08:51:57.301 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.301 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.301 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.302 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lssnap_latency": { 2024-04-21T08:51:57.302 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.302 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.302 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.302 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mkdir_latency": { 2024-04-21T08:51:57.302 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.303 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.303 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.303 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mknod_latency": { 2024-04-21T08:51:57.303 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.303 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.303 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.303 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mksnap_latency": { 2024-04-21T08:51:57.303 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.303 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.303 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.304 INFO:teuthology.orchestra.run.smithi043.stdout: "req_open_latency": { 2024-04-21T08:51:57.304 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.304 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.304 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.304 INFO:teuthology.orchestra.run.smithi043.stdout: "req_readdir_latency": { 2024-04-21T08:51:57.304 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.304 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.304 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.304 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rename_latency": { 2024-04-21T08:51:57.304 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.304 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.305 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.305 INFO:teuthology.orchestra.run.smithi043.stdout: "req_renamesnap_latency": { 2024-04-21T08:51:57.305 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.305 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.305 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.305 INFO:teuthology.orchestra.run.smithi043.stdout: "req_snapdiff_latency": { 2024-04-21T08:51:57.305 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.305 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.305 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.306 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmdir_latency": { 2024-04-21T08:51:57.306 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.306 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.306 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.306 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmsnap_latency": { 2024-04-21T08:51:57.306 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.306 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.306 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.306 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmxattr_latency": { 2024-04-21T08:51:57.306 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.306 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.306 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.307 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setattr_latency": { 2024-04-21T08:51:57.307 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.307 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.307 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.307 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setdirlayout_latency": { 2024-04-21T08:51:57.307 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.307 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.307 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.307 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setfilelock_latency": { 2024-04-21T08:51:57.307 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.307 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.308 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.308 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setlayout_latency": { 2024-04-21T08:51:57.308 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.308 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.308 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.308 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setxattr_latency": { 2024-04-21T08:51:57.308 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.308 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.308 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.308 INFO:teuthology.orchestra.run.smithi043.stdout: "req_symlink_latency": { 2024-04-21T08:51:57.308 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.308 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.309 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.309 INFO:teuthology.orchestra.run.smithi043.stdout: "req_unlink_latency": { 2024-04-21T08:51:57.309 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.309 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.309 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.309 INFO:teuthology.orchestra.run.smithi043.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:51:57.309 INFO:teuthology.orchestra.run.smithi043.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:51:57.309 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getvxattr_latency": { 2024-04-21T08:51:57.309 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.309 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.309 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.309 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:57.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.310 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_sessions": { 2024-04-21T08:51:57.310 INFO:teuthology.orchestra.run.smithi043.stdout: "session_count": 1, 2024-04-21T08:51:57.310 INFO:teuthology.orchestra.run.smithi043.stdout: "session_add": 2, 2024-04-21T08:51:57.310 INFO:teuthology.orchestra.run.smithi043.stdout: "session_remove": 1, 2024-04-21T08:51:57.310 INFO:teuthology.orchestra.run.smithi043.stdout: "sessions_open": 2, 2024-04-21T08:51:57.310 INFO:teuthology.orchestra.run.smithi043.stdout: "sessions_stale": 0, 2024-04-21T08:51:57.310 INFO:teuthology.orchestra.run.smithi043.stdout: "total_load": 0, 2024-04-21T08:51:57.310 INFO:teuthology.orchestra.run.smithi043.stdout: "average_load": 0, 2024-04-21T08:51:57.310 INFO:teuthology.orchestra.run.smithi043.stdout: "avg_session_uptime": 3161, 2024-04-21T08:51:57.310 INFO:teuthology.orchestra.run.smithi043.stdout: "mdthresh_evicted": 0 2024-04-21T08:51:57.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.310 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool": { 2024-04-21T08:51:57.310 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:51:57.310 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_items": 0, 2024-04-21T08:51:57.311 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:51:57.311 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:51:57.311 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:51:57.311 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:51:57.311 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:51:57.311 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:51:57.311 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:51:57.311 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:51:57.311 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:51:57.311 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:51:57.311 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:51:57.311 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:51:57.311 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:51:57.311 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_items": 0, 2024-04-21T08:51:57.311 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:51:57.312 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_items": 0, 2024-04-21T08:51:57.312 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:51:57.312 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:51:57.312 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:51:57.312 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:51:57.312 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:51:57.312 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:51:57.314 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:51:57.314 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_items": 0, 2024-04-21T08:51:57.314 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:51:57.314 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:51:57.314 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:51:57.314 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_items": 0, 2024-04-21T08:51:57.314 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_bytes": 0, 2024-04-21T08:51:57.314 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_items": 0, 2024-04-21T08:51:57.314 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:51:57.314 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:51:57.314 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:51:57.315 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:51:57.315 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_bytes": 197473, 2024-04-21T08:51:57.315 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_items": 55, 2024-04-21T08:51:57.315 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:51:57.315 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_items": 0, 2024-04-21T08:51:57.315 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bytes": 0, 2024-04-21T08:51:57.315 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_items": 0, 2024-04-21T08:51:57.315 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:51:57.315 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_items": 0, 2024-04-21T08:51:57.315 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:51:57.315 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_items": 0, 2024-04-21T08:51:57.315 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_bytes": 9800, 2024-04-21T08:51:57.315 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_items": 188, 2024-04-21T08:51:57.315 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:51:57.315 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:51:57.316 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_bytes": 0, 2024-04-21T08:51:57.316 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_items": 0, 2024-04-21T08:51:57.316 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_bytes": 8512, 2024-04-21T08:51:57.316 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_items": 13, 2024-04-21T08:51:57.316 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_bytes": 0, 2024-04-21T08:51:57.316 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_items": 0, 2024-04-21T08:51:57.316 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_bytes": 0, 2024-04-21T08:51:57.316 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_items": 0 2024-04-21T08:51:57.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.316 INFO:teuthology.orchestra.run.smithi043.stdout: "objecter": { 2024-04-21T08:51:57.316 INFO:teuthology.orchestra.run.smithi043.stdout: "op_active": 0, 2024-04-21T08:51:57.316 INFO:teuthology.orchestra.run.smithi043.stdout: "op_laggy": 0, 2024-04-21T08:51:57.316 INFO:teuthology.orchestra.run.smithi043.stdout: "op_send": 2600, 2024-04-21T08:51:57.316 INFO:teuthology.orchestra.run.smithi043.stdout: "op_send_bytes": 21, 2024-04-21T08:51:57.317 INFO:teuthology.orchestra.run.smithi043.stdout: "op_resend": 3, 2024-04-21T08:51:57.317 INFO:teuthology.orchestra.run.smithi043.stdout: "op_reply": 2600, 2024-04-21T08:51:57.317 INFO:teuthology.orchestra.run.smithi043.stdout: "op_latency": { 2024-04-21T08:51:57.317 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 2600, 2024-04-21T08:51:57.317 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 1.270984608, 2024-04-21T08:51:57.317 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000488840 2024-04-21T08:51:57.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.317 INFO:teuthology.orchestra.run.smithi043.stdout: "op_inflight": 0, 2024-04-21T08:51:57.317 INFO:teuthology.orchestra.run.smithi043.stdout: "oplen_avg": { 2024-04-21T08:51:57.317 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 2600, 2024-04-21T08:51:57.317 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 2601 2024-04-21T08:51:57.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.317 INFO:teuthology.orchestra.run.smithi043.stdout: "op": 2600, 2024-04-21T08:51:57.317 INFO:teuthology.orchestra.run.smithi043.stdout: "op_r": 2600, 2024-04-21T08:51:57.317 INFO:teuthology.orchestra.run.smithi043.stdout: "op_w": 0, 2024-04-21T08:51:57.318 INFO:teuthology.orchestra.run.smithi043.stdout: "op_rmw": 0, 2024-04-21T08:51:57.318 INFO:teuthology.orchestra.run.smithi043.stdout: "op_pg": 0, 2024-04-21T08:51:57.318 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_stat": 1686, 2024-04-21T08:51:57.318 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_create": 0, 2024-04-21T08:51:57.318 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_read": 912, 2024-04-21T08:51:57.318 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_write": 0, 2024-04-21T08:51:57.318 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_writefull": 0, 2024-04-21T08:51:57.318 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_writesame": 0, 2024-04-21T08:51:57.318 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_append": 0, 2024-04-21T08:51:57.318 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_zero": 0, 2024-04-21T08:51:57.318 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_truncate": 0, 2024-04-21T08:51:57.318 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_delete": 0, 2024-04-21T08:51:57.318 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_mapext": 0, 2024-04-21T08:51:57.318 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_sparse_read": 0, 2024-04-21T08:51:57.319 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_clonerange": 0, 2024-04-21T08:51:57.319 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_getxattr": 1, 2024-04-21T08:51:57.319 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_setxattr": 0, 2024-04-21T08:51:57.319 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:51:57.319 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_rmxattr": 0, 2024-04-21T08:51:57.319 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:51:57.319 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_call": 0, 2024-04-21T08:51:57.319 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_watch": 0, 2024-04-21T08:51:57.319 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_notify": 0, 2024-04-21T08:51:57.319 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:51:57.319 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_pgls": 0, 2024-04-21T08:51:57.319 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:51:57.320 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_other": 0, 2024-04-21T08:51:57.320 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_active": 0, 2024-04-21T08:51:57.320 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_send": 0, 2024-04-21T08:51:57.320 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_resend": 0, 2024-04-21T08:51:57.320 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_ping": 0, 2024-04-21T08:51:57.320 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_active": 0, 2024-04-21T08:51:57.320 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_send": 0, 2024-04-21T08:51:57.320 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_resend": 0, 2024-04-21T08:51:57.320 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_active": 0, 2024-04-21T08:51:57.320 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_send": 0, 2024-04-21T08:51:57.320 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_resend": 0, 2024-04-21T08:51:57.320 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_active": 0, 2024-04-21T08:51:57.320 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_send": 0, 2024-04-21T08:51:57.320 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_resend": 0, 2024-04-21T08:51:57.320 INFO:teuthology.orchestra.run.smithi043.stdout: "command_active": 0, 2024-04-21T08:51:57.321 INFO:teuthology.orchestra.run.smithi043.stdout: "command_send": 0, 2024-04-21T08:51:57.321 INFO:teuthology.orchestra.run.smithi043.stdout: "command_resend": 0, 2024-04-21T08:51:57.321 INFO:teuthology.orchestra.run.smithi043.stdout: "map_epoch": 108, 2024-04-21T08:51:57.321 INFO:teuthology.orchestra.run.smithi043.stdout: "map_full": 0, 2024-04-21T08:51:57.321 INFO:teuthology.orchestra.run.smithi043.stdout: "map_inc": 18, 2024-04-21T08:51:57.321 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_sessions": 6, 2024-04-21T08:51:57.321 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_session_open": 8, 2024-04-21T08:51:57.321 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_session_close": 2, 2024-04-21T08:51:57.321 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_laggy": 0, 2024-04-21T08:51:57.321 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_wr": 0, 2024-04-21T08:51:57.321 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_rd": 2, 2024-04-21T08:51:57.321 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_del": 0 2024-04-21T08:51:57.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.321 INFO:teuthology.orchestra.run.smithi043.stdout: "oft": { 2024-04-21T08:51:57.321 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_objs": 0, 2024-04-21T08:51:57.322 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:51:57.322 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_updates": 0, 2024-04-21T08:51:57.322 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_removes": 0 2024-04-21T08:51:57.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.322 INFO:teuthology.orchestra.run.smithi043.stdout: "osd-slow-ops": { 2024-04-21T08:51:57.322 INFO:teuthology.orchestra.run.smithi043.stdout: "slow_ops_count": 0 2024-04-21T08:51:57.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.322 INFO:teuthology.orchestra.run.smithi043.stdout: "purge_queue": { 2024-04-21T08:51:57.322 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_ops": 0, 2024-04-21T08:51:57.322 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T08:51:57.322 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing": 0, 2024-04-21T08:51:57.322 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_high_water": 0, 2024-04-21T08:51:57.322 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executed_ops": 0, 2024-04-21T08:51:57.322 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executed": 0, 2024-04-21T08:51:57.322 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_item_in_journal": 0 2024-04-21T08:51:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:51:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-21T08:51:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-21T08:51:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-21T08:51:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 2892, 2024-04-21T08:51:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 2114832, 2024-04-21T08:51:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 2892, 2024-04-21T08:51:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-21T08:51:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-21T08:51:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 2892, 2024-04-21T08:51:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 2114832, 2024-04-21T08:51:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-21T08:51:57.323 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-objecter_bytes": { 2024-04-21T08:51:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-21T08:51:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-21T08:51:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-21T08:51:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-21T08:51:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-21T08:51:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 2600, 2024-04-21T08:51:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 1516, 2024-04-21T08:51:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 2600, 2024-04-21T08:51:57.324 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 1516, 2024-04-21T08:51:57.325 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-21T08:51:57.325 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.325 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.325 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.325 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:57.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.325 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-objecter_ops": { 2024-04-21T08:51:57.325 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-21T08:51:57.325 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 1024, 2024-04-21T08:51:57.325 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-21T08:51:57.325 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-21T08:51:57.325 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-21T08:51:57.325 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:57.325 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:57.325 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 2600, 2024-04-21T08:51:57.326 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 2600, 2024-04-21T08:51:57.326 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 2600, 2024-04-21T08:51:57.326 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 2600, 2024-04-21T08:51:57.326 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-21T08:51:57.326 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.326 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.326 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.326 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:57.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.326 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:51:57.326 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-21T08:51:57.326 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 3758096384, 2024-04-21T08:51:57.326 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-21T08:51:57.326 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-21T08:51:57.326 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-21T08:51:57.327 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:57.327 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:57.327 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-21T08:51:57.327 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-21T08:51:57.327 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 0, 2024-04-21T08:51:57.327 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 0, 2024-04-21T08:51:57.327 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-21T08:51:57.327 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.327 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.327 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.327 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:57.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:51:57.327 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-write_buf_throttle-0x55ac928126e0": { 2024-04-21T08:51:57.327 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-21T08:51:57.327 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 3758096384, 2024-04-21T08:51:57.328 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-21T08:51:57.328 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-21T08:51:57.328 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-21T08:51:57.328 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:57.328 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:57.328 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-21T08:51:57.328 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-21T08:51:57.328 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 0, 2024-04-21T08:51:57.328 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 0, 2024-04-21T08:51:57.328 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-21T08:51:57.328 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:51:57.328 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:51:57.328 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:51:57.328 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:57.328 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:51:57.328 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:51:58.066 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-21T08:51:58.066 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-21T08:51:58.066 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-04-21T08:51:58.066 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-04-21T08:51:58.066 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-04-21T08:51:58.066 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:51:58.066 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:51:58.066 DEBUG:tasks.check_counter:Getting stats from j 2024-04-21T08:51:58.066 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-21T08:51:58.317 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:51:58.729 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:51:58 smithi043 ceph-mon[27978]: pgmap v407: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 13 op/s 2024-04-21T08:51:58.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:51:58 smithi175 ceph-mon[31920]: pgmap v407: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 13 op/s 2024-04-21T08:51:59.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:51:58 smithi112 ceph-mon[31085]: pgmap v407: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 13 op/s 2024-04-21T08:52:00.127 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-21T08:52:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:52:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 269, 2024-04-21T08:52:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 258, 2024-04-21T08:52:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 1603168, 2024-04-21T08:52:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 159120, 2024-04-21T08:52:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 4, 2024-04-21T08:52:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 0, 2024-04-21T08:52:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.052363843, 2024-04-21T08:52:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.017918813, 2024-04-21T08:52:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.036018092, 2024-04-21T08:52:00.127 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.005467897, 2024-04-21T08:52:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 257, 2024-04-21T08:52:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.012990715, 2024-04-21T08:52:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000050547 2024-04-21T08:52:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:52:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:52:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:52:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:52:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:52:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 1603168, 2024-04-21T08:52:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 159120 2024-04-21T08:52:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:52:00.128 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:52:00.129 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 6, 2024-04-21T08:52:00.129 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 199, 2024-04-21T08:52:00.129 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 3312, 2024-04-21T08:52:00.129 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 191200, 2024-04-21T08:52:00.129 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 5, 2024-04-21T08:52:00.129 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 3, 2024-04-21T08:52:00.129 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.018252427, 2024-04-21T08:52:00.129 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.012879582, 2024-04-21T08:52:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.002337897, 2024-04-21T08:52:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T08:52:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 197, 2024-04-21T08:52:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.012834559, 2024-04-21T08:52:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000065150 2024-04-21T08:52:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:52:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:52:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:52:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:52:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:52:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-21T08:52:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 191200 2024-04-21T08:52:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:52:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:52:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 0, 2024-04-21T08:52:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 2, 2024-04-21T08:52:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 0, 2024-04-21T08:52:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 0, 2024-04-21T08:52:00.131 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 2, 2024-04-21T08:52:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 1, 2024-04-21T08:52:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.000345077, 2024-04-21T08:52:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.000000000, 2024-04-21T08:52:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.000274361, 2024-04-21T08:52:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T08:52:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:52:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:52:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:52:00.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:52:00.133 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:00.133 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:52:00.133 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:52:00.133 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:52:00.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:52:00.133 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T08:52:00.133 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T08:52:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:52:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: "cct": { 2024-04-21T08:52:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: "total_workers": 0, 2024-04-21T08:52:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: "unhealthy_workers": 0 2024-04-21T08:52:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:52:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool": { 2024-04-21T08:52:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:52:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_items": 0, 2024-04-21T08:52:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:52:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:52:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:52:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:52:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:52:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:52:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:52:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:52:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:52:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:52:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:52:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:52:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:52:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_items": 0, 2024-04-21T08:52:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:52:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_items": 0, 2024-04-21T08:52:00.135 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:52:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:52:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:52:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:52:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:52:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:52:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:52:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_items": 0, 2024-04-21T08:52:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:52:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:52:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:52:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_items": 0, 2024-04-21T08:52:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_bytes": 0, 2024-04-21T08:52:00.136 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_items": 0, 2024-04-21T08:52:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:52:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:52:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:52:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:52:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_bytes": 148709, 2024-04-21T08:52:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_items": 31, 2024-04-21T08:52:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:52:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_items": 0, 2024-04-21T08:52:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bytes": 0, 2024-04-21T08:52:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_items": 0, 2024-04-21T08:52:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:52:00.137 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_items": 0, 2024-04-21T08:52:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:52:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_items": 0, 2024-04-21T08:52:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_bytes": 0, 2024-04-21T08:52:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_items": 0, 2024-04-21T08:52:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:52:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:52:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_bytes": 0, 2024-04-21T08:52:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_items": 0, 2024-04-21T08:52:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_bytes": 928, 2024-04-21T08:52:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_items": 2, 2024-04-21T08:52:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_bytes": 0, 2024-04-21T08:52:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_items": 0, 2024-04-21T08:52:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_bytes": 0, 2024-04-21T08:52:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_items": 0 2024-04-21T08:52:00.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-21T08:52:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:52:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-21T08:52:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-21T08:52:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-21T08:52:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 271, 2024-04-21T08:52:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 1565214, 2024-04-21T08:52:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 271, 2024-04-21T08:52:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-21T08:52:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-21T08:52:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 271, 2024-04-21T08:52:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 1565214, 2024-04-21T08:52:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-21T08:52:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-21T08:52:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-21T08:52:00.139 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-21T08:52:00.140 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:52:00.140 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-21T08:52:00.140 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-21T08:52:00.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:00 smithi043 ceph-mon[27978]: pgmap v408: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-21T08:52:00.934 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-21T08:52:00.934 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-21T08:52:00.934 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-21T08:52:00.934 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-21T08:52:00.934 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-21T08:52:00.934 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-21T08:52:00.934 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-21T08:52:00.934 DEBUG:tasks.check_counter:Getting stats from b 2024-04-21T08:52:00.935 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-21T08:52:00.961 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:00 smithi112 ceph-mon[31085]: pgmap v408: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-21T08:52:00.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:00 smithi175 ceph-mon[31920]: pgmap v408: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-21T08:52:01.184 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:52:02.944 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:02 smithi112 ceph-mon[31085]: pgmap v409: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-21T08:52:02.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:02 smithi175 ceph-mon[31920]: pgmap v409: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-21T08:52:03.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:02 smithi043 ceph-mon[27978]: pgmap v409: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-21T08:52:03.069 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-21T08:52:03.070 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:52:03.070 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 4882, 2024-04-21T08:52:03.070 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 5922, 2024-04-21T08:52:03.070 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 1515261, 2024-04-21T08:52:03.070 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 15069352, 2024-04-21T08:52:03.070 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 12, 2024-04-21T08:52:03.070 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 5, 2024-04-21T08:52:03.070 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.474277224, 2024-04-21T08:52:03.070 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.169160987, 2024-04-21T08:52:03.070 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.149716910, 2024-04-21T08:52:03.070 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.112988867, 2024-04-21T08:52:03.070 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:03.070 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 5920, 2024-04-21T08:52:03.071 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.394551198, 2024-04-21T08:52:03.071 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000066647 2024-04-21T08:52:03.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.071 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:03.071 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 4893, 2024-04-21T08:52:03.071 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.001313342, 2024-04-21T08:52:03.071 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000268 2024-04-21T08:52:03.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.071 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-21T08:52:03.071 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 373584 2024-04-21T08:52:03.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.071 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:52:03.071 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 4714, 2024-04-21T08:52:03.071 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 5607, 2024-04-21T08:52:03.072 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 3322848, 2024-04-21T08:52:03.072 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 9045008, 2024-04-21T08:52:03.072 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 39, 2024-04-21T08:52:03.072 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 5, 2024-04-21T08:52:03.072 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.605625017, 2024-04-21T08:52:03.072 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.204741813, 2024-04-21T08:52:03.072 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.271628193, 2024-04-21T08:52:03.072 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.144283130, 2024-04-21T08:52:03.072 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:03.072 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 5607, 2024-04-21T08:52:03.072 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.617428352, 2024-04-21T08:52:03.072 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000110117 2024-04-21T08:52:03.072 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.073 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:03.073 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 6038, 2024-04-21T08:52:03.073 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.001162163, 2024-04-21T08:52:03.073 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000192 2024-04-21T08:52:03.073 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.073 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3002160, 2024-04-21T08:52:03.073 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 1867008 2024-04-21T08:52:03.073 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.073 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:52:03.073 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 3629, 2024-04-21T08:52:03.073 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 4345, 2024-04-21T08:52:03.073 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 1264658, 2024-04-21T08:52:03.073 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 9908535, 2024-04-21T08:52:03.073 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 346, 2024-04-21T08:52:03.073 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 7, 2024-04-21T08:52:03.073 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.456414147, 2024-04-21T08:52:03.074 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.151984693, 2024-04-21T08:52:03.074 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.221436210, 2024-04-21T08:52:03.074 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.070898350, 2024-04-21T08:52:03.074 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:03.074 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 4342, 2024-04-21T08:52:03.074 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.253636797, 2024-04-21T08:52:03.074 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000058414 2024-04-21T08:52:03.074 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.074 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:03.074 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 5474, 2024-04-21T08:52:03.074 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.001251269, 2024-04-21T08:52:03.074 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000228 2024-04-21T08:52:03.074 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.075 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 84928, 2024-04-21T08:52:03.075 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 140864 2024-04-21T08:52:03.075 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.075 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-04-21T08:52:03.075 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 1, 2024-04-21T08:52:03.075 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-04-21T08:52:03.075 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.075 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-MDSRank": { 2024-04-21T08:52:03.075 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-04-21T08:52:03.075 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-04-21T08:52:03.075 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 6423, 2024-04-21T08:52:03.075 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 2.068593726, 2024-04-21T08:52:03.075 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000322060 2024-04-21T08:52:03.075 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:03.075 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.075 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-PurgeQueue": { 2024-04-21T08:52:03.076 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-04-21T08:52:03.076 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-04-21T08:52:03.076 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 16, 2024-04-21T08:52:03.076 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000366198, 2024-04-21T08:52:03.076 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000022887 2024-04-21T08:52:03.076 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:03.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.076 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-04-21T08:52:03.076 INFO:teuthology.orchestra.run.smithi112.stdout: "request": 811, 2024-04-21T08:52:03.076 INFO:teuthology.orchestra.run.smithi112.stdout: "reply": 811, 2024-04-21T08:52:03.076 INFO:teuthology.orchestra.run.smithi112.stdout: "reply_latency": { 2024-04-21T08:52:03.076 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 811, 2024-04-21T08:52:03.076 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 3.590054416, 2024-04-21T08:52:03.076 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.004426700 2024-04-21T08:52:03.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.077 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_reply": 0, 2024-04-21T08:52:03.077 INFO:teuthology.orchestra.run.smithi112.stdout: "forward": 0, 2024-04-21T08:52:03.077 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_complete": 0, 2024-04-21T08:52:03.077 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_keys": 0, 2024-04-21T08:52:03.077 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_commit": 378, 2024-04-21T08:52:03.077 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_split": 0, 2024-04-21T08:52:03.077 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_merge": 0, 2024-04-21T08:52:03.077 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes": 44, 2024-04-21T08:52:03.077 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_top": 15, 2024-04-21T08:52:03.077 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_bottom": 29, 2024-04-21T08:52:03.077 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pin_tail": 0, 2024-04-21T08:52:03.077 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pinned": 22, 2024-04-21T08:52:03.077 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_expired": 0, 2024-04-21T08:52:03.077 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_with_caps": 2, 2024-04-21T08:52:03.077 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": 2, 2024-04-21T08:52:03.077 INFO:teuthology.orchestra.run.smithi112.stdout: "subtrees": 2, 2024-04-21T08:52:03.078 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse": 903, 2024-04-21T08:52:03.078 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_hit": 892, 2024-04-21T08:52:03.078 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_forward": 0, 2024-04-21T08:52:03.078 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_discover": 0, 2024-04-21T08:52:03.078 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:52:03.078 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_remote_ino": 0, 2024-04-21T08:52:03.078 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_lock": 0, 2024-04-21T08:52:03.078 INFO:teuthology.orchestra.run.smithi112.stdout: "load_cent": 27, 2024-04-21T08:52:03.078 INFO:teuthology.orchestra.run.smithi112.stdout: "q": 0, 2024-04-21T08:52:03.078 INFO:teuthology.orchestra.run.smithi112.stdout: "exported": 0, 2024-04-21T08:52:03.078 INFO:teuthology.orchestra.run.smithi112.stdout: "exported_inodes": 0, 2024-04-21T08:52:03.078 INFO:teuthology.orchestra.run.smithi112.stdout: "imported": 0, 2024-04-21T08:52:03.078 INFO:teuthology.orchestra.run.smithi112.stdout: "imported_inodes": 0, 2024-04-21T08:52:03.078 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_dir_fetch": 0, 2024-04-21T08:52:03.079 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_backtrace_fetch": 415, 2024-04-21T08:52:03.079 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_peer_discover": 0, 2024-04-21T08:52:03.079 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rfiles": 4, 2024-04-21T08:52:03.079 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rbytes": 234, 2024-04-21T08:52:03.079 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rsnaps": 6, 2024-04-21T08:52:03.079 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_fetch": 415, 2024-04-21T08:52:03.079 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_set_tag": 0, 2024-04-21T08:52:03.079 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:52:03.079 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:52:03.079 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_inodes": 267, 2024-04-21T08:52:03.079 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_base_inodes": 30, 2024-04-21T08:52:03.079 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dirfrag_rstats": 237, 2024-04-21T08:52:03.079 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_file_inodes": 148, 2024-04-21T08:52:03.079 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:52:03.080 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_revoke": 627, 2024-04-21T08:52:03.080 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_grant": 623, 2024-04-21T08:52:03.080 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_trunc": 599, 2024-04-21T08:52:03.080 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:52:03.080 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:52:03.080 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps": 732, 2024-04-21T08:52:03.080 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps_dirty": 702, 2024-04-21T08:52:03.080 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_cap_release": 3, 2024-04-21T08:52:03.080 INFO:teuthology.orchestra.run.smithi112.stdout: "process_request_cap_release": 624 2024-04-21T08:52:03.080 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.080 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_cache": { 2024-04-21T08:52:03.080 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update": 20, 2024-04-21T08:52:03.080 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update_receipt": 0, 2024-04-21T08:52:03.080 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_try_discover": 0, 2024-04-21T08:52:03.081 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_send_discover": 0, 2024-04-21T08:52:03.081 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_handle_discover": 45, 2024-04-21T08:52:03.081 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays": 2, 2024-04-21T08:52:03.081 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_delayed": 0, 2024-04-21T08:52:03.081 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:52:03.081 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_created": 3, 2024-04-21T08:52:03.081 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_enqueued": 1, 2024-04-21T08:52:03.081 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_reintegrated": 0, 2024-04-21T08:52:03.081 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_migrated": 0, 2024-04-21T08:52:03.081 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_processing": 0, 2024-04-21T08:52:03.081 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:52:03.081 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:52:03.081 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_started": 0, 2024-04-21T08:52:03.081 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_completed": 0, 2024-04-21T08:52:03.081 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_path": 3, 2024-04-21T08:52:03.082 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_inode": 42, 2024-04-21T08:52:03.082 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_enqueue_scrub": 30, 2024-04-21T08:52:03.082 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_exportdir": 0, 2024-04-21T08:52:03.082 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_flush": 0, 2024-04-21T08:52:03.082 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:52:03.082 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragstats": 0, 2024-04-21T08:52:03.082 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_inodestats": 0 2024-04-21T08:52:03.082 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.082 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_log": { 2024-04-21T08:52:03.082 INFO:teuthology.orchestra.run.smithi112.stdout: "evlrg": 0, 2024-04-21T08:52:03.082 INFO:teuthology.orchestra.run.smithi112.stdout: "evadd": 2590, 2024-04-21T08:52:03.082 INFO:teuthology.orchestra.run.smithi112.stdout: "evex": 2545, 2024-04-21T08:52:03.082 INFO:teuthology.orchestra.run.smithi112.stdout: "evtrm": 2545, 2024-04-21T08:52:03.083 INFO:teuthology.orchestra.run.smithi112.stdout: "ev": 45, 2024-04-21T08:52:03.083 INFO:teuthology.orchestra.run.smithi112.stdout: "evexg": 0, 2024-04-21T08:52:03.083 INFO:teuthology.orchestra.run.smithi112.stdout: "evexd": 52, 2024-04-21T08:52:03.083 INFO:teuthology.orchestra.run.smithi112.stdout: "segadd": 63, 2024-04-21T08:52:03.083 INFO:teuthology.orchestra.run.smithi112.stdout: "segex": 62, 2024-04-21T08:52:03.083 INFO:teuthology.orchestra.run.smithi112.stdout: "segtrm": 62, 2024-04-21T08:52:03.083 INFO:teuthology.orchestra.run.smithi112.stdout: "seg": 1, 2024-04-21T08:52:03.083 INFO:teuthology.orchestra.run.smithi112.stdout: "segmjr": 1, 2024-04-21T08:52:03.083 INFO:teuthology.orchestra.run.smithi112.stdout: "segexg": 0, 2024-04-21T08:52:03.083 INFO:teuthology.orchestra.run.smithi112.stdout: "segexd": 1, 2024-04-21T08:52:03.083 INFO:teuthology.orchestra.run.smithi112.stdout: "expos": 26659424, 2024-04-21T08:52:03.083 INFO:teuthology.orchestra.run.smithi112.stdout: "wrpos": 27070100, 2024-04-21T08:52:03.083 INFO:teuthology.orchestra.run.smithi112.stdout: "rdpos": 0, 2024-04-21T08:52:03.083 INFO:teuthology.orchestra.run.smithi112.stdout: "jlat": { 2024-04-21T08:52:03.083 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2515, 2024-04-21T08:52:03.083 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 5.414030955, 2024-04-21T08:52:03.084 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.002152696 2024-04-21T08:52:03.084 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.084 INFO:teuthology.orchestra.run.smithi112.stdout: "replayed": 0 2024-04-21T08:52:03.084 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.084 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_mem": { 2024-04-21T08:52:03.084 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 37, 2024-04-21T08:52:03.084 INFO:teuthology.orchestra.run.smithi112.stdout: "ino+": 68, 2024-04-21T08:52:03.084 INFO:teuthology.orchestra.run.smithi112.stdout: "ino-": 31, 2024-04-21T08:52:03.084 INFO:teuthology.orchestra.run.smithi112.stdout: "dir": 20, 2024-04-21T08:52:03.084 INFO:teuthology.orchestra.run.smithi112.stdout: "dir+": 20, 2024-04-21T08:52:03.084 INFO:teuthology.orchestra.run.smithi112.stdout: "dir-": 0, 2024-04-21T08:52:03.084 INFO:teuthology.orchestra.run.smithi112.stdout: "dn": 44, 2024-04-21T08:52:03.084 INFO:teuthology.orchestra.run.smithi112.stdout: "dn+": 45, 2024-04-21T08:52:03.084 INFO:teuthology.orchestra.run.smithi112.stdout: "dn-": 1, 2024-04-21T08:52:03.084 INFO:teuthology.orchestra.run.smithi112.stdout: "cap": 2, 2024-04-21T08:52:03.085 INFO:teuthology.orchestra.run.smithi112.stdout: "cap+": 17, 2024-04-21T08:52:03.085 INFO:teuthology.orchestra.run.smithi112.stdout: "cap-": 15, 2024-04-21T08:52:03.085 INFO:teuthology.orchestra.run.smithi112.stdout: "rss": 60320, 2024-04-21T08:52:03.085 INFO:teuthology.orchestra.run.smithi112.stdout: "heap": 231800 2024-04-21T08:52:03.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.085 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_server": { 2024-04-21T08:52:03.085 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_client_request": 2072, 2024-04-21T08:52:03.085 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_server_request": 58, 2024-04-21T08:52:03.085 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_request": 811, 2024-04-21T08:52:03.085 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_session": 39, 2024-04-21T08:52:03.085 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_peer_request": 116, 2024-04-21T08:52:03.085 INFO:teuthology.orchestra.run.smithi112.stdout: "req_create_latency": { 2024-04-21T08:52:03.085 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 8, 2024-04-21T08:52:03.085 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.005795989, 2024-04-21T08:52:03.085 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000724498 2024-04-21T08:52:03.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.086 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getattr_latency": { 2024-04-21T08:52:03.086 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 142, 2024-04-21T08:52:03.086 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.029621297, 2024-04-21T08:52:03.086 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000208600 2024-04-21T08:52:03.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.086 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getfilelock_latency": { 2024-04-21T08:52:03.086 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:03.086 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:03.086 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:03.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.086 INFO:teuthology.orchestra.run.smithi112.stdout: "req_link_latency": { 2024-04-21T08:52:03.086 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:03.087 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:03.087 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:03.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.087 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookup_latency": { 2024-04-21T08:52:03.087 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 3, 2024-04-21T08:52:03.087 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.001628991, 2024-04-21T08:52:03.087 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000542997 2024-04-21T08:52:03.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.087 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookuphash_latency": { 2024-04-21T08:52:03.087 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:03.087 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:03.087 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:03.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.087 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupino_latency": { 2024-04-21T08:52:03.087 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:03.088 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:03.088 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:03.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.088 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupname_latency": { 2024-04-21T08:52:03.088 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:03.088 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:03.088 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:03.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.088 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupparent_latency": { 2024-04-21T08:52:03.088 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:03.088 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:03.088 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:03.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.088 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupsnap_latency": { 2024-04-21T08:52:03.088 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:03.088 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:03.089 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:03.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.089 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lssnap_latency": { 2024-04-21T08:52:03.089 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 13, 2024-04-21T08:52:03.089 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.002156051, 2024-04-21T08:52:03.089 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000165850 2024-04-21T08:52:03.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.089 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mkdir_latency": { 2024-04-21T08:52:03.090 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 8, 2024-04-21T08:52:03.090 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.015147192, 2024-04-21T08:52:03.090 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001893399 2024-04-21T08:52:03.090 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.090 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mknod_latency": { 2024-04-21T08:52:03.090 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:03.090 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:03.090 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:03.090 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.090 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mksnap_latency": { 2024-04-21T08:52:03.090 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 13, 2024-04-21T08:52:03.090 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.064093706, 2024-04-21T08:52:03.090 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.004930285 2024-04-21T08:52:03.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.091 INFO:teuthology.orchestra.run.smithi112.stdout: "req_open_latency": { 2024-04-21T08:52:03.091 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1, 2024-04-21T08:52:03.091 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000273769, 2024-04-21T08:52:03.091 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000273769 2024-04-21T08:52:03.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.091 INFO:teuthology.orchestra.run.smithi112.stdout: "req_readdir_latency": { 2024-04-21T08:52:03.091 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2, 2024-04-21T08:52:03.091 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.004793925, 2024-04-21T08:52:03.091 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.002396962 2024-04-21T08:52:03.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.091 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rename_latency": { 2024-04-21T08:52:03.092 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2, 2024-04-21T08:52:03.092 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.002879196, 2024-04-21T08:52:03.092 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001439598 2024-04-21T08:52:03.092 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.092 INFO:teuthology.orchestra.run.smithi112.stdout: "req_renamesnap_latency": { 2024-04-21T08:52:03.092 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:03.092 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:03.092 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:03.092 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.092 INFO:teuthology.orchestra.run.smithi112.stdout: "req_snapdiff_latency": { 2024-04-21T08:52:03.092 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:03.092 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:03.092 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:03.092 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.093 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmdir_latency": { 2024-04-21T08:52:03.093 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2, 2024-04-21T08:52:03.093 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.014504324, 2024-04-21T08:52:03.093 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.007252162 2024-04-21T08:52:03.093 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.093 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmsnap_latency": { 2024-04-21T08:52:03.093 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 7, 2024-04-21T08:52:03.093 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.031985988, 2024-04-21T08:52:03.093 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.004569426 2024-04-21T08:52:03.094 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.094 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmxattr_latency": { 2024-04-21T08:52:03.094 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:03.094 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:03.094 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:03.094 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.094 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setattr_latency": { 2024-04-21T08:52:03.094 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 601, 2024-04-21T08:52:03.095 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 3.392677409, 2024-04-21T08:52:03.095 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.005645053 2024-04-21T08:52:03.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.095 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setdirlayout_latency": { 2024-04-21T08:52:03.095 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:03.095 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:03.095 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:03.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.096 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setfilelock_latency": { 2024-04-21T08:52:03.096 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:03.096 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:03.096 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:03.096 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.096 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setlayout_latency": { 2024-04-21T08:52:03.096 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:03.096 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:03.096 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:03.097 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.097 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setxattr_latency": { 2024-04-21T08:52:03.097 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 8, 2024-04-21T08:52:03.097 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.021409219, 2024-04-21T08:52:03.097 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.002676152 2024-04-21T08:52:03.097 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.098 INFO:teuthology.orchestra.run.smithi112.stdout: "req_symlink_latency": { 2024-04-21T08:52:03.098 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:03.098 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:03.098 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:03.098 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.098 INFO:teuthology.orchestra.run.smithi112.stdout: "req_unlink_latency": { 2024-04-21T08:52:03.099 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1, 2024-04-21T08:52:03.099 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.003087360, 2024-04-21T08:52:03.099 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.003087360 2024-04-21T08:52:03.099 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.099 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:52:03.099 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:52:03.099 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getvxattr_latency": { 2024-04-21T08:52:03.099 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:03.100 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:03.100 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:03.100 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:03.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.100 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_sessions": { 2024-04-21T08:52:03.100 INFO:teuthology.orchestra.run.smithi112.stdout: "session_count": 2, 2024-04-21T08:52:03.100 INFO:teuthology.orchestra.run.smithi112.stdout: "session_add": 6, 2024-04-21T08:52:03.101 INFO:teuthology.orchestra.run.smithi112.stdout: "session_remove": 4, 2024-04-21T08:52:03.101 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_open": 2, 2024-04-21T08:52:03.101 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_stale": 0, 2024-04-21T08:52:03.101 INFO:teuthology.orchestra.run.smithi112.stdout: "total_load": 89, 2024-04-21T08:52:03.101 INFO:teuthology.orchestra.run.smithi112.stdout: "average_load": 44, 2024-04-21T08:52:03.101 INFO:teuthology.orchestra.run.smithi112.stdout: "avg_session_uptime": 752, 2024-04-21T08:52:03.101 INFO:teuthology.orchestra.run.smithi112.stdout: "mdthresh_evicted": 0 2024-04-21T08:52:03.101 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.102 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-04-21T08:52:03.102 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:52:03.102 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-04-21T08:52:03.102 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:52:03.102 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:52:03.102 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:52:03.102 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:52:03.103 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:52:03.103 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:52:03.103 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:52:03.103 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:52:03.103 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:52:03.103 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:52:03.104 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:52:03.104 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:52:03.104 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:52:03.104 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-04-21T08:52:03.104 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:52:03.104 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-04-21T08:52:03.105 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:52:03.105 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:52:03.105 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:52:03.105 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:52:03.105 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:52:03.105 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:52:03.105 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:52:03.105 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-04-21T08:52:03.105 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:52:03.105 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:52:03.105 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:52:03.106 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-04-21T08:52:03.106 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-04-21T08:52:03.106 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-04-21T08:52:03.106 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:52:03.106 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:52:03.106 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:52:03.106 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:52:03.106 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 335174, 2024-04-21T08:52:03.106 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 120, 2024-04-21T08:52:03.106 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:52:03.106 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-04-21T08:52:03.106 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-04-21T08:52:03.106 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-04-21T08:52:03.106 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:52:03.107 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-04-21T08:52:03.107 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:52:03.107 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-04-21T08:52:03.107 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 9800, 2024-04-21T08:52:03.107 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 188, 2024-04-21T08:52:03.107 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:52:03.107 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:52:03.107 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-04-21T08:52:03.107 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-04-21T08:52:03.107 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 181227, 2024-04-21T08:52:03.107 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 1427, 2024-04-21T08:52:03.107 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-04-21T08:52:03.107 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-04-21T08:52:03.107 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-04-21T08:52:03.107 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-04-21T08:52:03.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.108 INFO:teuthology.orchestra.run.smithi112.stdout: "objecter": { 2024-04-21T08:52:03.108 INFO:teuthology.orchestra.run.smithi112.stdout: "op_active": 0, 2024-04-21T08:52:03.108 INFO:teuthology.orchestra.run.smithi112.stdout: "op_laggy": 0, 2024-04-21T08:52:03.108 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send": 4853, 2024-04-21T08:52:03.108 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send_bytes": 24109985, 2024-04-21T08:52:03.108 INFO:teuthology.orchestra.run.smithi112.stdout: "op_resend": 32, 2024-04-21T08:52:03.108 INFO:teuthology.orchestra.run.smithi112.stdout: "op_reply": 4853, 2024-04-21T08:52:03.108 INFO:teuthology.orchestra.run.smithi112.stdout: "op_latency": { 2024-04-21T08:52:03.108 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 4853, 2024-04-21T08:52:03.108 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 27.175282586, 2024-04-21T08:52:03.108 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.005599687 2024-04-21T08:52:03.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.108 INFO:teuthology.orchestra.run.smithi112.stdout: "op_inflight": 0, 2024-04-21T08:52:03.109 INFO:teuthology.orchestra.run.smithi112.stdout: "oplen_avg": { 2024-04-21T08:52:03.109 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 4853, 2024-04-21T08:52:03.109 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 5652 2024-04-21T08:52:03.109 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.109 INFO:teuthology.orchestra.run.smithi112.stdout: "op": 4853, 2024-04-21T08:52:03.109 INFO:teuthology.orchestra.run.smithi112.stdout: "op_r": 476, 2024-04-21T08:52:03.109 INFO:teuthology.orchestra.run.smithi112.stdout: "op_w": 4377, 2024-04-21T08:52:03.109 INFO:teuthology.orchestra.run.smithi112.stdout: "op_rmw": 0, 2024-04-21T08:52:03.109 INFO:teuthology.orchestra.run.smithi112.stdout: "op_pg": 0, 2024-04-21T08:52:03.109 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_stat": 358, 2024-04-21T08:52:03.109 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_create": 30, 2024-04-21T08:52:03.109 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_read": 31, 2024-04-21T08:52:03.109 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_write": 2516, 2024-04-21T08:52:03.109 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writefull": 200, 2024-04-21T08:52:03.109 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writesame": 0, 2024-04-21T08:52:03.110 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_append": 0, 2024-04-21T08:52:03.110 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_zero": 0, 2024-04-21T08:52:03.110 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_truncate": 0, 2024-04-21T08:52:03.110 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_delete": 524, 2024-04-21T08:52:03.110 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_mapext": 0, 2024-04-21T08:52:03.110 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_sparse_read": 0, 2024-04-21T08:52:03.110 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_clonerange": 0, 2024-04-21T08:52:03.110 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_getxattr": 445, 2024-04-21T08:52:03.110 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_setxattr": 55, 2024-04-21T08:52:03.110 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:52:03.110 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_rmxattr": 0, 2024-04-21T08:52:03.110 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:52:03.110 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_call": 0, 2024-04-21T08:52:03.110 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_watch": 0, 2024-04-21T08:52:03.111 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_notify": 0, 2024-04-21T08:52:03.111 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:52:03.111 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls": 0, 2024-04-21T08:52:03.111 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:52:03.111 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_other": 599, 2024-04-21T08:52:03.111 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_active": 0, 2024-04-21T08:52:03.111 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_send": 0, 2024-04-21T08:52:03.111 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_resend": 0, 2024-04-21T08:52:03.111 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_ping": 0, 2024-04-21T08:52:03.111 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_active": 0, 2024-04-21T08:52:03.111 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_send": 0, 2024-04-21T08:52:03.111 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_resend": 0, 2024-04-21T08:52:03.111 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_active": 0, 2024-04-21T08:52:03.111 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_send": 0, 2024-04-21T08:52:03.111 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_resend": 0, 2024-04-21T08:52:03.112 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_active": 0, 2024-04-21T08:52:03.112 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_send": 0, 2024-04-21T08:52:03.112 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_resend": 0, 2024-04-21T08:52:03.112 INFO:teuthology.orchestra.run.smithi112.stdout: "command_active": 0, 2024-04-21T08:52:03.112 INFO:teuthology.orchestra.run.smithi112.stdout: "command_send": 0, 2024-04-21T08:52:03.112 INFO:teuthology.orchestra.run.smithi112.stdout: "command_resend": 0, 2024-04-21T08:52:03.112 INFO:teuthology.orchestra.run.smithi112.stdout: "map_epoch": 108, 2024-04-21T08:52:03.112 INFO:teuthology.orchestra.run.smithi112.stdout: "map_full": 0, 2024-04-21T08:52:03.112 INFO:teuthology.orchestra.run.smithi112.stdout: "map_inc": 23, 2024-04-21T08:52:03.112 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_sessions": 12, 2024-04-21T08:52:03.112 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_open": 12, 2024-04-21T08:52:03.112 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_close": 0, 2024-04-21T08:52:03.112 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_laggy": 0, 2024-04-21T08:52:03.112 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_wr": 882, 2024-04-21T08:52:03.113 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_rd": 0, 2024-04-21T08:52:03.113 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_del": 12 2024-04-21T08:52:03.113 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.113 INFO:teuthology.orchestra.run.smithi112.stdout: "oft": { 2024-04-21T08:52:03.113 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_objs": 1, 2024-04-21T08:52:03.113 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:52:03.113 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_updates": 0, 2024-04-21T08:52:03.113 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_removes": 0 2024-04-21T08:52:03.113 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.113 INFO:teuthology.orchestra.run.smithi112.stdout: "osd-slow-ops": { 2024-04-21T08:52:03.113 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_ops_count": 102 2024-04-21T08:52:03.113 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.113 INFO:teuthology.orchestra.run.smithi112.stdout: "purge_queue": { 2024-04-21T08:52:03.114 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops": 0, 2024-04-21T08:52:03.114 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops_high_water": 2, 2024-04-21T08:52:03.114 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing": 0, 2024-04-21T08:52:03.114 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_high_water": 1, 2024-04-21T08:52:03.114 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed_ops": 2, 2024-04-21T08:52:03.114 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed": 1, 2024-04-21T08:52:03.114 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_item_in_journal": 0 2024-04-21T08:52:03.114 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.114 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:52:03.114 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-21T08:52:03.114 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-21T08:52:03.114 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-21T08:52:03.114 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 13221, 2024-04-21T08:52:03.114 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 4737548, 2024-04-21T08:52:03.115 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:03.115 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 13221, 2024-04-21T08:52:03.115 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-21T08:52:03.115 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-21T08:52:03.115 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 13221, 2024-04-21T08:52:03.115 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 4737548, 2024-04-21T08:52:03.115 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-21T08:52:03.115 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:03.115 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:03.115 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:03.115 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:03.115 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.115 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_bytes": { 2024-04-21T08:52:03.115 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-21T08:52:03.115 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-21T08:52:03.116 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-21T08:52:03.116 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-21T08:52:03.116 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-21T08:52:03.116 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:03.116 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:03.116 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 4853, 2024-04-21T08:52:03.116 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 24110142, 2024-04-21T08:52:03.116 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 4853, 2024-04-21T08:52:03.116 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 24110142, 2024-04-21T08:52:03.116 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-21T08:52:03.116 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:03.116 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:03.116 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:03.116 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:03.116 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.117 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_ops": { 2024-04-21T08:52:03.117 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-21T08:52:03.117 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 1024, 2024-04-21T08:52:03.117 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-21T08:52:03.117 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-21T08:52:03.117 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-21T08:52:03.117 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:03.117 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:03.117 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 4853, 2024-04-21T08:52:03.117 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 4853, 2024-04-21T08:52:03.117 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 4853, 2024-04-21T08:52:03.117 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 4853, 2024-04-21T08:52:03.118 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-21T08:52:03.118 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:03.118 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:03.118 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:03.118 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:03.118 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.118 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:52:03.118 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-21T08:52:03.118 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-04-21T08:52:03.118 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-21T08:52:03.118 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 1, 2024-04-21T08:52:03.118 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 157, 2024-04-21T08:52:03.118 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:03.119 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 1, 2024-04-21T08:52:03.119 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-21T08:52:03.119 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-21T08:52:03.119 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 1, 2024-04-21T08:52:03.119 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 157, 2024-04-21T08:52:03.119 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-21T08:52:03.119 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:03.119 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:03.119 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:03.119 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:03.119 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:03.119 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle-0x56133b4fe500": { 2024-04-21T08:52:03.119 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 84, 2024-04-21T08:52:03.119 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-04-21T08:52:03.119 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-21T08:52:03.120 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 2590, 2024-04-21T08:52:03.120 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 22875796, 2024-04-21T08:52:03.120 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:03.120 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 2590, 2024-04-21T08:52:03.120 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-21T08:52:03.120 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-21T08:52:03.120 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 2515, 2024-04-21T08:52:03.120 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 22875712, 2024-04-21T08:52:03.120 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-21T08:52:03.120 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:03.120 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:03.120 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:03.121 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:03.121 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:03.121 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-21T08:52:03.945 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-21T08:52:03.945 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-21T08:52:03.945 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=20 2024-04-21T08:52:03.945 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-04-21T08:52:03.945 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-21T08:52:03.946 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=13 2024-04-21T08:52:03.946 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=7 2024-04-21T08:52:03.946 DEBUG:tasks.check_counter:Getting stats from e 2024-04-21T08:52:03.946 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-21T08:52:04.222 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:52:04.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:04 smithi175 ceph-mon[31920]: pgmap v410: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-21T08:52:05.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:04 smithi112 ceph-mon[31085]: pgmap v410: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-21T08:52:05.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:04 smithi043 ceph-mon[27978]: pgmap v410: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-21T08:52:05.973 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-21T08:52:05.974 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:52:05.974 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 2669, 2024-04-21T08:52:05.974 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 2671, 2024-04-21T08:52:05.974 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 756147, 2024-04-21T08:52:05.974 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 848749, 2024-04-21T08:52:05.974 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 6, 2024-04-21T08:52:05.974 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 2, 2024-04-21T08:52:05.974 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.204547847, 2024-04-21T08:52:05.974 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.068360223, 2024-04-21T08:52:05.974 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.048240457, 2024-04-21T08:52:05.974 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.070353120, 2024-04-21T08:52:05.974 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:05.974 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2669, 2024-04-21T08:52:05.975 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.086866205, 2024-04-21T08:52:05.975 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000032546 2024-04-21T08:52:05.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.975 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:05.975 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.975 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.975 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.975 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T08:52:05.975 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T08:52:05.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.975 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:52:05.975 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 8, 2024-04-21T08:52:05.976 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 202, 2024-04-21T08:52:05.976 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 3868, 2024-04-21T08:52:05.976 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 504746, 2024-04-21T08:52:05.976 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 7, 2024-04-21T08:52:05.976 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 4, 2024-04-21T08:52:05.976 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.012229366, 2024-04-21T08:52:05.976 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.007931324, 2024-04-21T08:52:05.976 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.002225215, 2024-04-21T08:52:05.976 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000044651, 2024-04-21T08:52:05.976 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:05.976 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 200, 2024-04-21T08:52:05.976 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.010596907, 2024-04-21T08:52:05.976 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000052984 2024-04-21T08:52:05.976 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.977 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:05.977 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.977 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.977 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.977 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.977 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-21T08:52:05.977 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 504048 2024-04-21T08:52:05.977 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.977 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:52:05.977 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 301, 2024-04-21T08:52:05.977 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 282, 2024-04-21T08:52:05.977 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 1678940, 2024-04-21T08:52:05.977 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 163438, 2024-04-21T08:52:05.977 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-04-21T08:52:05.978 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 2, 2024-04-21T08:52:05.978 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.058978594, 2024-04-21T08:52:05.978 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.018946987, 2024-04-21T08:52:05.978 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.041581576, 2024-04-21T08:52:05.978 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.007268505, 2024-04-21T08:52:05.978 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:05.978 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 281, 2024-04-21T08:52:05.978 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.016081866, 2024-04-21T08:52:05.978 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000057230 2024-04-21T08:52:05.978 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.978 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:05.978 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.978 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.979 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.979 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 1678672, 2024-04-21T08:52:05.979 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 163120 2024-04-21T08:52:05.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.979 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-04-21T08:52:05.979 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 1, 2024-04-21T08:52:05.979 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-04-21T08:52:05.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.979 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-MDSRank": { 2024-04-21T08:52:05.979 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-04-21T08:52:05.979 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-04-21T08:52:05.979 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 4502, 2024-04-21T08:52:05.979 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.112089843, 2024-04-21T08:52:05.979 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000024897 2024-04-21T08:52:05.980 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:05.980 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.980 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-PurgeQueue": { 2024-04-21T08:52:05.980 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-04-21T08:52:05.980 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-04-21T08:52:05.980 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.980 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.980 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.980 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:05.980 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.980 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-04-21T08:52:05.980 INFO:teuthology.orchestra.run.smithi112.stdout: "request": 0, 2024-04-21T08:52:05.980 INFO:teuthology.orchestra.run.smithi112.stdout: "reply": 0, 2024-04-21T08:52:05.980 INFO:teuthology.orchestra.run.smithi112.stdout: "reply_latency": { 2024-04-21T08:52:05.981 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.981 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.981 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.981 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.981 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_reply": 0, 2024-04-21T08:52:05.981 INFO:teuthology.orchestra.run.smithi112.stdout: "forward": 0, 2024-04-21T08:52:05.981 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_complete": 0, 2024-04-21T08:52:05.981 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_keys": 0, 2024-04-21T08:52:05.981 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_commit": 0, 2024-04-21T08:52:05.981 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_split": 0, 2024-04-21T08:52:05.981 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_merge": 0, 2024-04-21T08:52:05.981 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes": 0, 2024-04-21T08:52:05.981 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_top": 0, 2024-04-21T08:52:05.981 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_bottom": 0, 2024-04-21T08:52:05.981 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pin_tail": 0, 2024-04-21T08:52:05.982 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pinned": 0, 2024-04-21T08:52:05.982 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_expired": 0, 2024-04-21T08:52:05.982 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_with_caps": 0, 2024-04-21T08:52:05.982 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": 0, 2024-04-21T08:52:05.982 INFO:teuthology.orchestra.run.smithi112.stdout: "subtrees": 1, 2024-04-21T08:52:05.982 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse": 0, 2024-04-21T08:52:05.982 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_hit": 0, 2024-04-21T08:52:05.982 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_forward": 0, 2024-04-21T08:52:05.982 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_discover": 0, 2024-04-21T08:52:05.982 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:52:05.982 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_remote_ino": 0, 2024-04-21T08:52:05.982 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_lock": 0, 2024-04-21T08:52:05.982 INFO:teuthology.orchestra.run.smithi112.stdout: "load_cent": 0, 2024-04-21T08:52:05.982 INFO:teuthology.orchestra.run.smithi112.stdout: "q": 0, 2024-04-21T08:52:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: "exported": 0, 2024-04-21T08:52:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: "exported_inodes": 0, 2024-04-21T08:52:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: "imported": 0, 2024-04-21T08:52:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: "imported_inodes": 0, 2024-04-21T08:52:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_dir_fetch": 0, 2024-04-21T08:52:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_backtrace_fetch": 0, 2024-04-21T08:52:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_peer_discover": 0, 2024-04-21T08:52:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rfiles": 0, 2024-04-21T08:52:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rbytes": 0, 2024-04-21T08:52:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rsnaps": 0, 2024-04-21T08:52:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T08:52:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_set_tag": 0, 2024-04-21T08:52:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:52:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:52:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:52:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:52:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:52:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_file_inodes": 0, 2024-04-21T08:52:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:52:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T08:52:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_grant": 0, 2024-04-21T08:52:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:52:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:52:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:52:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps": 0, 2024-04-21T08:52:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps_dirty": 0, 2024-04-21T08:52:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_cap_release": 0, 2024-04-21T08:52:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: "process_request_cap_release": 0 2024-04-21T08:52:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_cache": { 2024-04-21T08:52:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update": 0, 2024-04-21T08:52:05.985 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update_receipt": 0, 2024-04-21T08:52:05.985 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_try_discover": 0, 2024-04-21T08:52:05.985 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_send_discover": 0, 2024-04-21T08:52:05.985 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_handle_discover": 0, 2024-04-21T08:52:05.985 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays": 0, 2024-04-21T08:52:05.985 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_delayed": 0, 2024-04-21T08:52:05.985 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:52:05.985 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_created": 0, 2024-04-21T08:52:05.985 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_enqueued": 0, 2024-04-21T08:52:05.985 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_reintegrated": 0, 2024-04-21T08:52:05.985 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_migrated": 0, 2024-04-21T08:52:05.985 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_processing": 0, 2024-04-21T08:52:05.985 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:52:05.985 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:52:05.985 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_started": 0, 2024-04-21T08:52:05.985 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_completed": 0, 2024-04-21T08:52:05.986 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_path": 0, 2024-04-21T08:52:05.986 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_inode": 0, 2024-04-21T08:52:05.986 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:52:05.986 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_exportdir": 0, 2024-04-21T08:52:05.986 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_flush": 0, 2024-04-21T08:52:05.986 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:52:05.986 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragstats": 0, 2024-04-21T08:52:05.986 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_inodestats": 0 2024-04-21T08:52:05.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.986 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_log": { 2024-04-21T08:52:05.986 INFO:teuthology.orchestra.run.smithi112.stdout: "evlrg": 0, 2024-04-21T08:52:05.986 INFO:teuthology.orchestra.run.smithi112.stdout: "evadd": 0, 2024-04-21T08:52:05.986 INFO:teuthology.orchestra.run.smithi112.stdout: "evex": 0, 2024-04-21T08:52:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: "evtrm": 0, 2024-04-21T08:52:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: "ev": 7, 2024-04-21T08:52:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: "evexg": 0, 2024-04-21T08:52:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: "evexd": 0, 2024-04-21T08:52:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: "segadd": 0, 2024-04-21T08:52:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: "segex": 0, 2024-04-21T08:52:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: "segtrm": 0, 2024-04-21T08:52:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: "seg": 2, 2024-04-21T08:52:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: "segmjr": 2, 2024-04-21T08:52:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: "segexg": 0, 2024-04-21T08:52:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: "segexd": 0, 2024-04-21T08:52:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: "expos": 4194304, 2024-04-21T08:52:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: "wrpos": 4196264, 2024-04-21T08:52:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: "rdpos": 4196123, 2024-04-21T08:52:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: "jlat": { 2024-04-21T08:52:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.988 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.988 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.988 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.988 INFO:teuthology.orchestra.run.smithi112.stdout: "replayed": 7 2024-04-21T08:52:05.988 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.988 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_mem": { 2024-04-21T08:52:05.988 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 3, 2024-04-21T08:52:05.988 INFO:teuthology.orchestra.run.smithi112.stdout: "ino+": 3, 2024-04-21T08:52:05.988 INFO:teuthology.orchestra.run.smithi112.stdout: "ino-": 0, 2024-04-21T08:52:05.988 INFO:teuthology.orchestra.run.smithi112.stdout: "dir": 1, 2024-04-21T08:52:05.988 INFO:teuthology.orchestra.run.smithi112.stdout: "dir+": 1, 2024-04-21T08:52:05.988 INFO:teuthology.orchestra.run.smithi112.stdout: "dir-": 0, 2024-04-21T08:52:05.988 INFO:teuthology.orchestra.run.smithi112.stdout: "dn": 0, 2024-04-21T08:52:05.988 INFO:teuthology.orchestra.run.smithi112.stdout: "dn+": 0, 2024-04-21T08:52:05.988 INFO:teuthology.orchestra.run.smithi112.stdout: "dn-": 0, 2024-04-21T08:52:05.989 INFO:teuthology.orchestra.run.smithi112.stdout: "cap": 0, 2024-04-21T08:52:05.989 INFO:teuthology.orchestra.run.smithi112.stdout: "cap+": 0, 2024-04-21T08:52:05.989 INFO:teuthology.orchestra.run.smithi112.stdout: "cap-": 0, 2024-04-21T08:52:05.989 INFO:teuthology.orchestra.run.smithi112.stdout: "rss": 53300, 2024-04-21T08:52:05.989 INFO:teuthology.orchestra.run.smithi112.stdout: "heap": 215416 2024-04-21T08:52:05.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.989 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_server": { 2024-04-21T08:52:05.989 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_client_request": 0, 2024-04-21T08:52:05.989 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_server_request": 0, 2024-04-21T08:52:05.989 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_request": 0, 2024-04-21T08:52:05.989 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_session": 0, 2024-04-21T08:52:05.989 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_peer_request": 0, 2024-04-21T08:52:05.989 INFO:teuthology.orchestra.run.smithi112.stdout: "req_create_latency": { 2024-04-21T08:52:05.989 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.989 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.990 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.990 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.990 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getattr_latency": { 2024-04-21T08:52:05.990 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.990 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.990 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.990 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.990 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getfilelock_latency": { 2024-04-21T08:52:05.990 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.990 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.990 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.990 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.990 INFO:teuthology.orchestra.run.smithi112.stdout: "req_link_latency": { 2024-04-21T08:52:05.990 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.990 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookup_latency": { 2024-04-21T08:52:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookuphash_latency": { 2024-04-21T08:52:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupino_latency": { 2024-04-21T08:52:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupname_latency": { 2024-04-21T08:52:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupparent_latency": { 2024-04-21T08:52:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupsnap_latency": { 2024-04-21T08:52:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.993 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lssnap_latency": { 2024-04-21T08:52:05.993 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.993 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.993 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.993 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.993 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mkdir_latency": { 2024-04-21T08:52:05.993 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.993 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.994 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.994 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.994 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mknod_latency": { 2024-04-21T08:52:05.994 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.994 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.994 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.994 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.994 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mksnap_latency": { 2024-04-21T08:52:05.994 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.994 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.994 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.994 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.995 INFO:teuthology.orchestra.run.smithi112.stdout: "req_open_latency": { 2024-04-21T08:52:05.995 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.995 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.995 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.995 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.995 INFO:teuthology.orchestra.run.smithi112.stdout: "req_readdir_latency": { 2024-04-21T08:52:05.995 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.995 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.995 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.995 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.995 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rename_latency": { 2024-04-21T08:52:05.995 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.995 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.995 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.995 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.996 INFO:teuthology.orchestra.run.smithi112.stdout: "req_renamesnap_latency": { 2024-04-21T08:52:05.996 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.996 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.996 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.996 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.996 INFO:teuthology.orchestra.run.smithi112.stdout: "req_snapdiff_latency": { 2024-04-21T08:52:05.996 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.996 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.996 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.996 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.996 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmdir_latency": { 2024-04-21T08:52:05.996 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.996 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.996 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.996 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.997 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmsnap_latency": { 2024-04-21T08:52:05.997 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.997 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.997 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.997 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmxattr_latency": { 2024-04-21T08:52:05.997 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.997 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.997 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.997 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setattr_latency": { 2024-04-21T08:52:05.997 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.997 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.997 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.998 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setdirlayout_latency": { 2024-04-21T08:52:05.998 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.998 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.998 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.998 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setfilelock_latency": { 2024-04-21T08:52:05.998 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.998 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.998 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.998 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setlayout_latency": { 2024-04-21T08:52:05.998 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.998 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.999 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.999 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setxattr_latency": { 2024-04-21T08:52:05.999 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.999 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.999 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.999 INFO:teuthology.orchestra.run.smithi112.stdout: "req_symlink_latency": { 2024-04-21T08:52:05.999 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.999 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:05.999 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:05.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:05.999 INFO:teuthology.orchestra.run.smithi112.stdout: "req_unlink_latency": { 2024-04-21T08:52:05.999 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:05.999 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:06.000 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:06.000 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:06.000 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:52:06.000 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:52:06.000 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getvxattr_latency": { 2024-04-21T08:52:06.000 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:06.000 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:06.000 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:06.000 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:06.000 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:06.000 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_sessions": { 2024-04-21T08:52:06.000 INFO:teuthology.orchestra.run.smithi112.stdout: "session_count": 1, 2024-04-21T08:52:06.000 INFO:teuthology.orchestra.run.smithi112.stdout: "session_add": 3, 2024-04-21T08:52:06.000 INFO:teuthology.orchestra.run.smithi112.stdout: "session_remove": 2, 2024-04-21T08:52:06.001 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_open": 2, 2024-04-21T08:52:06.001 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_stale": 0, 2024-04-21T08:52:06.001 INFO:teuthology.orchestra.run.smithi112.stdout: "total_load": 0, 2024-04-21T08:52:06.001 INFO:teuthology.orchestra.run.smithi112.stdout: "average_load": 0, 2024-04-21T08:52:06.001 INFO:teuthology.orchestra.run.smithi112.stdout: "avg_session_uptime": 3152, 2024-04-21T08:52:06.001 INFO:teuthology.orchestra.run.smithi112.stdout: "mdthresh_evicted": 0 2024-04-21T08:52:06.001 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:06.001 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-04-21T08:52:06.001 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:52:06.001 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-04-21T08:52:06.001 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:52:06.001 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:52:06.001 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:52:06.001 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:52:06.001 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:52:06.002 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:52:06.002 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:52:06.002 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:52:06.002 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:52:06.002 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:52:06.002 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:52:06.002 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:52:06.002 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:52:06.002 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-04-21T08:52:06.002 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:52:06.002 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-04-21T08:52:06.002 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:52:06.002 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:52:06.002 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:52:06.003 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:52:06.003 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:52:06.003 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:52:06.003 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:52:06.003 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-04-21T08:52:06.003 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:52:06.003 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:52:06.003 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:52:06.003 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-04-21T08:52:06.003 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-04-21T08:52:06.003 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-04-21T08:52:06.003 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:52:06.004 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:52:06.004 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:52:06.004 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:52:06.004 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 181393, 2024-04-21T08:52:06.004 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 49, 2024-04-21T08:52:06.004 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:52:06.004 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-04-21T08:52:06.004 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-04-21T08:52:06.004 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-04-21T08:52:06.004 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:52:06.004 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-04-21T08:52:06.004 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:52:06.004 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-04-21T08:52:06.004 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 9800, 2024-04-21T08:52:06.004 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 188, 2024-04-21T08:52:06.005 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:52:06.005 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:52:06.005 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-04-21T08:52:06.005 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-04-21T08:52:06.005 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 8512, 2024-04-21T08:52:06.005 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 13, 2024-04-21T08:52:06.005 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-04-21T08:52:06.005 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-04-21T08:52:06.005 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-04-21T08:52:06.005 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-04-21T08:52:06.005 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:06.005 INFO:teuthology.orchestra.run.smithi112.stdout: "objecter": { 2024-04-21T08:52:06.005 INFO:teuthology.orchestra.run.smithi112.stdout: "op_active": 0, 2024-04-21T08:52:06.005 INFO:teuthology.orchestra.run.smithi112.stdout: "op_laggy": 0, 2024-04-21T08:52:06.006 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send": 2672, 2024-04-21T08:52:06.006 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send_bytes": 21, 2024-04-21T08:52:06.006 INFO:teuthology.orchestra.run.smithi112.stdout: "op_resend": 3, 2024-04-21T08:52:06.006 INFO:teuthology.orchestra.run.smithi112.stdout: "op_reply": 2672, 2024-04-21T08:52:06.006 INFO:teuthology.orchestra.run.smithi112.stdout: "op_latency": { 2024-04-21T08:52:06.006 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2672, 2024-04-21T08:52:06.006 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 1.351964229, 2024-04-21T08:52:06.006 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000505974 2024-04-21T08:52:06.006 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:06.006 INFO:teuthology.orchestra.run.smithi112.stdout: "op_inflight": 0, 2024-04-21T08:52:06.006 INFO:teuthology.orchestra.run.smithi112.stdout: "oplen_avg": { 2024-04-21T08:52:06.006 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2672, 2024-04-21T08:52:06.006 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 2673 2024-04-21T08:52:06.006 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:06.006 INFO:teuthology.orchestra.run.smithi112.stdout: "op": 2672, 2024-04-21T08:52:06.007 INFO:teuthology.orchestra.run.smithi112.stdout: "op_r": 2672, 2024-04-21T08:52:06.007 INFO:teuthology.orchestra.run.smithi112.stdout: "op_w": 0, 2024-04-21T08:52:06.007 INFO:teuthology.orchestra.run.smithi112.stdout: "op_rmw": 0, 2024-04-21T08:52:06.007 INFO:teuthology.orchestra.run.smithi112.stdout: "op_pg": 0, 2024-04-21T08:52:06.007 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_stat": 1746, 2024-04-21T08:52:06.007 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_create": 0, 2024-04-21T08:52:06.007 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_read": 924, 2024-04-21T08:52:06.007 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_write": 0, 2024-04-21T08:52:06.007 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writefull": 0, 2024-04-21T08:52:06.007 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writesame": 0, 2024-04-21T08:52:06.007 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_append": 0, 2024-04-21T08:52:06.007 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_zero": 0, 2024-04-21T08:52:06.007 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_truncate": 0, 2024-04-21T08:52:06.007 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_delete": 0, 2024-04-21T08:52:06.008 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_mapext": 0, 2024-04-21T08:52:06.008 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_sparse_read": 0, 2024-04-21T08:52:06.008 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_clonerange": 0, 2024-04-21T08:52:06.008 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_getxattr": 1, 2024-04-21T08:52:06.008 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_setxattr": 0, 2024-04-21T08:52:06.008 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:52:06.008 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_rmxattr": 0, 2024-04-21T08:52:06.008 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:52:06.008 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_call": 0, 2024-04-21T08:52:06.008 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_watch": 0, 2024-04-21T08:52:06.008 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_notify": 0, 2024-04-21T08:52:06.008 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:52:06.008 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls": 0, 2024-04-21T08:52:06.008 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:52:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_other": 0, 2024-04-21T08:52:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_active": 0, 2024-04-21T08:52:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_send": 0, 2024-04-21T08:52:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_resend": 0, 2024-04-21T08:52:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_ping": 0, 2024-04-21T08:52:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_active": 0, 2024-04-21T08:52:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_send": 0, 2024-04-21T08:52:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_resend": 0, 2024-04-21T08:52:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_active": 0, 2024-04-21T08:52:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_send": 0, 2024-04-21T08:52:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_resend": 0, 2024-04-21T08:52:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_active": 0, 2024-04-21T08:52:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_send": 0, 2024-04-21T08:52:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_resend": 0, 2024-04-21T08:52:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "command_active": 0, 2024-04-21T08:52:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "command_send": 0, 2024-04-21T08:52:06.010 INFO:teuthology.orchestra.run.smithi112.stdout: "command_resend": 0, 2024-04-21T08:52:06.010 INFO:teuthology.orchestra.run.smithi112.stdout: "map_epoch": 108, 2024-04-21T08:52:06.010 INFO:teuthology.orchestra.run.smithi112.stdout: "map_full": 0, 2024-04-21T08:52:06.010 INFO:teuthology.orchestra.run.smithi112.stdout: "map_inc": 18, 2024-04-21T08:52:06.010 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_sessions": 4, 2024-04-21T08:52:06.010 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_open": 6, 2024-04-21T08:52:06.010 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_close": 2, 2024-04-21T08:52:06.010 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_laggy": 0, 2024-04-21T08:52:06.010 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_wr": 0, 2024-04-21T08:52:06.010 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_rd": 2, 2024-04-21T08:52:06.010 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_del": 0 2024-04-21T08:52:06.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:06.010 INFO:teuthology.orchestra.run.smithi112.stdout: "oft": { 2024-04-21T08:52:06.011 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_objs": 0, 2024-04-21T08:52:06.011 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:52:06.011 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_updates": 0, 2024-04-21T08:52:06.011 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_removes": 0 2024-04-21T08:52:06.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:06.011 INFO:teuthology.orchestra.run.smithi112.stdout: "osd-slow-ops": { 2024-04-21T08:52:06.011 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_ops_count": 0 2024-04-21T08:52:06.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:06.011 INFO:teuthology.orchestra.run.smithi112.stdout: "purge_queue": { 2024-04-21T08:52:06.011 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops": 0, 2024-04-21T08:52:06.011 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T08:52:06.011 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing": 0, 2024-04-21T08:52:06.011 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_high_water": 0, 2024-04-21T08:52:06.011 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed_ops": 0, 2024-04-21T08:52:06.011 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed": 0, 2024-04-21T08:52:06.012 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_item_in_journal": 0 2024-04-21T08:52:06.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:06.012 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:52:06.012 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-21T08:52:06.012 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-21T08:52:06.012 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-21T08:52:06.012 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 2974, 2024-04-21T08:52:06.012 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 2152970, 2024-04-21T08:52:06.012 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:06.012 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 2974, 2024-04-21T08:52:06.012 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-21T08:52:06.012 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-21T08:52:06.012 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 2974, 2024-04-21T08:52:06.012 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 2152970, 2024-04-21T08:52:06.012 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-21T08:52:06.013 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:06.013 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:06.013 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:06.013 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:06.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:06.013 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_bytes": { 2024-04-21T08:52:06.013 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-21T08:52:06.013 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-21T08:52:06.013 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-21T08:52:06.013 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-21T08:52:06.013 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-21T08:52:06.013 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:06.013 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:06.013 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 2672, 2024-04-21T08:52:06.013 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 1965, 2024-04-21T08:52:06.014 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 2672, 2024-04-21T08:52:06.014 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1965, 2024-04-21T08:52:06.014 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-21T08:52:06.014 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:06.014 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:06.014 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:06.014 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:06.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:06.014 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_ops": { 2024-04-21T08:52:06.014 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-21T08:52:06.014 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 1024, 2024-04-21T08:52:06.014 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-21T08:52:06.014 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-21T08:52:06.015 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-21T08:52:06.015 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:06.015 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:06.015 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 2672, 2024-04-21T08:52:06.015 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 2672, 2024-04-21T08:52:06.015 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 2672, 2024-04-21T08:52:06.015 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 2672, 2024-04-21T08:52:06.015 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-21T08:52:06.015 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:06.015 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:06.015 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:06.015 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:06.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:06.015 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:52:06.015 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-21T08:52:06.016 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-04-21T08:52:06.016 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-21T08:52:06.016 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-21T08:52:06.016 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-21T08:52:06.016 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:06.016 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:06.016 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-21T08:52:06.016 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-21T08:52:06.016 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 0, 2024-04-21T08:52:06.016 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 0, 2024-04-21T08:52:06.016 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-21T08:52:06.016 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:06.016 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:06.016 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:06.016 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:06.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:06.017 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle-0x5643c335e6e0": { 2024-04-21T08:52:06.017 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-21T08:52:06.017 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-04-21T08:52:06.017 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-21T08:52:06.017 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-21T08:52:06.017 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-21T08:52:06.017 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:06.017 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:06.017 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-21T08:52:06.017 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-21T08:52:06.017 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 0, 2024-04-21T08:52:06.017 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 0, 2024-04-21T08:52:06.017 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-21T08:52:06.017 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:06.017 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:06.018 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:06.018 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:06.018 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:06.018 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-21T08:52:06.744 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-21T08:52:06.744 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-21T08:52:06.744 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-21T08:52:06.745 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-21T08:52:06.745 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-04-21T08:52:06.745 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:52:06.745 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:52:06.745 DEBUG:tasks.check_counter:Getting stats from h 2024-04-21T08:52:06.745 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-21T08:52:06.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:06 smithi175 ceph-mon[31920]: pgmap v411: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-21T08:52:06.982 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:52:07.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:06 smithi112 ceph-mon[31085]: pgmap v411: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-21T08:52:07.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:06 smithi043 ceph-mon[27978]: pgmap v411: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-21T08:52:08.740 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:08 smithi112 ceph-mon[31085]: pgmap v412: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-21T08:52:08.740 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-21T08:52:08.741 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:52:08.741 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 489, 2024-04-21T08:52:08.741 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 474, 2024-04-21T08:52:08.741 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 1737931, 2024-04-21T08:52:08.741 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 986196, 2024-04-21T08:52:08.741 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 79, 2024-04-21T08:52:08.741 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 5, 2024-04-21T08:52:08.741 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.092963976, 2024-04-21T08:52:08.741 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.022525035, 2024-04-21T08:52:08.741 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.066563565, 2024-04-21T08:52:08.741 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.007216226, 2024-04-21T08:52:08.741 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:08.742 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 468, 2024-04-21T08:52:08.742 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.041980530, 2024-04-21T08:52:08.742 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000089701 2024-04-21T08:52:08.742 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.742 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:08.742 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 208, 2024-04-21T08:52:08.742 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000041738, 2024-04-21T08:52:08.742 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000200 2024-04-21T08:52:08.742 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.742 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 1693376, 2024-04-21T08:52:08.742 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 171888 2024-04-21T08:52:08.742 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.742 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:52:08.742 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 1587, 2024-04-21T08:52:08.743 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 1343, 2024-04-21T08:52:08.743 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 456535, 2024-04-21T08:52:08.743 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 914852, 2024-04-21T08:52:08.743 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 12, 2024-04-21T08:52:08.743 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 5, 2024-04-21T08:52:08.743 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.156359479, 2024-04-21T08:52:08.743 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.076565930, 2024-04-21T08:52:08.743 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.086438624, 2024-04-21T08:52:08.743 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.005370537, 2024-04-21T08:52:08.743 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:08.743 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1346, 2024-04-21T08:52:08.743 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.080282987, 2024-04-21T08:52:08.743 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000059645 2024-04-21T08:52:08.743 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.743 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:08.744 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2564, 2024-04-21T08:52:08.744 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000689768, 2024-04-21T08:52:08.744 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000269 2024-04-21T08:52:08.744 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.744 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-21T08:52:08.744 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 503696 2024-04-21T08:52:08.744 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.744 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:52:08.744 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 96, 2024-04-21T08:52:08.744 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 138, 2024-04-21T08:52:08.744 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 39141, 2024-04-21T08:52:08.744 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 58716, 2024-04-21T08:52:08.745 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 10, 2024-04-21T08:52:08.745 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 5, 2024-04-21T08:52:08.745 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.023566587, 2024-04-21T08:52:08.745 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.007062504, 2024-04-21T08:52:08.745 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.015030937, 2024-04-21T08:52:08.745 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000756275, 2024-04-21T08:52:08.745 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:08.745 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 136, 2024-04-21T08:52:08.745 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.025511298, 2024-04-21T08:52:08.745 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000187583 2024-04-21T08:52:08.745 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.746 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:08.746 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 176, 2024-04-21T08:52:08.746 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000045105, 2024-04-21T08:52:08.746 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000256 2024-04-21T08:52:08.746 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.746 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-04-21T08:52:08.746 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 13744 2024-04-21T08:52:08.746 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.746 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-04-21T08:52:08.746 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 1, 2024-04-21T08:52:08.746 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-04-21T08:52:08.746 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.746 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-MDSRank": { 2024-04-21T08:52:08.746 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-04-21T08:52:08.747 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-04-21T08:52:08.747 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 48, 2024-04-21T08:52:08.747 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.001920965, 2024-04-21T08:52:08.747 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000040020 2024-04-21T08:52:08.747 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:08.747 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.747 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-PurgeQueue": { 2024-04-21T08:52:08.747 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-04-21T08:52:08.747 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-04-21T08:52:08.747 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2, 2024-04-21T08:52:08.747 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000008748, 2024-04-21T08:52:08.747 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000004374 2024-04-21T08:52:08.747 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:08.747 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.747 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-04-21T08:52:08.748 INFO:teuthology.orchestra.run.smithi112.stdout: "request": 8, 2024-04-21T08:52:08.748 INFO:teuthology.orchestra.run.smithi112.stdout: "reply": 5, 2024-04-21T08:52:08.748 INFO:teuthology.orchestra.run.smithi112.stdout: "reply_latency": { 2024-04-21T08:52:08.748 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 5, 2024-04-21T08:52:08.748 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.001011128, 2024-04-21T08:52:08.748 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000202225 2024-04-21T08:52:08.748 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.748 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_reply": 0, 2024-04-21T08:52:08.748 INFO:teuthology.orchestra.run.smithi112.stdout: "forward": 3, 2024-04-21T08:52:08.748 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_complete": 0, 2024-04-21T08:52:08.748 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_keys": 0, 2024-04-21T08:52:08.748 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_commit": 11, 2024-04-21T08:52:08.748 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_split": 0, 2024-04-21T08:52:08.748 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_merge": 0, 2024-04-21T08:52:08.748 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes": 16, 2024-04-21T08:52:08.749 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_top": 0, 2024-04-21T08:52:08.749 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_bottom": 16, 2024-04-21T08:52:08.749 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pin_tail": 0, 2024-04-21T08:52:08.749 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pinned": 13, 2024-04-21T08:52:08.749 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_expired": 4, 2024-04-21T08:52:08.749 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_with_caps": 2, 2024-04-21T08:52:08.749 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": 2, 2024-04-21T08:52:08.749 INFO:teuthology.orchestra.run.smithi112.stdout: "subtrees": 3, 2024-04-21T08:52:08.749 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse": 28, 2024-04-21T08:52:08.749 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_hit": 17, 2024-04-21T08:52:08.749 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_forward": 1, 2024-04-21T08:52:08.749 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_discover": 8, 2024-04-21T08:52:08.749 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:52:08.749 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_remote_ino": 0, 2024-04-21T08:52:08.749 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_lock": 0, 2024-04-21T08:52:08.750 INFO:teuthology.orchestra.run.smithi112.stdout: "load_cent": 0, 2024-04-21T08:52:08.750 INFO:teuthology.orchestra.run.smithi112.stdout: "q": 0, 2024-04-21T08:52:08.750 INFO:teuthology.orchestra.run.smithi112.stdout: "exported": 0, 2024-04-21T08:52:08.750 INFO:teuthology.orchestra.run.smithi112.stdout: "exported_inodes": 0, 2024-04-21T08:52:08.750 INFO:teuthology.orchestra.run.smithi112.stdout: "imported": 0, 2024-04-21T08:52:08.750 INFO:teuthology.orchestra.run.smithi112.stdout: "imported_inodes": 0, 2024-04-21T08:52:08.750 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_dir_fetch": 0, 2024-04-21T08:52:08.750 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_backtrace_fetch": 0, 2024-04-21T08:52:08.750 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_peer_discover": 0, 2024-04-21T08:52:08.750 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rfiles": 5, 2024-04-21T08:52:08.750 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rbytes": 298, 2024-04-21T08:52:08.750 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rsnaps": 6, 2024-04-21T08:52:08.750 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T08:52:08.750 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_set_tag": 0, 2024-04-21T08:52:08.751 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:52:08.751 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:52:08.751 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:52:08.751 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:52:08.751 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:52:08.751 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_file_inodes": 0, 2024-04-21T08:52:08.751 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:52:08.751 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_revoke": 2, 2024-04-21T08:52:08.751 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_grant": 0, 2024-04-21T08:52:08.751 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:52:08.751 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:52:08.752 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:52:08.752 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps": 2, 2024-04-21T08:52:08.752 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps_dirty": 0, 2024-04-21T08:52:08.752 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_cap_release": 0, 2024-04-21T08:52:08.752 INFO:teuthology.orchestra.run.smithi112.stdout: "process_request_cap_release": 2 2024-04-21T08:52:08.752 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.752 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_cache": { 2024-04-21T08:52:08.752 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update": 0, 2024-04-21T08:52:08.752 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update_receipt": 17, 2024-04-21T08:52:08.752 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_try_discover": 12, 2024-04-21T08:52:08.752 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_send_discover": 12, 2024-04-21T08:52:08.752 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_handle_discover": 0, 2024-04-21T08:52:08.752 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays": 0, 2024-04-21T08:52:08.753 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_delayed": 0, 2024-04-21T08:52:08.753 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:52:08.753 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_created": 0, 2024-04-21T08:52:08.753 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_enqueued": 0, 2024-04-21T08:52:08.753 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_reintegrated": 0, 2024-04-21T08:52:08.753 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_migrated": 0, 2024-04-21T08:52:08.753 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_processing": 0, 2024-04-21T08:52:08.753 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:52:08.753 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:52:08.753 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_started": 0, 2024-04-21T08:52:08.753 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_completed": 0, 2024-04-21T08:52:08.753 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_path": 3, 2024-04-21T08:52:08.753 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_inode": 15, 2024-04-21T08:52:08.753 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:52:08.753 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_exportdir": 0, 2024-04-21T08:52:08.754 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_flush": 0, 2024-04-21T08:52:08.754 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:52:08.754 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragstats": 0, 2024-04-21T08:52:08.754 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_inodestats": 0 2024-04-21T08:52:08.754 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.754 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_log": { 2024-04-21T08:52:08.754 INFO:teuthology.orchestra.run.smithi112.stdout: "evlrg": 0, 2024-04-21T08:52:08.754 INFO:teuthology.orchestra.run.smithi112.stdout: "evadd": 7, 2024-04-21T08:52:08.754 INFO:teuthology.orchestra.run.smithi112.stdout: "evex": 0, 2024-04-21T08:52:08.754 INFO:teuthology.orchestra.run.smithi112.stdout: "evtrm": 0, 2024-04-21T08:52:08.754 INFO:teuthology.orchestra.run.smithi112.stdout: "ev": 7, 2024-04-21T08:52:08.754 INFO:teuthology.orchestra.run.smithi112.stdout: "evexg": 0, 2024-04-21T08:52:08.754 INFO:teuthology.orchestra.run.smithi112.stdout: "evexd": 0, 2024-04-21T08:52:08.754 INFO:teuthology.orchestra.run.smithi112.stdout: "segadd": 2, 2024-04-21T08:52:08.754 INFO:teuthology.orchestra.run.smithi112.stdout: "segex": 0, 2024-04-21T08:52:08.755 INFO:teuthology.orchestra.run.smithi112.stdout: "segtrm": 0, 2024-04-21T08:52:08.755 INFO:teuthology.orchestra.run.smithi112.stdout: "seg": 2, 2024-04-21T08:52:08.755 INFO:teuthology.orchestra.run.smithi112.stdout: "segmjr": 2, 2024-04-21T08:52:08.755 INFO:teuthology.orchestra.run.smithi112.stdout: "segexg": 0, 2024-04-21T08:52:08.755 INFO:teuthology.orchestra.run.smithi112.stdout: "segexd": 0, 2024-04-21T08:52:08.755 INFO:teuthology.orchestra.run.smithi112.stdout: "expos": 4194304, 2024-04-21T08:52:08.755 INFO:teuthology.orchestra.run.smithi112.stdout: "wrpos": 4196264, 2024-04-21T08:52:08.755 INFO:teuthology.orchestra.run.smithi112.stdout: "rdpos": 0, 2024-04-21T08:52:08.755 INFO:teuthology.orchestra.run.smithi112.stdout: "jlat": { 2024-04-21T08:52:08.755 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 6, 2024-04-21T08:52:08.755 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.010960360, 2024-04-21T08:52:08.755 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001826726 2024-04-21T08:52:08.755 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.758 INFO:teuthology.orchestra.run.smithi112.stdout: "replayed": 0 2024-04-21T08:52:08.758 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.758 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_mem": { 2024-04-21T08:52:08.758 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 20, 2024-04-21T08:52:08.758 INFO:teuthology.orchestra.run.smithi112.stdout: "ino+": 22, 2024-04-21T08:52:08.758 INFO:teuthology.orchestra.run.smithi112.stdout: "ino-": 2, 2024-04-21T08:52:08.758 INFO:teuthology.orchestra.run.smithi112.stdout: "dir": 19, 2024-04-21T08:52:08.758 INFO:teuthology.orchestra.run.smithi112.stdout: "dir+": 22, 2024-04-21T08:52:08.758 INFO:teuthology.orchestra.run.smithi112.stdout: "dir-": 3, 2024-04-21T08:52:08.758 INFO:teuthology.orchestra.run.smithi112.stdout: "dn": 16, 2024-04-21T08:52:08.758 INFO:teuthology.orchestra.run.smithi112.stdout: "dn+": 20, 2024-04-21T08:52:08.758 INFO:teuthology.orchestra.run.smithi112.stdout: "dn-": 4, 2024-04-21T08:52:08.758 INFO:teuthology.orchestra.run.smithi112.stdout: "cap": 2, 2024-04-21T08:52:08.759 INFO:teuthology.orchestra.run.smithi112.stdout: "cap+": 5, 2024-04-21T08:52:08.759 INFO:teuthology.orchestra.run.smithi112.stdout: "cap-": 3, 2024-04-21T08:52:08.759 INFO:teuthology.orchestra.run.smithi112.stdout: "rss": 56256, 2024-04-21T08:52:08.759 INFO:teuthology.orchestra.run.smithi112.stdout: "heap": 223608 2024-04-21T08:52:08.759 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.759 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_server": { 2024-04-21T08:52:08.759 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_client_request": 9, 2024-04-21T08:52:08.759 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_server_request": 0, 2024-04-21T08:52:08.759 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_request": 8, 2024-04-21T08:52:08.759 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_session": 19, 2024-04-21T08:52:08.759 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_peer_request": 15, 2024-04-21T08:52:08.759 INFO:teuthology.orchestra.run.smithi112.stdout: "req_create_latency": { 2024-04-21T08:52:08.759 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.760 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.760 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.760 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.760 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getattr_latency": { 2024-04-21T08:52:08.760 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2, 2024-04-21T08:52:08.760 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000473450, 2024-04-21T08:52:08.760 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000236725 2024-04-21T08:52:08.760 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.760 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getfilelock_latency": { 2024-04-21T08:52:08.760 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.760 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.760 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.760 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.760 INFO:teuthology.orchestra.run.smithi112.stdout: "req_link_latency": { 2024-04-21T08:52:08.760 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.761 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.761 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.761 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.761 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookup_latency": { 2024-04-21T08:52:08.761 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1, 2024-04-21T08:52:08.761 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000305956, 2024-04-21T08:52:08.761 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000305956 2024-04-21T08:52:08.761 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.761 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookuphash_latency": { 2024-04-21T08:52:08.761 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.761 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.761 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.761 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.761 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupino_latency": { 2024-04-21T08:52:08.762 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2, 2024-04-21T08:52:08.762 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000231722, 2024-04-21T08:52:08.762 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000115861 2024-04-21T08:52:08.762 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.762 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupname_latency": { 2024-04-21T08:52:08.762 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.762 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.762 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.762 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.762 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupparent_latency": { 2024-04-21T08:52:08.762 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.762 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.762 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.763 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.763 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupsnap_latency": { 2024-04-21T08:52:08.763 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.763 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.763 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.763 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.763 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lssnap_latency": { 2024-04-21T08:52:08.763 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.763 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.763 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.763 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.763 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mkdir_latency": { 2024-04-21T08:52:08.764 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.764 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.764 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.764 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.764 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mknod_latency": { 2024-04-21T08:52:08.764 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.764 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.764 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.765 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mksnap_latency": { 2024-04-21T08:52:08.765 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.765 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.765 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.765 INFO:teuthology.orchestra.run.smithi112.stdout: "req_open_latency": { 2024-04-21T08:52:08.765 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.765 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.765 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.765 INFO:teuthology.orchestra.run.smithi112.stdout: "req_readdir_latency": { 2024-04-21T08:52:08.765 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.766 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.766 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.766 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.766 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rename_latency": { 2024-04-21T08:52:08.766 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.766 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.766 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.766 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.766 INFO:teuthology.orchestra.run.smithi112.stdout: "req_renamesnap_latency": { 2024-04-21T08:52:08.766 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.766 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.766 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.766 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.766 INFO:teuthology.orchestra.run.smithi112.stdout: "req_snapdiff_latency": { 2024-04-21T08:52:08.767 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.767 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.767 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.767 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmdir_latency": { 2024-04-21T08:52:08.767 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.767 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.767 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.767 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmsnap_latency": { 2024-04-21T08:52:08.767 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.767 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.767 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.767 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmxattr_latency": { 2024-04-21T08:52:08.768 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.768 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.768 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.768 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.768 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setattr_latency": { 2024-04-21T08:52:08.768 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.768 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.768 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.768 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.768 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setdirlayout_latency": { 2024-04-21T08:52:08.768 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.768 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.768 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.768 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.769 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setfilelock_latency": { 2024-04-21T08:52:08.769 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.769 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.769 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.769 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setlayout_latency": { 2024-04-21T08:52:08.769 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.769 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.769 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.769 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setxattr_latency": { 2024-04-21T08:52:08.769 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.769 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.769 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.770 INFO:teuthology.orchestra.run.smithi112.stdout: "req_symlink_latency": { 2024-04-21T08:52:08.770 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.770 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.770 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.770 INFO:teuthology.orchestra.run.smithi112.stdout: "req_unlink_latency": { 2024-04-21T08:52:08.770 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.770 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.770 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.770 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:52:08.770 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:52:08.770 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getvxattr_latency": { 2024-04-21T08:52:08.770 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.771 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.771 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.771 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:08.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.771 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_sessions": { 2024-04-21T08:52:08.771 INFO:teuthology.orchestra.run.smithi112.stdout: "session_count": 1, 2024-04-21T08:52:08.771 INFO:teuthology.orchestra.run.smithi112.stdout: "session_add": 3, 2024-04-21T08:52:08.771 INFO:teuthology.orchestra.run.smithi112.stdout: "session_remove": 2, 2024-04-21T08:52:08.771 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_open": 1, 2024-04-21T08:52:08.771 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_stale": 0, 2024-04-21T08:52:08.771 INFO:teuthology.orchestra.run.smithi112.stdout: "total_load": 3, 2024-04-21T08:52:08.771 INFO:teuthology.orchestra.run.smithi112.stdout: "average_load": 1, 2024-04-21T08:52:08.771 INFO:teuthology.orchestra.run.smithi112.stdout: "avg_session_uptime": 778, 2024-04-21T08:52:08.771 INFO:teuthology.orchestra.run.smithi112.stdout: "mdthresh_evicted": 0 2024-04-21T08:52:08.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.772 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-04-21T08:52:08.772 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:52:08.772 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-04-21T08:52:08.772 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:52:08.772 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:52:08.772 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:52:08.772 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:52:08.772 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:52:08.772 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:52:08.772 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:52:08.772 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:52:08.772 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:52:08.772 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:52:08.772 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:52:08.772 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:52:08.773 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:52:08.773 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-04-21T08:52:08.773 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:52:08.773 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-04-21T08:52:08.773 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:52:08.773 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:52:08.773 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:52:08.773 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:52:08.773 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:52:08.773 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:52:08.773 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:52:08.773 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-04-21T08:52:08.773 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:52:08.773 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:52:08.773 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:52:08.774 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-04-21T08:52:08.774 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-04-21T08:52:08.774 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-04-21T08:52:08.774 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:52:08.774 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:52:08.774 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:52:08.774 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:52:08.774 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 262695, 2024-04-21T08:52:08.774 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 86, 2024-04-21T08:52:08.774 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:52:08.774 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-04-21T08:52:08.774 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-04-21T08:52:08.774 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-04-21T08:52:08.774 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:52:08.774 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-04-21T08:52:08.775 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:52:08.775 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-04-21T08:52:08.775 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 9800, 2024-04-21T08:52:08.775 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 188, 2024-04-21T08:52:08.775 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:52:08.775 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:52:08.775 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-04-21T08:52:08.775 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-04-21T08:52:08.775 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 87149, 2024-04-21T08:52:08.775 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 346, 2024-04-21T08:52:08.775 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-04-21T08:52:08.775 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-04-21T08:52:08.775 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-04-21T08:52:08.775 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-04-21T08:52:08.775 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.776 INFO:teuthology.orchestra.run.smithi112.stdout: "objecter": { 2024-04-21T08:52:08.776 INFO:teuthology.orchestra.run.smithi112.stdout: "op_active": 0, 2024-04-21T08:52:08.776 INFO:teuthology.orchestra.run.smithi112.stdout: "op_laggy": 0, 2024-04-21T08:52:08.776 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send": 46, 2024-04-21T08:52:08.776 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send_bytes": 12703, 2024-04-21T08:52:08.776 INFO:teuthology.orchestra.run.smithi112.stdout: "op_resend": 30, 2024-04-21T08:52:08.776 INFO:teuthology.orchestra.run.smithi112.stdout: "op_reply": 46, 2024-04-21T08:52:08.776 INFO:teuthology.orchestra.run.smithi112.stdout: "op_latency": { 2024-04-21T08:52:08.776 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 46, 2024-04-21T08:52:08.776 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.237993716, 2024-04-21T08:52:08.776 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.005173776 2024-04-21T08:52:08.776 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.776 INFO:teuthology.orchestra.run.smithi112.stdout: "op_inflight": 0, 2024-04-21T08:52:08.776 INFO:teuthology.orchestra.run.smithi112.stdout: "oplen_avg": { 2024-04-21T08:52:08.776 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 46, 2024-04-21T08:52:08.777 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 71 2024-04-21T08:52:08.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.777 INFO:teuthology.orchestra.run.smithi112.stdout: "op": 46, 2024-04-21T08:52:08.777 INFO:teuthology.orchestra.run.smithi112.stdout: "op_r": 0, 2024-04-21T08:52:08.777 INFO:teuthology.orchestra.run.smithi112.stdout: "op_w": 46, 2024-04-21T08:52:08.777 INFO:teuthology.orchestra.run.smithi112.stdout: "op_rmw": 0, 2024-04-21T08:52:08.777 INFO:teuthology.orchestra.run.smithi112.stdout: "op_pg": 0, 2024-04-21T08:52:08.777 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_stat": 0, 2024-04-21T08:52:08.777 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_create": 10, 2024-04-21T08:52:08.777 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_read": 0, 2024-04-21T08:52:08.777 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_write": 6, 2024-04-21T08:52:08.777 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writefull": 8, 2024-04-21T08:52:08.777 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writesame": 0, 2024-04-21T08:52:08.777 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_append": 0, 2024-04-21T08:52:08.777 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_zero": 0, 2024-04-21T08:52:08.778 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_truncate": 0, 2024-04-21T08:52:08.778 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_delete": 6, 2024-04-21T08:52:08.778 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_mapext": 0, 2024-04-21T08:52:08.778 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_sparse_read": 0, 2024-04-21T08:52:08.778 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_clonerange": 0, 2024-04-21T08:52:08.778 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_getxattr": 0, 2024-04-21T08:52:08.778 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_setxattr": 20, 2024-04-21T08:52:08.778 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:52:08.778 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_rmxattr": 0, 2024-04-21T08:52:08.778 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:52:08.778 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_call": 0, 2024-04-21T08:52:08.778 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_watch": 0, 2024-04-21T08:52:08.778 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_notify": 0, 2024-04-21T08:52:08.778 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:52:08.778 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls": 0, 2024-04-21T08:52:08.779 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:52:08.779 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_other": 0, 2024-04-21T08:52:08.779 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_active": 0, 2024-04-21T08:52:08.779 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_send": 0, 2024-04-21T08:52:08.779 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_resend": 0, 2024-04-21T08:52:08.779 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_ping": 0, 2024-04-21T08:52:08.779 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_active": 0, 2024-04-21T08:52:08.779 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_send": 0, 2024-04-21T08:52:08.779 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_resend": 0, 2024-04-21T08:52:08.779 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_active": 0, 2024-04-21T08:52:08.779 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_send": 0, 2024-04-21T08:52:08.779 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_resend": 0, 2024-04-21T08:52:08.779 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_active": 0, 2024-04-21T08:52:08.779 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_send": 0, 2024-04-21T08:52:08.779 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_resend": 0, 2024-04-21T08:52:08.780 INFO:teuthology.orchestra.run.smithi112.stdout: "command_active": 0, 2024-04-21T08:52:08.780 INFO:teuthology.orchestra.run.smithi112.stdout: "command_send": 0, 2024-04-21T08:52:08.780 INFO:teuthology.orchestra.run.smithi112.stdout: "command_resend": 0, 2024-04-21T08:52:08.780 INFO:teuthology.orchestra.run.smithi112.stdout: "map_epoch": 108, 2024-04-21T08:52:08.780 INFO:teuthology.orchestra.run.smithi112.stdout: "map_full": 0, 2024-04-21T08:52:08.780 INFO:teuthology.orchestra.run.smithi112.stdout: "map_inc": 18, 2024-04-21T08:52:08.780 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_sessions": 9, 2024-04-21T08:52:08.780 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_open": 13, 2024-04-21T08:52:08.780 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_close": 4, 2024-04-21T08:52:08.780 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_laggy": 0, 2024-04-21T08:52:08.780 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_wr": 20, 2024-04-21T08:52:08.780 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_rd": 0, 2024-04-21T08:52:08.780 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_del": 1 2024-04-21T08:52:08.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.781 INFO:teuthology.orchestra.run.smithi112.stdout: "oft": { 2024-04-21T08:52:08.781 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_objs": 1, 2024-04-21T08:52:08.781 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:52:08.781 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_updates": 0, 2024-04-21T08:52:08.781 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_removes": 0 2024-04-21T08:52:08.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.781 INFO:teuthology.orchestra.run.smithi112.stdout: "osd-slow-ops": { 2024-04-21T08:52:08.781 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_ops_count": 18 2024-04-21T08:52:08.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.781 INFO:teuthology.orchestra.run.smithi112.stdout: "purge_queue": { 2024-04-21T08:52:08.781 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops": 0, 2024-04-21T08:52:08.781 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T08:52:08.781 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing": 0, 2024-04-21T08:52:08.781 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_high_water": 0, 2024-04-21T08:52:08.781 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed_ops": 0, 2024-04-21T08:52:08.782 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed": 0, 2024-04-21T08:52:08.782 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_item_in_journal": 0 2024-04-21T08:52:08.782 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.782 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:52:08.782 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-21T08:52:08.782 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-21T08:52:08.782 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-21T08:52:08.782 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 2168, 2024-04-21T08:52:08.782 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 2017535, 2024-04-21T08:52:08.782 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:08.782 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 2168, 2024-04-21T08:52:08.782 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-21T08:52:08.782 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-21T08:52:08.782 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 2168, 2024-04-21T08:52:08.783 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 2017535, 2024-04-21T08:52:08.783 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-21T08:52:08.783 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.783 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.783 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.783 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:08.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.783 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_bytes": { 2024-04-21T08:52:08.783 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-21T08:52:08.783 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-21T08:52:08.783 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-21T08:52:08.783 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-21T08:52:08.783 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-21T08:52:08.783 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:08.783 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:08.784 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 46, 2024-04-21T08:52:08.784 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 12703, 2024-04-21T08:52:08.784 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 46, 2024-04-21T08:52:08.784 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 12703, 2024-04-21T08:52:08.784 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-21T08:52:08.784 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.784 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.784 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.784 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:08.784 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.784 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_ops": { 2024-04-21T08:52:08.784 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-21T08:52:08.784 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 1024, 2024-04-21T08:52:08.784 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-21T08:52:08.784 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-21T08:52:08.785 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-21T08:52:08.785 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:08.785 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:08.785 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 46, 2024-04-21T08:52:08.785 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 46, 2024-04-21T08:52:08.785 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 46, 2024-04-21T08:52:08.785 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 46, 2024-04-21T08:52:08.785 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-21T08:52:08.785 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.785 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.785 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.785 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:08.785 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.785 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:52:08.785 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-21T08:52:08.785 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-04-21T08:52:08.786 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-21T08:52:08.786 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-21T08:52:08.786 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-21T08:52:08.786 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:08.786 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:08.786 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-21T08:52:08.786 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-21T08:52:08.786 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 0, 2024-04-21T08:52:08.786 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 0, 2024-04-21T08:52:08.786 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-21T08:52:08.786 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.786 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.786 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.787 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:08.787 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:08.787 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle-0x56256bf4c500": { 2024-04-21T08:52:08.787 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-21T08:52:08.787 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-04-21T08:52:08.787 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-21T08:52:08.787 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 7, 2024-04-21T08:52:08.787 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 1960, 2024-04-21T08:52:08.787 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:08.787 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 7, 2024-04-21T08:52:08.787 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-21T08:52:08.787 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-21T08:52:08.787 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 6, 2024-04-21T08:52:08.787 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1960, 2024-04-21T08:52:08.787 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-21T08:52:08.787 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:08.788 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:08.788 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:08.788 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:08.788 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:08.788 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-21T08:52:08.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:08 smithi175 ceph-mon[31920]: pgmap v412: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-21T08:52:09.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:08 smithi043 ceph-mon[27978]: pgmap v412: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-21T08:52:09.564 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-21T08:52:09.564 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-21T08:52:09.564 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-21T08:52:09.564 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=17 2024-04-21T08:52:09.564 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-21T08:52:09.564 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:52:09.564 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:52:09.564 DEBUG:tasks.check_counter:Getting stats from k 2024-04-21T08:52:09.565 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-21T08:52:09.813 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:52:10.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:10 smithi175 ceph-mon[31920]: pgmap v413: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-21T08:52:10.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:10 smithi175 ceph-mon[31920]: osdmap e109: 12 total, 12 up, 12 in 2024-04-21T08:52:10.965 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:10 smithi112 ceph-mon[31085]: pgmap v413: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-21T08:52:10.965 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:10 smithi112 ceph-mon[31085]: osdmap e109: 12 total, 12 up, 12 in 2024-04-21T08:52:11.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:10 smithi043 ceph-mon[27978]: pgmap v413: 129 pgs: 129 active+clean; 3.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-21T08:52:11.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:10 smithi043 ceph-mon[27978]: osdmap e109: 12 total, 12 up, 12 in 2024-04-21T08:52:11.532 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-21T08:52:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:52:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 1537, 2024-04-21T08:52:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 1126, 2024-04-21T08:52:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 431309, 2024-04-21T08:52:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 548677, 2024-04-21T08:52:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 15, 2024-04-21T08:52:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-04-21T08:52:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.148440003, 2024-04-21T08:52:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.068342995, 2024-04-21T08:52:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.083301226, 2024-04-21T08:52:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.004696158, 2024-04-21T08:52:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1129, 2024-04-21T08:52:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.092135854, 2024-04-21T08:52:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000081608 2024-04-21T08:52:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2483, 2024-04-21T08:52:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000834290, 2024-04-21T08:52:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000336 2024-04-21T08:52:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 4880, 2024-04-21T08:52:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 7600 2024-04-21T08:52:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:52:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 473, 2024-04-21T08:52:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 455, 2024-04-21T08:52:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 1746999, 2024-04-21T08:52:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 238883, 2024-04-21T08:52:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 8, 2024-04-21T08:52:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 6, 2024-04-21T08:52:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.070665561, 2024-04-21T08:52:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.023243354, 2024-04-21T08:52:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.051235858, 2024-04-21T08:52:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.007057602, 2024-04-21T08:52:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 454, 2024-04-21T08:52:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.047626696, 2024-04-21T08:52:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000104904 2024-04-21T08:52:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 223, 2024-04-21T08:52:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000049487, 2024-04-21T08:52:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000221 2024-04-21T08:52:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 1681792, 2024-04-21T08:52:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 171296 2024-04-21T08:52:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:52:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 149, 2024-04-21T08:52:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 372, 2024-04-21T08:52:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 45055, 2024-04-21T08:52:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 1282019, 2024-04-21T08:52:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 76, 2024-04-21T08:52:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 8, 2024-04-21T08:52:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.052121629, 2024-04-21T08:52:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.015743674, 2024-04-21T08:52:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.032254257, 2024-04-21T08:52:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000122612, 2024-04-21T08:52:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 365, 2024-04-21T08:52:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.029223602, 2024-04-21T08:52:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000080064 2024-04-21T08:52:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 227, 2024-04-21T08:52:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000045676, 2024-04-21T08:52:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000201 2024-04-21T08:52:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3536, 2024-04-21T08:52:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 515072 2024-04-21T08:52:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-04-21T08:52:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 1, 2024-04-21T08:52:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-04-21T08:52:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-MDSRank": { 2024-04-21T08:52:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-04-21T08:52:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-04-21T08:52:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 57, 2024-04-21T08:52:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.001627603, 2024-04-21T08:52:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000028554 2024-04-21T08:52:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-PurgeQueue": { 2024-04-21T08:52:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-04-21T08:52:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-04-21T08:52:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2, 2024-04-21T08:52:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000005237, 2024-04-21T08:52:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000002618 2024-04-21T08:52:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-04-21T08:52:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "request": 1, 2024-04-21T08:52:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "reply": 0, 2024-04-21T08:52:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "reply_latency": { 2024-04-21T08:52:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_reply": 0, 2024-04-21T08:52:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "forward": 1, 2024-04-21T08:52:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_complete": 0, 2024-04-21T08:52:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_keys": 0, 2024-04-21T08:52:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_commit": 11, 2024-04-21T08:52:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_split": 0, 2024-04-21T08:52:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_merge": 0, 2024-04-21T08:52:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes": 16, 2024-04-21T08:52:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_top": 0, 2024-04-21T08:52:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_bottom": 16, 2024-04-21T08:52:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pin_tail": 0, 2024-04-21T08:52:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pinned": 13, 2024-04-21T08:52:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_expired": 4, 2024-04-21T08:52:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_with_caps": 0, 2024-04-21T08:52:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": 0, 2024-04-21T08:52:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "subtrees": 3, 2024-04-21T08:52:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse": 20, 2024-04-21T08:52:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_hit": 11, 2024-04-21T08:52:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_forward": 1, 2024-04-21T08:52:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_discover": 8, 2024-04-21T08:52:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:52:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_remote_ino": 0, 2024-04-21T08:52:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_lock": 0, 2024-04-21T08:52:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "load_cent": 0, 2024-04-21T08:52:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "q": 0, 2024-04-21T08:52:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "exported": 0, 2024-04-21T08:52:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "exported_inodes": 0, 2024-04-21T08:52:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "imported": 0, 2024-04-21T08:52:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "imported_inodes": 0, 2024-04-21T08:52:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_dir_fetch": 0, 2024-04-21T08:52:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_backtrace_fetch": 0, 2024-04-21T08:52:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_peer_discover": 0, 2024-04-21T08:52:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rfiles": 5, 2024-04-21T08:52:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rbytes": 298, 2024-04-21T08:52:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rsnaps": 6, 2024-04-21T08:52:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T08:52:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_set_tag": 0, 2024-04-21T08:52:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:52:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:52:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:52:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:52:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:52:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_file_inodes": 0, 2024-04-21T08:52:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:52:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T08:52:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_grant": 0, 2024-04-21T08:52:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:52:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:52:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:52:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps": 0, 2024-04-21T08:52:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps_dirty": 0, 2024-04-21T08:52:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_cap_release": 0, 2024-04-21T08:52:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "process_request_cap_release": 0 2024-04-21T08:52:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_cache": { 2024-04-21T08:52:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update": 0, 2024-04-21T08:52:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update_receipt": 18, 2024-04-21T08:52:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_try_discover": 13, 2024-04-21T08:52:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_send_discover": 11, 2024-04-21T08:52:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_handle_discover": 0, 2024-04-21T08:52:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays": 0, 2024-04-21T08:52:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_delayed": 0, 2024-04-21T08:52:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:52:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_created": 0, 2024-04-21T08:52:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_enqueued": 0, 2024-04-21T08:52:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_reintegrated": 0, 2024-04-21T08:52:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_migrated": 0, 2024-04-21T08:52:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_processing": 0, 2024-04-21T08:52:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:52:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:52:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_started": 0, 2024-04-21T08:52:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_completed": 0, 2024-04-21T08:52:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_path": 3, 2024-04-21T08:52:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_inode": 15, 2024-04-21T08:52:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:52:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_exportdir": 0, 2024-04-21T08:52:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_flush": 0, 2024-04-21T08:52:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:52:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragstats": 0, 2024-04-21T08:52:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_inodestats": 0 2024-04-21T08:52:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_log": { 2024-04-21T08:52:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "evlrg": 0, 2024-04-21T08:52:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "evadd": 6, 2024-04-21T08:52:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "evex": 0, 2024-04-21T08:52:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "evtrm": 0, 2024-04-21T08:52:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "ev": 6, 2024-04-21T08:52:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "evexg": 0, 2024-04-21T08:52:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "evexd": 0, 2024-04-21T08:52:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "segadd": 2, 2024-04-21T08:52:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "segex": 0, 2024-04-21T08:52:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "segtrm": 0, 2024-04-21T08:52:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "seg": 2, 2024-04-21T08:52:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "segmjr": 2, 2024-04-21T08:52:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "segexg": 0, 2024-04-21T08:52:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "segexd": 0, 2024-04-21T08:52:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "expos": 4194304, 2024-04-21T08:52:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "wrpos": 4196123, 2024-04-21T08:52:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "rdpos": 0, 2024-04-21T08:52:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "jlat": { 2024-04-21T08:52:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 5, 2024-04-21T08:52:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.008236620, 2024-04-21T08:52:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001647324 2024-04-21T08:52:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "replayed": 0 2024-04-21T08:52:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_mem": { 2024-04-21T08:52:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 20, 2024-04-21T08:52:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "ino+": 22, 2024-04-21T08:52:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "ino-": 2, 2024-04-21T08:52:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "dir": 19, 2024-04-21T08:52:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "dir+": 21, 2024-04-21T08:52:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "dir-": 2, 2024-04-21T08:52:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "dn": 16, 2024-04-21T08:52:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "dn+": 20, 2024-04-21T08:52:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "dn-": 4, 2024-04-21T08:52:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "cap": 0, 2024-04-21T08:52:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "cap+": 0, 2024-04-21T08:52:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "cap-": 0, 2024-04-21T08:52:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "rss": 52916, 2024-04-21T08:52:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "heap": 223608 2024-04-21T08:52:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_server": { 2024-04-21T08:52:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_client_request": 1, 2024-04-21T08:52:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_server_request": 0, 2024-04-21T08:52:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_request": 1, 2024-04-21T08:52:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_session": 25, 2024-04-21T08:52:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_peer_request": 15, 2024-04-21T08:52:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "req_create_latency": { 2024-04-21T08:52:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getattr_latency": { 2024-04-21T08:52:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getfilelock_latency": { 2024-04-21T08:52:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "req_link_latency": { 2024-04-21T08:52:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookup_latency": { 2024-04-21T08:52:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookuphash_latency": { 2024-04-21T08:52:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupino_latency": { 2024-04-21T08:52:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupname_latency": { 2024-04-21T08:52:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupparent_latency": { 2024-04-21T08:52:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupsnap_latency": { 2024-04-21T08:52:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lssnap_latency": { 2024-04-21T08:52:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mkdir_latency": { 2024-04-21T08:52:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mknod_latency": { 2024-04-21T08:52:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mksnap_latency": { 2024-04-21T08:52:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "req_open_latency": { 2024-04-21T08:52:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "req_readdir_latency": { 2024-04-21T08:52:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rename_latency": { 2024-04-21T08:52:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "req_renamesnap_latency": { 2024-04-21T08:52:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "req_snapdiff_latency": { 2024-04-21T08:52:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmdir_latency": { 2024-04-21T08:52:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmsnap_latency": { 2024-04-21T08:52:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmxattr_latency": { 2024-04-21T08:52:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setattr_latency": { 2024-04-21T08:52:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setdirlayout_latency": { 2024-04-21T08:52:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setfilelock_latency": { 2024-04-21T08:52:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setlayout_latency": { 2024-04-21T08:52:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setxattr_latency": { 2024-04-21T08:52:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "req_symlink_latency": { 2024-04-21T08:52:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "req_unlink_latency": { 2024-04-21T08:52:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:52:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:52:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getvxattr_latency": { 2024-04-21T08:52:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_sessions": { 2024-04-21T08:52:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "session_count": 2, 2024-04-21T08:52:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "session_add": 3, 2024-04-21T08:52:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "session_remove": 1, 2024-04-21T08:52:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_open": 2, 2024-04-21T08:52:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_stale": 0, 2024-04-21T08:52:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "total_load": 0, 2024-04-21T08:52:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "average_load": 0, 2024-04-21T08:52:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "avg_session_uptime": 763, 2024-04-21T08:52:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "mdthresh_evicted": 0 2024-04-21T08:52:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-04-21T08:52:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:52:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-04-21T08:52:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:52:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:52:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:52:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:52:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:52:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:52:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:52:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:52:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:52:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:52:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:52:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:52:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:52:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-04-21T08:52:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:52:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-04-21T08:52:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:52:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:52:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:52:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:52:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:52:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:52:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:52:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-04-21T08:52:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:52:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:52:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:52:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-04-21T08:52:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-04-21T08:52:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-04-21T08:52:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:52:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:52:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:52:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:52:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 255735, 2024-04-21T08:52:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 92, 2024-04-21T08:52:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:52:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-04-21T08:52:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-04-21T08:52:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-04-21T08:52:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:52:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-04-21T08:52:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:52:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-04-21T08:52:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 10040, 2024-04-21T08:52:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 194, 2024-04-21T08:52:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:52:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:52:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-04-21T08:52:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-04-21T08:52:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 86405, 2024-04-21T08:52:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 341, 2024-04-21T08:52:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-04-21T08:52:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-04-21T08:52:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-04-21T08:52:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-04-21T08:52:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "objecter": { 2024-04-21T08:52:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "op_active": 0, 2024-04-21T08:52:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "op_laggy": 0, 2024-04-21T08:52:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send": 42, 2024-04-21T08:52:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send_bytes": 12211, 2024-04-21T08:52:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "op_resend": 33, 2024-04-21T08:52:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "op_reply": 42, 2024-04-21T08:52:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "op_latency": { 2024-04-21T08:52:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 42, 2024-04-21T08:52:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.226994029, 2024-04-21T08:52:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.005404619 2024-04-21T08:52:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "op_inflight": 0, 2024-04-21T08:52:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "oplen_avg": { 2024-04-21T08:52:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 42, 2024-04-21T08:52:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 64 2024-04-21T08:52:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "op": 42, 2024-04-21T08:52:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "op_r": 0, 2024-04-21T08:52:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "op_w": 42, 2024-04-21T08:52:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "op_rmw": 0, 2024-04-21T08:52:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "op_pg": 0, 2024-04-21T08:52:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_stat": 0, 2024-04-21T08:52:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_create": 10, 2024-04-21T08:52:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_read": 0, 2024-04-21T08:52:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_write": 5, 2024-04-21T08:52:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writefull": 8, 2024-04-21T08:52:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writesame": 0, 2024-04-21T08:52:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_append": 0, 2024-04-21T08:52:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_zero": 0, 2024-04-21T08:52:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_truncate": 0, 2024-04-21T08:52:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_delete": 6, 2024-04-21T08:52:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_mapext": 0, 2024-04-21T08:52:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_sparse_read": 0, 2024-04-21T08:52:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_clonerange": 0, 2024-04-21T08:52:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_getxattr": 0, 2024-04-21T08:52:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_setxattr": 20, 2024-04-21T08:52:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:52:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_rmxattr": 0, 2024-04-21T08:52:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:52:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_call": 0, 2024-04-21T08:52:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_watch": 0, 2024-04-21T08:52:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_notify": 0, 2024-04-21T08:52:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:52:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls": 0, 2024-04-21T08:52:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:52:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_other": 0, 2024-04-21T08:52:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_active": 0, 2024-04-21T08:52:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_send": 0, 2024-04-21T08:52:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_resend": 0, 2024-04-21T08:52:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_ping": 0, 2024-04-21T08:52:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_active": 0, 2024-04-21T08:52:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_send": 0, 2024-04-21T08:52:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_resend": 0, 2024-04-21T08:52:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_active": 0, 2024-04-21T08:52:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_send": 0, 2024-04-21T08:52:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_resend": 0, 2024-04-21T08:52:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_active": 0, 2024-04-21T08:52:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_send": 0, 2024-04-21T08:52:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_resend": 0, 2024-04-21T08:52:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "command_active": 0, 2024-04-21T08:52:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "command_send": 0, 2024-04-21T08:52:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "command_resend": 0, 2024-04-21T08:52:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "map_epoch": 109, 2024-04-21T08:52:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "map_full": 0, 2024-04-21T08:52:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "map_inc": 19, 2024-04-21T08:52:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_sessions": 8, 2024-04-21T08:52:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_open": 11, 2024-04-21T08:52:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_close": 3, 2024-04-21T08:52:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_laggy": 0, 2024-04-21T08:52:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_wr": 14, 2024-04-21T08:52:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_rd": 0, 2024-04-21T08:52:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_del": 1 2024-04-21T08:52:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "oft": { 2024-04-21T08:52:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_objs": 1, 2024-04-21T08:52:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:52:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_updates": 0, 2024-04-21T08:52:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_removes": 0 2024-04-21T08:52:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "osd-slow-ops": { 2024-04-21T08:52:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_ops_count": 18 2024-04-21T08:52:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "purge_queue": { 2024-04-21T08:52:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops": 0, 2024-04-21T08:52:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T08:52:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing": 0, 2024-04-21T08:52:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_high_water": 0, 2024-04-21T08:52:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed_ops": 0, 2024-04-21T08:52:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed": 0, 2024-04-21T08:52:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_item_in_journal": 0 2024-04-21T08:52:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:52:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-21T08:52:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-21T08:52:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-21T08:52:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 2155, 2024-04-21T08:52:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 2008328, 2024-04-21T08:52:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 2155, 2024-04-21T08:52:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-21T08:52:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-21T08:52:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 2155, 2024-04-21T08:52:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 2008328, 2024-04-21T08:52:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-21T08:52:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_bytes": { 2024-04-21T08:52:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-21T08:52:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-21T08:52:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-21T08:52:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-21T08:52:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-21T08:52:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 42, 2024-04-21T08:52:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 12211, 2024-04-21T08:52:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 42, 2024-04-21T08:52:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 12211, 2024-04-21T08:52:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-21T08:52:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_ops": { 2024-04-21T08:52:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-21T08:52:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 1024, 2024-04-21T08:52:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-21T08:52:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-21T08:52:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-21T08:52:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 42, 2024-04-21T08:52:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 42, 2024-04-21T08:52:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 42, 2024-04-21T08:52:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 42, 2024-04-21T08:52:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-21T08:52:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:52:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-21T08:52:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-04-21T08:52:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-21T08:52:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-21T08:52:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-21T08:52:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-21T08:52:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-21T08:52:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 0, 2024-04-21T08:52:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 0, 2024-04-21T08:52:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-21T08:52:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-21T08:52:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle-0x55cc39bc0500": { 2024-04-21T08:52:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-21T08:52:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-04-21T08:52:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-21T08:52:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 6, 2024-04-21T08:52:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 1819, 2024-04-21T08:52:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 6, 2024-04-21T08:52:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-21T08:52:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-21T08:52:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 5, 2024-04-21T08:52:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1819, 2024-04-21T08:52:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-21T08:52:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-21T08:52:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-21T08:52:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-21T08:52:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-21T08:52:11.575 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-21T08:52:12.356 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-21T08:52:12.356 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-21T08:52:12.356 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-04-21T08:52:12.356 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=18 2024-04-21T08:52:12.356 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2024-04-21T08:52:12.356 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:52:12.356 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:52:12.356 DEBUG:tasks.check_counter:Getting stats from c 2024-04-21T08:52:12.356 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-21T08:52:12.608 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:52:12.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:12 smithi175 ceph-mon[31920]: pgmap v415: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-21T08:52:13.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:12 smithi112 ceph-mon[31085]: pgmap v415: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-21T08:52:13.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:12 smithi043 ceph-mon[27978]: pgmap v415: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-21T08:52:13.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:13 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:52:13.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:13 smithi175 ceph-mon[31920]: pgmap v416: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-21T08:52:13.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:13 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:52:13.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:13 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:52:13.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:13 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:52:14.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:13 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:52:14.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:13 smithi112 ceph-mon[31085]: pgmap v416: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-21T08:52:14.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:13 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:52:14.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:13 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:52:14.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:13 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:52:14.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:13 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:52:14.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:13 smithi043 ceph-mon[27978]: pgmap v416: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-21T08:52:14.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:13 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:52:14.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:13 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:52:14.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:13 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' 2024-04-21T08:52:14.422 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-04-21T08:52:14.422 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:52:14.423 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 641, 2024-04-21T08:52:14.423 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 643, 2024-04-21T08:52:14.423 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 4389268, 2024-04-21T08:52:14.423 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 203848, 2024-04-21T08:52:14.423 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 6, 2024-04-21T08:52:14.423 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 2, 2024-04-21T08:52:14.423 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.078502193, 2024-04-21T08:52:14.423 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.020820653, 2024-04-21T08:52:14.423 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.024815751, 2024-04-21T08:52:14.423 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.025657366, 2024-04-21T08:52:14.423 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:14.423 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 641, 2024-04-21T08:52:14.423 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.028228801, 2024-04-21T08:52:14.424 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000044038 2024-04-21T08:52:14.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.424 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:14.424 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.424 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.424 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.424 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T08:52:14.424 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T08:52:14.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.424 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:52:14.424 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 655, 2024-04-21T08:52:14.424 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 652, 2024-04-21T08:52:14.424 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 8039755, 2024-04-21T08:52:14.424 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 233284, 2024-04-21T08:52:14.425 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 4, 2024-04-21T08:52:14.425 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 3, 2024-04-21T08:52:14.425 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.069325994, 2024-04-21T08:52:14.425 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.021371607, 2024-04-21T08:52:14.425 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.026285791, 2024-04-21T08:52:14.425 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.017319151, 2024-04-21T08:52:14.425 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:14.425 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 652, 2024-04-21T08:52:14.425 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.027788457, 2024-04-21T08:52:14.425 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000042620 2024-04-21T08:52:14.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.425 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:14.425 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.425 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.426 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.426 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.426 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 199376, 2024-04-21T08:52:14.426 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 63088 2024-04-21T08:52:14.426 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.426 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:52:14.426 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 226, 2024-04-21T08:52:14.426 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 300, 2024-04-21T08:52:14.426 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 1962023, 2024-04-21T08:52:14.426 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 288335, 2024-04-21T08:52:14.426 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 5, 2024-04-21T08:52:14.427 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 4, 2024-04-21T08:52:14.427 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.027760785, 2024-04-21T08:52:14.427 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.009400248, 2024-04-21T08:52:14.427 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.008517434, 2024-04-21T08:52:14.427 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.007079756, 2024-04-21T08:52:14.427 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:14.427 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 298, 2024-04-21T08:52:14.427 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.013357785, 2024-04-21T08:52:14.427 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000044824 2024-04-21T08:52:14.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.427 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:14.427 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.427 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.427 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.428 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-21T08:52:14.428 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 216784 2024-04-21T08:52:14.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.428 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2024-04-21T08:52:14.428 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2024-04-21T08:52:14.428 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2024-04-21T08:52:14.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.428 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2024-04-21T08:52:14.428 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-04-21T08:52:14.428 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-04-21T08:52:14.428 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2402, 2024-04-21T08:52:14.428 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.070848022, 2024-04-21T08:52:14.428 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000029495 2024-04-21T08:52:14.429 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:14.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.429 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2024-04-21T08:52:14.429 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-04-21T08:52:14.429 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-04-21T08:52:14.429 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.429 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.429 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.429 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:14.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.429 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2024-04-21T08:52:14.429 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 0, 2024-04-21T08:52:14.429 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 0, 2024-04-21T08:52:14.430 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2024-04-21T08:52:14.430 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.430 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.430 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.430 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_reply": 0, 2024-04-21T08:52:14.430 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 0, 2024-04-21T08:52:14.430 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_complete": 0, 2024-04-21T08:52:14.430 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_keys": 0, 2024-04-21T08:52:14.430 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 0, 2024-04-21T08:52:14.430 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 0, 2024-04-21T08:52:14.430 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 0, 2024-04-21T08:52:14.430 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 19, 2024-04-21T08:52:14.430 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 4, 2024-04-21T08:52:14.431 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 15, 2024-04-21T08:52:14.431 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 0, 2024-04-21T08:52:14.431 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 13, 2024-04-21T08:52:14.431 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 0, 2024-04-21T08:52:14.431 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 0, 2024-04-21T08:52:14.431 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 0, 2024-04-21T08:52:14.431 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 2, 2024-04-21T08:52:14.431 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 0, 2024-04-21T08:52:14.431 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 0, 2024-04-21T08:52:14.431 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 0, 2024-04-21T08:52:14.431 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 0, 2024-04-21T08:52:14.431 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:52:14.431 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2024-04-21T08:52:14.431 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 0, 2024-04-21T08:52:14.431 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 0, 2024-04-21T08:52:14.431 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 0, 2024-04-21T08:52:14.432 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 0, 2024-04-21T08:52:14.432 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 0, 2024-04-21T08:52:14.432 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 0, 2024-04-21T08:52:14.432 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 0, 2024-04-21T08:52:14.432 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2024-04-21T08:52:14.432 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 0, 2024-04-21T08:52:14.432 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 0, 2024-04-21T08:52:14.432 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 4, 2024-04-21T08:52:14.432 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 234, 2024-04-21T08:52:14.432 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 6, 2024-04-21T08:52:14.432 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T08:52:14.432 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_set_tag": 0, 2024-04-21T08:52:14.432 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:52:14.432 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:52:14.432 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:52:14.433 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:52:14.433 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:52:14.433 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_file_inodes": 0, 2024-04-21T08:52:14.433 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:52:14.433 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T08:52:14.433 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 0, 2024-04-21T08:52:14.433 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:52:14.433 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:52:14.433 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:52:14.433 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 0, 2024-04-21T08:52:14.433 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 0, 2024-04-21T08:52:14.433 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 0, 2024-04-21T08:52:14.433 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 0 2024-04-21T08:52:14.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.433 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2024-04-21T08:52:14.434 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update": 0, 2024-04-21T08:52:14.434 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update_receipt": 0, 2024-04-21T08:52:14.434 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_try_discover": 0, 2024-04-21T08:52:14.434 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_send_discover": 0, 2024-04-21T08:52:14.434 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_handle_discover": 0, 2024-04-21T08:52:14.434 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 0, 2024-04-21T08:52:14.434 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2024-04-21T08:52:14.434 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:52:14.434 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 0, 2024-04-21T08:52:14.434 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 0, 2024-04-21T08:52:14.434 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 0, 2024-04-21T08:52:14.434 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2024-04-21T08:52:14.434 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2024-04-21T08:52:14.434 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:52:14.435 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:52:14.435 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2024-04-21T08:52:14.435 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2024-04-21T08:52:14.435 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_path": 0, 2024-04-21T08:52:14.435 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_inode": 0, 2024-04-21T08:52:14.435 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:52:14.435 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 0, 2024-04-21T08:52:14.435 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2024-04-21T08:52:14.435 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:52:14.435 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2024-04-21T08:52:14.435 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2024-04-21T08:52:14.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.435 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2024-04-21T08:52:14.436 INFO:teuthology.orchestra.run.smithi175.stdout: "evlrg": 0, 2024-04-21T08:52:14.436 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 0, 2024-04-21T08:52:14.436 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 0, 2024-04-21T08:52:14.436 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 0, 2024-04-21T08:52:14.436 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 1297, 2024-04-21T08:52:14.436 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2024-04-21T08:52:14.436 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 0, 2024-04-21T08:52:14.436 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 0, 2024-04-21T08:52:14.436 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 0, 2024-04-21T08:52:14.436 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 0, 2024-04-21T08:52:14.436 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 2, 2024-04-21T08:52:14.436 INFO:teuthology.orchestra.run.smithi175.stdout: "segmjr": 25, 2024-04-21T08:52:14.436 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2024-04-21T08:52:14.437 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 0, 2024-04-21T08:52:14.437 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 26659424, 2024-04-21T08:52:14.437 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 27070296, 2024-04-21T08:52:14.437 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 27070160, 2024-04-21T08:52:14.437 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2024-04-21T08:52:14.437 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.437 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.437 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.437 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 1297 2024-04-21T08:52:14.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.438 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2024-04-21T08:52:14.438 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 16, 2024-04-21T08:52:14.438 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 17, 2024-04-21T08:52:14.438 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 1, 2024-04-21T08:52:14.438 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 9, 2024-04-21T08:52:14.438 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 9, 2024-04-21T08:52:14.438 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 0, 2024-04-21T08:52:14.438 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 19, 2024-04-21T08:52:14.438 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 19, 2024-04-21T08:52:14.438 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 0, 2024-04-21T08:52:14.438 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 0, 2024-04-21T08:52:14.438 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 0, 2024-04-21T08:52:14.439 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 0, 2024-04-21T08:52:14.439 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 52368, 2024-04-21T08:52:14.439 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 215416 2024-04-21T08:52:14.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.439 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2024-04-21T08:52:14.439 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 0, 2024-04-21T08:52:14.439 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 0, 2024-04-21T08:52:14.439 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 0, 2024-04-21T08:52:14.439 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 0, 2024-04-21T08:52:14.439 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 0, 2024-04-21T08:52:14.439 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2024-04-21T08:52:14.439 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.439 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.439 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.440 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2024-04-21T08:52:14.440 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.440 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.440 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.440 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2024-04-21T08:52:14.440 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.440 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.440 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.440 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2024-04-21T08:52:14.440 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.440 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.441 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.441 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2024-04-21T08:52:14.441 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.441 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.441 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.441 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2024-04-21T08:52:14.441 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.441 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.441 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.441 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2024-04-21T08:52:14.441 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.441 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.442 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.442 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2024-04-21T08:52:14.442 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.442 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.442 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.442 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2024-04-21T08:52:14.442 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.442 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.442 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.443 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2024-04-21T08:52:14.443 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.443 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.443 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.443 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2024-04-21T08:52:14.443 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.443 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.443 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.443 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2024-04-21T08:52:14.443 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.444 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.444 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.445 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2024-04-21T08:52:14.445 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.445 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.445 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.445 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2024-04-21T08:52:14.445 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.445 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.445 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.445 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2024-04-21T08:52:14.445 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.445 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.445 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.446 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2024-04-21T08:52:14.446 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.446 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.446 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.446 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2024-04-21T08:52:14.446 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.446 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.446 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.446 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2024-04-21T08:52:14.447 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.447 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.447 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.447 INFO:teuthology.orchestra.run.smithi175.stdout: "req_snapdiff_latency": { 2024-04-21T08:52:14.447 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.447 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.447 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.447 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2024-04-21T08:52:14.447 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.447 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.447 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.448 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2024-04-21T08:52:14.448 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.448 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.448 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.448 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2024-04-21T08:52:14.448 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.448 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.448 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.448 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2024-04-21T08:52:14.448 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.448 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.448 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.449 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2024-04-21T08:52:14.449 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.449 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.449 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.449 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2024-04-21T08:52:14.449 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.449 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.449 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.449 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2024-04-21T08:52:14.449 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.449 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.450 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.450 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2024-04-21T08:52:14.450 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.450 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.450 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.450 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2024-04-21T08:52:14.450 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.450 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.450 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.450 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2024-04-21T08:52:14.450 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.450 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.451 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.451 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:52:14.451 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:52:14.451 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getvxattr_latency": { 2024-04-21T08:52:14.451 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.451 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.451 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.451 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:14.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.451 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2024-04-21T08:52:14.451 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 2, 2024-04-21T08:52:14.451 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 2, 2024-04-21T08:52:14.451 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 0, 2024-04-21T08:52:14.451 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 0, 2024-04-21T08:52:14.452 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2024-04-21T08:52:14.452 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 0, 2024-04-21T08:52:14.452 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 0, 2024-04-21T08:52:14.452 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 1977, 2024-04-21T08:52:14.452 INFO:teuthology.orchestra.run.smithi175.stdout: "mdthresh_evicted": 0 2024-04-21T08:52:14.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.452 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2024-04-21T08:52:14.452 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:52:14.452 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2024-04-21T08:52:14.452 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:52:14.452 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:52:14.452 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:52:14.452 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:52:14.452 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:52:14.453 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:52:14.453 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:52:14.453 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:52:14.453 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:52:14.453 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:52:14.453 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:52:14.453 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:52:14.453 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:52:14.453 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2024-04-21T08:52:14.453 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:52:14.453 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2024-04-21T08:52:14.453 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:52:14.453 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:52:14.453 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:52:14.453 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:52:14.454 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:52:14.454 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:52:14.454 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:52:14.454 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2024-04-21T08:52:14.454 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:52:14.454 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:52:14.454 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:52:14.454 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2024-04-21T08:52:14.454 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2024-04-21T08:52:14.454 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2024-04-21T08:52:14.454 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:52:14.454 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:52:14.454 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:52:14.454 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:52:14.455 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 198037, 2024-04-21T08:52:14.455 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 72, 2024-04-21T08:52:14.455 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:52:14.455 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2024-04-21T08:52:14.455 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2024-04-21T08:52:14.455 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2024-04-21T08:52:14.455 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:52:14.455 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2024-04-21T08:52:14.455 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:52:14.455 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2024-04-21T08:52:14.455 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 10040, 2024-04-21T08:52:14.455 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 194, 2024-04-21T08:52:14.455 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:52:14.456 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:52:14.456 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2024-04-21T08:52:14.456 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2024-04-21T08:52:14.456 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 97627, 2024-04-21T08:52:14.456 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 788, 2024-04-21T08:52:14.456 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2024-04-21T08:52:14.456 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2024-04-21T08:52:14.456 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2024-04-21T08:52:14.456 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2024-04-21T08:52:14.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.456 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2024-04-21T08:52:14.456 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2024-04-21T08:52:14.456 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2024-04-21T08:52:14.456 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 1401, 2024-04-21T08:52:14.457 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 26, 2024-04-21T08:52:14.457 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 4, 2024-04-21T08:52:14.457 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 1401, 2024-04-21T08:52:14.457 INFO:teuthology.orchestra.run.smithi175.stdout: "op_latency": { 2024-04-21T08:52:14.457 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1401, 2024-04-21T08:52:14.457 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.864985330, 2024-04-21T08:52:14.457 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000617405 2024-04-21T08:52:14.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.457 INFO:teuthology.orchestra.run.smithi175.stdout: "op_inflight": 0, 2024-04-21T08:52:14.457 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2024-04-21T08:52:14.457 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1401, 2024-04-21T08:52:14.457 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 1402 2024-04-21T08:52:14.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.457 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 1401, 2024-04-21T08:52:14.457 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 1401, 2024-04-21T08:52:14.458 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 0, 2024-04-21T08:52:14.458 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2024-04-21T08:52:14.458 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2024-04-21T08:52:14.458 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 714, 2024-04-21T08:52:14.458 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 0, 2024-04-21T08:52:14.458 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 684, 2024-04-21T08:52:14.458 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 0, 2024-04-21T08:52:14.458 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 0, 2024-04-21T08:52:14.458 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2024-04-21T08:52:14.458 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2024-04-21T08:52:14.458 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2024-04-21T08:52:14.458 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 0, 2024-04-21T08:52:14.458 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 0, 2024-04-21T08:52:14.458 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2024-04-21T08:52:14.459 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2024-04-21T08:52:14.459 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2024-04-21T08:52:14.459 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 2, 2024-04-21T08:52:14.459 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 0, 2024-04-21T08:52:14.459 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:52:14.459 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2024-04-21T08:52:14.459 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:52:14.459 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2024-04-21T08:52:14.459 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2024-04-21T08:52:14.459 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2024-04-21T08:52:14.459 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:52:14.459 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2024-04-21T08:52:14.459 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:52:14.459 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2024-04-21T08:52:14.459 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2024-04-21T08:52:14.460 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2024-04-21T08:52:14.460 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2024-04-21T08:52:14.460 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2024-04-21T08:52:14.460 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2024-04-21T08:52:14.460 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2024-04-21T08:52:14.460 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2024-04-21T08:52:14.460 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2024-04-21T08:52:14.460 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2024-04-21T08:52:14.460 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2024-04-21T08:52:14.460 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2024-04-21T08:52:14.460 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2024-04-21T08:52:14.461 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2024-04-21T08:52:14.461 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2024-04-21T08:52:14.461 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2024-04-21T08:52:14.461 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2024-04-21T08:52:14.461 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 109, 2024-04-21T08:52:14.461 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2024-04-21T08:52:14.461 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 11, 2024-04-21T08:52:14.461 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 6, 2024-04-21T08:52:14.461 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 6, 2024-04-21T08:52:14.461 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 0, 2024-04-21T08:52:14.461 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2024-04-21T08:52:14.461 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 0, 2024-04-21T08:52:14.461 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 2, 2024-04-21T08:52:14.462 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 0 2024-04-21T08:52:14.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.462 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2024-04-21T08:52:14.462 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 0, 2024-04-21T08:52:14.462 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:52:14.462 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2024-04-21T08:52:14.462 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2024-04-21T08:52:14.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.462 INFO:teuthology.orchestra.run.smithi175.stdout: "osd-slow-ops": { 2024-04-21T08:52:14.462 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_ops_count": 0 2024-04-21T08:52:14.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.463 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2024-04-21T08:52:14.463 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2024-04-21T08:52:14.463 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T08:52:14.463 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2024-04-21T08:52:14.463 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 0, 2024-04-21T08:52:14.463 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed_ops": 0, 2024-04-21T08:52:14.463 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 0, 2024-04-21T08:52:14.463 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2024-04-21T08:52:14.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.463 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:52:14.463 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-04-21T08:52:14.463 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-04-21T08:52:14.464 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-04-21T08:52:14.464 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 1518, 2024-04-21T08:52:14.464 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 14245501, 2024-04-21T08:52:14.464 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:14.464 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 1518, 2024-04-21T08:52:14.464 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-04-21T08:52:14.464 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-04-21T08:52:14.464 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 1518, 2024-04-21T08:52:14.464 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 14245501, 2024-04-21T08:52:14.464 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-04-21T08:52:14.464 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.464 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.465 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.465 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:14.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.465 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2024-04-21T08:52:14.465 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-04-21T08:52:14.465 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-04-21T08:52:14.465 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-04-21T08:52:14.465 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-04-21T08:52:14.465 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-04-21T08:52:14.465 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:14.465 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:14.465 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 1401, 2024-04-21T08:52:14.465 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 13803020, 2024-04-21T08:52:14.466 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 1401, 2024-04-21T08:52:14.466 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 13803020, 2024-04-21T08:52:14.466 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-04-21T08:52:14.466 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.466 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.466 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.466 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:14.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.466 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2024-04-21T08:52:14.466 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-04-21T08:52:14.466 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2024-04-21T08:52:14.466 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-04-21T08:52:14.466 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-04-21T08:52:14.466 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-04-21T08:52:14.467 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:14.467 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:14.467 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 1401, 2024-04-21T08:52:14.467 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 1401, 2024-04-21T08:52:14.467 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 1401, 2024-04-21T08:52:14.467 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 1401, 2024-04-21T08:52:14.467 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-04-21T08:52:14.467 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.467 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.467 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.467 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:14.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.467 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:52:14.467 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-04-21T08:52:14.468 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-04-21T08:52:14.468 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-04-21T08:52:14.468 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-04-21T08:52:14.468 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-04-21T08:52:14.468 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:14.468 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:14.468 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-04-21T08:52:14.468 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-04-21T08:52:14.468 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2024-04-21T08:52:14.468 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2024-04-21T08:52:14.468 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-04-21T08:52:14.468 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.468 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.468 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.468 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:14.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:14.469 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x563e807746e0": { 2024-04-21T08:52:14.469 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-04-21T08:52:14.469 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-04-21T08:52:14.469 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-04-21T08:52:14.469 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-04-21T08:52:14.469 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-04-21T08:52:14.469 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:14.469 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:14.469 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-04-21T08:52:14.469 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-04-21T08:52:14.469 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2024-04-21T08:52:14.469 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2024-04-21T08:52:14.470 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-04-21T08:52:14.470 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:14.470 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:14.470 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:14.470 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:14.470 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:14.470 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-04-21T08:52:15.230 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-21T08:52:15.230 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-21T08:52:15.230 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-21T08:52:15.230 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-04-21T08:52:15.230 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-21T08:52:15.230 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:52:15.230 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:52:15.230 DEBUG:tasks.check_counter:Getting stats from f 2024-04-21T08:52:15.231 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-21T08:52:15.489 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:52:16.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:16 smithi175 ceph-mon[31920]: osdmap e110: 12 total, 12 up, 12 in 2024-04-21T08:52:16.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:16 smithi175 ceph-mon[31920]: pgmap v418: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-21T08:52:16.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:16 smithi112 ceph-mon[31085]: osdmap e110: 12 total, 12 up, 12 in 2024-04-21T08:52:16.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:16 smithi112 ceph-mon[31085]: pgmap v418: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-21T08:52:16.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:16 smithi043 ceph-mon[27978]: osdmap e110: 12 total, 12 up, 12 in 2024-04-21T08:52:16.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:16 smithi043 ceph-mon[27978]: pgmap v418: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-21T08:52:17.206 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-04-21T08:52:17.207 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:52:17.207 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 305, 2024-04-21T08:52:17.207 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 286, 2024-04-21T08:52:17.207 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 1681874, 2024-04-21T08:52:17.207 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 165076, 2024-04-21T08:52:17.207 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 6, 2024-04-21T08:52:17.207 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 1, 2024-04-21T08:52:17.207 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.058503887, 2024-04-21T08:52:17.207 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.018768012, 2024-04-21T08:52:17.207 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.040873814, 2024-04-21T08:52:17.207 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.006925283, 2024-04-21T08:52:17.207 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:17.207 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 285, 2024-04-21T08:52:17.208 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.017159746, 2024-04-21T08:52:17.208 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000060209 2024-04-21T08:52:17.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.208 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:17.208 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.208 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.208 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.208 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 1681040, 2024-04-21T08:52:17.208 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 164752 2024-04-21T08:52:17.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.209 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:52:17.209 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 850, 2024-04-21T08:52:17.209 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 1046, 2024-04-21T08:52:17.209 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 210958, 2024-04-21T08:52:17.209 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 777645, 2024-04-21T08:52:17.209 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 7, 2024-04-21T08:52:17.209 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 5, 2024-04-21T08:52:17.209 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.100540875, 2024-04-21T08:52:17.209 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.031736553, 2024-04-21T08:52:17.209 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.023319334, 2024-04-21T08:52:17.209 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.035099527, 2024-04-21T08:52:17.209 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:17.209 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1044, 2024-04-21T08:52:17.210 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.046068423, 2024-04-21T08:52:17.210 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000044126 2024-04-21T08:52:17.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.210 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:17.210 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.210 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.210 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.210 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-21T08:52:17.210 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 509248 2024-04-21T08:52:17.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.210 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:52:17.211 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 1860, 2024-04-21T08:52:17.211 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 1862, 2024-04-21T08:52:17.211 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 557858, 2024-04-21T08:52:17.211 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 591538, 2024-04-21T08:52:17.211 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 4, 2024-04-21T08:52:17.211 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 3, 2024-04-21T08:52:17.211 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.195799003, 2024-04-21T08:52:17.211 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.062768023, 2024-04-21T08:52:17.211 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.051143182, 2024-04-21T08:52:17.211 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.063417441, 2024-04-21T08:52:17.211 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:17.211 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1860, 2024-04-21T08:52:17.212 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.079236281, 2024-04-21T08:52:17.212 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000042600 2024-04-21T08:52:17.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.212 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:17.212 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.212 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.212 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.212 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T08:52:17.212 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T08:52:17.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.212 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2024-04-21T08:52:17.212 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2024-04-21T08:52:17.213 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2024-04-21T08:52:17.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.213 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2024-04-21T08:52:17.213 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-04-21T08:52:17.213 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-04-21T08:52:17.213 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 4417, 2024-04-21T08:52:17.213 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.137369988, 2024-04-21T08:52:17.213 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000031100 2024-04-21T08:52:17.213 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:17.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.213 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2024-04-21T08:52:17.213 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-04-21T08:52:17.213 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-04-21T08:52:17.213 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.214 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.214 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.214 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:17.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.214 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2024-04-21T08:52:17.214 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 0, 2024-04-21T08:52:17.214 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 0, 2024-04-21T08:52:17.214 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2024-04-21T08:52:17.214 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.214 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.214 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.214 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_reply": 0, 2024-04-21T08:52:17.215 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 0, 2024-04-21T08:52:17.215 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_complete": 0, 2024-04-21T08:52:17.215 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_keys": 0, 2024-04-21T08:52:17.215 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 0, 2024-04-21T08:52:17.215 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 0, 2024-04-21T08:52:17.215 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 0, 2024-04-21T08:52:17.215 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 0, 2024-04-21T08:52:17.215 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 0, 2024-04-21T08:52:17.215 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 0, 2024-04-21T08:52:17.215 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 0, 2024-04-21T08:52:17.215 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 0, 2024-04-21T08:52:17.215 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 0, 2024-04-21T08:52:17.215 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 0, 2024-04-21T08:52:17.216 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 0, 2024-04-21T08:52:17.216 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 1, 2024-04-21T08:52:17.216 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 0, 2024-04-21T08:52:17.216 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 0, 2024-04-21T08:52:17.216 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 0, 2024-04-21T08:52:17.216 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 0, 2024-04-21T08:52:17.216 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:52:17.216 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2024-04-21T08:52:17.216 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 0, 2024-04-21T08:52:17.216 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 0, 2024-04-21T08:52:17.216 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 0, 2024-04-21T08:52:17.216 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 0, 2024-04-21T08:52:17.217 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 0, 2024-04-21T08:52:17.217 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 0, 2024-04-21T08:52:17.217 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 0, 2024-04-21T08:52:17.217 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2024-04-21T08:52:17.217 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 0, 2024-04-21T08:52:17.217 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 0, 2024-04-21T08:52:17.217 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 0, 2024-04-21T08:52:17.217 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 0, 2024-04-21T08:52:17.217 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 0, 2024-04-21T08:52:17.217 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T08:52:17.217 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_set_tag": 0, 2024-04-21T08:52:17.217 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:52:17.217 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:52:17.218 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:52:17.218 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:52:17.218 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:52:17.218 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_file_inodes": 0, 2024-04-21T08:52:17.218 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:52:17.218 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T08:52:17.218 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 0, 2024-04-21T08:52:17.218 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:52:17.218 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:52:17.218 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:52:17.218 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 0, 2024-04-21T08:52:17.218 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 0, 2024-04-21T08:52:17.218 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 0, 2024-04-21T08:52:17.219 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 0 2024-04-21T08:52:17.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.219 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2024-04-21T08:52:17.219 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update": 0, 2024-04-21T08:52:17.219 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update_receipt": 0, 2024-04-21T08:52:17.219 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_try_discover": 0, 2024-04-21T08:52:17.219 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_send_discover": 0, 2024-04-21T08:52:17.219 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_handle_discover": 0, 2024-04-21T08:52:17.219 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 0, 2024-04-21T08:52:17.219 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2024-04-21T08:52:17.219 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:52:17.219 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 0, 2024-04-21T08:52:17.219 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 0, 2024-04-21T08:52:17.219 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 0, 2024-04-21T08:52:17.220 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2024-04-21T08:52:17.220 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2024-04-21T08:52:17.220 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:52:17.220 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:52:17.220 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2024-04-21T08:52:17.220 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2024-04-21T08:52:17.220 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_path": 0, 2024-04-21T08:52:17.220 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_inode": 0, 2024-04-21T08:52:17.220 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:52:17.220 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 0, 2024-04-21T08:52:17.220 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2024-04-21T08:52:17.220 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:52:17.220 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2024-04-21T08:52:17.220 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2024-04-21T08:52:17.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.221 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2024-04-21T08:52:17.221 INFO:teuthology.orchestra.run.smithi175.stdout: "evlrg": 0, 2024-04-21T08:52:17.221 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 0, 2024-04-21T08:52:17.221 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 0, 2024-04-21T08:52:17.221 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 0, 2024-04-21T08:52:17.221 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 7, 2024-04-21T08:52:17.221 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2024-04-21T08:52:17.221 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 0, 2024-04-21T08:52:17.221 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 0, 2024-04-21T08:52:17.221 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 0, 2024-04-21T08:52:17.221 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 0, 2024-04-21T08:52:17.221 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 2, 2024-04-21T08:52:17.221 INFO:teuthology.orchestra.run.smithi175.stdout: "segmjr": 2, 2024-04-21T08:52:17.222 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2024-04-21T08:52:17.222 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 0, 2024-04-21T08:52:17.222 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 4194304, 2024-04-21T08:52:17.222 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 4196347, 2024-04-21T08:52:17.222 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 4196206, 2024-04-21T08:52:17.222 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2024-04-21T08:52:17.222 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.222 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.222 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.222 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 7 2024-04-21T08:52:17.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.223 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2024-04-21T08:52:17.223 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 3, 2024-04-21T08:52:17.223 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 3, 2024-04-21T08:52:17.223 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 0, 2024-04-21T08:52:17.223 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 1, 2024-04-21T08:52:17.223 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 1, 2024-04-21T08:52:17.223 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 0, 2024-04-21T08:52:17.223 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 0, 2024-04-21T08:52:17.223 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 0, 2024-04-21T08:52:17.223 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 0, 2024-04-21T08:52:17.223 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 0, 2024-04-21T08:52:17.223 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 0, 2024-04-21T08:52:17.223 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 0, 2024-04-21T08:52:17.224 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 52172, 2024-04-21T08:52:17.224 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 215416 2024-04-21T08:52:17.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.224 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2024-04-21T08:52:17.224 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 0, 2024-04-21T08:52:17.224 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 0, 2024-04-21T08:52:17.224 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 0, 2024-04-21T08:52:17.224 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 0, 2024-04-21T08:52:17.224 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 0, 2024-04-21T08:52:17.224 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2024-04-21T08:52:17.224 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.224 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.224 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.224 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2024-04-21T08:52:17.225 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.225 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.225 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.225 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2024-04-21T08:52:17.225 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.225 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.225 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.225 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2024-04-21T08:52:17.225 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.225 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.225 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.226 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2024-04-21T08:52:17.226 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.226 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.226 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.226 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2024-04-21T08:52:17.226 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.226 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.226 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.226 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2024-04-21T08:52:17.227 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.227 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.227 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.227 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2024-04-21T08:52:17.227 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.227 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.227 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.227 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2024-04-21T08:52:17.227 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.227 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.227 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.228 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2024-04-21T08:52:17.228 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.228 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.228 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.228 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2024-04-21T08:52:17.228 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.228 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.228 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.228 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2024-04-21T08:52:17.228 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.229 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.229 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.229 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2024-04-21T08:52:17.229 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.229 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.229 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.230 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2024-04-21T08:52:17.230 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.230 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.230 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.230 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2024-04-21T08:52:17.230 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.230 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.230 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.231 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2024-04-21T08:52:17.231 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.231 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.231 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.231 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2024-04-21T08:52:17.231 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.231 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.231 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.231 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2024-04-21T08:52:17.231 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.231 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.232 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.232 INFO:teuthology.orchestra.run.smithi175.stdout: "req_snapdiff_latency": { 2024-04-21T08:52:17.232 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.232 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.232 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.232 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2024-04-21T08:52:17.232 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.232 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.232 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.232 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2024-04-21T08:52:17.233 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.233 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.233 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.233 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2024-04-21T08:52:17.233 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.233 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.233 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.233 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2024-04-21T08:52:17.233 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.233 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.233 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.234 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2024-04-21T08:52:17.234 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.234 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.234 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.234 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2024-04-21T08:52:17.234 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.234 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.234 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.234 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2024-04-21T08:52:17.234 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.235 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.235 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.235 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2024-04-21T08:52:17.235 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.235 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.235 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.235 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2024-04-21T08:52:17.235 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.235 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.235 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.235 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2024-04-21T08:52:17.236 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.236 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.236 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.236 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:52:17.236 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:52:17.236 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getvxattr_latency": { 2024-04-21T08:52:17.236 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.236 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.236 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.236 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:17.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.236 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2024-04-21T08:52:17.237 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 1, 2024-04-21T08:52:17.237 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 3, 2024-04-21T08:52:17.237 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 2, 2024-04-21T08:52:17.237 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 2, 2024-04-21T08:52:17.237 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2024-04-21T08:52:17.237 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 0, 2024-04-21T08:52:17.237 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 0, 2024-04-21T08:52:17.237 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 3163, 2024-04-21T08:52:17.237 INFO:teuthology.orchestra.run.smithi175.stdout: "mdthresh_evicted": 0 2024-04-21T08:52:17.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.237 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2024-04-21T08:52:17.237 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:52:17.237 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2024-04-21T08:52:17.237 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:52:17.238 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:52:17.238 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:52:17.238 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:52:17.238 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:52:17.238 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:52:17.238 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:52:17.238 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:52:17.238 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:52:17.238 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:52:17.238 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:52:17.238 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:52:17.238 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:52:17.239 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2024-04-21T08:52:17.239 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:52:17.239 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2024-04-21T08:52:17.239 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:52:17.239 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:52:17.239 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:52:17.239 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:52:17.239 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:52:17.239 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:52:17.239 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:52:17.239 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2024-04-21T08:52:17.239 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:52:17.239 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:52:17.240 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:52:17.240 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2024-04-21T08:52:17.240 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2024-04-21T08:52:17.240 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2024-04-21T08:52:17.240 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:52:17.240 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:52:17.240 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:52:17.240 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:52:17.240 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 189423, 2024-04-21T08:52:17.240 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 51, 2024-04-21T08:52:17.240 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:52:17.241 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2024-04-21T08:52:17.241 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2024-04-21T08:52:17.241 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2024-04-21T08:52:17.241 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:52:17.241 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2024-04-21T08:52:17.241 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:52:17.241 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2024-04-21T08:52:17.241 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 9800, 2024-04-21T08:52:17.241 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 188, 2024-04-21T08:52:17.241 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:52:17.241 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:52:17.241 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2024-04-21T08:52:17.241 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2024-04-21T08:52:17.241 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 8512, 2024-04-21T08:52:17.242 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 13, 2024-04-21T08:52:17.242 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2024-04-21T08:52:17.242 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2024-04-21T08:52:17.242 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2024-04-21T08:52:17.242 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2024-04-21T08:52:17.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.242 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2024-04-21T08:52:17.242 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2024-04-21T08:52:17.242 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2024-04-21T08:52:17.242 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 2705, 2024-04-21T08:52:17.242 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 21, 2024-04-21T08:52:17.242 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 3, 2024-04-21T08:52:17.243 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 2705, 2024-04-21T08:52:17.243 INFO:teuthology.orchestra.run.smithi175.stdout: "op_latency": { 2024-04-21T08:52:17.243 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2705, 2024-04-21T08:52:17.243 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 1.483974898, 2024-04-21T08:52:17.243 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000548604 2024-04-21T08:52:17.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.243 INFO:teuthology.orchestra.run.smithi175.stdout: "op_inflight": 0, 2024-04-21T08:52:17.243 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2024-04-21T08:52:17.243 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2705, 2024-04-21T08:52:17.243 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 2706 2024-04-21T08:52:17.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.243 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 2705, 2024-04-21T08:52:17.243 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 2705, 2024-04-21T08:52:17.244 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 0, 2024-04-21T08:52:17.244 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2024-04-21T08:52:17.244 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2024-04-21T08:52:17.244 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 1768, 2024-04-21T08:52:17.244 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 0, 2024-04-21T08:52:17.244 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 935, 2024-04-21T08:52:17.244 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 0, 2024-04-21T08:52:17.244 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 0, 2024-04-21T08:52:17.244 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2024-04-21T08:52:17.244 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2024-04-21T08:52:17.244 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2024-04-21T08:52:17.244 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 0, 2024-04-21T08:52:17.244 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 0, 2024-04-21T08:52:17.244 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2024-04-21T08:52:17.245 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2024-04-21T08:52:17.245 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2024-04-21T08:52:17.245 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 1, 2024-04-21T08:52:17.245 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 0, 2024-04-21T08:52:17.245 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:52:17.245 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2024-04-21T08:52:17.245 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:52:17.245 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2024-04-21T08:52:17.245 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2024-04-21T08:52:17.245 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2024-04-21T08:52:17.245 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:52:17.245 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2024-04-21T08:52:17.245 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:52:17.246 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2024-04-21T08:52:17.246 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2024-04-21T08:52:17.246 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2024-04-21T08:52:17.246 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2024-04-21T08:52:17.246 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2024-04-21T08:52:17.246 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2024-04-21T08:52:17.246 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2024-04-21T08:52:17.246 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2024-04-21T08:52:17.246 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2024-04-21T08:52:17.246 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2024-04-21T08:52:17.246 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2024-04-21T08:52:17.247 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2024-04-21T08:52:17.247 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2024-04-21T08:52:17.247 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2024-04-21T08:52:17.247 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2024-04-21T08:52:17.247 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2024-04-21T08:52:17.247 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2024-04-21T08:52:17.247 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 110, 2024-04-21T08:52:17.247 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2024-04-21T08:52:17.247 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 20, 2024-04-21T08:52:17.247 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 5, 2024-04-21T08:52:17.247 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 6, 2024-04-21T08:52:17.248 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 1, 2024-04-21T08:52:17.248 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2024-04-21T08:52:17.248 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 0, 2024-04-21T08:52:17.248 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 2, 2024-04-21T08:52:17.248 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 0 2024-04-21T08:52:17.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.248 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2024-04-21T08:52:17.248 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 0, 2024-04-21T08:52:17.248 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:52:17.248 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2024-04-21T08:52:17.249 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2024-04-21T08:52:17.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.249 INFO:teuthology.orchestra.run.smithi175.stdout: "osd-slow-ops": { 2024-04-21T08:52:17.249 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_ops_count": 0 2024-04-21T08:52:17.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.249 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2024-04-21T08:52:17.249 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2024-04-21T08:52:17.249 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T08:52:17.249 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2024-04-21T08:52:17.249 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 0, 2024-04-21T08:52:17.249 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed_ops": 0, 2024-04-21T08:52:17.250 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 0, 2024-04-21T08:52:17.250 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2024-04-21T08:52:17.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.250 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:52:17.250 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-04-21T08:52:17.250 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-04-21T08:52:17.250 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-04-21T08:52:17.250 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 3011, 2024-04-21T08:52:17.250 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 2161175, 2024-04-21T08:52:17.250 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:17.251 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 3011, 2024-04-21T08:52:17.251 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-04-21T08:52:17.251 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-04-21T08:52:17.251 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 3011, 2024-04-21T08:52:17.251 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 2161175, 2024-04-21T08:52:17.251 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-04-21T08:52:17.251 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.251 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.251 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.251 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:17.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.251 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2024-04-21T08:52:17.251 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-04-21T08:52:17.252 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-04-21T08:52:17.252 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-04-21T08:52:17.252 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-04-21T08:52:17.252 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-04-21T08:52:17.252 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:17.252 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:17.252 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 2705, 2024-04-21T08:52:17.252 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 2048, 2024-04-21T08:52:17.252 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 2705, 2024-04-21T08:52:17.252 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 2048, 2024-04-21T08:52:17.252 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-04-21T08:52:17.252 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.253 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.253 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.253 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:17.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.253 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2024-04-21T08:52:17.253 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-04-21T08:52:17.253 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2024-04-21T08:52:17.253 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-04-21T08:52:17.253 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-04-21T08:52:17.253 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-04-21T08:52:17.253 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:17.254 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:17.254 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 2705, 2024-04-21T08:52:17.254 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 2705, 2024-04-21T08:52:17.254 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 2705, 2024-04-21T08:52:17.254 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 2705, 2024-04-21T08:52:17.254 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-04-21T08:52:17.254 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.254 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.254 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.254 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:17.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.254 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:52:17.254 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-04-21T08:52:17.255 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-04-21T08:52:17.255 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-04-21T08:52:17.255 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-04-21T08:52:17.255 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-04-21T08:52:17.255 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:17.255 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:17.255 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-04-21T08:52:17.255 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-04-21T08:52:17.255 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2024-04-21T08:52:17.255 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2024-04-21T08:52:17.255 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-04-21T08:52:17.255 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.255 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.256 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.256 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:17.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:17.256 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x564f007ea6e0": { 2024-04-21T08:52:17.256 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-04-21T08:52:17.256 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-04-21T08:52:17.256 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-04-21T08:52:17.256 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-04-21T08:52:17.256 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-04-21T08:52:17.256 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:17.256 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:17.256 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-04-21T08:52:17.257 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-04-21T08:52:17.257 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2024-04-21T08:52:17.257 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2024-04-21T08:52:17.257 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-04-21T08:52:17.257 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:17.257 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:17.257 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:17.257 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:17.257 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:17.257 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-04-21T08:52:18.124 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-21T08:52:18.124 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-21T08:52:18.124 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-21T08:52:18.124 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-21T08:52:18.124 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-21T08:52:18.124 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:52:18.124 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:52:18.124 DEBUG:tasks.check_counter:Getting stats from i 2024-04-21T08:52:18.125 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-21T08:52:18.364 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:52:18.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:18 smithi175 ceph-mon[31920]: pgmap v419: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-21T08:52:18.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:18 smithi112 ceph-mon[31085]: pgmap v419: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-21T08:52:18.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:18 smithi043 ceph-mon[27978]: pgmap v419: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-21T08:52:20.072 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-04-21T08:52:20.073 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:52:20.073 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 421, 2024-04-21T08:52:20.073 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 566, 2024-04-21T08:52:20.073 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 1727178, 2024-04-21T08:52:20.073 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 632609, 2024-04-21T08:52:20.073 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 15, 2024-04-21T08:52:20.073 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 6, 2024-04-21T08:52:20.073 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.081199940, 2024-04-21T08:52:20.073 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.027906998, 2024-04-21T08:52:20.073 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.046795066, 2024-04-21T08:52:20.073 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.007162715, 2024-04-21T08:52:20.073 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:20.073 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 560, 2024-04-21T08:52:20.074 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.071621651, 2024-04-21T08:52:20.074 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000127895 2024-04-21T08:52:20.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.074 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:20.074 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 133, 2024-04-21T08:52:20.074 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000036709, 2024-04-21T08:52:20.074 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000276 2024-04-21T08:52:20.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.074 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 1691456, 2024-04-21T08:52:20.074 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 545568 2024-04-21T08:52:20.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.074 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:52:20.074 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 158, 2024-04-21T08:52:20.075 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 215, 2024-04-21T08:52:20.075 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 48191, 2024-04-21T08:52:20.075 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 889958, 2024-04-21T08:52:20.075 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 73, 2024-04-21T08:52:20.075 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 5, 2024-04-21T08:52:20.075 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.036414379, 2024-04-21T08:52:20.075 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.007457489, 2024-04-21T08:52:20.075 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.026250468, 2024-04-21T08:52:20.075 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000426317, 2024-04-21T08:52:20.075 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:20.075 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 212, 2024-04-21T08:52:20.075 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.019210862, 2024-04-21T08:52:20.075 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000090617 2024-04-21T08:52:20.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.076 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:20.076 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 236, 2024-04-21T08:52:20.076 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000041294, 2024-04-21T08:52:20.076 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000174 2024-04-21T08:52:20.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.076 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 4192, 2024-04-21T08:52:20.076 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 142352 2024-04-21T08:52:20.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.076 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:52:20.076 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 1597, 2024-04-21T08:52:20.076 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 1161, 2024-04-21T08:52:20.076 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 459303, 2024-04-21T08:52:20.076 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 434007, 2024-04-21T08:52:20.076 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 12, 2024-04-21T08:52:20.077 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 2, 2024-04-21T08:52:20.077 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.122942877, 2024-04-21T08:52:20.077 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.048338563, 2024-04-21T08:52:20.077 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.081124295, 2024-04-21T08:52:20.077 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.006313465, 2024-04-21T08:52:20.077 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:20.077 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1165, 2024-04-21T08:52:20.077 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.056361742, 2024-04-21T08:52:20.077 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000048379 2024-04-21T08:52:20.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.077 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:20.077 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2609, 2024-04-21T08:52:20.077 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000597446, 2024-04-21T08:52:20.077 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000228 2024-04-21T08:52:20.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.078 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T08:52:20.078 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T08:52:20.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.078 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2024-04-21T08:52:20.078 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2024-04-21T08:52:20.078 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2024-04-21T08:52:20.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.078 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2024-04-21T08:52:20.078 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-04-21T08:52:20.078 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-04-21T08:52:20.078 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 54, 2024-04-21T08:52:20.078 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.001771354, 2024-04-21T08:52:20.078 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000032802 2024-04-21T08:52:20.079 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:20.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.079 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2024-04-21T08:52:20.079 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-04-21T08:52:20.079 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-04-21T08:52:20.079 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2, 2024-04-21T08:52:20.079 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000007532, 2024-04-21T08:52:20.079 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000003766 2024-04-21T08:52:20.079 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:20.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.079 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2024-04-21T08:52:20.079 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 4, 2024-04-21T08:52:20.079 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 3, 2024-04-21T08:52:20.079 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2024-04-21T08:52:20.080 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 3, 2024-04-21T08:52:20.080 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.001742578, 2024-04-21T08:52:20.080 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000580859 2024-04-21T08:52:20.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.080 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_reply": 0, 2024-04-21T08:52:20.080 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 1, 2024-04-21T08:52:20.080 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_complete": 0, 2024-04-21T08:52:20.080 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_keys": 0, 2024-04-21T08:52:20.080 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 11, 2024-04-21T08:52:20.080 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 0, 2024-04-21T08:52:20.080 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 0, 2024-04-21T08:52:20.080 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 16, 2024-04-21T08:52:20.080 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 0, 2024-04-21T08:52:20.080 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 16, 2024-04-21T08:52:20.080 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 0, 2024-04-21T08:52:20.081 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 13, 2024-04-21T08:52:20.081 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 5, 2024-04-21T08:52:20.081 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 0, 2024-04-21T08:52:20.081 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 0, 2024-04-21T08:52:20.081 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 3, 2024-04-21T08:52:20.081 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 28, 2024-04-21T08:52:20.081 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 16, 2024-04-21T08:52:20.081 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 1, 2024-04-21T08:52:20.081 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 9, 2024-04-21T08:52:20.081 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:52:20.081 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2024-04-21T08:52:20.081 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 0, 2024-04-21T08:52:20.081 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 2000, 2024-04-21T08:52:20.081 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 2, 2024-04-21T08:52:20.082 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 0, 2024-04-21T08:52:20.082 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 0, 2024-04-21T08:52:20.082 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 0, 2024-04-21T08:52:20.082 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 0, 2024-04-21T08:52:20.082 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2024-04-21T08:52:20.082 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 0, 2024-04-21T08:52:20.082 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 0, 2024-04-21T08:52:20.082 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 5, 2024-04-21T08:52:20.082 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 298, 2024-04-21T08:52:20.082 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 6, 2024-04-21T08:52:20.082 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T08:52:20.082 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_set_tag": 0, 2024-04-21T08:52:20.082 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:52:20.082 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:52:20.083 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:52:20.083 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:52:20.083 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:52:20.083 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_file_inodes": 0, 2024-04-21T08:52:20.083 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:52:20.083 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T08:52:20.083 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 0, 2024-04-21T08:52:20.083 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:52:20.083 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:52:20.083 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:52:20.083 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 0, 2024-04-21T08:52:20.083 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 0, 2024-04-21T08:52:20.083 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 0, 2024-04-21T08:52:20.084 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 0 2024-04-21T08:52:20.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.084 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2024-04-21T08:52:20.084 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update": 0, 2024-04-21T08:52:20.084 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update_receipt": 20, 2024-04-21T08:52:20.084 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_try_discover": 14, 2024-04-21T08:52:20.084 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_send_discover": 13, 2024-04-21T08:52:20.084 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_handle_discover": 0, 2024-04-21T08:52:20.084 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 0, 2024-04-21T08:52:20.084 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2024-04-21T08:52:20.084 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:52:20.084 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 0, 2024-04-21T08:52:20.084 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 0, 2024-04-21T08:52:20.084 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 0, 2024-04-21T08:52:20.085 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2024-04-21T08:52:20.085 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2024-04-21T08:52:20.085 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:52:20.085 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:52:20.085 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2024-04-21T08:52:20.085 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2024-04-21T08:52:20.085 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_path": 3, 2024-04-21T08:52:20.085 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_inode": 15, 2024-04-21T08:52:20.085 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:52:20.085 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 0, 2024-04-21T08:52:20.085 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2024-04-21T08:52:20.085 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:52:20.085 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2024-04-21T08:52:20.085 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2024-04-21T08:52:20.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.086 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2024-04-21T08:52:20.086 INFO:teuthology.orchestra.run.smithi175.stdout: "evlrg": 0, 2024-04-21T08:52:20.086 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 7, 2024-04-21T08:52:20.086 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 0, 2024-04-21T08:52:20.086 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 0, 2024-04-21T08:52:20.086 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 7, 2024-04-21T08:52:20.086 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2024-04-21T08:52:20.086 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 0, 2024-04-21T08:52:20.086 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 2, 2024-04-21T08:52:20.086 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 0, 2024-04-21T08:52:20.086 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 0, 2024-04-21T08:52:20.086 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 2, 2024-04-21T08:52:20.086 INFO:teuthology.orchestra.run.smithi175.stdout: "segmjr": 2, 2024-04-21T08:52:20.087 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2024-04-21T08:52:20.087 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 0, 2024-04-21T08:52:20.087 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 4194304, 2024-04-21T08:52:20.087 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 4196347, 2024-04-21T08:52:20.087 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 0, 2024-04-21T08:52:20.087 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2024-04-21T08:52:20.087 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 6, 2024-04-21T08:52:20.087 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.011704731, 2024-04-21T08:52:20.087 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.001950788 2024-04-21T08:52:20.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.087 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 0 2024-04-21T08:52:20.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.087 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2024-04-21T08:52:20.087 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 20, 2024-04-21T08:52:20.087 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 22, 2024-04-21T08:52:20.088 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 2, 2024-04-21T08:52:20.088 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 19, 2024-04-21T08:52:20.088 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 23, 2024-04-21T08:52:20.088 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 4, 2024-04-21T08:52:20.088 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 16, 2024-04-21T08:52:20.088 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 21, 2024-04-21T08:52:20.088 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 5, 2024-04-21T08:52:20.088 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 0, 2024-04-21T08:52:20.088 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 2, 2024-04-21T08:52:20.088 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 2, 2024-04-21T08:52:20.088 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 53208, 2024-04-21T08:52:20.088 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 223608 2024-04-21T08:52:20.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.088 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2024-04-21T08:52:20.089 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 8, 2024-04-21T08:52:20.089 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 0, 2024-04-21T08:52:20.089 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 4, 2024-04-21T08:52:20.089 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 18, 2024-04-21T08:52:20.089 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 16, 2024-04-21T08:52:20.089 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2024-04-21T08:52:20.089 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.089 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.089 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.089 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2024-04-21T08:52:20.089 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1, 2024-04-21T08:52:20.089 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.001160173, 2024-04-21T08:52:20.089 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.001160173 2024-04-21T08:52:20.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.090 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2024-04-21T08:52:20.090 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.090 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.090 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.090 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2024-04-21T08:52:20.090 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.090 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.090 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.090 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2024-04-21T08:52:20.090 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2, 2024-04-21T08:52:20.090 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000582405, 2024-04-21T08:52:20.091 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000291202 2024-04-21T08:52:20.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.091 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2024-04-21T08:52:20.091 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.091 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.091 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.091 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2024-04-21T08:52:20.091 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.091 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.091 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.091 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2024-04-21T08:52:20.091 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.092 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.092 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.092 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2024-04-21T08:52:20.092 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.092 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.092 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.092 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2024-04-21T08:52:20.092 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.092 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.092 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.092 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2024-04-21T08:52:20.093 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.093 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.093 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.093 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2024-04-21T08:52:20.094 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.094 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.094 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.094 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2024-04-21T08:52:20.094 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.094 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.094 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.094 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2024-04-21T08:52:20.094 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.094 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.094 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.095 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2024-04-21T08:52:20.095 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.095 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.095 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.095 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2024-04-21T08:52:20.095 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.095 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.095 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.095 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2024-04-21T08:52:20.095 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.095 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.096 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.096 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2024-04-21T08:52:20.096 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.096 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.096 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.096 INFO:teuthology.orchestra.run.smithi175.stdout: "req_snapdiff_latency": { 2024-04-21T08:52:20.096 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.096 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.096 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.096 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2024-04-21T08:52:20.096 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.097 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.097 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.097 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2024-04-21T08:52:20.097 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.097 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.097 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.097 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2024-04-21T08:52:20.097 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.097 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.097 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.097 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2024-04-21T08:52:20.097 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.098 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.098 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.098 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2024-04-21T08:52:20.098 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.098 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.098 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.098 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2024-04-21T08:52:20.098 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.098 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.098 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.099 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2024-04-21T08:52:20.099 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.099 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.099 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.099 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2024-04-21T08:52:20.099 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.099 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.099 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.099 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2024-04-21T08:52:20.099 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.099 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.099 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.100 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2024-04-21T08:52:20.100 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.100 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.100 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.100 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:52:20.100 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:52:20.100 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getvxattr_latency": { 2024-04-21T08:52:20.100 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.100 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.100 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.100 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:20.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.101 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2024-04-21T08:52:20.101 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 1, 2024-04-21T08:52:20.101 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 3, 2024-04-21T08:52:20.101 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 2, 2024-04-21T08:52:20.101 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 1, 2024-04-21T08:52:20.101 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2024-04-21T08:52:20.101 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 2, 2024-04-21T08:52:20.101 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 2, 2024-04-21T08:52:20.101 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 734, 2024-04-21T08:52:20.101 INFO:teuthology.orchestra.run.smithi175.stdout: "mdthresh_evicted": 0 2024-04-21T08:52:20.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.101 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2024-04-21T08:52:20.101 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:52:20.101 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2024-04-21T08:52:20.101 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:52:20.102 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:52:20.102 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:52:20.102 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:52:20.102 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:52:20.102 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:52:20.102 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:52:20.102 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:52:20.102 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:52:20.102 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:52:20.102 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:52:20.102 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:52:20.102 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:52:20.102 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2024-04-21T08:52:20.102 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:52:20.103 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2024-04-21T08:52:20.103 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:52:20.103 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:52:20.103 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:52:20.103 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:52:20.103 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:52:20.103 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:52:20.103 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:52:20.103 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2024-04-21T08:52:20.103 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:52:20.103 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:52:20.103 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:52:20.103 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2024-04-21T08:52:20.103 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2024-04-21T08:52:20.104 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2024-04-21T08:52:20.104 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:52:20.104 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:52:20.104 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:52:20.104 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:52:20.104 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 246583, 2024-04-21T08:52:20.104 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 81, 2024-04-21T08:52:20.104 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:52:20.104 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2024-04-21T08:52:20.104 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2024-04-21T08:52:20.104 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2024-04-21T08:52:20.104 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:52:20.104 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2024-04-21T08:52:20.104 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:52:20.104 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2024-04-21T08:52:20.105 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 9800, 2024-04-21T08:52:20.105 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 188, 2024-04-21T08:52:20.105 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:52:20.105 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:52:20.105 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2024-04-21T08:52:20.105 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2024-04-21T08:52:20.105 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 86461, 2024-04-21T08:52:20.105 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 348, 2024-04-21T08:52:20.105 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2024-04-21T08:52:20.105 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2024-04-21T08:52:20.105 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2024-04-21T08:52:20.105 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2024-04-21T08:52:20.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.105 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2024-04-21T08:52:20.106 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2024-04-21T08:52:20.106 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2024-04-21T08:52:20.106 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 46, 2024-04-21T08:52:20.106 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 12670, 2024-04-21T08:52:20.106 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 28, 2024-04-21T08:52:20.106 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 46, 2024-04-21T08:52:20.106 INFO:teuthology.orchestra.run.smithi175.stdout: "op_latency": { 2024-04-21T08:52:20.106 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 46, 2024-04-21T08:52:20.106 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.441992548, 2024-04-21T08:52:20.106 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.009608533 2024-04-21T08:52:20.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.106 INFO:teuthology.orchestra.run.smithi175.stdout: "op_inflight": 0, 2024-04-21T08:52:20.106 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2024-04-21T08:52:20.107 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 46, 2024-04-21T08:52:20.107 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 70 2024-04-21T08:52:20.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.107 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 46, 2024-04-21T08:52:20.107 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 0, 2024-04-21T08:52:20.107 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 46, 2024-04-21T08:52:20.107 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2024-04-21T08:52:20.107 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2024-04-21T08:52:20.107 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 0, 2024-04-21T08:52:20.107 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 10, 2024-04-21T08:52:20.107 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 0, 2024-04-21T08:52:20.107 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 6, 2024-04-21T08:52:20.107 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 9, 2024-04-21T08:52:20.107 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2024-04-21T08:52:20.107 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2024-04-21T08:52:20.108 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2024-04-21T08:52:20.108 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 0, 2024-04-21T08:52:20.108 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 6, 2024-04-21T08:52:20.108 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2024-04-21T08:52:20.108 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2024-04-21T08:52:20.108 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2024-04-21T08:52:20.108 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 0, 2024-04-21T08:52:20.108 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 20, 2024-04-21T08:52:20.108 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:52:20.108 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2024-04-21T08:52:20.108 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:52:20.108 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2024-04-21T08:52:20.108 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2024-04-21T08:52:20.109 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2024-04-21T08:52:20.109 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:52:20.109 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2024-04-21T08:52:20.109 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:52:20.109 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2024-04-21T08:52:20.109 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2024-04-21T08:52:20.109 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2024-04-21T08:52:20.109 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2024-04-21T08:52:20.109 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2024-04-21T08:52:20.109 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2024-04-21T08:52:20.109 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2024-04-21T08:52:20.109 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2024-04-21T08:52:20.109 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2024-04-21T08:52:20.109 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2024-04-21T08:52:20.109 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2024-04-21T08:52:20.110 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2024-04-21T08:52:20.110 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2024-04-21T08:52:20.110 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2024-04-21T08:52:20.110 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2024-04-21T08:52:20.110 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2024-04-21T08:52:20.110 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2024-04-21T08:52:20.110 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 110, 2024-04-21T08:52:20.110 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2024-04-21T08:52:20.110 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 20, 2024-04-21T08:52:20.110 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 8, 2024-04-21T08:52:20.110 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 12, 2024-04-21T08:52:20.110 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 4, 2024-04-21T08:52:20.110 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2024-04-21T08:52:20.110 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 17, 2024-04-21T08:52:20.111 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 0, 2024-04-21T08:52:20.111 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 2 2024-04-21T08:52:20.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.111 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2024-04-21T08:52:20.111 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 1, 2024-04-21T08:52:20.111 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:52:20.111 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2024-04-21T08:52:20.111 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2024-04-21T08:52:20.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.111 INFO:teuthology.orchestra.run.smithi175.stdout: "osd-slow-ops": { 2024-04-21T08:52:20.111 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_ops_count": 18 2024-04-21T08:52:20.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.111 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2024-04-21T08:52:20.111 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2024-04-21T08:52:20.111 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T08:52:20.111 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2024-04-21T08:52:20.112 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 0, 2024-04-21T08:52:20.112 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed_ops": 0, 2024-04-21T08:52:20.112 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 0, 2024-04-21T08:52:20.112 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2024-04-21T08:52:20.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.112 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:52:20.112 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-04-21T08:52:20.112 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-04-21T08:52:20.112 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-04-21T08:52:20.112 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 2172, 2024-04-21T08:52:20.112 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 2018584, 2024-04-21T08:52:20.112 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:20.112 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 2172, 2024-04-21T08:52:20.112 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-04-21T08:52:20.113 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-04-21T08:52:20.113 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 2172, 2024-04-21T08:52:20.113 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 2018584, 2024-04-21T08:52:20.113 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-04-21T08:52:20.113 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.113 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.113 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.113 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:20.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.113 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2024-04-21T08:52:20.113 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-04-21T08:52:20.113 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-04-21T08:52:20.113 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-04-21T08:52:20.113 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-04-21T08:52:20.113 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-04-21T08:52:20.114 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:20.114 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:20.114 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 46, 2024-04-21T08:52:20.114 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 12670, 2024-04-21T08:52:20.114 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 46, 2024-04-21T08:52:20.114 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 12670, 2024-04-21T08:52:20.114 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-04-21T08:52:20.114 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.114 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.114 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.114 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:20.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.114 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2024-04-21T08:52:20.114 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-04-21T08:52:20.115 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2024-04-21T08:52:20.115 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-04-21T08:52:20.115 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-04-21T08:52:20.115 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-04-21T08:52:20.115 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:20.115 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:20.115 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 46, 2024-04-21T08:52:20.115 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 46, 2024-04-21T08:52:20.115 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 46, 2024-04-21T08:52:20.115 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 46, 2024-04-21T08:52:20.115 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-04-21T08:52:20.115 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.115 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.115 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.115 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:20.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.116 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:52:20.116 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-04-21T08:52:20.116 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-04-21T08:52:20.116 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-04-21T08:52:20.116 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-04-21T08:52:20.116 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-04-21T08:52:20.116 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:20.116 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:20.116 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-04-21T08:52:20.116 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-04-21T08:52:20.116 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2024-04-21T08:52:20.116 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2024-04-21T08:52:20.116 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-04-21T08:52:20.116 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.117 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.117 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.117 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:20.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:20.117 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x55919e212500": { 2024-04-21T08:52:20.117 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-04-21T08:52:20.117 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-04-21T08:52:20.117 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-04-21T08:52:20.117 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 7, 2024-04-21T08:52:20.117 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 2043, 2024-04-21T08:52:20.117 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:20.117 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 7, 2024-04-21T08:52:20.117 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-04-21T08:52:20.117 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-04-21T08:52:20.117 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 6, 2024-04-21T08:52:20.117 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 2043, 2024-04-21T08:52:20.118 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-04-21T08:52:20.118 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:20.118 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:20.118 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:20.118 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:20.118 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:20.118 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-04-21T08:52:20.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:20 smithi175 ceph-mon[31920]: pgmap v420: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 216 B/s wr, 10 op/s 2024-04-21T08:52:20.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:20 smithi112 ceph-mon[31085]: pgmap v420: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 216 B/s wr, 10 op/s 2024-04-21T08:52:20.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:20 smithi043 ceph-mon[27978]: pgmap v420: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 216 B/s wr, 10 op/s 2024-04-21T08:52:20.878 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-21T08:52:20.879 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-21T08:52:20.879 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-04-21T08:52:20.879 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=20 2024-04-21T08:52:20.879 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-21T08:52:20.879 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:52:20.879 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:52:20.879 DEBUG:tasks.check_counter:Getting stats from l 2024-04-21T08:52:20.879 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-21T08:52:21.125 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:52:22.294 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:22 smithi175 ceph-mon[31920]: pgmap v421: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-21T08:52:22.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:22 smithi112 ceph-mon[31085]: pgmap v421: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-21T08:52:22.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:22 smithi043 ceph-mon[27978]: pgmap v421: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-21T08:52:22.841 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-04-21T08:52:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:52:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 84, 2024-04-21T08:52:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 256, 2024-04-21T08:52:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 14475, 2024-04-21T08:52:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 1115122, 2024-04-21T08:52:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 77, 2024-04-21T08:52:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 5, 2024-04-21T08:52:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.039580132, 2024-04-21T08:52:22.841 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.008869908, 2024-04-21T08:52:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.021909524, 2024-04-21T08:52:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000279776, 2024-04-21T08:52:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 247, 2024-04-21T08:52:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.020489384, 2024-04-21T08:52:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000082952 2024-04-21T08:52:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 70, 2024-04-21T08:52:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000007654, 2024-04-21T08:52:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000109 2024-04-21T08:52:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-04-21T08:52:22.842 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 379040 2024-04-21T08:52:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:52:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 1583, 2024-04-21T08:52:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 1221, 2024-04-21T08:52:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 451092, 2024-04-21T08:52:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 580814, 2024-04-21T08:52:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 16, 2024-04-21T08:52:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 4, 2024-04-21T08:52:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.126069118, 2024-04-21T08:52:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.051174400, 2024-04-21T08:52:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.079921324, 2024-04-21T08:52:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.006086482, 2024-04-21T08:52:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:22.843 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1223, 2024-04-21T08:52:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.060395273, 2024-04-21T08:52:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000049382 2024-04-21T08:52:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2629, 2024-04-21T08:52:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000729560, 2024-04-21T08:52:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000277 2024-04-21T08:52:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 4368, 2024-04-21T08:52:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 148192 2024-04-21T08:52:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:52:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 483, 2024-04-21T08:52:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 463, 2024-04-21T08:52:22.844 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 1742557, 2024-04-21T08:52:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 243425, 2024-04-21T08:52:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 8, 2024-04-21T08:52:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 4, 2024-04-21T08:52:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.071254159, 2024-04-21T08:52:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.024036383, 2024-04-21T08:52:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.057672050, 2024-04-21T08:52:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.007826830, 2024-04-21T08:52:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:52:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 465, 2024-04-21T08:52:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.037742489, 2024-04-21T08:52:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000081166 2024-04-21T08:52:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:52:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 226, 2024-04-21T08:52:22.845 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000047493, 2024-04-21T08:52:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000210 2024-04-21T08:52:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 1676560, 2024-04-21T08:52:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 174144 2024-04-21T08:52:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2024-04-21T08:52:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2024-04-21T08:52:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2024-04-21T08:52:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2024-04-21T08:52:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-04-21T08:52:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-04-21T08:52:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 33, 2024-04-21T08:52:22.846 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.002160087, 2024-04-21T08:52:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000065457 2024-04-21T08:52:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2024-04-21T08:52:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-04-21T08:52:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-04-21T08:52:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2, 2024-04-21T08:52:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000073325, 2024-04-21T08:52:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000036662 2024-04-21T08:52:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2024-04-21T08:52:22.847 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 6, 2024-04-21T08:52:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 3, 2024-04-21T08:52:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2024-04-21T08:52:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 3, 2024-04-21T08:52:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000516419, 2024-04-21T08:52:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000172139 2024-04-21T08:52:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_reply": 0, 2024-04-21T08:52:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 3, 2024-04-21T08:52:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_complete": 0, 2024-04-21T08:52:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_keys": 0, 2024-04-21T08:52:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 11, 2024-04-21T08:52:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 0, 2024-04-21T08:52:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 0, 2024-04-21T08:52:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 15, 2024-04-21T08:52:22.848 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 0, 2024-04-21T08:52:22.849 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 15, 2024-04-21T08:52:22.849 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 0, 2024-04-21T08:52:22.849 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 13, 2024-04-21T08:52:22.849 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 4, 2024-04-21T08:52:22.849 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 1, 2024-04-21T08:52:22.849 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 1, 2024-04-21T08:52:22.849 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 3, 2024-04-21T08:52:22.849 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 24, 2024-04-21T08:52:22.849 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 16, 2024-04-21T08:52:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 0, 2024-04-21T08:52:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 7, 2024-04-21T08:52:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:52:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2024-04-21T08:52:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 0, 2024-04-21T08:52:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 2000, 2024-04-21T08:52:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 2, 2024-04-21T08:52:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 0, 2024-04-21T08:52:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 0, 2024-04-21T08:52:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 0, 2024-04-21T08:52:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 0, 2024-04-21T08:52:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2024-04-21T08:52:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 0, 2024-04-21T08:52:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 0, 2024-04-21T08:52:22.850 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 5, 2024-04-21T08:52:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 298, 2024-04-21T08:52:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 6, 2024-04-21T08:52:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T08:52:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_set_tag": 0, 2024-04-21T08:52:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:52:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:52:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:52:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:52:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:52:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_file_inodes": 0, 2024-04-21T08:52:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:52:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 1, 2024-04-21T08:52:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 0, 2024-04-21T08:52:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:52:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:52:22.851 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:52:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 1, 2024-04-21T08:52:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 0, 2024-04-21T08:52:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 0, 2024-04-21T08:52:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 0 2024-04-21T08:52:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2024-04-21T08:52:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update": 0, 2024-04-21T08:52:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update_receipt": 15, 2024-04-21T08:52:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_try_discover": 11, 2024-04-21T08:52:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_send_discover": 9, 2024-04-21T08:52:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_handle_discover": 0, 2024-04-21T08:52:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 0, 2024-04-21T08:52:22.852 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2024-04-21T08:52:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:52:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 0, 2024-04-21T08:52:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 0, 2024-04-21T08:52:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 0, 2024-04-21T08:52:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2024-04-21T08:52:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2024-04-21T08:52:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:52:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:52:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2024-04-21T08:52:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2024-04-21T08:52:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_path": 3, 2024-04-21T08:52:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_inode": 12, 2024-04-21T08:52:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:52:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 0, 2024-04-21T08:52:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2024-04-21T08:52:22.853 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:52:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2024-04-21T08:52:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2024-04-21T08:52:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2024-04-21T08:52:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: "evlrg": 0, 2024-04-21T08:52:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 5, 2024-04-21T08:52:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 0, 2024-04-21T08:52:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 0, 2024-04-21T08:52:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 5, 2024-04-21T08:52:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2024-04-21T08:52:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 0, 2024-04-21T08:52:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 2, 2024-04-21T08:52:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 0, 2024-04-21T08:52:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 0, 2024-04-21T08:52:22.854 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 2, 2024-04-21T08:52:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: "segmjr": 2, 2024-04-21T08:52:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2024-04-21T08:52:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 0, 2024-04-21T08:52:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 4194304, 2024-04-21T08:52:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 4195815, 2024-04-21T08:52:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 0, 2024-04-21T08:52:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2024-04-21T08:52:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 4, 2024-04-21T08:52:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.007223832, 2024-04-21T08:52:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.001805958 2024-04-21T08:52:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 0 2024-04-21T08:52:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2024-04-21T08:52:22.855 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 19, 2024-04-21T08:52:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 21, 2024-04-21T08:52:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 2, 2024-04-21T08:52:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 18, 2024-04-21T08:52:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 20, 2024-04-21T08:52:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 2, 2024-04-21T08:52:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 15, 2024-04-21T08:52:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 19, 2024-04-21T08:52:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 4, 2024-04-21T08:52:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 1, 2024-04-21T08:52:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 1, 2024-04-21T08:52:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 0, 2024-04-21T08:52:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 56124, 2024-04-21T08:52:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 223608 2024-04-21T08:52:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2024-04-21T08:52:22.856 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 6, 2024-04-21T08:52:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 0, 2024-04-21T08:52:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 6, 2024-04-21T08:52:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 14, 2024-04-21T08:52:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 12, 2024-04-21T08:52:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2024-04-21T08:52:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2024-04-21T08:52:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 3, 2024-04-21T08:52:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000516419, 2024-04-21T08:52:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000172139 2024-04-21T08:52:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.857 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2024-04-21T08:52:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2024-04-21T08:52:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2024-04-21T08:52:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.858 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2024-04-21T08:52:22.859 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.859 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.859 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.859 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.859 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2024-04-21T08:52:22.859 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.859 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.859 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.859 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.859 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2024-04-21T08:52:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2024-04-21T08:52:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2024-04-21T08:52:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.860 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.861 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.861 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2024-04-21T08:52:22.861 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.861 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.861 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.861 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.861 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2024-04-21T08:52:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2024-04-21T08:52:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2024-04-21T08:52:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.862 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2024-04-21T08:52:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2024-04-21T08:52:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2024-04-21T08:52:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.863 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2024-04-21T08:52:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: "req_snapdiff_latency": { 2024-04-21T08:52:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2024-04-21T08:52:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.864 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2024-04-21T08:52:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2024-04-21T08:52:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2024-04-21T08:52:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2024-04-21T08:52:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2024-04-21T08:52:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2024-04-21T08:52:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.866 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2024-04-21T08:52:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2024-04-21T08:52:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.867 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2024-04-21T08:52:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:52:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:52:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getvxattr_latency": { 2024-04-21T08:52:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2024-04-21T08:52:22.868 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 1, 2024-04-21T08:52:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 2, 2024-04-21T08:52:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 1, 2024-04-21T08:52:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 1, 2024-04-21T08:52:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2024-04-21T08:52:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 2, 2024-04-21T08:52:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 1, 2024-04-21T08:52:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 741, 2024-04-21T08:52:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "mdthresh_evicted": 0 2024-04-21T08:52:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2024-04-21T08:52:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:52:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2024-04-21T08:52:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:52:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:52:22.869 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:52:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:52:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:52:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:52:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:52:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:52:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:52:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:52:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:52:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:52:22.870 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:52:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2024-04-21T08:52:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:52:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2024-04-21T08:52:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:52:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:52:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:52:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:52:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:52:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:52:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:52:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2024-04-21T08:52:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:52:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:52:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:52:22.871 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2024-04-21T08:52:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2024-04-21T08:52:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2024-04-21T08:52:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:52:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:52:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:52:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:52:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 263847, 2024-04-21T08:52:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 91, 2024-04-21T08:52:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 88, 2024-04-21T08:52:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 1, 2024-04-21T08:52:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2024-04-21T08:52:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2024-04-21T08:52:22.872 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:52:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2024-04-21T08:52:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:52:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2024-04-21T08:52:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 9800, 2024-04-21T08:52:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 188, 2024-04-21T08:52:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:52:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:52:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2024-04-21T08:52:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2024-04-21T08:52:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 82557, 2024-04-21T08:52:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 327, 2024-04-21T08:52:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2024-04-21T08:52:22.873 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2024-04-21T08:52:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2024-04-21T08:52:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2024-04-21T08:52:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2024-04-21T08:52:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2024-04-21T08:52:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2024-04-21T08:52:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 41, 2024-04-21T08:52:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 11903, 2024-04-21T08:52:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 33, 2024-04-21T08:52:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 41, 2024-04-21T08:52:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: "op_latency": { 2024-04-21T08:52:22.874 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 41, 2024-04-21T08:52:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.260995596, 2024-04-21T08:52:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.006365746 2024-04-21T08:52:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: "op_inflight": 0, 2024-04-21T08:52:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2024-04-21T08:52:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 41, 2024-04-21T08:52:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 63 2024-04-21T08:52:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 41, 2024-04-21T08:52:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 0, 2024-04-21T08:52:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 41, 2024-04-21T08:52:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2024-04-21T08:52:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2024-04-21T08:52:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 0, 2024-04-21T08:52:22.875 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 10, 2024-04-21T08:52:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 0, 2024-04-21T08:52:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 4, 2024-04-21T08:52:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 8, 2024-04-21T08:52:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2024-04-21T08:52:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2024-04-21T08:52:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2024-04-21T08:52:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 0, 2024-04-21T08:52:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 6, 2024-04-21T08:52:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2024-04-21T08:52:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2024-04-21T08:52:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2024-04-21T08:52:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 0, 2024-04-21T08:52:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 20, 2024-04-21T08:52:22.876 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:52:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2024-04-21T08:52:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:52:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2024-04-21T08:52:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2024-04-21T08:52:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2024-04-21T08:52:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:52:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2024-04-21T08:52:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:52:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2024-04-21T08:52:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2024-04-21T08:52:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2024-04-21T08:52:22.877 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2024-04-21T08:52:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2024-04-21T08:52:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2024-04-21T08:52:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2024-04-21T08:52:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2024-04-21T08:52:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2024-04-21T08:52:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2024-04-21T08:52:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2024-04-21T08:52:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2024-04-21T08:52:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2024-04-21T08:52:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2024-04-21T08:52:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2024-04-21T08:52:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2024-04-21T08:52:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2024-04-21T08:52:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 110, 2024-04-21T08:52:22.878 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2024-04-21T08:52:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 20, 2024-04-21T08:52:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 9, 2024-04-21T08:52:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 14, 2024-04-21T08:52:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 5, 2024-04-21T08:52:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2024-04-21T08:52:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 14, 2024-04-21T08:52:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 0, 2024-04-21T08:52:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 1 2024-04-21T08:52:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2024-04-21T08:52:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 1, 2024-04-21T08:52:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:52:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2024-04-21T08:52:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2024-04-21T08:52:22.879 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: "osd-slow-ops": { 2024-04-21T08:52:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_ops_count": 15 2024-04-21T08:52:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2024-04-21T08:52:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2024-04-21T08:52:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T08:52:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2024-04-21T08:52:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 0, 2024-04-21T08:52:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed_ops": 0, 2024-04-21T08:52:22.880 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 0, 2024-04-21T08:52:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2024-04-21T08:52:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:52:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-04-21T08:52:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-04-21T08:52:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-04-21T08:52:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 2146, 2024-04-21T08:52:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 1993925, 2024-04-21T08:52:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 2146, 2024-04-21T08:52:22.881 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-04-21T08:52:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-04-21T08:52:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 2146, 2024-04-21T08:52:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 1993925, 2024-04-21T08:52:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-04-21T08:52:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2024-04-21T08:52:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-04-21T08:52:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-04-21T08:52:22.882 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-04-21T08:52:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-04-21T08:52:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-04-21T08:52:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 41, 2024-04-21T08:52:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 11903, 2024-04-21T08:52:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 41, 2024-04-21T08:52:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 11903, 2024-04-21T08:52:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-04-21T08:52:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.883 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2024-04-21T08:52:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-04-21T08:52:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2024-04-21T08:52:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-04-21T08:52:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-04-21T08:52:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-04-21T08:52:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 41, 2024-04-21T08:52:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 41, 2024-04-21T08:52:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 41, 2024-04-21T08:52:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 41, 2024-04-21T08:52:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-04-21T08:52:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.884 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:52:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-04-21T08:52:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-04-21T08:52:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-04-21T08:52:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-04-21T08:52:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-04-21T08:52:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-04-21T08:52:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-04-21T08:52:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-04-21T08:52:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2024-04-21T08:52:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2024-04-21T08:52:22.885 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-04-21T08:52:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-21T08:52:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x558d08d52500": { 2024-04-21T08:52:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-04-21T08:52:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-04-21T08:52:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-04-21T08:52:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 5, 2024-04-21T08:52:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 1511, 2024-04-21T08:52:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-04-21T08:52:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 5, 2024-04-21T08:52:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-04-21T08:52:22.886 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-04-21T08:52:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 4, 2024-04-21T08:52:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 1511, 2024-04-21T08:52:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-04-21T08:52:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-04-21T08:52:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-04-21T08:52:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-04-21T08:52:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:22.887 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-21T08:52:22.887 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-04-21T08:52:23.768 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-21T08:52:23.769 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-21T08:52:23.769 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-04-21T08:52:23.769 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=15 2024-04-21T08:52:23.769 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=6 2024-04-21T08:52:23.769 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:52:23.769 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:52:23.769 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-21T08:52:23.778 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-21T08:52:23.778 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-21T08:52:23.779 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T08:52:23.800 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-21T08:52:23.800 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T08:52:23.860 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:52:24.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.019+0000 7f7b8b5b2640 1 -- 172.21.15.43:0/1101607671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b84107fe0 msgr2=0x7f7b841083c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:24.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.019+0000 7f7b8b5b2640 1 --2- 172.21.15.43:0/1101607671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b84107fe0 0x7f7b841083c0 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009900 tx=0x7f7b8002bc30 comp rx=0 tx=0).stop 2024-04-21T08:52:24.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.019+0000 7f7b8b5b2640 1 -- 172.21.15.43:0/1101607671 shutdown_connections 2024-04-21T08:52:24.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.019+0000 7f7b8b5b2640 1 --2- 172.21.15.43:0/1101607671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b84069a80 0x7f7b8410eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.019+0000 7f7b8b5b2640 1 --2- 172.21.15.43:0/1101607671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b840690e0 0x7f7b84069540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.019+0000 7f7b8b5b2640 1 --2- 172.21.15.43:0/1101607671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b84107fe0 0x7f7b841083c0 unknown :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.019+0000 7f7b8b5b2640 1 -- 172.21.15.43:0/1101607671 >> 172.21.15.43:0/1101607671 conn(0x7f7b84077e30 msgr2=0x7f7b84078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:24.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.020+0000 7f7b8b5b2640 1 -- 172.21.15.43:0/1101607671 shutdown_connections 2024-04-21T08:52:24.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.020+0000 7f7b8b5b2640 1 -- 172.21.15.43:0/1101607671 wait complete. 2024-04-21T08:52:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.020+0000 7f7b8b5b2640 1 Processor -- start 2024-04-21T08:52:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.020+0000 7f7b8b5b2640 1 -- start start 2024-04-21T08:52:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.020+0000 7f7b8b5b2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b840690e0 0x7f7b84101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.020+0000 7f7b8b5b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b84069a80 0x7f7b841017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.020+0000 7f7b8b5b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b84107fe0 0x7f7b84101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.020+0000 7f7b8b5b2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7b84112140 con 0x7f7b84107fe0 2024-04-21T08:52:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.020+0000 7f7b8b5b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7b84111fc0 con 0x7f7b84069a80 2024-04-21T08:52:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.020+0000 7f7b8b5b2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841122c0 con 0x7f7b840690e0 2024-04-21T08:52:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.021+0000 7f7b89b4f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b84107fe0 0x7f7b84101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.021+0000 7f7b89b4f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b84107fe0 0x7f7b84101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56440/0 (socket says 172.21.15.43:56440) 2024-04-21T08:52:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.021+0000 7f7b89b4f640 1 -- 172.21.15.43:0/1336640938 learned_addr learned my addr 172.21.15.43:0/1336640938 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:52:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.021+0000 7f7b88b4d640 1 --2- 172.21.15.43:0/1336640938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b84069a80 0x7f7b841017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.021+0000 7f7b8934e640 1 --2- 172.21.15.43:0/1336640938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b840690e0 0x7f7b84101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.021+0000 7f7b89b4f640 1 -- 172.21.15.43:0/1336640938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b840690e0 msgr2=0x7f7b84101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.021+0000 7f7b89b4f640 1 --2- 172.21.15.43:0/1336640938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b840690e0 0x7f7b84101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.021+0000 7f7b89b4f640 1 -- 172.21.15.43:0/1336640938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b84069a80 msgr2=0x7f7b841017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.021+0000 7f7b89b4f640 1 --2- 172.21.15.43:0/1336640938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b84069a80 0x7f7b841017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.021+0000 7f7b89b4f640 1 -- 172.21.15.43:0/1336640938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b841a6c00 con 0x7f7b84107fe0 2024-04-21T08:52:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.021+0000 7f7b8934e640 1 --2- 172.21.15.43:0/1336640938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b840690e0 0x7f7b84101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:52:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.021+0000 7f7b89b4f640 1 --2- 172.21.15.43:0/1336640938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b84107fe0 0x7f7b84101cf0 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00efd0 tx=0x7f7b7c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.021+0000 7f7b7a7fc640 1 -- 172.21.15.43:0/1336640938 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c003b70 con 0x7f7b84107fe0 2024-04-21T08:52:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.021+0000 7f7b7a7fc640 1 -- 172.21.15.43:0/1336640938 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b7c003d10 con 0x7f7b84107fe0 2024-04-21T08:52:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.021+0000 7f7b8b5b2640 1 -- 172.21.15.43:0/1336640938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b841a6ef0 con 0x7f7b84107fe0 2024-04-21T08:52:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.021+0000 7f7b7a7fc640 1 -- 172.21.15.43:0/1336640938 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c0108a0 con 0x7f7b84107fe0 2024-04-21T08:52:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.021+0000 7f7b8b5b2640 1 -- 172.21.15.43:0/1336640938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b841a7410 con 0x7f7b84107fe0 2024-04-21T08:52:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.022+0000 7f7b8b5b2640 1 -- 172.21.15.43:0/1336640938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b84103610 con 0x7f7b84107fe0 2024-04-21T08:52:24.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.023+0000 7f7b7a7fc640 1 -- 172.21.15.43:0/1336640938 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7b7c004b40 con 0x7f7b84107fe0 2024-04-21T08:52:24.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.025+0000 7f7b7a7fc640 1 --2- 172.21.15.43:0/1336640938 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7b700a3da0 0x7f7b700a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:24.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.025+0000 7f7b8934e640 1 --2- 172.21.15.43:0/1336640938 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7b700a3da0 0x7f7b700a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:24.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.025+0000 7f7b7a7fc640 1 -- 172.21.15.43:0/1336640938 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f7b7c0d13b0 con 0x7f7b84107fe0 2024-04-21T08:52:24.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.025+0000 7f7b8934e640 1 --2- 172.21.15.43:0/1336640938 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7b700a3da0 0x7f7b700a6260 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7b80036040 tx=0x7f7b80000bc0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:24.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.028+0000 7f7b7a7fc640 1 -- 172.21.15.43:0/1336640938 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b7c09d610 con 0x7f7b84107fe0 2024-04-21T08:52:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.218+0000 7f7b8b5b2640 1 -- 172.21.15.43:0/1336640938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7b84068de0 con 0x7f7b84107fe0 2024-04-21T08:52:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.220+0000 7f7b7a7fc640 1 -- 172.21.15.43:0/1336640938 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+25419 (secure 0 0 0) 0x7f7b84068de0 con 0x7f7b84107fe0 2024-04-21T08:52:24.220 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:52:24.220 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":110,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:52:15.080041+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/126475132":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/4288951239":"2024-04-22T08:38:54.881882+0000","172.21.15.43:6801/1577030772":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/2382670190":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/1174411284":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/1577030772":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]},{"pool":4,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:52:24.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.222+0000 7f7b8b5b2640 1 -- 172.21.15.43:0/1336640938 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7b700a3da0 msgr2=0x7f7b700a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:24.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.222+0000 7f7b8b5b2640 1 --2- 172.21.15.43:0/1336640938 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7b700a3da0 0x7f7b700a6260 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7b80036040 tx=0x7f7b80000bc0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.222+0000 7f7b8b5b2640 1 -- 172.21.15.43:0/1336640938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b84107fe0 msgr2=0x7f7b84101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:24.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.222+0000 7f7b8b5b2640 1 --2- 172.21.15.43:0/1336640938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b84107fe0 0x7f7b84101cf0 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00efd0 tx=0x7f7b7c00c6a0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.222+0000 7f7b8b5b2640 1 -- 172.21.15.43:0/1336640938 shutdown_connections 2024-04-21T08:52:24.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.222+0000 7f7b8b5b2640 1 --2- 172.21.15.43:0/1336640938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b84107fe0 0x7f7b84101cf0 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.222+0000 7f7b8b5b2640 1 --2- 172.21.15.43:0/1336640938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b84069a80 0x7f7b841017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.222+0000 7f7b8b5b2640 1 --2- 172.21.15.43:0/1336640938 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f7b700a3da0 0x7f7b700a6260 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.222+0000 7f7b8b5b2640 1 --2- 172.21.15.43:0/1336640938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b840690e0 0x7f7b84101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.222+0000 7f7b8b5b2640 1 -- 172.21.15.43:0/1336640938 >> 172.21.15.43:0/1336640938 conn(0x7f7b84077e30 msgr2=0x7f7b84100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:24.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.222+0000 7f7b8b5b2640 1 -- 172.21.15.43:0/1336640938 shutdown_connections 2024-04-21T08:52:24.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.222+0000 7f7b8b5b2640 1 -- 172.21.15.43:0/1336640938 wait complete. 2024-04-21T08:52:24.233 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:39.981405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:22.034751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:22.780795+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 109, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:25.984261+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 109, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:52:24.234 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:52:24.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.340+0000 7f24f37ca640 1 -- 172.21.15.43:0/2769120951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ec1031d0 msgr2=0x7f24ec1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:24.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.340+0000 7f24f37ca640 1 --2- 172.21.15.43:0/2769120951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ec1031d0 0x7f24ec1035b0 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009ca0 tx=0x7f24e802c230 comp rx=0 tx=0).stop 2024-04-21T08:52:24.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.340+0000 7f24f37ca640 1 -- 172.21.15.43:0/2769120951 shutdown_connections 2024-04-21T08:52:24.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.340+0000 7f24f37ca640 1 --2- 172.21.15.43:0/2769120951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24ec10c510 0x7f24ec10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.340+0000 7f24f37ca640 1 --2- 172.21.15.43:0/2769120951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24ec103af0 0x7f24ec10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.340+0000 7f24f37ca640 1 --2- 172.21.15.43:0/2769120951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ec1031d0 0x7f24ec1035b0 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.340+0000 7f24f37ca640 1 -- 172.21.15.43:0/2769120951 >> 172.21.15.43:0/2769120951 conn(0x7f24ec077f50 msgr2=0x7f24ec078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:24.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.340+0000 7f24f37ca640 1 -- 172.21.15.43:0/2769120951 shutdown_connections 2024-04-21T08:52:24.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.340+0000 7f24f37ca640 1 -- 172.21.15.43:0/2769120951 wait complete. 2024-04-21T08:52:24.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.340+0000 7f24f37ca640 1 Processor -- start 2024-04-21T08:52:24.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.340+0000 7f24f37ca640 1 -- start start 2024-04-21T08:52:24.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.341+0000 7f24f37ca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ec1031d0 0x7f24ec19e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:24.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.341+0000 7f24f37ca640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24ec103af0 0x7f24ec19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:24.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.341+0000 7f24f37ca640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24ec10c510 0x7f24ec1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:24.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.341+0000 7f24f37ca640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec1121a0 con 0x7f24ec1031d0 2024-04-21T08:52:24.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.341+0000 7f24f37ca640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec112020 con 0x7f24ec103af0 2024-04-21T08:52:24.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.341+0000 7f24f37ca640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec112320 con 0x7f24ec10c510 2024-04-21T08:52:24.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.341+0000 7f24f1566640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ec1031d0 0x7f24ec19e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:24.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.341+0000 7f24f1566640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ec1031d0 0x7f24ec19e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56448/0 (socket says 172.21.15.43:56448) 2024-04-21T08:52:24.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.341+0000 7f24f1566640 1 -- 172.21.15.43:0/617802409 learned_addr learned my addr 172.21.15.43:0/617802409 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:52:24.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.341+0000 7f24f0d65640 1 --2- 172.21.15.43:0/617802409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24ec103af0 0x7f24ec19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:24.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.341+0000 7f24f1d67640 1 --2- 172.21.15.43:0/617802409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24ec10c510 0x7f24ec1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:24.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.341+0000 7f24f1566640 1 -- 172.21.15.43:0/617802409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24ec10c510 msgr2=0x7f24ec1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:24.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.341+0000 7f24f1566640 1 --2- 172.21.15.43:0/617802409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24ec10c510 0x7f24ec1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.341+0000 7f24f1566640 1 -- 172.21.15.43:0/617802409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24ec103af0 msgr2=0x7f24ec19e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:24.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.341+0000 7f24f1566640 1 --2- 172.21.15.43:0/617802409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24ec103af0 0x7f24ec19e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.341+0000 7f24f1566640 1 -- 172.21.15.43:0/617802409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ec198d30 con 0x7f24ec1031d0 2024-04-21T08:52:24.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.341+0000 7f24f1566640 1 --2- 172.21.15.43:0/617802409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ec1031d0 0x7f24ec19e400 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f24e8005e30 tx=0x7f24e8035ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:24.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.341+0000 7f24e27fc640 1 -- 172.21.15.43:0/617802409 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8039040 con 0x7f24ec1031d0 2024-04-21T08:52:24.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.341+0000 7f24f37ca640 1 -- 172.21.15.43:0/617802409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24ec198fc0 con 0x7f24ec1031d0 2024-04-21T08:52:24.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.341+0000 7f24e27fc640 1 -- 172.21.15.43:0/617802409 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24e8044070 con 0x7f24ec1031d0 2024-04-21T08:52:24.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.341+0000 7f24e27fc640 1 -- 172.21.15.43:0/617802409 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e80328a0 con 0x7f24ec1031d0 2024-04-21T08:52:24.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.341+0000 7f24f37ca640 1 -- 172.21.15.43:0/617802409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24ec1a52c0 con 0x7f24ec1031d0 2024-04-21T08:52:24.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.342+0000 7f24f37ca640 1 -- 172.21.15.43:0/617802409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24ec068d70 con 0x7f24ec1031d0 2024-04-21T08:52:24.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.344+0000 7f24e27fc640 1 -- 172.21.15.43:0/617802409 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f24e8032a40 con 0x7f24ec1031d0 2024-04-21T08:52:24.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.345+0000 7f24e27fc640 1 --2- 172.21.15.43:0/617802409 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f24d80a3d20 0x7f24d80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:24.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.345+0000 7f24e27fc640 1 -- 172.21.15.43:0/617802409 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f24e80f36b0 con 0x7f24ec1031d0 2024-04-21T08:52:24.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.345+0000 7f24f0d65640 1 --2- 172.21.15.43:0/617802409 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f24d80a3d20 0x7f24d80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:24.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.346+0000 7f24f0d65640 1 --2- 172.21.15.43:0/617802409 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f24d80a3d20 0x7f24d80a61e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f24ec10e3b0 tx=0x7f24dc00b390 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:24.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.349+0000 7f24e27fc640 1 -- 172.21.15.43:0/617802409 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f24e80bf9e0 con 0x7f24ec1031d0 2024-04-21T08:52:24.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:24 smithi175 ceph-mon[31920]: pgmap v422: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-21T08:52:24.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.536+0000 7f24f37ca640 1 -- 172.21.15.43:0/617802409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f24ec107690 con 0x7f24ec1031d0 2024-04-21T08:52:24.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.536+0000 7f24e27fc640 1 -- 172.21.15.43:0/617802409 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) ==== 93+0+10 (secure 0 0 0) 0x7f24e80bf2e0 con 0x7f24ec1031d0 2024-04-21T08:52:24.535 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-21T08:52:24.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.538+0000 7f24f37ca640 1 -- 172.21.15.43:0/617802409 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f24d80a3d20 msgr2=0x7f24d80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:24.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.538+0000 7f24f37ca640 1 --2- 172.21.15.43:0/617802409 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f24d80a3d20 0x7f24d80a61e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f24ec10e3b0 tx=0x7f24dc00b390 comp rx=0 tx=0).stop 2024-04-21T08:52:24.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.538+0000 7f24f37ca640 1 -- 172.21.15.43:0/617802409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ec1031d0 msgr2=0x7f24ec19e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:24.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.538+0000 7f24f37ca640 1 --2- 172.21.15.43:0/617802409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ec1031d0 0x7f24ec19e400 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f24e8005e30 tx=0x7f24e8035ea0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.538+0000 7f24f37ca640 1 -- 172.21.15.43:0/617802409 shutdown_connections 2024-04-21T08:52:24.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.538+0000 7f24f37ca640 1 --2- 172.21.15.43:0/617802409 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f24d80a3d20 0x7f24d80a61e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.538+0000 7f24f37ca640 1 --2- 172.21.15.43:0/617802409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24ec10c510 0x7f24ec1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.538+0000 7f24f37ca640 1 --2- 172.21.15.43:0/617802409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24ec103af0 0x7f24ec19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.538+0000 7f24f37ca640 1 --2- 172.21.15.43:0/617802409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ec1031d0 0x7f24ec19e400 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.538+0000 7f24f37ca640 1 -- 172.21.15.43:0/617802409 >> 172.21.15.43:0/617802409 conn(0x7f24ec077f50 msgr2=0x7f24ec0fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:24.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.538+0000 7f24f37ca640 1 -- 172.21.15.43:0/617802409 shutdown_connections 2024-04-21T08:52:24.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.538+0000 7f24f37ca640 1 -- 172.21.15.43:0/617802409 wait complete. 2024-04-21T08:52:24.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:24 smithi112 ceph-mon[31085]: pgmap v422: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-21T08:52:24.548 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:24 smithi043 ceph-mon[27978]: pgmap v422: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-21T08:52:24.549 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:52:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.654+0000 7f3de9db6640 1 -- 172.21.15.43:0/1337523601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3de4107260 msgr2=0x7f3de410ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.654+0000 7f3de9db6640 1 --2- 172.21.15.43:0/1337523601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3de4107260 0x7f3de410ee30 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009ca0 tx=0x7f3ddc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.654+0000 7f3de9db6640 1 -- 172.21.15.43:0/1337523601 shutdown_connections 2024-04-21T08:52:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.654+0000 7f3de9db6640 1 --2- 172.21.15.43:0/1337523601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3de4107260 0x7f3de410ee30 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.654+0000 7f3de9db6640 1 --2- 172.21.15.43:0/1337523601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3de4102b40 0x7f3de4106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.654+0000 7f3de9db6640 1 --2- 172.21.15.43:0/1337523601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3de4102190 0x7f3de4102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.654+0000 7f3de9db6640 1 -- 172.21.15.43:0/1337523601 >> 172.21.15.43:0/1337523601 conn(0x7f3de40fd990 msgr2=0x7f3de40ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.654+0000 7f3de9db6640 1 -- 172.21.15.43:0/1337523601 shutdown_connections 2024-04-21T08:52:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.654+0000 7f3de9db6640 1 -- 172.21.15.43:0/1337523601 wait complete. 2024-04-21T08:52:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.655+0000 7f3de9db6640 1 Processor -- start 2024-04-21T08:52:24.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.655+0000 7f3de9db6640 1 -- start start 2024-04-21T08:52:24.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.655+0000 7f3de9db6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3de4102190 0x7f3de419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:24.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.655+0000 7f3de9db6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3de4107260 0x7f3de419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:24.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.655+0000 7f3de9db6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3de41985a0 0x7f3de4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:24.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.655+0000 7f3de9db6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3de4112220 con 0x7f3de4102190 2024-04-21T08:52:24.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.655+0000 7f3de9db6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41120a0 con 0x7f3de4107260 2024-04-21T08:52:24.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.655+0000 7f3de9db6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41123a0 con 0x7f3de41985a0 2024-04-21T08:52:24.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.655+0000 7f3de37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3de4102190 0x7f3de419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:24.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.655+0000 7f3de37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3de4102190 0x7f3de419e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56460/0 (socket says 172.21.15.43:56460) 2024-04-21T08:52:24.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.655+0000 7f3de37fe640 1 -- 172.21.15.43:0/4173493035 learned_addr learned my addr 172.21.15.43:0/4173493035 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:52:24.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.656+0000 7f3de2ffd640 1 --2- 172.21.15.43:0/4173493035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3de4107260 0x7f3de419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:24.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.656+0000 7f3de37fe640 1 -- 172.21.15.43:0/4173493035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3de41985a0 msgr2=0x7f3de4198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:24.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.656+0000 7f3de3fff640 1 --2- 172.21.15.43:0/4173493035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3de41985a0 0x7f3de4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:24.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.656+0000 7f3de37fe640 1 --2- 172.21.15.43:0/4173493035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3de41985a0 0x7f3de4198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.656+0000 7f3de37fe640 1 -- 172.21.15.43:0/4173493035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3de4107260 msgr2=0x7f3de419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:24.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.656+0000 7f3de37fe640 1 --2- 172.21.15.43:0/4173493035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3de4107260 0x7f3de419e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.656+0000 7f3de37fe640 1 -- 172.21.15.43:0/4173493035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de41990c0 con 0x7f3de4102190 2024-04-21T08:52:24.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.656+0000 7f3de2ffd640 1 --2- 172.21.15.43:0/4173493035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3de4107260 0x7f3de419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:52:24.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.656+0000 7f3de3fff640 1 --2- 172.21.15.43:0/4173493035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3de41985a0 0x7f3de4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:52:24.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.656+0000 7f3de37fe640 1 --2- 172.21.15.43:0/4173493035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3de4102190 0x7f3de419e3f0 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400d950 tx=0x7f3dd400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:24.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.656+0000 7f3de0ff9640 1 -- 172.21.15.43:0/4173493035 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4017070 con 0x7f3de4102190 2024-04-21T08:52:24.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.656+0000 7f3de0ff9640 1 -- 172.21.15.43:0/4173493035 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dd4009c30 con 0x7f3de4102190 2024-04-21T08:52:24.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.656+0000 7f3de0ff9640 1 -- 172.21.15.43:0/4173493035 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd400f620 con 0x7f3de4102190 2024-04-21T08:52:24.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.656+0000 7f3de9db6640 1 -- 172.21.15.43:0/4173493035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de41993b0 con 0x7f3de4102190 2024-04-21T08:52:24.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.656+0000 7f3de9db6640 1 -- 172.21.15.43:0/4173493035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de41a5600 con 0x7f3de4102190 2024-04-21T08:52:24.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.657+0000 7f3de9db6640 1 -- 172.21.15.43:0/4173493035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3de40691c0 con 0x7f3de4102190 2024-04-21T08:52:24.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.658+0000 7f3de0ff9640 1 -- 172.21.15.43:0/4173493035 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3dd4010460 con 0x7f3de4102190 2024-04-21T08:52:24.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.659+0000 7f3de0ff9640 1 --2- 172.21.15.43:0/4173493035 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3dcc0a3d20 0x7f3dcc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:24.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.659+0000 7f3de2ffd640 1 --2- 172.21.15.43:0/4173493035 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3dcc0a3d20 0x7f3dcc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:24.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.659+0000 7f3de0ff9640 1 -- 172.21.15.43:0/4173493035 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f3dd40d08f0 con 0x7f3de4102190 2024-04-21T08:52:24.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.660+0000 7f3de2ffd640 1 --2- 172.21.15.43:0/4173493035 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3dcc0a3d20 0x7f3dcc0a61e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3de4108800 tx=0x7f3dd8008040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:24.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.663+0000 7f3de0ff9640 1 -- 172.21.15.43:0/4173493035 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3dd409cb70 con 0x7f3de4102190 2024-04-21T08:52:24.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.849+0000 7f3de9db6640 1 -- 172.21.15.43:0/4173493035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3de4102b40 con 0x7f3de4102190 2024-04-21T08:52:24.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.849+0000 7f3de0ff9640 1 -- 172.21.15.43:0/4173493035 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) ==== 104+0+11 (secure 0 0 0) 0x7f3dd4009de0 con 0x7f3de4102190 2024-04-21T08:52:24.849 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:52:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.851+0000 7f3de9db6640 1 -- 172.21.15.43:0/4173493035 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3dcc0a3d20 msgr2=0x7f3dcc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.851+0000 7f3de9db6640 1 --2- 172.21.15.43:0/4173493035 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3dcc0a3d20 0x7f3dcc0a61e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3de4108800 tx=0x7f3dd8008040 comp rx=0 tx=0).stop 2024-04-21T08:52:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.851+0000 7f3de9db6640 1 -- 172.21.15.43:0/4173493035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3de4102190 msgr2=0x7f3de419e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.851+0000 7f3de9db6640 1 --2- 172.21.15.43:0/4173493035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3de4102190 0x7f3de419e3f0 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400d950 tx=0x7f3dd400de10 comp rx=0 tx=0).stop 2024-04-21T08:52:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.851+0000 7f3de9db6640 1 -- 172.21.15.43:0/4173493035 shutdown_connections 2024-04-21T08:52:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.851+0000 7f3de9db6640 1 --2- 172.21.15.43:0/4173493035 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f3dcc0a3d20 0x7f3dcc0a61e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.851+0000 7f3de9db6640 1 --2- 172.21.15.43:0/4173493035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3de41985a0 0x7f3de4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.851+0000 7f3de9db6640 1 --2- 172.21.15.43:0/4173493035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3de4107260 0x7f3de419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.851+0000 7f3de9db6640 1 --2- 172.21.15.43:0/4173493035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3de4102190 0x7f3de419e3f0 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.851+0000 7f3de9db6640 1 -- 172.21.15.43:0/4173493035 >> 172.21.15.43:0/4173493035 conn(0x7f3de40fd990 msgr2=0x7f3de40fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:24.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.851+0000 7f3de9db6640 1 -- 172.21.15.43:0/4173493035 shutdown_connections 2024-04-21T08:52:24.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.851+0000 7f3de9db6640 1 -- 172.21.15.43:0/4173493035 wait complete. 2024-04-21T08:52:24.862 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:52:24.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.968+0000 7f6efd6e0640 1 -- 172.21.15.43:0/1311819104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ef8102350 msgr2=0x7f6ef810dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:24.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.968+0000 7f6efd6e0640 1 --2- 172.21.15.43:0/1311819104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ef8102350 0x7f6ef810dd00 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0009cd0 tx=0x7f6ef002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.968+0000 7f6efd6e0640 1 -- 172.21.15.43:0/1311819104 shutdown_connections 2024-04-21T08:52:24.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.968+0000 7f6efd6e0640 1 --2- 172.21.15.43:0/1311819104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ef8102350 0x7f6ef810dd00 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.968+0000 7f6efd6e0640 1 --2- 172.21.15.43:0/1311819104 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ef8101990 0x7f6ef8101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.968+0000 7f6efd6e0640 1 --2- 172.21.15.43:0/1311819104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef8105e20 0x7f6ef8106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.968+0000 7f6efd6e0640 1 -- 172.21.15.43:0/1311819104 >> 172.21.15.43:0/1311819104 conn(0x7f6ef80760e0 msgr2=0x7f6ef80764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:24.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.968+0000 7f6efd6e0640 1 -- 172.21.15.43:0/1311819104 shutdown_connections 2024-04-21T08:52:24.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.968+0000 7f6efd6e0640 1 -- 172.21.15.43:0/1311819104 wait complete. 2024-04-21T08:52:24.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.969+0000 7f6efd6e0640 1 Processor -- start 2024-04-21T08:52:24.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.969+0000 7f6efd6e0640 1 -- start start 2024-04-21T08:52:24.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.969+0000 7f6efd6e0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ef8101990 0x7f6ef8197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:24.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.969+0000 7f6efd6e0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ef8102350 0x7f6ef8197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:24.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.969+0000 7f6efd6e0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef8105e20 0x7f6ef819be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:24.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.969+0000 7f6efd6e0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef81110a0 con 0x7f6ef8101990 2024-04-21T08:52:24.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.969+0000 7f6efd6e0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef8110f20 con 0x7f6ef8105e20 2024-04-21T08:52:24.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.969+0000 7f6efd6e0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef8111220 con 0x7f6ef8102350 2024-04-21T08:52:24.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.969+0000 7f6ef6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ef8101990 0x7f6ef8197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:24.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.969+0000 7f6ef6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ef8101990 0x7f6ef8197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56468/0 (socket says 172.21.15.43:56468) 2024-04-21T08:52:24.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.969+0000 7f6ef6ffd640 1 -- 172.21.15.43:0/4115975852 learned_addr learned my addr 172.21.15.43:0/4115975852 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:52:24.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.969+0000 7f6eeffff640 1 --2- 172.21.15.43:0/4115975852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ef8102350 0x7f6ef8197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:24.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.969+0000 7f6ef77fe640 1 --2- 172.21.15.43:0/4115975852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef8105e20 0x7f6ef819be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:24.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.969+0000 7f6ef6ffd640 1 -- 172.21.15.43:0/4115975852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ef8102350 msgr2=0x7f6ef8197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:24.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.969+0000 7f6ef6ffd640 1 --2- 172.21.15.43:0/4115975852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ef8102350 0x7f6ef8197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.969+0000 7f6ef6ffd640 1 -- 172.21.15.43:0/4115975852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef8105e20 msgr2=0x7f6ef819be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:24.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.969+0000 7f6ef6ffd640 1 --2- 172.21.15.43:0/4115975852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef8105e20 0x7f6ef819be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:24.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.969+0000 7f6ef6ffd640 1 -- 172.21.15.43:0/4115975852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef819c550 con 0x7f6ef8101990 2024-04-21T08:52:24.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.969+0000 7f6ef6ffd640 1 --2- 172.21.15.43:0/4115975852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ef8101990 0x7f6ef8197560 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f6ee800e9c0 tx=0x7f6ee800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:24.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.970+0000 7f6ef4ff9640 1 -- 172.21.15.43:0/4115975852 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee800cde0 con 0x7f6ef8101990 2024-04-21T08:52:24.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.970+0000 7f6ef4ff9640 1 -- 172.21.15.43:0/4115975852 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ee80108e0 con 0x7f6ef8101990 2024-04-21T08:52:24.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.970+0000 7f6ef4ff9640 1 -- 172.21.15.43:0/4115975852 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8018960 con 0x7f6ef8101990 2024-04-21T08:52:24.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.970+0000 7f6efd6e0640 1 -- 172.21.15.43:0/4115975852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef819c840 con 0x7f6ef8101990 2024-04-21T08:52:24.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.970+0000 7f6efd6e0640 1 -- 172.21.15.43:0/4115975852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef81a40d0 con 0x7f6ef8101990 2024-04-21T08:52:24.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.971+0000 7f6efd6e0640 1 -- 172.21.15.43:0/4115975852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef80691c0 con 0x7f6ef8101990 2024-04-21T08:52:24.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.973+0000 7f6ef4ff9640 1 -- 172.21.15.43:0/4115975852 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6ee801f020 con 0x7f6ef8101990 2024-04-21T08:52:24.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.973+0000 7f6ef4ff9640 1 --2- 172.21.15.43:0/4115975852 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6ee40a3d20 0x7f6ee40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:24.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.973+0000 7f6eeffff640 1 --2- 172.21.15.43:0/4115975852 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6ee40a3d20 0x7f6ee40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:24.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.973+0000 7f6ef4ff9640 1 -- 172.21.15.43:0/4115975852 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f6ee80d16a0 con 0x7f6ef8101990 2024-04-21T08:52:24.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.974+0000 7f6eeffff640 1 --2- 172.21.15.43:0/4115975852 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6ee40a3d20 0x7f6ee40a61e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6ef80766e0 tx=0x7f6ee0005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:24.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:24.977+0000 7f6ef4ff9640 1 -- 172.21.15.43:0/4115975852 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ee8026070 con 0x7f6ef8101990 2024-04-21T08:52:25.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.164+0000 7f6efd6e0640 1 -- 172.21.15.43:0/4115975852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6ef802d5c0 con 0x7f6ef8101990 2024-04-21T08:52:25.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.164+0000 7f6ef4ff9640 1 -- 172.21.15.43:0/4115975852 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) ==== 100+0+11 (secure 0 0 0) 0x7f6ee809d920 con 0x7f6ef8101990 2024-04-21T08:52:25.164 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-21T08:52:25.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.167+0000 7f6efd6e0640 1 -- 172.21.15.43:0/4115975852 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6ee40a3d20 msgr2=0x7f6ee40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.167+0000 7f6efd6e0640 1 --2- 172.21.15.43:0/4115975852 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6ee40a3d20 0x7f6ee40a61e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6ef80766e0 tx=0x7f6ee0005f90 comp rx=0 tx=0).stop 2024-04-21T08:52:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.167+0000 7f6efd6e0640 1 -- 172.21.15.43:0/4115975852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ef8101990 msgr2=0x7f6ef8197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.167+0000 7f6efd6e0640 1 --2- 172.21.15.43:0/4115975852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ef8101990 0x7f6ef8197560 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f6ee800e9c0 tx=0x7f6ee800ee80 comp rx=0 tx=0).stop 2024-04-21T08:52:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.167+0000 7f6efd6e0640 1 -- 172.21.15.43:0/4115975852 shutdown_connections 2024-04-21T08:52:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.167+0000 7f6efd6e0640 1 --2- 172.21.15.43:0/4115975852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ef8105e20 0x7f6ef819be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.167+0000 7f6efd6e0640 1 --2- 172.21.15.43:0/4115975852 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f6ee40a3d20 0x7f6ee40a61e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.167+0000 7f6efd6e0640 1 --2- 172.21.15.43:0/4115975852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ef8102350 0x7f6ef8197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.167+0000 7f6efd6e0640 1 --2- 172.21.15.43:0/4115975852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ef8101990 0x7f6ef8197560 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.167+0000 7f6efd6e0640 1 -- 172.21.15.43:0/4115975852 >> 172.21.15.43:0/4115975852 conn(0x7f6ef80760e0 msgr2=0x7f6ef80fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.167+0000 7f6efd6e0640 1 -- 172.21.15.43:0/4115975852 shutdown_connections 2024-04-21T08:52:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.167+0000 7f6efd6e0640 1 -- 172.21.15.43:0/4115975852 wait complete. 2024-04-21T08:52:25.178 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:52:25.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.286+0000 7fd1a229a640 1 -- 172.21.15.43:0/4137008532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19c103050 msgr2=0x7fd19c103430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:25.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.286+0000 7fd1a229a640 1 --2- 172.21.15.43:0/4137008532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19c103050 0x7fd19c103430 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009900 tx=0x7fd18c02bcd0 comp rx=0 tx=0).stop 2024-04-21T08:52:25.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.286+0000 7fd1a229a640 1 -- 172.21.15.43:0/4137008532 shutdown_connections 2024-04-21T08:52:25.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.286+0000 7fd1a229a640 1 --2- 172.21.15.43:0/4137008532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19c10c570 0x7fd19c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:25.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.286+0000 7fd1a229a640 1 --2- 172.21.15.43:0/4137008532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd19c103970 0x7fd19c10c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:25.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.286+0000 7fd1a229a640 1 --2- 172.21.15.43:0/4137008532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19c103050 0x7fd19c103430 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:25.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.286+0000 7fd1a229a640 1 -- 172.21.15.43:0/4137008532 >> 172.21.15.43:0/4137008532 conn(0x7fd19c077fb0 msgr2=0x7fd19c0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:25.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.286+0000 7fd1a229a640 1 -- 172.21.15.43:0/4137008532 shutdown_connections 2024-04-21T08:52:25.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.286+0000 7fd1a229a640 1 -- 172.21.15.43:0/4137008532 wait complete. 2024-04-21T08:52:25.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.286+0000 7fd1a229a640 1 Processor -- start 2024-04-21T08:52:25.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.286+0000 7fd1a229a640 1 -- start start 2024-04-21T08:52:25.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.287+0000 7fd1a229a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd19c103050 0x7fd19c19e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:25.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.287+0000 7fd1a229a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19c103970 0x7fd19c19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:25.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.287+0000 7fd1a229a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19c10c570 0x7fd19c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:25.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.287+0000 7fd1a229a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1121b0 con 0x7fd19c103970 2024-04-21T08:52:25.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.287+0000 7fd1a229a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c112030 con 0x7fd19c103050 2024-04-21T08:52:25.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.287+0000 7fd1a229a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c112330 con 0x7fd19c10c570 2024-04-21T08:52:25.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.287+0000 7fd19b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19c103970 0x7fd19c19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:25.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.287+0000 7fd19b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19c103970 0x7fd19c19e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56498/0 (socket says 172.21.15.43:56498) 2024-04-21T08:52:25.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.287+0000 7fd19b7fe640 1 -- 172.21.15.43:0/3127217614 learned_addr learned my addr 172.21.15.43:0/3127217614 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:52:25.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.287+0000 7fd19bfff640 1 --2- 172.21.15.43:0/3127217614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd19c103050 0x7fd19c19e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:25.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.287+0000 7fd1a0837640 1 --2- 172.21.15.43:0/3127217614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19c10c570 0x7fd19c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:25.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.287+0000 7fd19b7fe640 1 -- 172.21.15.43:0/3127217614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19c10c570 msgr2=0x7fd19c198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:25.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.287+0000 7fd19b7fe640 1 --2- 172.21.15.43:0/3127217614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19c10c570 0x7fd19c198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:25.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.287+0000 7fd19b7fe640 1 -- 172.21.15.43:0/3127217614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd19c103050 msgr2=0x7fd19c19e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:25.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.287+0000 7fd19b7fe640 1 --2- 172.21.15.43:0/3127217614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd19c103050 0x7fd19c19e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:25.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.287+0000 7fd19b7fe640 1 -- 172.21.15.43:0/3127217614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19c198d90 con 0x7fd19c103970 2024-04-21T08:52:25.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.288+0000 7fd19b7fe640 1 --2- 172.21.15.43:0/3127217614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19c103970 0x7fd19c19e9a0 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7fd19000d950 tx=0x7fd19000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:25.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.288+0000 7fd1997fa640 1 -- 172.21.15.43:0/3127217614 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd190017030 con 0x7fd19c103970 2024-04-21T08:52:25.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.288+0000 7fd1a229a640 1 -- 172.21.15.43:0/3127217614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19c199080 con 0x7fd19c103970 2024-04-21T08:52:25.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.288+0000 7fd1997fa640 1 -- 172.21.15.43:0/3127217614 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd190004020 con 0x7fd19c103970 2024-04-21T08:52:25.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.288+0000 7fd1997fa640 1 -- 172.21.15.43:0/3127217614 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1900047a0 con 0x7fd19c103970 2024-04-21T08:52:25.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.288+0000 7fd1a229a640 1 -- 172.21.15.43:0/3127217614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd19c1a5320 con 0x7fd19c103970 2024-04-21T08:52:25.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.289+0000 7fd1a229a640 1 -- 172.21.15.43:0/3127217614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd19c0684e0 con 0x7fd19c103970 2024-04-21T08:52:25.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.290+0000 7fd1997fa640 1 -- 172.21.15.43:0/3127217614 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd190009a30 con 0x7fd19c103970 2024-04-21T08:52:25.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.290+0000 7fd1997fa640 1 --2- 172.21.15.43:0/3127217614 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd1840a3d20 0x7fd1840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:25.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.291+0000 7fd19bfff640 1 --2- 172.21.15.43:0/3127217614 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd1840a3d20 0x7fd1840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:25.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.291+0000 7fd1997fa640 1 -- 172.21.15.43:0/3127217614 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fd1900d0d00 con 0x7fd19c103970 2024-04-21T08:52:25.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.291+0000 7fd19bfff640 1 --2- 172.21.15.43:0/3127217614 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd1840a3d20 0x7fd1840a61e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd18c006010 tx=0x7fd18c005e40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:25.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.294+0000 7fd1997fa640 1 -- 172.21.15.43:0/3127217614 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd19009cf20 con 0x7fd19c103970 2024-04-21T08:52:25.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:25 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1336640938' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:52:25.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:25 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/617802409' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:52:25.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:25 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/4173493035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:52:25.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:25 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/4115975852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:52:25.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.479+0000 7fd1a229a640 1 -- 172.21.15.43:0/3127217614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd19c1076f0 con 0x7fd19c103970 2024-04-21T08:52:25.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.479+0000 7fd1997fa640 1 -- 172.21.15.43:0/3127217614 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v110) ==== 103+0+11 (secure 0 0 0) 0x7fd19009c820 con 0x7fd19c103970 2024-04-21T08:52:25.479 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-21T08:52:25.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.482+0000 7fd1a229a640 1 -- 172.21.15.43:0/3127217614 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd1840a3d20 msgr2=0x7fd1840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:25.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.482+0000 7fd1a229a640 1 --2- 172.21.15.43:0/3127217614 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd1840a3d20 0x7fd1840a61e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd18c006010 tx=0x7fd18c005e40 comp rx=0 tx=0).stop 2024-04-21T08:52:25.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.482+0000 7fd1a229a640 1 -- 172.21.15.43:0/3127217614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19c103970 msgr2=0x7fd19c19e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:25.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.482+0000 7fd1a229a640 1 --2- 172.21.15.43:0/3127217614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19c103970 0x7fd19c19e9a0 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7fd19000d950 tx=0x7fd19000de10 comp rx=0 tx=0).stop 2024-04-21T08:52:25.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.482+0000 7fd1a229a640 1 -- 172.21.15.43:0/3127217614 shutdown_connections 2024-04-21T08:52:25.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.482+0000 7fd1a229a640 1 --2- 172.21.15.43:0/3127217614 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd1840a3d20 0x7fd1840a61e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:25.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.482+0000 7fd1a229a640 1 --2- 172.21.15.43:0/3127217614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19c10c570 0x7fd19c198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:25.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.482+0000 7fd1a229a640 1 --2- 172.21.15.43:0/3127217614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19c103970 0x7fd19c19e9a0 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:25.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.482+0000 7fd1a229a640 1 --2- 172.21.15.43:0/3127217614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd19c103050 0x7fd19c19e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:25.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.482+0000 7fd1a229a640 1 -- 172.21.15.43:0/3127217614 >> 172.21.15.43:0/3127217614 conn(0x7fd19c077fb0 msgr2=0x7fd19c0ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:25.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.482+0000 7fd1a229a640 1 -- 172.21.15.43:0/3127217614 shutdown_connections 2024-04-21T08:52:25.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.482+0000 7fd1a229a640 1 -- 172.21.15.43:0/3127217614 wait complete. 2024-04-21T08:52:25.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:25 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1336640938' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:52:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:25 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/617802409' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:52:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:25 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/4173493035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:52:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:25 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/4115975852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:52:25.493 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-21T08:52:25.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:25 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1336640938' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:52:25.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:25 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/617802409' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:52:25.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:25 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/4173493035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:52:25.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:25 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/4115975852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:52:25.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.598+0000 7f8739df4640 1 -- 172.21.15.43:0/2879646867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f873406a840 msgr2=0x7f873406ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:25.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.598+0000 7f8739df4640 1 --2- 172.21.15.43:0/2879646867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f873406a840 0x7f873406ac20 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f8724009900 tx=0x7f872402bc30 comp rx=0 tx=0).stop 2024-04-21T08:52:25.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.598+0000 7f8739df4640 1 -- 172.21.15.43:0/2879646867 shutdown_connections 2024-04-21T08:52:25.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.598+0000 7f8739df4640 1 --2- 172.21.15.43:0/2879646867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8734107270 0x7f873410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:25.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.598+0000 7f8739df4640 1 --2- 172.21.15.43:0/2879646867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f873406b1f0 0x7f8734106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:25.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.598+0000 7f8739df4640 1 --2- 172.21.15.43:0/2879646867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f873406a840 0x7f873406ac20 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:25.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.598+0000 7f8739df4640 1 -- 172.21.15.43:0/2879646867 >> 172.21.15.43:0/2879646867 conn(0x7f8734077eb0 msgr2=0x7f87340782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:25.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.598+0000 7f8739df4640 1 -- 172.21.15.43:0/2879646867 shutdown_connections 2024-04-21T08:52:25.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.598+0000 7f8739df4640 1 -- 172.21.15.43:0/2879646867 wait complete. 2024-04-21T08:52:25.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.598+0000 7f8739df4640 1 Processor -- start 2024-04-21T08:52:25.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.599+0000 7f8739df4640 1 -- start start 2024-04-21T08:52:25.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.599+0000 7f8739df4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f873406a840 0x7f8734198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:25.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.599+0000 7f8739df4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8734107270 0x7f8734198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:25.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.599+0000 7f8739df4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f873419cfe0 0x7f873419d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:25.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.599+0000 7f8739df4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f87341121e0 con 0x7f873406a840 2024-04-21T08:52:25.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.599+0000 7f8739df4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8734112060 con 0x7f8734107270 2024-04-21T08:52:25.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.599+0000 7f8739df4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8734112360 con 0x7f873419cfe0 2024-04-21T08:52:25.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.599+0000 7f87337fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f873406a840 0x7f8734198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:25.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.599+0000 7f87337fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f873406a840 0x7f8734198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56514/0 (socket says 172.21.15.43:56514) 2024-04-21T08:52:25.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.599+0000 7f87337fe640 1 -- 172.21.15.43:0/1460695046 learned_addr learned my addr 172.21.15.43:0/1460695046 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:52:25.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.599+0000 7f8732ffd640 1 --2- 172.21.15.43:0/1460695046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8734107270 0x7f8734198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:25.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.599+0000 7f87337fe640 1 -- 172.21.15.43:0/1460695046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f873419cfe0 msgr2=0x7f873419d490 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:52:25.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.599+0000 7f87337fe640 1 --2- 172.21.15.43:0/1460695046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f873419cfe0 0x7f873419d490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:25.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.599+0000 7f87337fe640 1 -- 172.21.15.43:0/1460695046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8734107270 msgr2=0x7f8734198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:25.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.599+0000 7f87337fe640 1 --2- 172.21.15.43:0/1460695046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8734107270 0x7f8734198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:25.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.599+0000 7f87337fe640 1 -- 172.21.15.43:0/1460695046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f873419dad0 con 0x7f873406a840 2024-04-21T08:52:25.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.599+0000 7f87337fe640 1 --2- 172.21.15.43:0/1460695046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f873406a840 0x7f8734198720 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f8724005940 tx=0x7f872402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:25.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.600+0000 7f8730ff9640 1 -- 172.21.15.43:0/1460695046 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8724043020 con 0x7f873406a840 2024-04-21T08:52:25.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.600+0000 7f8739df4640 1 -- 172.21.15.43:0/1460695046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f873419dd60 con 0x7f873406a840 2024-04-21T08:52:25.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.600+0000 7f8730ff9640 1 -- 172.21.15.43:0/1460695046 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f872402cbe0 con 0x7f873406a840 2024-04-21T08:52:25.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.600+0000 7f8730ff9640 1 -- 172.21.15.43:0/1460695046 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8724003bf0 con 0x7f873406a840 2024-04-21T08:52:25.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.600+0000 7f8739df4640 1 -- 172.21.15.43:0/1460695046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87341a5580 con 0x7f873406a840 2024-04-21T08:52:25.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.601+0000 7f8739df4640 1 -- 172.21.15.43:0/1460695046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87340691c0 con 0x7f873406a840 2024-04-21T08:52:25.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.602+0000 7f8730ff9640 1 -- 172.21.15.43:0/1460695046 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8724046020 con 0x7f873406a840 2024-04-21T08:52:25.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.603+0000 7f8730ff9640 1 --2- 172.21.15.43:0/1460695046 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f871c0a3d20 0x7f871c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:25.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.603+0000 7f8732ffd640 1 --2- 172.21.15.43:0/1460695046 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f871c0a3d20 0x7f871c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:25.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.603+0000 7f8730ff9640 1 -- 172.21.15.43:0/1460695046 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f87240f2d90 con 0x7f873406a840 2024-04-21T08:52:25.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.603+0000 7f8732ffd640 1 --2- 172.21.15.43:0/1460695046 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f871c0a3d20 0x7f871c0a61e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f87341009c0 tx=0x7f872800b390 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:25.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.607+0000 7f8730ff9640 1 -- 172.21.15.43:0/1460695046 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f872404d020 con 0x7f873406a840 2024-04-21T08:52:25.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.807+0000 7f8739df4640 1 -- 172.21.15.43:0/1460695046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f873406b1f0 con 0x7f873406a840 2024-04-21T08:52:25.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.807+0000 7f8730ff9640 1 -- 172.21.15.43:0/1460695046 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v110) ==== 81+0+943 (secure 0 0 0) 0x7f87240bf120 con 0x7f873406a840 2024-04-21T08:52:25.807 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/126475132 2024-04-22T08:38:54.881882+0000 2024-04-21T08:52:25.807 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/4288951239 2024-04-22T08:38:54.881882+0000 2024-04-21T08:52:25.807 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6801/1577030772 2024-04-22T08:38:54.881882+0000 2024-04-21T08:52:25.807 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/2382670190 2024-04-22T08:38:54.881882+0000 2024-04-21T08:52:25.807 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/1916182222 2024-04-22T08:29:13.889526+0000 2024-04-21T08:52:25.808 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6801/3200866696 2024-04-22T08:29:13.889526+0000 2024-04-21T08:52:25.808 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6801/1559586579 2024-04-22T08:28:46.855699+0000 2024-04-21T08:52:25.808 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6800/3200866696 2024-04-22T08:29:13.889526+0000 2024-04-21T08:52:25.808 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/4047106671 2024-04-22T08:29:13.889526+0000 2024-04-21T08:52:25.808 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6800/1559586579 2024-04-22T08:28:46.855699+0000 2024-04-21T08:52:25.808 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/1174411284 2024-04-22T08:38:54.881882+0000 2024-04-21T08:52:25.808 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/347461706 2024-04-22T08:29:13.889526+0000 2024-04-21T08:52:25.808 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/887926369 2024-04-22T08:28:46.855699+0000 2024-04-21T08:52:25.808 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6800/1577030772 2024-04-22T08:38:54.881882+0000 2024-04-21T08:52:25.808 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/2598599313 2024-04-22T08:28:46.855699+0000 2024-04-21T08:52:25.808 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/2126312819 2024-04-22T08:28:46.855699+0000 2024-04-21T08:52:25.809 INFO:teuthology.orchestra.run.smithi043.stderr:listed 16 entries 2024-04-21T08:52:25.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.809+0000 7f8739df4640 1 -- 172.21.15.43:0/1460695046 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f871c0a3d20 msgr2=0x7f871c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:25.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.809+0000 7f8739df4640 1 --2- 172.21.15.43:0/1460695046 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f871c0a3d20 0x7f871c0a61e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f87341009c0 tx=0x7f872800b390 comp rx=0 tx=0).stop 2024-04-21T08:52:25.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.809+0000 7f8739df4640 1 -- 172.21.15.43:0/1460695046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f873406a840 msgr2=0x7f8734198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:25.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.809+0000 7f8739df4640 1 --2- 172.21.15.43:0/1460695046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f873406a840 0x7f8734198720 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f8724005940 tx=0x7f872402c7e0 comp rx=0 tx=0).stop 2024-04-21T08:52:25.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.809+0000 7f8739df4640 1 -- 172.21.15.43:0/1460695046 shutdown_connections 2024-04-21T08:52:25.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.809+0000 7f8739df4640 1 --2- 172.21.15.43:0/1460695046 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f871c0a3d20 0x7f871c0a61e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:25.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.809+0000 7f8739df4640 1 --2- 172.21.15.43:0/1460695046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f873419cfe0 0x7f873419d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:25.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.809+0000 7f8739df4640 1 --2- 172.21.15.43:0/1460695046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8734107270 0x7f8734198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:25.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.809+0000 7f8739df4640 1 --2- 172.21.15.43:0/1460695046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f873406a840 0x7f8734198720 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:25.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.809+0000 7f8739df4640 1 -- 172.21.15.43:0/1460695046 >> 172.21.15.43:0/1460695046 conn(0x7f8734077eb0 msgr2=0x7f87340ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:25.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.809+0000 7f8739df4640 1 -- 172.21.15.43:0/1460695046 shutdown_connections 2024-04-21T08:52:25.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:25.809+0000 7f8739df4640 1 -- 172.21.15.43:0/1460695046 wait complete. 2024-04-21T08:52:25.820 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-21T08:52:25.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:52:25.820 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-21T08:52:25.863 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-21T08:52:25.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:52:25.863 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-21T08:52:25.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:52:25.919 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-21T08:52:25.975 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-21T08:52:25.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:52:25.975 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-21T08:52:26.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:52:26.041 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2024-04-21T08:52:26.135 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-21T08:52:26.135 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-21T08:52:26.135 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T08:52:26.152 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20582 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/b08fddaba5765ba87b548af2c196ded4fb07a2f2e1038d0e4632e5ab3ec67b84/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/V2V6HI34RPW5E57TIY3WZA7TC7:/var/lib/containers/storage/overlay/l/ZKLLQPOSKVNCQT3DY2CURGGYN3,upperdir=/var/lib/containers/storage/overlay/b08fddaba5765ba87b548af2c196ded4fb07a2f2e1038d0e4632e5ab3ec67b84/diff,workdir=/var/lib/containers/storage/overlay/b08fddaba5765ba87b548af2c196ded4fb07a2f2e1038d0e4632e5ab3ec67b84/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9e5a6cb4ef143bb6438a72bf3c8816dcede626b6eea575f1da94354d8ec6f0eb/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/V2V6HI34RPW5E57TIY3WZA7TC7:/var/lib/containers/storage/overlay/l/ZKLLQPOSKVNCQT3DY2CURGGYN3,upperdir=/var/lib/containers/storage/overlay/9e5a6cb4ef143bb6438a72bf3c8816dcede626b6eea575f1da94354d8ec6f0eb/diff,workdir=/var/lib/containers/storage/overlay/9e5a6cb4ef143bb6438a72bf3c8816dcede626b6eea575f1da94354d8ec6f0eb/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/9dd6b0958e358768394bc17b4abf868f2d1b0a2ce5d0b1b84a4b2d5b7ded8a7b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/V2V6HI34RPW5E57TIY3WZA7TC7:/var/lib/containers/storage/overlay/l/ZKLLQPOSKVNCQT3DY2CURGGYN3,upperdir=/var/lib/containers/storage/overlay/9dd6b0958e358768394bc17b4abf868f2d1b0a2ce5d0b1b84a4b2d5b7ded8a7b/diff,workdir=/var/lib/containers/storage/overlay/9dd6b0958e358768394bc17b4abf868f2d1b0a2ce5d0b1b84a4b2d5b7ded8a7b/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6a22612a34f2d841c5cbbc80abe942b562af933c31ead184189df69a824ba9dd/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/V2V6HI34RPW5E57TIY3WZA7TC7:/var/lib/containers/storage/overlay/l/ZKLLQPOSKVNCQT3DY2CURGGYN3,upperdir=/var/lib/containers/storage/overlay/6a22612a34f2d841c5cbbc80abe942b562af933c31ead184189df69a824ba9dd/diff,workdir=/var/lib/containers/storage/overlay/6a22612a34f2d841c5cbbc80abe942b562af933c31ead184189df69a824ba9dd/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/4e0ed9591beba11b4255655ea5e24f3c8dd7e9384c15db4f64cb6ff543528bc2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/V2V6HI34RPW5E57TIY3WZA7TC7:/var/lib/containers/storage/overlay/l/ZKLLQPOSKVNCQT3DY2CURGGYN3,upperdir=/var/lib/containers/storage/overlay/4e0ed9591beba11b4255655ea5e24f3c8dd7e9384c15db4f64cb6ff543528bc2/diff,workdir=/var/lib/containers/storage/overlay/4e0ed9591beba11b4255655ea5e24f3c8dd7e9384c15db4f64cb6ff543528bc2/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/8cb5d07af860c312b7cb9e2be97c9cdfd861c608a19c27d69a471a0a0bce8255/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/V2V6HI34RPW5E57TIY3WZA7TC7:/var/lib/containers/storage/overlay/l/ZKLLQPOSKVNCQT3DY2CURGGYN3,upperdir=/var/lib/containers/storage/overlay/8cb5d07af860c312b7cb9e2be97c9cdfd861c608a19c27d69a471a0a0bce8255/diff,workdir=/var/lib/containers/storage/overlay/8cb5d07af860c312b7cb9e2be97c9cdfd861c608a19c27d69a471a0a0bce8255/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b9f083ffb478b4d3c9f0d2c3c632fba97773098b47382b6ff398ca387587e451/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/V2V6HI34RPW5E57TIY3WZA7TC7:/var/lib/containers/storage/overlay/l/ZKLLQPOSKVNCQT3DY2CURGGYN3,upperdir=/var/lib/containers/storage/overlay/b9f083ffb478b4d3c9f0d2c3c632fba97773098b47382b6ff398ca387587e451/diff,workdir=/var/lib/containers/storage/overlay/b9f083ffb478b4d3c9f0d2c3c632fba97773098b47382b6ff398ca387587e451/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/0c7cabb1986c131ba65b5f5c27dea23c3fe6d0601dfb429c3f6f9486700a73b3/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/V2V6HI34RPW5E57TIY3WZA7TC7:/var/lib/containers/storage/overlay/l/ZKLLQPOSKVNCQT3DY2CURGGYN3,upperdir=/var/lib/containers/storage/overlay/0c7cabb1986c131ba65b5f5c27dea23c3fe6d0601dfb429c3f6f9486700a73b3/diff,workdir=/var/lib/containers/storage/overlay/0c7cabb1986c131ba65b5f5c27dea23c3fe6d0601dfb429c3f6f9486700a73b3/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b873585374a1b6d64e64d0b4d813089c25bf3bac08e643efc98f3eec6c6d90b3/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/V2V6HI34RPW5E57TIY3WZA7TC7:/var/lib/containers/storage/overlay/l/ZKLLQPOSKVNCQT3DY2CURGGYN3,upperdir=/var/lib/containers/storage/overlay/b873585374a1b6d64e64d0b4d813089c25bf3bac08e643efc98f3eec6c6d90b3/diff,workdir=/var/lib/containers/storage/overlay/b873585374a1b6d64e64d0b4d813089c25bf3bac08e643efc98f3eec6c6d90b3/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ef28d4e35d4092323b098b4349965dd1e24a2b0afd28dcf34148df9fbd164ec6/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/V2V6HI34RPW5E57TIY3WZA7TC7:/var/lib/containers/storage/overlay/l/ZKLLQPOSKVNCQT3DY2CURGGYN3,upperdir=/var/lib/containers/storage/overlay/ef28d4e35d4092323b098b4349965dd1e24a2b0afd28dcf34148df9fbd164ec6/diff,workdir=/var/lib/containers/storage/overlay/ef28d4e35d4092323b098b4349965dd1e24a2b0afd28dcf34148df9fbd164ec6/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 2024-04-21T08:52:26.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:52:26.152 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-21T08:52:26.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:52:26.208 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2024-04-21T08:52:26.275 INFO:teuthology.orchestra.run.smithi043.stderr:Cannot find device "ceph-brx" 2024-04-21T08:52:26.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:52:26.277 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-21T08:52:26.287 INFO:tasks.cephadm:Teardown begin 2024-04-21T08:52:26.287 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-21T08:52:26.288 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:52:26.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.421+0000 7fd890431640 1 -- 172.21.15.43:0/3043577576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd888101b60 msgr2=0x7fd88810dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:26.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.421+0000 7fd890431640 1 --2- 172.21.15.43:0/3043577576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd888101b60 0x7fd88810dc90 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7fd880009cd0 tx=0x7fd88002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:52:26.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.421+0000 7fd890431640 1 -- 172.21.15.43:0/3043577576 shutdown_connections 2024-04-21T08:52:26.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.421+0000 7fd890431640 1 --2- 172.21.15.43:0/3043577576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd888101b60 0x7fd88810dc90 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:26.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.421+0000 7fd890431640 1 --2- 172.21.15.43:0/3043577576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8881011a0 0x7fd888101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:26.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.421+0000 7fd890431640 1 --2- 172.21.15.43:0/3043577576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd888105e30 0x7fd888106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:26.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.421+0000 7fd890431640 1 -- 172.21.15.43:0/3043577576 >> 172.21.15.43:0/3043577576 conn(0x7fd888077e30 msgr2=0x7fd888078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:26.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.421+0000 7fd890431640 1 -- 172.21.15.43:0/3043577576 shutdown_connections 2024-04-21T08:52:26.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.421+0000 7fd890431640 1 -- 172.21.15.43:0/3043577576 wait complete. 2024-04-21T08:52:26.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.422+0000 7fd890431640 1 Processor -- start 2024-04-21T08:52:26.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.422+0000 7fd890431640 1 -- start start 2024-04-21T08:52:26.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.422+0000 7fd890431640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8881011a0 0x7fd888198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:26.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.422+0000 7fd890431640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd888101b60 0x7fd888198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:26.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.422+0000 7fd890431640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd888105e30 0x7fd88819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:26.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.422+0000 7fd890431640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881110e0 con 0x7fd8881011a0 2024-04-21T08:52:26.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.422+0000 7fd890431640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd888110f60 con 0x7fd888105e30 2024-04-21T08:52:26.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.422+0000 7fd890431640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd888111260 con 0x7fd888101b60 2024-04-21T08:52:26.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.422+0000 7fd88e1cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8881011a0 0x7fd888198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:26.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.422+0000 7fd88e1cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8881011a0 0x7fd888198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56534/0 (socket says 172.21.15.43:56534) 2024-04-21T08:52:26.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.422+0000 7fd88e1cd640 1 -- 172.21.15.43:0/1299181219 learned_addr learned my addr 172.21.15.43:0/1299181219 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:52:26.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.422+0000 7fd88d9cc640 1 --2- 172.21.15.43:0/1299181219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd888101b60 0x7fd888198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:26.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.422+0000 7fd88e9ce640 1 --2- 172.21.15.43:0/1299181219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd888105e30 0x7fd88819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:26.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.422+0000 7fd88e1cd640 1 -- 172.21.15.43:0/1299181219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd888101b60 msgr2=0x7fd888198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:26.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.422+0000 7fd88e1cd640 1 --2- 172.21.15.43:0/1299181219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd888101b60 0x7fd888198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:26.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.422+0000 7fd88e1cd640 1 -- 172.21.15.43:0/1299181219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd888105e30 msgr2=0x7fd88819cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:26.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.422+0000 7fd88e1cd640 1 --2- 172.21.15.43:0/1299181219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd888105e30 0x7fd88819cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:26.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.422+0000 7fd88e1cd640 1 -- 172.21.15.43:0/1299181219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88819d620 con 0x7fd8881011a0 2024-04-21T08:52:26.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.422+0000 7fd88e1cd640 1 --2- 172.21.15.43:0/1299181219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8881011a0 0x7fd888198680 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7fd88400c920 tx=0x7fd88400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:26.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.423+0000 7fd87f7fe640 1 -- 172.21.15.43:0/1299181219 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884007c70 con 0x7fd8881011a0 2024-04-21T08:52:26.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.423+0000 7fd87f7fe640 1 -- 172.21.15.43:0/1299181219 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd884007e10 con 0x7fd8881011a0 2024-04-21T08:52:26.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.423+0000 7fd890431640 1 -- 172.21.15.43:0/1299181219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88819d910 con 0x7fd8881011a0 2024-04-21T08:52:26.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.423+0000 7fd87f7fe640 1 -- 172.21.15.43:0/1299181219 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884018820 con 0x7fd8881011a0 2024-04-21T08:52:26.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.423+0000 7fd890431640 1 -- 172.21.15.43:0/1299181219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8881a51a0 con 0x7fd8881011a0 2024-04-21T08:52:26.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.424+0000 7fd890431640 1 -- 172.21.15.43:0/1299181219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd888068de0 con 0x7fd8881011a0 2024-04-21T08:52:26.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.425+0000 7fd87f7fe640 1 -- 172.21.15.43:0/1299181219 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd88400f450 con 0x7fd8881011a0 2024-04-21T08:52:26.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.425+0000 7fd87f7fe640 1 --2- 172.21.15.43:0/1299181219 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd8740a3d20 0x7fd8740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:26.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.426+0000 7fd88d9cc640 1 --2- 172.21.15.43:0/1299181219 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd8740a3d20 0x7fd8740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:26.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.426+0000 7fd87f7fe640 1 -- 172.21.15.43:0/1299181219 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fd8840d1050 con 0x7fd8881011a0 2024-04-21T08:52:26.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.426+0000 7fd88d9cc640 1 --2- 172.21.15.43:0/1299181219 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd8740a3d20 0x7fd8740a61e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd8880fe530 tx=0x7fd87800b390 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:26.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.429+0000 7fd87f7fe640 1 -- 172.21.15.43:0/1299181219 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd884014030 con 0x7fd8881011a0 2024-04-21T08:52:26.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:26 smithi175 ceph-mon[31920]: pgmap v423: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 101 B/s wr, 10 op/s 2024-04-21T08:52:26.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:26 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/3127217614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:52:26.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:26 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1460695046' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:52:26.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:26 smithi112 ceph-mon[31085]: pgmap v423: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 101 B/s wr, 10 op/s 2024-04-21T08:52:26.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:26 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/3127217614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:52:26.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:26 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1460695046' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:52:26.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:26 smithi043 ceph-mon[27978]: pgmap v423: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 101 B/s wr, 10 op/s 2024-04-21T08:52:26.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:26 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/3127217614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:52:26.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:26 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1460695046' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:52:26.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.658+0000 7fd890431640 1 -- 172.21.15.43:0/1299181219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd88810a390 con 0x7fd8881011a0 2024-04-21T08:52:26.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.659+0000 7fd87f7fe640 1 -- 172.21.15.43:0/1299181219 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11935 (secure 0 0 0) 0x7fd88409d300 con 0x7fd8881011a0 2024-04-21T08:52:26.659 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:52:26.659 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:46:17.283806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27556":{"gid":27556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/229985237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":229985237},{"type":"v1","addr":"172.21.15.175:6837","nonce":229985237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24425,24412,24487,24527,24491]},"id":1}]} 2024-04-21T08:52:26.659 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-04-21T08:52:26.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.661+0000 7fd890431640 1 -- 172.21.15.43:0/1299181219 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd8740a3d20 msgr2=0x7fd8740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:26.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.661+0000 7fd890431640 1 --2- 172.21.15.43:0/1299181219 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd8740a3d20 0x7fd8740a61e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd8880fe530 tx=0x7fd87800b390 comp rx=0 tx=0).stop 2024-04-21T08:52:26.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.661+0000 7fd890431640 1 -- 172.21.15.43:0/1299181219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8881011a0 msgr2=0x7fd888198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:26.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.661+0000 7fd890431640 1 --2- 172.21.15.43:0/1299181219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8881011a0 0x7fd888198680 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7fd88400c920 tx=0x7fd88400cde0 comp rx=0 tx=0).stop 2024-04-21T08:52:26.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.661+0000 7fd890431640 1 -- 172.21.15.43:0/1299181219 shutdown_connections 2024-04-21T08:52:26.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.661+0000 7fd890431640 1 --2- 172.21.15.43:0/1299181219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd888105e30 0x7fd88819cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:26.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.661+0000 7fd890431640 1 --2- 172.21.15.43:0/1299181219 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fd8740a3d20 0x7fd8740a61e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:26.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.661+0000 7fd890431640 1 --2- 172.21.15.43:0/1299181219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd888101b60 0x7fd888198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:26.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.661+0000 7fd890431640 1 --2- 172.21.15.43:0/1299181219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8881011a0 0x7fd888198680 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:26.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.661+0000 7fd890431640 1 -- 172.21.15.43:0/1299181219 >> 172.21.15.43:0/1299181219 conn(0x7fd888077e30 msgr2=0x7fd888069360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:26.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.661+0000 7fd890431640 1 -- 172.21.15.43:0/1299181219 shutdown_connections 2024-04-21T08:52:26.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.661+0000 7fd890431640 1 -- 172.21.15.43:0/1299181219 wait complete. 2024-04-21T08:52:26.672 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:52:26.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.775+0000 7f62a4315640 1 -- 172.21.15.43:0/1636099015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f629c107270 msgr2=0x7f629c10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:26.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.775+0000 7f62a4315640 1 --2- 172.21.15.43:0/1636099015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f629c107270 0x7f629c10ee40 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f6294009ca0 tx=0x7f629402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:52:26.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.775+0000 7f62a4315640 1 -- 172.21.15.43:0/1636099015 shutdown_connections 2024-04-21T08:52:26.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.775+0000 7f62a4315640 1 --2- 172.21.15.43:0/1636099015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f629c107270 0x7f629c10ee40 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:26.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.775+0000 7f62a4315640 1 --2- 172.21.15.43:0/1636099015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f629c102b50 0x7f629c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:26.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.775+0000 7f62a4315640 1 --2- 172.21.15.43:0/1636099015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f629c1021a0 0x7f629c102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:26.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.775+0000 7f62a4315640 1 -- 172.21.15.43:0/1636099015 >> 172.21.15.43:0/1636099015 conn(0x7f629c0793a0 msgr2=0x7f629c0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:26.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.775+0000 7f62a4315640 1 -- 172.21.15.43:0/1636099015 shutdown_connections 2024-04-21T08:52:26.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.775+0000 7f62a4315640 1 -- 172.21.15.43:0/1636099015 wait complete. 2024-04-21T08:52:26.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.775+0000 7f62a4315640 1 Processor -- start 2024-04-21T08:52:26.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.775+0000 7f62a4315640 1 -- start start 2024-04-21T08:52:26.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.776+0000 7f62a4315640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f629c1021a0 0x7f629c198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:26.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.776+0000 7f62a4315640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f629c107270 0x7f629c198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:26.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.776+0000 7f62a4315640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f629c19cec0 0x7f629c19d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:26.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.776+0000 7f62a4315640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f629c1121c0 con 0x7f629c107270 2024-04-21T08:52:26.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.776+0000 7f62a4315640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f629c112040 con 0x7f629c1021a0 2024-04-21T08:52:26.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.776+0000 7f62a4315640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f629c112340 con 0x7f629c19cec0 2024-04-21T08:52:26.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.776+0000 7f62a18b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f629c107270 0x7f629c198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:26.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.776+0000 7f62a18b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f629c107270 0x7f629c198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53796/0 (socket says 172.21.15.43:53796) 2024-04-21T08:52:26.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.776+0000 7f62a18b0640 1 -- 172.21.15.43:0/1275658751 learned_addr learned my addr 172.21.15.43:0/1275658751 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:52:26.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.776+0000 7f62a28b2640 1 --2- 172.21.15.43:0/1275658751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f629c19cec0 0x7f629c19d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:26.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.776+0000 7f62a20b1640 1 --2- 172.21.15.43:0/1275658751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f629c1021a0 0x7f629c198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:26.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.776+0000 7f62a18b0640 1 -- 172.21.15.43:0/1275658751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f629c19cec0 msgr2=0x7f629c19d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:26.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.776+0000 7f62a18b0640 1 --2- 172.21.15.43:0/1275658751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f629c19cec0 0x7f629c19d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:26.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.776+0000 7f62a18b0640 1 -- 172.21.15.43:0/1275658751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f629c1021a0 msgr2=0x7f629c198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:26.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.776+0000 7f62a18b0640 1 --2- 172.21.15.43:0/1275658751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f629c1021a0 0x7f629c198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:26.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.776+0000 7f62a18b0640 1 -- 172.21.15.43:0/1275658751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f629c19d9b0 con 0x7f629c107270 2024-04-21T08:52:26.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.776+0000 7f62a18b0640 1 --2- 172.21.15.43:0/1275658751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f629c107270 0x7f629c198b40 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f629800c970 tx=0x7f629800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:26.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.776+0000 7f62937fe640 1 -- 172.21.15.43:0/1275658751 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6298014070 con 0x7f629c107270 2024-04-21T08:52:26.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.776+0000 7f62937fe640 1 -- 172.21.15.43:0/1275658751 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6298004440 con 0x7f629c107270 2024-04-21T08:52:26.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.776+0000 7f62a4315640 1 -- 172.21.15.43:0/1275658751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f629c19dca0 con 0x7f629c107270 2024-04-21T08:52:26.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.776+0000 7f62937fe640 1 -- 172.21.15.43:0/1275658751 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6298003e50 con 0x7f629c107270 2024-04-21T08:52:26.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.776+0000 7f62a4315640 1 -- 172.21.15.43:0/1275658751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f629c1a5490 con 0x7f629c107270 2024-04-21T08:52:26.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.777+0000 7f62a4315640 1 -- 172.21.15.43:0/1275658751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f629c069230 con 0x7f629c107270 2024-04-21T08:52:26.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.778+0000 7f62937fe640 1 -- 172.21.15.43:0/1275658751 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6298017070 con 0x7f629c107270 2024-04-21T08:52:26.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.779+0000 7f62937fe640 1 --2- 172.21.15.43:0/1275658751 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f62880a3ec0 0x7f62880a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:26.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.779+0000 7f62a20b1640 1 --2- 172.21.15.43:0/1275658751 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f62880a3ec0 0x7f62880a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:26.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.779+0000 7f62937fe640 1 -- 172.21.15.43:0/1275658751 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f62980d0720 con 0x7f629c107270 2024-04-21T08:52:26.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.779+0000 7f62a20b1640 1 --2- 172.21.15.43:0/1275658751 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f62880a3ec0 0x7f62880a6380 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f628c005fd0 tx=0x7f628c005850 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:26.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:26.783+0000 7f62937fe640 1 -- 172.21.15.43:0/1275658751 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f629801e030 con 0x7f629c107270 2024-04-21T08:52:27.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.009+0000 7f62a4315640 1 -- 172.21.15.43:0/1275658751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f629c102d30 con 0x7f629c107270 2024-04-21T08:52:27.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.009+0000 7f62937fe640 1 -- 172.21.15.43:0/1275658751 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11935 (secure 0 0 0) 0x7f629809c9a0 con 0x7f629c107270 2024-04-21T08:52:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:52:27.010 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3697998894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3697998894},{"type":"v1","addr":"172.21.15.43:6835","nonce":3697998894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1236755039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1236755039},{"type":"v1","addr":"172.21.15.43:6837","nonce":1236755039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:23.821253+0000","modified":"2024-04-21T08:46:17.283806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24491,"mds_2":24487,"mds_3":24425,"mds_4":24412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6839/931821965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":931821965},{"type":"v1","addr":"172.21.15.43:6839","nonce":931821965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/623279035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":623279035},{"type":"v1","addr":"172.21.15.43:6841","nonce":623279035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.175:6833/187945366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":187945366},{"type":"v1","addr":"172.21.15.175:6833","nonce":187945366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.112:6833/1516781885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":1516781885},{"type":"v1","addr":"172.21.15.112:6833","nonce":1516781885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/466532477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":466532477},{"type":"v1","addr":"172.21.15.175:6835","nonce":466532477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/3159749962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3159749962},{"type":"v1","addr":"172.21.15.112:6835","nonce":3159749962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.175:6839/3664054463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3664054463},{"type":"v1","addr":"172.21.15.175:6839","nonce":3664054463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6837/3054508488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3054508488},{"type":"v1","addr":"172.21.15.112:6837","nonce":3054508488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/2143575300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2143575300},{"type":"v1","addr":"172.21.15.112:6839","nonce":2143575300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27556":{"gid":27556,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/229985237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":229985237},{"type":"v1","addr":"172.21.15.175:6837","nonce":229985237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24425,24412,24487,24527,24491]},"id":1}]} 2024-04-21T08:52:27.010 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-04-21T08:52:27.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.011+0000 7f62a4315640 1 -- 172.21.15.43:0/1275658751 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f62880a3ec0 msgr2=0x7f62880a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:27.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.011+0000 7f62a4315640 1 --2- 172.21.15.43:0/1275658751 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f62880a3ec0 0x7f62880a6380 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f628c005fd0 tx=0x7f628c005850 comp rx=0 tx=0).stop 2024-04-21T08:52:27.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.011+0000 7f62a4315640 1 -- 172.21.15.43:0/1275658751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f629c107270 msgr2=0x7f629c198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:27.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.011+0000 7f62a4315640 1 --2- 172.21.15.43:0/1275658751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f629c107270 0x7f629c198b40 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f629800c970 tx=0x7f629800ce30 comp rx=0 tx=0).stop 2024-04-21T08:52:27.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.012+0000 7f62a4315640 1 -- 172.21.15.43:0/1275658751 shutdown_connections 2024-04-21T08:52:27.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.012+0000 7f62a4315640 1 --2- 172.21.15.43:0/1275658751 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f62880a3ec0 0x7f62880a6380 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.012+0000 7f62a4315640 1 --2- 172.21.15.43:0/1275658751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f629c19cec0 0x7f629c19d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.012+0000 7f62a4315640 1 --2- 172.21.15.43:0/1275658751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f629c107270 0x7f629c198b40 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.012+0000 7f62a4315640 1 --2- 172.21.15.43:0/1275658751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f629c1021a0 0x7f629c198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.012+0000 7f62a4315640 1 -- 172.21.15.43:0/1275658751 >> 172.21.15.43:0/1275658751 conn(0x7f629c0793a0 msgr2=0x7f629c0ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:27.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.012+0000 7f62a4315640 1 -- 172.21.15.43:0/1275658751 shutdown_connections 2024-04-21T08:52:27.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.012+0000 7f62a4315640 1 -- 172.21.15.43:0/1275658751 wait complete. 2024-04-21T08:52:27.023 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:52:27.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.126+0000 7ff823985640 1 -- 172.21.15.43:0/2923660682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c103af0 msgr2=0x7ff81c10bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:27.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.126+0000 7ff823985640 1 --2- 172.21.15.43:0/2923660682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c103af0 0x7ff81c10bfd0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7ff80c009900 tx=0x7ff80c02bcd0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.126+0000 7ff823985640 1 -- 172.21.15.43:0/2923660682 shutdown_connections 2024-04-21T08:52:27.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.126+0000 7ff823985640 1 --2- 172.21.15.43:0/2923660682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff81c10c5a0 0x7ff81c10e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.126+0000 7ff823985640 1 --2- 172.21.15.43:0/2923660682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c103af0 0x7ff81c10bfd0 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.126+0000 7ff823985640 1 --2- 172.21.15.43:0/2923660682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff81c1031d0 0x7ff81c1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.126+0000 7ff823985640 1 -- 172.21.15.43:0/2923660682 >> 172.21.15.43:0/2923660682 conn(0x7ff81c0771d0 msgr2=0x7ff81c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:27.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.126+0000 7ff823985640 1 -- 172.21.15.43:0/2923660682 shutdown_connections 2024-04-21T08:52:27.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.126+0000 7ff823985640 1 -- 172.21.15.43:0/2923660682 wait complete. 2024-04-21T08:52:27.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.126+0000 7ff823985640 1 Processor -- start 2024-04-21T08:52:27.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.126+0000 7ff823985640 1 -- start start 2024-04-21T08:52:27.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.127+0000 7ff823985640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c1031d0 0x7ff81c1a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:27.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.127+0000 7ff823985640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff81c103af0 0x7ff81c1a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:27.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.127+0000 7ff823985640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff81c10c5a0 0x7ff81c19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:27.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.127+0000 7ff823985640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1121e0 con 0x7ff81c1031d0 2024-04-21T08:52:27.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.127+0000 7ff823985640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c112060 con 0x7ff81c103af0 2024-04-21T08:52:27.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.127+0000 7ff823985640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c112360 con 0x7ff81c10c5a0 2024-04-21T08:52:27.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.127+0000 7ff821721640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c1031d0 0x7ff81c1a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:27.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.127+0000 7ff821721640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c1031d0 0x7ff81c1a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53804/0 (socket says 172.21.15.43:53804) 2024-04-21T08:52:27.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.127+0000 7ff821721640 1 -- 172.21.15.43:0/174841395 learned_addr learned my addr 172.21.15.43:0/174841395 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:52:27.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.127+0000 7ff821f22640 1 --2- 172.21.15.43:0/174841395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff81c10c5a0 0x7ff81c19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:27.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.127+0000 7ff821721640 1 -- 172.21.15.43:0/174841395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff81c10c5a0 msgr2=0x7ff81c19a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:27.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.127+0000 7ff821721640 1 --2- 172.21.15.43:0/174841395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff81c10c5a0 0x7ff81c19a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.127+0000 7ff821721640 1 -- 172.21.15.43:0/174841395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff81c103af0 msgr2=0x7ff81c1a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:27.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.127+0000 7ff821721640 1 --2- 172.21.15.43:0/174841395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff81c103af0 0x7ff81c1a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.127+0000 7ff821721640 1 -- 172.21.15.43:0/174841395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81c19aff0 con 0x7ff81c1031d0 2024-04-21T08:52:27.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.127+0000 7ff821f22640 1 --2- 172.21.15.43:0/174841395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff81c10c5a0 0x7ff81c19a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:52:27.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.127+0000 7ff821721640 1 --2- 172.21.15.43:0/174841395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c1031d0 0x7ff81c1a05c0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7ff81800d950 tx=0x7ff81800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:27.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.127+0000 7ff8127fc640 1 -- 172.21.15.43:0/174841395 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818017070 con 0x7ff81c1031d0 2024-04-21T08:52:27.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.127+0000 7ff8127fc640 1 -- 172.21.15.43:0/174841395 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff818009c30 con 0x7ff81c1031d0 2024-04-21T08:52:27.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.127+0000 7ff823985640 1 -- 172.21.15.43:0/174841395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff81c19b2e0 con 0x7ff81c1031d0 2024-04-21T08:52:27.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.127+0000 7ff8127fc640 1 -- 172.21.15.43:0/174841395 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81800f620 con 0x7ff81c1031d0 2024-04-21T08:52:27.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.127+0000 7ff823985640 1 -- 172.21.15.43:0/174841395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff81c1a7410 con 0x7ff81c1031d0 2024-04-21T08:52:27.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.128+0000 7ff823985640 1 -- 172.21.15.43:0/174841395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff81c06ae60 con 0x7ff81c1031d0 2024-04-21T08:52:27.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.129+0000 7ff8127fc640 1 -- 172.21.15.43:0/174841395 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff818010460 con 0x7ff81c1031d0 2024-04-21T08:52:27.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.130+0000 7ff8127fc640 1 --2- 172.21.15.43:0/174841395 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff8080a3d20 0x7ff8080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:27.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.130+0000 7ff8127fc640 1 -- 172.21.15.43:0/174841395 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7ff8180d0910 con 0x7ff81c1031d0 2024-04-21T08:52:27.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.130+0000 7ff820f20640 1 --2- 172.21.15.43:0/174841395 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff8080a3d20 0x7ff8080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:27.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.130+0000 7ff820f20640 1 --2- 172.21.15.43:0/174841395 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff8080a3d20 0x7ff8080a61e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff80c0096c0 tx=0x7ff80c000bc0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:27.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.135+0000 7ff8127fc640 1 -- 172.21.15.43:0/174841395 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff81809cb90 con 0x7ff81c1031d0 2024-04-21T08:52:27.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:27 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1299181219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:52:27.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:27 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1275658751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:52:27.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.325+0000 7ff823985640 1 -- 172.21.15.43:0/174841395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff81c1a77a0 con 0x7ff81c1031d0 2024-04-21T08:52:27.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.326+0000 7ff8127fc640 1 -- 172.21.15.43:0/174841395 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+25419 (secure 0 0 0) 0x7ff818009de0 con 0x7ff81c1031d0 2024-04-21T08:52:27.326 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-21T08:52:27.326 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":110,"fsid":"d57a7d98-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:11.976386+0000","modified":"2024-04-21T08:52:15.080041+0000","last_up_change":"2024-04-21T08:35:32.546332+0000","last_in_change":"2024-04-21T08:35:12.799792+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:39.981405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:22.034751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:22.780795+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:25.984261+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"070c2bde-f131-4523-b54c-d376a5015826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6803","nonce":4223002768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6805","nonce":4223002768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6809","nonce":4223002768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4223002768},{"type":"v1","addr":"172.21.15.43:6807","nonce":4223002768}]},"public_addr":"172.21.15.43:6803/4223002768","cluster_addr":"172.21.15.43:6805/4223002768","heartbeat_back_addr":"172.21.15.43:6809/4223002768","heartbeat_front_addr":"172.21.15.43:6807/4223002768","state":["exists","up"]},{"osd":1,"uuid":"f297cc81-3025-40c8-8081-1b201be2c4bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6801","nonce":2362092039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6803","nonce":2362092039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6807","nonce":2362092039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2362092039},{"type":"v1","addr":"172.21.15.112:6805","nonce":2362092039}]},"public_addr":"172.21.15.112:6801/2362092039","cluster_addr":"172.21.15.112:6803/2362092039","heartbeat_back_addr":"172.21.15.112:6807/2362092039","heartbeat_front_addr":"172.21.15.112:6805/2362092039","state":["exists","up"]},{"osd":2,"uuid":"ca0c4942-852b-415e-9a75-8645f0435202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6801","nonce":2080591921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6803","nonce":2080591921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6807","nonce":2080591921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2080591921},{"type":"v1","addr":"172.21.15.175:6805","nonce":2080591921}]},"public_addr":"172.21.15.175:6801/2080591921","cluster_addr":"172.21.15.175:6803/2080591921","heartbeat_back_addr":"172.21.15.175:6807/2080591921","heartbeat_front_addr":"172.21.15.175:6805/2080591921","state":["exists","up"]},{"osd":3,"uuid":"e2c6a01e-1ff7-416d-8d5e-6f3a736f58c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6811","nonce":117814050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6813","nonce":117814050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6817","nonce":117814050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":117814050},{"type":"v1","addr":"172.21.15.43:6815","nonce":117814050}]},"public_addr":"172.21.15.43:6811/117814050","cluster_addr":"172.21.15.43:6813/117814050","heartbeat_back_addr":"172.21.15.43:6817/117814050","heartbeat_front_addr":"172.21.15.43:6815/117814050","state":["exists","up"]},{"osd":4,"uuid":"87d594ea-199e-42db-8e47-2d535cbca3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6809","nonce":3007534004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6811","nonce":3007534004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6815","nonce":3007534004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3007534004},{"type":"v1","addr":"172.21.15.112:6813","nonce":3007534004}]},"public_addr":"172.21.15.112:6809/3007534004","cluster_addr":"172.21.15.112:6811/3007534004","heartbeat_back_addr":"172.21.15.112:6815/3007534004","heartbeat_front_addr":"172.21.15.112:6813/3007534004","state":["exists","up"]},{"osd":5,"uuid":"93b83dce-3ad8-40ef-b472-5612b41fd044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6809","nonce":1901076703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6811","nonce":1901076703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6815","nonce":1901076703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1901076703},{"type":"v1","addr":"172.21.15.175:6813","nonce":1901076703}]},"public_addr":"172.21.15.175:6809/1901076703","cluster_addr":"172.21.15.175:6811/1901076703","heartbeat_back_addr":"172.21.15.175:6815/1901076703","heartbeat_front_addr":"172.21.15.175:6813/1901076703","state":["exists","up"]},{"osd":6,"uuid":"3abceb62-32a7-48ae-a367-41869d2471cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6819","nonce":4220470202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6821","nonce":4220470202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6825","nonce":4220470202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4220470202},{"type":"v1","addr":"172.21.15.43:6823","nonce":4220470202}]},"public_addr":"172.21.15.43:6819/4220470202","cluster_addr":"172.21.15.43:6821/4220470202","heartbeat_back_addr":"172.21.15.43:6825/4220470202","heartbeat_front_addr":"172.21.15.43:6823/4220470202","state":["exists","up"]},{"osd":7,"uuid":"223d7e91-e4a6-42e0-b5fb-bbbf4844efd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6817","nonce":1090541705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6819","nonce":1090541705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6823","nonce":1090541705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1090541705},{"type":"v1","addr":"172.21.15.112:6821","nonce":1090541705}]},"public_addr":"172.21.15.112:6817/1090541705","cluster_addr":"172.21.15.112:6819/1090541705","heartbeat_back_addr":"172.21.15.112:6823/1090541705","heartbeat_front_addr":"172.21.15.112:6821/1090541705","state":["exists","up"]},{"osd":8,"uuid":"277e3276-9d50-4233-9a4d-74b1a321f308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6817","nonce":1369698308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6819","nonce":1369698308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6823","nonce":1369698308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1369698308},{"type":"v1","addr":"172.21.15.175:6821","nonce":1369698308}]},"public_addr":"172.21.15.175:6817/1369698308","cluster_addr":"172.21.15.175:6819/1369698308","heartbeat_back_addr":"172.21.15.175:6823/1369698308","heartbeat_front_addr":"172.21.15.175:6821/1369698308","state":["exists","up"]},{"osd":9,"uuid":"f530826b-3c2f-411b-9795-f4b3c6053a23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6827","nonce":2518248011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6829","nonce":2518248011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6833","nonce":2518248011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":2518248011},{"type":"v1","addr":"172.21.15.43:6831","nonce":2518248011}]},"public_addr":"172.21.15.43:6827/2518248011","cluster_addr":"172.21.15.43:6829/2518248011","heartbeat_back_addr":"172.21.15.43:6833/2518248011","heartbeat_front_addr":"172.21.15.43:6831/2518248011","state":["exists","up"]},{"osd":10,"uuid":"307394bf-0915-448c-b025-42db9d0691c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6825","nonce":3645019377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6827","nonce":3645019377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6831","nonce":3645019377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3645019377},{"type":"v1","addr":"172.21.15.112:6829","nonce":3645019377}]},"public_addr":"172.21.15.112:6825/3645019377","cluster_addr":"172.21.15.112:6827/3645019377","heartbeat_back_addr":"172.21.15.112:6831/3645019377","heartbeat_front_addr":"172.21.15.112:6829/3645019377","state":["exists","up"]},{"osd":11,"uuid":"8114e33b-8fb2-4fe4-bf56-5f007c88ddbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6825","nonce":2527264089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6827","nonce":2527264089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6831","nonce":2527264089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2527264089},{"type":"v1","addr":"172.21.15.175:6829","nonce":2527264089}]},"public_addr":"172.21.15.175:6825/2527264089","cluster_addr":"172.21.15.175:6827/2527264089","heartbeat_back_addr":"172.21.15.175:6831/2527264089","heartbeat_front_addr":"172.21.15.175:6829/2527264089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:49.440785+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:13.951843+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:37.906975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:03.182275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:28.261429+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:53.280007+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.193834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:45.906378+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.391956+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:38.929197+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:04.717067+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:31.413031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/126475132":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/4288951239":"2024-04-22T08:38:54.881882+0000","172.21.15.43:6801/1577030772":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/2382670190":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/1916182222":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6801/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/3200866696":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/4047106671":"2024-04-22T08:29:13.889526+0000","172.21.15.43:6800/1559586579":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/1174411284":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/347461706":"2024-04-22T08:29:13.889526+0000","172.21.15.43:0/887926369":"2024-04-22T08:28:46.855699+0000","172.21.15.43:6800/1577030772":"2024-04-22T08:38:54.881882+0000","172.21.15.43:0/2598599313":"2024-04-22T08:28:46.855699+0000","172.21.15.43:0/2126312819":"2024-04-22T08:28:46.855699+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]},{"pool":4,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:52:27.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.328+0000 7ff823985640 1 -- 172.21.15.43:0/174841395 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff8080a3d20 msgr2=0x7ff8080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:27.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.328+0000 7ff823985640 1 --2- 172.21.15.43:0/174841395 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff8080a3d20 0x7ff8080a61e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff80c0096c0 tx=0x7ff80c000bc0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.328+0000 7ff823985640 1 -- 172.21.15.43:0/174841395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c1031d0 msgr2=0x7ff81c1a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:27.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.328+0000 7ff823985640 1 --2- 172.21.15.43:0/174841395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c1031d0 0x7ff81c1a05c0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7ff81800d950 tx=0x7ff81800de10 comp rx=0 tx=0).stop 2024-04-21T08:52:27.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.328+0000 7ff823985640 1 -- 172.21.15.43:0/174841395 shutdown_connections 2024-04-21T08:52:27.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.328+0000 7ff823985640 1 --2- 172.21.15.43:0/174841395 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7ff8080a3d20 0x7ff8080a61e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.328+0000 7ff823985640 1 --2- 172.21.15.43:0/174841395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff81c10c5a0 0x7ff81c19a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.328+0000 7ff823985640 1 --2- 172.21.15.43:0/174841395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff81c103af0 0x7ff81c1a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.328+0000 7ff823985640 1 --2- 172.21.15.43:0/174841395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c1031d0 0x7ff81c1a05c0 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.328+0000 7ff823985640 1 -- 172.21.15.43:0/174841395 >> 172.21.15.43:0/174841395 conn(0x7ff81c0771d0 msgr2=0x7ff81c0fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:27.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.328+0000 7ff823985640 1 -- 172.21.15.43:0/174841395 shutdown_connections 2024-04-21T08:52:27.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.328+0000 7ff823985640 1 -- 172.21.15.43:0/174841395 wait complete. 2024-04-21T08:52:27.340 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-21T08:52:27.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.443+0000 7f85118fd640 1 -- 172.21.15.43:0/2791651398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f850c103b10 msgr2=0x7f850c10bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:27.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.443+0000 7f85118fd640 1 --2- 172.21.15.43:0/2791651398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f850c103b10 0x7f850c10bff0 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f8500009900 tx=0x7f850002bcd0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.443+0000 7f85118fd640 1 -- 172.21.15.43:0/2791651398 shutdown_connections 2024-04-21T08:52:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.443+0000 7f85118fd640 1 --2- 172.21.15.43:0/2791651398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f850c10c530 0x7f850c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.443+0000 7f85118fd640 1 --2- 172.21.15.43:0/2791651398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f850c103b10 0x7f850c10bff0 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.443+0000 7f85118fd640 1 --2- 172.21.15.43:0/2791651398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f850c1031f0 0x7f850c1035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.443+0000 7f85118fd640 1 -- 172.21.15.43:0/2791651398 >> 172.21.15.43:0/2791651398 conn(0x7f850c077fb0 msgr2=0x7f850c0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.443+0000 7f85118fd640 1 -- 172.21.15.43:0/2791651398 shutdown_connections 2024-04-21T08:52:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.443+0000 7f85118fd640 1 -- 172.21.15.43:0/2791651398 wait complete. 2024-04-21T08:52:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.444+0000 7f85118fd640 1 Processor -- start 2024-04-21T08:52:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.444+0000 7f85118fd640 1 -- start start 2024-04-21T08:52:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.444+0000 7f85118fd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f850c1031f0 0x7f850c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.444+0000 7f85118fd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f850c103b10 0x7f850c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.444+0000 7f85118fd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f850c10c530 0x7f850c19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.444+0000 7f85118fd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f850c1121e0 con 0x7f850c103b10 2024-04-21T08:52:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.444+0000 7f85118fd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f850c112060 con 0x7f850c1031f0 2024-04-21T08:52:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.444+0000 7f85118fd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f850c112360 con 0x7f850c10c530 2024-04-21T08:52:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.445+0000 7f850a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f850c103b10 0x7f850c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.445+0000 7f850a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f850c103b10 0x7f850c198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53814/0 (socket says 172.21.15.43:53814) 2024-04-21T08:52:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.445+0000 7f850a7fc640 1 -- 172.21.15.43:0/1616556296 learned_addr learned my addr 172.21.15.43:0/1616556296 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:52:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.445+0000 7f850affd640 1 --2- 172.21.15.43:0/1616556296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f850c1031f0 0x7f850c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.445+0000 7f850b7fe640 1 --2- 172.21.15.43:0/1616556296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f850c10c530 0x7f850c19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.445+0000 7f850a7fc640 1 -- 172.21.15.43:0/1616556296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f850c10c530 msgr2=0x7f850c19d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.445+0000 7f850a7fc640 1 --2- 172.21.15.43:0/1616556296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f850c10c530 0x7f850c19d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.445+0000 7f850a7fc640 1 -- 172.21.15.43:0/1616556296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f850c1031f0 msgr2=0x7f850c198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.445+0000 7f850a7fc640 1 --2- 172.21.15.43:0/1616556296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f850c1031f0 0x7f850c198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.445+0000 7f850a7fc640 1 -- 172.21.15.43:0/1616556296 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850c19d730 con 0x7f850c103b10 2024-04-21T08:52:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.445+0000 7f850a7fc640 1 --2- 172.21.15.43:0/1616556296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f850c103b10 0x7f850c198c80 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f8500000c00 tx=0x7f850002d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.445+0000 7f85108fb640 1 -- 172.21.15.43:0/1616556296 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500043020 con 0x7f850c103b10 2024-04-21T08:52:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.445+0000 7f85118fd640 1 -- 172.21.15.43:0/1616556296 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850c19d9c0 con 0x7f850c103b10 2024-04-21T08:52:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.445+0000 7f85108fb640 1 -- 172.21.15.43:0/1616556296 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f850002dd50 con 0x7f850c103b10 2024-04-21T08:52:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.445+0000 7f85108fb640 1 -- 172.21.15.43:0/1616556296 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500003af0 con 0x7f850c103b10 2024-04-21T08:52:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.445+0000 7f85118fd640 1 -- 172.21.15.43:0/1616556296 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f850c1a52b0 con 0x7f850c103b10 2024-04-21T08:52:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.446+0000 7f85118fd640 1 -- 172.21.15.43:0/1616556296 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f850c0691c0 con 0x7f850c103b10 2024-04-21T08:52:27.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.447+0000 7f85108fb640 1 -- 172.21.15.43:0/1616556296 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8500046020 con 0x7f850c103b10 2024-04-21T08:52:27.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.448+0000 7f85108fb640 1 --2- 172.21.15.43:0/1616556296 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84f40a3d20 0x7f84f40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:27.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.448+0000 7f850affd640 1 --2- 172.21.15.43:0/1616556296 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84f40a3d20 0x7f84f40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:27.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.448+0000 7f85108fb640 1 -- 172.21.15.43:0/1616556296 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f85000f2f00 con 0x7f850c103b10 2024-04-21T08:52:27.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.449+0000 7f850affd640 1 --2- 172.21.15.43:0/1616556296 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84f40a3d20 0x7f84f40a61e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00bbe0 tx=0x7f84fc00b390 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:27.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.452+0000 7f85108fb640 1 -- 172.21.15.43:0/1616556296 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f850004d020 con 0x7f850c103b10 2024-04-21T08:52:27.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:27 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1299181219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:52:27.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:27 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1275658751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:52:27.466 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:27 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1299181219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:52:27.466 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:27 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1275658751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:52:27.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.679+0000 7f85118fd640 1 -- 172.21.15.43:0/1616556296 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f850c1076b0 con 0x7f850c103b10 2024-04-21T08:52:27.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.681+0000 7f85108fb640 1 -- 172.21.15.43:0/1616556296 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) ==== 161+0+0 (secure 0 0 0) 0x7f85000bf290 con 0x7f850c103b10 2024-04-21T08:52:27.680 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-21T08:52:27.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.684+0000 7f85118fd640 1 -- 172.21.15.43:0/1616556296 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84f40a3d20 msgr2=0x7f84f40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:27.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.684+0000 7f85118fd640 1 --2- 172.21.15.43:0/1616556296 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84f40a3d20 0x7f84f40a61e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00bbe0 tx=0x7f84fc00b390 comp rx=0 tx=0).stop 2024-04-21T08:52:27.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.684+0000 7f85118fd640 1 -- 172.21.15.43:0/1616556296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f850c103b10 msgr2=0x7f850c198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:27.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.684+0000 7f85118fd640 1 --2- 172.21.15.43:0/1616556296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f850c103b10 0x7f850c198c80 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f8500000c00 tx=0x7f850002d890 comp rx=0 tx=0).stop 2024-04-21T08:52:27.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.685+0000 7f85118fd640 1 -- 172.21.15.43:0/1616556296 shutdown_connections 2024-04-21T08:52:27.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.685+0000 7f85118fd640 1 --2- 172.21.15.43:0/1616556296 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f84f40a3d20 0x7f84f40a61e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.685+0000 7f85118fd640 1 --2- 172.21.15.43:0/1616556296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f850c10c530 0x7f850c19d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.685+0000 7f85118fd640 1 --2- 172.21.15.43:0/1616556296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f850c103b10 0x7f850c198c80 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.685+0000 7f85118fd640 1 --2- 172.21.15.43:0/1616556296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f850c1031f0 0x7f850c198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.685+0000 7f85118fd640 1 -- 172.21.15.43:0/1616556296 >> 172.21.15.43:0/1616556296 conn(0x7f850c077fb0 msgr2=0x7f850c0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:27.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.685+0000 7f85118fd640 1 -- 172.21.15.43:0/1616556296 shutdown_connections 2024-04-21T08:52:27.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.685+0000 7f85118fd640 1 -- 172.21.15.43:0/1616556296 wait complete. 2024-04-21T08:52:27.700 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-21T08:52:27.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.809+0000 7f44510c1640 1 -- 172.21.15.43:0/227970358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f444c108920 msgr2=0x7f444c06a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:27.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.809+0000 7f44510c1640 1 --2- 172.21.15.43:0/227970358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f444c108920 0x7f444c06a5d0 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f4440009900 tx=0x7f444002bcd0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.809+0000 7f44510c1640 1 -- 172.21.15.43:0/227970358 shutdown_connections 2024-04-21T08:52:27.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.809+0000 7f44510c1640 1 --2- 172.21.15.43:0/227970358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f444c06ace0 0x7f444c10ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.809+0000 7f44510c1640 1 --2- 172.21.15.43:0/227970358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f444c108920 0x7f444c06a5d0 unknown :-1 s=CLOSED pgs=2450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.809+0000 7f44510c1640 1 --2- 172.21.15.43:0/227970358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f444c108000 0x7f444c1083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.809+0000 7f44510c1640 1 -- 172.21.15.43:0/227970358 >> 172.21.15.43:0/227970358 conn(0x7f444c076c50 msgr2=0x7f444c077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:27.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.809+0000 7f44510c1640 1 -- 172.21.15.43:0/227970358 shutdown_connections 2024-04-21T08:52:27.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.809+0000 7f44510c1640 1 -- 172.21.15.43:0/227970358 wait complete. 2024-04-21T08:52:27.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.810+0000 7f44510c1640 1 Processor -- start 2024-04-21T08:52:27.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.810+0000 7f44510c1640 1 -- start start 2024-04-21T08:52:27.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.810+0000 7f44510c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f444c06ace0 0x7f444c198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:27.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.810+0000 7f44510c1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f444c108000 0x7f444c198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:27.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.810+0000 7f44510c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f444c108920 0x7f444c19cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:27.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.810+0000 7f44510c1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f444c112180 con 0x7f444c108920 2024-04-21T08:52:27.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.810+0000 7f44510c1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f444c112000 con 0x7f444c06ace0 2024-04-21T08:52:27.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.810+0000 7f44510c1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f444c112300 con 0x7f444c108000 2024-04-21T08:52:27.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.811+0000 7f444b59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f444c108920 0x7f444c19cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:27.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.811+0000 7f444b59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f444c108920 0x7f444c19cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53862/0 (socket says 172.21.15.43:53862) 2024-04-21T08:52:27.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.811+0000 7f444b59e640 1 -- 172.21.15.43:0/636291739 learned_addr learned my addr 172.21.15.43:0/636291739 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:52:27.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.811+0000 7f444ad9d640 1 --2- 172.21.15.43:0/636291739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f444c06ace0 0x7f444c198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:27.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.811+0000 7f444a59c640 1 --2- 172.21.15.43:0/636291739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f444c108000 0x7f444c198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:27.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.811+0000 7f444b59e640 1 -- 172.21.15.43:0/636291739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f444c108000 msgr2=0x7f444c198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:27.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.811+0000 7f444b59e640 1 --2- 172.21.15.43:0/636291739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f444c108000 0x7f444c198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.811+0000 7f444b59e640 1 -- 172.21.15.43:0/636291739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f444c06ace0 msgr2=0x7f444c198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:27.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.811+0000 7f444b59e640 1 --2- 172.21.15.43:0/636291739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f444c06ace0 0x7f444c198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:27.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.811+0000 7f444b59e640 1 -- 172.21.15.43:0/636291739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f444c19d710 con 0x7f444c108920 2024-04-21T08:52:27.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.811+0000 7f444b59e640 1 --2- 172.21.15.43:0/636291739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f444c108920 0x7f444c19cfe0 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f444400da40 tx=0x7f444400df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:27.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.811+0000 7f4433fff640 1 -- 172.21.15.43:0/636291739 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4444017070 con 0x7f444c108920 2024-04-21T08:52:27.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.811+0000 7f4433fff640 1 -- 172.21.15.43:0/636291739 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4444003e20 con 0x7f444c108920 2024-04-21T08:52:27.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.811+0000 7f44510c1640 1 -- 172.21.15.43:0/636291739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f444c19da00 con 0x7f444c108920 2024-04-21T08:52:27.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.811+0000 7f4433fff640 1 -- 172.21.15.43:0/636291739 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4444010460 con 0x7f444c108920 2024-04-21T08:52:27.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.811+0000 7f44510c1640 1 -- 172.21.15.43:0/636291739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f444c1a5290 con 0x7f444c108920 2024-04-21T08:52:27.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.812+0000 7f44510c1640 1 -- 172.21.15.43:0/636291739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f444c103660 con 0x7f444c108920 2024-04-21T08:52:27.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.814+0000 7f4433fff640 1 -- 172.21.15.43:0/636291739 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4444004020 con 0x7f444c108920 2024-04-21T08:52:27.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.814+0000 7f4433fff640 1 --2- 172.21.15.43:0/636291739 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f44340a3d20 0x7f44340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:27.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.815+0000 7f444ad9d640 1 --2- 172.21.15.43:0/636291739 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f44340a3d20 0x7f44340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:27.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.815+0000 7f4433fff640 1 -- 172.21.15.43:0/636291739 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9925+0+0 (secure 0 0 0) 0x7f44440d0a30 con 0x7f444c108920 2024-04-21T08:52:27.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.815+0000 7f444ad9d640 1 --2- 172.21.15.43:0/636291739 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f44340a3d20 0x7f44340a61e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f443c005fd0 tx=0x7f443c00b390 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:27.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:27.818+0000 7f4433fff640 1 -- 172.21.15.43:0/636291739 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f444409cb50 con 0x7f444c108920 2024-04-21T08:52:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.044+0000 7f44510c1640 1 -- 172.21.15.43:0/636291739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f444c199a70 con 0x7f444c108920 2024-04-21T08:52:28.045 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 21 08:52:27 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-k[60639]: ignoring --setuser ceph since I am not root 2024-04-21T08:52:28.045 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 21 08:52:27 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-k[60639]: ignoring --setgroup ceph since I am not root 2024-04-21T08:52:28.045 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 21 08:52:27 smithi112 ceph-mds[60643]: -- 172.21.15.112:0/208654279 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556cda4efa40 con 0x556cda4c2c00 2024-04-21T08:52:28.045 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 21 08:52:27 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-k[60639]: starting mds.k at 2024-04-21T08:52:28.045 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 21 08:52:27 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-e[60958]: ignoring --setuser ceph since I am not root 2024-04-21T08:52:28.045 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 21 08:52:27 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-e[60958]: ignoring --setgroup ceph since I am not root 2024-04-21T08:52:28.045 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 21 08:52:27 smithi112 ceph-mds[60962]: -- 172.21.15.112:0/2572185104 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a6b8705a40 con 0x55a6b86d8800 2024-04-21T08:52:28.046 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 21 08:52:27 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-e[60958]: starting mds.e at 2024-04-21T08:52:28.046 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 21 08:52:27 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-h[61265]: ignoring --setuser ceph since I am not root 2024-04-21T08:52:28.046 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 21 08:52:27 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-h[61265]: ignoring --setgroup ceph since I am not root 2024-04-21T08:52:28.046 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 21 08:52:27 smithi112 ceph-mds[61269]: -- 172.21.15.112:0/3543494809 <== mon.2 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557bb995ba40 con 0x557bb992e800 2024-04-21T08:52:28.046 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 21 08:52:27 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-h[61265]: starting mds.h at 2024-04-21T08:52:28.046 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:52:27 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-b[61578]: ignoring --setuser ceph since I am not root 2024-04-21T08:52:28.047 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:52:27 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-b[61578]: ignoring --setgroup ceph since I am not root 2024-04-21T08:52:28.047 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:52:27 smithi112 ceph-mds[61582]: -- 172.21.15.112:0/1739796622 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563fbfa7fa40 con 0x563fbfa52800 2024-04-21T08:52:28.047 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:52:27 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-b[61578]: starting mds.b at 2024-04-21T08:52:28.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:27 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-a[27974]: 2024-04-21T08:52:27.679+0000 7f3024e1e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-21T08:52:28.062 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 21 08:52:27 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-a[67537]: ignoring --setuser ceph since I am not root 2024-04-21T08:52:28.062 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 21 08:52:27 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-a[67537]: ignoring --setgroup ceph since I am not root 2024-04-21T08:52:28.062 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 21 08:52:27 smithi043 ceph-mds[67542]: -- 172.21.15.43:0/1768379390 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55af47f0fa40 con 0x55af47ee3000 2024-04-21T08:52:28.062 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 21 08:52:27 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-a[67537]: starting mds.a at 2024-04-21T08:52:28.062 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 21 08:52:27 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-g[67149]: ignoring --setuser ceph since I am not root 2024-04-21T08:52:28.062 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 21 08:52:27 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-g[67149]: ignoring --setgroup ceph since I am not root 2024-04-21T08:52:28.063 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 21 08:52:27 smithi043 ceph-mds[67153]: -- 172.21.15.43:0/2665057791 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5608f0301a40 con 0x5608f02d5000 2024-04-21T08:52:28.063 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 21 08:52:27 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-g[67149]: starting mds.g at 2024-04-21T08:52:28.208 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:52:27 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[66800]: ignoring --setuser ceph since I am not root 2024-04-21T08:52:28.209 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:52:27 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[66800]: ignoring --setgroup ceph since I am not root 2024-04-21T08:52:28.209 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:52:27 smithi175 ceph-mds[66804]: -- 172.21.15.175:0/2216166424 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55790fa65a40 con 0x55790fa38c00 2024-04-21T08:52:28.209 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:52:27 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[66800]: starting mds.c at 2024-04-21T08:52:28.209 INFO:journalctl@ceph.mds.f.smithi175.stdout:Apr 21 08:52:27 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-f[61854]: ignoring --setuser ceph since I am not root 2024-04-21T08:52:28.209 INFO:journalctl@ceph.mds.f.smithi175.stdout:Apr 21 08:52:27 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-f[61854]: ignoring --setgroup ceph since I am not root 2024-04-21T08:52:28.209 INFO:journalctl@ceph.mds.f.smithi175.stdout:Apr 21 08:52:27 smithi175 ceph-mds[61858]: -- 172.21.15.175:0/3127998879 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b3f38e1a40 con 0x55b3f38b4c00 2024-04-21T08:52:28.209 INFO:journalctl@ceph.mds.f.smithi175.stdout:Apr 21 08:52:27 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-f[61854]: starting mds.f at 2024-04-21T08:52:28.210 INFO:journalctl@ceph.mds.l.smithi175.stdout:Apr 21 08:52:27 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-l[61532]: ignoring --setuser ceph since I am not root 2024-04-21T08:52:28.210 INFO:journalctl@ceph.mds.l.smithi175.stdout:Apr 21 08:52:27 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-l[61532]: ignoring --setgroup ceph since I am not root 2024-04-21T08:52:28.210 INFO:journalctl@ceph.mds.l.smithi175.stdout:Apr 21 08:52:27 smithi175 ceph-mds[61536]: -- 172.21.15.175:0/4051826360 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ec21925a40 con 0x55ec218f8800 2024-04-21T08:52:28.210 INFO:journalctl@ceph.mds.l.smithi175.stdout:Apr 21 08:52:27 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-l[61532]: starting mds.l at 2024-04-21T08:52:28.210 INFO:journalctl@ceph.mds.i.smithi175.stdout:Apr 21 08:52:27 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-i[62469]: ignoring --setuser ceph since I am not root 2024-04-21T08:52:28.210 INFO:journalctl@ceph.mds.i.smithi175.stdout:Apr 21 08:52:27 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-i[62469]: ignoring --setgroup ceph since I am not root 2024-04-21T08:52:28.210 INFO:journalctl@ceph.mds.i.smithi175.stdout:Apr 21 08:52:27 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-i[62469]: starting mds.i at 2024-04-21T08:52:28.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:28 smithi175 ceph-mon[31920]: pgmap v424: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 13 op/s 2024-04-21T08:52:28.211 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:28 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/174841395' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:52:28.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:28 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1616556296' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-21T08:52:28.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:28 smithi175 ceph-mon[31920]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-21T08:52:28.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:28 smithi175 ceph-mon[31920]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-21T08:52:28.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:28 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1616556296' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-21T08:52:28.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:28 smithi175 ceph-mon[31920]: osdmap e111: 12 total, 12 up, 12 in 2024-04-21T08:52:28.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:28 smithi175 ceph-mon[31920]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-21T08:52:28.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:28 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/636291739' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:52:28.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:28 smithi112 ceph-mon[31085]: pgmap v424: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 13 op/s 2024-04-21T08:52:28.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:28 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/174841395' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:52:28.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:28 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1616556296' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-21T08:52:28.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:28 smithi112 ceph-mon[31085]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-21T08:52:28.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:28 smithi112 ceph-mon[31085]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-21T08:52:28.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:28 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1616556296' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-21T08:52:28.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:28 smithi112 ceph-mon[31085]: osdmap e111: 12 total, 12 up, 12 in 2024-04-21T08:52:28.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:28 smithi112 ceph-mon[31085]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-21T08:52:28.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:28 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/636291739' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:52:28.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:28 smithi043 ceph-mon[27978]: pgmap v424: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 13 op/s 2024-04-21T08:52:28.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:28 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/174841395' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:52:28.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:28 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1616556296' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-21T08:52:28.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:28 smithi043 ceph-mon[27978]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-21T08:52:28.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:28 smithi043 ceph-mon[27978]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-21T08:52:28.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:28 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1616556296' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-21T08:52:28.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:28 smithi043 ceph-mon[27978]: osdmap e111: 12 total, 12 up, 12 in 2024-04-21T08:52:28.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:28 smithi043 ceph-mon[27978]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-21T08:52:28.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:28 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/636291739' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:52:28.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.683+0000 7f4433fff640 1 -- 172.21.15.43:0/636291739 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) ==== 104+0+0 (secure 0 0 0) 0x7f444409c450 con 0x7f444c108920 2024-04-21T08:52:28.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.686+0000 7f44510c1640 1 -- 172.21.15.43:0/636291739 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f44340a3d20 msgr2=0x7f44340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:28.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.686+0000 7f44510c1640 1 --2- 172.21.15.43:0/636291739 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f44340a3d20 0x7f44340a61e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f443c005fd0 tx=0x7f443c00b390 comp rx=0 tx=0).stop 2024-04-21T08:52:28.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.686+0000 7f44510c1640 1 -- 172.21.15.43:0/636291739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f444c108920 msgr2=0x7f444c19cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:28.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.686+0000 7f44510c1640 1 --2- 172.21.15.43:0/636291739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f444c108920 0x7f444c19cfe0 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f444400da40 tx=0x7f444400df00 comp rx=0 tx=0).stop 2024-04-21T08:52:28.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.686+0000 7f44510c1640 1 -- 172.21.15.43:0/636291739 shutdown_connections 2024-04-21T08:52:28.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.686+0000 7f44510c1640 1 --2- 172.21.15.43:0/636291739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f444c108920 0x7f444c19cfe0 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:28.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.686+0000 7f44510c1640 1 --2- 172.21.15.43:0/636291739 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f44340a3d20 0x7f44340a61e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:28.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.686+0000 7f44510c1640 1 --2- 172.21.15.43:0/636291739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f444c108000 0x7f444c198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:28.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.686+0000 7f44510c1640 1 --2- 172.21.15.43:0/636291739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f444c06ace0 0x7f444c198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:28.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.686+0000 7f44510c1640 1 -- 172.21.15.43:0/636291739 >> 172.21.15.43:0/636291739 conn(0x7f444c076c50 msgr2=0x7f444c100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:28.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.686+0000 7f44510c1640 1 -- 172.21.15.43:0/636291739 shutdown_connections 2024-04-21T08:52:28.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.686+0000 7f44510c1640 1 -- 172.21.15.43:0/636291739 wait complete. 2024-04-21T08:52:28.703 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-21T08:52:28.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.819+0000 7f6043af2640 1 -- 172.21.15.43:0/3757343369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f603c1031c0 msgr2=0x7f603c1035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:28.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.819+0000 7f6043af2640 1 --2- 172.21.15.43:0/3757343369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f603c1031c0 0x7f603c1035a0 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f602c009900 tx=0x7f602c02bd60 comp rx=0 tx=0).stop 2024-04-21T08:52:28.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.819+0000 7f6043af2640 1 -- 172.21.15.43:0/3757343369 shutdown_connections 2024-04-21T08:52:28.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.819+0000 7f6043af2640 1 --2- 172.21.15.43:0/3757343369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f603c10c500 0x7f603c10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:28.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.819+0000 7f6043af2640 1 --2- 172.21.15.43:0/3757343369 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f603c103ae0 0x7f603c10bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:28.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.819+0000 7f6043af2640 1 --2- 172.21.15.43:0/3757343369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f603c1031c0 0x7f603c1035a0 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:28.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.819+0000 7f6043af2640 1 -- 172.21.15.43:0/3757343369 >> 172.21.15.43:0/3757343369 conn(0x7f603c078000 msgr2=0x7f603c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:28.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.819+0000 7f6043af2640 1 -- 172.21.15.43:0/3757343369 shutdown_connections 2024-04-21T08:52:28.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.819+0000 7f6043af2640 1 -- 172.21.15.43:0/3757343369 wait complete. 2024-04-21T08:52:28.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.819+0000 7f6043af2640 1 Processor -- start 2024-04-21T08:52:28.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.819+0000 7f6043af2640 1 -- start start 2024-04-21T08:52:28.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.820+0000 7f6043af2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f603c1031c0 0x7f603c100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:28.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.820+0000 7f6043af2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f603c103ae0 0x7f603c101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:28.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.820+0000 7f6043af2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f603c10c500 0x7f603c101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:28.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.820+0000 7f6043af2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f603c112170 con 0x7f603c1031c0 2024-04-21T08:52:28.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.820+0000 7f6043af2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f603c111ff0 con 0x7f603c103ae0 2024-04-21T08:52:28.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.820+0000 7f6043af2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f603c1122f0 con 0x7f603c10c500 2024-04-21T08:52:28.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.820+0000 7f604188e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f603c1031c0 0x7f603c100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:28.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.820+0000 7f604188e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f603c1031c0 0x7f603c100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53880/0 (socket says 172.21.15.43:53880) 2024-04-21T08:52:28.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.820+0000 7f604188e640 1 -- 172.21.15.43:0/1950334635 learned_addr learned my addr 172.21.15.43:0/1950334635 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:52:28.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.820+0000 7f604108d640 1 --2- 172.21.15.43:0/1950334635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f603c103ae0 0x7f603c101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:28.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.820+0000 7f604208f640 1 --2- 172.21.15.43:0/1950334635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f603c10c500 0x7f603c101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:28.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.820+0000 7f604188e640 1 -- 172.21.15.43:0/1950334635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f603c10c500 msgr2=0x7f603c101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:28.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.820+0000 7f604188e640 1 --2- 172.21.15.43:0/1950334635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f603c10c500 0x7f603c101890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:28.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.820+0000 7f604188e640 1 -- 172.21.15.43:0/1950334635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f603c103ae0 msgr2=0x7f603c101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:28.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.820+0000 7f604188e640 1 --2- 172.21.15.43:0/1950334635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f603c103ae0 0x7f603c101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:28.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.820+0000 7f604188e640 1 -- 172.21.15.43:0/1950334635 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603c1a6c10 con 0x7f603c1031c0 2024-04-21T08:52:28.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.820+0000 7f604208f640 1 --2- 172.21.15.43:0/1950334635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f603c10c500 0x7f603c101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:52:28.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.820+0000 7f604108d640 1 --2- 172.21.15.43:0/1950334635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f603c103ae0 0x7f603c101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:52:28.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.820+0000 7f604188e640 1 --2- 172.21.15.43:0/1950334635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f603c1031c0 0x7f603c100e10 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f602c005940 tx=0x7f602c02d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:28.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.820+0000 7f6032ffd640 1 -- 172.21.15.43:0/1950334635 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c043020 con 0x7f603c1031c0 2024-04-21T08:52:28.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.820+0000 7f6032ffd640 1 -- 172.21.15.43:0/1950334635 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602c02dda0 con 0x7f603c1031c0 2024-04-21T08:52:28.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.820+0000 7f6043af2640 1 -- 172.21.15.43:0/1950334635 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603c1a6ea0 con 0x7f603c1031c0 2024-04-21T08:52:28.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.820+0000 7f6032ffd640 1 -- 172.21.15.43:0/1950334635 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c003bf0 con 0x7f603c1031c0 2024-04-21T08:52:28.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.821+0000 7f6043af2640 1 -- 172.21.15.43:0/1950334635 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f603c1a7360 con 0x7f603c1031c0 2024-04-21T08:52:28.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.821+0000 7f6043af2640 1 -- 172.21.15.43:0/1950334635 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f603c068d70 con 0x7f603c1031c0 2024-04-21T08:52:28.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.823+0000 7f6032ffd640 1 -- 172.21.15.43:0/1950334635 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f602c046020 con 0x7f603c1031c0 2024-04-21T08:52:28.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.823+0000 7f6032ffd640 1 --2- 172.21.15.43:0/1950334635 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f60280a3d20 0x7f60280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:28.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.823+0000 7f604108d640 1 --2- 172.21.15.43:0/1950334635 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f60280a3d20 0x7f60280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:28.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.823+0000 7f6032ffd640 1 -- 172.21.15.43:0/1950334635 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9925+0+0 (secure 0 0 0) 0x7f602c0f3670 con 0x7f603c1031c0 2024-04-21T08:52:28.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.824+0000 7f604108d640 1 --2- 172.21.15.43:0/1950334635 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f60280a3d20 0x7f60280a61e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f603c1028a0 tx=0x7f6038005ec0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:28.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:28.827+0000 7f6032ffd640 1 -- 172.21.15.43:0/1950334635 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f602c04d020 con 0x7f603c1031c0 2024-04-21T08:52:29.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.015+0000 7f6043af2640 1 -- 172.21.15.43:0/1950334635 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f603c102340 con 0x7f603c1031c0 2024-04-21T08:52:29.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-21T08:52:29.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-21T08:52:29.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: Cluster is now healthy 2024-04-21T08:52:29.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/636291739' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-21T08:52:29.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.175:6832/2862861935,v1:172.21.15.175:6833/2862861935] up:boot 2024-04-21T08:52:29.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.43:6838/403626209,v1:172.21.15.43:6839/403626209] up:boot 2024-04-21T08:52:29.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.175:6834/4159442700,v1:172.21.15.175:6835/4159442700] up:boot 2024-04-21T08:52:29.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.43:6840/699495307,v1:172.21.15.43:6841/699495307] up:boot 2024-04-21T08:52:29.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.175:6836/2414773173,v1:172.21.15.175:6837/2414773173] up:boot 2024-04-21T08:52:29.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.112:6832/284955728,v1:172.21.15.112:6834/284955728] up:boot 2024-04-21T08:52:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.112:6833/3310702365,v1:172.21.15.112:6835/3310702365] up:boot 2024-04-21T08:52:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.112:6836/4217545775,v1:172.21.15.112:6839/4217545775] up:boot 2024-04-21T08:52:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.112:6837/1634637866,v1:172.21.15.112:6838/1634637866] up:boot 2024-04-21T08:52:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: mds.? [v2:172.21.15.175:6838/1707014301,v1:172.21.15.175:6839/1707014301] up:boot 2024-04-21T08:52:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: fsmap 12 up:standby 2024-04-21T08:52:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:52:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:52:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:52:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:52:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:52:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:52:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:52:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:52:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:52:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:52:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:29 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1950334635' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:52:29.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-21T08:52:29.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-21T08:52:29.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: Cluster is now healthy 2024-04-21T08:52:29.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/636291739' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-21T08:52:29.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.175:6832/2862861935,v1:172.21.15.175:6833/2862861935] up:boot 2024-04-21T08:52:29.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.43:6838/403626209,v1:172.21.15.43:6839/403626209] up:boot 2024-04-21T08:52:29.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.175:6834/4159442700,v1:172.21.15.175:6835/4159442700] up:boot 2024-04-21T08:52:29.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.43:6840/699495307,v1:172.21.15.43:6841/699495307] up:boot 2024-04-21T08:52:29.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.175:6836/2414773173,v1:172.21.15.175:6837/2414773173] up:boot 2024-04-21T08:52:29.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.112:6832/284955728,v1:172.21.15.112:6834/284955728] up:boot 2024-04-21T08:52:29.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.112:6833/3310702365,v1:172.21.15.112:6835/3310702365] up:boot 2024-04-21T08:52:29.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.112:6836/4217545775,v1:172.21.15.112:6839/4217545775] up:boot 2024-04-21T08:52:29.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.112:6837/1634637866,v1:172.21.15.112:6838/1634637866] up:boot 2024-04-21T08:52:29.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: mds.? [v2:172.21.15.175:6838/1707014301,v1:172.21.15.175:6839/1707014301] up:boot 2024-04-21T08:52:29.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: fsmap 12 up:standby 2024-04-21T08:52:29.465 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:52:29.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:52:29.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:52:29.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:52:29.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:52:29.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:52:29.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:52:29.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:52:29.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:52:29.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:52:29.466 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:29 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1950334635' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:52:29.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-21T08:52:29.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-21T08:52:29.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: Cluster is now healthy 2024-04-21T08:52:29.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/636291739' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-21T08:52:29.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.175:6832/2862861935,v1:172.21.15.175:6833/2862861935] up:boot 2024-04-21T08:52:29.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.43:6838/403626209,v1:172.21.15.43:6839/403626209] up:boot 2024-04-21T08:52:29.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.175:6834/4159442700,v1:172.21.15.175:6835/4159442700] up:boot 2024-04-21T08:52:29.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.43:6840/699495307,v1:172.21.15.43:6841/699495307] up:boot 2024-04-21T08:52:29.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.175:6836/2414773173,v1:172.21.15.175:6837/2414773173] up:boot 2024-04-21T08:52:29.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.112:6832/284955728,v1:172.21.15.112:6834/284955728] up:boot 2024-04-21T08:52:29.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.112:6833/3310702365,v1:172.21.15.112:6835/3310702365] up:boot 2024-04-21T08:52:29.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.112:6836/4217545775,v1:172.21.15.112:6839/4217545775] up:boot 2024-04-21T08:52:29.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.112:6837/1634637866,v1:172.21.15.112:6838/1634637866] up:boot 2024-04-21T08:52:29.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: mds.? [v2:172.21.15.175:6838/1707014301,v1:172.21.15.175:6839/1707014301] up:boot 2024-04-21T08:52:29.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: fsmap 12 up:standby 2024-04-21T08:52:29.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:52:29.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:52:29.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:52:29.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:52:29.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:52:29.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:52:29.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:52:29.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:52:29.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:52:29.547 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: from='mgr.15546 172.21.15.43:0/1294203359' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:52:29.547 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:29 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1950334635' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:52:29.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.688+0000 7f6032ffd640 1 -- 172.21.15.43:0/1950334635 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v112) ==== 181+0+0 (secure 0 0 0) 0x7f603c102340 con 0x7f603c1031c0 2024-04-21T08:52:29.688 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-04-21T08:52:29.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.691+0000 7f6043af2640 1 -- 172.21.15.43:0/1950334635 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f60280a3d20 msgr2=0x7f60280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:29.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.691+0000 7f6043af2640 1 --2- 172.21.15.43:0/1950334635 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f60280a3d20 0x7f60280a61e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f603c1028a0 tx=0x7f6038005ec0 comp rx=0 tx=0).stop 2024-04-21T08:52:29.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.691+0000 7f6043af2640 1 -- 172.21.15.43:0/1950334635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f603c1031c0 msgr2=0x7f603c100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:29.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.691+0000 7f6043af2640 1 --2- 172.21.15.43:0/1950334635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f603c1031c0 0x7f603c100e10 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f602c005940 tx=0x7f602c02d9a0 comp rx=0 tx=0).stop 2024-04-21T08:52:29.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.692+0000 7f6043af2640 1 -- 172.21.15.43:0/1950334635 shutdown_connections 2024-04-21T08:52:29.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.692+0000 7f6043af2640 1 --2- 172.21.15.43:0/1950334635 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7f60280a3d20 0x7f60280a61e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:29.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.692+0000 7f6043af2640 1 --2- 172.21.15.43:0/1950334635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f603c10c500 0x7f603c101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:29.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.692+0000 7f6043af2640 1 --2- 172.21.15.43:0/1950334635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f603c103ae0 0x7f603c101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:29.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.692+0000 7f6043af2640 1 --2- 172.21.15.43:0/1950334635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f603c1031c0 0x7f603c100e10 unknown :-1 s=CLOSED pgs=2453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:29.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.692+0000 7f6043af2640 1 -- 172.21.15.43:0/1950334635 >> 172.21.15.43:0/1950334635 conn(0x7f603c078000 msgr2=0x7f603c0fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:29.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.692+0000 7f6043af2640 1 -- 172.21.15.43:0/1950334635 shutdown_connections 2024-04-21T08:52:29.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.692+0000 7f6043af2640 1 -- 172.21.15.43:0/1950334635 wait complete. 2024-04-21T08:52:29.704 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-21T08:52:29.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.813+0000 7fc588a3f640 1 -- 172.21.15.43:0/2847187151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc584106160 msgr2=0x7fc58410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:29.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.813+0000 7fc588a3f640 1 --2- 172.21.15.43:0/2847187151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc584106160 0x7fc58410dd30 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7fc574009ca0 tx=0x7fc57402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:52:29.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.813+0000 7fc588a3f640 1 -- 172.21.15.43:0/2847187151 shutdown_connections 2024-04-21T08:52:29.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.813+0000 7fc588a3f640 1 --2- 172.21.15.43:0/2847187151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc584106160 0x7fc58410dd30 unknown :-1 s=CLOSED pgs=2454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:29.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.813+0000 7fc588a3f640 1 --2- 172.21.15.43:0/2847187151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc584101a40 0x7fc584105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:29.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.813+0000 7fc588a3f640 1 --2- 172.21.15.43:0/2847187151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc584101090 0x7fc584101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:29.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.813+0000 7fc588a3f640 1 -- 172.21.15.43:0/2847187151 >> 172.21.15.43:0/2847187151 conn(0x7fc584077f50 msgr2=0x7fc584078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:29.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.813+0000 7fc588a3f640 1 -- 172.21.15.43:0/2847187151 shutdown_connections 2024-04-21T08:52:29.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.813+0000 7fc588a3f640 1 -- 172.21.15.43:0/2847187151 wait complete. 2024-04-21T08:52:29.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.814+0000 7fc588a3f640 1 Processor -- start 2024-04-21T08:52:29.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.814+0000 7fc588a3f640 1 -- start start 2024-04-21T08:52:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.814+0000 7fc588a3f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc584101090 0x7fc5841985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.814+0000 7fc588a3f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc584106160 0x7fc584198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.814+0000 7fc588a3f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc58419ce50 0x7fc58419d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.814+0000 7fc588a3f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc5841110b0 con 0x7fc584101090 2024-04-21T08:52:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.814+0000 7fc588a3f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc584110f30 con 0x7fc584106160 2024-04-21T08:52:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.814+0000 7fc588a3f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc584111230 con 0x7fc58419ce50 2024-04-21T08:52:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.815+0000 7fc58259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc584101090 0x7fc5841985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.815+0000 7fc58259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc584101090 0x7fc5841985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53896/0 (socket says 172.21.15.43:53896) 2024-04-21T08:52:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.815+0000 7fc58259c640 1 -- 172.21.15.43:0/2249020527 learned_addr learned my addr 172.21.15.43:0/2249020527 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:52:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.815+0000 7fc581d9b640 1 --2- 172.21.15.43:0/2249020527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc584106160 0x7fc584198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.815+0000 7fc582d9d640 1 --2- 172.21.15.43:0/2249020527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc58419ce50 0x7fc58419d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.815+0000 7fc58259c640 1 -- 172.21.15.43:0/2249020527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc58419ce50 msgr2=0x7fc58419d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.815+0000 7fc58259c640 1 --2- 172.21.15.43:0/2249020527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc58419ce50 0x7fc58419d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.815+0000 7fc58259c640 1 -- 172.21.15.43:0/2249020527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc584106160 msgr2=0x7fc584198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.815+0000 7fc58259c640 1 --2- 172.21.15.43:0/2249020527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc584106160 0x7fc584198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:29.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.815+0000 7fc58259c640 1 -- 172.21.15.43:0/2249020527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58419d940 con 0x7fc584101090 2024-04-21T08:52:29.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.815+0000 7fc582d9d640 1 --2- 172.21.15.43:0/2249020527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc58419ce50 0x7fc58419d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:52:29.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.815+0000 7fc58259c640 1 --2- 172.21.15.43:0/2249020527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc584101090 0x7fc5841985e0 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00c920 tx=0x7fc57c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:29.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.815+0000 7fc57b7fe640 1 -- 172.21.15.43:0/2249020527 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c007bb0 con 0x7fc584101090 2024-04-21T08:52:29.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.815+0000 7fc57b7fe640 1 -- 172.21.15.43:0/2249020527 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc57c007d50 con 0x7fc584101090 2024-04-21T08:52:29.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.815+0000 7fc57b7fe640 1 -- 172.21.15.43:0/2249020527 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c018730 con 0x7fc584101090 2024-04-21T08:52:29.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.815+0000 7fc588a3f640 1 -- 172.21.15.43:0/2249020527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc58419dc30 con 0x7fc584101090 2024-04-21T08:52:29.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.815+0000 7fc588a3f640 1 -- 172.21.15.43:0/2249020527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5841a53d0 con 0x7fc584101090 2024-04-21T08:52:29.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.817+0000 7fc588a3f640 1 -- 172.21.15.43:0/2249020527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc584068d70 con 0x7fc584101090 2024-04-21T08:52:29.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.818+0000 7fc57b7fe640 1 -- 172.21.15.43:0/2249020527 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc57c00f450 con 0x7fc584101090 2024-04-21T08:52:29.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.818+0000 7fc57b7fe640 1 --2- 172.21.15.43:0/2249020527 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc5700a3d20 0x7fc5700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:29.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.818+0000 7fc581d9b640 1 --2- 172.21.15.43:0/2249020527 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc5700a3d20 0x7fc5700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:29.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.818+0000 7fc57b7fe640 1 -- 172.21.15.43:0/2249020527 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9400+0+0 (secure 0 0 0) 0x7fc57c0d0cb0 con 0x7fc584101090 2024-04-21T08:52:29.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.819+0000 7fc581d9b640 1 --2- 172.21.15.43:0/2249020527 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc5700a3d20 0x7fc5700a61e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc584105b40 tx=0x7fc56c00a2d0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:29.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:29.823+0000 7fc57b7fe640 1 -- 172.21.15.43:0/2249020527 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc57c014030 con 0x7fc584101090 2024-04-21T08:52:30.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.024+0000 7fc588a3f640 1 -- 172.21.15.43:0/2249020527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc5841998e0 con 0x7fc584101090 2024-04-21T08:52:30.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:30 smithi043 ceph-mon[27978]: pgmap v426: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 716 B/s wr, 10 op/s 2024-04-21T08:52:30.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:30 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/1950334635' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T08:52:30.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:30 smithi043 ceph-mon[27978]: osdmap e112: 12 total, 12 up, 12 in 2024-04-21T08:52:30.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:30 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2249020527' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:52:30.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:30 smithi175 ceph-mon[31920]: pgmap v426: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 716 B/s wr, 10 op/s 2024-04-21T08:52:30.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:30 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/1950334635' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T08:52:30.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:30 smithi175 ceph-mon[31920]: osdmap e112: 12 total, 12 up, 12 in 2024-04-21T08:52:30.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:30 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2249020527' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:52:30.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:30 smithi112 ceph-mon[31085]: pgmap v426: 129 pgs: 129 active+clean; 3.1 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 716 B/s wr, 10 op/s 2024-04-21T08:52:30.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:30 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/1950334635' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T08:52:30.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:30 smithi112 ceph-mon[31085]: osdmap e112: 12 total, 12 up, 12 in 2024-04-21T08:52:30.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:30 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2249020527' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:52:30.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.690+0000 7fc57b7fe640 1 -- 172.21.15.43:0/2249020527 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v113) ==== 169+0+0 (secure 0 0 0) 0x7fc57c085e20 con 0x7fc584101090 2024-04-21T08:52:30.690 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-04-21T08:52:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.694+0000 7fc588a3f640 1 -- 172.21.15.43:0/2249020527 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc5700a3d20 msgr2=0x7fc5700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.694+0000 7fc588a3f640 1 --2- 172.21.15.43:0/2249020527 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc5700a3d20 0x7fc5700a61e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc584105b40 tx=0x7fc56c00a2d0 comp rx=0 tx=0).stop 2024-04-21T08:52:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.694+0000 7fc588a3f640 1 -- 172.21.15.43:0/2249020527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc584101090 msgr2=0x7fc5841985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.694+0000 7fc588a3f640 1 --2- 172.21.15.43:0/2249020527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc584101090 0x7fc5841985e0 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00c920 tx=0x7fc57c00cd20 comp rx=0 tx=0).stop 2024-04-21T08:52:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.694+0000 7fc588a3f640 1 -- 172.21.15.43:0/2249020527 shutdown_connections 2024-04-21T08:52:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.694+0000 7fc588a3f640 1 --2- 172.21.15.43:0/2249020527 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fc5700a3d20 0x7fc5700a61e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.694+0000 7fc588a3f640 1 --2- 172.21.15.43:0/2249020527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc58419ce50 0x7fc58419d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.694+0000 7fc588a3f640 1 --2- 172.21.15.43:0/2249020527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc584106160 0x7fc584198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.694+0000 7fc588a3f640 1 --2- 172.21.15.43:0/2249020527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc584101090 0x7fc5841985e0 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:30.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.694+0000 7fc588a3f640 1 -- 172.21.15.43:0/2249020527 >> 172.21.15.43:0/2249020527 conn(0x7fc584077f50 msgr2=0x7fc584069390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:30.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.694+0000 7fc588a3f640 1 -- 172.21.15.43:0/2249020527 shutdown_connections 2024-04-21T08:52:30.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.694+0000 7fc588a3f640 1 -- 172.21.15.43:0/2249020527 wait complete. 2024-04-21T08:52:30.712 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-21T08:52:30.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.825+0000 7fbec5b15640 1 -- 172.21.15.43:0/3680406992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbec0106160 msgr2=0x7fbec010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:30.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.825+0000 7fbec5b15640 1 --2- 172.21.15.43:0/3680406992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbec0106160 0x7fbec010dd30 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8009ca0 tx=0x7fbeb802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:52:30.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.825+0000 7fbec5b15640 1 -- 172.21.15.43:0/3680406992 shutdown_connections 2024-04-21T08:52:30.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.825+0000 7fbec5b15640 1 --2- 172.21.15.43:0/3680406992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbec0106160 0x7fbec010dd30 unknown :-1 s=CLOSED pgs=2456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:30.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.825+0000 7fbec5b15640 1 --2- 172.21.15.43:0/3680406992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbec0101a40 0x7fbec0105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:30.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.825+0000 7fbec5b15640 1 --2- 172.21.15.43:0/3680406992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbec0101090 0x7fbec0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:30.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.825+0000 7fbec5b15640 1 -- 172.21.15.43:0/3680406992 >> 172.21.15.43:0/3680406992 conn(0x7fbec00771d0 msgr2=0x7fbec00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:30.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.825+0000 7fbec5b15640 1 -- 172.21.15.43:0/3680406992 shutdown_connections 2024-04-21T08:52:30.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.825+0000 7fbec5b15640 1 -- 172.21.15.43:0/3680406992 wait complete. 2024-04-21T08:52:30.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.826+0000 7fbec5b15640 1 Processor -- start 2024-04-21T08:52:30.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.826+0000 7fbec5b15640 1 -- start start 2024-04-21T08:52:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.826+0000 7fbec5b15640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbec0101090 0x7fbec01986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.826+0000 7fbec5b15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbec0106160 0x7fbec0198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.826+0000 7fbec5b15640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbec019cf70 0x7fbec019d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.826+0000 7fbec5b15640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbec01110d0 con 0x7fbec019cf70 2024-04-21T08:52:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.826+0000 7fbec5b15640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbec0110f50 con 0x7fbec0106160 2024-04-21T08:52:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.826+0000 7fbec5b15640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbec0111250 con 0x7fbec0101090 2024-04-21T08:52:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.826+0000 7fbebffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbec019cf70 0x7fbec019d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.827+0000 7fbebffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbec019cf70 0x7fbec019d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53910/0 (socket says 172.21.15.43:53910) 2024-04-21T08:52:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.827+0000 7fbebffff640 1 -- 172.21.15.43:0/999398398 learned_addr learned my addr 172.21.15.43:0/999398398 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-21T08:52:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.827+0000 7fbebf7fe640 1 --2- 172.21.15.43:0/999398398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbec0101090 0x7fbec01986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.827+0000 7fbebeffd640 1 --2- 172.21.15.43:0/999398398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbec0106160 0x7fbec0198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.827+0000 7fbebffff640 1 -- 172.21.15.43:0/999398398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbec0101090 msgr2=0x7fbec01986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.827+0000 7fbebffff640 1 --2- 172.21.15.43:0/999398398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbec0101090 0x7fbec01986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.827+0000 7fbebffff640 1 -- 172.21.15.43:0/999398398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbec0106160 msgr2=0x7fbec0198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.827+0000 7fbebffff640 1 --2- 172.21.15.43:0/999398398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbec0106160 0x7fbec0198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.827+0000 7fbebffff640 1 -- 172.21.15.43:0/999398398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec019da60 con 0x7fbec019cf70 2024-04-21T08:52:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.827+0000 7fbebffff640 1 --2- 172.21.15.43:0/999398398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbec019cf70 0x7fbec019d420 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8005e30 tx=0x7fbeb8034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.827+0000 7fbebcff9640 1 -- 172.21.15.43:0/999398398 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8039040 con 0x7fbec019cf70 2024-04-21T08:52:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.827+0000 7fbebcff9640 1 -- 172.21.15.43:0/999398398 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeb8044070 con 0x7fbec019cf70 2024-04-21T08:52:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.827+0000 7fbebcff9640 1 -- 172.21.15.43:0/999398398 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8003e60 con 0x7fbec019cf70 2024-04-21T08:52:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.827+0000 7fbec5b15640 1 -- 172.21.15.43:0/999398398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec019dcf0 con 0x7fbec019cf70 2024-04-21T08:52:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.827+0000 7fbec5b15640 1 -- 172.21.15.43:0/999398398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec01a5460 con 0x7fbec019cf70 2024-04-21T08:52:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.828+0000 7fbec5b15640 1 -- 172.21.15.43:0/999398398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbec0102560 con 0x7fbec019cf70 2024-04-21T08:52:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.829+0000 7fbebcff9640 1 -- 172.21.15.43:0/999398398 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbeb80324c0 con 0x7fbec019cf70 2024-04-21T08:52:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.830+0000 7fbebcff9640 1 --2- 172.21.15.43:0/999398398 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbea80a3da0 0x7fbea80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.830+0000 7fbebf7fe640 1 --2- 172.21.15.43:0/999398398 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbea80a3da0 0x7fbea80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.830+0000 7fbebcff9640 1 -- 172.21.15.43:0/999398398 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8754+0+0 (secure 0 0 0) 0x7fbeb80f2a50 con 0x7fbec019cf70 2024-04-21T08:52:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.831+0000 7fbebf7fe640 1 --2- 172.21.15.43:0/999398398 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbea80a3da0 0x7fbea80a6260 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0005d00 tx=0x7fbeb000b410 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:30.835+0000 7fbebcff9640 1 -- 172.21.15.43:0/999398398 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbeb80bf0b0 con 0x7fbec019cf70 2024-04-21T08:52:31.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:31.034+0000 7fbec5b15640 1 -- 172.21.15.43:0/999398398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbec0101a40 con 0x7fbec019cf70 2024-04-21T08:52:31.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:31.695+0000 7fbebcff9640 1 -- 172.21.15.43:0/999398398 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v114) ==== 178+0+0 (secure 0 0 0) 0x7fbeb80be9b0 con 0x7fbec019cf70 2024-04-21T08:52:31.695 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data_ec' removed 2024-04-21T08:52:31.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:31.699+0000 7fbec5b15640 1 -- 172.21.15.43:0/999398398 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbea80a3da0 msgr2=0x7fbea80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:31.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:31.699+0000 7fbec5b15640 1 --2- 172.21.15.43:0/999398398 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbea80a3da0 0x7fbea80a6260 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0005d00 tx=0x7fbeb000b410 comp rx=0 tx=0).stop 2024-04-21T08:52:31.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:31.699+0000 7fbec5b15640 1 -- 172.21.15.43:0/999398398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbec019cf70 msgr2=0x7fbec019d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:31.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:31.699+0000 7fbec5b15640 1 --2- 172.21.15.43:0/999398398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbec019cf70 0x7fbec019d420 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8005e30 tx=0x7fbeb8034760 comp rx=0 tx=0).stop 2024-04-21T08:52:31.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:31.700+0000 7fbec5b15640 1 -- 172.21.15.43:0/999398398 shutdown_connections 2024-04-21T08:52:31.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:31.700+0000 7fbec5b15640 1 --2- 172.21.15.43:0/999398398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbec019cf70 0x7fbec019d420 unknown :-1 s=CLOSED pgs=2457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:31.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:31.700+0000 7fbec5b15640 1 --2- 172.21.15.43:0/999398398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbec0106160 0x7fbec0198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:31.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:31.700+0000 7fbec5b15640 1 --2- 172.21.15.43:0/999398398 >> [v2:172.21.15.43:6800/1482273906,v1:172.21.15.43:6801/1482273906] conn(0x7fbea80a3da0 0x7fbea80a6260 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:31.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:31.700+0000 7fbec5b15640 1 --2- 172.21.15.43:0/999398398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbec0101090 0x7fbec01986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:31.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:31.700+0000 7fbec5b15640 1 -- 172.21.15.43:0/999398398 >> 172.21.15.43:0/999398398 conn(0x7fbec00771d0 msgr2=0x7fbec006a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:31.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:31.700+0000 7fbec5b15640 1 -- 172.21.15.43:0/999398398 shutdown_connections 2024-04-21T08:52:31.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-21T08:52:31.700+0000 7fbec5b15640 1 -- 172.21.15.43:0/999398398 wait complete. 2024-04-21T08:52:31.716 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:52:31.743 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:52:31.776 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:52:31.809 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-21T08:52:31.809 DEBUG:teuthology.orchestra.run.smithi043:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-21T08:52:31.826 DEBUG:teuthology.orchestra.run.smithi112:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-21T08:52:31.842 DEBUG:teuthology.orchestra.run.smithi175:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-21T08:52:31.866 INFO:tasks.cephadm:Stopping all daemons... 2024-04-21T08:52:31.866 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-21T08:52:31.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mon.a 2024-04-21T08:52:31.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:31 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/2249020527' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T08:52:31.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:31 smithi175 ceph-mon[31920]: osdmap e113: 12 total, 12 up, 12 in 2024-04-21T08:52:31.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:31 smithi175 ceph-mon[31920]: from='client.? 172.21.15.43:0/999398398' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:52:31.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:31 smithi175 ceph-mon[31920]: pgmap v429: 33 pgs: 33 active+clean; 577 KiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:52:32.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:31 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/2249020527' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T08:52:32.044 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:31 smithi112 ceph-mon[31085]: osdmap e113: 12 total, 12 up, 12 in 2024-04-21T08:52:32.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:31 smithi112 ceph-mon[31085]: from='client.? 172.21.15.43:0/999398398' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:52:32.045 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:31 smithi112 ceph-mon[31085]: pgmap v429: 33 pgs: 33 active+clean; 577 KiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:52:32.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:31 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/2249020527' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T08:52:32.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:31 smithi043 ceph-mon[27978]: osdmap e113: 12 total, 12 up, 12 in 2024-04-21T08:52:32.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:31 smithi043 ceph-mon[27978]: from='client.? 172.21.15.43:0/999398398' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:52:32.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:31 smithi043 ceph-mon[27978]: pgmap v429: 33 pgs: 33 active+clean; 577 KiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:52:32.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:31 smithi043 systemd[1]: Stopping Ceph mon.a for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:52:32.535 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:32 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-a[27974]: 2024-04-21T08:52:32.184+0000 7f302a629700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:52:32.535 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:32 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-a[27974]: 2024-04-21T08:52:32.184+0000 7f302a629700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-21T08:52:32.535 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:32 smithi043 podman[108130]: 2024-04-21 08:52:32.256372526 +0000 UTC m=+0.195388559 container died 25b38f5f08346eddca90a08429d1cbee8da43b3de422220168e724d9855a4ec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:52:32.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:32 smithi043 podman[108130]: 2024-04-21 08:52:32.66774988 +0000 UTC m=+0.606765926 container cleanup 25b38f5f08346eddca90a08429d1cbee8da43b3de422220168e724d9855a4ec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-21T08:52:32.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:32 smithi043 bash[108130]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-a 2024-04-21T08:52:33.162 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mon.a.service' 2024-04-21T08:52:33.184 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:32 smithi043 podman[108144]: 2024-04-21 08:52:32.919566113 +0000 UTC m=+0.661193417 container remove 25b38f5f08346eddca90a08429d1cbee8da43b3de422220168e724d9855a4ec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-a, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-21T08:52:33.184 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:33 smithi043 systemd[1]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-21T08:52:33.184 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:33 smithi043 systemd[1]: Stopped Ceph mon.a for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:52:33.184 INFO:journalctl@ceph.mon.a.smithi043.stdout:Apr 21 08:52:33 smithi043 systemd[1]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mon.a.service: Consumed 23.705s CPU time. 2024-04-21T08:52:33.210 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:52:33.210 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-21T08:52:33.210 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-21T08:52:33.210 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mon.b 2024-04-21T08:52:33.525 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:33 smithi112 systemd[1]: Stopping Ceph mon.b for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:52:33.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:33 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-b[31081]: 2024-04-21T08:52:33.525+0000 7f8a8b842700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:52:33.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:33 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-b[31081]: 2024-04-21T08:52:33.525+0000 7f8a8b842700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-21T08:52:33.794 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:33 smithi112 podman[67405]: 2024-04-21 08:52:33.618091909 +0000 UTC m=+0.212659545 container died 86479c34e9f656a463119cc52d43009ca0835b29fcf243c79a1f349f3cc47c1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-21T08:52:34.286 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:34 smithi112 podman[67405]: 2024-04-21 08:52:34.018961621 +0000 UTC m=+0.613529255 container cleanup 86479c34e9f656a463119cc52d43009ca0835b29fcf243c79a1f349f3cc47c1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-b, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:52:34.286 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 21 08:52:34 smithi112 bash[67405]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-b 2024-04-21T08:52:34.433 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mon.b.service' 2024-04-21T08:52:34.480 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:52:34.480 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-21T08:52:34.480 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-21T08:52:34.480 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mon.c 2024-04-21T08:52:34.776 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:34 smithi175 systemd[1]: Stopping Ceph mon.c for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:52:35.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:34 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-c[31903]: 2024-04-21T08:52:34.776+0000 7f1618012700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:52:35.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:34 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-c[31903]: 2024-04-21T08:52:34.776+0000 7f1618012700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-21T08:52:35.214 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:35 smithi175 podman[69128]: 2024-04-21 08:52:35.001080556 +0000 UTC m=+0.326317520 container died 44beb75e459f9b983e3d3775928b4d24608fb57db35119d9abdd709e72249d20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-21T08:52:35.611 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:35 smithi175 podman[69128]: 2024-04-21 08:52:35.393591541 +0000 UTC m=+0.718828536 container cleanup 44beb75e459f9b983e3d3775928b4d24608fb57db35119d9abdd709e72249d20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:52:35.611 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:35 smithi175 bash[69128]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-c 2024-04-21T08:52:35.862 INFO:journalctl@ceph.mon.c.smithi175.stdout:Apr 21 08:52:35 smithi175 podman[69142]: 2024-04-21 08:52:35.612143737 +0000 UTC m=+0.611988660 container remove 44beb75e459f9b983e3d3775928b4d24608fb57db35119d9abdd709e72249d20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-21T08:52:35.867 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mon.c.service' 2024-04-21T08:52:35.913 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:52:35.914 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-21T08:52:35.914 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-21T08:52:35.914 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mgr.x 2024-04-21T08:52:36.239 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:52:35 smithi043 systemd[1]: Stopping Ceph mgr.x for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:52:36.562 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:52:36 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:52:36.238+0000 7fa511920700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:52:36.562 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:52:36 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x[28212]: 2024-04-21T08:52:36.238+0000 7fa511920700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-21T08:52:36.562 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:52:36 smithi043 podman[108251]: 2024-04-21 08:52:36.270071244 +0000 UTC m=+0.148018107 container died 7477df00b6fb78720379d2845187ec7156d30543a4cffb70d358b6d6fab2e5dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-21T08:52:36.932 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:52:36 smithi043 podman[108251]: 2024-04-21 08:52:36.706502185 +0000 UTC m=+0.584449045 container cleanup 7477df00b6fb78720379d2845187ec7156d30543a4cffb70d358b6d6fab2e5dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True) 2024-04-21T08:52:36.932 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:52:36 smithi043 bash[108251]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x 2024-04-21T08:52:37.175 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mgr.x.service' 2024-04-21T08:52:37.198 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:52:36 smithi043 podman[108264]: 2024-04-21 08:52:36.933212763 +0000 UTC m=+0.658636218 container remove 7477df00b6fb78720379d2845187ec7156d30543a4cffb70d358b6d6fab2e5dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:52:37.198 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:52:37 smithi043 systemd[1]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mgr.x.service: Deactivated successfully. 2024-04-21T08:52:37.198 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:52:37 smithi043 systemd[1]: Stopped Ceph mgr.x for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:52:37.198 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Apr 21 08:52:37 smithi043 systemd[1]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mgr.x.service: Consumed 1min 36.072s CPU time. 2024-04-21T08:52:37.213 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:52:37.213 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-21T08:52:37.213 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-21T08:52:37.213 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mgr.y 2024-04-21T08:52:37.544 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:52:37 smithi112 systemd[1]: Stopping Ceph mgr.y for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:52:37.815 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:52:37 smithi112 podman[67528]: 2024-04-21 08:52:37.552786496 +0000 UTC m=+0.142504586 container died 905d62cd1f86b9d74a2509f442789538a4a2cd97b544879d9c1aa0fb44563545 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-21T08:52:38.154 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:52:37 smithi112 podman[67528]: 2024-04-21 08:52:37.915814166 +0000 UTC m=+0.505532295 container cleanup 905d62cd1f86b9d74a2509f442789538a4a2cd97b544879d9c1aa0fb44563545 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:52:38.154 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 21 08:52:37 smithi112 bash[67528]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-y 2024-04-21T08:52:38.343 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mgr.y.service' 2024-04-21T08:52:38.375 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:52:38.375 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-21T08:52:38.375 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-21T08:52:38.376 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mgr.z 2024-04-21T08:52:38.714 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:52:38 smithi175 systemd[1]: Stopping Ceph mgr.z for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:52:39.024 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:52:38 smithi175 podman[69249]: 2024-04-21 08:52:38.728004776 +0000 UTC m=+0.155531685 container died b7aab1eaed462ac8ec5008d5fe9e3014779cef450f60e2e4f1bda5a5f9d0efd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:52:39.409 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:52:39 smithi175 podman[69249]: 2024-04-21 08:52:39.124823976 +0000 UTC m=+0.552350926 container cleanup b7aab1eaed462ac8ec5008d5fe9e3014779cef450f60e2e4f1bda5a5f9d0efd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:52:39.409 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Apr 21 08:52:39 smithi175 bash[69249]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mgr-z 2024-04-21T08:52:39.555 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mgr.z.service' 2024-04-21T08:52:39.588 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:52:39.588 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-21T08:52:39.588 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-21T08:52:39.588 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.0 2024-04-21T08:52:39.952 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 21 08:52:39 smithi043 systemd[1]: Stopping Ceph osd.0 for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:52:40.311 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 21 08:52:39 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-0[40338]: 2024-04-21T08:52:39.952+0000 7fef11f87700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:52:40.312 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 21 08:52:39 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-0[40338]: 2024-04-21T08:52:39.952+0000 7fef11f87700 -1 osd.0 114 *** Got signal Terminated *** 2024-04-21T08:52:40.312 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 21 08:52:39 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-0[40338]: 2024-04-21T08:52:39.952+0000 7fef11f87700 -1 osd.0 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:52:45.562 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 21 08:52:45 smithi043 podman[108373]: 2024-04-21 08:52:45.292479424 +0000 UTC m=+5.505226243 container died 4c5fd87f8f4fcec37548aa49cf4621a3a5f360a6ad6266220b38c4d5b3718bbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-21T08:52:46.062 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 21 08:52:45 smithi043 podman[108373]: 2024-04-21 08:52:45.744668194 +0000 UTC m=+5.957415012 container cleanup 4c5fd87f8f4fcec37548aa49cf4621a3a5f360a6ad6266220b38c4d5b3718bbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:52:46.062 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 21 08:52:45 smithi043 bash[108373]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-0 2024-04-21T08:52:46.326 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 21 08:52:46 smithi043 podman[108390]: 2024-04-21 08:52:46.07094097 +0000 UTC m=+0.774949907 container remove 4c5fd87f8f4fcec37548aa49cf4621a3a5f360a6ad6266220b38c4d5b3718bbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:52:46.602 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 21 08:52:46 smithi043 podman[108477]: 2024-04-21 08:52:46.331817893 +0000 UTC m=+0.021533446 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:52:46.602 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 21 08:52:46 smithi043 podman[108477]: 2024-04-21 08:52:46.463849299 +0000 UTC m=+0.153564874 container create 3f95608f22a8d807be1d3cf767a24f0f99c31297a588101688cbfc06e7721f3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-0-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-21T08:52:47.049 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 21 08:52:46 smithi043 podman[108477]: 2024-04-21 08:52:46.741910452 +0000 UTC m=+0.431625988 container init 3f95608f22a8d807be1d3cf767a24f0f99c31297a588101688cbfc06e7721f3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-21T08:52:47.049 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 21 08:52:46 smithi043 podman[108477]: 2024-04-21 08:52:46.745940397 +0000 UTC m=+0.435655934 container start 3f95608f22a8d807be1d3cf767a24f0f99c31297a588101688cbfc06e7721f3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux ) 2024-04-21T08:52:47.049 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 21 08:52:46 smithi043 podman[108477]: 2024-04-21 08:52:46.841780372 +0000 UTC m=+0.531495920 container attach 3f95608f22a8d807be1d3cf767a24f0f99c31297a588101688cbfc06e7721f3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:52:47.311 INFO:journalctl@ceph.osd.0.smithi043.stdout:Apr 21 08:52:47 smithi043 podman[108477]: 2024-04-21 08:52:47.050000319 +0000 UTC m=+0.739715862 container died 3f95608f22a8d807be1d3cf767a24f0f99c31297a588101688cbfc06e7721f3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:52:47.681 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.0.service' 2024-04-21T08:52:47.721 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:52:47.721 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-21T08:52:47.721 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-21T08:52:47.722 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.1 2024-04-21T08:52:48.044 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 21 08:52:47 smithi112 systemd[1]: Stopping Ceph osd.1 for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:52:48.544 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 21 08:52:48 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-1[35682]: 2024-04-21T08:52:48.119+0000 7f3be6ae9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:52:48.544 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 21 08:52:48 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-1[35682]: 2024-04-21T08:52:48.119+0000 7f3be6ae9700 -1 osd.1 114 *** Got signal Terminated *** 2024-04-21T08:52:48.544 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 21 08:52:48 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-1[35682]: 2024-04-21T08:52:48.119+0000 7f3be6ae9700 -1 osd.1 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:52:53.912 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 21 08:52:53 smithi112 podman[67655]: 2024-04-21 08:52:53.618971505 +0000 UTC m=+5.709339526 container died 0afe0693ecae660ff8416e9b2c3e76807d8aaf01c98ea80054e3f54625639967 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-21T08:52:54.279 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 21 08:52:54 smithi112 podman[67655]: 2024-04-21 08:52:54.012134365 +0000 UTC m=+6.102502386 container cleanup 0afe0693ecae660ff8416e9b2c3e76807d8aaf01c98ea80054e3f54625639967 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-1, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:52:54.279 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 21 08:52:54 smithi112 bash[67655]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-1 2024-04-21T08:52:54.544 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 21 08:52:54 smithi112 podman[67673]: 2024-04-21 08:52:54.28046068 +0000 UTC m=+0.652669658 container remove 0afe0693ecae660ff8416e9b2c3e76807d8aaf01c98ea80054e3f54625639967 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-1, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:52:54.916 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 21 08:52:54 smithi112 podman[67766]: 2024-04-21 08:52:54.537071103 +0000 UTC m=+0.017178504 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:52:54.916 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 21 08:52:54 smithi112 podman[67766]: 2024-04-21 08:52:54.672958205 +0000 UTC m=+0.153065588 container create dae35f088b50401b7ff2e0a50a5bda32ddb5001e5dca6c00f0b005f694e0a906 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-21T08:52:54.917 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 21 08:52:54 smithi112 podman[67766]: 2024-04-21 08:52:54.917562557 +0000 UTC m=+0.397669940 container init dae35f088b50401b7ff2e0a50a5bda32ddb5001e5dca6c00f0b005f694e0a906 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-21T08:52:55.223 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 21 08:52:54 smithi112 podman[67766]: 2024-04-21 08:52:54.920955894 +0000 UTC m=+0.401063286 container start dae35f088b50401b7ff2e0a50a5bda32ddb5001e5dca6c00f0b005f694e0a906 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-21T08:52:55.223 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 21 08:52:55 smithi112 podman[67766]: 2024-04-21 08:52:55.017755629 +0000 UTC m=+0.497863023 container attach dae35f088b50401b7ff2e0a50a5bda32ddb5001e5dca6c00f0b005f694e0a906 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-21T08:52:55.493 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 21 08:52:55 smithi112 podman[67766]: 2024-04-21 08:52:55.22399053 +0000 UTC m=+0.704097905 container died dae35f088b50401b7ff2e0a50a5bda32ddb5001e5dca6c00f0b005f694e0a906 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-1-deactivate, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-21T08:52:56.048 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.1.service' 2024-04-21T08:52:56.084 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:52:56.085 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-21T08:52:56.085 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-21T08:52:56.085 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.2 2024-04-21T08:52:56.411 INFO:journalctl@ceph.osd.2.smithi175.stdout:Apr 21 08:52:56 smithi175 systemd[1]: Stopping Ceph osd.2 for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:52:56.673 INFO:journalctl@ceph.osd.2.smithi175.stdout:Apr 21 08:52:56 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-2[37058]: 2024-04-21T08:52:56.411+0000 7fc80d919700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:52:56.673 INFO:journalctl@ceph.osd.2.smithi175.stdout:Apr 21 08:52:56 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-2[37058]: 2024-04-21T08:52:56.411+0000 7fc80d919700 -1 osd.2 114 *** Got signal Terminated *** 2024-04-21T08:52:56.673 INFO:journalctl@ceph.osd.2.smithi175.stdout:Apr 21 08:52:56 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-2[37058]: 2024-04-21T08:52:56.411+0000 7fc80d919700 -1 osd.2 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:53:02.103 INFO:journalctl@ceph.osd.2.smithi175.stdout:Apr 21 08:53:01 smithi175 podman[69375]: 2024-04-21 08:53:01.807350936 +0000 UTC m=+5.534560999 container died 0a0f70c3df286704f24926fedb92b5d053e307d1f618a9d5430b616f43fcce87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS) 2024-04-21T08:53:02.464 INFO:journalctl@ceph.osd.2.smithi175.stdout:Apr 21 08:53:02 smithi175 podman[69375]: 2024-04-21 08:53:02.203554905 +0000 UTC m=+5.930764969 container cleanup 0a0f70c3df286704f24926fedb92b5d053e307d1f618a9d5430b616f43fcce87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:53:02.464 INFO:journalctl@ceph.osd.2.smithi175.stdout:Apr 21 08:53:02 smithi175 bash[69375]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-2 2024-04-21T08:53:02.828 INFO:journalctl@ceph.osd.2.smithi175.stdout:Apr 21 08:53:02 smithi175 podman[69406]: 2024-04-21 08:53:02.488511111 +0000 UTC m=+0.673533340 container remove 0a0f70c3df286704f24926fedb92b5d053e307d1f618a9d5430b616f43fcce87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:53:03.133 INFO:journalctl@ceph.osd.2.smithi175.stdout:Apr 21 08:53:02 smithi175 podman[69495]: 2024-04-21 08:53:02.733180244 +0000 UTC m=+0.018154265 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:53:03.133 INFO:journalctl@ceph.osd.2.smithi175.stdout:Apr 21 08:53:02 smithi175 podman[69495]: 2024-04-21 08:53:02.864915952 +0000 UTC m=+0.149890016 container create 470f21dad7a2be980351840398949f26fa4c3155f900d9f44482bb11fefd40ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-21T08:53:03.133 INFO:journalctl@ceph.osd.2.smithi175.stdout:Apr 21 08:53:03 smithi175 podman[69495]: 2024-04-21 08:53:03.134192396 +0000 UTC m=+0.419166416 container init 470f21dad7a2be980351840398949f26fa4c3155f900d9f44482bb11fefd40ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:53:03.426 INFO:journalctl@ceph.osd.2.smithi175.stdout:Apr 21 08:53:03 smithi175 podman[69495]: 2024-04-21 08:53:03.138036902 +0000 UTC m=+0.423010921 container start 470f21dad7a2be980351840398949f26fa4c3155f900d9f44482bb11fefd40ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416) 2024-04-21T08:53:03.426 INFO:journalctl@ceph.osd.2.smithi175.stdout:Apr 21 08:53:03 smithi175 podman[69495]: 2024-04-21 08:53:03.251225755 +0000 UTC m=+0.536199774 container attach 470f21dad7a2be980351840398949f26fa4c3155f900d9f44482bb11fefd40ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-21T08:53:03.714 INFO:journalctl@ceph.osd.2.smithi175.stdout:Apr 21 08:53:03 smithi175 podman[69495]: 2024-04-21 08:53:03.426999229 +0000 UTC m=+0.711973246 container died 470f21dad7a2be980351840398949f26fa4c3155f900d9f44482bb11fefd40ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True) 2024-04-21T08:53:04.133 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.2.service' 2024-04-21T08:53:04.173 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:53:04.173 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-21T08:53:04.173 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-21T08:53:04.173 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.3 2024-04-21T08:53:04.497 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 21 08:53:04 smithi043 systemd[1]: Stopping Ceph osd.3 for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:53:04.811 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 21 08:53:04 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-3[46872]: 2024-04-21T08:53:04.497+0000 7fd5defb2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:53:04.811 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 21 08:53:04 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-3[46872]: 2024-04-21T08:53:04.497+0000 7fd5defb2700 -1 osd.3 114 *** Got signal Terminated *** 2024-04-21T08:53:04.812 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 21 08:53:04 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-3[46872]: 2024-04-21T08:53:04.497+0000 7fd5defb2700 -1 osd.3 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:53:07.515 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 21 08:53:07 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-3[46872]: 2024-04-21T08:53:07.214+0000 7fd5db5a6700 -1 osd.3 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:40.385916+0000 front 2024-04-21T08:52:40.385951+0000 (oldest deadline 2024-04-21T08:53:06.286042+0000) 2024-04-21T08:53:08.311 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 21 08:53:08 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-3[46872]: 2024-04-21T08:53:08.193+0000 7fd5db5a6700 -1 osd.3 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:40.385916+0000 front 2024-04-21T08:52:40.385951+0000 (oldest deadline 2024-04-21T08:53:06.286042+0000) 2024-04-21T08:53:08.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:08 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:08.056+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:09.062 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:09 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:09.038+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:09.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:08 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:08.899+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:09.561 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 21 08:53:09 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-3[46872]: 2024-04-21T08:53:09.178+0000 7fd5db5a6700 -1 osd.3 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:40.385916+0000 front 2024-04-21T08:52:40.385951+0000 (oldest deadline 2024-04-21T08:53:06.286042+0000) 2024-04-21T08:53:10.062 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 21 08:53:09 smithi043 podman[108585]: 2024-04-21 08:53:09.856931399 +0000 UTC m=+5.478931841 container died 656d0767e0ea321e14d752e9cf4924382d2fcbcf3a9bda207cc9a3d868400044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-21T08:53:10.062 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:10 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:10.039+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:10.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:09 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:09.856+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:10.540 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 21 08:53:10 smithi043 podman[108585]: 2024-04-21 08:53:10.247971905 +0000 UTC m=+5.869972344 container cleanup 656d0767e0ea321e14d752e9cf4924382d2fcbcf3a9bda207cc9a3d868400044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-3, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-04-21T08:53:10.540 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 21 08:53:10 smithi043 bash[108585]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-3 2024-04-21T08:53:10.544 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:10 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:10.112+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:10.811 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 21 08:53:10 smithi043 podman[108614]: 2024-04-21 08:53:10.541066272 +0000 UTC m=+0.682348976 container remove 656d0767e0ea321e14d752e9cf4924382d2fcbcf3a9bda207cc9a3d868400044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-3, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:53:11.135 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:11 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:11.071+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:11.135 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 21 08:53:10 smithi043 podman[108703]: 2024-04-21 08:53:10.775962796 +0000 UTC m=+0.018504221 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:53:11.135 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 21 08:53:10 smithi043 podman[108703]: 2024-04-21 08:53:10.900270331 +0000 UTC m=+0.142811775 container create bb98f039e877bff931cecc391023604ae8c87b1bc270381b773bec206da8745d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0) 2024-04-21T08:53:11.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:10 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:10.855+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:11.446 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 21 08:53:11 smithi043 podman[108703]: 2024-04-21 08:53:11.136195933 +0000 UTC m=+0.378737332 container init bb98f039e877bff931cecc391023604ae8c87b1bc270381b773bec206da8745d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:53:11.447 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 21 08:53:11 smithi043 podman[108703]: 2024-04-21 08:53:11.139854948 +0000 UTC m=+0.382396354 container start bb98f039e877bff931cecc391023604ae8c87b1bc270381b773bec206da8745d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:53:11.447 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 21 08:53:11 smithi043 podman[108703]: 2024-04-21 08:53:11.236177063 +0000 UTC m=+0.478718474 container attach bb98f039e877bff931cecc391023604ae8c87b1bc270381b773bec206da8745d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:53:11.544 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:11 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:11.147+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:11.797 INFO:journalctl@ceph.osd.3.smithi043.stdout:Apr 21 08:53:11 smithi043 podman[108703]: 2024-04-21 08:53:11.447411334 +0000 UTC m=+0.689952745 container died bb98f039e877bff931cecc391023604ae8c87b1bc270381b773bec206da8745d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, ceph=True) 2024-04-21T08:53:12.093 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.3.service' 2024-04-21T08:53:12.133 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:53:12.133 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-21T08:53:12.133 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-21T08:53:12.134 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.4 2024-04-21T08:53:12.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:11 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:11.878+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:12.433 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 21 08:53:12 smithi112 systemd[1]: Stopping Ceph osd.4 for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:53:12.433 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:12 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:12.154+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:12.561 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:12 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:12.078+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:12.794 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 21 08:53:12 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-4[42328]: 2024-04-21T08:53:12.433+0000 7f957dad8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:53:12.794 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 21 08:53:12 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-4[42328]: 2024-04-21T08:53:12.433+0000 7f957dad8700 -1 osd.4 114 *** Got signal Terminated *** 2024-04-21T08:53:12.794 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 21 08:53:12 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-4[42328]: 2024-04-21T08:53:12.433+0000 7f957dad8700 -1 osd.4 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:53:13.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:12 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:12.839+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:13.544 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:13 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:13.201+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:13.561 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:13 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:13.118+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:14.185 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:13 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:13.859+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:14.544 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:14 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:14.246+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:14.544 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:14 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:14.246+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:14.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:14 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:14.166+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:14.562 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:14 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:14.073+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.646363+0000 front 2024-04-21T08:52:51.646409+0000 (oldest deadline 2024-04-21T08:53:13.946519+0000) 2024-04-21T08:53:15.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:14 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:14.840+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:15.312 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:15 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:15.046+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.646363+0000 front 2024-04-21T08:52:51.646409+0000 (oldest deadline 2024-04-21T08:53:13.946519+0000) 2024-04-21T08:53:15.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:15 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:15.135+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:15.544 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 21 08:53:15 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-4[42328]: 2024-04-21T08:53:15.266+0000 7f957a0cc700 -1 osd.4 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:47.469641+0000 front 2024-04-21T08:52:47.469679+0000 (oldest deadline 2024-04-21T08:53:14.470335+0000) 2024-04-21T08:53:15.545 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:15 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:15.391+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:15.545 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:15 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:15.249+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:15.545 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:15 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:15.249+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:15.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:15 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:15.793+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:15.964 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:15 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5[44012]: 2024-04-21T08:53:15.574+0000 7fb49e5b8700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:50.935160+0000 front 2024-04-21T08:52:50.935389+0000 (oldest deadline 2024-04-21T08:53:15.035081+0000) 2024-04-21T08:53:16.312 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:16 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:16.006+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.646363+0000 front 2024-04-21T08:52:51.646409+0000 (oldest deadline 2024-04-21T08:53:13.946519+0000) 2024-04-21T08:53:16.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:16 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:16.167+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:16.464 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:16 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:16.008+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:16.544 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 21 08:53:16 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-4[42328]: 2024-04-21T08:53:16.301+0000 7f957a0cc700 -1 osd.4 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:47.469641+0000 front 2024-04-21T08:52:47.469679+0000 (oldest deadline 2024-04-21T08:53:14.470335+0000) 2024-04-21T08:53:16.544 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:16 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:16.437+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:16.545 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:16 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:16.245+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:16.545 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:16 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:16.245+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:16.823 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:16 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5[44012]: 2024-04-21T08:53:16.531+0000 7fb49e5b8700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:50.935160+0000 front 2024-04-21T08:52:50.935389+0000 (oldest deadline 2024-04-21T08:53:15.035081+0000) 2024-04-21T08:53:17.214 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:17 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:17.034+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:17.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:16 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:16.823+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:17.311 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:17 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:17.001+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.646363+0000 front 2024-04-21T08:52:51.646409+0000 (oldest deadline 2024-04-21T08:53:13.946519+0000) 2024-04-21T08:53:17.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:17 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:17.132+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:17.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:17 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:17.132+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:17.544 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 21 08:53:17 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-4[42328]: 2024-04-21T08:53:17.299+0000 7f957a0cc700 -1 osd.4 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:47.469641+0000 front 2024-04-21T08:52:47.469679+0000 (oldest deadline 2024-04-21T08:53:14.470335+0000) 2024-04-21T08:53:17.545 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:17 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:17.425+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:17.545 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:17 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:17.263+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:17.545 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:17 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:17.263+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:17.856 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:17 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5[44012]: 2024-04-21T08:53:17.505+0000 7fb49e5b8700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:50.935160+0000 front 2024-04-21T08:52:50.935389+0000 (oldest deadline 2024-04-21T08:53:15.035081+0000) 2024-04-21T08:53:18.214 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:18 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:18.074+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:18.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:17 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:17.856+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:18.250 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 21 08:53:17 smithi112 podman[67882]: 2024-04-21 08:53:17.959203075 +0000 UTC m=+5.638404625 container died 828ee7946705ddd97e75982e09bb66305026768866e50903bcf9f0c0cf0b17f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-21T08:53:18.312 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:18 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:18.021+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.646363+0000 front 2024-04-21T08:52:51.646409+0000 (oldest deadline 2024-04-21T08:53:13.946519+0000) 2024-04-21T08:53:18.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:18 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:18.086+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:18.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:18 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:18.086+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:18.544 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:18 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:18.416+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:18.544 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:18 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:18.310+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:18.545 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:18 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:18.310+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:18.545 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 21 08:53:18 smithi112 podman[67882]: 2024-04-21 08:53:18.35026096 +0000 UTC m=+6.029462506 container cleanup 828ee7946705ddd97e75982e09bb66305026768866e50903bcf9f0c0cf0b17f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-4, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:53:18.545 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 21 08:53:18 smithi112 bash[67882]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-4 2024-04-21T08:53:18.714 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:18 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5[44012]: 2024-04-21T08:53:18.459+0000 7fb49e5b8700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:50.935160+0000 front 2024-04-21T08:52:50.935389+0000 (oldest deadline 2024-04-21T08:53:15.035081+0000) 2024-04-21T08:53:18.970 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 21 08:53:18 smithi112 podman[67899]: 2024-04-21 08:53:18.634792076 +0000 UTC m=+0.674499395 container remove 828ee7946705ddd97e75982e09bb66305026768866e50903bcf9f0c0cf0b17f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:53:19.109 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:18 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:18.830+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:19.110 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:18 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:18.830+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:19.229 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 21 08:53:18 smithi112 podman[67990]: 2024-04-21 08:53:18.874909028 +0000 UTC m=+0.018139462 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:53:19.229 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 21 08:53:18 smithi112 podman[67990]: 2024-04-21 08:53:18.994172275 +0000 UTC m=+0.137402737 container create 9c3580b2e86ff9f59da1c5fab114ad8882dd05a9c5584b7500b4f8111b6a6a5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-21T08:53:19.272 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:19 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:19.005+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.646363+0000 front 2024-04-21T08:52:51.646409+0000 (oldest deadline 2024-04-21T08:53:13.946519+0000) 2024-04-21T08:53:19.272 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:19 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:19.109+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:19.272 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:19 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:19.109+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:19.464 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:19 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:19.110+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:19.526 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 21 08:53:19 smithi112 podman[67990]: 2024-04-21 08:53:19.230353614 +0000 UTC m=+0.373584034 container init 9c3580b2e86ff9f59da1c5fab114ad8882dd05a9c5584b7500b4f8111b6a6a5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:53:19.526 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 21 08:53:19 smithi112 podman[67990]: 2024-04-21 08:53:19.233433776 +0000 UTC m=+0.376664200 container start 9c3580b2e86ff9f59da1c5fab114ad8882dd05a9c5584b7500b4f8111b6a6a5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-21T08:53:19.526 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 21 08:53:19 smithi112 podman[67990]: 2024-04-21 08:53:19.338797731 +0000 UTC m=+0.482028152 container attach 9c3580b2e86ff9f59da1c5fab114ad8882dd05a9c5584b7500b4f8111b6a6a5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:53:19.526 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:19 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:19.421+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:19.527 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:19 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:19.307+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:19.527 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:19 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:19.307+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:19.794 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 21 08:53:19 smithi112 podman[67990]: 2024-04-21 08:53:19.526946579 +0000 UTC m=+0.670176995 container died 9c3580b2e86ff9f59da1c5fab114ad8882dd05a9c5584b7500b4f8111b6a6a5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:53:19.831 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:19 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5[44012]: 2024-04-21T08:53:19.493+0000 7fb49e5b8700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:50.935160+0000 front 2024-04-21T08:52:50.935389+0000 (oldest deadline 2024-04-21T08:53:15.035081+0000) 2024-04-21T08:53:20.147 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:20 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:20.036+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.646363+0000 front 2024-04-21T08:52:51.646409+0000 (oldest deadline 2024-04-21T08:53:13.946519+0000) 2024-04-21T08:53:20.150 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:19 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:19.831+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:20.150 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:19 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:19.831+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:20.168 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.4.service' 2024-04-21T08:53:20.206 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:53:20.207 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-21T08:53:20.207 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-21T08:53:20.207 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.5 2024-04-21T08:53:20.402 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:20 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:20.273+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:20.403 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:20 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:20.273+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:20.454 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:20 smithi175 systemd[1]: Stopping Ceph osd.5 for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:53:20.455 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:20 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:20.150+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:20.561 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:20 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:20.147+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:20.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:20 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:20.147+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:20.714 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:20 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5[44012]: 2024-04-21T08:53:20.454+0000 7fb49e5b8700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:50.935160+0000 front 2024-04-21T08:52:50.935389+0000 (oldest deadline 2024-04-21T08:53:15.035081+0000) 2024-04-21T08:53:20.714 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:20 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5[44012]: 2024-04-21T08:53:20.548+0000 7fb4a5fcc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:53:20.714 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:20 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5[44012]: 2024-04-21T08:53:20.548+0000 7fb4a5fcc700 -1 osd.5 114 *** Got signal Terminated *** 2024-04-21T08:53:20.714 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:20 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5[44012]: 2024-04-21T08:53:20.548+0000 7fb4a5fcc700 -1 osd.5 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:53:20.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:20 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:20.403+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:21.165 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:20 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:20.842+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:21.165 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:20 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:20.842+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:21.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:21 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:21.162+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:21.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:21 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:21.162+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:21.312 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:21 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:21.020+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.646363+0000 front 2024-04-21T08:52:51.646409+0000 (oldest deadline 2024-04-21T08:53:13.946519+0000) 2024-04-21T08:53:21.443 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:21 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:21.166+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:21.714 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:21 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5[44012]: 2024-04-21T08:53:21.444+0000 7fb49e5b8700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:50.935160+0000 front 2024-04-21T08:52:50.935389+0000 (oldest deadline 2024-04-21T08:53:15.035081+0000) 2024-04-21T08:53:21.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:21 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:21.361+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:21.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:21 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:21.320+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:21.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:21 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:21.320+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:22.168 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:21 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:21.807+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:22.168 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:21 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:21.807+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:22.311 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:21 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:21.982+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.646363+0000 front 2024-04-21T08:52:51.646409+0000 (oldest deadline 2024-04-21T08:53:13.946519+0000) 2024-04-21T08:53:22.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:22 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:22.184+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:22.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:22 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:22.184+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:22.435 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:22 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:22.168+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:22.714 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:22 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5[44012]: 2024-04-21T08:53:22.435+0000 7fb49e5b8700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:50.935160+0000 front 2024-04-21T08:52:50.935389+0000 (oldest deadline 2024-04-21T08:53:15.035081+0000) 2024-04-21T08:53:22.714 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:22 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5[44012]: 2024-04-21T08:53:22.435+0000 7fb49e5b8700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:56.736176+0000 front 2024-04-21T08:52:56.736294+0000 (oldest deadline 2024-04-21T08:53:22.036107+0000) 2024-04-21T08:53:22.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:22 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:22.369+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:22.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:22 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:22.321+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:22.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:22 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:22.321+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:23.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:22 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:22.790+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:23.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:22 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:22.790+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:23.312 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:23 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:23.012+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.646363+0000 front 2024-04-21T08:52:51.646409+0000 (oldest deadline 2024-04-21T08:53:13.946519+0000) 2024-04-21T08:53:23.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:23 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:23.230+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:23.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:23 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:23.230+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:23.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:23 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:23.230+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:23.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:23 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:23.215+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:23.714 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:23 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5[44012]: 2024-04-21T08:53:23.421+0000 7fb49e5b8700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:50.935160+0000 front 2024-04-21T08:52:50.935389+0000 (oldest deadline 2024-04-21T08:53:15.035081+0000) 2024-04-21T08:53:23.714 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:23 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5[44012]: 2024-04-21T08:53:23.421+0000 7fb49e5b8700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:56.736176+0000 front 2024-04-21T08:52:56.736294+0000 (oldest deadline 2024-04-21T08:53:22.036107+0000) 2024-04-21T08:53:23.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:23 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:23.385+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:23.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:23 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:23.365+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:23.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:23 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:23.365+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:24.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:23 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:23.800+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:24.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:23 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:23.800+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:24.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:23 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:23.800+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:24.312 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:24 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:24.042+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.646363+0000 front 2024-04-21T08:52:51.646409+0000 (oldest deadline 2024-04-21T08:53:13.946519+0000) 2024-04-21T08:53:24.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:24 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:24.214+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:24.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:24 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:24.214+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:24.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:24 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:24.214+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:24.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:24 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:24.230+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:24.714 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:24 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5[44012]: 2024-04-21T08:53:24.381+0000 7fb49e5b8700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:50.935160+0000 front 2024-04-21T08:52:50.935389+0000 (oldest deadline 2024-04-21T08:53:15.035081+0000) 2024-04-21T08:53:24.714 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:24 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5[44012]: 2024-04-21T08:53:24.381+0000 7fb49e5b8700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:56.736176+0000 front 2024-04-21T08:52:56.736294+0000 (oldest deadline 2024-04-21T08:53:22.036107+0000) 2024-04-21T08:53:24.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:24 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:24.372+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:24.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:24 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:24.399+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:24.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:24 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:24.399+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:25.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:24 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:24.800+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:25.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:24 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:24.800+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:25.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:24 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:24.800+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:25.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:25 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:25.220+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:25.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:25 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:25.220+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:25.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:25 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:25.220+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:25.563 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:25 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:25.088+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.646363+0000 front 2024-04-21T08:52:51.646409+0000 (oldest deadline 2024-04-21T08:53:13.946519+0000) 2024-04-21T08:53:25.563 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:25 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:25.088+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.948431+0000 front 2024-04-21T08:53:00.948313+0000 (oldest deadline 2024-04-21T08:53:24.448312+0000) 2024-04-21T08:53:25.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:25 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:25.231+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:25.714 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:25 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5[44012]: 2024-04-21T08:53:25.415+0000 7fb49e5b8700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:50.935160+0000 front 2024-04-21T08:52:50.935389+0000 (oldest deadline 2024-04-21T08:53:15.035081+0000) 2024-04-21T08:53:25.714 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:25 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5[44012]: 2024-04-21T08:53:25.415+0000 7fb49e5b8700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:56.736176+0000 front 2024-04-21T08:52:56.736294+0000 (oldest deadline 2024-04-21T08:53:22.036107+0000) 2024-04-21T08:53:25.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:25 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:25.418+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:25.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:25 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:25.418+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.963839+0000 front 2024-04-21T08:53:00.963989+0000 (oldest deadline 2024-04-21T08:53:25.063990+0000) 2024-04-21T08:53:25.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:25 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:25.429+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:25.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:25 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:25.429+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:26.214 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:25 smithi175 podman[69602]: 2024-04-21 08:53:25.944023583 +0000 UTC m=+5.535016456 container died db669708d411a489b1663aa489f424c63193a9a7bffcb275d38d34f25b1fcd73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:53:26.215 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:25 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:25.799+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:26.215 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:25 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:25.799+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:26.215 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:25 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:25.800+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:26.561 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:26 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:26.229+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:26.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:26 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:26.229+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:26.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:26 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:26.229+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:26.562 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:26 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:26.098+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.646363+0000 front 2024-04-21T08:52:51.646409+0000 (oldest deadline 2024-04-21T08:53:13.946519+0000) 2024-04-21T08:53:26.562 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:26 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:26.098+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.948431+0000 front 2024-04-21T08:53:00.948313+0000 (oldest deadline 2024-04-21T08:53:24.448312+0000) 2024-04-21T08:53:26.608 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:26 smithi175 podman[69602]: 2024-04-21 08:53:26.35765516 +0000 UTC m=+5.948648023 container cleanup db669708d411a489b1663aa489f424c63193a9a7bffcb275d38d34f25b1fcd73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5) 2024-04-21T08:53:26.608 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:26 smithi175 bash[69602]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5 2024-04-21T08:53:26.608 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:26 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:26.241+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:26.608 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:26 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:26.241+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.614735+0000 front 2024-04-21T08:52:59.614778+0000 (oldest deadline 2024-04-21T08:53:25.514902+0000) 2024-04-21T08:53:26.696 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:26 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:26.473+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:26.696 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:26 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:26.473+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:26.696 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:26 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:26.473+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:26.696 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:26 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:26.421+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:26.696 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:26 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:26.421+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.963839+0000 front 2024-04-21T08:53:00.963989+0000 (oldest deadline 2024-04-21T08:53:25.063990+0000) 2024-04-21T08:53:26.964 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:26 smithi175 podman[69622]: 2024-04-21 08:53:26.609309439 +0000 UTC m=+0.658354956 container remove db669708d411a489b1663aa489f424c63193a9a7bffcb275d38d34f25b1fcd73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-21T08:53:26.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:26 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:26.822+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:26.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:26 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:26.822+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:26.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:26 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:26.822+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:27.320 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:27 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:27.255+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:27.321 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:27 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:27.255+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.614735+0000 front 2024-04-21T08:52:59.614778+0000 (oldest deadline 2024-04-21T08:53:25.514902+0000) 2024-04-21T08:53:27.321 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:27 smithi175 podman[69710]: 2024-04-21 08:53:26.935178413 +0000 UTC m=+0.017929246 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:53:27.321 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:27 smithi175 podman[69710]: 2024-04-21 08:53:27.074077104 +0000 UTC m=+0.156827927 container create c600699e751501d87f644404fabf80560fc5cb71258560d9fd20df24cb106c47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:53:27.321 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:27 smithi175 podman[69710]: 2024-04-21 08:53:27.321651828 +0000 UTC m=+0.404402647 container init c600699e751501d87f644404fabf80560fc5cb71258560d9fd20df24cb106c47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-21T08:53:27.562 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:27 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:27.076+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.646363+0000 front 2024-04-21T08:52:51.646409+0000 (oldest deadline 2024-04-21T08:53:13.946519+0000) 2024-04-21T08:53:27.562 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:27 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:27.076+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.948431+0000 front 2024-04-21T08:53:00.948313+0000 (oldest deadline 2024-04-21T08:53:24.448312+0000) 2024-04-21T08:53:27.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:27 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:27.201+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:27.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:27 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:27.201+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:27.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:27 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:27.201+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:27.614 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:27 smithi175 podman[69710]: 2024-04-21 08:53:27.324865479 +0000 UTC m=+0.407616299 container start c600699e751501d87f644404fabf80560fc5cb71258560d9fd20df24cb106c47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-21T08:53:27.615 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:27 smithi175 podman[69710]: 2024-04-21 08:53:27.421737207 +0000 UTC m=+0.504488041 container attach c600699e751501d87f644404fabf80560fc5cb71258560d9fd20df24cb106c47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-04-21T08:53:27.715 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:27 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:27.375+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:27.715 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:27 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:27.375+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.963839+0000 front 2024-04-21T08:53:00.963989+0000 (oldest deadline 2024-04-21T08:53:25.063990+0000) 2024-04-21T08:53:27.715 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:27 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:27.505+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:27.716 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:27 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:27.505+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:27.716 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:27 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:27.505+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:27.964 INFO:journalctl@ceph.osd.5.smithi175.stdout:Apr 21 08:53:27 smithi175 podman[69710]: 2024-04-21 08:53:27.61626459 +0000 UTC m=+0.699015419 container died c600699e751501d87f644404fabf80560fc5cb71258560d9fd20df24cb106c47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-21T08:53:27.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:27 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:27.787+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:27.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:27 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:27.787+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:27.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:27 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:27.787+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:28.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:28 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:28.196+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:28.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:28 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:28.196+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:28.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:28 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:28.196+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:28.312 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:28 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:28.049+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.646363+0000 front 2024-04-21T08:52:51.646409+0000 (oldest deadline 2024-04-21T08:53:13.946519+0000) 2024-04-21T08:53:28.312 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:28 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:28.049+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.948431+0000 front 2024-04-21T08:53:00.948313+0000 (oldest deadline 2024-04-21T08:53:24.448312+0000) 2024-04-21T08:53:28.317 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.5.service' 2024-04-21T08:53:28.357 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:53:28.358 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-21T08:53:28.358 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-21T08:53:28.358 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.6 2024-04-21T08:53:28.570 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:28 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:28.269+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:28.571 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:28 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:28.269+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.614735+0000 front 2024-04-21T08:52:59.614778+0000 (oldest deadline 2024-04-21T08:53:25.514902+0000) 2024-04-21T08:53:28.675 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:28 smithi043 systemd[1]: Stopping Ceph osd.6 for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:53:28.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:28 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:28.339+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:28.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:28 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:28.339+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.963839+0000 front 2024-04-21T08:53:00.963989+0000 (oldest deadline 2024-04-21T08:53:25.063990+0000) 2024-04-21T08:53:28.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:28 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:28.474+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:28.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:28 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:28.474+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:28.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:28 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:28.474+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:28.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:28 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:28.750+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:28.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:28 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:28.750+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:28.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:28 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:28.750+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:29.060 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:28 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:28.674+0000 7efcb4ca7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:53:29.061 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:28 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:28.674+0000 7efcb4ca7700 -1 osd.6 114 *** Got signal Terminated *** 2024-04-21T08:53:29.061 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:28 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:28.674+0000 7efcb4ca7700 -1 osd.6 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:53:29.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:29 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:29.197+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:29.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:29 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:29.197+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:29.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:29 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:29.197+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:29.312 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:29 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:29.061+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.646363+0000 front 2024-04-21T08:52:51.646409+0000 (oldest deadline 2024-04-21T08:53:13.946519+0000) 2024-04-21T08:53:29.312 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:29 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:29.061+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.948431+0000 front 2024-04-21T08:53:00.948313+0000 (oldest deadline 2024-04-21T08:53:24.448312+0000) 2024-04-21T08:53:29.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:29 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:29.254+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:29.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:29 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:29.254+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.614735+0000 front 2024-04-21T08:52:59.614778+0000 (oldest deadline 2024-04-21T08:53:25.514902+0000) 2024-04-21T08:53:29.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:29 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:29.350+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:29.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:29 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:29.350+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.963839+0000 front 2024-04-21T08:53:00.963989+0000 (oldest deadline 2024-04-21T08:53:25.063990+0000) 2024-04-21T08:53:29.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:29 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:29.428+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:29.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:29 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:29.428+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:29.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:29 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:29.428+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:30.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:29 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:29.715+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:30.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:29 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:29.715+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:30.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:29 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:29.715+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:30.561 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:30 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:30.091+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.646363+0000 front 2024-04-21T08:52:51.646409+0000 (oldest deadline 2024-04-21T08:53:13.946519+0000) 2024-04-21T08:53:30.562 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:30 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:30.091+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.948431+0000 front 2024-04-21T08:53:00.948313+0000 (oldest deadline 2024-04-21T08:53:24.448312+0000) 2024-04-21T08:53:30.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:30 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:30.227+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:30.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:30 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:30.227+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:30.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:30 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:30.227+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:30.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:30 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:30.231+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:30.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:30 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:30.232+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.614735+0000 front 2024-04-21T08:52:59.614778+0000 (oldest deadline 2024-04-21T08:53:25.514902+0000) 2024-04-21T08:53:30.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:30 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:30.389+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:30.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:30 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:30.389+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:30.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:30 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:30.389+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:30.795 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:30 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:30.386+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:30.795 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:30 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:30.386+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.963839+0000 front 2024-04-21T08:53:00.963989+0000 (oldest deadline 2024-04-21T08:53:25.063990+0000) 2024-04-21T08:53:31.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:30 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:30.743+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:31.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:30 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:30.743+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:31.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:30 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:30.743+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:31.562 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:31 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:31.131+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.646363+0000 front 2024-04-21T08:52:51.646409+0000 (oldest deadline 2024-04-21T08:53:13.946519+0000) 2024-04-21T08:53:31.562 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:31 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:31.131+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.948431+0000 front 2024-04-21T08:53:00.948313+0000 (oldest deadline 2024-04-21T08:53:24.448312+0000) 2024-04-21T08:53:31.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:31 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:31.260+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:31.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:31 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:31.260+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:31.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:31 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:31.260+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:31.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:31 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:31.282+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:31.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:31 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:31.282+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.614735+0000 front 2024-04-21T08:52:59.614778+0000 (oldest deadline 2024-04-21T08:53:25.514902+0000) 2024-04-21T08:53:31.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:31 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:31.359+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:31.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:31 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:31.359+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.963839+0000 front 2024-04-21T08:53:00.963989+0000 (oldest deadline 2024-04-21T08:53:25.063990+0000) 2024-04-21T08:53:31.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:31 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:31.343+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:31.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:31 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:31.343+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:31.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:31 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:31.343+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:32.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:31 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:31.770+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:32.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:31 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:31.770+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:32.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:31 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:31.770+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:32.561 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:32 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:32.086+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.646363+0000 front 2024-04-21T08:52:51.646409+0000 (oldest deadline 2024-04-21T08:53:13.946519+0000) 2024-04-21T08:53:32.562 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:32 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:32.086+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.948431+0000 front 2024-04-21T08:53:00.948313+0000 (oldest deadline 2024-04-21T08:53:24.448312+0000) 2024-04-21T08:53:32.562 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:32 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:32.086+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:07.949213+0000 front 2024-04-21T08:53:07.949264+0000 (oldest deadline 2024-04-21T08:53:32.049417+0000) 2024-04-21T08:53:32.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:32 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:32.217+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:32.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:32 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:32.217+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:32.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:32 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:32.217+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:32.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:32 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:32.250+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:32.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:32 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:32.250+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.614735+0000 front 2024-04-21T08:52:59.614778+0000 (oldest deadline 2024-04-21T08:53:25.514902+0000) 2024-04-21T08:53:32.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:32 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:32.342+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:32.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:32 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:32.342+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.963839+0000 front 2024-04-21T08:53:00.963989+0000 (oldest deadline 2024-04-21T08:53:25.063990+0000) 2024-04-21T08:53:32.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:32 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:32.304+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:32.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:32 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:32.304+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:32.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:32 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:32.304+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:33.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:32 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:32.792+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:33.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:32 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:32.792+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:33.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:32 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:32.792+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:33.562 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:33 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:33.129+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.646363+0000 front 2024-04-21T08:52:51.646409+0000 (oldest deadline 2024-04-21T08:53:13.946519+0000) 2024-04-21T08:53:33.562 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:33 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:33.129+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.948431+0000 front 2024-04-21T08:53:00.948313+0000 (oldest deadline 2024-04-21T08:53:24.448312+0000) 2024-04-21T08:53:33.562 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:33 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6[53720]: 2024-04-21T08:53:33.129+0000 7efcad293700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:07.949213+0000 front 2024-04-21T08:53:07.949264+0000 (oldest deadline 2024-04-21T08:53:32.049417+0000) 2024-04-21T08:53:33.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:33 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:33.261+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:33.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:33 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:33.261+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:33.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:33 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:33.261+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:33.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:33 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:33.261+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:33.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:33 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:33.296+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:33.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:33 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:33.296+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.614735+0000 front 2024-04-21T08:52:59.614778+0000 (oldest deadline 2024-04-21T08:53:25.514902+0000) 2024-04-21T08:53:33.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:33 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:33.296+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:07.215737+0000 front 2024-04-21T08:53:07.215725+0000 (oldest deadline 2024-04-21T08:53:32.515815+0000) 2024-04-21T08:53:33.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:33 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:33.362+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:33.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:33 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:33.362+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.963839+0000 front 2024-04-21T08:53:00.963989+0000 (oldest deadline 2024-04-21T08:53:25.063990+0000) 2024-04-21T08:53:33.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:33 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:33.350+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:33.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:33 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:33.350+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:33.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:33 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:33.350+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:33.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:33 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:33.350+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:34.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:33 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:33.807+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:34.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:33 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:33.807+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:34.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:33 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:33.807+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:34.442 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:34 smithi043 podman[108808]: 2024-04-21 08:53:34.132866655 +0000 UTC m=+5.578350090 container died 6d2108af23d5112b4839ab816e4ab2a3a2ade70d16a58ee7fbbed1ddc113a235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:53:34.442 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:34 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:34.234+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:34.442 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:34 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:34.234+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:34.442 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:34 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:34.234+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:34.442 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:34 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:34.234+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:34.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:34 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:34.298+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:34.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:34 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:34.298+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.614735+0000 front 2024-04-21T08:52:59.614778+0000 (oldest deadline 2024-04-21T08:53:25.514902+0000) 2024-04-21T08:53:34.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:34 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:34.298+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:07.215737+0000 front 2024-04-21T08:53:07.215725+0000 (oldest deadline 2024-04-21T08:53:32.515815+0000) 2024-04-21T08:53:34.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:34 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:34.319+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:34.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:34 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:34.319+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.963839+0000 front 2024-04-21T08:53:00.963989+0000 (oldest deadline 2024-04-21T08:53:25.063990+0000) 2024-04-21T08:53:34.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:34 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:34.355+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:34.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:34 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:34.355+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:34.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:34 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:34.355+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:34.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:34 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:34.355+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:34.801 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:34 smithi043 podman[108808]: 2024-04-21 08:53:34.559004096 +0000 UTC m=+6.004487528 container cleanup 6d2108af23d5112b4839ab816e4ab2a3a2ade70d16a58ee7fbbed1ddc113a235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:53:34.801 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:34 smithi043 bash[108808]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6 2024-04-21T08:53:35.061 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:34 smithi043 podman[108827]: 2024-04-21 08:53:34.802461696 +0000 UTC m=+0.664678649 container remove 6d2108af23d5112b4839ab816e4ab2a3a2ade70d16a58ee7fbbed1ddc113a235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-21T08:53:35.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:34 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:34.810+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:35.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:34 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:34.810+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:35.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:34 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:34.810+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:35.501 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:35 smithi043 podman[108915]: 2024-04-21 08:53:35.146641759 +0000 UTC m=+0.018411407 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:53:35.501 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:35 smithi043 podman[108915]: 2024-04-21 08:53:35.261508479 +0000 UTC m=+0.133278101 container create 96452c2a30829863e0c92402f85db74dc795eb2e6294c2a17e670e3cc1fd7b54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-21T08:53:35.501 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:35 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:35.229+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:35.501 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:35 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:35.229+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:35.501 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:35 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:35.229+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:35.502 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:35 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:35.229+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:35.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:35 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:35.265+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:35.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:35 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:35.265+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.614735+0000 front 2024-04-21T08:52:59.614778+0000 (oldest deadline 2024-04-21T08:53:25.514902+0000) 2024-04-21T08:53:35.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:35 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:35.265+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:07.215737+0000 front 2024-04-21T08:53:07.215725+0000 (oldest deadline 2024-04-21T08:53:32.515815+0000) 2024-04-21T08:53:35.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:35 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:35.363+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:35.795 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:35 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:35.363+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.963839+0000 front 2024-04-21T08:53:00.963989+0000 (oldest deadline 2024-04-21T08:53:25.063990+0000) 2024-04-21T08:53:35.795 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:35 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:35.363+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.765025+0000 front 2024-04-21T08:53:06.765004+0000 (oldest deadline 2024-04-21T08:53:34.365517+0000) 2024-04-21T08:53:35.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:35 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:35.365+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:35.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:35 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:35.365+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:35.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:35 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:35.365+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:35.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:35 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:35.365+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:35.812 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:35 smithi043 podman[108915]: 2024-04-21 08:53:35.597484209 +0000 UTC m=+0.469253830 container init 96452c2a30829863e0c92402f85db74dc795eb2e6294c2a17e670e3cc1fd7b54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:53:35.812 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:35 smithi043 podman[108915]: 2024-04-21 08:53:35.600739902 +0000 UTC m=+0.472509537 container start 96452c2a30829863e0c92402f85db74dc795eb2e6294c2a17e670e3cc1fd7b54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:53:35.812 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:35 smithi043 podman[108915]: 2024-04-21 08:53:35.697406162 +0000 UTC m=+0.569175801 container attach 96452c2a30829863e0c92402f85db74dc795eb2e6294c2a17e670e3cc1fd7b54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:53:36.210 INFO:journalctl@ceph.osd.6.smithi043.stdout:Apr 21 08:53:35 smithi043 podman[108915]: 2024-04-21 08:53:35.901222616 +0000 UTC m=+0.772992250 container died 96452c2a30829863e0c92402f85db74dc795eb2e6294c2a17e670e3cc1fd7b54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:53:36.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:35 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:35.784+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:36.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:35 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:35.784+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:36.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:35 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:35.784+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:36.543 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:36 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:36.210+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:36.543 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:36 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:36.210+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:36.543 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:36 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:36.210+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:36.543 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:36 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:36.210+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:36.567 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.6.service' 2024-04-21T08:53:36.613 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:53:36.613 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-21T08:53:36.613 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-21T08:53:36.613 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.7 2024-04-21T08:53:36.642 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:36 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:36.405+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:36.643 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:36 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:36.405+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.963839+0000 front 2024-04-21T08:53:00.963989+0000 (oldest deadline 2024-04-21T08:53:25.063990+0000) 2024-04-21T08:53:36.643 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:36 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:36.405+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.765025+0000 front 2024-04-21T08:53:06.765004+0000 (oldest deadline 2024-04-21T08:53:34.365517+0000) 2024-04-21T08:53:36.643 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:36 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:36.392+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:36.643 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:36 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:36.392+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:36.643 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:36 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:36.392+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:36.643 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:36 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:36.392+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:36.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:36 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:36.309+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:36.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:36 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:36.309+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.614735+0000 front 2024-04-21T08:52:59.614778+0000 (oldest deadline 2024-04-21T08:53:25.514902+0000) 2024-04-21T08:53:36.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:36 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:36.309+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:07.215737+0000 front 2024-04-21T08:53:07.215725+0000 (oldest deadline 2024-04-21T08:53:32.515815+0000) 2024-04-21T08:53:36.911 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:36 smithi112 systemd[1]: Stopping Ceph osd.7 for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:53:37.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:36 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:36.802+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:37.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:36 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:36.802+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:37.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:36 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:36.802+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:37.294 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:36 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:36.911+0000 7f4fe1c5b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:53:37.294 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:36 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:36.911+0000 7f4fe1c5b700 -1 osd.7 114 *** Got signal Terminated *** 2024-04-21T08:53:37.294 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:36 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:36.911+0000 7f4fe1c5b700 -1 osd.7 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:53:37.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:37 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:37.252+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:37.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:37 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:37.252+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:37.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:37 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:37.252+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:37.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:37 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:37.252+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:37.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:37 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:37.270+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:37.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:37 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:37.270+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.614735+0000 front 2024-04-21T08:52:59.614778+0000 (oldest deadline 2024-04-21T08:53:25.514902+0000) 2024-04-21T08:53:37.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:37 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:37.270+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:07.215737+0000 front 2024-04-21T08:53:07.215725+0000 (oldest deadline 2024-04-21T08:53:32.515815+0000) 2024-04-21T08:53:37.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:37 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:37.426+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:37.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:37 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:37.426+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.963839+0000 front 2024-04-21T08:53:00.963989+0000 (oldest deadline 2024-04-21T08:53:25.063990+0000) 2024-04-21T08:53:37.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:37 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:37.426+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.765025+0000 front 2024-04-21T08:53:06.765004+0000 (oldest deadline 2024-04-21T08:53:34.365517+0000) 2024-04-21T08:53:37.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:37 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:37.411+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:37.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:37 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:37.411+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:37.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:37 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:37.411+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:37.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:37 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:37.411+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:38.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:37 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:37.792+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:38.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:37 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:37.792+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:38.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:37 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:37.792+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:38.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:38 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:38.299+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:38.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:38 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:38.299+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:38.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:38 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:38.299+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:38.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:38 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:38.299+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:38.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:38 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:38.299+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.364283+0000 front 2024-04-21T08:53:16.364308+0000 (oldest deadline 2024-04-21T08:53:38.064000+0000) 2024-04-21T08:53:38.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:38 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:38.274+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:38.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:38 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:38.274+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.614735+0000 front 2024-04-21T08:52:59.614778+0000 (oldest deadline 2024-04-21T08:53:25.514902+0000) 2024-04-21T08:53:38.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:38 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:38.274+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:07.215737+0000 front 2024-04-21T08:53:07.215725+0000 (oldest deadline 2024-04-21T08:53:32.515815+0000) 2024-04-21T08:53:38.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:38 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:38.456+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:38.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:38 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:38.456+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.963839+0000 front 2024-04-21T08:53:00.963989+0000 (oldest deadline 2024-04-21T08:53:25.063990+0000) 2024-04-21T08:53:38.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:38 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:38.456+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.765025+0000 front 2024-04-21T08:53:06.765004+0000 (oldest deadline 2024-04-21T08:53:34.365517+0000) 2024-04-21T08:53:38.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:38 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:38.394+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:38.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:38 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:38.394+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:38.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:38 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:38.394+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:38.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:38 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:38.394+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:39.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:38 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:38.793+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:39.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:38 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:38.793+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:39.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:38 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:38.793+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:39.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:39 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:39.314+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:39.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:39 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:39.314+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.614735+0000 front 2024-04-21T08:52:59.614778+0000 (oldest deadline 2024-04-21T08:53:25.514902+0000) 2024-04-21T08:53:39.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:39 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:39.314+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:07.215737+0000 front 2024-04-21T08:53:07.215725+0000 (oldest deadline 2024-04-21T08:53:32.515815+0000) 2024-04-21T08:53:39.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:39 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:39.416+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:39.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:39 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:39.416+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.963839+0000 front 2024-04-21T08:53:00.963989+0000 (oldest deadline 2024-04-21T08:53:25.063990+0000) 2024-04-21T08:53:39.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:39 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:39.416+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.765025+0000 front 2024-04-21T08:53:06.765004+0000 (oldest deadline 2024-04-21T08:53:34.365517+0000) 2024-04-21T08:53:39.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:39 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:39.410+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:39.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:39 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:39.410+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:39.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:39 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:39.410+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:39.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:39 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:39.410+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:39.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:39 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:39.410+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:53:39.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:39 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:39.320+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:39.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:39 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:39.320+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:39.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:39 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:39.320+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:39.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:39 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:39.320+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:39.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:39 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:39.320+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.364283+0000 front 2024-04-21T08:53:16.364308+0000 (oldest deadline 2024-04-21T08:53:38.064000+0000) 2024-04-21T08:53:40.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:39 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:39.780+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:40.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:39 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:39.780+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:40.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:39 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:39.780+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:40.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:40 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:40.307+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:40.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:40 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:40.307+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.614735+0000 front 2024-04-21T08:52:59.614778+0000 (oldest deadline 2024-04-21T08:53:25.514902+0000) 2024-04-21T08:53:40.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:40 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:40.307+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:07.215737+0000 front 2024-04-21T08:53:07.215725+0000 (oldest deadline 2024-04-21T08:53:32.515815+0000) 2024-04-21T08:53:40.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:40 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:40.435+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:40.795 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:40 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:40.435+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.963839+0000 front 2024-04-21T08:53:00.963989+0000 (oldest deadline 2024-04-21T08:53:25.063990+0000) 2024-04-21T08:53:40.795 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:40 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:40.435+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.765025+0000 front 2024-04-21T08:53:06.765004+0000 (oldest deadline 2024-04-21T08:53:34.365517+0000) 2024-04-21T08:53:40.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:40 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:40.429+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:40.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:40 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:40.430+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:40.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:40 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:40.430+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:40.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:40 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:40.430+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:40.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:40 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:40.430+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:53:40.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:40 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:40.348+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:40.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:40 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:40.348+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:40.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:40 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:40.348+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:40.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:40 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:40.348+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:40.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:40 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:40.348+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.364283+0000 front 2024-04-21T08:53:16.364308+0000 (oldest deadline 2024-04-21T08:53:38.064000+0000) 2024-04-21T08:53:41.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:40 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:40.738+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:41.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:40 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:40.738+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:41.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:40 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:40.738+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:41.689 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:41 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:41.275+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:41.689 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:41 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:41.275+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.614735+0000 front 2024-04-21T08:52:59.614778+0000 (oldest deadline 2024-04-21T08:53:25.514902+0000) 2024-04-21T08:53:41.689 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:41 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:41.275+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:07.215737+0000 front 2024-04-21T08:53:07.215725+0000 (oldest deadline 2024-04-21T08:53:32.515815+0000) 2024-04-21T08:53:41.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:41 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:41.482+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.062127+0000 front 2024-04-21T08:52:51.062192+0000 (oldest deadline 2024-04-21T08:53:15.162284+0000) 2024-04-21T08:53:41.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:41 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:41.482+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.963839+0000 front 2024-04-21T08:53:00.963989+0000 (oldest deadline 2024-04-21T08:53:25.063990+0000) 2024-04-21T08:53:41.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:41 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7[49316]: 2024-04-21T08:53:41.482+0000 7f4fde24f700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.765025+0000 front 2024-04-21T08:53:06.765004+0000 (oldest deadline 2024-04-21T08:53:34.365517+0000) 2024-04-21T08:53:41.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:41 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:41.394+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:41.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:41 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:41.394+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:41.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:41 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:41.394+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:41.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:41 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:41.394+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:41.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:41 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:41.394+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:53:41.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:41 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:41.335+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:41.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:41 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:41.335+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:41.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:41 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:41.335+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:41.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:41 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:41.335+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:41.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:41 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:41.335+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.364283+0000 front 2024-04-21T08:53:16.364308+0000 (oldest deadline 2024-04-21T08:53:38.064000+0000) 2024-04-21T08:53:41.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:41 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:41.690+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:41.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:41 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:41.690+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:41.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:41 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:41.690+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:42.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:42 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:42.286+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:42.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:42 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:42.286+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.614735+0000 front 2024-04-21T08:52:59.614778+0000 (oldest deadline 2024-04-21T08:53:25.514902+0000) 2024-04-21T08:53:42.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:42 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:42.286+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:07.215737+0000 front 2024-04-21T08:53:07.215725+0000 (oldest deadline 2024-04-21T08:53:32.515815+0000) 2024-04-21T08:53:42.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:42 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:42.286+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:15.417411+0000 front 2024-04-21T08:53:15.417498+0000 (oldest deadline 2024-04-21T08:53:41.317055+0000) 2024-04-21T08:53:42.745 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:42 smithi112 podman[68109]: 2024-04-21 08:53:42.429832965 +0000 UTC m=+5.617857067 container died fe54368f5892346a0d4f0d62288087f860ab09255ae918ad95363309b8fead41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-21T08:53:42.746 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:42 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:42.392+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:42.746 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:42 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:42.392+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:42.746 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:42 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:42.392+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:42.746 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:42 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:42.392+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:42.746 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:42 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:42.392+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:53:42.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:42 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:42.320+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:42.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:42 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:42.320+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:42.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:42 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:42.320+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:42.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:42 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:42.320+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:42.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:42 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:42.320+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.364283+0000 front 2024-04-21T08:53:16.364308+0000 (oldest deadline 2024-04-21T08:53:38.064000+0000) 2024-04-21T08:53:43.045 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:42 smithi112 podman[68109]: 2024-04-21 08:53:42.828846457 +0000 UTC m=+6.016870570 container cleanup fe54368f5892346a0d4f0d62288087f860ab09255ae918ad95363309b8fead41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:53:43.046 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:42 smithi112 bash[68109]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7 2024-04-21T08:53:43.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:42 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:42.739+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:43.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:42 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:42.739+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:43.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:42 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:42.739+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:43.369 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:43 smithi112 podman[68127]: 2024-04-21 08:53:43.07171051 +0000 UTC m=+0.635819833 container remove fe54368f5892346a0d4f0d62288087f860ab09255ae918ad95363309b8fead41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:53:43.624 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:43 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:43.369+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:43.624 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:43 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:43.369+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:43.624 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:43 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:43.369+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:43.624 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:43 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:43.369+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:43.624 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:43 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:43.369+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:53:43.624 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:43 smithi112 podman[68220]: 2024-04-21 08:53:43.292688161 +0000 UTC m=+0.017863275 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:53:43.625 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:43 smithi112 podman[68220]: 2024-04-21 08:53:43.406102264 +0000 UTC m=+0.131277380 container create d182ddd191f6719eaabf4f27ca444dec8464ad48bff64abda6cecef4a06fa990 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-21T08:53:43.625 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:43 smithi112 podman[68220]: 2024-04-21 08:53:43.625023356 +0000 UTC m=+0.350198464 container init d182ddd191f6719eaabf4f27ca444dec8464ad48bff64abda6cecef4a06fa990 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:53:43.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:43 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:43.313+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:43.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:43 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:43.313+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.614735+0000 front 2024-04-21T08:52:59.614778+0000 (oldest deadline 2024-04-21T08:53:25.514902+0000) 2024-04-21T08:53:43.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:43 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:43.313+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:07.215737+0000 front 2024-04-21T08:53:07.215725+0000 (oldest deadline 2024-04-21T08:53:32.515815+0000) 2024-04-21T08:53:43.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:43 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:43.313+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:15.417411+0000 front 2024-04-21T08:53:15.417498+0000 (oldest deadline 2024-04-21T08:53:41.317055+0000) 2024-04-21T08:53:43.811 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:43 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:43.366+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:43.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:43 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:43.366+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:43.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:43 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:43.366+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:43.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:43 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:43.366+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:43.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:43 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:43.366+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.364283+0000 front 2024-04-21T08:53:16.364308+0000 (oldest deadline 2024-04-21T08:53:38.064000+0000) 2024-04-21T08:53:43.922 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:43 smithi112 podman[68220]: 2024-04-21 08:53:43.628163181 +0000 UTC m=+0.353338296 container start d182ddd191f6719eaabf4f27ca444dec8464ad48bff64abda6cecef4a06fa990 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-21T08:53:43.922 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:43 smithi112 podman[68220]: 2024-04-21 08:53:43.708417916 +0000 UTC m=+0.433593042 container attach d182ddd191f6719eaabf4f27ca444dec8464ad48bff64abda6cecef4a06fa990 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-21T08:53:44.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:43 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:43.786+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:44.215 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:43 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:43.786+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:44.215 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:43 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:43.786+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:44.215 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:43 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:43.786+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:53:44.219 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 21 08:53:43 smithi112 podman[68220]: 2024-04-21 08:53:43.923282274 +0000 UTC m=+0.648457394 container died d182ddd191f6719eaabf4f27ca444dec8464ad48bff64abda6cecef4a06fa990 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:53:44.512 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:44 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:44.328+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:44.512 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:44 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:44.328+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:44.512 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:44 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:44.328+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:44.512 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:44 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:44.328+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:44.513 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:44 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:44.328+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:53:44.541 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.7.service' 2024-04-21T08:53:44.620 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:53:44.620 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-21T08:53:44.620 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-21T08:53:44.620 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.8 2024-04-21T08:53:44.647 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:44 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:44.339+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:44.647 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:44 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:44.339+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.614735+0000 front 2024-04-21T08:52:59.614778+0000 (oldest deadline 2024-04-21T08:53:25.514902+0000) 2024-04-21T08:53:44.647 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:44 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:44.339+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:07.215737+0000 front 2024-04-21T08:53:07.215725+0000 (oldest deadline 2024-04-21T08:53:32.515815+0000) 2024-04-21T08:53:44.647 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:44 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:44.339+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:15.417411+0000 front 2024-04-21T08:53:15.417498+0000 (oldest deadline 2024-04-21T08:53:41.317055+0000) 2024-04-21T08:53:44.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:44 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:44.384+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:44.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:44 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:44.384+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:44.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:44 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:44.384+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:44.813 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:44 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:44.384+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:44.813 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:44 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:44.385+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.364283+0000 front 2024-04-21T08:53:16.364308+0000 (oldest deadline 2024-04-21T08:53:38.064000+0000) 2024-04-21T08:53:44.935 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:44 smithi175 systemd[1]: Stopping Ceph osd.8 for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:53:44.936 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:44 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:44.935+0000 7f7bcfbbf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:53:44.936 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:44 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:44.787+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:44.936 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:44 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:44.787+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:44.936 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:44 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:44.787+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:44.936 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:44 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:44.787+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:53:45.214 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:44 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:44.935+0000 7f7bcfbbf700 -1 osd.8 114 *** Got signal Terminated *** 2024-04-21T08:53:45.214 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:44 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:44.935+0000 7f7bcfbbf700 -1 osd.8 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:53:45.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:45 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:45.357+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:45.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:45 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:45.357+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.614735+0000 front 2024-04-21T08:52:59.614778+0000 (oldest deadline 2024-04-21T08:53:25.514902+0000) 2024-04-21T08:53:45.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:45 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:45.357+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:07.215737+0000 front 2024-04-21T08:53:07.215725+0000 (oldest deadline 2024-04-21T08:53:32.515815+0000) 2024-04-21T08:53:45.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:45 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:45.357+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:15.417411+0000 front 2024-04-21T08:53:15.417498+0000 (oldest deadline 2024-04-21T08:53:41.317055+0000) 2024-04-21T08:53:45.724 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:45 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:45.341+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:45.724 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:45 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:45.341+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:45.724 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:45 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:45.341+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:45.724 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:45 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:45.341+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:45.724 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:45 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:45.341+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:53:45.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:45 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:45.416+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:45.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:45 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:45.416+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:45.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:45 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:45.416+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:45.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:45 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:45.416+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:45.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:45 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:45.416+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.364283+0000 front 2024-04-21T08:53:16.364308+0000 (oldest deadline 2024-04-21T08:53:38.064000+0000) 2024-04-21T08:53:46.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:45 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:45.776+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:46.215 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:45 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:45.776+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:46.215 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:45 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:45.776+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:46.215 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:45 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:45.776+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:53:46.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:46 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:46.313+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:46.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:46 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:46.313+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.614735+0000 front 2024-04-21T08:52:59.614778+0000 (oldest deadline 2024-04-21T08:53:25.514902+0000) 2024-04-21T08:53:46.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:46 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:46.313+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:07.215737+0000 front 2024-04-21T08:53:07.215725+0000 (oldest deadline 2024-04-21T08:53:32.515815+0000) 2024-04-21T08:53:46.714 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:46 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:46.313+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:15.417411+0000 front 2024-04-21T08:53:15.417498+0000 (oldest deadline 2024-04-21T08:53:41.317055+0000) 2024-04-21T08:53:46.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:46 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:46.347+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:46.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:46 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:46.347+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:46.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:46 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:46.347+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:46.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:46 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:46.347+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:46.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:46 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:46.347+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:53:46.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:46 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:46.347+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.269936+0000 front 2024-04-21T08:53:24.270028+0000 (oldest deadline 2024-04-21T08:53:45.969918+0000) 2024-04-21T08:53:46.811 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:46 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:46.385+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:46.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:46 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:46.385+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:46.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:46 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:46.385+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:46.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:46 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:46.385+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:46.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:46 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:46.385+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.364283+0000 front 2024-04-21T08:53:16.364308+0000 (oldest deadline 2024-04-21T08:53:38.064000+0000) 2024-04-21T08:53:47.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:46 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:46.750+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:47.214 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:46 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:46.750+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:47.215 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:46 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:46.750+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:47.215 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:46 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:46.750+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:53:47.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:47 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:47.349+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:47.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:47 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:47.349+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.614735+0000 front 2024-04-21T08:52:59.614778+0000 (oldest deadline 2024-04-21T08:53:25.514902+0000) 2024-04-21T08:53:47.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:47 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:47.349+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:07.215737+0000 front 2024-04-21T08:53:07.215725+0000 (oldest deadline 2024-04-21T08:53:32.515815+0000) 2024-04-21T08:53:47.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:47 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:47.349+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:15.417411+0000 front 2024-04-21T08:53:15.417498+0000 (oldest deadline 2024-04-21T08:53:41.317055+0000) 2024-04-21T08:53:47.710 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:47 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:47.349+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:22.417954+0000 front 2024-04-21T08:53:22.417925+0000 (oldest deadline 2024-04-21T08:53:46.518054+0000) 2024-04-21T08:53:47.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:47 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:47.347+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:47.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:47 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:47.347+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:47.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:47 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:47.347+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:47.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:47 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:47.347+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:47.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:47 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:47.347+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:53:47.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:47 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:47.347+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.269936+0000 front 2024-04-21T08:53:24.270028+0000 (oldest deadline 2024-04-21T08:53:45.969918+0000) 2024-04-21T08:53:47.811 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:47 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:47.406+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:47.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:47 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:47.406+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:47.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:47 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:47.406+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:47.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:47 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:47.406+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:47.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:47 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:47.406+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.364283+0000 front 2024-04-21T08:53:16.364308+0000 (oldest deadline 2024-04-21T08:53:38.064000+0000) 2024-04-21T08:53:47.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:47 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:47.709+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:47.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:47 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:47.709+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:47.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:47 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:47.709+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:47.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:47 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:47.709+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:53:47.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:47 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:47.709+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:53:48.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:48 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:48.393+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:48.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:48 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:48.393+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:48.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:48 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:48.393+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:48.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:48 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:48.393+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:48.562 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:48 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:48.393+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.364283+0000 front 2024-04-21T08:53:16.364308+0000 (oldest deadline 2024-04-21T08:53:38.064000+0000) 2024-04-21T08:53:48.700 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:48 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:48.390+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:48.700 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:48 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:48.390+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.614735+0000 front 2024-04-21T08:52:59.614778+0000 (oldest deadline 2024-04-21T08:53:25.514902+0000) 2024-04-21T08:53:48.700 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:48 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:48.390+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:07.215737+0000 front 2024-04-21T08:53:07.215725+0000 (oldest deadline 2024-04-21T08:53:32.515815+0000) 2024-04-21T08:53:48.700 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:48 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:48.390+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:15.417411+0000 front 2024-04-21T08:53:15.417498+0000 (oldest deadline 2024-04-21T08:53:41.317055+0000) 2024-04-21T08:53:48.700 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:48 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:48.390+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:22.417954+0000 front 2024-04-21T08:53:22.417925+0000 (oldest deadline 2024-04-21T08:53:46.518054+0000) 2024-04-21T08:53:48.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:48 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:48.318+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:48.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:48 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:48.318+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:48.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:48 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:48.318+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:48.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:48 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:48.318+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:48.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:48 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:48.318+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:53:48.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:48 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:48.318+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.269936+0000 front 2024-04-21T08:53:24.270028+0000 (oldest deadline 2024-04-21T08:53:45.969918+0000) 2024-04-21T08:53:48.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:48 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:48.701+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:48.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:48 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:48.701+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:48.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:48 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:48.701+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:48.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:48 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:48.701+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:53:48.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:48 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:48.701+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:53:49.544 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:49 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:49.284+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:49.545 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:49 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:49.284+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:49.545 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:49 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:49.284+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:49.545 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:49 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:49.284+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:49.545 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:49 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:49.284+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:53:49.545 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:49 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:49.284+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.269936+0000 front 2024-04-21T08:53:24.270028+0000 (oldest deadline 2024-04-21T08:53:45.969918+0000) 2024-04-21T08:53:49.696 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:49 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:49.394+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.013532+0000 front 2024-04-21T08:52:52.013556+0000 (oldest deadline 2024-04-21T08:53:15.513699+0000) 2024-04-21T08:53:49.697 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:49 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:49.394+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.614735+0000 front 2024-04-21T08:52:59.614778+0000 (oldest deadline 2024-04-21T08:53:25.514902+0000) 2024-04-21T08:53:49.697 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:49 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:49.394+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:07.215737+0000 front 2024-04-21T08:53:07.215725+0000 (oldest deadline 2024-04-21T08:53:32.515815+0000) 2024-04-21T08:53:49.697 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:49 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:49.394+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:15.417411+0000 front 2024-04-21T08:53:15.417498+0000 (oldest deadline 2024-04-21T08:53:41.317055+0000) 2024-04-21T08:53:49.697 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:49 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8[50564]: 2024-04-21T08:53:49.394+0000 7f7bcc1b3700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:22.417954+0000 front 2024-04-21T08:53:22.417925+0000 (oldest deadline 2024-04-21T08:53:46.518054+0000) 2024-04-21T08:53:49.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:49 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:49.345+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:49.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:49 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:49.345+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:49.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:49 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:49.345+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:49.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:49 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:49.345+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:49.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:49 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:49.345+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.364283+0000 front 2024-04-21T08:53:16.364308+0000 (oldest deadline 2024-04-21T08:53:38.064000+0000) 2024-04-21T08:53:49.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:49 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:49.345+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.365817+0000 front 2024-04-21T08:53:24.365916+0000 (oldest deadline 2024-04-21T08:53:49.065713+0000) 2024-04-21T08:53:49.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:49 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:49.697+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:49.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:49 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:49.697+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:49.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:49 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:49.697+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:49.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:49 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:49.697+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:53:49.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:49 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:49.697+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:53:50.578 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:50 smithi175 podman[69826]: 2024-04-21 08:53:50.312496023 +0000 UTC m=+5.495770462 container died 72864771c99440e1d9354fc8590bf6cf8a1a0a382e67b153a4a313f969f9c988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-21T08:53:50.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:50 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:50.317+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:50.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:50 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:50.317+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:50.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:50 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:50.317+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:50.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:50 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:50.317+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:50.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:50 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:50.317+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:53:50.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:50 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:50.317+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.269936+0000 front 2024-04-21T08:53:24.270028+0000 (oldest deadline 2024-04-21T08:53:45.969918+0000) 2024-04-21T08:53:50.814 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:50 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:50.394+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:50.814 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:50 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:50.394+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:50.814 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:50 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:50.394+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:50.814 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:50 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:50.394+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:50.814 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:50 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:50.394+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.364283+0000 front 2024-04-21T08:53:16.364308+0000 (oldest deadline 2024-04-21T08:53:38.064000+0000) 2024-04-21T08:53:50.814 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:50 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:50.394+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.365817+0000 front 2024-04-21T08:53:24.365916+0000 (oldest deadline 2024-04-21T08:53:49.065713+0000) 2024-04-21T08:53:50.964 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:50 smithi175 podman[69826]: 2024-04-21 08:53:50.677989943 +0000 UTC m=+5.861264380 container cleanup 72864771c99440e1d9354fc8590bf6cf8a1a0a382e67b153a4a313f969f9c988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:53:50.964 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:50 smithi175 bash[69826]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8 2024-04-21T08:53:50.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:50 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:50.674+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:50.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:50 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:50.674+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:50.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:50 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:50.674+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:50.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:50 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:50.674+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:53:50.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:50 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:50.674+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:53:51.348 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:50 smithi175 podman[69847]: 2024-04-21 08:53:50.987678193 +0000 UTC m=+0.669720237 container remove 72864771c99440e1d9354fc8590bf6cf8a1a0a382e67b153a4a313f969f9c988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-21T08:53:51.544 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:51 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:51.288+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:51.544 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:51 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:51.288+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:51.544 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:51 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:51.288+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:51.545 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:51 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:51.288+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:51.545 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:51 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:51.288+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:53:51.545 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:51 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:51.288+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.269936+0000 front 2024-04-21T08:53:24.270028+0000 (oldest deadline 2024-04-21T08:53:45.969918+0000) 2024-04-21T08:53:51.633 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:51 smithi175 podman[69936]: 2024-04-21 08:53:51.252174242 +0000 UTC m=+0.017876350 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:53:51.633 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:51 smithi175 podman[69936]: 2024-04-21 08:53:51.397461722 +0000 UTC m=+0.163163813 container create eabb8f59d388b6bb8fcc4fc5626be941da99bb3b3345aa4986e7725cc2c4c42a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:53:51.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:51 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:51.383+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:51.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:51 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:51.383+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:51.813 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:51 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:51.383+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:51.813 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:51 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:51.383+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:51.813 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:51 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:51.383+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.364283+0000 front 2024-04-21T08:53:16.364308+0000 (oldest deadline 2024-04-21T08:53:38.064000+0000) 2024-04-21T08:53:51.813 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:51 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:51.383+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.365817+0000 front 2024-04-21T08:53:24.365916+0000 (oldest deadline 2024-04-21T08:53:49.065713+0000) 2024-04-21T08:53:51.932 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:51 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:51.703+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:51.933 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:51 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:51.703+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:51.933 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:51 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:51.703+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:51.933 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:51 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:51.703+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:53:51.933 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:51 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:51.703+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:53:51.933 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:51 smithi175 podman[69936]: 2024-04-21 08:53:51.633630172 +0000 UTC m=+0.399332269 container init eabb8f59d388b6bb8fcc4fc5626be941da99bb3b3345aa4986e7725cc2c4c42a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-04-21T08:53:51.933 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:51 smithi175 podman[69936]: 2024-04-21 08:53:51.637485596 +0000 UTC m=+0.403187688 container start eabb8f59d388b6bb8fcc4fc5626be941da99bb3b3345aa4986e7725cc2c4c42a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:53:51.933 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:51 smithi175 podman[69936]: 2024-04-21 08:53:51.758351111 +0000 UTC m=+0.524053204 container attach eabb8f59d388b6bb8fcc4fc5626be941da99bb3b3345aa4986e7725cc2c4c42a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-21T08:53:52.194 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:51 smithi175 podman[69936]: 2024-04-21 08:53:51.933542881 +0000 UTC m=+0.699244973 container died eabb8f59d388b6bb8fcc4fc5626be941da99bb3b3345aa4986e7725cc2c4c42a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-21T08:53:52.446 INFO:journalctl@ceph.osd.8.smithi175.stdout:Apr 21 08:53:52 smithi175 podman[69962]: 2024-04-21 08:53:52.445564568 +0000 UTC m=+0.506067381 container remove eabb8f59d388b6bb8fcc4fc5626be941da99bb3b3345aa4986e7725cc2c4c42a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-21T08:53:52.462 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.8.service' 2024-04-21T08:53:52.540 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:53:52.541 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-21T08:53:52.541 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-21T08:53:52.541 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.9 2024-04-21T08:53:52.544 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:52 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:52.284+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:52.544 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:52 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:52.284+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:52.544 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:52 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:52.284+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:52.544 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:52 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:52.284+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:52.544 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:52 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:52.284+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:53:52.544 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:52 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:52.284+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.269936+0000 front 2024-04-21T08:53:24.270028+0000 (oldest deadline 2024-04-21T08:53:45.969918+0000) 2024-04-21T08:53:52.714 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:52 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:52.700+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:52.714 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:52 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:52.700+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:52.714 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:52 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:52.700+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:52.714 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:52 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:52.700+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:53:52.714 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:52 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:52.700+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:53:52.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:52.432+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:52.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:52.432+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:52.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:52.432+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:52.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:52.432+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:52.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:52.432+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.364283+0000 front 2024-04-21T08:53:16.364308+0000 (oldest deadline 2024-04-21T08:53:38.064000+0000) 2024-04-21T08:53:52.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:52.432+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.365817+0000 front 2024-04-21T08:53:24.365916+0000 (oldest deadline 2024-04-21T08:53:49.065713+0000) 2024-04-21T08:53:52.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:52 smithi043 systemd[1]: Stopping Ceph osd.9 for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:53:53.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:52.869+0000 7fe600b40700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:53:53.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:52.869+0000 7fe600b40700 -1 osd.9 114 *** Got signal Terminated *** 2024-04-21T08:53:53.312 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:52 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:52.869+0000 7fe600b40700 -1 osd.9 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:53:53.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:53 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:53.309+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:53.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:53 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:53.309+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:53.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:53 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:53.309+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:53.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:53 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:53.309+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:53.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:53 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:53.309+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:53:53.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:53 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:53.309+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.269936+0000 front 2024-04-21T08:53:24.270028+0000 (oldest deadline 2024-04-21T08:53:45.969918+0000) 2024-04-21T08:53:53.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:53 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:53.459+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:53.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:53 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:53.459+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:53.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:53 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:53.459+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:53.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:53 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:53.459+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:53.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:53 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:53.459+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.364283+0000 front 2024-04-21T08:53:16.364308+0000 (oldest deadline 2024-04-21T08:53:38.064000+0000) 2024-04-21T08:53:53.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:53 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:53.459+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.365817+0000 front 2024-04-21T08:53:24.365916+0000 (oldest deadline 2024-04-21T08:53:49.065713+0000) 2024-04-21T08:53:53.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:53 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:53.665+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:53.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:53 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:53.665+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:53.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:53 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:53.665+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:53.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:53 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:53.665+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:53:53.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:53 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:53.665+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:53:54.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:54 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:54.328+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:54.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:54 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:54.328+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:54.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:54 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:54.328+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:54.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:54 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:54.328+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:54.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:54 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:54.328+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:53:54.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:54 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:54.328+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.269936+0000 front 2024-04-21T08:53:24.270028+0000 (oldest deadline 2024-04-21T08:53:45.969918+0000) 2024-04-21T08:53:54.811 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:54 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:54.445+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:54.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:54 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:54.445+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:54.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:54 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:54.445+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:54.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:54 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:54.445+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:54.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:54 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:54.445+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.364283+0000 front 2024-04-21T08:53:16.364308+0000 (oldest deadline 2024-04-21T08:53:38.064000+0000) 2024-04-21T08:53:54.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:54 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:54.445+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.365817+0000 front 2024-04-21T08:53:24.365916+0000 (oldest deadline 2024-04-21T08:53:49.065713+0000) 2024-04-21T08:53:54.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:54 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:54.649+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:54.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:54 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:54.649+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:54.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:54 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:54.649+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:54.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:54 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:54.649+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:53:54.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:54 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:54.649+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:53:55.544 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:55 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:55.297+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:55.545 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:55 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:55.297+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:55.545 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:55 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:55.297+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:55.545 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:55 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:55.297+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:55.545 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:55 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:55.297+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:53:55.545 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:55 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:55.297+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.269936+0000 front 2024-04-21T08:53:24.270028+0000 (oldest deadline 2024-04-21T08:53:45.969918+0000) 2024-04-21T08:53:55.545 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:55 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:55.297+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.171510+0000 front 2024-04-21T08:53:31.171546+0000 (oldest deadline 2024-04-21T08:53:55.271423+0000) 2024-04-21T08:53:55.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:55 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:55.482+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:55.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:55 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:55.482+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:55.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:55 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:55.482+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:55.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:55 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:55.482+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:55.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:55 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:55.482+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.364283+0000 front 2024-04-21T08:53:16.364308+0000 (oldest deadline 2024-04-21T08:53:38.064000+0000) 2024-04-21T08:53:55.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:55 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:55.482+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.365817+0000 front 2024-04-21T08:53:24.365916+0000 (oldest deadline 2024-04-21T08:53:49.065713+0000) 2024-04-21T08:53:55.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:55 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:55.688+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:55.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:55 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:55.688+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:55.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:55 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:55.688+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:55.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:55 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:55.688+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:53:55.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:55 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:55.688+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:53:55.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:55 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:55.688+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.909620+0000 front 2024-04-21T08:53:31.909513+0000 (oldest deadline 2024-04-21T08:53:54.809372+0000) 2024-04-21T08:53:56.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:56 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:56.311+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:56.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:56 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:56.311+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:56.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:56 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:56.311+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:56.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:56 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:56.311+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:56.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:56 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:56.311+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:53:56.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:56 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:56.311+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.269936+0000 front 2024-04-21T08:53:24.270028+0000 (oldest deadline 2024-04-21T08:53:45.969918+0000) 2024-04-21T08:53:56.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:56 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:56.311+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.171510+0000 front 2024-04-21T08:53:31.171546+0000 (oldest deadline 2024-04-21T08:53:55.271423+0000) 2024-04-21T08:53:56.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:56 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:56.443+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:56.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:56 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:56.443+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:56.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:56 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:56.443+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:56.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:56 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:56.443+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:56.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:56 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:56.443+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.364283+0000 front 2024-04-21T08:53:16.364308+0000 (oldest deadline 2024-04-21T08:53:38.064000+0000) 2024-04-21T08:53:56.813 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:56 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:56.443+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.365817+0000 front 2024-04-21T08:53:24.365916+0000 (oldest deadline 2024-04-21T08:53:49.065713+0000) 2024-04-21T08:53:56.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:56 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:56.640+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:56.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:56 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:56.640+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:56.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:56 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:56.640+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:56.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:56 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:56.640+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:53:56.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:56 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:56.640+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:53:56.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:56 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:56.640+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.909620+0000 front 2024-04-21T08:53:31.909513+0000 (oldest deadline 2024-04-21T08:53:54.809372+0000) 2024-04-21T08:53:57.568 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:57 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:57.295+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:57.568 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:57 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:57.295+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:57.568 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:57 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:57.295+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:57.568 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:57 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:57.295+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:57.568 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:57 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:57.295+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:53:57.568 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:57 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:57.295+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.269936+0000 front 2024-04-21T08:53:24.270028+0000 (oldest deadline 2024-04-21T08:53:45.969918+0000) 2024-04-21T08:53:57.568 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:57 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:57.295+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.171510+0000 front 2024-04-21T08:53:31.171546+0000 (oldest deadline 2024-04-21T08:53:55.271423+0000) 2024-04-21T08:53:57.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:57 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:57.445+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:41.556596+0000 front 2024-04-21T08:52:41.556623+0000 (oldest deadline 2024-04-21T08:53:07.957457+0000) 2024-04-21T08:53:57.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:57 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:57.445+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:51.359257+0000 front 2024-04-21T08:52:51.359212+0000 (oldest deadline 2024-04-21T08:53:16.659368+0000) 2024-04-21T08:53:57.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:57 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:57.445+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:01.261177+0000 front 2024-04-21T08:53:01.261559+0000 (oldest deadline 2024-04-21T08:53:22.361183+0000) 2024-04-21T08:53:57.813 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:57 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:57.445+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:06.962871+0000 front 2024-04-21T08:53:06.962910+0000 (oldest deadline 2024-04-21T08:53:32.863031+0000) 2024-04-21T08:53:57.813 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:57 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:57.445+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.364283+0000 front 2024-04-21T08:53:16.364308+0000 (oldest deadline 2024-04-21T08:53:38.064000+0000) 2024-04-21T08:53:57.813 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:57 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9[60642]: 2024-04-21T08:53:57.445+0000 7fe5f912c700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.365817+0000 front 2024-04-21T08:53:24.365916+0000 (oldest deadline 2024-04-21T08:53:49.065713+0000) 2024-04-21T08:53:57.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:57 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:57.652+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:57.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:57 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:57.652+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:57.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:57 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:57.652+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:57.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:57 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:57.652+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:53:57.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:57 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:57.652+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:53:57.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:57 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:57.652+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.909620+0000 front 2024-04-21T08:53:31.909513+0000 (oldest deadline 2024-04-21T08:53:54.809372+0000) 2024-04-21T08:53:58.494 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:58 smithi043 podman[109032]: 2024-04-21 08:53:58.235246985 +0000 UTC m=+5.493787194 container died 1dbf59ed06ab2fdae07d176cd6c26991684c3a5b9525ae6766d0327f02edd10e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-21T08:53:58.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:58 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:58.294+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:58.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:58 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:58.294+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:58.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:58 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:58.294+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:58.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:58 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:58.294+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:58.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:58 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:58.294+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:53:58.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:58 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:58.294+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.269936+0000 front 2024-04-21T08:53:24.270028+0000 (oldest deadline 2024-04-21T08:53:45.969918+0000) 2024-04-21T08:53:58.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:58 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:58.294+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.171510+0000 front 2024-04-21T08:53:31.171546+0000 (oldest deadline 2024-04-21T08:53:55.271423+0000) 2024-04-21T08:53:58.811 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:58 smithi043 podman[109032]: 2024-04-21 08:53:58.594745124 +0000 UTC m=+5.853285368 container cleanup 1dbf59ed06ab2fdae07d176cd6c26991684c3a5b9525ae6766d0327f02edd10e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-21T08:53:58.812 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:58 smithi043 bash[109032]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9 2024-04-21T08:53:58.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:58 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:58.670+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:58.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:58 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:58.670+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:58.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:58 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:58.670+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:58.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:58 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:58.670+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:53:58.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:58 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:58.670+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:53:58.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:58 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:58.670+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.909620+0000 front 2024-04-21T08:53:31.909513+0000 (oldest deadline 2024-04-21T08:53:54.809372+0000) 2024-04-21T08:53:59.228 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:58 smithi043 podman[109049]: 2024-04-21 08:53:58.887502842 +0000 UTC m=+0.650988553 container remove 1dbf59ed06ab2fdae07d176cd6c26991684c3a5b9525ae6766d0327f02edd10e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:53:59.498 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:59 smithi043 podman[109143]: 2024-04-21 08:53:59.22979893 +0000 UTC m=+0.115101603 container create c8091616198f4e08613dff5d081316f78ef42bae51cd43c8c152ffab3cb58719 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-21T08:53:59.498 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:59 smithi043 podman[109143]: 2024-04-21 08:53:59.133299248 +0000 UTC m=+0.018601934 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:53:59.498 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:59 smithi043 podman[109143]: 2024-04-21 08:53:59.415898439 +0000 UTC m=+0.301201113 container init c8091616198f4e08613dff5d081316f78ef42bae51cd43c8c152ffab3cb58719 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-21T08:53:59.498 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:59 smithi043 podman[109143]: 2024-04-21 08:53:59.419055662 +0000 UTC m=+0.304358337 container start c8091616198f4e08613dff5d081316f78ef42bae51cd43c8c152ffab3cb58719 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:53:59.499 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:59 smithi043 podman[109143]: 2024-04-21 08:53:59.499084194 +0000 UTC m=+0.384386884 container attach c8091616198f4e08613dff5d081316f78ef42bae51cd43c8c152ffab3cb58719 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9-deactivate, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-04-21T08:53:59.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:59 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:59.341+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:53:59.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:59 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:59.341+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:53:59.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:59 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:59.341+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:53:59.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:59 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:59.341+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:53:59.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:59 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:59.341+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:53:59.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:59 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:59.341+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.269936+0000 front 2024-04-21T08:53:24.270028+0000 (oldest deadline 2024-04-21T08:53:45.969918+0000) 2024-04-21T08:53:59.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:53:59 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:53:59.341+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.171510+0000 front 2024-04-21T08:53:31.171546+0000 (oldest deadline 2024-04-21T08:53:55.271423+0000) 2024-04-21T08:53:59.813 INFO:journalctl@ceph.osd.9.smithi043.stdout:Apr 21 08:53:59 smithi043 podman[109143]: 2024-04-21 08:53:59.725693705 +0000 UTC m=+0.610996381 container died c8091616198f4e08613dff5d081316f78ef42bae51cd43c8c152ffab3cb58719 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-9-deactivate, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-21T08:53:59.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:59 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:59.675+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:53:59.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:59 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:59.675+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:53:59.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:59 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:59.675+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:53:59.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:59 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:59.675+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:53:59.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:59 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:59.675+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:53:59.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:53:59 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:53:59.675+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.909620+0000 front 2024-04-21T08:53:31.909513+0000 (oldest deadline 2024-04-21T08:53:54.809372+0000) 2024-04-21T08:54:00.371 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.9.service' 2024-04-21T08:54:00.415 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:54:00.415 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-21T08:54:00.416 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-21T08:54:00.416 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.10 2024-04-21T08:54:00.723 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:00 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:00.327+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:54:00.723 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:00 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:00.327+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:54:00.723 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:00 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:00.327+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:54:00.723 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:00 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:00.327+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:54:00.723 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:00 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:00.327+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:54:00.723 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:00 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:00.327+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.269936+0000 front 2024-04-21T08:53:24.270028+0000 (oldest deadline 2024-04-21T08:53:45.969918+0000) 2024-04-21T08:54:00.724 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:00 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:00.327+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.171510+0000 front 2024-04-21T08:53:31.171546+0000 (oldest deadline 2024-04-21T08:53:55.271423+0000) 2024-04-21T08:54:00.724 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:00 smithi112 systemd[1]: Stopping Ceph osd.10 for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:54:00.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:00 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:00.656+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:54:00.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:00 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:00.656+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:54:00.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:00 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:00.656+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:54:00.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:00 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:00.656+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:54:00.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:00 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:00.656+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:54:00.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:00 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:00.656+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.909620+0000 front 2024-04-21T08:53:31.909513+0000 (oldest deadline 2024-04-21T08:53:54.809372+0000) 2024-04-21T08:54:01.044 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:00 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:00.722+0000 7f9e51c5d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:54:01.044 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:00 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:00.722+0000 7f9e51c5d700 -1 osd.10 114 *** Got signal Terminated *** 2024-04-21T08:54:01.044 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:00 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:00.722+0000 7f9e51c5d700 -1 osd.10 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:54:01.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:01 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:01.294+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:54:01.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:01 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:01.294+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:54:01.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:01 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:01.294+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:54:01.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:01 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:01.294+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:54:01.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:01 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:01.294+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:54:01.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:01 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:01.294+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.269936+0000 front 2024-04-21T08:53:24.270028+0000 (oldest deadline 2024-04-21T08:53:45.969918+0000) 2024-04-21T08:54:01.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:01 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:01.294+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.171510+0000 front 2024-04-21T08:53:31.171546+0000 (oldest deadline 2024-04-21T08:53:55.271423+0000) 2024-04-21T08:54:01.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:01 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:01.622+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:54:01.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:01 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:01.622+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:54:01.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:01 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:01.622+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:54:01.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:01 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:01.622+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:54:01.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:01 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:01.622+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:54:01.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:01 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:01.622+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.909620+0000 front 2024-04-21T08:53:31.909513+0000 (oldest deadline 2024-04-21T08:53:54.809372+0000) 2024-04-21T08:54:02.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:02 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:02.301+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:54:02.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:02 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:02.301+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:54:02.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:02 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:02.301+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:54:02.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:02 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:02.301+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:54:02.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:02 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:02.301+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:54:02.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:02 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:02.301+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.269936+0000 front 2024-04-21T08:53:24.270028+0000 (oldest deadline 2024-04-21T08:53:45.969918+0000) 2024-04-21T08:54:02.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:02 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:02.301+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.171510+0000 front 2024-04-21T08:53:31.171546+0000 (oldest deadline 2024-04-21T08:53:55.271423+0000) 2024-04-21T08:54:02.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:02 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:02.641+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:54:02.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:02 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:02.641+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:54:02.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:02 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:02.641+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:54:02.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:02 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:02.641+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:54:02.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:02 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:02.641+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:54:02.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:02 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:02.641+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.909620+0000 front 2024-04-21T08:53:31.909513+0000 (oldest deadline 2024-04-21T08:53:54.809372+0000) 2024-04-21T08:54:03.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:03 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:03.321+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:54:03.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:03 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:03.321+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:54:03.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:03 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:03.321+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:54:03.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:03 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:03.321+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:54:03.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:03 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:03.321+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:54:03.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:03 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:03.321+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.269936+0000 front 2024-04-21T08:53:24.270028+0000 (oldest deadline 2024-04-21T08:53:45.969918+0000) 2024-04-21T08:54:03.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:03 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:03.321+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.171510+0000 front 2024-04-21T08:53:31.171546+0000 (oldest deadline 2024-04-21T08:53:55.271423+0000) 2024-04-21T08:54:03.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:03 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:03.690+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:54:03.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:03 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:03.690+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:54:03.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:03 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:03.690+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:54:03.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:03 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:03.690+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:54:03.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:03 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:03.690+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:54:03.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:03 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:03.690+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.909620+0000 front 2024-04-21T08:53:31.909513+0000 (oldest deadline 2024-04-21T08:53:54.809372+0000) 2024-04-21T08:54:04.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:04 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:04.339+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:54:04.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:04 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:04.339+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:54:04.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:04 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:04.339+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:54:04.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:04 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:04.339+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:54:04.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:04 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:04.339+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:54:04.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:04 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:04.339+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.269936+0000 front 2024-04-21T08:53:24.270028+0000 (oldest deadline 2024-04-21T08:53:45.969918+0000) 2024-04-21T08:54:04.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:04 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:04.339+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.171510+0000 front 2024-04-21T08:53:31.171546+0000 (oldest deadline 2024-04-21T08:53:55.271423+0000) 2024-04-21T08:54:04.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:04 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:04.669+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:54:04.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:04 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:04.669+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:54:04.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:04 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:04.669+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:54:04.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:04 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:04.669+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:54:04.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:04 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:04.669+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:54:04.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:04 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:04.669+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.909620+0000 front 2024-04-21T08:53:31.909513+0000 (oldest deadline 2024-04-21T08:53:54.809372+0000) 2024-04-21T08:54:05.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:05 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:05.326+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.361599+0000 front 2024-04-21T08:52:43.361526+0000 (oldest deadline 2024-04-21T08:53:09.162203+0000) 2024-04-21T08:54:05.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:05 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:05.326+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:49.162547+0000 front 2024-04-21T08:52:49.162579+0000 (oldest deadline 2024-04-21T08:53:13.862713+0000) 2024-04-21T08:54:05.794 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:05 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:05.326+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:53:00.264316+0000 front 2024-04-21T08:53:00.264247+0000 (oldest deadline 2024-04-21T08:53:26.164135+0000) 2024-04-21T08:54:05.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:05 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:05.326+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-04-21T08:53:09.466253+0000 front 2024-04-21T08:53:09.466186+0000 (oldest deadline 2024-04-21T08:53:32.966157+0000) 2024-04-21T08:54:05.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:05 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:05.326+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:16.466914+0000 front 2024-04-21T08:53:16.466937+0000 (oldest deadline 2024-04-21T08:53:38.766993+0000) 2024-04-21T08:54:05.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:05 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:05.326+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:24.269936+0000 front 2024-04-21T08:53:24.270028+0000 (oldest deadline 2024-04-21T08:53:45.969918+0000) 2024-04-21T08:54:05.795 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:05 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10[55919]: 2024-04-21T08:54:05.326+0000 7f9e4a249700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.171510+0000 front 2024-04-21T08:53:31.171546+0000 (oldest deadline 2024-04-21T08:53:55.271423+0000) 2024-04-21T08:54:05.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:05.666+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:54:05.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:05.666+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:54:05.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:05.666+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:54:05.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:05.666+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:54:05.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:05.666+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:54:05.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:05.666+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.909620+0000 front 2024-04-21T08:53:31.909513+0000 (oldest deadline 2024-04-21T08:53:54.809372+0000) 2024-04-21T08:54:05.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:05 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:05.666+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-21T08:53:38.910472+0000 front 2024-04-21T08:53:38.910572+0000 (oldest deadline 2024-04-21T08:54:04.810362+0000) 2024-04-21T08:54:06.308 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:06 smithi112 podman[68348]: 2024-04-21 08:54:06.050882467 +0000 UTC m=+5.438536338 container died b835457b5eb94f5934096afc24b5ecfad82d7846fbffb9a15d1c6d1af2ca2091 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:54:06.683 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:06 smithi112 podman[68348]: 2024-04-21 08:54:06.408031069 +0000 UTC m=+5.795684933 container cleanup b835457b5eb94f5934096afc24b5ecfad82d7846fbffb9a15d1c6d1af2ca2091 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-21T08:54:06.683 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:06 smithi112 bash[68348]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10 2024-04-21T08:54:06.966 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:06 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:06.630+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:54:06.966 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:06 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:06.630+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:54:06.966 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:06 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:06.630+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:54:06.966 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:06 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:06.630+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:54:06.967 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:06 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:06.630+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:54:06.967 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:06 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:06.630+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.909620+0000 front 2024-04-21T08:53:31.909513+0000 (oldest deadline 2024-04-21T08:53:54.809372+0000) 2024-04-21T08:54:06.967 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:06 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:06.630+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-21T08:53:38.910472+0000 front 2024-04-21T08:53:38.910572+0000 (oldest deadline 2024-04-21T08:54:04.810362+0000) 2024-04-21T08:54:07.029 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:06 smithi112 podman[68366]: 2024-04-21 08:54:06.683932507 +0000 UTC m=+0.627753900 container remove b835457b5eb94f5934096afc24b5ecfad82d7846fbffb9a15d1c6d1af2ca2091 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, ceph=True) 2024-04-21T08:54:07.286 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:07 smithi112 podman[68456]: 2024-04-21 08:54:06.932853359 +0000 UTC m=+0.017131498 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:54:07.286 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:07 smithi112 podman[68456]: 2024-04-21 08:54:07.042948001 +0000 UTC m=+0.127226129 container create b00fca5832319d2ec7a3476f116eedbc197278f6827166d4b0c0c88af5f18b46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True) 2024-04-21T08:54:07.286 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:07 smithi112 podman[68456]: 2024-04-21 08:54:07.287168111 +0000 UTC m=+0.371446243 container init b00fca5832319d2ec7a3476f116eedbc197278f6827166d4b0c0c88af5f18b46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-21T08:54:07.544 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:07 smithi112 podman[68456]: 2024-04-21 08:54:07.290168759 +0000 UTC m=+0.374446892 container start b00fca5832319d2ec7a3476f116eedbc197278f6827166d4b0c0c88af5f18b46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-21T08:54:07.544 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:07 smithi112 podman[68456]: 2024-04-21 08:54:07.37868518 +0000 UTC m=+0.462963310 container attach b00fca5832319d2ec7a3476f116eedbc197278f6827166d4b0c0c88af5f18b46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, ceph=True) 2024-04-21T08:54:07.914 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 21 08:54:07 smithi112 podman[68456]: 2024-04-21 08:54:07.586372551 +0000 UTC m=+0.670650681 container died b00fca5832319d2ec7a3476f116eedbc197278f6827166d4b0c0c88af5f18b46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2) 2024-04-21T08:54:07.920 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:07 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:07.606+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:54:07.920 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:07 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:07.606+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:54:07.920 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:07 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:07.606+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:54:07.920 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:07 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:07.606+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:54:07.920 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:07 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:07.606+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:54:07.921 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:07 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:07.606+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.909620+0000 front 2024-04-21T08:53:31.909513+0000 (oldest deadline 2024-04-21T08:53:54.809372+0000) 2024-04-21T08:54:07.921 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:07 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:07.606+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-21T08:53:38.910472+0000 front 2024-04-21T08:53:38.910572+0000 (oldest deadline 2024-04-21T08:54:04.810362+0000) 2024-04-21T08:54:08.190 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.10.service' 2024-04-21T08:54:08.227 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:54:08.227 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-21T08:54:08.227 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-21T08:54:08.227 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.11 2024-04-21T08:54:08.530 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:08 smithi175 systemd[1]: Stopping Ceph osd.11 for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:54:08.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:08 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:08.530+0000 7f0d53f9f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:54:08.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:08 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:08.530+0000 7f0d53f9f700 -1 osd.11 114 *** Got signal Terminated *** 2024-04-21T08:54:08.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:08 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:08.530+0000 7f0d53f9f700 -1 osd.11 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:54:08.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:08 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:08.652+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:54:08.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:08 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:08.652+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:54:08.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:08 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:08.652+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:54:08.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:08 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:08.652+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:54:08.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:08 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:08.652+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:54:08.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:08 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:08.652+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.909620+0000 front 2024-04-21T08:53:31.909513+0000 (oldest deadline 2024-04-21T08:53:54.809372+0000) 2024-04-21T08:54:08.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:08 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:08.652+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-21T08:53:38.910472+0000 front 2024-04-21T08:53:38.910572+0000 (oldest deadline 2024-04-21T08:54:04.810362+0000) 2024-04-21T08:54:09.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:09 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:09.651+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:54:09.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:09 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:09.651+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:54:09.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:09 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:09.651+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:54:09.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:09 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:09.651+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:54:09.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:09 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:09.651+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:54:09.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:09 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:09.651+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.909620+0000 front 2024-04-21T08:53:31.909513+0000 (oldest deadline 2024-04-21T08:53:54.809372+0000) 2024-04-21T08:54:09.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:09 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:09.651+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-21T08:53:38.910472+0000 front 2024-04-21T08:53:38.910572+0000 (oldest deadline 2024-04-21T08:54:04.810362+0000) 2024-04-21T08:54:10.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:10 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:10.638+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:54:10.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:10 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:10.638+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:54:10.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:10 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:10.638+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:54:10.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:10 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:10.638+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:54:10.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:10 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:10.638+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:54:10.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:10 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:10.638+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.909620+0000 front 2024-04-21T08:53:31.909513+0000 (oldest deadline 2024-04-21T08:53:54.809372+0000) 2024-04-21T08:54:10.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:10 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:10.638+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-21T08:53:38.910472+0000 front 2024-04-21T08:53:38.910572+0000 (oldest deadline 2024-04-21T08:54:04.810362+0000) 2024-04-21T08:54:11.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:11 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:11.678+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:54:11.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:11 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:11.678+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:54:11.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:11 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:11.678+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:54:11.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:11 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:11.678+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:54:11.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:11 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:11.678+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:54:11.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:11 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:11.678+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.909620+0000 front 2024-04-21T08:53:31.909513+0000 (oldest deadline 2024-04-21T08:53:54.809372+0000) 2024-04-21T08:54:11.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:11 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:11.678+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-21T08:53:38.910472+0000 front 2024-04-21T08:53:38.910572+0000 (oldest deadline 2024-04-21T08:54:04.810362+0000) 2024-04-21T08:54:12.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:12 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:12.678+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-04-21T08:52:43.700804+0000 front 2024-04-21T08:52:43.700853+0000 (oldest deadline 2024-04-21T08:53:08.400860+0000) 2024-04-21T08:54:12.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:12 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:12.678+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-21T08:52:52.402552+0000 front 2024-04-21T08:52:52.402469+0000 (oldest deadline 2024-04-21T08:53:18.302493+0000) 2024-04-21T08:54:12.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:12 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:12.678+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-04-21T08:52:59.403524+0000 front 2024-04-21T08:52:59.403415+0000 (oldest deadline 2024-04-21T08:53:23.503670+0000) 2024-04-21T08:54:12.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:12 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:12.678+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-21T08:53:17.406860+0000 front 2024-04-21T08:53:17.406909+0000 (oldest deadline 2024-04-21T08:53:43.306781+0000) 2024-04-21T08:54:12.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:12 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:12.678+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-04-21T08:53:23.307297+0000 front 2024-04-21T08:53:23.307188+0000 (oldest deadline 2024-04-21T08:53:46.807337+0000) 2024-04-21T08:54:12.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:12 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:12.678+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-04-21T08:53:31.909620+0000 front 2024-04-21T08:53:31.909513+0000 (oldest deadline 2024-04-21T08:53:54.809372+0000) 2024-04-21T08:54:12.965 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:12 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11[57158]: 2024-04-21T08:54:12.678+0000 7f0d50593700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-21T08:53:38.910472+0000 front 2024-04-21T08:53:38.910572+0000 (oldest deadline 2024-04-21T08:54:04.810362+0000) 2024-04-21T08:54:14.129 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:13 smithi175 podman[70055]: 2024-04-21 08:54:13.846538089 +0000 UTC m=+5.436511028 container died a4905d0f54e2a430085915f791c6e385fc734173a65a37dd65aa312e3110d244 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-21T08:54:14.129 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:14 smithi175 podman[70055]: 2024-04-21 08:54:14.13014081 +0000 UTC m=+5.720113751 container cleanup a4905d0f54e2a430085915f791c6e385fc734173a65a37dd65aa312e3110d244 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-21T08:54:14.464 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:14 smithi175 bash[70055]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11 2024-04-21T08:54:14.464 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:14 smithi175 podman[70072]: 2024-04-21 08:54:14.364405055 +0000 UTC m=+0.513613811 container remove a4905d0f54e2a430085915f791c6e385fc734173a65a37dd65aa312e3110d244 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:54:14.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:14 smithi175 podman[70163]: 2024-04-21 08:54:14.681915885 +0000 UTC m=+0.114866439 container create 56f684461aabbbfa253c3b02e2e8f6f45382f04b78fa2906907923135f477f2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-21T08:54:14.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:14 smithi175 podman[70163]: 2024-04-21 08:54:14.585087713 +0000 UTC m=+0.018038236 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:54:14.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:14 smithi175 podman[70163]: 2024-04-21 08:54:14.918298461 +0000 UTC m=+0.351248972 container init 56f684461aabbbfa253c3b02e2e8f6f45382f04b78fa2906907923135f477f2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-21T08:54:14.964 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:14 smithi175 podman[70163]: 2024-04-21 08:54:14.921249381 +0000 UTC m=+0.354199895 container start 56f684461aabbbfa253c3b02e2e8f6f45382f04b78fa2906907923135f477f2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-21T08:54:15.464 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:15 smithi175 podman[70163]: 2024-04-21 08:54:15.010545484 +0000 UTC m=+0.443496013 container attach 56f684461aabbbfa253c3b02e2e8f6f45382f04b78fa2906907923135f477f2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-21T08:54:15.464 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:15 smithi175 podman[70163]: 2024-04-21 08:54:15.210225509 +0000 UTC m=+0.643176012 container died 56f684461aabbbfa253c3b02e2e8f6f45382f04b78fa2906907923135f477f2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-21T08:54:15.772 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:15 smithi175 podman[70190]: 2024-04-21 08:54:15.755721045 +0000 UTC m=+0.536179001 container remove 56f684461aabbbfa253c3b02e2e8f6f45382f04b78fa2906907923135f477f2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-21T08:54:15.773 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:15 smithi175 systemd[1]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.11.service: Deactivated successfully. 2024-04-21T08:54:15.773 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:15 smithi175 systemd[1]: Stopped Ceph osd.11 for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:54:15.773 INFO:journalctl@ceph.osd.11.smithi175.stdout:Apr 21 08:54:15 smithi175 systemd[1]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.11.service: Consumed 11.371s CPU time. 2024-04-21T08:54:15.774 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@osd.11.service' 2024-04-21T08:54:15.811 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:54:15.812 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-21T08:54:15.812 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-21T08:54:15.812 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.a 2024-04-21T08:54:16.146 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 21 08:54:15 smithi043 systemd[1]: Stopping Ceph mds.a for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:54:16.562 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 21 08:54:16 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-a[67537]: 2024-04-21T08:54:16.146+0000 7fb0bb0ea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:54:16.562 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 21 08:54:16 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-a[67537]: 2024-04-21T08:54:16.146+0000 7fb0bb0ea700 -1 mds.a *** got signal Terminated *** 2024-04-21T08:54:20.021 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 21 08:54:19 smithi043 podman[109261]: 2024-04-21 08:54:19.740472085 +0000 UTC m=+3.725674564 container died 22dc542087dbc3ddf5da4ff364fad092e7c84e78ed3840facbe4ad281af22a1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-21T08:54:20.312 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 21 08:54:20 smithi043 podman[109261]: 2024-04-21 08:54:20.254185387 +0000 UTC m=+4.239387855 container cleanup 22dc542087dbc3ddf5da4ff364fad092e7c84e78ed3840facbe4ad281af22a1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:54:20.312 INFO:journalctl@ceph.mds.a.smithi043.stdout:Apr 21 08:54:20 smithi043 bash[109261]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-a 2024-04-21T08:54:20.644 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.a.service' 2024-04-21T08:54:20.679 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:54:20.679 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-21T08:54:20.679 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-21T08:54:20.679 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.d 2024-04-21T08:54:21.062 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 21 08:54:20 smithi043 systemd[1]: Stopping Ceph mds.d for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:54:21.062 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 21 08:54:21 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-d[66541]: 2024-04-21T08:54:20.999+0000 7f6182aae700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:54:21.062 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 21 08:54:21 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-d[66541]: 2024-04-21T08:54:20.999+0000 7f6182aae700 -1 mds.d *** got signal Terminated *** 2024-04-21T08:54:23.999 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 21 08:54:23 smithi043 podman[109381]: 2024-04-21 08:54:23.720771121 +0000 UTC m=+2.816087845 container died 63a5476ed53054dfa441f882f7b5cc445f9388ed100e17fe5e1542c6e3bb610a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:54:24.312 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 21 08:54:24 smithi043 podman[109381]: 2024-04-21 08:54:24.082606698 +0000 UTC m=+3.177923425 container cleanup 63a5476ed53054dfa441f882f7b5cc445f9388ed100e17fe5e1542c6e3bb610a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-21T08:54:24.312 INFO:journalctl@ceph.mds.d.smithi043.stdout:Apr 21 08:54:24 smithi043 bash[109381]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-d 2024-04-21T08:54:24.521 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.d.service' 2024-04-21T08:54:24.554 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:54:24.554 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-21T08:54:24.555 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-21T08:54:24.555 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.g 2024-04-21T08:54:24.877 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 21 08:54:24 smithi043 systemd[1]: Stopping Ceph mds.g for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:54:25.312 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 21 08:54:24 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-g[67149]: 2024-04-21T08:54:24.877+0000 7f264eef8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:54:25.312 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 21 08:54:24 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-g[67149]: 2024-04-21T08:54:24.877+0000 7f264eef8700 -1 mds.g *** got signal Terminated *** 2024-04-21T08:54:28.062 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 21 08:54:27 smithi043 podman[109501]: 2024-04-21 08:54:27.753622911 +0000 UTC m=+2.970534394 container died a244b7e4086e783f885905983bb3b213e0c10f87d8068d8f73e30d9d4d2d1e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-21T08:54:28.495 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 21 08:54:28 smithi043 podman[109501]: 2024-04-21 08:54:28.236796056 +0000 UTC m=+3.453707538 container cleanup a244b7e4086e783f885905983bb3b213e0c10f87d8068d8f73e30d9d4d2d1e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-g, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-21T08:54:28.495 INFO:journalctl@ceph.mds.g.smithi043.stdout:Apr 21 08:54:28 smithi043 bash[109501]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-g 2024-04-21T08:54:28.617 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.g.service' 2024-04-21T08:54:28.648 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:54:28.649 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-21T08:54:28.649 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-21T08:54:28.649 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.j 2024-04-21T08:54:28.811 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 21 08:54:28 smithi043 systemd[1]: Stopping Ceph mds.j for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:54:29.312 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 21 08:54:29 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-j[66193]: 2024-04-21T08:54:29.023+0000 7fdfe907a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:54:29.312 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 21 08:54:29 smithi043 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-j[66193]: 2024-04-21T08:54:29.023+0000 7fdfe907a700 -1 mds.j *** got signal Terminated *** 2024-04-21T08:54:33.062 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 21 08:54:32 smithi043 podman[109623]: 2024-04-21 08:54:32.790730162 +0000 UTC m=+3.924735131 container died 3d09652be3c85cc996ca41c1f5a68ccace2a5c4d3e575f4950634fb3dfc7f6fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-21T08:54:33.434 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 21 08:54:33 smithi043 podman[109623]: 2024-04-21 08:54:33.150403782 +0000 UTC m=+4.284408739 container cleanup 3d09652be3c85cc996ca41c1f5a68ccace2a5c4d3e575f4950634fb3dfc7f6fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-04-21T08:54:33.434 INFO:journalctl@ceph.mds.j.smithi043.stdout:Apr 21 08:54:33 smithi043 bash[109623]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-j 2024-04-21T08:54:33.592 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.j.service' 2024-04-21T08:54:33.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:54:33.623 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-21T08:54:33.624 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-21T08:54:33.624 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.b 2024-04-21T08:54:33.924 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:54:33 smithi112 systemd[1]: Stopping Ceph mds.b for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:54:34.182 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:54:33 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-b[61578]: 2024-04-21T08:54:33.924+0000 7f425c7ec700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:54:34.182 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:54:33 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-b[61578]: 2024-04-21T08:54:33.924+0000 7f425c7ec700 -1 mds.b *** got signal Terminated *** 2024-04-21T08:54:36.040 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:54:35 smithi112 podman[68561]: 2024-04-21 08:54:35.760028584 +0000 UTC m=+1.939880003 container died bea88b79092d11d5191c0b8ba200742cf9867263235771011a250f3d37e35dad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:54:36.040 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:54:36 smithi112 podman[68561]: 2024-04-21 08:54:36.041290595 +0000 UTC m=+2.221142016 container cleanup bea88b79092d11d5191c0b8ba200742cf9867263235771011a250f3d37e35dad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:54:36.294 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:54:36 smithi112 bash[68561]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-b 2024-04-21T08:54:36.294 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 21 08:54:36 smithi112 podman[68587]: 2024-04-21 08:54:36.284028407 +0000 UTC m=+0.525612552 container remove bea88b79092d11d5191c0b8ba200742cf9867263235771011a250f3d37e35dad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-21T08:54:36.425 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.b.service' 2024-04-21T08:54:36.460 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:54:36.460 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-21T08:54:36.460 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-21T08:54:36.460 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.e 2024-04-21T08:54:36.778 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 21 08:54:36 smithi112 systemd[1]: Stopping Ceph mds.e for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:54:37.044 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 21 08:54:36 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-e[60958]: 2024-04-21T08:54:36.778+0000 7f4858ca4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:54:37.044 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 21 08:54:36 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-e[60958]: 2024-04-21T08:54:36.778+0000 7f4858ca4700 -1 mds.e *** got signal Terminated *** 2024-04-21T08:54:40.037 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 21 08:54:39 smithi112 podman[68694]: 2024-04-21 08:54:39.761038947 +0000 UTC m=+3.086147413 container died cc359f26aee18c802f4bb7ad38baa028380957f4ed1050a61d7ecd89338ef7f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-e, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-21T08:54:40.294 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 21 08:54:40 smithi112 podman[68694]: 2024-04-21 08:54:40.129197424 +0000 UTC m=+3.454305881 container cleanup cc359f26aee18c802f4bb7ad38baa028380957f4ed1050a61d7ecd89338ef7f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416) 2024-04-21T08:54:40.294 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 21 08:54:40 smithi112 bash[68694]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-e 2024-04-21T08:54:40.557 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.e.service' 2024-04-21T08:54:40.589 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:54:40.589 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-21T08:54:40.589 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-21T08:54:40.589 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.h 2024-04-21T08:54:41.044 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 21 08:54:40 smithi112 systemd[1]: Stopping Ceph mds.h for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:54:41.044 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 21 08:54:40 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-h[61265]: 2024-04-21T08:54:40.915+0000 7f190ba8f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:54:41.044 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 21 08:54:40 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-h[61265]: 2024-04-21T08:54:40.915+0000 7f190ba8f700 -1 mds.h *** got signal Terminated *** 2024-04-21T08:54:44.044 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 21 08:54:43 smithi112 podman[68814]: 2024-04-21 08:54:43.754646097 +0000 UTC m=+2.948425847 container died 74e1523b5639d2cccb7f21e93c565a16becdd07285169fd9f111f6c5dd45f226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-h, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-21T08:54:44.544 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 21 08:54:44 smithi112 podman[68814]: 2024-04-21 08:54:44.332578255 +0000 UTC m=+3.526358010 container cleanup 74e1523b5639d2cccb7f21e93c565a16becdd07285169fd9f111f6c5dd45f226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-21T08:54:44.544 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 21 08:54:44 smithi112 bash[68814]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-h 2024-04-21T08:54:44.754 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.h.service' 2024-04-21T08:54:44.785 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:54:44.785 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-21T08:54:44.785 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-21T08:54:44.786 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.k 2024-04-21T08:54:45.177 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 21 08:54:44 smithi112 systemd[1]: Stopping Ceph mds.k for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:54:45.177 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 21 08:54:45 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-k[60639]: 2024-04-21T08:54:45.094+0000 7f14fe459700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:54:45.177 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 21 08:54:45 smithi112 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-k[60639]: 2024-04-21T08:54:45.094+0000 7f14fe459700 -1 mds.k *** got signal Terminated *** 2024-04-21T08:54:48.044 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 21 08:54:47 smithi112 podman[68934]: 2024-04-21 08:54:47.752231414 +0000 UTC m=+2.754212979 container died 45235075818ce263e8c0d652558f4c40fe1560f798a1c095675557bdbdd2bba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-04-21T08:54:48.470 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 21 08:54:48 smithi112 podman[68934]: 2024-04-21 08:54:48.18670071 +0000 UTC m=+3.188682282 container cleanup 45235075818ce263e8c0d652558f4c40fe1560f798a1c095675557bdbdd2bba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:54:48.470 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 21 08:54:48 smithi112 bash[68934]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-k 2024-04-21T08:54:48.622 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.k.service' 2024-04-21T08:54:48.652 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:54:48.652 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-21T08:54:48.652 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-21T08:54:48.652 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.c 2024-04-21T08:54:48.963 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:54:48 smithi175 systemd[1]: Stopping Ceph mds.c for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:54:49.214 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:54:48 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[66800]: 2024-04-21T08:54:48.963+0000 7f65960d3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:54:49.214 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:54:48 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c[66800]: 2024-04-21T08:54:48.963+0000 7f65960d3700 -1 mds.c *** got signal Terminated *** 2024-04-21T08:54:52.022 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:54:51 smithi175 podman[70270]: 2024-04-21 08:54:51.742001737 +0000 UTC m=+2.899484406 container died 5397489a561c84c2806161ea4c962df51bf8893a6d3c47bf00f7d7503abd8b29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:54:52.347 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:54:52 smithi175 podman[70270]: 2024-04-21 08:54:52.106222273 +0000 UTC m=+3.263704950 container cleanup 5397489a561c84c2806161ea4c962df51bf8893a6d3c47bf00f7d7503abd8b29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, ceph=True) 2024-04-21T08:54:52.348 INFO:journalctl@ceph.mds.c.smithi175.stdout:Apr 21 08:54:52 smithi175 bash[70270]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-c 2024-04-21T08:54:52.477 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.c.service' 2024-04-21T08:54:52.511 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:54:52.512 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-21T08:54:52.512 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-21T08:54:52.512 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.f 2024-04-21T08:54:52.714 INFO:journalctl@ceph.mds.f.smithi175.stdout:Apr 21 08:54:52 smithi175 systemd[1]: Stopping Ceph mds.f for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:54:53.214 INFO:journalctl@ceph.mds.f.smithi175.stdout:Apr 21 08:54:52 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-f[61854]: 2024-04-21T08:54:52.834+0000 7f07b86f5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:54:53.214 INFO:journalctl@ceph.mds.f.smithi175.stdout:Apr 21 08:54:52 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-f[61854]: 2024-04-21T08:54:52.834+0000 7f07b86f5700 -1 mds.f *** got signal Terminated *** 2024-04-21T08:54:56.052 INFO:journalctl@ceph.mds.f.smithi175.stdout:Apr 21 08:54:55 smithi175 podman[70403]: 2024-04-21 08:54:55.764951375 +0000 UTC m=+3.035920212 container died 9b7c904433b3674f4e2bd30c41d9e7f3b7e4c16bf275a7a066f0935593355b76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-21T08:54:56.379 INFO:journalctl@ceph.mds.f.smithi175.stdout:Apr 21 08:54:56 smithi175 podman[70403]: 2024-04-21 08:54:56.135614377 +0000 UTC m=+3.406583199 container cleanup 9b7c904433b3674f4e2bd30c41d9e7f3b7e4c16bf275a7a066f0935593355b76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-f, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-21T08:54:56.380 INFO:journalctl@ceph.mds.f.smithi175.stdout:Apr 21 08:54:56 smithi175 bash[70403]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-f 2024-04-21T08:54:56.510 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.f.service' 2024-04-21T08:54:56.542 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:54:56.542 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-21T08:54:56.542 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-21T08:54:56.543 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.i 2024-04-21T08:54:56.714 INFO:journalctl@ceph.mds.i.smithi175.stdout:Apr 21 08:54:56 smithi175 systemd[1]: Stopping Ceph mds.i for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:54:57.214 INFO:journalctl@ceph.mds.i.smithi175.stdout:Apr 21 08:54:56 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-i[62469]: 2024-04-21T08:54:56.856+0000 7f97ce08b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:54:57.214 INFO:journalctl@ceph.mds.i.smithi175.stdout:Apr 21 08:54:56 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-i[62469]: 2024-04-21T08:54:56.856+0000 7f97ce08b700 -1 mds.i *** got signal Terminated *** 2024-04-21T08:54:59.966 INFO:journalctl@ceph.mds.i.smithi175.stdout:Apr 21 08:54:59 smithi175 podman[70525]: 2024-04-21 08:54:59.756629934 +0000 UTC m=+2.999886295 container died 03ad24450f0c3fae67212ea21f599898c554e679ad9b93e4fbabc7e339ef994a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-21T08:55:00.416 INFO:journalctl@ceph.mds.i.smithi175.stdout:Apr 21 08:55:00 smithi175 podman[70525]: 2024-04-21 08:55:00.199074702 +0000 UTC m=+3.442331073 container cleanup 03ad24450f0c3fae67212ea21f599898c554e679ad9b93e4fbabc7e339ef994a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-21T08:55:00.416 INFO:journalctl@ceph.mds.i.smithi175.stdout:Apr 21 08:55:00 smithi175 bash[70525]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-i 2024-04-21T08:55:00.657 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.i.service' 2024-04-21T08:55:00.677 INFO:journalctl@ceph.mds.i.smithi175.stdout:Apr 21 08:55:00 smithi175 podman[70539]: 2024-04-21 08:55:00.416955953 +0000 UTC m=+0.659149305 container remove 03ad24450f0c3fae67212ea21f599898c554e679ad9b93e4fbabc7e339ef994a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-i, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:55:00.677 INFO:journalctl@ceph.mds.i.smithi175.stdout:Apr 21 08:55:00 smithi175 systemd[1]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.i.service: Deactivated successfully. 2024-04-21T08:55:00.677 INFO:journalctl@ceph.mds.i.smithi175.stdout:Apr 21 08:55:00 smithi175 systemd[1]: Stopped Ceph mds.i for d57a7d98-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:55:00.677 INFO:journalctl@ceph.mds.i.smithi175.stdout:Apr 21 08:55:00 smithi175 systemd[1]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.i.service: Consumed 2.332s CPU time. 2024-04-21T08:55:00.687 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:55:00.688 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-21T08:55:00.688 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-21T08:55:00.688 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.l 2024-04-21T08:55:00.964 INFO:journalctl@ceph.mds.l.smithi175.stdout:Apr 21 08:55:00 smithi175 systemd[1]: Stopping Ceph mds.l for d57a7d98-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:55:01.464 INFO:journalctl@ceph.mds.l.smithi175.stdout:Apr 21 08:55:01 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-l[61532]: 2024-04-21T08:55:01.010+0000 7fc6ea731700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:55:01.464 INFO:journalctl@ceph.mds.l.smithi175.stdout:Apr 21 08:55:01 smithi175 ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-l[61532]: 2024-04-21T08:55:01.010+0000 7fc6ea731700 -1 mds.l *** got signal Terminated *** 2024-04-21T08:55:04.002 INFO:journalctl@ceph.mds.l.smithi175.stdout:Apr 21 08:55:03 smithi175 podman[70644]: 2024-04-21 08:55:03.749494269 +0000 UTC m=+2.849931746 container died 7b28e9f656202ef7d3de2ed5832054f59c2ad50578f82b13e6f5eb4a8ad21bfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:55:04.377 INFO:journalctl@ceph.mds.l.smithi175.stdout:Apr 21 08:55:04 smithi175 podman[70644]: 2024-04-21 08:55:04.102409336 +0000 UTC m=+3.202846798 container cleanup 7b28e9f656202ef7d3de2ed5832054f59c2ad50578f82b13e6f5eb4a8ad21bfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-l, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD) 2024-04-21T08:55:04.377 INFO:journalctl@ceph.mds.l.smithi175.stdout:Apr 21 08:55:04 smithi175 bash[70644]: ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968-mds-l 2024-04-21T08:55:04.531 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d57a7d98-ffb8-11ee-bc93-c7b262605968@mds.l.service' 2024-04-21T08:55:04.563 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:55:04.563 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-21T08:55:04.563 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 --force --keep-logs 2024-04-21T08:55:04.762 INFO:teuthology.orchestra.run.smithi043.stdout:Deleting cluster with fsid: d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:55:07.299 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 --force --keep-logs 2024-04-21T08:55:07.489 INFO:teuthology.orchestra.run.smithi112.stdout:Deleting cluster with fsid: d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:55:09.909 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 --force --keep-logs 2024-04-21T08:55:10.098 INFO:teuthology.orchestra.run.smithi175.stdout:Deleting cluster with fsid: d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:55:12.367 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:55:12.400 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:55:12.432 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:55:12.457 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-21T08:55:12.465 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666510/remote/smithi043/crash 2024-04-21T08:55:12.466 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/crash -- . 2024-04-21T08:55:12.510 INFO:teuthology.orchestra.run.smithi043.stderr:tar: /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-21T08:55:12.510 INFO:teuthology.orchestra.run.smithi043.stderr:tar: Error is not recoverable: exiting now 2024-04-21T08:55:12.643 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666510/remote/smithi112/crash 2024-04-21T08:55:12.660 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/crash -- . 2024-04-21T08:55:12.785 INFO:teuthology.orchestra.run.smithi112.stderr:tar: /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-21T08:55:12.785 INFO:teuthology.orchestra.run.smithi112.stderr:tar: Error is not recoverable: exiting now 2024-04-21T08:55:12.787 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666510/remote/smithi175/crash 2024-04-21T08:55:12.788 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/crash -- . 2024-04-21T08:55:12.831 INFO:teuthology.orchestra.run.smithi175.stderr:tar: /var/lib/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-21T08:55:12.832 INFO:teuthology.orchestra.run.smithi175.stderr:tar: Error is not recoverable: exiting now 2024-04-21T08:55:12.833 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-21T08:55:12.833 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-21T08:55:12.878 INFO:tasks.cephadm:Compressing logs... 2024-04-21T08:55:12.878 DEBUG:teuthology.orchestra.run.smithi043:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-21T08:55:12.920 DEBUG:teuthology.orchestra.run.smithi112:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-21T08:55:12.923 DEBUG:teuthology.orchestra.run.smithi175:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-21T08:55:12.945 INFO:teuthology.orchestra.run.smithi043.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.log 2024-04-21T08:55:12.945 INFO:teuthology.orchestra.run.smithi043.stderr:: No such file or directory 2024-04-21T08:55:12.945 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.audit.log 2024-04-21T08:55:12.948 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.g.log 2024-04-21T08:55:12.948 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.log: /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.j.log 2024-04-21T08:55:12.948 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.3.log 2024-04-21T08:55:12.948 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.0.log 2024-04-21T08:55:12.948 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mgr.x.log 2024-04-21T08:55:12.948 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mon.a.log 2024-04-21T08:55:12.948 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.6.log 2024-04-21T08:55:12.954 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mon.a.log: 87.1% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.log.gz 2024-04-21T08:55:12.954 INFO:teuthology.orchestra.run.smithi043.stderr:gzip 90.2% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.audit.log.gz 2024-04-21T08:55:12.954 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.cephadm.log 2024-04-21T08:55:12.954 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.a.log 2024-04-21T08:55:12.954 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.cephadm.log: 84.1% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-21T08:55:12.954 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-volume.log 2024-04-21T08:55:12.954 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.a.log: 94.6% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-21T08:55:12.963 INFO:teuthology.orchestra.run.smithi112.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.log 2024-04-21T08:55:12.963 INFO:teuthology.orchestra.run.smithi112.stderr:: No such file or directory 2024-04-21T08:55:12.963 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.audit.log 2024-04-21T08:55:12.963 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.7.log 2024-04-21T08:55:12.963 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.log: /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.b.log 2024-04-21T08:55:12.963 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.4.log 2024-04-21T08:55:12.963 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.1.log 2024-04-21T08:55:12.963 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mgr.y.log 2024-04-21T08:55:12.963 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.h.log 2024-04-21T08:55:12.963 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.10.log 2024-04-21T08:55:12.963 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.h.log: 90.4% 87.1% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.log.gz 2024-04-21T08:55:12.964 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.audit.log.gz 2024-04-21T08:55:12.964 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.cephadm.log 2024-04-21T08:55:12.964 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.e.log 2024-04-21T08:55:12.964 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-21T08:55:12.964 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-volume.log 2024-04-21T08:55:12.964 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.e.log: 92.1% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-21T08:55:12.964 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mon.b.log 2024-04-21T08:55:12.964 INFO:teuthology.orchestra.run.smithi175.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory 2024-04-21T08:55:12.964 INFO:teuthology.orchestra.run.smithi175.stderr: /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.log 2024-04-21T08:55:12.965 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.2.log 2024-04-21T08:55:12.965 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.f.log 2024-04-21T08:55:12.965 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.log: /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.audit.log 2024-04-21T08:55:12.965 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.5.log 2024-04-21T08:55:12.965 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.c.log 2024-04-21T08:55:12.965 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mon.c.log 2024-04-21T08:55:12.965 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mgr.z.log 2024-04-21T08:55:12.965 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.i.log 2024-04-21T08:55:12.965 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mgr.z.log: 87.1% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.log.gz 2024-04-21T08:55:12.965 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.cephadm.log 2024-04-21T08:55:12.965 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.i.log: 90.4% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.audit.log.gz 2024-04-21T08:55:12.965 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.11.log 2024-04-21T08:55:12.965 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.cephadm.log: 83.8% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-21T08:55:12.965 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.8.log 2024-04-21T08:55:12.965 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.11.log: 92.1% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-21T08:55:12.966 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-volume.log 2024-04-21T08:55:12.966 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.d.log 2024-04-21T08:55:12.970 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.l.log 2024-04-21T08:55:12.973 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.k.log 2024-04-21T08:55:12.976 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-volume.log.gz 2024-04-21T08:55:12.976 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.9.log 2024-04-21T08:55:12.978 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.d.log: 91.0% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-volume.log.gz 2024-04-21T08:55:12.979 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-21T08:55:12.980 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-21T08:55:12.982 INFO:teuthology.orchestra.run.smithi043.stderr: 94.6% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-21T08:55:12.982 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-21T08:55:12.995 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-21T08:55:12.996 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-21T08:55:12.997 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-volume.log.gz 2024-04-21T08:55:13.002 INFO:teuthology.orchestra.run.smithi175.stderr: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-21T08:55:13.020 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/cephadm.log: 93.3%/var/log/ceph/ceph-client.admin.log: 92.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-21T08:55:13.022 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-21T08:55:13.023 INFO:teuthology.orchestra.run.smithi043.stderr: 94.5% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-21T08:55:13.032 INFO:teuthology.orchestra.run.smithi043.stderr: 94.5% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-21T08:55:13.034 INFO:teuthology.orchestra.run.smithi112.stderr: 94.6% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-21T08:55:13.036 INFO:teuthology.orchestra.run.smithi175.stderr: 94.5% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-21T08:55:13.060 INFO:teuthology.orchestra.run.smithi112.stderr: 92.5% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-21T08:55:13.089 INFO:teuthology.orchestra.run.smithi175.stderr: 92.6% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-21T08:55:13.109 INFO:teuthology.orchestra.run.smithi175.stderr: 92.6% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-21T08:55:13.113 INFO:teuthology.orchestra.run.smithi112.stderr: 92.5% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-21T08:55:13.280 INFO:teuthology.orchestra.run.smithi175.stderr: 93.7% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-21T08:55:13.330 INFO:teuthology.orchestra.run.smithi112.stderr: 92.0% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-21T08:55:13.371 INFO:teuthology.orchestra.run.smithi043.stderr: 90.3% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-21T08:55:13.416 INFO:teuthology.orchestra.run.smithi175.stderr: 92.1% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-21T08:55:13.972 INFO:teuthology.orchestra.run.smithi175.stderr: 92.0% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-21T08:55:14.022 INFO:teuthology.orchestra.run.smithi112.stderr: 92.4% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-21T08:55:14.124 INFO:teuthology.orchestra.run.smithi043.stderr: 92.4% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-21T08:55:14.295 INFO:teuthology.orchestra.run.smithi175.stderr: 92.0% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-21T08:55:14.362 INFO:teuthology.orchestra.run.smithi043.stderr: 93.3% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-21T08:55:14.397 INFO:teuthology.orchestra.run.smithi043.stderr: 90.7% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-21T08:55:14.646 INFO:teuthology.orchestra.run.smithi112.stderr: 91.6% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-21T08:55:14.873 INFO:teuthology.orchestra.run.smithi112.stderr: 93.0% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-21T08:55:14.899 INFO:teuthology.orchestra.run.smithi043.stderr: 93.0% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-21T08:55:15.023 INFO:teuthology.orchestra.run.smithi112.stderr: 92.9% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-21T08:55:15.060 INFO:teuthology.orchestra.run.smithi043.stderr: 92.7% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-21T08:55:15.062 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-04-21T08:55:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:real 0m2.128s 2024-04-21T08:55:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:user 0m8.427s 2024-04-21T08:55:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:sys 0m0.268s 2024-04-21T08:55:15.115 INFO:teuthology.orchestra.run.smithi175.stderr: 91.5% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-21T08:55:15.365 INFO:teuthology.orchestra.run.smithi175.stderr: 92.4% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-21T08:55:15.366 INFO:teuthology.orchestra.run.smithi175.stderr: 2024-04-21T08:55:15.366 INFO:teuthology.orchestra.run.smithi175.stderr:real 0m2.429s 2024-04-21T08:55:15.367 INFO:teuthology.orchestra.run.smithi175.stderr:user 0m7.775s 2024-04-21T08:55:15.367 INFO:teuthology.orchestra.run.smithi175.stderr:sys 0m0.293s 2024-04-21T08:55:15.522 INFO:teuthology.orchestra.run.smithi112.stderr: 92.4% -- replaced with /var/log/ceph/d57a7d98-ffb8-11ee-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-21T08:55:15.524 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-21T08:55:15.524 INFO:teuthology.orchestra.run.smithi112.stderr:real 0m2.587s 2024-04-21T08:55:15.524 INFO:teuthology.orchestra.run.smithi112.stderr:user 0m9.665s 2024-04-21T08:55:15.524 INFO:teuthology.orchestra.run.smithi112.stderr:sys 0m0.339s 2024-04-21T08:55:15.524 INFO:tasks.cephadm:Archiving logs... 2024-04-21T08:55:15.524 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666510/remote/smithi043/log 2024-04-21T08:55:15.525 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-21T08:55:16.703 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666510/remote/smithi112/log 2024-04-21T08:55:16.703 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-21T08:55:18.441 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666510/remote/smithi175/log 2024-04-21T08:55:18.442 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-21T08:55:19.674 INFO:tasks.cephadm:Removing cluster... 2024-04-21T08:55:19.675 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 --force 2024-04-21T08:55:19.875 INFO:teuthology.orchestra.run.smithi043.stdout:Deleting cluster with fsid: d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:55:20.154 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 --force 2024-04-21T08:55:20.340 INFO:teuthology.orchestra.run.smithi112.stdout:Deleting cluster with fsid: d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:55:20.687 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d57a7d98-ffb8-11ee-bc93-c7b262605968 --force 2024-04-21T08:55:20.869 INFO:teuthology.orchestra.run.smithi175.stdout:Deleting cluster with fsid: d57a7d98-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:55:21.125 INFO:tasks.cephadm:Removing cephadm ... 2024-04-21T08:55:21.125 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-21T08:55:21.143 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-21T08:55:21.159 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-21T08:55:21.174 INFO:tasks.cephadm:Teardown complete 2024-04-21T08:55:21.175 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-21T08:55:21.189 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-21T08:55:21.189 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-21T08:55:21.191 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-21T08:55:21.202 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-21T08:55:21.309 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-21T08:55:21.321 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-21T08:55:21.321 DEBUG:teuthology.orchestra.run.smithi043:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T08:55:21.324 DEBUG:teuthology.orchestra.run.smithi112:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T08:55:21.326 DEBUG:teuthology.orchestra.run.smithi175:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T08:55:21.338 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-04-21T08:55:21.338 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-04-21T08:55:21.340 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-04-21T08:55:21.342 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T08:55:21.342 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-04-21T08:55:21.343 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 28 -25us[ -30us] +/- 77ms 2024-04-21T08:55:21.343 INFO:teuthology.orchestra.run.smithi112.stdout:^* hv02.front.sepia.ceph.com 3 6 377 26 -403us[ -408us] +/- 62ms 2024-04-21T08:55:21.343 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 +561us[ +556us] +/- 83ms 2024-04-21T08:55:21.343 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:55:21.344 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T08:55:21.344 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-04-21T08:55:21.344 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 26 -55us[ -55us] +/- 77ms 2024-04-21T08:55:21.345 INFO:teuthology.orchestra.run.smithi043.stdout:^* hv02.front.sepia.ceph.com 3 6 377 30 -398us[ -393us] +/- 62ms 2024-04-21T08:55:21.345 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 92 +568us[ +577us] +/- 82ms 2024-04-21T08:55:21.345 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:55:21.346 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T08:55:21.346 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-04-21T08:55:21.346 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 27 +32us[ +27us] +/- 78ms 2024-04-21T08:55:21.346 INFO:teuthology.orchestra.run.smithi175.stdout:^* hv02.front.sepia.ceph.com 3 6 377 27 -393us[ -398us] +/- 62ms 2024-04-21T08:55:21.346 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 +560us[ +560us] +/- 83ms 2024-04-21T08:55:21.346 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:55:21.346 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-21T08:55:21.358 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-21T08:55:21.358 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-21T08:55:21.370 DEBUG:teuthology.orchestra.run.smithi043:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-21T08:55:21.472 DEBUG:teuthology.orchestra.run.smithi112:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-21T08:55:21.537 DEBUG:teuthology.orchestra.run.smithi175:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-21T08:55:21.601 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T08:55:21.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:55:21.632 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T08:55:21.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:55:21.659 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T08:55:21.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:55:21.685 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-21T08:55:21.697 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-21T08:55:21.708 INFO:teuthology.task.internal:Duration was 2090.850343 seconds 2024-04-21T08:55:21.708 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-21T08:55:21.719 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-21T08:55:21.719 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-21T08:55:21.722 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-21T08:55:21.724 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-21T08:55:21.758 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T08:55:21.758 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T08:55:21.762 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T08:55:22.101 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-21T08:55:22.101 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi043.front.sepia.ceph.com 2024-04-21T08:55:22.101 DEBUG:teuthology.orchestra.run.smithi043:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-21T08:55:22.161 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi112.front.sepia.ceph.com 2024-04-21T08:55:22.162 DEBUG:teuthology.orchestra.run.smithi112:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-21T08:55:22.184 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi175.front.sepia.ceph.com 2024-04-21T08:55:22.185 DEBUG:teuthology.orchestra.run.smithi175:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-21T08:55:22.214 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-21T08:55:22.214 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-21T08:55:22.216 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-21T08:55:22.227 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-21T08:55:22.279 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-21T08:55:22.280 DEBUG:teuthology.orchestra.run.smithi043:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-21T08:55:22.284 DEBUG:teuthology.orchestra.run.smithi112:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-21T08:55:22.292 DEBUG:teuthology.orchestra.run.smithi175:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-21T08:55:22.696 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-21T08:55:22.762 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-21T08:55:22.762 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-21T08:55:22.787 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-21T08:55:22.811 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-21T08:55:22.835 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-21T08:55:22.849 DEBUG:teuthology.orchestra.run.smithi043:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-21T08:55:22.852 DEBUG:teuthology.orchestra.run.smithi112:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-21T08:55:22.854 DEBUG:teuthology.orchestra.run.smithi175:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-21T08:55:22.872 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = core 2024-04-21T08:55:22.873 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = core 2024-04-21T08:55:22.897 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = core 2024-04-21T08:55:22.922 DEBUG:teuthology.orchestra.run.smithi043:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-21T08:55:22.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:55:22.939 DEBUG:teuthology.orchestra.run.smithi112:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-21T08:55:22.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:55:22.953 DEBUG:teuthology.orchestra.run.smithi175:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-21T08:55:22.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:55:22.976 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-21T08:55:22.989 INFO:teuthology.task.internal:Transferring archived files... 2024-04-21T08:55:22.990 DEBUG:teuthology.misc:Transferring archived files from smithi043:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666510/remote/smithi043 2024-04-21T08:55:22.990 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-21T08:55:23.035 DEBUG:teuthology.misc:Transferring archived files from smithi112:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666510/remote/smithi112 2024-04-21T08:55:23.035 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-21T08:55:23.072 DEBUG:teuthology.misc:Transferring archived files from smithi175:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666510/remote/smithi175 2024-04-21T08:55:23.073 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-21T08:55:23.107 INFO:teuthology.task.internal:Removing archive directory... 2024-04-21T08:55:23.107 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-21T08:55:23.110 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-21T08:55:23.112 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-21T08:55:23.156 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-21T08:55:23.170 INFO:teuthology.task.internal:Not uploading archives. 2024-04-21T08:55:23.170 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-21T08:55:23.182 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-21T08:55:23.183 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-21T08:55:23.185 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-21T08:55:23.188 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-21T08:55:23.198 INFO:teuthology.orchestra.run.smithi043.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 21 08:55 /home/ubuntu/cephtest 2024-04-21T08:55:23.200 INFO:teuthology.orchestra.run.smithi112.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 21 08:55 /home/ubuntu/cephtest 2024-04-21T08:55:23.211 INFO:teuthology.orchestra.run.smithi175.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 21 08:55 /home/ubuntu/cephtest 2024-04-21T08:55:23.212 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-21T08:55:23.224 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-21T08:55:23.251 INFO:teuthology.nuke:Checking targets against current locks 2024-04-21T08:55:23.275 DEBUG:teuthology.nuke:shortname: smithi043 2024-04-21T08:55:23.276 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-21T08:55:23.295 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666510', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 08:15:11.958922', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHvyUXKR7gN3zHbaPnlKREYK3PV15Bktz8Y9kw+xlm9fM2/qeSOAhC4Pz/GpLf9q25pOs4Nvl2cUsxn154piXtQ='} 2024-04-21T08:55:23.299 DEBUG:teuthology.nuke:shortname: smithi112 2024-04-21T08:55:23.299 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-21T08:55:23.321 DEBUG:teuthology.nuke:shortname: smithi175 2024-04-21T08:55:23.321 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-21T08:55:23.324 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666510', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 08:15:11.960764', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFNvgHzHjc0QFXV1lqtW7olbE2CT+rABIqw7RYiHnBdu+PNNmnXviUOqVef8EeyRpn36i/TS7Pa1OrnYAXRLopA='} 2024-04-21T08:55:23.338 INFO:teuthology.orchestra.console.smithi043:Power off 2024-04-21T08:55:23.338 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-21T08:55:23.353 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666510', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 08:15:11.962373', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO/mG4hwt9g8AfooYwlYtJtUA1tsPvLLoRi+wriTHjKdOnztrgK6ljh7zcX82n49LeVgtRnAMHe3fwGZLsOIOIg='} 2024-04-21T08:55:23.363 DEBUG:teuthology.orchestra.console.smithi043:power off output: Chassis Power Control: Down/Off 2024-04-21T08:55:23.363 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T08:55:23.383 INFO:teuthology.orchestra.console.smithi112:Power off 2024-04-21T08:55:23.384 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-21T08:55:23.396 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is on 2024-04-21T08:55:23.406 DEBUG:teuthology.orchestra.console.smithi112:power off output: Chassis Power Control: Down/Off 2024-04-21T08:55:23.406 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T08:55:23.425 INFO:teuthology.orchestra.console.smithi175:Power off 2024-04-21T08:55:23.425 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-21T08:55:23.433 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-04-21T08:55:23.447 DEBUG:teuthology.orchestra.console.smithi175:power off output: Chassis Power Control: Down/Off 2024-04-21T08:55:23.447 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T08:55:23.469 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is on 2024-04-21T08:55:27.398 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T08:55:27.435 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T08:55:27.470 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T08:55:27.508 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is on 2024-04-21T08:55:27.544 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-04-21T08:55:27.579 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is on 2024-04-21T08:55:31.510 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T08:55:31.545 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T08:55:31.580 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T08:55:31.620 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is off 2024-04-21T08:55:31.655 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2024-04-21T08:55:31.690 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is off 2024-04-21T08:55:31.721 INFO:teuthology.orchestra.console.smithi043:Power off completed 2024-04-21T08:55:31.755 INFO:teuthology.orchestra.console.smithi112:Power off completed 2024-04-21T08:55:31.791 INFO:teuthology.orchestra.console.smithi175:Power off completed 2024-04-21T08:55:31.896 INFO:teuthology.run:Summary data: description: fs/workload/{0-ubuntu_22.04 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} duration: 2090.8503425121307 failure_reason: 'error during scrub thrashing: ' flavor: default owner: scheduled_leonidus@teuthology status: fail success: false 2024-04-21T08:55:31.896 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-21T08:55:31.990 INFO:teuthology.run:FAIL